2024-02-23T00:10:47.567 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-23T00:10:47.567 DEBUG:teuthology.run:Teuthology command: teuthology --description rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} --archive /home/teuthworker/archive/yuriw-2024-02-22_21:39:39-rados-pacific-release-distro-default-smithi/7571410 --verbose --owner scheduled_yuriw@teuthology --name yuriw-2024-02-22_21:39:39-rados-pacific-release-distro-default-smithi -- /home/teuthworker/archive/yuriw-2024-02-22_21:39:39-rados-pacific-release-distro-default-smithi/7571410/orig.config.yaml 2024-02-23T00:10:47.600 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-23T00:10:47.730 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-02-22_21:39:39-rados-pacific-release-distro-default-smithi/7571410 branch: pacific-release description: rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} email: null first_in_suite: false job_id: '7571410' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2024-02-22_21:39:39-rados-pacific-release-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 8.stream overrides: admin_socket: branch: pacific-release ceph: conf: client: debug ms: 1 debug objecter: 20 debug rados: 20 global: ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mgrc: 20 debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon warn on pool no app: false paxos service trim min: 10 osd: debug ms: 1 debug osd: 20 osd debug reject backfill probability: 0.3 osd delete sleep: 1 osd max backfills: 3 osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 2 crush_tunables: jewel flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - but it is still running - objects unfound and apparently lost - \(MON_DOWN\) - \(OSDMAP_FLAGS\) - flag\(s\) set - \(CACHE_POOL_NO_HIT_SET\) - \(PG_ - \(OSD_ - \(POOL_ - \(CEPHADM_STRAY_DAEMON\) - PG_ - degraded - backfill - mons down - OSD_ - is down - acting - reached quota - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) sha1: eb66ed921e744301e6863be6353618d63967ea59 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: root install: ceph: flavor: default sha1: eb66ed921e744301e6863be6353618d63967ea59 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-ksirivad-pacific-release-whitelist sha1: 4f758cd8e8854d1f126da86e254076852bf059a9 owner: scheduled_yuriw@teuthology priority: 61 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 264 sha1: eb66ed921e744301e6863be6353618d63967ea59 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-ksirivad-pacific-release-whitelist suite_path: /home/teuthworker/src/github.com_kamoltat_ceph_4f758cd8e8854d1f126da86e254076852bf059a9/qa suite_relpath: qa suite_repo: https://github.com/kamoltat/ceph.git suite_sha1: 4f758cd8e8854d1f126da86e254076852bf059a9 targets: smithi047.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLja5BdvI+gD/T43f1uKqib+w89XuUXApahtXx6mCwgoCRsQhYYDNbC3SDQBa7ys1ksjaC4kryDPsWTIpx5hQRQ= smithi196.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL+ORhKeGKcECKIxv1tEiqI+7yxlQrpkHN2W5F/HdE9Olj7ZS9IkJm7qM7ObGbr9Z/sC15f3TGLKL76uw+nnO3s= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - thrashosds: chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 timeout: 1200 - workunit: clients: client.0: - rados/test.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-02-22_21:39:39 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-23T00:10:47.731 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_kamoltat_ceph_4f758cd8e8854d1f126da86e254076852bf059a9/qa; will attempt to use it 2024-02-23T00:10:47.732 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_kamoltat_ceph_4f758cd8e8854d1f126da86e254076852bf059a9/qa/tasks 2024-02-23T00:10:47.732 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-23T00:10:47.733 INFO:teuthology.task.internal:Checking packages... 2024-02-23T00:10:47.758 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'eb66ed921e744301e6863be6353618d63967ea59' 2024-02-23T00:10:47.758 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-23T00:10:47.758 INFO:teuthology.packaging:ref: None 2024-02-23T00:10:47.758 INFO:teuthology.packaging:tag: None 2024-02-23T00:10:47.758 INFO:teuthology.packaging:branch: pacific-release 2024-02-23T00:10:47.758 INFO:teuthology.packaging:sha1: eb66ed921e744301e6863be6353618d63967ea59 2024-02-23T00:10:47.759 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=pacific-release 2024-02-23T00:10:47.920 INFO:teuthology.task.internal:Found packages for ceph version 16.2.14-550.geb66ed92 2024-02-23T00:10:47.922 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-23T00:10:47.928 INFO:teuthology.task.internal:no buildpackages task found 2024-02-23T00:10:47.928 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-23T00:10:47.953 INFO:teuthology.task.internal:Saving configuration 2024-02-23T00:10:47.966 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-23T00:10:48.004 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-23T00:10:48.034 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi047.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-22_21:39:39-rados-pacific-release-distro-default-smithi/7571410', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-23 00:05:39.230323', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLja5BdvI+gD/T43f1uKqib+w89XuUXApahtXx6mCwgoCRsQhYYDNbC3SDQBa7ys1ksjaC4kryDPsWTIpx5hQRQ='} 2024-02-23T00:10:48.073 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-22_21:39:39-rados-pacific-release-distro-default-smithi/7571410', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-23 00:05:39.227856', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL+ORhKeGKcECKIxv1tEiqI+7yxlQrpkHN2W5F/HdE9Olj7ZS9IkJm7qM7ObGbr9Z/sC15f3TGLKL76uw+nnO3s='} 2024-02-23T00:10:48.073 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-23T00:10:48.103 INFO:teuthology.task.internal:roles: ubuntu@smithi047.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-02-23T00:10:48.103 INFO:teuthology.task.internal:roles: ubuntu@smithi196.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-02-23T00:10:48.103 INFO:teuthology.run_tasks:Running task console_log... 2024-02-23T00:10:48.232 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f67eaab30d0>, signals=[15]) 2024-02-23T00:10:48.232 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-23T00:10:48.240 INFO:teuthology.task.internal:Opening connections... 2024-02-23T00:10:48.240 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi047.front.sepia.ceph.com 2024-02-23T00:10:48.243 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-23T00:10:48.330 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi196.front.sepia.ceph.com 2024-02-23T00:10:48.331 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-23T00:10:48.413 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-23T00:10:48.420 DEBUG:teuthology.orchestra.run.smithi047:> uname -m 2024-02-23T00:10:48.443 INFO:teuthology.orchestra.run.smithi047.stdout:x86_64 2024-02-23T00:10:48.443 DEBUG:teuthology.orchestra.run.smithi047:> cat /etc/os-release 2024-02-23T00:10:48.506 INFO:teuthology.orchestra.run.smithi047.stdout:NAME="CentOS Stream" 2024-02-23T00:10:48.506 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION="8" 2024-02-23T00:10:48.506 INFO:teuthology.orchestra.run.smithi047.stdout:ID="centos" 2024-02-23T00:10:48.506 INFO:teuthology.orchestra.run.smithi047.stdout:ID_LIKE="rhel fedora" 2024-02-23T00:10:48.506 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION_ID="8" 2024-02-23T00:10:48.506 INFO:teuthology.orchestra.run.smithi047.stdout:PLATFORM_ID="platform:el8" 2024-02-23T00:10:48.506 INFO:teuthology.orchestra.run.smithi047.stdout:PRETTY_NAME="CentOS Stream 8" 2024-02-23T00:10:48.506 INFO:teuthology.orchestra.run.smithi047.stdout:ANSI_COLOR="0;31" 2024-02-23T00:10:48.506 INFO:teuthology.orchestra.run.smithi047.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-02-23T00:10:48.506 INFO:teuthology.orchestra.run.smithi047.stdout:HOME_URL="https://centos.org/" 2024-02-23T00:10:48.507 INFO:teuthology.orchestra.run.smithi047.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-23T00:10:48.507 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-02-23T00:10:48.507 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-23T00:10:48.507 INFO:teuthology.lock.ops:Updating smithi047.front.sepia.ceph.com on lock server 2024-02-23T00:10:48.532 DEBUG:teuthology.orchestra.run.smithi196:> uname -m 2024-02-23T00:10:48.554 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2024-02-23T00:10:48.554 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-02-23T00:10:48.614 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="CentOS Stream" 2024-02-23T00:10:48.614 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="8" 2024-02-23T00:10:48.614 INFO:teuthology.orchestra.run.smithi196.stdout:ID="centos" 2024-02-23T00:10:48.614 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="rhel fedora" 2024-02-23T00:10:48.614 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="8" 2024-02-23T00:10:48.615 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el8" 2024-02-23T00:10:48.615 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="CentOS Stream 8" 2024-02-23T00:10:48.615 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31" 2024-02-23T00:10:48.615 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-02-23T00:10:48.615 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://centos.org/" 2024-02-23T00:10:48.615 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-23T00:10:48.615 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-02-23T00:10:48.615 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-23T00:10:48.615 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-02-23T00:10:48.656 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-23T00:10:48.667 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-23T00:10:48.789 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-23T00:10:48.789 DEBUG:teuthology.orchestra.run.smithi047:> test '!' -e /home/ubuntu/cephtest 2024-02-23T00:10:48.792 DEBUG:teuthology.orchestra.run.smithi196:> test '!' -e /home/ubuntu/cephtest 2024-02-23T00:10:48.813 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-23T00:10:48.836 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-23T00:10:48.837 DEBUG:teuthology.orchestra.run.smithi047:> test -z $(ls -A /var/lib/ceph) 2024-02-23T00:10:48.854 DEBUG:teuthology.orchestra.run.smithi196:> test -z $(ls -A /var/lib/ceph) 2024-02-23T00:10:48.886 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-23T00:10:48.961 INFO:teuthology.run_tasks:Running task kernel... 2024-02-23T00:10:48.977 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-02-23T00:10:48.977 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-02-23T00:10:48.977 DEBUG:teuthology.orchestra.run.smithi047:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-23T00:10:48.978 DEBUG:teuthology.orchestra.run.smithi196:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-23T00:10:48.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-23T00:10:48.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-23T00:10:48.999 DEBUG:teuthology.orchestra.run.smithi196:> uname -r 2024-02-23T00:10:49.000 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2024-02-23T00:10:49.061 INFO:teuthology.orchestra.run.smithi196.stdout:4.18.0-489.el8.x86_64 2024-02-23T00:10:49.061 INFO:teuthology.task.kernel:Running kernel on smithi196: 4.18.0-489.el8.x86_64 2024-02-23T00:10:49.061 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum install -y kernel 2024-02-23T00:10:49.062 INFO:teuthology.orchestra.run.smithi047.stdout:4.18.0-489.el8.x86_64 2024-02-23T00:10:49.063 INFO:teuthology.task.kernel:Running kernel on smithi047: 4.18.0-489.el8.x86_64 2024-02-23T00:10:49.063 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum install -y kernel 2024-02-23T00:10:54.599 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 8 - AppStream 9.1 MB/s | 27 MB 00:03 2024-02-23T00:10:54.648 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 8 - AppStream 9.0 MB/s | 27 MB 00:03 2024-02-23T00:11:00.398 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 8 - BaseOS 13 MB/s | 10 MB 00:00 2024-02-23T00:11:00.791 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 8 - BaseOS 10 MB/s | 10 MB 00:01 2024-02-23T00:11:02.223 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 8 - Extras 83 kB/s | 18 kB 00:00 2024-02-23T00:11:02.580 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 8 - Extras common packages 28 kB/s | 6.9 kB 00:00 2024-02-23T00:11:02.851 INFO:teuthology.orchestra.run.smithi196.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2024-02-23T00:11:02.883 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 8 - Extras 47 kB/s | 18 kB 00:00 2024-02-23T00:11:03.350 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 8 - Extras common packages 20 kB/s | 6.9 kB 00:00 2024-02-23T00:11:03.594 INFO:teuthology.orchestra.run.smithi047.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2024-02-23T00:11:04.444 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 21 MB/s | 16 MB 00:00 2024-02-23T00:11:07.259 INFO:teuthology.orchestra.run.smithi196.stdout:Extra Packages for Enterprise Linux 3.7 MB/s | 16 MB 00:04 2024-02-23T00:11:08.452 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 437 kB/s | 24 kB 00:00 2024-02-23T00:11:11.059 INFO:teuthology.orchestra.run.smithi196.stdout:lab-extras 396 kB/s | 24 kB 00:00 2024-02-23T00:11:11.649 INFO:teuthology.orchestra.run.smithi047.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2024-02-23T00:11:11.649 INFO:teuthology.orchestra.run.smithi047.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2024-02-23T00:11:11.766 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-02-23T00:11:11.767 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:11:11.767 INFO:teuthology.orchestra.run.smithi047.stdout: Package Architecture Version Repository Size 2024-02-23T00:11:11.767 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:11:11.767 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-02-23T00:11:11.768 INFO:teuthology.orchestra.run.smithi047.stdout: kernel x86_64 4.18.0-540.el8 baseos 10 M 2024-02-23T00:11:11.768 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-02-23T00:11:11.768 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-core x86_64 4.18.0-540.el8 baseos 44 M 2024-02-23T00:11:11.768 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-modules x86_64 4.18.0-540.el8 baseos 36 M 2024-02-23T00:11:11.768 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:11:11.768 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-02-23T00:11:11.768 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:11:11.768 INFO:teuthology.orchestra.run.smithi047.stdout:Install 3 Packages 2024-02-23T00:11:11.768 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:11:11.769 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 90 M 2024-02-23T00:11:11.769 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 97 M 2024-02-23T00:11:11.769 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-02-23T00:11:12.829 INFO:teuthology.orchestra.run.smithi047.stdout:(1/3): kernel-4.18.0-540.el8.x86_64.rpm 12 MB/s | 10 MB 00:00 2024-02-23T00:11:13.829 INFO:teuthology.orchestra.run.smithi047.stdout:(2/3): kernel-modules-4.18.0-540.el8.x86_64.rpm 19 MB/s | 36 MB 00:01 2024-02-23T00:11:14.014 INFO:teuthology.orchestra.run.smithi196.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2024-02-23T00:11:14.014 INFO:teuthology.orchestra.run.smithi196.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2024-02-23T00:11:14.147 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-02-23T00:11:14.148 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:11:14.148 INFO:teuthology.orchestra.run.smithi196.stdout: Package Architecture Version Repository Size 2024-02-23T00:11:14.148 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:11:14.148 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-02-23T00:11:14.148 INFO:teuthology.orchestra.run.smithi196.stdout: kernel x86_64 4.18.0-540.el8 baseos 10 M 2024-02-23T00:11:14.148 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-02-23T00:11:14.148 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-core x86_64 4.18.0-540.el8 baseos 44 M 2024-02-23T00:11:14.148 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules x86_64 4.18.0-540.el8 baseos 36 M 2024-02-23T00:11:14.148 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:11:14.148 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-02-23T00:11:14.148 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:11:14.148 INFO:teuthology.orchestra.run.smithi196.stdout:Install 3 Packages 2024-02-23T00:11:14.149 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:11:14.149 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 90 M 2024-02-23T00:11:14.149 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 97 M 2024-02-23T00:11:14.149 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-02-23T00:11:14.230 INFO:teuthology.orchestra.run.smithi047.stdout:(3/3): kernel-core-4.18.0-540.el8.x86_64.rpm 19 MB/s | 44 MB 00:02 2024-02-23T00:11:14.230 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:11:14.230 INFO:teuthology.orchestra.run.smithi047.stdout:Total 37 MB/s | 90 MB 00:02 2024-02-23T00:11:14.946 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-02-23T00:11:15.027 INFO:teuthology.orchestra.run.smithi196.stdout:(1/3): kernel-4.18.0-540.el8.x86_64.rpm 12 MB/s | 10 MB 00:00 2024-02-23T00:11:15.408 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-02-23T00:11:15.409 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-02-23T00:11:15.677 INFO:teuthology.orchestra.run.smithi196.stdout:(2/3): kernel-core-4.18.0-540.el8.x86_64.rpm 29 MB/s | 44 MB 00:01 2024-02-23T00:11:16.194 INFO:teuthology.orchestra.run.smithi196.stdout:(3/3): kernel-modules-4.18.0-540.el8.x86_64.rpm 18 MB/s | 36 MB 00:02 2024-02-23T00:11:16.194 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:11:16.194 INFO:teuthology.orchestra.run.smithi196.stdout:Total 44 MB/s | 90 MB 00:02 2024-02-23T00:11:16.704 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-02-23T00:11:16.722 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-02-23T00:11:16.875 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-02-23T00:11:17.339 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-02-23T00:11:17.340 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-02-23T00:11:18.579 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-02-23T00:11:18.603 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-02-23T00:11:19.681 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-02-23T00:11:20.226 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : kernel-core-4.18.0-540.el8.x86_64 1/3 2024-02-23T00:11:21.393 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-02-23T00:11:21.881 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-core-4.18.0-540.el8.x86_64 1/3 2024-02-23T00:11:24.027 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-core-4.18.0-540.el8.x86_64 1/3 2024-02-23T00:11:24.417 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : kernel-modules-4.18.0-540.el8.x86_64 2/3 2024-02-23T00:11:25.451 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-core-4.18.0-540.el8.x86_64 1/3 2024-02-23T00:11:25.855 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-modules-4.18.0-540.el8.x86_64 2/3 2024-02-23T00:11:28.097 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-modules-4.18.0-540.el8.x86_64 2/3 2024-02-23T00:11:28.396 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : kernel-4.18.0-540.el8.x86_64 3/3 2024-02-23T00:11:29.282 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-modules-4.18.0-540.el8.x86_64 2/3 2024-02-23T00:11:29.545 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-4.18.0-540.el8.x86_64 3/3 2024-02-23T00:11:51.466 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-core-4.18.0-540.el8.x86_64 3/3 2024-02-23T00:11:51.489 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-modules-4.18.0-540.el8.x86_64 3/3 2024-02-23T00:11:51.860 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-4.18.0-540.el8.x86_64 3/3 2024-02-23T00:11:51.860 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-4.18.0-540.el8.x86_64 1/3 2024-02-23T00:11:51.860 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-core-4.18.0-540.el8.x86_64 2/3 2024-02-23T00:11:51.945 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-core-4.18.0-540.el8.x86_64 3/3 2024-02-23T00:11:51.968 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-modules-4.18.0-540.el8.x86_64 3/3 2024-02-23T00:11:52.317 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-modules-4.18.0-540.el8.x86_64 3/3 2024-02-23T00:11:52.317 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:11:52.317 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-02-23T00:11:52.317 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-4.18.0-540.el8.x86_64 kernel-core-4.18.0-540.el8.x86_64 2024-02-23T00:11:52.317 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules-4.18.0-540.el8.x86_64 2024-02-23T00:11:52.317 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:11:52.317 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-02-23T00:11:52.323 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-4.18.0-540.el8.x86_64 3/3 2024-02-23T00:11:52.323 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : kernel-4.18.0-540.el8.x86_64 1/3 2024-02-23T00:11:52.323 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : kernel-core-4.18.0-540.el8.x86_64 2/3 2024-02-23T00:11:52.447 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q kernel | sort -rV | head -n 1 2024-02-23T00:11:52.708 INFO:teuthology.orchestra.run.smithi196.stdout:kernel-4.18.0-540.el8.x86_64 2024-02-23T00:11:52.708 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-540.el8.x86_64 2024-02-23T00:11:52.708 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-540.el8.x86_64 2024-02-23T00:11:52.708 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-23T00:11:52.708 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-23T00:11:52.709 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-02-23T00:11:52.709 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi196.front.sepia.ceph.com, path=None, version=distro) 2024-02-23T00:11:52.709 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q kernel | sort -rV | head -n 1 2024-02-23T00:11:52.784 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : kernel-modules-4.18.0-540.el8.x86_64 3/3 2024-02-23T00:11:52.785 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:11:52.785 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-02-23T00:11:52.785 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-4.18.0-540.el8.x86_64 kernel-core-4.18.0-540.el8.x86_64 2024-02-23T00:11:52.785 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-modules-4.18.0-540.el8.x86_64 2024-02-23T00:11:52.785 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:11:52.785 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-02-23T00:11:52.920 INFO:teuthology.orchestra.run.smithi196.stdout:kernel-4.18.0-540.el8.x86_64 2024-02-23T00:11:52.921 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-540.el8.x86_64 2024-02-23T00:11:52.921 DEBUG:teuthology.orchestra.run.smithi196:> sudo rpm -qi grub2-tools 2024-02-23T00:11:52.936 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q kernel | sort -rV | head -n 1 2024-02-23T00:11:52.955 INFO:teuthology.orchestra.run.smithi196.stdout:Name : grub2-tools 2024-02-23T00:11:52.955 INFO:teuthology.orchestra.run.smithi196.stdout:Epoch : 1 2024-02-23T00:11:52.955 INFO:teuthology.orchestra.run.smithi196.stdout:Version : 2.02 2024-02-23T00:11:52.955 INFO:teuthology.orchestra.run.smithi196.stdout:Release : 129.el8 2024-02-23T00:11:52.955 INFO:teuthology.orchestra.run.smithi196.stdout:Architecture: x86_64 2024-02-23T00:11:52.955 INFO:teuthology.orchestra.run.smithi196.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-02-23T00:11:52.955 INFO:teuthology.orchestra.run.smithi196.stdout:Group : System Environment/Base 2024-02-23T00:11:52.955 INFO:teuthology.orchestra.run.smithi196.stdout:Size : 9265586 2024-02-23T00:11:52.955 INFO:teuthology.orchestra.run.smithi196.stdout:License : GPLv3+ 2024-02-23T00:11:52.955 INFO:teuthology.orchestra.run.smithi196.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-02-23T00:11:52.956 INFO:teuthology.orchestra.run.smithi196.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-02-23T00:11:52.956 INFO:teuthology.orchestra.run.smithi196.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-02-23T00:11:52.956 INFO:teuthology.orchestra.run.smithi196.stdout:Build Host : kbuilder.bsys.centos.org 2024-02-23T00:11:52.956 INFO:teuthology.orchestra.run.smithi196.stdout:Relocations : (not relocatable) 2024-02-23T00:11:52.956 INFO:teuthology.orchestra.run.smithi196.stdout:Packager : CentOS BuildSystem 2024-02-23T00:11:52.956 INFO:teuthology.orchestra.run.smithi196.stdout:Vendor : CentOS 2024-02-23T00:11:52.956 INFO:teuthology.orchestra.run.smithi196.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-23T00:11:52.956 INFO:teuthology.orchestra.run.smithi196.stdout:Summary : Support tools for GRUB. 2024-02-23T00:11:52.956 INFO:teuthology.orchestra.run.smithi196.stdout:Description : 2024-02-23T00:11:52.956 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:11:52.956 INFO:teuthology.orchestra.run.smithi196.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-23T00:11:52.956 INFO:teuthology.orchestra.run.smithi196.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-23T00:11:52.956 INFO:teuthology.orchestra.run.smithi196.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-23T00:11:52.956 INFO:teuthology.orchestra.run.smithi196.stdout:hardware devices. 2024-02-23T00:11:52.957 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:11:52.957 INFO:teuthology.orchestra.run.smithi196.stdout:This subpackage provides tools for support of all platforms. 2024-02-23T00:11:52.957 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-23T00:11:52.957 INFO:teuthology.task.kernel:Updating grub on smithi196 to boot 4.18.0-540.el8.x86_64 2024-02-23T00:11:52.958 DEBUG:teuthology.orchestra.run.smithi196:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-23T00:11:53.184 INFO:teuthology.orchestra.run.smithi047.stdout:kernel-4.18.0-540.el8.x86_64 2024-02-23T00:11:53.184 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-540.el8.x86_64 2024-02-23T00:11:53.185 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-540.el8.x86_64 2024-02-23T00:11:53.185 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-23T00:11:53.185 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-23T00:11:53.185 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-02-23T00:11:53.185 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi047.front.sepia.ceph.com, path=None, version=distro) 2024-02-23T00:11:53.185 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q kernel | sort -rV | head -n 1 2024-02-23T00:11:53.404 INFO:teuthology.orchestra.run.smithi047.stdout:kernel-4.18.0-540.el8.x86_64 2024-02-23T00:11:53.405 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-540.el8.x86_64 2024-02-23T00:11:53.405 DEBUG:teuthology.orchestra.run.smithi047:> sudo rpm -qi grub2-tools 2024-02-23T00:11:53.440 INFO:teuthology.orchestra.run.smithi047.stdout:Name : grub2-tools 2024-02-23T00:11:53.440 INFO:teuthology.orchestra.run.smithi047.stdout:Epoch : 1 2024-02-23T00:11:53.440 INFO:teuthology.orchestra.run.smithi047.stdout:Version : 2.02 2024-02-23T00:11:53.440 INFO:teuthology.orchestra.run.smithi047.stdout:Release : 129.el8 2024-02-23T00:11:53.440 INFO:teuthology.orchestra.run.smithi047.stdout:Architecture: x86_64 2024-02-23T00:11:53.440 INFO:teuthology.orchestra.run.smithi047.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-02-23T00:11:53.440 INFO:teuthology.orchestra.run.smithi047.stdout:Group : System Environment/Base 2024-02-23T00:11:53.440 INFO:teuthology.orchestra.run.smithi047.stdout:Size : 9265586 2024-02-23T00:11:53.440 INFO:teuthology.orchestra.run.smithi047.stdout:License : GPLv3+ 2024-02-23T00:11:53.440 INFO:teuthology.orchestra.run.smithi047.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-02-23T00:11:53.440 INFO:teuthology.orchestra.run.smithi047.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-02-23T00:11:53.441 INFO:teuthology.orchestra.run.smithi047.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-02-23T00:11:53.441 INFO:teuthology.orchestra.run.smithi047.stdout:Build Host : kbuilder.bsys.centos.org 2024-02-23T00:11:53.441 INFO:teuthology.orchestra.run.smithi047.stdout:Relocations : (not relocatable) 2024-02-23T00:11:53.441 INFO:teuthology.orchestra.run.smithi047.stdout:Packager : CentOS BuildSystem 2024-02-23T00:11:53.441 INFO:teuthology.orchestra.run.smithi047.stdout:Vendor : CentOS 2024-02-23T00:11:53.441 INFO:teuthology.orchestra.run.smithi047.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-23T00:11:53.441 INFO:teuthology.orchestra.run.smithi047.stdout:Summary : Support tools for GRUB. 2024-02-23T00:11:53.441 INFO:teuthology.orchestra.run.smithi047.stdout:Description : 2024-02-23T00:11:53.441 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:11:53.442 INFO:teuthology.orchestra.run.smithi047.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-23T00:11:53.442 INFO:teuthology.orchestra.run.smithi047.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-23T00:11:53.442 INFO:teuthology.orchestra.run.smithi047.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-23T00:11:53.442 INFO:teuthology.orchestra.run.smithi047.stdout:hardware devices. 2024-02-23T00:11:53.442 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:11:53.442 INFO:teuthology.orchestra.run.smithi047.stdout:This subpackage provides tools for support of all platforms. 2024-02-23T00:11:53.443 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-23T00:11:53.443 INFO:teuthology.task.kernel:Updating grub on smithi047 to boot 4.18.0-540.el8.x86_64 2024-02-23T00:11:53.443 DEBUG:teuthology.orchestra.run.smithi047:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-23T00:11:54.550 INFO:teuthology.orchestra.run.smithi196.stderr:Generating grub configuration file ... 2024-02-23T00:11:55.092 INFO:teuthology.orchestra.run.smithi047.stderr:Generating grub configuration file ... 2024-02-23T00:11:56.601 INFO:teuthology.orchestra.run.smithi196.stderr:done 2024-02-23T00:11:56.602 DEBUG:teuthology.orchestra.run.smithi196:> mktemp 2024-02-23T00:11:56.653 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.5k7hij8WSu 2024-02-23T00:11:56.653 DEBUG:teuthology.orchestra.run.smithi196:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.5k7hij8WSu 2024-02-23T00:11:56.687 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.5k7hij8WSu 2024-02-23T00:11:56.839 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.5k7hij8WSu is 6KB 2024-02-23T00:11:56.887 DEBUG:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.5k7hij8WSu 2024-02-23T00:11:56.905 DEBUG:teuthology.orchestra.run.smithi196:> sudo /bin/ls /boot/loader/entries || true 2024-02-23T00:11:56.975 INFO:teuthology.orchestra.run.smithi196.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-02-23T00:11:56.975 INFO:teuthology.orchestra.run.smithi196.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-02-23T00:11:56.975 INFO:teuthology.orchestra.run.smithi196.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-02-23T00:11:56.975 INFO:teuthology.orchestra.run.smithi196.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-540.el8.x86_64.conf 2024-02-23T00:11:56.976 DEBUG:teuthology.orchestra.run.smithi196:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-540.el8.x86_64 2024-02-23T00:11:57.134 DEBUG:teuthology.orchestra.run.smithi196:> sudo shutdown -r now 2024-02-23T00:11:57.202 INFO:teuthology.orchestra.run.smithi047.stderr:done 2024-02-23T00:11:57.203 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2024-02-23T00:11:57.234 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.K4ZfQYPVVP 2024-02-23T00:11:57.234 DEBUG:teuthology.orchestra.run.smithi047:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.K4ZfQYPVVP 2024-02-23T00:11:57.307 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.K4ZfQYPVVP 2024-02-23T00:11:57.445 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.K4ZfQYPVVP is 6KB 2024-02-23T00:11:57.494 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.K4ZfQYPVVP 2024-02-23T00:11:57.512 DEBUG:teuthology.orchestra.run.smithi047:> sudo /bin/ls /boot/loader/entries || true 2024-02-23T00:11:57.582 INFO:teuthology.orchestra.run.smithi047.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-02-23T00:11:57.582 INFO:teuthology.orchestra.run.smithi047.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-02-23T00:11:57.582 INFO:teuthology.orchestra.run.smithi047.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-02-23T00:11:57.582 INFO:teuthology.orchestra.run.smithi047.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-540.el8.x86_64.conf 2024-02-23T00:11:57.583 DEBUG:teuthology.orchestra.run.smithi047:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-540.el8.x86_64 2024-02-23T00:11:57.738 DEBUG:teuthology.orchestra.run.smithi047:> sudo shutdown -r now 2024-02-23T00:12:27.160 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-02-23T00:12:27.161 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-02-23T00:12:27.162 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-23T00:12:27.742 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-02-23T00:12:27.742 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-02-23T00:12:27.742 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-23T00:13:27.233 DEBUG:teuthology.orchestra.remote:timed out 2024-02-23T00:13:27.753 DEBUG:teuthology.orchestra.remote:timed out 2024-02-23T00:13:36.242 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-02-23T00:13:36.243 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-23T00:13:36.594 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-02-23T00:13:36.754 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-02-23T00:13:36.755 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-23T00:13:36.944 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-02-23T00:13:36.944 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "4.18.0-540.el8.x86_64"... 2024-02-23T00:13:36.944 DEBUG:teuthology.orchestra.run.smithi196:> uname -r 2024-02-23T00:13:36.961 INFO:teuthology.orchestra.run.smithi196.stdout:4.18.0-540.el8.x86_64 2024-02-23T00:13:36.961 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-540.el8.x86_64 vs 4.18.0-540.el8.x86_64 2024-02-23T00:13:36.961 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-23T00:13:36.961 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-23T00:13:37.177 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-02-23T00:13:37.457 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-02-23T00:13:37.457 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-540.el8.x86_64"... 2024-02-23T00:13:37.457 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2024-02-23T00:13:37.474 INFO:teuthology.orchestra.run.smithi047.stdout:4.18.0-540.el8.x86_64 2024-02-23T00:13:37.474 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-540.el8.x86_64 vs 4.18.0-540.el8.x86_64 2024-02-23T00:13:37.474 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-23T00:13:37.474 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-23T00:13:37.962 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-02-23T00:13:37.962 DEBUG:teuthology.orchestra.run.smithi196:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-23T00:13:38.032 INFO:teuthology.orchestra.run.smithi196.stdout:ttyS1 2024-02-23T00:13:38.049 DEBUG:teuthology.parallel:result is None 2024-02-23T00:13:38.475 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-02-23T00:13:38.475 DEBUG:teuthology.orchestra.run.smithi047:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-23T00:13:38.648 INFO:teuthology.orchestra.run.smithi047.stdout:ttyS1 2024-02-23T00:13:38.679 DEBUG:teuthology.parallel:result is None 2024-02-23T00:13:38.679 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-23T00:13:38.685 INFO:teuthology.task.internal:Creating test directory... 2024-02-23T00:13:38.686 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-23T00:13:38.714 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-23T00:13:38.971 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-23T00:13:38.978 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-23T00:13:39.053 INFO:teuthology.task.internal:Creating archive directory... 2024-02-23T00:13:39.054 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-23T00:13:39.103 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-23T00:13:39.316 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-23T00:13:39.323 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-23T00:13:39.323 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-23T00:13:39.337 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-23T00:13:39.396 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-23T00:13:39.406 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-23T00:13:39.691 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-23T00:13:39.769 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-23T00:13:39.802 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-23T00:13:39.811 INFO:teuthology.task.internal:Configuring sudo... 2024-02-23T00:13:39.812 DEBUG:teuthology.orchestra.run.smithi047:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-23T00:13:39.850 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-23T00:13:40.061 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-23T00:13:40.070 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-23T00:13:40.070 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-23T00:13:40.100 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-23T00:13:40.299 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-23T00:13:40.352 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-23T00:13:40.399 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-23T00:13:40.492 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-23T00:13:40.528 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-02-23T00:13:40.528 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-23T00:13:40.596 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-23T00:13:40.649 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-23T00:13:40.692 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-23T00:13:40.781 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-23T00:13:40.812 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-02-23T00:13:40.812 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-23T00:13:40.877 DEBUG:teuthology.orchestra.run.smithi047:> sudo service rsyslog restart 2024-02-23T00:13:40.879 DEBUG:teuthology.orchestra.run.smithi196:> sudo service rsyslog restart 2024-02-23T00:13:40.959 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-23T00:13:40.982 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-23T00:13:41.173 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-23T00:13:41.181 INFO:teuthology.task.internal:Starting timer... 2024-02-23T00:13:41.181 INFO:teuthology.run_tasks:Running task pcp... 2024-02-23T00:13:41.211 INFO:teuthology.run_tasks:Running task selinux... 2024-02-23T00:13:41.272 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-23T00:13:41.273 DEBUG:teuthology.orchestra.run.smithi047:> sudo service auditd rotate 2024-02-23T00:13:41.369 INFO:teuthology.orchestra.run.smithi047.stdout:Rotating logs: 2024-02-23T00:13:41.371 DEBUG:teuthology.orchestra.run.smithi196:> sudo service auditd rotate 2024-02-23T00:13:41.451 INFO:teuthology.orchestra.run.smithi196.stdout:Rotating logs: 2024-02-23T00:13:41.453 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-23T00:13:41.453 DEBUG:teuthology.orchestra.run.smithi047:> /usr/sbin/getenforce 2024-02-23T00:13:41.504 INFO:teuthology.orchestra.run.smithi047.stdout:Permissive 2024-02-23T00:13:41.505 DEBUG:teuthology.orchestra.run.smithi196:> /usr/sbin/getenforce 2024-02-23T00:13:41.538 INFO:teuthology.orchestra.run.smithi196.stdout:Permissive 2024-02-23T00:13:41.538 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi047.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi196.front.sepia.ceph.com': 'permissive'} 2024-02-23T00:13:41.539 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-23T00:13:41.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-23T00:13:41.567 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-23T00:13:41.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-23T00:13:41.605 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-23T00:13:41.605 DEBUG:teuthology.orchestra.run.smithi047:> sudo /usr/sbin/setenforce permissive 2024-02-23T00:13:41.635 DEBUG:teuthology.orchestra.run.smithi196:> sudo /usr/sbin/setenforce permissive 2024-02-23T00:13:41.674 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-23T00:13:41.683 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-02-23T00:13:41.705 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-02-23T00:13:41.847 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-23T00:13:41.881 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-23T00:13:41.892 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi047.front.sepia.ceph.com,smithi196.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-23T00:24:02.694 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi047.front.sepia.ceph.com'), Remote(name='ubuntu@smithi196.front.sepia.ceph.com')] 2024-02-23T00:24:02.720 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-02-23T00:24:02.721 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-23T00:24:02.844 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-02-23T00:24:02.920 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-02-23T00:24:02.920 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-02-23T00:24:02.921 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-23T00:24:03.027 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-02-23T00:24:03.096 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-02-23T00:24:03.096 INFO:teuthology.run_tasks:Running task clock... 2024-02-23T00:24:03.107 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-23T00:24:03.107 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-23T00:24:03.108 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-23T00:24:03.110 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-23T00:24:03.110 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-23T00:24:03.148 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-23T00:24:03.161 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-23T00:24:03.171 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-23T00:24:03.186 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-23T00:24:03.227 INFO:teuthology.orchestra.run.smithi047.stderr:sudo: ntpd: command not found 2024-02-23T00:24:03.239 INFO:teuthology.orchestra.run.smithi047.stdout:506 Cannot talk to daemon 2024-02-23T00:24:03.255 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-23T00:24:03.258 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: ntpd: command not found 2024-02-23T00:24:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-23T00:24:03.270 INFO:teuthology.orchestra.run.smithi196.stdout:506 Cannot talk to daemon 2024-02-23T00:24:03.285 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-23T00:24:03.298 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-23T00:24:03.335 INFO:teuthology.orchestra.run.smithi047.stderr:bash: ntpq: command not found 2024-02-23T00:24:03.338 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-23T00:24:03.338 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2024-02-23T00:24:03.339 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-23T00:24:03.339 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-23T00:24:03.339 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-23T00:24:03.339 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-23T00:24:03.366 INFO:teuthology.orchestra.run.smithi196.stderr:bash: ntpq: command not found 2024-02-23T00:24:03.370 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-23T00:24:03.370 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-02-23T00:24:03.370 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-23T00:24:03.370 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-23T00:24:03.370 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-23T00:24:03.370 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-23T00:24:03.371 INFO:teuthology.run_tasks:Running task pexec... 2024-02-23T00:24:03.380 INFO:teuthology.task.pexec:Executing custom commands... 2024-02-23T00:24:03.380 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi047.front.sepia.ceph.com 2024-02-23T00:24:03.381 DEBUG:teuthology.orchestra.run.smithi047:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-23T00:24:03.381 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi196.front.sepia.ceph.com 2024-02-23T00:24:03.381 DEBUG:teuthology.orchestra.run.smithi196:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-23T00:24:04.199 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:23 ago on Fri 23 Feb 2024 12:22:41 AM UTC. 2024-02-23T00:24:04.208 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:31 ago on Fri 23 Feb 2024 12:22:33 AM UTC. 2024-02-23T00:24:06.249 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-02-23T00:24:06.250 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:24:06.250 INFO:teuthology.orchestra.run.smithi196.stdout: Package Architecture Version Repository Size 2024-02-23T00:24:06.250 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:24:06.250 INFO:teuthology.orchestra.run.smithi196.stdout:Resetting modules: 2024-02-23T00:24:06.250 INFO:teuthology.orchestra.run.smithi196.stdout: container-tools 2024-02-23T00:24:06.250 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:24:06.250 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-02-23T00:24:06.250 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:24:06.251 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:24:06.405 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-02-23T00:24:06.406 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:24:06.406 INFO:teuthology.orchestra.run.smithi047.stdout: Package Architecture Version Repository Size 2024-02-23T00:24:06.406 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:24:06.407 INFO:teuthology.orchestra.run.smithi047.stdout:Resetting modules: 2024-02-23T00:24:06.407 INFO:teuthology.orchestra.run.smithi047.stdout: container-tools 2024-02-23T00:24:06.407 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:24:06.407 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-02-23T00:24:06.407 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:24:06.407 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:24:06.506 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-02-23T00:24:06.688 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-02-23T00:24:07.348 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:34 ago on Fri 23 Feb 2024 12:22:33 AM UTC. 2024-02-23T00:24:07.578 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:26 ago on Fri 23 Feb 2024 12:22:41 AM UTC. 2024-02-23T00:24:09.396 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-02-23T00:24:09.431 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:24:09.431 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-02-23T00:24:09.431 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:24:09.431 INFO:teuthology.orchestra.run.smithi196.stdout:Installing group/module packages: 2024-02-23T00:24:09.431 INFO:teuthology.orchestra.run.smithi196.stdout: aardvark-dns x86_64 2:1.10.0-1.module_el8+871+552be958 appstream 1.0 M 2024-02-23T00:24:09.431 INFO:teuthology.orchestra.run.smithi196.stdout: buildah x86_64 2:1.33.5-1.module_el8+885+7da147f3 appstream 9.7 M 2024-02-23T00:24:09.431 INFO:teuthology.orchestra.run.smithi196.stdout: cockpit-podman noarch 83-1.module_el8+847+7863d4e6 appstream 688 k 2024-02-23T00:24:09.431 INFO:teuthology.orchestra.run.smithi196.stdout: crun x86_64 1.14.1-1.module_el8+951+32019cde appstream 257 k 2024-02-23T00:24:09.432 INFO:teuthology.orchestra.run.smithi196.stdout: netavark x86_64 2:1.10.3-1.module_el8+951+32019cde appstream 4.1 M 2024-02-23T00:24:09.432 INFO:teuthology.orchestra.run.smithi196.stdout: python3-podman noarch 4.9.0-1.module_el8+847+7863d4e6 appstream 180 k 2024-02-23T00:24:09.432 INFO:teuthology.orchestra.run.smithi196.stdout: skopeo x86_64 2:1.14.3-0.1.module_el8+885+7da147f3 appstream 8.8 M 2024-02-23T00:24:09.432 INFO:teuthology.orchestra.run.smithi196.stdout: toolbox x86_64 0.0.99.5-1.module_el8+847+7863d4e6 appstream 2.5 M 2024-02-23T00:24:09.432 INFO:teuthology.orchestra.run.smithi196.stdout: udica noarch 0.2.6-20.module_el8+804+f131391c appstream 53 k 2024-02-23T00:24:09.432 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-02-23T00:24:09.432 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 appstream 25 k 2024-02-23T00:24:09.432 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyxdg noarch 0.25-16.el8 appstream 94 k 2024-02-23T00:24:09.432 INFO:teuthology.orchestra.run.smithi196.stdout: yajl x86_64 2.1.0-12.el8 appstream 41 k 2024-02-23T00:24:09.432 INFO:teuthology.orchestra.run.smithi196.stdout:Installing weak dependencies: 2024-02-23T00:24:09.432 INFO:teuthology.orchestra.run.smithi196.stdout: criu-libs x86_64 3.15-3.module_el8.7.0+1216+b022c01d appstream 38 k 2024-02-23T00:24:09.432 INFO:teuthology.orchestra.run.smithi196.stdout:Installing module profiles: 2024-02-23T00:24:09.432 INFO:teuthology.orchestra.run.smithi196.stdout: container-tools/common 2024-02-23T00:24:09.432 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:24:09.433 INFO:teuthology.orchestra.run.smithi196.stdout:Enabling module streams: 2024-02-23T00:24:09.433 INFO:teuthology.orchestra.run.smithi196.stdout: container-tools rhel8 2024-02-23T00:24:09.433 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:24:09.433 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-02-23T00:24:09.433 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:24:09.433 INFO:teuthology.orchestra.run.smithi196.stdout:Install 13 Packages 2024-02-23T00:24:09.433 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:24:09.434 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 27 M 2024-02-23T00:24:09.434 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 91 M 2024-02-23T00:24:09.434 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-02-23T00:24:09.775 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-02-23T00:24:09.816 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:24:09.816 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-02-23T00:24:09.816 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:24:09.816 INFO:teuthology.orchestra.run.smithi047.stdout:Installing group/module packages: 2024-02-23T00:24:09.816 INFO:teuthology.orchestra.run.smithi047.stdout: aardvark-dns x86_64 2:1.10.0-1.module_el8+871+552be958 appstream 1.0 M 2024-02-23T00:24:09.816 INFO:teuthology.orchestra.run.smithi047.stdout: buildah x86_64 2:1.33.5-1.module_el8+885+7da147f3 appstream 9.7 M 2024-02-23T00:24:09.816 INFO:teuthology.orchestra.run.smithi047.stdout: cockpit-podman noarch 83-1.module_el8+847+7863d4e6 appstream 688 k 2024-02-23T00:24:09.816 INFO:teuthology.orchestra.run.smithi047.stdout: crun x86_64 1.14.1-1.module_el8+951+32019cde appstream 257 k 2024-02-23T00:24:09.816 INFO:teuthology.orchestra.run.smithi047.stdout: netavark x86_64 2:1.10.3-1.module_el8+951+32019cde appstream 4.1 M 2024-02-23T00:24:09.817 INFO:teuthology.orchestra.run.smithi047.stdout: python3-podman noarch 4.9.0-1.module_el8+847+7863d4e6 appstream 180 k 2024-02-23T00:24:09.817 INFO:teuthology.orchestra.run.smithi047.stdout: skopeo x86_64 2:1.14.3-0.1.module_el8+885+7da147f3 appstream 8.8 M 2024-02-23T00:24:09.817 INFO:teuthology.orchestra.run.smithi047.stdout: toolbox x86_64 0.0.99.5-1.module_el8+847+7863d4e6 appstream 2.5 M 2024-02-23T00:24:09.817 INFO:teuthology.orchestra.run.smithi047.stdout: udica noarch 0.2.6-20.module_el8+804+f131391c appstream 53 k 2024-02-23T00:24:09.817 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-02-23T00:24:09.817 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 appstream 25 k 2024-02-23T00:24:09.817 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyxdg noarch 0.25-16.el8 appstream 94 k 2024-02-23T00:24:09.817 INFO:teuthology.orchestra.run.smithi047.stdout: yajl x86_64 2.1.0-12.el8 appstream 41 k 2024-02-23T00:24:09.817 INFO:teuthology.orchestra.run.smithi047.stdout:Installing weak dependencies: 2024-02-23T00:24:09.817 INFO:teuthology.orchestra.run.smithi047.stdout: criu-libs x86_64 3.15-3.module_el8.7.0+1216+b022c01d appstream 38 k 2024-02-23T00:24:09.817 INFO:teuthology.orchestra.run.smithi047.stdout:Installing module profiles: 2024-02-23T00:24:09.817 INFO:teuthology.orchestra.run.smithi047.stdout: container-tools/common 2024-02-23T00:24:09.817 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:24:09.817 INFO:teuthology.orchestra.run.smithi047.stdout:Enabling module streams: 2024-02-23T00:24:09.817 INFO:teuthology.orchestra.run.smithi047.stdout: container-tools rhel8 2024-02-23T00:24:09.818 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:24:09.818 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-02-23T00:24:09.818 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:24:09.818 INFO:teuthology.orchestra.run.smithi047.stdout:Install 13 Packages 2024-02-23T00:24:09.818 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:24:09.819 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 27 M 2024-02-23T00:24:09.819 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 91 M 2024-02-23T00:24:09.819 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-02-23T00:24:09.845 INFO:teuthology.orchestra.run.smithi196.stdout:(1/13): cockpit-podman-83-1.module_el8+847+7863 3.0 MB/s | 688 kB 00:00 2024-02-23T00:24:09.895 INFO:teuthology.orchestra.run.smithi196.stdout:(2/13): aardvark-dns-1.10.0-1.module_el8+871+55 3.6 MB/s | 1.0 MB 00:00 2024-02-23T00:24:09.920 INFO:teuthology.orchestra.run.smithi196.stdout:(3/13): criu-libs-3.15-3.module_el8.7.0+1216+b0 503 kB/s | 38 kB 00:00 2024-02-23T00:24:09.954 INFO:teuthology.orchestra.run.smithi196.stdout:(4/13): crun-1.14.1-1.module_el8+951+32019cde.x 4.3 MB/s | 257 kB 00:00 2024-02-23T00:24:09.996 INFO:teuthology.orchestra.run.smithi196.stdout:(5/13): python3-podman-4.9.0-1.module_el8+847+7 4.2 MB/s | 180 kB 00:00 2024-02-23T00:24:10.038 INFO:teuthology.orchestra.run.smithi196.stdout:(6/13): python3-pytoml-0.1.14-5.git7dea353.el8. 595 kB/s | 25 kB 00:00 2024-02-23T00:24:10.113 INFO:teuthology.orchestra.run.smithi196.stdout:(7/13): python3-pyxdg-0.25-16.el8.noarch.rpm 1.2 MB/s | 94 kB 00:00 2024-02-23T00:24:10.205 INFO:teuthology.orchestra.run.smithi196.stdout:(8/13): netavark-1.10.3-1.module_el8+951+32019c 14 MB/s | 4.1 MB 00:00 2024-02-23T00:24:10.378 INFO:teuthology.orchestra.run.smithi196.stdout:(9/13): toolbox-0.0.99.5-1.module_el8+847+7863d 15 MB/s | 2.5 MB 00:00 2024-02-23T00:24:10.481 INFO:teuthology.orchestra.run.smithi196.stdout:(10/13): buildah-1.33.5-1.module_el8+885+7da147 11 MB/s | 9.7 MB 00:00 2024-02-23T00:24:10.506 INFO:teuthology.orchestra.run.smithi196.stdout:(11/13): udica-0.2.6-20.module_el8+804+f131391c 411 kB/s | 53 kB 00:00 2024-02-23T00:24:10.532 INFO:teuthology.orchestra.run.smithi196.stdout:(12/13): yajl-2.1.0-12.el8.x86_64.rpm 817 kB/s | 41 kB 00:00 2024-02-23T00:24:10.952 INFO:teuthology.orchestra.run.smithi196.stdout:(13/13): skopeo-1.14.3-0.1.module_el8+885+7da14 11 MB/s | 8.8 MB 00:00 2024-02-23T00:24:10.952 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:24:10.952 INFO:teuthology.orchestra.run.smithi196.stdout:Total 18 MB/s | 27 MB 00:01 2024-02-23T00:24:11.222 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-02-23T00:24:11.229 INFO:teuthology.orchestra.run.smithi047.stdout:(1/13): cockpit-podman-83-1.module_el8+847+7863 3.2 MB/s | 688 kB 00:00 2024-02-23T00:24:11.243 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-02-23T00:24:11.244 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-02-23T00:24:11.297 INFO:teuthology.orchestra.run.smithi047.stdout:(2/13): aardvark-dns-1.10.0-1.module_el8+871+55 3.7 MB/s | 1.0 MB 00:00 2024-02-23T00:24:11.322 INFO:teuthology.orchestra.run.smithi047.stdout:(3/13): criu-libs-3.15-3.module_el8.7.0+1216+b0 413 kB/s | 38 kB 00:00 2024-02-23T00:24:11.347 INFO:teuthology.orchestra.run.smithi047.stdout:(4/13): crun-1.14.1-1.module_el8+951+32019cde.x 5.0 MB/s | 257 kB 00:00 2024-02-23T00:24:11.397 INFO:teuthology.orchestra.run.smithi047.stdout:(5/13): python3-podman-4.9.0-1.module_el8+847+7 3.5 MB/s | 180 kB 00:00 2024-02-23T00:24:11.447 INFO:teuthology.orchestra.run.smithi047.stdout:(6/13): python3-pytoml-0.1.14-5.git7dea353.el8. 495 kB/s | 25 kB 00:00 2024-02-23T00:24:11.498 INFO:teuthology.orchestra.run.smithi047.stdout:(7/13): python3-pyxdg-0.25-16.el8.noarch.rpm 1.8 MB/s | 94 kB 00:00 2024-02-23T00:24:11.508 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-02-23T00:24:11.512 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-02-23T00:24:11.573 INFO:teuthology.orchestra.run.smithi047.stdout:(8/13): netavark-1.10.3-1.module_el8+951+32019c 16 MB/s | 4.1 MB 00:00 2024-02-23T00:24:11.707 INFO:teuthology.orchestra.run.smithi047.stdout:(9/13): toolbox-0.0.99.5-1.module_el8+847+7863d 19 MB/s | 2.5 MB 00:00 2024-02-23T00:24:11.824 INFO:teuthology.orchestra.run.smithi047.stdout:(10/13): buildah-1.33.5-1.module_el8+885+7da147 12 MB/s | 9.7 MB 00:00 2024-02-23T00:24:11.849 INFO:teuthology.orchestra.run.smithi047.stdout:(11/13): udica-0.2.6-20.module_el8+804+f131391c 371 kB/s | 53 kB 00:00 2024-02-23T00:24:11.874 INFO:teuthology.orchestra.run.smithi047.stdout:(12/13): yajl-2.1.0-12.el8.x86_64.rpm 820 kB/s | 41 kB 00:00 2024-02-23T00:24:12.075 INFO:teuthology.orchestra.run.smithi047.stdout:(13/13): skopeo-1.14.3-0.1.module_el8+885+7da14 15 MB/s | 8.8 MB 00:00 2024-02-23T00:24:12.075 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:24:12.075 INFO:teuthology.orchestra.run.smithi047.stdout:Total 12 MB/s | 27 MB 00:02 2024-02-23T00:24:12.225 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-02-23T00:24:12.353 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-02-23T00:24:12.376 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-02-23T00:24:12.376 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-02-23T00:24:12.652 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-02-23T00:24:12.656 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-02-23T00:24:13.212 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : yajl-2.1.0-12.el8.x86_64 1/13 2024-02-23T00:24:13.369 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-02-23T00:24:13.424 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2/13 2024-02-23T00:24:13.669 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/13 2024-02-23T00:24:13.896 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 4/13 2024-02-23T00:24:14.123 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 5/13 2024-02-23T00:24:14.422 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : yajl-2.1.0-12.el8.x86_64 1/13 2024-02-23T00:24:14.622 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2/13 2024-02-23T00:24:14.672 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 6/13 2024-02-23T00:24:14.850 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/13 2024-02-23T00:24:15.041 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 4/13 2024-02-23T00:24:15.283 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 5/13 2024-02-23T00:24:15.629 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 7/13 2024-02-23T00:24:15.886 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 8/13 2024-02-23T00:24:15.886 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 6/13 2024-02-23T00:24:16.081 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : crun-1.14.1-1.module_el8+951+32019cde.x86_64 9/13 2024-02-23T00:24:16.511 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 10/13 2024-02-23T00:24:16.727 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 11/13 2024-02-23T00:24:16.916 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 7/13 2024-02-23T00:24:16.972 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-02-23T00:24:17.151 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 13/13 2024-02-23T00:24:17.169 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 8/13 2024-02-23T00:24:17.451 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : crun-1.14.1-1.module_el8+951+32019cde.x86_64 9/13 2024-02-23T00:24:17.605 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 13/13 2024-02-23T00:24:17.605 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 1/13 2024-02-23T00:24:17.605 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 2/13 2024-02-23T00:24:17.605 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 3/13 2024-02-23T00:24:17.606 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 4/13 2024-02-23T00:24:17.606 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : crun-1.14.1-1.module_el8+951+32019cde.x86_64 5/13 2024-02-23T00:24:17.606 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 6/13 2024-02-23T00:24:17.606 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 7/13 2024-02-23T00:24:17.606 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 8/13 2024-02-23T00:24:17.606 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 9/13 2024-02-23T00:24:17.606 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 10/13 2024-02-23T00:24:17.606 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 11/13 2024-02-23T00:24:17.606 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-02-23T00:24:17.962 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 10/13 2024-02-23T00:24:18.149 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 13/13 2024-02-23T00:24:18.150 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:24:18.150 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-02-23T00:24:18.150 INFO:teuthology.orchestra.run.smithi196.stdout: aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x86_64 2024-02-23T00:24:18.150 INFO:teuthology.orchestra.run.smithi196.stdout: buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 2024-02-23T00:24:18.150 INFO:teuthology.orchestra.run.smithi196.stdout: cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 2024-02-23T00:24:18.150 INFO:teuthology.orchestra.run.smithi196.stdout: criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 2024-02-23T00:24:18.150 INFO:teuthology.orchestra.run.smithi196.stdout: crun-1.14.1-1.module_el8+951+32019cde.x86_64 2024-02-23T00:24:18.150 INFO:teuthology.orchestra.run.smithi196.stdout: netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 2024-02-23T00:24:18.150 INFO:teuthology.orchestra.run.smithi196.stdout: python3-podman-4.9.0-1.module_el8+847+7863d4e6.noarch 2024-02-23T00:24:18.150 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-02-23T00:24:18.150 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-02-23T00:24:18.150 INFO:teuthology.orchestra.run.smithi196.stdout: skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2024-02-23T00:24:18.150 INFO:teuthology.orchestra.run.smithi196.stdout: toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 2024-02-23T00:24:18.150 INFO:teuthology.orchestra.run.smithi196.stdout: udica-0.2.6-20.module_el8+804+f131391c.noarch 2024-02-23T00:24:18.151 INFO:teuthology.orchestra.run.smithi196.stdout: yajl-2.1.0-12.el8.x86_64 2024-02-23T00:24:18.151 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:24:18.151 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-02-23T00:24:18.185 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 11/13 2024-02-23T00:24:18.422 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-02-23T00:24:18.467 DEBUG:teuthology.parallel:result is None 2024-02-23T00:24:18.632 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 13/13 2024-02-23T00:24:19.066 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 13/13 2024-02-23T00:24:19.066 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 1/13 2024-02-23T00:24:19.066 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 2/13 2024-02-23T00:24:19.066 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 3/13 2024-02-23T00:24:19.066 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 4/13 2024-02-23T00:24:19.066 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : crun-1.14.1-1.module_el8+951+32019cde.x86_64 5/13 2024-02-23T00:24:19.066 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 6/13 2024-02-23T00:24:19.066 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 7/13 2024-02-23T00:24:19.066 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 8/13 2024-02-23T00:24:19.066 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 9/13 2024-02-23T00:24:19.067 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 10/13 2024-02-23T00:24:19.067 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 11/13 2024-02-23T00:24:19.067 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-02-23T00:24:19.607 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 13/13 2024-02-23T00:24:19.607 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:24:19.607 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-02-23T00:24:19.607 INFO:teuthology.orchestra.run.smithi047.stdout: aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x86_64 2024-02-23T00:24:19.607 INFO:teuthology.orchestra.run.smithi047.stdout: buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 2024-02-23T00:24:19.607 INFO:teuthology.orchestra.run.smithi047.stdout: cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 2024-02-23T00:24:19.607 INFO:teuthology.orchestra.run.smithi047.stdout: criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 2024-02-23T00:24:19.607 INFO:teuthology.orchestra.run.smithi047.stdout: crun-1.14.1-1.module_el8+951+32019cde.x86_64 2024-02-23T00:24:19.607 INFO:teuthology.orchestra.run.smithi047.stdout: netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 2024-02-23T00:24:19.607 INFO:teuthology.orchestra.run.smithi047.stdout: python3-podman-4.9.0-1.module_el8+847+7863d4e6.noarch 2024-02-23T00:24:19.608 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-02-23T00:24:19.608 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-02-23T00:24:19.608 INFO:teuthology.orchestra.run.smithi047.stdout: skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2024-02-23T00:24:19.608 INFO:teuthology.orchestra.run.smithi047.stdout: toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 2024-02-23T00:24:19.608 INFO:teuthology.orchestra.run.smithi047.stdout: udica-0.2.6-20.module_el8+804+f131391c.noarch 2024-02-23T00:24:19.608 INFO:teuthology.orchestra.run.smithi047.stdout: yajl-2.1.0-12.el8.x86_64 2024-02-23T00:24:19.608 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:24:19.608 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-02-23T00:24:19.886 DEBUG:teuthology.parallel:result is None 2024-02-23T00:24:19.886 INFO:teuthology.run_tasks:Running task install... 2024-02-23T00:24:19.905 DEBUG:teuthology.task.install:project ceph 2024-02-23T00:24:19.905 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'eb66ed921e744301e6863be6353618d63967ea59'}} 2024-02-23T00:24:19.905 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'eb66ed921e744301e6863be6353618d63967ea59'} 2024-02-23T00:24:19.906 INFO:teuthology.task.install:Using flavor: default 2024-02-23T00:24:19.912 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-23T00:24:19.913 INFO:teuthology.task.install:extra packages: [] 2024-02-23T00:24:19.913 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'eb66ed921e744301e6863be6353618d63967ea59', 'tag': None, 'wait_for_package': False} 2024-02-23T00:24:19.913 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=eb66ed921e744301e6863be6353618d63967ea59 2024-02-23T00:24:19.915 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'eb66ed921e744301e6863be6353618d63967ea59', 'tag': None, 'wait_for_package': False} 2024-02-23T00:24:19.916 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=eb66ed921e744301e6863be6353618d63967ea59 2024-02-23T00:24:20.089 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/pacific/eb66ed921e744301e6863be6353618d63967ea59/centos/8/flavors/default/ 2024-02-23T00:24:20.089 INFO:teuthology.task.install.rpm:Package version is 16.2.14-550.geb66ed92 2024-02-23T00:24:20.161 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/pacific/eb66ed921e744301e6863be6353618d63967ea59/centos/8/flavors/default/ 2024-02-23T00:24:20.161 INFO:teuthology.task.install.rpm:Package version is 16.2.14-550.geb66ed92 2024-02-23T00:24:20.205 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/pacific/eb66ed921e744301e6863be6353618d63967ea59/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/pacific/eb66ed921e744301e6863be6353618d63967ea59/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/pacific/eb66ed921e744301e6863be6353618d63967ea59/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-23T00:24:20.205 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-02-23T00:24:20.205 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-23T00:24:20.239 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-23T00:24:20.240 DEBUG:teuthology.orchestra.run.smithi196:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/eb66ed921e744301e6863be6353618d63967ea59/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-23T00:24:20.277 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/pacific/eb66ed921e744301e6863be6353618d63967ea59/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/pacific/eb66ed921e744301e6863be6353618d63967ea59/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/pacific/eb66ed921e744301e6863be6353618d63967ea59/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-23T00:24:20.277 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-02-23T00:24:20.277 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-23T00:24:20.307 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-23T00:24:20.307 DEBUG:teuthology.orchestra.run.smithi047:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/eb66ed921e744301e6863be6353618d63967ea59/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-23T00:24:20.313 DEBUG:teuthology.orchestra.run.smithi196:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-23T00:24:20.376 DEBUG:teuthology.orchestra.run.smithi047:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-23T00:24:20.397 DEBUG:teuthology.orchestra.run.smithi196:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-23T00:24:20.454 DEBUG:teuthology.orchestra.run.smithi047:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-23T00:24:20.466 INFO:teuthology.orchestra.run.smithi196.stdout:check_obsoletes = 1 2024-02-23T00:24:20.467 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum clean all 2024-02-23T00:24:20.522 INFO:teuthology.orchestra.run.smithi047.stdout:check_obsoletes = 1 2024-02-23T00:24:20.523 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum clean all 2024-02-23T00:24:20.737 INFO:teuthology.orchestra.run.smithi196.stdout:76 files removed 2024-02-23T00:24:20.765 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-radosgw 2024-02-23T00:24:20.802 INFO:teuthology.orchestra.run.smithi047.stdout:76 files removed 2024-02-23T00:24:20.831 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-radosgw 2024-02-23T00:24:21.431 INFO:teuthology.orchestra.run.smithi196.stdout:ceph packages for x86_64 295 kB/s | 76 kB 00:00 2024-02-23T00:24:21.517 INFO:teuthology.orchestra.run.smithi047.stdout:ceph packages for x86_64 298 kB/s | 76 kB 00:00 2024-02-23T00:24:21.732 INFO:teuthology.orchestra.run.smithi196.stdout:ceph noarch packages 58 kB/s | 15 kB 00:00 2024-02-23T00:24:21.918 INFO:teuthology.orchestra.run.smithi047.stdout:ceph noarch packages 43 kB/s | 15 kB 00:00 2024-02-23T00:24:22.091 INFO:teuthology.orchestra.run.smithi196.stdout:ceph source packages 4.4 kB/s | 1.4 kB 00:00 2024-02-23T00:24:22.227 INFO:teuthology.orchestra.run.smithi047.stdout:ceph source packages 5.2 kB/s | 1.4 kB 00:00 2024-02-23T00:24:24.908 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 8 - AppStream 9.8 MB/s | 27 MB 00:02 2024-02-23T00:24:25.670 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 8 - AppStream 8.0 MB/s | 27 MB 00:03 2024-02-23T00:24:30.594 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 8 - BaseOS 12 MB/s | 10 MB 00:00 2024-02-23T00:24:31.470 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 8 - BaseOS 17 MB/s | 10 MB 00:00 2024-02-23T00:24:32.528 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 8 - Extras 52 kB/s | 18 kB 00:00 2024-02-23T00:24:32.845 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 8 - Extras common packages 33 kB/s | 6.9 kB 00:00 2024-02-23T00:24:33.538 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 8 - Extras 48 kB/s | 18 kB 00:00 2024-02-23T00:24:33.880 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 8 - Extras common packages 31 kB/s | 6.9 kB 00:00 2024-02-23T00:28:57.930 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 8 - PowerTools 19 kB/s | 4.9 MB 04:24 2024-02-23T00:28:58.912 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 8 - PowerTools 19 kB/s | 4.9 MB 04:24 2024-02-23T00:28:59.212 INFO:teuthology.orchestra.run.smithi196.stdout:Copr repo for python3-asyncssh owned by ceph 10 kB/s | 3.5 kB 00:00 2024-02-23T00:29:00.071 INFO:teuthology.orchestra.run.smithi047.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2024-02-23T00:29:01.165 INFO:teuthology.orchestra.run.smithi196.stdout:Extra Packages for Enterprise Linux 8.8 MB/s | 16 MB 00:01 2024-02-23T00:29:04.066 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 4.2 MB/s | 16 MB 00:03 2024-02-23T00:29:04.929 INFO:teuthology.orchestra.run.smithi196.stdout:lab-extras 400 kB/s | 24 kB 00:00 2024-02-23T00:29:08.114 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 331 kB/s | 24 kB 00:00 2024-02-23T00:29:08.475 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-02-23T00:29:08.477 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:29:08.477 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-02-23T00:29:08.478 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:29:08.478 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-02-23T00:29:08.478 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw x86_64 2:16.2.14-550.geb66ed92.el8 ceph 12 M 2024-02-23T00:29:08.478 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrading: 2024-02-23T00:29:08.478 INFO:teuthology.orchestra.run.smithi196.stdout: librados2 x86_64 2:16.2.14-550.geb66ed92.el8 ceph 3.7 M 2024-02-23T00:29:08.478 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1 x86_64 2:16.2.14-550.geb66ed92.el8 ceph 4.2 M 2024-02-23T00:29:08.478 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-02-23T00:29:08.478 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base x86_64 2:16.2.14-550.geb66ed92.el8 ceph 6.1 M 2024-02-23T00:29:08.478 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common x86_64 2:16.2.14-550.geb66ed92.el8 ceph 24 M 2024-02-23T00:29:08.478 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux x86_64 2:16.2.14-550.geb66ed92.el8 ceph 26 k 2024-02-23T00:29:08.478 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-02-23T00:29:08.479 INFO:teuthology.orchestra.run.smithi196.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2024-02-23T00:29:08.479 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2 x86_64 2:16.2.14-550.geb66ed92.el8 ceph 825 k 2024-02-23T00:29:08.479 INFO:teuthology.orchestra.run.smithi196.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-02-23T00:29:08.479 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-02-23T00:29:08.479 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1 x86_64 2:16.2.14-550.geb66ed92.el8 ceph 494 k 2024-02-23T00:29:08.479 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-02-23T00:29:08.479 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2 x86_64 2:16.2.14-550.geb66ed92.el8 ceph 3.8 M 2024-02-23T00:29:08.479 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-02-23T00:29:08.479 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-02-23T00:29:08.479 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse x86_64 2:16.2.14-550.geb66ed92.el8 ceph 45 k 2024-02-23T00:29:08.479 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common x86_64 2:16.2.14-550.geb66ed92.el8 ceph 99 k 2024-02-23T00:29:08.479 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs x86_64 2:16.2.14-550.geb66ed92.el8 ceph 215 k 2024-02-23T00:29:08.479 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-02-23T00:29:08.480 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados x86_64 2:16.2.14-550.geb66ed92.el8 ceph 389 k 2024-02-23T00:29:08.480 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd x86_64 2:16.2.14-550.geb66ed92.el8 ceph 369 k 2024-02-23T00:29:08.480 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw x86_64 2:16.2.14-550.geb66ed92.el8 ceph 114 k 2024-02-23T00:29:08.480 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:29:08.480 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-02-23T00:29:08.480 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:29:08.480 INFO:teuthology.orchestra.run.smithi196.stdout:Install 21 Packages 2024-02-23T00:29:08.480 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrade 2 Packages 2024-02-23T00:29:08.480 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:29:08.481 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 58 M 2024-02-23T00:29:08.481 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-02-23T00:29:11.755 INFO:teuthology.orchestra.run.smithi196.stdout:(1/23): ceph-common-16.2.14-550.geb66ed92.el8.x 14 MB/s | 24 MB 00:01 2024-02-23T00:29:11.805 INFO:teuthology.orchestra.run.smithi196.stdout:(2/23): ceph-selinux-16.2.14-550.geb66ed92.el8. 515 kB/s | 26 kB 00:00 2024-02-23T00:29:11.906 INFO:teuthology.orchestra.run.smithi196.stdout:(3/23): libcephfs2-16.2.14-550.geb66ed92.el8.x8 8.1 MB/s | 825 kB 00:00 2024-02-23T00:29:11.922 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-02-23T00:29:11.925 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:29:11.925 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-02-23T00:29:11.925 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:29:11.925 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-02-23T00:29:11.926 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw x86_64 2:16.2.14-550.geb66ed92.el8 ceph 12 M 2024-02-23T00:29:11.926 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrading: 2024-02-23T00:29:11.926 INFO:teuthology.orchestra.run.smithi047.stdout: librados2 x86_64 2:16.2.14-550.geb66ed92.el8 ceph 3.7 M 2024-02-23T00:29:11.926 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1 x86_64 2:16.2.14-550.geb66ed92.el8 ceph 4.2 M 2024-02-23T00:29:11.926 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-02-23T00:29:11.926 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base x86_64 2:16.2.14-550.geb66ed92.el8 ceph 6.1 M 2024-02-23T00:29:11.926 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common x86_64 2:16.2.14-550.geb66ed92.el8 ceph 24 M 2024-02-23T00:29:11.926 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux x86_64 2:16.2.14-550.geb66ed92.el8 ceph 26 k 2024-02-23T00:29:11.926 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-02-23T00:29:11.926 INFO:teuthology.orchestra.run.smithi047.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2024-02-23T00:29:11.926 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2 x86_64 2:16.2.14-550.geb66ed92.el8 ceph 825 k 2024-02-23T00:29:11.926 INFO:teuthology.orchestra.run.smithi047.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-02-23T00:29:11.926 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-02-23T00:29:11.927 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1 x86_64 2:16.2.14-550.geb66ed92.el8 ceph 494 k 2024-02-23T00:29:11.927 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-02-23T00:29:11.927 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2 x86_64 2:16.2.14-550.geb66ed92.el8 ceph 3.8 M 2024-02-23T00:29:11.927 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-02-23T00:29:11.927 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-02-23T00:29:11.927 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse x86_64 2:16.2.14-550.geb66ed92.el8 ceph 45 k 2024-02-23T00:29:11.927 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common x86_64 2:16.2.14-550.geb66ed92.el8 ceph 99 k 2024-02-23T00:29:11.927 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs x86_64 2:16.2.14-550.geb66ed92.el8 ceph 215 k 2024-02-23T00:29:11.927 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-02-23T00:29:11.927 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados x86_64 2:16.2.14-550.geb66ed92.el8 ceph 389 k 2024-02-23T00:29:11.927 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd x86_64 2:16.2.14-550.geb66ed92.el8 ceph 369 k 2024-02-23T00:29:11.927 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw x86_64 2:16.2.14-550.geb66ed92.el8 ceph 114 k 2024-02-23T00:29:11.927 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:29:11.927 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-02-23T00:29:11.928 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:29:11.928 INFO:teuthology.orchestra.run.smithi047.stdout:Install 21 Packages 2024-02-23T00:29:11.928 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrade 2 Packages 2024-02-23T00:29:11.928 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:29:11.928 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 58 M 2024-02-23T00:29:11.928 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-02-23T00:29:12.048 INFO:teuthology.orchestra.run.smithi196.stdout:(4/23): ceph-radosgw-16.2.14-550.geb66ed92.el8. 5.7 MB/s | 12 MB 00:02 2024-02-23T00:29:12.073 INFO:teuthology.orchestra.run.smithi196.stdout:(5/23): libradosstriper1-16.2.14-550.geb66ed92. 2.9 MB/s | 494 kB 00:00 2024-02-23T00:29:12.123 INFO:teuthology.orchestra.run.smithi196.stdout:(6/23): python3-ceph-argparse-16.2.14-550.geb66 900 kB/s | 45 kB 00:00 2024-02-23T00:29:12.182 INFO:teuthology.orchestra.run.smithi196.stdout:(7/23): python3-ceph-common-16.2.14-550.geb66ed 1.7 MB/s | 99 kB 00:00 2024-02-23T00:29:12.240 INFO:teuthology.orchestra.run.smithi196.stdout:(8/23): python3-cephfs-16.2.14-550.geb66ed92.el 3.6 MB/s | 215 kB 00:00 2024-02-23T00:29:12.316 INFO:teuthology.orchestra.run.smithi196.stdout:(9/23): python3-rados-16.2.14-550.geb66ed92.el8 5.1 MB/s | 389 kB 00:00 2024-02-23T00:29:12.392 INFO:teuthology.orchestra.run.smithi196.stdout:(10/23): ceph-base-16.2.14-550.geb66ed92.el8.x8 2.6 MB/s | 6.1 MB 00:02 2024-02-23T00:29:12.417 INFO:teuthology.orchestra.run.smithi196.stdout:(11/23): python3-rbd-16.2.14-550.geb66ed92.el8. 3.6 MB/s | 369 kB 00:00 2024-02-23T00:29:12.475 INFO:teuthology.orchestra.run.smithi196.stdout:(12/23): librgw2-16.2.14-550.geb66ed92.el8.x86_ 8.9 MB/s | 3.8 MB 00:00 2024-02-23T00:29:12.501 INFO:teuthology.orchestra.run.smithi196.stdout:(13/23): python3-rgw-16.2.14-550.geb66ed92.el8. 1.0 MB/s | 114 kB 00:00 2024-02-23T00:29:12.609 INFO:teuthology.orchestra.run.smithi196.stdout:(14/23): python3-prettytable-0.7.2-14.el8.noarc 405 kB/s | 44 kB 00:00 2024-02-23T00:29:12.651 INFO:teuthology.orchestra.run.smithi196.stdout:(15/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.4 MB/s | 259 kB 00:00 2024-02-23T00:29:12.685 INFO:teuthology.orchestra.run.smithi196.stdout:(16/23): librdkafka-1.6.1-1.el8.x86_64.rpm 2.5 MB/s | 677 kB 00:00 2024-02-23T00:29:12.718 INFO:teuthology.orchestra.run.smithi196.stdout:(17/23): gperftools-libs-2.7-9.el8.x86_64.rpm 4.5 MB/s | 306 kB 00:00 2024-02-23T00:29:12.744 INFO:teuthology.orchestra.run.smithi196.stdout:(18/23): leveldb-1.22-1.el8.x86_64.rpm 3.0 MB/s | 181 kB 00:00 2024-02-23T00:29:12.761 INFO:teuthology.orchestra.run.smithi196.stdout:(19/23): librabbitmq-0.9.0-5.el8.x86_64.rpm 312 kB/s | 47 kB 00:00 2024-02-23T00:29:12.786 INFO:teuthology.orchestra.run.smithi196.stdout:(20/23): liboath-2.6.2-3.el8.x86_64.rpm 877 kB/s | 59 kB 00:00 2024-02-23T00:29:12.811 INFO:teuthology.orchestra.run.smithi196.stdout:(21/23): libunwind-1.3.1-3.el8.x86_64.rpm 1.1 MB/s | 75 kB 00:00 2024-02-23T00:29:13.270 INFO:teuthology.orchestra.run.smithi196.stdout:(22/23): librbd1-16.2.14-550.geb66ed92.el8.x86_ 8.6 MB/s | 4.2 MB 00:00 2024-02-23T00:29:13.740 INFO:teuthology.orchestra.run.smithi047.stdout:(1/23): ceph-radosgw-16.2.14-550.geb66ed92.el8. 8.3 MB/s | 12 MB 00:01 2024-02-23T00:29:13.831 INFO:teuthology.orchestra.run.smithi047.stdout:(2/23): ceph-base-16.2.14-550.geb66ed92.el8.x86 4.1 MB/s | 6.1 MB 00:01 2024-02-23T00:29:13.848 INFO:teuthology.orchestra.run.smithi047.stdout:(3/23): ceph-selinux-16.2.14-550.geb66ed92.el8. 236 kB/s | 26 kB 00:00 2024-02-23T00:29:13.949 INFO:teuthology.orchestra.run.smithi047.stdout:(4/23): libradosstriper1-16.2.14-550.geb66ed92. 4.8 MB/s | 494 kB 00:00 2024-02-23T00:29:14.033 INFO:teuthology.orchestra.run.smithi047.stdout:(5/23): libcephfs2-16.2.14-550.geb66ed92.el8.x8 4.0 MB/s | 825 kB 00:00 2024-02-23T00:29:14.036 INFO:teuthology.orchestra.run.smithi196.stdout:(23/23): librados2-16.2.14-550.geb66ed92.el8.x8 2.9 MB/s | 3.7 MB 00:01 2024-02-23T00:29:14.039 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:29:14.039 INFO:teuthology.orchestra.run.smithi196.stdout:Total 10 MB/s | 58 MB 00:05 2024-02-23T00:29:14.082 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-02-23T00:29:14.116 INFO:teuthology.orchestra.run.smithi047.stdout:(6/23): python3-ceph-argparse-16.2.14-550.geb66 541 kB/s | 45 kB 00:00 2024-02-23T00:29:14.130 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-02-23T00:29:14.130 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-02-23T00:29:14.275 INFO:teuthology.orchestra.run.smithi047.stdout:(7/23): python3-ceph-common-16.2.14-550.geb66ed 623 kB/s | 99 kB 00:00 2024-02-23T00:29:14.409 INFO:teuthology.orchestra.run.smithi047.stdout:(8/23): librgw2-16.2.14-550.geb66ed92.el8.x86_6 8.3 MB/s | 3.8 MB 00:00 2024-02-23T00:29:14.435 INFO:teuthology.orchestra.run.smithi047.stdout:(9/23): python3-cephfs-16.2.14-550.geb66ed92.el 1.3 MB/s | 215 kB 00:00 2024-02-23T00:29:14.476 INFO:teuthology.orchestra.run.smithi047.stdout:(10/23): python3-rados-16.2.14-550.geb66ed92.el 5.7 MB/s | 389 kB 00:00 2024-02-23T00:29:14.478 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-02-23T00:29:14.478 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-02-23T00:29:14.552 INFO:teuthology.orchestra.run.smithi047.stdout:(11/23): python3-rgw-16.2.14-550.geb66ed92.el8. 1.5 MB/s | 114 kB 00:00 2024-02-23T00:29:14.586 INFO:teuthology.orchestra.run.smithi047.stdout:(12/23): python3-rbd-16.2.14-550.geb66ed92.el8. 2.4 MB/s | 369 kB 00:00 2024-02-23T00:29:14.860 INFO:teuthology.orchestra.run.smithi047.stdout:(13/23): ceph-common-16.2.14-550.geb66ed92.el8. 9.7 MB/s | 24 MB 00:02 2024-02-23T00:29:14.944 INFO:teuthology.orchestra.run.smithi047.stdout:(14/23): librdkafka-1.6.1-1.el8.x86_64.rpm 1.7 MB/s | 677 kB 00:00 2024-02-23T00:29:14.969 INFO:teuthology.orchestra.run.smithi047.stdout:(15/23): python3-prettytable-0.7.2-14.el8.noarc 406 kB/s | 44 kB 00:00 2024-02-23T00:29:15.011 INFO:teuthology.orchestra.run.smithi047.stdout:(16/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 609 kB/s | 259 kB 00:00 2024-02-23T00:29:15.161 INFO:teuthology.orchestra.run.smithi047.stdout:(17/23): librabbitmq-0.9.0-5.el8.x86_64.rpm 216 kB/s | 47 kB 00:00 2024-02-23T00:29:15.220 INFO:teuthology.orchestra.run.smithi047.stdout:(18/23): leveldb-1.22-1.el8.x86_64.rpm 868 kB/s | 181 kB 00:00 2024-02-23T00:29:15.245 INFO:teuthology.orchestra.run.smithi047.stdout:(19/23): gperftools-libs-2.7-9.el8.x86_64.rpm 1.1 MB/s | 306 kB 00:00 2024-02-23T00:29:15.279 INFO:teuthology.orchestra.run.smithi047.stdout:(20/23): libunwind-1.3.1-3.el8.x86_64.rpm 1.2 MB/s | 75 kB 00:00 2024-02-23T00:29:15.321 INFO:teuthology.orchestra.run.smithi047.stdout:(21/23): liboath-2.6.2-3.el8.x86_64.rpm 369 kB/s | 59 kB 00:00 2024-02-23T00:29:15.561 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-02-23T00:29:15.621 INFO:teuthology.orchestra.run.smithi047.stdout:(22/23): librbd1-16.2.14-550.geb66ed92.el8.x86_ 12 MB/s | 4.2 MB 00:00 2024-02-23T00:29:15.787 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/25 2024-02-23T00:29:15.821 INFO:teuthology.orchestra.run.smithi047.stdout:(23/23): librados2-16.2.14-550.geb66ed92.el8.x8 6.5 MB/s | 3.7 MB 00:00 2024-02-23T00:29:15.823 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:29:15.823 INFO:teuthology.orchestra.run.smithi047.stdout:Total 15 MB/s | 58 MB 00:03 2024-02-23T00:29:15.867 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-02-23T00:29:15.918 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-02-23T00:29:15.919 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-02-23T00:29:16.004 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-02-23T00:29:16.294 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-02-23T00:29:16.297 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-02-23T00:29:16.391 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-02-23T00:29:16.612 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librados2-2:16.2.14-550.geb66ed92.el8.x86_64 3/25 2024-02-23T00:29:16.712 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:16.2.14-550.geb66ed92.el8.x86_64 3/25 2024-02-23T00:29:16.968 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rados-2:16.2.14-550.geb66ed92.el8.x86_64 4/25 2024-02-23T00:29:17.187 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs2-2:16.2.14-550.geb66ed92.el8.x86_64 5/25 2024-02-23T00:29:17.391 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-02-23T00:29:17.584 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/25 2024-02-23T00:29:17.597 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephfs2-2:16.2.14-550.geb66ed92.el8.x86_64 5/25 2024-02-23T00:29:17.793 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-02-23T00:29:17.812 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librbd1-2:16.2.14-550.geb66ed92.el8.x86_64 6/25 2024-02-23T00:29:17.874 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:16.2.14-550.geb66ed92.el8.x86_64 6/25 2024-02-23T00:29:18.104 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 7/25 2024-02-23T00:29:18.207 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-02-23T00:29:18.409 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librados2-2:16.2.14-550.geb66ed92.el8.x86_64 3/25 2024-02-23T00:29:18.506 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-2:16.2.14-550.geb66ed92.el8.x86_64 3/25 2024-02-23T00:29:18.625 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 8/25 2024-02-23T00:29:18.812 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librgw2-2:16.2.14-550.geb66ed92.el8.x86_64 9/25 2024-02-23T00:29:18.872 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librgw2-2:16.2.14-550.geb66ed92.el8.x86_64 9/25 2024-02-23T00:29:18.887 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rados-2:16.2.14-550.geb66ed92.el8.x86_64 4/25 2024-02-23T00:29:19.051 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : leveldb-1.22-1.el8.x86_64 10/25 2024-02-23T00:29:19.051 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs2-2:16.2.14-550.geb66ed92.el8.x86_64 5/25 2024-02-23T00:29:19.434 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-argparse-2:16.2.14-550.geb66ed92.el8. 11/25 2024-02-23T00:29:19.493 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephfs2-2:16.2.14-550.geb66ed92.el8.x86_64 5/25 2024-02-23T00:29:19.595 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cephfs-2:16.2.14-550.geb66ed92.el8.x86_64 12/25 2024-02-23T00:29:19.667 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librbd1-2:16.2.14-550.geb66ed92.el8.x86_64 6/25 2024-02-23T00:29:19.718 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-2:16.2.14-550.geb66ed92.el8.x86_64 6/25 2024-02-23T00:29:19.830 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rgw-2:16.2.14-550.geb66ed92.el8.x86_64 13/25 2024-02-23T00:29:19.947 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 7/25 2024-02-23T00:29:20.087 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rbd-2:16.2.14-550.geb66ed92.el8.x86_64 14/25 2024-02-23T00:29:20.278 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libradosstriper1-2:16.2.14-550.geb66ed92.el8.x86_6 15/25 2024-02-23T00:29:20.353 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libradosstriper1-2:16.2.14-550.geb66ed92.el8.x86_6 15/25 2024-02-23T00:29:20.484 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 8/25 2024-02-23T00:29:20.614 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 16/25 2024-02-23T00:29:20.626 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librgw2-2:16.2.14-550.geb66ed92.el8.x86_64 9/25 2024-02-23T00:29:20.708 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librgw2-2:16.2.14-550.geb66ed92.el8.x86_64 9/25 2024-02-23T00:29:20.820 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 17/25 2024-02-23T00:29:20.890 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : leveldb-1.22-1.el8.x86_64 10/25 2024-02-23T00:29:21.000 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2024-02-23T00:29:21.082 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-argparse-2:16.2.14-550.geb66ed92.el8. 11/25 2024-02-23T00:29:21.245 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-common-2:16.2.14-550.geb66ed92.el8.x8 19/25 2024-02-23T00:29:21.247 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cephfs-2:16.2.14-550.geb66ed92.el8.x86_64 12/25 2024-02-23T00:29:21.471 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rgw-2:16.2.14-550.geb66ed92.el8.x86_64 13/25 2024-02-23T00:29:21.662 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rbd-2:16.2.14-550.geb66ed92.el8.x86_64 14/25 2024-02-23T00:29:21.809 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libradosstriper1-2:16.2.14-550.geb66ed92.el8.x86_6 15/25 2024-02-23T00:29:21.880 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libradosstriper1-2:16.2.14-550.geb66ed92.el8.x86_6 15/25 2024-02-23T00:29:22.105 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 16/25 2024-02-23T00:29:22.260 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 17/25 2024-02-23T00:29:22.476 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2024-02-23T00:29:22.704 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-common-2:16.2.14-550.geb66ed92.el8.x8 19/25 2024-02-23T00:29:23.805 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:16.2.14-550.geb66ed92.el8.x86_64 20/25 2024-02-23T00:29:24.070 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-common-2:16.2.14-550.geb66ed92.el8.x86_64 20/25 2024-02-23T00:29:24.790 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:16.2.14-550.geb66ed92.el8.x86_64 20/25 2024-02-23T00:29:25.091 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-base-2:16.2.14-550.geb66ed92.el8.x86_64 21/25 2024-02-23T00:29:25.203 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-base-2:16.2.14-550.geb66ed92.el8.x86_64 21/25 2024-02-23T00:29:25.386 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-selinux-2:16.2.14-550.geb66ed92.el8.x86_64 22/25 2024-02-23T00:29:25.414 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:16.2.14-550.geb66ed92.el8.x86_64 20/25 2024-02-23T00:29:25.634 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-common-2:16.2.14-550.geb66ed92.el8.x86_64 20/25 2024-02-23T00:29:26.375 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:16.2.14-550.geb66ed92.el8.x86_64 20/25 2024-02-23T00:29:26.690 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-base-2:16.2.14-550.geb66ed92.el8.x86_64 21/25 2024-02-23T00:29:26.789 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-base-2:16.2.14-550.geb66ed92.el8.x86_64 21/25 2024-02-23T00:29:26.950 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-selinux-2:16.2.14-550.geb66ed92.el8.x86_64 22/25 2024-02-23T00:29:44.528 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-selinux-2:16.2.14-550.geb66ed92.el8.x86_64 22/25 2024-02-23T00:29:44.528 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /sys 2024-02-23T00:29:44.528 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /proc 2024-02-23T00:29:44.528 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /dev 2024-02-23T00:29:44.528 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /run 2024-02-23T00:29:44.529 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /mnt 2024-02-23T00:29:44.529 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /var/tmp 2024-02-23T00:29:44.529 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /home 2024-02-23T00:29:44.529 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /tmp 2024-02-23T00:29:44.529 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /dev 2024-02-23T00:29:44.529 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:29:45.762 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-radosgw-2:16.2.14-550.geb66ed92.el8.x86_64 23/25 2024-02-23T00:29:45.833 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-radosgw-2:16.2.14-550.geb66ed92.el8.x86_64 23/25 2024-02-23T00:29:45.834 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-02-23T00:29:46.076 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-02-23T00:29:46.076 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-02-23T00:29:46.785 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-02-23T00:29:46.785 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-base-2:16.2.14-550.geb66ed92.el8.x86_64 1/25 2024-02-23T00:29:46.785 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-common-2:16.2.14-550.geb66ed92.el8.x86_64 2/25 2024-02-23T00:29:46.785 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-radosgw-2:16.2.14-550.geb66ed92.el8.x86_64 3/25 2024-02-23T00:29:46.785 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-selinux-2:16.2.14-550.geb66ed92.el8.x86_64 4/25 2024-02-23T00:29:46.785 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs2-2:16.2.14-550.geb66ed92.el8.x86_64 5/25 2024-02-23T00:29:46.785 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libradosstriper1-2:16.2.14-550.geb66ed92.el8.x86_6 6/25 2024-02-23T00:29:46.785 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librgw2-2:16.2.14-550.geb66ed92.el8.x86_64 7/25 2024-02-23T00:29:46.785 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-argparse-2:16.2.14-550.geb66ed92.el8. 8/25 2024-02-23T00:29:46.785 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-common-2:16.2.14-550.geb66ed92.el8.x8 9/25 2024-02-23T00:29:46.786 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cephfs-2:16.2.14-550.geb66ed92.el8.x86_64 10/25 2024-02-23T00:29:46.786 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rados-2:16.2.14-550.geb66ed92.el8.x86_64 11/25 2024-02-23T00:29:46.786 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rbd-2:16.2.14-550.geb66ed92.el8.x86_64 12/25 2024-02-23T00:29:46.786 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rgw-2:16.2.14-550.geb66ed92.el8.x86_64 13/25 2024-02-23T00:29:46.786 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 14/25 2024-02-23T00:29:46.786 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2024-02-23T00:29:46.786 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2024-02-23T00:29:46.786 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 17/25 2024-02-23T00:29:46.786 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2024-02-23T00:29:46.786 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : leveldb-1.22-1.el8.x86_64 19/25 2024-02-23T00:29:46.786 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 20/25 2024-02-23T00:29:46.786 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 21/25 2024-02-23T00:29:46.786 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:16.2.14-550.geb66ed92.el8.x86_64 22/25 2024-02-23T00:29:46.787 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-02-23T00:29:46.787 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:16.2.14-550.geb66ed92.el8.x86_64 24/25 2024-02-23T00:29:47.070 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-selinux-2:16.2.14-550.geb66ed92.el8.x86_64 22/25 2024-02-23T00:29:47.070 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /sys 2024-02-23T00:29:47.070 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /proc 2024-02-23T00:29:47.070 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /dev 2024-02-23T00:29:47.070 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /run 2024-02-23T00:29:47.070 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /mnt 2024-02-23T00:29:47.070 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /var/tmp 2024-02-23T00:29:47.071 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /home 2024-02-23T00:29:47.071 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /tmp 2024-02-23T00:29:47.071 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /dev 2024-02-23T00:29:47.071 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:29:47.652 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-02-23T00:29:47.653 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:29:47.653 INFO:teuthology.orchestra.run.smithi196.stdout:Upgraded: 2024-02-23T00:29:47.653 INFO:teuthology.orchestra.run.smithi196.stdout: librados2-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:29:47.653 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:29:47.653 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-02-23T00:29:47.653 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:29:47.653 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:29:47.653 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:29:47.653 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:29:47.653 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-02-23T00:29:47.653 INFO:teuthology.orchestra.run.smithi196.stdout: leveldb-1.22-1.el8.x86_64 2024-02-23T00:29:47.653 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:29:47.654 INFO:teuthology.orchestra.run.smithi196.stdout: liboath-2.6.2-3.el8.x86_64 2024-02-23T00:29:47.654 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-02-23T00:29:47.654 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:29:47.654 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-02-23T00:29:47.654 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:29:47.654 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind-1.3.1-3.el8.x86_64 2024-02-23T00:29:47.654 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-02-23T00:29:47.654 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:29:47.654 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:29:47.654 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:29:47.654 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-02-23T00:29:47.654 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:29:47.654 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:29:47.654 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:29:47.655 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:29:47.655 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-02-23T00:29:47.981 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-test 2024-02-23T00:29:48.312 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-radosgw-2:16.2.14-550.geb66ed92.el8.x86_64 23/25 2024-02-23T00:29:48.418 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-radosgw-2:16.2.14-550.geb66ed92.el8.x86_64 23/25 2024-02-23T00:29:48.418 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-02-23T00:29:48.625 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-02-23T00:29:48.625 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-02-23T00:29:48.737 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:00:44 ago on Fri 23 Feb 2024 12:29:04 AM UTC. 2024-02-23T00:29:49.324 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-02-23T00:29:49.324 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-base-2:16.2.14-550.geb66ed92.el8.x86_64 1/25 2024-02-23T00:29:49.324 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-common-2:16.2.14-550.geb66ed92.el8.x86_64 2/25 2024-02-23T00:29:49.324 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-radosgw-2:16.2.14-550.geb66ed92.el8.x86_64 3/25 2024-02-23T00:29:49.324 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-selinux-2:16.2.14-550.geb66ed92.el8.x86_64 4/25 2024-02-23T00:29:49.324 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs2-2:16.2.14-550.geb66ed92.el8.x86_64 5/25 2024-02-23T00:29:49.324 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libradosstriper1-2:16.2.14-550.geb66ed92.el8.x86_6 6/25 2024-02-23T00:29:49.324 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librgw2-2:16.2.14-550.geb66ed92.el8.x86_64 7/25 2024-02-23T00:29:49.324 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-argparse-2:16.2.14-550.geb66ed92.el8. 8/25 2024-02-23T00:29:49.324 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-common-2:16.2.14-550.geb66ed92.el8.x8 9/25 2024-02-23T00:29:49.324 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cephfs-2:16.2.14-550.geb66ed92.el8.x86_64 10/25 2024-02-23T00:29:49.325 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rados-2:16.2.14-550.geb66ed92.el8.x86_64 11/25 2024-02-23T00:29:49.325 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rbd-2:16.2.14-550.geb66ed92.el8.x86_64 12/25 2024-02-23T00:29:49.325 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rgw-2:16.2.14-550.geb66ed92.el8.x86_64 13/25 2024-02-23T00:29:49.325 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 14/25 2024-02-23T00:29:49.325 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2024-02-23T00:29:49.325 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2024-02-23T00:29:49.325 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 17/25 2024-02-23T00:29:49.325 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2024-02-23T00:29:49.325 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : leveldb-1.22-1.el8.x86_64 19/25 2024-02-23T00:29:49.325 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 20/25 2024-02-23T00:29:49.325 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 21/25 2024-02-23T00:29:49.325 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-2:16.2.14-550.geb66ed92.el8.x86_64 22/25 2024-02-23T00:29:49.325 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-02-23T00:29:49.325 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-2:16.2.14-550.geb66ed92.el8.x86_64 24/25 2024-02-23T00:29:50.209 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-02-23T00:29:50.209 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:29:50.209 INFO:teuthology.orchestra.run.smithi047.stdout:Upgraded: 2024-02-23T00:29:50.209 INFO:teuthology.orchestra.run.smithi047.stdout: librados2-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:29:50.209 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:29:50.209 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-02-23T00:29:50.210 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:29:50.210 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:29:50.210 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:29:50.210 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:29:50.210 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-02-23T00:29:50.210 INFO:teuthology.orchestra.run.smithi047.stdout: leveldb-1.22-1.el8.x86_64 2024-02-23T00:29:50.210 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:29:50.210 INFO:teuthology.orchestra.run.smithi047.stdout: liboath-2.6.2-3.el8.x86_64 2024-02-23T00:29:50.210 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-02-23T00:29:50.210 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:29:50.210 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-02-23T00:29:50.210 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:29:50.211 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind-1.3.1-3.el8.x86_64 2024-02-23T00:29:50.211 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-02-23T00:29:50.211 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:29:50.211 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:29:50.211 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:29:50.211 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-02-23T00:29:50.211 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:29:50.211 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:29:50.211 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:29:50.211 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:29:50.212 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-02-23T00:29:50.311 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-02-23T00:29:50.312 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:29:50.313 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-02-23T00:29:50.313 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:29:50.313 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-02-23T00:29:50.313 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test x86_64 2:16.2.14-550.geb66ed92.el8 ceph 52 M 2024-02-23T00:29:50.313 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-02-23T00:29:50.313 INFO:teuthology.orchestra.run.smithi196.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-02-23T00:29:50.313 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite x86_64 2:16.2.14-550.geb66ed92.el8 ceph 174 k 2024-02-23T00:29:50.313 INFO:teuthology.orchestra.run.smithi196.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-02-23T00:29:50.313 INFO:teuthology.orchestra.run.smithi196.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-02-23T00:29:50.313 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-02-23T00:29:50.313 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:29:50.313 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-02-23T00:29:50.313 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:29:50.314 INFO:teuthology.orchestra.run.smithi196.stdout:Install 6 Packages 2024-02-23T00:29:50.314 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:29:50.314 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 53 M 2024-02-23T00:29:50.314 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 197 M 2024-02-23T00:29:50.314 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-02-23T00:29:50.567 INFO:teuthology.orchestra.run.smithi196.stdout:(1/6): libcephsqlite-16.2.14-550.geb66ed92.el8. 910 kB/s | 174 kB 00:00 2024-02-23T00:29:50.578 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-test 2024-02-23T00:29:50.825 INFO:teuthology.orchestra.run.smithi196.stdout:(2/6): jq-1.6-8.el8.x86_64.rpm 452 kB/s | 203 kB 00:00 2024-02-23T00:29:50.934 INFO:teuthology.orchestra.run.smithi196.stdout:(3/6): oniguruma-6.8.2-3.el8.x86_64.rpm 513 kB/s | 188 kB 00:00 2024-02-23T00:29:50.968 INFO:teuthology.orchestra.run.smithi196.stdout:(4/6): socat-1.7.4.1-1.el8.x86_64.rpm 2.2 MB/s | 323 kB 00:00 2024-02-23T00:29:51.127 INFO:teuthology.orchestra.run.smithi196.stdout:(5/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 368 kB/s | 71 kB 00:00 2024-02-23T00:29:51.366 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:00:43 ago on Fri 23 Feb 2024 12:29:08 AM UTC. 2024-02-23T00:29:53.056 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-02-23T00:29:53.057 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:29:53.057 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-02-23T00:29:53.057 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:29:53.057 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-02-23T00:29:53.057 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test x86_64 2:16.2.14-550.geb66ed92.el8 ceph 52 M 2024-02-23T00:29:53.057 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-02-23T00:29:53.057 INFO:teuthology.orchestra.run.smithi047.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-02-23T00:29:53.057 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite x86_64 2:16.2.14-550.geb66ed92.el8 ceph 174 k 2024-02-23T00:29:53.057 INFO:teuthology.orchestra.run.smithi047.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-02-23T00:29:53.057 INFO:teuthology.orchestra.run.smithi047.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-02-23T00:29:53.057 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-02-23T00:29:53.058 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:29:53.058 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-02-23T00:29:53.058 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:29:53.058 INFO:teuthology.orchestra.run.smithi047.stdout:Install 6 Packages 2024-02-23T00:29:53.058 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:29:53.058 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 53 M 2024-02-23T00:29:53.058 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 197 M 2024-02-23T00:29:53.059 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-02-23T00:29:53.487 INFO:teuthology.orchestra.run.smithi047.stdout:(1/6): jq-1.6-8.el8.x86_64.rpm 553 kB/s | 203 kB 00:00 2024-02-23T00:29:53.509 INFO:teuthology.orchestra.run.smithi196.stdout:(6/6): ceph-test-16.2.14-550.geb66ed92.el8.x86_ 17 MB/s | 52 MB 00:03 2024-02-23T00:29:53.511 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:29:53.512 INFO:teuthology.orchestra.run.smithi047.stdout:(2/6): libcephsqlite-16.2.14-550.geb66ed92.el8. 443 kB/s | 174 kB 00:00 2024-02-23T00:29:53.512 INFO:teuthology.orchestra.run.smithi196.stdout:Total 17 MB/s | 53 MB 00:03 2024-02-23T00:29:53.555 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-02-23T00:29:53.576 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-02-23T00:29:53.576 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-02-23T00:29:53.621 INFO:teuthology.orchestra.run.smithi047.stdout:(3/6): oniguruma-6.8.2-3.el8.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-02-23T00:29:53.696 INFO:teuthology.orchestra.run.smithi047.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 945 kB/s | 71 kB 00:00 2024-02-23T00:29:53.838 INFO:teuthology.orchestra.run.smithi047.stdout:(5/6): socat-1.7.4.1-1.el8.x86_64.rpm 991 kB/s | 323 kB 00:00 2024-02-23T00:29:53.889 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-02-23T00:29:53.892 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-02-23T00:29:54.585 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-02-23T00:29:54.834 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2024-02-23T00:29:55.091 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2024-02-23T00:29:55.255 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 3/6 2024-02-23T00:29:55.339 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 3/6 2024-02-23T00:29:55.569 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : jq-1.6-8.el8.x86_64 4/6 2024-02-23T00:29:55.680 INFO:teuthology.orchestra.run.smithi047.stdout:(6/6): ceph-test-16.2.14-550.geb66ed92.el8.x86_ 20 MB/s | 52 MB 00:02 2024-02-23T00:29:55.682 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:29:55.682 INFO:teuthology.orchestra.run.smithi047.stdout:Total 20 MB/s | 53 MB 00:02 2024-02-23T00:29:55.726 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-02-23T00:29:55.749 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-02-23T00:29:55.749 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-02-23T00:29:55.780 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephsqlite-2:16.2.14-550.geb66ed92.el8.x86_64 5/6 2024-02-23T00:29:56.078 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-02-23T00:29:56.081 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-02-23T00:29:56.583 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-02-23T00:29:56.817 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2024-02-23T00:29:57.033 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2024-02-23T00:29:57.154 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 3/6 2024-02-23T00:29:57.235 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 3/6 2024-02-23T00:29:57.445 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : jq-1.6-8.el8.x86_64 4/6 2024-02-23T00:29:57.646 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephsqlite-2:16.2.14-550.geb66ed92.el8.x86_64 5/6 2024-02-23T00:30:00.384 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephsqlite-2:16.2.14-550.geb66ed92.el8.x86_64 5/6 2024-02-23T00:30:00.686 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-test-2:16.2.14-550.geb66ed92.el8.x86_64 6/6 2024-02-23T00:30:01.001 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-test-2:16.2.14-550.geb66ed92.el8.x86_64 6/6 2024-02-23T00:30:01.001 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-test-2:16.2.14-550.geb66ed92.el8.x86_64 1/6 2024-02-23T00:30:01.001 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephsqlite-2:16.2.14-550.geb66ed92.el8.x86_64 2/6 2024-02-23T00:30:01.001 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : jq-1.6-8.el8.x86_64 3/6 2024-02-23T00:30:01.001 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 4/6 2024-02-23T00:30:01.001 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2024-02-23T00:30:01.482 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2024-02-23T00:30:01.482 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:30:01.482 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-02-23T00:30:01.483 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:30:01.483 INFO:teuthology.orchestra.run.smithi196.stdout: jq-1.6-8.el8.x86_64 2024-02-23T00:30:01.483 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:30:01.483 INFO:teuthology.orchestra.run.smithi196.stdout: oniguruma-6.8.2-3.el8.x86_64 2024-02-23T00:30:01.483 INFO:teuthology.orchestra.run.smithi196.stdout: socat-1.7.4.1-1.el8.x86_64 2024-02-23T00:30:01.483 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-02-23T00:30:01.483 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:30:01.483 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-02-23T00:30:01.753 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph 2024-02-23T00:30:02.514 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:00:58 ago on Fri 23 Feb 2024 12:29:04 AM UTC. 2024-02-23T00:30:02.535 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephsqlite-2:16.2.14-550.geb66ed92.el8.x86_64 5/6 2024-02-23T00:30:02.886 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-test-2:16.2.14-550.geb66ed92.el8.x86_64 6/6 2024-02-23T00:30:03.187 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-test-2:16.2.14-550.geb66ed92.el8.x86_64 6/6 2024-02-23T00:30:03.187 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-test-2:16.2.14-550.geb66ed92.el8.x86_64 1/6 2024-02-23T00:30:03.187 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephsqlite-2:16.2.14-550.geb66ed92.el8.x86_64 2/6 2024-02-23T00:30:03.187 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : jq-1.6-8.el8.x86_64 3/6 2024-02-23T00:30:03.187 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 4/6 2024-02-23T00:30:03.187 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2024-02-23T00:30:03.728 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2024-02-23T00:30:03.728 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:30:03.728 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-02-23T00:30:03.728 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:30:03.728 INFO:teuthology.orchestra.run.smithi047.stdout: jq-1.6-8.el8.x86_64 2024-02-23T00:30:03.728 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:30:03.728 INFO:teuthology.orchestra.run.smithi047.stdout: oniguruma-6.8.2-3.el8.x86_64 2024-02-23T00:30:03.728 INFO:teuthology.orchestra.run.smithi047.stdout: socat-1.7.4.1-1.el8.x86_64 2024-02-23T00:30:03.729 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-02-23T00:30:03.729 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:30:03.729 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-02-23T00:30:04.028 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph 2024-02-23T00:30:04.079 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-02-23T00:30:04.082 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:30:04.082 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-02-23T00:30:04.082 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:30:04.083 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-02-23T00:30:04.083 INFO:teuthology.orchestra.run.smithi196.stdout: ceph x86_64 2:16.2.14-550.geb66ed92.el8 ceph 6.4 k 2024-02-23T00:30:04.083 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-02-23T00:30:04.083 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds x86_64 2:16.2.14-550.geb66ed92.el8 ceph 2.4 M 2024-02-23T00:30:04.083 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr x86_64 2:16.2.14-550.geb66ed92.el8 ceph 1.6 M 2024-02-23T00:30:04.083 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core noarch 2:16.2.14-550.geb66ed92.el8 ceph-noarch 245 k 2024-02-23T00:30:04.083 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon x86_64 2:16.2.14-550.geb66ed92.el8 ceph 5.2 M 2024-02-23T00:30:04.083 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd x86_64 2:16.2.14-550.geb66ed92.el8 ceph 19 M 2024-02-23T00:30:04.083 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-02-23T00:30:04.084 INFO:teuthology.orchestra.run.smithi196.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-02-23T00:30:04.084 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-02-23T00:30:04.084 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-02-23T00:30:04.084 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-02-23T00:30:04.084 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-02-23T00:30:04.084 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-02-23T00:30:04.084 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-02-23T00:30:04.084 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-02-23T00:30:04.084 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-02-23T00:30:04.084 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-02-23T00:30:04.084 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-02-23T00:30:04.084 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-02-23T00:30:04.085 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-02-23T00:30:04.085 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-02-23T00:30:04.085 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-02-23T00:30:04.085 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-02-23T00:30:04.085 INFO:teuthology.orchestra.run.smithi196.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-02-23T00:30:04.085 INFO:teuthology.orchestra.run.smithi196.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-02-23T00:30:04.085 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-02-23T00:30:04.085 INFO:teuthology.orchestra.run.smithi196.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-02-23T00:30:04.085 INFO:teuthology.orchestra.run.smithi196.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-02-23T00:30:04.085 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-02-23T00:30:04.085 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-02-23T00:30:04.085 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-02-23T00:30:04.085 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-02-23T00:30:04.085 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:30:04.086 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-02-23T00:30:04.086 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:30:04.086 INFO:teuthology.orchestra.run.smithi196.stdout:Install 32 Packages 2024-02-23T00:30:04.086 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:30:04.086 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 32 M 2024-02-23T00:30:04.086 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 104 M 2024-02-23T00:30:04.087 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-02-23T00:30:04.812 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:00:56 ago on Fri 23 Feb 2024 12:29:08 AM UTC. 2024-02-23T00:30:06.494 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-02-23T00:30:06.497 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:30:06.497 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-02-23T00:30:06.497 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:30:06.497 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-02-23T00:30:06.498 INFO:teuthology.orchestra.run.smithi047.stdout: ceph x86_64 2:16.2.14-550.geb66ed92.el8 ceph 6.4 k 2024-02-23T00:30:06.498 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-02-23T00:30:06.498 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds x86_64 2:16.2.14-550.geb66ed92.el8 ceph 2.4 M 2024-02-23T00:30:06.498 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr x86_64 2:16.2.14-550.geb66ed92.el8 ceph 1.6 M 2024-02-23T00:30:06.498 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core noarch 2:16.2.14-550.geb66ed92.el8 ceph-noarch 245 k 2024-02-23T00:30:06.498 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon x86_64 2:16.2.14-550.geb66ed92.el8 ceph 5.2 M 2024-02-23T00:30:06.498 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd x86_64 2:16.2.14-550.geb66ed92.el8 ceph 19 M 2024-02-23T00:30:06.498 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-02-23T00:30:06.498 INFO:teuthology.orchestra.run.smithi047.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-02-23T00:30:06.498 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-02-23T00:30:06.498 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-02-23T00:30:06.498 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-02-23T00:30:06.498 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-02-23T00:30:06.498 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-02-23T00:30:06.499 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-02-23T00:30:06.499 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-02-23T00:30:06.499 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-02-23T00:30:06.499 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-02-23T00:30:06.499 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-02-23T00:30:06.499 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-02-23T00:30:06.499 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-02-23T00:30:06.499 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-02-23T00:30:06.499 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-02-23T00:30:06.499 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-02-23T00:30:06.499 INFO:teuthology.orchestra.run.smithi047.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-02-23T00:30:06.499 INFO:teuthology.orchestra.run.smithi047.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-02-23T00:30:06.499 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-02-23T00:30:06.499 INFO:teuthology.orchestra.run.smithi047.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-02-23T00:30:06.500 INFO:teuthology.orchestra.run.smithi047.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-02-23T00:30:06.500 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-02-23T00:30:06.500 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-02-23T00:30:06.500 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-02-23T00:30:06.500 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-02-23T00:30:06.500 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:30:06.500 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-02-23T00:30:06.500 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:30:06.500 INFO:teuthology.orchestra.run.smithi047.stdout:Install 32 Packages 2024-02-23T00:30:06.500 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:30:06.501 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 32 M 2024-02-23T00:30:06.501 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 104 M 2024-02-23T00:30:06.501 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-02-23T00:30:07.837 INFO:teuthology.orchestra.run.smithi196.stdout:(1/32): ceph-16.2.14-550.geb66ed92.el8.x86_64.r 36 kB/s | 6.4 kB 00:00 2024-02-23T00:30:08.021 INFO:teuthology.orchestra.run.smithi196.stdout:(2/32): ceph-mds-16.2.14-550.geb66ed92.el8.x86_ 6.8 MB/s | 2.4 MB 00:00 2024-02-23T00:30:08.394 INFO:teuthology.orchestra.run.smithi196.stdout:(3/32): ceph-mgr-16.2.14-550.geb66ed92.el8.x86_ 2.2 MB/s | 1.6 MB 00:00 2024-02-23T00:30:08.513 INFO:teuthology.orchestra.run.smithi196.stdout:(4/32): ceph-mgr-modules-core-16.2.14-550.geb66 2.0 MB/s | 245 kB 00:00 2024-02-23T00:30:08.846 INFO:teuthology.orchestra.run.smithi196.stdout:(5/32): python3-mako-1.0.6-14.el8.noarch.rpm 471 kB/s | 157 kB 00:00 2024-02-23T00:30:09.080 INFO:teuthology.orchestra.run.smithi196.stdout:(6/32): ceph-osd-16.2.14-550.geb66ed92.el8.x86_ 18 MB/s | 19 MB 00:01 2024-02-23T00:30:09.105 INFO:teuthology.orchestra.run.smithi196.stdout:(7/32): python3-markupsafe-0.23-19.el8.x86_64.r 150 kB/s | 39 kB 00:00 2024-02-23T00:30:09.181 INFO:teuthology.orchestra.run.smithi196.stdout:(8/32): ceph-mon-16.2.14-550.geb66ed92.el8.x86_ 3.9 MB/s | 5.2 MB 00:01 2024-02-23T00:30:09.206 INFO:teuthology.orchestra.run.smithi196.stdout:(9/32): python3-pytz-2017.2-11.el8.noarch.rpm 569 kB/s | 57 kB 00:00 2024-02-23T00:30:09.398 INFO:teuthology.orchestra.run.smithi196.stdout:(10/32): python3-pyOpenSSL-19.0.0-1.el8.noarch. 326 kB/s | 103 kB 00:00 2024-02-23T00:30:09.431 INFO:teuthology.orchestra.run.smithi196.stdout:(11/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.1 MB/s | 249 kB 00:00 2024-02-23T00:30:09.556 INFO:teuthology.orchestra.run.smithi196.stdout:(12/32): python3-werkzeug-0.12.2-4.el8.noarch.r 1.2 MB/s | 457 kB 00:00 2024-02-23T00:30:09.690 INFO:teuthology.orchestra.run.smithi196.stdout:(13/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 327 kB/s | 44 kB 00:00 2024-02-23T00:30:09.715 INFO:teuthology.orchestra.run.smithi196.stdout:(14/32): python3-pycparser-2.14-14.el8.noarch.r 384 kB/s | 109 kB 00:00 2024-02-23T00:30:09.774 INFO:teuthology.orchestra.run.smithi196.stdout:(15/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 2.2 MB/s | 185 kB 00:00 2024-02-23T00:30:09.841 INFO:teuthology.orchestra.run.smithi196.stdout:(16/32): python3-cherrypy-18.4.0-1.el8.noarch.r 5.7 MB/s | 384 kB 00:00 2024-02-23T00:30:09.916 INFO:teuthology.orchestra.run.smithi196.stdout:(17/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 861 kB/s | 173 kB 00:00 2024-02-23T00:30:09.942 INFO:teuthology.orchestra.run.smithi196.stdout:(18/32): python3-jaraco-6.2-6.el8.noarch.rpm 110 kB/s | 11 kB 00:00 2024-02-23T00:30:09.967 INFO:teuthology.orchestra.run.smithi196.stdout:(19/32): python3-jaraco-functools-2.0-4.el8.noa 348 kB/s | 18 kB 00:00 2024-02-23T00:30:09.992 INFO:teuthology.orchestra.run.smithi196.stdout:(20/32): python3-logutils-0.3.5-11.el8.noarch.r 968 kB/s | 49 kB 00:00 2024-02-23T00:30:10.017 INFO:teuthology.orchestra.run.smithi196.stdout:(21/32): python3-cryptography-3.2.1-7.el8.x86_6 1.0 MB/s | 641 kB 00:00 2024-02-23T00:30:10.043 INFO:teuthology.orchestra.run.smithi196.stdout:(22/32): python3-more-itertools-7.2.0-3.el8.noa 778 kB/s | 59 kB 00:00 2024-02-23T00:30:10.076 INFO:teuthology.orchestra.run.smithi196.stdout:(23/32): python3-pecan-1.3.2-9.el8.noarch.rpm 3.3 MB/s | 283 kB 00:00 2024-02-23T00:30:10.093 INFO:teuthology.orchestra.run.smithi196.stdout:(24/32): python3-simplegeneric-0.8.1-17.el8.noa 413 kB/s | 19 kB 00:00 2024-02-23T00:30:10.119 INFO:teuthology.orchestra.run.smithi196.stdout:(25/32): python3-singledispatch-3.4.0.3-18.el8. 577 kB/s | 24 kB 00:00 2024-02-23T00:30:10.162 INFO:teuthology.orchestra.run.smithi196.stdout:(26/32): python3-portend-2.6-1.el8.noarch.rpm 108 kB/s | 16 kB 00:00 2024-02-23T00:30:10.186 INFO:teuthology.orchestra.run.smithi196.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 317 kB/s | 29 kB 00:00 2024-02-23T00:30:10.203 INFO:teuthology.orchestra.run.smithi196.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 329 kB/s | 27 kB 00:00 2024-02-23T00:30:10.245 INFO:teuthology.orchestra.run.smithi196.stdout:(29/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 4.2 MB/s | 251 kB 00:00 2024-02-23T00:30:10.270 INFO:teuthology.orchestra.run.smithi196.stdout:(30/32): python3-waitress-1.4.3-1.el8.noarch.rp 2.3 MB/s | 255 kB 00:00 2024-02-23T00:30:10.287 INFO:teuthology.orchestra.run.smithi196.stdout:(31/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.0 MB/s | 85 kB 00:00 2024-02-23T00:30:10.304 INFO:teuthology.orchestra.run.smithi196.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 391 kB/s | 23 kB 00:00 2024-02-23T00:30:10.308 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:30:10.308 INFO:teuthology.orchestra.run.smithi196.stdout:Total 5.1 MB/s | 32 MB 00:06 2024-02-23T00:30:10.390 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-02-23T00:30:10.429 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-02-23T00:30:10.429 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-02-23T00:30:10.775 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-02-23T00:30:10.778 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-02-23T00:30:11.970 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-02-23T00:30:12.158 INFO:teuthology.orchestra.run.smithi047.stdout:(1/32): ceph-16.2.14-550.geb66ed92.el8.x86_64.r 22 kB/s | 6.4 kB 00:00 2024-02-23T00:30:12.224 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-02-23T00:30:12.484 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-02-23T00:30:12.736 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2024-02-23T00:30:12.783 INFO:teuthology.orchestra.run.smithi047.stdout:(2/32): ceph-mds-16.2.14-550.geb66ed92.el8.x86_ 2.7 MB/s | 2.4 MB 00:00 2024-02-23T00:30:12.833 INFO:teuthology.orchestra.run.smithi047.stdout:(3/32): ceph-mgr-16.2.14-550.geb66ed92.el8.x86_ 1.7 MB/s | 1.6 MB 00:00 2024-02-23T00:30:12.966 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2024-02-23T00:30:12.975 INFO:teuthology.orchestra.run.smithi047.stdout:(4/32): ceph-mgr-modules-core-16.2.14-550.geb66 1.7 MB/s | 245 kB 00:00 2024-02-23T00:30:13.206 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2024-02-23T00:30:13.309 INFO:teuthology.orchestra.run.smithi047.stdout:(5/32): ceph-mon-16.2.14-550.geb66ed92.el8.x86_ 4.5 MB/s | 5.2 MB 00:01 2024-02-23T00:30:13.334 INFO:teuthology.orchestra.run.smithi047.stdout:(6/32): python3-mako-1.0.6-14.el8.noarch.rpm 438 kB/s | 157 kB 00:00 2024-02-23T00:30:13.401 INFO:teuthology.orchestra.run.smithi047.stdout:(7/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 1.5 MB/s | 103 kB 00:00 2024-02-23T00:30:13.436 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2024-02-23T00:30:13.468 INFO:teuthology.orchestra.run.smithi047.stdout:(8/32): python3-pytz-2017.2-11.el8.noarch.rpm 857 kB/s | 57 kB 00:00 2024-02-23T00:30:13.493 INFO:teuthology.orchestra.run.smithi047.stdout:(9/32): python3-markupsafe-0.23-19.el8.x86_64.r 211 kB/s | 39 kB 00:00 2024-02-23T00:30:13.619 INFO:teuthology.orchestra.run.smithi047.stdout:(10/32): python3-werkzeug-0.12.2-4.el8.noarch.r 3.0 MB/s | 457 kB 00:00 2024-02-23T00:30:13.646 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2024-02-23T00:30:13.711 INFO:teuthology.orchestra.run.smithi047.stdout:(11/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.1 MB/s | 249 kB 00:00 2024-02-23T00:30:13.874 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2024-02-23T00:30:13.923 INFO:teuthology.orchestra.run.smithi047.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 515 kB/s | 109 kB 00:00 2024-02-23T00:30:14.083 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2024-02-23T00:30:14.145 INFO:teuthology.orchestra.run.smithi047.stdout:(13/32): python3-cryptography-3.2.1-7.el8.x86_6 1.2 MB/s | 641 kB 00:00 2024-02-23T00:30:14.178 INFO:teuthology.orchestra.run.smithi047.stdout:(14/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 171 kB/s | 44 kB 00:00 2024-02-23T00:30:14.337 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2024-02-23T00:30:14.362 INFO:teuthology.orchestra.run.smithi047.stdout:(15/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 941 kB/s | 173 kB 00:00 2024-02-23T00:30:14.487 INFO:teuthology.orchestra.run.smithi047.stdout:(16/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 541 kB/s | 185 kB 00:00 2024-02-23T00:30:14.547 INFO:teuthology.orchestra.run.smithi047.stdout:(17/32): python3-cherrypy-18.4.0-1.el8.noarch.r 2.0 MB/s | 384 kB 00:00 2024-02-23T00:30:14.580 INFO:teuthology.orchestra.run.smithi047.stdout:(18/32): python3-jaraco-6.2-6.el8.noarch.rpm 121 kB/s | 11 kB 00:00 2024-02-23T00:30:14.622 INFO:teuthology.orchestra.run.smithi047.stdout:(19/32): python3-jaraco-functools-2.0-4.el8.noa 238 kB/s | 18 kB 00:00 2024-02-23T00:30:14.634 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2024-02-23T00:30:14.672 INFO:teuthology.orchestra.run.smithi047.stdout:(20/32): python3-logutils-0.3.5-11.el8.noarch.r 530 kB/s | 49 kB 00:00 2024-02-23T00:30:14.714 INFO:teuthology.orchestra.run.smithi047.stdout:(21/32): python3-more-itertools-7.2.0-3.el8.noa 638 kB/s | 59 kB 00:00 2024-02-23T00:30:14.789 INFO:teuthology.orchestra.run.smithi047.stdout:(22/32): python3-portend-2.6-1.el8.noarch.rpm 208 kB/s | 16 kB 00:00 2024-02-23T00:30:14.814 INFO:teuthology.orchestra.run.smithi047.stdout:(23/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.9 MB/s | 283 kB 00:00 2024-02-23T00:30:14.881 INFO:teuthology.orchestra.run.smithi047.stdout:(24/32): python3-simplegeneric-0.8.1-17.el8.noa 203 kB/s | 19 kB 00:00 2024-02-23T00:30:14.907 INFO:teuthology.orchestra.run.smithi047.stdout:(25/32): python3-singledispatch-3.4.0.3-18.el8. 261 kB/s | 24 kB 00:00 2024-02-23T00:30:14.908 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2024-02-23T00:30:14.957 INFO:teuthology.orchestra.run.smithi047.stdout:(26/32): python3-tempora-1.14.1-5.el8.noarch.rp 389 kB/s | 29 kB 00:00 2024-02-23T00:30:14.982 INFO:teuthology.orchestra.run.smithi047.stdout:(27/32): python3-trustme-0.6.0-4.el8.noarch.rpm 365 kB/s | 27 kB 00:00 2024-02-23T00:30:15.057 INFO:teuthology.orchestra.run.smithi047.stdout:(28/32): python3-waitress-1.4.3-1.el8.noarch.rp 2.5 MB/s | 255 kB 00:00 2024-02-23T00:30:15.108 INFO:teuthology.orchestra.run.smithi047.stdout:(29/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.0 MB/s | 251 kB 00:00 2024-02-23T00:30:15.189 INFO:teuthology.orchestra.run.smithi047.stdout:(30/32): python3-webtest-2.0.33-1.el8.noarch.rp 652 kB/s | 85 kB 00:00 2024-02-23T00:30:15.217 INFO:teuthology.orchestra.run.smithi047.stdout:(31/32): python3-zc-lockfile-2.0-2.el8.noarch.r 211 kB/s | 23 kB 00:00 2024-02-23T00:30:15.346 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2024-02-23T00:30:15.639 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 14/32 2024-02-23T00:30:15.884 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2024-02-23T00:30:16.185 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2024-02-23T00:30:16.369 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-02-23T00:30:16.707 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2024-02-23T00:30:16.982 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2024-02-23T00:30:17.171 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2024-02-23T00:30:17.317 INFO:teuthology.orchestra.run.smithi047.stdout:(32/32): ceph-osd-16.2.14-550.geb66ed92.el8.x86 4.1 MB/s | 19 MB 00:04 2024-02-23T00:30:17.321 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:30:17.321 INFO:teuthology.orchestra.run.smithi047.stdout:Total 2.9 MB/s | 32 MB 00:10 2024-02-23T00:30:17.390 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-02-23T00:30:17.406 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-02-23T00:30:17.448 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-02-23T00:30:17.448 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-02-23T00:30:17.670 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-02-23T00:30:17.855 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-02-23T00:30:17.858 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-02-23T00:30:17.897 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-02-23T00:30:18.164 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2024-02-23T00:30:18.508 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2024-02-23T00:30:18.846 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-02-23T00:30:19.171 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-02-23T00:30:19.233 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-modules-core-2:16.2.14-550.geb66ed92.el8. 27/32 2024-02-23T00:30:19.428 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-02-23T00:30:19.430 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-2:16.2.14-550.geb66ed92.el8.x86_64 28/32 2024-02-23T00:30:19.676 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-02-23T00:30:19.958 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2024-02-23T00:30:20.183 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2024-02-23T00:30:20.415 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2024-02-23T00:30:20.639 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2024-02-23T00:30:20.822 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2024-02-23T00:30:20.946 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-2:16.2.14-550.geb66ed92.el8.x86_64 28/32 2024-02-23T00:30:21.008 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2024-02-23T00:30:21.134 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-osd-2:16.2.14-550.geb66ed92.el8.x86_64 29/32 2024-02-23T00:30:21.277 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2024-02-23T00:30:21.506 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2024-02-23T00:30:21.578 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-osd-2:16.2.14-550.geb66ed92.el8.x86_64 29/32 2024-02-23T00:30:21.801 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2024-02-23T00:30:21.809 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mon-2:16.2.14-550.geb66ed92.el8.x86_64 30/32 2024-02-23T00:30:22.041 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mon-2:16.2.14-550.geb66ed92.el8.x86_64 30/32 2024-02-23T00:30:22.069 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2024-02-23T00:30:22.199 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mds-2:16.2.14-550.geb66ed92.el8.x86_64 31/32 2024-02-23T00:30:22.217 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mds-2:16.2.14-550.geb66ed92.el8.x86_64 31/32 2024-02-23T00:30:22.366 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-2:16.2.14-550.geb66ed92.el8.x86_64 32/32 2024-02-23T00:30:22.492 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2024-02-23T00:30:22.747 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 14/32 2024-02-23T00:30:22.770 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-2:16.2.14-550.geb66ed92.el8.x86_64 32/32 2024-02-23T00:30:22.770 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-2:16.2.14-550.geb66ed92.el8.x86_64 1/32 2024-02-23T00:30:22.770 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mds-2:16.2.14-550.geb66ed92.el8.x86_64 2/32 2024-02-23T00:30:22.770 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-2:16.2.14-550.geb66ed92.el8.x86_64 3/32 2024-02-23T00:30:22.770 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mon-2:16.2.14-550.geb66ed92.el8.x86_64 4/32 2024-02-23T00:30:22.771 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-osd-2:16.2.14-550.geb66ed92.el8.x86_64 5/32 2024-02-23T00:30:22.771 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-modules-core-2:16.2.14-550.geb66ed92.el8. 6/32 2024-02-23T00:30:22.771 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2024-02-23T00:30:22.771 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2024-02-23T00:30:22.771 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2024-02-23T00:30:22.771 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2024-02-23T00:30:22.771 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2024-02-23T00:30:22.771 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2024-02-23T00:30:22.771 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 13/32 2024-02-23T00:30:22.771 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2024-02-23T00:30:22.771 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2024-02-23T00:30:22.771 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2024-02-23T00:30:22.771 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-02-23T00:30:22.772 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2024-02-23T00:30:22.772 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2024-02-23T00:30:22.772 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2024-02-23T00:30:22.772 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2024-02-23T00:30:22.772 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2024-02-23T00:30:22.772 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2024-02-23T00:30:22.772 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2024-02-23T00:30:22.772 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2024-02-23T00:30:22.772 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2024-02-23T00:30:22.772 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2024-02-23T00:30:22.772 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2024-02-23T00:30:22.772 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2024-02-23T00:30:22.772 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2024-02-23T00:30:22.773 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2024-02-23T00:30:23.144 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2024-02-23T00:30:23.408 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2024-02-23T00:30:23.612 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-02-23T00:30:23.699 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2024-02-23T00:30:23.700 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:30:23.700 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-02-23T00:30:23.700 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:30:23.700 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:30:23.700 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:30:23.700 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core-2:16.2.14-550.geb66ed92.el8.noarch 2024-02-23T00:30:23.700 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:30:23.700 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:30:23.700 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-02-23T00:30:23.700 INFO:teuthology.orchestra.run.smithi196.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-02-23T00:30:23.700 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-02-23T00:30:23.700 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-02-23T00:30:23.701 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-02-23T00:30:23.701 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-02-23T00:30:23.701 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-6.2-6.el8.noarch 2024-02-23T00:30:23.701 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-02-23T00:30:23.701 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-02-23T00:30:23.701 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako-1.0.6-14.el8.noarch 2024-02-23T00:30:23.701 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-02-23T00:30:23.701 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-02-23T00:30:23.701 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-02-23T00:30:23.701 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend-2.6-1.el8.noarch 2024-02-23T00:30:23.701 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-02-23T00:30:23.701 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser-2.14-14.el8.noarch 2024-02-23T00:30:23.701 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz-2017.2-11.el8.noarch 2024-02-23T00:30:23.701 INFO:teuthology.orchestra.run.smithi196.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-02-23T00:30:23.701 INFO:teuthology.orchestra.run.smithi196.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-02-23T00:30:23.702 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-02-23T00:30:23.702 INFO:teuthology.orchestra.run.smithi196.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-02-23T00:30:23.702 INFO:teuthology.orchestra.run.smithi196.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-02-23T00:30:23.702 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-02-23T00:30:23.702 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-02-23T00:30:23.702 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-02-23T00:30:23.702 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-02-23T00:30:23.702 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:30:23.702 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-02-23T00:30:23.955 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2024-02-23T00:30:23.979 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-base 2024-02-23T00:30:24.217 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2024-02-23T00:30:24.430 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2024-02-23T00:30:24.658 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-02-23T00:30:24.737 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:20 ago on Fri 23 Feb 2024 12:29:04 AM UTC. 2024-02-23T00:30:24.935 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-02-23T00:30:25.181 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-02-23T00:30:25.430 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2024-02-23T00:30:25.791 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2024-02-23T00:30:26.121 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-02-23T00:30:26.281 INFO:teuthology.orchestra.run.smithi196.stdout:Package ceph-base-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-23T00:30:26.327 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-02-23T00:30:26.329 INFO:teuthology.orchestra.run.smithi196.stdout:Nothing to do. 2024-02-23T00:30:26.329 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-02-23T00:30:26.438 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install cephadm 2024-02-23T00:30:26.519 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-modules-core-2:16.2.14-550.geb66ed92.el8. 27/32 2024-02-23T00:30:26.774 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-2:16.2.14-550.geb66ed92.el8.x86_64 28/32 2024-02-23T00:30:27.188 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:23 ago on Fri 23 Feb 2024 12:29:04 AM UTC. 2024-02-23T00:30:28.383 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-2:16.2.14-550.geb66ed92.el8.x86_64 28/32 2024-02-23T00:30:28.609 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-osd-2:16.2.14-550.geb66ed92.el8.x86_64 29/32 2024-02-23T00:30:28.739 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-02-23T00:30:28.739 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:30:28.739 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-02-23T00:30:28.739 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:30:28.739 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-02-23T00:30:28.740 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm noarch 2:16.2.14-550.geb66ed92.el8 ceph-noarch 85 k 2024-02-23T00:30:28.740 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:30:28.740 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-02-23T00:30:28.740 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:30:28.740 INFO:teuthology.orchestra.run.smithi196.stdout:Install 1 Package 2024-02-23T00:30:28.740 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:30:28.740 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 85 k 2024-02-23T00:30:28.741 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 352 k 2024-02-23T00:30:28.741 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-02-23T00:30:28.920 INFO:teuthology.orchestra.run.smithi196.stdout:cephadm-16.2.14-550.geb66ed92.el8.noarch.rpm 475 kB/s | 85 kB 00:00 2024-02-23T00:30:28.921 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:30:28.921 INFO:teuthology.orchestra.run.smithi196.stdout:Total 472 kB/s | 85 kB 00:00 2024-02-23T00:30:28.921 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-02-23T00:30:28.928 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-02-23T00:30:28.928 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-02-23T00:30:28.930 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-02-23T00:30:28.933 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-02-23T00:30:28.980 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-02-23T00:30:29.080 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-osd-2:16.2.14-550.geb66ed92.el8.x86_64 29/32 2024-02-23T00:30:29.267 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mon-2:16.2.14-550.geb66ed92.el8.x86_64 30/32 2024-02-23T00:30:29.511 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mon-2:16.2.14-550.geb66ed92.el8.x86_64 30/32 2024-02-23T00:30:29.565 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephadm-2:16.2.14-550.geb66ed92.el8.noarch 1/1 2024-02-23T00:30:29.733 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mds-2:16.2.14-550.geb66ed92.el8.x86_64 31/32 2024-02-23T00:30:29.750 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephadm-2:16.2.14-550.geb66ed92.el8.noarch 1/1 2024-02-23T00:30:29.752 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mds-2:16.2.14-550.geb66ed92.el8.x86_64 31/32 2024-02-23T00:30:29.876 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-2:16.2.14-550.geb66ed92.el8.x86_64 32/32 2024-02-23T00:30:29.975 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephadm-2:16.2.14-550.geb66ed92.el8.noarch 1/1 2024-02-23T00:30:30.306 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-2:16.2.14-550.geb66ed92.el8.x86_64 32/32 2024-02-23T00:30:30.307 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-2:16.2.14-550.geb66ed92.el8.x86_64 1/32 2024-02-23T00:30:30.307 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mds-2:16.2.14-550.geb66ed92.el8.x86_64 2/32 2024-02-23T00:30:30.307 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-2:16.2.14-550.geb66ed92.el8.x86_64 3/32 2024-02-23T00:30:30.307 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mon-2:16.2.14-550.geb66ed92.el8.x86_64 4/32 2024-02-23T00:30:30.307 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-osd-2:16.2.14-550.geb66ed92.el8.x86_64 5/32 2024-02-23T00:30:30.307 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-modules-core-2:16.2.14-550.geb66ed92.el8. 6/32 2024-02-23T00:30:30.307 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2024-02-23T00:30:30.307 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2024-02-23T00:30:30.307 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2024-02-23T00:30:30.307 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2024-02-23T00:30:30.307 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2024-02-23T00:30:30.307 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2024-02-23T00:30:30.308 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 13/32 2024-02-23T00:30:30.308 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2024-02-23T00:30:30.308 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2024-02-23T00:30:30.308 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2024-02-23T00:30:30.308 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-02-23T00:30:30.308 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2024-02-23T00:30:30.308 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2024-02-23T00:30:30.308 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2024-02-23T00:30:30.308 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2024-02-23T00:30:30.308 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2024-02-23T00:30:30.308 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2024-02-23T00:30:30.308 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2024-02-23T00:30:30.308 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2024-02-23T00:30:30.308 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2024-02-23T00:30:30.309 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2024-02-23T00:30:30.309 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2024-02-23T00:30:30.309 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2024-02-23T00:30:30.309 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2024-02-23T00:30:30.309 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2024-02-23T00:30:30.343 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephadm-2:16.2.14-550.geb66ed92.el8.noarch 1/1 2024-02-23T00:30:30.343 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:30:30.343 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-02-23T00:30:30.344 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm-2:16.2.14-550.geb66ed92.el8.noarch 2024-02-23T00:30:30.344 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:30:30.344 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-02-23T00:30:30.607 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-immutable-object-cache 2024-02-23T00:30:31.179 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2024-02-23T00:30:31.179 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:30:31.179 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-02-23T00:30:31.179 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:30:31.179 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:30:31.179 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:30:31.179 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core-2:16.2.14-550.geb66ed92.el8.noarch 2024-02-23T00:30:31.179 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:30:31.179 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:30:31.179 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-02-23T00:30:31.179 INFO:teuthology.orchestra.run.smithi047.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-02-23T00:30:31.180 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-02-23T00:30:31.180 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-02-23T00:30:31.180 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-02-23T00:30:31.180 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-02-23T00:30:31.180 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-6.2-6.el8.noarch 2024-02-23T00:30:31.180 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-02-23T00:30:31.180 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-02-23T00:30:31.180 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako-1.0.6-14.el8.noarch 2024-02-23T00:30:31.180 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-02-23T00:30:31.180 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-02-23T00:30:31.180 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-02-23T00:30:31.180 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend-2.6-1.el8.noarch 2024-02-23T00:30:31.180 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-02-23T00:30:31.180 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pycparser-2.14-14.el8.noarch 2024-02-23T00:30:31.181 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz-2017.2-11.el8.noarch 2024-02-23T00:30:31.181 INFO:teuthology.orchestra.run.smithi047.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-02-23T00:30:31.181 INFO:teuthology.orchestra.run.smithi047.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-02-23T00:30:31.181 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-02-23T00:30:31.181 INFO:teuthology.orchestra.run.smithi047.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-02-23T00:30:31.181 INFO:teuthology.orchestra.run.smithi047.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-02-23T00:30:31.181 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-02-23T00:30:31.181 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-02-23T00:30:31.181 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-02-23T00:30:31.181 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-02-23T00:30:31.181 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:30:31.181 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-02-23T00:30:31.384 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:27 ago on Fri 23 Feb 2024 12:29:04 AM UTC. 2024-02-23T00:30:31.442 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-base 2024-02-23T00:30:32.242 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:24 ago on Fri 23 Feb 2024 12:29:08 AM UTC. 2024-02-23T00:30:32.969 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-02-23T00:30:32.969 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:30:32.970 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repo Size 2024-02-23T00:30:32.970 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:30:32.970 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-02-23T00:30:32.970 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache x86_64 2:16.2.14-550.geb66ed92.el8 ceph 161 k 2024-02-23T00:30:32.970 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:30:32.970 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-02-23T00:30:32.970 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:30:32.970 INFO:teuthology.orchestra.run.smithi196.stdout:Install 1 Package 2024-02-23T00:30:32.970 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:30:32.970 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 161 k 2024-02-23T00:30:32.970 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 470 k 2024-02-23T00:30:32.970 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-02-23T00:30:33.151 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-immutable-object-cache-16.2.14-550.geb66ed 891 kB/s | 161 kB 00:00 2024-02-23T00:30:33.152 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:30:33.152 INFO:teuthology.orchestra.run.smithi196.stdout:Total 885 kB/s | 161 kB 00:00 2024-02-23T00:30:33.152 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-02-23T00:30:33.166 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-02-23T00:30:33.166 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-02-23T00:30:33.215 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-02-23T00:30:33.218 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-02-23T00:30:33.335 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-02-23T00:30:33.565 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-immutable-object-cache-2:16.2.14-550.geb66ed92. 1/1 2024-02-23T00:30:33.882 INFO:teuthology.orchestra.run.smithi047.stdout:Package ceph-base-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-23T00:30:33.933 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-02-23T00:30:33.934 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-02-23T00:30:33.934 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-02-23T00:30:33.964 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.2.14-550.geb66ed92. 1/1 2024-02-23T00:30:34.376 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-immutable-object-cache-2:16.2.14-550.geb66ed92. 1/1 2024-02-23T00:30:34.376 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:30:34.376 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-02-23T00:30:34.376 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:30:34.376 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:30:34.376 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-02-23T00:30:34.592 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install cephadm 2024-02-23T00:30:34.618 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-mgr 2024-02-23T00:30:35.384 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:31 ago on Fri 23 Feb 2024 12:29:04 AM UTC. 2024-02-23T00:30:35.388 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:27 ago on Fri 23 Feb 2024 12:29:08 AM UTC. 2024-02-23T00:30:36.908 INFO:teuthology.orchestra.run.smithi196.stdout:Package ceph-mgr-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-23T00:30:36.955 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-02-23T00:30:36.956 INFO:teuthology.orchestra.run.smithi196.stdout:Nothing to do. 2024-02-23T00:30:36.956 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-02-23T00:30:37.058 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-mgr-dashboard 2024-02-23T00:30:37.069 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-02-23T00:30:37.070 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:30:37.071 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-02-23T00:30:37.071 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:30:37.071 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-02-23T00:30:37.071 INFO:teuthology.orchestra.run.smithi047.stdout: cephadm noarch 2:16.2.14-550.geb66ed92.el8 ceph-noarch 85 k 2024-02-23T00:30:37.071 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:30:37.071 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-02-23T00:30:37.071 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:30:37.071 INFO:teuthology.orchestra.run.smithi047.stdout:Install 1 Package 2024-02-23T00:30:37.071 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:30:37.071 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 85 k 2024-02-23T00:30:37.072 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 352 k 2024-02-23T00:30:37.072 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-02-23T00:30:37.342 INFO:teuthology.orchestra.run.smithi047.stdout:cephadm-16.2.14-550.geb66ed92.el8.noarch.rpm 315 kB/s | 85 kB 00:00 2024-02-23T00:30:37.343 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:30:37.343 INFO:teuthology.orchestra.run.smithi047.stdout:Total 314 kB/s | 85 kB 00:00 2024-02-23T00:30:37.343 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-02-23T00:30:37.351 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-02-23T00:30:37.351 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-02-23T00:30:37.352 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-02-23T00:30:37.356 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-02-23T00:30:37.399 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-02-23T00:30:37.805 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:33 ago on Fri 23 Feb 2024 12:29:04 AM UTC. 2024-02-23T00:30:37.938 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: cephadm-2:16.2.14-550.geb66ed92.el8.noarch 1/1 2024-02-23T00:30:38.110 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephadm-2:16.2.14-550.geb66ed92.el8.noarch 1/1 2024-02-23T00:30:38.354 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: cephadm-2:16.2.14-550.geb66ed92.el8.noarch 1/1 2024-02-23T00:30:38.720 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephadm-2:16.2.14-550.geb66ed92.el8.noarch 1/1 2024-02-23T00:30:38.721 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:30:38.721 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-02-23T00:30:38.721 INFO:teuthology.orchestra.run.smithi047.stdout: cephadm-2:16.2.14-550.geb66ed92.el8.noarch 2024-02-23T00:30:38.721 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:30:38.721 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-02-23T00:30:38.985 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-immutable-object-cache 2024-02-23T00:30:39.387 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-02-23T00:30:39.388 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:30:39.388 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-02-23T00:30:39.388 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:30:39.388 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-02-23T00:30:39.388 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard noarch 2:16.2.14-550.geb66ed92.el8 ceph-noarch 3.1 M 2024-02-23T00:30:39.388 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-02-23T00:30:39.388 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards noarch 2:16.2.14-550.geb66ed92.el8 ceph-noarch 24 k 2024-02-23T00:30:39.388 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts noarch 2:16.2.14-550.geb66ed92.el8 ceph-noarch 15 k 2024-02-23T00:30:39.388 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2024-02-23T00:30:39.388 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-02-23T00:30:39.388 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-02-23T00:30:39.388 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:30:39.389 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-02-23T00:30:39.389 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:30:39.389 INFO:teuthology.orchestra.run.smithi196.stdout:Install 6 Packages 2024-02-23T00:30:39.389 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:30:39.389 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 3.4 M 2024-02-23T00:30:39.389 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 22 M 2024-02-23T00:30:39.390 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-02-23T00:30:39.746 INFO:teuthology.orchestra.run.smithi196.stdout:(1/6): ceph-grafana-dashboards-16.2.14-550.geb6 131 kB/s | 24 kB 00:00 2024-02-23T00:30:39.775 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:31 ago on Fri 23 Feb 2024 12:29:08 AM UTC. 2024-02-23T00:30:39.847 INFO:teuthology.orchestra.run.smithi196.stdout:(2/6): ceph-prometheus-alerts-16.2.14-550.geb66 52 kB/s | 15 kB 00:00 2024-02-23T00:30:39.905 INFO:teuthology.orchestra.run.smithi196.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 268 kB/s | 43 kB 00:00 2024-02-23T00:30:39.997 INFO:teuthology.orchestra.run.smithi196.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 224 kB/s | 34 kB 00:00 2024-02-23T00:30:40.064 INFO:teuthology.orchestra.run.smithi196.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 1.2 MB/s | 196 kB 00:00 2024-02-23T00:30:40.548 INFO:teuthology.orchestra.run.smithi196.stdout:(6/6): ceph-mgr-dashboard-16.2.14-550.geb66ed92 3.2 MB/s | 3.1 MB 00:00 2024-02-23T00:30:40.549 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:30:40.550 INFO:teuthology.orchestra.run.smithi196.stdout:Total 3.0 MB/s | 3.4 MB 00:01 2024-02-23T00:30:40.561 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-02-23T00:30:40.571 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-02-23T00:30:40.572 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-02-23T00:30:40.731 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-02-23T00:30:40.734 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-02-23T00:30:41.024 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-02-23T00:30:41.244 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2024-02-23T00:30:41.447 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-02-23T00:30:41.448 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:30:41.448 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repo Size 2024-02-23T00:30:41.448 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:30:41.448 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-02-23T00:30:41.448 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache x86_64 2:16.2.14-550.geb66ed92.el8 ceph 161 k 2024-02-23T00:30:41.448 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:30:41.448 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-02-23T00:30:41.448 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:30:41.448 INFO:teuthology.orchestra.run.smithi047.stdout:Install 1 Package 2024-02-23T00:30:41.449 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:30:41.449 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 161 k 2024-02-23T00:30:41.449 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 470 k 2024-02-23T00:30:41.449 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-02-23T00:30:41.533 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2024-02-23T00:30:41.749 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2024-02-23T00:30:41.755 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-immutable-object-cache-16.2.14-550.geb66ed 527 kB/s | 161 kB 00:00 2024-02-23T00:30:41.756 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:30:41.756 INFO:teuthology.orchestra.run.smithi047.stdout:Total 525 kB/s | 161 kB 00:00 2024-02-23T00:30:41.756 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-02-23T00:30:41.769 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-02-23T00:30:41.769 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-02-23T00:30:41.821 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-02-23T00:30:41.824 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-02-23T00:30:41.920 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-prometheus-alerts-2:16.2.14-550.geb66ed92.el8.n 4/6 2024-02-23T00:30:41.966 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-02-23T00:30:42.143 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-immutable-object-cache-2:16.2.14-550.geb66ed92. 1/1 2024-02-23T00:30:42.680 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.2.14-550.geb66ed92. 1/1 2024-02-23T00:30:43.014 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-grafana-dashboards-2:16.2.14-550.geb66ed92.el8. 5/6 2024-02-23T00:30:43.055 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-immutable-object-cache-2:16.2.14-550.geb66ed92. 1/1 2024-02-23T00:30:43.056 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:30:43.056 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-02-23T00:30:43.056 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:30:43.056 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:30:43.056 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-02-23T00:30:43.291 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-mgr 2024-02-23T00:30:43.376 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-dashboard-2:16.2.14-550.geb66ed92.el8.noarc 6/6 2024-02-23T00:30:43.657 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.14-550.geb66ed92.el8.noarc 6/6 2024-02-23T00:30:43.657 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-grafana-dashboards-2:16.2.14-550.geb66ed92.el8. 1/6 2024-02-23T00:30:43.657 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-dashboard-2:16.2.14-550.geb66ed92.el8.noarc 2/6 2024-02-23T00:30:43.657 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-prometheus-alerts-2:16.2.14-550.geb66ed92.el8.n 3/6 2024-02-23T00:30:43.657 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2024-02-23T00:30:43.657 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2024-02-23T00:30:44.052 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2024-02-23T00:30:44.052 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:30:44.052 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-02-23T00:30:44.052 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards-2:16.2.14-550.geb66ed92.el8.noarch 2024-02-23T00:30:44.052 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard-2:16.2.14-550.geb66ed92.el8.noarch 2024-02-23T00:30:44.052 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts-2:16.2.14-550.geb66ed92.el8.noarch 2024-02-23T00:30:44.052 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-02-23T00:30:44.053 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-02-23T00:30:44.053 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes-2.4.1-12.el8.noarch 2024-02-23T00:30:44.053 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:30:44.053 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-02-23T00:30:44.081 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:36 ago on Fri 23 Feb 2024 12:29:08 AM UTC. 2024-02-23T00:30:44.216 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-23T00:30:44.971 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:40 ago on Fri 23 Feb 2024 12:29:04 AM UTC. 2024-02-23T00:30:45.716 INFO:teuthology.orchestra.run.smithi047.stdout:Package ceph-mgr-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-23T00:30:45.767 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-02-23T00:30:45.768 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-02-23T00:30:45.768 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-02-23T00:30:45.883 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-mgr-dashboard 2024-02-23T00:30:46.527 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-02-23T00:30:46.528 INFO:teuthology.orchestra.run.smithi196.stdout:============================================================================================ 2024-02-23T00:30:46.529 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-02-23T00:30:46.529 INFO:teuthology.orchestra.run.smithi196.stdout:============================================================================================ 2024-02-23T00:30:46.529 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-02-23T00:30:46.529 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.14-550.geb66ed92.el8 ceph-noarch 8.6 M 2024-02-23T00:30:46.529 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-02-23T00:30:46.529 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2024-02-23T00:30:46.529 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2024-02-23T00:30:46.529 INFO:teuthology.orchestra.run.smithi196.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2024-02-23T00:30:46.529 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2024-02-23T00:30:46.529 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2024-02-23T00:30:46.529 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2024-02-23T00:30:46.529 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy x86_64 1.0.0-21.module_el8+762+77bd8591 appstream 14 M 2024-02-23T00:30:46.529 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:30:46.530 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-02-23T00:30:46.530 INFO:teuthology.orchestra.run.smithi196.stdout:============================================================================================ 2024-02-23T00:30:46.530 INFO:teuthology.orchestra.run.smithi196.stdout:Install 8 Packages 2024-02-23T00:30:46.530 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:30:46.530 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 37 M 2024-02-23T00:30:46.530 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 210 M 2024-02-23T00:30:46.530 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-02-23T00:30:46.667 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:38 ago on Fri 23 Feb 2024 12:29:08 AM UTC. 2024-02-23T00:30:47.310 INFO:teuthology.orchestra.run.smithi196.stdout:(1/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 8.9 MB/s | 4.9 MB 00:00 2024-02-23T00:30:47.394 INFO:teuthology.orchestra.run.smithi196.stdout:(2/8): openblas-0.3.15-4.el8.x86_64.rpm 7.5 MB/s | 4.7 MB 00:00 2024-02-23T00:30:47.511 INFO:teuthology.orchestra.run.smithi196.stdout:(3/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 19 MB/s | 3.7 MB 00:00 2024-02-23T00:30:47.536 INFO:teuthology.orchestra.run.smithi196.stdout:(4/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 1.5 MB/s | 225 kB 00:00 2024-02-23T00:30:47.853 INFO:teuthology.orchestra.run.smithi196.stdout:(5/8): ceph-mgr-diskprediction-local-16.2.14-55 7.9 MB/s | 8.6 MB 00:01 2024-02-23T00:30:48.018 INFO:teuthology.orchestra.run.smithi196.stdout:(6/8): python3-scipy-1.0.0-21.module_el8+762+77 28 MB/s | 14 MB 00:00 2024-02-23T00:30:48.179 INFO:teuthology.orchestra.run.smithi196.stdout:(7/8): libquadmath-8.5.0-21.el8.x86_64.rpm 530 kB/s | 172 kB 00:00 2024-02-23T00:30:48.347 INFO:teuthology.orchestra.run.smithi196.stdout:(8/8): libgfortran-8.5.0-21.el8.x86_64.rpm 797 kB/s | 645 kB 00:00 2024-02-23T00:30:48.348 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:30:48.348 INFO:teuthology.orchestra.run.smithi196.stdout:Total 20 MB/s | 37 MB 00:01 2024-02-23T00:30:48.350 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-02-23T00:30:48.351 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:30:48.351 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-02-23T00:30:48.351 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:30:48.352 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-02-23T00:30:48.352 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard noarch 2:16.2.14-550.geb66ed92.el8 ceph-noarch 3.1 M 2024-02-23T00:30:48.352 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-02-23T00:30:48.352 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-grafana-dashboards noarch 2:16.2.14-550.geb66ed92.el8 ceph-noarch 24 k 2024-02-23T00:30:48.352 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-prometheus-alerts noarch 2:16.2.14-550.geb66ed92.el8 ceph-noarch 15 k 2024-02-23T00:30:48.352 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2024-02-23T00:30:48.352 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-02-23T00:30:48.352 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-02-23T00:30:48.352 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:30:48.352 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-02-23T00:30:48.352 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:30:48.353 INFO:teuthology.orchestra.run.smithi047.stdout:Install 6 Packages 2024-02-23T00:30:48.353 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:30:48.353 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 3.4 M 2024-02-23T00:30:48.353 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 22 M 2024-02-23T00:30:48.353 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-02-23T00:30:48.582 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-02-23T00:30:48.601 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-02-23T00:30:48.601 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-02-23T00:30:48.883 INFO:teuthology.orchestra.run.smithi047.stdout:(1/6): ceph-prometheus-alerts-16.2.14-550.geb66 95 kB/s | 15 kB 00:00 2024-02-23T00:30:48.963 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-02-23T00:30:48.966 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-02-23T00:30:48.983 INFO:teuthology.orchestra.run.smithi047.stdout:(2/6): ceph-grafana-dashboards-16.2.14-550.geb6 95 kB/s | 24 kB 00:00 2024-02-23T00:30:49.044 INFO:teuthology.orchestra.run.smithi047.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 264 kB/s | 43 kB 00:00 2024-02-23T00:30:49.244 INFO:teuthology.orchestra.run.smithi047.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 129 kB/s | 34 kB 00:00 2024-02-23T00:30:49.411 INFO:teuthology.orchestra.run.smithi047.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 536 kB/s | 196 kB 00:00 2024-02-23T00:30:49.478 INFO:teuthology.orchestra.run.smithi047.stdout:(6/6): ceph-mgr-dashboard-16.2.14-550.geb66ed92 4.2 MB/s | 3.1 MB 00:00 2024-02-23T00:30:49.482 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:30:49.482 INFO:teuthology.orchestra.run.smithi047.stdout:Total 3.0 MB/s | 3.4 MB 00:01 2024-02-23T00:30:49.494 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-02-23T00:30:49.505 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-02-23T00:30:49.505 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-02-23T00:30:49.532 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-02-23T00:30:49.678 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 1/8 2024-02-23T00:30:49.682 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-02-23T00:30:49.686 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-02-23T00:30:49.817 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 1/8 2024-02-23T00:30:49.961 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 2/8 2024-02-23T00:30:49.994 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-02-23T00:30:50.546 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 2/8 2024-02-23T00:30:50.584 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2024-02-23T00:30:50.753 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2024-02-23T00:30:50.845 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2024-02-23T00:30:51.067 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2024-02-23T00:30:51.180 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-prometheus-alerts-2:16.2.14-550.geb66ed92.el8.n 4/6 2024-02-23T00:30:51.360 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2024-02-23T00:30:51.544 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-02-23T00:30:52.220 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-02-23T00:30:52.328 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-grafana-dashboards-2:16.2.14-550.geb66ed92.el8. 5/6 2024-02-23T00:30:52.590 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-02-23T00:30:52.603 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-dashboard-2:16.2.14-550.geb66ed92.el8.noarc 6/6 2024-02-23T00:30:52.852 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-02-23T00:30:52.896 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.14-550.geb66ed92.el8.noarc 6/6 2024-02-23T00:30:52.897 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-grafana-dashboards-2:16.2.14-550.geb66ed92.el8. 1/6 2024-02-23T00:30:52.897 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-dashboard-2:16.2.14-550.geb66ed92.el8.noarc 2/6 2024-02-23T00:30:52.897 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-prometheus-alerts-2:16.2.14-550.geb66ed92.el8.n 3/6 2024-02-23T00:30:52.897 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2024-02-23T00:30:52.897 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2024-02-23T00:30:53.285 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2024-02-23T00:30:53.285 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:30:53.285 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-02-23T00:30:53.285 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-grafana-dashboards-2:16.2.14-550.geb66ed92.el8.noarch 2024-02-23T00:30:53.285 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard-2:16.2.14-550.geb66ed92.el8.noarch 2024-02-23T00:30:53.286 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-prometheus-alerts-2:16.2.14-550.geb66ed92.el8.noarch 2024-02-23T00:30:53.286 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-02-23T00:30:53.286 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-02-23T00:30:53.286 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes-2.4.1-12.el8.noarch 2024-02-23T00:30:53.286 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:30:53.286 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-02-23T00:30:53.454 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-23T00:30:54.237 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:46 ago on Fri 23 Feb 2024 12:29:08 AM UTC. 2024-02-23T00:30:54.618 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-02-23T00:30:55.935 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-02-23T00:30:55.937 INFO:teuthology.orchestra.run.smithi047.stdout:============================================================================================ 2024-02-23T00:30:55.937 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-02-23T00:30:55.937 INFO:teuthology.orchestra.run.smithi047.stdout:============================================================================================ 2024-02-23T00:30:55.937 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-02-23T00:30:55.937 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.14-550.geb66ed92.el8 ceph-noarch 8.6 M 2024-02-23T00:30:55.937 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-02-23T00:30:55.937 INFO:teuthology.orchestra.run.smithi047.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2024-02-23T00:30:55.937 INFO:teuthology.orchestra.run.smithi047.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2024-02-23T00:30:55.937 INFO:teuthology.orchestra.run.smithi047.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2024-02-23T00:30:55.937 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2024-02-23T00:30:55.937 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2024-02-23T00:30:55.937 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2024-02-23T00:30:55.938 INFO:teuthology.orchestra.run.smithi047.stdout: python3-scipy x86_64 1.0.0-21.module_el8+762+77bd8591 appstream 14 M 2024-02-23T00:30:55.938 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:30:55.938 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-02-23T00:30:55.938 INFO:teuthology.orchestra.run.smithi047.stdout:============================================================================================ 2024-02-23T00:30:55.938 INFO:teuthology.orchestra.run.smithi047.stdout:Install 8 Packages 2024-02-23T00:30:55.938 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:30:55.938 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 37 M 2024-02-23T00:30:55.939 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 210 M 2024-02-23T00:30:55.939 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-02-23T00:30:56.052 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 7/8 2024-02-23T00:30:56.260 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.14-550.geb66ed9 8/8 2024-02-23T00:30:56.581 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.14-550.geb66ed9 8/8 2024-02-23T00:30:56.582 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.14-550.geb66ed9 1/8 2024-02-23T00:30:56.582 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2024-02-23T00:30:56.582 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2024-02-23T00:30:56.582 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2024-02-23T00:30:56.582 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2024-02-23T00:30:56.582 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 6/8 2024-02-23T00:30:56.583 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 7/8 2024-02-23T00:30:56.888 INFO:teuthology.orchestra.run.smithi047.stdout:(1/8): openblas-0.3.15-4.el8.x86_64.rpm 8.5 MB/s | 4.7 MB 00:00 2024-02-23T00:30:56.959 INFO:teuthology.orchestra.run.smithi047.stdout:(2/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 7.7 MB/s | 4.9 MB 00:00 2024-02-23T00:30:56.988 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 8/8 2024-02-23T00:30:56.989 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:30:56.989 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-02-23T00:30:56.989 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local-2:16.2.14-550.geb66ed92.el8.noarch 2024-02-23T00:30:56.989 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran-8.5.0-21.el8.x86_64 2024-02-23T00:30:56.989 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath-8.5.0-21.el8.x86_64 2024-02-23T00:30:56.989 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-0.3.15-4.el8.x86_64 2024-02-23T00:30:56.989 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-threads-0.3.15-4.el8.x86_64 2024-02-23T00:30:56.989 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-02-23T00:30:56.989 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-02-23T00:30:56.989 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_64 2024-02-23T00:30:56.989 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:30:56.989 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-02-23T00:30:57.035 INFO:teuthology.orchestra.run.smithi047.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 2.9 MB/s | 225 kB 00:00 2024-02-23T00:30:57.127 INFO:teuthology.orchestra.run.smithi047.stdout:(4/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 15 MB/s | 3.7 MB 00:00 2024-02-23T00:30:57.135 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-mgr-rook 2024-02-23T00:30:57.494 INFO:teuthology.orchestra.run.smithi047.stdout:(5/8): python3-scipy-1.0.0-21.module_el8+762+77 31 MB/s | 14 MB 00:00 2024-02-23T00:30:57.661 INFO:teuthology.orchestra.run.smithi047.stdout:(6/8): libquadmath-8.5.0-21.el8.x86_64.rpm 1.0 MB/s | 172 kB 00:00 2024-02-23T00:30:57.728 INFO:teuthology.orchestra.run.smithi047.stdout:(7/8): libgfortran-8.5.0-21.el8.x86_64.rpm 1.0 MB/s | 645 kB 00:00 2024-02-23T00:30:57.884 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:53 ago on Fri 23 Feb 2024 12:29:04 AM UTC. 2024-02-23T00:30:59.460 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-02-23T00:30:59.462 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:30:59.462 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-02-23T00:30:59.462 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:30:59.462 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-02-23T00:30:59.462 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook noarch 2:16.2.14-550.geb66ed92.el8 ceph-noarch 27 k 2024-02-23T00:30:59.462 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-02-23T00:30:59.462 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-02-23T00:30:59.462 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-02-23T00:30:59.462 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-02-23T00:30:59.463 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2024-02-23T00:30:59.463 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2024-02-23T00:30:59.463 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-02-23T00:30:59.463 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2024-02-23T00:30:59.463 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2024-02-23T00:30:59.463 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2024-02-23T00:30:59.463 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2024-02-23T00:30:59.463 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-02-23T00:30:59.463 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-02-23T00:30:59.463 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:30:59.463 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-02-23T00:30:59.463 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:30:59.463 INFO:teuthology.orchestra.run.smithi196.stdout:Install 13 Packages 2024-02-23T00:30:59.463 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:30:59.464 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 1.9 M 2024-02-23T00:30:59.464 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 24 M 2024-02-23T00:30:59.464 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-02-23T00:30:59.653 INFO:teuthology.orchestra.run.smithi047.stdout:(8/8): ceph-mgr-diskprediction-local-16.2.14-55 2.6 MB/s | 8.6 MB 00:03 2024-02-23T00:30:59.654 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:30:59.654 INFO:teuthology.orchestra.run.smithi047.stdout:Total 10 MB/s | 37 MB 00:03 2024-02-23T00:30:59.895 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-02-23T00:30:59.914 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-02-23T00:30:59.915 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-02-23T00:31:00.012 INFO:teuthology.orchestra.run.smithi196.stdout:(1/13): ceph-mgr-rook-16.2.14-550.geb66ed92.el8 191 kB/s | 27 kB 00:00 2024-02-23T00:31:00.038 INFO:teuthology.orchestra.run.smithi196.stdout:(2/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 157 kB/s | 27 kB 00:00 2024-02-23T00:31:00.063 INFO:teuthology.orchestra.run.smithi196.stdout:(3/13): python3-jsonpointer-1.10-11.el8.noarch. 103 kB/s | 20 kB 00:00 2024-02-23T00:31:00.197 INFO:teuthology.orchestra.run.smithi196.stdout:(4/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 687 kB/s | 126 kB 00:00 2024-02-23T00:31:00.222 INFO:teuthology.orchestra.run.smithi196.stdout:(5/13): python3-pyasn1-modules-0.3.7-6.el8.noar 695 kB/s | 110 kB 00:00 2024-02-23T00:31:00.247 INFO:teuthology.orchestra.run.smithi196.stdout:(6/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 843 kB/s | 155 kB 00:00 2024-02-23T00:31:00.317 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-02-23T00:31:00.321 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-02-23T00:31:00.330 INFO:teuthology.orchestra.run.smithi196.stdout:(7/13): python3-requests-oauthlib-1.0.0-1.el8.n 319 kB/s | 43 kB 00:00 2024-02-23T00:31:00.481 INFO:teuthology.orchestra.run.smithi196.stdout:(8/13): python3-certifi-2018.10.15-7.el8.noarch 70 kB/s | 16 kB 00:00 2024-02-23T00:31:00.507 INFO:teuthology.orchestra.run.smithi196.stdout:(9/13): python3-cachetools-3.1.1-4.el8.noarch.r 116 kB/s | 33 kB 00:00 2024-02-23T00:31:00.623 INFO:teuthology.orchestra.run.smithi196.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 278 kB/s | 81 kB 00:00 2024-02-23T00:31:00.648 INFO:teuthology.orchestra.run.smithi196.stdout:(11/13): python3-rsa-4.9-2.el8.noarch.rpm 453 kB/s | 64 kB 00:00 2024-02-23T00:31:00.757 INFO:teuthology.orchestra.run.smithi196.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 459 kB/s | 61 kB 00:00 2024-02-23T00:31:00.874 INFO:teuthology.orchestra.run.smithi196.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 3.0 MB/s | 1.2 MB 00:00 2024-02-23T00:31:00.876 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:31:00.876 INFO:teuthology.orchestra.run.smithi196.stdout:Total 1.4 MB/s | 1.9 MB 00:01 2024-02-23T00:31:00.933 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-02-23T00:31:00.950 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-02-23T00:31:00.950 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-02-23T00:31:01.038 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-02-23T00:31:01.042 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-02-23T00:31:01.588 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-02-23T00:31:01.670 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-02-23T00:31:01.746 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 1/8 2024-02-23T00:31:01.886 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 1/8 2024-02-23T00:31:01.911 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2024-02-23T00:31:02.062 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 2/8 2024-02-23T00:31:02.100 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2024-02-23T00:31:02.378 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/13 2024-02-23T00:31:02.632 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2024-02-23T00:31:02.679 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 2/8 2024-02-23T00:31:02.821 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2024-02-23T00:31:02.881 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2024-02-23T00:31:03.137 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2024-02-23T00:31:03.454 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2024-02-23T00:31:03.515 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2024-02-23T00:31:03.654 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-02-23T00:31:03.721 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2024-02-23T00:31:04.391 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-02-23T00:31:04.797 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2024-02-23T00:31:04.881 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-02-23T00:31:05.082 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2024-02-23T00:31:05.128 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-02-23T00:31:05.340 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2024-02-23T00:31:05.524 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2024-02-23T00:31:05.747 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-rook-2:16.2.14-550.geb66ed92.el8.noarch 13/13 2024-02-23T00:31:06.058 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.14-550.geb66ed92.el8.noarch 13/13 2024-02-23T00:31:06.059 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-rook-2:16.2.14-550.geb66ed92.el8.noarch 1/13 2024-02-23T00:31:06.059 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2024-02-23T00:31:06.059 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2024-02-23T00:31:06.059 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2024-02-23T00:31:06.059 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2024-02-23T00:31:06.059 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2024-02-23T00:31:06.060 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2024-02-23T00:31:06.060 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2024-02-23T00:31:06.060 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2024-02-23T00:31:06.060 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2024-02-23T00:31:06.060 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2024-02-23T00:31:06.060 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 12/13 2024-02-23T00:31:06.539 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2024-02-23T00:31:06.539 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:31:06.539 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-02-23T00:31:06.540 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook-2:16.2.14-550.geb66ed92.el8.noarch 2024-02-23T00:31:06.540 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-02-23T00:31:06.540 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-02-23T00:31:06.540 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-02-23T00:31:06.540 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-02-23T00:31:06.540 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-02-23T00:31:06.540 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-02-23T00:31:06.540 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-02-23T00:31:06.540 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-02-23T00:31:06.540 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-02-23T00:31:06.540 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-02-23T00:31:06.540 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa-4.9-2.el8.noarch 2024-02-23T00:31:06.541 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-02-23T00:31:06.541 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:31:06.541 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-02-23T00:31:06.679 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-mgr-cephadm 2024-02-23T00:31:07.016 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-02-23T00:31:07.426 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:02:03 ago on Fri 23 Feb 2024 12:29:04 AM UTC. 2024-02-23T00:31:08.413 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 7/8 2024-02-23T00:31:08.612 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.14-550.geb66ed9 8/8 2024-02-23T00:31:08.944 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.14-550.geb66ed9 8/8 2024-02-23T00:31:08.944 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.14-550.geb66ed9 1/8 2024-02-23T00:31:08.944 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2024-02-23T00:31:08.945 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2024-02-23T00:31:08.945 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2024-02-23T00:31:08.945 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2024-02-23T00:31:08.945 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 6/8 2024-02-23T00:31:08.945 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 7/8 2024-02-23T00:31:09.002 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-02-23T00:31:09.003 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:31:09.003 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-02-23T00:31:09.003 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:31:09.003 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-02-23T00:31:09.003 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm noarch 2:16.2.14-550.geb66ed92.el8 ceph-noarch 116 k 2024-02-23T00:31:09.003 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-02-23T00:31:09.003 INFO:teuthology.orchestra.run.smithi196.stdout: python3-apipkg noarch 1.5-6.el8 epel 21 k 2024-02-23T00:31:09.003 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2024-02-23T00:31:09.003 INFO:teuthology.orchestra.run.smithi196.stdout: python3-execnet noarch 1.7.1-1.el8 epel 355 k 2024-02-23T00:31:09.003 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2 noarch 2.10.1-4.el8 appstream 578 k 2024-02-23T00:31:09.003 INFO:teuthology.orchestra.run.smithi196.stdout: python3-remoto noarch 1.2.1-1.el8 epel 49 k 2024-02-23T00:31:09.003 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:31:09.004 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-02-23T00:31:09.004 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:31:09.004 INFO:teuthology.orchestra.run.smithi196.stdout:Install 6 Packages 2024-02-23T00:31:09.004 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:31:09.004 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 5.9 M 2024-02-23T00:31:09.004 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 24 M 2024-02-23T00:31:09.004 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-02-23T00:31:09.369 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 8/8 2024-02-23T00:31:09.369 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:31:09.369 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-02-23T00:31:09.369 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local-2:16.2.14-550.geb66ed92.el8.noarch 2024-02-23T00:31:09.369 INFO:teuthology.orchestra.run.smithi047.stdout: libgfortran-8.5.0-21.el8.x86_64 2024-02-23T00:31:09.369 INFO:teuthology.orchestra.run.smithi047.stdout: libquadmath-8.5.0-21.el8.x86_64 2024-02-23T00:31:09.370 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-0.3.15-4.el8.x86_64 2024-02-23T00:31:09.370 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-threads-0.3.15-4.el8.x86_64 2024-02-23T00:31:09.370 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-02-23T00:31:09.370 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-02-23T00:31:09.370 INFO:teuthology.orchestra.run.smithi047.stdout: python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_64 2024-02-23T00:31:09.370 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:31:09.370 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-02-23T00:31:09.504 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-mgr-rook 2024-02-23T00:31:09.582 INFO:teuthology.orchestra.run.smithi196.stdout:(1/6): python3-jinja2-2.10.1-4.el8.noarch.rpm 1.6 MB/s | 578 kB 00:00 2024-02-23T00:31:09.607 INFO:teuthology.orchestra.run.smithi196.stdout:(2/6): ceph-mgr-cephadm-16.2.14-550.geb66ed92.e 300 kB/s | 116 kB 00:00 2024-02-23T00:31:09.808 INFO:teuthology.orchestra.run.smithi196.stdout:(3/6): python3-apipkg-1.5-6.el8.noarch.rpm 94 kB/s | 21 kB 00:00 2024-02-23T00:31:09.874 INFO:teuthology.orchestra.run.smithi196.stdout:(4/6): python3-babel-2.5.1-7.el8.noarch.rpm 7.3 MB/s | 4.8 MB 00:00 2024-02-23T00:31:09.908 INFO:teuthology.orchestra.run.smithi196.stdout:(5/6): python3-remoto-1.2.1-1.el8.noarch.rpm 488 kB/s | 49 kB 00:00 2024-02-23T00:31:09.966 INFO:teuthology.orchestra.run.smithi196.stdout:(6/6): python3-execnet-1.7.1-1.el8.noarch.rpm 991 kB/s | 355 kB 00:00 2024-02-23T00:31:09.968 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:31:09.968 INFO:teuthology.orchestra.run.smithi196.stdout:Total 6.1 MB/s | 5.9 MB 00:00 2024-02-23T00:31:10.019 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-02-23T00:31:10.029 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-02-23T00:31:10.029 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-02-23T00:31:10.137 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-02-23T00:31:10.140 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-02-23T00:31:10.289 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:02:02 ago on Fri 23 Feb 2024 12:29:08 AM UTC. 2024-02-23T00:31:10.419 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-02-23T00:31:10.674 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 1/6 2024-02-23T00:31:10.968 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 2/6 2024-02-23T00:31:11.932 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-remoto-1.2.1-1.el8.noarch 3/6 2024-02-23T00:31:11.967 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-02-23T00:31:11.969 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:31:11.969 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-02-23T00:31:11.969 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:31:11.969 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-02-23T00:31:11.969 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-rook noarch 2:16.2.14-550.geb66ed92.el8 ceph-noarch 27 k 2024-02-23T00:31:11.969 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-02-23T00:31:11.969 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-02-23T00:31:11.969 INFO:teuthology.orchestra.run.smithi047.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-02-23T00:31:11.969 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-02-23T00:31:11.969 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2024-02-23T00:31:11.970 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2024-02-23T00:31:11.970 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-02-23T00:31:11.970 INFO:teuthology.orchestra.run.smithi047.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2024-02-23T00:31:11.970 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2024-02-23T00:31:11.970 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2024-02-23T00:31:11.970 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2024-02-23T00:31:11.970 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-02-23T00:31:11.970 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-02-23T00:31:11.970 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:31:11.970 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-02-23T00:31:11.971 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:31:11.971 INFO:teuthology.orchestra.run.smithi047.stdout:Install 13 Packages 2024-02-23T00:31:11.971 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:31:11.971 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 1.9 M 2024-02-23T00:31:11.971 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 24 M 2024-02-23T00:31:11.971 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-02-23T00:31:12.260 INFO:teuthology.orchestra.run.smithi047.stdout:(1/13): python3-jsonpointer-1.10-11.el8.noarch. 109 kB/s | 20 kB 00:00 2024-02-23T00:31:12.285 INFO:teuthology.orchestra.run.smithi047.stdout:(2/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 128 kB/s | 27 kB 00:00 2024-02-23T00:31:12.310 INFO:teuthology.orchestra.run.smithi047.stdout:(3/13): ceph-mgr-rook-16.2.14-550.geb66ed92.el8 117 kB/s | 27 kB 00:00 2024-02-23T00:31:12.320 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 4/6 2024-02-23T00:31:12.419 INFO:teuthology.orchestra.run.smithi047.stdout:(4/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 796 kB/s | 126 kB 00:00 2024-02-23T00:31:12.478 INFO:teuthology.orchestra.run.smithi047.stdout:(5/13): python3-pyasn1-modules-0.3.7-6.el8.noar 572 kB/s | 110 kB 00:00 2024-02-23T00:31:12.565 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jinja2-2.10.1-4.el8.noarch 5/6 2024-02-23T00:31:12.594 INFO:teuthology.orchestra.run.smithi047.stdout:(6/13): python3-requests-oauthlib-1.0.0-1.el8.n 243 kB/s | 43 kB 00:00 2024-02-23T00:31:12.620 INFO:teuthology.orchestra.run.smithi047.stdout:(7/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 501 kB/s | 155 kB 00:00 2024-02-23T00:31:12.728 INFO:teuthology.orchestra.run.smithi047.stdout:(8/13): python3-cachetools-3.1.1-4.el8.noarch.r 131 kB/s | 33 kB 00:00 2024-02-23T00:31:12.782 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-cephadm-2:16.2.14-550.geb66ed92.el8.noarch 6/6 2024-02-23T00:31:12.821 INFO:teuthology.orchestra.run.smithi047.stdout:(9/13): python3-certifi-2018.10.15-7.el8.noarch 72 kB/s | 16 kB 00:00 2024-02-23T00:31:12.912 INFO:teuthology.orchestra.run.smithi047.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 278 kB/s | 81 kB 00:00 2024-02-23T00:31:12.971 INFO:teuthology.orchestra.run.smithi047.stdout:(11/13): python3-rsa-4.9-2.el8.noarch.rpm 428 kB/s | 64 kB 00:00 2024-02-23T00:31:13.013 INFO:teuthology.orchestra.run.smithi047.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 613 kB/s | 61 kB 00:00 2024-02-23T00:31:13.108 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.2.14-550.geb66ed92.el8.noarch 6/6 2024-02-23T00:31:13.108 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-cephadm-2:16.2.14-550.geb66ed92.el8.noarch 1/6 2024-02-23T00:31:13.109 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/6 2024-02-23T00:31:13.109 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jinja2-2.10.1-4.el8.noarch 3/6 2024-02-23T00:31:13.109 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 4/6 2024-02-23T00:31:13.109 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 5/6 2024-02-23T00:31:13.113 INFO:teuthology.orchestra.run.smithi047.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 3.0 MB/s | 1.2 MB 00:00 2024-02-23T00:31:13.116 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:31:13.116 INFO:teuthology.orchestra.run.smithi047.stdout:Total 1.7 MB/s | 1.9 MB 00:01 2024-02-23T00:31:13.175 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-02-23T00:31:13.191 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-02-23T00:31:13.191 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-02-23T00:31:13.287 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-02-23T00:31:13.291 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-02-23T00:31:13.561 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-remoto-1.2.1-1.el8.noarch 6/6 2024-02-23T00:31:13.561 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:31:13.561 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-02-23T00:31:13.562 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm-2:16.2.14-550.geb66ed92.el8.noarch 2024-02-23T00:31:13.562 INFO:teuthology.orchestra.run.smithi196.stdout: python3-apipkg-1.5-6.el8.noarch 2024-02-23T00:31:13.562 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel-2.5.1-7.el8.noarch 2024-02-23T00:31:13.562 INFO:teuthology.orchestra.run.smithi196.stdout: python3-execnet-1.7.1-1.el8.noarch 2024-02-23T00:31:13.562 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2-2.10.1-4.el8.noarch 2024-02-23T00:31:13.562 INFO:teuthology.orchestra.run.smithi196.stdout: python3-remoto-1.2.1-1.el8.noarch 2024-02-23T00:31:13.562 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:31:13.562 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-02-23T00:31:13.695 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-fuse 2024-02-23T00:31:13.976 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-02-23T00:31:14.217 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2024-02-23T00:31:14.439 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2024-02-23T00:31:14.447 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:02:10 ago on Fri 23 Feb 2024 12:29:04 AM UTC. 2024-02-23T00:31:14.709 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/13 2024-02-23T00:31:15.103 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2024-02-23T00:31:15.361 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2024-02-23T00:31:15.612 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2024-02-23T00:31:15.958 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2024-02-23T00:31:16.013 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-02-23T00:31:16.013 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:31:16.014 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-02-23T00:31:16.014 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:31:16.014 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-02-23T00:31:16.014 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse x86_64 2:16.2.14-550.geb66ed92.el8 ceph 911 k 2024-02-23T00:31:16.014 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-02-23T00:31:16.014 INFO:teuthology.orchestra.run.smithi196.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2024-02-23T00:31:16.014 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:31:16.014 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-02-23T00:31:16.014 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:31:16.015 INFO:teuthology.orchestra.run.smithi196.stdout:Install 2 Packages 2024-02-23T00:31:16.015 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:31:16.015 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 994 k 2024-02-23T00:31:16.015 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 2.8 M 2024-02-23T00:31:16.015 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-02-23T00:31:16.194 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2024-02-23T00:31:16.213 INFO:teuthology.orchestra.run.smithi196.stdout:(1/2): fuse-2.9.7-17.el8.x86_64.rpm 527 kB/s | 83 kB 00:00 2024-02-23T00:31:16.313 INFO:teuthology.orchestra.run.smithi196.stdout:(2/2): ceph-fuse-16.2.14-550.geb66ed92.el8.x86_ 3.5 MB/s | 911 kB 00:00 2024-02-23T00:31:16.314 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:31:16.314 INFO:teuthology.orchestra.run.smithi196.stdout:Total 3.2 MB/s | 994 kB 00:00 2024-02-23T00:31:16.324 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-02-23T00:31:16.345 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-02-23T00:31:16.345 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-02-23T00:31:16.432 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-02-23T00:31:16.435 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-02-23T00:31:16.656 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-02-23T00:31:16.963 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : fuse-2.9.7-17.el8.x86_64 1/2 2024-02-23T00:31:17.198 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-fuse-2:16.2.14-550.geb66ed92.el8.x86_64 2/2 2024-02-23T00:31:17.323 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2024-02-23T00:31:17.597 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-fuse-2:16.2.14-550.geb66ed92.el8.x86_64 2/2 2024-02-23T00:31:17.597 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-fuse-2:16.2.14-550.geb66ed92.el8.x86_64 1/2 2024-02-23T00:31:17.610 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2024-02-23T00:31:17.852 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2024-02-23T00:31:18.518 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2024-02-23T00:31:18.518 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-rook-2:16.2.14-550.geb66ed92.el8.noarch 13/13 2024-02-23T00:31:18.518 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 2/2 2024-02-23T00:31:18.519 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:31:18.519 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-02-23T00:31:18.519 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse-2:16.2.14-550.geb66ed92.el8.x86_64 fuse-2.9.7-17.el8.x86_64 2024-02-23T00:31:18.519 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:31:18.519 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-02-23T00:31:18.519 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install librados-devel 2024-02-23T00:31:18.565 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.14-550.geb66ed92.el8.noarch 13/13 2024-02-23T00:31:18.565 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-rook-2:16.2.14-550.geb66ed92.el8.noarch 1/13 2024-02-23T00:31:18.565 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2024-02-23T00:31:18.565 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2024-02-23T00:31:18.566 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2024-02-23T00:31:18.566 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2024-02-23T00:31:18.566 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2024-02-23T00:31:18.566 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2024-02-23T00:31:18.566 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2024-02-23T00:31:18.566 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2024-02-23T00:31:18.566 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2024-02-23T00:31:18.566 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2024-02-23T00:31:18.566 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 12/13 2024-02-23T00:31:19.037 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2024-02-23T00:31:19.038 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:31:19.038 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-02-23T00:31:19.038 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-rook-2:16.2.14-550.geb66ed92.el8.noarch 2024-02-23T00:31:19.038 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-02-23T00:31:19.038 INFO:teuthology.orchestra.run.smithi047.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-02-23T00:31:19.038 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-02-23T00:31:19.038 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-02-23T00:31:19.038 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-02-23T00:31:19.038 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-02-23T00:31:19.038 INFO:teuthology.orchestra.run.smithi047.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-02-23T00:31:19.038 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-02-23T00:31:19.038 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-02-23T00:31:19.039 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-02-23T00:31:19.039 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa-4.9-2.el8.noarch 2024-02-23T00:31:19.039 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-02-23T00:31:19.039 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:31:19.039 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-02-23T00:31:19.179 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-mgr-cephadm 2024-02-23T00:31:19.284 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:02:15 ago on Fri 23 Feb 2024 12:29:04 AM UTC. 2024-02-23T00:31:19.961 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:02:11 ago on Fri 23 Feb 2024 12:29:08 AM UTC. 2024-02-23T00:31:20.861 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-02-23T00:31:20.861 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:31:20.861 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repo Size 2024-02-23T00:31:20.861 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:31:20.862 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-02-23T00:31:20.862 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel x86_64 2:16.2.14-550.geb66ed92.el8 ceph 131 k 2024-02-23T00:31:20.862 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:31:20.862 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-02-23T00:31:20.862 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:31:20.862 INFO:teuthology.orchestra.run.smithi196.stdout:Install 1 Package 2024-02-23T00:31:20.862 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:31:20.862 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 131 k 2024-02-23T00:31:20.863 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 432 k 2024-02-23T00:31:20.863 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-02-23T00:31:21.059 INFO:teuthology.orchestra.run.smithi196.stdout:librados-devel-16.2.14-550.geb66ed92.el8.x86_64 668 kB/s | 131 kB 00:00 2024-02-23T00:31:21.060 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:31:21.060 INFO:teuthology.orchestra.run.smithi196.stdout:Total 664 kB/s | 131 kB 00:00 2024-02-23T00:31:21.060 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-02-23T00:31:21.065 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-02-23T00:31:21.065 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-02-23T00:31:21.115 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-02-23T00:31:21.118 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-02-23T00:31:21.221 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-02-23T00:31:21.488 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librados-devel-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-23T00:31:21.654 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-02-23T00:31:21.655 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:31:21.655 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-02-23T00:31:21.655 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:31:21.655 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-02-23T00:31:21.655 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-cephadm noarch 2:16.2.14-550.geb66ed92.el8 ceph-noarch 116 k 2024-02-23T00:31:21.655 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-02-23T00:31:21.656 INFO:teuthology.orchestra.run.smithi047.stdout: python3-apipkg noarch 1.5-6.el8 epel 21 k 2024-02-23T00:31:21.656 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2024-02-23T00:31:21.656 INFO:teuthology.orchestra.run.smithi047.stdout: python3-execnet noarch 1.7.1-1.el8 epel 355 k 2024-02-23T00:31:21.656 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2 noarch 2.10.1-4.el8 appstream 578 k 2024-02-23T00:31:21.656 INFO:teuthology.orchestra.run.smithi047.stdout: python3-remoto noarch 1.2.1-1.el8 epel 49 k 2024-02-23T00:31:21.656 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:31:21.656 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-02-23T00:31:21.656 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:31:21.656 INFO:teuthology.orchestra.run.smithi047.stdout:Install 6 Packages 2024-02-23T00:31:21.656 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:31:21.657 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 5.9 M 2024-02-23T00:31:21.657 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 24 M 2024-02-23T00:31:21.657 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-02-23T00:31:21.780 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados-devel-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-23T00:31:22.167 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados-devel-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-23T00:31:22.167 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:31:22.167 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-02-23T00:31:22.167 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:31:22.167 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:31:22.167 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-02-23T00:31:22.222 INFO:teuthology.orchestra.run.smithi047.stdout:(1/6): ceph-mgr-cephadm-16.2.14-550.geb66ed92.e 628 kB/s | 116 kB 00:00 2024-02-23T00:31:22.381 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install libcephfs2 2024-02-23T00:31:22.450 INFO:teuthology.orchestra.run.smithi047.stdout:(2/6): python3-jinja2-2.10.1-4.el8.noarch.rpm 1.4 MB/s | 578 kB 00:00 2024-02-23T00:31:22.473 INFO:teuthology.orchestra.run.smithi047.stdout:(3/6): python3-apipkg-1.5-6.el8.noarch.rpm 85 kB/s | 21 kB 00:00 2024-02-23T00:31:22.614 INFO:teuthology.orchestra.run.smithi047.stdout:(4/6): python3-remoto-1.2.1-1.el8.noarch.rpm 345 kB/s | 49 kB 00:00 2024-02-23T00:31:22.682 INFO:teuthology.orchestra.run.smithi047.stdout:(5/6): python3-babel-2.5.1-7.el8.noarch.rpm 7.4 MB/s | 4.8 MB 00:00 2024-02-23T00:31:22.881 INFO:teuthology.orchestra.run.smithi047.stdout:(6/6): python3-execnet-1.7.1-1.el8.noarch.rpm 825 kB/s | 355 kB 00:00 2024-02-23T00:31:22.884 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:31:22.884 INFO:teuthology.orchestra.run.smithi047.stdout:Total 4.8 MB/s | 5.9 MB 00:01 2024-02-23T00:31:22.939 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-02-23T00:31:22.950 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-02-23T00:31:22.950 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-02-23T00:31:23.071 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-02-23T00:31:23.074 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-02-23T00:31:23.147 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:02:19 ago on Fri 23 Feb 2024 12:29:04 AM UTC. 2024-02-23T00:31:23.520 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-02-23T00:31:23.816 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 1/6 2024-02-23T00:31:24.266 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 2/6 2024-02-23T00:31:24.669 INFO:teuthology.orchestra.run.smithi196.stdout:Package libcephfs2-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-23T00:31:24.715 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-02-23T00:31:24.716 INFO:teuthology.orchestra.run.smithi196.stdout:Nothing to do. 2024-02-23T00:31:24.716 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-02-23T00:31:24.827 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install libcephfs-devel 2024-02-23T00:31:25.286 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-remoto-1.2.1-1.el8.noarch 3/6 2024-02-23T00:31:25.584 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:02:21 ago on Fri 23 Feb 2024 12:29:04 AM UTC. 2024-02-23T00:31:25.648 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 4/6 2024-02-23T00:31:25.861 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jinja2-2.10.1-4.el8.noarch 5/6 2024-02-23T00:31:26.068 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-cephadm-2:16.2.14-550.geb66ed92.el8.noarch 6/6 2024-02-23T00:31:26.369 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.2.14-550.geb66ed92.el8.noarch 6/6 2024-02-23T00:31:26.369 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-cephadm-2:16.2.14-550.geb66ed92.el8.noarch 1/6 2024-02-23T00:31:26.369 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/6 2024-02-23T00:31:26.369 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jinja2-2.10.1-4.el8.noarch 3/6 2024-02-23T00:31:26.370 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 4/6 2024-02-23T00:31:26.370 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 5/6 2024-02-23T00:31:26.775 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-remoto-1.2.1-1.el8.noarch 6/6 2024-02-23T00:31:26.775 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:31:26.775 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-02-23T00:31:26.775 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-cephadm-2:16.2.14-550.geb66ed92.el8.noarch 2024-02-23T00:31:26.775 INFO:teuthology.orchestra.run.smithi047.stdout: python3-apipkg-1.5-6.el8.noarch 2024-02-23T00:31:26.775 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel-2.5.1-7.el8.noarch 2024-02-23T00:31:26.775 INFO:teuthology.orchestra.run.smithi047.stdout: python3-execnet-1.7.1-1.el8.noarch 2024-02-23T00:31:26.775 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2-2.10.1-4.el8.noarch 2024-02-23T00:31:26.776 INFO:teuthology.orchestra.run.smithi047.stdout: python3-remoto-1.2.1-1.el8.noarch 2024-02-23T00:31:26.776 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:31:26.776 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-02-23T00:31:27.053 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-fuse 2024-02-23T00:31:27.127 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-02-23T00:31:27.128 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:31:27.128 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repo Size 2024-02-23T00:31:27.128 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:31:27.129 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-02-23T00:31:27.129 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel x86_64 2:16.2.14-550.geb66ed92.el8 ceph 25 k 2024-02-23T00:31:27.129 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:31:27.129 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-02-23T00:31:27.129 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:31:27.129 INFO:teuthology.orchestra.run.smithi196.stdout:Install 1 Package 2024-02-23T00:31:27.129 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:31:27.129 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 25 k 2024-02-23T00:31:27.129 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 107 k 2024-02-23T00:31:27.129 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-02-23T00:31:27.791 INFO:teuthology.orchestra.run.smithi196.stdout:libcephfs-devel-16.2.14-550.geb66ed92.el8.x86_6 39 kB/s | 25 kB 00:00 2024-02-23T00:31:27.791 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:31:27.792 INFO:teuthology.orchestra.run.smithi196.stdout:Total 38 kB/s | 25 kB 00:00 2024-02-23T00:31:27.792 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-02-23T00:31:27.796 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-02-23T00:31:27.796 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-02-23T00:31:27.798 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-02-23T00:31:27.801 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-02-23T00:31:27.854 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:02:19 ago on Fri 23 Feb 2024 12:29:08 AM UTC. 2024-02-23T00:31:27.855 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-02-23T00:31:28.099 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs-devel-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-23T00:31:28.375 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephfs-devel-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-23T00:31:28.743 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs-devel-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-23T00:31:28.743 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:31:28.743 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-02-23T00:31:28.743 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:31:28.743 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:31:28.743 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-02-23T00:31:28.856 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install librados2 2024-02-23T00:31:29.540 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-02-23T00:31:29.540 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:31:29.540 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-02-23T00:31:29.540 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:31:29.541 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-02-23T00:31:29.541 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse x86_64 2:16.2.14-550.geb66ed92.el8 ceph 911 k 2024-02-23T00:31:29.541 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-02-23T00:31:29.541 INFO:teuthology.orchestra.run.smithi047.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2024-02-23T00:31:29.541 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:31:29.541 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-02-23T00:31:29.541 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:31:29.541 INFO:teuthology.orchestra.run.smithi047.stdout:Install 2 Packages 2024-02-23T00:31:29.541 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:31:29.542 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 994 k 2024-02-23T00:31:29.542 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 2.8 M 2024-02-23T00:31:29.542 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-02-23T00:31:29.613 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:02:25 ago on Fri 23 Feb 2024 12:29:04 AM UTC. 2024-02-23T00:31:29.801 INFO:teuthology.orchestra.run.smithi047.stdout:(1/2): fuse-2.9.7-17.el8.x86_64.rpm 380 kB/s | 83 kB 00:00 2024-02-23T00:31:29.992 INFO:teuthology.orchestra.run.smithi047.stdout:(2/2): ceph-fuse-16.2.14-550.geb66ed92.el8.x86_ 2.2 MB/s | 911 kB 00:00 2024-02-23T00:31:29.993 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:31:29.993 INFO:teuthology.orchestra.run.smithi047.stdout:Total 2.1 MB/s | 994 kB 00:00 2024-02-23T00:31:30.004 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-02-23T00:31:30.026 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-02-23T00:31:30.026 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-02-23T00:31:30.122 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-02-23T00:31:30.125 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-02-23T00:31:30.300 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-02-23T00:31:30.562 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : fuse-2.9.7-17.el8.x86_64 1/2 2024-02-23T00:31:30.768 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-fuse-2:16.2.14-550.geb66ed92.el8.x86_64 2/2 2024-02-23T00:31:31.136 INFO:teuthology.orchestra.run.smithi196.stdout:Package librados2-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-23T00:31:31.181 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-02-23T00:31:31.182 INFO:teuthology.orchestra.run.smithi196.stdout:Nothing to do. 2024-02-23T00:31:31.183 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-02-23T00:31:31.206 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-fuse-2:16.2.14-550.geb66ed92.el8.x86_64 2/2 2024-02-23T00:31:31.207 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-fuse-2:16.2.14-550.geb66ed92.el8.x86_64 1/2 2024-02-23T00:31:31.292 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install librbd1 2024-02-23T00:31:31.600 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 2/2 2024-02-23T00:31:31.600 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:31:31.600 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-02-23T00:31:31.600 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse-2:16.2.14-550.geb66ed92.el8.x86_64 fuse-2.9.7-17.el8.x86_64 2024-02-23T00:31:31.600 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:31:31.600 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-02-23T00:31:31.863 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install librados-devel 2024-02-23T00:31:32.045 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:02:28 ago on Fri 23 Feb 2024 12:29:04 AM UTC. 2024-02-23T00:31:32.668 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:02:24 ago on Fri 23 Feb 2024 12:29:08 AM UTC. 2024-02-23T00:31:33.570 INFO:teuthology.orchestra.run.smithi196.stdout:Package librbd1-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-23T00:31:33.616 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-02-23T00:31:33.617 INFO:teuthology.orchestra.run.smithi196.stdout:Nothing to do. 2024-02-23T00:31:33.617 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-02-23T00:31:33.720 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install python3-rados 2024-02-23T00:31:34.355 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-02-23T00:31:34.355 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:31:34.355 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repo Size 2024-02-23T00:31:34.355 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:31:34.356 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-02-23T00:31:34.356 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel x86_64 2:16.2.14-550.geb66ed92.el8 ceph 131 k 2024-02-23T00:31:34.356 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:31:34.356 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-02-23T00:31:34.356 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:31:34.356 INFO:teuthology.orchestra.run.smithi047.stdout:Install 1 Package 2024-02-23T00:31:34.356 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:31:34.356 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 131 k 2024-02-23T00:31:34.357 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 432 k 2024-02-23T00:31:34.357 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-02-23T00:31:34.474 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:02:30 ago on Fri 23 Feb 2024 12:29:04 AM UTC. 2024-02-23T00:31:34.689 INFO:teuthology.orchestra.run.smithi047.stdout:librados-devel-16.2.14-550.geb66ed92.el8.x86_64 395 kB/s | 131 kB 00:00 2024-02-23T00:31:34.690 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:31:34.690 INFO:teuthology.orchestra.run.smithi047.stdout:Total 393 kB/s | 131 kB 00:00 2024-02-23T00:31:34.690 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-02-23T00:31:34.696 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-02-23T00:31:34.696 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-02-23T00:31:34.753 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-02-23T00:31:34.756 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-02-23T00:31:34.874 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-02-23T00:31:35.091 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librados-devel-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-23T00:31:35.437 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados-devel-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-23T00:31:35.886 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados-devel-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-23T00:31:35.886 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:31:35.886 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-02-23T00:31:35.886 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:31:35.886 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:31:35.886 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-02-23T00:31:35.987 INFO:teuthology.orchestra.run.smithi196.stdout:Package python3-rados-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-23T00:31:36.036 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-02-23T00:31:36.037 INFO:teuthology.orchestra.run.smithi196.stdout:Nothing to do. 2024-02-23T00:31:36.037 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-02-23T00:31:36.153 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install python3-rgw 2024-02-23T00:31:36.186 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install libcephfs2 2024-02-23T00:31:36.905 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:02:32 ago on Fri 23 Feb 2024 12:29:04 AM UTC. 2024-02-23T00:31:36.982 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:02:28 ago on Fri 23 Feb 2024 12:29:08 AM UTC. 2024-02-23T00:31:38.433 INFO:teuthology.orchestra.run.smithi196.stdout:Package python3-rgw-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-23T00:31:38.480 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-02-23T00:31:38.481 INFO:teuthology.orchestra.run.smithi196.stdout:Nothing to do. 2024-02-23T00:31:38.481 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-02-23T00:31:38.601 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install python3-cephfs 2024-02-23T00:31:38.623 INFO:teuthology.orchestra.run.smithi047.stdout:Package libcephfs2-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-23T00:31:38.674 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-02-23T00:31:38.675 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-02-23T00:31:38.675 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-02-23T00:31:38.923 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install libcephfs-devel 2024-02-23T00:31:39.357 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:02:35 ago on Fri 23 Feb 2024 12:29:04 AM UTC. 2024-02-23T00:31:39.712 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:02:31 ago on Fri 23 Feb 2024 12:29:08 AM UTC. 2024-02-23T00:31:40.863 INFO:teuthology.orchestra.run.smithi196.stdout:Package python3-cephfs-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-23T00:31:40.910 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-02-23T00:31:40.911 INFO:teuthology.orchestra.run.smithi196.stdout:Nothing to do. 2024-02-23T00:31:40.911 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-02-23T00:31:41.015 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install python3-rbd 2024-02-23T00:31:41.392 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-02-23T00:31:41.393 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:31:41.393 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repo Size 2024-02-23T00:31:41.393 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:31:41.393 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-02-23T00:31:41.393 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel x86_64 2:16.2.14-550.geb66ed92.el8 ceph 25 k 2024-02-23T00:31:41.393 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:31:41.393 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-02-23T00:31:41.393 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:31:41.393 INFO:teuthology.orchestra.run.smithi047.stdout:Install 1 Package 2024-02-23T00:31:41.394 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:31:41.394 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 25 k 2024-02-23T00:31:41.394 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 107 k 2024-02-23T00:31:41.394 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-02-23T00:31:41.546 INFO:teuthology.orchestra.run.smithi047.stdout:libcephfs-devel-16.2.14-550.geb66ed92.el8.x86_6 168 kB/s | 25 kB 00:00 2024-02-23T00:31:41.546 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:31:41.546 INFO:teuthology.orchestra.run.smithi047.stdout:Total 167 kB/s | 25 kB 00:00 2024-02-23T00:31:41.547 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-02-23T00:31:41.550 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-02-23T00:31:41.550 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-02-23T00:31:41.555 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-02-23T00:31:41.556 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-02-23T00:31:41.610 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-02-23T00:31:41.768 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:02:37 ago on Fri 23 Feb 2024 12:29:04 AM UTC. 2024-02-23T00:31:41.817 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs-devel-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-23T00:31:42.111 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephfs-devel-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-23T00:31:42.479 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs-devel-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-23T00:31:42.479 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:31:42.479 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-02-23T00:31:42.479 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:31:42.479 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:31:42.479 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-02-23T00:31:42.607 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install librados2 2024-02-23T00:31:43.279 INFO:teuthology.orchestra.run.smithi196.stdout:Package python3-rbd-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-23T00:31:43.328 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-02-23T00:31:43.329 INFO:teuthology.orchestra.run.smithi196.stdout:Nothing to do. 2024-02-23T00:31:43.330 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-02-23T00:31:43.393 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:02:35 ago on Fri 23 Feb 2024 12:29:08 AM UTC. 2024-02-23T00:31:43.440 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install rbd-fuse 2024-02-23T00:31:44.210 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:02:40 ago on Fri 23 Feb 2024 12:29:04 AM UTC. 2024-02-23T00:31:45.021 INFO:teuthology.orchestra.run.smithi047.stdout:Package librados2-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-23T00:31:45.071 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-02-23T00:31:45.072 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-02-23T00:31:45.072 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-02-23T00:31:45.191 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install librbd1 2024-02-23T00:31:45.770 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-02-23T00:31:45.770 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:31:45.770 INFO:teuthology.orchestra.run.smithi196.stdout: Package Architecture Version Repository Size 2024-02-23T00:31:45.770 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:31:45.771 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-02-23T00:31:45.771 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse x86_64 2:16.2.14-550.geb66ed92.el8 ceph 88 k 2024-02-23T00:31:45.771 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:31:45.771 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-02-23T00:31:45.771 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:31:45.771 INFO:teuthology.orchestra.run.smithi196.stdout:Install 1 Package 2024-02-23T00:31:45.771 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:31:45.771 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 88 k 2024-02-23T00:31:45.772 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 240 k 2024-02-23T00:31:45.772 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-02-23T00:31:45.981 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:02:37 ago on Fri 23 Feb 2024 12:29:08 AM UTC. 2024-02-23T00:31:46.114 INFO:teuthology.orchestra.run.smithi196.stdout:rbd-fuse-16.2.14-550.geb66ed92.el8.x86_64.rpm 258 kB/s | 88 kB 00:00 2024-02-23T00:31:46.115 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:31:46.115 INFO:teuthology.orchestra.run.smithi196.stdout:Total 257 kB/s | 88 kB 00:00 2024-02-23T00:31:46.115 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-02-23T00:31:46.124 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-02-23T00:31:46.124 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-02-23T00:31:46.175 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-02-23T00:31:46.178 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-02-23T00:31:46.289 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-02-23T00:31:46.500 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-fuse-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-23T00:31:46.783 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: rbd-fuse-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-23T00:31:47.146 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-fuse-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-23T00:31:47.146 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:31:47.146 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-02-23T00:31:47.146 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:31:47.146 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:31:47.147 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-02-23T00:31:47.380 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install rbd-mirror 2024-02-23T00:31:47.609 INFO:teuthology.orchestra.run.smithi047.stdout:Package librbd1-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-23T00:31:47.660 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-02-23T00:31:47.661 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-02-23T00:31:47.661 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-02-23T00:31:47.787 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install python3-rados 2024-02-23T00:31:48.142 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:02:44 ago on Fri 23 Feb 2024 12:29:04 AM UTC. 2024-02-23T00:31:48.571 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:02:40 ago on Fri 23 Feb 2024 12:29:08 AM UTC. 2024-02-23T00:31:49.690 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-02-23T00:31:49.691 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:31:49.691 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repo Size 2024-02-23T00:31:49.691 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:31:49.691 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-02-23T00:31:49.691 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror x86_64 2:16.2.14-550.geb66ed92.el8 ceph 3.7 M 2024-02-23T00:31:49.691 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:31:49.691 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-02-23T00:31:49.692 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:31:49.692 INFO:teuthology.orchestra.run.smithi196.stdout:Install 1 Package 2024-02-23T00:31:49.692 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:31:49.692 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 3.7 M 2024-02-23T00:31:49.692 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 15 M 2024-02-23T00:31:49.692 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-02-23T00:31:50.182 INFO:teuthology.orchestra.run.smithi047.stdout:Package python3-rados-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-23T00:31:50.232 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-02-23T00:31:50.233 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-02-23T00:31:50.234 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-02-23T00:31:50.343 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install python3-rgw 2024-02-23T00:31:50.800 INFO:teuthology.orchestra.run.smithi196.stdout:rbd-mirror-16.2.14-550.geb66ed92.el8.x86_64.rpm 3.3 MB/s | 3.7 MB 00:01 2024-02-23T00:31:50.800 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:31:50.801 INFO:teuthology.orchestra.run.smithi196.stdout:Total 3.3 MB/s | 3.7 MB 00:01 2024-02-23T00:31:50.801 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-02-23T00:31:50.814 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-02-23T00:31:50.814 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-02-23T00:31:50.883 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-02-23T00:31:50.886 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-02-23T00:31:51.134 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:02:43 ago on Fri 23 Feb 2024 12:29:08 AM UTC. 2024-02-23T00:31:51.336 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-02-23T00:31:51.575 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-mirror-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-23T00:31:51.990 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: rbd-mirror-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-23T00:31:52.450 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-mirror-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-23T00:31:52.450 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:31:52.450 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-02-23T00:31:52.450 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:31:52.450 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:31:52.451 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-02-23T00:31:52.700 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install rbd-nbd 2024-02-23T00:31:52.759 INFO:teuthology.orchestra.run.smithi047.stdout:Package python3-rgw-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-23T00:31:52.813 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-02-23T00:31:52.813 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-02-23T00:31:52.814 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-02-23T00:31:52.920 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install python3-cephfs 2024-02-23T00:31:53.466 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:02:49 ago on Fri 23 Feb 2024 12:29:04 AM UTC. 2024-02-23T00:31:53.711 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:02:45 ago on Fri 23 Feb 2024 12:29:08 AM UTC. 2024-02-23T00:31:55.038 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-02-23T00:31:55.039 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:31:55.039 INFO:teuthology.orchestra.run.smithi196.stdout: Package Architecture Version Repository Size 2024-02-23T00:31:55.039 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:31:55.039 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-02-23T00:31:55.039 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd x86_64 2:16.2.14-550.geb66ed92.el8 ceph 175 k 2024-02-23T00:31:55.039 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:31:55.039 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-02-23T00:31:55.039 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-23T00:31:55.039 INFO:teuthology.orchestra.run.smithi196.stdout:Install 1 Package 2024-02-23T00:31:55.039 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:31:55.039 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 175 k 2024-02-23T00:31:55.040 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 523 k 2024-02-23T00:31:55.040 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-02-23T00:31:55.236 INFO:teuthology.orchestra.run.smithi196.stdout:rbd-nbd-16.2.14-550.geb66ed92.el8.x86_64.rpm 892 kB/s | 175 kB 00:00 2024-02-23T00:31:55.237 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:31:55.237 INFO:teuthology.orchestra.run.smithi196.stdout:Total 886 kB/s | 175 kB 00:00 2024-02-23T00:31:55.237 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-02-23T00:31:55.247 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-02-23T00:31:55.247 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-02-23T00:31:55.298 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-02-23T00:31:55.301 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-02-23T00:31:55.340 INFO:teuthology.orchestra.run.smithi047.stdout:Package python3-cephfs-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-23T00:31:55.390 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-02-23T00:31:55.391 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-02-23T00:31:55.391 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-02-23T00:31:55.411 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-02-23T00:31:55.489 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install python3-rbd 2024-02-23T00:31:55.688 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-nbd-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-23T00:31:55.990 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: rbd-nbd-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-23T00:31:56.277 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:02:48 ago on Fri 23 Feb 2024 12:29:08 AM UTC. 2024-02-23T00:31:56.387 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-nbd-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-23T00:31:56.388 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:31:56.388 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-02-23T00:31:56.388 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:31:56.388 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:31:56.388 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-02-23T00:31:56.644 DEBUG:teuthology.parallel:result is None 2024-02-23T00:31:57.893 INFO:teuthology.orchestra.run.smithi047.stdout:Package python3-rbd-2:16.2.14-550.geb66ed92.el8.x86_64 is already installed. 2024-02-23T00:31:57.945 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-02-23T00:31:57.946 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-02-23T00:31:57.946 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-02-23T00:31:58.047 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install rbd-fuse 2024-02-23T00:31:58.834 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:02:50 ago on Fri 23 Feb 2024 12:29:08 AM UTC. 2024-02-23T00:32:00.534 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-02-23T00:32:00.535 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:32:00.535 INFO:teuthology.orchestra.run.smithi047.stdout: Package Architecture Version Repository Size 2024-02-23T00:32:00.535 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:32:00.535 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-02-23T00:32:00.535 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse x86_64 2:16.2.14-550.geb66ed92.el8 ceph 88 k 2024-02-23T00:32:00.535 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:32:00.535 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-02-23T00:32:00.535 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:32:00.535 INFO:teuthology.orchestra.run.smithi047.stdout:Install 1 Package 2024-02-23T00:32:00.535 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:32:00.535 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 88 k 2024-02-23T00:32:00.535 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 240 k 2024-02-23T00:32:00.536 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-02-23T00:32:00.795 INFO:teuthology.orchestra.run.smithi047.stdout:rbd-fuse-16.2.14-550.geb66ed92.el8.x86_64.rpm 342 kB/s | 88 kB 00:00 2024-02-23T00:32:00.796 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:32:00.796 INFO:teuthology.orchestra.run.smithi047.stdout:Total 340 kB/s | 88 kB 00:00 2024-02-23T00:32:00.796 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-02-23T00:32:00.806 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-02-23T00:32:00.806 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-02-23T00:32:00.860 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-02-23T00:32:00.864 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-02-23T00:32:01.002 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-02-23T00:32:01.204 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-fuse-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-23T00:32:01.535 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: rbd-fuse-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-23T00:32:01.925 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-fuse-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-23T00:32:01.926 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:32:01.926 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-02-23T00:32:01.926 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:32:01.926 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:32:01.926 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-02-23T00:32:02.137 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install rbd-mirror 2024-02-23T00:32:02.928 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:02:54 ago on Fri 23 Feb 2024 12:29:08 AM UTC. 2024-02-23T00:32:04.620 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-02-23T00:32:04.621 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:32:04.621 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repo Size 2024-02-23T00:32:04.621 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:32:04.621 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-02-23T00:32:04.621 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror x86_64 2:16.2.14-550.geb66ed92.el8 ceph 3.7 M 2024-02-23T00:32:04.621 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:32:04.622 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-02-23T00:32:04.622 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:32:04.622 INFO:teuthology.orchestra.run.smithi047.stdout:Install 1 Package 2024-02-23T00:32:04.622 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:32:04.622 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 3.7 M 2024-02-23T00:32:04.622 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 15 M 2024-02-23T00:32:04.622 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-02-23T00:32:05.614 INFO:teuthology.orchestra.run.smithi047.stdout:rbd-mirror-16.2.14-550.geb66ed92.el8.x86_64.rpm 3.7 MB/s | 3.7 MB 00:00 2024-02-23T00:32:05.615 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:32:05.615 INFO:teuthology.orchestra.run.smithi047.stdout:Total 3.7 MB/s | 3.7 MB 00:00 2024-02-23T00:32:05.616 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-02-23T00:32:05.630 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-02-23T00:32:05.630 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-02-23T00:32:05.703 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-02-23T00:32:05.707 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-02-23T00:32:06.272 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-02-23T00:32:06.488 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-mirror-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-23T00:32:06.929 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: rbd-mirror-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-23T00:32:07.435 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-mirror-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-23T00:32:07.435 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:32:07.435 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-02-23T00:32:07.435 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:32:07.435 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:32:07.435 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-02-23T00:32:07.674 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install rbd-nbd 2024-02-23T00:32:08.467 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:03:00 ago on Fri 23 Feb 2024 12:29:08 AM UTC. 2024-02-23T00:32:10.136 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-02-23T00:32:10.137 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:32:10.137 INFO:teuthology.orchestra.run.smithi047.stdout: Package Architecture Version Repository Size 2024-02-23T00:32:10.137 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:32:10.138 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-02-23T00:32:10.138 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd x86_64 2:16.2.14-550.geb66ed92.el8 ceph 175 k 2024-02-23T00:32:10.138 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:32:10.138 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-02-23T00:32:10.138 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-02-23T00:32:10.138 INFO:teuthology.orchestra.run.smithi047.stdout:Install 1 Package 2024-02-23T00:32:10.138 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:32:10.138 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 175 k 2024-02-23T00:32:10.138 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 523 k 2024-02-23T00:32:10.138 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-02-23T00:32:10.450 INFO:teuthology.orchestra.run.smithi047.stdout:rbd-nbd-16.2.14-550.geb66ed92.el8.x86_64.rpm 560 kB/s | 175 kB 00:00 2024-02-23T00:32:10.451 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-02-23T00:32:10.451 INFO:teuthology.orchestra.run.smithi047.stdout:Total 558 kB/s | 175 kB 00:00 2024-02-23T00:32:10.451 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-02-23T00:32:10.463 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-02-23T00:32:10.463 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-02-23T00:32:10.517 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-02-23T00:32:10.520 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-02-23T00:32:10.646 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-02-23T00:32:10.868 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-nbd-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-23T00:32:11.182 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: rbd-nbd-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-23T00:32:11.563 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-nbd-2:16.2.14-550.geb66ed92.el8.x86_64 1/1 2024-02-23T00:32:11.564 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:32:11.564 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-02-23T00:32:11.564 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd-2:16.2.14-550.geb66ed92.el8.x86_64 2024-02-23T00:32:11.564 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:32:11.564 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-02-23T00:32:11.800 DEBUG:teuthology.parallel:result is None 2024-02-23T00:32:11.801 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=eb66ed921e744301e6863be6353618d63967ea59 2024-02-23T00:32:12.035 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-23T00:32:12.063 INFO:teuthology.orchestra.run.smithi047.stdout:16.2.14-550.geb66ed92.el8 2024-02-23T00:32:12.064 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-550.geb66ed92.el8 2024-02-23T00:32:12.064 INFO:teuthology.task.install:The correct ceph version 16.2.14-550.geb66ed92 is installed. 2024-02-23T00:32:12.066 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=eb66ed921e744301e6863be6353618d63967ea59 2024-02-23T00:32:12.293 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-23T00:32:12.320 INFO:teuthology.orchestra.run.smithi196.stdout:16.2.14-550.geb66ed92.el8 2024-02-23T00:32:12.320 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-550.geb66ed92.el8 2024-02-23T00:32:12.320 INFO:teuthology.task.install:The correct ceph version 16.2.14-550.geb66ed92 is installed. 2024-02-23T00:32:12.322 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-23T00:32:12.322 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-02-23T00:32:12.322 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-23T00:32:12.353 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-02-23T00:32:12.353 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-23T00:32:12.391 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-23T00:32:12.392 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-02-23T00:32:12.392 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/daemon-helper 2024-02-23T00:32:12.423 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-23T00:32:12.493 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-02-23T00:32:12.493 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/daemon-helper 2024-02-23T00:32:12.523 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-23T00:32:12.591 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-23T00:32:12.592 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-02-23T00:32:12.592 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-23T00:32:12.620 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-23T00:32:12.689 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-02-23T00:32:12.689 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-23T00:32:12.718 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-23T00:32:12.787 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-23T00:32:12.787 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-02-23T00:32:12.787 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/stdin-killer 2024-02-23T00:32:12.816 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-23T00:32:12.885 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-02-23T00:32:12.885 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/stdin-killer 2024-02-23T00:32:12.912 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-23T00:32:12.979 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-23T00:32:13.056 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'ms type': 'async'}, 'mon': {'debug mgrc': 20, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon warn on pool no app': False, 'paxos service trim min': 10}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'crush_tunables': 'jewel', 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', '\\(MON_DOWN\\)', '\\(OSDMAP_FLAGS\\)', 'flag\\(s\\) set', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(PG_', '\\(OSD_', '\\(POOL_', '\\(CEPHADM_STRAY_DAEMON\\)', 'PG_', 'degraded', 'backfill', 'mons down', 'OSD_', 'is down', 'acting', 'reached quota', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)'], 'sha1': 'eb66ed921e744301e6863be6353618d63967ea59', 'cephadm_mode': 'root'} 2024-02-23T00:32:13.056 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 2024-02-23T00:32:13.057 INFO:tasks.cephadm:Cluster fsid is fce2617e-d1e2-11ee-95c0-87774f69a715 2024-02-23T00:32:13.057 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-23T00:32:13.057 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.47', 'mon.c': '[v2:172.21.15.47:3301,v1:172.21.15.47:6790]', 'mon.b': '172.21.15.196'} 2024-02-23T00:32:13.057 INFO:tasks.cephadm:First mon is mon.a on smithi047 2024-02-23T00:32:13.058 INFO:tasks.cephadm:First mgr is y 2024-02-23T00:32:13.058 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-23T00:32:13.058 DEBUG:teuthology.orchestra.run.smithi047:> sudo hostname $(hostname -s) 2024-02-23T00:32:13.087 DEBUG:teuthology.orchestra.run.smithi196:> sudo hostname $(hostname -s) 2024-02-23T00:32:13.115 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph-ci.git ref eb66ed921e744301e6863be6353618d63967ea59)... 2024-02-23T00:32:13.116 DEBUG:teuthology.orchestra.run.smithi047:> curl --silent https://raw.githubusercontent.com/ceph/ceph-ci/eb66ed921e744301e6863be6353618d63967ea59/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-23T00:32:13.445 INFO:teuthology.orchestra.run.smithi047.stdout:-rw-rw-r--. 1 ubuntu ubuntu 355729 Feb 23 00:32 /home/ubuntu/cephtest/cephadm 2024-02-23T00:32:13.446 DEBUG:teuthology.orchestra.run.smithi196:> curl --silent https://raw.githubusercontent.com/ceph/ceph-ci/eb66ed921e744301e6863be6353618d63967ea59/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-23T00:32:13.839 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-rw-r--. 1 ubuntu ubuntu 355729 Feb 23 00:32 /home/ubuntu/cephtest/cephadm 2024-02-23T00:32:13.839 DEBUG:teuthology.orchestra.run.smithi047:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-23T00:32:13.863 DEBUG:teuthology.orchestra.run.smithi196:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-23T00:32:13.928 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /etc/ceph 2024-02-23T00:32:13.961 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph 2024-02-23T00:32:13.993 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 777 /etc/ceph 2024-02-23T00:32:14.034 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 777 /etc/ceph 2024-02-23T00:32:14.065 INFO:tasks.cephadm:Writing seed config... 2024-02-23T00:32:14.066 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-23T00:32:14.066 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-23T00:32:14.066 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-02-23T00:32:14.066 INFO:tasks.cephadm: override: [client] debug objecter = 20 2024-02-23T00:32:14.066 INFO:tasks.cephadm: override: [client] debug rados = 20 2024-02-23T00:32:14.066 INFO:tasks.cephadm: override: [global] ms type = async 2024-02-23T00:32:14.066 INFO:tasks.cephadm: override: [mon] debug mgrc = 20 2024-02-23T00:32:14.066 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-23T00:32:14.066 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-23T00:32:14.066 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-23T00:32:14.066 INFO:tasks.cephadm: override: [mon] mon min osdmap epochs = 50 2024-02-23T00:32:14.066 INFO:tasks.cephadm: override: [mon] mon osdmap full prune interval = 2 2024-02-23T00:32:14.067 INFO:tasks.cephadm: override: [mon] mon osdmap full prune min = 15 2024-02-23T00:32:14.067 INFO:tasks.cephadm: override: [mon] mon osdmap full prune txsize = 2 2024-02-23T00:32:14.067 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-02-23T00:32:14.067 INFO:tasks.cephadm: override: [mon] paxos service trim min = 10 2024-02-23T00:32:14.067 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-23T00:32:14.067 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-23T00:32:14.067 INFO:tasks.cephadm: override: [osd] osd debug reject backfill probability = 0.3 2024-02-23T00:32:14.067 INFO:tasks.cephadm: override: [osd] osd delete sleep = 1 2024-02-23T00:32:14.067 INFO:tasks.cephadm: override: [osd] osd max backfills = 3 2024-02-23T00:32:14.067 INFO:tasks.cephadm: override: [osd] osd scrub max interval = 120 2024-02-23T00:32:14.067 INFO:tasks.cephadm: override: [osd] osd scrub min interval = 60 2024-02-23T00:32:14.067 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-02-23T00:32:14.067 INFO:tasks.cephadm: override: [osd] osd snap trim sleep = 2 2024-02-23T00:32:14.068 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-02-23T00:32:14.068 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-23T00:32:14.095 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = fce2617e-d1e2-11ee-95c0-87774f69a715 ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 120 osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd debug reject backfill probability = 0.3 osd delete sleep = 1 osd max backfills = 3 osd scrub min interval = 60 osd snap trim sleep = 2 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mgrc = 20 debug mon = 20 debug ms = 1 debug paxos = 20 mon min osdmap epochs = 50 mon osdmap full prune interval = 2 mon osdmap full prune min = 15 mon osdmap full prune txsize = 2 mon warn on pool no app = False paxos service trim min = 10 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] debug ms = 1 debug objecter = 20 debug rados = 20 2024-02-23T00:32:14.095 DEBUG:teuthology.orchestra.run.smithi047:mon.a> sudo journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@mon.a.service 2024-02-23T00:32:14.137 DEBUG:teuthology.orchestra.run.smithi047:mgr.y> sudo journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@mgr.y.service 2024-02-23T00:32:14.166 INFO:journalctl@ceph.mon.a.smithi047.stdout:-- Logs begin at Fri 2024-02-23 00:13:17 UTC. -- 2024-02-23T00:32:14.167 INFO:tasks.cephadm:Bootstrapping... 2024-02-23T00:32:14.167 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 -v bootstrap --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.47 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-23T00:32:14.196 INFO:journalctl@ceph.mgr.y.smithi047.stdout:-- Logs begin at Fri 2024-02-23 00:13:17 UTC. -- 2024-02-23T00:32:14.482 INFO:teuthology.orchestra.run.smithi047.stderr:-------------------------------------------------------------------------------- 2024-02-23T00:32:14.483 INFO:teuthology.orchestra.run.smithi047.stderr:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59', '-v', 'bootstrap', '--fsid', 'fce2617e-d1e2-11ee-95c0-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.47', '--skip-admin-label'] 2024-02-23T00:32:15.418 INFO:teuthology.orchestra.run.smithi047.stderr:/bin/podman: 4.9.3-dev 2024-02-23T00:32:15.429 INFO:teuthology.orchestra.run.smithi047.stderr:Verifying podman|docker is present... 2024-02-23T00:32:15.453 INFO:teuthology.orchestra.run.smithi047.stderr:/bin/podman: 4.9.3-dev 2024-02-23T00:32:15.466 INFO:teuthology.orchestra.run.smithi047.stderr:Verifying lvm2 is present... 2024-02-23T00:32:15.466 INFO:teuthology.orchestra.run.smithi047.stderr:Verifying time synchronization is in place... 2024-02-23T00:32:15.474 INFO:teuthology.orchestra.run.smithi047.stderr:systemctl: Failed to get unit file state for chrony.service: No such file or directory 2024-02-23T00:32:15.481 INFO:teuthology.orchestra.run.smithi047.stderr:systemctl: inactive 2024-02-23T00:32:15.488 INFO:teuthology.orchestra.run.smithi047.stderr:systemctl: enabled 2024-02-23T00:32:15.494 INFO:teuthology.orchestra.run.smithi047.stderr:systemctl: active 2024-02-23T00:32:15.495 INFO:teuthology.orchestra.run.smithi047.stderr:Unit chronyd.service is enabled and running 2024-02-23T00:32:15.495 INFO:teuthology.orchestra.run.smithi047.stderr:Repeating the final host check... 2024-02-23T00:32:15.516 INFO:teuthology.orchestra.run.smithi047.stderr:/bin/podman: 4.9.3-dev 2024-02-23T00:32:15.525 INFO:teuthology.orchestra.run.smithi047.stderr:podman (/bin/podman) version 4.9.3 is present 2024-02-23T00:32:15.526 INFO:teuthology.orchestra.run.smithi047.stderr:systemctl is present 2024-02-23T00:32:15.526 INFO:teuthology.orchestra.run.smithi047.stderr:lvcreate is present 2024-02-23T00:32:15.531 INFO:teuthology.orchestra.run.smithi047.stderr:systemctl: Failed to get unit file state for chrony.service: No such file or directory 2024-02-23T00:32:15.537 INFO:teuthology.orchestra.run.smithi047.stderr:systemctl: inactive 2024-02-23T00:32:15.544 INFO:teuthology.orchestra.run.smithi047.stderr:systemctl: enabled 2024-02-23T00:32:15.549 INFO:teuthology.orchestra.run.smithi047.stderr:systemctl: active 2024-02-23T00:32:15.551 INFO:teuthology.orchestra.run.smithi047.stderr:Unit chronyd.service is enabled and running 2024-02-23T00:32:15.551 INFO:teuthology.orchestra.run.smithi047.stderr:Host looks OK 2024-02-23T00:32:15.551 INFO:teuthology.orchestra.run.smithi047.stderr:Cluster fsid: fce2617e-d1e2-11ee-95c0-87774f69a715 2024-02-23T00:32:15.551 INFO:teuthology.orchestra.run.smithi047.stderr:Acquiring lock 140390409128424 on /run/cephadm/fce2617e-d1e2-11ee-95c0-87774f69a715.lock 2024-02-23T00:32:15.551 INFO:teuthology.orchestra.run.smithi047.stderr:Lock 140390409128424 acquired on /run/cephadm/fce2617e-d1e2-11ee-95c0-87774f69a715.lock 2024-02-23T00:32:15.552 INFO:teuthology.orchestra.run.smithi047.stderr:Verifying IP 172.21.15.47 port 3300 ... 2024-02-23T00:32:15.552 INFO:teuthology.orchestra.run.smithi047.stderr:Verifying IP 172.21.15.47 port 6789 ... 2024-02-23T00:32:15.552 INFO:teuthology.orchestra.run.smithi047.stderr:Base mon IP(s) is [172.21.15.47:3300, 172.21.15.47:6789], mon addrv is [v2:172.21.15.47:3300,v1:172.21.15.47:6789] 2024-02-23T00:32:15.555 INFO:teuthology.orchestra.run.smithi047.stderr:/sbin/ip: default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.47 metric 100 2024-02-23T00:32:15.555 INFO:teuthology.orchestra.run.smithi047.stderr:/sbin/ip: 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.47 metric 100 2024-02-23T00:32:15.558 INFO:teuthology.orchestra.run.smithi047.stderr:/sbin/ip: ::1 dev lo proto kernel metric 256 pref medium 2024-02-23T00:32:15.558 INFO:teuthology.orchestra.run.smithi047.stderr:/sbin/ip: fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-02-23T00:32:15.558 INFO:teuthology.orchestra.run.smithi047.stderr:/sbin/ip: default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1790sec hoplimit 64 pref medium 2024-02-23T00:32:15.561 INFO:teuthology.orchestra.run.smithi047.stderr:/sbin/ip: 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-23T00:32:15.561 INFO:teuthology.orchestra.run.smithi047.stderr:/sbin/ip: inet6 ::1/128 scope host 2024-02-23T00:32:15.561 INFO:teuthology.orchestra.run.smithi047.stderr:/sbin/ip: valid_lft forever preferred_lft forever 2024-02-23T00:32:15.561 INFO:teuthology.orchestra.run.smithi047.stderr:/sbin/ip: 4: ens1f0: mtu 1500 state UP qlen 1000 2024-02-23T00:32:15.561 INFO:teuthology.orchestra.run.smithi047.stderr:/sbin/ip: inet6 fe80::ec4:7aff:febd:148e/64 scope link 2024-02-23T00:32:15.562 INFO:teuthology.orchestra.run.smithi047.stderr:/sbin/ip: valid_lft forever preferred_lft forever 2024-02-23T00:32:15.562 INFO:teuthology.orchestra.run.smithi047.stderr:Mon IP `172.21.15.47` is in CIDR network `172.21.0.0/20` 2024-02-23T00:32:15.562 INFO:teuthology.orchestra.run.smithi047.stderr:Mon IP `172.21.15.47` is in CIDR network `172.21.0.0/20` 2024-02-23T00:32:15.562 INFO:teuthology.orchestra.run.smithi047.stderr:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-23T00:32:15.563 INFO:teuthology.orchestra.run.smithi047.stderr:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-23T00:32:15.564 INFO:teuthology.orchestra.run.smithi047.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59... 2024-02-23T00:32:15.666 INFO:teuthology.orchestra.run.smithi047.stderr:/bin/podman: Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59... 2024-02-23T00:32:15.835 INFO:teuthology.orchestra.run.smithi047.stderr:/bin/podman: Getting image source signatures 2024-02-23T00:32:15.843 INFO:teuthology.orchestra.run.smithi047.stderr:/bin/podman: Copying blob sha256:6a725db67f9d8a3d1854323a2987a594424da05f10d8a333e41377f3b137fc12 2024-02-23T00:32:15.843 INFO:teuthology.orchestra.run.smithi047.stderr:/bin/podman: Copying blob sha256:0f642e61a11afb42c0ccb5b05f5809250ad643ed143780d6ff49a5a4c492fc38 2024-02-23T00:32:45.637 INFO:teuthology.orchestra.run.smithi047.stderr:/bin/podman: Copying config sha256:7403d18623076b313def53d9a4ae24b05fb9f62358c436a81c22d7e96d9d9550 2024-02-23T00:32:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:/bin/podman: Writing manifest to image destination 2024-02-23T00:32:46.092 INFO:teuthology.orchestra.run.smithi047.stderr:/bin/podman: 7403d18623076b313def53d9a4ae24b05fb9f62358c436a81c22d7e96d9d9550 2024-02-23T00:32:46.798 INFO:teuthology.orchestra.run.smithi047.stderr:ceph: ceph version 16.2.14-550-geb66ed92 (eb66ed921e744301e6863be6353618d63967ea59) pacific (stable) 2024-02-23T00:32:47.385 INFO:teuthology.orchestra.run.smithi047.stderr:Ceph version: ceph version 16.2.14-550-geb66ed92 (eb66ed921e744301e6863be6353618d63967ea59) pacific (stable) 2024-02-23T00:32:47.385 INFO:teuthology.orchestra.run.smithi047.stderr:Extracting ceph user uid/gid from container image... 2024-02-23T00:32:47.907 INFO:teuthology.orchestra.run.smithi047.stderr:stat: 167 167 2024-02-23T00:32:48.473 INFO:teuthology.orchestra.run.smithi047.stderr:Creating initial keys... 2024-02-23T00:32:49.021 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-authtool: AQCw59dlKq74NBAAssXITBJHrI2Yhw8YOJdU2Q== 2024-02-23T00:32:50.110 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-authtool: AQCy59dlA5ZkARAAJ3LtMos5sih2M+TFASkXrg== 2024-02-23T00:32:51.266 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-authtool: AQCz59dlmkXmCRAACf7bky9zWXt4gVwcFvZzEA== 2024-02-23T00:32:51.869 INFO:teuthology.orchestra.run.smithi047.stderr:Creating initial monmap... 2024-02-23T00:32:52.380 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-23T00:32:52.380 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: set fsid to fce2617e-d1e2-11ee-95c0-87774f69a715 2024-02-23T00:32:52.380 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-23T00:32:53.168 INFO:teuthology.orchestra.run.smithi047.stderr:monmaptool for a [v2:172.21.15.47:3300,v1:172.21.15.47:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-23T00:32:53.168 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/monmaptool: set fsid to fce2617e-d1e2-11ee-95c0-87774f69a715 2024-02-23T00:32:53.168 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-23T00:32:53.168 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-02-23T00:32:53.168 INFO:teuthology.orchestra.run.smithi047.stderr:Creating mon... 2024-02-23T00:32:53.746 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.603+0000 7f57fdd93880 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-23T00:32:53.746 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.604+0000 7f57fdd93880 1 imported monmap: 2024-02-23T00:32:53.746 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: epoch 0 2024-02-23T00:32:53.746 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: fsid fce2617e-d1e2-11ee-95c0-87774f69a715 2024-02-23T00:32:53.746 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: last_changed 2024-02-23T00:32:52.273607+0000 2024-02-23T00:32:53.746 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: created 2024-02-23T00:32:52.273607+0000 2024-02-23T00:32:53.746 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: min_mon_release 0 (unknown) 2024-02-23T00:32:53.746 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: election_strategy: 1 2024-02-23T00:32:53.746 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: 0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-02-23T00:32:53.746 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: 2024-02-23T00:32:53.746 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.604+0000 7f57fdd93880 0 /usr/bin/ceph-mon: set fsid to fce2617e-d1e2-11ee-95c0-87774f69a715 2024-02-23T00:32:53.746 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: RocksDB version: 6.8.1 2024-02-23T00:32:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: 2024-02-23T00:32:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-23T00:32:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Compile date Feb 20 2024 2024-02-23T00:32:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: DB SUMMARY 2024-02-23T00:32:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: 2024-02-23T00:32:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 0, files: 2024-02-23T00:32:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: 2024-02-23T00:32:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 2024-02-23T00:32:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: 2024-02-23T00:32:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.error_if_exists: 0 2024-02-23T00:32:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.create_if_missing: 1 2024-02-23T00:32:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.paranoid_checks: 1 2024-02-23T00:32:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.env: 0x55a581f9a080 2024-02-23T00:32:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.fs: Posix File System 2024-02-23T00:32:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.info_log: 0x55a5835a7a00 2024-02-23T00:32:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-23T00:32:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.statistics: (nil) 2024-02-23T00:32:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.use_fsync: 0 2024-02-23T00:32:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.max_log_file_size: 0 2024-02-23T00:32:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-23T00:32:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-23T00:32:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-23T00:32:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-23T00:32:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.allow_fallocate: 1 2024-02-23T00:32:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-23T00:32:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-23T00:32:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.use_direct_reads: 0 2024-02-23T00:32:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-23T00:32:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.create_missing_column_families: 0 2024-02-23T00:32:53.749 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.db_log_dir: 2024-02-23T00:32:53.749 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-02-23T00:32:53.749 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-23T00:32:53.749 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.max_subcompactions: 1 2024-02-23T00:32:53.749 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.max_background_flushes: -1 2024-02-23T00:32:53.749 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-23T00:32:53.749 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-23T00:32:53.749 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-23T00:32:53.749 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-23T00:32:53.749 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-23T00:32:53.749 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.advise_random_on_open: 1 2024-02-23T00:32:53.749 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-23T00:32:53.749 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.write_buffer_manager: 0x55a5835ad920 2024-02-23T00:32:53.749 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-23T00:32:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-23T00:32:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-23T00:32:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-23T00:32:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.rate_limiter: (nil) 2024-02-23T00:32:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-23T00:32:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-23T00:32:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-23T00:32:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-23T00:32:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.unordered_write: 0 2024-02-23T00:32:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-23T00:32:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-23T00:32:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-23T00:32:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-23T00:32:53.751 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.row_cache: None 2024-02-23T00:32:53.751 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.wal_filter: None 2024-02-23T00:32:53.751 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-23T00:32:53.751 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-23T00:32:53.751 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.preserve_deletes: 0 2024-02-23T00:32:53.751 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.two_write_queues: 0 2024-02-23T00:32:53.751 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.manual_wal_flush: 0 2024-02-23T00:32:53.751 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.atomic_flush: 0 2024-02-23T00:32:53.751 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-23T00:32:53.751 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-23T00:32:53.751 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-23T00:32:53.751 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.log_readahead_size: 0 2024-02-23T00:32:53.751 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-23T00:32:53.751 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.max_background_jobs: 2 2024-02-23T00:32:53.751 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.max_background_compactions: -1 2024-02-23T00:32:53.752 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-23T00:32:53.752 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-23T00:32:53.752 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-23T00:32:53.752 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.max_total_wal_size: 0 2024-02-23T00:32:53.752 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-23T00:32:53.752 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-23T00:32:53.752 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-23T00:32:53.752 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-23T00:32:53.752 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.max_open_files: -1 2024-02-23T00:32:53.752 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.bytes_per_sync: 0 2024-02-23T00:32:53.752 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-23T00:32:53.752 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-23T00:32:53.752 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-23T00:32:53.753 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Compression algorithms supported: 2024-02-23T00:32:53.753 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-23T00:32:53.753 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: kZSTD supported: 0 2024-02-23T00:32:53.753 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: kXpressCompression supported: 0 2024-02-23T00:32:53.753 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-23T00:32:53.753 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: kLZ4Compression supported: 1 2024-02-23T00:32:53.753 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: kBZip2Compression supported: 0 2024-02-23T00:32:53.753 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: kZlibCompression supported: 1 2024-02-23T00:32:53.753 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: kSnappyCompression supported: 1 2024-02-23T00:32:53.753 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-23T00:32:53.753 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: [db_impl/db_impl_open.cc:273] Creating manifest 1 2024-02-23T00:32:53.754 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: 2024-02-23T00:32:53.754 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000001 2024-02-23T00:32:53.754 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: 2024-02-23T00:32:53.754 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-23T00:32:53.754 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: 2024-02-23T00:32:53.754 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-23T00:32:53.754 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.merge_operator: 2024-02-23T00:32:53.754 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.compaction_filter: None 2024-02-23T00:32:53.754 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.compaction_filter_factory: None 2024-02-23T00:32:53.754 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-23T00:32:53.754 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-23T00:32:53.754 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a5834bdd18) 2024-02-23T00:32:53.754 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: cache_index_and_filter_blocks: 1 2024-02-23T00:32:53.754 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-23T00:32:53.755 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-23T00:32:53.755 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: pin_top_level_index_and_filter: 1 2024-02-23T00:32:53.755 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: index_type: 0 2024-02-23T00:32:53.755 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: data_block_index_type: 0 2024-02-23T00:32:53.755 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: index_shortening: 1 2024-02-23T00:32:53.755 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: data_block_hash_table_util_ratio: 0.750000 2024-02-23T00:32:53.755 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: hash_index_allow_collision: 1 2024-02-23T00:32:53.755 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: checksum: 1 2024-02-23T00:32:53.755 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: no_block_cache: 0 2024-02-23T00:32:53.755 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: block_cache: 0x55a5834f2d10 2024-02-23T00:32:53.755 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: block_cache_name: BinnedLRUCache 2024-02-23T00:32:53.755 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: block_cache_options: 2024-02-23T00:32:53.755 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: capacity : 536870912 2024-02-23T00:32:53.755 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: num_shard_bits : 4 2024-02-23T00:32:53.756 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: strict_capacity_limit : 0 2024-02-23T00:32:53.756 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: high_pri_pool_ratio: 0.000 2024-02-23T00:32:53.756 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: block_cache_compressed: (nil) 2024-02-23T00:32:53.756 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: persistent_cache: (nil) 2024-02-23T00:32:53.756 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: block_size: 4096 2024-02-23T00:32:53.756 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: block_size_deviation: 10 2024-02-23T00:32:53.756 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: block_restart_interval: 16 2024-02-23T00:32:53.756 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: index_block_restart_interval: 1 2024-02-23T00:32:53.756 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: metadata_block_size: 4096 2024-02-23T00:32:53.756 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: partition_filters: 0 2024-02-23T00:32:53.756 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: use_delta_encoding: 1 2024-02-23T00:32:53.756 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-23T00:32:53.756 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: whole_key_filtering: 1 2024-02-23T00:32:53.756 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: verify_compression: 0 2024-02-23T00:32:53.756 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: read_amp_bytes_per_bit: 0 2024-02-23T00:32:53.757 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: format_version: 2 2024-02-23T00:32:53.757 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: enable_index_compression: 1 2024-02-23T00:32:53.757 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: block_align: 0 2024-02-23T00:32:53.757 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: 2024-02-23T00:32:53.757 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-23T00:32:53.757 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-23T00:32:53.757 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.compression: NoCompression 2024-02-23T00:32:53.757 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-23T00:32:53.757 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-23T00:32:53.757 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-23T00:32:53.757 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.num_levels: 7 2024-02-23T00:32:53.757 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-23T00:32:53.757 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-23T00:32:53.757 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-23T00:32:53.758 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-23T00:32:53.758 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-23T00:32:53.758 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-23T00:32:53.758 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-23T00:32:53.758 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-23T00:32:53.758 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-23T00:32:53.758 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-23T00:32:53.758 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.compression_opts.level: 32767 2024-02-23T00:32:53.758 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-23T00:32:53.758 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-23T00:32:53.758 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-23T00:32:53.758 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.compression_opts.enabled: false 2024-02-23T00:32:53.758 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-23T00:32:53.758 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-23T00:32:53.758 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-23T00:32:53.759 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-23T00:32:53.759 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-23T00:32:53.759 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-23T00:32:53.759 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-23T00:32:53.759 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-23T00:32:53.759 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-23T00:32:53.759 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-23T00:32:53.759 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-23T00:32:53.759 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-23T00:32:53.759 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-23T00:32:53.759 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-23T00:32:53.759 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-23T00:32:53.759 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-23T00:32:53.759 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-23T00:32:53.759 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.arena_block_size: 4194304 2024-02-23T00:32:53.760 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-23T00:32:53.760 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-23T00:32:53.760 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-23T00:32:53.760 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-23T00:32:53.760 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-23T00:32:53.760 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-23T00:32:53.760 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-23T00:32:53.760 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-23T00:32:53.760 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-23T00:32:53.760 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-23T00:32:53.760 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-23T00:32:53.760 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-23T00:32:53.760 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-23T00:32:53.761 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-23T00:32:53.761 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.table_properties_collectors: 2024-02-23T00:32:53.761 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.inplace_update_support: 0 2024-02-23T00:32:53.761 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-23T00:32:53.761 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-23T00:32:53.761 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-23T00:32:53.761 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-23T00:32:53.761 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.bloom_locality: 0 2024-02-23T00:32:53.761 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.max_successive_merges: 0 2024-02-23T00:32:53.761 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-23T00:32:53.761 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-23T00:32:53.761 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.force_consistency_checks: 0 2024-02-23T00:32:53.761 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-23T00:32:53.761 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.ttl: 2592000 2024-02-23T00:32:53.761 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.605+0000 7f57fdd93880 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-23T00:32:53.762 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.606+0000 7f57fdd93880 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-23T00:32:53.762 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: 2024-02-23T00:32:53.762 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.606+0000 7f57fdd93880 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 0 2024-02-23T00:32:53.762 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: 2024-02-23T00:32:53.762 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.607+0000 7f57fdd93880 4 rocksdb: DB pointer 0x55a5835bb800 2024-02-23T00:32:53.762 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.607+0000 7f57e6c06700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-02-23T00:32:53.762 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.607+0000 7f57e6c06700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-02-23T00:32:53.762 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: ** DB Stats ** 2024-02-23T00:32:53.762 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2024-02-23T00:32:53.762 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-23T00:32:53.762 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-23T00:32:53.762 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-23T00:32:53.762 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-23T00:32:53.763 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-23T00:32:53.763 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-23T00:32:53.763 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: 2024-02-23T00:32:53.763 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-02-23T00:32:53.763 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-23T00:32:53.763 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-23T00:32:53.763 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-23T00:32:53.763 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-23T00:32:53.763 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: 2024-02-23T00:32:53.763 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-02-23T00:32:53.763 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-23T00:32:53.763 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-23T00:32:53.763 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2024-02-23T00:32:53.763 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: Flush(GB): cumulative 0.000, interval 0.000 2024-02-23T00:32:53.763 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-23T00:32:53.764 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: AddFile(Total Files): cumulative 0, interval 0 2024-02-23T00:32:53.764 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: AddFile(L0 Files): cumulative 0, interval 0 2024-02-23T00:32:53.764 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: AddFile(Keys): cumulative 0, interval 0 2024-02-23T00:32:53.764 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-23T00:32:53.764 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-23T00:32:53.764 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-23T00:32:53.764 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: 2024-02-23T00:32:53.764 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: ** File Read Latency Histogram By Level [default] ** 2024-02-23T00:32:53.764 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: 2024-02-23T00:32:53.764 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-02-23T00:32:53.764 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-23T00:32:53.764 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-23T00:32:53.764 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-23T00:32:53.764 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-23T00:32:53.765 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: 2024-02-23T00:32:53.765 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-02-23T00:32:53.765 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-23T00:32:53.765 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-23T00:32:53.765 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2024-02-23T00:32:53.765 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: Flush(GB): cumulative 0.000, interval 0.000 2024-02-23T00:32:53.765 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-23T00:32:53.765 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: AddFile(Total Files): cumulative 0, interval 0 2024-02-23T00:32:53.765 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: AddFile(L0 Files): cumulative 0, interval 0 2024-02-23T00:32:53.765 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: AddFile(Keys): cumulative 0, interval 0 2024-02-23T00:32:53.765 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-23T00:32:53.765 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-23T00:32:53.765 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-23T00:32:53.765 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: 2024-02-23T00:32:53.765 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: ** File Read Latency Histogram By Level [default] ** 2024-02-23T00:32:53.766 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: 2024-02-23T00:32:53.766 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.608+0000 7f57fdd93880 4 rocksdb: [db_impl/db_impl.cc:397] Shutdown: canceling all background work 2024-02-23T00:32:53.766 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.608+0000 7f57fdd93880 4 rocksdb: [db_impl/db_impl.cc:573] Shutdown complete 2024-02-23T00:32:53.766 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph-mon: debug 2024-02-23T00:32:53.608+0000 7f57fdd93880 0 /usr/bin/ceph-mon: created monfs at /var/lib/ceph/mon/ceph-a for mon.a 2024-02-23T00:32:54.298 INFO:teuthology.orchestra.run.smithi047.stderr:create mon.a on 2024-02-23T00:32:54.313 INFO:teuthology.orchestra.run.smithi047.stderr:systemctl: Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-02-23T00:32:54.417 INFO:teuthology.orchestra.run.smithi047.stderr:systemctl: Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-23T00:32:54.515 INFO:teuthology.orchestra.run.smithi047.stderr:systemctl: Created symlink /etc/systemd/system/multi-user.target.wants/ceph-fce2617e-d1e2-11ee-95c0-87774f69a715.target → /etc/systemd/system/ceph-fce2617e-d1e2-11ee-95c0-87774f69a715.target. 2024-02-23T00:32:54.515 INFO:teuthology.orchestra.run.smithi047.stderr:systemctl: Created symlink /etc/systemd/system/ceph.target.wants/ceph-fce2617e-d1e2-11ee-95c0-87774f69a715.target → /etc/systemd/system/ceph-fce2617e-d1e2-11ee-95c0-87774f69a715.target. 2024-02-23T00:32:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:systemctl: Failed to reset failed state of unit ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@mon.a.service: Unit ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@mon.a.service not loaded. 2024-02-23T00:32:54.746 INFO:teuthology.orchestra.run.smithi047.stderr:systemctl: Created symlink /etc/systemd/system/ceph-fce2617e-d1e2-11ee-95c0-87774f69a715.target.wants/ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@mon.a.service → /etc/systemd/system/ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@.service. 2024-02-23T00:32:55.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:32:54 smithi047 systemd[1]: Starting Ceph mon.a for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T00:32:55.430 INFO:teuthology.orchestra.run.smithi047.stderr:systemctl: disabled 2024-02-23T00:32:55.438 INFO:teuthology.orchestra.run.smithi047.stderr:systemctl: inactive 2024-02-23T00:32:55.439 INFO:teuthology.orchestra.run.smithi047.stderr:firewalld.service is not enabled 2024-02-23T00:32:55.439 INFO:teuthology.orchestra.run.smithi047.stderr:Not possible to enable service . firewalld.service is not available 2024-02-23T00:32:55.439 INFO:teuthology.orchestra.run.smithi047.stderr:Waiting for mon to start... 2024-02-23T00:32:55.440 INFO:teuthology.orchestra.run.smithi047.stderr:Waiting for mon... 2024-02-23T00:32:55.688 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:32:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[37793]: cluster 2024-02-23T00:32:55.399555+0000 mon.a (mon.0) 0 : [INF] mkfs fce2617e-d1e2-11ee-95c0-87774f69a715 2024-02-23T00:32:55.688 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:32:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[37793]: cluster 2024-02-23T00:32:55.396074+0000 mon.a (mon.0) 1 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2024-02-23T00:32:55.689 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:32:55 smithi047 bash[37778]: 293dcf78bad484ff0bfb66ae7eec41eebe9dbb2f4aed9188406cb3b510db6010 2024-02-23T00:32:55.689 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:32:55 smithi047 systemd[1]: Started Ceph mon.a for fce2617e-d1e2-11ee-95c0-87774f69a715. 2024-02-23T00:32:56.015 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.012+0000 7f234a5d0700 1 Processor -- start 2024-02-23T00:32:56.015 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.012+0000 7f234a5d0700 1 -- start start 2024-02-23T00:32:56.015 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.012+0000 7f234a5d0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23440713a0 0x7f23440feac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-23T00:32:56.015 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.012+0000 7f234a5d0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23440fefb0 con 0x7f23440713a0 2024-02-23T00:32:56.020 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.013+0000 7f2343fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23440713a0 0x7f23440feac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:32:56.020 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.013+0000 7f2343fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23440713a0 0x7f23440feac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40420/0 (socket says 172.21.15.47:40420) 2024-02-23T00:32:56.020 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.013+0000 7f2343fff700 1 -- 172.21.15.47:0/1441014810 learned_addr learned my addr 172.21.15.47:0/1441014810 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:32:56.020 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.013+0000 7f2343fff700 1 -- 172.21.15.47:0/1441014810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23440ff0f0 con 0x7f23440713a0 2024-02-23T00:32:56.020 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.013+0000 7f2343fff700 1 --2- 172.21.15.47:0/1441014810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23440713a0 0x7f23440feac0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7f2334009730 tx=0x7f23340090f0).ready entity=mon.0 client_cookie=c4ac611a8f87fb63 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:32:56.020 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.014+0000 7f2342ffd700 1 -- 172.21.15.47:0/1441014810 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f233400b610 con 0x7f23440713a0 2024-02-23T00:32:56.020 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.014+0000 7f2342ffd700 1 -- 172.21.15.47:0/1441014810 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f233400b770 con 0x7f23440713a0 2024-02-23T00:32:56.020 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.014+0000 7f2342ffd700 1 -- 172.21.15.47:0/1441014810 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f233400ba20 con 0x7f23440713a0 2024-02-23T00:32:56.020 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.014+0000 7f234a5d0700 1 -- 172.21.15.47:0/1441014810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23440713a0 msgr2=0x7f23440feac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:32:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.014+0000 7f234a5d0700 1 --2- 172.21.15.47:0/1441014810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23440713a0 0x7f23440feac0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7f2334009730 tx=0x7f23340090f0).stop 2024-02-23T00:32:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.014+0000 7f234a5d0700 1 -- 172.21.15.47:0/1441014810 shutdown_connections 2024-02-23T00:32:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.014+0000 7f234a5d0700 1 --2- 172.21.15.47:0/1441014810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23440713a0 0x7f23440feac0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:32:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.014+0000 7f234a5d0700 1 -- 172.21.15.47:0/1441014810 >> 172.21.15.47:0/1441014810 conn(0x7f23440f6540 msgr2=0x7f23440f8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:32:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.014+0000 7f234a5d0700 1 -- 172.21.15.47:0/1441014810 shutdown_connections 2024-02-23T00:32:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.014+0000 7f234a5d0700 1 -- 172.21.15.47:0/1441014810 wait complete. 2024-02-23T00:32:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.015+0000 7f234a5d0700 1 librados: starting msgr at 2024-02-23T00:32:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.015+0000 7f234a5d0700 1 librados: starting objecter 2024-02-23T00:32:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.015+0000 7f234a5d0700 1 Processor -- start 2024-02-23T00:32:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.015+0000 7f234a5d0700 1 -- start start 2024-02-23T00:32:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.015+0000 7f234a5d0700 1 librados: setting wanted keys 2024-02-23T00:32:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.015+0000 7f234a5d0700 1 librados: calling monclient init 2024-02-23T00:32:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.015+0000 7f234a5d0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23440713a0 0x7f234410a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:32:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.015+0000 7f234a5d0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234410a780 con 0x7f23440713a0 2024-02-23T00:32:56.022 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.016+0000 7f2343fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23440713a0 0x7f234410a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:32:56.022 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.016+0000 7f2343fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23440713a0 0x7f234410a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40422/0 (socket says 172.21.15.47:40422) 2024-02-23T00:32:56.022 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.016+0000 7f2343fff700 1 -- 172.21.15.47:0/519285966 learned_addr learned my addr 172.21.15.47:0/519285966 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:32:56.022 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.016+0000 7f2343fff700 1 -- 172.21.15.47:0/519285966 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2334009370 con 0x7f23440713a0 2024-02-23T00:32:56.022 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.016+0000 7f2343fff700 1 --2- 172.21.15.47:0/519285966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23440713a0 0x7f234410a290 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7f2334009130 tx=0x7f2334017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:32:56.022 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.016+0000 7f23417fa700 10 client.?.objecter ms_handle_connect 0x7f23440713a0 2024-02-23T00:32:56.022 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.016+0000 7f23417fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:32:56.022 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.016+0000 7f23417fa700 1 -- 172.21.15.47:0/519285966 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f233400b610 con 0x7f23440713a0 2024-02-23T00:32:56.022 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.016+0000 7f23417fa700 1 -- 172.21.15.47:0/519285966 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f233400be90 con 0x7f23440713a0 2024-02-23T00:32:56.022 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.016+0000 7f23417fa700 1 -- 172.21.15.47:0/519285966 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f233401d570 con 0x7f23440713a0 2024-02-23T00:32:56.022 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.017+0000 7f234a5d0700 1 -- 172.21.15.47:0/519285966 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f234410a980 con 0x7f23440713a0 2024-02-23T00:32:56.022 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.017+0000 7f234a5d0700 10 client.4098.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:32:56.022 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.017+0000 7f234a5d0700 1 -- 172.21.15.47:0/519285966 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f234410ae10 con 0x7f23440713a0 2024-02-23T00:32:56.022 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.017+0000 7f234a5d0700 1 librados: init done 2024-02-23T00:32:56.023 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.017+0000 7f23417fa700 1 -- 172.21.15.47:0/519285966 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f233401d980 con 0x7f23440713a0 2024-02-23T00:32:56.023 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.017+0000 7f23417fa700 1 -- 172.21.15.47:0/519285966 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f2334026690 con 0x7f23440713a0 2024-02-23T00:32:56.023 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.017+0000 7f23417fa700 10 client.4098.objecter ms_dispatch 0x7f2344001550 osd_map(1..1 src has 1..1) v4 2024-02-23T00:32:56.023 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.017+0000 7f23417fa700 3 client.4098.objecter handle_osd_map got epochs [1,1] > 0 2024-02-23T00:32:56.023 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.017+0000 7f23417fa700 3 client.4098.objecter handle_osd_map decoding full epoch 1 2024-02-23T00:32:56.023 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.017+0000 7f23417fa700 20 client.4098.objecter dump_active .. 0 homeless 2024-02-23T00:32:56.023 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.018+0000 7f234a5d0700 1 -- 172.21.15.47:0/519285966 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2344046ce0 con 0x7f23440713a0 2024-02-23T00:32:56.023 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.020+0000 7f23417fa700 1 -- 172.21.15.47:0/519285966 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7f2334021030 con 0x7f23440713a0 2024-02-23T00:32:56.095 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.091+0000 7f234a5d0700 1 -- 172.21.15.47:0/519285966 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f2344103b90 con 0x7f23440713a0 2024-02-23T00:32:56.095 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.091+0000 7f23417fa700 1 -- 172.21.15.47:0/519285966 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f2334026970 con 0x7f23440713a0 2024-02-23T00:32:56.095 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: cluster: 2024-02-23T00:32:56.096 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: id: fce2617e-d1e2-11ee-95c0-87774f69a715 2024-02-23T00:32:56.096 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: health: HEALTH_OK 2024-02-23T00:32:56.096 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.096 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: services: 2024-02-23T00:32:56.096 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: mon: 1 daemons, quorum a (age 0.693858s) 2024-02-23T00:32:56.096 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: mgr: no daemons active 2024-02-23T00:32:56.096 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: osd: 0 osds: 0 up, 0 in 2024-02-23T00:32:56.096 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.096 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: data: 2024-02-23T00:32:56.096 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: pools: 0 pools, 0 pgs 2024-02-23T00:32:56.096 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: objects: 0 objects, 0 B 2024-02-23T00:32:56.096 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: usage: 0 B used, 0 B / 0 B avail 2024-02-23T00:32:56.096 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: pgs: 2024-02-23T00:32:56.096 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.097 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.092+0000 7f234a5d0700 10 librados: watch_flush enter 2024-02-23T00:32:56.097 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.092+0000 7f234a5d0700 10 librados: watch_flush exit 2024-02-23T00:32:56.097 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.092+0000 7f234a5d0700 20 client.4098.objecter shutdown clearing up homeless session... 2024-02-23T00:32:56.097 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.092+0000 7f234a5d0700 10 client.4098.objecter successfully canceled tick 2024-02-23T00:32:56.097 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.092+0000 7f234a5d0700 1 -- 172.21.15.47:0/519285966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23440713a0 msgr2=0x7f234410a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:32:56.097 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.092+0000 7f234a5d0700 1 --2- 172.21.15.47:0/519285966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23440713a0 0x7f234410a290 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7f2334009130 tx=0x7f2334017000).stop 2024-02-23T00:32:56.099 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.092+0000 7f234a5d0700 1 -- 172.21.15.47:0/519285966 shutdown_connections 2024-02-23T00:32:56.099 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.092+0000 7f234a5d0700 1 --2- 172.21.15.47:0/519285966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23440713a0 0x7f234410a290 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:32:56.099 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.092+0000 7f234a5d0700 1 -- 172.21.15.47:0/519285966 >> 172.21.15.47:0/519285966 conn(0x7f23440f6540 msgr2=0x7f23440f80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:32:56.099 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.092+0000 7f234a5d0700 1 -- 172.21.15.47:0/519285966 shutdown_connections 2024-02-23T00:32:56.099 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.092+0000 7f234a5d0700 1 -- 172.21.15.47:0/519285966 wait complete. 2024-02-23T00:32:56.099 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:32:56.092+0000 7f234a5d0700 1 librados: shutdown 2024-02-23T00:32:56.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:32:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[37793]: cluster 2024-02-23T00:32:55.398784+0000 mon.a (mon.0) 2 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2024-02-23T00:32:56.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:32:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[37793]: cluster 2024-02-23T00:32:55.398900+0000 mon.a (mon.0) 3 : cluster [DBG] monmap e1: 1 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0]} removed_ranks: {} 2024-02-23T00:32:56.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:32:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[37793]: cluster 2024-02-23T00:32:55.400647+0000 mon.a (mon.0) 4 : cluster [DBG] fsmap 2024-02-23T00:32:56.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:32:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[37793]: cluster 2024-02-23T00:32:55.402604+0000 mon.a (mon.0) 5 : cluster [DBG] osdmap e1: 0 total, 0 up, 0 in 2024-02-23T00:32:56.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:32:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[37793]: cluster 2024-02-23T00:32:55.402967+0000 mon.a (mon.0) 6 : cluster [DBG] mgrmap e1: no daemons active 2024-02-23T00:32:56.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:32:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[37793]: audit 2024-02-23T00:32:56.092589+0000 mon.a (mon.0) 7 : audit [DBG] from='client.? 172.21.15.47:0/519285966' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-23T00:32:58.675 INFO:teuthology.orchestra.run.smithi047.stderr:mon is available 2024-02-23T00:32:58.675 INFO:teuthology.orchestra.run.smithi047.stderr:Assimilating anything we can from ceph.conf... 2024-02-23T00:33:00.237 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.232+0000 7f1d995b5700 1 Processor -- start 2024-02-23T00:33:00.238 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.232+0000 7f1d995b5700 1 -- start start 2024-02-23T00:33:00.238 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.232+0000 7f1d995b5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d940fbe40 0x7f1d940fc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:00.238 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.232+0000 7f1d995b5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d940fc750 con 0x7f1d940fbe40 2024-02-23T00:33:00.238 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.232+0000 7f1d92ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d940fbe40 0x7f1d940fc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:00.238 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.232+0000 7f1d92ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d940fbe40 0x7f1d940fc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57978/0 (socket says 172.21.15.47:57978) 2024-02-23T00:33:00.238 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.232+0000 7f1d92ffd700 1 -- 172.21.15.47:0/639652550 learned_addr learned my addr 172.21.15.47:0/639652550 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:00.238 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.233+0000 7f1d92ffd700 1 -- 172.21.15.47:0/639652550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d940fc890 con 0x7f1d940fbe40 2024-02-23T00:33:00.238 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.233+0000 7f1d92ffd700 1 --2- 172.21.15.47:0/639652550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d940fbe40 0x7f1d940fc260 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7f1d84009730 tx=0x7f1d840090f0).ready entity=mon.0 client_cookie=2025e3282e31c5d6 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:00.238 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.233+0000 7f1d91ffb700 1 -- 172.21.15.47:0/639652550 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d8400b580 con 0x7f1d940fbe40 2024-02-23T00:33:00.238 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.233+0000 7f1d91ffb700 1 -- 172.21.15.47:0/639652550 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8400b6e0 con 0x7f1d940fbe40 2024-02-23T00:33:00.238 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.233+0000 7f1d91ffb700 1 -- 172.21.15.47:0/639652550 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d8400b990 con 0x7f1d940fbe40 2024-02-23T00:33:00.238 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.233+0000 7f1d995b5700 1 -- 172.21.15.47:0/639652550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d940fbe40 msgr2=0x7f1d940fc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:00.238 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.233+0000 7f1d995b5700 1 --2- 172.21.15.47:0/639652550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d940fbe40 0x7f1d940fc260 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7f1d84009730 tx=0x7f1d840090f0).stop 2024-02-23T00:33:00.238 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.233+0000 7f1d995b5700 1 -- 172.21.15.47:0/639652550 shutdown_connections 2024-02-23T00:33:00.239 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.233+0000 7f1d995b5700 1 --2- 172.21.15.47:0/639652550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d940fbe40 0x7f1d940fc260 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:00.239 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.234+0000 7f1d995b5700 1 -- 172.21.15.47:0/639652550 >> 172.21.15.47:0/639652550 conn(0x7f1d940f78c0 msgr2=0x7f1d940f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:00.239 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.234+0000 7f1d995b5700 1 -- 172.21.15.47:0/639652550 shutdown_connections 2024-02-23T00:33:00.239 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.234+0000 7f1d995b5700 1 -- 172.21.15.47:0/639652550 wait complete. 2024-02-23T00:33:00.239 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.234+0000 7f1d995b5700 1 librados: starting msgr at 2024-02-23T00:33:00.239 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.234+0000 7f1d995b5700 1 librados: starting objecter 2024-02-23T00:33:00.239 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.234+0000 7f1d995b5700 1 Processor -- start 2024-02-23T00:33:00.239 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.234+0000 7f1d995b5700 1 -- start start 2024-02-23T00:33:00.239 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.234+0000 7f1d995b5700 1 librados: setting wanted keys 2024-02-23T00:33:00.239 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.234+0000 7f1d995b5700 1 librados: calling monclient init 2024-02-23T00:33:00.239 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.235+0000 7f1d995b5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d940fbe40 0x7f1d94073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:00.240 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.235+0000 7f1d995b5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d94071a10 con 0x7f1d940fbe40 2024-02-23T00:33:00.240 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.235+0000 7f1d92ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d940fbe40 0x7f1d94073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:00.240 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.235+0000 7f1d92ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d940fbe40 0x7f1d94073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57988/0 (socket says 172.21.15.47:57988) 2024-02-23T00:33:00.240 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.235+0000 7f1d92ffd700 1 -- 172.21.15.47:0/3868787936 learned_addr learned my addr 172.21.15.47:0/3868787936 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:00.240 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.235+0000 7f1d92ffd700 1 -- 172.21.15.47:0/3868787936 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d84009370 con 0x7f1d940fbe40 2024-02-23T00:33:00.241 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.235+0000 7f1d92ffd700 1 --2- 172.21.15.47:0/3868787936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d940fbe40 0x7f1d94073590 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f1d84009130 tx=0x7f1d84017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:00.241 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.235+0000 7f1d7bfff700 10 client.?.objecter ms_handle_connect 0x7f1d940fbe40 2024-02-23T00:33:00.241 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.235+0000 7f1d7bfff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:33:00.241 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.235+0000 7f1d7bfff700 1 -- 172.21.15.47:0/3868787936 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d84011cd0 con 0x7f1d940fbe40 2024-02-23T00:33:00.241 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.235+0000 7f1d7bfff700 1 -- 172.21.15.47:0/3868787936 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d84011e30 con 0x7f1d940fbe40 2024-02-23T00:33:00.241 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.235+0000 7f1d995b5700 1 -- 172.21.15.47:0/3868787936 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d94071bb0 con 0x7f1d940fbe40 2024-02-23T00:33:00.241 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.235+0000 7f1d7bfff700 1 -- 172.21.15.47:0/3868787936 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d8401d570 con 0x7f1d940fbe40 2024-02-23T00:33:00.241 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.235+0000 7f1d995b5700 10 client.4100.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:33:00.241 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.235+0000 7f1d995b5700 1 -- 172.21.15.47:0/3868787936 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d94072040 con 0x7f1d940fbe40 2024-02-23T00:33:00.242 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.235+0000 7f1d995b5700 1 librados: init done 2024-02-23T00:33:00.242 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.236+0000 7f1d7bfff700 1 -- 172.21.15.47:0/3868787936 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f1d8401da00 con 0x7f1d940fbe40 2024-02-23T00:33:00.242 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.236+0000 7f1d7bfff700 1 -- 172.21.15.47:0/3868787936 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f1d84026710 con 0x7f1d940fbe40 2024-02-23T00:33:00.242 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.236+0000 7f1d7bfff700 10 client.4100.objecter ms_dispatch 0x7f1d94001550 osd_map(1..1 src has 1..1) v4 2024-02-23T00:33:00.242 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.236+0000 7f1d7bfff700 3 client.4100.objecter handle_osd_map got epochs [1,1] > 0 2024-02-23T00:33:00.242 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.236+0000 7f1d7bfff700 3 client.4100.objecter handle_osd_map decoding full epoch 1 2024-02-23T00:33:00.242 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.236+0000 7f1d7bfff700 20 client.4100.objecter dump_active .. 0 homeless 2024-02-23T00:33:00.242 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.237+0000 7f1d995b5700 1 -- 172.21.15.47:0/3868787936 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d9405a310 con 0x7f1d940fbe40 2024-02-23T00:33:00.242 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.239+0000 7f1d7bfff700 1 -- 172.21.15.47:0/3868787936 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7f1d84021030 con 0x7f1d940fbe40 2024-02-23T00:33:00.308 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.306+0000 7f1d995b5700 1 -- 172.21.15.47:0/3868787936 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f1d94046ce0 con 0x7f1d940fbe40 2024-02-23T00:33:00.314 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.309+0000 7f1d7bfff700 1 -- 172.21.15.47:0/3868787936 <== mon.0 v2:172.21.15.47:3300/0 7 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7f1d8401de90 con 0x7f1d940fbe40 2024-02-23T00:33:00.314 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.309+0000 7f1d7bfff700 1 -- 172.21.15.47:0/3868787936 <== mon.0 v2:172.21.15.47:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+363 (secure 0 0 0) 0x7f1d840269f0 con 0x7f1d940fbe40 2024-02-23T00:33:00.314 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.314 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: [global] 2024-02-23T00:33:00.314 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: fsid = fce2617e-d1e2-11ee-95c0-87774f69a715 2024-02-23T00:33:00.314 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: log_to_journald = false 2024-02-23T00:33:00.314 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: mon_host = [v2:172.21.15.47:3300,v1:172.21.15.47:6789] 2024-02-23T00:33:00.314 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: mon_osd_allow_pg_remap = true 2024-02-23T00:33:00.314 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: mon_osd_allow_primary_affinity = true 2024-02-23T00:33:00.314 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: mon_warn_on_no_sortbitwise = false 2024-02-23T00:33:00.315 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: osd_crush_chooseleaf_type = 0 2024-02-23T00:33:00.315 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.315 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: [mgr] 2024-02-23T00:33:00.315 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: mgr/telemetry/nag = false 2024-02-23T00:33:00.315 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.315 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: [osd] 2024-02-23T00:33:00.315 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: osd_map_max_advance = 10 2024-02-23T00:33:00.315 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: osd_sloppy_crc = true 2024-02-23T00:33:00.315 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.310+0000 7f1d995b5700 10 librados: watch_flush enter 2024-02-23T00:33:00.315 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.310+0000 7f1d995b5700 10 librados: watch_flush exit 2024-02-23T00:33:00.315 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.310+0000 7f1d995b5700 20 client.4100.objecter shutdown clearing up homeless session... 2024-02-23T00:33:00.315 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.310+0000 7f1d995b5700 10 client.4100.objecter successfully canceled tick 2024-02-23T00:33:00.315 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.310+0000 7f1d995b5700 1 -- 172.21.15.47:0/3868787936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d940fbe40 msgr2=0x7f1d94073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:00.315 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.310+0000 7f1d995b5700 1 --2- 172.21.15.47:0/3868787936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d940fbe40 0x7f1d94073590 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f1d84009130 tx=0x7f1d84017000).stop 2024-02-23T00:33:00.315 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.310+0000 7f1d995b5700 1 -- 172.21.15.47:0/3868787936 shutdown_connections 2024-02-23T00:33:00.316 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.310+0000 7f1d995b5700 1 --2- 172.21.15.47:0/3868787936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d940fbe40 0x7f1d94073590 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:00.316 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.310+0000 7f1d995b5700 1 -- 172.21.15.47:0/3868787936 >> 172.21.15.47:0/3868787936 conn(0x7f1d940f78c0 msgr2=0x7f1d940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:00.316 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.310+0000 7f1d995b5700 1 -- 172.21.15.47:0/3868787936 shutdown_connections 2024-02-23T00:33:00.316 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.310+0000 7f1d995b5700 1 -- 172.21.15.47:0/3868787936 wait complete. 2024-02-23T00:33:00.316 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:00.310+0000 7f1d995b5700 1 librados: shutdown 2024-02-23T00:33:00.709 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[37793]: audit 2024-02-23T00:33:00.307848+0000 mon.a (mon.0) 8 : audit [INF] from='client.? 172.21.15.47:0/3868787936' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-23T00:33:00.710 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[37793]: audit 2024-02-23T00:33:00.310760+0000 mon.a (mon.0) 9 : audit [INF] from='client.? 172.21.15.47:0/3868787936' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-23T00:33:01.005 INFO:teuthology.orchestra.run.smithi047.stderr:Generating new minimal ceph.conf... 2024-02-23T00:33:01.620 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.614+0000 7fb40b981700 1 Processor -- start 2024-02-23T00:33:01.620 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.615+0000 7fb40b981700 1 -- start start 2024-02-23T00:33:01.620 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.615+0000 7fb40b981700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4040fadc0 0x7fb4040fb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:01.620 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.615+0000 7fb40b981700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4040fb6d0 con 0x7fb4040fadc0 2024-02-23T00:33:01.620 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.615+0000 7fb40971d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4040fadc0 0x7fb4040fb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:01.620 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.615+0000 7fb40971d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4040fadc0 0x7fb4040fb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58002/0 (socket says 172.21.15.47:58002) 2024-02-23T00:33:01.620 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.615+0000 7fb40971d700 1 -- 172.21.15.47:0/64353474 learned_addr learned my addr 172.21.15.47:0/64353474 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:01.620 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.615+0000 7fb40971d700 1 -- 172.21.15.47:0/64353474 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4040fb810 con 0x7fb4040fadc0 2024-02-23T00:33:01.620 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.616+0000 7fb40971d700 1 --2- 172.21.15.47:0/64353474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4040fadc0 0x7fb4040fb1e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7fb400009730 tx=0x7fb4000090f0).ready entity=mon.0 client_cookie=a30de6b9bf185290 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:01.620 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.616+0000 7fb3f7fff700 1 -- 172.21.15.47:0/64353474 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb40000b580 con 0x7fb4040fadc0 2024-02-23T00:33:01.620 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.616+0000 7fb3f7fff700 1 -- 172.21.15.47:0/64353474 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7fb40000bb50 con 0x7fb4040fadc0 2024-02-23T00:33:01.621 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.616+0000 7fb3f7fff700 1 -- 172.21.15.47:0/64353474 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb400011ab0 con 0x7fb4040fadc0 2024-02-23T00:33:01.621 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.616+0000 7fb40b981700 1 -- 172.21.15.47:0/64353474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4040fadc0 msgr2=0x7fb4040fb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:01.621 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.616+0000 7fb40b981700 1 --2- 172.21.15.47:0/64353474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4040fadc0 0x7fb4040fb1e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7fb400009730 tx=0x7fb4000090f0).stop 2024-02-23T00:33:01.621 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.616+0000 7fb40b981700 1 -- 172.21.15.47:0/64353474 shutdown_connections 2024-02-23T00:33:01.621 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.616+0000 7fb40b981700 1 --2- 172.21.15.47:0/64353474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4040fadc0 0x7fb4040fb1e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:01.621 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.616+0000 7fb40b981700 1 -- 172.21.15.47:0/64353474 >> 172.21.15.47:0/64353474 conn(0x7fb4040f6540 msgr2=0x7fb4040f8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:01.621 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.617+0000 7fb40b981700 1 -- 172.21.15.47:0/64353474 shutdown_connections 2024-02-23T00:33:01.621 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.617+0000 7fb40b981700 1 -- 172.21.15.47:0/64353474 wait complete. 2024-02-23T00:33:01.621 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.617+0000 7fb40b981700 1 librados: starting msgr at 2024-02-23T00:33:01.621 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.617+0000 7fb40b981700 1 librados: starting objecter 2024-02-23T00:33:01.621 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.617+0000 7fb40b981700 1 Processor -- start 2024-02-23T00:33:01.621 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.617+0000 7fb40b981700 1 -- start start 2024-02-23T00:33:01.621 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.617+0000 7fb40b981700 1 librados: setting wanted keys 2024-02-23T00:33:01.621 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.617+0000 7fb40b981700 1 librados: calling monclient init 2024-02-23T00:33:01.621 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.618+0000 7fb40b981700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4040fadc0 0x7fb404101070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:01.622 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.618+0000 7fb40b981700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4040ff4f0 con 0x7fb4040fadc0 2024-02-23T00:33:01.622 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.618+0000 7fb40971d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4040fadc0 0x7fb404101070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:01.622 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.618+0000 7fb40971d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4040fadc0 0x7fb404101070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58016/0 (socket says 172.21.15.47:58016) 2024-02-23T00:33:01.622 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.618+0000 7fb40971d700 1 -- 172.21.15.47:0/1981070344 learned_addr learned my addr 172.21.15.47:0/1981070344 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:01.627 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.618+0000 7fb40971d700 1 -- 172.21.15.47:0/1981070344 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb400009370 con 0x7fb4040fadc0 2024-02-23T00:33:01.627 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.618+0000 7fb40971d700 1 --2- 172.21.15.47:0/1981070344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4040fadc0 0x7fb404101070 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fb40000bfa0 tx=0x7fb4000090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:01.627 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.618+0000 7fb3f67fc700 10 client.?.objecter ms_handle_connect 0x7fb4040fadc0 2024-02-23T00:33:01.627 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.618+0000 7fb3f67fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:33:01.628 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.618+0000 7fb3f67fc700 1 -- 172.21.15.47:0/1981070344 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb400011a70 con 0x7fb4040fadc0 2024-02-23T00:33:01.628 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.618+0000 7fb40b981700 1 -- 172.21.15.47:0/1981070344 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4040ff690 con 0x7fb4040fadc0 2024-02-23T00:33:01.628 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.618+0000 7fb3f67fc700 1 -- 172.21.15.47:0/1981070344 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7fb40001aab0 con 0x7fb4040fadc0 2024-02-23T00:33:01.628 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.618+0000 7fb3f67fc700 1 -- 172.21.15.47:0/1981070344 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4000239a0 con 0x7fb4040fadc0 2024-02-23T00:33:01.628 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.618+0000 7fb40b981700 10 client.4102.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:33:01.628 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.618+0000 7fb40b981700 1 -- 172.21.15.47:0/1981070344 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4040ffb20 con 0x7fb4040fadc0 2024-02-23T00:33:01.628 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.618+0000 7fb40b981700 1 librados: init done 2024-02-23T00:33:01.628 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.619+0000 7fb3f67fc700 1 -- 172.21.15.47:0/1981070344 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7fb40002c3f0 con 0x7fb4040fadc0 2024-02-23T00:33:01.628 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.619+0000 7fb3f67fc700 1 -- 172.21.15.47:0/1981070344 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7fb400022730 con 0x7fb4040fadc0 2024-02-23T00:33:01.628 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.619+0000 7fb3f67fc700 10 client.4102.objecter ms_dispatch 0x7fb404001550 osd_map(1..1 src has 1..1) v4 2024-02-23T00:33:01.628 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.619+0000 7fb3f67fc700 3 client.4102.objecter handle_osd_map got epochs [1,1] > 0 2024-02-23T00:33:01.628 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.619+0000 7fb3f67fc700 3 client.4102.objecter handle_osd_map decoding full epoch 1 2024-02-23T00:33:01.628 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.619+0000 7fb3f67fc700 20 client.4102.objecter dump_active .. 0 homeless 2024-02-23T00:33:01.628 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.620+0000 7fb40b981700 1 -- 172.21.15.47:0/1981070344 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb40405a310 con 0x7fb4040fadc0 2024-02-23T00:33:01.629 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.622+0000 7fb3f67fc700 1 -- 172.21.15.47:0/1981070344 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7fb40001e030 con 0x7fb4040fadc0 2024-02-23T00:33:01.692 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.690+0000 7fb40b981700 1 -- 172.21.15.47:0/1981070344 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fb404104bd0 con 0x7fb4040fadc0 2024-02-23T00:33:01.692 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.690+0000 7fb3f67fc700 1 -- 172.21.15.47:0/1981070344 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7fb40001ac20 con 0x7fb4040fadc0 2024-02-23T00:33:01.693 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.691+0000 7fb40b981700 10 librados: watch_flush enter 2024-02-23T00:33:01.694 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.691+0000 7fb40b981700 10 librados: watch_flush exit 2024-02-23T00:33:01.694 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.691+0000 7fb40b981700 20 client.4102.objecter shutdown clearing up homeless session... 2024-02-23T00:33:01.694 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.691+0000 7fb40b981700 10 client.4102.objecter successfully canceled tick 2024-02-23T00:33:01.694 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.691+0000 7fb40b981700 1 -- 172.21.15.47:0/1981070344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4040fadc0 msgr2=0x7fb404101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:01.694 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.691+0000 7fb40b981700 1 --2- 172.21.15.47:0/1981070344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4040fadc0 0x7fb404101070 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fb40000bfa0 tx=0x7fb4000090f0).stop 2024-02-23T00:33:01.694 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.691+0000 7fb40b981700 1 -- 172.21.15.47:0/1981070344 shutdown_connections 2024-02-23T00:33:01.694 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.691+0000 7fb40b981700 1 --2- 172.21.15.47:0/1981070344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4040fadc0 0x7fb404101070 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:01.694 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.691+0000 7fb40b981700 1 -- 172.21.15.47:0/1981070344 >> 172.21.15.47:0/1981070344 conn(0x7fb4040f6540 msgr2=0x7fb404071010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:01.694 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.691+0000 7fb40b981700 1 -- 172.21.15.47:0/1981070344 shutdown_connections 2024-02-23T00:33:01.694 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.691+0000 7fb40b981700 1 -- 172.21.15.47:0/1981070344 wait complete. 2024-02-23T00:33:01.694 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:01.691+0000 7fb40b981700 1 librados: shutdown 2024-02-23T00:33:02.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[37793]: audit 2024-02-23T00:33:01.691592+0000 mon.a (mon.0) 10 : audit [DBG] from='client.? 172.21.15.47:0/1981070344' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:33:02.357 INFO:teuthology.orchestra.run.smithi047.stderr:Restarting the monitor... 2024-02-23T00:33:02.619 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:02 smithi047 systemd[1]: Stopping Ceph mon.a for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T00:33:02.619 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:02 smithi047 bash[38129]: Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon.a" found: no such container 2024-02-23T00:33:02.870 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[37793]: debug 2024-02-23T00:33:02.616+0000 7fc75a68c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-23T00:33:02.870 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[37793]: debug 2024-02-23T00:33:02.616+0000 7fc75a68c700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-23T00:33:03.178 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:02 smithi047 bash[38169]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a 2024-02-23T00:33:03.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 systemd[1]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@mon.a.service: Succeeded. 2024-02-23T00:33:03.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 systemd[1]: Stopped Ceph mon.a for fce2617e-d1e2-11ee-95c0-87774f69a715. 2024-02-23T00:33:03.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 systemd[1]: Starting Ceph mon.a for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T00:33:03.954 INFO:teuthology.orchestra.run.smithi047.stderr:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-23T00:33:04.151 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.866+0000 7f3ca06fa880 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-23T00:33:04.152 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.866+0000 7f3ca06fa880 0 ceph version 16.2.14-550-geb66ed92 (eb66ed921e744301e6863be6353618d63967ea59) pacific (stable), process ceph-mon, pid 7 2024-02-23T00:33:04.152 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.866+0000 7f3ca06fa880 0 pidfile_write: ignore empty --pid-file 2024-02-23T00:33:04.152 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.870+0000 7f3ca06fa880 0 load: jerasure load: lrc load: isa 2024-02-23T00:33:04.152 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: RocksDB version: 6.8.1 2024-02-23T00:33:04.152 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: 2024-02-23T00:33:04.152 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-23T00:33:04.152 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Compile date Feb 20 2024 2024-02-23T00:33:04.152 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: DB SUMMARY 2024-02-23T00:33:04.152 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: 2024-02-23T00:33:04.152 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: CURRENT file: CURRENT 2024-02-23T00:33:04.152 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: 2024-02-23T00:33:04.152 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: IDENTITY file: IDENTITY 2024-02-23T00:33:04.152 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: 2024-02-23T00:33:04.153 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: MANIFEST file: MANIFEST-000005 size: 131 Bytes 2024-02-23T00:33:04.153 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: 2024-02-23T00:33:04.153 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000004.sst 2024-02-23T00:33:04.153 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: 2024-02-23T00:33:04.153 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000006.log size: 93697 ; 2024-02-23T00:33:04.153 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: 2024-02-23T00:33:04.153 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.error_if_exists: 0 2024-02-23T00:33:04.153 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.create_if_missing: 0 2024-02-23T00:33:04.153 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.paranoid_checks: 1 2024-02-23T00:33:04.153 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.env: 0x5588f1095080 2024-02-23T00:33:04.153 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.fs: Posix File System 2024-02-23T00:33:04.154 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.info_log: 0x5588f345a260 2024-02-23T00:33:04.154 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-23T00:33:04.154 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.statistics: (nil) 2024-02-23T00:33:04.154 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.use_fsync: 0 2024-02-23T00:33:04.154 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.max_log_file_size: 0 2024-02-23T00:33:04.154 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-23T00:33:04.154 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-23T00:33:04.154 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-23T00:33:04.154 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-23T00:33:04.154 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.allow_fallocate: 1 2024-02-23T00:33:04.154 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-23T00:33:04.154 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-23T00:33:04.154 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.use_direct_reads: 0 2024-02-23T00:33:04.154 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-23T00:33:04.155 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.create_missing_column_families: 0 2024-02-23T00:33:04.155 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.db_log_dir: 2024-02-23T00:33:04.155 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-02-23T00:33:04.155 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-23T00:33:04.155 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.max_subcompactions: 1 2024-02-23T00:33:04.155 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.max_background_flushes: -1 2024-02-23T00:33:04.155 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-23T00:33:04.155 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-23T00:33:04.155 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-23T00:33:04.155 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-23T00:33:04.155 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-23T00:33:04.156 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.advise_random_on_open: 1 2024-02-23T00:33:04.156 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-23T00:33:04.156 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.write_buffer_manager: 0x5588f414c3c0 2024-02-23T00:33:04.156 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-23T00:33:04.156 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-23T00:33:04.156 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-23T00:33:04.156 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-23T00:33:04.156 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.rate_limiter: (nil) 2024-02-23T00:33:04.157 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-23T00:33:04.157 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-23T00:33:04.157 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-23T00:33:04.157 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-23T00:33:04.157 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.unordered_write: 0 2024-02-23T00:33:04.157 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-23T00:33:04.157 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-23T00:33:04.157 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-23T00:33:04.157 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-23T00:33:04.157 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.row_cache: None 2024-02-23T00:33:04.157 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.wal_filter: None 2024-02-23T00:33:04.158 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-23T00:33:04.158 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-23T00:33:04.158 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.preserve_deletes: 0 2024-02-23T00:33:04.158 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.two_write_queues: 0 2024-02-23T00:33:04.158 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.manual_wal_flush: 0 2024-02-23T00:33:04.158 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.atomic_flush: 0 2024-02-23T00:33:04.158 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-23T00:33:04.158 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-23T00:33:04.158 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-23T00:33:04.158 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.log_readahead_size: 0 2024-02-23T00:33:04.158 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-23T00:33:04.158 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.max_background_jobs: 2 2024-02-23T00:33:04.158 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.max_background_compactions: -1 2024-02-23T00:33:04.158 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-23T00:33:04.159 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-23T00:33:04.159 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-23T00:33:04.159 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.max_total_wal_size: 0 2024-02-23T00:33:04.159 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-23T00:33:04.159 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-23T00:33:04.160 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-23T00:33:04.161 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-23T00:33:04.161 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.max_open_files: -1 2024-02-23T00:33:04.161 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.bytes_per_sync: 0 2024-02-23T00:33:04.161 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-23T00:33:04.161 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-23T00:33:04.161 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-23T00:33:04.161 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Compression algorithms supported: 2024-02-23T00:33:04.161 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-23T00:33:04.161 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: kZSTD supported: 0 2024-02-23T00:33:04.161 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: kXpressCompression supported: 0 2024-02-23T00:33:04.161 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-23T00:33:04.161 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: kLZ4Compression supported: 1 2024-02-23T00:33:04.161 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: kBZip2Compression supported: 0 2024-02-23T00:33:04.162 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: kZlibCompression supported: 1 2024-02-23T00:33:04.162 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: kSnappyCompression supported: 1 2024-02-23T00:33:04.162 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-23T00:33:04.162 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000005 2024-02-23T00:33:04.162 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: 2024-02-23T00:33:04.162 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-23T00:33:04.162 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: 2024-02-23T00:33:04.162 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-23T00:33:04.162 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.merge_operator: 2024-02-23T00:33:04.162 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.compaction_filter: None 2024-02-23T00:33:04.162 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.compaction_filter_factory: None 2024-02-23T00:33:04.162 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-23T00:33:04.162 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-23T00:33:04.162 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5588f33ffd10) 2024-02-23T00:33:04.162 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cache_index_and_filter_blocks: 1 2024-02-23T00:33:04.163 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-23T00:33:04.163 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-23T00:33:04.163 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: pin_top_level_index_and_filter: 1 2024-02-23T00:33:04.163 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: index_type: 0 2024-02-23T00:33:04.163 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: data_block_index_type: 0 2024-02-23T00:33:04.163 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: index_shortening: 1 2024-02-23T00:33:04.163 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: data_block_hash_table_util_ratio: 0.750000 2024-02-23T00:33:04.163 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: hash_index_allow_collision: 1 2024-02-23T00:33:04.163 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: checksum: 1 2024-02-23T00:33:04.163 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: no_block_cache: 0 2024-02-23T00:33:04.163 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: block_cache: 0x5588f3434f10 2024-02-23T00:33:04.163 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: block_cache_name: BinnedLRUCache 2024-02-23T00:33:04.163 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: block_cache_options: 2024-02-23T00:33:04.163 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: capacity : 536870912 2024-02-23T00:33:04.164 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: num_shard_bits : 4 2024-02-23T00:33:04.164 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: strict_capacity_limit : 0 2024-02-23T00:33:04.164 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: high_pri_pool_ratio: 0.000 2024-02-23T00:33:04.164 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: block_cache_compressed: (nil) 2024-02-23T00:33:04.164 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: persistent_cache: (nil) 2024-02-23T00:33:04.164 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: block_size: 4096 2024-02-23T00:33:04.164 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: block_size_deviation: 10 2024-02-23T00:33:04.164 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: block_restart_interval: 16 2024-02-23T00:33:04.164 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: index_block_restart_interval: 1 2024-02-23T00:33:04.164 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: metadata_block_size: 4096 2024-02-23T00:33:04.164 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: partition_filters: 0 2024-02-23T00:33:04.164 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: use_delta_encoding: 1 2024-02-23T00:33:04.164 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-23T00:33:04.164 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: whole_key_filtering: 1 2024-02-23T00:33:04.165 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: verify_compression: 0 2024-02-23T00:33:04.165 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: read_amp_bytes_per_bit: 0 2024-02-23T00:33:04.165 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: format_version: 2 2024-02-23T00:33:04.165 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: enable_index_compression: 1 2024-02-23T00:33:04.165 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: block_align: 0 2024-02-23T00:33:04.165 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: 2024-02-23T00:33:04.165 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-23T00:33:04.165 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-23T00:33:04.166 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.compression: NoCompression 2024-02-23T00:33:04.166 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-23T00:33:04.166 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-23T00:33:04.166 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-23T00:33:04.166 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.num_levels: 7 2024-02-23T00:33:04.166 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-23T00:33:04.166 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-23T00:33:04.166 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-23T00:33:04.166 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-23T00:33:04.166 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-23T00:33:04.166 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-23T00:33:04.166 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-23T00:33:04.166 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-23T00:33:04.166 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-23T00:33:04.167 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-23T00:33:04.167 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.compression_opts.level: 32767 2024-02-23T00:33:04.167 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-23T00:33:04.167 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-23T00:33:04.167 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-23T00:33:04.167 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.compression_opts.enabled: false 2024-02-23T00:33:04.167 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-23T00:33:04.167 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-23T00:33:04.167 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-23T00:33:04.167 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-23T00:33:04.167 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-23T00:33:04.167 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-23T00:33:04.167 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-23T00:33:04.167 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-23T00:33:04.167 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-23T00:33:04.168 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-23T00:33:04.168 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-23T00:33:04.168 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-23T00:33:04.168 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-23T00:33:04.168 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-23T00:33:04.168 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-23T00:33:04.168 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-23T00:33:04.168 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-23T00:33:04.168 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.arena_block_size: 4194304 2024-02-23T00:33:04.168 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-23T00:33:04.168 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-23T00:33:04.169 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-23T00:33:04.169 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-23T00:33:04.169 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-23T00:33:04.169 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-23T00:33:04.169 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-23T00:33:04.169 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-23T00:33:04.169 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-23T00:33:04.169 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-23T00:33:04.169 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-23T00:33:04.169 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-23T00:33:04.169 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-23T00:33:04.169 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-23T00:33:04.169 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.table_properties_collectors: 2024-02-23T00:33:04.169 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.inplace_update_support: 0 2024-02-23T00:33:04.170 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-23T00:33:04.170 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-23T00:33:04.170 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-23T00:33:04.170 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-23T00:33:04.170 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.bloom_locality: 0 2024-02-23T00:33:04.170 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.max_successive_merges: 0 2024-02-23T00:33:04.170 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-23T00:33:04.170 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-23T00:33:04.170 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.force_consistency_checks: 0 2024-02-23T00:33:04.170 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-23T00:33:04.170 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.ttl: 2592000 2024-02-23T00:33:04.171 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.871+0000 7f3ca06fa880 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-23T00:33:04.171 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.872+0000 7f3ca06fa880 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 5, log_number is 4,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-23T00:33:04.171 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: 2024-02-23T00:33:04.171 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.872+0000 7f3ca06fa880 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 4 2024-02-23T00:33:04.171 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: 2024-02-23T00:33:04.171 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.872+0000 7f3ca06fa880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708648383873333, "job": 1, "event": "recovery_started", "log_files": [6]} 2024-02-23T00:33:04.171 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.872+0000 7f3ca06fa880 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #6 mode 2 2024-02-23T00:33:04.171 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.872+0000 7f3ca06fa880 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-02-23T00:33:04.171 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.873+0000 7f3ca06fa880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708648383874260, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 7, "file_size": 89624, "table_properties": {"data_size": 87750, "index_size": 351, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 709, "raw_key_size": 11370, "raw_average_key_size": 49, "raw_value_size": 81480, "raw_average_value_size": 351, "num_data_blocks": 11, "num_entries": 232, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1708648383, "oldest_key_time": 3, "file_creation_time": 0}} 2024-02-23T00:33:04.171 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.873+0000 7f3ca06fa880 4 rocksdb: [version_set.cc:3826] Creating manifest 8 2024-02-23T00:33:04.171 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: 2024-02-23T00:33:04.171 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.873+0000 7f3ca06fa880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708648383874656, "job": 1, "event": "recovery_finished"} 2024-02-23T00:33:04.172 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.874+0000 7f3ca06fa880 4 rocksdb: DB pointer 0x5588f34fd800 2024-02-23T00:33:04.172 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.874+0000 7f3c874ec700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-02-23T00:33:04.172 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.874+0000 7f3c874ec700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-02-23T00:33:04.172 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: ** DB Stats ** 2024-02-23T00:33:04.172 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-23T00:33:04.172 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-23T00:33:04.172 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-23T00:33:04.172 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-23T00:33:04.172 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-23T00:33:04.172 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-23T00:33:04.172 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-23T00:33:04.172 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: 2024-02-23T00:33:04.172 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: ** Compaction Stats [default] ** 2024-02-23T00:33:04.172 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-23T00:33:04.173 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-23T00:33:04.173 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: L0 2/0 89.18 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.7 0.00 0.00 1 0.001 0 0 2024-02-23T00:33:04.173 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: Sum 2/0 89.18 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.7 0.00 0.00 1 0.001 0 0 2024-02-23T00:33:04.173 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.7 0.00 0.00 1 0.001 0 0 2024-02-23T00:33:04.173 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: 2024-02-23T00:33:04.173 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: ** Compaction Stats [default] ** 2024-02-23T00:33:04.173 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-23T00:33:04.173 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-23T00:33:04.173 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 141.7 0.00 0.00 1 0.001 0 0 2024-02-23T00:33:04.173 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-23T00:33:04.173 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-23T00:33:04.173 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-23T00:33:04.173 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: AddFile(Total Files): cumulative 0, interval 0 2024-02-23T00:33:04.173 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-23T00:33:04.174 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: AddFile(Keys): cumulative 0, interval 0 2024-02-23T00:33:04.174 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: Cumulative compaction: 0.00 GB write, 31.24 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-23T00:33:04.174 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: Interval compaction: 0.00 GB write, 31.24 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-23T00:33:04.174 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-23T00:33:04.174 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: 2024-02-23T00:33:04.174 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: ** File Read Latency Histogram By Level [default] ** 2024-02-23T00:33:04.174 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: 2024-02-23T00:33:04.174 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: ** Compaction Stats [default] ** 2024-02-23T00:33:04.174 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-23T00:33:04.174 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-23T00:33:04.174 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: L0 2/0 89.18 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.7 0.00 0.00 1 0.001 0 0 2024-02-23T00:33:04.174 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: Sum 2/0 89.18 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.7 0.00 0.00 1 0.001 0 0 2024-02-23T00:33:04.174 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-23T00:33:04.174 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: 2024-02-23T00:33:04.174 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: ** Compaction Stats [default] ** 2024-02-23T00:33:04.175 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-23T00:33:04.175 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-23T00:33:04.175 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 141.7 0.00 0.00 1 0.001 0 0 2024-02-23T00:33:04.175 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-23T00:33:04.175 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-23T00:33:04.175 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-23T00:33:04.175 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: AddFile(Total Files): cumulative 0, interval 0 2024-02-23T00:33:04.175 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-23T00:33:04.175 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: AddFile(Keys): cumulative 0, interval 0 2024-02-23T00:33:04.175 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: Cumulative compaction: 0.00 GB write, 30.86 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-23T00:33:04.175 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-23T00:33:04.175 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-23T00:33:04.175 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: 2024-02-23T00:33:04.175 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: ** File Read Latency Histogram By Level [default] ** 2024-02-23T00:33:04.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: 2024-02-23T00:33:04.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.874+0000 7f3ca06fa880 0 starting mon.a rank 0 at public addrs [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] at bind addrs [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid fce2617e-d1e2-11ee-95c0-87774f69a715 2024-02-23T00:33:04.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.875+0000 7f3ca06fa880 1 mon.a@-1(???) e1 preinit fsid fce2617e-d1e2-11ee-95c0-87774f69a715 2024-02-23T00:33:04.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.875+0000 7f3ca06fa880 0 mon.a@-1(???).mds e1 new map 2024-02-23T00:33:04.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.875+0000 7f3ca06fa880 0 mon.a@-1(???).mds e1 print_map 2024-02-23T00:33:04.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: e1 2024-02-23T00:33:04.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: enable_multiple, ever_enabled_multiple: 1,1 2024-02-23T00:33:04.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-23T00:33:04.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: legacy client fscid: -1 2024-02-23T00:33:04.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: 2024-02-23T00:33:04.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: No filesystems configured 2024-02-23T00:33:04.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: 2024-02-23T00:33:04.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.876+0000 7f3ca06fa880 0 mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-23T00:33:04.177 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.876+0000 7f3ca06fa880 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-23T00:33:04.177 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.876+0000 7f3ca06fa880 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-23T00:33:04.177 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.876+0000 7f3ca06fa880 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-23T00:33:04.177 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:01.691592+0000 mon.a (mon.0) 10 : audit [DBG] from='client.? 172.21.15.47:0/1981070344' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:33:04.177 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:33:03.876+0000 7f3ca06fa880 1 mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-23T00:33:04.177 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:33:03.892974+0000 mon.a (mon.0) 1 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2024-02-23T00:33:04.177 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:33:03.893016+0000 mon.a (mon.0) 2 : cluster [DBG] monmap e1: 1 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0]} removed_ranks: {} 2024-02-23T00:33:04.177 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:33:03.893445+0000 mon.a (mon.0) 3 : cluster [DBG] fsmap 2024-02-23T00:33:04.177 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:33:03.893474+0000 mon.a (mon.0) 4 : cluster [DBG] osdmap e1: 0 total, 0 up, 0 in 2024-02-23T00:33:04.177 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:33:03.893773+0000 mon.a (mon.0) 5 : cluster [DBG] mgrmap e1: no daemons active 2024-02-23T00:33:04.177 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 bash[38291]: 14b10697daaf80808b3cdd28f16c91c73ee2cd010164465924918723731cbcf6 2024-02-23T00:33:04.178 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:03 smithi047 systemd[1]: Started Ceph mon.a for fce2617e-d1e2-11ee-95c0-87774f69a715. 2024-02-23T00:33:04.473 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.469+0000 7efc62263700 1 Processor -- start 2024-02-23T00:33:04.473 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.470+0000 7efc62263700 1 -- start start 2024-02-23T00:33:04.473 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.470+0000 7efc62263700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc5c070ae0 0x7efc5c0feb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:04.473 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.470+0000 7efc62263700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c070f00 con 0x7efc5c070ae0 2024-02-23T00:33:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.470+0000 7efc5bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc5c070ae0 0x7efc5c0feb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.470+0000 7efc5bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc5c070ae0 0x7efc5c0feb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58032/0 (socket says 172.21.15.47:58032) 2024-02-23T00:33:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.470+0000 7efc5bfff700 1 -- 172.21.15.47:0/3388755959 learned_addr learned my addr 172.21.15.47:0/3388755959 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.471+0000 7efc5bfff700 1 -- 172.21.15.47:0/3388755959 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5c0ff040 con 0x7efc5c070ae0 2024-02-23T00:33:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.471+0000 7efc5bfff700 1 --2- 172.21.15.47:0/3388755959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc5c070ae0 0x7efc5c0feb50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7efc4c009730 tx=0x7efc4c0090f0).ready entity=mon.0 client_cookie=cf81efd8f94ec511 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.471+0000 7efc5affd700 1 -- 172.21.15.47:0/3388755959 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc4c00b580 con 0x7efc5c070ae0 2024-02-23T00:33:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.471+0000 7efc5affd700 1 -- 172.21.15.47:0/3388755959 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7efc4c00bb50 con 0x7efc5c070ae0 2024-02-23T00:33:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.471+0000 7efc5affd700 1 -- 172.21.15.47:0/3388755959 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc4c011ab0 con 0x7efc5c070ae0 2024-02-23T00:33:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.472+0000 7efc62263700 1 -- 172.21.15.47:0/3388755959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc5c070ae0 msgr2=0x7efc5c0feb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.472+0000 7efc62263700 1 --2- 172.21.15.47:0/3388755959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc5c070ae0 0x7efc5c0feb50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7efc4c009730 tx=0x7efc4c0090f0).stop 2024-02-23T00:33:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.472+0000 7efc62263700 1 -- 172.21.15.47:0/3388755959 shutdown_connections 2024-02-23T00:33:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.472+0000 7efc62263700 1 --2- 172.21.15.47:0/3388755959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc5c070ae0 0x7efc5c0feb50 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.472+0000 7efc62263700 1 -- 172.21.15.47:0/3388755959 >> 172.21.15.47:0/3388755959 conn(0x7efc5c0f6530 msgr2=0x7efc5c0f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.472+0000 7efc62263700 1 -- 172.21.15.47:0/3388755959 shutdown_connections 2024-02-23T00:33:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.472+0000 7efc62263700 1 -- 172.21.15.47:0/3388755959 wait complete. 2024-02-23T00:33:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.472+0000 7efc62263700 1 librados: starting msgr at 2024-02-23T00:33:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.472+0000 7efc62263700 1 librados: starting objecter 2024-02-23T00:33:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.473+0000 7efc62263700 1 Processor -- start 2024-02-23T00:33:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.473+0000 7efc62263700 1 -- start start 2024-02-23T00:33:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.473+0000 7efc62263700 1 librados: setting wanted keys 2024-02-23T00:33:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.473+0000 7efc62263700 1 librados: calling monclient init 2024-02-23T00:33:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.473+0000 7efc62263700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc5c070ae0 0x7efc5c10aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.473+0000 7efc62263700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c10af30 con 0x7efc5c070ae0 2024-02-23T00:33:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.473+0000 7efc5bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc5c070ae0 0x7efc5c10aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.473+0000 7efc5bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc5c070ae0 0x7efc5c10aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58044/0 (socket says 172.21.15.47:58044) 2024-02-23T00:33:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.473+0000 7efc5bfff700 1 -- 172.21.15.47:0/1469910874 learned_addr learned my addr 172.21.15.47:0/1469910874 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.473+0000 7efc5bfff700 1 -- 172.21.15.47:0/1469910874 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc4c009370 con 0x7efc5c070ae0 2024-02-23T00:33:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.474+0000 7efc5bfff700 1 --2- 172.21.15.47:0/1469910874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc5c070ae0 0x7efc5c10aa40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7efc4c012f20 tx=0x7efc4c0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.474+0000 7efc59775700 10 client.?.objecter ms_handle_connect 0x7efc5c070ae0 2024-02-23T00:33:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.474+0000 7efc59775700 10 client.?.objecter resend_mon_ops 2024-02-23T00:33:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.474+0000 7efc59775700 1 -- 172.21.15.47:0/1469910874 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc4c011a70 con 0x7efc5c070ae0 2024-02-23T00:33:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.474+0000 7efc59775700 1 -- 172.21.15.47:0/1469910874 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7efc4c01aab0 con 0x7efc5c070ae0 2024-02-23T00:33:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.474+0000 7efc59775700 1 -- 172.21.15.47:0/1469910874 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc4c023960 con 0x7efc5c070ae0 2024-02-23T00:33:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.474+0000 7efc62263700 1 -- 172.21.15.47:0/1469910874 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc5c10b130 con 0x7efc5c070ae0 2024-02-23T00:33:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.474+0000 7efc62263700 10 client.14098.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:33:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.474+0000 7efc62263700 1 -- 172.21.15.47:0/1469910874 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc5c10b5c0 con 0x7efc5c070ae0 2024-02-23T00:33:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.474+0000 7efc62263700 1 librados: init done 2024-02-23T00:33:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.474+0000 7efc59775700 1 -- 172.21.15.47:0/1469910874 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7efc4c023d20 con 0x7efc5c070ae0 2024-02-23T00:33:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.474+0000 7efc59775700 1 -- 172.21.15.47:0/1469910874 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7efc4c0223c0 con 0x7efc5c070ae0 2024-02-23T00:33:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.474+0000 7efc59775700 10 client.14098.objecter ms_dispatch 0x7efc5c001550 osd_map(1..1 src has 1..1) v4 2024-02-23T00:33:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.474+0000 7efc59775700 3 client.14098.objecter handle_osd_map got epochs [1,1] > 0 2024-02-23T00:33:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.474+0000 7efc59775700 3 client.14098.objecter handle_osd_map decoding full epoch 1 2024-02-23T00:33:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.474+0000 7efc59775700 20 client.14098.objecter dump_active .. 0 homeless 2024-02-23T00:33:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.475+0000 7efc62263700 1 -- 172.21.15.47:0/1469910874 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc5c046ce0 con 0x7efc5c070ae0 2024-02-23T00:33:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.477+0000 7efc59775700 1 -- 172.21.15.47:0/1469910874 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7efc4c01e030 con 0x7efc5c070ae0 2024-02-23T00:33:04.546 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.544+0000 7efc62263700 1 -- 172.21.15.47:0/1469910874 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7efc5c0ffd80 con 0x7efc5c070ae0 2024-02-23T00:33:04.548 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.546+0000 7efc59775700 1 -- 172.21.15.47:0/1469910874 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7efc4c01ac20 con 0x7efc5c070ae0 2024-02-23T00:33:04.549 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.547+0000 7efc62263700 10 librados: watch_flush enter 2024-02-23T00:33:04.549 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.547+0000 7efc62263700 10 librados: watch_flush exit 2024-02-23T00:33:04.550 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.547+0000 7efc62263700 20 client.14098.objecter shutdown clearing up homeless session... 2024-02-23T00:33:04.550 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.547+0000 7efc62263700 10 client.14098.objecter successfully canceled tick 2024-02-23T00:33:04.550 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.547+0000 7efc62263700 1 -- 172.21.15.47:0/1469910874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc5c070ae0 msgr2=0x7efc5c10aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:04.550 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.547+0000 7efc62263700 1 --2- 172.21.15.47:0/1469910874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc5c070ae0 0x7efc5c10aa40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7efc4c012f20 tx=0x7efc4c0090f0).stop 2024-02-23T00:33:04.550 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.547+0000 7efc62263700 1 -- 172.21.15.47:0/1469910874 shutdown_connections 2024-02-23T00:33:04.550 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.547+0000 7efc62263700 1 --2- 172.21.15.47:0/1469910874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc5c070ae0 0x7efc5c10aa40 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:04.550 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.547+0000 7efc62263700 1 -- 172.21.15.47:0/1469910874 >> 172.21.15.47:0/1469910874 conn(0x7efc5c0f6530 msgr2=0x7efc5c0f7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:04.550 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.547+0000 7efc62263700 1 -- 172.21.15.47:0/1469910874 shutdown_connections 2024-02-23T00:33:04.550 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.547+0000 7efc62263700 1 -- 172.21.15.47:0/1469910874 wait complete. 2024-02-23T00:33:04.550 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:04.547+0000 7efc62263700 1 librados: shutdown 2024-02-23T00:33:05.140 INFO:teuthology.orchestra.run.smithi047.stderr:Wrote config to /etc/ceph/ceph.conf 2024-02-23T00:33:05.140 INFO:teuthology.orchestra.run.smithi047.stderr:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-23T00:33:05.141 INFO:teuthology.orchestra.run.smithi047.stderr:Creating mgr... 2024-02-23T00:33:05.141 INFO:teuthology.orchestra.run.smithi047.stderr:Verifying port 9283 ... 2024-02-23T00:33:05.287 INFO:teuthology.orchestra.run.smithi047.stderr:systemctl: Failed to reset failed state of unit ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@mgr.y.service: Unit ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@mgr.y.service not loaded. 2024-02-23T00:33:05.293 INFO:teuthology.orchestra.run.smithi047.stderr:systemctl: Created symlink /etc/systemd/system/ceph-fce2617e-d1e2-11ee-95c0-87774f69a715.target.wants/ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@mgr.y.service → /etc/systemd/system/ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@.service. 2024-02-23T00:33:05.394 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:05 smithi047 systemd[1]: Starting Ceph mgr.y for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T00:33:05.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:04.547680+0000 mon.a (mon.0) 6 : audit [INF] from='client.? 172.21.15.47:0/1469910874' entity='client.admin' 2024-02-23T00:33:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:systemctl: disabled 2024-02-23T00:33:06.036 INFO:teuthology.orchestra.run.smithi047.stderr:systemctl: inactive 2024-02-23T00:33:06.037 INFO:teuthology.orchestra.run.smithi047.stderr:firewalld.service is not enabled 2024-02-23T00:33:06.037 INFO:teuthology.orchestra.run.smithi047.stderr:Not possible to enable service . firewalld.service is not available 2024-02-23T00:33:06.045 INFO:teuthology.orchestra.run.smithi047.stderr:systemctl: disabled 2024-02-23T00:33:06.054 INFO:teuthology.orchestra.run.smithi047.stderr:systemctl: inactive 2024-02-23T00:33:06.054 INFO:teuthology.orchestra.run.smithi047.stderr:firewalld.service is not enabled 2024-02-23T00:33:06.054 INFO:teuthology.orchestra.run.smithi047.stderr:Not possible to open ports <[9283]>. firewalld.service is not available 2024-02-23T00:33:06.054 INFO:teuthology.orchestra.run.smithi047.stderr:Waiting for mgr to start... 2024-02-23T00:33:06.055 INFO:teuthology.orchestra.run.smithi047.stderr:Waiting for mgr... 2024-02-23T00:33:06.289 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:06 smithi047 bash[38562]: 1569c4c2a749afcddc23fc3023a8206c2a7e53504411419768e5c02d0212117e 2024-02-23T00:33:06.289 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:06 smithi047 systemd[1]: Started Ceph mgr.y for fce2617e-d1e2-11ee-95c0-87774f69a715. 2024-02-23T00:33:06.289 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:06.085+0000 7fa8b9689680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-23T00:33:06.289 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:06.169+0000 7fa8b9689680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-23T00:33:06.609 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.605+0000 7f1f3bae2700 1 Processor -- start 2024-02-23T00:33:06.609 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.605+0000 7f1f3bae2700 1 -- start start 2024-02-23T00:33:06.609 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.605+0000 7f1f3bae2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f340733d0 0x7f1f340719c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:06.610 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.605+0000 7f1f3bae2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f340737f0 con 0x7f1f340733d0 2024-02-23T00:33:06.610 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.605+0000 7f1f3987e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f340733d0 0x7f1f340719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:06.610 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.606+0000 7f1f3987e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f340733d0 0x7f1f340719c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54026/0 (socket says 172.21.15.47:54026) 2024-02-23T00:33:06.610 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.606+0000 7f1f3987e700 1 -- 172.21.15.47:0/3056021719 learned_addr learned my addr 172.21.15.47:0/3056021719 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:06.610 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.606+0000 7f1f3987e700 1 -- 172.21.15.47:0/3056021719 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f34071eb0 con 0x7f1f340733d0 2024-02-23T00:33:06.610 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.606+0000 7f1f3987e700 1 --2- 172.21.15.47:0/3056021719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f340733d0 0x7f1f340719c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f1f30009730 tx=0x7f1f300090f0).ready entity=mon.0 client_cookie=20c8641f052972e3 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:06.610 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.606+0000 7f1f3887c700 1 -- 172.21.15.47:0/3056021719 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f3000b580 con 0x7f1f340733d0 2024-02-23T00:33:06.610 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.606+0000 7f1f3887c700 1 -- 172.21.15.47:0/3056021719 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7f1f3000bb50 con 0x7f1f340733d0 2024-02-23T00:33:06.610 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.606+0000 7f1f3887c700 1 -- 172.21.15.47:0/3056021719 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f30011ab0 con 0x7f1f340733d0 2024-02-23T00:33:06.610 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.607+0000 7f1f3bae2700 1 -- 172.21.15.47:0/3056021719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f340733d0 msgr2=0x7f1f340719c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:06.610 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.607+0000 7f1f3bae2700 1 --2- 172.21.15.47:0/3056021719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f340733d0 0x7f1f340719c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f1f30009730 tx=0x7f1f300090f0).stop 2024-02-23T00:33:06.610 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.607+0000 7f1f3bae2700 1 -- 172.21.15.47:0/3056021719 shutdown_connections 2024-02-23T00:33:06.610 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.607+0000 7f1f3bae2700 1 --2- 172.21.15.47:0/3056021719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f340733d0 0x7f1f340719c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:06.610 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.607+0000 7f1f3bae2700 1 -- 172.21.15.47:0/3056021719 >> 172.21.15.47:0/3056021719 conn(0x7f1f340f7850 msgr2=0x7f1f340f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:06.611 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.607+0000 7f1f3bae2700 1 -- 172.21.15.47:0/3056021719 shutdown_connections 2024-02-23T00:33:06.611 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.607+0000 7f1f3bae2700 1 -- 172.21.15.47:0/3056021719 wait complete. 2024-02-23T00:33:06.611 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.608+0000 7f1f3bae2700 1 librados: starting msgr at 2024-02-23T00:33:06.611 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.608+0000 7f1f3bae2700 1 librados: starting objecter 2024-02-23T00:33:06.613 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.608+0000 7f1f3bae2700 1 Processor -- start 2024-02-23T00:33:06.613 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.608+0000 7f1f3bae2700 1 -- start start 2024-02-23T00:33:06.613 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.608+0000 7f1f3bae2700 1 librados: setting wanted keys 2024-02-23T00:33:06.614 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.608+0000 7f1f3bae2700 1 librados: calling monclient init 2024-02-23T00:33:06.614 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.608+0000 7f1f3bae2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f340733d0 0x7f1f3410cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:06.614 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.608+0000 7f1f3bae2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3410d200 con 0x7f1f340733d0 2024-02-23T00:33:06.614 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.608+0000 7f1f3987e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f340733d0 0x7f1f3410cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:06.614 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.608+0000 7f1f3987e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f340733d0 0x7f1f3410cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54032/0 (socket says 172.21.15.47:54032) 2024-02-23T00:33:06.614 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.608+0000 7f1f3987e700 1 -- 172.21.15.47:0/614039594 learned_addr learned my addr 172.21.15.47:0/614039594 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:06.614 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.608+0000 7f1f3987e700 1 -- 172.21.15.47:0/614039594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f30009370 con 0x7f1f340733d0 2024-02-23T00:33:06.614 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.609+0000 7f1f3987e700 1 --2- 172.21.15.47:0/614039594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f340733d0 0x7f1f3410cd10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f1f30012f20 tx=0x7f1f300090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:06.614 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.609+0000 7f1f2af78700 10 client.?.objecter ms_handle_connect 0x7f1f340733d0 2024-02-23T00:33:06.615 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.609+0000 7f1f2af78700 10 client.?.objecter resend_mon_ops 2024-02-23T00:33:06.615 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.609+0000 7f1f2af78700 1 -- 172.21.15.47:0/614039594 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f30011a70 con 0x7f1f340733d0 2024-02-23T00:33:06.616 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.609+0000 7f1f2af78700 1 -- 172.21.15.47:0/614039594 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7f1f3001aab0 con 0x7f1f340733d0 2024-02-23T00:33:06.616 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.609+0000 7f1f3bae2700 1 -- 172.21.15.47:0/614039594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f3410d400 con 0x7f1f340733d0 2024-02-23T00:33:06.616 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.609+0000 7f1f2af78700 1 -- 172.21.15.47:0/614039594 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f300238f0 con 0x7f1f340733d0 2024-02-23T00:33:06.616 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.609+0000 7f1f3bae2700 10 client.14102.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:33:06.616 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.609+0000 7f1f3bae2700 1 -- 172.21.15.47:0/614039594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3410dfb0 con 0x7f1f340733d0 2024-02-23T00:33:06.616 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.609+0000 7f1f3bae2700 1 librados: init done 2024-02-23T00:33:06.616 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.610+0000 7f1f2af78700 1 -- 172.21.15.47:0/614039594 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f1f3002c3f0 con 0x7f1f340733d0 2024-02-23T00:33:06.616 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.610+0000 7f1f2af78700 1 -- 172.21.15.47:0/614039594 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f1f30022730 con 0x7f1f340733d0 2024-02-23T00:33:06.616 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.610+0000 7f1f2af78700 10 client.14102.objecter ms_dispatch 0x7f1f34001550 osd_map(1..1 src has 1..1) v4 2024-02-23T00:33:06.617 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.610+0000 7f1f2af78700 3 client.14102.objecter handle_osd_map got epochs [1,1] > 0 2024-02-23T00:33:06.617 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.610+0000 7f1f2af78700 3 client.14102.objecter handle_osd_map decoding full epoch 1 2024-02-23T00:33:06.617 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.610+0000 7f1f2af78700 20 client.14102.objecter dump_active .. 0 homeless 2024-02-23T00:33:06.617 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.610+0000 7f1f3bae2700 1 -- 172.21.15.47:0/614039594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f3405a310 con 0x7f1f340733d0 2024-02-23T00:33:06.617 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.612+0000 7f1f2af78700 1 -- 172.21.15.47:0/614039594 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7f1f3001e030 con 0x7f1f340733d0 2024-02-23T00:33:06.695 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.684+0000 7f1f3bae2700 1 -- 172.21.15.47:0/614039594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f1f34102080 con 0x7f1f340733d0 2024-02-23T00:33:06.695 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.685+0000 7f1f2af78700 1 -- 172.21.15.47:0/614039594 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1241 (secure 0 0 0) 0x7f1f3001ac20 con 0x7f1f340733d0 2024-02-23T00:33:06.695 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.695 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: { 2024-02-23T00:33:06.695 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "fsid": "fce2617e-d1e2-11ee-95c0-87774f69a715", 2024-02-23T00:33:06.695 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "health": { 2024-02-23T00:33:06.695 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-02-23T00:33:06.695 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "checks": {}, 2024-02-23T00:33:06.695 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "mutes": [] 2024-02-23T00:33:06.695 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: }, 2024-02-23T00:33:06.696 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-02-23T00:33:06.696 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "quorum": [ 2024-02-23T00:33:06.696 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 0 2024-02-23T00:33:06.696 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: ], 2024-02-23T00:33:06.696 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "quorum_names": [ 2024-02-23T00:33:06.696 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "a" 2024-02-23T00:33:06.696 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: ], 2024-02-23T00:33:06.696 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "quorum_age": 2, 2024-02-23T00:33:06.696 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "monmap": { 2024-02-23T00:33:06.696 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-23T00:33:06.696 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2024-02-23T00:33:06.697 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "num_mons": 1 2024-02-23T00:33:06.697 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: }, 2024-02-23T00:33:06.697 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "osdmap": { 2024-02-23T00:33:06.697 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-23T00:33:06.697 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "num_osds": 0, 2024-02-23T00:33:06.697 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-02-23T00:33:06.697 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-02-23T00:33:06.697 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-02-23T00:33:06.697 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-02-23T00:33:06.697 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-02-23T00:33:06.697 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: }, 2024-02-23T00:33:06.697 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "pgmap": { 2024-02-23T00:33:06.698 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-02-23T00:33:06.698 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-02-23T00:33:06.698 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "num_pools": 0, 2024-02-23T00:33:06.698 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "num_objects": 0, 2024-02-23T00:33:06.698 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-02-23T00:33:06.698 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-02-23T00:33:06.698 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-02-23T00:33:06.698 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "bytes_total": 0 2024-02-23T00:33:06.698 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: }, 2024-02-23T00:33:06.698 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "fsmap": { 2024-02-23T00:33:06.698 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-23T00:33:06.698 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "by_rank": [], 2024-02-23T00:33:06.698 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "up:standby": 0 2024-02-23T00:33:06.699 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: }, 2024-02-23T00:33:06.699 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "mgrmap": { 2024-02-23T00:33:06.699 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "available": false, 2024-02-23T00:33:06.699 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-02-23T00:33:06.699 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "modules": [ 2024-02-23T00:33:06.699 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "iostat", 2024-02-23T00:33:06.699 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "nfs", 2024-02-23T00:33:06.699 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "restful" 2024-02-23T00:33:06.699 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: ], 2024-02-23T00:33:06.699 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "services": {} 2024-02-23T00:33:06.700 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: }, 2024-02-23T00:33:06.700 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "servicemap": { 2024-02-23T00:33:06.700 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-23T00:33:06.700 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "modified": "2024-02-23T00:32:55.399919+0000", 2024-02-23T00:33:06.700 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "services": {} 2024-02-23T00:33:06.700 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: }, 2024-02-23T00:33:06.700 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "progress_events": {} 2024-02-23T00:33:06.700 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: } 2024-02-23T00:33:06.700 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.686+0000 7f1f3bae2700 10 librados: watch_flush enter 2024-02-23T00:33:06.700 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.686+0000 7f1f3bae2700 10 librados: watch_flush exit 2024-02-23T00:33:06.700 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.686+0000 7f1f3bae2700 20 client.14102.objecter shutdown clearing up homeless session... 2024-02-23T00:33:06.700 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.686+0000 7f1f3bae2700 10 client.14102.objecter successfully canceled tick 2024-02-23T00:33:06.701 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.686+0000 7f1f3bae2700 1 -- 172.21.15.47:0/614039594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f340733d0 msgr2=0x7f1f3410cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:06.701 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.686+0000 7f1f3bae2700 1 --2- 172.21.15.47:0/614039594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f340733d0 0x7f1f3410cd10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f1f30012f20 tx=0x7f1f300090f0).stop 2024-02-23T00:33:06.701 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.686+0000 7f1f3bae2700 1 -- 172.21.15.47:0/614039594 shutdown_connections 2024-02-23T00:33:06.701 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.686+0000 7f1f3bae2700 1 --2- 172.21.15.47:0/614039594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f340733d0 0x7f1f3410cd10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:06.701 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.686+0000 7f1f3bae2700 1 -- 172.21.15.47:0/614039594 >> 172.21.15.47:0/614039594 conn(0x7f1f340f7850 msgr2=0x7f1f340fa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:06.701 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.687+0000 7f1f3bae2700 1 -- 172.21.15.47:0/614039594 shutdown_connections 2024-02-23T00:33:06.701 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.687+0000 7f1f3bae2700 1 -- 172.21.15.47:0/614039594 wait complete. 2024-02-23T00:33:06.701 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:06.687+0000 7f1f3bae2700 1 librados: shutdown 2024-02-23T00:33:07.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:06.686241+0000 mon.a (mon.0) 7 : audit [DBG] from='client.? 172.21.15.47:0/614039594' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-23T00:33:07.056 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:06.566+0000 7fa8b9689680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-23T00:33:07.450 INFO:teuthology.orchestra.run.smithi047.stderr:mgr not available, waiting (1/15)... 2024-02-23T00:33:07.675 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:07.373+0000 7fa8b9689680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-23T00:33:07.676 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:07.455+0000 7fa8b9689680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-23T00:33:07.676 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:07.673+0000 7fa8b9689680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-23T00:33:08.971 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:08.662+0000 7fa8b9689680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-23T00:33:08.971 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:08.795+0000 7fa8b9689680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-23T00:33:08.971 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:08.876+0000 7fa8b9689680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-23T00:33:09.306 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:08.969+0000 7fa8b9689680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-23T00:33:09.306 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:09.170+0000 7fa8b9689680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-23T00:33:09.717 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:09.461+0000 7fa8b9689680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-23T00:33:09.717 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:09.565+0000 7fa8b9689680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-23T00:33:10.056 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.050+0000 7f0611ffe700 1 Processor -- start 2024-02-23T00:33:10.056 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.051+0000 7f0611ffe700 1 -- start start 2024-02-23T00:33:10.056 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.051+0000 7f0611ffe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f060c071300 0x7f060c0feb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:10.056 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.051+0000 7f0611ffe700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060c071720 con 0x7f060c071300 2024-02-23T00:33:10.056 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.051+0000 7f060b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f060c071300 0x7f060c0feb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:10.056 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.051+0000 7f060b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f060c071300 0x7f060c0feb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54036/0 (socket says 172.21.15.47:54036) 2024-02-23T00:33:10.056 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.051+0000 7f060b7fe700 1 -- 172.21.15.47:0/1823310422 learned_addr learned my addr 172.21.15.47:0/1823310422 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:10.057 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.051+0000 7f060b7fe700 1 -- 172.21.15.47:0/1823310422 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060c0ff050 con 0x7f060c071300 2024-02-23T00:33:10.057 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.051+0000 7f060b7fe700 1 --2- 172.21.15.47:0/1823310422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f060c071300 0x7f060c0feb60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f05fc009730 tx=0x7f05fc0090f0).ready entity=mon.0 client_cookie=da8224a7c2640130 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:10.057 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.052+0000 7f060a7fc700 1 -- 172.21.15.47:0/1823310422 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05fc00b580 con 0x7f060c071300 2024-02-23T00:33:10.057 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.052+0000 7f060a7fc700 1 -- 172.21.15.47:0/1823310422 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7f05fc00bb50 con 0x7f060c071300 2024-02-23T00:33:10.057 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.052+0000 7f060a7fc700 1 -- 172.21.15.47:0/1823310422 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05fc011ab0 con 0x7f060c071300 2024-02-23T00:33:10.057 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.052+0000 7f0611ffe700 1 -- 172.21.15.47:0/1823310422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f060c071300 msgr2=0x7f060c0feb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:10.057 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.052+0000 7f0611ffe700 1 --2- 172.21.15.47:0/1823310422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f060c071300 0x7f060c0feb60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f05fc009730 tx=0x7f05fc0090f0).stop 2024-02-23T00:33:10.057 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.052+0000 7f0611ffe700 1 -- 172.21.15.47:0/1823310422 shutdown_connections 2024-02-23T00:33:10.057 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.052+0000 7f0611ffe700 1 --2- 172.21.15.47:0/1823310422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f060c071300 0x7f060c0feb60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:10.057 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.052+0000 7f0611ffe700 1 -- 172.21.15.47:0/1823310422 >> 172.21.15.47:0/1823310422 conn(0x7f060c0f6540 msgr2=0x7f060c0f8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:10.057 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.052+0000 7f0611ffe700 1 -- 172.21.15.47:0/1823310422 shutdown_connections 2024-02-23T00:33:10.058 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.052+0000 7f0611ffe700 1 -- 172.21.15.47:0/1823310422 wait complete. 2024-02-23T00:33:10.058 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.053+0000 7f0611ffe700 1 librados: starting msgr at 2024-02-23T00:33:10.058 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.053+0000 7f0611ffe700 1 librados: starting objecter 2024-02-23T00:33:10.058 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.053+0000 7f0611ffe700 1 Processor -- start 2024-02-23T00:33:10.058 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.054+0000 7f0611ffe700 1 -- start start 2024-02-23T00:33:10.058 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.054+0000 7f0611ffe700 1 librados: setting wanted keys 2024-02-23T00:33:10.059 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.054+0000 7f0611ffe700 1 librados: calling monclient init 2024-02-23T00:33:10.059 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.054+0000 7f0611ffe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f060c071300 0x7f060c10aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:10.059 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.054+0000 7f0611ffe700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060c10af80 con 0x7f060c071300 2024-02-23T00:33:10.059 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.054+0000 7f060b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f060c071300 0x7f060c10aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:10.059 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.054+0000 7f060b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f060c071300 0x7f060c10aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54046/0 (socket says 172.21.15.47:54046) 2024-02-23T00:33:10.059 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.054+0000 7f060b7fe700 1 -- 172.21.15.47:0/785251794 learned_addr learned my addr 172.21.15.47:0/785251794 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:10.059 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.054+0000 7f060b7fe700 1 -- 172.21.15.47:0/785251794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05fc009370 con 0x7f060c071300 2024-02-23T00:33:10.060 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.054+0000 7f060b7fe700 1 --2- 172.21.15.47:0/785251794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f060c071300 0x7f060c10aa90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f05fc000c00 tx=0x7f05fc0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:10.060 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.054+0000 7f0608ff9700 10 client.?.objecter ms_handle_connect 0x7f060c071300 2024-02-23T00:33:10.060 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.054+0000 7f0608ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:33:10.060 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.055+0000 7f0608ff9700 1 -- 172.21.15.47:0/785251794 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05fc00b9e0 con 0x7f060c071300 2024-02-23T00:33:10.060 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.055+0000 7f0611ffe700 1 -- 172.21.15.47:0/785251794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f060c10b180 con 0x7f060c071300 2024-02-23T00:33:10.060 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.055+0000 7f0608ff9700 1 -- 172.21.15.47:0/785251794 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7f05fc011400 con 0x7f060c071300 2024-02-23T00:33:10.060 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.055+0000 7f0608ff9700 1 -- 172.21.15.47:0/785251794 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05fc0238b0 con 0x7f060c071300 2024-02-23T00:33:10.060 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.055+0000 7f0611ffe700 10 client.14104.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:33:10.060 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.055+0000 7f0611ffe700 1 -- 172.21.15.47:0/785251794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060c10b610 con 0x7f060c071300 2024-02-23T00:33:10.060 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.055+0000 7f0611ffe700 1 librados: init done 2024-02-23T00:33:10.061 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.055+0000 7f0608ff9700 1 -- 172.21.15.47:0/785251794 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f05fc02c3f0 con 0x7f060c071300 2024-02-23T00:33:10.061 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.055+0000 7f0608ff9700 1 -- 172.21.15.47:0/785251794 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f05fc022730 con 0x7f060c071300 2024-02-23T00:33:10.061 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.055+0000 7f0608ff9700 10 client.14104.objecter ms_dispatch 0x7f060c001550 osd_map(1..1 src has 1..1) v4 2024-02-23T00:33:10.061 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.055+0000 7f0608ff9700 3 client.14104.objecter handle_osd_map got epochs [1,1] > 0 2024-02-23T00:33:10.061 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.055+0000 7f0608ff9700 3 client.14104.objecter handle_osd_map decoding full epoch 1 2024-02-23T00:33:10.061 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.055+0000 7f0608ff9700 20 client.14104.objecter dump_active .. 0 homeless 2024-02-23T00:33:10.061 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.056+0000 7f0611ffe700 1 -- 172.21.15.47:0/785251794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f060c046ce0 con 0x7f060c071300 2024-02-23T00:33:10.061 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.058+0000 7f0608ff9700 1 -- 172.21.15.47:0/785251794 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7f05fc01e030 con 0x7f060c071300 2024-02-23T00:33:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.128+0000 7f0611ffe700 1 -- 172.21.15.47:0/785251794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f060c104590 con 0x7f060c071300 2024-02-23T00:33:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.129+0000 7f0608ff9700 1 -- 172.21.15.47:0/785251794 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1241 (secure 0 0 0) 0x7f05fc02c830 con 0x7f060c071300 2024-02-23T00:33:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: { 2024-02-23T00:33:10.142 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "fsid": "fce2617e-d1e2-11ee-95c0-87774f69a715", 2024-02-23T00:33:10.142 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "health": { 2024-02-23T00:33:10.142 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-02-23T00:33:10.142 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "checks": {}, 2024-02-23T00:33:10.142 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "mutes": [] 2024-02-23T00:33:10.142 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: }, 2024-02-23T00:33:10.142 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-02-23T00:33:10.142 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "quorum": [ 2024-02-23T00:33:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 0 2024-02-23T00:33:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: ], 2024-02-23T00:33:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "quorum_names": [ 2024-02-23T00:33:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "a" 2024-02-23T00:33:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: ], 2024-02-23T00:33:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "quorum_age": 6, 2024-02-23T00:33:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "monmap": { 2024-02-23T00:33:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-23T00:33:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2024-02-23T00:33:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "num_mons": 1 2024-02-23T00:33:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: }, 2024-02-23T00:33:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "osdmap": { 2024-02-23T00:33:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-23T00:33:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "num_osds": 0, 2024-02-23T00:33:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-02-23T00:33:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-02-23T00:33:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-02-23T00:33:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-02-23T00:33:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-02-23T00:33:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: }, 2024-02-23T00:33:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "pgmap": { 2024-02-23T00:33:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-02-23T00:33:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-02-23T00:33:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "num_pools": 0, 2024-02-23T00:33:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "num_objects": 0, 2024-02-23T00:33:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-02-23T00:33:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-02-23T00:33:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-02-23T00:33:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "bytes_total": 0 2024-02-23T00:33:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: }, 2024-02-23T00:33:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "fsmap": { 2024-02-23T00:33:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-23T00:33:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "by_rank": [], 2024-02-23T00:33:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "up:standby": 0 2024-02-23T00:33:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: }, 2024-02-23T00:33:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "mgrmap": { 2024-02-23T00:33:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "available": false, 2024-02-23T00:33:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-02-23T00:33:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "modules": [ 2024-02-23T00:33:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "iostat", 2024-02-23T00:33:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "nfs", 2024-02-23T00:33:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "restful" 2024-02-23T00:33:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: ], 2024-02-23T00:33:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "services": {} 2024-02-23T00:33:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: }, 2024-02-23T00:33:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "servicemap": { 2024-02-23T00:33:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-23T00:33:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "modified": "2024-02-23T00:32:55.399919+0000", 2024-02-23T00:33:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "services": {} 2024-02-23T00:33:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: }, 2024-02-23T00:33:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "progress_events": {} 2024-02-23T00:33:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: } 2024-02-23T00:33:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.129+0000 7f0611ffe700 10 librados: watch_flush enter 2024-02-23T00:33:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.130+0000 7f0611ffe700 10 librados: watch_flush exit 2024-02-23T00:33:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.130+0000 7f0611ffe700 20 client.14104.objecter shutdown clearing up homeless session... 2024-02-23T00:33:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.130+0000 7f0611ffe700 10 client.14104.objecter successfully canceled tick 2024-02-23T00:33:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.130+0000 7f0611ffe700 1 -- 172.21.15.47:0/785251794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f060c071300 msgr2=0x7f060c10aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.130+0000 7f0611ffe700 1 --2- 172.21.15.47:0/785251794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f060c071300 0x7f060c10aa90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f05fc000c00 tx=0x7f05fc0090f0).stop 2024-02-23T00:33:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.130+0000 7f0611ffe700 1 -- 172.21.15.47:0/785251794 shutdown_connections 2024-02-23T00:33:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.130+0000 7f0611ffe700 1 --2- 172.21.15.47:0/785251794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f060c071300 0x7f060c10aa90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.130+0000 7f0611ffe700 1 -- 172.21.15.47:0/785251794 >> 172.21.15.47:0/785251794 conn(0x7f060c0f6540 msgr2=0x7f060c0f7e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.130+0000 7f0611ffe700 1 -- 172.21.15.47:0/785251794 shutdown_connections 2024-02-23T00:33:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.130+0000 7f0611ffe700 1 -- 172.21.15.47:0/785251794 wait complete. 2024-02-23T00:33:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:10.130+0000 7f0611ffe700 1 librados: shutdown 2024-02-23T00:33:10.434 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:10.130157+0000 mon.a (mon.0) 8 : audit [DBG] from='client.? 172.21.15.47:0/785251794' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-23T00:33:10.434 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:10.348+0000 7fa8b9689680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-23T00:33:10.735 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:10.432+0000 7fa8b9689680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-23T00:33:10.736 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:10.527+0000 7fa8b9689680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-23T00:33:11.056 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:10.706+0000 7fa8b9689680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-23T00:33:11.056 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:10.974+0000 7fa8b9689680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-23T00:33:11.432 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:11.111+0000 7fa8b9689680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-23T00:33:11.432 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:11.250+0000 7fa8b9689680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-23T00:33:11.806 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:11.430+0000 7fa8b9689680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-23T00:33:11.806 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:11.516+0000 7fa8b9689680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-23T00:33:11.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:33:11.530037+0000 mon.a (mon.0) 9 : cluster [INF] Activating manager daemon y 2024-02-23T00:33:11.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:33:11.532529+0000 mon.a (mon.0) 10 : cluster [DBG] mgrmap e2: y(active, starting, since 0.00258695s) 2024-02-23T00:33:11.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:11.533395+0000 mon.a (mon.0) 11 : audit [DBG] from='mgr.14100 172.21.15.47:0/4054175394' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-23T00:33:11.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:11.533527+0000 mon.a (mon.0) 12 : audit [DBG] from='mgr.14100 172.21.15.47:0/4054175394' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-23T00:33:11.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:11.533642+0000 mon.a (mon.0) 13 : audit [DBG] from='mgr.14100 172.21.15.47:0/4054175394' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-23T00:33:11.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:11.533752+0000 mon.a (mon.0) 14 : audit [DBG] from='mgr.14100 172.21.15.47:0/4054175394' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-23T00:33:11.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:11.534144+0000 mon.a (mon.0) 15 : audit [DBG] from='mgr.14100 172.21.15.47:0/4054175394' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-23T00:33:11.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:33:11.547763+0000 mon.a (mon.0) 16 : cluster [INF] Manager daemon y is now available 2024-02-23T00:33:11.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:11.557530+0000 mon.a (mon.0) 17 : audit [INF] from='mgr.14100 172.21.15.47:0/4054175394' entity='mgr.y' 2024-02-23T00:33:11.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:11.562502+0000 mon.a (mon.0) 18 : audit [INF] from='mgr.14100 172.21.15.47:0/4054175394' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-23T00:33:11.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:11.565520+0000 mon.a (mon.0) 19 : audit [INF] from='mgr.14100 172.21.15.47:0/4054175394' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-23T00:33:11.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:11.567628+0000 mon.a (mon.0) 20 : audit [INF] from='mgr.14100 172.21.15.47:0/4054175394' entity='mgr.y' 2024-02-23T00:33:11.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:11.569424+0000 mon.a (mon.0) 21 : audit [INF] from='mgr.14100 172.21.15.47:0/4054175394' entity='mgr.y' 2024-02-23T00:33:13.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:33:12.535965+0000 mon.a (mon.0) 22 : cluster [DBG] mgrmap e3: y(active, since 1.00602s) 2024-02-23T00:33:14.028 INFO:teuthology.orchestra.run.smithi047.stderr:mgr not available, waiting (2/15)... 2024-02-23T00:33:15.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:33:14.536675+0000 mon.a (mon.0) 23 : cluster [DBG] mgrmap e4: y(active, since 3s) 2024-02-23T00:33:19.060 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.054+0000 7f3599d4f700 1 Processor -- start 2024-02-23T00:33:19.060 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.054+0000 7f3599d4f700 1 -- start start 2024-02-23T00:33:19.060 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.054+0000 7f3599d4f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35940713e0 0x7f35940fea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:19.060 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.054+0000 7f3599d4f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35940fef70 con 0x7f35940713e0 2024-02-23T00:33:19.060 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.054+0000 7f35937fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35940713e0 0x7f35940fea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:19.060 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.055+0000 7f35937fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35940713e0 0x7f35940fea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34176/0 (socket says 172.21.15.47:34176) 2024-02-23T00:33:19.060 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.055+0000 7f35937fe700 1 -- 172.21.15.47:0/3180704938 learned_addr learned my addr 172.21.15.47:0/3180704938 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:19.060 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.055+0000 7f35937fe700 1 -- 172.21.15.47:0/3180704938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35940ff0b0 con 0x7f35940713e0 2024-02-23T00:33:19.061 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.055+0000 7f35937fe700 1 --2- 172.21.15.47:0/3180704938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35940713e0 0x7f35940fea80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f3584009730 tx=0x7f35840090f0).ready entity=mon.0 client_cookie=e5f2d7a8bc914e36 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:19.061 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.055+0000 7f35927fc700 1 -- 172.21.15.47:0/3180704938 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f358400b580 con 0x7f35940713e0 2024-02-23T00:33:19.061 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.055+0000 7f35927fc700 1 -- 172.21.15.47:0/3180704938 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7f358400bb50 con 0x7f35940713e0 2024-02-23T00:33:19.061 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.055+0000 7f35927fc700 1 -- 172.21.15.47:0/3180704938 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3584011ab0 con 0x7f35940713e0 2024-02-23T00:33:19.061 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.056+0000 7f3599d4f700 1 -- 172.21.15.47:0/3180704938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35940713e0 msgr2=0x7f35940fea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:19.061 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.056+0000 7f3599d4f700 1 --2- 172.21.15.47:0/3180704938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35940713e0 0x7f35940fea80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f3584009730 tx=0x7f35840090f0).stop 2024-02-23T00:33:19.061 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.056+0000 7f3599d4f700 1 -- 172.21.15.47:0/3180704938 shutdown_connections 2024-02-23T00:33:19.061 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.056+0000 7f3599d4f700 1 --2- 172.21.15.47:0/3180704938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35940713e0 0x7f35940fea80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:19.061 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.056+0000 7f3599d4f700 1 -- 172.21.15.47:0/3180704938 >> 172.21.15.47:0/3180704938 conn(0x7f35940f6540 msgr2=0x7f35940f8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:19.061 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.056+0000 7f3599d4f700 1 -- 172.21.15.47:0/3180704938 shutdown_connections 2024-02-23T00:33:19.061 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.056+0000 7f3599d4f700 1 -- 172.21.15.47:0/3180704938 wait complete. 2024-02-23T00:33:19.061 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.056+0000 7f3599d4f700 1 librados: starting msgr at 2024-02-23T00:33:19.061 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.056+0000 7f3599d4f700 1 librados: starting objecter 2024-02-23T00:33:19.062 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.057+0000 7f3599d4f700 1 Processor -- start 2024-02-23T00:33:19.062 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.057+0000 7f3599d4f700 1 -- start start 2024-02-23T00:33:19.062 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.057+0000 7f3599d4f700 1 librados: setting wanted keys 2024-02-23T00:33:19.062 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.057+0000 7f3599d4f700 1 librados: calling monclient init 2024-02-23T00:33:19.062 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.057+0000 7f3599d4f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35940713e0 0x7f359410aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:19.062 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.057+0000 7f3599d4f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359410af60 con 0x7f35940713e0 2024-02-23T00:33:19.062 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.057+0000 7f35937fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35940713e0 0x7f359410aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:19.062 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.057+0000 7f35937fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35940713e0 0x7f359410aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34192/0 (socket says 172.21.15.47:34192) 2024-02-23T00:33:19.062 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.057+0000 7f35937fe700 1 -- 172.21.15.47:0/522275049 learned_addr learned my addr 172.21.15.47:0/522275049 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:19.062 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.057+0000 7f35937fe700 1 -- 172.21.15.47:0/522275049 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3584009370 con 0x7f35940713e0 2024-02-23T00:33:19.063 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.058+0000 7f35937fe700 1 --2- 172.21.15.47:0/522275049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35940713e0 0x7f359410aa70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f3584000c00 tx=0x7f35840090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:19.063 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.058+0000 7f3590ff9700 10 client.?.objecter ms_handle_connect 0x7f35940713e0 2024-02-23T00:33:19.063 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.058+0000 7f3590ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:33:19.063 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.058+0000 7f3590ff9700 1 -- 172.21.15.47:0/522275049 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f358400b9e0 con 0x7f35940713e0 2024-02-23T00:33:19.063 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.058+0000 7f3590ff9700 1 -- 172.21.15.47:0/522275049 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7f3584011400 con 0x7f35940713e0 2024-02-23T00:33:19.064 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.058+0000 7f3590ff9700 1 -- 172.21.15.47:0/522275049 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35840238b0 con 0x7f35940713e0 2024-02-23T00:33:19.064 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.058+0000 7f3599d4f700 1 -- 172.21.15.47:0/522275049 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f359410b160 con 0x7f35940713e0 2024-02-23T00:33:19.064 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.058+0000 7f3599d4f700 10 client.14110.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:33:19.064 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.058+0000 7f3599d4f700 1 -- 172.21.15.47:0/522275049 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f359410b5f0 con 0x7f35940713e0 2024-02-23T00:33:19.064 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.058+0000 7f3599d4f700 1 librados: init done 2024-02-23T00:33:19.064 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.059+0000 7f3590ff9700 1 -- 172.21.15.47:0/522275049 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 38606+0+0 (secure 0 0 0) 0x7f3584023b20 con 0x7f35940713e0 2024-02-23T00:33:19.064 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.059+0000 7f3590ff9700 1 --2- 172.21.15.47:0/522275049 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7f3580033f60 0x7f3580036420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:19.064 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.059+0000 7f3590ff9700 1 -- 172.21.15.47:0/522275049 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f3584044df0 con 0x7f35940713e0 2024-02-23T00:33:19.064 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.059+0000 7f3590ff9700 10 client.14110.objecter ms_dispatch 0x7f3594001550 osd_map(1..1 src has 1..1) v4 2024-02-23T00:33:19.064 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.059+0000 7f3590ff9700 3 client.14110.objecter handle_osd_map got epochs [1,1] > 0 2024-02-23T00:33:19.064 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.059+0000 7f3590ff9700 3 client.14110.objecter handle_osd_map decoding full epoch 1 2024-02-23T00:33:19.064 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.059+0000 7f3590ff9700 20 client.14110.objecter dump_active .. 0 homeless 2024-02-23T00:33:19.064 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.059+0000 7f3592ffd700 1 --2- 172.21.15.47:0/522275049 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7f3580033f60 0x7f3580036420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:19.064 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.059+0000 7f3599d4f700 1 -- 172.21.15.47:0/522275049 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3594046ce0 con 0x7f35940713e0 2024-02-23T00:33:19.065 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.060+0000 7f3592ffd700 1 --2- 172.21.15.47:0/522275049 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7f3580033f60 0x7f3580036420 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f357c009780 tx=0x7f357c006cc0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:19.065 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.060+0000 7f3590ff9700 10 client.14110.objecter ms_handle_connect 0x7f3580033f60 2024-02-23T00:33:19.066 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.064+0000 7f3590ff9700 1 -- 172.21.15.47:0/522275049 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f358401e070 con 0x7f35940713e0 2024-02-23T00:33:19.311 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.302+0000 7f3599d4f700 1 -- 172.21.15.47:0/522275049 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f35940ffe70 con 0x7f35940713e0 2024-02-23T00:33:19.311 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.302+0000 7f3590ff9700 1 -- 172.21.15.47:0/522275049 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1241 (secure 0 0 0) 0x7f358401e020 con 0x7f35940713e0 2024-02-23T00:33:19.311 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.311 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: { 2024-02-23T00:33:19.311 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "fsid": "fce2617e-d1e2-11ee-95c0-87774f69a715", 2024-02-23T00:33:19.311 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "health": { 2024-02-23T00:33:19.311 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-02-23T00:33:19.311 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "checks": {}, 2024-02-23T00:33:19.312 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "mutes": [] 2024-02-23T00:33:19.312 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: }, 2024-02-23T00:33:19.312 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-02-23T00:33:19.312 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "quorum": [ 2024-02-23T00:33:19.312 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 0 2024-02-23T00:33:19.312 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: ], 2024-02-23T00:33:19.312 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "quorum_names": [ 2024-02-23T00:33:19.312 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "a" 2024-02-23T00:33:19.312 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: ], 2024-02-23T00:33:19.312 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "quorum_age": 15, 2024-02-23T00:33:19.312 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "monmap": { 2024-02-23T00:33:19.312 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-23T00:33:19.312 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2024-02-23T00:33:19.312 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "num_mons": 1 2024-02-23T00:33:19.312 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: }, 2024-02-23T00:33:19.313 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "osdmap": { 2024-02-23T00:33:19.313 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-23T00:33:19.313 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "num_osds": 0, 2024-02-23T00:33:19.313 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-02-23T00:33:19.313 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-02-23T00:33:19.313 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-02-23T00:33:19.313 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-02-23T00:33:19.313 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-02-23T00:33:19.313 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: }, 2024-02-23T00:33:19.313 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "pgmap": { 2024-02-23T00:33:19.313 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-02-23T00:33:19.313 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-02-23T00:33:19.313 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "num_pools": 0, 2024-02-23T00:33:19.313 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "num_objects": 0, 2024-02-23T00:33:19.313 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-02-23T00:33:19.314 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-02-23T00:33:19.314 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-02-23T00:33:19.314 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "bytes_total": 0 2024-02-23T00:33:19.314 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: }, 2024-02-23T00:33:19.314 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "fsmap": { 2024-02-23T00:33:19.314 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-23T00:33:19.314 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "by_rank": [], 2024-02-23T00:33:19.314 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "up:standby": 0 2024-02-23T00:33:19.314 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: }, 2024-02-23T00:33:19.314 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "mgrmap": { 2024-02-23T00:33:19.314 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "available": true, 2024-02-23T00:33:19.314 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-02-23T00:33:19.314 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "modules": [ 2024-02-23T00:33:19.314 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "iostat", 2024-02-23T00:33:19.314 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "nfs", 2024-02-23T00:33:19.315 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "restful" 2024-02-23T00:33:19.315 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: ], 2024-02-23T00:33:19.315 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "services": {} 2024-02-23T00:33:19.315 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: }, 2024-02-23T00:33:19.315 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "servicemap": { 2024-02-23T00:33:19.315 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-23T00:33:19.315 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "modified": "2024-02-23T00:32:55.399919+0000", 2024-02-23T00:33:19.315 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "services": {} 2024-02-23T00:33:19.315 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: }, 2024-02-23T00:33:19.315 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "progress_events": {} 2024-02-23T00:33:19.315 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: } 2024-02-23T00:33:19.315 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.304+0000 7f3599d4f700 10 librados: watch_flush enter 2024-02-23T00:33:19.315 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.304+0000 7f3599d4f700 10 librados: watch_flush exit 2024-02-23T00:33:19.315 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.304+0000 7f3599d4f700 20 client.14110.objecter shutdown clearing up homeless session... 2024-02-23T00:33:19.315 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.304+0000 7f3599d4f700 10 client.14110.objecter successfully canceled tick 2024-02-23T00:33:19.316 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.304+0000 7f3599d4f700 1 -- 172.21.15.47:0/522275049 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7f3580033f60 msgr2=0x7f3580036420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:19.316 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.304+0000 7f3599d4f700 1 --2- 172.21.15.47:0/522275049 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7f3580033f60 0x7f3580036420 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f357c009780 tx=0x7f357c006cc0).stop 2024-02-23T00:33:19.316 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.304+0000 7f3599d4f700 1 -- 172.21.15.47:0/522275049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35940713e0 msgr2=0x7f359410aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:19.316 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.304+0000 7f3599d4f700 1 --2- 172.21.15.47:0/522275049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35940713e0 0x7f359410aa70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f3584000c00 tx=0x7f35840090f0).stop 2024-02-23T00:33:19.316 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.304+0000 7f3599d4f700 1 -- 172.21.15.47:0/522275049 shutdown_connections 2024-02-23T00:33:19.316 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.304+0000 7f3599d4f700 1 --2- 172.21.15.47:0/522275049 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7f3580033f60 0x7f3580036420 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:19.316 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.304+0000 7f3599d4f700 1 --2- 172.21.15.47:0/522275049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35940713e0 0x7f359410aa70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:19.316 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.304+0000 7f3599d4f700 1 -- 172.21.15.47:0/522275049 >> 172.21.15.47:0/522275049 conn(0x7f35940f6540 msgr2=0x7f35940f7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:19.316 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.304+0000 7f3599d4f700 1 -- 172.21.15.47:0/522275049 shutdown_connections 2024-02-23T00:33:19.316 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.304+0000 7f3599d4f700 1 -- 172.21.15.47:0/522275049 wait complete. 2024-02-23T00:33:19.316 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:19.304+0000 7f3599d4f700 1 librados: shutdown 2024-02-23T00:33:19.696 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:19.303443+0000 mon.a (mon.0) 24 : audit [DBG] from='client.? 172.21.15.47:0/522275049' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-23T00:33:19.974 INFO:teuthology.orchestra.run.smithi047.stderr:mgr is available 2024-02-23T00:33:20.600 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.594+0000 7f391ab6f700 1 Processor -- start 2024-02-23T00:33:20.600 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.594+0000 7f391ab6f700 1 -- start start 2024-02-23T00:33:20.600 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.594+0000 7f391ab6f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3914071380 0x7f3914100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:20.600 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.594+0000 7f391ab6f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39140fc8d0 con 0x7f3914071380 2024-02-23T00:33:20.600 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.594+0000 7f391890b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3914071380 0x7f3914100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:20.600 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.594+0000 7f391890b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3914071380 0x7f3914100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34194/0 (socket says 172.21.15.47:34194) 2024-02-23T00:33:20.600 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.594+0000 7f391890b700 1 -- 172.21.15.47:0/2411160906 learned_addr learned my addr 172.21.15.47:0/2411160906 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:20.601 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.594+0000 7f391890b700 1 -- 172.21.15.47:0/2411160906 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39141013d0 con 0x7f3914071380 2024-02-23T00:33:20.601 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.595+0000 7f391890b700 1 --2- 172.21.15.47:0/2411160906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3914071380 0x7f3914100ee0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f3908009730 tx=0x7f39080090f0).ready entity=mon.0 client_cookie=b714d09912c56036 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:20.601 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.595+0000 7f39137fe700 1 -- 172.21.15.47:0/2411160906 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f390800b580 con 0x7f3914071380 2024-02-23T00:33:20.601 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.595+0000 7f39137fe700 1 -- 172.21.15.47:0/2411160906 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7f390800bb50 con 0x7f3914071380 2024-02-23T00:33:20.601 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.595+0000 7f39137fe700 1 -- 172.21.15.47:0/2411160906 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3908011ab0 con 0x7f3914071380 2024-02-23T00:33:20.601 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.595+0000 7f391ab6f700 1 -- 172.21.15.47:0/2411160906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3914071380 msgr2=0x7f3914100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:20.601 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.595+0000 7f391ab6f700 1 --2- 172.21.15.47:0/2411160906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3914071380 0x7f3914100ee0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f3908009730 tx=0x7f39080090f0).stop 2024-02-23T00:33:20.601 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.595+0000 7f391ab6f700 1 -- 172.21.15.47:0/2411160906 shutdown_connections 2024-02-23T00:33:20.601 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.595+0000 7f391ab6f700 1 --2- 172.21.15.47:0/2411160906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3914071380 0x7f3914100ee0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:20.601 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.595+0000 7f391ab6f700 1 -- 172.21.15.47:0/2411160906 >> 172.21.15.47:0/2411160906 conn(0x7f39140f6540 msgr2=0x7f39140f8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:20.601 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.595+0000 7f391ab6f700 1 -- 172.21.15.47:0/2411160906 shutdown_connections 2024-02-23T00:33:20.601 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.595+0000 7f391ab6f700 1 -- 172.21.15.47:0/2411160906 wait complete. 2024-02-23T00:33:20.601 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.596+0000 7f391ab6f700 1 librados: starting msgr at 2024-02-23T00:33:20.601 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.596+0000 7f391ab6f700 1 librados: starting objecter 2024-02-23T00:33:20.601 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.596+0000 7f391ab6f700 1 Processor -- start 2024-02-23T00:33:20.602 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.596+0000 7f391ab6f700 1 -- start start 2024-02-23T00:33:20.602 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.596+0000 7f391ab6f700 1 librados: setting wanted keys 2024-02-23T00:33:20.602 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.596+0000 7f391ab6f700 1 librados: calling monclient init 2024-02-23T00:33:20.602 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.596+0000 7f391ab6f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3914071380 0x7f391410cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:20.602 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.596+0000 7f391ab6f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391410d240 con 0x7f3914071380 2024-02-23T00:33:20.602 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.596+0000 7f391890b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3914071380 0x7f391410cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:20.602 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.596+0000 7f391890b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3914071380 0x7f391410cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34206/0 (socket says 172.21.15.47:34206) 2024-02-23T00:33:20.602 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.596+0000 7f391890b700 1 -- 172.21.15.47:0/3541715097 learned_addr learned my addr 172.21.15.47:0/3541715097 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:20.602 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.597+0000 7f391890b700 1 -- 172.21.15.47:0/3541715097 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3908009370 con 0x7f3914071380 2024-02-23T00:33:20.602 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.597+0000 7f391890b700 1 --2- 172.21.15.47:0/3541715097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3914071380 0x7f391410cd50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f390800bef0 tx=0x7f39080090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:20.602 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.597+0000 7f3911ffb700 10 client.?.objecter ms_handle_connect 0x7f3914071380 2024-02-23T00:33:20.602 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.597+0000 7f3911ffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:33:20.602 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.597+0000 7f3911ffb700 1 -- 172.21.15.47:0/3541715097 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3908011a70 con 0x7f3914071380 2024-02-23T00:33:20.602 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.597+0000 7f3911ffb700 1 -- 172.21.15.47:0/3541715097 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7f390801aab0 con 0x7f3914071380 2024-02-23T00:33:20.602 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.597+0000 7f391ab6f700 1 -- 172.21.15.47:0/3541715097 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f391410d440 con 0x7f3914071380 2024-02-23T00:33:20.603 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.597+0000 7f3911ffb700 1 -- 172.21.15.47:0/3541715097 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3908023910 con 0x7f3914071380 2024-02-23T00:33:20.603 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.597+0000 7f391ab6f700 10 client.14112.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:33:20.603 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.597+0000 7f391ab6f700 1 -- 172.21.15.47:0/3541715097 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f391410e050 con 0x7f3914071380 2024-02-23T00:33:20.603 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.597+0000 7f391ab6f700 1 librados: init done 2024-02-23T00:33:20.603 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.598+0000 7f3911ffb700 1 -- 172.21.15.47:0/3541715097 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 38606+0+0 (secure 0 0 0) 0x7f3908023a70 con 0x7f3914071380 2024-02-23T00:33:20.603 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.598+0000 7f3911ffb700 1 --2- 172.21.15.47:0/3541715097 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7f3904033f60 0x7f3904036420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:20.604 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.598+0000 7f3911ffb700 1 -- 172.21.15.47:0/3541715097 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f3908044b00 con 0x7f3914071380 2024-02-23T00:33:20.604 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.598+0000 7f3911ffb700 10 client.14112.objecter ms_dispatch 0x7f3914001550 osd_map(1..1 src has 1..1) v4 2024-02-23T00:33:20.604 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.598+0000 7f3911ffb700 3 client.14112.objecter handle_osd_map got epochs [1,1] > 0 2024-02-23T00:33:20.604 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.598+0000 7f3911ffb700 3 client.14112.objecter handle_osd_map decoding full epoch 1 2024-02-23T00:33:20.604 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.598+0000 7f3911ffb700 20 client.14112.objecter dump_active .. 0 homeless 2024-02-23T00:33:20.604 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.598+0000 7f391ab6f700 1 -- 172.21.15.47:0/3541715097 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f391410d8d0 con 0x7f3914071380 2024-02-23T00:33:20.604 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.598+0000 7f3913fff700 1 --2- 172.21.15.47:0/3541715097 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7f3904033f60 0x7f3904036420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:20.605 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.599+0000 7f3911ffb700 10 client.14112.objecter ms_handle_connect 0x7f3904033f60 2024-02-23T00:33:20.605 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.599+0000 7f3913fff700 1 --2- 172.21.15.47:0/3541715097 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7f3904033f60 0x7f3904036420 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f3900009780 tx=0x7f3900006cc0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:20.605 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.603+0000 7f3911ffb700 1 -- 172.21.15.47:0/3541715097 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f391410d8d0 con 0x7f3914071380 2024-02-23T00:33:20.781 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.778+0000 7f391ab6f700 1 -- 172.21.15.47:0/3541715097 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f391410d8d0 con 0x7f3914071380 2024-02-23T00:33:20.784 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.779+0000 7f3911ffb700 1 -- 172.21.15.47:0/3541715097 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+307 (secure 0 0 0) 0x7f391410d8d0 con 0x7f3914071380 2024-02-23T00:33:20.784 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.784 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: [global] 2024-02-23T00:33:20.784 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: fsid = fce2617e-d1e2-11ee-95c0-87774f69a715 2024-02-23T00:33:20.784 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: log_to_journald = false 2024-02-23T00:33:20.785 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: mon_osd_allow_pg_remap = true 2024-02-23T00:33:20.785 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: mon_osd_allow_primary_affinity = true 2024-02-23T00:33:20.785 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: mon_warn_on_no_sortbitwise = false 2024-02-23T00:33:20.785 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: osd_crush_chooseleaf_type = 0 2024-02-23T00:33:20.785 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.785 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: [mgr] 2024-02-23T00:33:20.785 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: mgr/telemetry/nag = false 2024-02-23T00:33:20.785 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.785 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: [osd] 2024-02-23T00:33:20.785 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: osd_map_max_advance = 10 2024-02-23T00:33:20.785 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: osd_sloppy_crc = true 2024-02-23T00:33:20.785 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.781+0000 7f391ab6f700 10 librados: watch_flush enter 2024-02-23T00:33:20.785 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.781+0000 7f391ab6f700 10 librados: watch_flush exit 2024-02-23T00:33:20.785 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.781+0000 7f391ab6f700 20 client.14112.objecter shutdown clearing up homeless session... 2024-02-23T00:33:20.786 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.781+0000 7f391ab6f700 10 client.14112.objecter successfully canceled tick 2024-02-23T00:33:20.786 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.781+0000 7f391ab6f700 1 -- 172.21.15.47:0/3541715097 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7f3904033f60 msgr2=0x7f3904036420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:20.786 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.781+0000 7f391ab6f700 1 --2- 172.21.15.47:0/3541715097 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7f3904033f60 0x7f3904036420 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f3900009780 tx=0x7f3900006cc0).stop 2024-02-23T00:33:20.786 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.781+0000 7f391ab6f700 1 -- 172.21.15.47:0/3541715097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3914071380 msgr2=0x7f391410cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:20.786 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.781+0000 7f391ab6f700 1 --2- 172.21.15.47:0/3541715097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3914071380 0x7f391410cd50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f390800bef0 tx=0x7f39080090f0).stop 2024-02-23T00:33:20.786 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.782+0000 7f391ab6f700 1 -- 172.21.15.47:0/3541715097 shutdown_connections 2024-02-23T00:33:20.786 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.782+0000 7f391ab6f700 1 --2- 172.21.15.47:0/3541715097 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7f3904033f60 0x7f3904036420 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:20.786 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.782+0000 7f391ab6f700 1 --2- 172.21.15.47:0/3541715097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3914071380 0x7f391410cd50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:20.786 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.782+0000 7f391ab6f700 1 -- 172.21.15.47:0/3541715097 >> 172.21.15.47:0/3541715097 conn(0x7f39140f6540 msgr2=0x7f39140f7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:20.786 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.782+0000 7f391ab6f700 1 -- 172.21.15.47:0/3541715097 shutdown_connections 2024-02-23T00:33:20.786 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.782+0000 7f391ab6f700 1 -- 172.21.15.47:0/3541715097 wait complete. 2024-02-23T00:33:20.786 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:20.782+0000 7f391ab6f700 1 librados: shutdown 2024-02-23T00:33:21.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:20.780460+0000 mon.a (mon.0) 25 : audit [INF] from='client.? 172.21.15.47:0/3541715097' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-23T00:33:21.514 INFO:teuthology.orchestra.run.smithi047.stderr:Enabling cephadm module... 2024-02-23T00:33:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.120+0000 7fc96a93f700 1 Processor -- start 2024-02-23T00:33:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.120+0000 7fc96a93f700 1 -- start start 2024-02-23T00:33:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.120+0000 7fc96a93f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc964070bc0 0x7fc9640fea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.120+0000 7fc96a93f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9640fef60 con 0x7fc964070bc0 2024-02-23T00:33:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.120+0000 7fc963fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc964070bc0 0x7fc9640fea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.120+0000 7fc963fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc964070bc0 0x7fc9640fea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34220/0 (socket says 172.21.15.47:34220) 2024-02-23T00:33:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.120+0000 7fc963fff700 1 -- 172.21.15.47:0/2568899628 learned_addr learned my addr 172.21.15.47:0/2568899628 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.120+0000 7fc963fff700 1 -- 172.21.15.47:0/2568899628 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9640ff0a0 con 0x7fc964070bc0 2024-02-23T00:33:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.120+0000 7fc963fff700 1 --2- 172.21.15.47:0/2568899628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc964070bc0 0x7fc9640fea70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7fc94c009730 tx=0x7fc94c0090f0).ready entity=mon.0 client_cookie=1dbfe82acd477d99 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.120+0000 7fc962ffd700 1 -- 172.21.15.47:0/2568899628 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc94c00b580 con 0x7fc964070bc0 2024-02-23T00:33:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.121+0000 7fc962ffd700 1 -- 172.21.15.47:0/2568899628 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7fc94c00bb50 con 0x7fc964070bc0 2024-02-23T00:33:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.121+0000 7fc962ffd700 1 -- 172.21.15.47:0/2568899628 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc94c011ab0 con 0x7fc964070bc0 2024-02-23T00:33:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.121+0000 7fc96a93f700 1 -- 172.21.15.47:0/2568899628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc964070bc0 msgr2=0x7fc9640fea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.121+0000 7fc96a93f700 1 --2- 172.21.15.47:0/2568899628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc964070bc0 0x7fc9640fea70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7fc94c009730 tx=0x7fc94c0090f0).stop 2024-02-23T00:33:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.121+0000 7fc96a93f700 1 -- 172.21.15.47:0/2568899628 shutdown_connections 2024-02-23T00:33:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.121+0000 7fc96a93f700 1 --2- 172.21.15.47:0/2568899628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc964070bc0 0x7fc9640fea70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.121+0000 7fc96a93f700 1 -- 172.21.15.47:0/2568899628 >> 172.21.15.47:0/2568899628 conn(0x7fc9640f6530 msgr2=0x7fc9640f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.121+0000 7fc96a93f700 1 -- 172.21.15.47:0/2568899628 shutdown_connections 2024-02-23T00:33:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.121+0000 7fc96a93f700 1 -- 172.21.15.47:0/2568899628 wait complete. 2024-02-23T00:33:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.122+0000 7fc96a93f700 1 librados: starting msgr at 2024-02-23T00:33:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.122+0000 7fc96a93f700 1 librados: starting objecter 2024-02-23T00:33:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.122+0000 7fc96a93f700 1 Processor -- start 2024-02-23T00:33:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.122+0000 7fc96a93f700 1 -- start start 2024-02-23T00:33:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.122+0000 7fc96a93f700 1 librados: setting wanted keys 2024-02-23T00:33:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.122+0000 7fc96a93f700 1 librados: calling monclient init 2024-02-23T00:33:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.122+0000 7fc96a93f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc964070bc0 0x7fc96410aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.122+0000 7fc96a93f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96410af50 con 0x7fc964070bc0 2024-02-23T00:33:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.122+0000 7fc963fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc964070bc0 0x7fc96410aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.122+0000 7fc963fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc964070bc0 0x7fc96410aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34230/0 (socket says 172.21.15.47:34230) 2024-02-23T00:33:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.122+0000 7fc963fff700 1 -- 172.21.15.47:0/1123491492 learned_addr learned my addr 172.21.15.47:0/1123491492 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.122+0000 7fc963fff700 1 -- 172.21.15.47:0/1123491492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc94c009370 con 0x7fc964070bc0 2024-02-23T00:33:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.122+0000 7fc963fff700 1 --2- 172.21.15.47:0/1123491492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc964070bc0 0x7fc96410aa60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fc94c012f20 tx=0x7fc94c0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.122+0000 7fc9617fa700 10 client.?.objecter ms_handle_connect 0x7fc964070bc0 2024-02-23T00:33:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.122+0000 7fc9617fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:33:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.123+0000 7fc9617fa700 1 -- 172.21.15.47:0/1123491492 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc94c011a70 con 0x7fc964070bc0 2024-02-23T00:33:22.140 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.123+0000 7fc9617fa700 1 -- 172.21.15.47:0/1123491492 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7fc94c01aab0 con 0x7fc964070bc0 2024-02-23T00:33:22.140 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.123+0000 7fc96a93f700 1 -- 172.21.15.47:0/1123491492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc96410b150 con 0x7fc964070bc0 2024-02-23T00:33:22.140 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.123+0000 7fc9617fa700 1 -- 172.21.15.47:0/1123491492 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc94c023960 con 0x7fc964070bc0 2024-02-23T00:33:22.140 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.123+0000 7fc96a93f700 10 client.14114.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:33:22.140 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.123+0000 7fc96a93f700 1 -- 172.21.15.47:0/1123491492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc96410b5e0 con 0x7fc964070bc0 2024-02-23T00:33:22.140 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.123+0000 7fc96a93f700 1 librados: init done 2024-02-23T00:33:22.140 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.123+0000 7fc9617fa700 1 -- 172.21.15.47:0/1123491492 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 38606+0+0 (secure 0 0 0) 0x7fc94c023ac0 con 0x7fc964070bc0 2024-02-23T00:33:22.140 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.123+0000 7fc9617fa700 1 --2- 172.21.15.47:0/1123491492 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7fc950033f60 0x7fc950036420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:22.140 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.124+0000 7fc9617fa700 1 -- 172.21.15.47:0/1123491492 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7fc94c044ad0 con 0x7fc964070bc0 2024-02-23T00:33:22.140 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.124+0000 7fc9617fa700 10 client.14114.objecter ms_dispatch 0x7fc964001550 osd_map(1..1 src has 1..1) v4 2024-02-23T00:33:22.140 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.124+0000 7fc9617fa700 3 client.14114.objecter handle_osd_map got epochs [1,1] > 0 2024-02-23T00:33:22.141 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.124+0000 7fc9617fa700 3 client.14114.objecter handle_osd_map decoding full epoch 1 2024-02-23T00:33:22.141 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.124+0000 7fc9617fa700 20 client.14114.objecter dump_active .. 0 homeless 2024-02-23T00:33:22.141 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.124+0000 7fc9637fe700 1 --2- 172.21.15.47:0/1123491492 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7fc950033f60 0x7fc950036420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:22.141 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.124+0000 7fc96a93f700 1 -- 172.21.15.47:0/1123491492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc96405a310 con 0x7fc964070bc0 2024-02-23T00:33:22.141 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.124+0000 7fc9617fa700 10 client.14114.objecter ms_handle_connect 0x7fc950033f60 2024-02-23T00:33:22.141 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.124+0000 7fc9637fe700 1 --2- 172.21.15.47:0/1123491492 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7fc950033f60 0x7fc950036420 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fc954009780 tx=0x7fc954006cc0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:22.141 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.128+0000 7fc9617fa700 1 -- 172.21.15.47:0/1123491492 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fc94c01e020 con 0x7fc964070bc0 2024-02-23T00:33:22.348 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.346+0000 7fc96a93f700 1 -- 172.21.15.47:0/1123491492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fc964046ce0 con 0x7fc964070bc0 2024-02-23T00:33:22.401 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.399+0000 7fc9617fa700 1 -- 172.21.15.47:0/1123491492 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 5) v1 ==== 38617+0+0 (secure 0 0 0) 0x7fc94c02c910 con 0x7fc964070bc0 2024-02-23T00:33:22.402 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.400+0000 7fc9617fa700 1 -- 172.21.15.47:0/1123491492 <== mon.0 v2:172.21.15.47:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fc94c044130 con 0x7fc964070bc0 2024-02-23T00:33:22.405 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.402+0000 7fc96a93f700 10 librados: watch_flush enter 2024-02-23T00:33:22.405 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.402+0000 7fc96a93f700 10 librados: watch_flush exit 2024-02-23T00:33:22.405 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.402+0000 7fc96a93f700 20 client.14114.objecter shutdown clearing up homeless session... 2024-02-23T00:33:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.402+0000 7fc96a93f700 10 client.14114.objecter successfully canceled tick 2024-02-23T00:33:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.402+0000 7fc96a93f700 1 -- 172.21.15.47:0/1123491492 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7fc950033f60 msgr2=0x7fc950036420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.402+0000 7fc96a93f700 1 --2- 172.21.15.47:0/1123491492 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7fc950033f60 0x7fc950036420 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fc954009780 tx=0x7fc954006cc0).stop 2024-02-23T00:33:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.402+0000 7fc96a93f700 1 -- 172.21.15.47:0/1123491492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc964070bc0 msgr2=0x7fc96410aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.402+0000 7fc96a93f700 1 --2- 172.21.15.47:0/1123491492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc964070bc0 0x7fc96410aa60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fc94c012f20 tx=0x7fc94c0090f0).stop 2024-02-23T00:33:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.402+0000 7fc96a93f700 1 -- 172.21.15.47:0/1123491492 shutdown_connections 2024-02-23T00:33:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.402+0000 7fc96a93f700 1 --2- 172.21.15.47:0/1123491492 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7fc950033f60 0x7fc950036420 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.402+0000 7fc96a93f700 1 --2- 172.21.15.47:0/1123491492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc964070bc0 0x7fc96410aa60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.402+0000 7fc96a93f700 1 -- 172.21.15.47:0/1123491492 >> 172.21.15.47:0/1123491492 conn(0x7fc9640f6530 msgr2=0x7fc9640f7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.402+0000 7fc96a93f700 1 -- 172.21.15.47:0/1123491492 shutdown_connections 2024-02-23T00:33:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.402+0000 7fc96a93f700 1 -- 172.21.15.47:0/1123491492 wait complete. 2024-02-23T00:33:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:22.402+0000 7fc96a93f700 1 librados: shutdown 2024-02-23T00:33:22.801 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:22.348254+0000 mon.a (mon.0) 26 : audit [INF] from='client.? 172.21.15.47:0/1123491492' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-23T00:33:22.802 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ignoring --setuser ceph since I am not root 2024-02-23T00:33:22.802 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ignoring --setgroup ceph since I am not root 2024-02-23T00:33:22.802 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:22.557+0000 7fa56f4f4680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-23T00:33:22.802 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:22.641+0000 7fa56f4f4680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-23T00:33:23.056 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:23.021+0000 7fa56f4f4680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-23T00:33:23.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:22.401439+0000 mon.a (mon.0) 27 : audit [INF] from='client.? 172.21.15.47:0/1123491492' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-23T00:33:23.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:33:22.401483+0000 mon.a (mon.0) 28 : cluster [DBG] mgrmap e5: y(active, since 10s) 2024-02-23T00:33:23.813 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.810+0000 7f0f55164700 1 Processor -- start 2024-02-23T00:33:23.813 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.810+0000 7f0f55164700 1 -- start start 2024-02-23T00:33:23.813 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.810+0000 7f0f55164700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f50069960 0x7f0f50069d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:23.813 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.810+0000 7f0f55164700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5006a270 con 0x7f0f50069960 2024-02-23T00:33:23.813 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.810+0000 7f0f4ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f50069960 0x7f0f50069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:23.813 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.810+0000 7f0f4ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f50069960 0x7f0f50069d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34254/0 (socket says 172.21.15.47:34254) 2024-02-23T00:33:23.813 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.810+0000 7f0f4ed9d700 1 -- 172.21.15.47:0/3124772576 learned_addr learned my addr 172.21.15.47:0/3124772576 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:23.813 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.811+0000 7f0f4ed9d700 1 -- 172.21.15.47:0/3124772576 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5006a3b0 con 0x7f0f50069960 2024-02-23T00:33:23.814 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.811+0000 7f0f4ed9d700 1 --2- 172.21.15.47:0/3124772576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f50069960 0x7f0f50069d80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f0f48009e50 tx=0x7f0f48009840).ready entity=mon.0 client_cookie=d892aedf29e659bf server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:23.814 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.812+0000 7f0f4dd9b700 1 -- 172.21.15.47:0/3124772576 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f4800e070 con 0x7f0f50069960 2024-02-23T00:33:23.814 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.812+0000 7f0f4dd9b700 1 -- 172.21.15.47:0/3124772576 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7f0f4800be90 con 0x7f0f50069960 2024-02-23T00:33:23.814 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.812+0000 7f0f4dd9b700 1 -- 172.21.15.47:0/3124772576 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f48013d80 con 0x7f0f50069960 2024-02-23T00:33:23.815 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.813+0000 7f0f55164700 1 -- 172.21.15.47:0/3124772576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f50069960 msgr2=0x7f0f50069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:23.815 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.813+0000 7f0f55164700 1 --2- 172.21.15.47:0/3124772576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f50069960 0x7f0f50069d80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f0f48009e50 tx=0x7f0f48009840).stop 2024-02-23T00:33:23.815 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.813+0000 7f0f55164700 1 -- 172.21.15.47:0/3124772576 shutdown_connections 2024-02-23T00:33:23.815 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.813+0000 7f0f55164700 1 --2- 172.21.15.47:0/3124772576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f50069960 0x7f0f50069d80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:23.815 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.813+0000 7f0f55164700 1 -- 172.21.15.47:0/3124772576 >> 172.21.15.47:0/3124772576 conn(0x7f0f50063e40 msgr2=0x7f0f50066260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:23.815 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.813+0000 7f0f55164700 1 -- 172.21.15.47:0/3124772576 shutdown_connections 2024-02-23T00:33:23.818 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.813+0000 7f0f55164700 1 -- 172.21.15.47:0/3124772576 wait complete. 2024-02-23T00:33:23.818 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.814+0000 7f0f55164700 1 librados: starting msgr at 2024-02-23T00:33:23.818 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.814+0000 7f0f55164700 1 librados: starting objecter 2024-02-23T00:33:23.818 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.814+0000 7f0f55164700 1 Processor -- start 2024-02-23T00:33:23.818 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.815+0000 7f0f55164700 1 -- start start 2024-02-23T00:33:23.818 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.815+0000 7f0f55164700 1 librados: setting wanted keys 2024-02-23T00:33:23.818 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.815+0000 7f0f55164700 1 librados: calling monclient init 2024-02-23T00:33:23.818 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.815+0000 7f0f55164700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f50069960 0x7f0f50112480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:23.819 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.815+0000 7f0f55164700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f50112970 con 0x7f0f50069960 2024-02-23T00:33:23.819 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.815+0000 7f0f4ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f50069960 0x7f0f50112480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:23.819 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.815+0000 7f0f4ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f50069960 0x7f0f50112480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34260/0 (socket says 172.21.15.47:34260) 2024-02-23T00:33:23.819 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.815+0000 7f0f4ed9d700 1 -- 172.21.15.47:0/3786068239 learned_addr learned my addr 172.21.15.47:0/3786068239 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:23.819 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.816+0000 7f0f4ed9d700 1 -- 172.21.15.47:0/3786068239 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f48009ae0 con 0x7f0f50069960 2024-02-23T00:33:23.819 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.816+0000 7f0f4ed9d700 1 --2- 172.21.15.47:0/3786068239 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f50069960 0x7f0f50112480 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f0f48000f80 tx=0x7f0f48009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:23.819 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.816+0000 7f0f37fff700 10 client.?.objecter ms_handle_connect 0x7f0f50069960 2024-02-23T00:33:23.819 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.816+0000 7f0f37fff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:33:23.820 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.817+0000 7f0f37fff700 1 -- 172.21.15.47:0/3786068239 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f4800e040 con 0x7f0f50069960 2024-02-23T00:33:23.820 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.817+0000 7f0f37fff700 1 -- 172.21.15.47:0/3786068239 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7f0f48013be0 con 0x7f0f50069960 2024-02-23T00:33:23.820 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.817+0000 7f0f37fff700 1 -- 172.21.15.47:0/3786068239 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f4801c3f0 con 0x7f0f50069960 2024-02-23T00:33:23.820 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.817+0000 7f0f55164700 1 -- 172.21.15.47:0/3786068239 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f50112b70 con 0x7f0f50069960 2024-02-23T00:33:23.820 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.817+0000 7f0f55164700 10 client.14118.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:33:23.821 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.817+0000 7f0f55164700 1 -- 172.21.15.47:0/3786068239 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f50113000 con 0x7f0f50069960 2024-02-23T00:33:23.821 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.817+0000 7f0f55164700 1 librados: init done 2024-02-23T00:33:23.822 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.818+0000 7f0f37fff700 1 -- 172.21.15.47:0/3786068239 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 38617+0+0 (secure 0 0 0) 0x7f0f48003040 con 0x7f0f50069960 2024-02-23T00:33:23.822 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.819+0000 7f0f37fff700 1 --2- 172.21.15.47:0/3786068239 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7f0f38034090 0x7f0f38036550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:23.822 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.819+0000 7f0f37fff700 1 -- 172.21.15.47:0/3786068239 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f0f48043eb0 con 0x7f0f50069960 2024-02-23T00:33:23.822 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.819+0000 7f0f37fff700 10 client.14118.objecter ms_dispatch 0x7f0f50001550 osd_map(1..1 src has 1..1) v4 2024-02-23T00:33:23.822 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.819+0000 7f0f37fff700 3 client.14118.objecter handle_osd_map got epochs [1,1] > 0 2024-02-23T00:33:23.822 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.819+0000 7f0f37fff700 3 client.14118.objecter handle_osd_map decoding full epoch 1 2024-02-23T00:33:23.822 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.819+0000 7f0f37fff700 20 client.14118.objecter dump_active .. 0 homeless 2024-02-23T00:33:23.822 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.819+0000 7f0f4e59c700 1 -- 172.21.15.47:0/3786068239 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7f0f38034090 msgr2=0x7f0f38036550 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/2409691510 2024-02-23T00:33:23.822 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.819+0000 7f0f4e59c700 1 --2- 172.21.15.47:0/3786068239 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7f0f38034090 0x7f0f38036550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-02-23T00:33:23.823 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.820+0000 7f0f55164700 1 -- 172.21.15.47:0/3786068239 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f501132b0 con 0x7f0f50069960 2024-02-23T00:33:23.827 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:23.825+0000 7f0f37fff700 1 -- 172.21.15.47:0/3786068239 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f0f4801c970 con 0x7f0f50069960 2024-02-23T00:33:24.022 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:24.020+0000 7f0f4e59c700 1 -- 172.21.15.47:0/3786068239 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7f0f38034090 msgr2=0x7f0f38036550 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/2409691510 2024-02-23T00:33:24.022 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:24.020+0000 7f0f4e59c700 1 --2- 172.21.15.47:0/3786068239 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7f0f38034090 0x7f0f38036550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-02-23T00:33:24.042 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:24.040+0000 7f0f55164700 1 -- 172.21.15.47:0/3786068239 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f0f50068b00 con 0x7f0f50069960 2024-02-23T00:33:24.042 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:24.040+0000 7f0f37fff700 1 -- 172.21.15.47:0/3786068239 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f0f48025e20 con 0x7f0f50069960 2024-02-23T00:33:24.043 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: { 2024-02-23T00:33:24.043 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "epoch": 5, 2024-02-23T00:33:24.043 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "available": true, 2024-02-23T00:33:24.043 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "active_name": "y", 2024-02-23T00:33:24.043 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "num_standby": 0 2024-02-23T00:33:24.043 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: } 2024-02-23T00:33:24.045 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:24.042+0000 7f0f55164700 10 librados: watch_flush enter 2024-02-23T00:33:24.045 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:24.042+0000 7f0f55164700 10 librados: watch_flush exit 2024-02-23T00:33:24.045 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:24.042+0000 7f0f55164700 20 client.14118.objecter shutdown clearing up homeless session... 2024-02-23T00:33:24.045 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:24.042+0000 7f0f55164700 10 client.14118.objecter successfully canceled tick 2024-02-23T00:33:24.045 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:24.042+0000 7f0f55164700 1 -- 172.21.15.47:0/3786068239 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7f0f38034090 msgr2=0x7f0f38036550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-23T00:33:24.046 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:24.042+0000 7f0f55164700 1 --2- 172.21.15.47:0/3786068239 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7f0f38034090 0x7f0f38036550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:33:24.046 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:24.042+0000 7f0f55164700 1 -- 172.21.15.47:0/3786068239 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f50069960 msgr2=0x7f0f50112480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:24.046 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:24.042+0000 7f0f55164700 1 --2- 172.21.15.47:0/3786068239 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f50069960 0x7f0f50112480 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f0f48000f80 tx=0x7f0f48009840).stop 2024-02-23T00:33:24.046 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:24.042+0000 7f0f55164700 1 -- 172.21.15.47:0/3786068239 shutdown_connections 2024-02-23T00:33:24.046 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:24.042+0000 7f0f55164700 1 --2- 172.21.15.47:0/3786068239 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7f0f38034090 0x7f0f38036550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:33:24.046 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:24.042+0000 7f0f55164700 1 --2- 172.21.15.47:0/3786068239 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f50069960 0x7f0f50112480 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:24.046 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:24.042+0000 7f0f55164700 1 -- 172.21.15.47:0/3786068239 >> 172.21.15.47:0/3786068239 conn(0x7f0f50063e40 msgr2=0x7f0f50105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:24.046 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:24.042+0000 7f0f55164700 1 -- 172.21.15.47:0/3786068239 shutdown_connections 2024-02-23T00:33:24.046 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:24.042+0000 7f0f55164700 1 -- 172.21.15.47:0/3786068239 wait complete. 2024-02-23T00:33:24.046 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:24.042+0000 7f0f55164700 1 librados: shutdown 2024-02-23T00:33:24.127 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:23.817+0000 7fa56f4f4680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-23T00:33:24.127 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:23.902+0000 7fa56f4f4680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-23T00:33:24.686 INFO:teuthology.orchestra.run.smithi047.stderr:Waiting for the mgr to restart... 2024-02-23T00:33:24.686 INFO:teuthology.orchestra.run.smithi047.stderr:Waiting for mgr epoch 5... 2024-02-23T00:33:24.688 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:24.125+0000 7fa56f4f4680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-23T00:33:24.688 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:24.041459+0000 mon.a (mon.0) 29 : audit [DBG] from='client.? 172.21.15.47:0/3786068239' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-23T00:33:25.258 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:25.121+0000 7fa56f4f4680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-23T00:33:25.322 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.319+0000 7f28354f5700 1 Processor -- start 2024-02-23T00:33:25.323 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.319+0000 7f28354f5700 1 -- start start 2024-02-23T00:33:25.323 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.319+0000 7f28354f5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28300fadc0 0x7f28300fb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:25.323 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.319+0000 7f28354f5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28300fb6d0 con 0x7f28300fadc0 2024-02-23T00:33:25.323 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.320+0000 7f282effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28300fadc0 0x7f28300fb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:25.323 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.320+0000 7f282effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28300fadc0 0x7f28300fb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34276/0 (socket says 172.21.15.47:34276) 2024-02-23T00:33:25.323 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.320+0000 7f282effd700 1 -- 172.21.15.47:0/2916877599 learned_addr learned my addr 172.21.15.47:0/2916877599 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:25.323 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.320+0000 7f282effd700 1 -- 172.21.15.47:0/2916877599 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28300fb810 con 0x7f28300fadc0 2024-02-23T00:33:25.323 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.320+0000 7f282effd700 1 --2- 172.21.15.47:0/2916877599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28300fadc0 0x7f28300fb1e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f2820009730 tx=0x7f28200090f0).ready entity=mon.0 client_cookie=c928eebb98025387 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:25.323 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.320+0000 7f282dffb700 1 -- 172.21.15.47:0/2916877599 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f282000b580 con 0x7f28300fadc0 2024-02-23T00:33:25.323 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.320+0000 7f282dffb700 1 -- 172.21.15.47:0/2916877599 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7f282000bb50 con 0x7f28300fadc0 2024-02-23T00:33:25.323 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.320+0000 7f282dffb700 1 -- 172.21.15.47:0/2916877599 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2820011ab0 con 0x7f28300fadc0 2024-02-23T00:33:25.325 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.321+0000 7f28354f5700 1 -- 172.21.15.47:0/2916877599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28300fadc0 msgr2=0x7f28300fb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:25.325 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.321+0000 7f28354f5700 1 --2- 172.21.15.47:0/2916877599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28300fadc0 0x7f28300fb1e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f2820009730 tx=0x7f28200090f0).stop 2024-02-23T00:33:25.325 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.321+0000 7f28354f5700 1 -- 172.21.15.47:0/2916877599 shutdown_connections 2024-02-23T00:33:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.321+0000 7f28354f5700 1 --2- 172.21.15.47:0/2916877599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28300fadc0 0x7f28300fb1e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.321+0000 7f28354f5700 1 -- 172.21.15.47:0/2916877599 >> 172.21.15.47:0/2916877599 conn(0x7f28300f6540 msgr2=0x7f28300f8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.321+0000 7f28354f5700 1 -- 172.21.15.47:0/2916877599 shutdown_connections 2024-02-23T00:33:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.322+0000 7f28354f5700 1 -- 172.21.15.47:0/2916877599 wait complete. 2024-02-23T00:33:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.322+0000 7f28354f5700 1 librados: starting msgr at 2024-02-23T00:33:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.322+0000 7f28354f5700 1 librados: starting objecter 2024-02-23T00:33:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.322+0000 7f28354f5700 1 Processor -- start 2024-02-23T00:33:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.322+0000 7f28354f5700 1 -- start start 2024-02-23T00:33:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.322+0000 7f28354f5700 1 librados: setting wanted keys 2024-02-23T00:33:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.322+0000 7f28354f5700 1 librados: calling monclient init 2024-02-23T00:33:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.323+0000 7f28354f5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28300fadc0 0x7f283010aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.323+0000 7f28354f5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283010af20 con 0x7f28300fadc0 2024-02-23T00:33:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.323+0000 7f282effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28300fadc0 0x7f283010aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.323+0000 7f282effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28300fadc0 0x7f283010aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34286/0 (socket says 172.21.15.47:34286) 2024-02-23T00:33:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.323+0000 7f282effd700 1 -- 172.21.15.47:0/3568667069 learned_addr learned my addr 172.21.15.47:0/3568667069 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:25.328 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.323+0000 7f282effd700 1 -- 172.21.15.47:0/3568667069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2820009370 con 0x7f28300fadc0 2024-02-23T00:33:25.328 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.323+0000 7f282effd700 1 --2- 172.21.15.47:0/3568667069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28300fadc0 0x7f283010aa30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f282000bfa0 tx=0x7f28200090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:25.328 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.323+0000 7f2817fff700 10 client.?.objecter ms_handle_connect 0x7f28300fadc0 2024-02-23T00:33:25.328 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.323+0000 7f2817fff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:33:25.328 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.323+0000 7f2817fff700 1 -- 172.21.15.47:0/3568667069 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2820011a70 con 0x7f28300fadc0 2024-02-23T00:33:25.329 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.323+0000 7f28354f5700 1 -- 172.21.15.47:0/3568667069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f283010b120 con 0x7f28300fadc0 2024-02-23T00:33:25.329 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.323+0000 7f2817fff700 1 -- 172.21.15.47:0/3568667069 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7f282001aab0 con 0x7f28300fadc0 2024-02-23T00:33:25.329 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.323+0000 7f2817fff700 1 -- 172.21.15.47:0/3568667069 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f28200239a0 con 0x7f28300fadc0 2024-02-23T00:33:25.329 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.323+0000 7f28354f5700 10 client.14120.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:33:25.329 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.323+0000 7f28354f5700 1 -- 172.21.15.47:0/3568667069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f283010b530 con 0x7f28300fadc0 2024-02-23T00:33:25.329 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.323+0000 7f28354f5700 1 librados: init done 2024-02-23T00:33:25.329 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.324+0000 7f2817fff700 1 -- 172.21.15.47:0/3568667069 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 38617+0+0 (secure 0 0 0) 0x7f2820023c10 con 0x7f28300fadc0 2024-02-23T00:33:25.329 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.324+0000 7f2817fff700 1 --2- 172.21.15.47:0/3568667069 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7f2818033f60 0x7f2818036420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:25.329 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.324+0000 7f2817fff700 1 -- 172.21.15.47:0/3568667069 --> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2818036ae0 con 0x7f2818033f60 2024-02-23T00:33:25.329 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.325+0000 7f2817fff700 1 -- 172.21.15.47:0/3568667069 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f2820044b30 con 0x7f28300fadc0 2024-02-23T00:33:25.329 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.325+0000 7f2817fff700 10 client.14120.objecter ms_dispatch 0x7f2830001550 osd_map(1..1 src has 1..1) v4 2024-02-23T00:33:25.329 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.325+0000 7f2817fff700 3 client.14120.objecter handle_osd_map got epochs [1,1] > 0 2024-02-23T00:33:25.329 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.325+0000 7f2817fff700 3 client.14120.objecter handle_osd_map decoding full epoch 1 2024-02-23T00:33:25.329 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.325+0000 7f282e7fc700 1 -- 172.21.15.47:0/3568667069 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7f2818033f60 msgr2=0x7f2818036420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/2409691510 2024-02-23T00:33:25.330 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.325+0000 7f2817fff700 20 client.14120.objecter dump_active .. 0 homeless 2024-02-23T00:33:25.330 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.325+0000 7f282e7fc700 1 --2- 172.21.15.47:0/3568667069 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7f2818033f60 0x7f2818036420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-02-23T00:33:25.528 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.526+0000 7f282e7fc700 1 -- 172.21.15.47:0/3568667069 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7f2818033f60 msgr2=0x7f2818036420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/2409691510 2024-02-23T00:33:25.528 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.526+0000 7f282e7fc700 1 --2- 172.21.15.47:0/3568667069 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7f2818033f60 0x7f2818036420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-02-23T00:33:25.556 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:25.256+0000 7fa56f4f4680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-23T00:33:25.556 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:25.342+0000 7fa56f4f4680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-23T00:33:25.556 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:25.435+0000 7fa56f4f4680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-23T00:33:25.929 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.927+0000 7f282e7fc700 1 -- 172.21.15.47:0/3568667069 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7f2818033f60 msgr2=0x7f2818036420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/2409691510 2024-02-23T00:33:25.929 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:25.927+0000 7f282e7fc700 1 --2- 172.21.15.47:0/3568667069 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7f2818033f60 0x7f2818036420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2024-02-23T00:33:25.933 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:25.638+0000 7fa56f4f4680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-23T00:33:26.306 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:25.931+0000 7fa56f4f4680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-23T00:33:26.306 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:26.033+0000 7fa56f4f4680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-23T00:33:26.730 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:26.728+0000 7f282e7fc700 1 -- 172.21.15.47:0/3568667069 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7f2818033f60 msgr2=0x7f2818036420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/2409691510 2024-02-23T00:33:26.730 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:26.728+0000 7f282e7fc700 1 --2- 172.21.15.47:0/3568667069 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7f2818033f60 0x7f2818036420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2024-02-23T00:33:27.056 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:26.791+0000 7fa56f4f4680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-23T00:33:27.056 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:26.876+0000 7fa56f4f4680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-23T00:33:27.056 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:26.970+0000 7fa56f4f4680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-23T00:33:27.420 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:27.150+0000 7fa56f4f4680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-23T00:33:27.695 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:27.418+0000 7fa56f4f4680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-23T00:33:27.696 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:27.555+0000 7fa56f4f4680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-23T00:33:27.961 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:27.693+0000 7fa56f4f4680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-23T00:33:27.961 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:27.872+0000 7fa56f4f4680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-23T00:33:27.976 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:27.973+0000 7f2817fff700 1 -- 172.21.15.47:0/3568667069 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mgrmap(e 6) v1 ==== 38441+0+0 (secure 0 0 0) 0x7f282002c3f0 con 0x7f28300fadc0 2024-02-23T00:33:27.976 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:27.973+0000 7f2817fff700 1 -- 172.21.15.47:0/3568667069 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7f2818033f60 msgr2=0x7f2818036420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-23T00:33:27.976 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:27.973+0000 7f2817fff700 1 --2- 172.21.15.47:0/3568667069 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7f2818033f60 0x7f2818036420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:33:28.306 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:27.959+0000 7fa56f4f4680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-23T00:33:28.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:33:27.972921+0000 mon.a (mon.0) 30 : cluster [INF] Active manager daemon y restarted 2024-02-23T00:33:28.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:33:27.973206+0000 mon.a (mon.0) 31 : cluster [INF] Activating manager daemon y 2024-02-23T00:33:28.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:33:27.975987+0000 mon.a (mon.0) 32 : cluster [DBG] osdmap e2: 0 total, 0 up, 0 in 2024-02-23T00:33:28.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:33:27.976088+0000 mon.a (mon.0) 33 : cluster [DBG] mgrmap e6: y(active, starting, since 0.00297275s) 2024-02-23T00:33:28.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:27.977244+0000 mon.a (mon.0) 34 : audit [DBG] from='mgr.14116 172.21.15.47:0/1137377827' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-23T00:33:28.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:27.977989+0000 mon.a (mon.0) 35 : audit [DBG] from='mgr.14116 172.21.15.47:0/1137377827' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-23T00:33:28.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:27.978652+0000 mon.a (mon.0) 36 : audit [DBG] from='mgr.14116 172.21.15.47:0/1137377827' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-23T00:33:28.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:27.978797+0000 mon.a (mon.0) 37 : audit [DBG] from='mgr.14116 172.21.15.47:0/1137377827' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-23T00:33:28.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:27.978931+0000 mon.a (mon.0) 38 : audit [DBG] from='mgr.14116 172.21.15.47:0/1137377827' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-23T00:33:28.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:33:27.992661+0000 mon.a (mon.0) 39 : cluster [INF] Manager daemon y is now available 2024-02-23T00:33:28.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:28.005124+0000 mon.a (mon.0) 40 : audit [INF] from='mgr.14116 172.21.15.47:0/1137377827' entity='mgr.y' 2024-02-23T00:33:28.977 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:28.976+0000 7f2817fff700 1 -- 172.21.15.47:0/3568667069 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 7) v1 ==== 38511+0+0 (secure 0 0 0) 0x7f282002b350 con 0x7f28300fadc0 2024-02-23T00:33:28.977 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:28.976+0000 7f2817fff700 1 --2- 172.21.15.47:0/3568667069 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f2818037a30 0x7f2818039e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:28.977 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:28.976+0000 7f2817fff700 1 -- 172.21.15.47:0/3568667069 --> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2818036ae0 con 0x7f2818037a30 2024-02-23T00:33:28.977 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:28.977+0000 7f282e7fc700 1 --2- 172.21.15.47:0/3568667069 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f2818037a30 0x7f2818039e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:28.978 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:28.977+0000 7f282e7fc700 1 --2- 172.21.15.47:0/3568667069 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f2818037a30 0x7f2818039e20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7f2824003850 tx=0x7f2824006e20).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:28.978 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:28.977+0000 7f2817fff700 10 client.14120.objecter ms_handle_connect 0x7f2818037a30 2024-02-23T00:33:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:28.978+0000 7f2817fff700 1 -- 172.21.15.47:0/3568667069 <== mgr.14116 v2:172.21.15.47:6800/1916598887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6438 (secure 0 0 0) 0x7f2818036ae0 con 0x7f2818037a30 2024-02-23T00:33:28.986 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:28.985+0000 7f28354f5700 1 -- 172.21.15.47:0/3568667069 --> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f283005a310 con 0x7f2818037a30 2024-02-23T00:33:28.986 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:28.985+0000 7f2817fff700 1 -- 172.21.15.47:0/3568667069 <== mgr.14116 v2:172.21.15.47:6800/1916598887 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f283005a310 con 0x7f2818037a30 2024-02-23T00:33:28.986 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: { 2024-02-23T00:33:28.988 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "mgrmap_epoch": 7, 2024-02-23T00:33:28.988 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "initialized": true 2024-02-23T00:33:28.988 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: } 2024-02-23T00:33:28.988 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:28.985+0000 7f28354f5700 10 librados: watch_flush enter 2024-02-23T00:33:28.988 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:28.985+0000 7f28354f5700 10 librados: watch_flush exit 2024-02-23T00:33:28.988 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:28.985+0000 7f28354f5700 20 client.14120.objecter shutdown clearing up homeless session... 2024-02-23T00:33:28.988 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:28.985+0000 7f28354f5700 10 client.14120.objecter successfully canceled tick 2024-02-23T00:33:28.988 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:28.985+0000 7f28354f5700 1 -- 172.21.15.47:0/3568667069 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f2818037a30 msgr2=0x7f2818039e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:28.988 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:28.985+0000 7f28354f5700 1 --2- 172.21.15.47:0/3568667069 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f2818037a30 0x7f2818039e20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7f2824003850 tx=0x7f2824006e20).stop 2024-02-23T00:33:28.989 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:28.985+0000 7f28354f5700 1 -- 172.21.15.47:0/3568667069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28300fadc0 msgr2=0x7f283010aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:28.989 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:28.985+0000 7f28354f5700 1 --2- 172.21.15.47:0/3568667069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28300fadc0 0x7f283010aa30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f282000bfa0 tx=0x7f28200090f0).stop 2024-02-23T00:33:28.989 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:28.986+0000 7f28354f5700 1 -- 172.21.15.47:0/3568667069 shutdown_connections 2024-02-23T00:33:28.989 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:28.986+0000 7f28354f5700 1 --2- 172.21.15.47:0/3568667069 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f2818037a30 0x7f2818039e20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:28.989 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:28.986+0000 7f28354f5700 1 --2- 172.21.15.47:0/3568667069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28300fadc0 0x7f283010aa30 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:28.989 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:28.986+0000 7f28354f5700 1 --2- 172.21.15.47:0/3568667069 >> [v2:172.21.15.47:6800/2409691510,v1:172.21.15.47:6801/2409691510] conn(0x7f2818033f60 0x7f2818036420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:33:28.989 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:28.986+0000 7f28354f5700 1 -- 172.21.15.47:0/3568667069 >> 172.21.15.47:0/3568667069 conn(0x7f28300f6540 msgr2=0x7f28300f7e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:28.989 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:28.986+0000 7f28354f5700 1 -- 172.21.15.47:0/3568667069 shutdown_connections 2024-02-23T00:33:28.989 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:28.986+0000 7f28354f5700 1 -- 172.21.15.47:0/3568667069 wait complete. 2024-02-23T00:33:28.989 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:28.986+0000 7f28354f5700 1 librados: shutdown 2024-02-23T00:33:29.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:33:28.002586+0000 mgr.y (mgr.14116) 1 : cephadm [INF] Found migration_current of "None". Setting to last migration. 2024-02-23T00:33:29.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:28.095834+0000 mon.a (mon.0) 41 : audit [INF] from='mgr.14116 172.21.15.47:0/1137377827' entity='mgr.y' 2024-02-23T00:33:29.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:28.096803+0000 mon.a (mon.0) 42 : audit [DBG] from='mgr.14116 172.21.15.47:0/1137377827' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:33:29.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:28.098491+0000 mon.a (mon.0) 43 : audit [DBG] from='mgr.14116 172.21.15.47:0/1137377827' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:33:29.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:28.100619+0000 mon.a (mon.0) 44 : audit [DBG] from='mgr.14116 172.21.15.47:0/1137377827' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:33:29.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:28.114256+0000 mon.a (mon.0) 45 : audit [INF] from='mgr.14116 172.21.15.47:0/1137377827' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-23T00:33:29.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:28.117063+0000 mon.a (mon.0) 46 : audit [INF] from='mgr.14116 172.21.15.47:0/1137377827' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-23T00:33:29.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:33:28.979220+0000 mon.a (mon.0) 47 : cluster [DBG] mgrmap e7: y(active, since 1.0061s) 2024-02-23T00:33:29.684 INFO:teuthology.orchestra.run.smithi047.stderr:mgr epoch 5 is available 2024-02-23T00:33:29.684 INFO:teuthology.orchestra.run.smithi047.stderr:Setting orchestrator backend to cephadm... 2024-02-23T00:33:30.298 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.258+0000 7f319b810700 1 Processor -- start 2024-02-23T00:33:30.298 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.258+0000 7f319b810700 1 -- start start 2024-02-23T00:33:30.298 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.258+0000 7f319b810700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3194070bc0 0x7f31940fea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:30.298 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.258+0000 7f319b810700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31940fef60 con 0x7f3194070bc0 2024-02-23T00:33:30.298 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.259+0000 7f31995ac700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3194070bc0 0x7f31940fea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:30.298 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.259+0000 7f31995ac700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3194070bc0 0x7f31940fea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44730/0 (socket says 172.21.15.47:44730) 2024-02-23T00:33:30.298 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.259+0000 7f31995ac700 1 -- 172.21.15.47:0/363388454 learned_addr learned my addr 172.21.15.47:0/363388454 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:30.298 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.259+0000 7f31995ac700 1 -- 172.21.15.47:0/363388454 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31940ff0a0 con 0x7f3194070bc0 2024-02-23T00:33:30.298 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.259+0000 7f31995ac700 1 --2- 172.21.15.47:0/363388454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3194070bc0 0x7f31940fea70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f3184009730 tx=0x7f31840090f0).ready entity=mon.0 client_cookie=206870dd4ff25b6c server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:30.298 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.259+0000 7f318bfff700 1 -- 172.21.15.47:0/363388454 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f318400b580 con 0x7f3194070bc0 2024-02-23T00:33:30.299 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.259+0000 7f318bfff700 1 -- 172.21.15.47:0/363388454 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7f318400bb50 con 0x7f3194070bc0 2024-02-23T00:33:30.299 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.259+0000 7f318bfff700 1 -- 172.21.15.47:0/363388454 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3184011ab0 con 0x7f3194070bc0 2024-02-23T00:33:30.299 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.260+0000 7f319b810700 1 -- 172.21.15.47:0/363388454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3194070bc0 msgr2=0x7f31940fea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:30.299 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.260+0000 7f319b810700 1 --2- 172.21.15.47:0/363388454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3194070bc0 0x7f31940fea70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f3184009730 tx=0x7f31840090f0).stop 2024-02-23T00:33:30.299 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.260+0000 7f319b810700 1 -- 172.21.15.47:0/363388454 shutdown_connections 2024-02-23T00:33:30.299 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.260+0000 7f319b810700 1 --2- 172.21.15.47:0/363388454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3194070bc0 0x7f31940fea70 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:30.299 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.260+0000 7f319b810700 1 -- 172.21.15.47:0/363388454 >> 172.21.15.47:0/363388454 conn(0x7f31940f6530 msgr2=0x7f31940f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:30.299 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.260+0000 7f319b810700 1 -- 172.21.15.47:0/363388454 shutdown_connections 2024-02-23T00:33:30.299 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.260+0000 7f319b810700 1 -- 172.21.15.47:0/363388454 wait complete. 2024-02-23T00:33:30.299 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.260+0000 7f319b810700 1 librados: starting msgr at 2024-02-23T00:33:30.299 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.260+0000 7f319b810700 1 librados: starting objecter 2024-02-23T00:33:30.299 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.260+0000 7f319b810700 1 Processor -- start 2024-02-23T00:33:30.299 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.260+0000 7f319b810700 1 -- start start 2024-02-23T00:33:30.299 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.260+0000 7f319b810700 1 librados: setting wanted keys 2024-02-23T00:33:30.299 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.260+0000 7f319b810700 1 librados: calling monclient init 2024-02-23T00:33:30.300 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.261+0000 7f319b810700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3194070bc0 0x7f31940fc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:30.300 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.261+0000 7f319b810700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31940fa970 con 0x7f3194070bc0 2024-02-23T00:33:30.300 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.261+0000 7f31995ac700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3194070bc0 0x7f31940fc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:30.300 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.261+0000 7f31995ac700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3194070bc0 0x7f31940fc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44744/0 (socket says 172.21.15.47:44744) 2024-02-23T00:33:30.300 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.261+0000 7f31995ac700 1 -- 172.21.15.47:0/2766282137 learned_addr learned my addr 172.21.15.47:0/2766282137 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:30.300 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.261+0000 7f31995ac700 1 -- 172.21.15.47:0/2766282137 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3184009370 con 0x7f3194070bc0 2024-02-23T00:33:30.300 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.261+0000 7f31995ac700 1 --2- 172.21.15.47:0/2766282137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3194070bc0 0x7f31940fc4f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f3184012f20 tx=0x7f31840090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:30.300 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.261+0000 7f318a7fc700 10 client.?.objecter ms_handle_connect 0x7f3194070bc0 2024-02-23T00:33:30.300 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.261+0000 7f318a7fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:33:30.300 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.261+0000 7f318a7fc700 1 -- 172.21.15.47:0/2766282137 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3184011a70 con 0x7f3194070bc0 2024-02-23T00:33:30.300 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.261+0000 7f319b810700 1 -- 172.21.15.47:0/2766282137 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31940fab10 con 0x7f3194070bc0 2024-02-23T00:33:30.300 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.261+0000 7f318a7fc700 1 -- 172.21.15.47:0/2766282137 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7f318401aab0 con 0x7f3194070bc0 2024-02-23T00:33:30.300 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.261+0000 7f318a7fc700 1 -- 172.21.15.47:0/2766282137 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3184023960 con 0x7f3194070bc0 2024-02-23T00:33:30.300 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.261+0000 7f319b810700 10 client.14126.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:33:30.301 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.261+0000 7f319b810700 1 -- 172.21.15.47:0/2766282137 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31940fafa0 con 0x7f3194070bc0 2024-02-23T00:33:30.301 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.261+0000 7f319b810700 1 librados: init done 2024-02-23T00:33:30.301 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.262+0000 7f318a7fc700 1 -- 172.21.15.47:0/2766282137 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) v1 ==== 38617+0+0 (secure 0 0 0) 0x7f3184023b50 con 0x7f3194070bc0 2024-02-23T00:33:30.301 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.262+0000 7f319b810700 1 -- 172.21.15.47:0/2766282137 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f319405a310 con 0x7f3194070bc0 2024-02-23T00:33:30.301 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.262+0000 7f318a7fc700 1 --2- 172.21.15.47:0/2766282137 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f3180033f60 0x7f3180036420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:30.301 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.263+0000 7f318a7fc700 1 -- 172.21.15.47:0/2766282137 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7f3184044e90 con 0x7f3194070bc0 2024-02-23T00:33:30.301 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.263+0000 7f318a7fc700 10 client.14126.objecter ms_dispatch 0x7f3194001550 osd_map(2..2 src has 1..2) v4 2024-02-23T00:33:30.301 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.263+0000 7f318a7fc700 3 client.14126.objecter handle_osd_map got epochs [2,2] > 0 2024-02-23T00:33:30.301 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.263+0000 7f318a7fc700 3 client.14126.objecter handle_osd_map decoding full epoch 2 2024-02-23T00:33:30.301 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.263+0000 7f318a7fc700 20 client.14126.objecter dump_active .. 0 homeless 2024-02-23T00:33:30.301 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.263+0000 7f3198dab700 1 --2- 172.21.15.47:0/2766282137 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f3180033f60 0x7f3180036420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:30.301 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.263+0000 7f318a7fc700 10 client.14126.objecter ms_handle_connect 0x7f3180033f60 2024-02-23T00:33:30.301 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.263+0000 7f3198dab700 1 --2- 172.21.15.47:0/2766282137 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f3180033f60 0x7f3180036420 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f3190009780 tx=0x7f3190006cc0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:30.302 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.268+0000 7f318a7fc700 1 -- 172.21.15.47:0/2766282137 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f318402b530 con 0x7f3194070bc0 2024-02-23T00:33:30.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:28.978147+0000 mgr.y (mgr.14116) 2 : audit [DBG] from='client.14120 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-23T00:33:30.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:28.985358+0000 mgr.y (mgr.14116) 3 : audit [DBG] from='client.14120 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-23T00:33:30.452 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.451+0000 7f319b810700 1 -- 172.21.15.47:0/2766282137 --> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f31941045f0 con 0x7f3180033f60 2024-02-23T00:33:30.457 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.456+0000 7f318a7fc700 1 -- 172.21.15.47:0/2766282137 <== mgr.14116 v2:172.21.15.47:6800/1916598887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f31941045f0 con 0x7f3180033f60 2024-02-23T00:33:30.460 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.458+0000 7f319b810700 10 librados: watch_flush enter 2024-02-23T00:33:30.460 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.458+0000 7f319b810700 10 librados: watch_flush exit 2024-02-23T00:33:30.461 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.458+0000 7f319b810700 20 client.14126.objecter shutdown clearing up homeless session... 2024-02-23T00:33:30.461 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.458+0000 7f319b810700 10 client.14126.objecter successfully canceled tick 2024-02-23T00:33:30.461 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.458+0000 7f319b810700 1 -- 172.21.15.47:0/2766282137 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f3180033f60 msgr2=0x7f3180036420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:30.461 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.458+0000 7f319b810700 1 --2- 172.21.15.47:0/2766282137 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f3180033f60 0x7f3180036420 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f3190009780 tx=0x7f3190006cc0).stop 2024-02-23T00:33:30.461 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.458+0000 7f319b810700 1 -- 172.21.15.47:0/2766282137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3194070bc0 msgr2=0x7f31940fc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:30.461 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.458+0000 7f319b810700 1 --2- 172.21.15.47:0/2766282137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3194070bc0 0x7f31940fc4f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f3184012f20 tx=0x7f31840090f0).stop 2024-02-23T00:33:30.461 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.458+0000 7f319b810700 1 -- 172.21.15.47:0/2766282137 shutdown_connections 2024-02-23T00:33:30.461 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.458+0000 7f319b810700 1 --2- 172.21.15.47:0/2766282137 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f3180033f60 0x7f3180036420 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:30.461 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.458+0000 7f319b810700 1 --2- 172.21.15.47:0/2766282137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3194070bc0 0x7f31940fc4f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:30.462 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.458+0000 7f319b810700 1 -- 172.21.15.47:0/2766282137 >> 172.21.15.47:0/2766282137 conn(0x7f31940f6530 msgr2=0x7f31940f7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:30.462 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.458+0000 7f319b810700 1 -- 172.21.15.47:0/2766282137 shutdown_connections 2024-02-23T00:33:30.462 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.458+0000 7f319b810700 1 -- 172.21.15.47:0/2766282137 wait complete. 2024-02-23T00:33:30.462 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:30.458+0000 7f319b810700 1 librados: shutdown 2024-02-23T00:33:31.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:33:30.098324+0000 mon.a (mon.0) 48 : cluster [DBG] mgrmap e8: y(active, since 2s) 2024-02-23T00:33:31.369 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:30.456412+0000 mon.a (mon.0) 49 : audit [INF] from='mgr.14116 172.21.15.47:0/1137377827' entity='mgr.y' 2024-02-23T00:33:31.369 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:30.475628+0000 mon.a (mon.0) 50 : audit [DBG] from='mgr.14116 172.21.15.47:0/1137377827' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:33:31.714 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.704+0000 7fec7bc01700 1 Processor -- start 2024-02-23T00:33:31.714 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.705+0000 7fec7bc01700 1 -- start start 2024-02-23T00:33:31.715 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.705+0000 7fec7bc01700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec74073410 0x7fec74071a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:31.715 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.705+0000 7fec7bc01700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec74073830 con 0x7fec74073410 2024-02-23T00:33:31.715 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.705+0000 7fec7999d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec74073410 0x7fec74071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:31.715 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.705+0000 7fec7999d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec74073410 0x7fec74071a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44752/0 (socket says 172.21.15.47:44752) 2024-02-23T00:33:31.715 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.705+0000 7fec7999d700 1 -- 172.21.15.47:0/2515069350 learned_addr learned my addr 172.21.15.47:0/2515069350 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:31.715 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.705+0000 7fec7999d700 1 -- 172.21.15.47:0/2515069350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec74071ef0 con 0x7fec74073410 2024-02-23T00:33:31.715 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.706+0000 7fec7999d700 1 --2- 172.21.15.47:0/2515069350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec74073410 0x7fec74071a00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fec68009730 tx=0x7fec680090f0).ready entity=mon.0 client_cookie=3cd8240bb0ed512e server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:31.716 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.706+0000 7fec7899b700 1 -- 172.21.15.47:0/2515069350 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec6800b580 con 0x7fec74073410 2024-02-23T00:33:31.716 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.706+0000 7fec7899b700 1 -- 172.21.15.47:0/2515069350 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7fec6800bb50 con 0x7fec74073410 2024-02-23T00:33:31.716 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.706+0000 7fec7899b700 1 -- 172.21.15.47:0/2515069350 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec68011ab0 con 0x7fec74073410 2024-02-23T00:33:31.716 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.706+0000 7fec7bc01700 1 -- 172.21.15.47:0/2515069350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec74073410 msgr2=0x7fec74071a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:31.716 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.706+0000 7fec7bc01700 1 --2- 172.21.15.47:0/2515069350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec74073410 0x7fec74071a00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fec68009730 tx=0x7fec680090f0).stop 2024-02-23T00:33:31.716 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.706+0000 7fec7bc01700 1 -- 172.21.15.47:0/2515069350 shutdown_connections 2024-02-23T00:33:31.716 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.706+0000 7fec7bc01700 1 --2- 172.21.15.47:0/2515069350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec74073410 0x7fec74071a00 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:31.717 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.706+0000 7fec7bc01700 1 -- 172.21.15.47:0/2515069350 >> 172.21.15.47:0/2515069350 conn(0x7fec740f7850 msgr2=0x7fec740f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:31.717 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.707+0000 7fec7bc01700 1 -- 172.21.15.47:0/2515069350 shutdown_connections 2024-02-23T00:33:31.717 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.707+0000 7fec7bc01700 1 -- 172.21.15.47:0/2515069350 wait complete. 2024-02-23T00:33:31.717 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.707+0000 7fec7bc01700 1 librados: starting msgr at 2024-02-23T00:33:31.717 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.707+0000 7fec7bc01700 1 librados: starting objecter 2024-02-23T00:33:31.717 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.707+0000 7fec7bc01700 1 Processor -- start 2024-02-23T00:33:31.717 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.707+0000 7fec7bc01700 1 -- start start 2024-02-23T00:33:31.717 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.707+0000 7fec7bc01700 1 librados: setting wanted keys 2024-02-23T00:33:31.717 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.707+0000 7fec7bc01700 1 librados: calling monclient init 2024-02-23T00:33:31.718 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.708+0000 7fec7bc01700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec74073410 0x7fec7410cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:31.718 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.708+0000 7fec7bc01700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7410d220 con 0x7fec74073410 2024-02-23T00:33:31.718 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.708+0000 7fec7999d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec74073410 0x7fec7410cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:31.718 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.708+0000 7fec7999d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec74073410 0x7fec7410cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44760/0 (socket says 172.21.15.47:44760) 2024-02-23T00:33:31.718 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.708+0000 7fec7999d700 1 -- 172.21.15.47:0/1731035685 learned_addr learned my addr 172.21.15.47:0/1731035685 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:31.718 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.708+0000 7fec7999d700 1 -- 172.21.15.47:0/1731035685 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec68009370 con 0x7fec74073410 2024-02-23T00:33:31.718 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.708+0000 7fec7999d700 1 --2- 172.21.15.47:0/1731035685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec74073410 0x7fec7410cd30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fec68012f20 tx=0x7fec680090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:31.718 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.708+0000 7fec66ffd700 10 client.?.objecter ms_handle_connect 0x7fec74073410 2024-02-23T00:33:31.718 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.708+0000 7fec66ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:33:31.719 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.708+0000 7fec66ffd700 1 -- 172.21.15.47:0/1731035685 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec68011a70 con 0x7fec74073410 2024-02-23T00:33:31.719 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.708+0000 7fec66ffd700 1 -- 172.21.15.47:0/1731035685 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7fec6801aab0 con 0x7fec74073410 2024-02-23T00:33:31.719 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.708+0000 7fec66ffd700 1 -- 172.21.15.47:0/1731035685 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec680238f0 con 0x7fec74073410 2024-02-23T00:33:31.719 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.708+0000 7fec7bc01700 1 -- 172.21.15.47:0/1731035685 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec7410d420 con 0x7fec74073410 2024-02-23T00:33:31.719 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.708+0000 7fec7bc01700 10 client.14128.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:33:31.719 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.709+0000 7fec7bc01700 1 -- 172.21.15.47:0/1731035685 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec7410dfd0 con 0x7fec74073410 2024-02-23T00:33:31.719 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.709+0000 7fec7bc01700 1 librados: init done 2024-02-23T00:33:31.719 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.709+0000 7fec66ffd700 1 -- 172.21.15.47:0/1731035685 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) v1 ==== 38617+0+0 (secure 0 0 0) 0x7fec68023b60 con 0x7fec74073410 2024-02-23T00:33:31.720 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.709+0000 7fec66ffd700 1 --2- 172.21.15.47:0/1731035685 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7fec60033fb0 0x7fec60036470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:31.720 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.710+0000 7fec66ffd700 1 -- 172.21.15.47:0/1731035685 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7fec68044ec0 con 0x7fec74073410 2024-02-23T00:33:31.720 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.710+0000 7fec66ffd700 10 client.14128.objecter ms_dispatch 0x7fec74001550 osd_map(2..2 src has 1..2) v4 2024-02-23T00:33:31.720 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.710+0000 7fec66ffd700 3 client.14128.objecter handle_osd_map got epochs [2,2] > 0 2024-02-23T00:33:31.720 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.710+0000 7fec66ffd700 3 client.14128.objecter handle_osd_map decoding full epoch 2 2024-02-23T00:33:31.720 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.710+0000 7fec66ffd700 20 client.14128.objecter dump_active .. 0 homeless 2024-02-23T00:33:31.720 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.710+0000 7fec7919c700 1 --2- 172.21.15.47:0/1731035685 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7fec60033fb0 0x7fec60036470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:31.720 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.710+0000 7fec7919c700 1 --2- 172.21.15.47:0/1731035685 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7fec60033fb0 0x7fec60036470 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fec70009780 tx=0x7fec70006cc0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:31.721 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.710+0000 7fec66ffd700 10 client.14128.objecter ms_handle_connect 0x7fec60033fb0 2024-02-23T00:33:31.721 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.710+0000 7fec7bc01700 1 -- 172.21.15.47:0/1731035685 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec7405a310 con 0x7fec74073410 2024-02-23T00:33:31.721 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.715+0000 7fec66ffd700 1 -- 172.21.15.47:0/1731035685 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fec6801e070 con 0x7fec74073410 2024-02-23T00:33:31.889 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.888+0000 7fec7bc01700 1 -- 172.21.15.47:0/1731035685 --> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fec7410d8b0 con 0x7fec60033fb0 2024-02-23T00:33:31.890 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.889+0000 7fec66ffd700 1 -- 172.21.15.47:0/1731035685 <== mgr.14116 v2:172.21.15.47:6800/1916598887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fec7410d8b0 con 0x7fec60033fb0 2024-02-23T00:33:31.890 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: value unchanged 2024-02-23T00:33:31.893 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.891+0000 7fec7bc01700 10 librados: watch_flush enter 2024-02-23T00:33:31.893 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.891+0000 7fec7bc01700 10 librados: watch_flush exit 2024-02-23T00:33:31.893 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.891+0000 7fec7bc01700 20 client.14128.objecter shutdown clearing up homeless session... 2024-02-23T00:33:31.893 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.891+0000 7fec7bc01700 10 client.14128.objecter successfully canceled tick 2024-02-23T00:33:31.893 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.891+0000 7fec7bc01700 1 -- 172.21.15.47:0/1731035685 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7fec60033fb0 msgr2=0x7fec60036470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:31.893 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.891+0000 7fec7bc01700 1 --2- 172.21.15.47:0/1731035685 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7fec60033fb0 0x7fec60036470 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fec70009780 tx=0x7fec70006cc0).stop 2024-02-23T00:33:31.893 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.891+0000 7fec7bc01700 1 -- 172.21.15.47:0/1731035685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec74073410 msgr2=0x7fec7410cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:31.894 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.891+0000 7fec7bc01700 1 --2- 172.21.15.47:0/1731035685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec74073410 0x7fec7410cd30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fec68012f20 tx=0x7fec680090f0).stop 2024-02-23T00:33:31.894 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.891+0000 7fec7bc01700 1 -- 172.21.15.47:0/1731035685 shutdown_connections 2024-02-23T00:33:31.894 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.891+0000 7fec7bc01700 1 --2- 172.21.15.47:0/1731035685 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7fec60033fb0 0x7fec60036470 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:31.894 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.891+0000 7fec7bc01700 1 --2- 172.21.15.47:0/1731035685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec74073410 0x7fec7410cd30 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:31.894 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.891+0000 7fec7bc01700 1 -- 172.21.15.47:0/1731035685 >> 172.21.15.47:0/1731035685 conn(0x7fec740f7850 msgr2=0x7fec740fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:31.894 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.891+0000 7fec7bc01700 1 -- 172.21.15.47:0/1731035685 shutdown_connections 2024-02-23T00:33:31.894 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.891+0000 7fec7bc01700 1 -- 172.21.15.47:0/1731035685 wait complete. 2024-02-23T00:33:31.894 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:31.891+0000 7fec7bc01700 1 librados: shutdown 2024-02-23T00:33:32.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:30.452549+0000 mgr.y (mgr.14116) 4 : audit [DBG] from='client.14126 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:33:32.680 INFO:teuthology.orchestra.run.smithi047.stderr:Generating ssh key... 2024-02-23T00:33:33.201 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.197+0000 7f397f7d8700 1 Processor -- start 2024-02-23T00:33:33.201 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.198+0000 7f397f7d8700 1 -- start start 2024-02-23T00:33:33.201 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.198+0000 7f397f7d8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39780713e0 0x7f39780fea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:33.201 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.198+0000 7f397f7d8700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39780fef70 con 0x7f39780713e0 2024-02-23T00:33:33.201 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.198+0000 7f397d574700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39780713e0 0x7f39780fea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:33.201 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.198+0000 7f397d574700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39780713e0 0x7f39780fea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44776/0 (socket says 172.21.15.47:44776) 2024-02-23T00:33:33.201 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.198+0000 7f397d574700 1 -- 172.21.15.47:0/293879352 learned_addr learned my addr 172.21.15.47:0/293879352 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:33.201 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.198+0000 7f397d574700 1 -- 172.21.15.47:0/293879352 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39780ff0b0 con 0x7f39780713e0 2024-02-23T00:33:33.202 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.198+0000 7f397d574700 1 --2- 172.21.15.47:0/293879352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39780713e0 0x7f39780fea80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f3974009730 tx=0x7f39740090f0).ready entity=mon.0 client_cookie=2e7a8e5126414963 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:33.202 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.199+0000 7f396bfff700 1 -- 172.21.15.47:0/293879352 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f397400b580 con 0x7f39780713e0 2024-02-23T00:33:33.202 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.199+0000 7f396bfff700 1 -- 172.21.15.47:0/293879352 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7f397400bb50 con 0x7f39780713e0 2024-02-23T00:33:33.202 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.199+0000 7f396bfff700 1 -- 172.21.15.47:0/293879352 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3974011ab0 con 0x7f39780713e0 2024-02-23T00:33:33.202 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.199+0000 7f397f7d8700 1 -- 172.21.15.47:0/293879352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39780713e0 msgr2=0x7f39780fea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:33.202 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.199+0000 7f397f7d8700 1 --2- 172.21.15.47:0/293879352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39780713e0 0x7f39780fea80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f3974009730 tx=0x7f39740090f0).stop 2024-02-23T00:33:33.202 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.199+0000 7f397f7d8700 1 -- 172.21.15.47:0/293879352 shutdown_connections 2024-02-23T00:33:33.202 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.199+0000 7f397f7d8700 1 --2- 172.21.15.47:0/293879352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39780713e0 0x7f39780fea80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:33.202 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.199+0000 7f397f7d8700 1 -- 172.21.15.47:0/293879352 >> 172.21.15.47:0/293879352 conn(0x7f39780f6540 msgr2=0x7f39780f8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:33.202 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.199+0000 7f397f7d8700 1 -- 172.21.15.47:0/293879352 shutdown_connections 2024-02-23T00:33:33.202 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.199+0000 7f397f7d8700 1 -- 172.21.15.47:0/293879352 wait complete. 2024-02-23T00:33:33.202 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.200+0000 7f397f7d8700 1 librados: starting msgr at 2024-02-23T00:33:33.202 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.200+0000 7f397f7d8700 1 librados: starting objecter 2024-02-23T00:33:33.205 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.200+0000 7f397f7d8700 1 Processor -- start 2024-02-23T00:33:33.206 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.200+0000 7f397f7d8700 1 -- start start 2024-02-23T00:33:33.206 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.200+0000 7f397f7d8700 1 librados: setting wanted keys 2024-02-23T00:33:33.206 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.200+0000 7f397f7d8700 1 librados: calling monclient init 2024-02-23T00:33:33.206 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.200+0000 7f397f7d8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39780713e0 0x7f397810aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:33.206 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.200+0000 7f397f7d8700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397810af60 con 0x7f39780713e0 2024-02-23T00:33:33.206 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.201+0000 7f397d574700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39780713e0 0x7f397810aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:33.206 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.201+0000 7f397d574700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39780713e0 0x7f397810aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44792/0 (socket says 172.21.15.47:44792) 2024-02-23T00:33:33.206 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.201+0000 7f397d574700 1 -- 172.21.15.47:0/1881094912 learned_addr learned my addr 172.21.15.47:0/1881094912 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:33.206 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.201+0000 7f397d574700 1 -- 172.21.15.47:0/1881094912 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3974009370 con 0x7f39780713e0 2024-02-23T00:33:33.206 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.201+0000 7f397d574700 1 --2- 172.21.15.47:0/1881094912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39780713e0 0x7f397810aa70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f3974000c00 tx=0x7f39740090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:33.207 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.201+0000 7f396a7fc700 10 client.?.objecter ms_handle_connect 0x7f39780713e0 2024-02-23T00:33:33.207 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.201+0000 7f396a7fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:33:33.207 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.201+0000 7f396a7fc700 1 -- 172.21.15.47:0/1881094912 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f397400b9e0 con 0x7f39780713e0 2024-02-23T00:33:33.207 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.201+0000 7f396a7fc700 1 -- 172.21.15.47:0/1881094912 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7f3974011400 con 0x7f39780713e0 2024-02-23T00:33:33.207 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.201+0000 7f397f7d8700 1 -- 172.21.15.47:0/1881094912 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f397810b160 con 0x7f39780713e0 2024-02-23T00:33:33.207 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.201+0000 7f396a7fc700 1 -- 172.21.15.47:0/1881094912 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39740238b0 con 0x7f39780713e0 2024-02-23T00:33:33.207 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.201+0000 7f397f7d8700 10 client.14130.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:33:33.207 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.201+0000 7f397f7d8700 1 -- 172.21.15.47:0/1881094912 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f397810b5f0 con 0x7f39780713e0 2024-02-23T00:33:33.207 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.201+0000 7f397f7d8700 1 librados: init done 2024-02-23T00:33:33.207 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.202+0000 7f396a7fc700 1 -- 172.21.15.47:0/1881094912 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) v1 ==== 38617+0+0 (secure 0 0 0) 0x7f397401a640 con 0x7f39780713e0 2024-02-23T00:33:33.207 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.202+0000 7f396a7fc700 1 --2- 172.21.15.47:0/1881094912 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f3964033fb0 0x7f3964036470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:33.207 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.202+0000 7f396a7fc700 1 -- 172.21.15.47:0/1881094912 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7f3974044ab0 con 0x7f39780713e0 2024-02-23T00:33:33.207 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.202+0000 7f396a7fc700 10 client.14130.objecter ms_dispatch 0x7f3978001550 osd_map(2..2 src has 1..2) v4 2024-02-23T00:33:33.208 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.202+0000 7f396a7fc700 3 client.14130.objecter handle_osd_map got epochs [2,2] > 0 2024-02-23T00:33:33.208 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.202+0000 7f396a7fc700 3 client.14130.objecter handle_osd_map decoding full epoch 2 2024-02-23T00:33:33.208 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.202+0000 7f397f7d8700 1 -- 172.21.15.47:0/1881094912 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3978046ce0 con 0x7f39780713e0 2024-02-23T00:33:33.208 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.202+0000 7f396a7fc700 20 client.14130.objecter dump_active .. 0 homeless 2024-02-23T00:33:33.208 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.202+0000 7f397cd73700 1 --2- 172.21.15.47:0/1881094912 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f3964033fb0 0x7f3964036470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:33.208 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.203+0000 7f396a7fc700 10 client.14130.objecter ms_handle_connect 0x7f3964033fb0 2024-02-23T00:33:33.208 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.203+0000 7f397cd73700 1 --2- 172.21.15.47:0/1881094912 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f3964033fb0 0x7f3964036470 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f396c009780 tx=0x7f396c006cc0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:33.209 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.207+0000 7f396a7fc700 1 -- 172.21.15.47:0/1881094912 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f397401e020 con 0x7f39780713e0 2024-02-23T00:33:33.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:31.889524+0000 mgr.y (mgr.14116) 5 : audit [DBG] from='client.14128 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:33:33.383 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.382+0000 7f397f7d8700 1 -- 172.21.15.47:0/1881094912 --> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f397810be50 con 0x7f3964033fb0 2024-02-23T00:33:33.589 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.589+0000 7f396a7fc700 1 -- 172.21.15.47:0/1881094912 <== mgr.14116 v2:172.21.15.47:6800/1916598887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f397810be50 con 0x7f3964033fb0 2024-02-23T00:33:33.592 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.590+0000 7f397f7d8700 10 librados: watch_flush enter 2024-02-23T00:33:33.593 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.590+0000 7f397f7d8700 10 librados: watch_flush exit 2024-02-23T00:33:33.593 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.590+0000 7f397f7d8700 20 client.14130.objecter shutdown clearing up homeless session... 2024-02-23T00:33:33.593 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.590+0000 7f397f7d8700 10 client.14130.objecter successfully canceled tick 2024-02-23T00:33:33.593 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.590+0000 7f397f7d8700 1 -- 172.21.15.47:0/1881094912 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f3964033fb0 msgr2=0x7f3964036470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:33.593 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.590+0000 7f397f7d8700 1 --2- 172.21.15.47:0/1881094912 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f3964033fb0 0x7f3964036470 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f396c009780 tx=0x7f396c006cc0).stop 2024-02-23T00:33:33.593 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.590+0000 7f397f7d8700 1 -- 172.21.15.47:0/1881094912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39780713e0 msgr2=0x7f397810aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:33.593 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.590+0000 7f397f7d8700 1 --2- 172.21.15.47:0/1881094912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39780713e0 0x7f397810aa70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f3974000c00 tx=0x7f39740090f0).stop 2024-02-23T00:33:33.593 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.590+0000 7f397f7d8700 1 -- 172.21.15.47:0/1881094912 shutdown_connections 2024-02-23T00:33:33.593 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.591+0000 7f397f7d8700 1 --2- 172.21.15.47:0/1881094912 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f3964033fb0 0x7f3964036470 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:33.593 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.591+0000 7f397f7d8700 1 --2- 172.21.15.47:0/1881094912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39780713e0 0x7f397810aa70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:33.593 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.591+0000 7f397f7d8700 1 -- 172.21.15.47:0/1881094912 >> 172.21.15.47:0/1881094912 conn(0x7f39780f6540 msgr2=0x7f39780f7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:33.593 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.591+0000 7f397f7d8700 1 -- 172.21.15.47:0/1881094912 shutdown_connections 2024-02-23T00:33:33.593 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.591+0000 7f397f7d8700 1 -- 172.21.15.47:0/1881094912 wait complete. 2024-02-23T00:33:33.594 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:33.591+0000 7f397f7d8700 1 librados: shutdown 2024-02-23T00:33:33.901 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: Generating public/private rsa key pair. 2024-02-23T00:33:33.902 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: Your identification has been saved in /tmp/tmp1j7lqn0d/key. 2024-02-23T00:33:33.902 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: Your public key has been saved in /tmp/tmp1j7lqn0d/key.pub. 2024-02-23T00:33:33.902 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: The key fingerprint is: 2024-02-23T00:33:33.902 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: SHA256:V5tKwu4IQdVhhAQ3bf1hoaw1a4XOog1BLcgGRkS3tgs ceph-fce2617e-d1e2-11ee-95c0-87774f69a715 2024-02-23T00:33:33.902 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: The key's randomart image is: 2024-02-23T00:33:33.902 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: +---[RSA 3072]----+ 2024-02-23T00:33:33.902 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: | +*oo+**+o .. | 2024-02-23T00:33:33.902 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: | . .+=oo=..oo | 2024-02-23T00:33:33.902 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: | .+ .o *oo. | 2024-02-23T00:33:33.902 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: | o . o = =.o | 2024-02-23T00:33:33.902 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: | E o . S B o | 2024-02-23T00:33:33.902 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: | . o = * . | 2024-02-23T00:33:33.902 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: | o . o . | 2024-02-23T00:33:33.903 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: | . o | 2024-02-23T00:33:33.903 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: | . . | 2024-02-23T00:33:33.903 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: +----[SHA256]-----+ 2024-02-23T00:33:34.232 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:33.383117+0000 mgr.y (mgr.14116) 6 : audit [DBG] from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:33:34.233 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:33:33.383343+0000 mgr.y (mgr.14116) 7 : cephadm [INF] Generating ssh key... 2024-02-23T00:33:34.233 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:33.586485+0000 mon.a (mon.0) 51 : audit [INF] from='mgr.14116 172.21.15.47:0/1137377827' entity='mgr.y' 2024-02-23T00:33:34.233 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:33.588275+0000 mon.a (mon.0) 52 : audit [INF] from='mgr.14116 172.21.15.47:0/1137377827' entity='mgr.y' 2024-02-23T00:33:35.032 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.024+0000 7fa19ae46700 1 Processor -- start 2024-02-23T00:33:35.033 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.024+0000 7fa19ae46700 1 -- start start 2024-02-23T00:33:35.033 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.024+0000 7fa19ae46700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa194073430 0x7fa194071a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:35.033 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.024+0000 7fa19ae46700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa194073850 con 0x7fa194073430 2024-02-23T00:33:35.033 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.025+0000 7fa198be2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa194073430 0x7fa194071a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:35.033 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.025+0000 7fa198be2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa194073430 0x7fa194071a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44800/0 (socket says 172.21.15.47:44800) 2024-02-23T00:33:35.033 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.025+0000 7fa198be2700 1 -- 172.21.15.47:0/2725135653 learned_addr learned my addr 172.21.15.47:0/2725135653 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:35.033 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.025+0000 7fa198be2700 1 -- 172.21.15.47:0/2725135653 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa194071f10 con 0x7fa194073430 2024-02-23T00:33:35.033 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.025+0000 7fa198be2700 1 --2- 172.21.15.47:0/2725135653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa194073430 0x7fa194071a20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fa188009730 tx=0x7fa1880090f0).ready entity=mon.0 client_cookie=c33bb3157be91d77 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.025+0000 7fa1937fe700 1 -- 172.21.15.47:0/2725135653 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa18800b580 con 0x7fa194073430 2024-02-23T00:33:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.025+0000 7fa1937fe700 1 -- 172.21.15.47:0/2725135653 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7fa18800bb50 con 0x7fa194073430 2024-02-23T00:33:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.025+0000 7fa1937fe700 1 -- 172.21.15.47:0/2725135653 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa188011ab0 con 0x7fa194073430 2024-02-23T00:33:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.026+0000 7fa19ae46700 1 -- 172.21.15.47:0/2725135653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa194073430 msgr2=0x7fa194071a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.026+0000 7fa19ae46700 1 --2- 172.21.15.47:0/2725135653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa194073430 0x7fa194071a20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fa188009730 tx=0x7fa1880090f0).stop 2024-02-23T00:33:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.026+0000 7fa19ae46700 1 -- 172.21.15.47:0/2725135653 shutdown_connections 2024-02-23T00:33:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.026+0000 7fa19ae46700 1 --2- 172.21.15.47:0/2725135653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa194073430 0x7fa194071a20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.026+0000 7fa19ae46700 1 -- 172.21.15.47:0/2725135653 >> 172.21.15.47:0/2725135653 conn(0x7fa1940f7850 msgr2=0x7fa1940f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:35.035 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.026+0000 7fa19ae46700 1 -- 172.21.15.47:0/2725135653 shutdown_connections 2024-02-23T00:33:35.035 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.026+0000 7fa19ae46700 1 -- 172.21.15.47:0/2725135653 wait complete. 2024-02-23T00:33:35.035 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.027+0000 7fa19ae46700 1 librados: starting msgr at 2024-02-23T00:33:35.035 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.027+0000 7fa19ae46700 1 librados: starting objecter 2024-02-23T00:33:35.035 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.027+0000 7fa19ae46700 1 Processor -- start 2024-02-23T00:33:35.035 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.027+0000 7fa19ae46700 1 -- start start 2024-02-23T00:33:35.035 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.027+0000 7fa19ae46700 1 librados: setting wanted keys 2024-02-23T00:33:35.035 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.027+0000 7fa19ae46700 1 librados: calling monclient init 2024-02-23T00:33:35.035 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.027+0000 7fa19ae46700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa194073430 0x7fa19410cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:35.036 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.027+0000 7fa19ae46700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19410d220 con 0x7fa194073430 2024-02-23T00:33:35.036 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.027+0000 7fa198be2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa194073430 0x7fa19410cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:35.036 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.027+0000 7fa198be2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa194073430 0x7fa19410cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44806/0 (socket says 172.21.15.47:44806) 2024-02-23T00:33:35.036 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.027+0000 7fa198be2700 1 -- 172.21.15.47:0/3958717109 learned_addr learned my addr 172.21.15.47:0/3958717109 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:35.036 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.028+0000 7fa198be2700 1 -- 172.21.15.47:0/3958717109 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa188009370 con 0x7fa194073430 2024-02-23T00:33:35.036 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.028+0000 7fa198be2700 1 --2- 172.21.15.47:0/3958717109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa194073430 0x7fa19410cd30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fa188009f00 tx=0x7fa1880090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:35.036 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.028+0000 7fa191ffb700 10 client.?.objecter ms_handle_connect 0x7fa194073430 2024-02-23T00:33:35.036 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.028+0000 7fa191ffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:33:35.037 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.028+0000 7fa191ffb700 1 -- 172.21.15.47:0/3958717109 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1880118f0 con 0x7fa194073430 2024-02-23T00:33:35.037 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.028+0000 7fa191ffb700 1 -- 172.21.15.47:0/3958717109 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7fa188011400 con 0x7fa194073430 2024-02-23T00:33:35.037 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.028+0000 7fa19ae46700 1 -- 172.21.15.47:0/3958717109 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa19410d420 con 0x7fa194073430 2024-02-23T00:33:35.037 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.028+0000 7fa191ffb700 1 -- 172.21.15.47:0/3958717109 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa188023940 con 0x7fa194073430 2024-02-23T00:33:35.037 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.028+0000 7fa19ae46700 10 client.14132.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:33:35.037 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.028+0000 7fa19ae46700 1 -- 172.21.15.47:0/3958717109 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa19410dfd0 con 0x7fa194073430 2024-02-23T00:33:35.037 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.028+0000 7fa19ae46700 1 librados: init done 2024-02-23T00:33:35.037 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.029+0000 7fa191ffb700 1 -- 172.21.15.47:0/3958717109 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) v1 ==== 38617+0+0 (secure 0 0 0) 0x7fa18801a640 con 0x7fa194073430 2024-02-23T00:33:35.038 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.029+0000 7fa19ae46700 1 -- 172.21.15.47:0/3958717109 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa19405a310 con 0x7fa194073430 2024-02-23T00:33:35.038 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.029+0000 7fa191ffb700 1 --2- 172.21.15.47:0/3958717109 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7fa184033fb0 0x7fa184036470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:35.038 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.030+0000 7fa191ffb700 1 -- 172.21.15.47:0/3958717109 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7fa188044ae0 con 0x7fa194073430 2024-02-23T00:33:35.038 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.030+0000 7fa191ffb700 10 client.14132.objecter ms_dispatch 0x7fa194001550 osd_map(2..2 src has 1..2) v4 2024-02-23T00:33:35.038 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.030+0000 7fa191ffb700 3 client.14132.objecter handle_osd_map got epochs [2,2] > 0 2024-02-23T00:33:35.038 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.030+0000 7fa191ffb700 3 client.14132.objecter handle_osd_map decoding full epoch 2 2024-02-23T00:33:35.038 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.030+0000 7fa191ffb700 20 client.14132.objecter dump_active .. 0 homeless 2024-02-23T00:33:35.038 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.030+0000 7fa193fff700 1 --2- 172.21.15.47:0/3958717109 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7fa184033fb0 0x7fa184036470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:35.038 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.030+0000 7fa193fff700 1 --2- 172.21.15.47:0/3958717109 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7fa184033fb0 0x7fa184036470 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fa180009780 tx=0x7fa180006cc0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:35.039 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.030+0000 7fa191ffb700 10 client.14132.objecter ms_handle_connect 0x7fa184033fb0 2024-02-23T00:33:35.039 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.034+0000 7fa191ffb700 1 -- 172.21.15.47:0/3958717109 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fa188019960 con 0x7fa194073430 2024-02-23T00:33:35.209 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.208+0000 7fa19ae46700 1 -- 172.21.15.47:0/3958717109 --> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fa19410e210 con 0x7fa184033fb0 2024-02-23T00:33:35.210 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.209+0000 7fa191ffb700 1 -- 172.21.15.47:0/3958717109 <== mgr.14116 v2:172.21.15.47:6800/1916598887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fa19410e210 con 0x7fa184033fb0 2024-02-23T00:33:35.210 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzKgjPTR2hwImYTv0iLqANPGRqURcRKmXdJVbOiDqCxX3Bqga20+evwIci0xwlOXp+Ne6FILdQ3X8/v33Z0fFAPERRq3dNan5QJFuOS3G9mp78yy69ikoey8eb73epKR9UpenH/U10fvRayxwIgca+lvwRXaTQRStP1uyJG4/cXbRm443SsuIi7NEmNgYZNGZ4oDW8LBEt97eZxQ1p0TklvnF4ozQ/TgsxQvnDcD5fhNO9URl21Q2ekL59RYPG9xVLAmjO8G3Y3FS8TVeVcJ7z68ok6opskjegliBJlwfSA6Gxf/QZ+tyfgvCmuarcG5GQLVcfatBO8dxaUY6GxKKDeQwUlm1fzGhuswZnza+hQvn09qpxL8Cau8rA2F+/W7GSkxrjUhjX9b88NRKz8DVFfq88WZw0fMvc8teXRXzV2tEgUGYMH0pRIU+ehEedjLdmHzG+yAa5BmbUsWXn+G4o9PmNMV5e3dYUfQ09kFASPhHmV5kdf6KRgN7PmL6j8Nc= ceph-fce2617e-d1e2-11ee-95c0-87774f69a715 2024-02-23T00:33:35.211 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.211+0000 7fa19ae46700 10 librados: watch_flush enter 2024-02-23T00:33:35.212 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.211+0000 7fa19ae46700 10 librados: watch_flush exit 2024-02-23T00:33:35.212 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.211+0000 7fa19ae46700 20 client.14132.objecter shutdown clearing up homeless session... 2024-02-23T00:33:35.212 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.211+0000 7fa19ae46700 10 client.14132.objecter successfully canceled tick 2024-02-23T00:33:35.213 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.211+0000 7fa19ae46700 1 -- 172.21.15.47:0/3958717109 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7fa184033fb0 msgr2=0x7fa184036470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:35.213 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.211+0000 7fa19ae46700 1 --2- 172.21.15.47:0/3958717109 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7fa184033fb0 0x7fa184036470 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fa180009780 tx=0x7fa180006cc0).stop 2024-02-23T00:33:35.213 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.211+0000 7fa19ae46700 1 -- 172.21.15.47:0/3958717109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa194073430 msgr2=0x7fa19410cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:35.213 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.211+0000 7fa19ae46700 1 --2- 172.21.15.47:0/3958717109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa194073430 0x7fa19410cd30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fa188009f00 tx=0x7fa1880090f0).stop 2024-02-23T00:33:35.214 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.211+0000 7fa19ae46700 1 -- 172.21.15.47:0/3958717109 shutdown_connections 2024-02-23T00:33:35.214 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.211+0000 7fa19ae46700 1 --2- 172.21.15.47:0/3958717109 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7fa184033fb0 0x7fa184036470 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:35.214 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.211+0000 7fa19ae46700 1 --2- 172.21.15.47:0/3958717109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa194073430 0x7fa19410cd30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:35.214 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.211+0000 7fa19ae46700 1 -- 172.21.15.47:0/3958717109 >> 172.21.15.47:0/3958717109 conn(0x7fa1940f7850 msgr2=0x7fa1940fa0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:35.214 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.211+0000 7fa19ae46700 1 -- 172.21.15.47:0/3958717109 shutdown_connections 2024-02-23T00:33:35.214 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.211+0000 7fa19ae46700 1 -- 172.21.15.47:0/3958717109 wait complete. 2024-02-23T00:33:35.214 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:35.211+0000 7fa19ae46700 1 librados: shutdown 2024-02-23T00:33:35.787 INFO:teuthology.orchestra.run.smithi047.stderr:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-23T00:33:35.787 INFO:teuthology.orchestra.run.smithi047.stderr:Adding key to root@localhost authorized_keys... 2024-02-23T00:33:35.787 INFO:teuthology.orchestra.run.smithi047.stderr:Adding host smithi047... 2024-02-23T00:33:35.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:35.209472+0000 mgr.y (mgr.14116) 8 : audit [DBG] from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:33:36.347 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.341+0000 7f04a043d700 1 Processor -- start 2024-02-23T00:33:36.347 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.341+0000 7f04a043d700 1 -- start start 2024-02-23T00:33:36.347 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.341+0000 7f04a043d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0498070b80 0x7f04980feab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:36.347 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.342+0000 7f04a043d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04980fefa0 con 0x7f0498070b80 2024-02-23T00:33:36.347 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.342+0000 7f049e1d9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0498070b80 0x7f04980feab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:36.347 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.342+0000 7f049e1d9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0498070b80 0x7f04980feab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44820/0 (socket says 172.21.15.47:44820) 2024-02-23T00:33:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.342+0000 7f049e1d9700 1 -- 172.21.15.47:0/114377648 learned_addr learned my addr 172.21.15.47:0/114377648 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.342+0000 7f049e1d9700 1 -- 172.21.15.47:0/114377648 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04980ff0e0 con 0x7f0498070b80 2024-02-23T00:33:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.342+0000 7f049e1d9700 1 --2- 172.21.15.47:0/114377648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0498070b80 0x7f04980feab0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f0488009730 tx=0x7f04880090f0).ready entity=mon.0 client_cookie=83e4bf02addfd10e server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.342+0000 7f049d1d7700 1 -- 172.21.15.47:0/114377648 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f048800b610 con 0x7f0498070b80 2024-02-23T00:33:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.342+0000 7f049d1d7700 1 -- 172.21.15.47:0/114377648 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7f048800bbe0 con 0x7f0498070b80 2024-02-23T00:33:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.342+0000 7f049d1d7700 1 -- 172.21.15.47:0/114377648 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0488011b50 con 0x7f0498070b80 2024-02-23T00:33:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.343+0000 7f04a043d700 1 -- 172.21.15.47:0/114377648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0498070b80 msgr2=0x7f04980feab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.343+0000 7f04a043d700 1 --2- 172.21.15.47:0/114377648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0498070b80 0x7f04980feab0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f0488009730 tx=0x7f04880090f0).stop 2024-02-23T00:33:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.343+0000 7f04a043d700 1 -- 172.21.15.47:0/114377648 shutdown_connections 2024-02-23T00:33:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.343+0000 7f04a043d700 1 --2- 172.21.15.47:0/114377648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0498070b80 0x7f04980feab0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.343+0000 7f04a043d700 1 -- 172.21.15.47:0/114377648 >> 172.21.15.47:0/114377648 conn(0x7f04980f6530 msgr2=0x7f04980f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.343+0000 7f04a043d700 1 -- 172.21.15.47:0/114377648 shutdown_connections 2024-02-23T00:33:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.343+0000 7f04a043d700 1 -- 172.21.15.47:0/114377648 wait complete. 2024-02-23T00:33:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.343+0000 7f04a043d700 1 librados: starting msgr at 2024-02-23T00:33:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.343+0000 7f04a043d700 1 librados: starting objecter 2024-02-23T00:33:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.344+0000 7f04a043d700 1 Processor -- start 2024-02-23T00:33:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.344+0000 7f04a043d700 1 -- start start 2024-02-23T00:33:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.344+0000 7f04a043d700 1 librados: setting wanted keys 2024-02-23T00:33:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.344+0000 7f04a043d700 1 librados: calling monclient init 2024-02-23T00:33:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.344+0000 7f04a043d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0498070b80 0x7f049810a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.344+0000 7f04a043d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049810aee0 con 0x7f0498070b80 2024-02-23T00:33:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.344+0000 7f049e1d9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0498070b80 0x7f049810a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.344+0000 7f049e1d9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0498070b80 0x7f049810a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44828/0 (socket says 172.21.15.47:44828) 2024-02-23T00:33:36.350 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.344+0000 7f049e1d9700 1 -- 172.21.15.47:0/4274513776 learned_addr learned my addr 172.21.15.47:0/4274513776 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:36.350 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.344+0000 7f049e1d9700 1 -- 172.21.15.47:0/4274513776 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0488009370 con 0x7f0498070b80 2024-02-23T00:33:36.350 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.344+0000 7f049e1d9700 1 --2- 172.21.15.47:0/4274513776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0498070b80 0x7f049810a9f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f0488012fc0 tx=0x7f04880090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:36.350 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.344+0000 7f048f7fe700 10 client.?.objecter ms_handle_connect 0x7f0498070b80 2024-02-23T00:33:36.350 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.344+0000 7f048f7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:33:36.350 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.345+0000 7f048f7fe700 1 -- 172.21.15.47:0/4274513776 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f048801a640 con 0x7f0498070b80 2024-02-23T00:33:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.345+0000 7f048f7fe700 1 -- 172.21.15.47:0/4274513776 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7f048801ac10 con 0x7f0498070b80 2024-02-23T00:33:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.345+0000 7f04a043d700 1 -- 172.21.15.47:0/4274513776 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f049810b0e0 con 0x7f0498070b80 2024-02-23T00:33:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.345+0000 7f048f7fe700 1 -- 172.21.15.47:0/4274513776 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0488023b70 con 0x7f0498070b80 2024-02-23T00:33:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.345+0000 7f04a043d700 10 client.14134.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:33:36.352 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.345+0000 7f04a043d700 1 -- 172.21.15.47:0/4274513776 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f049810b570 con 0x7f0498070b80 2024-02-23T00:33:36.352 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.345+0000 7f04a043d700 1 librados: init done 2024-02-23T00:33:36.352 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.345+0000 7f048f7fe700 1 -- 172.21.15.47:0/4274513776 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) v1 ==== 38617+0+0 (secure 0 0 0) 0x7f048801a7a0 con 0x7f0498070b80 2024-02-23T00:33:36.352 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.346+0000 7f048f7fe700 1 --2- 172.21.15.47:0/4274513776 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f0484033f60 0x7f0484036420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:36.352 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.346+0000 7f048f7fe700 1 -- 172.21.15.47:0/4274513776 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7f0488044da0 con 0x7f0498070b80 2024-02-23T00:33:36.352 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.346+0000 7f04a043d700 1 -- 172.21.15.47:0/4274513776 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0498046ce0 con 0x7f0498070b80 2024-02-23T00:33:36.352 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.346+0000 7f048f7fe700 10 client.14134.objecter ms_dispatch 0x7f0498001550 osd_map(2..2 src has 1..2) v4 2024-02-23T00:33:36.352 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.346+0000 7f048f7fe700 3 client.14134.objecter handle_osd_map got epochs [2,2] > 0 2024-02-23T00:33:36.352 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.346+0000 7f048f7fe700 3 client.14134.objecter handle_osd_map decoding full epoch 2 2024-02-23T00:33:36.352 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.346+0000 7f048f7fe700 20 client.14134.objecter dump_active .. 0 homeless 2024-02-23T00:33:36.352 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.346+0000 7f049d9d8700 1 --2- 172.21.15.47:0/4274513776 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f0484033f60 0x7f0484036420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:36.352 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.346+0000 7f048f7fe700 10 client.14134.objecter ms_handle_connect 0x7f0484033f60 2024-02-23T00:33:36.353 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.346+0000 7f049d9d8700 1 --2- 172.21.15.47:0/4274513776 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f0484033f60 0x7f0484036420 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f0494009780 tx=0x7f0494006cc0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:36.353 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.350+0000 7f048f7fe700 1 -- 172.21.15.47:0/4274513776 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f048802c3f0 con 0x7f0498070b80 2024-02-23T00:33:36.531 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:36.529+0000 7f04a043d700 1 -- 172.21.15.47:0/4274513776 --> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi047", "addr": "172.21.15.47", "target": ["mon-mgr", ""]}) v1 -- 0x7f0498104630 con 0x7f0484033f60 2024-02-23T00:33:36.806 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [79B blob data] 2024-02-23T00:33:37.174 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: Added host 'smithi047' with addr '172.21.15.47' 2024-02-23T00:33:37.174 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:37.171+0000 7f048f7fe700 1 -- 172.21.15.47:0/4274513776 <== mgr.14116 v2:172.21.15.47:6800/1916598887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f0498104630 con 0x7f0484033f60 2024-02-23T00:33:37.176 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:37.173+0000 7f04a043d700 10 librados: watch_flush enter 2024-02-23T00:33:37.176 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:37.173+0000 7f04a043d700 10 librados: watch_flush exit 2024-02-23T00:33:37.176 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:37.173+0000 7f04a043d700 20 client.14134.objecter shutdown clearing up homeless session... 2024-02-23T00:33:37.176 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:37.173+0000 7f04a043d700 10 client.14134.objecter successfully canceled tick 2024-02-23T00:33:37.176 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:37.173+0000 7f04a043d700 1 -- 172.21.15.47:0/4274513776 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f0484033f60 msgr2=0x7f0484036420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:37.176 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:37.173+0000 7f04a043d700 1 --2- 172.21.15.47:0/4274513776 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f0484033f60 0x7f0484036420 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f0494009780 tx=0x7f0494006cc0).stop 2024-02-23T00:33:37.176 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:37.173+0000 7f04a043d700 1 -- 172.21.15.47:0/4274513776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0498070b80 msgr2=0x7f049810a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:37.177 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:37.173+0000 7f04a043d700 1 --2- 172.21.15.47:0/4274513776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0498070b80 0x7f049810a9f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f0488012fc0 tx=0x7f04880090f0).stop 2024-02-23T00:33:37.177 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:37.173+0000 7f04a043d700 1 -- 172.21.15.47:0/4274513776 shutdown_connections 2024-02-23T00:33:37.177 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:37.173+0000 7f04a043d700 1 --2- 172.21.15.47:0/4274513776 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f0484033f60 0x7f0484036420 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:37.177 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:37.173+0000 7f04a043d700 1 --2- 172.21.15.47:0/4274513776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0498070b80 0x7f049810a9f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:37.177 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:37.173+0000 7f04a043d700 1 -- 172.21.15.47:0/4274513776 >> 172.21.15.47:0/4274513776 conn(0x7f04980f6530 msgr2=0x7f04980f7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:37.177 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:37.173+0000 7f04a043d700 1 -- 172.21.15.47:0/4274513776 shutdown_connections 2024-02-23T00:33:37.177 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:37.173+0000 7f04a043d700 1 -- 172.21.15.47:0/4274513776 wait complete. 2024-02-23T00:33:37.177 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:37.173+0000 7f04a043d700 1 librados: shutdown 2024-02-23T00:33:37.868 INFO:teuthology.orchestra.run.smithi047.stderr:Deploying unmanaged mon service... 2024-02-23T00:33:38.517 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.459+0000 7ffaa5b66700 1 Processor -- start 2024-02-23T00:33:38.518 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.459+0000 7ffaa5b66700 1 -- start start 2024-02-23T00:33:38.518 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.459+0000 7ffaa5b66700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffaa0070b20 0x7ffaa0100f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:38.518 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.459+0000 7ffaa5b66700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa00fc900 con 0x7ffaa0070b20 2024-02-23T00:33:38.518 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.459+0000 7ffa9f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffaa0070b20 0x7ffaa0100f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:38.518 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.459+0000 7ffa9f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffaa0070b20 0x7ffaa0100f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39476/0 (socket says 172.21.15.47:39476) 2024-02-23T00:33:38.518 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.459+0000 7ffa9f7fe700 1 -- 172.21.15.47:0/1968682180 learned_addr learned my addr 172.21.15.47:0/1968682180 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:38.518 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.460+0000 7ffa9f7fe700 1 -- 172.21.15.47:0/1968682180 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa0101400 con 0x7ffaa0070b20 2024-02-23T00:33:38.518 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.460+0000 7ffa9f7fe700 1 --2- 172.21.15.47:0/1968682180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffaa0070b20 0x7ffaa0100f10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7ffa90009730 tx=0x7ffa900090f0).ready entity=mon.0 client_cookie=c88e268cb6d46852 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:38.518 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.460+0000 7ffa9e7fc700 1 -- 172.21.15.47:0/1968682180 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa9000b580 con 0x7ffaa0070b20 2024-02-23T00:33:38.518 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.460+0000 7ffa9e7fc700 1 -- 172.21.15.47:0/1968682180 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7ffa9000bb50 con 0x7ffaa0070b20 2024-02-23T00:33:38.519 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.460+0000 7ffa9e7fc700 1 -- 172.21.15.47:0/1968682180 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa90011ab0 con 0x7ffaa0070b20 2024-02-23T00:33:38.519 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.460+0000 7ffaa5b66700 1 -- 172.21.15.47:0/1968682180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffaa0070b20 msgr2=0x7ffaa0100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:38.519 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.460+0000 7ffaa5b66700 1 --2- 172.21.15.47:0/1968682180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffaa0070b20 0x7ffaa0100f10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7ffa90009730 tx=0x7ffa900090f0).stop 2024-02-23T00:33:38.519 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.460+0000 7ffaa5b66700 1 -- 172.21.15.47:0/1968682180 shutdown_connections 2024-02-23T00:33:38.519 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.460+0000 7ffaa5b66700 1 --2- 172.21.15.47:0/1968682180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffaa0070b20 0x7ffaa0100f10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:38.519 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.460+0000 7ffaa5b66700 1 -- 172.21.15.47:0/1968682180 >> 172.21.15.47:0/1968682180 conn(0x7ffaa00f6530 msgr2=0x7ffaa00f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:38.519 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.461+0000 7ffaa5b66700 1 -- 172.21.15.47:0/1968682180 shutdown_connections 2024-02-23T00:33:38.519 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.461+0000 7ffaa5b66700 1 -- 172.21.15.47:0/1968682180 wait complete. 2024-02-23T00:33:38.519 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.461+0000 7ffaa5b66700 1 librados: starting msgr at 2024-02-23T00:33:38.519 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.461+0000 7ffaa5b66700 1 librados: starting objecter 2024-02-23T00:33:38.519 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.461+0000 7ffaa5b66700 1 Processor -- start 2024-02-23T00:33:38.519 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.461+0000 7ffaa5b66700 1 -- start start 2024-02-23T00:33:38.519 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.461+0000 7ffaa5b66700 1 librados: setting wanted keys 2024-02-23T00:33:38.519 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.461+0000 7ffaa5b66700 1 librados: calling monclient init 2024-02-23T00:33:38.520 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.461+0000 7ffaa5b66700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffaa0070b20 0x7ffaa010cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:38.520 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.461+0000 7ffaa5b66700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa010d1f0 con 0x7ffaa0070b20 2024-02-23T00:33:38.520 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.462+0000 7ffa9f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffaa0070b20 0x7ffaa010cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:38.520 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.462+0000 7ffa9f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffaa0070b20 0x7ffaa010cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39478/0 (socket says 172.21.15.47:39478) 2024-02-23T00:33:38.520 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.462+0000 7ffa9f7fe700 1 -- 172.21.15.47:0/3450709559 learned_addr learned my addr 172.21.15.47:0/3450709559 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:38.520 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.462+0000 7ffa9f7fe700 1 -- 172.21.15.47:0/3450709559 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa90009370 con 0x7ffaa0070b20 2024-02-23T00:33:38.520 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.462+0000 7ffa9f7fe700 1 --2- 172.21.15.47:0/3450709559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffaa0070b20 0x7ffaa010cd00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7ffa90012f20 tx=0x7ffa900090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:38.520 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.462+0000 7ffa9cff9700 10 client.?.objecter ms_handle_connect 0x7ffaa0070b20 2024-02-23T00:33:38.520 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.462+0000 7ffa9cff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:33:38.520 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.462+0000 7ffa9cff9700 1 -- 172.21.15.47:0/3450709559 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa90011a70 con 0x7ffaa0070b20 2024-02-23T00:33:38.520 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.462+0000 7ffa9cff9700 1 -- 172.21.15.47:0/3450709559 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7ffa9001aab0 con 0x7ffaa0070b20 2024-02-23T00:33:38.520 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.462+0000 7ffaa5b66700 1 -- 172.21.15.47:0/3450709559 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa010d3f0 con 0x7ffaa0070b20 2024-02-23T00:33:38.520 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.462+0000 7ffa9cff9700 1 -- 172.21.15.47:0/3450709559 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa90023960 con 0x7ffaa0070b20 2024-02-23T00:33:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.462+0000 7ffaa5b66700 10 client.14136.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:33:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.462+0000 7ffaa5b66700 1 -- 172.21.15.47:0/3450709559 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa010dfa0 con 0x7ffaa0070b20 2024-02-23T00:33:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.462+0000 7ffaa5b66700 1 librados: init done 2024-02-23T00:33:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.463+0000 7ffa9cff9700 1 -- 172.21.15.47:0/3450709559 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) v1 ==== 38617+0+0 (secure 0 0 0) 0x7ffa9001a640 con 0x7ffaa0070b20 2024-02-23T00:33:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.463+0000 7ffaa5b66700 1 -- 172.21.15.47:0/3450709559 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa005a310 con 0x7ffaa0070b20 2024-02-23T00:33:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.463+0000 7ffa9cff9700 1 --2- 172.21.15.47:0/3450709559 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7ffa88033fb0 0x7ffa88036470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.463+0000 7ffa9cff9700 1 -- 172.21.15.47:0/3450709559 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7ffa90044d00 con 0x7ffaa0070b20 2024-02-23T00:33:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.463+0000 7ffa9cff9700 10 client.14136.objecter ms_dispatch 0x7ffaa0001550 osd_map(2..2 src has 1..2) v4 2024-02-23T00:33:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.463+0000 7ffa9cff9700 3 client.14136.objecter handle_osd_map got epochs [2,2] > 0 2024-02-23T00:33:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.463+0000 7ffa9cff9700 3 client.14136.objecter handle_osd_map decoding full epoch 2 2024-02-23T00:33:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.463+0000 7ffa9cff9700 20 client.14136.objecter dump_active .. 0 homeless 2024-02-23T00:33:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.463+0000 7ffa9effd700 1 --2- 172.21.15.47:0/3450709559 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7ffa88033fb0 0x7ffa88036470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.464+0000 7ffa9cff9700 10 client.14136.objecter ms_handle_connect 0x7ffa88033fb0 2024-02-23T00:33:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.464+0000 7ffa9effd700 1 --2- 172.21.15.47:0/3450709559 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7ffa88033fb0 0x7ffa88036470 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7ffa94009780 tx=0x7ffa94006cc0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.468+0000 7ffa9cff9700 1 -- 172.21.15.47:0/3450709559 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ffa900199e0 con 0x7ffaa0070b20 2024-02-23T00:33:38.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:36.531248+0000 mgr.y (mgr.14116) 9 : audit [DBG] from='client.14134 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi047", "addr": "172.21.15.47", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:33:38.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:37.171651+0000 mon.a (mon.0) 53 : audit [INF] from='mgr.14116 172.21.15.47:0/1137377827' entity='mgr.y' 2024-02-23T00:33:38.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:33:37.171959+0000 mgr.y (mgr.14116) 10 : cephadm [INF] Added host smithi047 2024-02-23T00:33:38.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:37.204155+0000 mon.a (mon.0) 54 : audit [DBG] from='mgr.14116 172.21.15.47:0/1137377827' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:33:38.654 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.652+0000 7ffaa5b66700 1 -- 172.21.15.47:0/3450709559 --> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7ffaa010e1e0 con 0x7ffa88033fb0 2024-02-23T00:33:38.658 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.656+0000 7ffa9cff9700 1 -- 172.21.15.47:0/3450709559 <== mgr.14116 v2:172.21.15.47:6800/1916598887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7ffaa010e1e0 con 0x7ffa88033fb0 2024-02-23T00:33:38.658 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: Scheduled mon update... 2024-02-23T00:33:38.661 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.658+0000 7ffaa5b66700 10 librados: watch_flush enter 2024-02-23T00:33:38.661 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.658+0000 7ffaa5b66700 10 librados: watch_flush exit 2024-02-23T00:33:38.662 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.658+0000 7ffaa5b66700 20 client.14136.objecter shutdown clearing up homeless session... 2024-02-23T00:33:38.662 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.658+0000 7ffaa5b66700 10 client.14136.objecter successfully canceled tick 2024-02-23T00:33:38.662 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.658+0000 7ffaa5b66700 1 -- 172.21.15.47:0/3450709559 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7ffa88033fb0 msgr2=0x7ffa88036470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:38.662 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.658+0000 7ffaa5b66700 1 --2- 172.21.15.47:0/3450709559 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7ffa88033fb0 0x7ffa88036470 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7ffa94009780 tx=0x7ffa94006cc0).stop 2024-02-23T00:33:38.662 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.658+0000 7ffaa5b66700 1 -- 172.21.15.47:0/3450709559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffaa0070b20 msgr2=0x7ffaa010cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:38.662 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.658+0000 7ffaa5b66700 1 --2- 172.21.15.47:0/3450709559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffaa0070b20 0x7ffaa010cd00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7ffa90012f20 tx=0x7ffa900090f0).stop 2024-02-23T00:33:38.662 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.658+0000 7ffaa5b66700 1 -- 172.21.15.47:0/3450709559 shutdown_connections 2024-02-23T00:33:38.662 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.658+0000 7ffaa5b66700 1 --2- 172.21.15.47:0/3450709559 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7ffa88033fb0 0x7ffa88036470 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:38.662 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.658+0000 7ffaa5b66700 1 --2- 172.21.15.47:0/3450709559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffaa0070b20 0x7ffaa010cd00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:38.662 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.658+0000 7ffaa5b66700 1 -- 172.21.15.47:0/3450709559 >> 172.21.15.47:0/3450709559 conn(0x7ffaa00f6530 msgr2=0x7ffaa00f7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:38.662 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.659+0000 7ffaa5b66700 1 -- 172.21.15.47:0/3450709559 shutdown_connections 2024-02-23T00:33:38.662 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.659+0000 7ffaa5b66700 1 -- 172.21.15.47:0/3450709559 wait complete. 2024-02-23T00:33:38.662 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:38.659+0000 7ffaa5b66700 1 librados: shutdown 2024-02-23T00:33:39.378 INFO:teuthology.orchestra.run.smithi047.stderr:Deploying unmanaged mgr service... 2024-02-23T00:33:39.912 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:38.654064+0000 mgr.y (mgr.14116) 11 : audit [DBG] from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:33:39.912 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:33:38.655341+0000 mgr.y (mgr.14116) 12 : cephadm [INF] Saving service mon spec with placement count:5 2024-02-23T00:33:39.912 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:38.657632+0000 mon.a (mon.0) 55 : audit [INF] from='mgr.14116 172.21.15.47:0/1137377827' entity='mgr.y' 2024-02-23T00:33:40.238 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.133+0000 7fb42ada1700 1 Processor -- start 2024-02-23T00:33:40.239 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.134+0000 7fb42ada1700 1 -- start start 2024-02-23T00:33:40.239 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.134+0000 7fb42ada1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb424073460 0x7fb424071a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:40.239 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.134+0000 7fb42ada1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb424073880 con 0x7fb424073460 2024-02-23T00:33:40.239 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.134+0000 7fb429d9f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb424073460 0x7fb424071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:40.239 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.134+0000 7fb429d9f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb424073460 0x7fb424071a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39488/0 (socket says 172.21.15.47:39488) 2024-02-23T00:33:40.239 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.134+0000 7fb429d9f700 1 -- 172.21.15.47:0/855795183 learned_addr learned my addr 172.21.15.47:0/855795183 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:40.239 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.134+0000 7fb429d9f700 1 -- 172.21.15.47:0/855795183 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb424071ef0 con 0x7fb424073460 2024-02-23T00:33:40.239 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.134+0000 7fb429d9f700 1 --2- 172.21.15.47:0/855795183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb424073460 0x7fb424071a00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fb420009730 tx=0x7fb4200090f0).ready entity=mon.0 client_cookie=e57f84e917e363e5 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:40.239 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.135+0000 7fb428d9d700 1 -- 172.21.15.47:0/855795183 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb42000b580 con 0x7fb424073460 2024-02-23T00:33:40.239 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.135+0000 7fb428d9d700 1 -- 172.21.15.47:0/855795183 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7fb42000bb50 con 0x7fb424073460 2024-02-23T00:33:40.239 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.135+0000 7fb428d9d700 1 -- 172.21.15.47:0/855795183 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb420011ab0 con 0x7fb424073460 2024-02-23T00:33:40.239 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.135+0000 7fb42ada1700 1 -- 172.21.15.47:0/855795183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb424073460 msgr2=0x7fb424071a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:40.239 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.135+0000 7fb42ada1700 1 --2- 172.21.15.47:0/855795183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb424073460 0x7fb424071a00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fb420009730 tx=0x7fb4200090f0).stop 2024-02-23T00:33:40.240 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.135+0000 7fb42ada1700 1 -- 172.21.15.47:0/855795183 shutdown_connections 2024-02-23T00:33:40.240 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.135+0000 7fb42ada1700 1 --2- 172.21.15.47:0/855795183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb424073460 0x7fb424071a00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:40.240 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.135+0000 7fb42ada1700 1 -- 172.21.15.47:0/855795183 >> 172.21.15.47:0/855795183 conn(0x7fb4240f7850 msgr2=0x7fb4240f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:40.240 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.135+0000 7fb42ada1700 1 -- 172.21.15.47:0/855795183 shutdown_connections 2024-02-23T00:33:40.240 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.135+0000 7fb42ada1700 1 -- 172.21.15.47:0/855795183 wait complete. 2024-02-23T00:33:40.240 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.136+0000 7fb42ada1700 1 librados: starting msgr at 2024-02-23T00:33:40.240 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.136+0000 7fb42ada1700 1 librados: starting objecter 2024-02-23T00:33:40.240 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.136+0000 7fb42ada1700 1 Processor -- start 2024-02-23T00:33:40.240 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.136+0000 7fb42ada1700 1 -- start start 2024-02-23T00:33:40.240 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.136+0000 7fb42ada1700 1 librados: setting wanted keys 2024-02-23T00:33:40.240 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.136+0000 7fb42ada1700 1 librados: calling monclient init 2024-02-23T00:33:40.240 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.136+0000 7fb42ada1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb424073460 0x7fb42410cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:40.240 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.137+0000 7fb42ada1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42410cfa0 con 0x7fb424073460 2024-02-23T00:33:40.241 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.137+0000 7fb429d9f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb424073460 0x7fb42410cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:40.241 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.137+0000 7fb429d9f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb424073460 0x7fb42410cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39504/0 (socket says 172.21.15.47:39504) 2024-02-23T00:33:40.241 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.137+0000 7fb429d9f700 1 -- 172.21.15.47:0/3113938240 learned_addr learned my addr 172.21.15.47:0/3113938240 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:40.241 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.137+0000 7fb429d9f700 1 -- 172.21.15.47:0/3113938240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb420009370 con 0x7fb424073460 2024-02-23T00:33:40.241 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.137+0000 7fb429d9f700 1 --2- 172.21.15.47:0/3113938240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb424073460 0x7fb42410cab0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fb420012f20 tx=0x7fb4200090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:40.241 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.137+0000 7fb41affd700 10 client.?.objecter ms_handle_connect 0x7fb424073460 2024-02-23T00:33:40.241 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.137+0000 7fb41affd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:33:40.241 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.137+0000 7fb41affd700 1 -- 172.21.15.47:0/3113938240 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb420011a70 con 0x7fb424073460 2024-02-23T00:33:40.241 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.137+0000 7fb41affd700 1 -- 172.21.15.47:0/3113938240 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1023+0+0 (secure 0 0 0) 0x7fb42001aab0 con 0x7fb424073460 2024-02-23T00:33:40.241 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.137+0000 7fb41affd700 1 -- 172.21.15.47:0/3113938240 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4200238f0 con 0x7fb424073460 2024-02-23T00:33:40.241 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.137+0000 7fb42ada1700 1 -- 172.21.15.47:0/3113938240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb42410d1a0 con 0x7fb424073460 2024-02-23T00:33:40.241 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.137+0000 7fb42ada1700 10 client.14138.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:33:40.241 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.137+0000 7fb42ada1700 1 -- 172.21.15.47:0/3113938240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb42410d630 con 0x7fb424073460 2024-02-23T00:33:40.241 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.137+0000 7fb42ada1700 1 librados: init done 2024-02-23T00:33:40.242 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.138+0000 7fb41affd700 1 -- 172.21.15.47:0/3113938240 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) v1 ==== 38617+0+0 (secure 0 0 0) 0x7fb42001a640 con 0x7fb424073460 2024-02-23T00:33:40.242 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.138+0000 7fb42ada1700 1 -- 172.21.15.47:0/3113938240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb424046ce0 con 0x7fb424073460 2024-02-23T00:33:40.242 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.138+0000 7fb41affd700 1 --2- 172.21.15.47:0/3113938240 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7fb410033fb0 0x7fb410036470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:40.242 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.138+0000 7fb41affd700 1 -- 172.21.15.47:0/3113938240 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7fb420044c60 con 0x7fb424073460 2024-02-23T00:33:40.242 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.138+0000 7fb41affd700 10 client.14138.objecter ms_dispatch 0x7fb424001550 osd_map(2..2 src has 1..2) v4 2024-02-23T00:33:40.242 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.138+0000 7fb41affd700 3 client.14138.objecter handle_osd_map got epochs [2,2] > 0 2024-02-23T00:33:40.242 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.138+0000 7fb41affd700 3 client.14138.objecter handle_osd_map decoding full epoch 2 2024-02-23T00:33:40.242 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.138+0000 7fb41affd700 20 client.14138.objecter dump_active .. 0 homeless 2024-02-23T00:33:40.242 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.138+0000 7fb42959e700 1 --2- 172.21.15.47:0/3113938240 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7fb410033fb0 0x7fb410036470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:40.242 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.139+0000 7fb41affd700 10 client.14138.objecter ms_handle_connect 0x7fb410033fb0 2024-02-23T00:33:40.242 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.139+0000 7fb42959e700 1 --2- 172.21.15.47:0/3113938240 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7fb410033fb0 0x7fb410036470 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fb414009780 tx=0x7fb414006cc0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:40.242 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.143+0000 7fb41affd700 1 -- 172.21.15.47:0/3113938240 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb4200199e0 con 0x7fb424073460 2024-02-23T00:33:40.335 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.333+0000 7fb42ada1700 1 -- 172.21.15.47:0/3113938240 --> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fb424106670 con 0x7fb410033fb0 2024-02-23T00:33:40.339 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.337+0000 7fb41affd700 1 -- 172.21.15.47:0/3113938240 <== mgr.14116 v2:172.21.15.47:6800/1916598887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fb424106670 con 0x7fb410033fb0 2024-02-23T00:33:40.339 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: Scheduled mgr update... 2024-02-23T00:33:40.342 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.338+0000 7fb42ada1700 10 librados: watch_flush enter 2024-02-23T00:33:40.342 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.338+0000 7fb42ada1700 10 librados: watch_flush exit 2024-02-23T00:33:40.342 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.339+0000 7fb42ada1700 20 client.14138.objecter shutdown clearing up homeless session... 2024-02-23T00:33:40.342 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.339+0000 7fb42ada1700 10 client.14138.objecter successfully canceled tick 2024-02-23T00:33:40.342 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.339+0000 7fb42ada1700 1 -- 172.21.15.47:0/3113938240 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7fb410033fb0 msgr2=0x7fb410036470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:40.342 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.339+0000 7fb42ada1700 1 --2- 172.21.15.47:0/3113938240 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7fb410033fb0 0x7fb410036470 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fb414009780 tx=0x7fb414006cc0).stop 2024-02-23T00:33:40.342 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.339+0000 7fb42ada1700 1 -- 172.21.15.47:0/3113938240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb424073460 msgr2=0x7fb42410cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:40.342 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.339+0000 7fb42ada1700 1 --2- 172.21.15.47:0/3113938240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb424073460 0x7fb42410cab0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fb420012f20 tx=0x7fb4200090f0).stop 2024-02-23T00:33:40.342 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.339+0000 7fb42ada1700 1 -- 172.21.15.47:0/3113938240 shutdown_connections 2024-02-23T00:33:40.343 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.339+0000 7fb42ada1700 1 --2- 172.21.15.47:0/3113938240 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7fb410033fb0 0x7fb410036470 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:40.343 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.339+0000 7fb42ada1700 1 --2- 172.21.15.47:0/3113938240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb424073460 0x7fb42410cab0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:40.343 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.339+0000 7fb42ada1700 1 -- 172.21.15.47:0/3113938240 >> 172.21.15.47:0/3113938240 conn(0x7fb4240f7850 msgr2=0x7fb4240fa100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:40.343 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.340+0000 7fb42ada1700 1 -- 172.21.15.47:0/3113938240 shutdown_connections 2024-02-23T00:33:40.343 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.340+0000 7fb42ada1700 1 -- 172.21.15.47:0/3113938240 wait complete. 2024-02-23T00:33:40.343 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:40.340+0000 7fb42ada1700 1 librados: shutdown 2024-02-23T00:33:41.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:40.335110+0000 mgr.y (mgr.14116) 13 : audit [DBG] from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:33:41.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:33:40.335956+0000 mgr.y (mgr.14116) 14 : cephadm [INF] Saving service mgr spec with placement count:2 2024-02-23T00:33:41.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:40.338133+0000 mon.a (mon.0) 56 : audit [INF] from='mgr.14116 172.21.15.47:0/1137377827' entity='mgr.y' 2024-02-23T00:33:42.179 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.129+0000 7f010d339700 1 Processor -- start 2024-02-23T00:33:42.179 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.129+0000 7f010d339700 1 -- start start 2024-02-23T00:33:42.180 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.129+0000 7f010d339700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01080fadc0 0x7f01080fb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:42.180 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.129+0000 7f010d339700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01080fb6d0 con 0x7f01080fadc0 2024-02-23T00:33:42.180 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.129+0000 7f0106ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01080fadc0 0x7f01080fb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:42.180 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.129+0000 7f0106ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01080fadc0 0x7f01080fb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39510/0 (socket says 172.21.15.47:39510) 2024-02-23T00:33:42.180 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.129+0000 7f0106ffd700 1 -- 172.21.15.47:0/741648764 learned_addr learned my addr 172.21.15.47:0/741648764 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:42.180 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.130+0000 7f0106ffd700 1 -- 172.21.15.47:0/741648764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01080fb810 con 0x7f01080fadc0 2024-02-23T00:33:42.180 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.130+0000 7f0106ffd700 1 --2- 172.21.15.47:0/741648764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01080fadc0 0x7f01080fb1e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f0100009730 tx=0x7f01000090f0).ready entity=mon.0 client_cookie=5a9e4abfb2033f0d server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:42.180 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.130+0000 7f0105ffb700 1 -- 172.21.15.47:0/741648764 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f010000b580 con 0x7f01080fadc0 2024-02-23T00:33:42.180 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.130+0000 7f0105ffb700 1 -- 172.21.15.47:0/741648764 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f010000bb70 con 0x7f01080fadc0 2024-02-23T00:33:42.180 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.130+0000 7f0105ffb700 1 -- 172.21.15.47:0/741648764 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0100011a70 con 0x7f01080fadc0 2024-02-23T00:33:42.180 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.130+0000 7f010d339700 1 -- 172.21.15.47:0/741648764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01080fadc0 msgr2=0x7f01080fb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:42.180 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.130+0000 7f010d339700 1 --2- 172.21.15.47:0/741648764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01080fadc0 0x7f01080fb1e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f0100009730 tx=0x7f01000090f0).stop 2024-02-23T00:33:42.180 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.131+0000 7f010d339700 1 -- 172.21.15.47:0/741648764 shutdown_connections 2024-02-23T00:33:42.180 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.131+0000 7f010d339700 1 --2- 172.21.15.47:0/741648764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01080fadc0 0x7f01080fb1e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:42.181 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.131+0000 7f010d339700 1 -- 172.21.15.47:0/741648764 >> 172.21.15.47:0/741648764 conn(0x7f01080f6540 msgr2=0x7f01080f8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:42.181 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.131+0000 7f010d339700 1 -- 172.21.15.47:0/741648764 shutdown_connections 2024-02-23T00:33:42.181 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.131+0000 7f010d339700 1 -- 172.21.15.47:0/741648764 wait complete. 2024-02-23T00:33:42.181 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.131+0000 7f010d339700 1 librados: starting msgr at 2024-02-23T00:33:42.181 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.131+0000 7f010d339700 1 librados: starting objecter 2024-02-23T00:33:42.181 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.131+0000 7f010d339700 1 Processor -- start 2024-02-23T00:33:42.181 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.131+0000 7f010d339700 1 -- start start 2024-02-23T00:33:42.181 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.131+0000 7f010d339700 1 librados: setting wanted keys 2024-02-23T00:33:42.181 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.131+0000 7f010d339700 1 librados: calling monclient init 2024-02-23T00:33:42.181 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.131+0000 7f010d339700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01080fadc0 0x7f010810a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:42.181 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.132+0000 7f010d339700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010810aeb0 con 0x7f01080fadc0 2024-02-23T00:33:42.181 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.132+0000 7f0106ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01080fadc0 0x7f010810a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:42.181 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.132+0000 7f0106ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01080fadc0 0x7f010810a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39522/0 (socket says 172.21.15.47:39522) 2024-02-23T00:33:42.181 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.132+0000 7f0106ffd700 1 -- 172.21.15.47:0/3697354678 learned_addr learned my addr 172.21.15.47:0/3697354678 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:42.182 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.132+0000 7f0106ffd700 1 -- 172.21.15.47:0/3697354678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0100009370 con 0x7f01080fadc0 2024-02-23T00:33:42.182 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.132+0000 7f0106ffd700 1 --2- 172.21.15.47:0/3697354678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01080fadc0 0x7f010810a9c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f0100006940 tx=0x7f01000090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:42.182 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.132+0000 7f00fffff700 10 client.?.objecter ms_handle_connect 0x7f01080fadc0 2024-02-23T00:33:42.182 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.132+0000 7f00fffff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:33:42.182 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.132+0000 7f00fffff700 1 -- 172.21.15.47:0/3697354678 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f010001a640 con 0x7f01080fadc0 2024-02-23T00:33:42.182 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.132+0000 7f00fffff700 1 -- 172.21.15.47:0/3697354678 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f010001ac30 con 0x7f01080fadc0 2024-02-23T00:33:42.182 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.132+0000 7f010d339700 1 -- 172.21.15.47:0/3697354678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f010810b0b0 con 0x7f01080fadc0 2024-02-23T00:33:42.182 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.132+0000 7f00fffff700 1 -- 172.21.15.47:0/3697354678 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0100023a80 con 0x7f01080fadc0 2024-02-23T00:33:42.182 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.132+0000 7f010d339700 10 client.14140.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:33:42.182 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.132+0000 7f010d339700 1 -- 172.21.15.47:0/3697354678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010810b540 con 0x7f01080fadc0 2024-02-23T00:33:42.182 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.132+0000 7f010d339700 1 librados: init done 2024-02-23T00:33:42.182 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.133+0000 7f00fffff700 1 -- 172.21.15.47:0/3697354678 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) v1 ==== 38617+0+0 (secure 0 0 0) 0x7f0100023be0 con 0x7f01080fadc0 2024-02-23T00:33:42.182 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.133+0000 7f010d339700 1 -- 172.21.15.47:0/3697354678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0108046ce0 con 0x7f01080fadc0 2024-02-23T00:33:42.182 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.133+0000 7f00fffff700 1 --2- 172.21.15.47:0/3697354678 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f00f4033fb0 0x7f00f4036470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:42.183 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.133+0000 7f00fffff700 1 -- 172.21.15.47:0/3697354678 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7f0100044c10 con 0x7f01080fadc0 2024-02-23T00:33:42.183 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.133+0000 7f00fffff700 10 client.14140.objecter ms_dispatch 0x7f0108001550 osd_map(2..2 src has 1..2) v4 2024-02-23T00:33:42.183 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.133+0000 7f00fffff700 3 client.14140.objecter handle_osd_map got epochs [2,2] > 0 2024-02-23T00:33:42.183 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.133+0000 7f00fffff700 3 client.14140.objecter handle_osd_map decoding full epoch 2 2024-02-23T00:33:42.183 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.133+0000 7f00fffff700 20 client.14140.objecter dump_active .. 0 homeless 2024-02-23T00:33:42.183 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.133+0000 7f01067fc700 1 --2- 172.21.15.47:0/3697354678 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f00f4033fb0 0x7f00f4036470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:42.183 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.133+0000 7f01067fc700 1 --2- 172.21.15.47:0/3697354678 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f00f4033fb0 0x7f00f4036470 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f00f0009780 tx=0x7f00f0006cc0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:42.183 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.133+0000 7f00fffff700 10 client.14140.objecter ms_handle_connect 0x7f00f4033fb0 2024-02-23T00:33:42.183 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.138+0000 7f00fffff700 1 -- 172.21.15.47:0/3697354678 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f0100021320 con 0x7f01080fadc0 2024-02-23T00:33:42.327 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.325+0000 7f010d339700 1 -- 172.21.15.47:0/3697354678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f01080fcb00 con 0x7f01080fadc0 2024-02-23T00:33:42.329 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.328+0000 7f00fffff700 1 -- 172.21.15.47:0/3697354678 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f010001e020 con 0x7f01080fadc0 2024-02-23T00:33:42.332 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.330+0000 7f010d339700 10 librados: watch_flush enter 2024-02-23T00:33:42.332 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.330+0000 7f010d339700 10 librados: watch_flush exit 2024-02-23T00:33:42.332 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.330+0000 7f010d339700 20 client.14140.objecter shutdown clearing up homeless session... 2024-02-23T00:33:42.334 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.330+0000 7f010d339700 10 client.14140.objecter successfully canceled tick 2024-02-23T00:33:42.334 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.330+0000 7f010d339700 1 -- 172.21.15.47:0/3697354678 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f00f4033fb0 msgr2=0x7f00f4036470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:42.334 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.330+0000 7f010d339700 1 --2- 172.21.15.47:0/3697354678 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f00f4033fb0 0x7f00f4036470 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f00f0009780 tx=0x7f00f0006cc0).stop 2024-02-23T00:33:42.334 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.330+0000 7f010d339700 1 -- 172.21.15.47:0/3697354678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01080fadc0 msgr2=0x7f010810a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:42.334 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.330+0000 7f010d339700 1 --2- 172.21.15.47:0/3697354678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01080fadc0 0x7f010810a9c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f0100006940 tx=0x7f01000090f0).stop 2024-02-23T00:33:42.334 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.330+0000 7f010d339700 1 -- 172.21.15.47:0/3697354678 shutdown_connections 2024-02-23T00:33:42.334 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.330+0000 7f010d339700 1 --2- 172.21.15.47:0/3697354678 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f00f4033fb0 0x7f00f4036470 secure :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0x7f00f0009780 tx=0x7f00f0006cc0).stop 2024-02-23T00:33:42.334 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.330+0000 7f010d339700 1 --2- 172.21.15.47:0/3697354678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01080fadc0 0x7f010810a9c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:42.334 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.330+0000 7f010d339700 1 -- 172.21.15.47:0/3697354678 >> 172.21.15.47:0/3697354678 conn(0x7f01080f6540 msgr2=0x7f0108070ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:42.334 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.331+0000 7f010d339700 1 -- 172.21.15.47:0/3697354678 shutdown_connections 2024-02-23T00:33:42.334 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.331+0000 7f010d339700 1 -- 172.21.15.47:0/3697354678 wait complete. 2024-02-23T00:33:42.334 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:42.331+0000 7f010d339700 1 librados: shutdown 2024-02-23T00:33:43.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:41.654117+0000 mon.a (mon.0) 57 : audit [INF] from='mgr.14116 172.21.15.47:0/1137377827' entity='mgr.y' 2024-02-23T00:33:43.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:41.902686+0000 mon.a (mon.0) 58 : audit [INF] from='mgr.14116 172.21.15.47:0/1137377827' entity='mgr.y' 2024-02-23T00:33:43.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:42.329116+0000 mon.a (mon.0) 59 : audit [INF] from='client.? 172.21.15.47:0/3697354678' entity='client.admin' 2024-02-23T00:33:44.368 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.364+0000 7f84953e4700 1 Processor -- start 2024-02-23T00:33:44.369 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.364+0000 7f84953e4700 1 -- start start 2024-02-23T00:33:44.369 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.365+0000 7f84953e4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84900fadb0 0x7f84900fb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:44.369 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.365+0000 7f84953e4700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84900fb6c0 con 0x7f84900fadb0 2024-02-23T00:33:44.369 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.365+0000 7f848effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84900fadb0 0x7f84900fb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:44.369 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.365+0000 7f848effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84900fadb0 0x7f84900fb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39526/0 (socket says 172.21.15.47:39526) 2024-02-23T00:33:44.369 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.365+0000 7f848effd700 1 -- 172.21.15.47:0/251586524 learned_addr learned my addr 172.21.15.47:0/251586524 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:44.369 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.365+0000 7f848effd700 1 -- 172.21.15.47:0/251586524 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84900fb800 con 0x7f84900fadb0 2024-02-23T00:33:44.369 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.365+0000 7f848effd700 1 --2- 172.21.15.47:0/251586524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84900fadb0 0x7f84900fb1d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f8488009e50 tx=0x7f8488009840).ready entity=mon.0 client_cookie=3039d11c06b87adc server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:44.369 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.365+0000 7f848dffb700 1 -- 172.21.15.47:0/251586524 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f848800e070 con 0x7f84900fadb0 2024-02-23T00:33:44.369 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.365+0000 7f848dffb700 1 -- 172.21.15.47:0/251586524 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f8488013420 con 0x7f84900fadb0 2024-02-23T00:33:44.369 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.365+0000 7f848dffb700 1 -- 172.21.15.47:0/251586524 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8488007400 con 0x7f84900fadb0 2024-02-23T00:33:44.369 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.366+0000 7f84953e4700 1 -- 172.21.15.47:0/251586524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84900fadb0 msgr2=0x7f84900fb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:44.370 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.366+0000 7f84953e4700 1 --2- 172.21.15.47:0/251586524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84900fadb0 0x7f84900fb1d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f8488009e50 tx=0x7f8488009840).stop 2024-02-23T00:33:44.370 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.366+0000 7f84953e4700 1 -- 172.21.15.47:0/251586524 shutdown_connections 2024-02-23T00:33:44.370 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.366+0000 7f84953e4700 1 --2- 172.21.15.47:0/251586524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84900fadb0 0x7f84900fb1d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:44.370 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.366+0000 7f84953e4700 1 -- 172.21.15.47:0/251586524 >> 172.21.15.47:0/251586524 conn(0x7f84900f6530 msgr2=0x7f84900f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:44.370 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.366+0000 7f84953e4700 1 -- 172.21.15.47:0/251586524 shutdown_connections 2024-02-23T00:33:44.370 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.366+0000 7f84953e4700 1 -- 172.21.15.47:0/251586524 wait complete. 2024-02-23T00:33:44.371 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.366+0000 7f84953e4700 1 librados: starting msgr at 2024-02-23T00:33:44.371 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.366+0000 7f84953e4700 1 librados: starting objecter 2024-02-23T00:33:44.371 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.367+0000 7f84953e4700 1 Processor -- start 2024-02-23T00:33:44.371 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.367+0000 7f84953e4700 1 -- start start 2024-02-23T00:33:44.371 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.367+0000 7f84953e4700 1 librados: setting wanted keys 2024-02-23T00:33:44.371 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.367+0000 7f84953e4700 1 librados: calling monclient init 2024-02-23T00:33:44.372 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.367+0000 7f84953e4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84900fadb0 0x7f849010cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:44.372 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.367+0000 7f84953e4700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849010d100 con 0x7f84900fadb0 2024-02-23T00:33:44.372 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.367+0000 7f848effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84900fadb0 0x7f849010cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:44.372 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.367+0000 7f848effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84900fadb0 0x7f849010cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39530/0 (socket says 172.21.15.47:39530) 2024-02-23T00:33:44.372 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.367+0000 7f848effd700 1 -- 172.21.15.47:0/4184315945 learned_addr learned my addr 172.21.15.47:0/4184315945 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:44.372 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.367+0000 7f848effd700 1 -- 172.21.15.47:0/4184315945 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8488009ae0 con 0x7f84900fadb0 2024-02-23T00:33:44.372 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.367+0000 7f848effd700 1 --2- 172.21.15.47:0/4184315945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84900fadb0 0x7f849010cc10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f848800bf80 tx=0x7f8488009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:44.372 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.367+0000 7f8487fff700 10 client.?.objecter ms_handle_connect 0x7f84900fadb0 2024-02-23T00:33:44.372 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.367+0000 7f8487fff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:33:44.373 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.368+0000 7f8487fff700 1 -- 172.21.15.47:0/4184315945 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f848800e040 con 0x7f84900fadb0 2024-02-23T00:33:44.373 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.368+0000 7f84953e4700 1 -- 172.21.15.47:0/4184315945 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f849010d300 con 0x7f84900fadb0 2024-02-23T00:33:44.373 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.368+0000 7f8487fff700 1 -- 172.21.15.47:0/4184315945 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f8488007890 con 0x7f84900fadb0 2024-02-23T00:33:44.373 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.368+0000 7f8487fff700 1 -- 172.21.15.47:0/4184315945 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f848801c6b0 con 0x7f84900fadb0 2024-02-23T00:33:44.373 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.368+0000 7f84953e4700 10 client.14142.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:33:44.373 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.368+0000 7f84953e4700 1 -- 172.21.15.47:0/4184315945 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f849010d790 con 0x7f84900fadb0 2024-02-23T00:33:44.373 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.368+0000 7f84953e4700 1 librados: init done 2024-02-23T00:33:44.373 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.369+0000 7f8487fff700 1 -- 172.21.15.47:0/4184315945 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) v1 ==== 38617+0+0 (secure 0 0 0) 0x7f8488003040 con 0x7f84900fadb0 2024-02-23T00:33:44.373 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.369+0000 7f84953e4700 1 -- 172.21.15.47:0/4184315945 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8490046ce0 con 0x7f84900fadb0 2024-02-23T00:33:44.374 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.369+0000 7f8487fff700 1 --2- 172.21.15.47:0/4184315945 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f847c033f60 0x7f847c036420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:44.374 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.369+0000 7f8487fff700 1 -- 172.21.15.47:0/4184315945 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7f8488044b20 con 0x7f84900fadb0 2024-02-23T00:33:44.374 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.369+0000 7f8487fff700 10 client.14142.objecter ms_dispatch 0x7f8490001550 osd_map(2..2 src has 1..2) v4 2024-02-23T00:33:44.374 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.369+0000 7f8487fff700 3 client.14142.objecter handle_osd_map got epochs [2,2] > 0 2024-02-23T00:33:44.374 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.369+0000 7f8487fff700 3 client.14142.objecter handle_osd_map decoding full epoch 2 2024-02-23T00:33:44.374 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.369+0000 7f8487fff700 20 client.14142.objecter dump_active .. 0 homeless 2024-02-23T00:33:44.374 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.369+0000 7f848e7fc700 1 --2- 172.21.15.47:0/4184315945 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f847c033f60 0x7f847c036420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:44.374 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.369+0000 7f848e7fc700 1 --2- 172.21.15.47:0/4184315945 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f847c033f60 0x7f847c036420 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f8478009780 tx=0x7f8478006cc0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:44.374 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.369+0000 7f8487fff700 10 client.14142.objecter ms_handle_connect 0x7f847c033f60 2024-02-23T00:33:44.375 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.374+0000 7f8487fff700 1 -- 172.21.15.47:0/4184315945 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f8488025400 con 0x7f84900fadb0 2024-02-23T00:33:44.569 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.567+0000 7f84953e4700 1 -- 172.21.15.47:0/4184315945 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f84900fcaf0 con 0x7f84900fadb0 2024-02-23T00:33:44.572 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.570+0000 7f8487fff700 1 -- 172.21.15.47:0/4184315945 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f8488024b50 con 0x7f84900fadb0 2024-02-23T00:33:44.575 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.572+0000 7f84953e4700 10 librados: watch_flush enter 2024-02-23T00:33:44.575 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.572+0000 7f84953e4700 10 librados: watch_flush exit 2024-02-23T00:33:44.576 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.572+0000 7f84953e4700 20 client.14142.objecter shutdown clearing up homeless session... 2024-02-23T00:33:44.576 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.572+0000 7f84953e4700 10 client.14142.objecter successfully canceled tick 2024-02-23T00:33:44.576 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.572+0000 7f84953e4700 1 -- 172.21.15.47:0/4184315945 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f847c033f60 msgr2=0x7f847c036420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:44.576 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.572+0000 7f84953e4700 1 --2- 172.21.15.47:0/4184315945 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f847c033f60 0x7f847c036420 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f8478009780 tx=0x7f8478006cc0).stop 2024-02-23T00:33:44.576 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.572+0000 7f84953e4700 1 -- 172.21.15.47:0/4184315945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84900fadb0 msgr2=0x7f849010cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:44.576 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.572+0000 7f84953e4700 1 --2- 172.21.15.47:0/4184315945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84900fadb0 0x7f849010cc10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f848800bf80 tx=0x7f8488009840).stop 2024-02-23T00:33:44.576 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.572+0000 7f84953e4700 1 -- 172.21.15.47:0/4184315945 shutdown_connections 2024-02-23T00:33:44.576 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.572+0000 7f84953e4700 1 --2- 172.21.15.47:0/4184315945 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f847c033f60 0x7f847c036420 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:44.576 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.572+0000 7f84953e4700 1 --2- 172.21.15.47:0/4184315945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84900fadb0 0x7f849010cc10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:44.576 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.572+0000 7f84953e4700 1 -- 172.21.15.47:0/4184315945 >> 172.21.15.47:0/4184315945 conn(0x7f84900f6530 msgr2=0x7f84900ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:44.576 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.573+0000 7f84953e4700 1 -- 172.21.15.47:0/4184315945 shutdown_connections 2024-02-23T00:33:44.576 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.573+0000 7f84953e4700 1 -- 172.21.15.47:0/4184315945 wait complete. 2024-02-23T00:33:44.576 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:44.573+0000 7f84953e4700 1 librados: shutdown 2024-02-23T00:33:45.261 INFO:teuthology.orchestra.run.smithi047.stderr:Enabling the dashboard module... 2024-02-23T00:33:45.264 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:44.085652+0000 mon.a (mon.0) 60 : audit [INF] from='mgr.14116 172.21.15.47:0/1137377827' entity='mgr.y' 2024-02-23T00:33:45.264 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:44.571675+0000 mon.a (mon.0) 61 : audit [INF] from='client.? 172.21.15.47:0/4184315945' entity='client.admin' 2024-02-23T00:33:46.638 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.205+0000 7f1f863bb700 1 Processor -- start 2024-02-23T00:33:46.638 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.205+0000 7f1f863bb700 1 -- start start 2024-02-23T00:33:46.638 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.205+0000 7f1f863bb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f800fadb0 0x7f1f800fb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:46.638 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.205+0000 7f1f863bb700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f800fb6c0 con 0x7f1f800fadb0 2024-02-23T00:33:46.638 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.205+0000 7f1f7ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f800fadb0 0x7f1f800fb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:46.638 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.205+0000 7f1f7ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f800fadb0 0x7f1f800fb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39532/0 (socket says 172.21.15.47:39532) 2024-02-23T00:33:46.638 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.205+0000 7f1f7ffff700 1 -- 172.21.15.47:0/4001559483 learned_addr learned my addr 172.21.15.47:0/4001559483 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:46.638 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.206+0000 7f1f7ffff700 1 -- 172.21.15.47:0/4001559483 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f800fb800 con 0x7f1f800fadb0 2024-02-23T00:33:46.638 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.206+0000 7f1f7ffff700 1 --2- 172.21.15.47:0/4001559483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f800fadb0 0x7f1f800fb1d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f1f70009730 tx=0x7f1f700090f0).ready entity=mon.0 client_cookie=ce90037c676d1bdc server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:46.638 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.206+0000 7f1f7effd700 1 -- 172.21.15.47:0/4001559483 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f7000b580 con 0x7f1f800fadb0 2024-02-23T00:33:46.638 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.206+0000 7f1f7effd700 1 -- 172.21.15.47:0/4001559483 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f1f7000bb70 con 0x7f1f800fadb0 2024-02-23T00:33:46.639 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.206+0000 7f1f7effd700 1 -- 172.21.15.47:0/4001559483 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f70011a70 con 0x7f1f800fadb0 2024-02-23T00:33:46.639 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.207+0000 7f1f863bb700 1 -- 172.21.15.47:0/4001559483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f800fadb0 msgr2=0x7f1f800fb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:46.639 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.207+0000 7f1f863bb700 1 --2- 172.21.15.47:0/4001559483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f800fadb0 0x7f1f800fb1d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f1f70009730 tx=0x7f1f700090f0).stop 2024-02-23T00:33:46.639 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.207+0000 7f1f863bb700 1 -- 172.21.15.47:0/4001559483 shutdown_connections 2024-02-23T00:33:46.639 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.207+0000 7f1f863bb700 1 --2- 172.21.15.47:0/4001559483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f800fadb0 0x7f1f800fb1d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:46.639 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.207+0000 7f1f863bb700 1 -- 172.21.15.47:0/4001559483 >> 172.21.15.47:0/4001559483 conn(0x7f1f800f6530 msgr2=0x7f1f800f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:46.639 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.207+0000 7f1f863bb700 1 -- 172.21.15.47:0/4001559483 shutdown_connections 2024-02-23T00:33:46.639 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.207+0000 7f1f863bb700 1 -- 172.21.15.47:0/4001559483 wait complete. 2024-02-23T00:33:46.639 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.207+0000 7f1f863bb700 1 librados: starting msgr at 2024-02-23T00:33:46.639 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.207+0000 7f1f863bb700 1 librados: starting objecter 2024-02-23T00:33:46.639 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.207+0000 7f1f863bb700 1 Processor -- start 2024-02-23T00:33:46.639 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.208+0000 7f1f863bb700 1 -- start start 2024-02-23T00:33:46.639 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.208+0000 7f1f863bb700 1 librados: setting wanted keys 2024-02-23T00:33:46.640 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.208+0000 7f1f863bb700 1 librados: calling monclient init 2024-02-23T00:33:46.640 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.208+0000 7f1f863bb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f800fadb0 0x7f1f80101020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:46.640 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.208+0000 7f1f863bb700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f800ff4a0 con 0x7f1f800fadb0 2024-02-23T00:33:46.640 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.208+0000 7f1f7ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f800fadb0 0x7f1f80101020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:46.640 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.208+0000 7f1f7ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f800fadb0 0x7f1f80101020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39542/0 (socket says 172.21.15.47:39542) 2024-02-23T00:33:46.640 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.208+0000 7f1f7ffff700 1 -- 172.21.15.47:0/3221389567 learned_addr learned my addr 172.21.15.47:0/3221389567 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:46.640 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.208+0000 7f1f7ffff700 1 -- 172.21.15.47:0/3221389567 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f70009370 con 0x7f1f800fadb0 2024-02-23T00:33:46.640 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.208+0000 7f1f7ffff700 1 --2- 172.21.15.47:0/3221389567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f800fadb0 0x7f1f80101020 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f1f70000c00 tx=0x7f1f700090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:46.640 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.208+0000 7f1f7d7fa700 10 client.?.objecter ms_handle_connect 0x7f1f800fadb0 2024-02-23T00:33:46.640 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.209+0000 7f1f7d7fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:33:46.640 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.209+0000 7f1f7d7fa700 1 -- 172.21.15.47:0/3221389567 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f70011a00 con 0x7f1f800fadb0 2024-02-23T00:33:46.640 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.209+0000 7f1f7d7fa700 1 -- 172.21.15.47:0/3221389567 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f1f7001a640 con 0x7f1f800fadb0 2024-02-23T00:33:46.640 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.209+0000 7f1f863bb700 1 -- 172.21.15.47:0/3221389567 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f800ff640 con 0x7f1f800fadb0 2024-02-23T00:33:46.640 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.209+0000 7f1f7d7fa700 1 -- 172.21.15.47:0/3221389567 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f700121d0 con 0x7f1f800fadb0 2024-02-23T00:33:46.640 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.209+0000 7f1f863bb700 10 client.14144.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:33:46.641 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.209+0000 7f1f863bb700 1 -- 172.21.15.47:0/3221389567 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f800ffad0 con 0x7f1f800fadb0 2024-02-23T00:33:46.641 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.209+0000 7f1f863bb700 1 librados: init done 2024-02-23T00:33:46.641 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.210+0000 7f1f863bb700 1 -- 172.21.15.47:0/3221389567 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f8005a310 con 0x7f1f800fadb0 2024-02-23T00:33:46.641 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.210+0000 7f1f7d7fa700 1 -- 172.21.15.47:0/3221389567 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) v1 ==== 38617+0+0 (secure 0 0 0) 0x7f1f7001a7b0 con 0x7f1f800fadb0 2024-02-23T00:33:46.641 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.210+0000 7f1f7d7fa700 1 --2- 172.21.15.47:0/3221389567 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f1f68033f60 0x7f1f68036420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:46.641 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.210+0000 7f1f7d7fa700 1 -- 172.21.15.47:0/3221389567 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7f1f70044b20 con 0x7f1f800fadb0 2024-02-23T00:33:46.641 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.210+0000 7f1f7d7fa700 10 client.14144.objecter ms_dispatch 0x7f1f80001550 osd_map(2..2 src has 1..2) v4 2024-02-23T00:33:46.641 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.210+0000 7f1f7d7fa700 3 client.14144.objecter handle_osd_map got epochs [2,2] > 0 2024-02-23T00:33:46.641 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.210+0000 7f1f7d7fa700 3 client.14144.objecter handle_osd_map decoding full epoch 2 2024-02-23T00:33:46.641 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.210+0000 7f1f7d7fa700 20 client.14144.objecter dump_active .. 0 homeless 2024-02-23T00:33:46.641 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.210+0000 7f1f7f7fe700 1 --2- 172.21.15.47:0/3221389567 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f1f68033f60 0x7f1f68036420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:46.641 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.211+0000 7f1f7d7fa700 10 client.14144.objecter ms_handle_connect 0x7f1f68033f60 2024-02-23T00:33:46.641 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.211+0000 7f1f7f7fe700 1 --2- 172.21.15.47:0/3221389567 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f1f68033f60 0x7f1f68036420 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f1f74009780 tx=0x7f1f74006cc0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:46.641 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.214+0000 7f1f7d7fa700 1 -- 172.21.15.47:0/3221389567 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f1f7002b350 con 0x7f1f800fadb0 2024-02-23T00:33:46.642 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.431+0000 7f1f863bb700 1 -- 172.21.15.47:0/3221389567 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f1f80046ce0 con 0x7f1f800fadb0 2024-02-23T00:33:46.642 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.484+0000 7f1f7d7fa700 1 -- 172.21.15.47:0/3221389567 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 9) v1 ==== 38630+0+0 (secure 0 0 0) 0x7f1f70012450 con 0x7f1f800fadb0 2024-02-23T00:33:46.642 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.485+0000 7f1f7d7fa700 1 -- 172.21.15.47:0/3221389567 <== mon.0 v2:172.21.15.47:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f1f70018710 con 0x7f1f800fadb0 2024-02-23T00:33:46.642 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.487+0000 7f1f863bb700 10 librados: watch_flush enter 2024-02-23T00:33:46.642 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.487+0000 7f1f863bb700 10 librados: watch_flush exit 2024-02-23T00:33:46.642 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.487+0000 7f1f863bb700 20 client.14144.objecter shutdown clearing up homeless session... 2024-02-23T00:33:46.642 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.487+0000 7f1f863bb700 10 client.14144.objecter successfully canceled tick 2024-02-23T00:33:46.642 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.487+0000 7f1f863bb700 1 -- 172.21.15.47:0/3221389567 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f1f68033f60 msgr2=0x7f1f68036420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:46.642 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.487+0000 7f1f863bb700 1 --2- 172.21.15.47:0/3221389567 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f1f68033f60 0x7f1f68036420 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f1f74009780 tx=0x7f1f74006cc0).stop 2024-02-23T00:33:46.642 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.487+0000 7f1f863bb700 1 -- 172.21.15.47:0/3221389567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f800fadb0 msgr2=0x7f1f80101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:46.642 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.487+0000 7f1f863bb700 1 --2- 172.21.15.47:0/3221389567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f800fadb0 0x7f1f80101020 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f1f70000c00 tx=0x7f1f700090f0).stop 2024-02-23T00:33:46.642 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.487+0000 7f1f863bb700 1 -- 172.21.15.47:0/3221389567 shutdown_connections 2024-02-23T00:33:46.642 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.487+0000 7f1f863bb700 1 --2- 172.21.15.47:0/3221389567 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f1f68033f60 0x7f1f68036420 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:46.642 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.487+0000 7f1f863bb700 1 --2- 172.21.15.47:0/3221389567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f800fadb0 0x7f1f80101020 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:46.643 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.487+0000 7f1f863bb700 1 -- 172.21.15.47:0/3221389567 >> 172.21.15.47:0/3221389567 conn(0x7f1f800f6530 msgr2=0x7f1f800707b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:46.643 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.487+0000 7f1f863bb700 1 -- 172.21.15.47:0/3221389567 shutdown_connections 2024-02-23T00:33:46.643 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.487+0000 7f1f863bb700 1 -- 172.21.15.47:0/3221389567 wait complete. 2024-02-23T00:33:46.643 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:46.487+0000 7f1f863bb700 1 librados: shutdown 2024-02-23T00:33:46.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:46.433195+0000 mon.a (mon.0) 62 : audit [INF] from='client.? 172.21.15.47:0/3221389567' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-23T00:33:46.806 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ignoring --setuser ceph since I am not root 2024-02-23T00:33:46.806 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ignoring --setgroup ceph since I am not root 2024-02-23T00:33:46.806 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:46.648+0000 7f8120c78680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-23T00:33:46.807 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:46.732+0000 7f8120c78680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-23T00:33:47.273 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:47.113+0000 7f8120c78680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-23T00:33:47.552 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:46.486455+0000 mon.a (mon.0) 63 : audit [INF] from='client.? 172.21.15.47:0/3221389567' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-23T00:33:47.552 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:33:46.486544+0000 mon.a (mon.0) 64 : cluster [DBG] mgrmap e9: y(active, since 18s) 2024-02-23T00:33:47.843 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.830+0000 7fa11d4a4700 1 Processor -- start 2024-02-23T00:33:47.843 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.830+0000 7fa11d4a4700 1 -- start start 2024-02-23T00:33:47.843 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.830+0000 7fa11d4a4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1180713a0 0x7fa1180feac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:47.843 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.830+0000 7fa11d4a4700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1180fefb0 con 0x7fa1180713a0 2024-02-23T00:33:47.843 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.830+0000 7fa116ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1180713a0 0x7fa1180feac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:47.843 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.830+0000 7fa116ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1180713a0 0x7fa1180feac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37392/0 (socket says 172.21.15.47:37392) 2024-02-23T00:33:47.844 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.830+0000 7fa116ffd700 1 -- 172.21.15.47:0/3207131102 learned_addr learned my addr 172.21.15.47:0/3207131102 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:47.844 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.830+0000 7fa116ffd700 1 -- 172.21.15.47:0/3207131102 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1180ff0f0 con 0x7fa1180713a0 2024-02-23T00:33:47.844 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.831+0000 7fa116ffd700 1 --2- 172.21.15.47:0/3207131102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1180713a0 0x7fa1180feac0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fa108009730 tx=0x7fa1080090f0).ready entity=mon.0 client_cookie=10d4e59a4f08c114 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:47.844 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.831+0000 7fa115ffb700 1 -- 172.21.15.47:0/3207131102 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa10800b610 con 0x7fa1180713a0 2024-02-23T00:33:47.844 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.831+0000 7fa115ffb700 1 -- 172.21.15.47:0/3207131102 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fa10800bc00 con 0x7fa1180713a0 2024-02-23T00:33:47.844 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.831+0000 7fa115ffb700 1 -- 172.21.15.47:0/3207131102 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa108011b50 con 0x7fa1180713a0 2024-02-23T00:33:47.845 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.831+0000 7fa11d4a4700 1 -- 172.21.15.47:0/3207131102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1180713a0 msgr2=0x7fa1180feac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:47.845 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.831+0000 7fa11d4a4700 1 --2- 172.21.15.47:0/3207131102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1180713a0 0x7fa1180feac0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fa108009730 tx=0x7fa1080090f0).stop 2024-02-23T00:33:47.845 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.831+0000 7fa11d4a4700 1 -- 172.21.15.47:0/3207131102 shutdown_connections 2024-02-23T00:33:47.845 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.831+0000 7fa11d4a4700 1 --2- 172.21.15.47:0/3207131102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1180713a0 0x7fa1180feac0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:47.845 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.831+0000 7fa11d4a4700 1 -- 172.21.15.47:0/3207131102 >> 172.21.15.47:0/3207131102 conn(0x7fa1180f6540 msgr2=0x7fa1180f8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:47.845 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.832+0000 7fa11d4a4700 1 -- 172.21.15.47:0/3207131102 shutdown_connections 2024-02-23T00:33:47.846 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.832+0000 7fa11d4a4700 1 -- 172.21.15.47:0/3207131102 wait complete. 2024-02-23T00:33:47.846 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.832+0000 7fa11d4a4700 1 librados: starting msgr at 2024-02-23T00:33:47.846 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.832+0000 7fa11d4a4700 1 librados: starting objecter 2024-02-23T00:33:47.846 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.832+0000 7fa11d4a4700 1 Processor -- start 2024-02-23T00:33:47.846 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.832+0000 7fa11d4a4700 1 -- start start 2024-02-23T00:33:47.846 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.832+0000 7fa11d4a4700 1 librados: setting wanted keys 2024-02-23T00:33:47.846 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.832+0000 7fa11d4a4700 1 librados: calling monclient init 2024-02-23T00:33:47.846 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.832+0000 7fa11d4a4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1180713a0 0x7fa11810aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:47.846 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.832+0000 7fa11d4a4700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11810af80 con 0x7fa1180713a0 2024-02-23T00:33:47.846 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.833+0000 7fa116ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1180713a0 0x7fa11810aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:47.846 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.833+0000 7fa116ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1180713a0 0x7fa11810aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37404/0 (socket says 172.21.15.47:37404) 2024-02-23T00:33:47.847 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.833+0000 7fa116ffd700 1 -- 172.21.15.47:0/2870677610 learned_addr learned my addr 172.21.15.47:0/2870677610 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:47.847 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.833+0000 7fa116ffd700 1 -- 172.21.15.47:0/2870677610 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa108009370 con 0x7fa1180713a0 2024-02-23T00:33:47.847 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.833+0000 7fa116ffd700 1 --2- 172.21.15.47:0/2870677610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1180713a0 0x7fa11810aa90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fa108000c00 tx=0x7fa1080090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:47.847 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.833+0000 7fa0fffff700 10 client.?.objecter ms_handle_connect 0x7fa1180713a0 2024-02-23T00:33:47.847 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.833+0000 7fa0fffff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:33:47.847 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.833+0000 7fa0fffff700 1 -- 172.21.15.47:0/2870677610 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa108011990 con 0x7fa1180713a0 2024-02-23T00:33:47.847 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.833+0000 7fa11d4a4700 1 -- 172.21.15.47:0/2870677610 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11810b180 con 0x7fa1180713a0 2024-02-23T00:33:47.847 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.833+0000 7fa0fffff700 1 -- 172.21.15.47:0/2870677610 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fa10801aad0 con 0x7fa1180713a0 2024-02-23T00:33:47.847 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.833+0000 7fa0fffff700 1 -- 172.21.15.47:0/2870677610 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1080239d0 con 0x7fa1180713a0 2024-02-23T00:33:47.847 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.833+0000 7fa11d4a4700 10 client.14148.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:33:47.847 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.833+0000 7fa11d4a4700 1 -- 172.21.15.47:0/2870677610 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11810b610 con 0x7fa1180713a0 2024-02-23T00:33:47.847 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.833+0000 7fa11d4a4700 1 librados: init done 2024-02-23T00:33:47.847 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.834+0000 7fa11d4a4700 1 -- 172.21.15.47:0/2870677610 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa118046ce0 con 0x7fa1180713a0 2024-02-23T00:33:47.848 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.834+0000 7fa0fffff700 1 -- 172.21.15.47:0/2870677610 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 9) v1 ==== 38630+0+0 (secure 0 0 0) 0x7fa108023c90 con 0x7fa1180713a0 2024-02-23T00:33:47.848 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.834+0000 7fa0fffff700 1 --2- 172.21.15.47:0/2870677610 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7fa100033fb0 0x7fa100036470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:47.848 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.835+0000 7fa0fffff700 1 -- 172.21.15.47:0/2870677610 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7fa1080451d0 con 0x7fa1180713a0 2024-02-23T00:33:47.848 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.835+0000 7fa0fffff700 10 client.14148.objecter ms_dispatch 0x7fa118001550 osd_map(2..2 src has 1..2) v4 2024-02-23T00:33:47.848 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.835+0000 7fa1167fc700 1 -- 172.21.15.47:0/2870677610 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7fa100033fb0 msgr2=0x7fa100036470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/1916598887 2024-02-23T00:33:47.848 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.835+0000 7fa0fffff700 3 client.14148.objecter handle_osd_map got epochs [2,2] > 0 2024-02-23T00:33:47.848 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.835+0000 7fa0fffff700 3 client.14148.objecter handle_osd_map decoding full epoch 2 2024-02-23T00:33:47.848 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.835+0000 7fa1167fc700 1 --2- 172.21.15.47:0/2870677610 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7fa100033fb0 0x7fa100036470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-02-23T00:33:47.848 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.835+0000 7fa0fffff700 20 client.14148.objecter dump_active .. 0 homeless 2024-02-23T00:33:47.849 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:47.842+0000 7fa0fffff700 1 -- 172.21.15.47:0/2870677610 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fa10802bb40 con 0x7fa1180713a0 2024-02-23T00:33:48.038 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:48.036+0000 7fa1167fc700 1 -- 172.21.15.47:0/2870677610 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7fa100033fb0 msgr2=0x7fa100036470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/1916598887 2024-02-23T00:33:48.038 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:48.036+0000 7fa1167fc700 1 --2- 172.21.15.47:0/2870677610 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7fa100033fb0 0x7fa100036470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-02-23T00:33:48.105 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:48.102+0000 7fa11d4a4700 1 -- 172.21.15.47:0/2870677610 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fa1181045f0 con 0x7fa1180713a0 2024-02-23T00:33:48.105 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:48.103+0000 7fa0fffff700 1 -- 172.21.15.47:0/2870677610 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fa10801e070 con 0x7fa1180713a0 2024-02-23T00:33:48.106 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: { 2024-02-23T00:33:48.106 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "epoch": 9, 2024-02-23T00:33:48.106 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "available": true, 2024-02-23T00:33:48.106 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "active_name": "y", 2024-02-23T00:33:48.106 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "num_standby": 0 2024-02-23T00:33:48.106 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: } 2024-02-23T00:33:48.108 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:48.105+0000 7fa11d4a4700 10 librados: watch_flush enter 2024-02-23T00:33:48.108 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:48.105+0000 7fa11d4a4700 10 librados: watch_flush exit 2024-02-23T00:33:48.108 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:48.105+0000 7fa11d4a4700 20 client.14148.objecter shutdown clearing up homeless session... 2024-02-23T00:33:48.108 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:48.105+0000 7fa11d4a4700 10 client.14148.objecter successfully canceled tick 2024-02-23T00:33:48.108 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:48.105+0000 7fa11d4a4700 1 -- 172.21.15.47:0/2870677610 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7fa100033fb0 msgr2=0x7fa100036470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-23T00:33:48.109 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:48.105+0000 7fa11d4a4700 1 --2- 172.21.15.47:0/2870677610 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7fa100033fb0 0x7fa100036470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:33:48.109 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:48.105+0000 7fa11d4a4700 1 -- 172.21.15.47:0/2870677610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1180713a0 msgr2=0x7fa11810aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:48.109 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:48.105+0000 7fa11d4a4700 1 --2- 172.21.15.47:0/2870677610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1180713a0 0x7fa11810aa90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fa108000c00 tx=0x7fa1080090f0).stop 2024-02-23T00:33:48.109 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:48.105+0000 7fa11d4a4700 1 -- 172.21.15.47:0/2870677610 shutdown_connections 2024-02-23T00:33:48.109 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:48.105+0000 7fa11d4a4700 1 --2- 172.21.15.47:0/2870677610 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7fa100033fb0 0x7fa100036470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:33:48.109 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:48.105+0000 7fa11d4a4700 1 --2- 172.21.15.47:0/2870677610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1180713a0 0x7fa11810aa90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:48.109 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:48.105+0000 7fa11d4a4700 1 -- 172.21.15.47:0/2870677610 >> 172.21.15.47:0/2870677610 conn(0x7fa1180f6540 msgr2=0x7fa1180f7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:48.109 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:48.105+0000 7fa11d4a4700 1 -- 172.21.15.47:0/2870677610 shutdown_connections 2024-02-23T00:33:48.109 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:48.105+0000 7fa11d4a4700 1 -- 172.21.15.47:0/2870677610 wait complete. 2024-02-23T00:33:48.109 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:48.105+0000 7fa11d4a4700 1 librados: shutdown 2024-02-23T00:33:48.229 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:47.922+0000 7f8120c78680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-23T00:33:48.229 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:48.004+0000 7f8120c78680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-23T00:33:48.488 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:48.227+0000 7f8120c78680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-23T00:33:48.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:48.104276+0000 mon.a (mon.0) 65 : audit [DBG] from='client.? 172.21.15.47:0/2870677610' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-23T00:33:48.827 INFO:teuthology.orchestra.run.smithi047.stderr:Waiting for the mgr to restart... 2024-02-23T00:33:48.827 INFO:teuthology.orchestra.run.smithi047.stderr:Waiting for mgr epoch 9... 2024-02-23T00:33:49.352 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:49.216+0000 7f8120c78680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-23T00:33:49.475 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.472+0000 7f6cc7c64700 1 Processor -- start 2024-02-23T00:33:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.472+0000 7f6cc7c64700 1 -- start start 2024-02-23T00:33:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.472+0000 7f6cc7c64700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cc0071380 0x7f6cc00feae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.473+0000 7f6cc7c64700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc00fefd0 con 0x7f6cc0071380 2024-02-23T00:33:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.473+0000 7f6cc5a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cc0071380 0x7f6cc00feae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.473+0000 7f6cc5a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cc0071380 0x7f6cc00feae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37420/0 (socket says 172.21.15.47:37420) 2024-02-23T00:33:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.473+0000 7f6cc5a00700 1 -- 172.21.15.47:0/1726168538 learned_addr learned my addr 172.21.15.47:0/1726168538 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.473+0000 7f6cc5a00700 1 -- 172.21.15.47:0/1726168538 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc00ff110 con 0x7f6cc0071380 2024-02-23T00:33:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.473+0000 7f6cc5a00700 1 --2- 172.21.15.47:0/1726168538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cc0071380 0x7f6cc00feae0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f6cbc009730 tx=0x7f6cbc0090f0).ready entity=mon.0 client_cookie=25d3c5024d07a2ad server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.473+0000 7f6cc49fe700 1 -- 172.21.15.47:0/1726168538 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6cbc00b580 con 0x7f6cc0071380 2024-02-23T00:33:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.473+0000 7f6cc49fe700 1 -- 172.21.15.47:0/1726168538 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f6cbc00bb70 con 0x7f6cc0071380 2024-02-23T00:33:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.473+0000 7f6cc49fe700 1 -- 172.21.15.47:0/1726168538 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6cbc011a70 con 0x7f6cc0071380 2024-02-23T00:33:49.478 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.474+0000 7f6cc7c64700 1 -- 172.21.15.47:0/1726168538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cc0071380 msgr2=0x7f6cc00feae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:49.478 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.474+0000 7f6cc7c64700 1 --2- 172.21.15.47:0/1726168538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cc0071380 0x7f6cc00feae0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f6cbc009730 tx=0x7f6cbc0090f0).stop 2024-02-23T00:33:49.478 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.474+0000 7f6cc7c64700 1 -- 172.21.15.47:0/1726168538 shutdown_connections 2024-02-23T00:33:49.478 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.474+0000 7f6cc7c64700 1 --2- 172.21.15.47:0/1726168538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cc0071380 0x7f6cc00feae0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:49.478 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.474+0000 7f6cc7c64700 1 -- 172.21.15.47:0/1726168538 >> 172.21.15.47:0/1726168538 conn(0x7f6cc00f6540 msgr2=0x7f6cc00f8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:49.479 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.474+0000 7f6cc7c64700 1 -- 172.21.15.47:0/1726168538 shutdown_connections 2024-02-23T00:33:49.479 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.475+0000 7f6cc7c64700 1 -- 172.21.15.47:0/1726168538 wait complete. 2024-02-23T00:33:49.479 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.475+0000 7f6cc7c64700 1 librados: starting msgr at 2024-02-23T00:33:49.479 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.475+0000 7f6cc7c64700 1 librados: starting objecter 2024-02-23T00:33:49.479 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.475+0000 7f6cc7c64700 1 Processor -- start 2024-02-23T00:33:49.479 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.475+0000 7f6cc7c64700 1 -- start start 2024-02-23T00:33:49.479 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.475+0000 7f6cc7c64700 1 librados: setting wanted keys 2024-02-23T00:33:49.479 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.475+0000 7f6cc7c64700 1 librados: calling monclient init 2024-02-23T00:33:49.479 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.476+0000 7f6cc7c64700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cc0071380 0x7f6cc010aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:49.479 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.476+0000 7f6cc7c64700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc010af30 con 0x7f6cc0071380 2024-02-23T00:33:49.479 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.476+0000 7f6cc5a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cc0071380 0x7f6cc010aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:49.479 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.476+0000 7f6cc5a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cc0071380 0x7f6cc010aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37430/0 (socket says 172.21.15.47:37430) 2024-02-23T00:33:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.476+0000 7f6cc5a00700 1 -- 172.21.15.47:0/1685318384 learned_addr learned my addr 172.21.15.47:0/1685318384 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.476+0000 7f6cc5a00700 1 -- 172.21.15.47:0/1685318384 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cbc009370 con 0x7f6cc0071380 2024-02-23T00:33:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.476+0000 7f6cc5a00700 1 --2- 172.21.15.47:0/1685318384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cc0071380 0x7f6cc010aa40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f6cbc000c00 tx=0x7f6cbc0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.476+0000 7f6cb6ffd700 10 client.?.objecter ms_handle_connect 0x7f6cc0071380 2024-02-23T00:33:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.476+0000 7f6cb6ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:33:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.476+0000 7f6cb6ffd700 1 -- 172.21.15.47:0/1685318384 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6cbc011400 con 0x7f6cc0071380 2024-02-23T00:33:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.476+0000 7f6cb6ffd700 1 -- 172.21.15.47:0/1685318384 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f6cbc01aad0 con 0x7f6cc0071380 2024-02-23T00:33:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.476+0000 7f6cc7c64700 1 -- 172.21.15.47:0/1685318384 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cc010b130 con 0x7f6cc0071380 2024-02-23T00:33:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.476+0000 7f6cb6ffd700 1 -- 172.21.15.47:0/1685318384 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6cbc0238a0 con 0x7f6cc0071380 2024-02-23T00:33:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.476+0000 7f6cc7c64700 10 client.14150.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:33:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.476+0000 7f6cc7c64700 1 -- 172.21.15.47:0/1685318384 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc010b5c0 con 0x7f6cc0071380 2024-02-23T00:33:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.477+0000 7f6cc7c64700 1 librados: init done 2024-02-23T00:33:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.477+0000 7f6cb6ffd700 1 -- 172.21.15.47:0/1685318384 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 9) v1 ==== 38630+0+0 (secure 0 0 0) 0x7f6cbc023b10 con 0x7f6cc0071380 2024-02-23T00:33:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.478+0000 7f6cb6ffd700 1 --2- 172.21.15.47:0/1685318384 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f6cac034000 0x7f6cac0364c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.478+0000 7f6cb6ffd700 1 -- 172.21.15.47:0/1685318384 --> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6cac036b80 con 0x7f6cac034000 2024-02-23T00:33:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.478+0000 7f6cb6ffd700 1 -- 172.21.15.47:0/1685318384 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7f6cbc044d80 con 0x7f6cc0071380 2024-02-23T00:33:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.478+0000 7f6cb6ffd700 10 client.14150.objecter ms_dispatch 0x7f6cc0001550 osd_map(2..2 src has 1..2) v4 2024-02-23T00:33:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.478+0000 7f6cb6ffd700 3 client.14150.objecter handle_osd_map got epochs [2,2] > 0 2024-02-23T00:33:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.478+0000 7f6cb6ffd700 3 client.14150.objecter handle_osd_map decoding full epoch 2 2024-02-23T00:33:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.478+0000 7f6cc51ff700 1 -- 172.21.15.47:0/1685318384 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f6cac034000 msgr2=0x7f6cac0364c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/1916598887 2024-02-23T00:33:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.478+0000 7f6cc51ff700 1 --2- 172.21.15.47:0/1685318384 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f6cac034000 0x7f6cac0364c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-02-23T00:33:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.478+0000 7f6cb6ffd700 20 client.14150.objecter dump_active .. 0 homeless 2024-02-23T00:33:49.681 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.679+0000 7f6cc51ff700 1 -- 172.21.15.47:0/1685318384 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f6cac034000 msgr2=0x7f6cac0364c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/1916598887 2024-02-23T00:33:49.681 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:49.679+0000 7f6cc51ff700 1 --2- 172.21.15.47:0/1685318384 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f6cac034000 0x7f6cac0364c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-02-23T00:33:49.734 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:49.350+0000 7f8120c78680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-23T00:33:49.735 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:49.435+0000 7f8120c78680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-23T00:33:49.735 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:49.531+0000 7f8120c78680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-23T00:33:50.027 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:49.732+0000 7f8120c78680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-23T00:33:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:50.080+0000 7f6cc51ff700 1 -- 172.21.15.47:0/1685318384 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f6cac034000 msgr2=0x7f6cac0364c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/1916598887 2024-02-23T00:33:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:50.080+0000 7f6cc51ff700 1 --2- 172.21.15.47:0/1685318384 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f6cac034000 0x7f6cac0364c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2024-02-23T00:33:50.306 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:50.025+0000 7f8120c78680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-23T00:33:50.306 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:50.127+0000 7f8120c78680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-23T00:33:50.883 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:50.881+0000 7f6cc51ff700 1 -- 172.21.15.47:0/1685318384 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f6cac034000 msgr2=0x7f6cac0364c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/1916598887 2024-02-23T00:33:50.883 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:50.881+0000 7f6cc51ff700 1 --2- 172.21.15.47:0/1685318384 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f6cac034000 0x7f6cac0364c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2024-02-23T00:33:51.240 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:50.879+0000 7f8120c78680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-23T00:33:51.241 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:50.964+0000 7f8120c78680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-23T00:33:51.241 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:51.059+0000 7f8120c78680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-23T00:33:51.785 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:51.238+0000 7f8120c78680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-23T00:33:51.785 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:51.505+0000 7f8120c78680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-23T00:33:51.785 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:51.644+0000 7f8120c78680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-23T00:33:52.049 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:51.783+0000 7f8120c78680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-23T00:33:52.050 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:51.962+0000 7f8120c78680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-23T00:33:52.065 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:52.062+0000 7f6cb6ffd700 1 -- 172.21.15.47:0/1685318384 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mgrmap(e 10) v1 ==== 38454+0+0 (secure 0 0 0) 0x7f6cbc01ac40 con 0x7f6cc0071380 2024-02-23T00:33:52.065 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:52.062+0000 7f6cb6ffd700 1 -- 172.21.15.47:0/1685318384 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f6cac034000 msgr2=0x7f6cac0364c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-23T00:33:52.065 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:52.062+0000 7f6cb6ffd700 1 --2- 172.21.15.47:0/1685318384 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f6cac034000 0x7f6cac0364c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:33:52.300 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:33:52.047+0000 7f8120c78680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-23T00:33:52.300 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:33:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [79B blob data] 2024-02-23T00:33:52.301 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:33:52.061897+0000 mon.a (mon.0) 66 : cluster [INF] Active manager daemon y restarted 2024-02-23T00:33:52.301 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:33:52.062113+0000 mon.a (mon.0) 67 : cluster [INF] Activating manager daemon y 2024-02-23T00:33:52.301 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:33:52.064828+0000 mon.a (mon.0) 68 : cluster [DBG] osdmap e3: 0 total, 0 up, 0 in 2024-02-23T00:33:52.301 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:33:52.064939+0000 mon.a (mon.0) 69 : cluster [DBG] mgrmap e10: y(active, starting, since 0.00291231s) 2024-02-23T00:33:52.301 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:52.066069+0000 mon.a (mon.0) 70 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-23T00:33:52.301 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:52.066674+0000 mon.a (mon.0) 71 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-23T00:33:52.301 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:52.067356+0000 mon.a (mon.0) 72 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-23T00:33:52.301 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:52.067519+0000 mon.a (mon.0) 73 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-23T00:33:52.301 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:52.067637+0000 mon.a (mon.0) 74 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-23T00:33:52.301 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:33:52.081711+0000 mon.a (mon.0) 75 : cluster [INF] Manager daemon y is now available 2024-02-23T00:33:53.065 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:53.063+0000 7f6cb6ffd700 1 -- 172.21.15.47:0/1685318384 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 11) v1 ==== 38524+0+0 (secure 0 0 0) 0x7f6cbc02c3f0 con 0x7f6cc0071380 2024-02-23T00:33:53.066 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:53.063+0000 7f6cb6ffd700 1 --2- 172.21.15.47:0/1685318384 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f6cac037c10 0x7f6cac03a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:53.066 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:53.063+0000 7f6cb6ffd700 1 -- 172.21.15.47:0/1685318384 --> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6cac036b80 con 0x7f6cac037c10 2024-02-23T00:33:53.066 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:53.064+0000 7f6cc51ff700 1 --2- 172.21.15.47:0/1685318384 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f6cac037c10 0x7f6cac03a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:53.067 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:53.065+0000 7f6cb6ffd700 10 client.14150.objecter ms_handle_connect 0x7f6cac037c10 2024-02-23T00:33:53.067 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:53.065+0000 7f6cc51ff700 1 --2- 172.21.15.47:0/1685318384 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f6cac037c10 0x7f6cac03a000 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7f6cb0003850 tx=0x7f6cb0009350).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:53.067 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:53.065+0000 7f6cb6ffd700 1 -- 172.21.15.47:0/1685318384 <== mgr.14146 v2:172.21.15.47:6800/622994871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6438 (secure 0 0 0) 0x7f6cac036b80 con 0x7f6cac037c10 2024-02-23T00:33:53.073 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:53.071+0000 7f6cc7c64700 1 -- 172.21.15.47:0/1685318384 --> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f6cc005a310 con 0x7f6cac037c10 2024-02-23T00:33:53.073 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:53.071+0000 7f6cb6ffd700 1 -- 172.21.15.47:0/1685318384 <== mgr.14146 v2:172.21.15.47:6800/622994871 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f6cc005a310 con 0x7f6cac037c10 2024-02-23T00:33:53.075 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: { 2024-02-23T00:33:53.075 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "mgrmap_epoch": 11, 2024-02-23T00:33:53.075 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: "initialized": true 2024-02-23T00:33:53.075 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: } 2024-02-23T00:33:53.075 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:53.072+0000 7f6cc7c64700 10 librados: watch_flush enter 2024-02-23T00:33:53.076 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:53.072+0000 7f6cc7c64700 10 librados: watch_flush exit 2024-02-23T00:33:53.076 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:53.072+0000 7f6cc7c64700 20 client.14150.objecter shutdown clearing up homeless session... 2024-02-23T00:33:53.076 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:53.072+0000 7f6cc7c64700 10 client.14150.objecter successfully canceled tick 2024-02-23T00:33:53.076 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:53.072+0000 7f6cc7c64700 1 -- 172.21.15.47:0/1685318384 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f6cac037c10 msgr2=0x7f6cac03a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:53.076 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:53.072+0000 7f6cc7c64700 1 --2- 172.21.15.47:0/1685318384 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f6cac037c10 0x7f6cac03a000 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7f6cb0003850 tx=0x7f6cb0009350).stop 2024-02-23T00:33:53.076 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:53.072+0000 7f6cc7c64700 1 -- 172.21.15.47:0/1685318384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cc0071380 msgr2=0x7f6cc010aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:53.076 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:53.072+0000 7f6cc7c64700 1 --2- 172.21.15.47:0/1685318384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cc0071380 0x7f6cc010aa40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f6cbc000c00 tx=0x7f6cbc0090f0).stop 2024-02-23T00:33:53.076 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:53.072+0000 7f6cc7c64700 1 -- 172.21.15.47:0/1685318384 shutdown_connections 2024-02-23T00:33:53.076 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:53.072+0000 7f6cc7c64700 1 --2- 172.21.15.47:0/1685318384 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f6cac037c10 0x7f6cac03a000 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:53.076 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:53.072+0000 7f6cc7c64700 1 --2- 172.21.15.47:0/1685318384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cc0071380 0x7f6cc010aa40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:53.076 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:53.072+0000 7f6cc7c64700 1 --2- 172.21.15.47:0/1685318384 >> [v2:172.21.15.47:6800/1916598887,v1:172.21.15.47:6801/1916598887] conn(0x7f6cac034000 0x7f6cac0364c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:33:53.076 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:53.072+0000 7f6cc7c64700 1 -- 172.21.15.47:0/1685318384 >> 172.21.15.47:0/1685318384 conn(0x7f6cc00f6540 msgr2=0x7f6cc00f7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:53.076 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:53.072+0000 7f6cc7c64700 1 -- 172.21.15.47:0/1685318384 shutdown_connections 2024-02-23T00:33:53.076 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:53.072+0000 7f6cc7c64700 1 -- 172.21.15.47:0/1685318384 wait complete. 2024-02-23T00:33:53.077 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:53.072+0000 7f6cc7c64700 1 librados: shutdown 2024-02-23T00:33:53.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:52.180308+0000 mon.a (mon.0) 76 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:33:53.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:52.182066+0000 mon.a (mon.0) 77 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:33:53.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:52.206125+0000 mon.a (mon.0) 78 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-23T00:33:53.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:52.209506+0000 mon.a (mon.0) 79 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-23T00:33:53.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:33:53.067039+0000 mon.a (mon.0) 80 : cluster [DBG] mgrmap e11: y(active, since 1.00501s) 2024-02-23T00:33:54.121 INFO:teuthology.orchestra.run.smithi047.stderr:mgr epoch 9 is available 2024-02-23T00:33:54.122 INFO:teuthology.orchestra.run.smithi047.stderr:Generating a dashboard self-signed certificate... 2024-02-23T00:33:54.373 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:53.066483+0000 mgr.y (mgr.14146) 1 : audit [DBG] from='client.14150 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-23T00:33:54.373 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:53.072870+0000 mgr.y (mgr.14146) 2 : audit [DBG] from='client.14150 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-23T00:33:54.373 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:53.810942+0000 mon.a (mon.0) 81 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:33:54.706 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.672+0000 7fc8f97e1700 1 Processor -- start 2024-02-23T00:33:54.706 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.672+0000 7fc8f97e1700 1 -- start start 2024-02-23T00:33:54.706 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.672+0000 7fc8f97e1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8f4071390 0x7fc8f40feb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:54.706 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.672+0000 7fc8f97e1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f40717b0 con 0x7fc8f4071390 2024-02-23T00:33:54.706 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.672+0000 7fc8f2ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8f4071390 0x7fc8f40feb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:54.707 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.672+0000 7fc8f2ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8f4071390 0x7fc8f40feb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37476/0 (socket says 172.21.15.47:37476) 2024-02-23T00:33:54.707 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.672+0000 7fc8f2ffd700 1 -- 172.21.15.47:0/3032608206 learned_addr learned my addr 172.21.15.47:0/3032608206 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:54.707 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.672+0000 7fc8f2ffd700 1 -- 172.21.15.47:0/3032608206 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f40ff060 con 0x7fc8f4071390 2024-02-23T00:33:54.707 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.673+0000 7fc8f2ffd700 1 --2- 172.21.15.47:0/3032608206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8f4071390 0x7fc8f40feb70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fc8e4009730 tx=0x7fc8e40090f0).ready entity=mon.0 client_cookie=dd3f04c033a95494 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:54.707 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.673+0000 7fc8f1ffb700 1 -- 172.21.15.47:0/3032608206 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8e400b580 con 0x7fc8f4071390 2024-02-23T00:33:54.711 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.673+0000 7fc8f1ffb700 1 -- 172.21.15.47:0/3032608206 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fc8e400bb70 con 0x7fc8f4071390 2024-02-23T00:33:54.711 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.673+0000 7fc8f1ffb700 1 -- 172.21.15.47:0/3032608206 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8e4011a70 con 0x7fc8f4071390 2024-02-23T00:33:54.711 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.673+0000 7fc8f97e1700 1 -- 172.21.15.47:0/3032608206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8f4071390 msgr2=0x7fc8f40feb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:54.711 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.673+0000 7fc8f97e1700 1 --2- 172.21.15.47:0/3032608206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8f4071390 0x7fc8f40feb70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fc8e4009730 tx=0x7fc8e40090f0).stop 2024-02-23T00:33:54.711 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.673+0000 7fc8f97e1700 1 -- 172.21.15.47:0/3032608206 shutdown_connections 2024-02-23T00:33:54.711 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.673+0000 7fc8f97e1700 1 --2- 172.21.15.47:0/3032608206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8f4071390 0x7fc8f40feb70 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:54.711 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.673+0000 7fc8f97e1700 1 -- 172.21.15.47:0/3032608206 >> 172.21.15.47:0/3032608206 conn(0x7fc8f40f6570 msgr2=0x7fc8f40f8990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:54.711 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.673+0000 7fc8f97e1700 1 -- 172.21.15.47:0/3032608206 shutdown_connections 2024-02-23T00:33:54.711 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.673+0000 7fc8f97e1700 1 -- 172.21.15.47:0/3032608206 wait complete. 2024-02-23T00:33:54.711 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.674+0000 7fc8f97e1700 1 librados: starting msgr at 2024-02-23T00:33:54.711 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.674+0000 7fc8f97e1700 1 librados: starting objecter 2024-02-23T00:33:54.712 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.674+0000 7fc8f97e1700 1 Processor -- start 2024-02-23T00:33:54.712 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.674+0000 7fc8f97e1700 1 -- start start 2024-02-23T00:33:54.712 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.674+0000 7fc8f97e1700 1 librados: setting wanted keys 2024-02-23T00:33:54.712 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.674+0000 7fc8f97e1700 1 librados: calling monclient init 2024-02-23T00:33:54.712 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.674+0000 7fc8f97e1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8f4071390 0x7fc8f410cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:54.712 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.674+0000 7fc8f97e1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f410d100 con 0x7fc8f4071390 2024-02-23T00:33:54.712 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.674+0000 7fc8f2ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8f4071390 0x7fc8f410cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:54.712 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.674+0000 7fc8f2ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8f4071390 0x7fc8f410cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37478/0 (socket says 172.21.15.47:37478) 2024-02-23T00:33:54.712 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.674+0000 7fc8f2ffd700 1 -- 172.21.15.47:0/1299986238 learned_addr learned my addr 172.21.15.47:0/1299986238 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:54.712 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.675+0000 7fc8f2ffd700 1 -- 172.21.15.47:0/1299986238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e4009370 con 0x7fc8f4071390 2024-02-23T00:33:54.712 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.675+0000 7fc8f2ffd700 1 --2- 172.21.15.47:0/1299986238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8f4071390 0x7fc8f410cc10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fc8e400b7b0 tx=0x7fc8e40090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:54.712 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.675+0000 7fc8ebfff700 10 client.?.objecter ms_handle_connect 0x7fc8f4071390 2024-02-23T00:33:54.712 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.675+0000 7fc8ebfff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:33:54.712 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.675+0000 7fc8ebfff700 1 -- 172.21.15.47:0/1299986238 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8e401a640 con 0x7fc8f4071390 2024-02-23T00:33:54.713 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.675+0000 7fc8f97e1700 1 -- 172.21.15.47:0/1299986238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8f410d300 con 0x7fc8f4071390 2024-02-23T00:33:54.713 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.675+0000 7fc8ebfff700 1 -- 172.21.15.47:0/1299986238 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fc8e401ac30 con 0x7fc8f4071390 2024-02-23T00:33:54.713 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.675+0000 7fc8ebfff700 1 -- 172.21.15.47:0/1299986238 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8e4023a00 con 0x7fc8f4071390 2024-02-23T00:33:54.713 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.675+0000 7fc8f97e1700 10 client.14156.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:33:54.713 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.675+0000 7fc8f97e1700 1 -- 172.21.15.47:0/1299986238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f410d790 con 0x7fc8f4071390 2024-02-23T00:33:54.713 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.675+0000 7fc8f97e1700 1 librados: init done 2024-02-23T00:33:54.713 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.676+0000 7fc8ebfff700 1 -- 172.21.15.47:0/1299986238 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 11) v1 ==== 38524+0+0 (secure 0 0 0) 0x7fc8e4023c80 con 0x7fc8f4071390 2024-02-23T00:33:54.713 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.676+0000 7fc8f97e1700 1 -- 172.21.15.47:0/1299986238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8f4046ce0 con 0x7fc8f4071390 2024-02-23T00:33:54.713 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.676+0000 7fc8ebfff700 1 --2- 172.21.15.47:0/1299986238 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fc8e0033e90 0x7fc8e0036350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:54.713 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.676+0000 7fc8ebfff700 1 -- 172.21.15.47:0/1299986238 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7fc8e4045020 con 0x7fc8f4071390 2024-02-23T00:33:54.713 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.676+0000 7fc8ebfff700 10 client.14156.objecter ms_dispatch 0x7fc8f4001550 osd_map(3..3 src has 1..3) v4 2024-02-23T00:33:54.713 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.676+0000 7fc8ebfff700 3 client.14156.objecter handle_osd_map got epochs [3,3] > 0 2024-02-23T00:33:54.713 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.676+0000 7fc8ebfff700 3 client.14156.objecter handle_osd_map decoding full epoch 3 2024-02-23T00:33:54.713 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.676+0000 7fc8ebfff700 20 client.14156.objecter dump_active .. 0 homeless 2024-02-23T00:33:54.714 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.676+0000 7fc8f27fc700 1 --2- 172.21.15.47:0/1299986238 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fc8e0033e90 0x7fc8e0036350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:54.714 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.676+0000 7fc8ebfff700 10 client.14156.objecter ms_handle_connect 0x7fc8e0033e90 2024-02-23T00:33:54.714 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.676+0000 7fc8f27fc700 1 --2- 172.21.15.47:0/1299986238 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fc8e0033e90 0x7fc8e0036350 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fc8dc009780 tx=0x7fc8dc006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:54.714 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.681+0000 7fc8ebfff700 1 -- 172.21.15.47:0/1299986238 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fc8e402b8b0 con 0x7fc8f4071390 2024-02-23T00:33:54.812 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.810+0000 7fc8ebfff700 1 -- 172.21.15.47:0/1299986238 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 12) v1 ==== 38630+0+0 (secure 0 0 0) 0x7fc8e402bd20 con 0x7fc8f4071390 2024-02-23T00:33:54.871 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.869+0000 7fc8f97e1700 1 -- 172.21.15.47:0/1299986238 --> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8f4106a50 con 0x7fc8e0033e90 2024-02-23T00:33:54.936 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.934+0000 7fc8ebfff700 1 -- 172.21.15.47:0/1299986238 <== mgr.14146 v2:172.21.15.47:6800/622994871 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fc8f4106a50 con 0x7fc8e0033e90 2024-02-23T00:33:54.936 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: Self-signed certificate created 2024-02-23T00:33:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.936+0000 7fc8f97e1700 10 librados: watch_flush enter 2024-02-23T00:33:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.936+0000 7fc8f97e1700 10 librados: watch_flush exit 2024-02-23T00:33:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.936+0000 7fc8f97e1700 20 client.14156.objecter shutdown clearing up homeless session... 2024-02-23T00:33:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.936+0000 7fc8f97e1700 10 client.14156.objecter successfully canceled tick 2024-02-23T00:33:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.936+0000 7fc8f97e1700 1 -- 172.21.15.47:0/1299986238 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fc8e0033e90 msgr2=0x7fc8e0036350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.936+0000 7fc8f97e1700 1 --2- 172.21.15.47:0/1299986238 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fc8e0033e90 0x7fc8e0036350 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fc8dc009780 tx=0x7fc8dc006cc0).stop 2024-02-23T00:33:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.936+0000 7fc8f97e1700 1 -- 172.21.15.47:0/1299986238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8f4071390 msgr2=0x7fc8f410cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:54.940 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.936+0000 7fc8f97e1700 1 --2- 172.21.15.47:0/1299986238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8f4071390 0x7fc8f410cc10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fc8e400b7b0 tx=0x7fc8e40090f0).stop 2024-02-23T00:33:54.940 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.936+0000 7fc8f97e1700 1 -- 172.21.15.47:0/1299986238 shutdown_connections 2024-02-23T00:33:54.940 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.936+0000 7fc8f97e1700 1 --2- 172.21.15.47:0/1299986238 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fc8e0033e90 0x7fc8e0036350 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:54.940 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.936+0000 7fc8f97e1700 1 --2- 172.21.15.47:0/1299986238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8f4071390 0x7fc8f410cc10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:54.940 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.936+0000 7fc8f97e1700 1 -- 172.21.15.47:0/1299986238 >> 172.21.15.47:0/1299986238 conn(0x7fc8f40f6570 msgr2=0x7fc8f40f7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:54.940 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.936+0000 7fc8f97e1700 1 -- 172.21.15.47:0/1299986238 shutdown_connections 2024-02-23T00:33:54.940 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.936+0000 7fc8f97e1700 1 -- 172.21.15.47:0/1299986238 wait complete. 2024-02-23T00:33:54.940 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:54.936+0000 7fc8f97e1700 1 librados: shutdown 2024-02-23T00:33:56.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:33:54.813332+0000 mon.a (mon.0) 82 : cluster [DBG] mgrmap e12: y(active, since 2s) 2024-02-23T00:33:56.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:54.871576+0000 mgr.y (mgr.14146) 3 : audit [DBG] from='client.14156 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:33:56.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:54.933344+0000 mon.a (mon.0) 83 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:33:56.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:54.935132+0000 mon.a (mon.0) 84 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:33:56.158 INFO:teuthology.orchestra.run.smithi047.stderr:Creating initial admin user... 2024-02-23T00:33:57.091 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.087+0000 7f65ab1f1700 1 Processor -- start 2024-02-23T00:33:57.091 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.087+0000 7f65ab1f1700 1 -- start start 2024-02-23T00:33:57.091 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.087+0000 7f65ab1f1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65a4073410 0x7f65a4071a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:57.091 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.087+0000 7f65ab1f1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a4073830 con 0x7f65a4073410 2024-02-23T00:33:57.091 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.087+0000 7f65a8f8d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65a4073410 0x7f65a4071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:57.091 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.087+0000 7f65a8f8d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65a4073410 0x7f65a4071a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59158/0 (socket says 172.21.15.47:59158) 2024-02-23T00:33:57.091 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.087+0000 7f65a8f8d700 1 -- 172.21.15.47:0/3317851676 learned_addr learned my addr 172.21.15.47:0/3317851676 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:57.091 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.087+0000 7f65a8f8d700 1 -- 172.21.15.47:0/3317851676 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a4071ef0 con 0x7f65a4073410 2024-02-23T00:33:57.091 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.088+0000 7f65a8f8d700 1 --2- 172.21.15.47:0/3317851676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65a4073410 0x7f65a4071a00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f659c009e50 tx=0x7f659c009840).ready entity=mon.0 client_cookie=e7ebe0723e0cf6fb server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:57.092 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.088+0000 7f65a37fe700 1 -- 172.21.15.47:0/3317851676 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f659c00e070 con 0x7f65a4073410 2024-02-23T00:33:57.092 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.088+0000 7f65a37fe700 1 -- 172.21.15.47:0/3317851676 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f659c013420 con 0x7f65a4073410 2024-02-23T00:33:57.092 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.088+0000 7f65a37fe700 1 -- 172.21.15.47:0/3317851676 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f659c007400 con 0x7f65a4073410 2024-02-23T00:33:57.092 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.088+0000 7f65ab1f1700 1 -- 172.21.15.47:0/3317851676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65a4073410 msgr2=0x7f65a4071a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:57.092 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.088+0000 7f65ab1f1700 1 --2- 172.21.15.47:0/3317851676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65a4073410 0x7f65a4071a00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f659c009e50 tx=0x7f659c009840).stop 2024-02-23T00:33:57.092 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.088+0000 7f65ab1f1700 1 -- 172.21.15.47:0/3317851676 shutdown_connections 2024-02-23T00:33:57.092 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.088+0000 7f65ab1f1700 1 --2- 172.21.15.47:0/3317851676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65a4073410 0x7f65a4071a00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:57.096 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.088+0000 7f65ab1f1700 1 -- 172.21.15.47:0/3317851676 >> 172.21.15.47:0/3317851676 conn(0x7f65a40f7850 msgr2=0x7f65a40f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:57.097 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.089+0000 7f65ab1f1700 1 -- 172.21.15.47:0/3317851676 shutdown_connections 2024-02-23T00:33:57.097 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.089+0000 7f65ab1f1700 1 -- 172.21.15.47:0/3317851676 wait complete. 2024-02-23T00:33:57.097 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.089+0000 7f65ab1f1700 1 librados: starting msgr at 2024-02-23T00:33:57.097 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.089+0000 7f65ab1f1700 1 librados: starting objecter 2024-02-23T00:33:57.097 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.089+0000 7f65ab1f1700 1 Processor -- start 2024-02-23T00:33:57.097 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.089+0000 7f65ab1f1700 1 -- start start 2024-02-23T00:33:57.097 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.089+0000 7f65ab1f1700 1 librados: setting wanted keys 2024-02-23T00:33:57.097 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.089+0000 7f65ab1f1700 1 librados: calling monclient init 2024-02-23T00:33:57.097 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.089+0000 7f65ab1f1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65a4073410 0x7f65a410cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:57.097 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.089+0000 7f65ab1f1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a410d240 con 0x7f65a4073410 2024-02-23T00:33:57.097 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.090+0000 7f65a8f8d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65a4073410 0x7f65a410cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:57.097 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.090+0000 7f65a8f8d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65a4073410 0x7f65a410cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59166/0 (socket says 172.21.15.47:59166) 2024-02-23T00:33:57.097 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.090+0000 7f65a8f8d700 1 -- 172.21.15.47:0/2899261285 learned_addr learned my addr 172.21.15.47:0/2899261285 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:57.097 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.090+0000 7f65a8f8d700 1 -- 172.21.15.47:0/2899261285 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659c009ae0 con 0x7f65a4073410 2024-02-23T00:33:57.098 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.090+0000 7f65a8f8d700 1 --2- 172.21.15.47:0/2899261285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65a4073410 0x7f65a410cd50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f659c00bf30 tx=0x7f659c009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:57.098 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.090+0000 7f65a1ffb700 10 client.?.objecter ms_handle_connect 0x7f65a4073410 2024-02-23T00:33:57.098 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.090+0000 7f65a1ffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:33:57.098 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.090+0000 7f65a1ffb700 1 -- 172.21.15.47:0/2899261285 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f659c00e040 con 0x7f65a4073410 2024-02-23T00:33:57.098 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.090+0000 7f65a1ffb700 1 -- 172.21.15.47:0/2899261285 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f659c007890 con 0x7f65a4073410 2024-02-23T00:33:57.098 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.090+0000 7f65ab1f1700 1 -- 172.21.15.47:0/2899261285 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a410d440 con 0x7f65a4073410 2024-02-23T00:33:57.098 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.090+0000 7f65a1ffb700 1 -- 172.21.15.47:0/2899261285 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f659c01c680 con 0x7f65a4073410 2024-02-23T00:33:57.098 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.090+0000 7f65ab1f1700 10 client.14158.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:33:57.098 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.090+0000 7f65ab1f1700 1 -- 172.21.15.47:0/2899261285 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a410dff0 con 0x7f65a4073410 2024-02-23T00:33:57.098 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.090+0000 7f65ab1f1700 1 librados: init done 2024-02-23T00:33:57.098 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.091+0000 7f65a1ffb700 1 -- 172.21.15.47:0/2899261285 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 12) v1 ==== 38630+0+0 (secure 0 0 0) 0x7f659c003040 con 0x7f65a4073410 2024-02-23T00:33:57.098 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.092+0000 7f65a1ffb700 1 --2- 172.21.15.47:0/2899261285 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f658c034000 0x7f658c0364c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:57.098 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.092+0000 7f65ab1f1700 1 -- 172.21.15.47:0/2899261285 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a405a310 con 0x7f65a4073410 2024-02-23T00:33:57.098 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.092+0000 7f65a1ffb700 1 -- 172.21.15.47:0/2899261285 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f659c0447a0 con 0x7f65a4073410 2024-02-23T00:33:57.099 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.092+0000 7f65a1ffb700 10 client.14158.objecter ms_dispatch 0x7f65a4001550 osd_map(3..3 src has 1..3) v4 2024-02-23T00:33:57.099 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.092+0000 7f65a1ffb700 3 client.14158.objecter handle_osd_map got epochs [3,3] > 0 2024-02-23T00:33:57.099 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.092+0000 7f65a1ffb700 3 client.14158.objecter handle_osd_map decoding full epoch 3 2024-02-23T00:33:57.099 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.092+0000 7f65a1ffb700 20 client.14158.objecter dump_active .. 0 homeless 2024-02-23T00:33:57.099 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.092+0000 7f65a3fff700 1 --2- 172.21.15.47:0/2899261285 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f658c034000 0x7f658c0364c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:57.099 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.092+0000 7f65a1ffb700 10 client.14158.objecter ms_handle_connect 0x7f658c034000 2024-02-23T00:33:57.099 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.092+0000 7f65a3fff700 1 --2- 172.21.15.47:0/2899261285 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f658c034000 0x7f658c0364c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f6598009780 tx=0x7f6598006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:57.099 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.096+0000 7f65a1ffb700 1 -- 172.21.15.47:0/2899261285 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f659c025400 con 0x7f65a4073410 2024-02-23T00:33:57.319 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.317+0000 7f65ab1f1700 1 -- 172.21.15.47:0/2899261285 --> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f65a405a130 con 0x7f658c034000 2024-02-23T00:33:57.569 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.567+0000 7f65a1ffb700 1 -- 172.21.15.47:0/2899261285 <== mgr.14146 v2:172.21.15.47:6800/622994871 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f65a405a130 con 0x7f658c034000 2024-02-23T00:33:57.570 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: {"username": "admin", "password": "$2b$12$BBv0W4c3TUqdwKz8UT//A.Jum4Qn33iGV0q9QVlGQxLAQzWC61KRa", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1708648437, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-23T00:33:57.572 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.569+0000 7f65ab1f1700 10 librados: watch_flush enter 2024-02-23T00:33:57.572 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.569+0000 7f65ab1f1700 10 librados: watch_flush exit 2024-02-23T00:33:57.572 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.569+0000 7f65ab1f1700 20 client.14158.objecter shutdown clearing up homeless session... 2024-02-23T00:33:57.572 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.569+0000 7f65ab1f1700 10 client.14158.objecter successfully canceled tick 2024-02-23T00:33:57.572 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.569+0000 7f65ab1f1700 1 -- 172.21.15.47:0/2899261285 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f658c034000 msgr2=0x7f658c0364c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:57.573 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.569+0000 7f65ab1f1700 1 --2- 172.21.15.47:0/2899261285 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f658c034000 0x7f658c0364c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f6598009780 tx=0x7f6598006cc0).stop 2024-02-23T00:33:57.573 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.569+0000 7f65ab1f1700 1 -- 172.21.15.47:0/2899261285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65a4073410 msgr2=0x7f65a410cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:57.573 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.569+0000 7f65ab1f1700 1 --2- 172.21.15.47:0/2899261285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65a4073410 0x7f65a410cd50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f659c00bf30 tx=0x7f659c009840).stop 2024-02-23T00:33:57.573 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.569+0000 7f65ab1f1700 1 -- 172.21.15.47:0/2899261285 shutdown_connections 2024-02-23T00:33:57.573 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.569+0000 7f65ab1f1700 1 --2- 172.21.15.47:0/2899261285 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f658c034000 0x7f658c0364c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:57.573 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.569+0000 7f65ab1f1700 1 --2- 172.21.15.47:0/2899261285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65a4073410 0x7f65a410cd50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:57.573 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.569+0000 7f65ab1f1700 1 -- 172.21.15.47:0/2899261285 >> 172.21.15.47:0/2899261285 conn(0x7f65a40f7850 msgr2=0x7f65a40fa150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:57.573 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.569+0000 7f65ab1f1700 1 -- 172.21.15.47:0/2899261285 shutdown_connections 2024-02-23T00:33:57.573 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.569+0000 7f65ab1f1700 1 -- 172.21.15.47:0/2899261285 wait complete. 2024-02-23T00:33:57.573 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:57.569+0000 7f65ab1f1700 1 librados: shutdown 2024-02-23T00:33:57.998 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:57.319069+0000 mgr.y (mgr.14146) 4 : audit [DBG] from='client.14158 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:33:57.998 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:57.568602+0000 mon.a (mon.0) 85 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:33:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:Fetching dashboard port number... 2024-02-23T00:33:58.970 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.967+0000 7f6d0267b700 1 Processor -- start 2024-02-23T00:33:58.970 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.967+0000 7f6d0267b700 1 -- start start 2024-02-23T00:33:58.971 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.968+0000 7f6d0267b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cfc070b60 0x7f6cfc100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:58.971 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.968+0000 7f6d0267b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cfc0fc8c0 con 0x7f6cfc070b60 2024-02-23T00:33:58.971 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.968+0000 7f6cfbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cfc070b60 0x7f6cfc100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:58.971 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.968+0000 7f6cfbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cfc070b60 0x7f6cfc100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59170/0 (socket says 172.21.15.47:59170) 2024-02-23T00:33:58.971 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.968+0000 7f6cfbfff700 1 -- 172.21.15.47:0/92666863 learned_addr learned my addr 172.21.15.47:0/92666863 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:58.971 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.968+0000 7f6cfbfff700 1 -- 172.21.15.47:0/92666863 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cfc1013c0 con 0x7f6cfc070b60 2024-02-23T00:33:58.971 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.968+0000 7f6cfbfff700 1 --2- 172.21.15.47:0/92666863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cfc070b60 0x7f6cfc100ed0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f6cec009730 tx=0x7f6cec0090f0).ready entity=mon.0 client_cookie=cf48a02194f2d34b server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:58.971 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.968+0000 7f6cfaffd700 1 -- 172.21.15.47:0/92666863 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6cec00b580 con 0x7f6cfc070b60 2024-02-23T00:33:58.975 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.968+0000 7f6cfaffd700 1 -- 172.21.15.47:0/92666863 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f6cec00bb70 con 0x7f6cfc070b60 2024-02-23T00:33:58.975 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.968+0000 7f6cfaffd700 1 -- 172.21.15.47:0/92666863 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6cec011a70 con 0x7f6cfc070b60 2024-02-23T00:33:58.975 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.969+0000 7f6d0267b700 1 -- 172.21.15.47:0/92666863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cfc070b60 msgr2=0x7f6cfc100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:58.975 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.969+0000 7f6d0267b700 1 --2- 172.21.15.47:0/92666863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cfc070b60 0x7f6cfc100ed0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f6cec009730 tx=0x7f6cec0090f0).stop 2024-02-23T00:33:58.975 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.969+0000 7f6d0267b700 1 -- 172.21.15.47:0/92666863 shutdown_connections 2024-02-23T00:33:58.975 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.969+0000 7f6d0267b700 1 --2- 172.21.15.47:0/92666863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cfc070b60 0x7f6cfc100ed0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:58.975 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.969+0000 7f6d0267b700 1 -- 172.21.15.47:0/92666863 >> 172.21.15.47:0/92666863 conn(0x7f6cfc0f6530 msgr2=0x7f6cfc0f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:58.975 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.969+0000 7f6d0267b700 1 -- 172.21.15.47:0/92666863 shutdown_connections 2024-02-23T00:33:58.976 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.969+0000 7f6d0267b700 1 -- 172.21.15.47:0/92666863 wait complete. 2024-02-23T00:33:58.976 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.969+0000 7f6d0267b700 1 librados: starting msgr at 2024-02-23T00:33:58.976 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.969+0000 7f6d0267b700 1 librados: starting objecter 2024-02-23T00:33:58.976 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.970+0000 7f6d0267b700 1 Processor -- start 2024-02-23T00:33:58.976 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.970+0000 7f6d0267b700 1 -- start start 2024-02-23T00:33:58.976 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.970+0000 7f6d0267b700 1 librados: setting wanted keys 2024-02-23T00:33:58.976 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.970+0000 7f6d0267b700 1 librados: calling monclient init 2024-02-23T00:33:58.976 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.970+0000 7f6d0267b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cfc070b60 0x7f6cfc10cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:58.976 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.970+0000 7f6d0267b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cfc10d0f0 con 0x7f6cfc070b60 2024-02-23T00:33:58.976 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.970+0000 7f6cfbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cfc070b60 0x7f6cfc10cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:58.976 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.970+0000 7f6cfbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cfc070b60 0x7f6cfc10cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59176/0 (socket says 172.21.15.47:59176) 2024-02-23T00:33:58.976 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.970+0000 7f6cfbfff700 1 -- 172.21.15.47:0/254074550 learned_addr learned my addr 172.21.15.47:0/254074550 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:33:58.976 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.970+0000 7f6cfbfff700 1 -- 172.21.15.47:0/254074550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cec009370 con 0x7f6cfc070b60 2024-02-23T00:33:58.976 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.971+0000 7f6cfbfff700 1 --2- 172.21.15.47:0/254074550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cfc070b60 0x7f6cfc10cc00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f6cec00b7b0 tx=0x7f6cec0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:58.977 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.971+0000 7f6cf97fa700 10 client.?.objecter ms_handle_connect 0x7f6cfc070b60 2024-02-23T00:33:58.977 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.971+0000 7f6cf97fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:33:58.977 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.971+0000 7f6cf97fa700 1 -- 172.21.15.47:0/254074550 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6cec01a640 con 0x7f6cfc070b60 2024-02-23T00:33:58.977 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.971+0000 7f6cf97fa700 1 -- 172.21.15.47:0/254074550 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f6cec01ac30 con 0x7f6cfc070b60 2024-02-23T00:33:58.977 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.971+0000 7f6cf97fa700 1 -- 172.21.15.47:0/254074550 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6cec023a00 con 0x7f6cfc070b60 2024-02-23T00:33:58.977 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.971+0000 7f6d0267b700 1 -- 172.21.15.47:0/254074550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cfc10d2f0 con 0x7f6cfc070b60 2024-02-23T00:33:58.977 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.971+0000 7f6d0267b700 10 client.14160.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:33:58.977 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.971+0000 7f6d0267b700 1 -- 172.21.15.47:0/254074550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cfc10d780 con 0x7f6cfc070b60 2024-02-23T00:33:58.977 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.971+0000 7f6d0267b700 1 librados: init done 2024-02-23T00:33:58.977 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.972+0000 7f6cf97fa700 1 -- 172.21.15.47:0/254074550 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 38673+0+0 (secure 0 0 0) 0x7f6cec01a7a0 con 0x7f6cfc070b60 2024-02-23T00:33:58.977 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.972+0000 7f6d0267b700 1 -- 172.21.15.47:0/254074550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cfc046ce0 con 0x7f6cfc070b60 2024-02-23T00:33:58.978 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.972+0000 7f6cf97fa700 1 --2- 172.21.15.47:0/254074550 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f6ce4034050 0x7f6ce4036510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:33:58.978 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.972+0000 7f6cf97fa700 1 -- 172.21.15.47:0/254074550 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f6cec044ee0 con 0x7f6cfc070b60 2024-02-23T00:33:58.978 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.972+0000 7f6cf97fa700 10 client.14160.objecter ms_dispatch 0x7f6cfc001550 osd_map(3..3 src has 1..3) v4 2024-02-23T00:33:58.978 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.972+0000 7f6cf97fa700 3 client.14160.objecter handle_osd_map got epochs [3,3] > 0 2024-02-23T00:33:58.978 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.972+0000 7f6cf97fa700 3 client.14160.objecter handle_osd_map decoding full epoch 3 2024-02-23T00:33:58.978 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.972+0000 7f6cf97fa700 20 client.14160.objecter dump_active .. 0 homeless 2024-02-23T00:33:58.978 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.973+0000 7f6cfb7fe700 1 --2- 172.21.15.47:0/254074550 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f6ce4034050 0x7f6ce4036510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:33:58.978 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.973+0000 7f6cf97fa700 10 client.14160.objecter ms_handle_connect 0x7f6ce4034050 2024-02-23T00:33:58.978 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.973+0000 7f6cfb7fe700 1 --2- 172.21.15.47:0/254074550 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f6ce4034050 0x7f6ce4036510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f6cf0009780 tx=0x7f6cf0006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:33:58.978 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:58.977+0000 7f6cf97fa700 1 -- 172.21.15.47:0/254074550 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f6cec0292a0 con 0x7f6cfc070b60 2024-02-23T00:33:59.148 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:59.146+0000 7f6d0267b700 1 -- 172.21.15.47:0/254074550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f6cfc102000 con 0x7f6cfc070b60 2024-02-23T00:33:59.148 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:59.146+0000 7f6cf97fa700 1 -- 172.21.15.47:0/254074550 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f6cec01e070 con 0x7f6cfc070b60 2024-02-23T00:33:59.149 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 8443 2024-02-23T00:33:59.151 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:59.148+0000 7f6d0267b700 10 librados: watch_flush enter 2024-02-23T00:33:59.151 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:59.148+0000 7f6d0267b700 10 librados: watch_flush exit 2024-02-23T00:33:59.151 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:59.148+0000 7f6d0267b700 20 client.14160.objecter shutdown clearing up homeless session... 2024-02-23T00:33:59.151 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:59.148+0000 7f6d0267b700 10 client.14160.objecter successfully canceled tick 2024-02-23T00:33:59.151 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:59.148+0000 7f6d0267b700 1 -- 172.21.15.47:0/254074550 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f6ce4034050 msgr2=0x7f6ce4036510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:59.151 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:59.148+0000 7f6d0267b700 1 --2- 172.21.15.47:0/254074550 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f6ce4034050 0x7f6ce4036510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f6cf0009780 tx=0x7f6cf0006cc0).stop 2024-02-23T00:33:59.152 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:59.148+0000 7f6d0267b700 1 -- 172.21.15.47:0/254074550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cfc070b60 msgr2=0x7f6cfc10cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:33:59.152 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:59.148+0000 7f6d0267b700 1 --2- 172.21.15.47:0/254074550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cfc070b60 0x7f6cfc10cc00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f6cec00b7b0 tx=0x7f6cec0090f0).stop 2024-02-23T00:33:59.152 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:59.148+0000 7f6d0267b700 1 -- 172.21.15.47:0/254074550 shutdown_connections 2024-02-23T00:33:59.152 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:59.148+0000 7f6d0267b700 1 --2- 172.21.15.47:0/254074550 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f6ce4034050 0x7f6ce4036510 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:59.152 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:59.148+0000 7f6d0267b700 1 --2- 172.21.15.47:0/254074550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cfc070b60 0x7f6cfc10cc00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:33:59.152 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:59.148+0000 7f6d0267b700 1 -- 172.21.15.47:0/254074550 >> 172.21.15.47:0/254074550 conn(0x7f6cfc0f6530 msgr2=0x7f6cfc0f7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:33:59.152 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:59.148+0000 7f6d0267b700 1 -- 172.21.15.47:0/254074550 shutdown_connections 2024-02-23T00:33:59.152 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:59.148+0000 7f6d0267b700 1 -- 172.21.15.47:0/254074550 wait complete. 2024-02-23T00:33:59.152 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:33:59.149+0000 7f6d0267b700 1 librados: shutdown 2024-02-23T00:33:59.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:33:58.581461+0000 mon.a (mon.0) 86 : cluster [DBG] mgrmap e13: y(active, since 6s) 2024-02-23T00:33:59.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:33:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:33:59.147863+0000 mon.a (mon.0) 87 : audit [DBG] from='client.? 172.21.15.47:0/254074550' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-23T00:33:59.863 INFO:teuthology.orchestra.run.smithi047.stderr:systemctl: disabled 2024-02-23T00:33:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:systemctl: inactive 2024-02-23T00:33:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:firewalld.service is not enabled 2024-02-23T00:33:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-23T00:33:59.875 INFO:teuthology.orchestra.run.smithi047.stderr:Ceph Dashboard is now available at: 2024-02-23T00:33:59.875 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-02-23T00:33:59.875 INFO:teuthology.orchestra.run.smithi047.stderr: URL: https://smithi047.front.sepia.ceph.com:8443/ 2024-02-23T00:33:59.875 INFO:teuthology.orchestra.run.smithi047.stderr: User: admin 2024-02-23T00:33:59.875 INFO:teuthology.orchestra.run.smithi047.stderr: Password: fxpjezggi2 2024-02-23T00:33:59.875 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-02-23T00:33:59.875 INFO:teuthology.orchestra.run.smithi047.stderr:Enabling autotune for osd_memory_target 2024-02-23T00:34:00.549 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.543+0000 7f6f83f60700 1 Processor -- start 2024-02-23T00:34:00.549 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.543+0000 7f6f83f60700 1 -- start start 2024-02-23T00:34:00.549 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.544+0000 7f6f83f60700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f7c070ae0 0x7f6f7c0feb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-23T00:34:00.549 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.544+0000 7f6f83f60700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c070f00 con 0x7f6f7c070ae0 2024-02-23T00:34:00.549 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.544+0000 7f6f81cfc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f7c070ae0 0x7f6f7c0feb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:34:00.550 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.544+0000 7f6f81cfc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f7c070ae0 0x7f6f7c0feb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59192/0 (socket says 172.21.15.47:59192) 2024-02-23T00:34:00.550 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.544+0000 7f6f81cfc700 1 -- 172.21.15.47:0/396471949 learned_addr learned my addr 172.21.15.47:0/396471949 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:34:00.550 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.544+0000 7f6f81cfc700 1 -- 172.21.15.47:0/396471949 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7c0ff040 con 0x7f6f7c070ae0 2024-02-23T00:34:00.550 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.544+0000 7f6f81cfc700 1 --2- 172.21.15.47:0/396471949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f7c070ae0 0x7f6f7c0feb50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f6f6c009730 tx=0x7f6f6c0090f0).ready entity=mon.0 client_cookie=51d0c81ffcc393c5 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:34:00.550 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.544+0000 7f6f80cfa700 1 -- 172.21.15.47:0/396471949 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f6c00b580 con 0x7f6f7c070ae0 2024-02-23T00:34:00.550 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.544+0000 7f6f80cfa700 1 -- 172.21.15.47:0/396471949 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f6f6c00bb70 con 0x7f6f7c070ae0 2024-02-23T00:34:00.550 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.544+0000 7f6f80cfa700 1 -- 172.21.15.47:0/396471949 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f6c011a70 con 0x7f6f7c070ae0 2024-02-23T00:34:00.550 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.545+0000 7f6f83f60700 1 -- 172.21.15.47:0/396471949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f7c070ae0 msgr2=0x7f6f7c0feb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:34:00.550 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.545+0000 7f6f83f60700 1 --2- 172.21.15.47:0/396471949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f7c070ae0 0x7f6f7c0feb50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f6f6c009730 tx=0x7f6f6c0090f0).stop 2024-02-23T00:34:00.550 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.545+0000 7f6f83f60700 1 -- 172.21.15.47:0/396471949 shutdown_connections 2024-02-23T00:34:00.550 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.545+0000 7f6f83f60700 1 --2- 172.21.15.47:0/396471949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f7c070ae0 0x7f6f7c0feb50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:34:00.550 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.545+0000 7f6f83f60700 1 -- 172.21.15.47:0/396471949 >> 172.21.15.47:0/396471949 conn(0x7f6f7c0f6530 msgr2=0x7f6f7c0f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:34:00.551 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.545+0000 7f6f83f60700 1 -- 172.21.15.47:0/396471949 shutdown_connections 2024-02-23T00:34:00.551 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.545+0000 7f6f83f60700 1 -- 172.21.15.47:0/396471949 wait complete. 2024-02-23T00:34:00.551 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.546+0000 7f6f83f60700 1 librados: starting msgr at 2024-02-23T00:34:00.551 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.546+0000 7f6f83f60700 1 librados: starting objecter 2024-02-23T00:34:00.551 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.546+0000 7f6f83f60700 1 Processor -- start 2024-02-23T00:34:00.551 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.546+0000 7f6f83f60700 1 -- start start 2024-02-23T00:34:00.551 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.546+0000 7f6f83f60700 1 librados: setting wanted keys 2024-02-23T00:34:00.551 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.546+0000 7f6f83f60700 1 librados: calling monclient init 2024-02-23T00:34:00.551 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.546+0000 7f6f83f60700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f7c070ae0 0x7f6f7c10a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:34:00.551 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.546+0000 7f6f83f60700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c10aed0 con 0x7f6f7c070ae0 2024-02-23T00:34:00.551 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.546+0000 7f6f81cfc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f7c070ae0 0x7f6f7c10a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:34:00.551 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.546+0000 7f6f81cfc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f7c070ae0 0x7f6f7c10a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59202/0 (socket says 172.21.15.47:59202) 2024-02-23T00:34:00.551 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.546+0000 7f6f81cfc700 1 -- 172.21.15.47:0/2120449356 learned_addr learned my addr 172.21.15.47:0/2120449356 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:34:00.552 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.546+0000 7f6f81cfc700 1 -- 172.21.15.47:0/2120449356 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f6c009370 con 0x7f6f7c070ae0 2024-02-23T00:34:00.552 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.547+0000 7f6f81cfc700 1 --2- 172.21.15.47:0/2120449356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f7c070ae0 0x7f6f7c10a9e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f6f6c012f10 tx=0x7f6f6c0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:34:00.552 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.547+0000 7f6f72ffd700 10 client.?.objecter ms_handle_connect 0x7f6f7c070ae0 2024-02-23T00:34:00.552 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.547+0000 7f6f72ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:34:00.552 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.547+0000 7f6f72ffd700 1 -- 172.21.15.47:0/2120449356 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f6c01a640 con 0x7f6f7c070ae0 2024-02-23T00:34:00.552 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.547+0000 7f6f83f60700 1 -- 172.21.15.47:0/2120449356 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f7c10b0d0 con 0x7f6f7c070ae0 2024-02-23T00:34:00.552 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.547+0000 7f6f72ffd700 1 -- 172.21.15.47:0/2120449356 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f6f6c01ac30 con 0x7f6f7c070ae0 2024-02-23T00:34:00.552 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.547+0000 7f6f72ffd700 1 -- 172.21.15.47:0/2120449356 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f6c0239e0 con 0x7f6f7c070ae0 2024-02-23T00:34:00.552 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.547+0000 7f6f83f60700 10 client.14162.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:34:00.552 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.547+0000 7f6f83f60700 1 -- 172.21.15.47:0/2120449356 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f7c10b4e0 con 0x7f6f7c070ae0 2024-02-23T00:34:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.547+0000 7f6f83f60700 1 librados: init done 2024-02-23T00:34:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.548+0000 7f6f72ffd700 1 -- 172.21.15.47:0/2120449356 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 38673+0+0 (secure 0 0 0) 0x7f6f6c01a7a0 con 0x7f6f7c070ae0 2024-02-23T00:34:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.548+0000 7f6f83f60700 1 -- 172.21.15.47:0/2120449356 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f7c046ce0 con 0x7f6f7c070ae0 2024-02-23T00:34:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.548+0000 7f6f72ffd700 1 --2- 172.21.15.47:0/2120449356 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f6f68038460 0x7f6f6803a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:34:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.548+0000 7f6f72ffd700 1 -- 172.21.15.47:0/2120449356 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f6f6c044e10 con 0x7f6f7c070ae0 2024-02-23T00:34:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.548+0000 7f6f72ffd700 10 client.14162.objecter ms_dispatch 0x7f6f7c001550 osd_map(3..3 src has 1..3) v4 2024-02-23T00:34:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.548+0000 7f6f72ffd700 3 client.14162.objecter handle_osd_map got epochs [3,3] > 0 2024-02-23T00:34:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.548+0000 7f6f72ffd700 3 client.14162.objecter handle_osd_map decoding full epoch 3 2024-02-23T00:34:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.548+0000 7f6f72ffd700 20 client.14162.objecter dump_active .. 0 homeless 2024-02-23T00:34:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.548+0000 7f6f814fb700 1 --2- 172.21.15.47:0/2120449356 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f6f68038460 0x7f6f6803a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:34:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.548+0000 7f6f72ffd700 10 client.14162.objecter ms_handle_connect 0x7f6f68038460 2024-02-23T00:34:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.548+0000 7f6f814fb700 1 --2- 172.21.15.47:0/2120449356 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f6f68038460 0x7f6f6803a920 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f6f78009780 tx=0x7f6f78006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:34:00.554 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.552+0000 7f6f72ffd700 1 -- 172.21.15.47:0/2120449356 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f6f6c019920 con 0x7f6f7c070ae0 2024-02-23T00:34:00.725 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.722+0000 7f6f83f60700 1 -- 172.21.15.47:0/2120449356 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f6f7c0ffe30 con 0x7f6f7c070ae0 2024-02-23T00:34:00.726 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.722+0000 7f6f72ffd700 1 -- 172.21.15.47:0/2120449356 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f6f6c01e020 con 0x7f6f7c070ae0 2024-02-23T00:34:00.728 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.724+0000 7f6f83f60700 10 librados: watch_flush enter 2024-02-23T00:34:00.728 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.724+0000 7f6f83f60700 10 librados: watch_flush exit 2024-02-23T00:34:00.728 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.724+0000 7f6f83f60700 20 client.14162.objecter shutdown clearing up homeless session... 2024-02-23T00:34:00.728 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.724+0000 7f6f83f60700 10 client.14162.objecter successfully canceled tick 2024-02-23T00:34:00.728 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.724+0000 7f6f83f60700 1 -- 172.21.15.47:0/2120449356 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f6f68038460 msgr2=0x7f6f6803a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:34:00.728 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.724+0000 7f6f83f60700 1 --2- 172.21.15.47:0/2120449356 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f6f68038460 0x7f6f6803a920 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f6f78009780 tx=0x7f6f78006cc0).stop 2024-02-23T00:34:00.728 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.724+0000 7f6f83f60700 1 -- 172.21.15.47:0/2120449356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f7c070ae0 msgr2=0x7f6f7c10a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:34:00.728 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.724+0000 7f6f83f60700 1 --2- 172.21.15.47:0/2120449356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f7c070ae0 0x7f6f7c10a9e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f6f6c012f10 tx=0x7f6f6c0090f0).stop 2024-02-23T00:34:00.728 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.724+0000 7f6f83f60700 1 -- 172.21.15.47:0/2120449356 shutdown_connections 2024-02-23T00:34:00.728 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.724+0000 7f6f83f60700 1 --2- 172.21.15.47:0/2120449356 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f6f68038460 0x7f6f6803a920 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:34:00.728 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.724+0000 7f6f83f60700 1 --2- 172.21.15.47:0/2120449356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f7c070ae0 0x7f6f7c10a9e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:34:00.728 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.724+0000 7f6f83f60700 1 -- 172.21.15.47:0/2120449356 >> 172.21.15.47:0/2120449356 conn(0x7f6f7c0f6530 msgr2=0x7f6f7c0f7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:34:00.729 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.724+0000 7f6f83f60700 1 -- 172.21.15.47:0/2120449356 shutdown_connections 2024-02-23T00:34:00.729 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.724+0000 7f6f83f60700 1 -- 172.21.15.47:0/2120449356 wait complete. 2024-02-23T00:34:00.729 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:00.725+0000 7f6f83f60700 1 librados: shutdown 2024-02-23T00:34:02.121 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.113+0000 7fe43a943700 1 Processor -- start 2024-02-23T00:34:02.121 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.113+0000 7fe43a943700 1 -- start start 2024-02-23T00:34:02.121 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.113+0000 7fe43a943700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe434070b40 0x7fe434100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-23T00:34:02.121 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.113+0000 7fe43a943700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4340fc8e0 con 0x7fe434070b40 2024-02-23T00:34:02.122 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.113+0000 7fe433fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe434070b40 0x7fe434100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:34:02.122 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.114+0000 7fe433fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe434070b40 0x7fe434100ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59210/0 (socket says 172.21.15.47:59210) 2024-02-23T00:34:02.122 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.114+0000 7fe433fff700 1 -- 172.21.15.47:0/4264821747 learned_addr learned my addr 172.21.15.47:0/4264821747 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:34:02.122 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.114+0000 7fe433fff700 1 -- 172.21.15.47:0/4264821747 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4341013e0 con 0x7fe434070b40 2024-02-23T00:34:02.122 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.114+0000 7fe433fff700 1 --2- 172.21.15.47:0/4264821747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe434070b40 0x7fe434100ef0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fe424009730 tx=0x7fe4240090f0).ready entity=mon.0 client_cookie=faf4dfb5d946a02a server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:34:02.122 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.114+0000 7fe432ffd700 1 -- 172.21.15.47:0/4264821747 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe42400b580 con 0x7fe434070b40 2024-02-23T00:34:02.122 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.114+0000 7fe432ffd700 1 -- 172.21.15.47:0/4264821747 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fe42400bb70 con 0x7fe434070b40 2024-02-23T00:34:02.122 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.114+0000 7fe432ffd700 1 -- 172.21.15.47:0/4264821747 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe424011a70 con 0x7fe434070b40 2024-02-23T00:34:02.122 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.115+0000 7fe43a943700 1 -- 172.21.15.47:0/4264821747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe434070b40 msgr2=0x7fe434100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:34:02.122 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.115+0000 7fe43a943700 1 --2- 172.21.15.47:0/4264821747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe434070b40 0x7fe434100ef0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fe424009730 tx=0x7fe4240090f0).stop 2024-02-23T00:34:02.122 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.115+0000 7fe43a943700 1 -- 172.21.15.47:0/4264821747 shutdown_connections 2024-02-23T00:34:02.122 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.115+0000 7fe43a943700 1 --2- 172.21.15.47:0/4264821747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe434070b40 0x7fe434100ef0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:34:02.122 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.115+0000 7fe43a943700 1 -- 172.21.15.47:0/4264821747 >> 172.21.15.47:0/4264821747 conn(0x7fe4340f6530 msgr2=0x7fe4340f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:34:02.122 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.115+0000 7fe43a943700 1 -- 172.21.15.47:0/4264821747 shutdown_connections 2024-02-23T00:34:02.122 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.115+0000 7fe43a943700 1 -- 172.21.15.47:0/4264821747 wait complete. 2024-02-23T00:34:02.123 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.115+0000 7fe43a943700 1 librados: starting msgr at 2024-02-23T00:34:02.123 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.115+0000 7fe43a943700 1 librados: starting objecter 2024-02-23T00:34:02.123 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.116+0000 7fe43a943700 1 Processor -- start 2024-02-23T00:34:02.123 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.116+0000 7fe43a943700 1 -- start start 2024-02-23T00:34:02.123 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.116+0000 7fe43a943700 1 librados: setting wanted keys 2024-02-23T00:34:02.123 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.116+0000 7fe43a943700 1 librados: calling monclient init 2024-02-23T00:34:02.123 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.116+0000 7fe43a943700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe434070b40 0x7fe43410cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:34:02.123 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.116+0000 7fe43a943700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43410d0c0 con 0x7fe434070b40 2024-02-23T00:34:02.123 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.116+0000 7fe433fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe434070b40 0x7fe43410cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:34:02.123 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.116+0000 7fe433fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe434070b40 0x7fe43410cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59218/0 (socket says 172.21.15.47:59218) 2024-02-23T00:34:02.123 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.116+0000 7fe433fff700 1 -- 172.21.15.47:0/2107748619 learned_addr learned my addr 172.21.15.47:0/2107748619 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:34:02.123 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.117+0000 7fe433fff700 1 -- 172.21.15.47:0/2107748619 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe424009370 con 0x7fe434070b40 2024-02-23T00:34:02.123 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.117+0000 7fe433fff700 1 --2- 172.21.15.47:0/2107748619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe434070b40 0x7fe43410cbd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fe424012f10 tx=0x7fe4240090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:34:02.123 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.117+0000 7fe4317fa700 10 client.?.objecter ms_handle_connect 0x7fe434070b40 2024-02-23T00:34:02.123 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.117+0000 7fe4317fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:34:02.123 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.117+0000 7fe4317fa700 1 -- 172.21.15.47:0/2107748619 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe42401a640 con 0x7fe434070b40 2024-02-23T00:34:02.124 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.117+0000 7fe4317fa700 1 -- 172.21.15.47:0/2107748619 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fe42401ac30 con 0x7fe434070b40 2024-02-23T00:34:02.124 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.117+0000 7fe4317fa700 1 -- 172.21.15.47:0/2107748619 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4240239e0 con 0x7fe434070b40 2024-02-23T00:34:02.124 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.117+0000 7fe43a943700 1 -- 172.21.15.47:0/2107748619 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe43410d2c0 con 0x7fe434070b40 2024-02-23T00:34:02.124 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.117+0000 7fe43a943700 10 client.14164.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:34:02.124 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.117+0000 7fe43a943700 1 -- 172.21.15.47:0/2107748619 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe43410d750 con 0x7fe434070b40 2024-02-23T00:34:02.124 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.117+0000 7fe43a943700 1 librados: init done 2024-02-23T00:34:02.124 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.118+0000 7fe4317fa700 1 -- 172.21.15.47:0/2107748619 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 38673+0+0 (secure 0 0 0) 0x7fe42401a7a0 con 0x7fe434070b40 2024-02-23T00:34:02.124 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.118+0000 7fe4317fa700 1 --2- 172.21.15.47:0/2107748619 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fe41c0340a0 0x7fe41c036560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:34:02.124 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.118+0000 7fe4317fa700 1 -- 172.21.15.47:0/2107748619 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7fe424044e60 con 0x7fe434070b40 2024-02-23T00:34:02.124 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.118+0000 7fe4317fa700 10 client.14164.objecter ms_dispatch 0x7fe434001550 osd_map(3..3 src has 1..3) v4 2024-02-23T00:34:02.124 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.118+0000 7fe4317fa700 3 client.14164.objecter handle_osd_map got epochs [3,3] > 0 2024-02-23T00:34:02.124 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.118+0000 7fe4317fa700 3 client.14164.objecter handle_osd_map decoding full epoch 3 2024-02-23T00:34:02.124 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.118+0000 7fe4317fa700 20 client.14164.objecter dump_active .. 0 homeless 2024-02-23T00:34:02.125 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.118+0000 7fe43a943700 1 -- 172.21.15.47:0/2107748619 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe434046ce0 con 0x7fe434070b40 2024-02-23T00:34:02.125 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.118+0000 7fe4337fe700 1 --2- 172.21.15.47:0/2107748619 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fe41c0340a0 0x7fe41c036560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:34:02.125 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.119+0000 7fe4317fa700 10 client.14164.objecter ms_handle_connect 0x7fe41c0340a0 2024-02-23T00:34:02.125 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.119+0000 7fe4337fe700 1 --2- 172.21.15.47:0/2107748619 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fe41c0340a0 0x7fe41c036560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fe428009780 tx=0x7fe428006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:34:02.125 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.123+0000 7fe4317fa700 1 -- 172.21.15.47:0/2107748619 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe42401e020 con 0x7fe434070b40 2024-02-23T00:34:02.390 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.388+0000 7fe43a943700 1 -- 172.21.15.47:0/2107748619 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fe4341020b0 con 0x7fe434070b40 2024-02-23T00:34:02.392 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.390+0000 7fe4317fa700 1 -- 172.21.15.47:0/2107748619 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v24)=0 set mgr/dashboard/cluster/status v24) v1 ==== 153+0+0 (secure 0 0 0) 0x7fe42402c3f0 con 0x7fe434070b40 2024-02-23T00:34:02.395 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.392+0000 7fe43a943700 10 librados: watch_flush enter 2024-02-23T00:34:02.395 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.392+0000 7fe43a943700 10 librados: watch_flush exit 2024-02-23T00:34:02.395 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.392+0000 7fe43a943700 20 client.14164.objecter shutdown clearing up homeless session... 2024-02-23T00:34:02.396 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.392+0000 7fe43a943700 10 client.14164.objecter successfully canceled tick 2024-02-23T00:34:02.396 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.392+0000 7fe43a943700 1 -- 172.21.15.47:0/2107748619 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fe41c0340a0 msgr2=0x7fe41c036560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:34:02.396 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.392+0000 7fe43a943700 1 --2- 172.21.15.47:0/2107748619 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fe41c0340a0 0x7fe41c036560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fe428009780 tx=0x7fe428006cc0).stop 2024-02-23T00:34:02.396 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.392+0000 7fe43a943700 1 -- 172.21.15.47:0/2107748619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe434070b40 msgr2=0x7fe43410cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:34:02.396 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.392+0000 7fe43a943700 1 --2- 172.21.15.47:0/2107748619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe434070b40 0x7fe43410cbd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fe424012f10 tx=0x7fe4240090f0).stop 2024-02-23T00:34:02.396 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.392+0000 7fe43a943700 1 -- 172.21.15.47:0/2107748619 shutdown_connections 2024-02-23T00:34:02.396 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.392+0000 7fe43a943700 1 --2- 172.21.15.47:0/2107748619 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fe41c0340a0 0x7fe41c036560 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:34:02.396 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.392+0000 7fe43a943700 1 --2- 172.21.15.47:0/2107748619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe434070b40 0x7fe43410cbd0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:34:02.396 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.392+0000 7fe43a943700 1 -- 172.21.15.47:0/2107748619 >> 172.21.15.47:0/2107748619 conn(0x7fe4340f6530 msgr2=0x7fe4340f7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:34:02.396 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.392+0000 7fe43a943700 1 -- 172.21.15.47:0/2107748619 shutdown_connections 2024-02-23T00:34:02.396 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.392+0000 7fe43a943700 1 -- 172.21.15.47:0/2107748619 wait complete. 2024-02-23T00:34:02.396 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: 2024-02-23T00:34:02.392+0000 7fe43a943700 1 librados: shutdown 2024-02-23T00:34:02.396 INFO:teuthology.orchestra.run.smithi047.stderr:/usr/bin/ceph: set mgr/dashboard/cluster/status 2024-02-23T00:34:03.066 INFO:teuthology.orchestra.run.smithi047.stderr:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-23T00:34:03.066 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-02-23T00:34:03.066 INFO:teuthology.orchestra.run.smithi047.stderr: sudo /home/ubuntu/cephtest/cephadm shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-23T00:34:03.066 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-02-23T00:34:03.066 INFO:teuthology.orchestra.run.smithi047.stderr:Or, if you are only running a single cluster on this host: 2024-02-23T00:34:03.066 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-02-23T00:34:03.066 INFO:teuthology.orchestra.run.smithi047.stderr: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-23T00:34:03.066 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-02-23T00:34:03.066 INFO:teuthology.orchestra.run.smithi047.stderr:Please consider enabling telemetry to help improve Ceph: 2024-02-23T00:34:03.067 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-02-23T00:34:03.067 INFO:teuthology.orchestra.run.smithi047.stderr: ceph telemetry on 2024-02-23T00:34:03.067 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-02-23T00:34:03.067 INFO:teuthology.orchestra.run.smithi047.stderr:For more information see: 2024-02-23T00:34:03.067 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-02-23T00:34:03.067 INFO:teuthology.orchestra.run.smithi047.stderr: https://docs.ceph.com/en/pacific/mgr/telemetry/ 2024-02-23T00:34:03.067 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-02-23T00:34:03.067 INFO:teuthology.orchestra.run.smithi047.stderr:Bootstrap complete. 2024-02-23T00:34:03.113 INFO:tasks.cephadm:Fetching config... 2024-02-23T00:34:03.114 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-02-23T00:34:03.114 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-23T00:34:03.175 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-23T00:34:03.175 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-02-23T00:34:03.175 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-23T00:34:03.234 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-23T00:34:03.234 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-02-23T00:34:03.234 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/keyring of=/dev/stdout 2024-02-23T00:34:03.304 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-23T00:34:03.304 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-02-23T00:34:03.304 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-23T00:34:03.363 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-23T00:34:03.364 DEBUG:teuthology.orchestra.run.smithi047:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzKgjPTR2hwImYTv0iLqANPGRqURcRKmXdJVbOiDqCxX3Bqga20+evwIci0xwlOXp+Ne6FILdQ3X8/v33Z0fFAPERRq3dNan5QJFuOS3G9mp78yy69ikoey8eb73epKR9UpenH/U10fvRayxwIgca+lvwRXaTQRStP1uyJG4/cXbRm443SsuIi7NEmNgYZNGZ4oDW8LBEt97eZxQ1p0TklvnF4ozQ/TgsxQvnDcD5fhNO9URl21Q2ekL59RYPG9xVLAmjO8G3Y3FS8TVeVcJ7z68ok6opskjegliBJlwfSA6Gxf/QZ+tyfgvCmuarcG5GQLVcfatBO8dxaUY6GxKKDeQwUlm1fzGhuswZnza+hQvn09qpxL8Cau8rA2F+/W7GSkxrjUhjX9b88NRKz8DVFfq88WZw0fMvc8teXRXzV2tEgUGYMH0pRIU+ehEedjLdmHzG+yAa5BmbUsWXn+G4o9PmNMV5e3dYUfQ09kFASPhHmV5kdf6KRgN7PmL6j8Nc= ceph-fce2617e-d1e2-11ee-95c0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-23T00:34:03.445 INFO:teuthology.orchestra.run.smithi047.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzKgjPTR2hwImYTv0iLqANPGRqURcRKmXdJVbOiDqCxX3Bqga20+evwIci0xwlOXp+Ne6FILdQ3X8/v33Z0fFAPERRq3dNan5QJFuOS3G9mp78yy69ikoey8eb73epKR9UpenH/U10fvRayxwIgca+lvwRXaTQRStP1uyJG4/cXbRm443SsuIi7NEmNgYZNGZ4oDW8LBEt97eZxQ1p0TklvnF4ozQ/TgsxQvnDcD5fhNO9URl21Q2ekL59RYPG9xVLAmjO8G3Y3FS8TVeVcJ7z68ok6opskjegliBJlwfSA6Gxf/QZ+tyfgvCmuarcG5GQLVcfatBO8dxaUY6GxKKDeQwUlm1fzGhuswZnza+hQvn09qpxL8Cau8rA2F+/W7GSkxrjUhjX9b88NRKz8DVFfq88WZw0fMvc8teXRXzV2tEgUGYMH0pRIU+ehEedjLdmHzG+yAa5BmbUsWXn+G4o9PmNMV5e3dYUfQ09kFASPhHmV5kdf6KRgN7PmL6j8Nc= ceph-fce2617e-d1e2-11ee-95c0-87774f69a715 2024-02-23T00:34:03.456 DEBUG:teuthology.orchestra.run.smithi196:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzKgjPTR2hwImYTv0iLqANPGRqURcRKmXdJVbOiDqCxX3Bqga20+evwIci0xwlOXp+Ne6FILdQ3X8/v33Z0fFAPERRq3dNan5QJFuOS3G9mp78yy69ikoey8eb73epKR9UpenH/U10fvRayxwIgca+lvwRXaTQRStP1uyJG4/cXbRm443SsuIi7NEmNgYZNGZ4oDW8LBEt97eZxQ1p0TklvnF4ozQ/TgsxQvnDcD5fhNO9URl21Q2ekL59RYPG9xVLAmjO8G3Y3FS8TVeVcJ7z68ok6opskjegliBJlwfSA6Gxf/QZ+tyfgvCmuarcG5GQLVcfatBO8dxaUY6GxKKDeQwUlm1fzGhuswZnza+hQvn09qpxL8Cau8rA2F+/W7GSkxrjUhjX9b88NRKz8DVFfq88WZw0fMvc8teXRXzV2tEgUGYMH0pRIU+ehEedjLdmHzG+yAa5BmbUsWXn+G4o9PmNMV5e3dYUfQ09kFASPhHmV5kdf6KRgN7PmL6j8Nc= ceph-fce2617e-d1e2-11ee-95c0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-23T00:34:03.500 INFO:teuthology.orchestra.run.smithi196.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzKgjPTR2hwImYTv0iLqANPGRqURcRKmXdJVbOiDqCxX3Bqga20+evwIci0xwlOXp+Ne6FILdQ3X8/v33Z0fFAPERRq3dNan5QJFuOS3G9mp78yy69ikoey8eb73epKR9UpenH/U10fvRayxwIgca+lvwRXaTQRStP1uyJG4/cXbRm443SsuIi7NEmNgYZNGZ4oDW8LBEt97eZxQ1p0TklvnF4ozQ/TgsxQvnDcD5fhNO9URl21Q2ekL59RYPG9xVLAmjO8G3Y3FS8TVeVcJ7z68ok6opskjegliBJlwfSA6Gxf/QZ+tyfgvCmuarcG5GQLVcfatBO8dxaUY6GxKKDeQwUlm1fzGhuswZnza+hQvn09qpxL8Cau8rA2F+/W7GSkxrjUhjX9b88NRKz8DVFfq88WZw0fMvc8teXRXzV2tEgUGYMH0pRIU+ehEedjLdmHzG+yAa5BmbUsWXn+G4o9PmNMV5e3dYUfQ09kFASPhHmV5kdf6KRgN7PmL6j8Nc= ceph-fce2617e-d1e2-11ee-95c0-87774f69a715 2024-02-23T00:34:03.512 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-23T00:34:03.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:02.391868+0000 mon.a (mon.0) 88 : audit [INF] from='client.? 172.21.15.47:0/2107748619' entity='client.admin' 2024-02-23T00:34:07.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:05.796999+0000 mon.a (mon.0) 89 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:34:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.147+0000 7f0449174700 1 -- 172.21.15.47:0/3072444573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04440f64b0 msgr2=0x7f04440f68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:34:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.147+0000 7f0449174700 1 --2- 172.21.15.47:0/3072444573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04440f64b0 0x7f04440f68d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f0434004770 tx=0x7f0434009cb0).stop 2024-02-23T00:34:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.147+0000 7f0449174700 1 -- 172.21.15.47:0/3072444573 shutdown_connections 2024-02-23T00:34:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.147+0000 7f0449174700 1 --2- 172.21.15.47:0/3072444573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04440f64b0 0x7f04440f68d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:34:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.147+0000 7f0449174700 1 -- 172.21.15.47:0/3072444573 >> 172.21.15.47:0/3072444573 conn(0x7f04440f1c10 msgr2=0x7f04440f4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:34:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.147+0000 7f0449174700 1 -- 172.21.15.47:0/3072444573 shutdown_connections 2024-02-23T00:34:07.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.147+0000 7f0449174700 1 -- 172.21.15.47:0/3072444573 wait complete. 2024-02-23T00:34:07.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.147+0000 7f0449174700 1 librados: starting msgr at 2024-02-23T00:34:07.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.147+0000 7f0449174700 1 librados: starting objecter 2024-02-23T00:34:07.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.148+0000 7f0449174700 1 Processor -- start 2024-02-23T00:34:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.148+0000 7f0449174700 1 -- start start 2024-02-23T00:34:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.148+0000 7f0449174700 1 librados: setting wanted keys 2024-02-23T00:34:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.148+0000 7f0449174700 1 librados: calling monclient init 2024-02-23T00:34:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.148+0000 7f0449174700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04440f64b0 0x7f0444106ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:34:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.148+0000 7f0449174700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0444107390 con 0x7f04440f64b0 2024-02-23T00:34:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.148+0000 7f0442d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04440f64b0 0x7f0444106ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:34:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.148+0000 7f0442d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04440f64b0 0x7f0444106ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35300/0 (socket says 172.21.15.47:35300) 2024-02-23T00:34:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.148+0000 7f0442d9d700 1 -- 172.21.15.47:0/421739475 learned_addr learned my addr 172.21.15.47:0/421739475 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:34:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.148+0000 7f0442d9d700 1 -- 172.21.15.47:0/421739475 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0434005040 con 0x7f04440f64b0 2024-02-23T00:34:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.148+0000 7f0442d9d700 1 --2- 172.21.15.47:0/421739475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04440f64b0 0x7f0444106ea0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f0434009b10 tx=0x7f0434009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:34:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.148+0000 7f042bfff700 10 client.?.objecter ms_handle_connect 0x7f04440f64b0 2024-02-23T00:34:07.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.149+0000 7f042bfff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:34:07.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.149+0000 7f042bfff700 1 -- 172.21.15.47:0/421739475 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f043400e030 con 0x7f04440f64b0 2024-02-23T00:34:07.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.149+0000 7f042bfff700 1 -- 172.21.15.47:0/421739475 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f0434013e60 con 0x7f04440f64b0 2024-02-23T00:34:07.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.149+0000 7f0449174700 1 -- 172.21.15.47:0/421739475 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0444107590 con 0x7f04440f64b0 2024-02-23T00:34:07.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.149+0000 7f042bfff700 1 -- 172.21.15.47:0/421739475 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0434018070 con 0x7f04440f64b0 2024-02-23T00:34:07.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.149+0000 7f0449174700 10 client.14166.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:34:07.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.149+0000 7f0449174700 1 -- 172.21.15.47:0/421739475 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0444108140 con 0x7f04440f64b0 2024-02-23T00:34:07.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.149+0000 7f0449174700 1 librados: init done 2024-02-23T00:34:07.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.150+0000 7f042bfff700 1 -- 172.21.15.47:0/421739475 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 38673+0+0 (secure 0 0 0) 0x7f0434027440 con 0x7f04440f64b0 2024-02-23T00:34:07.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.150+0000 7f0449174700 1 -- 172.21.15.47:0/421739475 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f044405a310 con 0x7f04440f64b0 2024-02-23T00:34:07.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.150+0000 7f042bfff700 1 --2- 172.21.15.47:0/421739475 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f042c034050 0x7f042c036510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:34:07.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.150+0000 7f042bfff700 1 -- 172.21.15.47:0/421739475 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f04340441d0 con 0x7f04440f64b0 2024-02-23T00:34:07.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.150+0000 7f042bfff700 10 client.14166.objecter ms_dispatch 0x7f0444001550 osd_map(3..3 src has 1..3) v4 2024-02-23T00:34:07.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.150+0000 7f042bfff700 3 client.14166.objecter handle_osd_map got epochs [3,3] > 0 2024-02-23T00:34:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.150+0000 7f042bfff700 3 client.14166.objecter handle_osd_map decoding full epoch 3 2024-02-23T00:34:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.150+0000 7f042bfff700 20 client.14166.objecter dump_active .. 0 homeless 2024-02-23T00:34:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.150+0000 7f044259c700 1 --2- 172.21.15.47:0/421739475 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f042c034050 0x7f042c036510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:34:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.150+0000 7f044259c700 1 --2- 172.21.15.47:0/421739475 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f042c034050 0x7f042c036510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f0438009780 tx=0x7f0438006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:34:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.150+0000 7f042bfff700 10 client.14166.objecter ms_handle_connect 0x7f042c034050 2024-02-23T00:34:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.155+0000 7f042bfff700 1 -- 172.21.15.47:0/421739475 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f043401de60 con 0x7f04440f64b0 2024-02-23T00:34:07.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.334+0000 7f0449174700 1 -- 172.21.15.47:0/421739475 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f0444107a20 con 0x7f04440f64b0 2024-02-23T00:34:07.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.337+0000 7f042bfff700 1 -- 172.21.15.47:0/421739475 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f0444107a20 con 0x7f04440f64b0 2024-02-23T00:34:07.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.338+0000 7f0449174700 10 librados: watch_flush enter 2024-02-23T00:34:07.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.339+0000 7f0449174700 10 librados: watch_flush exit 2024-02-23T00:34:07.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.339+0000 7f0449174700 20 client.14166.objecter shutdown clearing up homeless session... 2024-02-23T00:34:07.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.339+0000 7f0449174700 10 client.14166.objecter successfully canceled tick 2024-02-23T00:34:07.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.339+0000 7f0449174700 1 -- 172.21.15.47:0/421739475 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f042c034050 msgr2=0x7f042c036510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:34:07.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.339+0000 7f0449174700 1 --2- 172.21.15.47:0/421739475 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f042c034050 0x7f042c036510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f0438009780 tx=0x7f0438006cc0).stop 2024-02-23T00:34:07.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.339+0000 7f0449174700 1 -- 172.21.15.47:0/421739475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04440f64b0 msgr2=0x7f0444106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:34:07.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.339+0000 7f0449174700 1 --2- 172.21.15.47:0/421739475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04440f64b0 0x7f0444106ea0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f0434009b10 tx=0x7f0434009cb0).stop 2024-02-23T00:34:07.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.339+0000 7f0449174700 1 -- 172.21.15.47:0/421739475 shutdown_connections 2024-02-23T00:34:07.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.339+0000 7f0449174700 1 --2- 172.21.15.47:0/421739475 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f042c034050 0x7f042c036510 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:34:07.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.339+0000 7f0449174700 1 --2- 172.21.15.47:0/421739475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04440f64b0 0x7f0444106ea0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:34:07.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.339+0000 7f0449174700 1 -- 172.21.15.47:0/421739475 >> 172.21.15.47:0/421739475 conn(0x7f04440f1c10 msgr2=0x7f0444060da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:34:07.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.339+0000 7f0449174700 1 -- 172.21.15.47:0/421739475 shutdown_connections 2024-02-23T00:34:07.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.339+0000 7f0449174700 1 -- 172.21.15.47:0/421739475 wait complete. 2024-02-23T00:34:07.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:07.339+0000 7f0449174700 1 librados: shutdown 2024-02-23T00:34:08.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:06.951642+0000 mon.a (mon.0) 90 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi047", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:34:08.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:06.954949+0000 mon.a (mon.0) 91 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:34:08.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:06.956265+0000 mon.a (mon.0) 92 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:34:08.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:06.959128+0000 mon.a (mon.0) 93 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:34:08.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:07.338045+0000 mon.a (mon.0) 94 : audit [INF] from='client.? 172.21.15.47:0/421739475' entity='client.admin' 2024-02-23T00:34:08.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:07.374123+0000 mon.a (mon.0) 95 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:34:08.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:07.377648+0000 mon.a (mon.0) 96 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:34:08.438 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-23T00:34:08.439 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-23T00:34:10.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.449+0000 7fa5951d8700 1 -- 172.21.15.47:0/1190909133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5900f64b0 msgr2=0x7fa5900f68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:34:10.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.449+0000 7fa5951d8700 1 --2- 172.21.15.47:0/1190909133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5900f64b0 0x7fa5900f68d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fa580004770 tx=0x7fa580009cb0).stop 2024-02-23T00:34:10.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.449+0000 7fa5951d8700 1 -- 172.21.15.47:0/1190909133 shutdown_connections 2024-02-23T00:34:10.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.449+0000 7fa5951d8700 1 --2- 172.21.15.47:0/1190909133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5900f64b0 0x7fa5900f68d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:34:10.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.449+0000 7fa5951d8700 1 -- 172.21.15.47:0/1190909133 >> 172.21.15.47:0/1190909133 conn(0x7fa5900f1c10 msgr2=0x7fa5900f4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:34:10.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.449+0000 7fa5951d8700 1 -- 172.21.15.47:0/1190909133 shutdown_connections 2024-02-23T00:34:10.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.449+0000 7fa5951d8700 1 -- 172.21.15.47:0/1190909133 wait complete. 2024-02-23T00:34:10.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.450+0000 7fa5951d8700 1 librados: starting msgr at 2024-02-23T00:34:10.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.450+0000 7fa5951d8700 1 librados: starting objecter 2024-02-23T00:34:10.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.450+0000 7fa5951d8700 1 Processor -- start 2024-02-23T00:34:10.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.450+0000 7fa5951d8700 1 -- start start 2024-02-23T00:34:10.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.450+0000 7fa5951d8700 1 librados: setting wanted keys 2024-02-23T00:34:10.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.450+0000 7fa5951d8700 1 librados: calling monclient init 2024-02-23T00:34:10.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.451+0000 7fa5951d8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5900f64b0 0x7fa59010bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:34:10.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.451+0000 7fa5951d8700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa590107900 con 0x7fa5900f64b0 2024-02-23T00:34:10.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.451+0000 7fa58ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5900f64b0 0x7fa59010bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:34:10.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.451+0000 7fa58ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5900f64b0 0x7fa59010bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35318/0 (socket says 172.21.15.47:35318) 2024-02-23T00:34:10.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.451+0000 7fa58ed9d700 1 -- 172.21.15.47:0/1886900932 learned_addr learned my addr 172.21.15.47:0/1886900932 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:34:10.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.451+0000 7fa58ed9d700 1 -- 172.21.15.47:0/1886900932 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa580005040 con 0x7fa5900f64b0 2024-02-23T00:34:10.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.451+0000 7fa58ed9d700 1 --2- 172.21.15.47:0/1886900932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5900f64b0 0x7fa59010bf10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fa580009b10 tx=0x7fa580009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:34:10.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.451+0000 7fa577fff700 10 client.?.objecter ms_handle_connect 0x7fa5900f64b0 2024-02-23T00:34:10.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.451+0000 7fa577fff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:34:10.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.451+0000 7fa577fff700 1 -- 172.21.15.47:0/1886900932 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa58000e030 con 0x7fa5900f64b0 2024-02-23T00:34:10.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.451+0000 7fa577fff700 1 -- 172.21.15.47:0/1886900932 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fa580013e60 con 0x7fa5900f64b0 2024-02-23T00:34:10.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.451+0000 7fa5951d8700 1 -- 172.21.15.47:0/1886900932 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa590061190 con 0x7fa5900f64b0 2024-02-23T00:34:10.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.451+0000 7fa577fff700 1 -- 172.21.15.47:0/1886900932 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa580018070 con 0x7fa5900f64b0 2024-02-23T00:34:10.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.451+0000 7fa5951d8700 10 client.14168.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:34:10.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.451+0000 7fa5951d8700 1 -- 172.21.15.47:0/1886900932 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa59010c400 con 0x7fa5900f64b0 2024-02-23T00:34:10.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.451+0000 7fa5951d8700 1 librados: init done 2024-02-23T00:34:10.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.452+0000 7fa5951d8700 1 -- 172.21.15.47:0/1886900932 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa59005a310 con 0x7fa5900f64b0 2024-02-23T00:34:10.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.453+0000 7fa577fff700 1 -- 172.21.15.47:0/1886900932 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 38673+0+0 (secure 0 0 0) 0x7fa58001d600 con 0x7fa5900f64b0 2024-02-23T00:34:10.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.453+0000 7fa577fff700 1 --2- 172.21.15.47:0/1886900932 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fa578034050 0x7fa578036510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:34:10.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.453+0000 7fa577fff700 1 -- 172.21.15.47:0/1886900932 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7fa580044510 con 0x7fa5900f64b0 2024-02-23T00:34:10.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.453+0000 7fa577fff700 10 client.14168.objecter ms_dispatch 0x7fa590001550 osd_map(3..3 src has 1..3) v4 2024-02-23T00:34:10.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.453+0000 7fa577fff700 3 client.14168.objecter handle_osd_map got epochs [3,3] > 0 2024-02-23T00:34:10.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.453+0000 7fa577fff700 3 client.14168.objecter handle_osd_map decoding full epoch 3 2024-02-23T00:34:10.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.453+0000 7fa577fff700 20 client.14168.objecter dump_active .. 0 homeless 2024-02-23T00:34:10.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.453+0000 7fa58e59c700 1 --2- 172.21.15.47:0/1886900932 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fa578034050 0x7fa578036510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:34:10.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.453+0000 7fa577fff700 10 client.14168.objecter ms_handle_connect 0x7fa578034050 2024-02-23T00:34:10.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.453+0000 7fa58e59c700 1 --2- 172.21.15.47:0/1886900932 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fa578034050 0x7fa578036510 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fa584009780 tx=0x7fa584006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:34:10.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.457+0000 7fa577fff700 1 -- 172.21.15.47:0/1886900932 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fa58001de60 con 0x7fa5900f64b0 2024-02-23T00:34:10.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.634+0000 7fa5951d8700 1 -- 172.21.15.47:0/1886900932 --> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5900f2d30 con 0x7fa578034050 2024-02-23T00:34:10.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.637+0000 7fa577fff700 1 -- 172.21.15.47:0/1886900932 <== mgr.14146 v2:172.21.15.47:6800/622994871 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa5900f2d30 con 0x7fa578034050 2024-02-23T00:34:10.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.638+0000 7fa5951d8700 10 librados: watch_flush enter 2024-02-23T00:34:10.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.638+0000 7fa5951d8700 10 librados: watch_flush exit 2024-02-23T00:34:10.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.638+0000 7fa5951d8700 20 client.14168.objecter shutdown clearing up homeless session... 2024-02-23T00:34:10.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.638+0000 7fa5951d8700 10 client.14168.objecter successfully canceled tick 2024-02-23T00:34:10.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.638+0000 7fa5951d8700 1 -- 172.21.15.47:0/1886900932 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fa578034050 msgr2=0x7fa578036510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:34:10.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.638+0000 7fa5951d8700 1 --2- 172.21.15.47:0/1886900932 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fa578034050 0x7fa578036510 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fa584009780 tx=0x7fa584006cc0).stop 2024-02-23T00:34:10.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.639+0000 7fa5951d8700 1 -- 172.21.15.47:0/1886900932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5900f64b0 msgr2=0x7fa59010bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:34:10.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.639+0000 7fa5951d8700 1 --2- 172.21.15.47:0/1886900932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5900f64b0 0x7fa59010bf10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fa580009b10 tx=0x7fa580009cb0).stop 2024-02-23T00:34:10.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.639+0000 7fa5951d8700 1 -- 172.21.15.47:0/1886900932 shutdown_connections 2024-02-23T00:34:10.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.639+0000 7fa5951d8700 1 --2- 172.21.15.47:0/1886900932 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fa578034050 0x7fa578036510 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:34:10.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.639+0000 7fa5951d8700 1 --2- 172.21.15.47:0/1886900932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5900f64b0 0x7fa59010bf10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:34:10.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.639+0000 7fa5951d8700 1 -- 172.21.15.47:0/1886900932 >> 172.21.15.47:0/1886900932 conn(0x7fa5900f1c10 msgr2=0x7fa5900f2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:34:10.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.639+0000 7fa5951d8700 1 -- 172.21.15.47:0/1886900932 shutdown_connections 2024-02-23T00:34:10.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.639+0000 7fa5951d8700 1 -- 172.21.15.47:0/1886900932 wait complete. 2024-02-23T00:34:10.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:10.639+0000 7fa5951d8700 1 librados: shutdown 2024-02-23T00:34:11.534 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi196 2024-02-23T00:34:11.535 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-02-23T00:34:11.535 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.conf 2024-02-23T00:34:11.556 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-02-23T00:34:11.557 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-23T00:34:11.617 INFO:tasks.cephadm:Adding host smithi196 to orchestrator... 2024-02-23T00:34:11.617 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph orch host add smithi196 2024-02-23T00:34:11.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:10.638083+0000 mon.a (mon.0) 97 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:34:11.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:10.648809+0000 mon.a (mon.0) 98 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:34:11.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:10.650197+0000 mon.a (mon.0) 99 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:34:11.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:10.650786+0000 mon.a (mon.0) 100 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:34:11.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:34:10.651763+0000 mgr.y (mgr.14146) 6 : cephadm [INF] Updating smithi047:/etc/ceph/ceph.conf 2024-02-23T00:34:11.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:34:10.680844+0000 mgr.y (mgr.14146) 7 : cephadm [INF] Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-02-23T00:34:11.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:10.756676+0000 mon.a (mon.0) 101 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:34:11.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:10.758860+0000 mon.a (mon.0) 102 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:34:13.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:34:12.068536+0000 mgr.y (mgr.14146) 8 : cluster [DBG] pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:34:15.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:34:14.068876+0000 mgr.y (mgr.14146) 9 : cluster [DBG] pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:34:17.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:34:16.069264+0000 mgr.y (mgr.14146) 10 : cluster [DBG] pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:34:19.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:34:18.069571+0000 mgr.y (mgr.14146) 11 : cluster [DBG] pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:34:21.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:34:20.069861+0000 mgr.y (mgr.14146) 12 : cluster [DBG] pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:34:23.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:34:22.070211+0000 mgr.y (mgr.14146) 13 : cluster [DBG] pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:34:25.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:34:24.070623+0000 mgr.y (mgr.14146) 14 : cluster [DBG] pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:34:27.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:34:26.070909+0000 mgr.y (mgr.14146) 15 : cluster [DBG] pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:34:29.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:34:28.071251+0000 mgr.y (mgr.14146) 16 : cluster [DBG] pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:34:31.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:34:30.071541+0000 mgr.y (mgr.14146) 17 : cluster [DBG] pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:34:33.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:34:32.071857+0000 mgr.y (mgr.14146) 18 : cluster [DBG] pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:34:35.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:34:34.072207+0000 mgr.y (mgr.14146) 19 : cluster [DBG] pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:34:37.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:34:36.072484+0000 mgr.y (mgr.14146) 20 : cluster [DBG] pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:34:39.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:34:38.072774+0000 mgr.y (mgr.14146) 21 : cluster [DBG] pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:34:41.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.018+0000 7f08c3f78700 1 -- 172.21.15.196:0/1495921516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08bc0f64b0 msgr2=0x7f08bc0f68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:34:41.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.018+0000 7f08c3f78700 1 --2- 172.21.15.196:0/1495921516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08bc0f64b0 0x7f08bc0f68d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f08b8004770 tx=0x7f08b8009cb0).stop 2024-02-23T00:34:41.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.018+0000 7f08c3f78700 1 -- 172.21.15.196:0/1495921516 shutdown_connections 2024-02-23T00:34:41.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.018+0000 7f08c3f78700 1 --2- 172.21.15.196:0/1495921516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08bc0f64b0 0x7f08bc0f68d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:34:41.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.018+0000 7f08c3f78700 1 -- 172.21.15.196:0/1495921516 >> 172.21.15.196:0/1495921516 conn(0x7f08bc0f1c10 msgr2=0x7f08bc0f4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:34:41.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.018+0000 7f08c3f78700 1 -- 172.21.15.196:0/1495921516 shutdown_connections 2024-02-23T00:34:41.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.018+0000 7f08c3f78700 1 -- 172.21.15.196:0/1495921516 wait complete. 2024-02-23T00:34:41.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.019+0000 7f08c3f78700 1 librados: starting msgr at 2024-02-23T00:34:41.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.019+0000 7f08c3f78700 1 librados: starting objecter 2024-02-23T00:34:41.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.019+0000 7f08c3f78700 1 Processor -- start 2024-02-23T00:34:41.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.019+0000 7f08c3f78700 1 -- start start 2024-02-23T00:34:41.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.019+0000 7f08c3f78700 1 librados: setting wanted keys 2024-02-23T00:34:41.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.019+0000 7f08c3f78700 1 librados: calling monclient init 2024-02-23T00:34:41.025 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.019+0000 7f08c3f78700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08bc0f64b0 0x7f08bc106ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:34:41.025 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.019+0000 7f08c3f78700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc1073c0 con 0x7f08bc0f64b0 2024-02-23T00:34:41.025 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.019+0000 7f08c1d14700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08bc0f64b0 0x7f08bc106ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:34:41.025 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.019+0000 7f08c1d14700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08bc0f64b0 0x7f08bc106ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.196:52432/0 (socket says 172.21.15.196:52432) 2024-02-23T00:34:41.025 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.019+0000 7f08c1d14700 1 -- 172.21.15.196:0/1164001907 learned_addr learned my addr 172.21.15.196:0/1164001907 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-02-23T00:34:41.025 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.020+0000 7f08c1d14700 1 -- 172.21.15.196:0/1164001907 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b8005040 con 0x7f08bc0f64b0 2024-02-23T00:34:41.025 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.020+0000 7f08c1d14700 1 --2- 172.21.15.196:0/1164001907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08bc0f64b0 0x7f08bc106ed0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f08b80097f0 tx=0x7f08b8009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:34:41.025 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.020+0000 7f08b2ffd700 10 client.?.objecter ms_handle_connect 0x7f08bc0f64b0 2024-02-23T00:34:41.025 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.020+0000 7f08b2ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:34:41.025 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.020+0000 7f08b2ffd700 1 -- 172.21.15.196:0/1164001907 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f08b800e030 con 0x7f08bc0f64b0 2024-02-23T00:34:41.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.020+0000 7f08c3f78700 1 -- 172.21.15.196:0/1164001907 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08bc1075c0 con 0x7f08bc0f64b0 2024-02-23T00:34:41.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.020+0000 7f08b2ffd700 1 -- 172.21.15.196:0/1164001907 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f08b801d420 con 0x7f08bc0f64b0 2024-02-23T00:34:41.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.020+0000 7f08b2ffd700 1 -- 172.21.15.196:0/1164001907 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f08b8018070 con 0x7f08bc0f64b0 2024-02-23T00:34:41.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.020+0000 7f08c3f78700 10 client.14170.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:34:41.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.020+0000 7f08c3f78700 1 -- 172.21.15.196:0/1164001907 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08bc108170 con 0x7f08bc0f64b0 2024-02-23T00:34:41.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.020+0000 7f08c3f78700 1 librados: init done 2024-02-23T00:34:41.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.021+0000 7f08b2ffd700 1 -- 172.21.15.196:0/1164001907 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 38673+0+0 (secure 0 0 0) 0x7f08b8013a20 con 0x7f08bc0f64b0 2024-02-23T00:34:41.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.021+0000 7f08c3f78700 1 -- 172.21.15.196:0/1164001907 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08bc05a310 con 0x7f08bc0f64b0 2024-02-23T00:34:41.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.021+0000 7f08b2ffd700 1 --2- 172.21.15.196:0/1164001907 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f08a8034050 0x7f08a8036510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:34:41.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.021+0000 7f08b2ffd700 1 -- 172.21.15.196:0/1164001907 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f08b80441a0 con 0x7f08bc0f64b0 2024-02-23T00:34:41.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.021+0000 7f08b2ffd700 10 client.14170.objecter ms_dispatch 0x7f08bc001550 osd_map(3..3 src has 1..3) v4 2024-02-23T00:34:41.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.021+0000 7f08b2ffd700 3 client.14170.objecter handle_osd_map got epochs [3,3] > 0 2024-02-23T00:34:41.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.021+0000 7f08b2ffd700 3 client.14170.objecter handle_osd_map decoding full epoch 3 2024-02-23T00:34:41.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.021+0000 7f08b2ffd700 20 client.14170.objecter dump_active .. 0 homeless 2024-02-23T00:34:41.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.021+0000 7f08c1513700 1 --2- 172.21.15.196:0/1164001907 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f08a8034050 0x7f08a8036510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:34:41.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.021+0000 7f08b2ffd700 10 client.14170.objecter ms_handle_connect 0x7f08a8034050 2024-02-23T00:34:41.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.021+0000 7f08c1513700 1 --2- 172.21.15.196:0/1164001907 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f08a8034050 0x7f08a8036510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f08ac009780 tx=0x7f08ac006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:34:41.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.026+0000 7f08b2ffd700 1 -- 172.21.15.196:0/1164001907 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f08b8025710 con 0x7f08bc0f64b0 2024-02-23T00:34:41.193 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.192+0000 7f08c3f78700 1 -- 172.21.15.196:0/1164001907 --> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}) v1 -- 0x7f08bc0fb7a0 con 0x7f08a8034050 2024-02-23T00:34:41.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:34:40.073055+0000 mgr.y (mgr.14146) 22 : cluster [DBG] pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:34:41.556 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:34:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [90B blob data] 2024-02-23T00:34:41.816 INFO:teuthology.orchestra.run.smithi196.stdout:Added host 'smithi196' with addr '172.21.15.196' 2024-02-23T00:34:41.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.813+0000 7f08b2ffd700 1 -- 172.21.15.196:0/1164001907 <== mgr.14146 v2:172.21.15.47:6800/622994871 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f08bc0fb7a0 con 0x7f08a8034050 2024-02-23T00:34:41.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.816+0000 7f08c3f78700 10 librados: watch_flush enter 2024-02-23T00:34:41.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.816+0000 7f08c3f78700 10 librados: watch_flush exit 2024-02-23T00:34:41.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.816+0000 7f08c3f78700 20 client.14170.objecter shutdown clearing up homeless session... 2024-02-23T00:34:41.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.816+0000 7f08c3f78700 10 client.14170.objecter successfully canceled tick 2024-02-23T00:34:41.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.816+0000 7f08c3f78700 1 -- 172.21.15.196:0/1164001907 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f08a8034050 msgr2=0x7f08a8036510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:34:41.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.816+0000 7f08c3f78700 1 --2- 172.21.15.196:0/1164001907 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f08a8034050 0x7f08a8036510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f08ac009780 tx=0x7f08ac006cc0).stop 2024-02-23T00:34:41.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.816+0000 7f08c3f78700 1 -- 172.21.15.196:0/1164001907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08bc0f64b0 msgr2=0x7f08bc106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:34:41.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.816+0000 7f08c3f78700 1 --2- 172.21.15.196:0/1164001907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08bc0f64b0 0x7f08bc106ed0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f08b80097f0 tx=0x7f08b8009cb0).stop 2024-02-23T00:34:41.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.816+0000 7f08c3f78700 1 -- 172.21.15.196:0/1164001907 shutdown_connections 2024-02-23T00:34:41.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.816+0000 7f08c3f78700 1 --2- 172.21.15.196:0/1164001907 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f08a8034050 0x7f08a8036510 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:34:41.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.816+0000 7f08c3f78700 1 --2- 172.21.15.196:0/1164001907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08bc0f64b0 0x7f08bc106ed0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:34:41.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.816+0000 7f08c3f78700 1 -- 172.21.15.196:0/1164001907 >> 172.21.15.196:0/1164001907 conn(0x7f08bc0f1c10 msgr2=0x7f08bc0f2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:34:41.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.816+0000 7f08c3f78700 1 -- 172.21.15.196:0/1164001907 shutdown_connections 2024-02-23T00:34:41.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.816+0000 7f08c3f78700 1 -- 172.21.15.196:0/1164001907 wait complete. 2024-02-23T00:34:41.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:41.816+0000 7f08c3f78700 1 librados: shutdown 2024-02-23T00:34:42.306 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:34:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [80B blob data] 2024-02-23T00:34:42.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:41.194005+0000 mgr.y (mgr.14146) 23 : audit [DBG] from='client.14170 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:34:42.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:41.814119+0000 mon.a (mon.0) 103 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:34:42.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:41.857023+0000 mon.a (mon.0) 104 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:34:42.565 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph orch host ls --format=json 2024-02-23T00:34:43.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:34:41.814533+0000 mgr.y (mgr.14146) 24 : cephadm [INF] Added host smithi196 2024-02-23T00:34:43.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:34:42.073500+0000 mgr.y (mgr.14146) 25 : cluster [DBG] pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:34:43.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:42.644351+0000 mon.a (mon.0) 105 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:34:44.502 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.474+0000 7f3dc350c700 1 -- 172.21.15.196:0/1885178724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3dbc0f75c0 msgr2=0x7f3dbc0f79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:34:44.502 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.474+0000 7f3dc350c700 1 --2- 172.21.15.196:0/1885178724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3dbc0f75c0 0x7f3dbc0f79e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f3dac004770 tx=0x7f3dac009cb0).stop 2024-02-23T00:34:44.502 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.474+0000 7f3dc350c700 1 -- 172.21.15.196:0/1885178724 shutdown_connections 2024-02-23T00:34:44.502 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.474+0000 7f3dc350c700 1 --2- 172.21.15.196:0/1885178724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3dbc0f75c0 0x7f3dbc0f79e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:34:44.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.474+0000 7f3dc350c700 1 -- 172.21.15.196:0/1885178724 >> 172.21.15.196:0/1885178724 conn(0x7f3dbc0f2d40 msgr2=0x7f3dbc0f5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:34:44.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.474+0000 7f3dc350c700 1 -- 172.21.15.196:0/1885178724 shutdown_connections 2024-02-23T00:34:44.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.474+0000 7f3dc350c700 1 -- 172.21.15.196:0/1885178724 wait complete. 2024-02-23T00:34:44.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.474+0000 7f3dc350c700 1 librados: starting msgr at 2024-02-23T00:34:44.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.475+0000 7f3dc350c700 1 librados: starting objecter 2024-02-23T00:34:44.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.475+0000 7f3dc350c700 1 Processor -- start 2024-02-23T00:34:44.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.475+0000 7f3dc350c700 1 -- start start 2024-02-23T00:34:44.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.475+0000 7f3dc350c700 1 librados: setting wanted keys 2024-02-23T00:34:44.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.475+0000 7f3dc350c700 1 librados: calling monclient init 2024-02-23T00:34:44.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.475+0000 7f3dc350c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3dbc0f75c0 0x7f3dbc106ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:34:44.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.475+0000 7f3dc350c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dbc107390 con 0x7f3dbc0f75c0 2024-02-23T00:34:44.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.475+0000 7f3dc12a8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3dbc0f75c0 0x7f3dbc106ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:34:44.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.475+0000 7f3dc12a8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3dbc0f75c0 0x7f3dbc106ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.196:52440/0 (socket says 172.21.15.196:52440) 2024-02-23T00:34:44.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.475+0000 7f3dc12a8700 1 -- 172.21.15.196:0/3456610484 learned_addr learned my addr 172.21.15.196:0/3456610484 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-02-23T00:34:44.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.476+0000 7f3dc12a8700 1 -- 172.21.15.196:0/3456610484 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dac005040 con 0x7f3dbc0f75c0 2024-02-23T00:34:44.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.476+0000 7f3dc12a8700 1 --2- 172.21.15.196:0/3456610484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3dbc0f75c0 0x7f3dbc106ea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f3dac0097f0 tx=0x7f3dac009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:34:44.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.476+0000 7f3db27fc700 10 client.?.objecter ms_handle_connect 0x7f3dbc0f75c0 2024-02-23T00:34:44.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.476+0000 7f3db27fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:34:44.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.476+0000 7f3db27fc700 1 -- 172.21.15.196:0/3456610484 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3dac00e030 con 0x7f3dbc0f75c0 2024-02-23T00:34:44.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.476+0000 7f3db27fc700 1 -- 172.21.15.196:0/3456610484 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f3dac01d420 con 0x7f3dbc0f75c0 2024-02-23T00:34:44.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.476+0000 7f3dc350c700 1 -- 172.21.15.196:0/3456610484 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dbc107590 con 0x7f3dbc0f75c0 2024-02-23T00:34:44.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.476+0000 7f3db27fc700 1 -- 172.21.15.196:0/3456610484 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3dac018070 con 0x7f3dbc0f75c0 2024-02-23T00:34:44.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.476+0000 7f3dc350c700 10 client.14172.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:34:44.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.476+0000 7f3dc350c700 1 -- 172.21.15.196:0/3456610484 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dbc108140 con 0x7f3dbc0f75c0 2024-02-23T00:34:44.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.476+0000 7f3dc350c700 1 librados: init done 2024-02-23T00:34:44.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.477+0000 7f3db27fc700 1 -- 172.21.15.196:0/3456610484 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 38673+0+0 (secure 0 0 0) 0x7f3dac013a20 con 0x7f3dbc0f75c0 2024-02-23T00:34:44.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.477+0000 7f3db27fc700 1 --2- 172.21.15.196:0/3456610484 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f3da80340a0 0x7f3da8036560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:34:44.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.477+0000 7f3dc350c700 1 -- 172.21.15.196:0/3456610484 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dbc05a310 con 0x7f3dbc0f75c0 2024-02-23T00:34:44.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.477+0000 7f3db27fc700 1 -- 172.21.15.196:0/3456610484 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f3dac044090 con 0x7f3dbc0f75c0 2024-02-23T00:34:44.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.477+0000 7f3db27fc700 10 client.14172.objecter ms_dispatch 0x7f3dbc001550 osd_map(3..3 src has 1..3) v4 2024-02-23T00:34:44.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.477+0000 7f3db27fc700 3 client.14172.objecter handle_osd_map got epochs [3,3] > 0 2024-02-23T00:34:44.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.477+0000 7f3db27fc700 3 client.14172.objecter handle_osd_map decoding full epoch 3 2024-02-23T00:34:44.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.477+0000 7f3db27fc700 20 client.14172.objecter dump_active .. 0 homeless 2024-02-23T00:34:44.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.477+0000 7f3dc0aa7700 1 --2- 172.21.15.196:0/3456610484 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f3da80340a0 0x7f3da8036560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:34:44.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.477+0000 7f3db27fc700 10 client.14172.objecter ms_handle_connect 0x7f3da80340a0 2024-02-23T00:34:44.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.477+0000 7f3dc0aa7700 1 --2- 172.21.15.196:0/3456610484 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f3da80340a0 0x7f3da8036560 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f3db8009780 tx=0x7f3db8006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:34:44.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.482+0000 7f3db27fc700 1 -- 172.21.15.196:0/3456610484 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f3dac072f60 con 0x7f3dbc0f75c0 2024-02-23T00:34:44.670 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.669+0000 7f3dc350c700 1 -- 172.21.15.196:0/3456610484 --> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3dbc0fc8b0 con 0x7f3da80340a0 2024-02-23T00:34:44.672 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.670+0000 7f3db27fc700 1 -- 172.21.15.196:0/3456610484 <== mgr.14146 v2:172.21.15.47:6800/622994871 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f3dbc0fc8b0 con 0x7f3da80340a0 2024-02-23T00:34:44.672 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:34:44.672 INFO:teuthology.orchestra.run.smithi196.stdout:[{"addr": "172.21.15.47", "hostname": "smithi047", "labels": [], "status": ""}, {"addr": "172.21.15.196", "hostname": "smithi196", "labels": [], "status": ""}] 2024-02-23T00:34:44.674 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.672+0000 7f3dc350c700 10 librados: watch_flush enter 2024-02-23T00:34:44.674 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.672+0000 7f3dc350c700 10 librados: watch_flush exit 2024-02-23T00:34:44.674 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.672+0000 7f3dc350c700 20 client.14172.objecter shutdown clearing up homeless session... 2024-02-23T00:34:44.674 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.672+0000 7f3dc350c700 10 client.14172.objecter successfully canceled tick 2024-02-23T00:34:44.675 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.672+0000 7f3dc350c700 1 -- 172.21.15.196:0/3456610484 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f3da80340a0 msgr2=0x7f3da8036560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:34:44.675 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.672+0000 7f3dc350c700 1 --2- 172.21.15.196:0/3456610484 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f3da80340a0 0x7f3da8036560 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f3db8009780 tx=0x7f3db8006cc0).stop 2024-02-23T00:34:44.675 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.672+0000 7f3dc350c700 1 -- 172.21.15.196:0/3456610484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3dbc0f75c0 msgr2=0x7f3dbc106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:34:44.675 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.672+0000 7f3dc350c700 1 --2- 172.21.15.196:0/3456610484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3dbc0f75c0 0x7f3dbc106ea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f3dac0097f0 tx=0x7f3dac009cb0).stop 2024-02-23T00:34:44.675 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.672+0000 7f3dc350c700 1 -- 172.21.15.196:0/3456610484 shutdown_connections 2024-02-23T00:34:44.675 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.672+0000 7f3dc350c700 1 --2- 172.21.15.196:0/3456610484 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f3da80340a0 0x7f3da8036560 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:34:44.675 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.672+0000 7f3dc350c700 1 --2- 172.21.15.196:0/3456610484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3dbc0f75c0 0x7f3dbc106ea0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:34:44.675 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.672+0000 7f3dc350c700 1 -- 172.21.15.196:0/3456610484 >> 172.21.15.196:0/3456610484 conn(0x7f3dbc0f2d40 msgr2=0x7f3dbc0f3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:34:44.675 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.672+0000 7f3dc350c700 1 -- 172.21.15.196:0/3456610484 shutdown_connections 2024-02-23T00:34:44.675 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.672+0000 7f3dc350c700 1 -- 172.21.15.196:0/3456610484 wait complete. 2024-02-23T00:34:44.675 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:44.672+0000 7f3dc350c700 1 librados: shutdown 2024-02-23T00:34:45.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:43.986381+0000 mon.a (mon.0) 106 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:34:45.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:34:44.073855+0000 mgr.y (mgr.14146) 26 : cluster [DBG] pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:34:45.870 INFO:tasks.cephadm:Setting crush tunables to jewel 2024-02-23T00:34:45.871 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd crush tunables jewel 2024-02-23T00:34:46.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:44.671260+0000 mgr.y (mgr.14146) 27 : audit [DBG] from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:34:47.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:34:46.074157+0000 mgr.y (mgr.14146) 28 : cluster [DBG] pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:34:48.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.014+0000 7f552a969700 1 -- 172.21.15.47:0/924195761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55240fd620 msgr2=0x7f55240fda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:34:48.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.014+0000 7f552a969700 1 --2- 172.21.15.47:0/924195761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55240fd620 0x7f55240fda00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f5518004770 tx=0x7f5518009cb0).stop 2024-02-23T00:34:48.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.014+0000 7f552a969700 1 -- 172.21.15.47:0/924195761 shutdown_connections 2024-02-23T00:34:48.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.014+0000 7f552a969700 1 --2- 172.21.15.47:0/924195761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55240fd620 0x7f55240fda00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:34:48.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.014+0000 7f552a969700 1 -- 172.21.15.47:0/924195761 >> 172.21.15.47:0/924195761 conn(0x7f55240f3220 msgr2=0x7f55240f5640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:34:48.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.015+0000 7f552a969700 1 -- 172.21.15.47:0/924195761 shutdown_connections 2024-02-23T00:34:48.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.015+0000 7f552a969700 1 -- 172.21.15.47:0/924195761 wait complete. 2024-02-23T00:34:48.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.015+0000 7f552a969700 1 librados: starting msgr at 2024-02-23T00:34:48.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.015+0000 7f552a969700 1 librados: starting objecter 2024-02-23T00:34:48.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.015+0000 7f552a969700 1 Processor -- start 2024-02-23T00:34:48.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.015+0000 7f552a969700 1 -- start start 2024-02-23T00:34:48.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.015+0000 7f552a969700 1 librados: setting wanted keys 2024-02-23T00:34:48.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.015+0000 7f552a969700 1 librados: calling monclient init 2024-02-23T00:34:48.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.015+0000 7f552a969700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55240fd620 0x7f552406b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:34:48.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.015+0000 7f552a969700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552406b5c0 con 0x7f55240fd620 2024-02-23T00:34:48.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.016+0000 7f5523fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55240fd620 0x7f552406b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:34:48.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.016+0000 7f5523fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55240fd620 0x7f552406b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35786/0 (socket says 172.21.15.47:35786) 2024-02-23T00:34:48.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.016+0000 7f5523fff700 1 -- 172.21.15.47:0/2637232316 learned_addr learned my addr 172.21.15.47:0/2637232316 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:34:48.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.016+0000 7f5523fff700 1 -- 172.21.15.47:0/2637232316 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5518005040 con 0x7f55240fd620 2024-02-23T00:34:48.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.016+0000 7f5523fff700 1 --2- 172.21.15.47:0/2637232316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55240fd620 0x7f552406b0d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f55180097f0 tx=0x7f5518009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:34:48.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.016+0000 7f55217fa700 10 client.?.objecter ms_handle_connect 0x7f55240fd620 2024-02-23T00:34:48.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.016+0000 7f55217fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:34:48.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.016+0000 7f55217fa700 1 -- 172.21.15.47:0/2637232316 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f551800e070 con 0x7f55240fd620 2024-02-23T00:34:48.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.016+0000 7f55217fa700 1 -- 172.21.15.47:0/2637232316 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f5518013e30 con 0x7f55240fd620 2024-02-23T00:34:48.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.016+0000 7f552a969700 1 -- 172.21.15.47:0/2637232316 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f552406b7c0 con 0x7f55240fd620 2024-02-23T00:34:48.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.016+0000 7f55217fa700 1 -- 172.21.15.47:0/2637232316 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5518018070 con 0x7f55240fd620 2024-02-23T00:34:48.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.016+0000 7f552a969700 10 client.14174.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:34:48.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.016+0000 7f552a969700 1 -- 172.21.15.47:0/2637232316 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f552406bc50 con 0x7f55240fd620 2024-02-23T00:34:48.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.016+0000 7f552a969700 1 librados: init done 2024-02-23T00:34:48.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.017+0000 7f55217fa700 1 -- 172.21.15.47:0/2637232316 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 38673+0+0 (secure 0 0 0) 0x7f55180263f0 con 0x7f55240fd620 2024-02-23T00:34:48.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.017+0000 7f55217fa700 1 --2- 172.21.15.47:0/2637232316 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f550c034050 0x7f550c036510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:34:48.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.017+0000 7f55217fa700 1 -- 172.21.15.47:0/2637232316 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f551801d420 con 0x7f55240fd620 2024-02-23T00:34:48.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.017+0000 7f55217fa700 10 client.14174.objecter ms_dispatch 0x7f5524001550 osd_map(3..3 src has 1..3) v4 2024-02-23T00:34:48.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.017+0000 7f55217fa700 3 client.14174.objecter handle_osd_map got epochs [3,3] > 0 2024-02-23T00:34:48.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.017+0000 7f55217fa700 3 client.14174.objecter handle_osd_map decoding full epoch 3 2024-02-23T00:34:48.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.017+0000 7f55217fa700 20 client.14174.objecter dump_active .. 0 homeless 2024-02-23T00:34:48.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.017+0000 7f55237fe700 1 --2- 172.21.15.47:0/2637232316 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f550c034050 0x7f550c036510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:34:48.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.017+0000 7f552a969700 1 -- 172.21.15.47:0/2637232316 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f552405f150 con 0x7f55240fd620 2024-02-23T00:34:48.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.018+0000 7f55217fa700 10 client.14174.objecter ms_handle_connect 0x7f550c034050 2024-02-23T00:34:48.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.018+0000 7f55237fe700 1 --2- 172.21.15.47:0/2637232316 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f550c034050 0x7f550c036510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f5514009780 tx=0x7f5514006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:34:48.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.022+0000 7f55217fa700 1 -- 172.21.15.47:0/2637232316 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f551801de60 con 0x7f55240fd620 2024-02-23T00:34:48.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:48.201+0000 7f552a969700 1 -- 172.21.15.47:0/2637232316 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "jewel"} v 0) v1 -- 0x7f5524046ce0 con 0x7f55240fd620 2024-02-23T00:34:49.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:49.128+0000 7f55217fa700 1 -- 172.21.15.47:0/2637232316 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "jewel"}]=0 adjusted tunables profile to jewel v4) v1 ==== 120+0+0 (secure 0 0 0) 0x7f5518022210 con 0x7f55240fd620 2024-02-23T00:34:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:49.130+0000 7f552a969700 10 librados: watch_flush enter 2024-02-23T00:34:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:49.131+0000 7f552a969700 10 librados: watch_flush exit 2024-02-23T00:34:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:49.131+0000 7f552a969700 20 client.14174.objecter shutdown clearing up homeless session... 2024-02-23T00:34:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:49.131+0000 7f552a969700 10 client.14174.objecter successfully canceled tick 2024-02-23T00:34:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:49.131+0000 7f552a969700 1 -- 172.21.15.47:0/2637232316 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f550c034050 msgr2=0x7f550c036510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:34:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:49.131+0000 7f552a969700 1 --2- 172.21.15.47:0/2637232316 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f550c034050 0x7f550c036510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f5514009780 tx=0x7f5514006cc0).stop 2024-02-23T00:34:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:49.131+0000 7f552a969700 1 -- 172.21.15.47:0/2637232316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55240fd620 msgr2=0x7f552406b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:34:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:49.131+0000 7f552a969700 1 --2- 172.21.15.47:0/2637232316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55240fd620 0x7f552406b0d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f55180097f0 tx=0x7f5518009cb0).stop 2024-02-23T00:34:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:49.131+0000 7f552a969700 1 -- 172.21.15.47:0/2637232316 shutdown_connections 2024-02-23T00:34:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:49.131+0000 7f552a969700 1 --2- 172.21.15.47:0/2637232316 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f550c034050 0x7f550c036510 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:34:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:49.131+0000 7f552a969700 1 --2- 172.21.15.47:0/2637232316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55240fd620 0x7f552406b0d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:34:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:49.131+0000 7f552a969700 1 -- 172.21.15.47:0/2637232316 >> 172.21.15.47:0/2637232316 conn(0x7f55240f3220 msgr2=0x7f55240f3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:34:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:49.131+0000 7f552a969700 1 -- 172.21.15.47:0/2637232316 shutdown_connections 2024-02-23T00:34:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:49.131+0000 7f552a969700 1 -- 172.21.15.47:0/2637232316 wait complete. 2024-02-23T00:34:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:34:49.131+0000 7f552a969700 1 librados: shutdown 2024-02-23T00:34:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:adjusted tunables profile to jewel 2024-02-23T00:34:49.542 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:34:48.074385+0000 mgr.y (mgr.14146) 29 : cluster [DBG] pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:34:49.542 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:48.202834+0000 mon.a (mon.0) 107 : audit [INF] from='client.? 172.21.15.47:0/2637232316' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "jewel"}]: dispatch 2024-02-23T00:34:49.542 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:48.271400+0000 mon.a (mon.0) 108 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:34:49.542 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:48.801074+0000 mon.a (mon.0) 109 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:34:49.542 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:48.801850+0000 mon.a (mon.0) 110 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:34:49.542 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:48.803012+0000 mon.a (mon.0) 111 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:34:49.542 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:48.803969+0000 mon.a (mon.0) 112 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:34:49.542 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:48.861967+0000 mon.a (mon.0) 113 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:34:49.542 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:48.864648+0000 mon.a (mon.0) 114 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:34:49.820 INFO:tasks.cephadm:Adding mon.a on smithi047 2024-02-23T00:34:49.820 INFO:tasks.cephadm:Adding mon.c on smithi047 2024-02-23T00:34:49.820 INFO:tasks.cephadm:Adding mon.b on smithi196 2024-02-23T00:34:49.820 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph orch apply mon '3;smithi047:172.21.15.47=a;smithi047:[v2:172.21.15.47:3301,v1:172.21.15.47:6790]=c;smithi196:172.21.15.196=b' 2024-02-23T00:34:50.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:34:48.805350+0000 mgr.y (mgr.14146) 30 : cephadm [INF] Updating smithi196:/etc/ceph/ceph.conf 2024-02-23T00:34:50.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:34:48.834674+0000 mgr.y (mgr.14146) 31 : cephadm [INF] Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-02-23T00:34:50.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:49.129659+0000 mon.a (mon.0) 115 : audit [INF] from='client.? 172.21.15.47:0/2637232316' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "jewel"}]': finished 2024-02-23T00:34:50.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:34:49.129712+0000 mon.a (mon.0) 116 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-23T00:34:51.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:34:50.074746+0000 mgr.y (mgr.14146) 32 : cluster [DBG] pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:34:51.733 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.730+0000 7f60ff328700 1 -- 172.21.15.196:0/2546171942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60f80fc550 msgr2=0x7f60f80fc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:34:51.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.730+0000 7f60ff328700 1 --2- 172.21.15.196:0/2546171942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60f80fc550 0x7f60f80fc930 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f60f0009ae0 tx=0x7f60f0004d10).stop 2024-02-23T00:34:51.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.730+0000 7f60ff328700 1 -- 172.21.15.196:0/2546171942 shutdown_connections 2024-02-23T00:34:51.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.730+0000 7f60ff328700 1 --2- 172.21.15.196:0/2546171942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60f80fc550 0x7f60f80fc930 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:34:51.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.730+0000 7f60ff328700 1 -- 172.21.15.196:0/2546171942 >> 172.21.15.196:0/2546171942 conn(0x7f60f806c440 msgr2=0x7f60f806c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:34:51.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.730+0000 7f60ff328700 1 -- 172.21.15.196:0/2546171942 shutdown_connections 2024-02-23T00:34:51.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.730+0000 7f60ff328700 1 -- 172.21.15.196:0/2546171942 wait complete. 2024-02-23T00:34:51.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.730+0000 7f60ff328700 1 librados: starting msgr at 2024-02-23T00:34:51.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.730+0000 7f60ff328700 1 librados: starting objecter 2024-02-23T00:34:51.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.730+0000 7f60ff328700 1 Processor -- start 2024-02-23T00:34:51.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.730+0000 7f60ff328700 1 -- start start 2024-02-23T00:34:51.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.730+0000 7f60ff328700 1 librados: setting wanted keys 2024-02-23T00:34:51.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.730+0000 7f60ff328700 1 librados: calling monclient init 2024-02-23T00:34:51.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.730+0000 7f60ff328700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60f80fc550 0x7f60f810cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:34:51.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.730+0000 7f60ff328700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f810d3f0 con 0x7f60f80fc550 2024-02-23T00:34:51.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.731+0000 7f60fd0c4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60f80fc550 0x7f60f810cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:34:51.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.731+0000 7f60fd0c4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60f80fc550 0x7f60f810cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.196:37426/0 (socket says 172.21.15.196:37426) 2024-02-23T00:34:51.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.731+0000 7f60fd0c4700 1 -- 172.21.15.196:0/2191268577 learned_addr learned my addr 172.21.15.196:0/2191268577 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-02-23T00:34:51.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.731+0000 7f60fd0c4700 1 -- 172.21.15.196:0/2191268577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f00097c0 con 0x7f60f80fc550 2024-02-23T00:34:51.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.731+0000 7f60fd0c4700 1 --2- 172.21.15.196:0/2191268577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60f80fc550 0x7f60f810cf00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f60f0003380 tx=0x7f60f0004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:34:51.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.731+0000 7f60ee7fc700 10 client.?.objecter ms_handle_connect 0x7f60f80fc550 2024-02-23T00:34:51.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.731+0000 7f60ee7fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:34:51.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.731+0000 7f60ee7fc700 1 -- 172.21.15.196:0/2191268577 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60f000e070 con 0x7f60f80fc550 2024-02-23T00:34:51.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.731+0000 7f60ff328700 1 -- 172.21.15.196:0/2191268577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f810d5f0 con 0x7f60f80fc550 2024-02-23T00:34:51.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.731+0000 7f60ee7fc700 1 -- 172.21.15.196:0/2191268577 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f60f000bcc0 con 0x7f60f80fc550 2024-02-23T00:34:51.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.731+0000 7f60ee7fc700 1 -- 172.21.15.196:0/2191268577 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60f0018070 con 0x7f60f80fc550 2024-02-23T00:34:51.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.731+0000 7f60ff328700 10 client.14176.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:34:51.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.731+0000 7f60ff328700 1 -- 172.21.15.196:0/2191268577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f810da00 con 0x7f60f80fc550 2024-02-23T00:34:51.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.731+0000 7f60ff328700 1 librados: init done 2024-02-23T00:34:51.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.732+0000 7f60ee7fc700 1 -- 172.21.15.196:0/2191268577 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 38673+0+0 (secure 0 0 0) 0x7f60f001d420 con 0x7f60f80fc550 2024-02-23T00:34:51.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.732+0000 7f60ff328700 1 -- 172.21.15.196:0/2191268577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f805f1f0 con 0x7f60f80fc550 2024-02-23T00:34:51.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.732+0000 7f60ee7fc700 1 --2- 172.21.15.196:0/2191268577 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f60e4034050 0x7f60e4036510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:34:51.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.732+0000 7f60ee7fc700 1 -- 172.21.15.196:0/2191268577 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f60f0044390 con 0x7f60f80fc550 2024-02-23T00:34:51.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.732+0000 7f60ee7fc700 10 client.14176.objecter ms_dispatch 0x7f60f8001550 osd_map(4..4 src has 1..4) v4 2024-02-23T00:34:51.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.732+0000 7f60ee7fc700 3 client.14176.objecter handle_osd_map got epochs [4,4] > 0 2024-02-23T00:34:51.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.732+0000 7f60ee7fc700 3 client.14176.objecter handle_osd_map decoding full epoch 4 2024-02-23T00:34:51.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.732+0000 7f60ee7fc700 20 client.14176.objecter dump_active .. 0 homeless 2024-02-23T00:34:51.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.733+0000 7f60fc8c3700 1 --2- 172.21.15.196:0/2191268577 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f60e4034050 0x7f60e4036510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:34:51.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.733+0000 7f60ee7fc700 10 client.14176.objecter ms_handle_connect 0x7f60e4034050 2024-02-23T00:34:51.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.733+0000 7f60fc8c3700 1 --2- 172.21.15.196:0/2191268577 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f60e4034050 0x7f60e4036510 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f60e8009780 tx=0x7f60e8006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:34:51.739 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.738+0000 7f60ee7fc700 1 -- 172.21.15.196:0/2191268577 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f60f00118f0 con 0x7f60f80fc550 2024-02-23T00:34:51.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.910+0000 7f60ff328700 1 -- 172.21.15.196:0/2191268577 --> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi047:172.21.15.47=a;smithi047:[v2:172.21.15.47:3301,v1:172.21.15.47:6790]=c;smithi196:172.21.15.196=b", "target": ["mon-mgr", ""]}) v1 -- 0x7f60f8046d50 con 0x7f60e4034050 2024-02-23T00:34:51.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.917+0000 7f60ee7fc700 1 -- 172.21.15.196:0/2191268577 <== mgr.14146 v2:172.21.15.47:6800/622994871 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f60f8046d50 con 0x7f60e4034050 2024-02-23T00:34:51.918 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mon update... 2024-02-23T00:34:51.920 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.918+0000 7f60ff328700 10 librados: watch_flush enter 2024-02-23T00:34:51.920 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.918+0000 7f60ff328700 10 librados: watch_flush exit 2024-02-23T00:34:51.921 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.918+0000 7f60ff328700 20 client.14176.objecter shutdown clearing up homeless session... 2024-02-23T00:34:51.921 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.918+0000 7f60ff328700 10 client.14176.objecter successfully canceled tick 2024-02-23T00:34:51.921 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.918+0000 7f60ff328700 1 -- 172.21.15.196:0/2191268577 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f60e4034050 msgr2=0x7f60e4036510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:34:51.921 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.918+0000 7f60ff328700 1 --2- 172.21.15.196:0/2191268577 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f60e4034050 0x7f60e4036510 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f60e8009780 tx=0x7f60e8006cc0).stop 2024-02-23T00:34:51.921 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.918+0000 7f60ff328700 1 -- 172.21.15.196:0/2191268577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60f80fc550 msgr2=0x7f60f810cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:34:51.921 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.918+0000 7f60ff328700 1 --2- 172.21.15.196:0/2191268577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60f80fc550 0x7f60f810cf00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f60f0003380 tx=0x7f60f0004d10).stop 2024-02-23T00:34:51.921 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.918+0000 7f60ff328700 1 -- 172.21.15.196:0/2191268577 shutdown_connections 2024-02-23T00:34:51.921 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.918+0000 7f60ff328700 1 --2- 172.21.15.196:0/2191268577 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f60e4034050 0x7f60e4036510 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:34:51.921 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.918+0000 7f60ff328700 1 --2- 172.21.15.196:0/2191268577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60f80fc550 0x7f60f810cf00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:34:51.921 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.918+0000 7f60ff328700 1 -- 172.21.15.196:0/2191268577 >> 172.21.15.196:0/2191268577 conn(0x7f60f806c440 msgr2=0x7f60f8060500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:34:51.921 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.918+0000 7f60ff328700 1 -- 172.21.15.196:0/2191268577 shutdown_connections 2024-02-23T00:34:51.922 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.918+0000 7f60ff328700 1 -- 172.21.15.196:0/2191268577 wait complete. 2024-02-23T00:34:51.922 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:51.918+0000 7f60ff328700 1 librados: shutdown 2024-02-23T00:34:52.686 DEBUG:teuthology.orchestra.run.smithi047:mon.c> sudo journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@mon.c.service 2024-02-23T00:34:52.689 DEBUG:teuthology.orchestra.run.smithi196:mon.b> sudo journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@mon.b.service 2024-02-23T00:34:52.691 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-23T00:34:52.691 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph mon dump -f json 2024-02-23T00:34:52.724 INFO:journalctl@ceph.mon.b.smithi196.stdout:-- Logs begin at Fri 2024-02-23 00:13:21 UTC. -- 2024-02-23T00:34:52.726 INFO:journalctl@ceph.mon.c.smithi047.stdout:-- Logs begin at Fri 2024-02-23 00:13:17 UTC. -- 2024-02-23T00:34:53.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:51.912270+0000 mgr.y (mgr.14146) 33 : audit [DBG] from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi047:172.21.15.47=a;smithi047:[v2:172.21.15.47:3301,v1:172.21.15.47:6790]=c;smithi196:172.21.15.196=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:34:53.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:34:51.914698+0000 mgr.y (mgr.14146) 34 : cephadm [INF] Saving service mon spec with placement smithi047:172.21.15.47=a;smithi047:[v2:172.21.15.47:3301,v1:172.21.15.47:6790]=c;smithi196:172.21.15.196=b;count:3 2024-02-23T00:34:53.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:51.917377+0000 mon.a (mon.0) 117 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:34:53.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:51.937191+0000 mon.a (mon.0) 118 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:34:53.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:51.938886+0000 mon.a (mon.0) 119 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:34:53.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:51.939738+0000 mon.a (mon.0) 120 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:34:53.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:51.943167+0000 mon.a (mon.0) 121 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:34:53.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:51.944912+0000 mon.a (mon.0) 122 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-23T00:34:53.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:51.945704+0000 mon.a (mon.0) 123 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:34:53.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:34:51.946449+0000 mgr.y (mgr.14146) 35 : cephadm [INF] Deploying daemon mon.b on smithi196 2024-02-23T00:34:53.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:34:52.075097+0000 mgr.y (mgr.14146) 36 : cluster [DBG] pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:34:55.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:34:54.075463+0000 mgr.y (mgr.14146) 37 : cluster [DBG] pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:34:57.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:34:56.075900+0000 mgr.y (mgr.14146) 38 : cluster [DBG] pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:34:58.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.185+0000 7f3ac87ca700 1 -- 172.21.15.196:0/3147120198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ac00f6480 msgr2=0x7f3ac00f6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:34:58.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.185+0000 7f3ac87ca700 1 --2- 172.21.15.196:0/3147120198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ac00f6480 0x7f3ac00f6860 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f3abc004770 tx=0x7f3abc009cb0).stop 2024-02-23T00:34:58.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.185+0000 7f3ac87ca700 1 -- 172.21.15.196:0/3147120198 shutdown_connections 2024-02-23T00:34:58.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.185+0000 7f3ac87ca700 1 --2- 172.21.15.196:0/3147120198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ac00f6480 0x7f3ac00f6860 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:34:58.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.185+0000 7f3ac87ca700 1 -- 172.21.15.196:0/3147120198 >> 172.21.15.196:0/3147120198 conn(0x7f3ac006c440 msgr2=0x7f3ac006c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:34:58.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.185+0000 7f3ac87ca700 1 -- 172.21.15.196:0/3147120198 shutdown_connections 2024-02-23T00:34:58.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.185+0000 7f3ac87ca700 1 -- 172.21.15.196:0/3147120198 wait complete. 2024-02-23T00:34:58.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.186+0000 7f3ac87ca700 1 librados: starting msgr at 2024-02-23T00:34:58.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.186+0000 7f3ac87ca700 1 librados: starting objecter 2024-02-23T00:34:58.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.186+0000 7f3ac87ca700 1 Processor -- start 2024-02-23T00:34:58.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.186+0000 7f3ac87ca700 1 -- start start 2024-02-23T00:34:58.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.186+0000 7f3ac87ca700 1 librados: setting wanted keys 2024-02-23T00:34:58.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.186+0000 7f3ac87ca700 1 librados: calling monclient init 2024-02-23T00:34:58.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.186+0000 7f3ac87ca700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ac00f6480 0x7f3ac010f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:34:58.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.186+0000 7f3ac87ca700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac010f5c0 con 0x7f3ac00f6480 2024-02-23T00:34:58.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.187+0000 7f3ac6566700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ac00f6480 0x7f3ac010f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:34:58.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.187+0000 7f3ac6566700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ac00f6480 0x7f3ac010f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.196:37442/0 (socket says 172.21.15.196:37442) 2024-02-23T00:34:58.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.187+0000 7f3ac6566700 1 -- 172.21.15.196:0/1996023354 learned_addr learned my addr 172.21.15.196:0/1996023354 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-02-23T00:34:58.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.187+0000 7f3ac6566700 1 -- 172.21.15.196:0/1996023354 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3abc005040 con 0x7f3ac00f6480 2024-02-23T00:34:58.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.187+0000 7f3ac6566700 1 --2- 172.21.15.196:0/1996023354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ac00f6480 0x7f3ac010f0d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f3abc0097f0 tx=0x7f3abc009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:34:58.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.187+0000 7f3ab77fe700 10 client.?.objecter ms_handle_connect 0x7f3ac00f6480 2024-02-23T00:34:58.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.187+0000 7f3ab77fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:34:58.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.187+0000 7f3ab77fe700 1 -- 172.21.15.196:0/1996023354 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3abc00e070 con 0x7f3ac00f6480 2024-02-23T00:34:58.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.187+0000 7f3ab77fe700 1 -- 172.21.15.196:0/1996023354 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f3abc013e30 con 0x7f3ac00f6480 2024-02-23T00:34:58.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.187+0000 7f3ab77fe700 1 -- 172.21.15.196:0/1996023354 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3abc018070 con 0x7f3ac00f6480 2024-02-23T00:34:58.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.187+0000 7f3ac87ca700 1 -- 172.21.15.196:0/1996023354 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac010f7c0 con 0x7f3ac00f6480 2024-02-23T00:34:58.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.188+0000 7f3ac87ca700 10 client.14178.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:34:58.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.188+0000 7f3ac87ca700 1 -- 172.21.15.196:0/1996023354 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac010fc50 con 0x7f3ac00f6480 2024-02-23T00:34:58.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.188+0000 7f3ac87ca700 1 librados: init done 2024-02-23T00:34:58.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.189+0000 7f3ab77fe700 1 -- 172.21.15.196:0/1996023354 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 38673+0+0 (secure 0 0 0) 0x7f3abc012cf0 con 0x7f3ac00f6480 2024-02-23T00:34:58.305 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.189+0000 7f3ab77fe700 1 --2- 172.21.15.196:0/1996023354 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f3aac0340a0 0x7f3aac036560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:34:58.305 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.189+0000 7f3ab77fe700 1 -- 172.21.15.196:0/1996023354 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f3abc043cb0 con 0x7f3ac00f6480 2024-02-23T00:34:58.305 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.189+0000 7f3ab77fe700 10 client.14178.objecter ms_dispatch 0x7f3ac0001550 osd_map(4..4 src has 1..4) v4 2024-02-23T00:34:58.305 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.189+0000 7f3ab77fe700 3 client.14178.objecter handle_osd_map got epochs [4,4] > 0 2024-02-23T00:34:58.305 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.189+0000 7f3ab77fe700 3 client.14178.objecter handle_osd_map decoding full epoch 4 2024-02-23T00:34:58.305 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.189+0000 7f3ab77fe700 20 client.14178.objecter dump_active .. 0 homeless 2024-02-23T00:34:58.305 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.189+0000 7f3ac87ca700 1 -- 172.21.15.196:0/1996023354 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac005f1f0 con 0x7f3ac00f6480 2024-02-23T00:34:58.305 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.189+0000 7f3ac5d65700 1 --2- 172.21.15.196:0/1996023354 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f3aac0340a0 0x7f3aac036560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:34:58.305 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.190+0000 7f3ab77fe700 10 client.14178.objecter ms_handle_connect 0x7f3aac0340a0 2024-02-23T00:34:58.305 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.189+0000 7f3ac5d65700 1 --2- 172.21.15.196:0/1996023354 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f3aac0340a0 0x7f3aac036560 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f3ab0009780 tx=0x7f3ab0006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:34:58.305 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.194+0000 7f3ab77fe700 1 -- 172.21.15.196:0/1996023354 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f3abc0139a0 con 0x7f3ac00f6480 2024-02-23T00:34:58.434 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.432+0000 7f3ac87ca700 1 -- 172.21.15.196:0/1996023354 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3ac005a310 con 0x7f3ac00f6480 2024-02-23T00:34:58.435 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.433+0000 7f3ab77fe700 1 -- 172.21.15.196:0/1996023354 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+723 (secure 0 0 0) 0x7f3abc02a020 con 0x7f3ac00f6480 2024-02-23T00:34:58.435 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:34:58.435 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"fce2617e-d1e2-11ee-95c0-87774f69a715","modified":"2024-02-23T00:32:52.273607Z","created":"2024-02-23T00:32:52.273607Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-23T00:34:58.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.435+0000 7f3ac87ca700 10 librados: watch_flush enter 2024-02-23T00:34:58.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.435+0000 7f3ac87ca700 10 librados: watch_flush exit 2024-02-23T00:34:58.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.435+0000 7f3ac87ca700 20 client.14178.objecter shutdown clearing up homeless session... 2024-02-23T00:34:58.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.435+0000 7f3ac87ca700 10 client.14178.objecter successfully canceled tick 2024-02-23T00:34:58.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.435+0000 7f3ac87ca700 1 -- 172.21.15.196:0/1996023354 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f3aac0340a0 msgr2=0x7f3aac036560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:34:58.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.435+0000 7f3ac87ca700 1 --2- 172.21.15.196:0/1996023354 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f3aac0340a0 0x7f3aac036560 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f3ab0009780 tx=0x7f3ab0006cc0).stop 2024-02-23T00:34:58.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.435+0000 7f3ac87ca700 1 -- 172.21.15.196:0/1996023354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ac00f6480 msgr2=0x7f3ac010f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:34:58.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.435+0000 7f3ac87ca700 1 --2- 172.21.15.196:0/1996023354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ac00f6480 0x7f3ac010f0d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f3abc0097f0 tx=0x7f3abc009cb0).stop 2024-02-23T00:34:58.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.435+0000 7f3ac87ca700 1 -- 172.21.15.196:0/1996023354 shutdown_connections 2024-02-23T00:34:58.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.435+0000 7f3ac87ca700 1 --2- 172.21.15.196:0/1996023354 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f3aac0340a0 0x7f3aac036560 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:34:58.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.435+0000 7f3ac87ca700 1 --2- 172.21.15.196:0/1996023354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ac00f6480 0x7f3ac010f0d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:34:58.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.435+0000 7f3ac87ca700 1 -- 172.21.15.196:0/1996023354 >> 172.21.15.196:0/1996023354 conn(0x7f3ac006c440 msgr2=0x7f3ac00f4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:34:58.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.435+0000 7f3ac87ca700 1 -- 172.21.15.196:0/1996023354 shutdown_connections 2024-02-23T00:34:58.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.435+0000 7f3ac87ca700 1 -- 172.21.15.196:0/1996023354 wait complete. 2024-02-23T00:34:58.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:34:58.435+0000 7f3ac87ca700 1 librados: shutdown 2024-02-23T00:34:58.438 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-02-23T00:34:59.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:34:58.076237+0000 mgr.y (mgr.14146) 39 : cluster [DBG] pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:34:59.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:34:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:34:58.434538+0000 mon.a (mon.0) 124 : audit [DBG] from='client.? 172.21.15.196:0/1996023354' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-23T00:35:00.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 systemd[1]: Starting Ceph mon.b for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T00:35:01.011 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-23T00:35:01.011 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph mon dump -f json 2024-02-23T00:35:01.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.814+0000 7fd407e6e880 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-23T00:35:01.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.815+0000 7fd407e6e880 0 ceph version 16.2.14-550-geb66ed92 (eb66ed921e744301e6863be6353618d63967ea59) pacific (stable), process ceph-mon, pid 7 2024-02-23T00:35:01.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.815+0000 7fd407e6e880 0 pidfile_write: ignore empty --pid-file 2024-02-23T00:35:01.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.819+0000 7fd407e6e880 0 load: jerasure load: lrc load: isa 2024-02-23T00:35:01.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: RocksDB version: 6.8.1 2024-02-23T00:35:01.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: 2024-02-23T00:35:01.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-23T00:35:01.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Compile date Feb 20 2024 2024-02-23T00:35:01.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: DB SUMMARY 2024-02-23T00:35:01.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: 2024-02-23T00:35:01.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: CURRENT file: CURRENT 2024-02-23T00:35:01.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: 2024-02-23T00:35:01.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: IDENTITY file: IDENTITY 2024-02-23T00:35:01.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: 2024-02-23T00:35:01.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: MANIFEST file: MANIFEST-000001 size: 13 Bytes 2024-02-23T00:35:01.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: 2024-02-23T00:35:01.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-02-23T00:35:01.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: 2024-02-23T00:35:01.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000003.log size: 511 ; 2024-02-23T00:35:01.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: 2024-02-23T00:35:01.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.error_if_exists: 0 2024-02-23T00:35:01.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.create_if_missing: 0 2024-02-23T00:35:01.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.paranoid_checks: 1 2024-02-23T00:35:01.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.env: 0x5596c5f62080 2024-02-23T00:35:01.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.fs: Posix File System 2024-02-23T00:35:01.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.info_log: 0x5596c6882260 2024-02-23T00:35:01.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-23T00:35:01.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.statistics: (nil) 2024-02-23T00:35:01.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.use_fsync: 0 2024-02-23T00:35:01.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.max_log_file_size: 0 2024-02-23T00:35:01.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-23T00:35:01.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-23T00:35:01.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-23T00:35:01.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-23T00:35:01.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.allow_fallocate: 1 2024-02-23T00:35:01.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-23T00:35:01.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-23T00:35:01.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.use_direct_reads: 0 2024-02-23T00:35:01.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-23T00:35:01.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.create_missing_column_families: 0 2024-02-23T00:35:01.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.db_log_dir: 2024-02-23T00:35:01.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-b/store.db 2024-02-23T00:35:01.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-23T00:35:01.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.max_subcompactions: 1 2024-02-23T00:35:01.197 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.max_background_flushes: -1 2024-02-23T00:35:01.197 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-23T00:35:01.199 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-23T00:35:01.199 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-23T00:35:01.199 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-23T00:35:01.199 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-23T00:35:01.199 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.advise_random_on_open: 1 2024-02-23T00:35:01.199 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-23T00:35:01.199 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.write_buffer_manager: 0x5596c75743c0 2024-02-23T00:35:01.199 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-23T00:35:01.199 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-23T00:35:01.199 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-23T00:35:01.200 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-23T00:35:01.200 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.rate_limiter: (nil) 2024-02-23T00:35:01.200 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-23T00:35:01.200 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-23T00:35:01.200 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-23T00:35:01.200 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-23T00:35:01.200 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.unordered_write: 0 2024-02-23T00:35:01.200 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-23T00:35:01.200 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-23T00:35:01.200 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-23T00:35:01.200 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-23T00:35:01.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.row_cache: None 2024-02-23T00:35:01.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.wal_filter: None 2024-02-23T00:35:01.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-23T00:35:01.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-23T00:35:01.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.preserve_deletes: 0 2024-02-23T00:35:01.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.two_write_queues: 0 2024-02-23T00:35:01.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.manual_wal_flush: 0 2024-02-23T00:35:01.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.atomic_flush: 0 2024-02-23T00:35:01.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-23T00:35:01.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-23T00:35:01.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-23T00:35:01.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.log_readahead_size: 0 2024-02-23T00:35:01.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-23T00:35:01.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.max_background_jobs: 2 2024-02-23T00:35:01.202 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.max_background_compactions: -1 2024-02-23T00:35:01.202 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-23T00:35:01.202 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-23T00:35:01.202 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-23T00:35:01.202 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.max_total_wal_size: 0 2024-02-23T00:35:01.202 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-23T00:35:01.202 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-23T00:35:01.202 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-23T00:35:01.202 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-23T00:35:01.202 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.max_open_files: -1 2024-02-23T00:35:01.202 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.bytes_per_sync: 0 2024-02-23T00:35:01.202 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-23T00:35:01.202 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-23T00:35:01.202 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-23T00:35:01.203 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Compression algorithms supported: 2024-02-23T00:35:01.203 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-23T00:35:01.203 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: kZSTD supported: 0 2024-02-23T00:35:01.203 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: kXpressCompression supported: 0 2024-02-23T00:35:01.203 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-23T00:35:01.203 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: kLZ4Compression supported: 1 2024-02-23T00:35:01.203 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: kBZip2Compression supported: 0 2024-02-23T00:35:01.203 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: kZlibCompression supported: 1 2024-02-23T00:35:01.203 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: kSnappyCompression supported: 1 2024-02-23T00:35:01.203 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-23T00:35:01.203 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000001 2024-02-23T00:35:01.203 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: 2024-02-23T00:35:01.203 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-23T00:35:01.203 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: 2024-02-23T00:35:01.203 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-23T00:35:01.204 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.merge_operator: 2024-02-23T00:35:01.204 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.compaction_filter: None 2024-02-23T00:35:01.204 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.compaction_filter_factory: None 2024-02-23T00:35:01.204 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-23T00:35:01.204 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-23T00:35:01.204 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5596c6827d10) 2024-02-23T00:35:01.204 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cache_index_and_filter_blocks: 1 2024-02-23T00:35:01.204 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-23T00:35:01.204 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-23T00:35:01.204 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: pin_top_level_index_and_filter: 1 2024-02-23T00:35:01.204 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: index_type: 0 2024-02-23T00:35:01.204 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: data_block_index_type: 0 2024-02-23T00:35:01.204 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: index_shortening: 1 2024-02-23T00:35:01.204 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: data_block_hash_table_util_ratio: 0.750000 2024-02-23T00:35:01.205 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: hash_index_allow_collision: 1 2024-02-23T00:35:01.205 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: checksum: 1 2024-02-23T00:35:01.205 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: no_block_cache: 0 2024-02-23T00:35:01.205 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: block_cache: 0x5596c685cf10 2024-02-23T00:35:01.205 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: block_cache_name: BinnedLRUCache 2024-02-23T00:35:01.205 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: block_cache_options: 2024-02-23T00:35:01.205 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: capacity : 536870912 2024-02-23T00:35:01.205 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: num_shard_bits : 4 2024-02-23T00:35:01.205 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: strict_capacity_limit : 0 2024-02-23T00:35:01.205 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: high_pri_pool_ratio: 0.000 2024-02-23T00:35:01.205 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: block_cache_compressed: (nil) 2024-02-23T00:35:01.205 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: persistent_cache: (nil) 2024-02-23T00:35:01.205 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: block_size: 4096 2024-02-23T00:35:01.205 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: block_size_deviation: 10 2024-02-23T00:35:01.206 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: block_restart_interval: 16 2024-02-23T00:35:01.206 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: index_block_restart_interval: 1 2024-02-23T00:35:01.206 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: metadata_block_size: 4096 2024-02-23T00:35:01.206 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: partition_filters: 0 2024-02-23T00:35:01.206 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: use_delta_encoding: 1 2024-02-23T00:35:01.206 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-23T00:35:01.206 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: whole_key_filtering: 1 2024-02-23T00:35:01.206 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: verify_compression: 0 2024-02-23T00:35:01.206 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: read_amp_bytes_per_bit: 0 2024-02-23T00:35:01.206 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: format_version: 2 2024-02-23T00:35:01.206 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: enable_index_compression: 1 2024-02-23T00:35:01.206 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: block_align: 0 2024-02-23T00:35:01.206 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: 2024-02-23T00:35:01.206 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-23T00:35:01.207 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-23T00:35:01.207 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.compression: NoCompression 2024-02-23T00:35:01.207 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-23T00:35:01.207 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-23T00:35:01.207 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-23T00:35:01.207 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.num_levels: 7 2024-02-23T00:35:01.207 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-23T00:35:01.207 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-23T00:35:01.207 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-23T00:35:01.207 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-23T00:35:01.207 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-23T00:35:01.207 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-23T00:35:01.207 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-23T00:35:01.207 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-23T00:35:01.208 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-23T00:35:01.208 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-23T00:35:01.208 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.compression_opts.level: 32767 2024-02-23T00:35:01.208 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-23T00:35:01.208 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-23T00:35:01.208 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-23T00:35:01.208 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.compression_opts.enabled: false 2024-02-23T00:35:01.208 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-23T00:35:01.208 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-23T00:35:01.208 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-23T00:35:01.208 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-23T00:35:01.208 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-23T00:35:01.208 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-23T00:35:01.209 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-23T00:35:01.209 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-23T00:35:01.209 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-23T00:35:01.209 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-23T00:35:01.209 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-23T00:35:01.209 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-23T00:35:01.209 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-23T00:35:01.209 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-23T00:35:01.209 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-23T00:35:01.209 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-23T00:35:01.209 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-23T00:35:01.209 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.arena_block_size: 4194304 2024-02-23T00:35:01.209 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-23T00:35:01.209 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-23T00:35:01.210 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-23T00:35:01.210 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-23T00:35:01.210 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-23T00:35:01.210 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-23T00:35:01.210 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-23T00:35:01.210 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-23T00:35:01.210 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-23T00:35:01.210 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-23T00:35:01.210 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-23T00:35:01.210 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-23T00:35:01.210 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-23T00:35:01.210 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-23T00:35:01.210 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.table_properties_collectors: 2024-02-23T00:35:01.211 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.inplace_update_support: 0 2024-02-23T00:35:01.211 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-23T00:35:01.211 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-23T00:35:01.211 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-23T00:35:01.211 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-23T00:35:01.211 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.bloom_locality: 0 2024-02-23T00:35:01.211 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.max_successive_merges: 0 2024-02-23T00:35:01.211 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-23T00:35:01.211 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-23T00:35:01.211 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.force_consistency_checks: 0 2024-02-23T00:35:01.211 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-23T00:35:01.211 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.ttl: 2592000 2024-02-23T00:35:01.211 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.820+0000 7fd407e6e880 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-23T00:35:01.211 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.821+0000 7fd407e6e880 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-23T00:35:01.212 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: 2024-02-23T00:35:01.212 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.821+0000 7fd407e6e880 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 0 2024-02-23T00:35:01.212 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: 2024-02-23T00:35:01.212 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.821+0000 7fd407e6e880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708648500822333, "job": 1, "event": "recovery_started", "log_files": [3]} 2024-02-23T00:35:01.212 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.821+0000 7fd407e6e880 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #3 mode 2 2024-02-23T00:35:01.212 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.821+0000 7fd407e6e880 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-02-23T00:35:01.212 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.821+0000 7fd407e6e880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708648500822942, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 4, "file_size": 1437, "table_properties": {"data_size": 523, "index_size": 40, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1708648500, "oldest_key_time": 3, "file_creation_time": 0}} 2024-02-23T00:35:01.212 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.821+0000 7fd407e6e880 4 rocksdb: [version_set.cc:3826] Creating manifest 5 2024-02-23T00:35:01.212 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: 2024-02-23T00:35:01.212 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.822+0000 7fd407e6e880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708648500823447, "job": 1, "event": "recovery_finished"} 2024-02-23T00:35:01.212 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.823+0000 7fd407e6e880 4 rocksdb: DB pointer 0x5596c6925800 2024-02-23T00:35:01.212 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.823+0000 7fd3eec60700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-02-23T00:35:01.212 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.823+0000 7fd3eec60700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-02-23T00:35:01.213 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: ** DB Stats ** 2024-02-23T00:35:01.213 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-23T00:35:01.213 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-23T00:35:01.213 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-23T00:35:01.213 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-23T00:35:01.213 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-23T00:35:01.213 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-23T00:35:01.213 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-23T00:35:01.213 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: 2024-02-23T00:35:01.213 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: ** Compaction Stats [default] ** 2024-02-23T00:35:01.213 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-23T00:35:01.213 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-23T00:35:01.213 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: L0 1/0 1.40 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.4 0.00 0.00 1 0.001 0 0 2024-02-23T00:35:01.213 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: Sum 1/0 1.40 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.4 0.00 0.00 1 0.001 0 0 2024-02-23T00:35:01.214 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.4 0.00 0.00 1 0.001 0 0 2024-02-23T00:35:01.214 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: 2024-02-23T00:35:01.214 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: ** Compaction Stats [default] ** 2024-02-23T00:35:01.214 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-23T00:35:01.214 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-23T00:35:01.214 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.4 0.00 0.00 1 0.001 0 0 2024-02-23T00:35:01.214 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-23T00:35:01.214 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-23T00:35:01.214 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-23T00:35:01.214 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: AddFile(Total Files): cumulative 0, interval 0 2024-02-23T00:35:01.214 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-23T00:35:01.214 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: AddFile(Keys): cumulative 0, interval 0 2024-02-23T00:35:01.214 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: Cumulative compaction: 0.00 GB write, 0.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-23T00:35:01.214 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: Interval compaction: 0.00 GB write, 0.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-23T00:35:01.214 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-23T00:35:01.215 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: 2024-02-23T00:35:01.215 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: ** File Read Latency Histogram By Level [default] ** 2024-02-23T00:35:01.215 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: 2024-02-23T00:35:01.215 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: ** Compaction Stats [default] ** 2024-02-23T00:35:01.215 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-23T00:35:01.215 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-23T00:35:01.215 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: L0 1/0 1.40 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.4 0.00 0.00 1 0.001 0 0 2024-02-23T00:35:01.215 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: Sum 1/0 1.40 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.4 0.00 0.00 1 0.001 0 0 2024-02-23T00:35:01.215 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-23T00:35:01.215 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: 2024-02-23T00:35:01.215 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: ** Compaction Stats [default] ** 2024-02-23T00:35:01.215 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-23T00:35:01.215 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-23T00:35:01.215 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.4 0.00 0.00 1 0.001 0 0 2024-02-23T00:35:01.216 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-23T00:35:01.216 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-23T00:35:01.216 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-23T00:35:01.216 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: AddFile(Total Files): cumulative 0, interval 0 2024-02-23T00:35:01.216 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-23T00:35:01.216 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: AddFile(Keys): cumulative 0, interval 0 2024-02-23T00:35:01.216 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: Cumulative compaction: 0.00 GB write, 0.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-23T00:35:01.216 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-23T00:35:01.216 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-23T00:35:01.216 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: 2024-02-23T00:35:01.216 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: ** File Read Latency Histogram By Level [default] ** 2024-02-23T00:35:01.216 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: 2024-02-23T00:35:01.216 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.823+0000 7fd407e6e880 0 mon.b does not exist in monmap, will attempt to join an existing cluster 2024-02-23T00:35:01.217 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.823+0000 7fd407e6e880 0 using public_addr v2:172.21.15.196:0/0 -> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] 2024-02-23T00:35:01.217 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.824+0000 7fd407e6e880 0 starting mon.b rank -1 at public addrs [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] at bind addrs [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid fce2617e-d1e2-11ee-95c0-87774f69a715 2024-02-23T00:35:01.217 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.825+0000 7fd407e6e880 1 mon.b@-1(???) e0 preinit fsid fce2617e-d1e2-11ee-95c0-87774f69a715 2024-02-23T00:35:01.217 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.845+0000 7fd3ecc5c700 4 rocksdb: [db_impl/db_impl_write.cc:1668] [default] New memtable created with log file: #9. Immutable memtables: 0. 2024-02-23T00:35:01.217 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: 2024-02-23T00:35:01.217 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.845+0000 7fd3f6c70700 4 rocksdb: (Original Log Time 2024/02/23-00:35:00.847139) [db_impl/db_impl_compaction_flush.cc:2198] Calling FlushMemTableToOutputFile with column family [default], flush slots available 1, compaction slots available 1, flush slots scheduled 1, compaction slots scheduled 0 2024-02-23T00:35:01.217 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.845+0000 7fd3f6c70700 4 rocksdb: [flush_job.cc:321] [default] [JOB 3] Flushing memtable with next log file: 9 2024-02-23T00:35:01.217 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: 2024-02-23T00:35:01.217 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.845+0000 7fd3f6c70700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708648500847182, "job": 3, "event": "flush_started", "num_memtables": 1, "num_entries": 941, "num_deletes": 11, "total_data_size": 3884621, "memory_usage": 3901896, "flush_reason": "Manual Compaction"} 2024-02-23T00:35:01.217 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.845+0000 7fd3f6c70700 4 rocksdb: [flush_job.cc:350] [default] [JOB 3] Level-0 flush table #10: started 2024-02-23T00:35:01.217 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.846+0000 7fd3f1465700 0 mon.b@-1(synchronizing).mds e1 new map 2024-02-23T00:35:01.217 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.846+0000 7fd3f1465700 0 mon.b@-1(synchronizing).mds e1 print_map 2024-02-23T00:35:01.217 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: e1 2024-02-23T00:35:01.217 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: enable_multiple, ever_enabled_multiple: 1,1 2024-02-23T00:35:01.218 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-23T00:35:01.218 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: legacy client fscid: -1 2024-02-23T00:35:01.218 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: 2024-02-23T00:35:01.218 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: No filesystems configured 2024-02-23T00:35:01.218 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: 2024-02-23T00:35:01.218 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.846+0000 7fd3f1465700 1 mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-23T00:35:01.218 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.846+0000 7fd3f1465700 1 mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-23T00:35:01.218 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.846+0000 7fd3f1465700 1 mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-23T00:35:01.218 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.846+0000 7fd3f1465700 1 mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-23T00:35:01.218 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.846+0000 7fd3f1465700 1 mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-23T00:35:01.218 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.847+0000 7fd3f1465700 1 mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-23T00:35:01.218 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.847+0000 7fd3f1465700 0 mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-23T00:35:01.218 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.847+0000 7fd3f1465700 0 mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-23T00:35:01.218 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.847+0000 7fd3f1465700 0 mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-23T00:35:01.219 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.847+0000 7fd3f1465700 0 mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-23T00:35:01.219 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:34:58.076237+0000 mgr.y (mgr.14146) 39 : cluster [DBG] pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:01.219 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:34:58.434538+0000 mon.a (mon.0) 124 : audit [DBG] from='client.? 172.21.15.196:0/1996023354' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-23T00:35:01.219 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:35:00.847+0000 7fd3f1465700 1 mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-23T00:35:01.219 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 bash[39605]: c2dcfab6509bc3c6ff6ad290dd7bdbb63a095831a5412129ea059e08b5344c5a 2024-02-23T00:35:01.219 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:00 smithi196 systemd[1]: Started Ceph mon.b for fce2617e-d1e2-11ee-95c0-87774f69a715. 2024-02-23T00:35:06.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.048+0000 7fa76c6c0700 1 -- 172.21.15.196:0/1060665047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa750003400 msgr2=0x7fa750005890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:35:06.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.048+0000 7fa76c6c0700 1 --2- 172.21.15.196:0/1060665047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa750003400 0x7fa750005890 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fa7640f6590 tx=0x7fa76000b040).stop 2024-02-23T00:35:06.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.048+0000 7fa76c6c0700 1 -- 172.21.15.196:0/1060665047 shutdown_connections 2024-02-23T00:35:06.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.048+0000 7fa76c6c0700 1 --2- 172.21.15.196:0/1060665047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa750003400 0x7fa750005890 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:06.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.048+0000 7fa76c6c0700 1 -- 172.21.15.196:0/1060665047 >> 172.21.15.196:0/1060665047 conn(0x7fa76406c440 msgr2=0x7fa76406c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:35:06.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.048+0000 7fa76c6c0700 1 -- 172.21.15.196:0/1060665047 shutdown_connections 2024-02-23T00:35:06.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.049+0000 7fa76c6c0700 1 -- 172.21.15.196:0/1060665047 wait complete. 2024-02-23T00:35:06.054 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.049+0000 7fa76c6c0700 1 librados: starting msgr at 2024-02-23T00:35:06.054 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.049+0000 7fa76c6c0700 1 librados: starting objecter 2024-02-23T00:35:06.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.049+0000 7fa76c6c0700 1 Processor -- start 2024-02-23T00:35:06.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.050+0000 7fa76c6c0700 1 -- start start 2024-02-23T00:35:06.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.050+0000 7fa76c6c0700 1 librados: setting wanted keys 2024-02-23T00:35:06.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.050+0000 7fa76c6c0700 1 librados: calling monclient init 2024-02-23T00:35:06.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.050+0000 7fa76c6c0700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa750003400 0x7fa7640f8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:35:06.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.050+0000 7fa76c6c0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7640f8750 0x7fa7640f8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:35:06.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.050+0000 7fa76c6c0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa764113210 con 0x7fa7640f8750 2024-02-23T00:35:06.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.050+0000 7fa76c6c0700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa764113350 con 0x7fa750003400 2024-02-23T00:35:06.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.050+0000 7fa76a45c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa750003400 0x7fa7640f8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:35:06.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.050+0000 7fa76a45c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa750003400 0x7fa7640f8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56506/0 (socket says 172.21.15.196:56506) 2024-02-23T00:35:06.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.050+0000 7fa76a45c700 1 -- 172.21.15.196:0/2149567894 learned_addr learned my addr 172.21.15.196:0/2149567894 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-02-23T00:35:06.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.050+0000 7fa769c5b700 1 --2- 172.21.15.196:0/2149567894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7640f8750 0x7fa7640f8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:35:06.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.050+0000 7fa76a45c700 1 -- 172.21.15.196:0/2149567894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa750003400 msgr2=0x7fa7640f8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-23T00:35:06.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.050+0000 7fa76a45c700 1 -- 172.21.15.196:0/2149567894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa750003400 msgr2=0x7fa7640f8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-23T00:35:06.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.050+0000 7fa76a45c700 1 --2- 172.21.15.196:0/2149567894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa750003400 0x7fa7640f8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-02-23T00:35:06.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.050+0000 7fa76a45c700 1 --2- 172.21.15.196:0/2149567894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa750003400 0x7fa7640f8260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2024-02-23T00:35:06.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.051+0000 7fa769c5b700 1 -- 172.21.15.196:0/2149567894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa750003400 msgr2=0x7fa7640f8260 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-23T00:35:06.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.051+0000 7fa769c5b700 1 --2- 172.21.15.196:0/2149567894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa750003400 0x7fa7640f8260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:06.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.051+0000 7fa769c5b700 1 -- 172.21.15.196:0/2149567894 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa760005040 con 0x7fa7640f8750 2024-02-23T00:35:06.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.051+0000 7fa769c5b700 1 --2- 172.21.15.196:0/2149567894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7640f8750 0x7fa7640f8bd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fa75400b460 tx=0x7fa754009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:35:06.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.051+0000 7fa75b7fe700 10 client.?.objecter ms_handle_connect 0x7fa7640f8750 2024-02-23T00:35:06.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.051+0000 7fa75b7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:35:06.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.051+0000 7fa75b7fe700 1 -- 172.21.15.196:0/2149567894 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fa75400f6c0 con 0x7fa7640f8750 2024-02-23T00:35:06.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.051+0000 7fa75b7fe700 1 -- 172.21.15.196:0/2149567894 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fa75400fcb0 con 0x7fa7640f8750 2024-02-23T00:35:06.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.051+0000 7fa76c6c0700 1 -- 172.21.15.196:0/2149567894 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7641134f0 con 0x7fa7640f8750 2024-02-23T00:35:06.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.051+0000 7fa75b7fe700 1 -- 172.21.15.196:0/2149567894 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fa754011b00 con 0x7fa7640f8750 2024-02-23T00:35:06.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.051+0000 7fa76c6c0700 10 client.14182.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:35:06.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.051+0000 7fa76c6c0700 1 -- 172.21.15.196:0/2149567894 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa764113a80 con 0x7fa7640f8750 2024-02-23T00:35:06.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.051+0000 7fa76c6c0700 1 librados: init done 2024-02-23T00:35:06.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.052+0000 7fa75b7fe700 1 -- 172.21.15.196:0/2149567894 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 38673+0+0 (secure 0 0 0) 0x7fa75401a400 con 0x7fa7640f8750 2024-02-23T00:35:06.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.052+0000 7fa75b7fe700 1 --2- 172.21.15.196:0/2149567894 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fa7480341f0 0x7fa7480366b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:35:06.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.052+0000 7fa75b7fe700 1 -- 172.21.15.196:0/2149567894 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1098+0+0 (secure 0 0 0) 0x7fa754042db0 con 0x7fa7640f8750 2024-02-23T00:35:06.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.052+0000 7fa75b7fe700 10 client.14182.objecter ms_dispatch 0x7fa764001550 osd_map(4..4 src has 1..4) v4 2024-02-23T00:35:06.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.052+0000 7fa75b7fe700 3 client.14182.objecter handle_osd_map got epochs [4,4] > 0 2024-02-23T00:35:06.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.052+0000 7fa75b7fe700 3 client.14182.objecter handle_osd_map decoding full epoch 4 2024-02-23T00:35:06.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.052+0000 7fa75b7fe700 20 client.14182.objecter dump_active .. 0 homeless 2024-02-23T00:35:06.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.052+0000 7fa76a45c700 1 --2- 172.21.15.196:0/2149567894 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fa7480341f0 0x7fa7480366b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:35:06.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.053+0000 7fa76c6c0700 1 -- 172.21.15.196:0/2149567894 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa76405f1f0 con 0x7fa7640f8750 2024-02-23T00:35:06.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.053+0000 7fa76a45c700 1 --2- 172.21.15.196:0/2149567894 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fa7480341f0 0x7fa7480366b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fa760000ea0 tx=0x7fa76000b040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:35:06.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.053+0000 7fa75b7fe700 10 client.14182.objecter ms_handle_connect 0x7fa7480341f0 2024-02-23T00:35:06.059 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.058+0000 7fa75b7fe700 1 -- 172.21.15.196:0/2149567894 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fa75400e260 con 0x7fa7640f8750 2024-02-23T00:35:06.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:00.852624+0000 mon.a (mon.0) 125 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-23T00:35:06.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:00.855180+0000 mon.a (mon.0) 126 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-23T00:35:06.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:00.855382+0000 mon.a (mon.0) 127 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-23T00:35:06.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:00.855584+0000 mon.a (mon.0) 128 : cluster [INF] mon.a calling monitor election 2024-02-23T00:35:06.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:01.852355+0000 mon.a (mon.0) 129 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-23T00:35:06.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:02.076891+0000 mgr.y (mgr.14146) 41 : cluster [DBG] pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:06.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:02.852449+0000 mon.a (mon.0) 130 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-23T00:35:06.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:02.853413+0000 mon.b (mon.1) 1 : cluster [INF] mon.b calling monitor election 2024-02-23T00:35:06.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:03.852501+0000 mon.a (mon.0) 131 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-23T00:35:06.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:04.077153+0000 mgr.y (mgr.14146) 42 : cluster [DBG] pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:06.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:04.852742+0000 mon.a (mon.0) 132 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-23T00:35:06.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:05.852635+0000 mon.a (mon.0) 133 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-23T00:35:06.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:05.870127+0000 mon.a (mon.0) 134 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-23T00:35:06.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:05.873210+0000 mon.a (mon.0) 135 : cluster [DBG] monmap e2: 2 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0]} removed_ranks: {} 2024-02-23T00:35:06.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:05.885610+0000 mon.a (mon.0) 136 : cluster [DBG] fsmap 2024-02-23T00:35:06.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:05.885652+0000 mon.a (mon.0) 137 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-23T00:35:06.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:05.885856+0000 mon.a (mon.0) 138 : cluster [DBG] mgrmap e13: y(active, since 73s) 2024-02-23T00:35:06.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:05.885997+0000 mon.a (mon.0) 139 : cluster [INF] overall HEALTH_OK 2024-02-23T00:35:06.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:05.889020+0000 mon.a (mon.0) 140 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:06.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:05.890041+0000 mon.a (mon.0) 141 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-23T00:35:06.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:05.890863+0000 mon.a (mon.0) 142 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:06.290 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.288+0000 7fa76c6c0700 1 -- 172.21.15.196:0/2149567894 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa764046ce0 con 0x7fa7640f8750 2024-02-23T00:35:06.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.289+0000 7fa75b7fe700 1 -- 172.21.15.196:0/2149567894 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+993 (secure 0 0 0) 0x7fa754029090 con 0x7fa7640f8750 2024-02-23T00:35:06.291 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:35:06.291 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":2,"fsid":"fce2617e-d1e2-11ee-95c0-87774f69a715","modified":"2024-02-23T00:35:00.852302Z","created":"2024-02-23T00:32:52.273607Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-23T00:35:06.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.291+0000 7fa76c6c0700 10 librados: watch_flush enter 2024-02-23T00:35:06.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.291+0000 7fa76c6c0700 10 librados: watch_flush exit 2024-02-23T00:35:06.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.291+0000 7fa76c6c0700 20 client.14182.objecter shutdown clearing up homeless session... 2024-02-23T00:35:06.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.291+0000 7fa76c6c0700 10 client.14182.objecter successfully canceled tick 2024-02-23T00:35:06.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.291+0000 7fa76c6c0700 1 -- 172.21.15.196:0/2149567894 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fa7480341f0 msgr2=0x7fa7480366b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:35:06.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.291+0000 7fa76c6c0700 1 --2- 172.21.15.196:0/2149567894 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fa7480341f0 0x7fa7480366b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fa760000ea0 tx=0x7fa76000b040).stop 2024-02-23T00:35:06.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.291+0000 7fa76c6c0700 1 -- 172.21.15.196:0/2149567894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7640f8750 msgr2=0x7fa7640f8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:35:06.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.291+0000 7fa76c6c0700 1 --2- 172.21.15.196:0/2149567894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7640f8750 0x7fa7640f8bd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fa75400b460 tx=0x7fa754009060).stop 2024-02-23T00:35:06.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.291+0000 7fa76c6c0700 1 -- 172.21.15.196:0/2149567894 shutdown_connections 2024-02-23T00:35:06.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.291+0000 7fa76c6c0700 1 --2- 172.21.15.196:0/2149567894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa750003400 0x7fa7640f8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:06.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.291+0000 7fa76c6c0700 1 --2- 172.21.15.196:0/2149567894 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fa7480341f0 0x7fa7480366b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:06.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.291+0000 7fa76c6c0700 1 --2- 172.21.15.196:0/2149567894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7640f8750 0x7fa7640f8bd0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:06.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.291+0000 7fa76c6c0700 1 -- 172.21.15.196:0/2149567894 >> 172.21.15.196:0/2149567894 conn(0x7fa76406c440 msgr2=0x7fa7640faf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:35:06.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.291+0000 7fa76c6c0700 1 -- 172.21.15.196:0/2149567894 shutdown_connections 2024-02-23T00:35:06.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.291+0000 7fa76c6c0700 1 -- 172.21.15.196:0/2149567894 wait complete. 2024-02-23T00:35:06.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:06.291+0000 7fa76c6c0700 1 librados: shutdown 2024-02-23T00:35:06.294 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 2 2024-02-23T00:35:06.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:00.852624+0000 mon.a (mon.0) 125 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-23T00:35:06.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:00.855180+0000 mon.a (mon.0) 126 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-23T00:35:06.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:00.855382+0000 mon.a (mon.0) 127 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-23T00:35:06.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:00.855584+0000 mon.a (mon.0) 128 : cluster [INF] mon.a calling monitor election 2024-02-23T00:35:06.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:01.852355+0000 mon.a (mon.0) 129 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-23T00:35:06.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:02.076891+0000 mgr.y (mgr.14146) 41 : cluster [DBG] pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:06.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:02.852449+0000 mon.a (mon.0) 130 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-23T00:35:06.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:02.853413+0000 mon.b (mon.1) 1 : cluster [INF] mon.b calling monitor election 2024-02-23T00:35:06.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:03.852501+0000 mon.a (mon.0) 131 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-23T00:35:06.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:04.077153+0000 mgr.y (mgr.14146) 42 : cluster [DBG] pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:06.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:04.852742+0000 mon.a (mon.0) 132 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-23T00:35:06.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:05.852635+0000 mon.a (mon.0) 133 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-23T00:35:06.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:05.870127+0000 mon.a (mon.0) 134 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-23T00:35:06.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:05.873210+0000 mon.a (mon.0) 135 : cluster [DBG] monmap e2: 2 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0]} removed_ranks: {} 2024-02-23T00:35:06.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:05.885610+0000 mon.a (mon.0) 136 : cluster [DBG] fsmap 2024-02-23T00:35:06.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:05.885652+0000 mon.a (mon.0) 137 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-23T00:35:06.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:05.885856+0000 mon.a (mon.0) 138 : cluster [DBG] mgrmap e13: y(active, since 73s) 2024-02-23T00:35:06.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:05.885997+0000 mon.a (mon.0) 139 : cluster [INF] overall HEALTH_OK 2024-02-23T00:35:06.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:05.889020+0000 mon.a (mon.0) 140 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:06.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:05.890041+0000 mon.a (mon.0) 141 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-23T00:35:06.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:05.890863+0000 mon.a (mon.0) 142 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:07.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:35:05.891619+0000 mgr.y (mgr.14146) 43 : cephadm [INF] Deploying daemon mon.c on smithi047 2024-02-23T00:35:07.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:06.077383+0000 mgr.y (mgr.14146) 44 : cluster [DBG] pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:07.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:06.290386+0000 mon.a (mon.0) 143 : audit [DBG] from='client.? 172.21.15.196:0/2149567894' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-23T00:35:07.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:06.853117+0000 mon.a (mon.0) 144 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-23T00:35:07.211 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:35:05.891619+0000 mgr.y (mgr.14146) 43 : cephadm [INF] Deploying daemon mon.c on smithi047 2024-02-23T00:35:07.211 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:06.077383+0000 mgr.y (mgr.14146) 44 : cluster [DBG] pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:07.211 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:06.290386+0000 mon.a (mon.0) 143 : audit [DBG] from='client.? 172.21.15.196:0/2149567894' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-23T00:35:07.211 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:06.853117+0000 mon.a (mon.0) 144 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-23T00:35:09.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:08.077668+0000 mgr.y (mgr.14146) 45 : cluster [DBG] pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:09.496 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:08.077668+0000 mgr.y (mgr.14146) 45 : cluster [DBG] pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:09.496 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 systemd[1]: Starting Ceph mon.c for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T00:35:10.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.674+0000 7efccab6b880 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-23T00:35:10.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.674+0000 7efccab6b880 0 ceph version 16.2.14-550-geb66ed92 (eb66ed921e744301e6863be6353618d63967ea59) pacific (stable), process ceph-mon, pid 7 2024-02-23T00:35:10.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.674+0000 7efccab6b880 0 pidfile_write: ignore empty --pid-file 2024-02-23T00:35:10.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.678+0000 7efccab6b880 0 load: jerasure load: lrc load: isa 2024-02-23T00:35:10.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: RocksDB version: 6.8.1 2024-02-23T00:35:10.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: 2024-02-23T00:35:10.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-23T00:35:10.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Compile date Feb 20 2024 2024-02-23T00:35:10.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: DB SUMMARY 2024-02-23T00:35:10.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: 2024-02-23T00:35:10.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: CURRENT file: CURRENT 2024-02-23T00:35:10.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: 2024-02-23T00:35:10.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: IDENTITY file: IDENTITY 2024-02-23T00:35:10.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: 2024-02-23T00:35:10.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: MANIFEST file: MANIFEST-000001 size: 13 Bytes 2024-02-23T00:35:10.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: 2024-02-23T00:35:10.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-02-23T00:35:10.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: 2024-02-23T00:35:10.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000003.log size: 636 ; 2024-02-23T00:35:10.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: 2024-02-23T00:35:10.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.error_if_exists: 0 2024-02-23T00:35:10.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.create_if_missing: 0 2024-02-23T00:35:10.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.paranoid_checks: 1 2024-02-23T00:35:10.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.env: 0x557d87a47080 2024-02-23T00:35:10.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.fs: Posix File System 2024-02-23T00:35:10.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.info_log: 0x557d8922e260 2024-02-23T00:35:10.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-23T00:35:10.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.statistics: (nil) 2024-02-23T00:35:10.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.use_fsync: 0 2024-02-23T00:35:10.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.max_log_file_size: 0 2024-02-23T00:35:10.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-23T00:35:10.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-23T00:35:10.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-23T00:35:10.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-23T00:35:10.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.allow_fallocate: 1 2024-02-23T00:35:10.061 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-23T00:35:10.061 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-23T00:35:10.061 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.use_direct_reads: 0 2024-02-23T00:35:10.061 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-23T00:35:10.061 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.create_missing_column_families: 0 2024-02-23T00:35:10.061 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.db_log_dir: 2024-02-23T00:35:10.061 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-c/store.db 2024-02-23T00:35:10.061 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-23T00:35:10.061 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.max_subcompactions: 1 2024-02-23T00:35:10.061 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.max_background_flushes: -1 2024-02-23T00:35:10.061 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-23T00:35:10.061 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-23T00:35:10.061 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-23T00:35:10.062 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-23T00:35:10.062 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-23T00:35:10.062 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.advise_random_on_open: 1 2024-02-23T00:35:10.062 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-23T00:35:10.062 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.write_buffer_manager: 0x557d89f20390 2024-02-23T00:35:10.062 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-23T00:35:10.062 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-23T00:35:10.062 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-23T00:35:10.062 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-23T00:35:10.062 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.rate_limiter: (nil) 2024-02-23T00:35:10.062 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-23T00:35:10.062 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-23T00:35:10.062 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-23T00:35:10.062 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-23T00:35:10.063 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.unordered_write: 0 2024-02-23T00:35:10.063 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-23T00:35:10.063 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-23T00:35:10.063 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-23T00:35:10.063 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-23T00:35:10.063 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.row_cache: None 2024-02-23T00:35:10.063 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.wal_filter: None 2024-02-23T00:35:10.063 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-23T00:35:10.063 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-23T00:35:10.063 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.preserve_deletes: 0 2024-02-23T00:35:10.063 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.two_write_queues: 0 2024-02-23T00:35:10.063 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.manual_wal_flush: 0 2024-02-23T00:35:10.063 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.atomic_flush: 0 2024-02-23T00:35:10.063 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-23T00:35:10.063 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-23T00:35:10.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-23T00:35:10.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.log_readahead_size: 0 2024-02-23T00:35:10.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-23T00:35:10.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.max_background_jobs: 2 2024-02-23T00:35:10.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.max_background_compactions: -1 2024-02-23T00:35:10.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-23T00:35:10.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-23T00:35:10.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-23T00:35:10.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.max_total_wal_size: 0 2024-02-23T00:35:10.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-23T00:35:10.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-23T00:35:10.066 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-23T00:35:10.066 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-23T00:35:10.066 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.max_open_files: -1 2024-02-23T00:35:10.066 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.bytes_per_sync: 0 2024-02-23T00:35:10.066 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-23T00:35:10.066 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-23T00:35:10.066 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-23T00:35:10.066 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Compression algorithms supported: 2024-02-23T00:35:10.067 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-23T00:35:10.067 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: kZSTD supported: 0 2024-02-23T00:35:10.067 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: kXpressCompression supported: 0 2024-02-23T00:35:10.067 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-23T00:35:10.067 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: kLZ4Compression supported: 1 2024-02-23T00:35:10.067 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: kBZip2Compression supported: 0 2024-02-23T00:35:10.067 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: kZlibCompression supported: 1 2024-02-23T00:35:10.067 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: kSnappyCompression supported: 1 2024-02-23T00:35:10.067 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-23T00:35:10.067 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000001 2024-02-23T00:35:10.067 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: 2024-02-23T00:35:10.067 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-23T00:35:10.068 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: 2024-02-23T00:35:10.068 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-23T00:35:10.068 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.merge_operator: 2024-02-23T00:35:10.068 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.compaction_filter: None 2024-02-23T00:35:10.068 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.compaction_filter_factory: None 2024-02-23T00:35:10.068 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-23T00:35:10.068 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-23T00:35:10.068 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557d891d3d10) 2024-02-23T00:35:10.068 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cache_index_and_filter_blocks: 1 2024-02-23T00:35:10.068 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-23T00:35:10.068 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-23T00:35:10.068 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: pin_top_level_index_and_filter: 1 2024-02-23T00:35:10.069 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: index_type: 0 2024-02-23T00:35:10.069 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: data_block_index_type: 0 2024-02-23T00:35:10.069 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: index_shortening: 1 2024-02-23T00:35:10.069 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: data_block_hash_table_util_ratio: 0.750000 2024-02-23T00:35:10.069 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: hash_index_allow_collision: 1 2024-02-23T00:35:10.069 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: checksum: 1 2024-02-23T00:35:10.069 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: no_block_cache: 0 2024-02-23T00:35:10.069 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: block_cache: 0x557d89208f10 2024-02-23T00:35:10.069 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: block_cache_name: BinnedLRUCache 2024-02-23T00:35:10.069 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: block_cache_options: 2024-02-23T00:35:10.069 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: capacity : 536870912 2024-02-23T00:35:10.069 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: num_shard_bits : 4 2024-02-23T00:35:10.069 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: strict_capacity_limit : 0 2024-02-23T00:35:10.070 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: high_pri_pool_ratio: 0.000 2024-02-23T00:35:10.070 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: block_cache_compressed: (nil) 2024-02-23T00:35:10.070 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: persistent_cache: (nil) 2024-02-23T00:35:10.070 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: block_size: 4096 2024-02-23T00:35:10.070 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: block_size_deviation: 10 2024-02-23T00:35:10.070 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: block_restart_interval: 16 2024-02-23T00:35:10.070 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: index_block_restart_interval: 1 2024-02-23T00:35:10.070 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: metadata_block_size: 4096 2024-02-23T00:35:10.070 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: partition_filters: 0 2024-02-23T00:35:10.070 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: use_delta_encoding: 1 2024-02-23T00:35:10.070 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-23T00:35:10.070 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: whole_key_filtering: 1 2024-02-23T00:35:10.070 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: verify_compression: 0 2024-02-23T00:35:10.070 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: read_amp_bytes_per_bit: 0 2024-02-23T00:35:10.070 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: format_version: 2 2024-02-23T00:35:10.071 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: enable_index_compression: 1 2024-02-23T00:35:10.071 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: block_align: 0 2024-02-23T00:35:10.071 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: 2024-02-23T00:35:10.071 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-23T00:35:10.071 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-23T00:35:10.071 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.compression: NoCompression 2024-02-23T00:35:10.071 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-23T00:35:10.071 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-23T00:35:10.071 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-23T00:35:10.071 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.num_levels: 7 2024-02-23T00:35:10.071 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-23T00:35:10.071 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-23T00:35:10.071 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-23T00:35:10.072 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-23T00:35:10.072 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-23T00:35:10.072 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-23T00:35:10.072 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-23T00:35:10.072 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-23T00:35:10.072 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-23T00:35:10.072 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-23T00:35:10.072 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.compression_opts.level: 32767 2024-02-23T00:35:10.072 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-23T00:35:10.072 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-23T00:35:10.072 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-23T00:35:10.072 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.compression_opts.enabled: false 2024-02-23T00:35:10.072 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-23T00:35:10.073 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-23T00:35:10.073 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-23T00:35:10.073 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-23T00:35:10.073 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-23T00:35:10.073 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-23T00:35:10.073 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-23T00:35:10.073 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-23T00:35:10.073 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-23T00:35:10.073 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-23T00:35:10.073 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-23T00:35:10.073 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-23T00:35:10.073 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-23T00:35:10.073 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-23T00:35:10.073 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-23T00:35:10.074 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-23T00:35:10.074 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-23T00:35:10.074 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.arena_block_size: 4194304 2024-02-23T00:35:10.074 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-23T00:35:10.074 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-23T00:35:10.074 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-23T00:35:10.074 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-23T00:35:10.074 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-23T00:35:10.074 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-23T00:35:10.074 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-23T00:35:10.074 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-23T00:35:10.074 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-23T00:35:10.074 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-23T00:35:10.074 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-23T00:35:10.075 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-23T00:35:10.075 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-23T00:35:10.075 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-23T00:35:10.075 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.table_properties_collectors: 2024-02-23T00:35:10.075 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.inplace_update_support: 0 2024-02-23T00:35:10.075 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-23T00:35:10.075 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-23T00:35:10.075 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-23T00:35:10.075 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-23T00:35:10.075 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.bloom_locality: 0 2024-02-23T00:35:10.075 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.max_successive_merges: 0 2024-02-23T00:35:10.075 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-23T00:35:10.075 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-23T00:35:10.075 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.force_consistency_checks: 0 2024-02-23T00:35:10.076 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-23T00:35:10.076 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.ttl: 2592000 2024-02-23T00:35:10.076 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.679+0000 7efccab6b880 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-23T00:35:10.076 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.680+0000 7efccab6b880 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-23T00:35:10.080 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: 2024-02-23T00:35:10.080 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.680+0000 7efccab6b880 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 0 2024-02-23T00:35:10.080 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: 2024-02-23T00:35:10.080 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.680+0000 7efccab6b880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708648509681869, "job": 1, "event": "recovery_started", "log_files": [3]} 2024-02-23T00:35:10.080 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.680+0000 7efccab6b880 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #3 mode 2 2024-02-23T00:35:10.080 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.680+0000 7efccab6b880 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-02-23T00:35:10.081 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.681+0000 7efccab6b880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708648509682326, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 4, "file_size": 1562, "table_properties": {"data_size": 648, "index_size": 40, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1708648509, "oldest_key_time": 3, "file_creation_time": 0}} 2024-02-23T00:35:10.081 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.681+0000 7efccab6b880 4 rocksdb: [version_set.cc:3826] Creating manifest 5 2024-02-23T00:35:10.081 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: 2024-02-23T00:35:10.081 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.681+0000 7efccab6b880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708648509682729, "job": 1, "event": "recovery_finished"} 2024-02-23T00:35:10.081 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.682+0000 7efccab6b880 4 rocksdb: DB pointer 0x557d892d1800 2024-02-23T00:35:10.081 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.682+0000 7efcb195d700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-02-23T00:35:10.081 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.682+0000 7efcb195d700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-02-23T00:35:10.081 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: ** DB Stats ** 2024-02-23T00:35:10.081 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-23T00:35:10.081 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-23T00:35:10.081 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-23T00:35:10.081 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-23T00:35:10.081 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-23T00:35:10.081 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-23T00:35:10.082 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-23T00:35:10.082 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: 2024-02-23T00:35:10.082 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: ** Compaction Stats [default] ** 2024-02-23T00:35:10.082 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-23T00:35:10.082 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-23T00:35:10.082 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: L0 1/0 1.53 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 2024-02-23T00:35:10.082 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: Sum 1/0 1.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 2024-02-23T00:35:10.082 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 2024-02-23T00:35:10.082 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: 2024-02-23T00:35:10.082 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: ** Compaction Stats [default] ** 2024-02-23T00:35:10.082 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-23T00:35:10.082 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-23T00:35:10.082 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.6 0.00 0.00 1 0.000 0 0 2024-02-23T00:35:10.082 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-23T00:35:10.082 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-23T00:35:10.083 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-23T00:35:10.083 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: AddFile(Total Files): cumulative 0, interval 0 2024-02-23T00:35:10.083 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-23T00:35:10.083 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: AddFile(Keys): cumulative 0, interval 0 2024-02-23T00:35:10.083 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: Cumulative compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-23T00:35:10.083 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: Interval compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-23T00:35:10.083 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-23T00:35:10.083 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: 2024-02-23T00:35:10.083 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: ** File Read Latency Histogram By Level [default] ** 2024-02-23T00:35:10.083 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: 2024-02-23T00:35:10.083 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: ** Compaction Stats [default] ** 2024-02-23T00:35:10.083 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-23T00:35:10.083 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-23T00:35:10.084 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: L0 1/0 1.53 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 2024-02-23T00:35:10.084 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: Sum 1/0 1.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 2024-02-23T00:35:10.084 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-23T00:35:10.084 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: 2024-02-23T00:35:10.084 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: ** Compaction Stats [default] ** 2024-02-23T00:35:10.084 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-23T00:35:10.084 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-23T00:35:10.084 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.6 0.00 0.00 1 0.000 0 0 2024-02-23T00:35:10.084 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-23T00:35:10.084 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-23T00:35:10.084 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-23T00:35:10.084 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: AddFile(Total Files): cumulative 0, interval 0 2024-02-23T00:35:10.084 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-23T00:35:10.084 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: AddFile(Keys): cumulative 0, interval 0 2024-02-23T00:35:10.085 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: Cumulative compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-23T00:35:10.085 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-23T00:35:10.085 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-23T00:35:10.085 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: 2024-02-23T00:35:10.085 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: ** File Read Latency Histogram By Level [default] ** 2024-02-23T00:35:10.085 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: 2024-02-23T00:35:10.085 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.682+0000 7efccab6b880 0 mon.c does not exist in monmap, will attempt to join an existing cluster 2024-02-23T00:35:10.085 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.682+0000 7efccab6b880 0 using public_addrv [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-02-23T00:35:10.085 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.683+0000 7efccab6b880 0 starting mon.c rank -1 at public addrs [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] at bind addrs [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid fce2617e-d1e2-11ee-95c0-87774f69a715 2024-02-23T00:35:10.085 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.684+0000 7efccab6b880 1 mon.c@-1(???) e0 preinit fsid fce2617e-d1e2-11ee-95c0-87774f69a715 2024-02-23T00:35:10.085 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.702+0000 7efcaf959700 4 rocksdb: [db_impl/db_impl_write.cc:1668] [default] New memtable created with log file: #9. Immutable memtables: 0. 2024-02-23T00:35:10.085 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: 2024-02-23T00:35:10.085 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.702+0000 7efcb996d700 4 rocksdb: (Original Log Time 2024/02/23-00:35:09.704066) [db_impl/db_impl_compaction_flush.cc:2198] Calling FlushMemTableToOutputFile with column family [default], flush slots available 1, compaction slots available 1, flush slots scheduled 1, compaction slots scheduled 0 2024-02-23T00:35:10.085 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.702+0000 7efcb996d700 4 rocksdb: [flush_job.cc:321] [default] [JOB 3] Flushing memtable with next log file: 9 2024-02-23T00:35:10.086 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: 2024-02-23T00:35:10.086 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.702+0000 7efcb996d700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708648509704114, "job": 3, "event": "flush_started", "num_memtables": 1, "num_entries": 991, "num_deletes": 23, "total_data_size": 4052602, "memory_usage": 4070888, "flush_reason": "Manual Compaction"} 2024-02-23T00:35:10.086 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.702+0000 7efcb996d700 4 rocksdb: [flush_job.cc:350] [default] [JOB 3] Level-0 flush table #10: started 2024-02-23T00:35:10.086 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.703+0000 7efcb4162700 0 mon.c@-1(synchronizing).mds e1 new map 2024-02-23T00:35:10.086 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.703+0000 7efcb4162700 0 mon.c@-1(synchronizing).mds e1 print_map 2024-02-23T00:35:10.086 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: e1 2024-02-23T00:35:10.086 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: enable_multiple, ever_enabled_multiple: 1,1 2024-02-23T00:35:10.086 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-23T00:35:10.086 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: legacy client fscid: -1 2024-02-23T00:35:10.086 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: 2024-02-23T00:35:10.086 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: No filesystems configured 2024-02-23T00:35:10.086 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: 2024-02-23T00:35:10.086 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.703+0000 7efcb4162700 1 mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-23T00:35:10.086 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.703+0000 7efcb4162700 1 mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-23T00:35:10.087 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.703+0000 7efcb4162700 1 mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-23T00:35:10.087 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.703+0000 7efcb4162700 1 mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-23T00:35:10.087 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.703+0000 7efcb4162700 1 mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-23T00:35:10.087 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.703+0000 7efcb4162700 1 mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-23T00:35:10.087 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.703+0000 7efcb4162700 0 mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-23T00:35:10.087 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.703+0000 7efcb4162700 0 mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-23T00:35:10.087 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.703+0000 7efcb4162700 0 mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-23T00:35:10.087 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.703+0000 7efcb4162700 0 mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-23T00:35:10.087 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:08.077668+0000 mgr.y (mgr.14146) 45 : cluster [DBG] pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:10.087 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:35:09.703+0000 7efcb4162700 1 mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-23T00:35:10.087 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 bash[44771]: c163f9f8f8bf0464262ec7087419b0f65240ff21b73c845f41b129e63ca75103 2024-02-23T00:35:10.087 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:09 smithi047 systemd[1]: Started Ceph mon.c for fce2617e-d1e2-11ee-95c0-87774f69a715. 2024-02-23T00:35:10.087 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-23T00:35:10.088 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph mon dump -f json 2024-02-23T00:35:15.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:09.711705+0000 mon.a (mon.0) 146 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-23T00:35:15.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:09.711873+0000 mon.a (mon.0) 147 : cluster [INF] mon.a calling monitor election 2024-02-23T00:35:15.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:09.714096+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2024-02-23T00:35:15.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:09.726217+0000 mon.a (mon.0) 148 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-23T00:35:15.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:09.726342+0000 mon.a (mon.0) 149 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-23T00:35:15.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:10.078056+0000 mgr.y (mgr.14146) 46 : cluster [DBG] pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:15.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:10.708426+0000 mon.a (mon.0) 150 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-23T00:35:15.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:11.708253+0000 mon.a (mon.0) 151 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-23T00:35:15.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:11.710114+0000 mon.c (mon.2) 1 : cluster [INF] mon.c calling monitor election 2024-02-23T00:35:15.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:12.078451+0000 mgr.y (mgr.14146) 47 : cluster [DBG] pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:15.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:12.708625+0000 mon.a (mon.0) 152 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-23T00:35:15.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:13.708660+0000 mon.a (mon.0) 153 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-23T00:35:15.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:14.078717+0000 mgr.y (mgr.14146) 48 : cluster [DBG] pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:15.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:14.708739+0000 mon.a (mon.0) 154 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-23T00:35:15.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:14.740457+0000 mon.a (mon.0) 155 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-23T00:35:15.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:14.743763+0000 mon.a (mon.0) 156 : cluster [DBG] monmap e3: 3 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0],c=[v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0]} removed_ranks: {} 2024-02-23T00:35:15.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:14.744293+0000 mon.a (mon.0) 157 : cluster [DBG] fsmap 2024-02-23T00:35:15.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:09.711705+0000 mon.a (mon.0) 146 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-23T00:35:15.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:09.711873+0000 mon.a (mon.0) 147 : cluster [INF] mon.a calling monitor election 2024-02-23T00:35:15.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:09.714096+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2024-02-23T00:35:15.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:09.726217+0000 mon.a (mon.0) 148 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-23T00:35:15.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:09.726342+0000 mon.a (mon.0) 149 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-23T00:35:15.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:10.078056+0000 mgr.y (mgr.14146) 46 : cluster [DBG] pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:15.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:10.708426+0000 mon.a (mon.0) 150 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-23T00:35:15.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:11.708253+0000 mon.a (mon.0) 151 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-23T00:35:15.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:11.710114+0000 mon.c (mon.2) 1 : cluster [INF] mon.c calling monitor election 2024-02-23T00:35:15.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:12.078451+0000 mgr.y (mgr.14146) 47 : cluster [DBG] pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:15.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:12.708625+0000 mon.a (mon.0) 152 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-23T00:35:15.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:13.708660+0000 mon.a (mon.0) 153 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-23T00:35:15.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:14.078717+0000 mgr.y (mgr.14146) 48 : cluster [DBG] pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:15.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:14.708739+0000 mon.a (mon.0) 154 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-23T00:35:15.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:14.740457+0000 mon.a (mon.0) 155 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-23T00:35:15.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:14.743763+0000 mon.a (mon.0) 156 : cluster [DBG] monmap e3: 3 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0],c=[v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0]} removed_ranks: {} 2024-02-23T00:35:15.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:14.744293+0000 mon.a (mon.0) 157 : cluster [DBG] fsmap 2024-02-23T00:35:15.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:14.744344+0000 mon.a (mon.0) 158 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-23T00:35:15.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:14.744578+0000 mon.a (mon.0) 159 : cluster [DBG] mgrmap e13: y(active, since 82s) 2024-02-23T00:35:15.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:14.744879+0000 mon.a (mon.0) 160 : cluster [INF] overall HEALTH_OK 2024-02-23T00:35:15.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:14.749233+0000 mon.a (mon.0) 161 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:15.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:14.750602+0000 mon.a (mon.0) 162 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:35:15.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:14.744344+0000 mon.a (mon.0) 158 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-23T00:35:15.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:14.744578+0000 mon.a (mon.0) 159 : cluster [DBG] mgrmap e13: y(active, since 82s) 2024-02-23T00:35:15.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:14.744879+0000 mon.a (mon.0) 160 : cluster [INF] overall HEALTH_OK 2024-02-23T00:35:15.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:14.749233+0000 mon.a (mon.0) 161 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:15.061 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:14.750602+0000 mon.a (mon.0) 162 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:35:15.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:09.711705+0000 mon.a (mon.0) 146 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-23T00:35:15.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:09.711873+0000 mon.a (mon.0) 147 : cluster [INF] mon.a calling monitor election 2024-02-23T00:35:15.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:09.714096+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2024-02-23T00:35:15.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:09.726217+0000 mon.a (mon.0) 148 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-23T00:35:15.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:09.726342+0000 mon.a (mon.0) 149 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-23T00:35:15.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:10.078056+0000 mgr.y (mgr.14146) 46 : cluster [DBG] pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:15.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:10.708426+0000 mon.a (mon.0) 150 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-23T00:35:15.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:11.708253+0000 mon.a (mon.0) 151 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-23T00:35:15.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:11.710114+0000 mon.c (mon.2) 1 : cluster [INF] mon.c calling monitor election 2024-02-23T00:35:15.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:12.078451+0000 mgr.y (mgr.14146) 47 : cluster [DBG] pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:15.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:12.708625+0000 mon.a (mon.0) 152 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-23T00:35:15.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:13.708660+0000 mon.a (mon.0) 153 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-23T00:35:15.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:14.078717+0000 mgr.y (mgr.14146) 48 : cluster [DBG] pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:15.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:14.708739+0000 mon.a (mon.0) 154 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-23T00:35:15.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:14.740457+0000 mon.a (mon.0) 155 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-23T00:35:15.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:14.743763+0000 mon.a (mon.0) 156 : cluster [DBG] monmap e3: 3 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0],c=[v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0]} removed_ranks: {} 2024-02-23T00:35:15.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:14.744293+0000 mon.a (mon.0) 157 : cluster [DBG] fsmap 2024-02-23T00:35:15.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:14.744344+0000 mon.a (mon.0) 158 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-23T00:35:15.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:14.744578+0000 mon.a (mon.0) 159 : cluster [DBG] mgrmap e13: y(active, since 82s) 2024-02-23T00:35:15.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:14.744879+0000 mon.a (mon.0) 160 : cluster [INF] overall HEALTH_OK 2024-02-23T00:35:15.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:14.749233+0000 mon.a (mon.0) 161 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:15.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:14.750602+0000 mon.a (mon.0) 162 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:35:16.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:15.709032+0000 mon.a (mon.0) 163 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-23T00:35:16.056 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:15.709032+0000 mon.a (mon.0) 163 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-23T00:35:16.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:15.709032+0000 mon.a (mon.0) 163 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-23T00:35:17.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:15.958265+0000 mon.a (mon.0) 164 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:17.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:16.078979+0000 mgr.y (mgr.14146) 49 : cluster [DBG] pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:17.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:15.958265+0000 mon.a (mon.0) 164 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:17.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:16.078979+0000 mgr.y (mgr.14146) 49 : cluster [DBG] pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:17.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:15.958265+0000 mon.a (mon.0) 164 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:17.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:16.078979+0000 mgr.y (mgr.14146) 49 : cluster [DBG] pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:18.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.423+0000 7feb44c63700 1 -- 172.21.15.196:0/4028615351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb400fd5d0 msgr2=0x7feb400fd9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:35:18.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.423+0000 7feb44c63700 1 --2- 172.21.15.196:0/4028615351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb400fd5d0 0x7feb400fd9b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7feb30004770 tx=0x7feb30009cb0).stop 2024-02-23T00:35:18.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.423+0000 7feb44c63700 1 -- 172.21.15.196:0/4028615351 shutdown_connections 2024-02-23T00:35:18.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.423+0000 7feb44c63700 1 --2- 172.21.15.196:0/4028615351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb400fd5d0 0x7feb400fd9b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:18.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.423+0000 7feb44c63700 1 -- 172.21.15.196:0/4028615351 >> 172.21.15.196:0/4028615351 conn(0x7feb400f1890 msgr2=0x7feb400f3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:35:18.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.423+0000 7feb44c63700 1 -- 172.21.15.196:0/4028615351 shutdown_connections 2024-02-23T00:35:18.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.423+0000 7feb44c63700 1 -- 172.21.15.196:0/4028615351 wait complete. 2024-02-23T00:35:18.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.424+0000 7feb44c63700 1 librados: starting msgr at 2024-02-23T00:35:18.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.424+0000 7feb44c63700 1 librados: starting objecter 2024-02-23T00:35:18.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.424+0000 7feb44c63700 1 Processor -- start 2024-02-23T00:35:18.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.424+0000 7feb44c63700 1 -- start start 2024-02-23T00:35:18.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.424+0000 7feb44c63700 1 librados: setting wanted keys 2024-02-23T00:35:18.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.424+0000 7feb44c63700 1 librados: calling monclient init 2024-02-23T00:35:18.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.424+0000 7feb44c63700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb400fd5d0 0x7feb4010d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:35:18.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.424+0000 7feb44c63700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb4010d6d0 0x7feb40107170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:35:18.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.424+0000 7feb44c63700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb40107660 0x7feb40107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:35:18.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.424+0000 7feb44c63700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4010dd40 con 0x7feb40107660 2024-02-23T00:35:18.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.424+0000 7feb44c63700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb40114010 con 0x7feb4010d6d0 2024-02-23T00:35:18.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.424+0000 7feb44c63700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb40114150 con 0x7feb400fd5d0 2024-02-23T00:35:18.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.425+0000 7feb3dd9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb4010d6d0 0x7feb40107170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:35:18.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.425+0000 7feb3dd9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb4010d6d0 0x7feb40107170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:60414/0 (socket says 172.21.15.196:60414) 2024-02-23T00:35:18.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.425+0000 7feb3dd9b700 1 -- 172.21.15.196:0/54537304 learned_addr learned my addr 172.21.15.196:0/54537304 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-02-23T00:35:18.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.425+0000 7feb3ed9d700 1 --2- 172.21.15.196:0/54537304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb40107660 0x7feb40107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:35:18.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.425+0000 7feb3e59c700 1 --2- 172.21.15.196:0/54537304 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb400fd5d0 0x7feb4010d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:35:18.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.425+0000 7feb3dd9b700 1 -- 172.21.15.196:0/54537304 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb400fd5d0 msgr2=0x7feb4010d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:35:18.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.425+0000 7feb3dd9b700 1 --2- 172.21.15.196:0/54537304 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb400fd5d0 0x7feb4010d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:18.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.425+0000 7feb3dd9b700 1 -- 172.21.15.196:0/54537304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb40107660 msgr2=0x7feb40107ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:35:18.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.425+0000 7feb3dd9b700 1 --2- 172.21.15.196:0/54537304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb40107660 0x7feb40107ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:18.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.425+0000 7feb3dd9b700 1 -- 172.21.15.196:0/54537304 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb30005040 con 0x7feb4010d6d0 2024-02-23T00:35:18.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.425+0000 7feb3ed9d700 1 --2- 172.21.15.196:0/54537304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb40107660 0x7feb40107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:35:18.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.425+0000 7feb3dd9b700 1 --2- 172.21.15.196:0/54537304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb4010d6d0 0x7feb40107170 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7feb3400b5d0 tx=0x7feb340090f0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:35:18.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.425+0000 7feb2f7fe700 10 client.?.objecter ms_handle_connect 0x7feb4010d6d0 2024-02-23T00:35:18.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.425+0000 7feb2f7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:35:18.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.425+0000 7feb2f7fe700 1 -- 172.21.15.196:0/54537304 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3400f510 con 0x7feb4010d6d0 2024-02-23T00:35:18.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.425+0000 7feb2f7fe700 1 -- 172.21.15.196:0/54537304 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7feb3400fb00 con 0x7feb4010d6d0 2024-02-23T00:35:18.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.425+0000 7feb44c63700 1 -- 172.21.15.196:0/54537304 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb400f6d40 con 0x7feb4010d6d0 2024-02-23T00:35:18.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.425+0000 7feb2f7fe700 1 -- 172.21.15.196:0/54537304 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb34011970 con 0x7feb4010d6d0 2024-02-23T00:35:18.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.425+0000 7feb44c63700 10 client.24100.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:35:18.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.425+0000 7feb44c63700 1 -- 172.21.15.196:0/54537304 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb400f7300 con 0x7feb4010d6d0 2024-02-23T00:35:18.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.425+0000 7feb44c63700 1 librados: init done 2024-02-23T00:35:18.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.426+0000 7feb2f7fe700 1 -- 172.21.15.196:0/54537304 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 13) v1 ==== 38673+0+0 (secure 0 0 0) 0x7feb3401a400 con 0x7feb4010d6d0 2024-02-23T00:35:18.432 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.426+0000 7feb2f7fe700 1 --2- 172.21.15.196:0/54537304 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7feb28034530 0x7feb280369f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:35:18.432 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.427+0000 7feb2f7fe700 1 -- 172.21.15.196:0/54537304 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1098+0+0 (secure 0 0 0) 0x7feb34042c20 con 0x7feb4010d6d0 2024-02-23T00:35:18.432 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.427+0000 7feb2f7fe700 10 client.24100.objecter ms_dispatch 0x7feb40001550 osd_map(4..4 src has 1..4) v4 2024-02-23T00:35:18.432 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.427+0000 7feb44c63700 1 -- 172.21.15.196:0/54537304 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb4005f1f0 con 0x7feb4010d6d0 2024-02-23T00:35:18.432 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.427+0000 7feb2f7fe700 3 client.24100.objecter handle_osd_map got epochs [4,4] > 0 2024-02-23T00:35:18.432 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.427+0000 7feb2f7fe700 3 client.24100.objecter handle_osd_map decoding full epoch 4 2024-02-23T00:35:18.432 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.427+0000 7feb2f7fe700 20 client.24100.objecter dump_active .. 0 homeless 2024-02-23T00:35:18.432 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.427+0000 7feb3e59c700 1 --2- 172.21.15.196:0/54537304 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7feb28034530 0x7feb280369f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:35:18.432 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.427+0000 7feb3e59c700 1 --2- 172.21.15.196:0/54537304 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7feb28034530 0x7feb280369f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7feb30004e90 tx=0x7feb30009cb0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:35:18.432 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.427+0000 7feb2f7fe700 10 client.24100.objecter ms_handle_connect 0x7feb28034530 2024-02-23T00:35:18.433 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.432+0000 7feb2f7fe700 1 -- 172.21.15.196:0/54537304 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7feb3400e260 con 0x7feb4010d6d0 2024-02-23T00:35:18.442 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:17.209190+0000 mon.a (mon.0) 165 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:18.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:17.209190+0000 mon.a (mon.0) 165 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:18.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:17.209190+0000 mon.a (mon.0) 165 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:18.684 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.683+0000 7feb44c63700 1 -- 172.21.15.196:0/54537304 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7feb4005a310 con 0x7feb4010d6d0 2024-02-23T00:35:18.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.684+0000 7feb2f7fe700 1 -- 172.21.15.196:0/54537304 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1259 (secure 0 0 0) 0x7feb3400e260 con 0x7feb4010d6d0 2024-02-23T00:35:18.685 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-23T00:35:18.685 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":3,"fsid":"fce2617e-d1e2-11ee-95c0-87774f69a715","modified":"2024-02-23T00:35:09.708953Z","created":"2024-02-23T00:32:52.273607Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-02-23T00:35:18.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.685+0000 7feb44c63700 10 librados: watch_flush enter 2024-02-23T00:35:18.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.686+0000 7feb44c63700 10 librados: watch_flush exit 2024-02-23T00:35:18.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.686+0000 7feb44c63700 20 client.24100.objecter shutdown clearing up homeless session... 2024-02-23T00:35:18.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.686+0000 7feb44c63700 10 client.24100.objecter successfully canceled tick 2024-02-23T00:35:18.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.686+0000 7feb44c63700 1 -- 172.21.15.196:0/54537304 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7feb28034530 msgr2=0x7feb280369f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:35:18.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.686+0000 7feb44c63700 1 --2- 172.21.15.196:0/54537304 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7feb28034530 0x7feb280369f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7feb30004e90 tx=0x7feb30009cb0).stop 2024-02-23T00:35:18.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.686+0000 7feb44c63700 1 -- 172.21.15.196:0/54537304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb4010d6d0 msgr2=0x7feb40107170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:35:18.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.686+0000 7feb44c63700 1 --2- 172.21.15.196:0/54537304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb4010d6d0 0x7feb40107170 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7feb3400b5d0 tx=0x7feb340090f0).stop 2024-02-23T00:35:18.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.686+0000 7feb44c63700 1 -- 172.21.15.196:0/54537304 shutdown_connections 2024-02-23T00:35:18.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.686+0000 7feb44c63700 1 --2- 172.21.15.196:0/54537304 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7feb28034530 0x7feb280369f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:18.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.686+0000 7feb44c63700 1 --2- 172.21.15.196:0/54537304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb40107660 0x7feb40107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:18.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.686+0000 7feb44c63700 1 --2- 172.21.15.196:0/54537304 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb400fd5d0 0x7feb4010d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:18.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.686+0000 7feb44c63700 1 --2- 172.21.15.196:0/54537304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb4010d6d0 0x7feb40107170 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:18.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.686+0000 7feb44c63700 1 -- 172.21.15.196:0/54537304 >> 172.21.15.196:0/54537304 conn(0x7feb400f1890 msgr2=0x7feb400f1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:35:18.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.686+0000 7feb44c63700 1 -- 172.21.15.196:0/54537304 shutdown_connections 2024-02-23T00:35:18.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.686+0000 7feb44c63700 1 -- 172.21.15.196:0/54537304 wait complete. 2024-02-23T00:35:18.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:18.686+0000 7feb44c63700 1 librados: shutdown 2024-02-23T00:35:18.688 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 3 2024-02-23T00:35:19.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:18.079317+0000 mgr.y (mgr.14146) 50 : cluster [DBG] pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:19.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:18.684152+0000 mon.b (mon.1) 3 : audit [DBG] from='client.? 172.21.15.196:0/54537304' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-23T00:35:19.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:18.079317+0000 mgr.y (mgr.14146) 50 : cluster [DBG] pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:19.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:18.684152+0000 mon.b (mon.1) 3 : audit [DBG] from='client.? 172.21.15.196:0/54537304' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-23T00:35:19.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:18.079317+0000 mgr.y (mgr.14146) 50 : cluster [DBG] pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:19.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:18.684152+0000 mon.b (mon.1) 3 : audit [DBG] from='client.? 172.21.15.196:0/54537304' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-23T00:35:19.937 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-23T00:35:19.937 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph config generate-minimal-conf 2024-02-23T00:35:20.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:19.619311+0000 mon.a (mon.0) 166 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:20.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:19.620065+0000 mon.a (mon.0) 167 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:20.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:19.620945+0000 mon.a (mon.0) 168 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:35:20.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:35:19.622144+0000 mgr.y (mgr.14146) 51 : cephadm [INF] Updating smithi047:/etc/ceph/ceph.conf 2024-02-23T00:35:20.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:35:19.622447+0000 mgr.y (mgr.14146) 52 : cephadm [INF] Updating smithi196:/etc/ceph/ceph.conf 2024-02-23T00:35:20.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:19.653664+0000 mon.a (mon.0) 169 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:20.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:19.658762+0000 mon.a (mon.0) 170 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:20.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:19.662833+0000 mon.a (mon.0) 171 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:20.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:35:19.664099+0000 mgr.y (mgr.14146) 53 : cephadm [INF] Reconfiguring mon.a (unknown last config time)... 2024-02-23T00:35:20.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:19.664389+0000 mon.a (mon.0) 172 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-23T00:35:20.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:19.665134+0000 mon.a (mon.0) 173 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-23T00:35:20.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:19.665861+0000 mon.a (mon.0) 174 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:20.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:35:19.666576+0000 mgr.y (mgr.14146) 54 : cephadm [INF] Reconfiguring daemon mon.a on smithi047 2024-02-23T00:35:20.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:20.079529+0000 mgr.y (mgr.14146) 55 : cluster [DBG] pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:20.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:19.619311+0000 mon.a (mon.0) 166 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:20.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:19.620065+0000 mon.a (mon.0) 167 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:20.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:19.620945+0000 mon.a (mon.0) 168 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:35:20.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:35:19.622144+0000 mgr.y (mgr.14146) 51 : cephadm [INF] Updating smithi047:/etc/ceph/ceph.conf 2024-02-23T00:35:20.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:35:19.622447+0000 mgr.y (mgr.14146) 52 : cephadm [INF] Updating smithi196:/etc/ceph/ceph.conf 2024-02-23T00:35:20.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:19.653664+0000 mon.a (mon.0) 169 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:20.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:19.658762+0000 mon.a (mon.0) 170 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:20.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:19.662833+0000 mon.a (mon.0) 171 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:20.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:35:19.664099+0000 mgr.y (mgr.14146) 53 : cephadm [INF] Reconfiguring mon.a (unknown last config time)... 2024-02-23T00:35:20.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:19.664389+0000 mon.a (mon.0) 172 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-23T00:35:20.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:19.665134+0000 mon.a (mon.0) 173 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-23T00:35:20.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:19.665861+0000 mon.a (mon.0) 174 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:20.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:35:19.666576+0000 mgr.y (mgr.14146) 54 : cephadm [INF] Reconfiguring daemon mon.a on smithi047 2024-02-23T00:35:20.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:20.079529+0000 mgr.y (mgr.14146) 55 : cluster [DBG] pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:20.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:19.619311+0000 mon.a (mon.0) 166 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:20.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:19.620065+0000 mon.a (mon.0) 167 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:20.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:19.620945+0000 mon.a (mon.0) 168 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:35:20.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:35:19.622144+0000 mgr.y (mgr.14146) 51 : cephadm [INF] Updating smithi047:/etc/ceph/ceph.conf 2024-02-23T00:35:20.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:35:19.622447+0000 mgr.y (mgr.14146) 52 : cephadm [INF] Updating smithi196:/etc/ceph/ceph.conf 2024-02-23T00:35:20.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:19.653664+0000 mon.a (mon.0) 169 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:20.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:19.658762+0000 mon.a (mon.0) 170 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:20.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:19.662833+0000 mon.a (mon.0) 171 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:20.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:35:19.664099+0000 mgr.y (mgr.14146) 53 : cephadm [INF] Reconfiguring mon.a (unknown last config time)... 2024-02-23T00:35:20.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:19.664389+0000 mon.a (mon.0) 172 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-23T00:35:20.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:19.665134+0000 mon.a (mon.0) 173 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-23T00:35:20.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:19.665861+0000 mon.a (mon.0) 174 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:20.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:35:19.666576+0000 mgr.y (mgr.14146) 54 : cephadm [INF] Reconfiguring daemon mon.a on smithi047 2024-02-23T00:35:20.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:20.079529+0000 mgr.y (mgr.14146) 55 : cluster [DBG] pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:23.056 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:21.884031+0000 mon.a (mon.0) 175 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:23.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:35:21.884559+0000 mgr.y (mgr.14146) 56 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2024-02-23T00:35:23.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:21.884848+0000 mon.a (mon.0) 176 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-23T00:35:23.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:21.885554+0000 mon.a (mon.0) 177 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-23T00:35:23.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:21.886208+0000 mon.a (mon.0) 178 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:23.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:35:21.886832+0000 mgr.y (mgr.14146) 57 : cephadm [INF] Reconfiguring daemon mon.c on smithi047 2024-02-23T00:35:23.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:22.079828+0000 mgr.y (mgr.14146) 58 : cluster [DBG] pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:23.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:21.884031+0000 mon.a (mon.0) 175 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:23.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:35:21.884559+0000 mgr.y (mgr.14146) 56 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2024-02-23T00:35:23.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:21.884848+0000 mon.a (mon.0) 176 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-23T00:35:23.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:21.885554+0000 mon.a (mon.0) 177 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-23T00:35:23.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:21.886208+0000 mon.a (mon.0) 178 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:23.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:35:21.886832+0000 mgr.y (mgr.14146) 57 : cephadm [INF] Reconfiguring daemon mon.c on smithi047 2024-02-23T00:35:23.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:22.079828+0000 mgr.y (mgr.14146) 58 : cluster [DBG] pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:23.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:21.884031+0000 mon.a (mon.0) 175 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:23.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:35:21.884559+0000 mgr.y (mgr.14146) 56 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2024-02-23T00:35:23.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:21.884848+0000 mon.a (mon.0) 176 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-23T00:35:23.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:21.885554+0000 mon.a (mon.0) 177 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-23T00:35:23.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:21.886208+0000 mon.a (mon.0) 178 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:23.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:35:21.886832+0000 mgr.y (mgr.14146) 57 : cephadm [INF] Reconfiguring daemon mon.c on smithi047 2024-02-23T00:35:23.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:22.079828+0000 mgr.y (mgr.14146) 58 : cluster [DBG] pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:24.037 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:23.386715+0000 mon.a (mon.0) 179 : audit [DBG] from='client.? 172.21.15.47:0/3938808629' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:24.038 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:23.386715+0000 mon.a (mon.0) 179 : audit [DBG] from='client.? 172.21.15.47:0/3938808629' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:24.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.204+0000 7f0797f8b700 1 -- 172.21.15.47:0/3771454549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07900fdb60 msgr2=0x7f07900fdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:35:24.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.204+0000 7f0797f8b700 1 --2- 172.21.15.47:0/3771454549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07900fdb60 0x7f07900fdf40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f078c004870 tx=0x7f078c009cb0).stop 2024-02-23T00:35:24.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.204+0000 7f0797f8b700 1 -- 172.21.15.47:0/3771454549 shutdown_connections 2024-02-23T00:35:24.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.204+0000 7f0797f8b700 1 --2- 172.21.15.47:0/3771454549 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f07900f6a20 0x7f07901049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:24.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.204+0000 7f0797f8b700 1 --2- 172.21.15.47:0/3771454549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07900fdb60 0x7f07900fdf40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:24.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.204+0000 7f0797f8b700 1 --2- 172.21.15.47:0/3771454549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07900fe4c0 0x7f07900f6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:24.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.204+0000 7f0797f8b700 1 -- 172.21.15.47:0/3771454549 >> 172.21.15.47:0/3771454549 conn(0x7f079006c300 msgr2=0x7f079006c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:35:24.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.204+0000 7f0797f8b700 1 -- 172.21.15.47:0/3771454549 shutdown_connections 2024-02-23T00:35:24.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.204+0000 7f0797f8b700 1 -- 172.21.15.47:0/3771454549 wait complete. 2024-02-23T00:35:24.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.204+0000 7f0797f8b700 1 librados: starting msgr at 2024-02-23T00:35:24.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.204+0000 7f0797f8b700 1 librados: starting objecter 2024-02-23T00:35:24.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.204+0000 7f0797f8b700 1 Processor -- start 2024-02-23T00:35:24.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.204+0000 7f0797f8b700 1 -- start start 2024-02-23T00:35:24.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.204+0000 7f0797f8b700 1 librados: setting wanted keys 2024-02-23T00:35:24.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.204+0000 7f0797f8b700 1 librados: calling monclient init 2024-02-23T00:35:24.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.205+0000 7f0797f8b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07900f6a20 0x7f079010dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:35:24.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.205+0000 7f0797f8b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07900fdb60 0x7f079010e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:35:24.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.205+0000 7f0797f8b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f07900fe4c0 0x7f0790108080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:35:24.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.205+0000 7f0797f8b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0790108570 con 0x7f07900f6a20 2024-02-23T00:35:24.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.205+0000 7f0797f8b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07901086e0 con 0x7f07900fdb60 2024-02-23T00:35:24.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.205+0000 7f0797f8b700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07901088a0 con 0x7f07900fe4c0 2024-02-23T00:35:24.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.205+0000 7f0795d27700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07900f6a20 0x7f079010dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:35:24.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.205+0000 7f0796528700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f07900fe4c0 0x7f0790108080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:35:24.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.205+0000 7f0795d27700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07900f6a20 0x7f079010dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39272/0 (socket says 172.21.15.47:39272) 2024-02-23T00:35:24.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.205+0000 7f0795d27700 1 -- 172.21.15.47:0/3938808629 learned_addr learned my addr 172.21.15.47:0/3938808629 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:35:24.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.205+0000 7f0795526700 1 --2- 172.21.15.47:0/3938808629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07900fdb60 0x7f079010e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:35:24.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.205+0000 7f0795d27700 1 -- 172.21.15.47:0/3938808629 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f07900fe4c0 msgr2=0x7f0790108080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:35:24.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.205+0000 7f0795d27700 1 --2- 172.21.15.47:0/3938808629 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f07900fe4c0 0x7f0790108080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:24.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.205+0000 7f0795d27700 1 -- 172.21.15.47:0/3938808629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07900fdb60 msgr2=0x7f079010e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:35:24.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.205+0000 7f0795d27700 1 --2- 172.21.15.47:0/3938808629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07900fdb60 0x7f079010e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:24.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.205+0000 7f0795d27700 1 -- 172.21.15.47:0/3938808629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07880097c0 con 0x7f07900f6a20 2024-02-23T00:35:24.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.205+0000 7f0795526700 1 --2- 172.21.15.47:0/3938808629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07900fdb60 0x7f079010e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:35:24.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.205+0000 7f0796528700 1 --2- 172.21.15.47:0/3938808629 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f07900fe4c0 0x7f0790108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:35:24.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.205+0000 7f0795d27700 1 --2- 172.21.15.47:0/3938808629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07900f6a20 0x7f079010dee0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f078c009da0 tx=0x7f078c009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:35:24.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.205+0000 7f0786ffd700 10 client.?.objecter ms_handle_connect 0x7f07900f6a20 2024-02-23T00:35:24.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.205+0000 7f0786ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:35:24.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.205+0000 7f0786ffd700 1 -- 172.21.15.47:0/3938808629 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c00e030 con 0x7f07900f6a20 2024-02-23T00:35:24.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.205+0000 7f0797f8b700 1 -- 172.21.15.47:0/3938808629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f078c005040 con 0x7f07900f6a20 2024-02-23T00:35:24.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.205+0000 7f0786ffd700 1 -- 172.21.15.47:0/3938808629 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f078c013e70 con 0x7f07900f6a20 2024-02-23T00:35:24.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.205+0000 7f0786ffd700 1 -- 172.21.15.47:0/3938808629 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c018070 con 0x7f07900f6a20 2024-02-23T00:35:24.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.205+0000 7f0797f8b700 10 client.14193.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:35:24.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.205+0000 7f0797f8b700 1 -- 172.21.15.47:0/3938808629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0790108d40 con 0x7f07900f6a20 2024-02-23T00:35:24.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.205+0000 7f0797f8b700 1 librados: init done 2024-02-23T00:35:24.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.206+0000 7f0786ffd700 1 -- 172.21.15.47:0/3938808629 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 38673+0+0 (secure 0 0 0) 0x7f078c027440 con 0x7f07900f6a20 2024-02-23T00:35:24.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.206+0000 7f0786ffd700 1 --2- 172.21.15.47:0/3938808629 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f077c034270 0x7f077c036730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:35:24.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.206+0000 7f0786ffd700 1 -- 172.21.15.47:0/3938808629 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f078c044210 con 0x7f07900f6a20 2024-02-23T00:35:24.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.206+0000 7f0786ffd700 10 client.14193.objecter ms_dispatch 0x7f0790001550 osd_map(4..4 src has 1..4) v4 2024-02-23T00:35:24.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.206+0000 7f0786ffd700 3 client.14193.objecter handle_osd_map got epochs [4,4] > 0 2024-02-23T00:35:24.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.206+0000 7f0786ffd700 3 client.14193.objecter handle_osd_map decoding full epoch 4 2024-02-23T00:35:24.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.206+0000 7f0786ffd700 20 client.14193.objecter dump_active .. 0 homeless 2024-02-23T00:35:24.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.206+0000 7f0795526700 1 --2- 172.21.15.47:0/3938808629 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f077c034270 0x7f077c036730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:35:24.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.206+0000 7f0797f8b700 1 -- 172.21.15.47:0/3938808629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f079005ffd0 con 0x7f07900f6a20 2024-02-23T00:35:24.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.207+0000 7f0786ffd700 10 client.14193.objecter ms_handle_connect 0x7f077c034270 2024-02-23T00:35:24.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.207+0000 7f0795526700 1 --2- 172.21.15.47:0/3938808629 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f077c034270 0x7f077c036730 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f0790109820 tx=0x7f0780009040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:35:24.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.211+0000 7f0786ffd700 1 -- 172.21.15.47:0/3938808629 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f078c01de00 con 0x7f07900f6a20 2024-02-23T00:35:24.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.385+0000 7f0797f8b700 1 -- 172.21.15.47:0/3938808629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f0790106e20 con 0x7f07900f6a20 2024-02-23T00:35:24.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.385+0000 7f0786ffd700 1 -- 172.21.15.47:0/3938808629 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7f078c00f270 con 0x7f07900f6a20 2024-02-23T00:35:24.135 INFO:teuthology.orchestra.run.smithi047.stdout:# minimal ceph.conf for fce2617e-d1e2-11ee-95c0-87774f69a715 2024-02-23T00:35:24.136 INFO:teuthology.orchestra.run.smithi047.stdout:[global] 2024-02-23T00:35:24.136 INFO:teuthology.orchestra.run.smithi047.stdout: fsid = fce2617e-d1e2-11ee-95c0-87774f69a715 2024-02-23T00:35:24.136 INFO:teuthology.orchestra.run.smithi047.stdout: mon_host = [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-02-23T00:35:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.387+0000 7f0797f8b700 10 librados: watch_flush enter 2024-02-23T00:35:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.387+0000 7f0797f8b700 10 librados: watch_flush exit 2024-02-23T00:35:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.387+0000 7f0797f8b700 20 client.14193.objecter shutdown clearing up homeless session... 2024-02-23T00:35:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.387+0000 7f0797f8b700 10 client.14193.objecter successfully canceled tick 2024-02-23T00:35:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.387+0000 7f0797f8b700 1 -- 172.21.15.47:0/3938808629 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f077c034270 msgr2=0x7f077c036730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:35:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.387+0000 7f0797f8b700 1 --2- 172.21.15.47:0/3938808629 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f077c034270 0x7f077c036730 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f0790109820 tx=0x7f0780009040).stop 2024-02-23T00:35:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.387+0000 7f0797f8b700 1 -- 172.21.15.47:0/3938808629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07900f6a20 msgr2=0x7f079010dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:35:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.387+0000 7f0797f8b700 1 --2- 172.21.15.47:0/3938808629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07900f6a20 0x7f079010dee0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f078c009da0 tx=0x7f078c009cb0).stop 2024-02-23T00:35:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.387+0000 7f0797f8b700 1 -- 172.21.15.47:0/3938808629 shutdown_connections 2024-02-23T00:35:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.387+0000 7f0797f8b700 1 --2- 172.21.15.47:0/3938808629 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f07900fe4c0 0x7f0790108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.387+0000 7f0797f8b700 1 --2- 172.21.15.47:0/3938808629 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f077c034270 0x7f077c036730 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.387+0000 7f0797f8b700 1 --2- 172.21.15.47:0/3938808629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07900f6a20 0x7f079010dee0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.387+0000 7f0797f8b700 1 --2- 172.21.15.47:0/3938808629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07900fdb60 0x7f079010e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.387+0000 7f0797f8b700 1 -- 172.21.15.47:0/3938808629 >> 172.21.15.47:0/3938808629 conn(0x7f079006c300 msgr2=0x7f07900fb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:35:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.387+0000 7f0797f8b700 1 -- 172.21.15.47:0/3938808629 shutdown_connections 2024-02-23T00:35:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.387+0000 7f0797f8b700 1 -- 172.21.15.47:0/3938808629 wait complete. 2024-02-23T00:35:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:23.387+0000 7f0797f8b700 1 librados: shutdown 2024-02-23T00:35:24.192 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:23.386715+0000 mon.a (mon.0) 179 : audit [DBG] from='client.? 172.21.15.47:0/3938808629' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:24.780 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-23T00:35:24.780 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-02-23T00:35:24.780 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.conf 2024-02-23T00:35:24.814 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-02-23T00:35:24.815 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-23T00:35:24.885 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-02-23T00:35:24.885 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.conf 2024-02-23T00:35:24.917 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-02-23T00:35:24.917 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-23T00:35:24.983 INFO:tasks.cephadm:Adding mgr.y on smithi047 2024-02-23T00:35:24.984 INFO:tasks.cephadm:Adding mgr.x on smithi196 2024-02-23T00:35:24.984 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph orch apply mgr '2;smithi047=y;smithi196=x' 2024-02-23T00:35:25.192 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:24.080092+0000 mgr.y (mgr.14146) 59 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:25.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:24.087719+0000 mon.a (mon.0) 180 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:25.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:35:24.088291+0000 mgr.y (mgr.14146) 60 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2024-02-23T00:35:25.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:24.088532+0000 mon.a (mon.0) 181 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-23T00:35:25.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:24.089223+0000 mon.a (mon.0) 182 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-23T00:35:25.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:24.089829+0000 mon.a (mon.0) 183 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:25.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:35:24.090385+0000 mgr.y (mgr.14146) 61 : cephadm [INF] Reconfiguring daemon mon.b on smithi196 2024-02-23T00:35:25.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:24.080092+0000 mgr.y (mgr.14146) 59 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:25.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:24.087719+0000 mon.a (mon.0) 180 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:25.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:35:24.088291+0000 mgr.y (mgr.14146) 60 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2024-02-23T00:35:25.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:24.088532+0000 mon.a (mon.0) 181 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-23T00:35:25.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:24.089223+0000 mon.a (mon.0) 182 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-23T00:35:25.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:24.089829+0000 mon.a (mon.0) 183 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:25.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:35:24.090385+0000 mgr.y (mgr.14146) 61 : cephadm [INF] Reconfiguring daemon mon.b on smithi196 2024-02-23T00:35:25.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:24.080092+0000 mgr.y (mgr.14146) 59 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:25.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:24.087719+0000 mon.a (mon.0) 180 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:25.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:35:24.088291+0000 mgr.y (mgr.14146) 60 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2024-02-23T00:35:25.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:24.088532+0000 mon.a (mon.0) 181 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-23T00:35:25.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:24.089223+0000 mon.a (mon.0) 182 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-23T00:35:25.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:24.089829+0000 mon.a (mon.0) 183 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:25.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:35:24.090385+0000 mgr.y (mgr.14146) 61 : cephadm [INF] Reconfiguring daemon mon.b on smithi196 2024-02-23T00:35:26.887 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:25.585225+0000 mon.a (mon.0) 184 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:26.887 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:25.586193+0000 mon.a (mon.0) 185 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:35:26.887 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:25.587766+0000 mon.a (mon.0) 186 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:26.887 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:25.588555+0000 mon.a (mon.0) 187 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:35:26.887 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:25.592610+0000 mon.a (mon.0) 188 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:26.887 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:26.080421+0000 mgr.y (mgr.14146) 62 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:27.056 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:25.585225+0000 mon.a (mon.0) 184 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:27.056 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:25.586193+0000 mon.a (mon.0) 185 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:35:27.056 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:25.587766+0000 mon.a (mon.0) 186 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:27.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:25.588555+0000 mon.a (mon.0) 187 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:35:27.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:25.592610+0000 mon.a (mon.0) 188 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:27.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:26.080421+0000 mgr.y (mgr.14146) 62 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:27.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:25.585225+0000 mon.a (mon.0) 184 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:27.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:25.586193+0000 mon.a (mon.0) 185 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:35:27.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:25.587766+0000 mon.a (mon.0) 186 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:27.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:25.588555+0000 mon.a (mon.0) 187 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:35:27.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:25.592610+0000 mon.a (mon.0) 188 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:27.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:26.080421+0000 mgr.y (mgr.14146) 62 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:27.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.385+0000 7fd2882cf700 1 -- 172.21.15.196:0/2090413461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2800f69a0 msgr2=0x7fd2800f6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:35:27.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.385+0000 7fd2882cf700 1 --2- 172.21.15.196:0/2090413461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2800f69a0 0x7fd2800f6dc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fd27c0048a0 tx=0x7fd27c009cb0).stop 2024-02-23T00:35:27.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.386+0000 7fd2882cf700 1 -- 172.21.15.196:0/2090413461 shutdown_connections 2024-02-23T00:35:27.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.386+0000 7fd2882cf700 1 --2- 172.21.15.196:0/2090413461 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2800f8590 0x7fd2800fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:27.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.386+0000 7fd2882cf700 1 --2- 172.21.15.196:0/2090413461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2800f69a0 0x7fd2800f6dc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:27.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.386+0000 7fd2882cf700 1 --2- 172.21.15.196:0/2090413461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2800f7c40 0x7fd2800f80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:27.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.386+0000 7fd2882cf700 1 -- 172.21.15.196:0/2090413461 >> 172.21.15.196:0/2090413461 conn(0x7fd2800f21a0 msgr2=0x7fd2800f45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:35:27.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.386+0000 7fd2882cf700 1 -- 172.21.15.196:0/2090413461 shutdown_connections 2024-02-23T00:35:27.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.386+0000 7fd2882cf700 1 -- 172.21.15.196:0/2090413461 wait complete. 2024-02-23T00:35:27.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.386+0000 7fd2882cf700 1 librados: starting msgr at 2024-02-23T00:35:27.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.386+0000 7fd2882cf700 1 librados: starting objecter 2024-02-23T00:35:27.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.386+0000 7fd2882cf700 1 Processor -- start 2024-02-23T00:35:27.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.386+0000 7fd2882cf700 1 -- start start 2024-02-23T00:35:27.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.386+0000 7fd2882cf700 1 librados: setting wanted keys 2024-02-23T00:35:27.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.386+0000 7fd2882cf700 1 librados: calling monclient init 2024-02-23T00:35:27.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.386+0000 7fd2882cf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2800f69a0 0x7fd280107dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:35:27.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.386+0000 7fd2882cf700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2800f7c40 0x7fd2801082c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:35:27.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.386+0000 7fd2882cf700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2800f8590 0x7fd28010f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:35:27.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.386+0000 7fd2882cf700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28010f890 con 0x7fd2800f69a0 2024-02-23T00:35:27.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.386+0000 7fd2882cf700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28010f9d0 con 0x7fd2800f8590 2024-02-23T00:35:27.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.386+0000 7fd2882cf700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd28010fb10 con 0x7fd2800f7c40 2024-02-23T00:35:27.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.387+0000 7fd28686c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2800f8590 0x7fd28010f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:35:27.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.387+0000 7fd28686c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2800f8590 0x7fd28010f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:41966/0 (socket says 172.21.15.196:41966) 2024-02-23T00:35:27.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.387+0000 7fd28606b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2800f69a0 0x7fd280107dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:35:27.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.387+0000 7fd28686c700 1 -- 172.21.15.196:0/738124569 learned_addr learned my addr 172.21.15.196:0/738124569 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-02-23T00:35:27.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.387+0000 7fd28586a700 1 --2- 172.21.15.196:0/738124569 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2800f7c40 0x7fd2801082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:35:27.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.387+0000 7fd28686c700 1 -- 172.21.15.196:0/738124569 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2800f7c40 msgr2=0x7fd2801082c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:35:27.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.387+0000 7fd28686c700 1 --2- 172.21.15.196:0/738124569 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2800f7c40 0x7fd2801082c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:27.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.387+0000 7fd28686c700 1 -- 172.21.15.196:0/738124569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2800f69a0 msgr2=0x7fd280107dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:35:27.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.387+0000 7fd28686c700 1 --2- 172.21.15.196:0/738124569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2800f69a0 0x7fd280107dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:27.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.387+0000 7fd28686c700 1 -- 172.21.15.196:0/738124569 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd27c005040 con 0x7fd2800f8590 2024-02-23T00:35:27.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.387+0000 7fd28686c700 1 --2- 172.21.15.196:0/738124569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2800f8590 0x7fd28010f3a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fd27800d970 tx=0x7fd278004e60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:35:27.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.387+0000 7fd277779700 10 client.?.objecter ms_handle_connect 0x7fd2800f8590 2024-02-23T00:35:27.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.387+0000 7fd277779700 10 client.?.objecter resend_mon_ops 2024-02-23T00:35:27.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.387+0000 7fd277779700 1 -- 172.21.15.196:0/738124569 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd278004020 con 0x7fd2800f8590 2024-02-23T00:35:27.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.387+0000 7fd2882cf700 1 -- 172.21.15.196:0/738124569 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd28010fdf0 con 0x7fd2800f8590 2024-02-23T00:35:27.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.387+0000 7fd277779700 1 -- 172.21.15.196:0/738124569 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fd27800f040 con 0x7fd2800f8590 2024-02-23T00:35:27.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.387+0000 7fd277779700 1 -- 172.21.15.196:0/738124569 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd278003430 con 0x7fd2800f8590 2024-02-23T00:35:27.394 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.387+0000 7fd2882cf700 10 client.24109.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:35:27.394 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.387+0000 7fd2882cf700 1 -- 172.21.15.196:0/738124569 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd280110260 con 0x7fd2800f8590 2024-02-23T00:35:27.394 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.387+0000 7fd2882cf700 1 librados: init done 2024-02-23T00:35:27.394 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.388+0000 7fd277779700 1 -- 172.21.15.196:0/738124569 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 13) v1 ==== 38673+0+0 (secure 0 0 0) 0x7fd27800b510 con 0x7fd2800f8590 2024-02-23T00:35:27.394 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.388+0000 7fd277779700 1 --2- 172.21.15.196:0/738124569 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fd26c034270 0x7fd26c036730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:35:27.394 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.388+0000 7fd2882cf700 1 -- 172.21.15.196:0/738124569 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2801015a0 con 0x7fd2800f8590 2024-02-23T00:35:27.394 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.388+0000 7fd277779700 1 -- 172.21.15.196:0/738124569 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1098+0+0 (secure 0 0 0) 0x7fd278042880 con 0x7fd2800f8590 2024-02-23T00:35:27.394 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.388+0000 7fd277779700 10 client.24109.objecter ms_dispatch 0x7fd280001550 osd_map(4..4 src has 1..4) v4 2024-02-23T00:35:27.394 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.388+0000 7fd277779700 3 client.24109.objecter handle_osd_map got epochs [4,4] > 0 2024-02-23T00:35:27.394 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.388+0000 7fd277779700 3 client.24109.objecter handle_osd_map decoding full epoch 4 2024-02-23T00:35:27.394 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.388+0000 7fd277779700 20 client.24109.objecter dump_active .. 0 homeless 2024-02-23T00:35:27.394 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.388+0000 7fd28606b700 1 --2- 172.21.15.196:0/738124569 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fd26c034270 0x7fd26c036730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:35:27.394 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.389+0000 7fd277779700 10 client.24109.objecter ms_handle_connect 0x7fd26c034270 2024-02-23T00:35:27.394 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.389+0000 7fd28606b700 1 --2- 172.21.15.196:0/738124569 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fd26c034270 0x7fd26c036730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fd27c004fa0 tx=0x7fd27c009cb0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:35:27.395 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.393+0000 7fd277779700 1 -- 172.21.15.196:0/738124569 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fd278003820 con 0x7fd2800f8590 2024-02-23T00:35:27.566 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.565+0000 7fd2882cf700 1 -- 172.21.15.196:0/738124569 --> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi047=y;smithi196=x", "target": ["mon-mgr", ""]}) v1 -- 0x7fd280061150 con 0x7fd26c034270 2024-02-23T00:35:27.573 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.571+0000 7fd277779700 1 -- 172.21.15.196:0/738124569 <== mgr.14146 v2:172.21.15.47:6800/622994871 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd280061150 con 0x7fd26c034270 2024-02-23T00:35:27.573 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mgr update... 2024-02-23T00:35:27.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.573+0000 7fd2882cf700 10 librados: watch_flush enter 2024-02-23T00:35:27.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.573+0000 7fd2882cf700 10 librados: watch_flush exit 2024-02-23T00:35:27.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.573+0000 7fd2882cf700 20 client.24109.objecter shutdown clearing up homeless session... 2024-02-23T00:35:27.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.573+0000 7fd2882cf700 10 client.24109.objecter successfully canceled tick 2024-02-23T00:35:27.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.573+0000 7fd2882cf700 1 -- 172.21.15.196:0/738124569 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fd26c034270 msgr2=0x7fd26c036730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:35:27.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.573+0000 7fd2882cf700 1 --2- 172.21.15.196:0/738124569 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fd26c034270 0x7fd26c036730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fd27c004fa0 tx=0x7fd27c009cb0).stop 2024-02-23T00:35:27.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.573+0000 7fd2882cf700 1 -- 172.21.15.196:0/738124569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2800f8590 msgr2=0x7fd28010f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:35:27.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.573+0000 7fd2882cf700 1 --2- 172.21.15.196:0/738124569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2800f8590 0x7fd28010f3a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fd27800d970 tx=0x7fd278004e60).stop 2024-02-23T00:35:27.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.573+0000 7fd2882cf700 1 -- 172.21.15.196:0/738124569 shutdown_connections 2024-02-23T00:35:27.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.573+0000 7fd2882cf700 1 --2- 172.21.15.196:0/738124569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2800f8590 0x7fd28010f3a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:27.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.573+0000 7fd2882cf700 1 --2- 172.21.15.196:0/738124569 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fd26c034270 0x7fd26c036730 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:27.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.573+0000 7fd2882cf700 1 --2- 172.21.15.196:0/738124569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2800f69a0 0x7fd280107dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:27.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.573+0000 7fd2882cf700 1 --2- 172.21.15.196:0/738124569 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2800f7c40 0x7fd2801082c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:27.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.573+0000 7fd2882cf700 1 -- 172.21.15.196:0/738124569 >> 172.21.15.196:0/738124569 conn(0x7fd2800f21a0 msgr2=0x7fd280060a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:35:27.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.573+0000 7fd2882cf700 1 -- 172.21.15.196:0/738124569 shutdown_connections 2024-02-23T00:35:27.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.573+0000 7fd2882cf700 1 -- 172.21.15.196:0/738124569 wait complete. 2024-02-23T00:35:27.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:35:27.574+0000 7fd2882cf700 1 librados: shutdown 2024-02-23T00:35:28.287 DEBUG:teuthology.orchestra.run.smithi196:mgr.x> sudo journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@mgr.x.service 2024-02-23T00:35:28.289 INFO:tasks.cephadm:Deploying OSDs... 2024-02-23T00:35:28.290 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-02-23T00:35:28.290 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/scratch_devs of=/dev/stdout 2024-02-23T00:35:28.314 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-23T00:35:28.314 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_1 2024-02-23T00:35:28.319 INFO:journalctl@ceph.mgr.x.smithi196.stdout:-- Logs begin at Fri 2024-02-23 00:13:21 UTC. -- 2024-02-23T00:35:28.375 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-23T00:35:28.375 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-23T00:35:28.375 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 6h/6d Inode: 159679 Links: 1 2024-02-23T00:35:28.375 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-23T00:35:28.376 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-02-23T00:35:28.376 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-02-23 00:35:09.088980696 +0000 2024-02-23T00:35:28.376 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-02-23 00:23:30.960744237 +0000 2024-02-23T00:35:28.376 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-02-23 00:23:30.960744237 +0000 2024-02-23T00:35:28.376 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: - 2024-02-23T00:35:28.376 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-23T00:35:28.447 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-02-23T00:35:28.447 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-02-23T00:35:28.447 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000107669 s, 4.8 MB/s 2024-02-23T00:35:28.448 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-23T00:35:28.511 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_2 2024-02-23T00:35:28.572 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-23T00:35:28.572 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-23T00:35:28.573 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 6h/6d Inode: 165405 Links: 1 2024-02-23T00:35:28.573 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-23T00:35:28.573 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-02-23T00:35:28.573 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-02-23 00:35:09.088980696 +0000 2024-02-23T00:35:28.573 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-02-23 00:23:31.301736970 +0000 2024-02-23T00:35:28.573 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-02-23 00:23:31.301736970 +0000 2024-02-23T00:35:28.573 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: - 2024-02-23T00:35:28.573 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-23T00:35:28.642 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-02-23T00:35:28.642 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-02-23T00:35:28.642 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000110229 s, 4.6 MB/s 2024-02-23T00:35:28.643 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-23T00:35:28.704 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_3 2024-02-23T00:35:28.764 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-23T00:35:28.764 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-23T00:35:28.764 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 6h/6d Inode: 167641 Links: 1 2024-02-23T00:35:28.764 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-23T00:35:28.764 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-02-23T00:35:28.764 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-02-23 00:35:09.088980696 +0000 2024-02-23T00:35:28.764 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-02-23 00:23:31.653729468 +0000 2024-02-23T00:35:28.764 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-02-23 00:23:31.653729468 +0000 2024-02-23T00:35:28.765 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: - 2024-02-23T00:35:28.765 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-23T00:35:28.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:27.567386+0000 mgr.y (mgr.14146) 63 : audit [DBG] from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi047=y;smithi196=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:35:28.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:35:27.568838+0000 mgr.y (mgr.14146) 64 : cephadm [INF] Saving service mgr spec with placement smithi047=y;smithi196=x;count:2 2024-02-23T00:35:28.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:27.572498+0000 mon.a (mon.0) 189 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:28.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:27.613246+0000 mon.a (mon.0) 190 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:35:28.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:27.614917+0000 mon.a (mon.0) 191 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:28.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:27.615693+0000 mon.a (mon.0) 192 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:35:28.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:27.620050+0000 mon.a (mon.0) 193 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:28.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:27.621392+0000 mon.a (mon.0) 194 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-23T00:35:28.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:27.624644+0000 mon.a (mon.0) 195 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-23T00:35:28.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:27.625448+0000 mon.a (mon.0) 196 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-23T00:35:28.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:27.626345+0000 mon.a (mon.0) 197 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:28.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:35:27.627032+0000 mgr.y (mgr.14146) 65 : cephadm [INF] Deploying daemon mgr.x on smithi196 2024-02-23T00:35:28.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:28.080785+0000 mgr.y (mgr.14146) 66 : cluster [DBG] pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:28.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:27.567386+0000 mgr.y (mgr.14146) 63 : audit [DBG] from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi047=y;smithi196=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:35:28.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:35:27.568838+0000 mgr.y (mgr.14146) 64 : cephadm [INF] Saving service mgr spec with placement smithi047=y;smithi196=x;count:2 2024-02-23T00:35:28.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:27.572498+0000 mon.a (mon.0) 189 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:28.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:27.613246+0000 mon.a (mon.0) 190 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:35:28.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:27.614917+0000 mon.a (mon.0) 191 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:28.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:27.615693+0000 mon.a (mon.0) 192 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:35:28.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:27.620050+0000 mon.a (mon.0) 193 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:28.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:27.621392+0000 mon.a (mon.0) 194 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-23T00:35:28.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:27.624644+0000 mon.a (mon.0) 195 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-23T00:35:28.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:27.625448+0000 mon.a (mon.0) 196 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-23T00:35:28.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:27.626345+0000 mon.a (mon.0) 197 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:28.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:35:27.627032+0000 mgr.y (mgr.14146) 65 : cephadm [INF] Deploying daemon mgr.x on smithi196 2024-02-23T00:35:28.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:28.080785+0000 mgr.y (mgr.14146) 66 : cluster [DBG] pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:28.832 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-02-23T00:35:28.832 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-02-23T00:35:28.832 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000107668 s, 4.8 MB/s 2024-02-23T00:35:28.833 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-23T00:35:28.894 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_4 2024-02-23T00:35:28.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:27.567386+0000 mgr.y (mgr.14146) 63 : audit [DBG] from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi047=y;smithi196=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:35:28.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:35:27.568838+0000 mgr.y (mgr.14146) 64 : cephadm [INF] Saving service mgr spec with placement smithi047=y;smithi196=x;count:2 2024-02-23T00:35:28.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:27.572498+0000 mon.a (mon.0) 189 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:28.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:27.613246+0000 mon.a (mon.0) 190 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:35:28.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:27.614917+0000 mon.a (mon.0) 191 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:28.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:27.615693+0000 mon.a (mon.0) 192 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:35:28.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:27.620050+0000 mon.a (mon.0) 193 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:28.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:27.621392+0000 mon.a (mon.0) 194 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-23T00:35:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:27.624644+0000 mon.a (mon.0) 195 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-23T00:35:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:27.625448+0000 mon.a (mon.0) 196 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-23T00:35:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:27.626345+0000 mon.a (mon.0) 197 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:35:27.627032+0000 mgr.y (mgr.14146) 65 : cephadm [INF] Deploying daemon mgr.x on smithi196 2024-02-23T00:35:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:28.080785+0000 mgr.y (mgr.14146) 66 : cluster [DBG] pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:28.953 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-23T00:35:28.954 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-23T00:35:28.954 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 6h/6d Inode: 167714 Links: 1 2024-02-23T00:35:28.954 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-23T00:35:28.954 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-02-23T00:35:28.954 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-02-23 00:35:09.088980696 +0000 2024-02-23T00:35:28.954 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-02-23 00:23:32.011721839 +0000 2024-02-23T00:35:28.954 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-02-23 00:23:32.011721839 +0000 2024-02-23T00:35:28.954 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: - 2024-02-23T00:35:28.954 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-23T00:35:29.021 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-02-23T00:35:29.022 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-02-23T00:35:29.022 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 9.9196e-05 s, 5.2 MB/s 2024-02-23T00:35:29.023 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-23T00:35:29.083 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-02-23T00:35:29.083 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/scratch_devs of=/dev/stdout 2024-02-23T00:35:29.101 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-23T00:35:29.102 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_1 2024-02-23T00:35:29.160 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-23T00:35:29.160 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-23T00:35:29.160 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 6h/6d Inode: 166420 Links: 1 2024-02-23T00:35:29.160 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-23T00:35:29.160 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-02-23T00:35:29.160 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-02-23 00:35:00.242489994 +0000 2024-02-23T00:35:29.160 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-02-23 00:23:22.825124148 +0000 2024-02-23T00:35:29.160 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-02-23 00:23:22.825124148 +0000 2024-02-23T00:35:29.161 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: - 2024-02-23T00:35:29.161 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-23T00:35:29.228 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-02-23T00:35:29.228 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-02-23T00:35:29.228 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000259526 s, 2.0 MB/s 2024-02-23T00:35:29.229 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-23T00:35:29.289 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_2 2024-02-23T00:35:29.347 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-23T00:35:29.347 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-23T00:35:29.348 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 6h/6d Inode: 159492 Links: 1 2024-02-23T00:35:29.348 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-23T00:35:29.348 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-02-23T00:35:29.348 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-02-23 00:35:00.242489994 +0000 2024-02-23T00:35:29.348 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-02-23 00:23:23.176116571 +0000 2024-02-23T00:35:29.348 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-02-23 00:23:23.176116571 +0000 2024-02-23T00:35:29.348 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: - 2024-02-23T00:35:29.348 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-23T00:35:29.413 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-02-23T00:35:29.413 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-02-23T00:35:29.413 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.00023468 s, 2.2 MB/s 2024-02-23T00:35:29.414 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-23T00:35:29.473 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_3 2024-02-23T00:35:29.532 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-23T00:35:29.532 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-23T00:35:29.532 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 6h/6d Inode: 166521 Links: 1 2024-02-23T00:35:29.532 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-23T00:35:29.532 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-02-23T00:35:29.533 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-02-23 00:35:00.243489972 +0000 2024-02-23T00:35:29.533 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-02-23 00:23:23.535108821 +0000 2024-02-23T00:35:29.533 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-02-23 00:23:23.535108821 +0000 2024-02-23T00:35:29.533 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: - 2024-02-23T00:35:29.533 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-23T00:35:29.597 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-02-23T00:35:29.597 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-02-23T00:35:29.597 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000181526 s, 2.8 MB/s 2024-02-23T00:35:29.598 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-23T00:35:29.659 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_4 2024-02-23T00:35:29.718 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-23T00:35:29.718 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-23T00:35:29.718 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 6h/6d Inode: 168659 Links: 1 2024-02-23T00:35:29.718 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-23T00:35:29.718 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-02-23T00:35:29.718 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-02-23 00:35:29.703842449 +0000 2024-02-23T00:35:29.718 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-02-23 00:23:23.896101028 +0000 2024-02-23T00:35:29.718 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-02-23 00:23:23.896101028 +0000 2024-02-23T00:35:29.718 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: - 2024-02-23T00:35:29.719 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-23T00:35:29.872 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-02-23T00:35:29.872 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-02-23T00:35:29.872 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000233518 s, 2.2 MB/s 2024-02-23T00:35:29.873 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-23T00:35:29.893 INFO:tasks.cephadm:Deploying osd.0 on smithi047 with /dev/vg_nvme/lv_4... 2024-02-23T00:35:29.894 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-23T00:35:30.193 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:35:29 smithi196 systemd[1]: Starting Ceph mgr.x for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T00:35:30.506 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:35:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:35:30.421+0000 7f897e32c700 1 -- 172.21.15.196:0/409719398 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55909b1a64e0 con 0x55909b17ac00 2024-02-23T00:35:30.942 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:35:30 smithi196 bash[41072]: 3497102a3f83ec5da8152aa1e2cc71a190297184efc5bfbb43fcada7ba617a75 2024-02-23T00:35:30.943 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:35:30 smithi196 systemd[1]: Started Ceph mgr.x for fce2617e-d1e2-11ee-95c0-87774f69a715. 2024-02-23T00:35:30.943 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:35:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:35:30.551+0000 7f898e927680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-23T00:35:30.943 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:35:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:35:30.640+0000 7f898e927680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-23T00:35:31.313 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:35:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:35:31.035+0000 7f898e927680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-23T00:35:31.313 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:30.081080+0000 mgr.y (mgr.14146) 67 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:31.313 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:30.573546+0000 mon.a (mon.0) 198 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:31.313 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:30.575194+0000 mon.a (mon.0) 199 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:35:31.405 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:30.081080+0000 mgr.y (mgr.14146) 67 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:31.405 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:30.573546+0000 mon.a (mon.0) 198 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:31.405 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:30.575194+0000 mon.a (mon.0) 199 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:35:31.405 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:30.081080+0000 mgr.y (mgr.14146) 67 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:31.405 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:30.573546+0000 mon.a (mon.0) 198 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:31.405 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:30.575194+0000 mon.a (mon.0) 199 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:35:32.106 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:35:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:35:31.814+0000 7f898e927680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-23T00:35:32.107 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:35:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:35:31.897+0000 7f898e927680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-23T00:35:32.276 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-23T00:35:32.276 INFO:teuthology.orchestra.run.smithi047.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-23T00:35:32.276 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10+0 records in 2024-02-23T00:35:32.276 INFO:teuthology.orchestra.run.smithi047.stderr:10+0 records out 2024-02-23T00:35:32.276 INFO:teuthology.orchestra.run.smithi047.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00802096 s, 1.3 GB/s 2024-02-23T00:35:32.276 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping successful for: 2024-02-23T00:35:32.443 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:35:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:35:32.104+0000 7f898e927680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-23T00:35:33.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:31.730370+0000 mon.a (mon.0) 200 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:33.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:31.731043+0000 mon.a (mon.0) 201 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:33.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:31.731765+0000 mon.a (mon.0) 202 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:35:33.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:31.736019+0000 mon.a (mon.0) 203 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:33.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:35:31.737743+0000 mgr.y (mgr.14146) 68 : cephadm [INF] Reconfiguring mgr.y (unknown last config time)... 2024-02-23T00:35:33.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:31.738008+0000 mon.a (mon.0) 204 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-23T00:35:33.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:31.738720+0000 mon.a (mon.0) 205 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-23T00:35:33.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:31.739339+0000 mon.a (mon.0) 206 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:33.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:35:31.739954+0000 mgr.y (mgr.14146) 69 : cephadm [INF] Reconfiguring daemon mgr.y on smithi047 2024-02-23T00:35:33.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:32.081264+0000 mgr.y (mgr.14146) 70 : cluster [DBG] pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:33.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:32.214134+0000 mon.a (mon.0) 207 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:33.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:31.730370+0000 mon.a (mon.0) 200 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:33.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:31.731043+0000 mon.a (mon.0) 201 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:33.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:31.731765+0000 mon.a (mon.0) 202 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:35:33.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:31.736019+0000 mon.a (mon.0) 203 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:33.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:35:31.737743+0000 mgr.y (mgr.14146) 68 : cephadm [INF] Reconfiguring mgr.y (unknown last config time)... 2024-02-23T00:35:33.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:31.738008+0000 mon.a (mon.0) 204 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-23T00:35:33.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:31.738720+0000 mon.a (mon.0) 205 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-23T00:35:33.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:31.739339+0000 mon.a (mon.0) 206 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:33.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:35:31.739954+0000 mgr.y (mgr.14146) 69 : cephadm [INF] Reconfiguring daemon mgr.y on smithi047 2024-02-23T00:35:33.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:32.081264+0000 mgr.y (mgr.14146) 70 : cluster [DBG] pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:33.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:32.214134+0000 mon.a (mon.0) 207 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:33.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:31.730370+0000 mon.a (mon.0) 200 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:33.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:31.731043+0000 mon.a (mon.0) 201 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:33.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:31.731765+0000 mon.a (mon.0) 202 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:35:33.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:31.736019+0000 mon.a (mon.0) 203 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:33.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:35:31.737743+0000 mgr.y (mgr.14146) 68 : cephadm [INF] Reconfiguring mgr.y (unknown last config time)... 2024-02-23T00:35:33.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:31.738008+0000 mon.a (mon.0) 204 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-23T00:35:33.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:31.738720+0000 mon.a (mon.0) 205 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-23T00:35:33.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:31.739339+0000 mon.a (mon.0) 206 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:33.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:35:31.739954+0000 mgr.y (mgr.14146) 69 : cephadm [INF] Reconfiguring daemon mgr.y on smithi047 2024-02-23T00:35:33.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:32.081264+0000 mgr.y (mgr.14146) 70 : cluster [DBG] pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:33.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:32.214134+0000 mon.a (mon.0) 207 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:33.324 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:35:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:35:33.032+0000 7f898e927680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-23T00:35:33.324 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:35:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:35:33.157+0000 7f898e927680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-23T00:35:33.324 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:35:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:35:33.234+0000 7f898e927680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-23T00:35:33.457 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph orch daemon add osd smithi047:vg_nvme/lv_4 2024-02-23T00:35:33.692 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:35:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:35:33.322+0000 7f898e927680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-23T00:35:33.693 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:35:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:35:33.515+0000 7f898e927680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-23T00:35:34.192 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:35:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:35:33.790+0000 7f898e927680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-23T00:35:34.193 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:35:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:35:33.885+0000 7f898e927680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-23T00:35:34.943 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:35:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:35:34.602+0000 7f898e927680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-23T00:35:34.943 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:35:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:35:34.682+0000 7f898e927680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-23T00:35:34.943 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:35:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:35:34.772+0000 7f898e927680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-23T00:35:35.203 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:35:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:35:34.941+0000 7f898e927680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-23T00:35:35.204 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:34.081511+0000 mgr.y (mgr.14146) 71 : cluster [DBG] pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:35.204 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:34.112179+0000 mon.a (mon.0) 208 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:35.204 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:34.113337+0000 mon.a (mon.0) 209 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:35:35.204 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:34.114702+0000 mon.a (mon.0) 210 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:35.204 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:34.115396+0000 mon.a (mon.0) 211 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:35:35.204 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:34.119423+0000 mon.a (mon.0) 212 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:35.426 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:34.081511+0000 mgr.y (mgr.14146) 71 : cluster [DBG] pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:35.426 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:34.112179+0000 mon.a (mon.0) 208 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:35.427 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:34.113337+0000 mon.a (mon.0) 209 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:35:35.427 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:34.114702+0000 mon.a (mon.0) 210 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:35.427 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:34.115396+0000 mon.a (mon.0) 211 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:35:35.427 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:34.119423+0000 mon.a (mon.0) 212 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:35.427 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:34.081511+0000 mgr.y (mgr.14146) 71 : cluster [DBG] pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:35.427 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:34.112179+0000 mon.a (mon.0) 208 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:35.427 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:34.113337+0000 mon.a (mon.0) 209 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:35:35.427 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:34.114702+0000 mon.a (mon.0) 210 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:35.427 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:34.115396+0000 mon.a (mon.0) 211 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:35:35.427 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:34.119423+0000 mon.a (mon.0) 212 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:35.463 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:35:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:35:35.201+0000 7f898e927680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-23T00:35:35.463 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:35:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:35:35.329+0000 7f898e927680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-23T00:35:35.944 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:35:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:35:35.461+0000 7f898e927680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-23T00:35:35.944 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:35:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:35:35.628+0000 7f898e927680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-23T00:35:35.944 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:35:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:35:35.710+0000 7f898e927680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-23T00:35:36.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.002+0000 7f57ba323700 1 -- 172.21.15.47:0/2415333866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57b40f8d20 msgr2=0x7f57b40f9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:35:36.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.002+0000 7f57ba323700 1 --2- 172.21.15.47:0/2415333866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57b40f8d20 0x7f57b40f9180 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f579c004870 tx=0x7f579c009cb0).stop 2024-02-23T00:35:36.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.002+0000 7f57ba323700 1 -- 172.21.15.47:0/2415333866 shutdown_connections 2024-02-23T00:35:36.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.002+0000 7f57ba323700 1 --2- 172.21.15.47:0/2415333866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57b40f9670 0x7f57b40fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:36.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.002+0000 7f57ba323700 1 --2- 172.21.15.47:0/2415333866 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f57b40f7a80 0x7f57b40f7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:36.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.002+0000 7f57ba323700 1 --2- 172.21.15.47:0/2415333866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57b40f8d20 0x7f57b40f9180 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:36.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.002+0000 7f57ba323700 1 -- 172.21.15.47:0/2415333866 >> 172.21.15.47:0/2415333866 conn(0x7f57b40f32a0 msgr2=0x7f57b40f56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:35:36.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.002+0000 7f57ba323700 1 -- 172.21.15.47:0/2415333866 shutdown_connections 2024-02-23T00:35:36.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.002+0000 7f57ba323700 1 -- 172.21.15.47:0/2415333866 wait complete. 2024-02-23T00:35:36.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.003+0000 7f57ba323700 1 librados: starting msgr at 2024-02-23T00:35:36.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.003+0000 7f57ba323700 1 librados: starting objecter 2024-02-23T00:35:36.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.003+0000 7f57ba323700 1 Processor -- start 2024-02-23T00:35:36.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.003+0000 7f57ba323700 1 -- start start 2024-02-23T00:35:36.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.003+0000 7f57ba323700 1 librados: setting wanted keys 2024-02-23T00:35:36.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.003+0000 7f57ba323700 1 librados: calling monclient init 2024-02-23T00:35:36.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.003+0000 7f57ba323700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f57b40f7a80 0x7f57b406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:35:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.003+0000 7f57ba323700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57b40f8d20 0x7f57b406b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:35:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.003+0000 7f57ba323700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57b40f9670 0x7f57b406b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:35:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.004+0000 7f57ba323700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b406bb40 con 0x7f57b40f8d20 2024-02-23T00:35:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.004+0000 7f57ba323700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b406bcb0 con 0x7f57b40f9670 2024-02-23T00:35:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.004+0000 7f57ba323700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b406be70 con 0x7f57b40f7a80 2024-02-23T00:35:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.004+0000 7f57b37fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57b40f8d20 0x7f57b406b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:35:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.004+0000 7f57b3fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f57b40f7a80 0x7f57b406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:35:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.004+0000 7f57b37fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57b40f8d20 0x7f57b406b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54220/0 (socket says 172.21.15.47:54220) 2024-02-23T00:35:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.004+0000 7f57b37fe700 1 -- 172.21.15.47:0/208727751 learned_addr learned my addr 172.21.15.47:0/208727751 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:35:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.004+0000 7f57b88c0700 1 --2- 172.21.15.47:0/208727751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57b40f9670 0x7f57b406b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:35:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.004+0000 7f57b37fe700 1 -- 172.21.15.47:0/208727751 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f57b40f7a80 msgr2=0x7f57b406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:35:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.004+0000 7f57b37fe700 1 --2- 172.21.15.47:0/208727751 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f57b40f7a80 0x7f57b406c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.004+0000 7f57b37fe700 1 -- 172.21.15.47:0/208727751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57b40f9670 msgr2=0x7f57b406b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:35:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.004+0000 7f57b37fe700 1 --2- 172.21.15.47:0/208727751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57b40f9670 0x7f57b406b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.004+0000 7f57b37fe700 1 -- 172.21.15.47:0/208727751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a4005040 con 0x7f57b40f8d20 2024-02-23T00:35:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.004+0000 7f57b3fff700 1 --2- 172.21.15.47:0/208727751 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f57b40f7a80 0x7f57b406c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:35:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.004+0000 7f57b37fe700 1 --2- 172.21.15.47:0/208727751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57b40f8d20 0x7f57b406b030 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f579c013700 tx=0x7f579c009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:35:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.004+0000 7f57b17fa700 10 client.?.objecter ms_handle_connect 0x7f57b40f8d20 2024-02-23T00:35:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.004+0000 7f57b17fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:35:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.004+0000 7f57b17fa700 1 -- 172.21.15.47:0/208727751 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c00e030 con 0x7f57b40f8d20 2024-02-23T00:35:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.004+0000 7f57b17fa700 1 -- 172.21.15.47:0/208727751 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f579c013df0 con 0x7f57b40f8d20 2024-02-23T00:35:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.004+0000 7f57ba323700 1 -- 172.21.15.47:0/208727751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f579c005040 con 0x7f57b40f8d20 2024-02-23T00:35:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.004+0000 7f57b17fa700 1 -- 172.21.15.47:0/208727751 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c018070 con 0x7f57b40f8d20 2024-02-23T00:35:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.004+0000 7f57ba323700 10 client.14211.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:35:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.004+0000 7f57ba323700 1 -- 172.21.15.47:0/208727751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b4112410 con 0x7f57b40f8d20 2024-02-23T00:35:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.004+0000 7f57ba323700 1 librados: init done 2024-02-23T00:35:36.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.005+0000 7f57b17fa700 1 -- 172.21.15.47:0/208727751 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 38673+0+0 (secure 0 0 0) 0x7f579c027440 con 0x7f57b40f8d20 2024-02-23T00:35:36.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.005+0000 7f57ba323700 1 -- 172.21.15.47:0/208727751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b405f360 con 0x7f57b40f8d20 2024-02-23T00:35:36.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.005+0000 7f57b17fa700 1 --2- 172.21.15.47:0/208727751 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f57a0034270 0x7f57a0036730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:35:36.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.005+0000 7f57b17fa700 1 -- 172.21.15.47:0/208727751 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f579c043ff0 con 0x7f57b40f8d20 2024-02-23T00:35:36.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.005+0000 7f57b17fa700 10 client.14211.objecter ms_dispatch 0x7f57b4001550 osd_map(4..4 src has 1..4) v4 2024-02-23T00:35:36.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.005+0000 7f57b17fa700 3 client.14211.objecter handle_osd_map got epochs [4,4] > 0 2024-02-23T00:35:36.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.005+0000 7f57b17fa700 3 client.14211.objecter handle_osd_map decoding full epoch 4 2024-02-23T00:35:36.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.006+0000 7f57b17fa700 20 client.14211.objecter dump_active .. 0 homeless 2024-02-23T00:35:36.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.007+0000 7f57b3fff700 1 --2- 172.21.15.47:0/208727751 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f57a0034270 0x7f57a0036730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:35:36.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.007+0000 7f57b17fa700 10 client.14211.objecter ms_handle_connect 0x7f57a0034270 2024-02-23T00:35:36.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.007+0000 7f57b3fff700 1 --2- 172.21.15.47:0/208727751 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f57a0034270 0x7f57a0036730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f57b40f8b80 tx=0x7f57a400b040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:35:36.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.010+0000 7f57b17fa700 1 -- 172.21.15.47:0/208727751 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f579c072f60 con 0x7f57b40f8d20 2024-02-23T00:35:36.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.118+0000 7f57b17fa700 1 -- 172.21.15.47:0/208727751 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 14) v1 ==== 77064+0+0 (secure 0 0 0) 0x7f579c0263e0 con 0x7f57b40f8d20 2024-02-23T00:35:36.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:36.193+0000 7f57ba323700 1 -- 172.21.15.47:0/208727751 --> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f57b40ff5c0 con 0x7f57a0034270 2024-02-23T00:35:36.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:35.725042+0000 mon.a (mon.0) 213 : cluster [DBG] Standby manager daemon x started 2024-02-23T00:35:36.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:35.725342+0000 mon.b (mon.1) 4 : audit [DBG] from='mgr.? 172.21.15.196:0/2055825704' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-23T00:35:36.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:35.725659+0000 mon.b (mon.1) 5 : audit [DBG] from='mgr.? 172.21.15.196:0/2055825704' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-23T00:35:36.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:35.726546+0000 mon.b (mon.1) 6 : audit [DBG] from='mgr.? 172.21.15.196:0/2055825704' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-23T00:35:36.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:35.726856+0000 mon.b (mon.1) 7 : audit [DBG] from='mgr.? 172.21.15.196:0/2055825704' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-23T00:35:36.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:35.725042+0000 mon.a (mon.0) 213 : cluster [DBG] Standby manager daemon x started 2024-02-23T00:35:36.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:35.725342+0000 mon.b (mon.1) 4 : audit [DBG] from='mgr.? 172.21.15.196:0/2055825704' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-23T00:35:36.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:35.725659+0000 mon.b (mon.1) 5 : audit [DBG] from='mgr.? 172.21.15.196:0/2055825704' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-23T00:35:36.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:35.726546+0000 mon.b (mon.1) 6 : audit [DBG] from='mgr.? 172.21.15.196:0/2055825704' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-23T00:35:36.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:35.726856+0000 mon.b (mon.1) 7 : audit [DBG] from='mgr.? 172.21.15.196:0/2055825704' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-23T00:35:36.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:35.725042+0000 mon.a (mon.0) 213 : cluster [DBG] Standby manager daemon x started 2024-02-23T00:35:36.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:35.725342+0000 mon.b (mon.1) 4 : audit [DBG] from='mgr.? 172.21.15.196:0/2055825704' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-23T00:35:36.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:35.725659+0000 mon.b (mon.1) 5 : audit [DBG] from='mgr.? 172.21.15.196:0/2055825704' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-23T00:35:36.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:35.726546+0000 mon.b (mon.1) 6 : audit [DBG] from='mgr.? 172.21.15.196:0/2055825704' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-23T00:35:36.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:35.726856+0000 mon.b (mon.1) 7 : audit [DBG] from='mgr.? 172.21.15.196:0/2055825704' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-23T00:35:37.438 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:36.081760+0000 mgr.y (mgr.14146) 72 : cluster [DBG] pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:37.438 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:36.120656+0000 mon.a (mon.0) 214 : cluster [DBG] mgrmap e14: y(active, since 104s), standbys: x 2024-02-23T00:35:37.438 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:36.120762+0000 mon.a (mon.0) 215 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-23T00:35:37.438 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:36.195368+0000 mgr.y (mgr.14146) 73 : audit [DBG] from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:35:37.438 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:36.197585+0000 mon.a (mon.0) 216 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-23T00:35:37.438 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:36.199392+0000 mon.a (mon.0) 217 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-23T00:35:37.438 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:36.199882+0000 mon.a (mon.0) 218 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:37.438 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:36.081760+0000 mgr.y (mgr.14146) 72 : cluster [DBG] pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:37.439 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:36.120656+0000 mon.a (mon.0) 214 : cluster [DBG] mgrmap e14: y(active, since 104s), standbys: x 2024-02-23T00:35:37.439 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:36.120762+0000 mon.a (mon.0) 215 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-23T00:35:37.439 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:36.195368+0000 mgr.y (mgr.14146) 73 : audit [DBG] from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:35:37.439 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:36.197585+0000 mon.a (mon.0) 216 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-23T00:35:37.439 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:36.199392+0000 mon.a (mon.0) 217 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-23T00:35:37.439 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:36.199882+0000 mon.a (mon.0) 218 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:37.442 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:36.081760+0000 mgr.y (mgr.14146) 72 : cluster [DBG] pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:37.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:36.120656+0000 mon.a (mon.0) 214 : cluster [DBG] mgrmap e14: y(active, since 104s), standbys: x 2024-02-23T00:35:37.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:36.120762+0000 mon.a (mon.0) 215 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-23T00:35:37.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:36.195368+0000 mgr.y (mgr.14146) 73 : audit [DBG] from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:35:37.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:36.197585+0000 mon.a (mon.0) 216 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-23T00:35:37.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:36.199392+0000 mon.a (mon.0) 217 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-23T00:35:37.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:36.199882+0000 mon.a (mon.0) 218 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:39.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:38.082103+0000 mgr.y (mgr.14146) 74 : cluster [DBG] pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:39.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:38.769181+0000 mon.a (mon.0) 219 : audit [INF] from='client.? 172.21.15.47:0/3610561220' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "380050f9-9001-46dc-bb29-037987b4c8e7"}]: dispatch 2024-02-23T00:35:39.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:38.772652+0000 mon.a (mon.0) 220 : audit [INF] from='client.? 172.21.15.47:0/3610561220' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "380050f9-9001-46dc-bb29-037987b4c8e7"}]': finished 2024-02-23T00:35:39.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:38.772715+0000 mon.a (mon.0) 221 : cluster [DBG] osdmap e5: 1 total, 0 up, 1 in 2024-02-23T00:35:39.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:38.772825+0000 mon.a (mon.0) 222 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-23T00:35:39.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:38.082103+0000 mgr.y (mgr.14146) 74 : cluster [DBG] pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:39.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:38.769181+0000 mon.a (mon.0) 219 : audit [INF] from='client.? 172.21.15.47:0/3610561220' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "380050f9-9001-46dc-bb29-037987b4c8e7"}]: dispatch 2024-02-23T00:35:39.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:38.772652+0000 mon.a (mon.0) 220 : audit [INF] from='client.? 172.21.15.47:0/3610561220' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "380050f9-9001-46dc-bb29-037987b4c8e7"}]': finished 2024-02-23T00:35:39.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:38.772715+0000 mon.a (mon.0) 221 : cluster [DBG] osdmap e5: 1 total, 0 up, 1 in 2024-02-23T00:35:39.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:38.772825+0000 mon.a (mon.0) 222 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-23T00:35:39.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:38.082103+0000 mgr.y (mgr.14146) 74 : cluster [DBG] pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:39.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:38.769181+0000 mon.a (mon.0) 219 : audit [INF] from='client.? 172.21.15.47:0/3610561220' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "380050f9-9001-46dc-bb29-037987b4c8e7"}]: dispatch 2024-02-23T00:35:39.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:38.772652+0000 mon.a (mon.0) 220 : audit [INF] from='client.? 172.21.15.47:0/3610561220' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "380050f9-9001-46dc-bb29-037987b4c8e7"}]': finished 2024-02-23T00:35:39.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:38.772715+0000 mon.a (mon.0) 221 : cluster [DBG] osdmap e5: 1 total, 0 up, 1 in 2024-02-23T00:35:39.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:38.772825+0000 mon.a (mon.0) 222 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-23T00:35:40.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:39.442849+0000 mon.a (mon.0) 223 : audit [DBG] from='client.? 172.21.15.47:0/1073873445' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-23T00:35:40.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:39.442849+0000 mon.a (mon.0) 223 : audit [DBG] from='client.? 172.21.15.47:0/1073873445' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-23T00:35:40.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:39.442849+0000 mon.a (mon.0) 223 : audit [DBG] from='client.? 172.21.15.47:0/1073873445' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-23T00:35:41.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:41.005+0000 7f57b9b22700 10 client.14211.objecter tick 2024-02-23T00:35:41.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:40.082384+0000 mgr.y (mgr.14146) 75 : cluster [DBG] pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:41.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:40.082384+0000 mgr.y (mgr.14146) 75 : cluster [DBG] pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:41.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:40.082384+0000 mgr.y (mgr.14146) 75 : cluster [DBG] pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:43.209 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:42.082715+0000 mgr.y (mgr.14146) 76 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:43.209 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:42.082715+0000 mgr.y (mgr.14146) 76 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:43.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:42.082715+0000 mgr.y (mgr.14146) 76 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:45.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:44.083109+0000 mgr.y (mgr.14146) 77 : cluster [DBG] pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:45.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:44.083109+0000 mgr.y (mgr.14146) 77 : cluster [DBG] pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:45.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:44.083109+0000 mgr.y (mgr.14146) 77 : cluster [DBG] pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:46.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:46.007+0000 7f57b9b22700 10 client.14211.objecter tick 2024-02-23T00:35:46.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:45.701137+0000 mon.a (mon.0) 224 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-23T00:35:46.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:45.701859+0000 mon.a (mon.0) 225 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:46.550 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:45.701137+0000 mon.a (mon.0) 224 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-23T00:35:46.551 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:45.701859+0000 mon.a (mon.0) 225 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:46.551 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:45.701137+0000 mon.a (mon.0) 224 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-23T00:35:46.551 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:45.701859+0000 mon.a (mon.0) 225 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:35:47.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:35:45.702449+0000 mgr.y (mgr.14146) 78 : cephadm [INF] Deploying daemon osd.0 on smithi047 2024-02-23T00:35:47.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:46.083470+0000 mgr.y (mgr.14146) 79 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:47.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:35:45.702449+0000 mgr.y (mgr.14146) 78 : cephadm [INF] Deploying daemon osd.0 on smithi047 2024-02-23T00:35:47.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:46.083470+0000 mgr.y (mgr.14146) 79 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:47.442 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:35:45.702449+0000 mgr.y (mgr.14146) 78 : cephadm [INF] Deploying daemon osd.0 on smithi047 2024-02-23T00:35:47.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:46.083470+0000 mgr.y (mgr.14146) 79 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:49.442 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:48.083737+0000 mgr.y (mgr.14146) 80 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:49.471 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:48.083737+0000 mgr.y (mgr.14146) 80 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:49.472 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:48.083737+0000 mgr.y (mgr.14146) 80 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:51.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:51.008+0000 7f57b9b22700 10 client.14211.objecter tick 2024-02-23T00:35:51.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:50.083993+0000 mgr.y (mgr.14146) 81 : cluster [DBG] pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:51.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:50.083993+0000 mgr.y (mgr.14146) 81 : cluster [DBG] pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:51.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:50.083993+0000 mgr.y (mgr.14146) 81 : cluster [DBG] pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:52.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:51.435196+0000 mon.a (mon.0) 226 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:52.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:51.471704+0000 mon.a (mon.0) 227 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:35:52.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:51.435196+0000 mon.a (mon.0) 226 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:52.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:51.471704+0000 mon.a (mon.0) 227 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:35:52.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:51.435196+0000 mon.a (mon.0) 226 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:52.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:51.471704+0000 mon.a (mon.0) 227 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:35:53.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:52.084223+0000 mgr.y (mgr.14146) 82 : cluster [DBG] pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:53.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:52.084223+0000 mgr.y (mgr.14146) 82 : cluster [DBG] pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:53.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:52.084223+0000 mgr.y (mgr.14146) 82 : cluster [DBG] pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:55.052 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 0 on host 'smithi047' 2024-02-23T00:35:55.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:55.049+0000 7f57b17fa700 1 -- 172.21.15.47:0/208727751 <== mgr.14146 v2:172.21.15.47:6800/622994871 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f57b40ff5c0 con 0x7f57a0034270 2024-02-23T00:35:55.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:55.052+0000 7f57ba323700 10 librados: watch_flush enter 2024-02-23T00:35:55.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:55.052+0000 7f57ba323700 10 librados: watch_flush exit 2024-02-23T00:35:55.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:55.052+0000 7f57ba323700 20 client.14211.objecter shutdown clearing up homeless session... 2024-02-23T00:35:55.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:55.052+0000 7f57ba323700 10 client.14211.objecter successfully canceled tick 2024-02-23T00:35:55.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:55.052+0000 7f57ba323700 1 -- 172.21.15.47:0/208727751 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f57a0034270 msgr2=0x7f57a0036730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:35:55.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:55.052+0000 7f57ba323700 1 --2- 172.21.15.47:0/208727751 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f57a0034270 0x7f57a0036730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f57b40f8b80 tx=0x7f57a400b040).stop 2024-02-23T00:35:55.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:55.052+0000 7f57ba323700 1 -- 172.21.15.47:0/208727751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57b40f8d20 msgr2=0x7f57b406b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:35:55.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:55.052+0000 7f57ba323700 1 --2- 172.21.15.47:0/208727751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57b40f8d20 0x7f57b406b030 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f579c013700 tx=0x7f579c009cb0).stop 2024-02-23T00:35:55.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:55.052+0000 7f57ba323700 1 -- 172.21.15.47:0/208727751 shutdown_connections 2024-02-23T00:35:55.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:55.052+0000 7f57ba323700 1 --2- 172.21.15.47:0/208727751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57b40f9670 0x7f57b406b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:55.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:55.052+0000 7f57ba323700 1 --2- 172.21.15.47:0/208727751 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f57b40f7a80 0x7f57b406c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:55.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:55.052+0000 7f57ba323700 1 --2- 172.21.15.47:0/208727751 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f57a0034270 0x7f57a0036730 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:55.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:55.052+0000 7f57ba323700 1 --2- 172.21.15.47:0/208727751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57b40f8d20 0x7f57b406b030 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:35:55.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:55.052+0000 7f57ba323700 1 -- 172.21.15.47:0/208727751 >> 172.21.15.47:0/208727751 conn(0x7f57b40f32a0 msgr2=0x7f57b40fc770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:35:55.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:55.052+0000 7f57ba323700 1 -- 172.21.15.47:0/208727751 shutdown_connections 2024-02-23T00:35:55.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:55.052+0000 7f57ba323700 1 -- 172.21.15.47:0/208727751 wait complete. 2024-02-23T00:35:55.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:35:55.052+0000 7f57ba323700 1 librados: shutdown 2024-02-23T00:35:55.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:54.084397+0000 mgr.y (mgr.14146) 83 : cluster [DBG] pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:55.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:54.516763+0000 mon.c (mon.2) 2 : audit [INF] from='osd.0 [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-23T00:35:55.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:54.517092+0000 mon.a (mon.0) 228 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-23T00:35:55.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:54.084397+0000 mgr.y (mgr.14146) 83 : cluster [DBG] pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:55.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:54.516763+0000 mon.c (mon.2) 2 : audit [INF] from='osd.0 [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-23T00:35:55.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:54.517092+0000 mon.a (mon.0) 228 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-23T00:35:55.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:54.084397+0000 mgr.y (mgr.14146) 83 : cluster [DBG] pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:55.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:54.516763+0000 mon.c (mon.2) 2 : audit [INF] from='osd.0 [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-23T00:35:55.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:54.517092+0000 mon.a (mon.0) 228 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-23T00:35:55.775 DEBUG:teuthology.orchestra.run.smithi047:osd.0> sudo journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@osd.0.service 2024-02-23T00:35:55.778 INFO:tasks.cephadm:Deploying osd.1 on smithi047 with /dev/vg_nvme/lv_3... 2024-02-23T00:35:55.778 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-23T00:35:55.809 INFO:journalctl@ceph.osd.0.smithi047.stdout:-- Logs begin at Fri 2024-02-23 00:13:17 UTC. -- 2024-02-23T00:35:56.056 INFO:journalctl@ceph.osd.0.smithi047.stdout:Feb 23 00:35:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-0[48294]: debug 2024-02-23T00:35:55.919+0000 7f8861ea7700 -1 osd.0 0 waiting for initial osdmap 2024-02-23T00:35:56.056 INFO:journalctl@ceph.osd.0.smithi047.stdout:Feb 23 00:35:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-0[48294]: debug 2024-02-23T00:35:55.922+0000 7f885e623700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-23T00:35:56.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:54.911397+0000 mon.a (mon.0) 229 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-23T00:35:56.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:54.911489+0000 mon.a (mon.0) 230 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2024-02-23T00:35:56.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:54.911626+0000 mon.a (mon.0) 231 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-23T00:35:56.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:54.912077+0000 mon.c (mon.2) 3 : audit [INF] from='osd.0 [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-02-23T00:35:56.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:54.912410+0000 mon.a (mon.0) 232 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-02-23T00:35:56.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:55.050136+0000 mon.a (mon.0) 233 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:56.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:55.330313+0000 mon.a (mon.0) 234 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:56.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:54.911397+0000 mon.a (mon.0) 229 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-23T00:35:56.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:54.911489+0000 mon.a (mon.0) 230 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2024-02-23T00:35:56.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:54.911626+0000 mon.a (mon.0) 231 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-23T00:35:56.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:54.912077+0000 mon.c (mon.2) 3 : audit [INF] from='osd.0 [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-02-23T00:35:56.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:54.912410+0000 mon.a (mon.0) 232 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-02-23T00:35:56.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:55.050136+0000 mon.a (mon.0) 233 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:56.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:55.330313+0000 mon.a (mon.0) 234 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:56.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:54.911397+0000 mon.a (mon.0) 229 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-23T00:35:56.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:54.911489+0000 mon.a (mon.0) 230 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2024-02-23T00:35:56.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:54.911626+0000 mon.a (mon.0) 231 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-23T00:35:56.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:54.912077+0000 mon.c (mon.2) 3 : audit [INF] from='osd.0 [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-02-23T00:35:56.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:54.912410+0000 mon.a (mon.0) 232 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-02-23T00:35:56.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:55.050136+0000 mon.a (mon.0) 233 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:56.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:55.330313+0000 mon.a (mon.0) 234 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:35:57.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:55.913477+0000 mon.a (mon.0) 235 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-02-23T00:35:57.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:55.913575+0000 mon.a (mon.0) 236 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2024-02-23T00:35:57.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:55.913972+0000 mon.a (mon.0) 237 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-23T00:35:57.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:55.920219+0000 mon.a (mon.0) 238 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-23T00:35:57.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:56.084761+0000 mgr.y (mgr.14146) 84 : cluster [DBG] pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:57.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:55.913477+0000 mon.a (mon.0) 235 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-02-23T00:35:57.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:55.913575+0000 mon.a (mon.0) 236 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2024-02-23T00:35:57.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:55.913972+0000 mon.a (mon.0) 237 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-23T00:35:57.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:55.920219+0000 mon.a (mon.0) 238 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-23T00:35:57.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:56.084761+0000 mgr.y (mgr.14146) 84 : cluster [DBG] pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:57.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:55.913477+0000 mon.a (mon.0) 235 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-02-23T00:35:57.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:55.913575+0000 mon.a (mon.0) 236 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2024-02-23T00:35:57.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:55.913972+0000 mon.a (mon.0) 237 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-23T00:35:57.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:55.920219+0000 mon.a (mon.0) 238 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-23T00:35:57.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:56.084761+0000 mgr.y (mgr.14146) 84 : cluster [DBG] pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-23T00:35:58.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:55.491538+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:35:58.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:55.491587+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:35:58.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:56.917500+0000 mon.a (mon.0) 239 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-23T00:35:58.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:56.923687+0000 mon.a (mon.0) 240 : cluster [INF] osd.0 [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] boot 2024-02-23T00:35:58.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:56.923724+0000 mon.a (mon.0) 241 : cluster [DBG] osdmap e8: 1 total, 1 up, 1 in 2024-02-23T00:35:58.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:35:56.924013+0000 mon.a (mon.0) 242 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-23T00:35:58.287 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:55.491538+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:35:58.287 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:55.491587+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:35:58.287 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:56.917500+0000 mon.a (mon.0) 239 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-23T00:35:58.287 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:56.923687+0000 mon.a (mon.0) 240 : cluster [INF] osd.0 [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] boot 2024-02-23T00:35:58.287 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:56.923724+0000 mon.a (mon.0) 241 : cluster [DBG] osdmap e8: 1 total, 1 up, 1 in 2024-02-23T00:35:58.287 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:35:56.924013+0000 mon.a (mon.0) 242 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-23T00:35:58.288 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:55.491538+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:35:58.288 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:55.491587+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:35:58.288 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:56.917500+0000 mon.a (mon.0) 239 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-23T00:35:58.288 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:56.923687+0000 mon.a (mon.0) 240 : cluster [INF] osd.0 [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] boot 2024-02-23T00:35:58.288 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:56.923724+0000 mon.a (mon.0) 241 : cluster [DBG] osdmap e8: 1 total, 1 up, 1 in 2024-02-23T00:35:58.288 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:35:56.924013+0000 mon.a (mon.0) 242 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-23T00:35:59.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:57.927975+0000 mon.a (mon.0) 243 : cluster [DBG] osdmap e9: 1 total, 1 up, 1 in 2024-02-23T00:35:59.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:35:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:35:58.085043+0000 mgr.y (mgr.14146) 85 : cluster [DBG] pgmap v62: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:35:59.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:57.927975+0000 mon.a (mon.0) 243 : cluster [DBG] osdmap e9: 1 total, 1 up, 1 in 2024-02-23T00:35:59.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:35:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:35:58.085043+0000 mgr.y (mgr.14146) 85 : cluster [DBG] pgmap v62: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:35:59.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:57.927975+0000 mon.a (mon.0) 243 : cluster [DBG] osdmap e9: 1 total, 1 up, 1 in 2024-02-23T00:35:59.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:35:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:35:58.085043+0000 mgr.y (mgr.14146) 85 : cluster [DBG] pgmap v62: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:35:59.624 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-23T00:35:59.624 INFO:teuthology.orchestra.run.smithi047.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-23T00:35:59.624 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10+0 records in 2024-02-23T00:35:59.624 INFO:teuthology.orchestra.run.smithi047.stderr:10+0 records out 2024-02-23T00:35:59.624 INFO:teuthology.orchestra.run.smithi047.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00834291 s, 1.3 GB/s 2024-02-23T00:35:59.624 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping successful for: 2024-02-23T00:36:00.921 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph orch daemon add osd smithi047:vg_nvme/lv_3 2024-02-23T00:36:01.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:00.085457+0000 mgr.y (mgr.14146) 86 : cluster [DBG] pgmap v63: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:01.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:00.565752+0000 mon.a (mon.0) 244 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:01.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:00.566502+0000 mon.a (mon.0) 245 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:36:01.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:00.570008+0000 mon.a (mon.0) 246 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:01.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:00.570811+0000 mon.a (mon.0) 247 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:01.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:00.571440+0000 mon.a (mon.0) 248 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:36:01.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:00.574897+0000 mon.a (mon.0) 249 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:01.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:00.576621+0000 mon.a (mon.0) 250 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:36:01.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:00.577739+0000 mon.a (mon.0) 251 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:01.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:00.578265+0000 mon.a (mon.0) 252 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:36:01.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:00.581919+0000 mon.a (mon.0) 253 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:01.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:00.085457+0000 mgr.y (mgr.14146) 86 : cluster [DBG] pgmap v63: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:01.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:00.565752+0000 mon.a (mon.0) 244 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:01.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:00.566502+0000 mon.a (mon.0) 245 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:36:01.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:00.570008+0000 mon.a (mon.0) 246 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:01.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:00.570811+0000 mon.a (mon.0) 247 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:01.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:00.571440+0000 mon.a (mon.0) 248 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:36:01.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:00.574897+0000 mon.a (mon.0) 249 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:01.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:00.576621+0000 mon.a (mon.0) 250 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:36:01.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:00.577739+0000 mon.a (mon.0) 251 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:01.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:00.578265+0000 mon.a (mon.0) 252 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:36:01.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:00.581919+0000 mon.a (mon.0) 253 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:01.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:00.085457+0000 mgr.y (mgr.14146) 86 : cluster [DBG] pgmap v63: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:01.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:00.565752+0000 mon.a (mon.0) 244 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:01.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:00.566502+0000 mon.a (mon.0) 245 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:36:01.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:00.570008+0000 mon.a (mon.0) 246 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:01.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:00.570811+0000 mon.a (mon.0) 247 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:01.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:00.571440+0000 mon.a (mon.0) 248 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:36:01.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:00.574897+0000 mon.a (mon.0) 249 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:01.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:00.576621+0000 mon.a (mon.0) 250 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:36:01.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:00.577739+0000 mon.a (mon.0) 251 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:01.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:00.578265+0000 mon.a (mon.0) 252 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:36:01.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:00.581919+0000 mon.a (mon.0) 253 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:02.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:36:00.561384+0000 mgr.y (mgr.14146) 87 : cephadm [INF] Detected new or changed devices on smithi047 2024-02-23T00:36:02.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:36:00.566993+0000 mgr.y (mgr.14146) 88 : cephadm [INF] Adjusting osd_memory_target on smithi047 to 16102M 2024-02-23T00:36:02.549 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:36:00.561384+0000 mgr.y (mgr.14146) 87 : cephadm [INF] Detected new or changed devices on smithi047 2024-02-23T00:36:02.549 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:36:00.566993+0000 mgr.y (mgr.14146) 88 : cephadm [INF] Adjusting osd_memory_target on smithi047 to 16102M 2024-02-23T00:36:02.550 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:36:00.561384+0000 mgr.y (mgr.14146) 87 : cephadm [INF] Detected new or changed devices on smithi047 2024-02-23T00:36:02.550 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:36:00.566993+0000 mgr.y (mgr.14146) 88 : cephadm [INF] Adjusting osd_memory_target on smithi047 to 16102M 2024-02-23T00:36:03.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.130+0000 7fe8716ea700 1 -- 172.21.15.47:0/4027652530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe86c0f8d20 msgr2=0x7fe86c0f9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:36:03.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.130+0000 7fe8716ea700 1 --2- 172.21.15.47:0/4027652530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe86c0f8d20 0x7fe86c0f9180 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fe85c0047b0 tx=0x7fe85c009cb0).stop 2024-02-23T00:36:03.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.130+0000 7fe8716ea700 1 -- 172.21.15.47:0/4027652530 shutdown_connections 2024-02-23T00:36:03.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.130+0000 7fe8716ea700 1 --2- 172.21.15.47:0/4027652530 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe86c0f9670 0x7fe86c0fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:36:03.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.130+0000 7fe8716ea700 1 --2- 172.21.15.47:0/4027652530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe86c0f7a80 0x7fe86c0f7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:36:03.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.130+0000 7fe8716ea700 1 --2- 172.21.15.47:0/4027652530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe86c0f8d20 0x7fe86c0f9180 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:36:03.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.130+0000 7fe8716ea700 1 -- 172.21.15.47:0/4027652530 >> 172.21.15.47:0/4027652530 conn(0x7fe86c0f32a0 msgr2=0x7fe86c0f56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:36:03.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.130+0000 7fe8716ea700 1 -- 172.21.15.47:0/4027652530 shutdown_connections 2024-02-23T00:36:03.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.130+0000 7fe8716ea700 1 -- 172.21.15.47:0/4027652530 wait complete. 2024-02-23T00:36:03.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.131+0000 7fe8716ea700 1 librados: starting msgr at 2024-02-23T00:36:03.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.131+0000 7fe8716ea700 1 librados: starting objecter 2024-02-23T00:36:03.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.131+0000 7fe8716ea700 1 Processor -- start 2024-02-23T00:36:03.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.131+0000 7fe8716ea700 1 -- start start 2024-02-23T00:36:03.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.131+0000 7fe8716ea700 1 librados: setting wanted keys 2024-02-23T00:36:03.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.131+0000 7fe8716ea700 1 librados: calling monclient init 2024-02-23T00:36:03.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.131+0000 7fe8716ea700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe86c0f7a80 0x7fe86c107eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:36:03.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.131+0000 7fe8716ea700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe86c0f8d20 0x7fe86c1083a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:36:03.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.131+0000 7fe8716ea700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe86c0f9670 0x7fe86c10f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:36:03.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.131+0000 7fe8716ea700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c10f970 con 0x7fe86c0f7a80 2024-02-23T00:36:03.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.131+0000 7fe8716ea700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c10fab0 con 0x7fe86c0f8d20 2024-02-23T00:36:03.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.131+0000 7fe8716ea700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c10fbf0 con 0x7fe86c0f9670 2024-02-23T00:36:03.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.131+0000 7fe86affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe86c0f7a80 0x7fe86c107eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:36:03.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.131+0000 7fe86b7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe86c0f9670 0x7fe86c10f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:36:03.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.131+0000 7fe86affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe86c0f7a80 0x7fe86c107eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40124/0 (socket says 172.21.15.47:40124) 2024-02-23T00:36:03.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.131+0000 7fe86affd700 1 -- 172.21.15.47:0/2906564925 learned_addr learned my addr 172.21.15.47:0/2906564925 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:36:03.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.131+0000 7fe86a7fc700 1 --2- 172.21.15.47:0/2906564925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe86c0f8d20 0x7fe86c1083a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:36:03.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.131+0000 7fe86affd700 1 -- 172.21.15.47:0/2906564925 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe86c0f9670 msgr2=0x7fe86c10f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:36:03.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.131+0000 7fe86affd700 1 --2- 172.21.15.47:0/2906564925 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe86c0f9670 0x7fe86c10f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:36:03.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.132+0000 7fe86affd700 1 -- 172.21.15.47:0/2906564925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe86c0f8d20 msgr2=0x7fe86c1083a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:36:03.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.132+0000 7fe86affd700 1 --2- 172.21.15.47:0/2906564925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe86c0f8d20 0x7fe86c1083a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:36:03.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.132+0000 7fe86affd700 1 -- 172.21.15.47:0/2906564925 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8640097c0 con 0x7fe86c0f7a80 2024-02-23T00:36:03.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.132+0000 7fe86a7fc700 1 --2- 172.21.15.47:0/2906564925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe86c0f8d20 0x7fe86c1083a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-23T00:36:03.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.132+0000 7fe86b7fe700 1 --2- 172.21.15.47:0/2906564925 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe86c0f9670 0x7fe86c10f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:36:03.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.132+0000 7fe86affd700 1 --2- 172.21.15.47:0/2906564925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe86c0f7a80 0x7fe86c107eb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fe8600027f0 tx=0x7fe860002b00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:36:03.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.132+0000 7fe853fff700 10 client.?.objecter ms_handle_connect 0x7fe86c0f7a80 2024-02-23T00:36:03.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.132+0000 7fe853fff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:36:03.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.132+0000 7fe853fff700 1 -- 172.21.15.47:0/2906564925 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe860007b40 con 0x7fe86c0f7a80 2024-02-23T00:36:03.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.132+0000 7fe853fff700 1 -- 172.21.15.47:0/2906564925 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fe86000f040 con 0x7fe86c0f7a80 2024-02-23T00:36:03.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.132+0000 7fe8716ea700 1 -- 172.21.15.47:0/2906564925 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe85c005040 con 0x7fe86c0f7a80 2024-02-23T00:36:03.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.132+0000 7fe853fff700 1 -- 172.21.15.47:0/2906564925 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe860014690 con 0x7fe86c0f7a80 2024-02-23T00:36:03.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.132+0000 7fe8716ea700 10 client.14238.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:36:03.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.132+0000 7fe8716ea700 1 -- 172.21.15.47:0/2906564925 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe86c110240 con 0x7fe86c0f7a80 2024-02-23T00:36:03.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.132+0000 7fe8716ea700 1 librados: init done 2024-02-23T00:36:03.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.133+0000 7fe8716ea700 1 -- 172.21.15.47:0/2906564925 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe86c05f360 con 0x7fe86c0f7a80 2024-02-23T00:36:03.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.133+0000 7fe853fff700 1 -- 172.21.15.47:0/2906564925 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 14) v1 ==== 77064+0+0 (secure 0 0 0) 0x7fe860009110 con 0x7fe86c0f7a80 2024-02-23T00:36:03.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.134+0000 7fe853fff700 1 --2- 172.21.15.47:0/2906564925 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fe854062bd0 0x7fe854065090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:36:03.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.134+0000 7fe86a7fc700 1 --2- 172.21.15.47:0/2906564925 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fe854062bd0 0x7fe854065090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:36:03.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.134+0000 7fe853fff700 1 -- 172.21.15.47:0/2906564925 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1710+0+0 (secure 0 0 0) 0x7fe86007b520 con 0x7fe86c0f7a80 2024-02-23T00:36:03.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.134+0000 7fe853fff700 10 client.14238.objecter ms_dispatch 0x7fe86c001550 osd_map(9..9 src has 1..9) v4 2024-02-23T00:36:03.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.134+0000 7fe853fff700 3 client.14238.objecter handle_osd_map got epochs [9,9] > 0 2024-02-23T00:36:03.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.134+0000 7fe853fff700 3 client.14238.objecter handle_osd_map decoding full epoch 9 2024-02-23T00:36:03.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.134+0000 7fe853fff700 20 client.14238.objecter dump_active .. 0 homeless 2024-02-23T00:36:03.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.134+0000 7fe853fff700 10 client.14238.objecter ms_handle_connect 0x7fe854062bd0 2024-02-23T00:36:03.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.134+0000 7fe86a7fc700 1 --2- 172.21.15.47:0/2906564925 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fe854062bd0 0x7fe854065090 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fe85c004e20 tx=0x7fe85c00b040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:36:03.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.138+0000 7fe853fff700 1 -- 172.21.15.47:0/2906564925 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe86004c910 con 0x7fe86c0f7a80 2024-02-23T00:36:03.145 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:02.085833+0000 mgr.y (mgr.14146) 89 : cluster [DBG] pgmap v64: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:03.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:03.321+0000 7fe8716ea700 1 -- 172.21.15.47:0/2906564925 --> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fe86c0ff5c0 con 0x7fe854062bd0 2024-02-23T00:36:03.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:02.085833+0000 mgr.y (mgr.14146) 89 : cluster [DBG] pgmap v64: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:03.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:02.085833+0000 mgr.y (mgr.14146) 89 : cluster [DBG] pgmap v64: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:04.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:03.323625+0000 mgr.y (mgr.14146) 90 : audit [DBG] from='client.14238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:36:04.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:03.325663+0000 mon.a (mon.0) 254 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-23T00:36:04.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:03.327614+0000 mon.a (mon.0) 255 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-23T00:36:04.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:03.328186+0000 mon.a (mon.0) 256 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:04.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:03.323625+0000 mgr.y (mgr.14146) 90 : audit [DBG] from='client.14238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:36:04.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:03.325663+0000 mon.a (mon.0) 254 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-23T00:36:04.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:03.327614+0000 mon.a (mon.0) 255 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-23T00:36:04.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:03.328186+0000 mon.a (mon.0) 256 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:04.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:03.323625+0000 mgr.y (mgr.14146) 90 : audit [DBG] from='client.14238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:36:04.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:03.325663+0000 mon.a (mon.0) 254 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-23T00:36:04.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:03.327614+0000 mon.a (mon.0) 255 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-23T00:36:04.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:03.328186+0000 mon.a (mon.0) 256 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:05.289 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:04.086341+0000 mgr.y (mgr.14146) 91 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:05.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:04.086341+0000 mgr.y (mgr.14146) 91 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:05.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:04.086341+0000 mgr.y (mgr.14146) 91 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:06.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:06.119904+0000 mon.c (mon.2) 4 : audit [INF] from='client.? 172.21.15.47:0/2705840074' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b06d0cce-55d2-4a6a-a796-4db0b9bb9d16"}]: dispatch 2024-02-23T00:36:06.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:06.120232+0000 mon.a (mon.0) 257 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b06d0cce-55d2-4a6a-a796-4db0b9bb9d16"}]: dispatch 2024-02-23T00:36:06.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:06.123567+0000 mon.a (mon.0) 258 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b06d0cce-55d2-4a6a-a796-4db0b9bb9d16"}]': finished 2024-02-23T00:36:06.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:06.123620+0000 mon.a (mon.0) 259 : cluster [DBG] osdmap e10: 2 total, 1 up, 2 in 2024-02-23T00:36:06.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:06.123743+0000 mon.a (mon.0) 260 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-23T00:36:06.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:06.119904+0000 mon.c (mon.2) 4 : audit [INF] from='client.? 172.21.15.47:0/2705840074' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b06d0cce-55d2-4a6a-a796-4db0b9bb9d16"}]: dispatch 2024-02-23T00:36:06.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:06.120232+0000 mon.a (mon.0) 257 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b06d0cce-55d2-4a6a-a796-4db0b9bb9d16"}]: dispatch 2024-02-23T00:36:06.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:06.123567+0000 mon.a (mon.0) 258 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b06d0cce-55d2-4a6a-a796-4db0b9bb9d16"}]': finished 2024-02-23T00:36:06.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:06.123620+0000 mon.a (mon.0) 259 : cluster [DBG] osdmap e10: 2 total, 1 up, 2 in 2024-02-23T00:36:06.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:06.123743+0000 mon.a (mon.0) 260 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-23T00:36:06.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:06.119904+0000 mon.c (mon.2) 4 : audit [INF] from='client.? 172.21.15.47:0/2705840074' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b06d0cce-55d2-4a6a-a796-4db0b9bb9d16"}]: dispatch 2024-02-23T00:36:06.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:06.120232+0000 mon.a (mon.0) 257 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b06d0cce-55d2-4a6a-a796-4db0b9bb9d16"}]: dispatch 2024-02-23T00:36:06.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:06.123567+0000 mon.a (mon.0) 258 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b06d0cce-55d2-4a6a-a796-4db0b9bb9d16"}]': finished 2024-02-23T00:36:06.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:06.123620+0000 mon.a (mon.0) 259 : cluster [DBG] osdmap e10: 2 total, 1 up, 2 in 2024-02-23T00:36:06.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:06.123743+0000 mon.a (mon.0) 260 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-23T00:36:07.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:06.086639+0000 mgr.y (mgr.14146) 92 : cluster [DBG] pgmap v66: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:07.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:06.770901+0000 mon.c (mon.2) 5 : audit [DBG] from='client.? 172.21.15.47:0/121861731' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-23T00:36:07.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:06.086639+0000 mgr.y (mgr.14146) 92 : cluster [DBG] pgmap v66: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:07.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:06.770901+0000 mon.c (mon.2) 5 : audit [DBG] from='client.? 172.21.15.47:0/121861731' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-23T00:36:07.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:06.086639+0000 mgr.y (mgr.14146) 92 : cluster [DBG] pgmap v66: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:07.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:06.770901+0000 mon.c (mon.2) 5 : audit [DBG] from='client.? 172.21.15.47:0/121861731' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-23T00:36:08.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:08.133+0000 7fe870ee9700 10 client.14238.objecter tick 2024-02-23T00:36:09.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:08.086934+0000 mgr.y (mgr.14146) 93 : cluster [DBG] pgmap v68: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:09.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:08.086934+0000 mgr.y (mgr.14146) 93 : cluster [DBG] pgmap v68: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:09.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:08.086934+0000 mgr.y (mgr.14146) 93 : cluster [DBG] pgmap v68: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:11.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:10.087210+0000 mgr.y (mgr.14146) 94 : cluster [DBG] pgmap v69: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:11.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:10.087210+0000 mgr.y (mgr.14146) 94 : cluster [DBG] pgmap v69: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:11.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:10.087210+0000 mgr.y (mgr.14146) 94 : cluster [DBG] pgmap v69: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:13.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:13.134+0000 7fe870ee9700 10 client.14238.objecter tick 2024-02-23T00:36:13.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:12.087500+0000 mgr.y (mgr.14146) 95 : cluster [DBG] pgmap v70: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:13.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:12.087500+0000 mgr.y (mgr.14146) 95 : cluster [DBG] pgmap v70: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:13.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:12.087500+0000 mgr.y (mgr.14146) 95 : cluster [DBG] pgmap v70: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:14.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:13.252466+0000 mon.a (mon.0) 261 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-23T00:36:14.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:13.253260+0000 mon.a (mon.0) 262 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:14.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:36:13.253923+0000 mgr.y (mgr.14146) 96 : cephadm [INF] Deploying daemon osd.1 on smithi047 2024-02-23T00:36:14.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:13.252466+0000 mon.a (mon.0) 261 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-23T00:36:14.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:13.253260+0000 mon.a (mon.0) 262 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:14.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:36:13.253923+0000 mgr.y (mgr.14146) 96 : cephadm [INF] Deploying daemon osd.1 on smithi047 2024-02-23T00:36:14.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:13.252466+0000 mon.a (mon.0) 261 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-23T00:36:14.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:13.253260+0000 mon.a (mon.0) 262 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:14.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:36:13.253923+0000 mgr.y (mgr.14146) 96 : cephadm [INF] Deploying daemon osd.1 on smithi047 2024-02-23T00:36:15.167 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:14.087755+0000 mgr.y (mgr.14146) 97 : cluster [DBG] pgmap v71: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:15.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:14.087755+0000 mgr.y (mgr.14146) 97 : cluster [DBG] pgmap v71: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:15.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:14.087755+0000 mgr.y (mgr.14146) 97 : cluster [DBG] pgmap v71: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:17.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:16.088140+0000 mgr.y (mgr.14146) 98 : cluster [DBG] pgmap v72: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:17.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:16.088140+0000 mgr.y (mgr.14146) 98 : cluster [DBG] pgmap v72: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:17.442 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:16.088140+0000 mgr.y (mgr.14146) 98 : cluster [DBG] pgmap v72: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:18.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:18.135+0000 7fe870ee9700 10 client.14238.objecter tick 2024-02-23T00:36:19.173 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:18.088540+0000 mgr.y (mgr.14146) 99 : cluster [DBG] pgmap v73: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:19.173 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:19.030322+0000 mon.a (mon.0) 263 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:19.173 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:19.034698+0000 mon.a (mon.0) 264 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:36:19.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:18.088540+0000 mgr.y (mgr.14146) 99 : cluster [DBG] pgmap v73: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:19.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:19.030322+0000 mon.a (mon.0) 263 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:19.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:19.034698+0000 mon.a (mon.0) 264 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:36:19.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:18.088540+0000 mgr.y (mgr.14146) 99 : cluster [DBG] pgmap v73: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:19.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:19.030322+0000 mon.a (mon.0) 263 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:19.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:19.034698+0000 mon.a (mon.0) 264 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:36:21.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:20.088830+0000 mgr.y (mgr.14146) 100 : cluster [DBG] pgmap v74: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:21.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:20.088830+0000 mgr.y (mgr.14146) 100 : cluster [DBG] pgmap v74: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:21.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:20.088830+0000 mgr.y (mgr.14146) 100 : cluster [DBG] pgmap v74: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:22.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:22.712+0000 7fe853fff700 1 -- 172.21.15.47:0/2906564925 <== mgr.14146 v2:172.21.15.47:6800/622994871 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe86c0ff5c0 con 0x7fe854062bd0 2024-02-23T00:36:22.715 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 1 on host 'smithi047' 2024-02-23T00:36:22.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:22.714+0000 7fe8716ea700 10 librados: watch_flush enter 2024-02-23T00:36:22.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:22.714+0000 7fe8716ea700 10 librados: watch_flush exit 2024-02-23T00:36:22.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:22.714+0000 7fe8716ea700 20 client.14238.objecter shutdown clearing up homeless session... 2024-02-23T00:36:22.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:22.714+0000 7fe8716ea700 10 client.14238.objecter successfully canceled tick 2024-02-23T00:36:22.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:22.714+0000 7fe8716ea700 1 -- 172.21.15.47:0/2906564925 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fe854062bd0 msgr2=0x7fe854065090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:36:22.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:22.714+0000 7fe8716ea700 1 --2- 172.21.15.47:0/2906564925 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fe854062bd0 0x7fe854065090 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fe85c004e20 tx=0x7fe85c00b040).stop 2024-02-23T00:36:22.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:22.714+0000 7fe8716ea700 1 -- 172.21.15.47:0/2906564925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe86c0f7a80 msgr2=0x7fe86c107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:36:22.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:22.714+0000 7fe8716ea700 1 --2- 172.21.15.47:0/2906564925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe86c0f7a80 0x7fe86c107eb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fe8600027f0 tx=0x7fe860002b00).stop 2024-02-23T00:36:22.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:22.714+0000 7fe8716ea700 1 -- 172.21.15.47:0/2906564925 shutdown_connections 2024-02-23T00:36:22.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:22.714+0000 7fe8716ea700 1 --2- 172.21.15.47:0/2906564925 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe86c0f9670 0x7fe86c10f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:36:22.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:22.714+0000 7fe8716ea700 1 --2- 172.21.15.47:0/2906564925 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fe854062bd0 0x7fe854065090 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:36:22.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:22.714+0000 7fe8716ea700 1 --2- 172.21.15.47:0/2906564925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe86c0f7a80 0x7fe86c107eb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:36:22.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:22.714+0000 7fe8716ea700 1 --2- 172.21.15.47:0/2906564925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe86c0f8d20 0x7fe86c1083a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:36:22.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:22.714+0000 7fe8716ea700 1 -- 172.21.15.47:0/2906564925 >> 172.21.15.47:0/2906564925 conn(0x7fe86c0f32a0 msgr2=0x7fe86c0fc770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:36:22.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:22.714+0000 7fe8716ea700 1 -- 172.21.15.47:0/2906564925 shutdown_connections 2024-02-23T00:36:22.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:22.714+0000 7fe8716ea700 1 -- 172.21.15.47:0/2906564925 wait complete. 2024-02-23T00:36:22.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:22.714+0000 7fe8716ea700 1 librados: shutdown 2024-02-23T00:36:23.056 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:21.897923+0000 mon.a (mon.0) 265 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:23.056 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:22.081549+0000 mon.c (mon.2) 6 : audit [INF] from='osd.1 [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-23T00:36:23.056 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:22.081965+0000 mon.a (mon.0) 266 : audit [INF] from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-23T00:36:23.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:22.089122+0000 mgr.y (mgr.14146) 101 : cluster [DBG] pgmap v75: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:23.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:22.202368+0000 mon.a (mon.0) 267 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:23.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:22.203102+0000 mon.a (mon.0) 268 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:36:23.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:22.207104+0000 mon.a (mon.0) 269 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:23.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:22.712362+0000 mon.a (mon.0) 270 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:23.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:22.723546+0000 mon.a (mon.0) 271 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:36:23.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:21.897923+0000 mon.a (mon.0) 265 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:23.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:22.081549+0000 mon.c (mon.2) 6 : audit [INF] from='osd.1 [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-23T00:36:23.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:22.081965+0000 mon.a (mon.0) 266 : audit [INF] from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-23T00:36:23.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:22.089122+0000 mgr.y (mgr.14146) 101 : cluster [DBG] pgmap v75: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:23.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:22.202368+0000 mon.a (mon.0) 267 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:23.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:22.203102+0000 mon.a (mon.0) 268 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:36:23.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:22.207104+0000 mon.a (mon.0) 269 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:23.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:22.712362+0000 mon.a (mon.0) 270 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:23.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:22.723546+0000 mon.a (mon.0) 271 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:36:23.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:21.897923+0000 mon.a (mon.0) 265 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:23.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:22.081549+0000 mon.c (mon.2) 6 : audit [INF] from='osd.1 [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-23T00:36:23.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:22.081965+0000 mon.a (mon.0) 266 : audit [INF] from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-23T00:36:23.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:22.089122+0000 mgr.y (mgr.14146) 101 : cluster [DBG] pgmap v75: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:23.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:22.202368+0000 mon.a (mon.0) 267 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:23.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:22.203102+0000 mon.a (mon.0) 268 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:36:23.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:22.207104+0000 mon.a (mon.0) 269 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:23.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:22.712362+0000 mon.a (mon.0) 270 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:23.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:22.723546+0000 mon.a (mon.0) 271 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:36:23.468 DEBUG:teuthology.orchestra.run.smithi047:osd.1> sudo journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@osd.1.service 2024-02-23T00:36:23.470 INFO:tasks.cephadm:Deploying osd.2 on smithi047 with /dev/vg_nvme/lv_2... 2024-02-23T00:36:23.470 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-23T00:36:23.502 INFO:journalctl@ceph.osd.1.smithi047.stdout:-- Logs begin at Fri 2024-02-23 00:13:17 UTC. -- 2024-02-23T00:36:24.019 INFO:journalctl@ceph.osd.1.smithi047.stdout:Feb 23 00:36:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-1[52714]: debug 2024-02-23T00:36:23.906+0000 7f7fd5d64700 -1 osd.1 0 waiting for initial osdmap 2024-02-23T00:36:24.019 INFO:journalctl@ceph.osd.1.smithi047.stdout:Feb 23 00:36:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-1[52714]: debug 2024-02-23T00:36:23.910+0000 7f7fd04dc700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-23T00:36:24.019 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:22.904588+0000 mon.a (mon.0) 272 : audit [INF] from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-23T00:36:24.019 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:22.904673+0000 mon.a (mon.0) 273 : cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2024-02-23T00:36:24.019 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:22.904785+0000 mon.a (mon.0) 274 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-23T00:36:24.019 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:22.905220+0000 mon.c (mon.2) 7 : audit [INF] from='osd.1 [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-02-23T00:36:24.019 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:22.905524+0000 mon.a (mon.0) 275 : audit [INF] from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-02-23T00:36:24.020 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:22.904588+0000 mon.a (mon.0) 272 : audit [INF] from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-23T00:36:24.020 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:22.904673+0000 mon.a (mon.0) 273 : cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2024-02-23T00:36:24.020 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:22.904785+0000 mon.a (mon.0) 274 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-23T00:36:24.020 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:22.905220+0000 mon.c (mon.2) 7 : audit [INF] from='osd.1 [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-02-23T00:36:24.020 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:22.905524+0000 mon.a (mon.0) 275 : audit [INF] from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-02-23T00:36:24.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:22.904588+0000 mon.a (mon.0) 272 : audit [INF] from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-23T00:36:24.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:22.904673+0000 mon.a (mon.0) 273 : cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2024-02-23T00:36:24.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:22.904785+0000 mon.a (mon.0) 274 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-23T00:36:24.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:22.905220+0000 mon.c (mon.2) 7 : audit [INF] from='osd.1 [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-02-23T00:36:24.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:22.905524+0000 mon.a (mon.0) 275 : audit [INF] from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-02-23T00:36:25.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:23.904122+0000 mon.a (mon.0) 276 : audit [INF] from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-02-23T00:36:25.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:23.904185+0000 mon.a (mon.0) 277 : cluster [DBG] osdmap e12: 2 total, 1 up, 2 in 2024-02-23T00:36:25.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:23.904482+0000 mon.a (mon.0) 278 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-23T00:36:25.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:23.908143+0000 mon.a (mon.0) 279 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-23T00:36:25.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:24.089417+0000 mgr.y (mgr.14146) 102 : cluster [DBG] pgmap v78: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:25.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:23.904122+0000 mon.a (mon.0) 276 : audit [INF] from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-02-23T00:36:25.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:23.904185+0000 mon.a (mon.0) 277 : cluster [DBG] osdmap e12: 2 total, 1 up, 2 in 2024-02-23T00:36:25.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:23.904482+0000 mon.a (mon.0) 278 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-23T00:36:25.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:23.908143+0000 mon.a (mon.0) 279 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-23T00:36:25.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:24.089417+0000 mgr.y (mgr.14146) 102 : cluster [DBG] pgmap v78: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:25.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:23.904122+0000 mon.a (mon.0) 276 : audit [INF] from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-02-23T00:36:25.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:23.904185+0000 mon.a (mon.0) 277 : cluster [DBG] osdmap e12: 2 total, 1 up, 2 in 2024-02-23T00:36:25.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:23.904482+0000 mon.a (mon.0) 278 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-23T00:36:25.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:23.908143+0000 mon.a (mon.0) 279 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-23T00:36:25.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:24.089417+0000 mgr.y (mgr.14146) 102 : cluster [DBG] pgmap v78: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-23T00:36:26.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:23.125358+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:36:26.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:23.125405+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:36:26.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:24.909695+0000 mon.a (mon.0) 280 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-23T00:36:26.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:24.914722+0000 mon.a (mon.0) 281 : cluster [INF] osd.1 [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] boot 2024-02-23T00:36:26.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:24.914751+0000 mon.a (mon.0) 282 : cluster [DBG] osdmap e13: 2 total, 2 up, 2 in 2024-02-23T00:36:26.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:24.915025+0000 mon.a (mon.0) 283 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-23T00:36:26.234 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:23.125358+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:36:26.235 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:23.125405+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:36:26.235 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:24.909695+0000 mon.a (mon.0) 280 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-23T00:36:26.235 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:24.914722+0000 mon.a (mon.0) 281 : cluster [INF] osd.1 [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] boot 2024-02-23T00:36:26.235 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:24.914751+0000 mon.a (mon.0) 282 : cluster [DBG] osdmap e13: 2 total, 2 up, 2 in 2024-02-23T00:36:26.235 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:24.915025+0000 mon.a (mon.0) 283 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-23T00:36:26.235 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:23.125358+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:36:26.235 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:23.125405+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:36:26.235 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:24.909695+0000 mon.a (mon.0) 280 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-23T00:36:26.235 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:24.914722+0000 mon.a (mon.0) 281 : cluster [INF] osd.1 [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] boot 2024-02-23T00:36:26.236 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:24.914751+0000 mon.a (mon.0) 282 : cluster [DBG] osdmap e13: 2 total, 2 up, 2 in 2024-02-23T00:36:26.236 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:24.915025+0000 mon.a (mon.0) 283 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-23T00:36:27.044 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-23T00:36:27.044 INFO:teuthology.orchestra.run.smithi047.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-23T00:36:27.044 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10+0 records in 2024-02-23T00:36:27.044 INFO:teuthology.orchestra.run.smithi047.stderr:10+0 records out 2024-02-23T00:36:27.044 INFO:teuthology.orchestra.run.smithi047.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00921572 s, 1.1 GB/s 2024-02-23T00:36:27.045 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping successful for: 2024-02-23T00:36:27.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:25.919234+0000 mon.a (mon.0) 284 : cluster [DBG] osdmap e14: 2 total, 2 up, 2 in 2024-02-23T00:36:27.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:26.089797+0000 mgr.y (mgr.14146) 103 : cluster [DBG] pgmap v81: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:27.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:25.919234+0000 mon.a (mon.0) 284 : cluster [DBG] osdmap e14: 2 total, 2 up, 2 in 2024-02-23T00:36:27.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:26.089797+0000 mgr.y (mgr.14146) 103 : cluster [DBG] pgmap v81: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:27.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:25.919234+0000 mon.a (mon.0) 284 : cluster [DBG] osdmap e14: 2 total, 2 up, 2 in 2024-02-23T00:36:27.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:26.089797+0000 mgr.y (mgr.14146) 103 : cluster [DBG] pgmap v81: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:28.150 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph orch daemon add osd smithi047:vg_nvme/lv_2 2024-02-23T00:36:29.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:28.090129+0000 mgr.y (mgr.14146) 104 : cluster [DBG] pgmap v82: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:29.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:28.899177+0000 mon.a (mon.0) 285 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:29.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:28.899956+0000 mon.a (mon.0) 286 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:36:29.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:28.900541+0000 mon.a (mon.0) 287 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:36:29.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:28.904041+0000 mon.a (mon.0) 288 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:29.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:28.904962+0000 mon.a (mon.0) 289 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:29.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:28.905532+0000 mon.a (mon.0) 290 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:36:29.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:28.909902+0000 mon.a (mon.0) 291 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:29.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:28.090129+0000 mgr.y (mgr.14146) 104 : cluster [DBG] pgmap v82: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:29.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:28.899177+0000 mon.a (mon.0) 285 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:29.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:28.899956+0000 mon.a (mon.0) 286 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:36:29.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:28.900541+0000 mon.a (mon.0) 287 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:36:29.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:28.904041+0000 mon.a (mon.0) 288 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:29.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:28.904962+0000 mon.a (mon.0) 289 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:29.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:28.905532+0000 mon.a (mon.0) 290 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:36:29.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:28.909902+0000 mon.a (mon.0) 291 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:29.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:28.090129+0000 mgr.y (mgr.14146) 104 : cluster [DBG] pgmap v82: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:29.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:28.899177+0000 mon.a (mon.0) 285 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:29.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:28.899956+0000 mon.a (mon.0) 286 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:36:29.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:28.900541+0000 mon.a (mon.0) 287 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:36:29.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:28.904041+0000 mon.a (mon.0) 288 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:29.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:28.904962+0000 mon.a (mon.0) 289 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:29.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:28.905532+0000 mon.a (mon.0) 290 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:36:29.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:28.909902+0000 mon.a (mon.0) 291 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:30.388 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:36:28.894901+0000 mgr.y (mgr.14146) 105 : cephadm [INF] Detected new or changed devices on smithi047 2024-02-23T00:36:30.389 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:36:28.900941+0000 mgr.y (mgr.14146) 106 : cephadm [INF] Adjusting osd_memory_target on smithi047 to 8051M 2024-02-23T00:36:30.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:36:28.894901+0000 mgr.y (mgr.14146) 105 : cephadm [INF] Detected new or changed devices on smithi047 2024-02-23T00:36:30.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:36:28.900941+0000 mgr.y (mgr.14146) 106 : cephadm [INF] Adjusting osd_memory_target on smithi047 to 8051M 2024-02-23T00:36:30.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:36:28.894901+0000 mgr.y (mgr.14146) 105 : cephadm [INF] Detected new or changed devices on smithi047 2024-02-23T00:36:30.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:36:28.900941+0000 mgr.y (mgr.14146) 106 : cephadm [INF] Adjusting osd_memory_target on smithi047 to 8051M 2024-02-23T00:36:30.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.662+0000 7ff1e7a7a700 1 -- 172.21.15.47:0/2240812642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1e00f7a80 msgr2=0x7ff1e00f7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:36:30.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.662+0000 7ff1e7a7a700 1 --2- 172.21.15.47:0/2240812642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1e00f7a80 0x7ff1e00f7ea0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7ff1d0004870 tx=0x7ff1d0009cb0).stop 2024-02-23T00:36:30.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.662+0000 7ff1e7a7a700 1 -- 172.21.15.47:0/2240812642 shutdown_connections 2024-02-23T00:36:30.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.662+0000 7ff1e7a7a700 1 --2- 172.21.15.47:0/2240812642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1e00f9670 0x7ff1e00fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:36:30.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.662+0000 7ff1e7a7a700 1 --2- 172.21.15.47:0/2240812642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1e00f7a80 0x7ff1e00f7ea0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:36:30.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.662+0000 7ff1e7a7a700 1 --2- 172.21.15.47:0/2240812642 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1e00f8d20 0x7ff1e00f9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:36:30.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.662+0000 7ff1e7a7a700 1 -- 172.21.15.47:0/2240812642 >> 172.21.15.47:0/2240812642 conn(0x7ff1e00f32a0 msgr2=0x7ff1e00f56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:36:30.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.662+0000 7ff1e7a7a700 1 -- 172.21.15.47:0/2240812642 shutdown_connections 2024-02-23T00:36:30.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.662+0000 7ff1e7a7a700 1 -- 172.21.15.47:0/2240812642 wait complete. 2024-02-23T00:36:30.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.663+0000 7ff1e7a7a700 1 librados: starting msgr at 2024-02-23T00:36:30.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.663+0000 7ff1e7a7a700 1 librados: starting objecter 2024-02-23T00:36:30.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.663+0000 7ff1e7a7a700 1 Processor -- start 2024-02-23T00:36:30.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.663+0000 7ff1e7a7a700 1 -- start start 2024-02-23T00:36:30.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.663+0000 7ff1e7a7a700 1 librados: setting wanted keys 2024-02-23T00:36:30.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.663+0000 7ff1e7a7a700 1 librados: calling monclient init 2024-02-23T00:36:30.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.663+0000 7ff1e7a7a700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1e00f7a80 0x7ff1e0107f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:36:30.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.663+0000 7ff1e7a7a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1e00f8d20 0x7ff1e0108470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:36:30.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.663+0000 7ff1e7a7a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1e00f9670 0x7ff1e010f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:36:30.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.663+0000 7ff1e7a7a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e010fa40 con 0x7ff1e00f8d20 2024-02-23T00:36:30.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.663+0000 7ff1e7a7a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e010fb80 con 0x7ff1e00f9670 2024-02-23T00:36:30.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.663+0000 7ff1e7a7a700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e010fcc0 con 0x7ff1e00f7a80 2024-02-23T00:36:30.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.664+0000 7ff1e5015700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1e00f8d20 0x7ff1e0108470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:36:30.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.664+0000 7ff1e5816700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1e00f7a80 0x7ff1e0107f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:36:30.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.664+0000 7ff1e5816700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1e00f7a80 0x7ff1e0107f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:54516/0 (socket says 172.21.15.47:54516) 2024-02-23T00:36:30.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.664+0000 7ff1e5015700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1e00f8d20 0x7ff1e0108470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38428/0 (socket says 172.21.15.47:38428) 2024-02-23T00:36:30.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.664+0000 7ff1e5816700 1 -- 172.21.15.47:0/1814828977 learned_addr learned my addr 172.21.15.47:0/1814828977 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:36:30.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.664+0000 7ff1e6017700 1 --2- 172.21.15.47:0/1814828977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1e00f9670 0x7ff1e010f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:36:30.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.664+0000 7ff1e5816700 1 -- 172.21.15.47:0/1814828977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1e00f9670 msgr2=0x7ff1e010f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:36:30.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.664+0000 7ff1e5816700 1 --2- 172.21.15.47:0/1814828977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1e00f9670 0x7ff1e010f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:36:30.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.664+0000 7ff1e5816700 1 -- 172.21.15.47:0/1814828977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1e00f8d20 msgr2=0x7ff1e0108470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:36:30.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.664+0000 7ff1e5816700 1 --2- 172.21.15.47:0/1814828977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1e00f8d20 0x7ff1e0108470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:36:30.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.664+0000 7ff1e5816700 1 -- 172.21.15.47:0/1814828977 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1dc005040 con 0x7ff1e00f7a80 2024-02-23T00:36:30.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.664+0000 7ff1e5015700 1 --2- 172.21.15.47:0/1814828977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1e00f8d20 0x7ff1e0108470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:36:30.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.664+0000 7ff1e5816700 1 --2- 172.21.15.47:0/1814828977 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1e00f7a80 0x7ff1e0107f80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7ff1d000b120 tx=0x7ff1d0009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:36:30.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.664+0000 7ff1d6f78700 10 client.?.objecter ms_handle_connect 0x7ff1e00f7a80 2024-02-23T00:36:30.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.664+0000 7ff1d6f78700 10 client.?.objecter resend_mon_ops 2024-02-23T00:36:30.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.664+0000 7ff1d6f78700 1 -- 172.21.15.47:0/1814828977 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d000e030 con 0x7ff1e00f7a80 2024-02-23T00:36:30.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.664+0000 7ff1e7a7a700 1 -- 172.21.15.47:0/1814828977 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d0005040 con 0x7ff1e00f7a80 2024-02-23T00:36:30.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.664+0000 7ff1d6f78700 1 -- 172.21.15.47:0/1814828977 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7ff1d00096d0 con 0x7ff1e00f7a80 2024-02-23T00:36:30.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.664+0000 7ff1d6f78700 1 -- 172.21.15.47:0/1814828977 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0018070 con 0x7ff1e00f7a80 2024-02-23T00:36:30.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.664+0000 7ff1e7a7a700 10 client.24176.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:36:30.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.664+0000 7ff1e7a7a700 1 -- 172.21.15.47:0/1814828977 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e006b1b0 con 0x7ff1e00f7a80 2024-02-23T00:36:30.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.664+0000 7ff1e7a7a700 1 librados: init done 2024-02-23T00:36:30.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.665+0000 7ff1e7a7a700 1 -- 172.21.15.47:0/1814828977 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1e005f360 con 0x7ff1e00f7a80 2024-02-23T00:36:30.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.666+0000 7ff1d6f78700 1 -- 172.21.15.47:0/1814828977 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 14) v1 ==== 77064+0+0 (secure 0 0 0) 0x7ff1d0009240 con 0x7ff1e00f7a80 2024-02-23T00:36:30.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.666+0000 7ff1d6f78700 1 --2- 172.21.15.47:0/1814828977 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7ff1cc062bd0 0x7ff1cc065090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:36:30.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.666+0000 7ff1d6f78700 1 -- 172.21.15.47:0/1814828977 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2142+0+0 (secure 0 0 0) 0x7ff1d007cbb0 con 0x7ff1e00f7a80 2024-02-23T00:36:30.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.666+0000 7ff1d6f78700 10 client.24176.objecter ms_dispatch 0x7ff1e0001550 osd_map(14..14 src has 1..14) v4 2024-02-23T00:36:30.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.666+0000 7ff1d6f78700 3 client.24176.objecter handle_osd_map got epochs [14,14] > 0 2024-02-23T00:36:30.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.666+0000 7ff1d6f78700 3 client.24176.objecter handle_osd_map decoding full epoch 14 2024-02-23T00:36:30.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.666+0000 7ff1d6f78700 20 client.24176.objecter dump_active .. 0 homeless 2024-02-23T00:36:30.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.666+0000 7ff1e5015700 1 --2- 172.21.15.47:0/1814828977 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7ff1cc062bd0 0x7ff1cc065090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:36:30.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.667+0000 7ff1d6f78700 10 client.24176.objecter ms_handle_connect 0x7ff1cc062bd0 2024-02-23T00:36:30.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.667+0000 7ff1e5015700 1 --2- 172.21.15.47:0/1814828977 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7ff1cc062bd0 0x7ff1cc065090 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7ff1dc009760 tx=0x7ff1dc00b040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:36:30.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.671+0000 7ff1d6f78700 1 -- 172.21.15.47:0/1814828977 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ff1d004ddb0 con 0x7ff1e00f7a80 2024-02-23T00:36:30.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:30.850+0000 7ff1e7a7a700 1 -- 172.21.15.47:0/1814828977 --> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1e00ff510 con 0x7ff1cc062bd0 2024-02-23T00:36:31.414 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:30.090417+0000 mgr.y (mgr.14146) 107 : cluster [DBG] pgmap v83: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:31.414 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:30.854027+0000 mon.a (mon.0) 292 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-23T00:36:31.414 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:30.855846+0000 mon.a (mon.0) 293 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-23T00:36:31.414 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:30.856396+0000 mon.a (mon.0) 294 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:31.415 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:30.090417+0000 mgr.y (mgr.14146) 107 : cluster [DBG] pgmap v83: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:31.415 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:30.854027+0000 mon.a (mon.0) 292 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-23T00:36:31.415 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:30.855846+0000 mon.a (mon.0) 293 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-23T00:36:31.415 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:30.856396+0000 mon.a (mon.0) 294 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:31.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:30.090417+0000 mgr.y (mgr.14146) 107 : cluster [DBG] pgmap v83: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:31.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:30.854027+0000 mon.a (mon.0) 292 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-23T00:36:31.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:30.855846+0000 mon.a (mon.0) 293 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-23T00:36:31.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:30.856396+0000 mon.a (mon.0) 294 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:32.339 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:30.852060+0000 mgr.y (mgr.14146) 108 : audit [DBG] from='client.24176 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:36:32.340 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:30.852060+0000 mgr.y (mgr.14146) 108 : audit [DBG] from='client.24176 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:36:32.442 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:30.852060+0000 mgr.y (mgr.14146) 108 : audit [DBG] from='client.24176 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:36:33.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:32.090702+0000 mgr.y (mgr.14146) 109 : cluster [DBG] pgmap v84: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:33.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:32.090702+0000 mgr.y (mgr.14146) 109 : cluster [DBG] pgmap v84: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:33.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:32.090702+0000 mgr.y (mgr.14146) 109 : cluster [DBG] pgmap v84: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:34.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:33.479074+0000 mon.a (mon.0) 295 : audit [INF] from='client.? 172.21.15.47:0/968346980' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e44d7ca-67a1-4868-bb64-891cf477ebd5"}]: dispatch 2024-02-23T00:36:34.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:33.483405+0000 mon.a (mon.0) 296 : audit [INF] from='client.? 172.21.15.47:0/968346980' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e44d7ca-67a1-4868-bb64-891cf477ebd5"}]': finished 2024-02-23T00:36:34.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:33.483461+0000 mon.a (mon.0) 297 : cluster [DBG] osdmap e15: 3 total, 2 up, 3 in 2024-02-23T00:36:34.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:33.483579+0000 mon.a (mon.0) 298 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-23T00:36:34.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:34.132269+0000 mon.a (mon.0) 299 : audit [DBG] from='client.? 172.21.15.47:0/1625767133' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-23T00:36:34.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:33.479074+0000 mon.a (mon.0) 295 : audit [INF] from='client.? 172.21.15.47:0/968346980' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e44d7ca-67a1-4868-bb64-891cf477ebd5"}]: dispatch 2024-02-23T00:36:34.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:33.483405+0000 mon.a (mon.0) 296 : audit [INF] from='client.? 172.21.15.47:0/968346980' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e44d7ca-67a1-4868-bb64-891cf477ebd5"}]': finished 2024-02-23T00:36:34.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:33.483461+0000 mon.a (mon.0) 297 : cluster [DBG] osdmap e15: 3 total, 2 up, 3 in 2024-02-23T00:36:34.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:33.483579+0000 mon.a (mon.0) 298 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-23T00:36:34.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:34.132269+0000 mon.a (mon.0) 299 : audit [DBG] from='client.? 172.21.15.47:0/1625767133' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-23T00:36:34.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:33.479074+0000 mon.a (mon.0) 295 : audit [INF] from='client.? 172.21.15.47:0/968346980' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e44d7ca-67a1-4868-bb64-891cf477ebd5"}]: dispatch 2024-02-23T00:36:34.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:33.483405+0000 mon.a (mon.0) 296 : audit [INF] from='client.? 172.21.15.47:0/968346980' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e44d7ca-67a1-4868-bb64-891cf477ebd5"}]': finished 2024-02-23T00:36:34.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:33.483461+0000 mon.a (mon.0) 297 : cluster [DBG] osdmap e15: 3 total, 2 up, 3 in 2024-02-23T00:36:34.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:33.483579+0000 mon.a (mon.0) 298 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-23T00:36:34.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:34.132269+0000 mon.a (mon.0) 299 : audit [DBG] from='client.? 172.21.15.47:0/1625767133' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-23T00:36:35.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:34.090980+0000 mgr.y (mgr.14146) 110 : cluster [DBG] pgmap v86: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:35.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:34.090980+0000 mgr.y (mgr.14146) 110 : cluster [DBG] pgmap v86: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:35.597 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:34.090980+0000 mgr.y (mgr.14146) 110 : cluster [DBG] pgmap v86: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:35.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:35.666+0000 7ff1e7279700 10 client.24176.objecter tick 2024-02-23T00:36:37.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:36.091438+0000 mgr.y (mgr.14146) 111 : cluster [DBG] pgmap v87: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:37.306 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:36.091438+0000 mgr.y (mgr.14146) 111 : cluster [DBG] pgmap v87: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:37.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:36.091438+0000 mgr.y (mgr.14146) 111 : cluster [DBG] pgmap v87: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:39.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:38.091817+0000 mgr.y (mgr.14146) 112 : cluster [DBG] pgmap v88: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:39.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:38.091817+0000 mgr.y (mgr.14146) 112 : cluster [DBG] pgmap v88: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:39.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:38.091817+0000 mgr.y (mgr.14146) 112 : cluster [DBG] pgmap v88: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:40.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:40.667+0000 7ff1e7279700 10 client.24176.objecter tick 2024-02-23T00:36:41.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:40.092199+0000 mgr.y (mgr.14146) 113 : cluster [DBG] pgmap v89: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:41.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:40.209122+0000 mon.a (mon.0) 300 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-23T00:36:41.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:40.209903+0000 mon.a (mon.0) 301 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:41.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:36:40.210575+0000 mgr.y (mgr.14146) 114 : cephadm [INF] Deploying daemon osd.2 on smithi047 2024-02-23T00:36:41.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:40.092199+0000 mgr.y (mgr.14146) 113 : cluster [DBG] pgmap v89: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:41.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:40.209122+0000 mon.a (mon.0) 300 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-23T00:36:41.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:40.209903+0000 mon.a (mon.0) 301 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:41.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:36:40.210575+0000 mgr.y (mgr.14146) 114 : cephadm [INF] Deploying daemon osd.2 on smithi047 2024-02-23T00:36:41.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:40.092199+0000 mgr.y (mgr.14146) 113 : cluster [DBG] pgmap v89: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:41.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:40.209122+0000 mon.a (mon.0) 300 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-23T00:36:41.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:40.209903+0000 mon.a (mon.0) 301 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:41.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:36:40.210575+0000 mgr.y (mgr.14146) 114 : cephadm [INF] Deploying daemon osd.2 on smithi047 2024-02-23T00:36:43.416 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:42.092551+0000 mgr.y (mgr.14146) 115 : cluster [DBG] pgmap v90: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:43.417 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:42.092551+0000 mgr.y (mgr.14146) 115 : cluster [DBG] pgmap v90: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:43.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:42.092551+0000 mgr.y (mgr.14146) 115 : cluster [DBG] pgmap v90: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:45.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:44.092989+0000 mgr.y (mgr.14146) 116 : cluster [DBG] pgmap v91: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:45.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:44.092989+0000 mgr.y (mgr.14146) 116 : cluster [DBG] pgmap v91: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:45.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:44.092989+0000 mgr.y (mgr.14146) 116 : cluster [DBG] pgmap v91: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:45.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:45.667+0000 7ff1e7279700 10 client.24176.objecter tick 2024-02-23T00:36:47.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:46.093341+0000 mgr.y (mgr.14146) 117 : cluster [DBG] pgmap v92: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:47.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:46.160405+0000 mon.a (mon.0) 302 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:47.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:46.161560+0000 mon.a (mon.0) 303 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:36:47.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:46.093341+0000 mgr.y (mgr.14146) 117 : cluster [DBG] pgmap v92: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:47.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:46.160405+0000 mon.a (mon.0) 302 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:47.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:46.161560+0000 mon.a (mon.0) 303 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:36:47.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:46.093341+0000 mgr.y (mgr.14146) 117 : cluster [DBG] pgmap v92: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:47.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:46.160405+0000 mon.a (mon.0) 302 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:47.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:46.161560+0000 mon.a (mon.0) 303 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:36:49.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:48.093718+0000 mgr.y (mgr.14146) 118 : cluster [DBG] pgmap v93: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:49.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:49.070587+0000 mon.a (mon.0) 304 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:49.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:49.071372+0000 mon.a (mon.0) 305 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:49.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:49.072082+0000 mon.a (mon.0) 306 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:36:49.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:49.076125+0000 mon.a (mon.0) 307 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:49.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:48.093718+0000 mgr.y (mgr.14146) 118 : cluster [DBG] pgmap v93: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:49.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:49.070587+0000 mon.a (mon.0) 304 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:49.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:49.071372+0000 mon.a (mon.0) 305 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:49.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:49.072082+0000 mon.a (mon.0) 306 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:36:49.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:49.076125+0000 mon.a (mon.0) 307 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:49.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:48.093718+0000 mgr.y (mgr.14146) 118 : cluster [DBG] pgmap v93: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:49.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:49.070587+0000 mon.a (mon.0) 304 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:49.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:49.071372+0000 mon.a (mon.0) 305 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:49.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:49.072082+0000 mon.a (mon.0) 306 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:36:49.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:49.076125+0000 mon.a (mon.0) 307 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:49.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:49.935+0000 7ff1d6f78700 1 -- 172.21.15.47:0/1814828977 <== mgr.14146 v2:172.21.15.47:6800/622994871 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff1e00ff510 con 0x7ff1cc062bd0 2024-02-23T00:36:49.938 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 2 on host 'smithi047' 2024-02-23T00:36:49.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:49.937+0000 7ff1e7a7a700 10 librados: watch_flush enter 2024-02-23T00:36:49.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:49.937+0000 7ff1e7a7a700 10 librados: watch_flush exit 2024-02-23T00:36:49.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:49.937+0000 7ff1e7a7a700 20 client.24176.objecter shutdown clearing up homeless session... 2024-02-23T00:36:49.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:49.937+0000 7ff1e7a7a700 10 client.24176.objecter successfully canceled tick 2024-02-23T00:36:49.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:49.938+0000 7ff1e7a7a700 1 -- 172.21.15.47:0/1814828977 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7ff1cc062bd0 msgr2=0x7ff1cc065090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:36:49.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:49.938+0000 7ff1e7a7a700 1 --2- 172.21.15.47:0/1814828977 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7ff1cc062bd0 0x7ff1cc065090 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7ff1dc009760 tx=0x7ff1dc00b040).stop 2024-02-23T00:36:49.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:49.938+0000 7ff1e7a7a700 1 -- 172.21.15.47:0/1814828977 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1e00f7a80 msgr2=0x7ff1e0107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:36:49.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:49.938+0000 7ff1e7a7a700 1 --2- 172.21.15.47:0/1814828977 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1e00f7a80 0x7ff1e0107f80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7ff1d000b120 tx=0x7ff1d0009cb0).stop 2024-02-23T00:36:49.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:49.938+0000 7ff1e7a7a700 1 -- 172.21.15.47:0/1814828977 shutdown_connections 2024-02-23T00:36:49.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:49.938+0000 7ff1e7a7a700 1 --2- 172.21.15.47:0/1814828977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1e00f9670 0x7ff1e010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:36:49.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:49.938+0000 7ff1e7a7a700 1 --2- 172.21.15.47:0/1814828977 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1e00f7a80 0x7ff1e0107f80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:36:49.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:49.938+0000 7ff1e7a7a700 1 --2- 172.21.15.47:0/1814828977 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7ff1cc062bd0 0x7ff1cc065090 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:36:49.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:49.938+0000 7ff1e7a7a700 1 --2- 172.21.15.47:0/1814828977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1e00f8d20 0x7ff1e0108470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:36:49.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:49.938+0000 7ff1e7a7a700 1 -- 172.21.15.47:0/1814828977 >> 172.21.15.47:0/1814828977 conn(0x7ff1e00f32a0 msgr2=0x7ff1e00fc770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:36:49.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:49.938+0000 7ff1e7a7a700 1 -- 172.21.15.47:0/1814828977 shutdown_connections 2024-02-23T00:36:49.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:49.938+0000 7ff1e7a7a700 1 -- 172.21.15.47:0/1814828977 wait complete. 2024-02-23T00:36:49.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:49.938+0000 7ff1e7a7a700 1 librados: shutdown 2024-02-23T00:36:50.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:49.255257+0000 mon.c (mon.2) 8 : audit [INF] from='osd.2 [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-23T00:36:50.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:49.255615+0000 mon.a (mon.0) 308 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-23T00:36:50.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:49.935881+0000 mon.a (mon.0) 309 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:50.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:49.943609+0000 mon.a (mon.0) 310 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:36:50.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:49.255257+0000 mon.c (mon.2) 8 : audit [INF] from='osd.2 [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-23T00:36:50.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:49.255615+0000 mon.a (mon.0) 308 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-23T00:36:50.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:49.935881+0000 mon.a (mon.0) 309 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:50.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:49.943609+0000 mon.a (mon.0) 310 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:36:50.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:49.255257+0000 mon.c (mon.2) 8 : audit [INF] from='osd.2 [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-23T00:36:50.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:49.255615+0000 mon.a (mon.0) 308 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-23T00:36:50.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:49.935881+0000 mon.a (mon.0) 309 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:50.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:49.943609+0000 mon.a (mon.0) 310 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:36:50.736 DEBUG:teuthology.orchestra.run.smithi047:osd.2> sudo journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@osd.2.service 2024-02-23T00:36:50.739 INFO:tasks.cephadm:Deploying osd.3 on smithi047 with /dev/vg_nvme/lv_1... 2024-02-23T00:36:50.739 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-23T00:36:50.775 INFO:journalctl@ceph.osd.2.smithi047.stdout:-- Logs begin at Fri 2024-02-23 00:13:17 UTC. -- 2024-02-23T00:36:51.336 INFO:journalctl@ceph.osd.2.smithi047.stdout:Feb 23 00:36:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-2[57453]: debug 2024-02-23T00:36:51.173+0000 7f8f32083700 -1 osd.2 0 waiting for initial osdmap 2024-02-23T00:36:51.336 INFO:journalctl@ceph.osd.2.smithi047.stdout:Feb 23 00:36:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-2[57453]: debug 2024-02-23T00:36:51.177+0000 7f8f2e7ff700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-23T00:36:51.337 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:50.093993+0000 mgr.y (mgr.14146) 119 : cluster [DBG] pgmap v94: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:51.337 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:50.168823+0000 mon.a (mon.0) 311 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-23T00:36:51.337 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:50.168875+0000 mon.a (mon.0) 312 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2024-02-23T00:36:51.337 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:50.169010+0000 mon.a (mon.0) 313 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-23T00:36:51.337 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:50.169583+0000 mon.c (mon.2) 9 : audit [INF] from='osd.2 [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-02-23T00:36:51.337 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:50.169839+0000 mon.a (mon.0) 314 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-02-23T00:36:51.337 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:50.093993+0000 mgr.y (mgr.14146) 119 : cluster [DBG] pgmap v94: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:51.337 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:50.168823+0000 mon.a (mon.0) 311 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-23T00:36:51.337 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:50.168875+0000 mon.a (mon.0) 312 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2024-02-23T00:36:51.338 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:50.169010+0000 mon.a (mon.0) 313 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-23T00:36:51.338 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:50.169583+0000 mon.c (mon.2) 9 : audit [INF] from='osd.2 [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-02-23T00:36:51.338 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:50.169839+0000 mon.a (mon.0) 314 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-02-23T00:36:51.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:50.093993+0000 mgr.y (mgr.14146) 119 : cluster [DBG] pgmap v94: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:51.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:50.168823+0000 mon.a (mon.0) 311 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-23T00:36:51.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:50.168875+0000 mon.a (mon.0) 312 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2024-02-23T00:36:51.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:50.169010+0000 mon.a (mon.0) 313 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-23T00:36:51.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:50.169583+0000 mon.c (mon.2) 9 : audit [INF] from='osd.2 [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-02-23T00:36:51.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:50.169839+0000 mon.a (mon.0) 314 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-02-23T00:36:52.744 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:51.170695+0000 mon.a (mon.0) 315 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-02-23T00:36:52.744 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:51.170791+0000 mon.a (mon.0) 316 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2024-02-23T00:36:52.744 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:51.171111+0000 mon.a (mon.0) 317 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-23T00:36:52.745 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:51.175033+0000 mon.a (mon.0) 318 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-23T00:36:52.745 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:51.170695+0000 mon.a (mon.0) 315 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-02-23T00:36:52.745 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:51.170791+0000 mon.a (mon.0) 316 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2024-02-23T00:36:52.745 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:51.171111+0000 mon.a (mon.0) 317 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-23T00:36:52.745 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:51.175033+0000 mon.a (mon.0) 318 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-23T00:36:52.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:51.170695+0000 mon.a (mon.0) 315 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-02-23T00:36:52.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:51.170791+0000 mon.a (mon.0) 316 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2024-02-23T00:36:52.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:51.171111+0000 mon.a (mon.0) 317 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-23T00:36:52.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:51.175033+0000 mon.a (mon.0) 318 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-23T00:36:53.848 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:50.256995+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:36:53.848 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:50.257051+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:36:53.848 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:52.094374+0000 mgr.y (mgr.14146) 120 : cluster [DBG] pgmap v97: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:53.848 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:52.176359+0000 mon.a (mon.0) 319 : cluster [INF] osd.2 [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] boot 2024-02-23T00:36:53.848 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:52.176419+0000 mon.a (mon.0) 320 : cluster [DBG] osdmap e18: 3 total, 3 up, 3 in 2024-02-23T00:36:53.848 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:52.176818+0000 mon.a (mon.0) 321 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-23T00:36:53.848 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:52.177210+0000 mon.a (mon.0) 322 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-23T00:36:53.848 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:53.177124+0000 mon.a (mon.0) 323 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-02-23T00:36:53.848 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:53.177204+0000 mon.a (mon.0) 324 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2024-02-23T00:36:53.848 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:53.177874+0000 mon.a (mon.0) 325 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-23T00:36:53.849 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:50.256995+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:36:53.849 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:50.257051+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:36:53.849 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:52.094374+0000 mgr.y (mgr.14146) 120 : cluster [DBG] pgmap v97: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:53.849 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:52.176359+0000 mon.a (mon.0) 319 : cluster [INF] osd.2 [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] boot 2024-02-23T00:36:53.849 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:52.176419+0000 mon.a (mon.0) 320 : cluster [DBG] osdmap e18: 3 total, 3 up, 3 in 2024-02-23T00:36:53.849 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:52.176818+0000 mon.a (mon.0) 321 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-23T00:36:53.849 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:52.177210+0000 mon.a (mon.0) 322 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-23T00:36:53.849 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:53.177124+0000 mon.a (mon.0) 323 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-02-23T00:36:53.849 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:53.177204+0000 mon.a (mon.0) 324 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2024-02-23T00:36:53.849 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:53.177874+0000 mon.a (mon.0) 325 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-23T00:36:53.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:50.256995+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:36:53.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:50.257051+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:36:53.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:52.094374+0000 mgr.y (mgr.14146) 120 : cluster [DBG] pgmap v97: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-23T00:36:53.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:52.176359+0000 mon.a (mon.0) 319 : cluster [INF] osd.2 [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] boot 2024-02-23T00:36:53.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:52.176419+0000 mon.a (mon.0) 320 : cluster [DBG] osdmap e18: 3 total, 3 up, 3 in 2024-02-23T00:36:53.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:52.176818+0000 mon.a (mon.0) 321 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-23T00:36:53.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:52.177210+0000 mon.a (mon.0) 322 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-23T00:36:53.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:53.177124+0000 mon.a (mon.0) 323 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-02-23T00:36:53.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:53.177204+0000 mon.a (mon.0) 324 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2024-02-23T00:36:53.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:53.177874+0000 mon.a (mon.0) 325 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-23T00:36:54.994 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-23T00:36:54.994 INFO:teuthology.orchestra.run.smithi047.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-23T00:36:54.995 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10+0 records in 2024-02-23T00:36:54.995 INFO:teuthology.orchestra.run.smithi047.stderr:10+0 records out 2024-02-23T00:36:54.995 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00946641 s, 1.1 GB/s 2024-02-23T00:36:54.995 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping successful for: 2024-02-23T00:36:55.306 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:54.094773+0000 mgr.y (mgr.14146) 121 : cluster [DBG] pgmap v100: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:36:55.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:54.178586+0000 mon.a (mon.0) 326 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-23T00:36:55.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:54.178655+0000 mon.a (mon.0) 327 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2024-02-23T00:36:55.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:54.094773+0000 mgr.y (mgr.14146) 121 : cluster [DBG] pgmap v100: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:36:55.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:54.178586+0000 mon.a (mon.0) 326 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-23T00:36:55.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:54.178655+0000 mon.a (mon.0) 327 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2024-02-23T00:36:55.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:54.094773+0000 mgr.y (mgr.14146) 121 : cluster [DBG] pgmap v100: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:36:55.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:54.178586+0000 mon.a (mon.0) 326 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-23T00:36:55.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:54.178655+0000 mon.a (mon.0) 327 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2024-02-23T00:36:55.813 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph orch daemon add osd smithi047:vg_nvme/lv_1 2024-02-23T00:36:56.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:55.184780+0000 mon.a (mon.0) 328 : cluster [DBG] osdmap e21: 3 total, 3 up, 3 in 2024-02-23T00:36:56.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:36:55.322105+0000 mgr.y (mgr.14146) 122 : cephadm [INF] Detected new or changed devices on smithi047 2024-02-23T00:36:56.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:55.326495+0000 mon.a (mon.0) 329 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:56.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:55.327348+0000 mon.a (mon.0) 330 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:36:56.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:55.328016+0000 mon.a (mon.0) 331 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:36:56.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:55.328681+0000 mon.a (mon.0) 332 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:36:56.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:36:55.329175+0000 mgr.y (mgr.14146) 123 : cephadm [INF] Adjusting osd_memory_target on smithi047 to 5367M 2024-02-23T00:36:56.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:55.333078+0000 mon.a (mon.0) 333 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:56.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:55.334065+0000 mon.a (mon.0) 334 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:56.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:55.334653+0000 mon.a (mon.0) 335 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:36:56.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:55.338458+0000 mon.a (mon.0) 336 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:56.519 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:55.184780+0000 mon.a (mon.0) 328 : cluster [DBG] osdmap e21: 3 total, 3 up, 3 in 2024-02-23T00:36:56.519 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:36:55.322105+0000 mgr.y (mgr.14146) 122 : cephadm [INF] Detected new or changed devices on smithi047 2024-02-23T00:36:56.519 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:55.326495+0000 mon.a (mon.0) 329 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:56.519 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:55.327348+0000 mon.a (mon.0) 330 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:36:56.519 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:55.328016+0000 mon.a (mon.0) 331 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:36:56.519 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:55.328681+0000 mon.a (mon.0) 332 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:36:56.519 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:36:55.329175+0000 mgr.y (mgr.14146) 123 : cephadm [INF] Adjusting osd_memory_target on smithi047 to 5367M 2024-02-23T00:36:56.519 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:55.333078+0000 mon.a (mon.0) 333 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:56.519 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:55.334065+0000 mon.a (mon.0) 334 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:56.519 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:55.334653+0000 mon.a (mon.0) 335 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:36:56.520 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:55.338458+0000 mon.a (mon.0) 336 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:56.520 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:55.184780+0000 mon.a (mon.0) 328 : cluster [DBG] osdmap e21: 3 total, 3 up, 3 in 2024-02-23T00:36:56.520 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:36:55.322105+0000 mgr.y (mgr.14146) 122 : cephadm [INF] Detected new or changed devices on smithi047 2024-02-23T00:36:56.520 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:55.326495+0000 mon.a (mon.0) 329 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:56.520 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:55.327348+0000 mon.a (mon.0) 330 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:36:56.520 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:55.328016+0000 mon.a (mon.0) 331 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:36:56.520 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:55.328681+0000 mon.a (mon.0) 332 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:36:56.520 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:36:55.329175+0000 mgr.y (mgr.14146) 123 : cephadm [INF] Adjusting osd_memory_target on smithi047 to 5367M 2024-02-23T00:36:56.520 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:55.333078+0000 mon.a (mon.0) 333 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:56.520 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:55.334065+0000 mon.a (mon.0) 334 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:56.520 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:55.334653+0000 mon.a (mon.0) 335 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:36:56.520 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:55.338458+0000 mon.a (mon.0) 336 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:36:57.506 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:56.095075+0000 mgr.y (mgr.14146) 124 : cluster [DBG] pgmap v103: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:36:57.506 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:56.095075+0000 mgr.y (mgr.14146) 124 : cluster [DBG] pgmap v103: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:36:57.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:56.095075+0000 mgr.y (mgr.14146) 124 : cluster [DBG] pgmap v103: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:36:57.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.785+0000 7f052aeae700 1 -- 172.21.15.47:0/2160542852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05240fd680 msgr2=0x7f05240ffb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:36:57.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.785+0000 7f052aeae700 1 --2- 172.21.15.47:0/2160542852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05240fd680 0x7f05240ffb10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f051c009be0 tx=0x7f051c004d10).stop 2024-02-23T00:36:57.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.785+0000 7f052aeae700 1 -- 172.21.15.47:0/2160542852 shutdown_connections 2024-02-23T00:36:57.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.785+0000 7f052aeae700 1 --2- 172.21.15.47:0/2160542852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05240fd680 0x7f05240ffb10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:36:57.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.785+0000 7f052aeae700 1 --2- 172.21.15.47:0/2160542852 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0524061670 0x7f05240fa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:36:57.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.785+0000 7f052aeae700 1 --2- 172.21.15.47:0/2160542852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05240fac50 0x7f05240fd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:36:57.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.785+0000 7f052aeae700 1 -- 172.21.15.47:0/2160542852 >> 172.21.15.47:0/2160542852 conn(0x7f05240f2190 msgr2=0x7f05240f45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:36:57.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.785+0000 7f052aeae700 1 -- 172.21.15.47:0/2160542852 shutdown_connections 2024-02-23T00:36:57.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.785+0000 7f052aeae700 1 -- 172.21.15.47:0/2160542852 wait complete. 2024-02-23T00:36:57.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.786+0000 7f052aeae700 1 librados: starting msgr at 2024-02-23T00:36:57.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.786+0000 7f052aeae700 1 librados: starting objecter 2024-02-23T00:36:57.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.786+0000 7f052aeae700 1 Processor -- start 2024-02-23T00:36:57.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.786+0000 7f052aeae700 1 -- start start 2024-02-23T00:36:57.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.786+0000 7f052aeae700 1 librados: setting wanted keys 2024-02-23T00:36:57.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.786+0000 7f052aeae700 1 librados: calling monclient init 2024-02-23T00:36:57.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.786+0000 7f052aeae700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0524061670 0x7f05240f7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:36:57.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.786+0000 7f052aeae700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05240fac50 0x7f05240f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:36:57.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.786+0000 7f052aeae700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05240fd680 0x7f05240f6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:36:57.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.786+0000 7f052aeae700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05240f70e0 con 0x7f05240fac50 2024-02-23T00:36:57.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.786+0000 7f052aeae700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05240f7250 con 0x7f05240fd680 2024-02-23T00:36:57.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.786+0000 7f052aeae700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05240f7410 con 0x7f0524061670 2024-02-23T00:36:57.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.786+0000 7f0528c4a700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0524061670 0x7f05240f7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:36:57.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.786+0000 7f0523fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05240fac50 0x7f05240f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:36:57.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.786+0000 7f0528c4a700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0524061670 0x7f05240f7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52648/0 (socket says 172.21.15.47:52648) 2024-02-23T00:36:57.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.786+0000 7f0523fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05240fac50 0x7f05240f65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46076/0 (socket says 172.21.15.47:46076) 2024-02-23T00:36:57.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.786+0000 7f0528c4a700 1 -- 172.21.15.47:0/2723017977 learned_addr learned my addr 172.21.15.47:0/2723017977 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:36:57.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.787+0000 7f052944b700 1 --2- 172.21.15.47:0/2723017977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05240fd680 0x7f05240f6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:36:57.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.787+0000 7f0528c4a700 1 -- 172.21.15.47:0/2723017977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05240fd680 msgr2=0x7f05240f6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:36:57.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.787+0000 7f0528c4a700 1 --2- 172.21.15.47:0/2723017977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05240fd680 0x7f05240f6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:36:57.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.787+0000 7f0528c4a700 1 -- 172.21.15.47:0/2723017977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05240fac50 msgr2=0x7f05240f65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:36:57.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.787+0000 7f0528c4a700 1 --2- 172.21.15.47:0/2723017977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05240fac50 0x7f05240f65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:36:57.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.787+0000 7f0528c4a700 1 -- 172.21.15.47:0/2723017977 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f051c0097c0 con 0x7f0524061670 2024-02-23T00:36:57.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.787+0000 7f0523fff700 1 --2- 172.21.15.47:0/2723017977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05240fac50 0x7f05240f65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:36:57.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.787+0000 7f0528c4a700 1 --2- 172.21.15.47:0/2723017977 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0524061670 0x7f05240f7f70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f051800e970 tx=0x7f051800b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:36:57.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.787+0000 7f0521ffb700 10 client.?.objecter ms_handle_connect 0x7f0524061670 2024-02-23T00:36:57.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.787+0000 7f0521ffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:36:57.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.787+0000 7f0521ffb700 1 -- 172.21.15.47:0/2723017977 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051800ee60 con 0x7f0524061670 2024-02-23T00:36:57.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.787+0000 7f0521ffb700 1 -- 172.21.15.47:0/2723017977 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f051800f040 con 0x7f0524061670 2024-02-23T00:36:57.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.787+0000 7f0521ffb700 1 -- 172.21.15.47:0/2723017977 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051800c5b0 con 0x7f0524061670 2024-02-23T00:36:57.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.787+0000 7f052aeae700 1 -- 172.21.15.47:0/2723017977 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0524111f40 con 0x7f0524061670 2024-02-23T00:36:57.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.787+0000 7f052aeae700 10 client.24203.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:36:57.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.787+0000 7f052aeae700 1 -- 172.21.15.47:0/2723017977 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0524112340 con 0x7f0524061670 2024-02-23T00:36:57.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.787+0000 7f052aeae700 1 librados: init done 2024-02-23T00:36:57.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.788+0000 7f052aeae700 1 -- 172.21.15.47:0/2723017977 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f052405f360 con 0x7f0524061670 2024-02-23T00:36:57.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.789+0000 7f0521ffb700 1 -- 172.21.15.47:0/2723017977 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 14) v1 ==== 77064+0+0 (secure 0 0 0) 0x7f0518007b40 con 0x7f0524061670 2024-02-23T00:36:57.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.789+0000 7f0521ffb700 1 --2- 172.21.15.47:0/2723017977 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f0514062bd0 0x7f0514065090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:36:57.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.789+0000 7f0521ffb700 1 -- 172.21.15.47:0/2723017977 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 2990+0+0 (secure 0 0 0) 0x7f051807b880 con 0x7f0524061670 2024-02-23T00:36:57.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.789+0000 7f0523fff700 1 --2- 172.21.15.47:0/2723017977 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f0514062bd0 0x7f0514065090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:36:57.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.789+0000 7f0521ffb700 10 client.24203.objecter ms_dispatch 0x7f0524001550 osd_map(21..21 src has 1..21) v4 2024-02-23T00:36:57.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.789+0000 7f0521ffb700 3 client.24203.objecter handle_osd_map got epochs [21,21] > 0 2024-02-23T00:36:57.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.789+0000 7f0521ffb700 3 client.24203.objecter handle_osd_map decoding full epoch 21 2024-02-23T00:36:57.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.789+0000 7f0521ffb700 20 client.24203.objecter dump_active .. 0 homeless 2024-02-23T00:36:57.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.790+0000 7f0521ffb700 10 client.24203.objecter ms_handle_connect 0x7f0514062bd0 2024-02-23T00:36:57.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.790+0000 7f0523fff700 1 --2- 172.21.15.47:0/2723017977 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f0514062bd0 0x7f0514065090 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f0524061440 tx=0x7f0510009040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:36:57.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.794+0000 7f0521ffb700 1 -- 172.21.15.47:0/2723017977 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f051804af60 con 0x7f0524061670 2024-02-23T00:36:57.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:36:57.974+0000 7f052aeae700 1 -- 172.21.15.47:0/2723017977 --> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0524059120 con 0x7f0514062bd0 2024-02-23T00:36:58.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:57.977873+0000 mon.a (mon.0) 337 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-23T00:36:58.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:57.979929+0000 mon.a (mon.0) 338 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-23T00:36:58.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:57.980559+0000 mon.a (mon.0) 339 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:58.729 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:57.977873+0000 mon.a (mon.0) 337 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-23T00:36:58.729 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:57.979929+0000 mon.a (mon.0) 338 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-23T00:36:58.729 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:57.980559+0000 mon.a (mon.0) 339 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:58.729 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:57.977873+0000 mon.a (mon.0) 337 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-23T00:36:58.729 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:57.979929+0000 mon.a (mon.0) 338 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-23T00:36:58.729 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:57.980559+0000 mon.a (mon.0) 339 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:36:59.532 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:36:57.975820+0000 mgr.y (mgr.14146) 125 : audit [DBG] from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:36:59.532 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:36:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:36:58.095389+0000 mgr.y (mgr.14146) 126 : cluster [DBG] pgmap v104: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:36:59.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:36:57.975820+0000 mgr.y (mgr.14146) 125 : audit [DBG] from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:36:59.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:36:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:36:58.095389+0000 mgr.y (mgr.14146) 126 : cluster [DBG] pgmap v104: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:36:59.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:36:57.975820+0000 mgr.y (mgr.14146) 125 : audit [DBG] from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:36:59.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:36:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:36:58.095389+0000 mgr.y (mgr.14146) 126 : cluster [DBG] pgmap v104: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:37:01.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:00.095694+0000 mgr.y (mgr.14146) 127 : cluster [DBG] pgmap v105: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:37:01.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:00.668190+0000 mon.c (mon.2) 10 : audit [INF] from='client.? 172.21.15.47:0/3645441140' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c88debab-cb0b-46f6-a7d8-c5dc61fc059f"}]: dispatch 2024-02-23T00:37:01.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:00.668611+0000 mon.a (mon.0) 340 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c88debab-cb0b-46f6-a7d8-c5dc61fc059f"}]: dispatch 2024-02-23T00:37:01.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:00.672235+0000 mon.a (mon.0) 341 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c88debab-cb0b-46f6-a7d8-c5dc61fc059f"}]': finished 2024-02-23T00:37:01.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:00.672293+0000 mon.a (mon.0) 342 : cluster [DBG] osdmap e22: 4 total, 3 up, 4 in 2024-02-23T00:37:01.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:00.672419+0000 mon.a (mon.0) 343 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-23T00:37:01.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:01.290534+0000 mon.c (mon.2) 11 : audit [DBG] from='client.? 172.21.15.47:0/1124375659' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-23T00:37:01.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:00.095694+0000 mgr.y (mgr.14146) 127 : cluster [DBG] pgmap v105: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:37:01.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:00.668190+0000 mon.c (mon.2) 10 : audit [INF] from='client.? 172.21.15.47:0/3645441140' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c88debab-cb0b-46f6-a7d8-c5dc61fc059f"}]: dispatch 2024-02-23T00:37:01.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:00.668611+0000 mon.a (mon.0) 340 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c88debab-cb0b-46f6-a7d8-c5dc61fc059f"}]: dispatch 2024-02-23T00:37:01.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:00.672235+0000 mon.a (mon.0) 341 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c88debab-cb0b-46f6-a7d8-c5dc61fc059f"}]': finished 2024-02-23T00:37:01.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:00.672293+0000 mon.a (mon.0) 342 : cluster [DBG] osdmap e22: 4 total, 3 up, 4 in 2024-02-23T00:37:01.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:00.672419+0000 mon.a (mon.0) 343 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-23T00:37:01.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:01.290534+0000 mon.c (mon.2) 11 : audit [DBG] from='client.? 172.21.15.47:0/1124375659' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-23T00:37:01.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:00.095694+0000 mgr.y (mgr.14146) 127 : cluster [DBG] pgmap v105: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:37:01.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:00.668190+0000 mon.c (mon.2) 10 : audit [INF] from='client.? 172.21.15.47:0/3645441140' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c88debab-cb0b-46f6-a7d8-c5dc61fc059f"}]: dispatch 2024-02-23T00:37:01.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:00.668611+0000 mon.a (mon.0) 340 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c88debab-cb0b-46f6-a7d8-c5dc61fc059f"}]: dispatch 2024-02-23T00:37:01.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:00.672235+0000 mon.a (mon.0) 341 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c88debab-cb0b-46f6-a7d8-c5dc61fc059f"}]': finished 2024-02-23T00:37:01.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:00.672293+0000 mon.a (mon.0) 342 : cluster [DBG] osdmap e22: 4 total, 3 up, 4 in 2024-02-23T00:37:01.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:00.672419+0000 mon.a (mon.0) 343 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-23T00:37:01.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:01.290534+0000 mon.c (mon.2) 11 : audit [DBG] from='client.? 172.21.15.47:0/1124375659' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-23T00:37:02.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:37:02.788+0000 7f052a6ad700 10 client.24203.objecter tick 2024-02-23T00:37:03.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:02.096208+0000 mgr.y (mgr.14146) 128 : cluster [DBG] pgmap v107: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:37:03.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:02.096208+0000 mgr.y (mgr.14146) 128 : cluster [DBG] pgmap v107: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:37:03.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:02.096208+0000 mgr.y (mgr.14146) 128 : cluster [DBG] pgmap v107: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:37:05.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:04.096577+0000 mgr.y (mgr.14146) 129 : cluster [DBG] pgmap v108: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:37:05.745 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:04.096577+0000 mgr.y (mgr.14146) 129 : cluster [DBG] pgmap v108: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:37:05.746 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:04.096577+0000 mgr.y (mgr.14146) 129 : cluster [DBG] pgmap v108: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:37:07.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:06.096973+0000 mgr.y (mgr.14146) 130 : cluster [DBG] pgmap v109: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:37:07.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:07.241854+0000 mon.a (mon.0) 344 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-23T00:37:07.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:07.242548+0000 mon.a (mon.0) 345 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:37:07.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:06.096973+0000 mgr.y (mgr.14146) 130 : cluster [DBG] pgmap v109: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:37:07.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:07.241854+0000 mon.a (mon.0) 344 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-23T00:37:07.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:07.242548+0000 mon.a (mon.0) 345 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:37:07.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:06.096973+0000 mgr.y (mgr.14146) 130 : cluster [DBG] pgmap v109: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:37:07.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:07.241854+0000 mon.a (mon.0) 344 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-23T00:37:07.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:07.242548+0000 mon.a (mon.0) 345 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:37:07.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:37:07.789+0000 7f052a6ad700 10 client.24203.objecter tick 2024-02-23T00:37:08.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:37:07.243111+0000 mgr.y (mgr.14146) 131 : cephadm [INF] Deploying daemon osd.3 on smithi047 2024-02-23T00:37:08.776 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:37:07.243111+0000 mgr.y (mgr.14146) 131 : cephadm [INF] Deploying daemon osd.3 on smithi047 2024-02-23T00:37:08.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:37:07.243111+0000 mgr.y (mgr.14146) 131 : cephadm [INF] Deploying daemon osd.3 on smithi047 2024-02-23T00:37:09.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:08.097365+0000 mgr.y (mgr.14146) 132 : cluster [DBG] pgmap v110: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:37:09.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:08.097365+0000 mgr.y (mgr.14146) 132 : cluster [DBG] pgmap v110: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:37:09.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:08.097365+0000 mgr.y (mgr.14146) 132 : cluster [DBG] pgmap v110: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:37:11.616 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:10.097653+0000 mgr.y (mgr.14146) 133 : cluster [DBG] pgmap v111: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:37:11.617 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:10.097653+0000 mgr.y (mgr.14146) 133 : cluster [DBG] pgmap v111: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:37:11.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:10.097653+0000 mgr.y (mgr.14146) 133 : cluster [DBG] pgmap v111: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:37:12.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:37:12.790+0000 7f052a6ad700 10 client.24203.objecter tick 2024-02-23T00:37:13.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:12.097975+0000 mgr.y (mgr.14146) 134 : cluster [DBG] pgmap v112: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:37:13.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:12.935711+0000 mon.a (mon.0) 346 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:13.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:12.940109+0000 mon.a (mon.0) 347 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:37:13.767 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:12.097975+0000 mgr.y (mgr.14146) 134 : cluster [DBG] pgmap v112: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:37:13.767 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:12.935711+0000 mon.a (mon.0) 346 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:13.767 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:12.940109+0000 mon.a (mon.0) 347 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:37:13.768 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:12.097975+0000 mgr.y (mgr.14146) 134 : cluster [DBG] pgmap v112: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:37:13.768 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:12.935711+0000 mon.a (mon.0) 346 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:13.768 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:12.940109+0000 mon.a (mon.0) 347 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:37:15.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:13.801059+0000 osd.0 (osd.0) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:37:15.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:13.801144+0000 osd.0 (osd.0) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:37:15.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:14.098524+0000 mgr.y (mgr.14146) 135 : cluster [DBG] pgmap v113: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:37:15.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:14.371207+0000 mon.a (mon.0) 348 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2024-02-23T00:37:15.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:14.371384+0000 mon.a (mon.0) 349 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-23T00:37:15.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:13.801059+0000 osd.0 (osd.0) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:37:15.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:13.801144+0000 osd.0 (osd.0) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:37:15.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:14.098524+0000 mgr.y (mgr.14146) 135 : cluster [DBG] pgmap v113: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:37:15.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:14.371207+0000 mon.a (mon.0) 348 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2024-02-23T00:37:15.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:14.371384+0000 mon.a (mon.0) 349 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-23T00:37:15.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:13.801059+0000 osd.0 (osd.0) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:37:15.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:13.801144+0000 osd.0 (osd.0) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:37:15.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:14.098524+0000 mgr.y (mgr.14146) 135 : cluster [DBG] pgmap v113: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:37:15.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:14.371207+0000 mon.a (mon.0) 348 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2024-02-23T00:37:15.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:14.371384+0000 mon.a (mon.0) 349 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-23T00:37:16.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:37:16.652+0000 7f0521ffb700 1 -- 172.21.15.47:0/2723017977 <== mgr.14146 v2:172.21.15.47:6800/622994871 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0524059120 con 0x7f0514062bd0 2024-02-23T00:37:16.655 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 3 on host 'smithi047' 2024-02-23T00:37:16.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:37:16.654+0000 7f052aeae700 10 librados: watch_flush enter 2024-02-23T00:37:16.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:37:16.654+0000 7f052aeae700 10 librados: watch_flush exit 2024-02-23T00:37:16.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:37:16.654+0000 7f052aeae700 20 client.24203.objecter shutdown clearing up homeless session... 2024-02-23T00:37:16.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:37:16.654+0000 7f052aeae700 10 client.24203.objecter successfully canceled tick 2024-02-23T00:37:16.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:37:16.654+0000 7f052aeae700 1 -- 172.21.15.47:0/2723017977 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f0514062bd0 msgr2=0x7f0514065090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:37:16.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:37:16.654+0000 7f052aeae700 1 --2- 172.21.15.47:0/2723017977 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f0514062bd0 0x7f0514065090 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f0524061440 tx=0x7f0510009040).stop 2024-02-23T00:37:16.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:37:16.654+0000 7f052aeae700 1 -- 172.21.15.47:0/2723017977 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0524061670 msgr2=0x7f05240f7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:37:16.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:37:16.654+0000 7f052aeae700 1 --2- 172.21.15.47:0/2723017977 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0524061670 0x7f05240f7f70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f051800e970 tx=0x7f051800b040).stop 2024-02-23T00:37:16.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:37:16.654+0000 7f052aeae700 1 -- 172.21.15.47:0/2723017977 shutdown_connections 2024-02-23T00:37:16.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:37:16.655+0000 7f052aeae700 1 --2- 172.21.15.47:0/2723017977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05240fd680 0x7f05240f6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:37:16.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:37:16.655+0000 7f052aeae700 1 --2- 172.21.15.47:0/2723017977 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0524061670 0x7f05240f7f70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:37:16.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:37:16.655+0000 7f052aeae700 1 --2- 172.21.15.47:0/2723017977 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f0514062bd0 0x7f0514065090 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:37:16.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:37:16.655+0000 7f052aeae700 1 --2- 172.21.15.47:0/2723017977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05240fac50 0x7f05240f65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:37:16.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:37:16.655+0000 7f052aeae700 1 -- 172.21.15.47:0/2723017977 >> 172.21.15.47:0/2723017977 conn(0x7f05240f2190 msgr2=0x7f05240f8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:37:16.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:37:16.655+0000 7f052aeae700 1 -- 172.21.15.47:0/2723017977 shutdown_connections 2024-02-23T00:37:16.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:37:16.655+0000 7f052aeae700 1 -- 172.21.15.47:0/2723017977 wait complete. 2024-02-23T00:37:16.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:37:16.655+0000 7f052aeae700 1 librados: shutdown 2024-02-23T00:37:17.056 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:15.825618+0000 mon.a (mon.0) 350 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:17.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:15.826408+0000 mon.a (mon.0) 351 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:37:17.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:15.827133+0000 mon.a (mon.0) 352 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:37:17.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:15.831391+0000 mon.a (mon.0) 353 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:17.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:16.041578+0000 mon.a (mon.0) 354 : audit [INF] from='osd.3 [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-23T00:37:17.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:16.098996+0000 mgr.y (mgr.14146) 136 : cluster [DBG] pgmap v115: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:37:17.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:16.648522+0000 mon.a (mon.0) 355 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:37:17.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:16.651699+0000 mon.a (mon.0) 356 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:17.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:15.825618+0000 mon.a (mon.0) 350 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:17.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:15.826408+0000 mon.a (mon.0) 351 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:37:17.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:15.827133+0000 mon.a (mon.0) 352 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:37:17.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:15.831391+0000 mon.a (mon.0) 353 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:17.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:16.041578+0000 mon.a (mon.0) 354 : audit [INF] from='osd.3 [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-23T00:37:17.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:16.098996+0000 mgr.y (mgr.14146) 136 : cluster [DBG] pgmap v115: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:37:17.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:16.648522+0000 mon.a (mon.0) 355 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:37:17.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:16.651699+0000 mon.a (mon.0) 356 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:17.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:15.825618+0000 mon.a (mon.0) 350 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:17.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:15.826408+0000 mon.a (mon.0) 351 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:37:17.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:15.827133+0000 mon.a (mon.0) 352 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:37:17.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:15.831391+0000 mon.a (mon.0) 353 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:17.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:16.041578+0000 mon.a (mon.0) 354 : audit [INF] from='osd.3 [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-23T00:37:17.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:16.098996+0000 mgr.y (mgr.14146) 136 : cluster [DBG] pgmap v115: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:37:17.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:16.648522+0000 mon.a (mon.0) 355 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:37:17.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:16.651699+0000 mon.a (mon.0) 356 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:17.421 DEBUG:teuthology.orchestra.run.smithi047:osd.3> sudo journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@osd.3.service 2024-02-23T00:37:17.423 INFO:tasks.cephadm:Deploying osd.4 on smithi196 with /dev/vg_nvme/lv_4... 2024-02-23T00:37:17.424 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-23T00:37:17.458 INFO:journalctl@ceph.osd.3.smithi047.stdout:-- Logs begin at Fri 2024-02-23 00:13:17 UTC. -- 2024-02-23T00:37:18.056 INFO:journalctl@ceph.osd.3.smithi047.stdout:Feb 23 00:37:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-3[62151]: debug 2024-02-23T00:37:17.841+0000 7f781073e700 -1 osd.3 0 waiting for initial osdmap 2024-02-23T00:37:18.057 INFO:journalctl@ceph.osd.3.smithi047.stdout:Feb 23 00:37:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-3[62151]: debug 2024-02-23T00:37:17.847+0000 7f780c6b9700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-23T00:37:18.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:16.838838+0000 mon.a (mon.0) 357 : audit [INF] from='osd.3 [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-23T00:37:18.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:16.838896+0000 mon.a (mon.0) 358 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2024-02-23T00:37:18.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:16.839035+0000 mon.a (mon.0) 359 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-23T00:37:18.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:16.839357+0000 mon.a (mon.0) 360 : audit [INF] from='osd.3 [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-02-23T00:37:18.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:16.838838+0000 mon.a (mon.0) 357 : audit [INF] from='osd.3 [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-23T00:37:18.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:16.838896+0000 mon.a (mon.0) 358 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2024-02-23T00:37:18.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:16.839035+0000 mon.a (mon.0) 359 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-23T00:37:18.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:16.839357+0000 mon.a (mon.0) 360 : audit [INF] from='osd.3 [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-02-23T00:37:18.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:16.838838+0000 mon.a (mon.0) 357 : audit [INF] from='osd.3 [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-23T00:37:18.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:16.838896+0000 mon.a (mon.0) 358 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2024-02-23T00:37:18.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:16.839035+0000 mon.a (mon.0) 359 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-23T00:37:18.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:16.839357+0000 mon.a (mon.0) 360 : audit [INF] from='osd.3 [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-02-23T00:37:19.017 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:17.839565+0000 mon.a (mon.0) 361 : audit [INF] from='osd.3 [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-02-23T00:37:19.018 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:17.839613+0000 mon.a (mon.0) 362 : cluster [DBG] osdmap e25: 4 total, 3 up, 4 in 2024-02-23T00:37:19.018 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:17.839935+0000 mon.a (mon.0) 363 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-23T00:37:19.018 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:17.845458+0000 mon.a (mon.0) 364 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-23T00:37:19.018 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:18.099327+0000 mgr.y (mgr.14146) 137 : cluster [DBG] pgmap v118: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:37:19.147 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:17.839565+0000 mon.a (mon.0) 361 : audit [INF] from='osd.3 [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-02-23T00:37:19.147 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:17.839613+0000 mon.a (mon.0) 362 : cluster [DBG] osdmap e25: 4 total, 3 up, 4 in 2024-02-23T00:37:19.148 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:17.839935+0000 mon.a (mon.0) 363 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-23T00:37:19.148 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:17.845458+0000 mon.a (mon.0) 364 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-23T00:37:19.148 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:18.099327+0000 mgr.y (mgr.14146) 137 : cluster [DBG] pgmap v118: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:37:19.148 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:17.839565+0000 mon.a (mon.0) 361 : audit [INF] from='osd.3 [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-02-23T00:37:19.149 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:17.839613+0000 mon.a (mon.0) 362 : cluster [DBG] osdmap e25: 4 total, 3 up, 4 in 2024-02-23T00:37:19.149 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:17.839935+0000 mon.a (mon.0) 363 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-23T00:37:19.149 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:17.845458+0000 mon.a (mon.0) 364 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-23T00:37:19.149 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:18.099327+0000 mgr.y (mgr.14146) 137 : cluster [DBG] pgmap v118: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-23T00:37:19.871 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-23T00:37:19.871 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-23T00:37:19.871 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-02-23T00:37:19.871 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-02-23T00:37:19.871 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130107 s, 806 MB/s 2024-02-23T00:37:19.871 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-02-23T00:37:20.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:17.082776+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:37:20.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:17.082821+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:37:20.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:18.843230+0000 mon.a (mon.0) 365 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-23T00:37:20.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:18.846994+0000 mon.a (mon.0) 366 : cluster [INF] osd.3 [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] boot 2024-02-23T00:37:20.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:18.847041+0000 mon.a (mon.0) 367 : cluster [DBG] osdmap e26: 4 total, 4 up, 4 in 2024-02-23T00:37:20.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:18.847654+0000 mon.a (mon.0) 368 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-23T00:37:20.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:17.082776+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:37:20.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:17.082821+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:37:20.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:18.843230+0000 mon.a (mon.0) 365 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-23T00:37:20.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:18.846994+0000 mon.a (mon.0) 366 : cluster [INF] osd.3 [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] boot 2024-02-23T00:37:20.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:18.847041+0000 mon.a (mon.0) 367 : cluster [DBG] osdmap e26: 4 total, 4 up, 4 in 2024-02-23T00:37:20.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:18.847654+0000 mon.a (mon.0) 368 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-23T00:37:20.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:17.082776+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:37:20.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:17.082821+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:37:20.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:18.843230+0000 mon.a (mon.0) 365 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-23T00:37:20.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:18.846994+0000 mon.a (mon.0) 366 : cluster [INF] osd.3 [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] boot 2024-02-23T00:37:20.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:18.847041+0000 mon.a (mon.0) 367 : cluster [DBG] osdmap e26: 4 total, 4 up, 4 in 2024-02-23T00:37:20.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:18.847654+0000 mon.a (mon.0) 368 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-23T00:37:20.600 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph orch daemon add osd smithi196:vg_nvme/lv_4 2024-02-23T00:37:21.056 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:19.852169+0000 mon.a (mon.0) 369 : cluster [DBG] osdmap e27: 4 total, 4 up, 4 in 2024-02-23T00:37:21.056 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:20.099569+0000 mgr.y (mgr.14146) 138 : cluster [DBG] pgmap v121: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:21.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:19.852169+0000 mon.a (mon.0) 369 : cluster [DBG] osdmap e27: 4 total, 4 up, 4 in 2024-02-23T00:37:21.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:20.099569+0000 mgr.y (mgr.14146) 138 : cluster [DBG] pgmap v121: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:21.115 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:19.852169+0000 mon.a (mon.0) 369 : cluster [DBG] osdmap e27: 4 total, 4 up, 4 in 2024-02-23T00:37:21.115 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:20.099569+0000 mgr.y (mgr.14146) 138 : cluster [DBG] pgmap v121: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:22.112 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:37:20.931629+0000 mgr.y (mgr.14146) 139 : cephadm [INF] Detected new or changed devices on smithi047 2024-02-23T00:37:22.113 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:20.935807+0000 mon.a (mon.0) 370 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:22.113 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:20.936700+0000 mon.a (mon.0) 371 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:37:22.113 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:20.937331+0000 mon.a (mon.0) 372 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:37:22.113 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:20.937905+0000 mon.a (mon.0) 373 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:37:22.113 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:20.938511+0000 mon.a (mon.0) 374 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:37:22.113 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:37:20.938980+0000 mgr.y (mgr.14146) 140 : cephadm [INF] Adjusting osd_memory_target on smithi047 to 4025M 2024-02-23T00:37:22.113 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:20.942070+0000 mon.a (mon.0) 375 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:22.113 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:20.943763+0000 mon.a (mon.0) 376 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:37:22.113 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:20.944320+0000 mon.a (mon.0) 377 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:37:22.113 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:20.947937+0000 mon.a (mon.0) 378 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:22.306 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:37:20.931629+0000 mgr.y (mgr.14146) 139 : cephadm [INF] Detected new or changed devices on smithi047 2024-02-23T00:37:22.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:20.935807+0000 mon.a (mon.0) 370 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:22.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:20.936700+0000 mon.a (mon.0) 371 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:37:22.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:20.937331+0000 mon.a (mon.0) 372 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:37:22.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:20.937905+0000 mon.a (mon.0) 373 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:37:22.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:20.938511+0000 mon.a (mon.0) 374 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:37:22.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:37:20.938980+0000 mgr.y (mgr.14146) 140 : cephadm [INF] Adjusting osd_memory_target on smithi047 to 4025M 2024-02-23T00:37:22.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:20.942070+0000 mon.a (mon.0) 375 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:22.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:20.943763+0000 mon.a (mon.0) 376 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:37:22.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:20.944320+0000 mon.a (mon.0) 377 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:37:22.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:20.947937+0000 mon.a (mon.0) 378 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:22.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:37:20.931629+0000 mgr.y (mgr.14146) 139 : cephadm [INF] Detected new or changed devices on smithi047 2024-02-23T00:37:22.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:20.935807+0000 mon.a (mon.0) 370 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:22.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:20.936700+0000 mon.a (mon.0) 371 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:37:22.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:20.937331+0000 mon.a (mon.0) 372 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:37:22.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:20.937905+0000 mon.a (mon.0) 373 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:37:22.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:20.938511+0000 mon.a (mon.0) 374 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:37:22.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:37:20.938980+0000 mgr.y (mgr.14146) 140 : cephadm [INF] Adjusting osd_memory_target on smithi047 to 4025M 2024-02-23T00:37:22.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:20.942070+0000 mon.a (mon.0) 375 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:22.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:20.943763+0000 mon.a (mon.0) 376 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:37:22.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:20.944320+0000 mon.a (mon.0) 377 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:37:22.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:20.947937+0000 mon.a (mon.0) 378 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:22.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.639+0000 7f7d22301700 1 -- 172.21.15.196:0/727999100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d1c0f6990 msgr2=0x7f7d1c0f6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:37:22.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.639+0000 7f7d22301700 1 --2- 172.21.15.196:0/727999100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d1c0f6990 0x7f7d1c0f6db0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f7d0c0048a0 tx=0x7f7d0c009cb0).stop 2024-02-23T00:37:22.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.639+0000 7f7d22301700 1 -- 172.21.15.196:0/727999100 shutdown_connections 2024-02-23T00:37:22.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.639+0000 7f7d22301700 1 --2- 172.21.15.196:0/727999100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d1c0f8580 0x7f7d1c0fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:37:22.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.639+0000 7f7d22301700 1 --2- 172.21.15.196:0/727999100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d1c0f6990 0x7f7d1c0f6db0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:37:22.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.639+0000 7f7d22301700 1 --2- 172.21.15.196:0/727999100 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7d1c0f7c30 0x7f7d1c0f8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:37:22.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.639+0000 7f7d22301700 1 -- 172.21.15.196:0/727999100 >> 172.21.15.196:0/727999100 conn(0x7f7d1c0f2190 msgr2=0x7f7d1c0f45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:37:22.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.640+0000 7f7d22301700 1 -- 172.21.15.196:0/727999100 shutdown_connections 2024-02-23T00:37:22.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.640+0000 7f7d22301700 1 -- 172.21.15.196:0/727999100 wait complete. 2024-02-23T00:37:22.647 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.640+0000 7f7d22301700 1 librados: starting msgr at 2024-02-23T00:37:22.647 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.640+0000 7f7d22301700 1 librados: starting objecter 2024-02-23T00:37:22.647 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.640+0000 7f7d22301700 1 Processor -- start 2024-02-23T00:37:22.647 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.640+0000 7f7d22301700 1 -- start start 2024-02-23T00:37:22.647 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.640+0000 7f7d22301700 1 librados: setting wanted keys 2024-02-23T00:37:22.647 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.640+0000 7f7d22301700 1 librados: calling monclient init 2024-02-23T00:37:22.647 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.640+0000 7f7d22301700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d1c0f6990 0x7f7d1c0fa910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:37:22.647 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.640+0000 7f7d22301700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d1c0f7c30 0x7f7d1c0f8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:37:22.647 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.640+0000 7f7d22301700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7d1c0f8580 0x7f7d1c0f9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:37:22.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.640+0000 7f7d22301700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c0f9a80 con 0x7f7d1c0f6990 2024-02-23T00:37:22.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.640+0000 7f7d22301700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c0f9bf0 con 0x7f7d1c0f7c30 2024-02-23T00:37:22.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.640+0000 7f7d22301700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c0f9db0 con 0x7f7d1c0f8580 2024-02-23T00:37:22.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.641+0000 7f7d1b7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d1c0f7c30 0x7f7d1c0f8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:37:22.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.641+0000 7f7d1b7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d1c0f7c30 0x7f7d1c0f8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54674/0 (socket says 172.21.15.196:54674) 2024-02-23T00:37:22.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.641+0000 7f7d1b7fe700 1 -- 172.21.15.196:0/2532128866 learned_addr learned my addr 172.21.15.196:0/2532128866 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-02-23T00:37:22.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.641+0000 7f7d1bfff700 1 --2- 172.21.15.196:0/2532128866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d1c0f6990 0x7f7d1c0fa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:37:22.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.641+0000 7f7d2089e700 1 --2- 172.21.15.196:0/2532128866 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7d1c0f8580 0x7f7d1c0f9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:37:22.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.641+0000 7f7d1b7fe700 1 -- 172.21.15.196:0/2532128866 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7d1c0f8580 msgr2=0x7f7d1c0f9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:37:22.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.641+0000 7f7d1b7fe700 1 --2- 172.21.15.196:0/2532128866 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7d1c0f8580 0x7f7d1c0f9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:37:22.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.641+0000 7f7d1b7fe700 1 -- 172.21.15.196:0/2532128866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d1c0f6990 msgr2=0x7f7d1c0fa910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:37:22.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.641+0000 7f7d1b7fe700 1 --2- 172.21.15.196:0/2532128866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d1c0f6990 0x7f7d1c0fa910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:37:22.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.641+0000 7f7d1b7fe700 1 -- 172.21.15.196:0/2532128866 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d0c005040 con 0x7f7d1c0f7c30 2024-02-23T00:37:22.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.641+0000 7f7d1b7fe700 1 --2- 172.21.15.196:0/2532128866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d1c0f7c30 0x7f7d1c0f8f70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f7d1000b560 tx=0x7f7d100090f0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:37:22.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.641+0000 7f7d197fa700 10 client.?.objecter ms_handle_connect 0x7f7d1c0f7c30 2024-02-23T00:37:22.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.641+0000 7f7d197fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:37:22.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.641+0000 7f7d197fa700 1 -- 172.21.15.196:0/2532128866 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d10002320 con 0x7f7d1c0f7c30 2024-02-23T00:37:22.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.641+0000 7f7d197fa700 1 -- 172.21.15.196:0/2532128866 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f7d10002910 con 0x7f7d1c0f7c30 2024-02-23T00:37:22.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.641+0000 7f7d197fa700 1 -- 172.21.15.196:0/2532128866 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d100128e0 con 0x7f7d1c0f7c30 2024-02-23T00:37:22.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.641+0000 7f7d22301700 1 -- 172.21.15.196:0/2532128866 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d1c112150 con 0x7f7d1c0f7c30 2024-02-23T00:37:22.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.641+0000 7f7d22301700 10 client.24202.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:37:22.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.641+0000 7f7d22301700 1 -- 172.21.15.196:0/2532128866 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d1c1125f0 con 0x7f7d1c0f7c30 2024-02-23T00:37:22.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.641+0000 7f7d22301700 1 librados: init done 2024-02-23T00:37:22.650 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.642+0000 7f7d22301700 1 -- 172.21.15.196:0/2532128866 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d1c05f360 con 0x7f7d1c0f7c30 2024-02-23T00:37:22.650 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.642+0000 7f7d197fa700 1 -- 172.21.15.196:0/2532128866 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 14) v1 ==== 77064+0+0 (secure 0 0 0) 0x7f7d10002480 con 0x7f7d1c0f7c30 2024-02-23T00:37:22.650 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.643+0000 7f7d197fa700 1 --2- 172.21.15.196:0/2532128866 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f7d04062bd0 0x7f7d04065090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:37:22.650 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.643+0000 7f7d197fa700 1 -- 172.21.15.196:0/2532128866 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3422+0+0 (secure 0 0 0) 0x7f7d1007bf40 con 0x7f7d1c0f7c30 2024-02-23T00:37:22.650 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.643+0000 7f7d197fa700 10 client.24202.objecter ms_dispatch 0x7f7d1c001550 osd_map(27..27 src has 1..27) v4 2024-02-23T00:37:22.650 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.643+0000 7f7d197fa700 3 client.24202.objecter handle_osd_map got epochs [27,27] > 0 2024-02-23T00:37:22.650 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.643+0000 7f7d197fa700 3 client.24202.objecter handle_osd_map decoding full epoch 27 2024-02-23T00:37:22.650 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.643+0000 7f7d197fa700 20 client.24202.objecter dump_active .. 0 homeless 2024-02-23T00:37:22.650 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.643+0000 7f7d1bfff700 1 --2- 172.21.15.196:0/2532128866 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f7d04062bd0 0x7f7d04065090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:37:22.650 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.643+0000 7f7d197fa700 10 client.24202.objecter ms_handle_connect 0x7f7d04062bd0 2024-02-23T00:37:22.650 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.643+0000 7f7d1bfff700 1 --2- 172.21.15.196:0/2532128866 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f7d04062bd0 0x7f7d04065090 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f7d0c009a00 tx=0x7f7d0c009cb0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:37:22.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.647+0000 7f7d197fa700 1 -- 172.21.15.196:0/2532128866 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f7d1004cc40 con 0x7f7d1c0f7c30 2024-02-23T00:37:22.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:22.816+0000 7f7d22301700 1 -- 172.21.15.196:0/2532128866 --> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d1c061200 con 0x7f7d04062bd0 2024-02-23T00:37:23.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:22.099887+0000 mgr.y (mgr.14146) 141 : cluster [DBG] pgmap v122: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:23.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:22.820931+0000 mon.a (mon.0) 379 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-23T00:37:23.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:22.823404+0000 mon.a (mon.0) 380 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-23T00:37:23.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:22.824192+0000 mon.a (mon.0) 381 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:37:23.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:22.099887+0000 mgr.y (mgr.14146) 141 : cluster [DBG] pgmap v122: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:23.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:22.820931+0000 mon.a (mon.0) 379 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-23T00:37:23.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:22.823404+0000 mon.a (mon.0) 380 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-23T00:37:23.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:22.824192+0000 mon.a (mon.0) 381 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:37:23.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:22.099887+0000 mgr.y (mgr.14146) 141 : cluster [DBG] pgmap v122: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:23.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:22.820931+0000 mon.a (mon.0) 379 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-23T00:37:23.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:22.823404+0000 mon.a (mon.0) 380 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-23T00:37:23.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:22.824192+0000 mon.a (mon.0) 381 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:37:24.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:22.818381+0000 mgr.y (mgr.14146) 142 : audit [DBG] from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:37:24.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:22.818381+0000 mgr.y (mgr.14146) 142 : audit [DBG] from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:37:24.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:22.818381+0000 mgr.y (mgr.14146) 142 : audit [DBG] from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:37:25.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:24.100247+0000 mgr.y (mgr.14146) 143 : cluster [DBG] pgmap v123: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:25.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:24.100247+0000 mgr.y (mgr.14146) 143 : cluster [DBG] pgmap v123: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:25.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:24.100247+0000 mgr.y (mgr.14146) 143 : cluster [DBG] pgmap v123: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:26.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:25.436099+0000 mon.b (mon.1) 8 : audit [INF] from='client.? 172.21.15.196:0/1569909915' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d17966d7-eb12-4389-ac90-8d15efe290e4"}]: dispatch 2024-02-23T00:37:26.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:25.436567+0000 mon.a (mon.0) 382 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d17966d7-eb12-4389-ac90-8d15efe290e4"}]: dispatch 2024-02-23T00:37:26.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:25.441222+0000 mon.a (mon.0) 383 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d17966d7-eb12-4389-ac90-8d15efe290e4"}]': finished 2024-02-23T00:37:26.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:25.441306+0000 mon.a (mon.0) 384 : cluster [DBG] osdmap e28: 5 total, 4 up, 5 in 2024-02-23T00:37:26.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:25.441464+0000 mon.a (mon.0) 385 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-23T00:37:26.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:25.436099+0000 mon.b (mon.1) 8 : audit [INF] from='client.? 172.21.15.196:0/1569909915' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d17966d7-eb12-4389-ac90-8d15efe290e4"}]: dispatch 2024-02-23T00:37:26.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:25.436567+0000 mon.a (mon.0) 382 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d17966d7-eb12-4389-ac90-8d15efe290e4"}]: dispatch 2024-02-23T00:37:26.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:25.441222+0000 mon.a (mon.0) 383 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d17966d7-eb12-4389-ac90-8d15efe290e4"}]': finished 2024-02-23T00:37:26.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:25.441306+0000 mon.a (mon.0) 384 : cluster [DBG] osdmap e28: 5 total, 4 up, 5 in 2024-02-23T00:37:26.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:25.441464+0000 mon.a (mon.0) 385 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-23T00:37:26.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:25.436099+0000 mon.b (mon.1) 8 : audit [INF] from='client.? 172.21.15.196:0/1569909915' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d17966d7-eb12-4389-ac90-8d15efe290e4"}]: dispatch 2024-02-23T00:37:26.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:25.436567+0000 mon.a (mon.0) 382 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d17966d7-eb12-4389-ac90-8d15efe290e4"}]: dispatch 2024-02-23T00:37:26.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:25.441222+0000 mon.a (mon.0) 383 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d17966d7-eb12-4389-ac90-8d15efe290e4"}]': finished 2024-02-23T00:37:26.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:25.441306+0000 mon.a (mon.0) 384 : cluster [DBG] osdmap e28: 5 total, 4 up, 5 in 2024-02-23T00:37:26.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:25.441464+0000 mon.a (mon.0) 385 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-23T00:37:27.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:26.058378+0000 mon.b (mon.1) 9 : audit [DBG] from='client.? 172.21.15.196:0/1015346089' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-23T00:37:27.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:26.100701+0000 mgr.y (mgr.14146) 144 : cluster [DBG] pgmap v125: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:27.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:26.058378+0000 mon.b (mon.1) 9 : audit [DBG] from='client.? 172.21.15.196:0/1015346089' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-23T00:37:27.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:26.100701+0000 mgr.y (mgr.14146) 144 : cluster [DBG] pgmap v125: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:27.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:26.058378+0000 mon.b (mon.1) 9 : audit [DBG] from='client.? 172.21.15.196:0/1015346089' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-23T00:37:27.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:26.100701+0000 mgr.y (mgr.14146) 144 : cluster [DBG] pgmap v125: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:27.644 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:27.642+0000 7f7d21b00700 10 client.24202.objecter tick 2024-02-23T00:37:29.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:28.101139+0000 mgr.y (mgr.14146) 145 : cluster [DBG] pgmap v126: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:29.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:28.101139+0000 mgr.y (mgr.14146) 145 : cluster [DBG] pgmap v126: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:29.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:28.101139+0000 mgr.y (mgr.14146) 145 : cluster [DBG] pgmap v126: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:31.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:30.101485+0000 mgr.y (mgr.14146) 146 : cluster [DBG] pgmap v127: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:31.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:30.101485+0000 mgr.y (mgr.14146) 146 : cluster [DBG] pgmap v127: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:31.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:30.101485+0000 mgr.y (mgr.14146) 146 : cluster [DBG] pgmap v127: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:32.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:32.071846+0000 mon.a (mon.0) 386 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-23T00:37:32.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:32.072743+0000 mon.a (mon.0) 387 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:37:32.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:32.071846+0000 mon.a (mon.0) 386 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-23T00:37:32.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:32.072743+0000 mon.a (mon.0) 387 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:37:32.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:32.071846+0000 mon.a (mon.0) 386 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-23T00:37:32.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:32.072743+0000 mon.a (mon.0) 387 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:37:32.644 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:32.643+0000 7f7d21b00700 10 client.24202.objecter tick 2024-02-23T00:37:33.429 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:37:32.073528+0000 mgr.y (mgr.14146) 147 : cephadm [INF] Deploying daemon osd.4 on smithi196 2024-02-23T00:37:33.429 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:32.101907+0000 mgr.y (mgr.14146) 148 : cluster [DBG] pgmap v128: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:33.429 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:32.320814+0000 osd.1 (osd.1) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:37:33.429 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:32.320944+0000 osd.1 (osd.1) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:37:33.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:37:32.073528+0000 mgr.y (mgr.14146) 147 : cephadm [INF] Deploying daemon osd.4 on smithi196 2024-02-23T00:37:33.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:32.101907+0000 mgr.y (mgr.14146) 148 : cluster [DBG] pgmap v128: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:33.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:32.320814+0000 osd.1 (osd.1) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:37:33.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:32.320944+0000 osd.1 (osd.1) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:37:33.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:37:32.073528+0000 mgr.y (mgr.14146) 147 : cephadm [INF] Deploying daemon osd.4 on smithi196 2024-02-23T00:37:33.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:32.101907+0000 mgr.y (mgr.14146) 148 : cluster [DBG] pgmap v128: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:33.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:32.320814+0000 osd.1 (osd.1) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:37:33.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:32.320944+0000 osd.1 (osd.1) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:37:34.426 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:33.169463+0000 mon.a (mon.0) 388 : cluster [DBG] osdmap e29: 5 total, 4 up, 5 in 2024-02-23T00:37:34.426 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:33.176258+0000 mon.a (mon.0) 389 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-23T00:37:34.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:33.169463+0000 mon.a (mon.0) 388 : cluster [DBG] osdmap e29: 5 total, 4 up, 5 in 2024-02-23T00:37:34.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:33.176258+0000 mon.a (mon.0) 389 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-23T00:37:34.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:33.169463+0000 mon.a (mon.0) 388 : cluster [DBG] osdmap e29: 5 total, 4 up, 5 in 2024-02-23T00:37:34.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:33.176258+0000 mon.a (mon.0) 389 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-23T00:37:35.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:34.102376+0000 mgr.y (mgr.14146) 149 : cluster [DBG] pgmap v130: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:35.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:34.102376+0000 mgr.y (mgr.14146) 149 : cluster [DBG] pgmap v130: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:35.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:34.102376+0000 mgr.y (mgr.14146) 149 : cluster [DBG] pgmap v130: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:37.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:36.102845+0000 mgr.y (mgr.14146) 150 : cluster [DBG] pgmap v131: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:37.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:36.102845+0000 mgr.y (mgr.14146) 150 : cluster [DBG] pgmap v131: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:37.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:36.102845+0000 mgr.y (mgr.14146) 150 : cluster [DBG] pgmap v131: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:37.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:37.644+0000 7f7d21b00700 10 client.24202.objecter tick 2024-02-23T00:37:39.056 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:37.773129+0000 mon.a (mon.0) 390 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:39.056 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:37.804704+0000 mon.a (mon.0) 391 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:37:39.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:38.103198+0000 mgr.y (mgr.14146) 151 : cluster [DBG] pgmap v132: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:39.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:37.773129+0000 mon.a (mon.0) 390 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:39.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:37.804704+0000 mon.a (mon.0) 391 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:37:39.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:38.103198+0000 mgr.y (mgr.14146) 151 : cluster [DBG] pgmap v132: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:39.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:37.773129+0000 mon.a (mon.0) 390 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:39.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:37.804704+0000 mon.a (mon.0) 391 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:37:39.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:38.103198+0000 mgr.y (mgr.14146) 151 : cluster [DBG] pgmap v132: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:41.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:40.103526+0000 mgr.y (mgr.14146) 152 : cluster [DBG] pgmap v133: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:41.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:40.312113+0000 mon.a (mon.0) 392 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:41.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:40.312956+0000 mon.a (mon.0) 393 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:37:41.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:40.313828+0000 mon.a (mon.0) 394 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:37:41.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:40.318568+0000 mon.a (mon.0) 395 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:41.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:40.843694+0000 mon.b (mon.1) 10 : audit [INF] from='osd.4 [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-23T00:37:41.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:40.843964+0000 mon.a (mon.0) 396 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-23T00:37:41.484 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 4 on host 'smithi196' 2024-02-23T00:37:41.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:41.482+0000 7f7d197fa700 1 -- 172.21.15.196:0/2532128866 <== mgr.14146 v2:172.21.15.47:6800/622994871 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7d1c061200 con 0x7f7d04062bd0 2024-02-23T00:37:41.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:41.484+0000 7f7d22301700 10 librados: watch_flush enter 2024-02-23T00:37:41.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:41.484+0000 7f7d22301700 10 librados: watch_flush exit 2024-02-23T00:37:41.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:41.484+0000 7f7d22301700 20 client.24202.objecter shutdown clearing up homeless session... 2024-02-23T00:37:41.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:41.484+0000 7f7d22301700 10 client.24202.objecter successfully canceled tick 2024-02-23T00:37:41.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:41.484+0000 7f7d22301700 1 -- 172.21.15.196:0/2532128866 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f7d04062bd0 msgr2=0x7f7d04065090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:37:41.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:41.484+0000 7f7d22301700 1 --2- 172.21.15.196:0/2532128866 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f7d04062bd0 0x7f7d04065090 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f7d0c009a00 tx=0x7f7d0c009cb0).stop 2024-02-23T00:37:41.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:41.484+0000 7f7d22301700 1 -- 172.21.15.196:0/2532128866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d1c0f7c30 msgr2=0x7f7d1c0f8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:37:41.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:41.484+0000 7f7d22301700 1 --2- 172.21.15.196:0/2532128866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d1c0f7c30 0x7f7d1c0f8f70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f7d1000b560 tx=0x7f7d100090f0).stop 2024-02-23T00:37:41.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:41.484+0000 7f7d22301700 1 -- 172.21.15.196:0/2532128866 shutdown_connections 2024-02-23T00:37:41.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:41.484+0000 7f7d22301700 1 --2- 172.21.15.196:0/2532128866 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7d1c0f8580 0x7f7d1c0f9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:37:41.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:41.484+0000 7f7d22301700 1 --2- 172.21.15.196:0/2532128866 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f7d04062bd0 0x7f7d04065090 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:37:41.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:41.484+0000 7f7d22301700 1 --2- 172.21.15.196:0/2532128866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d1c0f6990 0x7f7d1c0fa910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:37:41.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:41.484+0000 7f7d22301700 1 --2- 172.21.15.196:0/2532128866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d1c0f7c30 0x7f7d1c0f8f70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:37:41.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:41.484+0000 7f7d22301700 1 -- 172.21.15.196:0/2532128866 >> 172.21.15.196:0/2532128866 conn(0x7f7d1c0f2190 msgr2=0x7f7d1c0f45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:37:41.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:41.484+0000 7f7d22301700 1 -- 172.21.15.196:0/2532128866 shutdown_connections 2024-02-23T00:37:41.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:41.484+0000 7f7d22301700 1 -- 172.21.15.196:0/2532128866 wait complete. 2024-02-23T00:37:41.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:41.484+0000 7f7d22301700 1 librados: shutdown 2024-02-23T00:37:41.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:40.103526+0000 mgr.y (mgr.14146) 152 : cluster [DBG] pgmap v133: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:41.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:40.312113+0000 mon.a (mon.0) 392 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:41.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:40.312956+0000 mon.a (mon.0) 393 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:37:41.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:40.313828+0000 mon.a (mon.0) 394 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:37:41.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:40.318568+0000 mon.a (mon.0) 395 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:41.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:40.843694+0000 mon.b (mon.1) 10 : audit [INF] from='osd.4 [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-23T00:37:41.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:40.843964+0000 mon.a (mon.0) 396 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-23T00:37:41.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:40.103526+0000 mgr.y (mgr.14146) 152 : cluster [DBG] pgmap v133: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:41.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:40.312113+0000 mon.a (mon.0) 392 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:41.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:40.312956+0000 mon.a (mon.0) 393 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:37:41.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:40.313828+0000 mon.a (mon.0) 394 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:37:41.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:40.318568+0000 mon.a (mon.0) 395 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:41.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:40.843694+0000 mon.b (mon.1) 10 : audit [INF] from='osd.4 [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-23T00:37:41.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:40.843964+0000 mon.a (mon.0) 396 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-23T00:37:42.337 DEBUG:teuthology.orchestra.run.smithi196:osd.4> sudo journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@osd.4.service 2024-02-23T00:37:42.339 INFO:tasks.cephadm:Deploying osd.5 on smithi196 with /dev/vg_nvme/lv_3... 2024-02-23T00:37:42.340 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-23T00:37:42.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:-- Logs begin at Fri 2024-02-23 00:13:21 UTC. -- 2024-02-23T00:37:42.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:41.320063+0000 mon.a (mon.0) 397 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-23T00:37:42.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:41.320147+0000 mon.a (mon.0) 398 : cluster [DBG] osdmap e30: 5 total, 4 up, 5 in 2024-02-23T00:37:42.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:41.320293+0000 mon.a (mon.0) 399 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-23T00:37:42.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:41.320796+0000 mon.b (mon.1) 11 : audit [INF] from='osd.4 [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-02-23T00:37:42.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:41.321072+0000 mon.a (mon.0) 400 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-02-23T00:37:42.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:41.482420+0000 mon.a (mon.0) 401 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:42.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:41.488066+0000 mon.a (mon.0) 402 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:37:42.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:41.320063+0000 mon.a (mon.0) 397 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-23T00:37:42.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:41.320147+0000 mon.a (mon.0) 398 : cluster [DBG] osdmap e30: 5 total, 4 up, 5 in 2024-02-23T00:37:42.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:41.320293+0000 mon.a (mon.0) 399 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-23T00:37:42.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:41.320796+0000 mon.b (mon.1) 11 : audit [INF] from='osd.4 [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-02-23T00:37:42.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:41.321072+0000 mon.a (mon.0) 400 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-02-23T00:37:42.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:41.482420+0000 mon.a (mon.0) 401 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:42.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:41.488066+0000 mon.a (mon.0) 402 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:37:42.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:41.320063+0000 mon.a (mon.0) 397 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-23T00:37:42.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:41.320147+0000 mon.a (mon.0) 398 : cluster [DBG] osdmap e30: 5 total, 4 up, 5 in 2024-02-23T00:37:42.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:41.320293+0000 mon.a (mon.0) 399 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-23T00:37:42.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:41.320796+0000 mon.b (mon.1) 11 : audit [INF] from='osd.4 [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-02-23T00:37:42.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:41.321072+0000 mon.a (mon.0) 400 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-02-23T00:37:42.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:41.482420+0000 mon.a (mon.0) 401 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:42.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:41.488066+0000 mon.a (mon.0) 402 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:37:43.578 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:42.103979+0000 mgr.y (mgr.14146) 153 : cluster [DBG] pgmap v135: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:43.578 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:42.322074+0000 mon.a (mon.0) 403 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-02-23T00:37:43.578 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:42.322278+0000 mon.a (mon.0) 404 : cluster [DBG] osdmap e31: 5 total, 4 up, 5 in 2024-02-23T00:37:43.579 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:42.322799+0000 mon.a (mon.0) 405 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-23T00:37:43.579 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:42.328003+0000 mon.a (mon.0) 406 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-23T00:37:43.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:42.103979+0000 mgr.y (mgr.14146) 153 : cluster [DBG] pgmap v135: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:43.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:42.322074+0000 mon.a (mon.0) 403 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-02-23T00:37:43.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:42.322278+0000 mon.a (mon.0) 404 : cluster [DBG] osdmap e31: 5 total, 4 up, 5 in 2024-02-23T00:37:43.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:42.322799+0000 mon.a (mon.0) 405 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-23T00:37:43.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:42.328003+0000 mon.a (mon.0) 406 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-23T00:37:43.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:42.103979+0000 mgr.y (mgr.14146) 153 : cluster [DBG] pgmap v135: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-23T00:37:43.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:42.322074+0000 mon.a (mon.0) 403 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-02-23T00:37:43.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:42.322278+0000 mon.a (mon.0) 404 : cluster [DBG] osdmap e31: 5 total, 4 up, 5 in 2024-02-23T00:37:43.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:42.322799+0000 mon.a (mon.0) 405 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-23T00:37:43.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:42.328003+0000 mon.a (mon.0) 406 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-23T00:37:44.689 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:41.836409+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:37:44.689 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:41.836506+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:37:44.689 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:43.323898+0000 mon.a (mon.0) 407 : cluster [INF] osd.4 [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] boot 2024-02-23T00:37:44.689 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:43.323955+0000 mon.a (mon.0) 408 : cluster [DBG] osdmap e32: 5 total, 5 up, 5 in 2024-02-23T00:37:44.689 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:43.324142+0000 mon.a (mon.0) 409 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-23T00:37:44.689 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:43.931700+0000 mon.a (mon.0) 410 : cluster [DBG] osdmap e33: 5 total, 5 up, 5 in 2024-02-23T00:37:44.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:41.836409+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:37:44.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:41.836506+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:37:44.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:43.323898+0000 mon.a (mon.0) 407 : cluster [INF] osd.4 [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] boot 2024-02-23T00:37:44.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:43.323955+0000 mon.a (mon.0) 408 : cluster [DBG] osdmap e32: 5 total, 5 up, 5 in 2024-02-23T00:37:44.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:43.324142+0000 mon.a (mon.0) 409 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-23T00:37:44.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:43.931700+0000 mon.a (mon.0) 410 : cluster [DBG] osdmap e33: 5 total, 5 up, 5 in 2024-02-23T00:37:44.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:41.836409+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:37:44.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:41.836506+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:37:44.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:43.323898+0000 mon.a (mon.0) 407 : cluster [INF] osd.4 [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] boot 2024-02-23T00:37:44.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:43.323955+0000 mon.a (mon.0) 408 : cluster [DBG] osdmap e32: 5 total, 5 up, 5 in 2024-02-23T00:37:44.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:43.324142+0000 mon.a (mon.0) 409 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-23T00:37:44.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:43.931700+0000 mon.a (mon.0) 410 : cluster [DBG] osdmap e33: 5 total, 5 up, 5 in 2024-02-23T00:37:45.563 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:44.104291+0000 mgr.y (mgr.14146) 154 : cluster [DBG] pgmap v139: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:37:45.563 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:44.940173+0000 mon.a (mon.0) 411 : cluster [DBG] osdmap e34: 5 total, 5 up, 5 in 2024-02-23T00:37:45.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:44.104291+0000 mgr.y (mgr.14146) 154 : cluster [DBG] pgmap v139: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:37:45.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:44.940173+0000 mon.a (mon.0) 411 : cluster [DBG] osdmap e34: 5 total, 5 up, 5 in 2024-02-23T00:37:45.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:44.104291+0000 mgr.y (mgr.14146) 154 : cluster [DBG] pgmap v139: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:37:45.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:44.940173+0000 mon.a (mon.0) 411 : cluster [DBG] osdmap e34: 5 total, 5 up, 5 in 2024-02-23T00:37:46.149 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-23T00:37:46.149 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-23T00:37:46.149 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-02-23T00:37:46.149 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-02-23T00:37:46.149 INFO:teuthology.orchestra.run.smithi196.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113092 s, 927 MB/s 2024-02-23T00:37:46.150 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-02-23T00:37:47.030 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph orch daemon add osd smithi196:vg_nvme/lv_3 2024-02-23T00:37:47.654 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:46.104614+0000 mgr.y (mgr.14146) 155 : cluster [DBG] pgmap v141: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:37:47.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:46.104614+0000 mgr.y (mgr.14146) 155 : cluster [DBG] pgmap v141: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:37:47.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:46.104614+0000 mgr.y (mgr.14146) 155 : cluster [DBG] pgmap v141: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:37:49.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.535+0000 7fd2e8930700 1 -- 172.21.15.196:0/3782054898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2e40f6990 msgr2=0x7fd2e40f6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:37:49.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.535+0000 7fd2e8930700 1 --2- 172.21.15.196:0/3782054898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2e40f6990 0x7fd2e40f6db0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fd2d40048a0 tx=0x7fd2d4009cb0).stop 2024-02-23T00:37:49.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.536+0000 7fd2e8930700 1 -- 172.21.15.196:0/3782054898 shutdown_connections 2024-02-23T00:37:49.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.536+0000 7fd2e8930700 1 --2- 172.21.15.196:0/3782054898 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2e40f8580 0x7fd2e40fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:37:49.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.536+0000 7fd2e8930700 1 --2- 172.21.15.196:0/3782054898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2e40f6990 0x7fd2e40f6db0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:37:49.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.536+0000 7fd2e8930700 1 --2- 172.21.15.196:0/3782054898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2e40f7c30 0x7fd2e40f8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:37:49.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.536+0000 7fd2e8930700 1 -- 172.21.15.196:0/3782054898 >> 172.21.15.196:0/3782054898 conn(0x7fd2e40f2190 msgr2=0x7fd2e40f45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:37:49.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.536+0000 7fd2e8930700 1 -- 172.21.15.196:0/3782054898 shutdown_connections 2024-02-23T00:37:49.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.536+0000 7fd2e8930700 1 -- 172.21.15.196:0/3782054898 wait complete. 2024-02-23T00:37:49.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.536+0000 7fd2e8930700 1 librados: starting msgr at 2024-02-23T00:37:49.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.536+0000 7fd2e8930700 1 librados: starting objecter 2024-02-23T00:37:49.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.536+0000 7fd2e8930700 1 Processor -- start 2024-02-23T00:37:49.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.536+0000 7fd2e8930700 1 -- start start 2024-02-23T00:37:49.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.536+0000 7fd2e8930700 1 librados: setting wanted keys 2024-02-23T00:37:49.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.536+0000 7fd2e8930700 1 librados: calling monclient init 2024-02-23T00:37:49.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.536+0000 7fd2e8930700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2e40f6990 0x7fd2e40614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:37:49.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.536+0000 7fd2e8930700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2e40f7c30 0x7fd2e410cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:37:49.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.537+0000 7fd2e8930700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2e40f8580 0x7fd2e410f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:37:49.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.537+0000 7fd2e8930700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e410f860 con 0x7fd2e40f6990 2024-02-23T00:37:49.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.537+0000 7fd2e8930700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e410f9d0 con 0x7fd2e40f7c30 2024-02-23T00:37:49.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.537+0000 7fd2e8930700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e410fb90 con 0x7fd2e40f8580 2024-02-23T00:37:49.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.537+0000 7fd2e1d9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2e40f7c30 0x7fd2e410cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:37:49.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.537+0000 7fd2e1d9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2e40f7c30 0x7fd2e410cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:47848/0 (socket says 172.21.15.196:47848) 2024-02-23T00:37:49.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.537+0000 7fd2e1d9b700 1 -- 172.21.15.196:0/1541358440 learned_addr learned my addr 172.21.15.196:0/1541358440 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-02-23T00:37:49.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.537+0000 7fd2e259c700 1 --2- 172.21.15.196:0/1541358440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2e40f6990 0x7fd2e40614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:37:49.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.537+0000 7fd2e2d9d700 1 --2- 172.21.15.196:0/1541358440 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2e40f8580 0x7fd2e410f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:37:49.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.537+0000 7fd2e1d9b700 1 -- 172.21.15.196:0/1541358440 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2e40f8580 msgr2=0x7fd2e410f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:37:49.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.537+0000 7fd2e1d9b700 1 --2- 172.21.15.196:0/1541358440 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2e40f8580 0x7fd2e410f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:37:49.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.537+0000 7fd2e1d9b700 1 -- 172.21.15.196:0/1541358440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2e40f6990 msgr2=0x7fd2e40614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:37:49.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.537+0000 7fd2e1d9b700 1 --2- 172.21.15.196:0/1541358440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2e40f6990 0x7fd2e40614f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:37:49.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.537+0000 7fd2e1d9b700 1 -- 172.21.15.196:0/1541358440 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d4005040 con 0x7fd2e40f7c30 2024-02-23T00:37:49.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.537+0000 7fd2e259c700 1 --2- 172.21.15.196:0/1541358440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2e40f6990 0x7fd2e40614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-23T00:37:49.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.537+0000 7fd2e1d9b700 1 --2- 172.21.15.196:0/1541358440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2e40f7c30 0x7fd2e410cc00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fd2d80041d0 tx=0x7fd2d8009f60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:37:49.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.537+0000 7fd2d37fe700 10 client.?.objecter ms_handle_connect 0x7fd2e40f7c30 2024-02-23T00:37:49.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.537+0000 7fd2d37fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:37:49.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.537+0000 7fd2d37fe700 1 -- 172.21.15.196:0/1541358440 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8007b40 con 0x7fd2e40f7c30 2024-02-23T00:37:49.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.537+0000 7fd2d37fe700 1 -- 172.21.15.196:0/1541358440 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fd2d800f040 con 0x7fd2e40f7c30 2024-02-23T00:37:49.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.537+0000 7fd2e8930700 1 -- 172.21.15.196:0/1541358440 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2e410fe70 con 0x7fd2e40f7c30 2024-02-23T00:37:49.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.537+0000 7fd2d37fe700 1 -- 172.21.15.196:0/1541358440 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d800d850 con 0x7fd2e40f7c30 2024-02-23T00:37:49.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.537+0000 7fd2e8930700 10 client.24229.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:37:49.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.537+0000 7fd2e8930700 1 -- 172.21.15.196:0/1541358440 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e4110350 con 0x7fd2e40f7c30 2024-02-23T00:37:49.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.537+0000 7fd2e8930700 1 librados: init done 2024-02-23T00:37:49.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.538+0000 7fd2d37fe700 1 -- 172.21.15.196:0/1541358440 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 14) v1 ==== 77064+0+0 (secure 0 0 0) 0x7fd2d800dc80 con 0x7fd2e40f7c30 2024-02-23T00:37:49.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.538+0000 7fd2e8930700 1 -- 172.21.15.196:0/1541358440 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2e405f360 con 0x7fd2e40f7c30 2024-02-23T00:37:49.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.539+0000 7fd2d37fe700 1 --2- 172.21.15.196:0/1541358440 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fd2cc062bd0 0x7fd2cc065090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:37:49.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.539+0000 7fd2d37fe700 1 -- 172.21.15.196:0/1541358440 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 3964+0+0 (secure 0 0 0) 0x7fd2d807c400 con 0x7fd2e40f7c30 2024-02-23T00:37:49.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.539+0000 7fd2d37fe700 10 client.24229.objecter ms_dispatch 0x7fd2e4001550 osd_map(34..34 src has 1..34) v4 2024-02-23T00:37:49.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.539+0000 7fd2d37fe700 3 client.24229.objecter handle_osd_map got epochs [34,34] > 0 2024-02-23T00:37:49.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.539+0000 7fd2d37fe700 3 client.24229.objecter handle_osd_map decoding full epoch 34 2024-02-23T00:37:49.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.539+0000 7fd2d37fe700 20 client.24229.objecter dump_active .. 0 homeless 2024-02-23T00:37:49.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.539+0000 7fd2e259c700 1 --2- 172.21.15.196:0/1541358440 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fd2cc062bd0 0x7fd2cc065090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:37:49.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.539+0000 7fd2d37fe700 10 client.24229.objecter ms_handle_connect 0x7fd2cc062bd0 2024-02-23T00:37:49.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.539+0000 7fd2e259c700 1 --2- 172.21.15.196:0/1541358440 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fd2cc062bd0 0x7fd2cc065090 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fd2d40045a0 tx=0x7fd2d4009cb0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:37:49.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.543+0000 7fd2d37fe700 1 -- 172.21.15.196:0/1541358440 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fd2d804cf70 con 0x7fd2e40f7c30 2024-02-23T00:37:49.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:48.104991+0000 mgr.y (mgr.14146) 156 : cluster [DBG] pgmap v142: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:37:49.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:37:48.526593+0000 mgr.y (mgr.14146) 157 : cephadm [INF] Detected new or changed devices on smithi196 2024-02-23T00:37:49.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:48.531172+0000 mon.a (mon.0) 412 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:49.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:48.532229+0000 mon.a (mon.0) 413 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:37:49.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:37:48.532909+0000 mgr.y (mgr.14146) 158 : cephadm [INF] Adjusting osd_memory_target on smithi196 to 17122M 2024-02-23T00:37:49.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:48.536993+0000 mon.a (mon.0) 414 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:49.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:48.538037+0000 mon.a (mon.0) 415 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:37:49.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:48.538850+0000 mon.a (mon.0) 416 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:37:49.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:48.543356+0000 mon.a (mon.0) 417 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:49.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:49.714+0000 7fd2e8930700 1 -- 172.21.15.196:0/1541358440 --> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fd2e40fe540 con 0x7fd2cc062bd0 2024-02-23T00:37:49.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:48.104991+0000 mgr.y (mgr.14146) 156 : cluster [DBG] pgmap v142: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:37:49.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:37:48.526593+0000 mgr.y (mgr.14146) 157 : cephadm [INF] Detected new or changed devices on smithi196 2024-02-23T00:37:49.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:48.531172+0000 mon.a (mon.0) 412 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:49.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:48.532229+0000 mon.a (mon.0) 413 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:37:49.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:37:48.532909+0000 mgr.y (mgr.14146) 158 : cephadm [INF] Adjusting osd_memory_target on smithi196 to 17122M 2024-02-23T00:37:49.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:48.536993+0000 mon.a (mon.0) 414 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:49.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:48.538037+0000 mon.a (mon.0) 415 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:37:49.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:48.538850+0000 mon.a (mon.0) 416 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:37:49.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:48.543356+0000 mon.a (mon.0) 417 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:49.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:48.104991+0000 mgr.y (mgr.14146) 156 : cluster [DBG] pgmap v142: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:37:49.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:37:48.526593+0000 mgr.y (mgr.14146) 157 : cephadm [INF] Detected new or changed devices on smithi196 2024-02-23T00:37:49.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:48.531172+0000 mon.a (mon.0) 412 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:49.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:48.532229+0000 mon.a (mon.0) 413 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:37:49.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:37:48.532909+0000 mgr.y (mgr.14146) 158 : cephadm [INF] Adjusting osd_memory_target on smithi196 to 17122M 2024-02-23T00:37:49.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:48.536993+0000 mon.a (mon.0) 414 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:49.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:48.538037+0000 mon.a (mon.0) 415 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:37:49.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:48.538850+0000 mon.a (mon.0) 416 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:37:49.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:48.543356+0000 mon.a (mon.0) 417 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:37:50.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:49.718947+0000 mon.a (mon.0) 418 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-23T00:37:50.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:49.721266+0000 mon.a (mon.0) 419 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-23T00:37:50.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:49.721997+0000 mon.a (mon.0) 420 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:37:50.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:49.718947+0000 mon.a (mon.0) 418 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-23T00:37:50.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:49.721266+0000 mon.a (mon.0) 419 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-23T00:37:50.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:49.721997+0000 mon.a (mon.0) 420 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:37:50.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:49.718947+0000 mon.a (mon.0) 418 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-23T00:37:50.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:49.721266+0000 mon.a (mon.0) 419 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-23T00:37:50.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:49.721997+0000 mon.a (mon.0) 420 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:37:51.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:49.716573+0000 mgr.y (mgr.14146) 159 : audit [DBG] from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:37:51.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:50.105401+0000 mgr.y (mgr.14146) 160 : cluster [DBG] pgmap v143: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:37:51.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:49.716573+0000 mgr.y (mgr.14146) 159 : audit [DBG] from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:37:51.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:50.105401+0000 mgr.y (mgr.14146) 160 : cluster [DBG] pgmap v143: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:37:51.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:49.716573+0000 mgr.y (mgr.14146) 159 : audit [DBG] from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:37:51.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:50.105401+0000 mgr.y (mgr.14146) 160 : cluster [DBG] pgmap v143: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:37:52.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:52.501416+0000 mon.b (mon.1) 12 : audit [INF] from='client.? 172.21.15.196:0/1382619448' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b729d646-4899-47b3-b917-acceba46d241"}]: dispatch 2024-02-23T00:37:52.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:52.501821+0000 mon.a (mon.0) 421 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b729d646-4899-47b3-b917-acceba46d241"}]: dispatch 2024-02-23T00:37:52.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:52.506691+0000 mon.a (mon.0) 422 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b729d646-4899-47b3-b917-acceba46d241"}]': finished 2024-02-23T00:37:52.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:52.506755+0000 mon.a (mon.0) 423 : cluster [DBG] osdmap e35: 6 total, 5 up, 6 in 2024-02-23T00:37:52.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:52.506904+0000 mon.a (mon.0) 424 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-23T00:37:52.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:52.501416+0000 mon.b (mon.1) 12 : audit [INF] from='client.? 172.21.15.196:0/1382619448' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b729d646-4899-47b3-b917-acceba46d241"}]: dispatch 2024-02-23T00:37:52.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:52.501821+0000 mon.a (mon.0) 421 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b729d646-4899-47b3-b917-acceba46d241"}]: dispatch 2024-02-23T00:37:52.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:52.506691+0000 mon.a (mon.0) 422 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b729d646-4899-47b3-b917-acceba46d241"}]': finished 2024-02-23T00:37:52.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:52.506755+0000 mon.a (mon.0) 423 : cluster [DBG] osdmap e35: 6 total, 5 up, 6 in 2024-02-23T00:37:52.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:52.506904+0000 mon.a (mon.0) 424 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-23T00:37:52.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:52.501416+0000 mon.b (mon.1) 12 : audit [INF] from='client.? 172.21.15.196:0/1382619448' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b729d646-4899-47b3-b917-acceba46d241"}]: dispatch 2024-02-23T00:37:52.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:52.501821+0000 mon.a (mon.0) 421 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b729d646-4899-47b3-b917-acceba46d241"}]: dispatch 2024-02-23T00:37:52.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:52.506691+0000 mon.a (mon.0) 422 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b729d646-4899-47b3-b917-acceba46d241"}]': finished 2024-02-23T00:37:52.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:52.506755+0000 mon.a (mon.0) 423 : cluster [DBG] osdmap e35: 6 total, 5 up, 6 in 2024-02-23T00:37:52.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:52.506904+0000 mon.a (mon.0) 424 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-23T00:37:53.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:52.105858+0000 mgr.y (mgr.14146) 161 : cluster [DBG] pgmap v144: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:37:53.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:53.154820+0000 mon.b (mon.1) 13 : audit [DBG] from='client.? 172.21.15.196:0/1859281879' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-23T00:37:53.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:52.105858+0000 mgr.y (mgr.14146) 161 : cluster [DBG] pgmap v144: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:37:53.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:53.154820+0000 mon.b (mon.1) 13 : audit [DBG] from='client.? 172.21.15.196:0/1859281879' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-23T00:37:53.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:52.105858+0000 mgr.y (mgr.14146) 161 : cluster [DBG] pgmap v144: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:37:53.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:53.154820+0000 mon.b (mon.1) 13 : audit [DBG] from='client.? 172.21.15.196:0/1859281879' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-23T00:37:54.540 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:54.538+0000 7fd2e3fff700 10 client.24229.objecter tick 2024-02-23T00:37:55.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:54.106419+0000 mgr.y (mgr.14146) 162 : cluster [DBG] pgmap v146: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:37:55.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:54.106419+0000 mgr.y (mgr.14146) 162 : cluster [DBG] pgmap v146: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:37:55.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:54.106419+0000 mgr.y (mgr.14146) 162 : cluster [DBG] pgmap v146: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:37:57.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:56.106780+0000 mgr.y (mgr.14146) 163 : cluster [DBG] pgmap v147: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:37:57.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:56.106780+0000 mgr.y (mgr.14146) 163 : cluster [DBG] pgmap v147: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:37:57.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:56.106780+0000 mgr.y (mgr.14146) 163 : cluster [DBG] pgmap v147: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:37:59.276 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:37:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:37:58.107273+0000 mgr.y (mgr.14146) 164 : cluster [DBG] pgmap v148: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:37:59.541 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:37:59.539+0000 7fd2e3fff700 10 client.24229.objecter tick 2024-02-23T00:37:59.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:37:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:37:58.107273+0000 mgr.y (mgr.14146) 164 : cluster [DBG] pgmap v148: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:37:59.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:37:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:37:58.107273+0000 mgr.y (mgr.14146) 164 : cluster [DBG] pgmap v148: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:38:00.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:59.295095+0000 mon.a (mon.0) 425 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-23T00:38:00.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:37:59.295943+0000 mon.a (mon.0) 426 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:38:00.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:37:59.296681+0000 mgr.y (mgr.14146) 165 : cephadm [INF] Deploying daemon osd.5 on smithi196 2024-02-23T00:38:00.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:59.295095+0000 mon.a (mon.0) 425 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-23T00:38:00.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:37:59.295943+0000 mon.a (mon.0) 426 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:38:00.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:37:59.296681+0000 mgr.y (mgr.14146) 165 : cephadm [INF] Deploying daemon osd.5 on smithi196 2024-02-23T00:38:00.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:59.295095+0000 mon.a (mon.0) 425 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-23T00:38:00.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:37:59.295943+0000 mon.a (mon.0) 426 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:38:00.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:37:59.296681+0000 mgr.y (mgr.14146) 165 : cephadm [INF] Deploying daemon osd.5 on smithi196 2024-02-23T00:38:01.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:00.107563+0000 mgr.y (mgr.14146) 166 : cluster [DBG] pgmap v149: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:38:01.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:00.107563+0000 mgr.y (mgr.14146) 166 : cluster [DBG] pgmap v149: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:38:01.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:00.107563+0000 mgr.y (mgr.14146) 166 : cluster [DBG] pgmap v149: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:38:03.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:02.107966+0000 mgr.y (mgr.14146) 167 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:38:03.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:02.107966+0000 mgr.y (mgr.14146) 167 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:38:03.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:02.107966+0000 mgr.y (mgr.14146) 167 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:38:04.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:04.540+0000 7fd2e3fff700 10 client.24229.objecter tick 2024-02-23T00:38:05.399 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:04.108361+0000 mgr.y (mgr.14146) 168 : cluster [DBG] pgmap v151: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:38:05.399 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:04.221499+0000 osd.2 (osd.2) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:38:05.399 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:04.221596+0000 osd.2 (osd.2) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:38:05.400 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:05.022974+0000 mon.a (mon.0) 427 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:05.400 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:05.055371+0000 mon.a (mon.0) 428 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:38:05.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:04.108361+0000 mgr.y (mgr.14146) 168 : cluster [DBG] pgmap v151: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:38:05.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:04.221499+0000 osd.2 (osd.2) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:38:05.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:04.221596+0000 osd.2 (osd.2) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:38:05.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:05.022974+0000 mon.a (mon.0) 427 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:05.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:05.055371+0000 mon.a (mon.0) 428 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:38:05.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:04.108361+0000 mgr.y (mgr.14146) 168 : cluster [DBG] pgmap v151: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:38:05.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:04.221499+0000 osd.2 (osd.2) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:38:05.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:04.221596+0000 osd.2 (osd.2) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:38:05.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:05.022974+0000 mon.a (mon.0) 427 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:05.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:05.055371+0000 mon.a (mon.0) 428 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:38:06.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:05.177555+0000 mon.a (mon.0) 429 : cluster [DBG] osdmap e36: 6 total, 5 up, 6 in 2024-02-23T00:38:06.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:05.177976+0000 mon.a (mon.0) 430 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-23T00:38:06.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:05.177555+0000 mon.a (mon.0) 429 : cluster [DBG] osdmap e36: 6 total, 5 up, 6 in 2024-02-23T00:38:06.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:05.177976+0000 mon.a (mon.0) 430 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-23T00:38:06.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:05.177555+0000 mon.a (mon.0) 429 : cluster [DBG] osdmap e36: 6 total, 5 up, 6 in 2024-02-23T00:38:06.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:05.177976+0000 mon.a (mon.0) 430 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-23T00:38:07.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:06.108765+0000 mgr.y (mgr.14146) 169 : cluster [DBG] pgmap v153: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:38:07.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:06.610746+0000 osd.1 (osd.1) 5 : cluster [DBG] 1.0 scrub starts 2024-02-23T00:38:07.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:06.611811+0000 osd.1 (osd.1) 6 : cluster [DBG] 1.0 scrub ok 2024-02-23T00:38:07.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:06.108765+0000 mgr.y (mgr.14146) 169 : cluster [DBG] pgmap v153: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:38:07.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:06.610746+0000 osd.1 (osd.1) 5 : cluster [DBG] 1.0 scrub starts 2024-02-23T00:38:07.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:06.611811+0000 osd.1 (osd.1) 6 : cluster [DBG] 1.0 scrub ok 2024-02-23T00:38:07.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:06.108765+0000 mgr.y (mgr.14146) 169 : cluster [DBG] pgmap v153: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:38:07.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:06.610746+0000 osd.1 (osd.1) 5 : cluster [DBG] 1.0 scrub starts 2024-02-23T00:38:07.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:06.611811+0000 osd.1 (osd.1) 6 : cluster [DBG] 1.0 scrub ok 2024-02-23T00:38:08.587 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 5 on host 'smithi196' 2024-02-23T00:38:08.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:08.585+0000 7fd2d37fe700 1 -- 172.21.15.196:0/1541358440 <== mgr.14146 v2:172.21.15.47:6800/622994871 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd2e40fe540 con 0x7fd2cc062bd0 2024-02-23T00:38:08.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:08.587+0000 7fd2e8930700 10 librados: watch_flush enter 2024-02-23T00:38:08.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:08.587+0000 7fd2e8930700 10 librados: watch_flush exit 2024-02-23T00:38:08.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:08.587+0000 7fd2e8930700 20 client.24229.objecter shutdown clearing up homeless session... 2024-02-23T00:38:08.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:08.587+0000 7fd2e8930700 10 client.24229.objecter successfully canceled tick 2024-02-23T00:38:08.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:08.587+0000 7fd2e8930700 1 -- 172.21.15.196:0/1541358440 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fd2cc062bd0 msgr2=0x7fd2cc065090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:38:08.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:08.587+0000 7fd2e8930700 1 --2- 172.21.15.196:0/1541358440 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fd2cc062bd0 0x7fd2cc065090 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fd2d40045a0 tx=0x7fd2d4009cb0).stop 2024-02-23T00:38:08.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:08.587+0000 7fd2e8930700 1 -- 172.21.15.196:0/1541358440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2e40f7c30 msgr2=0x7fd2e410cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:38:08.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:08.587+0000 7fd2e8930700 1 --2- 172.21.15.196:0/1541358440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2e40f7c30 0x7fd2e410cc00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fd2d80041d0 tx=0x7fd2d8009f60).stop 2024-02-23T00:38:08.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:08.587+0000 7fd2e8930700 1 -- 172.21.15.196:0/1541358440 shutdown_connections 2024-02-23T00:38:08.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:08.587+0000 7fd2e8930700 1 --2- 172.21.15.196:0/1541358440 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2e40f8580 0x7fd2e410f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:38:08.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:08.587+0000 7fd2e8930700 1 --2- 172.21.15.196:0/1541358440 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fd2cc062bd0 0x7fd2cc065090 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:38:08.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:08.587+0000 7fd2e8930700 1 --2- 172.21.15.196:0/1541358440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2e40f6990 0x7fd2e40614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:38:08.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:08.587+0000 7fd2e8930700 1 --2- 172.21.15.196:0/1541358440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2e40f7c30 0x7fd2e410cc00 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:38:08.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:08.587+0000 7fd2e8930700 1 -- 172.21.15.196:0/1541358440 >> 172.21.15.196:0/1541358440 conn(0x7fd2e40f2190 msgr2=0x7fd2e40fb680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:38:08.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:08.587+0000 7fd2e8930700 1 -- 172.21.15.196:0/1541358440 shutdown_connections 2024-02-23T00:38:08.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:08.587+0000 7fd2e8930700 1 -- 172.21.15.196:0/1541358440 wait complete. 2024-02-23T00:38:08.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:08.587+0000 7fd2e8930700 1 librados: shutdown 2024-02-23T00:38:09.298 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:07.945484+0000 mon.a (mon.0) 431 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:09.298 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:07.946323+0000 mon.a (mon.0) 432 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:38:09.298 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:07.947135+0000 mon.a (mon.0) 433 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:38:09.298 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:07.951592+0000 mon.a (mon.0) 434 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:09.298 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:08.109234+0000 mgr.y (mgr.14146) 170 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:38:09.298 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:08.125525+0000 mon.b (mon.1) 14 : audit [INF] from='osd.5 [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-23T00:38:09.298 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:08.125894+0000 mon.a (mon.0) 435 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-23T00:38:09.299 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:08.585544+0000 mon.a (mon.0) 436 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:09.299 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:08.619582+0000 mon.a (mon.0) 437 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:38:09.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:07.945484+0000 mon.a (mon.0) 431 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:09.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:07.946323+0000 mon.a (mon.0) 432 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:38:09.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:07.947135+0000 mon.a (mon.0) 433 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:38:09.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:07.951592+0000 mon.a (mon.0) 434 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:09.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:08.109234+0000 mgr.y (mgr.14146) 170 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:38:09.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:08.125525+0000 mon.b (mon.1) 14 : audit [INF] from='osd.5 [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-23T00:38:09.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:08.125894+0000 mon.a (mon.0) 435 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-23T00:38:09.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:08.585544+0000 mon.a (mon.0) 436 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:09.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:08.619582+0000 mon.a (mon.0) 437 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:38:09.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:07.945484+0000 mon.a (mon.0) 431 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:09.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:07.946323+0000 mon.a (mon.0) 432 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:38:09.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:07.947135+0000 mon.a (mon.0) 433 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:38:09.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:07.951592+0000 mon.a (mon.0) 434 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:09.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:08.109234+0000 mgr.y (mgr.14146) 170 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:38:09.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:08.125525+0000 mon.b (mon.1) 14 : audit [INF] from='osd.5 [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-23T00:38:09.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:08.125894+0000 mon.a (mon.0) 435 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-23T00:38:09.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:08.585544+0000 mon.a (mon.0) 436 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:09.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:08.619582+0000 mon.a (mon.0) 437 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:38:09.324 DEBUG:teuthology.orchestra.run.smithi196:osd.5> sudo journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@osd.5.service 2024-02-23T00:38:09.326 INFO:tasks.cephadm:Deploying osd.6 on smithi196 with /dev/vg_nvme/lv_2... 2024-02-23T00:38:09.326 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-23T00:38:09.357 INFO:journalctl@ceph.osd.5.smithi196.stdout:-- Logs begin at Fri 2024-02-23 00:13:21 UTC. -- 2024-02-23T00:38:10.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:08.958305+0000 mon.a (mon.0) 438 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-23T00:38:10.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:08.958408+0000 mon.a (mon.0) 439 : cluster [DBG] osdmap e37: 6 total, 5 up, 6 in 2024-02-23T00:38:10.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:08.958555+0000 mon.a (mon.0) 440 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-23T00:38:10.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:08.960338+0000 mon.b (mon.1) 15 : audit [INF] from='osd.5 [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-02-23T00:38:10.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:08.960801+0000 mon.a (mon.0) 441 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-02-23T00:38:10.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:08.958305+0000 mon.a (mon.0) 438 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-23T00:38:10.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:08.958408+0000 mon.a (mon.0) 439 : cluster [DBG] osdmap e37: 6 total, 5 up, 6 in 2024-02-23T00:38:10.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:08.958555+0000 mon.a (mon.0) 440 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-23T00:38:10.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:08.960338+0000 mon.b (mon.1) 15 : audit [INF] from='osd.5 [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-02-23T00:38:10.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:08.960801+0000 mon.a (mon.0) 441 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-02-23T00:38:10.443 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:38:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5[48426]: debug 2024-02-23T00:38:09.965+0000 7f06dd770700 -1 osd.5 0 waiting for initial osdmap 2024-02-23T00:38:10.443 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:38:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5[48426]: debug 2024-02-23T00:38:09.969+0000 7f06d7ee8700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-23T00:38:10.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:08.958305+0000 mon.a (mon.0) 438 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-23T00:38:10.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:08.958408+0000 mon.a (mon.0) 439 : cluster [DBG] osdmap e37: 6 total, 5 up, 6 in 2024-02-23T00:38:10.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:08.958555+0000 mon.a (mon.0) 440 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-23T00:38:10.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:08.960338+0000 mon.b (mon.1) 15 : audit [INF] from='osd.5 [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-02-23T00:38:10.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:08.960801+0000 mon.a (mon.0) 441 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-02-23T00:38:11.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:09.960473+0000 mon.a (mon.0) 442 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-02-23T00:38:11.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:09.960582+0000 mon.a (mon.0) 443 : cluster [DBG] osdmap e38: 6 total, 5 up, 6 in 2024-02-23T00:38:11.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:09.961003+0000 mon.a (mon.0) 444 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-23T00:38:11.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:09.966284+0000 mon.a (mon.0) 445 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-23T00:38:11.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:10.109672+0000 mgr.y (mgr.14146) 171 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:38:11.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:09.960473+0000 mon.a (mon.0) 442 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-02-23T00:38:11.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:09.960582+0000 mon.a (mon.0) 443 : cluster [DBG] osdmap e38: 6 total, 5 up, 6 in 2024-02-23T00:38:11.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:09.961003+0000 mon.a (mon.0) 444 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-23T00:38:11.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:09.966284+0000 mon.a (mon.0) 445 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-23T00:38:11.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:10.109672+0000 mgr.y (mgr.14146) 171 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:38:11.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:09.960473+0000 mon.a (mon.0) 442 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-02-23T00:38:11.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:09.960582+0000 mon.a (mon.0) 443 : cluster [DBG] osdmap e38: 6 total, 5 up, 6 in 2024-02-23T00:38:11.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:09.961003+0000 mon.a (mon.0) 444 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-23T00:38:11.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:09.966284+0000 mon.a (mon.0) 445 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-23T00:38:11.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:10.109672+0000 mgr.y (mgr.14146) 171 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-23T00:38:12.065 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:09.097204+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:38:12.065 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:09.097323+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:38:12.065 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:10.967207+0000 mon.a (mon.0) 446 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-23T00:38:12.065 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:10.971915+0000 mon.a (mon.0) 447 : cluster [INF] osd.5 [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] boot 2024-02-23T00:38:12.065 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:10.971975+0000 mon.a (mon.0) 448 : cluster [DBG] osdmap e39: 6 total, 6 up, 6 in 2024-02-23T00:38:12.065 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:10.972337+0000 mon.a (mon.0) 449 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-23T00:38:12.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:09.097204+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:38:12.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:09.097323+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:38:12.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:10.967207+0000 mon.a (mon.0) 446 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-23T00:38:12.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:10.971915+0000 mon.a (mon.0) 447 : cluster [INF] osd.5 [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] boot 2024-02-23T00:38:12.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:10.971975+0000 mon.a (mon.0) 448 : cluster [DBG] osdmap e39: 6 total, 6 up, 6 in 2024-02-23T00:38:12.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:10.972337+0000 mon.a (mon.0) 449 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-23T00:38:12.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:09.097204+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:38:12.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:09.097323+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:38:12.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:10.967207+0000 mon.a (mon.0) 446 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-23T00:38:12.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:10.971915+0000 mon.a (mon.0) 447 : cluster [INF] osd.5 [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] boot 2024-02-23T00:38:12.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:10.971975+0000 mon.a (mon.0) 448 : cluster [DBG] osdmap e39: 6 total, 6 up, 6 in 2024-02-23T00:38:12.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:10.972337+0000 mon.a (mon.0) 449 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-23T00:38:12.712 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-23T00:38:12.713 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-23T00:38:12.713 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-02-23T00:38:12.713 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-02-23T00:38:12.713 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0151934 s, 690 MB/s 2024-02-23T00:38:12.713 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-02-23T00:38:13.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:11.972221+0000 mon.a (mon.0) 450 : cluster [DBG] osdmap e40: 6 total, 6 up, 6 in 2024-02-23T00:38:13.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:12.110116+0000 mgr.y (mgr.14146) 172 : cluster [DBG] pgmap v160: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:13.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:11.972221+0000 mon.a (mon.0) 450 : cluster [DBG] osdmap e40: 6 total, 6 up, 6 in 2024-02-23T00:38:13.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:12.110116+0000 mgr.y (mgr.14146) 172 : cluster [DBG] pgmap v160: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:13.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:11.972221+0000 mon.a (mon.0) 450 : cluster [DBG] osdmap e40: 6 total, 6 up, 6 in 2024-02-23T00:38:13.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:12.110116+0000 mgr.y (mgr.14146) 172 : cluster [DBG] pgmap v160: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:14.108 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph orch daemon add osd smithi196:vg_nvme/lv_2 2024-02-23T00:38:14.306 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:12.981750+0000 mon.a (mon.0) 451 : cluster [DBG] osdmap e41: 6 total, 6 up, 6 in 2024-02-23T00:38:14.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:12.981750+0000 mon.a (mon.0) 451 : cluster [DBG] osdmap e41: 6 total, 6 up, 6 in 2024-02-23T00:38:14.438 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:12.981750+0000 mon.a (mon.0) 451 : cluster [DBG] osdmap e41: 6 total, 6 up, 6 in 2024-02-23T00:38:15.174 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:14.110541+0000 mgr.y (mgr.14146) 173 : cluster [DBG] pgmap v162: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:15.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:14.110541+0000 mgr.y (mgr.14146) 173 : cluster [DBG] pgmap v162: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:15.306 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:14.110541+0000 mgr.y (mgr.14146) 173 : cluster [DBG] pgmap v162: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:16.191 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:14.780085+0000 osd.0 (osd.0) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:38:16.191 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:14.780211+0000 osd.0 (osd.0) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:38:16.191 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:14.988540+0000 mon.a (mon.0) 452 : cluster [DBG] osdmap e42: 6 total, 6 up, 6 in 2024-02-23T00:38:16.191 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:38:15.217435+0000 mgr.y (mgr.14146) 174 : cephadm [INF] Detected new or changed devices on smithi196 2024-02-23T00:38:16.192 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:15.222088+0000 mon.a (mon.0) 453 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:16.192 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:15.838852+0000 mon.a (mon.0) 454 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:38:16.192 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:15.839798+0000 mon.a (mon.0) 455 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:38:16.192 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:15.845094+0000 mon.a (mon.0) 456 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:16.192 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:15.846157+0000 mon.a (mon.0) 457 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:38:16.192 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:15.846964+0000 mon.a (mon.0) 458 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:38:16.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:15.851540+0000 mon.a (mon.0) 459 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:16.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:14.780085+0000 osd.0 (osd.0) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:38:16.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:14.780211+0000 osd.0 (osd.0) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:38:16.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:14.988540+0000 mon.a (mon.0) 452 : cluster [DBG] osdmap e42: 6 total, 6 up, 6 in 2024-02-23T00:38:16.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:38:15.217435+0000 mgr.y (mgr.14146) 174 : cephadm [INF] Detected new or changed devices on smithi196 2024-02-23T00:38:16.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:15.222088+0000 mon.a (mon.0) 453 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:16.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:15.838852+0000 mon.a (mon.0) 454 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:38:16.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:15.839798+0000 mon.a (mon.0) 455 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:38:16.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:15.845094+0000 mon.a (mon.0) 456 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:16.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:15.846157+0000 mon.a (mon.0) 457 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:38:16.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:15.846964+0000 mon.a (mon.0) 458 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:38:16.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:15.851540+0000 mon.a (mon.0) 459 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:16.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:14.780085+0000 osd.0 (osd.0) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:38:16.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:14.780211+0000 osd.0 (osd.0) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:38:16.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:14.988540+0000 mon.a (mon.0) 452 : cluster [DBG] osdmap e42: 6 total, 6 up, 6 in 2024-02-23T00:38:16.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:38:15.217435+0000 mgr.y (mgr.14146) 174 : cephadm [INF] Detected new or changed devices on smithi196 2024-02-23T00:38:16.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:15.222088+0000 mon.a (mon.0) 453 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:16.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:15.838852+0000 mon.a (mon.0) 454 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:38:16.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:15.839798+0000 mon.a (mon.0) 455 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:38:16.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:15.845094+0000 mon.a (mon.0) 456 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:16.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:15.846157+0000 mon.a (mon.0) 457 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:38:16.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:15.846964+0000 mon.a (mon.0) 458 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:38:16.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:15.851540+0000 mon.a (mon.0) 459 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:16.805 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.799+0000 7fb268f48700 1 -- 172.21.15.196:0/1712170310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2640f7c30 msgr2=0x7fb2640f8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:38:16.805 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.799+0000 7fb268f48700 1 --2- 172.21.15.196:0/1712170310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2640f7c30 0x7fb2640f8090 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fb24c0048a0 tx=0x7fb24c009cb0).stop 2024-02-23T00:38:16.805 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.799+0000 7fb268f48700 1 -- 172.21.15.196:0/1712170310 shutdown_connections 2024-02-23T00:38:16.805 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.799+0000 7fb268f48700 1 --2- 172.21.15.196:0/1712170310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2640f8580 0x7fb2640fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:38:16.805 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.799+0000 7fb268f48700 1 --2- 172.21.15.196:0/1712170310 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb2640f6990 0x7fb2640f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:38:16.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.799+0000 7fb268f48700 1 --2- 172.21.15.196:0/1712170310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2640f7c30 0x7fb2640f8090 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:38:16.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.799+0000 7fb268f48700 1 -- 172.21.15.196:0/1712170310 >> 172.21.15.196:0/1712170310 conn(0x7fb2640f2190 msgr2=0x7fb2640f45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:38:16.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.799+0000 7fb268f48700 1 -- 172.21.15.196:0/1712170310 shutdown_connections 2024-02-23T00:38:16.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.799+0000 7fb268f48700 1 -- 172.21.15.196:0/1712170310 wait complete. 2024-02-23T00:38:16.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.800+0000 7fb268f48700 1 librados: starting msgr at 2024-02-23T00:38:16.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.800+0000 7fb268f48700 1 librados: starting objecter 2024-02-23T00:38:16.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.800+0000 7fb268f48700 1 Processor -- start 2024-02-23T00:38:16.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.800+0000 7fb268f48700 1 -- start start 2024-02-23T00:38:16.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.800+0000 7fb268f48700 1 librados: setting wanted keys 2024-02-23T00:38:16.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.800+0000 7fb268f48700 1 librados: calling monclient init 2024-02-23T00:38:16.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.800+0000 7fb268f48700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb2640f6990 0x7fb264106e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:38:16.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.800+0000 7fb268f48700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2640f7c30 0x7fb264107320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:38:16.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.800+0000 7fb268f48700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2640f8580 0x7fb26410e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:38:16.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.800+0000 7fb268f48700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26410e8f0 con 0x7fb2640f7c30 2024-02-23T00:38:16.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.800+0000 7fb268f48700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26410ea60 con 0x7fb2640f8580 2024-02-23T00:38:16.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.800+0000 7fb268f48700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb26410ec20 con 0x7fb2640f6990 2024-02-23T00:38:16.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.800+0000 7fb262d9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2640f8580 0x7fb26410e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:38:16.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.800+0000 7fb262d9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2640f8580 0x7fb26410e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49826/0 (socket says 172.21.15.196:49826) 2024-02-23T00:38:16.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.800+0000 7fb262d9d700 1 -- 172.21.15.196:0/2809910869 learned_addr learned my addr 172.21.15.196:0/2809910869 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-02-23T00:38:16.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.800+0000 7fb261d9b700 1 --2- 172.21.15.196:0/2809910869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2640f7c30 0x7fb264107320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:38:16.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.801+0000 7fb262d9d700 1 -- 172.21.15.196:0/2809910869 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb2640f6990 msgr2=0x7fb264106e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:38:16.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.801+0000 7fb26259c700 1 --2- 172.21.15.196:0/2809910869 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb2640f6990 0x7fb264106e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:38:16.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.801+0000 7fb262d9d700 1 --2- 172.21.15.196:0/2809910869 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb2640f6990 0x7fb264106e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:38:16.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.801+0000 7fb262d9d700 1 -- 172.21.15.196:0/2809910869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2640f7c30 msgr2=0x7fb264107320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:38:16.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.801+0000 7fb262d9d700 1 --2- 172.21.15.196:0/2809910869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2640f7c30 0x7fb264107320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:38:16.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.801+0000 7fb262d9d700 1 -- 172.21.15.196:0/2809910869 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb24c005040 con 0x7fb2640f8580 2024-02-23T00:38:16.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.801+0000 7fb261d9b700 1 --2- 172.21.15.196:0/2809910869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2640f7c30 0x7fb264107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-23T00:38:16.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.801+0000 7fb262d9d700 1 --2- 172.21.15.196:0/2809910869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2640f8580 0x7fb26410e400 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fb25400bc30 tx=0x7fb254009840).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:38:16.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.801+0000 7fb25b7fe700 10 client.?.objecter ms_handle_connect 0x7fb2640f8580 2024-02-23T00:38:16.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.801+0000 7fb25b7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:38:16.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.801+0000 7fb25b7fe700 1 -- 172.21.15.196:0/2809910869 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25400c7a0 con 0x7fb2640f8580 2024-02-23T00:38:16.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.801+0000 7fb268f48700 1 -- 172.21.15.196:0/2809910869 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb26410ef00 con 0x7fb2640f8580 2024-02-23T00:38:16.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.801+0000 7fb25b7fe700 1 -- 172.21.15.196:0/2809910869 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fb254010070 con 0x7fb2640f8580 2024-02-23T00:38:16.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.801+0000 7fb25b7fe700 1 -- 172.21.15.196:0/2809910869 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb254007840 con 0x7fb2640f8580 2024-02-23T00:38:16.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.801+0000 7fb268f48700 10 client.24256.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:38:16.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.801+0000 7fb268f48700 1 -- 172.21.15.196:0/2809910869 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb26410f3b0 con 0x7fb2640f8580 2024-02-23T00:38:16.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.801+0000 7fb268f48700 1 librados: init done 2024-02-23T00:38:16.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.802+0000 7fb268f48700 1 -- 172.21.15.196:0/2809910869 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb26405f360 con 0x7fb2640f8580 2024-02-23T00:38:16.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.803+0000 7fb25b7fe700 1 -- 172.21.15.196:0/2809910869 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 14) v1 ==== 77064+0+0 (secure 0 0 0) 0x7fb254007bb0 con 0x7fb2640f8580 2024-02-23T00:38:16.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.803+0000 7fb25b7fe700 1 --2- 172.21.15.196:0/2809910869 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fb250062bd0 0x7fb250065090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:38:16.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.803+0000 7fb25b7fe700 1 -- 172.21.15.196:0/2809910869 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4396+0+0 (secure 0 0 0) 0x7fb25407c050 con 0x7fb2640f8580 2024-02-23T00:38:16.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.803+0000 7fb25b7fe700 10 client.24256.objecter ms_dispatch 0x7fb264001550 osd_map(42..42 src has 1..42) v4 2024-02-23T00:38:16.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.803+0000 7fb25b7fe700 3 client.24256.objecter handle_osd_map got epochs [42,42] > 0 2024-02-23T00:38:16.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.803+0000 7fb25b7fe700 3 client.24256.objecter handle_osd_map decoding full epoch 42 2024-02-23T00:38:16.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.803+0000 7fb25b7fe700 20 client.24256.objecter dump_active .. 0 homeless 2024-02-23T00:38:16.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.803+0000 7fb26259c700 1 --2- 172.21.15.196:0/2809910869 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fb250062bd0 0x7fb250065090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:38:16.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.804+0000 7fb25b7fe700 10 client.24256.objecter ms_handle_connect 0x7fb250062bd0 2024-02-23T00:38:16.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.804+0000 7fb26259c700 1 --2- 172.21.15.196:0/2809910869 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fb250062bd0 0x7fb250065090 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fb2640f7a90 tx=0x7fb25c008040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:38:16.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.807+0000 7fb25b7fe700 1 -- 172.21.15.196:0/2809910869 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb25404ca20 con 0x7fb2640f8580 2024-02-23T00:38:16.978 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:16.977+0000 7fb268f48700 1 -- 172.21.15.196:0/2809910869 --> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2640fe560 con 0x7fb250062bd0 2024-02-23T00:38:17.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:38:15.840460+0000 mgr.y (mgr.14146) 175 : cephadm [INF] Adjusting osd_memory_target on smithi196 to 8561M 2024-02-23T00:38:17.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:16.111026+0000 mgr.y (mgr.14146) 176 : cluster [DBG] pgmap v164: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:17.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:16.981114+0000 mon.a (mon.0) 460 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-23T00:38:17.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:16.983547+0000 mon.a (mon.0) 461 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-23T00:38:17.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:16.984348+0000 mon.a (mon.0) 462 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:38:17.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:38:15.840460+0000 mgr.y (mgr.14146) 175 : cephadm [INF] Adjusting osd_memory_target on smithi196 to 8561M 2024-02-23T00:38:17.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:16.111026+0000 mgr.y (mgr.14146) 176 : cluster [DBG] pgmap v164: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:17.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:16.981114+0000 mon.a (mon.0) 460 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-23T00:38:17.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:16.983547+0000 mon.a (mon.0) 461 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-23T00:38:17.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:16.984348+0000 mon.a (mon.0) 462 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:38:17.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:38:15.840460+0000 mgr.y (mgr.14146) 175 : cephadm [INF] Adjusting osd_memory_target on smithi196 to 8561M 2024-02-23T00:38:17.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:16.111026+0000 mgr.y (mgr.14146) 176 : cluster [DBG] pgmap v164: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:17.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:16.981114+0000 mon.a (mon.0) 460 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-23T00:38:17.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:16.983547+0000 mon.a (mon.0) 461 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-23T00:38:17.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:16.984348+0000 mon.a (mon.0) 462 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:38:18.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:16.978577+0000 mgr.y (mgr.14146) 177 : audit [DBG] from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:38:18.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:16.978577+0000 mgr.y (mgr.14146) 177 : audit [DBG] from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:38:18.310 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:16.978577+0000 mgr.y (mgr.14146) 177 : audit [DBG] from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:38:19.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:18.111460+0000 mgr.y (mgr.14146) 178 : cluster [DBG] pgmap v165: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:19.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:18.111460+0000 mgr.y (mgr.14146) 178 : cluster [DBG] pgmap v165: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:19.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:18.111460+0000 mgr.y (mgr.14146) 178 : cluster [DBG] pgmap v165: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:20.306 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:19.771847+0000 mon.b (mon.1) 16 : audit [INF] from='client.? 172.21.15.196:0/832114024' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df337124-9488-4aad-98e5-3614b51fc757"}]: dispatch 2024-02-23T00:38:20.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:19.772240+0000 mon.a (mon.0) 463 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df337124-9488-4aad-98e5-3614b51fc757"}]: dispatch 2024-02-23T00:38:20.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:19.777187+0000 mon.a (mon.0) 464 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df337124-9488-4aad-98e5-3614b51fc757"}]': finished 2024-02-23T00:38:20.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:19.777290+0000 mon.a (mon.0) 465 : cluster [DBG] osdmap e43: 7 total, 6 up, 7 in 2024-02-23T00:38:20.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:19.777470+0000 mon.a (mon.0) 466 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-23T00:38:20.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:19.771847+0000 mon.b (mon.1) 16 : audit [INF] from='client.? 172.21.15.196:0/832114024' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df337124-9488-4aad-98e5-3614b51fc757"}]: dispatch 2024-02-23T00:38:20.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:19.772240+0000 mon.a (mon.0) 463 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df337124-9488-4aad-98e5-3614b51fc757"}]: dispatch 2024-02-23T00:38:20.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:19.777187+0000 mon.a (mon.0) 464 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df337124-9488-4aad-98e5-3614b51fc757"}]': finished 2024-02-23T00:38:20.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:19.777290+0000 mon.a (mon.0) 465 : cluster [DBG] osdmap e43: 7 total, 6 up, 7 in 2024-02-23T00:38:20.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:19.777470+0000 mon.a (mon.0) 466 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-23T00:38:20.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:19.771847+0000 mon.b (mon.1) 16 : audit [INF] from='client.? 172.21.15.196:0/832114024' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df337124-9488-4aad-98e5-3614b51fc757"}]: dispatch 2024-02-23T00:38:20.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:19.772240+0000 mon.a (mon.0) 463 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df337124-9488-4aad-98e5-3614b51fc757"}]: dispatch 2024-02-23T00:38:20.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:19.777187+0000 mon.a (mon.0) 464 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df337124-9488-4aad-98e5-3614b51fc757"}]': finished 2024-02-23T00:38:20.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:19.777290+0000 mon.a (mon.0) 465 : cluster [DBG] osdmap e43: 7 total, 6 up, 7 in 2024-02-23T00:38:20.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:19.777470+0000 mon.a (mon.0) 466 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-23T00:38:21.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:20.111837+0000 mgr.y (mgr.14146) 179 : cluster [DBG] pgmap v167: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:21.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:20.406297+0000 mon.b (mon.1) 17 : audit [DBG] from='client.? 172.21.15.196:0/249004457' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-23T00:38:21.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:20.111837+0000 mgr.y (mgr.14146) 179 : cluster [DBG] pgmap v167: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:21.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:20.406297+0000 mon.b (mon.1) 17 : audit [DBG] from='client.? 172.21.15.196:0/249004457' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-23T00:38:21.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:20.111837+0000 mgr.y (mgr.14146) 179 : cluster [DBG] pgmap v167: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:21.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:20.406297+0000 mon.b (mon.1) 17 : audit [DBG] from='client.? 172.21.15.196:0/249004457' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-23T00:38:21.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:21.802+0000 7fb263fff700 10 client.24256.objecter tick 2024-02-23T00:38:23.387 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:22.112238+0000 mgr.y (mgr.14146) 180 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:23.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:22.112238+0000 mgr.y (mgr.14146) 180 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:23.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:22.112238+0000 mgr.y (mgr.14146) 180 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:25.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:24.112597+0000 mgr.y (mgr.14146) 181 : cluster [DBG] pgmap v169: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:25.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:24.112597+0000 mgr.y (mgr.14146) 181 : cluster [DBG] pgmap v169: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:25.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:24.112597+0000 mgr.y (mgr.14146) 181 : cluster [DBG] pgmap v169: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:26.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:26.802+0000 7fb263fff700 10 client.24256.objecter tick 2024-02-23T00:38:27.176 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:26.113066+0000 mgr.y (mgr.14146) 182 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:27.176 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:26.389996+0000 mon.a (mon.0) 467 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-23T00:38:27.176 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:26.390828+0000 mon.a (mon.0) 468 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:38:27.177 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:38:26.391549+0000 mgr.y (mgr.14146) 183 : cephadm [INF] Deploying daemon osd.6 on smithi196 2024-02-23T00:38:27.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:26.113066+0000 mgr.y (mgr.14146) 182 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:27.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:26.389996+0000 mon.a (mon.0) 467 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-23T00:38:27.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:26.390828+0000 mon.a (mon.0) 468 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:38:27.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:38:26.391549+0000 mgr.y (mgr.14146) 183 : cephadm [INF] Deploying daemon osd.6 on smithi196 2024-02-23T00:38:27.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:26.113066+0000 mgr.y (mgr.14146) 182 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:27.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:26.389996+0000 mon.a (mon.0) 467 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-23T00:38:27.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:26.390828+0000 mon.a (mon.0) 468 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:38:27.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:38:26.391549+0000 mgr.y (mgr.14146) 183 : cephadm [INF] Deploying daemon osd.6 on smithi196 2024-02-23T00:38:29.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:28.113547+0000 mgr.y (mgr.14146) 184 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:29.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:28.113547+0000 mgr.y (mgr.14146) 184 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:29.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:28.113547+0000 mgr.y (mgr.14146) 184 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:31.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:30.113997+0000 mgr.y (mgr.14146) 185 : cluster [DBG] pgmap v172: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:31.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:30.113997+0000 mgr.y (mgr.14146) 185 : cluster [DBG] pgmap v172: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:31.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:30.113997+0000 mgr.y (mgr.14146) 185 : cluster [DBG] pgmap v172: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:31.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:31.803+0000 7fb263fff700 10 client.24256.objecter tick 2024-02-23T00:38:33.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:32.114399+0000 mgr.y (mgr.14146) 186 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:33.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:32.132779+0000 mon.a (mon.0) 469 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:33.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:32.159282+0000 mon.a (mon.0) 470 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:38:33.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:32.114399+0000 mgr.y (mgr.14146) 186 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:33.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:32.132779+0000 mon.a (mon.0) 469 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:33.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:32.159282+0000 mon.a (mon.0) 470 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:38:33.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:32.114399+0000 mgr.y (mgr.14146) 186 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:33.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:32.132779+0000 mon.a (mon.0) 469 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:33.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:32.159282+0000 mon.a (mon.0) 470 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:38:35.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:33.277313+0000 osd.1 (osd.1) 7 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:38:35.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:33.277421+0000 osd.1 (osd.1) 8 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:38:35.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:34.114705+0000 mgr.y (mgr.14146) 187 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:35.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:34.135921+0000 mon.a (mon.0) 471 : cluster [DBG] osdmap e44: 7 total, 6 up, 7 in 2024-02-23T00:38:35.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:34.136096+0000 mon.a (mon.0) 472 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-23T00:38:35.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:33.277313+0000 osd.1 (osd.1) 7 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:38:35.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:33.277421+0000 osd.1 (osd.1) 8 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:38:35.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:34.114705+0000 mgr.y (mgr.14146) 187 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:35.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:34.135921+0000 mon.a (mon.0) 471 : cluster [DBG] osdmap e44: 7 total, 6 up, 7 in 2024-02-23T00:38:35.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:34.136096+0000 mon.a (mon.0) 472 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-23T00:38:35.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:33.277313+0000 osd.1 (osd.1) 7 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:38:35.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:33.277421+0000 osd.1 (osd.1) 8 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:38:35.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:34.114705+0000 mgr.y (mgr.14146) 187 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:35.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:34.135921+0000 mon.a (mon.0) 471 : cluster [DBG] osdmap e44: 7 total, 6 up, 7 in 2024-02-23T00:38:35.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:34.136096+0000 mon.a (mon.0) 472 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-23T00:38:35.985 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:35.983+0000 7fb25b7fe700 1 -- 172.21.15.196:0/2809910869 <== mgr.14146 v2:172.21.15.47:6800/622994871 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb2640fe560 con 0x7fb250062bd0 2024-02-23T00:38:35.985 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 6 on host 'smithi196' 2024-02-23T00:38:35.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:35.985+0000 7fb268f48700 10 librados: watch_flush enter 2024-02-23T00:38:35.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:35.985+0000 7fb268f48700 10 librados: watch_flush exit 2024-02-23T00:38:35.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:35.985+0000 7fb268f48700 20 client.24256.objecter shutdown clearing up homeless session... 2024-02-23T00:38:35.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:35.985+0000 7fb268f48700 10 client.24256.objecter successfully canceled tick 2024-02-23T00:38:35.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:35.985+0000 7fb268f48700 1 -- 172.21.15.196:0/2809910869 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fb250062bd0 msgr2=0x7fb250065090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:38:35.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:35.985+0000 7fb268f48700 1 --2- 172.21.15.196:0/2809910869 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fb250062bd0 0x7fb250065090 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fb2640f7a90 tx=0x7fb25c008040).stop 2024-02-23T00:38:35.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:35.985+0000 7fb268f48700 1 -- 172.21.15.196:0/2809910869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2640f8580 msgr2=0x7fb26410e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:38:35.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:35.985+0000 7fb268f48700 1 --2- 172.21.15.196:0/2809910869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2640f8580 0x7fb26410e400 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fb25400bc30 tx=0x7fb254009840).stop 2024-02-23T00:38:35.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:35.985+0000 7fb268f48700 1 -- 172.21.15.196:0/2809910869 shutdown_connections 2024-02-23T00:38:35.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:35.985+0000 7fb268f48700 1 --2- 172.21.15.196:0/2809910869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2640f8580 0x7fb26410e400 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:38:35.989 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:35.985+0000 7fb268f48700 1 --2- 172.21.15.196:0/2809910869 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb2640f6990 0x7fb264106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:38:35.989 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:35.985+0000 7fb268f48700 1 --2- 172.21.15.196:0/2809910869 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fb250062bd0 0x7fb250065090 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:38:35.989 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:35.985+0000 7fb268f48700 1 --2- 172.21.15.196:0/2809910869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2640f7c30 0x7fb264107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:38:35.989 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:35.985+0000 7fb268f48700 1 -- 172.21.15.196:0/2809910869 >> 172.21.15.196:0/2809910869 conn(0x7fb2640f2190 msgr2=0x7fb2640fb680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:38:35.989 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:35.985+0000 7fb268f48700 1 -- 172.21.15.196:0/2809910869 shutdown_connections 2024-02-23T00:38:35.989 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:35.985+0000 7fb268f48700 1 -- 172.21.15.196:0/2809910869 wait complete. 2024-02-23T00:38:35.989 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:35.985+0000 7fb268f48700 1 librados: shutdown 2024-02-23T00:38:36.382 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:35.126266+0000 mon.a (mon.0) 473 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:36.382 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:35.127142+0000 mon.a (mon.0) 474 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:38:36.382 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:35.128028+0000 mon.a (mon.0) 475 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:38:36.382 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:35.132792+0000 mon.a (mon.0) 476 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:36.382 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:35.220091+0000 mon.b (mon.1) 18 : audit [INF] from='osd.6 [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-23T00:38:36.382 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:35.220407+0000 mon.a (mon.0) 477 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-23T00:38:36.382 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:35.983331+0000 mon.a (mon.0) 478 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:36.383 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:36.001098+0000 mon.a (mon.0) 479 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:38:36.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:35.126266+0000 mon.a (mon.0) 473 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:36.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:35.127142+0000 mon.a (mon.0) 474 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:38:36.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:35.128028+0000 mon.a (mon.0) 475 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:38:36.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:35.132792+0000 mon.a (mon.0) 476 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:36.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:35.220091+0000 mon.b (mon.1) 18 : audit [INF] from='osd.6 [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-23T00:38:36.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:35.220407+0000 mon.a (mon.0) 477 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-23T00:38:36.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:35.983331+0000 mon.a (mon.0) 478 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:36.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:36.001098+0000 mon.a (mon.0) 479 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:38:36.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:35.126266+0000 mon.a (mon.0) 473 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:36.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:35.127142+0000 mon.a (mon.0) 474 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:38:36.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:35.128028+0000 mon.a (mon.0) 475 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:38:36.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:35.132792+0000 mon.a (mon.0) 476 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:36.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:35.220091+0000 mon.b (mon.1) 18 : audit [INF] from='osd.6 [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-23T00:38:36.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:35.220407+0000 mon.a (mon.0) 477 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-23T00:38:36.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:35.983331+0000 mon.a (mon.0) 478 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:36.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:36.001098+0000 mon.a (mon.0) 479 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:38:36.752 DEBUG:teuthology.orchestra.run.smithi196:osd.6> sudo journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@osd.6.service 2024-02-23T00:38:36.754 INFO:tasks.cephadm:Deploying osd.7 on smithi196 with /dev/vg_nvme/lv_1... 2024-02-23T00:38:36.755 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-23T00:38:36.787 INFO:journalctl@ceph.osd.6.smithi196.stdout:-- Logs begin at Fri 2024-02-23 00:13:21 UTC. -- 2024-02-23T00:38:37.327 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:38:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:38:37.140+0000 7f2ce9dec700 -1 osd.6 0 waiting for initial osdmap 2024-02-23T00:38:37.327 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:38:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:38:37.146+0000 7f2ce6568700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-23T00:38:37.327 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:36.115175+0000 mgr.y (mgr.14146) 188 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:37.327 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:36.134566+0000 mon.a (mon.0) 480 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-23T00:38:37.328 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:36.134684+0000 mon.a (mon.0) 481 : cluster [DBG] osdmap e45: 7 total, 6 up, 7 in 2024-02-23T00:38:37.328 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:36.134823+0000 mon.a (mon.0) 482 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-23T00:38:37.328 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:36.135273+0000 mon.b (mon.1) 19 : audit [INF] from='osd.6 [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-02-23T00:38:37.328 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:36.135513+0000 mon.a (mon.0) 483 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-02-23T00:38:37.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:36.115175+0000 mgr.y (mgr.14146) 188 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:37.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:36.134566+0000 mon.a (mon.0) 480 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-23T00:38:37.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:36.134684+0000 mon.a (mon.0) 481 : cluster [DBG] osdmap e45: 7 total, 6 up, 7 in 2024-02-23T00:38:37.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:36.134823+0000 mon.a (mon.0) 482 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-23T00:38:37.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:36.135273+0000 mon.b (mon.1) 19 : audit [INF] from='osd.6 [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-02-23T00:38:37.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:36.135513+0000 mon.a (mon.0) 483 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-02-23T00:38:37.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:36.115175+0000 mgr.y (mgr.14146) 188 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:37.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:36.134566+0000 mon.a (mon.0) 480 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-23T00:38:37.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:36.134684+0000 mon.a (mon.0) 481 : cluster [DBG] osdmap e45: 7 total, 6 up, 7 in 2024-02-23T00:38:37.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:36.134823+0000 mon.a (mon.0) 482 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-23T00:38:37.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:36.135273+0000 mon.b (mon.1) 19 : audit [INF] from='osd.6 [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-02-23T00:38:37.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:36.135513+0000 mon.a (mon.0) 483 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-02-23T00:38:38.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:37.136829+0000 mon.a (mon.0) 484 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-02-23T00:38:38.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:37.136963+0000 mon.a (mon.0) 485 : cluster [DBG] osdmap e46: 7 total, 6 up, 7 in 2024-02-23T00:38:38.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:37.137458+0000 mon.a (mon.0) 486 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-23T00:38:38.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:37.141814+0000 mon.a (mon.0) 487 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-23T00:38:38.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:37.136829+0000 mon.a (mon.0) 484 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-02-23T00:38:38.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:37.136963+0000 mon.a (mon.0) 485 : cluster [DBG] osdmap e46: 7 total, 6 up, 7 in 2024-02-23T00:38:38.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:37.137458+0000 mon.a (mon.0) 486 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-23T00:38:38.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:37.141814+0000 mon.a (mon.0) 487 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-23T00:38:38.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:37.136829+0000 mon.a (mon.0) 484 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-02-23T00:38:38.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:37.136963+0000 mon.a (mon.0) 485 : cluster [DBG] osdmap e46: 7 total, 6 up, 7 in 2024-02-23T00:38:38.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:37.137458+0000 mon.a (mon.0) 486 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-23T00:38:38.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:37.141814+0000 mon.a (mon.0) 487 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-23T00:38:39.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:36.219359+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:38:39.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:36.219421+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:38:39.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:38.115497+0000 mgr.y (mgr.14146) 189 : cluster [DBG] pgmap v179: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:39.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:38.141953+0000 mon.a (mon.0) 488 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-23T00:38:39.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:38.146670+0000 mon.a (mon.0) 489 : cluster [INF] osd.6 [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] boot 2024-02-23T00:38:39.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:38.146732+0000 mon.a (mon.0) 490 : cluster [DBG] osdmap e47: 7 total, 7 up, 7 in 2024-02-23T00:38:39.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:38.147194+0000 mon.a (mon.0) 491 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-23T00:38:39.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:38.942588+0000 mon.a (mon.0) 492 : cluster [DBG] osdmap e48: 7 total, 7 up, 7 in 2024-02-23T00:38:39.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:36.219359+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:38:39.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:36.219421+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:38:39.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:38.115497+0000 mgr.y (mgr.14146) 189 : cluster [DBG] pgmap v179: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:39.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:38.141953+0000 mon.a (mon.0) 488 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-23T00:38:39.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:38.146670+0000 mon.a (mon.0) 489 : cluster [INF] osd.6 [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] boot 2024-02-23T00:38:39.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:38.146732+0000 mon.a (mon.0) 490 : cluster [DBG] osdmap e47: 7 total, 7 up, 7 in 2024-02-23T00:38:39.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:38.147194+0000 mon.a (mon.0) 491 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-23T00:38:39.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:38.942588+0000 mon.a (mon.0) 492 : cluster [DBG] osdmap e48: 7 total, 7 up, 7 in 2024-02-23T00:38:39.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:36.219359+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:38:39.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:36.219421+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:38:39.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:38.115497+0000 mgr.y (mgr.14146) 189 : cluster [DBG] pgmap v179: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:38:39.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:38.141953+0000 mon.a (mon.0) 488 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-23T00:38:39.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:38.146670+0000 mon.a (mon.0) 489 : cluster [INF] osd.6 [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] boot 2024-02-23T00:38:39.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:38.146732+0000 mon.a (mon.0) 490 : cluster [DBG] osdmap e47: 7 total, 7 up, 7 in 2024-02-23T00:38:39.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:38.147194+0000 mon.a (mon.0) 491 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-23T00:38:39.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:38.942588+0000 mon.a (mon.0) 492 : cluster [DBG] osdmap e48: 7 total, 7 up, 7 in 2024-02-23T00:38:40.726 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-23T00:38:40.726 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-23T00:38:40.726 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-02-23T00:38:40.726 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-02-23T00:38:40.726 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125314 s, 837 MB/s 2024-02-23T00:38:40.726 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-02-23T00:38:41.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:39.943822+0000 mon.a (mon.0) 493 : cluster [DBG] osdmap e49: 7 total, 7 up, 7 in 2024-02-23T00:38:41.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:40.115862+0000 mgr.y (mgr.14146) 190 : cluster [DBG] pgmap v183: 1 pgs: 1 peering; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:38:41.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:39.943822+0000 mon.a (mon.0) 493 : cluster [DBG] osdmap e49: 7 total, 7 up, 7 in 2024-02-23T00:38:41.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:40.115862+0000 mgr.y (mgr.14146) 190 : cluster [DBG] pgmap v183: 1 pgs: 1 peering; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:38:41.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:39.943822+0000 mon.a (mon.0) 493 : cluster [DBG] osdmap e49: 7 total, 7 up, 7 in 2024-02-23T00:38:41.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:40.115862+0000 mgr.y (mgr.14146) 190 : cluster [DBG] pgmap v183: 1 pgs: 1 peering; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:38:41.993 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph orch daemon add osd smithi196:vg_nvme/lv_1 2024-02-23T00:38:42.690 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:38:41.521968+0000 mgr.y (mgr.14146) 191 : cephadm [INF] Detected new or changed devices on smithi196 2024-02-23T00:38:42.690 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:41.527318+0000 mon.a (mon.0) 494 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:42.690 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:41.528405+0000 mon.a (mon.0) 495 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:38:42.690 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:41.529286+0000 mon.a (mon.0) 496 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:38:42.690 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:41.530021+0000 mon.a (mon.0) 497 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:38:42.690 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:38:41.530589+0000 mgr.y (mgr.14146) 192 : cephadm [INF] Adjusting osd_memory_target on smithi196 to 5707M 2024-02-23T00:38:42.690 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:41.534868+0000 mon.a (mon.0) 498 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:42.690 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:41.535875+0000 mon.a (mon.0) 499 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:38:42.690 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:41.536749+0000 mon.a (mon.0) 500 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:38:42.691 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:41.541557+0000 mon.a (mon.0) 501 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:42.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:38:41.521968+0000 mgr.y (mgr.14146) 191 : cephadm [INF] Detected new or changed devices on smithi196 2024-02-23T00:38:42.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:41.527318+0000 mon.a (mon.0) 494 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:42.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:41.528405+0000 mon.a (mon.0) 495 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:38:42.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:41.529286+0000 mon.a (mon.0) 496 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:38:42.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:41.530021+0000 mon.a (mon.0) 497 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:38:42.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:38:41.530589+0000 mgr.y (mgr.14146) 192 : cephadm [INF] Adjusting osd_memory_target on smithi196 to 5707M 2024-02-23T00:38:42.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:41.534868+0000 mon.a (mon.0) 498 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:42.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:41.535875+0000 mon.a (mon.0) 499 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:38:42.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:41.536749+0000 mon.a (mon.0) 500 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:38:42.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:41.541557+0000 mon.a (mon.0) 501 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:42.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:38:41.521968+0000 mgr.y (mgr.14146) 191 : cephadm [INF] Detected new or changed devices on smithi196 2024-02-23T00:38:42.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:41.527318+0000 mon.a (mon.0) 494 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:42.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:41.528405+0000 mon.a (mon.0) 495 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:38:42.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:41.529286+0000 mon.a (mon.0) 496 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:38:42.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:41.530021+0000 mon.a (mon.0) 497 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:38:42.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:38:41.530589+0000 mgr.y (mgr.14146) 192 : cephadm [INF] Adjusting osd_memory_target on smithi196 to 5707M 2024-02-23T00:38:42.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:41.534868+0000 mon.a (mon.0) 498 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:42.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:41.535875+0000 mon.a (mon.0) 499 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:38:42.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:41.536749+0000 mon.a (mon.0) 500 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:38:42.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:41.541557+0000 mon.a (mon.0) 501 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:38:43.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:42.091758+0000 osd.3 (osd.3) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:38:43.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:42.091859+0000 osd.3 (osd.3) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:38:43.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:42.116338+0000 mgr.y (mgr.14146) 193 : cluster [DBG] pgmap v184: 1 pgs: 1 peering; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:38:43.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:42.539869+0000 mon.a (mon.0) 502 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-23T00:38:43.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:42.546491+0000 mon.a (mon.0) 503 : cluster [DBG] osdmap e50: 7 total, 7 up, 7 in 2024-02-23T00:38:43.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:42.091758+0000 osd.3 (osd.3) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:38:43.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:42.091859+0000 osd.3 (osd.3) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:38:43.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:42.116338+0000 mgr.y (mgr.14146) 193 : cluster [DBG] pgmap v184: 1 pgs: 1 peering; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:38:43.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:42.539869+0000 mon.a (mon.0) 502 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-23T00:38:43.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:42.546491+0000 mon.a (mon.0) 503 : cluster [DBG] osdmap e50: 7 total, 7 up, 7 in 2024-02-23T00:38:43.815 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:42.091758+0000 osd.3 (osd.3) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:38:43.815 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:42.091859+0000 osd.3 (osd.3) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:38:43.815 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:42.116338+0000 mgr.y (mgr.14146) 193 : cluster [DBG] pgmap v184: 1 pgs: 1 peering; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:38:43.815 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:42.539869+0000 mon.a (mon.0) 502 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-23T00:38:43.815 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:42.546491+0000 mon.a (mon.0) 503 : cluster [DBG] osdmap e50: 7 total, 7 up, 7 in 2024-02-23T00:38:44.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.244+0000 7f4e5d6fe700 1 -- 172.21.15.196:0/3039789036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e580f6170 msgr2=0x7f4e580fd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:38:44.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.244+0000 7f4e5d6fe700 1 --2- 172.21.15.196:0/3039789036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e580f6170 0x7f4e580fd430 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f4e4c004870 tx=0x7f4e4c009cb0).stop 2024-02-23T00:38:44.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.244+0000 7f4e5d6fe700 1 -- 172.21.15.196:0/3039789036 shutdown_connections 2024-02-23T00:38:44.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.244+0000 7f4e5d6fe700 1 --2- 172.21.15.196:0/3039789036 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4e580fd920 0x7f4e580ffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:38:44.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.244+0000 7f4e5d6fe700 1 --2- 172.21.15.196:0/3039789036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e580f5860 0x7f4e580f5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:38:44.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.244+0000 7f4e5d6fe700 1 --2- 172.21.15.196:0/3039789036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e580f6170 0x7f4e580fd430 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:38:44.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.244+0000 7f4e5d6fe700 1 -- 172.21.15.196:0/3039789036 >> 172.21.15.196:0/3039789036 conn(0x7f4e580f14a0 msgr2=0x7f4e580f38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:38:44.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.245+0000 7f4e5d6fe700 1 -- 172.21.15.196:0/3039789036 shutdown_connections 2024-02-23T00:38:44.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.245+0000 7f4e5d6fe700 1 -- 172.21.15.196:0/3039789036 wait complete. 2024-02-23T00:38:44.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.245+0000 7f4e5d6fe700 1 librados: starting msgr at 2024-02-23T00:38:44.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.245+0000 7f4e5d6fe700 1 librados: starting objecter 2024-02-23T00:38:44.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.245+0000 7f4e5d6fe700 1 Processor -- start 2024-02-23T00:38:44.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.245+0000 7f4e5d6fe700 1 -- start start 2024-02-23T00:38:44.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.245+0000 7f4e5d6fe700 1 librados: setting wanted keys 2024-02-23T00:38:44.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.245+0000 7f4e5d6fe700 1 librados: calling monclient init 2024-02-23T00:38:44.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.245+0000 7f4e5d6fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4e580f5860 0x7f4e580f8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:38:44.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.245+0000 7f4e5d6fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e580f6170 0x7f4e580f7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:38:44.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.245+0000 7f4e5d6fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e580fd920 0x7f4e580f7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:38:44.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.245+0000 7f4e5d6fe700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e580f7e30 con 0x7f4e580fd920 2024-02-23T00:38:44.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.245+0000 7f4e5d6fe700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e580f7fa0 con 0x7f4e580f6170 2024-02-23T00:38:44.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.245+0000 7f4e5d6fe700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e580f8160 con 0x7f4e580f5860 2024-02-23T00:38:44.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.246+0000 7f4e567fc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e580f6170 0x7f4e580f7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:38:44.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.246+0000 7f4e567fc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e580f6170 0x7f4e580f7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:36298/0 (socket says 172.21.15.196:36298) 2024-02-23T00:38:44.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.246+0000 7f4e567fc700 1 -- 172.21.15.196:0/945964007 learned_addr learned my addr 172.21.15.196:0/945964007 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-02-23T00:38:44.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.246+0000 7f4e56ffd700 1 --2- 172.21.15.196:0/945964007 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4e580f5860 0x7f4e580f8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:38:44.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.246+0000 7f4e577fe700 1 --2- 172.21.15.196:0/945964007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e580fd920 0x7f4e580f7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:38:44.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.246+0000 7f4e567fc700 1 -- 172.21.15.196:0/945964007 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4e580f5860 msgr2=0x7f4e580f8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:38:44.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.246+0000 7f4e567fc700 1 --2- 172.21.15.196:0/945964007 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4e580f5860 0x7f4e580f8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:38:44.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.246+0000 7f4e567fc700 1 -- 172.21.15.196:0/945964007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e580fd920 msgr2=0x7f4e580f7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:38:44.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.246+0000 7f4e567fc700 1 --2- 172.21.15.196:0/945964007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e580fd920 0x7f4e580f7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:38:44.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.246+0000 7f4e567fc700 1 -- 172.21.15.196:0/945964007 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4c005040 con 0x7f4e580f6170 2024-02-23T00:38:44.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.246+0000 7f4e567fc700 1 --2- 172.21.15.196:0/945964007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e580f6170 0x7f4e580f7320 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f4e4c0137b0 tx=0x7f4e4c009cb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:38:44.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.246+0000 7f4e3ffff700 10 client.?.objecter ms_handle_connect 0x7f4e580f6170 2024-02-23T00:38:44.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.246+0000 7f4e3ffff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:38:44.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.246+0000 7f4e3ffff700 1 -- 172.21.15.196:0/945964007 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c00e030 con 0x7f4e580f6170 2024-02-23T00:38:44.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.246+0000 7f4e3ffff700 1 -- 172.21.15.196:0/945964007 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f4e4c0096d0 con 0x7f4e580f6170 2024-02-23T00:38:44.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.246+0000 7f4e3ffff700 1 -- 172.21.15.196:0/945964007 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c018070 con 0x7f4e580f6170 2024-02-23T00:38:44.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.246+0000 7f4e5d6fe700 1 -- 172.21.15.196:0/945964007 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e58112170 con 0x7f4e580f6170 2024-02-23T00:38:44.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.246+0000 7f4e5d6fe700 10 client.24283.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:38:44.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.246+0000 7f4e5d6fe700 1 -- 172.21.15.196:0/945964007 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e58112520 con 0x7f4e580f6170 2024-02-23T00:38:44.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.246+0000 7f4e5d6fe700 1 librados: init done 2024-02-23T00:38:44.253 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.247+0000 7f4e5d6fe700 1 -- 172.21.15.196:0/945964007 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e5805ffd0 con 0x7f4e580f6170 2024-02-23T00:38:44.253 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.248+0000 7f4e3ffff700 1 -- 172.21.15.196:0/945964007 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 14) v1 ==== 77064+0+0 (secure 0 0 0) 0x7f4e4c01d420 con 0x7f4e580f6170 2024-02-23T00:38:44.253 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.248+0000 7f4e3ffff700 1 --2- 172.21.15.196:0/945964007 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f4e40062bd0 0x7f4e40065090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:38:44.253 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.248+0000 7f4e3ffff700 1 -- 172.21.15.196:0/945964007 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4828+0+0 (secure 0 0 0) 0x7f4e4c07d640 con 0x7f4e580f6170 2024-02-23T00:38:44.253 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.248+0000 7f4e3ffff700 10 client.24283.objecter ms_dispatch 0x7f4e58001550 osd_map(50..50 src has 1..50) v4 2024-02-23T00:38:44.253 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.248+0000 7f4e3ffff700 3 client.24283.objecter handle_osd_map got epochs [50,50] > 0 2024-02-23T00:38:44.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.248+0000 7f4e3ffff700 3 client.24283.objecter handle_osd_map decoding full epoch 50 2024-02-23T00:38:44.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.248+0000 7f4e3ffff700 20 client.24283.objecter dump_active .. 0 homeless 2024-02-23T00:38:44.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.248+0000 7f4e56ffd700 1 --2- 172.21.15.196:0/945964007 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f4e40062bd0 0x7f4e40065090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:38:44.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.248+0000 7f4e3ffff700 10 client.24283.objecter ms_handle_connect 0x7f4e40062bd0 2024-02-23T00:38:44.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.248+0000 7f4e56ffd700 1 --2- 172.21.15.196:0/945964007 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f4e40062bd0 0x7f4e40065090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f4e5805f750 tx=0x7f4e4800b040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:38:44.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.252+0000 7f4e3ffff700 1 -- 172.21.15.196:0/945964007 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f4e4c04de60 con 0x7f4e580f6170 2024-02-23T00:38:44.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:44.420+0000 7f4e5d6fe700 1 -- 172.21.15.196:0/945964007 --> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e580fca70 con 0x7f4e40062bd0 2024-02-23T00:38:45.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:44.116864+0000 mgr.y (mgr.14146) 194 : cluster [DBG] pgmap v186: 1 pgs: 1 peering; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:38:45.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:44.422672+0000 mgr.y (mgr.14146) 195 : audit [DBG] from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:38:45.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:44.425174+0000 mon.a (mon.0) 504 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-23T00:38:45.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:44.427833+0000 mon.a (mon.0) 505 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-23T00:38:45.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:44.428640+0000 mon.a (mon.0) 506 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:38:45.306 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:44.116864+0000 mgr.y (mgr.14146) 194 : cluster [DBG] pgmap v186: 1 pgs: 1 peering; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:38:45.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:44.422672+0000 mgr.y (mgr.14146) 195 : audit [DBG] from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:38:45.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:44.425174+0000 mon.a (mon.0) 504 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-23T00:38:45.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:44.427833+0000 mon.a (mon.0) 505 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-23T00:38:45.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:44.428640+0000 mon.a (mon.0) 506 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:38:45.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:44.116864+0000 mgr.y (mgr.14146) 194 : cluster [DBG] pgmap v186: 1 pgs: 1 peering; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:38:45.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:44.422672+0000 mgr.y (mgr.14146) 195 : audit [DBG] from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:38:45.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:44.425174+0000 mon.a (mon.0) 504 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-23T00:38:45.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:44.427833+0000 mon.a (mon.0) 505 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-23T00:38:45.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:44.428640+0000 mon.a (mon.0) 506 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:38:46.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:46.167876+0000 mon.a (mon.0) 507 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-02-23T00:38:46.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:46.167943+0000 mon.a (mon.0) 508 : cluster [INF] Cluster is now healthy 2024-02-23T00:38:46.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:46.167876+0000 mon.a (mon.0) 507 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-02-23T00:38:46.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:46.167943+0000 mon.a (mon.0) 508 : cluster [INF] Cluster is now healthy 2024-02-23T00:38:46.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:46.167876+0000 mon.a (mon.0) 507 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-02-23T00:38:46.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:46.167943+0000 mon.a (mon.0) 508 : cluster [INF] Cluster is now healthy 2024-02-23T00:38:47.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:46.117237+0000 mgr.y (mgr.14146) 196 : cluster [DBG] pgmap v187: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:38:47.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:47.075691+0000 mon.b (mon.1) 20 : audit [INF] from='client.? 172.21.15.196:0/1582381036' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84fee1f2-1a88-447a-b1f3-56ff7efe7fec"}]: dispatch 2024-02-23T00:38:47.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:47.076233+0000 mon.a (mon.0) 509 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84fee1f2-1a88-447a-b1f3-56ff7efe7fec"}]: dispatch 2024-02-23T00:38:47.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:47.081226+0000 mon.a (mon.0) 510 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84fee1f2-1a88-447a-b1f3-56ff7efe7fec"}]': finished 2024-02-23T00:38:47.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:47.081315+0000 mon.a (mon.0) 511 : cluster [DBG] osdmap e51: 8 total, 7 up, 8 in 2024-02-23T00:38:47.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:47.081465+0000 mon.a (mon.0) 512 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-23T00:38:47.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:46.117237+0000 mgr.y (mgr.14146) 196 : cluster [DBG] pgmap v187: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:38:47.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:47.075691+0000 mon.b (mon.1) 20 : audit [INF] from='client.? 172.21.15.196:0/1582381036' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84fee1f2-1a88-447a-b1f3-56ff7efe7fec"}]: dispatch 2024-02-23T00:38:47.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:47.076233+0000 mon.a (mon.0) 509 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84fee1f2-1a88-447a-b1f3-56ff7efe7fec"}]: dispatch 2024-02-23T00:38:47.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:47.081226+0000 mon.a (mon.0) 510 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84fee1f2-1a88-447a-b1f3-56ff7efe7fec"}]': finished 2024-02-23T00:38:47.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:47.081315+0000 mon.a (mon.0) 511 : cluster [DBG] osdmap e51: 8 total, 7 up, 8 in 2024-02-23T00:38:47.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:47.081465+0000 mon.a (mon.0) 512 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-23T00:38:47.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:46.117237+0000 mgr.y (mgr.14146) 196 : cluster [DBG] pgmap v187: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:38:47.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:47.075691+0000 mon.b (mon.1) 20 : audit [INF] from='client.? 172.21.15.196:0/1582381036' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84fee1f2-1a88-447a-b1f3-56ff7efe7fec"}]: dispatch 2024-02-23T00:38:47.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:47.076233+0000 mon.a (mon.0) 509 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84fee1f2-1a88-447a-b1f3-56ff7efe7fec"}]: dispatch 2024-02-23T00:38:47.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:47.081226+0000 mon.a (mon.0) 510 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84fee1f2-1a88-447a-b1f3-56ff7efe7fec"}]': finished 2024-02-23T00:38:47.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:47.081315+0000 mon.a (mon.0) 511 : cluster [DBG] osdmap e51: 8 total, 7 up, 8 in 2024-02-23T00:38:47.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:47.081465+0000 mon.a (mon.0) 512 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-23T00:38:48.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:47.684074+0000 mon.b (mon.1) 21 : audit [DBG] from='client.? 172.21.15.196:0/3756794016' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-23T00:38:48.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:47.684074+0000 mon.b (mon.1) 21 : audit [DBG] from='client.? 172.21.15.196:0/3756794016' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-23T00:38:48.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:47.684074+0000 mon.b (mon.1) 21 : audit [DBG] from='client.? 172.21.15.196:0/3756794016' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-23T00:38:49.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:49.247+0000 7f4e5cefd700 10 client.24283.objecter tick 2024-02-23T00:38:49.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:48.117718+0000 mgr.y (mgr.14146) 197 : cluster [DBG] pgmap v189: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:38:49.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:48.117718+0000 mgr.y (mgr.14146) 197 : cluster [DBG] pgmap v189: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:38:49.697 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:48.117718+0000 mgr.y (mgr.14146) 197 : cluster [DBG] pgmap v189: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:38:51.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:50.118107+0000 mgr.y (mgr.14146) 198 : cluster [DBG] pgmap v190: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:38:51.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:50.118107+0000 mgr.y (mgr.14146) 198 : cluster [DBG] pgmap v190: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:38:51.559 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:50.118107+0000 mgr.y (mgr.14146) 198 : cluster [DBG] pgmap v190: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:38:53.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:52.118585+0000 mgr.y (mgr.14146) 199 : cluster [DBG] pgmap v191: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:38:53.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:52.118585+0000 mgr.y (mgr.14146) 199 : cluster [DBG] pgmap v191: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:38:53.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:52.118585+0000 mgr.y (mgr.14146) 199 : cluster [DBG] pgmap v191: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:38:54.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:54.248+0000 7f4e5cefd700 10 client.24283.objecter tick 2024-02-23T00:38:54.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:53.865031+0000 mon.a (mon.0) 513 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-23T00:38:54.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:53.865912+0000 mon.a (mon.0) 514 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:38:54.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:53.865031+0000 mon.a (mon.0) 513 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-23T00:38:54.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:53.865912+0000 mon.a (mon.0) 514 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:38:54.670 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:53.865031+0000 mon.a (mon.0) 513 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-23T00:38:54.671 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:53.865912+0000 mon.a (mon.0) 514 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:38:55.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:38:53.866659+0000 mgr.y (mgr.14146) 200 : cephadm [INF] Deploying daemon osd.7 on smithi196 2024-02-23T00:38:55.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:54.119039+0000 mgr.y (mgr.14146) 201 : cluster [DBG] pgmap v192: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:38:55.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:38:53.866659+0000 mgr.y (mgr.14146) 200 : cephadm [INF] Deploying daemon osd.7 on smithi196 2024-02-23T00:38:55.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:54.119039+0000 mgr.y (mgr.14146) 201 : cluster [DBG] pgmap v192: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:38:55.592 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:38:53.866659+0000 mgr.y (mgr.14146) 200 : cephadm [INF] Deploying daemon osd.7 on smithi196 2024-02-23T00:38:55.593 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:54.119039+0000 mgr.y (mgr.14146) 201 : cluster [DBG] pgmap v192: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:38:57.301 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:56.119457+0000 mgr.y (mgr.14146) 202 : cluster [DBG] pgmap v193: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:38:57.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:56.119457+0000 mgr.y (mgr.14146) 202 : cluster [DBG] pgmap v193: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:38:57.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:56.119457+0000 mgr.y (mgr.14146) 202 : cluster [DBG] pgmap v193: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:38:59.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:38:59.249+0000 7f4e5cefd700 10 client.24283.objecter tick 2024-02-23T00:38:59.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:38:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:38:58.119924+0000 mgr.y (mgr.14146) 203 : cluster [DBG] pgmap v194: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:38:59.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:38:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:38:58.119924+0000 mgr.y (mgr.14146) 203 : cluster [DBG] pgmap v194: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:38:59.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:38:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:38:58.119924+0000 mgr.y (mgr.14146) 203 : cluster [DBG] pgmap v194: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:39:00.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:59.572936+0000 mon.a (mon.0) 515 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:00.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:38:59.604048+0000 mon.a (mon.0) 516 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:39:01.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:59.572936+0000 mon.a (mon.0) 515 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:01.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:38:59.604048+0000 mon.a (mon.0) 516 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:39:01.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:59.572936+0000 mon.a (mon.0) 515 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:01.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:38:59.604048+0000 mon.a (mon.0) 516 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:39:01.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:00.120480+0000 mgr.y (mgr.14146) 204 : cluster [DBG] pgmap v195: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:39:02.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:00.120480+0000 mgr.y (mgr.14146) 204 : cluster [DBG] pgmap v195: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:39:02.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:00.120480+0000 mgr.y (mgr.14146) 204 : cluster [DBG] pgmap v195: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:39:03.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:03.573+0000 7f4e3ffff700 1 -- 172.21.15.196:0/945964007 <== mgr.14146 v2:172.21.15.47:6800/622994871 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4e580fca70 con 0x7f4e40062bd0 2024-02-23T00:39:03.575 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 7 on host 'smithi196' 2024-02-23T00:39:03.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:03.575+0000 7f4e5d6fe700 10 librados: watch_flush enter 2024-02-23T00:39:03.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:03.575+0000 7f4e5d6fe700 10 librados: watch_flush exit 2024-02-23T00:39:03.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:03.575+0000 7f4e5d6fe700 20 client.24283.objecter shutdown clearing up homeless session... 2024-02-23T00:39:03.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:03.575+0000 7f4e5d6fe700 10 client.24283.objecter successfully canceled tick 2024-02-23T00:39:03.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:03.576+0000 7f4e5d6fe700 1 -- 172.21.15.196:0/945964007 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f4e40062bd0 msgr2=0x7f4e40065090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:03.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:03.576+0000 7f4e5d6fe700 1 --2- 172.21.15.196:0/945964007 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f4e40062bd0 0x7f4e40065090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f4e5805f750 tx=0x7f4e4800b040).stop 2024-02-23T00:39:03.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:03.576+0000 7f4e5d6fe700 1 -- 172.21.15.196:0/945964007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e580f6170 msgr2=0x7f4e580f7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:03.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:03.576+0000 7f4e5d6fe700 1 --2- 172.21.15.196:0/945964007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e580f6170 0x7f4e580f7320 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f4e4c0137b0 tx=0x7f4e4c009cb0).stop 2024-02-23T00:39:03.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:03.576+0000 7f4e5d6fe700 1 -- 172.21.15.196:0/945964007 shutdown_connections 2024-02-23T00:39:03.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:03.576+0000 7f4e5d6fe700 1 --2- 172.21.15.196:0/945964007 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f4e40062bd0 0x7f4e40065090 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:03.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:03.576+0000 7f4e5d6fe700 1 --2- 172.21.15.196:0/945964007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e580fd920 0x7f4e580f7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:03.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:03.576+0000 7f4e5d6fe700 1 --2- 172.21.15.196:0/945964007 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4e580f5860 0x7f4e580f8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:03.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:03.576+0000 7f4e5d6fe700 1 --2- 172.21.15.196:0/945964007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e580f6170 0x7f4e580f7320 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:03.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:03.576+0000 7f4e5d6fe700 1 -- 172.21.15.196:0/945964007 >> 172.21.15.196:0/945964007 conn(0x7f4e580f14a0 msgr2=0x7f4e580f20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:39:03.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:03.576+0000 7f4e5d6fe700 1 -- 172.21.15.196:0/945964007 shutdown_connections 2024-02-23T00:39:03.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:03.576+0000 7f4e5d6fe700 1 -- 172.21.15.196:0/945964007 wait complete. 2024-02-23T00:39:03.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:03.576+0000 7f4e5d6fe700 1 librados: shutdown 2024-02-23T00:39:03.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:02.120924+0000 mgr.y (mgr.14146) 205 : cluster [DBG] pgmap v196: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:39:03.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:02.655448+0000 mon.a (mon.0) 517 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:03.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:02.656331+0000 mon.a (mon.0) 518 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:39:03.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:02.657262+0000 mon.a (mon.0) 519 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:39:03.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:02.662024+0000 mon.a (mon.0) 520 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:03.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:02.690112+0000 mon.b (mon.1) 22 : audit [INF] from='osd.7 [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-23T00:39:03.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:02.690479+0000 mon.a (mon.0) 521 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-23T00:39:03.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:03.573111+0000 mon.a (mon.0) 522 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:04.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:02.120924+0000 mgr.y (mgr.14146) 205 : cluster [DBG] pgmap v196: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:39:04.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:02.655448+0000 mon.a (mon.0) 517 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:04.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:02.656331+0000 mon.a (mon.0) 518 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:39:04.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:02.657262+0000 mon.a (mon.0) 519 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:39:04.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:02.662024+0000 mon.a (mon.0) 520 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:04.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:02.690112+0000 mon.b (mon.1) 22 : audit [INF] from='osd.7 [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-23T00:39:04.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:02.690479+0000 mon.a (mon.0) 521 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-23T00:39:04.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:03.573111+0000 mon.a (mon.0) 522 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:04.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:02.120924+0000 mgr.y (mgr.14146) 205 : cluster [DBG] pgmap v196: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:39:04.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:02.655448+0000 mon.a (mon.0) 517 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:04.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:02.656331+0000 mon.a (mon.0) 518 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:39:04.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:02.657262+0000 mon.a (mon.0) 519 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:39:04.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:02.662024+0000 mon.a (mon.0) 520 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:04.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:02.690112+0000 mon.b (mon.1) 22 : audit [INF] from='osd.7 [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-23T00:39:04.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:02.690479+0000 mon.a (mon.0) 521 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-23T00:39:04.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:03.573111+0000 mon.a (mon.0) 522 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:04.446 DEBUG:teuthology.orchestra.run.smithi196:osd.7> sudo journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@osd.7.service 2024-02-23T00:39:04.448 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-02-23T00:39:04.448 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd stat -f json 2024-02-23T00:39:04.477 INFO:journalctl@ceph.osd.7.smithi196.stdout:-- Logs begin at Fri 2024-02-23 00:13:21 UTC. -- 2024-02-23T00:39:04.668 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:03.583031+0000 mon.a (mon.0) 523 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:39:04.669 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:03.664429+0000 mon.a (mon.0) 524 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-23T00:39:04.669 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:03.664535+0000 mon.a (mon.0) 525 : cluster [DBG] osdmap e52: 8 total, 7 up, 8 in 2024-02-23T00:39:04.669 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:03.664702+0000 mon.a (mon.0) 526 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-23T00:39:04.669 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:03.665160+0000 mon.b (mon.1) 23 : audit [INF] from='osd.7 [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-02-23T00:39:04.669 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:03.665476+0000 mon.a (mon.0) 527 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-02-23T00:39:04.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:03.583031+0000 mon.a (mon.0) 523 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:39:04.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:03.664429+0000 mon.a (mon.0) 524 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-23T00:39:04.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:03.664535+0000 mon.a (mon.0) 525 : cluster [DBG] osdmap e52: 8 total, 7 up, 8 in 2024-02-23T00:39:04.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:03.664702+0000 mon.a (mon.0) 526 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-23T00:39:04.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:03.665160+0000 mon.b (mon.1) 23 : audit [INF] from='osd.7 [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-02-23T00:39:04.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:03.665476+0000 mon.a (mon.0) 527 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-02-23T00:39:04.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:03.583031+0000 mon.a (mon.0) 523 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:39:04.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:03.664429+0000 mon.a (mon.0) 524 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-23T00:39:04.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:03.664535+0000 mon.a (mon.0) 525 : cluster [DBG] osdmap e52: 8 total, 7 up, 8 in 2024-02-23T00:39:04.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:03.664702+0000 mon.a (mon.0) 526 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-23T00:39:04.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:03.665160+0000 mon.b (mon.1) 23 : audit [INF] from='osd.7 [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-02-23T00:39:04.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:03.665476+0000 mon.a (mon.0) 527 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-02-23T00:39:04.943 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:39:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:39:04.667+0000 7f18af0a1700 -1 osd.7 0 waiting for initial osdmap 2024-02-23T00:39:04.943 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:39:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:39:04.674+0000 7f18a8817700 -1 osd.7 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-23T00:39:05.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:04.121302+0000 mgr.y (mgr.14146) 206 : cluster [DBG] pgmap v198: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:39:05.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:04.665041+0000 mon.a (mon.0) 528 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-02-23T00:39:05.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:04.665121+0000 mon.a (mon.0) 529 : cluster [DBG] osdmap e53: 8 total, 7 up, 8 in 2024-02-23T00:39:05.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:04.665284+0000 mon.a (mon.0) 530 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-23T00:39:05.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:04.668303+0000 mon.a (mon.0) 531 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-23T00:39:05.839 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:04.121302+0000 mgr.y (mgr.14146) 206 : cluster [DBG] pgmap v198: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:39:05.839 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:04.665041+0000 mon.a (mon.0) 528 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-02-23T00:39:05.839 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:04.665121+0000 mon.a (mon.0) 529 : cluster [DBG] osdmap e53: 8 total, 7 up, 8 in 2024-02-23T00:39:05.839 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:04.665284+0000 mon.a (mon.0) 530 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-23T00:39:05.839 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:04.668303+0000 mon.a (mon.0) 531 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-23T00:39:05.840 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:04.121302+0000 mgr.y (mgr.14146) 206 : cluster [DBG] pgmap v198: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:39:05.840 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:04.665041+0000 mon.a (mon.0) 528 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-02-23T00:39:05.840 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:04.665121+0000 mon.a (mon.0) 529 : cluster [DBG] osdmap e53: 8 total, 7 up, 8 in 2024-02-23T00:39:05.840 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:04.665284+0000 mon.a (mon.0) 530 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-23T00:39:05.840 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:04.668303+0000 mon.a (mon.0) 531 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-23T00:39:06.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:03.696787+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:39:06.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:03.696870+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:39:06.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:05.667142+0000 mon.a (mon.0) 532 : cluster [INF] osd.7 [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] boot 2024-02-23T00:39:06.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:05.667211+0000 mon.a (mon.0) 533 : cluster [DBG] osdmap e54: 8 total, 8 up, 8 in 2024-02-23T00:39:06.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:05.667369+0000 mon.a (mon.0) 534 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-23T00:39:06.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:06.121664+0000 mgr.y (mgr.14146) 207 : cluster [DBG] pgmap v201: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:39:06.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:03.696787+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:39:06.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:03.696870+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:39:06.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:05.667142+0000 mon.a (mon.0) 532 : cluster [INF] osd.7 [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] boot 2024-02-23T00:39:06.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:05.667211+0000 mon.a (mon.0) 533 : cluster [DBG] osdmap e54: 8 total, 8 up, 8 in 2024-02-23T00:39:06.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:05.667369+0000 mon.a (mon.0) 534 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-23T00:39:06.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:06.121664+0000 mgr.y (mgr.14146) 207 : cluster [DBG] pgmap v201: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:39:06.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.786+0000 7f210b23c700 1 -- 172.21.15.47:0/1500992502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21040f7a80 msgr2=0x7f21040f7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:06.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.786+0000 7f210b23c700 1 --2- 172.21.15.47:0/1500992502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21040f7a80 0x7f21040f7ea0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f20f8004870 tx=0x7f20f8009cb0).stop 2024-02-23T00:39:06.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.786+0000 7f210b23c700 1 -- 172.21.15.47:0/1500992502 shutdown_connections 2024-02-23T00:39:06.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.786+0000 7f210b23c700 1 --2- 172.21.15.47:0/1500992502 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21040f9670 0x7f21040fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:06.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.786+0000 7f210b23c700 1 --2- 172.21.15.47:0/1500992502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21040f7a80 0x7f21040f7ea0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:06.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.786+0000 7f210b23c700 1 --2- 172.21.15.47:0/1500992502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21040f8d20 0x7f21040f9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:06.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.786+0000 7f210b23c700 1 -- 172.21.15.47:0/1500992502 >> 172.21.15.47:0/1500992502 conn(0x7f21040f32a0 msgr2=0x7f21040f56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:39:06.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.786+0000 7f210b23c700 1 -- 172.21.15.47:0/1500992502 shutdown_connections 2024-02-23T00:39:06.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.786+0000 7f210b23c700 1 -- 172.21.15.47:0/1500992502 wait complete. 2024-02-23T00:39:06.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.787+0000 7f210b23c700 1 librados: starting msgr at 2024-02-23T00:39:06.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.787+0000 7f210b23c700 1 librados: starting objecter 2024-02-23T00:39:06.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.787+0000 7f210b23c700 1 Processor -- start 2024-02-23T00:39:06.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.787+0000 7f210b23c700 1 -- start start 2024-02-23T00:39:06.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.787+0000 7f210b23c700 1 librados: setting wanted keys 2024-02-23T00:39:06.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.787+0000 7f210b23c700 1 librados: calling monclient init 2024-02-23T00:39:06.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.787+0000 7f210b23c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21040f7a80 0x7f2104107e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:06.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.787+0000 7f210b23c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21040f8d20 0x7f2104108380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:06.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.787+0000 7f210b23c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21040f9670 0x7f210410f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:06.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.787+0000 7f210b23c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210410f950 con 0x7f21040f7a80 2024-02-23T00:39:06.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.787+0000 7f210b23c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210410fa90 con 0x7f21040f9670 2024-02-23T00:39:06.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.787+0000 7f210b23c700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210410fbd0 con 0x7f21040f8d20 2024-02-23T00:39:06.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.787+0000 7f2108fd8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21040f7a80 0x7f2104107e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:06.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.787+0000 7f2108fd8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21040f7a80 0x7f2104107e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46334/0 (socket says 172.21.15.47:46334) 2024-02-23T00:39:06.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.787+0000 7f2103fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21040f8d20 0x7f2104108380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:06.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.787+0000 7f2108fd8700 1 -- 172.21.15.47:0/1138135926 learned_addr learned my addr 172.21.15.47:0/1138135926 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:39:06.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.787+0000 7f21097d9700 1 --2- 172.21.15.47:0/1138135926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21040f9670 0x7f210410f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:06.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.788+0000 7f2108fd8700 1 -- 172.21.15.47:0/1138135926 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21040f8d20 msgr2=0x7f2104108380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:06.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.788+0000 7f2108fd8700 1 --2- 172.21.15.47:0/1138135926 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21040f8d20 0x7f2104108380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:06.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.788+0000 7f2108fd8700 1 -- 172.21.15.47:0/1138135926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21040f9670 msgr2=0x7f210410f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:06.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.788+0000 7f2108fd8700 1 --2- 172.21.15.47:0/1138135926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21040f9670 0x7f210410f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:06.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.788+0000 7f2108fd8700 1 -- 172.21.15.47:0/1138135926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f8005040 con 0x7f21040f7a80 2024-02-23T00:39:06.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.788+0000 7f2103fff700 1 --2- 172.21.15.47:0/1138135926 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21040f8d20 0x7f2104108380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:39:06.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.788+0000 7f2108fd8700 1 --2- 172.21.15.47:0/1138135926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21040f7a80 0x7f2104107e90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f20f800b120 tx=0x7f20f8009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:39:06.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.788+0000 7f2101ffb700 10 client.?.objecter ms_handle_connect 0x7f21040f7a80 2024-02-23T00:39:06.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.788+0000 7f2101ffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:39:06.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.788+0000 7f2101ffb700 1 -- 172.21.15.47:0/1138135926 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f800e030 con 0x7f21040f7a80 2024-02-23T00:39:06.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.788+0000 7f2101ffb700 1 -- 172.21.15.47:0/1138135926 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f20f8013e70 con 0x7f21040f7a80 2024-02-23T00:39:06.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.788+0000 7f2101ffb700 1 -- 172.21.15.47:0/1138135926 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f8018070 con 0x7f21040f7a80 2024-02-23T00:39:06.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.788+0000 7f210b23c700 1 -- 172.21.15.47:0/1138135926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f210410fe50 con 0x7f21040f7a80 2024-02-23T00:39:06.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.788+0000 7f210b23c700 10 client.14400.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:39:06.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.788+0000 7f210b23c700 1 -- 172.21.15.47:0/1138135926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21041101a0 con 0x7f21040f7a80 2024-02-23T00:39:06.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.788+0000 7f210b23c700 1 librados: init done 2024-02-23T00:39:06.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.789+0000 7f210b23c700 1 -- 172.21.15.47:0/1138135926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f210405f360 con 0x7f21040f7a80 2024-02-23T00:39:06.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.790+0000 7f2101ffb700 1 -- 172.21.15.47:0/1138135926 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 14) v1 ==== 77064+0+0 (secure 0 0 0) 0x7f20f8009240 con 0x7f21040f7a80 2024-02-23T00:39:06.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.791+0000 7f2101ffb700 1 --2- 172.21.15.47:0/1138135926 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f20f4062bd0 0x7f20f4065090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:06.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.791+0000 7f2101ffb700 1 -- 172.21.15.47:0/1138135926 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5260+0+0 (secure 0 0 0) 0x7f20f807ddc0 con 0x7f21040f7a80 2024-02-23T00:39:06.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.791+0000 7f2101ffb700 10 client.14400.objecter ms_dispatch 0x7f2104001550 osd_map(55..55 src has 1..55) v4 2024-02-23T00:39:06.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.791+0000 7f2101ffb700 3 client.14400.objecter handle_osd_map got epochs [55,55] > 0 2024-02-23T00:39:06.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.791+0000 7f2101ffb700 3 client.14400.objecter handle_osd_map decoding full epoch 55 2024-02-23T00:39:06.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.791+0000 7f2103fff700 1 --2- 172.21.15.47:0/1138135926 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f20f4062bd0 0x7f20f4065090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:06.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.791+0000 7f2101ffb700 20 client.14400.objecter dump_active .. 0 homeless 2024-02-23T00:39:06.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.791+0000 7f2101ffb700 10 client.14400.objecter ms_handle_connect 0x7f20f4062bd0 2024-02-23T00:39:06.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.791+0000 7f2103fff700 1 --2- 172.21.15.47:0/1138135926 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f20f4062bd0 0x7f20f4065090 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f20f0004fd0 tx=0x7f20f000b040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:39:06.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.795+0000 7f2101ffb700 1 -- 172.21.15.47:0/1138135926 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f20f804e330 con 0x7f21040f7a80 2024-02-23T00:39:06.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:03.696787+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:39:06.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:03.696870+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:39:06.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:05.667142+0000 mon.a (mon.0) 532 : cluster [INF] osd.7 [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] boot 2024-02-23T00:39:06.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:05.667211+0000 mon.a (mon.0) 533 : cluster [DBG] osdmap e54: 8 total, 8 up, 8 in 2024-02-23T00:39:06.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:05.667369+0000 mon.a (mon.0) 534 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-23T00:39:06.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:06.121664+0000 mgr.y (mgr.14146) 207 : cluster [DBG] pgmap v201: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:39:06.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.970+0000 7f210b23c700 1 -- 172.21.15.47:0/1138135926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f21041026f0 con 0x7f21040f7a80 2024-02-23T00:39:06.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.970+0000 7f2101ffb700 1 -- 172.21.15.47:0/1138135926 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v55) v1 ==== 74+0+130 (secure 0 0 0) 0x7f20f8051950 con 0x7f21040f7a80 2024-02-23T00:39:06.972 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:39:06.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.972+0000 7f210b23c700 10 librados: watch_flush enter 2024-02-23T00:39:06.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.972+0000 7f210b23c700 10 librados: watch_flush exit 2024-02-23T00:39:06.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.972+0000 7f210b23c700 20 client.14400.objecter shutdown clearing up homeless session... 2024-02-23T00:39:06.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.972+0000 7f210b23c700 10 client.14400.objecter successfully canceled tick 2024-02-23T00:39:06.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.972+0000 7f210b23c700 1 -- 172.21.15.47:0/1138135926 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f20f4062bd0 msgr2=0x7f20f4065090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:06.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.972+0000 7f210b23c700 1 --2- 172.21.15.47:0/1138135926 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f20f4062bd0 0x7f20f4065090 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f20f0004fd0 tx=0x7f20f000b040).stop 2024-02-23T00:39:06.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.972+0000 7f210b23c700 1 -- 172.21.15.47:0/1138135926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21040f7a80 msgr2=0x7f2104107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:06.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.972+0000 7f210b23c700 1 --2- 172.21.15.47:0/1138135926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21040f7a80 0x7f2104107e90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f20f800b120 tx=0x7f20f8009cb0).stop 2024-02-23T00:39:06.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.972+0000 7f210b23c700 1 -- 172.21.15.47:0/1138135926 shutdown_connections 2024-02-23T00:39:06.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.972+0000 7f210b23c700 1 --2- 172.21.15.47:0/1138135926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21040f9670 0x7f210410f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:06.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.972+0000 7f210b23c700 1 --2- 172.21.15.47:0/1138135926 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f20f4062bd0 0x7f20f4065090 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:06.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.972+0000 7f210b23c700 1 --2- 172.21.15.47:0/1138135926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21040f7a80 0x7f2104107e90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:06.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.972+0000 7f210b23c700 1 --2- 172.21.15.47:0/1138135926 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21040f8d20 0x7f2104108380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:06.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.972+0000 7f210b23c700 1 -- 172.21.15.47:0/1138135926 >> 172.21.15.47:0/1138135926 conn(0x7f21040f32a0 msgr2=0x7f21040fc770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:39:06.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.972+0000 7f210b23c700 1 -- 172.21.15.47:0/1138135926 shutdown_connections 2024-02-23T00:39:06.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.972+0000 7f210b23c700 1 -- 172.21.15.47:0/1138135926 wait complete. 2024-02-23T00:39:06.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:06.972+0000 7f210b23c700 1 librados: shutdown 2024-02-23T00:39:07.675 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:06.672280+0000 mon.a (mon.0) 535 : cluster [DBG] osdmap e55: 8 total, 8 up, 8 in 2024-02-23T00:39:07.676 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:06.971600+0000 mon.a (mon.0) 536 : audit [DBG] from='client.? 172.21.15.47:0/1138135926' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-23T00:39:07.769 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":55,"num_osds":8,"num_up_osds":8,"osd_up_since":1708648745,"num_in_osds":8,"osd_in_since":1708648727,"num_remapped_pgs":0} 2024-02-23T00:39:07.769 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi047 2024-02-23T00:39:07.769 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph orch apply rgw foo.a --placement '1;smithi047=foo.a' 2024-02-23T00:39:07.798 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:06.672280+0000 mon.a (mon.0) 535 : cluster [DBG] osdmap e55: 8 total, 8 up, 8 in 2024-02-23T00:39:07.798 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:06.971600+0000 mon.a (mon.0) 536 : audit [DBG] from='client.? 172.21.15.47:0/1138135926' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-23T00:39:08.056 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:06.672280+0000 mon.a (mon.0) 535 : cluster [DBG] osdmap e55: 8 total, 8 up, 8 in 2024-02-23T00:39:08.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:06.971600+0000 mon.a (mon.0) 536 : audit [DBG] from='client.? 172.21.15.47:0/1138135926' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-23T00:39:08.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:07.672857+0000 mon.a (mon.0) 537 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2024-02-23T00:39:08.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:07.716042+0000 osd.4 (osd.4) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:39:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:07.716098+0000 osd.4 (osd.4) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:39:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:39:07.780783+0000 mgr.y (mgr.14146) 208 : cephadm [INF] Detected new or changed devices on smithi196 2024-02-23T00:39:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:07.785660+0000 mon.a (mon.0) 538 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:07.786687+0000 mon.a (mon.0) 539 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:07.787397+0000 mon.a (mon.0) 540 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:07.788028+0000 mon.a (mon.0) 541 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:07.788748+0000 mon.a (mon.0) 542 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:39:07.789264+0000 mgr.y (mgr.14146) 209 : cephadm [INF] Adjusting osd_memory_target on smithi196 to 4280M 2024-02-23T00:39:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:07.792992+0000 mon.a (mon.0) 543 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:07.793812+0000 mon.a (mon.0) 544 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:39:08.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:07.794454+0000 mon.a (mon.0) 545 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:39:08.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:07.798714+0000 mon.a (mon.0) 546 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:08.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:08.121992+0000 mgr.y (mgr.14146) 210 : cluster [DBG] pgmap v204: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:39:09.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:07.672857+0000 mon.a (mon.0) 537 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2024-02-23T00:39:09.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:07.716042+0000 osd.4 (osd.4) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:39:09.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:07.716098+0000 osd.4 (osd.4) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:39:09.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:39:07.780783+0000 mgr.y (mgr.14146) 208 : cephadm [INF] Detected new or changed devices on smithi196 2024-02-23T00:39:09.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:07.785660+0000 mon.a (mon.0) 538 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:09.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:07.786687+0000 mon.a (mon.0) 539 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:09.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:07.787397+0000 mon.a (mon.0) 540 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:09.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:07.788028+0000 mon.a (mon.0) 541 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:09.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:07.788748+0000 mon.a (mon.0) 542 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:09.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:39:07.789264+0000 mgr.y (mgr.14146) 209 : cephadm [INF] Adjusting osd_memory_target on smithi196 to 4280M 2024-02-23T00:39:09.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:07.792992+0000 mon.a (mon.0) 543 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:09.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:07.793812+0000 mon.a (mon.0) 544 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:39:09.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:07.794454+0000 mon.a (mon.0) 545 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:39:09.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:07.798714+0000 mon.a (mon.0) 546 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:09.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:08.121992+0000 mgr.y (mgr.14146) 210 : cluster [DBG] pgmap v204: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:39:09.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:07.672857+0000 mon.a (mon.0) 537 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2024-02-23T00:39:09.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:07.716042+0000 osd.4 (osd.4) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:39:09.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:07.716098+0000 osd.4 (osd.4) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:39:09.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:39:07.780783+0000 mgr.y (mgr.14146) 208 : cephadm [INF] Detected new or changed devices on smithi196 2024-02-23T00:39:09.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:07.785660+0000 mon.a (mon.0) 538 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:09.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:07.786687+0000 mon.a (mon.0) 539 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:09.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:07.787397+0000 mon.a (mon.0) 540 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:09.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:07.788028+0000 mon.a (mon.0) 541 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:09.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:07.788748+0000 mon.a (mon.0) 542 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:09.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:39:07.789264+0000 mgr.y (mgr.14146) 209 : cephadm [INF] Adjusting osd_memory_target on smithi196 to 4280M 2024-02-23T00:39:09.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:07.792992+0000 mon.a (mon.0) 543 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:09.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:07.793812+0000 mon.a (mon.0) 544 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:39:09.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:07.794454+0000 mon.a (mon.0) 545 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:39:09.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:07.798714+0000 mon.a (mon.0) 546 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:09.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:08.121992+0000 mgr.y (mgr.14146) 210 : cluster [DBG] pgmap v204: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:39:09.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:08.683220+0000 mon.a (mon.0) 547 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2024-02-23T00:39:09.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.951+0000 7fb837df9700 1 -- 172.21.15.196:0/3612516647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8300f7c30 msgr2=0x7fb8300f8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:09.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.951+0000 7fb837df9700 1 --2- 172.21.15.196:0/3612516647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8300f7c30 0x7fb8300f8090 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fb8200048a0 tx=0x7fb820009cb0).stop 2024-02-23T00:39:09.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.951+0000 7fb837df9700 1 -- 172.21.15.196:0/3612516647 shutdown_connections 2024-02-23T00:39:09.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.951+0000 7fb837df9700 1 --2- 172.21.15.196:0/3612516647 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb8300f8580 0x7fb8300fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:39:09.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.952+0000 7fb837df9700 1 --2- 172.21.15.196:0/3612516647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8300f6990 0x7fb8300f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:39:09.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.952+0000 7fb837df9700 1 --2- 172.21.15.196:0/3612516647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8300f7c30 0x7fb8300f8090 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:09.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.952+0000 7fb837df9700 1 -- 172.21.15.196:0/3612516647 >> 172.21.15.196:0/3612516647 conn(0x7fb8300f2190 msgr2=0x7fb8300f45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:39:09.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.952+0000 7fb837df9700 1 -- 172.21.15.196:0/3612516647 shutdown_connections 2024-02-23T00:39:09.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.952+0000 7fb837df9700 1 -- 172.21.15.196:0/3612516647 wait complete. 2024-02-23T00:39:09.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.952+0000 7fb837df9700 1 librados: starting msgr at 2024-02-23T00:39:09.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.952+0000 7fb837df9700 1 librados: starting objecter 2024-02-23T00:39:09.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.952+0000 7fb837df9700 1 Processor -- start 2024-02-23T00:39:09.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.952+0000 7fb837df9700 1 -- start start 2024-02-23T00:39:09.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.952+0000 7fb837df9700 1 librados: setting wanted keys 2024-02-23T00:39:09.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.952+0000 7fb837df9700 1 librados: calling monclient init 2024-02-23T00:39:09.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.953+0000 7fb837df9700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb8300f6990 0x7fb8300614b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:09.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.953+0000 7fb837df9700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8300f7c30 0x7fb83010cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:09.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.953+0000 7fb837df9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8300f8580 0x7fb83010f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:09.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.953+0000 7fb837df9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83010f9d0 con 0x7fb8300f8580 2024-02-23T00:39:09.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.953+0000 7fb837df9700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83010fb40 con 0x7fb8300f7c30 2024-02-23T00:39:09.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.953+0000 7fb837df9700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb83010fd00 con 0x7fb8300f6990 2024-02-23T00:39:09.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.953+0000 7fb835394700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8300f7c30 0x7fb83010cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:09.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.953+0000 7fb835394700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8300f7c30 0x7fb83010cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:48592/0 (socket says 172.21.15.196:48592) 2024-02-23T00:39:09.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.953+0000 7fb835394700 1 -- 172.21.15.196:0/3266082140 learned_addr learned my addr 172.21.15.196:0/3266082140 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-02-23T00:39:09.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.953+0000 7fb836396700 1 --2- 172.21.15.196:0/3266082140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8300f8580 0x7fb83010f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:09.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.953+0000 7fb835b95700 1 --2- 172.21.15.196:0/3266082140 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb8300f6990 0x7fb8300614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:09.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.953+0000 7fb835394700 1 -- 172.21.15.196:0/3266082140 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb8300f6990 msgr2=0x7fb8300614b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:09.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.953+0000 7fb835394700 1 --2- 172.21.15.196:0/3266082140 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb8300f6990 0x7fb8300614b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:09.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.953+0000 7fb835394700 1 -- 172.21.15.196:0/3266082140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8300f8580 msgr2=0x7fb83010f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:09.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.953+0000 7fb835394700 1 --2- 172.21.15.196:0/3266082140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8300f8580 0x7fb83010f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:09.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.953+0000 7fb835394700 1 -- 172.21.15.196:0/3266082140 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb820005040 con 0x7fb8300f7c30 2024-02-23T00:39:09.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.953+0000 7fb835394700 1 --2- 172.21.15.196:0/3266082140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8300f7c30 0x7fb83010cd20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fb8200045a0 tx=0x7fb820009cb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:39:09.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.953+0000 7fb826ffd700 10 client.?.objecter ms_handle_connect 0x7fb8300f7c30 2024-02-23T00:39:09.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.953+0000 7fb826ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:39:09.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.953+0000 7fb826ffd700 1 -- 172.21.15.196:0/3266082140 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82000e030 con 0x7fb8300f7c30 2024-02-23T00:39:09.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.953+0000 7fb826ffd700 1 -- 172.21.15.196:0/3266082140 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fb820009450 con 0x7fb8300f7c30 2024-02-23T00:39:09.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.953+0000 7fb826ffd700 1 -- 172.21.15.196:0/3266082140 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb820018070 con 0x7fb8300f7c30 2024-02-23T00:39:09.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.953+0000 7fb837df9700 1 -- 172.21.15.196:0/3266082140 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb83010ff80 con 0x7fb8300f7c30 2024-02-23T00:39:09.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.953+0000 7fb837df9700 10 client.24316.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:39:09.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.953+0000 7fb837df9700 1 -- 172.21.15.196:0/3266082140 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb830110330 con 0x7fb8300f7c30 2024-02-23T00:39:09.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.953+0000 7fb837df9700 1 librados: init done 2024-02-23T00:39:09.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.954+0000 7fb837df9700 1 -- 172.21.15.196:0/3266082140 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb83005f360 con 0x7fb8300f7c30 2024-02-23T00:39:09.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.955+0000 7fb826ffd700 1 -- 172.21.15.196:0/3266082140 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 14) v1 ==== 77064+0+0 (secure 0 0 0) 0x7fb82001d420 con 0x7fb8300f7c30 2024-02-23T00:39:09.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.955+0000 7fb826ffd700 1 --2- 172.21.15.196:0/3266082140 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fb81c062bd0 0x7fb81c065090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:09.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.955+0000 7fb826ffd700 1 -- 172.21.15.196:0/3266082140 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5260+0+0 (secure 0 0 0) 0x7fb82007da20 con 0x7fb8300f7c30 2024-02-23T00:39:09.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.955+0000 7fb826ffd700 10 client.24316.objecter ms_dispatch 0x7fb830001550 osd_map(57..57 src has 1..57) v4 2024-02-23T00:39:10.000 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.955+0000 7fb826ffd700 3 client.24316.objecter handle_osd_map got epochs [57,57] > 0 2024-02-23T00:39:10.000 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.955+0000 7fb826ffd700 3 client.24316.objecter handle_osd_map decoding full epoch 57 2024-02-23T00:39:10.000 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.955+0000 7fb826ffd700 20 client.24316.objecter dump_active .. 0 homeless 2024-02-23T00:39:10.000 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.955+0000 7fb835b95700 1 --2- 172.21.15.196:0/3266082140 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fb81c062bd0 0x7fb81c065090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:10.000 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.956+0000 7fb826ffd700 10 client.24316.objecter ms_handle_connect 0x7fb81c062bd0 2024-02-23T00:39:10.000 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.956+0000 7fb835b95700 1 --2- 172.21.15.196:0/3266082140 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fb81c062bd0 0x7fb81c065090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fb8300f7a90 tx=0x7fb82c008040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:39:10.000 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:09.960+0000 7fb826ffd700 1 -- 172.21.15.196:0/3266082140 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb82004dff0 con 0x7fb8300f7c30 2024-02-23T00:39:10.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:08.683220+0000 mon.a (mon.0) 547 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2024-02-23T00:39:10.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:08.683220+0000 mon.a (mon.0) 547 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2024-02-23T00:39:10.134 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:10.133+0000 7fb837df9700 1 -- 172.21.15.196:0/3266082140 --> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] -- mgr_command(tid 0: {"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi047=foo.a", "target": ["mon-mgr", ""]}) v1 -- 0x7fb830059120 con 0x7fb81c062bd0 2024-02-23T00:39:10.141 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:10.140+0000 7fb826ffd700 1 -- 172.21.15.196:0/3266082140 <== mgr.14146 v2:172.21.15.47:6800/622994871 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+30 (secure 0 0 0) 0x7fb830059120 con 0x7fb81c062bd0 2024-02-23T00:39:10.141 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled rgw.foo.a update... 2024-02-23T00:39:10.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:10.142+0000 7fb837df9700 10 librados: watch_flush enter 2024-02-23T00:39:10.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:10.142+0000 7fb837df9700 10 librados: watch_flush exit 2024-02-23T00:39:10.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:10.142+0000 7fb837df9700 20 client.24316.objecter shutdown clearing up homeless session... 2024-02-23T00:39:10.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:10.142+0000 7fb837df9700 10 client.24316.objecter successfully canceled tick 2024-02-23T00:39:10.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:10.142+0000 7fb837df9700 1 -- 172.21.15.196:0/3266082140 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fb81c062bd0 msgr2=0x7fb81c065090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:10.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:10.142+0000 7fb837df9700 1 --2- 172.21.15.196:0/3266082140 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fb81c062bd0 0x7fb81c065090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fb8300f7a90 tx=0x7fb82c008040).stop 2024-02-23T00:39:10.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:10.142+0000 7fb837df9700 1 -- 172.21.15.196:0/3266082140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8300f7c30 msgr2=0x7fb83010cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:10.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:10.142+0000 7fb837df9700 1 --2- 172.21.15.196:0/3266082140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8300f7c30 0x7fb83010cd20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fb8200045a0 tx=0x7fb820009cb0).stop 2024-02-23T00:39:10.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:10.142+0000 7fb837df9700 1 -- 172.21.15.196:0/3266082140 shutdown_connections 2024-02-23T00:39:10.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:10.142+0000 7fb837df9700 1 --2- 172.21.15.196:0/3266082140 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fb81c062bd0 0x7fb81c065090 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:10.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:10.142+0000 7fb837df9700 1 --2- 172.21.15.196:0/3266082140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8300f8580 0x7fb83010f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:10.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:10.142+0000 7fb837df9700 1 --2- 172.21.15.196:0/3266082140 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb8300f6990 0x7fb8300614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:10.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:10.142+0000 7fb837df9700 1 --2- 172.21.15.196:0/3266082140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8300f7c30 0x7fb83010cd20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:10.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:10.142+0000 7fb837df9700 1 -- 172.21.15.196:0/3266082140 >> 172.21.15.196:0/3266082140 conn(0x7fb8300f2190 msgr2=0x7fb8300fb680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:39:10.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:10.142+0000 7fb837df9700 1 -- 172.21.15.196:0/3266082140 shutdown_connections 2024-02-23T00:39:10.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:10.142+0000 7fb837df9700 1 -- 172.21.15.196:0/3266082140 wait complete. 2024-02-23T00:39:10.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:10.142+0000 7fb837df9700 1 librados: shutdown 2024-02-23T00:39:10.921 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:09.901412+0000 osd.5 (osd.5) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:39:10.921 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:09.901472+0000 osd.5 (osd.5) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:39:10.921 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:10.122396+0000 mgr.y (mgr.14146) 211 : cluster [DBG] pgmap v206: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:39:10.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:10.134410+0000 mgr.y (mgr.14146) 212 : audit [DBG] from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi047=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:39:10.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:39:10.136005+0000 mgr.y (mgr.14146) 213 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi047=foo.a;count:1 2024-02-23T00:39:10.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:10.140004+0000 mon.a (mon.0) 548 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:10.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:10.170767+0000 mon.a (mon.0) 549 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:39:10.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:10.172561+0000 mon.a (mon.0) 550 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:39:10.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:10.173424+0000 mon.a (mon.0) 551 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:39:10.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:10.178660+0000 mon.a (mon.0) 552 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:10.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:39:10.181354+0000 mgr.y (mgr.14146) 214 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi047=foo.a;count:1 2024-02-23T00:39:10.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:10.184381+0000 mon.a (mon.0) 553 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:10.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:10.185054+0000 mon.a (mon.0) 554 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-23T00:39:10.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:10.188134+0000 mon.a (mon.0) 555 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-23T00:39:10.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:10.192430+0000 mon.a (mon.0) 556 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:10.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:10.193318+0000 mon.a (mon.0) 557 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:39:10.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:39:10.194052+0000 mgr.y (mgr.14146) 215 : cephadm [INF] Deploying daemon rgw.foo.a on smithi047 2024-02-23T00:39:10.941 DEBUG:teuthology.orchestra.run.smithi047:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@rgw.foo.a.service 2024-02-23T00:39:10.943 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi196 2024-02-23T00:39:10.943 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd pool create iscsi 3 3 replicated 2024-02-23T00:39:10.962 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:09.901412+0000 osd.5 (osd.5) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:39:10.963 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:09.901472+0000 osd.5 (osd.5) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:39:10.963 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:10.122396+0000 mgr.y (mgr.14146) 211 : cluster [DBG] pgmap v206: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:39:10.963 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:10.134410+0000 mgr.y (mgr.14146) 212 : audit [DBG] from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi047=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:39:10.963 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:39:10.136005+0000 mgr.y (mgr.14146) 213 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi047=foo.a;count:1 2024-02-23T00:39:10.963 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:10.140004+0000 mon.a (mon.0) 548 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:10.963 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:10.170767+0000 mon.a (mon.0) 549 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:39:10.963 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:10.172561+0000 mon.a (mon.0) 550 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:39:10.963 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:10.173424+0000 mon.a (mon.0) 551 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:39:10.963 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:10.178660+0000 mon.a (mon.0) 552 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:10.963 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:39:10.181354+0000 mgr.y (mgr.14146) 214 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi047=foo.a;count:1 2024-02-23T00:39:10.963 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:10.184381+0000 mon.a (mon.0) 553 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:10.964 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:10.185054+0000 mon.a (mon.0) 554 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-23T00:39:10.964 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:10.188134+0000 mon.a (mon.0) 555 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-23T00:39:10.964 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:10.192430+0000 mon.a (mon.0) 556 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:10.964 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:10.193318+0000 mon.a (mon.0) 557 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:39:10.965 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:39:10.194052+0000 mgr.y (mgr.14146) 215 : cephadm [INF] Deploying daemon rgw.foo.a on smithi047 2024-02-23T00:39:10.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:09.901412+0000 osd.5 (osd.5) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:39:10.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:09.901472+0000 osd.5 (osd.5) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:39:10.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:10.122396+0000 mgr.y (mgr.14146) 211 : cluster [DBG] pgmap v206: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:39:10.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:10.134410+0000 mgr.y (mgr.14146) 212 : audit [DBG] from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi047=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:39:10.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:39:10.136005+0000 mgr.y (mgr.14146) 213 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi047=foo.a;count:1 2024-02-23T00:39:10.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:10.140004+0000 mon.a (mon.0) 548 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:10.966 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:10.170767+0000 mon.a (mon.0) 549 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:39:10.966 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:10.172561+0000 mon.a (mon.0) 550 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:39:10.966 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:10.173424+0000 mon.a (mon.0) 551 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:39:10.966 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:10.178660+0000 mon.a (mon.0) 552 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:10.966 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:39:10.181354+0000 mgr.y (mgr.14146) 214 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi047=foo.a;count:1 2024-02-23T00:39:10.966 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:10.184381+0000 mon.a (mon.0) 553 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:10.966 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:10.185054+0000 mon.a (mon.0) 554 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-23T00:39:10.966 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:10.188134+0000 mon.a (mon.0) 555 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-23T00:39:10.966 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:10.192430+0000 mon.a (mon.0) 556 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:10.966 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:10.193318+0000 mon.a (mon.0) 557 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:39:10.966 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:39:10.194052+0000 mgr.y (mgr.14146) 215 : cephadm [INF] Deploying daemon rgw.foo.a on smithi047 2024-02-23T00:39:10.975 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:-- Logs begin at Fri 2024-02-23 00:13:17 UTC. -- 2024-02-23T00:39:11.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:10.690304+0000 mon.a (mon.0) 558 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2024-02-23T00:39:11.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:10.690304+0000 mon.a (mon.0) 558 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2024-02-23T00:39:11.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:10.690304+0000 mon.a (mon.0) 558 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2024-02-23T00:39:12.199 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Feb 23 00:39:12 smithi047 systemd[1]: Starting Ceph rgw.foo.a for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T00:39:12.869 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:12.122717+0000 mgr.y (mgr.14146) 216 : cluster [DBG] pgmap v208: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:39:13.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:12.122717+0000 mgr.y (mgr.14146) 216 : cluster [DBG] pgmap v208: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:39:13.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:12.122717+0000 mgr.y (mgr.14146) 216 : cluster [DBG] pgmap v208: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:39:13.057 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Feb 23 00:39:12 smithi047 bash[64727]: aac63f6c46a1dfcbefeb8646f5aca5ed7993c9fe8b8d550e4fb2044924b1505d 2024-02-23T00:39:13.057 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Feb 23 00:39:12 smithi047 systemd[1]: Started Ceph rgw.foo.a for fce2617e-d1e2-11ee-95c0-87774f69a715. 2024-02-23T00:39:13.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.134+0000 7f1693eaf700 1 -- 172.21.15.196:0/4149004130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f168c0f7c30 msgr2=0x7f168c0f8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:13.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.134+0000 7f1693eaf700 1 --2- 172.21.15.196:0/4149004130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f168c0f7c30 0x7f168c0f8090 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f16880048a0 tx=0x7f1688009cb0).stop 2024-02-23T00:39:13.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.134+0000 7f1693eaf700 1 -- 172.21.15.196:0/4149004130 shutdown_connections 2024-02-23T00:39:13.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.134+0000 7f1693eaf700 1 --2- 172.21.15.196:0/4149004130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f168c0f8580 0x7f168c0fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:13.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.134+0000 7f1693eaf700 1 --2- 172.21.15.196:0/4149004130 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f168c0f6990 0x7f168c0f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:13.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.134+0000 7f1693eaf700 1 --2- 172.21.15.196:0/4149004130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f168c0f7c30 0x7f168c0f8090 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:13.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.134+0000 7f1693eaf700 1 -- 172.21.15.196:0/4149004130 >> 172.21.15.196:0/4149004130 conn(0x7f168c0f2190 msgr2=0x7f168c0f45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:39:13.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.134+0000 7f1693eaf700 1 -- 172.21.15.196:0/4149004130 shutdown_connections 2024-02-23T00:39:13.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.134+0000 7f1693eaf700 1 -- 172.21.15.196:0/4149004130 wait complete. 2024-02-23T00:39:13.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.134+0000 7f1693eaf700 1 librados: starting msgr at 2024-02-23T00:39:13.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.134+0000 7f1693eaf700 1 librados: starting objecter 2024-02-23T00:39:13.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.135+0000 7f1693eaf700 1 Processor -- start 2024-02-23T00:39:13.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.135+0000 7f1693eaf700 1 -- start start 2024-02-23T00:39:13.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.135+0000 7f1693eaf700 1 librados: setting wanted keys 2024-02-23T00:39:13.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.135+0000 7f1693eaf700 1 librados: calling monclient init 2024-02-23T00:39:13.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.135+0000 7f1693eaf700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f168c0f6990 0x7f168c061490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:13.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.135+0000 7f1693eaf700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f168c0f7c30 0x7f168c10cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:13.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.135+0000 7f1693eaf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f168c0f8580 0x7f168c10f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:13.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.135+0000 7f1693eaf700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168c10f8a0 con 0x7f168c0f8580 2024-02-23T00:39:13.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.135+0000 7f1693eaf700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168c10fa10 con 0x7f168c0f7c30 2024-02-23T00:39:13.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.135+0000 7f1693eaf700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168c10fbd0 con 0x7f168c0f6990 2024-02-23T00:39:13.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.135+0000 7f169144a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f168c0f7c30 0x7f168c10cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:13.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.135+0000 7f169144a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f168c0f7c30 0x7f168c10cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:55682/0 (socket says 172.21.15.196:55682) 2024-02-23T00:39:13.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.135+0000 7f169144a700 1 -- 172.21.15.196:0/3924536705 learned_addr learned my addr 172.21.15.196:0/3924536705 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-02-23T00:39:13.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.135+0000 7f1691c4b700 1 --2- 172.21.15.196:0/3924536705 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f168c0f6990 0x7f168c061490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:13.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.135+0000 7f169244c700 1 --2- 172.21.15.196:0/3924536705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f168c0f8580 0x7f168c10f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:13.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.135+0000 7f169144a700 1 -- 172.21.15.196:0/3924536705 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f168c0f6990 msgr2=0x7f168c061490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:13.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.135+0000 7f169144a700 1 --2- 172.21.15.196:0/3924536705 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f168c0f6990 0x7f168c061490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:13.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.135+0000 7f169144a700 1 -- 172.21.15.196:0/3924536705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f168c0f8580 msgr2=0x7f168c10f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:13.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.135+0000 7f169144a700 1 --2- 172.21.15.196:0/3924536705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f168c0f8580 0x7f168c10f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:13.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.135+0000 7f169144a700 1 -- 172.21.15.196:0/3924536705 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1688005040 con 0x7f168c0f7c30 2024-02-23T00:39:13.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.135+0000 7f169244c700 1 --2- 172.21.15.196:0/3924536705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f168c0f8580 0x7f168c10f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:39:13.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.135+0000 7f169144a700 1 --2- 172.21.15.196:0/3924536705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f168c0f7c30 0x7f168c10cca0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f1688004870 tx=0x7f1688009cb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:39:13.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.135+0000 7f1682ffd700 10 client.?.objecter ms_handle_connect 0x7f168c0f7c30 2024-02-23T00:39:13.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.135+0000 7f1682ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:39:13.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.136+0000 7f1682ffd700 1 -- 172.21.15.196:0/3924536705 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168800e070 con 0x7f168c0f7c30 2024-02-23T00:39:13.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.136+0000 7f1682ffd700 1 -- 172.21.15.196:0/3924536705 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f1688004100 con 0x7f168c0f7c30 2024-02-23T00:39:13.145 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.136+0000 7f1682ffd700 1 -- 172.21.15.196:0/3924536705 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1688018070 con 0x7f168c0f7c30 2024-02-23T00:39:13.145 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.136+0000 7f1693eaf700 1 -- 172.21.15.196:0/3924536705 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f168c10fe50 con 0x7f168c0f7c30 2024-02-23T00:39:13.145 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.136+0000 7f1693eaf700 10 client.24325.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:39:13.145 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.136+0000 7f1693eaf700 1 -- 172.21.15.196:0/3924536705 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f168c110250 con 0x7f168c0f7c30 2024-02-23T00:39:13.145 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.136+0000 7f1693eaf700 1 librados: init done 2024-02-23T00:39:13.145 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.137+0000 7f1693eaf700 1 -- 172.21.15.196:0/3924536705 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f168c05f360 con 0x7f168c0f7c30 2024-02-23T00:39:13.145 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.137+0000 7f1682ffd700 1 -- 172.21.15.196:0/3924536705 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 14) v1 ==== 77064+0+0 (secure 0 0 0) 0x7f168801d420 con 0x7f168c0f7c30 2024-02-23T00:39:13.145 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.137+0000 7f1682ffd700 1 --2- 172.21.15.196:0/3924536705 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f1678062bd0 0x7f1678065090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:13.145 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.137+0000 7f1682ffd700 1 -- 172.21.15.196:0/3924536705 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5260+0+0 (secure 0 0 0) 0x7f168807d040 con 0x7f168c0f7c30 2024-02-23T00:39:13.145 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.137+0000 7f1682ffd700 10 client.24325.objecter ms_dispatch 0x7f168c001550 osd_map(58..58 src has 1..58) v4 2024-02-23T00:39:13.145 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.137+0000 7f1682ffd700 3 client.24325.objecter handle_osd_map got epochs [58,58] > 0 2024-02-23T00:39:13.145 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.137+0000 7f1682ffd700 3 client.24325.objecter handle_osd_map decoding full epoch 58 2024-02-23T00:39:13.145 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.138+0000 7f1682ffd700 20 client.24325.objecter dump_active .. 0 homeless 2024-02-23T00:39:13.145 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.138+0000 7f1691c4b700 1 --2- 172.21.15.196:0/3924536705 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f1678062bd0 0x7f1678065090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:13.145 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.138+0000 7f1682ffd700 10 client.24325.objecter ms_handle_connect 0x7f1678062bd0 2024-02-23T00:39:13.146 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.138+0000 7f1691c4b700 1 --2- 172.21.15.196:0/3924536705 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f1678062bd0 0x7f1678065090 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f168c0f7a90 tx=0x7f167c00b040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:39:13.146 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.142+0000 7f1682ffd700 1 -- 172.21.15.196:0/3924536705 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f168804d6b0 con 0x7f168c0f7c30 2024-02-23T00:39:13.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.304+0000 7f1693eaf700 1 -- 172.21.15.196:0/3924536705 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} v 0) v1 -- 0x7f168c101580 con 0x7f168c0f7c30 2024-02-23T00:39:13.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.765+0000 7f1682ffd700 1 -- 172.21.15.196:0/3924536705 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]=0 pool 'iscsi' created v59) v1 ==== 154+0+0 (secure 0 0 0) 0x7f1688022020 con 0x7f168c0f7c30 2024-02-23T00:39:13.773 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.769+0000 7f1693eaf700 10 librados: watch_flush enter 2024-02-23T00:39:13.773 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.769+0000 7f1693eaf700 10 librados: watch_flush exit 2024-02-23T00:39:13.773 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.769+0000 7f1693eaf700 20 client.24325.objecter shutdown clearing up homeless session... 2024-02-23T00:39:13.773 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.769+0000 7f1693eaf700 10 client.24325.objecter successfully canceled tick 2024-02-23T00:39:13.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.770+0000 7f1693eaf700 1 -- 172.21.15.196:0/3924536705 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f1678062bd0 msgr2=0x7f1678065090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:13.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.770+0000 7f1693eaf700 1 --2- 172.21.15.196:0/3924536705 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f1678062bd0 0x7f1678065090 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f168c0f7a90 tx=0x7f167c00b040).stop 2024-02-23T00:39:13.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.770+0000 7f1693eaf700 1 -- 172.21.15.196:0/3924536705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f168c0f7c30 msgr2=0x7f168c10cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:13.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.770+0000 7f1693eaf700 1 --2- 172.21.15.196:0/3924536705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f168c0f7c30 0x7f168c10cca0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f1688004870 tx=0x7f1688009cb0).stop 2024-02-23T00:39:13.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.771+0000 7f1693eaf700 1 -- 172.21.15.196:0/3924536705 shutdown_connections 2024-02-23T00:39:13.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.771+0000 7f1693eaf700 1 --2- 172.21.15.196:0/3924536705 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f1678062bd0 0x7f1678065090 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:13.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.771+0000 7f1693eaf700 1 --2- 172.21.15.196:0/3924536705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f168c0f8580 0x7f168c10f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:13.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.771+0000 7f1693eaf700 1 --2- 172.21.15.196:0/3924536705 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f168c0f6990 0x7f168c061490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:13.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.771+0000 7f1693eaf700 1 --2- 172.21.15.196:0/3924536705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f168c0f7c30 0x7f168c10cca0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:13.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.771+0000 7f1693eaf700 1 -- 172.21.15.196:0/3924536705 >> 172.21.15.196:0/3924536705 conn(0x7f168c0f2190 msgr2=0x7f168c0fb680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:39:13.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.771+0000 7f1693eaf700 1 -- 172.21.15.196:0/3924536705 shutdown_connections 2024-02-23T00:39:13.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.771+0000 7f1693eaf700 1 -- 172.21.15.196:0/3924536705 wait complete. 2024-02-23T00:39:13.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:13.771+0000 7f1693eaf700 1 librados: shutdown 2024-02-23T00:39:13.775 INFO:teuthology.orchestra.run.smithi196.stderr:pool 'iscsi' created 2024-02-23T00:39:14.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:13.281219+0000 mon.a (mon.0) 559 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:14.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:13.282366+0000 mon.a (mon.0) 560 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:39:14.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:13.306142+0000 mon.b (mon.1) 24 : audit [INF] from='client.? 172.21.15.196:0/3924536705' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-23T00:39:14.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:13.306512+0000 mon.a (mon.0) 561 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-23T00:39:14.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:13.766171+0000 mon.a (mon.0) 562 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-23T00:39:14.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:13.766294+0000 mon.a (mon.0) 563 : cluster [DBG] osdmap e59: 8 total, 8 up, 8 in 2024-02-23T00:39:14.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:13.768909+0000 mon.c (mon.2) 12 : audit [INF] from='client.? 172.21.15.47:0/3723202188' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-23T00:39:14.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:13.769435+0000 mon.a (mon.0) 564 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-23T00:39:14.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:13.281219+0000 mon.a (mon.0) 559 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:14.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:13.282366+0000 mon.a (mon.0) 560 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:39:14.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:13.306142+0000 mon.b (mon.1) 24 : audit [INF] from='client.? 172.21.15.196:0/3924536705' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-23T00:39:14.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:13.306512+0000 mon.a (mon.0) 561 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-23T00:39:14.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:13.766171+0000 mon.a (mon.0) 562 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-23T00:39:14.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:13.766294+0000 mon.a (mon.0) 563 : cluster [DBG] osdmap e59: 8 total, 8 up, 8 in 2024-02-23T00:39:14.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:13.768909+0000 mon.c (mon.2) 12 : audit [INF] from='client.? 172.21.15.47:0/3723202188' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-23T00:39:14.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:13.769435+0000 mon.a (mon.0) 564 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-23T00:39:14.615 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:13.281219+0000 mon.a (mon.0) 559 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:14.615 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:13.282366+0000 mon.a (mon.0) 560 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:39:14.615 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:13.306142+0000 mon.b (mon.1) 24 : audit [INF] from='client.? 172.21.15.196:0/3924536705' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-23T00:39:14.615 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:13.306512+0000 mon.a (mon.0) 561 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-23T00:39:14.615 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:13.766171+0000 mon.a (mon.0) 562 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-23T00:39:14.615 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:13.766294+0000 mon.a (mon.0) 563 : cluster [DBG] osdmap e59: 8 total, 8 up, 8 in 2024-02-23T00:39:14.616 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:13.768909+0000 mon.c (mon.2) 12 : audit [INF] from='client.? 172.21.15.47:0/3723202188' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-23T00:39:14.616 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:13.769435+0000 mon.a (mon.0) 564 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-23T00:39:14.638 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd pool application enable iscsi rbd 2024-02-23T00:39:15.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:14.123070+0000 mgr.y (mgr.14146) 217 : cluster [DBG] pgmap v210: 36 pgs: 35 unknown, 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:39:15.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:14.692461+0000 mon.a (mon.0) 565 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:15.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:14.693158+0000 mon.a (mon.0) 566 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:39:15.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:14.693752+0000 mon.a (mon.0) 567 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:39:15.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:14.767354+0000 mon.a (mon.0) 568 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-23T00:39:15.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:14.767485+0000 mon.a (mon.0) 569 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2024-02-23T00:39:16.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:14.123070+0000 mgr.y (mgr.14146) 217 : cluster [DBG] pgmap v210: 36 pgs: 35 unknown, 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:39:16.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:14.692461+0000 mon.a (mon.0) 565 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:16.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:14.693158+0000 mon.a (mon.0) 566 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:39:16.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:14.693752+0000 mon.a (mon.0) 567 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:39:16.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:14.767354+0000 mon.a (mon.0) 568 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-23T00:39:16.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:14.767485+0000 mon.a (mon.0) 569 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2024-02-23T00:39:16.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:14.123070+0000 mgr.y (mgr.14146) 217 : cluster [DBG] pgmap v210: 36 pgs: 35 unknown, 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:39:16.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:14.692461+0000 mon.a (mon.0) 565 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:16.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:14.693158+0000 mon.a (mon.0) 566 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:39:16.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:14.693752+0000 mon.a (mon.0) 567 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:39:16.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:14.767354+0000 mon.a (mon.0) 568 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-23T00:39:16.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:14.767485+0000 mon.a (mon.0) 569 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2024-02-23T00:39:16.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.849+0000 7ff8729c8700 1 -- 172.21.15.196:0/1697094640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff86c0f7c30 msgr2=0x7ff86c0f8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:16.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.849+0000 7ff8729c8700 1 --2- 172.21.15.196:0/1697094640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff86c0f7c30 0x7ff86c0f8090 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7ff8540048a0 tx=0x7ff854009cb0).stop 2024-02-23T00:39:16.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.849+0000 7ff8729c8700 1 -- 172.21.15.196:0/1697094640 shutdown_connections 2024-02-23T00:39:16.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.849+0000 7ff8729c8700 1 --2- 172.21.15.196:0/1697094640 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff86c0f8580 0x7ff86c0fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:16.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.849+0000 7ff8729c8700 1 --2- 172.21.15.196:0/1697094640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff86c0f6990 0x7ff86c0f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:16.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.849+0000 7ff8729c8700 1 --2- 172.21.15.196:0/1697094640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff86c0f7c30 0x7ff86c0f8090 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:16.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.849+0000 7ff8729c8700 1 -- 172.21.15.196:0/1697094640 >> 172.21.15.196:0/1697094640 conn(0x7ff86c0f2190 msgr2=0x7ff86c0f45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:39:16.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.849+0000 7ff8729c8700 1 -- 172.21.15.196:0/1697094640 shutdown_connections 2024-02-23T00:39:16.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.849+0000 7ff8729c8700 1 -- 172.21.15.196:0/1697094640 wait complete. 2024-02-23T00:39:16.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.849+0000 7ff8729c8700 1 librados: starting msgr at 2024-02-23T00:39:16.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.849+0000 7ff8729c8700 1 librados: starting objecter 2024-02-23T00:39:16.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.850+0000 7ff8729c8700 1 Processor -- start 2024-02-23T00:39:16.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.850+0000 7ff8729c8700 1 -- start start 2024-02-23T00:39:16.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.850+0000 7ff8729c8700 1 librados: setting wanted keys 2024-02-23T00:39:16.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.850+0000 7ff8729c8700 1 librados: calling monclient init 2024-02-23T00:39:16.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.850+0000 7ff8729c8700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff86c0f6990 0x7ff86c0614b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:16.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.850+0000 7ff8729c8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff86c0f7c30 0x7ff86c10cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:16.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.850+0000 7ff8729c8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff86c0f8580 0x7ff86c10f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:16.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.850+0000 7ff8729c8700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c10f790 con 0x7ff86c0f8580 2024-02-23T00:39:16.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.850+0000 7ff8729c8700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c10f900 con 0x7ff86c0f7c30 2024-02-23T00:39:16.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.850+0000 7ff8729c8700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c10fac0 con 0x7ff86c0f6990 2024-02-23T00:39:16.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.850+0000 7ff863fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff86c0f7c30 0x7ff86c10cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:16.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.850+0000 7ff863fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff86c0f7c30 0x7ff86c10cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:55696/0 (socket says 172.21.15.196:55696) 2024-02-23T00:39:16.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.850+0000 7ff863fff700 1 -- 172.21.15.196:0/1592258808 learned_addr learned my addr 172.21.15.196:0/1592258808 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-02-23T00:39:16.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.850+0000 7ff86bfff700 1 --2- 172.21.15.196:0/1592258808 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff86c0f6990 0x7ff86c0614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:16.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.850+0000 7ff870f65700 1 --2- 172.21.15.196:0/1592258808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff86c0f8580 0x7ff86c10f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:16.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.850+0000 7ff863fff700 1 -- 172.21.15.196:0/1592258808 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff86c0f6990 msgr2=0x7ff86c0614b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:16.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.850+0000 7ff863fff700 1 --2- 172.21.15.196:0/1592258808 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff86c0f6990 0x7ff86c0614b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:16.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.850+0000 7ff863fff700 1 -- 172.21.15.196:0/1592258808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff86c0f8580 msgr2=0x7ff86c10f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:16.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.850+0000 7ff863fff700 1 --2- 172.21.15.196:0/1592258808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff86c0f8580 0x7ff86c10f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:16.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.850+0000 7ff863fff700 1 -- 172.21.15.196:0/1592258808 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff854005040 con 0x7ff86c0f7c30 2024-02-23T00:39:16.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.850+0000 7ff863fff700 1 --2- 172.21.15.196:0/1592258808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff86c0f7c30 0x7ff86c10cb70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7ff8540045a0 tx=0x7ff854009cb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:39:16.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.850+0000 7ff869ffb700 10 client.?.objecter ms_handle_connect 0x7ff86c0f7c30 2024-02-23T00:39:16.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.850+0000 7ff869ffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:39:16.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.851+0000 7ff869ffb700 1 -- 172.21.15.196:0/1592258808 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85400e030 con 0x7ff86c0f7c30 2024-02-23T00:39:16.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.851+0000 7ff869ffb700 1 -- 172.21.15.196:0/1592258808 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7ff8540096d0 con 0x7ff86c0f7c30 2024-02-23T00:39:16.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.851+0000 7ff8729c8700 1 -- 172.21.15.196:0/1592258808 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff86c10fd40 con 0x7ff86c0f7c30 2024-02-23T00:39:16.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.851+0000 7ff869ffb700 1 -- 172.21.15.196:0/1592258808 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff854018070 con 0x7ff86c0f7c30 2024-02-23T00:39:16.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.851+0000 7ff8729c8700 10 client.24355.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:39:16.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.851+0000 7ff8729c8700 1 -- 172.21.15.196:0/1592258808 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff86c1101d0 con 0x7ff86c0f7c30 2024-02-23T00:39:16.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.851+0000 7ff8729c8700 1 librados: init done 2024-02-23T00:39:16.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.851+0000 7ff8729c8700 1 -- 172.21.15.196:0/1592258808 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff86c05f360 con 0x7ff86c0f7c30 2024-02-23T00:39:16.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.852+0000 7ff869ffb700 1 -- 172.21.15.196:0/1592258808 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 14) v1 ==== 77064+0+0 (secure 0 0 0) 0x7ff85401d420 con 0x7ff86c0f7c30 2024-02-23T00:39:16.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.852+0000 7ff869ffb700 1 --2- 172.21.15.196:0/1592258808 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7ff858062bd0 0x7ff858065090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:16.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.852+0000 7ff869ffb700 1 -- 172.21.15.196:0/1592258808 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff85407dca0 con 0x7ff86c0f7c30 2024-02-23T00:39:16.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.852+0000 7ff869ffb700 10 client.24355.objecter ms_dispatch 0x7ff86c001550 osd_map(62..62 src has 1..62) v4 2024-02-23T00:39:16.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.852+0000 7ff869ffb700 3 client.24355.objecter handle_osd_map got epochs [62,62] > 0 2024-02-23T00:39:16.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.852+0000 7ff869ffb700 3 client.24355.objecter handle_osd_map decoding full epoch 62 2024-02-23T00:39:16.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.852+0000 7ff86bfff700 1 --2- 172.21.15.196:0/1592258808 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7ff858062bd0 0x7ff858065090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:16.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.852+0000 7ff869ffb700 20 client.24355.objecter dump_active .. 0 homeless 2024-02-23T00:39:16.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.853+0000 7ff86bfff700 1 --2- 172.21.15.196:0/1592258808 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7ff858062bd0 0x7ff858065090 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7ff86c0f7a90 tx=0x7ff85c00b040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:39:16.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.854+0000 7ff869ffb700 10 client.24355.objecter ms_handle_connect 0x7ff858062bd0 2024-02-23T00:39:16.861 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:16.859+0000 7ff869ffb700 1 -- 172.21.15.196:0/1592258808 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ff85404de40 con 0x7ff86c0f7c30 2024-02-23T00:39:16.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:39:14.697726+0000 mgr.y (mgr.14146) 218 : cephadm [INF] Checking dashboard <-> RGW credentials 2024-02-23T00:39:16.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:15.242329+0000 osd.0 (osd.0) 7 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:39:16.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:15.242366+0000 osd.0 (osd.0) 8 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:39:16.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:15.768700+0000 mon.a (mon.0) 570 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2024-02-23T00:39:16.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:15.771011+0000 mon.c (mon.2) 13 : audit [INF] from='client.? 172.21.15.47:0/3723202188' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-23T00:39:16.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:15.771376+0000 mon.a (mon.0) 571 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-23T00:39:16.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:15.772922+0000 mon.c (mon.2) 14 : audit [INF] from='client.? 172.21.15.47:0/752447959' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-23T00:39:16.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:15.773337+0000 mon.a (mon.0) 572 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-23T00:39:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:16.123547+0000 mgr.y (mgr.14146) 219 : cluster [DBG] pgmap v213: 68 pgs: 18 active+clean, 10 creating+peering, 40 unknown; 873 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:39:17.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:17.024+0000 7ff8729c8700 1 -- 172.21.15.196:0/1592258808 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"} v 0) v1 -- 0x7ff86c0fe420 con 0x7ff86c0f7c30 2024-02-23T00:39:17.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:39:14.697726+0000 mgr.y (mgr.14146) 218 : cephadm [INF] Checking dashboard <-> RGW credentials 2024-02-23T00:39:17.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:15.242329+0000 osd.0 (osd.0) 7 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:39:17.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:15.242366+0000 osd.0 (osd.0) 8 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:39:17.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:15.768700+0000 mon.a (mon.0) 570 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2024-02-23T00:39:17.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:15.771011+0000 mon.c (mon.2) 13 : audit [INF] from='client.? 172.21.15.47:0/3723202188' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-23T00:39:17.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:15.771376+0000 mon.a (mon.0) 571 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-23T00:39:17.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:15.772922+0000 mon.c (mon.2) 14 : audit [INF] from='client.? 172.21.15.47:0/752447959' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-23T00:39:17.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:15.773337+0000 mon.a (mon.0) 572 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-23T00:39:17.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:16.123547+0000 mgr.y (mgr.14146) 219 : cluster [DBG] pgmap v213: 68 pgs: 18 active+clean, 10 creating+peering, 40 unknown; 873 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:39:17.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:39:14.697726+0000 mgr.y (mgr.14146) 218 : cephadm [INF] Checking dashboard <-> RGW credentials 2024-02-23T00:39:17.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:15.242329+0000 osd.0 (osd.0) 7 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:39:17.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:15.242366+0000 osd.0 (osd.0) 8 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:39:17.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:15.768700+0000 mon.a (mon.0) 570 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2024-02-23T00:39:17.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:15.771011+0000 mon.c (mon.2) 13 : audit [INF] from='client.? 172.21.15.47:0/3723202188' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-23T00:39:17.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:15.771376+0000 mon.a (mon.0) 571 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-23T00:39:17.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:15.772922+0000 mon.c (mon.2) 14 : audit [INF] from='client.? 172.21.15.47:0/752447959' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-23T00:39:17.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:15.773337+0000 mon.a (mon.0) 572 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-23T00:39:17.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:16.123547+0000 mgr.y (mgr.14146) 219 : cluster [DBG] pgmap v213: 68 pgs: 18 active+clean, 10 creating+peering, 40 unknown; 873 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:39:17.786 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:17.783+0000 7ff869ffb700 1 -- 172.21.15.196:0/1592258808 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]=0 enabled application 'rbd' on pool 'iscsi' v63) v1 ==== 147+0+0 (secure 0 0 0) 0x7ff854051460 con 0x7ff86c0f7c30 2024-02-23T00:39:17.789 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:17.786+0000 7ff8729c8700 10 librados: watch_flush enter 2024-02-23T00:39:17.789 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:17.787+0000 7ff8729c8700 10 librados: watch_flush exit 2024-02-23T00:39:17.789 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:17.787+0000 7ff8729c8700 20 client.24355.objecter shutdown clearing up homeless session... 2024-02-23T00:39:17.789 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:17.787+0000 7ff8729c8700 10 client.24355.objecter successfully canceled tick 2024-02-23T00:39:17.789 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:17.787+0000 7ff8729c8700 1 -- 172.21.15.196:0/1592258808 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7ff858062bd0 msgr2=0x7ff858065090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:17.789 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:17.787+0000 7ff8729c8700 1 --2- 172.21.15.196:0/1592258808 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7ff858062bd0 0x7ff858065090 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7ff86c0f7a90 tx=0x7ff85c00b040).stop 2024-02-23T00:39:17.789 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:17.787+0000 7ff8729c8700 1 -- 172.21.15.196:0/1592258808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff86c0f7c30 msgr2=0x7ff86c10cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:17.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:17.787+0000 7ff8729c8700 1 --2- 172.21.15.196:0/1592258808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff86c0f7c30 0x7ff86c10cb70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7ff8540045a0 tx=0x7ff854009cb0).stop 2024-02-23T00:39:17.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:17.787+0000 7ff8729c8700 1 -- 172.21.15.196:0/1592258808 shutdown_connections 2024-02-23T00:39:17.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:17.787+0000 7ff8729c8700 1 --2- 172.21.15.196:0/1592258808 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7ff858062bd0 0x7ff858065090 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:17.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:17.787+0000 7ff8729c8700 1 --2- 172.21.15.196:0/1592258808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff86c0f8580 0x7ff86c10f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:17.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:17.787+0000 7ff8729c8700 1 --2- 172.21.15.196:0/1592258808 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff86c0f6990 0x7ff86c0614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:17.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:17.787+0000 7ff8729c8700 1 --2- 172.21.15.196:0/1592258808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff86c0f7c30 0x7ff86c10cb70 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:17.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:17.787+0000 7ff8729c8700 1 -- 172.21.15.196:0/1592258808 >> 172.21.15.196:0/1592258808 conn(0x7ff86c0f2190 msgr2=0x7ff86c0fb680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:39:17.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:17.787+0000 7ff8729c8700 1 -- 172.21.15.196:0/1592258808 shutdown_connections 2024-02-23T00:39:17.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:17.787+0000 7ff8729c8700 1 -- 172.21.15.196:0/1592258808 wait complete. 2024-02-23T00:39:17.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:17.787+0000 7ff8729c8700 1 librados: shutdown 2024-02-23T00:39:17.790 INFO:teuthology.orchestra.run.smithi196.stderr:enabled application 'rbd' on pool 'iscsi' 2024-02-23T00:39:18.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:16.773396+0000 mon.a (mon.0) 573 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-23T00:39:18.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:16.773519+0000 mon.a (mon.0) 574 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-23T00:39:18.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:16.773629+0000 mon.a (mon.0) 575 : cluster [DBG] osdmap e62: 8 total, 8 up, 8 in 2024-02-23T00:39:18.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:17.026067+0000 mon.b (mon.1) 25 : audit [INF] from='client.? 172.21.15.196:0/1592258808' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-23T00:39:18.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:17.654675+0000 mon.a (mon.0) 576 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-23T00:39:18.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:17.657435+0000 mon.a (mon.0) 577 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:18.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:16.773396+0000 mon.a (mon.0) 573 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-23T00:39:18.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:16.773519+0000 mon.a (mon.0) 574 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-23T00:39:18.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:16.773629+0000 mon.a (mon.0) 575 : cluster [DBG] osdmap e62: 8 total, 8 up, 8 in 2024-02-23T00:39:18.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:17.026067+0000 mon.b (mon.1) 25 : audit [INF] from='client.? 172.21.15.196:0/1592258808' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-23T00:39:18.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:17.654675+0000 mon.a (mon.0) 576 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-23T00:39:18.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:17.657435+0000 mon.a (mon.0) 577 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:18.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:16.773396+0000 mon.a (mon.0) 573 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-23T00:39:18.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:16.773519+0000 mon.a (mon.0) 574 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-23T00:39:18.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:16.773629+0000 mon.a (mon.0) 575 : cluster [DBG] osdmap e62: 8 total, 8 up, 8 in 2024-02-23T00:39:18.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:17.026067+0000 mon.b (mon.1) 25 : audit [INF] from='client.? 172.21.15.196:0/1592258808' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-23T00:39:18.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:17.654675+0000 mon.a (mon.0) 576 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-23T00:39:18.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:17.657435+0000 mon.a (mon.0) 577 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:18.589 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph orch apply iscsi iscsi user password --placement '1;smithi196=iscsi.a' 2024-02-23T00:39:18.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:17.777407+0000 mon.a (mon.0) 578 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]': finished 2024-02-23T00:39:18.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:17.777488+0000 mon.a (mon.0) 579 : cluster [DBG] osdmap e63: 8 total, 8 up, 8 in 2024-02-23T00:39:18.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:17.780439+0000 mon.c (mon.2) 15 : audit [INF] from='client.? 172.21.15.47:0/3723202188' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-23T00:39:18.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:17.781188+0000 mon.a (mon.0) 580 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-23T00:39:18.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:17.783302+0000 mon.c (mon.2) 16 : audit [INF] from='client.? 172.21.15.47:0/752447959' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-23T00:39:18.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:17.783770+0000 mon.a (mon.0) 581 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-23T00:39:18.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:18.124135+0000 mgr.y (mgr.14146) 220 : cluster [DBG] pgmap v216: 100 pgs: 33 active+clean, 13 creating+peering, 54 unknown; 1.2 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-23T00:39:19.056 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:17.777407+0000 mon.a (mon.0) 578 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]': finished 2024-02-23T00:39:19.056 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:17.777488+0000 mon.a (mon.0) 579 : cluster [DBG] osdmap e63: 8 total, 8 up, 8 in 2024-02-23T00:39:19.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:17.780439+0000 mon.c (mon.2) 15 : audit [INF] from='client.? 172.21.15.47:0/3723202188' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-23T00:39:19.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:17.781188+0000 mon.a (mon.0) 580 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-23T00:39:19.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:17.783302+0000 mon.c (mon.2) 16 : audit [INF] from='client.? 172.21.15.47:0/752447959' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-23T00:39:19.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:17.783770+0000 mon.a (mon.0) 581 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-23T00:39:19.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:18.124135+0000 mgr.y (mgr.14146) 220 : cluster [DBG] pgmap v216: 100 pgs: 33 active+clean, 13 creating+peering, 54 unknown; 1.2 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-23T00:39:19.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:17.777407+0000 mon.a (mon.0) 578 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]': finished 2024-02-23T00:39:19.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:17.777488+0000 mon.a (mon.0) 579 : cluster [DBG] osdmap e63: 8 total, 8 up, 8 in 2024-02-23T00:39:19.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:17.780439+0000 mon.c (mon.2) 15 : audit [INF] from='client.? 172.21.15.47:0/3723202188' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-23T00:39:19.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:17.781188+0000 mon.a (mon.0) 580 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-23T00:39:19.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:17.783302+0000 mon.c (mon.2) 16 : audit [INF] from='client.? 172.21.15.47:0/752447959' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-23T00:39:19.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:17.783770+0000 mon.a (mon.0) 581 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-23T00:39:19.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:18.124135+0000 mgr.y (mgr.14146) 220 : cluster [DBG] pgmap v216: 100 pgs: 33 active+clean, 13 creating+peering, 54 unknown; 1.2 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-23T00:39:20.056 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:18.784840+0000 mon.a (mon.0) 582 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-23T00:39:20.056 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:18.784998+0000 mon.a (mon.0) 583 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-23T00:39:20.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:18.785167+0000 mon.a (mon.0) 584 : cluster [DBG] osdmap e64: 8 total, 8 up, 8 in 2024-02-23T00:39:20.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:18.784840+0000 mon.a (mon.0) 582 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-23T00:39:20.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:18.784998+0000 mon.a (mon.0) 583 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-23T00:39:20.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:18.785167+0000 mon.a (mon.0) 584 : cluster [DBG] osdmap e64: 8 total, 8 up, 8 in 2024-02-23T00:39:20.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:18.784840+0000 mon.a (mon.0) 582 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-23T00:39:20.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:18.784998+0000 mon.a (mon.0) 583 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-23T00:39:20.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:18.785167+0000 mon.a (mon.0) 584 : cluster [DBG] osdmap e64: 8 total, 8 up, 8 in 2024-02-23T00:39:20.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.856+0000 7fbb1ebab700 1 -- 172.21.15.196:0/453374749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb18061670 msgr2=0x7fbb180fa6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:20.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.856+0000 7fbb1ebab700 1 --2- 172.21.15.196:0/453374749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb18061670 0x7fbb180fa6d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fbb0c004870 tx=0x7fbb0c009cb0).stop 2024-02-23T00:39:20.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.856+0000 7fbb1ebab700 1 -- 172.21.15.196:0/453374749 shutdown_connections 2024-02-23T00:39:20.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.856+0000 7fbb1ebab700 1 --2- 172.21.15.196:0/453374749 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbb180fd680 0x7fbb180ffb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:39:20.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.856+0000 7fbb1ebab700 1 --2- 172.21.15.196:0/453374749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb18061670 0x7fbb180fa6d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:20.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.856+0000 7fbb1ebab700 1 --2- 172.21.15.196:0/453374749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb180fac50 0x7fbb180fd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:39:20.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.856+0000 7fbb1ebab700 1 -- 172.21.15.196:0/453374749 >> 172.21.15.196:0/453374749 conn(0x7fbb180f2190 msgr2=0x7fbb180f45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:39:20.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.856+0000 7fbb1ebab700 1 -- 172.21.15.196:0/453374749 shutdown_connections 2024-02-23T00:39:20.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.856+0000 7fbb1ebab700 1 -- 172.21.15.196:0/453374749 wait complete. 2024-02-23T00:39:20.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.856+0000 7fbb1ebab700 1 librados: starting msgr at 2024-02-23T00:39:20.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.856+0000 7fbb1ebab700 1 librados: starting objecter 2024-02-23T00:39:20.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.856+0000 7fbb1ebab700 1 Processor -- start 2024-02-23T00:39:20.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.856+0000 7fbb1ebab700 1 -- start start 2024-02-23T00:39:20.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.856+0000 7fbb1ebab700 1 librados: setting wanted keys 2024-02-23T00:39:20.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.856+0000 7fbb1ebab700 1 librados: calling monclient init 2024-02-23T00:39:20.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.857+0000 7fbb1ebab700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb18061670 0x7fbb18108160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:20.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.857+0000 7fbb1ebab700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb180fac50 0x7fbb18108650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:20.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.857+0000 7fbb1ebab700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbb180fd680 0x7fbb1810f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:20.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.857+0000 7fbb1ebab700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1810fbd0 con 0x7fbb18061670 2024-02-23T00:39:20.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.857+0000 7fbb1ebab700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1810fd40 con 0x7fbb180fac50 2024-02-23T00:39:20.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.857+0000 7fbb1ebab700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1810ff00 con 0x7fbb180fd680 2024-02-23T00:39:20.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.857+0000 7fbb17fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb180fac50 0x7fbb18108650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:20.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.857+0000 7fbb17fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb180fac50 0x7fbb18108650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43080/0 (socket says 172.21.15.196:43080) 2024-02-23T00:39:20.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.857+0000 7fbb17fff700 1 -- 172.21.15.196:0/3205740869 learned_addr learned my addr 172.21.15.196:0/3205740869 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-02-23T00:39:20.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.857+0000 7fbb1c947700 1 --2- 172.21.15.196:0/3205740869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb18061670 0x7fbb18108160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:20.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.857+0000 7fbb1d148700 1 --2- 172.21.15.196:0/3205740869 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbb180fd680 0x7fbb1810f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:20.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.857+0000 7fbb17fff700 1 -- 172.21.15.196:0/3205740869 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbb180fd680 msgr2=0x7fbb1810f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:20.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.857+0000 7fbb17fff700 1 --2- 172.21.15.196:0/3205740869 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbb180fd680 0x7fbb1810f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:20.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.857+0000 7fbb17fff700 1 -- 172.21.15.196:0/3205740869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb18061670 msgr2=0x7fbb18108160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:20.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.857+0000 7fbb17fff700 1 --2- 172.21.15.196:0/3205740869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb18061670 0x7fbb18108160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:20.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.857+0000 7fbb17fff700 1 -- 172.21.15.196:0/3205740869 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb0c005040 con 0x7fbb180fac50 2024-02-23T00:39:20.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.857+0000 7fbb1c947700 1 --2- 172.21.15.196:0/3205740869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb18061670 0x7fbb18108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-23T00:39:20.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.857+0000 7fbb17fff700 1 --2- 172.21.15.196:0/3205740869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb180fac50 0x7fbb18108650 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fbb0400b500 tx=0x7fbb040090f0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:39:20.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.857+0000 7fbb15ffb700 10 client.?.objecter ms_handle_connect 0x7fbb180fac50 2024-02-23T00:39:20.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.857+0000 7fbb15ffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:39:20.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.858+0000 7fbb15ffb700 1 -- 172.21.15.196:0/3205740869 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0400d4a0 con 0x7fbb180fac50 2024-02-23T00:39:20.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.858+0000 7fbb15ffb700 1 -- 172.21.15.196:0/3205740869 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fbb0400da90 con 0x7fbb180fac50 2024-02-23T00:39:20.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.858+0000 7fbb1ebab700 1 -- 172.21.15.196:0/3205740869 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb181101e0 con 0x7fbb180fac50 2024-02-23T00:39:20.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.858+0000 7fbb15ffb700 1 -- 172.21.15.196:0/3205740869 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb040129c0 con 0x7fbb180fac50 2024-02-23T00:39:20.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.858+0000 7fbb1ebab700 10 client.24367.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:39:20.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.858+0000 7fbb1ebab700 1 -- 172.21.15.196:0/3205740869 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb18110720 con 0x7fbb180fac50 2024-02-23T00:39:20.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.858+0000 7fbb1ebab700 1 librados: init done 2024-02-23T00:39:20.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.859+0000 7fbb1ebab700 1 -- 172.21.15.196:0/3205740869 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1805f360 con 0x7fbb180fac50 2024-02-23T00:39:20.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.860+0000 7fbb15ffb700 1 -- 172.21.15.196:0/3205740869 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 14) v1 ==== 77064+0+0 (secure 0 0 0) 0x7fbb0400d600 con 0x7fbb180fac50 2024-02-23T00:39:20.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.860+0000 7fbb15ffb700 1 --2- 172.21.15.196:0/3205740869 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fbb08062bd0 0x7fbb08065090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:20.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.860+0000 7fbb15ffb700 1 -- 172.21.15.196:0/3205740869 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7074+0+0 (secure 0 0 0) 0x7fbb0407d2d0 con 0x7fbb180fac50 2024-02-23T00:39:20.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.860+0000 7fbb15ffb700 10 client.24367.objecter ms_dispatch 0x7fbb18001550 osd_map(66..66 src has 1..66) v4 2024-02-23T00:39:20.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.860+0000 7fbb15ffb700 3 client.24367.objecter handle_osd_map got epochs [66,66] > 0 2024-02-23T00:39:20.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.860+0000 7fbb15ffb700 3 client.24367.objecter handle_osd_map decoding full epoch 66 2024-02-23T00:39:20.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.860+0000 7fbb15ffb700 20 client.24367.objecter dump_active .. 0 homeless 2024-02-23T00:39:20.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.860+0000 7fbb1c947700 1 --2- 172.21.15.196:0/3205740869 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fbb08062bd0 0x7fbb08065090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:20.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.861+0000 7fbb1c947700 1 --2- 172.21.15.196:0/3205740869 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fbb08062bd0 0x7fbb08065090 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fbb0c009a00 tx=0x7fbb0c009cb0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:39:20.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.861+0000 7fbb15ffb700 10 client.24367.objecter ms_handle_connect 0x7fbb08062bd0 2024-02-23T00:39:20.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:20.864+0000 7fbb15ffb700 1 -- 172.21.15.196:0/3205740869 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fbb0404d220 con 0x7fbb180fac50 2024-02-23T00:39:20.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:19.784630+0000 mon.a (mon.0) 585 : cluster [DBG] osdmap e65: 8 total, 8 up, 8 in 2024-02-23T00:39:20.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:19.787951+0000 mon.a (mon.0) 586 : audit [INF] from='client.? 172.21.15.47:0/1636546524' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-23T00:39:20.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:19.793190+0000 mon.a (mon.0) 587 : audit [INF] from='client.? 172.21.15.47:0/1573276039' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-23T00:39:20.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:20.124505+0000 mgr.y (mgr.14146) 221 : cluster [DBG] pgmap v219: 132 pgs: 70 active+clean, 24 creating+peering, 38 unknown; 1.4 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-23T00:39:20.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:20.786887+0000 mon.a (mon.0) 588 : audit [INF] from='client.? 172.21.15.47:0/1636546524' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-23T00:39:20.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:20.787041+0000 mon.a (mon.0) 589 : audit [INF] from='client.? 172.21.15.47:0/1573276039' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-23T00:39:20.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:20.787154+0000 mon.a (mon.0) 590 : cluster [DBG] osdmap e66: 8 total, 8 up, 8 in 2024-02-23T00:39:20.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:20.789031+0000 mon.a (mon.0) 591 : audit [INF] from='client.? 172.21.15.47:0/1636546524' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-23T00:39:21.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:21.039+0000 7fbb1ebab700 1 -- 172.21.15.196:0/3205740869 --> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] -- mgr_command(tid 0: {"prefix": "orch apply iscsi", "pool": "iscsi", "api_user": "user", "api_password": "password", "placement": "1;smithi196=iscsi.a", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb180f9140 con 0x7fbb08062bd0 2024-02-23T00:39:21.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:21.048+0000 7fbb15ffb700 1 -- 172.21.15.196:0/3205740869 <== mgr.14146 v2:172.21.15.47:6800/622994871 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fbb0c020030 con 0x7fbb08062bd0 2024-02-23T00:39:21.050 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled iscsi.iscsi update... 2024-02-23T00:39:21.052 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:21.050+0000 7fbb1ebab700 10 librados: watch_flush enter 2024-02-23T00:39:21.052 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:21.050+0000 7fbb1ebab700 10 librados: watch_flush exit 2024-02-23T00:39:21.052 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:21.050+0000 7fbb1ebab700 20 client.24367.objecter shutdown clearing up homeless session... 2024-02-23T00:39:21.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:21.050+0000 7fbb1ebab700 10 client.24367.objecter successfully canceled tick 2024-02-23T00:39:21.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:21.050+0000 7fbb1ebab700 1 -- 172.21.15.196:0/3205740869 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fbb08062bd0 msgr2=0x7fbb08065090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:21.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:21.050+0000 7fbb1ebab700 1 --2- 172.21.15.196:0/3205740869 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fbb08062bd0 0x7fbb08065090 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fbb0c009a00 tx=0x7fbb0c009cb0).stop 2024-02-23T00:39:21.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:21.050+0000 7fbb1ebab700 1 -- 172.21.15.196:0/3205740869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb180fac50 msgr2=0x7fbb18108650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:21.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:21.050+0000 7fbb1ebab700 1 --2- 172.21.15.196:0/3205740869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb180fac50 0x7fbb18108650 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fbb0400b500 tx=0x7fbb040090f0).stop 2024-02-23T00:39:21.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:21.050+0000 7fbb1ebab700 1 -- 172.21.15.196:0/3205740869 shutdown_connections 2024-02-23T00:39:21.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:21.050+0000 7fbb1ebab700 1 --2- 172.21.15.196:0/3205740869 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbb180fd680 0x7fbb1810f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:21.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:21.050+0000 7fbb1ebab700 1 --2- 172.21.15.196:0/3205740869 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fbb08062bd0 0x7fbb08065090 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:21.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:21.050+0000 7fbb1ebab700 1 --2- 172.21.15.196:0/3205740869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb18061670 0x7fbb18108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:21.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:21.050+0000 7fbb1ebab700 1 --2- 172.21.15.196:0/3205740869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb180fac50 0x7fbb18108650 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:21.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:21.050+0000 7fbb1ebab700 1 -- 172.21.15.196:0/3205740869 >> 172.21.15.196:0/3205740869 conn(0x7fbb180f2190 msgr2=0x7fbb180f8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:39:21.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:21.050+0000 7fbb1ebab700 1 -- 172.21.15.196:0/3205740869 shutdown_connections 2024-02-23T00:39:21.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:21.050+0000 7fbb1ebab700 1 -- 172.21.15.196:0/3205740869 wait complete. 2024-02-23T00:39:21.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:21.050+0000 7fbb1ebab700 1 librados: shutdown 2024-02-23T00:39:21.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:19.784630+0000 mon.a (mon.0) 585 : cluster [DBG] osdmap e65: 8 total, 8 up, 8 in 2024-02-23T00:39:21.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:19.787951+0000 mon.a (mon.0) 586 : audit [INF] from='client.? 172.21.15.47:0/1636546524' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-23T00:39:21.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:19.793190+0000 mon.a (mon.0) 587 : audit [INF] from='client.? 172.21.15.47:0/1573276039' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-23T00:39:21.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:20.124505+0000 mgr.y (mgr.14146) 221 : cluster [DBG] pgmap v219: 132 pgs: 70 active+clean, 24 creating+peering, 38 unknown; 1.4 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-23T00:39:21.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:20.786887+0000 mon.a (mon.0) 588 : audit [INF] from='client.? 172.21.15.47:0/1636546524' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-23T00:39:21.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:20.787041+0000 mon.a (mon.0) 589 : audit [INF] from='client.? 172.21.15.47:0/1573276039' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-23T00:39:21.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:20.787154+0000 mon.a (mon.0) 590 : cluster [DBG] osdmap e66: 8 total, 8 up, 8 in 2024-02-23T00:39:21.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:20.789031+0000 mon.a (mon.0) 591 : audit [INF] from='client.? 172.21.15.47:0/1636546524' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-23T00:39:21.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:19.784630+0000 mon.a (mon.0) 585 : cluster [DBG] osdmap e65: 8 total, 8 up, 8 in 2024-02-23T00:39:21.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:19.787951+0000 mon.a (mon.0) 586 : audit [INF] from='client.? 172.21.15.47:0/1636546524' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-23T00:39:21.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:19.793190+0000 mon.a (mon.0) 587 : audit [INF] from='client.? 172.21.15.47:0/1573276039' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-23T00:39:21.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:20.124505+0000 mgr.y (mgr.14146) 221 : cluster [DBG] pgmap v219: 132 pgs: 70 active+clean, 24 creating+peering, 38 unknown; 1.4 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-23T00:39:21.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:20.786887+0000 mon.a (mon.0) 588 : audit [INF] from='client.? 172.21.15.47:0/1636546524' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-23T00:39:21.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:20.787041+0000 mon.a (mon.0) 589 : audit [INF] from='client.? 172.21.15.47:0/1573276039' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-23T00:39:21.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:20.787154+0000 mon.a (mon.0) 590 : cluster [DBG] osdmap e66: 8 total, 8 up, 8 in 2024-02-23T00:39:21.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:20.789031+0000 mon.a (mon.0) 591 : audit [INF] from='client.? 172.21.15.47:0/1636546524' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-23T00:39:21.789 DEBUG:teuthology.orchestra.run.smithi196:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@iscsi.iscsi.a.service 2024-02-23T00:39:21.792 INFO:tasks.cephadm:Adding prometheus.a on smithi196 2024-02-23T00:39:21.792 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph orch apply prometheus '1;smithi196=a' 2024-02-23T00:39:21.826 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:-- Logs begin at Fri 2024-02-23 00:13:21 UTC. -- 2024-02-23T00:39:22.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:20.802204+0000 mon.a (mon.0) 592 : audit [INF] from='client.? 172.21.15.47:0/1573276039' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-23T00:39:22.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:21.041383+0000 mgr.y (mgr.14146) 222 : audit [DBG] from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "iscsi", "api_user": "user", "api_password": "password", "placement": "1;smithi196=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:39:22.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:39:21.043749+0000 mgr.y (mgr.14146) 223 : cephadm [INF] Saving service iscsi.iscsi spec with placement smithi196=iscsi.a;count:1 2024-02-23T00:39:22.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:21.048803+0000 mon.a (mon.0) 593 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:22.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:21.788169+0000 mon.a (mon.0) 594 : audit [INF] from='client.? 172.21.15.47:0/1636546524' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-23T00:39:22.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:21.788273+0000 mon.a (mon.0) 595 : audit [INF] from='client.? 172.21.15.47:0/1573276039' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-23T00:39:22.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:21.788334+0000 mon.a (mon.0) 596 : cluster [DBG] osdmap e67: 8 total, 8 up, 8 in 2024-02-23T00:39:22.306 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:20.802204+0000 mon.a (mon.0) 592 : audit [INF] from='client.? 172.21.15.47:0/1573276039' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-23T00:39:22.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:21.041383+0000 mgr.y (mgr.14146) 222 : audit [DBG] from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "iscsi", "api_user": "user", "api_password": "password", "placement": "1;smithi196=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:39:22.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:39:21.043749+0000 mgr.y (mgr.14146) 223 : cephadm [INF] Saving service iscsi.iscsi spec with placement smithi196=iscsi.a;count:1 2024-02-23T00:39:22.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:21.048803+0000 mon.a (mon.0) 593 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:22.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:21.788169+0000 mon.a (mon.0) 594 : audit [INF] from='client.? 172.21.15.47:0/1636546524' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-23T00:39:22.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:21.788273+0000 mon.a (mon.0) 595 : audit [INF] from='client.? 172.21.15.47:0/1573276039' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-23T00:39:22.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:21.788334+0000 mon.a (mon.0) 596 : cluster [DBG] osdmap e67: 8 total, 8 up, 8 in 2024-02-23T00:39:22.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:20.802204+0000 mon.a (mon.0) 592 : audit [INF] from='client.? 172.21.15.47:0/1573276039' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-23T00:39:22.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:21.041383+0000 mgr.y (mgr.14146) 222 : audit [DBG] from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "iscsi", "api_user": "user", "api_password": "password", "placement": "1;smithi196=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:39:22.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:39:21.043749+0000 mgr.y (mgr.14146) 223 : cephadm [INF] Saving service iscsi.iscsi spec with placement smithi196=iscsi.a;count:1 2024-02-23T00:39:22.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:21.048803+0000 mon.a (mon.0) 593 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:22.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:21.788169+0000 mon.a (mon.0) 594 : audit [INF] from='client.? 172.21.15.47:0/1636546524' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-23T00:39:22.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:21.788273+0000 mon.a (mon.0) 595 : audit [INF] from='client.? 172.21.15.47:0/1573276039' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-23T00:39:22.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:21.788334+0000 mon.a (mon.0) 596 : cluster [DBG] osdmap e67: 8 total, 8 up, 8 in 2024-02-23T00:39:23.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:21.405571+0000 osd.2 (osd.2) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:39:23.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:21.405650+0000 osd.2 (osd.2) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:39:23.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:21.984434+0000 mon.a (mon.0) 597 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:23.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:21.991192+0000 mon.a (mon.0) 598 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:23.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:21.997410+0000 mon.a (mon.0) 599 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:23.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:39:22.002514+0000 mgr.y (mgr.14146) 224 : cephadm [INF] Checking pool "iscsi" exists for service iscsi.iscsi 2024-02-23T00:39:23.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:22.011697+0000 mon.a (mon.0) 600 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-23T00:39:23.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:22.015475+0000 mon.a (mon.0) 601 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-23T00:39:23.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:22.024970+0000 mon.a (mon.0) 602 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:39:23.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:39:22.025800+0000 mgr.y (mgr.14146) 225 : cephadm [INF] Deploying daemon iscsi.iscsi.a on smithi196 2024-02-23T00:39:23.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:22.125050+0000 mgr.y (mgr.14146) 226 : cluster [DBG] pgmap v222: 132 pgs: 94 active+clean, 1 creating+activating, 25 creating+peering, 12 unknown; 1.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 8.2 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-02-23T00:39:23.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:21.405571+0000 osd.2 (osd.2) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:39:23.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:21.405650+0000 osd.2 (osd.2) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:39:23.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:21.984434+0000 mon.a (mon.0) 597 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:23.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:21.991192+0000 mon.a (mon.0) 598 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:23.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:21.997410+0000 mon.a (mon.0) 599 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:23.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:39:22.002514+0000 mgr.y (mgr.14146) 224 : cephadm [INF] Checking pool "iscsi" exists for service iscsi.iscsi 2024-02-23T00:39:23.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:22.011697+0000 mon.a (mon.0) 600 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-23T00:39:23.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:22.015475+0000 mon.a (mon.0) 601 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-23T00:39:23.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:22.024970+0000 mon.a (mon.0) 602 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:39:23.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:39:22.025800+0000 mgr.y (mgr.14146) 225 : cephadm [INF] Deploying daemon iscsi.iscsi.a on smithi196 2024-02-23T00:39:23.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:22.125050+0000 mgr.y (mgr.14146) 226 : cluster [DBG] pgmap v222: 132 pgs: 94 active+clean, 1 creating+activating, 25 creating+peering, 12 unknown; 1.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 8.2 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-02-23T00:39:23.413 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:21.405571+0000 osd.2 (osd.2) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:39:23.413 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:21.405650+0000 osd.2 (osd.2) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:39:23.413 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:21.984434+0000 mon.a (mon.0) 597 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:23.413 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:21.991192+0000 mon.a (mon.0) 598 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:23.414 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:21.997410+0000 mon.a (mon.0) 599 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:23.414 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:39:22.002514+0000 mgr.y (mgr.14146) 224 : cephadm [INF] Checking pool "iscsi" exists for service iscsi.iscsi 2024-02-23T00:39:23.414 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:22.011697+0000 mon.a (mon.0) 600 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-23T00:39:23.414 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:22.015475+0000 mon.a (mon.0) 601 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-23T00:39:23.414 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:22.024970+0000 mon.a (mon.0) 602 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:39:23.414 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:39:22.025800+0000 mgr.y (mgr.14146) 225 : cephadm [INF] Deploying daemon iscsi.iscsi.a on smithi196 2024-02-23T00:39:23.414 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:22.125050+0000 mgr.y (mgr.14146) 226 : cluster [DBG] pgmap v222: 132 pgs: 94 active+clean, 1 creating+activating, 25 creating+peering, 12 unknown; 1.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 8.2 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-02-23T00:39:25.043 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:24.023064+0000 mon.a (mon.0) 603 : cluster [DBG] mgrmap e15: y(active, since 5m), standbys: x 2024-02-23T00:39:25.043 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:24.125688+0000 mgr.y (mgr.14146) 227 : cluster [DBG] pgmap v223: 132 pgs: 118 active+clean, 1 creating+activating, 13 creating+peering; 3.2 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 57 KiB/s rd, 3.6 KiB/s wr, 127 op/s 2024-02-23T00:39:25.043 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:24.306484+0000 mon.a (mon.0) 604 : cluster [DBG] osdmap e68: 8 total, 8 up, 8 in 2024-02-23T00:39:25.043 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:24.477924+0000 osd.7 (osd.7) 3 : cluster [DBG] 1.0 deep-scrub starts 2024-02-23T00:39:25.043 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:24.481046+0000 osd.7 (osd.7) 4 : cluster [DBG] 1.0 deep-scrub ok 2024-02-23T00:39:25.043 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:39:25 smithi196 systemd[1]: Starting Ceph iscsi.iscsi.a for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T00:39:25.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:24.023064+0000 mon.a (mon.0) 603 : cluster [DBG] mgrmap e15: y(active, since 5m), standbys: x 2024-02-23T00:39:25.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:24.125688+0000 mgr.y (mgr.14146) 227 : cluster [DBG] pgmap v223: 132 pgs: 118 active+clean, 1 creating+activating, 13 creating+peering; 3.2 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 57 KiB/s rd, 3.6 KiB/s wr, 127 op/s 2024-02-23T00:39:25.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:24.306484+0000 mon.a (mon.0) 604 : cluster [DBG] osdmap e68: 8 total, 8 up, 8 in 2024-02-23T00:39:25.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:24.477924+0000 osd.7 (osd.7) 3 : cluster [DBG] 1.0 deep-scrub starts 2024-02-23T00:39:25.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:24.481046+0000 osd.7 (osd.7) 4 : cluster [DBG] 1.0 deep-scrub ok 2024-02-23T00:39:25.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:24.023064+0000 mon.a (mon.0) 603 : cluster [DBG] mgrmap e15: y(active, since 5m), standbys: x 2024-02-23T00:39:25.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:24.125688+0000 mgr.y (mgr.14146) 227 : cluster [DBG] pgmap v223: 132 pgs: 118 active+clean, 1 creating+activating, 13 creating+peering; 3.2 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 57 KiB/s rd, 3.6 KiB/s wr, 127 op/s 2024-02-23T00:39:25.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:24.306484+0000 mon.a (mon.0) 604 : cluster [DBG] osdmap e68: 8 total, 8 up, 8 in 2024-02-23T00:39:25.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:24.477924+0000 osd.7 (osd.7) 3 : cluster [DBG] 1.0 deep-scrub starts 2024-02-23T00:39:25.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:24.481046+0000 osd.7 (osd.7) 4 : cluster [DBG] 1.0 deep-scrub ok 2024-02-23T00:39:25.449 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.443+0000 7f6119f37700 1 -- 172.21.15.196:0/3586792350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61140f6110 msgr2=0x7f61140f6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:25.449 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.443+0000 7f6119f37700 1 --2- 172.21.15.196:0/3586792350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61140f6110 0x7f61140f6530 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f60fc0048a0 tx=0x7f60fc009cb0).stop 2024-02-23T00:39:25.449 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.443+0000 7f6119f37700 1 -- 172.21.15.196:0/3586792350 shutdown_connections 2024-02-23T00:39:25.449 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.443+0000 7f6119f37700 1 --2- 172.21.15.196:0/3586792350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61140f7d00 0x7f61140fe600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:25.449 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.443+0000 7f6119f37700 1 --2- 172.21.15.196:0/3586792350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61140f6110 0x7f61140f6530 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:25.450 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.443+0000 7f6119f37700 1 --2- 172.21.15.196:0/3586792350 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f61140f73b0 0x7f61140f7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:25.450 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.443+0000 7f6119f37700 1 -- 172.21.15.196:0/3586792350 >> 172.21.15.196:0/3586792350 conn(0x7f61140f1930 msgr2=0x7f61140f3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:39:25.450 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.443+0000 7f6119f37700 1 -- 172.21.15.196:0/3586792350 shutdown_connections 2024-02-23T00:39:25.450 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.443+0000 7f6119f37700 1 -- 172.21.15.196:0/3586792350 wait complete. 2024-02-23T00:39:25.450 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.444+0000 7f6119f37700 1 librados: starting msgr at 2024-02-23T00:39:25.450 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.444+0000 7f6119f37700 1 librados: starting objecter 2024-02-23T00:39:25.450 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.444+0000 7f6119f37700 1 Processor -- start 2024-02-23T00:39:25.450 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.444+0000 7f6119f37700 1 -- start start 2024-02-23T00:39:25.450 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.444+0000 7f6119f37700 1 librados: setting wanted keys 2024-02-23T00:39:25.450 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.444+0000 7f6119f37700 1 librados: calling monclient init 2024-02-23T00:39:25.450 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.445+0000 7f6119f37700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f61140f6110 0x7f6114107df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:25.450 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.445+0000 7f6119f37700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61140f73b0 0x7f61141082e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:25.450 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.445+0000 7f6119f37700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61140f7d00 0x7f611410f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:25.451 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.445+0000 7f6119f37700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611410f8b0 con 0x7f61140f7d00 2024-02-23T00:39:25.451 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.445+0000 7f6119f37700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611410f9f0 con 0x7f61140f73b0 2024-02-23T00:39:25.451 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.445+0000 7f6119f37700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f611410fb30 con 0x7f61140f6110 2024-02-23T00:39:25.451 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.445+0000 7f6112ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61140f73b0 0x7f61141082e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:25.451 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.445+0000 7f6112ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61140f73b0 0x7f61141082e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43098/0 (socket says 172.21.15.196:43098) 2024-02-23T00:39:25.451 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.445+0000 7f6112ffd700 1 -- 172.21.15.196:0/432001819 learned_addr learned my addr 172.21.15.196:0/432001819 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-02-23T00:39:25.451 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.445+0000 7f61137fe700 1 --2- 172.21.15.196:0/432001819 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f61140f6110 0x7f6114107df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:25.451 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.445+0000 7f6113fff700 1 --2- 172.21.15.196:0/432001819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61140f7d00 0x7f611410f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:25.451 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.445+0000 7f6112ffd700 1 -- 172.21.15.196:0/432001819 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f61140f6110 msgr2=0x7f6114107df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:25.451 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.445+0000 7f6112ffd700 1 --2- 172.21.15.196:0/432001819 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f61140f6110 0x7f6114107df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:25.451 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.445+0000 7f6112ffd700 1 -- 172.21.15.196:0/432001819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61140f7d00 msgr2=0x7f611410f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:25.451 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.445+0000 7f6112ffd700 1 --2- 172.21.15.196:0/432001819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61140f7d00 0x7f611410f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:25.451 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.445+0000 7f6112ffd700 1 -- 172.21.15.196:0/432001819 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60fc005040 con 0x7f61140f73b0 2024-02-23T00:39:25.452 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.445+0000 7f6112ffd700 1 --2- 172.21.15.196:0/432001819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61140f73b0 0x7f61141082e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f61040027f0 tx=0x7f6104002b00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:39:25.452 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.445+0000 7f6110ff9700 10 client.?.objecter ms_handle_connect 0x7f61140f73b0 2024-02-23T00:39:25.452 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.445+0000 7f6110ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:39:25.452 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.445+0000 7f6110ff9700 1 -- 172.21.15.196:0/432001819 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6104007b40 con 0x7f61140f73b0 2024-02-23T00:39:25.452 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.445+0000 7f6110ff9700 1 -- 172.21.15.196:0/432001819 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f610400f040 con 0x7f61140f73b0 2024-02-23T00:39:25.452 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.445+0000 7f6119f37700 1 -- 172.21.15.196:0/432001819 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f611410fe10 con 0x7f61140f73b0 2024-02-23T00:39:25.452 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.445+0000 7f6110ff9700 1 -- 172.21.15.196:0/432001819 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6104014610 con 0x7f61140f73b0 2024-02-23T00:39:25.452 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.445+0000 7f6119f37700 10 client.24388.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:39:25.452 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.445+0000 7f6119f37700 1 -- 172.21.15.196:0/432001819 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61141102c0 con 0x7f61140f73b0 2024-02-23T00:39:25.452 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.445+0000 7f6119f37700 1 librados: init done 2024-02-23T00:39:25.452 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.446+0000 7f6119f37700 1 -- 172.21.15.196:0/432001819 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f611405ffd0 con 0x7f61140f73b0 2024-02-23T00:39:25.452 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.447+0000 7f6110ff9700 1 -- 172.21.15.196:0/432001819 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 15) v1 ==== 77115+0+0 (secure 0 0 0) 0x7f6104009110 con 0x7f61140f73b0 2024-02-23T00:39:25.452 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.447+0000 7f6110ff9700 1 --2- 172.21.15.196:0/432001819 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f6100062c60 0x7f6100065120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:25.453 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.447+0000 7f6110ff9700 1 -- 172.21.15.196:0/432001819 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7090+0+0 (secure 0 0 0) 0x7f610407cae0 con 0x7f61140f73b0 2024-02-23T00:39:25.453 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.447+0000 7f6110ff9700 10 client.24388.objecter ms_dispatch 0x7f6114001550 osd_map(68..68 src has 1..68) v4 2024-02-23T00:39:25.453 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.447+0000 7f6110ff9700 3 client.24388.objecter handle_osd_map got epochs [68,68] > 0 2024-02-23T00:39:25.453 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.447+0000 7f6110ff9700 3 client.24388.objecter handle_osd_map decoding full epoch 68 2024-02-23T00:39:25.453 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.447+0000 7f6110ff9700 20 client.24388.objecter dump_active .. 0 homeless 2024-02-23T00:39:25.453 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.447+0000 7f61137fe700 1 --2- 172.21.15.196:0/432001819 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f6100062c60 0x7f6100065120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:25.453 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.448+0000 7f6110ff9700 10 client.24388.objecter ms_handle_connect 0x7f6100062c60 2024-02-23T00:39:25.453 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.448+0000 7f61137fe700 1 --2- 172.21.15.196:0/432001819 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f6100062c60 0x7f6100065120 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f60fc004ed0 tx=0x7f60fc009cb0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:39:25.454 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.451+0000 7f6110ff9700 1 -- 172.21.15.196:0/432001819 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f610404c870 con 0x7f61140f73b0 2024-02-23T00:39:25.643 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.641+0000 7f6119f37700 1 -- 172.21.15.196:0/432001819 --> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi196=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f61140fdc00 con 0x7f6100062c60 2024-02-23T00:39:25.650 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.648+0000 7f6110ff9700 1 -- 172.21.15.196:0/432001819 <== mgr.14146 v2:172.21.15.47:6800/622994871 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f61140fdc00 con 0x7f6100062c60 2024-02-23T00:39:25.650 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled prometheus update... 2024-02-23T00:39:25.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.650+0000 7f6119f37700 10 librados: watch_flush enter 2024-02-23T00:39:25.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.650+0000 7f6119f37700 10 librados: watch_flush exit 2024-02-23T00:39:25.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.650+0000 7f6119f37700 20 client.24388.objecter shutdown clearing up homeless session... 2024-02-23T00:39:25.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.650+0000 7f6119f37700 10 client.24388.objecter successfully canceled tick 2024-02-23T00:39:25.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.650+0000 7f6119f37700 1 -- 172.21.15.196:0/432001819 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f6100062c60 msgr2=0x7f6100065120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:25.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.650+0000 7f6119f37700 1 --2- 172.21.15.196:0/432001819 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f6100062c60 0x7f6100065120 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f60fc004ed0 tx=0x7f60fc009cb0).stop 2024-02-23T00:39:25.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.650+0000 7f6119f37700 1 -- 172.21.15.196:0/432001819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61140f73b0 msgr2=0x7f61141082e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:25.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.650+0000 7f6119f37700 1 --2- 172.21.15.196:0/432001819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61140f73b0 0x7f61141082e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f61040027f0 tx=0x7f6104002b00).stop 2024-02-23T00:39:25.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.650+0000 7f6119f37700 1 -- 172.21.15.196:0/432001819 shutdown_connections 2024-02-23T00:39:25.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.650+0000 7f6119f37700 1 --2- 172.21.15.196:0/432001819 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f6100062c60 0x7f6100065120 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:25.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.650+0000 7f6119f37700 1 --2- 172.21.15.196:0/432001819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61140f7d00 0x7f611410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:25.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.650+0000 7f6119f37700 1 --2- 172.21.15.196:0/432001819 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f61140f6110 0x7f6114107df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:25.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.650+0000 7f6119f37700 1 --2- 172.21.15.196:0/432001819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61140f73b0 0x7f61141082e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:25.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.650+0000 7f6119f37700 1 -- 172.21.15.196:0/432001819 >> 172.21.15.196:0/432001819 conn(0x7f61140f1930 msgr2=0x7f61140fae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:39:25.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.650+0000 7f6119f37700 1 -- 172.21.15.196:0/432001819 shutdown_connections 2024-02-23T00:39:25.654 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.650+0000 7f6119f37700 1 -- 172.21.15.196:0/432001819 wait complete. 2024-02-23T00:39:25.654 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:25.650+0000 7f6119f37700 1 librados: shutdown 2024-02-23T00:39:26.394 DEBUG:teuthology.orchestra.run.smithi196:prometheus.a> sudo journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@prometheus.a.service 2024-02-23T00:39:26.397 INFO:tasks.cephadm:Adding node-exporter.a on smithi047 2024-02-23T00:39:26.397 INFO:tasks.cephadm:Adding node-exporter.b on smithi196 2024-02-23T00:39:26.397 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph orch apply node-exporter '2;smithi047=a;smithi196=b' 2024-02-23T00:39:26.428 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:-- Logs begin at Fri 2024-02-23 00:13:21 UTC. -- 2024-02-23T00:39:26.943 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:39:26 smithi196 bash[60612]: log file path now is '/var/log/tcmu-runner.log' 2024-02-23T00:39:26.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:25.643039+0000 mgr.y (mgr.14146) 228 : audit [DBG] from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi196=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:39:26.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:39:25.644795+0000 mgr.y (mgr.14146) 229 : cephadm [INF] Saving service prometheus spec with placement smithi196=a;count:1 2024-02-23T00:39:26.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:25.648895+0000 mon.a (mon.0) 605 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:26.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:26.126317+0000 mgr.y (mgr.14146) 230 : cluster [DBG] pgmap v225: 132 pgs: 132 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 103 KiB/s rd, 5.8 KiB/s wr, 231 op/s 2024-02-23T00:39:27.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:25.643039+0000 mgr.y (mgr.14146) 228 : audit [DBG] from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi196=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:39:27.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:39:25.644795+0000 mgr.y (mgr.14146) 229 : cephadm [INF] Saving service prometheus spec with placement smithi196=a;count:1 2024-02-23T00:39:27.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:25.648895+0000 mon.a (mon.0) 605 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:27.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:26.126317+0000 mgr.y (mgr.14146) 230 : cluster [DBG] pgmap v225: 132 pgs: 132 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 103 KiB/s rd, 5.8 KiB/s wr, 231 op/s 2024-02-23T00:39:27.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:25.643039+0000 mgr.y (mgr.14146) 228 : audit [DBG] from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi196=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:39:27.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:39:25.644795+0000 mgr.y (mgr.14146) 229 : cephadm [INF] Saving service prometheus spec with placement smithi196=a;count:1 2024-02-23T00:39:27.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:25.648895+0000 mon.a (mon.0) 605 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:27.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:26.126317+0000 mgr.y (mgr.14146) 230 : cluster [DBG] pgmap v225: 132 pgs: 132 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 103 KiB/s rd, 5.8 KiB/s wr, 231 op/s 2024-02-23T00:39:27.810 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:39:27 smithi196 bash[60712]: cb3bf15d6f76b050dbed4cdde518bfb7b206ebdd2ed26dea61bef85a1abb7172 2024-02-23T00:39:27.810 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:39:27 smithi196 systemd[1]: Started Ceph iscsi.iscsi.a for fce2617e-d1e2-11ee-95c0-87774f69a715. 2024-02-23T00:39:28.107 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:39:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug Started the configuration object watcher 2024-02-23T00:39:28.107 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:39:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug Checking for config object changes every 1s 2024-02-23T00:39:28.107 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:39:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug Processing osd blocklist entries for this node 2024-02-23T00:39:28.443 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:39:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug Reading the configuration object to update local LIO configuration 2024-02-23T00:39:28.443 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:39:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug Configuration does not have an entry for this host(smithi196.front.sepia.ceph.com) - nothing to define to LIO 2024-02-23T00:39:28.443 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:39:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug * Running on http://[::]:5000/ (Press CTRL+C to quit) 2024-02-23T00:39:28.928 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:27.732201+0000 mon.a (mon.0) 606 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:28.929 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:27.733610+0000 mon.a (mon.0) 607 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:39:28.929 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:28.092500+0000 mon.b (mon.1) 26 : audit [DBG] from='client.? 172.21.15.196:0/2892736071' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-23T00:39:28.929 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:28.126849+0000 mgr.y (mgr.14146) 231 : cluster [DBG] pgmap v226: 132 pgs: 132 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 93 KiB/s rd, 4.8 KiB/s wr, 204 op/s 2024-02-23T00:39:29.056 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:27.732201+0000 mon.a (mon.0) 606 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:29.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:27.733610+0000 mon.a (mon.0) 607 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:39:29.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:28.092500+0000 mon.b (mon.1) 26 : audit [DBG] from='client.? 172.21.15.196:0/2892736071' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-23T00:39:29.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:28.126849+0000 mgr.y (mgr.14146) 231 : cluster [DBG] pgmap v226: 132 pgs: 132 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 93 KiB/s rd, 4.8 KiB/s wr, 204 op/s 2024-02-23T00:39:29.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:27.732201+0000 mon.a (mon.0) 606 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:29.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:27.733610+0000 mon.a (mon.0) 607 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:39:29.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:28.092500+0000 mon.b (mon.1) 26 : audit [DBG] from='client.? 172.21.15.196:0/2892736071' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-23T00:39:29.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:28.126849+0000 mgr.y (mgr.14146) 231 : cluster [DBG] pgmap v226: 132 pgs: 132 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 93 KiB/s rd, 4.8 KiB/s wr, 204 op/s 2024-02-23T00:39:29.233 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.230+0000 7fbc45976700 1 -- 172.21.15.196:0/1596108488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc400f8370 msgr2=0x7fbc400feee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:29.233 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.230+0000 7fbc45976700 1 --2- 172.21.15.196:0/1596108488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc400f8370 0x7fbc400feee0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fbc38009be0 tx=0x7fbc38004d10).stop 2024-02-23T00:39:29.233 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.231+0000 7fbc45976700 1 -- 172.21.15.196:0/1596108488 shutdown_connections 2024-02-23T00:39:29.233 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.231+0000 7fbc45976700 1 --2- 172.21.15.196:0/1596108488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc400f8370 0x7fbc400feee0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:29.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.231+0000 7fbc45976700 1 --2- 172.21.15.196:0/1596108488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc400f6950 0x7fbc400f6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:29.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.231+0000 7fbc45976700 1 --2- 172.21.15.196:0/1596108488 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbc400f7260 0x7fbc400f7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:29.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.231+0000 7fbc45976700 1 -- 172.21.15.196:0/1596108488 >> 172.21.15.196:0/1596108488 conn(0x7fbc400f2150 msgr2=0x7fbc400f4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:39:29.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.231+0000 7fbc45976700 1 -- 172.21.15.196:0/1596108488 shutdown_connections 2024-02-23T00:39:29.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.231+0000 7fbc45976700 1 -- 172.21.15.196:0/1596108488 wait complete. 2024-02-23T00:39:29.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.231+0000 7fbc45976700 1 librados: starting msgr at 2024-02-23T00:39:29.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.231+0000 7fbc45976700 1 librados: starting objecter 2024-02-23T00:39:29.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.231+0000 7fbc45976700 1 Processor -- start 2024-02-23T00:39:29.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.232+0000 7fbc45976700 1 -- start start 2024-02-23T00:39:29.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.232+0000 7fbc45976700 1 librados: setting wanted keys 2024-02-23T00:39:29.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.232+0000 7fbc45976700 1 librados: calling monclient init 2024-02-23T00:39:29.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.232+0000 7fbc45976700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc400f6950 0x7fbc40106b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:29.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.232+0000 7fbc45976700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbc400f7260 0x7fbc40107000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:29.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.232+0000 7fbc45976700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc400f8370 0x7fbc4010e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:29.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.232+0000 7fbc45976700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4010e5d0 con 0x7fbc400f6950 2024-02-23T00:39:29.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.232+0000 7fbc45976700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4010e710 con 0x7fbc400f8370 2024-02-23T00:39:29.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.232+0000 7fbc45976700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4010e850 con 0x7fbc400f7260 2024-02-23T00:39:29.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.232+0000 7fbc3f7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc400f8370 0x7fbc4010e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:29.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.232+0000 7fbc3f7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc400f8370 0x7fbc4010e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43176/0 (socket says 172.21.15.196:43176) 2024-02-23T00:39:29.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.232+0000 7fbc3f7fe700 1 -- 172.21.15.196:0/1717633439 learned_addr learned my addr 172.21.15.196:0/1717633439 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-02-23T00:39:29.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.232+0000 7fbc3e7fc700 1 --2- 172.21.15.196:0/1717633439 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbc400f7260 0x7fbc40107000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:29.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.232+0000 7fbc3effd700 1 --2- 172.21.15.196:0/1717633439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc400f6950 0x7fbc40106b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:29.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.232+0000 7fbc3f7fe700 1 -- 172.21.15.196:0/1717633439 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbc400f7260 msgr2=0x7fbc40107000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:29.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.232+0000 7fbc3f7fe700 1 --2- 172.21.15.196:0/1717633439 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbc400f7260 0x7fbc40107000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:29.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.232+0000 7fbc3f7fe700 1 -- 172.21.15.196:0/1717633439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc400f6950 msgr2=0x7fbc40106b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:29.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.232+0000 7fbc3f7fe700 1 --2- 172.21.15.196:0/1717633439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc400f6950 0x7fbc40106b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:29.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.232+0000 7fbc3f7fe700 1 -- 172.21.15.196:0/1717633439 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc380097c0 con 0x7fbc400f8370 2024-02-23T00:39:29.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.232+0000 7fbc3effd700 1 --2- 172.21.15.196:0/1717633439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc400f6950 0x7fbc40106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:39:29.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.232+0000 7fbc3e7fc700 1 --2- 172.21.15.196:0/1717633439 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbc400f7260 0x7fbc40107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:39:29.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.233+0000 7fbc3f7fe700 1 --2- 172.21.15.196:0/1717633439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc400f8370 0x7fbc4010e0e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fbc38009790 tx=0x7fbc38004d10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:39:29.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.233+0000 7fbc27fff700 10 client.?.objecter ms_handle_connect 0x7fbc400f8370 2024-02-23T00:39:29.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.233+0000 7fbc27fff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:39:29.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.233+0000 7fbc27fff700 1 -- 172.21.15.196:0/1717633439 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3800e040 con 0x7fbc400f8370 2024-02-23T00:39:29.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.233+0000 7fbc27fff700 1 -- 172.21.15.196:0/1717633439 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fbc38018040 con 0x7fbc400f8370 2024-02-23T00:39:29.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.233+0000 7fbc27fff700 1 -- 172.21.15.196:0/1717633439 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3801d690 con 0x7fbc400f8370 2024-02-23T00:39:29.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.233+0000 7fbc45976700 1 -- 172.21.15.196:0/1717633439 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc4010ead0 con 0x7fbc400f8370 2024-02-23T00:39:29.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.233+0000 7fbc45976700 10 client.24415.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:39:29.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.233+0000 7fbc45976700 1 -- 172.21.15.196:0/1717633439 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc4010ef00 con 0x7fbc400f8370 2024-02-23T00:39:29.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.233+0000 7fbc45976700 1 librados: init done 2024-02-23T00:39:29.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.234+0000 7fbc45976700 1 -- 172.21.15.196:0/1717633439 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc401015c0 con 0x7fbc400f8370 2024-02-23T00:39:29.239 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.235+0000 7fbc27fff700 1 -- 172.21.15.196:0/1717633439 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 15) v1 ==== 77115+0+0 (secure 0 0 0) 0x7fbc38013420 con 0x7fbc400f8370 2024-02-23T00:39:29.239 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.235+0000 7fbc27fff700 1 --2- 172.21.15.196:0/1717633439 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fbc28062c60 0x7fbc28065120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:29.239 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.235+0000 7fbc27fff700 1 -- 172.21.15.196:0/1717633439 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7090+0+0 (secure 0 0 0) 0x7fbc3807eb80 con 0x7fbc400f8370 2024-02-23T00:39:29.239 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.235+0000 7fbc27fff700 10 client.24415.objecter ms_dispatch 0x7fbc40001550 osd_map(68..68 src has 1..68) v4 2024-02-23T00:39:29.239 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.235+0000 7fbc27fff700 3 client.24415.objecter handle_osd_map got epochs [68,68] > 0 2024-02-23T00:39:29.239 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.235+0000 7fbc27fff700 3 client.24415.objecter handle_osd_map decoding full epoch 68 2024-02-23T00:39:29.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.235+0000 7fbc27fff700 20 client.24415.objecter dump_active .. 0 homeless 2024-02-23T00:39:29.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.235+0000 7fbc3effd700 1 --2- 172.21.15.196:0/1717633439 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fbc28062c60 0x7fbc28065120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:29.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.236+0000 7fbc27fff700 10 client.24415.objecter ms_handle_connect 0x7fbc28062c60 2024-02-23T00:39:29.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.236+0000 7fbc3effd700 1 --2- 172.21.15.196:0/1717633439 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fbc28062c60 0x7fbc28065120 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fbc30009250 tx=0x7fbc30009d60).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:39:29.242 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.241+0000 7fbc27fff700 1 -- 172.21.15.196:0/1717633439 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fbc3804e9d0 con 0x7fbc400f8370 2024-02-23T00:39:29.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.425+0000 7fbc45976700 1 -- 172.21.15.196:0/1717633439 --> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi047=a;smithi196=b", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc40101750 con 0x7fbc28062c60 2024-02-23T00:39:29.434 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.432+0000 7fbc27fff700 1 -- 172.21.15.196:0/1717633439 <== mgr.14146 v2:172.21.15.47:6800/622994871 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7fbc40101750 con 0x7fbc28062c60 2024-02-23T00:39:29.434 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled node-exporter update... 2024-02-23T00:39:29.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.435+0000 7fbc45976700 10 librados: watch_flush enter 2024-02-23T00:39:29.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.435+0000 7fbc45976700 10 librados: watch_flush exit 2024-02-23T00:39:29.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.435+0000 7fbc45976700 20 client.24415.objecter shutdown clearing up homeless session... 2024-02-23T00:39:29.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.435+0000 7fbc45976700 10 client.24415.objecter successfully canceled tick 2024-02-23T00:39:29.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.435+0000 7fbc45976700 1 -- 172.21.15.196:0/1717633439 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fbc28062c60 msgr2=0x7fbc28065120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:29.441 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.435+0000 7fbc45976700 1 --2- 172.21.15.196:0/1717633439 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fbc28062c60 0x7fbc28065120 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fbc30009250 tx=0x7fbc30009d60).stop 2024-02-23T00:39:29.441 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.435+0000 7fbc45976700 1 -- 172.21.15.196:0/1717633439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc400f8370 msgr2=0x7fbc4010e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:29.441 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.435+0000 7fbc45976700 1 --2- 172.21.15.196:0/1717633439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc400f8370 0x7fbc4010e0e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fbc38009790 tx=0x7fbc38004d10).stop 2024-02-23T00:39:29.441 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.435+0000 7fbc45976700 1 -- 172.21.15.196:0/1717633439 shutdown_connections 2024-02-23T00:39:29.441 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.435+0000 7fbc45976700 1 --2- 172.21.15.196:0/1717633439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc400f8370 0x7fbc4010e0e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:29.441 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.435+0000 7fbc45976700 1 --2- 172.21.15.196:0/1717633439 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fbc28062c60 0x7fbc28065120 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:29.441 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.435+0000 7fbc45976700 1 --2- 172.21.15.196:0/1717633439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc400f6950 0x7fbc40106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:29.441 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.435+0000 7fbc45976700 1 --2- 172.21.15.196:0/1717633439 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbc400f7260 0x7fbc40107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:29.441 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.435+0000 7fbc45976700 1 -- 172.21.15.196:0/1717633439 >> 172.21.15.196:0/1717633439 conn(0x7fbc400f2150 msgr2=0x7fbc400faec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:39:29.441 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.435+0000 7fbc45976700 1 -- 172.21.15.196:0/1717633439 shutdown_connections 2024-02-23T00:39:29.441 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.435+0000 7fbc45976700 1 -- 172.21.15.196:0/1717633439 wait complete. 2024-02-23T00:39:29.441 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:29.435+0000 7fbc45976700 1 librados: shutdown 2024-02-23T00:39:30.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:29.307631+0000 mon.a (mon.0) 608 : cluster [DBG] osdmap e69: 8 total, 8 up, 8 in 2024-02-23T00:39:30.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:29.427679+0000 mgr.y (mgr.14146) 232 : audit [DBG] from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi047=a;smithi196=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:39:30.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:39:29.429174+0000 mgr.y (mgr.14146) 233 : cephadm [INF] Saving service node-exporter spec with placement smithi047=a;smithi196=b;count:2 2024-02-23T00:39:30.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:29.433214+0000 mon.a (mon.0) 609 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:30.496 DEBUG:teuthology.orchestra.run.smithi047:node-exporter.a> sudo journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@node-exporter.a.service 2024-02-23T00:39:30.499 DEBUG:teuthology.orchestra.run.smithi196:node-exporter.b> sudo journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@node-exporter.b.service 2024-02-23T00:39:30.502 INFO:tasks.cephadm:Adding alertmanager.a on smithi047 2024-02-23T00:39:30.502 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph orch apply alertmanager '1;smithi047=a' 2024-02-23T00:39:30.535 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:-- Logs begin at Fri 2024-02-23 00:13:21 UTC. -- 2024-02-23T00:39:30.537 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:-- Logs begin at Fri 2024-02-23 00:13:17 UTC. -- 2024-02-23T00:39:30.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:29.307631+0000 mon.a (mon.0) 608 : cluster [DBG] osdmap e69: 8 total, 8 up, 8 in 2024-02-23T00:39:30.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:29.427679+0000 mgr.y (mgr.14146) 232 : audit [DBG] from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi047=a;smithi196=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:39:30.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:39:29.429174+0000 mgr.y (mgr.14146) 233 : cephadm [INF] Saving service node-exporter spec with placement smithi047=a;smithi196=b;count:2 2024-02-23T00:39:30.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:29.433214+0000 mon.a (mon.0) 609 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:30.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:29.307631+0000 mon.a (mon.0) 608 : cluster [DBG] osdmap e69: 8 total, 8 up, 8 in 2024-02-23T00:39:30.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:29.427679+0000 mgr.y (mgr.14146) 232 : audit [DBG] from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi047=a;smithi196=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:39:30.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:39:29.429174+0000 mgr.y (mgr.14146) 233 : cephadm [INF] Saving service node-exporter spec with placement smithi047=a;smithi196=b;count:2 2024-02-23T00:39:30.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:29.433214+0000 mon.a (mon.0) 609 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:31.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:30.127566+0000 mgr.y (mgr.14146) 234 : cluster [DBG] pgmap v228: 132 pgs: 132 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 97 KiB/s rd, 4.0 KiB/s wr, 202 op/s 2024-02-23T00:39:31.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:30.127566+0000 mgr.y (mgr.14146) 234 : cluster [DBG] pgmap v228: 132 pgs: 132 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 97 KiB/s rd, 4.0 KiB/s wr, 202 op/s 2024-02-23T00:39:31.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:30.127566+0000 mgr.y (mgr.14146) 234 : cluster [DBG] pgmap v228: 132 pgs: 132 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 97 KiB/s rd, 4.0 KiB/s wr, 202 op/s 2024-02-23T00:39:32.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.835+0000 7fefde049700 1 -- 172.21.15.196:0/3812184472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefd80f8580 msgr2=0x7fefd80fee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:32.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.835+0000 7fefde049700 1 --2- 172.21.15.196:0/3812184472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefd80f8580 0x7fefd80fee80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fefd0009c10 tx=0x7fefd0004d10).stop 2024-02-23T00:39:32.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.835+0000 7fefde049700 1 -- 172.21.15.196:0/3812184472 shutdown_connections 2024-02-23T00:39:32.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.835+0000 7fefde049700 1 --2- 172.21.15.196:0/3812184472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefd80f8580 0x7fefd80fee80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:32.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.835+0000 7fefde049700 1 --2- 172.21.15.196:0/3812184472 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fefd80f6990 0x7fefd80f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:39:32.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.835+0000 7fefde049700 1 --2- 172.21.15.196:0/3812184472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefd80f7c30 0x7fefd80f8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:39:32.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.835+0000 7fefde049700 1 -- 172.21.15.196:0/3812184472 >> 172.21.15.196:0/3812184472 conn(0x7fefd80f2190 msgr2=0x7fefd80f45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:39:32.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.835+0000 7fefde049700 1 -- 172.21.15.196:0/3812184472 shutdown_connections 2024-02-23T00:39:32.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.835+0000 7fefde049700 1 -- 172.21.15.196:0/3812184472 wait complete. 2024-02-23T00:39:32.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.835+0000 7fefde049700 1 librados: starting msgr at 2024-02-23T00:39:32.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.835+0000 7fefde049700 1 librados: starting objecter 2024-02-23T00:39:32.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.835+0000 7fefde049700 1 Processor -- start 2024-02-23T00:39:32.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.836+0000 7fefde049700 1 -- start start 2024-02-23T00:39:32.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.836+0000 7fefde049700 1 librados: setting wanted keys 2024-02-23T00:39:32.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.836+0000 7fefde049700 1 librados: calling monclient init 2024-02-23T00:39:32.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.836+0000 7fefde049700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefd80f6990 0x7fefd80fa910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:32.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.836+0000 7fefde049700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefd80f7c30 0x7fefd80f8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:32.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.836+0000 7fefde049700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fefd80f8580 0x7fefd80f9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:32.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.836+0000 7fefde049700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd80f9b10 con 0x7fefd80f7c30 2024-02-23T00:39:32.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.836+0000 7fefde049700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd80f9c80 con 0x7fefd80f6990 2024-02-23T00:39:32.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.836+0000 7fefde049700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd80f9e40 con 0x7fefd80f8580 2024-02-23T00:39:32.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.836+0000 7fefd77fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefd80f6990 0x7fefd80fa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:32.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.836+0000 7fefd77fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefd80f6990 0x7fefd80fa910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:59782/0 (socket says 172.21.15.196:59782) 2024-02-23T00:39:32.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.836+0000 7fefd77fe700 1 -- 172.21.15.196:0/354761644 learned_addr learned my addr 172.21.15.196:0/354761644 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-02-23T00:39:32.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.836+0000 7fefd7fff700 1 --2- 172.21.15.196:0/354761644 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fefd80f8580 0x7fefd80f9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:32.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.836+0000 7fefd6ffd700 1 --2- 172.21.15.196:0/354761644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefd80f7c30 0x7fefd80f8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:32.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.836+0000 7fefd77fe700 1 -- 172.21.15.196:0/354761644 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fefd80f8580 msgr2=0x7fefd80f9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:32.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.836+0000 7fefd77fe700 1 --2- 172.21.15.196:0/354761644 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fefd80f8580 0x7fefd80f9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:32.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.836+0000 7fefd77fe700 1 -- 172.21.15.196:0/354761644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefd80f7c30 msgr2=0x7fefd80f8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:32.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.836+0000 7fefd77fe700 1 --2- 172.21.15.196:0/354761644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefd80f7c30 0x7fefd80f8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:32.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.836+0000 7fefd77fe700 1 -- 172.21.15.196:0/354761644 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd00097c0 con 0x7fefd80f6990 2024-02-23T00:39:32.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.836+0000 7fefd6ffd700 1 --2- 172.21.15.196:0/354761644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefd80f7c30 0x7fefd80f8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:39:32.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.836+0000 7fefd7fff700 1 --2- 172.21.15.196:0/354761644 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fefd80f8580 0x7fefd80f9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:39:32.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.836+0000 7fefd77fe700 1 --2- 172.21.15.196:0/354761644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefd80f6990 0x7fefd80fa910 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fefc800b500 tx=0x7fefc80090f0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:39:32.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.836+0000 7fefd4ff9700 10 client.?.objecter ms_handle_connect 0x7fefd80f6990 2024-02-23T00:39:32.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.836+0000 7fefd4ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:39:32.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.836+0000 7fefd4ff9700 1 -- 172.21.15.196:0/354761644 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc800d4a0 con 0x7fefd80f6990 2024-02-23T00:39:32.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.836+0000 7fefd4ff9700 1 -- 172.21.15.196:0/354761644 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fefc800da90 con 0x7fefd80f6990 2024-02-23T00:39:32.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.836+0000 7fefd4ff9700 1 -- 172.21.15.196:0/354761644 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc80129c0 con 0x7fefd80f6990 2024-02-23T00:39:32.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.836+0000 7fefde049700 1 -- 172.21.15.196:0/354761644 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefd8112060 con 0x7fefd80f6990 2024-02-23T00:39:32.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.836+0000 7fefde049700 10 client.24421.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:39:32.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.836+0000 7fefde049700 1 -- 172.21.15.196:0/354761644 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd8112500 con 0x7fefd80f6990 2024-02-23T00:39:32.853 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.836+0000 7fefde049700 1 librados: init done 2024-02-23T00:39:32.853 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.837+0000 7fefde049700 1 -- 172.21.15.196:0/354761644 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefd805f360 con 0x7fefd80f6990 2024-02-23T00:39:32.853 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.838+0000 7fefd4ff9700 1 -- 172.21.15.196:0/354761644 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 15) v1 ==== 77115+0+0 (secure 0 0 0) 0x7fefc800d600 con 0x7fefd80f6990 2024-02-23T00:39:32.853 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.838+0000 7fefd4ff9700 1 --2- 172.21.15.196:0/354761644 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fefc0062c60 0x7fefc0065120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:32.853 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.838+0000 7fefd4ff9700 1 -- 172.21.15.196:0/354761644 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7090+0+0 (secure 0 0 0) 0x7fefc807cee0 con 0x7fefd80f6990 2024-02-23T00:39:32.853 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.838+0000 7fefd4ff9700 10 client.24421.objecter ms_dispatch 0x7fefd8001550 osd_map(69..69 src has 1..69) v4 2024-02-23T00:39:32.853 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.838+0000 7fefd4ff9700 3 client.24421.objecter handle_osd_map got epochs [69,69] > 0 2024-02-23T00:39:32.853 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.838+0000 7fefd4ff9700 3 client.24421.objecter handle_osd_map decoding full epoch 69 2024-02-23T00:39:32.853 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.838+0000 7fefd4ff9700 20 client.24421.objecter dump_active .. 0 homeless 2024-02-23T00:39:32.853 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.838+0000 7fefd6ffd700 1 --2- 172.21.15.196:0/354761644 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fefc0062c60 0x7fefc0065120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:32.853 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.839+0000 7fefd4ff9700 10 client.24421.objecter ms_handle_connect 0x7fefc0062c60 2024-02-23T00:39:32.853 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.839+0000 7fefd6ffd700 1 --2- 172.21.15.196:0/354761644 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fefc0062c60 0x7fefc0065120 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fefcc009990 tx=0x7fefcc008040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:39:32.853 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:32.842+0000 7fefd4ff9700 1 -- 172.21.15.196:0/354761644 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fefc804cc10 con 0x7fefd80f6990 2024-02-23T00:39:33.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:33.028+0000 7fefde049700 1 -- 172.21.15.196:0/354761644 --> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi047=a", "target": ["mon-mgr", ""]}) v1 -- 0x7fefd80610f0 con 0x7fefc0062c60 2024-02-23T00:39:33.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:33.035+0000 7fefd4ff9700 1 -- 172.21.15.196:0/354761644 <== mgr.14146 v2:172.21.15.47:6800/622994871 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7fefd80610f0 con 0x7fefc0062c60 2024-02-23T00:39:33.037 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled alertmanager update... 2024-02-23T00:39:33.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:33.038+0000 7fefde049700 10 librados: watch_flush enter 2024-02-23T00:39:33.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:33.038+0000 7fefde049700 10 librados: watch_flush exit 2024-02-23T00:39:33.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:33.038+0000 7fefde049700 20 client.24421.objecter shutdown clearing up homeless session... 2024-02-23T00:39:33.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:33.038+0000 7fefde049700 10 client.24421.objecter successfully canceled tick 2024-02-23T00:39:33.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:33.038+0000 7fefde049700 1 -- 172.21.15.196:0/354761644 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fefc0062c60 msgr2=0x7fefc0065120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:33.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:33.038+0000 7fefde049700 1 --2- 172.21.15.196:0/354761644 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fefc0062c60 0x7fefc0065120 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fefcc009990 tx=0x7fefcc008040).stop 2024-02-23T00:39:33.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:33.038+0000 7fefde049700 1 -- 172.21.15.196:0/354761644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefd80f6990 msgr2=0x7fefd80fa910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:33.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:33.038+0000 7fefde049700 1 --2- 172.21.15.196:0/354761644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefd80f6990 0x7fefd80fa910 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fefc800b500 tx=0x7fefc80090f0).stop 2024-02-23T00:39:33.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:33.038+0000 7fefde049700 1 -- 172.21.15.196:0/354761644 shutdown_connections 2024-02-23T00:39:33.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:33.038+0000 7fefde049700 1 --2- 172.21.15.196:0/354761644 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fefd80f8580 0x7fefd80f9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:33.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:33.038+0000 7fefde049700 1 --2- 172.21.15.196:0/354761644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefd80f6990 0x7fefd80fa910 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:33.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:33.038+0000 7fefde049700 1 --2- 172.21.15.196:0/354761644 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7fefc0062c60 0x7fefc0065120 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:33.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:33.038+0000 7fefde049700 1 --2- 172.21.15.196:0/354761644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefd80f7c30 0x7fefd80f8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:33.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:33.038+0000 7fefde049700 1 -- 172.21.15.196:0/354761644 >> 172.21.15.196:0/354761644 conn(0x7fefd80f2190 msgr2=0x7fefd80609e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:39:33.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:33.038+0000 7fefde049700 1 -- 172.21.15.196:0/354761644 shutdown_connections 2024-02-23T00:39:33.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:33.038+0000 7fefde049700 1 -- 172.21.15.196:0/354761644 wait complete. 2024-02-23T00:39:33.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:33.038+0000 7fefde049700 1 librados: shutdown 2024-02-23T00:39:33.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:32.128203+0000 mgr.y (mgr.14146) 235 : cluster [DBG] pgmap v229: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 65 KiB/s rd, 2.2 KiB/s wr, 128 op/s 2024-02-23T00:39:33.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:32.670949+0000 mon.a (mon.0) 610 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:33.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:33.036271+0000 mon.a (mon.0) 611 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:33.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:32.128203+0000 mgr.y (mgr.14146) 235 : cluster [DBG] pgmap v229: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 65 KiB/s rd, 2.2 KiB/s wr, 128 op/s 2024-02-23T00:39:33.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:32.670949+0000 mon.a (mon.0) 610 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:33.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:33.036271+0000 mon.a (mon.0) 611 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:33.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:32.128203+0000 mgr.y (mgr.14146) 235 : cluster [DBG] pgmap v229: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 65 KiB/s rd, 2.2 KiB/s wr, 128 op/s 2024-02-23T00:39:33.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:32.670949+0000 mon.a (mon.0) 610 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:33.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:33.036271+0000 mon.a (mon.0) 611 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:34.363 DEBUG:teuthology.orchestra.run.smithi047:alertmanager.a> sudo journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@alertmanager.a.service 2024-02-23T00:39:34.366 INFO:tasks.cephadm:Adding grafana.a on smithi196 2024-02-23T00:39:34.366 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph orch apply grafana '1;smithi196=a' 2024-02-23T00:39:34.397 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:-- Logs begin at Fri 2024-02-23 00:13:17 UTC. -- 2024-02-23T00:39:34.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:33.030823+0000 mgr.y (mgr.14146) 236 : audit [DBG] from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi047=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:39:34.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:39:33.032249+0000 mgr.y (mgr.14146) 237 : cephadm [INF] Saving service alertmanager spec with placement smithi047=a;count:1 2024-02-23T00:39:34.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:33.791989+0000 mon.a (mon.0) 612 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:34.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:34.309406+0000 mon.a (mon.0) 613 : cluster [DBG] osdmap e70: 8 total, 8 up, 8 in 2024-02-23T00:39:34.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:33.030823+0000 mgr.y (mgr.14146) 236 : audit [DBG] from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi047=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:39:34.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:39:33.032249+0000 mgr.y (mgr.14146) 237 : cephadm [INF] Saving service alertmanager spec with placement smithi047=a;count:1 2024-02-23T00:39:34.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:33.791989+0000 mon.a (mon.0) 612 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:34.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:34.309406+0000 mon.a (mon.0) 613 : cluster [DBG] osdmap e70: 8 total, 8 up, 8 in 2024-02-23T00:39:34.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:33.030823+0000 mgr.y (mgr.14146) 236 : audit [DBG] from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi047=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:39:34.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:39:33.032249+0000 mgr.y (mgr.14146) 237 : cephadm [INF] Saving service alertmanager spec with placement smithi047=a;count:1 2024-02-23T00:39:34.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:33.791989+0000 mon.a (mon.0) 612 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:34.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:34.309406+0000 mon.a (mon.0) 613 : cluster [DBG] osdmap e70: 8 total, 8 up, 8 in 2024-02-23T00:39:35.693 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:39:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ignoring --setuser ceph since I am not root 2024-02-23T00:39:35.693 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:39:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ignoring --setgroup ceph since I am not root 2024-02-23T00:39:35.693 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:39:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:39:35.679+0000 7f6e09070680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-23T00:39:35.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:34.128821+0000 mgr.y (mgr.14146) 238 : cluster [DBG] pgmap v230: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 47 KiB/s rd, 1.6 KiB/s wr, 90 op/s 2024-02-23T00:39:35.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:34.513301+0000 mon.a (mon.0) 614 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:39:35.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:34.514039+0000 mon.a (mon.0) 615 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:39:35.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:34.518639+0000 mon.a (mon.0) 616 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:35.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:34.523370+0000 mon.a (mon.0) 617 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-23T00:39:35.694 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:34.128821+0000 mgr.y (mgr.14146) 238 : cluster [DBG] pgmap v230: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 47 KiB/s rd, 1.6 KiB/s wr, 90 op/s 2024-02-23T00:39:35.694 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:34.513301+0000 mon.a (mon.0) 614 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:39:35.695 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:34.514039+0000 mon.a (mon.0) 615 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:39:35.695 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:34.518639+0000 mon.a (mon.0) 616 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:35.695 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:34.523370+0000 mon.a (mon.0) 617 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-23T00:39:35.695 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ignoring --setuser ceph since I am not root 2024-02-23T00:39:35.695 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ignoring --setgroup ceph since I am not root 2024-02-23T00:39:35.695 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:39:35.563+0000 7f6798cbe700 1 -- 172.21.15.47:0/2608101763 <== mon.0 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x565398f644e0 con 0x565398f3b400 2024-02-23T00:39:35.695 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:39:35.692+0000 7f67a8e9b680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-23T00:39:35.696 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:34.128821+0000 mgr.y (mgr.14146) 238 : cluster [DBG] pgmap v230: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 47 KiB/s rd, 1.6 KiB/s wr, 90 op/s 2024-02-23T00:39:35.696 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:34.513301+0000 mon.a (mon.0) 614 : audit [DBG] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:39:35.696 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:34.514039+0000 mon.a (mon.0) 615 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:39:35.696 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:34.518639+0000 mon.a (mon.0) 616 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' 2024-02-23T00:39:35.696 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:34.523370+0000 mon.a (mon.0) 617 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-23T00:39:36.027 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:39:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:39:35.761+0000 7f6e09070680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-23T00:39:36.056 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:39:35.777+0000 7f67a8e9b680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-23T00:39:36.335 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:39:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:39:36.130+0000 7f6e09070680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-23T00:39:36.521 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:39:36.156+0000 7f67a8e9b680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-23T00:39:36.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.629+0000 7f7715dc5700 1 -- 172.21.15.196:0/688486164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f771006cbd0 msgr2=0x7f771006b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:36.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.629+0000 7f7715dc5700 1 --2- 172.21.15.196:0/688486164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f771006cbd0 0x7f771006b0d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f7700004870 tx=0x7f7700009cb0).stop 2024-02-23T00:39:36.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.629+0000 7f7715dc5700 1 -- 172.21.15.196:0/688486164 shutdown_connections 2024-02-23T00:39:36.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.629+0000 7f7715dc5700 1 --2- 172.21.15.196:0/688486164 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77100ff9b0 0x7f7710101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:36.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.629+0000 7f7715dc5700 1 --2- 172.21.15.196:0/688486164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f771006cbd0 0x7f771006b0d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:36.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.629+0000 7f7715dc5700 1 --2- 172.21.15.196:0/688486164 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f771006b5c0 0x7f771006ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:36.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.629+0000 7f7715dc5700 1 -- 172.21.15.196:0/688486164 >> 172.21.15.196:0/688486164 conn(0x7f77100f32a0 msgr2=0x7f77100f56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:39:36.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.629+0000 7f7715dc5700 1 -- 172.21.15.196:0/688486164 shutdown_connections 2024-02-23T00:39:36.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.629+0000 7f7715dc5700 1 -- 172.21.15.196:0/688486164 wait complete. 2024-02-23T00:39:36.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.630+0000 7f7715dc5700 1 librados: starting msgr at 2024-02-23T00:39:36.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.630+0000 7f7715dc5700 1 librados: starting objecter 2024-02-23T00:39:36.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.630+0000 7f7715dc5700 1 Processor -- start 2024-02-23T00:39:36.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.630+0000 7f7715dc5700 1 -- start start 2024-02-23T00:39:36.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.630+0000 7f7715dc5700 1 librados: setting wanted keys 2024-02-23T00:39:36.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.630+0000 7f7715dc5700 1 librados: calling monclient init 2024-02-23T00:39:36.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.630+0000 7f7715dc5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f771006b5c0 0x7f771010a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:36.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.630+0000 7f7715dc5700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f771006cbd0 0x7f771010a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:36.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.630+0000 7f7715dc5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77100ff9b0 0x7f7710111740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:36.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.630+0000 7f7715dc5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7710111c30 con 0x7f77100ff9b0 2024-02-23T00:39:36.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.630+0000 7f7715dc5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7710111d70 con 0x7f771006b5c0 2024-02-23T00:39:36.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.630+0000 7f7715dc5700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7710111eb0 con 0x7f771006cbd0 2024-02-23T00:39:36.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.630+0000 7f770f7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f771006b5c0 0x7f771010a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:36.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.630+0000 7f770effd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f771006cbd0 0x7f771010a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:36.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.630+0000 7f770f7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f771006b5c0 0x7f771010a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:59824/0 (socket says 172.21.15.196:59824) 2024-02-23T00:39:36.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.630+0000 7f770f7fe700 1 -- 172.21.15.196:0/2716205704 learned_addr learned my addr 172.21.15.196:0/2716205704 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-02-23T00:39:36.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.630+0000 7f770ffff700 1 --2- 172.21.15.196:0/2716205704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77100ff9b0 0x7f7710111740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:36.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.631+0000 7f770f7fe700 1 -- 172.21.15.196:0/2716205704 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f771006cbd0 msgr2=0x7f771010a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:36.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.631+0000 7f770f7fe700 1 --2- 172.21.15.196:0/2716205704 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f771006cbd0 0x7f771010a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:36.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.631+0000 7f770f7fe700 1 -- 172.21.15.196:0/2716205704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77100ff9b0 msgr2=0x7f7710111740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:36.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.631+0000 7f770f7fe700 1 --2- 172.21.15.196:0/2716205704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77100ff9b0 0x7f7710111740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:36.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.631+0000 7f770f7fe700 1 -- 172.21.15.196:0/2716205704 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7700005040 con 0x7f771006b5c0 2024-02-23T00:39:36.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.631+0000 7f770effd700 1 --2- 172.21.15.196:0/2716205704 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f771006cbd0 0x7f771010a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:39:36.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.631+0000 7f770f7fe700 1 --2- 172.21.15.196:0/2716205704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f771006b5c0 0x7f771010a170 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f7700013780 tx=0x7f7700009cb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:39:36.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.631+0000 7f770cff9700 10 client.?.objecter ms_handle_connect 0x7f771006b5c0 2024-02-23T00:39:36.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.631+0000 7f770cff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:39:36.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.631+0000 7f770cff9700 1 -- 172.21.15.196:0/2716205704 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770000e030 con 0x7f771006b5c0 2024-02-23T00:39:36.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.631+0000 7f770cff9700 1 -- 172.21.15.196:0/2716205704 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f77000096d0 con 0x7f771006b5c0 2024-02-23T00:39:36.637 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.631+0000 7f7715dc5700 1 -- 172.21.15.196:0/2716205704 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7710112130 con 0x7f771006b5c0 2024-02-23T00:39:36.637 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.631+0000 7f770cff9700 1 -- 172.21.15.196:0/2716205704 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7700018070 con 0x7f771006b5c0 2024-02-23T00:39:36.637 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.631+0000 7f7715dc5700 10 client.24439.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:39:36.637 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.631+0000 7f7715dc5700 1 -- 172.21.15.196:0/2716205704 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7710112530 con 0x7f771006b5c0 2024-02-23T00:39:36.637 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.631+0000 7f7715dc5700 1 librados: init done 2024-02-23T00:39:36.638 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.632+0000 7f7715dc5700 1 -- 172.21.15.196:0/2716205704 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f771005f360 con 0x7f771006b5c0 2024-02-23T00:39:36.638 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.633+0000 7f770cff9700 1 -- 172.21.15.196:0/2716205704 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) v1 ==== 77129+0+0 (secure 0 0 0) 0x7f7700027440 con 0x7f771006b5c0 2024-02-23T00:39:36.638 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.633+0000 7f770cff9700 1 --2- 172.21.15.196:0/2716205704 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f76f8062cb0 0x7f76f8065170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:36.638 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.633+0000 7f770cff9700 1 -- 172.21.15.196:0/2716205704 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7090+0+0 (secure 0 0 0) 0x7f770007e060 con 0x7f771006b5c0 2024-02-23T00:39:36.638 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.633+0000 7f770effd700 1 -- 172.21.15.196:0/2716205704 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f76f8062cb0 msgr2=0x7f76f8065170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/622994871 2024-02-23T00:39:36.638 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.633+0000 7f770cff9700 10 client.24439.objecter ms_dispatch 0x7f7710001550 osd_map(70..70 src has 1..70) v4 2024-02-23T00:39:36.638 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.633+0000 7f770cff9700 3 client.24439.objecter handle_osd_map got epochs [70,70] > 0 2024-02-23T00:39:36.638 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.633+0000 7f770cff9700 3 client.24439.objecter handle_osd_map decoding full epoch 70 2024-02-23T00:39:36.638 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.633+0000 7f770effd700 1 --2- 172.21.15.196:0/2716205704 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f76f8062cb0 0x7f76f8065170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-02-23T00:39:36.638 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.633+0000 7f770cff9700 20 client.24439.objecter dump_active .. 0 homeless 2024-02-23T00:39:36.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.637+0000 7f770cff9700 1 -- 172.21.15.196:0/2716205704 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f770004de50 con 0x7f771006b5c0 2024-02-23T00:39:36.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:35.525052+0000 mon.a (mon.0) 618 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-23T00:39:36.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:35.525129+0000 mon.a (mon.0) 619 : cluster [DBG] mgrmap e16: y(active, since 5m), standbys: x 2024-02-23T00:39:36.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:35.525052+0000 mon.a (mon.0) 618 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-23T00:39:36.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:35.525129+0000 mon.a (mon.0) 619 : cluster [DBG] mgrmap e16: y(active, since 5m), standbys: x 2024-02-23T00:39:36.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:35.525052+0000 mon.a (mon.0) 618 : audit [INF] from='mgr.14146 172.21.15.47:0/1999100149' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-23T00:39:36.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:35.525129+0000 mon.a (mon.0) 619 : cluster [DBG] mgrmap e16: y(active, since 5m), standbys: x 2024-02-23T00:39:36.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.819+0000 7f7715dc5700 1 -- 172.21.15.196:0/2716205704 --> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi196=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f77100f62e0 con 0x7f76f8062cb0 2024-02-23T00:39:36.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.834+0000 7f770effd700 1 -- 172.21.15.196:0/2716205704 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f76f8062cb0 msgr2=0x7f76f8065170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/622994871 2024-02-23T00:39:36.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:36.834+0000 7f770effd700 1 --2- 172.21.15.196:0/2716205704 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f76f8062cb0 0x7f76f8065170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-02-23T00:39:37.193 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:39:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:39:36.898+0000 7f6e09070680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-23T00:39:37.193 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:39:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:39:36.984+0000 7f6e09070680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-23T00:39:37.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:37.235+0000 7f770effd700 1 -- 172.21.15.196:0/2716205704 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f76f8062cb0 msgr2=0x7f76f8065170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/622994871 2024-02-23T00:39:37.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:37.235+0000 7f770effd700 1 --2- 172.21.15.196:0/2716205704 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f76f8062cb0 0x7f76f8065170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2024-02-23T00:39:37.253 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:39:36.948+0000 7f67a8e9b680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-23T00:39:37.253 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:39:37.031+0000 7f67a8e9b680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-23T00:39:37.523 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:39:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:39:37.195+0000 7f6e09070680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-23T00:39:37.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:37.023806+0000 osd.6 (osd.6) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:39:37.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:37.023881+0000 osd.6 (osd.6) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:39:37.807 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:39:37.251+0000 7f67a8e9b680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-23T00:39:37.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:37.023806+0000 osd.6 (osd.6) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:39:37.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:37.023881+0000 osd.6 (osd.6) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:39:37.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:37.023806+0000 osd.6 (osd.6) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:39:37.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:37.023881+0000 osd.6 (osd.6) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:39:38.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:38.036+0000 7f770effd700 1 -- 172.21.15.196:0/2716205704 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f76f8062cb0 msgr2=0x7f76f8065170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/622994871 2024-02-23T00:39:38.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:38.036+0000 7f770effd700 1 --2- 172.21.15.196:0/2716205704 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f76f8062cb0 0x7f76f8065170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2024-02-23T00:39:38.423 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:39:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:39:38.126+0000 7f6e09070680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-23T00:39:38.423 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:39:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:39:38.252+0000 7f6e09070680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-23T00:39:38.423 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:39:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:39:38.330+0000 7f6e09070680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-23T00:39:38.423 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:39:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:39:38.421+0000 7f6e09070680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-23T00:39:38.526 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:39:38.236+0000 7f67a8e9b680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-23T00:39:38.526 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:39:38.372+0000 7f67a8e9b680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-23T00:39:38.527 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:39:38.455+0000 7f67a8e9b680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-23T00:39:38.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:37.526679+0000 mon.a (mon.0) 620 : cluster [DBG] osdmap e71: 8 total, 8 up, 8 in 2024-02-23T00:39:38.693 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:39:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:39:38.619+0000 7f6e09070680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-23T00:39:38.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:37.526679+0000 mon.a (mon.0) 620 : cluster [DBG] osdmap e71: 8 total, 8 up, 8 in 2024-02-23T00:39:38.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:37.526679+0000 mon.a (mon.0) 620 : cluster [DBG] osdmap e71: 8 total, 8 up, 8 in 2024-02-23T00:39:38.807 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:39:38.550+0000 7f67a8e9b680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-23T00:39:38.807 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:39:38.752+0000 7f67a8e9b680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-23T00:39:39.193 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:39:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:39:38.898+0000 7f6e09070680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-23T00:39:39.193 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:39:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:39:38.996+0000 7f6e09070680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-23T00:39:39.306 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:39:39.049+0000 7f67a8e9b680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-23T00:39:39.306 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:39:39.151+0000 7f67a8e9b680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-23T00:39:39.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:39.637+0000 7f770effd700 1 -- 172.21.15.196:0/2716205704 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f76f8062cb0 msgr2=0x7f76f8065170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/622994871 2024-02-23T00:39:39.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:39.637+0000 7f770effd700 1 --2- 172.21.15.196:0/2716205704 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f76f8062cb0 0x7f76f8065170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2024-02-23T00:39:40.061 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:39:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:39:39.714+0000 7f6e09070680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-23T00:39:40.061 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:39:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:39:39.794+0000 7f6e09070680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-23T00:39:40.061 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:39:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:39:39.886+0000 7f6e09070680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-23T00:39:40.061 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:39:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:39:40.058+0000 7f6e09070680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-23T00:39:40.281 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:39:39.915+0000 7f67a8e9b680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-23T00:39:40.281 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:39:40.001+0000 7f67a8e9b680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-23T00:39:40.281 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:39:40.097+0000 7f67a8e9b680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-23T00:39:40.281 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:39:40.280+0000 7f67a8e9b680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-23T00:39:40.321 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:39.309628+0000 mon.a (mon.0) 621 : cluster [DBG] osdmap e72: 8 total, 8 up, 8 in 2024-02-23T00:39:40.549 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:39.309628+0000 mon.a (mon.0) 621 : cluster [DBG] osdmap e72: 8 total, 8 up, 8 in 2024-02-23T00:39:40.549 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:39.309628+0000 mon.a (mon.0) 621 : cluster [DBG] osdmap e72: 8 total, 8 up, 8 in 2024-02-23T00:39:40.582 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:39:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:39:40.319+0000 7f6e09070680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-23T00:39:40.582 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:39:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:39:40.448+0000 7f6e09070680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-23T00:39:40.582 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:39:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:39:40.580+0000 7f6e09070680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-23T00:39:40.806 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:39:40.548+0000 7f67a8e9b680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-23T00:39:40.806 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:39:40.685+0000 7f67a8e9b680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-23T00:39:40.836 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:39:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:39:40.750+0000 7f6e09070680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-23T00:39:41.098 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:39:40.826+0000 7f67a8e9b680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-23T00:39:41.098 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:39:41.009+0000 7f67a8e9b680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-23T00:39:41.121 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:41.119+0000 7f770cff9700 1 -- 172.21.15.196:0/2716205704 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mgrmap(e 17) v1 ==== 76859+0+0 (secure 0 0 0) 0x7f770001d420 con 0x7f771006b5c0 2024-02-23T00:39:41.121 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:41.119+0000 7f770cff9700 1 -- 172.21.15.196:0/2716205704 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f76f8062cb0 msgr2=0x7f76f8065170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-23T00:39:41.121 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:41.119+0000 7f770cff9700 1 --2- 172.21.15.196:0/2716205704 >> [v2:172.21.15.47:6800/622994871,v1:172.21.15.47:6801/622994871] conn(0x7f76f8062cb0 0x7f76f8065170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:39:41.193 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:39:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: debug 2024-02-23T00:39:40.834+0000 7f6e09070680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-23T00:39:41.193 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:39:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: [23/Feb/2024:00:39:40] ENGINE Bus STARTING 2024-02-23T00:39:41.193 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:39:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: CherryPy Checker: 2024-02-23T00:39:41.193 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:39:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: The Application mounted at '' has an empty config. 2024-02-23T00:39:41.193 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:39:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: 2024-02-23T00:39:41.193 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:39:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: [23/Feb/2024:00:39:40] ENGINE Serving on http://:::9283 2024-02-23T00:39:41.193 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:39:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: [23/Feb/2024:00:39:40] ENGINE Bus STARTED 2024-02-23T00:39:41.364 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:40.848739+0000 mon.a (mon.0) 622 : cluster [DBG] Standby manager daemon x restarted 2024-02-23T00:39:41.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:40.848844+0000 mon.a (mon.0) 623 : cluster [DBG] Standby manager daemon x started 2024-02-23T00:39:41.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:40.850498+0000 mon.b (mon.1) 27 : audit [DBG] from='mgr.? 172.21.15.196:0/3033848973' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-23T00:39:41.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:40.850989+0000 mon.b (mon.1) 28 : audit [DBG] from='mgr.? 172.21.15.196:0/3033848973' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-23T00:39:41.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:40.852785+0000 mon.b (mon.1) 29 : audit [DBG] from='mgr.? 172.21.15.196:0/3033848973' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-23T00:39:41.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:40.853715+0000 mon.b (mon.1) 30 : audit [DBG] from='mgr.? 172.21.15.196:0/3033848973' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-23T00:39:41.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:41.110140+0000 mon.a (mon.0) 624 : cluster [INF] Active manager daemon y restarted 2024-02-23T00:39:41.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:41.110430+0000 mon.a (mon.0) 625 : cluster [INF] Activating manager daemon y 2024-02-23T00:39:41.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:41.116501+0000 mon.a (mon.0) 626 : cluster [DBG] osdmap e73: 8 total, 8 up, 8 in 2024-02-23T00:39:41.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:41.116925+0000 mon.a (mon.0) 627 : cluster [DBG] mgrmap e17: y(active, starting, since 0.00657908s), standbys: x 2024-02-23T00:39:41.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:41.120619+0000 mon.a (mon.0) 628 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-23T00:39:41.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:41.120759+0000 mon.a (mon.0) 629 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-23T00:39:41.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:41.120899+0000 mon.a (mon.0) 630 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-23T00:39:41.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:41.121631+0000 mon.a (mon.0) 631 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-23T00:39:41.366 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:41.121837+0000 mon.a (mon.0) 632 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-23T00:39:41.366 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:41.122008+0000 mon.a (mon.0) 633 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-23T00:39:41.366 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:39:41.096+0000 7f67a8e9b680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-23T00:39:41.366 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:39:41] ENGINE Bus STARTING 2024-02-23T00:39:41.366 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [79B blob data] 2024-02-23T00:39:41.366 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [80B blob data] 2024-02-23T00:39:41.366 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:40.848739+0000 mon.a (mon.0) 622 : cluster [DBG] Standby manager daemon x restarted 2024-02-23T00:39:41.367 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:40.848844+0000 mon.a (mon.0) 623 : cluster [DBG] Standby manager daemon x started 2024-02-23T00:39:41.367 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:40.850498+0000 mon.b (mon.1) 27 : audit [DBG] from='mgr.? 172.21.15.196:0/3033848973' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-23T00:39:41.367 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:40.850989+0000 mon.b (mon.1) 28 : audit [DBG] from='mgr.? 172.21.15.196:0/3033848973' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-23T00:39:41.367 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:40.852785+0000 mon.b (mon.1) 29 : audit [DBG] from='mgr.? 172.21.15.196:0/3033848973' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-23T00:39:41.367 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:40.853715+0000 mon.b (mon.1) 30 : audit [DBG] from='mgr.? 172.21.15.196:0/3033848973' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-23T00:39:41.367 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:41.110140+0000 mon.a (mon.0) 624 : cluster [INF] Active manager daemon y restarted 2024-02-23T00:39:41.367 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:41.110430+0000 mon.a (mon.0) 625 : cluster [INF] Activating manager daemon y 2024-02-23T00:39:41.367 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:41.116501+0000 mon.a (mon.0) 626 : cluster [DBG] osdmap e73: 8 total, 8 up, 8 in 2024-02-23T00:39:41.367 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:41.116925+0000 mon.a (mon.0) 627 : cluster [DBG] mgrmap e17: y(active, starting, since 0.00657908s), standbys: x 2024-02-23T00:39:41.367 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:41.120619+0000 mon.a (mon.0) 628 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-23T00:39:41.367 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:41.120759+0000 mon.a (mon.0) 629 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-23T00:39:41.367 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:41.120899+0000 mon.a (mon.0) 630 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-23T00:39:41.367 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:41.121631+0000 mon.a (mon.0) 631 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-23T00:39:41.367 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:41.121837+0000 mon.a (mon.0) 632 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-23T00:39:41.367 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:41.122008+0000 mon.a (mon.0) 633 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-23T00:39:41.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:41.122212+0000 mon.a (mon.0) 634 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-23T00:39:41.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:41.122478+0000 mon.a (mon.0) 635 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-23T00:39:41.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:41.122769+0000 mon.a (mon.0) 636 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-23T00:39:41.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:41.123048+0000 mon.a (mon.0) 637 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-23T00:39:41.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:41.123369+0000 mon.a (mon.0) 638 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-23T00:39:41.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:41.123577+0000 mon.a (mon.0) 639 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-23T00:39:41.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:41.123801+0000 mon.a (mon.0) 640 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-23T00:39:41.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:41.123992+0000 mon.a (mon.0) 641 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-23T00:39:41.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:41.124152+0000 mon.a (mon.0) 642 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-23T00:39:41.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:41.124575+0000 mon.a (mon.0) 643 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-23T00:39:41.369 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:41.138632+0000 mon.a (mon.0) 644 : cluster [INF] Manager daemon y is now available 2024-02-23T00:39:41.369 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:41.241172+0000 mon.a (mon.0) 645 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:39:41.369 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:41.242649+0000 mon.a (mon.0) 646 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:39:41.369 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:41.261335+0000 mon.a (mon.0) 647 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:39:41.369 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:41.290030+0000 mon.a (mon.0) 648 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-23T00:39:41.369 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:41.122212+0000 mon.a (mon.0) 634 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-23T00:39:41.369 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:41.122478+0000 mon.a (mon.0) 635 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-23T00:39:41.369 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:41.122769+0000 mon.a (mon.0) 636 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-23T00:39:41.369 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:41.123048+0000 mon.a (mon.0) 637 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-23T00:39:41.369 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:41.123369+0000 mon.a (mon.0) 638 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-23T00:39:41.369 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:41.123577+0000 mon.a (mon.0) 639 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-23T00:39:41.369 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:41.123801+0000 mon.a (mon.0) 640 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-23T00:39:41.370 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:41.123992+0000 mon.a (mon.0) 641 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-23T00:39:41.370 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:41.124152+0000 mon.a (mon.0) 642 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-23T00:39:41.370 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:41.124575+0000 mon.a (mon.0) 643 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-23T00:39:41.370 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:41.138632+0000 mon.a (mon.0) 644 : cluster [INF] Manager daemon y is now available 2024-02-23T00:39:41.370 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:41.241172+0000 mon.a (mon.0) 645 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:39:41.370 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:41.242649+0000 mon.a (mon.0) 646 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:39:41.370 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:41.261335+0000 mon.a (mon.0) 647 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:39:41.370 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:41.290030+0000 mon.a (mon.0) 648 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-23T00:39:41.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:41.632+0000 7f77155c4700 10 client.24439.objecter tick 2024-02-23T00:39:41.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:40.848739+0000 mon.a (mon.0) 622 : cluster [DBG] Standby manager daemon x restarted 2024-02-23T00:39:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:40.848844+0000 mon.a (mon.0) 623 : cluster [DBG] Standby manager daemon x started 2024-02-23T00:39:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:40.850498+0000 mon.b (mon.1) 27 : audit [DBG] from='mgr.? 172.21.15.196:0/3033848973' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-23T00:39:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:40.850989+0000 mon.b (mon.1) 28 : audit [DBG] from='mgr.? 172.21.15.196:0/3033848973' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-23T00:39:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:40.852785+0000 mon.b (mon.1) 29 : audit [DBG] from='mgr.? 172.21.15.196:0/3033848973' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-23T00:39:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:40.853715+0000 mon.b (mon.1) 30 : audit [DBG] from='mgr.? 172.21.15.196:0/3033848973' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-23T00:39:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:41.110140+0000 mon.a (mon.0) 624 : cluster [INF] Active manager daemon y restarted 2024-02-23T00:39:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:41.110430+0000 mon.a (mon.0) 625 : cluster [INF] Activating manager daemon y 2024-02-23T00:39:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:41.116501+0000 mon.a (mon.0) 626 : cluster [DBG] osdmap e73: 8 total, 8 up, 8 in 2024-02-23T00:39:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:41.116925+0000 mon.a (mon.0) 627 : cluster [DBG] mgrmap e17: y(active, starting, since 0.00657908s), standbys: x 2024-02-23T00:39:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:41.120619+0000 mon.a (mon.0) 628 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-23T00:39:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:41.120759+0000 mon.a (mon.0) 629 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-23T00:39:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:41.120899+0000 mon.a (mon.0) 630 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-23T00:39:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:41.121631+0000 mon.a (mon.0) 631 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-23T00:39:41.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:41.121837+0000 mon.a (mon.0) 632 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-23T00:39:41.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:41.122008+0000 mon.a (mon.0) 633 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-23T00:39:41.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:41.122212+0000 mon.a (mon.0) 634 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-23T00:39:41.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:41.122478+0000 mon.a (mon.0) 635 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-23T00:39:41.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:41.122769+0000 mon.a (mon.0) 636 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-23T00:39:41.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:41.123048+0000 mon.a (mon.0) 637 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-23T00:39:41.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:41.123369+0000 mon.a (mon.0) 638 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-23T00:39:41.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:41.123577+0000 mon.a (mon.0) 639 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-23T00:39:41.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:41.123801+0000 mon.a (mon.0) 640 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-23T00:39:41.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:41.123992+0000 mon.a (mon.0) 641 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-23T00:39:41.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:41.124152+0000 mon.a (mon.0) 642 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-23T00:39:41.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:41.124575+0000 mon.a (mon.0) 643 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-23T00:39:41.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:41.138632+0000 mon.a (mon.0) 644 : cluster [INF] Manager daemon y is now available 2024-02-23T00:39:41.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:41.241172+0000 mon.a (mon.0) 645 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:39:41.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:41.242649+0000 mon.a (mon.0) 646 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:39:41.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:41.261335+0000 mon.a (mon.0) 647 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:39:41.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:41.290030+0000 mon.a (mon.0) 648 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-23T00:39:41.806 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: CherryPy Checker: 2024-02-23T00:39:41.806 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: The Application mounted at '' has an empty config. 2024-02-23T00:39:41.806 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: 2024-02-23T00:39:41.806 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:39:41] ENGINE Serving on http://:::9283 2024-02-23T00:39:41.806 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:39:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:39:41] ENGINE Bus STARTED 2024-02-23T00:39:42.121 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:42.118+0000 7f770cff9700 1 -- 172.21.15.196:0/2716205704 <== mon.1 v2:172.21.15.196:3300/0 8 ==== mgrmap(e 18) v1 ==== 76929+0+0 (secure 0 0 0) 0x7f770007bd90 con 0x7f771006b5c0 2024-02-23T00:39:42.121 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:42.119+0000 7f770cff9700 1 --2- 172.21.15.196:0/2716205704 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f76f80323e0 0x7f76f806e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:42.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:42.119+0000 7f770cff9700 1 -- 172.21.15.196:0/2716205704 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi196=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f77100f62e0 con 0x7f76f80323e0 2024-02-23T00:39:42.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:42.119+0000 7f770effd700 1 --2- 172.21.15.196:0/2716205704 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f76f80323e0 0x7f76f806e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:42.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:42.120+0000 7f770cff9700 10 client.24439.objecter ms_handle_connect 0x7f76f80323e0 2024-02-23T00:39:42.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:42.120+0000 7f770effd700 1 --2- 172.21.15.196:0/2716205704 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f76f80323e0 0x7f76f806e790 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f770400b010 tx=0x7f770400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:39:42.130 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:42.128+0000 7f770cff9700 1 -- 172.21.15.196:0/2716205704 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f77100f62e0 con 0x7f76f80323e0 2024-02-23T00:39:42.131 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled grafana update... 2024-02-23T00:39:42.133 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:42.130+0000 7f7715dc5700 10 librados: watch_flush enter 2024-02-23T00:39:42.133 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:42.131+0000 7f7715dc5700 10 librados: watch_flush exit 2024-02-23T00:39:42.133 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:42.131+0000 7f7715dc5700 20 client.24439.objecter shutdown clearing up homeless session... 2024-02-23T00:39:42.133 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:42.131+0000 7f7715dc5700 10 client.24439.objecter successfully canceled tick 2024-02-23T00:39:42.133 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:42.131+0000 7f7715dc5700 1 -- 172.21.15.196:0/2716205704 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f76f80323e0 msgr2=0x7f76f806e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:42.133 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:42.131+0000 7f7715dc5700 1 --2- 172.21.15.196:0/2716205704 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f76f80323e0 0x7f76f806e790 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f770400b010 tx=0x7f770400b040).stop 2024-02-23T00:39:42.133 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:42.131+0000 7f7715dc5700 1 -- 172.21.15.196:0/2716205704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f771006b5c0 msgr2=0x7f771010a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:42.133 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:42.131+0000 7f7715dc5700 1 --2- 172.21.15.196:0/2716205704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f771006b5c0 0x7f771010a170 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f7700013780 tx=0x7f7700009cb0).stop 2024-02-23T00:39:42.133 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:42.131+0000 7f770ffff700 1 -- 172.21.15.196:0/2716205704 reap_dead start 2024-02-23T00:39:42.134 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:42.131+0000 7f7715dc5700 1 -- 172.21.15.196:0/2716205704 shutdown_connections 2024-02-23T00:39:42.134 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:42.131+0000 7f7715dc5700 1 -- 172.21.15.196:0/2716205704 >> 172.21.15.196:0/2716205704 conn(0x7f77100f32a0 msgr2=0x7f77100f46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:39:42.134 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:42.131+0000 7f7715dc5700 1 -- 172.21.15.196:0/2716205704 shutdown_connections 2024-02-23T00:39:42.134 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:42.131+0000 7f7715dc5700 1 -- 172.21.15.196:0/2716205704 wait complete. 2024-02-23T00:39:42.134 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:42.131+0000 7f7715dc5700 1 librados: shutdown 2024-02-23T00:39:42.443 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:39:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:39:42.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:41.308981+0000 mon.a (mon.0) 649 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-23T00:39:42.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:42.120153+0000 mon.a (mon.0) 650 : cluster [DBG] mgrmap e18: y(active, since 1.0098s), standbys: x 2024-02-23T00:39:42.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:42.129608+0000 mon.a (mon.0) 651 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:39:42.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:41.308981+0000 mon.a (mon.0) 649 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-23T00:39:42.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:42.120153+0000 mon.a (mon.0) 650 : cluster [DBG] mgrmap e18: y(active, since 1.0098s), standbys: x 2024-02-23T00:39:42.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:42.129608+0000 mon.a (mon.0) 651 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:39:42.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:41.308981+0000 mon.a (mon.0) 649 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-23T00:39:42.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:42.120153+0000 mon.a (mon.0) 650 : cluster [DBG] mgrmap e18: y(active, since 1.0098s), standbys: x 2024-02-23T00:39:42.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:42.129608+0000 mon.a (mon.0) 651 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:39:43.102 DEBUG:teuthology.orchestra.run.smithi196:grafana.a> sudo journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@grafana.a.service 2024-02-23T00:39:43.105 INFO:tasks.cephadm:Setting up client nodes... 2024-02-23T00:39:43.105 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-23T00:39:43.138 INFO:journalctl@ceph.grafana.a.smithi196.stdout:-- Logs begin at Fri 2024-02-23 00:13:21 UTC. -- 2024-02-23T00:39:43.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:39:42.125810+0000 mgr.y (mgr.14517) 3 : cephadm [INF] Saving service grafana spec with placement smithi196=a;count:1 2024-02-23T00:39:43.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:42.126011+0000 mgr.y (mgr.14517) 4 : cluster [DBG] pgmap v3: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:39:43.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:42.969057+0000 mon.a (mon.0) 652 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:39:43.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:43.263685+0000 mon.a (mon.0) 653 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:43.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:43.264436+0000 mon.a (mon.0) 654 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:43.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:43.265033+0000 mon.a (mon.0) 655 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:43.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:43.265631+0000 mon.a (mon.0) 656 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:43.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:43.270210+0000 mon.a (mon.0) 657 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:39:43.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:39:42.125810+0000 mgr.y (mgr.14517) 3 : cephadm [INF] Saving service grafana spec with placement smithi196=a;count:1 2024-02-23T00:39:43.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:42.126011+0000 mgr.y (mgr.14517) 4 : cluster [DBG] pgmap v3: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:39:43.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:42.969057+0000 mon.a (mon.0) 652 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:39:43.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:43.263685+0000 mon.a (mon.0) 653 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:43.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:43.264436+0000 mon.a (mon.0) 654 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:43.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:43.265033+0000 mon.a (mon.0) 655 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:43.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:43.265631+0000 mon.a (mon.0) 656 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:43.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:43.270210+0000 mon.a (mon.0) 657 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:39:43.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:39:42.125810+0000 mgr.y (mgr.14517) 3 : cephadm [INF] Saving service grafana spec with placement smithi196=a;count:1 2024-02-23T00:39:43.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:42.126011+0000 mgr.y (mgr.14517) 4 : cluster [DBG] pgmap v3: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:39:43.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:42.969057+0000 mon.a (mon.0) 652 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:39:43.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:43.263685+0000 mon.a (mon.0) 653 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:43.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:43.264436+0000 mon.a (mon.0) 654 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:43.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:43.265033+0000 mon.a (mon.0) 655 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:43.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:43.265631+0000 mon.a (mon.0) 656 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:43.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:43.270210+0000 mon.a (mon.0) 657 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:39:44.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:43.123121+0000 mgr.y (mgr.14517) 5 : cluster [DBG] pgmap v4: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:39:44.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:39:43.266136+0000 mgr.y (mgr.14517) 6 : cephadm [INF] Adjusting osd_memory_target on smithi047 to 3769M 2024-02-23T00:39:44.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:44.125347+0000 mon.a (mon.0) 658 : cluster [DBG] mgrmap e19: y(active, since 3s), standbys: x 2024-02-23T00:39:44.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:44.296066+0000 mon.a (mon.0) 659 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:39:44.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:44.311892+0000 mon.a (mon.0) 660 : cluster [DBG] osdmap e74: 8 total, 8 up, 8 in 2024-02-23T00:39:44.755 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:43.123121+0000 mgr.y (mgr.14517) 5 : cluster [DBG] pgmap v4: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:39:44.755 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:39:43.266136+0000 mgr.y (mgr.14517) 6 : cephadm [INF] Adjusting osd_memory_target on smithi047 to 3769M 2024-02-23T00:39:44.756 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:44.125347+0000 mon.a (mon.0) 658 : cluster [DBG] mgrmap e19: y(active, since 3s), standbys: x 2024-02-23T00:39:44.756 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:44.296066+0000 mon.a (mon.0) 659 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:39:44.756 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:44.311892+0000 mon.a (mon.0) 660 : cluster [DBG] osdmap e74: 8 total, 8 up, 8 in 2024-02-23T00:39:44.756 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:43.123121+0000 mgr.y (mgr.14517) 5 : cluster [DBG] pgmap v4: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:39:44.756 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:39:43.266136+0000 mgr.y (mgr.14517) 6 : cephadm [INF] Adjusting osd_memory_target on smithi047 to 3769M 2024-02-23T00:39:44.756 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:44.125347+0000 mon.a (mon.0) 658 : cluster [DBG] mgrmap e19: y(active, since 3s), standbys: x 2024-02-23T00:39:44.756 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:44.296066+0000 mon.a (mon.0) 659 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:39:44.756 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:44.311892+0000 mon.a (mon.0) 660 : cluster [DBG] osdmap e74: 8 total, 8 up, 8 in 2024-02-23T00:39:45.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:44.313421+0000 mon.a (mon.0) 661 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-23T00:39:45.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:44.317239+0000 mon.a (mon.0) 662 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-23T00:39:45.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:44.581430+0000 mon.a (mon.0) 663 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:45.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:44.582234+0000 mon.a (mon.0) 664 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:45.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:44.582891+0000 mon.a (mon.0) 665 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:45.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:44.583553+0000 mon.a (mon.0) 666 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:45.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:44.588424+0000 mon.a (mon.0) 667 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:39:45.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:44.589355+0000 mon.a (mon.0) 668 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:39:45.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:44.590095+0000 mon.a (mon.0) 669 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:39:45.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:44.665538+0000 mon.a (mon.0) 670 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:39:45.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:44.700687+0000 mon.a (mon.0) 671 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:39:45.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:44.704761+0000 mon.a (mon.0) 672 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:39:45.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.692+0000 7fbf7b11f700 1 -- 172.21.15.47:0/3283723037 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbf740fe480 msgr2=0x7fbf740f7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:45.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.692+0000 7fbf7b11f700 1 --2- 172.21.15.47:0/3283723037 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbf740fe480 0x7fbf740f7cb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fbf60004870 tx=0x7fbf60009cb0).stop 2024-02-23T00:39:45.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.692+0000 7fbf7b11f700 1 -- 172.21.15.47:0/3283723037 shutdown_connections 2024-02-23T00:39:45.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.692+0000 7fbf7b11f700 1 --2- 172.21.15.47:0/3283723037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf740f8390 0x7fbf74104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:45.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.692+0000 7fbf7b11f700 1 --2- 172.21.15.47:0/3283723037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf740fdb20 0x7fbf740fdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:45.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.692+0000 7fbf7b11f700 1 --2- 172.21.15.47:0/3283723037 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbf740fe480 0x7fbf740f7cb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:45.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.692+0000 7fbf7b11f700 1 -- 172.21.15.47:0/3283723037 >> 172.21.15.47:0/3283723037 conn(0x7fbf740f3770 msgr2=0x7fbf740f5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:39:45.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.692+0000 7fbf7b11f700 1 -- 172.21.15.47:0/3283723037 shutdown_connections 2024-02-23T00:39:45.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.692+0000 7fbf7b11f700 1 -- 172.21.15.47:0/3283723037 wait complete. 2024-02-23T00:39:45.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.693+0000 7fbf7b11f700 1 librados: starting msgr at 2024-02-23T00:39:45.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.693+0000 7fbf7b11f700 1 librados: starting objecter 2024-02-23T00:39:45.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.693+0000 7fbf7b11f700 1 Processor -- start 2024-02-23T00:39:45.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.693+0000 7fbf7b11f700 1 -- start start 2024-02-23T00:39:45.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.693+0000 7fbf7b11f700 1 librados: setting wanted keys 2024-02-23T00:39:45.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.693+0000 7fbf7b11f700 1 librados: calling monclient init 2024-02-23T00:39:45.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.693+0000 7fbf7b11f700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbf740f8390 0x7fbf7406b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:45.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.693+0000 7fbf7b11f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf740fdb20 0x7fbf7406b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:45.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.693+0000 7fbf7b11f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf740fe480 0x7fbf7406bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:45.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.693+0000 7fbf7b11f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf741162d0 con 0x7fbf740fe480 2024-02-23T00:39:45.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.693+0000 7fbf7b11f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf74116440 con 0x7fbf740fdb20 2024-02-23T00:39:45.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.693+0000 7fbf7b11f700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf74116600 con 0x7fbf740f8390 2024-02-23T00:39:45.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.693+0000 7fbf78ebb700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbf740f8390 0x7fbf7406b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:45.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.693+0000 7fbf78ebb700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbf740f8390 0x7fbf7406b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:50170/0 (socket says 172.21.15.47:50170) 2024-02-23T00:39:45.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.693+0000 7fbf78ebb700 1 -- 172.21.15.47:0/1751455017 learned_addr learned my addr 172.21.15.47:0/1751455017 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:39:45.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.693+0000 7fbf796bc700 1 --2- 172.21.15.47:0/1751455017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf740fe480 0x7fbf7406bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:45.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.693+0000 7fbf73fff700 1 --2- 172.21.15.47:0/1751455017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf740fdb20 0x7fbf7406b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:45.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.694+0000 7fbf78ebb700 1 -- 172.21.15.47:0/1751455017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf740fdb20 msgr2=0x7fbf7406b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:45.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.694+0000 7fbf78ebb700 1 --2- 172.21.15.47:0/1751455017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf740fdb20 0x7fbf7406b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:45.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.694+0000 7fbf78ebb700 1 -- 172.21.15.47:0/1751455017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf740fe480 msgr2=0x7fbf7406bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:45.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.694+0000 7fbf78ebb700 1 --2- 172.21.15.47:0/1751455017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf740fe480 0x7fbf7406bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:45.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.694+0000 7fbf78ebb700 1 -- 172.21.15.47:0/1751455017 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf6c0097c0 con 0x7fbf740f8390 2024-02-23T00:39:45.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.694+0000 7fbf796bc700 1 --2- 172.21.15.47:0/1751455017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf740fe480 0x7fbf7406bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:39:45.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.694+0000 7fbf78ebb700 1 --2- 172.21.15.47:0/1751455017 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbf740f8390 0x7fbf7406b0d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fbf680041d0 tx=0x7fbf68009f60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:39:45.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.694+0000 7fbf71ffb700 10 client.?.objecter ms_handle_connect 0x7fbf740f8390 2024-02-23T00:39:45.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.694+0000 7fbf71ffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:39:45.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.694+0000 7fbf71ffb700 1 -- 172.21.15.47:0/1751455017 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf68007b40 con 0x7fbf740f8390 2024-02-23T00:39:45.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.694+0000 7fbf7b11f700 1 -- 172.21.15.47:0/1751455017 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf60005040 con 0x7fbf740f8390 2024-02-23T00:39:45.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.694+0000 7fbf71ffb700 1 -- 172.21.15.47:0/1751455017 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fbf6800f040 con 0x7fbf740f8390 2024-02-23T00:39:45.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.694+0000 7fbf71ffb700 1 -- 172.21.15.47:0/1751455017 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf68014690 con 0x7fbf740f8390 2024-02-23T00:39:45.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.694+0000 7fbf7b11f700 10 client.24434.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:39:45.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.694+0000 7fbf7b11f700 1 -- 172.21.15.47:0/1751455017 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf74116ce0 con 0x7fbf740f8390 2024-02-23T00:39:45.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.694+0000 7fbf7b11f700 1 librados: init done 2024-02-23T00:39:45.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.695+0000 7fbf7b11f700 1 -- 172.21.15.47:0/1751455017 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf74060b00 con 0x7fbf740f8390 2024-02-23T00:39:45.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.696+0000 7fbf71ffb700 1 -- 172.21.15.47:0/1751455017 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fbf68014ac0 con 0x7fbf740f8390 2024-02-23T00:39:45.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.696+0000 7fbf71ffb700 1 --2- 172.21.15.47:0/1751455017 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbf64062cc0 0x7fbf64065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:45.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.696+0000 7fbf71ffb700 1 -- 172.21.15.47:0/1751455017 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7305+0+0 (secure 0 0 0) 0x7fbf6807d2e0 con 0x7fbf740f8390 2024-02-23T00:39:45.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.696+0000 7fbf71ffb700 10 client.24434.objecter ms_dispatch 0x7fbf74001550 osd_map(75..75 src has 1..75) v4 2024-02-23T00:39:45.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.696+0000 7fbf71ffb700 3 client.24434.objecter handle_osd_map got epochs [75,75] > 0 2024-02-23T00:39:45.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.696+0000 7fbf71ffb700 3 client.24434.objecter handle_osd_map decoding full epoch 75 2024-02-23T00:39:45.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.696+0000 7fbf73fff700 1 --2- 172.21.15.47:0/1751455017 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbf64062cc0 0x7fbf64065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:45.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.697+0000 7fbf71ffb700 20 client.24434.objecter dump_active .. 0 homeless 2024-02-23T00:39:45.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.697+0000 7fbf71ffb700 10 client.24434.objecter ms_handle_connect 0x7fbf64062cc0 2024-02-23T00:39:45.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.697+0000 7fbf73fff700 1 --2- 172.21.15.47:0/1751455017 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbf64062cc0 0x7fbf64065180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fbf60004ee0 tx=0x7fbf6000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:39:45.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.700+0000 7fbf71ffb700 1 -- 172.21.15.47:0/1751455017 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fbf6804d150 con 0x7fbf740f8390 2024-02-23T00:39:45.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:44.313421+0000 mon.a (mon.0) 661 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-23T00:39:45.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:44.317239+0000 mon.a (mon.0) 662 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-23T00:39:45.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:44.581430+0000 mon.a (mon.0) 663 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:45.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:44.582234+0000 mon.a (mon.0) 664 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:45.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:44.582891+0000 mon.a (mon.0) 665 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:45.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:44.583553+0000 mon.a (mon.0) 666 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:45.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:44.588424+0000 mon.a (mon.0) 667 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:39:45.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:44.589355+0000 mon.a (mon.0) 668 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:39:45.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:44.590095+0000 mon.a (mon.0) 669 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:39:45.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:44.665538+0000 mon.a (mon.0) 670 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:39:45.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:44.700687+0000 mon.a (mon.0) 671 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:39:45.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:44.704761+0000 mon.a (mon.0) 672 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:39:45.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:44.313421+0000 mon.a (mon.0) 661 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-23T00:39:45.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:44.317239+0000 mon.a (mon.0) 662 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-23T00:39:45.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:44.581430+0000 mon.a (mon.0) 663 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:45.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:44.582234+0000 mon.a (mon.0) 664 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:45.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:44.582891+0000 mon.a (mon.0) 665 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:45.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:44.583553+0000 mon.a (mon.0) 666 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:39:45.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:44.588424+0000 mon.a (mon.0) 667 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:39:45.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:44.589355+0000 mon.a (mon.0) 668 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:39:45.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:44.590095+0000 mon.a (mon.0) 669 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:39:45.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:44.665538+0000 mon.a (mon.0) 670 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:39:45.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:44.700687+0000 mon.a (mon.0) 671 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:39:45.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:44.704761+0000 mon.a (mon.0) 672 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:39:45.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.958+0000 7fbf7b11f700 1 -- 172.21.15.47:0/1751455017 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fbf7405f360 con 0x7fbf740f8390 2024-02-23T00:39:45.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.962+0000 7fbf71ffb700 1 -- 172.21.15.47:0/1751455017 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7fbf68050770 con 0x7fbf740f8390 2024-02-23T00:39:45.964 INFO:teuthology.orchestra.run.smithi047.stdout:[client.0] 2024-02-23T00:39:45.964 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQBR6ddl75gwORAAraTVuX3IJmQyvlKYL5o2Eg== 2024-02-23T00:39:45.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.965+0000 7fbf7b11f700 10 librados: watch_flush enter 2024-02-23T00:39:45.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.965+0000 7fbf7b11f700 10 librados: watch_flush exit 2024-02-23T00:39:45.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.965+0000 7fbf7b11f700 20 client.24434.objecter shutdown clearing up homeless session... 2024-02-23T00:39:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.965+0000 7fbf7b11f700 10 client.24434.objecter successfully canceled tick 2024-02-23T00:39:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.965+0000 7fbf7b11f700 1 -- 172.21.15.47:0/1751455017 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbf64062cc0 msgr2=0x7fbf64065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.965+0000 7fbf7b11f700 1 --2- 172.21.15.47:0/1751455017 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbf64062cc0 0x7fbf64065180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fbf60004ee0 tx=0x7fbf6000b040).stop 2024-02-23T00:39:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.965+0000 7fbf7b11f700 1 -- 172.21.15.47:0/1751455017 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbf740f8390 msgr2=0x7fbf7406b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.965+0000 7fbf7b11f700 1 --2- 172.21.15.47:0/1751455017 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbf740f8390 0x7fbf7406b0d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fbf680041d0 tx=0x7fbf68009f60).stop 2024-02-23T00:39:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.965+0000 7fbf7b11f700 1 -- 172.21.15.47:0/1751455017 shutdown_connections 2024-02-23T00:39:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.965+0000 7fbf7b11f700 1 --2- 172.21.15.47:0/1751455017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf740fe480 0x7fbf7406bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.965+0000 7fbf7b11f700 1 --2- 172.21.15.47:0/1751455017 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbf64062cc0 0x7fbf64065180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.965+0000 7fbf7b11f700 1 --2- 172.21.15.47:0/1751455017 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbf740f8390 0x7fbf7406b0d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.965+0000 7fbf7b11f700 1 --2- 172.21.15.47:0/1751455017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf740fdb20 0x7fbf7406b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.965+0000 7fbf7b11f700 1 -- 172.21.15.47:0/1751455017 >> 172.21.15.47:0/1751455017 conn(0x7fbf740f3770 msgr2=0x7fbf740fbf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:39:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.965+0000 7fbf7b11f700 1 -- 172.21.15.47:0/1751455017 shutdown_connections 2024-02-23T00:39:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.965+0000 7fbf7b11f700 1 -- 172.21.15.47:0/1751455017 wait complete. 2024-02-23T00:39:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:45.965+0000 7fbf7b11f700 1 librados: shutdown 2024-02-23T00:39:46.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:39:44.584060+0000 mgr.y (mgr.14517) 7 : cephadm [INF] Adjusting osd_memory_target on smithi196 to 4024M 2024-02-23T00:39:46.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:39:44.591293+0000 mgr.y (mgr.14517) 8 : cephadm [INF] Updating smithi047:/etc/ceph/ceph.conf 2024-02-23T00:39:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:39:44.591584+0000 mgr.y (mgr.14517) 9 : cephadm [INF] Updating smithi196:/etc/ceph/ceph.conf 2024-02-23T00:39:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:39:44.628715+0000 mgr.y (mgr.14517) 10 : cephadm [INF] Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-02-23T00:39:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:39:44.643834+0000 mgr.y (mgr.14517) 11 : cephadm [INF] Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-02-23T00:39:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:39:44.715909+0000 mgr.y (mgr.14517) 12 : cephadm [INF] Deploying daemon alertmanager.a on smithi047 2024-02-23T00:39:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:45.123648+0000 mgr.y (mgr.14517) 13 : cluster [DBG] pgmap v6: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:39:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:45.318321+0000 mon.a (mon.0) 673 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-23T00:39:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:45.318410+0000 mon.a (mon.0) 674 : cluster [DBG] osdmap e75: 8 total, 8 up, 8 in 2024-02-23T00:39:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:45.958994+0000 mon.c (mon.2) 17 : audit [INF] from='client.? 172.21.15.47:0/1751455017' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-23T00:39:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:45.959327+0000 mon.a (mon.0) 675 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-23T00:39:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:45.962545+0000 mon.a (mon.0) 676 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-23T00:39:46.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:39:44.584060+0000 mgr.y (mgr.14517) 7 : cephadm [INF] Adjusting osd_memory_target on smithi196 to 4024M 2024-02-23T00:39:46.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:39:44.591293+0000 mgr.y (mgr.14517) 8 : cephadm [INF] Updating smithi047:/etc/ceph/ceph.conf 2024-02-23T00:39:46.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:39:44.591584+0000 mgr.y (mgr.14517) 9 : cephadm [INF] Updating smithi196:/etc/ceph/ceph.conf 2024-02-23T00:39:46.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:39:44.628715+0000 mgr.y (mgr.14517) 10 : cephadm [INF] Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-02-23T00:39:46.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:39:44.643834+0000 mgr.y (mgr.14517) 11 : cephadm [INF] Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-02-23T00:39:46.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:39:44.715909+0000 mgr.y (mgr.14517) 12 : cephadm [INF] Deploying daemon alertmanager.a on smithi047 2024-02-23T00:39:46.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:45.123648+0000 mgr.y (mgr.14517) 13 : cluster [DBG] pgmap v6: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:39:46.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:45.318321+0000 mon.a (mon.0) 673 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-23T00:39:46.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:45.318410+0000 mon.a (mon.0) 674 : cluster [DBG] osdmap e75: 8 total, 8 up, 8 in 2024-02-23T00:39:46.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:45.958994+0000 mon.c (mon.2) 17 : audit [INF] from='client.? 172.21.15.47:0/1751455017' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-23T00:39:46.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:45.959327+0000 mon.a (mon.0) 675 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-23T00:39:46.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:45.962545+0000 mon.a (mon.0) 676 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-23T00:39:46.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:39:44.584060+0000 mgr.y (mgr.14517) 7 : cephadm [INF] Adjusting osd_memory_target on smithi196 to 4024M 2024-02-23T00:39:46.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:39:44.591293+0000 mgr.y (mgr.14517) 8 : cephadm [INF] Updating smithi047:/etc/ceph/ceph.conf 2024-02-23T00:39:46.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:39:44.591584+0000 mgr.y (mgr.14517) 9 : cephadm [INF] Updating smithi196:/etc/ceph/ceph.conf 2024-02-23T00:39:46.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:39:44.628715+0000 mgr.y (mgr.14517) 10 : cephadm [INF] Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-02-23T00:39:46.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:39:44.643834+0000 mgr.y (mgr.14517) 11 : cephadm [INF] Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-02-23T00:39:46.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:39:44.715909+0000 mgr.y (mgr.14517) 12 : cephadm [INF] Deploying daemon alertmanager.a on smithi047 2024-02-23T00:39:46.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:45.123648+0000 mgr.y (mgr.14517) 13 : cluster [DBG] pgmap v6: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:39:46.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:45.318321+0000 mon.a (mon.0) 673 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-23T00:39:46.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:45.318410+0000 mon.a (mon.0) 674 : cluster [DBG] osdmap e75: 8 total, 8 up, 8 in 2024-02-23T00:39:46.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:45.958994+0000 mon.c (mon.2) 17 : audit [INF] from='client.? 172.21.15.47:0/1751455017' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-23T00:39:46.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:45.959327+0000 mon.a (mon.0) 675 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-23T00:39:46.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:45.962545+0000 mon.a (mon.0) 676 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-23T00:39:46.870 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-02-23T00:39:46.871 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-23T00:39:46.871 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-23T00:39:46.916 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-23T00:39:48.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:47.124065+0000 mgr.y (mgr.14517) 14 : cluster [DBG] pgmap v8: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:39:48.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:47.131132+0000 osd.3 (osd.3) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:39:48.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:47.131219+0000 osd.3 (osd.3) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:39:48.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:47.327662+0000 mon.a (mon.0) 677 : cluster [DBG] osdmap e76: 8 total, 8 up, 8 in 2024-02-23T00:39:48.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:47.124065+0000 mgr.y (mgr.14517) 14 : cluster [DBG] pgmap v8: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:39:48.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:47.131132+0000 osd.3 (osd.3) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:39:48.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:47.131219+0000 osd.3 (osd.3) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:39:48.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:47.327662+0000 mon.a (mon.0) 677 : cluster [DBG] osdmap e76: 8 total, 8 up, 8 in 2024-02-23T00:39:48.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:47.124065+0000 mgr.y (mgr.14517) 14 : cluster [DBG] pgmap v8: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:39:48.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:47.131132+0000 osd.3 (osd.3) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:39:48.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:47.131219+0000 osd.3 (osd.3) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:39:48.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:47.327662+0000 mon.a (mon.0) 677 : cluster [DBG] osdmap e76: 8 total, 8 up, 8 in 2024-02-23T00:39:49.153 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.146+0000 7f54b39b5700 1 -- 172.21.15.196:0/116037512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54ac0f72a0 msgr2=0x7f54ac1038a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:49.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.146+0000 7f54b39b5700 1 --2- 172.21.15.196:0/116037512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54ac0f72a0 0x7f54ac1038a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f54a4009be0 tx=0x7f54a4004d10).stop 2024-02-23T00:39:49.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.146+0000 7f54b39b5700 1 -- 172.21.15.196:0/116037512 shutdown_connections 2024-02-23T00:39:49.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.146+0000 7f54b39b5700 1 --2- 172.21.15.196:0/116037512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54ac0f72a0 0x7f54ac1038a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:49.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.146+0000 7f54b39b5700 1 --2- 172.21.15.196:0/116037512 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f54ac0fca30 0x7f54ac0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:49.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.146+0000 7f54b39b5700 1 --2- 172.21.15.196:0/116037512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54ac0fd390 0x7f54ac0f6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:49.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.146+0000 7f54b39b5700 1 -- 172.21.15.196:0/116037512 >> 172.21.15.196:0/116037512 conn(0x7f54ac0f2640 msgr2=0x7f54ac0f4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:39:49.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.146+0000 7f54b39b5700 1 -- 172.21.15.196:0/116037512 shutdown_connections 2024-02-23T00:39:49.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.146+0000 7f54b39b5700 1 -- 172.21.15.196:0/116037512 wait complete. 2024-02-23T00:39:49.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.146+0000 7f54b39b5700 1 librados: starting msgr at 2024-02-23T00:39:49.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.146+0000 7f54b39b5700 1 librados: starting objecter 2024-02-23T00:39:49.155 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.146+0000 7f54b39b5700 1 Processor -- start 2024-02-23T00:39:49.155 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.146+0000 7f54b39b5700 1 -- start start 2024-02-23T00:39:49.155 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.146+0000 7f54b39b5700 1 librados: setting wanted keys 2024-02-23T00:39:49.155 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.146+0000 7f54b39b5700 1 librados: calling monclient init 2024-02-23T00:39:49.155 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.146+0000 7f54b39b5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54ac0f72a0 0x7f54ac10cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:49.155 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.147+0000 7f54b39b5700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f54ac0fca30 0x7f54ac10cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:49.155 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.147+0000 7f54b39b5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54ac0fd390 0x7f54ac106c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:49.155 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.147+0000 7f54b39b5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac107150 con 0x7f54ac0f72a0 2024-02-23T00:39:49.155 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.147+0000 7f54b39b5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac1072c0 con 0x7f54ac0fd390 2024-02-23T00:39:49.155 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.147+0000 7f54b39b5700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac107480 con 0x7f54ac0fca30 2024-02-23T00:39:49.155 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.147+0000 7f54b1f52700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54ac0fd390 0x7f54ac106c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:49.155 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.147+0000 7f54b1751700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54ac0f72a0 0x7f54ac10cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:49.156 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.147+0000 7f54b1f52700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54ac0fd390 0x7f54ac106c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:37832/0 (socket says 172.21.15.196:37832) 2024-02-23T00:39:49.156 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.147+0000 7f54b1f52700 1 -- 172.21.15.196:0/1757931931 learned_addr learned my addr 172.21.15.196:0/1757931931 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-02-23T00:39:49.156 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.147+0000 7f54b0f50700 1 --2- 172.21.15.196:0/1757931931 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f54ac0fca30 0x7f54ac10cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:49.156 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.147+0000 7f54b1f52700 1 -- 172.21.15.196:0/1757931931 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f54ac0fca30 msgr2=0x7f54ac10cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:49.156 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.147+0000 7f54b1f52700 1 --2- 172.21.15.196:0/1757931931 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f54ac0fca30 0x7f54ac10cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:49.156 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.147+0000 7f54b1f52700 1 -- 172.21.15.196:0/1757931931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54ac0f72a0 msgr2=0x7f54ac10cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:49.156 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.147+0000 7f54b1f52700 1 --2- 172.21.15.196:0/1757931931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54ac0f72a0 0x7f54ac10cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:49.156 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.147+0000 7f54b1f52700 1 -- 172.21.15.196:0/1757931931 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a40097c0 con 0x7f54ac0fd390 2024-02-23T00:39:49.156 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.147+0000 7f54b1f52700 1 --2- 172.21.15.196:0/1757931931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54ac0fd390 0x7f54ac106c60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f54a400ba30 tx=0x7f54a4004d10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:39:49.156 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.147+0000 7f54a27fc700 10 client.?.objecter ms_handle_connect 0x7f54ac0fd390 2024-02-23T00:39:49.156 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.147+0000 7f54a27fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:39:49.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.147+0000 7f54a27fc700 1 -- 172.21.15.196:0/1757931931 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a400e040 con 0x7f54ac0fd390 2024-02-23T00:39:49.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.147+0000 7f54a27fc700 1 -- 172.21.15.196:0/1757931931 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f54a4018040 con 0x7f54ac0fd390 2024-02-23T00:39:49.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.147+0000 7f54b39b5700 1 -- 172.21.15.196:0/1757931931 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54ac107700 con 0x7f54ac0fd390 2024-02-23T00:39:49.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.147+0000 7f54a27fc700 1 -- 172.21.15.196:0/1757931931 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a401c3f0 con 0x7f54ac0fd390 2024-02-23T00:39:49.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.147+0000 7f54b39b5700 10 client.24457.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:39:49.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.147+0000 7f54b39b5700 1 -- 172.21.15.196:0/1757931931 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54ac1138f0 con 0x7f54ac0fd390 2024-02-23T00:39:49.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.147+0000 7f54b39b5700 1 librados: init done 2024-02-23T00:39:49.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.148+0000 7f54a27fc700 1 -- 172.21.15.196:0/1757931931 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f54a4013420 con 0x7f54ac0fd390 2024-02-23T00:39:49.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.149+0000 7f54b39b5700 1 -- 172.21.15.196:0/1757931931 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54ac05f860 con 0x7f54ac0fd390 2024-02-23T00:39:49.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.149+0000 7f54a27fc700 1 --2- 172.21.15.196:0/1757931931 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5498062cc0 0x7f5498065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:49.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.149+0000 7f54a27fc700 1 -- 172.21.15.196:0/1757931931 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7305+0+0 (secure 0 0 0) 0x7f54a407e1e0 con 0x7f54ac0fd390 2024-02-23T00:39:49.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.149+0000 7f54a27fc700 10 client.24457.objecter ms_dispatch 0x7f54ac001550 osd_map(76..76 src has 1..76) v4 2024-02-23T00:39:49.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.149+0000 7f54a27fc700 3 client.24457.objecter handle_osd_map got epochs [76,76] > 0 2024-02-23T00:39:49.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.149+0000 7f54a27fc700 3 client.24457.objecter handle_osd_map decoding full epoch 76 2024-02-23T00:39:49.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.149+0000 7f54b1751700 1 --2- 172.21.15.196:0/1757931931 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5498062cc0 0x7f5498065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:49.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.149+0000 7f54a27fc700 20 client.24457.objecter dump_active .. 0 homeless 2024-02-23T00:39:49.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.149+0000 7f54a27fc700 10 client.24457.objecter ms_handle_connect 0x7f5498062cc0 2024-02-23T00:39:49.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.149+0000 7f54b1751700 1 --2- 172.21.15.196:0/1757931931 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5498062cc0 0x7f5498065180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f549c004de0 tx=0x7f549c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:39:49.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.153+0000 7f54a27fc700 1 -- 172.21.15.196:0/1757931931 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f54a404df70 con 0x7f54ac0fd390 2024-02-23T00:39:49.380 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.379+0000 7f54b39b5700 1 -- 172.21.15.196:0/1757931931 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f54ac105d50 con 0x7f54ac0fd390 2024-02-23T00:39:49.384 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.383+0000 7f54a27fc700 1 -- 172.21.15.196:0/1757931931 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7f54a4021020 con 0x7f54ac0fd390 2024-02-23T00:39:49.384 INFO:teuthology.orchestra.run.smithi196.stdout:[client.1] 2024-02-23T00:39:49.384 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQBV6ddlgq+rFhAAlPiBwl4ZfRkDn0N+l37EwA== 2024-02-23T00:39:49.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.385+0000 7f54b39b5700 10 librados: watch_flush enter 2024-02-23T00:39:49.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.385+0000 7f54b39b5700 10 librados: watch_flush exit 2024-02-23T00:39:49.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.385+0000 7f54b39b5700 20 client.24457.objecter shutdown clearing up homeless session... 2024-02-23T00:39:49.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.385+0000 7f54b39b5700 10 client.24457.objecter successfully canceled tick 2024-02-23T00:39:49.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.385+0000 7f54b39b5700 1 -- 172.21.15.196:0/1757931931 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5498062cc0 msgr2=0x7f5498065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:49.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.385+0000 7f54b39b5700 1 --2- 172.21.15.196:0/1757931931 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5498062cc0 0x7f5498065180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f549c004de0 tx=0x7f549c00b040).stop 2024-02-23T00:39:49.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.385+0000 7f54b39b5700 1 -- 172.21.15.196:0/1757931931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54ac0fd390 msgr2=0x7f54ac106c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:49.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.385+0000 7f54b39b5700 1 --2- 172.21.15.196:0/1757931931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54ac0fd390 0x7f54ac106c60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f54a400ba30 tx=0x7f54a4004d10).stop 2024-02-23T00:39:49.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.385+0000 7f54b39b5700 1 -- 172.21.15.196:0/1757931931 shutdown_connections 2024-02-23T00:39:49.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.385+0000 7f54b39b5700 1 --2- 172.21.15.196:0/1757931931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54ac0fd390 0x7f54ac106c60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:49.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.385+0000 7f54b39b5700 1 --2- 172.21.15.196:0/1757931931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54ac0f72a0 0x7f54ac10cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:49.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.385+0000 7f54b39b5700 1 --2- 172.21.15.196:0/1757931931 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5498062cc0 0x7f5498065180 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:49.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.385+0000 7f54b39b5700 1 --2- 172.21.15.196:0/1757931931 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f54ac0fca30 0x7f54ac10cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:49.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.385+0000 7f54b39b5700 1 -- 172.21.15.196:0/1757931931 >> 172.21.15.196:0/1757931931 conn(0x7f54ac0f2640 msgr2=0x7f54ac0f3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:39:49.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.385+0000 7f54b39b5700 1 -- 172.21.15.196:0/1757931931 shutdown_connections 2024-02-23T00:39:49.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.385+0000 7f54b39b5700 1 -- 172.21.15.196:0/1757931931 wait complete. 2024-02-23T00:39:49.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-02-23T00:39:49.385+0000 7f54b39b5700 1 librados: shutdown 2024-02-23T00:39:50.210 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-02-23T00:39:50.210 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-02-23T00:39:50.210 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-23T00:39:50.253 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd dump --format=json 2024-02-23T00:39:50.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:49.124581+0000 mgr.y (mgr.14517) 15 : cluster [DBG] pgmap v10: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:39:50.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:49.379854+0000 mon.b (mon.1) 31 : audit [INF] from='client.? 172.21.15.196:0/1757931931' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-23T00:39:50.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:49.380153+0000 mon.a (mon.0) 678 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-23T00:39:50.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:49.383580+0000 mon.a (mon.0) 679 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-23T00:39:50.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:49.124581+0000 mgr.y (mgr.14517) 15 : cluster [DBG] pgmap v10: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:39:50.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:49.379854+0000 mon.b (mon.1) 31 : audit [INF] from='client.? 172.21.15.196:0/1757931931' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-23T00:39:50.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:49.380153+0000 mon.a (mon.0) 678 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-23T00:39:50.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:49.383580+0000 mon.a (mon.0) 679 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-23T00:39:50.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:49.124581+0000 mgr.y (mgr.14517) 15 : cluster [DBG] pgmap v10: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:39:50.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:49.379854+0000 mon.b (mon.1) 31 : audit [INF] from='client.? 172.21.15.196:0/1757931931' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-23T00:39:50.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:49.380153+0000 mon.a (mon.0) 678 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-23T00:39:50.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:49.383580+0000 mon.a (mon.0) 679 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-23T00:39:50.685 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:39:52.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:51.124895+0000 mgr.y (mgr.14517) 16 : cluster [DBG] pgmap v11: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 751 B/s rd, 0 op/s 2024-02-23T00:39:52.444 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:39:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:39:52.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:51.124895+0000 mgr.y (mgr.14517) 16 : cluster [DBG] pgmap v11: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 751 B/s rd, 0 op/s 2024-02-23T00:39:52.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:51.124895+0000 mgr.y (mgr.14517) 16 : cluster [DBG] pgmap v11: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 751 B/s rd, 0 op/s 2024-02-23T00:39:53.607 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:52.127350+0000 mgr.y (mgr.14517) 17 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:39:53.607 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:52.127350+0000 mgr.y (mgr.14517) 17 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:39:53.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:52.127350+0000 mgr.y (mgr.14517) 17 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:39:54.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:53.125599+0000 mgr.y (mgr.14517) 18 : cluster [DBG] pgmap v12: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:39:54.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:54.147465+0000 mon.a (mon.0) 680 : audit [DBG] from='client.? 172.21.15.47:0/1887090787' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-23T00:39:54.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:53.125599+0000 mgr.y (mgr.14517) 18 : cluster [DBG] pgmap v12: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:39:54.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:54.147465+0000 mon.a (mon.0) 680 : audit [DBG] from='client.? 172.21.15.47:0/1887090787' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-23T00:39:54.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:53.125599+0000 mgr.y (mgr.14517) 18 : cluster [DBG] pgmap v12: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:39:54.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:54.147465+0000 mon.a (mon.0) 680 : audit [DBG] from='client.? 172.21.15.47:0/1887090787' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-23T00:39:54.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.963+0000 7f66e7e63700 1 -- 172.21.15.47:0/1763014827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66e00f8bc0 msgr2=0x7f66e00f8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:54.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.963+0000 7f66e7e63700 1 --2- 172.21.15.47:0/1763014827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66e00f8bc0 0x7f66e00f8fa0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f66dc0047b0 tx=0x7f66dc009cb0).stop 2024-02-23T00:39:54.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.963+0000 7f66e7e63700 1 -- 172.21.15.47:0/1763014827 shutdown_connections 2024-02-23T00:39:54.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.963+0000 7f66e7e63700 1 --2- 172.21.15.47:0/1763014827 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f66e0102120 0x7f66e0104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:54.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.963+0000 7f66e7e63700 1 --2- 172.21.15.47:0/1763014827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66e00f8bc0 0x7f66e00f8fa0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:54.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.963+0000 7f66e7e63700 1 --2- 172.21.15.47:0/1763014827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66e00f9520 0x7f66e0101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:54.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.963+0000 7f66e7e63700 1 -- 172.21.15.47:0/1763014827 >> 172.21.15.47:0/1763014827 conn(0x7f66e00f26f0 msgr2=0x7f66e00f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:39:54.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.964+0000 7f66e7e63700 1 -- 172.21.15.47:0/1763014827 shutdown_connections 2024-02-23T00:39:54.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.964+0000 7f66e7e63700 1 -- 172.21.15.47:0/1763014827 wait complete. 2024-02-23T00:39:54.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.964+0000 7f66e7e63700 1 librados: starting msgr at 2024-02-23T00:39:54.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.964+0000 7f66e7e63700 1 librados: starting objecter 2024-02-23T00:39:54.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.964+0000 7f66e7e63700 1 Processor -- start 2024-02-23T00:39:54.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.964+0000 7f66e7e63700 1 -- start start 2024-02-23T00:39:54.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.964+0000 7f66e7e63700 1 librados: setting wanted keys 2024-02-23T00:39:54.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.964+0000 7f66e7e63700 1 librados: calling monclient init 2024-02-23T00:39:54.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.964+0000 7f66e7e63700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66e00f8bc0 0x7f66e00f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:54.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.964+0000 7f66e7e63700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66e00f9520 0x7f66e00f7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:54.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.964+0000 7f66e7e63700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f66e0102120 0x7f66e00f7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:54.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.964+0000 7f66e7e63700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e01164a0 con 0x7f66e00f8bc0 2024-02-23T00:39:54.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.964+0000 7f66e7e63700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e0116610 con 0x7f66e00f9520 2024-02-23T00:39:54.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.964+0000 7f66e7e63700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e01167d0 con 0x7f66e0102120 2024-02-23T00:39:54.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.965+0000 7f66e5bff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66e00f8bc0 0x7f66e00f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:54.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.965+0000 7f66e6400700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f66e0102120 0x7f66e00f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:54.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.965+0000 7f66e5bff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66e00f8bc0 0x7f66e00f6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60260/0 (socket says 172.21.15.47:60260) 2024-02-23T00:39:54.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.965+0000 7f66e6400700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f66e0102120 0x7f66e00f7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:48544/0 (socket says 172.21.15.47:48544) 2024-02-23T00:39:54.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.965+0000 7f66e5bff700 1 -- 172.21.15.47:0/1887090787 learned_addr learned my addr 172.21.15.47:0/1887090787 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:39:54.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.965+0000 7f66e53fe700 1 --2- 172.21.15.47:0/1887090787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66e00f9520 0x7f66e00f7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:54.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.965+0000 7f66e5bff700 1 -- 172.21.15.47:0/1887090787 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f66e0102120 msgr2=0x7f66e00f7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:54.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.965+0000 7f66e5bff700 1 --2- 172.21.15.47:0/1887090787 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f66e0102120 0x7f66e00f7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:54.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.965+0000 7f66e5bff700 1 -- 172.21.15.47:0/1887090787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66e00f9520 msgr2=0x7f66e00f7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:54.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.965+0000 7f66e5bff700 1 --2- 172.21.15.47:0/1887090787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66e00f9520 0x7f66e00f7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:54.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.965+0000 7f66e5bff700 1 -- 172.21.15.47:0/1887090787 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d80097c0 con 0x7f66e00f8bc0 2024-02-23T00:39:54.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.965+0000 7f66e6400700 1 --2- 172.21.15.47:0/1887090787 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f66e0102120 0x7f66e00f7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:39:54.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.965+0000 7f66e5bff700 1 --2- 172.21.15.47:0/1887090787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66e00f8bc0 0x7f66e00f6b70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f66dc009f20 tx=0x7f66dc009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:39:54.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.965+0000 7f66d6ffd700 10 client.?.objecter ms_handle_connect 0x7f66e00f8bc0 2024-02-23T00:39:54.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.965+0000 7f66d6ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:39:54.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.965+0000 7f66d6ffd700 1 -- 172.21.15.47:0/1887090787 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66dc018070 con 0x7f66e00f8bc0 2024-02-23T00:39:54.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.965+0000 7f66d6ffd700 1 -- 172.21.15.47:0/1887090787 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f66dc013dd0 con 0x7f66e00f8bc0 2024-02-23T00:39:54.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.965+0000 7f66e7e63700 1 -- 172.21.15.47:0/1887090787 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66dc005040 con 0x7f66e00f8bc0 2024-02-23T00:39:54.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.965+0000 7f66d6ffd700 1 -- 172.21.15.47:0/1887090787 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66dc01da10 con 0x7f66e00f8bc0 2024-02-23T00:39:54.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.965+0000 7f66e7e63700 10 client.14553.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:39:54.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.965+0000 7f66e7e63700 1 -- 172.21.15.47:0/1887090787 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e0116c70 con 0x7f66e00f8bc0 2024-02-23T00:39:54.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.965+0000 7f66e7e63700 1 librados: init done 2024-02-23T00:39:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:39:54.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.966+0000 7f66e7e63700 1 -- 172.21.15.47:0/1887090787 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66e00606d0 con 0x7f66e00f8bc0 2024-02-23T00:39:54.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.966+0000 7f66d6ffd700 1 -- 172.21.15.47:0/1887090787 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f66dc009240 con 0x7f66e00f8bc0 2024-02-23T00:39:54.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.967+0000 7f66d6ffd700 1 --2- 172.21.15.47:0/1887090787 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f66cc062cc0 0x7f66cc065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:54.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.967+0000 7f66d6ffd700 1 -- 172.21.15.47:0/1887090787 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7305+0+0 (secure 0 0 0) 0x7f66dc07e210 con 0x7f66e00f8bc0 2024-02-23T00:39:54.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.967+0000 7f66d6ffd700 10 client.14553.objecter ms_dispatch 0x7f66e0001550 osd_map(76..76 src has 1..76) v4 2024-02-23T00:39:54.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.967+0000 7f66d6ffd700 3 client.14553.objecter handle_osd_map got epochs [76,76] > 0 2024-02-23T00:39:54.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.967+0000 7f66d6ffd700 3 client.14553.objecter handle_osd_map decoding full epoch 76 2024-02-23T00:39:54.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.967+0000 7f66e53fe700 1 --2- 172.21.15.47:0/1887090787 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f66cc062cc0 0x7f66cc065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:54.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.967+0000 7f66d6ffd700 20 client.14553.objecter dump_active .. 0 homeless 2024-02-23T00:39:54.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.967+0000 7f66d6ffd700 10 client.14553.objecter ms_handle_connect 0x7f66cc062cc0 2024-02-23T00:39:54.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.967+0000 7f66e53fe700 1 --2- 172.21.15.47:0/1887090787 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f66cc062cc0 0x7f66cc065180 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f66e00f8310 tx=0x7f66d0009040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:39:54.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:53.972+0000 7f66d6ffd700 1 -- 172.21.15.47:0/1887090787 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f66dc04dfe0 con 0x7f66e00f8bc0 2024-02-23T00:39:54.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:54.146+0000 7f66e7e63700 1 -- 172.21.15.47:0/1887090787 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66e00f8050 con 0x7f66e00f8bc0 2024-02-23T00:39:54.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:54.147+0000 7f66d6ffd700 1 -- 172.21.15.47:0/1887090787 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+21164 (secure 0 0 0) 0x7f66dc0043d0 con 0x7f66e00f8bc0 2024-02-23T00:39:54.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:54.148+0000 7f66e7e63700 10 librados: watch_flush enter 2024-02-23T00:39:54.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:54.148+0000 7f66e7e63700 10 librados: watch_flush exit 2024-02-23T00:39:54.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:54.148+0000 7f66e7e63700 20 client.14553.objecter shutdown clearing up homeless session... 2024-02-23T00:39:54.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:54.148+0000 7f66e7e63700 10 client.14553.objecter successfully canceled tick 2024-02-23T00:39:54.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:54.148+0000 7f66e7e63700 1 -- 172.21.15.47:0/1887090787 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f66cc062cc0 msgr2=0x7f66cc065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:54.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:54.148+0000 7f66e7e63700 1 --2- 172.21.15.47:0/1887090787 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f66cc062cc0 0x7f66cc065180 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f66e00f8310 tx=0x7f66d0009040).stop 2024-02-23T00:39:54.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:54.148+0000 7f66e7e63700 1 -- 172.21.15.47:0/1887090787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66e00f8bc0 msgr2=0x7f66e00f6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:54.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:54.148+0000 7f66e7e63700 1 --2- 172.21.15.47:0/1887090787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66e00f8bc0 0x7f66e00f6b70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f66dc009f20 tx=0x7f66dc009cb0).stop 2024-02-23T00:39:54.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:54.148+0000 7f66e7e63700 1 -- 172.21.15.47:0/1887090787 shutdown_connections 2024-02-23T00:39:54.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:54.148+0000 7f66e7e63700 1 --2- 172.21.15.47:0/1887090787 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f66cc062cc0 0x7f66cc065180 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:54.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:54.148+0000 7f66e7e63700 1 --2- 172.21.15.47:0/1887090787 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f66e0102120 0x7f66e00f7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:54.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:54.148+0000 7f66e7e63700 1 --2- 172.21.15.47:0/1887090787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66e00f8bc0 0x7f66e00f6b70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:54.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:54.148+0000 7f66e7e63700 1 --2- 172.21.15.47:0/1887090787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66e00f9520 0x7f66e00f7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:54.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:54.148+0000 7f66e7e63700 1 -- 172.21.15.47:0/1887090787 >> 172.21.15.47:0/1887090787 conn(0x7f66e00f26f0 msgr2=0x7f66e00f33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:39:54.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:54.148+0000 7f66e7e63700 1 -- 172.21.15.47:0/1887090787 shutdown_connections 2024-02-23T00:39:54.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:54.148+0000 7f66e7e63700 1 -- 172.21.15.47:0/1887090787 wait complete. 2024-02-23T00:39:54.793 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":76,"fsid":"fce2617e-d1e2-11ee-95c0-87774f69a715","created":"2024-02-23T00:32:55.399437+0000","modified":"2024-02-23T00:39:47.321577+0000","last_up_change":"2024-02-23T00:39:05.662717+0000","last_in_change":"2024-02-23T00:38:47.076807+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-23T00:36:52.622994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-02-23T00:39:13.277309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"iscsi","create_time":"2024-02-23T00:39:13.761312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-23T00:39:15.255656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-23T00:39:17.226393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-23T00:39:19.300351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"380050f9-9001-46dc-bb29-037987b4c8e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":4042552860},{"type":"v1","addr":"172.21.15.47:6803","nonce":4042552860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":4042552860},{"type":"v1","addr":"172.21.15.47:6805","nonce":4042552860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":4042552860},{"type":"v1","addr":"172.21.15.47:6809","nonce":4042552860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":4042552860},{"type":"v1","addr":"172.21.15.47:6807","nonce":4042552860}]},"public_addr":"172.21.15.47:6803/4042552860","cluster_addr":"172.21.15.47:6805/4042552860","heartbeat_back_addr":"172.21.15.47:6809/4042552860","heartbeat_front_addr":"172.21.15.47:6807/4042552860","state":["exists","up"]},{"osd":1,"uuid":"b06d0cce-55d2-4a6a-a796-4db0b9bb9d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1945417179},{"type":"v1","addr":"172.21.15.47:6811","nonce":1945417179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1945417179},{"type":"v1","addr":"172.21.15.47:6813","nonce":1945417179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":1945417179},{"type":"v1","addr":"172.21.15.47:6817","nonce":1945417179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1945417179},{"type":"v1","addr":"172.21.15.47:6815","nonce":1945417179}]},"public_addr":"172.21.15.47:6811/1945417179","cluster_addr":"172.21.15.47:6813/1945417179","heartbeat_back_addr":"172.21.15.47:6817/1945417179","heartbeat_front_addr":"172.21.15.47:6815/1945417179","state":["exists","up"]},{"osd":2,"uuid":"6e44d7ca-67a1-4868-bb64-891cf477ebd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":3596356785},{"type":"v1","addr":"172.21.15.47:6819","nonce":3596356785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":3596356785},{"type":"v1","addr":"172.21.15.47:6821","nonce":3596356785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":3596356785},{"type":"v1","addr":"172.21.15.47:6825","nonce":3596356785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":3596356785},{"type":"v1","addr":"172.21.15.47:6823","nonce":3596356785}]},"public_addr":"172.21.15.47:6819/3596356785","cluster_addr":"172.21.15.47:6821/3596356785","heartbeat_back_addr":"172.21.15.47:6825/3596356785","heartbeat_front_addr":"172.21.15.47:6823/3596356785","state":["exists","up"]},{"osd":3,"uuid":"c88debab-cb0b-46f6-a7d8-c5dc61fc059f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1195438081},{"type":"v1","addr":"172.21.15.47:6827","nonce":1195438081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1195438081},{"type":"v1","addr":"172.21.15.47:6829","nonce":1195438081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1195438081},{"type":"v1","addr":"172.21.15.47:6833","nonce":1195438081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1195438081},{"type":"v1","addr":"172.21.15.47:6831","nonce":1195438081}]},"public_addr":"172.21.15.47:6827/1195438081","cluster_addr":"172.21.15.47:6829/1195438081","heartbeat_back_addr":"172.21.15.47:6833/1195438081","heartbeat_front_addr":"172.21.15.47:6831/1195438081","state":["exists","up"]},{"osd":4,"uuid":"d17966d7-eb12-4389-ac90-8d15efe290e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2530629682},{"type":"v1","addr":"172.21.15.196:6801","nonce":2530629682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2530629682},{"type":"v1","addr":"172.21.15.196:6803","nonce":2530629682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2530629682},{"type":"v1","addr":"172.21.15.196:6807","nonce":2530629682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2530629682},{"type":"v1","addr":"172.21.15.196:6805","nonce":2530629682}]},"public_addr":"172.21.15.196:6801/2530629682","cluster_addr":"172.21.15.196:6803/2530629682","heartbeat_back_addr":"172.21.15.196:6807/2530629682","heartbeat_front_addr":"172.21.15.196:6805/2530629682","state":["exists","up"]},{"osd":5,"uuid":"b729d646-4899-47b3-b917-acceba46d241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1017187491},{"type":"v1","addr":"172.21.15.196:6809","nonce":1017187491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1017187491},{"type":"v1","addr":"172.21.15.196:6811","nonce":1017187491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1017187491},{"type":"v1","addr":"172.21.15.196:6815","nonce":1017187491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1017187491},{"type":"v1","addr":"172.21.15.196:6813","nonce":1017187491}]},"public_addr":"172.21.15.196:6809/1017187491","cluster_addr":"172.21.15.196:6811/1017187491","heartbeat_back_addr":"172.21.15.196:6815/1017187491","heartbeat_front_addr":"172.21.15.196:6813/1017187491","state":["exists","up"]},{"osd":6,"uuid":"df337124-9488-4aad-98e5-3614b51fc757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3885668599},{"type":"v1","addr":"172.21.15.196:6817","nonce":3885668599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3885668599},{"type":"v1","addr":"172.21.15.196:6819","nonce":3885668599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3885668599},{"type":"v1","addr":"172.21.15.196:6823","nonce":3885668599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3885668599},{"type":"v1","addr":"172.21.15.196:6821","nonce":3885668599}]},"public_addr":"172.21.15.196:6817/3885668599","cluster_addr":"172.21.15.196:6819/3885668599","heartbeat_back_addr":"172.21.15.196:6823/3885668599","heartbeat_front_addr":"172.21.15.196:6821/3885668599","state":["exists","up"]},{"osd":7,"uuid":"84fee1f2-1a88-447a-b1f3-56ff7efe7fec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2261821675},{"type":"v1","addr":"172.21.15.196:6825","nonce":2261821675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2261821675},{"type":"v1","addr":"172.21.15.196:6827","nonce":2261821675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2261821675},{"type":"v1","addr":"172.21.15.196:6831","nonce":2261821675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2261821675},{"type":"v1","addr":"172.21.15.196:6829","nonce":2261821675}]},"public_addr":"172.21.15.196:6825/2261821675","cluster_addr":"172.21.15.196:6827/2261821675","heartbeat_back_addr":"172.21.15.196:6831/2261821675","heartbeat_front_addr":"172.21.15.196:6829/2261821675","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:39:15.242369+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:38:33.277424+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:39:21.405657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:39:47.131225+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:39:07.716100+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:39:09.901475+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:39:37.023883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:39:03.696875+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6800/622994871":"2024-02-24T00:39:41.110158+0000","172.21.15.47:0/1561174431":"2024-02-24T00:39:41.110158+0000","172.21.15.47:0/3349912606":"2024-02-24T00:39:41.110158+0000","172.21.15.47:0/710432524":"2024-02-24T00:39:41.110158+0000","172.21.15.47:0/2804352395":"2024-02-24T00:33:52.061914+0000","172.21.15.47:6801/1916598887":"2024-02-24T00:33:52.061914+0000","172.21.15.47:6800/1916598887":"2024-02-24T00:33:52.061914+0000","172.21.15.47:0/792211316":"2024-02-24T00:33:27.972945+0000","172.21.15.47:0/381995198":"2024-02-24T00:33:52.061914+0000","172.21.15.47:6800/2409691510":"2024-02-24T00:33:27.972945+0000","172.21.15.47:6801/2409691510":"2024-02-24T00:33:27.972945+0000","172.21.15.47:6801/622994871":"2024-02-24T00:39:41.110158+0000","172.21.15.47:0/1448327041":"2024-02-24T00:33:27.972945+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-23T00:39:54.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:54.149+0000 7f66e7e63700 1 librados: shutdown 2024-02-23T00:39:55.468 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-02-23T00:36:52.622994+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': '.rgw.root', 'create_time': '2024-02-23T00:39:13.277309+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 3, 'pool_name': 'iscsi', 'create_time': '2024-02-23T00:39:13.761312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-02-23T00:39:15.255656+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-02-23T00:39:17.226393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-02-23T00:39:19.300351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}] 2024-02-23T00:39:55.469 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd pool get device_health_metrics pg_num 2024-02-23T00:39:56.551 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:39:56.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:55.126049+0000 mgr.y (mgr.14517) 19 : cluster [DBG] pgmap v13: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-23T00:39:56.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:55.126049+0000 mgr.y (mgr.14517) 19 : cluster [DBG] pgmap v13: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-23T00:39:56.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:55.126049+0000 mgr.y (mgr.14517) 19 : cluster [DBG] pgmap v13: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-23T00:39:58.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:39:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:57.126594+0000 mgr.y (mgr.14517) 20 : cluster [DBG] pgmap v14: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:39:58.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:39:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:57.126594+0000 mgr.y (mgr.14517) 20 : cluster [DBG] pgmap v14: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:39:58.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:39:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:57.126594+0000 mgr.y (mgr.14517) 20 : cluster [DBG] pgmap v14: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:39:59.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.495+0000 7f1054dbb700 1 -- 172.21.15.47:0/3578188471 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f10500fd510 msgr2=0x7f10500f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:59.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.495+0000 7f1054dbb700 1 --2- 172.21.15.47:0/3578188471 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f10500fd510 0x7f10500f6d40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f10440047b0 tx=0x7f1044009cb0).stop 2024-02-23T00:39:59.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.495+0000 7f1054dbb700 1 -- 172.21.15.47:0/3578188471 shutdown_connections 2024-02-23T00:39:59.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.495+0000 7f1054dbb700 1 --2- 172.21.15.47:0/3578188471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10500f7400 0x7f1050103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:59.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.495+0000 7f1054dbb700 1 --2- 172.21.15.47:0/3578188471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10500fcbb0 0x7f10500fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:59.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.495+0000 7f1054dbb700 1 --2- 172.21.15.47:0/3578188471 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f10500fd510 0x7f10500f6d40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:59.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.495+0000 7f1054dbb700 1 -- 172.21.15.47:0/3578188471 >> 172.21.15.47:0/3578188471 conn(0x7f10500f2820 msgr2=0x7f10500f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:39:59.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.495+0000 7f1054dbb700 1 -- 172.21.15.47:0/3578188471 shutdown_connections 2024-02-23T00:39:59.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.495+0000 7f1054dbb700 1 -- 172.21.15.47:0/3578188471 wait complete. 2024-02-23T00:39:59.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.496+0000 7f1054dbb700 1 librados: starting msgr at 2024-02-23T00:39:59.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.496+0000 7f1054dbb700 1 librados: starting objecter 2024-02-23T00:39:59.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.496+0000 7f1054dbb700 1 Processor -- start 2024-02-23T00:39:59.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.496+0000 7f1054dbb700 1 -- start start 2024-02-23T00:39:59.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.496+0000 7f1054dbb700 1 librados: setting wanted keys 2024-02-23T00:39:59.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.496+0000 7f1054dbb700 1 librados: calling monclient init 2024-02-23T00:39:59.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.497+0000 7f1054dbb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10500f7400 0x7f10501102e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:59.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.497+0000 7f1054dbb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10500fcbb0 0x7f10501107d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:59.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.497+0000 7f1054dbb700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f10500fd510 0x7f105010a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:59.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.497+0000 7f1054dbb700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105010a970 con 0x7f10500fcbb0 2024-02-23T00:39:59.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.497+0000 7f1054dbb700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105010aae0 con 0x7f10500f7400 2024-02-23T00:39:59.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.497+0000 7f1054dbb700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f105010aca0 con 0x7f10500fd510 2024-02-23T00:39:59.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.497+0000 7f104ed9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f10500fd510 0x7f105010a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:59.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.497+0000 7f104ed9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f10500fd510 0x7f105010a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:45948/0 (socket says 172.21.15.47:45948) 2024-02-23T00:39:59.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.497+0000 7f104ed9d700 1 -- 172.21.15.47:0/2657924072 learned_addr learned my addr 172.21.15.47:0/2657924072 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:39:59.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.497+0000 7f104dd9b700 1 --2- 172.21.15.47:0/2657924072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10500fcbb0 0x7f10501107d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:59.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.497+0000 7f104e59c700 1 --2- 172.21.15.47:0/2657924072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10500f7400 0x7f10501102e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:59.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.497+0000 7f104ed9d700 1 -- 172.21.15.47:0/2657924072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10500f7400 msgr2=0x7f10501102e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:59.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.497+0000 7f104ed9d700 1 --2- 172.21.15.47:0/2657924072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10500f7400 0x7f10501102e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:59.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.497+0000 7f104ed9d700 1 -- 172.21.15.47:0/2657924072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10500fcbb0 msgr2=0x7f10501107d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:59.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.497+0000 7f104ed9d700 1 --2- 172.21.15.47:0/2657924072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10500fcbb0 0x7f10501107d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:59.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.497+0000 7f104ed9d700 1 -- 172.21.15.47:0/2657924072 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1040005040 con 0x7f10500fd510 2024-02-23T00:39:59.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.497+0000 7f104dd9b700 1 --2- 172.21.15.47:0/2657924072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10500fcbb0 0x7f10501107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:39:59.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.497+0000 7f104ed9d700 1 --2- 172.21.15.47:0/2657924072 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f10500fd510 0x7f105010a480 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f104800d970 tx=0x7f1048004e60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:39:59.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.497+0000 7f103f7fe700 10 client.?.objecter ms_handle_connect 0x7f10500fd510 2024-02-23T00:39:59.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.497+0000 7f103f7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:39:59.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.497+0000 7f103f7fe700 1 -- 172.21.15.47:0/2657924072 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1048004020 con 0x7f10500fd510 2024-02-23T00:39:59.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.497+0000 7f103f7fe700 1 -- 172.21.15.47:0/2657924072 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f104800f040 con 0x7f10500fd510 2024-02-23T00:39:59.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.497+0000 7f1054dbb700 1 -- 172.21.15.47:0/2657924072 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1044005040 con 0x7f10500fd510 2024-02-23T00:39:59.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.497+0000 7f103f7fe700 1 -- 172.21.15.47:0/2657924072 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1048003430 con 0x7f10500fd510 2024-02-23T00:39:59.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.497+0000 7f1054dbb700 10 client.24452.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:39:59.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.497+0000 7f1054dbb700 1 -- 172.21.15.47:0/2657924072 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105010b290 con 0x7f10500fd510 2024-02-23T00:39:59.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.498+0000 7f1054dbb700 1 librados: init done 2024-02-23T00:39:59.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.498+0000 7f1054dbb700 1 -- 172.21.15.47:0/2657924072 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f105005fa60 con 0x7f10500fd510 2024-02-23T00:39:59.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.499+0000 7f103f7fe700 1 -- 172.21.15.47:0/2657924072 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f1048003860 con 0x7f10500fd510 2024-02-23T00:39:59.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.499+0000 7f103f7fe700 1 --2- 172.21.15.47:0/2657924072 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1038062cc0 0x7f1038065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:39:59.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.499+0000 7f103f7fe700 1 -- 172.21.15.47:0/2657924072 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7305+0+0 (secure 0 0 0) 0x7f104807c9d0 con 0x7f10500fd510 2024-02-23T00:39:59.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.499+0000 7f103f7fe700 10 client.24452.objecter ms_dispatch 0x7f1050001550 osd_map(76..76 src has 1..76) v4 2024-02-23T00:39:59.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.499+0000 7f103f7fe700 3 client.24452.objecter handle_osd_map got epochs [76,76] > 0 2024-02-23T00:39:59.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.500+0000 7f103f7fe700 3 client.24452.objecter handle_osd_map decoding full epoch 76 2024-02-23T00:39:59.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.500+0000 7f103f7fe700 20 client.24452.objecter dump_active .. 0 homeless 2024-02-23T00:39:59.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.500+0000 7f104e59c700 1 --2- 172.21.15.47:0/2657924072 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1038062cc0 0x7f1038065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:39:59.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.500+0000 7f104e59c700 1 --2- 172.21.15.47:0/2657924072 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1038062cc0 0x7f1038065180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f1040004870 tx=0x7f104000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:39:59.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.500+0000 7f103f7fe700 10 client.24452.objecter ms_handle_connect 0x7f1038062cc0 2024-02-23T00:39:59.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.505+0000 7f103f7fe700 1 -- 172.21.15.47:0/2657924072 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f1048048f60 con 0x7f10500fd510 2024-02-23T00:39:59.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.681+0000 7f1054dbb700 1 -- 172.21.15.47:0/2657924072 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f1050105ea0 con 0x7f10500fd510 2024-02-23T00:39:59.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.681+0000 7f103f7fe700 1 -- 172.21.15.47:0/2657924072 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v76) v1 ==== 110+0+10 (secure 0 0 0) 0x7f104804c8f0 con 0x7f10500fd510 2024-02-23T00:39:59.684 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-02-23T00:39:59.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.683+0000 7f1054dbb700 10 librados: watch_flush enter 2024-02-23T00:39:59.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.683+0000 7f1054dbb700 10 librados: watch_flush exit 2024-02-23T00:39:59.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.683+0000 7f1054dbb700 20 client.24452.objecter shutdown clearing up homeless session... 2024-02-23T00:39:59.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.683+0000 7f1054dbb700 10 client.24452.objecter successfully canceled tick 2024-02-23T00:39:59.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.683+0000 7f1054dbb700 1 -- 172.21.15.47:0/2657924072 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1038062cc0 msgr2=0x7f1038065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:59.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.683+0000 7f1054dbb700 1 --2- 172.21.15.47:0/2657924072 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1038062cc0 0x7f1038065180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f1040004870 tx=0x7f104000b040).stop 2024-02-23T00:39:59.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.683+0000 7f1054dbb700 1 -- 172.21.15.47:0/2657924072 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f10500fd510 msgr2=0x7f105010a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:39:59.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.683+0000 7f1054dbb700 1 --2- 172.21.15.47:0/2657924072 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f10500fd510 0x7f105010a480 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f104800d970 tx=0x7f1048004e60).stop 2024-02-23T00:39:59.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.683+0000 7f1054dbb700 1 -- 172.21.15.47:0/2657924072 shutdown_connections 2024-02-23T00:39:59.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.683+0000 7f1054dbb700 1 --2- 172.21.15.47:0/2657924072 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1038062cc0 0x7f1038065180 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:59.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.683+0000 7f1054dbb700 1 --2- 172.21.15.47:0/2657924072 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f10500fd510 0x7f105010a480 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:59.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.683+0000 7f1054dbb700 1 --2- 172.21.15.47:0/2657924072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10500f7400 0x7f10501102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:59.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.683+0000 7f1054dbb700 1 --2- 172.21.15.47:0/2657924072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10500fcbb0 0x7f10501107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:39:59.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.683+0000 7f1054dbb700 1 -- 172.21.15.47:0/2657924072 >> 172.21.15.47:0/2657924072 conn(0x7f10500f2820 msgr2=0x7f10500f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:39:59.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.683+0000 7f1054dbb700 1 -- 172.21.15.47:0/2657924072 shutdown_connections 2024-02-23T00:39:59.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.683+0000 7f1054dbb700 1 -- 172.21.15.47:0/2657924072 wait complete. 2024-02-23T00:39:59.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:39:59.683+0000 7f1054dbb700 1 librados: shutdown 2024-02-23T00:39:59.806 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:39:59 smithi047 bash[67376]: fe3237bd3876813563b370cd58d9f0b01e6266c0d7f084187bbac39bac21e3f9 2024-02-23T00:39:59.807 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:39:59 smithi047 systemd[1]: Started Ceph alertmanager.a for fce2617e-d1e2-11ee-95c0-87774f69a715. 2024-02-23T00:40:00.362 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd pool get .rgw.root pg_num 2024-02-23T00:40:00.661 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:59.127105+0000 mgr.y (mgr.14517) 21 : cluster [DBG] pgmap v15: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 867 B/s rd, 0 op/s 2024-02-23T00:40:00.661 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:59.682552+0000 mon.c (mon.2) 18 : audit [DBG] from='client.? 172.21.15.47:0/2657924072' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-02-23T00:40:00.661 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:39:59.722879+0000 mon.a (mon.0) 681 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:00.661 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:59.996736+0000 osd.1 (osd.1) 9 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:40:00.661 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:39:59.996826+0000 osd.1 (osd.1) 10 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:40:00.661 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:00.000136+0000 mon.a (mon.0) 682 : cluster [INF] overall HEALTH_OK 2024-02-23T00:40:00.661 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:59.127105+0000 mgr.y (mgr.14517) 21 : cluster [DBG] pgmap v15: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 867 B/s rd, 0 op/s 2024-02-23T00:40:00.662 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:59.682552+0000 mon.c (mon.2) 18 : audit [DBG] from='client.? 172.21.15.47:0/2657924072' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-02-23T00:40:00.662 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:39:59.722879+0000 mon.a (mon.0) 681 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:00.662 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:59.996736+0000 osd.1 (osd.1) 9 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:40:00.662 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:39:59.996826+0000 osd.1 (osd.1) 10 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:40:00.662 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:00.000136+0000 mon.a (mon.0) 682 : cluster [INF] overall HEALTH_OK 2024-02-23T00:40:00.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:59.127105+0000 mgr.y (mgr.14517) 21 : cluster [DBG] pgmap v15: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 867 B/s rd, 0 op/s 2024-02-23T00:40:00.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:59.682552+0000 mon.c (mon.2) 18 : audit [DBG] from='client.? 172.21.15.47:0/2657924072' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-02-23T00:40:00.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:39:59.722879+0000 mon.a (mon.0) 681 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:00.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:59.996736+0000 osd.1 (osd.1) 9 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:40:00.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:39:59.996826+0000 osd.1 (osd.1) 10 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:40:00.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:00.000136+0000 mon.a (mon.0) 682 : cluster [INF] overall HEALTH_OK 2024-02-23T00:40:00.783 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:40:01.057 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:00 smithi047 systemd[1]: Starting Ceph node-exporter.a for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T00:40:01.057 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:00 smithi047 bash[67730]: Trying to pull quay.io/prometheus/node-exporter:v1.3.1... 2024-02-23T00:40:01.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:39:59.725515+0000 mgr.y (mgr.14517) 22 : cephadm [INF] Deploying daemon node-exporter.a on smithi047 2024-02-23T00:40:01.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:00.726251+0000 mon.a (mon.0) 683 : cluster [DBG] osdmap e77: 8 total, 8 up, 8 in 2024-02-23T00:40:01.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:01.274113+0000 mon.a (mon.0) 684 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:01.557 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:01 smithi047 bash[67730]: Getting image source signatures 2024-02-23T00:40:01.557 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:01 smithi047 bash[67730]: Copying blob sha256:b5db1e299295edf3005515ab7879c1df64a33c185d3a7a23aa4dcaa17d26f7b3 2024-02-23T00:40:01.557 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:01 smithi047 bash[67730]: Copying blob sha256:aa2a8d90b84cb2a9c422e7005cd166a008ccf22ef5d7d4f07128478585ce35ea 2024-02-23T00:40:01.557 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:01 smithi047 bash[67730]: Copying blob sha256:b45d31ee2d7f9f452678a85b0c837c29e12089f31ee8dbac6c8c24dfa4054a30 2024-02-23T00:40:01.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:39:59.725515+0000 mgr.y (mgr.14517) 22 : cephadm [INF] Deploying daemon node-exporter.a on smithi047 2024-02-23T00:40:01.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:00.726251+0000 mon.a (mon.0) 683 : cluster [DBG] osdmap e77: 8 total, 8 up, 8 in 2024-02-23T00:40:01.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:01.274113+0000 mon.a (mon.0) 684 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:01.558 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:40:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[67409]: level=info ts=2024-02-23T00:40:01.504Z caller=cluster.go:696 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000902413s 2024-02-23T00:40:01.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:39:59.725515+0000 mgr.y (mgr.14517) 22 : cephadm [INF] Deploying daemon node-exporter.a on smithi047 2024-02-23T00:40:01.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:00.726251+0000 mon.a (mon.0) 683 : cluster [DBG] osdmap e77: 8 total, 8 up, 8 in 2024-02-23T00:40:01.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:01.274113+0000 mon.a (mon.0) 684 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:02.443 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:40:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:40:02.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:01.127438+0000 mgr.y (mgr.14517) 23 : cluster [DBG] pgmap v17: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:40:02.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:01.127438+0000 mgr.y (mgr.14517) 23 : cluster [DBG] pgmap v17: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:40:02.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:01.127438+0000 mgr.y (mgr.14517) 23 : cluster [DBG] pgmap v17: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:40:02.557 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:02 smithi047 bash[67730]: Copying config sha256:1dbe0e931976487e20e5cfb272087e08a9779c88fd5e9617ed7042dd9751ec26 2024-02-23T00:40:02.915 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:02 smithi047 bash[67730]: Writing manifest to image destination 2024-02-23T00:40:03.629 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:02.129577+0000 mgr.y (mgr.14517) 24 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:40:03.630 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:02.129577+0000 mgr.y (mgr.14517) 24 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:40:03.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:02.129577+0000 mgr.y (mgr.14517) 24 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:40:03.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.815+0000 7ff27250e700 1 -- 172.21.15.47:0/60863522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff26c1044c0 msgr2=0x7ff26c1068b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:03.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.815+0000 7ff27250e700 1 --2- 172.21.15.47:0/60863522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff26c1044c0 0x7ff26c1068b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7ff264009be0 tx=0x7ff264004d10).stop 2024-02-23T00:40:03.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.815+0000 7ff27250e700 1 -- 172.21.15.47:0/60863522 shutdown_connections 2024-02-23T00:40:03.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.815+0000 7ff27250e700 1 --2- 172.21.15.47:0/60863522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff26c1044c0 0x7ff26c1068b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:03.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.815+0000 7ff27250e700 1 --2- 172.21.15.47:0/60863522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff26c06b0f0 0x7ff26c06b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:03.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.815+0000 7ff27250e700 1 --2- 172.21.15.47:0/60863522 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff26c06b9c0 0x7ff26c103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:03.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.815+0000 7ff27250e700 1 -- 172.21.15.47:0/60863522 >> 172.21.15.47:0/60863522 conn(0x7ff26c0f3800 msgr2=0x7ff26c0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:40:03.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.815+0000 7ff27250e700 1 -- 172.21.15.47:0/60863522 shutdown_connections 2024-02-23T00:40:03.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.815+0000 7ff27250e700 1 -- 172.21.15.47:0/60863522 wait complete. 2024-02-23T00:40:03.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.816+0000 7ff27250e700 1 librados: starting msgr at 2024-02-23T00:40:03.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.816+0000 7ff27250e700 1 librados: starting objecter 2024-02-23T00:40:03.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.816+0000 7ff27250e700 1 Processor -- start 2024-02-23T00:40:03.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.816+0000 7ff27250e700 1 -- start start 2024-02-23T00:40:03.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.816+0000 7ff27250e700 1 librados: setting wanted keys 2024-02-23T00:40:03.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.816+0000 7ff27250e700 1 librados: calling monclient init 2024-02-23T00:40:03.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.816+0000 7ff27250e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff26c06b0f0 0x7ff26c10ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:03.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.816+0000 7ff27250e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff26c06b9c0 0x7ff26c110450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:03.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.816+0000 7ff27250e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff26c1044c0 0x7ff26c10a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:03.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.816+0000 7ff27250e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c10a630 con 0x7ff26c1044c0 2024-02-23T00:40:03.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.816+0000 7ff27250e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c10a7a0 con 0x7ff26c06b9c0 2024-02-23T00:40:03.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.816+0000 7ff27250e700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c10a910 con 0x7ff26c06b0f0 2024-02-23T00:40:03.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.817+0000 7ff26bfff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff26c06b0f0 0x7ff26c10ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:03.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.817+0000 7ff26bfff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff26c06b0f0 0x7ff26c10ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:45962/0 (socket says 172.21.15.47:45962) 2024-02-23T00:40:03.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.817+0000 7ff26bfff700 1 -- 172.21.15.47:0/1954863607 learned_addr learned my addr 172.21.15.47:0/1954863607 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:40:03.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.817+0000 7ff270aab700 1 --2- 172.21.15.47:0/1954863607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff26c1044c0 0x7ff26c10a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:03.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.817+0000 7ff26b7fe700 1 --2- 172.21.15.47:0/1954863607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff26c06b9c0 0x7ff26c110450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:03.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.817+0000 7ff26bfff700 1 -- 172.21.15.47:0/1954863607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff26c06b9c0 msgr2=0x7ff26c110450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:03.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.817+0000 7ff26bfff700 1 --2- 172.21.15.47:0/1954863607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff26c06b9c0 0x7ff26c110450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:03.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.817+0000 7ff26bfff700 1 -- 172.21.15.47:0/1954863607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff26c1044c0 msgr2=0x7ff26c10a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:03.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.817+0000 7ff26bfff700 1 --2- 172.21.15.47:0/1954863607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff26c1044c0 0x7ff26c10a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:03.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.817+0000 7ff26bfff700 1 -- 172.21.15.47:0/1954863607 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff260005040 con 0x7ff26c06b0f0 2024-02-23T00:40:03.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.817+0000 7ff270aab700 1 --2- 172.21.15.47:0/1954863607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff26c1044c0 0x7ff26c10a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:40:03.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.817+0000 7ff26bfff700 1 --2- 172.21.15.47:0/1954863607 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff26c06b0f0 0x7ff26c10ff60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7ff25c0027f0 tx=0x7ff25c002b00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:03.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.817+0000 7ff2697fa700 10 client.?.objecter ms_handle_connect 0x7ff26c06b0f0 2024-02-23T00:40:03.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.817+0000 7ff2697fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:40:03.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.817+0000 7ff2697fa700 1 -- 172.21.15.47:0/1954863607 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c007b40 con 0x7ff26c06b0f0 2024-02-23T00:40:03.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.817+0000 7ff2697fa700 1 -- 172.21.15.47:0/1954863607 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7ff25c00f040 con 0x7ff26c06b0f0 2024-02-23T00:40:03.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.817+0000 7ff2697fa700 1 -- 172.21.15.47:0/1954863607 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c014610 con 0x7ff26c06b0f0 2024-02-23T00:40:03.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.817+0000 7ff27250e700 1 -- 172.21.15.47:0/1954863607 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2640097c0 con 0x7ff26c06b0f0 2024-02-23T00:40:03.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.817+0000 7ff27250e700 10 client.24458.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:40:03.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.817+0000 7ff27250e700 1 -- 172.21.15.47:0/1954863607 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff26c10af50 con 0x7ff26c06b0f0 2024-02-23T00:40:03.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.817+0000 7ff27250e700 1 librados: init done 2024-02-23T00:40:03.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.818+0000 7ff27250e700 1 -- 172.21.15.47:0/1954863607 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff26c1090d0 con 0x7ff26c06b0f0 2024-02-23T00:40:03.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.818+0000 7ff2697fa700 1 -- 172.21.15.47:0/1954863607 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7ff25c009110 con 0x7ff26c06b0f0 2024-02-23T00:40:03.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.819+0000 7ff2697fa700 1 --2- 172.21.15.47:0/1954863607 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff254062cc0 0x7ff254065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:03.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.819+0000 7ff2697fa700 1 -- 172.21.15.47:0/1954863607 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7305+0+0 (secure 0 0 0) 0x7ff25c07cbe0 con 0x7ff26c06b0f0 2024-02-23T00:40:03.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.819+0000 7ff2697fa700 10 client.24458.objecter ms_dispatch 0x7ff26c001550 osd_map(77..77 src has 1..77) v4 2024-02-23T00:40:03.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.819+0000 7ff2697fa700 3 client.24458.objecter handle_osd_map got epochs [77,77] > 0 2024-02-23T00:40:03.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.819+0000 7ff2697fa700 3 client.24458.objecter handle_osd_map decoding full epoch 77 2024-02-23T00:40:03.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.819+0000 7ff26b7fe700 1 --2- 172.21.15.47:0/1954863607 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff254062cc0 0x7ff254065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:03.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.819+0000 7ff2697fa700 20 client.24458.objecter dump_active .. 0 homeless 2024-02-23T00:40:03.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.819+0000 7ff2697fa700 10 client.24458.objecter ms_handle_connect 0x7ff254062cc0 2024-02-23T00:40:03.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.819+0000 7ff26b7fe700 1 --2- 172.21.15.47:0/1954863607 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff254062cc0 0x7ff254065180 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7ff26c10b8e0 tx=0x7ff26000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:03.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.823+0000 7ff2697fa700 1 -- 172.21.15.47:0/1954863607 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ff25c04c9b0 con 0x7ff26c06b0f0 2024-02-23T00:40:03.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.995+0000 7ff27250e700 1 -- 172.21.15.47:0/1954863607 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".rgw.root", "var": "pg_num"} v 0) v1 -- 0x7ff26c10b620 con 0x7ff26c06b0f0 2024-02-23T00:40:03.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.995+0000 7ff2697fa700 1 -- 172.21.15.47:0/1954863607 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".rgw.root", "var": "pg_num"}]=0 v77) v1 ==== 98+0+11 (secure 0 0 0) 0x7ff25c04ffd0 con 0x7ff26c06b0f0 2024-02-23T00:40:03.998 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-02-23T00:40:04.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.997+0000 7ff27250e700 10 librados: watch_flush enter 2024-02-23T00:40:04.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.997+0000 7ff27250e700 10 librados: watch_flush exit 2024-02-23T00:40:04.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.997+0000 7ff27250e700 20 client.24458.objecter shutdown clearing up homeless session... 2024-02-23T00:40:04.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.997+0000 7ff27250e700 10 client.24458.objecter successfully canceled tick 2024-02-23T00:40:04.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.997+0000 7ff27250e700 1 -- 172.21.15.47:0/1954863607 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff254062cc0 msgr2=0x7ff254065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:04.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.997+0000 7ff27250e700 1 --2- 172.21.15.47:0/1954863607 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff254062cc0 0x7ff254065180 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7ff26c10b8e0 tx=0x7ff26000b040).stop 2024-02-23T00:40:04.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.997+0000 7ff27250e700 1 -- 172.21.15.47:0/1954863607 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff26c06b0f0 msgr2=0x7ff26c10ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:04.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.997+0000 7ff27250e700 1 --2- 172.21.15.47:0/1954863607 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff26c06b0f0 0x7ff26c10ff60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7ff25c0027f0 tx=0x7ff25c002b00).stop 2024-02-23T00:40:04.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.997+0000 7ff27250e700 1 -- 172.21.15.47:0/1954863607 shutdown_connections 2024-02-23T00:40:04.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.997+0000 7ff27250e700 1 --2- 172.21.15.47:0/1954863607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff26c1044c0 0x7ff26c10a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:04.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.997+0000 7ff27250e700 1 --2- 172.21.15.47:0/1954863607 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff254062cc0 0x7ff254065180 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:04.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.997+0000 7ff27250e700 1 --2- 172.21.15.47:0/1954863607 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff26c06b0f0 0x7ff26c10ff60 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:04.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.997+0000 7ff27250e700 1 --2- 172.21.15.47:0/1954863607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff26c06b9c0 0x7ff26c110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:04.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.997+0000 7ff27250e700 1 -- 172.21.15.47:0/1954863607 >> 172.21.15.47:0/1954863607 conn(0x7ff26c0f3800 msgr2=0x7ff26c0f4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:40:04.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.997+0000 7ff27250e700 1 -- 172.21.15.47:0/1954863607 shutdown_connections 2024-02-23T00:40:04.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.998+0000 7ff27250e700 1 -- 172.21.15.47:0/1954863607 wait complete. 2024-02-23T00:40:04.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:03.998+0000 7ff27250e700 1 librados: shutdown 2024-02-23T00:40:04.307 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.994Z caller=node_exporter.go:182 level=info msg="Starting node_exporter" version="(version=1.3.1, branch=HEAD, revision=a2321e7b940ddcff26873612bccdf7cd4c42b6b6)" 2024-02-23T00:40:04.307 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.994Z caller=node_exporter.go:183 level=info msg="Build context" build_context="(go=go1.17.3, user=root@243aafa5525c, date=20211205-11:09:49)" 2024-02-23T00:40:04.307 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.994Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+)($|/) 2024-02-23T00:40:04.307 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.994Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-23T00:40:04.307 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.995Z caller=node_exporter.go:108 level=info msg="Enabled collectors" 2024-02-23T00:40:04.307 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.995Z caller=node_exporter.go:115 level=info collector=arp 2024-02-23T00:40:04.307 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.995Z caller=node_exporter.go:115 level=info collector=bcache 2024-02-23T00:40:04.307 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.995Z caller=node_exporter.go:115 level=info collector=bonding 2024-02-23T00:40:04.307 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.995Z caller=node_exporter.go:115 level=info collector=btrfs 2024-02-23T00:40:04.307 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.995Z caller=node_exporter.go:115 level=info collector=conntrack 2024-02-23T00:40:04.308 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.995Z caller=node_exporter.go:115 level=info collector=cpu 2024-02-23T00:40:04.308 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.995Z caller=node_exporter.go:115 level=info collector=cpufreq 2024-02-23T00:40:04.308 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.995Z caller=node_exporter.go:115 level=info collector=diskstats 2024-02-23T00:40:04.308 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.995Z caller=node_exporter.go:115 level=info collector=dmi 2024-02-23T00:40:04.308 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.995Z caller=node_exporter.go:115 level=info collector=edac 2024-02-23T00:40:04.308 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.995Z caller=node_exporter.go:115 level=info collector=entropy 2024-02-23T00:40:04.308 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.995Z caller=node_exporter.go:115 level=info collector=fibrechannel 2024-02-23T00:40:04.308 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.995Z caller=node_exporter.go:115 level=info collector=filefd 2024-02-23T00:40:04.308 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.995Z caller=node_exporter.go:115 level=info collector=filesystem 2024-02-23T00:40:04.308 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.995Z caller=node_exporter.go:115 level=info collector=hwmon 2024-02-23T00:40:04.308 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.995Z caller=node_exporter.go:115 level=info collector=infiniband 2024-02-23T00:40:04.308 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.995Z caller=node_exporter.go:115 level=info collector=ipvs 2024-02-23T00:40:04.308 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.995Z caller=node_exporter.go:115 level=info collector=loadavg 2024-02-23T00:40:04.309 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.995Z caller=node_exporter.go:115 level=info collector=mdadm 2024-02-23T00:40:04.309 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.995Z caller=node_exporter.go:115 level=info collector=meminfo 2024-02-23T00:40:04.309 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.996Z caller=node_exporter.go:115 level=info collector=netclass 2024-02-23T00:40:04.309 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.996Z caller=node_exporter.go:115 level=info collector=netdev 2024-02-23T00:40:04.309 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.996Z caller=node_exporter.go:115 level=info collector=netstat 2024-02-23T00:40:04.309 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.996Z caller=node_exporter.go:115 level=info collector=nfs 2024-02-23T00:40:04.309 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.996Z caller=node_exporter.go:115 level=info collector=nfsd 2024-02-23T00:40:04.309 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.996Z caller=node_exporter.go:115 level=info collector=nvme 2024-02-23T00:40:04.309 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.996Z caller=node_exporter.go:115 level=info collector=os 2024-02-23T00:40:04.309 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.996Z caller=node_exporter.go:115 level=info collector=powersupplyclass 2024-02-23T00:40:04.310 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.996Z caller=node_exporter.go:115 level=info collector=pressure 2024-02-23T00:40:04.310 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.996Z caller=node_exporter.go:115 level=info collector=rapl 2024-02-23T00:40:04.310 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.996Z caller=node_exporter.go:115 level=info collector=schedstat 2024-02-23T00:40:04.310 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.996Z caller=node_exporter.go:115 level=info collector=sockstat 2024-02-23T00:40:04.310 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.996Z caller=node_exporter.go:115 level=info collector=softnet 2024-02-23T00:40:04.310 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.996Z caller=node_exporter.go:115 level=info collector=stat 2024-02-23T00:40:04.310 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.996Z caller=node_exporter.go:115 level=info collector=tapestats 2024-02-23T00:40:04.310 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.996Z caller=node_exporter.go:115 level=info collector=textfile 2024-02-23T00:40:04.310 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.996Z caller=node_exporter.go:115 level=info collector=thermal_zone 2024-02-23T00:40:04.310 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.996Z caller=node_exporter.go:115 level=info collector=time 2024-02-23T00:40:04.310 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.996Z caller=node_exporter.go:115 level=info collector=udp_queues 2024-02-23T00:40:04.310 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.996Z caller=node_exporter.go:115 level=info collector=uname 2024-02-23T00:40:04.310 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.996Z caller=node_exporter.go:115 level=info collector=vmstat 2024-02-23T00:40:04.311 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.996Z caller=node_exporter.go:115 level=info collector=xfs 2024-02-23T00:40:04.311 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.996Z caller=node_exporter.go:115 level=info collector=zfs 2024-02-23T00:40:04.311 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.996Z caller=node_exporter.go:199 level=info msg="Listening on" address=:9100 2024-02-23T00:40:04.311 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a[67837]: ts=2024-02-23T00:40:03.996Z caller=tls_config.go:195 level=info msg="TLS is disabled." http2=false 2024-02-23T00:40:04.311 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:04 smithi047 bash[67730]: 8a4bb7c0a897f8403eee45d63919a506a840cff75626bc7b512c3b4698370f7a 2024-02-23T00:40:04.311 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 00:40:04 smithi047 systemd[1]: Started Ceph node-exporter.a for fce2617e-d1e2-11ee-95c0-87774f69a715. 2024-02-23T00:40:04.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:03.128338+0000 mgr.y (mgr.14517) 25 : cluster [DBG] pgmap v18: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:40:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:03.996843+0000 mon.c (mon.2) 19 : audit [DBG] from='client.? 172.21.15.47:0/1954863607' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".rgw.root", "var": "pg_num"}]: dispatch 2024-02-23T00:40:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:04.193251+0000 mon.a (mon.0) 685 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:04.316720+0000 mon.a (mon.0) 686 : cluster [DBG] osdmap e78: 8 total, 8 up, 8 in 2024-02-23T00:40:04.783 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:03.128338+0000 mgr.y (mgr.14517) 25 : cluster [DBG] pgmap v18: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:40:04.783 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:03.996843+0000 mon.c (mon.2) 19 : audit [DBG] from='client.? 172.21.15.47:0/1954863607' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".rgw.root", "var": "pg_num"}]: dispatch 2024-02-23T00:40:04.784 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:04.193251+0000 mon.a (mon.0) 685 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:04.784 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:04.316720+0000 mon.a (mon.0) 686 : cluster [DBG] osdmap e78: 8 total, 8 up, 8 in 2024-02-23T00:40:04.784 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:03.128338+0000 mgr.y (mgr.14517) 25 : cluster [DBG] pgmap v18: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:40:04.784 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:03.996843+0000 mon.c (mon.2) 19 : audit [DBG] from='client.? 172.21.15.47:0/1954863607' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".rgw.root", "var": "pg_num"}]: dispatch 2024-02-23T00:40:04.784 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:04.193251+0000 mon.a (mon.0) 685 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:04.784 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:04.316720+0000 mon.a (mon.0) 686 : cluster [DBG] osdmap e78: 8 total, 8 up, 8 in 2024-02-23T00:40:04.808 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd pool get iscsi pg_num 2024-02-23T00:40:05.092 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:40:05.193 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:04 smithi196 systemd[1]: Starting Ceph node-exporter.b for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T00:40:05.193 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:04 smithi196 bash[62620]: Trying to pull quay.io/prometheus/node-exporter:v1.3.1... 2024-02-23T00:40:05.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:40:04.194068+0000 mgr.y (mgr.14517) 26 : cephadm [INF] Deploying daemon node-exporter.b on smithi196 2024-02-23T00:40:05.693 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:05 smithi196 bash[62620]: Getting image source signatures 2024-02-23T00:40:05.693 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:05 smithi196 bash[62620]: Copying blob sha256:aa2a8d90b84cb2a9c422e7005cd166a008ccf22ef5d7d4f07128478585ce35ea 2024-02-23T00:40:05.693 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:05 smithi196 bash[62620]: Copying blob sha256:b45d31ee2d7f9f452678a85b0c837c29e12089f31ee8dbac6c8c24dfa4054a30 2024-02-23T00:40:05.693 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:05 smithi196 bash[62620]: Copying blob sha256:b5db1e299295edf3005515ab7879c1df64a33c185d3a7a23aa4dcaa17d26f7b3 2024-02-23T00:40:05.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:40:04.194068+0000 mgr.y (mgr.14517) 26 : cephadm [INF] Deploying daemon node-exporter.b on smithi196 2024-02-23T00:40:05.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:40:04.194068+0000 mgr.y (mgr.14517) 26 : cephadm [INF] Deploying daemon node-exporter.b on smithi196 2024-02-23T00:40:06.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:05.128702+0000 mgr.y (mgr.14517) 27 : cluster [DBG] pgmap v20: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:40:06.759 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:05.128702+0000 mgr.y (mgr.14517) 27 : cluster [DBG] pgmap v20: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:40:06.759 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:05.128702+0000 mgr.y (mgr.14517) 27 : cluster [DBG] pgmap v20: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:40:07.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.064+0000 7f7efedc5700 1 -- 172.21.15.47:0/3713288807 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7ef80f8cb0 msgr2=0x7f7ef80f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:07.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.064+0000 7f7efedc5700 1 --2- 172.21.15.47:0/3713288807 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7ef80f8cb0 0x7f7ef80f9090 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f7eec004870 tx=0x7f7eec009cb0).stop 2024-02-23T00:40:07.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.064+0000 7f7efedc5700 1 -- 172.21.15.47:0/3713288807 shutdown_connections 2024-02-23T00:40:07.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.064+0000 7f7efedc5700 1 --2- 172.21.15.47:0/3713288807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ef8102210 0x7f7ef8104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:07.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.064+0000 7f7efedc5700 1 --2- 172.21.15.47:0/3713288807 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7ef80f8cb0 0x7f7ef80f9090 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:07.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.064+0000 7f7efedc5700 1 --2- 172.21.15.47:0/3713288807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ef80f9610 0x7f7ef8101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:07.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.064+0000 7f7efedc5700 1 -- 172.21.15.47:0/3713288807 >> 172.21.15.47:0/3713288807 conn(0x7f7ef80f2820 msgr2=0x7f7ef80f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:40:07.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.064+0000 7f7efedc5700 1 -- 172.21.15.47:0/3713288807 shutdown_connections 2024-02-23T00:40:07.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.064+0000 7f7efedc5700 1 -- 172.21.15.47:0/3713288807 wait complete. 2024-02-23T00:40:07.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.064+0000 7f7efedc5700 1 librados: starting msgr at 2024-02-23T00:40:07.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.064+0000 7f7efedc5700 1 librados: starting objecter 2024-02-23T00:40:07.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.065+0000 7f7efedc5700 1 Processor -- start 2024-02-23T00:40:07.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.065+0000 7f7efedc5700 1 -- start start 2024-02-23T00:40:07.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.065+0000 7f7efedc5700 1 librados: setting wanted keys 2024-02-23T00:40:07.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.065+0000 7f7efedc5700 1 librados: calling monclient init 2024-02-23T00:40:07.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.065+0000 7f7efedc5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ef80f8cb0 0x7f7ef810e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:07.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.065+0000 7f7efedc5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ef80f9610 0x7f7ef810e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:07.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.065+0000 7f7efedc5700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7ef8102210 0x7f7ef8108280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:07.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.065+0000 7f7efedc5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef8108770 con 0x7f7ef80f8cb0 2024-02-23T00:40:07.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.065+0000 7f7efedc5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef81088e0 con 0x7f7ef80f9610 2024-02-23T00:40:07.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.065+0000 7f7efedc5700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef8108aa0 con 0x7f7ef8102210 2024-02-23T00:40:07.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.065+0000 7f7efcb61700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ef80f8cb0 0x7f7ef810e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:07.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.065+0000 7f7efcb61700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ef80f8cb0 0x7f7ef810e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37806/0 (socket says 172.21.15.47:37806) 2024-02-23T00:40:07.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.065+0000 7f7efcb61700 1 -- 172.21.15.47:0/176879040 learned_addr learned my addr 172.21.15.47:0/176879040 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:40:07.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.065+0000 7f7efd362700 1 --2- 172.21.15.47:0/176879040 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7ef8102210 0x7f7ef8108280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:07.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.065+0000 7f7ef7fff700 1 --2- 172.21.15.47:0/176879040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ef80f9610 0x7f7ef810e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:07.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.065+0000 7f7efcb61700 1 -- 172.21.15.47:0/176879040 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7ef8102210 msgr2=0x7f7ef8108280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:07.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.065+0000 7f7efcb61700 1 --2- 172.21.15.47:0/176879040 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7ef8102210 0x7f7ef8108280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:07.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.065+0000 7f7efcb61700 1 -- 172.21.15.47:0/176879040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ef80f9610 msgr2=0x7f7ef810e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:07.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.065+0000 7f7efcb61700 1 --2- 172.21.15.47:0/176879040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ef80f9610 0x7f7ef810e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:07.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.065+0000 7f7efcb61700 1 -- 172.21.15.47:0/176879040 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef00097c0 con 0x7f7ef80f8cb0 2024-02-23T00:40:07.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.065+0000 7f7efd362700 1 --2- 172.21.15.47:0/176879040 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7ef8102210 0x7f7ef8108280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:40:07.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.066+0000 7f7efcb61700 1 --2- 172.21.15.47:0/176879040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ef80f8cb0 0x7f7ef810e0e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f7eec000e50 tx=0x7f7eec009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:07.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.066+0000 7f7ef5ffb700 10 client.?.objecter ms_handle_connect 0x7f7ef80f8cb0 2024-02-23T00:40:07.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.066+0000 7f7ef5ffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:40:07.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.066+0000 7f7ef5ffb700 1 -- 172.21.15.47:0/176879040 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec00e030 con 0x7f7ef80f8cb0 2024-02-23T00:40:07.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.066+0000 7f7efedc5700 1 -- 172.21.15.47:0/176879040 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eec005040 con 0x7f7ef80f8cb0 2024-02-23T00:40:07.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.066+0000 7f7ef5ffb700 1 -- 172.21.15.47:0/176879040 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f7eec013d70 con 0x7f7ef80f8cb0 2024-02-23T00:40:07.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.066+0000 7f7ef5ffb700 1 -- 172.21.15.47:0/176879040 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec018070 con 0x7f7ef80f8cb0 2024-02-23T00:40:07.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.066+0000 7f7efedc5700 10 client.14571.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:40:07.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.066+0000 7f7efedc5700 1 -- 172.21.15.47:0/176879040 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ef8108f40 con 0x7f7ef80f8cb0 2024-02-23T00:40:07.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.066+0000 7f7efedc5700 1 librados: init done 2024-02-23T00:40:07.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.067+0000 7f7efedc5700 1 -- 172.21.15.47:0/176879040 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ef8109130 con 0x7f7ef80f8cb0 2024-02-23T00:40:07.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.067+0000 7f7ef5ffb700 1 -- 172.21.15.47:0/176879040 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f7eec009240 con 0x7f7ef80f8cb0 2024-02-23T00:40:07.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.068+0000 7f7ef5ffb700 1 --2- 172.21.15.47:0/176879040 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7ee8062cc0 0x7f7ee8065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:07.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.068+0000 7f7ef5ffb700 1 -- 172.21.15.47:0/176879040 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7305+0+0 (secure 0 0 0) 0x7f7eec07e050 con 0x7f7ef80f8cb0 2024-02-23T00:40:07.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.068+0000 7f7ef5ffb700 10 client.14571.objecter ms_dispatch 0x7f7ef8001550 osd_map(78..78 src has 1..78) v4 2024-02-23T00:40:07.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.068+0000 7f7ef5ffb700 3 client.14571.objecter handle_osd_map got epochs [78,78] > 0 2024-02-23T00:40:07.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.068+0000 7f7ef5ffb700 3 client.14571.objecter handle_osd_map decoding full epoch 78 2024-02-23T00:40:07.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.068+0000 7f7ef7fff700 1 --2- 172.21.15.47:0/176879040 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7ee8062cc0 0x7f7ee8065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:07.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.068+0000 7f7ef5ffb700 20 client.14571.objecter dump_active .. 0 homeless 2024-02-23T00:40:07.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.068+0000 7f7ef5ffb700 10 client.14571.objecter ms_handle_connect 0x7f7ee8062cc0 2024-02-23T00:40:07.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.068+0000 7f7ef7fff700 1 --2- 172.21.15.47:0/176879040 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7ee8062cc0 0x7f7ee8065180 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f7ef8109a20 tx=0x7f7ee400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:07.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.072+0000 7f7ef5ffb700 1 -- 172.21.15.47:0/176879040 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f7eec04de20 con 0x7f7ef80f8cb0 2024-02-23T00:40:07.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.243+0000 7f7efedc5700 1 -- 172.21.15.47:0/176879040 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "iscsi", "var": "pg_num"} v 0) v1 -- 0x7f7ef805fa60 con 0x7f7ef80f8cb0 2024-02-23T00:40:07.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.244+0000 7f7ef5ffb700 1 -- 172.21.15.47:0/176879040 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "iscsi", "var": "pg_num"}]=0 v78) v1 ==== 94+0+10 (secure 0 0 0) 0x7f7ef805fa60 con 0x7f7ef80f8cb0 2024-02-23T00:40:07.246 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 3 2024-02-23T00:40:07.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.245+0000 7f7efedc5700 10 librados: watch_flush enter 2024-02-23T00:40:07.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.245+0000 7f7efedc5700 10 librados: watch_flush exit 2024-02-23T00:40:07.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.245+0000 7f7efedc5700 20 client.14571.objecter shutdown clearing up homeless session... 2024-02-23T00:40:07.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.245+0000 7f7efedc5700 10 client.14571.objecter successfully canceled tick 2024-02-23T00:40:07.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.246+0000 7f7efedc5700 1 -- 172.21.15.47:0/176879040 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7ee8062cc0 msgr2=0x7f7ee8065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:07.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.246+0000 7f7efedc5700 1 --2- 172.21.15.47:0/176879040 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7ee8062cc0 0x7f7ee8065180 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f7ef8109a20 tx=0x7f7ee400b040).stop 2024-02-23T00:40:07.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.246+0000 7f7efedc5700 1 -- 172.21.15.47:0/176879040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ef80f8cb0 msgr2=0x7f7ef810e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:07.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.246+0000 7f7efedc5700 1 --2- 172.21.15.47:0/176879040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ef80f8cb0 0x7f7ef810e0e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f7eec000e50 tx=0x7f7eec009cb0).stop 2024-02-23T00:40:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.246+0000 7f7efedc5700 1 -- 172.21.15.47:0/176879040 shutdown_connections 2024-02-23T00:40:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.246+0000 7f7efedc5700 1 --2- 172.21.15.47:0/176879040 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7ee8062cc0 0x7f7ee8065180 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.246+0000 7f7efedc5700 1 --2- 172.21.15.47:0/176879040 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7ef8102210 0x7f7ef8108280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.246+0000 7f7efedc5700 1 --2- 172.21.15.47:0/176879040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ef80f8cb0 0x7f7ef810e0e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.246+0000 7f7efedc5700 1 --2- 172.21.15.47:0/176879040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ef80f9610 0x7f7ef810e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.246+0000 7f7efedc5700 1 -- 172.21.15.47:0/176879040 >> 172.21.15.47:0/176879040 conn(0x7f7ef80f2820 msgr2=0x7f7ef80f34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:40:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.246+0000 7f7efedc5700 1 -- 172.21.15.47:0/176879040 shutdown_connections 2024-02-23T00:40:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.246+0000 7f7efedc5700 1 -- 172.21.15.47:0/176879040 wait complete. 2024-02-23T00:40:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:07.246+0000 7f7efedc5700 1 librados: shutdown 2024-02-23T00:40:07.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:07.245290+0000 mon.a (mon.0) 687 : audit [DBG] from='client.? 172.21.15.47:0/176879040' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "iscsi", "var": "pg_num"}]: dispatch 2024-02-23T00:40:07.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:07.245290+0000 mon.a (mon.0) 687 : audit [DBG] from='client.? 172.21.15.47:0/176879040' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "iscsi", "var": "pg_num"}]: dispatch 2024-02-23T00:40:07.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:07.245290+0000 mon.a (mon.0) 687 : audit [DBG] from='client.? 172.21.15.47:0/176879040' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "iscsi", "var": "pg_num"}]: dispatch 2024-02-23T00:40:07.693 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:07 smithi196 bash[62620]: Copying config sha256:1dbe0e931976487e20e5cfb272087e08a9779c88fd5e9617ed7042dd9751ec26 2024-02-23T00:40:07.693 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:07 smithi196 bash[62620]: Writing manifest to image destination 2024-02-23T00:40:07.913 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd pool get default.rgw.log pg_num 2024-02-23T00:40:08.195 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:40:08.673 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:07.129267+0000 mgr.y (mgr.14517) 28 : cluster [DBG] pgmap v21: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:40:08.673 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:07.129267+0000 mgr.y (mgr.14517) 28 : cluster [DBG] pgmap v21: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:40:08.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:07.129267+0000 mgr.y (mgr.14517) 28 : cluster [DBG] pgmap v21: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:40:08.693 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.280Z caller=node_exporter.go:182 level=info msg="Starting node_exporter" version="(version=1.3.1, branch=HEAD, revision=a2321e7b940ddcff26873612bccdf7cd4c42b6b6)" 2024-02-23T00:40:08.694 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.280Z caller=node_exporter.go:183 level=info msg="Build context" build_context="(go=go1.17.3, user=root@243aafa5525c, date=20211205-11:09:49)" 2024-02-23T00:40:08.694 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+)($|/) 2024-02-23T00:40:08.694 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-23T00:40:08.694 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:108 level=info msg="Enabled collectors" 2024-02-23T00:40:08.694 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=arp 2024-02-23T00:40:08.694 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=bcache 2024-02-23T00:40:08.694 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=bonding 2024-02-23T00:40:08.694 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=btrfs 2024-02-23T00:40:08.694 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=conntrack 2024-02-23T00:40:08.694 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=cpu 2024-02-23T00:40:08.694 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=cpufreq 2024-02-23T00:40:08.694 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=diskstats 2024-02-23T00:40:08.694 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=dmi 2024-02-23T00:40:08.695 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=edac 2024-02-23T00:40:08.695 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=entropy 2024-02-23T00:40:08.695 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=fibrechannel 2024-02-23T00:40:08.695 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=filefd 2024-02-23T00:40:08.695 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=filesystem 2024-02-23T00:40:08.695 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=hwmon 2024-02-23T00:40:08.695 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=infiniband 2024-02-23T00:40:08.695 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=ipvs 2024-02-23T00:40:08.695 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=loadavg 2024-02-23T00:40:08.695 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=mdadm 2024-02-23T00:40:08.695 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=meminfo 2024-02-23T00:40:08.695 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=netclass 2024-02-23T00:40:08.696 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=netdev 2024-02-23T00:40:08.696 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=netstat 2024-02-23T00:40:08.696 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=nfs 2024-02-23T00:40:08.696 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=nfsd 2024-02-23T00:40:08.696 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=nvme 2024-02-23T00:40:08.696 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=os 2024-02-23T00:40:08.696 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=powersupplyclass 2024-02-23T00:40:08.696 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=pressure 2024-02-23T00:40:08.696 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=rapl 2024-02-23T00:40:08.696 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=schedstat 2024-02-23T00:40:08.696 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=sockstat 2024-02-23T00:40:08.696 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=softnet 2024-02-23T00:40:08.696 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=stat 2024-02-23T00:40:08.697 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=tapestats 2024-02-23T00:40:08.697 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=textfile 2024-02-23T00:40:08.697 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=thermal_zone 2024-02-23T00:40:08.697 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=time 2024-02-23T00:40:08.697 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=udp_queues 2024-02-23T00:40:08.697 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=uname 2024-02-23T00:40:08.697 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=vmstat 2024-02-23T00:40:08.697 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=xfs 2024-02-23T00:40:08.698 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:115 level=info collector=zfs 2024-02-23T00:40:08.698 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=node_exporter.go:199 level=info msg="Listening on" address=:9100 2024-02-23T00:40:08.698 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b[62672]: ts=2024-02-23T00:40:08.281Z caller=tls_config.go:195 level=info msg="TLS is disabled." http2=false 2024-02-23T00:40:08.698 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 bash[62620]: 768ec2ae32b224d5036447b5d02293005de63e5c59ad14af1b65cc89b5b18a82 2024-02-23T00:40:08.698 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 00:40:08 smithi196 systemd[1]: Started Ceph node-exporter.b for fce2617e-d1e2-11ee-95c0-87774f69a715. 2024-02-23T00:40:09.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:08.484278+0000 mon.a (mon.0) 688 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:09.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:40:08.497347+0000 mgr.y (mgr.14517) 29 : cephadm [INF] Deploying daemon prometheus.a on smithi196 2024-02-23T00:40:09.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:08.484278+0000 mon.a (mon.0) 688 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:09.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:40:08.497347+0000 mgr.y (mgr.14517) 29 : cephadm [INF] Deploying daemon prometheus.a on smithi196 2024-02-23T00:40:09.808 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:40:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[67409]: level=info ts=2024-02-23T00:40:09.506Z caller=cluster.go:688 component=cluster msg="gossip settled; proceeding" elapsed=10.002919282s 2024-02-23T00:40:09.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:08.484278+0000 mon.a (mon.0) 688 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:09.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:40:08.497347+0000 mgr.y (mgr.14517) 29 : cephadm [INF] Deploying daemon prometheus.a on smithi196 2024-02-23T00:40:10.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.199+0000 7f69ea0dc700 1 -- 172.21.15.47:0/2124699878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69e40fe510 msgr2=0x7f69e40f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:10.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.199+0000 7f69ea0dc700 1 --2- 172.21.15.47:0/2124699878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69e40fe510 0x7f69e40f7d40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f69d4004870 tx=0x7f69d4009cb0).stop 2024-02-23T00:40:10.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.199+0000 7f69ea0dc700 1 -- 172.21.15.47:0/2124699878 shutdown_connections 2024-02-23T00:40:10.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.199+0000 7f69ea0dc700 1 --2- 172.21.15.47:0/2124699878 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f69e40f8420 0x7f69e4104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:10.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.199+0000 7f69ea0dc700 1 --2- 172.21.15.47:0/2124699878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69e40fdbb0 0x7f69e40fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:10.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.199+0000 7f69ea0dc700 1 --2- 172.21.15.47:0/2124699878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69e40fe510 0x7f69e40f7d40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:10.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.199+0000 7f69ea0dc700 1 -- 172.21.15.47:0/2124699878 >> 172.21.15.47:0/2124699878 conn(0x7f69e40f3800 msgr2=0x7f69e40f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:40:10.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.199+0000 7f69ea0dc700 1 -- 172.21.15.47:0/2124699878 shutdown_connections 2024-02-23T00:40:10.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.199+0000 7f69ea0dc700 1 -- 172.21.15.47:0/2124699878 wait complete. 2024-02-23T00:40:10.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.200+0000 7f69ea0dc700 1 librados: starting msgr at 2024-02-23T00:40:10.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.200+0000 7f69ea0dc700 1 librados: starting objecter 2024-02-23T00:40:10.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.200+0000 7f69ea0dc700 1 Processor -- start 2024-02-23T00:40:10.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.200+0000 7f69ea0dc700 1 -- start start 2024-02-23T00:40:10.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.200+0000 7f69ea0dc700 1 librados: setting wanted keys 2024-02-23T00:40:10.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.200+0000 7f69ea0dc700 1 librados: calling monclient init 2024-02-23T00:40:10.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.200+0000 7f69ea0dc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f69e40f8420 0x7f69e406b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:10.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.200+0000 7f69ea0dc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69e40fdbb0 0x7f69e406b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:10.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.200+0000 7f69ea0dc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69e40fe510 0x7f69e406bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:10.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.200+0000 7f69ea0dc700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e4116380 con 0x7f69e40fe510 2024-02-23T00:40:10.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.200+0000 7f69ea0dc700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e41164f0 con 0x7f69e40fdbb0 2024-02-23T00:40:10.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.200+0000 7f69ea0dc700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e41166b0 con 0x7f69e40f8420 2024-02-23T00:40:10.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.200+0000 7f69e37fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f69e40f8420 0x7f69e406b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:10.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.200+0000 7f69e3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69e40fe510 0x7f69e406bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:10.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.200+0000 7f69e37fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f69e40f8420 0x7f69e406b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:58934/0 (socket says 172.21.15.47:58934) 2024-02-23T00:40:10.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.200+0000 7f69e37fe700 1 -- 172.21.15.47:0/1277109678 learned_addr learned my addr 172.21.15.47:0/1277109678 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:40:10.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.200+0000 7f69e2ffd700 1 --2- 172.21.15.47:0/1277109678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69e40fdbb0 0x7f69e406b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:10.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.201+0000 7f69e37fe700 1 -- 172.21.15.47:0/1277109678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69e40fdbb0 msgr2=0x7f69e406b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:10.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.201+0000 7f69e37fe700 1 --2- 172.21.15.47:0/1277109678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69e40fdbb0 0x7f69e406b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:10.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.201+0000 7f69e37fe700 1 -- 172.21.15.47:0/1277109678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69e40fe510 msgr2=0x7f69e406bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:10.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.201+0000 7f69e37fe700 1 --2- 172.21.15.47:0/1277109678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69e40fe510 0x7f69e406bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:10.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.201+0000 7f69e37fe700 1 -- 172.21.15.47:0/1277109678 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69dc0097c0 con 0x7f69e40f8420 2024-02-23T00:40:10.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.201+0000 7f69e3fff700 1 --2- 172.21.15.47:0/1277109678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69e40fe510 0x7f69e406bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:40:10.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.201+0000 7f69e2ffd700 1 --2- 172.21.15.47:0/1277109678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69e40fdbb0 0x7f69e406b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:40:10.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.201+0000 7f69e37fe700 1 --2- 172.21.15.47:0/1277109678 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f69e40f8420 0x7f69e406b0d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f69d800e970 tx=0x7f69d800b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:10.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.201+0000 7f69e0ff9700 10 client.?.objecter ms_handle_connect 0x7f69e40f8420 2024-02-23T00:40:10.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.201+0000 7f69e0ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:40:10.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.201+0000 7f69e0ff9700 1 -- 172.21.15.47:0/1277109678 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d8004660 con 0x7f69e40f8420 2024-02-23T00:40:10.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.201+0000 7f69e0ff9700 1 -- 172.21.15.47:0/1277109678 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f69d800f040 con 0x7f69e40f8420 2024-02-23T00:40:10.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.201+0000 7f69ea0dc700 1 -- 172.21.15.47:0/1277109678 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69d4005040 con 0x7f69e40f8420 2024-02-23T00:40:10.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.201+0000 7f69e0ff9700 1 -- 172.21.15.47:0/1277109678 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d80146f0 con 0x7f69e40f8420 2024-02-23T00:40:10.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.201+0000 7f69ea0dc700 10 client.24470.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:40:10.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.201+0000 7f69ea0dc700 1 -- 172.21.15.47:0/1277109678 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e4116d90 con 0x7f69e40f8420 2024-02-23T00:40:10.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.201+0000 7f69ea0dc700 1 librados: init done 2024-02-23T00:40:10.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.202+0000 7f69ea0dc700 1 -- 172.21.15.47:0/1277109678 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69e4106e60 con 0x7f69e40f8420 2024-02-23T00:40:10.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.203+0000 7f69e0ff9700 1 -- 172.21.15.47:0/1277109678 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f69d8007b40 con 0x7f69e40f8420 2024-02-23T00:40:10.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.203+0000 7f69e0ff9700 1 --2- 172.21.15.47:0/1277109678 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f69cc062cc0 0x7f69cc065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:10.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.204+0000 7f69e0ff9700 1 -- 172.21.15.47:0/1277109678 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7305+0+0 (secure 0 0 0) 0x7f69d807cce0 con 0x7f69e40f8420 2024-02-23T00:40:10.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.204+0000 7f69e2ffd700 1 --2- 172.21.15.47:0/1277109678 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f69cc062cc0 0x7f69cc065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:10.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.204+0000 7f69e0ff9700 10 client.24470.objecter ms_dispatch 0x7f69e4001550 osd_map(78..78 src has 1..78) v4 2024-02-23T00:40:10.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.204+0000 7f69e0ff9700 3 client.24470.objecter handle_osd_map got epochs [78,78] > 0 2024-02-23T00:40:10.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.204+0000 7f69e0ff9700 3 client.24470.objecter handle_osd_map decoding full epoch 78 2024-02-23T00:40:10.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.204+0000 7f69e0ff9700 20 client.24470.objecter dump_active .. 0 homeless 2024-02-23T00:40:10.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.204+0000 7f69e0ff9700 10 client.24470.objecter ms_handle_connect 0x7f69cc062cc0 2024-02-23T00:40:10.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.204+0000 7f69e2ffd700 1 --2- 172.21.15.47:0/1277109678 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f69cc062cc0 0x7f69cc065180 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f69d4004ee0 tx=0x7f69d400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:10.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.208+0000 7f69e0ff9700 1 -- 172.21.15.47:0/1277109678 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f69d804cba0 con 0x7f69e40f8420 2024-02-23T00:40:10.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.381+0000 7f69ea0dc700 1 -- 172.21.15.47:0/1277109678 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "default.rgw.log", "var": "pg_num"} v 0) v1 -- 0x7f69e405faf0 con 0x7f69e40f8420 2024-02-23T00:40:10.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.382+0000 7f69e0ff9700 1 -- 172.21.15.47:0/1277109678 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "default.rgw.log", "var": "pg_num"}]=0 v78) v1 ==== 104+0+11 (secure 0 0 0) 0x7f69d80230a0 con 0x7f69e40f8420 2024-02-23T00:40:10.384 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-02-23T00:40:10.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.383+0000 7f69ea0dc700 10 librados: watch_flush enter 2024-02-23T00:40:10.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.383+0000 7f69ea0dc700 10 librados: watch_flush exit 2024-02-23T00:40:10.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.383+0000 7f69ea0dc700 20 client.24470.objecter shutdown clearing up homeless session... 2024-02-23T00:40:10.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.383+0000 7f69ea0dc700 10 client.24470.objecter successfully canceled tick 2024-02-23T00:40:10.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.383+0000 7f69ea0dc700 1 -- 172.21.15.47:0/1277109678 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f69cc062cc0 msgr2=0x7f69cc065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:10.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.383+0000 7f69ea0dc700 1 --2- 172.21.15.47:0/1277109678 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f69cc062cc0 0x7f69cc065180 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f69d4004ee0 tx=0x7f69d400b040).stop 2024-02-23T00:40:10.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.383+0000 7f69ea0dc700 1 -- 172.21.15.47:0/1277109678 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f69e40f8420 msgr2=0x7f69e406b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:10.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.383+0000 7f69ea0dc700 1 --2- 172.21.15.47:0/1277109678 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f69e40f8420 0x7f69e406b0d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f69d800e970 tx=0x7f69d800b040).stop 2024-02-23T00:40:10.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.384+0000 7f69ea0dc700 1 -- 172.21.15.47:0/1277109678 shutdown_connections 2024-02-23T00:40:10.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.384+0000 7f69ea0dc700 1 --2- 172.21.15.47:0/1277109678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69e40fe510 0x7f69e406bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:10.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.384+0000 7f69ea0dc700 1 --2- 172.21.15.47:0/1277109678 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f69cc062cc0 0x7f69cc065180 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:10.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.384+0000 7f69ea0dc700 1 --2- 172.21.15.47:0/1277109678 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f69e40f8420 0x7f69e406b0d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:10.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.384+0000 7f69ea0dc700 1 --2- 172.21.15.47:0/1277109678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69e40fdbb0 0x7f69e406b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:10.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.384+0000 7f69ea0dc700 1 -- 172.21.15.47:0/1277109678 >> 172.21.15.47:0/1277109678 conn(0x7f69e40f3800 msgr2=0x7f69e40fb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:40:10.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.384+0000 7f69ea0dc700 1 -- 172.21.15.47:0/1277109678 shutdown_connections 2024-02-23T00:40:10.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.384+0000 7f69ea0dc700 1 -- 172.21.15.47:0/1277109678 wait complete. 2024-02-23T00:40:10.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:10.384+0000 7f69ea0dc700 1 librados: shutdown 2024-02-23T00:40:10.739 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:09.129612+0000 mgr.y (mgr.14517) 30 : cluster [DBG] pgmap v22: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:40:10.739 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:10.383234+0000 mon.c (mon.2) 20 : audit [DBG] from='client.? 172.21.15.47:0/1277109678' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.log", "var": "pg_num"}]: dispatch 2024-02-23T00:40:10.739 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:09.129612+0000 mgr.y (mgr.14517) 30 : cluster [DBG] pgmap v22: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:40:10.739 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:10.383234+0000 mon.c (mon.2) 20 : audit [DBG] from='client.? 172.21.15.47:0/1277109678' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.log", "var": "pg_num"}]: dispatch 2024-02-23T00:40:10.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:09.129612+0000 mgr.y (mgr.14517) 30 : cluster [DBG] pgmap v22: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:40:10.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:10.383234+0000 mon.c (mon.2) 20 : audit [DBG] from='client.? 172.21.15.47:0/1277109678' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.log", "var": "pg_num"}]: dispatch 2024-02-23T00:40:11.169 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd pool get default.rgw.control pg_num 2024-02-23T00:40:11.453 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:40:12.283 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:11.129932+0000 mgr.y (mgr.14517) 31 : cluster [DBG] pgmap v23: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:40:12.283 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:11.282184+0000 mon.a (mon.0) 689 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:12.443 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:40:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:40:12.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:11.129932+0000 mgr.y (mgr.14517) 31 : cluster [DBG] pgmap v23: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:40:12.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:11.282184+0000 mon.a (mon.0) 689 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:12.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:11.129932+0000 mgr.y (mgr.14517) 31 : cluster [DBG] pgmap v23: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:40:12.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:11.282184+0000 mon.a (mon.0) 689 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:13.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:12.132620+0000 mgr.y (mgr.14517) 32 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:40:13.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:12.401241+0000 osd.4 (osd.4) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:40:13.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:12.401286+0000 osd.4 (osd.4) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:40:13.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:12.132620+0000 mgr.y (mgr.14517) 32 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:40:13.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:12.401241+0000 osd.4 (osd.4) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:40:13.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:12.401286+0000 osd.4 (osd.4) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:40:13.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.366+0000 7ff765b7c700 1 -- 172.21.15.47:0/2244355311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7600f9570 msgr2=0x7ff760101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:13.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.366+0000 7ff765b7c700 1 --2- 172.21.15.47:0/2244355311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7600f9570 0x7ff760101c80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7ff754004870 tx=0x7ff754009cb0).stop 2024-02-23T00:40:13.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.366+0000 7ff765b7c700 1 -- 172.21.15.47:0/2244355311 shutdown_connections 2024-02-23T00:40:13.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.366+0000 7ff765b7c700 1 --2- 172.21.15.47:0/2244355311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff760102170 0x7ff760104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:13.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.366+0000 7ff765b7c700 1 --2- 172.21.15.47:0/2244355311 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff7600f8c10 0x7ff7600f8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:13.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.366+0000 7ff765b7c700 1 --2- 172.21.15.47:0/2244355311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7600f9570 0x7ff760101c80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:13.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.366+0000 7ff765b7c700 1 -- 172.21.15.47:0/2244355311 >> 172.21.15.47:0/2244355311 conn(0x7ff7600f26c0 msgr2=0x7ff7600f4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:40:13.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.366+0000 7ff765b7c700 1 -- 172.21.15.47:0/2244355311 shutdown_connections 2024-02-23T00:40:13.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.366+0000 7ff765b7c700 1 -- 172.21.15.47:0/2244355311 wait complete. 2024-02-23T00:40:13.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.367+0000 7ff765b7c700 1 librados: starting msgr at 2024-02-23T00:40:13.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.367+0000 7ff765b7c700 1 librados: starting objecter 2024-02-23T00:40:13.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.367+0000 7ff765b7c700 1 Processor -- start 2024-02-23T00:40:13.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.367+0000 7ff765b7c700 1 -- start start 2024-02-23T00:40:13.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.367+0000 7ff765b7c700 1 librados: setting wanted keys 2024-02-23T00:40:13.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.367+0000 7ff765b7c700 1 librados: calling monclient init 2024-02-23T00:40:13.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.367+0000 7ff765b7c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7600f8c10 0x7ff76010df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:13.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.368+0000 7ff765b7c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7600f9570 0x7ff76010e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:13.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.368+0000 7ff765b7c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff760102170 0x7ff7601080f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:13.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.368+0000 7ff765b7c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7601085e0 con 0x7ff7600f9570 2024-02-23T00:40:13.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.368+0000 7ff765b7c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff760108750 con 0x7ff7600f8c10 2024-02-23T00:40:13.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.368+0000 7ff765b7c700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff760108910 con 0x7ff760102170 2024-02-23T00:40:13.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.368+0000 7ff75effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7600f9570 0x7ff76010e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:13.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.368+0000 7ff75effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7600f9570 0x7ff76010e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37866/0 (socket says 172.21.15.47:37866) 2024-02-23T00:40:13.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.368+0000 7ff75ffff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff760102170 0x7ff7601080f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:13.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.368+0000 7ff75effd700 1 -- 172.21.15.47:0/1190155140 learned_addr learned my addr 172.21.15.47:0/1190155140 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:40:13.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.368+0000 7ff75f7fe700 1 --2- 172.21.15.47:0/1190155140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7600f8c10 0x7ff76010df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:13.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.368+0000 7ff75effd700 1 -- 172.21.15.47:0/1190155140 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff760102170 msgr2=0x7ff7601080f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:13.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.368+0000 7ff75effd700 1 --2- 172.21.15.47:0/1190155140 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff760102170 0x7ff7601080f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:13.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.368+0000 7ff75effd700 1 -- 172.21.15.47:0/1190155140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7600f8c10 msgr2=0x7ff76010df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:13.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.368+0000 7ff75effd700 1 --2- 172.21.15.47:0/1190155140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7600f8c10 0x7ff76010df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:13.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.368+0000 7ff75effd700 1 -- 172.21.15.47:0/1190155140 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7500097c0 con 0x7ff7600f9570 2024-02-23T00:40:13.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.368+0000 7ff75ffff700 1 --2- 172.21.15.47:0/1190155140 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff760102170 0x7ff7601080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:40:13.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.368+0000 7ff75effd700 1 --2- 172.21.15.47:0/1190155140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7600f9570 0x7ff76010e440 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7ff754009da0 tx=0x7ff754009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:13.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.368+0000 7ff75cff9700 10 client.?.objecter ms_handle_connect 0x7ff7600f9570 2024-02-23T00:40:13.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.368+0000 7ff75cff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:40:13.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.368+0000 7ff75cff9700 1 -- 172.21.15.47:0/1190155140 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75400e030 con 0x7ff7600f9570 2024-02-23T00:40:13.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.368+0000 7ff75cff9700 1 -- 172.21.15.47:0/1190155140 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7ff754013d70 con 0x7ff7600f9570 2024-02-23T00:40:13.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.368+0000 7ff765b7c700 1 -- 172.21.15.47:0/1190155140 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff754005040 con 0x7ff7600f9570 2024-02-23T00:40:13.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.368+0000 7ff75cff9700 1 -- 172.21.15.47:0/1190155140 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754018070 con 0x7ff7600f9570 2024-02-23T00:40:13.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.368+0000 7ff765b7c700 10 client.14583.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:40:13.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.368+0000 7ff765b7c700 1 -- 172.21.15.47:0/1190155140 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff760108db0 con 0x7ff7600f9570 2024-02-23T00:40:13.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.368+0000 7ff765b7c700 1 librados: init done 2024-02-23T00:40:13.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.369+0000 7ff765b7c700 1 -- 172.21.15.47:0/1190155140 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff760108fa0 con 0x7ff7600f9570 2024-02-23T00:40:13.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.370+0000 7ff75cff9700 1 -- 172.21.15.47:0/1190155140 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7ff754009240 con 0x7ff7600f9570 2024-02-23T00:40:13.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.370+0000 7ff75cff9700 1 --2- 172.21.15.47:0/1190155140 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff748062cc0 0x7ff748065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:13.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.370+0000 7ff75cff9700 1 -- 172.21.15.47:0/1190155140 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7305+0+0 (secure 0 0 0) 0x7ff75407e000 con 0x7ff7600f9570 2024-02-23T00:40:13.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.370+0000 7ff75cff9700 10 client.14583.objecter ms_dispatch 0x7ff760001550 osd_map(79..79 src has 1..79) v4 2024-02-23T00:40:13.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.370+0000 7ff75cff9700 3 client.14583.objecter handle_osd_map got epochs [79,79] > 0 2024-02-23T00:40:13.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.370+0000 7ff75cff9700 3 client.14583.objecter handle_osd_map decoding full epoch 79 2024-02-23T00:40:13.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.370+0000 7ff75f7fe700 1 --2- 172.21.15.47:0/1190155140 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff748062cc0 0x7ff748065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:13.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.370+0000 7ff75cff9700 20 client.14583.objecter dump_active .. 0 homeless 2024-02-23T00:40:13.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.371+0000 7ff75cff9700 10 client.14583.objecter ms_handle_connect 0x7ff748062cc0 2024-02-23T00:40:13.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.371+0000 7ff75f7fe700 1 --2- 172.21.15.47:0/1190155140 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff748062cc0 0x7ff748065180 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7ff750001490 tx=0x7ff750004c50).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:13.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.375+0000 7ff75cff9700 1 -- 172.21.15.47:0/1190155140 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ff75404de70 con 0x7ff7600f9570 2024-02-23T00:40:13.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.546+0000 7ff765b7c700 1 -- 172.21.15.47:0/1190155140 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "default.rgw.control", "var": "pg_num"} v 0) v1 -- 0x7ff760106f10 con 0x7ff7600f9570 2024-02-23T00:40:13.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.546+0000 7ff75cff9700 1 -- 172.21.15.47:0/1190155140 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "default.rgw.control", "var": "pg_num"}]=0 v79) v1 ==== 108+0+11 (secure 0 0 0) 0x7ff754051490 con 0x7ff7600f9570 2024-02-23T00:40:13.549 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-02-23T00:40:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.548+0000 7ff765b7c700 10 librados: watch_flush enter 2024-02-23T00:40:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.548+0000 7ff765b7c700 10 librados: watch_flush exit 2024-02-23T00:40:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.548+0000 7ff765b7c700 20 client.14583.objecter shutdown clearing up homeless session... 2024-02-23T00:40:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.548+0000 7ff765b7c700 10 client.14583.objecter successfully canceled tick 2024-02-23T00:40:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.548+0000 7ff765b7c700 1 -- 172.21.15.47:0/1190155140 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff748062cc0 msgr2=0x7ff748065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.548+0000 7ff765b7c700 1 --2- 172.21.15.47:0/1190155140 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff748062cc0 0x7ff748065180 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7ff750001490 tx=0x7ff750004c50).stop 2024-02-23T00:40:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.548+0000 7ff765b7c700 1 -- 172.21.15.47:0/1190155140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7600f9570 msgr2=0x7ff76010e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.548+0000 7ff765b7c700 1 --2- 172.21.15.47:0/1190155140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7600f9570 0x7ff76010e440 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7ff754009da0 tx=0x7ff754009cb0).stop 2024-02-23T00:40:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.548+0000 7ff765b7c700 1 -- 172.21.15.47:0/1190155140 shutdown_connections 2024-02-23T00:40:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.548+0000 7ff765b7c700 1 --2- 172.21.15.47:0/1190155140 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff748062cc0 0x7ff748065180 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.548+0000 7ff765b7c700 1 --2- 172.21.15.47:0/1190155140 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff760102170 0x7ff7601080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.548+0000 7ff765b7c700 1 --2- 172.21.15.47:0/1190155140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7600f8c10 0x7ff76010df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.548+0000 7ff765b7c700 1 --2- 172.21.15.47:0/1190155140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7600f9570 0x7ff76010e440 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.548+0000 7ff765b7c700 1 -- 172.21.15.47:0/1190155140 >> 172.21.15.47:0/1190155140 conn(0x7ff7600f26c0 msgr2=0x7ff7600f3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:40:13.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.548+0000 7ff765b7c700 1 -- 172.21.15.47:0/1190155140 shutdown_connections 2024-02-23T00:40:13.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.548+0000 7ff765b7c700 1 -- 172.21.15.47:0/1190155140 wait complete. 2024-02-23T00:40:13.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:13.548+0000 7ff765b7c700 1 librados: shutdown 2024-02-23T00:40:13.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:12.132620+0000 mgr.y (mgr.14517) 32 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:40:13.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:12.401241+0000 osd.4 (osd.4) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:40:13.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:12.401286+0000 osd.4 (osd.4) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:40:14.216 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd pool get default.rgw.meta pg_num 2024-02-23T00:40:14.500 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:40:14.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:13.130735+0000 mgr.y (mgr.14517) 33 : cluster [DBG] pgmap v24: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:40:14.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:13.289529+0000 mon.a (mon.0) 690 : cluster [DBG] osdmap e79: 8 total, 8 up, 8 in 2024-02-23T00:40:14.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:13.547806+0000 mon.a (mon.0) 691 : audit [DBG] from='client.? 172.21.15.47:0/1190155140' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.control", "var": "pg_num"}]: dispatch 2024-02-23T00:40:14.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:13.130735+0000 mgr.y (mgr.14517) 33 : cluster [DBG] pgmap v24: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:40:14.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:13.289529+0000 mon.a (mon.0) 690 : cluster [DBG] osdmap e79: 8 total, 8 up, 8 in 2024-02-23T00:40:14.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:13.547806+0000 mon.a (mon.0) 691 : audit [DBG] from='client.? 172.21.15.47:0/1190155140' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.control", "var": "pg_num"}]: dispatch 2024-02-23T00:40:14.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:13.130735+0000 mgr.y (mgr.14517) 33 : cluster [DBG] pgmap v24: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:40:14.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:13.289529+0000 mon.a (mon.0) 690 : cluster [DBG] osdmap e79: 8 total, 8 up, 8 in 2024-02-23T00:40:14.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:13.547806+0000 mon.a (mon.0) 691 : audit [DBG] from='client.? 172.21.15.47:0/1190155140' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.control", "var": "pg_num"}]: dispatch 2024-02-23T00:40:15.553 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:14.319217+0000 mon.a (mon.0) 692 : cluster [DBG] osdmap e80: 8 total, 8 up, 8 in 2024-02-23T00:40:15.554 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:14.319217+0000 mon.a (mon.0) 692 : cluster [DBG] osdmap e80: 8 total, 8 up, 8 in 2024-02-23T00:40:15.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:14.319217+0000 mon.a (mon.0) 692 : cluster [DBG] osdmap e80: 8 total, 8 up, 8 in 2024-02-23T00:40:16.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.538+0000 7f65a3be8700 1 -- 172.21.15.47:0/522579650 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f659c0fdbb0 msgr2=0x7f659c0fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:16.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.538+0000 7f65a3be8700 1 --2- 172.21.15.47:0/522579650 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f659c0fdbb0 0x7f659c0fdf90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f658c0047b0 tx=0x7f658c009cb0).stop 2024-02-23T00:40:16.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.538+0000 7f65a3be8700 1 -- 172.21.15.47:0/522579650 shutdown_connections 2024-02-23T00:40:16.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.538+0000 7f65a3be8700 1 --2- 172.21.15.47:0/522579650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f659c0f8420 0x7f659c104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:16.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.538+0000 7f65a3be8700 1 --2- 172.21.15.47:0/522579650 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f659c0fdbb0 0x7f659c0fdf90 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:16.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.538+0000 7f65a3be8700 1 --2- 172.21.15.47:0/522579650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f659c0fe510 0x7f659c0f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:16.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.538+0000 7f65a3be8700 1 -- 172.21.15.47:0/522579650 >> 172.21.15.47:0/522579650 conn(0x7f659c0f3800 msgr2=0x7f659c0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:40:16.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.538+0000 7f65a3be8700 1 -- 172.21.15.47:0/522579650 shutdown_connections 2024-02-23T00:40:16.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.538+0000 7f65a3be8700 1 -- 172.21.15.47:0/522579650 wait complete. 2024-02-23T00:40:16.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.539+0000 7f65a3be8700 1 librados: starting msgr at 2024-02-23T00:40:16.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.539+0000 7f65a3be8700 1 librados: starting objecter 2024-02-23T00:40:16.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.539+0000 7f65a3be8700 1 Processor -- start 2024-02-23T00:40:16.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.539+0000 7f65a3be8700 1 -- start start 2024-02-23T00:40:16.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.539+0000 7f65a3be8700 1 librados: setting wanted keys 2024-02-23T00:40:16.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.539+0000 7f65a3be8700 1 librados: calling monclient init 2024-02-23T00:40:16.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.539+0000 7f65a3be8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f659c0f8420 0x7f659c10df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:16.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.539+0000 7f65a3be8700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f659c0fdbb0 0x7f659c10e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:16.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.539+0000 7f65a3be8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f659c0fe510 0x7f659c108120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:16.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.539+0000 7f65a3be8700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659c108610 con 0x7f659c0f8420 2024-02-23T00:40:16.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.539+0000 7f65a3be8700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659c108780 con 0x7f659c0fe510 2024-02-23T00:40:16.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.539+0000 7f65a3be8700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f659c108940 con 0x7f659c0fdbb0 2024-02-23T00:40:16.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.539+0000 7f65a1183700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f659c0fdbb0 0x7f659c10e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:16.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.539+0000 7f65a1984700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f659c0f8420 0x7f659c10df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:16.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.539+0000 7f65a1183700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f659c0fdbb0 0x7f659c10e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:55606/0 (socket says 172.21.15.47:55606) 2024-02-23T00:40:16.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.539+0000 7f65a1183700 1 -- 172.21.15.47:0/1424668923 learned_addr learned my addr 172.21.15.47:0/1424668923 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:40:16.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.539+0000 7f65a2185700 1 --2- 172.21.15.47:0/1424668923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f659c0fe510 0x7f659c108120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:16.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.540+0000 7f65a1984700 1 -- 172.21.15.47:0/1424668923 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f659c0fdbb0 msgr2=0x7f659c10e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:16.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.540+0000 7f65a1984700 1 --2- 172.21.15.47:0/1424668923 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f659c0fdbb0 0x7f659c10e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:16.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.540+0000 7f65a1984700 1 -- 172.21.15.47:0/1424668923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f659c0fe510 msgr2=0x7f659c108120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:16.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.540+0000 7f65a1984700 1 --2- 172.21.15.47:0/1424668923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f659c0fe510 0x7f659c108120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:16.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.540+0000 7f65a1984700 1 -- 172.21.15.47:0/1424668923 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6598005040 con 0x7f659c0f8420 2024-02-23T00:40:16.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.540+0000 7f65a1984700 1 --2- 172.21.15.47:0/1424668923 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f659c0f8420 0x7f659c10df80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f658c004650 tx=0x7f658c009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:16.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.540+0000 7f6592ffd700 10 client.?.objecter ms_handle_connect 0x7f659c0f8420 2024-02-23T00:40:16.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.540+0000 7f6592ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:40:16.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.540+0000 7f6592ffd700 1 -- 172.21.15.47:0/1424668923 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c018070 con 0x7f659c0f8420 2024-02-23T00:40:16.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.540+0000 7f6592ffd700 1 -- 172.21.15.47:0/1424668923 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f658c013dd0 con 0x7f659c0f8420 2024-02-23T00:40:16.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.540+0000 7f6592ffd700 1 -- 172.21.15.47:0/1424668923 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c01da30 con 0x7f659c0f8420 2024-02-23T00:40:16.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.540+0000 7f65a3be8700 1 -- 172.21.15.47:0/1424668923 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f658c005040 con 0x7f659c0f8420 2024-02-23T00:40:16.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.540+0000 7f65a3be8700 10 client.14589.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:40:16.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.540+0000 7f65a3be8700 1 -- 172.21.15.47:0/1424668923 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f659c108e60 con 0x7f659c0f8420 2024-02-23T00:40:16.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.540+0000 7f65a3be8700 1 librados: init done 2024-02-23T00:40:16.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.541+0000 7f65a3be8700 1 -- 172.21.15.47:0/1424668923 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f659c05ee80 con 0x7f659c0f8420 2024-02-23T00:40:16.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.542+0000 7f6592ffd700 1 -- 172.21.15.47:0/1424668923 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f658c009240 con 0x7f659c0f8420 2024-02-23T00:40:16.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.542+0000 7f6592ffd700 1 --2- 172.21.15.47:0/1424668923 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6588062cc0 0x7f6588065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:16.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.542+0000 7f6592ffd700 1 -- 172.21.15.47:0/1424668923 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7305+0+0 (secure 0 0 0) 0x7f658c07e4e0 con 0x7f659c0f8420 2024-02-23T00:40:16.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.542+0000 7f6592ffd700 10 client.14589.objecter ms_dispatch 0x7f659c001550 osd_map(80..80 src has 1..80) v4 2024-02-23T00:40:16.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.542+0000 7f6592ffd700 3 client.14589.objecter handle_osd_map got epochs [80,80] > 0 2024-02-23T00:40:16.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.542+0000 7f6592ffd700 3 client.14589.objecter handle_osd_map decoding full epoch 80 2024-02-23T00:40:16.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.542+0000 7f65a1183700 1 --2- 172.21.15.47:0/1424668923 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6588062cc0 0x7f6588065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:16.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.542+0000 7f6592ffd700 20 client.14589.objecter dump_active .. 0 homeless 2024-02-23T00:40:16.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.543+0000 7f6592ffd700 10 client.14589.objecter ms_handle_connect 0x7f6588062cc0 2024-02-23T00:40:16.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.543+0000 7f65a1183700 1 --2- 172.21.15.47:0/1424668923 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6588062cc0 0x7f6588065180 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f659c1098c0 tx=0x7f659800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:16.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.546+0000 7f6592ffd700 1 -- 172.21.15.47:0/1424668923 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f658c04e400 con 0x7f659c0f8420 2024-02-23T00:40:16.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:15.131292+0000 mgr.y (mgr.14517) 34 : cluster [DBG] pgmap v27: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:40:16.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:15.940611+0000 osd.5 (osd.5) 5 : cluster [DBG] 2.18 deep-scrub starts 2024-02-23T00:40:16.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:15.942720+0000 osd.5 (osd.5) 6 : cluster [DBG] 2.18 deep-scrub ok 2024-02-23T00:40:16.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:15.131292+0000 mgr.y (mgr.14517) 34 : cluster [DBG] pgmap v27: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:40:16.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:15.940611+0000 osd.5 (osd.5) 5 : cluster [DBG] 2.18 deep-scrub starts 2024-02-23T00:40:16.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:15.942720+0000 osd.5 (osd.5) 6 : cluster [DBG] 2.18 deep-scrub ok 2024-02-23T00:40:16.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:15.131292+0000 mgr.y (mgr.14517) 34 : cluster [DBG] pgmap v27: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:40:16.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:15.940611+0000 osd.5 (osd.5) 5 : cluster [DBG] 2.18 deep-scrub starts 2024-02-23T00:40:16.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:15.942720+0000 osd.5 (osd.5) 6 : cluster [DBG] 2.18 deep-scrub ok 2024-02-23T00:40:16.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.717+0000 7f65a3be8700 1 -- 172.21.15.47:0/1424668923 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "default.rgw.meta", "var": "pg_num"} v 0) v1 -- 0x7f659c106e90 con 0x7f659c0f8420 2024-02-23T00:40:16.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.718+0000 7f6592ffd700 1 -- 172.21.15.47:0/1424668923 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "default.rgw.meta", "var": "pg_num"}]=0 v80) v1 ==== 105+0+11 (secure 0 0 0) 0x7f658c01de60 con 0x7f659c0f8420 2024-02-23T00:40:16.720 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-02-23T00:40:16.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.719+0000 7f65a3be8700 10 librados: watch_flush enter 2024-02-23T00:40:16.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.719+0000 7f65a3be8700 10 librados: watch_flush exit 2024-02-23T00:40:16.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.719+0000 7f65a3be8700 20 client.14589.objecter shutdown clearing up homeless session... 2024-02-23T00:40:16.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.719+0000 7f65a3be8700 10 client.14589.objecter successfully canceled tick 2024-02-23T00:40:16.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.719+0000 7f65a3be8700 1 -- 172.21.15.47:0/1424668923 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6588062cc0 msgr2=0x7f6588065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:16.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.719+0000 7f65a3be8700 1 --2- 172.21.15.47:0/1424668923 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6588062cc0 0x7f6588065180 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f659c1098c0 tx=0x7f659800b040).stop 2024-02-23T00:40:16.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.719+0000 7f65a3be8700 1 -- 172.21.15.47:0/1424668923 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f659c0f8420 msgr2=0x7f659c10df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:16.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.719+0000 7f65a3be8700 1 --2- 172.21.15.47:0/1424668923 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f659c0f8420 0x7f659c10df80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f658c004650 tx=0x7f658c009cb0).stop 2024-02-23T00:40:16.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.719+0000 7f65a3be8700 1 -- 172.21.15.47:0/1424668923 shutdown_connections 2024-02-23T00:40:16.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.719+0000 7f65a3be8700 1 --2- 172.21.15.47:0/1424668923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f659c0fe510 0x7f659c108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:16.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.719+0000 7f65a3be8700 1 --2- 172.21.15.47:0/1424668923 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f659c0f8420 0x7f659c10df80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:16.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.719+0000 7f65a3be8700 1 --2- 172.21.15.47:0/1424668923 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6588062cc0 0x7f6588065180 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:16.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.719+0000 7f65a3be8700 1 --2- 172.21.15.47:0/1424668923 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f659c0fdbb0 0x7f659c10e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:16.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.719+0000 7f65a3be8700 1 -- 172.21.15.47:0/1424668923 >> 172.21.15.47:0/1424668923 conn(0x7f659c0f3800 msgr2=0x7f659c0fb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:40:16.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.719+0000 7f65a3be8700 1 -- 172.21.15.47:0/1424668923 shutdown_connections 2024-02-23T00:40:16.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.719+0000 7f65a3be8700 1 -- 172.21.15.47:0/1424668923 wait complete. 2024-02-23T00:40:16.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:16.720+0000 7f65a3be8700 1 librados: shutdown 2024-02-23T00:40:17.450 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:15.583752+0000 osd.0 (osd.0) 9 : cluster [DBG] 3.1 scrub starts 2024-02-23T00:40:17.450 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:15.584727+0000 osd.0 (osd.0) 10 : cluster [DBG] 3.1 scrub ok 2024-02-23T00:40:17.450 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:16.546376+0000 osd.0 (osd.0) 11 : cluster [DBG] 2.13 scrub starts 2024-02-23T00:40:17.450 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:16.547319+0000 osd.0 (osd.0) 12 : cluster [DBG] 2.13 scrub ok 2024-02-23T00:40:17.450 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:16.719055+0000 mon.a (mon.0) 693 : audit [DBG] from='client.? 172.21.15.47:0/1424668923' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.meta", "var": "pg_num"}]: dispatch 2024-02-23T00:40:17.450 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:16.892937+0000 osd.6 (osd.6) 5 : cluster [DBG] 2.14 scrub starts 2024-02-23T00:40:17.450 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:16.893871+0000 osd.6 (osd.6) 6 : cluster [DBG] 2.14 scrub ok 2024-02-23T00:40:17.450 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:15.583752+0000 osd.0 (osd.0) 9 : cluster [DBG] 3.1 scrub starts 2024-02-23T00:40:17.451 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:15.584727+0000 osd.0 (osd.0) 10 : cluster [DBG] 3.1 scrub ok 2024-02-23T00:40:17.451 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:16.546376+0000 osd.0 (osd.0) 11 : cluster [DBG] 2.13 scrub starts 2024-02-23T00:40:17.451 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:16.547319+0000 osd.0 (osd.0) 12 : cluster [DBG] 2.13 scrub ok 2024-02-23T00:40:17.451 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:16.719055+0000 mon.a (mon.0) 693 : audit [DBG] from='client.? 172.21.15.47:0/1424668923' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.meta", "var": "pg_num"}]: dispatch 2024-02-23T00:40:17.451 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:16.892937+0000 osd.6 (osd.6) 5 : cluster [DBG] 2.14 scrub starts 2024-02-23T00:40:17.451 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:16.893871+0000 osd.6 (osd.6) 6 : cluster [DBG] 2.14 scrub ok 2024-02-23T00:40:17.471 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-23T00:40:17.471 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-23T00:40:17.471 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph mgr dump --format=json 2024-02-23T00:40:17.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:15.583752+0000 osd.0 (osd.0) 9 : cluster [DBG] 3.1 scrub starts 2024-02-23T00:40:17.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:15.584727+0000 osd.0 (osd.0) 10 : cluster [DBG] 3.1 scrub ok 2024-02-23T00:40:17.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:16.546376+0000 osd.0 (osd.0) 11 : cluster [DBG] 2.13 scrub starts 2024-02-23T00:40:17.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:16.547319+0000 osd.0 (osd.0) 12 : cluster [DBG] 2.13 scrub ok 2024-02-23T00:40:17.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:16.719055+0000 mon.a (mon.0) 693 : audit [DBG] from='client.? 172.21.15.47:0/1424668923' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.meta", "var": "pg_num"}]: dispatch 2024-02-23T00:40:17.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:16.892937+0000 osd.6 (osd.6) 5 : cluster [DBG] 2.14 scrub starts 2024-02-23T00:40:17.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:16.893871+0000 osd.6 (osd.6) 6 : cluster [DBG] 2.14 scrub ok 2024-02-23T00:40:17.751 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:40:18.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:17.131816+0000 mgr.y (mgr.14517) 35 : cluster [DBG] pgmap v28: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:40:18.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:17.493094+0000 osd.7 (osd.7) 5 : cluster [DBG] 2.0 scrub starts 2024-02-23T00:40:18.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:17.495989+0000 osd.7 (osd.7) 6 : cluster [DBG] 2.0 scrub ok 2024-02-23T00:40:18.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:17.131816+0000 mgr.y (mgr.14517) 35 : cluster [DBG] pgmap v28: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:40:18.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:17.493094+0000 osd.7 (osd.7) 5 : cluster [DBG] 2.0 scrub starts 2024-02-23T00:40:18.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:17.495989+0000 osd.7 (osd.7) 6 : cluster [DBG] 2.0 scrub ok 2024-02-23T00:40:18.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:17.131816+0000 mgr.y (mgr.14517) 35 : cluster [DBG] pgmap v28: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:40:18.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:17.493094+0000 osd.7 (osd.7) 5 : cluster [DBG] 2.0 scrub starts 2024-02-23T00:40:18.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:17.495989+0000 osd.7 (osd.7) 6 : cluster [DBG] 2.0 scrub ok 2024-02-23T00:40:19.517 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:18.803898+0000 osd.1 (osd.1) 11 : cluster [DBG] 4.f scrub starts 2024-02-23T00:40:19.517 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:18.805436+0000 osd.1 (osd.1) 12 : cluster [DBG] 4.f scrub ok 2024-02-23T00:40:19.518 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:18.803898+0000 osd.1 (osd.1) 11 : cluster [DBG] 4.f scrub starts 2024-02-23T00:40:19.518 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:18.805436+0000 osd.1 (osd.1) 12 : cluster [DBG] 4.f scrub ok 2024-02-23T00:40:19.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:18.803898+0000 osd.1 (osd.1) 11 : cluster [DBG] 4.f scrub starts 2024-02-23T00:40:19.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:18.805436+0000 osd.1 (osd.1) 12 : cluster [DBG] 4.f scrub ok 2024-02-23T00:40:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.764+0000 7fba4f9bc700 1 -- 172.21.15.47:0/1846124425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba480f7400 msgr2=0x7fba48103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.764+0000 7fba4f9bc700 1 --2- 172.21.15.47:0/1846124425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba480f7400 0x7fba48103a20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fba40009c30 tx=0x7fba40004c50).stop 2024-02-23T00:40:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.764+0000 7fba4f9bc700 1 -- 172.21.15.47:0/1846124425 shutdown_connections 2024-02-23T00:40:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.764+0000 7fba4f9bc700 1 --2- 172.21.15.47:0/1846124425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba480f7400 0x7fba48103a20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.764+0000 7fba4f9bc700 1 --2- 172.21.15.47:0/1846124425 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba480fcbb0 0x7fba480fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.764+0000 7fba4f9bc700 1 --2- 172.21.15.47:0/1846124425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba480fd510 0x7fba480f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.764+0000 7fba4f9bc700 1 -- 172.21.15.47:0/1846124425 >> 172.21.15.47:0/1846124425 conn(0x7fba480f2820 msgr2=0x7fba480f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:40:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.764+0000 7fba4f9bc700 1 -- 172.21.15.47:0/1846124425 shutdown_connections 2024-02-23T00:40:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.764+0000 7fba4f9bc700 1 -- 172.21.15.47:0/1846124425 wait complete. 2024-02-23T00:40:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.765+0000 7fba4f9bc700 1 librados: starting msgr at 2024-02-23T00:40:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.765+0000 7fba4f9bc700 1 librados: starting objecter 2024-02-23T00:40:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.765+0000 7fba4f9bc700 1 Processor -- start 2024-02-23T00:40:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.765+0000 7fba4f9bc700 1 -- start start 2024-02-23T00:40:19.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.765+0000 7fba4f9bc700 1 librados: setting wanted keys 2024-02-23T00:40:19.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.765+0000 7fba4f9bc700 1 librados: calling monclient init 2024-02-23T00:40:19.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.765+0000 7fba4f9bc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba480f7400 0x7fba4810de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:19.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.765+0000 7fba4f9bc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba480fcbb0 0x7fba4810e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:19.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.765+0000 7fba4f9bc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba480fd510 0x7fba48108020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:19.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.765+0000 7fba4f9bc700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba48108510 con 0x7fba480fd510 2024-02-23T00:40:19.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.765+0000 7fba4f9bc700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba48108680 con 0x7fba480f7400 2024-02-23T00:40:19.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.765+0000 7fba4f9bc700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba481087f0 con 0x7fba480fcbb0 2024-02-23T00:40:19.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.765+0000 7fba4cf57700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba480fcbb0 0x7fba4810e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:19.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.765+0000 7fba4df59700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba480fd510 0x7fba48108020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:19.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.765+0000 7fba4df59700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba480fd510 0x7fba48108020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41144/0 (socket says 172.21.15.47:41144) 2024-02-23T00:40:19.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.765+0000 7fba4cf57700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba480fcbb0 0x7fba4810e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:55634/0 (socket says 172.21.15.47:55634) 2024-02-23T00:40:19.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.765+0000 7fba4df59700 1 -- 172.21.15.47:0/924735508 learned_addr learned my addr 172.21.15.47:0/924735508 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:40:19.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.765+0000 7fba4d758700 1 --2- 172.21.15.47:0/924735508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba480f7400 0x7fba4810de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:19.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.765+0000 7fba4df59700 1 -- 172.21.15.47:0/924735508 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba480fcbb0 msgr2=0x7fba4810e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:19.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.765+0000 7fba4df59700 1 --2- 172.21.15.47:0/924735508 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba480fcbb0 0x7fba4810e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:19.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.765+0000 7fba4df59700 1 -- 172.21.15.47:0/924735508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba480f7400 msgr2=0x7fba4810de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:19.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.765+0000 7fba4df59700 1 --2- 172.21.15.47:0/924735508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba480f7400 0x7fba4810de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.765+0000 7fba4df59700 1 -- 172.21.15.47:0/924735508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba44005040 con 0x7fba480fd510 2024-02-23T00:40:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.765+0000 7fba4cf57700 1 --2- 172.21.15.47:0/924735508 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba480fcbb0 0x7fba4810e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:40:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.766+0000 7fba4df59700 1 --2- 172.21.15.47:0/924735508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba480fd510 0x7fba48108020 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fba400093e0 tx=0x7fba40004c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.766+0000 7fba3e7fc700 10 client.?.objecter ms_handle_connect 0x7fba480fd510 2024-02-23T00:40:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.766+0000 7fba3e7fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:40:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.766+0000 7fba3e7fc700 1 -- 172.21.15.47:0/924735508 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4000e040 con 0x7fba480fd510 2024-02-23T00:40:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.766+0000 7fba3e7fc700 1 -- 172.21.15.47:0/924735508 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fba40018040 con 0x7fba480fd510 2024-02-23T00:40:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.766+0000 7fba4f9bc700 1 -- 172.21.15.47:0/924735508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba400097c0 con 0x7fba480fd510 2024-02-23T00:40:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.766+0000 7fba3e7fc700 1 -- 172.21.15.47:0/924735508 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba40003940 con 0x7fba480fd510 2024-02-23T00:40:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.766+0000 7fba4f9bc700 10 client.14595.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:40:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.766+0000 7fba4f9bc700 1 -- 172.21.15.47:0/924735508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba48108c30 con 0x7fba480fd510 2024-02-23T00:40:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.766+0000 7fba4f9bc700 1 librados: init done 2024-02-23T00:40:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.767+0000 7fba4f9bc700 1 -- 172.21.15.47:0/924735508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba4805fa60 con 0x7fba480fd510 2024-02-23T00:40:19.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.768+0000 7fba3e7fc700 1 -- 172.21.15.47:0/924735508 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fba40004120 con 0x7fba480fd510 2024-02-23T00:40:19.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.768+0000 7fba3e7fc700 1 --2- 172.21.15.47:0/924735508 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fba34062cc0 0x7fba34065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:19.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.768+0000 7fba3e7fc700 1 -- 172.21.15.47:0/924735508 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7305+0+0 (secure 0 0 0) 0x7fba4007e6d0 con 0x7fba480fd510 2024-02-23T00:40:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.768+0000 7fba3e7fc700 10 client.14595.objecter ms_dispatch 0x7fba48001550 osd_map(80..80 src has 1..80) v4 2024-02-23T00:40:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.768+0000 7fba3e7fc700 3 client.14595.objecter handle_osd_map got epochs [80,80] > 0 2024-02-23T00:40:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.768+0000 7fba3e7fc700 3 client.14595.objecter handle_osd_map decoding full epoch 80 2024-02-23T00:40:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.768+0000 7fba4d758700 1 --2- 172.21.15.47:0/924735508 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fba34062cc0 0x7fba34065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.768+0000 7fba3e7fc700 20 client.14595.objecter dump_active .. 0 homeless 2024-02-23T00:40:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.769+0000 7fba3e7fc700 10 client.14595.objecter ms_handle_connect 0x7fba34062cc0 2024-02-23T00:40:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.769+0000 7fba4d758700 1 --2- 172.21.15.47:0/924735508 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fba34062cc0 0x7fba34065180 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fba44009f20 tx=0x7fba44009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.772+0000 7fba3e7fc700 1 -- 172.21.15.47:0/924735508 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fba4004e460 con 0x7fba480fd510 2024-02-23T00:40:19.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.986+0000 7fba4f9bc700 1 -- 172.21.15.47:0/924735508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fba48108eb0 con 0x7fba480fd510 2024-02-23T00:40:19.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.989+0000 7fba3e7fc700 1 -- 172.21.15.47:0/924735508 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+151996 (secure 0 0 0) 0x7fba4002a020 con 0x7fba480fd510 2024-02-23T00:40:19.994 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:40:19.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.991+0000 7fba4f9bc700 10 librados: watch_flush enter 2024-02-23T00:40:19.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.991+0000 7fba4f9bc700 10 librados: watch_flush exit 2024-02-23T00:40:19.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.991+0000 7fba4f9bc700 20 client.14595.objecter shutdown clearing up homeless session... 2024-02-23T00:40:19.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.991+0000 7fba4f9bc700 10 client.14595.objecter successfully canceled tick 2024-02-23T00:40:19.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.991+0000 7fba4f9bc700 1 -- 172.21.15.47:0/924735508 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fba34062cc0 msgr2=0x7fba34065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:19.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.991+0000 7fba4f9bc700 1 --2- 172.21.15.47:0/924735508 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fba34062cc0 0x7fba34065180 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fba44009f20 tx=0x7fba44009cb0).stop 2024-02-23T00:40:19.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.991+0000 7fba4f9bc700 1 -- 172.21.15.47:0/924735508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba480fd510 msgr2=0x7fba48108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:19.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.991+0000 7fba4f9bc700 1 --2- 172.21.15.47:0/924735508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba480fd510 0x7fba48108020 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fba400093e0 tx=0x7fba40004c50).stop 2024-02-23T00:40:19.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.991+0000 7fba4f9bc700 1 -- 172.21.15.47:0/924735508 shutdown_connections 2024-02-23T00:40:19.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.991+0000 7fba4f9bc700 1 --2- 172.21.15.47:0/924735508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba480fd510 0x7fba48108020 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:19.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.991+0000 7fba4f9bc700 1 --2- 172.21.15.47:0/924735508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba480f7400 0x7fba4810de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:19.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.991+0000 7fba4f9bc700 1 --2- 172.21.15.47:0/924735508 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fba34062cc0 0x7fba34065180 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:19.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.991+0000 7fba4f9bc700 1 --2- 172.21.15.47:0/924735508 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba480fcbb0 0x7fba4810e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:19.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.991+0000 7fba4f9bc700 1 -- 172.21.15.47:0/924735508 >> 172.21.15.47:0/924735508 conn(0x7fba480f2820 msgr2=0x7fba48060db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:40:19.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.991+0000 7fba4f9bc700 1 -- 172.21.15.47:0/924735508 shutdown_connections 2024-02-23T00:40:19.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.991+0000 7fba4f9bc700 1 -- 172.21.15.47:0/924735508 wait complete. 2024-02-23T00:40:19.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:19.991+0000 7fba4f9bc700 1 librados: shutdown 2024-02-23T00:40:20.398 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:19.121864+0000 osd.3 (osd.3) 7 : cluster [DBG] 2.1b scrub starts 2024-02-23T00:40:20.398 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:19.123119+0000 osd.3 (osd.3) 8 : cluster [DBG] 2.1b scrub ok 2024-02-23T00:40:20.398 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:19.132296+0000 mgr.y (mgr.14517) 36 : cluster [DBG] pgmap v29: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:40:20.398 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:19.484700+0000 osd.7 (osd.7) 7 : cluster [DBG] 2.b scrub starts 2024-02-23T00:40:20.398 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:19.487605+0000 osd.7 (osd.7) 8 : cluster [DBG] 2.b scrub ok 2024-02-23T00:40:20.398 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:19.988071+0000 mon.a (mon.0) 694 : audit [DBG] from='client.? 172.21.15.47:0/924735508' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-23T00:40:20.398 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 00:40:20 smithi196 systemd[1]: Starting Ceph prometheus.a for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T00:40:20.730 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:19.121864+0000 osd.3 (osd.3) 7 : cluster [DBG] 2.1b scrub starts 2024-02-23T00:40:20.730 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:19.123119+0000 osd.3 (osd.3) 8 : cluster [DBG] 2.1b scrub ok 2024-02-23T00:40:20.730 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:19.132296+0000 mgr.y (mgr.14517) 36 : cluster [DBG] pgmap v29: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:40:20.730 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:19.484700+0000 osd.7 (osd.7) 7 : cluster [DBG] 2.b scrub starts 2024-02-23T00:40:20.730 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:19.487605+0000 osd.7 (osd.7) 8 : cluster [DBG] 2.b scrub ok 2024-02-23T00:40:20.731 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:19.988071+0000 mon.a (mon.0) 694 : audit [DBG] from='client.? 172.21.15.47:0/924735508' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-23T00:40:20.731 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:19.121864+0000 osd.3 (osd.3) 7 : cluster [DBG] 2.1b scrub starts 2024-02-23T00:40:20.731 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:19.123119+0000 osd.3 (osd.3) 8 : cluster [DBG] 2.1b scrub ok 2024-02-23T00:40:20.731 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:19.132296+0000 mgr.y (mgr.14517) 36 : cluster [DBG] pgmap v29: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:40:20.731 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:19.484700+0000 osd.7 (osd.7) 7 : cluster [DBG] 2.b scrub starts 2024-02-23T00:40:20.731 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:19.487605+0000 osd.7 (osd.7) 8 : cluster [DBG] 2.b scrub ok 2024-02-23T00:40:20.731 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:19.988071+0000 mon.a (mon.0) 694 : audit [DBG] from='client.? 172.21.15.47:0/924735508' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-23T00:40:20.753 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":19,"active_gid":14517,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":1636898748},{"type":"v1","addr":"172.21.15.47:6801","nonce":1636898748}]},"active_addr":"172.21.15.47:6801/1636898748","active_change":"2024-02-23T00:39:41.110334+0000","active_mgr_features":4540138314316775423,"available":true,"standbys":[{"gid":24427,"name":"x","mgr_features":4540138314316775423,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.47:8443/","prometheus":"http://172.21.15.47:9283/"},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":73,"active_clients":[{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":658573650}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":3372597046}]}]} 2024-02-23T00:40:20.756 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-23T00:40:20.756 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-23T00:40:20.757 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd dump --format=json 2024-02-23T00:40:21.037 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:40:21.038 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 00:40:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-prometheus-a[63150]: ts=2024-02-23T00:40:21.015Z caller=main.go:512 level=info msg="Starting Prometheus" version="(version=2.33.4, branch=HEAD, revision=83032011a5d3e6102624fe58241a374a7201fee8)" 2024-02-23T00:40:21.038 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 00:40:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-prometheus-a[63150]: ts=2024-02-23T00:40:21.015Z caller=main.go:517 level=info build_context="(go=go1.17.7, user=root@d13bf69e7be8, date=20220222-16:51:28)" 2024-02-23T00:40:21.038 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 00:40:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-prometheus-a[63150]: ts=2024-02-23T00:40:21.015Z caller=main.go:518 level=info host_details="(Linux 4.18.0-540.el8.x86_64 #1 SMP Fri Feb 9 14:58:00 UTC 2024 x86_64 smithi196 (none))" 2024-02-23T00:40:21.038 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 00:40:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-prometheus-a[63150]: ts=2024-02-23T00:40:21.015Z caller=main.go:519 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-02-23T00:40:21.038 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 00:40:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-prometheus-a[63150]: ts=2024-02-23T00:40:21.015Z caller=main.go:520 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-02-23T00:40:21.038 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 00:40:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-prometheus-a[63150]: ts=2024-02-23T00:40:21.016Z caller=web.go:570 level=info component=web msg="Start listening for connections" address=:9095 2024-02-23T00:40:21.038 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 00:40:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-prometheus-a[63150]: ts=2024-02-23T00:40:21.017Z caller=main.go:923 level=info msg="Starting TSDB ..." 2024-02-23T00:40:21.038 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 00:40:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-prometheus-a[63150]: ts=2024-02-23T00:40:21.018Z caller=tls_config.go:195 level=info component=web msg="TLS is disabled." http2=false 2024-02-23T00:40:21.039 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 00:40:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-prometheus-a[63150]: ts=2024-02-23T00:40:21.020Z caller=head.go:493 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-02-23T00:40:21.039 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 00:40:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-prometheus-a[63150]: ts=2024-02-23T00:40:21.020Z caller=head.go:527 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.893µs 2024-02-23T00:40:21.039 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 00:40:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-prometheus-a[63150]: ts=2024-02-23T00:40:21.020Z caller=head.go:533 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-02-23T00:40:21.039 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 00:40:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-prometheus-a[63150]: ts=2024-02-23T00:40:21.020Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-02-23T00:40:21.039 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 00:40:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-prometheus-a[63150]: ts=2024-02-23T00:40:21.020Z caller=head.go:610 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=20.131µs wal_replay_duration=118.898µs total_replay_duration=153.09µs 2024-02-23T00:40:21.039 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 00:40:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-prometheus-a[63150]: ts=2024-02-23T00:40:21.021Z caller=main.go:944 level=info fs_type=XFS_SUPER_MAGIC 2024-02-23T00:40:21.039 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 00:40:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-prometheus-a[63150]: ts=2024-02-23T00:40:21.021Z caller=main.go:947 level=info msg="TSDB started" 2024-02-23T00:40:21.039 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 00:40:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-prometheus-a[63150]: ts=2024-02-23T00:40:21.021Z caller=main.go:1128 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-02-23T00:40:21.039 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 00:40:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-prometheus-a[63150]: ts=2024-02-23T00:40:21.037Z caller=main.go:1165 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=15.878088ms db_storage=691ns remote_storage=1.469µs web_handler=216ns query_engine=611ns scrape=452.244µs scrape_sd=21.028µs notify=17.888µs notify_sd=5.831µs rules=15.032273ms 2024-02-23T00:40:21.039 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 00:40:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-prometheus-a[63150]: ts=2024-02-23T00:40:21.037Z caller=main.go:896 level=info msg="Server is ready to receive web requests." 2024-02-23T00:40:21.343 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 00:40:21 smithi196 bash[63136]: 60bc8eab4fba3ebe27987941f661082a601092a1a4cda506f26e787f927db88c 2024-02-23T00:40:21.343 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 00:40:21 smithi196 systemd[1]: Started Ceph prometheus.a for fce2617e-d1e2-11ee-95c0-87774f69a715. 2024-02-23T00:40:21.556 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:40:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:40:21] ENGINE Bus STOPPING 2024-02-23T00:40:21.556 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:40:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:40:21] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-23T00:40:21.557 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:40:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:40:21] ENGINE Bus STOPPED 2024-02-23T00:40:21.557 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:40:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:40:21] ENGINE Bus STARTING 2024-02-23T00:40:21.557 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:40:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:40:21] ENGINE Serving on http://:::9283 2024-02-23T00:40:21.557 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:40:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:40:21] ENGINE Bus STARTED 2024-02-23T00:40:21.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:20.141525+0000 osd.3 (osd.3) 9 : cluster [DBG] 4.11 scrub starts 2024-02-23T00:40:21.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:20.143283+0000 osd.3 (osd.3) 10 : cluster [DBG] 4.11 scrub ok 2024-02-23T00:40:21.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:20.907984+0000 osd.6 (osd.6) 7 : cluster [DBG] 4.1f scrub starts 2024-02-23T00:40:21.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:20.909754+0000 osd.6 (osd.6) 8 : cluster [DBG] 4.1f scrub ok 2024-02-23T00:40:21.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:21.176936+0000 mon.a (mon.0) 695 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:21.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:21.289249+0000 mon.a (mon.0) 696 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:21.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:21.325874+0000 mon.a (mon.0) 697 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:21.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:21.328888+0000 mon.a (mon.0) 698 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:21.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:21.332133+0000 mon.a (mon.0) 699 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-23T00:40:21.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:20.141525+0000 osd.3 (osd.3) 9 : cluster [DBG] 4.11 scrub starts 2024-02-23T00:40:21.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:20.143283+0000 osd.3 (osd.3) 10 : cluster [DBG] 4.11 scrub ok 2024-02-23T00:40:21.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:20.907984+0000 osd.6 (osd.6) 7 : cluster [DBG] 4.1f scrub starts 2024-02-23T00:40:21.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:20.909754+0000 osd.6 (osd.6) 8 : cluster [DBG] 4.1f scrub ok 2024-02-23T00:40:21.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:21.176936+0000 mon.a (mon.0) 695 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:21.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:21.289249+0000 mon.a (mon.0) 696 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:21.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:21.325874+0000 mon.a (mon.0) 697 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:21.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:21.328888+0000 mon.a (mon.0) 698 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:21.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:21.332133+0000 mon.a (mon.0) 699 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-23T00:40:21.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:20.141525+0000 osd.3 (osd.3) 9 : cluster [DBG] 4.11 scrub starts 2024-02-23T00:40:21.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:20.143283+0000 osd.3 (osd.3) 10 : cluster [DBG] 4.11 scrub ok 2024-02-23T00:40:21.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:20.907984+0000 osd.6 (osd.6) 7 : cluster [DBG] 4.1f scrub starts 2024-02-23T00:40:21.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:20.909754+0000 osd.6 (osd.6) 8 : cluster [DBG] 4.1f scrub ok 2024-02-23T00:40:21.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:21.176936+0000 mon.a (mon.0) 695 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:21.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:21.289249+0000 mon.a (mon.0) 696 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:21.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:21.325874+0000 mon.a (mon.0) 697 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:21.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:21.328888+0000 mon.a (mon.0) 698 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:21.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:21.332133+0000 mon.a (mon.0) 699 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-23T00:40:22.443 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:40:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:40:22.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:21.094459+0000 osd.4 (osd.4) 7 : cluster [DBG] 4.18 scrub starts 2024-02-23T00:40:22.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:21.095670+0000 osd.4 (osd.4) 8 : cluster [DBG] 4.18 scrub ok 2024-02-23T00:40:22.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:21.132630+0000 mgr.y (mgr.14517) 37 : cluster [DBG] pgmap v30: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:40:22.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:21.305342+0000 osd.2 (osd.2) 7 : cluster [DBG] 4.1c scrub starts 2024-02-23T00:40:22.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:21.307237+0000 osd.2 (osd.2) 8 : cluster [DBG] 4.1c scrub ok 2024-02-23T00:40:22.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:21.332548+0000 mgr.y (mgr.14517) 38 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-23T00:40:22.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:21.336584+0000 mon.a (mon.0) 700 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:22.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:40:21.351226+0000 mgr.y (mgr.14517) 39 : cephadm [INF] Deploying daemon grafana.a on smithi196 2024-02-23T00:40:22.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:21.509885+0000 osd.0 (osd.0) 13 : cluster [DBG] 5.1e scrub starts 2024-02-23T00:40:22.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:21.510654+0000 osd.0 (osd.0) 14 : cluster [DBG] 5.1e scrub ok 2024-02-23T00:40:22.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:22.047588+0000 osd.5 (osd.5) 7 : cluster [DBG] 2.16 scrub starts 2024-02-23T00:40:22.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:22.050493+0000 osd.5 (osd.5) 8 : cluster [DBG] 2.16 scrub ok 2024-02-23T00:40:22.741 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:21.094459+0000 osd.4 (osd.4) 7 : cluster [DBG] 4.18 scrub starts 2024-02-23T00:40:22.741 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:21.095670+0000 osd.4 (osd.4) 8 : cluster [DBG] 4.18 scrub ok 2024-02-23T00:40:22.741 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:21.132630+0000 mgr.y (mgr.14517) 37 : cluster [DBG] pgmap v30: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:40:22.742 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:21.305342+0000 osd.2 (osd.2) 7 : cluster [DBG] 4.1c scrub starts 2024-02-23T00:40:22.742 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:21.307237+0000 osd.2 (osd.2) 8 : cluster [DBG] 4.1c scrub ok 2024-02-23T00:40:22.742 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:21.332548+0000 mgr.y (mgr.14517) 38 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-23T00:40:22.742 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:21.336584+0000 mon.a (mon.0) 700 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:22.742 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:40:21.351226+0000 mgr.y (mgr.14517) 39 : cephadm [INF] Deploying daemon grafana.a on smithi196 2024-02-23T00:40:22.742 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:21.509885+0000 osd.0 (osd.0) 13 : cluster [DBG] 5.1e scrub starts 2024-02-23T00:40:22.742 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:21.510654+0000 osd.0 (osd.0) 14 : cluster [DBG] 5.1e scrub ok 2024-02-23T00:40:22.742 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:22.047588+0000 osd.5 (osd.5) 7 : cluster [DBG] 2.16 scrub starts 2024-02-23T00:40:22.742 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:22.050493+0000 osd.5 (osd.5) 8 : cluster [DBG] 2.16 scrub ok 2024-02-23T00:40:22.742 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:21.094459+0000 osd.4 (osd.4) 7 : cluster [DBG] 4.18 scrub starts 2024-02-23T00:40:22.742 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:21.095670+0000 osd.4 (osd.4) 8 : cluster [DBG] 4.18 scrub ok 2024-02-23T00:40:22.742 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:21.132630+0000 mgr.y (mgr.14517) 37 : cluster [DBG] pgmap v30: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:40:22.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:21.305342+0000 osd.2 (osd.2) 7 : cluster [DBG] 4.1c scrub starts 2024-02-23T00:40:22.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:21.307237+0000 osd.2 (osd.2) 8 : cluster [DBG] 4.1c scrub ok 2024-02-23T00:40:22.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:21.332548+0000 mgr.y (mgr.14517) 38 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-23T00:40:22.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:21.336584+0000 mon.a (mon.0) 700 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:22.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:40:21.351226+0000 mgr.y (mgr.14517) 39 : cephadm [INF] Deploying daemon grafana.a on smithi196 2024-02-23T00:40:22.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:21.509885+0000 osd.0 (osd.0) 13 : cluster [DBG] 5.1e scrub starts 2024-02-23T00:40:22.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:21.510654+0000 osd.0 (osd.0) 14 : cluster [DBG] 5.1e scrub ok 2024-02-23T00:40:22.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:22.047588+0000 osd.5 (osd.5) 7 : cluster [DBG] 2.16 scrub starts 2024-02-23T00:40:22.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:22.050493+0000 osd.5 (osd.5) 8 : cluster [DBG] 2.16 scrub ok 2024-02-23T00:40:23.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.042+0000 7fdbd90d2700 1 -- 172.21.15.47:0/4100932374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbd40fcbb0 msgr2=0x7fdbd40fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:23.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.042+0000 7fdbd90d2700 1 --2- 172.21.15.47:0/4100932374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbd40fcbb0 0x7fdbd40fcf90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fdbc8004870 tx=0x7fdbc8009cb0).stop 2024-02-23T00:40:23.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.042+0000 7fdbd90d2700 1 -- 172.21.15.47:0/4100932374 shutdown_connections 2024-02-23T00:40:23.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.042+0000 7fdbd90d2700 1 --2- 172.21.15.47:0/4100932374 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdbd40f7400 0x7fdbd4103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:23.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.042+0000 7fdbd90d2700 1 --2- 172.21.15.47:0/4100932374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbd40fcbb0 0x7fdbd40fcf90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:23.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.042+0000 7fdbd90d2700 1 --2- 172.21.15.47:0/4100932374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbd40fd510 0x7fdbd40f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:23.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.042+0000 7fdbd90d2700 1 -- 172.21.15.47:0/4100932374 >> 172.21.15.47:0/4100932374 conn(0x7fdbd40f2820 msgr2=0x7fdbd40f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:40:23.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.042+0000 7fdbd90d2700 1 -- 172.21.15.47:0/4100932374 shutdown_connections 2024-02-23T00:40:23.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.042+0000 7fdbd90d2700 1 -- 172.21.15.47:0/4100932374 wait complete. 2024-02-23T00:40:23.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.042+0000 7fdbd90d2700 1 librados: starting msgr at 2024-02-23T00:40:23.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.042+0000 7fdbd90d2700 1 librados: starting objecter 2024-02-23T00:40:23.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.043+0000 7fdbd90d2700 1 Processor -- start 2024-02-23T00:40:23.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.043+0000 7fdbd90d2700 1 -- start start 2024-02-23T00:40:23.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.043+0000 7fdbd90d2700 1 librados: setting wanted keys 2024-02-23T00:40:23.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.043+0000 7fdbd90d2700 1 librados: calling monclient init 2024-02-23T00:40:23.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.043+0000 7fdbd90d2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbd40f7400 0x7fdbd410cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:23.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.043+0000 7fdbd90d2700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdbd40fcbb0 0x7fdbd410d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:23.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.043+0000 7fdbd90d2700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbd40fd510 0x7fdbd4107100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:23.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.043+0000 7fdbd90d2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd41075f0 con 0x7fdbd40f7400 2024-02-23T00:40:23.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.043+0000 7fdbd90d2700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd4107760 con 0x7fdbd40fd510 2024-02-23T00:40:23.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.043+0000 7fdbd90d2700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd4107920 con 0x7fdbd40fcbb0 2024-02-23T00:40:23.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.043+0000 7fdbd259c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdbd40fcbb0 0x7fdbd410d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:23.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.044+0000 7fdbd259c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdbd40fcbb0 0x7fdbd410d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:55650/0 (socket says 172.21.15.47:55650) 2024-02-23T00:40:23.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.044+0000 7fdbd259c700 1 -- 172.21.15.47:0/2625686537 learned_addr learned my addr 172.21.15.47:0/2625686537 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:40:23.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.044+0000 7fdbd2d9d700 1 --2- 172.21.15.47:0/2625686537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbd40f7400 0x7fdbd410cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:23.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.044+0000 7fdbd359e700 1 --2- 172.21.15.47:0/2625686537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbd40fd510 0x7fdbd4107100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:23.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.044+0000 7fdbd259c700 1 -- 172.21.15.47:0/2625686537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbd40fd510 msgr2=0x7fdbd4107100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.044+0000 7fdbd259c700 1 --2- 172.21.15.47:0/2625686537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbd40fd510 0x7fdbd4107100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.044+0000 7fdbd259c700 1 -- 172.21.15.47:0/2625686537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbd40f7400 msgr2=0x7fdbd410cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.044+0000 7fdbd259c700 1 --2- 172.21.15.47:0/2625686537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbd40f7400 0x7fdbd410cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.044+0000 7fdbd259c700 1 -- 172.21.15.47:0/2625686537 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc8005040 con 0x7fdbd40fcbb0 2024-02-23T00:40:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.044+0000 7fdbd2d9d700 1 --2- 172.21.15.47:0/2625686537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbd40f7400 0x7fdbd410cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:40:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.044+0000 7fdbd359e700 1 --2- 172.21.15.47:0/2625686537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbd40fd510 0x7fdbd4107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:40:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.044+0000 7fdbd259c700 1 --2- 172.21.15.47:0/2625686537 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdbd40fcbb0 0x7fdbd410d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fdbc400e970 tx=0x7fdbc400b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.044+0000 7fdbbbfff700 10 client.?.objecter ms_handle_connect 0x7fdbd40fcbb0 2024-02-23T00:40:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.044+0000 7fdbbbfff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:40:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.044+0000 7fdbbbfff700 1 -- 172.21.15.47:0/2625686537 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4004660 con 0x7fdbd40fcbb0 2024-02-23T00:40:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.044+0000 7fdbbbfff700 1 -- 172.21.15.47:0/2625686537 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fdbc400f040 con 0x7fdbd40fcbb0 2024-02-23T00:40:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.044+0000 7fdbd90d2700 1 -- 172.21.15.47:0/2625686537 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd4107ba0 con 0x7fdbd40fcbb0 2024-02-23T00:40:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.044+0000 7fdbbbfff700 1 -- 172.21.15.47:0/2625686537 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc40146f0 con 0x7fdbd40fcbb0 2024-02-23T00:40:23.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.044+0000 7fdbd90d2700 10 client.24494.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:40:23.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.044+0000 7fdbd90d2700 1 -- 172.21.15.47:0/2625686537 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd4060db0 con 0x7fdbd40fcbb0 2024-02-23T00:40:23.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.044+0000 7fdbd90d2700 1 librados: init done 2024-02-23T00:40:23.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.045+0000 7fdbd90d2700 1 -- 172.21.15.47:0/2625686537 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd405fa60 con 0x7fdbd40fcbb0 2024-02-23T00:40:23.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.046+0000 7fdbbbfff700 1 -- 172.21.15.47:0/2625686537 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fdbc4007b40 con 0x7fdbd40fcbb0 2024-02-23T00:40:23.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.046+0000 7fdbbbfff700 1 --2- 172.21.15.47:0/2625686537 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdbbc062cc0 0x7fdbbc065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:23.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.046+0000 7fdbbbfff700 1 -- 172.21.15.47:0/2625686537 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7305+0+0 (secure 0 0 0) 0x7fdbc407c980 con 0x7fdbd40fcbb0 2024-02-23T00:40:23.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.046+0000 7fdbbbfff700 10 client.24494.objecter ms_dispatch 0x7fdbd4001550 osd_map(80..80 src has 1..80) v4 2024-02-23T00:40:23.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.046+0000 7fdbbbfff700 3 client.24494.objecter handle_osd_map got epochs [80,80] > 0 2024-02-23T00:40:23.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.046+0000 7fdbbbfff700 3 client.24494.objecter handle_osd_map decoding full epoch 80 2024-02-23T00:40:23.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.046+0000 7fdbd2d9d700 1 --2- 172.21.15.47:0/2625686537 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdbbc062cc0 0x7fdbbc065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:23.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.046+0000 7fdbbbfff700 20 client.24494.objecter dump_active .. 0 homeless 2024-02-23T00:40:23.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.047+0000 7fdbbbfff700 10 client.24494.objecter ms_handle_connect 0x7fdbbc062cc0 2024-02-23T00:40:23.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.047+0000 7fdbd2d9d700 1 --2- 172.21.15.47:0/2625686537 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdbbc062cc0 0x7fdbbc065180 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fdbc8004ee0 tx=0x7fdbc800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:23.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.051+0000 7fdbbbfff700 1 -- 172.21.15.47:0/2625686537 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fdbc404c800 con 0x7fdbd40fcbb0 2024-02-23T00:40:23.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.225+0000 7fdbd90d2700 1 -- 172.21.15.47:0/2625686537 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdbd4105ea0 con 0x7fdbd40fcbb0 2024-02-23T00:40:23.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.226+0000 7fdbbbfff700 1 -- 172.21.15.47:0/2625686537 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+21164 (secure 0 0 0) 0x7fdbc4019090 con 0x7fdbd40fcbb0 2024-02-23T00:40:23.229 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:40:23.229 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":80,"fsid":"fce2617e-d1e2-11ee-95c0-87774f69a715","created":"2024-02-23T00:32:55.399437+0000","modified":"2024-02-23T00:40:14.314911+0000","last_up_change":"2024-02-23T00:39:05.662717+0000","last_in_change":"2024-02-23T00:38:47.076807+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-23T00:36:52.622994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-02-23T00:39:13.277309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"iscsi","create_time":"2024-02-23T00:39:13.761312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-23T00:39:15.255656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-23T00:39:17.226393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-23T00:39:19.300351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"380050f9-9001-46dc-bb29-037987b4c8e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":4042552860},{"type":"v1","addr":"172.21.15.47:6803","nonce":4042552860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":4042552860},{"type":"v1","addr":"172.21.15.47:6805","nonce":4042552860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":4042552860},{"type":"v1","addr":"172.21.15.47:6809","nonce":4042552860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":4042552860},{"type":"v1","addr":"172.21.15.47:6807","nonce":4042552860}]},"public_addr":"172.21.15.47:6803/4042552860","cluster_addr":"172.21.15.47:6805/4042552860","heartbeat_back_addr":"172.21.15.47:6809/4042552860","heartbeat_front_addr":"172.21.15.47:6807/4042552860","state":["exists","up"]},{"osd":1,"uuid":"b06d0cce-55d2-4a6a-a796-4db0b9bb9d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1945417179},{"type":"v1","addr":"172.21.15.47:6811","nonce":1945417179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1945417179},{"type":"v1","addr":"172.21.15.47:6813","nonce":1945417179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":1945417179},{"type":"v1","addr":"172.21.15.47:6817","nonce":1945417179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1945417179},{"type":"v1","addr":"172.21.15.47:6815","nonce":1945417179}]},"public_addr":"172.21.15.47:6811/1945417179","cluster_addr":"172.21.15.47:6813/1945417179","heartbeat_back_addr":"172.21.15.47:6817/1945417179","heartbeat_front_addr":"172.21.15.47:6815/1945417179","state":["exists","up"]},{"osd":2,"uuid":"6e44d7ca-67a1-4868-bb64-891cf477ebd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":3596356785},{"type":"v1","addr":"172.21.15.47:6819","nonce":3596356785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":3596356785},{"type":"v1","addr":"172.21.15.47:6821","nonce":3596356785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":3596356785},{"type":"v1","addr":"172.21.15.47:6825","nonce":3596356785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":3596356785},{"type":"v1","addr":"172.21.15.47:6823","nonce":3596356785}]},"public_addr":"172.21.15.47:6819/3596356785","cluster_addr":"172.21.15.47:6821/3596356785","heartbeat_back_addr":"172.21.15.47:6825/3596356785","heartbeat_front_addr":"172.21.15.47:6823/3596356785","state":["exists","up"]},{"osd":3,"uuid":"c88debab-cb0b-46f6-a7d8-c5dc61fc059f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1195438081},{"type":"v1","addr":"172.21.15.47:6827","nonce":1195438081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1195438081},{"type":"v1","addr":"172.21.15.47:6829","nonce":1195438081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1195438081},{"type":"v1","addr":"172.21.15.47:6833","nonce":1195438081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1195438081},{"type":"v1","addr":"172.21.15.47:6831","nonce":1195438081}]},"public_addr":"172.21.15.47:6827/1195438081","cluster_addr":"172.21.15.47:6829/1195438081","heartbeat_back_addr":"172.21.15.47:6833/1195438081","heartbeat_front_addr":"172.21.15.47:6831/1195438081","state":["exists","up"]},{"osd":4,"uuid":"d17966d7-eb12-4389-ac90-8d15efe290e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2530629682},{"type":"v1","addr":"172.21.15.196:6801","nonce":2530629682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2530629682},{"type":"v1","addr":"172.21.15.196:6803","nonce":2530629682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2530629682},{"type":"v1","addr":"172.21.15.196:6807","nonce":2530629682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2530629682},{"type":"v1","addr":"172.21.15.196:6805","nonce":2530629682}]},"public_addr":"172.21.15.196:6801/2530629682","cluster_addr":"172.21.15.196:6803/2530629682","heartbeat_back_addr":"172.21.15.196:6807/2530629682","heartbeat_front_addr":"172.21.15.196:6805/2530629682","state":["exists","up"]},{"osd":5,"uuid":"b729d646-4899-47b3-b917-acceba46d241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1017187491},{"type":"v1","addr":"172.21.15.196:6809","nonce":1017187491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1017187491},{"type":"v1","addr":"172.21.15.196:6811","nonce":1017187491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1017187491},{"type":"v1","addr":"172.21.15.196:6815","nonce":1017187491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1017187491},{"type":"v1","addr":"172.21.15.196:6813","nonce":1017187491}]},"public_addr":"172.21.15.196:6809/1017187491","cluster_addr":"172.21.15.196:6811/1017187491","heartbeat_back_addr":"172.21.15.196:6815/1017187491","heartbeat_front_addr":"172.21.15.196:6813/1017187491","state":["exists","up"]},{"osd":6,"uuid":"df337124-9488-4aad-98e5-3614b51fc757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3885668599},{"type":"v1","addr":"172.21.15.196:6817","nonce":3885668599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3885668599},{"type":"v1","addr":"172.21.15.196:6819","nonce":3885668599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3885668599},{"type":"v1","addr":"172.21.15.196:6823","nonce":3885668599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3885668599},{"type":"v1","addr":"172.21.15.196:6821","nonce":3885668599}]},"public_addr":"172.21.15.196:6817/3885668599","cluster_addr":"172.21.15.196:6819/3885668599","heartbeat_back_addr":"172.21.15.196:6823/3885668599","heartbeat_front_addr":"172.21.15.196:6821/3885668599","state":["exists","up"]},{"osd":7,"uuid":"84fee1f2-1a88-447a-b1f3-56ff7efe7fec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2261821675},{"type":"v1","addr":"172.21.15.196:6825","nonce":2261821675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2261821675},{"type":"v1","addr":"172.21.15.196:6827","nonce":2261821675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2261821675},{"type":"v1","addr":"172.21.15.196:6831","nonce":2261821675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2261821675},{"type":"v1","addr":"172.21.15.196:6829","nonce":2261821675}]},"public_addr":"172.21.15.196:6825/2261821675","cluster_addr":"172.21.15.196:6827/2261821675","heartbeat_back_addr":"172.21.15.196:6831/2261821675","heartbeat_front_addr":"172.21.15.196:6829/2261821675","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:39:15.242369+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:39:59.996830+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:39:21.405657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:39:47.131225+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:40:12.401288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:39:09.901475+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:39:37.023883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:39:03.696875+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6800/622994871":"2024-02-24T00:39:41.110158+0000","172.21.15.47:0/1561174431":"2024-02-24T00:39:41.110158+0000","172.21.15.47:0/3349912606":"2024-02-24T00:39:41.110158+0000","172.21.15.47:0/710432524":"2024-02-24T00:39:41.110158+0000","172.21.15.47:0/2804352395":"2024-02-24T00:33:52.061914+0000","172.21.15.47:6801/1916598887":"2024-02-24T00:33:52.061914+0000","172.21.15.47:6800/1916598887":"2024-02-24T00:33:52.061914+0000","172.21.15.47:0/792211316":"2024-02-24T00:33:27.972945+0000","172.21.15.47:0/381995198":"2024-02-24T00:33:52.061914+0000","172.21.15.47:6800/2409691510":"2024-02-24T00:33:27.972945+0000","172.21.15.47:6801/2409691510":"2024-02-24T00:33:27.972945+0000","172.21.15.47:6801/622994871":"2024-02-24T00:39:41.110158+0000","172.21.15.47:0/1448327041":"2024-02-24T00:33:27.972945+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-23T00:40:23.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.228+0000 7fdbd90d2700 10 librados: watch_flush enter 2024-02-23T00:40:23.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.228+0000 7fdbd90d2700 10 librados: watch_flush exit 2024-02-23T00:40:23.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.228+0000 7fdbd90d2700 20 client.24494.objecter shutdown clearing up homeless session... 2024-02-23T00:40:23.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.228+0000 7fdbd90d2700 10 client.24494.objecter successfully canceled tick 2024-02-23T00:40:23.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.228+0000 7fdbd90d2700 1 -- 172.21.15.47:0/2625686537 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdbbc062cc0 msgr2=0x7fdbbc065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:23.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.228+0000 7fdbd90d2700 1 --2- 172.21.15.47:0/2625686537 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdbbc062cc0 0x7fdbbc065180 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fdbc8004ee0 tx=0x7fdbc800b040).stop 2024-02-23T00:40:23.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.228+0000 7fdbd90d2700 1 -- 172.21.15.47:0/2625686537 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdbd40fcbb0 msgr2=0x7fdbd410d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:23.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.228+0000 7fdbd90d2700 1 --2- 172.21.15.47:0/2625686537 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdbd40fcbb0 0x7fdbd410d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fdbc400e970 tx=0x7fdbc400b040).stop 2024-02-23T00:40:23.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.228+0000 7fdbd90d2700 1 -- 172.21.15.47:0/2625686537 shutdown_connections 2024-02-23T00:40:23.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.228+0000 7fdbd90d2700 1 --2- 172.21.15.47:0/2625686537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbd40fd510 0x7fdbd4107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:23.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.228+0000 7fdbd90d2700 1 --2- 172.21.15.47:0/2625686537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbd40f7400 0x7fdbd410cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:23.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.228+0000 7fdbd90d2700 1 --2- 172.21.15.47:0/2625686537 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdbbc062cc0 0x7fdbbc065180 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:23.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.228+0000 7fdbd90d2700 1 --2- 172.21.15.47:0/2625686537 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdbd40fcbb0 0x7fdbd410d450 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:23.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.228+0000 7fdbd90d2700 1 -- 172.21.15.47:0/2625686537 >> 172.21.15.47:0/2625686537 conn(0x7fdbd40f2820 msgr2=0x7fdbd40fb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:40:23.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.228+0000 7fdbd90d2700 1 -- 172.21.15.47:0/2625686537 shutdown_connections 2024-02-23T00:40:23.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.228+0000 7fdbd90d2700 1 -- 172.21.15.47:0/2625686537 wait complete. 2024-02-23T00:40:23.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:23.228+0000 7fdbd90d2700 1 librados: shutdown 2024-02-23T00:40:23.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:22.142642+0000 mgr.y (mgr.14517) 40 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:40:23.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:23.226987+0000 mon.c (mon.2) 21 : audit [DBG] from='client.? 172.21.15.47:0/2625686537' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-23T00:40:23.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:22.142642+0000 mgr.y (mgr.14517) 40 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:40:23.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:23.226987+0000 mon.c (mon.2) 21 : audit [DBG] from='client.? 172.21.15.47:0/2625686537' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-23T00:40:23.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:22.142642+0000 mgr.y (mgr.14517) 40 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:40:23.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:23.226987+0000 mon.c (mon.2) 21 : audit [DBG] from='client.? 172.21.15.47:0/2625686537' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-23T00:40:24.049 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-23T00:40:24.050 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd dump --format=json 2024-02-23T00:40:24.335 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:40:24.621 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:23.102609+0000 osd.4 (osd.4) 9 : cluster [DBG] 5.1 scrub starts 2024-02-23T00:40:24.621 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:23.103578+0000 osd.4 (osd.4) 10 : cluster [DBG] 5.1 scrub ok 2024-02-23T00:40:24.621 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:23.133170+0000 mgr.y (mgr.14517) 41 : cluster [DBG] pgmap v31: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-23T00:40:24.621 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:23.358397+0000 osd.2 (osd.2) 9 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:40:24.621 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:23.358479+0000 osd.2 (osd.2) 10 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:40:24.621 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:23.571667+0000 osd.7 (osd.7) 9 : cluster [DBG] 2.8 scrub starts 2024-02-23T00:40:24.621 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:23.574807+0000 osd.7 (osd.7) 10 : cluster [DBG] 2.8 scrub ok 2024-02-23T00:40:24.622 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:23.828121+0000 osd.1 (osd.1) 13 : cluster [DBG] 5.1d scrub starts 2024-02-23T00:40:24.622 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:23.829013+0000 osd.1 (osd.1) 14 : cluster [DBG] 5.1d scrub ok 2024-02-23T00:40:24.622 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:23.102609+0000 osd.4 (osd.4) 9 : cluster [DBG] 5.1 scrub starts 2024-02-23T00:40:24.622 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:23.103578+0000 osd.4 (osd.4) 10 : cluster [DBG] 5.1 scrub ok 2024-02-23T00:40:24.622 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:23.133170+0000 mgr.y (mgr.14517) 41 : cluster [DBG] pgmap v31: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-23T00:40:24.622 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:23.358397+0000 osd.2 (osd.2) 9 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:40:24.622 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:23.358479+0000 osd.2 (osd.2) 10 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:40:24.622 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:23.571667+0000 osd.7 (osd.7) 9 : cluster [DBG] 2.8 scrub starts 2024-02-23T00:40:24.622 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:23.574807+0000 osd.7 (osd.7) 10 : cluster [DBG] 2.8 scrub ok 2024-02-23T00:40:24.622 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:23.828121+0000 osd.1 (osd.1) 13 : cluster [DBG] 5.1d scrub starts 2024-02-23T00:40:24.622 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:23.829013+0000 osd.1 (osd.1) 14 : cluster [DBG] 5.1d scrub ok 2024-02-23T00:40:24.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:23.102609+0000 osd.4 (osd.4) 9 : cluster [DBG] 5.1 scrub starts 2024-02-23T00:40:24.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:23.103578+0000 osd.4 (osd.4) 10 : cluster [DBG] 5.1 scrub ok 2024-02-23T00:40:24.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:23.133170+0000 mgr.y (mgr.14517) 41 : cluster [DBG] pgmap v31: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-23T00:40:24.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:23.358397+0000 osd.2 (osd.2) 9 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:40:24.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:23.358479+0000 osd.2 (osd.2) 10 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:40:24.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:23.571667+0000 osd.7 (osd.7) 9 : cluster [DBG] 2.8 scrub starts 2024-02-23T00:40:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:23.574807+0000 osd.7 (osd.7) 10 : cluster [DBG] 2.8 scrub ok 2024-02-23T00:40:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:23.828121+0000 osd.1 (osd.1) 13 : cluster [DBG] 5.1d scrub starts 2024-02-23T00:40:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:23.829013+0000 osd.1 (osd.1) 14 : cluster [DBG] 5.1d scrub ok 2024-02-23T00:40:25.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:24.128245+0000 osd.3 (osd.3) 11 : cluster [DBG] 6.b scrub starts 2024-02-23T00:40:25.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:24.129078+0000 osd.3 (osd.3) 12 : cluster [DBG] 6.b scrub ok 2024-02-23T00:40:25.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:24.351219+0000 mon.a (mon.0) 701 : cluster [DBG] osdmap e81: 8 total, 8 up, 8 in 2024-02-23T00:40:25.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:24.369438+0000 osd.2 (osd.2) 11 : cluster [DBG] 2.10 scrub starts 2024-02-23T00:40:25.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:24.370250+0000 osd.2 (osd.2) 12 : cluster [DBG] 2.10 scrub ok 2024-02-23T00:40:25.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:25.022932+0000 osd.5 (osd.5) 9 : cluster [DBG] 5.16 scrub starts 2024-02-23T00:40:25.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:25.025882+0000 osd.5 (osd.5) 10 : cluster [DBG] 5.16 scrub ok 2024-02-23T00:40:25.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:24.128245+0000 osd.3 (osd.3) 11 : cluster [DBG] 6.b scrub starts 2024-02-23T00:40:25.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:24.129078+0000 osd.3 (osd.3) 12 : cluster [DBG] 6.b scrub ok 2024-02-23T00:40:25.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:24.351219+0000 mon.a (mon.0) 701 : cluster [DBG] osdmap e81: 8 total, 8 up, 8 in 2024-02-23T00:40:25.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:24.369438+0000 osd.2 (osd.2) 11 : cluster [DBG] 2.10 scrub starts 2024-02-23T00:40:25.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:24.370250+0000 osd.2 (osd.2) 12 : cluster [DBG] 2.10 scrub ok 2024-02-23T00:40:25.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:25.022932+0000 osd.5 (osd.5) 9 : cluster [DBG] 5.16 scrub starts 2024-02-23T00:40:25.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:25.025882+0000 osd.5 (osd.5) 10 : cluster [DBG] 5.16 scrub ok 2024-02-23T00:40:25.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:24.128245+0000 osd.3 (osd.3) 11 : cluster [DBG] 6.b scrub starts 2024-02-23T00:40:25.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:24.129078+0000 osd.3 (osd.3) 12 : cluster [DBG] 6.b scrub ok 2024-02-23T00:40:25.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:24.351219+0000 mon.a (mon.0) 701 : cluster [DBG] osdmap e81: 8 total, 8 up, 8 in 2024-02-23T00:40:25.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:24.369438+0000 osd.2 (osd.2) 11 : cluster [DBG] 2.10 scrub starts 2024-02-23T00:40:25.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:24.370250+0000 osd.2 (osd.2) 12 : cluster [DBG] 2.10 scrub ok 2024-02-23T00:40:25.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:25.022932+0000 osd.5 (osd.5) 9 : cluster [DBG] 5.16 scrub starts 2024-02-23T00:40:25.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:25.025882+0000 osd.5 (osd.5) 10 : cluster [DBG] 5.16 scrub ok 2024-02-23T00:40:26.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:25.133539+0000 mgr.y (mgr.14517) 42 : cluster [DBG] pgmap v33: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:40:26.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:25.578059+0000 osd.7 (osd.7) 11 : cluster [DBG] 5.10 scrub starts 2024-02-23T00:40:26.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:25.580935+0000 osd.7 (osd.7) 12 : cluster [DBG] 5.10 scrub ok 2024-02-23T00:40:26.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:25.133539+0000 mgr.y (mgr.14517) 42 : cluster [DBG] pgmap v33: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:40:26.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:25.578059+0000 osd.7 (osd.7) 11 : cluster [DBG] 5.10 scrub starts 2024-02-23T00:40:26.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:25.580935+0000 osd.7 (osd.7) 12 : cluster [DBG] 5.10 scrub ok 2024-02-23T00:40:26.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:25.133539+0000 mgr.y (mgr.14517) 42 : cluster [DBG] pgmap v33: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:40:26.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:25.578059+0000 osd.7 (osd.7) 11 : cluster [DBG] 5.10 scrub starts 2024-02-23T00:40:26.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:25.580935+0000 osd.7 (osd.7) 12 : cluster [DBG] 5.10 scrub ok 2024-02-23T00:40:26.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.581+0000 7ffb9c589700 1 -- 172.21.15.47:0/2656659420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb940fe510 msgr2=0x7ffb940f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:26.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.581+0000 7ffb9c589700 1 --2- 172.21.15.47:0/2656659420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb940fe510 0x7ffb940f7d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7ffb84004870 tx=0x7ffb84009cb0).stop 2024-02-23T00:40:26.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.581+0000 7ffb9c589700 1 -- 172.21.15.47:0/2656659420 shutdown_connections 2024-02-23T00:40:26.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.581+0000 7ffb9c589700 1 --2- 172.21.15.47:0/2656659420 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffb940f8420 0x7ffb94104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:26.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.581+0000 7ffb9c589700 1 --2- 172.21.15.47:0/2656659420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb940fdbb0 0x7ffb940fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:26.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.581+0000 7ffb9c589700 1 --2- 172.21.15.47:0/2656659420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb940fe510 0x7ffb940f7d40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:26.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.581+0000 7ffb9c589700 1 -- 172.21.15.47:0/2656659420 >> 172.21.15.47:0/2656659420 conn(0x7ffb940f3800 msgr2=0x7ffb940f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:40:26.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.581+0000 7ffb9c589700 1 -- 172.21.15.47:0/2656659420 shutdown_connections 2024-02-23T00:40:26.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.581+0000 7ffb9c589700 1 -- 172.21.15.47:0/2656659420 wait complete. 2024-02-23T00:40:26.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.581+0000 7ffb9c589700 1 librados: starting msgr at 2024-02-23T00:40:26.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.581+0000 7ffb9c589700 1 librados: starting objecter 2024-02-23T00:40:26.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.582+0000 7ffb9c589700 1 Processor -- start 2024-02-23T00:40:26.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.582+0000 7ffb9c589700 1 -- start start 2024-02-23T00:40:26.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.582+0000 7ffb9c589700 1 librados: setting wanted keys 2024-02-23T00:40:26.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.582+0000 7ffb9c589700 1 librados: calling monclient init 2024-02-23T00:40:26.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.582+0000 7ffb9c589700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb940f8420 0x7ffb9410ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:26.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.582+0000 7ffb9c589700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb940fdbb0 0x7ffb9410e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:26.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.582+0000 7ffb9c589700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffb940fe510 0x7ffb94108070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:26.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.582+0000 7ffb9c589700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb94108560 con 0x7ffb940fdbb0 2024-02-23T00:40:26.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.582+0000 7ffb9c589700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb941086d0 con 0x7ffb940f8420 2024-02-23T00:40:26.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.582+0000 7ffb9c589700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb94108890 con 0x7ffb940fe510 2024-02-23T00:40:26.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.582+0000 7ffb99b24700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb940fdbb0 0x7ffb9410e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:26.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.582+0000 7ffb9ab26700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffb940fe510 0x7ffb94108070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:26.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.582+0000 7ffb9a325700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb940f8420 0x7ffb9410ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:26.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.582+0000 7ffb99b24700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb940fdbb0 0x7ffb9410e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39722/0 (socket says 172.21.15.47:39722) 2024-02-23T00:40:26.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.582+0000 7ffb9ab26700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffb940fe510 0x7ffb94108070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:45028/0 (socket says 172.21.15.47:45028) 2024-02-23T00:40:26.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.582+0000 7ffb99b24700 1 -- 172.21.15.47:0/3698681634 learned_addr learned my addr 172.21.15.47:0/3698681634 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:40:26.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.582+0000 7ffb9ab26700 1 -- 172.21.15.47:0/3698681634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb940f8420 msgr2=0x7ffb9410ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:26.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.582+0000 7ffb9ab26700 1 --2- 172.21.15.47:0/3698681634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb940f8420 0x7ffb9410ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:26.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.582+0000 7ffb9ab26700 1 -- 172.21.15.47:0/3698681634 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb940fdbb0 msgr2=0x7ffb9410e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:26.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.582+0000 7ffb9ab26700 1 --2- 172.21.15.47:0/3698681634 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb940fdbb0 0x7ffb9410e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:26.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.582+0000 7ffb9ab26700 1 -- 172.21.15.47:0/3698681634 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c0097c0 con 0x7ffb940fe510 2024-02-23T00:40:26.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.582+0000 7ffb9a325700 1 --2- 172.21.15.47:0/3698681634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb940f8420 0x7ffb9410ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:40:26.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.582+0000 7ffb99b24700 1 --2- 172.21.15.47:0/3698681634 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb940fdbb0 0x7ffb9410e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:40:26.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.582+0000 7ffb9ab26700 1 --2- 172.21.15.47:0/3698681634 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffb940fe510 0x7ffb94108070 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7ffb8c001cb0 tx=0x7ffb8c004c50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:26.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.582+0000 7ffb8b7fe700 10 client.?.objecter ms_handle_connect 0x7ffb940fe510 2024-02-23T00:40:26.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.582+0000 7ffb8b7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:40:26.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.583+0000 7ffb8b7fe700 1 -- 172.21.15.47:0/3698681634 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c0079b0 con 0x7ffb940fe510 2024-02-23T00:40:26.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.583+0000 7ffb9c589700 1 -- 172.21.15.47:0/3698681634 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb84005040 con 0x7ffb940fe510 2024-02-23T00:40:26.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.583+0000 7ffb8b7fe700 1 -- 172.21.15.47:0/3698681634 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7ffb8c0090d0 con 0x7ffb940fe510 2024-02-23T00:40:26.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.583+0000 7ffb8b7fe700 1 -- 172.21.15.47:0/3698681634 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c00fba0 con 0x7ffb940fe510 2024-02-23T00:40:26.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.583+0000 7ffb9c589700 10 client.24500.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:40:26.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.583+0000 7ffb9c589700 1 -- 172.21.15.47:0/3698681634 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb94108e80 con 0x7ffb940fe510 2024-02-23T00:40:26.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.583+0000 7ffb9c589700 1 librados: init done 2024-02-23T00:40:26.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.584+0000 7ffb9c589700 1 -- 172.21.15.47:0/3698681634 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb9405ee80 con 0x7ffb940fe510 2024-02-23T00:40:26.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.584+0000 7ffb8b7fe700 1 -- 172.21.15.47:0/3698681634 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7ffb8c007b10 con 0x7ffb940fe510 2024-02-23T00:40:26.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.584+0000 7ffb8b7fe700 1 --2- 172.21.15.47:0/3698681634 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ffb80062cc0 0x7ffb80065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:26.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.585+0000 7ffb8b7fe700 1 -- 172.21.15.47:0/3698681634 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7305+0+0 (secure 0 0 0) 0x7ffb8c07d6e0 con 0x7ffb940fe510 2024-02-23T00:40:26.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.585+0000 7ffb8b7fe700 10 client.24500.objecter ms_dispatch 0x7ffb94001550 osd_map(82..82 src has 1..82) v4 2024-02-23T00:40:26.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.585+0000 7ffb9a325700 1 --2- 172.21.15.47:0/3698681634 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ffb80062cc0 0x7ffb80065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:26.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.585+0000 7ffb8b7fe700 3 client.24500.objecter handle_osd_map got epochs [82,82] > 0 2024-02-23T00:40:26.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.585+0000 7ffb8b7fe700 3 client.24500.objecter handle_osd_map decoding full epoch 82 2024-02-23T00:40:26.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.585+0000 7ffb8b7fe700 20 client.24500.objecter dump_active .. 0 homeless 2024-02-23T00:40:26.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.585+0000 7ffb8b7fe700 10 client.24500.objecter ms_handle_connect 0x7ffb80062cc0 2024-02-23T00:40:26.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.585+0000 7ffb9a325700 1 --2- 172.21.15.47:0/3698681634 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ffb80062cc0 0x7ffb80065180 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7ffb90007bf0 tx=0x7ffb9000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:26.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.589+0000 7ffb8b7fe700 1 -- 172.21.15.47:0/3698681634 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ffb8c04d4b0 con 0x7ffb940fe510 2024-02-23T00:40:26.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.760+0000 7ffb9c589700 1 -- 172.21.15.47:0/3698681634 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb94109670 con 0x7ffb940fe510 2024-02-23T00:40:26.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.761+0000 7ffb8b7fe700 1 -- 172.21.15.47:0/3698681634 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+21164 (secure 0 0 0) 0x7ffb94109670 con 0x7ffb940fe510 2024-02-23T00:40:26.765 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:40:26.765 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":82,"fsid":"fce2617e-d1e2-11ee-95c0-87774f69a715","created":"2024-02-23T00:32:55.399437+0000","modified":"2024-02-23T00:40:26.357863+0000","last_up_change":"2024-02-23T00:39:05.662717+0000","last_in_change":"2024-02-23T00:38:47.076807+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-23T00:36:52.622994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-02-23T00:39:13.277309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"iscsi","create_time":"2024-02-23T00:39:13.761312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-23T00:39:15.255656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-23T00:39:17.226393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-23T00:39:19.300351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"380050f9-9001-46dc-bb29-037987b4c8e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":4042552860},{"type":"v1","addr":"172.21.15.47:6803","nonce":4042552860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":4042552860},{"type":"v1","addr":"172.21.15.47:6805","nonce":4042552860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":4042552860},{"type":"v1","addr":"172.21.15.47:6809","nonce":4042552860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":4042552860},{"type":"v1","addr":"172.21.15.47:6807","nonce":4042552860}]},"public_addr":"172.21.15.47:6803/4042552860","cluster_addr":"172.21.15.47:6805/4042552860","heartbeat_back_addr":"172.21.15.47:6809/4042552860","heartbeat_front_addr":"172.21.15.47:6807/4042552860","state":["exists","up"]},{"osd":1,"uuid":"b06d0cce-55d2-4a6a-a796-4db0b9bb9d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1945417179},{"type":"v1","addr":"172.21.15.47:6811","nonce":1945417179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1945417179},{"type":"v1","addr":"172.21.15.47:6813","nonce":1945417179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":1945417179},{"type":"v1","addr":"172.21.15.47:6817","nonce":1945417179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1945417179},{"type":"v1","addr":"172.21.15.47:6815","nonce":1945417179}]},"public_addr":"172.21.15.47:6811/1945417179","cluster_addr":"172.21.15.47:6813/1945417179","heartbeat_back_addr":"172.21.15.47:6817/1945417179","heartbeat_front_addr":"172.21.15.47:6815/1945417179","state":["exists","up"]},{"osd":2,"uuid":"6e44d7ca-67a1-4868-bb64-891cf477ebd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":3596356785},{"type":"v1","addr":"172.21.15.47:6819","nonce":3596356785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":3596356785},{"type":"v1","addr":"172.21.15.47:6821","nonce":3596356785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":3596356785},{"type":"v1","addr":"172.21.15.47:6825","nonce":3596356785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":3596356785},{"type":"v1","addr":"172.21.15.47:6823","nonce":3596356785}]},"public_addr":"172.21.15.47:6819/3596356785","cluster_addr":"172.21.15.47:6821/3596356785","heartbeat_back_addr":"172.21.15.47:6825/3596356785","heartbeat_front_addr":"172.21.15.47:6823/3596356785","state":["exists","up"]},{"osd":3,"uuid":"c88debab-cb0b-46f6-a7d8-c5dc61fc059f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1195438081},{"type":"v1","addr":"172.21.15.47:6827","nonce":1195438081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1195438081},{"type":"v1","addr":"172.21.15.47:6829","nonce":1195438081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1195438081},{"type":"v1","addr":"172.21.15.47:6833","nonce":1195438081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1195438081},{"type":"v1","addr":"172.21.15.47:6831","nonce":1195438081}]},"public_addr":"172.21.15.47:6827/1195438081","cluster_addr":"172.21.15.47:6829/1195438081","heartbeat_back_addr":"172.21.15.47:6833/1195438081","heartbeat_front_addr":"172.21.15.47:6831/1195438081","state":["exists","up"]},{"osd":4,"uuid":"d17966d7-eb12-4389-ac90-8d15efe290e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2530629682},{"type":"v1","addr":"172.21.15.196:6801","nonce":2530629682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2530629682},{"type":"v1","addr":"172.21.15.196:6803","nonce":2530629682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2530629682},{"type":"v1","addr":"172.21.15.196:6807","nonce":2530629682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2530629682},{"type":"v1","addr":"172.21.15.196:6805","nonce":2530629682}]},"public_addr":"172.21.15.196:6801/2530629682","cluster_addr":"172.21.15.196:6803/2530629682","heartbeat_back_addr":"172.21.15.196:6807/2530629682","heartbeat_front_addr":"172.21.15.196:6805/2530629682","state":["exists","up"]},{"osd":5,"uuid":"b729d646-4899-47b3-b917-acceba46d241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1017187491},{"type":"v1","addr":"172.21.15.196:6809","nonce":1017187491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1017187491},{"type":"v1","addr":"172.21.15.196:6811","nonce":1017187491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1017187491},{"type":"v1","addr":"172.21.15.196:6815","nonce":1017187491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1017187491},{"type":"v1","addr":"172.21.15.196:6813","nonce":1017187491}]},"public_addr":"172.21.15.196:6809/1017187491","cluster_addr":"172.21.15.196:6811/1017187491","heartbeat_back_addr":"172.21.15.196:6815/1017187491","heartbeat_front_addr":"172.21.15.196:6813/1017187491","state":["exists","up"]},{"osd":6,"uuid":"df337124-9488-4aad-98e5-3614b51fc757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3885668599},{"type":"v1","addr":"172.21.15.196:6817","nonce":3885668599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3885668599},{"type":"v1","addr":"172.21.15.196:6819","nonce":3885668599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3885668599},{"type":"v1","addr":"172.21.15.196:6823","nonce":3885668599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3885668599},{"type":"v1","addr":"172.21.15.196:6821","nonce":3885668599}]},"public_addr":"172.21.15.196:6817/3885668599","cluster_addr":"172.21.15.196:6819/3885668599","heartbeat_back_addr":"172.21.15.196:6823/3885668599","heartbeat_front_addr":"172.21.15.196:6821/3885668599","state":["exists","up"]},{"osd":7,"uuid":"84fee1f2-1a88-447a-b1f3-56ff7efe7fec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2261821675},{"type":"v1","addr":"172.21.15.196:6825","nonce":2261821675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2261821675},{"type":"v1","addr":"172.21.15.196:6827","nonce":2261821675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2261821675},{"type":"v1","addr":"172.21.15.196:6831","nonce":2261821675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2261821675},{"type":"v1","addr":"172.21.15.196:6829","nonce":2261821675}]},"public_addr":"172.21.15.196:6825/2261821675","cluster_addr":"172.21.15.196:6827/2261821675","heartbeat_back_addr":"172.21.15.196:6831/2261821675","heartbeat_front_addr":"172.21.15.196:6829/2261821675","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:39:15.242369+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:39:59.996830+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:40:23.358483+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:39:47.131225+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:40:12.401288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:39:09.901475+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:39:37.023883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:40:25.999652+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6800/622994871":"2024-02-24T00:39:41.110158+0000","172.21.15.47:0/1561174431":"2024-02-24T00:39:41.110158+0000","172.21.15.47:0/3349912606":"2024-02-24T00:39:41.110158+0000","172.21.15.47:0/710432524":"2024-02-24T00:39:41.110158+0000","172.21.15.47:0/2804352395":"2024-02-24T00:33:52.061914+0000","172.21.15.47:6801/1916598887":"2024-02-24T00:33:52.061914+0000","172.21.15.47:6800/1916598887":"2024-02-24T00:33:52.061914+0000","172.21.15.47:0/792211316":"2024-02-24T00:33:27.972945+0000","172.21.15.47:0/381995198":"2024-02-24T00:33:52.061914+0000","172.21.15.47:6800/2409691510":"2024-02-24T00:33:27.972945+0000","172.21.15.47:6801/2409691510":"2024-02-24T00:33:27.972945+0000","172.21.15.47:6801/622994871":"2024-02-24T00:39:41.110158+0000","172.21.15.47:0/1448327041":"2024-02-24T00:33:27.972945+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-23T00:40:26.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.763+0000 7ffb9c589700 10 librados: watch_flush enter 2024-02-23T00:40:26.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.763+0000 7ffb9c589700 10 librados: watch_flush exit 2024-02-23T00:40:26.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.763+0000 7ffb9c589700 20 client.24500.objecter shutdown clearing up homeless session... 2024-02-23T00:40:26.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.763+0000 7ffb9c589700 10 client.24500.objecter successfully canceled tick 2024-02-23T00:40:26.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.763+0000 7ffb9c589700 1 -- 172.21.15.47:0/3698681634 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ffb80062cc0 msgr2=0x7ffb80065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:26.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.763+0000 7ffb9c589700 1 --2- 172.21.15.47:0/3698681634 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ffb80062cc0 0x7ffb80065180 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7ffb90007bf0 tx=0x7ffb9000b040).stop 2024-02-23T00:40:26.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.763+0000 7ffb9c589700 1 -- 172.21.15.47:0/3698681634 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffb940fe510 msgr2=0x7ffb94108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:26.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.763+0000 7ffb9c589700 1 --2- 172.21.15.47:0/3698681634 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffb940fe510 0x7ffb94108070 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7ffb8c001cb0 tx=0x7ffb8c004c50).stop 2024-02-23T00:40:26.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.763+0000 7ffb9c589700 1 -- 172.21.15.47:0/3698681634 shutdown_connections 2024-02-23T00:40:26.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.763+0000 7ffb9c589700 1 --2- 172.21.15.47:0/3698681634 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ffb80062cc0 0x7ffb80065180 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:26.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.763+0000 7ffb9c589700 1 --2- 172.21.15.47:0/3698681634 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffb940fe510 0x7ffb94108070 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:26.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.763+0000 7ffb9c589700 1 --2- 172.21.15.47:0/3698681634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb940f8420 0x7ffb9410ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:26.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.763+0000 7ffb9c589700 1 --2- 172.21.15.47:0/3698681634 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb940fdbb0 0x7ffb9410e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:26.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.763+0000 7ffb9c589700 1 -- 172.21.15.47:0/3698681634 >> 172.21.15.47:0/3698681634 conn(0x7ffb940f3800 msgr2=0x7ffb940fb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:40:26.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.763+0000 7ffb9c589700 1 -- 172.21.15.47:0/3698681634 shutdown_connections 2024-02-23T00:40:26.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.763+0000 7ffb9c589700 1 -- 172.21.15.47:0/3698681634 wait complete. 2024-02-23T00:40:26.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:26.763+0000 7ffb9c589700 1 librados: shutdown 2024-02-23T00:40:27.498 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:25.999597+0000 osd.7 (osd.7) 13 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:40:27.498 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:25.999650+0000 osd.7 (osd.7) 14 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:40:27.498 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:26.170859+0000 osd.3 (osd.3) 13 : cluster [DBG] 5.17 scrub starts 2024-02-23T00:40:27.498 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:26.171863+0000 osd.3 (osd.3) 14 : cluster [DBG] 5.17 scrub ok 2024-02-23T00:40:27.499 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:26.364940+0000 mon.a (mon.0) 702 : cluster [DBG] osdmap e82: 8 total, 8 up, 8 in 2024-02-23T00:40:27.499 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:26.572231+0000 osd.7 (osd.7) 15 : cluster [DBG] 6.15 scrub starts 2024-02-23T00:40:27.499 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:26.576289+0000 osd.7 (osd.7) 16 : cluster [DBG] 6.15 scrub ok 2024-02-23T00:40:27.499 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:26.760823+0000 mon.c (mon.2) 22 : audit [DBG] from='client.? 172.21.15.47:0/3698681634' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-23T00:40:27.499 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:26.841751+0000 osd.1 (osd.1) 15 : cluster [DBG] 2.9 scrub starts 2024-02-23T00:40:27.499 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:26.844981+0000 osd.1 (osd.1) 16 : cluster [DBG] 2.9 scrub ok 2024-02-23T00:40:27.499 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:25.999597+0000 osd.7 (osd.7) 13 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:40:27.499 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:25.999650+0000 osd.7 (osd.7) 14 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:40:27.499 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:26.170859+0000 osd.3 (osd.3) 13 : cluster [DBG] 5.17 scrub starts 2024-02-23T00:40:27.499 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:26.171863+0000 osd.3 (osd.3) 14 : cluster [DBG] 5.17 scrub ok 2024-02-23T00:40:27.499 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:26.364940+0000 mon.a (mon.0) 702 : cluster [DBG] osdmap e82: 8 total, 8 up, 8 in 2024-02-23T00:40:27.499 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:26.572231+0000 osd.7 (osd.7) 15 : cluster [DBG] 6.15 scrub starts 2024-02-23T00:40:27.500 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:26.576289+0000 osd.7 (osd.7) 16 : cluster [DBG] 6.15 scrub ok 2024-02-23T00:40:27.500 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:26.760823+0000 mon.c (mon.2) 22 : audit [DBG] from='client.? 172.21.15.47:0/3698681634' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-23T00:40:27.500 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:26.841751+0000 osd.1 (osd.1) 15 : cluster [DBG] 2.9 scrub starts 2024-02-23T00:40:27.500 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:26.844981+0000 osd.1 (osd.1) 16 : cluster [DBG] 2.9 scrub ok 2024-02-23T00:40:27.522 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-23T00:40:27.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:25.999597+0000 osd.7 (osd.7) 13 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:40:27.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:25.999650+0000 osd.7 (osd.7) 14 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:40:27.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:26.170859+0000 osd.3 (osd.3) 13 : cluster [DBG] 5.17 scrub starts 2024-02-23T00:40:27.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:26.171863+0000 osd.3 (osd.3) 14 : cluster [DBG] 5.17 scrub ok 2024-02-23T00:40:27.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:26.364940+0000 mon.a (mon.0) 702 : cluster [DBG] osdmap e82: 8 total, 8 up, 8 in 2024-02-23T00:40:27.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:26.572231+0000 osd.7 (osd.7) 15 : cluster [DBG] 6.15 scrub starts 2024-02-23T00:40:27.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:26.576289+0000 osd.7 (osd.7) 16 : cluster [DBG] 6.15 scrub ok 2024-02-23T00:40:27.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:26.760823+0000 mon.c (mon.2) 22 : audit [DBG] from='client.? 172.21.15.47:0/3698681634' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-23T00:40:27.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:26.841751+0000 osd.1 (osd.1) 15 : cluster [DBG] 2.9 scrub starts 2024-02-23T00:40:27.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:26.844981+0000 osd.1 (osd.1) 16 : cluster [DBG] 2.9 scrub ok 2024-02-23T00:40:27.805 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:40:28.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:27.095677+0000 osd.4 (osd.4) 11 : cluster [DBG] 6.1a scrub starts 2024-02-23T00:40:28.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:27.096740+0000 osd.4 (osd.4) 12 : cluster [DBG] 6.1a scrub ok 2024-02-23T00:40:28.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:27.134101+0000 mgr.y (mgr.14517) 43 : cluster [DBG] pgmap v35: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:40:28.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:27.566007+0000 osd.7 (osd.7) 17 : cluster [DBG] 2.1d scrub starts 2024-02-23T00:40:28.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:27.568959+0000 osd.7 (osd.7) 18 : cluster [DBG] 2.1d scrub ok 2024-02-23T00:40:28.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:27.856785+0000 osd.1 (osd.1) 17 : cluster [DBG] 6.1 scrub starts 2024-02-23T00:40:28.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:27.859652+0000 osd.1 (osd.1) 18 : cluster [DBG] 6.1 scrub ok 2024-02-23T00:40:28.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:27.921886+0000 osd.6 (osd.6) 9 : cluster [DBG] 5.1f scrub starts 2024-02-23T00:40:28.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:27.922999+0000 osd.6 (osd.6) 10 : cluster [DBG] 5.1f scrub ok 2024-02-23T00:40:28.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:27.095677+0000 osd.4 (osd.4) 11 : cluster [DBG] 6.1a scrub starts 2024-02-23T00:40:28.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:27.096740+0000 osd.4 (osd.4) 12 : cluster [DBG] 6.1a scrub ok 2024-02-23T00:40:28.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:27.134101+0000 mgr.y (mgr.14517) 43 : cluster [DBG] pgmap v35: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:40:28.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:27.566007+0000 osd.7 (osd.7) 17 : cluster [DBG] 2.1d scrub starts 2024-02-23T00:40:28.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:27.568959+0000 osd.7 (osd.7) 18 : cluster [DBG] 2.1d scrub ok 2024-02-23T00:40:28.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:27.856785+0000 osd.1 (osd.1) 17 : cluster [DBG] 6.1 scrub starts 2024-02-23T00:40:28.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:27.859652+0000 osd.1 (osd.1) 18 : cluster [DBG] 6.1 scrub ok 2024-02-23T00:40:28.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:27.921886+0000 osd.6 (osd.6) 9 : cluster [DBG] 5.1f scrub starts 2024-02-23T00:40:28.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:27.922999+0000 osd.6 (osd.6) 10 : cluster [DBG] 5.1f scrub ok 2024-02-23T00:40:28.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:27.095677+0000 osd.4 (osd.4) 11 : cluster [DBG] 6.1a scrub starts 2024-02-23T00:40:28.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:27.096740+0000 osd.4 (osd.4) 12 : cluster [DBG] 6.1a scrub ok 2024-02-23T00:40:28.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:27.134101+0000 mgr.y (mgr.14517) 43 : cluster [DBG] pgmap v35: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:40:28.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:27.566007+0000 osd.7 (osd.7) 17 : cluster [DBG] 2.1d scrub starts 2024-02-23T00:40:28.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:27.568959+0000 osd.7 (osd.7) 18 : cluster [DBG] 2.1d scrub ok 2024-02-23T00:40:28.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:27.856785+0000 osd.1 (osd.1) 17 : cluster [DBG] 6.1 scrub starts 2024-02-23T00:40:28.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:27.859652+0000 osd.1 (osd.1) 18 : cluster [DBG] 6.1 scrub ok 2024-02-23T00:40:28.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:27.921886+0000 osd.6 (osd.6) 9 : cluster [DBG] 5.1f scrub starts 2024-02-23T00:40:28.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:27.922999+0000 osd.6 (osd.6) 10 : cluster [DBG] 5.1f scrub ok 2024-02-23T00:40:29.466 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:28.116644+0000 osd.3 (osd.3) 15 : cluster [DBG] 4.1d scrub starts 2024-02-23T00:40:29.467 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:28.118015+0000 osd.3 (osd.3) 16 : cluster [DBG] 4.1d scrub ok 2024-02-23T00:40:29.467 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:28.449511+0000 osd.2 (osd.2) 13 : cluster [DBG] 2.c deep-scrub starts 2024-02-23T00:40:29.467 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:28.450367+0000 osd.2 (osd.2) 14 : cluster [DBG] 2.c deep-scrub ok 2024-02-23T00:40:29.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:28.116644+0000 osd.3 (osd.3) 15 : cluster [DBG] 4.1d scrub starts 2024-02-23T00:40:29.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:28.118015+0000 osd.3 (osd.3) 16 : cluster [DBG] 4.1d scrub ok 2024-02-23T00:40:29.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:28.449511+0000 osd.2 (osd.2) 13 : cluster [DBG] 2.c deep-scrub starts 2024-02-23T00:40:29.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:28.450367+0000 osd.2 (osd.2) 14 : cluster [DBG] 2.c deep-scrub ok 2024-02-23T00:40:29.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:28.116644+0000 osd.3 (osd.3) 15 : cluster [DBG] 4.1d scrub starts 2024-02-23T00:40:29.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:28.118015+0000 osd.3 (osd.3) 16 : cluster [DBG] 4.1d scrub ok 2024-02-23T00:40:29.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:28.449511+0000 osd.2 (osd.2) 13 : cluster [DBG] 2.c deep-scrub starts 2024-02-23T00:40:29.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:28.450367+0000 osd.2 (osd.2) 14 : cluster [DBG] 2.c deep-scrub ok 2024-02-23T00:40:29.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.758+0000 7fcf18eab700 1 -- 172.21.15.47:0/587746881 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcf140f9670 msgr2=0x7fcf14101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:29.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.758+0000 7fcf18eab700 1 --2- 172.21.15.47:0/587746881 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcf140f9670 0x7fcf14101d80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fcf080047b0 tx=0x7fcf08009cb0).stop 2024-02-23T00:40:29.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.758+0000 7fcf18eab700 1 -- 172.21.15.47:0/587746881 shutdown_connections 2024-02-23T00:40:29.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.758+0000 7fcf18eab700 1 --2- 172.21.15.47:0/587746881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf14102270 0x7fcf14104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:29.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.758+0000 7fcf18eab700 1 --2- 172.21.15.47:0/587746881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf140f8d10 0x7fcf140f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:29.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.758+0000 7fcf18eab700 1 --2- 172.21.15.47:0/587746881 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcf140f9670 0x7fcf14101d80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:29.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.758+0000 7fcf18eab700 1 -- 172.21.15.47:0/587746881 >> 172.21.15.47:0/587746881 conn(0x7fcf140f2820 msgr2=0x7fcf140f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:40:29.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.758+0000 7fcf18eab700 1 -- 172.21.15.47:0/587746881 shutdown_connections 2024-02-23T00:40:29.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.758+0000 7fcf18eab700 1 -- 172.21.15.47:0/587746881 wait complete. 2024-02-23T00:40:29.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.759+0000 7fcf18eab700 1 librados: starting msgr at 2024-02-23T00:40:29.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.759+0000 7fcf18eab700 1 librados: starting objecter 2024-02-23T00:40:29.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.759+0000 7fcf18eab700 1 Processor -- start 2024-02-23T00:40:29.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.759+0000 7fcf18eab700 1 -- start start 2024-02-23T00:40:29.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.759+0000 7fcf18eab700 1 librados: setting wanted keys 2024-02-23T00:40:29.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.759+0000 7fcf18eab700 1 librados: calling monclient init 2024-02-23T00:40:29.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.759+0000 7fcf18eab700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf140f8d10 0x7fcf1410e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:29.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.759+0000 7fcf18eab700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf140f9670 0x7fcf1410e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:29.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.759+0000 7fcf18eab700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcf14102270 0x7fcf141081d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:29.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.759+0000 7fcf18eab700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf141086c0 con 0x7fcf140f8d10 2024-02-23T00:40:29.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.759+0000 7fcf18eab700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf14108830 con 0x7fcf140f9670 2024-02-23T00:40:29.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.759+0000 7fcf18eab700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf141089f0 con 0x7fcf14102270 2024-02-23T00:40:29.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.759+0000 7fcf1259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf140f8d10 0x7fcf1410e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:29.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.759+0000 7fcf12d9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcf14102270 0x7fcf141081d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:29.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.760+0000 7fcf12d9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcf14102270 0x7fcf141081d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:45054/0 (socket says 172.21.15.47:45054) 2024-02-23T00:40:29.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.760+0000 7fcf1259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf140f8d10 0x7fcf1410e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39746/0 (socket says 172.21.15.47:39746) 2024-02-23T00:40:29.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.760+0000 7fcf12d9d700 1 -- 172.21.15.47:0/3884059933 learned_addr learned my addr 172.21.15.47:0/3884059933 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:40:29.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.760+0000 7fcf11d9b700 1 --2- 172.21.15.47:0/3884059933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf140f9670 0x7fcf1410e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:29.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.760+0000 7fcf12d9d700 1 -- 172.21.15.47:0/3884059933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf140f9670 msgr2=0x7fcf1410e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:29.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.760+0000 7fcf12d9d700 1 --2- 172.21.15.47:0/3884059933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf140f9670 0x7fcf1410e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:29.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.760+0000 7fcf12d9d700 1 -- 172.21.15.47:0/3884059933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf140f8d10 msgr2=0x7fcf1410e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:29.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.760+0000 7fcf12d9d700 1 --2- 172.21.15.47:0/3884059933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf140f8d10 0x7fcf1410e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:29.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.760+0000 7fcf12d9d700 1 -- 172.21.15.47:0/3884059933 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf04005040 con 0x7fcf14102270 2024-02-23T00:40:29.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.760+0000 7fcf12d9d700 1 --2- 172.21.15.47:0/3884059933 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcf14102270 0x7fcf141081d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fcf0c00eb70 tx=0x7fcf0c00b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:29.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.760+0000 7fcf037fe700 10 client.?.objecter ms_handle_connect 0x7fcf14102270 2024-02-23T00:40:29.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.760+0000 7fcf037fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:40:29.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.760+0000 7fcf037fe700 1 -- 172.21.15.47:0/3884059933 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c009910 con 0x7fcf14102270 2024-02-23T00:40:29.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.760+0000 7fcf037fe700 1 -- 172.21.15.47:0/3884059933 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fcf0c00f040 con 0x7fcf14102270 2024-02-23T00:40:29.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.760+0000 7fcf037fe700 1 -- 172.21.15.47:0/3884059933 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c007d50 con 0x7fcf14102270 2024-02-23T00:40:29.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.760+0000 7fcf18eab700 1 -- 172.21.15.47:0/3884059933 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf08005040 con 0x7fcf14102270 2024-02-23T00:40:29.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.760+0000 7fcf18eab700 10 client.24506.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:40:29.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.760+0000 7fcf18eab700 1 -- 172.21.15.47:0/3884059933 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf14108f80 con 0x7fcf14102270 2024-02-23T00:40:29.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.760+0000 7fcf18eab700 1 librados: init done 2024-02-23T00:40:29.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.761+0000 7fcf18eab700 10 client.24506.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:40:29.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.761+0000 7fcf18eab700 15 client.24506.objecter _session_command_op_assign -1 1 2024-02-23T00:40:29.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.761+0000 7fcf18eab700 20 client.24506.objecter _get_session osd=-1 returning homeless 2024-02-23T00:40:29.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.761+0000 7fcf18eab700 10 client.24506.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:40:29.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.761+0000 7fcf18eab700 1 -- 172.21.15.47:0/3884059933 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcf1405fa60 con 0x7fcf14102270 2024-02-23T00:40:29.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.762+0000 7fcf037fe700 1 -- 172.21.15.47:0/3884059933 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fcf0c004020 con 0x7fcf14102270 2024-02-23T00:40:29.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.762+0000 7fcf037fe700 1 --2- 172.21.15.47:0/3884059933 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcefc062cc0 0x7fcefc065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:29.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.762+0000 7fcf037fe700 1 -- 172.21.15.47:0/3884059933 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7305+0+0 (secure 0 0 0) 0x7fcf0c07cb90 con 0x7fcf14102270 2024-02-23T00:40:29.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.762+0000 7fcf037fe700 10 client.24506.objecter ms_dispatch 0x7fcf14001550 osd_map(82..82 src has 1..82) v4 2024-02-23T00:40:29.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.762+0000 7fcf037fe700 3 client.24506.objecter handle_osd_map got epochs [82,82] > 0 2024-02-23T00:40:29.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.762+0000 7fcf037fe700 3 client.24506.objecter handle_osd_map decoding full epoch 82 2024-02-23T00:40:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.762+0000 7fcf1259c700 1 --2- 172.21.15.47:0/3884059933 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcefc062cc0 0x7fcefc065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.762+0000 7fcf037fe700 10 client.24506.objecter checking command 1 2024-02-23T00:40:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.762+0000 7fcf037fe700 1 --2- 172.21.15.47:0/3884059933 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fcefc06cab0 0x7fcefc06ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.762+0000 7fcf037fe700 20 client.24506.objecter _get_session s=0x7fcefc06c3e0 osd=0 3 2024-02-23T00:40:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.762+0000 7fcf037fe700 20 client.24506.objecter put_session s=0x7fcefc06c3e0 osd=0 3 2024-02-23T00:40:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.762+0000 7fcf037fe700 15 client.24506.objecter _session_command_op_remove -1 1 2024-02-23T00:40:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.762+0000 7fcf037fe700 20 client.24506.objecter _get_session s=0x7fcefc06c3e0 osd=0 3 2024-02-23T00:40:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.762+0000 7fcf037fe700 20 client.24506.objecter get_session s=0x7fcefc06c3e0 osd=0 3 2024-02-23T00:40:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.762+0000 7fcf037fe700 15 client.24506.objecter _session_command_op_assign 0 1 2024-02-23T00:40:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.762+0000 7fcf037fe700 20 client.24506.objecter put_session s=0x7fcefc06c3e0 osd=0 4 2024-02-23T00:40:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.762+0000 7fcf037fe700 10 client.24506.objecter _send_command 1 2024-02-23T00:40:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.762+0000 7fcf037fe700 1 -- 172.21.15.47:0/3884059933 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcefc06f5d0 con 0x7fcefc06cab0 2024-02-23T00:40:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.762+0000 7fcf037fe700 20 client.24506.objecter dump_active .. 0 homeless 2024-02-23T00:40:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.762+0000 7fcf037fe700 1 -- 172.21.15.47:0/3884059933 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf0c081220 con 0x7fcf14102270 2024-02-23T00:40:29.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.762+0000 7fcf11d9b700 1 --2- 172.21.15.47:0/3884059933 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fcefc06cab0 0x7fcefc06ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:29.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.763+0000 7fcf037fe700 10 client.24506.objecter ms_handle_connect 0x7fcefc062cc0 2024-02-23T00:40:29.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.763+0000 7fcf1259c700 1 --2- 172.21.15.47:0/3884059933 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcefc062cc0 0x7fcefc065180 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fcf04004870 tx=0x7fcf0400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:29.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.763+0000 7fcf037fe700 10 client.24506.objecter ms_handle_connect 0x7fcefc06cab0 2024-02-23T00:40:29.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.763+0000 7fcf11d9b700 1 --2- 172.21.15.47:0/3884059933 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fcefc06cab0 0x7fcefc06ef50 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:29.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.763+0000 7fcf037fe700 1 -- 172.21.15.47:0/3884059933 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcefc06f5d0 con 0x7fcefc06cab0 2024-02-23T00:40:29.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.763+0000 7fcf037fe700 10 client.24506.objecter ms_dispatch 0x7fcf14001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:40:29.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.763+0000 7fcf037fe700 10 client.24506.objecter _finish_command 1 = system:0 2024-02-23T00:40:29.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.763+0000 7fcf037fe700 20 client.24506.objecter put_session s=0x7fcefc06c3e0 osd=0 4 2024-02-23T00:40:29.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.763+0000 7fcf037fe700 15 client.24506.objecter _session_command_op_remove 0 1 2024-02-23T00:40:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.778+0000 7fcf18eab700 10 client.24506.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-02-23T00:40:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.778+0000 7fcf18eab700 15 client.24506.objecter _session_command_op_assign -1 2 2024-02-23T00:40:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.778+0000 7fcf18eab700 20 client.24506.objecter _get_session s=0x7fcefc06c3e0 osd=0 3 2024-02-23T00:40:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.778+0000 7fcf18eab700 20 client.24506.objecter put_session s=0x7fcefc06c3e0 osd=0 3 2024-02-23T00:40:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.778+0000 7fcf18eab700 20 client.24506.objecter _get_session s=0x7fcefc06c3e0 osd=0 3 2024-02-23T00:40:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.778+0000 7fcf18eab700 15 client.24506.objecter _session_command_op_remove -1 2 2024-02-23T00:40:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.778+0000 7fcf18eab700 20 client.24506.objecter get_session s=0x7fcefc06c3e0 osd=0 3 2024-02-23T00:40:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.778+0000 7fcf18eab700 15 client.24506.objecter _session_command_op_assign 0 2 2024-02-23T00:40:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.778+0000 7fcf18eab700 20 client.24506.objecter put_session s=0x7fcefc06c3e0 osd=0 4 2024-02-23T00:40:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.778+0000 7fcf18eab700 10 client.24506.objecter _send_command 2 2024-02-23T00:40:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.778+0000 7fcf18eab700 1 -- 172.21.15.47:0/3884059933 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcf1405ee80 con 0x7fcefc06cab0 2024-02-23T00:40:29.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.778+0000 7fcf037fe700 1 -- 172.21.15.47:0/3884059933 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcf1405ee80 con 0x7fcefc06cab0 2024-02-23T00:40:29.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.778+0000 7fcf037fe700 10 client.24506.objecter ms_dispatch 0x7fcf14001550 command_reply(tid 2: 0 ) v1 2024-02-23T00:40:29.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.778+0000 7fcf037fe700 10 client.24506.objecter _finish_command 2 = system:0 2024-02-23T00:40:29.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.778+0000 7fcf037fe700 20 client.24506.objecter put_session s=0x7fcefc06c3e0 osd=0 4 2024-02-23T00:40:29.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.778+0000 7fcf037fe700 15 client.24506.objecter _session_command_op_remove 0 2 2024-02-23T00:40:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.778+0000 7fcf18eab700 10 librados: watch_flush enter 2024-02-23T00:40:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.778+0000 7fcf18eab700 10 librados: watch_flush exit 2024-02-23T00:40:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.778+0000 7fcf18eab700 10 client.24506.objecter close_session for osd.0 2024-02-23T00:40:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.778+0000 7fcf18eab700 1 -- 172.21.15.47:0/3884059933 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fcefc06cab0 msgr2=0x7fcefc06ef50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.778+0000 7fcf18eab700 1 --2- 172.21.15.47:0/3884059933 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fcefc06cab0 0x7fcefc06ef50 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.778+0000 7fcf18eab700 20 client.24506.objecter put_session s=0x7fcefc06c3e0 osd=0 1 2024-02-23T00:40:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.778+0000 7fcf18eab700 20 client.24506.objecter shutdown clearing up homeless session... 2024-02-23T00:40:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.778+0000 7fcf18eab700 10 client.24506.objecter successfully canceled tick 2024-02-23T00:40:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.778+0000 7fcf18eab700 1 -- 172.21.15.47:0/3884059933 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcefc062cc0 msgr2=0x7fcefc065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.778+0000 7fcf18eab700 1 --2- 172.21.15.47:0/3884059933 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcefc062cc0 0x7fcefc065180 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fcf04004870 tx=0x7fcf0400b040).stop 2024-02-23T00:40:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.778+0000 7fcf18eab700 1 -- 172.21.15.47:0/3884059933 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcf14102270 msgr2=0x7fcf141081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.778+0000 7fcf18eab700 1 --2- 172.21.15.47:0/3884059933 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcf14102270 0x7fcf141081d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fcf0c00eb70 tx=0x7fcf0c00b040).stop 2024-02-23T00:40:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.778+0000 7fcf12d9d700 1 -- 172.21.15.47:0/3884059933 reap_dead start 2024-02-23T00:40:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.779+0000 7fcf18eab700 1 -- 172.21.15.47:0/3884059933 shutdown_connections 2024-02-23T00:40:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.779+0000 7fcf18eab700 1 -- 172.21.15.47:0/3884059933 >> 172.21.15.47:0/3884059933 conn(0x7fcf140f2820 msgr2=0x7fcf140f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:40:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.779+0000 7fcf18eab700 1 -- 172.21.15.47:0/3884059933 shutdown_connections 2024-02-23T00:40:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.779+0000 7fcf18eab700 1 -- 172.21.15.47:0/3884059933 wait complete. 2024-02-23T00:40:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:29.779+0000 7fcf18eab700 1 librados: shutdown 2024-02-23T00:40:30.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:29.134622+0000 mgr.y (mgr.14517) 44 : cluster [DBG] pgmap v36: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:40:30.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:29.593110+0000 osd.7 (osd.7) 19 : cluster [DBG] 6.1c scrub starts 2024-02-23T00:40:30.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:29.596573+0000 osd.7 (osd.7) 20 : cluster [DBG] 6.1c scrub ok 2024-02-23T00:40:30.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:29.976457+0000 osd.6 (osd.6) 11 : cluster [DBG] 2.17 scrub starts 2024-02-23T00:40:30.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:29.977267+0000 osd.6 (osd.6) 12 : cluster [DBG] 2.17 scrub ok 2024-02-23T00:40:30.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:29.989486+0000 osd.5 (osd.5) 11 : cluster [DBG] 2.12 deep-scrub starts 2024-02-23T00:40:30.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:29.990247+0000 osd.5 (osd.5) 12 : cluster [DBG] 2.12 deep-scrub ok 2024-02-23T00:40:30.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:29.134622+0000 mgr.y (mgr.14517) 44 : cluster [DBG] pgmap v36: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:40:30.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:29.593110+0000 osd.7 (osd.7) 19 : cluster [DBG] 6.1c scrub starts 2024-02-23T00:40:30.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:29.596573+0000 osd.7 (osd.7) 20 : cluster [DBG] 6.1c scrub ok 2024-02-23T00:40:30.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:29.976457+0000 osd.6 (osd.6) 11 : cluster [DBG] 2.17 scrub starts 2024-02-23T00:40:30.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:29.977267+0000 osd.6 (osd.6) 12 : cluster [DBG] 2.17 scrub ok 2024-02-23T00:40:30.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:29.989486+0000 osd.5 (osd.5) 11 : cluster [DBG] 2.12 deep-scrub starts 2024-02-23T00:40:30.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:29.990247+0000 osd.5 (osd.5) 12 : cluster [DBG] 2.12 deep-scrub ok 2024-02-23T00:40:30.605 INFO:teuthology.orchestra.run.smithi047.stdout:34359738425 2024-02-23T00:40:30.605 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-23T00:40:30.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:29.134622+0000 mgr.y (mgr.14517) 44 : cluster [DBG] pgmap v36: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:40:30.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:29.593110+0000 osd.7 (osd.7) 19 : cluster [DBG] 6.1c scrub starts 2024-02-23T00:40:30.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:29.596573+0000 osd.7 (osd.7) 20 : cluster [DBG] 6.1c scrub ok 2024-02-23T00:40:30.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:29.976457+0000 osd.6 (osd.6) 11 : cluster [DBG] 2.17 scrub starts 2024-02-23T00:40:30.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:29.977267+0000 osd.6 (osd.6) 12 : cluster [DBG] 2.17 scrub ok 2024-02-23T00:40:30.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:29.989486+0000 osd.5 (osd.5) 11 : cluster [DBG] 2.12 deep-scrub starts 2024-02-23T00:40:30.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:29.990247+0000 osd.5 (osd.5) 12 : cluster [DBG] 2.12 deep-scrub ok 2024-02-23T00:40:30.806 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:40:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:40:30] "GET /metrics HTTP/1.1" 200 172129 "" "Prometheus/2.33.4" 2024-02-23T00:40:30.892 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:40:31.636 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:30.957697+0000 osd.6 (osd.6) 13 : cluster [DBG] 2.1a scrub starts 2024-02-23T00:40:31.636 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:30.958627+0000 osd.6 (osd.6) 14 : cluster [DBG] 2.1a scrub ok 2024-02-23T00:40:31.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:30.957697+0000 osd.6 (osd.6) 13 : cluster [DBG] 2.1a scrub starts 2024-02-23T00:40:31.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:30.958627+0000 osd.6 (osd.6) 14 : cluster [DBG] 2.1a scrub ok 2024-02-23T00:40:31.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:30.957697+0000 osd.6 (osd.6) 13 : cluster [DBG] 2.1a scrub starts 2024-02-23T00:40:31.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:30.958627+0000 osd.6 (osd.6) 14 : cluster [DBG] 2.1a scrub ok 2024-02-23T00:40:32.419 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:40:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:40:32.420 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:30.997289+0000 osd.4 (osd.4) 13 : cluster [DBG] 5.e scrub starts 2024-02-23T00:40:32.420 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:30.998725+0000 osd.4 (osd.4) 14 : cluster [DBG] 5.e scrub ok 2024-02-23T00:40:32.420 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:31.135198+0000 mgr.y (mgr.14517) 45 : cluster [DBG] pgmap v37: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:40:32.680 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:40:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:40:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:40:32.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:30.997289+0000 osd.4 (osd.4) 13 : cluster [DBG] 5.e scrub starts 2024-02-23T00:40:32.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:30.998725+0000 osd.4 (osd.4) 14 : cluster [DBG] 5.e scrub ok 2024-02-23T00:40:32.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:31.135198+0000 mgr.y (mgr.14517) 45 : cluster [DBG] pgmap v37: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:40:32.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:30.997289+0000 osd.4 (osd.4) 13 : cluster [DBG] 5.e scrub starts 2024-02-23T00:40:32.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:30.998725+0000 osd.4 (osd.4) 14 : cluster [DBG] 5.e scrub ok 2024-02-23T00:40:32.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:31.135198+0000 mgr.y (mgr.14517) 45 : cluster [DBG] pgmap v37: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:40:33.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.449+0000 7f260ed29700 1 -- 172.21.15.47:0/3420648045 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f26080f8420 msgr2=0x7f2608104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:33.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.449+0000 7f260ed29700 1 --2- 172.21.15.47:0/3420648045 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f26080f8420 0x7f2608104a20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f2600009be0 tx=0x7f2600004d10).stop 2024-02-23T00:40:33.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.449+0000 7f260ed29700 1 -- 172.21.15.47:0/3420648045 shutdown_connections 2024-02-23T00:40:33.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.449+0000 7f260ed29700 1 --2- 172.21.15.47:0/3420648045 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f26080f8420 0x7f2608104a20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:33.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.449+0000 7f260ed29700 1 --2- 172.21.15.47:0/3420648045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26080fdbb0 0x7f26080fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:33.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.449+0000 7f260ed29700 1 --2- 172.21.15.47:0/3420648045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26080fe510 0x7f26080f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:33.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.449+0000 7f260ed29700 1 -- 172.21.15.47:0/3420648045 >> 172.21.15.47:0/3420648045 conn(0x7f26080f3800 msgr2=0x7f26080f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:40:33.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.449+0000 7f260ed29700 1 -- 172.21.15.47:0/3420648045 shutdown_connections 2024-02-23T00:40:33.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.449+0000 7f260ed29700 1 -- 172.21.15.47:0/3420648045 wait complete. 2024-02-23T00:40:33.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.449+0000 7f260ed29700 1 librados: starting msgr at 2024-02-23T00:40:33.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.449+0000 7f260ed29700 1 librados: starting objecter 2024-02-23T00:40:33.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.449+0000 7f260ed29700 1 Processor -- start 2024-02-23T00:40:33.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.449+0000 7f260ed29700 1 -- start start 2024-02-23T00:40:33.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.449+0000 7f260ed29700 1 librados: setting wanted keys 2024-02-23T00:40:33.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.449+0000 7f260ed29700 1 librados: calling monclient init 2024-02-23T00:40:33.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.450+0000 7f260ed29700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26080f8420 0x7f260806b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:33.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.450+0000 7f260ed29700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f26080fdbb0 0x7f260806b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:33.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.450+0000 7f260ed29700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26080fe510 0x7f260806bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:33.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.450+0000 7f260ed29700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26081162c0 con 0x7f26080fe510 2024-02-23T00:40:33.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.450+0000 7f260ed29700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2608116430 con 0x7f26080f8420 2024-02-23T00:40:33.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.450+0000 7f260ed29700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26081165f0 con 0x7f26080fdbb0 2024-02-23T00:40:33.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.450+0000 7f2607fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f26080fdbb0 0x7f260806b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:33.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.450+0000 7f2607fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f26080fdbb0 0x7f260806b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:45086/0 (socket says 172.21.15.47:45086) 2024-02-23T00:40:33.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.450+0000 7f2607fff700 1 -- 172.21.15.47:0/1223446576 learned_addr learned my addr 172.21.15.47:0/1223446576 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:40:33.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.450+0000 7f260d2c6700 1 --2- 172.21.15.47:0/1223446576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26080fe510 0x7f260806bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:33.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.450+0000 7f260cac5700 1 --2- 172.21.15.47:0/1223446576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26080f8420 0x7f260806b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:33.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.450+0000 7f2607fff700 1 -- 172.21.15.47:0/1223446576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26080f8420 msgr2=0x7f260806b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:33.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.450+0000 7f2607fff700 1 --2- 172.21.15.47:0/1223446576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26080f8420 0x7f260806b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:33.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.450+0000 7f2607fff700 1 -- 172.21.15.47:0/1223446576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26080fe510 msgr2=0x7f260806bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:33.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.450+0000 7f2607fff700 1 --2- 172.21.15.47:0/1223446576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26080fe510 0x7f260806bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:33.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.450+0000 7f2607fff700 1 -- 172.21.15.47:0/1223446576 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f4005040 con 0x7f26080fdbb0 2024-02-23T00:40:33.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.450+0000 7f260d2c6700 1 --2- 172.21.15.47:0/1223446576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26080fe510 0x7f260806bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:40:33.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.450+0000 7f2607fff700 1 --2- 172.21.15.47:0/1223446576 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f26080fdbb0 0x7f260806b5c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f25f40048f0 tx=0x7f25f4009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.450+0000 7f2605ffb700 10 client.?.objecter ms_handle_connect 0x7f26080fdbb0 2024-02-23T00:40:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.450+0000 7f2605ffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:40:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.450+0000 7f2605ffb700 1 -- 172.21.15.47:0/1223446576 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f4007b00 con 0x7f26080fdbb0 2024-02-23T00:40:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.450+0000 7f2605ffb700 1 -- 172.21.15.47:0/1223446576 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f25f400f040 con 0x7f26080fdbb0 2024-02-23T00:40:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.450+0000 7f2605ffb700 1 -- 172.21.15.47:0/1223446576 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f4014820 con 0x7f26080fdbb0 2024-02-23T00:40:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.451+0000 7f260ed29700 1 -- 172.21.15.47:0/1223446576 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26000097c0 con 0x7f26080fdbb0 2024-02-23T00:40:33.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.451+0000 7f260ed29700 10 client.24512.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:40:33.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.451+0000 7f260ed29700 1 -- 172.21.15.47:0/1223446576 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2608116cb0 con 0x7f26080fdbb0 2024-02-23T00:40:33.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.451+0000 7f260ed29700 1 librados: init done 2024-02-23T00:40:33.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.451+0000 7f260ed29700 10 client.24512.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:40:33.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.451+0000 7f260ed29700 15 client.24512.objecter _session_command_op_assign -1 1 2024-02-23T00:40:33.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.451+0000 7f260ed29700 20 client.24512.objecter _get_session osd=-1 returning homeless 2024-02-23T00:40:33.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.451+0000 7f260ed29700 10 client.24512.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:40:33.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.451+0000 7f260ed29700 1 -- 172.21.15.47:0/1223446576 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f260805ee80 con 0x7f26080fdbb0 2024-02-23T00:40:33.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.452+0000 7f2605ffb700 1 -- 172.21.15.47:0/1223446576 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f25f4014c50 con 0x7f26080fdbb0 2024-02-23T00:40:33.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.452+0000 7f2605ffb700 1 --2- 172.21.15.47:0/1223446576 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f25f8062cc0 0x7f25f8065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:33.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.452+0000 7f2605ffb700 1 -- 172.21.15.47:0/1223446576 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7305+0+0 (secure 0 0 0) 0x7f25f407c920 con 0x7f26080fdbb0 2024-02-23T00:40:33.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.452+0000 7f2605ffb700 10 client.24512.objecter ms_dispatch 0x7f2608001550 osd_map(82..82 src has 1..82) v4 2024-02-23T00:40:33.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.452+0000 7f2605ffb700 3 client.24512.objecter handle_osd_map got epochs [82,82] > 0 2024-02-23T00:40:33.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.452+0000 7f2605ffb700 3 client.24512.objecter handle_osd_map decoding full epoch 82 2024-02-23T00:40:33.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.452+0000 7f260cac5700 1 --2- 172.21.15.47:0/1223446576 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f25f8062cc0 0x7f25f8065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:33.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.452+0000 7f2605ffb700 10 client.24512.objecter checking command 1 2024-02-23T00:40:33.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.453+0000 7f2605ffb700 1 --2- 172.21.15.47:0/1223446576 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f25f806cab0 0x7f25f806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:33.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.453+0000 7f2605ffb700 20 client.24512.objecter _get_session s=0x7f25f806c3e0 osd=1 3 2024-02-23T00:40:33.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.453+0000 7f2605ffb700 20 client.24512.objecter put_session s=0x7f25f806c3e0 osd=1 3 2024-02-23T00:40:33.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.453+0000 7f2605ffb700 15 client.24512.objecter _session_command_op_remove -1 1 2024-02-23T00:40:33.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.453+0000 7f2605ffb700 20 client.24512.objecter _get_session s=0x7f25f806c3e0 osd=1 3 2024-02-23T00:40:33.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.453+0000 7f2605ffb700 20 client.24512.objecter get_session s=0x7f25f806c3e0 osd=1 3 2024-02-23T00:40:33.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.453+0000 7f2605ffb700 15 client.24512.objecter _session_command_op_assign 1 1 2024-02-23T00:40:33.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.453+0000 7f2605ffb700 20 client.24512.objecter put_session s=0x7f25f806c3e0 osd=1 4 2024-02-23T00:40:33.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.453+0000 7f2605ffb700 10 client.24512.objecter _send_command 1 2024-02-23T00:40:33.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.453+0000 7f2605ffb700 1 -- 172.21.15.47:0/1223446576 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f25f806f5d0 con 0x7f25f806cab0 2024-02-23T00:40:33.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.453+0000 7f2605ffb700 20 client.24512.objecter dump_active .. 0 homeless 2024-02-23T00:40:33.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.453+0000 7f2605ffb700 1 -- 172.21.15.47:0/1223446576 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25f4081220 con 0x7f26080fdbb0 2024-02-23T00:40:33.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.453+0000 7f260d2c6700 1 --2- 172.21.15.47:0/1223446576 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f25f806cab0 0x7f25f806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:33.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.453+0000 7f2605ffb700 10 client.24512.objecter ms_handle_connect 0x7f25f8062cc0 2024-02-23T00:40:33.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.453+0000 7f260cac5700 1 --2- 172.21.15.47:0/1223446576 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f25f8062cc0 0x7f25f8065180 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f25fc004ee0 tx=0x7f25fc00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:33.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.453+0000 7f2605ffb700 10 client.24512.objecter ms_handle_connect 0x7f25f806cab0 2024-02-23T00:40:33.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.453+0000 7f260d2c6700 1 --2- 172.21.15.47:0/1223446576 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f25f806cab0 0x7f25f806ef50 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:33.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.454+0000 7f2605ffb700 1 -- 172.21.15.47:0/1223446576 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f25f806f5d0 con 0x7f25f806cab0 2024-02-23T00:40:33.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.454+0000 7f2605ffb700 10 client.24512.objecter ms_dispatch 0x7f2608001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:40:33.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.454+0000 7f2605ffb700 10 client.24512.objecter _finish_command 1 = system:0 2024-02-23T00:40:33.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.454+0000 7f2605ffb700 20 client.24512.objecter put_session s=0x7f25f806c3e0 osd=1 4 2024-02-23T00:40:33.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.454+0000 7f2605ffb700 15 client.24512.objecter _session_command_op_remove 1 1 2024-02-23T00:40:33.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.468+0000 7f260ed29700 10 client.24512.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-02-23T00:40:33.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.468+0000 7f260ed29700 15 client.24512.objecter _session_command_op_assign -1 2 2024-02-23T00:40:33.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.468+0000 7f260ed29700 20 client.24512.objecter _get_session s=0x7f25f806c3e0 osd=1 3 2024-02-23T00:40:33.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.468+0000 7f260ed29700 20 client.24512.objecter put_session s=0x7f25f806c3e0 osd=1 3 2024-02-23T00:40:33.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.468+0000 7f260ed29700 20 client.24512.objecter _get_session s=0x7f25f806c3e0 osd=1 3 2024-02-23T00:40:33.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.468+0000 7f260ed29700 15 client.24512.objecter _session_command_op_remove -1 2 2024-02-23T00:40:33.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.468+0000 7f260ed29700 20 client.24512.objecter get_session s=0x7f25f806c3e0 osd=1 3 2024-02-23T00:40:33.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.468+0000 7f260ed29700 15 client.24512.objecter _session_command_op_assign 1 2 2024-02-23T00:40:33.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.468+0000 7f260ed29700 20 client.24512.objecter put_session s=0x7f25f806c3e0 osd=1 4 2024-02-23T00:40:33.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.468+0000 7f260ed29700 10 client.24512.objecter _send_command 2 2024-02-23T00:40:33.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.468+0000 7f260ed29700 1 -- 172.21.15.47:0/1223446576 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2608106ed0 con 0x7f25f806cab0 2024-02-23T00:40:33.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.468+0000 7f2605ffb700 1 -- 172.21.15.47:0/1223446576 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2608106ed0 con 0x7f25f806cab0 2024-02-23T00:40:33.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.468+0000 7f2605ffb700 10 client.24512.objecter ms_dispatch 0x7f2608001550 command_reply(tid 2: 0 ) v1 2024-02-23T00:40:33.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.468+0000 7f2605ffb700 10 client.24512.objecter _finish_command 2 = system:0 2024-02-23T00:40:33.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.468+0000 7f2605ffb700 20 client.24512.objecter put_session s=0x7f25f806c3e0 osd=1 4 2024-02-23T00:40:33.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.468+0000 7f2605ffb700 15 client.24512.objecter _session_command_op_remove 1 2 2024-02-23T00:40:33.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.469+0000 7f260ed29700 10 librados: watch_flush enter 2024-02-23T00:40:33.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.469+0000 7f260ed29700 10 librados: watch_flush exit 2024-02-23T00:40:33.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.469+0000 7f260ed29700 10 client.24512.objecter close_session for osd.1 2024-02-23T00:40:33.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.469+0000 7f260ed29700 1 -- 172.21.15.47:0/1223446576 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f25f806cab0 msgr2=0x7f25f806ef50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:33.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.469+0000 7f260ed29700 1 --2- 172.21.15.47:0/1223446576 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f25f806cab0 0x7f25f806ef50 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:33.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.469+0000 7f260ed29700 20 client.24512.objecter put_session s=0x7f25f806c3e0 osd=1 1 2024-02-23T00:40:33.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.469+0000 7f260ed29700 20 client.24512.objecter shutdown clearing up homeless session... 2024-02-23T00:40:33.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.469+0000 7f260ed29700 10 client.24512.objecter successfully canceled tick 2024-02-23T00:40:33.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.469+0000 7f260ed29700 1 -- 172.21.15.47:0/1223446576 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f25f8062cc0 msgr2=0x7f25f8065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:33.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.469+0000 7f260ed29700 1 --2- 172.21.15.47:0/1223446576 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f25f8062cc0 0x7f25f8065180 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f25fc004ee0 tx=0x7f25fc00b040).stop 2024-02-23T00:40:33.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.469+0000 7f260ed29700 1 -- 172.21.15.47:0/1223446576 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f26080fdbb0 msgr2=0x7f260806b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:33.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.469+0000 7f260ed29700 1 --2- 172.21.15.47:0/1223446576 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f26080fdbb0 0x7f260806b5c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f25f40048f0 tx=0x7f25f4009cb0).stop 2024-02-23T00:40:33.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.469+0000 7f260d2c6700 1 -- 172.21.15.47:0/1223446576 reap_dead start 2024-02-23T00:40:33.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.469+0000 7f260ed29700 1 -- 172.21.15.47:0/1223446576 shutdown_connections 2024-02-23T00:40:33.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.469+0000 7f260ed29700 1 -- 172.21.15.47:0/1223446576 >> 172.21.15.47:0/1223446576 conn(0x7f26080f3800 msgr2=0x7f26080f51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:40:33.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.469+0000 7f260ed29700 1 -- 172.21.15.47:0/1223446576 shutdown_connections 2024-02-23T00:40:33.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.469+0000 7f260ed29700 1 -- 172.21.15.47:0/1223446576 wait complete. 2024-02-23T00:40:33.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:33.469+0000 7f260ed29700 1 librados: shutdown 2024-02-23T00:40:33.495 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:31.985489+0000 osd.4 (osd.4) 15 : cluster [DBG] 4.1b deep-scrub starts 2024-02-23T00:40:33.495 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:31.989021+0000 osd.4 (osd.4) 16 : cluster [DBG] 4.1b deep-scrub ok 2024-02-23T00:40:33.495 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:32.153508+0000 mgr.y (mgr.14517) 46 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:40:33.495 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:31.985489+0000 osd.4 (osd.4) 15 : cluster [DBG] 4.1b deep-scrub starts 2024-02-23T00:40:33.495 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:31.989021+0000 osd.4 (osd.4) 16 : cluster [DBG] 4.1b deep-scrub ok 2024-02-23T00:40:33.495 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:32.153508+0000 mgr.y (mgr.14517) 46 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:40:33.599 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:31.985489+0000 osd.4 (osd.4) 15 : cluster [DBG] 4.1b deep-scrub starts 2024-02-23T00:40:33.599 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:31.989021+0000 osd.4 (osd.4) 16 : cluster [DBG] 4.1b deep-scrub ok 2024-02-23T00:40:33.599 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:32.153508+0000 mgr.y (mgr.14517) 46 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:40:33.856 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:33 smithi196 systemd[1]: Starting Ceph grafana.a for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T00:40:34.228 INFO:teuthology.orchestra.run.smithi047.stdout:55834574900 2024-02-23T00:40:34.228 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-23T00:40:34.512 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:40:34.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:33.135857+0000 mgr.y (mgr.14517) 47 : cluster [DBG] pgmap v38: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-23T00:40:34.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:33.491001+0000 osd.2 (osd.2) 15 : cluster [DBG] 5.b scrub starts 2024-02-23T00:40:34.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:33.492220+0000 osd.2 (osd.2) 16 : cluster [DBG] 5.b scrub ok 2024-02-23T00:40:34.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:33.622007+0000 osd.7 (osd.7) 21 : cluster [DBG] 6.3 scrub starts 2024-02-23T00:40:34.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:33.626490+0000 osd.7 (osd.7) 22 : cluster [DBG] 6.3 scrub ok 2024-02-23T00:40:34.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:33.135857+0000 mgr.y (mgr.14517) 47 : cluster [DBG] pgmap v38: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-23T00:40:34.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:33.491001+0000 osd.2 (osd.2) 15 : cluster [DBG] 5.b scrub starts 2024-02-23T00:40:34.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:33.492220+0000 osd.2 (osd.2) 16 : cluster [DBG] 5.b scrub ok 2024-02-23T00:40:34.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:33.622007+0000 osd.7 (osd.7) 21 : cluster [DBG] 6.3 scrub starts 2024-02-23T00:40:34.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:33.626490+0000 osd.7 (osd.7) 22 : cluster [DBG] 6.3 scrub ok 2024-02-23T00:40:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:33.135857+0000 mgr.y (mgr.14517) 47 : cluster [DBG] pgmap v38: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-23T00:40:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:33.491001+0000 osd.2 (osd.2) 15 : cluster [DBG] 5.b scrub starts 2024-02-23T00:40:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:33.492220+0000 osd.2 (osd.2) 16 : cluster [DBG] 5.b scrub ok 2024-02-23T00:40:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:33.622007+0000 osd.7 (osd.7) 21 : cluster [DBG] 6.3 scrub starts 2024-02-23T00:40:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:33.626490+0000 osd.7 (osd.7) 22 : cluster [DBG] 6.3 scrub ok 2024-02-23T00:40:34.641 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="The state of unified alerting is still not defined. The decision will be made during as we run the database migrations" logger=settings 2024-02-23T00:40:34.641 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=warn msg="falling back to legacy setting of 'min_interval_seconds'; please use the configuration option in the `unified_alerting` section if Grafana 8 alerts are enabled." logger=settings 2024-02-23T00:40:34.641 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Config loaded from" logger=settings file=/usr/share/grafana/conf/defaults.ini 2024-02-23T00:40:34.641 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Config loaded from" logger=settings file=/etc/grafana/grafana.ini 2024-02-23T00:40:34.641 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_DATA=/var/lib/grafana" 2024-02-23T00:40:34.641 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_LOGS=/var/log/grafana" 2024-02-23T00:40:34.641 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-02-23T00:40:34.641 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-02-23T00:40:34.641 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Path Home" logger=settings path=/usr/share/grafana 2024-02-23T00:40:34.641 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Path Data" logger=settings path=/var/lib/grafana 2024-02-23T00:40:34.641 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Path Logs" logger=settings path=/var/log/grafana 2024-02-23T00:40:34.641 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Path Plugins" logger=settings path=/var/lib/grafana/plugins 2024-02-23T00:40:34.642 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Path Provisioning" logger=settings path=/etc/grafana/provisioning 2024-02-23T00:40:34.642 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="App mode production" logger=settings 2024-02-23T00:40:34.642 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Connecting to DB" logger=sqlstore dbtype=sqlite3 2024-02-23T00:40:34.642 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=warn msg="SQLite database file has broader permissions than it should" logger=sqlstore path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-02-23T00:40:34.642 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Starting DB migrations" logger=migrator 2024-02-23T00:40:34.642 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create migration_log table" 2024-02-23T00:40:34.642 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create user table" 2024-02-23T00:40:34.642 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user.login" 2024-02-23T00:40:34.642 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user.email" 2024-02-23T00:40:34.642 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_user_login - v1" 2024-02-23T00:40:34.642 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_user_email - v1" 2024-02-23T00:40:34.642 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table user to user_v1 - v1" 2024-02-23T00:40:34.642 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create user table v2" 2024-02-23T00:40:34.642 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_user_login - v2" 2024-02-23T00:40:34.643 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_user_email - v2" 2024-02-23T00:40:34.643 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="copy data_source v1 to v2" 2024-02-23T00:40:34.643 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table user_v1" 2024-02-23T00:40:34.643 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column help_flags1 to user table" 2024-02-23T00:40:34.643 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Update user table charset" 2024-02-23T00:40:34.643 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add last_seen_at column to user" 2024-02-23T00:40:34.643 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add missing user data" 2024-02-23T00:40:34.643 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add is_disabled column to user" 2024-02-23T00:40:34.643 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add index user.login/user.email" 2024-02-23T00:40:34.643 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add is_service_account column to user" 2024-02-23T00:40:34.643 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create temp user table v1-7" 2024-02-23T00:40:34.643 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_email - v1-7" 2024-02-23T00:40:34.643 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_org_id - v1-7" 2024-02-23T00:40:34.643 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_code - v1-7" 2024-02-23T00:40:34.644 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_status - v1-7" 2024-02-23T00:40:34.644 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Update temp_user table charset" 2024-02-23T00:40:34.644 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_temp_user_email - v1" 2024-02-23T00:40:34.644 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_temp_user_org_id - v1" 2024-02-23T00:40:34.644 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_temp_user_code - v1" 2024-02-23T00:40:34.644 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_temp_user_status - v1" 2024-02-23T00:40:34.644 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-02-23T00:40:34.644 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create temp_user v2" 2024-02-23T00:40:34.644 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_email - v2" 2024-02-23T00:40:34.644 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_org_id - v2" 2024-02-23T00:40:34.644 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_code - v2" 2024-02-23T00:40:34.644 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_status - v2" 2024-02-23T00:40:34.645 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="copy temp_user v1 to v2" 2024-02-23T00:40:34.645 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop temp_user_tmp_qwerty" 2024-02-23T00:40:34.645 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Set created for temp users that will otherwise prematurely expire" 2024-02-23T00:40:34.645 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create star table" 2024-02-23T00:40:34.645 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index star.user_id_dashboard_id" 2024-02-23T00:40:34.645 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create org table v1" 2024-02-23T00:40:34.645 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_org_name - v1" 2024-02-23T00:40:34.645 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create org_user table v1" 2024-02-23T00:40:34.645 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_org_user_org_id - v1" 2024-02-23T00:40:34.645 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_org_user_org_id_user_id - v1" 2024-02-23T00:40:34.645 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_org_user_user_id - v1" 2024-02-23T00:40:34.645 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Update org table charset" 2024-02-23T00:40:34.645 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Update org_user table charset" 2024-02-23T00:40:34.645 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Migrate all Read Only Viewers to Viewers" 2024-02-23T00:40:34.646 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard table" 2024-02-23T00:40:34.646 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard.account_id" 2024-02-23T00:40:34.646 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_account_id_slug" 2024-02-23T00:40:34.646 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_tag table" 2024-02-23T00:40:34.646 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_tag.dasboard_id_term" 2024-02-23T00:40:34.646 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-02-23T00:40:34.646 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table dashboard to dashboard_v1 - v1" 2024-02-23T00:40:34.646 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard v2" 2024-02-23T00:40:34.646 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_org_id - v2" 2024-02-23T00:40:34.646 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_dashboard_org_id_slug - v2" 2024-02-23T00:40:34.646 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="copy dashboard v1 to v2" 2024-02-23T00:40:34.646 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 bash[63503]: d7172b2fbd09b6aa102abbdce70d9274c3592602c6ffba6d1d5513c79c4632e2 2024-02-23T00:40:34.646 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop table dashboard_v1" 2024-02-23T00:40:34.646 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="alter dashboard.data to mediumtext v1" 2024-02-23T00:40:34.647 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column updated_by in dashboard - v2" 2024-02-23T00:40:34.647 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column created_by in dashboard - v2" 2024-02-23T00:40:34.647 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 systemd[1]: Started Ceph grafana.a for fce2617e-d1e2-11ee-95c0-87774f69a715. 2024-02-23T00:40:34.647 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column gnetId in dashboard" 2024-02-23T00:40:34.647 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for gnetId in dashboard" 2024-02-23T00:40:34.647 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column plugin_id in dashboard" 2024-02-23T00:40:34.647 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for plugin_id in dashboard" 2024-02-23T00:40:34.647 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for dashboard_id in dashboard_tag" 2024-02-23T00:40:34.647 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard table charset" 2024-02-23T00:40:34.648 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard_tag table charset" 2024-02-23T00:40:34.648 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column folder_id in dashboard" 2024-02-23T00:40:34.648 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column isFolder in dashboard" 2024-02-23T00:40:34.648 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column has_acl in dashboard" 2024-02-23T00:40:34.648 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column uid in dashboard" 2024-02-23T00:40:34.648 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Update uid column values in dashboard" 2024-02-23T00:40:34.648 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index dashboard_org_id_uid" 2024-02-23T00:40:34.648 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Remove unique index org_id_slug" 2024-02-23T00:40:34.648 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard title length" 2024-02-23T00:40:34.648 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index for dashboard_org_id_title_folder_id" 2024-02-23T00:40:34.648 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_provisioning" 2024-02-23T00:40:34.649 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-02-23T00:40:34.649 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_provisioning v2" 2024-02-23T00:40:34.649 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-02-23T00:40:34.649 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-02-23T00:40:34.649 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="copy dashboard_provisioning v1 to v2" 2024-02-23T00:40:34.649 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop dashboard_provisioning_tmp_qwerty" 2024-02-23T00:40:34.649 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add check_sum column" 2024-02-23T00:40:34.649 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for dashboard_title" 2024-02-23T00:40:34.649 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="delete tags for deleted dashboards" 2024-02-23T00:40:34.649 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="delete stars for deleted dashboards" 2024-02-23T00:40:34.649 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for dashboard_is_folder" 2024-02-23T00:40:34.649 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create data_source table" 2024-02-23T00:40:34.649 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index data_source.account_id" 2024-02-23T00:40:34.649 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index data_source.account_id_name" 2024-02-23T00:40:34.650 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_data_source_account_id - v1" 2024-02-23T00:40:34.650 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_data_source_account_id_name - v1" 2024-02-23T00:40:34.650 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table data_source to data_source_v1 - v1" 2024-02-23T00:40:34.650 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create data_source table v2" 2024-02-23T00:40:34.650 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_data_source_org_id - v2" 2024-02-23T00:40:34.650 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_data_source_org_id_name - v2" 2024-02-23T00:40:34.650 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="copy data_source v1 to v2" 2024-02-23T00:40:34.650 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table data_source_v1 #2" 2024-02-23T00:40:34.650 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column with_credentials" 2024-02-23T00:40:34.650 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add secure json data column" 2024-02-23T00:40:34.650 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Update data_source table charset" 2024-02-23T00:40:34.650 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Update initial version to 1" 2024-02-23T00:40:34.650 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add read_only data column" 2024-02-23T00:40:34.651 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Migrate logging ds to loki ds" 2024-02-23T00:40:34.651 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Update json_data with nulls" 2024-02-23T00:40:34.651 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add uid column" 2024-02-23T00:40:34.651 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Update uid value" 2024-02-23T00:40:34.651 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index datasource_org_id_uid" 2024-02-23T00:40:34.651 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index datasource_org_id_is_default" 2024-02-23T00:40:34.651 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create api_key table" 2024-02-23T00:40:34.651 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index api_key.account_id" 2024-02-23T00:40:34.890 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index api_key.key" 2024-02-23T00:40:34.890 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index api_key.account_id_name" 2024-02-23T00:40:34.890 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_api_key_account_id - v1" 2024-02-23T00:40:34.890 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_api_key_key - v1" 2024-02-23T00:40:34.890 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_api_key_account_id_name - v1" 2024-02-23T00:40:34.890 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table api_key to api_key_v1 - v1" 2024-02-23T00:40:34.891 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create api_key table v2" 2024-02-23T00:40:34.891 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_api_key_org_id - v2" 2024-02-23T00:40:34.891 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_api_key_key - v2" 2024-02-23T00:40:34.891 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_api_key_org_id_name - v2" 2024-02-23T00:40:34.891 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="copy api_key v1 to v2" 2024-02-23T00:40:34.891 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table api_key_v1" 2024-02-23T00:40:34.891 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Update api_key table charset" 2024-02-23T00:40:34.891 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add expires to api_key table" 2024-02-23T00:40:34.891 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add service account foreign key" 2024-02-23T00:40:34.891 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_snapshot table v4" 2024-02-23T00:40:34.891 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop table dashboard_snapshot_v4 #1" 2024-02-23T00:40:34.891 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_snapshot table v5 #2" 2024-02-23T00:40:34.891 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_dashboard_snapshot_key - v5" 2024-02-23T00:40:34.892 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-02-23T00:40:34.892 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_snapshot_user_id - v5" 2024-02-23T00:40:34.893 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="alter dashboard_snapshot to mediumtext v2" 2024-02-23T00:40:34.893 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard_snapshot table charset" 2024-02-23T00:40:34.893 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column external_delete_url to dashboard_snapshots table" 2024-02-23T00:40:34.893 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add encrypted dashboard json column" 2024-02-23T00:40:34.893 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-02-23T00:40:34.893 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create quota table v1" 2024-02-23T00:40:34.893 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_quota_org_id_user_id_target - v1" 2024-02-23T00:40:34.893 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Update quota table charset" 2024-02-23T00:40:34.893 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create plugin_setting table" 2024-02-23T00:40:34.893 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-02-23T00:40:34.893 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column plugin_version to plugin_settings" 2024-02-23T00:40:34.893 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Update plugin_setting table charset" 2024-02-23T00:40:34.893 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create session table" 2024-02-23T00:40:34.893 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table playlist table" 2024-02-23T00:40:34.894 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table playlist_item table" 2024-02-23T00:40:34.894 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create playlist table v2" 2024-02-23T00:40:34.894 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create playlist item table v2" 2024-02-23T00:40:34.894 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Update playlist table charset" 2024-02-23T00:40:34.894 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Update playlist_item table charset" 2024-02-23T00:40:34.894 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop preferences table v2" 2024-02-23T00:40:34.894 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop preferences table v3" 2024-02-23T00:40:34.894 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create preferences table v3" 2024-02-23T00:40:34.894 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Update preferences table charset" 2024-02-23T00:40:34.894 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column team_id in preferences" 2024-02-23T00:40:34.894 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Update team_id column values in preferences" 2024-02-23T00:40:34.894 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column week_start in preferences" 2024-02-23T00:40:34.894 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create alert table v1" 2024-02-23T00:40:34.895 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert org_id & id " 2024-02-23T00:40:34.895 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert state" 2024-02-23T00:40:34.895 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert dashboard_id" 2024-02-23T00:40:34.895 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Create alert_rule_tag table v1" 2024-02-23T00:40:34.895 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-02-23T00:40:34.895 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-02-23T00:40:34.895 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-02-23T00:40:34.895 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Create alert_rule_tag table v2" 2024-02-23T00:40:34.895 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-02-23T00:40:34.895 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="copy alert_rule_tag v1 to v2" 2024-02-23T00:40:34.895 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop table alert_rule_tag_v1" 2024-02-23T00:40:34.895 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_notification table v1" 2024-02-23T00:40:34.895 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column is_default" 2024-02-23T00:40:34.895 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column frequency" 2024-02-23T00:40:34.896 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column send_reminder" 2024-02-23T00:40:34.896 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column disable_resolve_message" 2024-02-23T00:40:34.896 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert_notification org_id & name" 2024-02-23T00:40:34.896 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert table charset" 2024-02-23T00:40:34.896 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert_notification table charset" 2024-02-23T00:40:34.896 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create notification_journal table v1" 2024-02-23T00:40:34.896 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index notification_journal org_id & alert_id & notifier_id" 2024-02-23T00:40:34.896 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop alert_notification_journal" 2024-02-23T00:40:34.896 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_notification_state table v1" 2024-02-23T00:40:34.896 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-02-23T00:40:34.896 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add for to alert table" 2024-02-23T00:40:34.896 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column uid in alert_notification" 2024-02-23T00:40:34.896 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Update uid column values in alert_notification" 2024-02-23T00:40:34.896 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index alert_notification_org_id_uid" 2024-02-23T00:40:34.897 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Remove unique index org_id_name" 2024-02-23T00:40:34.897 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column secure_settings in alert_notification" 2024-02-23T00:40:34.897 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert.settings to mediumtext" 2024-02-23T00:40:34.897 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add non-unique index alert_notification_state_alert_id" 2024-02-23T00:40:34.897 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add non-unique index alert_rule_tag_alert_id" 2024-02-23T00:40:34.897 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old annotation table v4" 2024-02-23T00:40:34.897 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create annotation table v5" 2024-02-23T00:40:34.897 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 0 v3" 2024-02-23T00:40:34.897 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 1 v3" 2024-02-23T00:40:34.897 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 2 v3" 2024-02-23T00:40:34.897 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 3 v3" 2024-02-23T00:40:34.897 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 4 v3" 2024-02-23T00:40:34.897 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Update annotation table charset" 2024-02-23T00:40:34.897 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column region_id to annotation table" 2024-02-23T00:40:34.898 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Drop category_id index" 2024-02-23T00:40:34.898 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column tags to annotation table" 2024-02-23T00:40:34.898 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Create annotation_tag table v2" 2024-02-23T00:40:34.898 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index annotation_tag.annotation_id_tag_id" 2024-02-23T00:40:34.898 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-02-23T00:40:34.898 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-02-23T00:40:34.898 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Create annotation_tag table v3" 2024-02-23T00:40:34.898 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-02-23T00:40:34.898 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="copy annotation_tag v2 to v3" 2024-02-23T00:40:34.898 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop table annotation_tag_v2" 2024-02-23T00:40:34.898 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert annotations and set TEXT to empty" 2024-02-23T00:40:34.898 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add created time to annotation table" 2024-02-23T00:40:34.898 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add updated time to annotation table" 2024-02-23T00:40:34.898 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for created in annotation table" 2024-02-23T00:40:34.899 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for updated in annotation table" 2024-02-23T00:40:34.899 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Convert existing annotations from seconds to milliseconds" 2024-02-23T00:40:34.899 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add epoch_end column" 2024-02-23T00:40:34.899 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for epoch_end" 2024-02-23T00:40:34.899 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Make epoch_end the same as epoch" 2024-02-23T00:40:34.899 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Move region to single row" 2024-02-23T00:40:34.899 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_epoch from annotation table" 2024-02-23T00:40:34.899 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-02-23T00:40:34.899 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-02-23T00:40:34.899 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for org_id_epoch_end_epoch on annotation table" 2024-02-23T00:40:34.899 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_epoch_epoch_end from annotation table" 2024-02-23T00:40:34.899 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for alert_id on annotation table" 2024-02-23T00:40:34.899 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create test_data table" 2024-02-23T00:40:34.899 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_version table v1" 2024-02-23T00:40:34.900 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_version.dashboard_id" 2024-02-23T00:40:34.900 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-02-23T00:40:34.900 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Set dashboard version to 1 where 0" 2024-02-23T00:40:34.900 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="save existing dashboard data in dashboard_version table v1" 2024-02-23T00:40:34.900 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="alter dashboard_version.data to mediumtext v1" 2024-02-23T00:40:34.900 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create team table" 2024-02-23T00:40:34.900 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index team.org_id" 2024-02-23T00:40:34.900 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index team_org_id_name" 2024-02-23T00:40:34.900 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create team member table" 2024-02-23T00:40:34.900 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_member.org_id" 2024-02-23T00:40:34.900 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index team_member_org_id_team_id_user_id" 2024-02-23T00:40:34.900 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_member.team_id" 2024-02-23T00:40:34.900 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column email to team table" 2024-02-23T00:40:34.900 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column external to team_member table" 2024-02-23T00:40:34.901 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column permission to team_member table" 2024-02-23T00:40:34.901 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard acl table" 2024-02-23T00:40:34.901 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_dashboard_id" 2024-02-23T00:40:34.901 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_acl_dashboard_id_user_id" 2024-02-23T00:40:34.901 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_acl_dashboard_id_team_id" 2024-02-23T00:40:34.901 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_user_id" 2024-02-23T00:40:34.901 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_team_id" 2024-02-23T00:40:34.901 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_org_id_role" 2024-02-23T00:40:34.901 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_permission" 2024-02-23T00:40:34.901 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="save default acl rules in dashboard_acl table" 2024-02-23T00:40:34.901 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="delete acl rules for deleted dashboards and folders" 2024-02-23T00:40:34.901 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create tag table" 2024-02-23T00:40:34.901 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index tag.key_value" 2024-02-23T00:40:34.901 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create login attempt table" 2024-02-23T00:40:34.902 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index login_attempt.username" 2024-02-23T00:40:34.902 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_login_attempt_username - v1" 2024-02-23T00:40:34.902 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-02-23T00:40:34.902 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create login_attempt v2" 2024-02-23T00:40:34.902 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_login_attempt_username - v2" 2024-02-23T00:40:34.902 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="copy login_attempt v1 to v2" 2024-02-23T00:40:34.902 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop login_attempt_tmp_qwerty" 2024-02-23T00:40:34.902 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create user auth table" 2024-02-23T00:40:34.902 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-02-23T00:40:34.902 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="alter user_auth.auth_id to length 190" 2024-02-23T00:40:34.902 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth access token to user_auth" 2024-02-23T00:40:34.902 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth refresh token to user_auth" 2024-02-23T00:40:34.902 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth token type to user_auth" 2024-02-23T00:40:34.902 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth expiry to user_auth" 2024-02-23T00:40:34.903 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add index to user_id column in user_auth" 2024-02-23T00:40:34.903 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create server_lock table" 2024-02-23T00:40:34.903 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index server_lock.operation_uid" 2024-02-23T00:40:34.903 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create user auth token table" 2024-02-23T00:40:34.903 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user_auth_token.auth_token" 2024-02-23T00:40:34.903 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user_auth_token.prev_auth_token" 2024-02-23T00:40:34.903 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index user_auth_token.user_id" 2024-02-23T00:40:34.903 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add revoked_at to the user auth token" 2024-02-23T00:40:34.903 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create cache_data table" 2024-02-23T00:40:34.903 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index cache_data.cache_key" 2024-02-23T00:40:34.903 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create short_url table v1" 2024-02-23T00:40:34.903 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index short_url.org_id-uid" 2024-02-23T00:40:34.903 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="delete alert_definition table" 2024-02-23T00:40:34.903 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="recreate alert_definition table" 2024-02-23T00:40:34.904 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition on org_id and title columns" 2024-02-23T00:40:35.194 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition on org_id and uid columns" 2024-02-23T00:40:35.194 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_definition table data column to mediumtext in mysql" 2024-02-23T00:40:35.194 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop index in alert_definition on org_id and title columns" 2024-02-23T00:40:35.194 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop index in alert_definition on org_id and uid columns" 2024-02-23T00:40:35.194 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index in alert_definition on org_id and title columns" 2024-02-23T00:40:35.194 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index in alert_definition on org_id and uid columns" 2024-02-23T00:40:35.194 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column paused in alert_definition" 2024-02-23T00:40:35.195 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop alert_definition table" 2024-02-23T00:40:35.195 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="delete alert_definition_version table" 2024-02-23T00:40:35.195 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="recreate alert_definition_version table" 2024-02-23T00:40:35.195 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-02-23T00:40:35.195 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-02-23T00:40:35.195 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_definition_version table data column to mediumtext in mysql" 2024-02-23T00:40:35.195 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop alert_definition_version table" 2024-02-23T00:40:35.195 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_instance table" 2024-02-23T00:40:35.195 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-02-23T00:40:35.195 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_instance table on def_org_id, current_state columns" 2024-02-23T00:40:35.195 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add column current_state_end to alert_instance" 2024-02-23T00:40:35.195 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-02-23T00:40:35.195 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="remove index def_org_id, current_state on alert_instance" 2024-02-23T00:40:35.195 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="rename def_org_id to rule_org_id in alert_instance" 2024-02-23T00:40:35.196 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="rename def_uid to rule_uid in alert_instance" 2024-02-23T00:40:35.196 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-02-23T00:40:35.196 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index rule_org_id, current_state on alert_instance" 2024-02-23T00:40:35.196 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_rule table" 2024-02-23T00:40:35.196 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id and title columns" 2024-02-23T00:40:35.196 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id and uid columns" 2024-02-23T00:40:35.196 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-02-23T00:40:35.196 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_rule table data column to mediumtext in mysql" 2024-02-23T00:40:35.196 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add column for to alert_rule" 2024-02-23T00:40:35.196 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add column annotations to alert_rule" 2024-02-23T00:40:35.196 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add column labels to alert_rule" 2024-02-23T00:40:35.196 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="remove unique index from alert_rule on org_id, title columns" 2024-02-23T00:40:35.196 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-02-23T00:40:35.197 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add dashboard_uid column to alert_rule" 2024-02-23T00:40:35.197 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add panel_id column to alert_rule" 2024-02-23T00:40:35.197 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-02-23T00:40:35.197 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_rule_version table" 2024-02-23T00:40:35.197 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-02-23T00:40:35.197 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-02-23T00:40:35.197 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_rule_version table data column to mediumtext in mysql" 2024-02-23T00:40:35.197 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add column for to alert_rule_version" 2024-02-23T00:40:35.197 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add column annotations to alert_rule_version" 2024-02-23T00:40:35.197 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add column labels to alert_rule_version" 2024-02-23T00:40:35.198 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id=create_alert_configuration_table 2024-02-23T00:40:35.198 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column default in alert_configuration" 2024-02-23T00:40:35.198 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-02-23T00:40:35.198 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add column org_id in alert_configuration" 2024-02-23T00:40:35.198 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_configuration table on org_id column" 2024-02-23T00:40:35.198 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id=create_ngalert_configuration_table 2024-02-23T00:40:35.198 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index in ngalert_configuration on org_id column" 2024-02-23T00:40:35.199 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="clear migration entry \"remove unified alerting data\"" 2024-02-23T00:40:35.199 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="move dashboard alerts to unified alerting" 2024-02-23T00:40:35.199 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create library_element table v1" 2024-02-23T00:40:35.199 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index library_element org_id-folder_id-name-kind" 2024-02-23T00:40:35.199 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create library_element_connection table v1" 2024-02-23T00:40:35.199 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index library_element_connection element_id-kind-connection_id" 2024-02-23T00:40:35.199 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index library_element org_id_uid" 2024-02-23T00:40:35.199 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="clone move dashboard alerts to unified alerting" 2024-02-23T00:40:35.199 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create data_keys table" 2024-02-23T00:40:35.199 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create kv_store table v1" 2024-02-23T00:40:35.199 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index kv_store.org_id-namespace-key" 2024-02-23T00:40:35.199 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="update dashboard_uid and panel_id from existing annotations" 2024-02-23T00:40:35.200 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create permission table" 2024-02-23T00:40:35.200 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index permission.role_id" 2024-02-23T00:40:35.200 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index role_id_action_scope" 2024-02-23T00:40:35.200 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create role table" 2024-02-23T00:40:35.200 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add column display_name" 2024-02-23T00:40:35.200 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add column group_name" 2024-02-23T00:40:35.200 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index role.org_id" 2024-02-23T00:40:35.200 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index role_org_id_name" 2024-02-23T00:40:35.200 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index role_org_id_uid" 2024-02-23T00:40:35.200 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create team role table" 2024-02-23T00:40:35.200 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_role.org_id" 2024-02-23T00:40:35.200 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index team_role_org_id_team_id_role_id" 2024-02-23T00:40:35.200 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_role.team_id" 2024-02-23T00:40:35.200 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create user role table" 2024-02-23T00:40:35.201 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index user_role.org_id" 2024-02-23T00:40:35.201 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user_role_org_id_user_id_role_id" 2024-02-23T00:40:35.201 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index user_role.user_id" 2024-02-23T00:40:35.201 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create builtin role table" 2024-02-23T00:40:35.201 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index builtin_role.role_id" 2024-02-23T00:40:35.201 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index builtin_role.name" 2024-02-23T00:40:35.201 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column org_id to builtin_role table" 2024-02-23T00:40:35.201 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index builtin_role.org_id" 2024-02-23T00:40:35.201 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index builtin_role_org_id_role_id_role" 2024-02-23T00:40:35.201 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="Remove unique index role_org_id_uid" 2024-02-23T00:40:35.201 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index role.uid" 2024-02-23T00:40:35.201 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="create seed assignment table" 2024-02-23T00:40:35.201 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index builtin_role_role_name" 2024-02-23T00:40:35.201 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="migrations completed" logger=migrator performed=381 skipped=0 duration=461.056057ms 2024-02-23T00:40:35.201 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Created default organization" logger=sqlstore 2024-02-23T00:40:35.202 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Initialising plugins" logger=plugin.manager 2024-02-23T00:40:35.202 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:34+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=input 2024-02-23T00:40:35.202 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:35+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=grafana-piechart-panel 2024-02-23T00:40:35.202 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:35+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=vonage-status-panel 2024-02-23T00:40:35.202 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:35+0000 lvl=info msg="Live Push Gateway initialization" logger=live.push_http 2024-02-23T00:40:35.202 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:35+0000 lvl=warn msg="[Deprecated] the datasource provisioning config is outdated. please upgrade" logger=provisioning.datasources filename=/etc/grafana/provisioning/datasources/ceph-dashboard.yml 2024-02-23T00:40:35.202 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:35+0000 lvl=info msg="inserting datasource from configuration " logger=provisioning.datasources name=Dashboard1 uid=P43CA22E17D0F9596 2024-02-23T00:40:35.202 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:35+0000 lvl=info msg="warming cache for startup" logger=ngalert 2024-02-23T00:40:35.202 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:35+0000 lvl=info msg="HTTP Server Listen" logger=http.server address=[::]:3000 protocol=https subUrl= socket= 2024-02-23T00:40:35.202 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:40:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:40:35+0000 lvl=info msg="starting MultiOrg Alertmanager" logger=ngalert.multiorg.alertmanager 2024-02-23T00:40:35.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:34.532767+0000 osd.2 (osd.2) 17 : cluster [DBG] 2.e deep-scrub starts 2024-02-23T00:40:35.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:34.533728+0000 osd.2 (osd.2) 18 : cluster [DBG] 2.e deep-scrub ok 2024-02-23T00:40:35.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:34.626617+0000 mon.a (mon.0) 703 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:35.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:34.627694+0000 mon.a (mon.0) 704 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:40:35.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:34.907529+0000 osd.1 (osd.1) 19 : cluster [DBG] 2.15 scrub starts 2024-02-23T00:40:35.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:34.909131+0000 osd.1 (osd.1) 20 : cluster [DBG] 2.15 scrub ok 2024-02-23T00:40:35.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:35.136300+0000 mgr.y (mgr.14517) 48 : cluster [DBG] pgmap v39: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-23T00:40:36.036 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:34.532767+0000 osd.2 (osd.2) 17 : cluster [DBG] 2.e deep-scrub starts 2024-02-23T00:40:36.036 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:34.533728+0000 osd.2 (osd.2) 18 : cluster [DBG] 2.e deep-scrub ok 2024-02-23T00:40:36.037 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:34.626617+0000 mon.a (mon.0) 703 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:36.037 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:34.627694+0000 mon.a (mon.0) 704 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:40:36.037 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:34.907529+0000 osd.1 (osd.1) 19 : cluster [DBG] 2.15 scrub starts 2024-02-23T00:40:36.037 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:34.909131+0000 osd.1 (osd.1) 20 : cluster [DBG] 2.15 scrub ok 2024-02-23T00:40:36.037 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:35.136300+0000 mgr.y (mgr.14517) 48 : cluster [DBG] pgmap v39: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-23T00:40:36.037 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:34.532767+0000 osd.2 (osd.2) 17 : cluster [DBG] 2.e deep-scrub starts 2024-02-23T00:40:36.037 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:34.533728+0000 osd.2 (osd.2) 18 : cluster [DBG] 2.e deep-scrub ok 2024-02-23T00:40:36.037 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:34.626617+0000 mon.a (mon.0) 703 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:36.037 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:34.627694+0000 mon.a (mon.0) 704 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:40:36.037 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:34.907529+0000 osd.1 (osd.1) 19 : cluster [DBG] 2.15 scrub starts 2024-02-23T00:40:36.037 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:34.909131+0000 osd.1 (osd.1) 20 : cluster [DBG] 2.15 scrub ok 2024-02-23T00:40:36.038 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:35.136300+0000 mgr.y (mgr.14517) 48 : cluster [DBG] pgmap v39: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-23T00:40:36.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:35.617397+0000 osd.7 (osd.7) 23 : cluster [DBG] 2.1c scrub starts 2024-02-23T00:40:36.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:35.620287+0000 osd.7 (osd.7) 24 : cluster [DBG] 2.1c scrub ok 2024-02-23T00:40:36.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:35.858962+0000 osd.1 (osd.1) 21 : cluster [DBG] 5.c scrub starts 2024-02-23T00:40:36.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:35.861850+0000 osd.1 (osd.1) 22 : cluster [DBG] 5.c scrub ok 2024-02-23T00:40:36.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:36.297918+0000 mon.a (mon.0) 705 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:36.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:35.617397+0000 osd.7 (osd.7) 23 : cluster [DBG] 2.1c scrub starts 2024-02-23T00:40:36.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:35.620287+0000 osd.7 (osd.7) 24 : cluster [DBG] 2.1c scrub ok 2024-02-23T00:40:36.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:35.858962+0000 osd.1 (osd.1) 21 : cluster [DBG] 5.c scrub starts 2024-02-23T00:40:36.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:35.861850+0000 osd.1 (osd.1) 22 : cluster [DBG] 5.c scrub ok 2024-02-23T00:40:36.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:36.297918+0000 mon.a (mon.0) 705 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:36.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:35.617397+0000 osd.7 (osd.7) 23 : cluster [DBG] 2.1c scrub starts 2024-02-23T00:40:36.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:35.620287+0000 osd.7 (osd.7) 24 : cluster [DBG] 2.1c scrub ok 2024-02-23T00:40:36.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:35.858962+0000 osd.1 (osd.1) 21 : cluster [DBG] 5.c scrub starts 2024-02-23T00:40:36.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:35.861850+0000 osd.1 (osd.1) 22 : cluster [DBG] 5.c scrub ok 2024-02-23T00:40:36.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:36.297918+0000 mon.a (mon.0) 705 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:37.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.882+0000 7fc3fce96700 1 -- 172.21.15.47:0/3258801012 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc3f8102270 msgr2=0x7fc3f8104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:37.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.882+0000 7fc3fce96700 1 --2- 172.21.15.47:0/3258801012 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc3f8102270 0x7fc3f8104660 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fc3f0009be0 tx=0x7fc3f0004d10).stop 2024-02-23T00:40:37.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.882+0000 7fc3fce96700 1 -- 172.21.15.47:0/3258801012 shutdown_connections 2024-02-23T00:40:37.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.882+0000 7fc3fce96700 1 --2- 172.21.15.47:0/3258801012 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc3f8102270 0x7fc3f8104660 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:37.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.882+0000 7fc3fce96700 1 --2- 172.21.15.47:0/3258801012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3f80f8d10 0x7fc3f80f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:37.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.882+0000 7fc3fce96700 1 --2- 172.21.15.47:0/3258801012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3f80f9670 0x7fc3f8101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:37.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.882+0000 7fc3fce96700 1 -- 172.21.15.47:0/3258801012 >> 172.21.15.47:0/3258801012 conn(0x7fc3f80f2820 msgr2=0x7fc3f80f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:40:37.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.882+0000 7fc3fce96700 1 -- 172.21.15.47:0/3258801012 shutdown_connections 2024-02-23T00:40:37.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.882+0000 7fc3fce96700 1 -- 172.21.15.47:0/3258801012 wait complete. 2024-02-23T00:40:37.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.882+0000 7fc3fce96700 1 librados: starting msgr at 2024-02-23T00:40:37.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.882+0000 7fc3fce96700 1 librados: starting objecter 2024-02-23T00:40:37.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.882+0000 7fc3fce96700 1 Processor -- start 2024-02-23T00:40:37.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.883+0000 7fc3fce96700 1 -- start start 2024-02-23T00:40:37.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.883+0000 7fc3fce96700 1 librados: setting wanted keys 2024-02-23T00:40:37.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.883+0000 7fc3fce96700 1 librados: calling monclient init 2024-02-23T00:40:37.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.883+0000 7fc3fce96700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc3f80f8d10 0x7fc3f80f6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:37.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.883+0000 7fc3fce96700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3f80f9670 0x7fc3f80f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:37.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.883+0000 7fc3fce96700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3f8102270 0x7fc3f80f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:37.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.883+0000 7fc3fce96700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f8116340 con 0x7fc3f8102270 2024-02-23T00:40:37.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.883+0000 7fc3fce96700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f81164b0 con 0x7fc3f80f9670 2024-02-23T00:40:37.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.883+0000 7fc3fce96700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f8116670 con 0x7fc3f80f8d10 2024-02-23T00:40:37.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.883+0000 7fc3f659c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc3f80f8d10 0x7fc3f80f6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:37.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.883+0000 7fc3f659c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc3f80f8d10 0x7fc3f80f6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:46338/0 (socket says 172.21.15.47:46338) 2024-02-23T00:40:37.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.883+0000 7fc3f659c700 1 -- 172.21.15.47:0/3477504150 learned_addr learned my addr 172.21.15.47:0/3477504150 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:40:37.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.883+0000 7fc3f6d9d700 1 --2- 172.21.15.47:0/3477504150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3f8102270 0x7fc3f80f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:37.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.883+0000 7fc3f5d9b700 1 --2- 172.21.15.47:0/3477504150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3f80f9670 0x7fc3f80f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:37.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.883+0000 7fc3f659c700 1 -- 172.21.15.47:0/3477504150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3f80f9670 msgr2=0x7fc3f80f71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:37.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.883+0000 7fc3f659c700 1 --2- 172.21.15.47:0/3477504150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3f80f9670 0x7fc3f80f71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:37.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.883+0000 7fc3f659c700 1 -- 172.21.15.47:0/3477504150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3f8102270 msgr2=0x7fc3f80f76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:37.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.883+0000 7fc3f659c700 1 --2- 172.21.15.47:0/3477504150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3f8102270 0x7fc3f80f76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:37.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.883+0000 7fc3f659c700 1 -- 172.21.15.47:0/3477504150 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e8005040 con 0x7fc3f80f8d10 2024-02-23T00:40:37.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.883+0000 7fc3f659c700 1 --2- 172.21.15.47:0/3477504150 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc3f80f8d10 0x7fc3f80f6cc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fc3e80048f0 tx=0x7fc3e8009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:37.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.883+0000 7fc3e77fe700 10 client.?.objecter ms_handle_connect 0x7fc3f80f8d10 2024-02-23T00:40:37.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.883+0000 7fc3e77fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:40:37.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.884+0000 7fc3e77fe700 1 -- 172.21.15.47:0/3477504150 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8007b00 con 0x7fc3f80f8d10 2024-02-23T00:40:37.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.884+0000 7fc3fce96700 1 -- 172.21.15.47:0/3477504150 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f00097c0 con 0x7fc3f80f8d10 2024-02-23T00:40:37.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.884+0000 7fc3e77fe700 1 -- 172.21.15.47:0/3477504150 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fc3e800f040 con 0x7fc3f80f8d10 2024-02-23T00:40:37.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.884+0000 7fc3e77fe700 1 -- 172.21.15.47:0/3477504150 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8014820 con 0x7fc3f80f8d10 2024-02-23T00:40:37.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.884+0000 7fc3fce96700 10 client.24518.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:40:37.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.884+0000 7fc3fce96700 1 -- 172.21.15.47:0/3477504150 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f8116cb0 con 0x7fc3f80f8d10 2024-02-23T00:40:37.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.884+0000 7fc3fce96700 1 librados: init done 2024-02-23T00:40:37.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.884+0000 7fc3fce96700 10 client.24518.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:40:37.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.884+0000 7fc3fce96700 15 client.24518.objecter _session_command_op_assign -1 1 2024-02-23T00:40:37.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.884+0000 7fc3fce96700 20 client.24518.objecter _get_session osd=-1 returning homeless 2024-02-23T00:40:37.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.884+0000 7fc3fce96700 10 client.24518.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:40:37.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.884+0000 7fc3fce96700 1 -- 172.21.15.47:0/3477504150 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc3f805fa60 con 0x7fc3f80f8d10 2024-02-23T00:40:37.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.886+0000 7fc3e77fe700 1 -- 172.21.15.47:0/3477504150 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fc3e8014c50 con 0x7fc3f80f8d10 2024-02-23T00:40:37.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.886+0000 7fc3e77fe700 1 --2- 172.21.15.47:0/3477504150 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc3e0062cc0 0x7fc3e0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:37.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.887+0000 7fc3e77fe700 1 -- 172.21.15.47:0/3477504150 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7305+0+0 (secure 0 0 0) 0x7fc3e807ca10 con 0x7fc3f80f8d10 2024-02-23T00:40:37.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.887+0000 7fc3e77fe700 10 client.24518.objecter ms_dispatch 0x7fc3f8001550 osd_map(83..83 src has 1..83) v4 2024-02-23T00:40:37.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.887+0000 7fc3e77fe700 3 client.24518.objecter handle_osd_map got epochs [83,83] > 0 2024-02-23T00:40:37.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.887+0000 7fc3e77fe700 3 client.24518.objecter handle_osd_map decoding full epoch 83 2024-02-23T00:40:37.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.887+0000 7fc3e77fe700 10 client.24518.objecter checking command 1 2024-02-23T00:40:37.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.887+0000 7fc3e77fe700 1 --2- 172.21.15.47:0/3477504150 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fc3e006ca30 0x7fc3e006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:37.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.887+0000 7fc3e77fe700 20 client.24518.objecter _get_session s=0x7fc3e006c360 osd=2 3 2024-02-23T00:40:37.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.887+0000 7fc3e77fe700 20 client.24518.objecter put_session s=0x7fc3e006c360 osd=2 3 2024-02-23T00:40:37.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.887+0000 7fc3e77fe700 15 client.24518.objecter _session_command_op_remove -1 1 2024-02-23T00:40:37.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.887+0000 7fc3e77fe700 20 client.24518.objecter _get_session s=0x7fc3e006c360 osd=2 3 2024-02-23T00:40:37.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.887+0000 7fc3e77fe700 20 client.24518.objecter get_session s=0x7fc3e006c360 osd=2 3 2024-02-23T00:40:37.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.887+0000 7fc3e77fe700 15 client.24518.objecter _session_command_op_assign 2 1 2024-02-23T00:40:37.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.887+0000 7fc3e77fe700 20 client.24518.objecter put_session s=0x7fc3e006c360 osd=2 4 2024-02-23T00:40:37.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.887+0000 7fc3e77fe700 10 client.24518.objecter _send_command 1 2024-02-23T00:40:37.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.887+0000 7fc3e77fe700 1 -- 172.21.15.47:0/3477504150 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc3e006f550 con 0x7fc3e006ca30 2024-02-23T00:40:37.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.887+0000 7fc3e77fe700 20 client.24518.objecter dump_active .. 0 homeless 2024-02-23T00:40:37.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.887+0000 7fc3e77fe700 1 -- 172.21.15.47:0/3477504150 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3e8081220 con 0x7fc3f80f8d10 2024-02-23T00:40:37.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.887+0000 7fc3f6d9d700 1 --2- 172.21.15.47:0/3477504150 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fc3e006ca30 0x7fc3e006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:37.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.887+0000 7fc3f5d9b700 1 --2- 172.21.15.47:0/3477504150 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc3e0062cc0 0x7fc3e0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:37.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.887+0000 7fc3e77fe700 10 client.24518.objecter ms_handle_connect 0x7fc3e006ca30 2024-02-23T00:40:37.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.887+0000 7fc3f6d9d700 1 --2- 172.21.15.47:0/3477504150 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fc3e006ca30 0x7fc3e006eed0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:37.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.887+0000 7fc3e77fe700 10 client.24518.objecter ms_handle_connect 0x7fc3e0062cc0 2024-02-23T00:40:37.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.887+0000 7fc3f5d9b700 1 --2- 172.21.15.47:0/3477504150 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc3e0062cc0 0x7fc3e0065180 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fc3f80f8460 tx=0x7fc3ec00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:37.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.888+0000 7fc3e77fe700 1 -- 172.21.15.47:0/3477504150 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc3e006f550 con 0x7fc3e006ca30 2024-02-23T00:40:37.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.888+0000 7fc3e77fe700 10 client.24518.objecter ms_dispatch 0x7fc3f8001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:40:37.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.888+0000 7fc3e77fe700 10 client.24518.objecter _finish_command 1 = system:0 2024-02-23T00:40:37.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.888+0000 7fc3e77fe700 20 client.24518.objecter put_session s=0x7fc3e006c360 osd=2 4 2024-02-23T00:40:37.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.888+0000 7fc3e77fe700 15 client.24518.objecter _session_command_op_remove 2 1 2024-02-23T00:40:37.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.902+0000 7fc3fce96700 10 client.24518.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-02-23T00:40:37.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.902+0000 7fc3fce96700 15 client.24518.objecter _session_command_op_assign -1 2 2024-02-23T00:40:37.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.902+0000 7fc3fce96700 20 client.24518.objecter _get_session s=0x7fc3e006c360 osd=2 3 2024-02-23T00:40:37.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.902+0000 7fc3fce96700 20 client.24518.objecter put_session s=0x7fc3e006c360 osd=2 3 2024-02-23T00:40:37.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.902+0000 7fc3fce96700 20 client.24518.objecter _get_session s=0x7fc3e006c360 osd=2 3 2024-02-23T00:40:37.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.902+0000 7fc3fce96700 15 client.24518.objecter _session_command_op_remove -1 2 2024-02-23T00:40:37.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.902+0000 7fc3fce96700 20 client.24518.objecter get_session s=0x7fc3e006c360 osd=2 3 2024-02-23T00:40:37.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.902+0000 7fc3fce96700 15 client.24518.objecter _session_command_op_assign 2 2 2024-02-23T00:40:37.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.902+0000 7fc3fce96700 20 client.24518.objecter put_session s=0x7fc3e006c360 osd=2 4 2024-02-23T00:40:37.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.902+0000 7fc3fce96700 10 client.24518.objecter _send_command 2 2024-02-23T00:40:37.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.902+0000 7fc3fce96700 1 -- 172.21.15.47:0/3477504150 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc3f805ee80 con 0x7fc3e006ca30 2024-02-23T00:40:37.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.902+0000 7fc3e77fe700 1 -- 172.21.15.47:0/3477504150 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc3f805ee80 con 0x7fc3e006ca30 2024-02-23T00:40:37.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.902+0000 7fc3e77fe700 10 client.24518.objecter ms_dispatch 0x7fc3f8001550 command_reply(tid 2: 0 ) v1 2024-02-23T00:40:37.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.902+0000 7fc3e77fe700 10 client.24518.objecter _finish_command 2 = system:0 2024-02-23T00:40:37.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.902+0000 7fc3e77fe700 20 client.24518.objecter put_session s=0x7fc3e006c360 osd=2 4 2024-02-23T00:40:37.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.902+0000 7fc3e77fe700 15 client.24518.objecter _session_command_op_remove 2 2 2024-02-23T00:40:37.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.903+0000 7fc3fce96700 10 librados: watch_flush enter 2024-02-23T00:40:37.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.903+0000 7fc3fce96700 10 librados: watch_flush exit 2024-02-23T00:40:37.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.903+0000 7fc3fce96700 10 client.24518.objecter close_session for osd.2 2024-02-23T00:40:37.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.903+0000 7fc3fce96700 1 -- 172.21.15.47:0/3477504150 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fc3e006ca30 msgr2=0x7fc3e006eed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:37.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.903+0000 7fc3fce96700 1 --2- 172.21.15.47:0/3477504150 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fc3e006ca30 0x7fc3e006eed0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:37.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.903+0000 7fc3fce96700 20 client.24518.objecter put_session s=0x7fc3e006c360 osd=2 1 2024-02-23T00:40:37.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.903+0000 7fc3fce96700 20 client.24518.objecter shutdown clearing up homeless session... 2024-02-23T00:40:37.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.903+0000 7fc3fce96700 10 client.24518.objecter successfully canceled tick 2024-02-23T00:40:37.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.903+0000 7fc3fce96700 1 -- 172.21.15.47:0/3477504150 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc3e0062cc0 msgr2=0x7fc3e0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:37.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.903+0000 7fc3fce96700 1 --2- 172.21.15.47:0/3477504150 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc3e0062cc0 0x7fc3e0065180 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fc3f80f8460 tx=0x7fc3ec00b040).stop 2024-02-23T00:40:37.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.903+0000 7fc3fce96700 1 -- 172.21.15.47:0/3477504150 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc3f80f8d10 msgr2=0x7fc3f80f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:37.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.903+0000 7fc3fce96700 1 --2- 172.21.15.47:0/3477504150 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc3f80f8d10 0x7fc3f80f6cc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fc3e80048f0 tx=0x7fc3e8009cb0).stop 2024-02-23T00:40:37.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.903+0000 7fc3f6d9d700 1 -- 172.21.15.47:0/3477504150 reap_dead start 2024-02-23T00:40:37.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.903+0000 7fc3fce96700 1 -- 172.21.15.47:0/3477504150 shutdown_connections 2024-02-23T00:40:37.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.903+0000 7fc3fce96700 1 -- 172.21.15.47:0/3477504150 >> 172.21.15.47:0/3477504150 conn(0x7fc3f80f2820 msgr2=0x7fc3f80f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:40:37.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.903+0000 7fc3fce96700 1 -- 172.21.15.47:0/3477504150 shutdown_connections 2024-02-23T00:40:37.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.903+0000 7fc3fce96700 1 -- 172.21.15.47:0/3477504150 wait complete. 2024-02-23T00:40:37.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:36.903+0000 7fc3fce96700 1 librados: shutdown 2024-02-23T00:40:37.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:36.274404+0000 osd.5 (osd.5) 13 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:40:37.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:36.274451+0000 osd.5 (osd.5) 14 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:40:37.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:36.502501+0000 osd.2 (osd.2) 19 : cluster [DBG] 5.6 scrub starts 2024-02-23T00:40:37.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:36.503338+0000 osd.2 (osd.2) 20 : cluster [DBG] 5.6 scrub ok 2024-02-23T00:40:37.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:36.595369+0000 osd.7 (osd.7) 25 : cluster [DBG] 6.5 scrub starts 2024-02-23T00:40:37.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:36.598257+0000 osd.7 (osd.7) 26 : cluster [DBG] 6.5 scrub ok 2024-02-23T00:40:37.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:36.635567+0000 mon.a (mon.0) 706 : cluster [DBG] osdmap e83: 8 total, 8 up, 8 in 2024-02-23T00:40:37.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:36.886461+0000 osd.1 (osd.1) 23 : cluster [DBG] 2.6 scrub starts 2024-02-23T00:40:37.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:36.888120+0000 osd.1 (osd.1) 24 : cluster [DBG] 2.6 scrub ok 2024-02-23T00:40:37.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:37.136891+0000 mgr.y (mgr.14517) 49 : cluster [DBG] pgmap v41: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:40:38.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:36.274404+0000 osd.5 (osd.5) 13 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:40:38.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:36.274451+0000 osd.5 (osd.5) 14 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:40:38.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:36.502501+0000 osd.2 (osd.2) 19 : cluster [DBG] 5.6 scrub starts 2024-02-23T00:40:38.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:36.503338+0000 osd.2 (osd.2) 20 : cluster [DBG] 5.6 scrub ok 2024-02-23T00:40:38.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:36.595369+0000 osd.7 (osd.7) 25 : cluster [DBG] 6.5 scrub starts 2024-02-23T00:40:38.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:36.598257+0000 osd.7 (osd.7) 26 : cluster [DBG] 6.5 scrub ok 2024-02-23T00:40:38.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:36.635567+0000 mon.a (mon.0) 706 : cluster [DBG] osdmap e83: 8 total, 8 up, 8 in 2024-02-23T00:40:38.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:36.886461+0000 osd.1 (osd.1) 23 : cluster [DBG] 2.6 scrub starts 2024-02-23T00:40:38.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:36.888120+0000 osd.1 (osd.1) 24 : cluster [DBG] 2.6 scrub ok 2024-02-23T00:40:38.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:37.136891+0000 mgr.y (mgr.14517) 49 : cluster [DBG] pgmap v41: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:40:38.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:36.274404+0000 osd.5 (osd.5) 13 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:40:38.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:36.274451+0000 osd.5 (osd.5) 14 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:40:38.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:36.502501+0000 osd.2 (osd.2) 19 : cluster [DBG] 5.6 scrub starts 2024-02-23T00:40:38.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:36.503338+0000 osd.2 (osd.2) 20 : cluster [DBG] 5.6 scrub ok 2024-02-23T00:40:38.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:36.595369+0000 osd.7 (osd.7) 25 : cluster [DBG] 6.5 scrub starts 2024-02-23T00:40:38.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:36.598257+0000 osd.7 (osd.7) 26 : cluster [DBG] 6.5 scrub ok 2024-02-23T00:40:38.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:36.635567+0000 mon.a (mon.0) 706 : cluster [DBG] osdmap e83: 8 total, 8 up, 8 in 2024-02-23T00:40:38.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:36.886461+0000 osd.1 (osd.1) 23 : cluster [DBG] 2.6 scrub starts 2024-02-23T00:40:38.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:36.888120+0000 osd.1 (osd.1) 24 : cluster [DBG] 2.6 scrub ok 2024-02-23T00:40:38.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:37.136891+0000 mgr.y (mgr.14517) 49 : cluster [DBG] pgmap v41: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:40:38.198 INFO:teuthology.orchestra.run.smithi047.stdout:77309411375 2024-02-23T00:40:38.198 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-23T00:40:38.510 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:40:39.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:38.027568+0000 osd.4 (osd.4) 17 : cluster [DBG] 2.f scrub starts 2024-02-23T00:40:39.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:38.028991+0000 osd.4 (osd.4) 18 : cluster [DBG] 2.f scrub ok 2024-02-23T00:40:39.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:38.027568+0000 osd.4 (osd.4) 17 : cluster [DBG] 2.f scrub starts 2024-02-23T00:40:39.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:38.028991+0000 osd.4 (osd.4) 18 : cluster [DBG] 2.f scrub ok 2024-02-23T00:40:39.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:38.027568+0000 osd.4 (osd.4) 17 : cluster [DBG] 2.f scrub starts 2024-02-23T00:40:39.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:38.028991+0000 osd.4 (osd.4) 18 : cluster [DBG] 2.f scrub ok 2024-02-23T00:40:40.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:38.809350+0000 osd.5 (osd.5) 15 : cluster [DBG] 5.7 scrub starts 2024-02-23T00:40:40.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:38.812324+0000 osd.5 (osd.5) 16 : cluster [DBG] 5.7 scrub ok 2024-02-23T00:40:40.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:38.956654+0000 osd.6 (osd.6) 15 : cluster [DBG] 5.11 scrub starts 2024-02-23T00:40:40.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:38.957571+0000 osd.6 (osd.6) 16 : cluster [DBG] 5.11 scrub ok 2024-02-23T00:40:40.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:39.137318+0000 mgr.y (mgr.14517) 50 : cluster [DBG] pgmap v42: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:40:40.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:39.325115+0000 mon.a (mon.0) 707 : cluster [DBG] osdmap e84: 8 total, 8 up, 8 in 2024-02-23T00:40:40.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:39.446739+0000 mon.a (mon.0) 708 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:40.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:38.809350+0000 osd.5 (osd.5) 15 : cluster [DBG] 5.7 scrub starts 2024-02-23T00:40:40.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:38.812324+0000 osd.5 (osd.5) 16 : cluster [DBG] 5.7 scrub ok 2024-02-23T00:40:40.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:38.956654+0000 osd.6 (osd.6) 15 : cluster [DBG] 5.11 scrub starts 2024-02-23T00:40:40.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:38.957571+0000 osd.6 (osd.6) 16 : cluster [DBG] 5.11 scrub ok 2024-02-23T00:40:40.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:39.137318+0000 mgr.y (mgr.14517) 50 : cluster [DBG] pgmap v42: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:40:40.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:39.325115+0000 mon.a (mon.0) 707 : cluster [DBG] osdmap e84: 8 total, 8 up, 8 in 2024-02-23T00:40:40.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:39.446739+0000 mon.a (mon.0) 708 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:40.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:38.809350+0000 osd.5 (osd.5) 15 : cluster [DBG] 5.7 scrub starts 2024-02-23T00:40:40.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:38.812324+0000 osd.5 (osd.5) 16 : cluster [DBG] 5.7 scrub ok 2024-02-23T00:40:40.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:38.956654+0000 osd.6 (osd.6) 15 : cluster [DBG] 5.11 scrub starts 2024-02-23T00:40:40.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:38.957571+0000 osd.6 (osd.6) 16 : cluster [DBG] 5.11 scrub ok 2024-02-23T00:40:40.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:39.137318+0000 mgr.y (mgr.14517) 50 : cluster [DBG] pgmap v42: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:40:40.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:39.325115+0000 mon.a (mon.0) 707 : cluster [DBG] osdmap e84: 8 total, 8 up, 8 in 2024-02-23T00:40:40.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:39.446739+0000 mon.a (mon.0) 708 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:40.806 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:40:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:40:40] "GET /metrics HTTP/1.1" 200 172129 "" "Prometheus/2.33.4" 2024-02-23T00:40:41.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:39.821785+0000 osd.1 (osd.1) 25 : cluster [DBG] 2.a deep-scrub starts 2024-02-23T00:40:41.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:39.825007+0000 osd.1 (osd.1) 26 : cluster [DBG] 2.a deep-scrub ok 2024-02-23T00:40:41.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:39.858379+0000 osd.5 (osd.5) 17 : cluster [DBG] 6.a scrub starts 2024-02-23T00:40:41.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:39.859193+0000 osd.5 (osd.5) 18 : cluster [DBG] 6.a scrub ok 2024-02-23T00:40:41.306 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:39.821785+0000 osd.1 (osd.1) 25 : cluster [DBG] 2.a deep-scrub starts 2024-02-23T00:40:41.306 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:39.825007+0000 osd.1 (osd.1) 26 : cluster [DBG] 2.a deep-scrub ok 2024-02-23T00:40:41.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:39.858379+0000 osd.5 (osd.5) 17 : cluster [DBG] 6.a scrub starts 2024-02-23T00:40:41.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:39.859193+0000 osd.5 (osd.5) 18 : cluster [DBG] 6.a scrub ok 2024-02-23T00:40:41.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:39.821785+0000 osd.1 (osd.1) 25 : cluster [DBG] 2.a deep-scrub starts 2024-02-23T00:40:41.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:39.825007+0000 osd.1 (osd.1) 26 : cluster [DBG] 2.a deep-scrub ok 2024-02-23T00:40:41.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:39.858379+0000 osd.5 (osd.5) 17 : cluster [DBG] 6.a scrub starts 2024-02-23T00:40:41.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:39.859193+0000 osd.5 (osd.5) 18 : cluster [DBG] 6.a scrub ok 2024-02-23T00:40:41.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.636+0000 7ff17491d700 1 -- 172.21.15.47:0/316959026 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1700fe510 msgr2=0x7ff1700f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:41.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.636+0000 7ff17491d700 1 --2- 172.21.15.47:0/316959026 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1700fe510 0x7ff1700f7d40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7ff164004870 tx=0x7ff164009cb0).stop 2024-02-23T00:40:41.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.636+0000 7ff17491d700 1 -- 172.21.15.47:0/316959026 shutdown_connections 2024-02-23T00:40:41.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.636+0000 7ff17491d700 1 --2- 172.21.15.47:0/316959026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1700f8420 0x7ff170104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:41.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.636+0000 7ff17491d700 1 --2- 172.21.15.47:0/316959026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1700fdbb0 0x7ff1700fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:41.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.636+0000 7ff17491d700 1 --2- 172.21.15.47:0/316959026 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1700fe510 0x7ff1700f7d40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:41.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.636+0000 7ff17491d700 1 -- 172.21.15.47:0/316959026 >> 172.21.15.47:0/316959026 conn(0x7ff1700f3800 msgr2=0x7ff1700f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:40:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.636+0000 7ff17491d700 1 -- 172.21.15.47:0/316959026 shutdown_connections 2024-02-23T00:40:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.636+0000 7ff17491d700 1 -- 172.21.15.47:0/316959026 wait complete. 2024-02-23T00:40:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.636+0000 7ff17491d700 1 librados: starting msgr at 2024-02-23T00:40:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.636+0000 7ff17491d700 1 librados: starting objecter 2024-02-23T00:40:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.637+0000 7ff17491d700 1 Processor -- start 2024-02-23T00:40:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.637+0000 7ff17491d700 1 -- start start 2024-02-23T00:40:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.637+0000 7ff17491d700 1 librados: setting wanted keys 2024-02-23T00:40:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.637+0000 7ff17491d700 1 librados: calling monclient init 2024-02-23T00:40:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.637+0000 7ff17491d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1700f8420 0x7ff17010de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.637+0000 7ff17491d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1700fdbb0 0x7ff17010e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.637+0000 7ff17491d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1700fe510 0x7ff170108030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.637+0000 7ff17491d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff170108520 con 0x7ff1700fe510 2024-02-23T00:40:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.637+0000 7ff17491d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff170108690 con 0x7ff1700fdbb0 2024-02-23T00:40:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.637+0000 7ff17491d700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff170108850 con 0x7ff1700f8420 2024-02-23T00:40:41.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.637+0000 7ff16e59c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1700f8420 0x7ff17010de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:41.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.637+0000 7ff16e59c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1700f8420 0x7ff17010de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:46352/0 (socket says 172.21.15.47:46352) 2024-02-23T00:40:41.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.637+0000 7ff16e59c700 1 -- 172.21.15.47:0/2449172308 learned_addr learned my addr 172.21.15.47:0/2449172308 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:40:41.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.637+0000 7ff16ed9d700 1 --2- 172.21.15.47:0/2449172308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1700fe510 0x7ff170108030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:41.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.637+0000 7ff16dd9b700 1 --2- 172.21.15.47:0/2449172308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1700fdbb0 0x7ff17010e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:41.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.637+0000 7ff16e59c700 1 -- 172.21.15.47:0/2449172308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1700fdbb0 msgr2=0x7ff17010e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:41.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.637+0000 7ff16e59c700 1 --2- 172.21.15.47:0/2449172308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1700fdbb0 0x7ff17010e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:41.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.637+0000 7ff16e59c700 1 -- 172.21.15.47:0/2449172308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1700fe510 msgr2=0x7ff170108030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:41.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.637+0000 7ff16e59c700 1 --2- 172.21.15.47:0/2449172308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1700fe510 0x7ff170108030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:41.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.637+0000 7ff16e59c700 1 -- 172.21.15.47:0/2449172308 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff164005040 con 0x7ff1700f8420 2024-02-23T00:40:41.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.638+0000 7ff16e59c700 1 --2- 172.21.15.47:0/2449172308 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1700f8420 0x7ff17010de90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7ff16000e970 tx=0x7ff16000b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:41.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.638+0000 7ff15f7fe700 10 client.?.objecter ms_handle_connect 0x7ff1700f8420 2024-02-23T00:40:41.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.638+0000 7ff15f7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:40:41.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.638+0000 7ff15f7fe700 1 -- 172.21.15.47:0/2449172308 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16000ee60 con 0x7ff1700f8420 2024-02-23T00:40:41.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.638+0000 7ff15f7fe700 1 -- 172.21.15.47:0/2449172308 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7ff16000f040 con 0x7ff1700f8420 2024-02-23T00:40:41.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.638+0000 7ff15f7fe700 1 -- 172.21.15.47:0/2449172308 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16000c5b0 con 0x7ff1700f8420 2024-02-23T00:40:41.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.638+0000 7ff17491d700 1 -- 172.21.15.47:0/2449172308 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff170108b30 con 0x7ff1700f8420 2024-02-23T00:40:41.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.638+0000 7ff17491d700 10 client.24524.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:40:41.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.638+0000 7ff17491d700 1 -- 172.21.15.47:0/2449172308 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff170108eb0 con 0x7ff1700f8420 2024-02-23T00:40:41.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.638+0000 7ff17491d700 1 librados: init done 2024-02-23T00:40:41.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.639+0000 7ff17491d700 10 client.24524.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:40:41.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.639+0000 7ff17491d700 15 client.24524.objecter _session_command_op_assign -1 1 2024-02-23T00:40:41.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.639+0000 7ff17491d700 20 client.24524.objecter _get_session osd=-1 returning homeless 2024-02-23T00:40:41.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.639+0000 7ff17491d700 10 client.24524.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:40:41.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.639+0000 7ff17491d700 1 -- 172.21.15.47:0/2449172308 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff17005ee80 con 0x7ff1700f8420 2024-02-23T00:40:41.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.639+0000 7ff15f7fe700 1 -- 172.21.15.47:0/2449172308 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7ff160007b40 con 0x7ff1700f8420 2024-02-23T00:40:41.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.639+0000 7ff15f7fe700 1 --2- 172.21.15.47:0/2449172308 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff158062cc0 0x7ff158065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:41.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.640+0000 7ff15f7fe700 1 -- 172.21.15.47:0/2449172308 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7305+0+0 (secure 0 0 0) 0x7ff16007ca70 con 0x7ff1700f8420 2024-02-23T00:40:41.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.640+0000 7ff15f7fe700 10 client.24524.objecter ms_dispatch 0x7ff170001550 osd_map(84..84 src has 1..84) v4 2024-02-23T00:40:41.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.640+0000 7ff15f7fe700 3 client.24524.objecter handle_osd_map got epochs [84,84] > 0 2024-02-23T00:40:41.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.640+0000 7ff15f7fe700 3 client.24524.objecter handle_osd_map decoding full epoch 84 2024-02-23T00:40:41.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.640+0000 7ff16dd9b700 1 --2- 172.21.15.47:0/2449172308 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff158062cc0 0x7ff158065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:41.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.640+0000 7ff15f7fe700 10 client.24524.objecter checking command 1 2024-02-23T00:40:41.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.640+0000 7ff15f7fe700 1 --2- 172.21.15.47:0/2449172308 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7ff15806cab0 0x7ff15806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:41.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.640+0000 7ff15f7fe700 20 client.24524.objecter _get_session s=0x7ff15806c3e0 osd=3 3 2024-02-23T00:40:41.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.640+0000 7ff15f7fe700 20 client.24524.objecter put_session s=0x7ff15806c3e0 osd=3 3 2024-02-23T00:40:41.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.640+0000 7ff15f7fe700 15 client.24524.objecter _session_command_op_remove -1 1 2024-02-23T00:40:41.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.640+0000 7ff15f7fe700 20 client.24524.objecter _get_session s=0x7ff15806c3e0 osd=3 3 2024-02-23T00:40:41.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.640+0000 7ff15f7fe700 20 client.24524.objecter get_session s=0x7ff15806c3e0 osd=3 3 2024-02-23T00:40:41.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.640+0000 7ff15f7fe700 15 client.24524.objecter _session_command_op_assign 3 1 2024-02-23T00:40:41.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.640+0000 7ff15f7fe700 20 client.24524.objecter put_session s=0x7ff15806c3e0 osd=3 4 2024-02-23T00:40:41.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.640+0000 7ff15f7fe700 10 client.24524.objecter _send_command 1 2024-02-23T00:40:41.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.640+0000 7ff15f7fe700 1 -- 172.21.15.47:0/2449172308 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff15806f5d0 con 0x7ff15806cab0 2024-02-23T00:40:41.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.640+0000 7ff15f7fe700 20 client.24524.objecter dump_active .. 0 homeless 2024-02-23T00:40:41.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.640+0000 7ff15f7fe700 1 -- 172.21.15.47:0/2449172308 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff160081220 con 0x7ff1700f8420 2024-02-23T00:40:41.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.640+0000 7ff16ed9d700 1 --2- 172.21.15.47:0/2449172308 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7ff15806cab0 0x7ff15806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:41.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.640+0000 7ff15f7fe700 10 client.24524.objecter ms_handle_connect 0x7ff158062cc0 2024-02-23T00:40:41.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.640+0000 7ff16dd9b700 1 --2- 172.21.15.47:0/2449172308 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff158062cc0 0x7ff158065180 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7ff164004ee0 tx=0x7ff16400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:41.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.640+0000 7ff15f7fe700 10 client.24524.objecter ms_handle_connect 0x7ff15806cab0 2024-02-23T00:40:41.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.640+0000 7ff16ed9d700 1 --2- 172.21.15.47:0/2449172308 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7ff15806cab0 0x7ff15806ef50 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:41.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.641+0000 7ff15f7fe700 1 -- 172.21.15.47:0/2449172308 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff15806f5d0 con 0x7ff15806cab0 2024-02-23T00:40:41.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.641+0000 7ff15f7fe700 10 client.24524.objecter ms_dispatch 0x7ff170001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:40:41.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.641+0000 7ff15f7fe700 10 client.24524.objecter _finish_command 1 = system:0 2024-02-23T00:40:41.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.641+0000 7ff15f7fe700 20 client.24524.objecter put_session s=0x7ff15806c3e0 osd=3 4 2024-02-23T00:40:41.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.641+0000 7ff15f7fe700 15 client.24524.objecter _session_command_op_remove 3 1 2024-02-23T00:40:41.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.655+0000 7ff17491d700 10 client.24524.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-02-23T00:40:41.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.655+0000 7ff17491d700 15 client.24524.objecter _session_command_op_assign -1 2 2024-02-23T00:40:41.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.655+0000 7ff17491d700 20 client.24524.objecter _get_session s=0x7ff15806c3e0 osd=3 3 2024-02-23T00:40:41.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.655+0000 7ff17491d700 20 client.24524.objecter put_session s=0x7ff15806c3e0 osd=3 3 2024-02-23T00:40:41.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.655+0000 7ff17491d700 20 client.24524.objecter _get_session s=0x7ff15806c3e0 osd=3 3 2024-02-23T00:40:41.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.655+0000 7ff17491d700 15 client.24524.objecter _session_command_op_remove -1 2 2024-02-23T00:40:41.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.655+0000 7ff17491d700 20 client.24524.objecter get_session s=0x7ff15806c3e0 osd=3 3 2024-02-23T00:40:41.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.655+0000 7ff17491d700 15 client.24524.objecter _session_command_op_assign 3 2 2024-02-23T00:40:41.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.655+0000 7ff17491d700 20 client.24524.objecter put_session s=0x7ff15806c3e0 osd=3 4 2024-02-23T00:40:41.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.655+0000 7ff17491d700 10 client.24524.objecter _send_command 2 2024-02-23T00:40:41.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.655+0000 7ff17491d700 1 -- 172.21.15.47:0/2449172308 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff170106e60 con 0x7ff15806cab0 2024-02-23T00:40:41.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.655+0000 7ff15f7fe700 1 -- 172.21.15.47:0/2449172308 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff170106e60 con 0x7ff15806cab0 2024-02-23T00:40:41.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.655+0000 7ff15f7fe700 10 client.24524.objecter ms_dispatch 0x7ff170001550 command_reply(tid 2: 0 ) v1 2024-02-23T00:40:41.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.655+0000 7ff15f7fe700 10 client.24524.objecter _finish_command 2 = system:0 2024-02-23T00:40:41.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.655+0000 7ff15f7fe700 20 client.24524.objecter put_session s=0x7ff15806c3e0 osd=3 4 2024-02-23T00:40:41.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.655+0000 7ff15f7fe700 15 client.24524.objecter _session_command_op_remove 3 2 2024-02-23T00:40:41.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.656+0000 7ff17491d700 10 librados: watch_flush enter 2024-02-23T00:40:41.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.656+0000 7ff17491d700 10 librados: watch_flush exit 2024-02-23T00:40:41.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.656+0000 7ff17491d700 10 client.24524.objecter close_session for osd.3 2024-02-23T00:40:41.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.656+0000 7ff17491d700 1 -- 172.21.15.47:0/2449172308 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7ff15806cab0 msgr2=0x7ff15806ef50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:41.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.656+0000 7ff17491d700 1 --2- 172.21.15.47:0/2449172308 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7ff15806cab0 0x7ff15806ef50 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:41.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.656+0000 7ff17491d700 20 client.24524.objecter put_session s=0x7ff15806c3e0 osd=3 1 2024-02-23T00:40:41.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.656+0000 7ff17491d700 20 client.24524.objecter shutdown clearing up homeless session... 2024-02-23T00:40:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.656+0000 7ff17491d700 10 client.24524.objecter successfully canceled tick 2024-02-23T00:40:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.656+0000 7ff17491d700 1 -- 172.21.15.47:0/2449172308 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff158062cc0 msgr2=0x7ff158065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.656+0000 7ff17491d700 1 --2- 172.21.15.47:0/2449172308 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff158062cc0 0x7ff158065180 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7ff164004ee0 tx=0x7ff16400b040).stop 2024-02-23T00:40:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.656+0000 7ff17491d700 1 -- 172.21.15.47:0/2449172308 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1700f8420 msgr2=0x7ff17010de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.656+0000 7ff17491d700 1 --2- 172.21.15.47:0/2449172308 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1700f8420 0x7ff17010de90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7ff16000e970 tx=0x7ff16000b040).stop 2024-02-23T00:40:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.656+0000 7ff16ed9d700 1 -- 172.21.15.47:0/2449172308 reap_dead start 2024-02-23T00:40:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.656+0000 7ff17491d700 1 -- 172.21.15.47:0/2449172308 shutdown_connections 2024-02-23T00:40:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.656+0000 7ff17491d700 1 -- 172.21.15.47:0/2449172308 >> 172.21.15.47:0/2449172308 conn(0x7ff1700f3800 msgr2=0x7ff1700fb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:40:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.656+0000 7ff17491d700 1 -- 172.21.15.47:0/2449172308 shutdown_connections 2024-02-23T00:40:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.656+0000 7ff17491d700 1 -- 172.21.15.47:0/2449172308 wait complete. 2024-02-23T00:40:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:41.656+0000 7ff17491d700 1 librados: shutdown 2024-02-23T00:40:42.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:40.359616+0000 osd.2 (osd.2) 21 : cluster [DBG] 6.14 scrub starts 2024-02-23T00:40:42.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:40.360791+0000 osd.2 (osd.2) 22 : cluster [DBG] 6.14 scrub ok 2024-02-23T00:40:42.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:41.137686+0000 mgr.y (mgr.14517) 51 : cluster [DBG] pgmap v44: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:40:42.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:41.143355+0000 mon.a (mon.0) 709 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2024-02-23T00:40:42.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:41.143595+0000 mon.a (mon.0) 710 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-02-23T00:40:42.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:41.143757+0000 mon.a (mon.0) 711 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-02-23T00:40:42.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:41.143915+0000 mon.a (mon.0) 712 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-02-23T00:40:42.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:41.667998+0000 osd.7 (osd.7) 27 : cluster [DBG] 2.5 scrub starts 2024-02-23T00:40:42.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:41.670947+0000 osd.7 (osd.7) 28 : cluster [DBG] 2.5 scrub ok 2024-02-23T00:40:42.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:40.359616+0000 osd.2 (osd.2) 21 : cluster [DBG] 6.14 scrub starts 2024-02-23T00:40:42.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:40.360791+0000 osd.2 (osd.2) 22 : cluster [DBG] 6.14 scrub ok 2024-02-23T00:40:42.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:41.137686+0000 mgr.y (mgr.14517) 51 : cluster [DBG] pgmap v44: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:40:42.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:41.143355+0000 mon.a (mon.0) 709 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2024-02-23T00:40:42.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:41.143595+0000 mon.a (mon.0) 710 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-02-23T00:40:42.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:41.143757+0000 mon.a (mon.0) 711 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-02-23T00:40:42.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:41.143915+0000 mon.a (mon.0) 712 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-02-23T00:40:42.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:41.667998+0000 osd.7 (osd.7) 27 : cluster [DBG] 2.5 scrub starts 2024-02-23T00:40:42.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:41.670947+0000 osd.7 (osd.7) 28 : cluster [DBG] 2.5 scrub ok 2024-02-23T00:40:42.193 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:40:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:40:42.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:40.359616+0000 osd.2 (osd.2) 21 : cluster [DBG] 6.14 scrub starts 2024-02-23T00:40:42.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:40.360791+0000 osd.2 (osd.2) 22 : cluster [DBG] 6.14 scrub ok 2024-02-23T00:40:42.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:41.137686+0000 mgr.y (mgr.14517) 51 : cluster [DBG] pgmap v44: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:40:42.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:41.143355+0000 mon.a (mon.0) 709 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2024-02-23T00:40:42.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:41.143595+0000 mon.a (mon.0) 710 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-02-23T00:40:42.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:41.143757+0000 mon.a (mon.0) 711 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-02-23T00:40:42.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:41.143915+0000 mon.a (mon.0) 712 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-02-23T00:40:42.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:41.667998+0000 osd.7 (osd.7) 27 : cluster [DBG] 2.5 scrub starts 2024-02-23T00:40:42.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:41.670947+0000 osd.7 (osd.7) 28 : cluster [DBG] 2.5 scrub ok 2024-02-23T00:40:42.938 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:40:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:40:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:40:42.938 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:41.768696+0000 osd.1 (osd.1) 27 : cluster [DBG] 4.2 scrub starts 2024-02-23T00:40:42.939 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:41.772146+0000 osd.1 (osd.1) 28 : cluster [DBG] 4.2 scrub ok 2024-02-23T00:40:42.939 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:41.918356+0000 osd.6 (osd.6) 17 : cluster [DBG] 4.5 scrub starts 2024-02-23T00:40:42.939 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:41.920305+0000 osd.6 (osd.6) 18 : cluster [DBG] 4.5 scrub ok 2024-02-23T00:40:42.939 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:41.928678+0000 mon.a (mon.0) 713 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]': finished 2024-02-23T00:40:42.939 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:41.928749+0000 mon.a (mon.0) 714 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-02-23T00:40:42.939 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:41.928813+0000 mon.a (mon.0) 715 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-02-23T00:40:42.939 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:41.928874+0000 mon.a (mon.0) 716 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-02-23T00:40:42.939 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:41.928925+0000 mon.a (mon.0) 717 : cluster [DBG] osdmap e85: 8 total, 8 up, 8 in 2024-02-23T00:40:42.939 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:41.996013+0000 mon.a (mon.0) 718 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:42.939 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:41.996871+0000 mon.a (mon.0) 719 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:40:42.939 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:41.997540+0000 mon.a (mon.0) 720 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:40:42.940 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:42.001540+0000 mon.a (mon.0) 721 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:42.940 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:40:42.005439+0000 mgr.y (mgr.14517) 52 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2024-02-23T00:40:42.940 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:40:42.009195+0000 mgr.y (mgr.14517) 53 : cephadm [INF] Reconfiguring daemon alertmanager.a on smithi047 2024-02-23T00:40:42.940 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:42.158785+0000 mgr.y (mgr.14517) 54 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:40:42.940 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:42.664406+0000 osd.7 (osd.7) 29 : cluster [DBG] 5.1a scrub starts 2024-02-23T00:40:42.940 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:42.667287+0000 osd.7 (osd.7) 30 : cluster [DBG] 5.1a scrub ok 2024-02-23T00:40:43.047 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:41.768696+0000 osd.1 (osd.1) 27 : cluster [DBG] 4.2 scrub starts 2024-02-23T00:40:43.047 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:41.772146+0000 osd.1 (osd.1) 28 : cluster [DBG] 4.2 scrub ok 2024-02-23T00:40:43.047 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:41.918356+0000 osd.6 (osd.6) 17 : cluster [DBG] 4.5 scrub starts 2024-02-23T00:40:43.047 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:41.920305+0000 osd.6 (osd.6) 18 : cluster [DBG] 4.5 scrub ok 2024-02-23T00:40:43.048 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:41.928678+0000 mon.a (mon.0) 713 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]': finished 2024-02-23T00:40:43.048 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:41.928749+0000 mon.a (mon.0) 714 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-02-23T00:40:43.048 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:41.928813+0000 mon.a (mon.0) 715 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-02-23T00:40:43.048 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:41.928874+0000 mon.a (mon.0) 716 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-02-23T00:40:43.048 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:41.928925+0000 mon.a (mon.0) 717 : cluster [DBG] osdmap e85: 8 total, 8 up, 8 in 2024-02-23T00:40:43.048 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:41.996013+0000 mon.a (mon.0) 718 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:43.048 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:41.996871+0000 mon.a (mon.0) 719 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:40:43.048 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:41.997540+0000 mon.a (mon.0) 720 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:40:43.048 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:42.001540+0000 mon.a (mon.0) 721 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:43.048 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:40:42.005439+0000 mgr.y (mgr.14517) 52 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2024-02-23T00:40:43.048 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:40:42.009195+0000 mgr.y (mgr.14517) 53 : cephadm [INF] Reconfiguring daemon alertmanager.a on smithi047 2024-02-23T00:40:43.048 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:42.158785+0000 mgr.y (mgr.14517) 54 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:40:43.048 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:42.664406+0000 osd.7 (osd.7) 29 : cluster [DBG] 5.1a scrub starts 2024-02-23T00:40:43.048 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:42.667287+0000 osd.7 (osd.7) 30 : cluster [DBG] 5.1a scrub ok 2024-02-23T00:40:43.049 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:41.768696+0000 osd.1 (osd.1) 27 : cluster [DBG] 4.2 scrub starts 2024-02-23T00:40:43.049 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:41.772146+0000 osd.1 (osd.1) 28 : cluster [DBG] 4.2 scrub ok 2024-02-23T00:40:43.049 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:41.918356+0000 osd.6 (osd.6) 17 : cluster [DBG] 4.5 scrub starts 2024-02-23T00:40:43.049 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:41.920305+0000 osd.6 (osd.6) 18 : cluster [DBG] 4.5 scrub ok 2024-02-23T00:40:43.049 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:41.928678+0000 mon.a (mon.0) 713 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]': finished 2024-02-23T00:40:43.049 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:41.928749+0000 mon.a (mon.0) 714 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-02-23T00:40:43.049 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:41.928813+0000 mon.a (mon.0) 715 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-02-23T00:40:43.049 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:41.928874+0000 mon.a (mon.0) 716 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-02-23T00:40:43.049 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:41.928925+0000 mon.a (mon.0) 717 : cluster [DBG] osdmap e85: 8 total, 8 up, 8 in 2024-02-23T00:40:43.049 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:41.996013+0000 mon.a (mon.0) 718 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:43.049 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:41.996871+0000 mon.a (mon.0) 719 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:40:43.049 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:41.997540+0000 mon.a (mon.0) 720 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:40:43.050 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:42.001540+0000 mon.a (mon.0) 721 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:43.050 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:40:42.005439+0000 mgr.y (mgr.14517) 52 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2024-02-23T00:40:43.050 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:40:42.009195+0000 mgr.y (mgr.14517) 53 : cephadm [INF] Reconfiguring daemon alertmanager.a on smithi047 2024-02-23T00:40:43.050 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:42.158785+0000 mgr.y (mgr.14517) 54 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:40:43.050 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:42.664406+0000 osd.7 (osd.7) 29 : cluster [DBG] 5.1a scrub starts 2024-02-23T00:40:43.050 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:42.667287+0000 osd.7 (osd.7) 30 : cluster [DBG] 5.1a scrub ok 2024-02-23T00:40:43.860 INFO:teuthology.orchestra.run.smithi047.stdout:111669149737 2024-02-23T00:40:43.860 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-23T00:40:44.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:42.863573+0000 osd.5 (osd.5) 19 : cluster [DBG] 4.15 scrub starts 2024-02-23T00:40:44.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:42.867038+0000 osd.5 (osd.5) 20 : cluster [DBG] 4.15 scrub ok 2024-02-23T00:40:44.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:42.929385+0000 mon.a (mon.0) 722 : cluster [DBG] osdmap e86: 8 total, 8 up, 8 in 2024-02-23T00:40:44.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:43.138076+0000 mgr.y (mgr.14517) 55 : cluster [DBG] pgmap v47: 132 pgs: 4 peering, 128 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:40:44.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:42.863573+0000 osd.5 (osd.5) 19 : cluster [DBG] 4.15 scrub starts 2024-02-23T00:40:44.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:42.867038+0000 osd.5 (osd.5) 20 : cluster [DBG] 4.15 scrub ok 2024-02-23T00:40:44.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:42.929385+0000 mon.a (mon.0) 722 : cluster [DBG] osdmap e86: 8 total, 8 up, 8 in 2024-02-23T00:40:44.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:43.138076+0000 mgr.y (mgr.14517) 55 : cluster [DBG] pgmap v47: 132 pgs: 4 peering, 128 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:40:44.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:42.863573+0000 osd.5 (osd.5) 19 : cluster [DBG] 4.15 scrub starts 2024-02-23T00:40:44.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:42.867038+0000 osd.5 (osd.5) 20 : cluster [DBG] 4.15 scrub ok 2024-02-23T00:40:44.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:42.929385+0000 mon.a (mon.0) 722 : cluster [DBG] osdmap e86: 8 total, 8 up, 8 in 2024-02-23T00:40:44.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:43.138076+0000 mgr.y (mgr.14517) 55 : cluster [DBG] pgmap v47: 132 pgs: 4 peering, 128 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:40:45.439 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:40:45.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:44.811414+0000 osd.1 (osd.1) 29 : cluster [DBG] 4.7 scrub starts 2024-02-23T00:40:45.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:44.813501+0000 osd.1 (osd.1) 30 : cluster [DBG] 4.7 scrub ok 2024-02-23T00:40:45.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:44.905592+0000 osd.3 (osd.3) 17 : cluster [DBG] 2.19 scrub starts 2024-02-23T00:40:45.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:44.906595+0000 osd.3 (osd.3) 18 : cluster [DBG] 2.19 scrub ok 2024-02-23T00:40:45.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:44.811414+0000 osd.1 (osd.1) 29 : cluster [DBG] 4.7 scrub starts 2024-02-23T00:40:45.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:44.813501+0000 osd.1 (osd.1) 30 : cluster [DBG] 4.7 scrub ok 2024-02-23T00:40:45.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:44.905592+0000 osd.3 (osd.3) 17 : cluster [DBG] 2.19 scrub starts 2024-02-23T00:40:45.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:44.906595+0000 osd.3 (osd.3) 18 : cluster [DBG] 2.19 scrub ok 2024-02-23T00:40:45.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:44.811414+0000 osd.1 (osd.1) 29 : cluster [DBG] 4.7 scrub starts 2024-02-23T00:40:45.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:44.813501+0000 osd.1 (osd.1) 30 : cluster [DBG] 4.7 scrub ok 2024-02-23T00:40:45.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:44.905592+0000 osd.3 (osd.3) 17 : cluster [DBG] 2.19 scrub starts 2024-02-23T00:40:45.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:44.906595+0000 osd.3 (osd.3) 18 : cluster [DBG] 2.19 scrub ok 2024-02-23T00:40:46.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:44.710932+0000 osd.7 (osd.7) 31 : cluster [DBG] 5.9 scrub starts 2024-02-23T00:40:46.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:44.713820+0000 osd.7 (osd.7) 32 : cluster [DBG] 5.9 scrub ok 2024-02-23T00:40:46.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:45.138359+0000 mgr.y (mgr.14517) 56 : cluster [DBG] pgmap v48: 132 pgs: 4 peering, 128 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:40:46.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:44.710932+0000 osd.7 (osd.7) 31 : cluster [DBG] 5.9 scrub starts 2024-02-23T00:40:46.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:44.713820+0000 osd.7 (osd.7) 32 : cluster [DBG] 5.9 scrub ok 2024-02-23T00:40:46.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:45.138359+0000 mgr.y (mgr.14517) 56 : cluster [DBG] pgmap v48: 132 pgs: 4 peering, 128 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:40:46.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:44.710932+0000 osd.7 (osd.7) 31 : cluster [DBG] 5.9 scrub starts 2024-02-23T00:40:46.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:44.713820+0000 osd.7 (osd.7) 32 : cluster [DBG] 5.9 scrub ok 2024-02-23T00:40:46.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:45.138359+0000 mgr.y (mgr.14517) 56 : cluster [DBG] pgmap v48: 132 pgs: 4 peering, 128 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:40:47.044 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:40:46 smithi047 systemd[1]: Stopping Ceph alertmanager.a for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T00:40:47.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:45.970482+0000 osd.6 (osd.6) 19 : cluster [DBG] 2.7 deep-scrub starts 2024-02-23T00:40:47.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:45.971460+0000 osd.6 (osd.6) 20 : cluster [DBG] 2.7 deep-scrub ok 2024-02-23T00:40:47.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:45.990257+0000 osd.0 (osd.0) 15 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:40:47.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:45.990300+0000 osd.0 (osd.0) 16 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:40:47.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:46.125906+0000 mon.a (mon.0) 723 : cluster [DBG] osdmap e87: 8 total, 8 up, 8 in 2024-02-23T00:40:47.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:46.875331+0000 osd.3 (osd.3) 19 : cluster [DBG] 6.c deep-scrub starts 2024-02-23T00:40:47.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:46.876374+0000 osd.3 (osd.3) 20 : cluster [DBG] 6.c deep-scrub ok 2024-02-23T00:40:47.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:45.970482+0000 osd.6 (osd.6) 19 : cluster [DBG] 2.7 deep-scrub starts 2024-02-23T00:40:47.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:45.971460+0000 osd.6 (osd.6) 20 : cluster [DBG] 2.7 deep-scrub ok 2024-02-23T00:40:47.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:45.990257+0000 osd.0 (osd.0) 15 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:40:47.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:45.990300+0000 osd.0 (osd.0) 16 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:40:47.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:46.125906+0000 mon.a (mon.0) 723 : cluster [DBG] osdmap e87: 8 total, 8 up, 8 in 2024-02-23T00:40:47.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:46.875331+0000 osd.3 (osd.3) 19 : cluster [DBG] 6.c deep-scrub starts 2024-02-23T00:40:47.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:46.876374+0000 osd.3 (osd.3) 20 : cluster [DBG] 6.c deep-scrub ok 2024-02-23T00:40:47.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:45.970482+0000 osd.6 (osd.6) 19 : cluster [DBG] 2.7 deep-scrub starts 2024-02-23T00:40:47.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:45.971460+0000 osd.6 (osd.6) 20 : cluster [DBG] 2.7 deep-scrub ok 2024-02-23T00:40:47.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:45.990257+0000 osd.0 (osd.0) 15 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:40:47.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:45.990300+0000 osd.0 (osd.0) 16 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:40:47.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:46.125906+0000 mon.a (mon.0) 723 : cluster [DBG] osdmap e87: 8 total, 8 up, 8 in 2024-02-23T00:40:47.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:46.875331+0000 osd.3 (osd.3) 19 : cluster [DBG] 6.c deep-scrub starts 2024-02-23T00:40:47.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:46.876374+0000 osd.3 (osd.3) 20 : cluster [DBG] 6.c deep-scrub ok 2024-02-23T00:40:47.667 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:40:47 smithi047 bash[71059]: Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager.a" found: no such container 2024-02-23T00:40:48.056 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:40:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[67409]: level=info ts=2024-02-23T00:40:47.784Z caller=main.go:557 msg="Received SIGTERM, exiting gracefully..." 2024-02-23T00:40:48.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.135+0000 7fc7136be700 1 -- 172.21.15.47:0/909108582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc70c0fe510 msgr2=0x7fc70c0f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:48.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.135+0000 7fc7136be700 1 --2- 172.21.15.47:0/909108582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc70c0fe510 0x7fc70c0f7d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fc6fc0047b0 tx=0x7fc6fc009cb0).stop 2024-02-23T00:40:48.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.135+0000 7fc7136be700 1 -- 172.21.15.47:0/909108582 shutdown_connections 2024-02-23T00:40:48.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.135+0000 7fc7136be700 1 --2- 172.21.15.47:0/909108582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc70c0f8420 0x7fc70c104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.135+0000 7fc7136be700 1 --2- 172.21.15.47:0/909108582 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc70c0fdbb0 0x7fc70c0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.135+0000 7fc7136be700 1 --2- 172.21.15.47:0/909108582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc70c0fe510 0x7fc70c0f7d40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.135+0000 7fc7136be700 1 -- 172.21.15.47:0/909108582 >> 172.21.15.47:0/909108582 conn(0x7fc70c0f3800 msgr2=0x7fc70c0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:40:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.135+0000 7fc7136be700 1 -- 172.21.15.47:0/909108582 shutdown_connections 2024-02-23T00:40:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.135+0000 7fc7136be700 1 -- 172.21.15.47:0/909108582 wait complete. 2024-02-23T00:40:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.136+0000 7fc7136be700 1 librados: starting msgr at 2024-02-23T00:40:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.136+0000 7fc7136be700 1 librados: starting objecter 2024-02-23T00:40:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.136+0000 7fc7136be700 1 Processor -- start 2024-02-23T00:40:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.136+0000 7fc7136be700 1 -- start start 2024-02-23T00:40:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.136+0000 7fc7136be700 1 librados: setting wanted keys 2024-02-23T00:40:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.136+0000 7fc7136be700 1 librados: calling monclient init 2024-02-23T00:40:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.136+0000 7fc7136be700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc70c0f8420 0x7fc70c10df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.136+0000 7fc7136be700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc70c0fdbb0 0x7fc70c10e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.136+0000 7fc7136be700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc70c0fe510 0x7fc70c108130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.136+0000 7fc7136be700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c108620 con 0x7fc70c0fdbb0 2024-02-23T00:40:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.136+0000 7fc7136be700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c108790 con 0x7fc70c0f8420 2024-02-23T00:40:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.136+0000 7fc7136be700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c108950 con 0x7fc70c0fe510 2024-02-23T00:40:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.136+0000 7fc710c59700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc70c0fdbb0 0x7fc70c10e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.136+0000 7fc710c59700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc70c0fdbb0 0x7fc70c10e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34164/0 (socket says 172.21.15.47:34164) 2024-02-23T00:40:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.136+0000 7fc710c59700 1 -- 172.21.15.47:0/1601086108 learned_addr learned my addr 172.21.15.47:0/1601086108 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:40:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.136+0000 7fc711c5b700 1 --2- 172.21.15.47:0/1601086108 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc70c0fe510 0x7fc70c108130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.136+0000 7fc71145a700 1 --2- 172.21.15.47:0/1601086108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc70c0f8420 0x7fc70c10df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.136+0000 7fc710c59700 1 -- 172.21.15.47:0/1601086108 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc70c0fe510 msgr2=0x7fc70c108130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.136+0000 7fc710c59700 1 --2- 172.21.15.47:0/1601086108 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc70c0fe510 0x7fc70c108130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.136+0000 7fc710c59700 1 -- 172.21.15.47:0/1601086108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc70c0f8420 msgr2=0x7fc70c10df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.136+0000 7fc710c59700 1 --2- 172.21.15.47:0/1601086108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc70c0f8420 0x7fc70c10df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.136+0000 7fc710c59700 1 -- 172.21.15.47:0/1601086108 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc708005040 con 0x7fc70c0fdbb0 2024-02-23T00:40:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.137+0000 7fc710c59700 1 --2- 172.21.15.47:0/1601086108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc70c0fdbb0 0x7fc70c10e480 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fc6fc0044e0 tx=0x7fc6fc009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:48.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.137+0000 7fc7027fc700 10 client.?.objecter ms_handle_connect 0x7fc70c0fdbb0 2024-02-23T00:40:48.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.137+0000 7fc7027fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:40:48.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.137+0000 7fc7027fc700 1 -- 172.21.15.47:0/1601086108 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc018070 con 0x7fc70c0fdbb0 2024-02-23T00:40:48.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.137+0000 7fc7027fc700 1 -- 172.21.15.47:0/1601086108 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fc6fc013e50 con 0x7fc70c0fdbb0 2024-02-23T00:40:48.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.137+0000 7fc7136be700 1 -- 172.21.15.47:0/1601086108 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6fc005040 con 0x7fc70c0fdbb0 2024-02-23T00:40:48.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.137+0000 7fc7027fc700 1 -- 172.21.15.47:0/1601086108 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc01db60 con 0x7fc70c0fdbb0 2024-02-23T00:40:48.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.137+0000 7fc7136be700 10 client.14637.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:40:48.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.137+0000 7fc7136be700 1 -- 172.21.15.47:0/1601086108 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc70c108e70 con 0x7fc70c0fdbb0 2024-02-23T00:40:48.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.137+0000 7fc7136be700 1 librados: init done 2024-02-23T00:40:48.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.138+0000 7fc7136be700 10 client.14637.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:40:48.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.138+0000 7fc7136be700 15 client.14637.objecter _session_command_op_assign -1 1 2024-02-23T00:40:48.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.138+0000 7fc7136be700 20 client.14637.objecter _get_session osd=-1 returning homeless 2024-02-23T00:40:48.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.138+0000 7fc7136be700 10 client.14637.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:40:48.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.138+0000 7fc7136be700 1 -- 172.21.15.47:0/1601086108 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc70c0606d0 con 0x7fc70c0fdbb0 2024-02-23T00:40:48.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.138+0000 7fc7027fc700 1 -- 172.21.15.47:0/1601086108 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fc6fc009240 con 0x7fc70c0fdbb0 2024-02-23T00:40:48.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.139+0000 7fc7027fc700 1 --2- 172.21.15.47:0/1601086108 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc6f8062cc0 0x7fc6f8065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:48.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.139+0000 7fc7027fc700 1 -- 172.21.15.47:0/1601086108 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7421+0+0 (secure 0 0 0) 0x7fc6fc07e820 con 0x7fc70c0fdbb0 2024-02-23T00:40:48.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.139+0000 7fc7027fc700 10 client.14637.objecter ms_dispatch 0x7fc70c001550 osd_map(87..87 src has 1..87) v4 2024-02-23T00:40:48.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.139+0000 7fc71145a700 1 --2- 172.21.15.47:0/1601086108 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc6f8062cc0 0x7fc6f8065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:48.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.139+0000 7fc7027fc700 3 client.14637.objecter handle_osd_map got epochs [87,87] > 0 2024-02-23T00:40:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.139+0000 7fc7027fc700 3 client.14637.objecter handle_osd_map decoding full epoch 87 2024-02-23T00:40:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.139+0000 7fc7027fc700 10 client.14637.objecter checking command 1 2024-02-23T00:40:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.139+0000 7fc7027fc700 1 --2- 172.21.15.47:0/1601086108 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fc6f806cc50 0x7fc6f806f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.139+0000 7fc7027fc700 20 client.14637.objecter _get_session s=0x7fc6f806c580 osd=4 3 2024-02-23T00:40:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.139+0000 7fc7027fc700 20 client.14637.objecter put_session s=0x7fc6f806c580 osd=4 3 2024-02-23T00:40:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.139+0000 7fc7027fc700 15 client.14637.objecter _session_command_op_remove -1 1 2024-02-23T00:40:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.139+0000 7fc7027fc700 20 client.14637.objecter _get_session s=0x7fc6f806c580 osd=4 3 2024-02-23T00:40:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.139+0000 7fc7027fc700 20 client.14637.objecter get_session s=0x7fc6f806c580 osd=4 3 2024-02-23T00:40:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.139+0000 7fc7027fc700 15 client.14637.objecter _session_command_op_assign 4 1 2024-02-23T00:40:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.139+0000 7fc7027fc700 20 client.14637.objecter put_session s=0x7fc6f806c580 osd=4 4 2024-02-23T00:40:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.139+0000 7fc7027fc700 10 client.14637.objecter _send_command 1 2024-02-23T00:40:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.139+0000 7fc7027fc700 1 -- 172.21.15.47:0/1601086108 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc6f806f770 con 0x7fc6f806cc50 2024-02-23T00:40:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.139+0000 7fc7027fc700 20 client.14637.objecter dump_active .. 0 homeless 2024-02-23T00:40:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.139+0000 7fc7027fc700 1 -- 172.21.15.47:0/1601086108 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6fc07ec20 con 0x7fc70c0fdbb0 2024-02-23T00:40:48.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.139+0000 7fc7027fc700 10 client.14637.objecter ms_handle_connect 0x7fc6f8062cc0 2024-02-23T00:40:48.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.139+0000 7fc71145a700 1 --2- 172.21.15.47:0/1601086108 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc6f8062cc0 0x7fc6f8065180 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fc708004870 tx=0x7fc70800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:48.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.139+0000 7fc711c5b700 1 --2- 172.21.15.47:0/1601086108 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fc6f806cc50 0x7fc6f806f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:48.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.140+0000 7fc7027fc700 10 client.14637.objecter ms_handle_connect 0x7fc6f806cc50 2024-02-23T00:40:48.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.140+0000 7fc711c5b700 1 --2- 172.21.15.47:0/1601086108 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fc6f806cc50 0x7fc6f806f0f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:48.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.140+0000 7fc7027fc700 1 -- 172.21.15.47:0/1601086108 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc6f806f770 con 0x7fc6f806cc50 2024-02-23T00:40:48.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.140+0000 7fc7027fc700 10 client.14637.objecter ms_dispatch 0x7fc70c001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:40:48.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.140+0000 7fc7027fc700 10 client.14637.objecter _finish_command 1 = system:0 2024-02-23T00:40:48.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.140+0000 7fc7027fc700 20 client.14637.objecter put_session s=0x7fc6f806c580 osd=4 4 2024-02-23T00:40:48.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.140+0000 7fc7027fc700 15 client.14637.objecter _session_command_op_remove 4 1 2024-02-23T00:40:48.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.155+0000 7fc7136be700 10 client.14637.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-02-23T00:40:48.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.155+0000 7fc7136be700 15 client.14637.objecter _session_command_op_assign -1 2 2024-02-23T00:40:48.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.155+0000 7fc7136be700 20 client.14637.objecter _get_session s=0x7fc6f806c580 osd=4 3 2024-02-23T00:40:48.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.155+0000 7fc7136be700 20 client.14637.objecter put_session s=0x7fc6f806c580 osd=4 3 2024-02-23T00:40:48.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.155+0000 7fc7136be700 20 client.14637.objecter _get_session s=0x7fc6f806c580 osd=4 3 2024-02-23T00:40:48.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.155+0000 7fc7136be700 15 client.14637.objecter _session_command_op_remove -1 2 2024-02-23T00:40:48.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.155+0000 7fc7136be700 20 client.14637.objecter get_session s=0x7fc6f806c580 osd=4 3 2024-02-23T00:40:48.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.155+0000 7fc7136be700 15 client.14637.objecter _session_command_op_assign 4 2 2024-02-23T00:40:48.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.155+0000 7fc7136be700 20 client.14637.objecter put_session s=0x7fc6f806c580 osd=4 4 2024-02-23T00:40:48.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.155+0000 7fc7136be700 10 client.14637.objecter _send_command 2 2024-02-23T00:40:48.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.155+0000 7fc7136be700 1 -- 172.21.15.47:0/1601086108 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc70c05ee80 con 0x7fc6f806cc50 2024-02-23T00:40:48.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.155+0000 7fc7027fc700 1 -- 172.21.15.47:0/1601086108 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc70c05ee80 con 0x7fc6f806cc50 2024-02-23T00:40:48.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.155+0000 7fc7027fc700 10 client.14637.objecter ms_dispatch 0x7fc70c001550 command_reply(tid 2: 0 ) v1 2024-02-23T00:40:48.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.155+0000 7fc7027fc700 10 client.14637.objecter _finish_command 2 = system:0 2024-02-23T00:40:48.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.155+0000 7fc7027fc700 20 client.14637.objecter put_session s=0x7fc6f806c580 osd=4 4 2024-02-23T00:40:48.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.155+0000 7fc7027fc700 15 client.14637.objecter _session_command_op_remove 4 2 2024-02-23T00:40:48.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.156+0000 7fc7136be700 10 librados: watch_flush enter 2024-02-23T00:40:48.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.156+0000 7fc7136be700 10 librados: watch_flush exit 2024-02-23T00:40:48.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.156+0000 7fc7136be700 10 client.14637.objecter close_session for osd.4 2024-02-23T00:40:48.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.156+0000 7fc7136be700 1 -- 172.21.15.47:0/1601086108 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fc6f806cc50 msgr2=0x7fc6f806f0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:48.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.156+0000 7fc7136be700 1 --2- 172.21.15.47:0/1601086108 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fc6f806cc50 0x7fc6f806f0f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:48.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.156+0000 7fc7136be700 20 client.14637.objecter put_session s=0x7fc6f806c580 osd=4 1 2024-02-23T00:40:48.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.156+0000 7fc7136be700 20 client.14637.objecter shutdown clearing up homeless session... 2024-02-23T00:40:48.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.156+0000 7fc7136be700 10 client.14637.objecter successfully canceled tick 2024-02-23T00:40:48.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.156+0000 7fc7136be700 1 -- 172.21.15.47:0/1601086108 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc6f8062cc0 msgr2=0x7fc6f8065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:48.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.156+0000 7fc7136be700 1 --2- 172.21.15.47:0/1601086108 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc6f8062cc0 0x7fc6f8065180 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fc708004870 tx=0x7fc70800b040).stop 2024-02-23T00:40:48.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.156+0000 7fc7136be700 1 -- 172.21.15.47:0/1601086108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc70c0fdbb0 msgr2=0x7fc70c10e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:48.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.156+0000 7fc7136be700 1 --2- 172.21.15.47:0/1601086108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc70c0fdbb0 0x7fc70c10e480 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fc6fc0044e0 tx=0x7fc6fc009cb0).stop 2024-02-23T00:40:48.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.156+0000 7fc711c5b700 1 -- 172.21.15.47:0/1601086108 reap_dead start 2024-02-23T00:40:48.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.156+0000 7fc7136be700 1 -- 172.21.15.47:0/1601086108 shutdown_connections 2024-02-23T00:40:48.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.156+0000 7fc7136be700 1 -- 172.21.15.47:0/1601086108 >> 172.21.15.47:0/1601086108 conn(0x7fc70c0f3800 msgr2=0x7fc70c0fb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:40:48.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.156+0000 7fc7136be700 1 -- 172.21.15.47:0/1601086108 shutdown_connections 2024-02-23T00:40:48.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.156+0000 7fc7136be700 1 -- 172.21.15.47:0/1601086108 wait complete. 2024-02-23T00:40:48.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:48.156+0000 7fc7136be700 1 librados: shutdown 2024-02-23T00:40:48.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:47.139205+0000 mgr.y (mgr.14517) 57 : cluster [DBG] pgmap v50: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 96 B/s, 4 objects/s recovering 2024-02-23T00:40:48.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:47.139205+0000 mgr.y (mgr.14517) 57 : cluster [DBG] pgmap v50: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 96 B/s, 4 objects/s recovering 2024-02-23T00:40:48.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:47.139205+0000 mgr.y (mgr.14517) 57 : cluster [DBG] pgmap v50: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 96 B/s, 4 objects/s recovering 2024-02-23T00:40:49.241 INFO:teuthology.orchestra.run.smithi047.stdout:137438953511 2024-02-23T00:40:49.241 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-23T00:40:49.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:47.730821+0000 osd.0 (osd.0) 17 : cluster [DBG] 4.b scrub starts 2024-02-23T00:40:49.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:47.732513+0000 osd.0 (osd.0) 18 : cluster [DBG] 4.b scrub ok 2024-02-23T00:40:49.496 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:47.730821+0000 osd.0 (osd.0) 17 : cluster [DBG] 4.b scrub starts 2024-02-23T00:40:49.496 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:47.732513+0000 osd.0 (osd.0) 18 : cluster [DBG] 4.b scrub ok 2024-02-23T00:40:49.496 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:47.730821+0000 osd.0 (osd.0) 17 : cluster [DBG] 4.b scrub starts 2024-02-23T00:40:49.497 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:47.732513+0000 osd.0 (osd.0) 18 : cluster [DBG] 4.b scrub ok 2024-02-23T00:40:49.806 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:40:49 smithi047 bash[71132]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a 2024-02-23T00:40:49.905 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:40:50.412 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:48.695515+0000 osd.0 (osd.0) 19 : cluster [DBG] 4.6 scrub starts 2024-02-23T00:40:50.412 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:48.696891+0000 osd.0 (osd.0) 20 : cluster [DBG] 4.6 scrub ok 2024-02-23T00:40:50.412 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:48.787077+0000 osd.7 (osd.7) 33 : cluster [DBG] 5.4 scrub starts 2024-02-23T00:40:50.413 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:48.789986+0000 osd.7 (osd.7) 34 : cluster [DBG] 5.4 scrub ok 2024-02-23T00:40:50.413 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:48.800090+0000 osd.5 (osd.5) 21 : cluster [DBG] 4.4 scrub starts 2024-02-23T00:40:50.413 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:48.803889+0000 osd.5 (osd.5) 22 : cluster [DBG] 4.4 scrub ok 2024-02-23T00:40:50.413 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:49.139699+0000 mgr.y (mgr.14517) 58 : cluster [DBG] pgmap v51: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 80 B/s, 3 objects/s recovering 2024-02-23T00:40:50.413 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:49.327816+0000 mon.a (mon.0) 724 : cluster [DBG] osdmap e88: 8 total, 8 up, 8 in 2024-02-23T00:40:50.413 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:49.776646+0000 osd.1 (osd.1) 31 : cluster [DBG] 6.1d scrub starts 2024-02-23T00:40:50.413 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:49.777632+0000 osd.1 (osd.1) 32 : cluster [DBG] 6.1d scrub ok 2024-02-23T00:40:50.413 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:48.695515+0000 osd.0 (osd.0) 19 : cluster [DBG] 4.6 scrub starts 2024-02-23T00:40:50.414 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:48.696891+0000 osd.0 (osd.0) 20 : cluster [DBG] 4.6 scrub ok 2024-02-23T00:40:50.414 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:48.787077+0000 osd.7 (osd.7) 33 : cluster [DBG] 5.4 scrub starts 2024-02-23T00:40:50.414 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:48.789986+0000 osd.7 (osd.7) 34 : cluster [DBG] 5.4 scrub ok 2024-02-23T00:40:50.414 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:48.800090+0000 osd.5 (osd.5) 21 : cluster [DBG] 4.4 scrub starts 2024-02-23T00:40:50.414 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:48.803889+0000 osd.5 (osd.5) 22 : cluster [DBG] 4.4 scrub ok 2024-02-23T00:40:50.414 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:49.139699+0000 mgr.y (mgr.14517) 58 : cluster [DBG] pgmap v51: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 80 B/s, 3 objects/s recovering 2024-02-23T00:40:50.414 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:49.327816+0000 mon.a (mon.0) 724 : cluster [DBG] osdmap e88: 8 total, 8 up, 8 in 2024-02-23T00:40:50.414 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:49.776646+0000 osd.1 (osd.1) 31 : cluster [DBG] 6.1d scrub starts 2024-02-23T00:40:50.415 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:49.777632+0000 osd.1 (osd.1) 32 : cluster [DBG] 6.1d scrub ok 2024-02-23T00:40:50.415 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:40:50 smithi047 systemd[1]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@alertmanager.a.service: Succeeded. 2024-02-23T00:40:50.415 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:40:50 smithi047 systemd[1]: Stopped Ceph alertmanager.a for fce2617e-d1e2-11ee-95c0-87774f69a715. 2024-02-23T00:40:50.415 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:40:50 smithi047 systemd[1]: Starting Ceph alertmanager.a for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T00:40:50.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:48.695515+0000 osd.0 (osd.0) 19 : cluster [DBG] 4.6 scrub starts 2024-02-23T00:40:50.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:48.696891+0000 osd.0 (osd.0) 20 : cluster [DBG] 4.6 scrub ok 2024-02-23T00:40:50.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:48.787077+0000 osd.7 (osd.7) 33 : cluster [DBG] 5.4 scrub starts 2024-02-23T00:40:50.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:48.789986+0000 osd.7 (osd.7) 34 : cluster [DBG] 5.4 scrub ok 2024-02-23T00:40:50.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:48.800090+0000 osd.5 (osd.5) 21 : cluster [DBG] 4.4 scrub starts 2024-02-23T00:40:50.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:48.803889+0000 osd.5 (osd.5) 22 : cluster [DBG] 4.4 scrub ok 2024-02-23T00:40:50.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:49.139699+0000 mgr.y (mgr.14517) 58 : cluster [DBG] pgmap v51: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 80 B/s, 3 objects/s recovering 2024-02-23T00:40:50.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:49.327816+0000 mon.a (mon.0) 724 : cluster [DBG] osdmap e88: 8 total, 8 up, 8 in 2024-02-23T00:40:50.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:49.776646+0000 osd.1 (osd.1) 31 : cluster [DBG] 6.1d scrub starts 2024-02-23T00:40:50.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:49.777632+0000 osd.1 (osd.1) 32 : cluster [DBG] 6.1d scrub ok 2024-02-23T00:40:50.806 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:40:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:40:50] "GET /metrics HTTP/1.1" 200 172136 "" "Prometheus/2.33.4" 2024-02-23T00:40:51.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:49.364444+0000 osd.2 (osd.2) 23 : cluster [DBG] 5.8 deep-scrub starts 2024-02-23T00:40:51.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:49.365704+0000 osd.2 (osd.2) 24 : cluster [DBG] 5.8 deep-scrub ok 2024-02-23T00:40:51.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:49.936576+0000 osd.4 (osd.4) 19 : cluster [DBG] 6.2 scrub starts 2024-02-23T00:40:51.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:49.937521+0000 osd.4 (osd.4) 20 : cluster [DBG] 6.2 scrub ok 2024-02-23T00:40:51.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:50.027967+0000 osd.6 (osd.6) 21 : cluster [DBG] 5.2 scrub starts 2024-02-23T00:40:51.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:50.029081+0000 osd.6 (osd.6) 22 : cluster [DBG] 5.2 scrub ok 2024-02-23T00:40:51.547 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:49.364444+0000 osd.2 (osd.2) 23 : cluster [DBG] 5.8 deep-scrub starts 2024-02-23T00:40:51.547 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:49.365704+0000 osd.2 (osd.2) 24 : cluster [DBG] 5.8 deep-scrub ok 2024-02-23T00:40:51.547 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:49.936576+0000 osd.4 (osd.4) 19 : cluster [DBG] 6.2 scrub starts 2024-02-23T00:40:51.547 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:49.937521+0000 osd.4 (osd.4) 20 : cluster [DBG] 6.2 scrub ok 2024-02-23T00:40:51.547 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:50.027967+0000 osd.6 (osd.6) 21 : cluster [DBG] 5.2 scrub starts 2024-02-23T00:40:51.547 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:50.029081+0000 osd.6 (osd.6) 22 : cluster [DBG] 5.2 scrub ok 2024-02-23T00:40:51.548 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:49.364444+0000 osd.2 (osd.2) 23 : cluster [DBG] 5.8 deep-scrub starts 2024-02-23T00:40:51.548 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:49.365704+0000 osd.2 (osd.2) 24 : cluster [DBG] 5.8 deep-scrub ok 2024-02-23T00:40:51.548 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:49.936576+0000 osd.4 (osd.4) 19 : cluster [DBG] 6.2 scrub starts 2024-02-23T00:40:51.548 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:49.937521+0000 osd.4 (osd.4) 20 : cluster [DBG] 6.2 scrub ok 2024-02-23T00:40:51.548 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:50.027967+0000 osd.6 (osd.6) 21 : cluster [DBG] 5.2 scrub starts 2024-02-23T00:40:51.548 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:50.029081+0000 osd.6 (osd.6) 22 : cluster [DBG] 5.2 scrub ok 2024-02-23T00:40:52.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.239+0000 7f4b4b118700 1 -- 172.21.15.47:0/420244116 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b440fd510 msgr2=0x7f4b440f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:52.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.239+0000 7f4b4b118700 1 --2- 172.21.15.47:0/420244116 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b440fd510 0x7f4b440f6d40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f4b30004870 tx=0x7f4b30009cb0).stop 2024-02-23T00:40:52.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.239+0000 7f4b4b118700 1 -- 172.21.15.47:0/420244116 shutdown_connections 2024-02-23T00:40:52.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.239+0000 7f4b4b118700 1 --2- 172.21.15.47:0/420244116 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b440f7400 0x7f4b44103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:52.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.239+0000 7f4b4b118700 1 --2- 172.21.15.47:0/420244116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b440fcbb0 0x7f4b440fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:52.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.239+0000 7f4b4b118700 1 --2- 172.21.15.47:0/420244116 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b440fd510 0x7f4b440f6d40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:52.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.239+0000 7f4b4b118700 1 -- 172.21.15.47:0/420244116 >> 172.21.15.47:0/420244116 conn(0x7f4b440f2820 msgr2=0x7f4b440f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:40:52.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.239+0000 7f4b4b118700 1 -- 172.21.15.47:0/420244116 shutdown_connections 2024-02-23T00:40:52.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.239+0000 7f4b4b118700 1 -- 172.21.15.47:0/420244116 wait complete. 2024-02-23T00:40:52.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.239+0000 7f4b4b118700 1 librados: starting msgr at 2024-02-23T00:40:52.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.239+0000 7f4b4b118700 1 librados: starting objecter 2024-02-23T00:40:52.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.239+0000 7f4b4b118700 1 Processor -- start 2024-02-23T00:40:52.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.240+0000 7f4b4b118700 1 -- start start 2024-02-23T00:40:52.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.240+0000 7f4b4b118700 1 librados: setting wanted keys 2024-02-23T00:40:52.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.240+0000 7f4b4b118700 1 librados: calling monclient init 2024-02-23T00:40:52.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.240+0000 7f4b4b118700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b440f7400 0x7f4b440f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:52.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.240+0000 7f4b4b118700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b440fcbb0 0x7f4b440f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:52.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.240+0000 7f4b4b118700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b440fd510 0x7f4b440f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:52.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.240+0000 7f4b4b118700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b441164c0 con 0x7f4b440fcbb0 2024-02-23T00:40:52.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.240+0000 7f4b4b118700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b44116630 con 0x7f4b440f7400 2024-02-23T00:40:52.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.240+0000 7f4b4b118700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b441167f0 con 0x7f4b440fd510 2024-02-23T00:40:52.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.240+0000 7f4b496b5700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b440fd510 0x7f4b440f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:52.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.240+0000 7f4b496b5700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b440fd510 0x7f4b440f9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:55894/0 (socket says 172.21.15.47:55894) 2024-02-23T00:40:52.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.240+0000 7f4b496b5700 1 -- 172.21.15.47:0/4112735267 learned_addr learned my addr 172.21.15.47:0/4112735267 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:40:52.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.240+0000 7f4b43fff700 1 --2- 172.21.15.47:0/4112735267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b440fcbb0 0x7f4b440f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:52.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.240+0000 7f4b48eb4700 1 --2- 172.21.15.47:0/4112735267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b440f7400 0x7f4b440f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:52.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.240+0000 7f4b496b5700 1 -- 172.21.15.47:0/4112735267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b440f7400 msgr2=0x7f4b440f89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.240+0000 7f4b496b5700 1 --2- 172.21.15.47:0/4112735267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b440f7400 0x7f4b440f89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.240+0000 7f4b496b5700 1 -- 172.21.15.47:0/4112735267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b440fcbb0 msgr2=0x7f4b440f8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.240+0000 7f4b496b5700 1 --2- 172.21.15.47:0/4112735267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b440fcbb0 0x7f4b440f8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.240+0000 7f4b496b5700 1 -- 172.21.15.47:0/4112735267 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b3c0097c0 con 0x7f4b440fd510 2024-02-23T00:40:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.240+0000 7f4b43fff700 1 --2- 172.21.15.47:0/4112735267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b440fcbb0 0x7f4b440f8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:40:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.240+0000 7f4b496b5700 1 --2- 172.21.15.47:0/4112735267 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b440fd510 0x7f4b440f9390 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f4b3c001df0 tx=0x7f4b3c004c50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.240+0000 7f4b41ffb700 10 client.?.objecter ms_handle_connect 0x7f4b440fd510 2024-02-23T00:40:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.240+0000 7f4b41ffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:40:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.240+0000 7f4b41ffb700 1 -- 172.21.15.47:0/4112735267 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c009a10 con 0x7f4b440fd510 2024-02-23T00:40:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.241+0000 7f4b41ffb700 1 -- 172.21.15.47:0/4112735267 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f4b3c0090d0 con 0x7f4b440fd510 2024-02-23T00:40:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.241+0000 7f4b4b118700 1 -- 172.21.15.47:0/4112735267 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b30005040 con 0x7f4b440fd510 2024-02-23T00:40:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.241+0000 7f4b41ffb700 1 -- 172.21.15.47:0/4112735267 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c007be0 con 0x7f4b440fd510 2024-02-23T00:40:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.241+0000 7f4b4b118700 10 client.24536.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:40:52.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.241+0000 7f4b4b118700 1 -- 172.21.15.47:0/4112735267 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b44116db0 con 0x7f4b440fd510 2024-02-23T00:40:52.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.241+0000 7f4b4b118700 1 librados: init done 2024-02-23T00:40:52.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.241+0000 7f4b4b118700 10 client.24536.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:40:52.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.241+0000 7f4b4b118700 15 client.24536.objecter _session_command_op_assign -1 1 2024-02-23T00:40:52.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.241+0000 7f4b4b118700 20 client.24536.objecter _get_session osd=-1 returning homeless 2024-02-23T00:40:52.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.241+0000 7f4b4b118700 10 client.24536.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:40:52.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.241+0000 7f4b4b118700 1 -- 172.21.15.47:0/4112735267 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4b4405fa60 con 0x7f4b440fd510 2024-02-23T00:40:52.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.242+0000 7f4b41ffb700 1 -- 172.21.15.47:0/4112735267 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f4b3c009240 con 0x7f4b440fd510 2024-02-23T00:40:52.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.242+0000 7f4b41ffb700 1 --2- 172.21.15.47:0/4112735267 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4b34062cc0 0x7f4b34065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:52.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.242+0000 7f4b41ffb700 1 -- 172.21.15.47:0/4112735267 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 7421+0+0 (secure 0 0 0) 0x7f4b3c07d860 con 0x7f4b440fd510 2024-02-23T00:40:52.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.242+0000 7f4b41ffb700 10 client.24536.objecter ms_dispatch 0x7f4b44001550 osd_map(88..88 src has 1..88) v4 2024-02-23T00:40:52.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.242+0000 7f4b41ffb700 3 client.24536.objecter handle_osd_map got epochs [88,88] > 0 2024-02-23T00:40:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.242+0000 7f4b41ffb700 3 client.24536.objecter handle_osd_map decoding full epoch 88 2024-02-23T00:40:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.242+0000 7f4b48eb4700 1 --2- 172.21.15.47:0/4112735267 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4b34062cc0 0x7f4b34065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.243+0000 7f4b41ffb700 10 client.24536.objecter checking command 1 2024-02-23T00:40:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.243+0000 7f4b41ffb700 1 --2- 172.21.15.47:0/4112735267 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f4b3406cbd0 0x7f4b3406f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.243+0000 7f4b41ffb700 20 client.24536.objecter _get_session s=0x7f4b3406c500 osd=5 3 2024-02-23T00:40:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.243+0000 7f4b41ffb700 20 client.24536.objecter put_session s=0x7f4b3406c500 osd=5 3 2024-02-23T00:40:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.243+0000 7f4b41ffb700 15 client.24536.objecter _session_command_op_remove -1 1 2024-02-23T00:40:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.243+0000 7f4b41ffb700 20 client.24536.objecter _get_session s=0x7f4b3406c500 osd=5 3 2024-02-23T00:40:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.243+0000 7f4b41ffb700 20 client.24536.objecter get_session s=0x7f4b3406c500 osd=5 3 2024-02-23T00:40:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.243+0000 7f4b41ffb700 15 client.24536.objecter _session_command_op_assign 5 1 2024-02-23T00:40:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.243+0000 7f4b41ffb700 20 client.24536.objecter put_session s=0x7f4b3406c500 osd=5 4 2024-02-23T00:40:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.243+0000 7f4b41ffb700 10 client.24536.objecter _send_command 1 2024-02-23T00:40:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.243+0000 7f4b41ffb700 1 -- 172.21.15.47:0/4112735267 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4b3406f6f0 con 0x7f4b3406cbd0 2024-02-23T00:40:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.243+0000 7f4b41ffb700 20 client.24536.objecter dump_active .. 0 homeless 2024-02-23T00:40:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.243+0000 7f4b41ffb700 1 -- 172.21.15.47:0/4112735267 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b3c082220 con 0x7f4b440fd510 2024-02-23T00:40:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.243+0000 7f4b41ffb700 10 client.24536.objecter ms_handle_connect 0x7f4b34062cc0 2024-02-23T00:40:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.243+0000 7f4b48eb4700 1 --2- 172.21.15.47:0/4112735267 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4b34062cc0 0x7f4b34065180 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f4b38004e60 tx=0x7f4b3800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.243+0000 7f4b43fff700 1 --2- 172.21.15.47:0/4112735267 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f4b3406cbd0 0x7f4b3406f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.243+0000 7f4b41ffb700 10 client.24536.objecter ms_handle_connect 0x7f4b3406cbd0 2024-02-23T00:40:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.243+0000 7f4b43fff700 1 --2- 172.21.15.47:0/4112735267 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f4b3406cbd0 0x7f4b3406f070 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.244+0000 7f4b41ffb700 1 -- 172.21.15.47:0/4112735267 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4b3406f6f0 con 0x7f4b3406cbd0 2024-02-23T00:40:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.244+0000 7f4b41ffb700 10 client.24536.objecter ms_dispatch 0x7f4b44001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:40:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.244+0000 7f4b41ffb700 10 client.24536.objecter _finish_command 1 = system:0 2024-02-23T00:40:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.244+0000 7f4b41ffb700 20 client.24536.objecter put_session s=0x7f4b3406c500 osd=5 4 2024-02-23T00:40:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.244+0000 7f4b41ffb700 15 client.24536.objecter _session_command_op_remove 5 1 2024-02-23T00:40:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.258+0000 7f4b4b118700 10 client.24536.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-02-23T00:40:52.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.258+0000 7f4b4b118700 15 client.24536.objecter _session_command_op_assign -1 2 2024-02-23T00:40:52.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.258+0000 7f4b4b118700 20 client.24536.objecter _get_session s=0x7f4b3406c500 osd=5 3 2024-02-23T00:40:52.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.258+0000 7f4b4b118700 20 client.24536.objecter put_session s=0x7f4b3406c500 osd=5 3 2024-02-23T00:40:52.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.258+0000 7f4b4b118700 20 client.24536.objecter _get_session s=0x7f4b3406c500 osd=5 3 2024-02-23T00:40:52.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.258+0000 7f4b4b118700 15 client.24536.objecter _session_command_op_remove -1 2 2024-02-23T00:40:52.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.258+0000 7f4b4b118700 20 client.24536.objecter get_session s=0x7f4b3406c500 osd=5 3 2024-02-23T00:40:52.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.258+0000 7f4b4b118700 15 client.24536.objecter _session_command_op_assign 5 2 2024-02-23T00:40:52.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.258+0000 7f4b4b118700 20 client.24536.objecter put_session s=0x7f4b3406c500 osd=5 4 2024-02-23T00:40:52.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.258+0000 7f4b4b118700 10 client.24536.objecter _send_command 2 2024-02-23T00:40:52.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.258+0000 7f4b4b118700 1 -- 172.21.15.47:0/4112735267 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4b4405ee80 con 0x7f4b3406cbd0 2024-02-23T00:40:52.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.259+0000 7f4b41ffb700 1 -- 172.21.15.47:0/4112735267 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4b4405ee80 con 0x7f4b3406cbd0 2024-02-23T00:40:52.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.259+0000 7f4b41ffb700 10 client.24536.objecter ms_dispatch 0x7f4b44001550 command_reply(tid 2: 0 ) v1 2024-02-23T00:40:52.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.259+0000 7f4b41ffb700 10 client.24536.objecter _finish_command 2 = system:0 2024-02-23T00:40:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.259+0000 7f4b41ffb700 20 client.24536.objecter put_session s=0x7f4b3406c500 osd=5 4 2024-02-23T00:40:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.259+0000 7f4b41ffb700 15 client.24536.objecter _session_command_op_remove 5 2 2024-02-23T00:40:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.259+0000 7f4b4b118700 10 librados: watch_flush enter 2024-02-23T00:40:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.259+0000 7f4b4b118700 10 librados: watch_flush exit 2024-02-23T00:40:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.259+0000 7f4b4b118700 10 client.24536.objecter close_session for osd.5 2024-02-23T00:40:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.259+0000 7f4b4b118700 1 -- 172.21.15.47:0/4112735267 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f4b3406cbd0 msgr2=0x7f4b3406f070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.259+0000 7f4b4b118700 1 --2- 172.21.15.47:0/4112735267 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f4b3406cbd0 0x7f4b3406f070 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.259+0000 7f4b4b118700 20 client.24536.objecter put_session s=0x7f4b3406c500 osd=5 1 2024-02-23T00:40:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.259+0000 7f4b4b118700 20 client.24536.objecter shutdown clearing up homeless session... 2024-02-23T00:40:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.259+0000 7f4b4b118700 10 client.24536.objecter successfully canceled tick 2024-02-23T00:40:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.259+0000 7f4b4b118700 1 -- 172.21.15.47:0/4112735267 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4b34062cc0 msgr2=0x7f4b34065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.259+0000 7f4b4b118700 1 --2- 172.21.15.47:0/4112735267 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4b34062cc0 0x7f4b34065180 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f4b38004e60 tx=0x7f4b3800b040).stop 2024-02-23T00:40:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.259+0000 7f4b4b118700 1 -- 172.21.15.47:0/4112735267 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b440fd510 msgr2=0x7f4b440f9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.259+0000 7f4b4b118700 1 --2- 172.21.15.47:0/4112735267 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b440fd510 0x7f4b440f9390 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f4b3c001df0 tx=0x7f4b3c004c50).stop 2024-02-23T00:40:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.259+0000 7f4b496b5700 1 -- 172.21.15.47:0/4112735267 reap_dead start 2024-02-23T00:40:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.259+0000 7f4b4b118700 1 -- 172.21.15.47:0/4112735267 shutdown_connections 2024-02-23T00:40:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.259+0000 7f4b4b118700 1 -- 172.21.15.47:0/4112735267 >> 172.21.15.47:0/4112735267 conn(0x7f4b440f2820 msgr2=0x7f4b440f4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:40:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.259+0000 7f4b4b118700 1 -- 172.21.15.47:0/4112735267 shutdown_connections 2024-02-23T00:40:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.259+0000 7f4b4b118700 1 -- 172.21.15.47:0/4112735267 wait complete. 2024-02-23T00:40:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:52.259+0000 7f4b4b118700 1 librados: shutdown 2024-02-23T00:40:52.443 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:40:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:40:52.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:50.758931+0000 osd.5 (osd.5) 23 : cluster [DBG] 2.3 scrub starts 2024-02-23T00:40:52.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:50.760008+0000 osd.5 (osd.5) 24 : cluster [DBG] 2.3 scrub ok 2024-02-23T00:40:52.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:51.140143+0000 mgr.y (mgr.14517) 59 : cluster [DBG] pgmap v53: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 6.1 KiB/s rd, 511 B/s wr, 12 op/s; 72 B/s, 3 objects/s recovering 2024-02-23T00:40:52.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:50.758931+0000 osd.5 (osd.5) 23 : cluster [DBG] 2.3 scrub starts 2024-02-23T00:40:52.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:50.760008+0000 osd.5 (osd.5) 24 : cluster [DBG] 2.3 scrub ok 2024-02-23T00:40:52.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:51.140143+0000 mgr.y (mgr.14517) 59 : cluster [DBG] pgmap v53: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 6.1 KiB/s rd, 511 B/s wr, 12 op/s; 72 B/s, 3 objects/s recovering 2024-02-23T00:40:52.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:50.758931+0000 osd.5 (osd.5) 23 : cluster [DBG] 2.3 scrub starts 2024-02-23T00:40:52.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:50.760008+0000 osd.5 (osd.5) 24 : cluster [DBG] 2.3 scrub ok 2024-02-23T00:40:52.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:51.140143+0000 mgr.y (mgr.14517) 59 : cluster [DBG] pgmap v53: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 6.1 KiB/s rd, 511 B/s wr, 12 op/s; 72 B/s, 3 objects/s recovering 2024-02-23T00:40:52.557 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:40:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=info ts=2024-02-23T00:40:52.279Z caller=main.go:225 msg="Starting Alertmanager" version="(version=0.23.0, branch=HEAD, revision=61046b17771a57cfd4c4a51be370ab930a4d7d54)" 2024-02-23T00:40:52.558 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:40:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=info ts=2024-02-23T00:40:52.279Z caller=main.go:226 build_context="(go=go1.16.7, user=root@e21a959be8d2, date=20210825-10:48:55)" 2024-02-23T00:40:52.558 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:40:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=info ts=2024-02-23T00:40:52.280Z caller=cluster.go:184 component=cluster msg="setting advertise address explicitly" addr=172.21.15.47 port=9094 2024-02-23T00:40:52.558 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:40:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=info ts=2024-02-23T00:40:52.281Z caller=cluster.go:671 component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-02-23T00:40:52.558 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:40:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=info ts=2024-02-23T00:40:52.307Z caller=coordinator.go:113 component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-23T00:40:52.558 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:40:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=info ts=2024-02-23T00:40:52.307Z caller=coordinator.go:126 component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-23T00:40:52.558 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:40:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=info ts=2024-02-23T00:40:52.310Z caller=main.go:518 msg=Listening address=:9093 2024-02-23T00:40:52.558 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:40:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=info ts=2024-02-23T00:40:52.310Z caller=tls_config.go:191 msg="TLS is disabled." http2=false 2024-02-23T00:40:52.943 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:40:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:40:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:40:53.157 INFO:teuthology.orchestra.run.smithi047.stdout:167503724579 2024-02-23T00:40:53.157 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-02-23T00:40:53.410 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:40:53 smithi047 bash[71458]: c1d3511b69402900a777e24aca899d21ad68b01e82b5a0c5aa4ea5d20dba150f 2024-02-23T00:40:53.410 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:40:53 smithi047 systemd[1]: Started Ceph alertmanager.a for fce2617e-d1e2-11ee-95c0-87774f69a715. 2024-02-23T00:40:53.411 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:51.774028+0000 osd.7 (osd.7) 35 : cluster [DBG] 6.8 scrub starts 2024-02-23T00:40:53.411 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:51.777395+0000 osd.7 (osd.7) 36 : cluster [DBG] 6.8 scrub ok 2024-02-23T00:40:53.411 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:51.803087+0000 osd.5 (osd.5) 25 : cluster [DBG] 6.7 scrub starts 2024-02-23T00:40:53.411 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:51.806001+0000 osd.5 (osd.5) 26 : cluster [DBG] 6.7 scrub ok 2024-02-23T00:40:53.411 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:51.948516+0000 osd.4 (osd.4) 21 : cluster [DBG] 4.1a scrub starts 2024-02-23T00:40:53.411 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:51.950461+0000 osd.4 (osd.4) 22 : cluster [DBG] 4.1a scrub ok 2024-02-23T00:40:53.411 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:52.165645+0000 mgr.y (mgr.14517) 60 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:40:53.411 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:52.779632+0000 osd.1 (osd.1) 33 : cluster [DBG] 4.12 scrub starts 2024-02-23T00:40:53.411 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:52.783136+0000 osd.1 (osd.1) 34 : cluster [DBG] 4.12 scrub ok 2024-02-23T00:40:53.411 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:52.833540+0000 osd.3 (osd.3) 21 : cluster [DBG] 4.19 deep-scrub starts 2024-02-23T00:40:53.411 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:52.840698+0000 osd.3 (osd.3) 22 : cluster [DBG] 4.19 deep-scrub ok 2024-02-23T00:40:53.412 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:51.774028+0000 osd.7 (osd.7) 35 : cluster [DBG] 6.8 scrub starts 2024-02-23T00:40:53.412 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:51.777395+0000 osd.7 (osd.7) 36 : cluster [DBG] 6.8 scrub ok 2024-02-23T00:40:53.412 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:51.803087+0000 osd.5 (osd.5) 25 : cluster [DBG] 6.7 scrub starts 2024-02-23T00:40:53.412 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:51.806001+0000 osd.5 (osd.5) 26 : cluster [DBG] 6.7 scrub ok 2024-02-23T00:40:53.412 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:51.948516+0000 osd.4 (osd.4) 21 : cluster [DBG] 4.1a scrub starts 2024-02-23T00:40:53.412 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:51.950461+0000 osd.4 (osd.4) 22 : cluster [DBG] 4.1a scrub ok 2024-02-23T00:40:53.412 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:52.165645+0000 mgr.y (mgr.14517) 60 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:40:53.412 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:52.779632+0000 osd.1 (osd.1) 33 : cluster [DBG] 4.12 scrub starts 2024-02-23T00:40:53.412 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:52.783136+0000 osd.1 (osd.1) 34 : cluster [DBG] 4.12 scrub ok 2024-02-23T00:40:53.412 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:52.833540+0000 osd.3 (osd.3) 21 : cluster [DBG] 4.19 deep-scrub starts 2024-02-23T00:40:53.412 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:52.840698+0000 osd.3 (osd.3) 22 : cluster [DBG] 4.19 deep-scrub ok 2024-02-23T00:40:53.413 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:40:53] ENGINE Bus STOPPING 2024-02-23T00:40:53.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:51.774028+0000 osd.7 (osd.7) 35 : cluster [DBG] 6.8 scrub starts 2024-02-23T00:40:53.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:51.777395+0000 osd.7 (osd.7) 36 : cluster [DBG] 6.8 scrub ok 2024-02-23T00:40:53.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:51.803087+0000 osd.5 (osd.5) 25 : cluster [DBG] 6.7 scrub starts 2024-02-23T00:40:53.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:51.806001+0000 osd.5 (osd.5) 26 : cluster [DBG] 6.7 scrub ok 2024-02-23T00:40:53.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:51.948516+0000 osd.4 (osd.4) 21 : cluster [DBG] 4.1a scrub starts 2024-02-23T00:40:53.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:51.950461+0000 osd.4 (osd.4) 22 : cluster [DBG] 4.1a scrub ok 2024-02-23T00:40:53.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:52.165645+0000 mgr.y (mgr.14517) 60 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:40:53.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:52.779632+0000 osd.1 (osd.1) 33 : cluster [DBG] 4.12 scrub starts 2024-02-23T00:40:53.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:52.783136+0000 osd.1 (osd.1) 34 : cluster [DBG] 4.12 scrub ok 2024-02-23T00:40:53.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:52.833540+0000 osd.3 (osd.3) 21 : cluster [DBG] 4.19 deep-scrub starts 2024-02-23T00:40:53.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:52.840698+0000 osd.3 (osd.3) 22 : cluster [DBG] 4.19 deep-scrub ok 2024-02-23T00:40:53.451 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:40:53.747 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:40:53] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-23T00:40:53.748 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:40:53] ENGINE Bus STOPPED 2024-02-23T00:40:53.748 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:40:53] ENGINE Bus STARTING 2024-02-23T00:40:53.748 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:40:53] ENGINE Serving on http://:::9283 2024-02-23T00:40:53.748 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:40:53] ENGINE Bus STARTED 2024-02-23T00:40:53.748 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:40:53] ENGINE Bus STOPPING 2024-02-23T00:40:53.748 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:40:53] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-23T00:40:53.748 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:40:53] ENGINE Bus STOPPED 2024-02-23T00:40:53.748 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:40:53] ENGINE Bus STARTING 2024-02-23T00:40:53.748 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:40:53] ENGINE Serving on http://:::9283 2024-02-23T00:40:53.748 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:40:53] ENGINE Bus STARTED 2024-02-23T00:40:53.748 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:40:53] ENGINE Bus STOPPING 2024-02-23T00:40:53.748 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:40:53] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-23T00:40:53.748 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:40:53] ENGINE Bus STOPPED 2024-02-23T00:40:53.748 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:40:53] ENGINE Bus STARTING 2024-02-23T00:40:54.002 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:40:53] ENGINE Serving on http://:::9283 2024-02-23T00:40:54.002 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:40:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:40:53] ENGINE Bus STARTED 2024-02-23T00:40:54.284 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:52.749401+0000 osd.0 (osd.0) 21 : cluster [DBG] 4.3 scrub starts 2024-02-23T00:40:54.284 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:52.751592+0000 osd.0 (osd.0) 22 : cluster [DBG] 4.3 scrub ok 2024-02-23T00:40:54.284 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:53.111987+0000 osd.6 (osd.6) 23 : cluster [DBG] 2.11 scrub starts 2024-02-23T00:40:54.284 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:53.113228+0000 osd.6 (osd.6) 24 : cluster [DBG] 2.11 scrub ok 2024-02-23T00:40:54.284 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:53.140760+0000 mgr.y (mgr.14517) 61 : cluster [DBG] pgmap v54: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 6.7 KiB/s rd, 511 B/s wr, 12 op/s; 72 B/s, 3 objects/s recovering 2024-02-23T00:40:54.284 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:53.392621+0000 mon.a (mon.0) 725 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:54.284 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:53.396109+0000 mon.a (mon.0) 726 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-23T00:40:54.284 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:53.396472+0000 mgr.y (mgr.14517) 62 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-23T00:40:54.284 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:53.397248+0000 mon.a (mon.0) 727 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi047.front.sepia.ceph.com:9093"}]: dispatch 2024-02-23T00:40:54.284 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:53.397478+0000 mgr.y (mgr.14517) 63 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi047.front.sepia.ceph.com:9093"}]: dispatch 2024-02-23T00:40:54.285 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:53.402186+0000 mon.a (mon.0) 728 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:54.285 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:53.406175+0000 mon.a (mon.0) 729 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-23T00:40:54.285 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:53.406603+0000 mgr.y (mgr.14517) 64 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-23T00:40:54.285 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:53.413238+0000 mon.a (mon.0) 730 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi196.front.sepia.ceph.com:9095"}]: dispatch 2024-02-23T00:40:54.285 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:53.413594+0000 mgr.y (mgr.14517) 65 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi196.front.sepia.ceph.com:9095"}]: dispatch 2024-02-23T00:40:54.285 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:53.419394+0000 mon.a (mon.0) 731 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:54.285 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:53.423808+0000 mon.a (mon.0) 732 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-23T00:40:54.285 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:53.424199+0000 mgr.y (mgr.14517) 66 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-23T00:40:54.285 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:53.429175+0000 mon.a (mon.0) 733 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi196.front.sepia.ceph.com:3000"}]: dispatch 2024-02-23T00:40:54.285 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:53.429686+0000 mgr.y (mgr.14517) 67 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi196.front.sepia.ceph.com:3000"}]: dispatch 2024-02-23T00:40:54.285 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:53.434446+0000 mon.a (mon.0) 734 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:54.285 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:53.444579+0000 mon.a (mon.0) 735 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:40:54.285 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:53.809879+0000 osd.1 (osd.1) 35 : cluster [DBG] 5.19 scrub starts 2024-02-23T00:40:54.286 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:53.812889+0000 osd.1 (osd.1) 36 : cluster [DBG] 5.19 scrub ok 2024-02-23T00:40:54.286 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:53.823451+0000 osd.5 (osd.5) 27 : cluster [DBG] 2.2 scrub starts 2024-02-23T00:40:54.286 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:53.824157+0000 osd.5 (osd.5) 28 : cluster [DBG] 2.2 scrub ok 2024-02-23T00:40:54.286 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:52.749401+0000 osd.0 (osd.0) 21 : cluster [DBG] 4.3 scrub starts 2024-02-23T00:40:54.286 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:52.751592+0000 osd.0 (osd.0) 22 : cluster [DBG] 4.3 scrub ok 2024-02-23T00:40:54.286 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:53.111987+0000 osd.6 (osd.6) 23 : cluster [DBG] 2.11 scrub starts 2024-02-23T00:40:54.286 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:53.113228+0000 osd.6 (osd.6) 24 : cluster [DBG] 2.11 scrub ok 2024-02-23T00:40:54.286 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:53.140760+0000 mgr.y (mgr.14517) 61 : cluster [DBG] pgmap v54: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 6.7 KiB/s rd, 511 B/s wr, 12 op/s; 72 B/s, 3 objects/s recovering 2024-02-23T00:40:54.286 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:53.392621+0000 mon.a (mon.0) 725 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:54.287 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:53.396109+0000 mon.a (mon.0) 726 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-23T00:40:54.287 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:53.396472+0000 mgr.y (mgr.14517) 62 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-23T00:40:54.287 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:53.397248+0000 mon.a (mon.0) 727 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi047.front.sepia.ceph.com:9093"}]: dispatch 2024-02-23T00:40:54.287 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:53.397478+0000 mgr.y (mgr.14517) 63 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi047.front.sepia.ceph.com:9093"}]: dispatch 2024-02-23T00:40:54.287 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:53.402186+0000 mon.a (mon.0) 728 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:54.287 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:53.406175+0000 mon.a (mon.0) 729 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-23T00:40:54.287 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:53.406603+0000 mgr.y (mgr.14517) 64 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-23T00:40:54.287 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:53.413238+0000 mon.a (mon.0) 730 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi196.front.sepia.ceph.com:9095"}]: dispatch 2024-02-23T00:40:54.287 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:53.413594+0000 mgr.y (mgr.14517) 65 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi196.front.sepia.ceph.com:9095"}]: dispatch 2024-02-23T00:40:54.287 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:53.419394+0000 mon.a (mon.0) 731 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:54.287 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:53.423808+0000 mon.a (mon.0) 732 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-23T00:40:54.287 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:53.424199+0000 mgr.y (mgr.14517) 66 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-23T00:40:54.287 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:53.429175+0000 mon.a (mon.0) 733 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi196.front.sepia.ceph.com:3000"}]: dispatch 2024-02-23T00:40:54.288 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:53.429686+0000 mgr.y (mgr.14517) 67 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi196.front.sepia.ceph.com:3000"}]: dispatch 2024-02-23T00:40:54.288 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:53.434446+0000 mon.a (mon.0) 734 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:54.288 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:53.444579+0000 mon.a (mon.0) 735 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:40:54.288 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:53.809879+0000 osd.1 (osd.1) 35 : cluster [DBG] 5.19 scrub starts 2024-02-23T00:40:54.288 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:53.812889+0000 osd.1 (osd.1) 36 : cluster [DBG] 5.19 scrub ok 2024-02-23T00:40:54.288 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:53.823451+0000 osd.5 (osd.5) 27 : cluster [DBG] 2.2 scrub starts 2024-02-23T00:40:54.288 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:53.824157+0000 osd.5 (osd.5) 28 : cluster [DBG] 2.2 scrub ok 2024-02-23T00:40:54.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:52.749401+0000 osd.0 (osd.0) 21 : cluster [DBG] 4.3 scrub starts 2024-02-23T00:40:54.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:52.751592+0000 osd.0 (osd.0) 22 : cluster [DBG] 4.3 scrub ok 2024-02-23T00:40:54.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:53.111987+0000 osd.6 (osd.6) 23 : cluster [DBG] 2.11 scrub starts 2024-02-23T00:40:54.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:53.113228+0000 osd.6 (osd.6) 24 : cluster [DBG] 2.11 scrub ok 2024-02-23T00:40:54.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:53.140760+0000 mgr.y (mgr.14517) 61 : cluster [DBG] pgmap v54: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 6.7 KiB/s rd, 511 B/s wr, 12 op/s; 72 B/s, 3 objects/s recovering 2024-02-23T00:40:54.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:53.392621+0000 mon.a (mon.0) 725 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:54.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:53.396109+0000 mon.a (mon.0) 726 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-23T00:40:54.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:53.396472+0000 mgr.y (mgr.14517) 62 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-23T00:40:54.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:53.397248+0000 mon.a (mon.0) 727 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi047.front.sepia.ceph.com:9093"}]: dispatch 2024-02-23T00:40:54.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:53.397478+0000 mgr.y (mgr.14517) 63 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi047.front.sepia.ceph.com:9093"}]: dispatch 2024-02-23T00:40:54.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:53.402186+0000 mon.a (mon.0) 728 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:54.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:53.406175+0000 mon.a (mon.0) 729 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-23T00:40:54.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:53.406603+0000 mgr.y (mgr.14517) 64 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-23T00:40:54.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:53.413238+0000 mon.a (mon.0) 730 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi196.front.sepia.ceph.com:9095"}]: dispatch 2024-02-23T00:40:54.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:53.413594+0000 mgr.y (mgr.14517) 65 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi196.front.sepia.ceph.com:9095"}]: dispatch 2024-02-23T00:40:54.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:53.419394+0000 mon.a (mon.0) 731 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:54.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:53.423808+0000 mon.a (mon.0) 732 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-23T00:40:54.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:53.424199+0000 mgr.y (mgr.14517) 66 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-23T00:40:54.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:53.429175+0000 mon.a (mon.0) 733 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi196.front.sepia.ceph.com:3000"}]: dispatch 2024-02-23T00:40:54.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:53.429686+0000 mgr.y (mgr.14517) 67 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi196.front.sepia.ceph.com:3000"}]: dispatch 2024-02-23T00:40:54.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:53.434446+0000 mon.a (mon.0) 734 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:54.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:53.444579+0000 mon.a (mon.0) 735 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:40:54.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:53.809879+0000 osd.1 (osd.1) 35 : cluster [DBG] 5.19 scrub starts 2024-02-23T00:40:54.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:53.812889+0000 osd.1 (osd.1) 36 : cluster [DBG] 5.19 scrub ok 2024-02-23T00:40:54.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:53.823451+0000 osd.5 (osd.5) 27 : cluster [DBG] 2.2 scrub starts 2024-02-23T00:40:54.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:53.824157+0000 osd.5 (osd.5) 28 : cluster [DBG] 2.2 scrub ok 2024-02-23T00:40:54.543 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:40:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=info ts=2024-02-23T00:40:54.282Z caller=cluster.go:696 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000574067s 2024-02-23T00:40:55.213 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:53.767156+0000 osd.7 (osd.7) 37 : cluster [DBG] 1.0 deep-scrub starts 2024-02-23T00:40:55.213 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:53.770075+0000 osd.7 (osd.7) 38 : cluster [DBG] 1.0 deep-scrub ok 2024-02-23T00:40:55.213 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:54.143911+0000 osd.6 (osd.6) 25 : cluster [DBG] 4.a deep-scrub starts 2024-02-23T00:40:55.213 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:54.152233+0000 osd.6 (osd.6) 26 : cluster [DBG] 4.a deep-scrub ok 2024-02-23T00:40:55.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:53.767156+0000 osd.7 (osd.7) 37 : cluster [DBG] 1.0 deep-scrub starts 2024-02-23T00:40:55.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:53.770075+0000 osd.7 (osd.7) 38 : cluster [DBG] 1.0 deep-scrub ok 2024-02-23T00:40:55.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:54.143911+0000 osd.6 (osd.6) 25 : cluster [DBG] 4.a deep-scrub starts 2024-02-23T00:40:55.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:54.152233+0000 osd.6 (osd.6) 26 : cluster [DBG] 4.a deep-scrub ok 2024-02-23T00:40:55.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.431+0000 7f5f066a5700 1 -- 172.21.15.47:0/2106108493 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5f00102210 msgr2=0x7f5f00104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:55.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.431+0000 7f5f066a5700 1 --2- 172.21.15.47:0/2106108493 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5f00102210 0x7f5f00104600 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f5ef8009c30 tx=0x7f5ef8004c50).stop 2024-02-23T00:40:55.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.431+0000 7f5f066a5700 1 -- 172.21.15.47:0/2106108493 shutdown_connections 2024-02-23T00:40:55.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.431+0000 7f5f066a5700 1 --2- 172.21.15.47:0/2106108493 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5f00102210 0x7f5f00104600 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:55.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.431+0000 7f5f066a5700 1 --2- 172.21.15.47:0/2106108493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f000f8cb0 0x7f5f000f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:55.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.431+0000 7f5f066a5700 1 --2- 172.21.15.47:0/2106108493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f000f9610 0x7f5f00101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:55.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.431+0000 7f5f066a5700 1 -- 172.21.15.47:0/2106108493 >> 172.21.15.47:0/2106108493 conn(0x7f5f000f2820 msgr2=0x7f5f000f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:40:55.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.431+0000 7f5f066a5700 1 -- 172.21.15.47:0/2106108493 shutdown_connections 2024-02-23T00:40:55.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.431+0000 7f5f066a5700 1 -- 172.21.15.47:0/2106108493 wait complete. 2024-02-23T00:40:55.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.432+0000 7f5f066a5700 1 librados: starting msgr at 2024-02-23T00:40:55.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.432+0000 7f5f066a5700 1 librados: starting objecter 2024-02-23T00:40:55.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.432+0000 7f5f066a5700 1 Processor -- start 2024-02-23T00:40:55.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.432+0000 7f5f066a5700 1 -- start start 2024-02-23T00:40:55.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.432+0000 7f5f066a5700 1 librados: setting wanted keys 2024-02-23T00:40:55.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.432+0000 7f5f066a5700 1 librados: calling monclient init 2024-02-23T00:40:55.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.432+0000 7f5f066a5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f000f8cb0 0x7f5f000f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:55.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.432+0000 7f5f066a5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f000f9610 0x7f5f000f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:55.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.432+0000 7f5f066a5700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5f00102210 0x7f5f000f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:55.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.432+0000 7f5f066a5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f00116430 con 0x7f5f000f9610 2024-02-23T00:40:55.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.432+0000 7f5f066a5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f001165a0 con 0x7f5f000f8cb0 2024-02-23T00:40:55.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.432+0000 7f5f066a5700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f00116760 con 0x7f5f00102210 2024-02-23T00:40:55.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.432+0000 7f5eff7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f000f9610 0x7f5f000f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:55.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.432+0000 7f5f04c42700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5f00102210 0x7f5f000f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:55.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.432+0000 7f5eff7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f000f9610 0x7f5f000f7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34192/0 (socket says 172.21.15.47:34192) 2024-02-23T00:40:55.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.432+0000 7f5eff7fe700 1 -- 172.21.15.47:0/208050470 learned_addr learned my addr 172.21.15.47:0/208050470 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:40:55.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.432+0000 7f5efffff700 1 --2- 172.21.15.47:0/208050470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f000f8cb0 0x7f5f000f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:55.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.433+0000 7f5f04c42700 1 -- 172.21.15.47:0/208050470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f000f8cb0 msgr2=0x7f5f000f6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:55.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.433+0000 7f5f04c42700 1 --2- 172.21.15.47:0/208050470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f000f8cb0 0x7f5f000f6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:55.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.433+0000 7f5f04c42700 1 -- 172.21.15.47:0/208050470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f000f9610 msgr2=0x7f5f000f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:55.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.433+0000 7f5f04c42700 1 --2- 172.21.15.47:0/208050470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f000f9610 0x7f5f000f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:55.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.433+0000 7f5f04c42700 1 -- 172.21.15.47:0/208050470 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef80097c0 con 0x7f5f00102210 2024-02-23T00:40:55.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.433+0000 7f5eff7fe700 1 --2- 172.21.15.47:0/208050470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f000f9610 0x7f5f000f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:40:55.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.433+0000 7f5f04c42700 1 --2- 172.21.15.47:0/208050470 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5f00102210 0x7f5f000f7640 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f5ef8009ad0 tx=0x7f5ef8004c50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:55.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.433+0000 7f5efd7fa700 10 client.?.objecter ms_handle_connect 0x7f5f00102210 2024-02-23T00:40:55.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.433+0000 7f5efd7fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:40:55.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.433+0000 7f5efd7fa700 1 -- 172.21.15.47:0/208050470 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef800e040 con 0x7f5f00102210 2024-02-23T00:40:55.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.433+0000 7f5efd7fa700 1 -- 172.21.15.47:0/208050470 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f5ef8018040 con 0x7f5f00102210 2024-02-23T00:40:55.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.433+0000 7f5f066a5700 1 -- 172.21.15.47:0/208050470 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f001169e0 con 0x7f5f00102210 2024-02-23T00:40:55.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.433+0000 7f5efd7fa700 1 -- 172.21.15.47:0/208050470 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8013d30 con 0x7f5f00102210 2024-02-23T00:40:55.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.433+0000 7f5f066a5700 10 client.24542.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:40:55.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.433+0000 7f5f066a5700 1 -- 172.21.15.47:0/208050470 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f00116de0 con 0x7f5f00102210 2024-02-23T00:40:55.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.433+0000 7f5f066a5700 1 librados: init done 2024-02-23T00:40:55.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.434+0000 7f5f066a5700 10 client.24542.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:40:55.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.434+0000 7f5f066a5700 15 client.24542.objecter _session_command_op_assign -1 1 2024-02-23T00:40:55.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.434+0000 7f5f066a5700 20 client.24542.objecter _get_session osd=-1 returning homeless 2024-02-23T00:40:55.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.434+0000 7f5f066a5700 10 client.24542.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:40:55.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.434+0000 7f5f066a5700 1 -- 172.21.15.47:0/208050470 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5f0005fa60 con 0x7f5f00102210 2024-02-23T00:40:55.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.434+0000 7f5efd7fa700 1 -- 172.21.15.47:0/208050470 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f5ef8013540 con 0x7f5f00102210 2024-02-23T00:40:55.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.435+0000 7f5efd7fa700 1 --2- 172.21.15.47:0/208050470 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5eec062cc0 0x7f5eec065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:55.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.435+0000 7f5efd7fa700 1 -- 172.21.15.47:0/208050470 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(88..88 src has 38..88) v4 ==== 7421+0+0 (secure 0 0 0) 0x7f5ef807dd30 con 0x7f5f00102210 2024-02-23T00:40:55.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.435+0000 7f5efd7fa700 10 client.24542.objecter ms_dispatch 0x7f5f00001550 osd_map(88..88 src has 38..88) v4 2024-02-23T00:40:55.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.435+0000 7f5efd7fa700 3 client.24542.objecter handle_osd_map got epochs [88,88] > 0 2024-02-23T00:40:55.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.435+0000 7f5efd7fa700 3 client.24542.objecter handle_osd_map decoding full epoch 88 2024-02-23T00:40:55.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.435+0000 7f5efffff700 1 --2- 172.21.15.47:0/208050470 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5eec062cc0 0x7f5eec065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:55.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.435+0000 7f5efd7fa700 10 client.24542.objecter checking command 1 2024-02-23T00:40:55.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.435+0000 7f5efd7fa700 1 --2- 172.21.15.47:0/208050470 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f5eec06cc50 0x7f5eec06f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:55.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.435+0000 7f5efd7fa700 20 client.24542.objecter _get_session s=0x7f5eec06c580 osd=6 3 2024-02-23T00:40:55.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.435+0000 7f5efd7fa700 20 client.24542.objecter put_session s=0x7f5eec06c580 osd=6 3 2024-02-23T00:40:55.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.435+0000 7f5efd7fa700 15 client.24542.objecter _session_command_op_remove -1 1 2024-02-23T00:40:55.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.435+0000 7f5efd7fa700 20 client.24542.objecter _get_session s=0x7f5eec06c580 osd=6 3 2024-02-23T00:40:55.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.435+0000 7f5efd7fa700 20 client.24542.objecter get_session s=0x7f5eec06c580 osd=6 3 2024-02-23T00:40:55.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.435+0000 7f5efd7fa700 15 client.24542.objecter _session_command_op_assign 6 1 2024-02-23T00:40:55.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.435+0000 7f5efd7fa700 20 client.24542.objecter put_session s=0x7f5eec06c580 osd=6 4 2024-02-23T00:40:55.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.435+0000 7f5efd7fa700 10 client.24542.objecter _send_command 1 2024-02-23T00:40:55.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.435+0000 7f5efd7fa700 1 -- 172.21.15.47:0/208050470 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5eec06f770 con 0x7f5eec06cc50 2024-02-23T00:40:55.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.435+0000 7f5efd7fa700 20 client.24542.objecter dump_active .. 0 homeless 2024-02-23T00:40:55.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.435+0000 7f5efd7fa700 1 -- 172.21.15.47:0/208050470 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ef8080290 con 0x7f5f00102210 2024-02-23T00:40:55.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.435+0000 7f5efffff700 1 --2- 172.21.15.47:0/208050470 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5eec062cc0 0x7f5eec065180 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f5ee80048b0 tx=0x7f5ee800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:55.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.435+0000 7f5efd7fa700 10 client.24542.objecter ms_handle_connect 0x7f5eec062cc0 2024-02-23T00:40:55.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.435+0000 7f5eff7fe700 1 --2- 172.21.15.47:0/208050470 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f5eec06cc50 0x7f5eec06f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:55.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.435+0000 7f5efd7fa700 10 client.24542.objecter ms_handle_connect 0x7f5eec06cc50 2024-02-23T00:40:55.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.435+0000 7f5eff7fe700 1 --2- 172.21.15.47:0/208050470 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f5eec06cc50 0x7f5eec06f0f0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:55.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:53.767156+0000 osd.7 (osd.7) 37 : cluster [DBG] 1.0 deep-scrub starts 2024-02-23T00:40:55.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:53.770075+0000 osd.7 (osd.7) 38 : cluster [DBG] 1.0 deep-scrub ok 2024-02-23T00:40:55.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:54.143911+0000 osd.6 (osd.6) 25 : cluster [DBG] 4.a deep-scrub starts 2024-02-23T00:40:55.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:54.152233+0000 osd.6 (osd.6) 26 : cluster [DBG] 4.a deep-scrub ok 2024-02-23T00:40:55.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.436+0000 7f5efd7fa700 1 -- 172.21.15.47:0/208050470 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5eec06f770 con 0x7f5eec06cc50 2024-02-23T00:40:55.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.436+0000 7f5efd7fa700 10 client.24542.objecter ms_dispatch 0x7f5f00001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:40:55.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.436+0000 7f5efd7fa700 10 client.24542.objecter _finish_command 1 = system:0 2024-02-23T00:40:55.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.436+0000 7f5efd7fa700 20 client.24542.objecter put_session s=0x7f5eec06c580 osd=6 4 2024-02-23T00:40:55.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.436+0000 7f5efd7fa700 15 client.24542.objecter _session_command_op_remove 6 1 2024-02-23T00:40:55.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.453+0000 7f5f066a5700 10 client.24542.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-02-23T00:40:55.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.453+0000 7f5f066a5700 15 client.24542.objecter _session_command_op_assign -1 2 2024-02-23T00:40:55.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.453+0000 7f5f066a5700 20 client.24542.objecter _get_session s=0x7f5eec06c580 osd=6 3 2024-02-23T00:40:55.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.453+0000 7f5f066a5700 20 client.24542.objecter put_session s=0x7f5eec06c580 osd=6 3 2024-02-23T00:40:55.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.453+0000 7f5f066a5700 20 client.24542.objecter _get_session s=0x7f5eec06c580 osd=6 3 2024-02-23T00:40:55.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.453+0000 7f5f066a5700 15 client.24542.objecter _session_command_op_remove -1 2 2024-02-23T00:40:55.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.453+0000 7f5f066a5700 20 client.24542.objecter get_session s=0x7f5eec06c580 osd=6 3 2024-02-23T00:40:55.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.453+0000 7f5f066a5700 15 client.24542.objecter _session_command_op_assign 6 2 2024-02-23T00:40:55.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.453+0000 7f5f066a5700 20 client.24542.objecter put_session s=0x7f5eec06c580 osd=6 4 2024-02-23T00:40:55.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.453+0000 7f5f066a5700 10 client.24542.objecter _send_command 2 2024-02-23T00:40:55.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.453+0000 7f5f066a5700 1 -- 172.21.15.47:0/208050470 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5f0005ee80 con 0x7f5eec06cc50 2024-02-23T00:40:55.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.453+0000 7f5efd7fa700 1 -- 172.21.15.47:0/208050470 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5f0005ee80 con 0x7f5eec06cc50 2024-02-23T00:40:55.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.453+0000 7f5efd7fa700 10 client.24542.objecter ms_dispatch 0x7f5f00001550 command_reply(tid 2: 0 ) v1 2024-02-23T00:40:55.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.453+0000 7f5efd7fa700 10 client.24542.objecter _finish_command 2 = system:0 2024-02-23T00:40:55.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.453+0000 7f5efd7fa700 20 client.24542.objecter put_session s=0x7f5eec06c580 osd=6 4 2024-02-23T00:40:55.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.453+0000 7f5efd7fa700 15 client.24542.objecter _session_command_op_remove 6 2 2024-02-23T00:40:55.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.453+0000 7f5f066a5700 10 librados: watch_flush enter 2024-02-23T00:40:55.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.453+0000 7f5f066a5700 10 librados: watch_flush exit 2024-02-23T00:40:55.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.453+0000 7f5f066a5700 10 client.24542.objecter close_session for osd.6 2024-02-23T00:40:55.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.453+0000 7f5f066a5700 1 -- 172.21.15.47:0/208050470 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f5eec06cc50 msgr2=0x7f5eec06f0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:55.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.453+0000 7f5f066a5700 1 --2- 172.21.15.47:0/208050470 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f5eec06cc50 0x7f5eec06f0f0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:55.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.453+0000 7f5f066a5700 20 client.24542.objecter put_session s=0x7f5eec06c580 osd=6 1 2024-02-23T00:40:55.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.453+0000 7f5f066a5700 20 client.24542.objecter shutdown clearing up homeless session... 2024-02-23T00:40:55.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.453+0000 7f5f066a5700 10 client.24542.objecter successfully canceled tick 2024-02-23T00:40:55.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.453+0000 7f5f066a5700 1 -- 172.21.15.47:0/208050470 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5eec062cc0 msgr2=0x7f5eec065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:55.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.453+0000 7f5f066a5700 1 --2- 172.21.15.47:0/208050470 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5eec062cc0 0x7f5eec065180 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f5ee80048b0 tx=0x7f5ee800b040).stop 2024-02-23T00:40:55.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.453+0000 7f5f066a5700 1 -- 172.21.15.47:0/208050470 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5f00102210 msgr2=0x7f5f000f7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:55.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.453+0000 7f5f066a5700 1 --2- 172.21.15.47:0/208050470 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5f00102210 0x7f5f000f7640 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f5ef8009ad0 tx=0x7f5ef8004c50).stop 2024-02-23T00:40:55.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.454+0000 7f5f04c42700 1 -- 172.21.15.47:0/208050470 reap_dead start 2024-02-23T00:40:55.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.454+0000 7f5f066a5700 1 -- 172.21.15.47:0/208050470 shutdown_connections 2024-02-23T00:40:55.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.454+0000 7f5f066a5700 1 -- 172.21.15.47:0/208050470 >> 172.21.15.47:0/208050470 conn(0x7f5f000f2820 msgr2=0x7f5f000f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:40:55.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.454+0000 7f5f066a5700 1 -- 172.21.15.47:0/208050470 shutdown_connections 2024-02-23T00:40:55.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.454+0000 7f5f066a5700 1 -- 172.21.15.47:0/208050470 wait complete. 2024-02-23T00:40:55.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:55.454+0000 7f5f066a5700 1 librados: shutdown 2024-02-23T00:40:56.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:54.347631+0000 osd.2 (osd.2) 25 : cluster [DBG] 6.f scrub starts 2024-02-23T00:40:56.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:54.348832+0000 osd.2 (osd.2) 26 : cluster [DBG] 6.f scrub ok 2024-02-23T00:40:56.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:54.736465+0000 osd.7 (osd.7) 39 : cluster [DBG] 6.12 scrub starts 2024-02-23T00:40:56.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:54.739530+0000 osd.7 (osd.7) 40 : cluster [DBG] 6.12 scrub ok 2024-02-23T00:40:56.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:54.937345+0000 osd.4 (osd.4) 23 : cluster [DBG] 6.17 deep-scrub starts 2024-02-23T00:40:56.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:54.938195+0000 osd.4 (osd.4) 24 : cluster [DBG] 6.17 deep-scrub ok 2024-02-23T00:40:56.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:55.141153+0000 mgr.y (mgr.14517) 68 : cluster [DBG] pgmap v55: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 6.0 KiB/s rd, 454 B/s wr, 11 op/s; 64 B/s, 2 objects/s recovering 2024-02-23T00:40:56.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:55.720417+0000 osd.1 (osd.1) 37 : cluster [DBG] 2.4 scrub starts 2024-02-23T00:40:56.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:55.723306+0000 osd.1 (osd.1) 38 : cluster [DBG] 2.4 scrub ok 2024-02-23T00:40:56.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:55.854137+0000 osd.5 (osd.5) 29 : cluster [DBG] 6.19 deep-scrub starts 2024-02-23T00:40:56.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:55.854958+0000 osd.5 (osd.5) 30 : cluster [DBG] 6.19 deep-scrub ok 2024-02-23T00:40:56.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:54.347631+0000 osd.2 (osd.2) 25 : cluster [DBG] 6.f scrub starts 2024-02-23T00:40:56.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:54.348832+0000 osd.2 (osd.2) 26 : cluster [DBG] 6.f scrub ok 2024-02-23T00:40:56.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:54.736465+0000 osd.7 (osd.7) 39 : cluster [DBG] 6.12 scrub starts 2024-02-23T00:40:56.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:54.739530+0000 osd.7 (osd.7) 40 : cluster [DBG] 6.12 scrub ok 2024-02-23T00:40:56.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:54.937345+0000 osd.4 (osd.4) 23 : cluster [DBG] 6.17 deep-scrub starts 2024-02-23T00:40:56.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:54.938195+0000 osd.4 (osd.4) 24 : cluster [DBG] 6.17 deep-scrub ok 2024-02-23T00:40:56.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:55.141153+0000 mgr.y (mgr.14517) 68 : cluster [DBG] pgmap v55: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 6.0 KiB/s rd, 454 B/s wr, 11 op/s; 64 B/s, 2 objects/s recovering 2024-02-23T00:40:56.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:55.720417+0000 osd.1 (osd.1) 37 : cluster [DBG] 2.4 scrub starts 2024-02-23T00:40:56.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:55.723306+0000 osd.1 (osd.1) 38 : cluster [DBG] 2.4 scrub ok 2024-02-23T00:40:56.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:55.854137+0000 osd.5 (osd.5) 29 : cluster [DBG] 6.19 deep-scrub starts 2024-02-23T00:40:56.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:55.854958+0000 osd.5 (osd.5) 30 : cluster [DBG] 6.19 deep-scrub ok 2024-02-23T00:40:56.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:54.347631+0000 osd.2 (osd.2) 25 : cluster [DBG] 6.f scrub starts 2024-02-23T00:40:56.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:54.348832+0000 osd.2 (osd.2) 26 : cluster [DBG] 6.f scrub ok 2024-02-23T00:40:56.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:54.736465+0000 osd.7 (osd.7) 39 : cluster [DBG] 6.12 scrub starts 2024-02-23T00:40:56.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:54.739530+0000 osd.7 (osd.7) 40 : cluster [DBG] 6.12 scrub ok 2024-02-23T00:40:56.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:54.937345+0000 osd.4 (osd.4) 23 : cluster [DBG] 6.17 deep-scrub starts 2024-02-23T00:40:56.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:54.938195+0000 osd.4 (osd.4) 24 : cluster [DBG] 6.17 deep-scrub ok 2024-02-23T00:40:56.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:55.141153+0000 mgr.y (mgr.14517) 68 : cluster [DBG] pgmap v55: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 6.0 KiB/s rd, 454 B/s wr, 11 op/s; 64 B/s, 2 objects/s recovering 2024-02-23T00:40:56.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:55.720417+0000 osd.1 (osd.1) 37 : cluster [DBG] 2.4 scrub starts 2024-02-23T00:40:56.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:55.723306+0000 osd.1 (osd.1) 38 : cluster [DBG] 2.4 scrub ok 2024-02-23T00:40:56.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:55.854137+0000 osd.5 (osd.5) 29 : cluster [DBG] 6.19 deep-scrub starts 2024-02-23T00:40:56.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:55.854958+0000 osd.5 (osd.5) 30 : cluster [DBG] 6.19 deep-scrub ok 2024-02-23T00:40:56.632 INFO:teuthology.orchestra.run.smithi047.stdout:201863462942 2024-02-23T00:40:56.632 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-02-23T00:40:57.054 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:40:57.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:55.926517+0000 osd.4 (osd.4) 25 : cluster [DBG] 4.13 scrub starts 2024-02-23T00:40:57.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:55.928344+0000 osd.4 (osd.4) 26 : cluster [DBG] 4.13 scrub ok 2024-02-23T00:40:57.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:56.887881+0000 osd.5 (osd.5) 31 : cluster [DBG] 6.d scrub starts 2024-02-23T00:40:57.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:56.890789+0000 osd.5 (osd.5) 32 : cluster [DBG] 6.d scrub ok 2024-02-23T00:40:57.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:56.892051+0000 osd.3 (osd.3) 23 : cluster [DBG] 6.11 scrub starts 2024-02-23T00:40:57.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:56.892792+0000 osd.3 (osd.3) 24 : cluster [DBG] 6.11 scrub ok 2024-02-23T00:40:57.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:55.926517+0000 osd.4 (osd.4) 25 : cluster [DBG] 4.13 scrub starts 2024-02-23T00:40:57.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:55.928344+0000 osd.4 (osd.4) 26 : cluster [DBG] 4.13 scrub ok 2024-02-23T00:40:57.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:56.887881+0000 osd.5 (osd.5) 31 : cluster [DBG] 6.d scrub starts 2024-02-23T00:40:57.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:56.890789+0000 osd.5 (osd.5) 32 : cluster [DBG] 6.d scrub ok 2024-02-23T00:40:57.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:56.892051+0000 osd.3 (osd.3) 23 : cluster [DBG] 6.11 scrub starts 2024-02-23T00:40:57.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:56.892792+0000 osd.3 (osd.3) 24 : cluster [DBG] 6.11 scrub ok 2024-02-23T00:40:57.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:55.926517+0000 osd.4 (osd.4) 25 : cluster [DBG] 4.13 scrub starts 2024-02-23T00:40:57.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:55.928344+0000 osd.4 (osd.4) 26 : cluster [DBG] 4.13 scrub ok 2024-02-23T00:40:57.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:56.887881+0000 osd.5 (osd.5) 31 : cluster [DBG] 6.d scrub starts 2024-02-23T00:40:57.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:56.890789+0000 osd.5 (osd.5) 32 : cluster [DBG] 6.d scrub ok 2024-02-23T00:40:57.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:56.892051+0000 osd.3 (osd.3) 23 : cluster [DBG] 6.11 scrub starts 2024-02-23T00:40:57.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:56.892792+0000 osd.3 (osd.3) 24 : cluster [DBG] 6.11 scrub ok 2024-02-23T00:40:58.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:57.141760+0000 mgr.y (mgr.14517) 69 : cluster [DBG] pgmap v56: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:40:58.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:57.680779+0000 osd.1 (osd.1) 39 : cluster [DBG] 2.d scrub starts 2024-02-23T00:40:58.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:57.681720+0000 osd.1 (osd.1) 40 : cluster [DBG] 2.d scrub ok 2024-02-23T00:40:58.552 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:57.141760+0000 mgr.y (mgr.14517) 69 : cluster [DBG] pgmap v56: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:40:58.552 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:57.680779+0000 osd.1 (osd.1) 39 : cluster [DBG] 2.d scrub starts 2024-02-23T00:40:58.552 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:57.681720+0000 osd.1 (osd.1) 40 : cluster [DBG] 2.d scrub ok 2024-02-23T00:40:58.552 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:57.141760+0000 mgr.y (mgr.14517) 69 : cluster [DBG] pgmap v56: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:40:58.552 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:57.680779+0000 osd.1 (osd.1) 39 : cluster [DBG] 2.d scrub starts 2024-02-23T00:40:58.552 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:57.681720+0000 osd.1 (osd.1) 40 : cluster [DBG] 2.d scrub ok 2024-02-23T00:40:59.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.084+0000 7f56ee59e700 1 -- 172.21.15.47:0/526412787 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f56e80f8cb0 msgr2=0x7f56e80f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:59.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.084+0000 7f56ee59e700 1 --2- 172.21.15.47:0/526412787 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f56e80f8cb0 0x7f56e80f9090 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f56d8004870 tx=0x7f56d8009cb0).stop 2024-02-23T00:40:59.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.085+0000 7f56ee59e700 1 -- 172.21.15.47:0/526412787 shutdown_connections 2024-02-23T00:40:59.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.085+0000 7f56ee59e700 1 --2- 172.21.15.47:0/526412787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56e8102210 0x7f56e8104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:59.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.085+0000 7f56ee59e700 1 --2- 172.21.15.47:0/526412787 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f56e80f8cb0 0x7f56e80f9090 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:59.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.085+0000 7f56ee59e700 1 --2- 172.21.15.47:0/526412787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56e80f9610 0x7f56e8101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:59.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.085+0000 7f56ee59e700 1 -- 172.21.15.47:0/526412787 >> 172.21.15.47:0/526412787 conn(0x7f56e80f2820 msgr2=0x7f56e80f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:40:59.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.085+0000 7f56ee59e700 1 -- 172.21.15.47:0/526412787 shutdown_connections 2024-02-23T00:40:59.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.085+0000 7f56ee59e700 1 -- 172.21.15.47:0/526412787 wait complete. 2024-02-23T00:40:59.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.085+0000 7f56ee59e700 1 librados: starting msgr at 2024-02-23T00:40:59.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.085+0000 7f56ee59e700 1 librados: starting objecter 2024-02-23T00:40:59.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.085+0000 7f56ee59e700 1 Processor -- start 2024-02-23T00:40:59.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.085+0000 7f56ee59e700 1 -- start start 2024-02-23T00:40:59.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.085+0000 7f56ee59e700 1 librados: setting wanted keys 2024-02-23T00:40:59.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.085+0000 7f56ee59e700 1 librados: calling monclient init 2024-02-23T00:40:59.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.086+0000 7f56ee59e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f56e80f8cb0 0x7f56e80f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:59.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.086+0000 7f56ee59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56e80f9610 0x7f56e80f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:59.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.086+0000 7f56ee59e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56e8102210 0x7f56e80f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:59.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.086+0000 7f56ee59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e8116590 con 0x7f56e80f9610 2024-02-23T00:40:59.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.086+0000 7f56ee59e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e8116700 con 0x7f56e8102210 2024-02-23T00:40:59.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.086+0000 7f56ee59e700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e81168c0 con 0x7f56e80f8cb0 2024-02-23T00:40:59.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.086+0000 7f56e7fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f56e80f8cb0 0x7f56e80f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:59.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.086+0000 7f56e77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56e80f9610 0x7f56e80f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:59.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.086+0000 7f56e7fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f56e80f8cb0 0x7f56e80f6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:38850/0 (socket says 172.21.15.47:38850) 2024-02-23T00:40:59.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.086+0000 7f56e7fff700 1 -- 172.21.15.47:0/3759586287 learned_addr learned my addr 172.21.15.47:0/3759586287 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:40:59.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.086+0000 7f56e77fe700 1 --2- 172.21.15.47:0/3759586287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56e80f9610 0x7f56e80f7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37224/0 (socket says 172.21.15.47:37224) 2024-02-23T00:40:59.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.086+0000 7f56ecb3b700 1 --2- 172.21.15.47:0/3759586287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56e8102210 0x7f56e80f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:59.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.086+0000 7f56e7fff700 1 -- 172.21.15.47:0/3759586287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56e8102210 msgr2=0x7f56e80f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:59.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.086+0000 7f56e7fff700 1 --2- 172.21.15.47:0/3759586287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56e8102210 0x7f56e80f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:59.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.086+0000 7f56e7fff700 1 -- 172.21.15.47:0/3759586287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56e80f9610 msgr2=0x7f56e80f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:59.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.086+0000 7f56e7fff700 1 --2- 172.21.15.47:0/3759586287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56e80f9610 0x7f56e80f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:59.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.086+0000 7f56e7fff700 1 -- 172.21.15.47:0/3759586287 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56dc005040 con 0x7f56e80f8cb0 2024-02-23T00:40:59.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.086+0000 7f56e77fe700 1 --2- 172.21.15.47:0/3759586287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56e80f9610 0x7f56e80f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:40:59.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.086+0000 7f56e7fff700 1 --2- 172.21.15.47:0/3759586287 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f56e80f8cb0 0x7f56e80f6c60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f56d8009da0 tx=0x7f56d8009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:59.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.086+0000 7f56e57fa700 10 client.?.objecter ms_handle_connect 0x7f56e80f8cb0 2024-02-23T00:40:59.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.086+0000 7f56e57fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:40:59.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.086+0000 7f56e57fa700 1 -- 172.21.15.47:0/3759586287 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d800e030 con 0x7f56e80f8cb0 2024-02-23T00:40:59.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.086+0000 7f56e57fa700 1 -- 172.21.15.47:0/3759586287 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f56d8013d70 con 0x7f56e80f8cb0 2024-02-23T00:40:59.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.086+0000 7f56ee59e700 1 -- 172.21.15.47:0/3759586287 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56d8005040 con 0x7f56e80f8cb0 2024-02-23T00:40:59.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.086+0000 7f56e57fa700 1 -- 172.21.15.47:0/3759586287 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d8018070 con 0x7f56e80f8cb0 2024-02-23T00:40:59.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.086+0000 7f56ee59e700 10 client.24548.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:40:59.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.086+0000 7f56ee59e700 1 -- 172.21.15.47:0/3759586287 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e8116db0 con 0x7f56e80f8cb0 2024-02-23T00:40:59.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.086+0000 7f56ee59e700 1 librados: init done 2024-02-23T00:40:59.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.087+0000 7f56ee59e700 10 client.24548.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:40:59.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.087+0000 7f56ee59e700 15 client.24548.objecter _session_command_op_assign -1 1 2024-02-23T00:40:59.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.087+0000 7f56ee59e700 20 client.24548.objecter _get_session osd=-1 returning homeless 2024-02-23T00:40:59.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.087+0000 7f56ee59e700 10 client.24548.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:40:59.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.087+0000 7f56ee59e700 1 -- 172.21.15.47:0/3759586287 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f56e805fa60 con 0x7f56e80f8cb0 2024-02-23T00:40:59.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.088+0000 7f56e57fa700 1 -- 172.21.15.47:0/3759586287 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f56d8009240 con 0x7f56e80f8cb0 2024-02-23T00:40:59.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.088+0000 7f56e57fa700 1 --2- 172.21.15.47:0/3759586287 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f56d0062cc0 0x7f56d0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:59.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.088+0000 7f56e57fa700 1 -- 172.21.15.47:0/3759586287 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(88..88 src has 38..88) v4 ==== 7421+0+0 (secure 0 0 0) 0x7f56d807e960 con 0x7f56e80f8cb0 2024-02-23T00:40:59.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.088+0000 7f56e57fa700 10 client.24548.objecter ms_dispatch 0x7f56e8001550 osd_map(88..88 src has 38..88) v4 2024-02-23T00:40:59.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.088+0000 7f56e57fa700 3 client.24548.objecter handle_osd_map got epochs [88,88] > 0 2024-02-23T00:40:59.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.088+0000 7f56e57fa700 3 client.24548.objecter handle_osd_map decoding full epoch 88 2024-02-23T00:40:59.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.088+0000 7f56e77fe700 1 --2- 172.21.15.47:0/3759586287 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f56d0062cc0 0x7f56d0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:59.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.089+0000 7f56e57fa700 10 client.24548.objecter checking command 1 2024-02-23T00:40:59.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.089+0000 7f56e57fa700 1 --2- 172.21.15.47:0/3759586287 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f56d006cc50 0x7f56d006f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:40:59.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.089+0000 7f56e57fa700 20 client.24548.objecter _get_session s=0x7f56d006c580 osd=7 3 2024-02-23T00:40:59.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.089+0000 7f56e57fa700 20 client.24548.objecter put_session s=0x7f56d006c580 osd=7 3 2024-02-23T00:40:59.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.089+0000 7f56e57fa700 15 client.24548.objecter _session_command_op_remove -1 1 2024-02-23T00:40:59.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.089+0000 7f56e57fa700 20 client.24548.objecter _get_session s=0x7f56d006c580 osd=7 3 2024-02-23T00:40:59.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.089+0000 7f56e57fa700 20 client.24548.objecter get_session s=0x7f56d006c580 osd=7 3 2024-02-23T00:40:59.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.089+0000 7f56e57fa700 15 client.24548.objecter _session_command_op_assign 7 1 2024-02-23T00:40:59.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.089+0000 7f56e57fa700 20 client.24548.objecter put_session s=0x7f56d006c580 osd=7 4 2024-02-23T00:40:59.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.089+0000 7f56e57fa700 10 client.24548.objecter _send_command 1 2024-02-23T00:40:59.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.089+0000 7f56e57fa700 1 -- 172.21.15.47:0/3759586287 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56d006f770 con 0x7f56d006cc50 2024-02-23T00:40:59.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.089+0000 7f56e57fa700 20 client.24548.objecter dump_active .. 0 homeless 2024-02-23T00:40:59.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.089+0000 7f56e57fa700 1 -- 172.21.15.47:0/3759586287 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56d807ed60 con 0x7f56e80f8cb0 2024-02-23T00:40:59.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.089+0000 7f56ecb3b700 1 --2- 172.21.15.47:0/3759586287 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f56d006cc50 0x7f56d006f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:40:59.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.089+0000 7f56e57fa700 10 client.24548.objecter ms_handle_connect 0x7f56d0062cc0 2024-02-23T00:40:59.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.089+0000 7f56e77fe700 1 --2- 172.21.15.47:0/3759586287 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f56d0062cc0 0x7f56d0065180 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f56e80f8400 tx=0x7f56dc00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:59.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.089+0000 7f56e57fa700 10 client.24548.objecter ms_handle_connect 0x7f56d006cc50 2024-02-23T00:40:59.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.089+0000 7f56ecb3b700 1 --2- 172.21.15.47:0/3759586287 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f56d006cc50 0x7f56d006f0f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:40:59.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.090+0000 7f56e57fa700 1 -- 172.21.15.47:0/3759586287 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f56d006f770 con 0x7f56d006cc50 2024-02-23T00:40:59.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.090+0000 7f56e57fa700 10 client.24548.objecter ms_dispatch 0x7f56e8001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:40:59.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.090+0000 7f56e57fa700 10 client.24548.objecter _finish_command 1 = system:0 2024-02-23T00:40:59.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.090+0000 7f56e57fa700 20 client.24548.objecter put_session s=0x7f56d006c580 osd=7 4 2024-02-23T00:40:59.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.090+0000 7f56e57fa700 15 client.24548.objecter _session_command_op_remove 7 1 2024-02-23T00:40:59.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.104+0000 7f56ee59e700 10 client.24548.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-02-23T00:40:59.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.104+0000 7f56ee59e700 15 client.24548.objecter _session_command_op_assign -1 2 2024-02-23T00:40:59.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.104+0000 7f56ee59e700 20 client.24548.objecter _get_session s=0x7f56d006c580 osd=7 3 2024-02-23T00:40:59.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.104+0000 7f56ee59e700 20 client.24548.objecter put_session s=0x7f56d006c580 osd=7 3 2024-02-23T00:40:59.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.104+0000 7f56ee59e700 20 client.24548.objecter _get_session s=0x7f56d006c580 osd=7 3 2024-02-23T00:40:59.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.104+0000 7f56ee59e700 15 client.24548.objecter _session_command_op_remove -1 2 2024-02-23T00:40:59.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.104+0000 7f56ee59e700 20 client.24548.objecter get_session s=0x7f56d006c580 osd=7 3 2024-02-23T00:40:59.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.104+0000 7f56ee59e700 15 client.24548.objecter _session_command_op_assign 7 2 2024-02-23T00:40:59.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.104+0000 7f56ee59e700 20 client.24548.objecter put_session s=0x7f56d006c580 osd=7 4 2024-02-23T00:40:59.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.104+0000 7f56ee59e700 10 client.24548.objecter _send_command 2 2024-02-23T00:40:59.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.104+0000 7f56ee59e700 1 -- 172.21.15.47:0/3759586287 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f56e805ee80 con 0x7f56d006cc50 2024-02-23T00:40:59.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.104+0000 7f56e57fa700 1 -- 172.21.15.47:0/3759586287 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f56e805ee80 con 0x7f56d006cc50 2024-02-23T00:40:59.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.104+0000 7f56e57fa700 10 client.24548.objecter ms_dispatch 0x7f56e8001550 command_reply(tid 2: 0 ) v1 2024-02-23T00:40:59.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.104+0000 7f56e57fa700 10 client.24548.objecter _finish_command 2 = system:0 2024-02-23T00:40:59.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.104+0000 7f56e57fa700 20 client.24548.objecter put_session s=0x7f56d006c580 osd=7 4 2024-02-23T00:40:59.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.104+0000 7f56e57fa700 15 client.24548.objecter _session_command_op_remove 7 2 2024-02-23T00:40:59.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.105+0000 7f56ee59e700 10 librados: watch_flush enter 2024-02-23T00:40:59.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.105+0000 7f56ee59e700 10 librados: watch_flush exit 2024-02-23T00:40:59.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.105+0000 7f56ee59e700 10 client.24548.objecter close_session for osd.7 2024-02-23T00:40:59.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.105+0000 7f56ee59e700 1 -- 172.21.15.47:0/3759586287 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f56d006cc50 msgr2=0x7f56d006f0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:59.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.105+0000 7f56ee59e700 1 --2- 172.21.15.47:0/3759586287 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f56d006cc50 0x7f56d006f0f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:40:59.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.105+0000 7f56ee59e700 20 client.24548.objecter put_session s=0x7f56d006c580 osd=7 1 2024-02-23T00:40:59.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.105+0000 7f56ee59e700 20 client.24548.objecter shutdown clearing up homeless session... 2024-02-23T00:40:59.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.105+0000 7f56ee59e700 10 client.24548.objecter successfully canceled tick 2024-02-23T00:40:59.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.105+0000 7f56ee59e700 1 -- 172.21.15.47:0/3759586287 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f56d0062cc0 msgr2=0x7f56d0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:59.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.105+0000 7f56ee59e700 1 --2- 172.21.15.47:0/3759586287 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f56d0062cc0 0x7f56d0065180 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f56e80f8400 tx=0x7f56dc00b040).stop 2024-02-23T00:40:59.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.105+0000 7f56ee59e700 1 -- 172.21.15.47:0/3759586287 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f56e80f8cb0 msgr2=0x7f56e80f6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:40:59.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.105+0000 7f56ee59e700 1 --2- 172.21.15.47:0/3759586287 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f56e80f8cb0 0x7f56e80f6c60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f56d8009da0 tx=0x7f56d8009cb0).stop 2024-02-23T00:40:59.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.105+0000 7f56ecb3b700 1 -- 172.21.15.47:0/3759586287 reap_dead start 2024-02-23T00:40:59.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.105+0000 7f56ee59e700 1 -- 172.21.15.47:0/3759586287 shutdown_connections 2024-02-23T00:40:59.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.105+0000 7f56ee59e700 1 -- 172.21.15.47:0/3759586287 >> 172.21.15.47:0/3759586287 conn(0x7f56e80f2820 msgr2=0x7f56e80f34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:40:59.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.105+0000 7f56ee59e700 1 -- 172.21.15.47:0/3759586287 shutdown_connections 2024-02-23T00:40:59.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.105+0000 7f56ee59e700 1 -- 172.21.15.47:0/3759586287 wait complete. 2024-02-23T00:40:59.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:40:59.105+0000 7f56ee59e700 1 librados: shutdown 2024-02-23T00:40:59.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:58.329278+0000 osd.2 (osd.2) 27 : cluster [DBG] 5.d scrub starts 2024-02-23T00:40:59.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:58.331043+0000 osd.2 (osd.2) 28 : cluster [DBG] 5.d scrub ok 2024-02-23T00:40:59.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:40:58.722169+0000 mon.a (mon.0) 736 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:59.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:58.872003+0000 osd.0 (osd.0) 23 : cluster [DBG] 2.1f deep-scrub starts 2024-02-23T00:40:59.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:58.873008+0000 osd.0 (osd.0) 24 : cluster [DBG] 2.1f deep-scrub ok 2024-02-23T00:40:59.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:40:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:59.142106+0000 mgr.y (mgr.14517) 70 : cluster [DBG] pgmap v57: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:40:59.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:58.329278+0000 osd.2 (osd.2) 27 : cluster [DBG] 5.d scrub starts 2024-02-23T00:40:59.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:58.331043+0000 osd.2 (osd.2) 28 : cluster [DBG] 5.d scrub ok 2024-02-23T00:40:59.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:40:58.722169+0000 mon.a (mon.0) 736 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:40:59.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:58.872003+0000 osd.0 (osd.0) 23 : cluster [DBG] 2.1f deep-scrub starts 2024-02-23T00:40:59.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:58.873008+0000 osd.0 (osd.0) 24 : cluster [DBG] 2.1f deep-scrub ok 2024-02-23T00:40:59.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:40:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:59.142106+0000 mgr.y (mgr.14517) 70 : cluster [DBG] pgmap v57: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:40:59.947 INFO:teuthology.orchestra.run.smithi047.stdout:231928234008 2024-02-23T00:40:59.947 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-23T00:41:00.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:58.329278+0000 osd.2 (osd.2) 27 : cluster [DBG] 5.d scrub starts 2024-02-23T00:41:00.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:58.331043+0000 osd.2 (osd.2) 28 : cluster [DBG] 5.d scrub ok 2024-02-23T00:41:00.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:40:58.722169+0000 mon.a (mon.0) 736 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:41:00.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:58.872003+0000 osd.0 (osd.0) 23 : cluster [DBG] 2.1f deep-scrub starts 2024-02-23T00:41:00.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:58.873008+0000 osd.0 (osd.0) 24 : cluster [DBG] 2.1f deep-scrub ok 2024-02-23T00:41:00.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:40:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:59.142106+0000 mgr.y (mgr.14517) 70 : cluster [DBG] pgmap v57: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:41:00.728 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:41:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:41:00] "GET /metrics HTTP/1.1" 200 172152 "" "Prometheus/2.33.4" 2024-02-23T00:41:00.822 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:41:01.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:59.666879+0000 osd.1 (osd.1) 41 : cluster [DBG] 5.12 scrub starts 2024-02-23T00:41:01.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:59.669806+0000 osd.1 (osd.1) 42 : cluster [DBG] 5.12 scrub ok 2024-02-23T00:41:01.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:59.883702+0000 osd.5 (osd.5) 33 : cluster [DBG] 4.8 scrub starts 2024-02-23T00:41:01.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:59.885902+0000 osd.5 (osd.5) 34 : cluster [DBG] 4.8 scrub ok 2024-02-23T00:41:01.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:59.957743+0000 osd.3 (osd.3) 25 : cluster [DBG] 3.2 scrub starts 2024-02-23T00:41:01.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:40:59.958841+0000 osd.3 (osd.3) 26 : cluster [DBG] 3.2 scrub ok 2024-02-23T00:41:01.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:59.666879+0000 osd.1 (osd.1) 41 : cluster [DBG] 5.12 scrub starts 2024-02-23T00:41:01.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:59.669806+0000 osd.1 (osd.1) 42 : cluster [DBG] 5.12 scrub ok 2024-02-23T00:41:01.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:59.883702+0000 osd.5 (osd.5) 33 : cluster [DBG] 4.8 scrub starts 2024-02-23T00:41:01.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:59.885902+0000 osd.5 (osd.5) 34 : cluster [DBG] 4.8 scrub ok 2024-02-23T00:41:01.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:59.957743+0000 osd.3 (osd.3) 25 : cluster [DBG] 3.2 scrub starts 2024-02-23T00:41:01.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:40:59.958841+0000 osd.3 (osd.3) 26 : cluster [DBG] 3.2 scrub ok 2024-02-23T00:41:01.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:59.666879+0000 osd.1 (osd.1) 41 : cluster [DBG] 5.12 scrub starts 2024-02-23T00:41:01.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:59.669806+0000 osd.1 (osd.1) 42 : cluster [DBG] 5.12 scrub ok 2024-02-23T00:41:01.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:59.883702+0000 osd.5 (osd.5) 33 : cluster [DBG] 4.8 scrub starts 2024-02-23T00:41:01.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:59.885902+0000 osd.5 (osd.5) 34 : cluster [DBG] 4.8 scrub ok 2024-02-23T00:41:01.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:59.957743+0000 osd.3 (osd.3) 25 : cluster [DBG] 3.2 scrub starts 2024-02-23T00:41:01.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:40:59.958841+0000 osd.3 (osd.3) 26 : cluster [DBG] 3.2 scrub ok 2024-02-23T00:41:02.008 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:00.317262+0000 osd.2 (osd.2) 29 : cluster [DBG] 2.1 scrub starts 2024-02-23T00:41:02.009 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:00.318628+0000 osd.2 (osd.2) 30 : cluster [DBG] 2.1 scrub ok 2024-02-23T00:41:02.009 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:00.647153+0000 osd.1 (osd.1) 43 : cluster [DBG] 3.0 scrub starts 2024-02-23T00:41:02.009 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:00.648349+0000 osd.1 (osd.1) 44 : cluster [DBG] 3.0 scrub ok 2024-02-23T00:41:02.009 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:00.807706+0000 osd.0 (osd.0) 25 : cluster [DBG] 6.0 scrub starts 2024-02-23T00:41:02.009 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:00.809184+0000 osd.0 (osd.0) 26 : cluster [DBG] 6.0 scrub ok 2024-02-23T00:41:02.009 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:00.983313+0000 osd.6 (osd.6) 27 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:41:02.009 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:00.983372+0000 osd.6 (osd.6) 28 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:41:02.009 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:01.142424+0000 mgr.y (mgr.14517) 71 : cluster [DBG] pgmap v58: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 866 B/s rd, 0 op/s 2024-02-23T00:41:02.009 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:00.317262+0000 osd.2 (osd.2) 29 : cluster [DBG] 2.1 scrub starts 2024-02-23T00:41:02.010 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:00.318628+0000 osd.2 (osd.2) 30 : cluster [DBG] 2.1 scrub ok 2024-02-23T00:41:02.010 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:00.647153+0000 osd.1 (osd.1) 43 : cluster [DBG] 3.0 scrub starts 2024-02-23T00:41:02.010 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:00.648349+0000 osd.1 (osd.1) 44 : cluster [DBG] 3.0 scrub ok 2024-02-23T00:41:02.010 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:00.807706+0000 osd.0 (osd.0) 25 : cluster [DBG] 6.0 scrub starts 2024-02-23T00:41:02.010 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:00.809184+0000 osd.0 (osd.0) 26 : cluster [DBG] 6.0 scrub ok 2024-02-23T00:41:02.010 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:00.983313+0000 osd.6 (osd.6) 27 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:41:02.010 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:00.983372+0000 osd.6 (osd.6) 28 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:41:02.010 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:01.142424+0000 mgr.y (mgr.14517) 71 : cluster [DBG] pgmap v58: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 866 B/s rd, 0 op/s 2024-02-23T00:41:02.177 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:00.317262+0000 osd.2 (osd.2) 29 : cluster [DBG] 2.1 scrub starts 2024-02-23T00:41:02.177 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:00.318628+0000 osd.2 (osd.2) 30 : cluster [DBG] 2.1 scrub ok 2024-02-23T00:41:02.177 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:00.647153+0000 osd.1 (osd.1) 43 : cluster [DBG] 3.0 scrub starts 2024-02-23T00:41:02.177 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:00.648349+0000 osd.1 (osd.1) 44 : cluster [DBG] 3.0 scrub ok 2024-02-23T00:41:02.178 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:00.807706+0000 osd.0 (osd.0) 25 : cluster [DBG] 6.0 scrub starts 2024-02-23T00:41:02.178 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:00.809184+0000 osd.0 (osd.0) 26 : cluster [DBG] 6.0 scrub ok 2024-02-23T00:41:02.178 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:00.983313+0000 osd.6 (osd.6) 27 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:41:02.178 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:00.983372+0000 osd.6 (osd.6) 28 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:41:02.178 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:01.142424+0000 mgr.y (mgr.14517) 71 : cluster [DBG] pgmap v58: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 866 B/s rd, 0 op/s 2024-02-23T00:41:02.443 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:41:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:41:02.556 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:41:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=info ts=2024-02-23T00:41:02.285Z caller=cluster.go:688 component=cluster msg="gossip settled; proceeding" elapsed=10.003216506s 2024-02-23T00:41:02.943 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:41:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:41:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:41:02.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:01.062565+0000 osd.4 (osd.4) 27 : cluster [DBG] 5.1c deep-scrub starts 2024-02-23T00:41:02.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:01.063654+0000 osd.4 (osd.4) 28 : cluster [DBG] 5.1c deep-scrub ok 2024-02-23T00:41:02.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:01.735568+0000 mon.a (mon.0) 737 : cluster [DBG] osdmap e89: 8 total, 8 up, 8 in 2024-02-23T00:41:02.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:41:02.175786+0000 mgr.y (mgr.14517) 72 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:41:03.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:01.062565+0000 osd.4 (osd.4) 27 : cluster [DBG] 5.1c deep-scrub starts 2024-02-23T00:41:03.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:01.063654+0000 osd.4 (osd.4) 28 : cluster [DBG] 5.1c deep-scrub ok 2024-02-23T00:41:03.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:01.735568+0000 mon.a (mon.0) 737 : cluster [DBG] osdmap e89: 8 total, 8 up, 8 in 2024-02-23T00:41:03.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:41:02.175786+0000 mgr.y (mgr.14517) 72 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:41:03.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:01.062565+0000 osd.4 (osd.4) 27 : cluster [DBG] 5.1c deep-scrub starts 2024-02-23T00:41:03.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:01.063654+0000 osd.4 (osd.4) 28 : cluster [DBG] 5.1c deep-scrub ok 2024-02-23T00:41:03.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:01.735568+0000 mon.a (mon.0) 737 : cluster [DBG] osdmap e89: 8 total, 8 up, 8 in 2024-02-23T00:41:03.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:41:02.175786+0000 mgr.y (mgr.14517) 72 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:41:03.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.359+0000 7f6f956cb700 1 -- 172.21.15.47:0/474907786 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f900fd410 msgr2=0x7f6f900f6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:03.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.359+0000 7f6f956cb700 1 --2- 172.21.15.47:0/474907786 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f900fd410 0x7f6f900f6c40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f6f80004870 tx=0x7f6f80009cb0).stop 2024-02-23T00:41:03.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.359+0000 7f6f956cb700 1 -- 172.21.15.47:0/474907786 shutdown_connections 2024-02-23T00:41:03.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.359+0000 7f6f956cb700 1 --2- 172.21.15.47:0/474907786 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6f900f7320 0x7f6f90103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:03.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.359+0000 7f6f956cb700 1 --2- 172.21.15.47:0/474907786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f900fcab0 0x7f6f900fce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:03.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.359+0000 7f6f956cb700 1 --2- 172.21.15.47:0/474907786 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f900fd410 0x7f6f900f6c40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:03.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.359+0000 7f6f956cb700 1 -- 172.21.15.47:0/474907786 >> 172.21.15.47:0/474907786 conn(0x7f6f900f26e0 msgr2=0x7f6f900f4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:41:03.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.359+0000 7f6f956cb700 1 -- 172.21.15.47:0/474907786 shutdown_connections 2024-02-23T00:41:03.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.359+0000 7f6f956cb700 1 -- 172.21.15.47:0/474907786 wait complete. 2024-02-23T00:41:03.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.359+0000 7f6f956cb700 1 librados: starting msgr at 2024-02-23T00:41:03.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.359+0000 7f6f956cb700 1 librados: starting objecter 2024-02-23T00:41:03.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.359+0000 7f6f956cb700 1 Processor -- start 2024-02-23T00:41:03.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.360+0000 7f6f956cb700 1 -- start start 2024-02-23T00:41:03.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.360+0000 7f6f956cb700 1 librados: setting wanted keys 2024-02-23T00:41:03.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.360+0000 7f6f956cb700 1 librados: calling monclient init 2024-02-23T00:41:03.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.360+0000 7f6f956cb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f900f7320 0x7f6f900f88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:03.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.360+0000 7f6f956cb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f900fcab0 0x7f6f900f8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:03.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.360+0000 7f6f956cb700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6f900fd410 0x7f6f900f9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:03.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.360+0000 7f6f956cb700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f901162e0 con 0x7f6f900fcab0 2024-02-23T00:41:03.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.360+0000 7f6f956cb700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f90116450 con 0x7f6f900f7320 2024-02-23T00:41:03.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.360+0000 7f6f956cb700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f90116610 con 0x7f6f900fd410 2024-02-23T00:41:03.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.360+0000 7f6f8e7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f900fcab0 0x7f6f900f8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:03.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.360+0000 7f6f8f7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6f900fd410 0x7f6f900f9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:03.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.360+0000 7f6f8e7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f900fcab0 0x7f6f900f8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37236/0 (socket says 172.21.15.47:37236) 2024-02-23T00:41:03.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.360+0000 7f6f8e7fc700 1 -- 172.21.15.47:0/161029889 learned_addr learned my addr 172.21.15.47:0/161029889 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:41:03.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.360+0000 7f6f8f7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6f900fd410 0x7f6f900f9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:38864/0 (socket says 172.21.15.47:38864) 2024-02-23T00:41:03.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.360+0000 7f6f8effd700 1 --2- 172.21.15.47:0/161029889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f900f7320 0x7f6f900f88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:03.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.360+0000 7f6f8e7fc700 1 -- 172.21.15.47:0/161029889 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6f900fd410 msgr2=0x7f6f900f9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:03.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.360+0000 7f6f8e7fc700 1 --2- 172.21.15.47:0/161029889 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6f900fd410 0x7f6f900f9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:03.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.360+0000 7f6f8e7fc700 1 -- 172.21.15.47:0/161029889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f900f7320 msgr2=0x7f6f900f88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:03.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.360+0000 7f6f8e7fc700 1 --2- 172.21.15.47:0/161029889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f900f7320 0x7f6f900f88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:03.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.360+0000 7f6f8e7fc700 1 -- 172.21.15.47:0/161029889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f880097c0 con 0x7f6f900fcab0 2024-02-23T00:41:03.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.360+0000 7f6f8f7fe700 1 --2- 172.21.15.47:0/161029889 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6f900fd410 0x7f6f900f9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:41:03.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.360+0000 7f6f8e7fc700 1 --2- 172.21.15.47:0/161029889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f900fcab0 0x7f6f900f8da0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f6f80009da0 tx=0x7f6f80009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:03.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.360+0000 7f6f77fff700 10 client.?.objecter ms_handle_connect 0x7f6f900fcab0 2024-02-23T00:41:03.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.360+0000 7f6f77fff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:41:03.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.361+0000 7f6f77fff700 1 -- 172.21.15.47:0/161029889 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8000e030 con 0x7f6f900fcab0 2024-02-23T00:41:03.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.361+0000 7f6f77fff700 1 -- 172.21.15.47:0/161029889 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f6f80013d70 con 0x7f6f900fcab0 2024-02-23T00:41:03.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.361+0000 7f6f956cb700 1 -- 172.21.15.47:0/161029889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f80005040 con 0x7f6f900fcab0 2024-02-23T00:41:03.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.361+0000 7f6f77fff700 1 -- 172.21.15.47:0/161029889 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f80018070 con 0x7f6f900fcab0 2024-02-23T00:41:03.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.361+0000 7f6f956cb700 10 client.14661.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:41:03.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.361+0000 7f6f956cb700 1 -- 172.21.15.47:0/161029889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f90116b90 con 0x7f6f900fcab0 2024-02-23T00:41:03.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.361+0000 7f6f956cb700 1 librados: init done 2024-02-23T00:41:03.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.362+0000 7f6f956cb700 1 -- 172.21.15.47:0/161029889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f9005f9d0 con 0x7f6f900fcab0 2024-02-23T00:41:03.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.362+0000 7f6f77fff700 1 -- 172.21.15.47:0/161029889 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f6f8001dce0 con 0x7f6f900fcab0 2024-02-23T00:41:03.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.362+0000 7f6f77fff700 1 --2- 172.21.15.47:0/161029889 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6f78062cc0 0x7f6f78065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:03.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.362+0000 7f6f77fff700 1 -- 172.21.15.47:0/161029889 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 38..89) v4 ==== 7421+0+0 (secure 0 0 0) 0x7f6f8007e150 con 0x7f6f900fcab0 2024-02-23T00:41:03.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.362+0000 7f6f77fff700 10 client.14661.objecter ms_dispatch 0x7f6f90001550 osd_map(89..89 src has 38..89) v4 2024-02-23T00:41:03.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.362+0000 7f6f77fff700 3 client.14661.objecter handle_osd_map got epochs [89,89] > 0 2024-02-23T00:41:03.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.362+0000 7f6f77fff700 3 client.14661.objecter handle_osd_map decoding full epoch 89 2024-02-23T00:41:03.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.362+0000 7f6f8effd700 1 --2- 172.21.15.47:0/161029889 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6f78062cc0 0x7f6f78065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:03.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.363+0000 7f6f77fff700 20 client.14661.objecter dump_active .. 0 homeless 2024-02-23T00:41:03.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.363+0000 7f6f77fff700 10 client.14661.objecter ms_handle_connect 0x7f6f78062cc0 2024-02-23T00:41:03.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.363+0000 7f6f8effd700 1 --2- 172.21.15.47:0/161029889 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6f78062cc0 0x7f6f78065180 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f6f84004d30 tx=0x7f6f8400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:03.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.367+0000 7f6f77fff700 1 -- 172.21.15.47:0/161029889 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f6f8004deb0 con 0x7f6f900fcab0 2024-02-23T00:41:03.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.540+0000 7f6f956cb700 1 -- 172.21.15.47:0/161029889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6f90105df0 con 0x7f6f900fcab0 2024-02-23T00:41:03.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.540+0000 7f6f77fff700 1 -- 172.21.15.47:0/161029889 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6f800514d0 con 0x7f6f900fcab0 2024-02-23T00:41:03.543 INFO:teuthology.orchestra.run.smithi047.stdout:34359738431 2024-02-23T00:41:03.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.542+0000 7f6f956cb700 10 librados: watch_flush enter 2024-02-23T00:41:03.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.542+0000 7f6f956cb700 10 librados: watch_flush exit 2024-02-23T00:41:03.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.542+0000 7f6f956cb700 20 client.14661.objecter shutdown clearing up homeless session... 2024-02-23T00:41:03.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.542+0000 7f6f956cb700 10 client.14661.objecter successfully canceled tick 2024-02-23T00:41:03.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.542+0000 7f6f956cb700 1 -- 172.21.15.47:0/161029889 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6f78062cc0 msgr2=0x7f6f78065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:03.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.542+0000 7f6f956cb700 1 --2- 172.21.15.47:0/161029889 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6f78062cc0 0x7f6f78065180 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f6f84004d30 tx=0x7f6f8400b040).stop 2024-02-23T00:41:03.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.542+0000 7f6f956cb700 1 -- 172.21.15.47:0/161029889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f900fcab0 msgr2=0x7f6f900f8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:03.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.542+0000 7f6f956cb700 1 --2- 172.21.15.47:0/161029889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f900fcab0 0x7f6f900f8da0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f6f80009da0 tx=0x7f6f80009cb0).stop 2024-02-23T00:41:03.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.542+0000 7f6f956cb700 1 -- 172.21.15.47:0/161029889 shutdown_connections 2024-02-23T00:41:03.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.542+0000 7f6f956cb700 1 --2- 172.21.15.47:0/161029889 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6f78062cc0 0x7f6f78065180 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:03.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.542+0000 7f6f956cb700 1 --2- 172.21.15.47:0/161029889 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6f900fd410 0x7f6f900f9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:03.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.542+0000 7f6f956cb700 1 --2- 172.21.15.47:0/161029889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f900f7320 0x7f6f900f88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:03.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.542+0000 7f6f956cb700 1 --2- 172.21.15.47:0/161029889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f900fcab0 0x7f6f900f8da0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:03.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.542+0000 7f6f956cb700 1 -- 172.21.15.47:0/161029889 >> 172.21.15.47:0/161029889 conn(0x7f6f900f26e0 msgr2=0x7f6f90060ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:41:03.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.542+0000 7f6f956cb700 1 -- 172.21.15.47:0/161029889 shutdown_connections 2024-02-23T00:41:03.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.542+0000 7f6f956cb700 1 -- 172.21.15.47:0/161029889 wait complete. 2024-02-23T00:41:03.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:03.542+0000 7f6f956cb700 1 librados: shutdown 2024-02-23T00:41:04.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:02.036493+0000 osd.4 (osd.4) 29 : cluster [DBG] 4.d scrub starts 2024-02-23T00:41:04.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:02.038621+0000 osd.4 (osd.4) 30 : cluster [DBG] 4.d scrub ok 2024-02-23T00:41:04.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:02.375857+0000 osd.2 (osd.2) 31 : cluster [DBG] 5.a scrub starts 2024-02-23T00:41:04.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:02.376800+0000 osd.2 (osd.2) 32 : cluster [DBG] 5.a scrub ok 2024-02-23T00:41:04.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:02.711131+0000 osd.1 (osd.1) 45 : cluster [DBG] 6.4 scrub starts 2024-02-23T00:41:04.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:02.714016+0000 osd.1 (osd.1) 46 : cluster [DBG] 6.4 scrub ok 2024-02-23T00:41:04.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:41:02.826357+0000 mon.a (mon.0) 738 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:41:04.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:02.852061+0000 osd.3 (osd.3) 27 : cluster [DBG] 2.1e scrub starts 2024-02-23T00:41:04.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:02.853324+0000 osd.3 (osd.3) 28 : cluster [DBG] 2.1e scrub ok 2024-02-23T00:41:04.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:03.143081+0000 mgr.y (mgr.14517) 73 : cluster [DBG] pgmap v60: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:41:04.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:41:03.202848+0000 mon.a (mon.0) 739 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:41:04.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:41:03.203466+0000 mon.a (mon.0) 740 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:41:04.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:41:03.208621+0000 mon.a (mon.0) 741 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:41:04.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:41:03.541845+0000 mon.a (mon.0) 742 : audit [DBG] from='client.? 172.21.15.47:0/161029889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-23T00:41:04.195 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:02.036493+0000 osd.4 (osd.4) 29 : cluster [DBG] 4.d scrub starts 2024-02-23T00:41:04.195 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:02.038621+0000 osd.4 (osd.4) 30 : cluster [DBG] 4.d scrub ok 2024-02-23T00:41:04.195 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:02.375857+0000 osd.2 (osd.2) 31 : cluster [DBG] 5.a scrub starts 2024-02-23T00:41:04.195 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:02.376800+0000 osd.2 (osd.2) 32 : cluster [DBG] 5.a scrub ok 2024-02-23T00:41:04.195 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:02.711131+0000 osd.1 (osd.1) 45 : cluster [DBG] 6.4 scrub starts 2024-02-23T00:41:04.195 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:02.714016+0000 osd.1 (osd.1) 46 : cluster [DBG] 6.4 scrub ok 2024-02-23T00:41:04.195 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:41:02.826357+0000 mon.a (mon.0) 738 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:41:04.195 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:02.852061+0000 osd.3 (osd.3) 27 : cluster [DBG] 2.1e scrub starts 2024-02-23T00:41:04.195 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:02.853324+0000 osd.3 (osd.3) 28 : cluster [DBG] 2.1e scrub ok 2024-02-23T00:41:04.195 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:03.143081+0000 mgr.y (mgr.14517) 73 : cluster [DBG] pgmap v60: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:41:04.195 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:41:03.202848+0000 mon.a (mon.0) 739 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:41:04.195 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:41:03.203466+0000 mon.a (mon.0) 740 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:41:04.196 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:41:03.208621+0000 mon.a (mon.0) 741 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:41:04.196 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:41:03.541845+0000 mon.a (mon.0) 742 : audit [DBG] from='client.? 172.21.15.47:0/161029889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-23T00:41:04.196 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:02.036493+0000 osd.4 (osd.4) 29 : cluster [DBG] 4.d scrub starts 2024-02-23T00:41:04.196 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:02.038621+0000 osd.4 (osd.4) 30 : cluster [DBG] 4.d scrub ok 2024-02-23T00:41:04.196 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:02.375857+0000 osd.2 (osd.2) 31 : cluster [DBG] 5.a scrub starts 2024-02-23T00:41:04.196 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:02.376800+0000 osd.2 (osd.2) 32 : cluster [DBG] 5.a scrub ok 2024-02-23T00:41:04.196 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:02.711131+0000 osd.1 (osd.1) 45 : cluster [DBG] 6.4 scrub starts 2024-02-23T00:41:04.196 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:02.714016+0000 osd.1 (osd.1) 46 : cluster [DBG] 6.4 scrub ok 2024-02-23T00:41:04.196 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:41:02.826357+0000 mon.a (mon.0) 738 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:41:04.196 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:02.852061+0000 osd.3 (osd.3) 27 : cluster [DBG] 2.1e scrub starts 2024-02-23T00:41:04.197 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:02.853324+0000 osd.3 (osd.3) 28 : cluster [DBG] 2.1e scrub ok 2024-02-23T00:41:04.197 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:03.143081+0000 mgr.y (mgr.14517) 73 : cluster [DBG] pgmap v60: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:41:04.197 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:41:03.202848+0000 mon.a (mon.0) 739 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:41:04.197 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:41:03.203466+0000 mon.a (mon.0) 740 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:41:04.197 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:41:03.208621+0000 mon.a (mon.0) 741 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:41:04.197 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:41:03.541845+0000 mon.a (mon.0) 742 : audit [DBG] from='client.? 172.21.15.47:0/161029889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-23T00:41:04.615 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738425 got 34359738431 for osd.0 2024-02-23T00:41:04.616 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-23T00:41:04.901 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:41:05.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:04.836584+0000 osd.5 (osd.5) 35 : cluster [DBG] 5.f deep-scrub starts 2024-02-23T00:41:05.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:04.839550+0000 osd.5 (osd.5) 36 : cluster [DBG] 5.f deep-scrub ok 2024-02-23T00:41:05.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:04.836584+0000 osd.5 (osd.5) 35 : cluster [DBG] 5.f deep-scrub starts 2024-02-23T00:41:05.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:04.839550+0000 osd.5 (osd.5) 36 : cluster [DBG] 5.f deep-scrub ok 2024-02-23T00:41:05.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:04.836584+0000 osd.5 (osd.5) 35 : cluster [DBG] 5.f deep-scrub starts 2024-02-23T00:41:05.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:04.839550+0000 osd.5 (osd.5) 36 : cluster [DBG] 5.f deep-scrub ok 2024-02-23T00:41:06.511 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:05.000192+0000 osd.4 (osd.4) 31 : cluster [DBG] 4.1 scrub starts 2024-02-23T00:41:06.511 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:05.001902+0000 osd.4 (osd.4) 32 : cluster [DBG] 4.1 scrub ok 2024-02-23T00:41:06.511 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:05.143389+0000 mgr.y (mgr.14517) 74 : cluster [DBG] pgmap v61: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:41:06.512 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:05.000192+0000 osd.4 (osd.4) 31 : cluster [DBG] 4.1 scrub starts 2024-02-23T00:41:06.512 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:05.001902+0000 osd.4 (osd.4) 32 : cluster [DBG] 4.1 scrub ok 2024-02-23T00:41:06.512 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:05.143389+0000 mgr.y (mgr.14517) 74 : cluster [DBG] pgmap v61: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:41:06.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:05.000192+0000 osd.4 (osd.4) 31 : cluster [DBG] 4.1 scrub starts 2024-02-23T00:41:06.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:05.001902+0000 osd.4 (osd.4) 32 : cluster [DBG] 4.1 scrub ok 2024-02-23T00:41:06.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:05.143389+0000 mgr.y (mgr.14517) 74 : cluster [DBG] pgmap v61: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:41:06.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.755+0000 7f3f2014c700 1 -- 172.21.15.47:0/510957980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f180fe510 msgr2=0x7f3f180f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.755+0000 7f3f2014c700 1 --2- 172.21.15.47:0/510957980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f180fe510 0x7f3f180f7d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f3f08004870 tx=0x7f3f08009cb0).stop 2024-02-23T00:41:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.755+0000 7f3f2014c700 1 -- 172.21.15.47:0/510957980 shutdown_connections 2024-02-23T00:41:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.755+0000 7f3f2014c700 1 --2- 172.21.15.47:0/510957980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f180f8420 0x7f3f18104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.755+0000 7f3f2014c700 1 --2- 172.21.15.47:0/510957980 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3f180fdbb0 0x7f3f180fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.755+0000 7f3f2014c700 1 --2- 172.21.15.47:0/510957980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f180fe510 0x7f3f180f7d40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.755+0000 7f3f2014c700 1 -- 172.21.15.47:0/510957980 >> 172.21.15.47:0/510957980 conn(0x7f3f180f3800 msgr2=0x7f3f180f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:41:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.755+0000 7f3f2014c700 1 -- 172.21.15.47:0/510957980 shutdown_connections 2024-02-23T00:41:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.755+0000 7f3f2014c700 1 -- 172.21.15.47:0/510957980 wait complete. 2024-02-23T00:41:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.755+0000 7f3f2014c700 1 librados: starting msgr at 2024-02-23T00:41:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.755+0000 7f3f2014c700 1 librados: starting objecter 2024-02-23T00:41:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.756+0000 7f3f2014c700 1 Processor -- start 2024-02-23T00:41:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.756+0000 7f3f2014c700 1 -- start start 2024-02-23T00:41:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.756+0000 7f3f2014c700 1 librados: setting wanted keys 2024-02-23T00:41:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.756+0000 7f3f2014c700 1 librados: calling monclient init 2024-02-23T00:41:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.756+0000 7f3f2014c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f180f8420 0x7f3f1806b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.756+0000 7f3f2014c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f180fdbb0 0x7f3f1806b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.756+0000 7f3f2014c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3f180fe510 0x7f3f1806bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.756+0000 7f3f2014c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f181163c0 con 0x7f3f180fdbb0 2024-02-23T00:41:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.756+0000 7f3f2014c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f18116530 con 0x7f3f180f8420 2024-02-23T00:41:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.756+0000 7f3f2014c700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f181166f0 con 0x7f3f180fe510 2024-02-23T00:41:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.756+0000 7f3f1d6e7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f180fdbb0 0x7f3f1806b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.756+0000 7f3f1e6e9700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3f180fe510 0x7f3f1806bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.756+0000 7f3f1d6e7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f180fdbb0 0x7f3f1806b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58314/0 (socket says 172.21.15.47:58314) 2024-02-23T00:41:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.756+0000 7f3f1d6e7700 1 -- 172.21.15.47:0/1944621994 learned_addr learned my addr 172.21.15.47:0/1944621994 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:41:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.756+0000 7f3f1dee8700 1 --2- 172.21.15.47:0/1944621994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f180f8420 0x7f3f1806b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.756+0000 7f3f1d6e7700 1 -- 172.21.15.47:0/1944621994 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3f180fe510 msgr2=0x7f3f1806bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.756+0000 7f3f1d6e7700 1 --2- 172.21.15.47:0/1944621994 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3f180fe510 0x7f3f1806bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.756+0000 7f3f1d6e7700 1 -- 172.21.15.47:0/1944621994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f180f8420 msgr2=0x7f3f1806b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.756+0000 7f3f1d6e7700 1 --2- 172.21.15.47:0/1944621994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f180f8420 0x7f3f1806b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.756+0000 7f3f1d6e7700 1 -- 172.21.15.47:0/1944621994 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f14005040 con 0x7f3f180fdbb0 2024-02-23T00:41:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.756+0000 7f3f1e6e9700 1 --2- 172.21.15.47:0/1944621994 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3f180fe510 0x7f3f1806bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:41:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.756+0000 7f3f1d6e7700 1 --2- 172.21.15.47:0/1944621994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f180fdbb0 0x7f3f1806b5c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f3f08004320 tx=0x7f3f08009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.756+0000 7f3f0effd700 10 client.?.objecter ms_handle_connect 0x7f3f180fdbb0 2024-02-23T00:41:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.756+0000 7f3f0effd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:41:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.757+0000 7f3f0effd700 1 -- 172.21.15.47:0/1944621994 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0800e030 con 0x7f3f180fdbb0 2024-02-23T00:41:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.757+0000 7f3f0effd700 1 -- 172.21.15.47:0/1944621994 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f3f08013d70 con 0x7f3f180fdbb0 2024-02-23T00:41:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.757+0000 7f3f2014c700 1 -- 172.21.15.47:0/1944621994 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f08005040 con 0x7f3f180fdbb0 2024-02-23T00:41:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.757+0000 7f3f0effd700 1 -- 172.21.15.47:0/1944621994 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f08018070 con 0x7f3f180fdbb0 2024-02-23T00:41:06.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.757+0000 7f3f2014c700 10 client.14667.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:41:06.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.757+0000 7f3f2014c700 1 -- 172.21.15.47:0/1944621994 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f18116c70 con 0x7f3f180fdbb0 2024-02-23T00:41:06.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.757+0000 7f3f2014c700 1 librados: init done 2024-02-23T00:41:06.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.758+0000 7f3f2014c700 1 -- 172.21.15.47:0/1944621994 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f1805ee80 con 0x7f3f180fdbb0 2024-02-23T00:41:06.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.758+0000 7f3f0effd700 1 -- 172.21.15.47:0/1944621994 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f3f08009240 con 0x7f3f180fdbb0 2024-02-23T00:41:06.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.758+0000 7f3f0effd700 1 --2- 172.21.15.47:0/1944621994 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3f04062cc0 0x7f3f04065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:06.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.759+0000 7f3f0effd700 1 -- 172.21.15.47:0/1944621994 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 38..89) v4 ==== 7421+0+0 (secure 0 0 0) 0x7f3f0807e0b0 con 0x7f3f180fdbb0 2024-02-23T00:41:06.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.759+0000 7f3f0effd700 10 client.14667.objecter ms_dispatch 0x7f3f18001550 osd_map(89..89 src has 38..89) v4 2024-02-23T00:41:06.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.759+0000 7f3f0effd700 3 client.14667.objecter handle_osd_map got epochs [89,89] > 0 2024-02-23T00:41:06.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.759+0000 7f3f0effd700 3 client.14667.objecter handle_osd_map decoding full epoch 89 2024-02-23T00:41:06.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.759+0000 7f3f1dee8700 1 --2- 172.21.15.47:0/1944621994 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3f04062cc0 0x7f3f04065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:06.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.759+0000 7f3f0effd700 20 client.14667.objecter dump_active .. 0 homeless 2024-02-23T00:41:06.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.759+0000 7f3f0effd700 10 client.14667.objecter ms_handle_connect 0x7f3f04062cc0 2024-02-23T00:41:06.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.759+0000 7f3f1dee8700 1 --2- 172.21.15.47:0/1944621994 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3f04062cc0 0x7f3f04065180 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f3f14009350 tx=0x7f3f1400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:06.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.763+0000 7f3f0effd700 1 -- 172.21.15.47:0/1944621994 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f3f0804de10 con 0x7f3f180fdbb0 2024-02-23T00:41:06.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.936+0000 7f3f2014c700 1 -- 172.21.15.47:0/1944621994 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3f1805fa60 con 0x7f3f180fdbb0 2024-02-23T00:41:06.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.937+0000 7f3f0effd700 1 -- 172.21.15.47:0/1944621994 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3f08051430 con 0x7f3f180fdbb0 2024-02-23T00:41:06.940 INFO:teuthology.orchestra.run.smithi047.stdout:55834574906 2024-02-23T00:41:06.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.938+0000 7f3f2014c700 10 librados: watch_flush enter 2024-02-23T00:41:06.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.939+0000 7f3f2014c700 10 librados: watch_flush exit 2024-02-23T00:41:06.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.939+0000 7f3f2014c700 20 client.14667.objecter shutdown clearing up homeless session... 2024-02-23T00:41:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.939+0000 7f3f2014c700 10 client.14667.objecter successfully canceled tick 2024-02-23T00:41:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.939+0000 7f3f2014c700 1 -- 172.21.15.47:0/1944621994 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3f04062cc0 msgr2=0x7f3f04065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.939+0000 7f3f2014c700 1 --2- 172.21.15.47:0/1944621994 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3f04062cc0 0x7f3f04065180 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f3f14009350 tx=0x7f3f1400b040).stop 2024-02-23T00:41:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.939+0000 7f3f2014c700 1 -- 172.21.15.47:0/1944621994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f180fdbb0 msgr2=0x7f3f1806b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.939+0000 7f3f2014c700 1 --2- 172.21.15.47:0/1944621994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f180fdbb0 0x7f3f1806b5c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f3f08004320 tx=0x7f3f08009cb0).stop 2024-02-23T00:41:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.939+0000 7f3f2014c700 1 -- 172.21.15.47:0/1944621994 shutdown_connections 2024-02-23T00:41:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.939+0000 7f3f2014c700 1 --2- 172.21.15.47:0/1944621994 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3f04062cc0 0x7f3f04065180 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.939+0000 7f3f2014c700 1 --2- 172.21.15.47:0/1944621994 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3f180fe510 0x7f3f1806bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.939+0000 7f3f2014c700 1 --2- 172.21.15.47:0/1944621994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f180f8420 0x7f3f1806b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.939+0000 7f3f2014c700 1 --2- 172.21.15.47:0/1944621994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f180fdbb0 0x7f3f1806b5c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.939+0000 7f3f2014c700 1 -- 172.21.15.47:0/1944621994 >> 172.21.15.47:0/1944621994 conn(0x7f3f180f3800 msgr2=0x7f3f180fb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:41:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.939+0000 7f3f2014c700 1 -- 172.21.15.47:0/1944621994 shutdown_connections 2024-02-23T00:41:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.939+0000 7f3f2014c700 1 -- 172.21.15.47:0/1944621994 wait complete. 2024-02-23T00:41:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:06.939+0000 7f3f2014c700 1 librados: shutdown 2024-02-23T00:41:07.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:05.801566+0000 osd.0 (osd.0) 27 : cluster [DBG] 6.9 scrub starts 2024-02-23T00:41:07.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:05.802773+0000 osd.0 (osd.0) 28 : cluster [DBG] 6.9 scrub ok 2024-02-23T00:41:07.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:05.964358+0000 osd.4 (osd.4) 33 : cluster [DBG] 5.18 scrub starts 2024-02-23T00:41:07.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:05.965522+0000 osd.4 (osd.4) 34 : cluster [DBG] 5.18 scrub ok 2024-02-23T00:41:07.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:41:06.938249+0000 mon.a (mon.0) 743 : audit [DBG] from='client.? 172.21.15.47:0/1944621994' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-23T00:41:07.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:05.801566+0000 osd.0 (osd.0) 27 : cluster [DBG] 6.9 scrub starts 2024-02-23T00:41:07.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:05.802773+0000 osd.0 (osd.0) 28 : cluster [DBG] 6.9 scrub ok 2024-02-23T00:41:07.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:05.964358+0000 osd.4 (osd.4) 33 : cluster [DBG] 5.18 scrub starts 2024-02-23T00:41:07.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:05.965522+0000 osd.4 (osd.4) 34 : cluster [DBG] 5.18 scrub ok 2024-02-23T00:41:07.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:41:06.938249+0000 mon.a (mon.0) 743 : audit [DBG] from='client.? 172.21.15.47:0/1944621994' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-23T00:41:07.667 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574900 got 55834574906 for osd.1 2024-02-23T00:41:07.668 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-23T00:41:07.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:05.801566+0000 osd.0 (osd.0) 27 : cluster [DBG] 6.9 scrub starts 2024-02-23T00:41:07.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:05.802773+0000 osd.0 (osd.0) 28 : cluster [DBG] 6.9 scrub ok 2024-02-23T00:41:07.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:05.964358+0000 osd.4 (osd.4) 33 : cluster [DBG] 5.18 scrub starts 2024-02-23T00:41:07.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:05.965522+0000 osd.4 (osd.4) 34 : cluster [DBG] 5.18 scrub ok 2024-02-23T00:41:07.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:41:06.938249+0000 mon.a (mon.0) 743 : audit [DBG] from='client.? 172.21.15.47:0/1944621994' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-23T00:41:07.947 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:41:08.509 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:07.144067+0000 mgr.y (mgr.14517) 75 : cluster [DBG] pgmap v62: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:41:08.509 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:07.776503+0000 osd.5 (osd.5) 37 : cluster [DBG] 5.1b scrub starts 2024-02-23T00:41:08.509 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:07.777430+0000 osd.5 (osd.5) 38 : cluster [DBG] 5.1b scrub ok 2024-02-23T00:41:08.509 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:07.144067+0000 mgr.y (mgr.14517) 75 : cluster [DBG] pgmap v62: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:41:08.510 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:07.776503+0000 osd.5 (osd.5) 37 : cluster [DBG] 5.1b scrub starts 2024-02-23T00:41:08.510 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:07.777430+0000 osd.5 (osd.5) 38 : cluster [DBG] 5.1b scrub ok 2024-02-23T00:41:08.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:07.144067+0000 mgr.y (mgr.14517) 75 : cluster [DBG] pgmap v62: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:41:08.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:07.776503+0000 osd.5 (osd.5) 37 : cluster [DBG] 5.1b scrub starts 2024-02-23T00:41:08.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:07.777430+0000 osd.5 (osd.5) 38 : cluster [DBG] 5.1b scrub ok 2024-02-23T00:41:09.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:08.002280+0000 osd.4 (osd.4) 35 : cluster [DBG] 4.9 scrub starts 2024-02-23T00:41:09.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:08.004484+0000 osd.4 (osd.4) 36 : cluster [DBG] 4.9 scrub ok 2024-02-23T00:41:09.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:08.781501+0000 osd.5 (osd.5) 39 : cluster [DBG] 5.15 scrub starts 2024-02-23T00:41:09.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:08.784961+0000 osd.5 (osd.5) 40 : cluster [DBG] 5.15 scrub ok 2024-02-23T00:41:09.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:08.002280+0000 osd.4 (osd.4) 35 : cluster [DBG] 4.9 scrub starts 2024-02-23T00:41:09.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:08.004484+0000 osd.4 (osd.4) 36 : cluster [DBG] 4.9 scrub ok 2024-02-23T00:41:09.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:08.781501+0000 osd.5 (osd.5) 39 : cluster [DBG] 5.15 scrub starts 2024-02-23T00:41:09.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:08.784961+0000 osd.5 (osd.5) 40 : cluster [DBG] 5.15 scrub ok 2024-02-23T00:41:09.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:08.002280+0000 osd.4 (osd.4) 35 : cluster [DBG] 4.9 scrub starts 2024-02-23T00:41:09.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:08.004484+0000 osd.4 (osd.4) 36 : cluster [DBG] 4.9 scrub ok 2024-02-23T00:41:09.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:08.781501+0000 osd.5 (osd.5) 39 : cluster [DBG] 5.15 scrub starts 2024-02-23T00:41:09.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:08.784961+0000 osd.5 (osd.5) 40 : cluster [DBG] 5.15 scrub ok 2024-02-23T00:41:09.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.965+0000 7f6c52d79700 1 -- 172.21.15.47:0/3952184915 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6c4c0fdbb0 msgr2=0x7f6c4c0fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:09.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.965+0000 7f6c52d79700 1 --2- 172.21.15.47:0/3952184915 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6c4c0fdbb0 0x7f6c4c0fdf90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f6c3c004870 tx=0x7f6c3c009cb0).stop 2024-02-23T00:41:09.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.965+0000 7f6c52d79700 1 -- 172.21.15.47:0/3952184915 shutdown_connections 2024-02-23T00:41:09.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.965+0000 7f6c52d79700 1 --2- 172.21.15.47:0/3952184915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c4c0f8420 0x7f6c4c104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:09.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.965+0000 7f6c52d79700 1 --2- 172.21.15.47:0/3952184915 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6c4c0fdbb0 0x7f6c4c0fdf90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:09.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.965+0000 7f6c52d79700 1 --2- 172.21.15.47:0/3952184915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c4c0fe510 0x7f6c4c0f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:09.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.965+0000 7f6c52d79700 1 -- 172.21.15.47:0/3952184915 >> 172.21.15.47:0/3952184915 conn(0x7f6c4c0f3800 msgr2=0x7f6c4c0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:41:09.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.965+0000 7f6c52d79700 1 -- 172.21.15.47:0/3952184915 shutdown_connections 2024-02-23T00:41:09.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.965+0000 7f6c52d79700 1 -- 172.21.15.47:0/3952184915 wait complete. 2024-02-23T00:41:09.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.966+0000 7f6c52d79700 1 librados: starting msgr at 2024-02-23T00:41:09.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.966+0000 7f6c52d79700 1 librados: starting objecter 2024-02-23T00:41:09.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.966+0000 7f6c52d79700 1 Processor -- start 2024-02-23T00:41:09.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.966+0000 7f6c52d79700 1 -- start start 2024-02-23T00:41:09.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.966+0000 7f6c52d79700 1 librados: setting wanted keys 2024-02-23T00:41:09.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.966+0000 7f6c52d79700 1 librados: calling monclient init 2024-02-23T00:41:09.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.966+0000 7f6c52d79700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6c4c0f8420 0x7f6c4c10de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:09.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.966+0000 7f6c52d79700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c4c0fdbb0 0x7f6c4c10e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:09.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.966+0000 7f6c52d79700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c4c0fe510 0x7f6c4c108020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:09.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.966+0000 7f6c52d79700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c108510 con 0x7f6c4c0fe510 2024-02-23T00:41:09.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.966+0000 7f6c52d79700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c108680 con 0x7f6c4c0fdbb0 2024-02-23T00:41:09.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.966+0000 7f6c52d79700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c108840 con 0x7f6c4c0f8420 2024-02-23T00:41:09.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.966+0000 7f6c50b15700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6c4c0f8420 0x7f6c4c10de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:09.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.966+0000 7f6c51316700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c4c0fe510 0x7f6c4c108020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:09.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.966+0000 7f6c50b15700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6c4c0f8420 0x7f6c4c10de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:45378/0 (socket says 172.21.15.47:45378) 2024-02-23T00:41:09.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.966+0000 7f6c50b15700 1 -- 172.21.15.47:0/1586657934 learned_addr learned my addr 172.21.15.47:0/1586657934 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:41:09.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.966+0000 7f6c51316700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c4c0fe510 0x7f6c4c108020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58338/0 (socket says 172.21.15.47:58338) 2024-02-23T00:41:09.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.966+0000 7f6c4bfff700 1 --2- 172.21.15.47:0/1586657934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c4c0fdbb0 0x7f6c4c10e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:09.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.966+0000 7f6c51316700 1 -- 172.21.15.47:0/1586657934 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6c4c0f8420 msgr2=0x7f6c4c10de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:09.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.966+0000 7f6c51316700 1 --2- 172.21.15.47:0/1586657934 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6c4c0f8420 0x7f6c4c10de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:09.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.966+0000 7f6c51316700 1 -- 172.21.15.47:0/1586657934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c4c0fdbb0 msgr2=0x7f6c4c10e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:09.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.966+0000 7f6c51316700 1 --2- 172.21.15.47:0/1586657934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c4c0fdbb0 0x7f6c4c10e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:09.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.966+0000 7f6c51316700 1 -- 172.21.15.47:0/1586657934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c440097c0 con 0x7f6c4c0fe510 2024-02-23T00:41:09.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.966+0000 7f6c50b15700 1 --2- 172.21.15.47:0/1586657934 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6c4c0f8420 0x7f6c4c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:41:09.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.967+0000 7f6c51316700 1 --2- 172.21.15.47:0/1586657934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c4c0fe510 0x7f6c4c108020 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f6c44001cb0 tx=0x7f6c44004c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:09.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.967+0000 7f6c49ffb700 10 client.?.objecter ms_handle_connect 0x7f6c4c0fe510 2024-02-23T00:41:09.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.967+0000 7f6c49ffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:41:09.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.967+0000 7f6c49ffb700 1 -- 172.21.15.47:0/1586657934 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4400fa30 con 0x7f6c4c0fe510 2024-02-23T00:41:09.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.967+0000 7f6c49ffb700 1 -- 172.21.15.47:0/1586657934 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f6c440090d0 con 0x7f6c4c0fe510 2024-02-23T00:41:09.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.967+0000 7f6c52d79700 1 -- 172.21.15.47:0/1586657934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c3c005040 con 0x7f6c4c0fe510 2024-02-23T00:41:09.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.967+0000 7f6c49ffb700 1 -- 172.21.15.47:0/1586657934 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c44007b40 con 0x7f6c4c0fe510 2024-02-23T00:41:09.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.967+0000 7f6c52d79700 10 client.14673.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:41:09.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.967+0000 7f6c52d79700 1 -- 172.21.15.47:0/1586657934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c4c108e30 con 0x7f6c4c0fe510 2024-02-23T00:41:09.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.967+0000 7f6c52d79700 1 librados: init done 2024-02-23T00:41:09.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.968+0000 7f6c52d79700 1 -- 172.21.15.47:0/1586657934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c4c05ee80 con 0x7f6c4c0fe510 2024-02-23T00:41:09.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.968+0000 7f6c49ffb700 1 -- 172.21.15.47:0/1586657934 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f6c44009240 con 0x7f6c4c0fe510 2024-02-23T00:41:09.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.969+0000 7f6c49ffb700 1 --2- 172.21.15.47:0/1586657934 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6c38062cc0 0x7f6c38065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:09.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.969+0000 7f6c49ffb700 1 -- 172.21.15.47:0/1586657934 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 38..89) v4 ==== 7421+0+0 (secure 0 0 0) 0x7f6c4407d850 con 0x7f6c4c0fe510 2024-02-23T00:41:09.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.969+0000 7f6c49ffb700 10 client.14673.objecter ms_dispatch 0x7f6c4c001550 osd_map(89..89 src has 38..89) v4 2024-02-23T00:41:09.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.969+0000 7f6c49ffb700 3 client.14673.objecter handle_osd_map got epochs [89,89] > 0 2024-02-23T00:41:09.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.969+0000 7f6c49ffb700 3 client.14673.objecter handle_osd_map decoding full epoch 89 2024-02-23T00:41:09.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.969+0000 7f6c50b15700 1 --2- 172.21.15.47:0/1586657934 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6c38062cc0 0x7f6c38065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:09.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.969+0000 7f6c49ffb700 20 client.14673.objecter dump_active .. 0 homeless 2024-02-23T00:41:09.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.969+0000 7f6c49ffb700 10 client.14673.objecter ms_handle_connect 0x7f6c38062cc0 2024-02-23T00:41:09.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.969+0000 7f6c50b15700 1 --2- 172.21.15.47:0/1586657934 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6c38062cc0 0x7f6c38065180 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f6c3c004ee0 tx=0x7f6c3c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:09.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:09.973+0000 7f6c49ffb700 1 -- 172.21.15.47:0/1586657934 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f6c4404d5b0 con 0x7f6c4c0fe510 2024-02-23T00:41:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:10.145+0000 7f6c52d79700 1 -- 172.21.15.47:0/1586657934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6c4c109620 con 0x7f6c4c0fe510 2024-02-23T00:41:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:10.145+0000 7f6c49ffb700 1 -- 172.21.15.47:0/1586657934 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6c4c109620 con 0x7f6c4c0fe510 2024-02-23T00:41:10.148 INFO:teuthology.orchestra.run.smithi047.stdout:77309411382 2024-02-23T00:41:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:10.147+0000 7f6c52d79700 10 librados: watch_flush enter 2024-02-23T00:41:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:10.147+0000 7f6c52d79700 10 librados: watch_flush exit 2024-02-23T00:41:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:10.147+0000 7f6c52d79700 20 client.14673.objecter shutdown clearing up homeless session... 2024-02-23T00:41:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:10.147+0000 7f6c52d79700 10 client.14673.objecter successfully canceled tick 2024-02-23T00:41:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:10.147+0000 7f6c52d79700 1 -- 172.21.15.47:0/1586657934 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6c38062cc0 msgr2=0x7f6c38065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:10.147+0000 7f6c52d79700 1 --2- 172.21.15.47:0/1586657934 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6c38062cc0 0x7f6c38065180 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f6c3c004ee0 tx=0x7f6c3c00b040).stop 2024-02-23T00:41:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:10.147+0000 7f6c52d79700 1 -- 172.21.15.47:0/1586657934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c4c0fe510 msgr2=0x7f6c4c108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:10.147+0000 7f6c52d79700 1 --2- 172.21.15.47:0/1586657934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c4c0fe510 0x7f6c4c108020 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f6c44001cb0 tx=0x7f6c44004c50).stop 2024-02-23T00:41:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:10.147+0000 7f6c52d79700 1 -- 172.21.15.47:0/1586657934 shutdown_connections 2024-02-23T00:41:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:10.147+0000 7f6c52d79700 1 --2- 172.21.15.47:0/1586657934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c4c0fe510 0x7f6c4c108020 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:10.147+0000 7f6c52d79700 1 --2- 172.21.15.47:0/1586657934 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6c38062cc0 0x7f6c38065180 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:10.147+0000 7f6c52d79700 1 --2- 172.21.15.47:0/1586657934 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6c4c0f8420 0x7f6c4c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:10.147+0000 7f6c52d79700 1 --2- 172.21.15.47:0/1586657934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c4c0fdbb0 0x7f6c4c10e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:10.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:10.147+0000 7f6c52d79700 1 -- 172.21.15.47:0/1586657934 >> 172.21.15.47:0/1586657934 conn(0x7f6c4c0f3800 msgr2=0x7f6c4c0fb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:41:10.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:10.148+0000 7f6c52d79700 1 -- 172.21.15.47:0/1586657934 shutdown_connections 2024-02-23T00:41:10.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:10.148+0000 7f6c52d79700 1 -- 172.21.15.47:0/1586657934 wait complete. 2024-02-23T00:41:10.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:10.148+0000 7f6c52d79700 1 librados: shutdown 2024-02-23T00:41:10.455 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:08.905992+0000 osd.0 (osd.0) 29 : cluster [DBG] 6.18 scrub starts 2024-02-23T00:41:10.455 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:08.906924+0000 osd.0 (osd.0) 30 : cluster [DBG] 6.18 scrub ok 2024-02-23T00:41:10.455 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:09.144434+0000 mgr.y (mgr.14517) 76 : cluster [DBG] pgmap v63: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:41:10.455 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:09.878555+0000 osd.3 (osd.3) 29 : cluster [DBG] 5.0 scrub starts 2024-02-23T00:41:10.455 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:09.882007+0000 osd.3 (osd.3) 30 : cluster [DBG] 5.0 scrub ok 2024-02-23T00:41:10.455 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:41:10.146850+0000 mon.a (mon.0) 744 : audit [DBG] from='client.? 172.21.15.47:0/1586657934' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-23T00:41:10.455 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:08.905992+0000 osd.0 (osd.0) 29 : cluster [DBG] 6.18 scrub starts 2024-02-23T00:41:10.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:08.906924+0000 osd.0 (osd.0) 30 : cluster [DBG] 6.18 scrub ok 2024-02-23T00:41:10.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:09.144434+0000 mgr.y (mgr.14517) 76 : cluster [DBG] pgmap v63: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:41:10.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:09.878555+0000 osd.3 (osd.3) 29 : cluster [DBG] 5.0 scrub starts 2024-02-23T00:41:10.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:09.882007+0000 osd.3 (osd.3) 30 : cluster [DBG] 5.0 scrub ok 2024-02-23T00:41:10.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:41:10.146850+0000 mon.a (mon.0) 744 : audit [DBG] from='client.? 172.21.15.47:0/1586657934' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-23T00:41:10.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:08.905992+0000 osd.0 (osd.0) 29 : cluster [DBG] 6.18 scrub starts 2024-02-23T00:41:10.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:08.906924+0000 osd.0 (osd.0) 30 : cluster [DBG] 6.18 scrub ok 2024-02-23T00:41:10.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:09.144434+0000 mgr.y (mgr.14517) 76 : cluster [DBG] pgmap v63: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:41:10.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:09.878555+0000 osd.3 (osd.3) 29 : cluster [DBG] 5.0 scrub starts 2024-02-23T00:41:10.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:09.882007+0000 osd.3 (osd.3) 30 : cluster [DBG] 5.0 scrub ok 2024-02-23T00:41:10.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:41:10.146850+0000 mon.a (mon.0) 744 : audit [DBG] from='client.? 172.21.15.47:0/1586657934' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-23T00:41:10.806 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:41:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:41:10] "GET /metrics HTTP/1.1" 200 172152 "" "Prometheus/2.33.4" 2024-02-23T00:41:11.046 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411375 got 77309411382 for osd.2 2024-02-23T00:41:11.046 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-23T00:41:11.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:10.903206+0000 osd.1 (osd.1) 47 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:41:11.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:10.903266+0000 osd.1 (osd.1) 48 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:41:11.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:10.903206+0000 osd.1 (osd.1) 47 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:41:11.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:10.903266+0000 osd.1 (osd.1) 48 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:41:11.336 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:41:11.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:10.903206+0000 osd.1 (osd.1) 47 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:41:11.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:10.903266+0000 osd.1 (osd.1) 48 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:41:12.443 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:41:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:41:12.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:10.890910+0000 osd.0 (osd.0) 31 : cluster [DBG] 6.10 scrub starts 2024-02-23T00:41:12.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:10.892249+0000 osd.0 (osd.0) 32 : cluster [DBG] 6.10 scrub ok 2024-02-23T00:41:12.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:11.144756+0000 mgr.y (mgr.14517) 77 : cluster [DBG] pgmap v64: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:41:12.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:11.221785+0000 mon.a (mon.0) 745 : cluster [DBG] osdmap e90: 8 total, 8 up, 8 in 2024-02-23T00:41:12.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:10.890910+0000 osd.0 (osd.0) 31 : cluster [DBG] 6.10 scrub starts 2024-02-23T00:41:12.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:10.892249+0000 osd.0 (osd.0) 32 : cluster [DBG] 6.10 scrub ok 2024-02-23T00:41:12.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:11.144756+0000 mgr.y (mgr.14517) 77 : cluster [DBG] pgmap v64: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:41:12.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:11.221785+0000 mon.a (mon.0) 745 : cluster [DBG] osdmap e90: 8 total, 8 up, 8 in 2024-02-23T00:41:12.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:10.890910+0000 osd.0 (osd.0) 31 : cluster [DBG] 6.10 scrub starts 2024-02-23T00:41:12.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:10.892249+0000 osd.0 (osd.0) 32 : cluster [DBG] 6.10 scrub ok 2024-02-23T00:41:12.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:11.144756+0000 mgr.y (mgr.14517) 77 : cluster [DBG] pgmap v64: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:41:12.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:11.221785+0000 mon.a (mon.0) 745 : cluster [DBG] osdmap e90: 8 total, 8 up, 8 in 2024-02-23T00:41:12.943 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:41:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:41:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:41:13.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.445+0000 7f925cf7e700 1 -- 172.21.15.47:0/4100766303 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92580fcac0 msgr2=0x7f92580fcea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:13.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.445+0000 7f925cf7e700 1 --2- 172.21.15.47:0/4100766303 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92580fcac0 0x7f92580fcea0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f9248004870 tx=0x7f9248009cb0).stop 2024-02-23T00:41:13.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.445+0000 7f925cf7e700 1 -- 172.21.15.47:0/4100766303 shutdown_connections 2024-02-23T00:41:13.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.445+0000 7f925cf7e700 1 --2- 172.21.15.47:0/4100766303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92580f7330 0x7f9258103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:13.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.445+0000 7f925cf7e700 1 --2- 172.21.15.47:0/4100766303 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92580fcac0 0x7f92580fcea0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:13.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.445+0000 7f925cf7e700 1 --2- 172.21.15.47:0/4100766303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92580fd420 0x7f92580f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:13.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.445+0000 7f925cf7e700 1 -- 172.21.15.47:0/4100766303 >> 172.21.15.47:0/4100766303 conn(0x7f92580f26f0 msgr2=0x7f92580f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:41:13.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.445+0000 7f925cf7e700 1 -- 172.21.15.47:0/4100766303 shutdown_connections 2024-02-23T00:41:13.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.445+0000 7f925cf7e700 1 -- 172.21.15.47:0/4100766303 wait complete. 2024-02-23T00:41:13.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.446+0000 7f925cf7e700 1 librados: starting msgr at 2024-02-23T00:41:13.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.446+0000 7f925cf7e700 1 librados: starting objecter 2024-02-23T00:41:13.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.446+0000 7f925cf7e700 1 Processor -- start 2024-02-23T00:41:13.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.446+0000 7f925cf7e700 1 -- start start 2024-02-23T00:41:13.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.446+0000 7f925cf7e700 1 librados: setting wanted keys 2024-02-23T00:41:13.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.446+0000 7f925cf7e700 1 librados: calling monclient init 2024-02-23T00:41:13.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.446+0000 7f925cf7e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92580f7330 0x7f9258108f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:13.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.446+0000 7f925cf7e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92580fcac0 0x7f9258109430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:13.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.446+0000 7f925cf7e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92580fd420 0x7f92581099f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:13.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.446+0000 7f925cf7e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9258114130 con 0x7f92580f7330 2024-02-23T00:41:13.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.446+0000 7f925cf7e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92581142a0 con 0x7f92580fd420 2024-02-23T00:41:13.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.446+0000 7f925cf7e700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9258114460 con 0x7f92580fcac0 2024-02-23T00:41:13.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.446+0000 7f9255d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92580fcac0 0x7f9258109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:13.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.446+0000 7f925659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92580f7330 0x7f9258108f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:13.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.446+0000 7f9255d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92580fcac0 0x7f9258109430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:45394/0 (socket says 172.21.15.47:45394) 2024-02-23T00:41:13.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.446+0000 7f9255d9b700 1 -- 172.21.15.47:0/624424727 learned_addr learned my addr 172.21.15.47:0/624424727 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:41:13.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.446+0000 7f9256d9d700 1 --2- 172.21.15.47:0/624424727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92580fd420 0x7f92581099f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:13.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.447+0000 7f925659c700 1 -- 172.21.15.47:0/624424727 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92580fcac0 msgr2=0x7f9258109430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:13.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.447+0000 7f925659c700 1 --2- 172.21.15.47:0/624424727 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92580fcac0 0x7f9258109430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:13.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.447+0000 7f925659c700 1 -- 172.21.15.47:0/624424727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92580fd420 msgr2=0x7f92581099f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:13.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.447+0000 7f925659c700 1 --2- 172.21.15.47:0/624424727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92580fd420 0x7f92581099f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:13.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.447+0000 7f925659c700 1 -- 172.21.15.47:0/624424727 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92500097c0 con 0x7f92580f7330 2024-02-23T00:41:13.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.447+0000 7f9255d9b700 1 --2- 172.21.15.47:0/624424727 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92580fcac0 0x7f9258109430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:41:13.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.447+0000 7f925659c700 1 --2- 172.21.15.47:0/624424727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92580f7330 0x7f9258108f40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f9248009da0 tx=0x7f9248009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:13.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.447+0000 7f924f7fe700 10 client.?.objecter ms_handle_connect 0x7f92580f7330 2024-02-23T00:41:13.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.447+0000 7f924f7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:41:13.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.447+0000 7f924f7fe700 1 -- 172.21.15.47:0/624424727 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924800e030 con 0x7f92580f7330 2024-02-23T00:41:13.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.447+0000 7f924f7fe700 1 -- 172.21.15.47:0/624424727 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f9248013d70 con 0x7f92580f7330 2024-02-23T00:41:13.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.447+0000 7f925cf7e700 1 -- 172.21.15.47:0/624424727 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9248005040 con 0x7f92580f7330 2024-02-23T00:41:13.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.447+0000 7f924f7fe700 1 -- 172.21.15.47:0/624424727 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9248018070 con 0x7f92580f7330 2024-02-23T00:41:13.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.447+0000 7f925cf7e700 10 client.14679.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:41:13.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.447+0000 7f925cf7e700 1 -- 172.21.15.47:0/624424727 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9258114a30 con 0x7f92580f7330 2024-02-23T00:41:13.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.447+0000 7f925cf7e700 1 librados: init done 2024-02-23T00:41:13.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.448+0000 7f925cf7e700 1 -- 172.21.15.47:0/624424727 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f925805f990 con 0x7f92580f7330 2024-02-23T00:41:13.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.448+0000 7f924f7fe700 1 -- 172.21.15.47:0/624424727 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f9248009240 con 0x7f92580f7330 2024-02-23T00:41:13.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.449+0000 7f924f7fe700 1 --2- 172.21.15.47:0/624424727 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9244062cc0 0x7f9244065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:13.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.449+0000 7f924f7fe700 1 -- 172.21.15.47:0/624424727 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 38..91) v4 ==== 7421+0+0 (secure 0 0 0) 0x7f924807e090 con 0x7f92580f7330 2024-02-23T00:41:13.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.449+0000 7f924f7fe700 10 client.14679.objecter ms_dispatch 0x7f9258001550 osd_map(91..91 src has 38..91) v4 2024-02-23T00:41:13.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.449+0000 7f924f7fe700 3 client.14679.objecter handle_osd_map got epochs [91,91] > 0 2024-02-23T00:41:13.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.449+0000 7f924f7fe700 3 client.14679.objecter handle_osd_map decoding full epoch 91 2024-02-23T00:41:13.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.449+0000 7f9255d9b700 1 --2- 172.21.15.47:0/624424727 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9244062cc0 0x7f9244065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:13.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.449+0000 7f924f7fe700 20 client.14679.objecter dump_active .. 0 homeless 2024-02-23T00:41:13.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.449+0000 7f924f7fe700 10 client.14679.objecter ms_handle_connect 0x7f9244062cc0 2024-02-23T00:41:13.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.449+0000 7f9255d9b700 1 --2- 172.21.15.47:0/624424727 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9244062cc0 0x7f9244065180 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f9240004fd0 tx=0x7f924000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:13.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.453+0000 7f924f7fe700 1 -- 172.21.15.47:0/624424727 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f924804ddb0 con 0x7f92580f7330 2024-02-23T00:41:13.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:12.082001+0000 osd.3 (osd.3) 31 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:41:13.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:12.082079+0000 osd.3 (osd.3) 32 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:41:13.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:41:12.186851+0000 mgr.y (mgr.14517) 78 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:41:13.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:12.226434+0000 mon.a (mon.0) 746 : cluster [DBG] osdmap e91: 8 total, 8 up, 8 in 2024-02-23T00:41:13.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:12.082001+0000 osd.3 (osd.3) 31 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:41:13.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:12.082079+0000 osd.3 (osd.3) 32 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:41:13.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:41:12.186851+0000 mgr.y (mgr.14517) 78 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:41:13.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:12.226434+0000 mon.a (mon.0) 746 : cluster [DBG] osdmap e91: 8 total, 8 up, 8 in 2024-02-23T00:41:13.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.625+0000 7f925cf7e700 1 -- 172.21.15.47:0/624424727 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9258061530 con 0x7f92580f7330 2024-02-23T00:41:13.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.626+0000 7f924f7fe700 1 -- 172.21.15.47:0/624424727 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f92480513d0 con 0x7f92580f7330 2024-02-23T00:41:13.627 INFO:teuthology.orchestra.run.smithi047.stdout:111669149744 2024-02-23T00:41:13.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.627+0000 7f925cf7e700 10 librados: watch_flush enter 2024-02-23T00:41:13.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.627+0000 7f925cf7e700 10 librados: watch_flush exit 2024-02-23T00:41:13.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.627+0000 7f925cf7e700 20 client.14679.objecter shutdown clearing up homeless session... 2024-02-23T00:41:13.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.627+0000 7f925cf7e700 10 client.14679.objecter successfully canceled tick 2024-02-23T00:41:13.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.627+0000 7f925cf7e700 1 -- 172.21.15.47:0/624424727 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9244062cc0 msgr2=0x7f9244065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:13.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.627+0000 7f925cf7e700 1 --2- 172.21.15.47:0/624424727 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9244062cc0 0x7f9244065180 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f9240004fd0 tx=0x7f924000b040).stop 2024-02-23T00:41:13.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.627+0000 7f925cf7e700 1 -- 172.21.15.47:0/624424727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92580f7330 msgr2=0x7f9258108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:13.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.627+0000 7f925cf7e700 1 --2- 172.21.15.47:0/624424727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92580f7330 0x7f9258108f40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f9248009da0 tx=0x7f9248009cb0).stop 2024-02-23T00:41:13.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.627+0000 7f925cf7e700 1 -- 172.21.15.47:0/624424727 shutdown_connections 2024-02-23T00:41:13.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.627+0000 7f925cf7e700 1 --2- 172.21.15.47:0/624424727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92580fd420 0x7f92581099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:13.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.627+0000 7f925cf7e700 1 --2- 172.21.15.47:0/624424727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92580f7330 0x7f9258108f40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:13.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.627+0000 7f925cf7e700 1 --2- 172.21.15.47:0/624424727 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9244062cc0 0x7f9244065180 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:13.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.627+0000 7f925cf7e700 1 --2- 172.21.15.47:0/624424727 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92580fcac0 0x7f9258109430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:13.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.627+0000 7f925cf7e700 1 -- 172.21.15.47:0/624424727 >> 172.21.15.47:0/624424727 conn(0x7f92580f26f0 msgr2=0x7f92580fa810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:41:13.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.627+0000 7f925cf7e700 1 -- 172.21.15.47:0/624424727 shutdown_connections 2024-02-23T00:41:13.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.628+0000 7f925cf7e700 1 -- 172.21.15.47:0/624424727 wait complete. 2024-02-23T00:41:13.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:13.628+0000 7f925cf7e700 1 librados: shutdown 2024-02-23T00:41:13.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:12.082001+0000 osd.3 (osd.3) 31 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:41:13.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:12.082079+0000 osd.3 (osd.3) 32 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:41:13.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:41:12.186851+0000 mgr.y (mgr.14517) 78 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:41:13.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:12.226434+0000 mon.a (mon.0) 746 : cluster [DBG] osdmap e91: 8 total, 8 up, 8 in 2024-02-23T00:41:14.471 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:12.841734+0000 osd.0 (osd.0) 33 : cluster [DBG] 5.3 deep-scrub starts 2024-02-23T00:41:14.471 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:12.843726+0000 osd.0 (osd.0) 34 : cluster [DBG] 5.3 deep-scrub ok 2024-02-23T00:41:14.471 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:13.015492+0000 osd.4 (osd.4) 37 : cluster [DBG] 4.e scrub starts 2024-02-23T00:41:14.471 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:13.017038+0000 osd.4 (osd.4) 38 : cluster [DBG] 4.e scrub ok 2024-02-23T00:41:14.471 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:13.145180+0000 mgr.y (mgr.14517) 79 : cluster [DBG] pgmap v67: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:41:14.471 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:41:13.626127+0000 mon.a (mon.0) 747 : audit [DBG] from='client.? 172.21.15.47:0/624424727' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-23T00:41:14.471 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:12.841734+0000 osd.0 (osd.0) 33 : cluster [DBG] 5.3 deep-scrub starts 2024-02-23T00:41:14.471 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:12.843726+0000 osd.0 (osd.0) 34 : cluster [DBG] 5.3 deep-scrub ok 2024-02-23T00:41:14.471 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:13.015492+0000 osd.4 (osd.4) 37 : cluster [DBG] 4.e scrub starts 2024-02-23T00:41:14.471 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:13.017038+0000 osd.4 (osd.4) 38 : cluster [DBG] 4.e scrub ok 2024-02-23T00:41:14.472 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:13.145180+0000 mgr.y (mgr.14517) 79 : cluster [DBG] pgmap v67: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:41:14.472 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:41:13.626127+0000 mon.a (mon.0) 747 : audit [DBG] from='client.? 172.21.15.47:0/624424727' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-23T00:41:14.494 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149737 got 111669149744 for osd.3 2024-02-23T00:41:14.495 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-23T00:41:14.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:12.841734+0000 osd.0 (osd.0) 33 : cluster [DBG] 5.3 deep-scrub starts 2024-02-23T00:41:14.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:12.843726+0000 osd.0 (osd.0) 34 : cluster [DBG] 5.3 deep-scrub ok 2024-02-23T00:41:14.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:13.015492+0000 osd.4 (osd.4) 37 : cluster [DBG] 4.e scrub starts 2024-02-23T00:41:14.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:13.017038+0000 osd.4 (osd.4) 38 : cluster [DBG] 4.e scrub ok 2024-02-23T00:41:14.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:13.145180+0000 mgr.y (mgr.14517) 79 : cluster [DBG] pgmap v67: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:41:14.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:41:13.626127+0000 mon.a (mon.0) 747 : audit [DBG] from='client.? 172.21.15.47:0/624424727' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-23T00:41:14.778 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:41:15.345 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:14.057809+0000 osd.4 (osd.4) 39 : cluster [DBG] 4.c scrub starts 2024-02-23T00:41:15.345 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:14.059328+0000 osd.4 (osd.4) 40 : cluster [DBG] 4.c scrub ok 2024-02-23T00:41:15.346 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:14.057809+0000 osd.4 (osd.4) 39 : cluster [DBG] 4.c scrub starts 2024-02-23T00:41:15.346 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:14.059328+0000 osd.4 (osd.4) 40 : cluster [DBG] 4.c scrub ok 2024-02-23T00:41:15.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:14.057809+0000 osd.4 (osd.4) 39 : cluster [DBG] 4.c scrub starts 2024-02-23T00:41:15.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:14.059328+0000 osd.4 (osd.4) 40 : cluster [DBG] 4.c scrub ok 2024-02-23T00:41:16.444 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:15.080151+0000 osd.4 (osd.4) 41 : cluster [DBG] 6.e deep-scrub starts 2024-02-23T00:41:16.444 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:15.081147+0000 osd.4 (osd.4) 42 : cluster [DBG] 6.e deep-scrub ok 2024-02-23T00:41:16.445 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:15.145631+0000 mgr.y (mgr.14517) 80 : cluster [DBG] pgmap v68: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:41:16.445 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:15.080151+0000 osd.4 (osd.4) 41 : cluster [DBG] 6.e deep-scrub starts 2024-02-23T00:41:16.445 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:15.081147+0000 osd.4 (osd.4) 42 : cluster [DBG] 6.e deep-scrub ok 2024-02-23T00:41:16.445 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:15.145631+0000 mgr.y (mgr.14517) 80 : cluster [DBG] pgmap v68: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:41:16.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:15.080151+0000 osd.4 (osd.4) 41 : cluster [DBG] 6.e deep-scrub starts 2024-02-23T00:41:16.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:15.081147+0000 osd.4 (osd.4) 42 : cluster [DBG] 6.e deep-scrub ok 2024-02-23T00:41:16.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:15.145631+0000 mgr.y (mgr.14517) 80 : cluster [DBG] pgmap v68: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:41:16.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.720+0000 7f804f276700 1 -- 172.21.15.47:0/2212452015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80480f8420 msgr2=0x7f8048104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:16.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.720+0000 7f804f276700 1 --2- 172.21.15.47:0/2212452015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80480f8420 0x7f8048104a20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f8040009c30 tx=0x7f8040004c50).stop 2024-02-23T00:41:16.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.720+0000 7f804f276700 1 -- 172.21.15.47:0/2212452015 shutdown_connections 2024-02-23T00:41:16.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.720+0000 7f804f276700 1 --2- 172.21.15.47:0/2212452015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80480f8420 0x7f8048104a20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:16.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.720+0000 7f804f276700 1 --2- 172.21.15.47:0/2212452015 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f80480fdbb0 0x7f80480fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:16.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.720+0000 7f804f276700 1 --2- 172.21.15.47:0/2212452015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80480fe510 0x7f80480f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:16.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.720+0000 7f804f276700 1 -- 172.21.15.47:0/2212452015 >> 172.21.15.47:0/2212452015 conn(0x7f80480f3800 msgr2=0x7f80480f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:41:16.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.720+0000 7f804f276700 1 -- 172.21.15.47:0/2212452015 shutdown_connections 2024-02-23T00:41:16.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.720+0000 7f804f276700 1 -- 172.21.15.47:0/2212452015 wait complete. 2024-02-23T00:41:16.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.721+0000 7f804f276700 1 librados: starting msgr at 2024-02-23T00:41:16.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.721+0000 7f804f276700 1 librados: starting objecter 2024-02-23T00:41:16.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.721+0000 7f804f276700 1 Processor -- start 2024-02-23T00:41:16.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.721+0000 7f804f276700 1 -- start start 2024-02-23T00:41:16.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.721+0000 7f804f276700 1 librados: setting wanted keys 2024-02-23T00:41:16.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.721+0000 7f804f276700 1 librados: calling monclient init 2024-02-23T00:41:16.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.721+0000 7f804f276700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80480f8420 0x7f804810dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:16.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.721+0000 7f804f276700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80480fdbb0 0x7f804810e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:16.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.721+0000 7f804f276700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f80480fe510 0x7f8048107f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:16.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.721+0000 7f804f276700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8048108460 con 0x7f80480fdbb0 2024-02-23T00:41:16.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.721+0000 7f804f276700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80481085d0 con 0x7f80480f8420 2024-02-23T00:41:16.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.721+0000 7f804f276700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8048108740 con 0x7f80480fe510 2024-02-23T00:41:16.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.721+0000 7f804c811700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80480fdbb0 0x7f804810e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:16.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.721+0000 7f804d813700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f80480fe510 0x7f8048107f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:16.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.721+0000 7f804c811700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80480fdbb0 0x7f804810e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43614/0 (socket says 172.21.15.47:43614) 2024-02-23T00:41:16.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.721+0000 7f804c811700 1 -- 172.21.15.47:0/266105098 learned_addr learned my addr 172.21.15.47:0/266105098 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:41:16.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.721+0000 7f804d012700 1 --2- 172.21.15.47:0/266105098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80480f8420 0x7f804810dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:16.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.721+0000 7f804c811700 1 -- 172.21.15.47:0/266105098 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f80480fe510 msgr2=0x7f8048107f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:16.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.721+0000 7f804c811700 1 --2- 172.21.15.47:0/266105098 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f80480fe510 0x7f8048107f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:16.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.721+0000 7f804c811700 1 -- 172.21.15.47:0/266105098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80480f8420 msgr2=0x7f804810dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:16.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.721+0000 7f804c811700 1 --2- 172.21.15.47:0/266105098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80480f8420 0x7f804810dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:16.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.721+0000 7f804c811700 1 -- 172.21.15.47:0/266105098 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8044005040 con 0x7f80480fdbb0 2024-02-23T00:41:16.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.722+0000 7f804c811700 1 --2- 172.21.15.47:0/266105098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80480fdbb0 0x7f804810e280 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f803800e970 tx=0x7f803800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:16.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.722+0000 7f803e777700 10 client.?.objecter ms_handle_connect 0x7f80480fdbb0 2024-02-23T00:41:16.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.722+0000 7f803e777700 10 client.?.objecter resend_mon_ops 2024-02-23T00:41:16.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.722+0000 7f803e777700 1 -- 172.21.15.47:0/266105098 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80380046e0 con 0x7f80480fdbb0 2024-02-23T00:41:16.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.722+0000 7f803e777700 1 -- 172.21.15.47:0/266105098 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f803800f040 con 0x7f80480fdbb0 2024-02-23T00:41:16.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.722+0000 7f804f276700 1 -- 172.21.15.47:0/266105098 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80400097c0 con 0x7f80480fdbb0 2024-02-23T00:41:16.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.722+0000 7f803e777700 1 -- 172.21.15.47:0/266105098 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803800c5b0 con 0x7f80480fdbb0 2024-02-23T00:41:16.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.722+0000 7f804f276700 10 client.14685.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:41:16.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.722+0000 7f804f276700 1 -- 172.21.15.47:0/266105098 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8048114c00 con 0x7f80480fdbb0 2024-02-23T00:41:16.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.722+0000 7f804f276700 1 librados: init done 2024-02-23T00:41:16.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.723+0000 7f804f276700 1 -- 172.21.15.47:0/266105098 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f804805ee80 con 0x7f80480fdbb0 2024-02-23T00:41:16.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.723+0000 7f803e777700 1 -- 172.21.15.47:0/266105098 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f8038007b40 con 0x7f80480fdbb0 2024-02-23T00:41:16.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.724+0000 7f803e777700 1 --2- 172.21.15.47:0/266105098 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8034062cc0 0x7f8034065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:16.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.724+0000 7f804d012700 1 --2- 172.21.15.47:0/266105098 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8034062cc0 0x7f8034065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:16.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.724+0000 7f803e777700 1 -- 172.21.15.47:0/266105098 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 38..91) v4 ==== 7421+0+0 (secure 0 0 0) 0x7f803807c9b0 con 0x7f80480fdbb0 2024-02-23T00:41:16.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.724+0000 7f803e777700 10 client.14685.objecter ms_dispatch 0x7f8048001550 osd_map(91..91 src has 38..91) v4 2024-02-23T00:41:16.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.724+0000 7f803e777700 3 client.14685.objecter handle_osd_map got epochs [91,91] > 0 2024-02-23T00:41:16.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.724+0000 7f803e777700 3 client.14685.objecter handle_osd_map decoding full epoch 91 2024-02-23T00:41:16.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.724+0000 7f803e777700 20 client.14685.objecter dump_active .. 0 homeless 2024-02-23T00:41:16.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.724+0000 7f803e777700 10 client.14685.objecter ms_handle_connect 0x7f8034062cc0 2024-02-23T00:41:16.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.724+0000 7f804d012700 1 --2- 172.21.15.47:0/266105098 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8034062cc0 0x7f8034065180 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f80440092f0 tx=0x7f8044009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:16.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.728+0000 7f803e777700 1 -- 172.21.15.47:0/266105098 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f803804c7c0 con 0x7f80480fdbb0 2024-02-23T00:41:16.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.903+0000 7f804f276700 1 -- 172.21.15.47:0/266105098 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f804805fa60 con 0x7f80480fdbb0 2024-02-23T00:41:16.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.904+0000 7f803e777700 1 -- 172.21.15.47:0/266105098 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8038019020 con 0x7f80480fdbb0 2024-02-23T00:41:16.905 INFO:teuthology.orchestra.run.smithi047.stdout:137438953516 2024-02-23T00:41:16.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.905+0000 7f804f276700 10 librados: watch_flush enter 2024-02-23T00:41:16.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.905+0000 7f804f276700 10 librados: watch_flush exit 2024-02-23T00:41:16.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.905+0000 7f804f276700 20 client.14685.objecter shutdown clearing up homeless session... 2024-02-23T00:41:16.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.905+0000 7f804f276700 10 client.14685.objecter successfully canceled tick 2024-02-23T00:41:16.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.905+0000 7f804f276700 1 -- 172.21.15.47:0/266105098 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8034062cc0 msgr2=0x7f8034065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:16.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.905+0000 7f804f276700 1 --2- 172.21.15.47:0/266105098 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8034062cc0 0x7f8034065180 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f80440092f0 tx=0x7f8044009cb0).stop 2024-02-23T00:41:16.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.905+0000 7f804f276700 1 -- 172.21.15.47:0/266105098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80480fdbb0 msgr2=0x7f804810e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:16.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.905+0000 7f804f276700 1 --2- 172.21.15.47:0/266105098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80480fdbb0 0x7f804810e280 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f803800e970 tx=0x7f803800b040).stop 2024-02-23T00:41:16.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.905+0000 7f804f276700 1 -- 172.21.15.47:0/266105098 shutdown_connections 2024-02-23T00:41:16.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.905+0000 7f804f276700 1 --2- 172.21.15.47:0/266105098 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8034062cc0 0x7f8034065180 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:16.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.905+0000 7f804f276700 1 --2- 172.21.15.47:0/266105098 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f80480fe510 0x7f8048107f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:16.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.905+0000 7f804f276700 1 --2- 172.21.15.47:0/266105098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80480f8420 0x7f804810dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:16.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.905+0000 7f804f276700 1 --2- 172.21.15.47:0/266105098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80480fdbb0 0x7f804810e280 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:16.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.906+0000 7f804f276700 1 -- 172.21.15.47:0/266105098 >> 172.21.15.47:0/266105098 conn(0x7f80480f3800 msgr2=0x7f80480f51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:41:16.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.906+0000 7f804f276700 1 -- 172.21.15.47:0/266105098 shutdown_connections 2024-02-23T00:41:16.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.906+0000 7f804f276700 1 -- 172.21.15.47:0/266105098 wait complete. 2024-02-23T00:41:16.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:16.906+0000 7f804f276700 1 librados: shutdown 2024-02-23T00:41:17.242 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:16.047249+0000 osd.4 (osd.4) 43 : cluster [DBG] 6.1e scrub starts 2024-02-23T00:41:17.242 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:16.048201+0000 osd.4 (osd.4) 44 : cluster [DBG] 6.1e scrub ok 2024-02-23T00:41:17.243 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:41:16.904261+0000 mon.a (mon.0) 748 : audit [DBG] from='client.? 172.21.15.47:0/266105098' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-23T00:41:17.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:16.047249+0000 osd.4 (osd.4) 43 : cluster [DBG] 6.1e scrub starts 2024-02-23T00:41:17.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:16.048201+0000 osd.4 (osd.4) 44 : cluster [DBG] 6.1e scrub ok 2024-02-23T00:41:17.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:41:16.904261+0000 mon.a (mon.0) 748 : audit [DBG] from='client.? 172.21.15.47:0/266105098' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-23T00:41:17.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:16.047249+0000 osd.4 (osd.4) 43 : cluster [DBG] 6.1e scrub starts 2024-02-23T00:41:17.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:16.048201+0000 osd.4 (osd.4) 44 : cluster [DBG] 6.1e scrub ok 2024-02-23T00:41:17.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:41:16.904261+0000 mon.a (mon.0) 748 : audit [DBG] from='client.? 172.21.15.47:0/266105098' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-23T00:41:17.798 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953511 got 137438953516 for osd.4 2024-02-23T00:41:17.798 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-23T00:41:18.083 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:41:18.533 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:17.146301+0000 mgr.y (mgr.14517) 81 : cluster [DBG] pgmap v69: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:41:18.534 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:17.930538+0000 osd.3 (osd.3) 33 : cluster [DBG] 5.13 deep-scrub starts 2024-02-23T00:41:18.534 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:17.931410+0000 osd.3 (osd.3) 34 : cluster [DBG] 5.13 deep-scrub ok 2024-02-23T00:41:18.534 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:17.146301+0000 mgr.y (mgr.14517) 81 : cluster [DBG] pgmap v69: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:41:18.534 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:17.930538+0000 osd.3 (osd.3) 33 : cluster [DBG] 5.13 deep-scrub starts 2024-02-23T00:41:18.534 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:17.931410+0000 osd.3 (osd.3) 34 : cluster [DBG] 5.13 deep-scrub ok 2024-02-23T00:41:18.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:17.146301+0000 mgr.y (mgr.14517) 81 : cluster [DBG] pgmap v69: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:41:18.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:17.930538+0000 osd.3 (osd.3) 33 : cluster [DBG] 5.13 deep-scrub starts 2024-02-23T00:41:18.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:17.931410+0000 osd.3 (osd.3) 34 : cluster [DBG] 5.13 deep-scrub ok 2024-02-23T00:41:19.502 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:18.909529+0000 osd.3 (osd.3) 35 : cluster [DBG] 6.6 scrub starts 2024-02-23T00:41:19.502 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:18.910598+0000 osd.3 (osd.3) 36 : cluster [DBG] 6.6 scrub ok 2024-02-23T00:41:19.502 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:18.909529+0000 osd.3 (osd.3) 35 : cluster [DBG] 6.6 scrub starts 2024-02-23T00:41:19.502 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:18.910598+0000 osd.3 (osd.3) 36 : cluster [DBG] 6.6 scrub ok 2024-02-23T00:41:19.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:18.909529+0000 osd.3 (osd.3) 35 : cluster [DBG] 6.6 scrub starts 2024-02-23T00:41:19.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:18.910598+0000 osd.3 (osd.3) 36 : cluster [DBG] 6.6 scrub ok 2024-02-23T00:41:20.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.013+0000 7f2d81c9e700 1 -- 172.21.15.47:0/1580129103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d7c0f8420 msgr2=0x7f2d7c104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:20.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.013+0000 7f2d81c9e700 1 --2- 172.21.15.47:0/1580129103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d7c0f8420 0x7f2d7c104a20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f2d74009be0 tx=0x7f2d74004d10).stop 2024-02-23T00:41:20.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.013+0000 7f2d81c9e700 1 -- 172.21.15.47:0/1580129103 shutdown_connections 2024-02-23T00:41:20.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.013+0000 7f2d81c9e700 1 --2- 172.21.15.47:0/1580129103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d7c0f8420 0x7f2d7c104a20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:20.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.013+0000 7f2d81c9e700 1 --2- 172.21.15.47:0/1580129103 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2d7c0fdbb0 0x7f2d7c0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:20.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.013+0000 7f2d81c9e700 1 --2- 172.21.15.47:0/1580129103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d7c0fe510 0x7f2d7c0f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:20.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.013+0000 7f2d81c9e700 1 -- 172.21.15.47:0/1580129103 >> 172.21.15.47:0/1580129103 conn(0x7f2d7c0f3800 msgr2=0x7f2d7c0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:41:20.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.014+0000 7f2d81c9e700 1 -- 172.21.15.47:0/1580129103 shutdown_connections 2024-02-23T00:41:20.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.014+0000 7f2d81c9e700 1 -- 172.21.15.47:0/1580129103 wait complete. 2024-02-23T00:41:20.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.014+0000 7f2d81c9e700 1 librados: starting msgr at 2024-02-23T00:41:20.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.014+0000 7f2d81c9e700 1 librados: starting objecter 2024-02-23T00:41:20.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.014+0000 7f2d81c9e700 1 Processor -- start 2024-02-23T00:41:20.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.014+0000 7f2d81c9e700 1 -- start start 2024-02-23T00:41:20.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.014+0000 7f2d81c9e700 1 librados: setting wanted keys 2024-02-23T00:41:20.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.014+0000 7f2d81c9e700 1 librados: calling monclient init 2024-02-23T00:41:20.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.015+0000 7f2d81c9e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d7c0f8420 0x7f2d7c06b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:20.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.015+0000 7f2d81c9e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2d7c0fdbb0 0x7f2d7c06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:20.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.015+0000 7f2d81c9e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d7c0fe510 0x7f2d7c06bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:20.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.015+0000 7f2d81c9e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c116130 con 0x7f2d7c0f8420 2024-02-23T00:41:20.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.015+0000 7f2d81c9e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c1162a0 con 0x7f2d7c0fe510 2024-02-23T00:41:20.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.015+0000 7f2d81c9e700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c116460 con 0x7f2d7c0fdbb0 2024-02-23T00:41:20.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.015+0000 7f2d7b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d7c0f8420 0x7f2d7c06b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:20.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.015+0000 7f2d7affd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2d7c0fdbb0 0x7f2d7c06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:20.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.015+0000 7f2d7b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d7c0f8420 0x7f2d7c06b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43634/0 (socket says 172.21.15.47:43634) 2024-02-23T00:41:20.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.015+0000 7f2d7b7fe700 1 -- 172.21.15.47:0/866701509 learned_addr learned my addr 172.21.15.47:0/866701509 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:41:20.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.015+0000 7f2d7bfff700 1 --2- 172.21.15.47:0/866701509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d7c0fe510 0x7f2d7c06bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:20.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.015+0000 7f2d7b7fe700 1 -- 172.21.15.47:0/866701509 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2d7c0fdbb0 msgr2=0x7f2d7c06b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:20.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.015+0000 7f2d7b7fe700 1 --2- 172.21.15.47:0/866701509 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2d7c0fdbb0 0x7f2d7c06b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:20.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.015+0000 7f2d7b7fe700 1 -- 172.21.15.47:0/866701509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d7c0fe510 msgr2=0x7f2d7c06bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:20.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.015+0000 7f2d7b7fe700 1 --2- 172.21.15.47:0/866701509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d7c0fe510 0x7f2d7c06bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:20.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.015+0000 7f2d7b7fe700 1 -- 172.21.15.47:0/866701509 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d70005040 con 0x7f2d7c0f8420 2024-02-23T00:41:20.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.015+0000 7f2d7affd700 1 --2- 172.21.15.47:0/866701509 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2d7c0fdbb0 0x7f2d7c06b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:41:20.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.015+0000 7f2d7b7fe700 1 --2- 172.21.15.47:0/866701509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d7c0f8420 0x7f2d7c06b0d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f2d700048f0 tx=0x7f2d70009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:20.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.015+0000 7f2d78ff9700 10 client.?.objecter ms_handle_connect 0x7f2d7c0f8420 2024-02-23T00:41:20.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.015+0000 7f2d78ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:41:20.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.015+0000 7f2d78ff9700 1 -- 172.21.15.47:0/866701509 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d70007b00 con 0x7f2d7c0f8420 2024-02-23T00:41:20.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.015+0000 7f2d78ff9700 1 -- 172.21.15.47:0/866701509 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f2d7000f040 con 0x7f2d7c0f8420 2024-02-23T00:41:20.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.015+0000 7f2d78ff9700 1 -- 172.21.15.47:0/866701509 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d70014820 con 0x7f2d7c0f8420 2024-02-23T00:41:20.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.015+0000 7f2d81c9e700 1 -- 172.21.15.47:0/866701509 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d740097c0 con 0x7f2d7c0f8420 2024-02-23T00:41:20.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.015+0000 7f2d81c9e700 10 client.14691.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:41:20.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.015+0000 7f2d81c9e700 1 -- 172.21.15.47:0/866701509 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7c116aa0 con 0x7f2d7c0f8420 2024-02-23T00:41:20.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.015+0000 7f2d81c9e700 1 librados: init done 2024-02-23T00:41:20.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.016+0000 7f2d81c9e700 1 -- 172.21.15.47:0/866701509 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d7c106ed0 con 0x7f2d7c0f8420 2024-02-23T00:41:20.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.017+0000 7f2d78ff9700 1 -- 172.21.15.47:0/866701509 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f2d70014c50 con 0x7f2d7c0f8420 2024-02-23T00:41:20.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.018+0000 7f2d78ff9700 1 --2- 172.21.15.47:0/866701509 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2d64062cc0 0x7f2d64065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:20.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.018+0000 7f2d7affd700 1 --2- 172.21.15.47:0/866701509 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2d64062cc0 0x7f2d64065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:20.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.018+0000 7f2d78ff9700 1 -- 172.21.15.47:0/866701509 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 38..91) v4 ==== 7421+0+0 (secure 0 0 0) 0x7f2d7007ceb0 con 0x7f2d7c0f8420 2024-02-23T00:41:20.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.018+0000 7f2d78ff9700 10 client.14691.objecter ms_dispatch 0x7f2d7c001550 osd_map(91..91 src has 38..91) v4 2024-02-23T00:41:20.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.018+0000 7f2d78ff9700 3 client.14691.objecter handle_osd_map got epochs [91,91] > 0 2024-02-23T00:41:20.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.018+0000 7f2d78ff9700 3 client.14691.objecter handle_osd_map decoding full epoch 91 2024-02-23T00:41:20.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.018+0000 7f2d78ff9700 20 client.14691.objecter dump_active .. 0 homeless 2024-02-23T00:41:20.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.018+0000 7f2d78ff9700 10 client.14691.objecter ms_handle_connect 0x7f2d64062cc0 2024-02-23T00:41:20.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.018+0000 7f2d7affd700 1 --2- 172.21.15.47:0/866701509 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2d64062cc0 0x7f2d64065180 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f2d7c06c820 tx=0x7f2d6c009040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:20.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.022+0000 7f2d78ff9700 1 -- 172.21.15.47:0/866701509 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f2d7004cbd0 con 0x7f2d7c0f8420 2024-02-23T00:41:20.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.192+0000 7f2d81c9e700 1 -- 172.21.15.47:0/866701509 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f2d7c05fb80 con 0x7f2d7c0f8420 2024-02-23T00:41:20.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.193+0000 7f2d78ff9700 1 -- 172.21.15.47:0/866701509 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2d70022090 con 0x7f2d7c0f8420 2024-02-23T00:41:20.194 INFO:teuthology.orchestra.run.smithi047.stdout:167503724584 2024-02-23T00:41:20.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.194+0000 7f2d81c9e700 10 librados: watch_flush enter 2024-02-23T00:41:20.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.194+0000 7f2d81c9e700 10 librados: watch_flush exit 2024-02-23T00:41:20.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.194+0000 7f2d81c9e700 20 client.14691.objecter shutdown clearing up homeless session... 2024-02-23T00:41:20.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.194+0000 7f2d81c9e700 10 client.14691.objecter successfully canceled tick 2024-02-23T00:41:20.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.194+0000 7f2d81c9e700 1 -- 172.21.15.47:0/866701509 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2d64062cc0 msgr2=0x7f2d64065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:20.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.194+0000 7f2d81c9e700 1 --2- 172.21.15.47:0/866701509 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2d64062cc0 0x7f2d64065180 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f2d7c06c820 tx=0x7f2d6c009040).stop 2024-02-23T00:41:20.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.194+0000 7f2d81c9e700 1 -- 172.21.15.47:0/866701509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d7c0f8420 msgr2=0x7f2d7c06b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:20.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.194+0000 7f2d81c9e700 1 --2- 172.21.15.47:0/866701509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d7c0f8420 0x7f2d7c06b0d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f2d700048f0 tx=0x7f2d70009cb0).stop 2024-02-23T00:41:20.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.195+0000 7f2d81c9e700 1 -- 172.21.15.47:0/866701509 shutdown_connections 2024-02-23T00:41:20.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.195+0000 7f2d81c9e700 1 --2- 172.21.15.47:0/866701509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d7c0fe510 0x7f2d7c06bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:20.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.195+0000 7f2d81c9e700 1 --2- 172.21.15.47:0/866701509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d7c0f8420 0x7f2d7c06b0d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:20.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.195+0000 7f2d81c9e700 1 --2- 172.21.15.47:0/866701509 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2d64062cc0 0x7f2d64065180 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:20.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.195+0000 7f2d81c9e700 1 --2- 172.21.15.47:0/866701509 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2d7c0fdbb0 0x7f2d7c06b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:20.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.195+0000 7f2d81c9e700 1 -- 172.21.15.47:0/866701509 >> 172.21.15.47:0/866701509 conn(0x7f2d7c0f3800 msgr2=0x7f2d7c0f4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:41:20.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.195+0000 7f2d81c9e700 1 -- 172.21.15.47:0/866701509 shutdown_connections 2024-02-23T00:41:20.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.195+0000 7f2d81c9e700 1 -- 172.21.15.47:0/866701509 wait complete. 2024-02-23T00:41:20.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:20.195+0000 7f2d81c9e700 1 librados: shutdown 2024-02-23T00:41:20.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:19.146734+0000 mgr.y (mgr.14517) 82 : cluster [DBG] pgmap v70: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:41:20.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:41:20.193342+0000 mon.a (mon.0) 749 : audit [DBG] from='client.? 172.21.15.47:0/866701509' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-23T00:41:20.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:19.146734+0000 mgr.y (mgr.14517) 82 : cluster [DBG] pgmap v70: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:41:20.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:41:20.193342+0000 mon.a (mon.0) 749 : audit [DBG] from='client.? 172.21.15.47:0/866701509' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-23T00:41:20.557 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:41:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:41:20] "GET /metrics HTTP/1.1" 200 172153 "" "Prometheus/2.33.4" 2024-02-23T00:41:20.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:19.146734+0000 mgr.y (mgr.14517) 82 : cluster [DBG] pgmap v70: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:41:20.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:41:20.193342+0000 mon.a (mon.0) 749 : audit [DBG] from='client.? 172.21.15.47:0/866701509' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-23T00:41:21.051 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724579 got 167503724584 for osd.5 2024-02-23T00:41:21.052 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-02-23T00:41:21.334 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:41:21.630 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:19.898714+0000 osd.0 (osd.0) 35 : cluster [DBG] 5.5 scrub starts 2024-02-23T00:41:21.630 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:19.899706+0000 osd.0 (osd.0) 36 : cluster [DBG] 5.5 scrub ok 2024-02-23T00:41:21.630 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:20.943943+0000 osd.3 (osd.3) 37 : cluster [DBG] 4.10 scrub starts 2024-02-23T00:41:21.630 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:20.947372+0000 osd.3 (osd.3) 38 : cluster [DBG] 4.10 scrub ok 2024-02-23T00:41:21.630 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:19.898714+0000 osd.0 (osd.0) 35 : cluster [DBG] 5.5 scrub starts 2024-02-23T00:41:21.630 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:19.899706+0000 osd.0 (osd.0) 36 : cluster [DBG] 5.5 scrub ok 2024-02-23T00:41:21.630 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:20.943943+0000 osd.3 (osd.3) 37 : cluster [DBG] 4.10 scrub starts 2024-02-23T00:41:21.631 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:20.947372+0000 osd.3 (osd.3) 38 : cluster [DBG] 4.10 scrub ok 2024-02-23T00:41:21.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:19.898714+0000 osd.0 (osd.0) 35 : cluster [DBG] 5.5 scrub starts 2024-02-23T00:41:21.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:19.899706+0000 osd.0 (osd.0) 36 : cluster [DBG] 5.5 scrub ok 2024-02-23T00:41:21.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:20.943943+0000 osd.3 (osd.3) 37 : cluster [DBG] 4.10 scrub starts 2024-02-23T00:41:21.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:20.947372+0000 osd.3 (osd.3) 38 : cluster [DBG] 4.10 scrub ok 2024-02-23T00:41:22.528 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:41:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:41:22.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:21.147075+0000 mgr.y (mgr.14517) 83 : cluster [DBG] pgmap v71: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-23T00:41:22.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:21.905618+0000 osd.3 (osd.3) 39 : cluster [DBG] 4.0 scrub starts 2024-02-23T00:41:22.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:21.909335+0000 osd.3 (osd.3) 40 : cluster [DBG] 4.0 scrub ok 2024-02-23T00:41:22.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:21.147075+0000 mgr.y (mgr.14517) 83 : cluster [DBG] pgmap v71: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-23T00:41:22.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:21.905618+0000 osd.3 (osd.3) 39 : cluster [DBG] 4.0 scrub starts 2024-02-23T00:41:22.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:21.909335+0000 osd.3 (osd.3) 40 : cluster [DBG] 4.0 scrub ok 2024-02-23T00:41:22.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:21.147075+0000 mgr.y (mgr.14517) 83 : cluster [DBG] pgmap v71: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-23T00:41:22.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:21.905618+0000 osd.3 (osd.3) 39 : cluster [DBG] 4.0 scrub starts 2024-02-23T00:41:22.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:21.909335+0000 osd.3 (osd.3) 40 : cluster [DBG] 4.0 scrub ok 2024-02-23T00:41:22.943 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:41:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:41:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:41:23.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.355+0000 7f96da318700 1 -- 172.21.15.47:0/3209838073 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f96d40fdbb0 msgr2=0x7f96d40fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:23.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.355+0000 7f96da318700 1 --2- 172.21.15.47:0/3209838073 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f96d40fdbb0 0x7f96d40fdf90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f96c4004870 tx=0x7f96c4009cb0).stop 2024-02-23T00:41:23.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.355+0000 7f96da318700 1 -- 172.21.15.47:0/3209838073 shutdown_connections 2024-02-23T00:41:23.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.355+0000 7f96da318700 1 --2- 172.21.15.47:0/3209838073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96d40f8420 0x7f96d4104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:23.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.355+0000 7f96da318700 1 --2- 172.21.15.47:0/3209838073 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f96d40fdbb0 0x7f96d40fdf90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:23.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.355+0000 7f96da318700 1 --2- 172.21.15.47:0/3209838073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96d40fe510 0x7f96d40f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:23.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.355+0000 7f96da318700 1 -- 172.21.15.47:0/3209838073 >> 172.21.15.47:0/3209838073 conn(0x7f96d40f3800 msgr2=0x7f96d40f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:41:23.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.355+0000 7f96da318700 1 -- 172.21.15.47:0/3209838073 shutdown_connections 2024-02-23T00:41:23.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.355+0000 7f96da318700 1 -- 172.21.15.47:0/3209838073 wait complete. 2024-02-23T00:41:23.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.356+0000 7f96da318700 1 librados: starting msgr at 2024-02-23T00:41:23.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.356+0000 7f96da318700 1 librados: starting objecter 2024-02-23T00:41:23.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.356+0000 7f96da318700 1 Processor -- start 2024-02-23T00:41:23.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.356+0000 7f96da318700 1 -- start start 2024-02-23T00:41:23.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.356+0000 7f96da318700 1 librados: setting wanted keys 2024-02-23T00:41:23.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.356+0000 7f96da318700 1 librados: calling monclient init 2024-02-23T00:41:23.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.356+0000 7f96da318700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f96d40f8420 0x7f96d410dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:23.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.356+0000 7f96da318700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96d40fdbb0 0x7f96d410e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:23.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.356+0000 7f96da318700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96d40fe510 0x7f96d41081d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:23.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.356+0000 7f96da318700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d41086c0 con 0x7f96d40fdbb0 2024-02-23T00:41:23.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.356+0000 7f96da318700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d4108830 con 0x7f96d40fe510 2024-02-23T00:41:23.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.356+0000 7f96da318700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d41089a0 con 0x7f96d40f8420 2024-02-23T00:41:23.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.357+0000 7f96d37fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96d40fdbb0 0x7f96d410e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:23.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.357+0000 7f96d37fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96d40fdbb0 0x7f96d410e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43650/0 (socket says 172.21.15.47:43650) 2024-02-23T00:41:23.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.357+0000 7f96d37fe700 1 -- 172.21.15.47:0/2311194106 learned_addr learned my addr 172.21.15.47:0/2311194106 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:41:23.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.357+0000 7f96d3fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f96d40f8420 0x7f96d410dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:23.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.357+0000 7f96d88b5700 1 --2- 172.21.15.47:0/2311194106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96d40fe510 0x7f96d41081d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:23.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.357+0000 7f96d37fe700 1 -- 172.21.15.47:0/2311194106 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f96d40f8420 msgr2=0x7f96d410dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:23.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.357+0000 7f96d37fe700 1 --2- 172.21.15.47:0/2311194106 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f96d40f8420 0x7f96d410dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:23.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.357+0000 7f96d37fe700 1 -- 172.21.15.47:0/2311194106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96d40fe510 msgr2=0x7f96d41081d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:23.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.357+0000 7f96d37fe700 1 --2- 172.21.15.47:0/2311194106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96d40fe510 0x7f96d41081d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:23.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.357+0000 7f96d37fe700 1 -- 172.21.15.47:0/2311194106 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96cc0097c0 con 0x7f96d40fdbb0 2024-02-23T00:41:23.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.357+0000 7f96d3fff700 1 --2- 172.21.15.47:0/2311194106 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f96d40f8420 0x7f96d410dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:41:23.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.357+0000 7f96d88b5700 1 --2- 172.21.15.47:0/2311194106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96d40fe510 0x7f96d41081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:41:23.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.357+0000 7f96d37fe700 1 --2- 172.21.15.47:0/2311194106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96d40fdbb0 0x7f96d410e4e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f96bc00e970 tx=0x7f96bc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:23.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.357+0000 7f96d17fa700 10 client.?.objecter ms_handle_connect 0x7f96d40fdbb0 2024-02-23T00:41:23.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.357+0000 7f96d17fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:41:23.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.357+0000 7f96d17fa700 1 -- 172.21.15.47:0/2311194106 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96bc00ee60 con 0x7f96d40fdbb0 2024-02-23T00:41:23.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.357+0000 7f96d17fa700 1 -- 172.21.15.47:0/2311194106 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f96bc00f040 con 0x7f96d40fdbb0 2024-02-23T00:41:23.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.357+0000 7f96da318700 1 -- 172.21.15.47:0/2311194106 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c4005040 con 0x7f96d40fdbb0 2024-02-23T00:41:23.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.357+0000 7f96d17fa700 1 -- 172.21.15.47:0/2311194106 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96bc00c5b0 con 0x7f96d40fdbb0 2024-02-23T00:41:23.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.357+0000 7f96da318700 10 client.14697.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:41:23.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.357+0000 7f96da318700 1 -- 172.21.15.47:0/2311194106 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d4109030 con 0x7f96d40fdbb0 2024-02-23T00:41:23.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.357+0000 7f96da318700 1 librados: init done 2024-02-23T00:41:23.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.358+0000 7f96da318700 1 -- 172.21.15.47:0/2311194106 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d4106eb0 con 0x7f96d40fdbb0 2024-02-23T00:41:23.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.359+0000 7f96d17fa700 1 -- 172.21.15.47:0/2311194106 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f96bc007b40 con 0x7f96d40fdbb0 2024-02-23T00:41:23.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.359+0000 7f96d17fa700 1 --2- 172.21.15.47:0/2311194106 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f96c0062cc0 0x7f96c0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:23.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.359+0000 7f96d3fff700 1 --2- 172.21.15.47:0/2311194106 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f96c0062cc0 0x7f96c0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:23.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.359+0000 7f96d17fa700 1 -- 172.21.15.47:0/2311194106 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 38..91) v4 ==== 7421+0+0 (secure 0 0 0) 0x7f96bc07caf0 con 0x7f96d40fdbb0 2024-02-23T00:41:23.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.359+0000 7f96d17fa700 10 client.14697.objecter ms_dispatch 0x7f96d4001550 osd_map(91..91 src has 38..91) v4 2024-02-23T00:41:23.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.359+0000 7f96d17fa700 3 client.14697.objecter handle_osd_map got epochs [91,91] > 0 2024-02-23T00:41:23.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.359+0000 7f96d17fa700 3 client.14697.objecter handle_osd_map decoding full epoch 91 2024-02-23T00:41:23.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.359+0000 7f96d17fa700 20 client.14697.objecter dump_active .. 0 homeless 2024-02-23T00:41:23.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.360+0000 7f96d17fa700 10 client.14697.objecter ms_handle_connect 0x7f96c0062cc0 2024-02-23T00:41:23.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.360+0000 7f96d3fff700 1 --2- 172.21.15.47:0/2311194106 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f96c0062cc0 0x7f96c0065180 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f96c4005010 tx=0x7f96c4009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:23.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.364+0000 7f96d17fa700 1 -- 172.21.15.47:0/2311194106 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f96bc04af60 con 0x7f96d40fdbb0 2024-02-23T00:41:23.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.535+0000 7f96da318700 1 -- 172.21.15.47:0/2311194106 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f96d405fa90 con 0x7f96d40fdbb0 2024-02-23T00:41:23.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.536+0000 7f96d17fa700 1 -- 172.21.15.47:0/2311194106 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f96bc019070 con 0x7f96d40fdbb0 2024-02-23T00:41:23.538 INFO:teuthology.orchestra.run.smithi047.stdout:201863462948 2024-02-23T00:41:23.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.537+0000 7f96da318700 10 librados: watch_flush enter 2024-02-23T00:41:23.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.537+0000 7f96da318700 10 librados: watch_flush exit 2024-02-23T00:41:23.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.537+0000 7f96da318700 20 client.14697.objecter shutdown clearing up homeless session... 2024-02-23T00:41:23.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.537+0000 7f96da318700 10 client.14697.objecter successfully canceled tick 2024-02-23T00:41:23.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.537+0000 7f96da318700 1 -- 172.21.15.47:0/2311194106 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f96c0062cc0 msgr2=0x7f96c0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:23.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.537+0000 7f96da318700 1 --2- 172.21.15.47:0/2311194106 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f96c0062cc0 0x7f96c0065180 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f96c4005010 tx=0x7f96c4009cb0).stop 2024-02-23T00:41:23.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.538+0000 7f96da318700 1 -- 172.21.15.47:0/2311194106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96d40fdbb0 msgr2=0x7f96d410e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:23.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.538+0000 7f96da318700 1 --2- 172.21.15.47:0/2311194106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96d40fdbb0 0x7f96d410e4e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f96bc00e970 tx=0x7f96bc00b040).stop 2024-02-23T00:41:23.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.538+0000 7f96da318700 1 -- 172.21.15.47:0/2311194106 shutdown_connections 2024-02-23T00:41:23.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.538+0000 7f96da318700 1 --2- 172.21.15.47:0/2311194106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96d40fe510 0x7f96d41081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:23.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.538+0000 7f96da318700 1 --2- 172.21.15.47:0/2311194106 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f96c0062cc0 0x7f96c0065180 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:23.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.538+0000 7f96da318700 1 --2- 172.21.15.47:0/2311194106 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f96d40f8420 0x7f96d410dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:23.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.538+0000 7f96da318700 1 --2- 172.21.15.47:0/2311194106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96d40fdbb0 0x7f96d410e4e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:23.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.538+0000 7f96da318700 1 -- 172.21.15.47:0/2311194106 >> 172.21.15.47:0/2311194106 conn(0x7f96d40f3800 msgr2=0x7f96d40fba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:41:23.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.538+0000 7f96da318700 1 -- 172.21.15.47:0/2311194106 shutdown_connections 2024-02-23T00:41:23.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.538+0000 7f96da318700 1 -- 172.21.15.47:0/2311194106 wait complete. 2024-02-23T00:41:23.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:23.538+0000 7f96da318700 1 librados: shutdown 2024-02-23T00:41:23.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:41:22.194493+0000 mgr.y (mgr.14517) 84 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:41:23.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:41:22.194493+0000 mgr.y (mgr.14517) 84 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:41:23.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:41:22.194493+0000 mgr.y (mgr.14517) 84 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:41:24.256 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462942 got 201863462948 for osd.6 2024-02-23T00:41:24.256 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-02-23T00:41:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:41:24.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:22.854466+0000 osd.0 (osd.0) 37 : cluster [DBG] 6.16 scrub starts 2024-02-23T00:41:24.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:22.855512+0000 osd.0 (osd.0) 38 : cluster [DBG] 6.16 scrub ok 2024-02-23T00:41:24.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:23.147685+0000 mgr.y (mgr.14517) 85 : cluster [DBG] pgmap v72: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 937 B/s rd, 0 op/s 2024-02-23T00:41:24.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:41:23.537180+0000 mon.a (mon.0) 750 : audit [DBG] from='client.? 172.21.15.47:0/2311194106' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-23T00:41:24.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:23.903105+0000 osd.3 (osd.3) 41 : cluster [DBG] 4.17 scrub starts 2024-02-23T00:41:24.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:23.904505+0000 osd.3 (osd.3) 42 : cluster [DBG] 4.17 scrub ok 2024-02-23T00:41:24.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:22.854466+0000 osd.0 (osd.0) 37 : cluster [DBG] 6.16 scrub starts 2024-02-23T00:41:24.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:22.855512+0000 osd.0 (osd.0) 38 : cluster [DBG] 6.16 scrub ok 2024-02-23T00:41:24.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:23.147685+0000 mgr.y (mgr.14517) 85 : cluster [DBG] pgmap v72: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 937 B/s rd, 0 op/s 2024-02-23T00:41:24.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:41:23.537180+0000 mon.a (mon.0) 750 : audit [DBG] from='client.? 172.21.15.47:0/2311194106' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-23T00:41:24.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:23.903105+0000 osd.3 (osd.3) 41 : cluster [DBG] 4.17 scrub starts 2024-02-23T00:41:24.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:23.904505+0000 osd.3 (osd.3) 42 : cluster [DBG] 4.17 scrub ok 2024-02-23T00:41:24.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:22.854466+0000 osd.0 (osd.0) 37 : cluster [DBG] 6.16 scrub starts 2024-02-23T00:41:24.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:22.855512+0000 osd.0 (osd.0) 38 : cluster [DBG] 6.16 scrub ok 2024-02-23T00:41:24.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:23.147685+0000 mgr.y (mgr.14517) 85 : cluster [DBG] pgmap v72: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 937 B/s rd, 0 op/s 2024-02-23T00:41:24.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:41:23.537180+0000 mon.a (mon.0) 750 : audit [DBG] from='client.? 172.21.15.47:0/2311194106' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-23T00:41:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:23.903105+0000 osd.3 (osd.3) 41 : cluster [DBG] 4.17 scrub starts 2024-02-23T00:41:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:23.904505+0000 osd.3 (osd.3) 42 : cluster [DBG] 4.17 scrub ok 2024-02-23T00:41:25.356 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:25.167460+0000 osd.2 (osd.2) 33 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:41:25.357 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:25.167511+0000 osd.2 (osd.2) 34 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:41:25.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:25.167460+0000 osd.2 (osd.2) 33 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:41:25.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:25.167511+0000 osd.2 (osd.2) 34 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:41:25.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:25.167460+0000 osd.2 (osd.2) 33 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:41:25.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:25.167511+0000 osd.2 (osd.2) 34 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:41:26.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.629+0000 7fd464158700 1 -- 172.21.15.47:0/2238843527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd45c0fe510 msgr2=0x7fd45c0f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:26.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.629+0000 7fd464158700 1 --2- 172.21.15.47:0/2238843527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd45c0fe510 0x7fd45c0f7d40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7fd458004870 tx=0x7fd458009cb0).stop 2024-02-23T00:41:26.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.629+0000 7fd464158700 1 -- 172.21.15.47:0/2238843527 shutdown_connections 2024-02-23T00:41:26.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.629+0000 7fd464158700 1 --2- 172.21.15.47:0/2238843527 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd45c0f8420 0x7fd45c104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:26.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.629+0000 7fd464158700 1 --2- 172.21.15.47:0/2238843527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd45c0fdbb0 0x7fd45c0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:26.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.629+0000 7fd464158700 1 --2- 172.21.15.47:0/2238843527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd45c0fe510 0x7fd45c0f7d40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:26.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.629+0000 7fd464158700 1 -- 172.21.15.47:0/2238843527 >> 172.21.15.47:0/2238843527 conn(0x7fd45c0f3800 msgr2=0x7fd45c0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:41:26.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.629+0000 7fd464158700 1 -- 172.21.15.47:0/2238843527 shutdown_connections 2024-02-23T00:41:26.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.629+0000 7fd464158700 1 -- 172.21.15.47:0/2238843527 wait complete. 2024-02-23T00:41:26.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.630+0000 7fd464158700 1 librados: starting msgr at 2024-02-23T00:41:26.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.630+0000 7fd464158700 1 librados: starting objecter 2024-02-23T00:41:26.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.630+0000 7fd464158700 1 Processor -- start 2024-02-23T00:41:26.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.630+0000 7fd464158700 1 -- start start 2024-02-23T00:41:26.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.630+0000 7fd464158700 1 librados: setting wanted keys 2024-02-23T00:41:26.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.630+0000 7fd464158700 1 librados: calling monclient init 2024-02-23T00:41:26.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.630+0000 7fd464158700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd45c0f8420 0x7fd45c110090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:26.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.630+0000 7fd464158700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd45c0fdbb0 0x7fd45c110580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:26.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.630+0000 7fd464158700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd45c0fe510 0x7fd45c10a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:26.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.630+0000 7fd464158700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c10a720 con 0x7fd45c0f8420 2024-02-23T00:41:26.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.630+0000 7fd464158700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c10a890 con 0x7fd45c0fdbb0 2024-02-23T00:41:26.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.630+0000 7fd464158700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c10aa50 con 0x7fd45c0fe510 2024-02-23T00:41:26.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.630+0000 7fd461ef4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd45c0f8420 0x7fd45c110090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:26.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.630+0000 7fd461ef4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd45c0f8420 0x7fd45c110090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41488/0 (socket says 172.21.15.47:41488) 2024-02-23T00:41:26.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.630+0000 7fd4626f5700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd45c0fe510 0x7fd45c10a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:26.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.630+0000 7fd461ef4700 1 -- 172.21.15.47:0/753482564 learned_addr learned my addr 172.21.15.47:0/753482564 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:41:26.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.630+0000 7fd461ef4700 1 -- 172.21.15.47:0/753482564 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd45c0fe510 msgr2=0x7fd45c10a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:26.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.630+0000 7fd461ef4700 1 --2- 172.21.15.47:0/753482564 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd45c0fe510 0x7fd45c10a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:26.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.630+0000 7fd4616f3700 1 --2- 172.21.15.47:0/753482564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd45c0fdbb0 0x7fd45c110580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:26.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.630+0000 7fd461ef4700 1 -- 172.21.15.47:0/753482564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd45c0fdbb0 msgr2=0x7fd45c110580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:26.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.630+0000 7fd461ef4700 1 --2- 172.21.15.47:0/753482564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd45c0fdbb0 0x7fd45c110580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:26.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.630+0000 7fd461ef4700 1 -- 172.21.15.47:0/753482564 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd458005040 con 0x7fd45c0f8420 2024-02-23T00:41:26.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.630+0000 7fd4626f5700 1 --2- 172.21.15.47:0/753482564 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd45c0fe510 0x7fd45c10a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:41:26.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.630+0000 7fd4616f3700 1 --2- 172.21.15.47:0/753482564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd45c0fdbb0 0x7fd45c110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-23T00:41:26.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.631+0000 7fd461ef4700 1 --2- 172.21.15.47:0/753482564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd45c0f8420 0x7fd45c110090 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fd4500027f0 tx=0x7fd450002b00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:26.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.631+0000 7fd44effd700 10 client.?.objecter ms_handle_connect 0x7fd45c0f8420 2024-02-23T00:41:26.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.631+0000 7fd44effd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:41:26.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.631+0000 7fd44effd700 1 -- 172.21.15.47:0/753482564 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd450007b40 con 0x7fd45c0f8420 2024-02-23T00:41:26.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.631+0000 7fd44effd700 1 -- 172.21.15.47:0/753482564 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fd45000f040 con 0x7fd45c0f8420 2024-02-23T00:41:26.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.631+0000 7fd464158700 1 -- 172.21.15.47:0/753482564 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd45c10ad30 con 0x7fd45c0f8420 2024-02-23T00:41:26.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.631+0000 7fd44effd700 1 -- 172.21.15.47:0/753482564 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45000d660 con 0x7fd45c0f8420 2024-02-23T00:41:26.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.631+0000 7fd464158700 10 client.14703.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:41:26.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.631+0000 7fd464158700 1 -- 172.21.15.47:0/753482564 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd45c116ec0 con 0x7fd45c0f8420 2024-02-23T00:41:26.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.631+0000 7fd464158700 1 librados: init done 2024-02-23T00:41:26.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.632+0000 7fd464158700 1 -- 172.21.15.47:0/753482564 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd45c106eb0 con 0x7fd45c0f8420 2024-02-23T00:41:26.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.632+0000 7fd44effd700 1 -- 172.21.15.47:0/753482564 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fd450009110 con 0x7fd45c0f8420 2024-02-23T00:41:26.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.632+0000 7fd44effd700 1 --2- 172.21.15.47:0/753482564 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd448062cc0 0x7fd448065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:26.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.633+0000 7fd44effd700 1 -- 172.21.15.47:0/753482564 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 38..92) v4 ==== 7421+0+0 (secure 0 0 0) 0x7fd45007cee0 con 0x7fd45c0f8420 2024-02-23T00:41:26.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.633+0000 7fd44effd700 10 client.14703.objecter ms_dispatch 0x7fd45c001550 osd_map(92..92 src has 38..92) v4 2024-02-23T00:41:26.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.633+0000 7fd44effd700 3 client.14703.objecter handle_osd_map got epochs [92,92] > 0 2024-02-23T00:41:26.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.633+0000 7fd44effd700 3 client.14703.objecter handle_osd_map decoding full epoch 92 2024-02-23T00:41:26.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.633+0000 7fd4616f3700 1 --2- 172.21.15.47:0/753482564 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd448062cc0 0x7fd448065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:26.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.633+0000 7fd44effd700 20 client.14703.objecter dump_active .. 0 homeless 2024-02-23T00:41:26.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.633+0000 7fd44effd700 10 client.14703.objecter ms_handle_connect 0x7fd448062cc0 2024-02-23T00:41:26.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.633+0000 7fd4616f3700 1 --2- 172.21.15.47:0/753482564 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd448062cc0 0x7fd448065180 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fd458004ee0 tx=0x7fd458009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:26.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.637+0000 7fd44effd700 1 -- 172.21.15.47:0/753482564 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fd45004cbd0 con 0x7fd45c0f8420 2024-02-23T00:41:26.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:25.148232+0000 mgr.y (mgr.14517) 86 : cluster [DBG] pgmap v73: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:41:26.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:25.355780+0000 mon.a (mon.0) 751 : cluster [DBG] osdmap e92: 8 total, 8 up, 8 in 2024-02-23T00:41:26.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:25.940921+0000 osd.3 (osd.3) 43 : cluster [DBG] 5.14 scrub starts 2024-02-23T00:41:26.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:25.942816+0000 osd.3 (osd.3) 44 : cluster [DBG] 5.14 scrub ok 2024-02-23T00:41:26.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:25.148232+0000 mgr.y (mgr.14517) 86 : cluster [DBG] pgmap v73: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:41:26.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:25.355780+0000 mon.a (mon.0) 751 : cluster [DBG] osdmap e92: 8 total, 8 up, 8 in 2024-02-23T00:41:26.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:25.940921+0000 osd.3 (osd.3) 43 : cluster [DBG] 5.14 scrub starts 2024-02-23T00:41:26.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:25.942816+0000 osd.3 (osd.3) 44 : cluster [DBG] 5.14 scrub ok 2024-02-23T00:41:26.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:25.148232+0000 mgr.y (mgr.14517) 86 : cluster [DBG] pgmap v73: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:41:26.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:25.355780+0000 mon.a (mon.0) 751 : cluster [DBG] osdmap e92: 8 total, 8 up, 8 in 2024-02-23T00:41:26.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:25.940921+0000 osd.3 (osd.3) 43 : cluster [DBG] 5.14 scrub starts 2024-02-23T00:41:26.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:25.942816+0000 osd.3 (osd.3) 44 : cluster [DBG] 5.14 scrub ok 2024-02-23T00:41:26.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.811+0000 7fd464158700 1 -- 172.21.15.47:0/753482564 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fd45c05fb30 con 0x7fd45c0f8420 2024-02-23T00:41:26.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.812+0000 7fd44effd700 1 -- 172.21.15.47:0/753482564 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd450004570 con 0x7fd45c0f8420 2024-02-23T00:41:26.814 INFO:teuthology.orchestra.run.smithi047.stdout:231928234013 2024-02-23T00:41:26.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.813+0000 7fd464158700 10 librados: watch_flush enter 2024-02-23T00:41:26.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.813+0000 7fd464158700 10 librados: watch_flush exit 2024-02-23T00:41:26.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.813+0000 7fd464158700 20 client.14703.objecter shutdown clearing up homeless session... 2024-02-23T00:41:26.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.813+0000 7fd464158700 10 client.14703.objecter successfully canceled tick 2024-02-23T00:41:26.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.813+0000 7fd464158700 1 -- 172.21.15.47:0/753482564 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd448062cc0 msgr2=0x7fd448065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:26.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.813+0000 7fd464158700 1 --2- 172.21.15.47:0/753482564 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd448062cc0 0x7fd448065180 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fd458004ee0 tx=0x7fd458009cb0).stop 2024-02-23T00:41:26.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.813+0000 7fd464158700 1 -- 172.21.15.47:0/753482564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd45c0f8420 msgr2=0x7fd45c110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:26.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.813+0000 7fd464158700 1 --2- 172.21.15.47:0/753482564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd45c0f8420 0x7fd45c110090 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fd4500027f0 tx=0x7fd450002b00).stop 2024-02-23T00:41:26.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.813+0000 7fd464158700 1 -- 172.21.15.47:0/753482564 shutdown_connections 2024-02-23T00:41:26.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.813+0000 7fd464158700 1 --2- 172.21.15.47:0/753482564 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd448062cc0 0x7fd448065180 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:26.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.813+0000 7fd464158700 1 --2- 172.21.15.47:0/753482564 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd45c0fe510 0x7fd45c10a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:26.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.813+0000 7fd464158700 1 --2- 172.21.15.47:0/753482564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd45c0f8420 0x7fd45c110090 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:26.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.813+0000 7fd464158700 1 --2- 172.21.15.47:0/753482564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd45c0fdbb0 0x7fd45c110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:26.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.813+0000 7fd464158700 1 -- 172.21.15.47:0/753482564 >> 172.21.15.47:0/753482564 conn(0x7fd45c0f3800 msgr2=0x7fd45c06af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:41:26.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.814+0000 7fd464158700 1 -- 172.21.15.47:0/753482564 shutdown_connections 2024-02-23T00:41:26.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.814+0000 7fd464158700 1 -- 172.21.15.47:0/753482564 wait complete. 2024-02-23T00:41:26.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:26.814+0000 7fd464158700 1 librados: shutdown 2024-02-23T00:41:27.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:41:26.813138+0000 mon.a (mon.0) 752 : audit [DBG] from='client.? 172.21.15.47:0/753482564' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-23T00:41:27.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:41:26.813138+0000 mon.a (mon.0) 752 : audit [DBG] from='client.? 172.21.15.47:0/753482564' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-23T00:41:27.627 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234008 got 231928234013 for osd.7 2024-02-23T00:41:27.627 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-23T00:41:27.627 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph pg dump --format=json 2024-02-23T00:41:27.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:41:26.813138+0000 mon.a (mon.0) 752 : audit [DBG] from='client.? 172.21.15.47:0/753482564' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-23T00:41:27.908 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:41:28.471 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:26.833833+0000 osd.7 (osd.7) 41 : cluster [DBG] 2.0 scrub starts 2024-02-23T00:41:28.472 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:26.836814+0000 osd.7 (osd.7) 42 : cluster [DBG] 2.0 scrub ok 2024-02-23T00:41:28.472 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:27.148718+0000 mgr.y (mgr.14517) 87 : cluster [DBG] pgmap v75: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:41:28.472 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:26.833833+0000 osd.7 (osd.7) 41 : cluster [DBG] 2.0 scrub starts 2024-02-23T00:41:28.472 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:26.836814+0000 osd.7 (osd.7) 42 : cluster [DBG] 2.0 scrub ok 2024-02-23T00:41:28.472 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:27.148718+0000 mgr.y (mgr.14517) 87 : cluster [DBG] pgmap v75: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:41:28.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:26.833833+0000 osd.7 (osd.7) 41 : cluster [DBG] 2.0 scrub starts 2024-02-23T00:41:28.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:26.836814+0000 osd.7 (osd.7) 42 : cluster [DBG] 2.0 scrub ok 2024-02-23T00:41:28.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:27.148718+0000 mgr.y (mgr.14517) 87 : cluster [DBG] pgmap v75: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:41:29.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.958+0000 7f496107b700 1 -- 172.21.15.47:0/205100845 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f495c0fdbb0 msgr2=0x7f495c0fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:29.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.958+0000 7f496107b700 1 --2- 172.21.15.47:0/205100845 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f495c0fdbb0 0x7f495c0fdf90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f494c0047b0 tx=0x7f494c009cb0).stop 2024-02-23T00:41:29.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.958+0000 7f496107b700 1 -- 172.21.15.47:0/205100845 shutdown_connections 2024-02-23T00:41:29.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.958+0000 7f496107b700 1 --2- 172.21.15.47:0/205100845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f495c0f8420 0x7f495c104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:29.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.958+0000 7f496107b700 1 --2- 172.21.15.47:0/205100845 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f495c0fdbb0 0x7f495c0fdf90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:29.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.958+0000 7f496107b700 1 --2- 172.21.15.47:0/205100845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f495c0fe510 0x7f495c0f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:29.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.958+0000 7f496107b700 1 -- 172.21.15.47:0/205100845 >> 172.21.15.47:0/205100845 conn(0x7f495c0f3800 msgr2=0x7f495c0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:41:29.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.959+0000 7f496107b700 1 -- 172.21.15.47:0/205100845 shutdown_connections 2024-02-23T00:41:29.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.959+0000 7f496107b700 1 -- 172.21.15.47:0/205100845 wait complete. 2024-02-23T00:41:29.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.959+0000 7f496107b700 1 librados: starting msgr at 2024-02-23T00:41:29.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.959+0000 7f496107b700 1 librados: starting objecter 2024-02-23T00:41:29.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.959+0000 7f496107b700 1 Processor -- start 2024-02-23T00:41:29.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.960+0000 7f496107b700 1 -- start start 2024-02-23T00:41:29.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.960+0000 7f496107b700 1 librados: setting wanted keys 2024-02-23T00:41:29.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.960+0000 7f496107b700 1 librados: calling monclient init 2024-02-23T00:41:29.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.960+0000 7f496107b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f495c0f8420 0x7f495c10de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:29.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.960+0000 7f496107b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f495c0fdbb0 0x7f495c10e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:29.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.960+0000 7f496107b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f495c0fe510 0x7f495c108000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:29.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.960+0000 7f496107b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495c1084f0 con 0x7f495c0fdbb0 2024-02-23T00:41:29.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.960+0000 7f496107b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495c108660 con 0x7f495c0f8420 2024-02-23T00:41:29.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.960+0000 7f496107b700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f495c108820 con 0x7f495c0fe510 2024-02-23T00:41:29.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.960+0000 7f495a59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f495c0fdbb0 0x7f495c10e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:29.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.960+0000 7f495b59e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f495c0fe510 0x7f495c108000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:29.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.960+0000 7f495a59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f495c0fdbb0 0x7f495c10e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41510/0 (socket says 172.21.15.47:41510) 2024-02-23T00:41:29.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.960+0000 7f495a59c700 1 -- 172.21.15.47:0/1100897515 learned_addr learned my addr 172.21.15.47:0/1100897515 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:41:29.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.960+0000 7f495ad9d700 1 --2- 172.21.15.47:0/1100897515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f495c0f8420 0x7f495c10de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:29.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.960+0000 7f495a59c700 1 -- 172.21.15.47:0/1100897515 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f495c0fe510 msgr2=0x7f495c108000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:29.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.960+0000 7f495a59c700 1 --2- 172.21.15.47:0/1100897515 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f495c0fe510 0x7f495c108000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:29.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.960+0000 7f495a59c700 1 -- 172.21.15.47:0/1100897515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f495c0f8420 msgr2=0x7f495c10de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:29.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.960+0000 7f495a59c700 1 --2- 172.21.15.47:0/1100897515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f495c0f8420 0x7f495c10de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:29.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.960+0000 7f495a59c700 1 -- 172.21.15.47:0/1100897515 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49540097c0 con 0x7f495c0fdbb0 2024-02-23T00:41:29.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.960+0000 7f495b59e700 1 --2- 172.21.15.47:0/1100897515 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f495c0fe510 0x7f495c108000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:41:29.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.960+0000 7f495a59c700 1 --2- 172.21.15.47:0/1100897515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f495c0fdbb0 0x7f495c10e350 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f495000c7c0 tx=0x7f4950007ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:29.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.960+0000 7f4943fff700 10 client.?.objecter ms_handle_connect 0x7f495c0fdbb0 2024-02-23T00:41:29.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.960+0000 7f4943fff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:41:29.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.961+0000 7f4943fff700 1 -- 172.21.15.47:0/1100897515 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4950004430 con 0x7f495c0fdbb0 2024-02-23T00:41:29.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.961+0000 7f496107b700 1 -- 172.21.15.47:0/1100897515 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f494c005040 con 0x7f495c0fdbb0 2024-02-23T00:41:29.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.961+0000 7f4943fff700 1 -- 172.21.15.47:0/1100897515 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f495000e930 con 0x7f495c0fdbb0 2024-02-23T00:41:29.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.961+0000 7f4943fff700 1 -- 172.21.15.47:0/1100897515 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49500139a0 con 0x7f495c0fdbb0 2024-02-23T00:41:29.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.961+0000 7f496107b700 10 client.14709.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:41:29.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.961+0000 7f496107b700 1 -- 172.21.15.47:0/1100897515 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f495c114c90 con 0x7f495c0fdbb0 2024-02-23T00:41:29.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.961+0000 7f496107b700 1 librados: init done 2024-02-23T00:41:29.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.962+0000 7f496107b700 1 -- 172.21.15.47:0/1100897515 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f495c05ee80 con 0x7f495c0fdbb0 2024-02-23T00:41:29.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.963+0000 7f4943fff700 1 -- 172.21.15.47:0/1100897515 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f495000eaa0 con 0x7f495c0fdbb0 2024-02-23T00:41:29.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.963+0000 7f4943fff700 1 --2- 172.21.15.47:0/1100897515 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4944062cc0 0x7f4944065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:29.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.963+0000 7f4943fff700 1 -- 172.21.15.47:0/1100897515 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 38..92) v4 ==== 7421+0+0 (secure 0 0 0) 0x7f495007cda0 con 0x7f495c0fdbb0 2024-02-23T00:41:29.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.963+0000 7f4943fff700 10 client.14709.objecter ms_dispatch 0x7f495c001550 osd_map(92..92 src has 38..92) v4 2024-02-23T00:41:29.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.963+0000 7f4943fff700 3 client.14709.objecter handle_osd_map got epochs [92,92] > 0 2024-02-23T00:41:29.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.963+0000 7f4943fff700 3 client.14709.objecter handle_osd_map decoding full epoch 92 2024-02-23T00:41:29.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.963+0000 7f495ad9d700 1 --2- 172.21.15.47:0/1100897515 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4944062cc0 0x7f4944065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:29.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.964+0000 7f4943fff700 20 client.14709.objecter dump_active .. 0 homeless 2024-02-23T00:41:29.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.964+0000 7f4943fff700 10 client.14709.objecter ms_handle_connect 0x7f4944062cc0 2024-02-23T00:41:29.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.964+0000 7f495ad9d700 1 --2- 172.21.15.47:0/1100897515 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4944062cc0 0x7f4944065180 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f494c005010 tx=0x7f494c009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:29.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:29.967+0000 7f4943fff700 1 -- 172.21.15.47:0/1100897515 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f495004cac0 con 0x7f495c0fdbb0 2024-02-23T00:41:30.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:30.134+0000 7f496107b700 1 -- 172.21.15.47:0/1100897515 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f495c05fa60 con 0x7f4944062cc0 2024-02-23T00:41:30.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:30.146+0000 7f4943fff700 1 -- 172.21.15.47:0/1100897515 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+357231 (secure 0 0 0) 0x7f495c05fa60 con 0x7f4944062cc0 2024-02-23T00:41:30.153 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:41:30.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:30.149+0000 7f496107b700 10 librados: watch_flush enter 2024-02-23T00:41:30.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:30.149+0000 7f496107b700 10 librados: watch_flush exit 2024-02-23T00:41:30.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:30.149+0000 7f496107b700 20 client.14709.objecter shutdown clearing up homeless session... 2024-02-23T00:41:30.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:30.149+0000 7f496107b700 10 client.14709.objecter successfully canceled tick 2024-02-23T00:41:30.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:30.149+0000 7f496107b700 1 -- 172.21.15.47:0/1100897515 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4944062cc0 msgr2=0x7f4944065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:30.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:30.149+0000 7f496107b700 1 --2- 172.21.15.47:0/1100897515 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4944062cc0 0x7f4944065180 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f494c005010 tx=0x7f494c009cb0).stop 2024-02-23T00:41:30.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:30.149+0000 7f496107b700 1 -- 172.21.15.47:0/1100897515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f495c0fdbb0 msgr2=0x7f495c10e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:30.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:30.149+0000 7f496107b700 1 --2- 172.21.15.47:0/1100897515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f495c0fdbb0 0x7f495c10e350 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f495000c7c0 tx=0x7f4950007ef0).stop 2024-02-23T00:41:30.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:30.149+0000 7f496107b700 1 -- 172.21.15.47:0/1100897515 shutdown_connections 2024-02-23T00:41:30.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:30.149+0000 7f496107b700 1 --2- 172.21.15.47:0/1100897515 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4944062cc0 0x7f4944065180 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:30.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:30.149+0000 7f496107b700 1 --2- 172.21.15.47:0/1100897515 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f495c0fe510 0x7f495c108000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:30.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:30.149+0000 7f496107b700 1 --2- 172.21.15.47:0/1100897515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f495c0f8420 0x7f495c10de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:30.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:30.149+0000 7f496107b700 1 --2- 172.21.15.47:0/1100897515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f495c0fdbb0 0x7f495c10e350 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:30.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:30.149+0000 7f496107b700 1 -- 172.21.15.47:0/1100897515 >> 172.21.15.47:0/1100897515 conn(0x7f495c0f3800 msgr2=0x7f495c0fb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:41:30.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:30.149+0000 7f496107b700 1 -- 172.21.15.47:0/1100897515 shutdown_connections 2024-02-23T00:41:30.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:30.149+0000 7f496107b700 1 -- 172.21.15.47:0/1100897515 wait complete. 2024-02-23T00:41:30.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:30.149+0000 7f496107b700 1 librados: shutdown 2024-02-23T00:41:30.157 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-02-23T00:41:30.455 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:29.149138+0000 mgr.y (mgr.14517) 88 : cluster [DBG] pgmap v76: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:41:30.455 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:29.991382+0000 osd.3 (osd.3) 45 : cluster [DBG] 6.1f scrub starts 2024-02-23T00:41:30.455 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:29.992408+0000 osd.3 (osd.3) 46 : cluster [DBG] 6.1f scrub ok 2024-02-23T00:41:30.455 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:29.149138+0000 mgr.y (mgr.14517) 88 : cluster [DBG] pgmap v76: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:41:30.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:29.991382+0000 osd.3 (osd.3) 45 : cluster [DBG] 6.1f scrub starts 2024-02-23T00:41:30.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:29.992408+0000 osd.3 (osd.3) 46 : cluster [DBG] 6.1f scrub ok 2024-02-23T00:41:30.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:29.149138+0000 mgr.y (mgr.14517) 88 : cluster [DBG] pgmap v76: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:41:30.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:29.991382+0000 osd.3 (osd.3) 45 : cluster [DBG] 6.1f scrub starts 2024-02-23T00:41:30.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:29.992408+0000 osd.3 (osd.3) 46 : cluster [DBG] 6.1f scrub ok 2024-02-23T00:41:30.806 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:41:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:41:30] "GET /metrics HTTP/1.1" 200 172154 "" "Prometheus/2.33.4" 2024-02-23T00:41:30.907 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-02-23T00:41:29.148942+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5796,"num_objects":224,"num_object_clones":0,"num_object_copies":672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":0,"num_read":1229,"num_read_kb":949,"num_write":620,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":687,"ondisk_log_size":687,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2382540,"kb_used_data":5772,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":747578676,"statfs":{"total":767960285184,"available":765520564224,"internally_reserved":0,"allocated":5910528,"data_stored":2947548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002412"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.225936+0000","last_change":"2024-02-23T00:39:21.162471+0000","last_active":"2024-02-23T00:41:12.225936+0000","last_peered":"2024-02-23T00:41:12.225936+0000","last_clean":"2024-02-23T00:41:12.225936+0000","last_became_active":"2024-02-23T00:39:21.162083+0000","last_became_peered":"2024-02-23T00:39:21.162083+0000","last_unstale":"2024-02-23T00:41:12.225936+0000","last_undegraded":"2024-02-23T00:41:12.225936+0000","last_fullsized":"2024-02-23T00:41:12.225936+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:39:19.779989+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.225496+0000","last_change":"2024-02-23T00:41:05.965631+0000","last_active":"2024-02-23T00:41:12.225496+0000","last_peered":"2024-02-23T00:41:12.225496+0000","last_clean":"2024-02-23T00:41:12.225496+0000","last_became_active":"2024-02-23T00:39:18.799723+0000","last_became_peered":"2024-02-23T00:39:18.799723+0000","last_unstale":"2024-02-23T00:41:12.225496+0000","last_undegraded":"2024-02-23T00:41:12.225496+0000","last_fullsized":"2024-02-23T00:41:12.225496+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:05.965531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:05.965531+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.457993+0000","last_change":"2024-02-23T00:40:58.873093+0000","last_active":"2024-02-23T00:41:12.457993+0000","last_peered":"2024-02-23T00:41:12.457993+0000","last_clean":"2024-02-23T00:41:12.457993+0000","last_became_active":"2024-02-23T00:39:14.782000+0000","last_became_peered":"2024-02-23T00:39:14.782000+0000","last_unstale":"2024-02-23T00:41:12.457993+0000","last_undegraded":"2024-02-23T00:41:12.457993+0000","last_fullsized":"2024-02-23T00:41:12.457993+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_clean_scrub_stamp":"2024-02-23T00:40:58.873013+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"88'50","reported_seq":140,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.227141+0000","last_change":"2024-02-23T00:40:52.840769+0000","last_active":"2024-02-23T00:41:12.227141+0000","last_peered":"2024-02-23T00:41:12.227141+0000","last_clean":"2024-02-23T00:41:12.227141+0000","last_became_active":"2024-02-23T00:39:16.780144+0000","last_became_peered":"2024-02-23T00:39:16.780144+0000","last_unstale":"2024-02-23T00:41:12.227141+0000","last_undegraded":"2024-02-23T00:41:12.227141+0000","last_fullsized":"2024-02-23T00:41:12.227141+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'50","last_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_deep_scrub":"88'50","last_deep_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_clean_scrub_stamp":"2024-02-23T00:40:52.840702+0000","log_size":50,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.225739+0000","last_change":"2024-02-23T00:40:27.096850+0000","last_active":"2024-02-23T00:41:12.225739+0000","last_peered":"2024-02-23T00:41:12.225739+0000","last_clean":"2024-02-23T00:41:12.225739+0000","last_became_active":"2024-02-23T00:39:20.795665+0000","last_became_peered":"2024-02-23T00:39:20.795665+0000","last_unstale":"2024-02-23T00:41:12.225739+0000","last_undegraded":"2024-02-23T00:41:12.225739+0000","last_fullsized":"2024-02-23T00:41:12.225739+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:27.096747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:27.096747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:13.177559+0000","last_change":"2024-02-23T00:40:53.812966+0000","last_active":"2024-02-23T00:41:13.177559+0000","last_peered":"2024-02-23T00:41:13.177559+0000","last_clean":"2024-02-23T00:41:13.177559+0000","last_became_active":"2024-02-23T00:39:18.796312+0000","last_became_peered":"2024-02-23T00:39:18.796312+0000","last_unstale":"2024-02-23T00:41:13.177559+0000","last_undegraded":"2024-02-23T00:41:13.177559+0000","last_fullsized":"2024-02-23T00:41:13.177559+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:53.812893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:53.812893+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.226317+0000","last_change":"2024-02-23T00:41:02.853395+0000","last_active":"2024-02-23T00:41:12.226317+0000","last_peered":"2024-02-23T00:41:12.226317+0000","last_clean":"2024-02-23T00:41:12.226317+0000","last_became_active":"2024-02-23T00:39:14.785362+0000","last_became_peered":"2024-02-23T00:39:14.785362+0000","last_unstale":"2024-02-23T00:41:12.226317+0000","last_undegraded":"2024-02-23T00:41:12.226317+0000","last_fullsized":"2024-02-23T00:41:12.226317+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:02.853328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:02.853328+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"68'15","reported_seq":74,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.225734+0000","last_change":"2024-02-23T00:40:21.095743+0000","last_active":"2024-02-23T00:41:12.225734+0000","last_peered":"2024-02-23T00:41:12.225734+0000","last_clean":"2024-02-23T00:41:12.225734+0000","last_became_active":"2024-02-23T00:39:16.784606+0000","last_became_peered":"2024-02-23T00:39:16.784606+0000","last_unstale":"2024-02-23T00:41:12.225734+0000","last_undegraded":"2024-02-23T00:41:12.225734+0000","last_fullsized":"2024-02-23T00:41:12.225734+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:40:21.095675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:21.095675+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.840028+0000","last_change":"2024-02-23T00:40:55.855032+0000","last_active":"2024-02-23T00:41:12.840028+0000","last_peered":"2024-02-23T00:41:12.840028+0000","last_clean":"2024-02-23T00:41:12.840028+0000","last_became_active":"2024-02-23T00:39:20.793590+0000","last_became_peered":"2024-02-23T00:39:20.793590+0000","last_unstale":"2024-02-23T00:41:12.840028+0000","last_undegraded":"2024-02-23T00:41:12.840028+0000","last_fullsized":"2024-02-23T00:41:12.840028+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:55.854964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:55.854964+0000","last_clean_scrub_stamp":"2024-02-23T00:40:55.854964+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.607422+0000","last_change":"2024-02-23T00:40:42.667373+0000","last_active":"2024-02-23T00:41:12.607422+0000","last_peered":"2024-02-23T00:41:12.607422+0000","last_clean":"2024-02-23T00:41:12.607422+0000","last_became_active":"2024-02-23T00:39:18.796952+0000","last_became_peered":"2024-02-23T00:39:18.796952+0000","last_unstale":"2024-02-23T00:41:12.607422+0000","last_undegraded":"2024-02-23T00:41:12.607422+0000","last_fullsized":"2024-02-23T00:41:12.607422+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:42.667291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:42.667291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.607729+0000","last_change":"2024-02-23T00:40:27.569090+0000","last_active":"2024-02-23T00:41:12.607729+0000","last_peered":"2024-02-23T00:41:12.607729+0000","last_clean":"2024-02-23T00:41:12.607729+0000","last_became_active":"2024-02-23T00:39:14.783469+0000","last_became_peered":"2024-02-23T00:39:14.783469+0000","last_unstale":"2024-02-23T00:41:12.607729+0000","last_undegraded":"2024-02-23T00:41:12.607729+0000","last_fullsized":"2024-02-23T00:41:12.607729+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:27.568970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:27.568970+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"82'15","reported_seq":72,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.225895+0000","last_change":"2024-02-23T00:40:31.989101+0000","last_active":"2024-02-23T00:41:12.225895+0000","last_peered":"2024-02-23T00:41:12.225895+0000","last_clean":"2024-02-23T00:41:12.225895+0000","last_became_active":"2024-02-23T00:39:16.782762+0000","last_became_peered":"2024-02-23T00:39:16.782762+0000","last_unstale":"2024-02-23T00:41:12.225895+0000","last_undegraded":"2024-02-23T00:41:12.225895+0000","last_fullsized":"2024-02-23T00:41:12.225895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'15","last_scrub_stamp":"2024-02-23T00:40:31.989027+0000","last_deep_scrub":"82'15","last_deep_scrub_stamp":"2024-02-23T00:40:31.989027+0000","last_clean_scrub_stamp":"2024-02-23T00:40:31.989027+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.457429+0000","last_change":"2024-02-23T00:41:08.906994+0000","last_active":"2024-02-23T00:41:12.457429+0000","last_peered":"2024-02-23T00:41:12.457429+0000","last_clean":"2024-02-23T00:41:12.457429+0000","last_became_active":"2024-02-23T00:39:20.803344+0000","last_became_peered":"2024-02-23T00:39:20.803344+0000","last_unstale":"2024-02-23T00:41:12.457429+0000","last_undegraded":"2024-02-23T00:41:12.457429+0000","last_fullsized":"2024-02-23T00:41:12.457429+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:08.906928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:08.906928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.841446+0000","last_change":"2024-02-23T00:41:07.777516+0000","last_active":"2024-02-23T00:41:12.841446+0000","last_peered":"2024-02-23T00:41:12.841446+0000","last_clean":"2024-02-23T00:41:12.841446+0000","last_became_active":"2024-02-23T00:39:18.796431+0000","last_became_peered":"2024-02-23T00:39:18.796431+0000","last_unstale":"2024-02-23T00:41:12.841446+0000","last_undegraded":"2024-02-23T00:41:12.841446+0000","last_fullsized":"2024-02-23T00:41:12.841446+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:07.777435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:07.777435+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.607517+0000","last_change":"2024-02-23T00:40:35.620371+0000","last_active":"2024-02-23T00:41:12.607517+0000","last_peered":"2024-02-23T00:41:12.607517+0000","last_clean":"2024-02-23T00:41:12.607517+0000","last_became_active":"2024-02-23T00:39:14.787148+0000","last_became_peered":"2024-02-23T00:39:14.787148+0000","last_unstale":"2024-02-23T00:41:12.607517+0000","last_undegraded":"2024-02-23T00:41:12.607517+0000","last_fullsized":"2024-02-23T00:41:12.607517+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:35.620292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:35.620292+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"68'21","reported_seq":89,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.225770+0000","last_change":"2024-02-23T00:40:51.950549+0000","last_active":"2024-02-23T00:41:12.225770+0000","last_peered":"2024-02-23T00:41:12.225770+0000","last_clean":"2024-02-23T00:41:12.225770+0000","last_became_active":"2024-02-23T00:39:16.786791+0000","last_became_peered":"2024-02-23T00:39:16.786791+0000","last_unstale":"2024-02-23T00:41:12.225770+0000","last_undegraded":"2024-02-23T00:41:12.225770+0000","last_fullsized":"2024-02-23T00:41:12.225770+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'21","last_scrub_stamp":"2024-02-23T00:40:51.950467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:51.950467+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.225798+0000","last_change":"2024-02-23T00:39:21.162451+0000","last_active":"2024-02-23T00:41:12.225798+0000","last_peered":"2024-02-23T00:41:12.225798+0000","last_clean":"2024-02-23T00:41:12.225798+0000","last_became_active":"2024-02-23T00:39:21.162053+0000","last_became_peered":"2024-02-23T00:39:21.162053+0000","last_unstale":"2024-02-23T00:41:12.225798+0000","last_undegraded":"2024-02-23T00:41:12.225798+0000","last_fullsized":"2024-02-23T00:41:12.225798+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:39:19.779989+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.226055+0000","last_change":"2024-02-23T00:41:01.063755+0000","last_active":"2024-02-23T00:41:12.226055+0000","last_peered":"2024-02-23T00:41:12.226055+0000","last_clean":"2024-02-23T00:41:12.226055+0000","last_became_active":"2024-02-23T00:39:18.794903+0000","last_became_peered":"2024-02-23T00:39:18.794903+0000","last_unstale":"2024-02-23T00:41:12.226055+0000","last_undegraded":"2024-02-23T00:41:12.226055+0000","last_fullsized":"2024-02-23T00:41:12.226055+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_clean_scrub_stamp":"2024-02-23T00:41:01.063660+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.226132+0000","last_change":"2024-02-23T00:40:19.123225+0000","last_active":"2024-02-23T00:41:12.226132+0000","last_peered":"2024-02-23T00:41:12.226132+0000","last_clean":"2024-02-23T00:41:12.226132+0000","last_became_active":"2024-02-23T00:39:14.782163+0000","last_became_peered":"2024-02-23T00:39:14.782163+0000","last_unstale":"2024-02-23T00:41:12.226132+0000","last_undegraded":"2024-02-23T00:41:12.226132+0000","last_fullsized":"2024-02-23T00:41:12.226132+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:19.123123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:19.123123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"68'18","reported_seq":82,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.226642+0000","last_change":"2024-02-23T00:40:28.118091+0000","last_active":"2024-02-23T00:41:12.226642+0000","last_peered":"2024-02-23T00:41:12.226642+0000","last_clean":"2024-02-23T00:41:12.226642+0000","last_became_active":"2024-02-23T00:39:16.782803+0000","last_became_peered":"2024-02-23T00:39:16.782803+0000","last_unstale":"2024-02-23T00:41:12.226642+0000","last_undegraded":"2024-02-23T00:41:12.226642+0000","last_fullsized":"2024-02-23T00:41:12.226642+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'18","last_scrub_stamp":"2024-02-23T00:40:28.118021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:28.118021+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:16.048341+0000","last_change":"2024-02-23T00:41:16.048306+0000","last_active":"2024-02-23T00:41:16.048341+0000","last_peered":"2024-02-23T00:41:16.048341+0000","last_clean":"2024-02-23T00:41:16.048341+0000","last_became_active":"2024-02-23T00:39:21.159750+0000","last_became_peered":"2024-02-23T00:39:21.159750+0000","last_unstale":"2024-02-23T00:41:16.048341+0000","last_undegraded":"2024-02-23T00:41:16.048341+0000","last_fullsized":"2024-02-23T00:41:16.048341+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:16.048207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:16.048207+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:13.178334+0000","last_change":"2024-02-23T00:40:23.829090+0000","last_active":"2024-02-23T00:41:13.178334+0000","last_peered":"2024-02-23T00:41:13.178334+0000","last_clean":"2024-02-23T00:41:13.178334+0000","last_became_active":"2024-02-23T00:39:18.801039+0000","last_became_peered":"2024-02-23T00:39:18.801039+0000","last_unstale":"2024-02-23T00:41:13.178334+0000","last_undegraded":"2024-02-23T00:41:13.178334+0000","last_fullsized":"2024-02-23T00:41:13.178334+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:23.829017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:23.829017+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.841186+0000","last_change":"2024-02-23T00:40:30.958705+0000","last_active":"2024-02-23T00:41:12.841186+0000","last_peered":"2024-02-23T00:41:12.841186+0000","last_clean":"2024-02-23T00:41:12.841186+0000","last_became_active":"2024-02-23T00:39:14.782494+0000","last_became_peered":"2024-02-23T00:39:14.782494+0000","last_unstale":"2024-02-23T00:41:12.841186+0000","last_undegraded":"2024-02-23T00:41:12.841186+0000","last_fullsized":"2024-02-23T00:41:12.841186+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:30.958634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:30.958634+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"67'15","reported_seq":68,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.226009+0000","last_change":"2024-02-23T00:40:21.307315+0000","last_active":"2024-02-23T00:41:12.226009+0000","last_peered":"2024-02-23T00:41:12.226009+0000","last_clean":"2024-02-23T00:41:12.226009+0000","last_became_active":"2024-02-23T00:39:16.787138+0000","last_became_peered":"2024-02-23T00:39:16.787138+0000","last_unstale":"2024-02-23T00:41:12.226009+0000","last_undegraded":"2024-02-23T00:41:12.226009+0000","last_fullsized":"2024-02-23T00:41:12.226009+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:40:21.307241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:21.307241+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:13.177266+0000","last_change":"2024-02-23T00:40:49.777703+0000","last_active":"2024-02-23T00:41:13.177266+0000","last_peered":"2024-02-23T00:41:13.177266+0000","last_clean":"2024-02-23T00:41:13.177266+0000","last_became_active":"2024-02-23T00:39:20.795524+0000","last_became_peered":"2024-02-23T00:39:20.795524+0000","last_unstale":"2024-02-23T00:41:13.177266+0000","last_undegraded":"2024-02-23T00:41:13.177266+0000","last_fullsized":"2024-02-23T00:41:13.177266+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:49.777636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:49.777636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.458095+0000","last_change":"2024-02-23T00:40:21.510727+0000","last_active":"2024-02-23T00:41:12.458095+0000","last_peered":"2024-02-23T00:41:12.458095+0000","last_clean":"2024-02-23T00:41:12.458095+0000","last_became_active":"2024-02-23T00:39:18.790602+0000","last_became_peered":"2024-02-23T00:39:18.790602+0000","last_unstale":"2024-02-23T00:41:12.458095+0000","last_undegraded":"2024-02-23T00:41:12.458095+0000","last_fullsized":"2024-02-23T00:41:12.458095+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:21.510658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:21.510658+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.226792+0000","last_change":"2024-02-23T00:40:44.906676+0000","last_active":"2024-02-23T00:41:12.226792+0000","last_peered":"2024-02-23T00:41:12.226792+0000","last_clean":"2024-02-23T00:41:12.226792+0000","last_became_active":"2024-02-23T00:39:14.781560+0000","last_became_peered":"2024-02-23T00:39:14.781560+0000","last_unstale":"2024-02-23T00:41:12.226792+0000","last_undegraded":"2024-02-23T00:41:12.226792+0000","last_fullsized":"2024-02-23T00:41:12.226792+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:44.906599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:44.906599+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"68'17","reported_seq":77,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.841194+0000","last_change":"2024-02-23T00:40:20.909829+0000","last_active":"2024-02-23T00:41:12.841194+0000","last_peered":"2024-02-23T00:41:12.841194+0000","last_clean":"2024-02-23T00:41:12.841194+0000","last_became_active":"2024-02-23T00:39:16.782660+0000","last_became_peered":"2024-02-23T00:39:16.782660+0000","last_unstale":"2024-02-23T00:41:12.841194+0000","last_undegraded":"2024-02-23T00:41:12.841194+0000","last_fullsized":"2024-02-23T00:41:12.841194+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:40:20.909759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:20.909759+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"67'1","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.608186+0000","last_change":"2024-02-23T00:40:29.596674+0000","last_active":"2024-02-23T00:41:12.608186+0000","last_peered":"2024-02-23T00:41:12.608186+0000","last_clean":"2024-02-23T00:41:12.608186+0000","last_became_active":"2024-02-23T00:39:20.795085+0000","last_became_peered":"2024-02-23T00:39:20.795085+0000","last_unstale":"2024-02-23T00:41:12.608186+0000","last_undegraded":"2024-02-23T00:41:12.608186+0000","last_fullsized":"2024-02-23T00:41:12.608186+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:40:29.596581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:29.596581+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"67'8","reported_seq":78,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:22.865946+0000","last_change":"2024-02-23T00:40:27.923078+0000","last_active":"2024-02-23T00:41:22.865946+0000","last_peered":"2024-02-23T00:41:22.865946+0000","last_clean":"2024-02-23T00:41:22.865946+0000","last_became_active":"2024-02-23T00:39:18.796680+0000","last_became_peered":"2024-02-23T00:39:18.796680+0000","last_unstale":"2024-02-23T00:41:22.865946+0000","last_undegraded":"2024-02-23T00:41:22.865946+0000","last_fullsized":"2024-02-23T00:41:22.865946+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:40:27.923003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:27.923003+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.840664+0000","last_change":"2024-02-23T00:40:15.942790+0000","last_active":"2024-02-23T00:41:12.840664+0000","last_peered":"2024-02-23T00:41:12.840664+0000","last_clean":"2024-02-23T00:41:12.840664+0000","last_became_active":"2024-02-23T00:39:14.785363+0000","last_became_peered":"2024-02-23T00:39:14.785363+0000","last_unstale":"2024-02-23T00:41:12.840664+0000","last_undegraded":"2024-02-23T00:41:12.840664+0000","last_fullsized":"2024-02-23T00:41:12.840664+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"80'2","last_scrub_stamp":"2024-02-23T00:40:15.942725+0000","last_deep_scrub":"80'2","last_deep_scrub_stamp":"2024-02-23T00:40:15.942725+0000","last_clean_scrub_stamp":"2024-02-23T00:40:15.942725+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"68'16","reported_seq":69,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.458087+0000","last_change":"2024-02-23T00:39:16.859294+0000","last_active":"2024-02-23T00:41:12.458087+0000","last_peered":"2024-02-23T00:41:12.458087+0000","last_clean":"2024-02-23T00:41:12.458087+0000","last_became_active":"2024-02-23T00:39:16.858945+0000","last_became_peered":"2024-02-23T00:39:16.858945+0000","last_unstale":"2024-02-23T00:41:12.458087+0000","last_undegraded":"2024-02-23T00:41:12.458087+0000","last_fullsized":"2024-02-23T00:41:12.458087+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:39:15.764507+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.226405+0000","last_change":"2024-02-23T00:39:21.160824+0000","last_active":"2024-02-23T00:41:12.226405+0000","last_peered":"2024-02-23T00:41:12.226405+0000","last_clean":"2024-02-23T00:41:12.226405+0000","last_became_active":"2024-02-23T00:39:21.160376+0000","last_became_peered":"2024-02-23T00:39:21.160376+0000","last_unstale":"2024-02-23T00:41:12.226405+0000","last_undegraded":"2024-02-23T00:41:12.226405+0000","last_fullsized":"2024-02-23T00:41:12.226405+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:39:19.779989+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.607648+0000","last_change":"2024-02-23T00:40:25.581008+0000","last_active":"2024-02-23T00:41:12.607648+0000","last_peered":"2024-02-23T00:41:12.607648+0000","last_clean":"2024-02-23T00:41:12.607648+0000","last_became_active":"2024-02-23T00:39:18.791457+0000","last_became_peered":"2024-02-23T00:39:18.791457+0000","last_unstale":"2024-02-23T00:41:12.607648+0000","last_undegraded":"2024-02-23T00:41:12.607648+0000","last_fullsized":"2024-02-23T00:41:12.607648+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:25.580938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:25.580938+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.841353+0000","last_change":"2024-02-23T00:40:29.977372+0000","last_active":"2024-02-23T00:41:12.841353+0000","last_peered":"2024-02-23T00:41:12.841353+0000","last_clean":"2024-02-23T00:41:12.841353+0000","last_became_active":"2024-02-23T00:39:14.784092+0000","last_became_peered":"2024-02-23T00:39:14.784092+0000","last_unstale":"2024-02-23T00:41:12.841353+0000","last_undegraded":"2024-02-23T00:41:12.841353+0000","last_fullsized":"2024-02-23T00:41:12.841353+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:29.977275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:29.977275+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"68'17","reported_seq":78,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.227088+0000","last_change":"2024-02-23T00:40:20.143358+0000","last_active":"2024-02-23T00:41:12.227088+0000","last_peered":"2024-02-23T00:41:12.227088+0000","last_clean":"2024-02-23T00:41:12.227088+0000","last_became_active":"2024-02-23T00:39:16.857232+0000","last_became_peered":"2024-02-23T00:39:16.857232+0000","last_unstale":"2024-02-23T00:41:12.227088+0000","last_undegraded":"2024-02-23T00:41:12.227088+0000","last_fullsized":"2024-02-23T00:41:12.227088+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:40:20.143288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:20.143288+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.607208+0000","last_change":"2024-02-23T00:40:54.739607+0000","last_active":"2024-02-23T00:41:12.607208+0000","last_peered":"2024-02-23T00:41:12.607208+0000","last_clean":"2024-02-23T00:41:12.607208+0000","last_became_active":"2024-02-23T00:39:20.795351+0000","last_became_peered":"2024-02-23T00:39:20.795351+0000","last_unstale":"2024-02-23T00:41:12.607208+0000","last_undegraded":"2024-02-23T00:41:12.607208+0000","last_fullsized":"2024-02-23T00:41:12.607208+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:54.739534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:54.739534+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.840598+0000","last_change":"2024-02-23T00:40:38.957682+0000","last_active":"2024-02-23T00:41:12.840598+0000","last_peered":"2024-02-23T00:41:12.840598+0000","last_clean":"2024-02-23T00:41:12.840598+0000","last_became_active":"2024-02-23T00:39:18.797287+0000","last_became_peered":"2024-02-23T00:39:18.797287+0000","last_unstale":"2024-02-23T00:41:12.840598+0000","last_undegraded":"2024-02-23T00:41:12.840598+0000","last_fullsized":"2024-02-23T00:41:12.840598+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:38.957578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:38.957578+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.841926+0000","last_change":"2024-02-23T00:40:22.050580+0000","last_active":"2024-02-23T00:41:12.841926+0000","last_peered":"2024-02-23T00:41:12.841926+0000","last_clean":"2024-02-23T00:41:12.841926+0000","last_became_active":"2024-02-23T00:39:14.783325+0000","last_became_peered":"2024-02-23T00:39:14.783325+0000","last_unstale":"2024-02-23T00:41:12.841926+0000","last_undegraded":"2024-02-23T00:41:12.841926+0000","last_fullsized":"2024-02-23T00:41:12.841926+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:22.050497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:22.050497+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"67'4","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:20.947503+0000","last_change":"2024-02-23T00:41:20.947463+0000","last_active":"2024-02-23T00:41:20.947503+0000","last_peered":"2024-02-23T00:41:20.947503+0000","last_clean":"2024-02-23T00:41:20.947503+0000","last_became_active":"2024-02-23T00:39:16.785830+0000","last_became_peered":"2024-02-23T00:39:16.785830+0000","last_unstale":"2024-02-23T00:41:20.947503+0000","last_undegraded":"2024-02-23T00:41:20.947503+0000","last_fullsized":"2024-02-23T00:41:20.947503+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'4","last_scrub_stamp":"2024-02-23T00:41:20.947377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:20.947377+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.226647+0000","last_change":"2024-02-23T00:40:56.892857+0000","last_active":"2024-02-23T00:41:12.226647+0000","last_peered":"2024-02-23T00:41:12.226647+0000","last_clean":"2024-02-23T00:41:12.226647+0000","last_became_active":"2024-02-23T00:39:20.799053+0000","last_became_peered":"2024-02-23T00:39:20.799053+0000","last_unstale":"2024-02-23T00:41:12.226647+0000","last_undegraded":"2024-02-23T00:41:12.226647+0000","last_fullsized":"2024-02-23T00:41:12.226647+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:56.892797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:56.892797+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:13.178044+0000","last_change":"2024-02-23T00:40:59.669880+0000","last_active":"2024-02-23T00:41:13.178044+0000","last_peered":"2024-02-23T00:41:13.178044+0000","last_clean":"2024-02-23T00:41:13.178044+0000","last_became_active":"2024-02-23T00:39:18.800382+0000","last_became_peered":"2024-02-23T00:39:18.800382+0000","last_unstale":"2024-02-23T00:41:13.178044+0000","last_undegraded":"2024-02-23T00:41:13.178044+0000","last_fullsized":"2024-02-23T00:41:13.178044+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:59.669810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:59.669810+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"60'1","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:13.178211+0000","last_change":"2024-02-23T00:40:34.909212+0000","last_active":"2024-02-23T00:41:13.178211+0000","last_peered":"2024-02-23T00:41:13.178211+0000","last_clean":"2024-02-23T00:41:13.178211+0000","last_became_active":"2024-02-23T00:39:14.775804+0000","last_became_peered":"2024-02-23T00:39:14.775804+0000","last_unstale":"2024-02-23T00:41:13.178211+0000","last_undegraded":"2024-02-23T00:41:13.178211+0000","last_fullsized":"2024-02-23T00:41:13.178211+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'1","last_scrub_stamp":"2024-02-23T00:40:34.909135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:34.909135+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"67'11","reported_seq":62,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.226013+0000","last_change":"2024-02-23T00:40:55.928416+0000","last_active":"2024-02-23T00:41:12.226013+0000","last_peered":"2024-02-23T00:41:12.226013+0000","last_clean":"2024-02-23T00:41:12.226013+0000","last_became_active":"2024-02-23T00:39:16.783896+0000","last_became_peered":"2024-02-23T00:39:16.783896+0000","last_unstale":"2024-02-23T00:41:12.226013+0000","last_undegraded":"2024-02-23T00:41:12.226013+0000","last_fullsized":"2024-02-23T00:41:12.226013+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'11","last_scrub_stamp":"2024-02-23T00:40:55.928349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:55.928349+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.457102+0000","last_change":"2024-02-23T00:41:10.892383+0000","last_active":"2024-02-23T00:41:12.457102+0000","last_peered":"2024-02-23T00:41:12.457102+0000","last_clean":"2024-02-23T00:41:12.457102+0000","last_became_active":"2024-02-23T00:39:20.802648+0000","last_became_peered":"2024-02-23T00:39:20.802648+0000","last_unstale":"2024-02-23T00:41:12.457102+0000","last_undegraded":"2024-02-23T00:41:12.457102+0000","last_fullsized":"2024-02-23T00:41:12.457102+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:10.892259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:10.892259+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:17.931503+0000","last_change":"2024-02-23T00:41:17.931479+0000","last_active":"2024-02-23T00:41:17.931503+0000","last_peered":"2024-02-23T00:41:17.931503+0000","last_clean":"2024-02-23T00:41:17.931503+0000","last_became_active":"2024-02-23T00:39:18.799052+0000","last_became_peered":"2024-02-23T00:39:18.799052+0000","last_unstale":"2024-02-23T00:41:17.931503+0000","last_undegraded":"2024-02-23T00:41:17.931503+0000","last_fullsized":"2024-02-23T00:41:17.931503+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_clean_scrub_stamp":"2024-02-23T00:41:17.931413+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.841028+0000","last_change":"2024-02-23T00:40:16.893963+0000","last_active":"2024-02-23T00:41:12.841028+0000","last_peered":"2024-02-23T00:41:12.841028+0000","last_clean":"2024-02-23T00:41:12.841028+0000","last_became_active":"2024-02-23T00:39:14.778455+0000","last_became_peered":"2024-02-23T00:39:14.778455+0000","last_unstale":"2024-02-23T00:41:12.841028+0000","last_undegraded":"2024-02-23T00:41:12.841028+0000","last_fullsized":"2024-02-23T00:41:12.841028+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:16.893878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:16.893878+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"67'9","reported_seq":59,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:13.177930+0000","last_change":"2024-02-23T00:40:52.783268+0000","last_active":"2024-02-23T00:41:13.177930+0000","last_peered":"2024-02-23T00:41:13.177930+0000","last_clean":"2024-02-23T00:41:13.177930+0000","last_became_active":"2024-02-23T00:39:16.786936+0000","last_became_peered":"2024-02-23T00:39:16.786936+0000","last_unstale":"2024-02-23T00:41:13.177930+0000","last_undegraded":"2024-02-23T00:41:13.177930+0000","last_fullsized":"2024-02-23T00:41:13.177930+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'9","last_scrub_stamp":"2024-02-23T00:40:52.783146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:52.783146+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.226653+0000","last_change":"2024-02-23T00:40:54.938288+0000","last_active":"2024-02-23T00:41:12.226653+0000","last_peered":"2024-02-23T00:41:12.226653+0000","last_clean":"2024-02-23T00:41:12.226653+0000","last_became_active":"2024-02-23T00:39:20.795210+0000","last_became_peered":"2024-02-23T00:39:20.795210+0000","last_unstale":"2024-02-23T00:41:12.226653+0000","last_undegraded":"2024-02-23T00:41:12.226653+0000","last_fullsized":"2024-02-23T00:41:12.226653+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:54.938209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:54.938209+0000","last_clean_scrub_stamp":"2024-02-23T00:40:54.938209+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"67'8","reported_seq":76,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.942914+0000","last_change":"2024-02-23T00:41:25.942914+0000","last_active":"2024-02-23T00:41:25.942914+0000","last_peered":"2024-02-23T00:41:25.942914+0000","last_clean":"2024-02-23T00:41:25.942914+0000","last_became_active":"2024-02-23T00:39:18.787313+0000","last_became_peered":"2024-02-23T00:39:18.787313+0000","last_unstale":"2024-02-23T00:41:25.942914+0000","last_undegraded":"2024-02-23T00:41:25.942914+0000","last_fullsized":"2024-02-23T00:41:25.942914+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:41:25.942822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:25.942822+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.457536+0000","last_change":"2024-02-23T00:40:16.547440+0000","last_active":"2024-02-23T00:41:12.457536+0000","last_peered":"2024-02-23T00:41:12.457536+0000","last_clean":"2024-02-23T00:41:12.457536+0000","last_became_active":"2024-02-23T00:39:14.773772+0000","last_became_peered":"2024-02-23T00:39:14.773772+0000","last_unstale":"2024-02-23T00:41:12.457536+0000","last_undegraded":"2024-02-23T00:41:12.457536+0000","last_fullsized":"2024-02-23T00:41:12.457536+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:16.547328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:16.547328+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"68'15","reported_seq":75,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.841697+0000","last_change":"2024-02-23T00:40:42.867136+0000","last_active":"2024-02-23T00:41:12.841697+0000","last_peered":"2024-02-23T00:41:12.841697+0000","last_clean":"2024-02-23T00:41:12.841697+0000","last_became_active":"2024-02-23T00:39:16.858953+0000","last_became_peered":"2024-02-23T00:39:16.858953+0000","last_unstale":"2024-02-23T00:41:12.841697+0000","last_undegraded":"2024-02-23T00:41:12.841697+0000","last_fullsized":"2024-02-23T00:41:12.841697+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:40:42.867043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:42.867043+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:22.855631+0000","last_change":"2024-02-23T00:41:22.855592+0000","last_active":"2024-02-23T00:41:22.855631+0000","last_peered":"2024-02-23T00:41:22.855631+0000","last_clean":"2024-02-23T00:41:22.855631+0000","last_became_active":"2024-02-23T00:39:20.800422+0000","last_became_peered":"2024-02-23T00:39:20.800422+0000","last_unstale":"2024-02-23T00:41:22.855631+0000","last_undegraded":"2024-02-23T00:41:22.855631+0000","last_fullsized":"2024-02-23T00:41:22.855631+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:22.855516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:22.855516+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"67'8","reported_seq":76,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:22.865939+0000","last_change":"2024-02-23T00:41:08.785056+0000","last_active":"2024-02-23T00:41:22.865939+0000","last_peered":"2024-02-23T00:41:22.865939+0000","last_clean":"2024-02-23T00:41:22.865939+0000","last_became_active":"2024-02-23T00:39:18.803566+0000","last_became_peered":"2024-02-23T00:39:18.803566+0000","last_unstale":"2024-02-23T00:41:22.865939+0000","last_undegraded":"2024-02-23T00:41:22.865939+0000","last_fullsized":"2024-02-23T00:41:22.865939+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:41:08.784965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:08.784965+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.841205+0000","last_change":"2024-02-23T00:40:29.990344+0000","last_active":"2024-02-23T00:41:12.841205+0000","last_peered":"2024-02-23T00:41:12.841205+0000","last_clean":"2024-02-23T00:41:12.841205+0000","last_became_active":"2024-02-23T00:39:14.785590+0000","last_became_peered":"2024-02-23T00:39:14.785590+0000","last_unstale":"2024-02-23T00:41:12.841205+0000","last_undegraded":"2024-02-23T00:41:12.841205+0000","last_fullsized":"2024-02-23T00:41:12.841205+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_clean_scrub_stamp":"2024-02-23T00:40:29.990255+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"68'22","reported_seq":84,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.227023+0000","last_change":"2024-02-23T00:39:16.857807+0000","last_active":"2024-02-23T00:41:12.227023+0000","last_peered":"2024-02-23T00:41:12.227023+0000","last_clean":"2024-02-23T00:41:12.227023+0000","last_became_active":"2024-02-23T00:39:16.857437+0000","last_became_peered":"2024-02-23T00:39:16.857437+0000","last_unstale":"2024-02-23T00:41:12.227023+0000","last_undegraded":"2024-02-23T00:41:12.227023+0000","last_fullsized":"2024-02-23T00:41:12.227023+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:39:15.764507+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":39,"num_read_kb":32,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.607726+0000","last_change":"2024-02-23T00:40:26.576403+0000","last_active":"2024-02-23T00:41:12.607726+0000","last_peered":"2024-02-23T00:41:12.607726+0000","last_clean":"2024-02-23T00:41:12.607726+0000","last_became_active":"2024-02-23T00:39:21.162036+0000","last_became_peered":"2024-02-23T00:39:21.162036+0000","last_unstale":"2024-02-23T00:41:12.607726+0000","last_undegraded":"2024-02-23T00:41:12.607726+0000","last_fullsized":"2024-02-23T00:41:12.607726+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:26.576296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:26.576296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.841800+0000","last_change":"2024-02-23T00:40:25.025989+0000","last_active":"2024-02-23T00:41:12.841800+0000","last_peered":"2024-02-23T00:41:12.841800+0000","last_clean":"2024-02-23T00:41:12.841800+0000","last_became_active":"2024-02-23T00:39:18.800297+0000","last_became_peered":"2024-02-23T00:39:18.800297+0000","last_unstale":"2024-02-23T00:41:12.841800+0000","last_undegraded":"2024-02-23T00:41:12.841800+0000","last_fullsized":"2024-02-23T00:41:12.841800+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:25.025890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:25.025890+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.840682+0000","last_change":"2024-02-23T00:40:53.113312+0000","last_active":"2024-02-23T00:41:12.840682+0000","last_peered":"2024-02-23T00:41:12.840682+0000","last_clean":"2024-02-23T00:41:12.840682+0000","last_became_active":"2024-02-23T00:39:14.783164+0000","last_became_peered":"2024-02-23T00:39:14.783164+0000","last_unstale":"2024-02-23T00:41:12.840682+0000","last_undegraded":"2024-02-23T00:41:12.840682+0000","last_fullsized":"2024-02-23T00:41:12.840682+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:53.113232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:53.113232+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"68'12","reported_seq":68,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:23.904628+0000","last_change":"2024-02-23T00:41:23.904595+0000","last_active":"2024-02-23T00:41:23.904628+0000","last_peered":"2024-02-23T00:41:23.904628+0000","last_clean":"2024-02-23T00:41:23.904628+0000","last_became_active":"2024-02-23T00:39:16.858449+0000","last_became_peered":"2024-02-23T00:39:16.858449+0000","last_unstale":"2024-02-23T00:41:23.904628+0000","last_undegraded":"2024-02-23T00:41:23.904628+0000","last_fullsized":"2024-02-23T00:41:23.904628+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'12","last_scrub_stamp":"2024-02-23T00:41:23.904511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:23.904511+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.225808+0000","last_change":"2024-02-23T00:40:40.360889+0000","last_active":"2024-02-23T00:41:12.225808+0000","last_peered":"2024-02-23T00:41:12.225808+0000","last_clean":"2024-02-23T00:41:12.225808+0000","last_became_active":"2024-02-23T00:39:20.796041+0000","last_became_peered":"2024-02-23T00:39:20.796041+0000","last_unstale":"2024-02-23T00:41:12.225808+0000","last_undegraded":"2024-02-23T00:41:12.225808+0000","last_fullsized":"2024-02-23T00:41:12.225808+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:40.360797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:40.360797+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.225884+0000","last_change":"2024-02-23T00:40:26.171967+0000","last_active":"2024-02-23T00:41:12.225884+0000","last_peered":"2024-02-23T00:41:12.225884+0000","last_clean":"2024-02-23T00:41:12.225884+0000","last_became_active":"2024-02-23T00:39:18.798624+0000","last_became_peered":"2024-02-23T00:39:18.798624+0000","last_unstale":"2024-02-23T00:41:12.225884+0000","last_undegraded":"2024-02-23T00:41:12.225884+0000","last_fullsized":"2024-02-23T00:41:12.225884+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:26.171869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:26.171869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.225342+0000","last_change":"2024-02-23T00:40:24.370317+0000","last_active":"2024-02-23T00:41:12.225342+0000","last_peered":"2024-02-23T00:41:12.225342+0000","last_clean":"2024-02-23T00:41:12.225342+0000","last_became_active":"2024-02-23T00:39:14.774244+0000","last_became_peered":"2024-02-23T00:39:14.774244+0000","last_unstale":"2024-02-23T00:41:12.225342+0000","last_undegraded":"2024-02-23T00:41:12.225342+0000","last_fullsized":"2024-02-23T00:41:12.225342+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:24.370253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:24.370253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"68'21","reported_seq":85,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.457974+0000","last_change":"2024-02-23T00:39:16.859093+0000","last_active":"2024-02-23T00:41:12.457974+0000","last_peered":"2024-02-23T00:41:12.457974+0000","last_clean":"2024-02-23T00:41:12.457974+0000","last_became_active":"2024-02-23T00:39:16.858760+0000","last_became_peered":"2024-02-23T00:39:16.858760+0000","last_unstale":"2024-02-23T00:41:12.457974+0000","last_undegraded":"2024-02-23T00:41:12.457974+0000","last_fullsized":"2024-02-23T00:41:12.457974+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:39:15.764507+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.226313+0000","last_change":"2024-02-23T00:40:24.129150+0000","last_active":"2024-02-23T00:41:12.226313+0000","last_peered":"2024-02-23T00:41:12.226313+0000","last_clean":"2024-02-23T00:41:12.226313+0000","last_became_active":"2024-02-23T00:39:20.793710+0000","last_became_peered":"2024-02-23T00:39:20.793710+0000","last_unstale":"2024-02-23T00:41:12.226313+0000","last_undegraded":"2024-02-23T00:41:12.226313+0000","last_fullsized":"2024-02-23T00:41:12.226313+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:24.129083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:24.129083+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"60'2","reported_seq":58,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.226471+0000","last_change":"2024-02-23T00:40:38.029108+0000","last_active":"2024-02-23T00:41:12.226471+0000","last_peered":"2024-02-23T00:41:12.226471+0000","last_clean":"2024-02-23T00:41:12.226471+0000","last_became_active":"2024-02-23T00:39:14.779205+0000","last_became_peered":"2024-02-23T00:39:14.779205+0000","last_unstale":"2024-02-23T00:41:12.226471+0000","last_undegraded":"2024-02-23T00:41:12.226471+0000","last_fullsized":"2024-02-23T00:41:12.226471+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-02-23T00:40:38.029000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:38.029000+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"68'25","reported_seq":97,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.225833+0000","last_change":"2024-02-23T00:41:08.004573+0000","last_active":"2024-02-23T00:41:12.225833+0000","last_peered":"2024-02-23T00:41:12.225833+0000","last_clean":"2024-02-23T00:41:12.225833+0000","last_became_active":"2024-02-23T00:39:16.783926+0000","last_became_peered":"2024-02-23T00:39:16.783926+0000","last_unstale":"2024-02-23T00:41:12.225833+0000","last_undegraded":"2024-02-23T00:41:12.225833+0000","last_fullsized":"2024-02-23T00:41:12.225833+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'25","last_scrub_stamp":"2024-02-23T00:41:08.004489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:08.004489+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.225254+0000","last_change":"2024-02-23T00:40:49.365809+0000","last_active":"2024-02-23T00:41:12.225254+0000","last_peered":"2024-02-23T00:41:12.225254+0000","last_clean":"2024-02-23T00:41:12.225254+0000","last_became_active":"2024-02-23T00:39:18.796323+0000","last_became_peered":"2024-02-23T00:39:18.796323+0000","last_unstale":"2024-02-23T00:41:12.225254+0000","last_undegraded":"2024-02-23T00:41:12.225254+0000","last_fullsized":"2024-02-23T00:41:12.225254+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_clean_scrub_stamp":"2024-02-23T00:40:49.365711+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"67'1","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.457106+0000","last_change":"2024-02-23T00:41:00.809258+0000","last_active":"2024-02-23T00:41:12.457106+0000","last_peered":"2024-02-23T00:41:12.457106+0000","last_clean":"2024-02-23T00:41:12.457106+0000","last_became_active":"2024-02-23T00:39:20.800858+0000","last_became_peered":"2024-02-23T00:39:20.800858+0000","last_unstale":"2024-02-23T00:41:12.457106+0000","last_undegraded":"2024-02-23T00:41:12.457106+0000","last_fullsized":"2024-02-23T00:41:12.457106+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:41:00.809190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:00.809190+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:13.178095+0000","last_change":"2024-02-23T00:40:55.723385+0000","last_active":"2024-02-23T00:41:13.178095+0000","last_peered":"2024-02-23T00:41:13.178095+0000","last_clean":"2024-02-23T00:41:13.178095+0000","last_became_active":"2024-02-23T00:39:14.779952+0000","last_became_peered":"2024-02-23T00:39:14.779952+0000","last_unstale":"2024-02-23T00:41:13.178095+0000","last_undegraded":"2024-02-23T00:41:13.178095+0000","last_fullsized":"2024-02-23T00:41:13.178095+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:55.723309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:55.723309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"68'16","reported_seq":73,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:13.177965+0000","last_change":"2024-02-23T00:40:41.772244+0000","last_active":"2024-02-23T00:41:13.177965+0000","last_peered":"2024-02-23T00:41:13.177965+0000","last_clean":"2024-02-23T00:41:13.177965+0000","last_became_active":"2024-02-23T00:39:16.787468+0000","last_became_peered":"2024-02-23T00:39:16.787468+0000","last_unstale":"2024-02-23T00:41:13.177965+0000","last_undegraded":"2024-02-23T00:41:13.177965+0000","last_fullsized":"2024-02-23T00:41:13.177965+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:40:41.772150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:41.772150+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"91'9","reported_seq":77,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:22.865700+0000","last_change":"2024-02-23T00:41:12.843814+0000","last_active":"2024-02-23T00:41:22.865700+0000","last_peered":"2024-02-23T00:41:22.865700+0000","last_clean":"2024-02-23T00:41:22.865700+0000","last_became_active":"2024-02-23T00:39:18.801297+0000","last_became_peered":"2024-02-23T00:39:18.801297+0000","last_unstale":"2024-02-23T00:41:22.865700+0000","last_undegraded":"2024-02-23T00:41:22.865700+0000","last_fullsized":"2024-02-23T00:41:22.865700+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'9","last_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_deep_scrub":"91'9","last_deep_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_clean_scrub_stamp":"2024-02-23T00:41:12.843733+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:18.910709+0000","last_change":"2024-02-23T00:41:18.910682+0000","last_active":"2024-02-23T00:41:18.910709+0000","last_peered":"2024-02-23T00:41:18.910709+0000","last_clean":"2024-02-23T00:41:18.910709+0000","last_became_active":"2024-02-23T00:39:20.790053+0000","last_became_peered":"2024-02-23T00:39:20.790053+0000","last_unstale":"2024-02-23T00:41:18.910709+0000","last_undegraded":"2024-02-23T00:41:18.910709+0000","last_fullsized":"2024-02-23T00:41:18.910709+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:18.910605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:18.910605+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.841209+0000","last_change":"2024-02-23T00:40:53.824228+0000","last_active":"2024-02-23T00:41:12.841209+0000","last_peered":"2024-02-23T00:41:12.841209+0000","last_clean":"2024-02-23T00:41:12.841209+0000","last_became_active":"2024-02-23T00:39:14.784015+0000","last_became_peered":"2024-02-23T00:39:14.784015+0000","last_unstale":"2024-02-23T00:41:12.841209+0000","last_undegraded":"2024-02-23T00:41:12.841209+0000","last_fullsized":"2024-02-23T00:41:12.841209+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:53.824161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:53.824161+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"86'35","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:22.866020+0000","last_change":"2024-02-23T00:40:48.803965+0000","last_active":"2024-02-23T00:41:22.866020+0000","last_peered":"2024-02-23T00:41:22.866020+0000","last_clean":"2024-02-23T00:41:22.866020+0000","last_became_active":"2024-02-23T00:40:42.933322+0000","last_became_peered":"2024-02-23T00:40:42.933322+0000","last_unstale":"2024-02-23T00:41:22.866020+0000","last_undegraded":"2024-02-23T00:41:22.866020+0000","last_fullsized":"2024-02-23T00:41:22.866020+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'35","last_scrub_stamp":"2024-02-23T00:40:48.803893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:48.803893+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:19.899822+0000","last_change":"2024-02-23T00:41:19.899792+0000","last_active":"2024-02-23T00:41:19.899822+0000","last_peered":"2024-02-23T00:41:19.899822+0000","last_clean":"2024-02-23T00:41:19.899822+0000","last_became_active":"2024-02-23T00:39:18.802134+0000","last_became_peered":"2024-02-23T00:39:18.802134+0000","last_unstale":"2024-02-23T00:41:19.899822+0000","last_undegraded":"2024-02-23T00:41:19.899822+0000","last_fullsized":"2024-02-23T00:41:19.899822+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:19.899710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:19.899710+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.607894+0000","last_change":"2024-02-23T00:40:36.598336+0000","last_active":"2024-02-23T00:41:12.607894+0000","last_peered":"2024-02-23T00:41:12.607894+0000","last_clean":"2024-02-23T00:41:12.607894+0000","last_became_active":"2024-02-23T00:39:21.160457+0000","last_became_peered":"2024-02-23T00:39:21.160457+0000","last_unstale":"2024-02-23T00:41:12.607894+0000","last_undegraded":"2024-02-23T00:41:12.607894+0000","last_fullsized":"2024-02-23T00:41:12.607894+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:36.598261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:36.598261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"68'5","reported_seq":173,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:26.018675+0000","last_change":"2024-02-23T00:41:00.648420+0000","last_active":"2024-02-23T00:41:26.018675+0000","last_peered":"2024-02-23T00:41:26.018675+0000","last_clean":"2024-02-23T00:41:26.018675+0000","last_became_active":"2024-02-23T00:39:14.779109+0000","last_became_peered":"2024-02-23T00:39:14.779109+0000","last_unstale":"2024-02-23T00:41:26.018675+0000","last_undegraded":"2024-02-23T00:41:26.018675+0000","last_fullsized":"2024-02-23T00:41:26.018675+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'5","last_scrub_stamp":"2024-02-23T00:41:00.648353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:00.648353+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":125,"num_read_kb":120,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.225606+0000","last_change":"2024-02-23T00:41:00.318702+0000","last_active":"2024-02-23T00:41:12.225606+0000","last_peered":"2024-02-23T00:41:12.225606+0000","last_clean":"2024-02-23T00:41:12.225606+0000","last_became_active":"2024-02-23T00:39:14.772411+0000","last_became_peered":"2024-02-23T00:39:14.772411+0000","last_unstale":"2024-02-23T00:41:12.225606+0000","last_undegraded":"2024-02-23T00:41:12.225606+0000","last_fullsized":"2024-02-23T00:41:12.225606+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:00.318633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:00.318633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"68'19","reported_seq":85,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:13.178299+0000","last_change":"2024-02-23T00:40:44.813607+0000","last_active":"2024-02-23T00:41:13.178299+0000","last_peered":"2024-02-23T00:41:13.178299+0000","last_clean":"2024-02-23T00:41:13.178299+0000","last_became_active":"2024-02-23T00:39:16.787484+0000","last_became_peered":"2024-02-23T00:39:16.787484+0000","last_unstale":"2024-02-23T00:41:13.178299+0000","last_undegraded":"2024-02-23T00:41:13.178299+0000","last_fullsized":"2024-02-23T00:41:13.178299+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'19","last_scrub_stamp":"2024-02-23T00:40:44.813506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:44.813506+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.226860+0000","last_change":"2024-02-23T00:40:36.503413+0000","last_active":"2024-02-23T00:41:12.226860+0000","last_peered":"2024-02-23T00:41:12.226860+0000","last_clean":"2024-02-23T00:41:12.226860+0000","last_became_active":"2024-02-23T00:39:18.789330+0000","last_became_peered":"2024-02-23T00:39:18.789330+0000","last_unstale":"2024-02-23T00:41:12.226860+0000","last_undegraded":"2024-02-23T00:41:12.226860+0000","last_fullsized":"2024-02-23T00:41:12.226860+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:36.503342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:36.503342+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.840280+0000","last_change":"2024-02-23T00:40:51.806086+0000","last_active":"2024-02-23T00:41:12.840280+0000","last_peered":"2024-02-23T00:41:12.840280+0000","last_clean":"2024-02-23T00:41:12.840280+0000","last_became_active":"2024-02-23T00:39:20.797226+0000","last_became_peered":"2024-02-23T00:39:20.797226+0000","last_unstale":"2024-02-23T00:41:12.840280+0000","last_undegraded":"2024-02-23T00:41:12.840280+0000","last_fullsized":"2024-02-23T00:41:12.840280+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:51.806007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:51.806007+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.225592+0000","last_change":"2024-02-23T00:40:59.958953+0000","last_active":"2024-02-23T00:41:12.225592+0000","last_peered":"2024-02-23T00:41:12.225592+0000","last_clean":"2024-02-23T00:41:12.225592+0000","last_became_active":"2024-02-23T00:39:14.783489+0000","last_became_peered":"2024-02-23T00:39:14.783489+0000","last_unstale":"2024-02-23T00:41:12.225592+0000","last_undegraded":"2024-02-23T00:41:12.225592+0000","last_fullsized":"2024-02-23T00:41:12.225592+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:59.958850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:59.958850+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.841435+0000","last_change":"2024-02-23T00:40:50.760096+0000","last_active":"2024-02-23T00:41:12.841435+0000","last_peered":"2024-02-23T00:41:12.841435+0000","last_clean":"2024-02-23T00:41:12.841435+0000","last_became_active":"2024-02-23T00:39:14.788888+0000","last_became_peered":"2024-02-23T00:39:14.788888+0000","last_unstale":"2024-02-23T00:41:12.841435+0000","last_undegraded":"2024-02-23T00:41:12.841435+0000","last_fullsized":"2024-02-23T00:41:12.841435+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:50.760015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:50.760015+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.607147+0000","last_change":"2024-02-23T00:40:53.770177+0000","last_active":"2024-02-23T00:41:12.607147+0000","last_peered":"2024-02-23T00:41:12.607147+0000","last_clean":"2024-02-23T00:41:12.607147+0000","last_became_active":"2024-02-23T00:39:08.248249+0000","last_became_peered":"2024-02-23T00:39:08.248249+0000","last_unstale":"2024-02-23T00:41:12.607147+0000","last_undegraded":"2024-02-23T00:41:12.607147+0000","last_fullsized":"2024-02-23T00:41:12.607147+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:53.770082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:53.770082+0000","last_clean_scrub_stamp":"2024-02-23T00:40:53.770082+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"67'14","reported_seq":65,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.840560+0000","last_change":"2024-02-23T00:40:41.920396+0000","last_active":"2024-02-23T00:41:12.840560+0000","last_peered":"2024-02-23T00:41:12.840560+0000","last_clean":"2024-02-23T00:41:12.840560+0000","last_became_active":"2024-02-23T00:39:16.858880+0000","last_became_peered":"2024-02-23T00:39:16.858880+0000","last_unstale":"2024-02-23T00:41:12.840560+0000","last_undegraded":"2024-02-23T00:41:12.840560+0000","last_fullsized":"2024-02-23T00:41:12.840560+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'14","last_scrub_stamp":"2024-02-23T00:40:41.920313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:41.920313+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.607787+0000","last_change":"2024-02-23T00:40:48.790072+0000","last_active":"2024-02-23T00:41:12.607787+0000","last_peered":"2024-02-23T00:41:12.607787+0000","last_clean":"2024-02-23T00:41:12.607787+0000","last_became_active":"2024-02-23T00:39:18.793801+0000","last_became_peered":"2024-02-23T00:39:18.793801+0000","last_unstale":"2024-02-23T00:41:12.607787+0000","last_undegraded":"2024-02-23T00:41:12.607787+0000","last_fullsized":"2024-02-23T00:41:12.607787+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:48.789992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:48.789992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:13.177241+0000","last_change":"2024-02-23T00:41:02.714085+0000","last_active":"2024-02-23T00:41:13.177241+0000","last_peered":"2024-02-23T00:41:13.177241+0000","last_clean":"2024-02-23T00:41:13.177241+0000","last_became_active":"2024-02-23T00:39:20.797242+0000","last_became_peered":"2024-02-23T00:39:20.797242+0000","last_unstale":"2024-02-23T00:41:13.177241+0000","last_undegraded":"2024-02-23T00:41:13.177241+0000","last_fullsized":"2024-02-23T00:41:13.177241+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:02.714020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:02.714020+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.457379+0000","last_change":"2024-02-23T00:40:15.584807+0000","last_active":"2024-02-23T00:41:12.457379+0000","last_peered":"2024-02-23T00:41:12.457379+0000","last_clean":"2024-02-23T00:41:12.457379+0000","last_became_active":"2024-02-23T00:39:14.775446+0000","last_became_peered":"2024-02-23T00:39:14.775446+0000","last_unstale":"2024-02-23T00:41:12.457379+0000","last_undegraded":"2024-02-23T00:41:12.457379+0000","last_fullsized":"2024-02-23T00:41:12.457379+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:15.584732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:15.584732+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:26.836896+0000","last_change":"2024-02-23T00:41:26.836896+0000","last_active":"2024-02-23T00:41:26.836896+0000","last_peered":"2024-02-23T00:41:26.836896+0000","last_clean":"2024-02-23T00:41:26.836896+0000","last_became_active":"2024-02-23T00:39:14.784578+0000","last_became_peered":"2024-02-23T00:39:14.784578+0000","last_unstale":"2024-02-23T00:41:26.836896+0000","last_undegraded":"2024-02-23T00:41:26.836896+0000","last_fullsized":"2024-02-23T00:41:26.836896+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:26.836819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:26.836819+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"67'12","reported_seq":62,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.457871+0000","last_change":"2024-02-23T00:40:48.696967+0000","last_active":"2024-02-23T00:41:12.457871+0000","last_peered":"2024-02-23T00:41:12.457871+0000","last_clean":"2024-02-23T00:41:12.457871+0000","last_became_active":"2024-02-23T00:39:16.787143+0000","last_became_peered":"2024-02-23T00:39:16.787143+0000","last_unstale":"2024-02-23T00:41:12.457871+0000","last_undegraded":"2024-02-23T00:41:12.457871+0000","last_fullsized":"2024-02-23T00:41:12.457871+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'12","last_scrub_stamp":"2024-02-23T00:40:48.696895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:48.696895+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.842140+0000","last_change":"2024-02-23T00:40:38.812399+0000","last_active":"2024-02-23T00:41:12.842140+0000","last_peered":"2024-02-23T00:41:12.842140+0000","last_clean":"2024-02-23T00:41:12.842140+0000","last_became_active":"2024-02-23T00:39:18.800378+0000","last_became_peered":"2024-02-23T00:39:18.800378+0000","last_unstale":"2024-02-23T00:41:12.842140+0000","last_undegraded":"2024-02-23T00:41:12.842140+0000","last_fullsized":"2024-02-23T00:41:12.842140+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:38.812327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:38.812327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:13.177124+0000","last_change":"2024-02-23T00:40:27.859722+0000","last_active":"2024-02-23T00:41:13.177124+0000","last_peered":"2024-02-23T00:41:13.177124+0000","last_clean":"2024-02-23T00:41:13.177124+0000","last_became_active":"2024-02-23T00:39:21.160060+0000","last_became_peered":"2024-02-23T00:39:21.160060+0000","last_unstale":"2024-02-23T00:41:13.177124+0000","last_undegraded":"2024-02-23T00:41:13.177124+0000","last_fullsized":"2024-02-23T00:41:13.177124+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:27.859657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:27.859657+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.608010+0000","last_change":"2024-02-23T00:40:41.671069+0000","last_active":"2024-02-23T00:41:12.608010+0000","last_peered":"2024-02-23T00:41:12.608010+0000","last_clean":"2024-02-23T00:41:12.608010+0000","last_became_active":"2024-02-23T00:39:14.783662+0000","last_became_peered":"2024-02-23T00:39:14.783662+0000","last_unstale":"2024-02-23T00:41:12.608010+0000","last_undegraded":"2024-02-23T00:41:12.608010+0000","last_fullsized":"2024-02-23T00:41:12.608010+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:41.670962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:41.670962+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"68'25","reported_seq":95,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.457456+0000","last_change":"2024-02-23T00:40:52.751667+0000","last_active":"2024-02-23T00:41:12.457456+0000","last_peered":"2024-02-23T00:41:12.457456+0000","last_clean":"2024-02-23T00:41:12.457456+0000","last_became_active":"2024-02-23T00:39:16.857278+0000","last_became_peered":"2024-02-23T00:39:16.857278+0000","last_unstale":"2024-02-23T00:41:12.457456+0000","last_undegraded":"2024-02-23T00:41:12.457456+0000","last_fullsized":"2024-02-23T00:41:12.457456+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'25","last_scrub_stamp":"2024-02-23T00:40:52.751597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:52.751597+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.841327+0000","last_change":"2024-02-23T00:40:50.029161+0000","last_active":"2024-02-23T00:41:12.841327+0000","last_peered":"2024-02-23T00:41:12.841327+0000","last_clean":"2024-02-23T00:41:12.841327+0000","last_became_active":"2024-02-23T00:39:18.796664+0000","last_became_peered":"2024-02-23T00:39:18.796664+0000","last_unstale":"2024-02-23T00:41:12.841327+0000","last_undegraded":"2024-02-23T00:41:12.841327+0000","last_fullsized":"2024-02-23T00:41:12.841327+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:50.029087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:50.029087+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.226055+0000","last_change":"2024-02-23T00:40:49.937604+0000","last_active":"2024-02-23T00:41:12.226055+0000","last_peered":"2024-02-23T00:41:12.226055+0000","last_clean":"2024-02-23T00:41:12.226055+0000","last_became_active":"2024-02-23T00:39:20.795271+0000","last_became_peered":"2024-02-23T00:39:20.795271+0000","last_unstale":"2024-02-23T00:41:12.226055+0000","last_undegraded":"2024-02-23T00:41:12.226055+0000","last_fullsized":"2024-02-23T00:41:12.226055+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:49.937526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:49.937526+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:13.178272+0000","last_change":"2024-02-23T00:40:36.888232+0000","last_active":"2024-02-23T00:41:13.178272+0000","last_peered":"2024-02-23T00:41:13.178272+0000","last_clean":"2024-02-23T00:41:13.178272+0000","last_became_active":"2024-02-23T00:39:14.779193+0000","last_became_peered":"2024-02-23T00:39:14.779193+0000","last_unstale":"2024-02-23T00:41:13.178272+0000","last_undegraded":"2024-02-23T00:41:13.178272+0000","last_fullsized":"2024-02-23T00:41:13.178272+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:36.888124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:36.888124+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"68'30","reported_seq":106,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:21.909463+0000","last_change":"2024-02-23T00:41:21.909427+0000","last_active":"2024-02-23T00:41:21.909463+0000","last_peered":"2024-02-23T00:41:21.909463+0000","last_clean":"2024-02-23T00:41:21.909463+0000","last_became_active":"2024-02-23T00:39:16.859002+0000","last_became_peered":"2024-02-23T00:39:16.859002+0000","last_unstale":"2024-02-23T00:41:21.909463+0000","last_undegraded":"2024-02-23T00:41:21.909463+0000","last_fullsized":"2024-02-23T00:41:21.909463+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'30","last_scrub_stamp":"2024-02-23T00:41:21.909340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:21.909340+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.225898+0000","last_change":"2024-02-23T00:40:23.103679+0000","last_active":"2024-02-23T00:41:12.225898+0000","last_peered":"2024-02-23T00:41:12.225898+0000","last_clean":"2024-02-23T00:41:12.225898+0000","last_became_active":"2024-02-23T00:39:18.798829+0000","last_became_peered":"2024-02-23T00:39:18.798829+0000","last_unstale":"2024-02-23T00:41:12.225898+0000","last_undegraded":"2024-02-23T00:41:12.225898+0000","last_fullsized":"2024-02-23T00:41:12.225898+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:23.103586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:23.103586+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.608087+0000","last_change":"2024-02-23T00:40:33.626648+0000","last_active":"2024-02-23T00:41:12.608087+0000","last_peered":"2024-02-23T00:41:12.608087+0000","last_clean":"2024-02-23T00:41:12.608087+0000","last_became_active":"2024-02-23T00:39:21.161928+0000","last_became_peered":"2024-02-23T00:39:21.161928+0000","last_unstale":"2024-02-23T00:41:12.608087+0000","last_undegraded":"2024-02-23T00:41:12.608087+0000","last_fullsized":"2024-02-23T00:41:12.608087+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:33.626500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:33.626500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.841314+0000","last_change":"2024-02-23T00:40:45.971552+0000","last_active":"2024-02-23T00:41:12.841314+0000","last_peered":"2024-02-23T00:41:12.841314+0000","last_clean":"2024-02-23T00:41:12.841314+0000","last_became_active":"2024-02-23T00:39:14.781999+0000","last_became_peered":"2024-02-23T00:39:14.781999+0000","last_unstale":"2024-02-23T00:41:12.841314+0000","last_undegraded":"2024-02-23T00:41:12.841314+0000","last_fullsized":"2024-02-23T00:41:12.841314+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:45.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:45.971465+0000","last_clean_scrub_stamp":"2024-02-23T00:40:45.971465+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"68'20","reported_seq":79,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.225695+0000","last_change":"2024-02-23T00:41:05.002039+0000","last_active":"2024-02-23T00:41:12.225695+0000","last_peered":"2024-02-23T00:41:12.225695+0000","last_clean":"2024-02-23T00:41:12.225695+0000","last_became_active":"2024-02-23T00:39:16.780130+0000","last_became_peered":"2024-02-23T00:39:16.780130+0000","last_unstale":"2024-02-23T00:41:12.225695+0000","last_undegraded":"2024-02-23T00:41:12.225695+0000","last_fullsized":"2024-02-23T00:41:12.225695+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'20","last_scrub_stamp":"2024-02-23T00:41:05.001912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:05.001912+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":76,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:22.865859+0000","last_change":"2024-02-23T00:41:09.882111+0000","last_active":"2024-02-23T00:41:22.865859+0000","last_peered":"2024-02-23T00:41:22.865859+0000","last_clean":"2024-02-23T00:41:22.865859+0000","last_became_active":"2024-02-23T00:39:18.798536+0000","last_became_peered":"2024-02-23T00:39:18.798536+0000","last_unstale":"2024-02-23T00:41:22.865859+0000","last_undegraded":"2024-02-23T00:41:22.865859+0000","last_fullsized":"2024-02-23T00:41:22.865859+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:41:09.882012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:09.882012+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.226826+0000","last_change":"2024-02-23T00:40:46.876448+0000","last_active":"2024-02-23T00:41:12.226826+0000","last_peered":"2024-02-23T00:41:12.226826+0000","last_clean":"2024-02-23T00:41:12.226826+0000","last_became_active":"2024-02-23T00:39:21.160443+0000","last_became_peered":"2024-02-23T00:39:21.160443+0000","last_unstale":"2024-02-23T00:41:12.226826+0000","last_undegraded":"2024-02-23T00:41:12.226826+0000","last_fullsized":"2024-02-23T00:41:12.226826+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:46.876380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:46.876380+0000","last_clean_scrub_stamp":"2024-02-23T00:40:46.876380+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.842080+0000","last_change":"2024-02-23T00:41:04.839634+0000","last_active":"2024-02-23T00:41:12.842080+0000","last_peered":"2024-02-23T00:41:12.842080+0000","last_clean":"2024-02-23T00:41:12.842080+0000","last_became_active":"2024-02-23T00:39:18.800777+0000","last_became_peered":"2024-02-23T00:39:18.800777+0000","last_unstale":"2024-02-23T00:41:12.842080+0000","last_undegraded":"2024-02-23T00:41:12.842080+0000","last_fullsized":"2024-02-23T00:41:12.842080+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_clean_scrub_stamp":"2024-02-23T00:41:04.839555+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.607189+0000","last_change":"2024-02-23T00:40:23.574894+0000","last_active":"2024-02-23T00:41:12.607189+0000","last_peered":"2024-02-23T00:41:12.607189+0000","last_clean":"2024-02-23T00:41:12.607189+0000","last_became_active":"2024-02-23T00:39:14.784121+0000","last_became_peered":"2024-02-23T00:39:14.784121+0000","last_unstale":"2024-02-23T00:41:12.607189+0000","last_undegraded":"2024-02-23T00:41:12.607189+0000","last_fullsized":"2024-02-23T00:41:12.607189+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:23.574813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:23.574813+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"68'17","reported_seq":100,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:13.017153+0000","last_change":"2024-02-23T00:41:13.017122+0000","last_active":"2024-02-23T00:41:13.017153+0000","last_peered":"2024-02-23T00:41:13.017153+0000","last_clean":"2024-02-23T00:41:13.017153+0000","last_became_active":"2024-02-23T00:40:42.933759+0000","last_became_peered":"2024-02-23T00:40:42.933759+0000","last_unstale":"2024-02-23T00:41:13.017153+0000","last_undegraded":"2024-02-23T00:41:13.017153+0000","last_fullsized":"2024-02-23T00:41:13.017153+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:41:13.017043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:13.017043+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.840814+0000","last_change":"2024-02-23T00:40:56.890866+0000","last_active":"2024-02-23T00:41:12.840814+0000","last_peered":"2024-02-23T00:41:12.840814+0000","last_clean":"2024-02-23T00:41:12.840814+0000","last_became_active":"2024-02-23T00:39:20.800800+0000","last_became_peered":"2024-02-23T00:39:20.800800+0000","last_unstale":"2024-02-23T00:41:12.840814+0000","last_undegraded":"2024-02-23T00:41:12.840814+0000","last_fullsized":"2024-02-23T00:41:12.840814+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:56.890792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:56.890792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":75,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:22.865922+0000","last_change":"2024-02-23T00:40:30.998799+0000","last_active":"2024-02-23T00:41:22.865922+0000","last_peered":"2024-02-23T00:41:22.865922+0000","last_clean":"2024-02-23T00:41:22.865922+0000","last_became_active":"2024-02-23T00:39:18.800440+0000","last_became_peered":"2024-02-23T00:39:18.800440+0000","last_unstale":"2024-02-23T00:41:22.865922+0000","last_undegraded":"2024-02-23T00:41:22.865922+0000","last_fullsized":"2024-02-23T00:41:22.865922+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:40:30.998729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:30.998729+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:13.177650+0000","last_change":"2024-02-23T00:40:26.845049+0000","last_active":"2024-02-23T00:41:13.177650+0000","last_peered":"2024-02-23T00:41:13.177650+0000","last_clean":"2024-02-23T00:41:13.177650+0000","last_became_active":"2024-02-23T00:39:14.780526+0000","last_became_peered":"2024-02-23T00:39:14.780526+0000","last_unstale":"2024-02-23T00:41:13.177650+0000","last_undegraded":"2024-02-23T00:41:13.177650+0000","last_fullsized":"2024-02-23T00:41:13.177650+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:26.844984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:26.844984+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"67'15","reported_seq":68,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:13.178393+0000","last_change":"2024-02-23T00:40:18.805518+0000","last_active":"2024-02-23T00:41:13.178393+0000","last_peered":"2024-02-23T00:41:13.178393+0000","last_clean":"2024-02-23T00:41:13.178393+0000","last_became_active":"2024-02-23T00:39:16.789757+0000","last_became_peered":"2024-02-23T00:39:16.789757+0000","last_unstale":"2024-02-23T00:41:13.178393+0000","last_undegraded":"2024-02-23T00:41:13.178393+0000","last_fullsized":"2024-02-23T00:41:13.178393+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:40:18.805441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:18.805441+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:15.081304+0000","last_change":"2024-02-23T00:41:15.081250+0000","last_active":"2024-02-23T00:41:15.081304+0000","last_peered":"2024-02-23T00:41:15.081304+0000","last_clean":"2024-02-23T00:41:15.081304+0000","last_became_active":"2024-02-23T00:39:20.795097+0000","last_became_peered":"2024-02-23T00:39:20.795097+0000","last_unstale":"2024-02-23T00:41:15.081304+0000","last_undegraded":"2024-02-23T00:41:15.081304+0000","last_fullsized":"2024-02-23T00:41:15.081304+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:15.081153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:15.081153+0000","last_clean_scrub_stamp":"2024-02-23T00:41:15.081153+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"67'8","reported_seq":75,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:22.865732+0000","last_change":"2024-02-23T00:40:58.331136+0000","last_active":"2024-02-23T00:41:22.865732+0000","last_peered":"2024-02-23T00:41:22.865732+0000","last_clean":"2024-02-23T00:41:22.865732+0000","last_became_active":"2024-02-23T00:39:18.799544+0000","last_became_peered":"2024-02-23T00:39:18.799544+0000","last_unstale":"2024-02-23T00:41:22.865732+0000","last_undegraded":"2024-02-23T00:41:22.865732+0000","last_fullsized":"2024-02-23T00:41:22.865732+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:40:58.331048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:58.331048+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:13.177845+0000","last_change":"2024-02-23T00:40:39.825090+0000","last_active":"2024-02-23T00:41:13.177845+0000","last_peered":"2024-02-23T00:41:13.177845+0000","last_clean":"2024-02-23T00:41:13.177845+0000","last_became_active":"2024-02-23T00:39:14.780379+0000","last_became_peered":"2024-02-23T00:39:14.780379+0000","last_unstale":"2024-02-23T00:41:13.177845+0000","last_undegraded":"2024-02-23T00:41:13.177845+0000","last_fullsized":"2024-02-23T00:41:13.177845+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:39.825013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:39.825013+0000","last_clean_scrub_stamp":"2024-02-23T00:40:39.825013+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"67'10","reported_seq":58,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:14.059467+0000","last_change":"2024-02-23T00:41:14.059424+0000","last_active":"2024-02-23T00:41:14.059467+0000","last_peered":"2024-02-23T00:41:14.059467+0000","last_clean":"2024-02-23T00:41:14.059467+0000","last_became_active":"2024-02-23T00:39:16.777530+0000","last_became_peered":"2024-02-23T00:39:16.777530+0000","last_unstale":"2024-02-23T00:41:14.059467+0000","last_undegraded":"2024-02-23T00:41:14.059467+0000","last_fullsized":"2024-02-23T00:41:14.059467+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'10","last_scrub_stamp":"2024-02-23T00:41:14.059334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:14.059334+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.226307+0000","last_change":"2024-02-23T00:40:54.348946+0000","last_active":"2024-02-23T00:41:12.226307+0000","last_peered":"2024-02-23T00:41:12.226307+0000","last_clean":"2024-02-23T00:41:12.226307+0000","last_became_active":"2024-02-23T00:39:20.800943+0000","last_became_peered":"2024-02-23T00:39:20.800943+0000","last_unstale":"2024-02-23T00:41:12.226307+0000","last_undegraded":"2024-02-23T00:41:12.226307+0000","last_fullsized":"2024-02-23T00:41:12.226307+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:54.348843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:54.348843+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:13.177975+0000","last_change":"2024-02-23T00:40:35.861920+0000","last_active":"2024-02-23T00:41:13.177975+0000","last_peered":"2024-02-23T00:41:13.177975+0000","last_clean":"2024-02-23T00:41:13.177975+0000","last_became_active":"2024-02-23T00:39:18.799875+0000","last_became_peered":"2024-02-23T00:39:18.799875+0000","last_unstale":"2024-02-23T00:41:13.177975+0000","last_undegraded":"2024-02-23T00:41:13.177975+0000","last_fullsized":"2024-02-23T00:41:13.177975+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:35.861854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:35.861854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.227262+0000","last_change":"2024-02-23T00:40:42.931442+0000","last_active":"2024-02-23T00:41:12.227262+0000","last_peered":"2024-02-23T00:41:12.227262+0000","last_clean":"2024-02-23T00:41:12.227262+0000","last_became_active":"2024-02-23T00:40:42.931041+0000","last_became_peered":"2024-02-23T00:40:42.931041+0000","last_unstale":"2024-02-23T00:41:12.227262+0000","last_undegraded":"2024-02-23T00:41:12.227262+0000","last_fullsized":"2024-02-23T00:41:12.227262+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:19.487609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:19.487609+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"67'17","reported_seq":98,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.226434+0000","last_change":"2024-02-23T00:41:02.038717+0000","last_active":"2024-02-23T00:41:12.226434+0000","last_peered":"2024-02-23T00:41:12.226434+0000","last_clean":"2024-02-23T00:41:12.226434+0000","last_became_active":"2024-02-23T00:40:42.933806+0000","last_became_peered":"2024-02-23T00:40:42.933806+0000","last_unstale":"2024-02-23T00:41:12.226434+0000","last_undegraded":"2024-02-23T00:41:12.226434+0000","last_fullsized":"2024-02-23T00:41:12.226434+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'17","last_scrub_stamp":"2024-02-23T00:41:02.038627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:02.038627+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.607027+0000","last_change":"2024-02-23T00:40:51.777535+0000","last_active":"2024-02-23T00:41:12.607027+0000","last_peered":"2024-02-23T00:41:12.607027+0000","last_clean":"2024-02-23T00:41:12.607027+0000","last_became_active":"2024-02-23T00:39:20.798989+0000","last_became_peered":"2024-02-23T00:39:20.798989+0000","last_unstale":"2024-02-23T00:41:12.607027+0000","last_undegraded":"2024-02-23T00:41:12.607027+0000","last_fullsized":"2024-02-23T00:41:12.607027+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:51.777405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:51.777405+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.227376+0000","last_change":"2024-02-23T00:40:33.492291+0000","last_active":"2024-02-23T00:41:12.227376+0000","last_peered":"2024-02-23T00:41:12.227376+0000","last_clean":"2024-02-23T00:41:12.227376+0000","last_became_active":"2024-02-23T00:39:18.792718+0000","last_became_peered":"2024-02-23T00:39:18.792718+0000","last_unstale":"2024-02-23T00:41:12.227376+0000","last_undegraded":"2024-02-23T00:41:12.227376+0000","last_fullsized":"2024-02-23T00:41:12.227376+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:33.492224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:33.492224+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.225920+0000","last_change":"2024-02-23T00:40:28.450480+0000","last_active":"2024-02-23T00:41:12.225920+0000","last_peered":"2024-02-23T00:41:12.225920+0000","last_clean":"2024-02-23T00:41:12.225920+0000","last_became_active":"2024-02-23T00:39:14.776705+0000","last_became_peered":"2024-02-23T00:39:14.776705+0000","last_unstale":"2024-02-23T00:41:12.225920+0000","last_undegraded":"2024-02-23T00:41:12.225920+0000","last_fullsized":"2024-02-23T00:41:12.225920+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_clean_scrub_stamp":"2024-02-23T00:40:28.450377+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"88'35","reported_seq":99,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.841180+0000","last_change":"2024-02-23T00:40:54.152313+0000","last_active":"2024-02-23T00:41:12.841180+0000","last_peered":"2024-02-23T00:41:12.841180+0000","last_clean":"2024-02-23T00:41:12.841180+0000","last_became_active":"2024-02-23T00:39:16.857413+0000","last_became_peered":"2024-02-23T00:39:16.857413+0000","last_unstale":"2024-02-23T00:41:12.841180+0000","last_undegraded":"2024-02-23T00:41:12.841180+0000","last_fullsized":"2024-02-23T00:41:12.841180+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'35","last_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_deep_scrub":"88'35","last_deep_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_clean_scrub_stamp":"2024-02-23T00:40:54.152239+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.457214+0000","last_change":"2024-02-23T00:41:05.802853+0000","last_active":"2024-02-23T00:41:12.457214+0000","last_peered":"2024-02-23T00:41:12.457214+0000","last_clean":"2024-02-23T00:41:12.457214+0000","last_became_active":"2024-02-23T00:39:20.802938+0000","last_became_peered":"2024-02-23T00:39:20.802938+0000","last_unstale":"2024-02-23T00:41:12.457214+0000","last_undegraded":"2024-02-23T00:41:12.457214+0000","last_fullsized":"2024-02-23T00:41:12.457214+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:05.802777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:05.802777+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.225517+0000","last_change":"2024-02-23T00:41:02.376871+0000","last_active":"2024-02-23T00:41:12.225517+0000","last_peered":"2024-02-23T00:41:12.225517+0000","last_clean":"2024-02-23T00:41:12.225517+0000","last_became_active":"2024-02-23T00:39:18.794476+0000","last_became_peered":"2024-02-23T00:39:18.794476+0000","last_unstale":"2024-02-23T00:41:12.225517+0000","last_undegraded":"2024-02-23T00:41:12.225517+0000","last_fullsized":"2024-02-23T00:41:12.225517+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:02.376804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:02.376804+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:13.178233+0000","last_change":"2024-02-23T00:40:57.681793+0000","last_active":"2024-02-23T00:41:13.178233+0000","last_peered":"2024-02-23T00:41:13.178233+0000","last_clean":"2024-02-23T00:41:13.178233+0000","last_became_active":"2024-02-23T00:39:14.772493+0000","last_became_peered":"2024-02-23T00:39:14.772493+0000","last_unstale":"2024-02-23T00:41:13.178233+0000","last_undegraded":"2024-02-23T00:41:13.178233+0000","last_fullsized":"2024-02-23T00:41:13.178233+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:57.681723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:57.681723+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"68'15","reported_seq":75,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.457578+0000","last_change":"2024-02-23T00:40:47.732636+0000","last_active":"2024-02-23T00:41:12.457578+0000","last_peered":"2024-02-23T00:41:12.457578+0000","last_clean":"2024-02-23T00:41:12.457578+0000","last_became_active":"2024-02-23T00:39:16.783701+0000","last_became_peered":"2024-02-23T00:39:16.783701+0000","last_unstale":"2024-02-23T00:41:12.457578+0000","last_undegraded":"2024-02-23T00:41:12.457578+0000","last_fullsized":"2024-02-23T00:41:12.457578+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:40:47.732524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:47.732524+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.840957+0000","last_change":"2024-02-23T00:40:39.859273+0000","last_active":"2024-02-23T00:41:12.840957+0000","last_peered":"2024-02-23T00:41:12.840957+0000","last_clean":"2024-02-23T00:41:12.840957+0000","last_became_active":"2024-02-23T00:39:21.160429+0000","last_became_peered":"2024-02-23T00:39:21.160429+0000","last_unstale":"2024-02-23T00:41:12.840957+0000","last_undegraded":"2024-02-23T00:41:12.840957+0000","last_fullsized":"2024-02-23T00:41:12.840957+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:39.859197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:39.859197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.226734+0000","last_change":"2024-02-23T00:40:34.533819+0000","last_active":"2024-02-23T00:41:12.226734+0000","last_peered":"2024-02-23T00:41:12.226734+0000","last_clean":"2024-02-23T00:41:12.226734+0000","last_became_active":"2024-02-23T00:39:14.784436+0000","last_became_peered":"2024-02-23T00:39:14.784436+0000","last_unstale":"2024-02-23T00:41:12.226734+0000","last_undegraded":"2024-02-23T00:41:12.226734+0000","last_fullsized":"2024-02-23T00:41:12.226734+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:34.533743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:34.533743+0000","last_clean_scrub_stamp":"2024-02-23T00:40:34.533743+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"68'27","reported_seq":99,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:12.840722+0000","last_change":"2024-02-23T00:40:59.885996+0000","last_active":"2024-02-23T00:41:12.840722+0000","last_peered":"2024-02-23T00:41:12.840722+0000","last_clean":"2024-02-23T00:41:12.840722+0000","last_became_active":"2024-02-23T00:39:16.859401+0000","last_became_peered":"2024-02-23T00:39:16.859401+0000","last_unstale":"2024-02-23T00:41:12.840722+0000","last_undegraded":"2024-02-23T00:41:12.840722+0000","last_fullsized":"2024-02-23T00:41:12.840722+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'27","last_scrub_stamp":"2024-02-23T00:40:59.885908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:59.885908+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"67'8","reported_seq":77,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-23T00:41:22.865936+0000","last_change":"2024-02-23T00:40:44.713885+0000","last_active":"2024-02-23T00:41:22.865936+0000","last_peered":"2024-02-23T00:41:22.865936+0000","last_clean":"2024-02-23T00:41:22.865936+0000","last_became_active":"2024-02-23T00:39:18.793414+0000","last_became_peered":"2024-02-23T00:39:18.793414+0000","last_unstale":"2024-02-23T00:41:22.865936+0000","last_undegraded":"2024-02-23T00:41:22.865936+0000","last_fullsized":"2024-02-23T00:41:22.865936+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:40:44.713825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:44.713825+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1082,"num_read_kb":807,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":610,"ondisk_log_size":610,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":125,"num_read_kb":120,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234014,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297816,"kb_used_data":720,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447336,"statfs":{"total":95995035648,"available":95690072064,"internally_reserved":0,"allocated":737280,"data_stored":368343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.374,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.69499999999999995,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.499,"15min":0.505},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.69999999999999996,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.51000000000000001}]},{"osd":1,"last update":"Fri Feb 23 00:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.54400000000000004,"15min":0.55000000000000004},"min":{"1min":0.38400000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.75600000000000001,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.54100000000000004,"15min":0.54700000000000004},"min":{"1min":0.39600000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.64600000000000002,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.44600000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.53000000000000003,"15min":0.53800000000000003},"min":{"1min":0.40000000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72699999999999998,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.52600000000000002,"15min":0.53200000000000003},"min":{"1min":0.34899999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.65300000000000002,"5min":1.024,"15min":1.024},"last":0.46800000000000003}]},{"osd":3,"last update":"Fri Feb 23 00:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.52100000000000002,"15min":0.52500000000000002},"min":{"1min":0.38200000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.68400000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.55600000000000005,"15min":0.56399999999999995},"min":{"1min":0.38300000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72499999999999998,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Feb 23 00:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.41999999999999998,"15min":0.42799999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.56899999999999995,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.443,"15min":0.45000000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.61099999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.56299999999999994}]},{"osd":5,"last update":"Fri Feb 23 00:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.46600000000000003,"15min":0.47399999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.58399999999999996,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.47099999999999997,"15min":0.47399999999999998},"min":{"1min":0.307,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.625,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Feb 23 00:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.497,"15min":0.503},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.496,"15min":0.503},"min":{"1min":0.33900000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.65500000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.61799999999999999}]}]},{"osd":6,"up_from":47,"seq":201863462949,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297796,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447356,"statfs":{"total":95995035648,"available":95690092544,"internally_reserved":0,"allocated":716800,"data_stored":367693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73199999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.64100000000000001}]},{"osd":1,"last update":"Fri Feb 23 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54900000000000004,"15min":0.55200000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72699999999999998,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77900000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.57199999999999995}]},{"osd":2,"last update":"Fri Feb 23 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58199999999999996,"15min":0.58399999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71199999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.749,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.68600000000000005}]},{"osd":3,"last update":"Fri Feb 23 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57899999999999996,"15min":0.58199999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81399999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59399999999999997,"15min":0.59499999999999997},"min":{"1min":0.41799999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.71599999999999997}]},{"osd":4,"last update":"Fri Feb 23 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.40999999999999998,"15min":0.40400000000000003},"min":{"1min":0.28199999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.433,"15min":0.42699999999999999},"min":{"1min":0.29899999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.48299999999999998}]},{"osd":5,"last update":"Fri Feb 23 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.41899999999999998,"15min":0.41199999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.497,"5min":0.44500000000000001,"15min":0.437},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.44700000000000001}]},{"osd":7,"last update":"Fri Feb 23 00:41:11 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.48999999999999999,"15min":0.48799999999999999},"min":{"1min":0.37,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65200000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76700000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.51300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297824,"kb_used_data":728,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447328,"statfs":{"total":95995035648,"available":95690063872,"internally_reserved":0,"allocated":745472,"data_stored":368977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:40:37 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36899999999999999,"15min":0.34499999999999997},"min":{"1min":0.22600000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.57599999999999996,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.39200000000000002,"15min":0.36099999999999999},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.60299999999999998,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.375}]},{"osd":2,"last update":"Fri Feb 23 00:40:58 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.39600000000000002,"15min":0.39400000000000002},"min":{"1min":0.245,"5min":0.188,"15min":0.188},"max":{"1min":0.53500000000000003,"5min":0.63,"15min":0.63},"last":0.433},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.441,"15min":0.435},"min":{"1min":0.29999999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.67900000000000005,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.33900000000000002}]},{"osd":3,"last update":"Fri Feb 23 00:40:27 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.48599999999999999,"15min":0.48999999999999999},"min":{"1min":0.315,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71699999999999997,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.45500000000000002,"15min":0.45900000000000002},"min":{"1min":0.27300000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.63500000000000001,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.35899999999999999}]},{"osd":4,"last update":"Fri Feb 23 00:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60599999999999998,"15min":0.61199999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.875,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.629},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61199999999999999,"15min":0.61299999999999999},"min":{"1min":0.39500000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83199999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.66600000000000004}]},{"osd":5,"last update":"Fri Feb 23 00:41:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63500000000000001,"15min":0.65700000000000003},"min":{"1min":0.36899999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84599999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69599999999999995,"15min":0.71599999999999997},"min":{"1min":0.442,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.72699999999999998}]},{"osd":6,"last update":"Fri Feb 23 00:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63700000000000001,"15min":0.64200000000000002},"min":{"1min":0.40699999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81999999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.64400000000000002,"15min":0.65100000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.84399999999999997,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.63600000000000001}]},{"osd":7,"last update":"Fri Feb 23 00:41:09 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.46899999999999997,"5min":0.439,"15min":0.439},"max":{"1min":0.85799999999999998,"5min":0.875,"15min":0.875},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.622,"15min":0.622},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.61099999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297820,"kb_used_data":724,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447332,"statfs":{"total":95995035648,"available":95690067968,"internally_reserved":0,"allocated":741376,"data_stored":368568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 23 00:40:33 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.36899999999999999,"15min":0.33500000000000002},"min":{"1min":0.26200000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.56499999999999995,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.223},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.39300000000000002,"15min":0.34999999999999998},"min":{"1min":0.27400000000000002,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.60399999999999998,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.35499999999999998}]},{"osd":2,"last update":"Fri Feb 23 00:41:01 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.39200000000000002},"min":{"1min":0.28799999999999998,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.56799999999999995,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39300000000000002,"15min":0.38},"min":{"1min":0.26200000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.62,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.25}]},{"osd":3,"last update":"Fri Feb 23 00:40:25 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.57099999999999995,"5min":0.754,"15min":0.754},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.503,"15min":0.505},"min":{"1min":0.34799999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.68500000000000005,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.38900000000000001}]},{"osd":4,"last update":"Fri Feb 23 00:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.65700000000000003,"15min":0.69199999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83099999999999996,"5min":3.2909999999999999,"15min":3.2909999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.69599999999999995,"15min":0.74099999999999999},"min":{"1min":0.42599999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.83699999999999997,"5min":3.319,"15min":3.319},"last":0.49399999999999999}]},{"osd":5,"last update":"Fri Feb 23 00:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59499999999999997,"15min":0.60799999999999998},"min":{"1min":0.42599999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.71999999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.65900000000000003,"15min":0.66900000000000004},"min":{"1min":0.45200000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.73199999999999998,"5min":1.131,"15min":1.131},"last":0.44500000000000001}]},{"osd":6,"last update":"Fri Feb 23 00:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.63300000000000001,"15min":0.63500000000000001},"min":{"1min":0.45000000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.85899999999999999,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.52800000000000002}]},{"osd":7,"last update":"Fri Feb 23 00:41:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.66100000000000003,"15min":0.67300000000000004},"min":{"1min":0.41699999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.67300000000000004,"5min":1.163,"15min":1.163},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.48399999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.75600000000000001,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.45900000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297804,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447348,"statfs":{"total":95995035648,"available":95690084352,"internally_reserved":0,"allocated":724992,"data_stored":367949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:40:54 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.38300000000000001,"15min":0.378},"min":{"1min":0.245,"5min":0.222,"15min":0.222},"max":{"1min":0.57399999999999995,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39800000000000002,"15min":0.38600000000000001},"min":{"1min":0.27100000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.41699999999999998}]},{"osd":1,"last update":"Fri Feb 23 00:40:54 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39000000000000001,"15min":0.376},"min":{"1min":0.27500000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.79800000000000004,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.316},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41599999999999998,"15min":0.40000000000000002},"min":{"1min":0.28399999999999997,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.60899999999999999,"5min":0.745,"15min":0.745},"last":0.35799999999999998}]},{"osd":3,"last update":"Fri Feb 23 00:41:25 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.45100000000000001,"15min":0.47999999999999998},"min":{"1min":0.30599999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.622,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.44900000000000001,"15min":0.46600000000000003},"min":{"1min":0.29399999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.53900000000000003,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.375}]},{"osd":4,"last update":"Fri Feb 23 00:40:54 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.59699999999999998},"min":{"1min":0.43099999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.374,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.50900000000000001}]},{"osd":5,"last update":"Fri Feb 23 00:41:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57699999999999996,"15min":0.58199999999999996},"min":{"1min":0.41799999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.748,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.47099999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0740000000000001,"5min":1.089,"15min":1.089},"last":0.55200000000000005}]},{"osd":6,"last update":"Fri Feb 23 00:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.624,"15min":0.624},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59999999999999998,"15min":0.59799999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.48299999999999998}]},{"osd":7,"last update":"Fri Feb 23 00:41:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.66800000000000004,"15min":0.67200000000000004},"min":{"1min":0.50900000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.64100000000000001,"15min":0.64800000000000002},"min":{"1min":0.42799999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.044,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.45600000000000002}]}]},{"osd":3,"up_from":26,"seq":111669149747,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297836,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447316,"statfs":{"total":95995035648,"available":95690051584,"internally_reserved":0,"allocated":757760,"data_stored":368496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.42599999999999999,"15min":0.438},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.66000000000000003,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.25},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.44600000000000001,"15min":0.45800000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.61899999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.307}]},{"osd":1,"last update":"Fri Feb 23 00:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46800000000000003,"15min":0.47899999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.64900000000000002,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.442,"15min":0.45200000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.63800000000000001,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.28699999999999998}]},{"osd":2,"last update":"Fri Feb 23 00:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.48899999999999999,"15min":0.497},"min":{"1min":0.32600000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.47099999999999997,"15min":0.48499999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.72599999999999998,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.32900000000000001}]},{"osd":4,"last update":"Fri Feb 23 00:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.436,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.79400000000000004,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.58199999999999996,"15min":0.58999999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.83999999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.53300000000000003}]},{"osd":5,"last update":"Fri Feb 23 00:41:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.59099999999999997,"15min":0.60299999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74099999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.61299999999999999,"15min":0.623},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85299999999999998,"5min":1.024,"15min":1.024},"last":0.76100000000000001}]},{"osd":6,"last update":"Fri Feb 23 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.59999999999999998,"15min":0.60799999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.75,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.61799999999999999,"15min":0.624},"min":{"1min":0.46300000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.77400000000000002,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.69799999999999995}]},{"osd":7,"last update":"Fri Feb 23 00:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63400000000000001,"15min":0.63600000000000001},"min":{"1min":0.433,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.64900000000000002,"15min":0.65500000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.77300000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.67200000000000004}]}]},{"osd":4,"up_from":32,"seq":137438953519,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297820,"kb_used_data":724,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447332,"statfs":{"total":95995035648,"available":95690067968,"internally_reserved":0,"allocated":741376,"data_stored":368194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55600000000000005,"15min":0.57999999999999996},"min":{"1min":0.32200000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66300000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56799999999999995,"15min":0.58399999999999996},"min":{"1min":0.35699999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71199999999999997,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.629}]},{"osd":1,"last update":"Fri Feb 23 00:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.56100000000000005,"15min":0.58299999999999996},"min":{"1min":0.33000000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.75700000000000001,"5min":1.175,"15min":1.175},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56699999999999995,"15min":0.57499999999999996},"min":{"1min":0.32600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.69999999999999996,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.55600000000000005}]},{"osd":2,"last update":"Fri Feb 23 00:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56999999999999995,"15min":0.57699999999999996},"min":{"1min":0.41399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.753,"5min":1.161,"15min":1.161},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.57399999999999995,"15min":0.58799999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79000000000000004,"5min":1.113,"15min":1.113},"last":0.68700000000000006}]},{"osd":3,"last update":"Fri Feb 23 00:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57399999999999995,"15min":0.57999999999999996},"min":{"1min":0.38200000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81999999999999995,"5min":1.143,"15min":1.143},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.57199999999999995,"15min":0.58199999999999996},"min":{"1min":0.36199999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79100000000000004,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.53500000000000003}]},{"osd":5,"last update":"Fri Feb 23 00:41:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.39600000000000002,"15min":0.371},"min":{"1min":0.251,"5min":0.246,"15min":0.246},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.44500000000000001,"15min":0.41599999999999998},"min":{"1min":0.29999999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.67300000000000004}]},{"osd":6,"last update":"Fri Feb 23 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.40200000000000002,"15min":0.39700000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.40300000000000002,"15min":0.39600000000000002},"min":{"1min":0.30499999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.59699999999999998}]},{"osd":7,"last update":"Fri Feb 23 00:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.48999999999999999,"15min":0.48399999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.70999999999999996}]}]},{"osd":5,"up_from":39,"seq":167503724586,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297824,"kb_used_data":728,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447328,"statfs":{"total":95995035648,"available":95690063872,"internally_reserved":0,"allocated":745472,"data_stored":369328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:41:17 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.50800000000000001,"15min":0.50900000000000001},"min":{"1min":0.32200000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.67600000000000005,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.51800000000000002,"15min":0.52300000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64700000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.58399999999999996}]},{"osd":1,"last update":"Fri Feb 23 00:41:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.65800000000000003,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.53300000000000003,"15min":0.54100000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68500000000000005,"5min":1.004,"15min":1.004},"last":0.54000000000000004}]},{"osd":2,"last update":"Fri Feb 23 00:41:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.53900000000000003,"15min":0.54700000000000004},"min":{"1min":0.38,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.70999999999999996,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.36099999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69799999999999995,"5min":1.026,"15min":1.026},"last":0.59699999999999998}]},{"osd":3,"last update":"Fri Feb 23 00:41:17 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.52500000000000002,"15min":0.53000000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.629,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55800000000000005,"15min":0.55900000000000005},"min":{"1min":0.41699999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72199999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.55700000000000005}]},{"osd":4,"last update":"Fri Feb 23 00:41:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.39600000000000002,"15min":0.38300000000000001},"min":{"1min":0.27100000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.63700000000000001,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.41299999999999998,"15min":0.39000000000000001},"min":{"1min":0.32400000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.51900000000000002}]},{"osd":6,"last update":"Fri Feb 23 00:40:39 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.42699999999999999,"15min":0.42499999999999999},"min":{"1min":0.313,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.44900000000000001,"15min":0.441},"min":{"1min":0.314,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.50600000000000001}]},{"osd":7,"last update":"Fri Feb 23 00:41:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51500000000000001,"15min":0.51100000000000001},"min":{"1min":0.34300000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72299999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.60899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-23T00:41:30.918 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph pg dump --format=json 2024-02-23T00:41:31.196 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:41:31.644 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:30.115465+0000 osd.7 (osd.7) 43 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:41:31.644 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:30.115509+0000 osd.7 (osd.7) 44 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:41:31.644 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:41:30.136237+0000 mgr.y (mgr.14517) 89 : audit [DBG] from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:41:31.645 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:30.365755+0000 mon.a (mon.0) 753 : cluster [DBG] osdmap e93: 8 total, 8 up, 8 in 2024-02-23T00:41:31.645 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:30.115465+0000 osd.7 (osd.7) 43 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:41:31.645 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:30.115509+0000 osd.7 (osd.7) 44 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:41:31.645 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:41:30.136237+0000 mgr.y (mgr.14517) 89 : audit [DBG] from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:41:31.645 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:30.365755+0000 mon.a (mon.0) 753 : cluster [DBG] osdmap e93: 8 total, 8 up, 8 in 2024-02-23T00:41:31.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:30.115465+0000 osd.7 (osd.7) 43 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:41:31.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:30.115509+0000 osd.7 (osd.7) 44 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:41:31.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:41:30.136237+0000 mgr.y (mgr.14517) 89 : audit [DBG] from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:41:31.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:30.365755+0000 mon.a (mon.0) 753 : cluster [DBG] osdmap e93: 8 total, 8 up, 8 in 2024-02-23T00:41:32.527 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:41:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:41:32.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:31.149483+0000 mgr.y (mgr.14517) 90 : cluster [DBG] pgmap v78: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:41:32.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:31.149483+0000 mgr.y (mgr.14517) 90 : cluster [DBG] pgmap v78: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:41:32.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:31.149483+0000 mgr.y (mgr.14517) 90 : cluster [DBG] pgmap v78: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:41:32.943 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:41:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:41:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:41:33.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.126+0000 7fdea6c26700 1 -- 172.21.15.47:0/2159986391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdea00fd510 msgr2=0x7fdea00f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:33.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.126+0000 7fdea6c26700 1 --2- 172.21.15.47:0/2159986391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdea00fd510 0x7fdea00f6d40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fde8c0047b0 tx=0x7fde8c009cb0).stop 2024-02-23T00:41:33.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.126+0000 7fdea6c26700 1 -- 172.21.15.47:0/2159986391 shutdown_connections 2024-02-23T00:41:33.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.126+0000 7fdea6c26700 1 --2- 172.21.15.47:0/2159986391 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdea00f7400 0x7fdea0103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:33.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.126+0000 7fdea6c26700 1 --2- 172.21.15.47:0/2159986391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdea00fcbb0 0x7fdea00fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:33.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.126+0000 7fdea6c26700 1 --2- 172.21.15.47:0/2159986391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdea00fd510 0x7fdea00f6d40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:33.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.126+0000 7fdea6c26700 1 -- 172.21.15.47:0/2159986391 >> 172.21.15.47:0/2159986391 conn(0x7fdea00f2820 msgr2=0x7fdea00f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:41:33.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.126+0000 7fdea6c26700 1 -- 172.21.15.47:0/2159986391 shutdown_connections 2024-02-23T00:41:33.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.126+0000 7fdea6c26700 1 -- 172.21.15.47:0/2159986391 wait complete. 2024-02-23T00:41:33.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.126+0000 7fdea6c26700 1 librados: starting msgr at 2024-02-23T00:41:33.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.126+0000 7fdea6c26700 1 librados: starting objecter 2024-02-23T00:41:33.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.126+0000 7fdea6c26700 1 Processor -- start 2024-02-23T00:41:33.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.126+0000 7fdea6c26700 1 -- start start 2024-02-23T00:41:33.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.126+0000 7fdea6c26700 1 librados: setting wanted keys 2024-02-23T00:41:33.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.126+0000 7fdea6c26700 1 librados: calling monclient init 2024-02-23T00:41:33.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.126+0000 7fdea6c26700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdea00f7400 0x7fdea00f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:33.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.127+0000 7fdea6c26700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdea00fcbb0 0x7fdea00f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:33.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.127+0000 7fdea6c26700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdea00fd510 0x7fdea00f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:33.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.127+0000 7fdea6c26700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea0116450 con 0x7fdea00fd510 2024-02-23T00:41:33.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.127+0000 7fdea6c26700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea01165c0 con 0x7fdea00f7400 2024-02-23T00:41:33.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.127+0000 7fdea6c26700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea0116780 con 0x7fdea00fcbb0 2024-02-23T00:41:33.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.127+0000 7fde9ffff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdea00fcbb0 0x7fdea00f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:33.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.127+0000 7fdea51c3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdea00fd510 0x7fdea00f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:33.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.127+0000 7fde9ffff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdea00fcbb0 0x7fdea00f8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:45570/0 (socket says 172.21.15.47:45570) 2024-02-23T00:41:33.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.127+0000 7fde9ffff700 1 -- 172.21.15.47:0/1377881460 learned_addr learned my addr 172.21.15.47:0/1377881460 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:41:33.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.127+0000 7fdea49c2700 1 --2- 172.21.15.47:0/1377881460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdea00f7400 0x7fdea00f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:33.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.127+0000 7fde9ffff700 1 -- 172.21.15.47:0/1377881460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdea00f7400 msgr2=0x7fdea00f89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:33.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.127+0000 7fde9ffff700 1 --2- 172.21.15.47:0/1377881460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdea00f7400 0x7fdea00f89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:33.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.127+0000 7fde9ffff700 1 -- 172.21.15.47:0/1377881460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdea00fd510 msgr2=0x7fdea00f9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:33.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.127+0000 7fde9ffff700 1 --2- 172.21.15.47:0/1377881460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdea00fd510 0x7fdea00f9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:33.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.127+0000 7fde9ffff700 1 -- 172.21.15.47:0/1377881460 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde980097c0 con 0x7fdea00fcbb0 2024-02-23T00:41:33.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.127+0000 7fdea51c3700 1 --2- 172.21.15.47:0/1377881460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdea00fd510 0x7fdea00f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:41:33.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.127+0000 7fde9ffff700 1 --2- 172.21.15.47:0/1377881460 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdea00fcbb0 0x7fdea00f8ea0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fde8c009f20 tx=0x7fde8c009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:33.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.127+0000 7fde9dffb700 10 client.?.objecter ms_handle_connect 0x7fdea00fcbb0 2024-02-23T00:41:33.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.127+0000 7fde9dffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:41:33.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.127+0000 7fde9dffb700 1 -- 172.21.15.47:0/1377881460 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c018070 con 0x7fdea00fcbb0 2024-02-23T00:41:33.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.127+0000 7fde9dffb700 1 -- 172.21.15.47:0/1377881460 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fde8c013dd0 con 0x7fdea00fcbb0 2024-02-23T00:41:33.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.127+0000 7fdea6c26700 1 -- 172.21.15.47:0/1377881460 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde8c005040 con 0x7fdea00fcbb0 2024-02-23T00:41:33.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.127+0000 7fde9dffb700 1 -- 172.21.15.47:0/1377881460 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c01da10 con 0x7fdea00fcbb0 2024-02-23T00:41:33.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.127+0000 7fdea6c26700 10 client.24608.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:41:33.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.127+0000 7fdea6c26700 1 -- 172.21.15.47:0/1377881460 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea0116c20 con 0x7fdea00fcbb0 2024-02-23T00:41:33.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.127+0000 7fdea6c26700 1 librados: init done 2024-02-23T00:41:33.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.129+0000 7fdea6c26700 1 -- 172.21.15.47:0/1377881460 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdea005fa60 con 0x7fdea00fcbb0 2024-02-23T00:41:33.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.129+0000 7fde9dffb700 1 -- 172.21.15.47:0/1377881460 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fde8c009240 con 0x7fdea00fcbb0 2024-02-23T00:41:33.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.129+0000 7fde9dffb700 1 --2- 172.21.15.47:0/1377881460 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fde90062cc0 0x7fde90065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:33.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.129+0000 7fde9dffb700 1 -- 172.21.15.47:0/1377881460 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(93..93 src has 38..93) v4 ==== 7421+0+0 (secure 0 0 0) 0x7fde8c07e280 con 0x7fdea00fcbb0 2024-02-23T00:41:33.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.129+0000 7fde9dffb700 10 client.24608.objecter ms_dispatch 0x7fdea0001550 osd_map(93..93 src has 38..93) v4 2024-02-23T00:41:33.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.129+0000 7fde9dffb700 3 client.24608.objecter handle_osd_map got epochs [93,93] > 0 2024-02-23T00:41:33.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.129+0000 7fde9dffb700 3 client.24608.objecter handle_osd_map decoding full epoch 93 2024-02-23T00:41:33.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.129+0000 7fdea49c2700 1 --2- 172.21.15.47:0/1377881460 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fde90062cc0 0x7fde90065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:33.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.129+0000 7fde9dffb700 20 client.24608.objecter dump_active .. 0 homeless 2024-02-23T00:41:33.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.130+0000 7fde9dffb700 10 client.24608.objecter ms_handle_connect 0x7fde90062cc0 2024-02-23T00:41:33.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.130+0000 7fdea49c2700 1 --2- 172.21.15.47:0/1377881460 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fde90062cc0 0x7fde90065180 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fde94004d30 tx=0x7fde9400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:33.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.134+0000 7fde9dffb700 1 -- 172.21.15.47:0/1377881460 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fde8c04dfe0 con 0x7fdea00fcbb0 2024-02-23T00:41:33.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.300+0000 7fdea6c26700 1 -- 172.21.15.47:0/1377881460 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdea00f9e90 con 0x7fde90062cc0 2024-02-23T00:41:33.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.311+0000 7fde9dffb700 1 -- 172.21.15.47:0/1377881460 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+357216 (secure 0 0 0) 0x7fdea00f9e90 con 0x7fde90062cc0 2024-02-23T00:41:33.317 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:41:33.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.314+0000 7fdea6c26700 10 librados: watch_flush enter 2024-02-23T00:41:33.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.314+0000 7fdea6c26700 10 librados: watch_flush exit 2024-02-23T00:41:33.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.314+0000 7fdea6c26700 20 client.24608.objecter shutdown clearing up homeless session... 2024-02-23T00:41:33.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.314+0000 7fdea6c26700 10 client.24608.objecter successfully canceled tick 2024-02-23T00:41:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.314+0000 7fdea6c26700 1 -- 172.21.15.47:0/1377881460 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fde90062cc0 msgr2=0x7fde90065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.314+0000 7fdea6c26700 1 --2- 172.21.15.47:0/1377881460 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fde90062cc0 0x7fde90065180 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fde94004d30 tx=0x7fde9400b040).stop 2024-02-23T00:41:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.314+0000 7fdea6c26700 1 -- 172.21.15.47:0/1377881460 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdea00fcbb0 msgr2=0x7fdea00f8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.314+0000 7fdea6c26700 1 --2- 172.21.15.47:0/1377881460 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdea00fcbb0 0x7fdea00f8ea0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fde8c009f20 tx=0x7fde8c009cb0).stop 2024-02-23T00:41:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.314+0000 7fdea6c26700 1 -- 172.21.15.47:0/1377881460 shutdown_connections 2024-02-23T00:41:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.314+0000 7fdea6c26700 1 --2- 172.21.15.47:0/1377881460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdea00fd510 0x7fdea00f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.314+0000 7fdea6c26700 1 --2- 172.21.15.47:0/1377881460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdea00f7400 0x7fdea00f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.314+0000 7fdea6c26700 1 --2- 172.21.15.47:0/1377881460 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fde90062cc0 0x7fde90065180 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.314+0000 7fdea6c26700 1 --2- 172.21.15.47:0/1377881460 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdea00fcbb0 0x7fdea00f8ea0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.314+0000 7fdea6c26700 1 -- 172.21.15.47:0/1377881460 >> 172.21.15.47:0/1377881460 conn(0x7fdea00f2820 msgr2=0x7fdea0060e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:41:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.314+0000 7fdea6c26700 1 -- 172.21.15.47:0/1377881460 shutdown_connections 2024-02-23T00:41:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.314+0000 7fdea6c26700 1 -- 172.21.15.47:0/1377881460 wait complete. 2024-02-23T00:41:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:33.314+0000 7fdea6c26700 1 librados: shutdown 2024-02-23T00:41:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-02-23T00:41:33.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:41:32.205576+0000 mgr.y (mgr.14517) 91 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:41:33.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:32.700959+0000 osd.5 (osd.5) 41 : cluster [DBG] 2.18 scrub starts 2024-02-23T00:41:33.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:32.702567+0000 osd.5 (osd.5) 42 : cluster [DBG] 2.18 scrub ok 2024-02-23T00:41:33.705 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:41:32.205576+0000 mgr.y (mgr.14517) 91 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:41:33.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:32.700959+0000 osd.5 (osd.5) 41 : cluster [DBG] 2.18 scrub starts 2024-02-23T00:41:33.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:32.702567+0000 osd.5 (osd.5) 42 : cluster [DBG] 2.18 scrub ok 2024-02-23T00:41:33.706 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:41:32.205576+0000 mgr.y (mgr.14517) 91 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:41:33.706 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:32.700959+0000 osd.5 (osd.5) 41 : cluster [DBG] 2.18 scrub starts 2024-02-23T00:41:33.706 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:32.702567+0000 osd.5 (osd.5) 42 : cluster [DBG] 2.18 scrub ok 2024-02-23T00:41:34.036 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2024-02-23T00:41:33.149627+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5796,"num_objects":224,"num_object_clones":0,"num_object_copies":672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":0,"num_read":1234,"num_read_kb":954,"num_write":620,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":687,"ondisk_log_size":687,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2382636,"kb_used_data":5868,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":747578580,"statfs":{"total":767960285184,"available":765520465920,"internally_reserved":0,"allocated":6008832,"data_stored":3008852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001717"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.354325+0000","last_change":"2024-02-23T00:39:21.162471+0000","last_active":"2024-02-23T00:41:25.354325+0000","last_peered":"2024-02-23T00:41:25.354325+0000","last_clean":"2024-02-23T00:41:25.354325+0000","last_became_active":"2024-02-23T00:39:21.162083+0000","last_became_peered":"2024-02-23T00:39:21.162083+0000","last_unstale":"2024-02-23T00:41:25.354325+0000","last_undegraded":"2024-02-23T00:41:25.354325+0000","last_fullsized":"2024-02-23T00:41:25.354325+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:39:19.779989+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409082+0000","last_change":"2024-02-23T00:41:05.965631+0000","last_active":"2024-02-23T00:41:25.409082+0000","last_peered":"2024-02-23T00:41:25.409082+0000","last_clean":"2024-02-23T00:41:25.409082+0000","last_became_active":"2024-02-23T00:39:18.799723+0000","last_became_peered":"2024-02-23T00:39:18.799723+0000","last_unstale":"2024-02-23T00:41:25.409082+0000","last_undegraded":"2024-02-23T00:41:25.409082+0000","last_fullsized":"2024-02-23T00:41:25.409082+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:05.965531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:05.965531+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.356578+0000","last_change":"2024-02-23T00:40:58.873093+0000","last_active":"2024-02-23T00:41:25.356578+0000","last_peered":"2024-02-23T00:41:25.356578+0000","last_clean":"2024-02-23T00:41:25.356578+0000","last_became_active":"2024-02-23T00:39:14.782000+0000","last_became_peered":"2024-02-23T00:39:14.782000+0000","last_unstale":"2024-02-23T00:41:25.356578+0000","last_undegraded":"2024-02-23T00:41:25.356578+0000","last_fullsized":"2024-02-23T00:41:25.356578+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_clean_scrub_stamp":"2024-02-23T00:40:58.873013+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"88'50","reported_seq":141,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.354710+0000","last_change":"2024-02-23T00:40:52.840769+0000","last_active":"2024-02-23T00:41:25.354710+0000","last_peered":"2024-02-23T00:41:25.354710+0000","last_clean":"2024-02-23T00:41:25.354710+0000","last_became_active":"2024-02-23T00:39:16.780144+0000","last_became_peered":"2024-02-23T00:39:16.780144+0000","last_unstale":"2024-02-23T00:41:25.354710+0000","last_undegraded":"2024-02-23T00:41:25.354710+0000","last_fullsized":"2024-02-23T00:41:25.354710+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'50","last_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_deep_scrub":"88'50","last_deep_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_clean_scrub_stamp":"2024-02-23T00:40:52.840702+0000","log_size":50,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409962+0000","last_change":"2024-02-23T00:40:27.096850+0000","last_active":"2024-02-23T00:41:25.409962+0000","last_peered":"2024-02-23T00:41:25.409962+0000","last_clean":"2024-02-23T00:41:25.409962+0000","last_became_active":"2024-02-23T00:39:20.795665+0000","last_became_peered":"2024-02-23T00:39:20.795665+0000","last_unstale":"2024-02-23T00:41:25.409962+0000","last_undegraded":"2024-02-23T00:41:25.409962+0000","last_fullsized":"2024-02-23T00:41:25.409962+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:27.096747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:27.096747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409613+0000","last_change":"2024-02-23T00:40:53.812966+0000","last_active":"2024-02-23T00:41:25.409613+0000","last_peered":"2024-02-23T00:41:25.409613+0000","last_clean":"2024-02-23T00:41:25.409613+0000","last_became_active":"2024-02-23T00:39:18.796312+0000","last_became_peered":"2024-02-23T00:39:18.796312+0000","last_unstale":"2024-02-23T00:41:25.409613+0000","last_undegraded":"2024-02-23T00:41:25.409613+0000","last_fullsized":"2024-02-23T00:41:25.409613+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:53.812893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:53.812893+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.355031+0000","last_change":"2024-02-23T00:41:02.853395+0000","last_active":"2024-02-23T00:41:25.355031+0000","last_peered":"2024-02-23T00:41:25.355031+0000","last_clean":"2024-02-23T00:41:25.355031+0000","last_became_active":"2024-02-23T00:39:14.785362+0000","last_became_peered":"2024-02-23T00:39:14.785362+0000","last_unstale":"2024-02-23T00:41:25.355031+0000","last_undegraded":"2024-02-23T00:41:25.355031+0000","last_fullsized":"2024-02-23T00:41:25.355031+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:02.853328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:02.853328+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"68'15","reported_seq":75,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409296+0000","last_change":"2024-02-23T00:40:21.095743+0000","last_active":"2024-02-23T00:41:25.409296+0000","last_peered":"2024-02-23T00:41:25.409296+0000","last_clean":"2024-02-23T00:41:25.409296+0000","last_became_active":"2024-02-23T00:39:16.784606+0000","last_became_peered":"2024-02-23T00:39:16.784606+0000","last_unstale":"2024-02-23T00:41:25.409296+0000","last_undegraded":"2024-02-23T00:41:25.409296+0000","last_fullsized":"2024-02-23T00:41:25.409296+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:40:21.095675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:21.095675+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409027+0000","last_change":"2024-02-23T00:40:55.855032+0000","last_active":"2024-02-23T00:41:25.409027+0000","last_peered":"2024-02-23T00:41:25.409027+0000","last_clean":"2024-02-23T00:41:25.409027+0000","last_became_active":"2024-02-23T00:39:20.793590+0000","last_became_peered":"2024-02-23T00:39:20.793590+0000","last_unstale":"2024-02-23T00:41:25.409027+0000","last_undegraded":"2024-02-23T00:41:25.409027+0000","last_fullsized":"2024-02-23T00:41:25.409027+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:55.854964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:55.854964+0000","last_clean_scrub_stamp":"2024-02-23T00:40:55.854964+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.359678+0000","last_change":"2024-02-23T00:40:42.667373+0000","last_active":"2024-02-23T00:41:25.359678+0000","last_peered":"2024-02-23T00:41:25.359678+0000","last_clean":"2024-02-23T00:41:25.359678+0000","last_became_active":"2024-02-23T00:39:18.796952+0000","last_became_peered":"2024-02-23T00:39:18.796952+0000","last_unstale":"2024-02-23T00:41:25.359678+0000","last_undegraded":"2024-02-23T00:41:25.359678+0000","last_fullsized":"2024-02-23T00:41:25.359678+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:42.667291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:42.667291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.359980+0000","last_change":"2024-02-23T00:40:27.569090+0000","last_active":"2024-02-23T00:41:25.359980+0000","last_peered":"2024-02-23T00:41:25.359980+0000","last_clean":"2024-02-23T00:41:25.359980+0000","last_became_active":"2024-02-23T00:39:14.783469+0000","last_became_peered":"2024-02-23T00:39:14.783469+0000","last_unstale":"2024-02-23T00:41:25.359980+0000","last_undegraded":"2024-02-23T00:41:25.359980+0000","last_fullsized":"2024-02-23T00:41:25.359980+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:27.568970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:27.568970+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"82'15","reported_seq":73,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409667+0000","last_change":"2024-02-23T00:40:31.989101+0000","last_active":"2024-02-23T00:41:25.409667+0000","last_peered":"2024-02-23T00:41:25.409667+0000","last_clean":"2024-02-23T00:41:25.409667+0000","last_became_active":"2024-02-23T00:39:16.782762+0000","last_became_peered":"2024-02-23T00:39:16.782762+0000","last_unstale":"2024-02-23T00:41:25.409667+0000","last_undegraded":"2024-02-23T00:41:25.409667+0000","last_fullsized":"2024-02-23T00:41:25.409667+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'15","last_scrub_stamp":"2024-02-23T00:40:31.989027+0000","last_deep_scrub":"82'15","last_deep_scrub_stamp":"2024-02-23T00:40:31.989027+0000","last_clean_scrub_stamp":"2024-02-23T00:40:31.989027+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.355996+0000","last_change":"2024-02-23T00:41:08.906994+0000","last_active":"2024-02-23T00:41:25.355996+0000","last_peered":"2024-02-23T00:41:25.355996+0000","last_clean":"2024-02-23T00:41:25.355996+0000","last_became_active":"2024-02-23T00:39:20.803344+0000","last_became_peered":"2024-02-23T00:39:20.803344+0000","last_unstale":"2024-02-23T00:41:25.355996+0000","last_undegraded":"2024-02-23T00:41:25.355996+0000","last_fullsized":"2024-02-23T00:41:25.355996+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:08.906928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:08.906928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409953+0000","last_change":"2024-02-23T00:41:07.777516+0000","last_active":"2024-02-23T00:41:25.409953+0000","last_peered":"2024-02-23T00:41:25.409953+0000","last_clean":"2024-02-23T00:41:25.409953+0000","last_became_active":"2024-02-23T00:39:18.796431+0000","last_became_peered":"2024-02-23T00:39:18.796431+0000","last_unstale":"2024-02-23T00:41:25.409953+0000","last_undegraded":"2024-02-23T00:41:25.409953+0000","last_fullsized":"2024-02-23T00:41:25.409953+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:07.777435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:07.777435+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.359824+0000","last_change":"2024-02-23T00:40:35.620371+0000","last_active":"2024-02-23T00:41:25.359824+0000","last_peered":"2024-02-23T00:41:25.359824+0000","last_clean":"2024-02-23T00:41:25.359824+0000","last_became_active":"2024-02-23T00:39:14.787148+0000","last_became_peered":"2024-02-23T00:39:14.787148+0000","last_unstale":"2024-02-23T00:41:25.359824+0000","last_undegraded":"2024-02-23T00:41:25.359824+0000","last_fullsized":"2024-02-23T00:41:25.359824+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:35.620292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:35.620292+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"68'21","reported_seq":90,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.410054+0000","last_change":"2024-02-23T00:40:51.950549+0000","last_active":"2024-02-23T00:41:25.410054+0000","last_peered":"2024-02-23T00:41:25.410054+0000","last_clean":"2024-02-23T00:41:25.410054+0000","last_became_active":"2024-02-23T00:39:16.786791+0000","last_became_peered":"2024-02-23T00:39:16.786791+0000","last_unstale":"2024-02-23T00:41:25.410054+0000","last_undegraded":"2024-02-23T00:41:25.410054+0000","last_fullsized":"2024-02-23T00:41:25.410054+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'21","last_scrub_stamp":"2024-02-23T00:40:51.950467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:51.950467+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:29.992562+0000","last_change":"2024-02-23T00:41:29.992520+0000","last_active":"2024-02-23T00:41:29.992562+0000","last_peered":"2024-02-23T00:41:29.992562+0000","last_clean":"2024-02-23T00:41:29.992562+0000","last_became_active":"2024-02-23T00:39:21.162053+0000","last_became_peered":"2024-02-23T00:39:21.162053+0000","last_unstale":"2024-02-23T00:41:29.992562+0000","last_undegraded":"2024-02-23T00:41:29.992562+0000","last_fullsized":"2024-02-23T00:41:29.992562+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:29.992417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:29.992417+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409977+0000","last_change":"2024-02-23T00:41:01.063755+0000","last_active":"2024-02-23T00:41:25.409977+0000","last_peered":"2024-02-23T00:41:25.409977+0000","last_clean":"2024-02-23T00:41:25.409977+0000","last_became_active":"2024-02-23T00:39:18.794903+0000","last_became_peered":"2024-02-23T00:39:18.794903+0000","last_unstale":"2024-02-23T00:41:25.409977+0000","last_undegraded":"2024-02-23T00:41:25.409977+0000","last_fullsized":"2024-02-23T00:41:25.409977+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_clean_scrub_stamp":"2024-02-23T00:41:01.063660+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.354985+0000","last_change":"2024-02-23T00:40:19.123225+0000","last_active":"2024-02-23T00:41:25.354985+0000","last_peered":"2024-02-23T00:41:25.354985+0000","last_clean":"2024-02-23T00:41:25.354985+0000","last_became_active":"2024-02-23T00:39:14.782163+0000","last_became_peered":"2024-02-23T00:39:14.782163+0000","last_unstale":"2024-02-23T00:41:25.354985+0000","last_undegraded":"2024-02-23T00:41:25.354985+0000","last_fullsized":"2024-02-23T00:41:25.354985+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:19.123123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:19.123123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"68'18","reported_seq":83,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.354822+0000","last_change":"2024-02-23T00:40:28.118091+0000","last_active":"2024-02-23T00:41:25.354822+0000","last_peered":"2024-02-23T00:41:25.354822+0000","last_clean":"2024-02-23T00:41:25.354822+0000","last_became_active":"2024-02-23T00:39:16.782803+0000","last_became_peered":"2024-02-23T00:39:16.782803+0000","last_unstale":"2024-02-23T00:41:25.354822+0000","last_undegraded":"2024-02-23T00:41:25.354822+0000","last_fullsized":"2024-02-23T00:41:25.354822+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'18","last_scrub_stamp":"2024-02-23T00:40:28.118021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:28.118021+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.411222+0000","last_change":"2024-02-23T00:41:16.048306+0000","last_active":"2024-02-23T00:41:25.411222+0000","last_peered":"2024-02-23T00:41:25.411222+0000","last_clean":"2024-02-23T00:41:25.411222+0000","last_became_active":"2024-02-23T00:39:21.159750+0000","last_became_peered":"2024-02-23T00:39:21.159750+0000","last_unstale":"2024-02-23T00:41:25.411222+0000","last_undegraded":"2024-02-23T00:41:25.411222+0000","last_fullsized":"2024-02-23T00:41:25.411222+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:16.048207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:16.048207+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.410465+0000","last_change":"2024-02-23T00:40:23.829090+0000","last_active":"2024-02-23T00:41:25.410465+0000","last_peered":"2024-02-23T00:41:25.410465+0000","last_clean":"2024-02-23T00:41:25.410465+0000","last_became_active":"2024-02-23T00:39:18.801039+0000","last_became_peered":"2024-02-23T00:39:18.801039+0000","last_unstale":"2024-02-23T00:41:25.410465+0000","last_undegraded":"2024-02-23T00:41:25.410465+0000","last_fullsized":"2024-02-23T00:41:25.410465+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:23.829017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:23.829017+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.411304+0000","last_change":"2024-02-23T00:40:30.958705+0000","last_active":"2024-02-23T00:41:25.411304+0000","last_peered":"2024-02-23T00:41:25.411304+0000","last_clean":"2024-02-23T00:41:25.411304+0000","last_became_active":"2024-02-23T00:39:14.782494+0000","last_became_peered":"2024-02-23T00:39:14.782494+0000","last_unstale":"2024-02-23T00:41:25.411304+0000","last_undegraded":"2024-02-23T00:41:25.411304+0000","last_fullsized":"2024-02-23T00:41:25.411304+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:30.958634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:30.958634+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"67'15","reported_seq":69,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409617+0000","last_change":"2024-02-23T00:40:21.307315+0000","last_active":"2024-02-23T00:41:25.409617+0000","last_peered":"2024-02-23T00:41:25.409617+0000","last_clean":"2024-02-23T00:41:25.409617+0000","last_became_active":"2024-02-23T00:39:16.787138+0000","last_became_peered":"2024-02-23T00:39:16.787138+0000","last_unstale":"2024-02-23T00:41:25.409617+0000","last_undegraded":"2024-02-23T00:41:25.409617+0000","last_fullsized":"2024-02-23T00:41:25.409617+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:40:21.307241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:21.307241+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.410183+0000","last_change":"2024-02-23T00:40:49.777703+0000","last_active":"2024-02-23T00:41:25.410183+0000","last_peered":"2024-02-23T00:41:25.410183+0000","last_clean":"2024-02-23T00:41:25.410183+0000","last_became_active":"2024-02-23T00:39:20.795524+0000","last_became_peered":"2024-02-23T00:39:20.795524+0000","last_unstale":"2024-02-23T00:41:25.410183+0000","last_undegraded":"2024-02-23T00:41:25.410183+0000","last_fullsized":"2024-02-23T00:41:25.410183+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:49.777636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:49.777636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.356807+0000","last_change":"2024-02-23T00:40:21.510727+0000","last_active":"2024-02-23T00:41:25.356807+0000","last_peered":"2024-02-23T00:41:25.356807+0000","last_clean":"2024-02-23T00:41:25.356807+0000","last_became_active":"2024-02-23T00:39:18.790602+0000","last_became_peered":"2024-02-23T00:39:18.790602+0000","last_unstale":"2024-02-23T00:41:25.356807+0000","last_undegraded":"2024-02-23T00:41:25.356807+0000","last_fullsized":"2024-02-23T00:41:25.356807+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:21.510658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:21.510658+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.354245+0000","last_change":"2024-02-23T00:40:44.906676+0000","last_active":"2024-02-23T00:41:25.354245+0000","last_peered":"2024-02-23T00:41:25.354245+0000","last_clean":"2024-02-23T00:41:25.354245+0000","last_became_active":"2024-02-23T00:39:14.781560+0000","last_became_peered":"2024-02-23T00:39:14.781560+0000","last_unstale":"2024-02-23T00:41:25.354245+0000","last_undegraded":"2024-02-23T00:41:25.354245+0000","last_fullsized":"2024-02-23T00:41:25.354245+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:44.906599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:44.906599+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"68'17","reported_seq":78,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.410536+0000","last_change":"2024-02-23T00:40:20.909829+0000","last_active":"2024-02-23T00:41:25.410536+0000","last_peered":"2024-02-23T00:41:25.410536+0000","last_clean":"2024-02-23T00:41:25.410536+0000","last_became_active":"2024-02-23T00:39:16.782660+0000","last_became_peered":"2024-02-23T00:39:16.782660+0000","last_unstale":"2024-02-23T00:41:25.410536+0000","last_undegraded":"2024-02-23T00:41:25.410536+0000","last_fullsized":"2024-02-23T00:41:25.410536+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:40:20.909759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:20.909759+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"67'1","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.360130+0000","last_change":"2024-02-23T00:40:29.596674+0000","last_active":"2024-02-23T00:41:25.360130+0000","last_peered":"2024-02-23T00:41:25.360130+0000","last_clean":"2024-02-23T00:41:25.360130+0000","last_became_active":"2024-02-23T00:39:20.795085+0000","last_became_peered":"2024-02-23T00:39:20.795085+0000","last_unstale":"2024-02-23T00:41:25.360130+0000","last_undegraded":"2024-02-23T00:41:25.360130+0000","last_fullsized":"2024-02-23T00:41:25.360130+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:40:29.596581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:29.596581+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"67'8","reported_seq":80,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:27.867078+0000","last_change":"2024-02-23T00:40:27.923078+0000","last_active":"2024-02-23T00:41:27.867078+0000","last_peered":"2024-02-23T00:41:27.867078+0000","last_clean":"2024-02-23T00:41:27.867078+0000","last_became_active":"2024-02-23T00:39:18.796680+0000","last_became_peered":"2024-02-23T00:39:18.796680+0000","last_unstale":"2024-02-23T00:41:27.867078+0000","last_undegraded":"2024-02-23T00:41:27.867078+0000","last_fullsized":"2024-02-23T00:41:27.867078+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:40:27.923003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:27.923003+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.408939+0000","last_change":"2024-02-23T00:40:15.942790+0000","last_active":"2024-02-23T00:41:25.408939+0000","last_peered":"2024-02-23T00:41:25.408939+0000","last_clean":"2024-02-23T00:41:25.408939+0000","last_became_active":"2024-02-23T00:39:14.785363+0000","last_became_peered":"2024-02-23T00:39:14.785363+0000","last_unstale":"2024-02-23T00:41:25.408939+0000","last_undegraded":"2024-02-23T00:41:25.408939+0000","last_fullsized":"2024-02-23T00:41:25.408939+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"80'2","last_scrub_stamp":"2024-02-23T00:40:15.942725+0000","last_deep_scrub":"80'2","last_deep_scrub_stamp":"2024-02-23T00:40:15.942725+0000","last_clean_scrub_stamp":"2024-02-23T00:40:15.942725+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"68'16","reported_seq":70,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.356763+0000","last_change":"2024-02-23T00:39:16.859294+0000","last_active":"2024-02-23T00:41:25.356763+0000","last_peered":"2024-02-23T00:41:25.356763+0000","last_clean":"2024-02-23T00:41:25.356763+0000","last_became_active":"2024-02-23T00:39:16.858945+0000","last_became_peered":"2024-02-23T00:39:16.858945+0000","last_unstale":"2024-02-23T00:41:25.356763+0000","last_undegraded":"2024-02-23T00:41:25.356763+0000","last_fullsized":"2024-02-23T00:41:25.356763+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:39:15.764507+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.354708+0000","last_change":"2024-02-23T00:39:21.160824+0000","last_active":"2024-02-23T00:41:25.354708+0000","last_peered":"2024-02-23T00:41:25.354708+0000","last_clean":"2024-02-23T00:41:25.354708+0000","last_became_active":"2024-02-23T00:39:21.160376+0000","last_became_peered":"2024-02-23T00:39:21.160376+0000","last_unstale":"2024-02-23T00:41:25.354708+0000","last_undegraded":"2024-02-23T00:41:25.354708+0000","last_fullsized":"2024-02-23T00:41:25.354708+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:39:19.779989+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.359650+0000","last_change":"2024-02-23T00:40:25.581008+0000","last_active":"2024-02-23T00:41:25.359650+0000","last_peered":"2024-02-23T00:41:25.359650+0000","last_clean":"2024-02-23T00:41:25.359650+0000","last_became_active":"2024-02-23T00:39:18.791457+0000","last_became_peered":"2024-02-23T00:39:18.791457+0000","last_unstale":"2024-02-23T00:41:25.359650+0000","last_undegraded":"2024-02-23T00:41:25.359650+0000","last_fullsized":"2024-02-23T00:41:25.359650+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:25.580938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:25.580938+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.410842+0000","last_change":"2024-02-23T00:40:29.977372+0000","last_active":"2024-02-23T00:41:25.410842+0000","last_peered":"2024-02-23T00:41:25.410842+0000","last_clean":"2024-02-23T00:41:25.410842+0000","last_became_active":"2024-02-23T00:39:14.784092+0000","last_became_peered":"2024-02-23T00:39:14.784092+0000","last_unstale":"2024-02-23T00:41:25.410842+0000","last_undegraded":"2024-02-23T00:41:25.410842+0000","last_fullsized":"2024-02-23T00:41:25.410842+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:29.977275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:29.977275+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"68'17","reported_seq":79,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.354706+0000","last_change":"2024-02-23T00:40:20.143358+0000","last_active":"2024-02-23T00:41:25.354706+0000","last_peered":"2024-02-23T00:41:25.354706+0000","last_clean":"2024-02-23T00:41:25.354706+0000","last_became_active":"2024-02-23T00:39:16.857232+0000","last_became_peered":"2024-02-23T00:39:16.857232+0000","last_unstale":"2024-02-23T00:41:25.354706+0000","last_undegraded":"2024-02-23T00:41:25.354706+0000","last_fullsized":"2024-02-23T00:41:25.354706+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:40:20.143288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:20.143288+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.359639+0000","last_change":"2024-02-23T00:40:54.739607+0000","last_active":"2024-02-23T00:41:25.359639+0000","last_peered":"2024-02-23T00:41:25.359639+0000","last_clean":"2024-02-23T00:41:25.359639+0000","last_became_active":"2024-02-23T00:39:20.795351+0000","last_became_peered":"2024-02-23T00:39:20.795351+0000","last_unstale":"2024-02-23T00:41:25.359639+0000","last_undegraded":"2024-02-23T00:41:25.359639+0000","last_fullsized":"2024-02-23T00:41:25.359639+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:54.739534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:54.739534+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.410695+0000","last_change":"2024-02-23T00:40:38.957682+0000","last_active":"2024-02-23T00:41:25.410695+0000","last_peered":"2024-02-23T00:41:25.410695+0000","last_clean":"2024-02-23T00:41:25.410695+0000","last_became_active":"2024-02-23T00:39:18.797287+0000","last_became_peered":"2024-02-23T00:39:18.797287+0000","last_unstale":"2024-02-23T00:41:25.410695+0000","last_undegraded":"2024-02-23T00:41:25.410695+0000","last_fullsized":"2024-02-23T00:41:25.410695+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:38.957578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:38.957578+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.410996+0000","last_change":"2024-02-23T00:40:22.050580+0000","last_active":"2024-02-23T00:41:25.410996+0000","last_peered":"2024-02-23T00:41:25.410996+0000","last_clean":"2024-02-23T00:41:25.410996+0000","last_became_active":"2024-02-23T00:39:14.783325+0000","last_became_peered":"2024-02-23T00:39:14.783325+0000","last_unstale":"2024-02-23T00:41:25.410996+0000","last_undegraded":"2024-02-23T00:41:25.410996+0000","last_fullsized":"2024-02-23T00:41:25.410996+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:22.050497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:22.050497+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"67'4","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.354270+0000","last_change":"2024-02-23T00:41:20.947463+0000","last_active":"2024-02-23T00:41:25.354270+0000","last_peered":"2024-02-23T00:41:25.354270+0000","last_clean":"2024-02-23T00:41:25.354270+0000","last_became_active":"2024-02-23T00:39:16.785830+0000","last_became_peered":"2024-02-23T00:39:16.785830+0000","last_unstale":"2024-02-23T00:41:25.354270+0000","last_undegraded":"2024-02-23T00:41:25.354270+0000","last_fullsized":"2024-02-23T00:41:25.354270+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'4","last_scrub_stamp":"2024-02-23T00:41:20.947377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:20.947377+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.354136+0000","last_change":"2024-02-23T00:40:56.892857+0000","last_active":"2024-02-23T00:41:25.354136+0000","last_peered":"2024-02-23T00:41:25.354136+0000","last_clean":"2024-02-23T00:41:25.354136+0000","last_became_active":"2024-02-23T00:39:20.799053+0000","last_became_peered":"2024-02-23T00:39:20.799053+0000","last_unstale":"2024-02-23T00:41:25.354136+0000","last_undegraded":"2024-02-23T00:41:25.354136+0000","last_fullsized":"2024-02-23T00:41:25.354136+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:56.892797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:56.892797+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409917+0000","last_change":"2024-02-23T00:40:59.669880+0000","last_active":"2024-02-23T00:41:25.409917+0000","last_peered":"2024-02-23T00:41:25.409917+0000","last_clean":"2024-02-23T00:41:25.409917+0000","last_became_active":"2024-02-23T00:39:18.800382+0000","last_became_peered":"2024-02-23T00:39:18.800382+0000","last_unstale":"2024-02-23T00:41:25.409917+0000","last_undegraded":"2024-02-23T00:41:25.409917+0000","last_fullsized":"2024-02-23T00:41:25.409917+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:59.669810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:59.669810+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"60'1","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.410363+0000","last_change":"2024-02-23T00:40:34.909212+0000","last_active":"2024-02-23T00:41:25.410363+0000","last_peered":"2024-02-23T00:41:25.410363+0000","last_clean":"2024-02-23T00:41:25.410363+0000","last_became_active":"2024-02-23T00:39:14.775804+0000","last_became_peered":"2024-02-23T00:39:14.775804+0000","last_unstale":"2024-02-23T00:41:25.410363+0000","last_undegraded":"2024-02-23T00:41:25.410363+0000","last_fullsized":"2024-02-23T00:41:25.410363+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'1","last_scrub_stamp":"2024-02-23T00:40:34.909135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:34.909135+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"67'11","reported_seq":63,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409837+0000","last_change":"2024-02-23T00:40:55.928416+0000","last_active":"2024-02-23T00:41:25.409837+0000","last_peered":"2024-02-23T00:41:25.409837+0000","last_clean":"2024-02-23T00:41:25.409837+0000","last_became_active":"2024-02-23T00:39:16.783896+0000","last_became_peered":"2024-02-23T00:39:16.783896+0000","last_unstale":"2024-02-23T00:41:25.409837+0000","last_undegraded":"2024-02-23T00:41:25.409837+0000","last_fullsized":"2024-02-23T00:41:25.409837+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'11","last_scrub_stamp":"2024-02-23T00:40:55.928349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:55.928349+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.355543+0000","last_change":"2024-02-23T00:41:10.892383+0000","last_active":"2024-02-23T00:41:25.355543+0000","last_peered":"2024-02-23T00:41:25.355543+0000","last_clean":"2024-02-23T00:41:25.355543+0000","last_became_active":"2024-02-23T00:39:20.802648+0000","last_became_peered":"2024-02-23T00:39:20.802648+0000","last_unstale":"2024-02-23T00:41:25.355543+0000","last_undegraded":"2024-02-23T00:41:25.355543+0000","last_fullsized":"2024-02-23T00:41:25.355543+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:10.892259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:10.892259+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.354534+0000","last_change":"2024-02-23T00:41:17.931479+0000","last_active":"2024-02-23T00:41:25.354534+0000","last_peered":"2024-02-23T00:41:25.354534+0000","last_clean":"2024-02-23T00:41:25.354534+0000","last_became_active":"2024-02-23T00:39:18.799052+0000","last_became_peered":"2024-02-23T00:39:18.799052+0000","last_unstale":"2024-02-23T00:41:25.354534+0000","last_undegraded":"2024-02-23T00:41:25.354534+0000","last_fullsized":"2024-02-23T00:41:25.354534+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_clean_scrub_stamp":"2024-02-23T00:41:17.931413+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.410237+0000","last_change":"2024-02-23T00:40:16.893963+0000","last_active":"2024-02-23T00:41:25.410237+0000","last_peered":"2024-02-23T00:41:25.410237+0000","last_clean":"2024-02-23T00:41:25.410237+0000","last_became_active":"2024-02-23T00:39:14.778455+0000","last_became_peered":"2024-02-23T00:39:14.778455+0000","last_unstale":"2024-02-23T00:41:25.410237+0000","last_undegraded":"2024-02-23T00:41:25.410237+0000","last_fullsized":"2024-02-23T00:41:25.410237+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:16.893878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:16.893878+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"67'9","reported_seq":60,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409819+0000","last_change":"2024-02-23T00:40:52.783268+0000","last_active":"2024-02-23T00:41:25.409819+0000","last_peered":"2024-02-23T00:41:25.409819+0000","last_clean":"2024-02-23T00:41:25.409819+0000","last_became_active":"2024-02-23T00:39:16.786936+0000","last_became_peered":"2024-02-23T00:39:16.786936+0000","last_unstale":"2024-02-23T00:41:25.409819+0000","last_undegraded":"2024-02-23T00:41:25.409819+0000","last_fullsized":"2024-02-23T00:41:25.409819+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'9","last_scrub_stamp":"2024-02-23T00:40:52.783146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:52.783146+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.410793+0000","last_change":"2024-02-23T00:40:54.938288+0000","last_active":"2024-02-23T00:41:25.410793+0000","last_peered":"2024-02-23T00:41:25.410793+0000","last_clean":"2024-02-23T00:41:25.410793+0000","last_became_active":"2024-02-23T00:39:20.795210+0000","last_became_peered":"2024-02-23T00:39:20.795210+0000","last_unstale":"2024-02-23T00:41:25.410793+0000","last_undegraded":"2024-02-23T00:41:25.410793+0000","last_fullsized":"2024-02-23T00:41:25.410793+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:54.938209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:54.938209+0000","last_clean_scrub_stamp":"2024-02-23T00:40:54.938209+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"67'8","reported_seq":78,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:27.866632+0000","last_change":"2024-02-23T00:41:25.942914+0000","last_active":"2024-02-23T00:41:27.866632+0000","last_peered":"2024-02-23T00:41:27.866632+0000","last_clean":"2024-02-23T00:41:27.866632+0000","last_became_active":"2024-02-23T00:39:18.787313+0000","last_became_peered":"2024-02-23T00:39:18.787313+0000","last_unstale":"2024-02-23T00:41:27.866632+0000","last_undegraded":"2024-02-23T00:41:27.866632+0000","last_fullsized":"2024-02-23T00:41:27.866632+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:41:25.942822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:25.942822+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.356429+0000","last_change":"2024-02-23T00:40:16.547440+0000","last_active":"2024-02-23T00:41:25.356429+0000","last_peered":"2024-02-23T00:41:25.356429+0000","last_clean":"2024-02-23T00:41:25.356429+0000","last_became_active":"2024-02-23T00:39:14.773772+0000","last_became_peered":"2024-02-23T00:39:14.773772+0000","last_unstale":"2024-02-23T00:41:25.356429+0000","last_undegraded":"2024-02-23T00:41:25.356429+0000","last_fullsized":"2024-02-23T00:41:25.356429+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:16.547328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:16.547328+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"68'15","reported_seq":76,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409847+0000","last_change":"2024-02-23T00:40:42.867136+0000","last_active":"2024-02-23T00:41:25.409847+0000","last_peered":"2024-02-23T00:41:25.409847+0000","last_clean":"2024-02-23T00:41:25.409847+0000","last_became_active":"2024-02-23T00:39:16.858953+0000","last_became_peered":"2024-02-23T00:39:16.858953+0000","last_unstale":"2024-02-23T00:41:25.409847+0000","last_undegraded":"2024-02-23T00:41:25.409847+0000","last_fullsized":"2024-02-23T00:41:25.409847+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:40:42.867043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:42.867043+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.356477+0000","last_change":"2024-02-23T00:41:22.855592+0000","last_active":"2024-02-23T00:41:25.356477+0000","last_peered":"2024-02-23T00:41:25.356477+0000","last_clean":"2024-02-23T00:41:25.356477+0000","last_became_active":"2024-02-23T00:39:20.800422+0000","last_became_peered":"2024-02-23T00:39:20.800422+0000","last_unstale":"2024-02-23T00:41:25.356477+0000","last_undegraded":"2024-02-23T00:41:25.356477+0000","last_fullsized":"2024-02-23T00:41:25.356477+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:22.855516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:22.855516+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"67'8","reported_seq":78,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:27.867071+0000","last_change":"2024-02-23T00:41:08.785056+0000","last_active":"2024-02-23T00:41:27.867071+0000","last_peered":"2024-02-23T00:41:27.867071+0000","last_clean":"2024-02-23T00:41:27.867071+0000","last_became_active":"2024-02-23T00:39:18.803566+0000","last_became_peered":"2024-02-23T00:39:18.803566+0000","last_unstale":"2024-02-23T00:41:27.867071+0000","last_undegraded":"2024-02-23T00:41:27.867071+0000","last_fullsized":"2024-02-23T00:41:27.867071+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:41:08.784965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:08.784965+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409274+0000","last_change":"2024-02-23T00:40:29.990344+0000","last_active":"2024-02-23T00:41:25.409274+0000","last_peered":"2024-02-23T00:41:25.409274+0000","last_clean":"2024-02-23T00:41:25.409274+0000","last_became_active":"2024-02-23T00:39:14.785590+0000","last_became_peered":"2024-02-23T00:39:14.785590+0000","last_unstale":"2024-02-23T00:41:25.409274+0000","last_undegraded":"2024-02-23T00:41:25.409274+0000","last_fullsized":"2024-02-23T00:41:25.409274+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_clean_scrub_stamp":"2024-02-23T00:40:29.990255+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"68'22","reported_seq":85,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.354905+0000","last_change":"2024-02-23T00:39:16.857807+0000","last_active":"2024-02-23T00:41:25.354905+0000","last_peered":"2024-02-23T00:41:25.354905+0000","last_clean":"2024-02-23T00:41:25.354905+0000","last_became_active":"2024-02-23T00:39:16.857437+0000","last_became_peered":"2024-02-23T00:39:16.857437+0000","last_unstale":"2024-02-23T00:41:25.354905+0000","last_undegraded":"2024-02-23T00:41:25.354905+0000","last_fullsized":"2024-02-23T00:41:25.354905+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:39:15.764507+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":39,"num_read_kb":32,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.359878+0000","last_change":"2024-02-23T00:40:26.576403+0000","last_active":"2024-02-23T00:41:25.359878+0000","last_peered":"2024-02-23T00:41:25.359878+0000","last_clean":"2024-02-23T00:41:25.359878+0000","last_became_active":"2024-02-23T00:39:21.162036+0000","last_became_peered":"2024-02-23T00:39:21.162036+0000","last_unstale":"2024-02-23T00:41:25.359878+0000","last_undegraded":"2024-02-23T00:41:25.359878+0000","last_fullsized":"2024-02-23T00:41:25.359878+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:26.576296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:26.576296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409302+0000","last_change":"2024-02-23T00:40:25.025989+0000","last_active":"2024-02-23T00:41:25.409302+0000","last_peered":"2024-02-23T00:41:25.409302+0000","last_clean":"2024-02-23T00:41:25.409302+0000","last_became_active":"2024-02-23T00:39:18.800297+0000","last_became_peered":"2024-02-23T00:39:18.800297+0000","last_unstale":"2024-02-23T00:41:25.409302+0000","last_undegraded":"2024-02-23T00:41:25.409302+0000","last_fullsized":"2024-02-23T00:41:25.409302+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:25.025890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:25.025890+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.410783+0000","last_change":"2024-02-23T00:40:53.113312+0000","last_active":"2024-02-23T00:41:25.410783+0000","last_peered":"2024-02-23T00:41:25.410783+0000","last_clean":"2024-02-23T00:41:25.410783+0000","last_became_active":"2024-02-23T00:39:14.783164+0000","last_became_peered":"2024-02-23T00:39:14.783164+0000","last_unstale":"2024-02-23T00:41:25.410783+0000","last_undegraded":"2024-02-23T00:41:25.410783+0000","last_fullsized":"2024-02-23T00:41:25.410783+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:53.113232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:53.113232+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"68'12","reported_seq":69,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.355380+0000","last_change":"2024-02-23T00:41:23.904595+0000","last_active":"2024-02-23T00:41:25.355380+0000","last_peered":"2024-02-23T00:41:25.355380+0000","last_clean":"2024-02-23T00:41:25.355380+0000","last_became_active":"2024-02-23T00:39:16.858449+0000","last_became_peered":"2024-02-23T00:39:16.858449+0000","last_unstale":"2024-02-23T00:41:25.355380+0000","last_undegraded":"2024-02-23T00:41:25.355380+0000","last_fullsized":"2024-02-23T00:41:25.355380+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'12","last_scrub_stamp":"2024-02-23T00:41:23.904511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:23.904511+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409416+0000","last_change":"2024-02-23T00:40:40.360889+0000","last_active":"2024-02-23T00:41:25.409416+0000","last_peered":"2024-02-23T00:41:25.409416+0000","last_clean":"2024-02-23T00:41:25.409416+0000","last_became_active":"2024-02-23T00:39:20.796041+0000","last_became_peered":"2024-02-23T00:39:20.796041+0000","last_unstale":"2024-02-23T00:41:25.409416+0000","last_undegraded":"2024-02-23T00:41:25.409416+0000","last_fullsized":"2024-02-23T00:41:25.409416+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:40.360797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:40.360797+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.355534+0000","last_change":"2024-02-23T00:40:26.171967+0000","last_active":"2024-02-23T00:41:25.355534+0000","last_peered":"2024-02-23T00:41:25.355534+0000","last_clean":"2024-02-23T00:41:25.355534+0000","last_became_active":"2024-02-23T00:39:18.798624+0000","last_became_peered":"2024-02-23T00:39:18.798624+0000","last_unstale":"2024-02-23T00:41:25.355534+0000","last_undegraded":"2024-02-23T00:41:25.355534+0000","last_fullsized":"2024-02-23T00:41:25.355534+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:26.171869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:26.171869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409095+0000","last_change":"2024-02-23T00:40:24.370317+0000","last_active":"2024-02-23T00:41:25.409095+0000","last_peered":"2024-02-23T00:41:25.409095+0000","last_clean":"2024-02-23T00:41:25.409095+0000","last_became_active":"2024-02-23T00:39:14.774244+0000","last_became_peered":"2024-02-23T00:39:14.774244+0000","last_unstale":"2024-02-23T00:41:25.409095+0000","last_undegraded":"2024-02-23T00:41:25.409095+0000","last_fullsized":"2024-02-23T00:41:25.409095+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:24.370253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:24.370253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"68'21","reported_seq":86,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.356656+0000","last_change":"2024-02-23T00:39:16.859093+0000","last_active":"2024-02-23T00:41:25.356656+0000","last_peered":"2024-02-23T00:41:25.356656+0000","last_clean":"2024-02-23T00:41:25.356656+0000","last_became_active":"2024-02-23T00:39:16.858760+0000","last_became_peered":"2024-02-23T00:39:16.858760+0000","last_unstale":"2024-02-23T00:41:25.356656+0000","last_undegraded":"2024-02-23T00:41:25.356656+0000","last_fullsized":"2024-02-23T00:41:25.356656+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:39:15.764507+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.354620+0000","last_change":"2024-02-23T00:40:24.129150+0000","last_active":"2024-02-23T00:41:25.354620+0000","last_peered":"2024-02-23T00:41:25.354620+0000","last_clean":"2024-02-23T00:41:25.354620+0000","last_became_active":"2024-02-23T00:39:20.793710+0000","last_became_peered":"2024-02-23T00:39:20.793710+0000","last_unstale":"2024-02-23T00:41:25.354620+0000","last_undegraded":"2024-02-23T00:41:25.354620+0000","last_fullsized":"2024-02-23T00:41:25.354620+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:24.129083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:24.129083+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"60'2","reported_seq":59,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.410613+0000","last_change":"2024-02-23T00:40:38.029108+0000","last_active":"2024-02-23T00:41:25.410613+0000","last_peered":"2024-02-23T00:41:25.410613+0000","last_clean":"2024-02-23T00:41:25.410613+0000","last_became_active":"2024-02-23T00:39:14.779205+0000","last_became_peered":"2024-02-23T00:39:14.779205+0000","last_unstale":"2024-02-23T00:41:25.410613+0000","last_undegraded":"2024-02-23T00:41:25.410613+0000","last_fullsized":"2024-02-23T00:41:25.410613+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-02-23T00:40:38.029000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:38.029000+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"68'25","reported_seq":98,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409691+0000","last_change":"2024-02-23T00:41:08.004573+0000","last_active":"2024-02-23T00:41:25.409691+0000","last_peered":"2024-02-23T00:41:25.409691+0000","last_clean":"2024-02-23T00:41:25.409691+0000","last_became_active":"2024-02-23T00:39:16.783926+0000","last_became_peered":"2024-02-23T00:39:16.783926+0000","last_unstale":"2024-02-23T00:41:25.409691+0000","last_undegraded":"2024-02-23T00:41:25.409691+0000","last_fullsized":"2024-02-23T00:41:25.409691+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'25","last_scrub_stamp":"2024-02-23T00:41:08.004489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:08.004489+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409010+0000","last_change":"2024-02-23T00:40:49.365809+0000","last_active":"2024-02-23T00:41:25.409010+0000","last_peered":"2024-02-23T00:41:25.409010+0000","last_clean":"2024-02-23T00:41:25.409010+0000","last_became_active":"2024-02-23T00:39:18.796323+0000","last_became_peered":"2024-02-23T00:39:18.796323+0000","last_unstale":"2024-02-23T00:41:25.409010+0000","last_undegraded":"2024-02-23T00:41:25.409010+0000","last_fullsized":"2024-02-23T00:41:25.409010+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_clean_scrub_stamp":"2024-02-23T00:40:49.365711+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"67'1","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.355635+0000","last_change":"2024-02-23T00:41:00.809258+0000","last_active":"2024-02-23T00:41:25.355635+0000","last_peered":"2024-02-23T00:41:25.355635+0000","last_clean":"2024-02-23T00:41:25.355635+0000","last_became_active":"2024-02-23T00:39:20.800858+0000","last_became_peered":"2024-02-23T00:39:20.800858+0000","last_unstale":"2024-02-23T00:41:25.355635+0000","last_undegraded":"2024-02-23T00:41:25.355635+0000","last_fullsized":"2024-02-23T00:41:25.355635+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:41:00.809190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:00.809190+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.410189+0000","last_change":"2024-02-23T00:40:55.723385+0000","last_active":"2024-02-23T00:41:25.410189+0000","last_peered":"2024-02-23T00:41:25.410189+0000","last_clean":"2024-02-23T00:41:25.410189+0000","last_became_active":"2024-02-23T00:39:14.779952+0000","last_became_peered":"2024-02-23T00:39:14.779952+0000","last_unstale":"2024-02-23T00:41:25.410189+0000","last_undegraded":"2024-02-23T00:41:25.410189+0000","last_fullsized":"2024-02-23T00:41:25.410189+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:55.723309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:55.723309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"68'16","reported_seq":74,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409855+0000","last_change":"2024-02-23T00:40:41.772244+0000","last_active":"2024-02-23T00:41:25.409855+0000","last_peered":"2024-02-23T00:41:25.409855+0000","last_clean":"2024-02-23T00:41:25.409855+0000","last_became_active":"2024-02-23T00:39:16.787468+0000","last_became_peered":"2024-02-23T00:39:16.787468+0000","last_unstale":"2024-02-23T00:41:25.409855+0000","last_undegraded":"2024-02-23T00:41:25.409855+0000","last_fullsized":"2024-02-23T00:41:25.409855+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:40:41.772150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:41.772150+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"91'9","reported_seq":79,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:27.866628+0000","last_change":"2024-02-23T00:41:12.843814+0000","last_active":"2024-02-23T00:41:27.866628+0000","last_peered":"2024-02-23T00:41:27.866628+0000","last_clean":"2024-02-23T00:41:27.866628+0000","last_became_active":"2024-02-23T00:39:18.801297+0000","last_became_peered":"2024-02-23T00:39:18.801297+0000","last_unstale":"2024-02-23T00:41:27.866628+0000","last_undegraded":"2024-02-23T00:41:27.866628+0000","last_fullsized":"2024-02-23T00:41:27.866628+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'9","last_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_deep_scrub":"91'9","last_deep_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_clean_scrub_stamp":"2024-02-23T00:41:12.843733+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.355117+0000","last_change":"2024-02-23T00:41:18.910682+0000","last_active":"2024-02-23T00:41:25.355117+0000","last_peered":"2024-02-23T00:41:25.355117+0000","last_clean":"2024-02-23T00:41:25.355117+0000","last_became_active":"2024-02-23T00:39:20.790053+0000","last_became_peered":"2024-02-23T00:39:20.790053+0000","last_unstale":"2024-02-23T00:41:25.355117+0000","last_undegraded":"2024-02-23T00:41:25.355117+0000","last_fullsized":"2024-02-23T00:41:25.355117+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:18.910605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:18.910605+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409348+0000","last_change":"2024-02-23T00:40:53.824228+0000","last_active":"2024-02-23T00:41:25.409348+0000","last_peered":"2024-02-23T00:41:25.409348+0000","last_clean":"2024-02-23T00:41:25.409348+0000","last_became_active":"2024-02-23T00:39:14.784015+0000","last_became_peered":"2024-02-23T00:39:14.784015+0000","last_unstale":"2024-02-23T00:41:25.409348+0000","last_undegraded":"2024-02-23T00:41:25.409348+0000","last_fullsized":"2024-02-23T00:41:25.409348+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:53.824161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:53.824161+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"86'35","reported_seq":56,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:27.867193+0000","last_change":"2024-02-23T00:40:48.803965+0000","last_active":"2024-02-23T00:41:27.867193+0000","last_peered":"2024-02-23T00:41:27.867193+0000","last_clean":"2024-02-23T00:41:27.867193+0000","last_became_active":"2024-02-23T00:40:42.933322+0000","last_became_peered":"2024-02-23T00:40:42.933322+0000","last_unstale":"2024-02-23T00:41:27.867193+0000","last_undegraded":"2024-02-23T00:41:27.867193+0000","last_fullsized":"2024-02-23T00:41:27.867193+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'35","last_scrub_stamp":"2024-02-23T00:40:48.803893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:48.803893+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.356567+0000","last_change":"2024-02-23T00:41:19.899792+0000","last_active":"2024-02-23T00:41:25.356567+0000","last_peered":"2024-02-23T00:41:25.356567+0000","last_clean":"2024-02-23T00:41:25.356567+0000","last_became_active":"2024-02-23T00:39:18.802134+0000","last_became_peered":"2024-02-23T00:39:18.802134+0000","last_unstale":"2024-02-23T00:41:25.356567+0000","last_undegraded":"2024-02-23T00:41:25.356567+0000","last_fullsized":"2024-02-23T00:41:25.356567+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:19.899710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:19.899710+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.360186+0000","last_change":"2024-02-23T00:40:36.598336+0000","last_active":"2024-02-23T00:41:25.360186+0000","last_peered":"2024-02-23T00:41:25.360186+0000","last_clean":"2024-02-23T00:41:25.360186+0000","last_became_active":"2024-02-23T00:39:21.160457+0000","last_became_peered":"2024-02-23T00:39:21.160457+0000","last_unstale":"2024-02-23T00:41:25.360186+0000","last_undegraded":"2024-02-23T00:41:25.360186+0000","last_fullsized":"2024-02-23T00:41:25.360186+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:36.598261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:36.598261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"68'5","reported_seq":179,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-23T00:41:31.026744+0000","last_change":"2024-02-23T00:41:00.648420+0000","last_active":"2024-02-23T00:41:31.026744+0000","last_peered":"2024-02-23T00:41:31.026744+0000","last_clean":"2024-02-23T00:41:31.026744+0000","last_became_active":"2024-02-23T00:39:14.779109+0000","last_became_peered":"2024-02-23T00:39:14.779109+0000","last_unstale":"2024-02-23T00:41:31.026744+0000","last_undegraded":"2024-02-23T00:41:31.026744+0000","last_fullsized":"2024-02-23T00:41:31.026744+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'5","last_scrub_stamp":"2024-02-23T00:41:00.648353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:00.648353+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":130,"num_read_kb":125,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409069+0000","last_change":"2024-02-23T00:41:00.318702+0000","last_active":"2024-02-23T00:41:25.409069+0000","last_peered":"2024-02-23T00:41:25.409069+0000","last_clean":"2024-02-23T00:41:25.409069+0000","last_became_active":"2024-02-23T00:39:14.772411+0000","last_became_peered":"2024-02-23T00:39:14.772411+0000","last_unstale":"2024-02-23T00:41:25.409069+0000","last_undegraded":"2024-02-23T00:41:25.409069+0000","last_fullsized":"2024-02-23T00:41:25.409069+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:00.318633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:00.318633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"68'19","reported_seq":86,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.410380+0000","last_change":"2024-02-23T00:40:44.813607+0000","last_active":"2024-02-23T00:41:25.410380+0000","last_peered":"2024-02-23T00:41:25.410380+0000","last_clean":"2024-02-23T00:41:25.410380+0000","last_became_active":"2024-02-23T00:39:16.787484+0000","last_became_peered":"2024-02-23T00:39:16.787484+0000","last_unstale":"2024-02-23T00:41:25.410380+0000","last_undegraded":"2024-02-23T00:41:25.410380+0000","last_fullsized":"2024-02-23T00:41:25.410380+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'19","last_scrub_stamp":"2024-02-23T00:40:44.813506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:44.813506+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409519+0000","last_change":"2024-02-23T00:40:36.503413+0000","last_active":"2024-02-23T00:41:25.409519+0000","last_peered":"2024-02-23T00:41:25.409519+0000","last_clean":"2024-02-23T00:41:25.409519+0000","last_became_active":"2024-02-23T00:39:18.789330+0000","last_became_peered":"2024-02-23T00:39:18.789330+0000","last_unstale":"2024-02-23T00:41:25.409519+0000","last_undegraded":"2024-02-23T00:41:25.409519+0000","last_fullsized":"2024-02-23T00:41:25.409519+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:36.503342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:36.503342+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.411404+0000","last_change":"2024-02-23T00:40:51.806086+0000","last_active":"2024-02-23T00:41:25.411404+0000","last_peered":"2024-02-23T00:41:25.411404+0000","last_clean":"2024-02-23T00:41:25.411404+0000","last_became_active":"2024-02-23T00:39:20.797226+0000","last_became_peered":"2024-02-23T00:39:20.797226+0000","last_unstale":"2024-02-23T00:41:25.411404+0000","last_undegraded":"2024-02-23T00:41:25.411404+0000","last_fullsized":"2024-02-23T00:41:25.411404+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:51.806007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:51.806007+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.354323+0000","last_change":"2024-02-23T00:40:59.958953+0000","last_active":"2024-02-23T00:41:25.354323+0000","last_peered":"2024-02-23T00:41:25.354323+0000","last_clean":"2024-02-23T00:41:25.354323+0000","last_became_active":"2024-02-23T00:39:14.783489+0000","last_became_peered":"2024-02-23T00:39:14.783489+0000","last_unstale":"2024-02-23T00:41:25.354323+0000","last_undegraded":"2024-02-23T00:41:25.354323+0000","last_fullsized":"2024-02-23T00:41:25.354323+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:59.958850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:59.958850+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409866+0000","last_change":"2024-02-23T00:40:50.760096+0000","last_active":"2024-02-23T00:41:25.409866+0000","last_peered":"2024-02-23T00:41:25.409866+0000","last_clean":"2024-02-23T00:41:25.409866+0000","last_became_active":"2024-02-23T00:39:14.788888+0000","last_became_peered":"2024-02-23T00:39:14.788888+0000","last_unstale":"2024-02-23T00:41:25.409866+0000","last_undegraded":"2024-02-23T00:41:25.409866+0000","last_fullsized":"2024-02-23T00:41:25.409866+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:50.760015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:50.760015+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.359439+0000","last_change":"2024-02-23T00:40:53.770177+0000","last_active":"2024-02-23T00:41:25.359439+0000","last_peered":"2024-02-23T00:41:25.359439+0000","last_clean":"2024-02-23T00:41:25.359439+0000","last_became_active":"2024-02-23T00:39:08.248249+0000","last_became_peered":"2024-02-23T00:39:08.248249+0000","last_unstale":"2024-02-23T00:41:25.359439+0000","last_undegraded":"2024-02-23T00:41:25.359439+0000","last_fullsized":"2024-02-23T00:41:25.359439+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:53.770082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:53.770082+0000","last_clean_scrub_stamp":"2024-02-23T00:40:53.770082+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"67'14","reported_seq":66,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.410915+0000","last_change":"2024-02-23T00:40:41.920396+0000","last_active":"2024-02-23T00:41:25.410915+0000","last_peered":"2024-02-23T00:41:25.410915+0000","last_clean":"2024-02-23T00:41:25.410915+0000","last_became_active":"2024-02-23T00:39:16.858880+0000","last_became_peered":"2024-02-23T00:39:16.858880+0000","last_unstale":"2024-02-23T00:41:25.410915+0000","last_undegraded":"2024-02-23T00:41:25.410915+0000","last_fullsized":"2024-02-23T00:41:25.410915+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'14","last_scrub_stamp":"2024-02-23T00:40:41.920313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:41.920313+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.360024+0000","last_change":"2024-02-23T00:40:48.790072+0000","last_active":"2024-02-23T00:41:25.360024+0000","last_peered":"2024-02-23T00:41:25.360024+0000","last_clean":"2024-02-23T00:41:25.360024+0000","last_became_active":"2024-02-23T00:39:18.793801+0000","last_became_peered":"2024-02-23T00:39:18.793801+0000","last_unstale":"2024-02-23T00:41:25.360024+0000","last_undegraded":"2024-02-23T00:41:25.360024+0000","last_fullsized":"2024-02-23T00:41:25.360024+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:48.789992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:48.789992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.410040+0000","last_change":"2024-02-23T00:41:02.714085+0000","last_active":"2024-02-23T00:41:25.410040+0000","last_peered":"2024-02-23T00:41:25.410040+0000","last_clean":"2024-02-23T00:41:25.410040+0000","last_became_active":"2024-02-23T00:39:20.797242+0000","last_became_peered":"2024-02-23T00:39:20.797242+0000","last_unstale":"2024-02-23T00:41:25.410040+0000","last_undegraded":"2024-02-23T00:41:25.410040+0000","last_fullsized":"2024-02-23T00:41:25.410040+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:02.714020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:02.714020+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.356261+0000","last_change":"2024-02-23T00:40:15.584807+0000","last_active":"2024-02-23T00:41:25.356261+0000","last_peered":"2024-02-23T00:41:25.356261+0000","last_clean":"2024-02-23T00:41:25.356261+0000","last_became_active":"2024-02-23T00:39:14.775446+0000","last_became_peered":"2024-02-23T00:39:14.775446+0000","last_unstale":"2024-02-23T00:41:25.356261+0000","last_undegraded":"2024-02-23T00:41:25.356261+0000","last_fullsized":"2024-02-23T00:41:25.356261+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:15.584732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:15.584732+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:26.836926+0000","last_change":"2024-02-23T00:41:26.836896+0000","last_active":"2024-02-23T00:41:26.836926+0000","last_peered":"2024-02-23T00:41:26.836926+0000","last_clean":"2024-02-23T00:41:26.836926+0000","last_became_active":"2024-02-23T00:39:14.784578+0000","last_became_peered":"2024-02-23T00:39:14.784578+0000","last_unstale":"2024-02-23T00:41:26.836926+0000","last_undegraded":"2024-02-23T00:41:26.836926+0000","last_fullsized":"2024-02-23T00:41:26.836926+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:26.836819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:26.836819+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"67'12","reported_seq":63,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.356572+0000","last_change":"2024-02-23T00:40:48.696967+0000","last_active":"2024-02-23T00:41:25.356572+0000","last_peered":"2024-02-23T00:41:25.356572+0000","last_clean":"2024-02-23T00:41:25.356572+0000","last_became_active":"2024-02-23T00:39:16.787143+0000","last_became_peered":"2024-02-23T00:39:16.787143+0000","last_unstale":"2024-02-23T00:41:25.356572+0000","last_undegraded":"2024-02-23T00:41:25.356572+0000","last_fullsized":"2024-02-23T00:41:25.356572+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'12","last_scrub_stamp":"2024-02-23T00:40:48.696895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:48.696895+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.411357+0000","last_change":"2024-02-23T00:40:38.812399+0000","last_active":"2024-02-23T00:41:25.411357+0000","last_peered":"2024-02-23T00:41:25.411357+0000","last_clean":"2024-02-23T00:41:25.411357+0000","last_became_active":"2024-02-23T00:39:18.800378+0000","last_became_peered":"2024-02-23T00:39:18.800378+0000","last_unstale":"2024-02-23T00:41:25.411357+0000","last_undegraded":"2024-02-23T00:41:25.411357+0000","last_fullsized":"2024-02-23T00:41:25.411357+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:38.812327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:38.812327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409535+0000","last_change":"2024-02-23T00:40:27.859722+0000","last_active":"2024-02-23T00:41:25.409535+0000","last_peered":"2024-02-23T00:41:25.409535+0000","last_clean":"2024-02-23T00:41:25.409535+0000","last_became_active":"2024-02-23T00:39:21.160060+0000","last_became_peered":"2024-02-23T00:39:21.160060+0000","last_unstale":"2024-02-23T00:41:25.409535+0000","last_undegraded":"2024-02-23T00:41:25.409535+0000","last_fullsized":"2024-02-23T00:41:25.409535+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:27.859657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:27.859657+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.360192+0000","last_change":"2024-02-23T00:40:41.671069+0000","last_active":"2024-02-23T00:41:25.360192+0000","last_peered":"2024-02-23T00:41:25.360192+0000","last_clean":"2024-02-23T00:41:25.360192+0000","last_became_active":"2024-02-23T00:39:14.783662+0000","last_became_peered":"2024-02-23T00:39:14.783662+0000","last_unstale":"2024-02-23T00:41:25.360192+0000","last_undegraded":"2024-02-23T00:41:25.360192+0000","last_fullsized":"2024-02-23T00:41:25.360192+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:41.670962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:41.670962+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"68'25","reported_seq":96,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.356361+0000","last_change":"2024-02-23T00:40:52.751667+0000","last_active":"2024-02-23T00:41:25.356361+0000","last_peered":"2024-02-23T00:41:25.356361+0000","last_clean":"2024-02-23T00:41:25.356361+0000","last_became_active":"2024-02-23T00:39:16.857278+0000","last_became_peered":"2024-02-23T00:39:16.857278+0000","last_unstale":"2024-02-23T00:41:25.356361+0000","last_undegraded":"2024-02-23T00:41:25.356361+0000","last_fullsized":"2024-02-23T00:41:25.356361+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'25","last_scrub_stamp":"2024-02-23T00:40:52.751597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:52.751597+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.411359+0000","last_change":"2024-02-23T00:40:50.029161+0000","last_active":"2024-02-23T00:41:25.411359+0000","last_peered":"2024-02-23T00:41:25.411359+0000","last_clean":"2024-02-23T00:41:25.411359+0000","last_became_active":"2024-02-23T00:39:18.796664+0000","last_became_peered":"2024-02-23T00:39:18.796664+0000","last_unstale":"2024-02-23T00:41:25.411359+0000","last_undegraded":"2024-02-23T00:41:25.411359+0000","last_fullsized":"2024-02-23T00:41:25.411359+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:50.029087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:50.029087+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.410400+0000","last_change":"2024-02-23T00:40:49.937604+0000","last_active":"2024-02-23T00:41:25.410400+0000","last_peered":"2024-02-23T00:41:25.410400+0000","last_clean":"2024-02-23T00:41:25.410400+0000","last_became_active":"2024-02-23T00:39:20.795271+0000","last_became_peered":"2024-02-23T00:39:20.795271+0000","last_unstale":"2024-02-23T00:41:25.410400+0000","last_undegraded":"2024-02-23T00:41:25.410400+0000","last_fullsized":"2024-02-23T00:41:25.410400+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:49.937526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:49.937526+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.410267+0000","last_change":"2024-02-23T00:40:36.888232+0000","last_active":"2024-02-23T00:41:25.410267+0000","last_peered":"2024-02-23T00:41:25.410267+0000","last_clean":"2024-02-23T00:41:25.410267+0000","last_became_active":"2024-02-23T00:39:14.779193+0000","last_became_peered":"2024-02-23T00:39:14.779193+0000","last_unstale":"2024-02-23T00:41:25.410267+0000","last_undegraded":"2024-02-23T00:41:25.410267+0000","last_fullsized":"2024-02-23T00:41:25.410267+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:36.888124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:36.888124+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"68'30","reported_seq":107,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.354172+0000","last_change":"2024-02-23T00:41:21.909427+0000","last_active":"2024-02-23T00:41:25.354172+0000","last_peered":"2024-02-23T00:41:25.354172+0000","last_clean":"2024-02-23T00:41:25.354172+0000","last_became_active":"2024-02-23T00:39:16.859002+0000","last_became_peered":"2024-02-23T00:39:16.859002+0000","last_unstale":"2024-02-23T00:41:25.354172+0000","last_undegraded":"2024-02-23T00:41:25.354172+0000","last_fullsized":"2024-02-23T00:41:25.354172+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'30","last_scrub_stamp":"2024-02-23T00:41:21.909340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:21.909340+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409784+0000","last_change":"2024-02-23T00:40:23.103679+0000","last_active":"2024-02-23T00:41:25.409784+0000","last_peered":"2024-02-23T00:41:25.409784+0000","last_clean":"2024-02-23T00:41:25.409784+0000","last_became_active":"2024-02-23T00:39:18.798829+0000","last_became_peered":"2024-02-23T00:39:18.798829+0000","last_unstale":"2024-02-23T00:41:25.409784+0000","last_undegraded":"2024-02-23T00:41:25.409784+0000","last_fullsized":"2024-02-23T00:41:25.409784+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:23.103586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:23.103586+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.359899+0000","last_change":"2024-02-23T00:40:33.626648+0000","last_active":"2024-02-23T00:41:25.359899+0000","last_peered":"2024-02-23T00:41:25.359899+0000","last_clean":"2024-02-23T00:41:25.359899+0000","last_became_active":"2024-02-23T00:39:21.161928+0000","last_became_peered":"2024-02-23T00:39:21.161928+0000","last_unstale":"2024-02-23T00:41:25.359899+0000","last_undegraded":"2024-02-23T00:41:25.359899+0000","last_fullsized":"2024-02-23T00:41:25.359899+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:33.626500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:33.626500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.410759+0000","last_change":"2024-02-23T00:40:45.971552+0000","last_active":"2024-02-23T00:41:25.410759+0000","last_peered":"2024-02-23T00:41:25.410759+0000","last_clean":"2024-02-23T00:41:25.410759+0000","last_became_active":"2024-02-23T00:39:14.781999+0000","last_became_peered":"2024-02-23T00:39:14.781999+0000","last_unstale":"2024-02-23T00:41:25.410759+0000","last_undegraded":"2024-02-23T00:41:25.410759+0000","last_fullsized":"2024-02-23T00:41:25.410759+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:45.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:45.971465+0000","last_clean_scrub_stamp":"2024-02-23T00:40:45.971465+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"68'20","reported_seq":80,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409514+0000","last_change":"2024-02-23T00:41:05.002039+0000","last_active":"2024-02-23T00:41:25.409514+0000","last_peered":"2024-02-23T00:41:25.409514+0000","last_clean":"2024-02-23T00:41:25.409514+0000","last_became_active":"2024-02-23T00:39:16.780130+0000","last_became_peered":"2024-02-23T00:39:16.780130+0000","last_unstale":"2024-02-23T00:41:25.409514+0000","last_undegraded":"2024-02-23T00:41:25.409514+0000","last_fullsized":"2024-02-23T00:41:25.409514+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'20","last_scrub_stamp":"2024-02-23T00:41:05.001912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:05.001912+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":78,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:27.866638+0000","last_change":"2024-02-23T00:41:09.882111+0000","last_active":"2024-02-23T00:41:27.866638+0000","last_peered":"2024-02-23T00:41:27.866638+0000","last_clean":"2024-02-23T00:41:27.866638+0000","last_became_active":"2024-02-23T00:39:18.798536+0000","last_became_peered":"2024-02-23T00:39:18.798536+0000","last_unstale":"2024-02-23T00:41:27.866638+0000","last_undegraded":"2024-02-23T00:41:27.866638+0000","last_fullsized":"2024-02-23T00:41:27.866638+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:41:09.882012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:09.882012+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.354626+0000","last_change":"2024-02-23T00:40:46.876448+0000","last_active":"2024-02-23T00:41:25.354626+0000","last_peered":"2024-02-23T00:41:25.354626+0000","last_clean":"2024-02-23T00:41:25.354626+0000","last_became_active":"2024-02-23T00:39:21.160443+0000","last_became_peered":"2024-02-23T00:39:21.160443+0000","last_unstale":"2024-02-23T00:41:25.354626+0000","last_undegraded":"2024-02-23T00:41:25.354626+0000","last_fullsized":"2024-02-23T00:41:25.354626+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:46.876380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:46.876380+0000","last_clean_scrub_stamp":"2024-02-23T00:40:46.876380+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.411279+0000","last_change":"2024-02-23T00:41:04.839634+0000","last_active":"2024-02-23T00:41:25.411279+0000","last_peered":"2024-02-23T00:41:25.411279+0000","last_clean":"2024-02-23T00:41:25.411279+0000","last_became_active":"2024-02-23T00:39:18.800777+0000","last_became_peered":"2024-02-23T00:39:18.800777+0000","last_unstale":"2024-02-23T00:41:25.411279+0000","last_undegraded":"2024-02-23T00:41:25.411279+0000","last_fullsized":"2024-02-23T00:41:25.411279+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_clean_scrub_stamp":"2024-02-23T00:41:04.839555+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.359420+0000","last_change":"2024-02-23T00:40:23.574894+0000","last_active":"2024-02-23T00:41:25.359420+0000","last_peered":"2024-02-23T00:41:25.359420+0000","last_clean":"2024-02-23T00:41:25.359420+0000","last_became_active":"2024-02-23T00:39:14.784121+0000","last_became_peered":"2024-02-23T00:39:14.784121+0000","last_unstale":"2024-02-23T00:41:25.359420+0000","last_undegraded":"2024-02-23T00:41:25.359420+0000","last_fullsized":"2024-02-23T00:41:25.359420+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:23.574813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:23.574813+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"68'17","reported_seq":101,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.410545+0000","last_change":"2024-02-23T00:41:13.017122+0000","last_active":"2024-02-23T00:41:25.410545+0000","last_peered":"2024-02-23T00:41:25.410545+0000","last_clean":"2024-02-23T00:41:25.410545+0000","last_became_active":"2024-02-23T00:40:42.933759+0000","last_became_peered":"2024-02-23T00:40:42.933759+0000","last_unstale":"2024-02-23T00:41:25.410545+0000","last_undegraded":"2024-02-23T00:41:25.410545+0000","last_fullsized":"2024-02-23T00:41:25.410545+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:41:13.017043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:13.017043+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.410163+0000","last_change":"2024-02-23T00:40:56.890866+0000","last_active":"2024-02-23T00:41:25.410163+0000","last_peered":"2024-02-23T00:41:25.410163+0000","last_clean":"2024-02-23T00:41:25.410163+0000","last_became_active":"2024-02-23T00:39:20.800800+0000","last_became_peered":"2024-02-23T00:39:20.800800+0000","last_unstale":"2024-02-23T00:41:25.410163+0000","last_undegraded":"2024-02-23T00:41:25.410163+0000","last_fullsized":"2024-02-23T00:41:25.410163+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:56.890792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:56.890792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":77,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:27.867032+0000","last_change":"2024-02-23T00:40:30.998799+0000","last_active":"2024-02-23T00:41:27.867032+0000","last_peered":"2024-02-23T00:41:27.867032+0000","last_clean":"2024-02-23T00:41:27.867032+0000","last_became_active":"2024-02-23T00:39:18.800440+0000","last_became_peered":"2024-02-23T00:39:18.800440+0000","last_unstale":"2024-02-23T00:41:27.867032+0000","last_undegraded":"2024-02-23T00:41:27.867032+0000","last_fullsized":"2024-02-23T00:41:27.867032+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:40:30.998729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:30.998729+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409753+0000","last_change":"2024-02-23T00:40:26.845049+0000","last_active":"2024-02-23T00:41:25.409753+0000","last_peered":"2024-02-23T00:41:25.409753+0000","last_clean":"2024-02-23T00:41:25.409753+0000","last_became_active":"2024-02-23T00:39:14.780526+0000","last_became_peered":"2024-02-23T00:39:14.780526+0000","last_unstale":"2024-02-23T00:41:25.409753+0000","last_undegraded":"2024-02-23T00:41:25.409753+0000","last_fullsized":"2024-02-23T00:41:25.409753+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:26.844984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:26.844984+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"67'15","reported_seq":69,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.410444+0000","last_change":"2024-02-23T00:40:18.805518+0000","last_active":"2024-02-23T00:41:25.410444+0000","last_peered":"2024-02-23T00:41:25.410444+0000","last_clean":"2024-02-23T00:41:25.410444+0000","last_became_active":"2024-02-23T00:39:16.789757+0000","last_became_peered":"2024-02-23T00:39:16.789757+0000","last_unstale":"2024-02-23T00:41:25.410444+0000","last_undegraded":"2024-02-23T00:41:25.410444+0000","last_fullsized":"2024-02-23T00:41:25.410444+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:40:18.805441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:18.805441+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.410403+0000","last_change":"2024-02-23T00:41:15.081250+0000","last_active":"2024-02-23T00:41:25.410403+0000","last_peered":"2024-02-23T00:41:25.410403+0000","last_clean":"2024-02-23T00:41:25.410403+0000","last_became_active":"2024-02-23T00:39:20.795097+0000","last_became_peered":"2024-02-23T00:39:20.795097+0000","last_unstale":"2024-02-23T00:41:25.410403+0000","last_undegraded":"2024-02-23T00:41:25.410403+0000","last_fullsized":"2024-02-23T00:41:25.410403+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:15.081153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:15.081153+0000","last_clean_scrub_stamp":"2024-02-23T00:41:15.081153+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"67'8","reported_seq":77,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:27.866648+0000","last_change":"2024-02-23T00:40:58.331136+0000","last_active":"2024-02-23T00:41:27.866648+0000","last_peered":"2024-02-23T00:41:27.866648+0000","last_clean":"2024-02-23T00:41:27.866648+0000","last_became_active":"2024-02-23T00:39:18.799544+0000","last_became_peered":"2024-02-23T00:39:18.799544+0000","last_unstale":"2024-02-23T00:41:27.866648+0000","last_undegraded":"2024-02-23T00:41:27.866648+0000","last_fullsized":"2024-02-23T00:41:27.866648+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:40:58.331048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:58.331048+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409728+0000","last_change":"2024-02-23T00:40:39.825090+0000","last_active":"2024-02-23T00:41:25.409728+0000","last_peered":"2024-02-23T00:41:25.409728+0000","last_clean":"2024-02-23T00:41:25.409728+0000","last_became_active":"2024-02-23T00:39:14.780379+0000","last_became_peered":"2024-02-23T00:39:14.780379+0000","last_unstale":"2024-02-23T00:41:25.409728+0000","last_undegraded":"2024-02-23T00:41:25.409728+0000","last_fullsized":"2024-02-23T00:41:25.409728+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:39.825013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:39.825013+0000","last_clean_scrub_stamp":"2024-02-23T00:40:39.825013+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"67'10","reported_seq":59,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.410071+0000","last_change":"2024-02-23T00:41:14.059424+0000","last_active":"2024-02-23T00:41:25.410071+0000","last_peered":"2024-02-23T00:41:25.410071+0000","last_clean":"2024-02-23T00:41:25.410071+0000","last_became_active":"2024-02-23T00:39:16.777530+0000","last_became_peered":"2024-02-23T00:39:16.777530+0000","last_unstale":"2024-02-23T00:41:25.410071+0000","last_undegraded":"2024-02-23T00:41:25.410071+0000","last_fullsized":"2024-02-23T00:41:25.410071+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'10","last_scrub_stamp":"2024-02-23T00:41:14.059334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:14.059334+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409331+0000","last_change":"2024-02-23T00:40:54.348946+0000","last_active":"2024-02-23T00:41:25.409331+0000","last_peered":"2024-02-23T00:41:25.409331+0000","last_clean":"2024-02-23T00:41:25.409331+0000","last_became_active":"2024-02-23T00:39:20.800943+0000","last_became_peered":"2024-02-23T00:39:20.800943+0000","last_unstale":"2024-02-23T00:41:25.409331+0000","last_undegraded":"2024-02-23T00:41:25.409331+0000","last_fullsized":"2024-02-23T00:41:25.409331+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:54.348843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:54.348843+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.410064+0000","last_change":"2024-02-23T00:40:35.861920+0000","last_active":"2024-02-23T00:41:25.410064+0000","last_peered":"2024-02-23T00:41:25.410064+0000","last_clean":"2024-02-23T00:41:25.410064+0000","last_became_active":"2024-02-23T00:39:18.799875+0000","last_became_peered":"2024-02-23T00:39:18.799875+0000","last_unstale":"2024-02-23T00:41:25.410064+0000","last_undegraded":"2024-02-23T00:41:25.410064+0000","last_fullsized":"2024-02-23T00:41:25.410064+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:35.861854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:35.861854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409214+0000","last_change":"2024-02-23T00:40:42.931442+0000","last_active":"2024-02-23T00:41:25.409214+0000","last_peered":"2024-02-23T00:41:25.409214+0000","last_clean":"2024-02-23T00:41:25.409214+0000","last_became_active":"2024-02-23T00:40:42.931041+0000","last_became_peered":"2024-02-23T00:40:42.931041+0000","last_unstale":"2024-02-23T00:41:25.409214+0000","last_undegraded":"2024-02-23T00:41:25.409214+0000","last_fullsized":"2024-02-23T00:41:25.409214+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:19.487609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:19.487609+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"67'17","reported_seq":99,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.410911+0000","last_change":"2024-02-23T00:41:02.038717+0000","last_active":"2024-02-23T00:41:25.410911+0000","last_peered":"2024-02-23T00:41:25.410911+0000","last_clean":"2024-02-23T00:41:25.410911+0000","last_became_active":"2024-02-23T00:40:42.933806+0000","last_became_peered":"2024-02-23T00:40:42.933806+0000","last_unstale":"2024-02-23T00:41:25.410911+0000","last_undegraded":"2024-02-23T00:41:25.410911+0000","last_fullsized":"2024-02-23T00:41:25.410911+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'17","last_scrub_stamp":"2024-02-23T00:41:02.038627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:02.038627+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.359284+0000","last_change":"2024-02-23T00:40:51.777535+0000","last_active":"2024-02-23T00:41:25.359284+0000","last_peered":"2024-02-23T00:41:25.359284+0000","last_clean":"2024-02-23T00:41:25.359284+0000","last_became_active":"2024-02-23T00:39:20.798989+0000","last_became_peered":"2024-02-23T00:39:20.798989+0000","last_unstale":"2024-02-23T00:41:25.359284+0000","last_undegraded":"2024-02-23T00:41:25.359284+0000","last_fullsized":"2024-02-23T00:41:25.359284+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:51.777405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:51.777405+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409384+0000","last_change":"2024-02-23T00:40:33.492291+0000","last_active":"2024-02-23T00:41:25.409384+0000","last_peered":"2024-02-23T00:41:25.409384+0000","last_clean":"2024-02-23T00:41:25.409384+0000","last_became_active":"2024-02-23T00:39:18.792718+0000","last_became_peered":"2024-02-23T00:39:18.792718+0000","last_unstale":"2024-02-23T00:41:25.409384+0000","last_undegraded":"2024-02-23T00:41:25.409384+0000","last_fullsized":"2024-02-23T00:41:25.409384+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:33.492224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:33.492224+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409569+0000","last_change":"2024-02-23T00:40:28.450480+0000","last_active":"2024-02-23T00:41:25.409569+0000","last_peered":"2024-02-23T00:41:25.409569+0000","last_clean":"2024-02-23T00:41:25.409569+0000","last_became_active":"2024-02-23T00:39:14.776705+0000","last_became_peered":"2024-02-23T00:39:14.776705+0000","last_unstale":"2024-02-23T00:41:25.409569+0000","last_undegraded":"2024-02-23T00:41:25.409569+0000","last_fullsized":"2024-02-23T00:41:25.409569+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_clean_scrub_stamp":"2024-02-23T00:40:28.450377+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"88'35","reported_seq":100,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.411294+0000","last_change":"2024-02-23T00:40:54.152313+0000","last_active":"2024-02-23T00:41:25.411294+0000","last_peered":"2024-02-23T00:41:25.411294+0000","last_clean":"2024-02-23T00:41:25.411294+0000","last_became_active":"2024-02-23T00:39:16.857413+0000","last_became_peered":"2024-02-23T00:39:16.857413+0000","last_unstale":"2024-02-23T00:41:25.411294+0000","last_undegraded":"2024-02-23T00:41:25.411294+0000","last_fullsized":"2024-02-23T00:41:25.411294+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'35","last_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_deep_scrub":"88'35","last_deep_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_clean_scrub_stamp":"2024-02-23T00:40:54.152239+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.355546+0000","last_change":"2024-02-23T00:41:05.802853+0000","last_active":"2024-02-23T00:41:25.355546+0000","last_peered":"2024-02-23T00:41:25.355546+0000","last_clean":"2024-02-23T00:41:25.355546+0000","last_became_active":"2024-02-23T00:39:20.802938+0000","last_became_peered":"2024-02-23T00:39:20.802938+0000","last_unstale":"2024-02-23T00:41:25.355546+0000","last_undegraded":"2024-02-23T00:41:25.355546+0000","last_fullsized":"2024-02-23T00:41:25.355546+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:05.802777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:05.802777+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409264+0000","last_change":"2024-02-23T00:41:02.376871+0000","last_active":"2024-02-23T00:41:25.409264+0000","last_peered":"2024-02-23T00:41:25.409264+0000","last_clean":"2024-02-23T00:41:25.409264+0000","last_became_active":"2024-02-23T00:39:18.794476+0000","last_became_peered":"2024-02-23T00:39:18.794476+0000","last_unstale":"2024-02-23T00:41:25.409264+0000","last_undegraded":"2024-02-23T00:41:25.409264+0000","last_fullsized":"2024-02-23T00:41:25.409264+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:02.376804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:02.376804+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.410371+0000","last_change":"2024-02-23T00:40:57.681793+0000","last_active":"2024-02-23T00:41:25.410371+0000","last_peered":"2024-02-23T00:41:25.410371+0000","last_clean":"2024-02-23T00:41:25.410371+0000","last_became_active":"2024-02-23T00:39:14.772493+0000","last_became_peered":"2024-02-23T00:39:14.772493+0000","last_unstale":"2024-02-23T00:41:25.410371+0000","last_undegraded":"2024-02-23T00:41:25.410371+0000","last_fullsized":"2024-02-23T00:41:25.410371+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:57.681723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:57.681723+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"68'15","reported_seq":76,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.356465+0000","last_change":"2024-02-23T00:40:47.732636+0000","last_active":"2024-02-23T00:41:25.356465+0000","last_peered":"2024-02-23T00:41:25.356465+0000","last_clean":"2024-02-23T00:41:25.356465+0000","last_became_active":"2024-02-23T00:39:16.783701+0000","last_became_peered":"2024-02-23T00:39:16.783701+0000","last_unstale":"2024-02-23T00:41:25.356465+0000","last_undegraded":"2024-02-23T00:41:25.356465+0000","last_fullsized":"2024-02-23T00:41:25.356465+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:40:47.732524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:47.732524+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409246+0000","last_change":"2024-02-23T00:40:39.859273+0000","last_active":"2024-02-23T00:41:25.409246+0000","last_peered":"2024-02-23T00:41:25.409246+0000","last_clean":"2024-02-23T00:41:25.409246+0000","last_became_active":"2024-02-23T00:39:21.160429+0000","last_became_peered":"2024-02-23T00:39:21.160429+0000","last_unstale":"2024-02-23T00:41:25.409246+0000","last_undegraded":"2024-02-23T00:41:25.409246+0000","last_fullsized":"2024-02-23T00:41:25.409246+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:39.859197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:39.859197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409316+0000","last_change":"2024-02-23T00:40:34.533819+0000","last_active":"2024-02-23T00:41:25.409316+0000","last_peered":"2024-02-23T00:41:25.409316+0000","last_clean":"2024-02-23T00:41:25.409316+0000","last_became_active":"2024-02-23T00:39:14.784436+0000","last_became_peered":"2024-02-23T00:39:14.784436+0000","last_unstale":"2024-02-23T00:41:25.409316+0000","last_undegraded":"2024-02-23T00:41:25.409316+0000","last_fullsized":"2024-02-23T00:41:25.409316+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:34.533743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:34.533743+0000","last_clean_scrub_stamp":"2024-02-23T00:40:34.533743+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"68'27","reported_seq":100,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:25.409036+0000","last_change":"2024-02-23T00:40:59.885996+0000","last_active":"2024-02-23T00:41:25.409036+0000","last_peered":"2024-02-23T00:41:25.409036+0000","last_clean":"2024-02-23T00:41:25.409036+0000","last_became_active":"2024-02-23T00:39:16.859401+0000","last_became_peered":"2024-02-23T00:39:16.859401+0000","last_unstale":"2024-02-23T00:41:25.409036+0000","last_undegraded":"2024-02-23T00:41:25.409036+0000","last_fullsized":"2024-02-23T00:41:25.409036+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'27","last_scrub_stamp":"2024-02-23T00:40:59.885908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:59.885908+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"67'8","reported_seq":79,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-02-23T00:41:27.867083+0000","last_change":"2024-02-23T00:40:44.713885+0000","last_active":"2024-02-23T00:41:27.867083+0000","last_peered":"2024-02-23T00:41:27.867083+0000","last_clean":"2024-02-23T00:41:27.867083+0000","last_became_active":"2024-02-23T00:39:18.793414+0000","last_became_peered":"2024-02-23T00:39:18.793414+0000","last_unstale":"2024-02-23T00:41:27.867083+0000","last_undegraded":"2024-02-23T00:41:27.867083+0000","last_fullsized":"2024-02-23T00:41:27.867083+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:40:44.713825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:44.713825+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1082,"num_read_kb":807,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":610,"ondisk_log_size":610,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":130,"num_read_kb":125,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234015,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297828,"kb_used_data":732,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447324,"statfs":{"total":95995035648,"available":95690059776,"internally_reserved":0,"allocated":749568,"data_stored":376006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.374,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.69499999999999995,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.499,"15min":0.505},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.69999999999999996,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.51000000000000001}]},{"osd":1,"last update":"Fri Feb 23 00:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.54400000000000004,"15min":0.55000000000000004},"min":{"1min":0.38400000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.75600000000000001,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.54100000000000004,"15min":0.54700000000000004},"min":{"1min":0.39600000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.64600000000000002,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.44600000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.53000000000000003,"15min":0.53800000000000003},"min":{"1min":0.40000000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72699999999999998,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.52600000000000002,"15min":0.53200000000000003},"min":{"1min":0.34899999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.65300000000000002,"5min":1.024,"15min":1.024},"last":0.46800000000000003}]},{"osd":3,"last update":"Fri Feb 23 00:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.52100000000000002,"15min":0.52500000000000002},"min":{"1min":0.38200000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.68400000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.55600000000000005,"15min":0.56399999999999995},"min":{"1min":0.38300000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72499999999999998,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Feb 23 00:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.41999999999999998,"15min":0.42799999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.56899999999999995,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.443,"15min":0.45000000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.61099999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.56299999999999994}]},{"osd":5,"last update":"Fri Feb 23 00:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.46600000000000003,"15min":0.47399999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.58399999999999996,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.47099999999999997,"15min":0.47399999999999998},"min":{"1min":0.307,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.625,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Feb 23 00:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.497,"15min":0.503},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.496,"15min":0.503},"min":{"1min":0.33900000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.65500000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.61799999999999999}]}]},{"osd":6,"up_from":47,"seq":201863462950,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297808,"kb_used_data":712,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447344,"statfs":{"total":95995035648,"available":95690080256,"internally_reserved":0,"allocated":729088,"data_stored":375356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73199999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.58499999999999996}]},{"osd":1,"last update":"Fri Feb 23 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54900000000000004,"15min":0.55200000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72699999999999998,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77900000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.60999999999999999}]},{"osd":2,"last update":"Fri Feb 23 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58199999999999996,"15min":0.58399999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71199999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.749,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.66100000000000003}]},{"osd":3,"last update":"Fri Feb 23 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57899999999999996,"15min":0.58199999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81399999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59399999999999997,"15min":0.59499999999999997},"min":{"1min":0.41799999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.71599999999999997}]},{"osd":4,"last update":"Fri Feb 23 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.40999999999999998,"15min":0.40400000000000003},"min":{"1min":0.28199999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.433,"15min":0.42699999999999999},"min":{"1min":0.29899999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.42699999999999999}]},{"osd":5,"last update":"Fri Feb 23 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.41899999999999998,"15min":0.41199999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.44500000000000001,"15min":0.437},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.497}]},{"osd":7,"last update":"Fri Feb 23 00:41:11 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.48999999999999999,"15min":0.48799999999999999},"min":{"1min":0.37,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65200000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76700000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.47299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297836,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447316,"statfs":{"total":95995035648,"available":95690051584,"internally_reserved":0,"allocated":757760,"data_stored":376640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:40:37 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36899999999999999,"15min":0.34499999999999997},"min":{"1min":0.22600000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.57599999999999996,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.39200000000000002,"15min":0.36099999999999999},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.60299999999999998,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.42499999999999999}]},{"osd":2,"last update":"Fri Feb 23 00:40:58 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.39600000000000002,"15min":0.39400000000000002},"min":{"1min":0.245,"5min":0.188,"15min":0.188},"max":{"1min":0.53500000000000003,"5min":0.63,"15min":0.63},"last":0.307},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.441,"15min":0.435},"min":{"1min":0.29999999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.67900000000000005,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.40799999999999997}]},{"osd":3,"last update":"Fri Feb 23 00:41:31 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.47199999999999998,"15min":0.48499999999999999},"min":{"1min":0.314,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.62,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.374},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.443,"15min":0.45500000000000002},"min":{"1min":0.28999999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.66900000000000004,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.36199999999999999}]},{"osd":4,"last update":"Fri Feb 23 00:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60599999999999998,"15min":0.61199999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.875,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61199999999999999,"15min":0.61299999999999999},"min":{"1min":0.39500000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83199999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.70899999999999996}]},{"osd":5,"last update":"Fri Feb 23 00:41:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63500000000000001,"15min":0.65700000000000003},"min":{"1min":0.36899999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84599999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69599999999999995,"15min":0.71599999999999997},"min":{"1min":0.442,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.74299999999999999}]},{"osd":6,"last update":"Fri Feb 23 00:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63700000000000001,"15min":0.64200000000000002},"min":{"1min":0.40699999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81999999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.64400000000000002,"15min":0.65100000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.84399999999999997,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.57899999999999996}]},{"osd":7,"last update":"Fri Feb 23 00:41:09 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.46899999999999997,"5min":0.439,"15min":0.439},"max":{"1min":0.85799999999999998,"5min":0.875,"15min":0.875},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.622,"15min":0.622},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.79300000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297832,"kb_used_data":736,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447320,"statfs":{"total":95995035648,"available":95690055680,"internally_reserved":0,"allocated":753664,"data_stored":376231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 23 00:40:33 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.36899999999999999,"15min":0.33500000000000002},"min":{"1min":0.26200000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.56499999999999995,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.39300000000000002,"15min":0.34999999999999998},"min":{"1min":0.27400000000000002,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.60399999999999998,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.307}]},{"osd":2,"last update":"Fri Feb 23 00:41:01 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.39200000000000002},"min":{"1min":0.28799999999999998,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.56799999999999995,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39300000000000002,"15min":0.38},"min":{"1min":0.26200000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.62,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.27400000000000002}]},{"osd":3,"last update":"Fri Feb 23 00:41:28 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.434,"15min":0.44400000000000001},"min":{"1min":0.28199999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.54400000000000004,"5min":0.754,"15min":0.754},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.48499999999999999,"15min":0.499},"min":{"1min":0.313,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.64700000000000002,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.38600000000000001}]},{"osd":4,"last update":"Fri Feb 23 00:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.65700000000000003,"15min":0.69199999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83099999999999996,"5min":3.2909999999999999,"15min":3.2909999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.69599999999999995,"15min":0.74099999999999999},"min":{"1min":0.42599999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.83699999999999997,"5min":3.319,"15min":3.319},"last":0.45200000000000001}]},{"osd":5,"last update":"Fri Feb 23 00:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59499999999999997,"15min":0.60799999999999998},"min":{"1min":0.42599999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.71999999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.65900000000000003,"15min":0.66900000000000004},"min":{"1min":0.45200000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.73199999999999998,"5min":1.131,"15min":1.131},"last":0.65500000000000003}]},{"osd":6,"last update":"Fri Feb 23 00:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.63300000000000001,"15min":0.63500000000000001},"min":{"1min":0.45000000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.85899999999999999,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.66400000000000003}]},{"osd":7,"last update":"Fri Feb 23 00:41:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.66100000000000003,"15min":0.67300000000000004},"min":{"1min":0.41699999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.67300000000000004,"5min":1.163,"15min":1.163},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.48399999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.75600000000000001,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.63}]}]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297816,"kb_used_data":720,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447336,"statfs":{"total":95995035648,"available":95690072064,"internally_reserved":0,"allocated":737280,"data_stored":375612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:40:54 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.38300000000000001,"15min":0.378},"min":{"1min":0.245,"5min":0.222,"15min":0.222},"max":{"1min":0.57399999999999995,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39800000000000002,"15min":0.38600000000000001},"min":{"1min":0.27100000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.308}]},{"osd":1,"last update":"Fri Feb 23 00:40:54 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39000000000000001,"15min":0.376},"min":{"1min":0.27500000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.79800000000000004,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.38},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41599999999999998,"15min":0.40000000000000002},"min":{"1min":0.28399999999999997,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.60899999999999999,"5min":0.745,"15min":0.745},"last":0.318}]},{"osd":3,"last update":"Fri Feb 23 00:41:25 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.45100000000000001,"15min":0.47999999999999998},"min":{"1min":0.30599999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.622,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.44900000000000001,"15min":0.46600000000000003},"min":{"1min":0.29399999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.53900000000000003,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.33700000000000002}]},{"osd":4,"last update":"Fri Feb 23 00:40:54 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.59699999999999998},"min":{"1min":0.43099999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.374,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.63200000000000001}]},{"osd":5,"last update":"Fri Feb 23 00:41:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57699999999999996,"15min":0.58199999999999996},"min":{"1min":0.41799999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.748,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.47099999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0740000000000001,"5min":1.089,"15min":1.089},"last":0.63900000000000001}]},{"osd":6,"last update":"Fri Feb 23 00:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.624,"15min":0.624},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59999999999999998,"15min":0.59799999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.66900000000000004}]},{"osd":7,"last update":"Fri Feb 23 00:41:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.66800000000000004,"15min":0.67200000000000004},"min":{"1min":0.50900000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.64100000000000001,"15min":0.64800000000000002},"min":{"1min":0.42799999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.044,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.60999999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149748,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297848,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447304,"statfs":{"total":95995035648,"available":95690039296,"internally_reserved":0,"allocated":770048,"data_stored":376159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:41:29 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.41599999999999998,"15min":0.435},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.60999999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.43099999999999999,"15min":0.45300000000000001},"min":{"1min":0.28100000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.52900000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.48099999999999998}]},{"osd":1,"last update":"Fri Feb 23 00:41:29 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.45100000000000001,"15min":0.47299999999999998},"min":{"1min":0.23400000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.60399999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.42799999999999999,"15min":0.44800000000000001},"min":{"1min":0.28699999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.51100000000000001,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.46000000000000002}]},{"osd":2,"last update":"Fri Feb 23 00:41:29 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.47499999999999998,"15min":0.49199999999999999},"min":{"1min":0.32900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.45900000000000002,"15min":0.48099999999999998},"min":{"1min":0.30599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64700000000000002,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.57199999999999995}]},{"osd":4,"last update":"Fri Feb 23 00:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.436,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.79400000000000004,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.58199999999999996,"15min":0.58999999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.83999999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.85499999999999998}]},{"osd":5,"last update":"Fri Feb 23 00:41:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.59099999999999997,"15min":0.60299999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74099999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.61299999999999999,"15min":0.623},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85299999999999998,"5min":1.024,"15min":1.024},"last":0.82299999999999995}]},{"osd":6,"last update":"Fri Feb 23 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.59999999999999998,"15min":0.60799999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.75,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.61799999999999999,"15min":0.624},"min":{"1min":0.46300000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.77400000000000002,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.88}]},{"osd":7,"last update":"Fri Feb 23 00:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63400000000000001,"15min":0.63600000000000001},"min":{"1min":0.433,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.64900000000000002,"15min":0.65500000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.77300000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.81000000000000005}]}]},{"osd":4,"up_from":32,"seq":137438953520,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297832,"kb_used_data":736,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447320,"statfs":{"total":95995035648,"available":95690055680,"internally_reserved":0,"allocated":753664,"data_stored":375857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55600000000000005,"15min":0.57999999999999996},"min":{"1min":0.32200000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66300000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56799999999999995,"15min":0.58399999999999996},"min":{"1min":0.35699999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71199999999999997,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.68799999999999994}]},{"osd":1,"last update":"Fri Feb 23 00:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.56100000000000005,"15min":0.58299999999999996},"min":{"1min":0.33000000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.75700000000000001,"5min":1.175,"15min":1.175},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56699999999999995,"15min":0.57499999999999996},"min":{"1min":0.32600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.69999999999999996,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.54800000000000004}]},{"osd":2,"last update":"Fri Feb 23 00:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56999999999999995,"15min":0.57699999999999996},"min":{"1min":0.41399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.753,"5min":1.161,"15min":1.161},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.57399999999999995,"15min":0.58799999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79000000000000004,"5min":1.113,"15min":1.113},"last":0.85999999999999999}]},{"osd":3,"last update":"Fri Feb 23 00:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57399999999999995,"15min":0.57999999999999996},"min":{"1min":0.38200000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81999999999999995,"5min":1.143,"15min":1.143},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.57199999999999995,"15min":0.58199999999999996},"min":{"1min":0.36199999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79100000000000004,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.67200000000000004}]},{"osd":5,"last update":"Fri Feb 23 00:41:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.39600000000000002,"15min":0.371},"min":{"1min":0.251,"5min":0.246,"15min":0.246},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.44500000000000001,"15min":0.41599999999999998},"min":{"1min":0.29999999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.80800000000000005}]},{"osd":6,"last update":"Fri Feb 23 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.40200000000000002,"15min":0.39700000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.40300000000000002,"15min":0.39600000000000002},"min":{"1min":0.30499999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.59499999999999997}]},{"osd":7,"last update":"Fri Feb 23 00:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.48999999999999999,"15min":0.48399999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.90200000000000002}]}]},{"osd":5,"up_from":39,"seq":167503724587,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297836,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447316,"statfs":{"total":95995035648,"available":95690051584,"internally_reserved":0,"allocated":757760,"data_stored":376991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:41:17 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.50800000000000001,"15min":0.50900000000000001},"min":{"1min":0.32200000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.67600000000000005,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.51800000000000002,"15min":0.52300000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64700000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri Feb 23 00:41:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.65800000000000003,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.53300000000000003,"15min":0.54100000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68500000000000005,"5min":1.004,"15min":1.004},"last":0.46300000000000002}]},{"osd":2,"last update":"Fri Feb 23 00:41:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.53900000000000003,"15min":0.54700000000000004},"min":{"1min":0.38,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.70999999999999996,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.36099999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69799999999999995,"5min":1.026,"15min":1.026},"last":0.44600000000000001}]},{"osd":3,"last update":"Fri Feb 23 00:41:17 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.52500000000000002,"15min":0.53000000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.629,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55800000000000005,"15min":0.55900000000000005},"min":{"1min":0.41699999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72199999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.53900000000000003}]},{"osd":4,"last update":"Fri Feb 23 00:41:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.39600000000000002,"15min":0.38300000000000001},"min":{"1min":0.27100000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.63700000000000001,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.41299999999999998,"15min":0.39000000000000001},"min":{"1min":0.32400000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.56100000000000005}]},{"osd":6,"last update":"Fri Feb 23 00:40:39 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.42699999999999999,"15min":0.42499999999999999},"min":{"1min":0.313,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.629},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.44900000000000001,"15min":0.441},"min":{"1min":0.314,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.61899999999999999}]},{"osd":7,"last update":"Fri Feb 23 00:41:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51500000000000001,"15min":0.51100000000000001},"min":{"1min":0.34300000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72299999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.55300000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-23T00:41:34.042 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-23T00:41:34.042 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-23T00:41:34.042 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-23T00:41:34.043 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph health --format=json 2024-02-23T00:41:34.325 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:41:34.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:33.062463+0000 osd.4 (osd.4) 45 : cluster [DBG] 4.18 scrub starts 2024-02-23T00:41:34.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:33.063984+0000 osd.4 (osd.4) 46 : cluster [DBG] 4.18 scrub ok 2024-02-23T00:41:34.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:33.150017+0000 mgr.y (mgr.14517) 92 : cluster [DBG] pgmap v79: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:41:34.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:41:33.302075+0000 mgr.y (mgr.14517) 93 : audit [DBG] from='client.24608 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:41:34.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:33.992153+0000 osd.6 (osd.6) 29 : cluster [DBG] 2.14 scrub starts 2024-02-23T00:41:34.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:33.993066+0000 osd.6 (osd.6) 30 : cluster [DBG] 2.14 scrub ok 2024-02-23T00:41:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:33.999838+0000 osd.3 (osd.3) 47 : cluster [DBG] 6.1b scrub starts 2024-02-23T00:41:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:34.000837+0000 osd.3 (osd.3) 48 : cluster [DBG] 6.1b scrub ok 2024-02-23T00:41:34.738 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:33.062463+0000 osd.4 (osd.4) 45 : cluster [DBG] 4.18 scrub starts 2024-02-23T00:41:34.738 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:33.063984+0000 osd.4 (osd.4) 46 : cluster [DBG] 4.18 scrub ok 2024-02-23T00:41:34.738 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:33.150017+0000 mgr.y (mgr.14517) 92 : cluster [DBG] pgmap v79: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:41:34.738 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:41:33.302075+0000 mgr.y (mgr.14517) 93 : audit [DBG] from='client.24608 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:41:34.738 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:33.992153+0000 osd.6 (osd.6) 29 : cluster [DBG] 2.14 scrub starts 2024-02-23T00:41:34.738 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:33.993066+0000 osd.6 (osd.6) 30 : cluster [DBG] 2.14 scrub ok 2024-02-23T00:41:34.738 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:33.999838+0000 osd.3 (osd.3) 47 : cluster [DBG] 6.1b scrub starts 2024-02-23T00:41:34.738 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:34.000837+0000 osd.3 (osd.3) 48 : cluster [DBG] 6.1b scrub ok 2024-02-23T00:41:34.738 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:33.062463+0000 osd.4 (osd.4) 45 : cluster [DBG] 4.18 scrub starts 2024-02-23T00:41:34.739 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:33.063984+0000 osd.4 (osd.4) 46 : cluster [DBG] 4.18 scrub ok 2024-02-23T00:41:34.739 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:33.150017+0000 mgr.y (mgr.14517) 92 : cluster [DBG] pgmap v79: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:41:34.739 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:41:33.302075+0000 mgr.y (mgr.14517) 93 : audit [DBG] from='client.24608 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:41:34.739 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:33.992153+0000 osd.6 (osd.6) 29 : cluster [DBG] 2.14 scrub starts 2024-02-23T00:41:34.739 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:33.993066+0000 osd.6 (osd.6) 30 : cluster [DBG] 2.14 scrub ok 2024-02-23T00:41:34.739 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:33.999838+0000 osd.3 (osd.3) 47 : cluster [DBG] 6.1b scrub starts 2024-02-23T00:41:34.739 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:34.000837+0000 osd.3 (osd.3) 48 : cluster [DBG] 6.1b scrub ok 2024-02-23T00:41:35.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:33.899189+0000 osd.0 (osd.0) 39 : cluster [DBG] 4.1e scrub starts 2024-02-23T00:41:35.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:33.901017+0000 osd.0 (osd.0) 40 : cluster [DBG] 4.1e scrub ok 2024-02-23T00:41:35.804 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:33.899189+0000 osd.0 (osd.0) 39 : cluster [DBG] 4.1e scrub starts 2024-02-23T00:41:35.804 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:33.901017+0000 osd.0 (osd.0) 40 : cluster [DBG] 4.1e scrub ok 2024-02-23T00:41:35.804 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:33.899189+0000 osd.0 (osd.0) 39 : cluster [DBG] 4.1e scrub starts 2024-02-23T00:41:35.804 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:33.901017+0000 osd.0 (osd.0) 40 : cluster [DBG] 4.1e scrub ok 2024-02-23T00:41:36.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.352+0000 7fbd7ffe7700 1 -- 172.21.15.47:0/153121635 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbd780fd510 msgr2=0x7fbd780f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:36.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.352+0000 7fbd7ffe7700 1 --2- 172.21.15.47:0/153121635 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbd780fd510 0x7fbd780f6d40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fbd6c004870 tx=0x7fbd6c009cb0).stop 2024-02-23T00:41:36.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.352+0000 7fbd7ffe7700 1 -- 172.21.15.47:0/153121635 shutdown_connections 2024-02-23T00:41:36.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.352+0000 7fbd7ffe7700 1 --2- 172.21.15.47:0/153121635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd780f7400 0x7fbd78103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:36.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.352+0000 7fbd7ffe7700 1 --2- 172.21.15.47:0/153121635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd780fcbb0 0x7fbd780fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:36.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.352+0000 7fbd7ffe7700 1 --2- 172.21.15.47:0/153121635 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbd780fd510 0x7fbd780f6d40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:36.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.352+0000 7fbd7ffe7700 1 -- 172.21.15.47:0/153121635 >> 172.21.15.47:0/153121635 conn(0x7fbd780f2820 msgr2=0x7fbd780f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:41:36.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.352+0000 7fbd7ffe7700 1 -- 172.21.15.47:0/153121635 shutdown_connections 2024-02-23T00:41:36.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.352+0000 7fbd7ffe7700 1 -- 172.21.15.47:0/153121635 wait complete. 2024-02-23T00:41:36.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.353+0000 7fbd7ffe7700 1 librados: starting msgr at 2024-02-23T00:41:36.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.353+0000 7fbd7ffe7700 1 librados: starting objecter 2024-02-23T00:41:36.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.353+0000 7fbd7ffe7700 1 Processor -- start 2024-02-23T00:41:36.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.353+0000 7fbd7ffe7700 1 -- start start 2024-02-23T00:41:36.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.353+0000 7fbd7ffe7700 1 librados: setting wanted keys 2024-02-23T00:41:36.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.353+0000 7fbd7ffe7700 1 librados: calling monclient init 2024-02-23T00:41:36.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.353+0000 7fbd7ffe7700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbd780f7400 0x7fbd780f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:36.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.353+0000 7fbd7ffe7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd780fcbb0 0x7fbd780f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:36.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.353+0000 7fbd7ffe7700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd780fd510 0x7fbd780f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:36.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.353+0000 7fbd7ffe7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd781164f0 con 0x7fbd780fcbb0 2024-02-23T00:41:36.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.353+0000 7fbd7ffe7700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd78116660 con 0x7fbd780fd510 2024-02-23T00:41:36.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.353+0000 7fbd7ffe7700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd78116820 con 0x7fbd780f7400 2024-02-23T00:41:36.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.353+0000 7fbd7d582700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd780fcbb0 0x7fbd780f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:36.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.353+0000 7fbd7d582700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd780fcbb0 0x7fbd780f8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41550/0 (socket says 172.21.15.47:41550) 2024-02-23T00:41:36.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.353+0000 7fbd7d582700 1 -- 172.21.15.47:0/17702142 learned_addr learned my addr 172.21.15.47:0/17702142 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:41:36.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.353+0000 7fbd7e584700 1 --2- 172.21.15.47:0/17702142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd780fd510 0x7fbd780f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:36.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.353+0000 7fbd7d582700 1 -- 172.21.15.47:0/17702142 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbd780f7400 msgr2=0x7fbd780f89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:36.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.353+0000 7fbd7d582700 1 --2- 172.21.15.47:0/17702142 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbd780f7400 0x7fbd780f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:41:36.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.353+0000 7fbd7d582700 1 -- 172.21.15.47:0/17702142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd780fd510 msgr2=0x7fbd780f9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:36.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.353+0000 7fbd7d582700 1 --2- 172.21.15.47:0/17702142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd780fd510 0x7fbd780f9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:36.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.353+0000 7fbd7d582700 1 -- 172.21.15.47:0/17702142 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd74005040 con 0x7fbd780fcbb0 2024-02-23T00:41:36.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.353+0000 7fbd7e584700 1 --2- 172.21.15.47:0/17702142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd780fd510 0x7fbd780f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-23T00:41:36.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.354+0000 7fbd7d582700 1 --2- 172.21.15.47:0/17702142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd780fcbb0 0x7fbd780f8ea0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fbd6c009da0 tx=0x7fbd6c009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:36.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.354+0000 7fbd6affd700 10 client.?.objecter ms_handle_connect 0x7fbd780fcbb0 2024-02-23T00:41:36.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.354+0000 7fbd6affd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:41:36.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.354+0000 7fbd6affd700 1 -- 172.21.15.47:0/17702142 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c00e030 con 0x7fbd780fcbb0 2024-02-23T00:41:36.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.354+0000 7fbd6affd700 1 -- 172.21.15.47:0/17702142 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fbd6c0096d0 con 0x7fbd780fcbb0 2024-02-23T00:41:36.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.354+0000 7fbd7ffe7700 1 -- 172.21.15.47:0/17702142 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd6c005040 con 0x7fbd780fcbb0 2024-02-23T00:41:36.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.354+0000 7fbd6affd700 1 -- 172.21.15.47:0/17702142 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c018070 con 0x7fbd780fcbb0 2024-02-23T00:41:36.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.354+0000 7fbd7ffe7700 10 client.14721.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:41:36.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.354+0000 7fbd7ffe7700 1 -- 172.21.15.47:0/17702142 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd78116cc0 con 0x7fbd780fcbb0 2024-02-23T00:41:36.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.354+0000 7fbd7ffe7700 1 librados: init done 2024-02-23T00:41:36.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.355+0000 7fbd7ffe7700 1 -- 172.21.15.47:0/17702142 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd7805fa60 con 0x7fbd780fcbb0 2024-02-23T00:41:36.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.355+0000 7fbd6affd700 1 -- 172.21.15.47:0/17702142 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fbd6c009240 con 0x7fbd780fcbb0 2024-02-23T00:41:36.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.355+0000 7fbd6affd700 1 --2- 172.21.15.47:0/17702142 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbd64062cc0 0x7fbd64065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:36.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.356+0000 7fbd6affd700 1 -- 172.21.15.47:0/17702142 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(93..93 src has 38..93) v4 ==== 7421+0+0 (secure 0 0 0) 0x7fbd6c07e240 con 0x7fbd780fcbb0 2024-02-23T00:41:36.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.356+0000 7fbd6affd700 10 client.14721.objecter ms_dispatch 0x7fbd78001550 osd_map(93..93 src has 38..93) v4 2024-02-23T00:41:36.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.356+0000 7fbd6affd700 3 client.14721.objecter handle_osd_map got epochs [93,93] > 0 2024-02-23T00:41:36.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.356+0000 7fbd6affd700 3 client.14721.objecter handle_osd_map decoding full epoch 93 2024-02-23T00:41:36.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.356+0000 7fbd7dd83700 1 --2- 172.21.15.47:0/17702142 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbd64062cc0 0x7fbd64065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:36.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.356+0000 7fbd6affd700 20 client.14721.objecter dump_active .. 0 homeless 2024-02-23T00:41:36.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.356+0000 7fbd6affd700 10 client.14721.objecter ms_handle_connect 0x7fbd64062cc0 2024-02-23T00:41:36.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.356+0000 7fbd7dd83700 1 --2- 172.21.15.47:0/17702142 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbd64062cc0 0x7fbd64065180 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fbd74009170 tx=0x7fbd74009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:36.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.360+0000 7fbd6affd700 1 -- 172.21.15.47:0/17702142 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fbd6c04dee0 con 0x7fbd780fcbb0 2024-02-23T00:41:36.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.572+0000 7fbd7ffe7700 1 -- 172.21.15.47:0/17702142 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fbd7805ee80 con 0x7fbd780fcbb0 2024-02-23T00:41:36.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.573+0000 7fbd6affd700 1 -- 172.21.15.47:0/17702142 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7fbd6c051500 con 0x7fbd780fcbb0 2024-02-23T00:41:36.575 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:41:36.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.574+0000 7fbd7ffe7700 10 librados: watch_flush enter 2024-02-23T00:41:36.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.574+0000 7fbd7ffe7700 10 librados: watch_flush exit 2024-02-23T00:41:36.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.574+0000 7fbd7ffe7700 20 client.14721.objecter shutdown clearing up homeless session... 2024-02-23T00:41:36.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.574+0000 7fbd7ffe7700 10 client.14721.objecter successfully canceled tick 2024-02-23T00:41:36.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.574+0000 7fbd7ffe7700 1 -- 172.21.15.47:0/17702142 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbd64062cc0 msgr2=0x7fbd64065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:36.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.574+0000 7fbd7ffe7700 1 --2- 172.21.15.47:0/17702142 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbd64062cc0 0x7fbd64065180 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fbd74009170 tx=0x7fbd74009cb0).stop 2024-02-23T00:41:36.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.574+0000 7fbd7ffe7700 1 -- 172.21.15.47:0/17702142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd780fcbb0 msgr2=0x7fbd780f8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:36.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.574+0000 7fbd7ffe7700 1 --2- 172.21.15.47:0/17702142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd780fcbb0 0x7fbd780f8ea0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fbd6c009da0 tx=0x7fbd6c009cb0).stop 2024-02-23T00:41:36.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.574+0000 7fbd7ffe7700 1 -- 172.21.15.47:0/17702142 shutdown_connections 2024-02-23T00:41:36.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.574+0000 7fbd7ffe7700 1 --2- 172.21.15.47:0/17702142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd780fd510 0x7fbd780f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:36.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.574+0000 7fbd7ffe7700 1 --2- 172.21.15.47:0/17702142 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbd64062cc0 0x7fbd64065180 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:36.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.574+0000 7fbd7ffe7700 1 --2- 172.21.15.47:0/17702142 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbd780f7400 0x7fbd780f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:41:36.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.574+0000 7fbd7ffe7700 1 --2- 172.21.15.47:0/17702142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd780fcbb0 0x7fbd780f8ea0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:36.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.574+0000 7fbd7ffe7700 1 -- 172.21.15.47:0/17702142 >> 172.21.15.47:0/17702142 conn(0x7fbd780f2820 msgr2=0x7fbd78060e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:41:36.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.575+0000 7fbd7ffe7700 1 -- 172.21.15.47:0/17702142 shutdown_connections 2024-02-23T00:41:36.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.575+0000 7fbd7ffe7700 1 -- 172.21.15.47:0/17702142 wait complete. 2024-02-23T00:41:36.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:36.575+0000 7fbd7ffe7700 1 librados: shutdown 2024-02-23T00:41:36.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:35.150355+0000 mgr.y (mgr.14517) 94 : cluster [DBG] pgmap v80: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-23T00:41:36.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:35.652186+0000 osd.5 (osd.5) 43 : cluster [DBG] 5.16 scrub starts 2024-02-23T00:41:36.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:35.655205+0000 osd.5 (osd.5) 44 : cluster [DBG] 5.16 scrub ok 2024-02-23T00:41:36.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:35.949674+0000 osd.6 (osd.6) 31 : cluster [DBG] 4.1f deep-scrub starts 2024-02-23T00:41:36.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:35.954510+0000 osd.6 (osd.6) 32 : cluster [DBG] 4.1f deep-scrub ok 2024-02-23T00:41:36.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:35.150355+0000 mgr.y (mgr.14517) 94 : cluster [DBG] pgmap v80: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-23T00:41:36.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:35.652186+0000 osd.5 (osd.5) 43 : cluster [DBG] 5.16 scrub starts 2024-02-23T00:41:36.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:35.655205+0000 osd.5 (osd.5) 44 : cluster [DBG] 5.16 scrub ok 2024-02-23T00:41:36.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:35.949674+0000 osd.6 (osd.6) 31 : cluster [DBG] 4.1f deep-scrub starts 2024-02-23T00:41:36.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:35.954510+0000 osd.6 (osd.6) 32 : cluster [DBG] 4.1f deep-scrub ok 2024-02-23T00:41:36.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:35.150355+0000 mgr.y (mgr.14517) 94 : cluster [DBG] pgmap v80: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-23T00:41:36.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:35.652186+0000 osd.5 (osd.5) 43 : cluster [DBG] 5.16 scrub starts 2024-02-23T00:41:36.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:35.655205+0000 osd.5 (osd.5) 44 : cluster [DBG] 5.16 scrub ok 2024-02-23T00:41:36.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:35.949674+0000 osd.6 (osd.6) 31 : cluster [DBG] 4.1f deep-scrub starts 2024-02-23T00:41:36.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:35.954510+0000 osd.6 (osd.6) 32 : cluster [DBG] 4.1f deep-scrub ok 2024-02-23T00:41:37.415 INFO:teuthology.orchestra.run.smithi047.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-23T00:41:37.416 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-23T00:41:37.416 INFO:tasks.cephadm:Setup complete, yielding 2024-02-23T00:41:37.416 INFO:teuthology.run_tasks:Running task thrashosds... 2024-02-23T00:41:37.426 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-02-23T00:41:37.427 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'pacific-release'}, 'ceph': {'conf': {'client': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mgrc': 20, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon warn on pool no app': False, 'paxos service trim min': 10}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'crush_tunables': 'jewel', 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', '\\(MON_DOWN\\)', '\\(OSDMAP_FLAGS\\)', 'flag\\(s\\) set', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(PG_', '\\(OSD_', '\\(POOL_', '\\(CEPHADM_STRAY_DAEMON\\)', 'PG_', 'degraded', 'backfill', 'mons down', 'OSD_', 'is down', 'acting', 'reached quota', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)'], 'sha1': 'eb66ed921e744301e6863be6353618d63967ea59'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'cephadm': {'cephadm_mode': 'root'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'eb66ed921e744301e6863be6353618d63967ea59'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'workunit': {'branch': 'wip-ksirivad-pacific-release-whitelist', 'sha1': '4f758cd8e8854d1f126da86e254076852bf059a9'}} 2024-02-23T00:41:37.427 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-02-23T00:41:37.427 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-02-23T00:41:37.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:41:36.574186+0000 mon.a (mon.0) 754 : audit [DBG] from='client.? 172.21.15.47:0/17702142' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-23T00:41:37.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:37.163938+0000 osd.2 (osd.2) 35 : cluster [DBG] 2.10 scrub starts 2024-02-23T00:41:37.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:37.165346+0000 osd.2 (osd.2) 36 : cluster [DBG] 2.10 scrub ok 2024-02-23T00:41:37.709 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:41:37.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:41:36.574186+0000 mon.a (mon.0) 754 : audit [DBG] from='client.? 172.21.15.47:0/17702142' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-23T00:41:37.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:37.163938+0000 osd.2 (osd.2) 35 : cluster [DBG] 2.10 scrub starts 2024-02-23T00:41:37.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:37.165346+0000 osd.2 (osd.2) 36 : cluster [DBG] 2.10 scrub ok 2024-02-23T00:41:37.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:41:36.574186+0000 mon.a (mon.0) 754 : audit [DBG] from='client.? 172.21.15.47:0/17702142' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-23T00:41:37.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:37.163938+0000 osd.2 (osd.2) 35 : cluster [DBG] 2.10 scrub starts 2024-02-23T00:41:37.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:37.165346+0000 osd.2 (osd.2) 36 : cluster [DBG] 2.10 scrub ok 2024-02-23T00:41:38.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:37.150994+0000 mgr.y (mgr.14517) 95 : cluster [DBG] pgmap v81: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:41:38.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:37.150994+0000 mgr.y (mgr.14517) 95 : cluster [DBG] pgmap v81: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:41:38.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:37.150994+0000 mgr.y (mgr.14517) 95 : cluster [DBG] pgmap v81: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:41:39.402 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:38.878823+0000 osd.6 (osd.6) 33 : cluster [DBG] 5.1f deep-scrub starts 2024-02-23T00:41:39.402 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:38.880982+0000 osd.6 (osd.6) 34 : cluster [DBG] 5.1f deep-scrub ok 2024-02-23T00:41:39.402 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:38.948515+0000 osd.3 (osd.3) 49 : cluster [DBG] 4.14 scrub starts 2024-02-23T00:41:39.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.660+0000 7f07cd249700 1 -- 172.21.15.47:0/3059671774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07c80fe510 msgr2=0x7f07c80f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:39.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.660+0000 7f07cd249700 1 --2- 172.21.15.47:0/3059671774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07c80fe510 0x7f07c80f7d40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f07b00047b0 tx=0x7f07b0009cb0).stop 2024-02-23T00:41:39.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.660+0000 7f07cd249700 1 -- 172.21.15.47:0/3059671774 shutdown_connections 2024-02-23T00:41:39.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.660+0000 7f07cd249700 1 --2- 172.21.15.47:0/3059671774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07c80f8420 0x7f07c8104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:39.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.660+0000 7f07cd249700 1 --2- 172.21.15.47:0/3059671774 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f07c80fdbb0 0x7f07c80fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:39.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.660+0000 7f07cd249700 1 --2- 172.21.15.47:0/3059671774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07c80fe510 0x7f07c80f7d40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:39.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.660+0000 7f07cd249700 1 -- 172.21.15.47:0/3059671774 >> 172.21.15.47:0/3059671774 conn(0x7f07c80f3800 msgr2=0x7f07c80f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:41:39.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.660+0000 7f07cd249700 1 -- 172.21.15.47:0/3059671774 shutdown_connections 2024-02-23T00:41:39.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.660+0000 7f07cd249700 1 -- 172.21.15.47:0/3059671774 wait complete. 2024-02-23T00:41:39.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.661+0000 7f07cd249700 1 librados: starting msgr at 2024-02-23T00:41:39.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.661+0000 7f07cd249700 1 librados: starting objecter 2024-02-23T00:41:39.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.661+0000 7f07cd249700 1 Processor -- start 2024-02-23T00:41:39.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.661+0000 7f07cd249700 1 -- start start 2024-02-23T00:41:39.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.661+0000 7f07cd249700 1 librados: setting wanted keys 2024-02-23T00:41:39.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.661+0000 7f07cd249700 1 librados: calling monclient init 2024-02-23T00:41:39.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.661+0000 7f07cd249700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07c80f8420 0x7f07c810df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:39.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.661+0000 7f07cd249700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07c80fdbb0 0x7f07c810e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:39.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.661+0000 7f07cd249700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f07c80fe510 0x7f07c81080d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:39.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.661+0000 7f07cd249700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c81085c0 con 0x7f07c80f8420 2024-02-23T00:41:39.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.661+0000 7f07cd249700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c8108730 con 0x7f07c80fdbb0 2024-02-23T00:41:39.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.661+0000 7f07cd249700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07c81088f0 con 0x7f07c80fe510 2024-02-23T00:41:39.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.662+0000 7f07c6d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07c80f8420 0x7f07c810df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:39.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.662+0000 7f07c759e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f07c80fe510 0x7f07c81080d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:39.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.662+0000 7f07c6d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07c80f8420 0x7f07c810df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42592/0 (socket says 172.21.15.47:42592) 2024-02-23T00:41:39.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.662+0000 7f07c6d9d700 1 -- 172.21.15.47:0/151799082 learned_addr learned my addr 172.21.15.47:0/151799082 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:41:39.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.662+0000 7f07c659c700 1 --2- 172.21.15.47:0/151799082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07c80fdbb0 0x7f07c810e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:39.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.662+0000 7f07c6d9d700 1 -- 172.21.15.47:0/151799082 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f07c80fe510 msgr2=0x7f07c81080d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:39.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.662+0000 7f07c6d9d700 1 --2- 172.21.15.47:0/151799082 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f07c80fe510 0x7f07c81080d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:39.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.662+0000 7f07c6d9d700 1 -- 172.21.15.47:0/151799082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07c80fdbb0 msgr2=0x7f07c810e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:39.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.662+0000 7f07c6d9d700 1 --2- 172.21.15.47:0/151799082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07c80fdbb0 0x7f07c810e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:39.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.662+0000 7f07c6d9d700 1 -- 172.21.15.47:0/151799082 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b8005040 con 0x7f07c80f8420 2024-02-23T00:41:39.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.662+0000 7f07c6d9d700 1 --2- 172.21.15.47:0/151799082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07c80f8420 0x7f07c810df30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f07b8009400 tx=0x7f07b8009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:39.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.662+0000 7f07bffff700 10 client.?.objecter ms_handle_connect 0x7f07c80f8420 2024-02-23T00:41:39.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.662+0000 7f07bffff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:41:39.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.662+0000 7f07bffff700 1 -- 172.21.15.47:0/151799082 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b8007a40 con 0x7f07c80f8420 2024-02-23T00:41:39.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.662+0000 7f07cd249700 1 -- 172.21.15.47:0/151799082 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07b0005040 con 0x7f07c80f8420 2024-02-23T00:41:39.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.662+0000 7f07bffff700 1 -- 172.21.15.47:0/151799082 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f07b800f040 con 0x7f07c80f8420 2024-02-23T00:41:39.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.662+0000 7f07bffff700 1 -- 172.21.15.47:0/151799082 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b8014900 con 0x7f07c80f8420 2024-02-23T00:41:39.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.662+0000 7f07cd249700 10 client.14727.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:41:39.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.662+0000 7f07cd249700 1 -- 172.21.15.47:0/151799082 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07c8108f30 con 0x7f07c80f8420 2024-02-23T00:41:39.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.662+0000 7f07cd249700 1 librados: init done 2024-02-23T00:41:39.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.663+0000 7f07cd249700 1 -- 172.21.15.47:0/151799082 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07c805ee80 con 0x7f07c80f8420 2024-02-23T00:41:39.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.664+0000 7f07bffff700 1 -- 172.21.15.47:0/151799082 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f07b8014d30 con 0x7f07c80f8420 2024-02-23T00:41:39.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.664+0000 7f07bffff700 1 --2- 172.21.15.47:0/151799082 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f07b4062cc0 0x7f07b4065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:39.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.664+0000 7f07bffff700 1 -- 172.21.15.47:0/151799082 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(94..94 src has 38..94) v4 ==== 7421+0+0 (secure 0 0 0) 0x7f07b807cad0 con 0x7f07c80f8420 2024-02-23T00:41:39.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.664+0000 7f07bffff700 10 client.14727.objecter ms_dispatch 0x7f07c8001550 osd_map(94..94 src has 38..94) v4 2024-02-23T00:41:39.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.664+0000 7f07bffff700 3 client.14727.objecter handle_osd_map got epochs [94,94] > 0 2024-02-23T00:41:39.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.664+0000 7f07bffff700 3 client.14727.objecter handle_osd_map decoding full epoch 94 2024-02-23T00:41:39.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.664+0000 7f07c659c700 1 --2- 172.21.15.47:0/151799082 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f07b4062cc0 0x7f07b4065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:39.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.664+0000 7f07bffff700 20 client.14727.objecter dump_active .. 0 homeless 2024-02-23T00:41:39.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.664+0000 7f07bffff700 10 client.14727.objecter ms_handle_connect 0x7f07b4062cc0 2024-02-23T00:41:39.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.664+0000 7f07c659c700 1 --2- 172.21.15.47:0/151799082 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f07b4062cc0 0x7f07b4065180 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f07c8109870 tx=0x7f07b0009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:39.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.668+0000 7f07bffff700 1 -- 172.21.15.47:0/151799082 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f07b8049f60 con 0x7f07c80f8420 2024-02-23T00:41:39.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:38.878823+0000 osd.6 (osd.6) 33 : cluster [DBG] 5.1f deep-scrub starts 2024-02-23T00:41:39.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:38.880982+0000 osd.6 (osd.6) 34 : cluster [DBG] 5.1f deep-scrub ok 2024-02-23T00:41:39.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:38.948515+0000 osd.3 (osd.3) 49 : cluster [DBG] 4.14 scrub starts 2024-02-23T00:41:39.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:38.950530+0000 osd.3 (osd.3) 50 : cluster [DBG] 4.14 scrub ok 2024-02-23T00:41:39.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:38.950530+0000 osd.3 (osd.3) 50 : cluster [DBG] 4.14 scrub ok 2024-02-23T00:41:39.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:38.878823+0000 osd.6 (osd.6) 33 : cluster [DBG] 5.1f deep-scrub starts 2024-02-23T00:41:39.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:38.880982+0000 osd.6 (osd.6) 34 : cluster [DBG] 5.1f deep-scrub ok 2024-02-23T00:41:39.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:38.948515+0000 osd.3 (osd.3) 49 : cluster [DBG] 4.14 scrub starts 2024-02-23T00:41:39.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:38.950530+0000 osd.3 (osd.3) 50 : cluster [DBG] 4.14 scrub ok 2024-02-23T00:41:39.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.843+0000 7f07cd249700 1 -- 172.21.15.47:0/151799082 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mgr_debug_aggressive_pg_num_changes}] v 0) v1 -- 0x7f07c805fa60 con 0x7f07c80f8420 2024-02-23T00:41:39.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.847+0000 7f07bffff700 1 -- 172.21.15.47:0/151799082 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr_debug_aggressive_pg_num_changes}]=0 v27)=0 v27) v1 ==== 136+0+0 (secure 0 0 0) 0x7f07b804c830 con 0x7f07c80f8420 2024-02-23T00:41:39.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.848+0000 7f07cd249700 10 librados: watch_flush enter 2024-02-23T00:41:39.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.848+0000 7f07cd249700 10 librados: watch_flush exit 2024-02-23T00:41:39.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.848+0000 7f07cd249700 20 client.14727.objecter shutdown clearing up homeless session... 2024-02-23T00:41:39.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.848+0000 7f07cd249700 10 client.14727.objecter successfully canceled tick 2024-02-23T00:41:39.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.849+0000 7f07cd249700 1 -- 172.21.15.47:0/151799082 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f07b4062cc0 msgr2=0x7f07b4065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:39.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.849+0000 7f07cd249700 1 --2- 172.21.15.47:0/151799082 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f07b4062cc0 0x7f07b4065180 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f07c8109870 tx=0x7f07b0009cb0).stop 2024-02-23T00:41:39.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.849+0000 7f07cd249700 1 -- 172.21.15.47:0/151799082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07c80f8420 msgr2=0x7f07c810df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:39.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.849+0000 7f07cd249700 1 --2- 172.21.15.47:0/151799082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07c80f8420 0x7f07c810df30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f07b8009400 tx=0x7f07b8009cb0).stop 2024-02-23T00:41:39.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.849+0000 7f07cd249700 1 -- 172.21.15.47:0/151799082 shutdown_connections 2024-02-23T00:41:39.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.849+0000 7f07cd249700 1 --2- 172.21.15.47:0/151799082 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f07b4062cc0 0x7f07b4065180 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:39.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.849+0000 7f07cd249700 1 --2- 172.21.15.47:0/151799082 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f07c80fe510 0x7f07c81080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:39.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.849+0000 7f07cd249700 1 --2- 172.21.15.47:0/151799082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07c80f8420 0x7f07c810df30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:39.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.849+0000 7f07cd249700 1 --2- 172.21.15.47:0/151799082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07c80fdbb0 0x7f07c810e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:39.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.849+0000 7f07cd249700 1 -- 172.21.15.47:0/151799082 >> 172.21.15.47:0/151799082 conn(0x7f07c80f3800 msgr2=0x7f07c80fb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:41:39.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.849+0000 7f07cd249700 1 -- 172.21.15.47:0/151799082 shutdown_connections 2024-02-23T00:41:39.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.849+0000 7f07cd249700 1 -- 172.21.15.47:0/151799082 wait complete. 2024-02-23T00:41:39.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:39.849+0000 7f07cd249700 1 librados: shutdown 2024-02-23T00:41:40.236 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:41:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:41:39] ENGINE Bus STOPPING 2024-02-23T00:41:40.236 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:41:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:41:39] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-23T00:41:40.236 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:41:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:41:39] ENGINE Bus STOPPED 2024-02-23T00:41:40.236 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:41:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:41:39] ENGINE Bus STARTING 2024-02-23T00:41:40.236 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:41:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:41:39] ENGINE Serving on http://:::9283 2024-02-23T00:41:40.236 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:41:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:41:39] ENGINE Bus STARTED 2024-02-23T00:41:40.556 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:41:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:41:40] "GET /metrics HTTP/1.1" 200 172154 "" "Prometheus/2.33.4" 2024-02-23T00:41:40.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:39.151477+0000 mgr.y (mgr.14517) 96 : cluster [DBG] pgmap v82: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:41:40.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:39.290832+0000 osd.5 (osd.5) 45 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:41:40.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:39.290880+0000 osd.5 (osd.5) 46 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:41:40.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:39.403244+0000 mon.a (mon.0) 755 : cluster [DBG] osdmap e94: 8 total, 8 up, 8 in 2024-02-23T00:41:40.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:39.680549+0000 osd.1 (osd.1) 49 : cluster [DBG] 4.f scrub starts 2024-02-23T00:41:40.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:39.682175+0000 osd.1 (osd.1) 50 : cluster [DBG] 4.f scrub ok 2024-02-23T00:41:40.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:41:39.848073+0000 mon.a (mon.0) 756 : audit [INF] from='client.? 172.21.15.47:0/151799082' entity='client.admin' 2024-02-23T00:41:40.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:41:39.900589+0000 mon.a (mon.0) 757 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:41:40.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:40.127500+0000 osd.2 (osd.2) 37 : cluster [DBG] 2.b scrub starts 2024-02-23T00:41:40.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:40.128393+0000 osd.2 (osd.2) 38 : cluster [DBG] 2.b scrub ok 2024-02-23T00:41:40.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:39.151477+0000 mgr.y (mgr.14517) 96 : cluster [DBG] pgmap v82: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:41:40.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:39.290832+0000 osd.5 (osd.5) 45 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:41:40.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:39.290880+0000 osd.5 (osd.5) 46 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:41:40.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:39.403244+0000 mon.a (mon.0) 755 : cluster [DBG] osdmap e94: 8 total, 8 up, 8 in 2024-02-23T00:41:40.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:39.680549+0000 osd.1 (osd.1) 49 : cluster [DBG] 4.f scrub starts 2024-02-23T00:41:40.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:39.682175+0000 osd.1 (osd.1) 50 : cluster [DBG] 4.f scrub ok 2024-02-23T00:41:40.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:41:39.848073+0000 mon.a (mon.0) 756 : audit [INF] from='client.? 172.21.15.47:0/151799082' entity='client.admin' 2024-02-23T00:41:40.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:41:39.900589+0000 mon.a (mon.0) 757 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:41:40.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:40.127500+0000 osd.2 (osd.2) 37 : cluster [DBG] 2.b scrub starts 2024-02-23T00:41:40.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:40.128393+0000 osd.2 (osd.2) 38 : cluster [DBG] 2.b scrub ok 2024-02-23T00:41:40.612 INFO:tasks.thrashosds:Beginning thrashosds... 2024-02-23T00:41:40.612 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-23T00:41:40.612 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph pg dump --format=json 2024-02-23T00:41:40.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:39.151477+0000 mgr.y (mgr.14517) 96 : cluster [DBG] pgmap v82: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:41:40.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:39.290832+0000 osd.5 (osd.5) 45 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:41:40.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:39.290880+0000 osd.5 (osd.5) 46 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:41:40.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:39.403244+0000 mon.a (mon.0) 755 : cluster [DBG] osdmap e94: 8 total, 8 up, 8 in 2024-02-23T00:41:40.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:39.680549+0000 osd.1 (osd.1) 49 : cluster [DBG] 4.f scrub starts 2024-02-23T00:41:40.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:39.682175+0000 osd.1 (osd.1) 50 : cluster [DBG] 4.f scrub ok 2024-02-23T00:41:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:41:39.848073+0000 mon.a (mon.0) 756 : audit [INF] from='client.? 172.21.15.47:0/151799082' entity='client.admin' 2024-02-23T00:41:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:41:39.900589+0000 mon.a (mon.0) 757 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:41:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:40.127500+0000 osd.2 (osd.2) 37 : cluster [DBG] 2.b scrub starts 2024-02-23T00:41:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:40.128393+0000 osd.2 (osd.2) 38 : cluster [DBG] 2.b scrub ok 2024-02-23T00:41:41.516 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:41:41.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:39.940996+0000 osd.4 (osd.4) 47 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:41:41.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:39.941055+0000 osd.4 (osd.4) 48 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:41:41.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:40.849532+0000 mon.a (mon.0) 758 : cluster [DBG] osdmap e95: 8 total, 8 up, 8 in 2024-02-23T00:41:41.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:40.923666+0000 osd.7 (osd.7) 45 : cluster [DBG] 2.1d scrub starts 2024-02-23T00:41:41.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:40.926847+0000 osd.7 (osd.7) 46 : cluster [DBG] 2.1d scrub ok 2024-02-23T00:41:41.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:40.940496+0000 osd.3 (osd.3) 51 : cluster [DBG] 6.13 scrub starts 2024-02-23T00:41:41.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:40.941376+0000 osd.3 (osd.3) 52 : cluster [DBG] 6.13 scrub ok 2024-02-23T00:41:41.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:41.119137+0000 osd.2 (osd.2) 39 : cluster [DBG] 4.1c scrub starts 2024-02-23T00:41:41.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:41.120935+0000 osd.2 (osd.2) 40 : cluster [DBG] 4.1c scrub ok 2024-02-23T00:41:41.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:41.151799+0000 mgr.y (mgr.14517) 97 : cluster [DBG] pgmap v85: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:41:42.056 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:39.940996+0000 osd.4 (osd.4) 47 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:41:42.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:39.941055+0000 osd.4 (osd.4) 48 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:41:42.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:40.849532+0000 mon.a (mon.0) 758 : cluster [DBG] osdmap e95: 8 total, 8 up, 8 in 2024-02-23T00:41:42.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:40.923666+0000 osd.7 (osd.7) 45 : cluster [DBG] 2.1d scrub starts 2024-02-23T00:41:42.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:40.926847+0000 osd.7 (osd.7) 46 : cluster [DBG] 2.1d scrub ok 2024-02-23T00:41:42.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:40.940496+0000 osd.3 (osd.3) 51 : cluster [DBG] 6.13 scrub starts 2024-02-23T00:41:42.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:40.941376+0000 osd.3 (osd.3) 52 : cluster [DBG] 6.13 scrub ok 2024-02-23T00:41:42.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:41.119137+0000 osd.2 (osd.2) 39 : cluster [DBG] 4.1c scrub starts 2024-02-23T00:41:42.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:41.120935+0000 osd.2 (osd.2) 40 : cluster [DBG] 4.1c scrub ok 2024-02-23T00:41:42.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:41.151799+0000 mgr.y (mgr.14517) 97 : cluster [DBG] pgmap v85: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:41:42.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:39.940996+0000 osd.4 (osd.4) 47 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:41:42.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:39.941055+0000 osd.4 (osd.4) 48 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:41:42.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:40.849532+0000 mon.a (mon.0) 758 : cluster [DBG] osdmap e95: 8 total, 8 up, 8 in 2024-02-23T00:41:42.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:40.923666+0000 osd.7 (osd.7) 45 : cluster [DBG] 2.1d scrub starts 2024-02-23T00:41:42.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:40.926847+0000 osd.7 (osd.7) 46 : cluster [DBG] 2.1d scrub ok 2024-02-23T00:41:42.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:40.940496+0000 osd.3 (osd.3) 51 : cluster [DBG] 6.13 scrub starts 2024-02-23T00:41:42.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:40.941376+0000 osd.3 (osd.3) 52 : cluster [DBG] 6.13 scrub ok 2024-02-23T00:41:42.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:41.119137+0000 osd.2 (osd.2) 39 : cluster [DBG] 4.1c scrub starts 2024-02-23T00:41:42.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:41.120935+0000 osd.2 (osd.2) 40 : cluster [DBG] 4.1c scrub ok 2024-02-23T00:41:42.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:41.151799+0000 mgr.y (mgr.14517) 97 : cluster [DBG] pgmap v85: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:41:42.527 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:41:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:41:42.853 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:41:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:41:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:41:43.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:41.647540+0000 osd.5 (osd.5) 47 : cluster [DBG] 2.16 scrub starts 2024-02-23T00:41:43.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:41.650431+0000 osd.5 (osd.5) 48 : cluster [DBG] 2.16 scrub ok 2024-02-23T00:41:43.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:41:42.216416+0000 mgr.y (mgr.14517) 98 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:41:43.300 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:41.647540+0000 osd.5 (osd.5) 47 : cluster [DBG] 2.16 scrub starts 2024-02-23T00:41:43.300 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:41.650431+0000 osd.5 (osd.5) 48 : cluster [DBG] 2.16 scrub ok 2024-02-23T00:41:43.300 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:41:42.216416+0000 mgr.y (mgr.14517) 98 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:41:43.300 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:41.647540+0000 osd.5 (osd.5) 47 : cluster [DBG] 2.16 scrub starts 2024-02-23T00:41:43.300 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:41.650431+0000 osd.5 (osd.5) 48 : cluster [DBG] 2.16 scrub ok 2024-02-23T00:41:43.300 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:41:42.216416+0000 mgr.y (mgr.14517) 98 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:41:43.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.750+0000 7f118c22c700 1 -- 172.21.15.47:0/2154154095 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11840f7400 msgr2=0x7f1184103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:43.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.750+0000 7f118c22c700 1 --2- 172.21.15.47:0/2154154095 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11840f7400 0x7f1184103a20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f117c009be0 tx=0x7f117c004d10).stop 2024-02-23T00:41:43.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.750+0000 7f118c22c700 1 -- 172.21.15.47:0/2154154095 shutdown_connections 2024-02-23T00:41:43.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.750+0000 7f118c22c700 1 --2- 172.21.15.47:0/2154154095 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11840f7400 0x7f1184103a20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:43.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.750+0000 7f118c22c700 1 --2- 172.21.15.47:0/2154154095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11840fcbb0 0x7f11840fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:43.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.750+0000 7f118c22c700 1 --2- 172.21.15.47:0/2154154095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11840fd510 0x7f11840f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:43.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.750+0000 7f118c22c700 1 -- 172.21.15.47:0/2154154095 >> 172.21.15.47:0/2154154095 conn(0x7f11840f2820 msgr2=0x7f11840f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:41:43.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.750+0000 7f118c22c700 1 -- 172.21.15.47:0/2154154095 shutdown_connections 2024-02-23T00:41:43.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.750+0000 7f118c22c700 1 -- 172.21.15.47:0/2154154095 wait complete. 2024-02-23T00:41:43.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.751+0000 7f118c22c700 1 librados: starting msgr at 2024-02-23T00:41:43.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.751+0000 7f118c22c700 1 librados: starting objecter 2024-02-23T00:41:43.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.751+0000 7f118c22c700 1 Processor -- start 2024-02-23T00:41:43.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.751+0000 7f118c22c700 1 -- start start 2024-02-23T00:41:43.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.751+0000 7f118c22c700 1 librados: setting wanted keys 2024-02-23T00:41:43.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.751+0000 7f118c22c700 1 librados: calling monclient init 2024-02-23T00:41:43.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.751+0000 7f118c22c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11840f7400 0x7f118410de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:43.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.751+0000 7f118c22c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11840fcbb0 0x7f118410e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:43.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.751+0000 7f118c22c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11840fd510 0x7f1184108030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:43.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.751+0000 7f118c22c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1184108520 con 0x7f11840fcbb0 2024-02-23T00:41:43.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.751+0000 7f118c22c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1184108690 con 0x7f11840fd510 2024-02-23T00:41:43.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.751+0000 7f118c22c700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1184108850 con 0x7f11840f7400 2024-02-23T00:41:43.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.751+0000 7f1189fc8700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11840f7400 0x7f118410de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:43.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.751+0000 7f11897c7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11840fcbb0 0x7f118410e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:43.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.751+0000 7f1189fc8700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11840f7400 0x7f118410de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:36206/0 (socket says 172.21.15.47:36206) 2024-02-23T00:41:43.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.751+0000 7f11897c7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11840fcbb0 0x7f118410e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42612/0 (socket says 172.21.15.47:42612) 2024-02-23T00:41:43.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.751+0000 7f1189fc8700 1 -- 172.21.15.47:0/1071625640 learned_addr learned my addr 172.21.15.47:0/1071625640 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:41:43.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.751+0000 7f118a7c9700 1 --2- 172.21.15.47:0/1071625640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11840fd510 0x7f1184108030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:43.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.752+0000 7f1189fc8700 1 -- 172.21.15.47:0/1071625640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11840fd510 msgr2=0x7f1184108030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:43.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.752+0000 7f1189fc8700 1 --2- 172.21.15.47:0/1071625640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11840fd510 0x7f1184108030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:43.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.752+0000 7f1189fc8700 1 -- 172.21.15.47:0/1071625640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11840fcbb0 msgr2=0x7f118410e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:43.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.752+0000 7f1189fc8700 1 --2- 172.21.15.47:0/1071625640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11840fcbb0 0x7f118410e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:43.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.752+0000 7f1189fc8700 1 -- 172.21.15.47:0/1071625640 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1178005040 con 0x7f11840f7400 2024-02-23T00:41:43.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.752+0000 7f118a7c9700 1 --2- 172.21.15.47:0/1071625640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11840fd510 0x7f1184108030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:41:43.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.752+0000 7f11897c7700 1 --2- 172.21.15.47:0/1071625640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11840fcbb0 0x7f118410e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:41:43.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.752+0000 7f1189fc8700 1 --2- 172.21.15.47:0/1071625640 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11840f7400 0x7f118410de90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f11780048f0 tx=0x7f1178009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:43.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.752+0000 7f1176ffd700 10 client.?.objecter ms_handle_connect 0x7f11840f7400 2024-02-23T00:41:43.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.752+0000 7f1176ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:41:43.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.752+0000 7f1176ffd700 1 -- 172.21.15.47:0/1071625640 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1178007b00 con 0x7f11840f7400 2024-02-23T00:41:43.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.752+0000 7f118c22c700 1 -- 172.21.15.47:0/1071625640 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f117c0097c0 con 0x7f11840f7400 2024-02-23T00:41:43.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.752+0000 7f1176ffd700 1 -- 172.21.15.47:0/1071625640 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f117800f040 con 0x7f11840f7400 2024-02-23T00:41:43.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.752+0000 7f1176ffd700 1 -- 172.21.15.47:0/1071625640 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1178014890 con 0x7f11840f7400 2024-02-23T00:41:43.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.752+0000 7f118c22c700 10 client.24623.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:41:43.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.752+0000 7f118c22c700 1 -- 172.21.15.47:0/1071625640 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1184108e30 con 0x7f11840f7400 2024-02-23T00:41:43.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.752+0000 7f118c22c700 1 librados: init done 2024-02-23T00:41:43.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.753+0000 7f118c22c700 1 -- 172.21.15.47:0/1071625640 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f118405fa60 con 0x7f11840f7400 2024-02-23T00:41:43.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.753+0000 7f1176ffd700 1 -- 172.21.15.47:0/1071625640 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f1178014cc0 con 0x7f11840f7400 2024-02-23T00:41:43.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.754+0000 7f1176ffd700 1 --2- 172.21.15.47:0/1071625640 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1170062cc0 0x7f1170065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:43.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.754+0000 7f1176ffd700 1 -- 172.21.15.47:0/1071625640 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(95..95 src has 38..95) v4 ==== 7421+0+0 (secure 0 0 0) 0x7f117807cb30 con 0x7f11840f7400 2024-02-23T00:41:43.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.754+0000 7f1176ffd700 10 client.24623.objecter ms_dispatch 0x7f1184001550 osd_map(95..95 src has 38..95) v4 2024-02-23T00:41:43.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.754+0000 7f1176ffd700 3 client.24623.objecter handle_osd_map got epochs [95,95] > 0 2024-02-23T00:41:43.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.754+0000 7f1176ffd700 3 client.24623.objecter handle_osd_map decoding full epoch 95 2024-02-23T00:41:43.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.754+0000 7f11897c7700 1 --2- 172.21.15.47:0/1071625640 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1170062cc0 0x7f1170065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:43.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.754+0000 7f1176ffd700 20 client.24623.objecter dump_active .. 0 homeless 2024-02-23T00:41:43.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.754+0000 7f1176ffd700 10 client.24623.objecter ms_handle_connect 0x7f1170062cc0 2024-02-23T00:41:43.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.754+0000 7f11897c7700 1 --2- 172.21.15.47:0/1071625640 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1170062cc0 0x7f1170065180 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f11841097d0 tx=0x7f118000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:43.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.758+0000 7f1176ffd700 1 -- 172.21.15.47:0/1071625640 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f1178049f60 con 0x7f11840f7400 2024-02-23T00:41:43.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.929+0000 7f118c22c700 1 -- 172.21.15.47:0/1071625640 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1184109510 con 0x7f1170062cc0 2024-02-23T00:41:43.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.939+0000 7f1176ffd700 1 -- 172.21.15.47:0/1071625640 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+357096 (secure 0 0 0) 0x7f1184109510 con 0x7f1170062cc0 2024-02-23T00:41:43.946 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:41:43.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.942+0000 7f118c22c700 10 librados: watch_flush enter 2024-02-23T00:41:43.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.942+0000 7f118c22c700 10 librados: watch_flush exit 2024-02-23T00:41:43.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.942+0000 7f118c22c700 20 client.24623.objecter shutdown clearing up homeless session... 2024-02-23T00:41:43.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.942+0000 7f118c22c700 10 client.24623.objecter successfully canceled tick 2024-02-23T00:41:43.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.942+0000 7f118c22c700 1 -- 172.21.15.47:0/1071625640 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1170062cc0 msgr2=0x7f1170065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:43.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.942+0000 7f118c22c700 1 --2- 172.21.15.47:0/1071625640 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1170062cc0 0x7f1170065180 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f11841097d0 tx=0x7f118000b040).stop 2024-02-23T00:41:43.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.942+0000 7f118c22c700 1 -- 172.21.15.47:0/1071625640 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11840f7400 msgr2=0x7f118410de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:43.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.942+0000 7f118c22c700 1 --2- 172.21.15.47:0/1071625640 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11840f7400 0x7f118410de90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f11780048f0 tx=0x7f1178009cb0).stop 2024-02-23T00:41:43.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.942+0000 7f118c22c700 1 -- 172.21.15.47:0/1071625640 shutdown_connections 2024-02-23T00:41:43.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.942+0000 7f118c22c700 1 --2- 172.21.15.47:0/1071625640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11840fd510 0x7f1184108030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:43.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.942+0000 7f118c22c700 1 --2- 172.21.15.47:0/1071625640 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1170062cc0 0x7f1170065180 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:43.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.942+0000 7f118c22c700 1 --2- 172.21.15.47:0/1071625640 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11840f7400 0x7f118410de90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:43.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.942+0000 7f118c22c700 1 --2- 172.21.15.47:0/1071625640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11840fcbb0 0x7f118410e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:43.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.942+0000 7f118c22c700 1 -- 172.21.15.47:0/1071625640 >> 172.21.15.47:0/1071625640 conn(0x7f11840f2820 msgr2=0x7f1184060d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:41:43.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.942+0000 7f118c22c700 1 -- 172.21.15.47:0/1071625640 shutdown_connections 2024-02-23T00:41:43.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.943+0000 7f118c22c700 1 -- 172.21.15.47:0/1071625640 wait complete. 2024-02-23T00:41:43.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:43.943+0000 7f118c22c700 1 librados: shutdown 2024-02-23T00:41:43.950 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-02-23T00:41:43.973 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:42.975792+0000 osd.7 (osd.7) 47 : cluster [DBG] 2.8 scrub starts 2024-02-23T00:41:43.973 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:42.978901+0000 osd.7 (osd.7) 48 : cluster [DBG] 2.8 scrub ok 2024-02-23T00:41:43.973 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:43.152514+0000 mgr.y (mgr.14517) 99 : cluster [DBG] pgmap v86: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:41:43.973 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:42.975792+0000 osd.7 (osd.7) 47 : cluster [DBG] 2.8 scrub starts 2024-02-23T00:41:43.973 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:42.978901+0000 osd.7 (osd.7) 48 : cluster [DBG] 2.8 scrub ok 2024-02-23T00:41:43.974 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:43.152514+0000 mgr.y (mgr.14517) 99 : cluster [DBG] pgmap v86: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:41:44.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:42.975792+0000 osd.7 (osd.7) 47 : cluster [DBG] 2.8 scrub starts 2024-02-23T00:41:44.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:42.978901+0000 osd.7 (osd.7) 48 : cluster [DBG] 2.8 scrub ok 2024-02-23T00:41:44.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:43.152514+0000 mgr.y (mgr.14517) 99 : cluster [DBG] pgmap v86: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:41:45.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:43.872245+0000 osd.4 (osd.4) 49 : cluster [DBG] 2.f deep-scrub starts 2024-02-23T00:41:45.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:41:43.930735+0000 mgr.y (mgr.14517) 100 : audit [DBG] from='client.24623 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:41:45.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:43.998399+0000 osd.7 (osd.7) 49 : cluster [DBG] 6.5 scrub starts 2024-02-23T00:41:45.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:44.001403+0000 osd.7 (osd.7) 50 : cluster [DBG] 6.5 scrub ok 2024-02-23T00:41:45.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:41:44.550669+0000 mon.a (mon.0) 759 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:41:45.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:43.872245+0000 osd.4 (osd.4) 49 : cluster [DBG] 2.f deep-scrub starts 2024-02-23T00:41:45.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:41:43.930735+0000 mgr.y (mgr.14517) 100 : audit [DBG] from='client.24623 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:41:45.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:43.998399+0000 osd.7 (osd.7) 49 : cluster [DBG] 6.5 scrub starts 2024-02-23T00:41:45.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:44.001403+0000 osd.7 (osd.7) 50 : cluster [DBG] 6.5 scrub ok 2024-02-23T00:41:45.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:41:44.550669+0000 mon.a (mon.0) 759 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:41:45.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:43.872245+0000 osd.4 (osd.4) 49 : cluster [DBG] 2.f deep-scrub starts 2024-02-23T00:41:45.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:41:43.930735+0000 mgr.y (mgr.14517) 100 : audit [DBG] from='client.24623 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:41:45.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:43.998399+0000 osd.7 (osd.7) 49 : cluster [DBG] 6.5 scrub starts 2024-02-23T00:41:45.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:44.001403+0000 osd.7 (osd.7) 50 : cluster [DBG] 6.5 scrub ok 2024-02-23T00:41:45.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:41:44.550669+0000 mon.a (mon.0) 759 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:41:45.206 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":86,"stamp":"2024-02-23T00:41:43.151995+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5796,"num_objects":224,"num_object_clones":0,"num_object_copies":672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":0,"num_read":1244,"num_read_kb":964,"num_write":620,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":694,"ondisk_log_size":694,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2382828,"kb_used_data":6060,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":747578388,"statfs":{"total":767960285184,"available":765520269312,"internally_reserved":0,"allocated":6205440,"data_stored":3131460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001825"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.403418+0000","last_change":"2024-02-23T00:41:34.000934+0000","last_active":"2024-02-23T00:41:39.403418+0000","last_peered":"2024-02-23T00:41:39.403418+0000","last_clean":"2024-02-23T00:41:39.403418+0000","last_became_active":"2024-02-23T00:39:21.162083+0000","last_became_peered":"2024-02-23T00:39:21.162083+0000","last_unstale":"2024-02-23T00:41:39.403418+0000","last_undegraded":"2024-02-23T00:41:39.403418+0000","last_fullsized":"2024-02-23T00:41:39.403418+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:34.000844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:34.000844+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.404474+0000","last_change":"2024-02-23T00:41:05.965631+0000","last_active":"2024-02-23T00:41:39.404474+0000","last_peered":"2024-02-23T00:41:39.404474+0000","last_clean":"2024-02-23T00:41:39.404474+0000","last_became_active":"2024-02-23T00:39:18.799723+0000","last_became_peered":"2024-02-23T00:39:18.799723+0000","last_unstale":"2024-02-23T00:41:39.404474+0000","last_undegraded":"2024-02-23T00:41:39.404474+0000","last_fullsized":"2024-02-23T00:41:39.404474+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:05.965531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:05.965531+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679892+0000","last_change":"2024-02-23T00:40:58.873093+0000","last_active":"2024-02-23T00:41:39.679892+0000","last_peered":"2024-02-23T00:41:39.679892+0000","last_clean":"2024-02-23T00:41:39.679892+0000","last_became_active":"2024-02-23T00:39:14.782000+0000","last_became_peered":"2024-02-23T00:39:14.782000+0000","last_unstale":"2024-02-23T00:41:39.679892+0000","last_undegraded":"2024-02-23T00:41:39.679892+0000","last_fullsized":"2024-02-23T00:41:39.679892+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_clean_scrub_stamp":"2024-02-23T00:40:58.873013+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"88'50","reported_seq":143,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.402927+0000","last_change":"2024-02-23T00:40:52.840769+0000","last_active":"2024-02-23T00:41:39.402927+0000","last_peered":"2024-02-23T00:41:39.402927+0000","last_clean":"2024-02-23T00:41:39.402927+0000","last_became_active":"2024-02-23T00:39:16.780144+0000","last_became_peered":"2024-02-23T00:39:16.780144+0000","last_unstale":"2024-02-23T00:41:39.402927+0000","last_undegraded":"2024-02-23T00:41:39.402927+0000","last_fullsized":"2024-02-23T00:41:39.402927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'50","last_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_deep_scrub":"88'50","last_deep_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_clean_scrub_stamp":"2024-02-23T00:40:52.840702+0000","log_size":50,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.404733+0000","last_change":"2024-02-23T00:40:27.096850+0000","last_active":"2024-02-23T00:41:39.404733+0000","last_peered":"2024-02-23T00:41:39.404733+0000","last_clean":"2024-02-23T00:41:39.404733+0000","last_became_active":"2024-02-23T00:39:20.795665+0000","last_became_peered":"2024-02-23T00:39:20.795665+0000","last_unstale":"2024-02-23T00:41:39.404733+0000","last_undegraded":"2024-02-23T00:41:39.404733+0000","last_fullsized":"2024-02-23T00:41:39.404733+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:27.096747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:27.096747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679007+0000","last_change":"2024-02-23T00:40:53.812966+0000","last_active":"2024-02-23T00:41:39.679007+0000","last_peered":"2024-02-23T00:41:39.679007+0000","last_clean":"2024-02-23T00:41:39.679007+0000","last_became_active":"2024-02-23T00:39:18.796312+0000","last_became_peered":"2024-02-23T00:39:18.796312+0000","last_unstale":"2024-02-23T00:41:39.679007+0000","last_undegraded":"2024-02-23T00:41:39.679007+0000","last_fullsized":"2024-02-23T00:41:39.679007+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:53.812893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:53.812893+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.404426+0000","last_change":"2024-02-23T00:41:02.853395+0000","last_active":"2024-02-23T00:41:39.404426+0000","last_peered":"2024-02-23T00:41:39.404426+0000","last_clean":"2024-02-23T00:41:39.404426+0000","last_became_active":"2024-02-23T00:39:14.785362+0000","last_became_peered":"2024-02-23T00:39:14.785362+0000","last_unstale":"2024-02-23T00:41:39.404426+0000","last_undegraded":"2024-02-23T00:41:39.404426+0000","last_fullsized":"2024-02-23T00:41:39.404426+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:02.853328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:02.853328+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"68'15","reported_seq":82,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.404668+0000","last_change":"2024-02-23T00:41:33.064109+0000","last_active":"2024-02-23T00:41:39.404668+0000","last_peered":"2024-02-23T00:41:39.404668+0000","last_clean":"2024-02-23T00:41:39.404668+0000","last_became_active":"2024-02-23T00:39:16.784606+0000","last_became_peered":"2024-02-23T00:39:16.784606+0000","last_unstale":"2024-02-23T00:41:39.404668+0000","last_undegraded":"2024-02-23T00:41:39.404668+0000","last_fullsized":"2024-02-23T00:41:39.404668+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:41:33.063992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:33.063992+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.678957+0000","last_change":"2024-02-23T00:40:55.855032+0000","last_active":"2024-02-23T00:41:39.678957+0000","last_peered":"2024-02-23T00:41:39.678957+0000","last_clean":"2024-02-23T00:41:39.678957+0000","last_became_active":"2024-02-23T00:39:20.793590+0000","last_became_peered":"2024-02-23T00:39:20.793590+0000","last_unstale":"2024-02-23T00:41:39.678957+0000","last_undegraded":"2024-02-23T00:41:39.678957+0000","last_fullsized":"2024-02-23T00:41:39.678957+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:55.854964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:55.854964+0000","last_clean_scrub_stamp":"2024-02-23T00:40:55.854964+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.680092+0000","last_change":"2024-02-23T00:40:42.667373+0000","last_active":"2024-02-23T00:41:39.680092+0000","last_peered":"2024-02-23T00:41:39.680092+0000","last_clean":"2024-02-23T00:41:39.680092+0000","last_became_active":"2024-02-23T00:39:18.796952+0000","last_became_peered":"2024-02-23T00:39:18.796952+0000","last_unstale":"2024-02-23T00:41:39.680092+0000","last_undegraded":"2024-02-23T00:41:39.680092+0000","last_fullsized":"2024-02-23T00:41:39.680092+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:42.667291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:42.667291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":52,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-23T00:41:40.926914+0000","last_change":"2024-02-23T00:41:40.926914+0000","last_active":"2024-02-23T00:41:40.926914+0000","last_peered":"2024-02-23T00:41:40.926914+0000","last_clean":"2024-02-23T00:41:40.926914+0000","last_became_active":"2024-02-23T00:39:14.783469+0000","last_became_peered":"2024-02-23T00:39:14.783469+0000","last_unstale":"2024-02-23T00:41:40.926914+0000","last_undegraded":"2024-02-23T00:41:40.926914+0000","last_fullsized":"2024-02-23T00:41:40.926914+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:40.926851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:40.926851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"82'15","reported_seq":75,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.404811+0000","last_change":"2024-02-23T00:40:31.989101+0000","last_active":"2024-02-23T00:41:39.404811+0000","last_peered":"2024-02-23T00:41:39.404811+0000","last_clean":"2024-02-23T00:41:39.404811+0000","last_became_active":"2024-02-23T00:39:16.782762+0000","last_became_peered":"2024-02-23T00:39:16.782762+0000","last_unstale":"2024-02-23T00:41:39.404811+0000","last_undegraded":"2024-02-23T00:41:39.404811+0000","last_fullsized":"2024-02-23T00:41:39.404811+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'15","last_scrub_stamp":"2024-02-23T00:40:31.989027+0000","last_deep_scrub":"82'15","last_deep_scrub_stamp":"2024-02-23T00:40:31.989027+0000","last_clean_scrub_stamp":"2024-02-23T00:40:31.989027+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.678768+0000","last_change":"2024-02-23T00:41:08.906994+0000","last_active":"2024-02-23T00:41:39.678768+0000","last_peered":"2024-02-23T00:41:39.678768+0000","last_clean":"2024-02-23T00:41:39.678768+0000","last_became_active":"2024-02-23T00:39:20.803344+0000","last_became_peered":"2024-02-23T00:39:20.803344+0000","last_unstale":"2024-02-23T00:41:39.678768+0000","last_undegraded":"2024-02-23T00:41:39.678768+0000","last_fullsized":"2024-02-23T00:41:39.678768+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:08.906928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:08.906928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679403+0000","last_change":"2024-02-23T00:41:07.777516+0000","last_active":"2024-02-23T00:41:39.679403+0000","last_peered":"2024-02-23T00:41:39.679403+0000","last_clean":"2024-02-23T00:41:39.679403+0000","last_became_active":"2024-02-23T00:39:18.796431+0000","last_became_peered":"2024-02-23T00:39:18.796431+0000","last_unstale":"2024-02-23T00:41:39.679403+0000","last_undegraded":"2024-02-23T00:41:39.679403+0000","last_fullsized":"2024-02-23T00:41:39.679403+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:07.777435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:07.777435+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679943+0000","last_change":"2024-02-23T00:40:35.620371+0000","last_active":"2024-02-23T00:41:39.679943+0000","last_peered":"2024-02-23T00:41:39.679943+0000","last_clean":"2024-02-23T00:41:39.679943+0000","last_became_active":"2024-02-23T00:39:14.787148+0000","last_became_peered":"2024-02-23T00:39:14.787148+0000","last_unstale":"2024-02-23T00:41:39.679943+0000","last_undegraded":"2024-02-23T00:41:39.679943+0000","last_fullsized":"2024-02-23T00:41:39.679943+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:35.620292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:35.620292+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"68'21","reported_seq":92,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.404796+0000","last_change":"2024-02-23T00:40:51.950549+0000","last_active":"2024-02-23T00:41:39.404796+0000","last_peered":"2024-02-23T00:41:39.404796+0000","last_clean":"2024-02-23T00:41:39.404796+0000","last_became_active":"2024-02-23T00:39:16.786791+0000","last_became_peered":"2024-02-23T00:39:16.786791+0000","last_unstale":"2024-02-23T00:41:39.404796+0000","last_undegraded":"2024-02-23T00:41:39.404796+0000","last_fullsized":"2024-02-23T00:41:39.404796+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'21","last_scrub_stamp":"2024-02-23T00:40:51.950467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:51.950467+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.404170+0000","last_change":"2024-02-23T00:41:29.992520+0000","last_active":"2024-02-23T00:41:39.404170+0000","last_peered":"2024-02-23T00:41:39.404170+0000","last_clean":"2024-02-23T00:41:39.404170+0000","last_became_active":"2024-02-23T00:39:21.162053+0000","last_became_peered":"2024-02-23T00:39:21.162053+0000","last_unstale":"2024-02-23T00:41:39.404170+0000","last_undegraded":"2024-02-23T00:41:39.404170+0000","last_fullsized":"2024-02-23T00:41:39.404170+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:29.992417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:29.992417+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.405021+0000","last_change":"2024-02-23T00:41:01.063755+0000","last_active":"2024-02-23T00:41:39.405021+0000","last_peered":"2024-02-23T00:41:39.405021+0000","last_clean":"2024-02-23T00:41:39.405021+0000","last_became_active":"2024-02-23T00:39:18.794903+0000","last_became_peered":"2024-02-23T00:39:18.794903+0000","last_unstale":"2024-02-23T00:41:39.405021+0000","last_undegraded":"2024-02-23T00:41:39.405021+0000","last_fullsized":"2024-02-23T00:41:39.405021+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_clean_scrub_stamp":"2024-02-23T00:41:01.063660+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.403550+0000","last_change":"2024-02-23T00:40:19.123225+0000","last_active":"2024-02-23T00:41:39.403550+0000","last_peered":"2024-02-23T00:41:39.403550+0000","last_clean":"2024-02-23T00:41:39.403550+0000","last_became_active":"2024-02-23T00:39:14.782163+0000","last_became_peered":"2024-02-23T00:39:14.782163+0000","last_unstale":"2024-02-23T00:41:39.403550+0000","last_undegraded":"2024-02-23T00:41:39.403550+0000","last_fullsized":"2024-02-23T00:41:39.403550+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:19.123123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:19.123123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"68'18","reported_seq":85,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.403959+0000","last_change":"2024-02-23T00:40:28.118091+0000","last_active":"2024-02-23T00:41:39.403959+0000","last_peered":"2024-02-23T00:41:39.403959+0000","last_clean":"2024-02-23T00:41:39.403959+0000","last_became_active":"2024-02-23T00:39:16.782803+0000","last_became_peered":"2024-02-23T00:39:16.782803+0000","last_unstale":"2024-02-23T00:41:39.403959+0000","last_undegraded":"2024-02-23T00:41:39.403959+0000","last_fullsized":"2024-02-23T00:41:39.403959+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'18","last_scrub_stamp":"2024-02-23T00:40:28.118021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:28.118021+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.405432+0000","last_change":"2024-02-23T00:41:16.048306+0000","last_active":"2024-02-23T00:41:39.405432+0000","last_peered":"2024-02-23T00:41:39.405432+0000","last_clean":"2024-02-23T00:41:39.405432+0000","last_became_active":"2024-02-23T00:39:21.159750+0000","last_became_peered":"2024-02-23T00:39:21.159750+0000","last_unstale":"2024-02-23T00:41:39.405432+0000","last_undegraded":"2024-02-23T00:41:39.405432+0000","last_fullsized":"2024-02-23T00:41:39.405432+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:16.048207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:16.048207+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679934+0000","last_change":"2024-02-23T00:40:23.829090+0000","last_active":"2024-02-23T00:41:39.679934+0000","last_peered":"2024-02-23T00:41:39.679934+0000","last_clean":"2024-02-23T00:41:39.679934+0000","last_became_active":"2024-02-23T00:39:18.801039+0000","last_became_peered":"2024-02-23T00:39:18.801039+0000","last_unstale":"2024-02-23T00:41:39.679934+0000","last_undegraded":"2024-02-23T00:41:39.679934+0000","last_fullsized":"2024-02-23T00:41:39.679934+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:23.829017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:23.829017+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.645352+0000","last_change":"2024-02-23T00:40:30.958705+0000","last_active":"2024-02-23T00:41:39.645352+0000","last_peered":"2024-02-23T00:41:39.645352+0000","last_clean":"2024-02-23T00:41:39.645352+0000","last_became_active":"2024-02-23T00:39:14.782494+0000","last_became_peered":"2024-02-23T00:39:14.782494+0000","last_unstale":"2024-02-23T00:41:39.645352+0000","last_undegraded":"2024-02-23T00:41:39.645352+0000","last_fullsized":"2024-02-23T00:41:39.645352+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:30.958634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:30.958634+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"67'15","reported_seq":76,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-23T00:41:41.121005+0000","last_change":"2024-02-23T00:41:41.121005+0000","last_active":"2024-02-23T00:41:41.121005+0000","last_peered":"2024-02-23T00:41:41.121005+0000","last_clean":"2024-02-23T00:41:41.121005+0000","last_became_active":"2024-02-23T00:39:16.787138+0000","last_became_peered":"2024-02-23T00:39:16.787138+0000","last_unstale":"2024-02-23T00:41:41.121005+0000","last_undegraded":"2024-02-23T00:41:41.121005+0000","last_fullsized":"2024-02-23T00:41:41.121005+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:41:41.120939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:41.120939+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679507+0000","last_change":"2024-02-23T00:40:49.777703+0000","last_active":"2024-02-23T00:41:39.679507+0000","last_peered":"2024-02-23T00:41:39.679507+0000","last_clean":"2024-02-23T00:41:39.679507+0000","last_became_active":"2024-02-23T00:39:20.795524+0000","last_became_peered":"2024-02-23T00:39:20.795524+0000","last_unstale":"2024-02-23T00:41:39.679507+0000","last_undegraded":"2024-02-23T00:41:39.679507+0000","last_fullsized":"2024-02-23T00:41:39.679507+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:49.777636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:49.777636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679788+0000","last_change":"2024-02-23T00:40:21.510727+0000","last_active":"2024-02-23T00:41:39.679788+0000","last_peered":"2024-02-23T00:41:39.679788+0000","last_clean":"2024-02-23T00:41:39.679788+0000","last_became_active":"2024-02-23T00:39:18.790602+0000","last_became_peered":"2024-02-23T00:39:18.790602+0000","last_unstale":"2024-02-23T00:41:39.679788+0000","last_undegraded":"2024-02-23T00:41:39.679788+0000","last_fullsized":"2024-02-23T00:41:39.679788+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:21.510658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:21.510658+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.402729+0000","last_change":"2024-02-23T00:40:44.906676+0000","last_active":"2024-02-23T00:41:39.402729+0000","last_peered":"2024-02-23T00:41:39.402729+0000","last_clean":"2024-02-23T00:41:39.402729+0000","last_became_active":"2024-02-23T00:39:14.781560+0000","last_became_peered":"2024-02-23T00:39:14.781560+0000","last_unstale":"2024-02-23T00:41:39.402729+0000","last_undegraded":"2024-02-23T00:41:39.402729+0000","last_fullsized":"2024-02-23T00:41:39.402729+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:44.906599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:44.906599+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"93'23","reported_seq":91,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.645631+0000","last_change":"2024-02-23T00:41:35.954600+0000","last_active":"2024-02-23T00:41:39.645631+0000","last_peered":"2024-02-23T00:41:39.645631+0000","last_clean":"2024-02-23T00:41:39.645631+0000","last_became_active":"2024-02-23T00:39:16.782660+0000","last_became_peered":"2024-02-23T00:39:16.782660+0000","last_unstale":"2024-02-23T00:41:39.645631+0000","last_undegraded":"2024-02-23T00:41:39.645631+0000","last_fullsized":"2024-02-23T00:41:39.645631+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'23","last_scrub_stamp":"2024-02-23T00:41:35.954516+0000","last_deep_scrub":"93'23","last_deep_scrub_stamp":"2024-02-23T00:41:35.954516+0000","last_clean_scrub_stamp":"2024-02-23T00:41:35.954516+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"67'1","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.680346+0000","last_change":"2024-02-23T00:40:29.596674+0000","last_active":"2024-02-23T00:41:39.680346+0000","last_peered":"2024-02-23T00:41:39.680346+0000","last_clean":"2024-02-23T00:41:39.680346+0000","last_became_active":"2024-02-23T00:39:20.795085+0000","last_became_peered":"2024-02-23T00:39:20.795085+0000","last_unstale":"2024-02-23T00:41:39.680346+0000","last_undegraded":"2024-02-23T00:41:39.680346+0000","last_fullsized":"2024-02-23T00:41:39.680346+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:40:29.596581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:29.596581+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"93'9","reported_seq":90,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.645666+0000","last_change":"2024-02-23T00:41:38.881071+0000","last_active":"2024-02-23T00:41:39.645666+0000","last_peered":"2024-02-23T00:41:39.645666+0000","last_clean":"2024-02-23T00:41:39.645666+0000","last_became_active":"2024-02-23T00:39:18.796680+0000","last_became_peered":"2024-02-23T00:39:18.796680+0000","last_unstale":"2024-02-23T00:41:39.645666+0000","last_undegraded":"2024-02-23T00:41:39.645666+0000","last_fullsized":"2024-02-23T00:41:39.645666+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-02-23T00:41:38.880989+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-02-23T00:41:38.880989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:38.880989+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":61,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.678869+0000","last_change":"2024-02-23T00:41:32.702659+0000","last_active":"2024-02-23T00:41:39.678869+0000","last_peered":"2024-02-23T00:41:39.678869+0000","last_clean":"2024-02-23T00:41:39.678869+0000","last_became_active":"2024-02-23T00:39:14.785363+0000","last_became_peered":"2024-02-23T00:39:14.785363+0000","last_unstale":"2024-02-23T00:41:39.678869+0000","last_undegraded":"2024-02-23T00:41:39.678869+0000","last_fullsized":"2024-02-23T00:41:39.678869+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"80'2","last_scrub_stamp":"2024-02-23T00:41:32.702574+0000","last_deep_scrub":"80'2","last_deep_scrub_stamp":"2024-02-23T00:40:15.942725+0000","last_clean_scrub_stamp":"2024-02-23T00:41:32.702574+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"68'16","reported_seq":77,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679704+0000","last_change":"2024-02-23T00:41:33.901108+0000","last_active":"2024-02-23T00:41:39.679704+0000","last_peered":"2024-02-23T00:41:39.679704+0000","last_clean":"2024-02-23T00:41:39.679704+0000","last_became_active":"2024-02-23T00:39:16.858945+0000","last_became_peered":"2024-02-23T00:39:16.858945+0000","last_unstale":"2024-02-23T00:41:39.679704+0000","last_undegraded":"2024-02-23T00:41:39.679704+0000","last_fullsized":"2024-02-23T00:41:39.679704+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:41:33.901023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:33.901023+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-23T00:41:40.941453+0000","last_change":"2024-02-23T00:41:40.941453+0000","last_active":"2024-02-23T00:41:40.941453+0000","last_peered":"2024-02-23T00:41:40.941453+0000","last_clean":"2024-02-23T00:41:40.941453+0000","last_became_active":"2024-02-23T00:39:21.160376+0000","last_became_peered":"2024-02-23T00:39:21.160376+0000","last_unstale":"2024-02-23T00:41:40.941453+0000","last_undegraded":"2024-02-23T00:41:40.941453+0000","last_fullsized":"2024-02-23T00:41:40.941453+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:40.941380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:40.941380+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679442+0000","last_change":"2024-02-23T00:40:25.581008+0000","last_active":"2024-02-23T00:41:39.679442+0000","last_peered":"2024-02-23T00:41:39.679442+0000","last_clean":"2024-02-23T00:41:39.679442+0000","last_became_active":"2024-02-23T00:39:18.791457+0000","last_became_peered":"2024-02-23T00:39:18.791457+0000","last_unstale":"2024-02-23T00:41:39.679442+0000","last_undegraded":"2024-02-23T00:41:39.679442+0000","last_fullsized":"2024-02-23T00:41:39.679442+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:25.580938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:25.580938+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.645722+0000","last_change":"2024-02-23T00:40:29.977372+0000","last_active":"2024-02-23T00:41:39.645722+0000","last_peered":"2024-02-23T00:41:39.645722+0000","last_clean":"2024-02-23T00:41:39.645722+0000","last_became_active":"2024-02-23T00:39:14.784092+0000","last_became_peered":"2024-02-23T00:39:14.784092+0000","last_unstale":"2024-02-23T00:41:39.645722+0000","last_undegraded":"2024-02-23T00:41:39.645722+0000","last_fullsized":"2024-02-23T00:41:39.645722+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:29.977275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:29.977275+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"68'17","reported_seq":81,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.402882+0000","last_change":"2024-02-23T00:40:20.143358+0000","last_active":"2024-02-23T00:41:39.402882+0000","last_peered":"2024-02-23T00:41:39.402882+0000","last_clean":"2024-02-23T00:41:39.402882+0000","last_became_active":"2024-02-23T00:39:16.857232+0000","last_became_peered":"2024-02-23T00:39:16.857232+0000","last_unstale":"2024-02-23T00:41:39.402882+0000","last_undegraded":"2024-02-23T00:41:39.402882+0000","last_fullsized":"2024-02-23T00:41:39.402882+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:40:20.143288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:20.143288+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679629+0000","last_change":"2024-02-23T00:40:54.739607+0000","last_active":"2024-02-23T00:41:39.679629+0000","last_peered":"2024-02-23T00:41:39.679629+0000","last_clean":"2024-02-23T00:41:39.679629+0000","last_became_active":"2024-02-23T00:39:20.795351+0000","last_became_peered":"2024-02-23T00:39:20.795351+0000","last_unstale":"2024-02-23T00:41:39.679629+0000","last_undegraded":"2024-02-23T00:41:39.679629+0000","last_fullsized":"2024-02-23T00:41:39.679629+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:54.739534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:54.739534+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.645118+0000","last_change":"2024-02-23T00:40:38.957682+0000","last_active":"2024-02-23T00:41:39.645118+0000","last_peered":"2024-02-23T00:41:39.645118+0000","last_clean":"2024-02-23T00:41:39.645118+0000","last_became_active":"2024-02-23T00:39:18.797287+0000","last_became_peered":"2024-02-23T00:39:18.797287+0000","last_unstale":"2024-02-23T00:41:39.645118+0000","last_undegraded":"2024-02-23T00:41:39.645118+0000","last_fullsized":"2024-02-23T00:41:39.645118+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:38.957578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:38.957578+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":54,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-23T00:41:41.650513+0000","last_change":"2024-02-23T00:41:41.650513+0000","last_active":"2024-02-23T00:41:41.650513+0000","last_peered":"2024-02-23T00:41:41.650513+0000","last_clean":"2024-02-23T00:41:41.650513+0000","last_became_active":"2024-02-23T00:39:14.783325+0000","last_became_peered":"2024-02-23T00:39:14.783325+0000","last_unstale":"2024-02-23T00:41:41.650513+0000","last_undegraded":"2024-02-23T00:41:41.650513+0000","last_fullsized":"2024-02-23T00:41:41.650513+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:41.650434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:41.650434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"67'4","reported_seq":53,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.402981+0000","last_change":"2024-02-23T00:41:20.947463+0000","last_active":"2024-02-23T00:41:39.402981+0000","last_peered":"2024-02-23T00:41:39.402981+0000","last_clean":"2024-02-23T00:41:39.402981+0000","last_became_active":"2024-02-23T00:39:16.785830+0000","last_became_peered":"2024-02-23T00:39:16.785830+0000","last_unstale":"2024-02-23T00:41:39.402981+0000","last_undegraded":"2024-02-23T00:41:39.402981+0000","last_fullsized":"2024-02-23T00:41:39.402981+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'4","last_scrub_stamp":"2024-02-23T00:41:20.947377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:20.947377+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.402628+0000","last_change":"2024-02-23T00:40:56.892857+0000","last_active":"2024-02-23T00:41:39.402628+0000","last_peered":"2024-02-23T00:41:39.402628+0000","last_clean":"2024-02-23T00:41:39.402628+0000","last_became_active":"2024-02-23T00:39:20.799053+0000","last_became_peered":"2024-02-23T00:39:20.799053+0000","last_unstale":"2024-02-23T00:41:39.402628+0000","last_undegraded":"2024-02-23T00:41:39.402628+0000","last_fullsized":"2024-02-23T00:41:39.402628+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:56.892797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:56.892797+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679332+0000","last_change":"2024-02-23T00:40:59.669880+0000","last_active":"2024-02-23T00:41:39.679332+0000","last_peered":"2024-02-23T00:41:39.679332+0000","last_clean":"2024-02-23T00:41:39.679332+0000","last_became_active":"2024-02-23T00:39:18.800382+0000","last_became_peered":"2024-02-23T00:39:18.800382+0000","last_unstale":"2024-02-23T00:41:39.679332+0000","last_undegraded":"2024-02-23T00:41:39.679332+0000","last_fullsized":"2024-02-23T00:41:39.679332+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:59.669810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:59.669810+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"60'1","reported_seq":55,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679717+0000","last_change":"2024-02-23T00:40:34.909212+0000","last_active":"2024-02-23T00:41:39.679717+0000","last_peered":"2024-02-23T00:41:39.679717+0000","last_clean":"2024-02-23T00:41:39.679717+0000","last_became_active":"2024-02-23T00:39:14.775804+0000","last_became_peered":"2024-02-23T00:39:14.775804+0000","last_unstale":"2024-02-23T00:41:39.679717+0000","last_undegraded":"2024-02-23T00:41:39.679717+0000","last_fullsized":"2024-02-23T00:41:39.679717+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'1","last_scrub_stamp":"2024-02-23T00:40:34.909135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:34.909135+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"67'11","reported_seq":65,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.405063+0000","last_change":"2024-02-23T00:40:55.928416+0000","last_active":"2024-02-23T00:41:39.405063+0000","last_peered":"2024-02-23T00:41:39.405063+0000","last_clean":"2024-02-23T00:41:39.405063+0000","last_became_active":"2024-02-23T00:39:16.783896+0000","last_became_peered":"2024-02-23T00:39:16.783896+0000","last_unstale":"2024-02-23T00:41:39.405063+0000","last_undegraded":"2024-02-23T00:41:39.405063+0000","last_fullsized":"2024-02-23T00:41:39.405063+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'11","last_scrub_stamp":"2024-02-23T00:40:55.928349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:55.928349+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.678518+0000","last_change":"2024-02-23T00:41:10.892383+0000","last_active":"2024-02-23T00:41:39.678518+0000","last_peered":"2024-02-23T00:41:39.678518+0000","last_clean":"2024-02-23T00:41:39.678518+0000","last_became_active":"2024-02-23T00:39:20.802648+0000","last_became_peered":"2024-02-23T00:39:20.802648+0000","last_unstale":"2024-02-23T00:41:39.678518+0000","last_undegraded":"2024-02-23T00:41:39.678518+0000","last_fullsized":"2024-02-23T00:41:39.678518+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:10.892259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:10.892259+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.403592+0000","last_change":"2024-02-23T00:41:17.931479+0000","last_active":"2024-02-23T00:41:39.403592+0000","last_peered":"2024-02-23T00:41:39.403592+0000","last_clean":"2024-02-23T00:41:39.403592+0000","last_became_active":"2024-02-23T00:39:18.799052+0000","last_became_peered":"2024-02-23T00:39:18.799052+0000","last_unstale":"2024-02-23T00:41:39.403592+0000","last_undegraded":"2024-02-23T00:41:39.403592+0000","last_fullsized":"2024-02-23T00:41:39.403592+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_clean_scrub_stamp":"2024-02-23T00:41:17.931413+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":53,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.645344+0000","last_change":"2024-02-23T00:41:33.993147+0000","last_active":"2024-02-23T00:41:39.645344+0000","last_peered":"2024-02-23T00:41:39.645344+0000","last_clean":"2024-02-23T00:41:39.645344+0000","last_became_active":"2024-02-23T00:39:14.778455+0000","last_became_peered":"2024-02-23T00:39:14.778455+0000","last_unstale":"2024-02-23T00:41:39.645344+0000","last_undegraded":"2024-02-23T00:41:39.645344+0000","last_fullsized":"2024-02-23T00:41:39.645344+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:33.993070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:33.993070+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"67'9","reported_seq":62,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679250+0000","last_change":"2024-02-23T00:40:52.783268+0000","last_active":"2024-02-23T00:41:39.679250+0000","last_peered":"2024-02-23T00:41:39.679250+0000","last_clean":"2024-02-23T00:41:39.679250+0000","last_became_active":"2024-02-23T00:39:16.786936+0000","last_became_peered":"2024-02-23T00:39:16.786936+0000","last_unstale":"2024-02-23T00:41:39.679250+0000","last_undegraded":"2024-02-23T00:41:39.679250+0000","last_fullsized":"2024-02-23T00:41:39.679250+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'9","last_scrub_stamp":"2024-02-23T00:40:52.783146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:52.783146+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.405610+0000","last_change":"2024-02-23T00:40:54.938288+0000","last_active":"2024-02-23T00:41:39.405610+0000","last_peered":"2024-02-23T00:41:39.405610+0000","last_clean":"2024-02-23T00:41:39.405610+0000","last_became_active":"2024-02-23T00:39:20.795210+0000","last_became_peered":"2024-02-23T00:39:20.795210+0000","last_unstale":"2024-02-23T00:41:39.405610+0000","last_undegraded":"2024-02-23T00:41:39.405610+0000","last_fullsized":"2024-02-23T00:41:39.405610+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:54.938209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:54.938209+0000","last_clean_scrub_stamp":"2024-02-23T00:40:54.938209+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"67'8","reported_seq":82,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.404848+0000","last_change":"2024-02-23T00:41:25.942914+0000","last_active":"2024-02-23T00:41:39.404848+0000","last_peered":"2024-02-23T00:41:39.404848+0000","last_clean":"2024-02-23T00:41:39.404848+0000","last_became_active":"2024-02-23T00:39:18.787313+0000","last_became_peered":"2024-02-23T00:39:18.787313+0000","last_unstale":"2024-02-23T00:41:39.404848+0000","last_undegraded":"2024-02-23T00:41:39.404848+0000","last_fullsized":"2024-02-23T00:41:39.404848+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:41:25.942822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:25.942822+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679462+0000","last_change":"2024-02-23T00:40:16.547440+0000","last_active":"2024-02-23T00:41:39.679462+0000","last_peered":"2024-02-23T00:41:39.679462+0000","last_clean":"2024-02-23T00:41:39.679462+0000","last_became_active":"2024-02-23T00:39:14.773772+0000","last_became_peered":"2024-02-23T00:39:14.773772+0000","last_unstale":"2024-02-23T00:41:39.679462+0000","last_undegraded":"2024-02-23T00:41:39.679462+0000","last_fullsized":"2024-02-23T00:41:39.679462+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:16.547328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:16.547328+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"68'15","reported_seq":78,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.680472+0000","last_change":"2024-02-23T00:40:42.867136+0000","last_active":"2024-02-23T00:41:39.680472+0000","last_peered":"2024-02-23T00:41:39.680472+0000","last_clean":"2024-02-23T00:41:39.680472+0000","last_became_active":"2024-02-23T00:39:16.858953+0000","last_became_peered":"2024-02-23T00:39:16.858953+0000","last_unstale":"2024-02-23T00:41:39.680472+0000","last_undegraded":"2024-02-23T00:41:39.680472+0000","last_fullsized":"2024-02-23T00:41:39.680472+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:40:42.867043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:42.867043+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679315+0000","last_change":"2024-02-23T00:41:22.855592+0000","last_active":"2024-02-23T00:41:39.679315+0000","last_peered":"2024-02-23T00:41:39.679315+0000","last_clean":"2024-02-23T00:41:39.679315+0000","last_became_active":"2024-02-23T00:39:20.800422+0000","last_became_peered":"2024-02-23T00:39:20.800422+0000","last_unstale":"2024-02-23T00:41:39.679315+0000","last_undegraded":"2024-02-23T00:41:39.679315+0000","last_fullsized":"2024-02-23T00:41:39.679315+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:22.855516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:22.855516+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"67'8","reported_seq":82,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.680603+0000","last_change":"2024-02-23T00:41:08.785056+0000","last_active":"2024-02-23T00:41:39.680603+0000","last_peered":"2024-02-23T00:41:39.680603+0000","last_clean":"2024-02-23T00:41:39.680603+0000","last_became_active":"2024-02-23T00:39:18.803566+0000","last_became_peered":"2024-02-23T00:39:18.803566+0000","last_unstale":"2024-02-23T00:41:39.680603+0000","last_undegraded":"2024-02-23T00:41:39.680603+0000","last_fullsized":"2024-02-23T00:41:39.680603+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:41:08.784965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:08.784965+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679223+0000","last_change":"2024-02-23T00:40:29.990344+0000","last_active":"2024-02-23T00:41:39.679223+0000","last_peered":"2024-02-23T00:41:39.679223+0000","last_clean":"2024-02-23T00:41:39.679223+0000","last_became_active":"2024-02-23T00:39:14.785590+0000","last_became_peered":"2024-02-23T00:39:14.785590+0000","last_unstale":"2024-02-23T00:41:39.679223+0000","last_undegraded":"2024-02-23T00:41:39.679223+0000","last_fullsized":"2024-02-23T00:41:39.679223+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_clean_scrub_stamp":"2024-02-23T00:40:29.990255+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"68'22","reported_seq":92,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.404740+0000","last_change":"2024-02-23T00:41:38.950622+0000","last_active":"2024-02-23T00:41:39.404740+0000","last_peered":"2024-02-23T00:41:39.404740+0000","last_clean":"2024-02-23T00:41:39.404740+0000","last_became_active":"2024-02-23T00:39:16.857437+0000","last_became_peered":"2024-02-23T00:39:16.857437+0000","last_unstale":"2024-02-23T00:41:39.404740+0000","last_undegraded":"2024-02-23T00:41:39.404740+0000","last_fullsized":"2024-02-23T00:41:39.404740+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'22","last_scrub_stamp":"2024-02-23T00:41:38.950535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:38.950535+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":39,"num_read_kb":32,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679478+0000","last_change":"2024-02-23T00:40:26.576403+0000","last_active":"2024-02-23T00:41:39.679478+0000","last_peered":"2024-02-23T00:41:39.679478+0000","last_clean":"2024-02-23T00:41:39.679478+0000","last_became_active":"2024-02-23T00:39:21.162036+0000","last_became_peered":"2024-02-23T00:39:21.162036+0000","last_unstale":"2024-02-23T00:41:39.679478+0000","last_undegraded":"2024-02-23T00:41:39.679478+0000","last_fullsized":"2024-02-23T00:41:39.679478+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:26.576296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:26.576296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.680580+0000","last_change":"2024-02-23T00:41:35.655313+0000","last_active":"2024-02-23T00:41:39.680580+0000","last_peered":"2024-02-23T00:41:39.680580+0000","last_clean":"2024-02-23T00:41:39.680580+0000","last_became_active":"2024-02-23T00:39:18.800297+0000","last_became_peered":"2024-02-23T00:39:18.800297+0000","last_unstale":"2024-02-23T00:41:39.680580+0000","last_undegraded":"2024-02-23T00:41:39.680580+0000","last_fullsized":"2024-02-23T00:41:39.680580+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:35.655209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:35.655209+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.645090+0000","last_change":"2024-02-23T00:40:53.113312+0000","last_active":"2024-02-23T00:41:39.645090+0000","last_peered":"2024-02-23T00:41:39.645090+0000","last_clean":"2024-02-23T00:41:39.645090+0000","last_became_active":"2024-02-23T00:39:14.783164+0000","last_became_peered":"2024-02-23T00:39:14.783164+0000","last_unstale":"2024-02-23T00:41:39.645090+0000","last_undegraded":"2024-02-23T00:41:39.645090+0000","last_fullsized":"2024-02-23T00:41:39.645090+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:53.113232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:53.113232+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"68'12","reported_seq":71,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.404118+0000","last_change":"2024-02-23T00:41:23.904595+0000","last_active":"2024-02-23T00:41:39.404118+0000","last_peered":"2024-02-23T00:41:39.404118+0000","last_clean":"2024-02-23T00:41:39.404118+0000","last_became_active":"2024-02-23T00:39:16.858449+0000","last_became_peered":"2024-02-23T00:39:16.858449+0000","last_unstale":"2024-02-23T00:41:39.404118+0000","last_undegraded":"2024-02-23T00:41:39.404118+0000","last_fullsized":"2024-02-23T00:41:39.404118+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'12","last_scrub_stamp":"2024-02-23T00:41:23.904511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:23.904511+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.402034+0000","last_change":"2024-02-23T00:40:40.360889+0000","last_active":"2024-02-23T00:41:39.402034+0000","last_peered":"2024-02-23T00:41:39.402034+0000","last_clean":"2024-02-23T00:41:39.402034+0000","last_became_active":"2024-02-23T00:39:20.796041+0000","last_became_peered":"2024-02-23T00:39:20.796041+0000","last_unstale":"2024-02-23T00:41:39.402034+0000","last_undegraded":"2024-02-23T00:41:39.402034+0000","last_fullsized":"2024-02-23T00:41:39.402034+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:40.360797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:40.360797+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.404213+0000","last_change":"2024-02-23T00:40:26.171967+0000","last_active":"2024-02-23T00:41:39.404213+0000","last_peered":"2024-02-23T00:41:39.404213+0000","last_clean":"2024-02-23T00:41:39.404213+0000","last_became_active":"2024-02-23T00:39:18.798624+0000","last_became_peered":"2024-02-23T00:39:18.798624+0000","last_unstale":"2024-02-23T00:41:39.404213+0000","last_undegraded":"2024-02-23T00:41:39.404213+0000","last_fullsized":"2024-02-23T00:41:39.404213+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:26.171869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:26.171869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":53,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.402073+0000","last_change":"2024-02-23T00:41:37.165479+0000","last_active":"2024-02-23T00:41:39.402073+0000","last_peered":"2024-02-23T00:41:39.402073+0000","last_clean":"2024-02-23T00:41:39.402073+0000","last_became_active":"2024-02-23T00:39:14.774244+0000","last_became_peered":"2024-02-23T00:39:14.774244+0000","last_unstale":"2024-02-23T00:41:39.402073+0000","last_undegraded":"2024-02-23T00:41:39.402073+0000","last_fullsized":"2024-02-23T00:41:39.402073+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:37.165357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:37.165357+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"68'21","reported_seq":88,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679495+0000","last_change":"2024-02-23T00:39:16.859093+0000","last_active":"2024-02-23T00:41:39.679495+0000","last_peered":"2024-02-23T00:41:39.679495+0000","last_clean":"2024-02-23T00:41:39.679495+0000","last_became_active":"2024-02-23T00:39:16.858760+0000","last_became_peered":"2024-02-23T00:39:16.858760+0000","last_unstale":"2024-02-23T00:41:39.679495+0000","last_undegraded":"2024-02-23T00:41:39.679495+0000","last_fullsized":"2024-02-23T00:41:39.679495+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:39:15.764507+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.403667+0000","last_change":"2024-02-23T00:40:24.129150+0000","last_active":"2024-02-23T00:41:39.403667+0000","last_peered":"2024-02-23T00:41:39.403667+0000","last_clean":"2024-02-23T00:41:39.403667+0000","last_became_active":"2024-02-23T00:39:20.793710+0000","last_became_peered":"2024-02-23T00:39:20.793710+0000","last_unstale":"2024-02-23T00:41:39.403667+0000","last_undegraded":"2024-02-23T00:41:39.403667+0000","last_fullsized":"2024-02-23T00:41:39.403667+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:24.129083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:24.129083+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"60'2","reported_seq":61,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.405446+0000","last_change":"2024-02-23T00:40:38.029108+0000","last_active":"2024-02-23T00:41:39.405446+0000","last_peered":"2024-02-23T00:41:39.405446+0000","last_clean":"2024-02-23T00:41:39.405446+0000","last_became_active":"2024-02-23T00:39:14.779205+0000","last_became_peered":"2024-02-23T00:39:14.779205+0000","last_unstale":"2024-02-23T00:41:39.405446+0000","last_undegraded":"2024-02-23T00:41:39.405446+0000","last_fullsized":"2024-02-23T00:41:39.405446+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-02-23T00:40:38.029000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:38.029000+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"68'25","reported_seq":100,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.404773+0000","last_change":"2024-02-23T00:41:08.004573+0000","last_active":"2024-02-23T00:41:39.404773+0000","last_peered":"2024-02-23T00:41:39.404773+0000","last_clean":"2024-02-23T00:41:39.404773+0000","last_became_active":"2024-02-23T00:39:16.783926+0000","last_became_peered":"2024-02-23T00:39:16.783926+0000","last_unstale":"2024-02-23T00:41:39.404773+0000","last_undegraded":"2024-02-23T00:41:39.404773+0000","last_fullsized":"2024-02-23T00:41:39.404773+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'25","last_scrub_stamp":"2024-02-23T00:41:08.004489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:08.004489+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.401987+0000","last_change":"2024-02-23T00:40:49.365809+0000","last_active":"2024-02-23T00:41:39.401987+0000","last_peered":"2024-02-23T00:41:39.401987+0000","last_clean":"2024-02-23T00:41:39.401987+0000","last_became_active":"2024-02-23T00:39:18.796323+0000","last_became_peered":"2024-02-23T00:39:18.796323+0000","last_unstale":"2024-02-23T00:41:39.401987+0000","last_undegraded":"2024-02-23T00:41:39.401987+0000","last_fullsized":"2024-02-23T00:41:39.401987+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_clean_scrub_stamp":"2024-02-23T00:40:49.365711+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"67'1","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.678533+0000","last_change":"2024-02-23T00:41:00.809258+0000","last_active":"2024-02-23T00:41:39.678533+0000","last_peered":"2024-02-23T00:41:39.678533+0000","last_clean":"2024-02-23T00:41:39.678533+0000","last_became_active":"2024-02-23T00:39:20.800858+0000","last_became_peered":"2024-02-23T00:39:20.800858+0000","last_unstale":"2024-02-23T00:41:39.678533+0000","last_undegraded":"2024-02-23T00:41:39.678533+0000","last_fullsized":"2024-02-23T00:41:39.678533+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:41:00.809190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:00.809190+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.680336+0000","last_change":"2024-02-23T00:40:55.723385+0000","last_active":"2024-02-23T00:41:39.680336+0000","last_peered":"2024-02-23T00:41:39.680336+0000","last_clean":"2024-02-23T00:41:39.680336+0000","last_became_active":"2024-02-23T00:39:14.779952+0000","last_became_peered":"2024-02-23T00:39:14.779952+0000","last_unstale":"2024-02-23T00:41:39.680336+0000","last_undegraded":"2024-02-23T00:41:39.680336+0000","last_fullsized":"2024-02-23T00:41:39.680336+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:55.723309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:55.723309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"68'16","reported_seq":76,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679306+0000","last_change":"2024-02-23T00:40:41.772244+0000","last_active":"2024-02-23T00:41:39.679306+0000","last_peered":"2024-02-23T00:41:39.679306+0000","last_clean":"2024-02-23T00:41:39.679306+0000","last_became_active":"2024-02-23T00:39:16.787468+0000","last_became_peered":"2024-02-23T00:39:16.787468+0000","last_unstale":"2024-02-23T00:41:39.679306+0000","last_undegraded":"2024-02-23T00:41:39.679306+0000","last_fullsized":"2024-02-23T00:41:39.679306+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:40:41.772150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:41.772150+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"91'9","reported_seq":83,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679980+0000","last_change":"2024-02-23T00:41:12.843814+0000","last_active":"2024-02-23T00:41:39.679980+0000","last_peered":"2024-02-23T00:41:39.679980+0000","last_clean":"2024-02-23T00:41:39.679980+0000","last_became_active":"2024-02-23T00:39:18.801297+0000","last_became_peered":"2024-02-23T00:39:18.801297+0000","last_unstale":"2024-02-23T00:41:39.679980+0000","last_undegraded":"2024-02-23T00:41:39.679980+0000","last_fullsized":"2024-02-23T00:41:39.679980+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'9","last_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_deep_scrub":"91'9","last_deep_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_clean_scrub_stamp":"2024-02-23T00:41:12.843733+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.404470+0000","last_change":"2024-02-23T00:41:18.910682+0000","last_active":"2024-02-23T00:41:39.404470+0000","last_peered":"2024-02-23T00:41:39.404470+0000","last_clean":"2024-02-23T00:41:39.404470+0000","last_became_active":"2024-02-23T00:39:20.790053+0000","last_became_peered":"2024-02-23T00:39:20.790053+0000","last_unstale":"2024-02-23T00:41:39.404470+0000","last_undegraded":"2024-02-23T00:41:39.404470+0000","last_fullsized":"2024-02-23T00:41:39.404470+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:18.910605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:18.910605+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679312+0000","last_change":"2024-02-23T00:40:53.824228+0000","last_active":"2024-02-23T00:41:39.679312+0000","last_peered":"2024-02-23T00:41:39.679312+0000","last_clean":"2024-02-23T00:41:39.679312+0000","last_became_active":"2024-02-23T00:39:14.784015+0000","last_became_peered":"2024-02-23T00:39:14.784015+0000","last_unstale":"2024-02-23T00:41:39.679312+0000","last_undegraded":"2024-02-23T00:41:39.679312+0000","last_fullsized":"2024-02-23T00:41:39.679312+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:53.824161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:53.824161+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"86'35","reported_seq":60,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679105+0000","last_change":"2024-02-23T00:40:48.803965+0000","last_active":"2024-02-23T00:41:39.679105+0000","last_peered":"2024-02-23T00:41:39.679105+0000","last_clean":"2024-02-23T00:41:39.679105+0000","last_became_active":"2024-02-23T00:40:42.933322+0000","last_became_peered":"2024-02-23T00:40:42.933322+0000","last_unstale":"2024-02-23T00:41:39.679105+0000","last_undegraded":"2024-02-23T00:41:39.679105+0000","last_fullsized":"2024-02-23T00:41:39.679105+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'35","last_scrub_stamp":"2024-02-23T00:40:48.803893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:48.803893+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679759+0000","last_change":"2024-02-23T00:41:19.899792+0000","last_active":"2024-02-23T00:41:39.679759+0000","last_peered":"2024-02-23T00:41:39.679759+0000","last_clean":"2024-02-23T00:41:39.679759+0000","last_became_active":"2024-02-23T00:39:18.802134+0000","last_became_peered":"2024-02-23T00:39:18.802134+0000","last_unstale":"2024-02-23T00:41:39.679759+0000","last_undegraded":"2024-02-23T00:41:39.679759+0000","last_fullsized":"2024-02-23T00:41:39.679759+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:19.899710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:19.899710+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679783+0000","last_change":"2024-02-23T00:40:36.598336+0000","last_active":"2024-02-23T00:41:39.679783+0000","last_peered":"2024-02-23T00:41:39.679783+0000","last_clean":"2024-02-23T00:41:39.679783+0000","last_became_active":"2024-02-23T00:39:21.160457+0000","last_became_peered":"2024-02-23T00:39:21.160457+0000","last_unstale":"2024-02-23T00:41:39.679783+0000","last_undegraded":"2024-02-23T00:41:39.679783+0000","last_fullsized":"2024-02-23T00:41:39.679783+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:36.598261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:36.598261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"68'5","reported_seq":191,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-23T00:41:41.117843+0000","last_change":"2024-02-23T00:41:00.648420+0000","last_active":"2024-02-23T00:41:41.117843+0000","last_peered":"2024-02-23T00:41:41.117843+0000","last_clean":"2024-02-23T00:41:41.117843+0000","last_became_active":"2024-02-23T00:39:14.779109+0000","last_became_peered":"2024-02-23T00:39:14.779109+0000","last_unstale":"2024-02-23T00:41:41.117843+0000","last_undegraded":"2024-02-23T00:41:41.117843+0000","last_fullsized":"2024-02-23T00:41:41.117843+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'5","last_scrub_stamp":"2024-02-23T00:41:00.648353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:00.648353+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":140,"num_read_kb":135,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.401827+0000","last_change":"2024-02-23T00:41:00.318702+0000","last_active":"2024-02-23T00:41:39.401827+0000","last_peered":"2024-02-23T00:41:39.401827+0000","last_clean":"2024-02-23T00:41:39.401827+0000","last_became_active":"2024-02-23T00:39:14.772411+0000","last_became_peered":"2024-02-23T00:39:14.772411+0000","last_unstale":"2024-02-23T00:41:39.401827+0000","last_undegraded":"2024-02-23T00:41:39.401827+0000","last_fullsized":"2024-02-23T00:41:39.401827+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:00.318633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:00.318633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"68'19","reported_seq":88,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679613+0000","last_change":"2024-02-23T00:40:44.813607+0000","last_active":"2024-02-23T00:41:39.679613+0000","last_peered":"2024-02-23T00:41:39.679613+0000","last_clean":"2024-02-23T00:41:39.679613+0000","last_became_active":"2024-02-23T00:39:16.787484+0000","last_became_peered":"2024-02-23T00:39:16.787484+0000","last_unstale":"2024-02-23T00:41:39.679613+0000","last_undegraded":"2024-02-23T00:41:39.679613+0000","last_fullsized":"2024-02-23T00:41:39.679613+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'19","last_scrub_stamp":"2024-02-23T00:40:44.813506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:44.813506+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.402327+0000","last_change":"2024-02-23T00:40:36.503413+0000","last_active":"2024-02-23T00:41:39.402327+0000","last_peered":"2024-02-23T00:41:39.402327+0000","last_clean":"2024-02-23T00:41:39.402327+0000","last_became_active":"2024-02-23T00:39:18.789330+0000","last_became_peered":"2024-02-23T00:39:18.789330+0000","last_unstale":"2024-02-23T00:41:39.402327+0000","last_undegraded":"2024-02-23T00:41:39.402327+0000","last_fullsized":"2024-02-23T00:41:39.402327+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:36.503342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:36.503342+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.681065+0000","last_change":"2024-02-23T00:40:51.806086+0000","last_active":"2024-02-23T00:41:39.681065+0000","last_peered":"2024-02-23T00:41:39.681065+0000","last_clean":"2024-02-23T00:41:39.681065+0000","last_became_active":"2024-02-23T00:39:20.797226+0000","last_became_peered":"2024-02-23T00:39:20.797226+0000","last_unstale":"2024-02-23T00:41:39.681065+0000","last_undegraded":"2024-02-23T00:41:39.681065+0000","last_fullsized":"2024-02-23T00:41:39.681065+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:51.806007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:51.806007+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.402627+0000","last_change":"2024-02-23T00:40:59.958953+0000","last_active":"2024-02-23T00:41:39.402627+0000","last_peered":"2024-02-23T00:41:39.402627+0000","last_clean":"2024-02-23T00:41:39.402627+0000","last_became_active":"2024-02-23T00:39:14.783489+0000","last_became_peered":"2024-02-23T00:39:14.783489+0000","last_unstale":"2024-02-23T00:41:39.402627+0000","last_undegraded":"2024-02-23T00:41:39.402627+0000","last_fullsized":"2024-02-23T00:41:39.402627+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:59.958850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:59.958850+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679314+0000","last_change":"2024-02-23T00:40:50.760096+0000","last_active":"2024-02-23T00:41:39.679314+0000","last_peered":"2024-02-23T00:41:39.679314+0000","last_clean":"2024-02-23T00:41:39.679314+0000","last_became_active":"2024-02-23T00:39:14.788888+0000","last_became_peered":"2024-02-23T00:39:14.788888+0000","last_unstale":"2024-02-23T00:41:39.679314+0000","last_undegraded":"2024-02-23T00:41:39.679314+0000","last_fullsized":"2024-02-23T00:41:39.679314+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:50.760015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:50.760015+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":57,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679259+0000","last_change":"2024-02-23T00:40:53.770177+0000","last_active":"2024-02-23T00:41:39.679259+0000","last_peered":"2024-02-23T00:41:39.679259+0000","last_clean":"2024-02-23T00:41:39.679259+0000","last_became_active":"2024-02-23T00:39:08.248249+0000","last_became_peered":"2024-02-23T00:39:08.248249+0000","last_unstale":"2024-02-23T00:41:39.679259+0000","last_undegraded":"2024-02-23T00:41:39.679259+0000","last_fullsized":"2024-02-23T00:41:39.679259+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:53.770082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:53.770082+0000","last_clean_scrub_stamp":"2024-02-23T00:40:53.770082+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"67'14","reported_seq":68,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.645548+0000","last_change":"2024-02-23T00:40:41.920396+0000","last_active":"2024-02-23T00:41:39.645548+0000","last_peered":"2024-02-23T00:41:39.645548+0000","last_clean":"2024-02-23T00:41:39.645548+0000","last_became_active":"2024-02-23T00:39:16.858880+0000","last_became_peered":"2024-02-23T00:39:16.858880+0000","last_unstale":"2024-02-23T00:41:39.645548+0000","last_undegraded":"2024-02-23T00:41:39.645548+0000","last_fullsized":"2024-02-23T00:41:39.645548+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'14","last_scrub_stamp":"2024-02-23T00:40:41.920313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:41.920313+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.680255+0000","last_change":"2024-02-23T00:40:48.790072+0000","last_active":"2024-02-23T00:41:39.680255+0000","last_peered":"2024-02-23T00:41:39.680255+0000","last_clean":"2024-02-23T00:41:39.680255+0000","last_became_active":"2024-02-23T00:39:18.793801+0000","last_became_peered":"2024-02-23T00:39:18.793801+0000","last_unstale":"2024-02-23T00:41:39.680255+0000","last_undegraded":"2024-02-23T00:41:39.680255+0000","last_fullsized":"2024-02-23T00:41:39.680255+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:48.789992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:48.789992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.680059+0000","last_change":"2024-02-23T00:41:02.714085+0000","last_active":"2024-02-23T00:41:39.680059+0000","last_peered":"2024-02-23T00:41:39.680059+0000","last_clean":"2024-02-23T00:41:39.680059+0000","last_became_active":"2024-02-23T00:39:20.797242+0000","last_became_peered":"2024-02-23T00:39:20.797242+0000","last_unstale":"2024-02-23T00:41:39.680059+0000","last_undegraded":"2024-02-23T00:41:39.680059+0000","last_fullsized":"2024-02-23T00:41:39.680059+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:02.714020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:02.714020+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679894+0000","last_change":"2024-02-23T00:40:15.584807+0000","last_active":"2024-02-23T00:41:39.679894+0000","last_peered":"2024-02-23T00:41:39.679894+0000","last_clean":"2024-02-23T00:41:39.679894+0000","last_became_active":"2024-02-23T00:39:14.775446+0000","last_became_peered":"2024-02-23T00:39:14.775446+0000","last_unstale":"2024-02-23T00:41:39.679894+0000","last_undegraded":"2024-02-23T00:41:39.679894+0000","last_fullsized":"2024-02-23T00:41:39.679894+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:15.584732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:15.584732+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":52,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679112+0000","last_change":"2024-02-23T00:41:26.836896+0000","last_active":"2024-02-23T00:41:39.679112+0000","last_peered":"2024-02-23T00:41:39.679112+0000","last_clean":"2024-02-23T00:41:39.679112+0000","last_became_active":"2024-02-23T00:39:14.784578+0000","last_became_peered":"2024-02-23T00:39:14.784578+0000","last_unstale":"2024-02-23T00:41:39.679112+0000","last_undegraded":"2024-02-23T00:41:39.679112+0000","last_fullsized":"2024-02-23T00:41:39.679112+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:26.836819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:26.836819+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"67'12","reported_seq":65,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679408+0000","last_change":"2024-02-23T00:40:48.696967+0000","last_active":"2024-02-23T00:41:39.679408+0000","last_peered":"2024-02-23T00:41:39.679408+0000","last_clean":"2024-02-23T00:41:39.679408+0000","last_became_active":"2024-02-23T00:39:16.787143+0000","last_became_peered":"2024-02-23T00:39:16.787143+0000","last_unstale":"2024-02-23T00:41:39.679408+0000","last_undegraded":"2024-02-23T00:41:39.679408+0000","last_fullsized":"2024-02-23T00:41:39.679408+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'12","last_scrub_stamp":"2024-02-23T00:40:48.696895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:48.696895+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.681002+0000","last_change":"2024-02-23T00:40:38.812399+0000","last_active":"2024-02-23T00:41:39.681002+0000","last_peered":"2024-02-23T00:41:39.681002+0000","last_clean":"2024-02-23T00:41:39.681002+0000","last_became_active":"2024-02-23T00:39:18.800378+0000","last_became_peered":"2024-02-23T00:39:18.800378+0000","last_unstale":"2024-02-23T00:41:39.681002+0000","last_undegraded":"2024-02-23T00:41:39.681002+0000","last_fullsized":"2024-02-23T00:41:39.681002+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:38.812327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:38.812327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.678923+0000","last_change":"2024-02-23T00:40:27.859722+0000","last_active":"2024-02-23T00:41:39.678923+0000","last_peered":"2024-02-23T00:41:39.678923+0000","last_clean":"2024-02-23T00:41:39.678923+0000","last_became_active":"2024-02-23T00:39:21.160060+0000","last_became_peered":"2024-02-23T00:39:21.160060+0000","last_unstale":"2024-02-23T00:41:39.678923+0000","last_undegraded":"2024-02-23T00:41:39.678923+0000","last_fullsized":"2024-02-23T00:41:39.678923+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:27.859657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:27.859657+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679919+0000","last_change":"2024-02-23T00:40:41.671069+0000","last_active":"2024-02-23T00:41:39.679919+0000","last_peered":"2024-02-23T00:41:39.679919+0000","last_clean":"2024-02-23T00:41:39.679919+0000","last_became_active":"2024-02-23T00:39:14.783662+0000","last_became_peered":"2024-02-23T00:39:14.783662+0000","last_unstale":"2024-02-23T00:41:39.679919+0000","last_undegraded":"2024-02-23T00:41:39.679919+0000","last_fullsized":"2024-02-23T00:41:39.679919+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:41.670962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:41.670962+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"68'25","reported_seq":98,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679329+0000","last_change":"2024-02-23T00:40:52.751667+0000","last_active":"2024-02-23T00:41:39.679329+0000","last_peered":"2024-02-23T00:41:39.679329+0000","last_clean":"2024-02-23T00:41:39.679329+0000","last_became_active":"2024-02-23T00:39:16.857278+0000","last_became_peered":"2024-02-23T00:39:16.857278+0000","last_unstale":"2024-02-23T00:41:39.679329+0000","last_undegraded":"2024-02-23T00:41:39.679329+0000","last_fullsized":"2024-02-23T00:41:39.679329+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'25","last_scrub_stamp":"2024-02-23T00:40:52.751597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:52.751597+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.645341+0000","last_change":"2024-02-23T00:40:50.029161+0000","last_active":"2024-02-23T00:41:39.645341+0000","last_peered":"2024-02-23T00:41:39.645341+0000","last_clean":"2024-02-23T00:41:39.645341+0000","last_became_active":"2024-02-23T00:39:18.796664+0000","last_became_peered":"2024-02-23T00:39:18.796664+0000","last_unstale":"2024-02-23T00:41:39.645341+0000","last_undegraded":"2024-02-23T00:41:39.645341+0000","last_fullsized":"2024-02-23T00:41:39.645341+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:50.029087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:50.029087+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.405120+0000","last_change":"2024-02-23T00:40:49.937604+0000","last_active":"2024-02-23T00:41:39.405120+0000","last_peered":"2024-02-23T00:41:39.405120+0000","last_clean":"2024-02-23T00:41:39.405120+0000","last_became_active":"2024-02-23T00:39:20.795271+0000","last_became_peered":"2024-02-23T00:39:20.795271+0000","last_unstale":"2024-02-23T00:41:39.405120+0000","last_undegraded":"2024-02-23T00:41:39.405120+0000","last_fullsized":"2024-02-23T00:41:39.405120+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:49.937526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:49.937526+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679591+0000","last_change":"2024-02-23T00:40:36.888232+0000","last_active":"2024-02-23T00:41:39.679591+0000","last_peered":"2024-02-23T00:41:39.679591+0000","last_clean":"2024-02-23T00:41:39.679591+0000","last_became_active":"2024-02-23T00:39:14.779193+0000","last_became_peered":"2024-02-23T00:39:14.779193+0000","last_unstale":"2024-02-23T00:41:39.679591+0000","last_undegraded":"2024-02-23T00:41:39.679591+0000","last_fullsized":"2024-02-23T00:41:39.679591+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:36.888124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:36.888124+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"68'30","reported_seq":109,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.401717+0000","last_change":"2024-02-23T00:41:21.909427+0000","last_active":"2024-02-23T00:41:39.401717+0000","last_peered":"2024-02-23T00:41:39.401717+0000","last_clean":"2024-02-23T00:41:39.401717+0000","last_became_active":"2024-02-23T00:39:16.859002+0000","last_became_peered":"2024-02-23T00:39:16.859002+0000","last_unstale":"2024-02-23T00:41:39.401717+0000","last_undegraded":"2024-02-23T00:41:39.401717+0000","last_fullsized":"2024-02-23T00:41:39.401717+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'30","last_scrub_stamp":"2024-02-23T00:41:21.909340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:21.909340+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.404902+0000","last_change":"2024-02-23T00:40:23.103679+0000","last_active":"2024-02-23T00:41:39.404902+0000","last_peered":"2024-02-23T00:41:39.404902+0000","last_clean":"2024-02-23T00:41:39.404902+0000","last_became_active":"2024-02-23T00:39:18.798829+0000","last_became_peered":"2024-02-23T00:39:18.798829+0000","last_unstale":"2024-02-23T00:41:39.404902+0000","last_undegraded":"2024-02-23T00:41:39.404902+0000","last_fullsized":"2024-02-23T00:41:39.404902+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:23.103586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:23.103586+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.680042+0000","last_change":"2024-02-23T00:40:33.626648+0000","last_active":"2024-02-23T00:41:39.680042+0000","last_peered":"2024-02-23T00:41:39.680042+0000","last_clean":"2024-02-23T00:41:39.680042+0000","last_became_active":"2024-02-23T00:39:21.161928+0000","last_became_peered":"2024-02-23T00:39:21.161928+0000","last_unstale":"2024-02-23T00:41:39.680042+0000","last_undegraded":"2024-02-23T00:41:39.680042+0000","last_fullsized":"2024-02-23T00:41:39.680042+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:33.626500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:33.626500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.645694+0000","last_change":"2024-02-23T00:40:45.971552+0000","last_active":"2024-02-23T00:41:39.645694+0000","last_peered":"2024-02-23T00:41:39.645694+0000","last_clean":"2024-02-23T00:41:39.645694+0000","last_became_active":"2024-02-23T00:39:14.781999+0000","last_became_peered":"2024-02-23T00:39:14.781999+0000","last_unstale":"2024-02-23T00:41:39.645694+0000","last_undegraded":"2024-02-23T00:41:39.645694+0000","last_fullsized":"2024-02-23T00:41:39.645694+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:45.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:45.971465+0000","last_clean_scrub_stamp":"2024-02-23T00:40:45.971465+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"68'20","reported_seq":82,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.404631+0000","last_change":"2024-02-23T00:41:05.002039+0000","last_active":"2024-02-23T00:41:39.404631+0000","last_peered":"2024-02-23T00:41:39.404631+0000","last_clean":"2024-02-23T00:41:39.404631+0000","last_became_active":"2024-02-23T00:39:16.780130+0000","last_became_peered":"2024-02-23T00:39:16.780130+0000","last_unstale":"2024-02-23T00:41:39.404631+0000","last_undegraded":"2024-02-23T00:41:39.404631+0000","last_fullsized":"2024-02-23T00:41:39.404631+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'20","last_scrub_stamp":"2024-02-23T00:41:05.001912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:05.001912+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":82,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.401655+0000","last_change":"2024-02-23T00:41:09.882111+0000","last_active":"2024-02-23T00:41:39.401655+0000","last_peered":"2024-02-23T00:41:39.401655+0000","last_clean":"2024-02-23T00:41:39.401655+0000","last_became_active":"2024-02-23T00:39:18.798536+0000","last_became_peered":"2024-02-23T00:39:18.798536+0000","last_unstale":"2024-02-23T00:41:39.401655+0000","last_undegraded":"2024-02-23T00:41:39.401655+0000","last_fullsized":"2024-02-23T00:41:39.401655+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:41:09.882012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:09.882012+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.404596+0000","last_change":"2024-02-23T00:40:46.876448+0000","last_active":"2024-02-23T00:41:39.404596+0000","last_peered":"2024-02-23T00:41:39.404596+0000","last_clean":"2024-02-23T00:41:39.404596+0000","last_became_active":"2024-02-23T00:39:21.160443+0000","last_became_peered":"2024-02-23T00:39:21.160443+0000","last_unstale":"2024-02-23T00:41:39.404596+0000","last_undegraded":"2024-02-23T00:41:39.404596+0000","last_fullsized":"2024-02-23T00:41:39.404596+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:46.876380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:46.876380+0000","last_clean_scrub_stamp":"2024-02-23T00:40:46.876380+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.680977+0000","last_change":"2024-02-23T00:41:04.839634+0000","last_active":"2024-02-23T00:41:39.680977+0000","last_peered":"2024-02-23T00:41:39.680977+0000","last_clean":"2024-02-23T00:41:39.680977+0000","last_became_active":"2024-02-23T00:39:18.800777+0000","last_became_peered":"2024-02-23T00:39:18.800777+0000","last_unstale":"2024-02-23T00:41:39.680977+0000","last_undegraded":"2024-02-23T00:41:39.680977+0000","last_fullsized":"2024-02-23T00:41:39.680977+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_clean_scrub_stamp":"2024-02-23T00:41:04.839555+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679249+0000","last_change":"2024-02-23T00:40:23.574894+0000","last_active":"2024-02-23T00:41:39.679249+0000","last_peered":"2024-02-23T00:41:39.679249+0000","last_clean":"2024-02-23T00:41:39.679249+0000","last_became_active":"2024-02-23T00:39:14.784121+0000","last_became_peered":"2024-02-23T00:39:14.784121+0000","last_unstale":"2024-02-23T00:41:39.679249+0000","last_undegraded":"2024-02-23T00:41:39.679249+0000","last_fullsized":"2024-02-23T00:41:39.679249+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:23.574813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:23.574813+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"68'17","reported_seq":103,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.405489+0000","last_change":"2024-02-23T00:41:13.017122+0000","last_active":"2024-02-23T00:41:39.405489+0000","last_peered":"2024-02-23T00:41:39.405489+0000","last_clean":"2024-02-23T00:41:39.405489+0000","last_became_active":"2024-02-23T00:40:42.933759+0000","last_became_peered":"2024-02-23T00:40:42.933759+0000","last_unstale":"2024-02-23T00:41:39.405489+0000","last_undegraded":"2024-02-23T00:41:39.405489+0000","last_fullsized":"2024-02-23T00:41:39.405489+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:41:13.017043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:13.017043+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.680674+0000","last_change":"2024-02-23T00:40:56.890866+0000","last_active":"2024-02-23T00:41:39.680674+0000","last_peered":"2024-02-23T00:41:39.680674+0000","last_clean":"2024-02-23T00:41:39.680674+0000","last_became_active":"2024-02-23T00:39:20.800800+0000","last_became_peered":"2024-02-23T00:39:20.800800+0000","last_unstale":"2024-02-23T00:41:39.680674+0000","last_undegraded":"2024-02-23T00:41:39.680674+0000","last_fullsized":"2024-02-23T00:41:39.680674+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:56.890792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:56.890792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":81,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.405335+0000","last_change":"2024-02-23T00:40:30.998799+0000","last_active":"2024-02-23T00:41:39.405335+0000","last_peered":"2024-02-23T00:41:39.405335+0000","last_clean":"2024-02-23T00:41:39.405335+0000","last_became_active":"2024-02-23T00:39:18.800440+0000","last_became_peered":"2024-02-23T00:39:18.800440+0000","last_unstale":"2024-02-23T00:41:39.405335+0000","last_undegraded":"2024-02-23T00:41:39.405335+0000","last_fullsized":"2024-02-23T00:41:39.405335+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:40:30.998729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:30.998729+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679152+0000","last_change":"2024-02-23T00:40:26.845049+0000","last_active":"2024-02-23T00:41:39.679152+0000","last_peered":"2024-02-23T00:41:39.679152+0000","last_clean":"2024-02-23T00:41:39.679152+0000","last_became_active":"2024-02-23T00:39:14.780526+0000","last_became_peered":"2024-02-23T00:39:14.780526+0000","last_unstale":"2024-02-23T00:41:39.679152+0000","last_undegraded":"2024-02-23T00:41:39.679152+0000","last_fullsized":"2024-02-23T00:41:39.679152+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:26.844984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:26.844984+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"67'15","reported_seq":76,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.682330+0000","last_change":"2024-02-23T00:41:39.682283+0000","last_active":"2024-02-23T00:41:39.682330+0000","last_peered":"2024-02-23T00:41:39.682330+0000","last_clean":"2024-02-23T00:41:39.682330+0000","last_became_active":"2024-02-23T00:39:16.789757+0000","last_became_peered":"2024-02-23T00:39:16.789757+0000","last_unstale":"2024-02-23T00:41:39.682330+0000","last_undegraded":"2024-02-23T00:41:39.682330+0000","last_fullsized":"2024-02-23T00:41:39.682330+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:41:39.682182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:39.682182+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.405283+0000","last_change":"2024-02-23T00:41:15.081250+0000","last_active":"2024-02-23T00:41:39.405283+0000","last_peered":"2024-02-23T00:41:39.405283+0000","last_clean":"2024-02-23T00:41:39.405283+0000","last_became_active":"2024-02-23T00:39:20.795097+0000","last_became_peered":"2024-02-23T00:39:20.795097+0000","last_unstale":"2024-02-23T00:41:39.405283+0000","last_undegraded":"2024-02-23T00:41:39.405283+0000","last_fullsized":"2024-02-23T00:41:39.405283+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:15.081153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:15.081153+0000","last_clean_scrub_stamp":"2024-02-23T00:41:15.081153+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"67'8","reported_seq":81,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.402257+0000","last_change":"2024-02-23T00:40:58.331136+0000","last_active":"2024-02-23T00:41:39.402257+0000","last_peered":"2024-02-23T00:41:39.402257+0000","last_clean":"2024-02-23T00:41:39.402257+0000","last_became_active":"2024-02-23T00:39:18.799544+0000","last_became_peered":"2024-02-23T00:39:18.799544+0000","last_unstale":"2024-02-23T00:41:39.402257+0000","last_undegraded":"2024-02-23T00:41:39.402257+0000","last_fullsized":"2024-02-23T00:41:39.402257+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:40:58.331048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:58.331048+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679209+0000","last_change":"2024-02-23T00:40:39.825090+0000","last_active":"2024-02-23T00:41:39.679209+0000","last_peered":"2024-02-23T00:41:39.679209+0000","last_clean":"2024-02-23T00:41:39.679209+0000","last_became_active":"2024-02-23T00:39:14.780379+0000","last_became_peered":"2024-02-23T00:39:14.780379+0000","last_unstale":"2024-02-23T00:41:39.679209+0000","last_undegraded":"2024-02-23T00:41:39.679209+0000","last_fullsized":"2024-02-23T00:41:39.679209+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:39.825013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:39.825013+0000","last_clean_scrub_stamp":"2024-02-23T00:40:39.825013+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"67'10","reported_seq":61,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.404961+0000","last_change":"2024-02-23T00:41:14.059424+0000","last_active":"2024-02-23T00:41:39.404961+0000","last_peered":"2024-02-23T00:41:39.404961+0000","last_clean":"2024-02-23T00:41:39.404961+0000","last_became_active":"2024-02-23T00:39:16.777530+0000","last_became_peered":"2024-02-23T00:39:16.777530+0000","last_unstale":"2024-02-23T00:41:39.404961+0000","last_undegraded":"2024-02-23T00:41:39.404961+0000","last_fullsized":"2024-02-23T00:41:39.404961+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'10","last_scrub_stamp":"2024-02-23T00:41:14.059334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:14.059334+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.402328+0000","last_change":"2024-02-23T00:40:54.348946+0000","last_active":"2024-02-23T00:41:39.402328+0000","last_peered":"2024-02-23T00:41:39.402328+0000","last_clean":"2024-02-23T00:41:39.402328+0000","last_became_active":"2024-02-23T00:39:20.800943+0000","last_became_peered":"2024-02-23T00:39:20.800943+0000","last_unstale":"2024-02-23T00:41:39.402328+0000","last_undegraded":"2024-02-23T00:41:39.402328+0000","last_fullsized":"2024-02-23T00:41:39.402328+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:54.348843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:54.348843+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.680134+0000","last_change":"2024-02-23T00:40:35.861920+0000","last_active":"2024-02-23T00:41:39.680134+0000","last_peered":"2024-02-23T00:41:39.680134+0000","last_clean":"2024-02-23T00:41:39.680134+0000","last_became_active":"2024-02-23T00:39:18.799875+0000","last_became_peered":"2024-02-23T00:39:18.799875+0000","last_unstale":"2024-02-23T00:41:39.680134+0000","last_undegraded":"2024-02-23T00:41:39.680134+0000","last_fullsized":"2024-02-23T00:41:39.680134+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:35.861854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:35.861854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:40.128491+0000","last_change":"2024-02-23T00:41:40.128467+0000","last_active":"2024-02-23T00:41:40.128491+0000","last_peered":"2024-02-23T00:41:40.128491+0000","last_clean":"2024-02-23T00:41:40.128491+0000","last_became_active":"2024-02-23T00:40:42.931041+0000","last_became_peered":"2024-02-23T00:40:42.931041+0000","last_unstale":"2024-02-23T00:41:40.128491+0000","last_undegraded":"2024-02-23T00:41:40.128491+0000","last_fullsized":"2024-02-23T00:41:40.128491+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:40.128397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:40.128397+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"67'17","reported_seq":101,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.405317+0000","last_change":"2024-02-23T00:41:02.038717+0000","last_active":"2024-02-23T00:41:39.405317+0000","last_peered":"2024-02-23T00:41:39.405317+0000","last_clean":"2024-02-23T00:41:39.405317+0000","last_became_active":"2024-02-23T00:40:42.933806+0000","last_became_peered":"2024-02-23T00:40:42.933806+0000","last_unstale":"2024-02-23T00:41:39.405317+0000","last_undegraded":"2024-02-23T00:41:39.405317+0000","last_fullsized":"2024-02-23T00:41:39.405317+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'17","last_scrub_stamp":"2024-02-23T00:41:02.038627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:02.038627+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679119+0000","last_change":"2024-02-23T00:40:51.777535+0000","last_active":"2024-02-23T00:41:39.679119+0000","last_peered":"2024-02-23T00:41:39.679119+0000","last_clean":"2024-02-23T00:41:39.679119+0000","last_became_active":"2024-02-23T00:39:20.798989+0000","last_became_peered":"2024-02-23T00:39:20.798989+0000","last_unstale":"2024-02-23T00:41:39.679119+0000","last_undegraded":"2024-02-23T00:41:39.679119+0000","last_fullsized":"2024-02-23T00:41:39.679119+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:51.777405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:51.777405+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.402655+0000","last_change":"2024-02-23T00:40:33.492291+0000","last_active":"2024-02-23T00:41:39.402655+0000","last_peered":"2024-02-23T00:41:39.402655+0000","last_clean":"2024-02-23T00:41:39.402655+0000","last_became_active":"2024-02-23T00:39:18.792718+0000","last_became_peered":"2024-02-23T00:39:18.792718+0000","last_unstale":"2024-02-23T00:41:39.402655+0000","last_undegraded":"2024-02-23T00:41:39.402655+0000","last_fullsized":"2024-02-23T00:41:39.402655+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:33.492224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:33.492224+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.402242+0000","last_change":"2024-02-23T00:40:28.450480+0000","last_active":"2024-02-23T00:41:39.402242+0000","last_peered":"2024-02-23T00:41:39.402242+0000","last_clean":"2024-02-23T00:41:39.402242+0000","last_became_active":"2024-02-23T00:39:14.776705+0000","last_became_peered":"2024-02-23T00:39:14.776705+0000","last_unstale":"2024-02-23T00:41:39.402242+0000","last_undegraded":"2024-02-23T00:41:39.402242+0000","last_fullsized":"2024-02-23T00:41:39.402242+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_clean_scrub_stamp":"2024-02-23T00:40:28.450377+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"88'35","reported_seq":102,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.645205+0000","last_change":"2024-02-23T00:40:54.152313+0000","last_active":"2024-02-23T00:41:39.645205+0000","last_peered":"2024-02-23T00:41:39.645205+0000","last_clean":"2024-02-23T00:41:39.645205+0000","last_became_active":"2024-02-23T00:39:16.857413+0000","last_became_peered":"2024-02-23T00:39:16.857413+0000","last_unstale":"2024-02-23T00:41:39.645205+0000","last_undegraded":"2024-02-23T00:41:39.645205+0000","last_fullsized":"2024-02-23T00:41:39.645205+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'35","last_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_deep_scrub":"88'35","last_deep_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_clean_scrub_stamp":"2024-02-23T00:40:54.152239+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.678584+0000","last_change":"2024-02-23T00:41:05.802853+0000","last_active":"2024-02-23T00:41:39.678584+0000","last_peered":"2024-02-23T00:41:39.678584+0000","last_clean":"2024-02-23T00:41:39.678584+0000","last_became_active":"2024-02-23T00:39:20.802938+0000","last_became_peered":"2024-02-23T00:39:20.802938+0000","last_unstale":"2024-02-23T00:41:39.678584+0000","last_undegraded":"2024-02-23T00:41:39.678584+0000","last_fullsized":"2024-02-23T00:41:39.678584+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:05.802777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:05.802777+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.402107+0000","last_change":"2024-02-23T00:41:02.376871+0000","last_active":"2024-02-23T00:41:39.402107+0000","last_peered":"2024-02-23T00:41:39.402107+0000","last_clean":"2024-02-23T00:41:39.402107+0000","last_became_active":"2024-02-23T00:39:18.794476+0000","last_became_peered":"2024-02-23T00:39:18.794476+0000","last_unstale":"2024-02-23T00:41:39.402107+0000","last_undegraded":"2024-02-23T00:41:39.402107+0000","last_fullsized":"2024-02-23T00:41:39.402107+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:02.376804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:02.376804+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679796+0000","last_change":"2024-02-23T00:40:57.681793+0000","last_active":"2024-02-23T00:41:39.679796+0000","last_peered":"2024-02-23T00:41:39.679796+0000","last_clean":"2024-02-23T00:41:39.679796+0000","last_became_active":"2024-02-23T00:39:14.772493+0000","last_became_peered":"2024-02-23T00:39:14.772493+0000","last_unstale":"2024-02-23T00:41:39.679796+0000","last_undegraded":"2024-02-23T00:41:39.679796+0000","last_fullsized":"2024-02-23T00:41:39.679796+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:57.681723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:57.681723+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"68'15","reported_seq":78,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679808+0000","last_change":"2024-02-23T00:40:47.732636+0000","last_active":"2024-02-23T00:41:39.679808+0000","last_peered":"2024-02-23T00:41:39.679808+0000","last_clean":"2024-02-23T00:41:39.679808+0000","last_became_active":"2024-02-23T00:39:16.783701+0000","last_became_peered":"2024-02-23T00:39:16.783701+0000","last_unstale":"2024-02-23T00:41:39.679808+0000","last_undegraded":"2024-02-23T00:41:39.679808+0000","last_fullsized":"2024-02-23T00:41:39.679808+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:40:47.732524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:47.732524+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679119+0000","last_change":"2024-02-23T00:40:39.859273+0000","last_active":"2024-02-23T00:41:39.679119+0000","last_peered":"2024-02-23T00:41:39.679119+0000","last_clean":"2024-02-23T00:41:39.679119+0000","last_became_active":"2024-02-23T00:39:21.160429+0000","last_became_peered":"2024-02-23T00:39:21.160429+0000","last_unstale":"2024-02-23T00:41:39.679119+0000","last_undegraded":"2024-02-23T00:41:39.679119+0000","last_fullsized":"2024-02-23T00:41:39.679119+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:39.859197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:39.859197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.402727+0000","last_change":"2024-02-23T00:40:34.533819+0000","last_active":"2024-02-23T00:41:39.402727+0000","last_peered":"2024-02-23T00:41:39.402727+0000","last_clean":"2024-02-23T00:41:39.402727+0000","last_became_active":"2024-02-23T00:39:14.784436+0000","last_became_peered":"2024-02-23T00:39:14.784436+0000","last_unstale":"2024-02-23T00:41:39.402727+0000","last_undegraded":"2024-02-23T00:41:39.402727+0000","last_fullsized":"2024-02-23T00:41:39.402727+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:34.533743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:34.533743+0000","last_clean_scrub_stamp":"2024-02-23T00:40:34.533743+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"68'27","reported_seq":102,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.678948+0000","last_change":"2024-02-23T00:40:59.885996+0000","last_active":"2024-02-23T00:41:39.678948+0000","last_peered":"2024-02-23T00:41:39.678948+0000","last_clean":"2024-02-23T00:41:39.678948+0000","last_became_active":"2024-02-23T00:39:16.859401+0000","last_became_peered":"2024-02-23T00:39:16.859401+0000","last_unstale":"2024-02-23T00:41:39.678948+0000","last_undegraded":"2024-02-23T00:41:39.678948+0000","last_fullsized":"2024-02-23T00:41:39.678948+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'27","last_scrub_stamp":"2024-02-23T00:40:59.885908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:59.885908+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"67'8","reported_seq":83,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679889+0000","last_change":"2024-02-23T00:40:44.713885+0000","last_active":"2024-02-23T00:41:39.679889+0000","last_peered":"2024-02-23T00:41:39.679889+0000","last_clean":"2024-02-23T00:41:39.679889+0000","last_became_active":"2024-02-23T00:39:18.793414+0000","last_became_peered":"2024-02-23T00:39:18.793414+0000","last_unstale":"2024-02-23T00:41:39.679889+0000","last_undegraded":"2024-02-23T00:41:39.679889+0000","last_fullsized":"2024-02-23T00:41:39.679889+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:40:44.713825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:44.713825+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1082,"num_read_kb":807,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":616,"ondisk_log_size":616,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":140,"num_read_kb":135,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234017,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297852,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447300,"statfs":{"total":95995035648,"available":95690035200,"internally_reserved":0,"allocated":774144,"data_stored":391332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.374,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.69499999999999995,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.499,"15min":0.505},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.69999999999999996,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.496}]},{"osd":1,"last update":"Fri Feb 23 00:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.54400000000000004,"15min":0.55000000000000004},"min":{"1min":0.38400000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.75600000000000001,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.54100000000000004,"15min":0.54700000000000004},"min":{"1min":0.39600000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.64600000000000002,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.63800000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.53000000000000003,"15min":0.53800000000000003},"min":{"1min":0.40000000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72699999999999998,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.52600000000000002,"15min":0.53200000000000003},"min":{"1min":0.34899999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.65300000000000002,"5min":1.024,"15min":1.024},"last":0.65300000000000002}]},{"osd":3,"last update":"Fri Feb 23 00:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.52100000000000002,"15min":0.52500000000000002},"min":{"1min":0.38200000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.68400000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.55600000000000005,"15min":0.56399999999999995},"min":{"1min":0.38300000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72499999999999998,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.73499999999999999}]},{"osd":4,"last update":"Fri Feb 23 00:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.41999999999999998,"15min":0.42799999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.56899999999999995,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.443,"15min":0.45000000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.61099999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.70999999999999996}]},{"osd":5,"last update":"Fri Feb 23 00:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.46600000000000003,"15min":0.47399999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.58399999999999996,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.47099999999999997,"15min":0.47399999999999998},"min":{"1min":0.307,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.625,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.753}]},{"osd":6,"last update":"Fri Feb 23 00:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.497,"15min":0.503},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.496,"15min":0.503},"min":{"1min":0.33900000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.65500000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.77300000000000002}]}]},{"osd":6,"up_from":47,"seq":201863462952,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297832,"kb_used_data":736,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447320,"statfs":{"total":95995035648,"available":95690055680,"internally_reserved":0,"allocated":753664,"data_stored":390682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73199999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.47899999999999998}]},{"osd":1,"last update":"Fri Feb 23 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54900000000000004,"15min":0.55200000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72699999999999998,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.435},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77900000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.53000000000000003}]},{"osd":2,"last update":"Fri Feb 23 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58199999999999996,"15min":0.58399999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71199999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.749,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.46400000000000002}]},{"osd":3,"last update":"Fri Feb 23 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57899999999999996,"15min":0.58199999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81399999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59399999999999997,"15min":0.59499999999999997},"min":{"1min":0.41799999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.64200000000000002}]},{"osd":4,"last update":"Fri Feb 23 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.40999999999999998,"15min":0.40400000000000003},"min":{"1min":0.28199999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.433,"15min":0.42699999999999999},"min":{"1min":0.29899999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.66200000000000003}]},{"osd":5,"last update":"Fri Feb 23 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.41899999999999998,"15min":0.41199999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.497,"5min":0.44500000000000001,"15min":0.437},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.58499999999999996}]},{"osd":7,"last update":"Fri Feb 23 00:41:11 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.48999999999999999,"15min":0.48799999999999999},"min":{"1min":0.37,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65200000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76700000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.68200000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297860,"kb_used_data":764,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447292,"statfs":{"total":95995035648,"available":95690027008,"internally_reserved":0,"allocated":782336,"data_stored":391966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:41:37 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.379,"15min":0.34799999999999998},"min":{"1min":0.22600000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.60799999999999998,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.40300000000000002,"15min":0.36499999999999999},"min":{"1min":0.27500000000000002,"5min":0.191,"15min":0.191},"max":{"1min":0.57599999999999996,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.252}]},{"osd":2,"last update":"Fri Feb 23 00:40:58 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.39600000000000002,"15min":0.39400000000000002},"min":{"1min":0.245,"5min":0.188,"15min":0.188},"max":{"1min":0.53500000000000003,"5min":0.63,"15min":0.63},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.441,"15min":0.435},"min":{"1min":0.29999999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.67900000000000005,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.313}]},{"osd":3,"last update":"Fri Feb 23 00:41:31 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.47199999999999998,"15min":0.48499999999999999},"min":{"1min":0.314,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.62,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.443,"15min":0.45500000000000002},"min":{"1min":0.28999999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.66900000000000004,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.27700000000000002}]},{"osd":4,"last update":"Fri Feb 23 00:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60599999999999998,"15min":0.61199999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.875,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61199999999999999,"15min":0.61299999999999999},"min":{"1min":0.39500000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83199999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.373}]},{"osd":5,"last update":"Fri Feb 23 00:41:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63500000000000001,"15min":0.65700000000000003},"min":{"1min":0.36899999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84599999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69599999999999995,"15min":0.71599999999999997},"min":{"1min":0.442,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.39500000000000002}]},{"osd":6,"last update":"Fri Feb 23 00:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63700000000000001,"15min":0.64200000000000002},"min":{"1min":0.40699999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81999999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.64400000000000002,"15min":0.65100000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.84399999999999997,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.44900000000000001}]},{"osd":7,"last update":"Fri Feb 23 00:41:09 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.46899999999999997,"5min":0.439,"15min":0.439},"max":{"1min":0.85799999999999998,"5min":0.875,"15min":0.875},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.622,"15min":0.622},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.495}]}]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297856,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447296,"statfs":{"total":95995035648,"available":95690031104,"internally_reserved":0,"allocated":778240,"data_stored":391557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 23 00:41:34 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.36899999999999999,"15min":0.33500000000000002},"min":{"1min":0.223,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.42999999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.40200000000000002,"15min":0.35299999999999998},"min":{"1min":0.28100000000000003,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.48899999999999999,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.39900000000000002}]},{"osd":2,"last update":"Fri Feb 23 00:41:01 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.39200000000000002},"min":{"1min":0.28799999999999998,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.56799999999999995,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.318},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39300000000000002,"15min":0.38},"min":{"1min":0.26200000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.62,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.36399999999999999}]},{"osd":3,"last update":"Fri Feb 23 00:41:28 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.434,"15min":0.44400000000000001},"min":{"1min":0.28199999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.54400000000000004,"5min":0.754,"15min":0.754},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.48499999999999999,"15min":0.499},"min":{"1min":0.313,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.64700000000000002,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.42699999999999999}]},{"osd":4,"last update":"Fri Feb 23 00:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.65700000000000003,"15min":0.69199999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83099999999999996,"5min":3.2909999999999999,"15min":3.2909999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.69599999999999995,"15min":0.74099999999999999},"min":{"1min":0.42599999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.83699999999999997,"5min":3.319,"15min":3.319},"last":0.51400000000000001}]},{"osd":5,"last update":"Fri Feb 23 00:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59499999999999997,"15min":0.60799999999999998},"min":{"1min":0.42599999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.71999999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.65900000000000003,"15min":0.66900000000000004},"min":{"1min":0.45200000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.73199999999999998,"5min":1.131,"15min":1.131},"last":0.629}]},{"osd":6,"last update":"Fri Feb 23 00:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.63300000000000001,"15min":0.63500000000000001},"min":{"1min":0.45000000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.85899999999999999,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.58199999999999996}]},{"osd":7,"last update":"Fri Feb 23 00:41:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.66100000000000003,"15min":0.67300000000000004},"min":{"1min":0.41699999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.67300000000000004,"5min":1.163,"15min":1.163},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.48399999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.75600000000000001,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.54300000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297840,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447312,"statfs":{"total":95995035648,"available":95690047488,"internally_reserved":0,"allocated":761856,"data_stored":390938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:40:54 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.38300000000000001,"15min":0.378},"min":{"1min":0.245,"5min":0.222,"15min":0.222},"max":{"1min":0.57399999999999995,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39800000000000002,"15min":0.38600000000000001},"min":{"1min":0.27100000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.29999999999999999}]},{"osd":1,"last update":"Fri Feb 23 00:40:54 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39000000000000001,"15min":0.376},"min":{"1min":0.27500000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.79800000000000004,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41599999999999998,"15min":0.40000000000000002},"min":{"1min":0.28399999999999997,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.60899999999999999,"5min":0.745,"15min":0.745},"last":0.39200000000000002}]},{"osd":3,"last update":"Fri Feb 23 00:41:25 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.45100000000000001,"15min":0.47999999999999998},"min":{"1min":0.30599999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.622,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.44900000000000001,"15min":0.46600000000000003},"min":{"1min":0.29399999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.53900000000000003,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.37}]},{"osd":4,"last update":"Fri Feb 23 00:40:54 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.59699999999999998},"min":{"1min":0.43099999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.374,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.45500000000000002}]},{"osd":5,"last update":"Fri Feb 23 00:41:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57699999999999996,"15min":0.58199999999999996},"min":{"1min":0.41799999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.748,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.47099999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0740000000000001,"5min":1.089,"15min":1.089},"last":0.54300000000000004}]},{"osd":6,"last update":"Fri Feb 23 00:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.624,"15min":0.624},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.625},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59999999999999998,"15min":0.59799999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.63200000000000001}]},{"osd":7,"last update":"Fri Feb 23 00:41:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.66800000000000004,"15min":0.67200000000000004},"min":{"1min":0.50900000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.64100000000000001,"15min":0.64800000000000002},"min":{"1min":0.42799999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.044,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.60499999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149750,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297872,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447280,"statfs":{"total":95995035648,"available":95690014720,"internally_reserved":0,"allocated":794624,"data_stored":391485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:41:29 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.41599999999999998,"15min":0.435},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.60999999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.43099999999999999,"15min":0.45300000000000001},"min":{"1min":0.28100000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.52900000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.41799999999999998}]},{"osd":1,"last update":"Fri Feb 23 00:41:29 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.45100000000000001,"15min":0.47299999999999998},"min":{"1min":0.23400000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.60399999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.42799999999999999,"15min":0.44800000000000001},"min":{"1min":0.28699999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.51100000000000001,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.38}]},{"osd":2,"last update":"Fri Feb 23 00:41:29 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.47499999999999998,"15min":0.49199999999999999},"min":{"1min":0.32900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.45900000000000002,"15min":0.48099999999999998},"min":{"1min":0.30599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64700000000000002,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.42999999999999999}]},{"osd":4,"last update":"Fri Feb 23 00:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.436,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.79400000000000004,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.58199999999999996,"15min":0.58999999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.83999999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.50700000000000001}]},{"osd":5,"last update":"Fri Feb 23 00:41:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.59099999999999997,"15min":0.60299999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74099999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.61299999999999999,"15min":0.623},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85299999999999998,"5min":1.024,"15min":1.024},"last":0.495}]},{"osd":6,"last update":"Fri Feb 23 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.60699999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.76400000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.628,"15min":0.628},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.88,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.52900000000000003}]},{"osd":7,"last update":"Fri Feb 23 00:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63400000000000001,"15min":0.63600000000000001},"min":{"1min":0.433,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.64900000000000002,"15min":0.65500000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.77300000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.58699999999999997}]}]},{"osd":4,"up_from":32,"seq":137438953522,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297856,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447296,"statfs":{"total":95995035648,"available":95690031104,"internally_reserved":0,"allocated":778240,"data_stored":391183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55600000000000005,"15min":0.57999999999999996},"min":{"1min":0.32200000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66300000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.754},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56799999999999995,"15min":0.58399999999999996},"min":{"1min":0.35699999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71199999999999997,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.44}]},{"osd":1,"last update":"Fri Feb 23 00:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.56100000000000005,"15min":0.58299999999999996},"min":{"1min":0.33000000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.75700000000000001,"5min":1.175,"15min":1.175},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56699999999999995,"15min":0.57499999999999996},"min":{"1min":0.32600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.69999999999999996,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.60499999999999998}]},{"osd":2,"last update":"Fri Feb 23 00:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56999999999999995,"15min":0.57699999999999996},"min":{"1min":0.41399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.753,"5min":1.161,"15min":1.161},"last":0.5},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.57399999999999995,"15min":0.58799999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79000000000000004,"5min":1.113,"15min":1.113},"last":0.54200000000000004}]},{"osd":3,"last update":"Fri Feb 23 00:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57399999999999995,"15min":0.57999999999999996},"min":{"1min":0.38200000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81999999999999995,"5min":1.143,"15min":1.143},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.57199999999999995,"15min":0.58199999999999996},"min":{"1min":0.36199999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79100000000000004,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.68100000000000005}]},{"osd":5,"last update":"Fri Feb 23 00:41:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.39600000000000002,"15min":0.371},"min":{"1min":0.251,"5min":0.246,"15min":0.246},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.622},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.44500000000000001,"15min":0.41599999999999998},"min":{"1min":0.29999999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.71199999999999997}]},{"osd":6,"last update":"Fri Feb 23 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.40200000000000002,"15min":0.39700000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.40300000000000002,"15min":0.39600000000000002},"min":{"1min":0.30499999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.72699999999999998}]},{"osd":7,"last update":"Fri Feb 23 00:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.48999999999999999,"15min":0.48399999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.81499999999999995}]}]},{"osd":5,"up_from":39,"seq":167503724589,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297860,"kb_used_data":764,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447292,"statfs":{"total":95995035648,"available":95690027008,"internally_reserved":0,"allocated":782336,"data_stored":392317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:41:17 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.50800000000000001,"15min":0.50900000000000001},"min":{"1min":0.32200000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.67600000000000005,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.51800000000000002,"15min":0.52300000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64700000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.47399999999999998}]},{"osd":1,"last update":"Fri Feb 23 00:41:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.65800000000000003,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.53300000000000003,"15min":0.54100000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68500000000000005,"5min":1.004,"15min":1.004},"last":0.503}]},{"osd":2,"last update":"Fri Feb 23 00:41:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.53900000000000003,"15min":0.54700000000000004},"min":{"1min":0.38,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.70999999999999996,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.371},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.36099999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69799999999999995,"5min":1.026,"15min":1.026},"last":0.33400000000000002}]},{"osd":3,"last update":"Fri Feb 23 00:41:17 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.52500000000000002,"15min":0.53000000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.629,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55800000000000005,"15min":0.55900000000000005},"min":{"1min":0.41699999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72199999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.39300000000000002}]},{"osd":4,"last update":"Fri Feb 23 00:41:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.39600000000000002,"15min":0.38300000000000001},"min":{"1min":0.27100000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.63700000000000001,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.41299999999999998,"15min":0.39000000000000001},"min":{"1min":0.32400000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.40500000000000003}]},{"osd":6,"last update":"Fri Feb 23 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.44700000000000001,"15min":0.43099999999999999},"min":{"1min":0.29399999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.66700000000000004,"5min":0.754,"15min":0.754},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.46800000000000003,"15min":0.44800000000000001},"min":{"1min":0.35199999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68799999999999994,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.41699999999999998}]},{"osd":7,"last update":"Fri Feb 23 00:41:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51500000000000001,"15min":0.51100000000000001},"min":{"1min":0.34300000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72299999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.435},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.51700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-23T00:41:45.212 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph pg dump --format=json 2024-02-23T00:41:46.070 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:41:46.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:43.874392+0000 osd.4 (osd.4) 50 : cluster [DBG] 2.f deep-scrub ok 2024-02-23T00:41:46.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:44.936754+0000 osd.3 (osd.3) 53 : cluster [DBG] 2.1b scrub starts 2024-02-23T00:41:46.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:44.937773+0000 osd.3 (osd.3) 54 : cluster [DBG] 2.1b scrub ok 2024-02-23T00:41:46.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:45.153013+0000 mgr.y (mgr.14517) 101 : cluster [DBG] pgmap v87: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:41:46.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:43.874392+0000 osd.4 (osd.4) 50 : cluster [DBG] 2.f deep-scrub ok 2024-02-23T00:41:46.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:44.936754+0000 osd.3 (osd.3) 53 : cluster [DBG] 2.1b scrub starts 2024-02-23T00:41:46.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:44.937773+0000 osd.3 (osd.3) 54 : cluster [DBG] 2.1b scrub ok 2024-02-23T00:41:46.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:45.153013+0000 mgr.y (mgr.14517) 101 : cluster [DBG] pgmap v87: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:41:46.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:43.874392+0000 osd.4 (osd.4) 50 : cluster [DBG] 2.f deep-scrub ok 2024-02-23T00:41:46.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:44.936754+0000 osd.3 (osd.3) 53 : cluster [DBG] 2.1b scrub starts 2024-02-23T00:41:46.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:44.937773+0000 osd.3 (osd.3) 54 : cluster [DBG] 2.1b scrub ok 2024-02-23T00:41:46.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:45.153013+0000 mgr.y (mgr.14517) 101 : cluster [DBG] pgmap v87: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:41:47.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:45.924964+0000 osd.7 (osd.7) 51 : cluster [DBG] 6.15 scrub starts 2024-02-23T00:41:47.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:45.927921+0000 osd.7 (osd.7) 52 : cluster [DBG] 6.15 scrub ok 2024-02-23T00:41:47.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:45.977786+0000 osd.3 (osd.3) 55 : cluster [DBG] 4.11 scrub starts 2024-02-23T00:41:47.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:45.979814+0000 osd.3 (osd.3) 56 : cluster [DBG] 4.11 scrub ok 2024-02-23T00:41:47.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:41:46.101335+0000 mon.a (mon.0) 760 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:41:47.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:41:46.102197+0000 mon.a (mon.0) 761 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:41:47.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:41:46.102960+0000 mon.a (mon.0) 762 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:41:47.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:41:46.108205+0000 mon.a (mon.0) 763 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:41:47.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:45.924964+0000 osd.7 (osd.7) 51 : cluster [DBG] 6.15 scrub starts 2024-02-23T00:41:47.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:45.927921+0000 osd.7 (osd.7) 52 : cluster [DBG] 6.15 scrub ok 2024-02-23T00:41:47.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:45.977786+0000 osd.3 (osd.3) 55 : cluster [DBG] 4.11 scrub starts 2024-02-23T00:41:47.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:45.979814+0000 osd.3 (osd.3) 56 : cluster [DBG] 4.11 scrub ok 2024-02-23T00:41:47.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:41:46.101335+0000 mon.a (mon.0) 760 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:41:47.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:41:46.102197+0000 mon.a (mon.0) 761 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:41:47.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:41:46.102960+0000 mon.a (mon.0) 762 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:41:47.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:41:46.108205+0000 mon.a (mon.0) 763 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:41:47.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:45.924964+0000 osd.7 (osd.7) 51 : cluster [DBG] 6.15 scrub starts 2024-02-23T00:41:47.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:45.927921+0000 osd.7 (osd.7) 52 : cluster [DBG] 6.15 scrub ok 2024-02-23T00:41:47.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:45.977786+0000 osd.3 (osd.3) 55 : cluster [DBG] 4.11 scrub starts 2024-02-23T00:41:47.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:45.979814+0000 osd.3 (osd.3) 56 : cluster [DBG] 4.11 scrub ok 2024-02-23T00:41:47.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:41:46.101335+0000 mon.a (mon.0) 760 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:41:47.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:41:46.102197+0000 mon.a (mon.0) 761 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:41:47.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:41:46.102960+0000 mon.a (mon.0) 762 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:41:47.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:41:46.108205+0000 mon.a (mon.0) 763 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:41:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:46.923064+0000 osd.7 (osd.7) 53 : cluster [DBG] 6.1c scrub starts 2024-02-23T00:41:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:46.926547+0000 osd.7 (osd.7) 54 : cluster [DBG] 6.1c scrub ok 2024-02-23T00:41:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:46.998059+0000 osd.3 (osd.3) 57 : cluster [DBG] 5.17 deep-scrub starts 2024-02-23T00:41:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:46.998871+0000 osd.3 (osd.3) 58 : cluster [DBG] 5.17 deep-scrub ok 2024-02-23T00:41:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:47.033128+0000 osd.6 (osd.6) 35 : cluster [DBG] 2.17 scrub starts 2024-02-23T00:41:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:47.034059+0000 osd.6 (osd.6) 36 : cluster [DBG] 2.17 scrub ok 2024-02-23T00:41:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:47.153801+0000 mgr.y (mgr.14517) 102 : cluster [DBG] pgmap v88: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:41:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:46.923064+0000 osd.7 (osd.7) 53 : cluster [DBG] 6.1c scrub starts 2024-02-23T00:41:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:46.926547+0000 osd.7 (osd.7) 54 : cluster [DBG] 6.1c scrub ok 2024-02-23T00:41:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:46.998059+0000 osd.3 (osd.3) 57 : cluster [DBG] 5.17 deep-scrub starts 2024-02-23T00:41:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:46.998871+0000 osd.3 (osd.3) 58 : cluster [DBG] 5.17 deep-scrub ok 2024-02-23T00:41:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:47.033128+0000 osd.6 (osd.6) 35 : cluster [DBG] 2.17 scrub starts 2024-02-23T00:41:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:47.034059+0000 osd.6 (osd.6) 36 : cluster [DBG] 2.17 scrub ok 2024-02-23T00:41:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:47.153801+0000 mgr.y (mgr.14517) 102 : cluster [DBG] pgmap v88: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:41:48.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.096+0000 7f9747c91700 1 -- 172.21.15.47:0/1996507188 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f97400fd510 msgr2=0x7f97400f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:48.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.096+0000 7f9747c91700 1 --2- 172.21.15.47:0/1996507188 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f97400fd510 0x7f97400f6d40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f9734004870 tx=0x7f9734009cb0).stop 2024-02-23T00:41:48.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.096+0000 7f9747c91700 1 -- 172.21.15.47:0/1996507188 shutdown_connections 2024-02-23T00:41:48.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.096+0000 7f9747c91700 1 --2- 172.21.15.47:0/1996507188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97400f7400 0x7f9740103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:48.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.096+0000 7f9747c91700 1 --2- 172.21.15.47:0/1996507188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97400fcbb0 0x7f97400fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:48.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.096+0000 7f9747c91700 1 --2- 172.21.15.47:0/1996507188 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f97400fd510 0x7f97400f6d40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:48.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.096+0000 7f9747c91700 1 -- 172.21.15.47:0/1996507188 >> 172.21.15.47:0/1996507188 conn(0x7f97400f2820 msgr2=0x7f97400f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:41:48.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.096+0000 7f9747c91700 1 -- 172.21.15.47:0/1996507188 shutdown_connections 2024-02-23T00:41:48.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.096+0000 7f9747c91700 1 -- 172.21.15.47:0/1996507188 wait complete. 2024-02-23T00:41:48.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.097+0000 7f9747c91700 1 librados: starting msgr at 2024-02-23T00:41:48.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.097+0000 7f9747c91700 1 librados: starting objecter 2024-02-23T00:41:48.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.097+0000 7f9747c91700 1 Processor -- start 2024-02-23T00:41:48.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.097+0000 7f9747c91700 1 -- start start 2024-02-23T00:41:48.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.097+0000 7f9747c91700 1 librados: setting wanted keys 2024-02-23T00:41:48.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.097+0000 7f9747c91700 1 librados: calling monclient init 2024-02-23T00:41:48.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.097+0000 7f9747c91700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97400f7400 0x7f9740110270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:48.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.097+0000 7f9747c91700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f97400fcbb0 0x7f9740110760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:48.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.097+0000 7f9747c91700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97400fd510 0x7f974010a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:48.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.097+0000 7f9747c91700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974010a900 con 0x7f97400fd510 2024-02-23T00:41:48.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.097+0000 7f9747c91700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974010aa70 con 0x7f97400f7400 2024-02-23T00:41:48.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.097+0000 7f9747c91700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f974010ac30 con 0x7f97400fcbb0 2024-02-23T00:41:48.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.097+0000 7f974522c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f97400fcbb0 0x7f9740110760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:48.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.097+0000 7f974522c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f97400fcbb0 0x7f9740110760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:42734/0 (socket says 172.21.15.47:42734) 2024-02-23T00:41:48.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.097+0000 7f974522c700 1 -- 172.21.15.47:0/3263762493 learned_addr learned my addr 172.21.15.47:0/3263762493 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:41:48.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.097+0000 7f974622e700 1 --2- 172.21.15.47:0/3263762493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97400fd510 0x7f974010a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:48.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.098+0000 7f9745a2d700 1 --2- 172.21.15.47:0/3263762493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97400f7400 0x7f9740110270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:48.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.098+0000 7f974522c700 1 -- 172.21.15.47:0/3263762493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97400f7400 msgr2=0x7f9740110270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:48.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.098+0000 7f974522c700 1 --2- 172.21.15.47:0/3263762493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97400f7400 0x7f9740110270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:48.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.098+0000 7f974522c700 1 -- 172.21.15.47:0/3263762493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97400fd510 msgr2=0x7f974010a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:48.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.098+0000 7f974522c700 1 --2- 172.21.15.47:0/3263762493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97400fd510 0x7f974010a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:48.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.098+0000 7f974522c700 1 -- 172.21.15.47:0/3263762493 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9734005040 con 0x7f97400fcbb0 2024-02-23T00:41:48.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.098+0000 7f974622e700 1 --2- 172.21.15.47:0/3263762493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97400fd510 0x7f974010a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:41:48.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.098+0000 7f974522c700 1 --2- 172.21.15.47:0/3263762493 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f97400fcbb0 0x7f9740110760 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f9734009da0 tx=0x7f9734009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:48.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.098+0000 7f9732ffd700 10 client.?.objecter ms_handle_connect 0x7f97400fcbb0 2024-02-23T00:41:48.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.098+0000 7f9732ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:41:48.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.098+0000 7f9732ffd700 1 -- 172.21.15.47:0/3263762493 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973400e030 con 0x7f97400fcbb0 2024-02-23T00:41:48.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.098+0000 7f9747c91700 1 -- 172.21.15.47:0/3263762493 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f974010aeb0 con 0x7f97400fcbb0 2024-02-23T00:41:48.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.098+0000 7f9732ffd700 1 -- 172.21.15.47:0/3263762493 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f9734013d70 con 0x7f97400fcbb0 2024-02-23T00:41:48.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.098+0000 7f9732ffd700 1 -- 172.21.15.47:0/3263762493 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9734018070 con 0x7f97400fcbb0 2024-02-23T00:41:48.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.098+0000 7f9747c91700 10 client.24629.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:41:48.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.098+0000 7f9747c91700 1 -- 172.21.15.47:0/3263762493 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97401170a0 con 0x7f97400fcbb0 2024-02-23T00:41:48.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.098+0000 7f9747c91700 1 librados: init done 2024-02-23T00:41:48.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.099+0000 7f9747c91700 1 -- 172.21.15.47:0/3263762493 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f974010b2f0 con 0x7f97400fcbb0 2024-02-23T00:41:48.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.100+0000 7f9732ffd700 1 -- 172.21.15.47:0/3263762493 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f9734009240 con 0x7f97400fcbb0 2024-02-23T00:41:48.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.100+0000 7f9732ffd700 1 --2- 172.21.15.47:0/3263762493 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f972c062cc0 0x7f972c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:48.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.100+0000 7f9732ffd700 1 -- 172.21.15.47:0/3263762493 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(95..95 src has 38..95) v4 ==== 7421+0+0 (secure 0 0 0) 0x7f973407e090 con 0x7f97400fcbb0 2024-02-23T00:41:48.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.100+0000 7f9732ffd700 10 client.24629.objecter ms_dispatch 0x7f9740001550 osd_map(95..95 src has 38..95) v4 2024-02-23T00:41:48.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.100+0000 7f9745a2d700 1 --2- 172.21.15.47:0/3263762493 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f972c062cc0 0x7f972c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:48.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.100+0000 7f9732ffd700 3 client.24629.objecter handle_osd_map got epochs [95,95] > 0 2024-02-23T00:41:48.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.100+0000 7f9732ffd700 3 client.24629.objecter handle_osd_map decoding full epoch 95 2024-02-23T00:41:48.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.100+0000 7f9732ffd700 20 client.24629.objecter dump_active .. 0 homeless 2024-02-23T00:41:48.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.101+0000 7f9732ffd700 10 client.24629.objecter ms_handle_connect 0x7f972c062cc0 2024-02-23T00:41:48.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.101+0000 7f9745a2d700 1 --2- 172.21.15.47:0/3263762493 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f972c062cc0 0x7f972c065180 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f973c0048b0 tx=0x7f973c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:48.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.105+0000 7f9732ffd700 1 -- 172.21.15.47:0/3263762493 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f973404ddf0 con 0x7f97400fcbb0 2024-02-23T00:41:48.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:46.923064+0000 osd.7 (osd.7) 53 : cluster [DBG] 6.1c scrub starts 2024-02-23T00:41:48.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:46.926547+0000 osd.7 (osd.7) 54 : cluster [DBG] 6.1c scrub ok 2024-02-23T00:41:48.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:46.998059+0000 osd.3 (osd.3) 57 : cluster [DBG] 5.17 deep-scrub starts 2024-02-23T00:41:48.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:46.998871+0000 osd.3 (osd.3) 58 : cluster [DBG] 5.17 deep-scrub ok 2024-02-23T00:41:48.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:47.033128+0000 osd.6 (osd.6) 35 : cluster [DBG] 2.17 scrub starts 2024-02-23T00:41:48.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:47.034059+0000 osd.6 (osd.6) 36 : cluster [DBG] 2.17 scrub ok 2024-02-23T00:41:48.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:47.153801+0000 mgr.y (mgr.14517) 102 : cluster [DBG] pgmap v88: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:41:48.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.272+0000 7f9747c91700 1 -- 172.21.15.47:0/3263762493 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f974005ef20 con 0x7f972c062cc0 2024-02-23T00:41:48.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.282+0000 7f9732ffd700 1 -- 172.21.15.47:0/3263762493 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+357186 (secure 0 0 0) 0x7f974005ef20 con 0x7f972c062cc0 2024-02-23T00:41:48.289 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:41:48.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.285+0000 7f9747c91700 10 librados: watch_flush enter 2024-02-23T00:41:48.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.285+0000 7f9747c91700 10 librados: watch_flush exit 2024-02-23T00:41:48.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.285+0000 7f9747c91700 20 client.24629.objecter shutdown clearing up homeless session... 2024-02-23T00:41:48.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.285+0000 7f9747c91700 10 client.24629.objecter successfully canceled tick 2024-02-23T00:41:48.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.285+0000 7f9747c91700 1 -- 172.21.15.47:0/3263762493 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f972c062cc0 msgr2=0x7f972c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:48.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.285+0000 7f9747c91700 1 --2- 172.21.15.47:0/3263762493 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f972c062cc0 0x7f972c065180 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f973c0048b0 tx=0x7f973c00b040).stop 2024-02-23T00:41:48.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.285+0000 7f9747c91700 1 -- 172.21.15.47:0/3263762493 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f97400fcbb0 msgr2=0x7f9740110760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:48.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.285+0000 7f9747c91700 1 --2- 172.21.15.47:0/3263762493 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f97400fcbb0 0x7f9740110760 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f9734009da0 tx=0x7f9734009cb0).stop 2024-02-23T00:41:48.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.285+0000 7f9747c91700 1 -- 172.21.15.47:0/3263762493 shutdown_connections 2024-02-23T00:41:48.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.285+0000 7f9747c91700 1 --2- 172.21.15.47:0/3263762493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97400fd510 0x7f974010a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:48.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.285+0000 7f9747c91700 1 --2- 172.21.15.47:0/3263762493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97400f7400 0x7f9740110270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:48.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.285+0000 7f9747c91700 1 --2- 172.21.15.47:0/3263762493 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f972c062cc0 0x7f972c065180 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:48.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.285+0000 7f9747c91700 1 --2- 172.21.15.47:0/3263762493 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f97400fcbb0 0x7f9740110760 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:48.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.285+0000 7f9747c91700 1 -- 172.21.15.47:0/3263762493 >> 172.21.15.47:0/3263762493 conn(0x7f97400f2820 msgr2=0x7f97400f42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:41:48.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.286+0000 7f9747c91700 1 -- 172.21.15.47:0/3263762493 shutdown_connections 2024-02-23T00:41:48.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.286+0000 7f9747c91700 1 -- 172.21.15.47:0/3263762493 wait complete. 2024-02-23T00:41:48.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:48.286+0000 7f9747c91700 1 librados: shutdown 2024-02-23T00:41:48.293 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-02-23T00:41:48.950 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2024-02-23T00:41:47.153242+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5796,"num_objects":224,"num_object_clones":0,"num_object_copies":672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":0,"num_read":1249,"num_read_kb":969,"num_write":620,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":696,"ondisk_log_size":696,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2382828,"kb_used_data":6060,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":747578388,"statfs":{"total":767960285184,"available":765520269312,"internally_reserved":0,"allocated":6205440,"data_stored":3131460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002001"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.403418+0000","last_change":"2024-02-23T00:41:34.000934+0000","last_active":"2024-02-23T00:41:39.403418+0000","last_peered":"2024-02-23T00:41:39.403418+0000","last_clean":"2024-02-23T00:41:39.403418+0000","last_became_active":"2024-02-23T00:39:21.162083+0000","last_became_peered":"2024-02-23T00:39:21.162083+0000","last_unstale":"2024-02-23T00:41:39.403418+0000","last_undegraded":"2024-02-23T00:41:39.403418+0000","last_fullsized":"2024-02-23T00:41:39.403418+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:34.000844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:34.000844+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.404474+0000","last_change":"2024-02-23T00:41:05.965631+0000","last_active":"2024-02-23T00:41:39.404474+0000","last_peered":"2024-02-23T00:41:39.404474+0000","last_clean":"2024-02-23T00:41:39.404474+0000","last_became_active":"2024-02-23T00:39:18.799723+0000","last_became_peered":"2024-02-23T00:39:18.799723+0000","last_unstale":"2024-02-23T00:41:39.404474+0000","last_undegraded":"2024-02-23T00:41:39.404474+0000","last_fullsized":"2024-02-23T00:41:39.404474+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:05.965531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:05.965531+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679892+0000","last_change":"2024-02-23T00:40:58.873093+0000","last_active":"2024-02-23T00:41:39.679892+0000","last_peered":"2024-02-23T00:41:39.679892+0000","last_clean":"2024-02-23T00:41:39.679892+0000","last_became_active":"2024-02-23T00:39:14.782000+0000","last_became_peered":"2024-02-23T00:39:14.782000+0000","last_unstale":"2024-02-23T00:41:39.679892+0000","last_undegraded":"2024-02-23T00:41:39.679892+0000","last_fullsized":"2024-02-23T00:41:39.679892+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_clean_scrub_stamp":"2024-02-23T00:40:58.873013+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"88'50","reported_seq":143,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.402927+0000","last_change":"2024-02-23T00:40:52.840769+0000","last_active":"2024-02-23T00:41:39.402927+0000","last_peered":"2024-02-23T00:41:39.402927+0000","last_clean":"2024-02-23T00:41:39.402927+0000","last_became_active":"2024-02-23T00:39:16.780144+0000","last_became_peered":"2024-02-23T00:39:16.780144+0000","last_unstale":"2024-02-23T00:41:39.402927+0000","last_undegraded":"2024-02-23T00:41:39.402927+0000","last_fullsized":"2024-02-23T00:41:39.402927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'50","last_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_deep_scrub":"88'50","last_deep_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_clean_scrub_stamp":"2024-02-23T00:40:52.840702+0000","log_size":50,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.404733+0000","last_change":"2024-02-23T00:40:27.096850+0000","last_active":"2024-02-23T00:41:39.404733+0000","last_peered":"2024-02-23T00:41:39.404733+0000","last_clean":"2024-02-23T00:41:39.404733+0000","last_became_active":"2024-02-23T00:39:20.795665+0000","last_became_peered":"2024-02-23T00:39:20.795665+0000","last_unstale":"2024-02-23T00:41:39.404733+0000","last_undegraded":"2024-02-23T00:41:39.404733+0000","last_fullsized":"2024-02-23T00:41:39.404733+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:27.096747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:27.096747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679007+0000","last_change":"2024-02-23T00:40:53.812966+0000","last_active":"2024-02-23T00:41:39.679007+0000","last_peered":"2024-02-23T00:41:39.679007+0000","last_clean":"2024-02-23T00:41:39.679007+0000","last_became_active":"2024-02-23T00:39:18.796312+0000","last_became_peered":"2024-02-23T00:39:18.796312+0000","last_unstale":"2024-02-23T00:41:39.679007+0000","last_undegraded":"2024-02-23T00:41:39.679007+0000","last_fullsized":"2024-02-23T00:41:39.679007+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:53.812893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:53.812893+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.404426+0000","last_change":"2024-02-23T00:41:02.853395+0000","last_active":"2024-02-23T00:41:39.404426+0000","last_peered":"2024-02-23T00:41:39.404426+0000","last_clean":"2024-02-23T00:41:39.404426+0000","last_became_active":"2024-02-23T00:39:14.785362+0000","last_became_peered":"2024-02-23T00:39:14.785362+0000","last_unstale":"2024-02-23T00:41:39.404426+0000","last_undegraded":"2024-02-23T00:41:39.404426+0000","last_fullsized":"2024-02-23T00:41:39.404426+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:02.853328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:02.853328+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"68'15","reported_seq":82,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.404668+0000","last_change":"2024-02-23T00:41:33.064109+0000","last_active":"2024-02-23T00:41:39.404668+0000","last_peered":"2024-02-23T00:41:39.404668+0000","last_clean":"2024-02-23T00:41:39.404668+0000","last_became_active":"2024-02-23T00:39:16.784606+0000","last_became_peered":"2024-02-23T00:39:16.784606+0000","last_unstale":"2024-02-23T00:41:39.404668+0000","last_undegraded":"2024-02-23T00:41:39.404668+0000","last_fullsized":"2024-02-23T00:41:39.404668+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:41:33.063992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:33.063992+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.678957+0000","last_change":"2024-02-23T00:40:55.855032+0000","last_active":"2024-02-23T00:41:39.678957+0000","last_peered":"2024-02-23T00:41:39.678957+0000","last_clean":"2024-02-23T00:41:39.678957+0000","last_became_active":"2024-02-23T00:39:20.793590+0000","last_became_peered":"2024-02-23T00:39:20.793590+0000","last_unstale":"2024-02-23T00:41:39.678957+0000","last_undegraded":"2024-02-23T00:41:39.678957+0000","last_fullsized":"2024-02-23T00:41:39.678957+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:55.854964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:55.854964+0000","last_clean_scrub_stamp":"2024-02-23T00:40:55.854964+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.680092+0000","last_change":"2024-02-23T00:40:42.667373+0000","last_active":"2024-02-23T00:41:39.680092+0000","last_peered":"2024-02-23T00:41:39.680092+0000","last_clean":"2024-02-23T00:41:39.680092+0000","last_became_active":"2024-02-23T00:39:18.796952+0000","last_became_peered":"2024-02-23T00:39:18.796952+0000","last_unstale":"2024-02-23T00:41:39.680092+0000","last_undegraded":"2024-02-23T00:41:39.680092+0000","last_fullsized":"2024-02-23T00:41:39.680092+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:42.667291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:42.667291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":52,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-23T00:41:40.926914+0000","last_change":"2024-02-23T00:41:40.926914+0000","last_active":"2024-02-23T00:41:40.926914+0000","last_peered":"2024-02-23T00:41:40.926914+0000","last_clean":"2024-02-23T00:41:40.926914+0000","last_became_active":"2024-02-23T00:39:14.783469+0000","last_became_peered":"2024-02-23T00:39:14.783469+0000","last_unstale":"2024-02-23T00:41:40.926914+0000","last_undegraded":"2024-02-23T00:41:40.926914+0000","last_fullsized":"2024-02-23T00:41:40.926914+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:40.926851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:40.926851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"82'15","reported_seq":75,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.404811+0000","last_change":"2024-02-23T00:40:31.989101+0000","last_active":"2024-02-23T00:41:39.404811+0000","last_peered":"2024-02-23T00:41:39.404811+0000","last_clean":"2024-02-23T00:41:39.404811+0000","last_became_active":"2024-02-23T00:39:16.782762+0000","last_became_peered":"2024-02-23T00:39:16.782762+0000","last_unstale":"2024-02-23T00:41:39.404811+0000","last_undegraded":"2024-02-23T00:41:39.404811+0000","last_fullsized":"2024-02-23T00:41:39.404811+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'15","last_scrub_stamp":"2024-02-23T00:40:31.989027+0000","last_deep_scrub":"82'15","last_deep_scrub_stamp":"2024-02-23T00:40:31.989027+0000","last_clean_scrub_stamp":"2024-02-23T00:40:31.989027+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.678768+0000","last_change":"2024-02-23T00:41:08.906994+0000","last_active":"2024-02-23T00:41:39.678768+0000","last_peered":"2024-02-23T00:41:39.678768+0000","last_clean":"2024-02-23T00:41:39.678768+0000","last_became_active":"2024-02-23T00:39:20.803344+0000","last_became_peered":"2024-02-23T00:39:20.803344+0000","last_unstale":"2024-02-23T00:41:39.678768+0000","last_undegraded":"2024-02-23T00:41:39.678768+0000","last_fullsized":"2024-02-23T00:41:39.678768+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:08.906928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:08.906928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679403+0000","last_change":"2024-02-23T00:41:07.777516+0000","last_active":"2024-02-23T00:41:39.679403+0000","last_peered":"2024-02-23T00:41:39.679403+0000","last_clean":"2024-02-23T00:41:39.679403+0000","last_became_active":"2024-02-23T00:39:18.796431+0000","last_became_peered":"2024-02-23T00:39:18.796431+0000","last_unstale":"2024-02-23T00:41:39.679403+0000","last_undegraded":"2024-02-23T00:41:39.679403+0000","last_fullsized":"2024-02-23T00:41:39.679403+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:07.777435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:07.777435+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679943+0000","last_change":"2024-02-23T00:40:35.620371+0000","last_active":"2024-02-23T00:41:39.679943+0000","last_peered":"2024-02-23T00:41:39.679943+0000","last_clean":"2024-02-23T00:41:39.679943+0000","last_became_active":"2024-02-23T00:39:14.787148+0000","last_became_peered":"2024-02-23T00:39:14.787148+0000","last_unstale":"2024-02-23T00:41:39.679943+0000","last_undegraded":"2024-02-23T00:41:39.679943+0000","last_fullsized":"2024-02-23T00:41:39.679943+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:35.620292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:35.620292+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"68'21","reported_seq":92,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.404796+0000","last_change":"2024-02-23T00:40:51.950549+0000","last_active":"2024-02-23T00:41:39.404796+0000","last_peered":"2024-02-23T00:41:39.404796+0000","last_clean":"2024-02-23T00:41:39.404796+0000","last_became_active":"2024-02-23T00:39:16.786791+0000","last_became_peered":"2024-02-23T00:39:16.786791+0000","last_unstale":"2024-02-23T00:41:39.404796+0000","last_undegraded":"2024-02-23T00:41:39.404796+0000","last_fullsized":"2024-02-23T00:41:39.404796+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'21","last_scrub_stamp":"2024-02-23T00:40:51.950467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:51.950467+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.404170+0000","last_change":"2024-02-23T00:41:29.992520+0000","last_active":"2024-02-23T00:41:39.404170+0000","last_peered":"2024-02-23T00:41:39.404170+0000","last_clean":"2024-02-23T00:41:39.404170+0000","last_became_active":"2024-02-23T00:39:21.162053+0000","last_became_peered":"2024-02-23T00:39:21.162053+0000","last_unstale":"2024-02-23T00:41:39.404170+0000","last_undegraded":"2024-02-23T00:41:39.404170+0000","last_fullsized":"2024-02-23T00:41:39.404170+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:29.992417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:29.992417+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.405021+0000","last_change":"2024-02-23T00:41:01.063755+0000","last_active":"2024-02-23T00:41:39.405021+0000","last_peered":"2024-02-23T00:41:39.405021+0000","last_clean":"2024-02-23T00:41:39.405021+0000","last_became_active":"2024-02-23T00:39:18.794903+0000","last_became_peered":"2024-02-23T00:39:18.794903+0000","last_unstale":"2024-02-23T00:41:39.405021+0000","last_undegraded":"2024-02-23T00:41:39.405021+0000","last_fullsized":"2024-02-23T00:41:39.405021+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_clean_scrub_stamp":"2024-02-23T00:41:01.063660+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":54,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-23T00:41:44.937852+0000","last_change":"2024-02-23T00:41:44.937852+0000","last_active":"2024-02-23T00:41:44.937852+0000","last_peered":"2024-02-23T00:41:44.937852+0000","last_clean":"2024-02-23T00:41:44.937852+0000","last_became_active":"2024-02-23T00:39:14.782163+0000","last_became_peered":"2024-02-23T00:39:14.782163+0000","last_unstale":"2024-02-23T00:41:44.937852+0000","last_undegraded":"2024-02-23T00:41:44.937852+0000","last_fullsized":"2024-02-23T00:41:44.937852+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:44.937776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:44.937776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"68'18","reported_seq":85,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.403959+0000","last_change":"2024-02-23T00:40:28.118091+0000","last_active":"2024-02-23T00:41:39.403959+0000","last_peered":"2024-02-23T00:41:39.403959+0000","last_clean":"2024-02-23T00:41:39.403959+0000","last_became_active":"2024-02-23T00:39:16.782803+0000","last_became_peered":"2024-02-23T00:39:16.782803+0000","last_unstale":"2024-02-23T00:41:39.403959+0000","last_undegraded":"2024-02-23T00:41:39.403959+0000","last_fullsized":"2024-02-23T00:41:39.403959+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'18","last_scrub_stamp":"2024-02-23T00:40:28.118021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:28.118021+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.405432+0000","last_change":"2024-02-23T00:41:16.048306+0000","last_active":"2024-02-23T00:41:39.405432+0000","last_peered":"2024-02-23T00:41:39.405432+0000","last_clean":"2024-02-23T00:41:39.405432+0000","last_became_active":"2024-02-23T00:39:21.159750+0000","last_became_peered":"2024-02-23T00:39:21.159750+0000","last_unstale":"2024-02-23T00:41:39.405432+0000","last_undegraded":"2024-02-23T00:41:39.405432+0000","last_fullsized":"2024-02-23T00:41:39.405432+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:16.048207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:16.048207+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679934+0000","last_change":"2024-02-23T00:40:23.829090+0000","last_active":"2024-02-23T00:41:39.679934+0000","last_peered":"2024-02-23T00:41:39.679934+0000","last_clean":"2024-02-23T00:41:39.679934+0000","last_became_active":"2024-02-23T00:39:18.801039+0000","last_became_peered":"2024-02-23T00:39:18.801039+0000","last_unstale":"2024-02-23T00:41:39.679934+0000","last_undegraded":"2024-02-23T00:41:39.679934+0000","last_fullsized":"2024-02-23T00:41:39.679934+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:23.829017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:23.829017+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.645352+0000","last_change":"2024-02-23T00:40:30.958705+0000","last_active":"2024-02-23T00:41:39.645352+0000","last_peered":"2024-02-23T00:41:39.645352+0000","last_clean":"2024-02-23T00:41:39.645352+0000","last_became_active":"2024-02-23T00:39:14.782494+0000","last_became_peered":"2024-02-23T00:39:14.782494+0000","last_unstale":"2024-02-23T00:41:39.645352+0000","last_undegraded":"2024-02-23T00:41:39.645352+0000","last_fullsized":"2024-02-23T00:41:39.645352+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:30.958634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:30.958634+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"67'15","reported_seq":76,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-23T00:41:41.121005+0000","last_change":"2024-02-23T00:41:41.121005+0000","last_active":"2024-02-23T00:41:41.121005+0000","last_peered":"2024-02-23T00:41:41.121005+0000","last_clean":"2024-02-23T00:41:41.121005+0000","last_became_active":"2024-02-23T00:39:16.787138+0000","last_became_peered":"2024-02-23T00:39:16.787138+0000","last_unstale":"2024-02-23T00:41:41.121005+0000","last_undegraded":"2024-02-23T00:41:41.121005+0000","last_fullsized":"2024-02-23T00:41:41.121005+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:41:41.120939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:41.120939+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679507+0000","last_change":"2024-02-23T00:40:49.777703+0000","last_active":"2024-02-23T00:41:39.679507+0000","last_peered":"2024-02-23T00:41:39.679507+0000","last_clean":"2024-02-23T00:41:39.679507+0000","last_became_active":"2024-02-23T00:39:20.795524+0000","last_became_peered":"2024-02-23T00:39:20.795524+0000","last_unstale":"2024-02-23T00:41:39.679507+0000","last_undegraded":"2024-02-23T00:41:39.679507+0000","last_fullsized":"2024-02-23T00:41:39.679507+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:49.777636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:49.777636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679788+0000","last_change":"2024-02-23T00:40:21.510727+0000","last_active":"2024-02-23T00:41:39.679788+0000","last_peered":"2024-02-23T00:41:39.679788+0000","last_clean":"2024-02-23T00:41:39.679788+0000","last_became_active":"2024-02-23T00:39:18.790602+0000","last_became_peered":"2024-02-23T00:39:18.790602+0000","last_unstale":"2024-02-23T00:41:39.679788+0000","last_undegraded":"2024-02-23T00:41:39.679788+0000","last_fullsized":"2024-02-23T00:41:39.679788+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:21.510658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:21.510658+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.402729+0000","last_change":"2024-02-23T00:40:44.906676+0000","last_active":"2024-02-23T00:41:39.402729+0000","last_peered":"2024-02-23T00:41:39.402729+0000","last_clean":"2024-02-23T00:41:39.402729+0000","last_became_active":"2024-02-23T00:39:14.781560+0000","last_became_peered":"2024-02-23T00:39:14.781560+0000","last_unstale":"2024-02-23T00:41:39.402729+0000","last_undegraded":"2024-02-23T00:41:39.402729+0000","last_fullsized":"2024-02-23T00:41:39.402729+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:44.906599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:44.906599+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"93'23","reported_seq":91,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.645631+0000","last_change":"2024-02-23T00:41:35.954600+0000","last_active":"2024-02-23T00:41:39.645631+0000","last_peered":"2024-02-23T00:41:39.645631+0000","last_clean":"2024-02-23T00:41:39.645631+0000","last_became_active":"2024-02-23T00:39:16.782660+0000","last_became_peered":"2024-02-23T00:39:16.782660+0000","last_unstale":"2024-02-23T00:41:39.645631+0000","last_undegraded":"2024-02-23T00:41:39.645631+0000","last_fullsized":"2024-02-23T00:41:39.645631+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'23","last_scrub_stamp":"2024-02-23T00:41:35.954516+0000","last_deep_scrub":"93'23","last_deep_scrub_stamp":"2024-02-23T00:41:35.954516+0000","last_clean_scrub_stamp":"2024-02-23T00:41:35.954516+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"67'1","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-23T00:41:46.926637+0000","last_change":"2024-02-23T00:41:46.926637+0000","last_active":"2024-02-23T00:41:46.926637+0000","last_peered":"2024-02-23T00:41:46.926637+0000","last_clean":"2024-02-23T00:41:46.926637+0000","last_became_active":"2024-02-23T00:39:20.795085+0000","last_became_peered":"2024-02-23T00:39:20.795085+0000","last_unstale":"2024-02-23T00:41:46.926637+0000","last_undegraded":"2024-02-23T00:41:46.926637+0000","last_fullsized":"2024-02-23T00:41:46.926637+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:41:46.926552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:46.926552+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"93'9","reported_seq":91,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-23T00:41:40.848692+0000","last_change":"2024-02-23T00:41:38.881071+0000","last_active":"2024-02-23T00:41:40.848692+0000","last_peered":"2024-02-23T00:41:40.848692+0000","last_clean":"2024-02-23T00:41:40.848692+0000","last_became_active":"2024-02-23T00:39:18.796680+0000","last_became_peered":"2024-02-23T00:39:18.796680+0000","last_unstale":"2024-02-23T00:41:40.848692+0000","last_undegraded":"2024-02-23T00:41:40.848692+0000","last_fullsized":"2024-02-23T00:41:40.848692+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-02-23T00:41:38.880989+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-02-23T00:41:38.880989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:38.880989+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":61,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.678869+0000","last_change":"2024-02-23T00:41:32.702659+0000","last_active":"2024-02-23T00:41:39.678869+0000","last_peered":"2024-02-23T00:41:39.678869+0000","last_clean":"2024-02-23T00:41:39.678869+0000","last_became_active":"2024-02-23T00:39:14.785363+0000","last_became_peered":"2024-02-23T00:39:14.785363+0000","last_unstale":"2024-02-23T00:41:39.678869+0000","last_undegraded":"2024-02-23T00:41:39.678869+0000","last_fullsized":"2024-02-23T00:41:39.678869+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"80'2","last_scrub_stamp":"2024-02-23T00:41:32.702574+0000","last_deep_scrub":"80'2","last_deep_scrub_stamp":"2024-02-23T00:40:15.942725+0000","last_clean_scrub_stamp":"2024-02-23T00:41:32.702574+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"68'16","reported_seq":77,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679704+0000","last_change":"2024-02-23T00:41:33.901108+0000","last_active":"2024-02-23T00:41:39.679704+0000","last_peered":"2024-02-23T00:41:39.679704+0000","last_clean":"2024-02-23T00:41:39.679704+0000","last_became_active":"2024-02-23T00:39:16.858945+0000","last_became_peered":"2024-02-23T00:39:16.858945+0000","last_unstale":"2024-02-23T00:41:39.679704+0000","last_undegraded":"2024-02-23T00:41:39.679704+0000","last_fullsized":"2024-02-23T00:41:39.679704+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:41:33.901023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:33.901023+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-23T00:41:40.941453+0000","last_change":"2024-02-23T00:41:40.941453+0000","last_active":"2024-02-23T00:41:40.941453+0000","last_peered":"2024-02-23T00:41:40.941453+0000","last_clean":"2024-02-23T00:41:40.941453+0000","last_became_active":"2024-02-23T00:39:21.160376+0000","last_became_peered":"2024-02-23T00:39:21.160376+0000","last_unstale":"2024-02-23T00:41:40.941453+0000","last_undegraded":"2024-02-23T00:41:40.941453+0000","last_fullsized":"2024-02-23T00:41:40.941453+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:40.941380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:40.941380+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679442+0000","last_change":"2024-02-23T00:40:25.581008+0000","last_active":"2024-02-23T00:41:39.679442+0000","last_peered":"2024-02-23T00:41:39.679442+0000","last_clean":"2024-02-23T00:41:39.679442+0000","last_became_active":"2024-02-23T00:39:18.791457+0000","last_became_peered":"2024-02-23T00:39:18.791457+0000","last_unstale":"2024-02-23T00:41:39.679442+0000","last_undegraded":"2024-02-23T00:41:39.679442+0000","last_fullsized":"2024-02-23T00:41:39.679442+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:25.580938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:25.580938+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":53,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-23T00:41:47.034138+0000","last_change":"2024-02-23T00:41:47.034138+0000","last_active":"2024-02-23T00:41:47.034138+0000","last_peered":"2024-02-23T00:41:47.034138+0000","last_clean":"2024-02-23T00:41:47.034138+0000","last_became_active":"2024-02-23T00:39:14.784092+0000","last_became_peered":"2024-02-23T00:39:14.784092+0000","last_unstale":"2024-02-23T00:41:47.034138+0000","last_undegraded":"2024-02-23T00:41:47.034138+0000","last_fullsized":"2024-02-23T00:41:47.034138+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:47.034064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:47.034064+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"68'17","reported_seq":86,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-23T00:41:45.979896+0000","last_change":"2024-02-23T00:41:45.979896+0000","last_active":"2024-02-23T00:41:45.979896+0000","last_peered":"2024-02-23T00:41:45.979896+0000","last_clean":"2024-02-23T00:41:45.979896+0000","last_became_active":"2024-02-23T00:39:16.857232+0000","last_became_peered":"2024-02-23T00:39:16.857232+0000","last_unstale":"2024-02-23T00:41:45.979896+0000","last_undegraded":"2024-02-23T00:41:45.979896+0000","last_fullsized":"2024-02-23T00:41:45.979896+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:41:45.979819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:45.979819+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679629+0000","last_change":"2024-02-23T00:40:54.739607+0000","last_active":"2024-02-23T00:41:39.679629+0000","last_peered":"2024-02-23T00:41:39.679629+0000","last_clean":"2024-02-23T00:41:39.679629+0000","last_became_active":"2024-02-23T00:39:20.795351+0000","last_became_peered":"2024-02-23T00:39:20.795351+0000","last_unstale":"2024-02-23T00:41:39.679629+0000","last_undegraded":"2024-02-23T00:41:39.679629+0000","last_fullsized":"2024-02-23T00:41:39.679629+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:54.739534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:54.739534+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.645118+0000","last_change":"2024-02-23T00:40:38.957682+0000","last_active":"2024-02-23T00:41:39.645118+0000","last_peered":"2024-02-23T00:41:39.645118+0000","last_clean":"2024-02-23T00:41:39.645118+0000","last_became_active":"2024-02-23T00:39:18.797287+0000","last_became_peered":"2024-02-23T00:39:18.797287+0000","last_unstale":"2024-02-23T00:41:39.645118+0000","last_undegraded":"2024-02-23T00:41:39.645118+0000","last_fullsized":"2024-02-23T00:41:39.645118+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:38.957578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:38.957578+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":54,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-23T00:41:41.650513+0000","last_change":"2024-02-23T00:41:41.650513+0000","last_active":"2024-02-23T00:41:41.650513+0000","last_peered":"2024-02-23T00:41:41.650513+0000","last_clean":"2024-02-23T00:41:41.650513+0000","last_became_active":"2024-02-23T00:39:14.783325+0000","last_became_peered":"2024-02-23T00:39:14.783325+0000","last_unstale":"2024-02-23T00:41:41.650513+0000","last_undegraded":"2024-02-23T00:41:41.650513+0000","last_fullsized":"2024-02-23T00:41:41.650513+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:41.650434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:41.650434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"67'4","reported_seq":53,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.402981+0000","last_change":"2024-02-23T00:41:20.947463+0000","last_active":"2024-02-23T00:41:39.402981+0000","last_peered":"2024-02-23T00:41:39.402981+0000","last_clean":"2024-02-23T00:41:39.402981+0000","last_became_active":"2024-02-23T00:39:16.785830+0000","last_became_peered":"2024-02-23T00:39:16.785830+0000","last_unstale":"2024-02-23T00:41:39.402981+0000","last_undegraded":"2024-02-23T00:41:39.402981+0000","last_fullsized":"2024-02-23T00:41:39.402981+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'4","last_scrub_stamp":"2024-02-23T00:41:20.947377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:20.947377+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.402628+0000","last_change":"2024-02-23T00:40:56.892857+0000","last_active":"2024-02-23T00:41:39.402628+0000","last_peered":"2024-02-23T00:41:39.402628+0000","last_clean":"2024-02-23T00:41:39.402628+0000","last_became_active":"2024-02-23T00:39:20.799053+0000","last_became_peered":"2024-02-23T00:39:20.799053+0000","last_unstale":"2024-02-23T00:41:39.402628+0000","last_undegraded":"2024-02-23T00:41:39.402628+0000","last_fullsized":"2024-02-23T00:41:39.402628+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:56.892797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:56.892797+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679332+0000","last_change":"2024-02-23T00:40:59.669880+0000","last_active":"2024-02-23T00:41:39.679332+0000","last_peered":"2024-02-23T00:41:39.679332+0000","last_clean":"2024-02-23T00:41:39.679332+0000","last_became_active":"2024-02-23T00:39:18.800382+0000","last_became_peered":"2024-02-23T00:39:18.800382+0000","last_unstale":"2024-02-23T00:41:39.679332+0000","last_undegraded":"2024-02-23T00:41:39.679332+0000","last_fullsized":"2024-02-23T00:41:39.679332+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:59.669810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:59.669810+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"60'1","reported_seq":55,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679717+0000","last_change":"2024-02-23T00:40:34.909212+0000","last_active":"2024-02-23T00:41:39.679717+0000","last_peered":"2024-02-23T00:41:39.679717+0000","last_clean":"2024-02-23T00:41:39.679717+0000","last_became_active":"2024-02-23T00:39:14.775804+0000","last_became_peered":"2024-02-23T00:39:14.775804+0000","last_unstale":"2024-02-23T00:41:39.679717+0000","last_undegraded":"2024-02-23T00:41:39.679717+0000","last_fullsized":"2024-02-23T00:41:39.679717+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'1","last_scrub_stamp":"2024-02-23T00:40:34.909135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:34.909135+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"67'11","reported_seq":65,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.405063+0000","last_change":"2024-02-23T00:40:55.928416+0000","last_active":"2024-02-23T00:41:39.405063+0000","last_peered":"2024-02-23T00:41:39.405063+0000","last_clean":"2024-02-23T00:41:39.405063+0000","last_became_active":"2024-02-23T00:39:16.783896+0000","last_became_peered":"2024-02-23T00:39:16.783896+0000","last_unstale":"2024-02-23T00:41:39.405063+0000","last_undegraded":"2024-02-23T00:41:39.405063+0000","last_fullsized":"2024-02-23T00:41:39.405063+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'11","last_scrub_stamp":"2024-02-23T00:40:55.928349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:55.928349+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.678518+0000","last_change":"2024-02-23T00:41:10.892383+0000","last_active":"2024-02-23T00:41:39.678518+0000","last_peered":"2024-02-23T00:41:39.678518+0000","last_clean":"2024-02-23T00:41:39.678518+0000","last_became_active":"2024-02-23T00:39:20.802648+0000","last_became_peered":"2024-02-23T00:39:20.802648+0000","last_unstale":"2024-02-23T00:41:39.678518+0000","last_undegraded":"2024-02-23T00:41:39.678518+0000","last_fullsized":"2024-02-23T00:41:39.678518+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:10.892259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:10.892259+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.403592+0000","last_change":"2024-02-23T00:41:17.931479+0000","last_active":"2024-02-23T00:41:39.403592+0000","last_peered":"2024-02-23T00:41:39.403592+0000","last_clean":"2024-02-23T00:41:39.403592+0000","last_became_active":"2024-02-23T00:39:18.799052+0000","last_became_peered":"2024-02-23T00:39:18.799052+0000","last_unstale":"2024-02-23T00:41:39.403592+0000","last_undegraded":"2024-02-23T00:41:39.403592+0000","last_fullsized":"2024-02-23T00:41:39.403592+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_clean_scrub_stamp":"2024-02-23T00:41:17.931413+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":53,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.645344+0000","last_change":"2024-02-23T00:41:33.993147+0000","last_active":"2024-02-23T00:41:39.645344+0000","last_peered":"2024-02-23T00:41:39.645344+0000","last_clean":"2024-02-23T00:41:39.645344+0000","last_became_active":"2024-02-23T00:39:14.778455+0000","last_became_peered":"2024-02-23T00:39:14.778455+0000","last_unstale":"2024-02-23T00:41:39.645344+0000","last_undegraded":"2024-02-23T00:41:39.645344+0000","last_fullsized":"2024-02-23T00:41:39.645344+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:33.993070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:33.993070+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"67'9","reported_seq":62,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679250+0000","last_change":"2024-02-23T00:40:52.783268+0000","last_active":"2024-02-23T00:41:39.679250+0000","last_peered":"2024-02-23T00:41:39.679250+0000","last_clean":"2024-02-23T00:41:39.679250+0000","last_became_active":"2024-02-23T00:39:16.786936+0000","last_became_peered":"2024-02-23T00:39:16.786936+0000","last_unstale":"2024-02-23T00:41:39.679250+0000","last_undegraded":"2024-02-23T00:41:39.679250+0000","last_fullsized":"2024-02-23T00:41:39.679250+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'9","last_scrub_stamp":"2024-02-23T00:40:52.783146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:52.783146+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.405610+0000","last_change":"2024-02-23T00:40:54.938288+0000","last_active":"2024-02-23T00:41:39.405610+0000","last_peered":"2024-02-23T00:41:39.405610+0000","last_clean":"2024-02-23T00:41:39.405610+0000","last_became_active":"2024-02-23T00:39:20.795210+0000","last_became_peered":"2024-02-23T00:39:20.795210+0000","last_unstale":"2024-02-23T00:41:39.405610+0000","last_undegraded":"2024-02-23T00:41:39.405610+0000","last_fullsized":"2024-02-23T00:41:39.405610+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:54.938209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:54.938209+0000","last_clean_scrub_stamp":"2024-02-23T00:40:54.938209+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"67'8","reported_seq":83,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-23T00:41:40.850613+0000","last_change":"2024-02-23T00:41:25.942914+0000","last_active":"2024-02-23T00:41:40.850613+0000","last_peered":"2024-02-23T00:41:40.850613+0000","last_clean":"2024-02-23T00:41:40.850613+0000","last_became_active":"2024-02-23T00:39:18.787313+0000","last_became_peered":"2024-02-23T00:39:18.787313+0000","last_unstale":"2024-02-23T00:41:40.850613+0000","last_undegraded":"2024-02-23T00:41:40.850613+0000","last_fullsized":"2024-02-23T00:41:40.850613+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:41:25.942822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:25.942822+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679462+0000","last_change":"2024-02-23T00:40:16.547440+0000","last_active":"2024-02-23T00:41:39.679462+0000","last_peered":"2024-02-23T00:41:39.679462+0000","last_clean":"2024-02-23T00:41:39.679462+0000","last_became_active":"2024-02-23T00:39:14.773772+0000","last_became_peered":"2024-02-23T00:39:14.773772+0000","last_unstale":"2024-02-23T00:41:39.679462+0000","last_undegraded":"2024-02-23T00:41:39.679462+0000","last_fullsized":"2024-02-23T00:41:39.679462+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:16.547328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:16.547328+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"68'15","reported_seq":78,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.680472+0000","last_change":"2024-02-23T00:40:42.867136+0000","last_active":"2024-02-23T00:41:39.680472+0000","last_peered":"2024-02-23T00:41:39.680472+0000","last_clean":"2024-02-23T00:41:39.680472+0000","last_became_active":"2024-02-23T00:39:16.858953+0000","last_became_peered":"2024-02-23T00:39:16.858953+0000","last_unstale":"2024-02-23T00:41:39.680472+0000","last_undegraded":"2024-02-23T00:41:39.680472+0000","last_fullsized":"2024-02-23T00:41:39.680472+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:40:42.867043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:42.867043+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679315+0000","last_change":"2024-02-23T00:41:22.855592+0000","last_active":"2024-02-23T00:41:39.679315+0000","last_peered":"2024-02-23T00:41:39.679315+0000","last_clean":"2024-02-23T00:41:39.679315+0000","last_became_active":"2024-02-23T00:39:20.800422+0000","last_became_peered":"2024-02-23T00:39:20.800422+0000","last_unstale":"2024-02-23T00:41:39.679315+0000","last_undegraded":"2024-02-23T00:41:39.679315+0000","last_fullsized":"2024-02-23T00:41:39.679315+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:22.855516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:22.855516+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"67'8","reported_seq":83,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-23T00:41:41.181652+0000","last_change":"2024-02-23T00:41:08.785056+0000","last_active":"2024-02-23T00:41:41.181652+0000","last_peered":"2024-02-23T00:41:41.181652+0000","last_clean":"2024-02-23T00:41:41.181652+0000","last_became_active":"2024-02-23T00:39:18.803566+0000","last_became_peered":"2024-02-23T00:39:18.803566+0000","last_unstale":"2024-02-23T00:41:41.181652+0000","last_undegraded":"2024-02-23T00:41:41.181652+0000","last_fullsized":"2024-02-23T00:41:41.181652+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:41:08.784965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:08.784965+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679223+0000","last_change":"2024-02-23T00:40:29.990344+0000","last_active":"2024-02-23T00:41:39.679223+0000","last_peered":"2024-02-23T00:41:39.679223+0000","last_clean":"2024-02-23T00:41:39.679223+0000","last_became_active":"2024-02-23T00:39:14.785590+0000","last_became_peered":"2024-02-23T00:39:14.785590+0000","last_unstale":"2024-02-23T00:41:39.679223+0000","last_undegraded":"2024-02-23T00:41:39.679223+0000","last_fullsized":"2024-02-23T00:41:39.679223+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_clean_scrub_stamp":"2024-02-23T00:40:29.990255+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"68'22","reported_seq":92,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.404740+0000","last_change":"2024-02-23T00:41:38.950622+0000","last_active":"2024-02-23T00:41:39.404740+0000","last_peered":"2024-02-23T00:41:39.404740+0000","last_clean":"2024-02-23T00:41:39.404740+0000","last_became_active":"2024-02-23T00:39:16.857437+0000","last_became_peered":"2024-02-23T00:39:16.857437+0000","last_unstale":"2024-02-23T00:41:39.404740+0000","last_undegraded":"2024-02-23T00:41:39.404740+0000","last_fullsized":"2024-02-23T00:41:39.404740+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'22","last_scrub_stamp":"2024-02-23T00:41:38.950535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:38.950535+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":39,"num_read_kb":32,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-23T00:41:45.928004+0000","last_change":"2024-02-23T00:41:45.928004+0000","last_active":"2024-02-23T00:41:45.928004+0000","last_peered":"2024-02-23T00:41:45.928004+0000","last_clean":"2024-02-23T00:41:45.928004+0000","last_became_active":"2024-02-23T00:39:21.162036+0000","last_became_peered":"2024-02-23T00:39:21.162036+0000","last_unstale":"2024-02-23T00:41:45.928004+0000","last_undegraded":"2024-02-23T00:41:45.928004+0000","last_fullsized":"2024-02-23T00:41:45.928004+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:45.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:45.927925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.680580+0000","last_change":"2024-02-23T00:41:35.655313+0000","last_active":"2024-02-23T00:41:39.680580+0000","last_peered":"2024-02-23T00:41:39.680580+0000","last_clean":"2024-02-23T00:41:39.680580+0000","last_became_active":"2024-02-23T00:39:18.800297+0000","last_became_peered":"2024-02-23T00:39:18.800297+0000","last_unstale":"2024-02-23T00:41:39.680580+0000","last_undegraded":"2024-02-23T00:41:39.680580+0000","last_fullsized":"2024-02-23T00:41:39.680580+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:35.655209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:35.655209+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.645090+0000","last_change":"2024-02-23T00:40:53.113312+0000","last_active":"2024-02-23T00:41:39.645090+0000","last_peered":"2024-02-23T00:41:39.645090+0000","last_clean":"2024-02-23T00:41:39.645090+0000","last_became_active":"2024-02-23T00:39:14.783164+0000","last_became_peered":"2024-02-23T00:39:14.783164+0000","last_unstale":"2024-02-23T00:41:39.645090+0000","last_undegraded":"2024-02-23T00:41:39.645090+0000","last_fullsized":"2024-02-23T00:41:39.645090+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:53.113232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:53.113232+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"68'12","reported_seq":71,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.404118+0000","last_change":"2024-02-23T00:41:23.904595+0000","last_active":"2024-02-23T00:41:39.404118+0000","last_peered":"2024-02-23T00:41:39.404118+0000","last_clean":"2024-02-23T00:41:39.404118+0000","last_became_active":"2024-02-23T00:39:16.858449+0000","last_became_peered":"2024-02-23T00:39:16.858449+0000","last_unstale":"2024-02-23T00:41:39.404118+0000","last_undegraded":"2024-02-23T00:41:39.404118+0000","last_fullsized":"2024-02-23T00:41:39.404118+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'12","last_scrub_stamp":"2024-02-23T00:41:23.904511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:23.904511+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.402034+0000","last_change":"2024-02-23T00:40:40.360889+0000","last_active":"2024-02-23T00:41:39.402034+0000","last_peered":"2024-02-23T00:41:39.402034+0000","last_clean":"2024-02-23T00:41:39.402034+0000","last_became_active":"2024-02-23T00:39:20.796041+0000","last_became_peered":"2024-02-23T00:39:20.796041+0000","last_unstale":"2024-02-23T00:41:39.402034+0000","last_undegraded":"2024-02-23T00:41:39.402034+0000","last_fullsized":"2024-02-23T00:41:39.402034+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:40.360797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:40.360797+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-23T00:41:46.998945+0000","last_change":"2024-02-23T00:41:46.998945+0000","last_active":"2024-02-23T00:41:46.998945+0000","last_peered":"2024-02-23T00:41:46.998945+0000","last_clean":"2024-02-23T00:41:46.998945+0000","last_became_active":"2024-02-23T00:39:18.798624+0000","last_became_peered":"2024-02-23T00:39:18.798624+0000","last_unstale":"2024-02-23T00:41:46.998945+0000","last_undegraded":"2024-02-23T00:41:46.998945+0000","last_fullsized":"2024-02-23T00:41:46.998945+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:46.998875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:46.998875+0000","last_clean_scrub_stamp":"2024-02-23T00:41:46.998875+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":53,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.402073+0000","last_change":"2024-02-23T00:41:37.165479+0000","last_active":"2024-02-23T00:41:39.402073+0000","last_peered":"2024-02-23T00:41:39.402073+0000","last_clean":"2024-02-23T00:41:39.402073+0000","last_became_active":"2024-02-23T00:39:14.774244+0000","last_became_peered":"2024-02-23T00:39:14.774244+0000","last_unstale":"2024-02-23T00:41:39.402073+0000","last_undegraded":"2024-02-23T00:41:39.402073+0000","last_fullsized":"2024-02-23T00:41:39.402073+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:37.165357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:37.165357+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"68'21","reported_seq":88,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679495+0000","last_change":"2024-02-23T00:39:16.859093+0000","last_active":"2024-02-23T00:41:39.679495+0000","last_peered":"2024-02-23T00:41:39.679495+0000","last_clean":"2024-02-23T00:41:39.679495+0000","last_became_active":"2024-02-23T00:39:16.858760+0000","last_became_peered":"2024-02-23T00:39:16.858760+0000","last_unstale":"2024-02-23T00:41:39.679495+0000","last_undegraded":"2024-02-23T00:41:39.679495+0000","last_fullsized":"2024-02-23T00:41:39.679495+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:39:15.764507+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.403667+0000","last_change":"2024-02-23T00:40:24.129150+0000","last_active":"2024-02-23T00:41:39.403667+0000","last_peered":"2024-02-23T00:41:39.403667+0000","last_clean":"2024-02-23T00:41:39.403667+0000","last_became_active":"2024-02-23T00:39:20.793710+0000","last_became_peered":"2024-02-23T00:39:20.793710+0000","last_unstale":"2024-02-23T00:41:39.403667+0000","last_undegraded":"2024-02-23T00:41:39.403667+0000","last_fullsized":"2024-02-23T00:41:39.403667+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:24.129083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:24.129083+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"95'4","reported_seq":68,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-23T00:41:43.874460+0000","last_change":"2024-02-23T00:41:43.874460+0000","last_active":"2024-02-23T00:41:43.874460+0000","last_peered":"2024-02-23T00:41:43.874460+0000","last_clean":"2024-02-23T00:41:43.874460+0000","last_became_active":"2024-02-23T00:39:14.779205+0000","last_became_peered":"2024-02-23T00:39:14.779205+0000","last_unstale":"2024-02-23T00:41:43.874460+0000","last_undegraded":"2024-02-23T00:41:43.874460+0000","last_fullsized":"2024-02-23T00:41:43.874460+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'4","last_scrub_stamp":"2024-02-23T00:41:43.874397+0000","last_deep_scrub":"95'4","last_deep_scrub_stamp":"2024-02-23T00:41:43.874397+0000","last_clean_scrub_stamp":"2024-02-23T00:41:43.874397+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"68'25","reported_seq":100,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.404773+0000","last_change":"2024-02-23T00:41:08.004573+0000","last_active":"2024-02-23T00:41:39.404773+0000","last_peered":"2024-02-23T00:41:39.404773+0000","last_clean":"2024-02-23T00:41:39.404773+0000","last_became_active":"2024-02-23T00:39:16.783926+0000","last_became_peered":"2024-02-23T00:39:16.783926+0000","last_unstale":"2024-02-23T00:41:39.404773+0000","last_undegraded":"2024-02-23T00:41:39.404773+0000","last_fullsized":"2024-02-23T00:41:39.404773+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'25","last_scrub_stamp":"2024-02-23T00:41:08.004489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:08.004489+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.401987+0000","last_change":"2024-02-23T00:40:49.365809+0000","last_active":"2024-02-23T00:41:39.401987+0000","last_peered":"2024-02-23T00:41:39.401987+0000","last_clean":"2024-02-23T00:41:39.401987+0000","last_became_active":"2024-02-23T00:39:18.796323+0000","last_became_peered":"2024-02-23T00:39:18.796323+0000","last_unstale":"2024-02-23T00:41:39.401987+0000","last_undegraded":"2024-02-23T00:41:39.401987+0000","last_fullsized":"2024-02-23T00:41:39.401987+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_clean_scrub_stamp":"2024-02-23T00:40:49.365711+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"67'1","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.678533+0000","last_change":"2024-02-23T00:41:00.809258+0000","last_active":"2024-02-23T00:41:39.678533+0000","last_peered":"2024-02-23T00:41:39.678533+0000","last_clean":"2024-02-23T00:41:39.678533+0000","last_became_active":"2024-02-23T00:39:20.800858+0000","last_became_peered":"2024-02-23T00:39:20.800858+0000","last_unstale":"2024-02-23T00:41:39.678533+0000","last_undegraded":"2024-02-23T00:41:39.678533+0000","last_fullsized":"2024-02-23T00:41:39.678533+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:41:00.809190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:00.809190+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.680336+0000","last_change":"2024-02-23T00:40:55.723385+0000","last_active":"2024-02-23T00:41:39.680336+0000","last_peered":"2024-02-23T00:41:39.680336+0000","last_clean":"2024-02-23T00:41:39.680336+0000","last_became_active":"2024-02-23T00:39:14.779952+0000","last_became_peered":"2024-02-23T00:39:14.779952+0000","last_unstale":"2024-02-23T00:41:39.680336+0000","last_undegraded":"2024-02-23T00:41:39.680336+0000","last_fullsized":"2024-02-23T00:41:39.680336+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:55.723309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:55.723309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"68'16","reported_seq":76,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679306+0000","last_change":"2024-02-23T00:40:41.772244+0000","last_active":"2024-02-23T00:41:39.679306+0000","last_peered":"2024-02-23T00:41:39.679306+0000","last_clean":"2024-02-23T00:41:39.679306+0000","last_became_active":"2024-02-23T00:39:16.787468+0000","last_became_peered":"2024-02-23T00:39:16.787468+0000","last_unstale":"2024-02-23T00:41:39.679306+0000","last_undegraded":"2024-02-23T00:41:39.679306+0000","last_fullsized":"2024-02-23T00:41:39.679306+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:40:41.772150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:41.772150+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"91'9","reported_seq":84,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-23T00:41:40.926651+0000","last_change":"2024-02-23T00:41:12.843814+0000","last_active":"2024-02-23T00:41:40.926651+0000","last_peered":"2024-02-23T00:41:40.926651+0000","last_clean":"2024-02-23T00:41:40.926651+0000","last_became_active":"2024-02-23T00:39:18.801297+0000","last_became_peered":"2024-02-23T00:39:18.801297+0000","last_unstale":"2024-02-23T00:41:40.926651+0000","last_undegraded":"2024-02-23T00:41:40.926651+0000","last_fullsized":"2024-02-23T00:41:40.926651+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'9","last_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_deep_scrub":"91'9","last_deep_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_clean_scrub_stamp":"2024-02-23T00:41:12.843733+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.404470+0000","last_change":"2024-02-23T00:41:18.910682+0000","last_active":"2024-02-23T00:41:39.404470+0000","last_peered":"2024-02-23T00:41:39.404470+0000","last_clean":"2024-02-23T00:41:39.404470+0000","last_became_active":"2024-02-23T00:39:20.790053+0000","last_became_peered":"2024-02-23T00:39:20.790053+0000","last_unstale":"2024-02-23T00:41:39.404470+0000","last_undegraded":"2024-02-23T00:41:39.404470+0000","last_fullsized":"2024-02-23T00:41:39.404470+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:18.910605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:18.910605+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679312+0000","last_change":"2024-02-23T00:40:53.824228+0000","last_active":"2024-02-23T00:41:39.679312+0000","last_peered":"2024-02-23T00:41:39.679312+0000","last_clean":"2024-02-23T00:41:39.679312+0000","last_became_active":"2024-02-23T00:39:14.784015+0000","last_became_peered":"2024-02-23T00:39:14.784015+0000","last_unstale":"2024-02-23T00:41:39.679312+0000","last_undegraded":"2024-02-23T00:41:39.679312+0000","last_fullsized":"2024-02-23T00:41:39.679312+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:53.824161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:53.824161+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"86'35","reported_seq":61,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-23T00:41:41.180885+0000","last_change":"2024-02-23T00:40:48.803965+0000","last_active":"2024-02-23T00:41:41.180885+0000","last_peered":"2024-02-23T00:41:41.180885+0000","last_clean":"2024-02-23T00:41:41.180885+0000","last_became_active":"2024-02-23T00:40:42.933322+0000","last_became_peered":"2024-02-23T00:40:42.933322+0000","last_unstale":"2024-02-23T00:41:41.180885+0000","last_undegraded":"2024-02-23T00:41:41.180885+0000","last_fullsized":"2024-02-23T00:41:41.180885+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'35","last_scrub_stamp":"2024-02-23T00:40:48.803893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:48.803893+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679759+0000","last_change":"2024-02-23T00:41:19.899792+0000","last_active":"2024-02-23T00:41:39.679759+0000","last_peered":"2024-02-23T00:41:39.679759+0000","last_clean":"2024-02-23T00:41:39.679759+0000","last_became_active":"2024-02-23T00:39:18.802134+0000","last_became_peered":"2024-02-23T00:39:18.802134+0000","last_unstale":"2024-02-23T00:41:39.679759+0000","last_undegraded":"2024-02-23T00:41:39.679759+0000","last_fullsized":"2024-02-23T00:41:39.679759+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:19.899710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:19.899710+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-23T00:41:44.001469+0000","last_change":"2024-02-23T00:41:44.001469+0000","last_active":"2024-02-23T00:41:44.001469+0000","last_peered":"2024-02-23T00:41:44.001469+0000","last_clean":"2024-02-23T00:41:44.001469+0000","last_became_active":"2024-02-23T00:39:21.160457+0000","last_became_peered":"2024-02-23T00:39:21.160457+0000","last_unstale":"2024-02-23T00:41:44.001469+0000","last_undegraded":"2024-02-23T00:41:44.001469+0000","last_fullsized":"2024-02-23T00:41:44.001469+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:44.001408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:44.001408+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"68'5","reported_seq":196,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-23T00:41:46.053430+0000","last_change":"2024-02-23T00:41:00.648420+0000","last_active":"2024-02-23T00:41:46.053430+0000","last_peered":"2024-02-23T00:41:46.053430+0000","last_clean":"2024-02-23T00:41:46.053430+0000","last_became_active":"2024-02-23T00:39:14.779109+0000","last_became_peered":"2024-02-23T00:39:14.779109+0000","last_unstale":"2024-02-23T00:41:46.053430+0000","last_undegraded":"2024-02-23T00:41:46.053430+0000","last_fullsized":"2024-02-23T00:41:46.053430+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'5","last_scrub_stamp":"2024-02-23T00:41:00.648353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:00.648353+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":145,"num_read_kb":140,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.401827+0000","last_change":"2024-02-23T00:41:00.318702+0000","last_active":"2024-02-23T00:41:39.401827+0000","last_peered":"2024-02-23T00:41:39.401827+0000","last_clean":"2024-02-23T00:41:39.401827+0000","last_became_active":"2024-02-23T00:39:14.772411+0000","last_became_peered":"2024-02-23T00:39:14.772411+0000","last_unstale":"2024-02-23T00:41:39.401827+0000","last_undegraded":"2024-02-23T00:41:39.401827+0000","last_fullsized":"2024-02-23T00:41:39.401827+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:00.318633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:00.318633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"68'19","reported_seq":88,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679613+0000","last_change":"2024-02-23T00:40:44.813607+0000","last_active":"2024-02-23T00:41:39.679613+0000","last_peered":"2024-02-23T00:41:39.679613+0000","last_clean":"2024-02-23T00:41:39.679613+0000","last_became_active":"2024-02-23T00:39:16.787484+0000","last_became_peered":"2024-02-23T00:39:16.787484+0000","last_unstale":"2024-02-23T00:41:39.679613+0000","last_undegraded":"2024-02-23T00:41:39.679613+0000","last_fullsized":"2024-02-23T00:41:39.679613+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'19","last_scrub_stamp":"2024-02-23T00:40:44.813506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:44.813506+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.402327+0000","last_change":"2024-02-23T00:40:36.503413+0000","last_active":"2024-02-23T00:41:39.402327+0000","last_peered":"2024-02-23T00:41:39.402327+0000","last_clean":"2024-02-23T00:41:39.402327+0000","last_became_active":"2024-02-23T00:39:18.789330+0000","last_became_peered":"2024-02-23T00:39:18.789330+0000","last_unstale":"2024-02-23T00:41:39.402327+0000","last_undegraded":"2024-02-23T00:41:39.402327+0000","last_fullsized":"2024-02-23T00:41:39.402327+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:36.503342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:36.503342+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.681065+0000","last_change":"2024-02-23T00:40:51.806086+0000","last_active":"2024-02-23T00:41:39.681065+0000","last_peered":"2024-02-23T00:41:39.681065+0000","last_clean":"2024-02-23T00:41:39.681065+0000","last_became_active":"2024-02-23T00:39:20.797226+0000","last_became_peered":"2024-02-23T00:39:20.797226+0000","last_unstale":"2024-02-23T00:41:39.681065+0000","last_undegraded":"2024-02-23T00:41:39.681065+0000","last_fullsized":"2024-02-23T00:41:39.681065+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:51.806007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:51.806007+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.402627+0000","last_change":"2024-02-23T00:40:59.958953+0000","last_active":"2024-02-23T00:41:39.402627+0000","last_peered":"2024-02-23T00:41:39.402627+0000","last_clean":"2024-02-23T00:41:39.402627+0000","last_became_active":"2024-02-23T00:39:14.783489+0000","last_became_peered":"2024-02-23T00:39:14.783489+0000","last_unstale":"2024-02-23T00:41:39.402627+0000","last_undegraded":"2024-02-23T00:41:39.402627+0000","last_fullsized":"2024-02-23T00:41:39.402627+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:59.958850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:59.958850+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679314+0000","last_change":"2024-02-23T00:40:50.760096+0000","last_active":"2024-02-23T00:41:39.679314+0000","last_peered":"2024-02-23T00:41:39.679314+0000","last_clean":"2024-02-23T00:41:39.679314+0000","last_became_active":"2024-02-23T00:39:14.788888+0000","last_became_peered":"2024-02-23T00:39:14.788888+0000","last_unstale":"2024-02-23T00:41:39.679314+0000","last_undegraded":"2024-02-23T00:41:39.679314+0000","last_fullsized":"2024-02-23T00:41:39.679314+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:50.760015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:50.760015+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":57,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679259+0000","last_change":"2024-02-23T00:40:53.770177+0000","last_active":"2024-02-23T00:41:39.679259+0000","last_peered":"2024-02-23T00:41:39.679259+0000","last_clean":"2024-02-23T00:41:39.679259+0000","last_became_active":"2024-02-23T00:39:08.248249+0000","last_became_peered":"2024-02-23T00:39:08.248249+0000","last_unstale":"2024-02-23T00:41:39.679259+0000","last_undegraded":"2024-02-23T00:41:39.679259+0000","last_fullsized":"2024-02-23T00:41:39.679259+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:53.770082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:53.770082+0000","last_clean_scrub_stamp":"2024-02-23T00:40:53.770082+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"67'14","reported_seq":68,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.645548+0000","last_change":"2024-02-23T00:40:41.920396+0000","last_active":"2024-02-23T00:41:39.645548+0000","last_peered":"2024-02-23T00:41:39.645548+0000","last_clean":"2024-02-23T00:41:39.645548+0000","last_became_active":"2024-02-23T00:39:16.858880+0000","last_became_peered":"2024-02-23T00:39:16.858880+0000","last_unstale":"2024-02-23T00:41:39.645548+0000","last_undegraded":"2024-02-23T00:41:39.645548+0000","last_fullsized":"2024-02-23T00:41:39.645548+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'14","last_scrub_stamp":"2024-02-23T00:40:41.920313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:41.920313+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.680255+0000","last_change":"2024-02-23T00:40:48.790072+0000","last_active":"2024-02-23T00:41:39.680255+0000","last_peered":"2024-02-23T00:41:39.680255+0000","last_clean":"2024-02-23T00:41:39.680255+0000","last_became_active":"2024-02-23T00:39:18.793801+0000","last_became_peered":"2024-02-23T00:39:18.793801+0000","last_unstale":"2024-02-23T00:41:39.680255+0000","last_undegraded":"2024-02-23T00:41:39.680255+0000","last_fullsized":"2024-02-23T00:41:39.680255+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:48.789992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:48.789992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.680059+0000","last_change":"2024-02-23T00:41:02.714085+0000","last_active":"2024-02-23T00:41:39.680059+0000","last_peered":"2024-02-23T00:41:39.680059+0000","last_clean":"2024-02-23T00:41:39.680059+0000","last_became_active":"2024-02-23T00:39:20.797242+0000","last_became_peered":"2024-02-23T00:39:20.797242+0000","last_unstale":"2024-02-23T00:41:39.680059+0000","last_undegraded":"2024-02-23T00:41:39.680059+0000","last_fullsized":"2024-02-23T00:41:39.680059+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:02.714020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:02.714020+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679894+0000","last_change":"2024-02-23T00:40:15.584807+0000","last_active":"2024-02-23T00:41:39.679894+0000","last_peered":"2024-02-23T00:41:39.679894+0000","last_clean":"2024-02-23T00:41:39.679894+0000","last_became_active":"2024-02-23T00:39:14.775446+0000","last_became_peered":"2024-02-23T00:39:14.775446+0000","last_unstale":"2024-02-23T00:41:39.679894+0000","last_undegraded":"2024-02-23T00:41:39.679894+0000","last_fullsized":"2024-02-23T00:41:39.679894+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:15.584732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:15.584732+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":52,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679112+0000","last_change":"2024-02-23T00:41:26.836896+0000","last_active":"2024-02-23T00:41:39.679112+0000","last_peered":"2024-02-23T00:41:39.679112+0000","last_clean":"2024-02-23T00:41:39.679112+0000","last_became_active":"2024-02-23T00:39:14.784578+0000","last_became_peered":"2024-02-23T00:39:14.784578+0000","last_unstale":"2024-02-23T00:41:39.679112+0000","last_undegraded":"2024-02-23T00:41:39.679112+0000","last_fullsized":"2024-02-23T00:41:39.679112+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:26.836819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:26.836819+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"67'12","reported_seq":65,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679408+0000","last_change":"2024-02-23T00:40:48.696967+0000","last_active":"2024-02-23T00:41:39.679408+0000","last_peered":"2024-02-23T00:41:39.679408+0000","last_clean":"2024-02-23T00:41:39.679408+0000","last_became_active":"2024-02-23T00:39:16.787143+0000","last_became_peered":"2024-02-23T00:39:16.787143+0000","last_unstale":"2024-02-23T00:41:39.679408+0000","last_undegraded":"2024-02-23T00:41:39.679408+0000","last_fullsized":"2024-02-23T00:41:39.679408+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'12","last_scrub_stamp":"2024-02-23T00:40:48.696895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:48.696895+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.681002+0000","last_change":"2024-02-23T00:40:38.812399+0000","last_active":"2024-02-23T00:41:39.681002+0000","last_peered":"2024-02-23T00:41:39.681002+0000","last_clean":"2024-02-23T00:41:39.681002+0000","last_became_active":"2024-02-23T00:39:18.800378+0000","last_became_peered":"2024-02-23T00:39:18.800378+0000","last_unstale":"2024-02-23T00:41:39.681002+0000","last_undegraded":"2024-02-23T00:41:39.681002+0000","last_fullsized":"2024-02-23T00:41:39.681002+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:38.812327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:38.812327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.678923+0000","last_change":"2024-02-23T00:40:27.859722+0000","last_active":"2024-02-23T00:41:39.678923+0000","last_peered":"2024-02-23T00:41:39.678923+0000","last_clean":"2024-02-23T00:41:39.678923+0000","last_became_active":"2024-02-23T00:39:21.160060+0000","last_became_peered":"2024-02-23T00:39:21.160060+0000","last_unstale":"2024-02-23T00:41:39.678923+0000","last_undegraded":"2024-02-23T00:41:39.678923+0000","last_fullsized":"2024-02-23T00:41:39.678923+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:27.859657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:27.859657+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679919+0000","last_change":"2024-02-23T00:40:41.671069+0000","last_active":"2024-02-23T00:41:39.679919+0000","last_peered":"2024-02-23T00:41:39.679919+0000","last_clean":"2024-02-23T00:41:39.679919+0000","last_became_active":"2024-02-23T00:39:14.783662+0000","last_became_peered":"2024-02-23T00:39:14.783662+0000","last_unstale":"2024-02-23T00:41:39.679919+0000","last_undegraded":"2024-02-23T00:41:39.679919+0000","last_fullsized":"2024-02-23T00:41:39.679919+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:41.670962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:41.670962+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"68'25","reported_seq":98,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679329+0000","last_change":"2024-02-23T00:40:52.751667+0000","last_active":"2024-02-23T00:41:39.679329+0000","last_peered":"2024-02-23T00:41:39.679329+0000","last_clean":"2024-02-23T00:41:39.679329+0000","last_became_active":"2024-02-23T00:39:16.857278+0000","last_became_peered":"2024-02-23T00:39:16.857278+0000","last_unstale":"2024-02-23T00:41:39.679329+0000","last_undegraded":"2024-02-23T00:41:39.679329+0000","last_fullsized":"2024-02-23T00:41:39.679329+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'25","last_scrub_stamp":"2024-02-23T00:40:52.751597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:52.751597+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.645341+0000","last_change":"2024-02-23T00:40:50.029161+0000","last_active":"2024-02-23T00:41:39.645341+0000","last_peered":"2024-02-23T00:41:39.645341+0000","last_clean":"2024-02-23T00:41:39.645341+0000","last_became_active":"2024-02-23T00:39:18.796664+0000","last_became_peered":"2024-02-23T00:39:18.796664+0000","last_unstale":"2024-02-23T00:41:39.645341+0000","last_undegraded":"2024-02-23T00:41:39.645341+0000","last_fullsized":"2024-02-23T00:41:39.645341+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:50.029087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:50.029087+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.405120+0000","last_change":"2024-02-23T00:40:49.937604+0000","last_active":"2024-02-23T00:41:39.405120+0000","last_peered":"2024-02-23T00:41:39.405120+0000","last_clean":"2024-02-23T00:41:39.405120+0000","last_became_active":"2024-02-23T00:39:20.795271+0000","last_became_peered":"2024-02-23T00:39:20.795271+0000","last_unstale":"2024-02-23T00:41:39.405120+0000","last_undegraded":"2024-02-23T00:41:39.405120+0000","last_fullsized":"2024-02-23T00:41:39.405120+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:49.937526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:49.937526+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679591+0000","last_change":"2024-02-23T00:40:36.888232+0000","last_active":"2024-02-23T00:41:39.679591+0000","last_peered":"2024-02-23T00:41:39.679591+0000","last_clean":"2024-02-23T00:41:39.679591+0000","last_became_active":"2024-02-23T00:39:14.779193+0000","last_became_peered":"2024-02-23T00:39:14.779193+0000","last_unstale":"2024-02-23T00:41:39.679591+0000","last_undegraded":"2024-02-23T00:41:39.679591+0000","last_fullsized":"2024-02-23T00:41:39.679591+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:36.888124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:36.888124+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"68'30","reported_seq":109,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.401717+0000","last_change":"2024-02-23T00:41:21.909427+0000","last_active":"2024-02-23T00:41:39.401717+0000","last_peered":"2024-02-23T00:41:39.401717+0000","last_clean":"2024-02-23T00:41:39.401717+0000","last_became_active":"2024-02-23T00:39:16.859002+0000","last_became_peered":"2024-02-23T00:39:16.859002+0000","last_unstale":"2024-02-23T00:41:39.401717+0000","last_undegraded":"2024-02-23T00:41:39.401717+0000","last_fullsized":"2024-02-23T00:41:39.401717+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'30","last_scrub_stamp":"2024-02-23T00:41:21.909340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:21.909340+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.404902+0000","last_change":"2024-02-23T00:40:23.103679+0000","last_active":"2024-02-23T00:41:39.404902+0000","last_peered":"2024-02-23T00:41:39.404902+0000","last_clean":"2024-02-23T00:41:39.404902+0000","last_became_active":"2024-02-23T00:39:18.798829+0000","last_became_peered":"2024-02-23T00:39:18.798829+0000","last_unstale":"2024-02-23T00:41:39.404902+0000","last_undegraded":"2024-02-23T00:41:39.404902+0000","last_fullsized":"2024-02-23T00:41:39.404902+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:23.103586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:23.103586+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.680042+0000","last_change":"2024-02-23T00:40:33.626648+0000","last_active":"2024-02-23T00:41:39.680042+0000","last_peered":"2024-02-23T00:41:39.680042+0000","last_clean":"2024-02-23T00:41:39.680042+0000","last_became_active":"2024-02-23T00:39:21.161928+0000","last_became_peered":"2024-02-23T00:39:21.161928+0000","last_unstale":"2024-02-23T00:41:39.680042+0000","last_undegraded":"2024-02-23T00:41:39.680042+0000","last_fullsized":"2024-02-23T00:41:39.680042+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:33.626500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:33.626500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.645694+0000","last_change":"2024-02-23T00:40:45.971552+0000","last_active":"2024-02-23T00:41:39.645694+0000","last_peered":"2024-02-23T00:41:39.645694+0000","last_clean":"2024-02-23T00:41:39.645694+0000","last_became_active":"2024-02-23T00:39:14.781999+0000","last_became_peered":"2024-02-23T00:39:14.781999+0000","last_unstale":"2024-02-23T00:41:39.645694+0000","last_undegraded":"2024-02-23T00:41:39.645694+0000","last_fullsized":"2024-02-23T00:41:39.645694+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:45.971465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:45.971465+0000","last_clean_scrub_stamp":"2024-02-23T00:40:45.971465+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"68'20","reported_seq":82,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.404631+0000","last_change":"2024-02-23T00:41:05.002039+0000","last_active":"2024-02-23T00:41:39.404631+0000","last_peered":"2024-02-23T00:41:39.404631+0000","last_clean":"2024-02-23T00:41:39.404631+0000","last_became_active":"2024-02-23T00:39:16.780130+0000","last_became_peered":"2024-02-23T00:39:16.780130+0000","last_unstale":"2024-02-23T00:41:39.404631+0000","last_undegraded":"2024-02-23T00:41:39.404631+0000","last_fullsized":"2024-02-23T00:41:39.404631+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'20","last_scrub_stamp":"2024-02-23T00:41:05.001912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:05.001912+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":83,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-23T00:41:40.848105+0000","last_change":"2024-02-23T00:41:09.882111+0000","last_active":"2024-02-23T00:41:40.848105+0000","last_peered":"2024-02-23T00:41:40.848105+0000","last_clean":"2024-02-23T00:41:40.848105+0000","last_became_active":"2024-02-23T00:39:18.798536+0000","last_became_peered":"2024-02-23T00:39:18.798536+0000","last_unstale":"2024-02-23T00:41:40.848105+0000","last_undegraded":"2024-02-23T00:41:40.848105+0000","last_fullsized":"2024-02-23T00:41:40.848105+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:41:09.882012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:09.882012+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.404596+0000","last_change":"2024-02-23T00:40:46.876448+0000","last_active":"2024-02-23T00:41:39.404596+0000","last_peered":"2024-02-23T00:41:39.404596+0000","last_clean":"2024-02-23T00:41:39.404596+0000","last_became_active":"2024-02-23T00:39:21.160443+0000","last_became_peered":"2024-02-23T00:39:21.160443+0000","last_unstale":"2024-02-23T00:41:39.404596+0000","last_undegraded":"2024-02-23T00:41:39.404596+0000","last_fullsized":"2024-02-23T00:41:39.404596+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:46.876380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:46.876380+0000","last_clean_scrub_stamp":"2024-02-23T00:40:46.876380+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.680977+0000","last_change":"2024-02-23T00:41:04.839634+0000","last_active":"2024-02-23T00:41:39.680977+0000","last_peered":"2024-02-23T00:41:39.680977+0000","last_clean":"2024-02-23T00:41:39.680977+0000","last_became_active":"2024-02-23T00:39:18.800777+0000","last_became_peered":"2024-02-23T00:39:18.800777+0000","last_unstale":"2024-02-23T00:41:39.680977+0000","last_undegraded":"2024-02-23T00:41:39.680977+0000","last_fullsized":"2024-02-23T00:41:39.680977+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_clean_scrub_stamp":"2024-02-23T00:41:04.839555+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":52,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-23T00:41:42.978987+0000","last_change":"2024-02-23T00:41:42.978987+0000","last_active":"2024-02-23T00:41:42.978987+0000","last_peered":"2024-02-23T00:41:42.978987+0000","last_clean":"2024-02-23T00:41:42.978987+0000","last_became_active":"2024-02-23T00:39:14.784121+0000","last_became_peered":"2024-02-23T00:39:14.784121+0000","last_unstale":"2024-02-23T00:41:42.978987+0000","last_undegraded":"2024-02-23T00:41:42.978987+0000","last_fullsized":"2024-02-23T00:41:42.978987+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:42.978905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:42.978905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"68'17","reported_seq":103,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.405489+0000","last_change":"2024-02-23T00:41:13.017122+0000","last_active":"2024-02-23T00:41:39.405489+0000","last_peered":"2024-02-23T00:41:39.405489+0000","last_clean":"2024-02-23T00:41:39.405489+0000","last_became_active":"2024-02-23T00:40:42.933759+0000","last_became_peered":"2024-02-23T00:40:42.933759+0000","last_unstale":"2024-02-23T00:41:39.405489+0000","last_undegraded":"2024-02-23T00:41:39.405489+0000","last_fullsized":"2024-02-23T00:41:39.405489+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:41:13.017043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:13.017043+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.680674+0000","last_change":"2024-02-23T00:40:56.890866+0000","last_active":"2024-02-23T00:41:39.680674+0000","last_peered":"2024-02-23T00:41:39.680674+0000","last_clean":"2024-02-23T00:41:39.680674+0000","last_became_active":"2024-02-23T00:39:20.800800+0000","last_became_peered":"2024-02-23T00:39:20.800800+0000","last_unstale":"2024-02-23T00:41:39.680674+0000","last_undegraded":"2024-02-23T00:41:39.680674+0000","last_fullsized":"2024-02-23T00:41:39.680674+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:56.890792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:56.890792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":82,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-23T00:41:40.903008+0000","last_change":"2024-02-23T00:40:30.998799+0000","last_active":"2024-02-23T00:41:40.903008+0000","last_peered":"2024-02-23T00:41:40.903008+0000","last_clean":"2024-02-23T00:41:40.903008+0000","last_became_active":"2024-02-23T00:39:18.800440+0000","last_became_peered":"2024-02-23T00:39:18.800440+0000","last_unstale":"2024-02-23T00:41:40.903008+0000","last_undegraded":"2024-02-23T00:41:40.903008+0000","last_fullsized":"2024-02-23T00:41:40.903008+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:40:30.998729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:30.998729+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679152+0000","last_change":"2024-02-23T00:40:26.845049+0000","last_active":"2024-02-23T00:41:39.679152+0000","last_peered":"2024-02-23T00:41:39.679152+0000","last_clean":"2024-02-23T00:41:39.679152+0000","last_became_active":"2024-02-23T00:39:14.780526+0000","last_became_peered":"2024-02-23T00:39:14.780526+0000","last_unstale":"2024-02-23T00:41:39.679152+0000","last_undegraded":"2024-02-23T00:41:39.679152+0000","last_fullsized":"2024-02-23T00:41:39.679152+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:26.844984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:26.844984+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"67'15","reported_seq":76,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.682330+0000","last_change":"2024-02-23T00:41:39.682283+0000","last_active":"2024-02-23T00:41:39.682330+0000","last_peered":"2024-02-23T00:41:39.682330+0000","last_clean":"2024-02-23T00:41:39.682330+0000","last_became_active":"2024-02-23T00:39:16.789757+0000","last_became_peered":"2024-02-23T00:39:16.789757+0000","last_unstale":"2024-02-23T00:41:39.682330+0000","last_undegraded":"2024-02-23T00:41:39.682330+0000","last_fullsized":"2024-02-23T00:41:39.682330+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:41:39.682182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:39.682182+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.405283+0000","last_change":"2024-02-23T00:41:15.081250+0000","last_active":"2024-02-23T00:41:39.405283+0000","last_peered":"2024-02-23T00:41:39.405283+0000","last_clean":"2024-02-23T00:41:39.405283+0000","last_became_active":"2024-02-23T00:39:20.795097+0000","last_became_peered":"2024-02-23T00:39:20.795097+0000","last_unstale":"2024-02-23T00:41:39.405283+0000","last_undegraded":"2024-02-23T00:41:39.405283+0000","last_fullsized":"2024-02-23T00:41:39.405283+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:15.081153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:15.081153+0000","last_clean_scrub_stamp":"2024-02-23T00:41:15.081153+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"67'8","reported_seq":82,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-23T00:41:40.849991+0000","last_change":"2024-02-23T00:40:58.331136+0000","last_active":"2024-02-23T00:41:40.849991+0000","last_peered":"2024-02-23T00:41:40.849991+0000","last_clean":"2024-02-23T00:41:40.849991+0000","last_became_active":"2024-02-23T00:39:18.799544+0000","last_became_peered":"2024-02-23T00:39:18.799544+0000","last_unstale":"2024-02-23T00:41:40.849991+0000","last_undegraded":"2024-02-23T00:41:40.849991+0000","last_fullsized":"2024-02-23T00:41:40.849991+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:40:58.331048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:58.331048+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679209+0000","last_change":"2024-02-23T00:40:39.825090+0000","last_active":"2024-02-23T00:41:39.679209+0000","last_peered":"2024-02-23T00:41:39.679209+0000","last_clean":"2024-02-23T00:41:39.679209+0000","last_became_active":"2024-02-23T00:39:14.780379+0000","last_became_peered":"2024-02-23T00:39:14.780379+0000","last_unstale":"2024-02-23T00:41:39.679209+0000","last_undegraded":"2024-02-23T00:41:39.679209+0000","last_fullsized":"2024-02-23T00:41:39.679209+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:39.825013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:39.825013+0000","last_clean_scrub_stamp":"2024-02-23T00:40:39.825013+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"67'10","reported_seq":61,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.404961+0000","last_change":"2024-02-23T00:41:14.059424+0000","last_active":"2024-02-23T00:41:39.404961+0000","last_peered":"2024-02-23T00:41:39.404961+0000","last_clean":"2024-02-23T00:41:39.404961+0000","last_became_active":"2024-02-23T00:39:16.777530+0000","last_became_peered":"2024-02-23T00:39:16.777530+0000","last_unstale":"2024-02-23T00:41:39.404961+0000","last_undegraded":"2024-02-23T00:41:39.404961+0000","last_fullsized":"2024-02-23T00:41:39.404961+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'10","last_scrub_stamp":"2024-02-23T00:41:14.059334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:14.059334+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.402328+0000","last_change":"2024-02-23T00:40:54.348946+0000","last_active":"2024-02-23T00:41:39.402328+0000","last_peered":"2024-02-23T00:41:39.402328+0000","last_clean":"2024-02-23T00:41:39.402328+0000","last_became_active":"2024-02-23T00:39:20.800943+0000","last_became_peered":"2024-02-23T00:39:20.800943+0000","last_unstale":"2024-02-23T00:41:39.402328+0000","last_undegraded":"2024-02-23T00:41:39.402328+0000","last_fullsized":"2024-02-23T00:41:39.402328+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:54.348843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:54.348843+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.680134+0000","last_change":"2024-02-23T00:40:35.861920+0000","last_active":"2024-02-23T00:41:39.680134+0000","last_peered":"2024-02-23T00:41:39.680134+0000","last_clean":"2024-02-23T00:41:39.680134+0000","last_became_active":"2024-02-23T00:39:18.799875+0000","last_became_peered":"2024-02-23T00:39:18.799875+0000","last_unstale":"2024-02-23T00:41:39.680134+0000","last_undegraded":"2024-02-23T00:41:39.680134+0000","last_fullsized":"2024-02-23T00:41:39.680134+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:35.861854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:35.861854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:40.128491+0000","last_change":"2024-02-23T00:41:40.128467+0000","last_active":"2024-02-23T00:41:40.128491+0000","last_peered":"2024-02-23T00:41:40.128491+0000","last_clean":"2024-02-23T00:41:40.128491+0000","last_became_active":"2024-02-23T00:40:42.931041+0000","last_became_peered":"2024-02-23T00:40:42.931041+0000","last_unstale":"2024-02-23T00:41:40.128491+0000","last_undegraded":"2024-02-23T00:41:40.128491+0000","last_fullsized":"2024-02-23T00:41:40.128491+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:40.128397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:40.128397+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"67'17","reported_seq":101,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.405317+0000","last_change":"2024-02-23T00:41:02.038717+0000","last_active":"2024-02-23T00:41:39.405317+0000","last_peered":"2024-02-23T00:41:39.405317+0000","last_clean":"2024-02-23T00:41:39.405317+0000","last_became_active":"2024-02-23T00:40:42.933806+0000","last_became_peered":"2024-02-23T00:40:42.933806+0000","last_unstale":"2024-02-23T00:41:39.405317+0000","last_undegraded":"2024-02-23T00:41:39.405317+0000","last_fullsized":"2024-02-23T00:41:39.405317+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'17","last_scrub_stamp":"2024-02-23T00:41:02.038627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:02.038627+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679119+0000","last_change":"2024-02-23T00:40:51.777535+0000","last_active":"2024-02-23T00:41:39.679119+0000","last_peered":"2024-02-23T00:41:39.679119+0000","last_clean":"2024-02-23T00:41:39.679119+0000","last_became_active":"2024-02-23T00:39:20.798989+0000","last_became_peered":"2024-02-23T00:39:20.798989+0000","last_unstale":"2024-02-23T00:41:39.679119+0000","last_undegraded":"2024-02-23T00:41:39.679119+0000","last_fullsized":"2024-02-23T00:41:39.679119+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:51.777405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:51.777405+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.402655+0000","last_change":"2024-02-23T00:40:33.492291+0000","last_active":"2024-02-23T00:41:39.402655+0000","last_peered":"2024-02-23T00:41:39.402655+0000","last_clean":"2024-02-23T00:41:39.402655+0000","last_became_active":"2024-02-23T00:39:18.792718+0000","last_became_peered":"2024-02-23T00:39:18.792718+0000","last_unstale":"2024-02-23T00:41:39.402655+0000","last_undegraded":"2024-02-23T00:41:39.402655+0000","last_fullsized":"2024-02-23T00:41:39.402655+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:33.492224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:33.492224+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.402242+0000","last_change":"2024-02-23T00:40:28.450480+0000","last_active":"2024-02-23T00:41:39.402242+0000","last_peered":"2024-02-23T00:41:39.402242+0000","last_clean":"2024-02-23T00:41:39.402242+0000","last_became_active":"2024-02-23T00:39:14.776705+0000","last_became_peered":"2024-02-23T00:39:14.776705+0000","last_unstale":"2024-02-23T00:41:39.402242+0000","last_undegraded":"2024-02-23T00:41:39.402242+0000","last_fullsized":"2024-02-23T00:41:39.402242+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_clean_scrub_stamp":"2024-02-23T00:40:28.450377+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"88'35","reported_seq":102,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.645205+0000","last_change":"2024-02-23T00:40:54.152313+0000","last_active":"2024-02-23T00:41:39.645205+0000","last_peered":"2024-02-23T00:41:39.645205+0000","last_clean":"2024-02-23T00:41:39.645205+0000","last_became_active":"2024-02-23T00:39:16.857413+0000","last_became_peered":"2024-02-23T00:39:16.857413+0000","last_unstale":"2024-02-23T00:41:39.645205+0000","last_undegraded":"2024-02-23T00:41:39.645205+0000","last_fullsized":"2024-02-23T00:41:39.645205+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'35","last_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_deep_scrub":"88'35","last_deep_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_clean_scrub_stamp":"2024-02-23T00:40:54.152239+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.678584+0000","last_change":"2024-02-23T00:41:05.802853+0000","last_active":"2024-02-23T00:41:39.678584+0000","last_peered":"2024-02-23T00:41:39.678584+0000","last_clean":"2024-02-23T00:41:39.678584+0000","last_became_active":"2024-02-23T00:39:20.802938+0000","last_became_peered":"2024-02-23T00:39:20.802938+0000","last_unstale":"2024-02-23T00:41:39.678584+0000","last_undegraded":"2024-02-23T00:41:39.678584+0000","last_fullsized":"2024-02-23T00:41:39.678584+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:05.802777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:05.802777+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.402107+0000","last_change":"2024-02-23T00:41:02.376871+0000","last_active":"2024-02-23T00:41:39.402107+0000","last_peered":"2024-02-23T00:41:39.402107+0000","last_clean":"2024-02-23T00:41:39.402107+0000","last_became_active":"2024-02-23T00:39:18.794476+0000","last_became_peered":"2024-02-23T00:39:18.794476+0000","last_unstale":"2024-02-23T00:41:39.402107+0000","last_undegraded":"2024-02-23T00:41:39.402107+0000","last_fullsized":"2024-02-23T00:41:39.402107+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:02.376804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:02.376804+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679796+0000","last_change":"2024-02-23T00:40:57.681793+0000","last_active":"2024-02-23T00:41:39.679796+0000","last_peered":"2024-02-23T00:41:39.679796+0000","last_clean":"2024-02-23T00:41:39.679796+0000","last_became_active":"2024-02-23T00:39:14.772493+0000","last_became_peered":"2024-02-23T00:39:14.772493+0000","last_unstale":"2024-02-23T00:41:39.679796+0000","last_undegraded":"2024-02-23T00:41:39.679796+0000","last_fullsized":"2024-02-23T00:41:39.679796+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:57.681723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:57.681723+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"68'15","reported_seq":78,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679808+0000","last_change":"2024-02-23T00:40:47.732636+0000","last_active":"2024-02-23T00:41:39.679808+0000","last_peered":"2024-02-23T00:41:39.679808+0000","last_clean":"2024-02-23T00:41:39.679808+0000","last_became_active":"2024-02-23T00:39:16.783701+0000","last_became_peered":"2024-02-23T00:39:16.783701+0000","last_unstale":"2024-02-23T00:41:39.679808+0000","last_undegraded":"2024-02-23T00:41:39.679808+0000","last_fullsized":"2024-02-23T00:41:39.679808+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:40:47.732524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:47.732524+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.679119+0000","last_change":"2024-02-23T00:40:39.859273+0000","last_active":"2024-02-23T00:41:39.679119+0000","last_peered":"2024-02-23T00:41:39.679119+0000","last_clean":"2024-02-23T00:41:39.679119+0000","last_became_active":"2024-02-23T00:39:21.160429+0000","last_became_peered":"2024-02-23T00:39:21.160429+0000","last_unstale":"2024-02-23T00:41:39.679119+0000","last_undegraded":"2024-02-23T00:41:39.679119+0000","last_fullsized":"2024-02-23T00:41:39.679119+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:39.859197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:39.859197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.402727+0000","last_change":"2024-02-23T00:40:34.533819+0000","last_active":"2024-02-23T00:41:39.402727+0000","last_peered":"2024-02-23T00:41:39.402727+0000","last_clean":"2024-02-23T00:41:39.402727+0000","last_became_active":"2024-02-23T00:39:14.784436+0000","last_became_peered":"2024-02-23T00:39:14.784436+0000","last_unstale":"2024-02-23T00:41:39.402727+0000","last_undegraded":"2024-02-23T00:41:39.402727+0000","last_fullsized":"2024-02-23T00:41:39.402727+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:34.533743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:34.533743+0000","last_clean_scrub_stamp":"2024-02-23T00:40:34.533743+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"68'27","reported_seq":102,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-23T00:41:39.678948+0000","last_change":"2024-02-23T00:40:59.885996+0000","last_active":"2024-02-23T00:41:39.678948+0000","last_peered":"2024-02-23T00:41:39.678948+0000","last_clean":"2024-02-23T00:41:39.678948+0000","last_became_active":"2024-02-23T00:39:16.859401+0000","last_became_peered":"2024-02-23T00:39:16.859401+0000","last_unstale":"2024-02-23T00:41:39.678948+0000","last_undegraded":"2024-02-23T00:41:39.678948+0000","last_fullsized":"2024-02-23T00:41:39.678948+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'27","last_scrub_stamp":"2024-02-23T00:40:59.885908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:59.885908+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"67'8","reported_seq":84,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-23T00:41:40.924386+0000","last_change":"2024-02-23T00:40:44.713885+0000","last_active":"2024-02-23T00:41:40.924386+0000","last_peered":"2024-02-23T00:41:40.924386+0000","last_clean":"2024-02-23T00:41:40.924386+0000","last_became_active":"2024-02-23T00:39:18.793414+0000","last_became_peered":"2024-02-23T00:39:18.793414+0000","last_unstale":"2024-02-23T00:41:40.924386+0000","last_undegraded":"2024-02-23T00:41:40.924386+0000","last_fullsized":"2024-02-23T00:41:40.924386+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:40:44.713825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:44.713825+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1082,"num_read_kb":807,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":616,"ondisk_log_size":616,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":145,"num_read_kb":140,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234018,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297852,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447300,"statfs":{"total":95995035648,"available":95690035200,"internally_reserved":0,"allocated":774144,"data_stored":391332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.374,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.69499999999999995,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.499,"15min":0.505},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.69999999999999996,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.60799999999999998}]},{"osd":1,"last update":"Fri Feb 23 00:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.54400000000000004,"15min":0.55000000000000004},"min":{"1min":0.38400000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.75600000000000001,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.501,"5min":0.54100000000000004,"15min":0.54700000000000004},"min":{"1min":0.39600000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.64600000000000002,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.57099999999999995}]},{"osd":2,"last update":"Fri Feb 23 00:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.53000000000000003,"15min":0.53800000000000003},"min":{"1min":0.40000000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72699999999999998,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.502},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.52600000000000002,"15min":0.53200000000000003},"min":{"1min":0.34899999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.65300000000000002,"5min":1.024,"15min":1.024},"last":0.64000000000000001}]},{"osd":3,"last update":"Fri Feb 23 00:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.52100000000000002,"15min":0.52500000000000002},"min":{"1min":0.38200000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.68400000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.625},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.55600000000000005,"15min":0.56399999999999995},"min":{"1min":0.38300000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72499999999999998,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.47499999999999998}]},{"osd":4,"last update":"Fri Feb 23 00:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.41999999999999998,"15min":0.42799999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.56899999999999995,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.443,"15min":0.45000000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.61099999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.45400000000000001}]},{"osd":5,"last update":"Fri Feb 23 00:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.46600000000000003,"15min":0.47399999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.58399999999999996,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.47099999999999997,"15min":0.47399999999999998},"min":{"1min":0.307,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.625,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.48699999999999999}]},{"osd":6,"last update":"Fri Feb 23 00:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.497,"15min":0.503},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.496,"15min":0.503},"min":{"1min":0.33900000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.65500000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.53700000000000003}]}]},{"osd":6,"up_from":47,"seq":201863462953,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297832,"kb_used_data":736,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447320,"statfs":{"total":95995035648,"available":95690055680,"internally_reserved":0,"allocated":753664,"data_stored":390682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:41:44 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56000000000000005,"15min":0.56899999999999995},"min":{"1min":0.36399999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70899999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.56599999999999995,"15min":0.57999999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.69399999999999995,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.47699999999999998}]},{"osd":1,"last update":"Fri Feb 23 00:41:44 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.54100000000000004,"15min":0.54900000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.67400000000000004,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56799999999999995,"15min":0.58099999999999996},"min":{"1min":0.34100000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.64600000000000002,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.54000000000000004}]},{"osd":2,"last update":"Fri Feb 23 00:41:44 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.57199999999999995,"15min":0.58099999999999996},"min":{"1min":0.378,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.66500000000000004,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.55300000000000005,"15min":0.55800000000000005},"min":{"1min":0.39400000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.68600000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.44800000000000001}]},{"osd":3,"last update":"Fri Feb 23 00:41:44 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57299999999999995,"15min":0.57999999999999996},"min":{"1min":0.43099999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68300000000000005,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.58599999999999997,"15min":0.59199999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71599999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.438}]},{"osd":4,"last update":"Fri Feb 23 00:41:44 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.41899999999999998,"15min":0.40699999999999997},"min":{"1min":0.29899999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.69199999999999995,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.443,"15min":0.43099999999999999},"min":{"1min":0.34000000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71199999999999997,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.377}]},{"osd":5,"last update":"Fri Feb 23 00:41:44 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.43099999999999999,"15min":0.41599999999999998},"min":{"1min":0.32100000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.66600000000000004,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.45700000000000002,"15min":0.441},"min":{"1min":0.34200000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76100000000000001,"5min":0.877,"15min":0.877},"last":0.34200000000000003}]},{"osd":7,"last update":"Fri Feb 23 00:41:11 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.48999999999999999,"15min":0.48799999999999999},"min":{"1min":0.37,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65200000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76700000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.42499999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297860,"kb_used_data":764,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447292,"statfs":{"total":95995035648,"available":95690027008,"internally_reserved":0,"allocated":782336,"data_stored":391966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:41:37 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.379,"15min":0.34799999999999998},"min":{"1min":0.22600000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.60799999999999998,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.40300000000000002,"15min":0.36499999999999999},"min":{"1min":0.27500000000000002,"5min":0.191,"15min":0.191},"max":{"1min":0.57599999999999996,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.26400000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:40:58 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.39600000000000002,"15min":0.39400000000000002},"min":{"1min":0.245,"5min":0.188,"15min":0.188},"max":{"1min":0.53500000000000003,"5min":0.63,"15min":0.63},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.441,"15min":0.435},"min":{"1min":0.29999999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.67900000000000005,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.32100000000000001}]},{"osd":3,"last update":"Fri Feb 23 00:41:31 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.47199999999999998,"15min":0.48499999999999999},"min":{"1min":0.314,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.62,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.443,"15min":0.45500000000000002},"min":{"1min":0.28999999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.66900000000000004,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.41099999999999998}]},{"osd":4,"last update":"Fri Feb 23 00:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60599999999999998,"15min":0.61199999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.875,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61199999999999999,"15min":0.61299999999999999},"min":{"1min":0.39500000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83199999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.72699999999999998}]},{"osd":5,"last update":"Fri Feb 23 00:41:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63500000000000001,"15min":0.65700000000000003},"min":{"1min":0.36899999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84599999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69599999999999995,"15min":0.71599999999999997},"min":{"1min":0.442,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.76000000000000001}]},{"osd":6,"last update":"Fri Feb 23 00:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63700000000000001,"15min":0.64200000000000002},"min":{"1min":0.40699999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81999999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.64400000000000002,"15min":0.65100000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.84399999999999997,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.54300000000000004}]},{"osd":7,"last update":"Fri Feb 23 00:41:09 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.46899999999999997,"5min":0.439,"15min":0.439},"max":{"1min":0.85799999999999998,"5min":0.875,"15min":0.875},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.622,"15min":0.622},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.63700000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297856,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447296,"statfs":{"total":95995035648,"available":95690031104,"internally_reserved":0,"allocated":778240,"data_stored":391557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 23 00:41:34 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.36899999999999999,"15min":0.33500000000000002},"min":{"1min":0.223,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.42999999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.376,"5min":0.40200000000000002,"15min":0.35299999999999998},"min":{"1min":0.28100000000000003,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.48899999999999999,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.34499999999999997}]},{"osd":2,"last update":"Fri Feb 23 00:41:01 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.39200000000000002},"min":{"1min":0.28799999999999998,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.56799999999999995,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39300000000000002,"15min":0.38},"min":{"1min":0.26200000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.62,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.32900000000000001}]},{"osd":3,"last update":"Fri Feb 23 00:41:28 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.434,"15min":0.44400000000000001},"min":{"1min":0.28199999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.54400000000000004,"5min":0.754,"15min":0.754},"last":0.316},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.48499999999999999,"15min":0.499},"min":{"1min":0.313,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.64700000000000002,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.41899999999999998}]},{"osd":4,"last update":"Fri Feb 23 00:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.65700000000000003,"15min":0.69199999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83099999999999996,"5min":3.2909999999999999,"15min":3.2909999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.69599999999999995,"15min":0.74099999999999999},"min":{"1min":0.42599999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.83699999999999997,"5min":3.319,"15min":3.319},"last":0.68899999999999995}]},{"osd":5,"last update":"Fri Feb 23 00:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59499999999999997,"15min":0.60799999999999998},"min":{"1min":0.42599999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.71999999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.65900000000000003,"15min":0.66900000000000004},"min":{"1min":0.45200000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.73199999999999998,"5min":1.131,"15min":1.131},"last":0.69499999999999995}]},{"osd":6,"last update":"Fri Feb 23 00:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.63300000000000001,"15min":0.63500000000000001},"min":{"1min":0.45000000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.85899999999999999,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.56100000000000005}]},{"osd":7,"last update":"Fri Feb 23 00:41:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.66100000000000003,"15min":0.67300000000000004},"min":{"1min":0.41699999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.67300000000000004,"5min":1.163,"15min":1.163},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.48399999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.75600000000000001,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.622}]}]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297840,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447312,"statfs":{"total":95995035648,"available":95690047488,"internally_reserved":0,"allocated":761856,"data_stored":390938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:40:54 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.38300000000000001,"15min":0.378},"min":{"1min":0.245,"5min":0.222,"15min":0.222},"max":{"1min":0.57399999999999995,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.44},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39800000000000002,"15min":0.38600000000000001},"min":{"1min":0.27100000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.40000000000000002}]},{"osd":1,"last update":"Fri Feb 23 00:40:54 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39000000000000001,"15min":0.376},"min":{"1min":0.27500000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.79800000000000004,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41599999999999998,"15min":0.40000000000000002},"min":{"1min":0.28399999999999997,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.60899999999999999,"5min":0.745,"15min":0.745},"last":0.47899999999999998}]},{"osd":3,"last update":"Fri Feb 23 00:41:25 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.45100000000000001,"15min":0.47999999999999998},"min":{"1min":0.30599999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.622,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.44900000000000001,"15min":0.46600000000000003},"min":{"1min":0.29399999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.53900000000000003,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.501}]},{"osd":4,"last update":"Fri Feb 23 00:40:54 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.59699999999999998},"min":{"1min":0.43099999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.374,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.69999999999999996}]},{"osd":5,"last update":"Fri Feb 23 00:41:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57699999999999996,"15min":0.58199999999999996},"min":{"1min":0.41799999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.748,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.47099999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0740000000000001,"5min":1.089,"15min":1.089},"last":0.57799999999999996}]},{"osd":6,"last update":"Fri Feb 23 00:41:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.49299999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.61399999999999999,"15min":0.60199999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.64200000000000002}]},{"osd":7,"last update":"Fri Feb 23 00:41:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.66800000000000004,"15min":0.67200000000000004},"min":{"1min":0.50900000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.64100000000000001,"15min":0.64800000000000002},"min":{"1min":0.42799999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.044,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.68000000000000005}]}]},{"osd":3,"up_from":26,"seq":111669149751,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297872,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447280,"statfs":{"total":95995035648,"available":95690014720,"internally_reserved":0,"allocated":794624,"data_stored":391485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:41:29 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.41599999999999998,"15min":0.435},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.60999999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.312},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.43099999999999999,"15min":0.45300000000000001},"min":{"1min":0.28100000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.52900000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.33700000000000002}]},{"osd":1,"last update":"Fri Feb 23 00:41:29 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.45100000000000001,"15min":0.47299999999999998},"min":{"1min":0.23400000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.60399999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.377},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.42799999999999999,"15min":0.44800000000000001},"min":{"1min":0.28699999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.51100000000000001,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.41199999999999998}]},{"osd":2,"last update":"Fri Feb 23 00:41:29 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.47499999999999998,"15min":0.49199999999999999},"min":{"1min":0.32900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.45900000000000002,"15min":0.48099999999999998},"min":{"1min":0.30599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64700000000000002,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.34999999999999998}]},{"osd":4,"last update":"Fri Feb 23 00:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.436,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.79400000000000004,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.58199999999999996,"15min":0.58999999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.83999999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.53800000000000003}]},{"osd":5,"last update":"Fri Feb 23 00:41:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.59099999999999997,"15min":0.60299999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74099999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.61299999999999999,"15min":0.623},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85299999999999998,"5min":1.024,"15min":1.024},"last":0.51000000000000001}]},{"osd":6,"last update":"Fri Feb 23 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.60699999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.76400000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.497},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.628,"15min":0.628},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.88,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.48899999999999999}]},{"osd":7,"last update":"Fri Feb 23 00:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63400000000000001,"15min":0.63600000000000001},"min":{"1min":0.433,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.64900000000000002,"15min":0.65500000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.77300000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.58999999999999997}]}]},{"osd":4,"up_from":32,"seq":137438953523,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297856,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447296,"statfs":{"total":95995035648,"available":95690031104,"internally_reserved":0,"allocated":778240,"data_stored":391183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55600000000000005,"15min":0.57999999999999996},"min":{"1min":0.32200000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66300000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56799999999999995,"15min":0.58399999999999996},"min":{"1min":0.35699999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71199999999999997,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.41499999999999998}]},{"osd":1,"last update":"Fri Feb 23 00:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.56100000000000005,"15min":0.58299999999999996},"min":{"1min":0.33000000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.75700000000000001,"5min":1.175,"15min":1.175},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56699999999999995,"15min":0.57499999999999996},"min":{"1min":0.32600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.69999999999999996,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.63700000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56999999999999995,"15min":0.57699999999999996},"min":{"1min":0.41399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.753,"5min":1.161,"15min":1.161},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.57399999999999995,"15min":0.58799999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79000000000000004,"5min":1.113,"15min":1.113},"last":0.54100000000000004}]},{"osd":3,"last update":"Fri Feb 23 00:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57399999999999995,"15min":0.57999999999999996},"min":{"1min":0.38200000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81999999999999995,"5min":1.143,"15min":1.143},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.57199999999999995,"15min":0.58199999999999996},"min":{"1min":0.36199999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79100000000000004,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.56399999999999995}]},{"osd":5,"last update":"Fri Feb 23 00:41:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.39600000000000002,"15min":0.371},"min":{"1min":0.251,"5min":0.246,"15min":0.246},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.502},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.44500000000000001,"15min":0.41599999999999998},"min":{"1min":0.29999999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.66200000000000003}]},{"osd":6,"last update":"Fri Feb 23 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.432,"15min":0.40699999999999997},"min":{"1min":0.29899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.435,"15min":0.40699999999999997},"min":{"1min":0.33500000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.61099999999999999}]},{"osd":7,"last update":"Fri Feb 23 00:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.48999999999999999,"15min":0.48399999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.628}]}]},{"osd":5,"up_from":39,"seq":167503724590,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297860,"kb_used_data":764,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447292,"statfs":{"total":95995035648,"available":95690027008,"internally_reserved":0,"allocated":782336,"data_stored":392317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:41:17 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.50800000000000001,"15min":0.50900000000000001},"min":{"1min":0.32200000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.67600000000000005,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.51800000000000002,"15min":0.52300000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64700000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.55400000000000005}]},{"osd":1,"last update":"Fri Feb 23 00:41:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.65800000000000003,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.625},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.53300000000000003,"15min":0.54100000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68500000000000005,"5min":1.004,"15min":1.004},"last":0.60699999999999998}]},{"osd":2,"last update":"Fri Feb 23 00:41:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.53900000000000003,"15min":0.54700000000000004},"min":{"1min":0.38,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.70999999999999996,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.36099999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69799999999999995,"5min":1.026,"15min":1.026},"last":0.70999999999999996}]},{"osd":3,"last update":"Fri Feb 23 00:41:17 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.52500000000000002,"15min":0.53000000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.629,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55800000000000005,"15min":0.55900000000000005},"min":{"1min":0.41699999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72199999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.745}]},{"osd":4,"last update":"Fri Feb 23 00:41:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.39600000000000002,"15min":0.38300000000000001},"min":{"1min":0.27100000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.63700000000000001,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.501},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.41299999999999998,"15min":0.39000000000000001},"min":{"1min":0.32400000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.58599999999999997}]},{"osd":6,"last update":"Fri Feb 23 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.44700000000000001,"15min":0.43099999999999999},"min":{"1min":0.29399999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.66700000000000004,"5min":0.754,"15min":0.754},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.46800000000000003,"15min":0.44800000000000001},"min":{"1min":0.35199999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68799999999999994,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.57699999999999996}]},{"osd":7,"last update":"Fri Feb 23 00:41:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51500000000000001,"15min":0.51100000000000001},"min":{"1min":0.34300000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72299999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.64500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-23T00:41:48.956 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-23T00:41:48.956 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd dump 2024-02-23T00:41:49.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:48.030546+0000 osd.3 (osd.3) 59 : cluster [DBG] 6.b scrub starts 2024-02-23T00:41:49.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:48.031628+0000 osd.3 (osd.3) 60 : cluster [DBG] 6.b scrub ok 2024-02-23T00:41:49.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:48.039162+0000 osd.6 (osd.6) 37 : cluster [DBG] 4.5 scrub starts 2024-02-23T00:41:49.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:48.041100+0000 osd.6 (osd.6) 38 : cluster [DBG] 4.5 scrub ok 2024-02-23T00:41:49.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:48.149685+0000 osd.0 (osd.0) 41 : cluster [DBG] 4.16 scrub starts 2024-02-23T00:41:49.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:48.151307+0000 osd.0 (osd.0) 42 : cluster [DBG] 4.16 scrub ok 2024-02-23T00:41:49.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:41:48.273907+0000 mgr.y (mgr.14517) 103 : audit [DBG] from='client.24629 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:41:49.240 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:41:49.306 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:48.030546+0000 osd.3 (osd.3) 59 : cluster [DBG] 6.b scrub starts 2024-02-23T00:41:49.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:48.031628+0000 osd.3 (osd.3) 60 : cluster [DBG] 6.b scrub ok 2024-02-23T00:41:49.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:48.039162+0000 osd.6 (osd.6) 37 : cluster [DBG] 4.5 scrub starts 2024-02-23T00:41:49.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:48.041100+0000 osd.6 (osd.6) 38 : cluster [DBG] 4.5 scrub ok 2024-02-23T00:41:49.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:48.149685+0000 osd.0 (osd.0) 41 : cluster [DBG] 4.16 scrub starts 2024-02-23T00:41:49.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:48.151307+0000 osd.0 (osd.0) 42 : cluster [DBG] 4.16 scrub ok 2024-02-23T00:41:49.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:41:48.273907+0000 mgr.y (mgr.14517) 103 : audit [DBG] from='client.24629 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:41:49.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:48.030546+0000 osd.3 (osd.3) 59 : cluster [DBG] 6.b scrub starts 2024-02-23T00:41:49.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:48.031628+0000 osd.3 (osd.3) 60 : cluster [DBG] 6.b scrub ok 2024-02-23T00:41:49.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:48.039162+0000 osd.6 (osd.6) 37 : cluster [DBG] 4.5 scrub starts 2024-02-23T00:41:49.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:48.041100+0000 osd.6 (osd.6) 38 : cluster [DBG] 4.5 scrub ok 2024-02-23T00:41:49.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:48.149685+0000 osd.0 (osd.0) 41 : cluster [DBG] 4.16 scrub starts 2024-02-23T00:41:49.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:48.151307+0000 osd.0 (osd.0) 42 : cluster [DBG] 4.16 scrub ok 2024-02-23T00:41:49.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:41:48.273907+0000 mgr.y (mgr.14517) 103 : audit [DBG] from='client.24629 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:41:50.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:49.154127+0000 mgr.y (mgr.14517) 104 : cluster [DBG] pgmap v89: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-23T00:41:50.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:49.195806+0000 osd.0 (osd.0) 43 : cluster [DBG] 3.1 scrub starts 2024-02-23T00:41:50.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:49.196949+0000 osd.0 (osd.0) 44 : cluster [DBG] 3.1 scrub ok 2024-02-23T00:41:50.306 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:49.154127+0000 mgr.y (mgr.14517) 104 : cluster [DBG] pgmap v89: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-23T00:41:50.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:49.195806+0000 osd.0 (osd.0) 43 : cluster [DBG] 3.1 scrub starts 2024-02-23T00:41:50.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:49.196949+0000 osd.0 (osd.0) 44 : cluster [DBG] 3.1 scrub ok 2024-02-23T00:41:50.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:49.154127+0000 mgr.y (mgr.14517) 104 : cluster [DBG] pgmap v89: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-23T00:41:50.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:49.195806+0000 osd.0 (osd.0) 43 : cluster [DBG] 3.1 scrub starts 2024-02-23T00:41:50.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:49.196949+0000 osd.0 (osd.0) 44 : cluster [DBG] 3.1 scrub ok 2024-02-23T00:41:50.806 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:41:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:41:50] "GET /metrics HTTP/1.1" 200 172161 "" "Prometheus/2.33.4" 2024-02-23T00:41:51.149 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:49.925665+0000 osd.4 (osd.4) 51 : cluster [DBG] 5.e scrub starts 2024-02-23T00:41:51.150 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:49.927319+0000 osd.4 (osd.4) 52 : cluster [DBG] 5.e scrub ok 2024-02-23T00:41:51.150 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:49.925665+0000 osd.4 (osd.4) 51 : cluster [DBG] 5.e scrub starts 2024-02-23T00:41:51.150 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:49.927319+0000 osd.4 (osd.4) 52 : cluster [DBG] 5.e scrub ok 2024-02-23T00:41:51.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:49.925665+0000 osd.4 (osd.4) 51 : cluster [DBG] 5.e scrub starts 2024-02-23T00:41:51.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:49.927319+0000 osd.4 (osd.4) 52 : cluster [DBG] 5.e scrub ok 2024-02-23T00:41:51.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.424+0000 7f370dece700 1 -- 172.21.15.47:0/1367054951 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f37080f9670 msgr2=0x7f3708101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:51.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.424+0000 7f370dece700 1 --2- 172.21.15.47:0/1367054951 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f37080f9670 0x7f3708101d80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f36f8004870 tx=0x7f36f8009cb0).stop 2024-02-23T00:41:51.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.424+0000 7f370dece700 1 -- 172.21.15.47:0/1367054951 shutdown_connections 2024-02-23T00:41:51.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.424+0000 7f370dece700 1 --2- 172.21.15.47:0/1367054951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3708102270 0x7f3708104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:51.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.424+0000 7f370dece700 1 --2- 172.21.15.47:0/1367054951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37080f8d10 0x7f37080f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:51.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.424+0000 7f370dece700 1 --2- 172.21.15.47:0/1367054951 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f37080f9670 0x7f3708101d80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:51.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.424+0000 7f370dece700 1 -- 172.21.15.47:0/1367054951 >> 172.21.15.47:0/1367054951 conn(0x7f37080f2820 msgr2=0x7f37080f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:41:51.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.424+0000 7f370dece700 1 -- 172.21.15.47:0/1367054951 shutdown_connections 2024-02-23T00:41:51.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.424+0000 7f370dece700 1 -- 172.21.15.47:0/1367054951 wait complete. 2024-02-23T00:41:51.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.424+0000 7f370dece700 1 librados: starting msgr at 2024-02-23T00:41:51.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.424+0000 7f370dece700 1 librados: starting objecter 2024-02-23T00:41:51.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.424+0000 7f370dece700 1 Processor -- start 2024-02-23T00:41:51.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.425+0000 7f370dece700 1 -- start start 2024-02-23T00:41:51.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.425+0000 7f370dece700 1 librados: setting wanted keys 2024-02-23T00:41:51.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.425+0000 7f370dece700 1 librados: calling monclient init 2024-02-23T00:41:51.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.425+0000 7f370dece700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f37080f8d10 0x7f370810dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:51.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.425+0000 7f370dece700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37080f9670 0x7f370810e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:51.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.425+0000 7f370dece700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3708102270 0x7f3708108190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:51.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.425+0000 7f370dece700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3708108680 con 0x7f37080f9670 2024-02-23T00:41:51.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.425+0000 7f370dece700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37081087f0 con 0x7f3708102270 2024-02-23T00:41:51.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.425+0000 7f370dece700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37081089b0 con 0x7f37080f8d10 2024-02-23T00:41:51.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.425+0000 7f3706ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37080f9670 0x7f370810e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:51.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.425+0000 7f3706ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37080f9670 0x7f370810e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54888/0 (socket says 172.21.15.47:54888) 2024-02-23T00:41:51.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.425+0000 7f3706ffd700 1 -- 172.21.15.47:0/4146025896 learned_addr learned my addr 172.21.15.47:0/4146025896 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:41:51.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.425+0000 7f37077fe700 1 --2- 172.21.15.47:0/4146025896 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f37080f8d10 0x7f370810dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:51.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.425+0000 7f3707fff700 1 --2- 172.21.15.47:0/4146025896 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3708102270 0x7f3708108190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:51.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.425+0000 7f3706ffd700 1 -- 172.21.15.47:0/4146025896 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f37080f8d10 msgr2=0x7f370810dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:51.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.425+0000 7f3706ffd700 1 --2- 172.21.15.47:0/4146025896 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f37080f8d10 0x7f370810dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:51.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.425+0000 7f3706ffd700 1 -- 172.21.15.47:0/4146025896 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3708102270 msgr2=0x7f3708108190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:51.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.425+0000 7f3706ffd700 1 --2- 172.21.15.47:0/4146025896 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3708102270 0x7f3708108190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:51.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.425+0000 7f3706ffd700 1 -- 172.21.15.47:0/4146025896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37000097c0 con 0x7f37080f9670 2024-02-23T00:41:51.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.425+0000 7f37077fe700 1 --2- 172.21.15.47:0/4146025896 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f37080f8d10 0x7f370810dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:41:51.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.425+0000 7f3706ffd700 1 --2- 172.21.15.47:0/4146025896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37080f9670 0x7f370810e4e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f36f8009da0 tx=0x7f36f8009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:51.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.425+0000 7f3704ff9700 10 client.?.objecter ms_handle_connect 0x7f37080f9670 2024-02-23T00:41:51.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.425+0000 7f3704ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:41:51.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.426+0000 7f3704ff9700 1 -- 172.21.15.47:0/4146025896 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f800e030 con 0x7f37080f9670 2024-02-23T00:41:51.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.426+0000 7f3704ff9700 1 -- 172.21.15.47:0/4146025896 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f36f8013d70 con 0x7f37080f9670 2024-02-23T00:41:51.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.426+0000 7f370dece700 1 -- 172.21.15.47:0/4146025896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36f8005040 con 0x7f37080f9670 2024-02-23T00:41:51.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.426+0000 7f3704ff9700 1 -- 172.21.15.47:0/4146025896 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f8018070 con 0x7f37080f9670 2024-02-23T00:41:51.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.426+0000 7f370dece700 10 client.14745.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:41:51.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.426+0000 7f370dece700 1 -- 172.21.15.47:0/4146025896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3708108e50 con 0x7f37080f9670 2024-02-23T00:41:51.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.426+0000 7f370dece700 1 librados: init done 2024-02-23T00:41:51.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.427+0000 7f370dece700 1 -- 172.21.15.47:0/4146025896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3708109040 con 0x7f37080f9670 2024-02-23T00:41:51.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.427+0000 7f3704ff9700 1 -- 172.21.15.47:0/4146025896 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f36f8009240 con 0x7f37080f9670 2024-02-23T00:41:51.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.428+0000 7f3704ff9700 1 --2- 172.21.15.47:0/4146025896 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f36f4062cc0 0x7f36f4065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:51.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.428+0000 7f3704ff9700 1 -- 172.21.15.47:0/4146025896 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 38..95) v4 ==== 7421+0+0 (secure 0 0 0) 0x7f36f807e170 con 0x7f37080f9670 2024-02-23T00:41:51.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.428+0000 7f3704ff9700 10 client.14745.objecter ms_dispatch 0x7f3708001550 osd_map(95..95 src has 38..95) v4 2024-02-23T00:41:51.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.428+0000 7f3704ff9700 3 client.14745.objecter handle_osd_map got epochs [95,95] > 0 2024-02-23T00:41:51.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.428+0000 7f3704ff9700 3 client.14745.objecter handle_osd_map decoding full epoch 95 2024-02-23T00:41:51.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.428+0000 7f37077fe700 1 --2- 172.21.15.47:0/4146025896 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f36f4062cc0 0x7f36f4065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:51.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.428+0000 7f3704ff9700 20 client.14745.objecter dump_active .. 0 homeless 2024-02-23T00:41:51.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.428+0000 7f3704ff9700 10 client.14745.objecter ms_handle_connect 0x7f36f4062cc0 2024-02-23T00:41:51.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.428+0000 7f37077fe700 1 --2- 172.21.15.47:0/4146025896 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f36f4062cc0 0x7f36f4065180 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f36f00103c0 tx=0x7f36f000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:51.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.432+0000 7f3704ff9700 1 -- 172.21.15.47:0/4146025896 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f36f804ded0 con 0x7f37080f9670 2024-02-23T00:41:51.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.605+0000 7f370dece700 1 -- 172.21.15.47:0/4146025896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f370805fa60 con 0x7f37080f9670 2024-02-23T00:41:51.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.605+0000 7f3704ff9700 1 -- 172.21.15.47:0/4146025896 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v95) v1 ==== 56+0+4803 (secure 0 0 0) 0x7f36f80514f0 con 0x7f37080f9670 2024-02-23T00:41:51.607 INFO:teuthology.orchestra.run.smithi047.stdout:epoch 95 2024-02-23T00:41:51.607 INFO:teuthology.orchestra.run.smithi047.stdout:fsid fce2617e-d1e2-11ee-95c0-87774f69a715 2024-02-23T00:41:51.608 INFO:teuthology.orchestra.run.smithi047.stdout:created 2024-02-23T00:32:55.399437+0000 2024-02-23T00:41:51.608 INFO:teuthology.orchestra.run.smithi047.stdout:modified 2024-02-23T00:41:40.845621+0000 2024-02-23T00:41:51.608 INFO:teuthology.orchestra.run.smithi047.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-02-23T00:41:51.608 INFO:teuthology.orchestra.run.smithi047.stdout:crush_version 18 2024-02-23T00:41:51.608 INFO:teuthology.orchestra.run.smithi047.stdout:full_ratio 0.95 2024-02-23T00:41:51.608 INFO:teuthology.orchestra.run.smithi047.stdout:backfillfull_ratio 0.9 2024-02-23T00:41:51.608 INFO:teuthology.orchestra.run.smithi047.stdout:nearfull_ratio 0.85 2024-02-23T00:41:51.608 INFO:teuthology.orchestra.run.smithi047.stdout:require_min_compat_client luminous 2024-02-23T00:41:51.608 INFO:teuthology.orchestra.run.smithi047.stdout:min_compat_client luminous 2024-02-23T00:41:51.608 INFO:teuthology.orchestra.run.smithi047.stdout:require_osd_release pacific 2024-02-23T00:41:51.608 INFO:teuthology.orchestra.run.smithi047.stdout:stretch_mode_enabled false 2024-02-23T00:41:51.609 INFO:teuthology.orchestra.run.smithi047.stdout:pool 1 'device_health_metrics' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 75 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr_devicehealth 2024-02-23T00:41:51.609 INFO:teuthology.orchestra.run.smithi047.stdout:pool 2 '.rgw.root' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 61 flags hashpspool stripe_width 0 application rgw 2024-02-23T00:41:51.609 INFO:teuthology.orchestra.run.smithi047.stdout:pool 3 'iscsi' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 3 pgp_num 3 autoscale_mode off last_change 63 flags hashpspool stripe_width 0 application rbd 2024-02-23T00:41:51.609 INFO:teuthology.orchestra.run.smithi047.stdout:pool 4 'default.rgw.log' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 63 flags hashpspool stripe_width 0 application rgw 2024-02-23T00:41:51.609 INFO:teuthology.orchestra.run.smithi047.stdout:pool 5 'default.rgw.control' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 65 flags hashpspool stripe_width 0 application rgw 2024-02-23T00:41:51.609 INFO:teuthology.orchestra.run.smithi047.stdout:pool 6 'default.rgw.meta' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 67 flags hashpspool stripe_width 0 pg_autoscale_bias 4 application rgw 2024-02-23T00:41:51.609 INFO:teuthology.orchestra.run.smithi047.stdout:max_osd 8 2024-02-23T00:41:51.609 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0 up in weight 1 up_from 8 up_thru 65 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] [v2:172.21.15.47:6804/4042552860,v1:172.21.15.47:6805/4042552860] exists,up 380050f9-9001-46dc-bb29-037987b4c8e7 2024-02-23T00:41:51.609 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1 up in weight 1 up_from 13 up_thru 65 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] [v2:172.21.15.47:6812/1945417179,v1:172.21.15.47:6813/1945417179] exists,up b06d0cce-55d2-4a6a-a796-4db0b9bb9d16 2024-02-23T00:41:51.609 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2 up in weight 1 up_from 18 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] [v2:172.21.15.47:6820/3596356785,v1:172.21.15.47:6821/3596356785] exists,up 6e44d7ca-67a1-4868-bb64-891cf477ebd5 2024-02-23T00:41:51.609 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3 up in weight 1 up_from 26 up_thru 65 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] [v2:172.21.15.47:6828/1195438081,v1:172.21.15.47:6829/1195438081] exists,up c88debab-cb0b-46f6-a7d8-c5dc61fc059f 2024-02-23T00:41:51.609 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4 up in weight 1 up_from 32 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] [v2:172.21.15.196:6802/2530629682,v1:172.21.15.196:6803/2530629682] exists,up d17966d7-eb12-4389-ac90-8d15efe290e4 2024-02-23T00:41:51.609 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5 up in weight 1 up_from 39 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] [v2:172.21.15.196:6810/1017187491,v1:172.21.15.196:6811/1017187491] exists,up b729d646-4899-47b3-b917-acceba46d241 2024-02-23T00:41:51.610 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6 up in weight 1 up_from 47 up_thru 63 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] [v2:172.21.15.196:6818/3885668599,v1:172.21.15.196:6819/3885668599] exists,up df337124-9488-4aad-98e5-3614b51fc757 2024-02-23T00:41:51.610 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7 up in weight 1 up_from 54 up_thru 65 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] [v2:172.21.15.196:6826/2261821675,v1:172.21.15.196:6827/2261821675] exists,up 84fee1f2-1a88-447a-b1f3-56ff7efe7fec 2024-02-23T00:41:51.610 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 2.b [7,2] 2024-02-23T00:41:51.610 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 4.4 [1,5] 2024-02-23T00:41:51.610 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 4.d [1,5] 2024-02-23T00:41:51.610 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 4.e [1,2] 2024-02-23T00:41:51.610 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:6800/622994871 expires 2024-02-24T00:39:41.110158+0000 2024-02-23T00:41:51.610 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/1561174431 expires 2024-02-24T00:39:41.110158+0000 2024-02-23T00:41:51.610 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/3349912606 expires 2024-02-24T00:39:41.110158+0000 2024-02-23T00:41:51.610 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/710432524 expires 2024-02-24T00:39:41.110158+0000 2024-02-23T00:41:51.610 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/2804352395 expires 2024-02-24T00:33:52.061914+0000 2024-02-23T00:41:51.610 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:6801/1916598887 expires 2024-02-24T00:33:52.061914+0000 2024-02-23T00:41:51.610 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:6800/1916598887 expires 2024-02-24T00:33:52.061914+0000 2024-02-23T00:41:51.610 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/792211316 expires 2024-02-24T00:33:27.972945+0000 2024-02-23T00:41:51.611 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/381995198 expires 2024-02-24T00:33:52.061914+0000 2024-02-23T00:41:51.611 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:6800/2409691510 expires 2024-02-24T00:33:27.972945+0000 2024-02-23T00:41:51.611 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:6801/2409691510 expires 2024-02-24T00:33:27.972945+0000 2024-02-23T00:41:51.611 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:6801/622994871 expires 2024-02-24T00:39:41.110158+0000 2024-02-23T00:41:51.611 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/1448327041 expires 2024-02-24T00:33:27.972945+0000 2024-02-23T00:41:51.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.607+0000 7f370dece700 10 librados: watch_flush enter 2024-02-23T00:41:51.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.607+0000 7f370dece700 10 librados: watch_flush exit 2024-02-23T00:41:51.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.607+0000 7f370dece700 20 client.14745.objecter shutdown clearing up homeless session... 2024-02-23T00:41:51.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.607+0000 7f370dece700 10 client.14745.objecter successfully canceled tick 2024-02-23T00:41:51.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.607+0000 7f370dece700 1 -- 172.21.15.47:0/4146025896 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f36f4062cc0 msgr2=0x7f36f4065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:51.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.607+0000 7f370dece700 1 --2- 172.21.15.47:0/4146025896 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f36f4062cc0 0x7f36f4065180 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f36f00103c0 tx=0x7f36f000b040).stop 2024-02-23T00:41:51.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.607+0000 7f370dece700 1 -- 172.21.15.47:0/4146025896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37080f9670 msgr2=0x7f370810e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:51.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.607+0000 7f370dece700 1 --2- 172.21.15.47:0/4146025896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37080f9670 0x7f370810e4e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f36f8009da0 tx=0x7f36f8009cb0).stop 2024-02-23T00:41:51.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.607+0000 7f370dece700 1 -- 172.21.15.47:0/4146025896 shutdown_connections 2024-02-23T00:41:51.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.607+0000 7f370dece700 1 --2- 172.21.15.47:0/4146025896 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3708102270 0x7f3708108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:51.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.607+0000 7f370dece700 1 --2- 172.21.15.47:0/4146025896 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f36f4062cc0 0x7f36f4065180 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:51.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.607+0000 7f370dece700 1 --2- 172.21.15.47:0/4146025896 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f37080f8d10 0x7f370810dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:51.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.607+0000 7f370dece700 1 --2- 172.21.15.47:0/4146025896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37080f9670 0x7f370810e4e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:51.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.607+0000 7f370dece700 1 -- 172.21.15.47:0/4146025896 >> 172.21.15.47:0/4146025896 conn(0x7f37080f2820 msgr2=0x7f37080f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:41:51.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.607+0000 7f370dece700 1 -- 172.21.15.47:0/4146025896 shutdown_connections 2024-02-23T00:41:51.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.607+0000 7f370dece700 1 -- 172.21.15.47:0/4146025896 wait complete. 2024-02-23T00:41:51.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:51.607+0000 7f370dece700 1 librados: shutdown 2024-02-23T00:41:52.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:50.653541+0000 osd.1 (osd.1) 51 : cluster [DBG] 6.1 scrub starts 2024-02-23T00:41:52.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:50.656704+0000 osd.1 (osd.1) 52 : cluster [DBG] 6.1 scrub ok 2024-02-23T00:41:52.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:50.973288+0000 osd.4 (osd.4) 53 : cluster [DBG] 4.1b deep-scrub starts 2024-02-23T00:41:52.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:50.975677+0000 osd.4 (osd.4) 54 : cluster [DBG] 4.1b deep-scrub ok 2024-02-23T00:41:52.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:51.154485+0000 mgr.y (mgr.14517) 105 : cluster [DBG] pgmap v90: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 993 B/s rd, 0 op/s 2024-02-23T00:41:52.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:41:51.606409+0000 mon.a (mon.0) 764 : audit [DBG] from='client.? 172.21.15.47:0/4146025896' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-23T00:41:52.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:50.653541+0000 osd.1 (osd.1) 51 : cluster [DBG] 6.1 scrub starts 2024-02-23T00:41:52.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:50.656704+0000 osd.1 (osd.1) 52 : cluster [DBG] 6.1 scrub ok 2024-02-23T00:41:52.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:50.973288+0000 osd.4 (osd.4) 53 : cluster [DBG] 4.1b deep-scrub starts 2024-02-23T00:41:52.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:50.975677+0000 osd.4 (osd.4) 54 : cluster [DBG] 4.1b deep-scrub ok 2024-02-23T00:41:52.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:51.154485+0000 mgr.y (mgr.14517) 105 : cluster [DBG] pgmap v90: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 993 B/s rd, 0 op/s 2024-02-23T00:41:52.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:41:51.606409+0000 mon.a (mon.0) 764 : audit [DBG] from='client.? 172.21.15.47:0/4146025896' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-23T00:41:52.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:50.653541+0000 osd.1 (osd.1) 51 : cluster [DBG] 6.1 scrub starts 2024-02-23T00:41:52.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:50.656704+0000 osd.1 (osd.1) 52 : cluster [DBG] 6.1 scrub ok 2024-02-23T00:41:52.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:50.973288+0000 osd.4 (osd.4) 53 : cluster [DBG] 4.1b deep-scrub starts 2024-02-23T00:41:52.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:50.975677+0000 osd.4 (osd.4) 54 : cluster [DBG] 4.1b deep-scrub ok 2024-02-23T00:41:52.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:51.154485+0000 mgr.y (mgr.14517) 105 : cluster [DBG] pgmap v90: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 993 B/s rd, 0 op/s 2024-02-23T00:41:52.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:41:51.606409+0000 mon.a (mon.0) 764 : audit [DBG] from='client.? 172.21.15.47:0/4146025896' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-23T00:41:52.413 INFO:tasks.cephadm.ceph_manager.ceph:['osd.0 up in weight 1 up_from 8 up_thru 65 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] [v2:172.21.15.47:6804/4042552860,v1:172.21.15.47:6805/4042552860] exists,up 380050f9-9001-46dc-bb29-037987b4c8e7', 'osd.1 up in weight 1 up_from 13 up_thru 65 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] [v2:172.21.15.47:6812/1945417179,v1:172.21.15.47:6813/1945417179] exists,up b06d0cce-55d2-4a6a-a796-4db0b9bb9d16', 'osd.2 up in weight 1 up_from 18 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] [v2:172.21.15.47:6820/3596356785,v1:172.21.15.47:6821/3596356785] exists,up 6e44d7ca-67a1-4868-bb64-891cf477ebd5', 'osd.3 up in weight 1 up_from 26 up_thru 65 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] [v2:172.21.15.47:6828/1195438081,v1:172.21.15.47:6829/1195438081] exists,up c88debab-cb0b-46f6-a7d8-c5dc61fc059f', 'osd.4 up in weight 1 up_from 32 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] [v2:172.21.15.196:6802/2530629682,v1:172.21.15.196:6803/2530629682] exists,up d17966d7-eb12-4389-ac90-8d15efe290e4', 'osd.5 up in weight 1 up_from 39 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] [v2:172.21.15.196:6810/1017187491,v1:172.21.15.196:6811/1017187491] exists,up b729d646-4899-47b3-b917-acceba46d241', 'osd.6 up in weight 1 up_from 47 up_thru 63 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] [v2:172.21.15.196:6818/3885668599,v1:172.21.15.196:6819/3885668599] exists,up df337124-9488-4aad-98e5-3614b51fc757', 'osd.7 up in weight 1 up_from 54 up_thru 65 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] [v2:172.21.15.196:6826/2261821675,v1:172.21.15.196:6827/2261821675] exists,up 84fee1f2-1a88-447a-b1f3-56ff7efe7fec'] 2024-02-23T00:41:52.413 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon mon.a config show 2024-02-23T00:41:52.528 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:41:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:41:52.694 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:41:52.933 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:41:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:41:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:41:53.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:52.081246+0000 osd.6 (osd.6) 39 : cluster [DBG] 5.11 scrub starts 2024-02-23T00:41:53.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:52.082265+0000 osd.6 (osd.6) 40 : cluster [DBG] 5.11 scrub ok 2024-02-23T00:41:53.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:52.150244+0000 osd.0 (osd.0) 45 : cluster [DBG] 2.13 scrub starts 2024-02-23T00:41:53.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:52.151115+0000 osd.0 (osd.0) 46 : cluster [DBG] 2.13 scrub ok 2024-02-23T00:41:53.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:41:52.227419+0000 mgr.y (mgr.14517) 106 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:41:53.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:52.263291+0000 osd.2 (osd.2) 41 : cluster [DBG] 5.6 deep-scrub starts 2024-02-23T00:41:53.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:52.264235+0000 osd.2 (osd.2) 42 : cluster [DBG] 5.6 deep-scrub ok 2024-02-23T00:41:53.286 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:52.081246+0000 osd.6 (osd.6) 39 : cluster [DBG] 5.11 scrub starts 2024-02-23T00:41:53.286 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:52.082265+0000 osd.6 (osd.6) 40 : cluster [DBG] 5.11 scrub ok 2024-02-23T00:41:53.286 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:52.150244+0000 osd.0 (osd.0) 45 : cluster [DBG] 2.13 scrub starts 2024-02-23T00:41:53.286 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:52.151115+0000 osd.0 (osd.0) 46 : cluster [DBG] 2.13 scrub ok 2024-02-23T00:41:53.286 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:41:52.227419+0000 mgr.y (mgr.14517) 106 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:41:53.286 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:52.263291+0000 osd.2 (osd.2) 41 : cluster [DBG] 5.6 deep-scrub starts 2024-02-23T00:41:53.287 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:52.264235+0000 osd.2 (osd.2) 42 : cluster [DBG] 5.6 deep-scrub ok 2024-02-23T00:41:53.287 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:52.081246+0000 osd.6 (osd.6) 39 : cluster [DBG] 5.11 scrub starts 2024-02-23T00:41:53.287 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:52.082265+0000 osd.6 (osd.6) 40 : cluster [DBG] 5.11 scrub ok 2024-02-23T00:41:53.287 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:52.150244+0000 osd.0 (osd.0) 45 : cluster [DBG] 2.13 scrub starts 2024-02-23T00:41:53.287 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:52.151115+0000 osd.0 (osd.0) 46 : cluster [DBG] 2.13 scrub ok 2024-02-23T00:41:53.287 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:41:52.227419+0000 mgr.y (mgr.14517) 106 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:41:53.287 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:52.263291+0000 osd.2 (osd.2) 41 : cluster [DBG] 5.6 deep-scrub starts 2024-02-23T00:41:53.287 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:52.264235+0000 osd.2 (osd.2) 42 : cluster [DBG] 5.6 deep-scrub ok 2024-02-23T00:41:54.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:52.640677+0000 osd.1 (osd.1) 53 : cluster [DBG] 5.1d deep-scrub starts 2024-02-23T00:41:54.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:52.641681+0000 osd.1 (osd.1) 54 : cluster [DBG] 5.1d deep-scrub ok 2024-02-23T00:41:54.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:52.964091+0000 osd.4 (osd.4) 55 : cluster [DBG] 5.1 scrub starts 2024-02-23T00:41:54.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:52.965246+0000 osd.4 (osd.4) 56 : cluster [DBG] 5.1 scrub ok 2024-02-23T00:41:54.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:53.155134+0000 mgr.y (mgr.14517) 107 : cluster [DBG] pgmap v91: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:41:54.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:53.189035+0000 osd.0 (osd.0) 47 : cluster [DBG] 5.1e deep-scrub starts 2024-02-23T00:41:54.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:53.189877+0000 osd.0 (osd.0) 48 : cluster [DBG] 5.1e deep-scrub ok 2024-02-23T00:41:54.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:52.640677+0000 osd.1 (osd.1) 53 : cluster [DBG] 5.1d deep-scrub starts 2024-02-23T00:41:54.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:52.641681+0000 osd.1 (osd.1) 54 : cluster [DBG] 5.1d deep-scrub ok 2024-02-23T00:41:54.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:52.964091+0000 osd.4 (osd.4) 55 : cluster [DBG] 5.1 scrub starts 2024-02-23T00:41:54.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:52.965246+0000 osd.4 (osd.4) 56 : cluster [DBG] 5.1 scrub ok 2024-02-23T00:41:54.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:53.155134+0000 mgr.y (mgr.14517) 107 : cluster [DBG] pgmap v91: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:41:54.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:53.189035+0000 osd.0 (osd.0) 47 : cluster [DBG] 5.1e deep-scrub starts 2024-02-23T00:41:54.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:53.189877+0000 osd.0 (osd.0) 48 : cluster [DBG] 5.1e deep-scrub ok 2024-02-23T00:41:54.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:52.640677+0000 osd.1 (osd.1) 53 : cluster [DBG] 5.1d deep-scrub starts 2024-02-23T00:41:54.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:52.641681+0000 osd.1 (osd.1) 54 : cluster [DBG] 5.1d deep-scrub ok 2024-02-23T00:41:54.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:52.964091+0000 osd.4 (osd.4) 55 : cluster [DBG] 5.1 scrub starts 2024-02-23T00:41:54.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:52.965246+0000 osd.4 (osd.4) 56 : cluster [DBG] 5.1 scrub ok 2024-02-23T00:41:54.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:53.155134+0000 mgr.y (mgr.14517) 107 : cluster [DBG] pgmap v91: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:41:54.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:53.189035+0000 osd.0 (osd.0) 47 : cluster [DBG] 5.1e deep-scrub starts 2024-02-23T00:41:54.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:53.189877+0000 osd.0 (osd.0) 48 : cluster [DBG] 5.1e deep-scrub ok 2024-02-23T00:41:54.680 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:41:54.680 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mon.a", 2024-02-23T00:41:54.680 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster": "ceph", 2024-02-23T00:41:54.681 INFO:teuthology.orchestra.run.smithi047.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-02-23T00:41:54.681 INFO:teuthology.orchestra.run.smithi047.stdout: "admin_socket_mode": "", 2024-02-23T00:41:54.681 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_allow_insecure_global_id_reclaim": "false", 2024-02-23T00:41:54.681 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_client_required": "cephx, none", 2024-02-23T00:41:54.681 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_cluster_required": "cephx", 2024-02-23T00:41:54.681 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_debug": "true", 2024-02-23T00:41:54.681 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-02-23T00:41:54.681 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-02-23T00:41:54.681 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_service_required": "cephx", 2024-02-23T00:41:54.681 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_service_ticket_ttl": "240.000000", 2024-02-23T00:41:54.681 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_supported": "", 2024-02-23T00:41:54.682 INFO:teuthology.orchestra.run.smithi047.stdout: "bdev_aio": "true", 2024-02-23T00:41:54.682 INFO:teuthology.orchestra.run.smithi047.stdout: "bdev_aio_max_queue_depth": "1024", 2024-02-23T00:41:54.682 INFO:teuthology.orchestra.run.smithi047.stdout: "bdev_aio_poll_ms": "250", 2024-02-23T00:41:54.682 INFO:teuthology.orchestra.run.smithi047.stdout: "bdev_aio_reap_max": "16", 2024-02-23T00:41:54.682 INFO:teuthology.orchestra.run.smithi047.stdout: "bdev_async_discard": "false", 2024-02-23T00:41:54.682 INFO:teuthology.orchestra.run.smithi047.stdout: "bdev_block_size": "4096", 2024-02-23T00:41:54.682 INFO:teuthology.orchestra.run.smithi047.stdout: "bdev_debug_aio": "false", 2024-02-23T00:41:54.682 INFO:teuthology.orchestra.run.smithi047.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-02-23T00:41:54.682 INFO:teuthology.orchestra.run.smithi047.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-02-23T00:41:54.682 INFO:teuthology.orchestra.run.smithi047.stdout: "bdev_debug_inflight_ios": "false", 2024-02-23T00:41:54.682 INFO:teuthology.orchestra.run.smithi047.stdout: "bdev_enable_discard": "false", 2024-02-23T00:41:54.682 INFO:teuthology.orchestra.run.smithi047.stdout: "bdev_flock_retry": "3", 2024-02-23T00:41:54.683 INFO:teuthology.orchestra.run.smithi047.stdout: "bdev_flock_retry_interval": "0.100000", 2024-02-23T00:41:54.683 INFO:teuthology.orchestra.run.smithi047.stdout: "bdev_inject_crash": "0", 2024-02-23T00:41:54.683 INFO:teuthology.orchestra.run.smithi047.stdout: "bdev_inject_crash_flush_delay": "2", 2024-02-23T00:41:54.683 INFO:teuthology.orchestra.run.smithi047.stdout: "bdev_ioring": "false", 2024-02-23T00:41:54.683 INFO:teuthology.orchestra.run.smithi047.stdout: "bdev_ioring_hipri": "false", 2024-02-23T00:41:54.683 INFO:teuthology.orchestra.run.smithi047.stdout: "bdev_ioring_sqthread_poll": "false", 2024-02-23T00:41:54.683 INFO:teuthology.orchestra.run.smithi047.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-02-23T00:41:54.683 INFO:teuthology.orchestra.run.smithi047.stdout: "bdev_type": "", 2024-02-23T00:41:54.683 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_alloc_size": "1048576", 2024-02-23T00:41:54.683 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_allocator": "hybrid", 2024-02-23T00:41:54.683 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_buffered_io": "true", 2024-02-23T00:41:54.684 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_check_for_zeros": "false", 2024-02-23T00:41:54.684 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_check_volume_selector_often": "false", 2024-02-23T00:41:54.684 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-02-23T00:41:54.684 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_compact_log_sync": "false", 2024-02-23T00:41:54.684 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-02-23T00:41:54.684 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-02-23T00:41:54.684 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_log_compact_min_size": "16777216", 2024-02-23T00:41:54.684 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_log_replay_check_allocations": "true", 2024-02-23T00:41:54.684 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_max_log_runway": "4194304", 2024-02-23T00:41:54.684 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_max_prefetch": "1048576", 2024-02-23T00:41:54.684 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_min_flush_size": "524288", 2024-02-23T00:41:54.685 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_min_log_runway": "1048576", 2024-02-23T00:41:54.685 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_replay_recovery": "false", 2024-02-23T00:41:54.685 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-02-23T00:41:54.685 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_shared_alloc_size": "65536", 2024-02-23T00:41:54.685 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_sync_write": "false", 2024-02-23T00:41:54.685 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-02-23T00:41:54.685 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-02-23T00:41:54.685 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-02-23T00:41:54.685 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_allocator": "hybrid", 2024-02-23T00:41:54.685 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-02-23T00:41:54.685 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-02-23T00:41:54.685 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-02-23T00:41:54.686 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-02-23T00:41:54.686 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-02-23T00:41:54.686 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-02-23T00:41:54.686 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blobid_prealloc": "10240", 2024-02-23T00:41:54.686 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_block_create": "true", 2024-02-23T00:41:54.686 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_block_db_create": "false", 2024-02-23T00:41:54.686 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_block_db_path": "", 2024-02-23T00:41:54.686 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_block_db_size": "0", 2024-02-23T00:41:54.686 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_block_path": "", 2024-02-23T00:41:54.686 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_block_preallocate_file": "false", 2024-02-23T00:41:54.686 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_block_size": "107374182400", 2024-02-23T00:41:54.686 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_block_wal_create": "false", 2024-02-23T00:41:54.687 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_block_wal_path": "", 2024-02-23T00:41:54.687 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_block_wal_size": "100663296", 2024-02-23T00:41:54.687 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_bluefs": "true", 2024-02-23T00:41:54.687 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-02-23T00:41:54.687 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_bluefs_env_mirror": "false", 2024-02-23T00:41:54.687 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_autotune": "true", 2024-02-23T00:41:54.687 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-02-23T00:41:54.687 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-02-23T00:41:54.687 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-02-23T00:41:54.687 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-02-23T00:41:54.687 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_size": "0", 2024-02-23T00:41:54.688 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-02-23T00:41:54.688 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-02-23T00:41:54.688 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-02-23T00:41:54.688 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-02-23T00:41:54.688 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_type": "2q", 2024-02-23T00:41:54.688 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_clone_cow": "true", 2024-02-23T00:41:54.688 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_compression_algorithm": "snappy", 2024-02-23T00:41:54.688 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_compression_max_blob_size": "0", 2024-02-23T00:41:54.688 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-02-23T00:41:54.688 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-02-23T00:41:54.688 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_compression_min_blob_size": "0", 2024-02-23T00:41:54.689 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-02-23T00:41:54.689 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_compression_min_blob_size_ssd": "8192", 2024-02-23T00:41:54.689 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_compression_mode": "none", 2024-02-23T00:41:54.689 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-02-23T00:41:54.689 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_csum_type": "crc32c", 2024-02-23T00:41:54.689 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_debug_enforce_settings": "default", 2024-02-23T00:41:54.689 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_debug_freelist": "false", 2024-02-23T00:41:54.689 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_debug_fsck_abort": "false", 2024-02-23T00:41:54.689 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_debug_inject_bug21040": "false", 2024-02-23T00:41:54.689 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-02-23T00:41:54.689 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_debug_inject_read_err": "false", 2024-02-23T00:41:54.689 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_debug_legacy_omap": "false", 2024-02-23T00:41:54.689 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-02-23T00:41:54.690 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-02-23T00:41:54.690 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-02-23T00:41:54.690 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-02-23T00:41:54.690 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_debug_prefill": "0.000000", 2024-02-23T00:41:54.690 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-02-23T00:41:54.690 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-02-23T00:41:54.690 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-02-23T00:41:54.690 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_debug_small_allocations": "0", 2024-02-23T00:41:54.690 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-02-23T00:41:54.690 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_default_buffered_read": "true", 2024-02-23T00:41:54.690 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_default_buffered_write": "false", 2024-02-23T00:41:54.690 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_deferred_batch_ops": "0", 2024-02-23T00:41:54.691 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-02-23T00:41:54.691 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-02-23T00:41:54.691 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-02-23T00:41:54.691 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-02-23T00:41:54.691 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-02-23T00:41:54.691 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-02-23T00:41:54.691 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-02-23T00:41:54.691 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-02-23T00:41:54.691 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-02-23T00:41:54.691 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-02-23T00:41:54.691 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-02-23T00:41:54.691 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_on_mkfs": "true", 2024-02-23T00:41:54.691 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-02-23T00:41:54.692 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_on_mount": "false", 2024-02-23T00:41:54.692 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-02-23T00:41:54.692 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_on_umount": "false", 2024-02-23T00:41:54.692 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-02-23T00:41:54.692 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-02-23T00:41:54.692 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-02-23T00:41:54.692 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-02-23T00:41:54.692 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-02-23T00:41:54.692 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-02-23T00:41:54.692 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-02-23T00:41:54.692 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-02-23T00:41:54.693 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_ignore_data_csum": "false", 2024-02-23T00:41:54.693 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-02-23T00:41:54.693 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_kvbackend": "rocksdb", 2024-02-23T00:41:54.693 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-02-23T00:41:54.693 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-02-23T00:41:54.693 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_log_op_age": "5.000000", 2024-02-23T00:41:54.693 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_max_alloc_size": "0", 2024-02-23T00:41:54.693 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_max_blob_size": "0", 2024-02-23T00:41:54.693 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-02-23T00:41:54.693 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-02-23T00:41:54.694 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_max_defer_interval": "3.000000", 2024-02-23T00:41:54.694 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_max_deferred_txc": "32", 2024-02-23T00:41:54.694 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_min_alloc_size": "0", 2024-02-23T00:41:54.694 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-02-23T00:41:54.694 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-02-23T00:41:54.694 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_nid_prealloc": "1024", 2024-02-23T00:41:54.694 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_prefer_deferred_size": "0", 2024-02-23T00:41:54.694 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-02-23T00:41:54.694 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-02-23T00:41:54.694 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_retry_disk_reads": "3", 2024-02-23T00:41:54.694 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_rocksdb_cf": "true", 2024-02-23T00:41:54.694 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L P", 2024-02-23T00:41:54.695 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2,max_total_wal_size=1073741824", 2024-02-23T00:41:54.695 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_rocksdb_options_annex": "", 2024-02-23T00:41:54.695 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_spdk_coremask": "0x1", 2024-02-23T00:41:54.695 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_spdk_io_sleep": "5", 2024-02-23T00:41:54.695 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_spdk_max_io_completion": "0", 2024-02-23T00:41:54.695 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_spdk_mem": "512", 2024-02-23T00:41:54.695 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_sync_submit_transaction": "false", 2024-02-23T00:41:54.695 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_throttle_bytes": "67108864", 2024-02-23T00:41:54.695 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_throttle_cost_per_io": "0", 2024-02-23T00:41:54.695 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-02-23T00:41:54.695 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-02-23T00:41:54.695 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-02-23T00:41:54.696 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-02-23T00:41:54.696 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_tracing": "false", 2024-02-23T00:41:54.696 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-02-23T00:41:54.696 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_volume_selection_reserved": "0", 2024-02-23T00:41:54.696 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-02-23T00:41:54.696 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-02-23T00:41:54.696 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-02-23T00:41:54.696 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-02-23T00:41:54.696 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-02-23T00:41:54.696 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-02-23T00:41:54.696 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-02-23T00:41:54.697 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_mirror_action_update_interval": "2", 2024-02-23T00:41:54.697 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-02-23T00:41:54.697 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-02-23T00:41:54.697 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-02-23T00:41:54.697 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-02-23T00:41:54.697 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_mirror_mount_timeout": "10", 2024-02-23T00:41:54.697 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-02-23T00:41:54.697 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-02-23T00:41:54.697 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-02-23T00:41:54.697 INFO:teuthology.orchestra.run.smithi047.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-02-23T00:41:54.697 INFO:teuthology.orchestra.run.smithi047.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-02-23T00:41:54.698 INFO:teuthology.orchestra.run.smithi047.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-02-23T00:41:54.698 INFO:teuthology.orchestra.run.smithi047.stdout: "cephx_cluster_require_signatures": "false", 2024-02-23T00:41:54.698 INFO:teuthology.orchestra.run.smithi047.stdout: "cephx_cluster_require_version": "2", 2024-02-23T00:41:54.698 INFO:teuthology.orchestra.run.smithi047.stdout: "cephx_require_signatures": "false", 2024-02-23T00:41:54.698 INFO:teuthology.orchestra.run.smithi047.stdout: "cephx_require_version": "2", 2024-02-23T00:41:54.698 INFO:teuthology.orchestra.run.smithi047.stdout: "cephx_service_require_signatures": "false", 2024-02-23T00:41:54.698 INFO:teuthology.orchestra.run.smithi047.stdout: "cephx_service_require_version": "2", 2024-02-23T00:41:54.698 INFO:teuthology.orchestra.run.smithi047.stdout: "cephx_sign_messages": "true", 2024-02-23T00:41:54.698 INFO:teuthology.orchestra.run.smithi047.stdout: "chdir": "", 2024-02-23T00:41:54.698 INFO:teuthology.orchestra.run.smithi047.stdout: "client_acl_type": "", 2024-02-23T00:41:54.698 INFO:teuthology.orchestra.run.smithi047.stdout: "client_asio_thread_count": "2", 2024-02-23T00:41:54.699 INFO:teuthology.orchestra.run.smithi047.stdout: "client_cache_mid": "0.750000", 2024-02-23T00:41:54.699 INFO:teuthology.orchestra.run.smithi047.stdout: "client_cache_size": "16384", 2024-02-23T00:41:54.699 INFO:teuthology.orchestra.run.smithi047.stdout: "client_caps_release_delay": "5", 2024-02-23T00:41:54.699 INFO:teuthology.orchestra.run.smithi047.stdout: "client_check_pool_perm": "true", 2024-02-23T00:41:54.699 INFO:teuthology.orchestra.run.smithi047.stdout: "client_collect_and_send_global_metrics": "false", 2024-02-23T00:41:54.699 INFO:teuthology.orchestra.run.smithi047.stdout: "client_debug_force_sync_read": "false", 2024-02-23T00:41:54.699 INFO:teuthology.orchestra.run.smithi047.stdout: "client_debug_getattr_caps": "false", 2024-02-23T00:41:54.699 INFO:teuthology.orchestra.run.smithi047.stdout: "client_debug_inject_tick_delay": "0", 2024-02-23T00:41:54.699 INFO:teuthology.orchestra.run.smithi047.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-02-23T00:41:54.699 INFO:teuthology.orchestra.run.smithi047.stdout: "client_die_on_failed_remount": "false", 2024-02-23T00:41:54.699 INFO:teuthology.orchestra.run.smithi047.stdout: "client_dirsize_rbytes": "true", 2024-02-23T00:41:54.699 INFO:teuthology.orchestra.run.smithi047.stdout: "client_force_lazyio": "false", 2024-02-23T00:41:54.700 INFO:teuthology.orchestra.run.smithi047.stdout: "client_fs": "", 2024-02-23T00:41:54.700 INFO:teuthology.orchestra.run.smithi047.stdout: "client_inject_fixed_oldest_tid": "false", 2024-02-23T00:41:54.700 INFO:teuthology.orchestra.run.smithi047.stdout: "client_inject_release_failure": "false", 2024-02-23T00:41:54.700 INFO:teuthology.orchestra.run.smithi047.stdout: "client_max_inline_size": "4096", 2024-02-23T00:41:54.700 INFO:teuthology.orchestra.run.smithi047.stdout: "client_max_retries_on_remount_failure": "5", 2024-02-23T00:41:54.700 INFO:teuthology.orchestra.run.smithi047.stdout: "client_mds_namespace": "", 2024-02-23T00:41:54.700 INFO:teuthology.orchestra.run.smithi047.stdout: "client_metadata": "", 2024-02-23T00:41:54.700 INFO:teuthology.orchestra.run.smithi047.stdout: "client_mount_gid": "-1", 2024-02-23T00:41:54.700 INFO:teuthology.orchestra.run.smithi047.stdout: "client_mount_timeout": "300", 2024-02-23T00:41:54.700 INFO:teuthology.orchestra.run.smithi047.stdout: "client_mount_uid": "-1", 2024-02-23T00:41:54.700 INFO:teuthology.orchestra.run.smithi047.stdout: "client_mountpoint": "/", 2024-02-23T00:41:54.700 INFO:teuthology.orchestra.run.smithi047.stdout: "client_notify_timeout": "10", 2024-02-23T00:41:54.701 INFO:teuthology.orchestra.run.smithi047.stdout: "client_oc": "true", 2024-02-23T00:41:54.701 INFO:teuthology.orchestra.run.smithi047.stdout: "client_oc_max_dirty": "104857600", 2024-02-23T00:41:54.701 INFO:teuthology.orchestra.run.smithi047.stdout: "client_oc_max_dirty_age": "5.000000", 2024-02-23T00:41:54.701 INFO:teuthology.orchestra.run.smithi047.stdout: "client_oc_max_objects": "1000", 2024-02-23T00:41:54.701 INFO:teuthology.orchestra.run.smithi047.stdout: "client_oc_size": "209715200", 2024-02-23T00:41:54.701 INFO:teuthology.orchestra.run.smithi047.stdout: "client_oc_target_dirty": "8388608", 2024-02-23T00:41:54.701 INFO:teuthology.orchestra.run.smithi047.stdout: "client_permissions": "true", 2024-02-23T00:41:54.701 INFO:teuthology.orchestra.run.smithi047.stdout: "client_quota": "true", 2024-02-23T00:41:54.701 INFO:teuthology.orchestra.run.smithi047.stdout: "client_quota_df": "true", 2024-02-23T00:41:54.701 INFO:teuthology.orchestra.run.smithi047.stdout: "client_readahead_max_bytes": "0", 2024-02-23T00:41:54.701 INFO:teuthology.orchestra.run.smithi047.stdout: "client_readahead_max_periods": "4", 2024-02-23T00:41:54.702 INFO:teuthology.orchestra.run.smithi047.stdout: "client_readahead_min": "131072", 2024-02-23T00:41:54.702 INFO:teuthology.orchestra.run.smithi047.stdout: "client_reconnect_stale": "false", 2024-02-23T00:41:54.702 INFO:teuthology.orchestra.run.smithi047.stdout: "client_shutdown_timeout": "30", 2024-02-23T00:41:54.702 INFO:teuthology.orchestra.run.smithi047.stdout: "client_snapdir": ".snap", 2024-02-23T00:41:54.702 INFO:teuthology.orchestra.run.smithi047.stdout: "client_tick_interval": "1", 2024-02-23T00:41:54.702 INFO:teuthology.orchestra.run.smithi047.stdout: "client_trace": "", 2024-02-23T00:41:54.702 INFO:teuthology.orchestra.run.smithi047.stdout: "client_try_dentry_invalidate": "false", 2024-02-23T00:41:54.702 INFO:teuthology.orchestra.run.smithi047.stdout: "client_use_faked_inos": "false", 2024-02-23T00:41:54.702 INFO:teuthology.orchestra.run.smithi047.stdout: "client_use_random_mds": "false", 2024-02-23T00:41:54.702 INFO:teuthology.orchestra.run.smithi047.stdout: "clog_to_graylog": "false", 2024-02-23T00:41:54.702 INFO:teuthology.orchestra.run.smithi047.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-02-23T00:41:54.703 INFO:teuthology.orchestra.run.smithi047.stdout: "clog_to_graylog_port": "12201", 2024-02-23T00:41:54.703 INFO:teuthology.orchestra.run.smithi047.stdout: "clog_to_monitors": "default=true", 2024-02-23T00:41:54.703 INFO:teuthology.orchestra.run.smithi047.stdout: "clog_to_syslog": "false", 2024-02-23T00:41:54.703 INFO:teuthology.orchestra.run.smithi047.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-02-23T00:41:54.703 INFO:teuthology.orchestra.run.smithi047.stdout: "clog_to_syslog_level": "info", 2024-02-23T00:41:54.703 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "-", 2024-02-23T00:41:54.703 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_network": "", 2024-02-23T00:41:54.703 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_network_interface": "", 2024-02-23T00:41:54.703 INFO:teuthology.orchestra.run.smithi047.stdout: "compressor_zlib_isal": "false", 2024-02-23T00:41:54.703 INFO:teuthology.orchestra.run.smithi047.stdout: "compressor_zlib_level": "5", 2024-02-23T00:41:54.703 INFO:teuthology.orchestra.run.smithi047.stdout: "compressor_zlib_winsize": "-15", 2024-02-23T00:41:54.703 INFO:teuthology.orchestra.run.smithi047.stdout: "compressor_zstd_level": "1", 2024-02-23T00:41:54.704 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:44277403522cf37edeee63b04643fd6c6e9416f13c3731a771e3cc8e53b7d010", 2024-02-23T00:41:54.704 INFO:teuthology.orchestra.run.smithi047.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-02-23T00:41:54.704 INFO:teuthology.orchestra.run.smithi047.stdout: "crimson_osd_obc_lru_size": "10", 2024-02-23T00:41:54.704 INFO:teuthology.orchestra.run.smithi047.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-02-23T00:41:54.704 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "", 2024-02-23T00:41:54.704 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location_hook": "", 2024-02-23T00:41:54.704 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location_hook_timeout": "10", 2024-02-23T00:41:54.704 INFO:teuthology.orchestra.run.smithi047.stdout: "daemonize": "false", 2024-02-23T00:41:54.704 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_allow_any_pool_priority": "false", 2024-02-23T00:41:54.704 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_asok": "1/5", 2024-02-23T00:41:54.704 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_asok_assert_abort": "false", 2024-02-23T00:41:54.705 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_asserts_on_shutdown": "true", 2024-02-23T00:41:54.705 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_auth": "1/5", 2024-02-23T00:41:54.705 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_bdev": "1/3", 2024-02-23T00:41:54.705 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_bluefs": "1/5", 2024-02-23T00:41:54.705 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_bluestore": "1/5", 2024-02-23T00:41:54.705 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_buffer": "0/1", 2024-02-23T00:41:54.705 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_cephfs_mirror": "0/5", 2024-02-23T00:41:54.705 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_cephsqlite": "0/5", 2024-02-23T00:41:54.705 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_civetweb": "1/10", 2024-02-23T00:41:54.705 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_client": "0/5", 2024-02-23T00:41:54.705 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_compressor": "1/5", 2024-02-23T00:41:54.705 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_context": "0/1", 2024-02-23T00:41:54.706 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_crush": "1/1", 2024-02-23T00:41:54.706 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_crypto": "1/5", 2024-02-23T00:41:54.706 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_deliberately_leak_memory": "false", 2024-02-23T00:41:54.706 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_disable_randomized_ping": "false", 2024-02-23T00:41:54.706 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_dpdk": "1/5", 2024-02-23T00:41:54.706 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_eventtrace": "1/5", 2024-02-23T00:41:54.706 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_filer": "0/1", 2024-02-23T00:41:54.706 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_filestore": "1/3", 2024-02-23T00:41:54.706 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_finisher": "1/1", 2024-02-23T00:41:54.706 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_fuse": "1/5", 2024-02-23T00:41:54.706 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_heartbeat_testing_span": "0", 2024-02-23T00:41:54.706 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_heartbeatmap": "1/5", 2024-02-23T00:41:54.707 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_immutable_obj_cache": "0/5", 2024-02-23T00:41:54.707 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_javaclient": "1/5", 2024-02-23T00:41:54.707 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_journal": "1/3", 2024-02-23T00:41:54.707 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_journaler": "0/5", 2024-02-23T00:41:54.707 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_kstore": "1/5", 2024-02-23T00:41:54.707 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_leveldb": "4/5", 2024-02-23T00:41:54.707 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_lockdep": "0/1", 2024-02-23T00:41:54.707 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_mds": "1/5", 2024-02-23T00:41:54.707 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_mds_balancer": "1/5", 2024-02-23T00:41:54.707 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_mds_locker": "1/5", 2024-02-23T00:41:54.707 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_mds_log": "1/5", 2024-02-23T00:41:54.707 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_mds_log_expire": "1/5", 2024-02-23T00:41:54.708 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_mds_migrator": "1/5", 2024-02-23T00:41:54.708 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_memdb": "4/5", 2024-02-23T00:41:54.708 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_mgr": "2/5", 2024-02-23T00:41:54.708 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_mgrc": "20/20", 2024-02-23T00:41:54.708 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_mon": "20/20", 2024-02-23T00:41:54.708 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_monc": "0/10", 2024-02-23T00:41:54.708 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_ms": "1/1", 2024-02-23T00:41:54.708 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_none": "0/5", 2024-02-23T00:41:54.708 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_objclass": "0/5", 2024-02-23T00:41:54.708 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_objectcacher": "0/5", 2024-02-23T00:41:54.708 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_objecter": "0/1", 2024-02-23T00:41:54.708 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_optracker": "0/5", 2024-02-23T00:41:54.709 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_osd": "1/5", 2024-02-23T00:41:54.709 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_paxos": "20/20", 2024-02-23T00:41:54.709 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_perfcounter": "1/5", 2024-02-23T00:41:54.709 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_prioritycache": "1/5", 2024-02-23T00:41:54.709 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_rados": "0/5", 2024-02-23T00:41:54.709 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_rbd": "0/5", 2024-02-23T00:41:54.709 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_rbd_mirror": "0/5", 2024-02-23T00:41:54.709 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_rbd_pwl": "0/5", 2024-02-23T00:41:54.709 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_rbd_replay": "0/5", 2024-02-23T00:41:54.709 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_refs": "0/0", 2024-02-23T00:41:54.709 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_reserver": "1/1", 2024-02-23T00:41:54.709 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_rgw": "1/5", 2024-02-23T00:41:54.710 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_rgw_sync": "1/5", 2024-02-23T00:41:54.710 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_rocksdb": "4/5", 2024-02-23T00:41:54.710 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_striper": "0/1", 2024-02-23T00:41:54.710 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_test": "0/5", 2024-02-23T00:41:54.710 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_throttle": "1/1", 2024-02-23T00:41:54.710 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_timer": "0/1", 2024-02-23T00:41:54.710 INFO:teuthology.orchestra.run.smithi047.stdout: "debug_tp": "0/5", 2024-02-23T00:41:54.710 INFO:teuthology.orchestra.run.smithi047.stdout: "defer_client_eviction_on_laggy_osds": "true", 2024-02-23T00:41:54.710 INFO:teuthology.orchestra.run.smithi047.stdout: "device_failure_prediction_mode": "none", 2024-02-23T00:41:54.710 INFO:teuthology.orchestra.run.smithi047.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-02-23T00:41:54.710 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-02-23T00:41:54.710 INFO:teuthology.orchestra.run.smithi047.stdout: "err_to_graylog": "false", 2024-02-23T00:41:54.711 INFO:teuthology.orchestra.run.smithi047.stdout: "err_to_stderr": "true", 2024-02-23T00:41:54.711 INFO:teuthology.orchestra.run.smithi047.stdout: "err_to_syslog": "false", 2024-02-23T00:41:54.711 INFO:teuthology.orchestra.run.smithi047.stdout: "event_tracing": "false", 2024-02-23T00:41:54.711 INFO:teuthology.orchestra.run.smithi047.stdout: "fake_statfs_for_testing": "0", 2024-02-23T00:41:54.711 INFO:teuthology.orchestra.run.smithi047.stdout: "fatal_signal_handlers": "true", 2024-02-23T00:41:54.711 INFO:teuthology.orchestra.run.smithi047.stdout: "filer_max_purge_ops": "10", 2024-02-23T00:41:54.711 INFO:teuthology.orchestra.run.smithi047.stdout: "filer_max_truncate_ops": "128", 2024-02-23T00:41:54.711 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_apply_finisher_threads": "1", 2024-02-23T00:41:54.711 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_blackhole": "false", 2024-02-23T00:41:54.711 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_btrfs_clone_range": "true", 2024-02-23T00:41:54.711 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_btrfs_snap": "true", 2024-02-23T00:41:54.711 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_caller_concurrency": "10", 2024-02-23T00:41:54.711 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_collect_device_partition_information": "true", 2024-02-23T00:41:54.712 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_commit_timeout": "600.000000", 2024-02-23T00:41:54.712 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_debug_inject_read_err": "false", 2024-02-23T00:41:54.712 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_debug_omap_check": "false", 2024-02-23T00:41:54.712 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_debug_random_read_err": "0.000000", 2024-02-23T00:41:54.712 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_debug_verify_split": "false", 2024-02-23T00:41:54.712 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_dump_file": "", 2024-02-23T00:41:54.712 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-02-23T00:41:54.712 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-02-23T00:41:54.712 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_fadvise": "true", 2024-02-23T00:41:54.712 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_fail_eio": "true", 2024-02-23T00:41:54.713 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_fd_cache_shards": "16", 2024-02-23T00:41:54.713 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_fd_cache_size": "128", 2024-02-23T00:41:54.713 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_fiemap": "false", 2024-02-23T00:41:54.713 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_fiemap_threshold": "4096", 2024-02-23T00:41:54.713 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-02-23T00:41:54.713 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_index_retry_probability": "0.000000", 2024-02-23T00:41:54.713 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_inject_stall": "0", 2024-02-23T00:41:54.713 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_journal_parallel": "false", 2024-02-23T00:41:54.713 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_journal_trailing": "false", 2024-02-23T00:41:54.713 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_journal_writeahead": "false", 2024-02-23T00:41:54.713 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_kill_at": "0", 2024-02-23T00:41:54.713 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-02-23T00:41:54.713 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_max_inline_xattr_size": "0", 2024-02-23T00:41:54.714 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-02-23T00:41:54.714 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-02-23T00:41:54.714 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-02-23T00:41:54.714 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_max_inline_xattrs": "0", 2024-02-23T00:41:54.714 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-02-23T00:41:54.714 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_max_inline_xattrs_other": "2", 2024-02-23T00:41:54.714 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-02-23T00:41:54.714 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_max_sync_interval": "5.000000", 2024-02-23T00:41:54.714 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_max_xattr_value_size": "0", 2024-02-23T00:41:54.714 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-02-23T00:41:54.715 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-02-23T00:41:54.715 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-02-23T00:41:54.715 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_merge_threshold": "-10", 2024-02-23T00:41:54.715 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_min_sync_interval": "0.010000", 2024-02-23T00:41:54.715 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_odsync_write": "false", 2024-02-23T00:41:54.715 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_omap_backend": "rocksdb", 2024-02-23T00:41:54.715 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_omap_backend_path": "", 2024-02-23T00:41:54.715 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_omap_header_cache_size": "1024", 2024-02-23T00:41:54.715 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_ondisk_finisher_threads": "1", 2024-02-23T00:41:54.715 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-02-23T00:41:54.715 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_op_thread_timeout": "60", 2024-02-23T00:41:54.715 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_op_threads": "2", 2024-02-23T00:41:54.716 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_punch_hole": "false", 2024-02-23T00:41:54.716 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-02-23T00:41:54.716 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-02-23T00:41:54.716 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-02-23T00:41:54.716 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-02-23T00:41:54.716 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-02-23T00:41:54.716 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_queue_max_bytes": "104857600", 2024-02-23T00:41:54.716 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-02-23T00:41:54.716 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-02-23T00:41:54.716 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-02-23T00:41:54.716 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_queue_max_ops": "50", 2024-02-23T00:41:54.717 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-02-23T00:41:54.717 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_seek_data_hole": "false", 2024-02-23T00:41:54.717 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_sloppy_crc": "false", 2024-02-23T00:41:54.717 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-02-23T00:41:54.717 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_splice": "false", 2024-02-23T00:41:54.717 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_split_multiple": "2", 2024-02-23T00:41:54.717 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_split_rand_factor": "20", 2024-02-23T00:41:54.717 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_update_to": "1000", 2024-02-23T00:41:54.717 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-02-23T00:41:54.717 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-02-23T00:41:54.717 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-02-23T00:41:54.718 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-02-23T00:41:54.718 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-02-23T00:41:54.718 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-02-23T00:41:54.718 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_wbthrottle_enable": "true", 2024-02-23T00:41:54.718 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-02-23T00:41:54.718 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-02-23T00:41:54.718 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-02-23T00:41:54.718 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-02-23T00:41:54.718 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-02-23T00:41:54.718 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-02-23T00:41:54.718 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_xfs_extsize": "false", 2024-02-23T00:41:54.719 INFO:teuthology.orchestra.run.smithi047.stdout: "filestore_zfs_snap": "false", 2024-02-23T00:41:54.719 INFO:teuthology.orchestra.run.smithi047.stdout: "fio_dir": "/tmp/fio", 2024-02-23T00:41:54.719 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "fce2617e-d1e2-11ee-95c0-87774f69a715", 2024-02-23T00:41:54.719 INFO:teuthology.orchestra.run.smithi047.stdout: "fuse_allow_other": "true", 2024-02-23T00:41:54.719 INFO:teuthology.orchestra.run.smithi047.stdout: "fuse_atomic_o_trunc": "true", 2024-02-23T00:41:54.719 INFO:teuthology.orchestra.run.smithi047.stdout: "fuse_big_writes": "true", 2024-02-23T00:41:54.719 INFO:teuthology.orchestra.run.smithi047.stdout: "fuse_debug": "false", 2024-02-23T00:41:54.719 INFO:teuthology.orchestra.run.smithi047.stdout: "fuse_default_permissions": "false", 2024-02-23T00:41:54.719 INFO:teuthology.orchestra.run.smithi047.stdout: "fuse_disable_pagecache": "false", 2024-02-23T00:41:54.719 INFO:teuthology.orchestra.run.smithi047.stdout: "fuse_max_write": "0", 2024-02-23T00:41:54.719 INFO:teuthology.orchestra.run.smithi047.stdout: "fuse_multithreaded": "true", 2024-02-23T00:41:54.719 INFO:teuthology.orchestra.run.smithi047.stdout: "fuse_require_active_mds": "true", 2024-02-23T00:41:54.720 INFO:teuthology.orchestra.run.smithi047.stdout: "fuse_set_user_groups": "true", 2024-02-23T00:41:54.720 INFO:teuthology.orchestra.run.smithi047.stdout: "fuse_splice_move": "true", 2024-02-23T00:41:54.720 INFO:teuthology.orchestra.run.smithi047.stdout: "fuse_splice_read": "true", 2024-02-23T00:41:54.720 INFO:teuthology.orchestra.run.smithi047.stdout: "fuse_splice_write": "true", 2024-02-23T00:41:54.720 INFO:teuthology.orchestra.run.smithi047.stdout: "fuse_syncfs_on_mksnap": "true", 2024-02-23T00:41:54.720 INFO:teuthology.orchestra.run.smithi047.stdout: "fuse_use_invalidate_cb": "true", 2024-02-23T00:41:54.720 INFO:teuthology.orchestra.run.smithi047.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-02-23T00:41:54.720 INFO:teuthology.orchestra.run.smithi047.stdout: "gss_target_name": "ceph", 2024-02-23T00:41:54.720 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_file": "", 2024-02-23T00:41:54.720 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_inject_failure": "0", 2024-02-23T00:41:54.720 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_interval": "5", 2024-02-23T00:41:54.721 INFO:teuthology.orchestra.run.smithi047.stdout: "host": "", 2024-02-23T00:41:54.721 INFO:teuthology.orchestra.run.smithi047.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-02-23T00:41:54.721 INFO:teuthology.orchestra.run.smithi047.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-02-23T00:41:54.721 INFO:teuthology.orchestra.run.smithi047.stdout: "immutable_object_cache_max_size": "1073741824", 2024-02-23T00:41:54.721 INFO:teuthology.orchestra.run.smithi047.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-02-23T00:41:54.721 INFO:teuthology.orchestra.run.smithi047.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-02-23T00:41:54.721 INFO:teuthology.orchestra.run.smithi047.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-02-23T00:41:54.721 INFO:teuthology.orchestra.run.smithi047.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-02-23T00:41:54.721 INFO:teuthology.orchestra.run.smithi047.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-02-23T00:41:54.721 INFO:teuthology.orchestra.run.smithi047.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-02-23T00:41:54.721 INFO:teuthology.orchestra.run.smithi047.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-02-23T00:41:54.721 INFO:teuthology.orchestra.run.smithi047.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-02-23T00:41:54.722 INFO:teuthology.orchestra.run.smithi047.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-02-23T00:41:54.722 INFO:teuthology.orchestra.run.smithi047.stdout: "immutable_object_cache_watermark": "0.900000", 2024-02-23T00:41:54.722 INFO:teuthology.orchestra.run.smithi047.stdout: "inject_early_sigterm": "false", 2024-02-23T00:41:54.722 INFO:teuthology.orchestra.run.smithi047.stdout: "journal_aio": "true", 2024-02-23T00:41:54.722 INFO:teuthology.orchestra.run.smithi047.stdout: "journal_align_min_size": "65536", 2024-02-23T00:41:54.722 INFO:teuthology.orchestra.run.smithi047.stdout: "journal_block_align": "true", 2024-02-23T00:41:54.722 INFO:teuthology.orchestra.run.smithi047.stdout: "journal_block_size": "4096", 2024-02-23T00:41:54.722 INFO:teuthology.orchestra.run.smithi047.stdout: "journal_dio": "true", 2024-02-23T00:41:54.722 INFO:teuthology.orchestra.run.smithi047.stdout: "journal_discard": "false", 2024-02-23T00:41:54.722 INFO:teuthology.orchestra.run.smithi047.stdout: "journal_force_aio": "false", 2024-02-23T00:41:54.722 INFO:teuthology.orchestra.run.smithi047.stdout: "journal_ignore_corruption": "false", 2024-02-23T00:41:54.722 INFO:teuthology.orchestra.run.smithi047.stdout: "journal_max_write_bytes": "10485760", 2024-02-23T00:41:54.722 INFO:teuthology.orchestra.run.smithi047.stdout: "journal_max_write_entries": "100", 2024-02-23T00:41:54.723 INFO:teuthology.orchestra.run.smithi047.stdout: "journal_replay_from": "0", 2024-02-23T00:41:54.723 INFO:teuthology.orchestra.run.smithi047.stdout: "journal_throttle_high_multiple": "0.000000", 2024-02-23T00:41:54.723 INFO:teuthology.orchestra.run.smithi047.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-02-23T00:41:54.723 INFO:teuthology.orchestra.run.smithi047.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-02-23T00:41:54.723 INFO:teuthology.orchestra.run.smithi047.stdout: "journal_throttle_max_multiple": "0.000000", 2024-02-23T00:41:54.723 INFO:teuthology.orchestra.run.smithi047.stdout: "journal_write_header_frequency": "0", 2024-02-23T00:41:54.723 INFO:teuthology.orchestra.run.smithi047.stdout: "journal_zero_on_create": "false", 2024-02-23T00:41:54.723 INFO:teuthology.orchestra.run.smithi047.stdout: "journaler_prefetch_periods": "10", 2024-02-23T00:41:54.723 INFO:teuthology.orchestra.run.smithi047.stdout: "journaler_prezero_periods": "5", 2024-02-23T00:41:54.723 INFO:teuthology.orchestra.run.smithi047.stdout: "journaler_write_head_interval": "15", 2024-02-23T00:41:54.723 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "", 2024-02-23T00:41:54.723 INFO:teuthology.orchestra.run.smithi047.stdout: "keyfile": "", 2024-02-23T00:41:54.723 INFO:teuthology.orchestra.run.smithi047.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-02-23T00:41:54.724 INFO:teuthology.orchestra.run.smithi047.stdout: "kstore_backend": "rocksdb", 2024-02-23T00:41:54.724 INFO:teuthology.orchestra.run.smithi047.stdout: "kstore_default_stripe_size": "65536", 2024-02-23T00:41:54.724 INFO:teuthology.orchestra.run.smithi047.stdout: "kstore_fsck_on_mount": "false", 2024-02-23T00:41:54.724 INFO:teuthology.orchestra.run.smithi047.stdout: "kstore_fsck_on_mount_deep": "true", 2024-02-23T00:41:54.724 INFO:teuthology.orchestra.run.smithi047.stdout: "kstore_max_bytes": "67108864", 2024-02-23T00:41:54.724 INFO:teuthology.orchestra.run.smithi047.stdout: "kstore_max_ops": "512", 2024-02-23T00:41:54.724 INFO:teuthology.orchestra.run.smithi047.stdout: "kstore_nid_prealloc": "1024", 2024-02-23T00:41:54.724 INFO:teuthology.orchestra.run.smithi047.stdout: "kstore_onode_map_size": "1024", 2024-02-23T00:41:54.724 INFO:teuthology.orchestra.run.smithi047.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-02-23T00:41:54.724 INFO:teuthology.orchestra.run.smithi047.stdout: "kstore_sync_submit_transaction": "false", 2024-02-23T00:41:54.724 INFO:teuthology.orchestra.run.smithi047.stdout: "kstore_sync_transaction": "false", 2024-02-23T00:41:54.724 INFO:teuthology.orchestra.run.smithi047.stdout: "leveldb_block_size": "65536", 2024-02-23T00:41:54.725 INFO:teuthology.orchestra.run.smithi047.stdout: "leveldb_bloom_size": "0", 2024-02-23T00:41:54.725 INFO:teuthology.orchestra.run.smithi047.stdout: "leveldb_cache_size": "536870912", 2024-02-23T00:41:54.725 INFO:teuthology.orchestra.run.smithi047.stdout: "leveldb_compact_on_mount": "false", 2024-02-23T00:41:54.725 INFO:teuthology.orchestra.run.smithi047.stdout: "leveldb_compression": "false", 2024-02-23T00:41:54.725 INFO:teuthology.orchestra.run.smithi047.stdout: "leveldb_log": "", 2024-02-23T00:41:54.725 INFO:teuthology.orchestra.run.smithi047.stdout: "leveldb_log_to_ceph_log": "true", 2024-02-23T00:41:54.725 INFO:teuthology.orchestra.run.smithi047.stdout: "leveldb_max_open_files": "0", 2024-02-23T00:41:54.725 INFO:teuthology.orchestra.run.smithi047.stdout: "leveldb_paranoid": "false", 2024-02-23T00:41:54.725 INFO:teuthology.orchestra.run.smithi047.stdout: "leveldb_write_buffer_size": "33554432", 2024-02-23T00:41:54.725 INFO:teuthology.orchestra.run.smithi047.stdout: "librados_thread_count": "2", 2024-02-23T00:41:54.725 INFO:teuthology.orchestra.run.smithi047.stdout: "lockdep": "false", 2024-02-23T00:41:54.725 INFO:teuthology.orchestra.run.smithi047.stdout: "lockdep_force_backtrace": "false", 2024-02-23T00:41:54.725 INFO:teuthology.orchestra.run.smithi047.stdout: "log_coarse_timestamps": "true", 2024-02-23T00:41:54.726 INFO:teuthology.orchestra.run.smithi047.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-02-23T00:41:54.726 INFO:teuthology.orchestra.run.smithi047.stdout: "log_flush_on_exit": "false", 2024-02-23T00:41:54.726 INFO:teuthology.orchestra.run.smithi047.stdout: "log_graylog_host": "127.0.0.1", 2024-02-23T00:41:54.726 INFO:teuthology.orchestra.run.smithi047.stdout: "log_graylog_port": "12201", 2024-02-23T00:41:54.726 INFO:teuthology.orchestra.run.smithi047.stdout: "log_max_new": "1000", 2024-02-23T00:41:54.726 INFO:teuthology.orchestra.run.smithi047.stdout: "log_max_recent": "10000", 2024-02-23T00:41:54.726 INFO:teuthology.orchestra.run.smithi047.stdout: "log_stderr_prefix": "debug ", 2024-02-23T00:41:54.726 INFO:teuthology.orchestra.run.smithi047.stdout: "log_stop_at_utilization": "0.970000", 2024-02-23T00:41:54.726 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": "true", 2024-02-23T00:41:54.726 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_graylog": "false", 2024-02-23T00:41:54.726 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_stderr": "false", 2024-02-23T00:41:54.727 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_syslog": "false", 2024-02-23T00:41:54.727 INFO:teuthology.orchestra.run.smithi047.stdout: "max_rotating_auth_attempts": "10", 2024-02-23T00:41:54.727 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-02-23T00:41:54.727 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_action_on_write_error": "1", 2024-02-23T00:41:54.727 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_alternate_name_max": "8192", 2024-02-23T00:41:54.727 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_asio_thread_count": "2", 2024-02-23T00:41:54.727 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_bal_export_pin": "true", 2024-02-23T00:41:54.727 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_bal_fragment_dirs": "true", 2024-02-23T00:41:54.727 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-02-23T00:41:54.727 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_bal_fragment_interval": "5", 2024-02-23T00:41:54.727 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_bal_fragment_size_max": "100000", 2024-02-23T00:41:54.728 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_bal_idle_threshold": "0.000000", 2024-02-23T00:41:54.728 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_bal_interval": "10", 2024-02-23T00:41:54.728 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_bal_max": "-1", 2024-02-23T00:41:54.728 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_bal_max_until": "-1", 2024-02-23T00:41:54.728 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_bal_merge_size": "50", 2024-02-23T00:41:54.728 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_bal_midchunk": "0.300000", 2024-02-23T00:41:54.728 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_bal_min_rebalance": "0.100000", 2024-02-23T00:41:54.728 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_bal_min_start": "0.200000", 2024-02-23T00:41:54.728 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_bal_minchunk": "0.001000", 2024-02-23T00:41:54.728 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_bal_mode": "0", 2024-02-23T00:41:54.728 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_bal_need_max": "1.200000", 2024-02-23T00:41:54.728 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_bal_need_min": "0.800000", 2024-02-23T00:41:54.729 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-02-23T00:41:54.729 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_bal_sample_interval": "3.000000", 2024-02-23T00:41:54.729 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_bal_split_bits": "3", 2024-02-23T00:41:54.729 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_bal_split_rd": "25000.000000", 2024-02-23T00:41:54.729 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_bal_split_size": "10000", 2024-02-23T00:41:54.729 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_bal_split_wr": "10000.000000", 2024-02-23T00:41:54.729 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_bal_target_decay": "10.000000", 2024-02-23T00:41:54.729 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-02-23T00:41:54.729 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_beacon_grace": "15.000000", 2024-02-23T00:41:54.729 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_beacon_interval": "4.000000", 2024-02-23T00:41:54.729 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_beacon_mon_down_grace": "60", 2024-02-23T00:41:54.729 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_cache_memory_limit": "4294967296", 2024-02-23T00:41:54.730 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_cache_mid": "0.700000", 2024-02-23T00:41:54.730 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_cache_release_free_interval": "10", 2024-02-23T00:41:54.730 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_cache_reservation": "0.050000", 2024-02-23T00:41:54.730 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-02-23T00:41:54.730 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_cache_trim_interval": "1", 2024-02-23T00:41:54.730 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_cache_trim_threshold": "262144", 2024-02-23T00:41:54.730 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-02-23T00:41:54.730 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-02-23T00:41:54.730 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_client_delegate_inos_pct": "50", 2024-02-23T00:41:54.730 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_client_prealloc_inos": "1000", 2024-02-23T00:41:54.730 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-02-23T00:41:54.730 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_connect_bootstrapping": "false", 2024-02-23T00:41:54.730 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_damage_table_max_entries": "10000", 2024-02-23T00:41:54.731 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-02-23T00:41:54.731 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_debug_auth_pins": "false", 2024-02-23T00:41:54.731 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_debug_frag": "false", 2024-02-23T00:41:54.731 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_debug_scatterstat": "false", 2024-02-23T00:41:54.731 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_debug_subtrees": "false", 2024-02-23T00:41:54.731 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_decay_halflife": "5.000000", 2024-02-23T00:41:54.731 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_default_dir_hash": "2", 2024-02-23T00:41:54.731 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_defer_session_stale": "true", 2024-02-23T00:41:54.731 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_deny_all_reconnect": "false", 2024-02-23T00:41:54.731 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_dir_keys_per_op": "16384", 2024-02-23T00:41:54.731 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_dir_max_commit_size": "10", 2024-02-23T00:41:54.731 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_dir_max_entries": "0", 2024-02-23T00:41:54.732 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_dirstat_min_interval": "1.000000", 2024-02-23T00:41:54.732 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_dump_cache_after_rejoin": "false", 2024-02-23T00:41:54.732 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_dump_cache_on_map": "false", 2024-02-23T00:41:54.732 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_dump_cache_threshold_file": "0", 2024-02-23T00:41:54.732 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-02-23T00:41:54.732 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_early_reply": "true", 2024-02-23T00:41:54.732 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_enable_op_tracker": "true", 2024-02-23T00:41:54.732 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_enforce_unique_name": "true", 2024-02-23T00:41:54.732 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_export_ephemeral_distributed": "true", 2024-02-23T00:41:54.732 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-02-23T00:41:54.732 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_export_ephemeral_random": "true", 2024-02-23T00:41:54.732 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-02-23T00:41:54.733 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_forward_all_requests_to_auth": "false", 2024-02-23T00:41:54.733 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-02-23T00:41:54.733 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-02-23T00:41:54.733 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-02-23T00:41:54.733 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_health_cache_threshold": "1.500000", 2024-02-23T00:41:54.733 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_health_summarize_threshold": "10", 2024-02-23T00:41:54.733 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_heartbeat_grace": "15.000000", 2024-02-23T00:41:54.733 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_heartbeat_reset_grace": "1000", 2024-02-23T00:41:54.733 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_inject_health_dummy": "false", 2024-02-23T00:41:54.733 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-02-23T00:41:54.733 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_inject_migrator_session_race": "false", 2024-02-23T00:41:54.733 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-02-23T00:41:54.733 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-02-23T00:41:54.734 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-02-23T00:41:54.734 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_join_fs": "", 2024-02-23T00:41:54.734 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_journal_format": "1", 2024-02-23T00:41:54.734 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_kill_create_at": "0", 2024-02-23T00:41:54.734 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_kill_export_at": "0", 2024-02-23T00:41:54.734 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_kill_import_at": "0", 2024-02-23T00:41:54.734 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_kill_journal_at": "0", 2024-02-23T00:41:54.734 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_kill_journal_expire_at": "0", 2024-02-23T00:41:54.734 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_kill_journal_replay_at": "0", 2024-02-23T00:41:54.734 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_kill_link_at": "0", 2024-02-23T00:41:54.734 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_kill_mdstable_at": "0", 2024-02-23T00:41:54.734 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_kill_openc_at": "0", 2024-02-23T00:41:54.735 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_kill_rename_at": "0", 2024-02-23T00:41:54.735 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_kill_skip_replaying_inotable": "false", 2024-02-23T00:41:54.735 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_log_events_per_segment": "1024", 2024-02-23T00:41:54.735 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_log_max_events": "-1", 2024-02-23T00:41:54.735 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_log_max_segments": "128", 2024-02-23T00:41:54.735 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_log_pause": "false", 2024-02-23T00:41:54.735 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_log_segment_size": "0", 2024-02-23T00:41:54.735 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_log_skip_corrupt_events": "false", 2024-02-23T00:41:54.735 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_log_warn_factor": "2.000000", 2024-02-23T00:41:54.735 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_max_caps_per_client": "1048576", 2024-02-23T00:41:54.735 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_max_completed_flushes": "100000", 2024-02-23T00:41:54.735 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_max_completed_requests": "100000", 2024-02-23T00:41:54.736 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_max_export_size": "20971520", 2024-02-23T00:41:54.736 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_max_file_recover": "32", 2024-02-23T00:41:54.736 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_max_purge_files": "64", 2024-02-23T00:41:54.736 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_max_purge_ops": "8192", 2024-02-23T00:41:54.736 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-02-23T00:41:54.736 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-02-23T00:41:54.736 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_max_snaps_per_dir": "100", 2024-02-23T00:41:54.736 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_max_xattr_pairs_size": "65536", 2024-02-23T00:41:54.736 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_metrics_update_interval": "2", 2024-02-23T00:41:54.736 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_min_caps_per_client": "100", 2024-02-23T00:41:54.736 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_min_caps_working_set": "10000", 2024-02-23T00:41:54.736 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-02-23T00:41:54.737 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_numa_node": "-1", 2024-02-23T00:41:54.737 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-02-23T00:41:54.737 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_op_complaint_time": "30.000000", 2024-02-23T00:41:54.737 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_op_history_duration": "600", 2024-02-23T00:41:54.737 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_op_history_size": "20", 2024-02-23T00:41:54.737 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_op_log_threshold": "5", 2024-02-23T00:41:54.737 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_ping_grace": "15", 2024-02-23T00:41:54.737 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_ping_interval": "5", 2024-02-23T00:41:54.737 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-02-23T00:41:54.737 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-02-23T00:41:54.737 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_recall_max_caps": "30000", 2024-02-23T00:41:54.737 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-02-23T00:41:54.737 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_recall_max_decay_threshold": "131072", 2024-02-23T00:41:54.738 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-02-23T00:41:54.738 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_recall_warning_threshold": "262144", 2024-02-23T00:41:54.738 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_reconnect_timeout": "45.000000", 2024-02-23T00:41:54.738 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_replay_interval": "1.000000", 2024-02-23T00:41:54.738 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-02-23T00:41:54.738 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-02-23T00:41:54.738 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_root_ino_gid": "0", 2024-02-23T00:41:54.738 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_root_ino_uid": "0", 2024-02-23T00:41:54.738 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-02-23T00:41:54.738 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_session_blocklist_on_evict": "true", 2024-02-23T00:41:54.738 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_session_blocklist_on_timeout": "true", 2024-02-23T00:41:54.738 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-02-23T00:41:54.739 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-02-23T00:41:54.739 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-02-23T00:41:54.739 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-02-23T00:41:54.739 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-02-23T00:41:54.739 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_session_metadata_threshold": "16777216", 2024-02-23T00:41:54.739 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-02-23T00:41:54.739 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_shutdown_check": "0", 2024-02-23T00:41:54.739 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_skip_ino": "0", 2024-02-23T00:41:54.739 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_sleep_rank_change": "0.000000", 2024-02-23T00:41:54.739 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_snap_max_uid": "4294967294", 2024-02-23T00:41:54.739 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_snap_min_uid": "0", 2024-02-23T00:41:54.739 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_snap_rstat": "false", 2024-02-23T00:41:54.739 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_standby_replay_damaged": "false", 2024-02-23T00:41:54.740 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_task_status_update_interval": "2.000000", 2024-02-23T00:41:54.740 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_thrash_exports": "0", 2024-02-23T00:41:54.740 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_thrash_fragments": "0", 2024-02-23T00:41:54.740 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_tick_interval": "5.000000", 2024-02-23T00:41:54.740 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_valgrind_exit": "false", 2024-02-23T00:41:54.740 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_verify_backtrace": "1", 2024-02-23T00:41:54.740 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_verify_scatter": "false", 2024-02-23T00:41:54.740 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_wipe_ino_prealloc": "false", 2024-02-23T00:41:54.740 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_wipe_sessions": "false", 2024-02-23T00:41:54.740 INFO:teuthology.orchestra.run.smithi047.stdout: "mempool_debug": "false", 2024-02-23T00:41:54.740 INFO:teuthology.orchestra.run.smithi047.stdout: "memstore_debug_omit_block_device_write": "false", 2024-02-23T00:41:54.741 INFO:teuthology.orchestra.run.smithi047.stdout: "memstore_device_bytes": "1073741824", 2024-02-23T00:41:54.741 INFO:teuthology.orchestra.run.smithi047.stdout: "memstore_page_set": "false", 2024-02-23T00:41:54.741 INFO:teuthology.orchestra.run.smithi047.stdout: "memstore_page_size": "65536", 2024-02-23T00:41:54.741 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_client_bytes": "134217728", 2024-02-23T00:41:54.741 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_client_messages": "512", 2024-02-23T00:41:54.741 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-02-23T00:41:54.741 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_connect_retry_interval": "1.000000", 2024-02-23T00:41:54.741 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-02-23T00:41:54.741 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-02-23T00:41:54.741 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_disabled_modules": "", 2024-02-23T00:41:54.742 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-02-23T00:41:54.742 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_max_pg_num_change": "128", 2024-02-23T00:41:54.742 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_mds_bytes": "134217728", 2024-02-23T00:41:54.742 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_mds_messages": "128", 2024-02-23T00:41:54.742 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-02-23T00:41:54.742 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_mon_bytes": "134217728", 2024-02-23T00:41:54.742 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_mon_messages": "128", 2024-02-23T00:41:54.742 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_osd_bytes": "536870912", 2024-02-23T00:41:54.742 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_osd_messages": "8192", 2024-02-23T00:41:54.742 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_service_beacon_grace": "60.000000", 2024-02-23T00:41:54.742 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_standby_modules": "true", 2024-02-23T00:41:54.743 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_stats_period": "5", 2024-02-23T00:41:54.743 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_stats_threshold": "5", 2024-02-23T00:41:54.743 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_tick_period": "2", 2024-02-23T00:41:54.743 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-02-23T00:41:54.743 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_accept_timeout_factor": "2.000000", 2024-02-23T00:41:54.743 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_allow_pool_delete": "true", 2024-02-23T00:41:54.743 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_allow_pool_size_one": "false", 2024-02-23T00:41:54.743 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_auth_validate_all_caps": "true", 2024-02-23T00:41:54.743 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-02-23T00:41:54.743 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-02-23T00:41:54.744 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_client_bytes": "104857600", 2024-02-23T00:41:54.744 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_client_directed_command_retry": "2", 2024-02-23T00:41:54.744 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_client_hunt_interval": "3.000000", 2024-02-23T00:41:54.744 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-02-23T00:41:54.744 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-02-23T00:41:54.744 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-02-23T00:41:54.744 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_client_hunt_parallel": "3", 2024-02-23T00:41:54.744 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_client_log_interval": "1.000000", 2024-02-23T00:41:54.744 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-02-23T00:41:54.744 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_client_ping_interval": "10.000000", 2024-02-23T00:41:54.744 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_client_ping_timeout": "30.000000", 2024-02-23T00:41:54.744 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_clock_drift_allowed": "1.000000", 2024-02-23T00:41:54.745 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-02-23T00:41:54.745 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-02-23T00:41:54.745 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_cluster_log_file_level": "debug", 2024-02-23T00:41:54.745 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_cluster_log_to_file": "true", 2024-02-23T00:41:54.745 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_cluster_log_to_graylog": "false", 2024-02-23T00:41:54.745 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-02-23T00:41:54.745 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-02-23T00:41:54.745 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_cluster_log_to_stderr": "true", 2024-02-23T00:41:54.745 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-02-23T00:41:54.745 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-02-23T00:41:54.745 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-02-23T00:41:54.745 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_compact_on_bootstrap": "false", 2024-02-23T00:41:54.745 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_compact_on_start": "false", 2024-02-23T00:41:54.746 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_compact_on_trim": "true", 2024-02-23T00:41:54.746 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_con_tracker_persist_interval": "10", 2024-02-23T00:41:54.746 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_con_tracker_score_halflife": "43200", 2024-02-23T00:41:54.746 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_config_key_max_entry_size": "65536", 2024-02-23T00:41:54.746 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_cpu_threads": "4", 2024-02-23T00:41:54.746 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_crush_min_required_version": "hammer", 2024-02-23T00:41:54.746 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_daemon_bytes": "419430400", 2024-02-23T00:41:54.746 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-02-23T00:41:54.746 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_data_avail_crit": "5", 2024-02-23T00:41:54.746 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_data_avail_warn": "5", 2024-02-23T00:41:54.746 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_data_size_warn": "16106127360", 2024-02-23T00:41:54.746 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_debug_block_osdmap_trim": "false", 2024-02-23T00:41:54.747 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-02-23T00:41:54.747 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_debug_dump_json": "false", 2024-02-23T00:41:54.747 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-02-23T00:41:54.747 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_debug_dump_transactions": "false", 2024-02-23T00:41:54.747 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_debug_extra_checks": "false", 2024-02-23T00:41:54.747 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-02-23T00:41:54.747 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-02-23T00:41:54.747 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_debug_no_require_octopus": "false", 2024-02-23T00:41:54.747 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_debug_no_require_pacific": "false", 2024-02-23T00:41:54.747 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-02-23T00:41:54.747 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_delta_reset_interval": "10.000000", 2024-02-23T00:41:54.747 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_dns_srv_name": "ceph-mon", 2024-02-23T00:41:54.747 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_down_mkfs_grace": "60", 2024-02-23T00:41:54.748 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_election_default_strategy": "1", 2024-02-23T00:41:54.748 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_election_timeout": "5.000000", 2024-02-23T00:41:54.748 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-02-23T00:41:54.748 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_elector_ping_divisor": "2", 2024-02-23T00:41:54.748 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_elector_ping_timeout": "2.000000", 2024-02-23T00:41:54.748 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_enable_op_tracker": "true", 2024-02-23T00:41:54.748 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_fake_pool_delete": "false", 2024-02-23T00:41:54.748 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_force_quorum_join": "false", 2024-02-23T00:41:54.748 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_globalid_prealloc": "10000", 2024-02-23T00:41:54.748 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_health_detail_to_clog": "true", 2024-02-23T00:41:54.748 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_health_log_update_period": "5", 2024-02-23T00:41:54.749 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_health_max_detail": "50", 2024-02-23T00:41:54.749 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_health_to_clog": "true", 2024-02-23T00:41:54.749 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_health_to_clog_interval": "600", 2024-02-23T00:41:54.749 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-02-23T00:41:54.749 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_host": "[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0]", 2024-02-23T00:41:54.749 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_host_override": "", 2024-02-23T00:41:54.749 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_initial_members": "", 2024-02-23T00:41:54.749 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-02-23T00:41:54.749 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-02-23T00:41:54.749 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-02-23T00:41:54.749 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-02-23T00:41:54.749 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_keyvaluedb": "rocksdb", 2024-02-23T00:41:54.749 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_lease": "5.000000", 2024-02-23T00:41:54.750 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-02-23T00:41:54.750 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-02-23T00:41:54.750 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_log_max_summary": "50", 2024-02-23T00:41:54.750 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_max_log_entries_per_event": "4096", 2024-02-23T00:41:54.750 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_max_log_epochs": "500", 2024-02-23T00:41:54.750 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_max_mdsmap_epochs": "500", 2024-02-23T00:41:54.750 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_max_mgrmap_epochs": "500", 2024-02-23T00:41:54.750 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_max_osd": "10000", 2024-02-23T00:41:54.750 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_max_pg_per_osd": "10000", 2024-02-23T00:41:54.750 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_max_pool_pg_num": "65536", 2024-02-23T00:41:54.750 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-02-23T00:41:54.750 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-02-23T00:41:54.751 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_mds_force_trim_to": "0", 2024-02-23T00:41:54.751 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_mds_skip_sanity": "false", 2024-02-23T00:41:54.751 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_memory_autotune": "true", 2024-02-23T00:41:54.751 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_memory_target": "2147483648", 2024-02-23T00:41:54.751 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_mgr_beacon_grace": "30", 2024-02-23T00:41:54.751 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-02-23T00:41:54.751 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_mgr_digest_period": "5", 2024-02-23T00:41:54.751 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_mgr_inactive_grace": "60", 2024-02-23T00:41:54.751 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_mgr_mkfs_grace": "240", 2024-02-23T00:41:54.751 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-02-23T00:41:54.751 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_min_osdmap_epochs": "50", 2024-02-23T00:41:54.751 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_op_complaint_time": "30", 2024-02-23T00:41:54.752 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_op_history_duration": "600", 2024-02-23T00:41:54.752 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_op_history_size": "20", 2024-02-23T00:41:54.752 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_op_history_slow_op_size": "20", 2024-02-23T00:41:54.752 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_op_history_slow_op_threshold": "10", 2024-02-23T00:41:54.752 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_op_log_threshold": "5", 2024-02-23T00:41:54.752 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-02-23T00:41:54.752 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-02-23T00:41:54.752 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-02-23T00:41:54.752 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_auto_mark_in": "false", 2024-02-23T00:41:54.753 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_auto_mark_new_in": "true", 2024-02-23T00:41:54.753 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-02-23T00:41:54.753 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-02-23T00:41:54.753 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_cache_size": "500", 2024-02-23T00:41:54.753 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_cache_size_min": "134217728", 2024-02-23T00:41:54.753 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_crush_smoke_test": "true", 2024-02-23T00:41:54.753 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_destroyed_out_interval": "600", 2024-02-23T00:41:54.753 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_down_out_interval": "600", 2024-02-23T00:41:54.753 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-02-23T00:41:54.753 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-02-23T00:41:54.754 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_force_trim_to": "0", 2024-02-23T00:41:54.754 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_full_ratio": "0.950000", 2024-02-23T00:41:54.754 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-02-23T00:41:54.754 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_laggy_halflife": "3600", 2024-02-23T00:41:54.754 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_laggy_max_interval": "300", 2024-02-23T00:41:54.754 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_laggy_weight": "0.300000", 2024-02-23T00:41:54.754 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-02-23T00:41:54.754 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_max_creating_pgs": "1024", 2024-02-23T00:41:54.754 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_max_initial_pgs": "1024", 2024-02-23T00:41:54.754 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_min_down_reporters": "2", 2024-02-23T00:41:54.754 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-02-23T00:41:54.754 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-02-23T00:41:54.754 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-02-23T00:41:54.755 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_prime_pg_temp": "true", 2024-02-23T00:41:54.755 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-02-23T00:41:54.755 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-02-23T00:41:54.755 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_report_timeout": "900", 2024-02-23T00:41:54.755 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-02-23T00:41:54.755 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-02-23T00:41:54.755 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_warn_num_repaired": "10", 2024-02-23T00:41:54.755 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osd_warn_op_age": "32.000000", 2024-02-23T00:41:54.755 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-02-23T00:41:54.755 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osdmap_full_prune_interval": "2", 2024-02-23T00:41:54.755 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osdmap_full_prune_min": "15", 2024-02-23T00:41:54.755 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_osdmap_full_prune_txsize": "2", 2024-02-23T00:41:54.756 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-02-23T00:41:54.756 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_pg_stuck_threshold": "60", 2024-02-23T00:41:54.756 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-02-23T00:41:54.756 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_pg_warn_min_objects": "10000", 2024-02-23T00:41:54.756 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_pg_warn_min_per_osd": "0", 2024-02-23T00:41:54.756 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-02-23T00:41:54.756 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_pool_quota_crit_threshold": "0", 2024-02-23T00:41:54.756 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_pool_quota_warn_threshold": "0", 2024-02-23T00:41:54.756 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_probe_timeout": "2.000000", 2024-02-23T00:41:54.756 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_reweight_max_change": "0.050000", 2024-02-23T00:41:54.756 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_reweight_max_osds": "4", 2024-02-23T00:41:54.756 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-02-23T00:41:54.757 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-02-23T00:41:54.757 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-02-23T00:41:54.757 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-02-23T00:41:54.757 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-02-23T00:41:54.757 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_scrub_interval": "86400", 2024-02-23T00:41:54.757 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_scrub_max_keys": "100", 2024-02-23T00:41:54.757 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_scrub_timeout": "300", 2024-02-23T00:41:54.757 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_session_timeout": "300", 2024-02-23T00:41:54.757 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_smart_report_timeout": "5", 2024-02-23T00:41:54.757 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_stat_smooth_intervals": "6", 2024-02-23T00:41:54.757 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-02-23T00:41:54.757 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_stretch_pool_min_size": "2", 2024-02-23T00:41:54.758 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_stretch_pool_size": "4", 2024-02-23T00:41:54.758 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-02-23T00:41:54.758 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_subscribe_interval": "86400.000000", 2024-02-23T00:41:54.758 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_sync_debug": "false", 2024-02-23T00:41:54.758 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_sync_max_payload_keys": "2000", 2024-02-23T00:41:54.758 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_sync_max_payload_size": "1048576", 2024-02-23T00:41:54.758 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_sync_provider_kill_at": "0", 2024-02-23T00:41:54.758 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_sync_requester_kill_at": "0", 2024-02-23T00:41:54.758 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_sync_timeout": "60.000000", 2024-02-23T00:41:54.758 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_target_pg_per_osd": "100", 2024-02-23T00:41:54.758 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_tick_interval": "5", 2024-02-23T00:41:54.758 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_timecheck_interval": "300.000000", 2024-02-23T00:41:54.759 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-02-23T00:41:54.759 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_warn_older_version_delay": "604800", 2024-02-23T00:41:54.759 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-02-23T00:41:54.759 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-02-23T00:41:54.759 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-02-23T00:41:54.759 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-02-23T00:41:54.759 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-02-23T00:41:54.759 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-02-23T00:41:54.759 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_warn_on_misplaced": "false", 2024-02-23T00:41:54.759 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-02-23T00:41:54.759 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_warn_on_older_version": "true", 2024-02-23T00:41:54.759 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-02-23T00:41:54.760 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_warn_on_pool_no_app": "false", 2024-02-23T00:41:54.760 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_warn_on_pool_no_redundancy": "true", 2024-02-23T00:41:54.760 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-02-23T00:41:54.760 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-02-23T00:41:54.760 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-02-23T00:41:54.760 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_warn_on_too_few_osds": "false", 2024-02-23T00:41:54.760 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-02-23T00:41:54.760 INFO:teuthology.orchestra.run.smithi047.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-02-23T00:41:54.760 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": "", 2024-02-23T00:41:54.760 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_async_op_threads": "3", 2024-02-23T00:41:54.760 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_async_rdma_buffer_size": "131072", 2024-02-23T00:41:54.761 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_async_rdma_cm": "false", 2024-02-23T00:41:54.761 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_async_rdma_device_name": "", 2024-02-23T00:41:54.761 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_async_rdma_dscp": "96", 2024-02-23T00:41:54.761 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-02-23T00:41:54.761 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_async_rdma_gid_idx": "0", 2024-02-23T00:41:54.761 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_async_rdma_local_gid": "", 2024-02-23T00:41:54.761 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_async_rdma_polling_us": "1000", 2024-02-23T00:41:54.761 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_async_rdma_port_num": "1", 2024-02-23T00:41:54.761 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-02-23T00:41:54.761 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-02-23T00:41:54.761 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_async_rdma_roce_ver": "1", 2024-02-23T00:41:54.761 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_async_rdma_send_buffers": "1024", 2024-02-23T00:41:54.761 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_async_rdma_sl": "3", 2024-02-23T00:41:54.762 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_async_rdma_support_srq": "true", 2024-02-23T00:41:54.762 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_async_rdma_type": "ib", 2024-02-23T00:41:54.762 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_async_reap_threshold": "5", 2024-02-23T00:41:54.762 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_bind_before_connect": "false", 2024-02-23T00:41:54.762 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_bind_ipv4": "true", 2024-02-23T00:41:54.762 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_bind_ipv6": "false", 2024-02-23T00:41:54.762 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_bind_msgr1": "true", 2024-02-23T00:41:54.762 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_bind_msgr2": "true", 2024-02-23T00:41:54.762 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_bind_port_max": "7300", 2024-02-23T00:41:54.762 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_bind_port_min": "6800", 2024-02-23T00:41:54.762 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_bind_prefer_ipv4": "false", 2024-02-23T00:41:54.762 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_bind_retry_count": "3", 2024-02-23T00:41:54.763 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_bind_retry_delay": "5", 2024-02-23T00:41:54.763 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_blackhole_client": "false", 2024-02-23T00:41:54.763 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_blackhole_mds": "false", 2024-02-23T00:41:54.763 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_blackhole_mgr": "false", 2024-02-23T00:41:54.763 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_blackhole_mon": "false", 2024-02-23T00:41:54.763 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_blackhole_osd": "false", 2024-02-23T00:41:54.763 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_client_mode": "crc secure", 2024-02-23T00:41:54.763 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_cluster_mode": "crc secure", 2024-02-23T00:41:54.763 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_cluster_type": "", 2024-02-23T00:41:54.763 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_connection_idle_timeout": "900", 2024-02-23T00:41:54.763 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_connection_ready_timeout": "10", 2024-02-23T00:41:54.763 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_crc_data": "true", 2024-02-23T00:41:54.764 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_crc_header": "true", 2024-02-23T00:41:54.764 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_die_on_bad_msg": "false", 2024-02-23T00:41:54.764 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_die_on_bug": "true", 2024-02-23T00:41:54.764 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_die_on_old_message": "true", 2024-02-23T00:41:54.764 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_die_on_skipped_message": "false", 2024-02-23T00:41:54.764 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_die_on_unhandled_msg": "false", 2024-02-23T00:41:54.764 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-02-23T00:41:54.764 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_dpdk_coremask": "0xF", 2024-02-23T00:41:54.764 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-02-23T00:41:54.764 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-02-23T00:41:54.764 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-02-23T00:41:54.765 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_dpdk_hugepages": "", 2024-02-23T00:41:54.765 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_dpdk_hw_flow_control": "true", 2024-02-23T00:41:54.765 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-02-23T00:41:54.765 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_dpdk_lro": "true", 2024-02-23T00:41:54.765 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_dpdk_memory_channel": "4", 2024-02-23T00:41:54.765 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-02-23T00:41:54.765 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_dpdk_pmd": "", 2024-02-23T00:41:54.765 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_dpdk_port_id": "0", 2024-02-23T00:41:54.765 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-02-23T00:41:54.765 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_dump_corrupt_message_level": "1", 2024-02-23T00:41:54.765 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_dump_on_send": "false", 2024-02-23T00:41:54.765 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_initial_backoff": "0.200000", 2024-02-23T00:41:54.766 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_inject_delay_max": "1.000000", 2024-02-23T00:41:54.766 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_inject_delay_probability": "0.000000", 2024-02-23T00:41:54.766 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_inject_delay_type": "", 2024-02-23T00:41:54.766 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_inject_internal_delays": "0.000000", 2024-02-23T00:41:54.766 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_inject_socket_failures": "0", 2024-02-23T00:41:54.766 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_learn_addr_from_peer": "true", 2024-02-23T00:41:54.766 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_max_accept_failures": "4", 2024-02-23T00:41:54.766 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_max_backoff": "15.000000", 2024-02-23T00:41:54.766 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_mon_client_mode": "secure crc", 2024-02-23T00:41:54.766 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_mon_cluster_mode": "secure crc", 2024-02-23T00:41:54.766 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_mon_service_mode": "secure crc", 2024-02-23T00:41:54.766 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-02-23T00:41:54.767 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_pq_min_cost": "65536", 2024-02-23T00:41:54.767 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_public_type": "", 2024-02-23T00:41:54.767 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_service_mode": "crc secure", 2024-02-23T00:41:54.767 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_tcp_listen_backlog": "512", 2024-02-23T00:41:54.767 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_tcp_nodelay": "true", 2024-02-23T00:41:54.767 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-02-23T00:41:54.767 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_tcp_rcvbuf": "0", 2024-02-23T00:41:54.767 INFO:teuthology.orchestra.run.smithi047.stdout: "ms_type": "async+posix", 2024-02-23T00:41:54.767 INFO:teuthology.orchestra.run.smithi047.stdout: "no_config_file": "false", 2024-02-23T00:41:54.767 INFO:teuthology.orchestra.run.smithi047.stdout: "objecter_completion_locks_per_session": "32", 2024-02-23T00:41:54.767 INFO:teuthology.orchestra.run.smithi047.stdout: "objecter_debug_inject_relock_delay": "false", 2024-02-23T00:41:54.767 INFO:teuthology.orchestra.run.smithi047.stdout: "objecter_inflight_op_bytes": "104857600", 2024-02-23T00:41:54.768 INFO:teuthology.orchestra.run.smithi047.stdout: "objecter_inflight_ops": "1024", 2024-02-23T00:41:54.768 INFO:teuthology.orchestra.run.smithi047.stdout: "objecter_inject_no_watch_ping": "false", 2024-02-23T00:41:54.768 INFO:teuthology.orchestra.run.smithi047.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-02-23T00:41:54.768 INFO:teuthology.orchestra.run.smithi047.stdout: "objecter_tick_interval": "5.000000", 2024-02-23T00:41:54.768 INFO:teuthology.orchestra.run.smithi047.stdout: "objecter_timeout": "10.000000", 2024-02-23T00:41:54.768 INFO:teuthology.orchestra.run.smithi047.stdout: "objectstore_blackhole": "false", 2024-02-23T00:41:54.768 INFO:teuthology.orchestra.run.smithi047.stdout: "openssl_engine_opts": "", 2024-02-23T00:41:54.768 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_agent_delay_time": "5.000000", 2024-02-23T00:41:54.768 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_agent_hist_halflife": "1000", 2024-02-23T00:41:54.768 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_agent_max_low_ops": "2", 2024-02-23T00:41:54.768 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_agent_max_ops": "4", 2024-02-23T00:41:54.768 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-02-23T00:41:54.769 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_agent_quantize_effort": "0.100000", 2024-02-23T00:41:54.769 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_agent_slop": "0.020000", 2024-02-23T00:41:54.769 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-02-23T00:41:54.769 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_allow_recovery_below_min_size": "true", 2024-02-23T00:41:54.769 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_asio_thread_count": "2", 2024-02-23T00:41:54.769 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_async_recovery_min_cost": "100", 2024-02-23T00:41:54.769 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_auto_mark_unfound_lost": "false", 2024-02-23T00:41:54.769 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_backfill_retry_interval": "30.000000", 2024-02-23T00:41:54.769 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_backfill_scan_max": "512", 2024-02-23T00:41:54.769 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_backfill_scan_min": "64", 2024-02-23T00:41:54.769 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_backoff_on_degraded": "false", 2024-02-23T00:41:54.769 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_backoff_on_peering": "false", 2024-02-23T00:41:54.770 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_backoff_on_unfound": "true", 2024-02-23T00:41:54.770 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_beacon_report_interval": "300", 2024-02-23T00:41:54.770 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_bench_duration": "30", 2024-02-23T00:41:54.770 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-02-23T00:41:54.770 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_bench_max_block_size": "67108864", 2024-02-23T00:41:54.770 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_bench_small_size_max_iops": "100", 2024-02-23T00:41:54.770 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_blkin_trace_all": "false", 2024-02-23T00:41:54.770 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-02-23T00:41:54.770 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-02-23T00:41:54.770 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_check_for_log_corruption": "false", 2024-02-23T00:41:54.770 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-02-23T00:41:54.770 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-02-23T00:41:54.771 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-02-23T00:41:54.771 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-02-23T00:41:54.771 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_class_update_on_start": "true", 2024-02-23T00:41:54.771 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_client_message_cap": "256", 2024-02-23T00:41:54.771 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_client_message_size_cap": "524288000", 2024-02-23T00:41:54.771 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_client_op_priority": "63", 2024-02-23T00:41:54.771 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_client_watch_timeout": "30", 2024-02-23T00:41:54.771 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_command_max_records": "256", 2024-02-23T00:41:54.771 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_command_thread_suicide_timeout": "900", 2024-02-23T00:41:54.771 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_command_thread_timeout": "600", 2024-02-23T00:41:54.771 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_compact_on_start": "false", 2024-02-23T00:41:54.771 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-02-23T00:41:54.772 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_crush_chooseleaf_type": "1", 2024-02-23T00:41:54.772 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_crush_initial_weight": "-1.000000", 2024-02-23T00:41:54.772 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_crush_update_on_start": "true", 2024-02-23T00:41:54.772 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_crush_update_weight_set": "true", 2024-02-23T00:41:54.772 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-02-23T00:41:54.772 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-02-23T00:41:54.772 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-02-23T00:41:54.772 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_debug_drop_ping_duration": "0", 2024-02-23T00:41:54.772 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-02-23T00:41:54.772 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_debug_feed_pullee": "-1", 2024-02-23T00:41:54.772 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-02-23T00:41:54.772 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-02-23T00:41:54.773 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-02-23T00:41:54.773 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_debug_misdirected_ops": "false", 2024-02-23T00:41:54.773 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_debug_no_acting_change": "false", 2024-02-23T00:41:54.773 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_debug_no_purge_strays": "false", 2024-02-23T00:41:54.773 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_debug_op_order": "false", 2024-02-23T00:41:54.773 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_debug_pg_log_writeout": "false", 2024-02-23T00:41:54.773 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_debug_pretend_recovery_active": "false", 2024-02-23T00:41:54.773 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-02-23T00:41:54.773 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-02-23T00:41:54.773 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_debug_shutdown": "false", 2024-02-23T00:41:54.773 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-02-23T00:41:54.774 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-02-23T00:41:54.774 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_debug_verify_cached_snaps": "false", 2024-02-23T00:41:54.774 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_debug_verify_missing_on_start": "false", 2024-02-23T00:41:54.774 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_debug_verify_snaps": "false", 2024-02-23T00:41:54.774 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-02-23T00:41:54.774 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-02-23T00:41:54.774 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_deep_scrub_keys": "1024", 2024-02-23T00:41:54.774 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-02-23T00:41:54.774 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-02-23T00:41:54.774 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-02-23T00:41:54.774 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_deep_scrub_stride": "524288", 2024-02-23T00:41:54.775 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-02-23T00:41:54.775 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_default_data_pool_replay_window": "45", 2024-02-23T00:41:54.775 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_default_notify_timeout": "30", 2024-02-23T00:41:54.775 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_delete_sleep": "0.000000", 2024-02-23T00:41:54.775 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-02-23T00:41:54.775 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-02-23T00:41:54.775 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-02-23T00:41:54.775 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_discard_disconnected_ops": "true", 2024-02-23T00:41:54.775 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_enable_op_tracker": "true", 2024-02-23T00:41:54.775 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_erasure_code_plugins": "jerasure lrc isa", 2024-02-23T00:41:54.775 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-02-23T00:41:54.775 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-02-23T00:41:54.776 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_fast_info": "true", 2024-02-23T00:41:54.776 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_fast_shutdown": "true", 2024-02-23T00:41:54.776 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-02-23T00:41:54.776 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-02-23T00:41:54.776 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-02-23T00:41:54.776 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-02-23T00:41:54.776 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_function_tracing": "false", 2024-02-23T00:41:54.776 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_heartbeat_grace": "20", 2024-02-23T00:41:54.776 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_heartbeat_interval": "6", 2024-02-23T00:41:54.776 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-02-23T00:41:54.776 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_heartbeat_min_peers": "10", 2024-02-23T00:41:54.776 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_heartbeat_min_size": "2000", 2024-02-23T00:41:54.777 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_heartbeat_stale": "600", 2024-02-23T00:41:54.777 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-02-23T00:41:54.777 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_hit_set_max_size": "100000", 2024-02-23T00:41:54.777 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_hit_set_min_size": "1000", 2024-02-23T00:41:54.777 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-02-23T00:41:54.777 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-02-23T00:41:54.777 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-02-23T00:41:54.777 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-02-23T00:41:54.777 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-02-23T00:41:54.777 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_journal_flush_on_shutdown": "true", 2024-02-23T00:41:54.777 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_journal_size": "5120", 2024-02-23T00:41:54.777 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_kill_backfill_at": "0", 2024-02-23T00:41:54.778 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_loop_before_reset_tphandle": "64", 2024-02-23T00:41:54.778 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_map_cache_size": "50", 2024-02-23T00:41:54.778 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_map_dedup": "true", 2024-02-23T00:41:54.778 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_map_message_max": "40", 2024-02-23T00:41:54.778 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_map_message_max_bytes": "10485760", 2024-02-23T00:41:54.778 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_map_share_max_epochs": "40", 2024-02-23T00:41:54.778 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_max_attr_name_len": "100", 2024-02-23T00:41:54.778 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_max_attr_size": "0", 2024-02-23T00:41:54.778 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_max_backfills": "1", 2024-02-23T00:41:54.778 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_max_markdown_count": "5", 2024-02-23T00:41:54.778 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_max_markdown_period": "600", 2024-02-23T00:41:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_max_object_name_len": "2048", 2024-02-23T00:41:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_max_object_namespace_len": "256", 2024-02-23T00:41:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_max_object_size": "134217728", 2024-02-23T00:41:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-02-23T00:41:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_max_omap_entries_per_request": "1024", 2024-02-23T00:41:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_max_pg_blocked_by": "16", 2024-02-23T00:41:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_max_pg_log_entries": "10000", 2024-02-23T00:41:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-02-23T00:41:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_max_pgls": "1024", 2024-02-23T00:41:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_max_push_cost": "8388608", 2024-02-23T00:41:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_max_push_objects": "10", 2024-02-23T00:41:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_max_scrubs": "1", 2024-02-23T00:41:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-02-23T00:41:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_max_trimming_pgs": "2", 2024-02-23T00:41:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_max_write_op_reply_len": "64", 2024-02-23T00:41:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_max_write_size": "90", 2024-02-23T00:41:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mclock_cost_per_byte_usec": "0.000000", 2024-02-23T00:41:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mclock_cost_per_byte_usec_hdd": "5.200000", 2024-02-23T00:41:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mclock_cost_per_byte_usec_ssd": "0.011000", 2024-02-23T00:41:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mclock_cost_per_io_usec": "0.000000", 2024-02-23T00:41:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mclock_cost_per_io_usec_hdd": "25000.000000", 2024-02-23T00:41:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mclock_cost_per_io_usec_ssd": "50.000000", 2024-02-23T00:41:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-02-23T00:41:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-02-23T00:41:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-02-23T00:41:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mclock_profile": "high_client_ops", 2024-02-23T00:41:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-02-23T00:41:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mclock_scheduler_background_best_effort_lim": "999999", 2024-02-23T00:41:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mclock_scheduler_background_best_effort_res": "1", 2024-02-23T00:41:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-02-23T00:41:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mclock_scheduler_background_recovery_lim": "999999", 2024-02-23T00:41:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mclock_scheduler_background_recovery_res": "1", 2024-02-23T00:41:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-02-23T00:41:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mclock_scheduler_client_lim": "999999", 2024-02-23T00:41:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mclock_scheduler_client_res": "1", 2024-02-23T00:41:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-02-23T00:41:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mclock_skip_benchmark": "false", 2024-02-23T00:41:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_memory_base": "805306368", 2024-02-23T00:41:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_memory_cache_min": "134217728", 2024-02-23T00:41:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-02-23T00:41:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-02-23T00:41:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_memory_target": "4294967296", 2024-02-23T00:41:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_memory_target_autotune": "false", 2024-02-23T00:41:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-02-23T00:41:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_min_pg_log_entries": "250", 2024-02-23T00:41:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_min_recovery_priority": "0", 2024-02-23T00:41:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mon_heartbeat_interval": "30", 2024-02-23T00:41:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-02-23T00:41:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mon_report_interval": "5", 2024-02-23T00:41:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mon_report_max_in_flight": "2", 2024-02-23T00:41:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-02-23T00:41:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_num_cache_shards": "32", 2024-02-23T00:41:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_num_op_tracker_shard": "32", 2024-02-23T00:41:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_numa_auto_affinity": "true", 2024-02-23T00:41:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_numa_node": "-1", 2024-02-23T00:41:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_numa_prefer_iface": "true", 2024-02-23T00:41:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-02-23T00:41:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_objecter_finishers": "1", 2024-02-23T00:41:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_objectstore": "bluestore", 2024-02-23T00:41:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_objectstore_fuse": "false", 2024-02-23T00:41:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_objectstore_tracing": "false", 2024-02-23T00:41:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_op_complaint_time": "30.000000", 2024-02-23T00:41:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_op_history_duration": "600", 2024-02-23T00:41:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_op_history_size": "20", 2024-02-23T00:41:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_op_history_slow_op_size": "20", 2024-02-23T00:41:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-02-23T00:41:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_op_log_threshold": "5", 2024-02-23T00:41:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_op_num_shards": "0", 2024-02-23T00:41:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_op_num_shards_hdd": "5", 2024-02-23T00:41:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_op_num_shards_ssd": "8", 2024-02-23T00:41:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_op_num_threads_per_shard": "0", 2024-02-23T00:41:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-02-23T00:41:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-02-23T00:41:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-02-23T00:41:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_op_pq_min_cost": "65536", 2024-02-23T00:41:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_op_queue": "wpq", 2024-02-23T00:41:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_op_queue_cut_off": "high", 2024-02-23T00:41:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_op_thread_suicide_timeout": "150", 2024-02-23T00:41:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_op_thread_timeout": "15", 2024-02-23T00:41:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_open_classes_on_start": "true", 2024-02-23T00:41:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_os_flags": "0", 2024-02-23T00:41:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_peering_op_priority": "255", 2024-02-23T00:41:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pg_delete_cost": "1048576", 2024-02-23T00:41:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pg_delete_priority": "5", 2024-02-23T00:41:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-02-23T00:41:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-02-23T00:41:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pg_log_dups_tracked": "3000", 2024-02-23T00:41:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pg_log_trim_max": "10000", 2024-02-23T00:41:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pg_log_trim_min": "100", 2024-02-23T00:41:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-02-23T00:41:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pg_object_context_cache_count": "64", 2024-02-23T00:41:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pg_stat_report_interval_max": "500", 2024-02-23T00:41:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-02-23T00:41:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-02-23T00:41:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-02-23T00:41:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-02-23T00:41:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-02-23T00:41:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-02-23T00:41:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pool_default_crush_rule": "-1", 2024-02-23T00:41:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pool_default_ec_fast_read": "false", 2024-02-23T00:41:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd", 2024-02-23T00:41:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pool_default_flag_bulk": "false", 2024-02-23T00:41:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-02-23T00:41:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pool_default_flag_nodelete": "false", 2024-02-23T00:41:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-02-23T00:41:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-02-23T00:41:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pool_default_flags": "0", 2024-02-23T00:41:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-02-23T00:41:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pool_default_min_size": "0", 2024-02-23T00:41:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-02-23T00:41:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pool_default_pg_num": "32", 2024-02-23T00:41:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pool_default_pgp_num": "0", 2024-02-23T00:41:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-02-23T00:41:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pool_default_size": "3", 2024-02-23T00:41:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pool_default_type": "replicated", 2024-02-23T00:41:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-02-23T00:41:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pool_use_gmt_hitset": "true", 2024-02-23T00:41:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_push_per_object_cost": "1000", 2024-02-23T00:41:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_read_ec_check_for_errors": "false", 2024-02-23T00:41:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_recover_clone_overlap": "true", 2024-02-23T00:41:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_recover_clone_overlap_limit": "10", 2024-02-23T00:41:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_recovery_cost": "20971520", 2024-02-23T00:41:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_recovery_delay_start": "0.000000", 2024-02-23T00:41:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_recovery_max_active": "0", 2024-02-23T00:41:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_recovery_max_active_hdd": "3", 2024-02-23T00:41:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_recovery_max_active_ssd": "10", 2024-02-23T00:41:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_recovery_max_chunk": "8388608", 2024-02-23T00:41:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-02-23T00:41:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_recovery_max_single_start": "1", 2024-02-23T00:41:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_recovery_op_priority": "3", 2024-02-23T00:41:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_recovery_op_warn_multiple": "16", 2024-02-23T00:41:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_recovery_priority": "5", 2024-02-23T00:41:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_recovery_retry_interval": "30.000000", 2024-02-23T00:41:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_recovery_sleep": "0.000000", 2024-02-23T00:41:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-02-23T00:41:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-02-23T00:41:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-02-23T00:41:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_repair_during_recovery": "false", 2024-02-23T00:41:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_requested_scrub_priority": "120", 2024-02-23T00:41:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-02-23T00:41:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_rollback_to_cluster_snap": "", 2024-02-23T00:41:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_scrub_auto_repair": "false", 2024-02-23T00:41:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-02-23T00:41:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-02-23T00:41:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_scrub_begin_hour": "0", 2024-02-23T00:41:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_scrub_begin_week_day": "0", 2024-02-23T00:41:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_scrub_chunk_max": "25", 2024-02-23T00:41:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_scrub_chunk_min": "5", 2024-02-23T00:41:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_scrub_cost": "52428800", 2024-02-23T00:41:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_scrub_during_recovery": "false", 2024-02-23T00:41:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_scrub_end_hour": "0", 2024-02-23T00:41:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_scrub_end_week_day": "0", 2024-02-23T00:41:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-02-23T00:41:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-02-23T00:41:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_scrub_invalid_stats": "true", 2024-02-23T00:41:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_scrub_load_threshold": "0.500000", 2024-02-23T00:41:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_scrub_max_interval": "604800.000000", 2024-02-23T00:41:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_scrub_max_preemptions": "5", 2024-02-23T00:41:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_scrub_min_interval": "86400.000000", 2024-02-23T00:41:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_scrub_priority": "5", 2024-02-23T00:41:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_scrub_sleep": "0.000000", 2024-02-23T00:41:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_shutdown_pgref_assert": "false", 2024-02-23T00:41:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_skip_data_digest": "false", 2024-02-23T00:41:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_smart_report_timeout": "5", 2024-02-23T00:41:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_snap_trim_cost": "1048576", 2024-02-23T00:41:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_snap_trim_priority": "5", 2024-02-23T00:41:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_snap_trim_sleep": "0.000000", 2024-02-23T00:41:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-02-23T00:41:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-02-23T00:41:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-02-23T00:41:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-02-23T00:41:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_target_transaction_size": "30", 2024-02-23T00:41:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-02-23T00:41:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-02-23T00:41:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-02-23T00:41:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-02-23T00:41:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-02-23T00:41:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-02-23T00:41:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-02-23T00:41:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_tier_default_cache_mode": "writeback", 2024-02-23T00:41:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-02-23T00:41:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-02-23T00:41:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_tracing": "false", 2024-02-23T00:41:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_use_stale_snap": "false", 2024-02-23T00:41:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-02-23T00:41:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "osdc_blkin_trace_all": "false", 2024-02-23T00:41:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "paxos_kill_at": "0", 2024-02-23T00:41:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "paxos_max_join_drift": "10", 2024-02-23T00:41:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "paxos_min": "500", 2024-02-23T00:41:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "paxos_min_wait": "0.050000", 2024-02-23T00:41:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "paxos_propose_interval": "1.000000", 2024-02-23T00:41:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "paxos_service_trim_max": "500", 2024-02-23T00:41:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "paxos_service_trim_max_multiplier": "20", 2024-02-23T00:41:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "paxos_service_trim_min": "10", 2024-02-23T00:41:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "paxos_stash_full_interval": "25", 2024-02-23T00:41:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: "paxos_trim_max": "500", 2024-02-23T00:41:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: "paxos_trim_min": "250", 2024-02-23T00:41:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: "perf": "true", 2024-02-23T00:41:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: "pid_file": "", 2024-02-23T00:41:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-02-23T00:41:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-02-23T00:41:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "-", 2024-02-23T00:41:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrv": "", 2024-02-23T00:41:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: "public_bind_addr": "-", 2024-02-23T00:41:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: "public_network": "", 2024-02-23T00:41:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: "public_network_interface": "", 2024-02-23T00:41:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: "qat_compressor_enabled": "false", 2024-02-23T00:41:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: "rados_mon_op_timeout": "0", 2024-02-23T00:41:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: "rados_osd_op_timeout": "0", 2024-02-23T00:41:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: "rados_tracing": "false", 2024-02-23T00:41:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_atime_update_interval": "60", 2024-02-23T00:41:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-02-23T00:41:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_balance_parent_reads": "false", 2024-02-23T00:41:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_balance_snap_reads": "false", 2024-02-23T00:41:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_blkin_trace_all": "false", 2024-02-23T00:41:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_blocklist_expire_seconds": "0", 2024-02-23T00:41:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_blocklist_on_break_lock": "true", 2024-02-23T00:41:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_cache": "true", 2024-02-23T00:41:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_cache_block_writes_upfront": "false", 2024-02-23T00:41:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_cache_max_dirty": "25165824", 2024-02-23T00:41:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-02-23T00:41:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_cache_max_dirty_object": "0", 2024-02-23T00:41:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_cache_policy": "writearound", 2024-02-23T00:41:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_cache_size": "33554432", 2024-02-23T00:41:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_cache_target_dirty": "16777216", 2024-02-23T00:41:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-02-23T00:41:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_clone_copy_on_read": "false", 2024-02-23T00:41:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_compression_hint": "none", 2024-02-23T00:41:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_concurrent_management_ops": "10", 2024-02-23T00:41:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-02-23T00:41:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_default_clone_format": "auto", 2024-02-23T00:41:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_default_data_pool": "", 2024-02-23T00:41:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_default_features": "61", 2024-02-23T00:41:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_default_format": "2", 2024-02-23T00:41:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_default_map_options": "", 2024-02-23T00:41:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_default_order": "22", 2024-02-23T00:41:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_default_pool": "rbd", 2024-02-23T00:41:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-02-23T00:41:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_default_stripe_count": "0", 2024-02-23T00:41:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_default_stripe_unit": "0", 2024-02-23T00:41:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_disable_zero_copy_writes": "true", 2024-02-23T00:41:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_discard_granularity_bytes": "65536", 2024-02-23T00:41:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-02-23T00:41:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_enable_alloc_hint": "true", 2024-02-23T00:41:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-02-23T00:41:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_io_scheduler": "simple", 2024-02-23T00:41:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-02-23T00:41:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_journal_commit_age": "5.000000", 2024-02-23T00:41:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-02-23T00:41:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-02-23T00:41:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-02-23T00:41:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-02-23T00:41:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_journal_object_flush_interval": "0", 2024-02-23T00:41:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-02-23T00:41:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-02-23T00:41:54.800 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_journal_order": "24", 2024-02-23T00:41:54.800 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_journal_pool": "", 2024-02-23T00:41:54.800 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_journal_splay_width": "4", 2024-02-23T00:41:54.800 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_localize_parent_reads": "false", 2024-02-23T00:41:54.800 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_localize_snap_reads": "false", 2024-02-23T00:41:54.800 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-02-23T00:41:54.800 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-02-23T00:41:54.800 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-02-23T00:41:54.800 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-02-23T00:41:54.800 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-02-23T00:41:54.800 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-02-23T00:41:54.800 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_mirror_image_policy_type": "simple", 2024-02-23T00:41:54.801 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-02-23T00:41:54.801 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-02-23T00:41:54.801 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-02-23T00:41:54.801 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-02-23T00:41:54.801 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-02-23T00:41:54.801 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-02-23T00:41:54.801 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-02-23T00:41:54.801 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_mirror_memory_autotune": "true", 2024-02-23T00:41:54.801 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_mirror_memory_base": "805306368", 2024-02-23T00:41:54.801 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-02-23T00:41:54.801 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-02-23T00:41:54.801 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-02-23T00:41:54.801 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-02-23T00:41:54.802 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_mirror_memory_target": "4294967296", 2024-02-23T00:41:54.802 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-02-23T00:41:54.802 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-02-23T00:41:54.802 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-02-23T00:41:54.802 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_mirroring_delete_delay": "0", 2024-02-23T00:41:54.802 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-02-23T00:41:54.802 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_mirroring_replay_delay": "0", 2024-02-23T00:41:54.802 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-02-23T00:41:54.802 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-02-23T00:41:54.802 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_move_to_trash_on_remove": "false", 2024-02-23T00:41:54.802 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-02-23T00:41:54.802 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_mtime_update_interval": "60", 2024-02-23T00:41:54.803 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_non_blocking_aio": "true", 2024-02-23T00:41:54.803 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_op_thread_timeout": "60", 2024-02-23T00:41:54.803 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_op_threads": "1", 2024-02-23T00:41:54.803 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_parent_cache_enabled": "false", 2024-02-23T00:41:54.803 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_persistent_cache_mode": "disabled", 2024-02-23T00:41:54.803 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_persistent_cache_path": "/tmp", 2024-02-23T00:41:54.803 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_persistent_cache_size": "1073741824", 2024-02-23T00:41:54.803 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_plugins": "", 2024-02-23T00:41:54.803 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_qos_bps_burst": "0", 2024-02-23T00:41:54.803 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-02-23T00:41:54.803 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_qos_bps_limit": "0", 2024-02-23T00:41:54.803 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_qos_iops_burst": "0", 2024-02-23T00:41:54.803 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-02-23T00:41:54.804 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_qos_iops_limit": "0", 2024-02-23T00:41:54.804 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_qos_read_bps_burst": "0", 2024-02-23T00:41:54.804 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-02-23T00:41:54.804 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_qos_read_bps_limit": "0", 2024-02-23T00:41:54.804 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_qos_read_iops_burst": "0", 2024-02-23T00:41:54.804 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-02-23T00:41:54.804 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_qos_read_iops_limit": "0", 2024-02-23T00:41:54.804 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_qos_schedule_tick_min": "50", 2024-02-23T00:41:54.804 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_qos_write_bps_burst": "0", 2024-02-23T00:41:54.804 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-02-23T00:41:54.804 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_qos_write_bps_limit": "0", 2024-02-23T00:41:54.805 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_qos_write_iops_burst": "0", 2024-02-23T00:41:54.805 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-02-23T00:41:54.805 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_qos_write_iops_limit": "0", 2024-02-23T00:41:54.805 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_quiesce_notification_attempts": "10", 2024-02-23T00:41:54.805 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_read_from_replica_policy": "default", 2024-02-23T00:41:54.805 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-02-23T00:41:54.805 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_readahead_max_bytes": "524288", 2024-02-23T00:41:54.805 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_readahead_trigger_requests": "10", 2024-02-23T00:41:54.805 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_request_timed_out_seconds": "30", 2024-02-23T00:41:54.805 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_skip_partial_discard": "true", 2024-02-23T00:41:54.805 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-02-23T00:41:54.805 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_tracing": "false", 2024-02-23T00:41:54.806 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_validate_names": "true", 2024-02-23T00:41:54.806 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_validate_pool": "true", 2024-02-23T00:41:54.806 INFO:teuthology.orchestra.run.smithi047.stdout: "restapi_base_url": "", 2024-02-23T00:41:54.806 INFO:teuthology.orchestra.run.smithi047.stdout: "restapi_log_level": "", 2024-02-23T00:41:54.806 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_acl_grants_max_num": "100", 2024-02-23T00:41:54.806 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_admin_entry": "admin", 2024-02-23T00:41:54.806 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_barbican_url": "", 2024-02-23T00:41:54.806 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-02-23T00:41:54.806 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-02-23T00:41:54.806 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_bucket_index_max_aio": "128", 2024-02-23T00:41:54.806 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-02-23T00:41:54.806 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_bucket_quota_ttl": "600", 2024-02-23T00:41:54.807 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_cache_enabled": "true", 2024-02-23T00:41:54.807 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_cache_expiry_interval": "900", 2024-02-23T00:41:54.807 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_cache_lru_size": "10000", 2024-02-23T00:41:54.807 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_content_length_compat": "false", 2024-02-23T00:41:54.807 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_copy_obj_progress": "true", 2024-02-23T00:41:54.807 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-02-23T00:41:54.807 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_cors_rules_max_num": "100", 2024-02-23T00:41:54.807 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_cross_domain_policy": "", 2024-02-23T00:41:54.807 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_crypt_default_encryption_key": "", 2024-02-23T00:41:54.807 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_crypt_kmip_addr": "", 2024-02-23T00:41:54.807 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_crypt_kmip_ca_path": "", 2024-02-23T00:41:54.807 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_crypt_kmip_client_cert": "", 2024-02-23T00:41:54.807 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_crypt_kmip_client_key": "", 2024-02-23T00:41:54.808 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-02-23T00:41:54.808 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_crypt_kmip_password": "", 2024-02-23T00:41:54.808 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-02-23T00:41:54.808 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_crypt_kmip_username": "", 2024-02-23T00:41:54.808 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_crypt_require_ssl": "true", 2024-02-23T00:41:54.808 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-02-23T00:41:54.808 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-02-23T00:41:54.808 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_crypt_suppress_logs": "true", 2024-02-23T00:41:54.808 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_crypt_vault_addr": "", 2024-02-23T00:41:54.808 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_crypt_vault_auth": "token", 2024-02-23T00:41:54.808 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_crypt_vault_namespace": "", 2024-02-23T00:41:54.808 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_crypt_vault_prefix": "", 2024-02-23T00:41:54.809 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-02-23T00:41:54.809 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-02-23T00:41:54.809 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-02-23T00:41:54.809 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-02-23T00:41:54.809 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_crypt_vault_token_file": "", 2024-02-23T00:41:54.809 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-02-23T00:41:54.809 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_curl_buffersize": "524288", 2024-02-23T00:41:54.809 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_curl_low_speed_limit": "1024", 2024-02-23T00:41:54.809 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_curl_low_speed_time": "300", 2024-02-23T00:41:54.809 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-02-23T00:41:54.809 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-02-23T00:41:54.809 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_data_log_changes_size": "1000", 2024-02-23T00:41:54.809 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_data_log_num_shards": "128", 2024-02-23T00:41:54.810 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-02-23T00:41:54.810 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_data_log_window": "30", 2024-02-23T00:41:54.810 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_data_notify_interval_msec": "200", 2024-02-23T00:41:54.810 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-02-23T00:41:54.810 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-02-23T00:41:54.810 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_default_data_log_backing": "fifo", 2024-02-23T00:41:54.810 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-02-23T00:41:54.810 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_default_region_info_oid": "default.region", 2024-02-23T00:41:54.810 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-02-23T00:41:54.810 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-02-23T00:41:54.810 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_defer_to_bucket_acls": "", 2024-02-23T00:41:54.811 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-02-23T00:41:54.811 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-02-23T00:41:54.811 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-02-23T00:41:54.811 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-02-23T00:41:54.811 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-02-23T00:41:54.811 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-02-23T00:41:54.811 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-02-23T00:41:54.811 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-02-23T00:41:54.811 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_dmclock_data_res": "500.000000", 2024-02-23T00:41:54.811 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-02-23T00:41:54.811 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-02-23T00:41:54.811 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-02-23T00:41:54.811 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-02-23T00:41:54.812 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_dns_name": "", 2024-02-23T00:41:54.812 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_dns_s3website_name": "", 2024-02-23T00:41:54.812 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_dynamic_resharding": "true", 2024-02-23T00:41:54.812 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-02-23T00:41:54.812 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_enable_gc_threads": "true", 2024-02-23T00:41:54.812 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_enable_lc_threads": "true", 2024-02-23T00:41:54.812 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_enable_ops_log": "false", 2024-02-23T00:41:54.812 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_enable_quota_threads": "true", 2024-02-23T00:41:54.812 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_enable_static_website": "false", 2024-02-23T00:41:54.812 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_enable_usage_log": "false", 2024-02-23T00:41:54.812 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_enforce_swift_acls": "true", 2024-02-23T00:41:54.813 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_exit_timeout_secs": "120", 2024-02-23T00:41:54.813 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_expose_bucket": "false", 2024-02-23T00:41:54.813 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_extended_http_attrs": "", 2024-02-23T00:41:54.813 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_fcgi_socket_backlog": "1024", 2024-02-23T00:41:54.813 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-02-23T00:41:54.813 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_frontends": "beast port=7480", 2024-02-23T00:41:54.813 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_gc_max_concurrent_io": "10", 2024-02-23T00:41:54.813 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_gc_max_deferred": "50", 2024-02-23T00:41:54.813 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-02-23T00:41:54.813 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_gc_max_objs": "32", 2024-02-23T00:41:54.813 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_gc_max_queue_size": "134213632", 2024-02-23T00:41:54.814 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_gc_max_trim_chunk": "16", 2024-02-23T00:41:54.814 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_gc_obj_min_wait": "7200", 2024-02-23T00:41:54.814 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_gc_processor_max_time": "3600", 2024-02-23T00:41:54.814 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_gc_processor_period": "3600", 2024-02-23T00:41:54.814 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-02-23T00:41:54.814 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_get_obj_window_size": "16777216", 2024-02-23T00:41:54.814 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_healthcheck_disabling_path": "", 2024-02-23T00:41:54.814 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_host": "", 2024-02-23T00:41:54.814 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_ignore_get_invalid_range": "false", 2024-02-23T00:41:54.814 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_init_timeout": "300", 2024-02-23T00:41:54.814 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-02-23T00:41:54.815 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-02-23T00:41:54.815 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-02-23T00:41:54.815 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_keystone_admin_domain": "", 2024-02-23T00:41:54.815 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_keystone_admin_password": "", 2024-02-23T00:41:54.815 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_keystone_admin_password_path": "", 2024-02-23T00:41:54.815 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_keystone_admin_project": "", 2024-02-23T00:41:54.815 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_keystone_admin_tenant": "", 2024-02-23T00:41:54.815 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_keystone_admin_token": "", 2024-02-23T00:41:54.815 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_keystone_admin_token_path": "", 2024-02-23T00:41:54.815 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_keystone_admin_user": "", 2024-02-23T00:41:54.815 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_keystone_api_version": "2", 2024-02-23T00:41:54.815 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_keystone_barbican_domain": "", 2024-02-23T00:41:54.815 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_keystone_barbican_password": "", 2024-02-23T00:41:54.816 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_keystone_barbican_project": "", 2024-02-23T00:41:54.816 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_keystone_barbican_tenant": "", 2024-02-23T00:41:54.816 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_keystone_barbican_user": "", 2024-02-23T00:41:54.816 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_keystone_implicit_tenants": "false", 2024-02-23T00:41:54.816 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_keystone_token_cache_size": "10000", 2024-02-23T00:41:54.816 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_keystone_url": "", 2024-02-23T00:41:54.816 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_keystone_verify_ssl": "true", 2024-02-23T00:41:54.816 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_lc_debug_interval": "-1", 2024-02-23T00:41:54.816 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_lc_lock_max_time": "90", 2024-02-23T00:41:54.816 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_lc_max_objs": "32", 2024-02-23T00:41:54.816 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_lc_max_rules": "1000", 2024-02-23T00:41:54.816 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_lc_max_worker": "3", 2024-02-23T00:41:54.817 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_lc_max_wp_worker": "3", 2024-02-23T00:41:54.817 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_lc_thread_delay": "0", 2024-02-23T00:41:54.817 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-02-23T00:41:54.817 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_ldap_dnattr": "uid", 2024-02-23T00:41:54.817 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-02-23T00:41:54.817 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_ldap_searchfilter": "", 2024-02-23T00:41:54.817 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-02-23T00:41:54.817 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_ldap_uri": "ldaps://", 2024-02-23T00:41:54.817 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-02-23T00:41:54.817 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-02-23T00:41:54.817 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-02-23T00:41:54.817 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_log_http_headers": "", 2024-02-23T00:41:54.818 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_log_nonexistent_bucket": "false", 2024-02-23T00:41:54.818 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-02-23T00:41:54.818 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_log_object_name_utc": "false", 2024-02-23T00:41:54.818 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-02-23T00:41:54.818 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_max_attr_name_len": "0", 2024-02-23T00:41:54.818 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_max_attr_size": "0", 2024-02-23T00:41:54.818 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_max_attrs_num_in_req": "0", 2024-02-23T00:41:54.818 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_max_chunk_size": "4194304", 2024-02-23T00:41:54.818 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_max_concurrent_requests": "1024", 2024-02-23T00:41:54.818 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_max_dynamic_shards": "1999", 2024-02-23T00:41:54.818 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_max_listing_results": "1000", 2024-02-23T00:41:54.818 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_max_notify_retries": "10", 2024-02-23T00:41:54.819 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_max_objs_per_shard": "100000", 2024-02-23T00:41:54.819 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_max_put_param_size": "1048576", 2024-02-23T00:41:54.819 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_max_put_size": "5368709120", 2024-02-23T00:41:54.819 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_max_slo_entries": "1000", 2024-02-23T00:41:54.819 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_md_log_max_shards": "64", 2024-02-23T00:41:54.819 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_md_notify_interval_msec": "200", 2024-02-23T00:41:54.819 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-02-23T00:41:54.819 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_mp_lock_max_time": "600", 2024-02-23T00:41:54.819 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-02-23T00:41:54.819 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_multipart_min_part_size": "5242880", 2024-02-23T00:41:54.819 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-02-23T00:41:54.819 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-02-23T00:41:54.820 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_nfs_fhcache_size": "2017", 2024-02-23T00:41:54.820 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-02-23T00:41:54.820 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_nfs_lru_lanes": "5", 2024-02-23T00:41:54.820 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_nfs_max_gc": "300", 2024-02-23T00:41:54.820 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-02-23T00:41:54.820 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_nfs_run_gc_threads": "false", 2024-02-23T00:41:54.820 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_nfs_run_lc_threads": "false", 2024-02-23T00:41:54.820 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_nfs_run_quota_threads": "false", 2024-02-23T00:41:54.820 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_nfs_run_sync_thread": "false", 2024-02-23T00:41:54.820 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-02-23T00:41:54.820 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-02-23T00:41:54.820 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_num_async_rados_threads": "32", 2024-02-23T00:41:54.821 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_num_control_oids": "8", 2024-02-23T00:41:54.821 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_num_rados_handles": "1", 2024-02-23T00:41:54.821 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_numa_node": "-1", 2024-02-23T00:41:54.821 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_obj_stripe_size": "4194304", 2024-02-23T00:41:54.821 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-02-23T00:41:54.821 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_objexp_chunk_size": "100", 2024-02-23T00:41:54.821 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_objexp_gc_interval": "600", 2024-02-23T00:41:54.821 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_objexp_hints_num_shards": "127", 2024-02-23T00:41:54.821 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-02-23T00:41:54.821 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-02-23T00:41:54.821 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_op_thread_timeout": "600", 2024-02-23T00:41:54.821 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_op_tracing": "false", 2024-02-23T00:41:54.821 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_opa_token": "", 2024-02-23T00:41:54.822 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_opa_url": "", 2024-02-23T00:41:54.822 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_opa_verify_ssl": "true", 2024-02-23T00:41:54.822 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-02-23T00:41:54.822 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_ops_log_file_path": "", 2024-02-23T00:41:54.822 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_ops_log_rados": "true", 2024-02-23T00:41:54.822 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_ops_log_socket_path": "", 2024-02-23T00:41:54.822 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-02-23T00:41:54.822 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-02-23T00:41:54.822 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_period_push_interval": "2.000000", 2024-02-23T00:41:54.822 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_period_push_interval_max": "30.000000", 2024-02-23T00:41:54.822 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_period_root_pool": ".rgw.root", 2024-02-23T00:41:54.822 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_port": "", 2024-02-23T00:41:54.822 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_print_continue": "true", 2024-02-23T00:41:54.823 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_print_prohibited_content_length": "false", 2024-02-23T00:41:54.823 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-02-23T00:41:54.823 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-02-23T00:41:54.823 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-02-23T00:41:54.823 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-02-23T00:41:54.823 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_rados_tracing": "false", 2024-02-23T00:41:54.823 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_realm": "", 2024-02-23T00:41:54.823 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-02-23T00:41:54.823 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_region": "", 2024-02-23T00:41:54.823 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_region_root_pool": ".rgw.root", 2024-02-23T00:41:54.823 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_relaxed_region_enforcement": "false", 2024-02-23T00:41:54.823 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-02-23T00:41:54.824 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-02-23T00:41:54.824 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_request_uri": "", 2024-02-23T00:41:54.824 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_reshard_batch_size": "64", 2024-02-23T00:41:54.824 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-02-23T00:41:54.824 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_reshard_max_aio": "128", 2024-02-23T00:41:54.824 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_reshard_num_logs": "16", 2024-02-23T00:41:54.824 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_reshard_thread_interval": "600", 2024-02-23T00:41:54.824 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_resolve_cname": "false", 2024-02-23T00:41:54.824 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_rest_getusage_op_compat": "false", 2024-02-23T00:41:54.824 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_run_sync_thread": "true", 2024-02-23T00:41:54.824 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-02-23T00:41:54.824 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_s3_auth_use_keystone": "false", 2024-02-23T00:41:54.825 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_s3_auth_use_ldap": "false", 2024-02-23T00:41:54.825 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_s3_auth_use_rados": "true", 2024-02-23T00:41:54.825 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_s3_auth_use_sts": "false", 2024-02-23T00:41:54.825 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_s3_success_create_obj_status": "0", 2024-02-23T00:41:54.825 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-02-23T00:41:54.825 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_scheduler_type": "throttler", 2024-02-23T00:41:54.825 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_script_uri": "", 2024-02-23T00:41:54.825 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_service_provider_name": "", 2024-02-23T00:41:54.825 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-02-23T00:41:54.825 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_socket_path": "", 2024-02-23T00:41:54.825 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_sts_client_id": "", 2024-02-23T00:41:54.825 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_sts_client_secret": "", 2024-02-23T00:41:54.826 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_sts_entry": "sts", 2024-02-23T00:41:54.826 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_sts_key": "sts", 2024-02-23T00:41:54.826 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_sts_max_session_duration": "43200", 2024-02-23T00:41:54.826 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_sts_min_session_duration": "900", 2024-02-23T00:41:54.826 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_sts_token_introspection_url": "", 2024-02-23T00:41:54.826 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_swift_account_in_url": "false", 2024-02-23T00:41:54.826 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_swift_auth_entry": "auth", 2024-02-23T00:41:54.826 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_swift_auth_url": "", 2024-02-23T00:41:54.826 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_swift_custom_header": "", 2024-02-23T00:41:54.826 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_swift_enforce_content_length": "false", 2024-02-23T00:41:54.826 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_swift_need_stats": "true", 2024-02-23T00:41:54.826 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_swift_tenant_name": "", 2024-02-23T00:41:54.827 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_swift_token_expiration": "86400", 2024-02-23T00:41:54.827 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_swift_url": "", 2024-02-23T00:41:54.827 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_swift_url_prefix": "swift", 2024-02-23T00:41:54.827 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_swift_versioning_enabled": "false", 2024-02-23T00:41:54.827 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-02-23T00:41:54.827 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_sync_lease_period": "120", 2024-02-23T00:41:54.827 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-02-23T00:41:54.827 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_sync_log_trim_interval": "1200", 2024-02-23T00:41:54.827 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-02-23T00:41:54.827 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-02-23T00:41:54.827 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-02-23T00:41:54.827 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_sync_obj_etag_verify": "false", 2024-02-23T00:41:54.828 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_sync_trace_history_size": "4096", 2024-02-23T00:41:54.828 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-02-23T00:41:54.828 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-02-23T00:41:54.828 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_thread_pool_size": "512", 2024-02-23T00:41:54.828 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_torrent_comment": "", 2024-02-23T00:41:54.828 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_torrent_createby": "", 2024-02-23T00:41:54.828 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_torrent_encoding": "", 2024-02-23T00:41:54.828 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_torrent_flag": "false", 2024-02-23T00:41:54.828 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_torrent_origin": "", 2024-02-23T00:41:54.828 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_torrent_sha_unit": "524288", 2024-02-23T00:41:54.828 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_torrent_tracker": "", 2024-02-23T00:41:54.828 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_trust_forwarded_https": "false", 2024-02-23T00:41:54.829 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-02-23T00:41:54.829 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_usage_log_tick_interval": "30", 2024-02-23T00:41:54.829 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_usage_max_shards": "32", 2024-02-23T00:41:54.829 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_usage_max_user_shards": "1", 2024-02-23T00:41:54.829 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_use_opa_authz": "false", 2024-02-23T00:41:54.829 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-02-23T00:41:54.829 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_user_default_quota_max_size": "-1", 2024-02-23T00:41:54.829 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_user_max_buckets": "1000", 2024-02-23T00:41:54.829 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-02-23T00:41:54.829 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-02-23T00:41:54.829 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_user_quota_sync_interval": "86400", 2024-02-23T00:41:54.829 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-02-23T00:41:54.830 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_user_unique_email": "true", 2024-02-23T00:41:54.830 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_verify_ssl": "true", 2024-02-23T00:41:54.830 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_website_routing_rules_max_num": "50", 2024-02-23T00:41:54.830 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_zone": "", 2024-02-23T00:41:54.830 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-02-23T00:41:54.830 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_zonegroup": "", 2024-02-23T00:41:54.830 INFO:teuthology.orchestra.run.smithi047.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-02-23T00:41:54.830 INFO:teuthology.orchestra.run.smithi047.stdout: "rocksdb_block_size": "4096", 2024-02-23T00:41:54.830 INFO:teuthology.orchestra.run.smithi047.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-02-23T00:41:54.830 INFO:teuthology.orchestra.run.smithi047.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-02-23T00:41:54.830 INFO:teuthology.orchestra.run.smithi047.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-02-23T00:41:54.831 INFO:teuthology.orchestra.run.smithi047.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-02-23T00:41:54.831 INFO:teuthology.orchestra.run.smithi047.stdout: "rocksdb_cache_shard_bits": "4", 2024-02-23T00:41:54.831 INFO:teuthology.orchestra.run.smithi047.stdout: "rocksdb_cache_size": "536870912", 2024-02-23T00:41:54.831 INFO:teuthology.orchestra.run.smithi047.stdout: "rocksdb_cache_type": "binned_lru", 2024-02-23T00:41:54.831 INFO:teuthology.orchestra.run.smithi047.stdout: "rocksdb_cf_compact_on_deletion": "false", 2024-02-23T00:41:54.831 INFO:teuthology.orchestra.run.smithi047.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-02-23T00:41:54.831 INFO:teuthology.orchestra.run.smithi047.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-02-23T00:41:54.831 INFO:teuthology.orchestra.run.smithi047.stdout: "rocksdb_collect_compaction_stats": "false", 2024-02-23T00:41:54.831 INFO:teuthology.orchestra.run.smithi047.stdout: "rocksdb_collect_extended_stats": "false", 2024-02-23T00:41:54.831 INFO:teuthology.orchestra.run.smithi047.stdout: "rocksdb_collect_memory_stats": "false", 2024-02-23T00:41:54.831 INFO:teuthology.orchestra.run.smithi047.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-02-23T00:41:54.831 INFO:teuthology.orchestra.run.smithi047.stdout: "rocksdb_index_type": "binary_search", 2024-02-23T00:41:54.832 INFO:teuthology.orchestra.run.smithi047.stdout: "rocksdb_log_to_ceph_log": "true", 2024-02-23T00:41:54.832 INFO:teuthology.orchestra.run.smithi047.stdout: "rocksdb_metadata_block_size": "4096", 2024-02-23T00:41:54.832 INFO:teuthology.orchestra.run.smithi047.stdout: "rocksdb_partition_filters": "false", 2024-02-23T00:41:54.832 INFO:teuthology.orchestra.run.smithi047.stdout: "rocksdb_perf": "false", 2024-02-23T00:41:54.832 INFO:teuthology.orchestra.run.smithi047.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-02-23T00:41:54.832 INFO:teuthology.orchestra.run.smithi047.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-02-23T00:41:54.832 INFO:teuthology.orchestra.run.smithi047.stdout: "rotating_keys_renewal_timeout": "10", 2024-02-23T00:41:54.832 INFO:teuthology.orchestra.run.smithi047.stdout: "run_dir": "/var/run/ceph", 2024-02-23T00:41:54.832 INFO:teuthology.orchestra.run.smithi047.stdout: "setgroup": "ceph", 2024-02-23T00:41:54.832 INFO:teuthology.orchestra.run.smithi047.stdout: "setuser": "ceph", 2024-02-23T00:41:54.832 INFO:teuthology.orchestra.run.smithi047.stdout: "setuser_match_path": "", 2024-02-23T00:41:54.833 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_misplaced_ratio": "0.050000", 2024-02-23T00:41:54.833 INFO:teuthology.orchestra.run.smithi047.stdout: "thp": "false", 2024-02-23T00:41:54.833 INFO:teuthology.orchestra.run.smithi047.stdout: "threadpool_default_timeout": "60", 2024-02-23T00:41:54.833 INFO:teuthology.orchestra.run.smithi047.stdout: "threadpool_empty_queue_max_wait": "2", 2024-02-23T00:41:54.833 INFO:teuthology.orchestra.run.smithi047.stdout: "throttler_perf_counter": "true" 2024-02-23T00:41:54.833 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:41:55.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:53.689061+0000 osd.1 (osd.1) 55 : cluster [DBG] 5.c scrub starts 2024-02-23T00:41:55.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:53.692175+0000 osd.1 (osd.1) 56 : cluster [DBG] 5.c scrub ok 2024-02-23T00:41:55.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:53.689061+0000 osd.1 (osd.1) 55 : cluster [DBG] 5.c scrub starts 2024-02-23T00:41:55.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:53.692175+0000 osd.1 (osd.1) 56 : cluster [DBG] 5.c scrub ok 2024-02-23T00:41:55.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:53.689061+0000 osd.1 (osd.1) 55 : cluster [DBG] 5.c scrub starts 2024-02-23T00:41:55.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:53.692175+0000 osd.1 (osd.1) 56 : cluster [DBG] 5.c scrub ok 2024-02-23T00:41:55.682 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-02-23T00:41:55.968 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:41:56.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:54.080535+0000 osd.3 (osd.3) 61 : cluster [DBG] 4.1d scrub starts 2024-02-23T00:41:56.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:54.081972+0000 osd.3 (osd.3) 62 : cluster [DBG] 4.1d scrub ok 2024-02-23T00:41:56.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:54.302759+0000 osd.2 (osd.2) 43 : cluster [DBG] 2.c scrub starts 2024-02-23T00:41:56.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:54.303785+0000 osd.2 (osd.2) 44 : cluster [DBG] 2.c scrub ok 2024-02-23T00:41:56.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:55.155576+0000 mgr.y (mgr.14517) 108 : cluster [DBG] pgmap v92: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:41:56.237 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:54.080535+0000 osd.3 (osd.3) 61 : cluster [DBG] 4.1d scrub starts 2024-02-23T00:41:56.238 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:54.081972+0000 osd.3 (osd.3) 62 : cluster [DBG] 4.1d scrub ok 2024-02-23T00:41:56.238 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:54.302759+0000 osd.2 (osd.2) 43 : cluster [DBG] 2.c scrub starts 2024-02-23T00:41:56.238 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:54.303785+0000 osd.2 (osd.2) 44 : cluster [DBG] 2.c scrub ok 2024-02-23T00:41:56.238 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:55.155576+0000 mgr.y (mgr.14517) 108 : cluster [DBG] pgmap v92: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:41:56.238 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:54.080535+0000 osd.3 (osd.3) 61 : cluster [DBG] 4.1d scrub starts 2024-02-23T00:41:56.238 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:54.081972+0000 osd.3 (osd.3) 62 : cluster [DBG] 4.1d scrub ok 2024-02-23T00:41:56.238 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:54.302759+0000 osd.2 (osd.2) 43 : cluster [DBG] 2.c scrub starts 2024-02-23T00:41:56.238 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:54.303785+0000 osd.2 (osd.2) 44 : cluster [DBG] 2.c scrub ok 2024-02-23T00:41:56.238 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:55.155576+0000 mgr.y (mgr.14517) 108 : cluster [DBG] pgmap v92: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:41:57.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:55.723386+0000 osd.5 (osd.5) 49 : cluster [DBG] 6.7 scrub starts 2024-02-23T00:41:57.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:55.726315+0000 osd.5 (osd.5) 50 : cluster [DBG] 6.7 scrub ok 2024-02-23T00:41:57.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:55.915923+0000 osd.4 (osd.4) 57 : cluster [DBG] 4.1a deep-scrub starts 2024-02-23T00:41:57.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:55.919891+0000 osd.4 (osd.4) 58 : cluster [DBG] 4.1a deep-scrub ok 2024-02-23T00:41:57.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:55.723386+0000 osd.5 (osd.5) 49 : cluster [DBG] 6.7 scrub starts 2024-02-23T00:41:57.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:55.726315+0000 osd.5 (osd.5) 50 : cluster [DBG] 6.7 scrub ok 2024-02-23T00:41:57.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:55.915923+0000 osd.4 (osd.4) 57 : cluster [DBG] 4.1a deep-scrub starts 2024-02-23T00:41:57.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:55.919891+0000 osd.4 (osd.4) 58 : cluster [DBG] 4.1a deep-scrub ok 2024-02-23T00:41:57.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:55.723386+0000 osd.5 (osd.5) 49 : cluster [DBG] 6.7 scrub starts 2024-02-23T00:41:57.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:55.726315+0000 osd.5 (osd.5) 50 : cluster [DBG] 6.7 scrub ok 2024-02-23T00:41:57.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:55.915923+0000 osd.4 (osd.4) 57 : cluster [DBG] 4.1a deep-scrub starts 2024-02-23T00:41:57.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:55.919891+0000 osd.4 (osd.4) 58 : cluster [DBG] 4.1a deep-scrub ok 2024-02-23T00:41:57.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.968+0000 7f59c8627700 1 -- 172.21.15.47:0/3231642350 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f59c00fdbb0 msgr2=0x7f59c00fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:57.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.968+0000 7f59c8627700 1 --2- 172.21.15.47:0/3231642350 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f59c00fdbb0 0x7f59c00fdf90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f59bc004870 tx=0x7f59bc009cb0).stop 2024-02-23T00:41:57.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.968+0000 7f59c8627700 1 -- 172.21.15.47:0/3231642350 shutdown_connections 2024-02-23T00:41:57.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.968+0000 7f59c8627700 1 --2- 172.21.15.47:0/3231642350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59c00f8420 0x7f59c0104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:57.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.968+0000 7f59c8627700 1 --2- 172.21.15.47:0/3231642350 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f59c00fdbb0 0x7f59c00fdf90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:57.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.968+0000 7f59c8627700 1 --2- 172.21.15.47:0/3231642350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59c00fe510 0x7f59c00f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:57.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.968+0000 7f59c8627700 1 -- 172.21.15.47:0/3231642350 >> 172.21.15.47:0/3231642350 conn(0x7f59c00f3800 msgr2=0x7f59c00f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:41:57.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.968+0000 7f59c8627700 1 -- 172.21.15.47:0/3231642350 shutdown_connections 2024-02-23T00:41:57.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.968+0000 7f59c8627700 1 -- 172.21.15.47:0/3231642350 wait complete. 2024-02-23T00:41:57.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.968+0000 7f59c8627700 1 librados: starting msgr at 2024-02-23T00:41:57.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.968+0000 7f59c8627700 1 librados: starting objecter 2024-02-23T00:41:57.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.969+0000 7f59c8627700 1 Processor -- start 2024-02-23T00:41:57.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.969+0000 7f59c8627700 1 -- start start 2024-02-23T00:41:57.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.969+0000 7f59c8627700 1 librados: setting wanted keys 2024-02-23T00:41:57.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.969+0000 7f59c8627700 1 librados: calling monclient init 2024-02-23T00:41:57.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.969+0000 7f59c8627700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59c00f8420 0x7f59c010df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:57.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.969+0000 7f59c8627700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59c00fdbb0 0x7f59c010e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:57.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.969+0000 7f59c8627700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f59c00fe510 0x7f59c01080a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:57.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.969+0000 7f59c8627700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c0108590 con 0x7f59c00fdbb0 2024-02-23T00:41:57.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.969+0000 7f59c8627700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c0108700 con 0x7f59c00f8420 2024-02-23T00:41:57.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.969+0000 7f59c8627700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59c01088c0 con 0x7f59c00fe510 2024-02-23T00:41:57.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.969+0000 7f59c5bc2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59c00fdbb0 0x7f59c010e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:57.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.969+0000 7f59c6bc4700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f59c00fe510 0x7f59c01080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:57.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.969+0000 7f59c5bc2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59c00fdbb0 0x7f59c010e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52876/0 (socket says 172.21.15.47:52876) 2024-02-23T00:41:57.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.969+0000 7f59c5bc2700 1 -- 172.21.15.47:0/98830989 learned_addr learned my addr 172.21.15.47:0/98830989 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:41:57.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.969+0000 7f59c6bc4700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f59c00fe510 0x7f59c01080a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:33280/0 (socket says 172.21.15.47:33280) 2024-02-23T00:41:57.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.969+0000 7f59c63c3700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59c00f8420 0x7f59c010df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:57.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.969+0000 7f59c6bc4700 1 -- 172.21.15.47:0/98830989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59c00f8420 msgr2=0x7f59c010df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:57.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.969+0000 7f59c6bc4700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59c00f8420 0x7f59c010df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:57.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.969+0000 7f59c6bc4700 1 -- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59c00fdbb0 msgr2=0x7f59c010e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:57.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.969+0000 7f59c6bc4700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59c00fdbb0 0x7f59c010e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:57.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.969+0000 7f59c6bc4700 1 -- 172.21.15.47:0/98830989 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b0005040 con 0x7f59c00fe510 2024-02-23T00:41:57.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.969+0000 7f59c5bc2700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59c00fdbb0 0x7f59c010e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:41:57.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.970+0000 7f59c6bc4700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f59c00fe510 0x7f59c01080a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f59b800c7a0 tx=0x7f59b8007d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:57.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.970+0000 7f59b77fe700 10 client.?.objecter ms_handle_connect 0x7f59c00fe510 2024-02-23T00:41:57.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.970+0000 7f59b77fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:41:57.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.970+0000 7f59b77fe700 1 -- 172.21.15.47:0/98830989 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b800e4a0 con 0x7f59c00fe510 2024-02-23T00:41:57.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.970+0000 7f59c8627700 1 -- 172.21.15.47:0/98830989 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59bc005040 con 0x7f59c00fe510 2024-02-23T00:41:57.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.970+0000 7f59b77fe700 1 -- 172.21.15.47:0/98830989 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f59b800ea90 con 0x7f59c00fe510 2024-02-23T00:41:57.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.970+0000 7f59b77fe700 1 -- 172.21.15.47:0/98830989 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b80129f0 con 0x7f59c00fe510 2024-02-23T00:41:57.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.970+0000 7f59c8627700 10 client.24641.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:41:57.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.970+0000 7f59c8627700 1 -- 172.21.15.47:0/98830989 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59c0108f00 con 0x7f59c00fe510 2024-02-23T00:41:57.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.970+0000 7f59c8627700 1 librados: init done 2024-02-23T00:41:57.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.970+0000 7f59c8627700 1 -- 172.21.15.47:0/98830989 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f59c005eec0 con 0x7f59c00fe510 2024-02-23T00:41:57.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.971+0000 7f59b77fe700 1 -- 172.21.15.47:0/98830989 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f59b800e600 con 0x7f59c00fe510 2024-02-23T00:41:57.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.971+0000 7f59b77fe700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f59ac062cc0 0x7f59ac065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:57.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.972+0000 7f59b77fe700 1 -- 172.21.15.47:0/98830989 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(95..95 src has 38..95) v4 ==== 7421+0+0 (secure 0 0 0) 0x7f59b807c860 con 0x7f59c00fe510 2024-02-23T00:41:57.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.972+0000 7f59b77fe700 10 client.24641.objecter ms_dispatch 0x7f59c0001550 osd_map(95..95 src has 38..95) v4 2024-02-23T00:41:57.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.972+0000 7f59b77fe700 3 client.24641.objecter handle_osd_map got epochs [95,95] > 0 2024-02-23T00:41:57.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.972+0000 7f59b77fe700 3 client.24641.objecter handle_osd_map decoding full epoch 95 2024-02-23T00:41:58.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.972+0000 7f59c63c3700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f59ac062cc0 0x7f59ac065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:58.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.972+0000 7f59b77fe700 20 client.24641.objecter dump_active .. 0 homeless 2024-02-23T00:41:58.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.972+0000 7f59b77fe700 1 -- 172.21.15.47:0/98830989 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1259 (secure 0 0 0) 0x7f59b8081220 con 0x7f59c00fe510 2024-02-23T00:41:58.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.972+0000 7f59b77fe700 10 client.24641.objecter ms_handle_connect 0x7f59ac062cc0 2024-02-23T00:41:58.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.972+0000 7f59c63c3700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f59ac062cc0 0x7f59ac065180 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f59bc004ee0 tx=0x7f59bc00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:58.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.972+0000 7f59c8627700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59c00fe950 0x7f59c00fed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:58.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.972+0000 7f59c8627700 1 -- 172.21.15.47:0/98830989 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c0106e20 con 0x7f59c00fe950 2024-02-23T00:41:58.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.972+0000 7f59c5bc2700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59c00fe950 0x7f59c00fed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:58.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.972+0000 7f59c5bc2700 1 -- 172.21.15.47:0/98830989 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "get_command_descriptions"}) v1 -- 0x7f59c0109580 con 0x7f59c00fe950 2024-02-23T00:41:58.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.972+0000 7f59b77fe700 10 client.24641.objecter ms_handle_connect 0x7f59c00fe950 2024-02-23T00:41:58.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.972+0000 7f59b77fe700 10 client.24641.objecter resend_mon_ops 2024-02-23T00:41:58.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.972+0000 7f59c5bc2700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59c00fe950 0x7f59c00fed30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f59b0004e20 tx=0x7f59b000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:58.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.973+0000 7f59b77fe700 1 -- 172.21.15.47:0/98830989 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b000e030 con 0x7f59c00fe950 2024-02-23T00:41:58.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.973+0000 7f59b77fe700 1 -- 172.21.15.47:0/98830989 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+8167 (secure 0 0 0) 0x7f59b000f040 con 0x7f59c00fe950 2024-02-23T00:41:58.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.973+0000 7f59b77fe700 1 -- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59c00fe950 msgr2=0x7f59c00fed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:58.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.973+0000 7f59b77fe700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59c00fe950 0x7f59c00fed30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f59b0004e20 tx=0x7f59b000b040).stop 2024-02-23T00:41:58.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.973+0000 7f59b77fe700 1 -- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59c00fe950 msgr2=0x7f59c00fed30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-02-23T00:41:58.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.973+0000 7f59b77fe700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59c00fe950 0x7f59c00fed30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:58.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.980+0000 7f59c8627700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59c00ff220 0x7f59c0118e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:58.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.980+0000 7f59c8627700 1 -- 172.21.15.47:0/98830989 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c005fa60 con 0x7f59c00ff220 2024-02-23T00:41:58.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.980+0000 7f59c5bc2700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59c00ff220 0x7f59c0118e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:58.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.980+0000 7f59c5bc2700 1 -- 172.21.15.47:0/98830989 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 3: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=0"]}) v1 -- 0x7f59c005fba0 con 0x7f59c00ff220 2024-02-23T00:41:58.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.980+0000 7f59b77fe700 10 client.24641.objecter ms_handle_connect 0x7f59c00ff220 2024-02-23T00:41:58.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.980+0000 7f59b77fe700 10 client.24641.objecter resend_mon_ops 2024-02-23T00:41:58.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.980+0000 7f59c5bc2700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59c00ff220 0x7f59c0118e40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f59b0004e20 tx=0x7f59b001a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:58.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.980+0000 7f59b77fe700 1 -- 172.21.15.47:0/98830989 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b0014ad0 con 0x7f59c00ff220 2024-02-23T00:41:58.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.981+0000 7f59b77fe700 1 -- 172.21.15.47:0/98830989 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 3: 0 mon_osd_down_out_interval = '0' (not observed, change may require restart) ) v1 ==== 83+0+3 (secure 0 0 0) 0x7f59b0011b40 con 0x7f59c00ff220 2024-02-23T00:41:58.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.981+0000 7f59b77fe700 1 -- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59c00ff220 msgr2=0x7f59c0118e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:58.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.981+0000 7f59b77fe700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59c00ff220 0x7f59c0118e40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f59b0004e20 tx=0x7f59b001a000).stop 2024-02-23T00:41:58.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.981+0000 7f59b77fe700 1 -- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59c00ff220 msgr2=0x7f59c0118e40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-02-23T00:41:58.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.981+0000 7f59b77fe700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59c00ff220 0x7f59c0118e40 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:58.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.982+0000 7f59c8627700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59c00273c0 0x7f59c00277d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:58.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.982+0000 7f59c8627700 1 -- 172.21.15.47:0/98830989 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c00ff910 con 0x7f59c00273c0 2024-02-23T00:41:58.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.982+0000 7f59c5bc2700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59c00273c0 0x7f59c00277d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:58.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.982+0000 7f59c5bc2700 1 -- 172.21.15.47:0/98830989 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- command(tid 4: {"prefix": "get_command_descriptions"}) v1 -- 0x7f59c0027cc0 con 0x7f59c00273c0 2024-02-23T00:41:58.002 INFO:teuthology.orchestra.run.smithi047.stdout:mon.a: {} 2024-02-23T00:41:58.002 INFO:teuthology.orchestra.run.smithi047.stdout:mon.b: {} 2024-02-23T00:41:58.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.982+0000 7f59b77fe700 10 client.24641.objecter ms_handle_connect 0x7f59c00273c0 2024-02-23T00:41:58.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.982+0000 7f59b77fe700 10 client.24641.objecter resend_mon_ops 2024-02-23T00:41:58.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.982+0000 7f59c5bc2700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59c00273c0 0x7f59c00277d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f59c0108b70 tx=0x7f59b0023000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:58.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.983+0000 7f59b77fe700 1 -- 172.21.15.47:0/98830989 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b002e7e0 con 0x7f59c00273c0 2024-02-23T00:41:58.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.983+0000 7f59b77fe700 1 -- 172.21.15.47:0/98830989 <== mon.1 v2:172.21.15.196:3300/0 2 ==== command_reply(tid 4: 0 ) v1 ==== 8+0+8167 (secure 0 0 0) 0x7f59b0014860 con 0x7f59c00273c0 2024-02-23T00:41:58.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.983+0000 7f59b77fe700 1 -- 172.21.15.47:0/98830989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59c00273c0 msgr2=0x7f59c00277d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:58.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.983+0000 7f59b77fe700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59c00273c0 0x7f59c00277d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f59c0108b70 tx=0x7f59b0023000).stop 2024-02-23T00:41:58.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.983+0000 7f59b77fe700 1 -- 172.21.15.47:0/98830989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59c00273c0 msgr2=0x7f59c00277d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-02-23T00:41:58.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.983+0000 7f59b77fe700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59c00273c0 0x7f59c00277d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:58.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.983+0000 7f59c6bc4700 1 -- 172.21.15.47:0/98830989 reap_dead start 2024-02-23T00:41:58.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.990+0000 7f59c8627700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59c010e0b0 0x7f59c0027e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:58.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.990+0000 7f59c8627700 1 -- 172.21.15.47:0/98830989 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c010e530 con 0x7f59c010e0b0 2024-02-23T00:41:58.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.990+0000 7f59c5bc2700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59c010e0b0 0x7f59c0027e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:58.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.990+0000 7f59c5bc2700 1 -- 172.21.15.47:0/98830989 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- command(tid 5: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=0"]}) v1 -- 0x7f59c010e670 con 0x7f59c010e0b0 2024-02-23T00:41:58.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.990+0000 7f59b77fe700 10 client.24641.objecter ms_handle_connect 0x7f59c010e0b0 2024-02-23T00:41:58.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.990+0000 7f59c5bc2700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59c010e0b0 0x7f59c0027e90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f59c00ffa50 tx=0x7f59b0030040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:58.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.990+0000 7f59b77fe700 10 client.24641.objecter resend_mon_ops 2024-02-23T00:41:58.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.991+0000 7f59b77fe700 1 -- 172.21.15.47:0/98830989 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b002b6d0 con 0x7f59c010e0b0 2024-02-23T00:41:58.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.992+0000 7f59b77fe700 1 -- 172.21.15.47:0/98830989 <== mon.1 v2:172.21.15.196:3300/0 2 ==== command_reply(tid 5: 0 mon_osd_down_out_interval = '0' (not observed, change may require restart) ) v1 ==== 83+0+3 (secure 0 0 0) 0x7f59b002ee90 con 0x7f59c010e0b0 2024-02-23T00:41:58.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.992+0000 7f59b77fe700 1 -- 172.21.15.47:0/98830989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59c010e0b0 msgr2=0x7f59c0027e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:58.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.992+0000 7f59b77fe700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59c010e0b0 0x7f59c0027e90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f59c00ffa50 tx=0x7f59b0030040).stop 2024-02-23T00:41:58.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.992+0000 7f59b77fe700 1 -- 172.21.15.47:0/98830989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59c010e0b0 msgr2=0x7f59c0027e90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-02-23T00:41:58.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.992+0000 7f59b77fe700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59c010e0b0 0x7f59c0027e90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:41:58.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.992+0000 7f59c8627700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f59c0028470 0x7f59c011b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:58.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.992+0000 7f59c8627700 1 -- 172.21.15.47:0/98830989 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59c011df90 con 0x7f59c0028470 2024-02-23T00:41:58.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.992+0000 7f59c5bc2700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f59c0028470 0x7f59c011b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:58.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.992+0000 7f59c5bc2700 1 -- 172.21.15.47:0/98830989 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 6: {"prefix": "get_command_descriptions"}) v1 -- 0x7f59c011e0d0 con 0x7f59c0028470 2024-02-23T00:41:58.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.992+0000 7f59b77fe700 10 client.24641.objecter ms_handle_connect 0x7f59c0028470 2024-02-23T00:41:58.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.992+0000 7f59b77fe700 10 client.24641.objecter resend_mon_ops 2024-02-23T00:41:58.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.992+0000 7f59c5bc2700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f59c0028470 0x7f59c011b340 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f59c00ff660 tx=0x7f59b0038000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:58.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.993+0000 7f59b77fe700 1 -- 172.21.15.47:0/98830989 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b0045070 con 0x7f59c0028470 2024-02-23T00:41:58.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.993+0000 7f59b77fe700 1 -- 172.21.15.47:0/98830989 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 6: 0 ) v1 ==== 8+0+8167 (secure 0 0 0) 0x7f59b002b400 con 0x7f59c0028470 2024-02-23T00:41:58.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.993+0000 7f59b77fe700 1 -- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f59c0028470 msgr2=0x7f59c011b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:58.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.993+0000 7f59b77fe700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f59c0028470 0x7f59c011b340 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f59c00ff660 tx=0x7f59b0038000).stop 2024-02-23T00:41:58.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.993+0000 7f59b77fe700 1 -- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f59c0028470 msgr2=0x7f59c011b340 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-02-23T00:41:58.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.993+0000 7f59b77fe700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f59c0028470 0x7f59c011b340 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0x7f59c00ff660 tx=0x7f59b0038000).stop 2024-02-23T00:41:58.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.999+0000 7f59c8627700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f59c011e3f0 0x7f59c011d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:41:58.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:57.999+0000 7f59c8627700 1 -- 172.21.15.47:0/98830989 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59c011ddd0 con 0x7f59c011e3f0 2024-02-23T00:41:58.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:58.000+0000 7f59c5bc2700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f59c011e3f0 0x7f59c011d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:41:58.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:58.000+0000 7f59c5bc2700 1 -- 172.21.15.47:0/98830989 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=0"]}) v1 -- 0x7f59c011fdb0 con 0x7f59c011e3f0 2024-02-23T00:41:58.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:58.000+0000 7f59b77fe700 10 client.24641.objecter ms_handle_connect 0x7f59c011e3f0 2024-02-23T00:41:58.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:58.000+0000 7f59b77fe700 10 client.24641.objecter resend_mon_ops 2024-02-23T00:41:58.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:58.000+0000 7f59c5bc2700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f59c011e3f0 0x7f59c011d840 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f59c0028880 tx=0x7f59b0040000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:41:58.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:58.000+0000 7f59b77fe700 1 -- 172.21.15.47:0/98830989 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b00143b0 con 0x7f59c011e3f0 2024-02-23T00:41:58.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:58.001+0000 7f59b77fe700 1 -- 172.21.15.47:0/98830989 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 7: 0 mon_osd_down_out_interval = '0' (not observed, change may require restart) ) v1 ==== 83+0+3 (secure 0 0 0) 0x7f59b0039440 con 0x7f59c011e3f0 2024-02-23T00:41:58.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:58.001+0000 7f59b77fe700 1 -- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f59c011e3f0 msgr2=0x7f59c011d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:58.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:58.001+0000 7f59b77fe700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f59c011e3f0 0x7f59c011d840 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f59c0028880 tx=0x7f59b0040000).stop 2024-02-23T00:41:58.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:58.001+0000 7f59b77fe700 1 -- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f59c011e3f0 msgr2=0x7f59c011d840 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-02-23T00:41:58.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:58.001+0000 7f59b77fe700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f59c011e3f0 0x7f59c011d840 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0x7f59c0028880 tx=0x7f59b0040000).stop 2024-02-23T00:41:58.007 INFO:teuthology.orchestra.run.smithi047.stdout:mon.c: {} 2024-02-23T00:41:58.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:58.001+0000 7f59c8627700 10 librados: watch_flush enter 2024-02-23T00:41:58.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:58.001+0000 7f59c8627700 10 librados: watch_flush exit 2024-02-23T00:41:58.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:58.001+0000 7f59c8627700 20 client.24641.objecter shutdown clearing up homeless session... 2024-02-23T00:41:58.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:58.001+0000 7f59c8627700 10 client.24641.objecter successfully canceled tick 2024-02-23T00:41:58.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:58.002+0000 7f59c8627700 1 -- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f59ac062cc0 msgr2=0x7f59ac065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:58.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:58.002+0000 7f59c8627700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f59ac062cc0 0x7f59ac065180 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f59bc004ee0 tx=0x7f59bc00b040).stop 2024-02-23T00:41:58.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:58.002+0000 7f59c8627700 1 -- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f59c00fe510 msgr2=0x7f59c01080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:41:58.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:58.002+0000 7f59c8627700 1 --2- 172.21.15.47:0/98830989 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f59c00fe510 0x7f59c01080a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f59b800c7a0 tx=0x7f59b8007d10).stop 2024-02-23T00:41:58.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:58.002+0000 7f59c6bc4700 1 -- 172.21.15.47:0/98830989 reap_dead start 2024-02-23T00:41:58.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:58.002+0000 7f59c8627700 1 -- 172.21.15.47:0/98830989 shutdown_connections 2024-02-23T00:41:58.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:58.002+0000 7f59c8627700 1 -- 172.21.15.47:0/98830989 >> 172.21.15.47:0/98830989 conn(0x7f59c00f3800 msgr2=0x7f59c00fb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:41:58.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:58.002+0000 7f59c8627700 1 -- 172.21.15.47:0/98830989 shutdown_connections 2024-02-23T00:41:58.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:58.002+0000 7f59c8627700 1 -- 172.21.15.47:0/98830989 wait complete. 2024-02-23T00:41:58.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:41:58.002+0000 7f59c8627700 1 librados: shutdown 2024-02-23T00:41:58.009 INFO:teuthology.orchestra.run.smithi047.stderr:mon.a: mon_osd_down_out_interval = '0' (not observed, change may require restart) 2024-02-23T00:41:58.009 INFO:teuthology.orchestra.run.smithi047.stderr:mon.b: mon_osd_down_out_interval = '0' (not observed, change may require restart) 2024-02-23T00:41:58.009 INFO:teuthology.orchestra.run.smithi047.stderr:mon.c: mon_osd_down_out_interval = '0' (not observed, change may require restart) 2024-02-23T00:41:58.033 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:56.725545+0000 osd.1 (osd.1) 57 : cluster [DBG] 2.9 scrub starts 2024-02-23T00:41:58.033 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:56.728567+0000 osd.1 (osd.1) 58 : cluster [DBG] 2.9 scrub ok 2024-02-23T00:41:58.033 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:57.052184+0000 osd.3 (osd.3) 63 : cluster [DBG] 2.19 scrub starts 2024-02-23T00:41:58.033 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:57.053290+0000 osd.3 (osd.3) 64 : cluster [DBG] 2.19 scrub ok 2024-02-23T00:41:58.033 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:57.137540+0000 osd.6 (osd.6) 41 : cluster [DBG] 2.1a scrub starts 2024-02-23T00:41:58.033 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:57.138509+0000 osd.6 (osd.6) 42 : cluster [DBG] 2.1a scrub ok 2024-02-23T00:41:58.033 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:57.156190+0000 mgr.y (mgr.14517) 109 : cluster [DBG] pgmap v93: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:41:58.033 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:56.725545+0000 osd.1 (osd.1) 57 : cluster [DBG] 2.9 scrub starts 2024-02-23T00:41:58.033 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:56.728567+0000 osd.1 (osd.1) 58 : cluster [DBG] 2.9 scrub ok 2024-02-23T00:41:58.034 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:57.052184+0000 osd.3 (osd.3) 63 : cluster [DBG] 2.19 scrub starts 2024-02-23T00:41:58.034 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:57.053290+0000 osd.3 (osd.3) 64 : cluster [DBG] 2.19 scrub ok 2024-02-23T00:41:58.034 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:57.137540+0000 osd.6 (osd.6) 41 : cluster [DBG] 2.1a scrub starts 2024-02-23T00:41:58.034 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:57.138509+0000 osd.6 (osd.6) 42 : cluster [DBG] 2.1a scrub ok 2024-02-23T00:41:58.034 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:57.156190+0000 mgr.y (mgr.14517) 109 : cluster [DBG] pgmap v93: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:41:58.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:56.725545+0000 osd.1 (osd.1) 57 : cluster [DBG] 2.9 scrub starts 2024-02-23T00:41:58.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:56.728567+0000 osd.1 (osd.1) 58 : cluster [DBG] 2.9 scrub ok 2024-02-23T00:41:58.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:57.052184+0000 osd.3 (osd.3) 63 : cluster [DBG] 2.19 scrub starts 2024-02-23T00:41:58.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:57.053290+0000 osd.3 (osd.3) 64 : cluster [DBG] 2.19 scrub ok 2024-02-23T00:41:58.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:57.137540+0000 osd.6 (osd.6) 41 : cluster [DBG] 2.1a scrub starts 2024-02-23T00:41:58.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:57.138509+0000 osd.6 (osd.6) 42 : cluster [DBG] 2.1a scrub ok 2024-02-23T00:41:58.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:57.156190+0000 mgr.y (mgr.14517) 109 : cluster [DBG] pgmap v93: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:41:58.795 INFO:teuthology.run_tasks:Running task workunit... 2024-02-23T00:41:58.808 INFO:tasks.workunit:Pulling workunits from ref 4f758cd8e8854d1f126da86e254076852bf059a9 2024-02-23T00:41:58.809 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-02-23T00:41:58.809 DEBUG:teuthology.orchestra.run.smithi047:> stat -- /home/ubuntu/cephtest/mnt.0 2024-02-23T00:41:58.810 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-02-23T00:41:58.810 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-23T00:41:58.810 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-23T00:41:58.810 INFO:tasks.thrashosds.thrasher:Reweighting osd 4 to 0.9988540557681002 2024-02-23T00:41:58.810 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight 4 0.9988540557681002 2024-02-23T00:41:58.811 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-02-23T00:41:58.811 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:41:58.811 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-02-23T00:41:58.811 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:41:58.813 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-02-23T00:41:58.813 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.0 dump_ops_in_flight 2024-02-23T00:41:58.813 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-02-23T00:41:58.814 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set noscrub 2024-02-23T00:41:58.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-23T00:41:58.840 INFO:teuthology.orchestra.run.smithi047.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-02-23T00:41:58.841 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-02-23T00:41:58.851 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:41:58.875 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:41:58.877 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:41:58.878 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:41:58.902 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-02-23T00:41:58.902 DEBUG:teuthology.orchestra.run.smithi047:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-02-23T00:41:58.963 INFO:tasks.workunit:timeout=3h 2024-02-23T00:41:58.964 INFO:tasks.workunit:cleanup=True 2024-02-23T00:41:58.964 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/kamoltat/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 4f758cd8e8854d1f126da86e254076852bf059a9 2024-02-23T00:41:58.979 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:41:59.033 INFO:tasks.workunit.client.0.smithi047.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-02-23T00:41:59.044 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:57.375462+0000 osd.2 (osd.2) 45 : cluster [DBG] 5.8 scrub starts 2024-02-23T00:41:59.044 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:57.376569+0000 osd.2 (osd.2) 46 : cluster [DBG] 5.8 scrub ok 2024-02-23T00:41:59.044 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:57.685063+0000 osd.1 (osd.1) 59 : cluster [DBG] 2.15 scrub starts 2024-02-23T00:41:59.044 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:57.686953+0000 osd.1 (osd.1) 60 : cluster [DBG] 2.15 scrub ok 2024-02-23T00:41:59.044 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:41:57.972882+0000 mon.c (mon.2) 23 : audit [DBG] from='client.? 172.21.15.47:0/98830989' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-23T00:41:59.044 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:58.118065+0000 osd.6 (osd.6) 43 : cluster [DBG] 4.a scrub starts 2024-02-23T00:41:59.044 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:58.120554+0000 osd.6 (osd.6) 44 : cluster [DBG] 4.a scrub ok 2024-02-23T00:41:59.046 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:57.375462+0000 osd.2 (osd.2) 45 : cluster [DBG] 5.8 scrub starts 2024-02-23T00:41:59.046 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:57.376569+0000 osd.2 (osd.2) 46 : cluster [DBG] 5.8 scrub ok 2024-02-23T00:41:59.046 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:57.685063+0000 osd.1 (osd.1) 59 : cluster [DBG] 2.15 scrub starts 2024-02-23T00:41:59.046 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:57.686953+0000 osd.1 (osd.1) 60 : cluster [DBG] 2.15 scrub ok 2024-02-23T00:41:59.046 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:41:57.972882+0000 mon.c (mon.2) 23 : audit [DBG] from='client.? 172.21.15.47:0/98830989' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-23T00:41:59.046 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:58.118065+0000 osd.6 (osd.6) 43 : cluster [DBG] 4.a scrub starts 2024-02-23T00:41:59.046 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:58.120554+0000 osd.6 (osd.6) 44 : cluster [DBG] 4.a scrub ok 2024-02-23T00:41:59.047 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:41:59.076 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:41:59.080 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:41:59.080 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:41:59.179 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:41:59.181 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:41:59.186 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:41:59.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:57.375462+0000 osd.2 (osd.2) 45 : cluster [DBG] 5.8 scrub starts 2024-02-23T00:41:59.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:57.376569+0000 osd.2 (osd.2) 46 : cluster [DBG] 5.8 scrub ok 2024-02-23T00:41:59.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:57.685063+0000 osd.1 (osd.1) 59 : cluster [DBG] 2.15 scrub starts 2024-02-23T00:41:59.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:57.686953+0000 osd.1 (osd.1) 60 : cluster [DBG] 2.15 scrub ok 2024-02-23T00:41:59.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:41:57.972882+0000 mon.c (mon.2) 23 : audit [DBG] from='client.? 172.21.15.47:0/98830989' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-23T00:41:59.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:58.118065+0000 osd.6 (osd.6) 43 : cluster [DBG] 4.a scrub starts 2024-02-23T00:41:59.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:58.120554+0000 osd.6 (osd.6) 44 : cluster [DBG] 4.a scrub ok 2024-02-23T00:41:59.224 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:41:59.412 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:41:59.416 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:41:59.774 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:41:59.777 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:41:59.778 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:41:59.879 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:41:59.911 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:41:59.933 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:41:59.936 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:41:59.936 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:42:00.037 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:42:00.067 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:00.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:41:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:59.156525+0000 mgr.y (mgr.14517) 110 : cluster [DBG] pgmap v94: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:42:00.265 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:42:00.267 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:41:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:59.156525+0000 mgr.y (mgr.14517) 110 : cluster [DBG] pgmap v94: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:42:00.267 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:41:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:59.156525+0000 mgr.y (mgr.14517) 110 : cluster [DBG] pgmap v94: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:42:00.268 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:00.268 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:42:00.369 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:42:00.402 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:00.423 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:42:00.427 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:00.428 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:42:00.528 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:42:00.556 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:42:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:42:00] "GET /metrics HTTP/1.1" 200 172167 "" "Prometheus/2.33.4" 2024-02-23T00:42:00.557 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:00.577 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:42:00.580 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:00.580 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:42:00.681 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:42:00.708 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:00.727 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:42:00.730 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:00.730 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:42:00.831 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:42:00.861 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:01.056 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:59.691145+0000 osd.5 (osd.5) 51 : cluster [DBG] 5.7 deep-scrub starts 2024-02-23T00:42:01.056 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:41:59.694029+0000 osd.5 (osd.5) 52 : cluster [DBG] 5.7 deep-scrub ok 2024-02-23T00:42:01.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:00.003636+0000 osd.4 (osd.4) 59 : cluster [DBG] 6.1a scrub starts 2024-02-23T00:42:01.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:00.004512+0000 osd.4 (osd.4) 60 : cluster [DBG] 6.1a scrub ok 2024-02-23T00:42:01.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:59.691145+0000 osd.5 (osd.5) 51 : cluster [DBG] 5.7 deep-scrub starts 2024-02-23T00:42:01.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:41:59.694029+0000 osd.5 (osd.5) 52 : cluster [DBG] 5.7 deep-scrub ok 2024-02-23T00:42:01.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:00.003636+0000 osd.4 (osd.4) 59 : cluster [DBG] 6.1a scrub starts 2024-02-23T00:42:01.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:00.004512+0000 osd.4 (osd.4) 60 : cluster [DBG] 6.1a scrub ok 2024-02-23T00:42:01.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:59.691145+0000 osd.5 (osd.5) 51 : cluster [DBG] 5.7 deep-scrub starts 2024-02-23T00:42:01.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:41:59.694029+0000 osd.5 (osd.5) 52 : cluster [DBG] 5.7 deep-scrub ok 2024-02-23T00:42:01.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:00.003636+0000 osd.4 (osd.4) 59 : cluster [DBG] 6.1a scrub starts 2024-02-23T00:42:01.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:00.004512+0000 osd.4 (osd.4) 60 : cluster [DBG] 6.1a scrub ok 2024-02-23T00:42:01.711 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:42:01.715 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:01.715 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:42:01.816 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:42:01.846 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:01.944 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:42:01.949 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:01.949 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:42:02.050 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:42:02.084 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:02.186 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:42:02.189 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:02.190 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:42:02.194 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:00.995991+0000 osd.7 (osd.7) 55 : cluster [DBG] 5.1a scrub starts 2024-02-23T00:42:02.194 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:00.998880+0000 osd.7 (osd.7) 56 : cluster [DBG] 5.1a scrub ok 2024-02-23T00:42:02.194 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:01.156801+0000 mgr.y (mgr.14517) 111 : cluster [DBG] pgmap v95: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:42:02.194 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:01.190424+0000 osd.0 (osd.0) 49 : cluster [DBG] 4.6 deep-scrub starts 2024-02-23T00:42:02.194 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:01.195540+0000 osd.0 (osd.0) 50 : cluster [DBG] 4.6 deep-scrub ok 2024-02-23T00:42:02.194 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:00.995991+0000 osd.7 (osd.7) 55 : cluster [DBG] 5.1a scrub starts 2024-02-23T00:42:02.195 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:00.998880+0000 osd.7 (osd.7) 56 : cluster [DBG] 5.1a scrub ok 2024-02-23T00:42:02.195 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:01.156801+0000 mgr.y (mgr.14517) 111 : cluster [DBG] pgmap v95: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:42:02.195 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:01.190424+0000 osd.0 (osd.0) 49 : cluster [DBG] 4.6 deep-scrub starts 2024-02-23T00:42:02.195 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:01.195540+0000 osd.0 (osd.0) 50 : cluster [DBG] 4.6 deep-scrub ok 2024-02-23T00:42:02.240 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:00.995991+0000 osd.7 (osd.7) 55 : cluster [DBG] 5.1a scrub starts 2024-02-23T00:42:02.240 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:00.998880+0000 osd.7 (osd.7) 56 : cluster [DBG] 5.1a scrub ok 2024-02-23T00:42:02.240 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:01.156801+0000 mgr.y (mgr.14517) 111 : cluster [DBG] pgmap v95: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:42:02.240 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:01.190424+0000 osd.0 (osd.0) 49 : cluster [DBG] 4.6 deep-scrub starts 2024-02-23T00:42:02.240 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:01.195540+0000 osd.0 (osd.0) 50 : cluster [DBG] 4.6 deep-scrub ok 2024-02-23T00:42:02.291 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:42:02.324 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:02.345 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:42:02.347 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:02.348 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:42:02.449 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:42:02.481 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:02.528 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:42:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:42:02.559 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:42:02.561 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:02.562 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:42:02.662 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:42:02.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.450+0000 7fc34b29e700 1 -- 172.21.15.47:0/2724929955 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc3440fd510 msgr2=0x7fc3440f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:02.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.450+0000 7fc34b29e700 1 --2- 172.21.15.47:0/2724929955 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc3440fd510 0x7fc3440f6d40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fc334004870 tx=0x7fc334009cb0).stop 2024-02-23T00:42:02.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.450+0000 7fc34b29e700 1 -- 172.21.15.47:0/2724929955 shutdown_connections 2024-02-23T00:42:02.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.450+0000 7fc34b29e700 1 --2- 172.21.15.47:0/2724929955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3440f7400 0x7fc344103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:02.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.450+0000 7fc34b29e700 1 --2- 172.21.15.47:0/2724929955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3440fcbb0 0x7fc3440fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:02.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.450+0000 7fc34b29e700 1 --2- 172.21.15.47:0/2724929955 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc3440fd510 0x7fc3440f6d40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:02.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.450+0000 7fc34b29e700 1 -- 172.21.15.47:0/2724929955 >> 172.21.15.47:0/2724929955 conn(0x7fc3440f2820 msgr2=0x7fc3440f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:42:02.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.450+0000 7fc34b29e700 1 -- 172.21.15.47:0/2724929955 shutdown_connections 2024-02-23T00:42:02.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.450+0000 7fc34b29e700 1 -- 172.21.15.47:0/2724929955 wait complete. 2024-02-23T00:42:02.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.451+0000 7fc34b29e700 1 librados: starting msgr at 2024-02-23T00:42:02.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.451+0000 7fc34b29e700 1 librados: starting objecter 2024-02-23T00:42:02.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.451+0000 7fc34b29e700 1 Processor -- start 2024-02-23T00:42:02.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.451+0000 7fc34b29e700 1 -- start start 2024-02-23T00:42:02.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.451+0000 7fc34b29e700 1 librados: setting wanted keys 2024-02-23T00:42:02.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.451+0000 7fc34b29e700 1 librados: calling monclient init 2024-02-23T00:42:02.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.451+0000 7fc34b29e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc3440f7400 0x7fc3440fdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:02.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.451+0000 7fc34b29e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3440fcbb0 0x7fc3440fe0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:02.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.451+0000 7fc34b29e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3440fd510 0x7fc3440fe590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:02.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.451+0000 7fc34b29e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3441165e0 con 0x7fc3440fd510 2024-02-23T00:42:02.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.451+0000 7fc34b29e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc344116750 con 0x7fc3440fcbb0 2024-02-23T00:42:02.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.451+0000 7fc34b29e700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc344116910 con 0x7fc3440f7400 2024-02-23T00:42:02.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.451+0000 7fc34903a700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc3440f7400 0x7fc3440fdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:02.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.451+0000 7fc34903a700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc3440f7400 0x7fc3440fdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:33322/0 (socket says 172.21.15.47:33322) 2024-02-23T00:42:02.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.451+0000 7fc34903a700 1 -- 172.21.15.47:0/2781021069 learned_addr learned my addr 172.21.15.47:0/2781021069 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:42:02.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.451+0000 7fc34983b700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3440fd510 0x7fc3440fe590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:02.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.451+0000 7fc348839700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3440fcbb0 0x7fc3440fe0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:02.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.452+0000 7fc34903a700 1 -- 172.21.15.47:0/2781021069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3440fcbb0 msgr2=0x7fc3440fe0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:02.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.452+0000 7fc34903a700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3440fcbb0 0x7fc3440fe0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:02.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.452+0000 7fc34903a700 1 -- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3440fd510 msgr2=0x7fc3440fe590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:02.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.452+0000 7fc34903a700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3440fd510 0x7fc3440fe590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:02.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.452+0000 7fc34903a700 1 -- 172.21.15.47:0/2781021069 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc340005040 con 0x7fc3440f7400 2024-02-23T00:42:02.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.452+0000 7fc34903a700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc3440f7400 0x7fc3440fdbb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fc3400048f0 tx=0x7fc340009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:02.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.452+0000 7fc33a777700 10 client.?.objecter ms_handle_connect 0x7fc3440f7400 2024-02-23T00:42:02.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.452+0000 7fc33a777700 10 client.?.objecter resend_mon_ops 2024-02-23T00:42:02.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.452+0000 7fc33a777700 1 -- 172.21.15.47:0/2781021069 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc340007b00 con 0x7fc3440f7400 2024-02-23T00:42:02.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.452+0000 7fc33a777700 1 -- 172.21.15.47:0/2781021069 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fc34000f040 con 0x7fc3440f7400 2024-02-23T00:42:02.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.452+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc334005040 con 0x7fc3440f7400 2024-02-23T00:42:02.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.452+0000 7fc33a777700 1 -- 172.21.15.47:0/2781021069 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc340014820 con 0x7fc3440f7400 2024-02-23T00:42:02.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.452+0000 7fc34b29e700 10 client.24653.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:42:02.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.452+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc344116fa0 con 0x7fc3440f7400 2024-02-23T00:42:02.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.452+0000 7fc34b29e700 1 librados: init done 2024-02-23T00:42:02.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.453+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fc34405fa60 con 0x7fc3440f7400 2024-02-23T00:42:02.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.454+0000 7fc33a777700 1 -- 172.21.15.47:0/2781021069 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fc340014c50 con 0x7fc3440f7400 2024-02-23T00:42:02.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.454+0000 7fc33a777700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc330062cc0 0x7fc330065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:02.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.454+0000 7fc33a777700 1 -- 172.21.15.47:0/2781021069 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(95..95 src has 38..95) v4 ==== 7421+0+0 (secure 0 0 0) 0x7fc34007c9b0 con 0x7fc3440f7400 2024-02-23T00:42:02.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.454+0000 7fc33a777700 10 client.24653.objecter ms_dispatch 0x7fc344001550 osd_map(95..95 src has 38..95) v4 2024-02-23T00:42:02.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.454+0000 7fc33a777700 3 client.24653.objecter handle_osd_map got epochs [95,95] > 0 2024-02-23T00:42:02.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.454+0000 7fc33a777700 3 client.24653.objecter handle_osd_map decoding full epoch 95 2024-02-23T00:42:02.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.454+0000 7fc33a777700 20 client.24653.objecter dump_active .. 0 homeless 2024-02-23T00:42:02.696 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:42:02.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.454+0000 7fc33a777700 1 -- 172.21.15.47:0/2781021069 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v95) v1 ==== 54+0+16 (secure 0 0 0) 0x7fc340081220 con 0x7fc3440f7400 2024-02-23T00:42:02.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.454+0000 7fc348839700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc330062cc0 0x7fc330065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:02.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.454+0000 7fc34b29e700 10 client.24653.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:02.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.454+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_assign -1 1 2024-02-23T00:42:02.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.454+0000 7fc34b29e700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fc344100110 0x7fc344100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:02.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.454+0000 7fc34b29e700 20 client.24653.objecter _get_session s=0x7fc34405ee80 osd=0 3 2024-02-23T00:42:02.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.454+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc34405ee80 osd=0 3 2024-02-23T00:42:02.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.454+0000 7fc34b29e700 20 client.24653.objecter _get_session s=0x7fc34405ee80 osd=0 3 2024-02-23T00:42:02.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.454+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_remove -1 1 2024-02-23T00:42:02.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.454+0000 7fc34b29e700 20 client.24653.objecter get_session s=0x7fc34405ee80 osd=0 3 2024-02-23T00:42:02.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.454+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_assign 0 1 2024-02-23T00:42:02.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.454+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc34405ee80 osd=0 4 2024-02-23T00:42:02.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.454+0000 7fc34b29e700 10 client.24653.objecter _send_command 1 2024-02-23T00:42:02.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.454+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc344105ef0 con 0x7fc344100110 2024-02-23T00:42:02.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.455+0000 7fc34983b700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fc344100110 0x7fc344100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:02.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.455+0000 7fc33a777700 10 client.24653.objecter ms_handle_connect 0x7fc330062cc0 2024-02-23T00:42:02.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.455+0000 7fc348839700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc330062cc0 0x7fc330065180 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fc3440ff350 tx=0x7fc334009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:02.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.455+0000 7fc33a777700 10 client.24653.objecter ms_handle_connect 0x7fc344100110 2024-02-23T00:42:02.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.455+0000 7fc34983b700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fc344100110 0x7fc344100530 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:02.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.456+0000 7fc33a777700 1 -- 172.21.15.47:0/2781021069 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc344105ef0 con 0x7fc344100110 2024-02-23T00:42:02.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.456+0000 7fc33a777700 10 client.24653.objecter ms_dispatch 0x7fc344001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:42:02.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.456+0000 7fc33a777700 10 client.24653.objecter _finish_command 1 = system:0 2024-02-23T00:42:02.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.456+0000 7fc33a777700 20 client.24653.objecter put_session s=0x7fc34405ee80 osd=0 4 2024-02-23T00:42:02.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.456+0000 7fc33a777700 15 client.24653.objecter _session_command_op_remove 0 1 2024-02-23T00:42:02.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.473+0000 7fc34b29e700 10 client.24653.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:42:02.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.473+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_assign -1 2 2024-02-23T00:42:02.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.473+0000 7fc34b29e700 20 client.24653.objecter _get_session s=0x7fc34405ee80 osd=0 3 2024-02-23T00:42:02.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.473+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc34405ee80 osd=0 3 2024-02-23T00:42:02.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.473+0000 7fc34b29e700 20 client.24653.objecter _get_session s=0x7fc34405ee80 osd=0 3 2024-02-23T00:42:02.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.473+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_remove -1 2 2024-02-23T00:42:02.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.473+0000 7fc34b29e700 20 client.24653.objecter get_session s=0x7fc34405ee80 osd=0 3 2024-02-23T00:42:02.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.473+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_assign 0 2 2024-02-23T00:42:02.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.473+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc34405ee80 osd=0 4 2024-02-23T00:42:02.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.473+0000 7fc34b29e700 10 client.24653.objecter _send_command 2 2024-02-23T00:42:02.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.473+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fc3440273c0 con 0x7fc344100110 2024-02-23T00:42:02.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.475+0000 7fc33a777700 1 -- 172.21.15.47:0/2781021069 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fc3440273c0 con 0x7fc344100110 2024-02-23T00:42:02.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.475+0000 7fc33a777700 10 client.24653.objecter ms_dispatch 0x7fc344001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:42:02.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.475+0000 7fc33a777700 10 client.24653.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:42:02.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.475+0000 7fc33a777700 20 client.24653.objecter put_session s=0x7fc34405ee80 osd=0 4 2024-02-23T00:42:02.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.475+0000 7fc33a777700 15 client.24653.objecter _session_command_op_remove 0 2 2024-02-23T00:42:02.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.475+0000 7fc34b29e700 10 client.24653.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:02.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.475+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_assign -1 3 2024-02-23T00:42:02.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.475+0000 7fc34b29e700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fc3440598f0 0x7fc344059d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:02.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.475+0000 7fc34b29e700 20 client.24653.objecter _get_session s=0x7fc344028710 osd=1 3 2024-02-23T00:42:02.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.475+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc344028710 osd=1 3 2024-02-23T00:42:02.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.475+0000 7fc34b29e700 20 client.24653.objecter _get_session s=0x7fc344028710 osd=1 3 2024-02-23T00:42:02.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.475+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_remove -1 3 2024-02-23T00:42:02.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.475+0000 7fc34b29e700 20 client.24653.objecter get_session s=0x7fc344028710 osd=1 3 2024-02-23T00:42:02.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.475+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_assign 1 3 2024-02-23T00:42:02.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.475+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc344028710 osd=1 4 2024-02-23T00:42:02.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.475+0000 7fc34b29e700 10 client.24653.objecter _send_command 3 2024-02-23T00:42:02.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.475+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc34411d910 con 0x7fc3440598f0 2024-02-23T00:42:02.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.475+0000 7fc34903a700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fc3440598f0 0x7fc344059d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:02.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.476+0000 7fc33a777700 10 client.24653.objecter ms_handle_connect 0x7fc3440598f0 2024-02-23T00:42:02.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.476+0000 7fc34903a700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fc3440598f0 0x7fc344059d10 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:02.700 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:42:02.701 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:42:02.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.477+0000 7fc33a777700 1 -- 172.21.15.47:0/2781021069 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc34411d910 con 0x7fc3440598f0 2024-02-23T00:42:02.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.477+0000 7fc33a777700 10 client.24653.objecter ms_dispatch 0x7fc344001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:42:02.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.477+0000 7fc33a777700 10 client.24653.objecter _finish_command 3 = system:0 2024-02-23T00:42:02.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.477+0000 7fc33a777700 20 client.24653.objecter put_session s=0x7fc344028710 osd=1 4 2024-02-23T00:42:02.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.477+0000 7fc33a777700 15 client.24653.objecter _session_command_op_remove 1 3 2024-02-23T00:42:02.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.496+0000 7fc34b29e700 10 client.24653.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:42:02.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.496+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_assign -1 4 2024-02-23T00:42:02.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.496+0000 7fc34b29e700 20 client.24653.objecter _get_session s=0x7fc344028710 osd=1 3 2024-02-23T00:42:02.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.496+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc344028710 osd=1 3 2024-02-23T00:42:02.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.496+0000 7fc34b29e700 20 client.24653.objecter _get_session s=0x7fc344028710 osd=1 3 2024-02-23T00:42:02.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.496+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_remove -1 4 2024-02-23T00:42:02.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.496+0000 7fc34b29e700 20 client.24653.objecter get_session s=0x7fc344028710 osd=1 3 2024-02-23T00:42:02.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.496+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_assign 1 4 2024-02-23T00:42:02.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.496+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc344028710 osd=1 4 2024-02-23T00:42:02.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.496+0000 7fc34b29e700 10 client.24653.objecter _send_command 4 2024-02-23T00:42:02.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.496+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fc344059360 con 0x7fc3440598f0 2024-02-23T00:42:02.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.498+0000 7fc33a777700 1 -- 172.21.15.47:0/2781021069 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fc344059360 con 0x7fc3440598f0 2024-02-23T00:42:02.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.498+0000 7fc33a777700 10 client.24653.objecter ms_dispatch 0x7fc344001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:42:02.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.498+0000 7fc33a777700 10 client.24653.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:42:02.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.498+0000 7fc33a777700 20 client.24653.objecter put_session s=0x7fc344028710 osd=1 4 2024-02-23T00:42:02.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.498+0000 7fc33a777700 15 client.24653.objecter _session_command_op_remove 1 4 2024-02-23T00:42:02.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.499+0000 7fc34b29e700 10 client.24653.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:02.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.499+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_assign -1 5 2024-02-23T00:42:02.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.499+0000 7fc34b29e700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fc344028270 0x7fc34411c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:02.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.499+0000 7fc34b29e700 20 client.24653.objecter _get_session s=0x7fc344059770 osd=2 3 2024-02-23T00:42:02.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.499+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc344059770 osd=2 3 2024-02-23T00:42:02.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.499+0000 7fc34b29e700 20 client.24653.objecter _get_session s=0x7fc344059770 osd=2 3 2024-02-23T00:42:02.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.499+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_remove -1 5 2024-02-23T00:42:02.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.499+0000 7fc34b29e700 20 client.24653.objecter get_session s=0x7fc344059770 osd=2 3 2024-02-23T00:42:02.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.499+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_assign 2 5 2024-02-23T00:42:02.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.499+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc344059770 osd=2 4 2024-02-23T00:42:02.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.499+0000 7fc34b29e700 10 client.24653.objecter _send_command 5 2024-02-23T00:42:02.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.499+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc34411c820 con 0x7fc344028270 2024-02-23T00:42:02.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.500+0000 7fc348839700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fc344028270 0x7fc34411c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:02.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.500+0000 7fc33a777700 10 client.24653.objecter ms_handle_connect 0x7fc344028270 2024-02-23T00:42:02.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.500+0000 7fc348839700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fc344028270 0x7fc34411c330 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:02.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.501+0000 7fc33a777700 1 -- 172.21.15.47:0/2781021069 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc34411c820 con 0x7fc344028270 2024-02-23T00:42:02.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.501+0000 7fc33a777700 10 client.24653.objecter ms_dispatch 0x7fc344001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:42:02.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.501+0000 7fc33a777700 10 client.24653.objecter _finish_command 5 = system:0 2024-02-23T00:42:02.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.501+0000 7fc33a777700 20 client.24653.objecter put_session s=0x7fc344059770 osd=2 4 2024-02-23T00:42:02.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.501+0000 7fc33a777700 15 client.24653.objecter _session_command_op_remove 2 5 2024-02-23T00:42:02.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.517+0000 7fc34b29e700 10 client.24653.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:42:02.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.517+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_assign -1 6 2024-02-23T00:42:02.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.517+0000 7fc34b29e700 20 client.24653.objecter _get_session s=0x7fc344059770 osd=2 3 2024-02-23T00:42:02.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.517+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc344059770 osd=2 3 2024-02-23T00:42:02.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.517+0000 7fc34b29e700 20 client.24653.objecter _get_session s=0x7fc344059770 osd=2 3 2024-02-23T00:42:02.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.517+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_remove -1 6 2024-02-23T00:42:02.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.517+0000 7fc34b29e700 20 client.24653.objecter get_session s=0x7fc344059770 osd=2 3 2024-02-23T00:42:02.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.517+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_assign 2 6 2024-02-23T00:42:02.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.517+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc344059770 osd=2 4 2024-02-23T00:42:02.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.517+0000 7fc34b29e700 10 client.24653.objecter _send_command 6 2024-02-23T00:42:02.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.517+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fc344027830 con 0x7fc344028270 2024-02-23T00:42:02.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.518+0000 7fc33a777700 1 -- 172.21.15.47:0/2781021069 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fc344027830 con 0x7fc344028270 2024-02-23T00:42:02.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.518+0000 7fc33a777700 10 client.24653.objecter ms_dispatch 0x7fc344001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:42:02.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.518+0000 7fc33a777700 10 client.24653.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:42:02.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.518+0000 7fc33a777700 20 client.24653.objecter put_session s=0x7fc344059770 osd=2 4 2024-02-23T00:42:02.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.518+0000 7fc33a777700 15 client.24653.objecter _session_command_op_remove 2 6 2024-02-23T00:42:02.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.518+0000 7fc34b29e700 10 client.24653.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:02.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.518+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_assign -1 7 2024-02-23T00:42:02.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.518+0000 7fc34b29e700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fc34411fa90 0x7fc344121e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:02.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.518+0000 7fc34b29e700 20 client.24653.objecter _get_session s=0x7fc34411d190 osd=3 3 2024-02-23T00:42:02.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.518+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc34411d190 osd=3 3 2024-02-23T00:42:02.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.518+0000 7fc34b29e700 20 client.24653.objecter _get_session s=0x7fc34411d190 osd=3 3 2024-02-23T00:42:02.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.518+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_remove -1 7 2024-02-23T00:42:02.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.518+0000 7fc34b29e700 20 client.24653.objecter get_session s=0x7fc34411d190 osd=3 3 2024-02-23T00:42:02.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.518+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_assign 3 7 2024-02-23T00:42:02.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.518+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc34411d190 osd=3 4 2024-02-23T00:42:02.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.518+0000 7fc34b29e700 10 client.24653.objecter _send_command 7 2024-02-23T00:42:02.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.518+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc344124b20 con 0x7fc34411fa90 2024-02-23T00:42:02.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.519+0000 7fc34983b700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fc34411fa90 0x7fc344121e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:02.706 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:42:02.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.519+0000 7fc33a777700 10 client.24653.objecter ms_handle_connect 0x7fc34411fa90 2024-02-23T00:42:02.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.519+0000 7fc34983b700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fc34411fa90 0x7fc344121e80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:02.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.520+0000 7fc33a777700 1 -- 172.21.15.47:0/2781021069 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc344124b20 con 0x7fc34411fa90 2024-02-23T00:42:02.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.520+0000 7fc33a777700 10 client.24653.objecter ms_dispatch 0x7fc344001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:42:02.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.520+0000 7fc33a777700 10 client.24653.objecter _finish_command 7 = system:0 2024-02-23T00:42:02.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.520+0000 7fc33a777700 20 client.24653.objecter put_session s=0x7fc34411d190 osd=3 4 2024-02-23T00:42:02.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.520+0000 7fc33a777700 15 client.24653.objecter _session_command_op_remove 3 7 2024-02-23T00:42:02.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.535+0000 7fc34b29e700 10 client.24653.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:42:02.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.535+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_assign -1 8 2024-02-23T00:42:02.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.535+0000 7fc34b29e700 20 client.24653.objecter _get_session s=0x7fc34411d190 osd=3 3 2024-02-23T00:42:02.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.535+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc34411d190 osd=3 3 2024-02-23T00:42:02.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.535+0000 7fc34b29e700 20 client.24653.objecter _get_session s=0x7fc34411d190 osd=3 3 2024-02-23T00:42:02.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.535+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_remove -1 8 2024-02-23T00:42:02.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.535+0000 7fc34b29e700 20 client.24653.objecter get_session s=0x7fc34411d190 osd=3 3 2024-02-23T00:42:02.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.535+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_assign 3 8 2024-02-23T00:42:02.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.535+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc34411d190 osd=3 4 2024-02-23T00:42:02.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.535+0000 7fc34b29e700 10 client.24653.objecter _send_command 8 2024-02-23T00:42:02.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.535+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fc34411cc00 con 0x7fc34411fa90 2024-02-23T00:42:02.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.536+0000 7fc33a777700 1 -- 172.21.15.47:0/2781021069 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fc34411cc00 con 0x7fc34411fa90 2024-02-23T00:42:02.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.536+0000 7fc33a777700 10 client.24653.objecter ms_dispatch 0x7fc344001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:42:02.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.536+0000 7fc33a777700 10 client.24653.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:42:02.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.536+0000 7fc33a777700 20 client.24653.objecter put_session s=0x7fc34411d190 osd=3 4 2024-02-23T00:42:02.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.536+0000 7fc33a777700 15 client.24653.objecter _session_command_op_remove 3 8 2024-02-23T00:42:02.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.536+0000 7fc34b29e700 10 client.24653.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:02.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.536+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_assign -1 9 2024-02-23T00:42:02.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.536+0000 7fc34b29e700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fc34411be90 0x7fc3441234f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:02.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.536+0000 7fc34b29e700 20 client.24653.objecter _get_session s=0x7fc34411c9c0 osd=4 3 2024-02-23T00:42:02.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.536+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc34411c9c0 osd=4 3 2024-02-23T00:42:02.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.536+0000 7fc34b29e700 20 client.24653.objecter _get_session s=0x7fc34411c9c0 osd=4 3 2024-02-23T00:42:02.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.536+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_remove -1 9 2024-02-23T00:42:02.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.536+0000 7fc34b29e700 20 client.24653.objecter get_session s=0x7fc34411c9c0 osd=4 3 2024-02-23T00:42:02.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.536+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_assign 4 9 2024-02-23T00:42:02.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.536+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc34411c9c0 osd=4 4 2024-02-23T00:42:02.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.536+0000 7fc34b29e700 10 client.24653.objecter _send_command 9 2024-02-23T00:42:02.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.536+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc34411cbe0 con 0x7fc34411be90 2024-02-23T00:42:02.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.537+0000 7fc34903a700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fc34411be90 0x7fc3441234f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:02.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.537+0000 7fc33a777700 10 client.24653.objecter ms_handle_connect 0x7fc34411be90 2024-02-23T00:42:02.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.537+0000 7fc34903a700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fc34411be90 0x7fc3441234f0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:02.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.538+0000 7fc33a777700 1 -- 172.21.15.47:0/2781021069 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc34411cbe0 con 0x7fc34411be90 2024-02-23T00:42:02.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.538+0000 7fc33a777700 10 client.24653.objecter ms_dispatch 0x7fc344001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:42:02.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.538+0000 7fc33a777700 10 client.24653.objecter _finish_command 9 = system:0 2024-02-23T00:42:02.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.538+0000 7fc33a777700 20 client.24653.objecter put_session s=0x7fc34411c9c0 osd=4 4 2024-02-23T00:42:02.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.538+0000 7fc33a777700 15 client.24653.objecter _session_command_op_remove 4 9 2024-02-23T00:42:02.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.556+0000 7fc34b29e700 10 client.24653.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:42:02.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.556+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_assign -1 10 2024-02-23T00:42:02.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.556+0000 7fc34b29e700 20 client.24653.objecter _get_session s=0x7fc34411c9c0 osd=4 3 2024-02-23T00:42:02.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.556+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc34411c9c0 osd=4 3 2024-02-23T00:42:02.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.556+0000 7fc34b29e700 20 client.24653.objecter _get_session s=0x7fc34411c9c0 osd=4 3 2024-02-23T00:42:02.710 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:42:02.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.556+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_remove -1 10 2024-02-23T00:42:02.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.556+0000 7fc34b29e700 20 client.24653.objecter get_session s=0x7fc34411c9c0 osd=4 3 2024-02-23T00:42:02.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.556+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_assign 4 10 2024-02-23T00:42:02.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.556+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc34411c9c0 osd=4 4 2024-02-23T00:42:02.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.556+0000 7fc34b29e700 10 client.24653.objecter _send_command 10 2024-02-23T00:42:02.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.556+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fc34411cbe0 con 0x7fc34411be90 2024-02-23T00:42:02.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.558+0000 7fc33a777700 1 -- 172.21.15.47:0/2781021069 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fc34411cbe0 con 0x7fc34411be90 2024-02-23T00:42:02.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.558+0000 7fc33a777700 10 client.24653.objecter ms_dispatch 0x7fc344001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:42:02.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.558+0000 7fc33a777700 10 client.24653.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:42:02.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.558+0000 7fc33a777700 20 client.24653.objecter put_session s=0x7fc34411c9c0 osd=4 4 2024-02-23T00:42:02.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.558+0000 7fc33a777700 15 client.24653.objecter _session_command_op_remove 4 10 2024-02-23T00:42:02.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.558+0000 7fc34b29e700 10 client.24653.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:02.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.558+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_assign -1 11 2024-02-23T00:42:02.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.558+0000 7fc34b29e700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fc344123f70 0x7fc3441243b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:02.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.558+0000 7fc34b29e700 20 client.24653.objecter _get_session s=0x7fc34411cbe0 osd=5 3 2024-02-23T00:42:02.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.558+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc34411cbe0 osd=5 3 2024-02-23T00:42:02.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.558+0000 7fc34b29e700 20 client.24653.objecter _get_session s=0x7fc34411cbe0 osd=5 3 2024-02-23T00:42:02.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.558+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_remove -1 11 2024-02-23T00:42:02.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.558+0000 7fc34b29e700 20 client.24653.objecter get_session s=0x7fc34411cbe0 osd=5 3 2024-02-23T00:42:02.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.558+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_assign 5 11 2024-02-23T00:42:02.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.558+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc34411cbe0 osd=5 4 2024-02-23T00:42:02.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.558+0000 7fc34b29e700 10 client.24653.objecter _send_command 11 2024-02-23T00:42:02.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.558+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc34412b410 con 0x7fc344123f70 2024-02-23T00:42:02.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.558+0000 7fc348839700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fc344123f70 0x7fc3441243b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:02.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.559+0000 7fc33a777700 10 client.24653.objecter ms_handle_connect 0x7fc344123f70 2024-02-23T00:42:02.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.559+0000 7fc348839700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fc344123f70 0x7fc3441243b0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:02.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.559+0000 7fc33a777700 1 -- 172.21.15.47:0/2781021069 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc34412b410 con 0x7fc344123f70 2024-02-23T00:42:02.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.559+0000 7fc33a777700 10 client.24653.objecter ms_dispatch 0x7fc344001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:42:02.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.559+0000 7fc33a777700 10 client.24653.objecter _finish_command 11 = system:0 2024-02-23T00:42:02.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.559+0000 7fc33a777700 20 client.24653.objecter put_session s=0x7fc34411cbe0 osd=5 4 2024-02-23T00:42:02.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.559+0000 7fc33a777700 15 client.24653.objecter _session_command_op_remove 5 11 2024-02-23T00:42:02.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.589+0000 7fc34b29e700 10 client.24653.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:42:02.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.589+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_assign -1 12 2024-02-23T00:42:02.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.589+0000 7fc34b29e700 20 client.24653.objecter _get_session s=0x7fc34411cbe0 osd=5 3 2024-02-23T00:42:02.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.589+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc34411cbe0 osd=5 3 2024-02-23T00:42:02.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.589+0000 7fc34b29e700 20 client.24653.objecter _get_session s=0x7fc34411cbe0 osd=5 3 2024-02-23T00:42:02.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.589+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_remove -1 12 2024-02-23T00:42:02.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.589+0000 7fc34b29e700 20 client.24653.objecter get_session s=0x7fc34411cbe0 osd=5 3 2024-02-23T00:42:02.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.589+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_assign 5 12 2024-02-23T00:42:02.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.589+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc34411cbe0 osd=5 4 2024-02-23T00:42:02.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.589+0000 7fc34b29e700 10 client.24653.objecter _send_command 12 2024-02-23T00:42:02.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.589+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fc344128cb0 con 0x7fc344123f70 2024-02-23T00:42:02.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.591+0000 7fc33a777700 1 -- 172.21.15.47:0/2781021069 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fc344128cb0 con 0x7fc344123f70 2024-02-23T00:42:02.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.591+0000 7fc33a777700 10 client.24653.objecter ms_dispatch 0x7fc344001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:42:02.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.591+0000 7fc33a777700 10 client.24653.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:42:02.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.591+0000 7fc33a777700 20 client.24653.objecter put_session s=0x7fc34411cbe0 osd=5 4 2024-02-23T00:42:02.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.591+0000 7fc33a777700 15 client.24653.objecter _session_command_op_remove 5 12 2024-02-23T00:42:02.715 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:42:02.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.591+0000 7fc34b29e700 10 client.24653.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:02.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.591+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_assign -1 13 2024-02-23T00:42:02.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.591+0000 7fc34b29e700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fc344129330 0x7fc34412b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:02.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.591+0000 7fc34b29e700 20 client.24653.objecter _get_session s=0x7fc344128cb0 osd=6 3 2024-02-23T00:42:02.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.591+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc344128cb0 osd=6 3 2024-02-23T00:42:02.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.591+0000 7fc34b29e700 20 client.24653.objecter _get_session s=0x7fc344128cb0 osd=6 3 2024-02-23T00:42:02.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.591+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_remove -1 13 2024-02-23T00:42:02.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.591+0000 7fc34b29e700 20 client.24653.objecter get_session s=0x7fc344128cb0 osd=6 3 2024-02-23T00:42:02.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.591+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_assign 6 13 2024-02-23T00:42:02.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.591+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc344128cb0 osd=6 4 2024-02-23T00:42:02.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.591+0000 7fc34b29e700 10 client.24653.objecter _send_command 13 2024-02-23T00:42:02.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.591+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc34412e560 con 0x7fc344129330 2024-02-23T00:42:02.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.592+0000 7fc34983b700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fc344129330 0x7fc34412b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:02.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.592+0000 7fc33a777700 10 client.24653.objecter ms_handle_connect 0x7fc344129330 2024-02-23T00:42:02.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.592+0000 7fc34983b700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fc344129330 0x7fc34412b720 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:02.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.593+0000 7fc33a777700 1 -- 172.21.15.47:0/2781021069 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc34412e560 con 0x7fc344129330 2024-02-23T00:42:02.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.593+0000 7fc33a777700 10 client.24653.objecter ms_dispatch 0x7fc344001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:42:02.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.593+0000 7fc33a777700 10 client.24653.objecter _finish_command 13 = system:0 2024-02-23T00:42:02.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.593+0000 7fc33a777700 20 client.24653.objecter put_session s=0x7fc344128cb0 osd=6 4 2024-02-23T00:42:02.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.593+0000 7fc33a777700 15 client.24653.objecter _session_command_op_remove 6 13 2024-02-23T00:42:02.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.622+0000 7fc34b29e700 10 client.24653.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:42:02.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.622+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_assign -1 14 2024-02-23T00:42:02.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.622+0000 7fc34b29e700 20 client.24653.objecter _get_session s=0x7fc344128cb0 osd=6 3 2024-02-23T00:42:02.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.622+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc344128cb0 osd=6 3 2024-02-23T00:42:02.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.622+0000 7fc34b29e700 20 client.24653.objecter _get_session s=0x7fc344128cb0 osd=6 3 2024-02-23T00:42:02.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.622+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_remove -1 14 2024-02-23T00:42:02.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.622+0000 7fc34b29e700 20 client.24653.objecter get_session s=0x7fc344128cb0 osd=6 3 2024-02-23T00:42:02.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.622+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_assign 6 14 2024-02-23T00:42:02.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.622+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc344128cb0 osd=6 4 2024-02-23T00:42:02.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.622+0000 7fc34b29e700 10 client.24653.objecter _send_command 14 2024-02-23T00:42:02.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.622+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fc34412e560 con 0x7fc344129330 2024-02-23T00:42:02.718 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:42:02.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.624+0000 7fc33a777700 1 -- 172.21.15.47:0/2781021069 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fc34412e560 con 0x7fc344129330 2024-02-23T00:42:02.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.624+0000 7fc33a777700 10 client.24653.objecter ms_dispatch 0x7fc344001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:42:02.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.624+0000 7fc33a777700 10 client.24653.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:42:02.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.624+0000 7fc33a777700 20 client.24653.objecter put_session s=0x7fc344128cb0 osd=6 4 2024-02-23T00:42:02.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.624+0000 7fc33a777700 15 client.24653.objecter _session_command_op_remove 6 14 2024-02-23T00:42:02.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.624+0000 7fc34b29e700 10 client.24653.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:02.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.624+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_assign -1 15 2024-02-23T00:42:02.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.624+0000 7fc34b29e700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fc34412d890 0x7fc34412dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:02.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.624+0000 7fc34b29e700 20 client.24653.objecter _get_session s=0x7fc34412e560 osd=7 3 2024-02-23T00:42:02.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.624+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc34412e560 osd=7 3 2024-02-23T00:42:02.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.624+0000 7fc34b29e700 20 client.24653.objecter _get_session s=0x7fc34412e560 osd=7 3 2024-02-23T00:42:02.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.624+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_remove -1 15 2024-02-23T00:42:02.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.624+0000 7fc34b29e700 20 client.24653.objecter get_session s=0x7fc34412e560 osd=7 3 2024-02-23T00:42:02.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.624+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_assign 7 15 2024-02-23T00:42:02.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.624+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc34412e560 osd=7 4 2024-02-23T00:42:02.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.624+0000 7fc34b29e700 10 client.24653.objecter _send_command 15 2024-02-23T00:42:02.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.624+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc3441318f0 con 0x7fc34412d890 2024-02-23T00:42:02.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.625+0000 7fc34903a700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fc34412d890 0x7fc34412dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:02.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.627+0000 7fc33a777700 10 client.24653.objecter ms_handle_connect 0x7fc34412d890 2024-02-23T00:42:02.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.627+0000 7fc34903a700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fc34412d890 0x7fc34412dd10 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:02.720 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:42:02.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.628+0000 7fc33a777700 1 -- 172.21.15.47:0/2781021069 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc3441318f0 con 0x7fc34412d890 2024-02-23T00:42:02.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.628+0000 7fc33a777700 10 client.24653.objecter ms_dispatch 0x7fc344001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:42:02.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.628+0000 7fc33a777700 10 client.24653.objecter _finish_command 15 = system:0 2024-02-23T00:42:02.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.628+0000 7fc33a777700 20 client.24653.objecter put_session s=0x7fc34412e560 osd=7 4 2024-02-23T00:42:02.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.628+0000 7fc33a777700 15 client.24653.objecter _session_command_op_remove 7 15 2024-02-23T00:42:02.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.657+0000 7fc34b29e700 10 client.24653.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:42:02.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.657+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_assign -1 16 2024-02-23T00:42:02.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.657+0000 7fc34b29e700 20 client.24653.objecter _get_session s=0x7fc34412e560 osd=7 3 2024-02-23T00:42:02.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.657+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc34412e560 osd=7 3 2024-02-23T00:42:02.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.657+0000 7fc34b29e700 20 client.24653.objecter _get_session s=0x7fc34412e560 osd=7 3 2024-02-23T00:42:02.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.657+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_remove -1 16 2024-02-23T00:42:02.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.657+0000 7fc34b29e700 20 client.24653.objecter get_session s=0x7fc34412e560 osd=7 3 2024-02-23T00:42:02.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.657+0000 7fc34b29e700 15 client.24653.objecter _session_command_op_assign 7 16 2024-02-23T00:42:02.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.657+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc34412e560 osd=7 4 2024-02-23T00:42:02.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.657+0000 7fc34b29e700 10 client.24653.objecter _send_command 16 2024-02-23T00:42:02.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.657+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fc34412bd20 con 0x7fc34412d890 2024-02-23T00:42:02.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc33a777700 1 -- 172.21.15.47:0/2781021069 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fc34412bd20 con 0x7fc34412d890 2024-02-23T00:42:02.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc33a777700 10 client.24653.objecter ms_dispatch 0x7fc344001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:42:02.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc33a777700 10 client.24653.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:42:02.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc33a777700 20 client.24653.objecter put_session s=0x7fc34412e560 osd=7 4 2024-02-23T00:42:02.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc33a777700 15 client.24653.objecter _session_command_op_remove 7 16 2024-02-23T00:42:02.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 10 librados: watch_flush enter 2024-02-23T00:42:02.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 10 librados: watch_flush exit 2024-02-23T00:42:02.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 10 client.24653.objecter close_session for osd.0 2024-02-23T00:42:02.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fc344100110 msgr2=0x7fc344100530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:02.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fc344100110 0x7fc344100530 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:02.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc34405ee80 osd=0 1 2024-02-23T00:42:02.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 10 client.24653.objecter close_session for osd.1 2024-02-23T00:42:02.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fc3440598f0 msgr2=0x7fc344059d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:02.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fc3440598f0 0x7fc344059d10 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:02.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc344028710 osd=1 1 2024-02-23T00:42:02.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 10 client.24653.objecter close_session for osd.2 2024-02-23T00:42:02.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fc344028270 msgr2=0x7fc34411c330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:02.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fc344028270 0x7fc34411c330 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:02.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc344059770 osd=2 1 2024-02-23T00:42:02.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 10 client.24653.objecter close_session for osd.3 2024-02-23T00:42:02.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fc34411fa90 msgr2=0x7fc344121e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:02.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fc34411fa90 0x7fc344121e80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:02.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc34411d190 osd=3 1 2024-02-23T00:42:02.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 10 client.24653.objecter close_session for osd.4 2024-02-23T00:42:02.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fc34411be90 msgr2=0x7fc3441234f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:02.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fc34411be90 0x7fc3441234f0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:02.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc34411c9c0 osd=4 1 2024-02-23T00:42:02.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 10 client.24653.objecter close_session for osd.5 2024-02-23T00:42:02.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fc344123f70 msgr2=0x7fc3441243b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:02.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fc344123f70 0x7fc3441243b0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:02.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc34411cbe0 osd=5 1 2024-02-23T00:42:02.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 10 client.24653.objecter close_session for osd.6 2024-02-23T00:42:02.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fc344129330 msgr2=0x7fc34412b720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:02.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34983b700 1 -- 172.21.15.47:0/2781021069 reap_dead start 2024-02-23T00:42:02.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fc344129330 0x7fc34412b720 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:02.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34983b700 1 -- 172.21.15.47:0/2781021069 reap_dead start 2024-02-23T00:42:02.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34983b700 1 -- 172.21.15.47:0/2781021069 reap_dead start 2024-02-23T00:42:02.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc344128cb0 osd=6 1 2024-02-23T00:42:02.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 10 client.24653.objecter close_session for osd.7 2024-02-23T00:42:02.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fc34412d890 msgr2=0x7fc34412dd10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:02.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fc34412d890 0x7fc34412dd10 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:02.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 20 client.24653.objecter put_session s=0x7fc34412e560 osd=7 1 2024-02-23T00:42:02.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 20 client.24653.objecter shutdown clearing up homeless session... 2024-02-23T00:42:02.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 10 client.24653.objecter successfully canceled tick 2024-02-23T00:42:02.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc330062cc0 msgr2=0x7fc330065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:02.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc330062cc0 0x7fc330065180 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fc3440ff350 tx=0x7fc334009cb0).stop 2024-02-23T00:42:02.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc3440f7400 msgr2=0x7fc3440fdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:02.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc3440f7400 0x7fc3440fdbb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fc3400048f0 tx=0x7fc340009cb0).stop 2024-02-23T00:42:02.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 shutdown_connections 2024-02-23T00:42:02.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fc344129330 0x7fc34412b720 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:02.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc3440f7400 0x7fc3440fdbb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:02.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc330062cc0 0x7fc330065180 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:02.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 1 --2- 172.21.15.47:0/2781021069 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fc34412d890 0x7fc34412dd10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:02.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 >> 172.21.15.47:0/2781021069 conn(0x7fc3440f2820 msgr2=0x7fc3440f3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:42:02.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 shutdown_connections 2024-02-23T00:42:02.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.659+0000 7fc34b29e700 1 -- 172.21.15.47:0/2781021069 wait complete. 2024-02-23T00:42:02.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:02.660+0000 7fc34b29e700 1 librados: shutdown 2024-02-23T00:42:02.726 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:42:02.726 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:42:02.726 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:42:02.727 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:42:02.727 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:42:02.727 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:42:02.727 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:42:02.727 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:42:02.727 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:02.943 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:42:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:42:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:42:03.306 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:02.238408+0000 mgr.y (mgr.14517) 112 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:42:03.306 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:02.454469+0000 mon.c (mon.2) 24 : audit [DBG] from='client.? 172.21.15.47:0/2781021069' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:42:03.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:02.238408+0000 mgr.y (mgr.14517) 112 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:42:03.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:02.454469+0000 mon.c (mon.2) 24 : audit [DBG] from='client.? 172.21.15.47:0/2781021069' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:42:03.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:02.238408+0000 mgr.y (mgr.14517) 112 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:42:03.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:02.454469+0000 mon.c (mon.2) 24 : audit [DBG] from='client.? 172.21.15.47:0/2781021069' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:42:04.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:02.692763+0000 osd.5 (osd.5) 53 : cluster [DBG] 2.12 scrub starts 2024-02-23T00:42:04.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:02.694017+0000 osd.5 (osd.5) 54 : cluster [DBG] 2.12 scrub ok 2024-02-23T00:42:04.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:02.962468+0000 osd.7 (osd.7) 57 : cluster [DBG] 5.10 scrub starts 2024-02-23T00:42:04.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:02.965450+0000 osd.7 (osd.7) 58 : cluster [DBG] 5.10 scrub ok 2024-02-23T00:42:04.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:03.157502+0000 mgr.y (mgr.14517) 113 : cluster [DBG] pgmap v96: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:42:04.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:03.290224+0000 osd.2 (osd.2) 47 : cluster [DBG] 5.b scrub starts 2024-02-23T00:42:04.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:03.291427+0000 osd.2 (osd.2) 48 : cluster [DBG] 5.b scrub ok 2024-02-23T00:42:04.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:02.692763+0000 osd.5 (osd.5) 53 : cluster [DBG] 2.12 scrub starts 2024-02-23T00:42:04.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:02.694017+0000 osd.5 (osd.5) 54 : cluster [DBG] 2.12 scrub ok 2024-02-23T00:42:04.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:02.962468+0000 osd.7 (osd.7) 57 : cluster [DBG] 5.10 scrub starts 2024-02-23T00:42:04.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:02.965450+0000 osd.7 (osd.7) 58 : cluster [DBG] 5.10 scrub ok 2024-02-23T00:42:04.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:03.157502+0000 mgr.y (mgr.14517) 113 : cluster [DBG] pgmap v96: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:42:04.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:03.290224+0000 osd.2 (osd.2) 47 : cluster [DBG] 5.b scrub starts 2024-02-23T00:42:04.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:03.291427+0000 osd.2 (osd.2) 48 : cluster [DBG] 5.b scrub ok 2024-02-23T00:42:04.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:02.692763+0000 osd.5 (osd.5) 53 : cluster [DBG] 2.12 scrub starts 2024-02-23T00:42:04.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:02.694017+0000 osd.5 (osd.5) 54 : cluster [DBG] 2.12 scrub ok 2024-02-23T00:42:04.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:02.962468+0000 osd.7 (osd.7) 57 : cluster [DBG] 5.10 scrub starts 2024-02-23T00:42:04.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:02.965450+0000 osd.7 (osd.7) 58 : cluster [DBG] 5.10 scrub ok 2024-02-23T00:42:04.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:03.157502+0000 mgr.y (mgr.14517) 113 : cluster [DBG] pgmap v96: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:42:04.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:03.290224+0000 osd.2 (osd.2) 47 : cluster [DBG] 5.b scrub starts 2024-02-23T00:42:04.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:03.291427+0000 osd.2 (osd.2) 48 : cluster [DBG] 5.b scrub ok 2024-02-23T00:42:05.042 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:42:05.045 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:05.046 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:42:05.146 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:42:05.179 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:05.183 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:04.190018+0000 osd.6 (osd.6) 45 : cluster [DBG] 2.7 deep-scrub starts 2024-02-23T00:42:05.183 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:04.190930+0000 osd.6 (osd.6) 46 : cluster [DBG] 2.7 deep-scrub ok 2024-02-23T00:42:05.183 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:04.190018+0000 osd.6 (osd.6) 45 : cluster [DBG] 2.7 deep-scrub starts 2024-02-23T00:42:05.183 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:04.190930+0000 osd.6 (osd.6) 46 : cluster [DBG] 2.7 deep-scrub ok 2024-02-23T00:42:05.200 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:42:05.203 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:05.203 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:42:05.304 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:42:05.336 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:05.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:04.190018+0000 osd.6 (osd.6) 45 : cluster [DBG] 2.7 deep-scrub starts 2024-02-23T00:42:05.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:04.190930+0000 osd.6 (osd.6) 46 : cluster [DBG] 2.7 deep-scrub ok 2024-02-23T00:42:06.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.275+0000 7fee26f32700 1 -- 172.21.15.47:0/3264962578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee201006b0 msgr2=0x7fee200f9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:06.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.275+0000 7fee26f32700 1 --2- 172.21.15.47:0/3264962578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee201006b0 0x7fee200f9f70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fee0c004870 tx=0x7fee0c009cb0).stop 2024-02-23T00:42:06.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.276+0000 7fee26f32700 1 -- 172.21.15.47:0/3264962578 shutdown_connections 2024-02-23T00:42:06.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.276+0000 7fee26f32700 1 --2- 172.21.15.47:0/3264962578 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fee200fa540 0x7fee200fa9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.276+0000 7fee26f32700 1 --2- 172.21.15.47:0/3264962578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee200ffde0 0x7fee201001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.276+0000 7fee26f32700 1 --2- 172.21.15.47:0/3264962578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee201006b0 0x7fee200f9f70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.276+0000 7fee26f32700 1 -- 172.21.15.47:0/3264962578 >> 172.21.15.47:0/3264962578 conn(0x7fee200f59f0 msgr2=0x7fee200f7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:42:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.276+0000 7fee26f32700 1 -- 172.21.15.47:0/3264962578 shutdown_connections 2024-02-23T00:42:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.276+0000 7fee26f32700 1 -- 172.21.15.47:0/3264962578 wait complete. 2024-02-23T00:42:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.276+0000 7fee26f32700 1 librados: starting msgr at 2024-02-23T00:42:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.276+0000 7fee26f32700 1 librados: starting objecter 2024-02-23T00:42:06.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.277+0000 7fee26f32700 1 Processor -- start 2024-02-23T00:42:06.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.277+0000 7fee26f32700 1 -- start start 2024-02-23T00:42:06.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.277+0000 7fee26f32700 1 librados: setting wanted keys 2024-02-23T00:42:06.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.277+0000 7fee26f32700 1 librados: calling monclient init 2024-02-23T00:42:06.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.277+0000 7fee26f32700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee200fa540 0x7fee20110110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:06.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.277+0000 7fee26f32700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee200ffde0 0x7fee20110600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:06.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.277+0000 7fee26f32700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fee201006b0 0x7fee2010a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:06.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.277+0000 7fee26f32700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2010a7e0 con 0x7fee200fa540 2024-02-23T00:42:06.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.277+0000 7fee26f32700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2010a950 con 0x7fee200ffde0 2024-02-23T00:42:06.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.277+0000 7fee26f32700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2010aac0 con 0x7fee201006b0 2024-02-23T00:42:06.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.277+0000 7fee24cce700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee200fa540 0x7fee20110110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:06.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.277+0000 7fee24cce700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee200fa540 0x7fee20110110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52912/0 (socket says 172.21.15.47:52912) 2024-02-23T00:42:06.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.277+0000 7fee24cce700 1 -- 172.21.15.47:0/1368274456 learned_addr learned my addr 172.21.15.47:0/1368274456 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:42:06.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.277+0000 7fee254cf700 1 --2- 172.21.15.47:0/1368274456 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fee201006b0 0x7fee2010a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:06.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.277+0000 7fee1ffff700 1 --2- 172.21.15.47:0/1368274456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee200ffde0 0x7fee20110600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:06.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.277+0000 7fee24cce700 1 -- 172.21.15.47:0/1368274456 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fee201006b0 msgr2=0x7fee2010a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:06.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.277+0000 7fee24cce700 1 --2- 172.21.15.47:0/1368274456 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fee201006b0 0x7fee2010a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:06.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.277+0000 7fee24cce700 1 -- 172.21.15.47:0/1368274456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee200ffde0 msgr2=0x7fee20110600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:06.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.277+0000 7fee24cce700 1 --2- 172.21.15.47:0/1368274456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee200ffde0 0x7fee20110600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:06.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.277+0000 7fee24cce700 1 -- 172.21.15.47:0/1368274456 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee180097c0 con 0x7fee200fa540 2024-02-23T00:42:06.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.278+0000 7fee24cce700 1 --2- 172.21.15.47:0/1368274456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee200fa540 0x7fee20110110 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7fee140041d0 tx=0x7fee14009f60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:06.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.278+0000 7fee1dffb700 10 client.?.objecter ms_handle_connect 0x7fee200fa540 2024-02-23T00:42:06.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.278+0000 7fee1dffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:42:06.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.278+0000 7fee1dffb700 1 -- 172.21.15.47:0/1368274456 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee14007b40 con 0x7fee200fa540 2024-02-23T00:42:06.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.278+0000 7fee1dffb700 1 -- 172.21.15.47:0/1368274456 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fee1400f040 con 0x7fee200fa540 2024-02-23T00:42:06.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.278+0000 7fee26f32700 1 -- 172.21.15.47:0/1368274456 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee0c005040 con 0x7fee200fa540 2024-02-23T00:42:06.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.278+0000 7fee1dffb700 1 -- 172.21.15.47:0/1368274456 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee14014780 con 0x7fee200fa540 2024-02-23T00:42:06.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.278+0000 7fee26f32700 10 client.14769.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:42:06.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.278+0000 7fee26f32700 1 -- 172.21.15.47:0/1368274456 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee2010b150 con 0x7fee200fa540 2024-02-23T00:42:06.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.278+0000 7fee26f32700 1 librados: init done 2024-02-23T00:42:06.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.279+0000 7fee26f32700 1 -- 172.21.15.47:0/1368274456 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee201090f0 con 0x7fee200fa540 2024-02-23T00:42:06.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.279+0000 7fee1dffb700 1 -- 172.21.15.47:0/1368274456 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fee14014b30 con 0x7fee200fa540 2024-02-23T00:42:06.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.280+0000 7fee1dffb700 1 --2- 172.21.15.47:0/1368274456 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fee10062cc0 0x7fee10065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:06.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.280+0000 7fee1dffb700 1 -- 172.21.15.47:0/1368274456 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 38..95) v4 ==== 7421+0+0 (secure 0 0 0) 0x7fee1407cb60 con 0x7fee200fa540 2024-02-23T00:42:06.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.280+0000 7fee1dffb700 10 client.14769.objecter ms_dispatch 0x7fee20001550 osd_map(95..95 src has 38..95) v4 2024-02-23T00:42:06.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.280+0000 7fee1dffb700 3 client.14769.objecter handle_osd_map got epochs [95,95] > 0 2024-02-23T00:42:06.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.280+0000 7fee1dffb700 3 client.14769.objecter handle_osd_map decoding full epoch 95 2024-02-23T00:42:06.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.280+0000 7fee1ffff700 1 --2- 172.21.15.47:0/1368274456 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fee10062cc0 0x7fee10065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:06.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.280+0000 7fee1dffb700 20 client.14769.objecter dump_active .. 0 homeless 2024-02-23T00:42:06.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.280+0000 7fee1dffb700 10 client.14769.objecter ms_handle_connect 0x7fee10062cc0 2024-02-23T00:42:06.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.280+0000 7fee1ffff700 1 --2- 172.21.15.47:0/1368274456 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fee10062cc0 0x7fee10065180 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fee0c009a00 tx=0x7fee0c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:06.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.284+0000 7fee1dffb700 1 -- 172.21.15.47:0/1368274456 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fee1404ca10 con 0x7fee200fa540 2024-02-23T00:42:06.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:04.329051+0000 osd.2 (osd.2) 49 : cluster [DBG] 2.e deep-scrub starts 2024-02-23T00:42:06.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:04.330420+0000 osd.2 (osd.2) 50 : cluster [DBG] 2.e deep-scrub ok 2024-02-23T00:42:06.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:04.622765+0000 osd.1 (osd.1) 61 : cluster [DBG] 4.7 scrub starts 2024-02-23T00:42:06.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:04.624998+0000 osd.1 (osd.1) 62 : cluster [DBG] 4.7 scrub ok 2024-02-23T00:42:06.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:04.707370+0000 osd.5 (osd.5) 55 : cluster [DBG] 2.2 scrub starts 2024-02-23T00:42:06.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:04.708276+0000 osd.5 (osd.5) 56 : cluster [DBG] 2.2 scrub ok 2024-02-23T00:42:06.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:05.157878+0000 mgr.y (mgr.14517) 114 : cluster [DBG] pgmap v97: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:42:06.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:04.329051+0000 osd.2 (osd.2) 49 : cluster [DBG] 2.e deep-scrub starts 2024-02-23T00:42:06.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:04.330420+0000 osd.2 (osd.2) 50 : cluster [DBG] 2.e deep-scrub ok 2024-02-23T00:42:06.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:04.622765+0000 osd.1 (osd.1) 61 : cluster [DBG] 4.7 scrub starts 2024-02-23T00:42:06.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:04.624998+0000 osd.1 (osd.1) 62 : cluster [DBG] 4.7 scrub ok 2024-02-23T00:42:06.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:04.707370+0000 osd.5 (osd.5) 55 : cluster [DBG] 2.2 scrub starts 2024-02-23T00:42:06.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:04.708276+0000 osd.5 (osd.5) 56 : cluster [DBG] 2.2 scrub ok 2024-02-23T00:42:06.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:05.157878+0000 mgr.y (mgr.14517) 114 : cluster [DBG] pgmap v97: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:42:06.317 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:42:06.320 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:06.321 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:42:06.422 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:42:06.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:04.329051+0000 osd.2 (osd.2) 49 : cluster [DBG] 2.e deep-scrub starts 2024-02-23T00:42:06.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:04.330420+0000 osd.2 (osd.2) 50 : cluster [DBG] 2.e deep-scrub ok 2024-02-23T00:42:06.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:04.622765+0000 osd.1 (osd.1) 61 : cluster [DBG] 4.7 scrub starts 2024-02-23T00:42:06.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:04.624998+0000 osd.1 (osd.1) 62 : cluster [DBG] 4.7 scrub ok 2024-02-23T00:42:06.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:04.707370+0000 osd.5 (osd.5) 55 : cluster [DBG] 2.2 scrub starts 2024-02-23T00:42:06.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:04.708276+0000 osd.5 (osd.5) 56 : cluster [DBG] 2.2 scrub ok 2024-02-23T00:42:06.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:05.157878+0000 mgr.y (mgr.14517) 114 : cluster [DBG] pgmap v97: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:42:06.451 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:06.482 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:42:06.486 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:06.487 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:42:06.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.485+0000 7fee26f32700 1 -- 172.21.15.47:0/1368274456 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd reweight", "id": 4, "weight": 0.9988540557681002} v 0) v1 -- 0x7fee2005fa60 con 0x7fee200fa540 2024-02-23T00:42:06.588 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:42:06.620 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:06.642 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:42:06.644 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:06.645 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:42:06.696 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:42:06.697 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:42:06.746 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:42:06.774 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:06.972 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:42:06.975 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:06.975 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:42:06.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.977+0000 7fee1dffb700 1 -- 172.21.15.47:0/1368274456 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 4, "weight": 0.9988540557681002}]=0 reweighted osd.4 to 0.998854 (ffb4) v96) v1 ==== 134+0+0 (secure 0 0 0) 0x7fee14050030 con 0x7fee200fa540 2024-02-23T00:42:06.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.979+0000 7fee26f32700 10 librados: watch_flush enter 2024-02-23T00:42:06.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.979+0000 7fee26f32700 10 librados: watch_flush exit 2024-02-23T00:42:06.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.979+0000 7fee26f32700 20 client.14769.objecter shutdown clearing up homeless session... 2024-02-23T00:42:06.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.979+0000 7fee26f32700 10 client.14769.objecter successfully canceled tick 2024-02-23T00:42:06.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.979+0000 7fee26f32700 1 -- 172.21.15.47:0/1368274456 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fee10062cc0 msgr2=0x7fee10065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:06.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.979+0000 7fee26f32700 1 --2- 172.21.15.47:0/1368274456 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fee10062cc0 0x7fee10065180 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fee0c009a00 tx=0x7fee0c00b040).stop 2024-02-23T00:42:06.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.979+0000 7fee26f32700 1 -- 172.21.15.47:0/1368274456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee200fa540 msgr2=0x7fee20110110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:06.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.979+0000 7fee26f32700 1 --2- 172.21.15.47:0/1368274456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee200fa540 0x7fee20110110 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7fee140041d0 tx=0x7fee14009f60).stop 2024-02-23T00:42:06.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.979+0000 7fee26f32700 1 -- 172.21.15.47:0/1368274456 shutdown_connections 2024-02-23T00:42:06.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.979+0000 7fee26f32700 1 --2- 172.21.15.47:0/1368274456 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fee10062cc0 0x7fee10065180 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:06.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.979+0000 7fee26f32700 1 --2- 172.21.15.47:0/1368274456 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fee201006b0 0x7fee2010a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:06.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.979+0000 7fee26f32700 1 --2- 172.21.15.47:0/1368274456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee200fa540 0x7fee20110110 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:06.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.979+0000 7fee26f32700 1 --2- 172.21.15.47:0/1368274456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee200ffde0 0x7fee20110600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:06.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.980+0000 7fee26f32700 1 -- 172.21.15.47:0/1368274456 >> 172.21.15.47:0/1368274456 conn(0x7fee200f59f0 msgr2=0x7fee200fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:42:06.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.980+0000 7fee26f32700 1 -- 172.21.15.47:0/1368274456 shutdown_connections 2024-02-23T00:42:06.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.980+0000 7fee26f32700 1 -- 172.21.15.47:0/1368274456 wait complete. 2024-02-23T00:42:06.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:06.980+0000 7fee26f32700 1 librados: shutdown 2024-02-23T00:42:06.983 INFO:teuthology.orchestra.run.smithi047.stderr:reweighted osd.4 to 0.998854 (ffb4) 2024-02-23T00:42:07.076 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:42:07.108 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:07.129 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:42:07.131 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:07.132 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:42:07.233 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:42:07.258 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:05.328718+0000 osd.2 (osd.2) 51 : cluster [DBG] 2.1 scrub starts 2024-02-23T00:42:07.258 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:05.329748+0000 osd.2 (osd.2) 52 : cluster [DBG] 2.1 scrub ok 2024-02-23T00:42:07.258 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:05.724271+0000 osd.5 (osd.5) 57 : cluster [DBG] 6.a scrub starts 2024-02-23T00:42:07.258 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:05.725791+0000 osd.5 (osd.5) 58 : cluster [DBG] 6.a scrub ok 2024-02-23T00:42:07.258 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:06.485960+0000 mon.a (mon.0) 765 : audit [INF] from='client.? 172.21.15.47:0/1368274456' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 0.9988540557681002}]: dispatch 2024-02-23T00:42:07.258 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:05.328718+0000 osd.2 (osd.2) 51 : cluster [DBG] 2.1 scrub starts 2024-02-23T00:42:07.259 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:05.329748+0000 osd.2 (osd.2) 52 : cluster [DBG] 2.1 scrub ok 2024-02-23T00:42:07.259 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:05.724271+0000 osd.5 (osd.5) 57 : cluster [DBG] 6.a scrub starts 2024-02-23T00:42:07.259 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:05.725791+0000 osd.5 (osd.5) 58 : cluster [DBG] 6.a scrub ok 2024-02-23T00:42:07.259 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:06.485960+0000 mon.a (mon.0) 765 : audit [INF] from='client.? 172.21.15.47:0/1368274456' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 0.9988540557681002}]: dispatch 2024-02-23T00:42:07.260 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:07.284 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:42:07.287 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:07.287 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:42:07.388 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:42:07.417 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:07.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:05.328718+0000 osd.2 (osd.2) 51 : cluster [DBG] 2.1 scrub starts 2024-02-23T00:42:07.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:05.329748+0000 osd.2 (osd.2) 52 : cluster [DBG] 2.1 scrub ok 2024-02-23T00:42:07.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:05.724271+0000 osd.5 (osd.5) 57 : cluster [DBG] 6.a scrub starts 2024-02-23T00:42:07.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:05.725791+0000 osd.5 (osd.5) 58 : cluster [DBG] 6.a scrub ok 2024-02-23T00:42:07.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:06.485960+0000 mon.a (mon.0) 765 : audit [INF] from='client.? 172.21.15.47:0/1368274456' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 0.9988540557681002}]: dispatch 2024-02-23T00:42:07.446 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:42:07.451 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:07.451 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:42:07.552 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:42:07.584 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:07.607 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:42:07.608 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:07.609 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:42:07.710 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:42:07.741 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:07.761 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:42:07.764 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:07.764 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:42:07.865 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:42:07.896 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:07.917 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:42:07.920 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:07.920 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:42:08.021 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:42:08.028 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:42:08.050 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:08.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:06.572222+0000 osd.1 (osd.1) 63 : cluster [DBG] 2.a deep-scrub starts 2024-02-23T00:42:08.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:06.575234+0000 osd.1 (osd.1) 64 : cluster [DBG] 2.a deep-scrub ok 2024-02-23T00:42:08.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:06.942697+0000 osd.7 (osd.7) 59 : cluster [DBG] 6.3 deep-scrub starts 2024-02-23T00:42:08.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:06.945634+0000 osd.7 (osd.7) 60 : cluster [DBG] 6.3 deep-scrub ok 2024-02-23T00:42:08.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:06.978202+0000 mon.a (mon.0) 766 : audit [INF] from='client.? 172.21.15.47:0/1368274456' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 0.9988540557681002}]': finished 2024-02-23T00:42:08.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:06.978273+0000 mon.a (mon.0) 767 : cluster [DBG] osdmap e96: 8 total, 8 up, 8 in 2024-02-23T00:42:08.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:07.158407+0000 mgr.y (mgr.14517) 115 : cluster [DBG] pgmap v99: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:42:08.306 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:06.572222+0000 osd.1 (osd.1) 63 : cluster [DBG] 2.a deep-scrub starts 2024-02-23T00:42:08.306 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:06.575234+0000 osd.1 (osd.1) 64 : cluster [DBG] 2.a deep-scrub ok 2024-02-23T00:42:08.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:06.942697+0000 osd.7 (osd.7) 59 : cluster [DBG] 6.3 deep-scrub starts 2024-02-23T00:42:08.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:06.945634+0000 osd.7 (osd.7) 60 : cluster [DBG] 6.3 deep-scrub ok 2024-02-23T00:42:08.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:06.978202+0000 mon.a (mon.0) 766 : audit [INF] from='client.? 172.21.15.47:0/1368274456' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 0.9988540557681002}]': finished 2024-02-23T00:42:08.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:06.978273+0000 mon.a (mon.0) 767 : cluster [DBG] osdmap e96: 8 total, 8 up, 8 in 2024-02-23T00:42:08.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:07.158407+0000 mgr.y (mgr.14517) 115 : cluster [DBG] pgmap v99: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:42:08.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:06.572222+0000 osd.1 (osd.1) 63 : cluster [DBG] 2.a deep-scrub starts 2024-02-23T00:42:08.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:06.575234+0000 osd.1 (osd.1) 64 : cluster [DBG] 2.a deep-scrub ok 2024-02-23T00:42:08.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:06.942697+0000 osd.7 (osd.7) 59 : cluster [DBG] 6.3 deep-scrub starts 2024-02-23T00:42:08.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:06.945634+0000 osd.7 (osd.7) 60 : cluster [DBG] 6.3 deep-scrub ok 2024-02-23T00:42:08.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:06.978202+0000 mon.a (mon.0) 766 : audit [INF] from='client.? 172.21.15.47:0/1368274456' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 0.9988540557681002}]': finished 2024-02-23T00:42:08.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:06.978273+0000 mon.a (mon.0) 767 : cluster [DBG] osdmap e96: 8 total, 8 up, 8 in 2024-02-23T00:42:08.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:07.158407+0000 mgr.y (mgr.14517) 115 : cluster [DBG] pgmap v99: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:42:08.919 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.0 dump_blocked_ops 2024-02-23T00:42:09.149 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:07.938197+0000 osd.4 (osd.4) 61 : cluster [DBG] 5.1c scrub starts 2024-02-23T00:42:09.149 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:07.939055+0000 osd.4 (osd.4) 62 : cluster [DBG] 5.1c scrub ok 2024-02-23T00:42:09.150 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:08.060667+0000 osd.3 (osd.3) 65 : cluster [DBG] 6.11 scrub starts 2024-02-23T00:42:09.150 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:08.062149+0000 osd.3 (osd.3) 66 : cluster [DBG] 6.11 scrub ok 2024-02-23T00:42:09.150 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:08.355050+0000 osd.0 (osd.0) 51 : cluster [DBG] 4.b scrub starts 2024-02-23T00:42:09.150 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:08.357015+0000 osd.0 (osd.0) 52 : cluster [DBG] 4.b scrub ok 2024-02-23T00:42:09.150 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:07.938197+0000 osd.4 (osd.4) 61 : cluster [DBG] 5.1c scrub starts 2024-02-23T00:42:09.150 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:07.939055+0000 osd.4 (osd.4) 62 : cluster [DBG] 5.1c scrub ok 2024-02-23T00:42:09.150 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:08.060667+0000 osd.3 (osd.3) 65 : cluster [DBG] 6.11 scrub starts 2024-02-23T00:42:09.150 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:08.062149+0000 osd.3 (osd.3) 66 : cluster [DBG] 6.11 scrub ok 2024-02-23T00:42:09.150 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:08.355050+0000 osd.0 (osd.0) 51 : cluster [DBG] 4.b scrub starts 2024-02-23T00:42:09.150 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:08.357015+0000 osd.0 (osd.0) 52 : cluster [DBG] 4.b scrub ok 2024-02-23T00:42:09.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.413+0000 7f37e8521700 1 -- 172.21.15.47:0/1044704004 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f37e00f6b20 msgr2=0x7f37e00f6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:09.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.413+0000 7f37e8521700 1 --2- 172.21.15.47:0/1044704004 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f37e00f6b20 0x7f37e00f6f00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f37dc004870 tx=0x7f37dc009cb0).stop 2024-02-23T00:42:09.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.413+0000 7f37e8521700 1 -- 172.21.15.47:0/1044704004 shutdown_connections 2024-02-23T00:42:09.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.413+0000 7f37e8521700 1 --2- 172.21.15.47:0/1044704004 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37e0104680 0x7f37e0106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:09.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.413+0000 7f37e8521700 1 --2- 172.21.15.47:0/1044704004 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f37e00f6b20 0x7f37e00f6f00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:09.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.413+0000 7f37e8521700 1 --2- 172.21.15.47:0/1044704004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37e00f73f0 0x7f37e0104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:09.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.413+0000 7f37e8521700 1 -- 172.21.15.47:0/1044704004 >> 172.21.15.47:0/1044704004 conn(0x7f37e00f2820 msgr2=0x7f37e00f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:42:09.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.413+0000 7f37e8521700 1 -- 172.21.15.47:0/1044704004 shutdown_connections 2024-02-23T00:42:09.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.413+0000 7f37e8521700 1 -- 172.21.15.47:0/1044704004 wait complete. 2024-02-23T00:42:09.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.414+0000 7f37e8521700 1 librados: starting msgr at 2024-02-23T00:42:09.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.414+0000 7f37e8521700 1 librados: starting objecter 2024-02-23T00:42:09.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.414+0000 7f37e8521700 1 Processor -- start 2024-02-23T00:42:09.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.414+0000 7f37e8521700 1 -- start start 2024-02-23T00:42:09.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.414+0000 7f37e8521700 1 librados: setting wanted keys 2024-02-23T00:42:09.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.414+0000 7f37e8521700 1 librados: calling monclient init 2024-02-23T00:42:09.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.414+0000 7f37e8521700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37e00f6b20 0x7f37e0110260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:09.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.414+0000 7f37e8521700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f37e00f73f0 0x7f37e0110750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:09.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.414+0000 7f37e8521700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37e0104680 0x7f37e010a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:09.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.414+0000 7f37e8521700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e010a8f0 con 0x7f37e00f6b20 2024-02-23T00:42:09.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.414+0000 7f37e8521700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e010aa60 con 0x7f37e0104680 2024-02-23T00:42:09.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.414+0000 7f37e8521700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e010ac20 con 0x7f37e00f73f0 2024-02-23T00:42:09.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.414+0000 7f37e62bd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37e00f6b20 0x7f37e0110260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:09.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.414+0000 7f37e62bd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37e00f6b20 0x7f37e0110260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48512/0 (socket says 172.21.15.47:48512) 2024-02-23T00:42:09.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.414+0000 7f37e62bd700 1 -- 172.21.15.47:0/2163008561 learned_addr learned my addr 172.21.15.47:0/2163008561 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:42:09.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.414+0000 7f37e5abc700 1 --2- 172.21.15.47:0/2163008561 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f37e00f73f0 0x7f37e0110750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:09.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.415+0000 7f37e62bd700 1 -- 172.21.15.47:0/2163008561 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f37e00f73f0 msgr2=0x7f37e0110750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:09.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.415+0000 7f37e62bd700 1 --2- 172.21.15.47:0/2163008561 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f37e00f73f0 0x7f37e0110750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:09.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.415+0000 7f37e62bd700 1 -- 172.21.15.47:0/2163008561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37e0104680 msgr2=0x7f37e010a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:09.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.415+0000 7f37e6abe700 1 --2- 172.21.15.47:0/2163008561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37e0104680 0x7f37e010a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:09.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.415+0000 7f37e62bd700 1 --2- 172.21.15.47:0/2163008561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37e0104680 0x7f37e010a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:09.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.415+0000 7f37e62bd700 1 -- 172.21.15.47:0/2163008561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d80097c0 con 0x7f37e00f6b20 2024-02-23T00:42:09.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.415+0000 7f37e6abe700 1 --2- 172.21.15.47:0/2163008561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37e0104680 0x7f37e010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-23T00:42:09.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.415+0000 7f37e62bd700 1 --2- 172.21.15.47:0/2163008561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37e00f6b20 0x7f37e0110260 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f37dc009da0 tx=0x7f37dc009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:09.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.415+0000 7f37d77fe700 10 client.?.objecter ms_handle_connect 0x7f37e00f6b20 2024-02-23T00:42:09.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.415+0000 7f37d77fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:42:09.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.415+0000 7f37d77fe700 1 -- 172.21.15.47:0/2163008561 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc00e030 con 0x7f37e00f6b20 2024-02-23T00:42:09.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.415+0000 7f37e8521700 1 -- 172.21.15.47:0/2163008561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37dc005040 con 0x7f37e00f6b20 2024-02-23T00:42:09.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.415+0000 7f37d77fe700 1 -- 172.21.15.47:0/2163008561 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f37dc013e70 con 0x7f37e00f6b20 2024-02-23T00:42:09.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.415+0000 7f37d77fe700 1 -- 172.21.15.47:0/2163008561 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc018070 con 0x7f37e00f6b20 2024-02-23T00:42:09.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.415+0000 7f37e8521700 10 client.14775.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:42:09.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.415+0000 7f37e8521700 1 -- 172.21.15.47:0/2163008561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37e010b0c0 con 0x7f37e00f6b20 2024-02-23T00:42:09.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.415+0000 7f37e8521700 1 librados: init done 2024-02-23T00:42:09.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.417+0000 7f37d77fe700 1 -- 172.21.15.47:0/2163008561 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f37dc009240 con 0x7f37e00f6b20 2024-02-23T00:42:09.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.417+0000 7f37e8521700 1 -- 172.21.15.47:0/2163008561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37e010b2b0 con 0x7f37e00f6b20 2024-02-23T00:42:09.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.417+0000 7f37d77fe700 1 --2- 172.21.15.47:0/2163008561 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f37cc062cc0 0x7f37cc065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:09.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.417+0000 7f37d77fe700 1 -- 172.21.15.47:0/2163008561 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 38..96) v4 ==== 7421+0+0 (secure 0 0 0) 0x7f37dc07e170 con 0x7f37e00f6b20 2024-02-23T00:42:09.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.417+0000 7f37d77fe700 10 client.14775.objecter ms_dispatch 0x7f37e0001550 osd_map(96..96 src has 38..96) v4 2024-02-23T00:42:09.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.417+0000 7f37d77fe700 3 client.14775.objecter handle_osd_map got epochs [96,96] > 0 2024-02-23T00:42:09.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.417+0000 7f37d77fe700 3 client.14775.objecter handle_osd_map decoding full epoch 96 2024-02-23T00:42:09.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.417+0000 7f37e5abc700 1 --2- 172.21.15.47:0/2163008561 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f37cc062cc0 0x7f37cc065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:09.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.417+0000 7f37d77fe700 20 client.14775.objecter dump_active .. 0 homeless 2024-02-23T00:42:09.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.417+0000 7f37d77fe700 10 client.14775.objecter ms_handle_connect 0x7f37cc062cc0 2024-02-23T00:42:09.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.417+0000 7f37e5abc700 1 --2- 172.21.15.47:0/2163008561 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f37cc062cc0 0x7f37cc065180 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f37e010bba0 tx=0x7f37d000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:09.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.422+0000 7f37d77fe700 1 -- 172.21.15.47:0/2163008561 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f37dc04ded0 con 0x7f37e00f6b20 2024-02-23T00:42:09.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:07.938197+0000 osd.4 (osd.4) 61 : cluster [DBG] 5.1c scrub starts 2024-02-23T00:42:09.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:07.939055+0000 osd.4 (osd.4) 62 : cluster [DBG] 5.1c scrub ok 2024-02-23T00:42:09.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:08.060667+0000 osd.3 (osd.3) 65 : cluster [DBG] 6.11 scrub starts 2024-02-23T00:42:09.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:08.062149+0000 osd.3 (osd.3) 66 : cluster [DBG] 6.11 scrub ok 2024-02-23T00:42:09.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:08.355050+0000 osd.0 (osd.0) 51 : cluster [DBG] 4.b scrub starts 2024-02-23T00:42:09.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:08.357015+0000 osd.0 (osd.0) 52 : cluster [DBG] 4.b scrub ok 2024-02-23T00:42:09.475 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:42:09.505 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:42:09.509 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:09.509 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:42:09.513 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:42:09.610 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:42:09.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.630+0000 7f37e8521700 1 -- 172.21.15.47:0/2163008561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7f37e005fa60 con 0x7f37e00f6b20 2024-02-23T00:42:09.642 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:09.663 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:42:09.666 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:09.666 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:42:09.767 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:42:09.796 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:09.816 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:42:09.819 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:09.819 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:42:09.920 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:42:09.950 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:09.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.994+0000 7f37d77fe700 1 -- 172.21.15.47:0/2163008561 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v97) v1 ==== 87+0+0 (secure 0 0 0) 0x7f37e005fa60 con 0x7f37e00f6b20 2024-02-23T00:42:09.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.996+0000 7f37e8521700 10 librados: watch_flush enter 2024-02-23T00:42:09.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.997+0000 7f37e8521700 10 librados: watch_flush exit 2024-02-23T00:42:09.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.997+0000 7f37e8521700 20 client.14775.objecter shutdown clearing up homeless session... 2024-02-23T00:42:09.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.997+0000 7f37e8521700 10 client.14775.objecter successfully canceled tick 2024-02-23T00:42:09.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.997+0000 7f37e8521700 1 -- 172.21.15.47:0/2163008561 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f37cc062cc0 msgr2=0x7f37cc065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:10.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.997+0000 7f37e8521700 1 --2- 172.21.15.47:0/2163008561 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f37cc062cc0 0x7f37cc065180 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f37e010bba0 tx=0x7f37d000b040).stop 2024-02-23T00:42:10.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.997+0000 7f37e8521700 1 -- 172.21.15.47:0/2163008561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37e00f6b20 msgr2=0x7f37e0110260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:10.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.997+0000 7f37e8521700 1 --2- 172.21.15.47:0/2163008561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37e00f6b20 0x7f37e0110260 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f37dc009da0 tx=0x7f37dc009cb0).stop 2024-02-23T00:42:10.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.997+0000 7f37e8521700 1 -- 172.21.15.47:0/2163008561 shutdown_connections 2024-02-23T00:42:10.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.997+0000 7f37e8521700 1 --2- 172.21.15.47:0/2163008561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37e0104680 0x7f37e010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:10.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.997+0000 7f37e8521700 1 --2- 172.21.15.47:0/2163008561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37e00f6b20 0x7f37e0110260 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:10.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.997+0000 7f37e8521700 1 --2- 172.21.15.47:0/2163008561 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f37cc062cc0 0x7f37cc065180 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:10.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.997+0000 7f37e8521700 1 --2- 172.21.15.47:0/2163008561 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f37e00f73f0 0x7f37e0110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:10.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.997+0000 7f37e8521700 1 -- 172.21.15.47:0/2163008561 >> 172.21.15.47:0/2163008561 conn(0x7f37e00f2820 msgr2=0x7f37e00f4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:42:10.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.997+0000 7f37e8521700 1 -- 172.21.15.47:0/2163008561 shutdown_connections 2024-02-23T00:42:10.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.997+0000 7f37e8521700 1 -- 172.21.15.47:0/2163008561 wait complete. 2024-02-23T00:42:10.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:09.997+0000 7f37e8521700 1 librados: shutdown 2024-02-23T00:42:10.001 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is set 2024-02-23T00:42:10.056 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:08.334369+0000 osd.2 (osd.2) 53 : cluster [DBG] 5.a deep-scrub starts 2024-02-23T00:42:10.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:08.335326+0000 osd.2 (osd.2) 54 : cluster [DBG] 5.a deep-scrub ok 2024-02-23T00:42:10.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:08.694323+0000 osd.5 (osd.5) 59 : cluster [DBG] 6.19 scrub starts 2024-02-23T00:42:10.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:08.695265+0000 osd.5 (osd.5) 60 : cluster [DBG] 6.19 scrub ok 2024-02-23T00:42:10.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:09.160377+0000 mgr.y (mgr.14517) 116 : cluster [DBG] pgmap v100: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:42:10.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:09.632154+0000 mon.a (mon.0) 768 : audit [INF] from='client.? 172.21.15.47:0/2163008561' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:42:10.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:08.334369+0000 osd.2 (osd.2) 53 : cluster [DBG] 5.a deep-scrub starts 2024-02-23T00:42:10.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:08.335326+0000 osd.2 (osd.2) 54 : cluster [DBG] 5.a deep-scrub ok 2024-02-23T00:42:10.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:08.694323+0000 osd.5 (osd.5) 59 : cluster [DBG] 6.19 scrub starts 2024-02-23T00:42:10.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:08.695265+0000 osd.5 (osd.5) 60 : cluster [DBG] 6.19 scrub ok 2024-02-23T00:42:10.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:09.160377+0000 mgr.y (mgr.14517) 116 : cluster [DBG] pgmap v100: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:42:10.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:09.632154+0000 mon.a (mon.0) 768 : audit [INF] from='client.? 172.21.15.47:0/2163008561' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:42:10.200 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:42:10.202 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:10.203 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:42:10.303 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:42:10.334 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:08.334369+0000 osd.2 (osd.2) 53 : cluster [DBG] 5.a deep-scrub starts 2024-02-23T00:42:10.334 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:08.335326+0000 osd.2 (osd.2) 54 : cluster [DBG] 5.a deep-scrub ok 2024-02-23T00:42:10.334 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:08.694323+0000 osd.5 (osd.5) 59 : cluster [DBG] 6.19 scrub starts 2024-02-23T00:42:10.335 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:08.695265+0000 osd.5 (osd.5) 60 : cluster [DBG] 6.19 scrub ok 2024-02-23T00:42:10.335 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:09.160377+0000 mgr.y (mgr.14517) 116 : cluster [DBG] pgmap v100: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:42:10.335 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:09.632154+0000 mon.a (mon.0) 768 : audit [INF] from='client.? 172.21.15.47:0/2163008561' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:42:10.336 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:10.357 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:42:10.360 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:10.360 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:42:10.461 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:42:10.490 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:10.806 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:42:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:42:10] "GET /metrics HTTP/1.1" 200 172167 "" "Prometheus/2.33.4" 2024-02-23T00:42:11.306 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:09.078798+0000 osd.3 (osd.3) 67 : cluster [DBG] 6.c deep-scrub starts 2024-02-23T00:42:11.306 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:09.079910+0000 osd.3 (osd.3) 68 : cluster [DBG] 6.c deep-scrub ok 2024-02-23T00:42:11.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:09.987833+0000 mon.a (mon.0) 769 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:42:11.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:09.995180+0000 mon.a (mon.0) 770 : audit [INF] from='client.? 172.21.15.47:0/2163008561' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:42:11.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:09.995247+0000 mon.a (mon.0) 771 : cluster [DBG] osdmap e97: 8 total, 8 up, 8 in 2024-02-23T00:42:11.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:10.004045+0000 osd.0 (osd.0) 53 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:42:11.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:10.004099+0000 osd.0 (osd.0) 54 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:42:11.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:09.078798+0000 osd.3 (osd.3) 67 : cluster [DBG] 6.c deep-scrub starts 2024-02-23T00:42:11.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:09.079910+0000 osd.3 (osd.3) 68 : cluster [DBG] 6.c deep-scrub ok 2024-02-23T00:42:11.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:09.987833+0000 mon.a (mon.0) 769 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:42:11.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:09.995180+0000 mon.a (mon.0) 770 : audit [INF] from='client.? 172.21.15.47:0/2163008561' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:42:11.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:09.995247+0000 mon.a (mon.0) 771 : cluster [DBG] osdmap e97: 8 total, 8 up, 8 in 2024-02-23T00:42:11.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:10.004045+0000 osd.0 (osd.0) 53 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:42:11.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:10.004099+0000 osd.0 (osd.0) 54 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:42:11.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:09.078798+0000 osd.3 (osd.3) 67 : cluster [DBG] 6.c deep-scrub starts 2024-02-23T00:42:11.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:09.079910+0000 osd.3 (osd.3) 68 : cluster [DBG] 6.c deep-scrub ok 2024-02-23T00:42:11.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:09.987833+0000 mon.a (mon.0) 769 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:42:11.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:09.995180+0000 mon.a (mon.0) 770 : audit [INF] from='client.? 172.21.15.47:0/2163008561' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:42:11.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:09.995247+0000 mon.a (mon.0) 771 : cluster [DBG] osdmap e97: 8 total, 8 up, 8 in 2024-02-23T00:42:11.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:10.004045+0000 osd.0 (osd.0) 53 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:42:11.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:10.004099+0000 osd.0 (osd.0) 54 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:42:12.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:10.596047+0000 osd.6 (osd.6) 47 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:42:12.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:10.596150+0000 osd.6 (osd.6) 48 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:42:12.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:10.662984+0000 osd.5 (osd.5) 61 : cluster [DBG] 6.d deep-scrub starts 2024-02-23T00:42:12.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:10.666291+0000 osd.5 (osd.5) 62 : cluster [DBG] 6.d deep-scrub ok 2024-02-23T00:42:12.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:11.000069+0000 mon.a (mon.0) 772 : cluster [DBG] osdmap e98: 8 total, 8 up, 8 in 2024-02-23T00:42:12.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:11.160740+0000 mgr.y (mgr.14517) 117 : cluster [DBG] pgmap v103: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-23T00:42:12.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:11.284733+0000 mon.a (mon.0) 773 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:42:12.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:10.596047+0000 osd.6 (osd.6) 47 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:42:12.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:10.596150+0000 osd.6 (osd.6) 48 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:42:12.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:10.662984+0000 osd.5 (osd.5) 61 : cluster [DBG] 6.d deep-scrub starts 2024-02-23T00:42:12.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:10.666291+0000 osd.5 (osd.5) 62 : cluster [DBG] 6.d deep-scrub ok 2024-02-23T00:42:12.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:11.000069+0000 mon.a (mon.0) 772 : cluster [DBG] osdmap e98: 8 total, 8 up, 8 in 2024-02-23T00:42:12.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:11.160740+0000 mgr.y (mgr.14517) 117 : cluster [DBG] pgmap v103: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-23T00:42:12.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:11.284733+0000 mon.a (mon.0) 773 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:42:12.443 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:42:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:42:12.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:10.596047+0000 osd.6 (osd.6) 47 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:42:12.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:10.596150+0000 osd.6 (osd.6) 48 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:42:12.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:10.662984+0000 osd.5 (osd.5) 61 : cluster [DBG] 6.d deep-scrub starts 2024-02-23T00:42:12.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:10.666291+0000 osd.5 (osd.5) 62 : cluster [DBG] 6.d deep-scrub ok 2024-02-23T00:42:12.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:11.000069+0000 mon.a (mon.0) 772 : cluster [DBG] osdmap e98: 8 total, 8 up, 8 in 2024-02-23T00:42:12.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:11.160740+0000 mgr.y (mgr.14517) 117 : cluster [DBG] pgmap v103: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-23T00:42:12.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:11.284733+0000 mon.a (mon.0) 773 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:42:12.927 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-23T00:42:12.927 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-23T00:42:12.927 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-02-23T00:42:12.927 INFO:tasks.thrashosds.thrasher:Setting osd 0 primary_affinity to 1.000000 2024-02-23T00:42:12.928 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd primary-affinity 0 1 2024-02-23T00:42:12.943 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:42:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:42:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:42:13.306 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:12.241028+0000 mgr.y (mgr.14517) 118 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:42:13.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:12.241028+0000 mgr.y (mgr.14517) 118 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:42:13.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:12.241028+0000 mgr.y (mgr.14517) 118 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:42:14.306 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:13.161343+0000 mgr.y (mgr.14517) 119 : cluster [DBG] pgmap v104: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-02-23T00:42:14.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:13.161343+0000 mgr.y (mgr.14517) 119 : cluster [DBG] pgmap v104: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-02-23T00:42:14.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:13.161343+0000 mgr.y (mgr.14517) 119 : cluster [DBG] pgmap v104: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-02-23T00:42:15.947 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:42:15.950 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:15.951 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:42:16.052 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:42:16.084 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:16.105 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:42:16.108 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:16.108 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:42:16.209 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:42:16.238 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:16.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:15.161704+0000 mgr.y (mgr.14517) 120 : cluster [DBG] pgmap v105: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:42:16.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:15.161704+0000 mgr.y (mgr.14517) 120 : cluster [DBG] pgmap v105: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:42:16.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:15.161704+0000 mgr.y (mgr.14517) 120 : cluster [DBG] pgmap v105: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:42:16.990 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:42:17.001 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:17.002 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:42:17.103 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:42:17.143 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:17.499 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:42:17.509 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:17.509 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:42:17.523 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:42:17.610 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:42:17.783 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:17.805 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:42:17.807 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:17.808 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:42:17.909 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:42:17.939 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:17.959 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:42:17.961 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:17.962 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:42:18.063 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:42:18.110 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:18.117 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-23T00:42:18.173 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:42:18.180 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:18.181 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:42:18.282 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:42:18.322 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:18.442 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:42:18.446 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:18.446 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:42:18.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.479+0000 7fbb006f5700 1 -- 172.21.15.47:0/373483241 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbaf80fcbb0 msgr2=0x7fbaf80fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:18.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.479+0000 7fbb006f5700 1 --2- 172.21.15.47:0/373483241 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbaf80fcbb0 0x7fbaf80fcf90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fbaf40047b0 tx=0x7fbaf4009cb0).stop 2024-02-23T00:42:18.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.480+0000 7fbb006f5700 1 -- 172.21.15.47:0/373483241 shutdown_connections 2024-02-23T00:42:18.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.480+0000 7fbb006f5700 1 --2- 172.21.15.47:0/373483241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbaf80f7400 0x7fbaf8103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:18.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.480+0000 7fbb006f5700 1 --2- 172.21.15.47:0/373483241 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbaf80fcbb0 0x7fbaf80fcf90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:18.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.480+0000 7fbb006f5700 1 --2- 172.21.15.47:0/373483241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbaf80fd510 0x7fbaf80f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:18.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.480+0000 7fbb006f5700 1 -- 172.21.15.47:0/373483241 >> 172.21.15.47:0/373483241 conn(0x7fbaf80f2820 msgr2=0x7fbaf80f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:42:18.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.480+0000 7fbb006f5700 1 -- 172.21.15.47:0/373483241 shutdown_connections 2024-02-23T00:42:18.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.480+0000 7fbb006f5700 1 -- 172.21.15.47:0/373483241 wait complete. 2024-02-23T00:42:18.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.480+0000 7fbb006f5700 1 librados: starting msgr at 2024-02-23T00:42:18.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.480+0000 7fbb006f5700 1 librados: starting objecter 2024-02-23T00:42:18.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.480+0000 7fbb006f5700 1 Processor -- start 2024-02-23T00:42:18.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.481+0000 7fbb006f5700 1 -- start start 2024-02-23T00:42:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.481+0000 7fbb006f5700 1 librados: setting wanted keys 2024-02-23T00:42:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.481+0000 7fbb006f5700 1 librados: calling monclient init 2024-02-23T00:42:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.481+0000 7fbb006f5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbaf80f7400 0x7fbaf80633d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.481+0000 7fbb006f5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbaf80fcbb0 0x7fbaf80638c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.481+0000 7fbb006f5700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbaf80fd510 0x7fbaf8109410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.481+0000 7fbb006f5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf8109900 con 0x7fbaf80fcbb0 2024-02-23T00:42:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.481+0000 7fbb006f5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf8109a70 con 0x7fbaf80f7400 2024-02-23T00:42:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.481+0000 7fbb006f5700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf8109c30 con 0x7fbaf80fd510 2024-02-23T00:42:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.481+0000 7fbafe491700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbaf80f7400 0x7fbaf80633d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.481+0000 7fbafe491700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbaf80f7400 0x7fbaf80633d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.47:36922/0 (socket says 172.21.15.47:36922) 2024-02-23T00:42:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.481+0000 7fbafe491700 1 -- 172.21.15.47:0/3924270949 learned_addr learned my addr 172.21.15.47:0/3924270949 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:42:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.481+0000 7fbafec92700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbaf80fd510 0x7fbaf8109410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.481+0000 7fbafdc90700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbaf80fcbb0 0x7fbaf80638c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.482+0000 7fbafe491700 1 -- 172.21.15.47:0/3924270949 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbaf80fd510 msgr2=0x7fbaf8109410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:18.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.482+0000 7fbafe491700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbaf80fd510 0x7fbaf8109410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:18.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.482+0000 7fbafe491700 1 -- 172.21.15.47:0/3924270949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbaf80fcbb0 msgr2=0x7fbaf80638c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:18.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.482+0000 7fbafe491700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbaf80fcbb0 0x7fbaf80638c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:18.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.482+0000 7fbafe491700 1 -- 172.21.15.47:0/3924270949 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf00049b0 con 0x7fbaf80f7400 2024-02-23T00:42:18.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.482+0000 7fbafe491700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbaf80f7400 0x7fbaf80633d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fbaf4009f20 tx=0x7fbaf4009cb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:18.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.482+0000 7fbaef7fe700 10 client.?.objecter ms_handle_connect 0x7fbaf80f7400 2024-02-23T00:42:18.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.482+0000 7fbaef7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:42:18.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.482+0000 7fbaef7fe700 1 -- 172.21.15.47:0/3924270949 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4018070 con 0x7fbaf80f7400 2024-02-23T00:42:18.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.482+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaf4005040 con 0x7fbaf80f7400 2024-02-23T00:42:18.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.482+0000 7fbb006f5700 10 client.24661.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:42:18.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.482+0000 7fbaef7fe700 1 -- 172.21.15.47:0/3924270949 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fbaf4013dd0 con 0x7fbaf80f7400 2024-02-23T00:42:18.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.482+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf810a120 con 0x7fbaf80f7400 2024-02-23T00:42:18.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.482+0000 7fbb006f5700 1 librados: init done 2024-02-23T00:42:18.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.482+0000 7fbaef7fe700 1 -- 172.21.15.47:0/3924270949 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf401da10 con 0x7fbaf80f7400 2024-02-23T00:42:18.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.483+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fbadc004f30 con 0x7fbaf80f7400 2024-02-23T00:42:18.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.485+0000 7fbaef7fe700 1 -- 172.21.15.47:0/3924270949 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fbaf4009240 con 0x7fbaf80f7400 2024-02-23T00:42:18.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.485+0000 7fbaef7fe700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbae4062cc0 0x7fbae4065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:18.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.485+0000 7fbaef7fe700 1 -- 172.21.15.47:0/3924270949 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(99..99 src has 48..99) v4 ==== 7421+0+0 (secure 0 0 0) 0x7fbaf407e220 con 0x7fbaf80f7400 2024-02-23T00:42:18.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.485+0000 7fbaef7fe700 10 client.24661.objecter ms_dispatch 0x7fbaf8001550 osd_map(99..99 src has 48..99) v4 2024-02-23T00:42:18.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.485+0000 7fbaef7fe700 3 client.24661.objecter handle_osd_map got epochs [99,99] > 0 2024-02-23T00:42:18.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.485+0000 7fbaef7fe700 3 client.24661.objecter handle_osd_map decoding full epoch 99 2024-02-23T00:42:18.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.485+0000 7fbaef7fe700 20 client.24661.objecter dump_active .. 0 homeless 2024-02-23T00:42:18.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.485+0000 7fbaef7fe700 1 -- 172.21.15.47:0/3924270949 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v99) v1 ==== 54+0+16 (secure 0 0 0) 0x7fbaf4085070 con 0x7fbaf80f7400 2024-02-23T00:42:18.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.486+0000 7fbb006f5700 10 client.24661.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:18.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.486+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_assign -1 1 2024-02-23T00:42:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.486+0000 7fbb006f5700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fbadc004560 0x7fbadc0049a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.486+0000 7fbb006f5700 20 client.24661.objecter _get_session s=0x7fbadc0051d0 osd=0 3 2024-02-23T00:42:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.486+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc0051d0 osd=0 3 2024-02-23T00:42:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.486+0000 7fbb006f5700 20 client.24661.objecter _get_session s=0x7fbadc0051d0 osd=0 3 2024-02-23T00:42:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.486+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_remove -1 1 2024-02-23T00:42:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.486+0000 7fbb006f5700 20 client.24661.objecter get_session s=0x7fbadc0051d0 osd=0 3 2024-02-23T00:42:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.486+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_assign 0 1 2024-02-23T00:42:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.486+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc0051d0 osd=0 4 2024-02-23T00:42:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.486+0000 7fbb006f5700 10 client.24661.objecter _send_command 1 2024-02-23T00:42:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.486+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbadc008960 con 0x7fbadc004560 2024-02-23T00:42:18.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.487+0000 7fbafdc90700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbae4062cc0 0x7fbae4065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:18.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.487+0000 7fbafdc90700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbae4062cc0 0x7fbae4065180 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fbaf80649b0 tx=0x7fbae8009040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:18.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.487+0000 7fbaef7fe700 10 client.24661.objecter ms_handle_connect 0x7fbae4062cc0 2024-02-23T00:42:18.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.488+0000 7fbafec92700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fbadc004560 0x7fbadc0049a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:18.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.488+0000 7fbaef7fe700 10 client.24661.objecter ms_handle_connect 0x7fbadc004560 2024-02-23T00:42:18.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.488+0000 7fbafec92700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fbadc004560 0x7fbadc0049a0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:18.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.489+0000 7fbaef7fe700 1 -- 172.21.15.47:0/3924270949 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fbadc008960 con 0x7fbadc004560 2024-02-23T00:42:18.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.489+0000 7fbaef7fe700 10 client.24661.objecter ms_dispatch 0x7fbaf8001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:42:18.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.489+0000 7fbaef7fe700 10 client.24661.objecter _finish_command 1 = system:0 2024-02-23T00:42:18.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.489+0000 7fbaef7fe700 20 client.24661.objecter put_session s=0x7fbadc0051d0 osd=0 4 2024-02-23T00:42:18.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.489+0000 7fbaef7fe700 15 client.24661.objecter _session_command_op_remove 0 1 2024-02-23T00:42:18.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.504+0000 7fbb006f5700 10 client.24661.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:42:18.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.504+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_assign -1 2 2024-02-23T00:42:18.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.504+0000 7fbb006f5700 20 client.24661.objecter _get_session s=0x7fbadc0051d0 osd=0 3 2024-02-23T00:42:18.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.504+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc0051d0 osd=0 3 2024-02-23T00:42:18.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.504+0000 7fbb006f5700 20 client.24661.objecter _get_session s=0x7fbadc0051d0 osd=0 3 2024-02-23T00:42:18.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.504+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_remove -1 2 2024-02-23T00:42:18.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.504+0000 7fbb006f5700 20 client.24661.objecter get_session s=0x7fbadc0051d0 osd=0 3 2024-02-23T00:42:18.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.504+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_assign 0 2 2024-02-23T00:42:18.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.504+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc0051d0 osd=0 4 2024-02-23T00:42:18.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.504+0000 7fbb006f5700 10 client.24661.objecter _send_command 2 2024-02-23T00:42:18.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.504+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fbadc003f80 con 0x7fbadc004560 2024-02-23T00:42:18.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.506+0000 7fbaef7fe700 1 -- 172.21.15.47:0/3924270949 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fbadc003f80 con 0x7fbadc004560 2024-02-23T00:42:18.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.506+0000 7fbaef7fe700 10 client.24661.objecter ms_dispatch 0x7fbaf8001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:42:18.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.506+0000 7fbaef7fe700 10 client.24661.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:42:18.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.506+0000 7fbaef7fe700 20 client.24661.objecter put_session s=0x7fbadc0051d0 osd=0 4 2024-02-23T00:42:18.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.506+0000 7fbaef7fe700 15 client.24661.objecter _session_command_op_remove 0 2 2024-02-23T00:42:18.509 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:42:18.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.506+0000 7fbb006f5700 10 client.24661.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:18.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.506+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_assign -1 3 2024-02-23T00:42:18.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.506+0000 7fbb006f5700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fbadc002740 0x7fbadc002bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:18.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.506+0000 7fbb006f5700 20 client.24661.objecter _get_session s=0x7fbadc007fb0 osd=1 3 2024-02-23T00:42:18.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.506+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc007fb0 osd=1 3 2024-02-23T00:42:18.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.506+0000 7fbb006f5700 20 client.24661.objecter _get_session s=0x7fbadc007fb0 osd=1 3 2024-02-23T00:42:18.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.506+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_remove -1 3 2024-02-23T00:42:18.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.506+0000 7fbb006f5700 20 client.24661.objecter get_session s=0x7fbadc007fb0 osd=1 3 2024-02-23T00:42:18.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.506+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_assign 1 3 2024-02-23T00:42:18.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.506+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc007fb0 osd=1 4 2024-02-23T00:42:18.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.506+0000 7fbb006f5700 10 client.24661.objecter _send_command 3 2024-02-23T00:42:18.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.506+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbadc008790 con 0x7fbadc002740 2024-02-23T00:42:18.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.507+0000 7fbafe491700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fbadc002740 0x7fbadc002bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:18.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.507+0000 7fbafe491700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fbadc002740 0x7fbadc002bc0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:18.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.507+0000 7fbaef7fe700 10 client.24661.objecter ms_handle_connect 0x7fbadc002740 2024-02-23T00:42:18.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.508+0000 7fbaef7fe700 1 -- 172.21.15.47:0/3924270949 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fbadc008790 con 0x7fbadc002740 2024-02-23T00:42:18.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.508+0000 7fbaef7fe700 10 client.24661.objecter ms_dispatch 0x7fbaf8001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:42:18.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.508+0000 7fbaef7fe700 10 client.24661.objecter _finish_command 3 = system:0 2024-02-23T00:42:18.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.508+0000 7fbaef7fe700 20 client.24661.objecter put_session s=0x7fbadc007fb0 osd=1 4 2024-02-23T00:42:18.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.508+0000 7fbaef7fe700 15 client.24661.objecter _session_command_op_remove 1 3 2024-02-23T00:42:18.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.523+0000 7fbb006f5700 10 client.24661.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:42:18.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.523+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_assign -1 4 2024-02-23T00:42:18.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.523+0000 7fbb006f5700 20 client.24661.objecter _get_session s=0x7fbadc007fb0 osd=1 3 2024-02-23T00:42:18.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.523+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc007fb0 osd=1 3 2024-02-23T00:42:18.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.523+0000 7fbb006f5700 20 client.24661.objecter _get_session s=0x7fbadc007fb0 osd=1 3 2024-02-23T00:42:18.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.523+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_remove -1 4 2024-02-23T00:42:18.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.523+0000 7fbb006f5700 20 client.24661.objecter get_session s=0x7fbadc007fb0 osd=1 3 2024-02-23T00:42:18.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.523+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_assign 1 4 2024-02-23T00:42:18.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.523+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc007fb0 osd=1 4 2024-02-23T00:42:18.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.523+0000 7fbb006f5700 10 client.24661.objecter _send_command 4 2024-02-23T00:42:18.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.523+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fbadc0038b0 con 0x7fbadc002740 2024-02-23T00:42:18.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.525+0000 7fbaef7fe700 1 -- 172.21.15.47:0/3924270949 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fbadc0038b0 con 0x7fbadc002740 2024-02-23T00:42:18.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.525+0000 7fbaef7fe700 10 client.24661.objecter ms_dispatch 0x7fbaf8001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:42:18.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.525+0000 7fbaef7fe700 10 client.24661.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:42:18.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.525+0000 7fbaef7fe700 20 client.24661.objecter put_session s=0x7fbadc007fb0 osd=1 4 2024-02-23T00:42:18.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.525+0000 7fbaef7fe700 15 client.24661.objecter _session_command_op_remove 1 4 2024-02-23T00:42:18.527 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:42:18.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.525+0000 7fbb006f5700 10 client.24661.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:18.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.525+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_assign -1 5 2024-02-23T00:42:18.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.525+0000 7fbb006f5700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fbadc00aff0 0x7fbadc00d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:18.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.525+0000 7fbb006f5700 20 client.24661.objecter _get_session s=0x7fbadc007a80 osd=2 3 2024-02-23T00:42:18.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.525+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc007a80 osd=2 3 2024-02-23T00:42:18.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.525+0000 7fbb006f5700 20 client.24661.objecter _get_session s=0x7fbadc007a80 osd=2 3 2024-02-23T00:42:18.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.525+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_remove -1 5 2024-02-23T00:42:18.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.525+0000 7fbb006f5700 20 client.24661.objecter get_session s=0x7fbadc007a80 osd=2 3 2024-02-23T00:42:18.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.525+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_assign 2 5 2024-02-23T00:42:18.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.525+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc007a80 osd=2 4 2024-02-23T00:42:18.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.525+0000 7fbb006f5700 10 client.24661.objecter _send_command 5 2024-02-23T00:42:18.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.525+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbadc010030 con 0x7fbadc00aff0 2024-02-23T00:42:18.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.525+0000 7fbafdc90700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fbadc00aff0 0x7fbadc00d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:18.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.525+0000 7fbaef7fe700 10 client.24661.objecter ms_handle_connect 0x7fbadc00aff0 2024-02-23T00:42:18.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.525+0000 7fbafdc90700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fbadc00aff0 0x7fbadc00d3e0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:18.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.529+0000 7fbaef7fe700 1 -- 172.21.15.47:0/3924270949 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fbadc010030 con 0x7fbadc00aff0 2024-02-23T00:42:18.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.529+0000 7fbaef7fe700 10 client.24661.objecter ms_dispatch 0x7fbaf8001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:42:18.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.529+0000 7fbaef7fe700 10 client.24661.objecter _finish_command 5 = system:0 2024-02-23T00:42:18.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.529+0000 7fbaef7fe700 20 client.24661.objecter put_session s=0x7fbadc007a80 osd=2 4 2024-02-23T00:42:18.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.529+0000 7fbaef7fe700 15 client.24661.objecter _session_command_op_remove 2 5 2024-02-23T00:42:18.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.544+0000 7fbb006f5700 10 client.24661.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:42:18.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.544+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_assign -1 6 2024-02-23T00:42:18.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.544+0000 7fbb006f5700 20 client.24661.objecter _get_session s=0x7fbadc007a80 osd=2 3 2024-02-23T00:42:18.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.544+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc007a80 osd=2 3 2024-02-23T00:42:18.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.544+0000 7fbb006f5700 20 client.24661.objecter _get_session s=0x7fbadc007a80 osd=2 3 2024-02-23T00:42:18.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.544+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_remove -1 6 2024-02-23T00:42:18.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.544+0000 7fbb006f5700 20 client.24661.objecter get_session s=0x7fbadc007a80 osd=2 3 2024-02-23T00:42:18.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.544+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_assign 2 6 2024-02-23T00:42:18.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.544+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc007a80 osd=2 4 2024-02-23T00:42:18.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.544+0000 7fbb006f5700 10 client.24661.objecter _send_command 6 2024-02-23T00:42:18.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.544+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fbadc0032f0 con 0x7fbadc00aff0 2024-02-23T00:42:18.547 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:42:18.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.546+0000 7fbaef7fe700 1 -- 172.21.15.47:0/3924270949 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fbadc0032f0 con 0x7fbadc00aff0 2024-02-23T00:42:18.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.546+0000 7fbaef7fe700 10 client.24661.objecter ms_dispatch 0x7fbaf8001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:42:18.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.546+0000 7fbaef7fe700 10 client.24661.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:42:18.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.546+0000 7fbaef7fe700 20 client.24661.objecter put_session s=0x7fbadc007a80 osd=2 4 2024-02-23T00:42:18.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.546+0000 7fbaef7fe700 15 client.24661.objecter _session_command_op_remove 2 6 2024-02-23T00:42:18.549 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:42:18.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.546+0000 7fbb006f5700 10 client.24661.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:18.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.546+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_assign -1 7 2024-02-23T00:42:18.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.546+0000 7fbb006f5700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fbadc00ea50 0x7fbadc00eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:18.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.546+0000 7fbb006f5700 20 client.24661.objecter _get_session s=0x7fbadc001bb0 osd=3 3 2024-02-23T00:42:18.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.546+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc001bb0 osd=3 3 2024-02-23T00:42:18.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.546+0000 7fbb006f5700 20 client.24661.objecter _get_session s=0x7fbadc001bb0 osd=3 3 2024-02-23T00:42:18.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.546+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_remove -1 7 2024-02-23T00:42:18.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.546+0000 7fbb006f5700 20 client.24661.objecter get_session s=0x7fbadc001bb0 osd=3 3 2024-02-23T00:42:18.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.546+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_assign 3 7 2024-02-23T00:42:18.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.546+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc001bb0 osd=3 4 2024-02-23T00:42:18.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.546+0000 7fbb006f5700 10 client.24661.objecter _send_command 7 2024-02-23T00:42:18.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.546+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbadc002370 con 0x7fbadc00ea50 2024-02-23T00:42:18.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.546+0000 7fbafec92700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fbadc00ea50 0x7fbadc00eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:18.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.547+0000 7fbafec92700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fbadc00ea50 0x7fbadc00eee0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:18.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.547+0000 7fbaef7fe700 10 client.24661.objecter ms_handle_connect 0x7fbadc00ea50 2024-02-23T00:42:18.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.547+0000 7fbaef7fe700 1 -- 172.21.15.47:0/3924270949 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fbadc002370 con 0x7fbadc00ea50 2024-02-23T00:42:18.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.547+0000 7fbaef7fe700 10 client.24661.objecter ms_dispatch 0x7fbaf8001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:42:18.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.547+0000 7fbaef7fe700 10 client.24661.objecter _finish_command 7 = system:0 2024-02-23T00:42:18.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.547+0000 7fbaef7fe700 20 client.24661.objecter put_session s=0x7fbadc001bb0 osd=3 4 2024-02-23T00:42:18.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.547+0000 7fbaef7fe700 15 client.24661.objecter _session_command_op_remove 3 7 2024-02-23T00:42:18.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.566+0000 7fbb006f5700 10 client.24661.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:42:18.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.566+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_assign -1 8 2024-02-23T00:42:18.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.566+0000 7fbb006f5700 20 client.24661.objecter _get_session s=0x7fbadc001bb0 osd=3 3 2024-02-23T00:42:18.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.566+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc001bb0 osd=3 3 2024-02-23T00:42:18.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.566+0000 7fbb006f5700 20 client.24661.objecter _get_session s=0x7fbadc001bb0 osd=3 3 2024-02-23T00:42:18.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.566+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_remove -1 8 2024-02-23T00:42:18.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.566+0000 7fbb006f5700 20 client.24661.objecter get_session s=0x7fbadc001bb0 osd=3 3 2024-02-23T00:42:18.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.566+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_assign 3 8 2024-02-23T00:42:18.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.566+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc001bb0 osd=3 4 2024-02-23T00:42:18.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.566+0000 7fbb006f5700 10 client.24661.objecter _send_command 8 2024-02-23T00:42:18.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.566+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fbadc00fbd0 con 0x7fbadc00ea50 2024-02-23T00:42:18.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.568+0000 7fbaef7fe700 1 -- 172.21.15.47:0/3924270949 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fbadc00fbd0 con 0x7fbadc00ea50 2024-02-23T00:42:18.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.568+0000 7fbaef7fe700 10 client.24661.objecter ms_dispatch 0x7fbaf8001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:42:18.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.568+0000 7fbaef7fe700 10 client.24661.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:42:18.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.568+0000 7fbaef7fe700 20 client.24661.objecter put_session s=0x7fbadc001bb0 osd=3 4 2024-02-23T00:42:18.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.568+0000 7fbaef7fe700 15 client.24661.objecter _session_command_op_remove 3 8 2024-02-23T00:42:18.571 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:42:18.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.569+0000 7fbb006f5700 10 client.24661.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:18.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.569+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_assign -1 9 2024-02-23T00:42:18.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.569+0000 7fbb006f5700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fbadc0121b0 0x7fbadc0145a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:18.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.569+0000 7fbb006f5700 20 client.24661.objecter _get_session s=0x7fbadc00f400 osd=4 3 2024-02-23T00:42:18.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.569+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc00f400 osd=4 3 2024-02-23T00:42:18.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.569+0000 7fbb006f5700 20 client.24661.objecter _get_session s=0x7fbadc00f400 osd=4 3 2024-02-23T00:42:18.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.569+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_remove -1 9 2024-02-23T00:42:18.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.569+0000 7fbb006f5700 20 client.24661.objecter get_session s=0x7fbadc00f400 osd=4 3 2024-02-23T00:42:18.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.569+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_assign 4 9 2024-02-23T00:42:18.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.569+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc00f400 osd=4 4 2024-02-23T00:42:18.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.569+0000 7fbb006f5700 10 client.24661.objecter _send_command 9 2024-02-23T00:42:18.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.569+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbadc0171f0 con 0x7fbadc0121b0 2024-02-23T00:42:18.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.570+0000 7fbafe491700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fbadc0121b0 0x7fbadc0145a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:18.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.570+0000 7fbaef7fe700 10 client.24661.objecter ms_handle_connect 0x7fbadc0121b0 2024-02-23T00:42:18.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.570+0000 7fbafe491700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fbadc0121b0 0x7fbadc0145a0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:18.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.571+0000 7fbaef7fe700 1 -- 172.21.15.47:0/3924270949 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fbadc0171f0 con 0x7fbadc0121b0 2024-02-23T00:42:18.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.571+0000 7fbaef7fe700 10 client.24661.objecter ms_dispatch 0x7fbaf8001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:42:18.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.571+0000 7fbaef7fe700 10 client.24661.objecter _finish_command 9 = system:0 2024-02-23T00:42:18.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.571+0000 7fbaef7fe700 20 client.24661.objecter put_session s=0x7fbadc00f400 osd=4 4 2024-02-23T00:42:18.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.571+0000 7fbaef7fe700 15 client.24661.objecter _session_command_op_remove 4 9 2024-02-23T00:42:18.584 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:17.162965+0000 mgr.y (mgr.14517) 121 : cluster [DBG] pgmap v106: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:42:18.584 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:17.194991+0000 osd.3 (osd.3) 69 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:42:18.584 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:17.195056+0000 osd.3 (osd.3) 70 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:42:18.584 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:17.358033+0000 mon.a (mon.0) 774 : cluster [DBG] osdmap e99: 8 total, 8 up, 8 in 2024-02-23T00:42:18.585 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:17.162965+0000 mgr.y (mgr.14517) 121 : cluster [DBG] pgmap v106: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:42:18.585 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:17.194991+0000 osd.3 (osd.3) 69 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:42:18.585 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:17.195056+0000 osd.3 (osd.3) 70 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:42:18.585 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:17.358033+0000 mon.a (mon.0) 774 : cluster [DBG] osdmap e99: 8 total, 8 up, 8 in 2024-02-23T00:42:18.587 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:18.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.593+0000 7fbb006f5700 10 client.24661.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:42:18.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.593+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_assign -1 10 2024-02-23T00:42:18.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.593+0000 7fbb006f5700 20 client.24661.objecter _get_session s=0x7fbadc00f400 osd=4 3 2024-02-23T00:42:18.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.593+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc00f400 osd=4 3 2024-02-23T00:42:18.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.593+0000 7fbb006f5700 20 client.24661.objecter _get_session s=0x7fbadc00f400 osd=4 3 2024-02-23T00:42:18.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.593+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_remove -1 10 2024-02-23T00:42:18.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.593+0000 7fbb006f5700 20 client.24661.objecter get_session s=0x7fbadc00f400 osd=4 3 2024-02-23T00:42:18.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.593+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_assign 4 10 2024-02-23T00:42:18.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.593+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc00f400 osd=4 4 2024-02-23T00:42:18.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.593+0000 7fbb006f5700 10 client.24661.objecter _send_command 10 2024-02-23T00:42:18.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.593+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fbadc014a90 con 0x7fbadc0121b0 2024-02-23T00:42:18.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.595+0000 7fbaef7fe700 1 -- 172.21.15.47:0/3924270949 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fbadc014a90 con 0x7fbadc0121b0 2024-02-23T00:42:18.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.595+0000 7fbaef7fe700 10 client.24661.objecter ms_dispatch 0x7fbaf8001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:42:18.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.595+0000 7fbaef7fe700 10 client.24661.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:42:18.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.595+0000 7fbaef7fe700 20 client.24661.objecter put_session s=0x7fbadc00f400 osd=4 4 2024-02-23T00:42:18.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.595+0000 7fbaef7fe700 15 client.24661.objecter _session_command_op_remove 4 10 2024-02-23T00:42:18.600 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:42:18.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.598+0000 7fbb006f5700 10 client.24661.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:18.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.598+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_assign -1 11 2024-02-23T00:42:18.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.598+0000 7fbb006f5700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fbadc015170 0x7fbadc0175c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:18.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.598+0000 7fbb006f5700 20 client.24661.objecter _get_session s=0x7fbadc014a90 osd=5 3 2024-02-23T00:42:18.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.598+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc014a90 osd=5 3 2024-02-23T00:42:18.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.598+0000 7fbb006f5700 20 client.24661.objecter _get_session s=0x7fbadc014a90 osd=5 3 2024-02-23T00:42:18.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.598+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_remove -1 11 2024-02-23T00:42:18.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.598+0000 7fbb006f5700 20 client.24661.objecter get_session s=0x7fbadc014a90 osd=5 3 2024-02-23T00:42:18.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.598+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_assign 5 11 2024-02-23T00:42:18.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.598+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc014a90 osd=5 4 2024-02-23T00:42:18.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.598+0000 7fbb006f5700 10 client.24661.objecter _send_command 11 2024-02-23T00:42:18.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.598+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbadc01a400 con 0x7fbadc015170 2024-02-23T00:42:18.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.598+0000 7fbafdc90700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fbadc015170 0x7fbadc0175c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:18.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.599+0000 7fbaef7fe700 10 client.24661.objecter ms_handle_connect 0x7fbadc015170 2024-02-23T00:42:18.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.599+0000 7fbafdc90700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fbadc015170 0x7fbadc0175c0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:18.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.600+0000 7fbaef7fe700 1 -- 172.21.15.47:0/3924270949 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fbadc01a400 con 0x7fbadc015170 2024-02-23T00:42:18.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.600+0000 7fbaef7fe700 10 client.24661.objecter ms_dispatch 0x7fbaf8001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:42:18.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.600+0000 7fbaef7fe700 10 client.24661.objecter _finish_command 11 = system:0 2024-02-23T00:42:18.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.600+0000 7fbaef7fe700 20 client.24661.objecter put_session s=0x7fbadc014a90 osd=5 4 2024-02-23T00:42:18.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.600+0000 7fbaef7fe700 15 client.24661.objecter _session_command_op_remove 5 11 2024-02-23T00:42:18.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.617+0000 7fbb006f5700 10 client.24661.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:42:18.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.617+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_assign -1 12 2024-02-23T00:42:18.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.617+0000 7fbb006f5700 20 client.24661.objecter _get_session s=0x7fbadc014a90 osd=5 3 2024-02-23T00:42:18.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.617+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc014a90 osd=5 3 2024-02-23T00:42:18.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.617+0000 7fbb006f5700 20 client.24661.objecter _get_session s=0x7fbadc014a90 osd=5 3 2024-02-23T00:42:18.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.617+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_remove -1 12 2024-02-23T00:42:18.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.617+0000 7fbb006f5700 20 client.24661.objecter get_session s=0x7fbadc014a90 osd=5 3 2024-02-23T00:42:18.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.617+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_assign 5 12 2024-02-23T00:42:18.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.617+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc014a90 osd=5 4 2024-02-23T00:42:18.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.617+0000 7fbb006f5700 10 client.24661.objecter _send_command 12 2024-02-23T00:42:18.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.617+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fbadc01a400 con 0x7fbadc015170 2024-02-23T00:42:18.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.618+0000 7fbaef7fe700 1 -- 172.21.15.47:0/3924270949 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fbadc01a400 con 0x7fbadc015170 2024-02-23T00:42:18.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.618+0000 7fbaef7fe700 10 client.24661.objecter ms_dispatch 0x7fbaf8001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:42:18.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.618+0000 7fbaef7fe700 10 client.24661.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:42:18.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.618+0000 7fbaef7fe700 20 client.24661.objecter put_session s=0x7fbadc014a90 osd=5 4 2024-02-23T00:42:18.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.618+0000 7fbaef7fe700 15 client.24661.objecter _session_command_op_remove 5 12 2024-02-23T00:42:18.621 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:42:18.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.620+0000 7fbb006f5700 10 client.24661.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:18.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.620+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_assign -1 13 2024-02-23T00:42:18.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.620+0000 7fbb006f5700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fbadc019730 0x7fbadc019bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:18.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.620+0000 7fbb006f5700 20 client.24661.objecter _get_session s=0x7fbadc01a400 osd=6 3 2024-02-23T00:42:18.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.620+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc01a400 osd=6 3 2024-02-23T00:42:18.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.620+0000 7fbb006f5700 20 client.24661.objecter _get_session s=0x7fbadc01a400 osd=6 3 2024-02-23T00:42:18.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.620+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_remove -1 13 2024-02-23T00:42:18.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.620+0000 7fbb006f5700 20 client.24661.objecter get_session s=0x7fbadc01a400 osd=6 3 2024-02-23T00:42:18.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.620+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_assign 6 13 2024-02-23T00:42:18.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.620+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc01a400 osd=6 4 2024-02-23T00:42:18.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.620+0000 7fbb006f5700 10 client.24661.objecter _send_command 13 2024-02-23T00:42:18.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.620+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbadc01d760 con 0x7fbadc019730 2024-02-23T00:42:18.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.620+0000 7fbafec92700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fbadc019730 0x7fbadc019bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:18.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.620+0000 7fbaef7fe700 10 client.24661.objecter ms_handle_connect 0x7fbadc019730 2024-02-23T00:42:18.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.620+0000 7fbafec92700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fbadc019730 0x7fbadc019bb0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:18.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.621+0000 7fbaef7fe700 1 -- 172.21.15.47:0/3924270949 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fbadc01d760 con 0x7fbadc019730 2024-02-23T00:42:18.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.621+0000 7fbaef7fe700 10 client.24661.objecter ms_dispatch 0x7fbaf8001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:42:18.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.621+0000 7fbaef7fe700 10 client.24661.objecter _finish_command 13 = system:0 2024-02-23T00:42:18.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.621+0000 7fbaef7fe700 20 client.24661.objecter put_session s=0x7fbadc01a400 osd=6 4 2024-02-23T00:42:18.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.621+0000 7fbaef7fe700 15 client.24661.objecter _session_command_op_remove 6 13 2024-02-23T00:42:18.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.637+0000 7fbb006f5700 10 client.24661.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:42:18.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.637+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_assign -1 14 2024-02-23T00:42:18.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.637+0000 7fbb006f5700 20 client.24661.objecter _get_session s=0x7fbadc01a400 osd=6 3 2024-02-23T00:42:18.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.637+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc01a400 osd=6 3 2024-02-23T00:42:18.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.637+0000 7fbb006f5700 20 client.24661.objecter _get_session s=0x7fbadc01a400 osd=6 3 2024-02-23T00:42:18.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.637+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_remove -1 14 2024-02-23T00:42:18.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.637+0000 7fbb006f5700 20 client.24661.objecter get_session s=0x7fbadc01a400 osd=6 3 2024-02-23T00:42:18.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.637+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_assign 6 14 2024-02-23T00:42:18.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.637+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc01a400 osd=6 4 2024-02-23T00:42:18.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.637+0000 7fbb006f5700 10 client.24661.objecter _send_command 14 2024-02-23T00:42:18.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.637+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fbadc017bc0 con 0x7fbadc019730 2024-02-23T00:42:18.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.639+0000 7fbaef7fe700 1 -- 172.21.15.47:0/3924270949 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fbadc017bc0 con 0x7fbadc019730 2024-02-23T00:42:18.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.639+0000 7fbaef7fe700 10 client.24661.objecter ms_dispatch 0x7fbaf8001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:42:18.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.639+0000 7fbaef7fe700 10 client.24661.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:42:18.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.639+0000 7fbaef7fe700 20 client.24661.objecter put_session s=0x7fbadc01a400 osd=6 4 2024-02-23T00:42:18.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.639+0000 7fbaef7fe700 15 client.24661.objecter _session_command_op_remove 6 14 2024-02-23T00:42:18.642 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:42:18.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.639+0000 7fbb006f5700 10 client.24661.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:18.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.639+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_assign -1 15 2024-02-23T00:42:18.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.639+0000 7fbb006f5700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fbadc018240 0x7fbadc0186a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:18.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.639+0000 7fbb006f5700 20 client.24661.objecter _get_session s=0x7fbadc017bc0 osd=7 3 2024-02-23T00:42:18.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.639+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc017bc0 osd=7 3 2024-02-23T00:42:18.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.639+0000 7fbb006f5700 20 client.24661.objecter _get_session s=0x7fbadc017bc0 osd=7 3 2024-02-23T00:42:18.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.639+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_remove -1 15 2024-02-23T00:42:18.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.639+0000 7fbb006f5700 20 client.24661.objecter get_session s=0x7fbadc017bc0 osd=7 3 2024-02-23T00:42:18.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.639+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_assign 7 15 2024-02-23T00:42:18.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.639+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc017bc0 osd=7 4 2024-02-23T00:42:18.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.639+0000 7fbb006f5700 10 client.24661.objecter _send_command 15 2024-02-23T00:42:18.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.639+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbadc018d30 con 0x7fbadc018240 2024-02-23T00:42:18.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.640+0000 7fbafe491700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fbadc018240 0x7fbadc0186a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:18.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.640+0000 7fbaef7fe700 10 client.24661.objecter ms_handle_connect 0x7fbadc018240 2024-02-23T00:42:18.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.640+0000 7fbafe491700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fbadc018240 0x7fbadc0186a0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:18.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.641+0000 7fbaef7fe700 1 -- 172.21.15.47:0/3924270949 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fbadc018d30 con 0x7fbadc018240 2024-02-23T00:42:18.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.641+0000 7fbaef7fe700 10 client.24661.objecter ms_dispatch 0x7fbaf8001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:42:18.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.641+0000 7fbaef7fe700 10 client.24661.objecter _finish_command 15 = system:0 2024-02-23T00:42:18.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.641+0000 7fbaef7fe700 20 client.24661.objecter put_session s=0x7fbadc017bc0 osd=7 4 2024-02-23T00:42:18.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.641+0000 7fbaef7fe700 15 client.24661.objecter _session_command_op_remove 7 15 2024-02-23T00:42:18.654 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:42:18.659 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:18.660 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:42:18.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.659+0000 7fbb006f5700 10 client.24661.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:42:18.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.659+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_assign -1 16 2024-02-23T00:42:18.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.659+0000 7fbb006f5700 20 client.24661.objecter _get_session s=0x7fbadc017bc0 osd=7 3 2024-02-23T00:42:18.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.660+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc017bc0 osd=7 3 2024-02-23T00:42:18.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.660+0000 7fbb006f5700 20 client.24661.objecter _get_session s=0x7fbadc017bc0 osd=7 3 2024-02-23T00:42:18.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.660+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_remove -1 16 2024-02-23T00:42:18.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.660+0000 7fbb006f5700 20 client.24661.objecter get_session s=0x7fbadc017bc0 osd=7 3 2024-02-23T00:42:18.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.660+0000 7fbb006f5700 15 client.24661.objecter _session_command_op_assign 7 16 2024-02-23T00:42:18.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.660+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc017bc0 osd=7 4 2024-02-23T00:42:18.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.660+0000 7fbb006f5700 10 client.24661.objecter _send_command 16 2024-02-23T00:42:18.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.660+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fbadc018d30 con 0x7fbadc018240 2024-02-23T00:42:18.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.661+0000 7fbaef7fe700 1 -- 172.21.15.47:0/3924270949 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fbadc018d30 con 0x7fbadc018240 2024-02-23T00:42:18.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.661+0000 7fbaef7fe700 10 client.24661.objecter ms_dispatch 0x7fbaf8001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:42:18.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.661+0000 7fbaef7fe700 10 client.24661.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:42:18.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.661+0000 7fbaef7fe700 20 client.24661.objecter put_session s=0x7fbadc017bc0 osd=7 4 2024-02-23T00:42:18.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.661+0000 7fbaef7fe700 15 client.24661.objecter _session_command_op_remove 7 16 2024-02-23T00:42:18.665 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:42:18.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 10 librados: watch_flush enter 2024-02-23T00:42:18.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 10 librados: watch_flush exit 2024-02-23T00:42:18.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 10 client.24661.objecter close_session for osd.0 2024-02-23T00:42:18.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fbadc004560 msgr2=0x7fbadc0049a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:18.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fbadc004560 0x7fbadc0049a0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:18.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc0051d0 osd=0 1 2024-02-23T00:42:18.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 10 client.24661.objecter close_session for osd.1 2024-02-23T00:42:18.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fbadc002740 msgr2=0x7fbadc002bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:18.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fbadc002740 0x7fbadc002bc0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:18.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc007fb0 osd=1 1 2024-02-23T00:42:18.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 10 client.24661.objecter close_session for osd.2 2024-02-23T00:42:18.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fbadc00aff0 msgr2=0x7fbadc00d3e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:18.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fbadc00aff0 0x7fbadc00d3e0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:18.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc007a80 osd=2 1 2024-02-23T00:42:18.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 10 client.24661.objecter close_session for osd.3 2024-02-23T00:42:18.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fbadc00ea50 msgr2=0x7fbadc00eee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:18.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fbadc00ea50 0x7fbadc00eee0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:18.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc001bb0 osd=3 1 2024-02-23T00:42:18.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 10 client.24661.objecter close_session for osd.4 2024-02-23T00:42:18.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fbadc0121b0 msgr2=0x7fbadc0145a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:18.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fbadc0121b0 0x7fbadc0145a0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:18.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc00f400 osd=4 1 2024-02-23T00:42:18.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 10 client.24661.objecter close_session for osd.5 2024-02-23T00:42:18.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fbadc015170 msgr2=0x7fbadc0175c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:18.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fbadc015170 0x7fbadc0175c0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:18.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc014a90 osd=5 1 2024-02-23T00:42:18.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 10 client.24661.objecter close_session for osd.6 2024-02-23T00:42:18.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fbadc019730 msgr2=0x7fbadc019bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:18.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fbadc019730 0x7fbadc019bb0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:18.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc01a400 osd=6 1 2024-02-23T00:42:18.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 10 client.24661.objecter close_session for osd.7 2024-02-23T00:42:18.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fbadc018240 msgr2=0x7fbadc0186a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:18.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fbadc018240 0x7fbadc0186a0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:18.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 20 client.24661.objecter put_session s=0x7fbadc017bc0 osd=7 1 2024-02-23T00:42:18.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 20 client.24661.objecter shutdown clearing up homeless session... 2024-02-23T00:42:18.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 10 client.24661.objecter successfully canceled tick 2024-02-23T00:42:18.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbae4062cc0 msgr2=0x7fbae4065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:18.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbae4062cc0 0x7fbae4065180 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fbaf80649b0 tx=0x7fbae8009040).stop 2024-02-23T00:42:18.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbaf80f7400 msgr2=0x7fbaf80633d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:18.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.662+0000 7fbb006f5700 1 --2- 172.21.15.47:0/3924270949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbaf80f7400 0x7fbaf80633d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fbaf4009f20 tx=0x7fbaf4009cb0).stop 2024-02-23T00:42:18.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.665+0000 7fbafec92700 1 -- 172.21.15.47:0/3924270949 reap_dead start 2024-02-23T00:42:18.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.665+0000 7fbafec92700 1 -- 172.21.15.47:0/3924270949 reap_dead start 2024-02-23T00:42:18.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.665+0000 7fbafec92700 1 -- 172.21.15.47:0/3924270949 reap_dead start 2024-02-23T00:42:18.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.665+0000 7fbafec92700 1 -- 172.21.15.47:0/3924270949 reap_dead start 2024-02-23T00:42:18.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.665+0000 7fbafec92700 1 -- 172.21.15.47:0/3924270949 reap_dead start 2024-02-23T00:42:18.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.668+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 shutdown_connections 2024-02-23T00:42:18.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.668+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 >> 172.21.15.47:0/3924270949 conn(0x7fbaf80f2820 msgr2=0x7fbaf80fa940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:42:18.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.669+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 shutdown_connections 2024-02-23T00:42:18.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.669+0000 7fbb006f5700 1 -- 172.21.15.47:0/3924270949 wait complete. 2024-02-23T00:42:18.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:18.669+0000 7fbb006f5700 1 librados: shutdown 2024-02-23T00:42:18.672 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:42:18.672 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:42:18.672 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:42:18.672 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:42:18.672 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:42:18.672 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:42:18.672 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:42:18.673 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:42:18.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:17.162965+0000 mgr.y (mgr.14517) 121 : cluster [DBG] pgmap v106: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:42:18.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:17.194991+0000 osd.3 (osd.3) 69 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:42:18.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:17.195056+0000 osd.3 (osd.3) 70 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:42:18.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:17.358033+0000 mon.a (mon.0) 774 : cluster [DBG] osdmap e99: 8 total, 8 up, 8 in 2024-02-23T00:42:18.761 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:42:18.794 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:18.816 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:42:18.818 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:18.819 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:42:18.919 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:42:18.950 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:19.100 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:42:19.106 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:19.106 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:42:19.207 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:42:19.239 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:19.261 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:42:19.263 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:19.264 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:42:19.365 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:42:19.404 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:19.508 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:42:19.513 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:19.513 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:42:19.614 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:42:19.647 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:18.486372+0000 mon.b (mon.1) 32 : audit [DBG] from='client.? 172.21.15.47:0/3924270949' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:42:19.648 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:18.486372+0000 mon.b (mon.1) 32 : audit [DBG] from='client.? 172.21.15.47:0/3924270949' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:42:19.651 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:19.689 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:42:19.691 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:42:19.691 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-02-23T00:42:19.691 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-02-23T00:42:19.691 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-02-23T00:42:19.691 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:42:19.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:18.486372+0000 mon.b (mon.1) 32 : audit [DBG] from='client.? 172.21.15.47:0/3924270949' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:42:19.706 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:19.706 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:42:19.807 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:42:19.839 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:20.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:19.163365+0000 mgr.y (mgr.14517) 122 : cluster [DBG] pgmap v108: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:42:20.726 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:19.163365+0000 mgr.y (mgr.14517) 122 : cluster [DBG] pgmap v108: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:42:20.727 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:19.163365+0000 mgr.y (mgr.14517) 122 : cluster [DBG] pgmap v108: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:42:20.727 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:42:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:42:20] "GET /metrics HTTP/1.1" 200 172253 "" "Prometheus/2.33.4" 2024-02-23T00:42:21.047 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:42:21.637 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.0 dump_historic_ops 2024-02-23T00:42:22.195 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:42:22.199 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:22.200 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:42:22.301 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:42:22.329 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:42:22.347 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:22.527 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:42:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:42:22.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:21.163731+0000 mgr.y (mgr.14517) 123 : cluster [DBG] pgmap v109: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:42:22.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:21.163731+0000 mgr.y (mgr.14517) 123 : cluster [DBG] pgmap v109: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:42:22.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:21.163731+0000 mgr.y (mgr.14517) 123 : cluster [DBG] pgmap v109: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:42:22.706 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:42:22.713 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:22.713 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:42:22.814 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:42:22.847 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:22.869 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:42:22.880 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:22.880 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:42:22.943 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:42:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:42:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:42:22.981 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:42:23.010 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:23.265 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:42:23.269 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:23.269 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:42:23.370 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:42:23.408 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:23.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:22.251984+0000 mgr.y (mgr.14517) 124 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:42:23.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:22.251984+0000 mgr.y (mgr.14517) 124 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:42:23.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:22.251984+0000 mgr.y (mgr.14517) 124 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:42:23.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.554+0000 7f618bf99700 1 -- 172.21.15.47:0/2544140159 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6184104660 msgr2=0x7f6184106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:23.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.554+0000 7f618bf99700 1 --2- 172.21.15.47:0/2544140159 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6184104660 0x7f6184106a50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f617c009be0 tx=0x7f617c004d10).stop 2024-02-23T00:42:23.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.554+0000 7f618bf99700 1 -- 172.21.15.47:0/2544140159 shutdown_connections 2024-02-23T00:42:23.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.554+0000 7f618bf99700 1 --2- 172.21.15.47:0/2544140159 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6184104660 0x7f6184106a50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:23.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.554+0000 7f618bf99700 1 --2- 172.21.15.47:0/2544140159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61840f6b00 0x7f61840f6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:42:23.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.554+0000 7f618bf99700 1 --2- 172.21.15.47:0/2544140159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61840f73d0 0x7f6184104170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:23.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.554+0000 7f618bf99700 1 -- 172.21.15.47:0/2544140159 >> 172.21.15.47:0/2544140159 conn(0x7f61840f2820 msgr2=0x7f61840f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:42:23.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.554+0000 7f618bf99700 1 -- 172.21.15.47:0/2544140159 shutdown_connections 2024-02-23T00:42:23.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.554+0000 7f618bf99700 1 -- 172.21.15.47:0/2544140159 wait complete. 2024-02-23T00:42:23.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.554+0000 7f618bf99700 1 librados: starting msgr at 2024-02-23T00:42:23.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.554+0000 7f618bf99700 1 librados: starting objecter 2024-02-23T00:42:23.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.555+0000 7f618bf99700 1 Processor -- start 2024-02-23T00:42:23.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.555+0000 7f618bf99700 1 -- start start 2024-02-23T00:42:23.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.555+0000 7f618bf99700 1 librados: setting wanted keys 2024-02-23T00:42:23.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.555+0000 7f618bf99700 1 librados: calling monclient init 2024-02-23T00:42:23.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.555+0000 7f618bf99700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f61840f6b00 0x7f6184103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:23.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.555+0000 7f618bf99700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61840f73d0 0x7f6184103fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:23.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.555+0000 7f618bf99700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6184104660 0x7f6184100d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:23.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.555+0000 7f618bf99700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6184101430 con 0x7f6184104660 2024-02-23T00:42:23.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.555+0000 7f618bf99700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61841015a0 con 0x7f61840f73d0 2024-02-23T00:42:23.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.555+0000 7f618bf99700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6184101760 con 0x7f61840f6b00 2024-02-23T00:42:23.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.555+0000 7f618a536700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6184104660 0x7f6184100d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:23.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.555+0000 7f618a536700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6184104660 0x7f6184100d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59076/0 (socket says 172.21.15.47:59076) 2024-02-23T00:42:23.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.556+0000 7f618a536700 1 -- 172.21.15.47:0/609245625 learned_addr learned my addr 172.21.15.47:0/609245625 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:42:23.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.556+0000 7f6189d35700 1 --2- 172.21.15.47:0/609245625 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f61840f6b00 0x7f6184103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:23.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.556+0000 7f6189534700 1 --2- 172.21.15.47:0/609245625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61840f73d0 0x7f6184103fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:23.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.556+0000 7f618a536700 1 -- 172.21.15.47:0/609245625 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f61840f6b00 msgr2=0x7f6184103ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:23.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.556+0000 7f618a536700 1 --2- 172.21.15.47:0/609245625 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f61840f6b00 0x7f6184103ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:23.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.556+0000 7f618a536700 1 -- 172.21.15.47:0/609245625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61840f73d0 msgr2=0x7f6184103fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:23.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.556+0000 7f618a536700 1 --2- 172.21.15.47:0/609245625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61840f73d0 0x7f6184103fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:23.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.556+0000 7f618a536700 1 -- 172.21.15.47:0/609245625 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f617c0097c0 con 0x7f6184104660 2024-02-23T00:42:23.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.556+0000 7f6189d35700 1 --2- 172.21.15.47:0/609245625 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f61840f6b00 0x7f6184103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:42:23.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.556+0000 7f618a536700 1 --2- 172.21.15.47:0/609245625 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6184104660 0x7f6184100d40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f617c009760 tx=0x7f617c004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:23.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.556+0000 7f617affd700 10 client.?.objecter ms_handle_connect 0x7f6184104660 2024-02-23T00:42:23.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.556+0000 7f617affd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:42:23.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.556+0000 7f617affd700 1 -- 172.21.15.47:0/609245625 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c00e040 con 0x7f6184104660 2024-02-23T00:42:23.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.556+0000 7f617affd700 1 -- 172.21.15.47:0/609245625 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f617c018040 con 0x7f6184104660 2024-02-23T00:42:23.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.556+0000 7f617affd700 1 -- 172.21.15.47:0/609245625 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c013e50 con 0x7f6184104660 2024-02-23T00:42:23.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.556+0000 7f618bf99700 1 -- 172.21.15.47:0/609245625 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61841019e0 con 0x7f6184104660 2024-02-23T00:42:23.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.557+0000 7f618bf99700 10 client.14784.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:42:23.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.557+0000 7f618bf99700 1 -- 172.21.15.47:0/609245625 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f618411b270 con 0x7f6184104660 2024-02-23T00:42:23.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.557+0000 7f618bf99700 1 librados: init done 2024-02-23T00:42:23.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.558+0000 7f617affd700 1 -- 172.21.15.47:0/609245625 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f617c013420 con 0x7f6184104660 2024-02-23T00:42:23.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.558+0000 7f617affd700 1 --2- 172.21.15.47:0/609245625 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6170062cc0 0x7f6170065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:23.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.558+0000 7f617affd700 1 -- 172.21.15.47:0/609245625 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(99..99 src has 48..99) v4 ==== 7421+0+0 (secure 0 0 0) 0x7f617c07dcb0 con 0x7f6184104660 2024-02-23T00:42:23.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.558+0000 7f617affd700 10 client.14784.objecter ms_dispatch 0x7f6184001550 osd_map(99..99 src has 48..99) v4 2024-02-23T00:42:23.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.558+0000 7f617affd700 3 client.14784.objecter handle_osd_map got epochs [99,99] > 0 2024-02-23T00:42:23.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.558+0000 7f617affd700 3 client.14784.objecter handle_osd_map decoding full epoch 99 2024-02-23T00:42:23.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.558+0000 7f618bf99700 1 -- 172.21.15.47:0/609245625 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f618405fa60 con 0x7f6184104660 2024-02-23T00:42:23.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.559+0000 7f617affd700 20 client.14784.objecter dump_active .. 0 homeless 2024-02-23T00:42:23.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.559+0000 7f6189d35700 1 --2- 172.21.15.47:0/609245625 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6170062cc0 0x7f6170065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:23.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.559+0000 7f6189d35700 1 --2- 172.21.15.47:0/609245625 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6170062cc0 0x7f6170065180 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f6180009990 tx=0x7f6180008040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:23.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.563+0000 7f617affd700 10 client.14784.objecter ms_handle_connect 0x7f6170062cc0 2024-02-23T00:42:23.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.564+0000 7f617affd700 1 -- 172.21.15.47:0/609245625 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f617c021020 con 0x7f6184104660 2024-02-23T00:42:23.796 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:42:23.803 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:42:23.803 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:23.804 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:42:23.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.805+0000 7f618bf99700 1 -- 172.21.15.47:0/609245625 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} v 0) v1 -- 0x7f6184109270 con 0x7f6184104660 2024-02-23T00:42:23.905 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:42:23.937 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:23.958 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:42:23.961 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:23.962 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:42:24.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.993+0000 7fac2b9a4700 1 -- 172.21.15.47:0/3047211192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac240fcaf0 msgr2=0x7fac240fced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:24.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.993+0000 7fac2b9a4700 1 --2- 172.21.15.47:0/3047211192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac240fcaf0 0x7fac240fced0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7fac20004870 tx=0x7fac20009cb0).stop 2024-02-23T00:42:24.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.995+0000 7fac2b9a4700 1 -- 172.21.15.47:0/3047211192 shutdown_connections 2024-02-23T00:42:24.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.995+0000 7fac2b9a4700 1 --2- 172.21.15.47:0/3047211192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac240f7a50 0x7fac24103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:24.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.995+0000 7fac2b9a4700 1 --2- 172.21.15.47:0/3047211192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac240fcaf0 0x7fac240fced0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:24.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.995+0000 7fac2b9a4700 1 --2- 172.21.15.47:0/3047211192 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fac240fd450 0x7fac240f7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:24.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.995+0000 7fac2b9a4700 1 -- 172.21.15.47:0/3047211192 >> 172.21.15.47:0/3047211192 conn(0x7fac24070030 msgr2=0x7fac240f4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:42:24.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.995+0000 7fac2b9a4700 1 -- 172.21.15.47:0/3047211192 shutdown_connections 2024-02-23T00:42:24.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.995+0000 7fac2b9a4700 1 -- 172.21.15.47:0/3047211192 wait complete. 2024-02-23T00:42:24.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.995+0000 7fac2b9a4700 1 librados: starting msgr at 2024-02-23T00:42:24.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.995+0000 7fac2b9a4700 1 librados: starting objecter 2024-02-23T00:42:24.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.995+0000 7fac2b9a4700 1 Processor -- start 2024-02-23T00:42:24.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.996+0000 7fac2b9a4700 1 -- start start 2024-02-23T00:42:24.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.996+0000 7fac2b9a4700 1 librados: setting wanted keys 2024-02-23T00:42:24.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.996+0000 7fac2b9a4700 1 librados: calling monclient init 2024-02-23T00:42:24.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.996+0000 7fac2b9a4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac240f7a50 0x7fac2410df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:24.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.996+0000 7fac2b9a4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac240fcaf0 0x7fac2410e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:24.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.996+0000 7fac2b9a4700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fac240fd450 0x7fac241080b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:24.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.996+0000 7fac2b9a4700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac241085a0 con 0x7fac240fcaf0 2024-02-23T00:42:24.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.996+0000 7fac2b9a4700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac24108710 con 0x7fac240f7a50 2024-02-23T00:42:24.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.996+0000 7fac2b9a4700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac241088d0 con 0x7fac240fd450 2024-02-23T00:42:24.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.996+0000 7fac28f3f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac240fcaf0 0x7fac2410e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:24.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.996+0000 7fac28f3f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac240fcaf0 0x7fac2410e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59094/0 (socket says 172.21.15.47:59094) 2024-02-23T00:42:24.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.996+0000 7fac28f3f700 1 -- 172.21.15.47:0/2659087284 learned_addr learned my addr 172.21.15.47:0/2659087284 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:42:24.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.996+0000 7fac29f41700 1 --2- 172.21.15.47:0/2659087284 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fac240fd450 0x7fac241080b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:24.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.996+0000 7fac29740700 1 --2- 172.21.15.47:0/2659087284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac240f7a50 0x7fac2410df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:24.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.997+0000 7fac29f41700 1 -- 172.21.15.47:0/2659087284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac240f7a50 msgr2=0x7fac2410df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:24.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.997+0000 7fac29f41700 1 --2- 172.21.15.47:0/2659087284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac240f7a50 0x7fac2410df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:24.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.997+0000 7fac29f41700 1 -- 172.21.15.47:0/2659087284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac240fcaf0 msgr2=0x7fac2410e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:24.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.997+0000 7fac29f41700 1 --2- 172.21.15.47:0/2659087284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac240fcaf0 0x7fac2410e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:24.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.997+0000 7fac29f41700 1 -- 172.21.15.47:0/2659087284 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac20005040 con 0x7fac240fd450 2024-02-23T00:42:24.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.997+0000 7fac29f41700 1 --2- 172.21.15.47:0/2659087284 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fac240fd450 0x7fac241080b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fac1c00eb70 tx=0x7fac1c00b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:24.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.997+0000 7fac1a7fc700 10 client.?.objecter ms_handle_connect 0x7fac240fd450 2024-02-23T00:42:24.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.997+0000 7fac1a7fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:42:24.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.997+0000 7fac1a7fc700 1 -- 172.21.15.47:0/2659087284 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c009870 con 0x7fac240fd450 2024-02-23T00:42:24.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.997+0000 7fac2b9a4700 1 -- 172.21.15.47:0/2659087284 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac24108b50 con 0x7fac240fd450 2024-02-23T00:42:24.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.997+0000 7fac1a7fc700 1 -- 172.21.15.47:0/2659087284 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fac1c00f040 con 0x7fac240fd450 2024-02-23T00:42:24.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.997+0000 7fac1a7fc700 1 -- 172.21.15.47:0/2659087284 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c007c70 con 0x7fac240fd450 2024-02-23T00:42:24.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.997+0000 7fac2b9a4700 10 client.24680.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:42:24.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.997+0000 7fac2b9a4700 1 -- 172.21.15.47:0/2659087284 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac24114d40 con 0x7fac240fd450 2024-02-23T00:42:24.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.997+0000 7fac2b9a4700 1 librados: init done 2024-02-23T00:42:24.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.998+0000 7fac2b9a4700 1 -- 172.21.15.47:0/2659087284 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac2405f990 con 0x7fac240fd450 2024-02-23T00:42:24.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.999+0000 7fac1a7fc700 1 -- 172.21.15.47:0/2659087284 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fac1c0040b0 con 0x7fac240fd450 2024-02-23T00:42:24.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.999+0000 7fac1a7fc700 1 --2- 172.21.15.47:0/2659087284 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fac10062cc0 0x7fac10065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:24.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.999+0000 7fac1a7fc700 1 -- 172.21.15.47:0/2659087284 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(99..99 src has 48..99) v4 ==== 7421+0+0 (secure 0 0 0) 0x7fac1c07cc90 con 0x7fac240fd450 2024-02-23T00:42:24.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.999+0000 7fac1a7fc700 10 client.24680.objecter ms_dispatch 0x7fac24001550 osd_map(99..99 src has 48..99) v4 2024-02-23T00:42:24.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.999+0000 7fac1a7fc700 3 client.24680.objecter handle_osd_map got epochs [99,99] > 0 2024-02-23T00:42:24.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.999+0000 7fac1a7fc700 3 client.24680.objecter handle_osd_map decoding full epoch 99 2024-02-23T00:42:24.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.999+0000 7fac29740700 1 --2- 172.21.15.47:0/2659087284 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fac10062cc0 0x7fac10065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:24.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:23.999+0000 7fac1a7fc700 20 client.24680.objecter dump_active .. 0 homeless 2024-02-23T00:42:24.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.000+0000 7fac1a7fc700 10 client.24680.objecter ms_handle_connect 0x7fac10062cc0 2024-02-23T00:42:24.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.000+0000 7fac29740700 1 --2- 172.21.15.47:0/2659087284 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fac10062cc0 0x7fac10065180 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fac20004ee0 tx=0x7fac2000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:24.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.007+0000 7fac1a7fc700 1 -- 172.21.15.47:0/2659087284 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fac1c049f60 con 0x7fac240fd450 2024-02-23T00:42:24.062 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:42:24.096 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:24.251 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:42:24.257 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:24.257 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:42:24.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.349+0000 7fac2b9a4700 1 -- 172.21.15.47:0/2659087284 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7fac24106fc0 con 0x7fac240fd450 2024-02-23T00:42:24.359 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:42:24.402 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:24.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.410+0000 7f617affd700 1 -- 172.21.15.47:0/609245625 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]=0 set osd.0 primary-affinity to 1 (8655362) v100) v1 ==== 133+0+0 (secure 0 0 0) 0x7f617c04dab0 con 0x7f6184104660 2024-02-23T00:42:24.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.414+0000 7f618bf99700 10 librados: watch_flush enter 2024-02-23T00:42:24.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.414+0000 7f618bf99700 10 librados: watch_flush exit 2024-02-23T00:42:24.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.414+0000 7f618bf99700 20 client.14784.objecter shutdown clearing up homeless session... 2024-02-23T00:42:24.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.414+0000 7f618bf99700 10 client.14784.objecter successfully canceled tick 2024-02-23T00:42:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.415+0000 7fac1a7fc700 1 -- 172.21.15.47:0/2659087284 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v100) v1 ==== 97+0+0 (secure 0 0 0) 0x7fac1c04c990 con 0x7fac240fd450 2024-02-23T00:42:24.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.416+0000 7f618bf99700 1 -- 172.21.15.47:0/609245625 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6170062cc0 msgr2=0x7f6170065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:24.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.416+0000 7f618bf99700 1 --2- 172.21.15.47:0/609245625 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6170062cc0 0x7f6170065180 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f6180009990 tx=0x7f6180008040).stop 2024-02-23T00:42:24.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.416+0000 7f618bf99700 1 -- 172.21.15.47:0/609245625 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6184104660 msgr2=0x7f6184100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:24.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.416+0000 7f618bf99700 1 --2- 172.21.15.47:0/609245625 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6184104660 0x7f6184100d40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f617c009760 tx=0x7f617c004d10).stop 2024-02-23T00:42:24.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.418+0000 7f618bf99700 1 -- 172.21.15.47:0/609245625 shutdown_connections 2024-02-23T00:42:24.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.418+0000 7f618bf99700 1 --2- 172.21.15.47:0/609245625 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6184104660 0x7f6184100d40 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:24.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.418+0000 7f618bf99700 1 --2- 172.21.15.47:0/609245625 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6170062cc0 0x7f6170065180 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:24.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.418+0000 7f618bf99700 1 --2- 172.21.15.47:0/609245625 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f61840f6b00 0x7f6184103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:24.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.418+0000 7f618bf99700 1 --2- 172.21.15.47:0/609245625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61840f73d0 0x7f6184103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:24.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.418+0000 7f618bf99700 1 -- 172.21.15.47:0/609245625 >> 172.21.15.47:0/609245625 conn(0x7f61840f2820 msgr2=0x7f61840f4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:42:24.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.422+0000 7f618bf99700 1 -- 172.21.15.47:0/609245625 shutdown_connections 2024-02-23T00:42:24.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.424+0000 7f618bf99700 1 -- 172.21.15.47:0/609245625 wait complete. 2024-02-23T00:42:24.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.424+0000 7f618bf99700 1 librados: shutdown 2024-02-23T00:42:24.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.422+0000 7fac2b9a4700 10 librados: watch_flush enter 2024-02-23T00:42:24.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.422+0000 7fac2b9a4700 10 librados: watch_flush exit 2024-02-23T00:42:24.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.422+0000 7fac2b9a4700 20 client.24680.objecter shutdown clearing up homeless session... 2024-02-23T00:42:24.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.422+0000 7fac2b9a4700 10 client.24680.objecter successfully canceled tick 2024-02-23T00:42:24.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.425+0000 7fac2b9a4700 1 -- 172.21.15.47:0/2659087284 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fac10062cc0 msgr2=0x7fac10065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:24.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.425+0000 7fac2b9a4700 1 --2- 172.21.15.47:0/2659087284 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fac10062cc0 0x7fac10065180 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fac20004ee0 tx=0x7fac2000b040).stop 2024-02-23T00:42:24.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.425+0000 7fac2b9a4700 1 -- 172.21.15.47:0/2659087284 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fac240fd450 msgr2=0x7fac241080b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:24.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.425+0000 7fac2b9a4700 1 --2- 172.21.15.47:0/2659087284 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fac240fd450 0x7fac241080b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fac1c00eb70 tx=0x7fac1c00b040).stop 2024-02-23T00:42:24.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.425+0000 7fac2b9a4700 1 -- 172.21.15.47:0/2659087284 shutdown_connections 2024-02-23T00:42:24.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.425+0000 7fac2b9a4700 1 --2- 172.21.15.47:0/2659087284 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fac10062cc0 0x7fac10065180 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:24.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.425+0000 7fac2b9a4700 1 --2- 172.21.15.47:0/2659087284 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fac240fd450 0x7fac241080b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:24.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.425+0000 7fac2b9a4700 1 --2- 172.21.15.47:0/2659087284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac240f7a50 0x7fac2410df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:24.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.425+0000 7fac2b9a4700 1 --2- 172.21.15.47:0/2659087284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac240fcaf0 0x7fac2410e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:24.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.425+0000 7fac2b9a4700 1 -- 172.21.15.47:0/2659087284 >> 172.21.15.47:0/2659087284 conn(0x7fac24070030 msgr2=0x7fac2406d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:42:24.431 INFO:teuthology.orchestra.run.smithi047.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-02-23T00:42:24.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.429+0000 7fac2b9a4700 1 -- 172.21.15.47:0/2659087284 shutdown_connections 2024-02-23T00:42:24.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.429+0000 7fac2b9a4700 1 -- 172.21.15.47:0/2659087284 wait complete. 2024-02-23T00:42:24.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:24.429+0000 7fac2b9a4700 1 librados: shutdown 2024-02-23T00:42:24.440 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is set 2024-02-23T00:42:24.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:23.164452+0000 mgr.y (mgr.14517) 125 : cluster [DBG] pgmap v110: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:42:24.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:23.814631+0000 mon.a (mon.0) 775 : audit [INF] from='client.? 172.21.15.47:0/609245625' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-02-23T00:42:24.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:24.351314+0000 mon.c (mon.2) 25 : audit [INF] from='client.? 172.21.15.47:0/2659087284' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:42:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:24.354886+0000 mon.a (mon.0) 776 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:42:24.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:23.164452+0000 mgr.y (mgr.14517) 125 : cluster [DBG] pgmap v110: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:42:24.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:23.814631+0000 mon.a (mon.0) 775 : audit [INF] from='client.? 172.21.15.47:0/609245625' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-02-23T00:42:24.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:24.351314+0000 mon.c (mon.2) 25 : audit [INF] from='client.? 172.21.15.47:0/2659087284' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:42:24.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:24.354886+0000 mon.a (mon.0) 776 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:42:24.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:23.164452+0000 mgr.y (mgr.14517) 125 : cluster [DBG] pgmap v110: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:42:24.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:23.814631+0000 mon.a (mon.0) 775 : audit [INF] from='client.? 172.21.15.47:0/609245625' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-02-23T00:42:24.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:24.351314+0000 mon.c (mon.2) 25 : audit [INF] from='client.? 172.21.15.47:0/2659087284' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:42:24.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:24.354886+0000 mon.a (mon.0) 776 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:42:25.271 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:42:25.276 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:25.276 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:42:25.377 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:42:25.412 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:25.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:24.405036+0000 mon.a (mon.0) 777 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:42:25.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:24.411779+0000 mon.a (mon.0) 778 : audit [INF] from='client.? 172.21.15.47:0/609245625' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-02-23T00:42:25.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:24.411851+0000 mon.a (mon.0) 779 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:42:25.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:24.411906+0000 mon.a (mon.0) 780 : cluster [DBG] osdmap e100: 8 total, 8 up, 8 in 2024-02-23T00:42:25.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:24.405036+0000 mon.a (mon.0) 777 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:42:25.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:24.411779+0000 mon.a (mon.0) 778 : audit [INF] from='client.? 172.21.15.47:0/609245625' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-02-23T00:42:25.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:24.411851+0000 mon.a (mon.0) 779 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:42:25.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:24.411906+0000 mon.a (mon.0) 780 : cluster [DBG] osdmap e100: 8 total, 8 up, 8 in 2024-02-23T00:42:25.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:24.405036+0000 mon.a (mon.0) 777 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:42:25.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:24.411779+0000 mon.a (mon.0) 778 : audit [INF] from='client.? 172.21.15.47:0/609245625' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-02-23T00:42:25.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:24.411851+0000 mon.a (mon.0) 779 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:42:25.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:24.411906+0000 mon.a (mon.0) 780 : cluster [DBG] osdmap e100: 8 total, 8 up, 8 in 2024-02-23T00:42:25.770 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:42:25.776 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:25.777 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:42:25.877 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:42:25.909 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:25.931 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:42:25.933 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:25.934 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:42:26.035 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:42:26.066 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:26.302 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:42:26.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:25.164837+0000 mgr.y (mgr.14517) 126 : cluster [DBG] pgmap v112: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:42:26.701 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:25.164837+0000 mgr.y (mgr.14517) 126 : cluster [DBG] pgmap v112: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:42:26.702 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:25.164837+0000 mgr.y (mgr.14517) 126 : cluster [DBG] pgmap v112: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:42:26.718 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:42:26.728 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:26.729 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:42:26.829 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:42:26.858 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:26.892 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:42:26.895 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:26.895 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:42:26.996 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:42:27.028 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:27.049 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:42:27.052 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:27.052 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:42:27.153 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:42:27.186 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:27.541 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:42:27.545 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:27.545 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:42:27.646 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:42:27.683 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:27.939 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:42:27.941 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:27.942 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:42:28.043 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:42:28.075 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:28.097 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:42:28.100 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:28.100 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:42:28.201 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:42:28.241 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:28.312 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset noscrub 2024-02-23T00:42:28.492 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:42:28.495 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:27.165588+0000 mgr.y (mgr.14517) 127 : cluster [DBG] pgmap v113: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-23T00:42:28.495 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:27.165588+0000 mgr.y (mgr.14517) 127 : cluster [DBG] pgmap v113: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-23T00:42:28.495 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:28.496 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:42:28.597 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:42:28.600 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:42:28.600 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 20, 2024-02-23T00:42:28.600 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 600, 2024-02-23T00:42:28.600 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [ 2024-02-23T00:42:28.600 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.600 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24311.0:102 5.3 5:c609908c:::notify.5:head [watch watch cookie 94587070403072] snapc 0=[] ondisk+write+known_if_redirected e64)", 2024-02-23T00:42:28.600 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:39:18.811442+0000", 2024-02-23T00:42:28.600 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 188.78208580800001, 2024-02-23T00:42:28.600 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0016704179999999999, 2024-02-23T00:42:28.601 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:42:28.601 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:42:28.601 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:42:28.601 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24311", 2024-02-23T00:42:28.601 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/3723202188", 2024-02-23T00:42:28.601 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 102 2024-02-23T00:42:28.601 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.601 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:42:28.601 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.601 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:42:28.601 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:18.811442+0000", 2024-02-23T00:42:28.601 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:42:28.602 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.602 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.602 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:42:28.602 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:18.811442+0000", 2024-02-23T00:42:28.602 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.999999 2024-02-23T00:42:28.602 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.602 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.602 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:42:28.602 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:18.811441+0000", 2024-02-23T00:42:28.602 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.2520000000000002e-06 2024-02-23T00:42:28.602 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.602 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.603 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:42:28.603 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:18.811448+0000", 2024-02-23T00:42:28.603 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.043e-05 2024-02-23T00:42:28.603 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.603 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.603 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:42:28.603 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:18.811459+0000", 2024-02-23T00:42:28.603 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.7400000000000004e-06 2024-02-23T00:42:28.603 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.603 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.603 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:42:28.603 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:18.811467+0000", 2024-02-23T00:42:28.604 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.1523000000000003e-05 2024-02-23T00:42:28.604 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.604 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.604 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:42:28.604 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:18.811498+0000", 2024-02-23T00:42:28.604 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.2312999999999997e-05 2024-02-23T00:42:28.604 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.604 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.604 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for rw locks", 2024-02-23T00:42:28.604 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:18.811570+0000", 2024-02-23T00:42:28.604 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00029964999999999999 2024-02-23T00:42:28.604 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.605 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.605 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:42:28.605 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:18.811870+0000", 2024-02-23T00:42:28.605 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.9330000000000002e-05 2024-02-23T00:42:28.605 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.605 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.606 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:42:28.606 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:18.811939+0000", 2024-02-23T00:42:28.606 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00016896599999999999 2024-02-23T00:42:28.606 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.606 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.606 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 5,6", 2024-02-23T00:42:28.606 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:18.812108+0000", 2024-02-23T00:42:28.606 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000439517 2024-02-23T00:42:28.606 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.606 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.606 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:42:28.606 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:18.812548+0000", 2024-02-23T00:42:28.607 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00021023099999999999 2024-02-23T00:42:28.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.607 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.607 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.607 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:18.812758+0000", 2024-02-23T00:42:28.607 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000265028 2024-02-23T00:42:28.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.607 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.607 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.607 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:18.813023+0000", 2024-02-23T00:42:28.607 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.2320000000000002e-05 2024-02-23T00:42:28.608 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.608 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.608 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:42:28.608 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:18.813055+0000", 2024-02-23T00:42:28.608 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.7188999999999997e-05 2024-02-23T00:42:28.608 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.608 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.608 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:42:28.608 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:18.813113+0000", 2024-02-23T00:42:28.608 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0016704179999999999 2024-02-23T00:42:28.608 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.609 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:42:28.609 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.609 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.609 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.609 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24341.0:64 5.3 5:c609908c:::notify.5:head [watch watch cookie 94435141295584] snapc 0=[] ondisk+write+known_if_redirected e64)", 2024-02-23T00:42:28.609 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:39:18.812021+0000", 2024-02-23T00:42:28.609 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 188.78150747000001, 2024-02-23T00:42:28.609 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0022579369999999998, 2024-02-23T00:42:28.609 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:42:28.609 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:42:28.609 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:42:28.610 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24341", 2024-02-23T00:42:28.610 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/752447959", 2024-02-23T00:42:28.610 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 64 2024-02-23T00:42:28.610 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.610 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:42:28.610 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.610 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:42:28.610 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:18.812021+0000", 2024-02-23T00:42:28.610 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:42:28.610 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.610 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.610 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:42:28.611 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:18.812021+0000", 2024-02-23T00:42:28.611 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999986 2024-02-23T00:42:28.611 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.611 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.611 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:42:28.611 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:18.812019+0000", 2024-02-23T00:42:28.611 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.9179999999999994e-06 2024-02-23T00:42:28.611 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.611 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.611 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:42:28.611 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:18.812027+0000", 2024-02-23T00:42:28.611 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3813e-05 2024-02-23T00:42:28.612 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.612 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.612 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:42:28.612 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:18.812041+0000", 2024-02-23T00:42:28.612 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.8759999999999994e-06 2024-02-23T00:42:28.612 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.612 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.612 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:42:28.612 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:18.812050+0000", 2024-02-23T00:42:28.612 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00030912099999999998 2024-02-23T00:42:28.612 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.612 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.612 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:42:28.613 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:18.812359+0000", 2024-02-23T00:42:28.613 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.5003e-05 2024-02-23T00:42:28.613 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.613 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.613 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for rw locks", 2024-02-23T00:42:28.614 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:18.812434+0000", 2024-02-23T00:42:28.614 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00072214600000000001 2024-02-23T00:42:28.614 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.614 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.614 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:42:28.614 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:18.813156+0000", 2024-02-23T00:42:28.614 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.6188999999999998e-05 2024-02-23T00:42:28.614 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.614 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.614 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:42:28.614 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:18.813232+0000", 2024-02-23T00:42:28.614 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00016630400000000001 2024-02-23T00:42:28.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.615 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.615 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 5,6", 2024-02-23T00:42:28.615 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:18.813398+0000", 2024-02-23T00:42:28.615 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00036940999999999997 2024-02-23T00:42:28.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.615 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.615 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:42:28.615 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:18.813768+0000", 2024-02-23T00:42:28.615 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00032894499999999998 2024-02-23T00:42:28.616 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.616 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.616 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.616 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:18.814097+0000", 2024-02-23T00:42:28.616 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.5347999999999999e-05 2024-02-23T00:42:28.616 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.616 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.616 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.616 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:18.814152+0000", 2024-02-23T00:42:28.617 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.9784000000000003e-05 2024-02-23T00:42:28.617 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.617 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.617 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:42:28.617 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:18.814202+0000", 2024-02-23T00:42:28.617 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.6528999999999999e-05 2024-02-23T00:42:28.617 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.617 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.617 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:42:28.617 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:18.814278+0000", 2024-02-23T00:42:28.617 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0022579369999999998 2024-02-23T00:42:28.617 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.617 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:42:28.617 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.618 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.618 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.618 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.14460.0:140 4.6 4:66924383:reshard::reshard.0000000005:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected e67)", 2024-02-23T00:42:28.618 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:39:21.883094+0000", 2024-02-23T00:42:28.618 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 185.71043425100001, 2024-02-23T00:42:28.618 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.003579558, 2024-02-23T00:42:28.618 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:42:28.618 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:42:28.618 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:42:28.618 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.14460", 2024-02-23T00:42:28.618 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1573276039", 2024-02-23T00:42:28.618 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 140 2024-02-23T00:42:28.618 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.618 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:42:28.618 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:42:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.883094+0000", 2024-02-23T00:42:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:42:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:42:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.883094+0000", 2024-02-23T00:42:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:42:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:42:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.883093+0000", 2024-02-23T00:42:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.825e-06 2024-02-23T00:42:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.620 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:42:28.620 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.883103+0000", 2024-02-23T00:42:28.620 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.3600000000000001e-06 2024-02-23T00:42:28.620 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.620 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.620 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:42:28.620 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.883109+0000", 2024-02-23T00:42:28.620 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8529999999999992e-06 2024-02-23T00:42:28.620 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.620 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.620 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:42:28.620 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.883117+0000", 2024-02-23T00:42:28.620 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.9536000000000005e-05 2024-02-23T00:42:28.620 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:42:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.883207+0000", 2024-02-23T00:42:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000118379 2024-02-23T00:42:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:42:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.883325+0000", 2024-02-23T00:42:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000172256 2024-02-23T00:42:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 1,2", 2024-02-23T00:42:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.883498+0000", 2024-02-23T00:42:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00076410499999999995 2024-02-23T00:42:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:42:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.884262+0000", 2024-02-23T00:42:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001524292 2024-02-23T00:42:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.885786+0000", 2024-02-23T00:42:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00080175099999999996 2024-02-23T00:42:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.886588+0000", 2024-02-23T00:42:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.7649999999999999e-05 2024-02-23T00:42:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:42:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.886635+0000", 2024-02-23T00:42:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.8000000000000002e-05 2024-02-23T00:42:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:42:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.886673+0000", 2024-02-23T00:42:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.003579558 2024-02-23T00:42:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:42:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.14460.0:152 4.6 4:66924383:reshard::reshard.0000000005:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected e67)", 2024-02-23T00:42:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:39:21.888064+0000", 2024-02-23T00:42:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 185.70546441100001, 2024-02-23T00:42:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0032968920000000001, 2024-02-23T00:42:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:42:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:42:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:42:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.14460", 2024-02-23T00:42:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1573276039", 2024-02-23T00:42:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 152 2024-02-23T00:42:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:42:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:42:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.888064+0000", 2024-02-23T00:42:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:42:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:42:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.888064+0000", 2024-02-23T00:42:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999986 2024-02-23T00:42:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:42:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.888062+0000", 2024-02-23T00:42:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.9090000000000006e-06 2024-02-23T00:42:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:42:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.888071+0000", 2024-02-23T00:42:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4683e-05 2024-02-23T00:42:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:42:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.888086+0000", 2024-02-23T00:42:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.1629999999999999e-06 2024-02-23T00:42:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:42:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.888093+0000", 2024-02-23T00:42:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.4929000000000003e-05 2024-02-23T00:42:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:42:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.888138+0000", 2024-02-23T00:42:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.3318000000000003e-05 2024-02-23T00:42:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:42:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.888231+0000", 2024-02-23T00:42:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00018802700000000001 2024-02-23T00:42:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 1,2", 2024-02-23T00:42:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.888419+0000", 2024-02-23T00:42:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00091187199999999999 2024-02-23T00:42:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:42:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.889331+0000", 2024-02-23T00:42:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00016116599999999999 2024-02-23T00:42:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.889492+0000", 2024-02-23T00:42:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0017811000000000001 2024-02-23T00:42:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.891274+0000", 2024-02-23T00:42:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.8683999999999997e-05 2024-02-23T00:42:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.629 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.629 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:42:28.629 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.891322+0000", 2024-02-23T00:42:28.629 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.8279999999999999e-05 2024-02-23T00:42:28.629 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.629 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.629 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:42:28.629 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.891360+0000", 2024-02-23T00:42:28.629 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0032968920000000001 2024-02-23T00:42:28.629 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.630 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:42:28.630 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.630 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.630 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.630 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.14460.0:153 4.3 4:c60da066:gc::gc.4:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected e67)", 2024-02-23T00:42:28.630 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:39:21.888106+0000", 2024-02-23T00:42:28.631 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 185.705421973, 2024-02-23T00:42:28.631 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0028418050000000002, 2024-02-23T00:42:28.631 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:42:28.631 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:42:28.631 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:42:28.631 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.14460", 2024-02-23T00:42:28.631 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1573276039", 2024-02-23T00:42:28.631 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 153 2024-02-23T00:42:28.631 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.632 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:42:28.632 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.632 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:42:28.632 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.888106+0000", 2024-02-23T00:42:28.632 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:42:28.632 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.632 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.633 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:42:28.633 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.888106+0000", 2024-02-23T00:42:28.633 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.999999 2024-02-23T00:42:28.633 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.633 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.633 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:42:28.634 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.888105+0000", 2024-02-23T00:42:28.634 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.2450000000000003e-06 2024-02-23T00:42:28.634 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.634 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.634 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:42:28.634 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.888111+0000", 2024-02-23T00:42:28.635 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.1420000000000004e-06 2024-02-23T00:42:28.635 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.635 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.635 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:42:28.635 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.888119+0000", 2024-02-23T00:42:28.635 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.6290000000000001e-06 2024-02-23T00:42:28.635 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.635 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.636 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:42:28.636 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.888123+0000", 2024-02-23T00:42:28.636 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000587113 2024-02-23T00:42:28.636 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.636 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.636 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:42:28.636 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.888710+0000", 2024-02-23T00:42:28.636 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8467999999999995e-05 2024-02-23T00:42:28.636 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.636 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.637 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:42:28.637 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.888789+0000", 2024-02-23T00:42:28.637 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00019196299999999999 2024-02-23T00:42:28.637 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.637 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.637 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 5,7", 2024-02-23T00:42:28.637 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.888981+0000", 2024-02-23T00:42:28.638 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000489519 2024-02-23T00:42:28.638 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.638 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.638 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:42:28.638 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.889470+0000", 2024-02-23T00:42:28.638 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00061353199999999997 2024-02-23T00:42:28.638 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.639 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.639 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.639 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.890084+0000", 2024-02-23T00:42:28.639 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00078013300000000002 2024-02-23T00:42:28.639 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.639 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.639 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.639 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.890864+0000", 2024-02-23T00:42:28.640 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.7809000000000003e-05 2024-02-23T00:42:28.640 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.640 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.640 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:42:28.640 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.890912+0000", 2024-02-23T00:42:28.640 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.6059000000000002e-05 2024-02-23T00:42:28.640 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.641 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.641 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:42:28.641 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.890948+0000", 2024-02-23T00:42:28.641 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0028418050000000002 2024-02-23T00:42:28.641 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.641 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:42:28.641 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.641 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.642 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.642 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.14460.0:155 4.1e 4:7e82ab18:lc::lc.13:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected e67)", 2024-02-23T00:42:28.642 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:39:21.888136+0000", 2024-02-23T00:42:28.642 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 185.70539193299999, 2024-02-23T00:42:28.642 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0023594599999999999, 2024-02-23T00:42:28.642 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:42:28.642 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:42:28.642 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:42:28.643 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.14460", 2024-02-23T00:42:28.643 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1573276039", 2024-02-23T00:42:28.643 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 155 2024-02-23T00:42:28.643 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.643 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:42:28.643 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.643 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:42:28.644 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.888136+0000", 2024-02-23T00:42:28.644 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:42:28.644 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.644 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.644 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:42:28.644 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.888136+0000", 2024-02-23T00:42:28.644 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:42:28.644 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.645 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.645 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:42:28.645 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.888135+0000", 2024-02-23T00:42:28.645 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.2810000000000002e-06 2024-02-23T00:42:28.645 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.645 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.645 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:42:28.646 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.888140+0000", 2024-02-23T00:42:28.646 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.5530000000000004e-06 2024-02-23T00:42:28.646 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.646 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.646 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:42:28.646 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.888147+0000", 2024-02-23T00:42:28.646 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.6670000000000004e-06 2024-02-23T00:42:28.646 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.647 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.647 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:42:28.647 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.888152+0000", 2024-02-23T00:42:28.647 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.5648000000000001e-05 2024-02-23T00:42:28.647 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.647 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.647 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:42:28.648 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.888198+0000", 2024-02-23T00:42:28.648 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.9375999999999994e-05 2024-02-23T00:42:28.648 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.648 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.648 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:42:28.648 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.888277+0000", 2024-02-23T00:42:28.648 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00019205700000000001 2024-02-23T00:42:28.649 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.649 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.649 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 3,7", 2024-02-23T00:42:28.649 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.888469+0000", 2024-02-23T00:42:28.649 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00085116399999999996 2024-02-23T00:42:28.649 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.649 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.649 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.650 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.889320+0000", 2024-02-23T00:42:28.650 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00015068699999999999 2024-02-23T00:42:28.650 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.650 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.650 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:42:28.650 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.889471+0000", 2024-02-23T00:42:28.650 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00094138100000000003 2024-02-23T00:42:28.651 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.651 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.651 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.651 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.890412+0000", 2024-02-23T00:42:28.651 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.7821000000000002e-05 2024-02-23T00:42:28.651 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.651 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.651 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:42:28.651 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.890460+0000", 2024-02-23T00:42:28.651 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.5482999999999999e-05 2024-02-23T00:42:28.651 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.651 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.652 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:42:28.652 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.890496+0000", 2024-02-23T00:42:28.652 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0023594599999999999 2024-02-23T00:42:28.652 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.652 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:42:28.652 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.652 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.652 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.653 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.14460.0:159 4.b 4:d5e0c857:lc::lc.5:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected e67)", 2024-02-23T00:42:28.653 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:39:21.890776+0000", 2024-02-23T00:42:28.653 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 185.702752495, 2024-02-23T00:42:28.653 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0020400150000000001, 2024-02-23T00:42:28.653 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:42:28.653 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:42:28.653 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:42:28.654 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.14460", 2024-02-23T00:42:28.654 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1573276039", 2024-02-23T00:42:28.654 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 159 2024-02-23T00:42:28.654 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.654 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:42:28.654 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.654 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:42:28.654 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.890776+0000", 2024-02-23T00:42:28.654 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:42:28.654 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.654 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.654 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:42:28.654 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.890776+0000", 2024-02-23T00:42:28.654 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.999999 2024-02-23T00:42:28.654 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.655 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.655 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:42:28.655 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.890775+0000", 2024-02-23T00:42:28.655 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.2419999999999997e-06 2024-02-23T00:42:28.655 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.655 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.655 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:42:28.655 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.890781+0000", 2024-02-23T00:42:28.655 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2034e-05 2024-02-23T00:42:28.655 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.655 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.655 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:42:28.655 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.890793+0000", 2024-02-23T00:42:28.655 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.553e-06 2024-02-23T00:42:28.656 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.656 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.656 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:42:28.656 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.890798+0000", 2024-02-23T00:42:28.656 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00017803900000000001 2024-02-23T00:42:28.656 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.656 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.656 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:42:28.656 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.890976+0000", 2024-02-23T00:42:28.656 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.4056999999999999e-05 2024-02-23T00:42:28.656 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.656 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.656 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:42:28.657 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.891050+0000", 2024-02-23T00:42:28.657 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000160784 2024-02-23T00:42:28.657 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.657 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.657 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 1,4", 2024-02-23T00:42:28.657 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.891211+0000", 2024-02-23T00:42:28.657 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00036531299999999998 2024-02-23T00:42:28.657 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.657 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.657 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:42:28.657 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.891577+0000", 2024-02-23T00:42:28.657 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000724576 2024-02-23T00:42:28.657 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.657 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.657 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.658 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.892301+0000", 2024-02-23T00:42:28.658 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00043468499999999998 2024-02-23T00:42:28.658 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.658 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.658 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.658 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.892736+0000", 2024-02-23T00:42:28.658 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.3665000000000003e-05 2024-02-23T00:42:28.658 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.658 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.658 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:42:28.658 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.892779+0000", 2024-02-23T00:42:28.658 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.6047000000000002e-05 2024-02-23T00:42:28.658 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.658 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.658 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:42:28.659 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.892816+0000", 2024-02-23T00:42:28.659 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0020400150000000001 2024-02-23T00:42:28.659 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.659 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:42:28.659 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.659 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.659 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.659 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.14460.0:179 4.16 4:6a988a9b:reshard::reshard.0000000007:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected e67)", 2024-02-23T00:42:28.659 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:39:21.896063+0000", 2024-02-23T00:42:28.659 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 185.69746487, 2024-02-23T00:42:28.659 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0016405009999999999, 2024-02-23T00:42:28.659 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:42:28.659 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:42:28.659 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:42:28.659 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.14460", 2024-02-23T00:42:28.660 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1573276039", 2024-02-23T00:42:28.660 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 179 2024-02-23T00:42:28.660 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.660 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:42:28.660 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.660 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:42:28.660 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.896063+0000", 2024-02-23T00:42:28.660 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:42:28.660 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.660 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.660 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:42:28.660 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.896063+0000", 2024-02-23T00:42:28.660 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.999999 2024-02-23T00:42:28.661 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.661 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.661 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:42:28.661 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.896062+0000", 2024-02-23T00:42:28.661 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.6640000000000003e-06 2024-02-23T00:42:28.661 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.661 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.661 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:42:28.661 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.896068+0000", 2024-02-23T00:42:28.661 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.419e-05 2024-02-23T00:42:28.661 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.661 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.661 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:42:28.661 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.896082+0000", 2024-02-23T00:42:28.661 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.006e-05 2024-02-23T00:42:28.662 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.662 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.662 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:42:28.662 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.896092+0000", 2024-02-23T00:42:28.662 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.5535000000000002e-05 2024-02-23T00:42:28.662 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.662 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.662 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:42:28.662 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.896158+0000", 2024-02-23T00:42:28.662 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00011319000000000001 2024-02-23T00:42:28.662 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.662 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.662 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:42:28.662 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.896271+0000", 2024-02-23T00:42:28.662 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00020222299999999999 2024-02-23T00:42:28.663 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.663 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.663 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 3,7", 2024-02-23T00:42:28.663 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.896473+0000", 2024-02-23T00:42:28.663 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00028732399999999999 2024-02-23T00:42:28.663 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.663 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.663 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:42:28.663 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.896760+0000", 2024-02-23T00:42:28.663 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00056229200000000004 2024-02-23T00:42:28.663 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.663 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.663 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.663 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.897323+0000", 2024-02-23T00:42:28.663 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000301482 2024-02-23T00:42:28.664 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.664 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.664 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.664 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.897624+0000", 2024-02-23T00:42:28.664 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.4189e-05 2024-02-23T00:42:28.664 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.664 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.664 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:42:28.664 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.897668+0000", 2024-02-23T00:42:28.664 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.5287999999999998e-05 2024-02-23T00:42:28.664 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.664 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.664 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:42:28.664 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.897704+0000", 2024-02-23T00:42:28.665 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0016405009999999999 2024-02-23T00:42:28.665 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.665 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:42:28.665 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.665 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.665 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.665 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.14460.0:206 4.16 4:6b0d4d95:lc::lc.20:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected e67)", 2024-02-23T00:42:28.665 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:39:21.902696+0000", 2024-02-23T00:42:28.665 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 185.690831648, 2024-02-23T00:42:28.665 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001696807, 2024-02-23T00:42:28.665 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:42:28.665 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:42:28.665 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:42:28.665 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.14460", 2024-02-23T00:42:28.665 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1573276039", 2024-02-23T00:42:28.666 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 206 2024-02-23T00:42:28.666 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.666 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:42:28.666 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.666 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:42:28.666 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.902696+0000", 2024-02-23T00:42:28.666 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:42:28.666 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.666 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.667 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:42:28.667 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.902696+0000", 2024-02-23T00:42:28.667 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.999999 2024-02-23T00:42:28.667 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.667 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.667 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:42:28.667 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.902695+0000", 2024-02-23T00:42:28.668 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.0249999999999999e-06 2024-02-23T00:42:28.668 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.668 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.668 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:42:28.668 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.902701+0000", 2024-02-23T00:42:28.668 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1375e-05 2024-02-23T00:42:28.668 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.669 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.669 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:42:28.669 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.902713+0000", 2024-02-23T00:42:28.669 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.4900000000000003e-06 2024-02-23T00:42:28.669 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.669 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.669 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:42:28.669 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.902720+0000", 2024-02-23T00:42:28.669 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.5547000000000003e-05 2024-02-23T00:42:28.669 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.670 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.670 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:42:28.670 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.902766+0000", 2024-02-23T00:42:28.670 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012236899999999999 2024-02-23T00:42:28.670 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.670 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.670 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:42:28.670 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.902888+0000", 2024-02-23T00:42:28.670 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000204506 2024-02-23T00:42:28.670 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.670 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.670 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 3,7", 2024-02-23T00:42:28.670 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.903093+0000", 2024-02-23T00:42:28.670 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0011657849999999999 2024-02-23T00:42:28.671 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.671 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.671 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.671 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.904258+0000", 2024-02-23T00:42:28.671 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.3611000000000001e-05 2024-02-23T00:42:28.671 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.671 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.671 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.671 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.904292+0000", 2024-02-23T00:42:28.671 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6540000000000001e-05 2024-02-23T00:42:28.671 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.671 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.671 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:42:28.671 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.904309+0000", 2024-02-23T00:42:28.671 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.8905e-05 2024-02-23T00:42:28.672 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.672 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.672 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:42:28.672 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.904357+0000", 2024-02-23T00:42:28.672 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.5669e-05 2024-02-23T00:42:28.672 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.672 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.672 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:42:28.672 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.904393+0000", 2024-02-23T00:42:28.672 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001696807 2024-02-23T00:42:28.672 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.672 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:42:28.672 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.673 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.673 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.673 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.14460.0:223 4.6 4:64760185:::obj_delete_at_hint.0000000010:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected e67)", 2024-02-23T00:42:28.673 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:39:21.907757+0000", 2024-02-23T00:42:28.673 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 185.68577122599999, 2024-02-23T00:42:28.673 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0019687659999999998, 2024-02-23T00:42:28.673 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:42:28.673 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:42:28.673 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:42:28.673 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.14460", 2024-02-23T00:42:28.673 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1573276039", 2024-02-23T00:42:28.673 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 223 2024-02-23T00:42:28.673 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.673 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:42:28.673 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.674 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:42:28.674 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.907757+0000", 2024-02-23T00:42:28.674 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:42:28.674 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.674 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.674 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:42:28.674 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.907757+0000", 2024-02-23T00:42:28.674 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.999999 2024-02-23T00:42:28.674 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.674 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.674 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:42:28.674 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.907756+0000", 2024-02-23T00:42:28.674 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.8239999999999996e-06 2024-02-23T00:42:28.674 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.675 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.675 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:42:28.675 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.907762+0000", 2024-02-23T00:42:28.675 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2176e-05 2024-02-23T00:42:28.675 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.675 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.675 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:42:28.675 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.907774+0000", 2024-02-23T00:42:28.675 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.2990000000000002e-06 2024-02-23T00:42:28.675 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.675 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.675 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:42:28.675 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.907783+0000", 2024-02-23T00:42:28.675 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.5315999999999998e-05 2024-02-23T00:42:28.675 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.676 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.676 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:42:28.676 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.907828+0000", 2024-02-23T00:42:28.676 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.9258999999999997e-05 2024-02-23T00:42:28.676 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.676 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.676 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:42:28.676 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.907878+0000", 2024-02-23T00:42:28.676 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000189011 2024-02-23T00:42:28.676 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.676 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.676 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 1,2", 2024-02-23T00:42:28.676 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.908067+0000", 2024-02-23T00:42:28.677 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00061395000000000004 2024-02-23T00:42:28.677 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.677 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.677 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:42:28.677 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.908681+0000", 2024-02-23T00:42:28.677 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000505191 2024-02-23T00:42:28.677 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.677 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.677 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.677 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.909186+0000", 2024-02-23T00:42:28.677 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000459297 2024-02-23T00:42:28.677 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.677 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.677 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.677 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.909645+0000", 2024-02-23T00:42:28.678 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.5590000000000002e-05 2024-02-23T00:42:28.678 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.678 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.678 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:42:28.678 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.909691+0000", 2024-02-23T00:42:28.678 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.4866999999999997e-05 2024-02-23T00:42:28.678 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.678 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.678 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:42:28.678 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.909726+0000", 2024-02-23T00:42:28.678 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0019687659999999998 2024-02-23T00:42:28.678 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.678 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:42:28.678 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.678 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.679 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.679 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.14460.0:241 4.16 4:6b0d4d95:lc::lc.20:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected e67)", 2024-02-23T00:42:28.679 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:39:21.911802+0000", 2024-02-23T00:42:28.679 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 185.681726217, 2024-02-23T00:42:28.679 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00177815, 2024-02-23T00:42:28.679 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:42:28.679 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:42:28.679 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:42:28.679 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.14460", 2024-02-23T00:42:28.679 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1573276039", 2024-02-23T00:42:28.679 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 241 2024-02-23T00:42:28.680 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.680 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:42:28.680 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.680 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:42:28.680 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.911802+0000", 2024-02-23T00:42:28.680 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:42:28.680 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.680 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.680 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:42:28.680 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.911802+0000", 2024-02-23T00:42:28.680 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.999999 2024-02-23T00:42:28.680 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.681 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.681 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:42:28.681 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.911801+0000", 2024-02-23T00:42:28.681 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.0039999999999999e-06 2024-02-23T00:42:28.681 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.681 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.681 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:42:28.681 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.911806+0000", 2024-02-23T00:42:28.681 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.9699999999999994e-06 2024-02-23T00:42:28.681 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.681 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.681 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:42:28.681 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.911816+0000", 2024-02-23T00:42:28.682 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.3489999999999997e-06 2024-02-23T00:42:28.682 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.682 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.682 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:42:28.682 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.911822+0000", 2024-02-23T00:42:28.682 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.4354999999999998e-05 2024-02-23T00:42:28.682 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.682 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.682 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:42:28.682 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.911867+0000", 2024-02-23T00:42:28.682 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.5179999999999995e-05 2024-02-23T00:42:28.683 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.683 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.683 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:42:28.683 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.911942+0000", 2024-02-23T00:42:28.683 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000174668 2024-02-23T00:42:28.683 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.683 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.683 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 3,7", 2024-02-23T00:42:28.684 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.912117+0000", 2024-02-23T00:42:28.684 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000592546 2024-02-23T00:42:28.684 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.684 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.684 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.684 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.912709+0000", 2024-02-23T00:42:28.684 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00020293399999999999 2024-02-23T00:42:28.685 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.685 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.685 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:42:28.685 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.912912+0000", 2024-02-23T00:42:28.685 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00058646800000000004 2024-02-23T00:42:28.685 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.685 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.685 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.686 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.913498+0000", 2024-02-23T00:42:28.686 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.6913000000000003e-05 2024-02-23T00:42:28.686 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.686 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.686 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:42:28.686 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.913545+0000", 2024-02-23T00:42:28.687 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.4566000000000001e-05 2024-02-23T00:42:28.687 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.687 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.687 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:42:28.687 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.913580+0000", 2024-02-23T00:42:28.687 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00177815 2024-02-23T00:42:28.687 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.687 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:42:28.687 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.687 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.687 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.687 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.14460.0:289 4.3 4:c57bc89b:::obj_delete_at_hint.0000000013:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected e67)", 2024-02-23T00:42:28.687 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:39:21.921919+0000", 2024-02-23T00:42:28.687 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 185.671608529, 2024-02-23T00:42:28.687 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002155213, 2024-02-23T00:42:28.688 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:42:28.688 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:42:28.688 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:42:28.688 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.14460", 2024-02-23T00:42:28.688 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1573276039", 2024-02-23T00:42:28.688 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 289 2024-02-23T00:42:28.688 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.688 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:42:28.688 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.688 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:42:28.688 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.921919+0000", 2024-02-23T00:42:28.688 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:42:28.688 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.688 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.689 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:42:28.689 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.921919+0000", 2024-02-23T00:42:28.689 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.999999 2024-02-23T00:42:28.689 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.689 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.689 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:42:28.689 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.921918+0000", 2024-02-23T00:42:28.689 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.9920000000000007e-06 2024-02-23T00:42:28.689 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.689 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.689 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:42:28.689 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.921926+0000", 2024-02-23T00:42:28.689 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4601e-05 2024-02-23T00:42:28.689 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.690 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.690 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:42:28.690 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.921941+0000", 2024-02-23T00:42:28.690 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.3940000000000001e-06 2024-02-23T00:42:28.690 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.690 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.690 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:42:28.690 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.921947+0000", 2024-02-23T00:42:28.690 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00031193699999999998 2024-02-23T00:42:28.690 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.690 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.690 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:42:28.690 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.922259+0000", 2024-02-23T00:42:28.690 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.3324999999999993e-05 2024-02-23T00:42:28.690 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.691 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.691 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:42:28.691 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.922333+0000", 2024-02-23T00:42:28.691 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00016821100000000001 2024-02-23T00:42:28.691 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.691 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.691 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 5,7", 2024-02-23T00:42:28.691 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.922501+0000", 2024-02-23T00:42:28.691 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00091051399999999996 2024-02-23T00:42:28.691 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.691 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.691 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:42:28.691 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.923411+0000", 2024-02-23T00:42:28.691 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00051968000000000001 2024-02-23T00:42:28.691 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.692 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.692 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.692 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.923931+0000", 2024-02-23T00:42:28.692 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.2173999999999999e-05 2024-02-23T00:42:28.692 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.692 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.692 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.692 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.923983+0000", 2024-02-23T00:42:28.692 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.3874e-05 2024-02-23T00:42:28.692 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.692 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.692 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:42:28.692 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.924027+0000", 2024-02-23T00:42:28.692 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.7555000000000001e-05 2024-02-23T00:42:28.693 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.693 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.693 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:42:28.693 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.924075+0000", 2024-02-23T00:42:28.693 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002155213 2024-02-23T00:42:28.693 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.693 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:42:28.693 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.693 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.693 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.693 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.14460.0:361 4.16 4:6b0d4d95:lc::lc.20:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected e67)", 2024-02-23T00:42:28.693 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:39:21.939142+0000", 2024-02-23T00:42:28.693 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 185.65438573200001, 2024-02-23T00:42:28.693 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002368935, 2024-02-23T00:42:28.693 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:42:28.694 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:42:28.694 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:42:28.694 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.14460", 2024-02-23T00:42:28.694 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1573276039", 2024-02-23T00:42:28.694 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 361 2024-02-23T00:42:28.694 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.694 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:42:28.694 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.694 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:42:28.694 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.939142+0000", 2024-02-23T00:42:28.694 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:42:28.694 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.694 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.694 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:42:28.694 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.939142+0000", 2024-02-23T00:42:28.695 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.999999 2024-02-23T00:42:28.695 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.695 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.695 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:42:28.695 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.939141+0000", 2024-02-23T00:42:28.695 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.5390000000000002e-06 2024-02-23T00:42:28.695 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.695 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.695 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:42:28.695 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.939149+0000", 2024-02-23T00:42:28.695 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.5562e-05 2024-02-23T00:42:28.695 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.695 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.695 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:42:28.695 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.939174+0000", 2024-02-23T00:42:28.696 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.1209999999999998e-06 2024-02-23T00:42:28.696 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.696 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.696 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:42:28.696 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.939179+0000", 2024-02-23T00:42:28.696 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.4697999999999998e-05 2024-02-23T00:42:28.696 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.696 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.696 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:42:28.696 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.939224+0000", 2024-02-23T00:42:28.696 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.9083000000000002e-05 2024-02-23T00:42:28.696 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.696 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.696 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:42:28.696 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.939303+0000", 2024-02-23T00:42:28.697 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012870199999999999 2024-02-23T00:42:28.697 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.697 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.697 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 3,7", 2024-02-23T00:42:28.697 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.939432+0000", 2024-02-23T00:42:28.697 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00062134699999999996 2024-02-23T00:42:28.697 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.697 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.697 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.697 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.940053+0000", 2024-02-23T00:42:28.697 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001166353 2024-02-23T00:42:28.697 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.697 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.697 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:42:28.697 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.941220+0000", 2024-02-23T00:42:28.698 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00024023499999999999 2024-02-23T00:42:28.698 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.698 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.698 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.698 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.941460+0000", 2024-02-23T00:42:28.698 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0809000000000003e-05 2024-02-23T00:42:28.698 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.698 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.698 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:42:28.698 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.941491+0000", 2024-02-23T00:42:28.698 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0652000000000001e-05 2024-02-23T00:42:28.698 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.698 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.698 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:42:28.698 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.941511+0000", 2024-02-23T00:42:28.698 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002368935 2024-02-23T00:42:28.699 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.699 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:42:28.699 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.699 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.699 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.699 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(client.14460.0:360 4.0 e67/61 4:0667bb13:gc::gc.13:head v 67'7, mlcod=67'5)", 2024-02-23T00:42:28.699 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:39:21.939282+0000", 2024-02-23T00:42:28.699 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 185.65424564099999, 2024-02-23T00:42:28.699 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001996163, 2024-02-23T00:42:28.699 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:42:28.699 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:42:28.699 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:42:28.699 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.699 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:42:28.700 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.939282+0000", 2024-02-23T00:42:28.700 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:42:28.700 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.700 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.700 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:42:28.700 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.939282+0000", 2024-02-23T00:42:28.700 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.999999 2024-02-23T00:42:28.700 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.700 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.700 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:42:28.700 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.939281+0000", 2024-02-23T00:42:28.700 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.6709999999999994e-06 2024-02-23T00:42:28.700 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.700 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.700 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:42:28.701 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.939291+0000", 2024-02-23T00:42:28.701 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0599e-05 2024-02-23T00:42:28.701 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.701 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.701 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:42:28.701 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.939302+0000", 2024-02-23T00:42:28.701 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.7180000000000001e-06 2024-02-23T00:42:28.701 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.701 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.701 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:42:28.701 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.939307+0000", 2024-02-23T00:42:28.701 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.7236999999999993e-05 2024-02-23T00:42:28.701 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.701 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.701 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:42:28.702 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.939375+0000", 2024-02-23T00:42:28.702 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8490000000000001e-05 2024-02-23T00:42:28.702 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.702 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.702 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:42:28.702 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.939393+0000", 2024-02-23T00:42:28.702 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0018569050000000001 2024-02-23T00:42:28.702 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.702 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.702 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:42:28.702 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.941250+0000", 2024-02-23T00:42:28.702 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8441000000000001e-05 2024-02-23T00:42:28.702 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.702 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.702 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:42:28.703 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.941279+0000", 2024-02-23T00:42:28.703 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001996163 2024-02-23T00:42:28.703 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.703 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:42:28.703 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.703 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.703 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.703 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.14460.0:433 4.b 4:d201668e:::obj_delete_at_hint.0000000023:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected e67)", 2024-02-23T00:42:28.703 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:39:21.959815+0000", 2024-02-23T00:42:28.703 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 185.63371279699999, 2024-02-23T00:42:28.703 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0023397259999999999, 2024-02-23T00:42:28.703 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:42:28.703 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:42:28.703 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:42:28.703 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.14460", 2024-02-23T00:42:28.704 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1573276039", 2024-02-23T00:42:28.704 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 433 2024-02-23T00:42:28.704 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.704 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:42:28.704 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.704 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:42:28.704 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.959815+0000", 2024-02-23T00:42:28.704 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:42:28.704 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.704 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.704 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:42:28.704 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.959815+0000", 2024-02-23T00:42:28.704 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:42:28.704 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.704 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.705 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:42:28.705 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.959815+0000", 2024-02-23T00:42:28.705 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.9520000000000002e-06 2024-02-23T00:42:28.705 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.705 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.705 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:42:28.705 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.959820+0000", 2024-02-23T00:42:28.705 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1090000000000001e-05 2024-02-23T00:42:28.705 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.705 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.705 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:42:28.705 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.959831+0000", 2024-02-23T00:42:28.705 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.301e-06 2024-02-23T00:42:28.705 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.705 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.706 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:42:28.706 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.959835+0000", 2024-02-23T00:42:28.706 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00050332700000000003 2024-02-23T00:42:28.706 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.706 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.706 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:42:28.706 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.960338+0000", 2024-02-23T00:42:28.706 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.9874e-05 2024-02-23T00:42:28.706 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.706 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.706 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:42:28.706 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.960428+0000", 2024-02-23T00:42:28.706 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000143883 2024-02-23T00:42:28.706 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.706 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.707 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 1,4", 2024-02-23T00:42:28.707 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.960572+0000", 2024-02-23T00:42:28.707 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00032645599999999998 2024-02-23T00:42:28.707 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.707 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.707 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:42:28.707 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.960899+0000", 2024-02-23T00:42:28.707 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00062859500000000002 2024-02-23T00:42:28.707 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.707 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.707 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.707 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.961527+0000", 2024-02-23T00:42:28.707 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00054101699999999997 2024-02-23T00:42:28.707 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.707 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.708 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.708 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.962068+0000", 2024-02-23T00:42:28.708 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.4428999999999998e-05 2024-02-23T00:42:28.708 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.708 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.708 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:42:28.708 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.962113+0000", 2024-02-23T00:42:28.708 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.2372e-05 2024-02-23T00:42:28.708 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.708 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.708 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:42:28.708 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.962155+0000", 2024-02-23T00:42:28.708 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0023397259999999999 2024-02-23T00:42:28.709 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.709 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:42:28.709 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.709 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.709 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.709 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.14460.0:441 4.b 4:d5e0c857:lc::lc.5:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected e67)", 2024-02-23T00:42:28.709 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:39:21.962045+0000", 2024-02-23T00:42:28.709 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 185.631483377, 2024-02-23T00:42:28.709 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0016350119999999999, 2024-02-23T00:42:28.709 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:42:28.709 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:42:28.709 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:42:28.709 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.14460", 2024-02-23T00:42:28.709 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1573276039", 2024-02-23T00:42:28.709 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 441 2024-02-23T00:42:28.710 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.710 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:42:28.710 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.710 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:42:28.710 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.962045+0000", 2024-02-23T00:42:28.710 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:42:28.710 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.710 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.710 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:42:28.710 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.962045+0000", 2024-02-23T00:42:28.710 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:42:28.710 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.710 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.710 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:42:28.710 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.962044+0000", 2024-02-23T00:42:28.711 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.425e-06 2024-02-23T00:42:28.711 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.711 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.711 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:42:28.711 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.962048+0000", 2024-02-23T00:42:28.711 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.7589999999999997e-06 2024-02-23T00:42:28.711 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.711 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.711 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:42:28.711 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.962056+0000", 2024-02-23T00:42:28.711 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.775e-06 2024-02-23T00:42:28.711 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.711 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.711 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:42:28.711 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.962061+0000", 2024-02-23T00:42:28.712 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000207288 2024-02-23T00:42:28.712 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.712 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.712 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:42:28.712 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.962268+0000", 2024-02-23T00:42:28.712 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.8337999999999997e-05 2024-02-23T00:42:28.712 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.712 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.712 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:42:28.712 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.962317+0000", 2024-02-23T00:42:28.712 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.5931e-05 2024-02-23T00:42:28.712 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.712 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.712 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 1,4", 2024-02-23T00:42:28.713 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.962412+0000", 2024-02-23T00:42:28.713 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000398842 2024-02-23T00:42:28.713 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.713 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.713 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:42:28.713 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.962811+0000", 2024-02-23T00:42:28.713 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00028907500000000002 2024-02-23T00:42:28.713 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.713 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.713 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.713 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.963100+0000", 2024-02-23T00:42:28.713 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00052340299999999998 2024-02-23T00:42:28.713 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.713 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.713 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.714 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.963624+0000", 2024-02-23T00:42:28.714 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.3776000000000003e-05 2024-02-23T00:42:28.714 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.714 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.714 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:42:28.714 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.963658+0000", 2024-02-23T00:42:28.714 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.2113e-05 2024-02-23T00:42:28.714 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.714 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.714 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:42:28.714 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.963680+0000", 2024-02-23T00:42:28.714 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0016350119999999999 2024-02-23T00:42:28.714 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.714 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:42:28.714 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.715 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.715 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.715 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.14460.0:545 4.3 4:c134263f:lc::lc.25:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected e67)", 2024-02-23T00:42:28.715 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:39:21.990459+0000", 2024-02-23T00:42:28.715 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 185.603068511, 2024-02-23T00:42:28.715 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0017673960000000001, 2024-02-23T00:42:28.715 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:42:28.715 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:42:28.715 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:42:28.715 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.14460", 2024-02-23T00:42:28.715 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1573276039", 2024-02-23T00:42:28.715 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 545 2024-02-23T00:42:28.715 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.716 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:42:28.716 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.716 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:42:28.716 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.990459+0000", 2024-02-23T00:42:28.716 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:42:28.716 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.716 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.716 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:42:28.716 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.990459+0000", 2024-02-23T00:42:28.716 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999976 2024-02-23T00:42:28.716 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.716 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.716 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:42:28.716 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.990457+0000", 2024-02-23T00:42:28.717 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2329e-05 2024-02-23T00:42:28.717 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.717 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.717 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:42:28.717 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.990470+0000", 2024-02-23T00:42:28.717 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6864e-05 2024-02-23T00:42:28.717 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.717 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.717 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:42:28.717 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.990486+0000", 2024-02-23T00:42:28.717 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.7059999999999998e-06 2024-02-23T00:42:28.717 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.717 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.717 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:42:28.717 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.990493+0000", 2024-02-23T00:42:28.718 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.6193999999999999e-05 2024-02-23T00:42:28.718 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.718 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.718 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:42:28.718 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.990529+0000", 2024-02-23T00:42:28.718 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.9186999999999998e-05 2024-02-23T00:42:28.718 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.718 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.718 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:42:28.718 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.990619+0000", 2024-02-23T00:42:28.718 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00015146199999999999 2024-02-23T00:42:28.718 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.718 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.718 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 5,7", 2024-02-23T00:42:28.718 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.990770+0000", 2024-02-23T00:42:28.719 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00045413400000000003 2024-02-23T00:42:28.719 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.719 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.719 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:42:28.719 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.991224+0000", 2024-02-23T00:42:28.719 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00055745199999999995 2024-02-23T00:42:28.719 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.719 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.719 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.719 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.991782+0000", 2024-02-23T00:42:28.719 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00035099000000000003 2024-02-23T00:42:28.719 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.719 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.719 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.720 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.992133+0000", 2024-02-23T00:42:28.720 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.6076999999999999e-05 2024-02-23T00:42:28.720 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.720 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.720 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:42:28.720 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.992189+0000", 2024-02-23T00:42:28.720 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.8231e-05 2024-02-23T00:42:28.720 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.720 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.720 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:42:28.720 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.992227+0000", 2024-02-23T00:42:28.720 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0017673960000000001 2024-02-23T00:42:28.720 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.720 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:42:28.721 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.721 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.721 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.721 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.14460.0:557 4.3 4:c134263f:lc::lc.25:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected e67)", 2024-02-23T00:42:28.721 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:39:21.994195+0000", 2024-02-23T00:42:28.721 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 185.59933305800001, 2024-02-23T00:42:28.721 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0021453779999999999, 2024-02-23T00:42:28.721 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:42:28.721 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:42:28.721 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:42:28.721 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.14460", 2024-02-23T00:42:28.721 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1573276039", 2024-02-23T00:42:28.721 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 557 2024-02-23T00:42:28.721 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.721 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:42:28.722 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.722 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:42:28.722 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.994195+0000", 2024-02-23T00:42:28.722 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:42:28.722 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.722 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.722 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:42:28.722 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.994195+0000", 2024-02-23T00:42:28.722 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.999999 2024-02-23T00:42:28.722 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.722 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.722 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:42:28.722 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.994194+0000", 2024-02-23T00:42:28.722 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.19e-06 2024-02-23T00:42:28.722 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.723 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.723 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:42:28.723 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.994200+0000", 2024-02-23T00:42:28.723 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.9018999999999999e-05 2024-02-23T00:42:28.723 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.723 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.723 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:42:28.723 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.994219+0000", 2024-02-23T00:42:28.723 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.5969999999999997e-06 2024-02-23T00:42:28.723 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.723 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.723 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:42:28.723 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.994226+0000", 2024-02-23T00:42:28.723 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.8105000000000002e-05 2024-02-23T00:42:28.723 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.724 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.724 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:42:28.724 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.994264+0000", 2024-02-23T00:42:28.724 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.1020999999999998e-05 2024-02-23T00:42:28.724 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.724 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.724 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:42:28.724 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.994315+0000", 2024-02-23T00:42:28.724 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000109233 2024-02-23T00:42:28.724 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.724 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.724 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 5,7", 2024-02-23T00:42:28.724 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.994424+0000", 2024-02-23T00:42:28.724 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00033866300000000001 2024-02-23T00:42:28.725 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.725 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.725 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:42:28.725 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.994763+0000", 2024-02-23T00:42:28.725 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001376967 2024-02-23T00:42:28.725 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.725 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.725 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.725 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.996140+0000", 2024-02-23T00:42:28.725 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012170799999999999 2024-02-23T00:42:28.725 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.725 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.725 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.725 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.996261+0000", 2024-02-23T00:42:28.726 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.4121e-05 2024-02-23T00:42:28.726 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.726 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.726 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:42:28.726 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.996306+0000", 2024-02-23T00:42:28.726 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.4734000000000003e-05 2024-02-23T00:42:28.726 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.726 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.726 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:42:28.726 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:21.996340+0000", 2024-02-23T00:42:28.726 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0021453779999999999 2024-02-23T00:42:28.726 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.726 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:42:28.726 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.726 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.726 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.727 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.14460.0:617 4.16 4:6d4431ed:lc::lc.7:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected e67)", 2024-02-23T00:42:28.727 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:39:22.013969+0000", 2024-02-23T00:42:28.727 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 185.57955896300001, 2024-02-23T00:42:28.727 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0019263430000000001, 2024-02-23T00:42:28.727 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:42:28.727 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:42:28.727 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:42:28.727 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.14460", 2024-02-23T00:42:28.727 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1573276039", 2024-02-23T00:42:28.727 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 617 2024-02-23T00:42:28.727 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.727 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:42:28.727 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.727 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:42:28.728 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:22.013969+0000", 2024-02-23T00:42:28.728 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:42:28.728 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.728 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.728 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:42:28.728 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:22.013969+0000", 2024-02-23T00:42:28.728 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.999999 2024-02-23T00:42:28.728 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.728 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.728 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:42:28.728 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:22.013968+0000", 2024-02-23T00:42:28.728 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.0459999999999996e-06 2024-02-23T00:42:28.728 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.728 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.728 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:42:28.729 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:22.013973+0000", 2024-02-23T00:42:28.729 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.7409999999999998e-06 2024-02-23T00:42:28.729 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.729 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.729 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:42:28.729 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:22.013982+0000", 2024-02-23T00:42:28.729 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.4390000000000002e-06 2024-02-23T00:42:28.729 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.729 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.729 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:42:28.729 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:22.013986+0000", 2024-02-23T00:42:28.729 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.8081999999999999e-05 2024-02-23T00:42:28.729 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.729 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.729 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:42:28.730 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:22.014025+0000", 2024-02-23T00:42:28.730 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.7556000000000002e-05 2024-02-23T00:42:28.730 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.730 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.730 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:42:28.730 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:22.014092+0000", 2024-02-23T00:42:28.730 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00017512100000000001 2024-02-23T00:42:28.730 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.730 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.730 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 3,7", 2024-02-23T00:42:28.730 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:22.014267+0000", 2024-02-23T00:42:28.730 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00046706200000000002 2024-02-23T00:42:28.730 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.730 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.730 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.731 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:22.014734+0000", 2024-02-23T00:42:28.731 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00016377500000000001 2024-02-23T00:42:28.731 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.731 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.731 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:42:28.731 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:22.014898+0000", 2024-02-23T00:42:28.731 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000905607 2024-02-23T00:42:28.731 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.731 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.731 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:42:28.731 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:22.015804+0000", 2024-02-23T00:42:28.731 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.4215000000000002e-05 2024-02-23T00:42:28.731 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.731 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.731 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:42:28.732 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:22.015858+0000", 2024-02-23T00:42:28.732 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7487999999999998e-05 2024-02-23T00:42:28.732 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.732 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.732 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:42:28.732 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:22.015895+0000", 2024-02-23T00:42:28.732 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0019263430000000001 2024-02-23T00:42:28.732 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.732 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:42:28.732 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.732 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.732 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.732 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(client.14460.0:615 4.1a e67/61 4:5909e671:gc::gc.27:head v 67'12, mlcod=67'10)", 2024-02-23T00:42:28.732 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:39:22.014055+0000", 2024-02-23T00:42:28.733 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 185.579473065, 2024-02-23T00:42:28.733 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001752263, 2024-02-23T00:42:28.733 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:42:28.733 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:42:28.733 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:42:28.733 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.733 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:42:28.733 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:22.014055+0000", 2024-02-23T00:42:28.733 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:42:28.733 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.733 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.733 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:42:28.733 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:22.014055+0000", 2024-02-23T00:42:28.733 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:42:28.733 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.734 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.734 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:42:28.734 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:22.014054+0000", 2024-02-23T00:42:28.734 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.6079999999999998e-06 2024-02-23T00:42:28.734 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.734 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.734 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:42:28.734 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:22.014059+0000", 2024-02-23T00:42:28.734 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.7320000000000003e-06 2024-02-23T00:42:28.734 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.734 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.734 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:42:28.734 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:22.014065+0000", 2024-02-23T00:42:28.734 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0850000000000001e-06 2024-02-23T00:42:28.734 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.735 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.735 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:42:28.735 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:22.014068+0000", 2024-02-23T00:42:28.735 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.1513999999999997e-05 2024-02-23T00:42:28.735 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.735 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.735 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:42:28.735 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:22.014099+0000", 2024-02-23T00:42:28.735 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2687999999999999e-05 2024-02-23T00:42:28.735 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.735 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.735 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:42:28.735 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:22.014112+0000", 2024-02-23T00:42:28.735 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0016670529999999999 2024-02-23T00:42:28.735 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.735 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.736 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:42:28.736 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:22.015779+0000", 2024-02-23T00:42:28.736 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8070000000000001e-05 2024-02-23T00:42:28.736 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:42:28.736 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:42:28.736 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:42:28.736 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:39:22.015807+0000", 2024-02-23T00:42:28.736 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001752263 2024-02-23T00:42:28.736 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.736 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:42:28.736 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.736 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:42:28.736 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:42:28.736 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:42:28.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:27.165588+0000 mgr.y (mgr.14517) 127 : cluster [DBG] pgmap v113: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-23T00:42:28.768 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:28.789 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:42:28.792 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:28.792 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:42:28.893 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:42:28.926 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:29.272 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.1 dump_ops_in_flight 2024-02-23T00:42:29.482 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:42:29.491 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:29.492 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:42:29.593 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:42:29.622 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:29.720 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:42:29.723 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:29.724 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:42:29.813 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:42:29.825 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:42:29.858 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:29.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.870+0000 7fdfa43ac700 1 -- 172.21.15.47:0/3034335895 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdf9c0fd510 msgr2=0x7fdf9c0f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:29.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.870+0000 7fdfa43ac700 1 --2- 172.21.15.47:0/3034335895 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdf9c0fd510 0x7fdf9c0f6d40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fdf8c004870 tx=0x7fdf8c009cb0).stop 2024-02-23T00:42:29.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.870+0000 7fdfa43ac700 1 -- 172.21.15.47:0/3034335895 shutdown_connections 2024-02-23T00:42:29.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.870+0000 7fdfa43ac700 1 --2- 172.21.15.47:0/3034335895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf9c0f7400 0x7fdf9c103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:29.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.870+0000 7fdfa43ac700 1 --2- 172.21.15.47:0/3034335895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf9c0fcbb0 0x7fdf9c0fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:29.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.870+0000 7fdfa43ac700 1 --2- 172.21.15.47:0/3034335895 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdf9c0fd510 0x7fdf9c0f6d40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:29.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.870+0000 7fdfa43ac700 1 -- 172.21.15.47:0/3034335895 >> 172.21.15.47:0/3034335895 conn(0x7fdf9c0f2820 msgr2=0x7fdf9c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:42:29.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.870+0000 7fdfa43ac700 1 -- 172.21.15.47:0/3034335895 shutdown_connections 2024-02-23T00:42:29.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.870+0000 7fdfa43ac700 1 -- 172.21.15.47:0/3034335895 wait complete. 2024-02-23T00:42:29.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.871+0000 7fdfa43ac700 1 librados: starting msgr at 2024-02-23T00:42:29.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.871+0000 7fdfa43ac700 1 librados: starting objecter 2024-02-23T00:42:29.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.871+0000 7fdfa43ac700 1 Processor -- start 2024-02-23T00:42:29.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.871+0000 7fdfa43ac700 1 -- start start 2024-02-23T00:42:29.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.871+0000 7fdfa43ac700 1 librados: setting wanted keys 2024-02-23T00:42:29.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.871+0000 7fdfa43ac700 1 librados: calling monclient init 2024-02-23T00:42:29.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.871+0000 7fdfa43ac700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf9c0f7400 0x7fdf9c1101e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:29.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.871+0000 7fdfa43ac700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdf9c0fcbb0 0x7fdf9c1106d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:29.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.871+0000 7fdfa43ac700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf9c0fd510 0x7fdf9c10a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:29.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.871+0000 7fdfa43ac700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c110bc0 con 0x7fdf9c0f7400 2024-02-23T00:42:29.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.871+0000 7fdfa43ac700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c110d30 con 0x7fdf9c0fd510 2024-02-23T00:42:29.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.871+0000 7fdfa43ac700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c10a8e0 con 0x7fdf9c0fcbb0 2024-02-23T00:42:29.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.871+0000 7fdfa2148700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf9c0f7400 0x7fdf9c1101e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:29.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.871+0000 7fdfa2949700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf9c0fd510 0x7fdf9c10a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:29.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.871+0000 7fdfa2148700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf9c0f7400 0x7fdf9c1101e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60568/0 (socket says 172.21.15.47:60568) 2024-02-23T00:42:29.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.871+0000 7fdfa2949700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf9c0fd510 0x7fdf9c10a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.47:44006/0 (socket says 172.21.15.47:44006) 2024-02-23T00:42:29.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.871+0000 7fdfa2148700 1 -- 172.21.15.47:0/2315457400 learned_addr learned my addr 172.21.15.47:0/2315457400 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:42:29.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.871+0000 7fdfa1947700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdf9c0fcbb0 0x7fdf9c1106d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:29.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.872+0000 7fdfa2949700 1 -- 172.21.15.47:0/2315457400 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdf9c0fcbb0 msgr2=0x7fdf9c1106d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:29.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.872+0000 7fdfa2949700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdf9c0fcbb0 0x7fdf9c1106d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:29.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.872+0000 7fdfa2949700 1 -- 172.21.15.47:0/2315457400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf9c0f7400 msgr2=0x7fdf9c1101e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:29.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.872+0000 7fdfa2949700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf9c0f7400 0x7fdf9c1101e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:29.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.872+0000 7fdfa2949700 1 -- 172.21.15.47:0/2315457400 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf8c005040 con 0x7fdf9c0fd510 2024-02-23T00:42:29.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.872+0000 7fdfa2148700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf9c0f7400 0x7fdf9c1101e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:42:29.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.872+0000 7fdfa2949700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf9c0fd510 0x7fdf9c10a3f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fdf9400eb70 tx=0x7fdf9400b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:29.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.872+0000 7fdf937fe700 10 client.?.objecter ms_handle_connect 0x7fdf9c0fd510 2024-02-23T00:42:29.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.872+0000 7fdf937fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:42:29.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.872+0000 7fdf937fe700 1 -- 172.21.15.47:0/2315457400 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf94009870 con 0x7fdf9c0fd510 2024-02-23T00:42:29.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.872+0000 7fdf937fe700 1 -- 172.21.15.47:0/2315457400 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fdf9400f040 con 0x7fdf9c0fd510 2024-02-23T00:42:29.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.872+0000 7fdf937fe700 1 -- 172.21.15.47:0/2315457400 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf94007c70 con 0x7fdf9c0fd510 2024-02-23T00:42:29.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.872+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf9c10aaa0 con 0x7fdf9c0fd510 2024-02-23T00:42:29.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.872+0000 7fdfa43ac700 10 client.24679.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:42:29.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.872+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf9c10af10 con 0x7fdf9c0fd510 2024-02-23T00:42:29.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.872+0000 7fdfa43ac700 1 librados: init done 2024-02-23T00:42:29.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.873+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fdf9c05fa60 con 0x7fdf9c0fd510 2024-02-23T00:42:29.880 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:42:29.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.874+0000 7fdf937fe700 1 -- 172.21.15.47:0/2315457400 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fdf940040b0 con 0x7fdf9c0fd510 2024-02-23T00:42:29.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.875+0000 7fdf937fe700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdf88062cc0 0x7fdf88065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:29.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.875+0000 7fdf937fe700 1 -- 172.21.15.47:0/2315457400 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(100..100 src has 48..100) v4 ==== 7453+0+0 (secure 0 0 0) 0x7fdf9407d380 con 0x7fdf9c0fd510 2024-02-23T00:42:29.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.875+0000 7fdf937fe700 10 client.24679.objecter ms_dispatch 0x7fdf9c001550 osd_map(100..100 src has 48..100) v4 2024-02-23T00:42:29.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.875+0000 7fdf937fe700 3 client.24679.objecter handle_osd_map got epochs [100,100] > 0 2024-02-23T00:42:29.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.875+0000 7fdf937fe700 3 client.24679.objecter handle_osd_map decoding full epoch 100 2024-02-23T00:42:29.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.875+0000 7fdfa2148700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdf88062cc0 0x7fdf88065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:29.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.875+0000 7fdf937fe700 20 client.24679.objecter dump_active .. 0 homeless 2024-02-23T00:42:29.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.875+0000 7fdf937fe700 1 -- 172.21.15.47:0/2315457400 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v100) v1 ==== 54+0+16 (secure 0 0 0) 0x7fdf9407d8b0 con 0x7fdf9c0fd510 2024-02-23T00:42:29.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.875+0000 7fdfa43ac700 10 client.24679.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:29.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.875+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_assign -1 1 2024-02-23T00:42:29.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.875+0000 7fdfa43ac700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fdf9c028540 0x7fdf9c0fd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:29.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.875+0000 7fdfa43ac700 20 client.24679.objecter _get_session s=0x7fdf9c05ee80 osd=0 3 2024-02-23T00:42:29.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.875+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c05ee80 osd=0 3 2024-02-23T00:42:29.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.875+0000 7fdfa43ac700 20 client.24679.objecter _get_session s=0x7fdf9c05ee80 osd=0 3 2024-02-23T00:42:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.875+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_remove -1 1 2024-02-23T00:42:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.875+0000 7fdfa43ac700 20 client.24679.objecter get_session s=0x7fdf9c05ee80 osd=0 3 2024-02-23T00:42:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.875+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_assign 0 1 2024-02-23T00:42:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.875+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c05ee80 osd=0 4 2024-02-23T00:42:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.875+0000 7fdfa43ac700 10 client.24679.objecter _send_command 1 2024-02-23T00:42:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.875+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf9c104e20 con 0x7fdf9c028540 2024-02-23T00:42:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.875+0000 7fdfa1947700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fdf9c028540 0x7fdf9c0fd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.876+0000 7fdf937fe700 10 client.24679.objecter ms_handle_connect 0x7fdf88062cc0 2024-02-23T00:42:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.876+0000 7fdfa2148700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdf88062cc0 0x7fdf88065180 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fdf98007980 tx=0x7fdf9800d040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.876+0000 7fdf937fe700 10 client.24679.objecter ms_handle_connect 0x7fdf9c028540 2024-02-23T00:42:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.876+0000 7fdfa1947700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fdf9c028540 0x7fdf9c0fd9b0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.877+0000 7fdf937fe700 1 -- 172.21.15.47:0/2315457400 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fdf9c104e20 con 0x7fdf9c028540 2024-02-23T00:42:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.877+0000 7fdf937fe700 10 client.24679.objecter ms_dispatch 0x7fdf9c001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:42:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.877+0000 7fdf937fe700 10 client.24679.objecter _finish_command 1 = system:0 2024-02-23T00:42:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.877+0000 7fdf937fe700 20 client.24679.objecter put_session s=0x7fdf9c05ee80 osd=0 4 2024-02-23T00:42:29.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.877+0000 7fdf937fe700 15 client.24679.objecter _session_command_op_remove 0 1 2024-02-23T00:42:29.883 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:29.884 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:42:29.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.906+0000 7fdfa43ac700 10 client.24679.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:42:29.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.906+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_assign -1 2 2024-02-23T00:42:29.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.906+0000 7fdfa43ac700 20 client.24679.objecter _get_session s=0x7fdf9c05ee80 osd=0 3 2024-02-23T00:42:29.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.906+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c05ee80 osd=0 3 2024-02-23T00:42:29.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.906+0000 7fdfa43ac700 20 client.24679.objecter _get_session s=0x7fdf9c05ee80 osd=0 3 2024-02-23T00:42:29.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.906+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_remove -1 2 2024-02-23T00:42:29.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.906+0000 7fdfa43ac700 20 client.24679.objecter get_session s=0x7fdf9c05ee80 osd=0 3 2024-02-23T00:42:29.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.906+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_assign 0 2 2024-02-23T00:42:29.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.906+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c05ee80 osd=0 4 2024-02-23T00:42:29.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.906+0000 7fdfa43ac700 10 client.24679.objecter _send_command 2 2024-02-23T00:42:29.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.906+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fdf9c059e00 con 0x7fdf9c028540 2024-02-23T00:42:29.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.909+0000 7fdf937fe700 1 -- 172.21.15.47:0/2315457400 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fdf9c059e00 con 0x7fdf9c028540 2024-02-23T00:42:29.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.909+0000 7fdf937fe700 10 client.24679.objecter ms_dispatch 0x7fdf9c001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:42:29.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.909+0000 7fdf937fe700 10 client.24679.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:42:29.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.909+0000 7fdf937fe700 20 client.24679.objecter put_session s=0x7fdf9c05ee80 osd=0 4 2024-02-23T00:42:29.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.909+0000 7fdf937fe700 15 client.24679.objecter _session_command_op_remove 0 2 2024-02-23T00:42:29.911 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:42:29.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.909+0000 7fdfa43ac700 10 client.24679.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:29.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.909+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_assign -1 3 2024-02-23T00:42:29.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.909+0000 7fdfa43ac700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fdf9c0fe520 0x7fdf9c0fe900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:29.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.909+0000 7fdfa43ac700 20 client.24679.objecter _get_session s=0x7fdf9c0fdea0 osd=1 3 2024-02-23T00:42:29.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.909+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c0fdea0 osd=1 3 2024-02-23T00:42:29.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.909+0000 7fdfa43ac700 20 client.24679.objecter _get_session s=0x7fdf9c0fdea0 osd=1 3 2024-02-23T00:42:29.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.909+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_remove -1 3 2024-02-23T00:42:29.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.909+0000 7fdfa43ac700 20 client.24679.objecter get_session s=0x7fdf9c0fdea0 osd=1 3 2024-02-23T00:42:29.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.909+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_assign 1 3 2024-02-23T00:42:29.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.909+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c0fdea0 osd=1 4 2024-02-23T00:42:29.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.909+0000 7fdfa43ac700 10 client.24679.objecter _send_command 3 2024-02-23T00:42:29.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.909+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf9c0fedf0 con 0x7fdf9c0fe520 2024-02-23T00:42:29.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.911+0000 7fdfa2148700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fdf9c0fe520 0x7fdf9c0fe900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:29.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.912+0000 7fdf937fe700 10 client.24679.objecter ms_handle_connect 0x7fdf9c0fe520 2024-02-23T00:42:29.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.912+0000 7fdfa2148700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fdf9c0fe520 0x7fdf9c0fe900 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:29.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.913+0000 7fdf937fe700 1 -- 172.21.15.47:0/2315457400 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fdf9c0fedf0 con 0x7fdf9c0fe520 2024-02-23T00:42:29.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.913+0000 7fdf937fe700 10 client.24679.objecter ms_dispatch 0x7fdf9c001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:42:29.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.913+0000 7fdf937fe700 10 client.24679.objecter _finish_command 3 = system:0 2024-02-23T00:42:29.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.913+0000 7fdf937fe700 20 client.24679.objecter put_session s=0x7fdf9c0fdea0 osd=1 4 2024-02-23T00:42:29.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.913+0000 7fdf937fe700 15 client.24679.objecter _session_command_op_remove 1 3 2024-02-23T00:42:29.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.942+0000 7fdfa43ac700 10 client.24679.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:42:29.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.942+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_assign -1 4 2024-02-23T00:42:29.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.942+0000 7fdfa43ac700 20 client.24679.objecter _get_session s=0x7fdf9c0fdea0 osd=1 3 2024-02-23T00:42:29.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.942+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c0fdea0 osd=1 3 2024-02-23T00:42:29.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.942+0000 7fdfa43ac700 20 client.24679.objecter _get_session s=0x7fdf9c0fdea0 osd=1 3 2024-02-23T00:42:29.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.942+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_remove -1 4 2024-02-23T00:42:29.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.942+0000 7fdfa43ac700 20 client.24679.objecter get_session s=0x7fdf9c0fdea0 osd=1 3 2024-02-23T00:42:29.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.942+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_assign 1 4 2024-02-23T00:42:29.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.942+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c0fdea0 osd=1 4 2024-02-23T00:42:29.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.942+0000 7fdfa43ac700 10 client.24679.objecter _send_command 4 2024-02-23T00:42:29.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.942+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fdf9c115040 con 0x7fdf9c0fe520 2024-02-23T00:42:29.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.944+0000 7fdf937fe700 1 -- 172.21.15.47:0/2315457400 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fdf9c115040 con 0x7fdf9c0fe520 2024-02-23T00:42:29.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.944+0000 7fdf937fe700 10 client.24679.objecter ms_dispatch 0x7fdf9c001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:42:29.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.944+0000 7fdf937fe700 10 client.24679.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:42:29.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.944+0000 7fdf937fe700 20 client.24679.objecter put_session s=0x7fdf9c0fdea0 osd=1 4 2024-02-23T00:42:29.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.944+0000 7fdf937fe700 15 client.24679.objecter _session_command_op_remove 1 4 2024-02-23T00:42:29.947 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:42:29.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.944+0000 7fdfa43ac700 10 client.24679.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:29.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.944+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_assign -1 5 2024-02-23T00:42:29.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.944+0000 7fdfa43ac700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fdf9c0594d0 0x7fdf9c1183a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:29.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.944+0000 7fdfa43ac700 20 client.24679.objecter _get_session s=0x7fdf9c059360 osd=2 3 2024-02-23T00:42:29.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.944+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c059360 osd=2 3 2024-02-23T00:42:29.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.944+0000 7fdfa43ac700 20 client.24679.objecter _get_session s=0x7fdf9c059360 osd=2 3 2024-02-23T00:42:29.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.944+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_remove -1 5 2024-02-23T00:42:29.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.944+0000 7fdfa43ac700 20 client.24679.objecter get_session s=0x7fdf9c059360 osd=2 3 2024-02-23T00:42:29.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.944+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_assign 2 5 2024-02-23T00:42:29.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.944+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c059360 osd=2 4 2024-02-23T00:42:29.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.944+0000 7fdfa43ac700 10 client.24679.objecter _send_command 5 2024-02-23T00:42:29.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.944+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf9c028130 con 0x7fdf9c0594d0 2024-02-23T00:42:29.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.945+0000 7fdfa1947700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fdf9c0594d0 0x7fdf9c1183a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:29.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.945+0000 7fdf937fe700 10 client.24679.objecter ms_handle_connect 0x7fdf9c0594d0 2024-02-23T00:42:29.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.945+0000 7fdfa1947700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fdf9c0594d0 0x7fdf9c1183a0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:29.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.946+0000 7fdf937fe700 1 -- 172.21.15.47:0/2315457400 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fdf9c028130 con 0x7fdf9c0594d0 2024-02-23T00:42:29.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.946+0000 7fdf937fe700 10 client.24679.objecter ms_dispatch 0x7fdf9c001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:42:29.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.946+0000 7fdf937fe700 10 client.24679.objecter _finish_command 5 = system:0 2024-02-23T00:42:29.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.946+0000 7fdf937fe700 20 client.24679.objecter put_session s=0x7fdf9c059360 osd=2 4 2024-02-23T00:42:29.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.946+0000 7fdf937fe700 15 client.24679.objecter _session_command_op_remove 2 5 2024-02-23T00:42:29.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.975+0000 7fdfa43ac700 10 client.24679.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:42:29.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.975+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_assign -1 6 2024-02-23T00:42:29.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.975+0000 7fdfa43ac700 20 client.24679.objecter _get_session s=0x7fdf9c059360 osd=2 3 2024-02-23T00:42:29.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.975+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c059360 osd=2 3 2024-02-23T00:42:29.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.975+0000 7fdfa43ac700 20 client.24679.objecter _get_session s=0x7fdf9c059360 osd=2 3 2024-02-23T00:42:29.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.975+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_remove -1 6 2024-02-23T00:42:29.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.975+0000 7fdfa43ac700 20 client.24679.objecter get_session s=0x7fdf9c059360 osd=2 3 2024-02-23T00:42:29.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.975+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_assign 2 6 2024-02-23T00:42:29.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.975+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c059360 osd=2 4 2024-02-23T00:42:29.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.975+0000 7fdfa43ac700 10 client.24679.objecter _send_command 6 2024-02-23T00:42:29.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.975+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fdf9c118890 con 0x7fdf9c0594d0 2024-02-23T00:42:29.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.976+0000 7fdf937fe700 1 -- 172.21.15.47:0/2315457400 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fdf9c118890 con 0x7fdf9c0594d0 2024-02-23T00:42:29.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.976+0000 7fdf937fe700 10 client.24679.objecter ms_dispatch 0x7fdf9c001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:42:29.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.976+0000 7fdf937fe700 10 client.24679.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:42:29.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.976+0000 7fdf937fe700 20 client.24679.objecter put_session s=0x7fdf9c059360 osd=2 4 2024-02-23T00:42:29.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.976+0000 7fdf937fe700 15 client.24679.objecter _session_command_op_remove 2 6 2024-02-23T00:42:29.982 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:42:29.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.977+0000 7fdfa43ac700 10 client.24679.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:29.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.977+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_assign -1 7 2024-02-23T00:42:29.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.977+0000 7fdfa43ac700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fdf9c027770 0x7fdf9c12e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:29.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.977+0000 7fdfa43ac700 20 client.24679.objecter _get_session s=0x7fdf9c027600 osd=3 3 2024-02-23T00:42:29.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.977+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c027600 osd=3 3 2024-02-23T00:42:29.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.977+0000 7fdfa43ac700 20 client.24679.objecter _get_session s=0x7fdf9c027600 osd=3 3 2024-02-23T00:42:29.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.977+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_remove -1 7 2024-02-23T00:42:29.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.977+0000 7fdfa43ac700 20 client.24679.objecter get_session s=0x7fdf9c027600 osd=3 3 2024-02-23T00:42:29.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.977+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_assign 3 7 2024-02-23T00:42:29.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.977+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c027600 osd=3 4 2024-02-23T00:42:29.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.977+0000 7fdfa43ac700 10 client.24679.objecter _send_command 7 2024-02-23T00:42:29.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.977+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf9c131590 con 0x7fdf9c027770 2024-02-23T00:42:29.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.977+0000 7fdfa2949700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fdf9c027770 0x7fdf9c12e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:29.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.977+0000 7fdf937fe700 10 client.24679.objecter ms_handle_connect 0x7fdf9c027770 2024-02-23T00:42:29.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.977+0000 7fdfa2949700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fdf9c027770 0x7fdf9c12e860 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:29.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.978+0000 7fdf937fe700 1 -- 172.21.15.47:0/2315457400 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fdf9c131590 con 0x7fdf9c027770 2024-02-23T00:42:29.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.978+0000 7fdf937fe700 10 client.24679.objecter ms_dispatch 0x7fdf9c001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:42:29.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.978+0000 7fdf937fe700 10 client.24679.objecter _finish_command 7 = system:0 2024-02-23T00:42:29.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.978+0000 7fdf937fe700 20 client.24679.objecter put_session s=0x7fdf9c027600 osd=3 4 2024-02-23T00:42:29.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.978+0000 7fdf937fe700 15 client.24679.objecter _session_command_op_remove 3 7 2024-02-23T00:42:29.987 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:42:29.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.993+0000 7fdfa43ac700 10 client.24679.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:42:29.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.993+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_assign -1 8 2024-02-23T00:42:29.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.993+0000 7fdfa43ac700 20 client.24679.objecter _get_session s=0x7fdf9c027600 osd=3 3 2024-02-23T00:42:29.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.993+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c027600 osd=3 3 2024-02-23T00:42:29.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.993+0000 7fdfa43ac700 20 client.24679.objecter _get_session s=0x7fdf9c027600 osd=3 3 2024-02-23T00:42:29.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.993+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_remove -1 8 2024-02-23T00:42:29.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.993+0000 7fdfa43ac700 20 client.24679.objecter get_session s=0x7fdf9c027600 osd=3 3 2024-02-23T00:42:29.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.993+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_assign 3 8 2024-02-23T00:42:29.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.993+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c027600 osd=3 4 2024-02-23T00:42:29.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.993+0000 7fdfa43ac700 10 client.24679.objecter _send_command 8 2024-02-23T00:42:29.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.993+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fdf9c118c40 con 0x7fdf9c027770 2024-02-23T00:42:29.998 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:42:29.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.995+0000 7fdf937fe700 1 -- 172.21.15.47:0/2315457400 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fdf9c118c40 con 0x7fdf9c027770 2024-02-23T00:42:29.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.995+0000 7fdf937fe700 10 client.24679.objecter ms_dispatch 0x7fdf9c001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:42:29.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.995+0000 7fdf937fe700 10 client.24679.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:42:29.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.995+0000 7fdf937fe700 20 client.24679.objecter put_session s=0x7fdf9c027600 osd=3 4 2024-02-23T00:42:29.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.995+0000 7fdf937fe700 15 client.24679.objecter _session_command_op_remove 3 8 2024-02-23T00:42:29.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.995+0000 7fdfa43ac700 10 client.24679.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:29.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.995+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_assign -1 9 2024-02-23T00:42:29.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.995+0000 7fdfa43ac700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fdf9c115e90 0x7fdf9c12ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:30.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.995+0000 7fdfa43ac700 20 client.24679.objecter _get_session s=0x7fdf9c118e00 osd=4 3 2024-02-23T00:42:30.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.995+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c118e00 osd=4 3 2024-02-23T00:42:30.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.995+0000 7fdfa43ac700 20 client.24679.objecter _get_session s=0x7fdf9c118e00 osd=4 3 2024-02-23T00:42:30.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.995+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_remove -1 9 2024-02-23T00:42:30.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.995+0000 7fdfa43ac700 20 client.24679.objecter get_session s=0x7fdf9c118e00 osd=4 3 2024-02-23T00:42:30.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.995+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_assign 4 9 2024-02-23T00:42:30.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.995+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c118e00 osd=4 4 2024-02-23T00:42:30.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.995+0000 7fdfa43ac700 10 client.24679.objecter _send_command 9 2024-02-23T00:42:30.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.995+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf9c119000 con 0x7fdf9c115e90 2024-02-23T00:42:30.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.996+0000 7fdfa2148700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fdf9c115e90 0x7fdf9c12ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:30.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.996+0000 7fdf937fe700 10 client.24679.objecter ms_handle_connect 0x7fdf9c115e90 2024-02-23T00:42:30.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.996+0000 7fdfa2148700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fdf9c115e90 0x7fdf9c12ed50 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:30.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.997+0000 7fdf937fe700 1 -- 172.21.15.47:0/2315457400 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fdf9c119000 con 0x7fdf9c115e90 2024-02-23T00:42:30.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.997+0000 7fdf937fe700 10 client.24679.objecter ms_dispatch 0x7fdf9c001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:42:30.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.997+0000 7fdf937fe700 10 client.24679.objecter _finish_command 9 = system:0 2024-02-23T00:42:30.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.997+0000 7fdf937fe700 20 client.24679.objecter put_session s=0x7fdf9c118e00 osd=4 4 2024-02-23T00:42:30.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:29.997+0000 7fdf937fe700 15 client.24679.objecter _session_command_op_remove 4 9 2024-02-23T00:42:30.017 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:30.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.016+0000 7fdfa43ac700 10 client.24679.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:42:30.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.016+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_assign -1 10 2024-02-23T00:42:30.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.016+0000 7fdfa43ac700 20 client.24679.objecter _get_session s=0x7fdf9c118e00 osd=4 3 2024-02-23T00:42:30.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.016+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c118e00 osd=4 3 2024-02-23T00:42:30.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.016+0000 7fdfa43ac700 20 client.24679.objecter _get_session s=0x7fdf9c118e00 osd=4 3 2024-02-23T00:42:30.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.016+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_remove -1 10 2024-02-23T00:42:30.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.016+0000 7fdfa43ac700 20 client.24679.objecter get_session s=0x7fdf9c118e00 osd=4 3 2024-02-23T00:42:30.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.016+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_assign 4 10 2024-02-23T00:42:30.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.016+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c118e00 osd=4 4 2024-02-23T00:42:30.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.016+0000 7fdfa43ac700 10 client.24679.objecter _send_command 10 2024-02-23T00:42:30.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.016+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fdf9c119000 con 0x7fdf9c115e90 2024-02-23T00:42:30.021 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:42:30.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.017+0000 7fdf937fe700 1 -- 172.21.15.47:0/2315457400 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fdf9c119000 con 0x7fdf9c115e90 2024-02-23T00:42:30.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.017+0000 7fdf937fe700 10 client.24679.objecter ms_dispatch 0x7fdf9c001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:42:30.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.017+0000 7fdf937fe700 10 client.24679.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:42:30.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.017+0000 7fdf937fe700 20 client.24679.objecter put_session s=0x7fdf9c118e00 osd=4 4 2024-02-23T00:42:30.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.017+0000 7fdf937fe700 15 client.24679.objecter _session_command_op_remove 4 10 2024-02-23T00:42:30.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.018+0000 7fdfa43ac700 10 client.24679.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:30.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.018+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_assign -1 11 2024-02-23T00:42:30.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.018+0000 7fdfa43ac700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fdf9c12f780 0x7fdf9c12fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:30.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.018+0000 7fdfa43ac700 20 client.24679.objecter _get_session s=0x7fdf9c119000 osd=5 3 2024-02-23T00:42:30.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.018+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c119000 osd=5 3 2024-02-23T00:42:30.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.018+0000 7fdfa43ac700 20 client.24679.objecter _get_session s=0x7fdf9c119000 osd=5 3 2024-02-23T00:42:30.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.018+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_remove -1 11 2024-02-23T00:42:30.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.018+0000 7fdfa43ac700 20 client.24679.objecter get_session s=0x7fdf9c119000 osd=5 3 2024-02-23T00:42:30.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.018+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_assign 5 11 2024-02-23T00:42:30.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.018+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c119000 osd=5 4 2024-02-23T00:42:30.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.018+0000 7fdfa43ac700 10 client.24679.objecter _send_command 11 2024-02-23T00:42:30.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.018+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf9c137eb0 con 0x7fdf9c12f780 2024-02-23T00:42:30.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.018+0000 7fdfa1947700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fdf9c12f780 0x7fdf9c12fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:30.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.018+0000 7fdf937fe700 10 client.24679.objecter ms_handle_connect 0x7fdf9c12f780 2024-02-23T00:42:30.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.018+0000 7fdfa1947700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fdf9c12f780 0x7fdf9c12fbe0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:30.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.019+0000 7fdf937fe700 1 -- 172.21.15.47:0/2315457400 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fdf9c137eb0 con 0x7fdf9c12f780 2024-02-23T00:42:30.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.019+0000 7fdf937fe700 10 client.24679.objecter ms_dispatch 0x7fdf9c001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:42:30.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.019+0000 7fdf937fe700 10 client.24679.objecter _finish_command 11 = system:0 2024-02-23T00:42:30.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.019+0000 7fdf937fe700 20 client.24679.objecter put_session s=0x7fdf9c119000 osd=5 4 2024-02-23T00:42:30.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.019+0000 7fdf937fe700 15 client.24679.objecter _session_command_op_remove 5 11 2024-02-23T00:42:30.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.034+0000 7fdfa43ac700 10 client.24679.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:42:30.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.034+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_assign -1 12 2024-02-23T00:42:30.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.034+0000 7fdfa43ac700 20 client.24679.objecter _get_session s=0x7fdf9c119000 osd=5 3 2024-02-23T00:42:30.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.034+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c119000 osd=5 3 2024-02-23T00:42:30.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.034+0000 7fdfa43ac700 20 client.24679.objecter _get_session s=0x7fdf9c119000 osd=5 3 2024-02-23T00:42:30.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.034+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_remove -1 12 2024-02-23T00:42:30.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.034+0000 7fdfa43ac700 20 client.24679.objecter get_session s=0x7fdf9c119000 osd=5 3 2024-02-23T00:42:30.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.034+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_assign 5 12 2024-02-23T00:42:30.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.034+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c119000 osd=5 4 2024-02-23T00:42:30.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.034+0000 7fdfa43ac700 10 client.24679.objecter _send_command 12 2024-02-23T00:42:30.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.034+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fdf9c135750 con 0x7fdf9c12f780 2024-02-23T00:42:30.039 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:42:30.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.036+0000 7fdf937fe700 1 -- 172.21.15.47:0/2315457400 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fdf9c135750 con 0x7fdf9c12f780 2024-02-23T00:42:30.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.036+0000 7fdf937fe700 10 client.24679.objecter ms_dispatch 0x7fdf9c001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:42:30.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.036+0000 7fdf937fe700 10 client.24679.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:42:30.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.036+0000 7fdf937fe700 20 client.24679.objecter put_session s=0x7fdf9c119000 osd=5 4 2024-02-23T00:42:30.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.036+0000 7fdf937fe700 15 client.24679.objecter _session_command_op_remove 5 12 2024-02-23T00:42:30.041 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:42:30.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.036+0000 7fdfa43ac700 10 client.24679.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:30.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.036+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_assign -1 13 2024-02-23T00:42:30.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.036+0000 7fdfa43ac700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fdf9c135dd0 0x7fdf9c138240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:30.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.036+0000 7fdfa43ac700 20 client.24679.objecter _get_session s=0x7fdf9c135750 osd=6 3 2024-02-23T00:42:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.036+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c135750 osd=6 3 2024-02-23T00:42:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.036+0000 7fdfa43ac700 20 client.24679.objecter _get_session s=0x7fdf9c135750 osd=6 3 2024-02-23T00:42:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.036+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_remove -1 13 2024-02-23T00:42:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.036+0000 7fdfa43ac700 20 client.24679.objecter get_session s=0x7fdf9c135750 osd=6 3 2024-02-23T00:42:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.036+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_assign 6 13 2024-02-23T00:42:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.036+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c135750 osd=6 4 2024-02-23T00:42:30.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.036+0000 7fdfa43ac700 10 client.24679.objecter _send_command 13 2024-02-23T00:42:30.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.036+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf9c13b080 con 0x7fdf9c135dd0 2024-02-23T00:42:30.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.036+0000 7fdfa2949700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fdf9c135dd0 0x7fdf9c138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:30.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.037+0000 7fdf937fe700 10 client.24679.objecter ms_handle_connect 0x7fdf9c135dd0 2024-02-23T00:42:30.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.037+0000 7fdfa2949700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fdf9c135dd0 0x7fdf9c138240 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:30.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.037+0000 7fdf937fe700 1 -- 172.21.15.47:0/2315457400 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fdf9c13b080 con 0x7fdf9c135dd0 2024-02-23T00:42:30.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.037+0000 7fdf937fe700 10 client.24679.objecter ms_dispatch 0x7fdf9c001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:42:30.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.037+0000 7fdf937fe700 10 client.24679.objecter _finish_command 13 = system:0 2024-02-23T00:42:30.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.037+0000 7fdf937fe700 20 client.24679.objecter put_session s=0x7fdf9c135750 osd=6 4 2024-02-23T00:42:30.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.037+0000 7fdf937fe700 15 client.24679.objecter _session_command_op_remove 6 13 2024-02-23T00:42:30.046 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:30.047 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:42:30.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.052+0000 7fdfa43ac700 10 client.24679.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:42:30.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.052+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_assign -1 14 2024-02-23T00:42:30.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.052+0000 7fdfa43ac700 20 client.24679.objecter _get_session s=0x7fdf9c135750 osd=6 3 2024-02-23T00:42:30.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.052+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c135750 osd=6 3 2024-02-23T00:42:30.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.052+0000 7fdfa43ac700 20 client.24679.objecter _get_session s=0x7fdf9c135750 osd=6 3 2024-02-23T00:42:30.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.052+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_remove -1 14 2024-02-23T00:42:30.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.052+0000 7fdfa43ac700 20 client.24679.objecter get_session s=0x7fdf9c135750 osd=6 3 2024-02-23T00:42:30.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.052+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_assign 6 14 2024-02-23T00:42:30.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.052+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c135750 osd=6 4 2024-02-23T00:42:30.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.052+0000 7fdfa43ac700 10 client.24679.objecter _send_command 14 2024-02-23T00:42:30.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.052+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fdf9c13b080 con 0x7fdf9c135dd0 2024-02-23T00:42:30.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.054+0000 7fdf937fe700 1 -- 172.21.15.47:0/2315457400 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fdf9c13b080 con 0x7fdf9c135dd0 2024-02-23T00:42:30.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.054+0000 7fdf937fe700 10 client.24679.objecter ms_dispatch 0x7fdf9c001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:42:30.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.054+0000 7fdf937fe700 10 client.24679.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:42:30.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.054+0000 7fdf937fe700 20 client.24679.objecter put_session s=0x7fdf9c135750 osd=6 4 2024-02-23T00:42:30.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.054+0000 7fdf937fe700 15 client.24679.objecter _session_command_op_remove 6 14 2024-02-23T00:42:30.058 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:42:30.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.054+0000 7fdfa43ac700 10 client.24679.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:30.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.054+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_assign -1 15 2024-02-23T00:42:30.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.054+0000 7fdfa43ac700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fdf9c13a3f0 0x7fdf9c13a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:30.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.054+0000 7fdfa43ac700 20 client.24679.objecter _get_session s=0x7fdf9c13b080 osd=7 3 2024-02-23T00:42:30.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.054+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c13b080 osd=7 3 2024-02-23T00:42:30.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.054+0000 7fdfa43ac700 20 client.24679.objecter _get_session s=0x7fdf9c13b080 osd=7 3 2024-02-23T00:42:30.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.054+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_remove -1 15 2024-02-23T00:42:30.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.054+0000 7fdfa43ac700 20 client.24679.objecter get_session s=0x7fdf9c13b080 osd=7 3 2024-02-23T00:42:30.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.054+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_assign 7 15 2024-02-23T00:42:30.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.054+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c13b080 osd=7 4 2024-02-23T00:42:30.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.054+0000 7fdfa43ac700 10 client.24679.objecter _send_command 15 2024-02-23T00:42:30.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.054+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf9c13e3e0 con 0x7fdf9c13a3f0 2024-02-23T00:42:30.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.054+0000 7fdfa2148700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fdf9c13a3f0 0x7fdf9c13a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:30.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.055+0000 7fdf937fe700 10 client.24679.objecter ms_handle_connect 0x7fdf9c13a3f0 2024-02-23T00:42:30.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.055+0000 7fdfa2148700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fdf9c13a3f0 0x7fdf9c13a870 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:30.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.055+0000 7fdf937fe700 1 -- 172.21.15.47:0/2315457400 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fdf9c13e3e0 con 0x7fdf9c13a3f0 2024-02-23T00:42:30.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.055+0000 7fdf937fe700 10 client.24679.objecter ms_dispatch 0x7fdf9c001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:42:30.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.055+0000 7fdf937fe700 10 client.24679.objecter _finish_command 15 = system:0 2024-02-23T00:42:30.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.056+0000 7fdf937fe700 20 client.24679.objecter put_session s=0x7fdf9c13b080 osd=7 4 2024-02-23T00:42:30.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.056+0000 7fdf937fe700 15 client.24679.objecter _session_command_op_remove 7 15 2024-02-23T00:42:30.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.084+0000 7fdfa43ac700 10 client.24679.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:42:30.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.084+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_assign -1 16 2024-02-23T00:42:30.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.084+0000 7fdfa43ac700 20 client.24679.objecter _get_session s=0x7fdf9c13b080 osd=7 3 2024-02-23T00:42:30.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.084+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c13b080 osd=7 3 2024-02-23T00:42:30.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.084+0000 7fdfa43ac700 20 client.24679.objecter _get_session s=0x7fdf9c13b080 osd=7 3 2024-02-23T00:42:30.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.084+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_remove -1 16 2024-02-23T00:42:30.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.084+0000 7fdfa43ac700 20 client.24679.objecter get_session s=0x7fdf9c13b080 osd=7 3 2024-02-23T00:42:30.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.084+0000 7fdfa43ac700 15 client.24679.objecter _session_command_op_assign 7 16 2024-02-23T00:42:30.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.084+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c13b080 osd=7 4 2024-02-23T00:42:30.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.084+0000 7fdfa43ac700 10 client.24679.objecter _send_command 16 2024-02-23T00:42:30.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.084+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fdf9c138840 con 0x7fdf9c13a3f0 2024-02-23T00:42:30.090 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:42:30.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdf937fe700 1 -- 172.21.15.47:0/2315457400 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fdf9c138840 con 0x7fdf9c13a3f0 2024-02-23T00:42:30.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdf937fe700 10 client.24679.objecter ms_dispatch 0x7fdf9c001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:42:30.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdf937fe700 10 client.24679.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:42:30.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdf937fe700 20 client.24679.objecter put_session s=0x7fdf9c13b080 osd=7 4 2024-02-23T00:42:30.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdf937fe700 15 client.24679.objecter _session_command_op_remove 7 16 2024-02-23T00:42:30.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdfa43ac700 10 librados: watch_flush enter 2024-02-23T00:42:30.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdfa43ac700 10 librados: watch_flush exit 2024-02-23T00:42:30.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdfa43ac700 10 client.24679.objecter close_session for osd.0 2024-02-23T00:42:30.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fdf9c028540 msgr2=0x7fdf9c0fd9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:30.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdfa43ac700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fdf9c028540 0x7fdf9c0fd9b0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:30.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c05ee80 osd=0 1 2024-02-23T00:42:30.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdfa43ac700 10 client.24679.objecter close_session for osd.1 2024-02-23T00:42:30.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fdf9c0fe520 msgr2=0x7fdf9c0fe900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:30.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdfa43ac700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fdf9c0fe520 0x7fdf9c0fe900 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:30.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c0fdea0 osd=1 1 2024-02-23T00:42:30.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdfa43ac700 10 client.24679.objecter close_session for osd.2 2024-02-23T00:42:30.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fdf9c0594d0 msgr2=0x7fdf9c1183a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:30.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdfa43ac700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fdf9c0594d0 0x7fdf9c1183a0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:30.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c059360 osd=2 1 2024-02-23T00:42:30.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdfa43ac700 10 client.24679.objecter close_session for osd.3 2024-02-23T00:42:30.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fdf9c027770 msgr2=0x7fdf9c12e860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:30.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdfa43ac700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fdf9c027770 0x7fdf9c12e860 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:30.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c027600 osd=3 1 2024-02-23T00:42:30.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdfa43ac700 10 client.24679.objecter close_session for osd.4 2024-02-23T00:42:30.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fdf9c115e90 msgr2=0x7fdf9c12ed50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:30.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdfa43ac700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fdf9c115e90 0x7fdf9c12ed50 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:30.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c118e00 osd=4 1 2024-02-23T00:42:30.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdfa43ac700 10 client.24679.objecter close_session for osd.5 2024-02-23T00:42:30.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fdf9c12f780 msgr2=0x7fdf9c12fbe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:30.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdfa43ac700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fdf9c12f780 0x7fdf9c12fbe0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:30.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c119000 osd=5 1 2024-02-23T00:42:30.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdfa43ac700 10 client.24679.objecter close_session for osd.6 2024-02-23T00:42:30.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.087+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fdf9c135dd0 msgr2=0x7fdf9c138240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:30.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.086+0000 7fdfa2949700 1 -- 172.21.15.47:0/2315457400 reap_dead start 2024-02-23T00:42:30.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.087+0000 7fdfa43ac700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fdf9c135dd0 0x7fdf9c138240 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:30.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.087+0000 7fdfa2949700 1 -- 172.21.15.47:0/2315457400 reap_dead start 2024-02-23T00:42:30.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.087+0000 7fdfa2949700 1 -- 172.21.15.47:0/2315457400 reap_dead start 2024-02-23T00:42:30.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.087+0000 7fdfa2949700 1 -- 172.21.15.47:0/2315457400 reap_dead start 2024-02-23T00:42:30.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.087+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c135750 osd=6 1 2024-02-23T00:42:30.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.087+0000 7fdfa43ac700 10 client.24679.objecter close_session for osd.7 2024-02-23T00:42:30.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.087+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fdf9c13a3f0 msgr2=0x7fdf9c13a870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:30.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.087+0000 7fdfa43ac700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fdf9c13a3f0 0x7fdf9c13a870 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:30.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.087+0000 7fdfa43ac700 20 client.24679.objecter put_session s=0x7fdf9c13b080 osd=7 1 2024-02-23T00:42:30.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.087+0000 7fdfa43ac700 20 client.24679.objecter shutdown clearing up homeless session... 2024-02-23T00:42:30.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.087+0000 7fdfa43ac700 10 client.24679.objecter successfully canceled tick 2024-02-23T00:42:30.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.087+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdf88062cc0 msgr2=0x7fdf88065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:30.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.087+0000 7fdfa43ac700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdf88062cc0 0x7fdf88065180 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fdf98007980 tx=0x7fdf9800d040).stop 2024-02-23T00:42:30.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.087+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf9c0fd510 msgr2=0x7fdf9c10a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:30.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.087+0000 7fdfa43ac700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf9c0fd510 0x7fdf9c10a3f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fdf9400eb70 tx=0x7fdf9400b040).stop 2024-02-23T00:42:30.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.087+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 shutdown_connections 2024-02-23T00:42:30.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.087+0000 7fdfa43ac700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdf88062cc0 0x7fdf88065180 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:30.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.087+0000 7fdfa43ac700 1 --2- 172.21.15.47:0/2315457400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf9c0fd510 0x7fdf9c10a3f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:30.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.087+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 >> 172.21.15.47:0/2315457400 conn(0x7fdf9c0f2820 msgr2=0x7fdf9c0fb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:42:30.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.087+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 shutdown_connections 2024-02-23T00:42:30.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.087+0000 7fdfa43ac700 1 -- 172.21.15.47:0/2315457400 wait complete. 2024-02-23T00:42:30.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:30.087+0000 7fdfa43ac700 1 librados: shutdown 2024-02-23T00:42:30.095 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:42:30.095 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:42:30.095 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:42:30.095 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:42:30.095 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:42:30.095 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:42:30.096 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:42:30.096 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:42:30.148 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:42:30.188 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:30.392 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:42:30.396 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:30.396 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:42:30.497 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:42:30.533 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:30.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:29.165965+0000 mgr.y (mgr.14517) 128 : cluster [DBG] pgmap v114: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:42:30.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:29.875038+0000 mon.b (mon.1) 33 : audit [DBG] from='client.? 172.21.15.47:0/2315457400' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:42:30.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:29.950712+0000 osd.2 (osd.2) 55 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:42:30.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:29.950775+0000 osd.2 (osd.2) 56 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:42:30.807 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:42:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:42:30] "GET /metrics HTTP/1.1" 200 172263 "" "Prometheus/2.33.4" 2024-02-23T00:42:30.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:29.165965+0000 mgr.y (mgr.14517) 128 : cluster [DBG] pgmap v114: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:42:30.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:29.875038+0000 mon.b (mon.1) 33 : audit [DBG] from='client.? 172.21.15.47:0/2315457400' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:42:30.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:29.950712+0000 osd.2 (osd.2) 55 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:42:30.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:29.950775+0000 osd.2 (osd.2) 56 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:42:30.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:29.165965+0000 mgr.y (mgr.14517) 128 : cluster [DBG] pgmap v114: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:42:30.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:29.875038+0000 mon.b (mon.1) 33 : audit [DBG] from='client.? 172.21.15.47:0/2315457400' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:42:30.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:29.950712+0000 osd.2 (osd.2) 55 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:42:30.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:29.950775+0000 osd.2 (osd.2) 56 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:42:31.085 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:42:31.092 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:31.092 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:42:31.193 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:42:31.223 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:31.476 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:42:31.479 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:31.480 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:42:31.581 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:42:31.614 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:31.635 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:42:31.638 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:31.638 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:42:31.732 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-23T00:42:31.732 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-23T00:42:31.732 INFO:tasks.thrashosds.thrasher:Shrinking pool default.rgw.meta 2024-02-23T00:42:31.732 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool default.rgw.meta by 10 min 8 2024-02-23T00:42:31.733 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph pg dump --format=json 2024-02-23T00:42:31.739 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:42:31.763 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:30.442740+0000 mon.a (mon.0) 781 : cluster [DBG] osdmap e101: 8 total, 8 up, 8 in 2024-02-23T00:42:31.764 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:30.442740+0000 mon.a (mon.0) 781 : cluster [DBG] osdmap e101: 8 total, 8 up, 8 in 2024-02-23T00:42:31.796 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:31.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:30.442740+0000 mon.a (mon.0) 781 : cluster [DBG] osdmap e101: 8 total, 8 up, 8 in 2024-02-23T00:42:32.326 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:42:32.356 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:42:32.363 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:32.364 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:42:32.381 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:42:32.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.403+0000 7f4f5bd1d700 1 -- 172.21.15.47:0/2897808599 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4f54104220 msgr2=0x7f4f54104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:32.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.403+0000 7f4f5bd1d700 1 --2- 172.21.15.47:0/2897808599 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4f54104220 0x7f4f54104600 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f4f50004870 tx=0x7f4f50009cb0).stop 2024-02-23T00:42:32.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.403+0000 7f4f5bd1d700 1 -- 172.21.15.47:0/2897808599 shutdown_connections 2024-02-23T00:42:32.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.403+0000 7f4f5bd1d700 1 --2- 172.21.15.47:0/2897808599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f54069cd0 0x7f4f54070f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:42:32.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.403+0000 7f4f5bd1d700 1 --2- 172.21.15.47:0/2897808599 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4f54104220 0x7f4f54104600 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:32.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.403+0000 7f4f5bd1d700 1 --2- 172.21.15.47:0/2897808599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f54104b80 0x7f4f540695f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:42:32.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.403+0000 7f4f5bd1d700 1 -- 172.21.15.47:0/2897808599 >> 172.21.15.47:0/2897808599 conn(0x7f4f54063db0 msgr2=0x7f4f54067560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:42:32.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.404+0000 7f4f5bd1d700 1 -- 172.21.15.47:0/2897808599 shutdown_connections 2024-02-23T00:42:32.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.408+0000 7f4f5bd1d700 1 -- 172.21.15.47:0/2897808599 wait complete. 2024-02-23T00:42:32.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.408+0000 7f4f5bd1d700 1 librados: starting msgr at 2024-02-23T00:42:32.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.408+0000 7f4f5bd1d700 1 librados: starting objecter 2024-02-23T00:42:32.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.409+0000 7f4f5bd1d700 1 Processor -- start 2024-02-23T00:42:32.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.409+0000 7f4f5bd1d700 1 -- start start 2024-02-23T00:42:32.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.409+0000 7f4f5bd1d700 1 librados: setting wanted keys 2024-02-23T00:42:32.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.409+0000 7f4f5bd1d700 1 librados: calling monclient init 2024-02-23T00:42:32.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.409+0000 7f4f5bd1d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f54069cd0 0x7f4f54070c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:32.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.409+0000 7f4f5bd1d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f54104220 0x7f4f5407ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:32.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.409+0000 7f4f5bd1d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4f54104b80 0x7f4f5407b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:32.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.409+0000 7f4f5bd1d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5407b870 con 0x7f4f54069cd0 2024-02-23T00:42:32.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.409+0000 7f4f5bd1d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5407b9b0 con 0x7f4f54104220 2024-02-23T00:42:32.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.409+0000 7f4f5bd1d700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5407bb70 con 0x7f4f54104b80 2024-02-23T00:42:32.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.409+0000 7f4f592b8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f54104220 0x7f4f5407ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:32.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.410+0000 7f4f592b8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f54104220 0x7f4f5407ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.47:44010/0 (socket says 172.21.15.47:44010) 2024-02-23T00:42:32.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.410+0000 7f4f592b8700 1 -- 172.21.15.47:0/3903471552 learned_addr learned my addr 172.21.15.47:0/3903471552 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:42:32.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.410+0000 7f4f59ab9700 1 --2- 172.21.15.47:0/3903471552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f54069cd0 0x7f4f54070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:32.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.410+0000 7f4f592b8700 1 -- 172.21.15.47:0/3903471552 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4f54104b80 msgr2=0x7f4f5407b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:32.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.410+0000 7f4f592b8700 1 --2- 172.21.15.47:0/3903471552 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4f54104b80 0x7f4f5407b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:42:32.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.410+0000 7f4f592b8700 1 -- 172.21.15.47:0/3903471552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f54069cd0 msgr2=0x7f4f54070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:32.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.410+0000 7f4f592b8700 1 --2- 172.21.15.47:0/3903471552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f54069cd0 0x7f4f54070c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:32.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.410+0000 7f4f592b8700 1 -- 172.21.15.47:0/3903471552 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f50005040 con 0x7f4f54104220 2024-02-23T00:42:32.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.410+0000 7f4f592b8700 1 --2- 172.21.15.47:0/3903471552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f54104220 0x7f4f5407ae00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f4f4400b490 tx=0x7f4f44009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:32.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.410+0000 7f4f4affd700 10 client.?.objecter ms_handle_connect 0x7f4f54104220 2024-02-23T00:42:32.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.410+0000 7f4f4affd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:42:32.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.410+0000 7f4f4affd700 1 -- 172.21.15.47:0/3903471552 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f440090d0 con 0x7f4f54104220 2024-02-23T00:42:32.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.410+0000 7f4f5bd1d700 1 -- 172.21.15.47:0/3903471552 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f5406e800 con 0x7f4f54104220 2024-02-23T00:42:32.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.410+0000 7f4f4affd700 1 -- 172.21.15.47:0/3903471552 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f4f440128a0 con 0x7f4f54104220 2024-02-23T00:42:32.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.410+0000 7f4f4affd700 1 -- 172.21.15.47:0/3903471552 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4400d750 con 0x7f4f54104220 2024-02-23T00:42:32.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.410+0000 7f4f5bd1d700 10 client.24682.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:42:32.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.411+0000 7f4f5bd1d700 1 -- 172.21.15.47:0/3903471552 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f5406ecf0 con 0x7f4f54104220 2024-02-23T00:42:32.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.411+0000 7f4f5bd1d700 1 librados: init done 2024-02-23T00:42:32.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.413+0000 7f4f4affd700 1 -- 172.21.15.47:0/3903471552 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f4f4400db80 con 0x7f4f54104220 2024-02-23T00:42:32.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.413+0000 7f4f4affd700 1 --2- 172.21.15.47:0/3903471552 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4f40062d10 0x7f4f400651d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:32.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.413+0000 7f4f4affd700 1 -- 172.21.15.47:0/3903471552 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(102..102 src has 48..102) v4 ==== 7453+0+0 (secure 0 0 0) 0x7f4f4407dba0 con 0x7f4f54104220 2024-02-23T00:42:32.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.413+0000 7f4f4affd700 10 client.24682.objecter ms_dispatch 0x7f4f54001550 osd_map(102..102 src has 48..102) v4 2024-02-23T00:42:32.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.413+0000 7f4f4affd700 3 client.24682.objecter handle_osd_map got epochs [102,102] > 0 2024-02-23T00:42:32.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.413+0000 7f4f4affd700 3 client.24682.objecter handle_osd_map decoding full epoch 102 2024-02-23T00:42:32.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.413+0000 7f4f4affd700 20 client.24682.objecter dump_active .. 0 homeless 2024-02-23T00:42:32.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.413+0000 7f4f59ab9700 1 --2- 172.21.15.47:0/3903471552 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4f40062d10 0x7f4f400651d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:32.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.415+0000 7f4f4affd700 10 client.24682.objecter ms_handle_connect 0x7f4f40062d10 2024-02-23T00:42:32.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.415+0000 7f4f59ab9700 1 --2- 172.21.15.47:0/3903471552 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4f40062d10 0x7f4f400651d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f4f50004540 tx=0x7f4f50009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:32.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.417+0000 7f4f5bd1d700 1 -- 172.21.15.47:0/3903471552 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f38004f50 con 0x7f4f54104220 2024-02-23T00:42:32.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.422+0000 7f4f4affd700 1 -- 172.21.15.47:0/3903471552 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f4f4404d980 con 0x7f4f54104220 2024-02-23T00:42:32.465 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:42:32.498 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:32.527 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:42:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:42:32.527 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:31.166279+0000 mgr.y (mgr.14517) 129 : cluster [DBG] pgmap v116: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:42:32.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:31.394172+0000 osd.1 (osd.1) 65 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:42:32.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:31.394233+0000 osd.1 (osd.1) 66 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:42:32.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:31.459504+0000 mon.a (mon.0) 782 : cluster [DBG] osdmap e102: 8 total, 8 up, 8 in 2024-02-23T00:42:32.645 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:42:32.650 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:31.166279+0000 mgr.y (mgr.14517) 129 : cluster [DBG] pgmap v116: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:42:32.651 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:31.394172+0000 osd.1 (osd.1) 65 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:42:32.651 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:31.394233+0000 osd.1 (osd.1) 66 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:42:32.651 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:31.459504+0000 mon.a (mon.0) 782 : cluster [DBG] osdmap e102: 8 total, 8 up, 8 in 2024-02-23T00:42:32.651 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:31.166279+0000 mgr.y (mgr.14517) 129 : cluster [DBG] pgmap v116: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:42:32.651 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:31.394172+0000 osd.1 (osd.1) 65 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:42:32.651 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:31.394233+0000 osd.1 (osd.1) 66 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:42:32.651 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:31.459504+0000 mon.a (mon.0) 782 : cluster [DBG] osdmap e102: 8 total, 8 up, 8 in 2024-02-23T00:42:32.652 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:32.652 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:42:32.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:32.724+0000 7f4f5bd1d700 1 -- 172.21.15.47:0/3903471552 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7f4f38005200 con 0x7f4f54104220 2024-02-23T00:42:32.753 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:42:32.786 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:32.943 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:42:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:42:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:42:33.343 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:42:33.346 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:33.347 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:42:33.448 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:42:33.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:33.474+0000 7f4f4affd700 1 -- 172.21.15.47:0/3903471552 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v103) v1 ==== 91+0+0 (secure 0 0 0) 0x7f4f44050fa0 con 0x7f4f54104220 2024-02-23T00:42:33.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:33.477+0000 7f4f5bd1d700 10 librados: watch_flush enter 2024-02-23T00:42:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:33.481+0000 7f4f5bd1d700 10 librados: watch_flush exit 2024-02-23T00:42:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:33.481+0000 7f4f5bd1d700 20 client.24682.objecter shutdown clearing up homeless session... 2024-02-23T00:42:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:33.481+0000 7f4f5bd1d700 10 client.24682.objecter successfully canceled tick 2024-02-23T00:42:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:33.481+0000 7f4f5bd1d700 1 -- 172.21.15.47:0/3903471552 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4f40062d10 msgr2=0x7f4f400651d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:33.481+0000 7f4f5bd1d700 1 --2- 172.21.15.47:0/3903471552 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4f40062d10 0x7f4f400651d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f4f50004540 tx=0x7f4f50009cb0).stop 2024-02-23T00:42:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:33.481+0000 7f4f5bd1d700 1 -- 172.21.15.47:0/3903471552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f54104220 msgr2=0x7f4f5407ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:33.481+0000 7f4f5bd1d700 1 --2- 172.21.15.47:0/3903471552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f54104220 0x7f4f5407ae00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f4f4400b490 tx=0x7f4f44009060).stop 2024-02-23T00:42:33.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:33.481+0000 7f4f5bd1d700 1 -- 172.21.15.47:0/3903471552 shutdown_connections 2024-02-23T00:42:33.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:33.481+0000 7f4f5bd1d700 1 --2- 172.21.15.47:0/3903471552 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4f40062d10 0x7f4f400651d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:33.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:33.481+0000 7f4f5bd1d700 1 --2- 172.21.15.47:0/3903471552 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4f54104b80 0x7f4f5407b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:42:33.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:33.481+0000 7f4f5bd1d700 1 --2- 172.21.15.47:0/3903471552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f54069cd0 0x7f4f54070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:33.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:33.481+0000 7f4f5bd1d700 1 --2- 172.21.15.47:0/3903471552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f54104220 0x7f4f5407ae00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:33.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:33.481+0000 7f4f5bd1d700 1 -- 172.21.15.47:0/3903471552 >> 172.21.15.47:0/3903471552 conn(0x7f4f54063db0 msgr2=0x7f4f540ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:42:33.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:33.481+0000 7f4f5bd1d700 1 -- 172.21.15.47:0/3903471552 shutdown_connections 2024-02-23T00:42:33.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:33.481+0000 7f4f5bd1d700 1 -- 172.21.15.47:0/3903471552 wait complete. 2024-02-23T00:42:33.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:33.482+0000 7f4f5bd1d700 1 librados: shutdown 2024-02-23T00:42:33.485 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is unset 2024-02-23T00:42:33.494 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:33.517 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:42:33.521 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:33.521 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:42:33.622 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:42:33.654 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:33.675 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:42:33.678 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:33.679 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:42:33.779 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:42:33.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:32.261618+0000 mgr.y (mgr.14517) 130 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:42:33.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:32.726827+0000 mon.b (mon.1) 34 : audit [INF] from='client.? 172.21.15.47:0/3903471552' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:42:33.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:32.727624+0000 mon.a (mon.0) 783 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:42:33.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:32.261618+0000 mgr.y (mgr.14517) 130 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:42:33.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:32.726827+0000 mon.b (mon.1) 34 : audit [INF] from='client.? 172.21.15.47:0/3903471552' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:42:33.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:32.727624+0000 mon.a (mon.0) 783 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:42:33.810 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:33.925 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:42:33.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:32.261618+0000 mgr.y (mgr.14517) 130 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:42:33.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:32.726827+0000 mon.b (mon.1) 34 : audit [INF] from='client.? 172.21.15.47:0/3903471552' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:42:33.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:32.727624+0000 mon.a (mon.0) 783 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:42:34.260 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:42:34.264 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:34.264 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:42:34.365 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:42:34.393 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:34.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:33.167116+0000 mgr.y (mgr.14517) 131 : cluster [DBG] pgmap v118: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-23T00:42:34.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:33.458057+0000 mon.a (mon.0) 784 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:42:34.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:33.473892+0000 mon.a (mon.0) 785 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:42:34.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:33.474593+0000 mon.a (mon.0) 786 : cluster [DBG] osdmap e103: 8 total, 8 up, 8 in 2024-02-23T00:42:34.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:33.167116+0000 mgr.y (mgr.14517) 131 : cluster [DBG] pgmap v118: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-23T00:42:34.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:33.458057+0000 mon.a (mon.0) 784 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:42:34.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:33.473892+0000 mon.a (mon.0) 785 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:42:34.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:33.474593+0000 mon.a (mon.0) 786 : cluster [DBG] osdmap e103: 8 total, 8 up, 8 in 2024-02-23T00:42:34.762 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:42:34.769 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:34.769 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:42:34.870 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:42:34.898 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:34.927 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:42:34.937 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:34.938 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:42:34.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:33.167116+0000 mgr.y (mgr.14517) 131 : cluster [DBG] pgmap v118: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-23T00:42:34.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:33.458057+0000 mon.a (mon.0) 784 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:42:34.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:33.473892+0000 mon.a (mon.0) 785 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:42:34.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:33.474593+0000 mon.a (mon.0) 786 : cluster [DBG] osdmap e103: 8 total, 8 up, 8 in 2024-02-23T00:42:35.039 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:42:35.076 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:35.333 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:42:35.337 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:35.337 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:42:35.438 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:42:35.471 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:35.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:34.244908+0000 osd.0 (osd.0) 55 : cluster [DBG] 4.3 scrub starts 2024-02-23T00:42:35.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:34.248363+0000 osd.0 (osd.0) 56 : cluster [DBG] 4.3 scrub ok 2024-02-23T00:42:35.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:34.630031+0000 osd.1 (osd.1) 67 : cluster [DBG] 4.2 scrub starts 2024-02-23T00:42:35.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:34.633545+0000 osd.1 (osd.1) 68 : cluster [DBG] 4.2 scrub ok 2024-02-23T00:42:35.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:34.825302+0000 osd.5 (osd.5) 63 : cluster [DBG] 4.15 scrub starts 2024-02-23T00:42:35.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:34.828987+0000 osd.5 (osd.5) 64 : cluster [DBG] 4.15 scrub ok 2024-02-23T00:42:35.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:34.983520+0000 osd.3 (osd.3) 71 : cluster [DBG] 3.2 scrub starts 2024-02-23T00:42:35.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:34.984676+0000 osd.3 (osd.3) 72 : cluster [DBG] 3.2 scrub ok 2024-02-23T00:42:35.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:35.225300+0000 osd.2 (osd.2) 57 : cluster [DBG] 6.14 scrub starts 2024-02-23T00:42:35.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:35.226334+0000 osd.2 (osd.2) 58 : cluster [DBG] 6.14 scrub ok 2024-02-23T00:42:35.492 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:42:35.495 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:35.495 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:42:35.596 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:42:35.624 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:34.244908+0000 osd.0 (osd.0) 55 : cluster [DBG] 4.3 scrub starts 2024-02-23T00:42:35.624 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:34.248363+0000 osd.0 (osd.0) 56 : cluster [DBG] 4.3 scrub ok 2024-02-23T00:42:35.624 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:34.630031+0000 osd.1 (osd.1) 67 : cluster [DBG] 4.2 scrub starts 2024-02-23T00:42:35.624 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:34.633545+0000 osd.1 (osd.1) 68 : cluster [DBG] 4.2 scrub ok 2024-02-23T00:42:35.624 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:34.825302+0000 osd.5 (osd.5) 63 : cluster [DBG] 4.15 scrub starts 2024-02-23T00:42:35.625 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:34.828987+0000 osd.5 (osd.5) 64 : cluster [DBG] 4.15 scrub ok 2024-02-23T00:42:35.625 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:34.983520+0000 osd.3 (osd.3) 71 : cluster [DBG] 3.2 scrub starts 2024-02-23T00:42:35.625 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:34.984676+0000 osd.3 (osd.3) 72 : cluster [DBG] 3.2 scrub ok 2024-02-23T00:42:35.625 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:35.225300+0000 osd.2 (osd.2) 57 : cluster [DBG] 6.14 scrub starts 2024-02-23T00:42:35.625 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:35.226334+0000 osd.2 (osd.2) 58 : cluster [DBG] 6.14 scrub ok 2024-02-23T00:42:35.625 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:35.658 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:42:35.665 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:35.666 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:42:35.767 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:42:35.797 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:34.244908+0000 osd.0 (osd.0) 55 : cluster [DBG] 4.3 scrub starts 2024-02-23T00:42:35.797 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:34.248363+0000 osd.0 (osd.0) 56 : cluster [DBG] 4.3 scrub ok 2024-02-23T00:42:35.797 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:34.630031+0000 osd.1 (osd.1) 67 : cluster [DBG] 4.2 scrub starts 2024-02-23T00:42:35.797 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:34.633545+0000 osd.1 (osd.1) 68 : cluster [DBG] 4.2 scrub ok 2024-02-23T00:42:35.797 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:34.825302+0000 osd.5 (osd.5) 63 : cluster [DBG] 4.15 scrub starts 2024-02-23T00:42:35.797 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:34.828987+0000 osd.5 (osd.5) 64 : cluster [DBG] 4.15 scrub ok 2024-02-23T00:42:35.797 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:34.983520+0000 osd.3 (osd.3) 71 : cluster [DBG] 3.2 scrub starts 2024-02-23T00:42:35.798 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:34.984676+0000 osd.3 (osd.3) 72 : cluster [DBG] 3.2 scrub ok 2024-02-23T00:42:35.798 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:35.225300+0000 osd.2 (osd.2) 57 : cluster [DBG] 6.14 scrub starts 2024-02-23T00:42:35.798 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:35.226334+0000 osd.2 (osd.2) 58 : cluster [DBG] 6.14 scrub ok 2024-02-23T00:42:35.799 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:35.820 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:42:35.823 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:35.824 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:42:35.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.916+0000 7f339db56700 1 -- 172.21.15.47:0/3867990699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33980f9610 msgr2=0x7f3398101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:35.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.916+0000 7f339db56700 1 --2- 172.21.15.47:0/3867990699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33980f9610 0x7f3398101d20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f338c004870 tx=0x7f338c009cb0).stop 2024-02-23T00:42:35.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.916+0000 7f339db56700 1 -- 172.21.15.47:0/3867990699 shutdown_connections 2024-02-23T00:42:35.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.916+0000 7f339db56700 1 --2- 172.21.15.47:0/3867990699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3398102210 0x7f3398104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:35.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.916+0000 7f339db56700 1 --2- 172.21.15.47:0/3867990699 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f33980f8cb0 0x7f33980f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:35.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.916+0000 7f339db56700 1 --2- 172.21.15.47:0/3867990699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33980f9610 0x7f3398101d20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:35.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.916+0000 7f339db56700 1 -- 172.21.15.47:0/3867990699 >> 172.21.15.47:0/3867990699 conn(0x7f33980f2820 msgr2=0x7f33980f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:42:35.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.916+0000 7f339db56700 1 -- 172.21.15.47:0/3867990699 shutdown_connections 2024-02-23T00:42:35.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.916+0000 7f339db56700 1 -- 172.21.15.47:0/3867990699 wait complete. 2024-02-23T00:42:35.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.917+0000 7f339db56700 1 librados: starting msgr at 2024-02-23T00:42:35.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.917+0000 7f339db56700 1 librados: starting objecter 2024-02-23T00:42:35.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.917+0000 7f339db56700 1 Processor -- start 2024-02-23T00:42:35.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.918+0000 7f339db56700 1 -- start start 2024-02-23T00:42:35.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.918+0000 7f339db56700 1 librados: setting wanted keys 2024-02-23T00:42:35.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.918+0000 7f339db56700 1 librados: calling monclient init 2024-02-23T00:42:35.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.918+0000 7f339db56700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33980f8cb0 0x7f33980f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:35.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.918+0000 7f339db56700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33980f9610 0x7f33980f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:35.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.918+0000 7f339db56700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3398102210 0x7f33980f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:35.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.918+0000 7f339db56700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3398110040 con 0x7f33980f9610 2024-02-23T00:42:35.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.918+0000 7f339db56700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33981101b0 con 0x7f33980f8cb0 2024-02-23T00:42:35.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.918+0000 7f339db56700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f339811ac00 con 0x7f3398102210 2024-02-23T00:42:35.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.918+0000 7f33977fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33980f8cb0 0x7f33980f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:35.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.918+0000 7f33977fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33980f8cb0 0x7f33980f6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.47:44016/0 (socket says 172.21.15.47:44016) 2024-02-23T00:42:35.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.918+0000 7f33977fe700 1 -- 172.21.15.47:0/67016437 learned_addr learned my addr 172.21.15.47:0/67016437 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:42:35.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.918+0000 7f3396ffd700 1 --2- 172.21.15.47:0/67016437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33980f9610 0x7f33980f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:35.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.919+0000 7f33977fe700 1 -- 172.21.15.47:0/67016437 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3398102210 msgr2=0x7f33980f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:35.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.919+0000 7f33977fe700 1 --2- 172.21.15.47:0/67016437 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3398102210 0x7f33980f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:42:35.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.919+0000 7f33977fe700 1 -- 172.21.15.47:0/67016437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33980f9610 msgr2=0x7f33980f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:35.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.919+0000 7f33977fe700 1 --2- 172.21.15.47:0/67016437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33980f9610 0x7f33980f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:35.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.919+0000 7f33977fe700 1 -- 172.21.15.47:0/67016437 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338c005040 con 0x7f33980f8cb0 2024-02-23T00:42:35.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.919+0000 7f3396ffd700 1 --2- 172.21.15.47:0/67016437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33980f9610 0x7f33980f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:42:35.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.919+0000 7f33977fe700 1 --2- 172.21.15.47:0/67016437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33980f8cb0 0x7f33980f6c60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f338800c7c0 tx=0x7f3388007ef0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:35.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.919+0000 7f3394ff9700 10 client.?.objecter ms_handle_connect 0x7f33980f8cb0 2024-02-23T00:42:35.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.919+0000 7f3394ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:42:35.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.919+0000 7f3394ff9700 1 -- 172.21.15.47:0/67016437 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3388004430 con 0x7f33980f8cb0 2024-02-23T00:42:35.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.919+0000 7f3394ff9700 1 -- 172.21.15.47:0/67016437 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f338800e930 con 0x7f33980f8cb0 2024-02-23T00:42:35.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.919+0000 7f3394ff9700 1 -- 172.21.15.47:0/67016437 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3388013990 con 0x7f33980f8cb0 2024-02-23T00:42:35.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.919+0000 7f339db56700 1 -- 172.21.15.47:0/67016437 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33981104e0 con 0x7f33980f8cb0 2024-02-23T00:42:35.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.919+0000 7f339db56700 10 client.24688.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:42:35.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.919+0000 7f339db56700 1 -- 172.21.15.47:0/67016437 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3398110980 con 0x7f33980f8cb0 2024-02-23T00:42:35.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.919+0000 7f339db56700 1 librados: init done 2024-02-23T00:42:35.926 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:42:35.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.921+0000 7f339db56700 1 -- 172.21.15.47:0/67016437 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f339805fa60 con 0x7f33980f8cb0 2024-02-23T00:42:35.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.921+0000 7f3394ff9700 1 -- 172.21.15.47:0/67016437 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f338800e4a0 con 0x7f33980f8cb0 2024-02-23T00:42:35.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.922+0000 7f3394ff9700 1 --2- 172.21.15.47:0/67016437 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3380062cc0 0x7f3380065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:35.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.922+0000 7f3394ff9700 1 -- 172.21.15.47:0/67016437 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(103..103 src has 48..103) v4 ==== 7453+0+0 (secure 0 0 0) 0x7f338807d0f0 con 0x7f33980f8cb0 2024-02-23T00:42:35.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.922+0000 7f3394ff9700 10 client.24688.objecter ms_dispatch 0x7f3398001550 osd_map(103..103 src has 48..103) v4 2024-02-23T00:42:35.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.922+0000 7f3394ff9700 3 client.24688.objecter handle_osd_map got epochs [103,103] > 0 2024-02-23T00:42:35.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.922+0000 7f3394ff9700 3 client.24688.objecter handle_osd_map decoding full epoch 103 2024-02-23T00:42:35.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.922+0000 7f3394ff9700 20 client.24688.objecter dump_active .. 0 homeless 2024-02-23T00:42:35.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.922+0000 7f3396ffd700 1 --2- 172.21.15.47:0/67016437 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3380062cc0 0x7f3380065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:35.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.922+0000 7f3394ff9700 10 client.24688.objecter ms_handle_connect 0x7f3380062cc0 2024-02-23T00:42:35.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.922+0000 7f3396ffd700 1 --2- 172.21.15.47:0/67016437 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3380062cc0 0x7f3380065180 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f33980f8400 tx=0x7f338c009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:35.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:35.927+0000 7f3394ff9700 1 -- 172.21.15.47:0/67016437 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f338804ced0 con 0x7f33980f8cb0 2024-02-23T00:42:35.955 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:35.976 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:42:35.978 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:35.979 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:42:36.029 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:42:36.079 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:42:36.119 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:36.153 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:42:36.170 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:36.170 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:42:36.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:36.175+0000 7f339db56700 1 -- 172.21.15.47:0/67016437 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f33980f81b0 con 0x7f3380062cc0 2024-02-23T00:42:36.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:36.188+0000 7f3394ff9700 1 -- 172.21.15.47:0/67016437 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+357409 (secure 0 0 0) 0x7f33980f81b0 con 0x7f3380062cc0 2024-02-23T00:42:36.195 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:42:36.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:36.191+0000 7f339db56700 10 librados: watch_flush enter 2024-02-23T00:42:36.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:36.192+0000 7f339db56700 10 librados: watch_flush exit 2024-02-23T00:42:36.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:36.192+0000 7f339db56700 20 client.24688.objecter shutdown clearing up homeless session... 2024-02-23T00:42:36.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:36.192+0000 7f339db56700 10 client.24688.objecter successfully canceled tick 2024-02-23T00:42:36.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:36.192+0000 7f339db56700 1 -- 172.21.15.47:0/67016437 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3380062cc0 msgr2=0x7f3380065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:36.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:36.192+0000 7f339db56700 1 --2- 172.21.15.47:0/67016437 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3380062cc0 0x7f3380065180 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f33980f8400 tx=0x7f338c009cb0).stop 2024-02-23T00:42:36.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:36.192+0000 7f339db56700 1 -- 172.21.15.47:0/67016437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33980f8cb0 msgr2=0x7f33980f6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:36.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:36.192+0000 7f339db56700 1 --2- 172.21.15.47:0/67016437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33980f8cb0 0x7f33980f6c60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f338800c7c0 tx=0x7f3388007ef0).stop 2024-02-23T00:42:36.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:36.192+0000 7f339db56700 1 -- 172.21.15.47:0/67016437 shutdown_connections 2024-02-23T00:42:36.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:36.192+0000 7f339db56700 1 --2- 172.21.15.47:0/67016437 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3380062cc0 0x7f3380065180 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:36.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:36.192+0000 7f339db56700 1 --2- 172.21.15.47:0/67016437 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3398102210 0x7f33980f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:42:36.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:36.192+0000 7f339db56700 1 --2- 172.21.15.47:0/67016437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33980f8cb0 0x7f33980f6c60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:36.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:36.192+0000 7f339db56700 1 --2- 172.21.15.47:0/67016437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33980f9610 0x7f33980f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:36.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:36.192+0000 7f339db56700 1 -- 172.21.15.47:0/67016437 >> 172.21.15.47:0/67016437 conn(0x7f33980f2820 msgr2=0x7f33980f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:42:36.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:36.192+0000 7f339db56700 1 -- 172.21.15.47:0/67016437 shutdown_connections 2024-02-23T00:42:36.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:36.193+0000 7f339db56700 1 -- 172.21.15.47:0/67016437 wait complete. 2024-02-23T00:42:36.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:36.193+0000 7f339db56700 1 librados: shutdown 2024-02-23T00:42:36.200 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-02-23T00:42:36.271 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:42:36.305 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:36.481 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:35.167505+0000 mgr.y (mgr.14517) 132 : cluster [DBG] pgmap v120: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-02-23T00:42:36.482 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:36.212987+0000 osd.2 (osd.2) 59 : cluster [DBG] 5.d scrub starts 2024-02-23T00:42:36.482 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:36.214820+0000 osd.2 (osd.2) 60 : cluster [DBG] 5.d scrub ok 2024-02-23T00:42:36.482 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:35.167505+0000 mgr.y (mgr.14517) 132 : cluster [DBG] pgmap v120: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-02-23T00:42:36.482 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:36.212987+0000 osd.2 (osd.2) 59 : cluster [DBG] 5.d scrub starts 2024-02-23T00:42:36.482 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:36.214820+0000 osd.2 (osd.2) 60 : cluster [DBG] 5.d scrub ok 2024-02-23T00:42:36.657 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:42:36.663 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:36.663 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:42:36.764 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:42:36.795 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:35.167505+0000 mgr.y (mgr.14517) 132 : cluster [DBG] pgmap v120: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-02-23T00:42:36.795 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:36.212987+0000 osd.2 (osd.2) 59 : cluster [DBG] 5.d scrub starts 2024-02-23T00:42:36.795 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:36.214820+0000 osd.2 (osd.2) 60 : cluster [DBG] 5.d scrub ok 2024-02-23T00:42:36.797 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:36.818 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:42:36.822 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:36.822 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:42:36.923 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:42:36.953 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:36.973 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:42:36.976 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:36.976 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:42:37.077 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:42:37.108 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:37.135 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:42:37.140 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:37.140 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:42:37.241 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:42:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:37.356 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-23T00:42:37.648 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:36.112527+0000 osd.4 (osd.4) 63 : cluster [DBG] 6.2 scrub starts 2024-02-23T00:42:37.648 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:36.113464+0000 osd.4 (osd.4) 64 : cluster [DBG] 6.2 scrub ok 2024-02-23T00:42:37.648 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:36.177274+0000 mgr.y (mgr.14517) 133 : audit [DBG] from='client.24688 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:42:37.648 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:36.453742+0000 osd.6 (osd.6) 49 : cluster [DBG] 5.2 scrub starts 2024-02-23T00:42:37.648 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:36.454651+0000 osd.6 (osd.6) 50 : cluster [DBG] 5.2 scrub ok 2024-02-23T00:42:37.648 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:37.179157+0000 osd.2 (osd.2) 61 : cluster [DBG] 6.f scrub starts 2024-02-23T00:42:37.648 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:37.180466+0000 osd.2 (osd.2) 62 : cluster [DBG] 6.f scrub ok 2024-02-23T00:42:37.652 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:36.112527+0000 osd.4 (osd.4) 63 : cluster [DBG] 6.2 scrub starts 2024-02-23T00:42:37.652 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:36.113464+0000 osd.4 (osd.4) 64 : cluster [DBG] 6.2 scrub ok 2024-02-23T00:42:37.652 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:36.177274+0000 mgr.y (mgr.14517) 133 : audit [DBG] from='client.24688 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:42:37.652 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:36.453742+0000 osd.6 (osd.6) 49 : cluster [DBG] 5.2 scrub starts 2024-02-23T00:42:37.652 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:36.454651+0000 osd.6 (osd.6) 50 : cluster [DBG] 5.2 scrub ok 2024-02-23T00:42:37.652 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:37.179157+0000 osd.2 (osd.2) 61 : cluster [DBG] 6.f scrub starts 2024-02-23T00:42:37.652 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:37.180466+0000 osd.2 (osd.2) 62 : cluster [DBG] 6.f scrub ok 2024-02-23T00:42:37.674 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2024-02-23T00:42:35.167293+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5796,"num_objects":224,"num_object_clones":0,"num_object_copies":672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":0,"num_read":1293,"num_read_kb":1013,"num_write":620,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":708,"ondisk_log_size":708,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2386640,"kb_used_data":5776,"kb_used_omap":0,"kb_used_meta":2380800,"kb_avail":747574576,"statfs":{"total":767960285184,"available":765516365824,"internally_reserved":0,"allocated":5914624,"data_stored":3135341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2437939200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001543"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.445596+0000","last_change":"2024-02-23T00:41:34.000934+0000","last_active":"2024-02-23T00:42:30.445596+0000","last_peered":"2024-02-23T00:42:30.445596+0000","last_clean":"2024-02-23T00:42:30.445596+0000","last_became_active":"2024-02-23T00:39:21.162083+0000","last_became_peered":"2024-02-23T00:39:21.162083+0000","last_unstale":"2024-02-23T00:42:30.445596+0000","last_undegraded":"2024-02-23T00:42:30.445596+0000","last_fullsized":"2024-02-23T00:42:30.445596+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:34.000844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:34.000844+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.889277+0000","last_change":"2024-02-23T00:41:05.965631+0000","last_active":"2024-02-23T00:42:30.889277+0000","last_peered":"2024-02-23T00:42:30.889277+0000","last_clean":"2024-02-23T00:42:30.889277+0000","last_became_active":"2024-02-23T00:39:18.799723+0000","last_became_peered":"2024-02-23T00:39:18.799723+0000","last_unstale":"2024-02-23T00:42:30.889277+0000","last_undegraded":"2024-02-23T00:42:30.889277+0000","last_fullsized":"2024-02-23T00:42:30.889277+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:05.965531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:05.965531+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":56,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.666552+0000","last_change":"2024-02-23T00:40:58.873093+0000","last_active":"2024-02-23T00:42:30.666552+0000","last_peered":"2024-02-23T00:42:30.666552+0000","last_clean":"2024-02-23T00:42:30.666552+0000","last_became_active":"2024-02-23T00:39:14.782000+0000","last_became_peered":"2024-02-23T00:39:14.782000+0000","last_unstale":"2024-02-23T00:42:30.666552+0000","last_undegraded":"2024-02-23T00:42:30.666552+0000","last_fullsized":"2024-02-23T00:42:30.666552+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_clean_scrub_stamp":"2024-02-23T00:40:58.873013+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"88'50","reported_seq":150,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.445453+0000","last_change":"2024-02-23T00:40:52.840769+0000","last_active":"2024-02-23T00:42:30.445453+0000","last_peered":"2024-02-23T00:42:30.445453+0000","last_clean":"2024-02-23T00:42:30.445453+0000","last_became_active":"2024-02-23T00:39:16.780144+0000","last_became_peered":"2024-02-23T00:39:16.780144+0000","last_unstale":"2024-02-23T00:42:30.445453+0000","last_undegraded":"2024-02-23T00:42:30.445453+0000","last_fullsized":"2024-02-23T00:42:30.445453+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'50","last_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_deep_scrub":"88'50","last_deep_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_clean_scrub_stamp":"2024-02-23T00:40:52.840702+0000","log_size":50,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.889760+0000","last_change":"2024-02-23T00:42:00.004626+0000","last_active":"2024-02-23T00:42:30.889760+0000","last_peered":"2024-02-23T00:42:30.889760+0000","last_clean":"2024-02-23T00:42:30.889760+0000","last_became_active":"2024-02-23T00:39:20.795665+0000","last_became_peered":"2024-02-23T00:39:20.795665+0000","last_unstale":"2024-02-23T00:42:30.889760+0000","last_undegraded":"2024-02-23T00:42:30.889760+0000","last_fullsized":"2024-02-23T00:42:30.889760+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:00.004521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:00.004521+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.443384+0000","last_change":"2024-02-23T00:40:53.812966+0000","last_active":"2024-02-23T00:42:30.443384+0000","last_peered":"2024-02-23T00:42:30.443384+0000","last_clean":"2024-02-23T00:42:30.443384+0000","last_became_active":"2024-02-23T00:39:18.796312+0000","last_became_peered":"2024-02-23T00:39:18.796312+0000","last_unstale":"2024-02-23T00:42:30.443384+0000","last_undegraded":"2024-02-23T00:42:30.443384+0000","last_fullsized":"2024-02-23T00:42:30.443384+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:53.812893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:53.812893+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":56,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.446443+0000","last_change":"2024-02-23T00:41:02.853395+0000","last_active":"2024-02-23T00:42:30.446443+0000","last_peered":"2024-02-23T00:42:30.446443+0000","last_clean":"2024-02-23T00:42:30.446443+0000","last_became_active":"2024-02-23T00:39:14.785362+0000","last_became_peered":"2024-02-23T00:39:14.785362+0000","last_unstale":"2024-02-23T00:42:30.446443+0000","last_undegraded":"2024-02-23T00:42:30.446443+0000","last_fullsized":"2024-02-23T00:42:30.446443+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:02.853328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:02.853328+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"68'15","reported_seq":89,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.889657+0000","last_change":"2024-02-23T00:41:33.064109+0000","last_active":"2024-02-23T00:42:30.889657+0000","last_peered":"2024-02-23T00:42:30.889657+0000","last_clean":"2024-02-23T00:42:30.889657+0000","last_became_active":"2024-02-23T00:39:16.784606+0000","last_became_peered":"2024-02-23T00:39:16.784606+0000","last_unstale":"2024-02-23T00:42:30.889657+0000","last_undegraded":"2024-02-23T00:42:30.889657+0000","last_fullsized":"2024-02-23T00:42:30.889657+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:41:33.063992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:33.063992+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.828710+0000","last_change":"2024-02-23T00:42:08.695352+0000","last_active":"2024-02-23T00:42:30.828710+0000","last_peered":"2024-02-23T00:42:30.828710+0000","last_clean":"2024-02-23T00:42:30.828710+0000","last_became_active":"2024-02-23T00:39:20.793590+0000","last_became_peered":"2024-02-23T00:39:20.793590+0000","last_unstale":"2024-02-23T00:42:30.828710+0000","last_undegraded":"2024-02-23T00:42:30.828710+0000","last_fullsized":"2024-02-23T00:42:30.828710+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:08.695270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:55.854964+0000","last_clean_scrub_stamp":"2024-02-23T00:42:08.695270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":57,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.889113+0000","last_change":"2024-02-23T00:42:00.998947+0000","last_active":"2024-02-23T00:42:30.889113+0000","last_peered":"2024-02-23T00:42:30.889113+0000","last_clean":"2024-02-23T00:42:30.889113+0000","last_became_active":"2024-02-23T00:39:18.796952+0000","last_became_peered":"2024-02-23T00:39:18.796952+0000","last_unstale":"2024-02-23T00:42:30.889113+0000","last_undegraded":"2024-02-23T00:42:30.889113+0000","last_fullsized":"2024-02-23T00:42:30.889113+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:00.998885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:00.998885+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":59,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.891245+0000","last_change":"2024-02-23T00:41:40.926914+0000","last_active":"2024-02-23T00:42:30.891245+0000","last_peered":"2024-02-23T00:42:30.891245+0000","last_clean":"2024-02-23T00:42:30.891245+0000","last_became_active":"2024-02-23T00:39:14.783469+0000","last_became_peered":"2024-02-23T00:39:14.783469+0000","last_unstale":"2024-02-23T00:42:30.891245+0000","last_undegraded":"2024-02-23T00:42:30.891245+0000","last_fullsized":"2024-02-23T00:42:30.891245+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:40.926851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:40.926851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"82'15","reported_seq":87,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.889678+0000","last_change":"2024-02-23T00:41:50.975772+0000","last_active":"2024-02-23T00:42:30.889678+0000","last_peered":"2024-02-23T00:42:30.889678+0000","last_clean":"2024-02-23T00:42:30.889678+0000","last_became_active":"2024-02-23T00:39:16.782762+0000","last_became_peered":"2024-02-23T00:39:16.782762+0000","last_unstale":"2024-02-23T00:42:30.889678+0000","last_undegraded":"2024-02-23T00:42:30.889678+0000","last_fullsized":"2024-02-23T00:42:30.889678+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'15","last_scrub_stamp":"2024-02-23T00:41:50.975683+0000","last_deep_scrub":"82'15","last_deep_scrub_stamp":"2024-02-23T00:41:50.975683+0000","last_clean_scrub_stamp":"2024-02-23T00:41:50.975683+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.665318+0000","last_change":"2024-02-23T00:41:08.906994+0000","last_active":"2024-02-23T00:42:30.665318+0000","last_peered":"2024-02-23T00:42:30.665318+0000","last_clean":"2024-02-23T00:42:30.665318+0000","last_became_active":"2024-02-23T00:39:20.803344+0000","last_became_peered":"2024-02-23T00:39:20.803344+0000","last_unstale":"2024-02-23T00:42:30.665318+0000","last_undegraded":"2024-02-23T00:42:30.665318+0000","last_fullsized":"2024-02-23T00:42:30.665318+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:08.906928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:08.906928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.828481+0000","last_change":"2024-02-23T00:41:07.777516+0000","last_active":"2024-02-23T00:42:30.828481+0000","last_peered":"2024-02-23T00:42:30.828481+0000","last_clean":"2024-02-23T00:42:30.828481+0000","last_became_active":"2024-02-23T00:39:18.796431+0000","last_became_peered":"2024-02-23T00:39:18.796431+0000","last_unstale":"2024-02-23T00:42:30.828481+0000","last_undegraded":"2024-02-23T00:42:30.828481+0000","last_fullsized":"2024-02-23T00:42:30.828481+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:07.777435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:07.777435+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.891133+0000","last_change":"2024-02-23T00:40:35.620371+0000","last_active":"2024-02-23T00:42:30.891133+0000","last_peered":"2024-02-23T00:42:30.891133+0000","last_clean":"2024-02-23T00:42:30.891133+0000","last_became_active":"2024-02-23T00:39:14.787148+0000","last_became_peered":"2024-02-23T00:39:14.787148+0000","last_unstale":"2024-02-23T00:42:30.891133+0000","last_undegraded":"2024-02-23T00:42:30.891133+0000","last_fullsized":"2024-02-23T00:42:30.891133+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:35.620292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:35.620292+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"95'27","reported_seq":110,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.890151+0000","last_change":"2024-02-23T00:41:55.920001+0000","last_active":"2024-02-23T00:42:30.890151+0000","last_peered":"2024-02-23T00:42:30.890151+0000","last_clean":"2024-02-23T00:42:30.890151+0000","last_became_active":"2024-02-23T00:39:16.786791+0000","last_became_peered":"2024-02-23T00:39:16.786791+0000","last_unstale":"2024-02-23T00:42:30.890151+0000","last_undegraded":"2024-02-23T00:42:30.890151+0000","last_fullsized":"2024-02-23T00:42:30.890151+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'27","last_scrub_stamp":"2024-02-23T00:41:55.919895+0000","last_deep_scrub":"95'27","last_deep_scrub_stamp":"2024-02-23T00:41:55.919895+0000","last_clean_scrub_stamp":"2024-02-23T00:41:55.919895+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.446632+0000","last_change":"2024-02-23T00:41:29.992520+0000","last_active":"2024-02-23T00:42:30.446632+0000","last_peered":"2024-02-23T00:42:30.446632+0000","last_clean":"2024-02-23T00:42:30.446632+0000","last_became_active":"2024-02-23T00:39:21.162053+0000","last_became_peered":"2024-02-23T00:39:21.162053+0000","last_unstale":"2024-02-23T00:42:30.446632+0000","last_undegraded":"2024-02-23T00:42:30.446632+0000","last_fullsized":"2024-02-23T00:42:30.446632+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:29.992417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:29.992417+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":56,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.889082+0000","last_change":"2024-02-23T00:42:07.939130+0000","last_active":"2024-02-23T00:42:30.889082+0000","last_peered":"2024-02-23T00:42:30.889082+0000","last_clean":"2024-02-23T00:42:30.889082+0000","last_became_active":"2024-02-23T00:39:18.794903+0000","last_became_peered":"2024-02-23T00:39:18.794903+0000","last_unstale":"2024-02-23T00:42:30.889082+0000","last_undegraded":"2024-02-23T00:42:30.889082+0000","last_fullsized":"2024-02-23T00:42:30.889082+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:07.939058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_clean_scrub_stamp":"2024-02-23T00:42:07.939058+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":61,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.445744+0000","last_change":"2024-02-23T00:41:44.937852+0000","last_active":"2024-02-23T00:42:30.445744+0000","last_peered":"2024-02-23T00:42:30.445744+0000","last_clean":"2024-02-23T00:42:30.445744+0000","last_became_active":"2024-02-23T00:39:14.782163+0000","last_became_peered":"2024-02-23T00:39:14.782163+0000","last_unstale":"2024-02-23T00:42:30.445744+0000","last_undegraded":"2024-02-23T00:42:30.445744+0000","last_fullsized":"2024-02-23T00:42:30.445744+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:44.937776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:44.937776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"68'18","reported_seq":97,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.445955+0000","last_change":"2024-02-23T00:41:54.082061+0000","last_active":"2024-02-23T00:42:30.445955+0000","last_peered":"2024-02-23T00:42:30.445955+0000","last_clean":"2024-02-23T00:42:30.445955+0000","last_became_active":"2024-02-23T00:39:16.782803+0000","last_became_peered":"2024-02-23T00:39:16.782803+0000","last_unstale":"2024-02-23T00:42:30.445955+0000","last_undegraded":"2024-02-23T00:42:30.445955+0000","last_fullsized":"2024-02-23T00:42:30.445955+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'18","last_scrub_stamp":"2024-02-23T00:41:54.081977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:54.081977+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.889972+0000","last_change":"2024-02-23T00:41:16.048306+0000","last_active":"2024-02-23T00:42:30.889972+0000","last_peered":"2024-02-23T00:42:30.889972+0000","last_clean":"2024-02-23T00:42:30.889972+0000","last_became_active":"2024-02-23T00:39:21.159750+0000","last_became_peered":"2024-02-23T00:39:21.159750+0000","last_unstale":"2024-02-23T00:42:30.889972+0000","last_undegraded":"2024-02-23T00:42:30.889972+0000","last_fullsized":"2024-02-23T00:42:30.889972+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:16.048207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:16.048207+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":56,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.443991+0000","last_change":"2024-02-23T00:41:52.641797+0000","last_active":"2024-02-23T00:42:30.443991+0000","last_peered":"2024-02-23T00:42:30.443991+0000","last_clean":"2024-02-23T00:42:30.443991+0000","last_became_active":"2024-02-23T00:39:18.801039+0000","last_became_peered":"2024-02-23T00:39:18.801039+0000","last_unstale":"2024-02-23T00:42:30.443991+0000","last_undegraded":"2024-02-23T00:42:30.443991+0000","last_fullsized":"2024-02-23T00:42:30.443991+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:52.641689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.641689+0000","last_clean_scrub_stamp":"2024-02-23T00:41:52.641689+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":60,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.442437+0000","last_change":"2024-02-23T00:41:57.138591+0000","last_active":"2024-02-23T00:42:30.442437+0000","last_peered":"2024-02-23T00:42:30.442437+0000","last_clean":"2024-02-23T00:42:30.442437+0000","last_became_active":"2024-02-23T00:39:14.782494+0000","last_became_peered":"2024-02-23T00:39:14.782494+0000","last_unstale":"2024-02-23T00:42:30.442437+0000","last_undegraded":"2024-02-23T00:42:30.442437+0000","last_fullsized":"2024-02-23T00:42:30.442437+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:57.138515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:57.138515+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"67'15","reported_seq":83,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.890141+0000","last_change":"2024-02-23T00:41:41.121005+0000","last_active":"2024-02-23T00:42:30.890141+0000","last_peered":"2024-02-23T00:42:30.890141+0000","last_clean":"2024-02-23T00:42:30.890141+0000","last_became_active":"2024-02-23T00:39:16.787138+0000","last_became_peered":"2024-02-23T00:39:16.787138+0000","last_unstale":"2024-02-23T00:42:30.890141+0000","last_undegraded":"2024-02-23T00:42:30.890141+0000","last_fullsized":"2024-02-23T00:42:30.890141+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:41:41.120939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:41.120939+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.443732+0000","last_change":"2024-02-23T00:40:49.777703+0000","last_active":"2024-02-23T00:42:30.443732+0000","last_peered":"2024-02-23T00:42:30.443732+0000","last_clean":"2024-02-23T00:42:30.443732+0000","last_became_active":"2024-02-23T00:39:20.795524+0000","last_became_peered":"2024-02-23T00:39:20.795524+0000","last_unstale":"2024-02-23T00:42:30.443732+0000","last_undegraded":"2024-02-23T00:42:30.443732+0000","last_fullsized":"2024-02-23T00:42:30.443732+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:49.777636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:49.777636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":57,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.667185+0000","last_change":"2024-02-23T00:41:53.189947+0000","last_active":"2024-02-23T00:42:30.667185+0000","last_peered":"2024-02-23T00:42:30.667185+0000","last_clean":"2024-02-23T00:42:30.667185+0000","last_became_active":"2024-02-23T00:39:18.790602+0000","last_became_peered":"2024-02-23T00:39:18.790602+0000","last_unstale":"2024-02-23T00:42:30.667185+0000","last_undegraded":"2024-02-23T00:42:30.667185+0000","last_fullsized":"2024-02-23T00:42:30.667185+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:53.189881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:53.189881+0000","last_clean_scrub_stamp":"2024-02-23T00:41:53.189881+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":61,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.445260+0000","last_change":"2024-02-23T00:41:57.053372+0000","last_active":"2024-02-23T00:42:30.445260+0000","last_peered":"2024-02-23T00:42:30.445260+0000","last_clean":"2024-02-23T00:42:30.445260+0000","last_became_active":"2024-02-23T00:39:14.781560+0000","last_became_peered":"2024-02-23T00:39:14.781560+0000","last_unstale":"2024-02-23T00:42:30.445260+0000","last_undegraded":"2024-02-23T00:42:30.445260+0000","last_fullsized":"2024-02-23T00:42:30.445260+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:57.053295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:57.053295+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"93'23","reported_seq":98,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.442781+0000","last_change":"2024-02-23T00:41:35.954600+0000","last_active":"2024-02-23T00:42:30.442781+0000","last_peered":"2024-02-23T00:42:30.442781+0000","last_clean":"2024-02-23T00:42:30.442781+0000","last_became_active":"2024-02-23T00:39:16.782660+0000","last_became_peered":"2024-02-23T00:39:16.782660+0000","last_unstale":"2024-02-23T00:42:30.442781+0000","last_undegraded":"2024-02-23T00:42:30.442781+0000","last_fullsized":"2024-02-23T00:42:30.442781+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'23","last_scrub_stamp":"2024-02-23T00:41:35.954516+0000","last_deep_scrub":"93'23","last_deep_scrub_stamp":"2024-02-23T00:41:35.954516+0000","last_clean_scrub_stamp":"2024-02-23T00:41:35.954516+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"67'1","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.891361+0000","last_change":"2024-02-23T00:41:46.926637+0000","last_active":"2024-02-23T00:42:30.891361+0000","last_peered":"2024-02-23T00:42:30.891361+0000","last_clean":"2024-02-23T00:42:30.891361+0000","last_became_active":"2024-02-23T00:39:20.795085+0000","last_became_peered":"2024-02-23T00:39:20.795085+0000","last_unstale":"2024-02-23T00:42:30.891361+0000","last_undegraded":"2024-02-23T00:42:30.891361+0000","last_fullsized":"2024-02-23T00:42:30.891361+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:41:46.926552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:46.926552+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"93'9","reported_seq":107,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.442838+0000","last_change":"2024-02-23T00:41:38.881071+0000","last_active":"2024-02-23T00:42:30.442838+0000","last_peered":"2024-02-23T00:42:30.442838+0000","last_clean":"2024-02-23T00:42:30.442838+0000","last_became_active":"2024-02-23T00:39:18.796680+0000","last_became_peered":"2024-02-23T00:39:18.796680+0000","last_unstale":"2024-02-23T00:42:30.442838+0000","last_undegraded":"2024-02-23T00:42:30.442838+0000","last_fullsized":"2024-02-23T00:42:30.442838+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-02-23T00:41:38.880989+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-02-23T00:41:38.880989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:38.880989+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":68,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.828784+0000","last_change":"2024-02-23T00:41:32.702659+0000","last_active":"2024-02-23T00:42:30.828784+0000","last_peered":"2024-02-23T00:42:30.828784+0000","last_clean":"2024-02-23T00:42:30.828784+0000","last_became_active":"2024-02-23T00:39:14.785363+0000","last_became_peered":"2024-02-23T00:39:14.785363+0000","last_unstale":"2024-02-23T00:42:30.828784+0000","last_undegraded":"2024-02-23T00:42:30.828784+0000","last_fullsized":"2024-02-23T00:42:30.828784+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"80'2","last_scrub_stamp":"2024-02-23T00:41:32.702574+0000","last_deep_scrub":"80'2","last_deep_scrub_stamp":"2024-02-23T00:40:15.942725+0000","last_clean_scrub_stamp":"2024-02-23T00:41:32.702574+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"68'16","reported_seq":84,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.667097+0000","last_change":"2024-02-23T00:41:33.901108+0000","last_active":"2024-02-23T00:42:30.667097+0000","last_peered":"2024-02-23T00:42:30.667097+0000","last_clean":"2024-02-23T00:42:30.667097+0000","last_became_active":"2024-02-23T00:39:16.858945+0000","last_became_peered":"2024-02-23T00:39:16.858945+0000","last_unstale":"2024-02-23T00:42:30.667097+0000","last_undegraded":"2024-02-23T00:42:30.667097+0000","last_fullsized":"2024-02-23T00:42:30.667097+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:41:33.901023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:33.901023+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.445915+0000","last_change":"2024-02-23T00:41:40.941453+0000","last_active":"2024-02-23T00:42:30.445915+0000","last_peered":"2024-02-23T00:42:30.445915+0000","last_clean":"2024-02-23T00:42:30.445915+0000","last_became_active":"2024-02-23T00:39:21.160376+0000","last_became_peered":"2024-02-23T00:39:21.160376+0000","last_unstale":"2024-02-23T00:42:30.445915+0000","last_undegraded":"2024-02-23T00:42:30.445915+0000","last_fullsized":"2024-02-23T00:42:30.445915+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:40.941380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:40.941380+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.889317+0000","last_change":"2024-02-23T00:42:02.965536+0000","last_active":"2024-02-23T00:42:30.889317+0000","last_peered":"2024-02-23T00:42:30.889317+0000","last_clean":"2024-02-23T00:42:30.889317+0000","last_became_active":"2024-02-23T00:39:18.791457+0000","last_became_peered":"2024-02-23T00:39:18.791457+0000","last_unstale":"2024-02-23T00:42:30.889317+0000","last_undegraded":"2024-02-23T00:42:30.889317+0000","last_fullsized":"2024-02-23T00:42:30.889317+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:02.965455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:02.965455+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":60,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.442911+0000","last_change":"2024-02-23T00:41:47.034138+0000","last_active":"2024-02-23T00:42:30.442911+0000","last_peered":"2024-02-23T00:42:30.442911+0000","last_clean":"2024-02-23T00:42:30.442911+0000","last_became_active":"2024-02-23T00:39:14.784092+0000","last_became_peered":"2024-02-23T00:39:14.784092+0000","last_unstale":"2024-02-23T00:42:30.442911+0000","last_undegraded":"2024-02-23T00:42:30.442911+0000","last_fullsized":"2024-02-23T00:42:30.442911+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:47.034064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:47.034064+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"68'17","reported_seq":93,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.445410+0000","last_change":"2024-02-23T00:41:45.979896+0000","last_active":"2024-02-23T00:42:30.445410+0000","last_peered":"2024-02-23T00:42:30.445410+0000","last_clean":"2024-02-23T00:42:30.445410+0000","last_became_active":"2024-02-23T00:39:16.857232+0000","last_became_peered":"2024-02-23T00:39:16.857232+0000","last_unstale":"2024-02-23T00:42:30.445410+0000","last_undegraded":"2024-02-23T00:42:30.445410+0000","last_fullsized":"2024-02-23T00:42:30.445410+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:41:45.979819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:45.979819+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.889326+0000","last_change":"2024-02-23T00:40:54.739607+0000","last_active":"2024-02-23T00:42:30.889326+0000","last_peered":"2024-02-23T00:42:30.889326+0000","last_clean":"2024-02-23T00:42:30.889326+0000","last_became_active":"2024-02-23T00:39:20.795351+0000","last_became_peered":"2024-02-23T00:39:20.795351+0000","last_unstale":"2024-02-23T00:42:30.889326+0000","last_undegraded":"2024-02-23T00:42:30.889326+0000","last_fullsized":"2024-02-23T00:42:30.889326+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:54.739534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:54.739534+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":56,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.442113+0000","last_change":"2024-02-23T00:41:52.082354+0000","last_active":"2024-02-23T00:42:30.442113+0000","last_peered":"2024-02-23T00:42:30.442113+0000","last_clean":"2024-02-23T00:42:30.442113+0000","last_became_active":"2024-02-23T00:39:18.797287+0000","last_became_peered":"2024-02-23T00:39:18.797287+0000","last_unstale":"2024-02-23T00:42:30.442113+0000","last_undegraded":"2024-02-23T00:42:30.442113+0000","last_fullsized":"2024-02-23T00:42:30.442113+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:52.082271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:52.082271+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":61,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.829415+0000","last_change":"2024-02-23T00:41:41.650513+0000","last_active":"2024-02-23T00:42:30.829415+0000","last_peered":"2024-02-23T00:42:30.829415+0000","last_clean":"2024-02-23T00:42:30.829415+0000","last_became_active":"2024-02-23T00:39:14.783325+0000","last_became_peered":"2024-02-23T00:39:14.783325+0000","last_unstale":"2024-02-23T00:42:30.829415+0000","last_undegraded":"2024-02-23T00:42:30.829415+0000","last_fullsized":"2024-02-23T00:42:30.829415+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:41.650434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:41.650434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"67'4","reported_seq":60,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.445014+0000","last_change":"2024-02-23T00:41:20.947463+0000","last_active":"2024-02-23T00:42:30.445014+0000","last_peered":"2024-02-23T00:42:30.445014+0000","last_clean":"2024-02-23T00:42:30.445014+0000","last_became_active":"2024-02-23T00:39:16.785830+0000","last_became_peered":"2024-02-23T00:39:16.785830+0000","last_unstale":"2024-02-23T00:42:30.445014+0000","last_undegraded":"2024-02-23T00:42:30.445014+0000","last_fullsized":"2024-02-23T00:42:30.445014+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'4","last_scrub_stamp":"2024-02-23T00:41:20.947377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:20.947377+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.445122+0000","last_change":"2024-02-23T00:42:08.062268+0000","last_active":"2024-02-23T00:42:30.445122+0000","last_peered":"2024-02-23T00:42:30.445122+0000","last_clean":"2024-02-23T00:42:30.445122+0000","last_became_active":"2024-02-23T00:39:20.799053+0000","last_became_peered":"2024-02-23T00:39:20.799053+0000","last_unstale":"2024-02-23T00:42:30.445122+0000","last_undegraded":"2024-02-23T00:42:30.445122+0000","last_fullsized":"2024-02-23T00:42:30.445122+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:08.062156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:08.062156+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.443031+0000","last_change":"2024-02-23T00:40:59.669880+0000","last_active":"2024-02-23T00:42:30.443031+0000","last_peered":"2024-02-23T00:42:30.443031+0000","last_clean":"2024-02-23T00:42:30.443031+0000","last_became_active":"2024-02-23T00:39:18.800382+0000","last_became_peered":"2024-02-23T00:39:18.800382+0000","last_unstale":"2024-02-23T00:42:30.443031+0000","last_undegraded":"2024-02-23T00:42:30.443031+0000","last_fullsized":"2024-02-23T00:42:30.443031+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:59.669810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:59.669810+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"60'1","reported_seq":67,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.443881+0000","last_change":"2024-02-23T00:41:57.687046+0000","last_active":"2024-02-23T00:42:30.443881+0000","last_peered":"2024-02-23T00:42:30.443881+0000","last_clean":"2024-02-23T00:42:30.443881+0000","last_became_active":"2024-02-23T00:39:14.775804+0000","last_became_peered":"2024-02-23T00:39:14.775804+0000","last_unstale":"2024-02-23T00:42:30.443881+0000","last_undegraded":"2024-02-23T00:42:30.443881+0000","last_fullsized":"2024-02-23T00:42:30.443881+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'1","last_scrub_stamp":"2024-02-23T00:41:57.686958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:57.686958+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"67'11","reported_seq":72,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.890027+0000","last_change":"2024-02-23T00:40:55.928416+0000","last_active":"2024-02-23T00:42:30.890027+0000","last_peered":"2024-02-23T00:42:30.890027+0000","last_clean":"2024-02-23T00:42:30.890027+0000","last_became_active":"2024-02-23T00:39:16.783896+0000","last_became_peered":"2024-02-23T00:39:16.783896+0000","last_unstale":"2024-02-23T00:42:30.890027+0000","last_undegraded":"2024-02-23T00:42:30.890027+0000","last_fullsized":"2024-02-23T00:42:30.890027+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'11","last_scrub_stamp":"2024-02-23T00:40:55.928349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:55.928349+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.665062+0000","last_change":"2024-02-23T00:41:10.892383+0000","last_active":"2024-02-23T00:42:30.665062+0000","last_peered":"2024-02-23T00:42:30.665062+0000","last_clean":"2024-02-23T00:42:30.665062+0000","last_became_active":"2024-02-23T00:39:20.802648+0000","last_became_peered":"2024-02-23T00:39:20.802648+0000","last_unstale":"2024-02-23T00:42:30.665062+0000","last_undegraded":"2024-02-23T00:42:30.665062+0000","last_fullsized":"2024-02-23T00:42:30.665062+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:10.892259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:10.892259+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.445817+0000","last_change":"2024-02-23T00:41:17.931479+0000","last_active":"2024-02-23T00:42:30.445817+0000","last_peered":"2024-02-23T00:42:30.445817+0000","last_clean":"2024-02-23T00:42:30.445817+0000","last_became_active":"2024-02-23T00:39:18.799052+0000","last_became_peered":"2024-02-23T00:39:18.799052+0000","last_unstale":"2024-02-23T00:42:30.445817+0000","last_undegraded":"2024-02-23T00:42:30.445817+0000","last_fullsized":"2024-02-23T00:42:30.445817+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_clean_scrub_stamp":"2024-02-23T00:41:17.931413+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":60,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.442516+0000","last_change":"2024-02-23T00:41:33.993147+0000","last_active":"2024-02-23T00:42:30.442516+0000","last_peered":"2024-02-23T00:42:30.442516+0000","last_clean":"2024-02-23T00:42:30.442516+0000","last_became_active":"2024-02-23T00:39:14.778455+0000","last_became_peered":"2024-02-23T00:39:14.778455+0000","last_unstale":"2024-02-23T00:42:30.442516+0000","last_undegraded":"2024-02-23T00:42:30.442516+0000","last_fullsized":"2024-02-23T00:42:30.442516+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:33.993070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:33.993070+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"67'9","reported_seq":69,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.442880+0000","last_change":"2024-02-23T00:40:52.783268+0000","last_active":"2024-02-23T00:42:30.442880+0000","last_peered":"2024-02-23T00:42:30.442880+0000","last_clean":"2024-02-23T00:42:30.442880+0000","last_became_active":"2024-02-23T00:39:16.786936+0000","last_became_peered":"2024-02-23T00:39:16.786936+0000","last_unstale":"2024-02-23T00:42:30.442880+0000","last_undegraded":"2024-02-23T00:42:30.442880+0000","last_fullsized":"2024-02-23T00:42:30.442880+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'9","last_scrub_stamp":"2024-02-23T00:40:52.783146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:52.783146+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.890347+0000","last_change":"2024-02-23T00:40:54.938288+0000","last_active":"2024-02-23T00:42:30.890347+0000","last_peered":"2024-02-23T00:42:30.890347+0000","last_clean":"2024-02-23T00:42:30.890347+0000","last_became_active":"2024-02-23T00:39:20.795210+0000","last_became_peered":"2024-02-23T00:39:20.795210+0000","last_unstale":"2024-02-23T00:42:30.890347+0000","last_undegraded":"2024-02-23T00:42:30.890347+0000","last_fullsized":"2024-02-23T00:42:30.890347+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:54.938209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:54.938209+0000","last_clean_scrub_stamp":"2024-02-23T00:40:54.938209+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"67'8","reported_seq":99,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.446287+0000","last_change":"2024-02-23T00:41:25.942914+0000","last_active":"2024-02-23T00:42:30.446287+0000","last_peered":"2024-02-23T00:42:30.446287+0000","last_clean":"2024-02-23T00:42:30.446287+0000","last_became_active":"2024-02-23T00:39:18.787313+0000","last_became_peered":"2024-02-23T00:39:18.787313+0000","last_unstale":"2024-02-23T00:42:30.446287+0000","last_undegraded":"2024-02-23T00:42:30.446287+0000","last_fullsized":"2024-02-23T00:42:30.446287+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:41:25.942822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:25.942822+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":61,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.665798+0000","last_change":"2024-02-23T00:41:52.151201+0000","last_active":"2024-02-23T00:42:30.665798+0000","last_peered":"2024-02-23T00:42:30.665798+0000","last_clean":"2024-02-23T00:42:30.665798+0000","last_became_active":"2024-02-23T00:39:14.773772+0000","last_became_peered":"2024-02-23T00:39:14.773772+0000","last_unstale":"2024-02-23T00:42:30.665798+0000","last_undegraded":"2024-02-23T00:42:30.665798+0000","last_fullsized":"2024-02-23T00:42:30.665798+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:52.151119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:52.151119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"68'15","reported_seq":85,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.829126+0000","last_change":"2024-02-23T00:40:42.867136+0000","last_active":"2024-02-23T00:42:30.829126+0000","last_peered":"2024-02-23T00:42:30.829126+0000","last_clean":"2024-02-23T00:42:30.829126+0000","last_became_active":"2024-02-23T00:39:16.858953+0000","last_became_peered":"2024-02-23T00:39:16.858953+0000","last_unstale":"2024-02-23T00:42:30.829126+0000","last_undegraded":"2024-02-23T00:42:30.829126+0000","last_fullsized":"2024-02-23T00:42:30.829126+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:40:42.867043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:42.867043+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.666699+0000","last_change":"2024-02-23T00:41:22.855592+0000","last_active":"2024-02-23T00:42:30.666699+0000","last_peered":"2024-02-23T00:42:30.666699+0000","last_clean":"2024-02-23T00:42:30.666699+0000","last_became_active":"2024-02-23T00:39:20.800422+0000","last_became_peered":"2024-02-23T00:39:20.800422+0000","last_unstale":"2024-02-23T00:42:30.666699+0000","last_undegraded":"2024-02-23T00:42:30.666699+0000","last_fullsized":"2024-02-23T00:42:30.666699+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:22.855516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:22.855516+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"67'8","reported_seq":99,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.828432+0000","last_change":"2024-02-23T00:41:08.785056+0000","last_active":"2024-02-23T00:42:30.828432+0000","last_peered":"2024-02-23T00:42:30.828432+0000","last_clean":"2024-02-23T00:42:30.828432+0000","last_became_active":"2024-02-23T00:39:18.803566+0000","last_became_peered":"2024-02-23T00:39:18.803566+0000","last_unstale":"2024-02-23T00:42:30.828432+0000","last_undegraded":"2024-02-23T00:42:30.828432+0000","last_fullsized":"2024-02-23T00:42:30.828432+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:41:08.784965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:08.784965+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":61,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.829037+0000","last_change":"2024-02-23T00:42:02.694103+0000","last_active":"2024-02-23T00:42:30.829037+0000","last_peered":"2024-02-23T00:42:30.829037+0000","last_clean":"2024-02-23T00:42:30.829037+0000","last_became_active":"2024-02-23T00:39:14.785590+0000","last_became_peered":"2024-02-23T00:39:14.785590+0000","last_unstale":"2024-02-23T00:42:30.829037+0000","last_undegraded":"2024-02-23T00:42:30.829037+0000","last_fullsized":"2024-02-23T00:42:30.829037+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:02.694022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_clean_scrub_stamp":"2024-02-23T00:42:02.694022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"68'22","reported_seq":99,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.446116+0000","last_change":"2024-02-23T00:41:38.950622+0000","last_active":"2024-02-23T00:42:30.446116+0000","last_peered":"2024-02-23T00:42:30.446116+0000","last_clean":"2024-02-23T00:42:30.446116+0000","last_became_active":"2024-02-23T00:39:16.857437+0000","last_became_peered":"2024-02-23T00:39:16.857437+0000","last_unstale":"2024-02-23T00:42:30.446116+0000","last_undegraded":"2024-02-23T00:42:30.446116+0000","last_fullsized":"2024-02-23T00:42:30.446116+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'22","last_scrub_stamp":"2024-02-23T00:41:38.950535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:38.950535+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":39,"num_read_kb":32,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":53,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.891243+0000","last_change":"2024-02-23T00:41:45.928004+0000","last_active":"2024-02-23T00:42:30.891243+0000","last_peered":"2024-02-23T00:42:30.891243+0000","last_clean":"2024-02-23T00:42:30.891243+0000","last_became_active":"2024-02-23T00:39:21.162036+0000","last_became_peered":"2024-02-23T00:39:21.162036+0000","last_unstale":"2024-02-23T00:42:30.891243+0000","last_undegraded":"2024-02-23T00:42:30.891243+0000","last_fullsized":"2024-02-23T00:42:30.891243+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:45.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:45.927925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":57,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.828438+0000","last_change":"2024-02-23T00:41:35.655313+0000","last_active":"2024-02-23T00:42:30.828438+0000","last_peered":"2024-02-23T00:42:30.828438+0000","last_clean":"2024-02-23T00:42:30.828438+0000","last_became_active":"2024-02-23T00:39:18.800297+0000","last_became_peered":"2024-02-23T00:39:18.800297+0000","last_unstale":"2024-02-23T00:42:30.828438+0000","last_undegraded":"2024-02-23T00:42:30.828438+0000","last_fullsized":"2024-02-23T00:42:30.828438+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:35.655209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:35.655209+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.442129+0000","last_change":"2024-02-23T00:40:53.113312+0000","last_active":"2024-02-23T00:42:30.442129+0000","last_peered":"2024-02-23T00:42:30.442129+0000","last_clean":"2024-02-23T00:42:30.442129+0000","last_became_active":"2024-02-23T00:39:14.783164+0000","last_became_peered":"2024-02-23T00:39:14.783164+0000","last_unstale":"2024-02-23T00:42:30.442129+0000","last_undegraded":"2024-02-23T00:42:30.442129+0000","last_fullsized":"2024-02-23T00:42:30.442129+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:53.113232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:53.113232+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"68'12","reported_seq":78,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.446590+0000","last_change":"2024-02-23T00:41:23.904595+0000","last_active":"2024-02-23T00:42:30.446590+0000","last_peered":"2024-02-23T00:42:30.446590+0000","last_clean":"2024-02-23T00:42:30.446590+0000","last_became_active":"2024-02-23T00:39:16.858449+0000","last_became_peered":"2024-02-23T00:39:16.858449+0000","last_unstale":"2024-02-23T00:42:30.446590+0000","last_undegraded":"2024-02-23T00:42:30.446590+0000","last_fullsized":"2024-02-23T00:42:30.446590+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'12","last_scrub_stamp":"2024-02-23T00:41:23.904511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:23.904511+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.889894+0000","last_change":"2024-02-23T00:40:40.360889+0000","last_active":"2024-02-23T00:42:30.889894+0000","last_peered":"2024-02-23T00:42:30.889894+0000","last_clean":"2024-02-23T00:42:30.889894+0000","last_became_active":"2024-02-23T00:39:20.796041+0000","last_became_peered":"2024-02-23T00:39:20.796041+0000","last_unstale":"2024-02-23T00:42:30.889894+0000","last_undegraded":"2024-02-23T00:42:30.889894+0000","last_fullsized":"2024-02-23T00:42:30.889894+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:40.360797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:40.360797+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":57,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.446669+0000","last_change":"2024-02-23T00:41:46.998945+0000","last_active":"2024-02-23T00:42:30.446669+0000","last_peered":"2024-02-23T00:42:30.446669+0000","last_clean":"2024-02-23T00:42:30.446669+0000","last_became_active":"2024-02-23T00:39:18.798624+0000","last_became_peered":"2024-02-23T00:39:18.798624+0000","last_unstale":"2024-02-23T00:42:30.446669+0000","last_undegraded":"2024-02-23T00:42:30.446669+0000","last_fullsized":"2024-02-23T00:42:30.446669+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:46.998875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:46.998875+0000","last_clean_scrub_stamp":"2024-02-23T00:41:46.998875+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":60,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.889134+0000","last_change":"2024-02-23T00:41:37.165479+0000","last_active":"2024-02-23T00:42:30.889134+0000","last_peered":"2024-02-23T00:42:30.889134+0000","last_clean":"2024-02-23T00:42:30.889134+0000","last_became_active":"2024-02-23T00:39:14.774244+0000","last_became_peered":"2024-02-23T00:39:14.774244+0000","last_unstale":"2024-02-23T00:42:30.889134+0000","last_undegraded":"2024-02-23T00:42:30.889134+0000","last_fullsized":"2024-02-23T00:42:30.889134+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:37.165357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:37.165357+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"68'21","reported_seq":100,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.666884+0000","last_change":"2024-02-23T00:41:48.151380+0000","last_active":"2024-02-23T00:42:30.666884+0000","last_peered":"2024-02-23T00:42:30.666884+0000","last_clean":"2024-02-23T00:42:30.666884+0000","last_became_active":"2024-02-23T00:39:16.858760+0000","last_became_peered":"2024-02-23T00:39:16.858760+0000","last_unstale":"2024-02-23T00:42:30.666884+0000","last_undegraded":"2024-02-23T00:42:30.666884+0000","last_fullsized":"2024-02-23T00:42:30.666884+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'21","last_scrub_stamp":"2024-02-23T00:41:48.151312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:48.151312+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.445820+0000","last_change":"2024-02-23T00:41:48.031707+0000","last_active":"2024-02-23T00:42:30.445820+0000","last_peered":"2024-02-23T00:42:30.445820+0000","last_clean":"2024-02-23T00:42:30.445820+0000","last_became_active":"2024-02-23T00:39:20.793710+0000","last_became_peered":"2024-02-23T00:39:20.793710+0000","last_unstale":"2024-02-23T00:42:30.445820+0000","last_undegraded":"2024-02-23T00:42:30.445820+0000","last_fullsized":"2024-02-23T00:42:30.445820+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:48.031633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:48.031633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"95'4","reported_seq":75,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.890004+0000","last_change":"2024-02-23T00:41:43.874460+0000","last_active":"2024-02-23T00:42:30.890004+0000","last_peered":"2024-02-23T00:42:30.890004+0000","last_clean":"2024-02-23T00:42:30.890004+0000","last_became_active":"2024-02-23T00:39:14.779205+0000","last_became_peered":"2024-02-23T00:39:14.779205+0000","last_unstale":"2024-02-23T00:42:30.890004+0000","last_undegraded":"2024-02-23T00:42:30.890004+0000","last_fullsized":"2024-02-23T00:42:30.890004+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'4","last_scrub_stamp":"2024-02-23T00:41:43.874397+0000","last_deep_scrub":"95'4","last_deep_scrub_stamp":"2024-02-23T00:41:43.874397+0000","last_clean_scrub_stamp":"2024-02-23T00:41:43.874397+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"68'25","reported_seq":107,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.890004+0000","last_change":"2024-02-23T00:41:08.004573+0000","last_active":"2024-02-23T00:42:30.890004+0000","last_peered":"2024-02-23T00:42:30.890004+0000","last_clean":"2024-02-23T00:42:30.890004+0000","last_became_active":"2024-02-23T00:39:16.783926+0000","last_became_peered":"2024-02-23T00:39:16.783926+0000","last_unstale":"2024-02-23T00:42:30.890004+0000","last_undegraded":"2024-02-23T00:42:30.890004+0000","last_fullsized":"2024-02-23T00:42:30.890004+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'25","last_scrub_stamp":"2024-02-23T00:41:08.004489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:08.004489+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":56,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.888839+0000","last_change":"2024-02-23T00:41:57.376669+0000","last_active":"2024-02-23T00:42:30.888839+0000","last_peered":"2024-02-23T00:42:30.888839+0000","last_clean":"2024-02-23T00:42:30.888839+0000","last_became_active":"2024-02-23T00:39:18.796323+0000","last_became_peered":"2024-02-23T00:39:18.796323+0000","last_unstale":"2024-02-23T00:42:30.888839+0000","last_undegraded":"2024-02-23T00:42:30.888839+0000","last_fullsized":"2024-02-23T00:42:30.888839+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:57.376576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_clean_scrub_stamp":"2024-02-23T00:41:57.376576+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"67'1","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.665062+0000","last_change":"2024-02-23T00:41:00.809258+0000","last_active":"2024-02-23T00:42:30.665062+0000","last_peered":"2024-02-23T00:42:30.665062+0000","last_clean":"2024-02-23T00:42:30.665062+0000","last_became_active":"2024-02-23T00:39:20.800858+0000","last_became_peered":"2024-02-23T00:39:20.800858+0000","last_unstale":"2024-02-23T00:42:30.665062+0000","last_undegraded":"2024-02-23T00:42:30.665062+0000","last_fullsized":"2024-02-23T00:42:30.665062+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:41:00.809190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:00.809190+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.443645+0000","last_change":"2024-02-23T00:40:55.723385+0000","last_active":"2024-02-23T00:42:30.443645+0000","last_peered":"2024-02-23T00:42:30.443645+0000","last_clean":"2024-02-23T00:42:30.443645+0000","last_became_active":"2024-02-23T00:39:14.779952+0000","last_became_peered":"2024-02-23T00:39:14.779952+0000","last_unstale":"2024-02-23T00:42:30.443645+0000","last_undegraded":"2024-02-23T00:42:30.443645+0000","last_fullsized":"2024-02-23T00:42:30.443645+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:55.723309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:55.723309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"68'16","reported_seq":83,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.442960+0000","last_change":"2024-02-23T00:40:41.772244+0000","last_active":"2024-02-23T00:42:30.442960+0000","last_peered":"2024-02-23T00:42:30.442960+0000","last_clean":"2024-02-23T00:42:30.442960+0000","last_became_active":"2024-02-23T00:39:16.787468+0000","last_became_peered":"2024-02-23T00:39:16.787468+0000","last_unstale":"2024-02-23T00:42:30.442960+0000","last_undegraded":"2024-02-23T00:42:30.442960+0000","last_fullsized":"2024-02-23T00:42:30.442960+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:40:41.772150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:41.772150+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"91'9","reported_seq":100,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.666003+0000","last_change":"2024-02-23T00:41:12.843814+0000","last_active":"2024-02-23T00:42:30.666003+0000","last_peered":"2024-02-23T00:42:30.666003+0000","last_clean":"2024-02-23T00:42:30.666003+0000","last_became_active":"2024-02-23T00:39:18.801297+0000","last_became_peered":"2024-02-23T00:39:18.801297+0000","last_unstale":"2024-02-23T00:42:30.666003+0000","last_undegraded":"2024-02-23T00:42:30.666003+0000","last_fullsized":"2024-02-23T00:42:30.666003+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'9","last_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_deep_scrub":"91'9","last_deep_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_clean_scrub_stamp":"2024-02-23T00:41:12.843733+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.446500+0000","last_change":"2024-02-23T00:41:18.910682+0000","last_active":"2024-02-23T00:42:30.446500+0000","last_peered":"2024-02-23T00:42:30.446500+0000","last_clean":"2024-02-23T00:42:30.446500+0000","last_became_active":"2024-02-23T00:39:20.790053+0000","last_became_peered":"2024-02-23T00:39:20.790053+0000","last_unstale":"2024-02-23T00:42:30.446500+0000","last_undegraded":"2024-02-23T00:42:30.446500+0000","last_fullsized":"2024-02-23T00:42:30.446500+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:18.910605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:18.910605+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":61,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.829072+0000","last_change":"2024-02-23T00:42:04.708358+0000","last_active":"2024-02-23T00:42:30.829072+0000","last_peered":"2024-02-23T00:42:30.829072+0000","last_clean":"2024-02-23T00:42:30.829072+0000","last_became_active":"2024-02-23T00:39:14.784015+0000","last_became_peered":"2024-02-23T00:39:14.784015+0000","last_unstale":"2024-02-23T00:42:30.829072+0000","last_undegraded":"2024-02-23T00:42:30.829072+0000","last_fullsized":"2024-02-23T00:42:30.829072+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:04.708280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:04.708280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"86'35","reported_seq":77,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.829111+0000","last_change":"2024-02-23T00:40:48.803965+0000","last_active":"2024-02-23T00:42:30.829111+0000","last_peered":"2024-02-23T00:42:30.829111+0000","last_clean":"2024-02-23T00:42:30.829111+0000","last_became_active":"2024-02-23T00:40:42.933322+0000","last_became_peered":"2024-02-23T00:40:42.933322+0000","last_unstale":"2024-02-23T00:42:30.829111+0000","last_undegraded":"2024-02-23T00:42:30.829111+0000","last_fullsized":"2024-02-23T00:42:30.829111+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'35","last_scrub_stamp":"2024-02-23T00:40:48.803893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:48.803893+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.666597+0000","last_change":"2024-02-23T00:41:19.899792+0000","last_active":"2024-02-23T00:42:30.666597+0000","last_peered":"2024-02-23T00:42:30.666597+0000","last_clean":"2024-02-23T00:42:30.666597+0000","last_became_active":"2024-02-23T00:39:18.802134+0000","last_became_peered":"2024-02-23T00:39:18.802134+0000","last_unstale":"2024-02-23T00:42:30.666597+0000","last_undegraded":"2024-02-23T00:42:30.666597+0000","last_fullsized":"2024-02-23T00:42:30.666597+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:19.899710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:19.899710+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":53,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.891375+0000","last_change":"2024-02-23T00:41:44.001469+0000","last_active":"2024-02-23T00:42:30.891375+0000","last_peered":"2024-02-23T00:42:30.891375+0000","last_clean":"2024-02-23T00:42:30.891375+0000","last_became_active":"2024-02-23T00:39:21.160457+0000","last_became_peered":"2024-02-23T00:39:21.160457+0000","last_unstale":"2024-02-23T00:42:30.891375+0000","last_undegraded":"2024-02-23T00:42:30.891375+0000","last_fullsized":"2024-02-23T00:42:30.891375+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:44.001408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:44.001408+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"68'5","reported_seq":247,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:31.141314+0000","last_change":"2024-02-23T00:41:00.648420+0000","last_active":"2024-02-23T00:42:31.141314+0000","last_peered":"2024-02-23T00:42:31.141314+0000","last_clean":"2024-02-23T00:42:31.141314+0000","last_became_active":"2024-02-23T00:39:14.779109+0000","last_became_peered":"2024-02-23T00:39:14.779109+0000","last_unstale":"2024-02-23T00:42:31.141314+0000","last_undegraded":"2024-02-23T00:42:31.141314+0000","last_fullsized":"2024-02-23T00:42:31.141314+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'5","last_scrub_stamp":"2024-02-23T00:41:00.648353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:00.648353+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":189,"num_read_kb":184,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":60,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.889149+0000","last_change":"2024-02-23T00:42:05.329858+0000","last_active":"2024-02-23T00:42:30.889149+0000","last_peered":"2024-02-23T00:42:30.889149+0000","last_clean":"2024-02-23T00:42:30.889149+0000","last_became_active":"2024-02-23T00:39:14.772411+0000","last_became_peered":"2024-02-23T00:39:14.772411+0000","last_unstale":"2024-02-23T00:42:30.889149+0000","last_undegraded":"2024-02-23T00:42:30.889149+0000","last_fullsized":"2024-02-23T00:42:30.889149+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:05.329757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:05.329757+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"68'19","reported_seq":100,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.444366+0000","last_change":"2024-02-23T00:42:04.625118+0000","last_active":"2024-02-23T00:42:30.444366+0000","last_peered":"2024-02-23T00:42:30.444366+0000","last_clean":"2024-02-23T00:42:30.444366+0000","last_became_active":"2024-02-23T00:39:16.787484+0000","last_became_peered":"2024-02-23T00:39:16.787484+0000","last_unstale":"2024-02-23T00:42:30.444366+0000","last_undegraded":"2024-02-23T00:42:30.444366+0000","last_fullsized":"2024-02-23T00:42:30.444366+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'19","last_scrub_stamp":"2024-02-23T00:42:04.625006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:04.625006+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":56,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.890515+0000","last_change":"2024-02-23T00:41:52.264308+0000","last_active":"2024-02-23T00:42:30.890515+0000","last_peered":"2024-02-23T00:42:30.890515+0000","last_clean":"2024-02-23T00:42:30.890515+0000","last_became_active":"2024-02-23T00:39:18.789330+0000","last_became_peered":"2024-02-23T00:39:18.789330+0000","last_unstale":"2024-02-23T00:42:30.890515+0000","last_undegraded":"2024-02-23T00:42:30.890515+0000","last_fullsized":"2024-02-23T00:42:30.890515+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:52.264240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.264240+0000","last_clean_scrub_stamp":"2024-02-23T00:41:52.264240+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.829518+0000","last_change":"2024-02-23T00:41:55.726397+0000","last_active":"2024-02-23T00:42:30.829518+0000","last_peered":"2024-02-23T00:42:30.829518+0000","last_clean":"2024-02-23T00:42:30.829518+0000","last_became_active":"2024-02-23T00:39:20.797226+0000","last_became_peered":"2024-02-23T00:39:20.797226+0000","last_unstale":"2024-02-23T00:42:30.829518+0000","last_undegraded":"2024-02-23T00:42:30.829518+0000","last_fullsized":"2024-02-23T00:42:30.829518+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:55.726321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:55.726321+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":56,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.445543+0000","last_change":"2024-02-23T00:40:59.958953+0000","last_active":"2024-02-23T00:42:30.445543+0000","last_peered":"2024-02-23T00:42:30.445543+0000","last_clean":"2024-02-23T00:42:30.445543+0000","last_became_active":"2024-02-23T00:39:14.783489+0000","last_became_peered":"2024-02-23T00:39:14.783489+0000","last_unstale":"2024-02-23T00:42:30.445543+0000","last_undegraded":"2024-02-23T00:42:30.445543+0000","last_fullsized":"2024-02-23T00:42:30.445543+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:59.958850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:59.958850+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":56,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.829000+0000","last_change":"2024-02-23T00:40:50.760096+0000","last_active":"2024-02-23T00:42:30.829000+0000","last_peered":"2024-02-23T00:42:30.829000+0000","last_clean":"2024-02-23T00:42:30.829000+0000","last_became_active":"2024-02-23T00:39:14.788888+0000","last_became_peered":"2024-02-23T00:39:14.788888+0000","last_unstale":"2024-02-23T00:42:30.829000+0000","last_undegraded":"2024-02-23T00:42:30.829000+0000","last_fullsized":"2024-02-23T00:42:30.829000+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:50.760015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:50.760015+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.889696+0000","last_change":"2024-02-23T00:40:53.770177+0000","last_active":"2024-02-23T00:42:30.889696+0000","last_peered":"2024-02-23T00:42:30.889696+0000","last_clean":"2024-02-23T00:42:30.889696+0000","last_became_active":"2024-02-23T00:39:08.248249+0000","last_became_peered":"2024-02-23T00:39:08.248249+0000","last_unstale":"2024-02-23T00:42:30.889696+0000","last_undegraded":"2024-02-23T00:42:30.889696+0000","last_fullsized":"2024-02-23T00:42:30.889696+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:53.770082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:53.770082+0000","last_clean_scrub_stamp":"2024-02-23T00:40:53.770082+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"67'14","reported_seq":80,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.442678+0000","last_change":"2024-02-23T00:41:48.041191+0000","last_active":"2024-02-23T00:42:30.442678+0000","last_peered":"2024-02-23T00:42:30.442678+0000","last_clean":"2024-02-23T00:42:30.442678+0000","last_became_active":"2024-02-23T00:39:16.858880+0000","last_became_peered":"2024-02-23T00:39:16.858880+0000","last_unstale":"2024-02-23T00:42:30.442678+0000","last_undegraded":"2024-02-23T00:42:30.442678+0000","last_fullsized":"2024-02-23T00:42:30.442678+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'14","last_scrub_stamp":"2024-02-23T00:41:48.041106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:48.041106+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.891360+0000","last_change":"2024-02-23T00:40:48.790072+0000","last_active":"2024-02-23T00:42:30.891360+0000","last_peered":"2024-02-23T00:42:30.891360+0000","last_clean":"2024-02-23T00:42:30.891360+0000","last_became_active":"2024-02-23T00:39:18.793801+0000","last_became_peered":"2024-02-23T00:39:18.793801+0000","last_unstale":"2024-02-23T00:42:30.891360+0000","last_undegraded":"2024-02-23T00:42:30.891360+0000","last_fullsized":"2024-02-23T00:42:30.891360+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:48.789992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:48.789992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.443471+0000","last_change":"2024-02-23T00:41:02.714085+0000","last_active":"2024-02-23T00:42:30.443471+0000","last_peered":"2024-02-23T00:42:30.443471+0000","last_clean":"2024-02-23T00:42:30.443471+0000","last_became_active":"2024-02-23T00:39:20.797242+0000","last_became_peered":"2024-02-23T00:39:20.797242+0000","last_unstale":"2024-02-23T00:42:30.443471+0000","last_undegraded":"2024-02-23T00:42:30.443471+0000","last_fullsized":"2024-02-23T00:42:30.443471+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:02.714020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:02.714020+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":61,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.665430+0000","last_change":"2024-02-23T00:41:49.197029+0000","last_active":"2024-02-23T00:42:30.665430+0000","last_peered":"2024-02-23T00:42:30.665430+0000","last_clean":"2024-02-23T00:42:30.665430+0000","last_became_active":"2024-02-23T00:39:14.775446+0000","last_became_peered":"2024-02-23T00:39:14.775446+0000","last_unstale":"2024-02-23T00:42:30.665430+0000","last_undegraded":"2024-02-23T00:42:30.665430+0000","last_fullsized":"2024-02-23T00:42:30.665430+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:49.196953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:49.196953+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":59,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.889583+0000","last_change":"2024-02-23T00:41:26.836896+0000","last_active":"2024-02-23T00:42:30.889583+0000","last_peered":"2024-02-23T00:42:30.889583+0000","last_clean":"2024-02-23T00:42:30.889583+0000","last_became_active":"2024-02-23T00:39:14.784578+0000","last_became_peered":"2024-02-23T00:39:14.784578+0000","last_unstale":"2024-02-23T00:42:30.889583+0000","last_undegraded":"2024-02-23T00:42:30.889583+0000","last_fullsized":"2024-02-23T00:42:30.889583+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:26.836819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:26.836819+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"95'18","reported_seq":83,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.666790+0000","last_change":"2024-02-23T00:42:01.195610+0000","last_active":"2024-02-23T00:42:30.666790+0000","last_peered":"2024-02-23T00:42:30.666790+0000","last_clean":"2024-02-23T00:42:30.666790+0000","last_became_active":"2024-02-23T00:39:16.787143+0000","last_became_peered":"2024-02-23T00:39:16.787143+0000","last_unstale":"2024-02-23T00:42:30.666790+0000","last_undegraded":"2024-02-23T00:42:30.666790+0000","last_fullsized":"2024-02-23T00:42:30.666790+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'18","last_scrub_stamp":"2024-02-23T00:42:01.195545+0000","last_deep_scrub":"95'18","last_deep_scrub_stamp":"2024-02-23T00:42:01.195545+0000","last_clean_scrub_stamp":"2024-02-23T00:42:01.195545+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":57,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.828494+0000","last_change":"2024-02-23T00:41:59.694132+0000","last_active":"2024-02-23T00:42:30.828494+0000","last_peered":"2024-02-23T00:42:30.828494+0000","last_clean":"2024-02-23T00:42:30.828494+0000","last_became_active":"2024-02-23T00:39:18.800378+0000","last_became_peered":"2024-02-23T00:39:18.800378+0000","last_unstale":"2024-02-23T00:42:30.828494+0000","last_undegraded":"2024-02-23T00:42:30.828494+0000","last_fullsized":"2024-02-23T00:42:30.828494+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:59.694033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:59.694033+0000","last_clean_scrub_stamp":"2024-02-23T00:41:59.694033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.443307+0000","last_change":"2024-02-23T00:41:50.656785+0000","last_active":"2024-02-23T00:42:30.443307+0000","last_peered":"2024-02-23T00:42:30.443307+0000","last_clean":"2024-02-23T00:42:30.443307+0000","last_became_active":"2024-02-23T00:39:21.160060+0000","last_became_peered":"2024-02-23T00:39:21.160060+0000","last_unstale":"2024-02-23T00:42:30.443307+0000","last_undegraded":"2024-02-23T00:42:30.443307+0000","last_fullsized":"2024-02-23T00:42:30.443307+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:50.656709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:50.656709+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.891439+0000","last_change":"2024-02-23T00:40:41.671069+0000","last_active":"2024-02-23T00:42:30.891439+0000","last_peered":"2024-02-23T00:42:30.891439+0000","last_clean":"2024-02-23T00:42:30.891439+0000","last_became_active":"2024-02-23T00:39:14.783662+0000","last_became_peered":"2024-02-23T00:39:14.783662+0000","last_unstale":"2024-02-23T00:42:30.891439+0000","last_undegraded":"2024-02-23T00:42:30.891439+0000","last_fullsized":"2024-02-23T00:42:30.891439+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:41.670962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:41.670962+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"68'25","reported_seq":105,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.665671+0000","last_change":"2024-02-23T00:40:52.751667+0000","last_active":"2024-02-23T00:42:30.665671+0000","last_peered":"2024-02-23T00:42:30.665671+0000","last_clean":"2024-02-23T00:42:30.665671+0000","last_became_active":"2024-02-23T00:39:16.857278+0000","last_became_peered":"2024-02-23T00:39:16.857278+0000","last_unstale":"2024-02-23T00:42:30.665671+0000","last_undegraded":"2024-02-23T00:42:30.665671+0000","last_fullsized":"2024-02-23T00:42:30.665671+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'25","last_scrub_stamp":"2024-02-23T00:40:52.751597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:52.751597+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.442616+0000","last_change":"2024-02-23T00:40:50.029161+0000","last_active":"2024-02-23T00:42:30.442616+0000","last_peered":"2024-02-23T00:42:30.442616+0000","last_clean":"2024-02-23T00:42:30.442616+0000","last_became_active":"2024-02-23T00:39:18.796664+0000","last_became_peered":"2024-02-23T00:39:18.796664+0000","last_unstale":"2024-02-23T00:42:30.442616+0000","last_undegraded":"2024-02-23T00:42:30.442616+0000","last_fullsized":"2024-02-23T00:42:30.442616+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:50.029087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:50.029087+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.890488+0000","last_change":"2024-02-23T00:40:49.937604+0000","last_active":"2024-02-23T00:42:30.890488+0000","last_peered":"2024-02-23T00:42:30.890488+0000","last_clean":"2024-02-23T00:42:30.890488+0000","last_became_active":"2024-02-23T00:39:20.795271+0000","last_became_peered":"2024-02-23T00:39:20.795271+0000","last_unstale":"2024-02-23T00:42:30.890488+0000","last_undegraded":"2024-02-23T00:42:30.890488+0000","last_fullsized":"2024-02-23T00:42:30.890488+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:49.937526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:49.937526+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.444130+0000","last_change":"2024-02-23T00:40:36.888232+0000","last_active":"2024-02-23T00:42:30.444130+0000","last_peered":"2024-02-23T00:42:30.444130+0000","last_clean":"2024-02-23T00:42:30.444130+0000","last_became_active":"2024-02-23T00:39:14.779193+0000","last_became_peered":"2024-02-23T00:39:14.779193+0000","last_unstale":"2024-02-23T00:42:30.444130+0000","last_undegraded":"2024-02-23T00:42:30.444130+0000","last_fullsized":"2024-02-23T00:42:30.444130+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:36.888124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:36.888124+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"68'30","reported_seq":116,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.444911+0000","last_change":"2024-02-23T00:41:21.909427+0000","last_active":"2024-02-23T00:42:30.444911+0000","last_peered":"2024-02-23T00:42:30.444911+0000","last_clean":"2024-02-23T00:42:30.444911+0000","last_became_active":"2024-02-23T00:39:16.859002+0000","last_became_peered":"2024-02-23T00:39:16.859002+0000","last_unstale":"2024-02-23T00:42:30.444911+0000","last_undegraded":"2024-02-23T00:42:30.444911+0000","last_fullsized":"2024-02-23T00:42:30.444911+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'30","last_scrub_stamp":"2024-02-23T00:41:21.909340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:21.909340+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":56,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.889171+0000","last_change":"2024-02-23T00:41:52.965337+0000","last_active":"2024-02-23T00:42:30.889171+0000","last_peered":"2024-02-23T00:42:30.889171+0000","last_clean":"2024-02-23T00:42:30.889171+0000","last_became_active":"2024-02-23T00:39:18.798829+0000","last_became_peered":"2024-02-23T00:39:18.798829+0000","last_unstale":"2024-02-23T00:42:30.889171+0000","last_undegraded":"2024-02-23T00:42:30.889171+0000","last_fullsized":"2024-02-23T00:42:30.889171+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:52.965253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:52.965253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":53,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.891095+0000","last_change":"2024-02-23T00:42:06.945763+0000","last_active":"2024-02-23T00:42:30.891095+0000","last_peered":"2024-02-23T00:42:30.891095+0000","last_clean":"2024-02-23T00:42:30.891095+0000","last_became_active":"2024-02-23T00:39:21.161928+0000","last_became_peered":"2024-02-23T00:39:21.161928+0000","last_unstale":"2024-02-23T00:42:30.891095+0000","last_undegraded":"2024-02-23T00:42:30.891095+0000","last_fullsized":"2024-02-23T00:42:30.891095+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:06.945642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.945642+0000","last_clean_scrub_stamp":"2024-02-23T00:42:06.945642+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":62,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.442873+0000","last_change":"2024-02-23T00:42:04.191017+0000","last_active":"2024-02-23T00:42:30.442873+0000","last_peered":"2024-02-23T00:42:30.442873+0000","last_clean":"2024-02-23T00:42:30.442873+0000","last_became_active":"2024-02-23T00:39:14.781999+0000","last_became_peered":"2024-02-23T00:39:14.781999+0000","last_unstale":"2024-02-23T00:42:30.442873+0000","last_undegraded":"2024-02-23T00:42:30.442873+0000","last_fullsized":"2024-02-23T00:42:30.442873+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:04.190935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:04.190935+0000","last_clean_scrub_stamp":"2024-02-23T00:42:04.190935+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"68'20","reported_seq":89,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.889728+0000","last_change":"2024-02-23T00:41:05.002039+0000","last_active":"2024-02-23T00:42:30.889728+0000","last_peered":"2024-02-23T00:42:30.889728+0000","last_clean":"2024-02-23T00:42:30.889728+0000","last_became_active":"2024-02-23T00:39:16.780130+0000","last_became_peered":"2024-02-23T00:39:16.780130+0000","last_unstale":"2024-02-23T00:42:30.889728+0000","last_undegraded":"2024-02-23T00:42:30.889728+0000","last_fullsized":"2024-02-23T00:42:30.889728+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'20","last_scrub_stamp":"2024-02-23T00:41:05.001912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:05.001912+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":99,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.444831+0000","last_change":"2024-02-23T00:41:09.882111+0000","last_active":"2024-02-23T00:42:30.444831+0000","last_peered":"2024-02-23T00:42:30.444831+0000","last_clean":"2024-02-23T00:42:30.444831+0000","last_became_active":"2024-02-23T00:39:18.798536+0000","last_became_peered":"2024-02-23T00:39:18.798536+0000","last_unstale":"2024-02-23T00:42:30.444831+0000","last_undegraded":"2024-02-23T00:42:30.444831+0000","last_fullsized":"2024-02-23T00:42:30.444831+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:41:09.882012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:09.882012+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.445963+0000","last_change":"2024-02-23T00:42:09.080023+0000","last_active":"2024-02-23T00:42:30.445963+0000","last_peered":"2024-02-23T00:42:30.445963+0000","last_clean":"2024-02-23T00:42:30.445963+0000","last_became_active":"2024-02-23T00:39:21.160443+0000","last_became_peered":"2024-02-23T00:39:21.160443+0000","last_unstale":"2024-02-23T00:42:30.445963+0000","last_undegraded":"2024-02-23T00:42:30.445963+0000","last_fullsized":"2024-02-23T00:42:30.445963+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:09.079918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:09.079918+0000","last_clean_scrub_stamp":"2024-02-23T00:42:09.079918+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.828774+0000","last_change":"2024-02-23T00:41:04.839634+0000","last_active":"2024-02-23T00:42:30.828774+0000","last_peered":"2024-02-23T00:42:30.828774+0000","last_clean":"2024-02-23T00:42:30.828774+0000","last_became_active":"2024-02-23T00:39:18.800777+0000","last_became_peered":"2024-02-23T00:39:18.800777+0000","last_unstale":"2024-02-23T00:42:30.828774+0000","last_undegraded":"2024-02-23T00:42:30.828774+0000","last_fullsized":"2024-02-23T00:42:30.828774+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_clean_scrub_stamp":"2024-02-23T00:41:04.839555+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":59,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.889777+0000","last_change":"2024-02-23T00:41:42.978987+0000","last_active":"2024-02-23T00:42:30.889777+0000","last_peered":"2024-02-23T00:42:30.889777+0000","last_clean":"2024-02-23T00:42:30.889777+0000","last_became_active":"2024-02-23T00:39:14.784121+0000","last_became_peered":"2024-02-23T00:39:14.784121+0000","last_unstale":"2024-02-23T00:42:30.889777+0000","last_undegraded":"2024-02-23T00:42:30.889777+0000","last_fullsized":"2024-02-23T00:42:30.889777+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:42.978905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:42.978905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"68'17","reported_seq":110,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.889799+0000","last_change":"2024-02-23T00:41:13.017122+0000","last_active":"2024-02-23T00:42:30.889799+0000","last_peered":"2024-02-23T00:42:30.889799+0000","last_clean":"2024-02-23T00:42:30.889799+0000","last_became_active":"2024-02-23T00:40:42.933759+0000","last_became_peered":"2024-02-23T00:40:42.933759+0000","last_unstale":"2024-02-23T00:42:30.889799+0000","last_undegraded":"2024-02-23T00:42:30.889799+0000","last_fullsized":"2024-02-23T00:42:30.889799+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:41:13.017043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:13.017043+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.829431+0000","last_change":"2024-02-23T00:42:10.666402+0000","last_active":"2024-02-23T00:42:30.829431+0000","last_peered":"2024-02-23T00:42:30.829431+0000","last_clean":"2024-02-23T00:42:30.829431+0000","last_became_active":"2024-02-23T00:39:20.800800+0000","last_became_peered":"2024-02-23T00:39:20.800800+0000","last_unstale":"2024-02-23T00:42:30.829431+0000","last_undegraded":"2024-02-23T00:42:30.829431+0000","last_fullsized":"2024-02-23T00:42:30.829431+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:10.666299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:10.666299+0000","last_clean_scrub_stamp":"2024-02-23T00:42:10.666299+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":103,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.889240+0000","last_change":"2024-02-23T00:41:49.927409+0000","last_active":"2024-02-23T00:42:30.889240+0000","last_peered":"2024-02-23T00:42:30.889240+0000","last_clean":"2024-02-23T00:42:30.889240+0000","last_became_active":"2024-02-23T00:39:18.800440+0000","last_became_peered":"2024-02-23T00:39:18.800440+0000","last_unstale":"2024-02-23T00:42:30.889240+0000","last_undegraded":"2024-02-23T00:42:30.889240+0000","last_fullsized":"2024-02-23T00:42:30.889240+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:41:49.927326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:49.927326+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":60,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.443521+0000","last_change":"2024-02-23T00:41:56.728679+0000","last_active":"2024-02-23T00:42:30.443521+0000","last_peered":"2024-02-23T00:42:30.443521+0000","last_clean":"2024-02-23T00:42:30.443521+0000","last_became_active":"2024-02-23T00:39:14.780526+0000","last_became_peered":"2024-02-23T00:39:14.780526+0000","last_unstale":"2024-02-23T00:42:30.443521+0000","last_undegraded":"2024-02-23T00:42:30.443521+0000","last_fullsized":"2024-02-23T00:42:30.443521+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:56.728577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:56.728577+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"67'15","reported_seq":83,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.444495+0000","last_change":"2024-02-23T00:41:39.682283+0000","last_active":"2024-02-23T00:42:30.444495+0000","last_peered":"2024-02-23T00:42:30.444495+0000","last_clean":"2024-02-23T00:42:30.444495+0000","last_became_active":"2024-02-23T00:39:16.789757+0000","last_became_peered":"2024-02-23T00:39:16.789757+0000","last_unstale":"2024-02-23T00:42:30.444495+0000","last_undegraded":"2024-02-23T00:42:30.444495+0000","last_fullsized":"2024-02-23T00:42:30.444495+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:41:39.682182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:39.682182+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.889649+0000","last_change":"2024-02-23T00:41:15.081250+0000","last_active":"2024-02-23T00:42:30.889649+0000","last_peered":"2024-02-23T00:42:30.889649+0000","last_clean":"2024-02-23T00:42:30.889649+0000","last_became_active":"2024-02-23T00:39:20.795097+0000","last_became_peered":"2024-02-23T00:39:20.795097+0000","last_unstale":"2024-02-23T00:42:30.889649+0000","last_undegraded":"2024-02-23T00:42:30.889649+0000","last_fullsized":"2024-02-23T00:42:30.889649+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:15.081153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:15.081153+0000","last_clean_scrub_stamp":"2024-02-23T00:41:15.081153+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"67'8","reported_seq":98,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.889970+0000","last_change":"2024-02-23T00:40:58.331136+0000","last_active":"2024-02-23T00:42:30.889970+0000","last_peered":"2024-02-23T00:42:30.889970+0000","last_clean":"2024-02-23T00:42:30.889970+0000","last_became_active":"2024-02-23T00:39:18.799544+0000","last_became_peered":"2024-02-23T00:39:18.799544+0000","last_unstale":"2024-02-23T00:42:30.889970+0000","last_undegraded":"2024-02-23T00:42:30.889970+0000","last_fullsized":"2024-02-23T00:42:30.889970+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:40:58.331048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:58.331048+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":60,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.442793+0000","last_change":"2024-02-23T00:42:06.575346+0000","last_active":"2024-02-23T00:42:30.442793+0000","last_peered":"2024-02-23T00:42:30.442793+0000","last_clean":"2024-02-23T00:42:30.442793+0000","last_became_active":"2024-02-23T00:39:14.780379+0000","last_became_peered":"2024-02-23T00:39:14.780379+0000","last_unstale":"2024-02-23T00:42:30.442793+0000","last_undegraded":"2024-02-23T00:42:30.442793+0000","last_fullsized":"2024-02-23T00:42:30.442793+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:06.575242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.575242+0000","last_clean_scrub_stamp":"2024-02-23T00:42:06.575242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"67'10","reported_seq":68,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.889942+0000","last_change":"2024-02-23T00:41:14.059424+0000","last_active":"2024-02-23T00:42:30.889942+0000","last_peered":"2024-02-23T00:42:30.889942+0000","last_clean":"2024-02-23T00:42:30.889942+0000","last_became_active":"2024-02-23T00:39:16.777530+0000","last_became_peered":"2024-02-23T00:39:16.777530+0000","last_unstale":"2024-02-23T00:42:30.889942+0000","last_undegraded":"2024-02-23T00:42:30.889942+0000","last_fullsized":"2024-02-23T00:42:30.889942+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'10","last_scrub_stamp":"2024-02-23T00:41:14.059334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:14.059334+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.890404+0000","last_change":"2024-02-23T00:40:54.348946+0000","last_active":"2024-02-23T00:42:30.890404+0000","last_peered":"2024-02-23T00:42:30.890404+0000","last_clean":"2024-02-23T00:42:30.890404+0000","last_became_active":"2024-02-23T00:39:20.800943+0000","last_became_peered":"2024-02-23T00:39:20.800943+0000","last_unstale":"2024-02-23T00:42:30.890404+0000","last_undegraded":"2024-02-23T00:42:30.890404+0000","last_fullsized":"2024-02-23T00:42:30.890404+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:54.348843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:54.348843+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":56,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.443537+0000","last_change":"2024-02-23T00:41:53.692312+0000","last_active":"2024-02-23T00:42:30.443537+0000","last_peered":"2024-02-23T00:42:30.443537+0000","last_clean":"2024-02-23T00:42:30.443537+0000","last_became_active":"2024-02-23T00:39:18.799875+0000","last_became_peered":"2024-02-23T00:39:18.799875+0000","last_unstale":"2024-02-23T00:42:30.443537+0000","last_undegraded":"2024-02-23T00:42:30.443537+0000","last_fullsized":"2024-02-23T00:42:30.443537+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:53.692185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:53.692185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.888974+0000","last_change":"2024-02-23T00:41:40.128467+0000","last_active":"2024-02-23T00:42:30.888974+0000","last_peered":"2024-02-23T00:42:30.888974+0000","last_clean":"2024-02-23T00:42:30.888974+0000","last_became_active":"2024-02-23T00:40:42.931041+0000","last_became_peered":"2024-02-23T00:40:42.931041+0000","last_unstale":"2024-02-23T00:42:30.888974+0000","last_undegraded":"2024-02-23T00:42:30.888974+0000","last_fullsized":"2024-02-23T00:42:30.888974+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:40.128397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:40.128397+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"67'17","reported_seq":108,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.890452+0000","last_change":"2024-02-23T00:41:02.038717+0000","last_active":"2024-02-23T00:42:30.890452+0000","last_peered":"2024-02-23T00:42:30.890452+0000","last_clean":"2024-02-23T00:42:30.890452+0000","last_became_active":"2024-02-23T00:40:42.933806+0000","last_became_peered":"2024-02-23T00:40:42.933806+0000","last_unstale":"2024-02-23T00:42:30.890452+0000","last_undegraded":"2024-02-23T00:42:30.890452+0000","last_fullsized":"2024-02-23T00:42:30.890452+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'17","last_scrub_stamp":"2024-02-23T00:41:02.038627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:02.038627+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.889503+0000","last_change":"2024-02-23T00:40:51.777535+0000","last_active":"2024-02-23T00:42:30.889503+0000","last_peered":"2024-02-23T00:42:30.889503+0000","last_clean":"2024-02-23T00:42:30.889503+0000","last_became_active":"2024-02-23T00:39:20.798989+0000","last_became_peered":"2024-02-23T00:39:20.798989+0000","last_unstale":"2024-02-23T00:42:30.889503+0000","last_undegraded":"2024-02-23T00:42:30.889503+0000","last_fullsized":"2024-02-23T00:42:30.889503+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:51.777405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:40:51.777405+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":56,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.888853+0000","last_change":"2024-02-23T00:42:03.291538+0000","last_active":"2024-02-23T00:42:30.888853+0000","last_peered":"2024-02-23T00:42:30.888853+0000","last_clean":"2024-02-23T00:42:30.888853+0000","last_became_active":"2024-02-23T00:39:18.792718+0000","last_became_peered":"2024-02-23T00:39:18.792718+0000","last_unstale":"2024-02-23T00:42:30.888853+0000","last_undegraded":"2024-02-23T00:42:30.888853+0000","last_fullsized":"2024-02-23T00:42:30.888853+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:03.291436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:03.291436+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":60,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.890075+0000","last_change":"2024-02-23T00:41:54.303858+0000","last_active":"2024-02-23T00:42:30.890075+0000","last_peered":"2024-02-23T00:42:30.890075+0000","last_clean":"2024-02-23T00:42:30.890075+0000","last_became_active":"2024-02-23T00:39:14.776705+0000","last_became_peered":"2024-02-23T00:39:14.776705+0000","last_unstale":"2024-02-23T00:42:30.890075+0000","last_undegraded":"2024-02-23T00:42:30.890075+0000","last_fullsized":"2024-02-23T00:42:30.890075+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:54.303790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_clean_scrub_stamp":"2024-02-23T00:41:54.303790+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"88'35","reported_seq":114,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.442389+0000","last_change":"2024-02-23T00:41:58.120657+0000","last_active":"2024-02-23T00:42:30.442389+0000","last_peered":"2024-02-23T00:42:30.442389+0000","last_clean":"2024-02-23T00:42:30.442389+0000","last_became_active":"2024-02-23T00:39:16.857413+0000","last_became_peered":"2024-02-23T00:39:16.857413+0000","last_unstale":"2024-02-23T00:42:30.442389+0000","last_undegraded":"2024-02-23T00:42:30.442389+0000","last_fullsized":"2024-02-23T00:42:30.442389+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'35","last_scrub_stamp":"2024-02-23T00:41:58.120561+0000","last_deep_scrub":"88'35","last_deep_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_clean_scrub_stamp":"2024-02-23T00:41:58.120561+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.665132+0000","last_change":"2024-02-23T00:41:05.802853+0000","last_active":"2024-02-23T00:42:30.665132+0000","last_peered":"2024-02-23T00:42:30.665132+0000","last_clean":"2024-02-23T00:42:30.665132+0000","last_became_active":"2024-02-23T00:39:20.802938+0000","last_became_peered":"2024-02-23T00:39:20.802938+0000","last_unstale":"2024-02-23T00:42:30.665132+0000","last_undegraded":"2024-02-23T00:42:30.665132+0000","last_fullsized":"2024-02-23T00:42:30.665132+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:05.802777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:05.802777+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":56,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.889369+0000","last_change":"2024-02-23T00:42:08.335396+0000","last_active":"2024-02-23T00:42:30.889369+0000","last_peered":"2024-02-23T00:42:30.889369+0000","last_clean":"2024-02-23T00:42:30.889369+0000","last_became_active":"2024-02-23T00:39:18.794476+0000","last_became_peered":"2024-02-23T00:39:18.794476+0000","last_unstale":"2024-02-23T00:42:30.889369+0000","last_undegraded":"2024-02-23T00:42:30.889369+0000","last_fullsized":"2024-02-23T00:42:30.889369+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:08.335330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:08.335330+0000","last_clean_scrub_stamp":"2024-02-23T00:42:08.335330+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.443921+0000","last_change":"2024-02-23T00:40:57.681793+0000","last_active":"2024-02-23T00:42:30.443921+0000","last_peered":"2024-02-23T00:42:30.443921+0000","last_clean":"2024-02-23T00:42:30.443921+0000","last_became_active":"2024-02-23T00:39:14.772493+0000","last_became_peered":"2024-02-23T00:39:14.772493+0000","last_unstale":"2024-02-23T00:42:30.443921+0000","last_undegraded":"2024-02-23T00:42:30.443921+0000","last_fullsized":"2024-02-23T00:42:30.443921+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:57.681723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:40:57.681723+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"68'15","reported_seq":90,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.665878+0000","last_change":"2024-02-23T00:42:08.357114+0000","last_active":"2024-02-23T00:42:30.665878+0000","last_peered":"2024-02-23T00:42:30.665878+0000","last_clean":"2024-02-23T00:42:30.665878+0000","last_became_active":"2024-02-23T00:39:16.783701+0000","last_became_peered":"2024-02-23T00:39:16.783701+0000","last_unstale":"2024-02-23T00:42:30.665878+0000","last_undegraded":"2024-02-23T00:42:30.665878+0000","last_fullsized":"2024-02-23T00:42:30.665878+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:42:08.357019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:08.357019+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.828922+0000","last_change":"2024-02-23T00:42:05.725881+0000","last_active":"2024-02-23T00:42:30.828922+0000","last_peered":"2024-02-23T00:42:30.828922+0000","last_clean":"2024-02-23T00:42:30.828922+0000","last_became_active":"2024-02-23T00:39:21.160429+0000","last_became_peered":"2024-02-23T00:39:21.160429+0000","last_unstale":"2024-02-23T00:42:30.828922+0000","last_undegraded":"2024-02-23T00:42:30.828922+0000","last_fullsized":"2024-02-23T00:42:30.828922+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:05.725796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:05.725796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":60,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.890334+0000","last_change":"2024-02-23T00:42:04.330493+0000","last_active":"2024-02-23T00:42:30.890334+0000","last_peered":"2024-02-23T00:42:30.890334+0000","last_clean":"2024-02-23T00:42:30.890334+0000","last_became_active":"2024-02-23T00:39:14.784436+0000","last_became_peered":"2024-02-23T00:39:14.784436+0000","last_unstale":"2024-02-23T00:42:30.890334+0000","last_undegraded":"2024-02-23T00:42:30.890334+0000","last_fullsized":"2024-02-23T00:42:30.890334+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:04.330426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:04.330426+0000","last_clean_scrub_stamp":"2024-02-23T00:42:04.330426+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"68'27","reported_seq":109,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.828741+0000","last_change":"2024-02-23T00:40:59.885996+0000","last_active":"2024-02-23T00:42:30.828741+0000","last_peered":"2024-02-23T00:42:30.828741+0000","last_clean":"2024-02-23T00:42:30.828741+0000","last_became_active":"2024-02-23T00:39:16.859401+0000","last_became_peered":"2024-02-23T00:39:16.859401+0000","last_unstale":"2024-02-23T00:42:30.828741+0000","last_undegraded":"2024-02-23T00:42:30.828741+0000","last_fullsized":"2024-02-23T00:42:30.828741+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'27","last_scrub_stamp":"2024-02-23T00:40:59.885908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:40:59.885908+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"67'8","reported_seq":100,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-02-23T00:42:30.890495+0000","last_change":"2024-02-23T00:40:44.713885+0000","last_active":"2024-02-23T00:42:30.890495+0000","last_peered":"2024-02-23T00:42:30.890495+0000","last_clean":"2024-02-23T00:42:30.890495+0000","last_became_active":"2024-02-23T00:39:18.793414+0000","last_became_peered":"2024-02-23T00:39:18.793414+0000","last_unstale":"2024-02-23T00:42:30.890495+0000","last_undegraded":"2024-02-23T00:42:30.890495+0000","last_fullsized":"2024-02-23T00:42:30.890495+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:40:44.713825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:44.713825+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1082,"num_read_kb":807,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":628,"ondisk_log_size":628,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":189,"num_read_kb":184,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234027,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297820,"kb_used_data":724,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447332,"statfs":{"total":95995035648,"available":95690067968,"internally_reserved":0,"allocated":741376,"data_stored":394624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:42:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.54800000000000004,"15min":0.54700000000000004},"min":{"1min":0.40600000000000003,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.73699999999999999,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.46100000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76400000000000001,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.39500000000000002}]},{"osd":1,"last update":"Fri Feb 23 00:42:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.54900000000000004,"15min":0.55100000000000005},"min":{"1min":0.39700000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.44600000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.83299999999999996,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.64100000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:42:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53400000000000003,"15min":0.53900000000000003},"min":{"1min":0.41099999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.71099999999999997,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.54000000000000004,"15min":0.53600000000000003},"min":{"1min":0.46800000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79500000000000004,"5min":1.024,"15min":1.024},"last":0.68200000000000005}]},{"osd":3,"last update":"Fri Feb 23 00:42:13 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.54000000000000004,"15min":0.53100000000000003},"min":{"1min":0.48799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.41499999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80200000000000005,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":1.7150000000000001}]},{"osd":4,"last update":"Fri Feb 23 00:42:13 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.435,"15min":0.433},"min":{"1min":0.34899999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.46100000000000002,"15min":0.45600000000000002},"min":{"1min":0.38800000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.60699999999999998}]},{"osd":5,"last update":"Fri Feb 23 00:42:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.48199999999999998,"15min":0.47999999999999998},"min":{"1min":0.34599999999999997,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78700000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.5},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.496,"15min":0.48199999999999998},"min":{"1min":0.39100000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.72299999999999998}]},{"osd":6,"last update":"Fri Feb 23 00:42:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.51600000000000001,"15min":0.50900000000000001},"min":{"1min":0.45000000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.51200000000000001,"15min":0.50800000000000001},"min":{"1min":0.436,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.65900000000000003}]}]},{"osd":6,"up_from":47,"seq":201863462962,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297788,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447364,"statfs":{"total":95995035648,"available":95690100736,"internally_reserved":0,"allocated":708608,"data_stored":386279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:41:44 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56000000000000005,"15min":0.56899999999999995},"min":{"1min":0.36399999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70899999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.56599999999999995,"15min":0.57999999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.69399999999999995,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.46100000000000002}]},{"osd":1,"last update":"Fri Feb 23 00:41:44 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.54100000000000004,"15min":0.54900000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.67400000000000004,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56799999999999995,"15min":0.58099999999999996},"min":{"1min":0.34100000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.64600000000000002,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.51600000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:41:44 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.57199999999999995,"15min":0.58099999999999996},"min":{"1min":0.378,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.66500000000000004,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.55300000000000005,"15min":0.55800000000000005},"min":{"1min":0.39400000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.68600000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":2.8530000000000002}]},{"osd":3,"last update":"Fri Feb 23 00:41:44 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57299999999999995,"15min":0.57999999999999996},"min":{"1min":0.43099999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68300000000000005,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.58599999999999997,"15min":0.59199999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71599999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.60499999999999998}]},{"osd":4,"last update":"Fri Feb 23 00:41:44 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.41899999999999998,"15min":0.40699999999999997},"min":{"1min":0.29899999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.69199999999999995,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.443,"15min":0.43099999999999999},"min":{"1min":0.34000000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71199999999999997,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.64600000000000002}]},{"osd":5,"last update":"Fri Feb 23 00:41:44 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.43099999999999999,"15min":0.41599999999999998},"min":{"1min":0.32100000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.66600000000000004,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.45700000000000002,"15min":0.441},"min":{"1min":0.34200000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76100000000000001,"5min":0.877,"15min":0.877},"last":0.59499999999999997}]},{"osd":7,"last update":"Fri Feb 23 00:42:11 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.497,"15min":0.49099999999999999},"min":{"1min":0.36799999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.66600000000000004,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52000000000000002,"15min":0.51600000000000001},"min":{"1min":0.40799999999999997,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70299999999999996,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.69499999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":301928,"kb_used_data":736,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":93443224,"statfs":{"total":95995035648,"available":95685861376,"internally_reserved":0,"allocated":753664,"data_stored":395573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:41:37 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.379,"15min":0.34799999999999998},"min":{"1min":0.22600000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.60799999999999998,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":1.103},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.40300000000000002,"15min":0.36499999999999999},"min":{"1min":0.27500000000000002,"5min":0.191,"15min":0.191},"max":{"1min":0.57599999999999996,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.60899999999999999}]},{"osd":2,"last update":"Fri Feb 23 00:41:59 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.39500000000000002,"15min":0.39300000000000002},"min":{"1min":0.23899999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.438,"15min":0.434},"min":{"1min":0.27000000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.60799999999999998,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":1.0169999999999999}]},{"osd":3,"last update":"Fri Feb 23 00:41:31 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.47199999999999998,"15min":0.48499999999999999},"min":{"1min":0.314,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.62,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.443,"15min":0.45500000000000002},"min":{"1min":0.28999999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.66900000000000004,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.995}]},{"osd":4,"last update":"Fri Feb 23 00:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.38300000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":8.9359999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.62,"15min":0.61599999999999999},"min":{"1min":0.373,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":3.9079999999999999}]},{"osd":5,"last update":"Fri Feb 23 00:42:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.624,"15min":0.65300000000000002},"min":{"1min":0.42599999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":8.9719999999999995},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68600000000000005,"15min":0.71199999999999997},"min":{"1min":0.39500000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.997,"5min":1.008,"15min":1.008},"last":0.59599999999999997}]},{"osd":6,"last update":"Fri Feb 23 00:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.63100000000000001,"15min":0.64000000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":1.044},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64300000000000002,"15min":0.65100000000000002},"min":{"1min":0.42899999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.58199999999999996}]},{"osd":7,"last update":"Fri Feb 23 00:42:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.64000000000000001,"15min":0.629},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.621,"15min":0.621},"min":{"1min":0.442,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93000000000000005,"5min":1.022,"15min":1.022},"last":3.9380000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297812,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447340,"statfs":{"total":95995035648,"available":95690076160,"internally_reserved":0,"allocated":733184,"data_stored":387154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 23 00:41:34 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.36899999999999999,"15min":0.33500000000000002},"min":{"1min":0.223,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.42999999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.376,"5min":0.40200000000000002,"15min":0.35299999999999998},"min":{"1min":0.28100000000000003,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.48899999999999999,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.30599999999999999}]},{"osd":2,"last update":"Fri Feb 23 00:42:03 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.40899999999999997,"15min":0.39000000000000001},"min":{"1min":0.28299999999999997,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.47299999999999998,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.38600000000000001,"15min":0.378},"min":{"1min":0.24299999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.46700000000000003,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.58199999999999996}]},{"osd":3,"last update":"Fri Feb 23 00:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.52000000000000002,"15min":0.47199999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":7.7110000000000003,"5min":7.7110000000000003,"15min":7.7110000000000003},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.54300000000000004,"15min":0.51800000000000002},"min":{"1min":0.33400000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":6.6929999999999996,"5min":6.6929999999999996,"15min":6.6929999999999996},"last":0.68799999999999994}]},{"osd":4,"last update":"Fri Feb 23 00:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.63100000000000001,"15min":0.68300000000000005},"min":{"1min":0.42999999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73399999999999999,"5min":3.2909999999999999,"15min":3.2909999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.66400000000000003,"15min":0.72999999999999998},"min":{"1min":0.45200000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72999999999999998,"5min":3.319,"15min":3.319},"last":0.57199999999999995}]},{"osd":5,"last update":"Fri Feb 23 00:42:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.57699999999999996,"15min":0.60199999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.65100000000000002,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.64700000000000002,"15min":0.66500000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.76700000000000002,"5min":1.131,"15min":1.131},"last":0.64400000000000002}]},{"osd":6,"last update":"Fri Feb 23 00:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.46600000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72099999999999997,"5min":1.093,"15min":1.093},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.622,"15min":0.63100000000000001},"min":{"1min":0.44800000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.67900000000000005,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.754}]},{"osd":7,"last update":"Fri Feb 23 00:42:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.64600000000000002,"15min":0.66800000000000004},"min":{"1min":0.47199999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73999999999999999,"5min":1.163,"15min":1.163},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.624,"15min":0.64200000000000002},"min":{"1min":0.45200000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.63,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.65600000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297800,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447352,"statfs":{"total":95995035648,"available":95690088448,"internally_reserved":0,"allocated":720896,"data_stored":386850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:41:54 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.38,"15min":0.377},"min":{"1min":0.23799999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.53200000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.442},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40000000000000002,"15min":0.38600000000000001},"min":{"1min":0.29399999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.56200000000000006,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":6.8879999999999999}]},{"osd":1,"last update":"Fri Feb 23 00:41:54 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.39000000000000001,"15min":0.376},"min":{"1min":0.26800000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.53400000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.41399999999999998,"15min":0.39900000000000002},"min":{"1min":0.29699999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.48999999999999999,"5min":0.745,"15min":0.745},"last":0.47399999999999998}]},{"osd":3,"last update":"Fri Feb 23 00:42:26 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.51400000000000001,"15min":0.501},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":7.0289999999999999,"5min":7.0289999999999999,"15min":7.0289999999999999},"last":7.0289999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.46400000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.71899999999999997,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.49199999999999999}]},{"osd":4,"last update":"Fri Feb 23 00:41:54 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.46999999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.748},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.60599999999999998,"15min":0.58599999999999997},"min":{"1min":0.45500000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.76900000000000002}]},{"osd":5,"last update":"Fri Feb 23 00:42:18 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.59299999999999997,"15min":0.58699999999999997},"min":{"1min":0.41899999999999998,"5min":0.378,"15min":0.378},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.63600000000000001,"15min":0.621},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":7.0019999999999998}]},{"osd":6,"last update":"Fri Feb 23 00:41:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.49299999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":6.9390000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.61399999999999999,"15min":0.60199999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.66300000000000003}]},{"osd":7,"last update":"Fri Feb 23 00:42:08 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":6.9740000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.65300000000000002,"15min":0.65200000000000002},"min":{"1min":0.45600000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.69299999999999995}]}]},{"osd":3,"up_from":26,"seq":111669149760,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297840,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447312,"statfs":{"total":95995035648,"available":95690047488,"internally_reserved":0,"allocated":761856,"data_stored":394777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.39700000000000002,"15min":0.42899999999999999},"min":{"1min":0.23200000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.254},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42299999999999999,"15min":0.45000000000000001},"min":{"1min":0.29399999999999998,"5min":0.248,"15min":0.248},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.502}]},{"osd":1,"last update":"Fri Feb 23 00:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.432,"15min":0.46700000000000003},"min":{"1min":0.25900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.69499999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.438,"15min":0.45100000000000001},"min":{"1min":0.26200000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.42499999999999999}]},{"osd":2,"last update":"Fri Feb 23 00:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.48499999999999999,"15min":0.495},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":2.698,"5min":2.698,"15min":2.698},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.45400000000000001,"15min":0.47999999999999998},"min":{"1min":0.30399999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.46600000000000003}]},{"osd":4,"last update":"Fri Feb 23 00:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.45900000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89800000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58099999999999996,"15min":0.58999999999999997},"min":{"1min":0.38600000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.85499999999999998,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.64800000000000002}]},{"osd":5,"last update":"Fri Feb 23 00:42:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.59799999999999998,"15min":0.60499999999999998},"min":{"1min":0.41099999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.61599999999999999,"15min":0.624},"min":{"1min":0.45800000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.84099999999999997,"5min":1.024,"15min":1.024},"last":0.77200000000000002}]},{"osd":6,"last update":"Fri Feb 23 00:41:41 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.60699999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.76400000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.628,"15min":0.628},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.88,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.70399999999999996}]},{"osd":7,"last update":"Fri Feb 23 00:42:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.44,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64600000000000002,"15min":0.65400000000000003},"min":{"1min":0.49399999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.84399999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.81399999999999995}]}]},{"osd":4,"up_from":32,"seq":137438953532,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297824,"kb_used_data":728,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447328,"statfs":{"total":95995035648,"available":95690063872,"internally_reserved":0,"allocated":745472,"data_stored":394475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55000000000000004,"15min":0.57799999999999996},"min":{"1min":0.38400000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.75800000000000001,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55900000000000005,"15min":0.58099999999999996},"min":{"1min":0.40500000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68799999999999994,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":1.823}]},{"osd":1,"last update":"Fri Feb 23 00:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.55100000000000005,"15min":0.57899999999999996},"min":{"1min":0.40000000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77500000000000002,"5min":1.175,"15min":1.175},"last":0.433},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56200000000000006,"15min":0.57299999999999995},"min":{"1min":0.39700000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.67700000000000005,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.441}]},{"osd":2,"last update":"Fri Feb 23 00:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.40799999999999997,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76500000000000001,"5min":1.161,"15min":1.161},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57099999999999995,"15min":0.58699999999999997},"min":{"1min":0.42899999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.85999999999999999,"5min":1.113,"15min":1.113},"last":0.40600000000000003}]},{"osd":3,"last update":"Fri Feb 23 00:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.376,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73899999999999999,"5min":1.143,"15min":1.143},"last":2.9239999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57199999999999995,"15min":0.58199999999999996},"min":{"1min":0.42799999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.70799999999999996,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.53400000000000003}]},{"osd":5,"last update":"Fri Feb 23 00:42:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.42199999999999999,"15min":0.38},"min":{"1min":0.32500000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.47799999999999998,"15min":0.42699999999999999},"min":{"1min":0.36599999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.33100000000000002}]},{"osd":6,"last update":"Fri Feb 23 00:41:43 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.432,"15min":0.40699999999999997},"min":{"1min":0.29899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.435,"15min":0.40699999999999997},"min":{"1min":0.33500000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.39200000000000002}]},{"osd":7,"last update":"Fri Feb 23 00:42:11 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.50800000000000001,"15min":0.48999999999999999},"min":{"1min":0.33900000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.73999999999999999,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.56100000000000005,"15min":0.54300000000000004},"min":{"1min":0.38400000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.502}]}]},{"osd":5,"up_from":39,"seq":167503724599,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297828,"kb_used_data":732,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447324,"statfs":{"total":95995035648,"available":95690059776,"internally_reserved":0,"allocated":749568,"data_stored":395609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:42:17 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.50600000000000001,"15min":0.50800000000000001},"min":{"1min":0.315,"5min":0.313,"15min":0.313},"max":{"1min":0.67800000000000005,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.51800000000000002,"15min":0.52300000000000002},"min":{"1min":0.40600000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64900000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.49299999999999999}]},{"osd":1,"last update":"Fri Feb 23 00:42:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52500000000000002,"15min":0.52600000000000002},"min":{"1min":0.36499999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.64400000000000002,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.52600000000000002,"15min":0.53800000000000003},"min":{"1min":0.35199999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.61599999999999999,"5min":1.004,"15min":1.004},"last":0.39100000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:42:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54100000000000004,"15min":0.54800000000000004},"min":{"1min":0.371,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.72699999999999998,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.70999999999999996,"5min":1.026,"15min":1.026},"last":0.40899999999999997}]},{"osd":3,"last update":"Fri Feb 23 00:42:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52900000000000003,"15min":0.53100000000000003},"min":{"1min":0.39000000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.75700000000000001,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55700000000000005,"15min":0.55900000000000005},"min":{"1min":0.39300000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80000000000000004,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.57999999999999996}]},{"osd":4,"last update":"Fri Feb 23 00:42:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.41799999999999998,"15min":0.39000000000000001},"min":{"1min":0.34599999999999997,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.628,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.44400000000000001,"15min":0.40000000000000002},"min":{"1min":0.39700000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.52300000000000002}]},{"osd":6,"last update":"Fri Feb 23 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.44700000000000001,"15min":0.43099999999999999},"min":{"1min":0.29399999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.66700000000000004,"5min":0.754,"15min":0.754},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.46800000000000003,"15min":0.44800000000000001},"min":{"1min":0.35199999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68799999999999994,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.54300000000000004}]},{"osd":7,"last update":"Fri Feb 23 00:42:12 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.53300000000000003,"15min":0.51700000000000002},"min":{"1min":0.435,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72699999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.40500000000000003,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.69099999999999995,"5min":0.754,"15min":0.754},"last":0.56799999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-23T00:42:37.681 INFO:tasks.cephadm.ceph_manager.ceph:decrease pool size by 10 2024-02-23T00:42:37.681 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd pool set default.rgw.meta pg_num 22 2024-02-23T00:42:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:42:37.938 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:37.939 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:42:37.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:36.112527+0000 osd.4 (osd.4) 63 : cluster [DBG] 6.2 scrub starts 2024-02-23T00:42:37.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:36.113464+0000 osd.4 (osd.4) 64 : cluster [DBG] 6.2 scrub ok 2024-02-23T00:42:37.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:36.177274+0000 mgr.y (mgr.14517) 133 : audit [DBG] from='client.24688 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:42:37.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:36.453742+0000 osd.6 (osd.6) 49 : cluster [DBG] 5.2 scrub starts 2024-02-23T00:42:37.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:36.454651+0000 osd.6 (osd.6) 50 : cluster [DBG] 5.2 scrub ok 2024-02-23T00:42:37.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:37.179157+0000 osd.2 (osd.2) 61 : cluster [DBG] 6.f scrub starts 2024-02-23T00:42:37.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:37.180466+0000 osd.2 (osd.2) 62 : cluster [DBG] 6.f scrub ok 2024-02-23T00:42:38.040 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:42:38.145 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:38.169 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:42:38.172 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:38.173 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:42:38.221 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:42:38.273 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:42:38.274 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:42:38.274 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:42:38.309 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:38.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:37.168230+0000 mgr.y (mgr.14517) 134 : cluster [DBG] pgmap v121: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-23T00:42:38.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:37.947919+0000 osd.3 (osd.3) 73 : cluster [DBG] 2.1e scrub starts 2024-02-23T00:42:38.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:37.949037+0000 osd.3 (osd.3) 74 : cluster [DBG] 2.1e scrub ok 2024-02-23T00:42:38.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:37.168230+0000 mgr.y (mgr.14517) 134 : cluster [DBG] pgmap v121: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-23T00:42:38.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:37.947919+0000 osd.3 (osd.3) 73 : cluster [DBG] 2.1e scrub starts 2024-02-23T00:42:38.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:37.949037+0000 osd.3 (osd.3) 74 : cluster [DBG] 2.1e scrub ok 2024-02-23T00:42:38.662 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:42:38.666 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:38.667 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:42:38.768 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:42:38.797 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:37.168230+0000 mgr.y (mgr.14517) 134 : cluster [DBG] pgmap v121: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-23T00:42:38.798 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:37.947919+0000 osd.3 (osd.3) 73 : cluster [DBG] 2.1e scrub starts 2024-02-23T00:42:38.798 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:37.949037+0000 osd.3 (osd.3) 74 : cluster [DBG] 2.1e scrub ok 2024-02-23T00:42:38.800 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:38.822 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:42:38.825 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:38.826 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:42:38.926 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:42:38.957 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:38.977 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:42:38.980 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:38.981 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:42:39.082 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:42:39.110 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:39.130 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:42:39.133 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:39.133 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:42:39.234 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:42:39.262 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:39.282 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:42:39.284 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:39.285 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:42:39.385 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:42:39.418 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:39.599 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:42:39.709 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:38.203053+0000 osd.0 (osd.0) 57 : cluster [DBG] 2.1f scrub starts 2024-02-23T00:42:39.709 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:38.204278+0000 osd.0 (osd.0) 58 : cluster [DBG] 2.1f scrub ok 2024-02-23T00:42:39.709 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:38.203053+0000 osd.0 (osd.0) 57 : cluster [DBG] 2.1f scrub starts 2024-02-23T00:42:39.709 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:38.204278+0000 osd.0 (osd.0) 58 : cluster [DBG] 2.1f scrub ok 2024-02-23T00:42:39.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:38.203053+0000 osd.0 (osd.0) 57 : cluster [DBG] 2.1f scrub starts 2024-02-23T00:42:39.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:38.204278+0000 osd.0 (osd.0) 58 : cluster [DBG] 2.1f scrub ok 2024-02-23T00:42:40.405 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.1 dump_blocked_ops 2024-02-23T00:42:40.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:39.168629+0000 mgr.y (mgr.14517) 135 : cluster [DBG] pgmap v122: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-23T00:42:40.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:39.937078+0000 osd.3 (osd.3) 75 : cluster [DBG] 4.19 scrub starts 2024-02-23T00:42:40.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:39.939820+0000 osd.3 (osd.3) 76 : cluster [DBG] 4.19 scrub ok 2024-02-23T00:42:40.808 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:42:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:42:40] "GET /metrics HTTP/1.1" 200 172263 "" "Prometheus/2.33.4" 2024-02-23T00:42:40.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:39.168629+0000 mgr.y (mgr.14517) 135 : cluster [DBG] pgmap v122: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-23T00:42:40.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:39.937078+0000 osd.3 (osd.3) 75 : cluster [DBG] 4.19 scrub starts 2024-02-23T00:42:40.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:39.939820+0000 osd.3 (osd.3) 76 : cluster [DBG] 4.19 scrub ok 2024-02-23T00:42:40.872 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:42:40.875 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:40.875 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:42:40.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:39.168629+0000 mgr.y (mgr.14517) 135 : cluster [DBG] pgmap v122: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-23T00:42:40.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:39.937078+0000 osd.3 (osd.3) 75 : cluster [DBG] 4.19 scrub starts 2024-02-23T00:42:40.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:39.939820+0000 osd.3 (osd.3) 76 : cluster [DBG] 4.19 scrub ok 2024-02-23T00:42:40.976 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:42:41.008 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:41.030 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:42:41.032 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:41.033 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:42:41.134 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:42:41.173 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:41.287 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:42:41.291 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:41.291 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:42:41.392 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:42:41.423 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:41.528 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:42:41.531 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:41.532 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:42:41.632 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:42:41.664 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:41.684 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:42:41.687 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:41.688 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:42:41.789 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:42:41.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:40.043687+0000 osd.7 (osd.7) 61 : cluster [DBG] 2.1c scrub starts 2024-02-23T00:42:41.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:40.046643+0000 osd.7 (osd.7) 62 : cluster [DBG] 2.1c scrub ok 2024-02-23T00:42:41.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:40.533033+0000 osd.1 (osd.1) 69 : cluster [DBG] 2.6 scrub starts 2024-02-23T00:42:41.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:40.534278+0000 osd.1 (osd.1) 70 : cluster [DBG] 2.6 scrub ok 2024-02-23T00:42:41.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:40.869484+0000 osd.5 (osd.5) 65 : cluster [DBG] 5.15 scrub starts 2024-02-23T00:42:41.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:40.872872+0000 osd.5 (osd.5) 66 : cluster [DBG] 5.15 scrub ok 2024-02-23T00:42:41.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:40.894796+0000 osd.3 (osd.3) 77 : cluster [DBG] 5.13 scrub starts 2024-02-23T00:42:41.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:40.896240+0000 osd.3 (osd.3) 78 : cluster [DBG] 5.13 scrub ok 2024-02-23T00:42:41.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:40.043687+0000 osd.7 (osd.7) 61 : cluster [DBG] 2.1c scrub starts 2024-02-23T00:42:41.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:40.046643+0000 osd.7 (osd.7) 62 : cluster [DBG] 2.1c scrub ok 2024-02-23T00:42:41.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:40.533033+0000 osd.1 (osd.1) 69 : cluster [DBG] 2.6 scrub starts 2024-02-23T00:42:41.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:40.534278+0000 osd.1 (osd.1) 70 : cluster [DBG] 2.6 scrub ok 2024-02-23T00:42:41.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:40.869484+0000 osd.5 (osd.5) 65 : cluster [DBG] 5.15 scrub starts 2024-02-23T00:42:41.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:40.872872+0000 osd.5 (osd.5) 66 : cluster [DBG] 5.15 scrub ok 2024-02-23T00:42:41.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:40.894796+0000 osd.3 (osd.3) 77 : cluster [DBG] 5.13 scrub starts 2024-02-23T00:42:41.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:40.896240+0000 osd.3 (osd.3) 78 : cluster [DBG] 5.13 scrub ok 2024-02-23T00:42:41.820 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:41.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.835+0000 7fd09d6ee700 1 -- 172.21.15.47:0/1745858839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0980f9610 msgr2=0x7fd098101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:41.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.835+0000 7fd09d6ee700 1 --2- 172.21.15.47:0/1745858839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0980f9610 0x7fd098101d20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7fd08c004870 tx=0x7fd08c009cb0).stop 2024-02-23T00:42:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.835+0000 7fd09d6ee700 1 -- 172.21.15.47:0/1745858839 shutdown_connections 2024-02-23T00:42:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.835+0000 7fd09d6ee700 1 --2- 172.21.15.47:0/1745858839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd098102210 0x7fd098104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.835+0000 7fd09d6ee700 1 --2- 172.21.15.47:0/1745858839 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd0980f8cb0 0x7fd0980f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.835+0000 7fd09d6ee700 1 --2- 172.21.15.47:0/1745858839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0980f9610 0x7fd098101d20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.835+0000 7fd09d6ee700 1 -- 172.21.15.47:0/1745858839 >> 172.21.15.47:0/1745858839 conn(0x7fd0980f2820 msgr2=0x7fd0980f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:42:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.836+0000 7fd09d6ee700 1 -- 172.21.15.47:0/1745858839 shutdown_connections 2024-02-23T00:42:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.836+0000 7fd09d6ee700 1 -- 172.21.15.47:0/1745858839 wait complete. 2024-02-23T00:42:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.836+0000 7fd09d6ee700 1 librados: starting msgr at 2024-02-23T00:42:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.836+0000 7fd09d6ee700 1 librados: starting objecter 2024-02-23T00:42:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.836+0000 7fd09d6ee700 1 Processor -- start 2024-02-23T00:42:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.837+0000 7fd09d6ee700 1 -- start start 2024-02-23T00:42:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.837+0000 7fd09d6ee700 1 librados: setting wanted keys 2024-02-23T00:42:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.837+0000 7fd09d6ee700 1 librados: calling monclient init 2024-02-23T00:42:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.837+0000 7fd09d6ee700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd0980f8cb0 0x7fd0980f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.837+0000 7fd09d6ee700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0980f9610 0x7fd0980f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.837+0000 7fd09d6ee700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd098102210 0x7fd0980f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.837+0000 7fd09d6ee700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd098116590 con 0x7fd0980f9610 2024-02-23T00:42:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.837+0000 7fd09d6ee700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd098116700 con 0x7fd098102210 2024-02-23T00:42:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.837+0000 7fd09d6ee700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0981168c0 con 0x7fd0980f8cb0 2024-02-23T00:42:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.837+0000 7fd0967fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0980f9610 0x7fd0980f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.837+0000 7fd0967fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0980f9610 0x7fd0980f7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37506/0 (socket says 172.21.15.47:37506) 2024-02-23T00:42:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.837+0000 7fd0967fc700 1 -- 172.21.15.47:0/2181864249 learned_addr learned my addr 172.21.15.47:0/2181864249 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:42:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.837+0000 7fd0977fe700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd098102210 0x7fd0980f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.838+0000 7fd096ffd700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd0980f8cb0 0x7fd0980f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.838+0000 7fd096ffd700 1 -- 172.21.15.47:0/2181864249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd098102210 msgr2=0x7fd0980f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.838+0000 7fd096ffd700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd098102210 0x7fd0980f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.838+0000 7fd096ffd700 1 -- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0980f9610 msgr2=0x7fd0980f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.838+0000 7fd096ffd700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0980f9610 0x7fd0980f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.838+0000 7fd096ffd700 1 -- 172.21.15.47:0/2181864249 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd088005040 con 0x7fd0980f8cb0 2024-02-23T00:42:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.838+0000 7fd0967fc700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0980f9610 0x7fd0980f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:42:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.838+0000 7fd096ffd700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd0980f8cb0 0x7fd0980f6c60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fd088004920 tx=0x7fd088009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.838+0000 7fd07ffff700 10 client.?.objecter ms_handle_connect 0x7fd0980f8cb0 2024-02-23T00:42:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.838+0000 7fd07ffff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:42:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.838+0000 7fd07ffff700 1 -- 172.21.15.47:0/2181864249 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd088007a40 con 0x7fd0980f8cb0 2024-02-23T00:42:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.838+0000 7fd07ffff700 1 -- 172.21.15.47:0/2181864249 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fd08800f040 con 0x7fd0980f8cb0 2024-02-23T00:42:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.838+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd08c005040 con 0x7fd0980f8cb0 2024-02-23T00:42:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.838+0000 7fd09d6ee700 10 client.24695.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:42:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.838+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd098116f50 con 0x7fd0980f8cb0 2024-02-23T00:42:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.838+0000 7fd09d6ee700 1 librados: init done 2024-02-23T00:42:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:42:41.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.838+0000 7fd07ffff700 1 -- 172.21.15.47:0/2181864249 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd088014900 con 0x7fd0980f8cb0 2024-02-23T00:42:41.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.839+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fd084004f30 con 0x7fd0980f8cb0 2024-02-23T00:42:41.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.841+0000 7fd07ffff700 1 -- 172.21.15.47:0/2181864249 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fd088014a60 con 0x7fd0980f8cb0 2024-02-23T00:42:41.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.841+0000 7fd07ffff700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd080062cc0 0x7fd080065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:41.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.842+0000 7fd07ffff700 1 -- 172.21.15.47:0/2181864249 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(103..103 src has 48..103) v4 ==== 7453+0+0 (secure 0 0 0) 0x7fd08807d4f0 con 0x7fd0980f8cb0 2024-02-23T00:42:41.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.842+0000 7fd07ffff700 10 client.24695.objecter ms_dispatch 0x7fd098001550 osd_map(103..103 src has 48..103) v4 2024-02-23T00:42:41.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.842+0000 7fd07ffff700 3 client.24695.objecter handle_osd_map got epochs [103,103] > 0 2024-02-23T00:42:41.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.842+0000 7fd07ffff700 3 client.24695.objecter handle_osd_map decoding full epoch 103 2024-02-23T00:42:41.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.842+0000 7fd0967fc700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd080062cc0 0x7fd080065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:41.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.842+0000 7fd07ffff700 20 client.24695.objecter dump_active .. 0 homeless 2024-02-23T00:42:41.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.842+0000 7fd07ffff700 1 -- 172.21.15.47:0/2181864249 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v103) v1 ==== 54+0+16 (secure 0 0 0) 0x7fd08807d910 con 0x7fd0980f8cb0 2024-02-23T00:42:41.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.842+0000 7fd07ffff700 10 client.24695.objecter ms_handle_connect 0x7fd080062cc0 2024-02-23T00:42:41.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.842+0000 7fd0967fc700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd080062cc0 0x7fd080065180 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fd08c007fd0 tx=0x7fd08c009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:41.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.842+0000 7fd09d6ee700 10 client.24695.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:41.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.842+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_assign -1 1 2024-02-23T00:42:41.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.842+0000 7fd09d6ee700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fd084004560 0x7fd084004980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:41.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.842+0000 7fd09d6ee700 20 client.24695.objecter _get_session s=0x7fd0840051d0 osd=0 3 2024-02-23T00:42:41.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.842+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd0840051d0 osd=0 3 2024-02-23T00:42:41.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.842+0000 7fd09d6ee700 20 client.24695.objecter _get_session s=0x7fd0840051d0 osd=0 3 2024-02-23T00:42:41.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.842+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_remove -1 1 2024-02-23T00:42:41.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.842+0000 7fd09d6ee700 20 client.24695.objecter get_session s=0x7fd0840051d0 osd=0 3 2024-02-23T00:42:41.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.842+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_assign 0 1 2024-02-23T00:42:41.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.842+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd0840051d0 osd=0 4 2024-02-23T00:42:41.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.842+0000 7fd09d6ee700 10 client.24695.objecter _send_command 1 2024-02-23T00:42:41.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.842+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd084008960 con 0x7fd084004560 2024-02-23T00:42:41.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.842+0000 7fd0977fe700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fd084004560 0x7fd084004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:41.847 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:41.847 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:42:41.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.843+0000 7fd07ffff700 10 client.24695.objecter ms_handle_connect 0x7fd084004560 2024-02-23T00:42:41.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.843+0000 7fd0977fe700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fd084004560 0x7fd084004980 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:41.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.843+0000 7fd07ffff700 1 -- 172.21.15.47:0/2181864249 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd084008960 con 0x7fd084004560 2024-02-23T00:42:41.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.843+0000 7fd07ffff700 10 client.24695.objecter ms_dispatch 0x7fd098001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:42:41.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.843+0000 7fd07ffff700 10 client.24695.objecter _finish_command 1 = system:0 2024-02-23T00:42:41.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.843+0000 7fd07ffff700 20 client.24695.objecter put_session s=0x7fd0840051d0 osd=0 4 2024-02-23T00:42:41.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.843+0000 7fd07ffff700 15 client.24695.objecter _session_command_op_remove 0 1 2024-02-23T00:42:41.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.858+0000 7fd09d6ee700 10 client.24695.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:42:41.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.858+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_assign -1 2 2024-02-23T00:42:41.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.858+0000 7fd09d6ee700 20 client.24695.objecter _get_session s=0x7fd0840051d0 osd=0 3 2024-02-23T00:42:41.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.858+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd0840051d0 osd=0 3 2024-02-23T00:42:41.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.858+0000 7fd09d6ee700 20 client.24695.objecter _get_session s=0x7fd0840051d0 osd=0 3 2024-02-23T00:42:41.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.858+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_remove -1 2 2024-02-23T00:42:41.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.858+0000 7fd09d6ee700 20 client.24695.objecter get_session s=0x7fd0840051d0 osd=0 3 2024-02-23T00:42:41.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.858+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_assign 0 2 2024-02-23T00:42:41.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.858+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd0840051d0 osd=0 4 2024-02-23T00:42:41.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.858+0000 7fd09d6ee700 10 client.24695.objecter _send_command 2 2024-02-23T00:42:41.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.858+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd084003fd0 con 0x7fd084004560 2024-02-23T00:42:41.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.860+0000 7fd07ffff700 1 -- 172.21.15.47:0/2181864249 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd084003fd0 con 0x7fd084004560 2024-02-23T00:42:41.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.860+0000 7fd07ffff700 10 client.24695.objecter ms_dispatch 0x7fd098001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:42:41.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.860+0000 7fd07ffff700 10 client.24695.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:42:41.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.860+0000 7fd07ffff700 20 client.24695.objecter put_session s=0x7fd0840051d0 osd=0 4 2024-02-23T00:42:41.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.860+0000 7fd07ffff700 15 client.24695.objecter _session_command_op_remove 0 2 2024-02-23T00:42:41.863 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:42:41.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.860+0000 7fd09d6ee700 10 client.24695.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.860+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_assign -1 3 2024-02-23T00:42:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.860+0000 7fd09d6ee700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fd084008510 0x7fd084002760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.860+0000 7fd09d6ee700 20 client.24695.objecter _get_session s=0x7fd084004380 osd=1 3 2024-02-23T00:42:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.860+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd084004380 osd=1 3 2024-02-23T00:42:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.860+0000 7fd09d6ee700 20 client.24695.objecter _get_session s=0x7fd084004380 osd=1 3 2024-02-23T00:42:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.860+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_remove -1 3 2024-02-23T00:42:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.860+0000 7fd09d6ee700 20 client.24695.objecter get_session s=0x7fd084004380 osd=1 3 2024-02-23T00:42:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.860+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_assign 1 3 2024-02-23T00:42:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.860+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd084004380 osd=1 4 2024-02-23T00:42:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.860+0000 7fd09d6ee700 10 client.24695.objecter _send_command 3 2024-02-23T00:42:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.860+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd084002e40 con 0x7fd084008510 2024-02-23T00:42:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.860+0000 7fd096ffd700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fd084008510 0x7fd084002760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.861+0000 7fd07ffff700 10 client.24695.objecter ms_handle_connect 0x7fd084008510 2024-02-23T00:42:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.861+0000 7fd096ffd700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fd084008510 0x7fd084002760 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.861+0000 7fd07ffff700 1 -- 172.21.15.47:0/2181864249 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd084002e40 con 0x7fd084008510 2024-02-23T00:42:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.861+0000 7fd07ffff700 10 client.24695.objecter ms_dispatch 0x7fd098001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:42:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.861+0000 7fd07ffff700 10 client.24695.objecter _finish_command 3 = system:0 2024-02-23T00:42:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.861+0000 7fd07ffff700 20 client.24695.objecter put_session s=0x7fd084004380 osd=1 4 2024-02-23T00:42:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.861+0000 7fd07ffff700 15 client.24695.objecter _session_command_op_remove 1 3 2024-02-23T00:42:41.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.876+0000 7fd09d6ee700 10 client.24695.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:42:41.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.876+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_assign -1 4 2024-02-23T00:42:41.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.876+0000 7fd09d6ee700 20 client.24695.objecter _get_session s=0x7fd084004380 osd=1 3 2024-02-23T00:42:41.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.876+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd084004380 osd=1 3 2024-02-23T00:42:41.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.876+0000 7fd09d6ee700 20 client.24695.objecter _get_session s=0x7fd084004380 osd=1 3 2024-02-23T00:42:41.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.876+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_remove -1 4 2024-02-23T00:42:41.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.876+0000 7fd09d6ee700 20 client.24695.objecter get_session s=0x7fd084004380 osd=1 3 2024-02-23T00:42:41.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.876+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_assign 1 4 2024-02-23T00:42:41.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.876+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd084004380 osd=1 4 2024-02-23T00:42:41.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.876+0000 7fd09d6ee700 10 client.24695.objecter _send_command 4 2024-02-23T00:42:41.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.876+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd084007a80 con 0x7fd084008510 2024-02-23T00:42:41.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.878+0000 7fd07ffff700 1 -- 172.21.15.47:0/2181864249 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd084007a80 con 0x7fd084008510 2024-02-23T00:42:41.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.878+0000 7fd07ffff700 10 client.24695.objecter ms_dispatch 0x7fd098001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:42:41.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.878+0000 7fd07ffff700 10 client.24695.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:42:41.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.878+0000 7fd07ffff700 20 client.24695.objecter put_session s=0x7fd084004380 osd=1 4 2024-02-23T00:42:41.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.878+0000 7fd07ffff700 15 client.24695.objecter _session_command_op_remove 1 4 2024-02-23T00:42:41.881 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:42:41.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.878+0000 7fd09d6ee700 10 client.24695.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:41.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.878+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_assign -1 5 2024-02-23T00:42:41.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.878+0000 7fd09d6ee700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fd084003950 0x7fd08400d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:41.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.878+0000 7fd09d6ee700 20 client.24695.objecter _get_session s=0x7fd0840037e0 osd=2 3 2024-02-23T00:42:41.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.878+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd0840037e0 osd=2 3 2024-02-23T00:42:41.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.878+0000 7fd09d6ee700 20 client.24695.objecter _get_session s=0x7fd0840037e0 osd=2 3 2024-02-23T00:42:41.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.878+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_remove -1 5 2024-02-23T00:42:41.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.878+0000 7fd09d6ee700 20 client.24695.objecter get_session s=0x7fd0840037e0 osd=2 3 2024-02-23T00:42:41.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.878+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_assign 2 5 2024-02-23T00:42:41.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.878+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd0840037e0 osd=2 4 2024-02-23T00:42:41.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.878+0000 7fd09d6ee700 10 client.24695.objecter _send_command 5 2024-02-23T00:42:41.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.878+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd08400ff40 con 0x7fd084003950 2024-02-23T00:42:41.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.878+0000 7fd0967fc700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fd084003950 0x7fd08400d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:41.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.879+0000 7fd07ffff700 10 client.24695.objecter ms_handle_connect 0x7fd084003950 2024-02-23T00:42:41.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.879+0000 7fd0967fc700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fd084003950 0x7fd08400d040 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:41.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.879+0000 7fd07ffff700 1 -- 172.21.15.47:0/2181864249 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd08400ff40 con 0x7fd084003950 2024-02-23T00:42:41.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.879+0000 7fd07ffff700 10 client.24695.objecter ms_dispatch 0x7fd098001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:42:41.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.879+0000 7fd07ffff700 10 client.24695.objecter _finish_command 5 = system:0 2024-02-23T00:42:41.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.879+0000 7fd07ffff700 20 client.24695.objecter put_session s=0x7fd0840037e0 osd=2 4 2024-02-23T00:42:41.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.879+0000 7fd07ffff700 15 client.24695.objecter _session_command_op_remove 2 5 2024-02-23T00:42:41.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.894+0000 7fd09d6ee700 10 client.24695.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:42:41.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.894+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_assign -1 6 2024-02-23T00:42:41.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.894+0000 7fd09d6ee700 20 client.24695.objecter _get_session s=0x7fd0840037e0 osd=2 3 2024-02-23T00:42:41.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.894+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd0840037e0 osd=2 3 2024-02-23T00:42:41.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.894+0000 7fd09d6ee700 20 client.24695.objecter _get_session s=0x7fd0840037e0 osd=2 3 2024-02-23T00:42:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.894+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_remove -1 6 2024-02-23T00:42:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.894+0000 7fd09d6ee700 20 client.24695.objecter get_session s=0x7fd0840037e0 osd=2 3 2024-02-23T00:42:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.894+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_assign 2 6 2024-02-23T00:42:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.894+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd0840037e0 osd=2 4 2024-02-23T00:42:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.894+0000 7fd09d6ee700 10 client.24695.objecter _send_command 6 2024-02-23T00:42:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.894+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd084003250 con 0x7fd084003950 2024-02-23T00:42:41.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.896+0000 7fd07ffff700 1 -- 172.21.15.47:0/2181864249 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd084003250 con 0x7fd084003950 2024-02-23T00:42:41.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.896+0000 7fd07ffff700 10 client.24695.objecter ms_dispatch 0x7fd098001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:42:41.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.896+0000 7fd07ffff700 10 client.24695.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:42:41.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.896+0000 7fd07ffff700 20 client.24695.objecter put_session s=0x7fd0840037e0 osd=2 4 2024-02-23T00:42:41.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.896+0000 7fd07ffff700 15 client.24695.objecter _session_command_op_remove 2 6 2024-02-23T00:42:41.898 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:42:41.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.896+0000 7fd09d6ee700 10 client.24695.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:41.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.896+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_assign -1 7 2024-02-23T00:42:41.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.896+0000 7fd09d6ee700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fd08400e7f0 0x7fd08400ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:41.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.896+0000 7fd09d6ee700 20 client.24695.objecter _get_session s=0x7fd084001b70 osd=3 3 2024-02-23T00:42:41.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.896+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd084001b70 osd=3 3 2024-02-23T00:42:41.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.896+0000 7fd09d6ee700 20 client.24695.objecter _get_session s=0x7fd084001b70 osd=3 3 2024-02-23T00:42:41.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.896+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_remove -1 7 2024-02-23T00:42:41.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.896+0000 7fd09d6ee700 20 client.24695.objecter get_session s=0x7fd084001b70 osd=3 3 2024-02-23T00:42:41.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.896+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_assign 3 7 2024-02-23T00:42:41.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.896+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd084001b70 osd=3 4 2024-02-23T00:42:41.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.896+0000 7fd09d6ee700 10 client.24695.objecter _send_command 7 2024-02-23T00:42:41.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.896+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd0840022f0 con 0x7fd08400e7f0 2024-02-23T00:42:41.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.896+0000 7fd0977fe700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fd08400e7f0 0x7fd08400ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:41.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.896+0000 7fd0977fe700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fd08400e7f0 0x7fd08400ec70 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:41.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.896+0000 7fd07ffff700 10 client.24695.objecter ms_handle_connect 0x7fd08400e7f0 2024-02-23T00:42:41.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.897+0000 7fd07ffff700 1 -- 172.21.15.47:0/2181864249 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd0840022f0 con 0x7fd08400e7f0 2024-02-23T00:42:41.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.897+0000 7fd07ffff700 10 client.24695.objecter ms_dispatch 0x7fd098001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:42:41.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.897+0000 7fd07ffff700 10 client.24695.objecter _finish_command 7 = system:0 2024-02-23T00:42:41.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.897+0000 7fd07ffff700 20 client.24695.objecter put_session s=0x7fd084001b70 osd=3 4 2024-02-23T00:42:41.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.897+0000 7fd07ffff700 15 client.24695.objecter _session_command_op_remove 3 7 2024-02-23T00:42:41.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.911+0000 7fd09d6ee700 10 client.24695.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:42:41.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.911+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_assign -1 8 2024-02-23T00:42:41.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.911+0000 7fd09d6ee700 20 client.24695.objecter _get_session s=0x7fd084001b70 osd=3 3 2024-02-23T00:42:41.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.911+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd084001b70 osd=3 3 2024-02-23T00:42:41.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.911+0000 7fd09d6ee700 20 client.24695.objecter _get_session s=0x7fd084001b70 osd=3 3 2024-02-23T00:42:41.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.912+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_remove -1 8 2024-02-23T00:42:41.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.912+0000 7fd09d6ee700 20 client.24695.objecter get_session s=0x7fd084001b70 osd=3 3 2024-02-23T00:42:41.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.912+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_assign 3 8 2024-02-23T00:42:41.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.912+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd084001b70 osd=3 4 2024-02-23T00:42:41.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.912+0000 7fd09d6ee700 10 client.24695.objecter _send_command 8 2024-02-23T00:42:41.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.912+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd08400f930 con 0x7fd08400e7f0 2024-02-23T00:42:41.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.913+0000 7fd07ffff700 1 -- 172.21.15.47:0/2181864249 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd08400f930 con 0x7fd08400e7f0 2024-02-23T00:42:41.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.913+0000 7fd07ffff700 10 client.24695.objecter ms_dispatch 0x7fd098001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:42:41.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.913+0000 7fd07ffff700 10 client.24695.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:42:41.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.913+0000 7fd07ffff700 20 client.24695.objecter put_session s=0x7fd084001b70 osd=3 4 2024-02-23T00:42:41.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.913+0000 7fd07ffff700 15 client.24695.objecter _session_command_op_remove 3 8 2024-02-23T00:42:41.916 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:42:41.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.913+0000 7fd09d6ee700 10 client.24695.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:41.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.913+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_assign -1 9 2024-02-23T00:42:41.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.913+0000 7fd09d6ee700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fd084012170 0x7fd084014560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:41.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.913+0000 7fd09d6ee700 20 client.24695.objecter _get_session s=0x7fd08400f160 osd=4 3 2024-02-23T00:42:41.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.913+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd08400f160 osd=4 3 2024-02-23T00:42:41.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.913+0000 7fd09d6ee700 20 client.24695.objecter _get_session s=0x7fd08400f160 osd=4 3 2024-02-23T00:42:41.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.913+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_remove -1 9 2024-02-23T00:42:41.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.913+0000 7fd09d6ee700 20 client.24695.objecter get_session s=0x7fd08400f160 osd=4 3 2024-02-23T00:42:41.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.913+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_assign 4 9 2024-02-23T00:42:41.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.913+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd08400f160 osd=4 4 2024-02-23T00:42:41.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.913+0000 7fd09d6ee700 10 client.24695.objecter _send_command 9 2024-02-23T00:42:41.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.913+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd08400f930 con 0x7fd084012170 2024-02-23T00:42:41.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.914+0000 7fd096ffd700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fd084012170 0x7fd084014560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:41.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.914+0000 7fd07ffff700 10 client.24695.objecter ms_handle_connect 0x7fd084012170 2024-02-23T00:42:41.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.914+0000 7fd096ffd700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fd084012170 0x7fd084014560 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:41.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.915+0000 7fd07ffff700 1 -- 172.21.15.47:0/2181864249 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd08400f930 con 0x7fd084012170 2024-02-23T00:42:41.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.915+0000 7fd07ffff700 10 client.24695.objecter ms_dispatch 0x7fd098001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:42:41.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.915+0000 7fd07ffff700 10 client.24695.objecter _finish_command 9 = system:0 2024-02-23T00:42:41.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.915+0000 7fd07ffff700 20 client.24695.objecter put_session s=0x7fd08400f160 osd=4 4 2024-02-23T00:42:41.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.915+0000 7fd07ffff700 15 client.24695.objecter _session_command_op_remove 4 9 2024-02-23T00:42:41.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.930+0000 7fd09d6ee700 10 client.24695.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:42:41.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.930+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_assign -1 10 2024-02-23T00:42:41.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.930+0000 7fd09d6ee700 20 client.24695.objecter _get_session s=0x7fd08400f160 osd=4 3 2024-02-23T00:42:41.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.930+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd08400f160 osd=4 3 2024-02-23T00:42:41.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.930+0000 7fd09d6ee700 20 client.24695.objecter _get_session s=0x7fd08400f160 osd=4 3 2024-02-23T00:42:41.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.930+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_remove -1 10 2024-02-23T00:42:41.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.930+0000 7fd09d6ee700 20 client.24695.objecter get_session s=0x7fd08400f160 osd=4 3 2024-02-23T00:42:41.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.930+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_assign 4 10 2024-02-23T00:42:41.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.930+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd08400f160 osd=4 4 2024-02-23T00:42:41.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.930+0000 7fd09d6ee700 10 client.24695.objecter _send_command 10 2024-02-23T00:42:41.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.930+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd08400f930 con 0x7fd084012170 2024-02-23T00:42:41.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.932+0000 7fd07ffff700 1 -- 172.21.15.47:0/2181864249 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd08400f930 con 0x7fd084012170 2024-02-23T00:42:41.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.932+0000 7fd07ffff700 10 client.24695.objecter ms_dispatch 0x7fd098001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:42:41.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.932+0000 7fd07ffff700 10 client.24695.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:42:41.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.932+0000 7fd07ffff700 20 client.24695.objecter put_session s=0x7fd08400f160 osd=4 4 2024-02-23T00:42:41.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.932+0000 7fd07ffff700 15 client.24695.objecter _session_command_op_remove 4 10 2024-02-23T00:42:41.935 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:42:41.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.933+0000 7fd09d6ee700 10 client.24695.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:41.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.933+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_assign -1 11 2024-02-23T00:42:41.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.933+0000 7fd09d6ee700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fd084014fb0 0x7fd084015410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:41.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.933+0000 7fd09d6ee700 20 client.24695.objecter _get_session s=0x7fd08400f930 osd=5 3 2024-02-23T00:42:41.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.933+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd08400f930 osd=5 3 2024-02-23T00:42:41.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.933+0000 7fd09d6ee700 20 client.24695.objecter _get_session s=0x7fd08400f930 osd=5 3 2024-02-23T00:42:41.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.933+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_remove -1 11 2024-02-23T00:42:41.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.933+0000 7fd09d6ee700 20 client.24695.objecter get_session s=0x7fd08400f930 osd=5 3 2024-02-23T00:42:41.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.933+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_assign 5 11 2024-02-23T00:42:41.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.933+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd08400f930 osd=5 4 2024-02-23T00:42:41.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.933+0000 7fd09d6ee700 10 client.24695.objecter _send_command 11 2024-02-23T00:42:41.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.933+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd08401a3a0 con 0x7fd084014fb0 2024-02-23T00:42:41.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.933+0000 7fd0967fc700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fd084014fb0 0x7fd084015410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:41.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.933+0000 7fd07ffff700 10 client.24695.objecter ms_handle_connect 0x7fd084014fb0 2024-02-23T00:42:41.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.933+0000 7fd0967fc700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fd084014fb0 0x7fd084015410 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:41.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.934+0000 7fd07ffff700 1 -- 172.21.15.47:0/2181864249 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd08401a3a0 con 0x7fd084014fb0 2024-02-23T00:42:41.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.934+0000 7fd07ffff700 10 client.24695.objecter ms_dispatch 0x7fd098001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:42:41.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.934+0000 7fd07ffff700 10 client.24695.objecter _finish_command 11 = system:0 2024-02-23T00:42:41.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.934+0000 7fd07ffff700 20 client.24695.objecter put_session s=0x7fd08400f930 osd=5 4 2024-02-23T00:42:41.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.934+0000 7fd07ffff700 15 client.24695.objecter _session_command_op_remove 5 11 2024-02-23T00:42:41.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:40.043687+0000 osd.7 (osd.7) 61 : cluster [DBG] 2.1c scrub starts 2024-02-23T00:42:41.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:40.046643+0000 osd.7 (osd.7) 62 : cluster [DBG] 2.1c scrub ok 2024-02-23T00:42:41.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:40.533033+0000 osd.1 (osd.1) 69 : cluster [DBG] 2.6 scrub starts 2024-02-23T00:42:41.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:40.534278+0000 osd.1 (osd.1) 70 : cluster [DBG] 2.6 scrub ok 2024-02-23T00:42:41.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:40.869484+0000 osd.5 (osd.5) 65 : cluster [DBG] 5.15 scrub starts 2024-02-23T00:42:41.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:40.872872+0000 osd.5 (osd.5) 66 : cluster [DBG] 5.15 scrub ok 2024-02-23T00:42:41.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:40.894796+0000 osd.3 (osd.3) 77 : cluster [DBG] 5.13 scrub starts 2024-02-23T00:42:41.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:40.896240+0000 osd.3 (osd.3) 78 : cluster [DBG] 5.13 scrub ok 2024-02-23T00:42:41.948 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:42:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.949+0000 7fd09d6ee700 10 client.24695.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:42:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.949+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_assign -1 12 2024-02-23T00:42:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.949+0000 7fd09d6ee700 20 client.24695.objecter _get_session s=0x7fd08400f930 osd=5 3 2024-02-23T00:42:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.949+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd08400f930 osd=5 3 2024-02-23T00:42:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.949+0000 7fd09d6ee700 20 client.24695.objecter _get_session s=0x7fd08400f930 osd=5 3 2024-02-23T00:42:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.949+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_remove -1 12 2024-02-23T00:42:41.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.949+0000 7fd09d6ee700 20 client.24695.objecter get_session s=0x7fd08400f930 osd=5 3 2024-02-23T00:42:41.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.949+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_assign 5 12 2024-02-23T00:42:41.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.949+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd08400f930 osd=5 4 2024-02-23T00:42:41.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.949+0000 7fd09d6ee700 10 client.24695.objecter _send_command 12 2024-02-23T00:42:41.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.949+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd084015a10 con 0x7fd084014fb0 2024-02-23T00:42:41.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.951+0000 7fd07ffff700 1 -- 172.21.15.47:0/2181864249 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd084015a10 con 0x7fd084014fb0 2024-02-23T00:42:41.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.951+0000 7fd07ffff700 10 client.24695.objecter ms_dispatch 0x7fd098001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:42:41.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.951+0000 7fd07ffff700 10 client.24695.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:42:41.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.951+0000 7fd07ffff700 20 client.24695.objecter put_session s=0x7fd08400f930 osd=5 4 2024-02-23T00:42:41.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.951+0000 7fd07ffff700 15 client.24695.objecter _session_command_op_remove 5 12 2024-02-23T00:42:41.954 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:42:41.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.951+0000 7fd09d6ee700 10 client.24695.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:41.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.951+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_assign -1 13 2024-02-23T00:42:41.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.951+0000 7fd09d6ee700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fd0840160f0 0x7fd084016550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:41.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.951+0000 7fd09d6ee700 20 client.24695.objecter _get_session s=0x7fd084015a10 osd=6 3 2024-02-23T00:42:41.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.951+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd084015a10 osd=6 3 2024-02-23T00:42:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.951+0000 7fd09d6ee700 20 client.24695.objecter _get_session s=0x7fd084015a10 osd=6 3 2024-02-23T00:42:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.951+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_remove -1 13 2024-02-23T00:42:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.951+0000 7fd09d6ee700 20 client.24695.objecter get_session s=0x7fd084015a10 osd=6 3 2024-02-23T00:42:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.951+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_assign 6 13 2024-02-23T00:42:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.951+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd084015a10 osd=6 4 2024-02-23T00:42:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.951+0000 7fd09d6ee700 10 client.24695.objecter _send_command 13 2024-02-23T00:42:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.951+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd084016ba0 con 0x7fd0840160f0 2024-02-23T00:42:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.952+0000 7fd0977fe700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fd0840160f0 0x7fd084016550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.952+0000 7fd07ffff700 10 client.24695.objecter ms_handle_connect 0x7fd0840160f0 2024-02-23T00:42:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.952+0000 7fd0977fe700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fd0840160f0 0x7fd084016550 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.953+0000 7fd07ffff700 1 -- 172.21.15.47:0/2181864249 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd084016ba0 con 0x7fd0840160f0 2024-02-23T00:42:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.953+0000 7fd07ffff700 10 client.24695.objecter ms_dispatch 0x7fd098001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:42:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.953+0000 7fd07ffff700 10 client.24695.objecter _finish_command 13 = system:0 2024-02-23T00:42:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.953+0000 7fd07ffff700 20 client.24695.objecter put_session s=0x7fd084015a10 osd=6 4 2024-02-23T00:42:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.953+0000 7fd07ffff700 15 client.24695.objecter _session_command_op_remove 6 13 2024-02-23T00:42:41.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.970+0000 7fd09d6ee700 10 client.24695.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:42:41.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.970+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_assign -1 14 2024-02-23T00:42:41.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.970+0000 7fd09d6ee700 20 client.24695.objecter _get_session s=0x7fd084015a10 osd=6 3 2024-02-23T00:42:41.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.970+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd084015a10 osd=6 3 2024-02-23T00:42:41.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.970+0000 7fd09d6ee700 20 client.24695.objecter _get_session s=0x7fd084015a10 osd=6 3 2024-02-23T00:42:41.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.970+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_remove -1 14 2024-02-23T00:42:41.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.970+0000 7fd09d6ee700 20 client.24695.objecter get_session s=0x7fd084015a10 osd=6 3 2024-02-23T00:42:41.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.970+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_assign 6 14 2024-02-23T00:42:41.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.970+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd084015a10 osd=6 4 2024-02-23T00:42:41.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.970+0000 7fd09d6ee700 10 client.24695.objecter _send_command 14 2024-02-23T00:42:41.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.970+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd084016ba0 con 0x7fd0840160f0 2024-02-23T00:42:41.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.972+0000 7fd07ffff700 1 -- 172.21.15.47:0/2181864249 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd084016ba0 con 0x7fd0840160f0 2024-02-23T00:42:41.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.972+0000 7fd07ffff700 10 client.24695.objecter ms_dispatch 0x7fd098001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:42:41.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.972+0000 7fd07ffff700 10 client.24695.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:42:41.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.972+0000 7fd07ffff700 20 client.24695.objecter put_session s=0x7fd084015a10 osd=6 4 2024-02-23T00:42:41.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.972+0000 7fd07ffff700 15 client.24695.objecter _session_command_op_remove 6 14 2024-02-23T00:42:41.975 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:42:41.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.973+0000 7fd09d6ee700 10 client.24695.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:41.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.973+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_assign -1 15 2024-02-23T00:42:41.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.973+0000 7fd09d6ee700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fd08401b740 0x7fd08401db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:41.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.973+0000 7fd09d6ee700 20 client.24695.objecter _get_session s=0x7fd084016de0 osd=7 3 2024-02-23T00:42:41.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.973+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd084016de0 osd=7 3 2024-02-23T00:42:41.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.973+0000 7fd09d6ee700 20 client.24695.objecter _get_session s=0x7fd084016de0 osd=7 3 2024-02-23T00:42:41.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.973+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_remove -1 15 2024-02-23T00:42:41.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.973+0000 7fd09d6ee700 20 client.24695.objecter get_session s=0x7fd084016de0 osd=7 3 2024-02-23T00:42:41.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.973+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_assign 7 15 2024-02-23T00:42:41.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.973+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd084016de0 osd=7 4 2024-02-23T00:42:41.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.973+0000 7fd09d6ee700 10 client.24695.objecter _send_command 15 2024-02-23T00:42:41.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.973+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd084020780 con 0x7fd08401b740 2024-02-23T00:42:41.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.973+0000 7fd096ffd700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fd08401b740 0x7fd08401db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:41.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.973+0000 7fd07ffff700 10 client.24695.objecter ms_handle_connect 0x7fd08401b740 2024-02-23T00:42:41.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.973+0000 7fd096ffd700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fd08401b740 0x7fd08401db30 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:41.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.974+0000 7fd07ffff700 1 -- 172.21.15.47:0/2181864249 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd084020780 con 0x7fd08401b740 2024-02-23T00:42:41.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.974+0000 7fd07ffff700 10 client.24695.objecter ms_dispatch 0x7fd098001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:42:41.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.974+0000 7fd07ffff700 10 client.24695.objecter _finish_command 15 = system:0 2024-02-23T00:42:41.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.974+0000 7fd07ffff700 20 client.24695.objecter put_session s=0x7fd084016de0 osd=7 4 2024-02-23T00:42:41.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.974+0000 7fd07ffff700 15 client.24695.objecter _session_command_op_remove 7 15 2024-02-23T00:42:41.979 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:41.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.994+0000 7fd09d6ee700 10 client.24695.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:42:41.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.994+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_assign -1 16 2024-02-23T00:42:41.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.994+0000 7fd09d6ee700 20 client.24695.objecter _get_session s=0x7fd084016de0 osd=7 3 2024-02-23T00:42:41.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.994+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd084016de0 osd=7 3 2024-02-23T00:42:41.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.994+0000 7fd09d6ee700 20 client.24695.objecter _get_session s=0x7fd084016de0 osd=7 3 2024-02-23T00:42:41.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.994+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_remove -1 16 2024-02-23T00:42:41.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.994+0000 7fd09d6ee700 20 client.24695.objecter get_session s=0x7fd084016de0 osd=7 3 2024-02-23T00:42:41.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.994+0000 7fd09d6ee700 15 client.24695.objecter _session_command_op_assign 7 16 2024-02-23T00:42:41.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.994+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd084016de0 osd=7 4 2024-02-23T00:42:41.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.994+0000 7fd09d6ee700 10 client.24695.objecter _send_command 16 2024-02-23T00:42:41.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.994+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd08401e020 con 0x7fd08401b740 2024-02-23T00:42:41.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd07ffff700 1 -- 172.21.15.47:0/2181864249 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd08401e020 con 0x7fd08401b740 2024-02-23T00:42:41.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd07ffff700 10 client.24695.objecter ms_dispatch 0x7fd098001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:42:41.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd07ffff700 10 client.24695.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:42:41.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd07ffff700 20 client.24695.objecter put_session s=0x7fd084016de0 osd=7 4 2024-02-23T00:42:41.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd07ffff700 15 client.24695.objecter _session_command_op_remove 7 16 2024-02-23T00:42:41.998 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:42:42.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 10 librados: watch_flush enter 2024-02-23T00:42:42.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 10 librados: watch_flush exit 2024-02-23T00:42:42.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 10 client.24695.objecter close_session for osd.0 2024-02-23T00:42:42.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fd084004560 msgr2=0x7fd084004980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:42.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fd084004560 0x7fd084004980 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:42.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd0840051d0 osd=0 1 2024-02-23T00:42:42.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 10 client.24695.objecter close_session for osd.1 2024-02-23T00:42:42.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fd084008510 msgr2=0x7fd084002760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:42.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fd084008510 0x7fd084002760 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:42.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd084004380 osd=1 1 2024-02-23T00:42:42.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 10 client.24695.objecter close_session for osd.2 2024-02-23T00:42:42.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fd084003950 msgr2=0x7fd08400d040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:42.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fd084003950 0x7fd08400d040 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:42.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd0840037e0 osd=2 1 2024-02-23T00:42:42.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 10 client.24695.objecter close_session for osd.3 2024-02-23T00:42:42.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fd08400e7f0 msgr2=0x7fd08400ec70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:42.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd0977fe700 1 -- 172.21.15.47:0/2181864249 reap_dead start 2024-02-23T00:42:42.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fd08400e7f0 0x7fd08400ec70 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:42.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd084001b70 osd=3 1 2024-02-23T00:42:42.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 10 client.24695.objecter close_session for osd.4 2024-02-23T00:42:42.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fd084012170 msgr2=0x7fd084014560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:42.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fd084012170 0x7fd084014560 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:42.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd08400f160 osd=4 1 2024-02-23T00:42:42.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 10 client.24695.objecter close_session for osd.5 2024-02-23T00:42:42.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fd084014fb0 msgr2=0x7fd084015410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:42.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fd084014fb0 0x7fd084015410 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:42.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd08400f930 osd=5 1 2024-02-23T00:42:42.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 10 client.24695.objecter close_session for osd.6 2024-02-23T00:42:42.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fd0840160f0 msgr2=0x7fd084016550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:42.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fd0840160f0 0x7fd084016550 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:42.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd084015a10 osd=6 1 2024-02-23T00:42:42.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 10 client.24695.objecter close_session for osd.7 2024-02-23T00:42:42.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fd08401b740 msgr2=0x7fd08401db30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:42.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fd08401b740 0x7fd08401db30 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:42.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 20 client.24695.objecter put_session s=0x7fd084016de0 osd=7 1 2024-02-23T00:42:42.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 20 client.24695.objecter shutdown clearing up homeless session... 2024-02-23T00:42:42.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd09d6ee700 10 client.24695.objecter successfully canceled tick 2024-02-23T00:42:42.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.996+0000 7fd0977fe700 1 -- 172.21.15.47:0/2181864249 reap_dead start 2024-02-23T00:42:42.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.997+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd080062cc0 msgr2=0x7fd080065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:42.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.997+0000 7fd09d6ee700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd080062cc0 0x7fd080065180 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fd08c007fd0 tx=0x7fd08c009cb0).stop 2024-02-23T00:42:42.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.997+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd0980f8cb0 msgr2=0x7fd0980f6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:42.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.997+0000 7fd09d6ee700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd0980f8cb0 0x7fd0980f6c60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fd088004920 tx=0x7fd088009cb0).stop 2024-02-23T00:42:42.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.997+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 shutdown_connections 2024-02-23T00:42:42.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.997+0000 7fd09d6ee700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd0980f8cb0 0x7fd0980f6c60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:42.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.997+0000 7fd09d6ee700 1 --2- 172.21.15.47:0/2181864249 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd080062cc0 0x7fd080065180 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:42.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.997+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 >> 172.21.15.47:0/2181864249 conn(0x7fd0980f2820 msgr2=0x7fd0980f34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:42:42.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.997+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 shutdown_connections 2024-02-23T00:42:42.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.997+0000 7fd09d6ee700 1 -- 172.21.15.47:0/2181864249 wait complete. 2024-02-23T00:42:42.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:41.997+0000 7fd09d6ee700 1 librados: shutdown 2024-02-23T00:42:42.003 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:42:42.003 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:42:42.003 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:42:42.003 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:42:42.003 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:42:42.003 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:42:42.003 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:42:42.004 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:42:42.527 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:42:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:42:42.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:41.169481+0000 mgr.y (mgr.14517) 136 : cluster [DBG] pgmap v123: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 948 B/s rd, 0 op/s 2024-02-23T00:42:42.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:41.428089+0000 osd.6 (osd.6) 51 : cluster [DBG] 2.11 scrub starts 2024-02-23T00:42:42.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:41.429068+0000 osd.6 (osd.6) 52 : cluster [DBG] 2.11 scrub ok 2024-02-23T00:42:42.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:41.841620+0000 mon.c (mon.2) 26 : audit [DBG] from='client.? 172.21.15.47:0/2181864249' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:42:42.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:41.935105+0000 mon.a (mon.0) 787 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [3, 2]}]: dispatch 2024-02-23T00:42:42.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:41.169481+0000 mgr.y (mgr.14517) 136 : cluster [DBG] pgmap v123: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 948 B/s rd, 0 op/s 2024-02-23T00:42:42.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:41.428089+0000 osd.6 (osd.6) 51 : cluster [DBG] 2.11 scrub starts 2024-02-23T00:42:42.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:41.429068+0000 osd.6 (osd.6) 52 : cluster [DBG] 2.11 scrub ok 2024-02-23T00:42:42.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:41.841620+0000 mon.c (mon.2) 26 : audit [DBG] from='client.? 172.21.15.47:0/2181864249' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:42:42.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:41.935105+0000 mon.a (mon.0) 787 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [3, 2]}]: dispatch 2024-02-23T00:42:42.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:41.169481+0000 mgr.y (mgr.14517) 136 : cluster [DBG] pgmap v123: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 948 B/s rd, 0 op/s 2024-02-23T00:42:42.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:41.428089+0000 osd.6 (osd.6) 51 : cluster [DBG] 2.11 scrub starts 2024-02-23T00:42:42.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:41.429068+0000 osd.6 (osd.6) 52 : cluster [DBG] 2.11 scrub ok 2024-02-23T00:42:42.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:41.841620+0000 mon.c (mon.2) 26 : audit [DBG] from='client.? 172.21.15.47:0/2181864249' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:42:42.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:41.935105+0000 mon.a (mon.0) 787 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [3, 2]}]: dispatch 2024-02-23T00:42:42.943 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:42:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:42:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:42:43.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:42.026664+0000 osd.7 (osd.7) 63 : cluster [DBG] 5.4 scrub starts 2024-02-23T00:42:43.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:42.029629+0000 osd.7 (osd.7) 64 : cluster [DBG] 5.4 scrub ok 2024-02-23T00:42:43.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:42.201840+0000 osd.4 (osd.4) 65 : cluster [DBG] 4.1 scrub starts 2024-02-23T00:42:43.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:42.203902+0000 osd.4 (osd.4) 66 : cluster [DBG] 4.1 scrub ok 2024-02-23T00:42:43.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:42.272631+0000 mgr.y (mgr.14517) 137 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:42:43.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:42.282034+0000 osd.0 (osd.0) 59 : cluster [DBG] 6.18 scrub starts 2024-02-23T00:42:43.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:42.283241+0000 osd.0 (osd.0) 60 : cluster [DBG] 6.18 scrub ok 2024-02-23T00:42:43.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:42.460084+0000 osd.6 (osd.6) 53 : cluster [DBG] 2.14 scrub starts 2024-02-23T00:42:43.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:42.461013+0000 osd.6 (osd.6) 54 : cluster [DBG] 2.14 scrub ok 2024-02-23T00:42:43.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:42.503570+0000 mon.a (mon.0) 788 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [3, 2]}]': finished 2024-02-23T00:42:43.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:42.503637+0000 mon.a (mon.0) 789 : cluster [DBG] osdmap e104: 8 total, 8 up, 8 in 2024-02-23T00:42:43.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:42.930488+0000 osd.3 (osd.3) 79 : cluster [DBG] 4.10 scrub starts 2024-02-23T00:42:43.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:42.933952+0000 osd.3 (osd.3) 80 : cluster [DBG] 4.10 scrub ok 2024-02-23T00:42:43.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:42.026664+0000 osd.7 (osd.7) 63 : cluster [DBG] 5.4 scrub starts 2024-02-23T00:42:43.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:42.029629+0000 osd.7 (osd.7) 64 : cluster [DBG] 5.4 scrub ok 2024-02-23T00:42:43.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:42.201840+0000 osd.4 (osd.4) 65 : cluster [DBG] 4.1 scrub starts 2024-02-23T00:42:43.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:42.203902+0000 osd.4 (osd.4) 66 : cluster [DBG] 4.1 scrub ok 2024-02-23T00:42:43.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:42.272631+0000 mgr.y (mgr.14517) 137 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:42:43.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:42.282034+0000 osd.0 (osd.0) 59 : cluster [DBG] 6.18 scrub starts 2024-02-23T00:42:43.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:42.283241+0000 osd.0 (osd.0) 60 : cluster [DBG] 6.18 scrub ok 2024-02-23T00:42:43.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:42.460084+0000 osd.6 (osd.6) 53 : cluster [DBG] 2.14 scrub starts 2024-02-23T00:42:43.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:42.461013+0000 osd.6 (osd.6) 54 : cluster [DBG] 2.14 scrub ok 2024-02-23T00:42:43.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:42.503570+0000 mon.a (mon.0) 788 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [3, 2]}]': finished 2024-02-23T00:42:43.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:42.503637+0000 mon.a (mon.0) 789 : cluster [DBG] osdmap e104: 8 total, 8 up, 8 in 2024-02-23T00:42:43.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:42.930488+0000 osd.3 (osd.3) 79 : cluster [DBG] 4.10 scrub starts 2024-02-23T00:42:43.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:42.933952+0000 osd.3 (osd.3) 80 : cluster [DBG] 4.10 scrub ok 2024-02-23T00:42:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:42:43.834 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:42:43.837 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:43.838 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:42:43.938 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:42:43.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:42.026664+0000 osd.7 (osd.7) 63 : cluster [DBG] 5.4 scrub starts 2024-02-23T00:42:43.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:42.029629+0000 osd.7 (osd.7) 64 : cluster [DBG] 5.4 scrub ok 2024-02-23T00:42:43.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:42.201840+0000 osd.4 (osd.4) 65 : cluster [DBG] 4.1 scrub starts 2024-02-23T00:42:43.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:42.203902+0000 osd.4 (osd.4) 66 : cluster [DBG] 4.1 scrub ok 2024-02-23T00:42:43.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:42.272631+0000 mgr.y (mgr.14517) 137 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:42:43.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:42.282034+0000 osd.0 (osd.0) 59 : cluster [DBG] 6.18 scrub starts 2024-02-23T00:42:43.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:42.283241+0000 osd.0 (osd.0) 60 : cluster [DBG] 6.18 scrub ok 2024-02-23T00:42:43.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:42.460084+0000 osd.6 (osd.6) 53 : cluster [DBG] 2.14 scrub starts 2024-02-23T00:42:43.948 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:42.461013+0000 osd.6 (osd.6) 54 : cluster [DBG] 2.14 scrub ok 2024-02-23T00:42:43.948 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:42.503570+0000 mon.a (mon.0) 788 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [3, 2]}]': finished 2024-02-23T00:42:43.948 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:42.503637+0000 mon.a (mon.0) 789 : cluster [DBG] osdmap e104: 8 total, 8 up, 8 in 2024-02-23T00:42:43.948 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:42.930488+0000 osd.3 (osd.3) 79 : cluster [DBG] 4.10 scrub starts 2024-02-23T00:42:43.948 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:42.933952+0000 osd.3 (osd.3) 80 : cluster [DBG] 4.10 scrub ok 2024-02-23T00:42:43.968 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:44.091 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:42:44.094 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:44.095 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:42:44.196 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:42:44.230 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:44.250 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:42:44.253 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:44.254 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:42:44.355 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:42:44.388 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:44.409 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:42:44.412 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:44.412 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:42:44.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.422+0000 7f8134f9d700 1 -- 172.21.15.47:0/3960629385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8130101f30 msgr2=0x7f8130104320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:44.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.422+0000 7f8134f9d700 1 --2- 172.21.15.47:0/3960629385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8130101f30 0x7f8130104320 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f8128009be0 tx=0x7f8128004d10).stop 2024-02-23T00:42:44.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.422+0000 7f8134f9d700 1 -- 172.21.15.47:0/3960629385 shutdown_connections 2024-02-23T00:42:44.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.422+0000 7f8134f9d700 1 --2- 172.21.15.47:0/3960629385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8130101f30 0x7f8130104320 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:44.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.422+0000 7f8134f9d700 1 --2- 172.21.15.47:0/3960629385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81300f89d0 0x7f81300f8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:44.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.422+0000 7f8134f9d700 1 --2- 172.21.15.47:0/3960629385 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f81300f9330 0x7f8130101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:44.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.422+0000 7f8134f9d700 1 -- 172.21.15.47:0/3960629385 >> 172.21.15.47:0/3960629385 conn(0x7f81300f2500 msgr2=0x7f81300f4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:42:44.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.422+0000 7f8134f9d700 1 -- 172.21.15.47:0/3960629385 shutdown_connections 2024-02-23T00:42:44.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.422+0000 7f8134f9d700 1 -- 172.21.15.47:0/3960629385 wait complete. 2024-02-23T00:42:44.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.423+0000 7f8134f9d700 1 librados: starting msgr at 2024-02-23T00:42:44.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.423+0000 7f8134f9d700 1 librados: starting objecter 2024-02-23T00:42:44.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.423+0000 7f8134f9d700 1 Processor -- start 2024-02-23T00:42:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.423+0000 7f8134f9d700 1 -- start start 2024-02-23T00:42:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.423+0000 7f8134f9d700 1 librados: setting wanted keys 2024-02-23T00:42:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.423+0000 7f8134f9d700 1 librados: calling monclient init 2024-02-23T00:42:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.423+0000 7f8134f9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81300f89d0 0x7f81300f6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.423+0000 7f8134f9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81300f9330 0x7f81300f6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.423+0000 7f8134f9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8130101f30 0x7f81300f7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.423+0000 7f8134f9d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8130116270 con 0x7f81300f89d0 2024-02-23T00:42:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.423+0000 7f8134f9d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81301163e0 con 0x7f81300f9330 2024-02-23T00:42:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.423+0000 7f8134f9d700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81301165a0 con 0x7f8130101f30 2024-02-23T00:42:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.423+0000 7f812ffff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8130101f30 0x7f81300f7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.423+0000 7f812f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81300f89d0 0x7f81300f6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.423+0000 7f812ffff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8130101f30 0x7f81300f7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:57456/0 (socket says 172.21.15.47:57456) 2024-02-23T00:42:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.423+0000 7f812ffff700 1 -- 172.21.15.47:0/3179945336 learned_addr learned my addr 172.21.15.47:0/3179945336 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:42:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.423+0000 7f812effd700 1 --2- 172.21.15.47:0/3179945336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81300f9330 0x7f81300f6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.424+0000 7f812f7fe700 1 -- 172.21.15.47:0/3179945336 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8130101f30 msgr2=0x7f81300f7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.424+0000 7f812f7fe700 1 --2- 172.21.15.47:0/3179945336 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8130101f30 0x7f81300f7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.424+0000 7f812f7fe700 1 -- 172.21.15.47:0/3179945336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81300f9330 msgr2=0x7f81300f6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.424+0000 7f812f7fe700 1 --2- 172.21.15.47:0/3179945336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81300f9330 0x7f81300f6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.424+0000 7f812f7fe700 1 -- 172.21.15.47:0/3179945336 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81280097c0 con 0x7f81300f89d0 2024-02-23T00:42:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.424+0000 7f812f7fe700 1 --2- 172.21.15.47:0/3179945336 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81300f89d0 0x7f81300f6980 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f812000e8f0 tx=0x7f812000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.424+0000 7f812cff9700 10 client.?.objecter ms_handle_connect 0x7f81300f89d0 2024-02-23T00:42:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.424+0000 7f812cff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:42:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.424+0000 7f812cff9700 1 -- 172.21.15.47:0/3179945336 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8120004620 con 0x7f81300f89d0 2024-02-23T00:42:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.424+0000 7f812cff9700 1 -- 172.21.15.47:0/3179945336 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f812000f040 con 0x7f81300f89d0 2024-02-23T00:42:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.424+0000 7f8134f9d700 1 -- 172.21.15.47:0/3179945336 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8130116880 con 0x7f81300f89d0 2024-02-23T00:42:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.424+0000 7f812cff9700 1 -- 172.21.15.47:0/3179945336 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81200146f0 con 0x7f81300f89d0 2024-02-23T00:42:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.424+0000 7f8134f9d700 10 client.14817.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:42:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.424+0000 7f8134f9d700 1 -- 172.21.15.47:0/3179945336 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8130116dc0 con 0x7f81300f89d0 2024-02-23T00:42:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.424+0000 7f8134f9d700 1 librados: init done 2024-02-23T00:42:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.425+0000 7f8134f9d700 1 -- 172.21.15.47:0/3179945336 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8130061ec0 con 0x7f81300f89d0 2024-02-23T00:42:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.425+0000 7f812cff9700 1 -- 172.21.15.47:0/3179945336 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f8120007b40 con 0x7f81300f89d0 2024-02-23T00:42:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.426+0000 7f812cff9700 1 --2- 172.21.15.47:0/3179945336 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8118062cc0 0x7f8118065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.426+0000 7f812cff9700 1 -- 172.21.15.47:0/3179945336 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(105..105 src has 48..105) v4 ==== 7482+0+0 (secure 0 0 0) 0x7f812007c820 con 0x7f81300f89d0 2024-02-23T00:42:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.426+0000 7f812cff9700 10 client.14817.objecter ms_dispatch 0x7f8130001550 osd_map(105..105 src has 48..105) v4 2024-02-23T00:42:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.426+0000 7f812cff9700 3 client.14817.objecter handle_osd_map got epochs [105,105] > 0 2024-02-23T00:42:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.426+0000 7f812cff9700 3 client.14817.objecter handle_osd_map decoding full epoch 105 2024-02-23T00:42:44.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.426+0000 7f812effd700 1 --2- 172.21.15.47:0/3179945336 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8118062cc0 0x7f8118065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:44.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.426+0000 7f812cff9700 20 client.14817.objecter dump_active .. 0 homeless 2024-02-23T00:42:44.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.426+0000 7f812cff9700 10 client.14817.objecter ms_handle_connect 0x7f8118062cc0 2024-02-23T00:42:44.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.426+0000 7f812effd700 1 --2- 172.21.15.47:0/3179945336 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8118062cc0 0x7f8118065180 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f81300f8120 tx=0x7f812400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:44.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.430+0000 7f812cff9700 1 -- 172.21.15.47:0/3179945336 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f812004c690 con 0x7f81300f89d0 2024-02-23T00:42:44.513 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:42:44.541 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:44.561 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:42:44.563 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:44.564 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:42:44.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:44.631+0000 7f8134f9d700 1 -- 172.21.15.47:0/3179945336 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7f81300f7e60 con 0x7f81300f89d0 2024-02-23T00:42:44.664 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:42:44.692 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:44.711 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:42:44.713 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:44.714 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:42:44.800 INFO:tasks.workunit.client.0.smithi047.stderr:Note: switching to '4f758cd8e8854d1f126da86e254076852bf059a9'. 2024-02-23T00:42:44.800 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-02-23T00:42:44.800 INFO:tasks.workunit.client.0.smithi047.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-23T00:42:44.800 INFO:tasks.workunit.client.0.smithi047.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-23T00:42:44.801 INFO:tasks.workunit.client.0.smithi047.stderr:state without impacting any branches by switching back to a branch. 2024-02-23T00:42:44.801 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-02-23T00:42:44.801 INFO:tasks.workunit.client.0.smithi047.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-23T00:42:44.801 INFO:tasks.workunit.client.0.smithi047.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-23T00:42:44.801 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-02-23T00:42:44.801 INFO:tasks.workunit.client.0.smithi047.stderr: git switch -c 2024-02-23T00:42:44.801 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-02-23T00:42:44.801 INFO:tasks.workunit.client.0.smithi047.stderr:Or undo this operation with: 2024-02-23T00:42:44.801 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-02-23T00:42:44.801 INFO:tasks.workunit.client.0.smithi047.stderr: git switch - 2024-02-23T00:42:44.801 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-02-23T00:42:44.801 INFO:tasks.workunit.client.0.smithi047.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-23T00:42:44.801 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-02-23T00:42:44.801 INFO:tasks.workunit.client.0.smithi047.stderr:HEAD is now at 4f758cd8e88 [DNM] qa/suites: added more whitelisting 2024-02-23T00:42:44.805 DEBUG:teuthology.orchestra.run.smithi047:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-02-23T00:42:44.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:43.024001+0000 osd.7 (osd.7) 65 : cluster [DBG] 6.8 scrub starts 2024-02-23T00:42:44.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:43.027022+0000 osd.7 (osd.7) 66 : cluster [DBG] 6.8 scrub ok 2024-02-23T00:42:44.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:43.169816+0000 mgr.y (mgr.14517) 138 : cluster [DBG] pgmap v125: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-23T00:42:44.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:43.441254+0000 osd.5 (osd.5) 67 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:42:44.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:43.441339+0000 osd.5 (osd.5) 68 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:42:44.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:43.514143+0000 mon.a (mon.0) 790 : cluster [DBG] osdmap e105: 8 total, 8 up, 8 in 2024-02-23T00:42:44.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:43.024001+0000 osd.7 (osd.7) 65 : cluster [DBG] 6.8 scrub starts 2024-02-23T00:42:44.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:43.027022+0000 osd.7 (osd.7) 66 : cluster [DBG] 6.8 scrub ok 2024-02-23T00:42:44.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:43.169816+0000 mgr.y (mgr.14517) 138 : cluster [DBG] pgmap v125: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-23T00:42:44.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:43.441254+0000 osd.5 (osd.5) 67 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:42:44.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:43.441339+0000 osd.5 (osd.5) 68 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:42:44.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:43.514143+0000 mon.a (mon.0) 790 : cluster [DBG] osdmap e105: 8 total, 8 up, 8 in 2024-02-23T00:42:44.814 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:42:44.878 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:44.906 INFO:tasks.workunit.client.0.smithi047.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-23T00:42:44.908 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-23T00:42:44.908 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-23T00:42:44.931 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:42:44.935 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:44.936 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:42:44.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:43.024001+0000 osd.7 (osd.7) 65 : cluster [DBG] 6.8 scrub starts 2024-02-23T00:42:44.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:43.027022+0000 osd.7 (osd.7) 66 : cluster [DBG] 6.8 scrub ok 2024-02-23T00:42:44.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:43.169816+0000 mgr.y (mgr.14517) 138 : cluster [DBG] pgmap v125: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-23T00:42:44.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:43.441254+0000 osd.5 (osd.5) 67 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:42:44.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:43.441339+0000 osd.5 (osd.5) 68 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:42:44.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:43.514143+0000 mon.a (mon.0) 790 : cluster [DBG] osdmap e105: 8 total, 8 up, 8 in 2024-02-23T00:42:45.037 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:42:45.066 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:45.338 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:42:45.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:45.520+0000 7f812cff9700 1 -- 172.21.15.47:0/3179945336 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v106) v1 ==== 101+0+0 (secure 0 0 0) 0x7f8120019090 con 0x7f81300f89d0 2024-02-23T00:42:45.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:45.523+0000 7f8134f9d700 10 librados: watch_flush enter 2024-02-23T00:42:45.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:45.523+0000 7f8134f9d700 10 librados: watch_flush exit 2024-02-23T00:42:45.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:45.523+0000 7f8134f9d700 20 client.14817.objecter shutdown clearing up homeless session... 2024-02-23T00:42:45.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:45.523+0000 7f8134f9d700 10 client.14817.objecter successfully canceled tick 2024-02-23T00:42:45.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:45.523+0000 7f8134f9d700 1 -- 172.21.15.47:0/3179945336 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8118062cc0 msgr2=0x7f8118065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:45.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:45.523+0000 7f8134f9d700 1 --2- 172.21.15.47:0/3179945336 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8118062cc0 0x7f8118065180 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f81300f8120 tx=0x7f812400b040).stop 2024-02-23T00:42:45.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:45.523+0000 7f8134f9d700 1 -- 172.21.15.47:0/3179945336 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81300f89d0 msgr2=0x7f81300f6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:45.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:45.523+0000 7f8134f9d700 1 --2- 172.21.15.47:0/3179945336 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81300f89d0 0x7f81300f6980 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f812000e8f0 tx=0x7f812000b040).stop 2024-02-23T00:42:45.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:45.524+0000 7f8134f9d700 1 -- 172.21.15.47:0/3179945336 shutdown_connections 2024-02-23T00:42:45.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:45.524+0000 7f8134f9d700 1 --2- 172.21.15.47:0/3179945336 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8118062cc0 0x7f8118065180 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:45.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:45.524+0000 7f8134f9d700 1 --2- 172.21.15.47:0/3179945336 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8130101f30 0x7f81300f7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:45.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:45.524+0000 7f8134f9d700 1 --2- 172.21.15.47:0/3179945336 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81300f89d0 0x7f81300f6980 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:45.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:45.524+0000 7f8134f9d700 1 --2- 172.21.15.47:0/3179945336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81300f9330 0x7f81300f6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:45.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:45.524+0000 7f8134f9d700 1 -- 172.21.15.47:0/3179945336 >> 172.21.15.47:0/3179945336 conn(0x7f81300f2500 msgr2=0x7f81300f3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:42:45.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:45.524+0000 7f8134f9d700 1 -- 172.21.15.47:0/3179945336 shutdown_connections 2024-02-23T00:42:45.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:45.524+0000 7f8134f9d700 1 -- 172.21.15.47:0/3179945336 wait complete. 2024-02-23T00:42:45.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:45.524+0000 7f8134f9d700 1 librados: shutdown 2024-02-23T00:42:45.527 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is unset 2024-02-23T00:42:45.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:43.987589+0000 osd.7 (osd.7) 67 : cluster [DBG] 2.5 scrub starts 2024-02-23T00:42:45.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:43.990482+0000 osd.7 (osd.7) 68 : cluster [DBG] 2.5 scrub ok 2024-02-23T00:42:45.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:44.342226+0000 osd.2 (osd.2) 63 : cluster [DBG] 4.14 scrub starts 2024-02-23T00:42:45.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:44.344434+0000 osd.2 (osd.2) 64 : cluster [DBG] 4.14 scrub ok 2024-02-23T00:42:45.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:44.550665+0000 osd.1 (osd.1) 71 : cluster [DBG] 5.19 scrub starts 2024-02-23T00:42:45.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:44.553611+0000 osd.1 (osd.1) 72 : cluster [DBG] 5.19 scrub ok 2024-02-23T00:42:45.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:44.633239+0000 mon.a (mon.0) 791 : audit [INF] from='client.? 172.21.15.47:0/3179945336' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:42:45.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:43.987589+0000 osd.7 (osd.7) 67 : cluster [DBG] 2.5 scrub starts 2024-02-23T00:42:45.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:43.990482+0000 osd.7 (osd.7) 68 : cluster [DBG] 2.5 scrub ok 2024-02-23T00:42:45.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:44.342226+0000 osd.2 (osd.2) 63 : cluster [DBG] 4.14 scrub starts 2024-02-23T00:42:45.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:44.344434+0000 osd.2 (osd.2) 64 : cluster [DBG] 4.14 scrub ok 2024-02-23T00:42:45.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:44.550665+0000 osd.1 (osd.1) 71 : cluster [DBG] 5.19 scrub starts 2024-02-23T00:42:45.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:44.553611+0000 osd.1 (osd.1) 72 : cluster [DBG] 5.19 scrub ok 2024-02-23T00:42:45.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:44.633239+0000 mon.a (mon.0) 791 : audit [INF] from='client.? 172.21.15.47:0/3179945336' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:42:45.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:43.987589+0000 osd.7 (osd.7) 67 : cluster [DBG] 2.5 scrub starts 2024-02-23T00:42:45.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:43.990482+0000 osd.7 (osd.7) 68 : cluster [DBG] 2.5 scrub ok 2024-02-23T00:42:45.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:44.342226+0000 osd.2 (osd.2) 63 : cluster [DBG] 4.14 scrub starts 2024-02-23T00:42:45.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:44.344434+0000 osd.2 (osd.2) 64 : cluster [DBG] 4.14 scrub ok 2024-02-23T00:42:45.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:44.550665+0000 osd.1 (osd.1) 71 : cluster [DBG] 5.19 scrub starts 2024-02-23T00:42:45.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:44.553611+0000 osd.1 (osd.1) 72 : cluster [DBG] 5.19 scrub ok 2024-02-23T00:42:45.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:44.633239+0000 mon.a (mon.0) 791 : audit [INF] from='client.? 172.21.15.47:0/3179945336' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:42:46.416 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:42:46.419 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:46.419 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:42:46.520 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:42:46.548 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:46.662 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-23T00:42:46.794 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-23T00:42:46.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:44.980904+0000 osd.7 (osd.7) 69 : cluster [DBG] 6.12 scrub starts 2024-02-23T00:42:46.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:44.983810+0000 osd.7 (osd.7) 70 : cluster [DBG] 6.12 scrub ok 2024-02-23T00:42:46.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:45.170332+0000 mgr.y (mgr.14517) 139 : cluster [DBG] pgmap v127: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:42:46.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:45.511808+0000 mon.a (mon.0) 792 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:42:46.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:45.511840+0000 mon.a (mon.0) 793 : cluster [INF] Cluster is now healthy 2024-02-23T00:42:46.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:45.520331+0000 mon.a (mon.0) 794 : audit [INF] from='client.? 172.21.15.47:0/3179945336' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:42:46.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:45.520383+0000 mon.a (mon.0) 795 : cluster [DBG] osdmap e106: 8 total, 8 up, 8 in 2024-02-23T00:42:46.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:46.113378+0000 mon.a (mon.0) 796 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:42:46.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:44.980904+0000 osd.7 (osd.7) 69 : cluster [DBG] 6.12 scrub starts 2024-02-23T00:42:46.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:44.983810+0000 osd.7 (osd.7) 70 : cluster [DBG] 6.12 scrub ok 2024-02-23T00:42:46.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:45.170332+0000 mgr.y (mgr.14517) 139 : cluster [DBG] pgmap v127: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:42:46.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:45.511808+0000 mon.a (mon.0) 792 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:42:46.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:45.511840+0000 mon.a (mon.0) 793 : cluster [INF] Cluster is now healthy 2024-02-23T00:42:46.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:45.520331+0000 mon.a (mon.0) 794 : audit [INF] from='client.? 172.21.15.47:0/3179945336' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:42:46.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:45.520383+0000 mon.a (mon.0) 795 : cluster [DBG] osdmap e106: 8 total, 8 up, 8 in 2024-02-23T00:42:46.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:46.113378+0000 mon.a (mon.0) 796 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:42:46.842 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-23T00:42:46.844 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-23T00:42:46.844 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-23T00:42:46.893 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-23T00:42:46.896 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-02-23T00:42:46.896 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-02-23T00:42:46.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:44.980904+0000 osd.7 (osd.7) 69 : cluster [DBG] 6.12 scrub starts 2024-02-23T00:42:46.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:44.983810+0000 osd.7 (osd.7) 70 : cluster [DBG] 6.12 scrub ok 2024-02-23T00:42:46.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:45.170332+0000 mgr.y (mgr.14517) 139 : cluster [DBG] pgmap v127: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:42:46.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:45.511808+0000 mon.a (mon.0) 792 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:42:46.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:45.511840+0000 mon.a (mon.0) 793 : cluster [INF] Cluster is now healthy 2024-02-23T00:42:46.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:45.520331+0000 mon.a (mon.0) 794 : audit [INF] from='client.? 172.21.15.47:0/3179945336' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:42:46.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:45.520383+0000 mon.a (mon.0) 795 : cluster [DBG] osdmap e106: 8 total, 8 up, 8 in 2024-02-23T00:42:46.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:46.113378+0000 mon.a (mon.0) 796 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:42:46.957 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-02-23T00:42:46.958 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-02-23T00:42:46.958 DEBUG:teuthology.orchestra.run.smithi047:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4f758cd8e8854d1f126da86e254076852bf059a9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2024-02-23T00:42:47.041 INFO:tasks.workunit.client.0.smithi047.stderr:+ parallel=1 2024-02-23T00:42:47.041 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' '' = --serial ']' 2024-02-23T00:42:47.041 INFO:tasks.workunit.client.0.smithi047.stderr:+ color= 2024-02-23T00:42:47.041 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' -t 1 ']' 2024-02-23T00:42:47.041 INFO:tasks.workunit.client.0.smithi047.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-02-23T00:42:47.041 INFO:tasks.workunit.client.0.smithi047.stderr:+ declare -A pids 2024-02-23T00:42:47.041 INFO:tasks.workunit.client.0.smithi047.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-23T00:42:47.042 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' 1 -eq 1 ']' 2024-02-23T00:42:47.042 INFO:tasks.workunit.client.0.smithi047.stderr:++ printf %25s api_aio 2024-02-23T00:42:47.042 INFO:tasks.workunit.client.0.smithi047.stderr:+ r=' api_aio' 2024-02-23T00:42:47.042 INFO:tasks.workunit.client.0.smithi047.stderr:++ echo api_aio 2024-02-23T00:42:47.042 INFO:tasks.workunit.client.0.smithi047.stderr:++ awk '{print $1}' 2024-02-23T00:42:47.043 INFO:tasks.workunit.client.0.smithi047.stderr:+ ff=api_aio 2024-02-23T00:42:47.043 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82373 2024-02-23T00:42:47.043 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo 'test api_aio on pid 82373' 2024-02-23T00:42:47.043 INFO:tasks.workunit.client.0.smithi047.stdout:test api_aio on pid 82373 2024-02-23T00:42:47.044 INFO:tasks.workunit.client.0.smithi047.stderr:+ pids[$f]=82373 2024-02-23T00:42:47.044 INFO:tasks.workunit.client.0.smithi047.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-23T00:42:47.044 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' 1 -eq 1 ']' 2024-02-23T00:42:47.044 INFO:tasks.workunit.client.0.smithi047.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2024-02-23T00:42:47.045 INFO:tasks.workunit.client.0.smithi047.stderr:++ printf %25s api_aio_pp 2024-02-23T00:42:47.045 INFO:tasks.workunit.client.0.smithi047.stderr:+ r=' api_aio_pp' 2024-02-23T00:42:47.045 INFO:tasks.workunit.client.0.smithi047.stderr:++ echo api_aio_pp 2024-02-23T00:42:47.045 INFO:tasks.workunit.client.0.smithi047.stderr:++ awk '{print $1}' 2024-02-23T00:42:47.045 INFO:tasks.workunit.client.0.smithi047.stderr:+ ceph_test_rados_api_aio 2024-02-23T00:42:47.045 INFO:tasks.workunit.client.0.smithi047.stderr:+ tee ceph_test_rados_api_aio.log 2024-02-23T00:42:47.045 INFO:tasks.workunit.client.0.smithi047.stderr:+ sed 's/^/ api_aio: /' 2024-02-23T00:42:47.046 INFO:tasks.workunit.client.0.smithi047.stderr:+ ff=api_aio_pp 2024-02-23T00:42:47.046 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82381 2024-02-23T00:42:47.046 INFO:tasks.workunit.client.0.smithi047.stdout:test api_aio_pp on pid 82381 2024-02-23T00:42:47.047 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo 'test api_aio_pp on pid 82381' 2024-02-23T00:42:47.047 INFO:tasks.workunit.client.0.smithi047.stderr:+ pids[$f]=82381 2024-02-23T00:42:47.047 INFO:tasks.workunit.client.0.smithi047.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-23T00:42:47.047 INFO:tasks.workunit.client.0.smithi047.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2024-02-23T00:42:47.048 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' 1 -eq 1 ']' 2024-02-23T00:42:47.048 INFO:tasks.workunit.client.0.smithi047.stderr:++ printf %25s api_io 2024-02-23T00:42:47.048 INFO:tasks.workunit.client.0.smithi047.stderr:+ r=' api_io' 2024-02-23T00:42:47.048 INFO:tasks.workunit.client.0.smithi047.stderr:++ echo api_io 2024-02-23T00:42:47.048 INFO:tasks.workunit.client.0.smithi047.stderr:++ awk '{print $1}' 2024-02-23T00:42:47.048 INFO:tasks.workunit.client.0.smithi047.stderr:+ ceph_test_rados_api_aio_pp 2024-02-23T00:42:47.048 INFO:tasks.workunit.client.0.smithi047.stderr:+ tee ceph_test_rados_api_aio_pp.log 2024-02-23T00:42:47.048 INFO:tasks.workunit.client.0.smithi047.stderr:+ sed 's/^/ api_aio_pp: /' 2024-02-23T00:42:47.048 INFO:tasks.workunit.client.0.smithi047.stderr:+ ff=api_io 2024-02-23T00:42:47.048 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82389 2024-02-23T00:42:47.048 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo 'test api_io on pid 82389' 2024-02-23T00:42:47.049 INFO:tasks.workunit.client.0.smithi047.stdout:test api_io on pid 82389 2024-02-23T00:42:47.050 INFO:tasks.workunit.client.0.smithi047.stderr:+ pids[$f]=82389 2024-02-23T00:42:47.050 INFO:tasks.workunit.client.0.smithi047.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-23T00:42:47.050 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' 1 -eq 1 ']' 2024-02-23T00:42:47.050 INFO:tasks.workunit.client.0.smithi047.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2024-02-23T00:42:47.050 INFO:tasks.workunit.client.0.smithi047.stderr:++ printf %25s api_io_pp 2024-02-23T00:42:47.050 INFO:tasks.workunit.client.0.smithi047.stderr:+ r=' api_io_pp' 2024-02-23T00:42:47.050 INFO:tasks.workunit.client.0.smithi047.stderr:++ echo api_io_pp 2024-02-23T00:42:47.050 INFO:tasks.workunit.client.0.smithi047.stderr:++ awk '{print $1}' 2024-02-23T00:42:47.050 INFO:tasks.workunit.client.0.smithi047.stderr:+ ceph_test_rados_api_io 2024-02-23T00:42:47.050 INFO:tasks.workunit.client.0.smithi047.stderr:+ tee ceph_test_rados_api_io.log 2024-02-23T00:42:47.051 INFO:tasks.workunit.client.0.smithi047.stderr:+ sed 's/^/ api_io: /' 2024-02-23T00:42:47.051 INFO:tasks.workunit.client.0.smithi047.stderr:+ ff=api_io_pp 2024-02-23T00:42:47.051 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82397 2024-02-23T00:42:47.051 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo 'test api_io_pp on pid 82397' 2024-02-23T00:42:47.051 INFO:tasks.workunit.client.0.smithi047.stdout:test api_io_pp on pid 82397 2024-02-23T00:42:47.052 INFO:tasks.workunit.client.0.smithi047.stderr:+ pids[$f]=82397 2024-02-23T00:42:47.053 INFO:tasks.workunit.client.0.smithi047.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-23T00:42:47.053 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' 1 -eq 1 ']' 2024-02-23T00:42:47.053 INFO:tasks.workunit.client.0.smithi047.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2024-02-23T00:42:47.053 INFO:tasks.workunit.client.0.smithi047.stderr:++ printf %25s api_asio 2024-02-23T00:42:47.053 INFO:tasks.workunit.client.0.smithi047.stderr:+ r=' api_asio' 2024-02-23T00:42:47.053 INFO:tasks.workunit.client.0.smithi047.stderr:++ echo api_asio 2024-02-23T00:42:47.053 INFO:tasks.workunit.client.0.smithi047.stderr:++ awk '{print $1}' 2024-02-23T00:42:47.053 INFO:tasks.workunit.client.0.smithi047.stderr:+ ceph_test_rados_api_io_pp 2024-02-23T00:42:47.053 INFO:tasks.workunit.client.0.smithi047.stderr:+ tee ceph_test_rados_api_io_pp.log 2024-02-23T00:42:47.053 INFO:tasks.workunit.client.0.smithi047.stderr:+ sed 's/^/ api_io_pp: /' 2024-02-23T00:42:47.053 INFO:tasks.workunit.client.0.smithi047.stderr:+ ff=api_asio 2024-02-23T00:42:47.053 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82405 2024-02-23T00:42:47.053 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo 'test api_asio on pid 82405' 2024-02-23T00:42:47.054 INFO:tasks.workunit.client.0.smithi047.stdout:test api_asio on pid 82405 2024-02-23T00:42:47.055 INFO:tasks.workunit.client.0.smithi047.stderr:+ pids[$f]=82405 2024-02-23T00:42:47.055 INFO:tasks.workunit.client.0.smithi047.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-23T00:42:47.055 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' 1 -eq 1 ']' 2024-02-23T00:42:47.055 INFO:tasks.workunit.client.0.smithi047.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2024-02-23T00:42:47.055 INFO:tasks.workunit.client.0.smithi047.stderr:++ printf %25s api_list 2024-02-23T00:42:47.055 INFO:tasks.workunit.client.0.smithi047.stderr:+ r=' api_list' 2024-02-23T00:42:47.055 INFO:tasks.workunit.client.0.smithi047.stderr:++ echo api_list 2024-02-23T00:42:47.055 INFO:tasks.workunit.client.0.smithi047.stderr:++ awk '{print $1}' 2024-02-23T00:42:47.055 INFO:tasks.workunit.client.0.smithi047.stderr:+ ceph_test_rados_api_asio 2024-02-23T00:42:47.055 INFO:tasks.workunit.client.0.smithi047.stderr:+ tee ceph_test_rados_api_asio.log 2024-02-23T00:42:47.056 INFO:tasks.workunit.client.0.smithi047.stderr:+ sed 's/^/ api_asio: /' 2024-02-23T00:42:47.056 INFO:tasks.workunit.client.0.smithi047.stderr:+ ff=api_list 2024-02-23T00:42:47.056 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82413 2024-02-23T00:42:47.056 INFO:tasks.workunit.client.0.smithi047.stdout:test api_list on pid 82413 2024-02-23T00:42:47.057 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo 'test api_list on pid 82413' 2024-02-23T00:42:47.057 INFO:tasks.workunit.client.0.smithi047.stderr:+ pids[$f]=82413 2024-02-23T00:42:47.057 INFO:tasks.workunit.client.0.smithi047.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-23T00:42:47.057 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' 1 -eq 1 ']' 2024-02-23T00:42:47.058 INFO:tasks.workunit.client.0.smithi047.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2024-02-23T00:42:47.058 INFO:tasks.workunit.client.0.smithi047.stderr:++ printf %25s api_lock 2024-02-23T00:42:47.058 INFO:tasks.workunit.client.0.smithi047.stderr:+ r=' api_lock' 2024-02-23T00:42:47.058 INFO:tasks.workunit.client.0.smithi047.stderr:++ echo api_lock 2024-02-23T00:42:47.058 INFO:tasks.workunit.client.0.smithi047.stderr:++ awk '{print $1}' 2024-02-23T00:42:47.058 INFO:tasks.workunit.client.0.smithi047.stderr:+ ceph_test_rados_api_list 2024-02-23T00:42:47.058 INFO:tasks.workunit.client.0.smithi047.stderr:+ tee ceph_test_rados_api_list.log 2024-02-23T00:42:47.058 INFO:tasks.workunit.client.0.smithi047.stderr:+ sed 's/^/ api_list: /' 2024-02-23T00:42:47.058 INFO:tasks.workunit.client.0.smithi047.stderr:+ ff=api_lock 2024-02-23T00:42:47.059 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82423 2024-02-23T00:42:47.059 INFO:tasks.workunit.client.0.smithi047.stdout:test api_lock on pid 82423 2024-02-23T00:42:47.060 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo 'test api_lock on pid 82423' 2024-02-23T00:42:47.060 INFO:tasks.workunit.client.0.smithi047.stderr:+ pids[$f]=82423 2024-02-23T00:42:47.060 INFO:tasks.workunit.client.0.smithi047.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-23T00:42:47.060 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' 1 -eq 1 ']' 2024-02-23T00:42:47.060 INFO:tasks.workunit.client.0.smithi047.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2024-02-23T00:42:47.061 INFO:tasks.workunit.client.0.smithi047.stderr:++ printf %25s api_lock_pp 2024-02-23T00:42:47.061 INFO:tasks.workunit.client.0.smithi047.stderr:+ r=' api_lock_pp' 2024-02-23T00:42:47.061 INFO:tasks.workunit.client.0.smithi047.stderr:++ echo api_lock_pp 2024-02-23T00:42:47.061 INFO:tasks.workunit.client.0.smithi047.stderr:++ awk '{print $1}' 2024-02-23T00:42:47.061 INFO:tasks.workunit.client.0.smithi047.stderr:+ ceph_test_rados_api_lock 2024-02-23T00:42:47.061 INFO:tasks.workunit.client.0.smithi047.stderr:+ tee ceph_test_rados_api_lock.log 2024-02-23T00:42:47.061 INFO:tasks.workunit.client.0.smithi047.stderr:+ sed 's/^/ api_lock: /' 2024-02-23T00:42:47.061 INFO:tasks.workunit.client.0.smithi047.stderr:+ ff=api_lock_pp 2024-02-23T00:42:47.061 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82433 2024-02-23T00:42:47.062 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo 'test api_lock_pp on pid 82433' 2024-02-23T00:42:47.062 INFO:tasks.workunit.client.0.smithi047.stdout:test api_lock_pp on pid 82433 2024-02-23T00:42:47.063 INFO:tasks.workunit.client.0.smithi047.stderr:+ pids[$f]=82433 2024-02-23T00:42:47.063 INFO:tasks.workunit.client.0.smithi047.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-23T00:42:47.063 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' 1 -eq 1 ']' 2024-02-23T00:42:47.063 INFO:tasks.workunit.client.0.smithi047.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2024-02-23T00:42:47.063 INFO:tasks.workunit.client.0.smithi047.stderr:++ printf %25s api_misc 2024-02-23T00:42:47.063 INFO:tasks.workunit.client.0.smithi047.stderr:+ r=' api_misc' 2024-02-23T00:42:47.063 INFO:tasks.workunit.client.0.smithi047.stderr:++ echo api_misc 2024-02-23T00:42:47.063 INFO:tasks.workunit.client.0.smithi047.stderr:++ awk '{print $1}' 2024-02-23T00:42:47.063 INFO:tasks.workunit.client.0.smithi047.stderr:+ ceph_test_rados_api_lock_pp 2024-02-23T00:42:47.064 INFO:tasks.workunit.client.0.smithi047.stderr:+ tee ceph_test_rados_api_lock_pp.log 2024-02-23T00:42:47.064 INFO:tasks.workunit.client.0.smithi047.stderr:+ sed 's/^/ api_lock_pp: /' 2024-02-23T00:42:47.064 INFO:tasks.workunit.client.0.smithi047.stderr:+ ff=api_misc 2024-02-23T00:42:47.064 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82442 2024-02-23T00:42:47.064 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo 'test api_misc on pid 82442' 2024-02-23T00:42:47.065 INFO:tasks.workunit.client.0.smithi047.stdout:test api_misc on pid 82442 2024-02-23T00:42:47.066 INFO:tasks.workunit.client.0.smithi047.stderr:+ pids[$f]=82442 2024-02-23T00:42:47.066 INFO:tasks.workunit.client.0.smithi047.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-23T00:42:47.066 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' 1 -eq 1 ']' 2024-02-23T00:42:47.066 INFO:tasks.workunit.client.0.smithi047.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2024-02-23T00:42:47.066 INFO:tasks.workunit.client.0.smithi047.stderr:++ printf %25s api_misc_pp 2024-02-23T00:42:47.066 INFO:tasks.workunit.client.0.smithi047.stderr:+ r=' api_misc_pp' 2024-02-23T00:42:47.066 INFO:tasks.workunit.client.0.smithi047.stderr:++ echo api_misc_pp 2024-02-23T00:42:47.066 INFO:tasks.workunit.client.0.smithi047.stderr:++ awk '{print $1}' 2024-02-23T00:42:47.066 INFO:tasks.workunit.client.0.smithi047.stderr:+ ceph_test_rados_api_misc 2024-02-23T00:42:47.066 INFO:tasks.workunit.client.0.smithi047.stderr:+ sed 's/^/ api_misc: /' 2024-02-23T00:42:47.067 INFO:tasks.workunit.client.0.smithi047.stderr:+ ff=api_misc_pp 2024-02-23T00:42:47.067 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82451 2024-02-23T00:42:47.067 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo 'test api_misc_pp on pid 82451' 2024-02-23T00:42:47.067 INFO:tasks.workunit.client.0.smithi047.stdout:test api_misc_pp on pid 82451 2024-02-23T00:42:47.068 INFO:tasks.workunit.client.0.smithi047.stderr:+ pids[$f]=82451 2024-02-23T00:42:47.069 INFO:tasks.workunit.client.0.smithi047.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-23T00:42:47.069 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' 1 -eq 1 ']' 2024-02-23T00:42:47.069 INFO:tasks.workunit.client.0.smithi047.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2024-02-23T00:42:47.069 INFO:tasks.workunit.client.0.smithi047.stderr:++ printf %25s api_tier_pp 2024-02-23T00:42:47.069 INFO:tasks.workunit.client.0.smithi047.stderr:+ r=' api_tier_pp' 2024-02-23T00:42:47.069 INFO:tasks.workunit.client.0.smithi047.stderr:+ tee ceph_test_rados_api_misc.log 2024-02-23T00:42:47.069 INFO:tasks.workunit.client.0.smithi047.stderr:++ echo api_tier_pp 2024-02-23T00:42:47.069 INFO:tasks.workunit.client.0.smithi047.stderr:++ awk '{print $1}' 2024-02-23T00:42:47.069 INFO:tasks.workunit.client.0.smithi047.stderr:+ ceph_test_rados_api_misc_pp 2024-02-23T00:42:47.069 INFO:tasks.workunit.client.0.smithi047.stderr:+ sed 's/^/ api_misc_pp: /' 2024-02-23T00:42:47.070 INFO:tasks.workunit.client.0.smithi047.stderr:+ ff=api_tier_pp 2024-02-23T00:42:47.070 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82460 2024-02-23T00:42:47.071 INFO:tasks.workunit.client.0.smithi047.stdout:test api_tier_pp on pid 82460 2024-02-23T00:42:47.071 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo 'test api_tier_pp on pid 82460' 2024-02-23T00:42:47.072 INFO:tasks.workunit.client.0.smithi047.stderr:+ pids[$f]=82460 2024-02-23T00:42:47.072 INFO:tasks.workunit.client.0.smithi047.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-23T00:42:47.072 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' 1 -eq 1 ']' 2024-02-23T00:42:47.072 INFO:tasks.workunit.client.0.smithi047.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2024-02-23T00:42:47.072 INFO:tasks.workunit.client.0.smithi047.stderr:+ tee ceph_test_rados_api_misc_pp.log 2024-02-23T00:42:47.072 INFO:tasks.workunit.client.0.smithi047.stderr:++ printf %25s api_pool 2024-02-23T00:42:47.072 INFO:tasks.workunit.client.0.smithi047.stderr:+ r=' api_pool' 2024-02-23T00:42:47.072 INFO:tasks.workunit.client.0.smithi047.stderr:++ echo api_pool 2024-02-23T00:42:47.072 INFO:tasks.workunit.client.0.smithi047.stderr:++ awk '{print $1}' 2024-02-23T00:42:47.073 INFO:tasks.workunit.client.0.smithi047.stderr:+ ceph_test_rados_api_tier_pp 2024-02-23T00:42:47.073 INFO:tasks.workunit.client.0.smithi047.stderr:+ tee ceph_test_rados_api_tier_pp.log 2024-02-23T00:42:47.074 INFO:tasks.workunit.client.0.smithi047.stderr:+ ff=api_pool 2024-02-23T00:42:47.074 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82470 2024-02-23T00:42:47.074 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo 'test api_pool on pid 82470' 2024-02-23T00:42:47.074 INFO:tasks.workunit.client.0.smithi047.stdout:test api_pool on pid 82470 2024-02-23T00:42:47.075 INFO:tasks.workunit.client.0.smithi047.stderr:+ pids[$f]=82470 2024-02-23T00:42:47.075 INFO:tasks.workunit.client.0.smithi047.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-23T00:42:47.075 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' 1 -eq 1 ']' 2024-02-23T00:42:47.075 INFO:tasks.workunit.client.0.smithi047.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2024-02-23T00:42:47.075 INFO:tasks.workunit.client.0.smithi047.stderr:++ printf %25s api_snapshots 2024-02-23T00:42:47.075 INFO:tasks.workunit.client.0.smithi047.stderr:+ r=' api_snapshots' 2024-02-23T00:42:47.075 INFO:tasks.workunit.client.0.smithi047.stderr:++ echo api_snapshots 2024-02-23T00:42:47.075 INFO:tasks.workunit.client.0.smithi047.stderr:++ awk '{print $1}' 2024-02-23T00:42:47.077 INFO:tasks.workunit.client.0.smithi047.stderr:+ ceph_test_rados_api_pool 2024-02-23T00:42:47.079 INFO:tasks.workunit.client.0.smithi047.stderr:+ sed 's/^/ api_pool: /' 2024-02-23T00:42:47.080 INFO:tasks.workunit.client.0.smithi047.stderr:+ sed 's/^/ api_tier_pp: /' 2024-02-23T00:42:47.080 INFO:tasks.workunit.client.0.smithi047.stderr:+ ff=api_snapshots 2024-02-23T00:42:47.080 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82481 2024-02-23T00:42:47.081 INFO:tasks.workunit.client.0.smithi047.stdout:test api_snapshots on pid 82481 2024-02-23T00:42:47.081 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo 'test api_snapshots on pid 82481' 2024-02-23T00:42:47.081 INFO:tasks.workunit.client.0.smithi047.stderr:+ pids[$f]=82481 2024-02-23T00:42:47.081 INFO:tasks.workunit.client.0.smithi047.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-23T00:42:47.082 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' 1 -eq 1 ']' 2024-02-23T00:42:47.082 INFO:tasks.workunit.client.0.smithi047.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2024-02-23T00:42:47.082 INFO:tasks.workunit.client.0.smithi047.stderr:+ tee ceph_test_rados_api_pool.log 2024-02-23T00:42:47.082 INFO:tasks.workunit.client.0.smithi047.stderr:++ printf %25s api_snapshots_pp 2024-02-23T00:42:47.082 INFO:tasks.workunit.client.0.smithi047.stderr:+ r=' api_snapshots_pp' 2024-02-23T00:42:47.084 INFO:tasks.workunit.client.0.smithi047.stderr:++ echo api_snapshots_pp 2024-02-23T00:42:47.084 INFO:tasks.workunit.client.0.smithi047.stderr:++ awk '{print $1}' 2024-02-23T00:42:47.084 INFO:tasks.workunit.client.0.smithi047.stderr:+ ceph_test_rados_api_snapshots 2024-02-23T00:42:47.084 INFO:tasks.workunit.client.0.smithi047.stderr:+ tee ceph_test_rados_api_snapshots.log 2024-02-23T00:42:47.084 INFO:tasks.workunit.client.0.smithi047.stderr:+ ff=api_snapshots_pp 2024-02-23T00:42:47.085 INFO:tasks.workunit.client.0.smithi047.stdout:test api_snapshots_pp on pid 82491 2024-02-23T00:42:47.085 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82491 2024-02-23T00:42:47.085 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo 'test api_snapshots_pp on pid 82491' 2024-02-23T00:42:47.085 INFO:tasks.workunit.client.0.smithi047.stderr:+ pids[$f]=82491 2024-02-23T00:42:47.085 INFO:tasks.workunit.client.0.smithi047.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-23T00:42:47.085 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' 1 -eq 1 ']' 2024-02-23T00:42:47.085 INFO:tasks.workunit.client.0.smithi047.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2024-02-23T00:42:47.086 INFO:tasks.workunit.client.0.smithi047.stderr:++ printf %25s api_stat 2024-02-23T00:42:47.086 INFO:tasks.workunit.client.0.smithi047.stderr:+ r=' api_stat' 2024-02-23T00:42:47.086 INFO:tasks.workunit.client.0.smithi047.stderr:++ awk '{print $1}' 2024-02-23T00:42:47.087 INFO:tasks.workunit.client.0.smithi047.stderr:+ sed 's/^/ api_snapshots: /' 2024-02-23T00:42:47.088 INFO:tasks.workunit.client.0.smithi047.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2024-02-23T00:42:47.088 INFO:tasks.workunit.client.0.smithi047.stderr:+ sed 's/^/ api_snapshots_pp: /' 2024-02-23T00:42:47.091 INFO:tasks.workunit.client.0.smithi047.stderr:++ echo api_stat 2024-02-23T00:42:47.091 INFO:tasks.workunit.client.0.smithi047.stderr:+ ceph_test_rados_api_snapshots_pp 2024-02-23T00:42:47.092 INFO:tasks.workunit.client.0.smithi047.stdout:test api_stat on pid 82499 2024-02-23T00:42:47.092 INFO:tasks.workunit.client.0.smithi047.stderr:+ ff=api_stat 2024-02-23T00:42:47.092 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82499 2024-02-23T00:42:47.092 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo 'test api_stat on pid 82499' 2024-02-23T00:42:47.093 INFO:tasks.workunit.client.0.smithi047.stderr:+ pids[$f]=82499 2024-02-23T00:42:47.093 INFO:tasks.workunit.client.0.smithi047.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2024-02-23T00:42:47.093 INFO:tasks.workunit.client.0.smithi047.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-23T00:42:47.093 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' 1 -eq 1 ']' 2024-02-23T00:42:47.093 INFO:tasks.workunit.client.0.smithi047.stderr:++ printf %25s api_stat_pp 2024-02-23T00:42:47.093 INFO:tasks.workunit.client.0.smithi047.stderr:+ r=' api_stat_pp' 2024-02-23T00:42:47.093 INFO:tasks.workunit.client.0.smithi047.stderr:++ awk '{print $1}' 2024-02-23T00:42:47.095 INFO:tasks.workunit.client.0.smithi047.stderr:+ sed 's/^/ api_stat: /' 2024-02-23T00:42:47.096 INFO:tasks.workunit.client.0.smithi047.stderr:++ echo api_stat_pp 2024-02-23T00:42:47.097 INFO:tasks.workunit.client.0.smithi047.stdout:test api_stat_pp on pid 82507 2024-02-23T00:42:47.097 INFO:tasks.workunit.client.0.smithi047.stderr:+ ff=api_stat_pp 2024-02-23T00:42:47.097 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82507 2024-02-23T00:42:47.097 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo 'test api_stat_pp on pid 82507' 2024-02-23T00:42:47.097 INFO:tasks.workunit.client.0.smithi047.stderr:+ pids[$f]=82507 2024-02-23T00:42:47.097 INFO:tasks.workunit.client.0.smithi047.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2024-02-23T00:42:47.097 INFO:tasks.workunit.client.0.smithi047.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-23T00:42:47.098 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' 1 -eq 1 ']' 2024-02-23T00:42:47.098 INFO:tasks.workunit.client.0.smithi047.stderr:++ printf %25s api_watch_notify 2024-02-23T00:42:47.098 INFO:tasks.workunit.client.0.smithi047.stderr:+ r=' api_watch_notify' 2024-02-23T00:42:47.098 INFO:tasks.workunit.client.0.smithi047.stderr:+ ceph_test_rados_api_stat 2024-02-23T00:42:47.098 INFO:tasks.workunit.client.0.smithi047.stderr:++ echo api_watch_notify 2024-02-23T00:42:47.098 INFO:tasks.workunit.client.0.smithi047.stderr:++ awk '{print $1}' 2024-02-23T00:42:47.098 INFO:tasks.workunit.client.0.smithi047.stderr:+ ceph_test_rados_api_stat_pp 2024-02-23T00:42:47.098 INFO:tasks.workunit.client.0.smithi047.stderr:+ sed 's/^/ api_stat_pp: /' 2024-02-23T00:42:47.099 INFO:tasks.workunit.client.0.smithi047.stderr:+ tee ceph_test_rados_api_stat.log 2024-02-23T00:42:47.100 INFO:tasks.workunit.client.0.smithi047.stderr:+ ff=api_watch_notify 2024-02-23T00:42:47.100 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82515 2024-02-23T00:42:47.100 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo 'test api_watch_notify on pid 82515' 2024-02-23T00:42:47.100 INFO:tasks.workunit.client.0.smithi047.stdout:test api_watch_notify on pid 82515 2024-02-23T00:42:47.101 INFO:tasks.workunit.client.0.smithi047.stderr:+ pids[$f]=82515 2024-02-23T00:42:47.101 INFO:tasks.workunit.client.0.smithi047.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-23T00:42:47.101 INFO:tasks.workunit.client.0.smithi047.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2024-02-23T00:42:47.101 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' 1 -eq 1 ']' 2024-02-23T00:42:47.101 INFO:tasks.workunit.client.0.smithi047.stderr:+ tee ceph_test_rados_api_stat_pp.log 2024-02-23T00:42:47.102 INFO:tasks.workunit.client.0.smithi047.stderr:+ tee ceph_test_rados_api_watch_notify.log 2024-02-23T00:42:47.102 INFO:tasks.workunit.client.0.smithi047.stderr:++ printf %25s api_watch_notify_pp 2024-02-23T00:42:47.102 INFO:tasks.workunit.client.0.smithi047.stderr:+ r=' api_watch_notify_pp' 2024-02-23T00:42:47.102 INFO:tasks.workunit.client.0.smithi047.stderr:+ ceph_test_rados_api_watch_notify 2024-02-23T00:42:47.102 INFO:tasks.workunit.client.0.smithi047.stderr:+ sed 's/^/ api_watch_notify: /' 2024-02-23T00:42:47.103 INFO:tasks.workunit.client.0.smithi047.stderr:++ echo api_watch_notify_pp 2024-02-23T00:42:47.104 INFO:tasks.workunit.client.0.smithi047.stderr:++ awk '{print $1}' 2024-02-23T00:42:47.105 INFO:tasks.workunit.client.0.smithi047.stderr:+ ff=api_watch_notify_pp 2024-02-23T00:42:47.105 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82523 2024-02-23T00:42:47.105 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo 'test api_watch_notify_pp on pid 82523' 2024-02-23T00:42:47.106 INFO:tasks.workunit.client.0.smithi047.stdout:test api_watch_notify_pp on pid 82523 2024-02-23T00:42:47.107 INFO:tasks.workunit.client.0.smithi047.stderr:+ pids[$f]=82523 2024-02-23T00:42:47.107 INFO:tasks.workunit.client.0.smithi047.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-23T00:42:47.107 INFO:tasks.workunit.client.0.smithi047.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2024-02-23T00:42:47.107 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' 1 -eq 1 ']' 2024-02-23T00:42:47.107 INFO:tasks.workunit.client.0.smithi047.stderr:++ printf %25s api_cmd 2024-02-23T00:42:47.107 INFO:tasks.workunit.client.0.smithi047.stderr:+ r=' api_cmd' 2024-02-23T00:42:47.107 INFO:tasks.workunit.client.0.smithi047.stderr:++ echo api_cmd 2024-02-23T00:42:47.107 INFO:tasks.workunit.client.0.smithi047.stderr:++ awk '{print $1}' 2024-02-23T00:42:47.107 INFO:tasks.workunit.client.0.smithi047.stderr:+ ceph_test_rados_api_watch_notify_pp 2024-02-23T00:42:47.107 INFO:tasks.workunit.client.0.smithi047.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2024-02-23T00:42:47.107 INFO:tasks.workunit.client.0.smithi047.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2024-02-23T00:42:47.108 INFO:tasks.workunit.client.0.smithi047.stderr:+ ff=api_cmd 2024-02-23T00:42:47.108 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82531 2024-02-23T00:42:47.108 INFO:tasks.workunit.client.0.smithi047.stdout:test api_cmd on pid 82531 2024-02-23T00:42:47.110 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo 'test api_cmd on pid 82531' 2024-02-23T00:42:47.110 INFO:tasks.workunit.client.0.smithi047.stderr:+ pids[$f]=82531 2024-02-23T00:42:47.110 INFO:tasks.workunit.client.0.smithi047.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-23T00:42:47.110 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' 1 -eq 1 ']' 2024-02-23T00:42:47.110 INFO:tasks.workunit.client.0.smithi047.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2024-02-23T00:42:47.110 INFO:tasks.workunit.client.0.smithi047.stderr:++ printf %25s api_cmd_pp 2024-02-23T00:42:47.110 INFO:tasks.workunit.client.0.smithi047.stderr:+ r=' api_cmd_pp' 2024-02-23T00:42:47.110 INFO:tasks.workunit.client.0.smithi047.stderr:++ echo api_cmd_pp 2024-02-23T00:42:47.110 INFO:tasks.workunit.client.0.smithi047.stderr:++ awk '{print $1}' 2024-02-23T00:42:47.110 INFO:tasks.workunit.client.0.smithi047.stderr:+ ceph_test_rados_api_cmd 2024-02-23T00:42:47.110 INFO:tasks.workunit.client.0.smithi047.stderr:+ tee ceph_test_rados_api_cmd.log 2024-02-23T00:42:47.110 INFO:tasks.workunit.client.0.smithi047.stderr:+ sed 's/^/ api_cmd: /' 2024-02-23T00:42:47.112 INFO:tasks.workunit.client.0.smithi047.stderr:+ ff=api_cmd_pp 2024-02-23T00:42:47.112 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82539 2024-02-23T00:42:47.112 INFO:tasks.workunit.client.0.smithi047.stdout:test api_cmd_pp on pid 82539 2024-02-23T00:42:47.113 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo 'test api_cmd_pp on pid 82539' 2024-02-23T00:42:47.113 INFO:tasks.workunit.client.0.smithi047.stderr:+ pids[$f]=82539 2024-02-23T00:42:47.113 INFO:tasks.workunit.client.0.smithi047.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-23T00:42:47.113 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' 1 -eq 1 ']' 2024-02-23T00:42:47.113 INFO:tasks.workunit.client.0.smithi047.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2024-02-23T00:42:47.114 INFO:tasks.workunit.client.0.smithi047.stderr:++ printf %25s api_service 2024-02-23T00:42:47.114 INFO:tasks.workunit.client.0.smithi047.stderr:+ r=' api_service' 2024-02-23T00:42:47.114 INFO:tasks.workunit.client.0.smithi047.stderr:++ echo api_service 2024-02-23T00:42:47.114 INFO:tasks.workunit.client.0.smithi047.stderr:++ awk '{print $1}' 2024-02-23T00:42:47.114 INFO:tasks.workunit.client.0.smithi047.stderr:+ ceph_test_rados_api_cmd_pp 2024-02-23T00:42:47.114 INFO:tasks.workunit.client.0.smithi047.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2024-02-23T00:42:47.114 INFO:tasks.workunit.client.0.smithi047.stderr:+ sed 's/^/ api_cmd_pp: /' 2024-02-23T00:42:47.115 INFO:tasks.workunit.client.0.smithi047.stderr:+ ff=api_service 2024-02-23T00:42:47.115 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82547 2024-02-23T00:42:47.115 INFO:tasks.workunit.client.0.smithi047.stdout:test api_service on pid 82547 2024-02-23T00:42:47.116 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo 'test api_service on pid 82547' 2024-02-23T00:42:47.116 INFO:tasks.workunit.client.0.smithi047.stderr:+ pids[$f]=82547 2024-02-23T00:42:47.116 INFO:tasks.workunit.client.0.smithi047.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-23T00:42:47.116 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' 1 -eq 1 ']' 2024-02-23T00:42:47.116 INFO:tasks.workunit.client.0.smithi047.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2024-02-23T00:42:47.117 INFO:tasks.workunit.client.0.smithi047.stderr:++ printf %25s api_service_pp 2024-02-23T00:42:47.117 INFO:tasks.workunit.client.0.smithi047.stderr:+ r=' api_service_pp' 2024-02-23T00:42:47.117 INFO:tasks.workunit.client.0.smithi047.stderr:++ echo api_service_pp 2024-02-23T00:42:47.117 INFO:tasks.workunit.client.0.smithi047.stderr:++ awk '{print $1}' 2024-02-23T00:42:47.117 INFO:tasks.workunit.client.0.smithi047.stderr:+ ceph_test_rados_api_service 2024-02-23T00:42:47.117 INFO:tasks.workunit.client.0.smithi047.stderr:+ tee ceph_test_rados_api_service.log 2024-02-23T00:42:47.117 INFO:tasks.workunit.client.0.smithi047.stderr:+ sed 's/^/ api_service: /' 2024-02-23T00:42:47.117 INFO:tasks.workunit.client.0.smithi047.stderr:+ ff=api_service_pp 2024-02-23T00:42:47.117 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82555 2024-02-23T00:42:47.118 INFO:tasks.workunit.client.0.smithi047.stdout:test api_service_pp on pid 82555 2024-02-23T00:42:47.119 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo 'test api_service_pp on pid 82555' 2024-02-23T00:42:47.119 INFO:tasks.workunit.client.0.smithi047.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2024-02-23T00:42:47.119 INFO:tasks.workunit.client.0.smithi047.stderr:+ pids[$f]=82555 2024-02-23T00:42:47.119 INFO:tasks.workunit.client.0.smithi047.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-23T00:42:47.119 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' 1 -eq 1 ']' 2024-02-23T00:42:47.119 INFO:tasks.workunit.client.0.smithi047.stderr:++ printf %25s api_c_write_operations 2024-02-23T00:42:47.119 INFO:tasks.workunit.client.0.smithi047.stderr:+ r=' api_c_write_operations' 2024-02-23T00:42:47.120 INFO:tasks.workunit.client.0.smithi047.stderr:++ echo api_c_write_operations 2024-02-23T00:42:47.120 INFO:tasks.workunit.client.0.smithi047.stderr:++ awk '{print $1}' 2024-02-23T00:42:47.120 INFO:tasks.workunit.client.0.smithi047.stderr:+ ceph_test_rados_api_service_pp 2024-02-23T00:42:47.120 INFO:tasks.workunit.client.0.smithi047.stderr:+ tee ceph_test_rados_api_service_pp.log 2024-02-23T00:42:47.120 INFO:tasks.workunit.client.0.smithi047.stderr:+ sed 's/^/ api_service_pp: /' 2024-02-23T00:42:47.121 INFO:tasks.workunit.client.0.smithi047.stderr:+ ff=api_c_write_operations 2024-02-23T00:42:47.121 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82563 2024-02-23T00:42:47.121 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo 'test api_c_write_operations on pid 82563' 2024-02-23T00:42:47.121 INFO:tasks.workunit.client.0.smithi047.stdout:test api_c_write_operations on pid 82563 2024-02-23T00:42:47.122 INFO:tasks.workunit.client.0.smithi047.stderr:+ pids[$f]=82563 2024-02-23T00:42:47.123 INFO:tasks.workunit.client.0.smithi047.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-23T00:42:47.123 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' 1 -eq 1 ']' 2024-02-23T00:42:47.123 INFO:tasks.workunit.client.0.smithi047.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2024-02-23T00:42:47.123 INFO:tasks.workunit.client.0.smithi047.stderr:++ printf %25s api_c_read_operations 2024-02-23T00:42:47.123 INFO:tasks.workunit.client.0.smithi047.stderr:+ r=' api_c_read_operations' 2024-02-23T00:42:47.123 INFO:tasks.workunit.client.0.smithi047.stderr:++ echo api_c_read_operations 2024-02-23T00:42:47.123 INFO:tasks.workunit.client.0.smithi047.stderr:++ awk '{print $1}' 2024-02-23T00:42:47.123 INFO:tasks.workunit.client.0.smithi047.stderr:+ ceph_test_rados_api_c_write_operations 2024-02-23T00:42:47.123 INFO:tasks.workunit.client.0.smithi047.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2024-02-23T00:42:47.123 INFO:tasks.workunit.client.0.smithi047.stderr:+ sed 's/^/ api_c_write_operations: /' 2024-02-23T00:42:47.124 INFO:tasks.workunit.client.0.smithi047.stderr:+ ff=api_c_read_operations 2024-02-23T00:42:47.124 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82571 2024-02-23T00:42:47.124 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo 'test api_c_read_operations on pid 82571' 2024-02-23T00:42:47.124 INFO:tasks.workunit.client.0.smithi047.stdout:test api_c_read_operations on pid 82571 2024-02-23T00:42:47.125 INFO:tasks.workunit.client.0.smithi047.stderr:+ pids[$f]=82571 2024-02-23T00:42:47.125 INFO:tasks.workunit.client.0.smithi047.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-23T00:42:47.125 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' 1 -eq 1 ']' 2024-02-23T00:42:47.125 INFO:tasks.workunit.client.0.smithi047.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2024-02-23T00:42:47.125 INFO:tasks.workunit.client.0.smithi047.stderr:++ printf %25s list_parallel 2024-02-23T00:42:47.126 INFO:tasks.workunit.client.0.smithi047.stderr:+ r=' list_parallel' 2024-02-23T00:42:47.126 INFO:tasks.workunit.client.0.smithi047.stderr:++ echo list_parallel 2024-02-23T00:42:47.126 INFO:tasks.workunit.client.0.smithi047.stderr:++ awk '{print $1}' 2024-02-23T00:42:47.126 INFO:tasks.workunit.client.0.smithi047.stderr:+ ceph_test_rados_api_c_read_operations 2024-02-23T00:42:47.126 INFO:tasks.workunit.client.0.smithi047.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2024-02-23T00:42:47.126 INFO:tasks.workunit.client.0.smithi047.stderr:+ sed 's/^/ api_c_read_operations: /' 2024-02-23T00:42:47.126 INFO:tasks.workunit.client.0.smithi047.stderr:+ ff=list_parallel 2024-02-23T00:42:47.127 INFO:tasks.workunit.client.0.smithi047.stdout:test list_parallel on pid 82579 2024-02-23T00:42:47.127 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82579 2024-02-23T00:42:47.127 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo 'test list_parallel on pid 82579' 2024-02-23T00:42:47.127 INFO:tasks.workunit.client.0.smithi047.stderr:+ pids[$f]=82579 2024-02-23T00:42:47.127 INFO:tasks.workunit.client.0.smithi047.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2024-02-23T00:42:47.128 INFO:tasks.workunit.client.0.smithi047.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-23T00:42:47.128 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' 1 -eq 1 ']' 2024-02-23T00:42:47.128 INFO:tasks.workunit.client.0.smithi047.stderr:++ printf %25s open_pools_parallel 2024-02-23T00:42:47.128 INFO:tasks.workunit.client.0.smithi047.stderr:+ r=' open_pools_parallel' 2024-02-23T00:42:47.128 INFO:tasks.workunit.client.0.smithi047.stderr:++ echo open_pools_parallel 2024-02-23T00:42:47.128 INFO:tasks.workunit.client.0.smithi047.stderr:++ awk '{print $1}' 2024-02-23T00:42:47.128 INFO:tasks.workunit.client.0.smithi047.stderr:+ ceph_test_rados_list_parallel 2024-02-23T00:42:47.129 INFO:tasks.workunit.client.0.smithi047.stderr:+ tee ceph_test_rados_list_parallel.log 2024-02-23T00:42:47.129 INFO:tasks.workunit.client.0.smithi047.stderr:+ sed 's/^/ list_parallel: /' 2024-02-23T00:42:47.130 INFO:tasks.workunit.client.0.smithi047.stdout:test open_pools_parallel on pid 82587 2024-02-23T00:42:47.130 INFO:tasks.workunit.client.0.smithi047.stderr:+ ff=open_pools_parallel 2024-02-23T00:42:47.131 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82587 2024-02-23T00:42:47.131 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo 'test open_pools_parallel on pid 82587' 2024-02-23T00:42:47.131 INFO:tasks.workunit.client.0.smithi047.stderr:+ pids[$f]=82587 2024-02-23T00:42:47.131 INFO:tasks.workunit.client.0.smithi047.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-23T00:42:47.131 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' 1 -eq 1 ']' 2024-02-23T00:42:47.131 INFO:tasks.workunit.client.0.smithi047.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2024-02-23T00:42:47.131 INFO:tasks.workunit.client.0.smithi047.stderr:++ printf %25s delete_pools_parallel 2024-02-23T00:42:47.131 INFO:tasks.workunit.client.0.smithi047.stderr:+ r=' delete_pools_parallel' 2024-02-23T00:42:47.131 INFO:tasks.workunit.client.0.smithi047.stderr:++ echo delete_pools_parallel 2024-02-23T00:42:47.131 INFO:tasks.workunit.client.0.smithi047.stderr:++ awk '{print $1}' 2024-02-23T00:42:47.132 INFO:tasks.workunit.client.0.smithi047.stderr:+ ceph_test_rados_open_pools_parallel 2024-02-23T00:42:47.132 INFO:tasks.workunit.client.0.smithi047.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2024-02-23T00:42:47.132 INFO:tasks.workunit.client.0.smithi047.stderr:+ sed 's/^/ open_pools_parallel: /' 2024-02-23T00:42:47.133 INFO:tasks.workunit.client.0.smithi047.stdout:test delete_pools_parallel on pid 82595 2024-02-23T00:42:47.133 INFO:tasks.workunit.client.0.smithi047.stderr:+ ff=delete_pools_parallel 2024-02-23T00:42:47.133 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82595 2024-02-23T00:42:47.133 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo 'test delete_pools_parallel on pid 82595' 2024-02-23T00:42:47.133 INFO:tasks.workunit.client.0.smithi047.stderr:+ pids[$f]=82595 2024-02-23T00:42:47.133 INFO:tasks.workunit.client.0.smithi047.stderr:+ ret=0 2024-02-23T00:42:47.133 INFO:tasks.workunit.client.0.smithi047.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2024-02-23T00:42:47.133 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' 1 -eq 1 ']' 2024-02-23T00:42:47.133 INFO:tasks.workunit.client.0.smithi047.stderr:+ for t in "${!pids[@]}" 2024-02-23T00:42:47.134 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82481 2024-02-23T00:42:47.134 INFO:tasks.workunit.client.0.smithi047.stderr:+ wait 82481 2024-02-23T00:42:47.134 INFO:tasks.workunit.client.0.smithi047.stderr:+ ceph_test_rados_delete_pools_parallel 2024-02-23T00:42:47.134 INFO:tasks.workunit.client.0.smithi047.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2024-02-23T00:42:47.134 INFO:tasks.workunit.client.0.smithi047.stderr:+ sed 's/^/ delete_pools_parallel: /' 2024-02-23T00:42:47.500 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:42:47.504 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:47.505 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:42:47.606 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:42:47.634 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:46.358502+0000 osd.0 (osd.0) 61 : cluster [DBG] 6.9 scrub starts 2024-02-23T00:42:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:46.359638+0000 osd.0 (osd.0) 62 : cluster [DBG] 6.9 scrub ok 2024-02-23T00:42:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:46.520464+0000 mon.a (mon.0) 797 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:42:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:46.521134+0000 mon.a (mon.0) 798 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:42:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:46.525478+0000 mon.a (mon.0) 799 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:42:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:46.603212+0000 osd.1 (osd.1) 73 : cluster [DBG] 2.4 scrub starts 2024-02-23T00:42:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:46.606182+0000 osd.1 (osd.1) 74 : cluster [DBG] 2.4 scrub ok 2024-02-23T00:42:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:46.944542+0000 osd.5 (osd.5) 69 : cluster [DBG] 2.3 scrub starts 2024-02-23T00:42:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:46.945353+0000 osd.5 (osd.5) 70 : cluster [DBG] 2.3 scrub ok 2024-02-23T00:42:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:47.298935+0000 mon.b (mon.1) 35 : audit [INF] from='client.? 172.21.15.47:0/2670125835' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:42:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:46.358502+0000 osd.0 (osd.0) 61 : cluster [DBG] 6.9 scrub starts 2024-02-23T00:42:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:46.359638+0000 osd.0 (osd.0) 62 : cluster [DBG] 6.9 scrub ok 2024-02-23T00:42:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:46.520464+0000 mon.a (mon.0) 797 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:42:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:46.521134+0000 mon.a (mon.0) 798 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:42:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:46.525478+0000 mon.a (mon.0) 799 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:42:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:46.603212+0000 osd.1 (osd.1) 73 : cluster [DBG] 2.4 scrub starts 2024-02-23T00:42:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:46.606182+0000 osd.1 (osd.1) 74 : cluster [DBG] 2.4 scrub ok 2024-02-23T00:42:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:46.944542+0000 osd.5 (osd.5) 69 : cluster [DBG] 2.3 scrub starts 2024-02-23T00:42:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:46.945353+0000 osd.5 (osd.5) 70 : cluster [DBG] 2.3 scrub ok 2024-02-23T00:42:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:47.298935+0000 mon.b (mon.1) 35 : audit [INF] from='client.? 172.21.15.47:0/2670125835' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:42:48.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:46.358502+0000 osd.0 (osd.0) 61 : cluster [DBG] 6.9 scrub starts 2024-02-23T00:42:48.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:46.359638+0000 osd.0 (osd.0) 62 : cluster [DBG] 6.9 scrub ok 2024-02-23T00:42:48.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:46.520464+0000 mon.a (mon.0) 797 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:42:48.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:46.521134+0000 mon.a (mon.0) 798 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:42:48.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:46.525478+0000 mon.a (mon.0) 799 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:42:48.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:46.603212+0000 osd.1 (osd.1) 73 : cluster [DBG] 2.4 scrub starts 2024-02-23T00:42:48.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:46.606182+0000 osd.1 (osd.1) 74 : cluster [DBG] 2.4 scrub ok 2024-02-23T00:42:48.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:46.944542+0000 osd.5 (osd.5) 69 : cluster [DBG] 2.3 scrub starts 2024-02-23T00:42:48.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:46.945353+0000 osd.5 (osd.5) 70 : cluster [DBG] 2.3 scrub ok 2024-02-23T00:42:48.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:47.298935+0000 mon.b (mon.1) 35 : audit [INF] from='client.? 172.21.15.47:0/2670125835' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:42:48.706 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:42:48.706 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-02-23T00:42:48.706 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-02-23T00:42:48.706 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-02-23T00:42:48.706 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:42:49.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.574+0000 7f0e88623700 1 -- 172.21.15.47:0/1176888456 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0e80102210 msgr2=0x7f0e80104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:49.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.574+0000 7f0e88623700 1 --2- 172.21.15.47:0/1176888456 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0e80102210 0x7f0e80104600 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f0e78009be0 tx=0x7f0e78004d10).stop 2024-02-23T00:42:49.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.574+0000 7f0e88623700 1 -- 172.21.15.47:0/1176888456 shutdown_connections 2024-02-23T00:42:49.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.574+0000 7f0e88623700 1 --2- 172.21.15.47:0/1176888456 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0e80102210 0x7f0e80104600 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:49.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.574+0000 7f0e88623700 1 --2- 172.21.15.47:0/1176888456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e800f8cb0 0x7f0e800f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:49.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.574+0000 7f0e88623700 1 --2- 172.21.15.47:0/1176888456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e800f9610 0x7f0e80101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:49.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.574+0000 7f0e88623700 1 -- 172.21.15.47:0/1176888456 >> 172.21.15.47:0/1176888456 conn(0x7f0e800f2820 msgr2=0x7f0e800f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:42:49.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.574+0000 7f0e88623700 1 -- 172.21.15.47:0/1176888456 shutdown_connections 2024-02-23T00:42:49.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.574+0000 7f0e88623700 1 -- 172.21.15.47:0/1176888456 wait complete. 2024-02-23T00:42:49.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.574+0000 7f0e88623700 1 librados: starting msgr at 2024-02-23T00:42:49.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.574+0000 7f0e88623700 1 librados: starting objecter 2024-02-23T00:42:49.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.574+0000 7f0e88623700 1 Processor -- start 2024-02-23T00:42:49.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.574+0000 7f0e88623700 1 -- start start 2024-02-23T00:42:49.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.574+0000 7f0e88623700 1 librados: setting wanted keys 2024-02-23T00:42:49.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.574+0000 7f0e88623700 1 librados: calling monclient init 2024-02-23T00:42:49.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.575+0000 7f0e88623700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0e800f8cb0 0x7f0e800f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:49.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.575+0000 7f0e88623700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e800f9610 0x7f0e800f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:49.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.575+0000 7f0e88623700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e80102210 0x7f0e800f7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:49.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.575+0000 7f0e88623700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e80116290 con 0x7f0e80102210 2024-02-23T00:42:49.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.575+0000 7f0e88623700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e80116400 con 0x7f0e800f9610 2024-02-23T00:42:49.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.575+0000 7f0e88623700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e801165c0 con 0x7f0e800f8cb0 2024-02-23T00:42:49.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.575+0000 7f0e863bf700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0e800f8cb0 0x7f0e800f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:49.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.575+0000 7f0e86bc0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e80102210 0x7f0e800f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:49.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.575+0000 7f0e863bf700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0e800f8cb0 0x7f0e800f6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:39120/0 (socket says 172.21.15.47:39120) 2024-02-23T00:42:49.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.575+0000 7f0e863bf700 1 -- 172.21.15.47:0/344941712 learned_addr learned my addr 172.21.15.47:0/344941712 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:42:49.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.575+0000 7f0e85bbe700 1 --2- 172.21.15.47:0/344941712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e800f9610 0x7f0e800f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:49.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.575+0000 7f0e863bf700 1 -- 172.21.15.47:0/344941712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e800f9610 msgr2=0x7f0e800f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:49.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.575+0000 7f0e863bf700 1 --2- 172.21.15.47:0/344941712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e800f9610 0x7f0e800f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:49.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.575+0000 7f0e863bf700 1 -- 172.21.15.47:0/344941712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e80102210 msgr2=0x7f0e800f7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:49.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.575+0000 7f0e863bf700 1 --2- 172.21.15.47:0/344941712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e80102210 0x7f0e800f7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:49.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.575+0000 7f0e863bf700 1 -- 172.21.15.47:0/344941712 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e780097c0 con 0x7f0e800f8cb0 2024-02-23T00:42:49.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.575+0000 7f0e863bf700 1 --2- 172.21.15.47:0/344941712 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0e800f8cb0 0x7f0e800f6c60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f0e7c0027f0 tx=0x7f0e7c002b00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:49.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.575+0000 7f0e777fe700 10 client.?.objecter ms_handle_connect 0x7f0e800f8cb0 2024-02-23T00:42:49.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.575+0000 7f0e777fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:42:49.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.576+0000 7f0e777fe700 1 -- 172.21.15.47:0/344941712 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c007b40 con 0x7f0e800f8cb0 2024-02-23T00:42:49.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.576+0000 7f0e88623700 1 -- 172.21.15.47:0/344941712 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e801168a0 con 0x7f0e800f8cb0 2024-02-23T00:42:49.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.576+0000 7f0e777fe700 1 -- 172.21.15.47:0/344941712 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f0e7c00f040 con 0x7f0e800f8cb0 2024-02-23T00:42:49.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.576+0000 7f0e777fe700 1 -- 172.21.15.47:0/344941712 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c014610 con 0x7f0e800f8cb0 2024-02-23T00:42:49.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.576+0000 7f0e88623700 10 client.24809.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:42:49.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.576+0000 7f0e88623700 1 -- 172.21.15.47:0/344941712 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e80116d60 con 0x7f0e800f8cb0 2024-02-23T00:42:49.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.576+0000 7f0e88623700 1 librados: init done 2024-02-23T00:42:49.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.577+0000 7f0e88623700 1 -- 172.21.15.47:0/344941712 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e8005fa60 con 0x7f0e800f8cb0 2024-02-23T00:42:49.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.577+0000 7f0e777fe700 1 -- 172.21.15.47:0/344941712 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f0e7c009110 con 0x7f0e800f8cb0 2024-02-23T00:42:49.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.577+0000 7f0e777fe700 1 --2- 172.21.15.47:0/344941712 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0e6c062cc0 0x7f0e6c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:49.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.578+0000 7f0e85bbe700 1 --2- 172.21.15.47:0/344941712 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0e6c062cc0 0x7f0e6c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:49.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.578+0000 7f0e777fe700 10 client.24809.objecter ms_handle_connect 0x7f0e6c062cc0 2024-02-23T00:42:49.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.578+0000 7f0e85bbe700 1 --2- 172.21.15.47:0/344941712 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0e6c062cc0 0x7f0e6c065180 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f0e800f8400 tx=0x7f0e7000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:49.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.579+0000 7f0e777fe700 1 -- 172.21.15.47:0/344941712 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(107..107 src has 48..107) v4 ==== 7851+0+0 (secure 0 0 0) 0x7f0e7c050020 con 0x7f0e800f8cb0 2024-02-23T00:42:49.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.579+0000 7f0e777fe700 10 client.24809.objecter ms_dispatch 0x7f0e80001550 osd_map(107..107 src has 48..107) v4 2024-02-23T00:42:49.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.579+0000 7f0e777fe700 3 client.24809.objecter handle_osd_map got epochs [107,107] > 0 2024-02-23T00:42:49.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.579+0000 7f0e777fe700 3 client.24809.objecter handle_osd_map decoding full epoch 107 2024-02-23T00:42:49.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.579+0000 7f0e777fe700 20 client.24809.objecter dump_active .. 0 homeless 2024-02-23T00:42:49.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.582+0000 7f0e777fe700 1 -- 172.21.15.47:0/344941712 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f0e7c081060 con 0x7f0e800f8cb0 2024-02-23T00:42:49.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:48.757+0000 7f0e88623700 1 -- 172.21.15.47:0/344941712 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "22"} v 0) v1 -- 0x7f0e8005ee80 con 0x7f0e800f8cb0 2024-02-23T00:42:49.040 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:42:49.383 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:42:49.388 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:49.388 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:42:49.489 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:42:49.522 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:49.543 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:42:49.545 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:49.546 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:42:49.647 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:42:49.676 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:49.695 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:42:49.698 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:49.698 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:42:49.799 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:42:49.835 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:49.835 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set noscrub 2024-02-23T00:42:49.857 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:42:49.860 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:49.860 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:42:49.886 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.1 dump_historic_ops 2024-02-23T00:42:49.962 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:42:49.992 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:50.012 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:42:50.015 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:50.015 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:42:50.116 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:42:50.144 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:50.343 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:42:50.346 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:50.346 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:42:50.394 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:42:50.448 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:42:50.476 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:50.599 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:42:50.606 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:50.607 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:42:50.707 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:42:50.739 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:50.761 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:42:50.763 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:50.764 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:42:50.806 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:42:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:42:50] "GET /metrics HTTP/1.1" 200 172258 "" "Prometheus/2.33.4" 2024-02-23T00:42:50.864 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:42:50.893 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:50.913 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:42:50.915 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:50.916 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:42:51.017 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:42:51.044 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:51.064 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:42:51.066 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:51.067 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:42:51.168 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:42:51.194 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:51.213 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:42:51.215 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:51.215 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:42:51.316 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:42:51.343 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:51.361 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:42:51.363 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:51.364 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:42:51.465 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:42:51.491 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:51.510 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:42:51.512 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:51.513 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:42:51.614 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:42:51.640 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:51.659 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:42:51.661 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:51.662 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:42:51.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.590+0000 7f414a0e7700 1 -- 172.21.15.47:0/1154935054 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f41440f8cb0 msgr2=0x7f41440f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:51.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.590+0000 7f414a0e7700 1 --2- 172.21.15.47:0/1154935054 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f41440f8cb0 0x7f41440f9090 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f4134004870 tx=0x7f4134009cb0).stop 2024-02-23T00:42:51.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.590+0000 7f414a0e7700 1 -- 172.21.15.47:0/1154935054 shutdown_connections 2024-02-23T00:42:51.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.590+0000 7f414a0e7700 1 --2- 172.21.15.47:0/1154935054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4144102210 0x7f4144104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:42:51.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.590+0000 7f414a0e7700 1 --2- 172.21.15.47:0/1154935054 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f41440f8cb0 0x7f41440f9090 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:51.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.590+0000 7f414a0e7700 1 --2- 172.21.15.47:0/1154935054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41440f9610 0x7f4144101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:51.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.590+0000 7f414a0e7700 1 -- 172.21.15.47:0/1154935054 >> 172.21.15.47:0/1154935054 conn(0x7f41440f2820 msgr2=0x7f41440f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:42:51.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.590+0000 7f414a0e7700 1 -- 172.21.15.47:0/1154935054 shutdown_connections 2024-02-23T00:42:51.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.590+0000 7f414a0e7700 1 -- 172.21.15.47:0/1154935054 wait complete. 2024-02-23T00:42:51.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.590+0000 7f414a0e7700 1 librados: starting msgr at 2024-02-23T00:42:51.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.590+0000 7f414a0e7700 1 librados: starting objecter 2024-02-23T00:42:51.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.591+0000 7f414a0e7700 1 Processor -- start 2024-02-23T00:42:51.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.591+0000 7f414a0e7700 1 -- start start 2024-02-23T00:42:51.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.591+0000 7f414a0e7700 1 librados: setting wanted keys 2024-02-23T00:42:51.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.591+0000 7f414a0e7700 1 librados: calling monclient init 2024-02-23T00:42:51.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.591+0000 7f414a0e7700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f41440f8cb0 0x7f414406a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:51.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.591+0000 7f414a0e7700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41440f9610 0x7f41440f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:51.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.591+0000 7f414a0e7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4144102210 0x7f41440f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:51.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.591+0000 7f414a0e7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414406acf0 con 0x7f4144102210 2024-02-23T00:42:51.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.591+0000 7f414a0e7700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41440f7640 con 0x7f41440f9610 2024-02-23T00:42:51.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.591+0000 7f414a0e7700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41440f7800 con 0x7f41440f8cb0 2024-02-23T00:42:51.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.591+0000 7f4143fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4144102210 0x7f41440f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:51.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.591+0000 7f4143fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4144102210 0x7f41440f7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58128/0 (socket says 172.21.15.47:58128) 2024-02-23T00:42:51.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.592+0000 7f4143fff700 1 -- 172.21.15.47:0/2224696171 learned_addr learned my addr 172.21.15.47:0/2224696171 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:42:51.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.592+0000 7f41437fe700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f41440f8cb0 0x7f414406a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:51.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.593+0000 7f41437fe700 1 -- 172.21.15.47:0/2224696171 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41440f9610 msgr2=0x7f41440f6c60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-23T00:42:51.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.593+0000 7f41437fe700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41440f9610 0x7f41440f6c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:42:51.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.593+0000 7f41437fe700 1 -- 172.21.15.47:0/2224696171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4144102210 msgr2=0x7f41440f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:51.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.593+0000 7f41437fe700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4144102210 0x7f41440f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:51.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.593+0000 7f41437fe700 1 -- 172.21.15.47:0/2224696171 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4134005040 con 0x7f41440f8cb0 2024-02-23T00:42:51.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.593+0000 7f4143fff700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4144102210 0x7f41440f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:42:51.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.593+0000 7f41437fe700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f41440f8cb0 0x7f414406a670 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f4134009da0 tx=0x7f4134009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:51.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.593+0000 7f4140ff9700 10 client.?.objecter ms_handle_connect 0x7f41440f8cb0 2024-02-23T00:42:51.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.593+0000 7f4140ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:42:51.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.594+0000 7f4140ff9700 1 -- 172.21.15.47:0/2224696171 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413400e030 con 0x7f41440f8cb0 2024-02-23T00:42:51.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.594+0000 7f4140ff9700 1 -- 172.21.15.47:0/2224696171 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f4134013df0 con 0x7f41440f8cb0 2024-02-23T00:42:51.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.594+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41440f7a80 con 0x7f41440f8cb0 2024-02-23T00:42:51.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.594+0000 7f4140ff9700 1 -- 172.21.15.47:0/2224696171 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4134018070 con 0x7f41440f8cb0 2024-02-23T00:42:51.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.594+0000 7f414a0e7700 10 client.24827.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:42:51.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.594+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f414411f900 con 0x7f41440f8cb0 2024-02-23T00:42:51.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.594+0000 7f414a0e7700 1 librados: init done 2024-02-23T00:42:51.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.596+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f414405fa60 con 0x7f41440f8cb0 2024-02-23T00:42:51.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.597+0000 7f4140ff9700 1 -- 172.21.15.47:0/2224696171 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f4134009240 con 0x7f41440f8cb0 2024-02-23T00:42:51.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.598+0000 7f4140ff9700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f412c062cc0 0x7f412c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:51.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.598+0000 7f4140ff9700 1 -- 172.21.15.47:0/2224696171 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(107..107 src has 48..107) v4 ==== 7851+0+0 (secure 0 0 0) 0x7f413407e8a0 con 0x7f41440f8cb0 2024-02-23T00:42:51.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.598+0000 7f4140ff9700 10 client.24827.objecter ms_dispatch 0x7f4144001550 osd_map(107..107 src has 48..107) v4 2024-02-23T00:42:51.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.598+0000 7f4140ff9700 3 client.24827.objecter handle_osd_map got epochs [107,107] > 0 2024-02-23T00:42:51.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.598+0000 7f4140ff9700 3 client.24827.objecter handle_osd_map decoding full epoch 107 2024-02-23T00:42:51.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.598+0000 7f4142ffd700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f412c062cc0 0x7f412c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:51.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.598+0000 7f4140ff9700 20 client.24827.objecter dump_active .. 0 homeless 2024-02-23T00:42:51.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.598+0000 7f4140ff9700 1 -- 172.21.15.47:0/2224696171 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v107) v1 ==== 54+0+16 (secure 0 0 0) 0x7f413407f0a0 con 0x7f41440f8cb0 2024-02-23T00:42:51.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.598+0000 7f414a0e7700 10 client.24827.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:51.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.598+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_assign -1 1 2024-02-23T00:42:51.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.598+0000 7f414a0e7700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f4144028540 0x7f41440fbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:51.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.598+0000 7f414a0e7700 20 client.24827.objecter _get_session s=0x7f414405ee80 osd=0 3 2024-02-23T00:42:51.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.598+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f414405ee80 osd=0 3 2024-02-23T00:42:51.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.598+0000 7f414a0e7700 20 client.24827.objecter _get_session s=0x7f414405ee80 osd=0 3 2024-02-23T00:42:51.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.598+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_remove -1 1 2024-02-23T00:42:51.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.598+0000 7f414a0e7700 20 client.24827.objecter get_session s=0x7f414405ee80 osd=0 3 2024-02-23T00:42:51.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.598+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_assign 0 1 2024-02-23T00:42:51.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.598+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f414405ee80 osd=0 4 2024-02-23T00:42:51.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.598+0000 7f414a0e7700 10 client.24827.objecter _send_command 1 2024-02-23T00:42:51.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.598+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4144107050 con 0x7f4144028540 2024-02-23T00:42:51.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.598+0000 7f4140ff9700 10 client.24827.objecter ms_handle_connect 0x7f412c062cc0 2024-02-23T00:42:51.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.598+0000 7f4142ffd700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f412c062cc0 0x7f412c065180 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f41440f8400 tx=0x7f413800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:51.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.598+0000 7f4143fff700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f4144028540 0x7f41440fbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:51.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.599+0000 7f4140ff9700 10 client.24827.objecter ms_handle_connect 0x7f4144028540 2024-02-23T00:42:51.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.599+0000 7f4143fff700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f4144028540 0x7f41440fbcd0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:51.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.600+0000 7f4140ff9700 1 -- 172.21.15.47:0/2224696171 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4144107050 con 0x7f4144028540 2024-02-23T00:42:51.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.600+0000 7f4140ff9700 10 client.24827.objecter ms_dispatch 0x7f4144001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:42:51.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.600+0000 7f4140ff9700 10 client.24827.objecter _finish_command 1 = system:0 2024-02-23T00:42:51.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.600+0000 7f4140ff9700 20 client.24827.objecter put_session s=0x7f414405ee80 osd=0 4 2024-02-23T00:42:51.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.600+0000 7f4140ff9700 15 client.24827.objecter _session_command_op_remove 0 1 2024-02-23T00:42:51.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.618+0000 7f414a0e7700 10 client.24827.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:42:51.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.618+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_assign -1 2 2024-02-23T00:42:51.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.618+0000 7f414a0e7700 20 client.24827.objecter _get_session s=0x7f414405ee80 osd=0 3 2024-02-23T00:42:51.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.618+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f414405ee80 osd=0 3 2024-02-23T00:42:51.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.618+0000 7f414a0e7700 20 client.24827.objecter _get_session s=0x7f414405ee80 osd=0 3 2024-02-23T00:42:51.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.618+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_remove -1 2 2024-02-23T00:42:51.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.618+0000 7f414a0e7700 20 client.24827.objecter get_session s=0x7f414405ee80 osd=0 3 2024-02-23T00:42:51.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.618+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_assign 0 2 2024-02-23T00:42:51.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.618+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f414405ee80 osd=0 4 2024-02-23T00:42:51.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.618+0000 7f414a0e7700 10 client.24827.objecter _send_command 2 2024-02-23T00:42:51.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.618+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f414405a040 con 0x7f4144028540 2024-02-23T00:42:51.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.621+0000 7f4140ff9700 1 -- 172.21.15.47:0/2224696171 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f414405a040 con 0x7f4144028540 2024-02-23T00:42:51.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.621+0000 7f4140ff9700 10 client.24827.objecter ms_dispatch 0x7f4144001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:42:51.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.621+0000 7f4140ff9700 10 client.24827.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:42:51.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.621+0000 7f4140ff9700 20 client.24827.objecter put_session s=0x7f414405ee80 osd=0 4 2024-02-23T00:42:51.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.621+0000 7f4140ff9700 15 client.24827.objecter _session_command_op_remove 0 2 2024-02-23T00:42:51.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.622+0000 7f414a0e7700 10 client.24827.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:51.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.622+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_assign -1 3 2024-02-23T00:42:51.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.622+0000 7f414a0e7700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f4144027b90 0x7f4144027f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:51.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.622+0000 7f414a0e7700 20 client.24827.objecter _get_session s=0x7f41440fd340 osd=1 3 2024-02-23T00:42:51.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.622+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f41440fd340 osd=1 3 2024-02-23T00:42:51.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.622+0000 7f414a0e7700 20 client.24827.objecter _get_session s=0x7f41440fd340 osd=1 3 2024-02-23T00:42:51.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.622+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_remove -1 3 2024-02-23T00:42:51.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.622+0000 7f414a0e7700 20 client.24827.objecter get_session s=0x7f41440fd340 osd=1 3 2024-02-23T00:42:51.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.622+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_assign 1 3 2024-02-23T00:42:51.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.622+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f41440fd340 osd=1 4 2024-02-23T00:42:51.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.622+0000 7f414a0e7700 10 client.24827.objecter _send_command 3 2024-02-23T00:42:51.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.622+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f41440fd4b0 con 0x7f4144027b90 2024-02-23T00:42:51.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.622+0000 7f41437fe700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f4144027b90 0x7f4144027f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:51.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.622+0000 7f4140ff9700 10 client.24827.objecter ms_handle_connect 0x7f4144027b90 2024-02-23T00:42:51.691 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:42:51.691 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:42:51.691 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:42:51.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.622+0000 7f41437fe700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f4144027b90 0x7f4144027f90 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:51.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.623+0000 7f4140ff9700 1 -- 172.21.15.47:0/2224696171 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f41440fd4b0 con 0x7f4144027b90 2024-02-23T00:42:51.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.623+0000 7f4140ff9700 10 client.24827.objecter ms_dispatch 0x7f4144001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:42:51.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.623+0000 7f4140ff9700 10 client.24827.objecter _finish_command 3 = system:0 2024-02-23T00:42:51.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.623+0000 7f4140ff9700 20 client.24827.objecter put_session s=0x7f41440fd340 osd=1 4 2024-02-23T00:42:51.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.623+0000 7f4140ff9700 15 client.24827.objecter _session_command_op_remove 1 3 2024-02-23T00:42:51.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.638+0000 7f414a0e7700 10 client.24827.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:42:51.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.638+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_assign -1 4 2024-02-23T00:42:51.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.638+0000 7f414a0e7700 20 client.24827.objecter _get_session s=0x7f41440fd340 osd=1 3 2024-02-23T00:42:51.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.638+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f41440fd340 osd=1 3 2024-02-23T00:42:51.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.638+0000 7f414a0e7700 20 client.24827.objecter _get_session s=0x7f41440fd340 osd=1 3 2024-02-23T00:42:51.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.638+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_remove -1 4 2024-02-23T00:42:51.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.638+0000 7f414a0e7700 20 client.24827.objecter get_session s=0x7f41440fd340 osd=1 3 2024-02-23T00:42:51.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.638+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_assign 1 4 2024-02-23T00:42:51.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.638+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f41440fd340 osd=1 4 2024-02-23T00:42:51.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.638+0000 7f414a0e7700 10 client.24827.objecter _send_command 4 2024-02-23T00:42:51.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.638+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f4144027600 con 0x7f4144027b90 2024-02-23T00:42:51.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.640+0000 7f4140ff9700 1 -- 172.21.15.47:0/2224696171 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f4144027600 con 0x7f4144027b90 2024-02-23T00:42:51.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.640+0000 7f4140ff9700 10 client.24827.objecter ms_dispatch 0x7f4144001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:42:51.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.640+0000 7f4140ff9700 10 client.24827.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:42:51.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.640+0000 7f4140ff9700 20 client.24827.objecter put_session s=0x7f41440fd340 osd=1 4 2024-02-23T00:42:51.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.640+0000 7f4140ff9700 15 client.24827.objecter _session_command_op_remove 1 4 2024-02-23T00:42:51.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.640+0000 7f414a0e7700 10 client.24827.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:51.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.640+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_assign -1 5 2024-02-23T00:42:51.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.640+0000 7f414a0e7700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f41440fcea0 0x7f4144126d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:51.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.640+0000 7f414a0e7700 20 client.24827.objecter _get_session s=0x7f41440279b0 osd=2 3 2024-02-23T00:42:51.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.640+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f41440279b0 osd=2 3 2024-02-23T00:42:51.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.640+0000 7f414a0e7700 20 client.24827.objecter _get_session s=0x7f41440279b0 osd=2 3 2024-02-23T00:42:51.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.640+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_remove -1 5 2024-02-23T00:42:51.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.640+0000 7f414a0e7700 20 client.24827.objecter get_session s=0x7f41440279b0 osd=2 3 2024-02-23T00:42:51.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.640+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_assign 2 5 2024-02-23T00:42:51.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.640+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f41440279b0 osd=2 4 2024-02-23T00:42:51.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.640+0000 7f414a0e7700 10 client.24827.objecter _send_command 5 2024-02-23T00:42:51.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.640+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f41441299b0 con 0x7f41440fcea0 2024-02-23T00:42:51.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.640+0000 7f4142ffd700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f41440fcea0 0x7f4144126d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:51.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.640+0000 7f4140ff9700 10 client.24827.objecter ms_handle_connect 0x7f41440fcea0 2024-02-23T00:42:51.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.640+0000 7f4142ffd700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f41440fcea0 0x7f4144126d60 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:51.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.642+0000 7f4140ff9700 1 -- 172.21.15.47:0/2224696171 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f41441299b0 con 0x7f41440fcea0 2024-02-23T00:42:51.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.642+0000 7f4140ff9700 10 client.24827.objecter ms_dispatch 0x7f4144001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:42:51.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.642+0000 7f4140ff9700 10 client.24827.objecter _finish_command 5 = system:0 2024-02-23T00:42:51.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.642+0000 7f4140ff9700 20 client.24827.objecter put_session s=0x7f41440279b0 osd=2 4 2024-02-23T00:42:51.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.642+0000 7f4140ff9700 15 client.24827.objecter _session_command_op_remove 2 5 2024-02-23T00:42:51.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.663+0000 7f414a0e7700 10 client.24827.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:42:51.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.663+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_assign -1 6 2024-02-23T00:42:51.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.663+0000 7f414a0e7700 20 client.24827.objecter _get_session s=0x7f41440279b0 osd=2 3 2024-02-23T00:42:51.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.663+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f41440279b0 osd=2 3 2024-02-23T00:42:51.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.663+0000 7f414a0e7700 20 client.24827.objecter _get_session s=0x7f41440279b0 osd=2 3 2024-02-23T00:42:51.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.663+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_remove -1 6 2024-02-23T00:42:51.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.663+0000 7f414a0e7700 20 client.24827.objecter get_session s=0x7f41440279b0 osd=2 3 2024-02-23T00:42:51.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.663+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_assign 2 6 2024-02-23T00:42:51.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.663+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f41440279b0 osd=2 4 2024-02-23T00:42:51.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.663+0000 7f414a0e7700 10 client.24827.objecter _send_command 6 2024-02-23T00:42:51.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.663+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f41440fc400 con 0x7f41440fcea0 2024-02-23T00:42:51.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.666+0000 7f4140ff9700 1 -- 172.21.15.47:0/2224696171 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f41440fc400 con 0x7f41440fcea0 2024-02-23T00:42:51.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.666+0000 7f4140ff9700 10 client.24827.objecter ms_dispatch 0x7f4144001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:42:51.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.666+0000 7f4140ff9700 10 client.24827.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:42:51.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.666+0000 7f4140ff9700 20 client.24827.objecter put_session s=0x7f41440279b0 osd=2 4 2024-02-23T00:42:51.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.666+0000 7f4140ff9700 15 client.24827.objecter _session_command_op_remove 2 6 2024-02-23T00:42:51.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.667+0000 7f414a0e7700 10 client.24827.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:51.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.667+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_assign -1 7 2024-02-23T00:42:51.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.667+0000 7f414a0e7700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f41441248b0 0x7f41441283d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:51.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.667+0000 7f414a0e7700 20 client.24827.objecter _get_session s=0x7f41440fc800 osd=3 3 2024-02-23T00:42:51.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.667+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f41440fc800 osd=3 3 2024-02-23T00:42:51.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.667+0000 7f414a0e7700 20 client.24827.objecter _get_session s=0x7f41440fc800 osd=3 3 2024-02-23T00:42:51.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.667+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_remove -1 7 2024-02-23T00:42:51.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.667+0000 7f414a0e7700 20 client.24827.objecter get_session s=0x7f41440fc800 osd=3 3 2024-02-23T00:42:51.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.667+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_assign 3 7 2024-02-23T00:42:51.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.667+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f41440fc800 osd=3 4 2024-02-23T00:42:51.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.667+0000 7f414a0e7700 10 client.24827.objecter _send_command 7 2024-02-23T00:42:51.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.667+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4144128a20 con 0x7f41441248b0 2024-02-23T00:42:51.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.667+0000 7f4143fff700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f41441248b0 0x7f41441283d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:51.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.667+0000 7f4140ff9700 10 client.24827.objecter ms_handle_connect 0x7f41441248b0 2024-02-23T00:42:51.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.667+0000 7f4143fff700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f41441248b0 0x7f41441283d0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:51.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.668+0000 7f4140ff9700 1 -- 172.21.15.47:0/2224696171 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4144128a20 con 0x7f41441248b0 2024-02-23T00:42:51.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.668+0000 7f4140ff9700 10 client.24827.objecter ms_dispatch 0x7f4144001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:42:51.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.668+0000 7f4140ff9700 10 client.24827.objecter _finish_command 7 = system:0 2024-02-23T00:42:51.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.668+0000 7f4140ff9700 20 client.24827.objecter put_session s=0x7f41440fc800 osd=3 4 2024-02-23T00:42:51.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.668+0000 7f4140ff9700 15 client.24827.objecter _session_command_op_remove 3 7 2024-02-23T00:42:51.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.683+0000 7f414a0e7700 10 client.24827.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:42:51.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.683+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_assign -1 8 2024-02-23T00:42:51.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.683+0000 7f414a0e7700 20 client.24827.objecter _get_session s=0x7f41440fc800 osd=3 3 2024-02-23T00:42:51.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.683+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f41440fc800 osd=3 3 2024-02-23T00:42:51.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.683+0000 7f414a0e7700 20 client.24827.objecter _get_session s=0x7f41440fc800 osd=3 3 2024-02-23T00:42:51.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.683+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_remove -1 8 2024-02-23T00:42:51.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.683+0000 7f414a0e7700 20 client.24827.objecter get_session s=0x7f41440fc800 osd=3 3 2024-02-23T00:42:51.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.683+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_assign 3 8 2024-02-23T00:42:51.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.683+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f41440fc800 osd=3 4 2024-02-23T00:42:51.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.683+0000 7f414a0e7700 10 client.24827.objecter _send_command 8 2024-02-23T00:42:51.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.683+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f4144123e10 con 0x7f41441248b0 2024-02-23T00:42:51.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.684+0000 7f4140ff9700 1 -- 172.21.15.47:0/2224696171 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f4144123e10 con 0x7f41441248b0 2024-02-23T00:42:51.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.684+0000 7f4140ff9700 10 client.24827.objecter ms_dispatch 0x7f4144001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:42:51.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.684+0000 7f4140ff9700 10 client.24827.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:42:51.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.684+0000 7f4140ff9700 20 client.24827.objecter put_session s=0x7f41440fc800 osd=3 4 2024-02-23T00:42:51.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.684+0000 7f4140ff9700 15 client.24827.objecter _session_command_op_remove 3 8 2024-02-23T00:42:51.698 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:42:51.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.685+0000 7f414a0e7700 10 client.24827.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:51.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.685+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_assign -1 9 2024-02-23T00:42:51.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.685+0000 7f414a0e7700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f414412bb30 0x7f414412df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:51.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.685+0000 7f414a0e7700 20 client.24827.objecter _get_session s=0x7f4144129620 osd=4 3 2024-02-23T00:42:51.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.685+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f4144129620 osd=4 3 2024-02-23T00:42:51.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.685+0000 7f414a0e7700 20 client.24827.objecter _get_session s=0x7f4144129620 osd=4 3 2024-02-23T00:42:51.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.685+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_remove -1 9 2024-02-23T00:42:51.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.685+0000 7f414a0e7700 20 client.24827.objecter get_session s=0x7f4144129620 osd=4 3 2024-02-23T00:42:51.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.685+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_assign 4 9 2024-02-23T00:42:51.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.685+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f4144129620 osd=4 4 2024-02-23T00:42:51.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.685+0000 7f414a0e7700 10 client.24827.objecter _send_command 9 2024-02-23T00:42:51.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.685+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f41441291b0 con 0x7f414412bb30 2024-02-23T00:42:51.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.685+0000 7f41437fe700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f414412bb30 0x7f414412df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:51.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.685+0000 7f4140ff9700 10 client.24827.objecter ms_handle_connect 0x7f414412bb30 2024-02-23T00:42:51.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.685+0000 7f41437fe700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f414412bb30 0x7f414412df20 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:51.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.686+0000 7f4140ff9700 1 -- 172.21.15.47:0/2224696171 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f41441291b0 con 0x7f414412bb30 2024-02-23T00:42:51.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.686+0000 7f4140ff9700 10 client.24827.objecter ms_dispatch 0x7f4144001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:42:51.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.686+0000 7f4140ff9700 10 client.24827.objecter _finish_command 9 = system:0 2024-02-23T00:42:51.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.686+0000 7f4140ff9700 20 client.24827.objecter put_session s=0x7f4144129620 osd=4 4 2024-02-23T00:42:51.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.686+0000 7f4140ff9700 15 client.24827.objecter _session_command_op_remove 4 9 2024-02-23T00:42:51.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.701+0000 7f414a0e7700 10 client.24827.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:42:51.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.701+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_assign -1 10 2024-02-23T00:42:51.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.701+0000 7f414a0e7700 20 client.24827.objecter _get_session s=0x7f4144129620 osd=4 3 2024-02-23T00:42:51.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.701+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f4144129620 osd=4 3 2024-02-23T00:42:51.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.701+0000 7f414a0e7700 20 client.24827.objecter _get_session s=0x7f4144129620 osd=4 3 2024-02-23T00:42:51.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.701+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_remove -1 10 2024-02-23T00:42:51.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.701+0000 7f414a0e7700 20 client.24827.objecter get_session s=0x7f4144129620 osd=4 3 2024-02-23T00:42:51.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.701+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_assign 4 10 2024-02-23T00:42:51.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.701+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f4144129620 osd=4 4 2024-02-23T00:42:51.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.701+0000 7f414a0e7700 10 client.24827.objecter _send_command 10 2024-02-23T00:42:51.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.701+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f41441291b0 con 0x7f414412bb30 2024-02-23T00:42:51.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.703+0000 7f4140ff9700 1 -- 172.21.15.47:0/2224696171 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f41441291b0 con 0x7f414412bb30 2024-02-23T00:42:51.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.703+0000 7f4140ff9700 10 client.24827.objecter ms_dispatch 0x7f4144001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:42:51.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.703+0000 7f4140ff9700 10 client.24827.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:42:51.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.703+0000 7f4140ff9700 20 client.24827.objecter put_session s=0x7f4144129620 osd=4 4 2024-02-23T00:42:51.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.703+0000 7f4140ff9700 15 client.24827.objecter _session_command_op_remove 4 10 2024-02-23T00:42:51.705 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:42:51.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.703+0000 7f414a0e7700 10 client.24827.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:51.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.703+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_assign -1 11 2024-02-23T00:42:51.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.703+0000 7f414a0e7700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f414412e920 0x7f4144130d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:51.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.703+0000 7f414a0e7700 20 client.24827.objecter _get_session s=0x7f41441291b0 osd=5 3 2024-02-23T00:42:51.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.703+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f41441291b0 osd=5 3 2024-02-23T00:42:51.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.703+0000 7f414a0e7700 20 client.24827.objecter _get_session s=0x7f41441291b0 osd=5 3 2024-02-23T00:42:51.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.703+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_remove -1 11 2024-02-23T00:42:51.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.703+0000 7f414a0e7700 20 client.24827.objecter get_session s=0x7f41441291b0 osd=5 3 2024-02-23T00:42:51.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.703+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_assign 5 11 2024-02-23T00:42:51.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.703+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f41441291b0 osd=5 4 2024-02-23T00:42:51.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.703+0000 7f414a0e7700 10 client.24827.objecter _send_command 11 2024-02-23T00:42:51.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.703+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4144133b70 con 0x7f414412e920 2024-02-23T00:42:51.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.703+0000 7f4142ffd700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f414412e920 0x7f4144130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:51.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.704+0000 7f4140ff9700 10 client.24827.objecter ms_handle_connect 0x7f414412e920 2024-02-23T00:42:51.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.704+0000 7f4142ffd700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f414412e920 0x7f4144130d70 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:51.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.704+0000 7f4140ff9700 1 -- 172.21.15.47:0/2224696171 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4144133b70 con 0x7f414412e920 2024-02-23T00:42:51.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.704+0000 7f4140ff9700 10 client.24827.objecter ms_dispatch 0x7f4144001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:42:51.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.704+0000 7f4140ff9700 10 client.24827.objecter _finish_command 11 = system:0 2024-02-23T00:42:51.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.704+0000 7f4140ff9700 20 client.24827.objecter put_session s=0x7f41441291b0 osd=5 4 2024-02-23T00:42:51.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.704+0000 7f4140ff9700 15 client.24827.objecter _session_command_op_remove 5 11 2024-02-23T00:42:51.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.740+0000 7f414a0e7700 10 client.24827.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:42:51.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.740+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_assign -1 12 2024-02-23T00:42:51.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.740+0000 7f414a0e7700 20 client.24827.objecter _get_session s=0x7f41441291b0 osd=5 3 2024-02-23T00:42:51.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.740+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f41441291b0 osd=5 3 2024-02-23T00:42:51.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.740+0000 7f414a0e7700 20 client.24827.objecter _get_session s=0x7f41441291b0 osd=5 3 2024-02-23T00:42:51.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.740+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_remove -1 12 2024-02-23T00:42:51.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.740+0000 7f414a0e7700 20 client.24827.objecter get_session s=0x7f41441291b0 osd=5 3 2024-02-23T00:42:51.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.740+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_assign 5 12 2024-02-23T00:42:51.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.740+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f41441291b0 osd=5 4 2024-02-23T00:42:51.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.740+0000 7f414a0e7700 10 client.24827.objecter _send_command 12 2024-02-23T00:42:51.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.740+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f4144133b70 con 0x7f414412e920 2024-02-23T00:42:51.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.742+0000 7f4140ff9700 1 -- 172.21.15.47:0/2224696171 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f4144133b70 con 0x7f414412e920 2024-02-23T00:42:51.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.742+0000 7f4140ff9700 10 client.24827.objecter ms_dispatch 0x7f4144001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:42:51.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.742+0000 7f4140ff9700 10 client.24827.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:42:51.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.742+0000 7f4140ff9700 20 client.24827.objecter put_session s=0x7f41441291b0 osd=5 4 2024-02-23T00:42:51.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.742+0000 7f4140ff9700 15 client.24827.objecter _session_command_op_remove 5 12 2024-02-23T00:42:51.744 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:42:51.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.742+0000 7f414a0e7700 10 client.24827.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:51.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.742+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_assign -1 13 2024-02-23T00:42:51.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.742+0000 7f414a0e7700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f4144132ee0 0x7f4144133340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:51.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.742+0000 7f414a0e7700 20 client.24827.objecter _get_session s=0x7f4144133b70 osd=6 3 2024-02-23T00:42:51.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.742+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f4144133b70 osd=6 3 2024-02-23T00:42:51.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.742+0000 7f414a0e7700 20 client.24827.objecter _get_session s=0x7f4144133b70 osd=6 3 2024-02-23T00:42:51.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.742+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_remove -1 13 2024-02-23T00:42:51.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.742+0000 7f414a0e7700 20 client.24827.objecter get_session s=0x7f4144133b70 osd=6 3 2024-02-23T00:42:51.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.742+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_assign 6 13 2024-02-23T00:42:51.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.742+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f4144133b70 osd=6 4 2024-02-23T00:42:51.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.742+0000 7f414a0e7700 10 client.24827.objecter _send_command 13 2024-02-23T00:42:51.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.742+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4144136f20 con 0x7f4144132ee0 2024-02-23T00:42:51.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.743+0000 7f4143fff700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f4144132ee0 0x7f4144133340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:51.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.743+0000 7f4140ff9700 10 client.24827.objecter ms_handle_connect 0x7f4144132ee0 2024-02-23T00:42:51.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.743+0000 7f4143fff700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f4144132ee0 0x7f4144133340 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:51.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.744+0000 7f4140ff9700 1 -- 172.21.15.47:0/2224696171 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4144136f20 con 0x7f4144132ee0 2024-02-23T00:42:51.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.744+0000 7f4140ff9700 10 client.24827.objecter ms_dispatch 0x7f4144001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:42:51.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.744+0000 7f4140ff9700 10 client.24827.objecter _finish_command 13 = system:0 2024-02-23T00:42:51.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.744+0000 7f4140ff9700 20 client.24827.objecter put_session s=0x7f4144133b70 osd=6 4 2024-02-23T00:42:51.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.744+0000 7f4140ff9700 15 client.24827.objecter _session_command_op_remove 6 13 2024-02-23T00:42:51.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.758+0000 7f414a0e7700 10 client.24827.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:42:51.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.758+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_assign -1 14 2024-02-23T00:42:51.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.758+0000 7f414a0e7700 20 client.24827.objecter _get_session s=0x7f4144133b70 osd=6 3 2024-02-23T00:42:51.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.758+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f4144133b70 osd=6 3 2024-02-23T00:42:51.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.758+0000 7f414a0e7700 20 client.24827.objecter _get_session s=0x7f4144133b70 osd=6 3 2024-02-23T00:42:51.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.758+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_remove -1 14 2024-02-23T00:42:51.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.758+0000 7f414a0e7700 20 client.24827.objecter get_session s=0x7f4144133b70 osd=6 3 2024-02-23T00:42:51.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.758+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_assign 6 14 2024-02-23T00:42:51.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.758+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f4144133b70 osd=6 4 2024-02-23T00:42:51.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.758+0000 7f414a0e7700 10 client.24827.objecter _send_command 14 2024-02-23T00:42:51.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.758+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f4144131370 con 0x7f4144132ee0 2024-02-23T00:42:51.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.760+0000 7f4140ff9700 1 -- 172.21.15.47:0/2224696171 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f4144131370 con 0x7f4144132ee0 2024-02-23T00:42:51.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.760+0000 7f4140ff9700 10 client.24827.objecter ms_dispatch 0x7f4144001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:42:51.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.760+0000 7f4140ff9700 10 client.24827.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:42:51.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.760+0000 7f4140ff9700 20 client.24827.objecter put_session s=0x7f4144133b70 osd=6 4 2024-02-23T00:42:51.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.760+0000 7f4140ff9700 15 client.24827.objecter _session_command_op_remove 6 14 2024-02-23T00:42:51.762 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:42:51.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.760+0000 7f414a0e7700 10 client.24827.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:51.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.760+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_assign -1 15 2024-02-23T00:42:51.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.760+0000 7f414a0e7700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f41441319f0 0x7f4144131e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:51.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.760+0000 7f414a0e7700 20 client.24827.objecter _get_session s=0x7f4144131370 osd=7 3 2024-02-23T00:42:51.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.760+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f4144131370 osd=7 3 2024-02-23T00:42:51.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.760+0000 7f414a0e7700 20 client.24827.objecter _get_session s=0x7f4144131370 osd=7 3 2024-02-23T00:42:51.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.760+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_remove -1 15 2024-02-23T00:42:51.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.760+0000 7f414a0e7700 20 client.24827.objecter get_session s=0x7f4144131370 osd=7 3 2024-02-23T00:42:51.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.760+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_assign 7 15 2024-02-23T00:42:51.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.760+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f4144131370 osd=7 4 2024-02-23T00:42:51.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.760+0000 7f414a0e7700 10 client.24827.objecter _send_command 15 2024-02-23T00:42:51.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.760+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f41441324e0 con 0x7f41441319f0 2024-02-23T00:42:51.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.761+0000 7f41437fe700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f41441319f0 0x7f4144131e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:51.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.761+0000 7f4140ff9700 10 client.24827.objecter ms_handle_connect 0x7f41441319f0 2024-02-23T00:42:51.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.761+0000 7f41437fe700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f41441319f0 0x7f4144131e50 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:51.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.762+0000 7f4140ff9700 1 -- 172.21.15.47:0/2224696171 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f41441324e0 con 0x7f41441319f0 2024-02-23T00:42:51.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.762+0000 7f4140ff9700 10 client.24827.objecter ms_dispatch 0x7f4144001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:42:51.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.762+0000 7f4140ff9700 10 client.24827.objecter _finish_command 15 = system:0 2024-02-23T00:42:51.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.762+0000 7f4140ff9700 20 client.24827.objecter put_session s=0x7f4144131370 osd=7 4 2024-02-23T00:42:51.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.762+0000 7f4140ff9700 15 client.24827.objecter _session_command_op_remove 7 15 2024-02-23T00:42:51.764 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:42:51.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.777+0000 7f414a0e7700 10 client.24827.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:42:51.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.777+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_assign -1 16 2024-02-23T00:42:51.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.777+0000 7f414a0e7700 20 client.24827.objecter _get_session s=0x7f4144131370 osd=7 3 2024-02-23T00:42:51.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.777+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f4144131370 osd=7 3 2024-02-23T00:42:51.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.777+0000 7f414a0e7700 20 client.24827.objecter _get_session s=0x7f4144131370 osd=7 3 2024-02-23T00:42:51.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.777+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_remove -1 16 2024-02-23T00:42:51.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.777+0000 7f414a0e7700 20 client.24827.objecter get_session s=0x7f4144131370 osd=7 3 2024-02-23T00:42:51.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.777+0000 7f414a0e7700 15 client.24827.objecter _session_command_op_assign 7 16 2024-02-23T00:42:51.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.777+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f4144131370 osd=7 4 2024-02-23T00:42:51.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.777+0000 7f414a0e7700 10 client.24827.objecter _send_command 16 2024-02-23T00:42:51.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.777+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f41441324e0 con 0x7f41441319f0 2024-02-23T00:42:51.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f4140ff9700 1 -- 172.21.15.47:0/2224696171 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f41441324e0 con 0x7f41441319f0 2024-02-23T00:42:51.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f4140ff9700 10 client.24827.objecter ms_dispatch 0x7f4144001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:42:51.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f4140ff9700 10 client.24827.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:42:51.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f4140ff9700 20 client.24827.objecter put_session s=0x7f4144131370 osd=7 4 2024-02-23T00:42:51.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f4140ff9700 15 client.24827.objecter _session_command_op_remove 7 16 2024-02-23T00:42:51.780 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:42:51.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 10 librados: watch_flush enter 2024-02-23T00:42:51.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 10 librados: watch_flush exit 2024-02-23T00:42:51.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 10 client.24827.objecter close_session for osd.0 2024-02-23T00:42:51.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f4144028540 msgr2=0x7f41440fbcd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:51.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f4144028540 0x7f41440fbcd0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:51.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f414405ee80 osd=0 1 2024-02-23T00:42:51.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 10 client.24827.objecter close_session for osd.1 2024-02-23T00:42:51.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f4144027b90 msgr2=0x7f4144027f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:51.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f4144027b90 0x7f4144027f90 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:51.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f41440fd340 osd=1 1 2024-02-23T00:42:51.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 10 client.24827.objecter close_session for osd.2 2024-02-23T00:42:51.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f41440fcea0 msgr2=0x7f4144126d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:51.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f41440fcea0 0x7f4144126d60 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:51.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f41440279b0 osd=2 1 2024-02-23T00:42:51.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 10 client.24827.objecter close_session for osd.3 2024-02-23T00:42:51.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f41441248b0 msgr2=0x7f41441283d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:51.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f41441248b0 0x7f41441283d0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:51.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f41440fc800 osd=3 1 2024-02-23T00:42:51.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f4143fff700 1 -- 172.21.15.47:0/2224696171 reap_dead start 2024-02-23T00:42:51.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 10 client.24827.objecter close_session for osd.4 2024-02-23T00:42:51.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f414412bb30 msgr2=0x7f414412df20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:51.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f414412bb30 0x7f414412df20 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:51.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f4144129620 osd=4 1 2024-02-23T00:42:51.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 10 client.24827.objecter close_session for osd.5 2024-02-23T00:42:51.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f4143fff700 1 -- 172.21.15.47:0/2224696171 reap_dead start 2024-02-23T00:42:51.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f414412e920 msgr2=0x7f4144130d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:51.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f414412e920 0x7f4144130d70 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:51.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f41441291b0 osd=5 1 2024-02-23T00:42:51.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 10 client.24827.objecter close_session for osd.6 2024-02-23T00:42:51.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f4144132ee0 msgr2=0x7f4144133340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:51.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f4144132ee0 0x7f4144133340 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:51.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f4144133b70 osd=6 1 2024-02-23T00:42:51.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 10 client.24827.objecter close_session for osd.7 2024-02-23T00:42:51.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f41441319f0 msgr2=0x7f4144131e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:51.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f41441319f0 0x7f4144131e50 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:51.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 20 client.24827.objecter put_session s=0x7f4144131370 osd=7 1 2024-02-23T00:42:51.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 20 client.24827.objecter shutdown clearing up homeless session... 2024-02-23T00:42:51.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 10 client.24827.objecter successfully canceled tick 2024-02-23T00:42:51.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f412c062cc0 msgr2=0x7f412c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:51.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f412c062cc0 0x7f412c065180 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f41440f8400 tx=0x7f413800b040).stop 2024-02-23T00:42:51.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f41440f8cb0 msgr2=0x7f414406a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:51.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f414a0e7700 1 --2- 172.21.15.47:0/2224696171 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f41440f8cb0 0x7f414406a670 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f4134009da0 tx=0x7f4134009cb0).stop 2024-02-23T00:42:51.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.779+0000 7f4143fff700 1 -- 172.21.15.47:0/2224696171 reap_dead start 2024-02-23T00:42:51.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.780+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 shutdown_connections 2024-02-23T00:42:51.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.780+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 >> 172.21.15.47:0/2224696171 conn(0x7f41440f2820 msgr2=0x7f41440f34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:42:51.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.780+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 shutdown_connections 2024-02-23T00:42:51.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.780+0000 7f414a0e7700 1 -- 172.21.15.47:0/2224696171 wait complete. 2024-02-23T00:42:51.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:51.780+0000 7f414a0e7700 1 librados: shutdown 2024-02-23T00:42:51.786 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:42:51.786 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:42:51.786 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:42:51.786 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:42:51.786 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:42:51.786 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:42:51.786 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:42:51.786 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:42:51.791 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:51.814 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:42:51.814 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:51.815 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:42:51.915 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:42:51.946 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:52.071 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:42:52.074 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:52.074 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:42:52.175 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:42:52.204 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:52.223 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:42:52.225 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:52.225 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:42:52.326 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:42:52.358 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:52.527 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:42:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:42:52.943 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:42:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:42:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:42:53.229 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:42:53.233 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:53.233 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:42:53.334 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:42:53.365 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:53.403 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:42:53.408 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:47.170887+0000 mgr.y (mgr.14517) 140 : cluster [DBG] pgmap v129: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 0 B/s, 1 objects/s recovering 2024-02-23T00:42:53.408 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:47.300686+0000 mon.a (mon.0) 800 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:42:53.408 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:47.307236+0000 mon.b (mon.1) 36 : audit [INF] from='client.? 172.21.15.47:0/2670125835' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:42:53.408 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:47.348952+0000 osd.0 (osd.0) 63 : cluster [DBG] 5.5 scrub starts 2024-02-23T00:42:53.408 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:47.349841+0000 osd.0 (osd.0) 64 : cluster [DBG] 5.5 scrub ok 2024-02-23T00:42:53.408 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:47.766004+0000 mon.a (mon.0) 801 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:42:53.409 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:47.170887+0000 mgr.y (mgr.14517) 140 : cluster [DBG] pgmap v129: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 0 B/s, 1 objects/s recovering 2024-02-23T00:42:53.409 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:47.300686+0000 mon.a (mon.0) 800 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:42:53.409 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:47.307236+0000 mon.b (mon.1) 36 : audit [INF] from='client.? 172.21.15.47:0/2670125835' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:42:53.409 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:47.348952+0000 osd.0 (osd.0) 63 : cluster [DBG] 5.5 scrub starts 2024-02-23T00:42:53.409 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:47.349841+0000 osd.0 (osd.0) 64 : cluster [DBG] 5.5 scrub ok 2024-02-23T00:42:53.409 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:47.766004+0000 mon.a (mon.0) 801 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:42:53.409 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:53.410 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:42:53.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:47.170887+0000 mgr.y (mgr.14517) 140 : cluster [DBG] pgmap v129: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 0 B/s, 1 objects/s recovering 2024-02-23T00:42:53.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:47.300686+0000 mon.a (mon.0) 800 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:42:53.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:47.307236+0000 mon.b (mon.1) 36 : audit [INF] from='client.? 172.21.15.47:0/2670125835' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:42:53.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:47.348952+0000 osd.0 (osd.0) 63 : cluster [DBG] 5.5 scrub starts 2024-02-23T00:42:53.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:47.349841+0000 osd.0 (osd.0) 64 : cluster [DBG] 5.5 scrub ok 2024-02-23T00:42:53.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:47.766004+0000 mon.a (mon.0) 801 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:42:53.510 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:42:53.542 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:53.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:53.576+0000 7f0e87e22700 10 client.24809.objecter tick 2024-02-23T00:42:53.779 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:42:53.791 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:42:53.794 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:53.794 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:42:53.802 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:42:53.802 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:42:53.895 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:42:53.927 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:53.948 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:42:53.951 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:53.951 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:42:54.052 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:42:54.081 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:54.100 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:42:54.102 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:54.103 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:42:54.204 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:42:54.234 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:54.767 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:42:54.801 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:42:54.804 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:54.804 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:42:54.905 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:42:54.933 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:54.959 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:42:54.963 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:54.963 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:42:55.064 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:42:55.096 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:55.117 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:42:55.120 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:55.121 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:42:55.221 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:42:55.252 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:55.350 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:42:55.352 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:55.353 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:42:55.454 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:42:55.481 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:55.790 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.2 dump_ops_in_flight 2024-02-23T00:42:55.833 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:42:55.838 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:55.838 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:42:55.939 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:42:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:56.074 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:42:56.077 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:56.077 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:42:56.178 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:42:56.208 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:56.228 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:42:56.231 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:56.232 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:42:56.333 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:42:56.364 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:56.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.379+0000 7f844b8c5700 1 -- 172.21.15.47:0/2254919718 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f84440fcbb0 msgr2=0x7f84440fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:56.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.379+0000 7f844b8c5700 1 --2- 172.21.15.47:0/2254919718 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f84440fcbb0 0x7f84440fcf90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f8434004870 tx=0x7f8434009cb0).stop 2024-02-23T00:42:56.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.380+0000 7f844b8c5700 1 -- 172.21.15.47:0/2254919718 shutdown_connections 2024-02-23T00:42:56.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.380+0000 7f844b8c5700 1 --2- 172.21.15.47:0/2254919718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84440f7400 0x7f8444103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:56.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.380+0000 7f844b8c5700 1 --2- 172.21.15.47:0/2254919718 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f84440fcbb0 0x7f84440fcf90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:56.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.380+0000 7f844b8c5700 1 --2- 172.21.15.47:0/2254919718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84440fd510 0x7f84440f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:56.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.380+0000 7f844b8c5700 1 -- 172.21.15.47:0/2254919718 >> 172.21.15.47:0/2254919718 conn(0x7f84440f2820 msgr2=0x7f84440f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:42:56.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.380+0000 7f844b8c5700 1 -- 172.21.15.47:0/2254919718 shutdown_connections 2024-02-23T00:42:56.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.380+0000 7f844b8c5700 1 -- 172.21.15.47:0/2254919718 wait complete. 2024-02-23T00:42:56.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.381+0000 7f844b8c5700 1 librados: starting msgr at 2024-02-23T00:42:56.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.381+0000 7f844b8c5700 1 librados: starting objecter 2024-02-23T00:42:56.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.381+0000 7f844b8c5700 1 Processor -- start 2024-02-23T00:42:56.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.381+0000 7f844b8c5700 1 -- start start 2024-02-23T00:42:56.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.381+0000 7f844b8c5700 1 librados: setting wanted keys 2024-02-23T00:42:56.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.381+0000 7f844b8c5700 1 librados: calling monclient init 2024-02-23T00:42:56.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.382+0000 7f844b8c5700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f84440f7400 0x7f8444106050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:56.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.382+0000 7f844b8c5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84440fcbb0 0x7f8444106540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:56.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.382+0000 7f844b8c5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84440fd510 0x7f8444106a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:56.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.382+0000 7f844b8c5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844410a120 con 0x7f84440fd510 2024-02-23T00:42:56.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.382+0000 7f844b8c5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844410a2a0 con 0x7f84440fcbb0 2024-02-23T00:42:56.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.382+0000 7f844b8c5700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f844410a460 con 0x7f84440f7400 2024-02-23T00:42:56.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.382+0000 7f8449661700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f84440f7400 0x7f8444106050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:56.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.382+0000 7f8449661700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f84440f7400 0x7f8444106050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:39508/0 (socket says 172.21.15.47:39508) 2024-02-23T00:42:56.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.382+0000 7f8449661700 1 -- 172.21.15.47:0/1523089110 learned_addr learned my addr 172.21.15.47:0/1523089110 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:42:56.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.382+0000 7f8448e60700 1 --2- 172.21.15.47:0/1523089110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84440fcbb0 0x7f8444106540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:56.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.382+0000 7f8449661700 1 -- 172.21.15.47:0/1523089110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84440fcbb0 msgr2=0x7f8444106540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:56.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.382+0000 7f8449661700 1 --2- 172.21.15.47:0/1523089110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84440fcbb0 0x7f8444106540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:56.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.382+0000 7f8449661700 1 -- 172.21.15.47:0/1523089110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84440fd510 msgr2=0x7f8444106a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:56.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.382+0000 7f8449661700 1 --2- 172.21.15.47:0/1523089110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84440fd510 0x7f8444106a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:42:56.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.382+0000 7f8449661700 1 -- 172.21.15.47:0/1523089110 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8434005040 con 0x7f84440f7400 2024-02-23T00:42:56.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.383+0000 7f8449661700 1 --2- 172.21.15.47:0/1523089110 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f84440f7400 0x7f8444106050 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f8434013680 tx=0x7f8434009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:56.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.383+0000 7f843a7fc700 10 client.?.objecter ms_handle_connect 0x7f84440f7400 2024-02-23T00:42:56.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.383+0000 7f843a7fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:42:56.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.383+0000 7f843a7fc700 1 -- 172.21.15.47:0/1523089110 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843400e030 con 0x7f84440f7400 2024-02-23T00:42:56.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.383+0000 7f843a7fc700 1 -- 172.21.15.47:0/1523089110 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f8434013e70 con 0x7f84440f7400 2024-02-23T00:42:56.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.383+0000 7f844b8c5700 1 -- 172.21.15.47:0/1523089110 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f844410a6e0 con 0x7f84440f7400 2024-02-23T00:42:56.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.383+0000 7f843a7fc700 1 -- 172.21.15.47:0/1523089110 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8434018070 con 0x7f84440f7400 2024-02-23T00:42:56.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.383+0000 7f844b8c5700 10 client.24914.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:42:56.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.384+0000 7f844b8c5700 1 -- 172.21.15.47:0/1523089110 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f844410ab70 con 0x7f84440f7400 2024-02-23T00:42:56.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.384+0000 7f844b8c5700 1 librados: init done 2024-02-23T00:42:56.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.384+0000 7f843a7fc700 1 -- 172.21.15.47:0/1523089110 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f843401c3c0 con 0x7f84440f7400 2024-02-23T00:42:56.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.385+0000 7f843a7fc700 1 --2- 172.21.15.47:0/1523089110 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8430062cc0 0x7f8430065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:56.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.385+0000 7f844b8c5700 1 -- 172.21.15.47:0/1523089110 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f844405fa60 con 0x7f84440f7400 2024-02-23T00:42:56.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.385+0000 7f843a7fc700 1 -- 172.21.15.47:0/1523089110 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(108..108 src has 48..108) v4 ==== 12299+0+0 (secure 0 0 0) 0x7f8434081170 con 0x7f84440f7400 2024-02-23T00:42:56.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.385+0000 7f843a7fc700 10 client.24914.objecter ms_dispatch 0x7f8444001550 osd_map(108..108 src has 48..108) v4 2024-02-23T00:42:56.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.385+0000 7f843a7fc700 3 client.24914.objecter handle_osd_map got epochs [108,108] > 0 2024-02-23T00:42:56.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.385+0000 7f843a7fc700 3 client.24914.objecter handle_osd_map decoding full epoch 108 2024-02-23T00:42:56.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.385+0000 7f8448e60700 1 --2- 172.21.15.47:0/1523089110 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8430062cc0 0x7f8430065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:56.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.385+0000 7f843a7fc700 20 client.24914.objecter dump_active .. 0 homeless 2024-02-23T00:42:56.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.385+0000 7f843a7fc700 10 client.24914.objecter ms_handle_connect 0x7f8430062cc0 2024-02-23T00:42:56.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.385+0000 7f8448e60700 1 --2- 172.21.15.47:0/1523089110 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8430062cc0 0x7f8430065180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f84441077f0 tx=0x7f844000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:56.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.392+0000 7f843a7fc700 1 -- 172.21.15.47:0/1523089110 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f8434082060 con 0x7f84440f7400 2024-02-23T00:42:56.400 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:42:56.403 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:56.403 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:42:56.428 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:42:56.439 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:42:56.505 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:42:56.537 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:56.562 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:42:56.565 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:56.565 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:42:56.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:56.593+0000 7f844b8c5700 1 -- 172.21.15.47:0/1523089110 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7f844405ee80 con 0x7f84440f7400 2024-02-23T00:42:56.666 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:42:56.695 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:56.720 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:42:56.721 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:56.721 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:42:56.822 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:42:56.851 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:57.479 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:42:57.482 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:57.482 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:42:57.583 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:42:57.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:57.593+0000 7f0e777fe700 1 -- 172.21.15.47:0/344941712 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "22"}]=0 set pool 6 pg_num to 22 v109) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0e7c081050 con 0x7f0e800f8cb0 2024-02-23T00:42:57.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:57.596+0000 7f0e88623700 10 librados: watch_flush enter 2024-02-23T00:42:57.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:57.596+0000 7f0e88623700 10 librados: watch_flush exit 2024-02-23T00:42:57.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:57.596+0000 7f0e88623700 20 client.24809.objecter shutdown clearing up homeless session... 2024-02-23T00:42:57.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:57.596+0000 7f0e88623700 10 client.24809.objecter successfully canceled tick 2024-02-23T00:42:57.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:57.596+0000 7f0e88623700 1 -- 172.21.15.47:0/344941712 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0e6c062cc0 msgr2=0x7f0e6c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:57.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:57.596+0000 7f0e88623700 1 --2- 172.21.15.47:0/344941712 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0e6c062cc0 0x7f0e6c065180 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f0e800f8400 tx=0x7f0e7000b040).stop 2024-02-23T00:42:57.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:57.596+0000 7f0e88623700 1 -- 172.21.15.47:0/344941712 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0e800f8cb0 msgr2=0x7f0e800f6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:57.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:57.596+0000 7f0e88623700 1 --2- 172.21.15.47:0/344941712 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0e800f8cb0 0x7f0e800f6c60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f0e7c0027f0 tx=0x7f0e7c002b00).stop 2024-02-23T00:42:57.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:57.596+0000 7f0e88623700 1 -- 172.21.15.47:0/344941712 shutdown_connections 2024-02-23T00:42:57.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:57.596+0000 7f0e88623700 1 --2- 172.21.15.47:0/344941712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e80102210 0x7f0e800f7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:57.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:57.596+0000 7f0e88623700 1 --2- 172.21.15.47:0/344941712 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0e6c062cc0 0x7f0e6c065180 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0x7f0e800f8400 tx=0x7f0e7000b040).stop 2024-02-23T00:42:57.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:57.596+0000 7f0e88623700 1 --2- 172.21.15.47:0/344941712 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0e800f8cb0 0x7f0e800f6c60 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0x7f0e7c0027f0 tx=0x7f0e7c002b00).stop 2024-02-23T00:42:57.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:57.596+0000 7f0e88623700 1 --2- 172.21.15.47:0/344941712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e800f9610 0x7f0e800f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:57.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:57.596+0000 7f0e88623700 1 -- 172.21.15.47:0/344941712 >> 172.21.15.47:0/344941712 conn(0x7f0e800f2820 msgr2=0x7f0e800f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:42:57.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:57.601+0000 7f0e88623700 1 -- 172.21.15.47:0/344941712 shutdown_connections 2024-02-23T00:42:57.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:57.602+0000 7f0e88623700 1 -- 172.21.15.47:0/344941712 wait complete. 2024-02-23T00:42:57.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:57.604+0000 7f0e88623700 1 librados: shutdown 2024-02-23T00:42:57.633 INFO:teuthology.orchestra.run.smithi047.stderr:set pool 6 pg_num to 22 2024-02-23T00:42:57.648 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:57.670 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:42:57.673 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:57.673 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:42:57.774 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:42:57.801 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:47.209177+0000 osd.4 (osd.4) 67 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:42:57.801 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:47.209238+0000 osd.4 (osd.4) 68 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:42:57.801 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:47.574956+0000 osd.1 (osd.1) 75 : cluster [DBG] 6.1d deep-scrub starts 2024-02-23T00:42:57.801 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:47.575801+0000 osd.1 (osd.1) 76 : cluster [DBG] 6.1d deep-scrub ok 2024-02-23T00:42:57.801 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:47.778785+0000 mon.b (mon.1) 37 : audit [INF] from='client.? 172.21.15.47:0/4172106799' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi047-82402-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.801 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:47.982098+0000 osd.7 (osd.7) 71 : cluster [DBG] 5.9 scrub starts 2024-02-23T00:42:57.801 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:47.985186+0000 osd.5 (osd.5) 71 : cluster [DBG] 4.4 scrub starts 2024-02-23T00:42:57.801 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:47.985568+0000 osd.7 (osd.7) 72 : cluster [DBG] 5.9 scrub ok 2024-02-23T00:42:57.802 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:47.988672+0000 osd.5 (osd.5) 72 : cluster [DBG] 4.4 scrub ok 2024-02-23T00:42:57.802 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:48.117744+0000 osd.4 (osd.4) 69 : cluster [DBG] 6.17 scrub starts 2024-02-23T00:42:57.802 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:48.118580+0000 osd.4 (osd.4) 70 : cluster [DBG] 6.17 scrub ok 2024-02-23T00:42:57.802 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:48.360325+0000 osd.0 (osd.0) 65 : cluster [DBG] 6.0 deep-scrub starts 2024-02-23T00:42:57.802 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:48.362731+0000 osd.0 (osd.0) 66 : cluster [DBG] 6.0 deep-scrub ok 2024-02-23T00:42:57.802 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:48.522903+0000 osd.6 (osd.6) 55 : cluster [DBG] 4.1f scrub starts 2024-02-23T00:42:57.802 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:48.524766+0000 osd.6 (osd.6) 56 : cluster [DBG] 4.1f scrub ok 2024-02-23T00:42:57.802 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:48.590031+0000 osd.1 (osd.1) 77 : cluster [DBG] 2.d scrub starts 2024-02-23T00:42:57.802 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:48.591478+0000 osd.1 (osd.1) 78 : cluster [DBG] 2.d scrub ok 2024-02-23T00:42:57.802 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:48.757887+0000 mon.c (mon.2) 27 : audit [INF] from='client.? 172.21.15.47:0/344941712' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "22"}]: dispatch 2024-02-23T00:42:57.802 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:49.171225+0000 mgr.y (mgr.14517) 141 : cluster [DBG] pgmap v131: 164 pgs: 32 unknown, 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 0 B/s, 1 objects/s recovering 2024-02-23T00:42:57.802 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:49.409001+0000 osd.0 (osd.0) 67 : cluster [DBG] 5.3 scrub starts 2024-02-23T00:42:57.802 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:49.410254+0000 osd.0 (osd.0) 68 : cluster [DBG] 5.3 scrub ok 2024-02-23T00:42:57.803 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:49.537547+0000 mon.b (mon.1) 38 : audit [INF] from='client.? 172.21.15.47:0/2670125835' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:42:57.803 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:49.966975+0000 osd.3 (osd.3) 81 : cluster [DBG] 4.0 scrub starts 2024-02-23T00:42:57.803 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:49.970748+0000 osd.3 (osd.3) 82 : cluster [DBG] 4.0 scrub ok 2024-02-23T00:42:57.803 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:50.424542+0000 osd.0 (osd.0) 69 : cluster [DBG] 6.10 scrub starts 2024-02-23T00:42:57.803 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:50.425629+0000 osd.0 (osd.0) 70 : cluster [DBG] 6.10 scrub ok 2024-02-23T00:42:57.803 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:51.171541+0000 mgr.y (mgr.14517) 142 : cluster [DBG] pgmap v132: 164 pgs: 32 unknown, 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 401 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-02-23T00:42:57.803 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:51.597128+0000 mon.c (mon.2) 28 : audit [DBG] from='client.? 172.21.15.47:0/2224696171' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:42:57.803 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:52.283442+0000 mgr.y (mgr.14517) 143 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:42:57.803 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:52.961502+0000 osd.3 (osd.3) 83 : cluster [DBG] 4.17 deep-scrub starts 2024-02-23T00:42:57.803 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:52.964506+0000 osd.3 (osd.3) 84 : cluster [DBG] 4.17 deep-scrub ok 2024-02-23T00:42:57.803 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:53.113051+0000 mon.a (mon.0) 802 : cluster [DBG] osdmap e107: 8 total, 8 up, 8 in 2024-02-23T00:42:57.803 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:53.124954+0000 mon.a (mon.0) 803 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi047-82402-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.804 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:53.127486+0000 mon.a (mon.0) 804 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "22"}]: dispatch 2024-02-23T00:42:57.804 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:53.130749+0000 mon.a (mon.0) 805 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:42:57.804 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:53.139846+0000 mon.b (mon.1) 39 : audit [INF] from='client.? 172.21.15.47:0/820656190' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi047-82428-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.804 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:47.209177+0000 osd.4 (osd.4) 67 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:42:57.804 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:47.209238+0000 osd.4 (osd.4) 68 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:42:57.804 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:47.574956+0000 osd.1 (osd.1) 75 : cluster [DBG] 6.1d deep-scrub starts 2024-02-23T00:42:57.804 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:47.575801+0000 osd.1 (osd.1) 76 : cluster [DBG] 6.1d deep-scrub ok 2024-02-23T00:42:57.804 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:47.778785+0000 mon.b (mon.1) 37 : audit [INF] from='client.? 172.21.15.47:0/4172106799' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi047-82402-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.805 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:47.982098+0000 osd.7 (osd.7) 71 : cluster [DBG] 5.9 scrub starts 2024-02-23T00:42:57.805 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:47.985186+0000 osd.5 (osd.5) 71 : cluster [DBG] 4.4 scrub starts 2024-02-23T00:42:57.805 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:47.985568+0000 osd.7 (osd.7) 72 : cluster [DBG] 5.9 scrub ok 2024-02-23T00:42:57.805 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:47.988672+0000 osd.5 (osd.5) 72 : cluster [DBG] 4.4 scrub ok 2024-02-23T00:42:57.805 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:48.117744+0000 osd.4 (osd.4) 69 : cluster [DBG] 6.17 scrub starts 2024-02-23T00:42:57.805 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:48.118580+0000 osd.4 (osd.4) 70 : cluster [DBG] 6.17 scrub ok 2024-02-23T00:42:57.805 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:48.360325+0000 osd.0 (osd.0) 65 : cluster [DBG] 6.0 deep-scrub starts 2024-02-23T00:42:57.805 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:48.362731+0000 osd.0 (osd.0) 66 : cluster [DBG] 6.0 deep-scrub ok 2024-02-23T00:42:57.805 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:48.522903+0000 osd.6 (osd.6) 55 : cluster [DBG] 4.1f scrub starts 2024-02-23T00:42:57.805 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:48.524766+0000 osd.6 (osd.6) 56 : cluster [DBG] 4.1f scrub ok 2024-02-23T00:42:57.805 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:48.590031+0000 osd.1 (osd.1) 77 : cluster [DBG] 2.d scrub starts 2024-02-23T00:42:57.805 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:48.591478+0000 osd.1 (osd.1) 78 : cluster [DBG] 2.d scrub ok 2024-02-23T00:42:57.805 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:48.757887+0000 mon.c (mon.2) 27 : audit [INF] from='client.? 172.21.15.47:0/344941712' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "22"}]: dispatch 2024-02-23T00:42:57.805 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:49.171225+0000 mgr.y (mgr.14517) 141 : cluster [DBG] pgmap v131: 164 pgs: 32 unknown, 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 0 B/s, 1 objects/s recovering 2024-02-23T00:42:57.805 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:49.409001+0000 osd.0 (osd.0) 67 : cluster [DBG] 5.3 scrub starts 2024-02-23T00:42:57.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:49.410254+0000 osd.0 (osd.0) 68 : cluster [DBG] 5.3 scrub ok 2024-02-23T00:42:57.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:53.140363+0000 mon.b (mon.1) 40 : audit [INF] from='client.? 172.21.15.47:0/1902177795' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi047-82386-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:53.140866+0000 mon.b (mon.1) 41 : audit [INF] from='client.? 172.21.15.47:0/2798582167' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi047-82512-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:53.152427+0000 mon.b (mon.1) 42 : audit [INF] from='client.? 172.21.15.47:0/2529329466' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi047-82504-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:53.152708+0000 mon.b (mon.1) 43 : audit [INF] from='client.? 172.21.15.47:0/4150674825' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi047-82465-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:53.159054+0000 mon.b (mon.1) 44 : audit [INF] from='client.? 172.21.15.47:0/3614542636' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi047-82438-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:53.159569+0000 mon.b (mon.1) 45 : audit [INF] from='client.? 172.21.15.47:0/750830915' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi047-82487-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:53.159746+0000 mon.b (mon.1) 46 : audit [INF] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi047-82419-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:53.160522+0000 mon.c (mon.2) 29 : audit [INF] from='client.? 172.21.15.47:0/1013021106' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:53.161104+0000 mon.b (mon.1) 47 : audit [INF] from='client.? 172.21.15.47:0/2668755850' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi047-82378-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:53.974453+0000 osd.3 (osd.3) 85 : cluster [DBG] 5.0 scrub starts 2024-02-23T00:42:57.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:53.977943+0000 osd.3 (osd.3) 86 : cluster [DBG] 5.0 scrub ok 2024-02-23T00:42:57.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:56.595370+0000 mon.c (mon.2) 30 : audit [INF] from='client.? 172.21.15.47:0/1523089110' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:42:57.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:57.118378+0000 mon.a (mon.0) 806 : cluster [DBG] osdmap e108: 8 total, 8 up, 8 in 2024-02-23T00:42:57.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:49.537547+0000 mon.b (mon.1) 38 : audit [INF] from='client.? 172.21.15.47:0/2670125835' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:42:57.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:49.966975+0000 osd.3 (osd.3) 81 : cluster [DBG] 4.0 scrub starts 2024-02-23T00:42:57.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:49.970748+0000 osd.3 (osd.3) 82 : cluster [DBG] 4.0 scrub ok 2024-02-23T00:42:57.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:50.424542+0000 osd.0 (osd.0) 69 : cluster [DBG] 6.10 scrub starts 2024-02-23T00:42:57.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:50.425629+0000 osd.0 (osd.0) 70 : cluster [DBG] 6.10 scrub ok 2024-02-23T00:42:57.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:51.171541+0000 mgr.y (mgr.14517) 142 : cluster [DBG] pgmap v132: 164 pgs: 32 unknown, 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 401 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-02-23T00:42:57.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:51.597128+0000 mon.c (mon.2) 28 : audit [DBG] from='client.? 172.21.15.47:0/2224696171' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:42:57.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:52.283442+0000 mgr.y (mgr.14517) 143 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:42:57.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:52.961502+0000 osd.3 (osd.3) 83 : cluster [DBG] 4.17 deep-scrub starts 2024-02-23T00:42:57.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:52.964506+0000 osd.3 (osd.3) 84 : cluster [DBG] 4.17 deep-scrub ok 2024-02-23T00:42:57.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:53.113051+0000 mon.a (mon.0) 802 : cluster [DBG] osdmap e107: 8 total, 8 up, 8 in 2024-02-23T00:42:57.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:53.124954+0000 mon.a (mon.0) 803 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi047-82402-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:53.127486+0000 mon.a (mon.0) 804 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "22"}]: dispatch 2024-02-23T00:42:57.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:53.130749+0000 mon.a (mon.0) 805 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:42:57.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:53.139846+0000 mon.b (mon.1) 39 : audit [INF] from='client.? 172.21.15.47:0/820656190' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi047-82428-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:53.140363+0000 mon.b (mon.1) 40 : audit [INF] from='client.? 172.21.15.47:0/1902177795' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi047-82386-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:53.140866+0000 mon.b (mon.1) 41 : audit [INF] from='client.? 172.21.15.47:0/2798582167' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi047-82512-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:53.152427+0000 mon.b (mon.1) 42 : audit [INF] from='client.? 172.21.15.47:0/2529329466' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi047-82504-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:53.152708+0000 mon.b (mon.1) 43 : audit [INF] from='client.? 172.21.15.47:0/4150674825' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi047-82465-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:53.159054+0000 mon.b (mon.1) 44 : audit [INF] from='client.? 172.21.15.47:0/3614542636' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi047-82438-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:53.159569+0000 mon.b (mon.1) 45 : audit [INF] from='client.? 172.21.15.47:0/750830915' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi047-82487-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:53.159746+0000 mon.b (mon.1) 46 : audit [INF] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi047-82419-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:53.160522+0000 mon.c (mon.2) 29 : audit [INF] from='client.? 172.21.15.47:0/1013021106' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:53.161104+0000 mon.b (mon.1) 47 : audit [INF] from='client.? 172.21.15.47:0/2668755850' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi047-82378-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:53.974453+0000 osd.3 (osd.3) 85 : cluster [DBG] 5.0 scrub starts 2024-02-23T00:42:57.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:53.977943+0000 osd.3 (osd.3) 86 : cluster [DBG] 5.0 scrub ok 2024-02-23T00:42:57.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:56.595370+0000 mon.c (mon.2) 30 : audit [INF] from='client.? 172.21.15.47:0/1523089110' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:42:57.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:57.118378+0000 mon.a (mon.0) 806 : cluster [DBG] osdmap e108: 8 total, 8 up, 8 in 2024-02-23T00:42:57.810 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:57.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:47.209177+0000 osd.4 (osd.4) 67 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:42:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:47.209238+0000 osd.4 (osd.4) 68 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:42:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:47.574956+0000 osd.1 (osd.1) 75 : cluster [DBG] 6.1d deep-scrub starts 2024-02-23T00:42:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:47.575801+0000 osd.1 (osd.1) 76 : cluster [DBG] 6.1d deep-scrub ok 2024-02-23T00:42:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:47.778785+0000 mon.b (mon.1) 37 : audit [INF] from='client.? 172.21.15.47:0/4172106799' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi047-82402-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:47.982098+0000 osd.7 (osd.7) 71 : cluster [DBG] 5.9 scrub starts 2024-02-23T00:42:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:47.985186+0000 osd.5 (osd.5) 71 : cluster [DBG] 4.4 scrub starts 2024-02-23T00:42:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:47.985568+0000 osd.7 (osd.7) 72 : cluster [DBG] 5.9 scrub ok 2024-02-23T00:42:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:47.988672+0000 osd.5 (osd.5) 72 : cluster [DBG] 4.4 scrub ok 2024-02-23T00:42:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:48.117744+0000 osd.4 (osd.4) 69 : cluster [DBG] 6.17 scrub starts 2024-02-23T00:42:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:48.118580+0000 osd.4 (osd.4) 70 : cluster [DBG] 6.17 scrub ok 2024-02-23T00:42:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:48.360325+0000 osd.0 (osd.0) 65 : cluster [DBG] 6.0 deep-scrub starts 2024-02-23T00:42:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:48.362731+0000 osd.0 (osd.0) 66 : cluster [DBG] 6.0 deep-scrub ok 2024-02-23T00:42:57.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:48.522903+0000 osd.6 (osd.6) 55 : cluster [DBG] 4.1f scrub starts 2024-02-23T00:42:57.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:48.524766+0000 osd.6 (osd.6) 56 : cluster [DBG] 4.1f scrub ok 2024-02-23T00:42:57.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:48.590031+0000 osd.1 (osd.1) 77 : cluster [DBG] 2.d scrub starts 2024-02-23T00:42:57.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:48.591478+0000 osd.1 (osd.1) 78 : cluster [DBG] 2.d scrub ok 2024-02-23T00:42:57.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:48.757887+0000 mon.c (mon.2) 27 : audit [INF] from='client.? 172.21.15.47:0/344941712' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "22"}]: dispatch 2024-02-23T00:42:57.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:49.171225+0000 mgr.y (mgr.14517) 141 : cluster [DBG] pgmap v131: 164 pgs: 32 unknown, 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 0 B/s, 1 objects/s recovering 2024-02-23T00:42:57.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:49.409001+0000 osd.0 (osd.0) 67 : cluster [DBG] 5.3 scrub starts 2024-02-23T00:42:57.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:49.410254+0000 osd.0 (osd.0) 68 : cluster [DBG] 5.3 scrub ok 2024-02-23T00:42:57.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:49.537547+0000 mon.b (mon.1) 38 : audit [INF] from='client.? 172.21.15.47:0/2670125835' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:42:57.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:49.966975+0000 osd.3 (osd.3) 81 : cluster [DBG] 4.0 scrub starts 2024-02-23T00:42:57.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:49.970748+0000 osd.3 (osd.3) 82 : cluster [DBG] 4.0 scrub ok 2024-02-23T00:42:57.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:50.424542+0000 osd.0 (osd.0) 69 : cluster [DBG] 6.10 scrub starts 2024-02-23T00:42:57.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:50.425629+0000 osd.0 (osd.0) 70 : cluster [DBG] 6.10 scrub ok 2024-02-23T00:42:57.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:51.171541+0000 mgr.y (mgr.14517) 142 : cluster [DBG] pgmap v132: 164 pgs: 32 unknown, 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 401 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-02-23T00:42:57.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:51.597128+0000 mon.c (mon.2) 28 : audit [DBG] from='client.? 172.21.15.47:0/2224696171' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:42:57.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:52.283442+0000 mgr.y (mgr.14517) 143 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:42:57.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:52.961502+0000 osd.3 (osd.3) 83 : cluster [DBG] 4.17 deep-scrub starts 2024-02-23T00:42:57.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:52.964506+0000 osd.3 (osd.3) 84 : cluster [DBG] 4.17 deep-scrub ok 2024-02-23T00:42:57.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:53.113051+0000 mon.a (mon.0) 802 : cluster [DBG] osdmap e107: 8 total, 8 up, 8 in 2024-02-23T00:42:57.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:53.124954+0000 mon.a (mon.0) 803 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi047-82402-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:53.127486+0000 mon.a (mon.0) 804 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "22"}]: dispatch 2024-02-23T00:42:57.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:53.130749+0000 mon.a (mon.0) 805 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:42:57.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:53.139846+0000 mon.b (mon.1) 39 : audit [INF] from='client.? 172.21.15.47:0/820656190' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi047-82428-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:53.140363+0000 mon.b (mon.1) 40 : audit [INF] from='client.? 172.21.15.47:0/1902177795' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi047-82386-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:53.140866+0000 mon.b (mon.1) 41 : audit [INF] from='client.? 172.21.15.47:0/2798582167' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi047-82512-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:53.152427+0000 mon.b (mon.1) 42 : audit [INF] from='client.? 172.21.15.47:0/2529329466' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi047-82504-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:53.152708+0000 mon.b (mon.1) 43 : audit [INF] from='client.? 172.21.15.47:0/4150674825' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi047-82465-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.947 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:53.159054+0000 mon.b (mon.1) 44 : audit [INF] from='client.? 172.21.15.47:0/3614542636' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi047-82438-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.947 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:53.159569+0000 mon.b (mon.1) 45 : audit [INF] from='client.? 172.21.15.47:0/750830915' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi047-82487-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.947 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:53.159746+0000 mon.b (mon.1) 46 : audit [INF] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi047-82419-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.947 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:53.160522+0000 mon.c (mon.2) 29 : audit [INF] from='client.? 172.21.15.47:0/1013021106' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.947 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:53.161104+0000 mon.b (mon.1) 47 : audit [INF] from='client.? 172.21.15.47:0/2668755850' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi047-82378-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:57.947 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:53.974453+0000 osd.3 (osd.3) 85 : cluster [DBG] 5.0 scrub starts 2024-02-23T00:42:57.947 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:53.977943+0000 osd.3 (osd.3) 86 : cluster [DBG] 5.0 scrub ok 2024-02-23T00:42:57.947 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:56.595370+0000 mon.c (mon.2) 30 : audit [INF] from='client.? 172.21.15.47:0/1523089110' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:42:57.947 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:57.118378+0000 mon.a (mon.0) 806 : cluster [DBG] osdmap e108: 8 total, 8 up, 8 in 2024-02-23T00:42:58.003 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:42:58.006 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:58.007 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:42:58.107 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:42:58.137 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:58.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:58.598+0000 7f843a7fc700 1 -- 172.21.15.47:0/1523089110 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v110) v1 ==== 87+0+0 (secure 0 0 0) 0x7f843404d5c0 con 0x7f84440f7400 2024-02-23T00:42:58.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:58.601+0000 7f844b8c5700 10 librados: watch_flush enter 2024-02-23T00:42:58.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:58.601+0000 7f844b8c5700 10 librados: watch_flush exit 2024-02-23T00:42:58.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:58.601+0000 7f844b8c5700 20 client.24914.objecter shutdown clearing up homeless session... 2024-02-23T00:42:58.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:58.601+0000 7f844b8c5700 10 client.24914.objecter successfully canceled tick 2024-02-23T00:42:58.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:58.601+0000 7f844b8c5700 1 -- 172.21.15.47:0/1523089110 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8430062cc0 msgr2=0x7f8430065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:58.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:58.601+0000 7f844b8c5700 1 --2- 172.21.15.47:0/1523089110 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8430062cc0 0x7f8430065180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f84441077f0 tx=0x7f844000b040).stop 2024-02-23T00:42:58.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:58.602+0000 7f844b8c5700 1 -- 172.21.15.47:0/1523089110 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f84440f7400 msgr2=0x7f8444106050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:58.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:58.602+0000 7f844b8c5700 1 --2- 172.21.15.47:0/1523089110 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f84440f7400 0x7f8444106050 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f8434013680 tx=0x7f8434009cb0).stop 2024-02-23T00:42:58.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:58.606+0000 7f844b8c5700 1 -- 172.21.15.47:0/1523089110 shutdown_connections 2024-02-23T00:42:58.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:58.606+0000 7f844b8c5700 1 --2- 172.21.15.47:0/1523089110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84440fd510 0x7f8444106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:42:58.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:58.606+0000 7f844b8c5700 1 --2- 172.21.15.47:0/1523089110 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8430062cc0 0x7f8430065180 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:58.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:58.606+0000 7f844b8c5700 1 --2- 172.21.15.47:0/1523089110 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f84440f7400 0x7f8444106050 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:58.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:58.606+0000 7f844b8c5700 1 --2- 172.21.15.47:0/1523089110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84440fcbb0 0x7f8444106540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:58.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:58.606+0000 7f844b8c5700 1 -- 172.21.15.47:0/1523089110 >> 172.21.15.47:0/1523089110 conn(0x7f84440f2820 msgr2=0x7f84440fb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:42:58.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:58.613+0000 7f844b8c5700 1 -- 172.21.15.47:0/1523089110 shutdown_connections 2024-02-23T00:42:58.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:58.614+0000 7f844b8c5700 1 -- 172.21.15.47:0/1523089110 wait complete. 2024-02-23T00:42:58.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:58.616+0000 7f844b8c5700 1 librados: shutdown 2024-02-23T00:42:58.636 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is set 2024-02-23T00:42:58.857 INFO:tasks.workunit.client.0.smithi047.stdout: api_asio: [==========] Running 12 tests from 1 test suite. 2024-02-23T00:42:58.857 INFO:tasks.workunit.client.0.smithi047.stdout: api_asio: [----------] Global test environment set-up. 2024-02-23T00:42:58.857 INFO:tasks.workunit.client.0.smithi047.stdout: api_asio: [----------] 12 tests from AsioRados 2024-02-23T00:42:58.857 INFO:tasks.workunit.client.0.smithi047.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2024-02-23T00:42:58.857 INFO:tasks.workunit.client.0.smithi047.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (16 ms) 2024-02-23T00:42:58.857 INFO:tasks.workunit.client.0.smithi047.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2024-02-23T00:42:58.857 INFO:tasks.workunit.client.0.smithi047.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (6 ms) 2024-02-23T00:42:58.857 INFO:tasks.workunit.client.0.smithi047.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2024-02-23T00:42:58.857 INFO:tasks.workunit.client.0.smithi047.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (1 ms) 2024-02-23T00:42:58.857 INFO:tasks.workunit.client.0.smithi047.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2024-02-23T00:42:58.858 INFO:tasks.workunit.client.0.smithi047.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (4 ms) 2024-02-23T00:42:58.858 INFO:tasks.workunit.client.0.smithi047.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2024-02-23T00:42:58.858 INFO:tasks.workunit.client.0.smithi047.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (2 ms) 2024-02-23T00:42:58.858 INFO:tasks.workunit.client.0.smithi047.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2024-02-23T00:42:58.858 INFO:tasks.workunit.client.0.smithi047.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (1 ms) 2024-02-23T00:42:58.858 INFO:tasks.workunit.client.0.smithi047.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2024-02-23T00:42:58.858 INFO:tasks.workunit.client.0.smithi047.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (1 ms) 2024-02-23T00:42:58.858 INFO:tasks.workunit.client.0.smithi047.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2024-02-23T00:42:58.858 INFO:tasks.workunit.client.0.smithi047.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (0 ms) 2024-02-23T00:42:58.858 INFO:tasks.workunit.client.0.smithi047.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2024-02-23T00:42:58.858 INFO:tasks.workunit.client.0.smithi047.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (1 ms) 2024-02-23T00:42:58.858 INFO:tasks.workunit.client.0.smithi047.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2024-02-23T00:42:58.859 INFO:tasks.workunit.client.0.smithi047.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (1 ms) 2024-02-23T00:42:58.859 INFO:tasks.workunit.client.0.smithi047.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2024-02-23T00:42:58.859 INFO:tasks.workunit.client.0.smithi047.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (4 ms) 2024-02-23T00:42:58.859 INFO:tasks.workunit.client.0.smithi047.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2024-02-23T00:42:58.859 INFO:tasks.workunit.client.0.smithi047.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (1 ms) 2024-02-23T00:42:58.859 INFO:tasks.workunit.client.0.smithi047.stdout: api_asio: [----------] 12 tests from AsioRados (38 ms total) 2024-02-23T00:42:58.859 INFO:tasks.workunit.client.0.smithi047.stdout: api_asio: 2024-02-23T00:42:58.859 INFO:tasks.workunit.client.0.smithi047.stdout: api_asio: [----------] Global test environment tear-down 2024-02-23T00:42:58.859 INFO:tasks.workunit.client.0.smithi047.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (11543 ms total) 2024-02-23T00:42:58.859 INFO:tasks.workunit.client.0.smithi047.stdout: api_asio: [ PASSED ] 12 tests. 2024-02-23T00:42:58.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:49.923564+0000 osd.5 (osd.5) 73 : cluster [DBG] 4.8 scrub starts 2024-02-23T00:42:58.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:49.925668+0000 osd.5 (osd.5) 74 : cluster [DBG] 4.8 scrub ok 2024-02-23T00:42:58.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:49.974291+0000 osd.7 (osd.7) 73 : cluster [DBG] 1.0 deep-scrub starts 2024-02-23T00:42:58.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:49.977238+0000 osd.7 (osd.7) 74 : cluster [DBG] 1.0 deep-scrub ok 2024-02-23T00:42:58.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:50.076491+0000 osd.4 (osd.4) 71 : cluster [DBG] 4.c scrub starts 2024-02-23T00:42:58.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:50.078033+0000 osd.4 (osd.4) 72 : cluster [DBG] 4.c scrub ok 2024-02-23T00:42:58.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:50.944456+0000 osd.7 (osd.7) 75 : cluster [DBG] 2.0 scrub starts 2024-02-23T00:42:58.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:50.947346+0000 osd.7 (osd.7) 76 : cluster [DBG] 2.0 scrub ok 2024-02-23T00:42:58.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:51.966732+0000 osd.5 (osd.5) 75 : cluster [DBG] 5.1b scrub starts 2024-02-23T00:42:58.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:51.967521+0000 osd.5 (osd.5) 76 : cluster [DBG] 5.1b scrub ok 2024-02-23T00:42:58.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:52.316553+0000 osd.2 (osd.2) 65 : cluster [DBG] 2.10 scrub starts 2024-02-23T00:42:58.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:52.317398+0000 osd.2 (osd.2) 66 : cluster [DBG] 2.10 scrub ok 2024-02-23T00:42:58.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:53.172956+0000 mgr.y (mgr.14517) 144 : cluster [DBG] pgmap v134: 548 pgs: 27 creating+peering, 132 active+clean, 389 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-02-23T00:42:58.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:53.442933+0000 osd.0 (osd.0) 71 : cluster [DBG] 4.1e scrub starts 2024-02-23T00:42:58.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:53.444506+0000 osd.0 (osd.0) 72 : cluster [DBG] 4.1e scrub ok 2024-02-23T00:42:58.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:53.962033+0000 osd.5 (osd.5) 77 : cluster [DBG] 5.f scrub starts 2024-02-23T00:42:58.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:53.964930+0000 osd.5 (osd.5) 78 : cluster [DBG] 5.f scrub ok 2024-02-23T00:42:58.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:55.173492+0000 mgr.y (mgr.14517) 145 : cluster [DBG] pgmap v135: 548 pgs: 27 creating+peering, 132 active+clean, 389 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:42:58.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:56.455580+0000 osd.0 (osd.0) 73 : cluster [DBG] 6.16 deep-scrub starts 2024-02-23T00:42:58.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:56.456806+0000 osd.0 (osd.0) 74 : cluster [DBG] 6.16 deep-scrub ok 2024-02-23T00:42:58.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:57.174889+0000 mgr.y (mgr.14517) 146 : cluster [DBG] pgmap v136: 548 pgs: 416 creating+peering, 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-23T00:42:58.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:57.573211+0000 osd.6 (osd.6) 57 : cluster [DBG] 5.11 scrub starts 2024-02-23T00:42:58.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:57.574363+0000 osd.6 (osd.6) 58 : cluster [DBG] 5.11 scrub ok 2024-02-23T00:42:58.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:57.577778+0000 mon.a (mon.0) 807 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:58.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:57.578758+0000 mon.a (mon.0) 808 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi047-82428-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:58.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:57.578891+0000 mon.a (mon.0) 809 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:42:58.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:57.579125+0000 mon.a (mon.0) 810 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi047-82386-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:58.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:57.579310+0000 mon.a (mon.0) 811 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi047-82512-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:58.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:57.579608+0000 mon.a (mon.0) 812 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi047-82504-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:58.947 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:57.579853+0000 mon.a (mon.0) 813 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi047-82465-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:58.947 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:57.580085+0000 mon.a (mon.0) 814 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi047-82438-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:58.947 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:57.590812+0000 mon.c (mon.2) 31 : audit [DBG] from='client.? 172.21.15.47:0/2550794552' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-23T00:42:58.947 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:57.592458+0000 mon.a (mon.0) 815 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi047-82402-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:58.947 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:57.592634+0000 mon.a (mon.0) 816 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "22"}]': finished 2024-02-23T00:42:58.947 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:57.592776+0000 mon.a (mon.0) 817 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:42:58.947 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:57.593648+0000 mon.a (mon.0) 818 : cluster [DBG] osdmap e109: 8 total, 8 up, 8 in 2024-02-23T00:42:58.947 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:57.594708+0000 mon.b (mon.1) 48 : audit [INF] from='client.? 172.21.15.47:0/2893985577' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi047-82517-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:58.947 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:57.601628+0000 mon.b (mon.1) 49 : audit [INF] from='client.? 172.21.15.47:0/2670125835' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi047-82528-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:42:58.947 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:57.605263+0000 mon.b (mon.1) 50 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi047-82457-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:58.948 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:57.606592+0000 mon.a (mon.0) 819 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi047-82487-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:58.948 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:57.606907+0000 mon.a (mon.0) 820 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi047-82419-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:58.948 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:57.607125+0000 mon.a (mon.0) 821 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi047-82378-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:58.948 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:57.607834+0000 mon.a (mon.0) 822 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi047-82517-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:58.948 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:57.608263+0000 mon.a (mon.0) 823 : audit [INF] from='client.? 172.21.15.47:0/2649136881' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82544-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:58.948 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:57.608482+0000 mon.a (mon.0) 824 : audit [INF] from='client.? 172.21.15.47:0/1494681915' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi047-82496-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:58.948 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:57.612382+0000 mon.a (mon.0) 825 : audit [INF] from='client.? 172.21.15.47:0/2014338467' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi047-82576-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:58.948 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:57.613475+0000 mon.a (mon.0) 826 : audit [INF] from='client.? 172.21.15.47:0/446573282' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi047-82394-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:58.948 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:57.614939+0000 mon.a (mon.0) 827 : audit [DBG] from='client.? 172.21.15.47:0/2068339748' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-23T00:42:58.948 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:57.620120+0000 mon.a (mon.0) 828 : audit [INF] from='client.? 172.21.15.47:0/1911970904' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:58.949 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:57.628032+0000 mon.a (mon.0) 829 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi047-82528-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:42:58.949 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:57.629863+0000 mon.a (mon.0) 830 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi047-82457-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:58.949 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:57.636651+0000 mon.a (mon.0) 831 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:42:58.949 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:57.700473+0000 osd.1 (osd.1) 79 : cluster [DBG] 4.12 scrub starts 2024-02-23T00:42:58.949 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:57.705253+0000 osd.1 (osd.1) 80 : cluster [DBG] 4.12 scrub ok 2024-02-23T00:42:59.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:49.923564+0000 osd.5 (osd.5) 73 : cluster [DBG] 4.8 scrub starts 2024-02-23T00:42:59.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:49.925668+0000 osd.5 (osd.5) 74 : cluster [DBG] 4.8 scrub ok 2024-02-23T00:42:59.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:49.974291+0000 osd.7 (osd.7) 73 : cluster [DBG] 1.0 deep-scrub starts 2024-02-23T00:42:59.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:49.977238+0000 osd.7 (osd.7) 74 : cluster [DBG] 1.0 deep-scrub ok 2024-02-23T00:42:59.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:50.076491+0000 osd.4 (osd.4) 71 : cluster [DBG] 4.c scrub starts 2024-02-23T00:42:59.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:50.078033+0000 osd.4 (osd.4) 72 : cluster [DBG] 4.c scrub ok 2024-02-23T00:42:59.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:50.944456+0000 osd.7 (osd.7) 75 : cluster [DBG] 2.0 scrub starts 2024-02-23T00:42:59.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:50.947346+0000 osd.7 (osd.7) 76 : cluster [DBG] 2.0 scrub ok 2024-02-23T00:42:59.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:51.966732+0000 osd.5 (osd.5) 75 : cluster [DBG] 5.1b scrub starts 2024-02-23T00:42:59.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:51.967521+0000 osd.5 (osd.5) 76 : cluster [DBG] 5.1b scrub ok 2024-02-23T00:42:59.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:52.316553+0000 osd.2 (osd.2) 65 : cluster [DBG] 2.10 scrub starts 2024-02-23T00:42:59.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:52.317398+0000 osd.2 (osd.2) 66 : cluster [DBG] 2.10 scrub ok 2024-02-23T00:42:59.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:53.172956+0000 mgr.y (mgr.14517) 144 : cluster [DBG] pgmap v134: 548 pgs: 27 creating+peering, 132 active+clean, 389 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-02-23T00:42:59.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:53.442933+0000 osd.0 (osd.0) 71 : cluster [DBG] 4.1e scrub starts 2024-02-23T00:42:59.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:53.444506+0000 osd.0 (osd.0) 72 : cluster [DBG] 4.1e scrub ok 2024-02-23T00:42:59.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:53.962033+0000 osd.5 (osd.5) 77 : cluster [DBG] 5.f scrub starts 2024-02-23T00:42:59.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:53.964930+0000 osd.5 (osd.5) 78 : cluster [DBG] 5.f scrub ok 2024-02-23T00:42:59.061 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:55.173492+0000 mgr.y (mgr.14517) 145 : cluster [DBG] pgmap v135: 548 pgs: 27 creating+peering, 132 active+clean, 389 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:42:59.061 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:56.455580+0000 osd.0 (osd.0) 73 : cluster [DBG] 6.16 deep-scrub starts 2024-02-23T00:42:59.061 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:56.456806+0000 osd.0 (osd.0) 74 : cluster [DBG] 6.16 deep-scrub ok 2024-02-23T00:42:59.061 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:57.174889+0000 mgr.y (mgr.14517) 146 : cluster [DBG] pgmap v136: 548 pgs: 416 creating+peering, 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-23T00:42:59.061 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:57.573211+0000 osd.6 (osd.6) 57 : cluster [DBG] 5.11 scrub starts 2024-02-23T00:42:59.061 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:57.574363+0000 osd.6 (osd.6) 58 : cluster [DBG] 5.11 scrub ok 2024-02-23T00:42:59.062 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:57.577778+0000 mon.a (mon.0) 807 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.062 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:57.578758+0000 mon.a (mon.0) 808 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi047-82428-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.062 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:57.578891+0000 mon.a (mon.0) 809 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:42:59.062 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:57.579125+0000 mon.a (mon.0) 810 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi047-82386-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.062 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:57.579310+0000 mon.a (mon.0) 811 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi047-82512-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.062 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:57.579608+0000 mon.a (mon.0) 812 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi047-82504-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.062 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:57.579853+0000 mon.a (mon.0) 813 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi047-82465-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:57.580085+0000 mon.a (mon.0) 814 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi047-82438-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:57.590812+0000 mon.c (mon.2) 31 : audit [DBG] from='client.? 172.21.15.47:0/2550794552' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-23T00:42:59.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:57.592458+0000 mon.a (mon.0) 815 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi047-82402-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:57.592634+0000 mon.a (mon.0) 816 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "22"}]': finished 2024-02-23T00:42:59.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:57.592776+0000 mon.a (mon.0) 817 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:42:59.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:57.593648+0000 mon.a (mon.0) 818 : cluster [DBG] osdmap e109: 8 total, 8 up, 8 in 2024-02-23T00:42:59.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:57.594708+0000 mon.b (mon.1) 48 : audit [INF] from='client.? 172.21.15.47:0/2893985577' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi047-82517-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:57.601628+0000 mon.b (mon.1) 49 : audit [INF] from='client.? 172.21.15.47:0/2670125835' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi047-82528-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:42:59.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:57.605263+0000 mon.b (mon.1) 50 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi047-82457-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:57.606592+0000 mon.a (mon.0) 819 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi047-82487-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:57.606907+0000 mon.a (mon.0) 820 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi047-82419-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:57.607125+0000 mon.a (mon.0) 821 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi047-82378-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:57.607834+0000 mon.a (mon.0) 822 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi047-82517-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:57.608263+0000 mon.a (mon.0) 823 : audit [INF] from='client.? 172.21.15.47:0/2649136881' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82544-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:57.608482+0000 mon.a (mon.0) 824 : audit [INF] from='client.? 172.21.15.47:0/1494681915' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi047-82496-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:57.612382+0000 mon.a (mon.0) 825 : audit [INF] from='client.? 172.21.15.47:0/2014338467' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi047-82576-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.066 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:49.923564+0000 osd.5 (osd.5) 73 : cluster [DBG] 4.8 scrub starts 2024-02-23T00:42:59.066 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:49.925668+0000 osd.5 (osd.5) 74 : cluster [DBG] 4.8 scrub ok 2024-02-23T00:42:59.066 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:49.974291+0000 osd.7 (osd.7) 73 : cluster [DBG] 1.0 deep-scrub starts 2024-02-23T00:42:59.066 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:49.977238+0000 osd.7 (osd.7) 74 : cluster [DBG] 1.0 deep-scrub ok 2024-02-23T00:42:59.066 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:50.076491+0000 osd.4 (osd.4) 71 : cluster [DBG] 4.c scrub starts 2024-02-23T00:42:59.067 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:50.078033+0000 osd.4 (osd.4) 72 : cluster [DBG] 4.c scrub ok 2024-02-23T00:42:59.067 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:50.944456+0000 osd.7 (osd.7) 75 : cluster [DBG] 2.0 scrub starts 2024-02-23T00:42:59.067 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:50.947346+0000 osd.7 (osd.7) 76 : cluster [DBG] 2.0 scrub ok 2024-02-23T00:42:59.067 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:51.966732+0000 osd.5 (osd.5) 75 : cluster [DBG] 5.1b scrub starts 2024-02-23T00:42:59.067 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:51.967521+0000 osd.5 (osd.5) 76 : cluster [DBG] 5.1b scrub ok 2024-02-23T00:42:59.067 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:52.316553+0000 osd.2 (osd.2) 65 : cluster [DBG] 2.10 scrub starts 2024-02-23T00:42:59.068 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:52.317398+0000 osd.2 (osd.2) 66 : cluster [DBG] 2.10 scrub ok 2024-02-23T00:42:59.068 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:53.172956+0000 mgr.y (mgr.14517) 144 : cluster [DBG] pgmap v134: 548 pgs: 27 creating+peering, 132 active+clean, 389 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-02-23T00:42:59.068 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:53.442933+0000 osd.0 (osd.0) 71 : cluster [DBG] 4.1e scrub starts 2024-02-23T00:42:59.069 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:53.444506+0000 osd.0 (osd.0) 72 : cluster [DBG] 4.1e scrub ok 2024-02-23T00:42:59.069 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:53.962033+0000 osd.5 (osd.5) 77 : cluster [DBG] 5.f scrub starts 2024-02-23T00:42:59.069 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:53.964930+0000 osd.5 (osd.5) 78 : cluster [DBG] 5.f scrub ok 2024-02-23T00:42:59.069 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:55.173492+0000 mgr.y (mgr.14517) 145 : cluster [DBG] pgmap v135: 548 pgs: 27 creating+peering, 132 active+clean, 389 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:42:59.069 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:56.455580+0000 osd.0 (osd.0) 73 : cluster [DBG] 6.16 deep-scrub starts 2024-02-23T00:42:59.069 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:56.456806+0000 osd.0 (osd.0) 74 : cluster [DBG] 6.16 deep-scrub ok 2024-02-23T00:42:59.069 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:57.174889+0000 mgr.y (mgr.14517) 146 : cluster [DBG] pgmap v136: 548 pgs: 416 creating+peering, 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-23T00:42:59.070 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:57.573211+0000 osd.6 (osd.6) 57 : cluster [DBG] 5.11 scrub starts 2024-02-23T00:42:59.070 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:57.574363+0000 osd.6 (osd.6) 58 : cluster [DBG] 5.11 scrub ok 2024-02-23T00:42:59.070 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:57.577778+0000 mon.a (mon.0) 807 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.070 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:57.578758+0000 mon.a (mon.0) 808 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi047-82428-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.070 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:57.578891+0000 mon.a (mon.0) 809 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:42:59.070 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:57.579125+0000 mon.a (mon.0) 810 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi047-82386-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.071 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:57.579310+0000 mon.a (mon.0) 811 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi047-82512-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.071 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:57.579608+0000 mon.a (mon.0) 812 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi047-82504-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.071 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:57.579853+0000 mon.a (mon.0) 813 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi047-82465-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.071 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:57.580085+0000 mon.a (mon.0) 814 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi047-82438-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.071 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:57.590812+0000 mon.c (mon.2) 31 : audit [DBG] from='client.? 172.21.15.47:0/2550794552' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-23T00:42:59.071 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:57.592458+0000 mon.a (mon.0) 815 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi047-82402-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.072 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:57.592634+0000 mon.a (mon.0) 816 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "22"}]': finished 2024-02-23T00:42:59.072 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:57.592776+0000 mon.a (mon.0) 817 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:42:59.072 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:57.593648+0000 mon.a (mon.0) 818 : cluster [DBG] osdmap e109: 8 total, 8 up, 8 in 2024-02-23T00:42:59.072 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:57.594708+0000 mon.b (mon.1) 48 : audit [INF] from='client.? 172.21.15.47:0/2893985577' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi047-82517-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.072 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:57.601628+0000 mon.b (mon.1) 49 : audit [INF] from='client.? 172.21.15.47:0/2670125835' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi047-82528-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:42:59.073 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:57.605263+0000 mon.b (mon.1) 50 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi047-82457-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.073 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:57.606592+0000 mon.a (mon.0) 819 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi047-82487-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.073 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:57.606907+0000 mon.a (mon.0) 820 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi047-82419-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.073 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:57.607125+0000 mon.a (mon.0) 821 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi047-82378-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.073 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:57.607834+0000 mon.a (mon.0) 822 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi047-82517-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.073 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:57.608263+0000 mon.a (mon.0) 823 : audit [INF] from='client.? 172.21.15.47:0/2649136881' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82544-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.074 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:57.608482+0000 mon.a (mon.0) 824 : audit [INF] from='client.? 172.21.15.47:0/1494681915' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi047-82496-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.074 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:57.612382+0000 mon.a (mon.0) 825 : audit [INF] from='client.? 172.21.15.47:0/2014338467' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi047-82576-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.074 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:57.613475+0000 mon.a (mon.0) 826 : audit [INF] from='client.? 172.21.15.47:0/446573282' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi047-82394-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.074 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:57.614939+0000 mon.a (mon.0) 827 : audit [DBG] from='client.? 172.21.15.47:0/2068339748' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-23T00:42:59.074 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:57.620120+0000 mon.a (mon.0) 828 : audit [INF] from='client.? 172.21.15.47:0/1911970904' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.074 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:57.628032+0000 mon.a (mon.0) 829 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi047-82528-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:42:59.075 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:57.629863+0000 mon.a (mon.0) 830 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi047-82457-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.075 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:57.636651+0000 mon.a (mon.0) 831 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:42:59.075 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:57.700473+0000 osd.1 (osd.1) 79 : cluster [DBG] 4.12 scrub starts 2024-02-23T00:42:59.075 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:57.705253+0000 osd.1 (osd.1) 80 : cluster [DBG] 4.12 scrub ok 2024-02-23T00:42:59.076 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:57.613475+0000 mon.a (mon.0) 826 : audit [INF] from='client.? 172.21.15.47:0/446573282' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi047-82394-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.076 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:57.614939+0000 mon.a (mon.0) 827 : audit [DBG] from='client.? 172.21.15.47:0/2068339748' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-23T00:42:59.076 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:57.620120+0000 mon.a (mon.0) 828 : audit [INF] from='client.? 172.21.15.47:0/1911970904' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.076 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:57.628032+0000 mon.a (mon.0) 829 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi047-82528-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:42:59.076 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:57.629863+0000 mon.a (mon.0) 830 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi047-82457-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.076 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:57.636651+0000 mon.a (mon.0) 831 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:42:59.077 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:57.700473+0000 osd.1 (osd.1) 79 : cluster [DBG] 4.12 scrub starts 2024-02-23T00:42:59.077 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:57.705253+0000 osd.1 (osd.1) 80 : cluster [DBG] 4.12 scrub ok 2024-02-23T00:42:59.286 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [==========] Running 11 tests from 3 test suites. 2024-02-23T00:42:59.286 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [----------] Global test environment set-up. 2024-02-23T00:42:59.286 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [----------] 7 tests from LibRadosList 2024-02-23T00:42:59.287 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2024-02-23T00:42:59.287 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [ OK ] LibRadosList.ListObjects (334 ms) 2024-02-23T00:42:59.287 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2024-02-23T00:42:59.287 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (30 ms) 2024-02-23T00:42:59.287 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2024-02-23T00:42:59.287 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: myset foo1,foo2,foo3 2024-02-23T00:42:59.287 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: foo1 2024-02-23T00:42:59.287 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: foo2 2024-02-23T00:42:59.287 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: foo3 2024-02-23T00:42:59.287 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: myset foo1,foo4,foo5 2024-02-23T00:42:59.287 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: foo4 2024-02-23T00:42:59.287 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: foo5 2024-02-23T00:42:59.287 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: foo1 2024-02-23T00:42:59.287 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: myset foo6,foo7 2024-02-23T00:42:59.287 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: foo7 2024-02-23T00:42:59.288 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: foo6 2024-02-23T00:42:59.288 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-02-23T00:42:59.288 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: ns1:foo4 2024-02-23T00:42:59.288 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: ns1:foo5 2024-02-23T00:42:59.288 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: ns2:foo7 2024-02-23T00:42:59.288 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: ns2:foo6 2024-02-23T00:42:59.288 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: ns1:foo1 2024-02-23T00:42:59.288 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: :foo1 2024-02-23T00:42:59.288 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: :foo2 2024-02-23T00:42:59.288 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: :foo3 2024-02-23T00:42:59.288 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (81 ms) 2024-02-23T00:42:59.288 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2024-02-23T00:42:59.288 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 1 0 2024-02-23T00:42:59.289 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 10 0 2024-02-23T00:42:59.289 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 13 0 2024-02-23T00:42:59.289 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 7 0 2024-02-23T00:42:59.289 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 14 0 2024-02-23T00:42:59.291 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 0 0 2024-02-23T00:42:59.291 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 15 0 2024-02-23T00:42:59.291 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 11 0 2024-02-23T00:42:59.291 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 5 0 2024-02-23T00:42:59.291 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 8 0 2024-02-23T00:42:59.291 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 6 0 2024-02-23T00:42:59.291 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 3 0 2024-02-23T00:42:59.291 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 4 0 2024-02-23T00:42:59.291 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 12 0 2024-02-23T00:42:59.291 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 9 0 2024-02-23T00:42:59.291 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 2 0 2024-02-23T00:42:59.292 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-02-23T00:42:59.292 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (64 ms) 2024-02-23T00:42:59.292 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2024-02-23T00:42:59.292 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: x cursor=MIN 2024-02-23T00:42:59.292 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > oid=1 cursor=15:02547ec2:::1:head 2024-02-23T00:42:59.292 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > oid=10 cursor=15:52ea6a34:::10:head 2024-02-23T00:42:59.293 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > oid=13 cursor=15:566253c9:::13:head 2024-02-23T00:42:59.293 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > oid=7 cursor=15:5c6b0b28:::7:head 2024-02-23T00:42:59.293 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > oid=14 cursor=15:62a1935d:::14:head 2024-02-23T00:42:59.293 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > oid=0 cursor=15:6cac518f:::0:head 2024-02-23T00:42:59.293 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > oid=15 cursor=15:863748b0:::15:head 2024-02-23T00:42:59.293 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > oid=11 cursor=15:89d3ae78:::11:head 2024-02-23T00:42:59.293 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > oid=5 cursor=15:b29083e3:::5:head 2024-02-23T00:42:59.293 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > oid=8 cursor=15:bd63b0f1:::8:head 2024-02-23T00:42:59.293 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > oid=6 cursor=15:c4fdafeb:::6:head 2024-02-23T00:42:59.293 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > oid=3 cursor=15:cfc208b3:::3:head 2024-02-23T00:42:59.293 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > oid=4 cursor=15:d83876eb:::4:head 2024-02-23T00:42:59.293 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > oid=12 cursor=15:de5d7c5f:::12:head 2024-02-23T00:42:59.293 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > oid=9 cursor=15:e960b815:::9:head 2024-02-23T00:42:59.293 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > oid=2 cursor=15:f905c69b:::2:head 2024-02-23T00:42:59.294 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: FIRST> seek to MIN oid=1 2024-02-23T00:42:59.294 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : oid=1 cursor=15:02547ec2:::1:head 2024-02-23T00:42:59.294 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : seek to 15:02547ec2:::1:head 2024-02-23T00:42:59.294 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > 15:02547ec2:::1:head -> 1 2024-02-23T00:42:59.294 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : oid=10 cursor=15:52ea6a34:::10:head 2024-02-23T00:42:59.294 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : seek to 15:52ea6a34:::10:head 2024-02-23T00:42:59.294 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > 15:52ea6a34:::10:head -> 10 2024-02-23T00:42:59.294 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : oid=13 cursor=15:566253c9:::13:head 2024-02-23T00:42:59.294 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : seek to 15:566253c9:::13:head 2024-02-23T00:42:59.294 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > 15:566253c9:::13:head -> 13 2024-02-23T00:42:59.294 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : oid=7 cursor=15:5c6b0b28:::7:head 2024-02-23T00:42:59.294 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : seek to 15:5c6b0b28:::7:head 2024-02-23T00:42:59.294 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > 15:5c6b0b28:::7:head -> 7 2024-02-23T00:42:59.296 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:42:59.317 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:59.318 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:42:59.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : oid=14 cursor=15:62a1935d:::14:head 2024-02-23T00:42:59.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : seek to 15:62a1935d:::14:head 2024-02-23T00:42:59.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > 15:62a1935d:::14:head -> 14 2024-02-23T00:42:59.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : oid=0 cursor=15:6cac518f:::0:head 2024-02-23T00:42:59.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : seek to 15:6cac518f:::0:head 2024-02-23T00:42:59.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > 15:6cac518f:::0:head -> 0 2024-02-23T00:42:59.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : oid=15 cursor=15:863748b0:::15:head 2024-02-23T00:42:59.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : seek to 15:863748b0:::15:head 2024-02-23T00:42:59.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > 15:863748b0:::15:head -> 15 2024-02-23T00:42:59.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : oid=11 cursor=15:89d3ae78:::11:head 2024-02-23T00:42:59.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : seek to 15:89d3ae78:::11:head 2024-02-23T00:42:59.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > 15:89d3ae78:::11:head -> 11 2024-02-23T00:42:59.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : oid=5 cursor=15:b29083e3:::5:head 2024-02-23T00:42:59.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : seek to 15:b29083e3:::5:head 2024-02-23T00:42:59.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > 15:b29083e3:::5:head -> 5 2024-02-23T00:42:59.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : oid=8 cursor=15:bd63b0f1:::8:head 2024-02-23T00:42:59.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : seek to 15:bd63b0f1:::8:head 2024-02-23T00:42:59.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > 15:bd63b0f1:::8:head -> 8 2024-02-23T00:42:59.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : oid=6 cursor=15:c4fdafeb:::6:head 2024-02-23T00:42:59.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : seek to 15:c4fdafeb:::6:head 2024-02-23T00:42:59.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > 15:c4fdafeb:::6:head -> 6 2024-02-23T00:42:59.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : oid=3 cursor=15:cfc208b3:::3:head 2024-02-23T00:42:59.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : seek to 15:cfc208b3:::3:head 2024-02-23T00:42:59.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > 15:cfc208b3:::3:head -> 3 2024-02-23T00:42:59.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : oid=4 cursor=15:d83876eb:::4:head 2024-02-23T00:42:59.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : seek to 15:d83876eb:::4:head 2024-02-23T00:42:59.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > 15:d83876eb:::4:head -> 4 2024-02-23T00:42:59.357 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : oid=12 cursor=15:de5d7c5f:::12:head 2024-02-23T00:42:59.357 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : seek to 15:de5d7c5f:::12:head 2024-02-23T00:42:59.357 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > 15:de5d7c5f:::12:head -> 12 2024-02-23T00:42:59.357 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : oid=9 cursor=15:e960b815:::9:head 2024-02-23T00:42:59.357 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : seek to 15:e960b815:::9:head 2024-02-23T00:42:59.357 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > 15:e960b815:::9:head -> 9 2024-02-23T00:42:59.357 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : oid=2 cursor=15:f905c69b:::2:head 2024-02-23T00:42:59.357 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : seek to 15:f905c69b:::2:head 2024-02-23T00:42:59.357 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > 15:f905c69b:::2:head -> 2 2024-02-23T00:42:59.357 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : seek to 15:62a1935d:::14:head 2024-02-23T00:42:59.357 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : cursor()=15:62a1935d:::14:head expected=15:62a1935d:::14:head 2024-02-23T00:42:59.357 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > 15:62a1935d:::14:head -> 14 2024-02-23T00:42:59.357 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : entry=14 expected=14 2024-02-23T00:42:59.357 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : seek to 15:566253c9:::13:head 2024-02-23T00:42:59.357 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : cursor()=15:566253c9:::13:head expected=15:566253c9:::13:head 2024-02-23T00:42:59.358 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > 15:566253c9:::13:head -> 13 2024-02-23T00:42:59.358 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : entry=13 expected=13 2024-02-23T00:42:59.358 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : seek to 15:f905c69b:::2:head 2024-02-23T00:42:59.358 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : cursor()=15:f905c69b:::2:head expected=15:f905c69b:::2:head 2024-02-23T00:42:59.358 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > 15:f905c69b:::2:head -> 2 2024-02-23T00:42:59.358 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : entry=2 expected=2 2024-02-23T00:42:59.358 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : seek to 15:e960b815:::9:head 2024-02-23T00:42:59.358 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : cursor()=15:e960b815:::9:head expected=15:e960b815:::9:head 2024-02-23T00:42:59.358 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > 15:e960b815:::9:head -> 9 2024-02-23T00:42:59.358 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : entry=9 expected=9 2024-02-23T00:42:59.358 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : seek to 15:de5d7c5f:::12:head 2024-02-23T00:42:59.358 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : cursor()=15:de5d7c5f:::12:head expected=15:de5d7c5f:::12:head 2024-02-23T00:42:59.358 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > 15:de5d7c5f:::12:head -> 12 2024-02-23T00:42:59.358 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : entry=12 expected=12 2024-02-23T00:42:59.358 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : seek to 15:d83876eb:::4:head 2024-02-23T00:42:59.359 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : cursor()=15:d83876eb:::4:head expected=15:d83876eb:::4:head 2024-02-23T00:42:59.359 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > 15:d83876eb:::4:head -> 4 2024-02-23T00:42:59.359 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : entry=4 expected=4 2024-02-23T00:42:59.359 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : seek to 15:cfc208b3:::3:head 2024-02-23T00:42:59.359 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : cursor()=15:cfc208b3:::3:head expected=15:cfc208b3:::3:head 2024-02-23T00:42:59.359 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > 15:cfc208b3:::3:head -> 3 2024-02-23T00:42:59.359 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : entry=3 expected=3 2024-02-23T00:42:59.359 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : seek to 15:c4fdafeb:::6:head 2024-02-23T00:42:59.359 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : cursor()=15:c4fdafeb:::6:head expected=15:c4fdafeb:::6:head 2024-02-23T00:42:59.359 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > 15:c4fdafeb:::6:head -> 6 2024-02-23T00:42:59.420 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:42:59.463 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:42:59.768 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:58.583149+0000 mon.a (mon.0) 832 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:42:59.769 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:58.589953+0000 mon.a (mon.0) 833 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.769 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:58.590034+0000 mon.a (mon.0) 834 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi047-82428-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.769 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:58.590098+0000 mon.a (mon.0) 835 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:42:59.769 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:58.590156+0000 mon.a (mon.0) 836 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi047-82386-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.769 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:58.590233+0000 mon.a (mon.0) 837 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi047-82512-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.769 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:58.590300+0000 mon.a (mon.0) 838 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi047-82504-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.769 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:58.590363+0000 mon.a (mon.0) 839 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi047-82465-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.769 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:58.590427+0000 mon.a (mon.0) 840 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi047-82438-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.769 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:58.590509+0000 mon.a (mon.0) 841 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi047-82487-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.769 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:58.590577+0000 mon.a (mon.0) 842 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi047-82419-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.769 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:58.590640+0000 mon.a (mon.0) 843 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi047-82378-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.769 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:58.590742+0000 mon.a (mon.0) 844 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi047-82517-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.770 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:58.590823+0000 mon.a (mon.0) 845 : audit [INF] from='client.? 172.21.15.47:0/2649136881' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82544-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.770 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:58.590883+0000 mon.a (mon.0) 846 : audit [INF] from='client.? 172.21.15.47:0/1494681915' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi047-82496-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.770 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:58.590938+0000 mon.a (mon.0) 847 : audit [INF] from='client.? 172.21.15.47:0/2014338467' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi047-82576-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.770 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:58.590988+0000 mon.a (mon.0) 848 : audit [INF] from='client.? 172.21.15.47:0/446573282' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi047-82394-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.770 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:58.591045+0000 mon.a (mon.0) 849 : audit [INF] from='client.? 172.21.15.47:0/1911970904' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.770 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:59.131532+0000 mon.a (mon.0) 850 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi047-82457-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.770 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:59.131775+0000 mon.a (mon.0) 851 : cluster [DBG] osdmap e110: 8 total, 8 up, 8 in 2024-02-23T00:42:59.770 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:59.153675+0000 mon.a (mon.0) 852 : audit [INF] from='client.? 172.21.15.47:0/2832234163' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82536-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.770 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:59.162238+0000 mon.b (mon.1) 51 : audit [INF] from='client.? 172.21.15.47:0/1505701467' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:42:59.770 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:59.179138+0000 mon.a (mon.0) 853 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num_actual", "val": "22"}]: dispatch 2024-02-23T00:42:59.770 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:58.583149+0000 mon.a (mon.0) 832 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:42:59.770 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:58.589953+0000 mon.a (mon.0) 833 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.771 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:58.590034+0000 mon.a (mon.0) 834 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi047-82428-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.771 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:58.590098+0000 mon.a (mon.0) 835 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:42:59.771 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:58.590156+0000 mon.a (mon.0) 836 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi047-82386-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.771 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:58.590233+0000 mon.a (mon.0) 837 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi047-82512-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.771 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:58.590300+0000 mon.a (mon.0) 838 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi047-82504-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.771 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:58.590363+0000 mon.a (mon.0) 839 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi047-82465-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.771 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:58.590427+0000 mon.a (mon.0) 840 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi047-82438-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.771 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:58.590509+0000 mon.a (mon.0) 841 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi047-82487-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.771 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:58.590577+0000 mon.a (mon.0) 842 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi047-82419-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.771 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:58.590640+0000 mon.a (mon.0) 843 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi047-82378-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.771 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:58.590742+0000 mon.a (mon.0) 844 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi047-82517-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.771 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:58.590823+0000 mon.a (mon.0) 845 : audit [INF] from='client.? 172.21.15.47:0/2649136881' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82544-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.771 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:58.590883+0000 mon.a (mon.0) 846 : audit [INF] from='client.? 172.21.15.47:0/1494681915' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi047-82496-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.771 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:58.590938+0000 mon.a (mon.0) 847 : audit [INF] from='client.? 172.21.15.47:0/2014338467' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi047-82576-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.772 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:58.590988+0000 mon.a (mon.0) 848 : audit [INF] from='client.? 172.21.15.47:0/446573282' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi047-82394-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.772 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:58.591045+0000 mon.a (mon.0) 849 : audit [INF] from='client.? 172.21.15.47:0/1911970904' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.772 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:59.131532+0000 mon.a (mon.0) 850 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi047-82457-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:42:59.772 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:59.131775+0000 mon.a (mon.0) 851 : cluster [DBG] osdmap e110: 8 total, 8 up, 8 in 2024-02-23T00:42:59.772 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:59.153675+0000 mon.a (mon.0) 852 : audit [INF] from='client.? 172.21.15.47:0/2832234163' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82536-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:42:59.772 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:59.162238+0000 mon.b (mon.1) 51 : audit [INF] from='client.? 172.21.15.47:0/1505701467' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:42:59.772 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:42:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:59.179138+0000 mon.a (mon.0) 853 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num_actual", "val": "22"}]: dispatch 2024-02-23T00:42:59.869 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:42:59.869 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:42:59.869 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:42:59.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.546+0000 7f03acd6e700 1 -- 172.21.15.47:0/492160729 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f03a80fdbb0 msgr2=0x7f03a80fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:59.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.546+0000 7f03acd6e700 1 --2- 172.21.15.47:0/492160729 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f03a80fdbb0 0x7f03a80fdf90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f0390004870 tx=0x7f0390009cb0).stop 2024-02-23T00:42:59.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.547+0000 7f03acd6e700 1 -- 172.21.15.47:0/492160729 shutdown_connections 2024-02-23T00:42:59.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.547+0000 7f03acd6e700 1 --2- 172.21.15.47:0/492160729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03a80f8420 0x7f03a8104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:59.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.547+0000 7f03acd6e700 1 --2- 172.21.15.47:0/492160729 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f03a80fdbb0 0x7f03a80fdf90 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:59.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.547+0000 7f03acd6e700 1 --2- 172.21.15.47:0/492160729 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a80fe510 0x7f03a80f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:59.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.547+0000 7f03acd6e700 1 -- 172.21.15.47:0/492160729 >> 172.21.15.47:0/492160729 conn(0x7f03a80f3800 msgr2=0x7f03a80f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:42:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.547+0000 7f03acd6e700 1 -- 172.21.15.47:0/492160729 shutdown_connections 2024-02-23T00:42:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.547+0000 7f03acd6e700 1 -- 172.21.15.47:0/492160729 wait complete. 2024-02-23T00:42:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.547+0000 7f03acd6e700 1 librados: starting msgr at 2024-02-23T00:42:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.547+0000 7f03acd6e700 1 librados: starting objecter 2024-02-23T00:42:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.548+0000 7f03acd6e700 1 Processor -- start 2024-02-23T00:42:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.548+0000 7f03acd6e700 1 -- start start 2024-02-23T00:42:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.548+0000 7f03acd6e700 1 librados: setting wanted keys 2024-02-23T00:42:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.548+0000 7f03acd6e700 1 librados: calling monclient init 2024-02-23T00:42:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.548+0000 7f03acd6e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f03a80f8420 0x7f03a810ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.548+0000 7f03acd6e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03a80fdbb0 0x7f03a810b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.548+0000 7f03acd6e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a80fe510 0x7f03a8103630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.548+0000 7f03acd6e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a8103bb0 con 0x7f03a80fe510 2024-02-23T00:42:59.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.548+0000 7f03acd6e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a8103d20 con 0x7f03a80fdbb0 2024-02-23T00:42:59.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.548+0000 7f03acd6e700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a8103ee0 con 0x7f03a80f8420 2024-02-23T00:42:59.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.549+0000 7f03a659c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f03a80f8420 0x7f03a810ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:59.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.549+0000 7f03a659c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f03a80f8420 0x7f03a810ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:47072/0 (socket says 172.21.15.47:47072) 2024-02-23T00:42:59.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.549+0000 7f03a659c700 1 -- 172.21.15.47:0/826605645 learned_addr learned my addr 172.21.15.47:0/826605645 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:42:59.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.549+0000 7f03a6d9d700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a80fe510 0x7f03a8103630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:59.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.549+0000 7f03a5d9b700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03a80fdbb0 0x7f03a810b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:59.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.549+0000 7f03a659c700 1 -- 172.21.15.47:0/826605645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03a80fdbb0 msgr2=0x7f03a810b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:59.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.549+0000 7f03a659c700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03a80fdbb0 0x7f03a810b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:59.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.549+0000 7f03a659c700 1 -- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a80fe510 msgr2=0x7f03a8103630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:59.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.549+0000 7f03a659c700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a80fe510 0x7f03a8103630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:59.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.549+0000 7f03a659c700 1 -- 172.21.15.47:0/826605645 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0390005040 con 0x7f03a80f8420 2024-02-23T00:42:59.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.549+0000 7f03a659c700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f03a80f8420 0x7f03a810ab70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f0390009da0 tx=0x7f0390009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:59.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.549+0000 7f039f7fe700 10 client.?.objecter ms_handle_connect 0x7f03a80f8420 2024-02-23T00:42:59.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.549+0000 7f039f7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:42:59.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.549+0000 7f039f7fe700 1 -- 172.21.15.47:0/826605645 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039000e030 con 0x7f03a80f8420 2024-02-23T00:42:59.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.549+0000 7f039f7fe700 1 -- 172.21.15.47:0/826605645 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f0390013d70 con 0x7f03a80f8420 2024-02-23T00:42:59.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.549+0000 7f039f7fe700 1 -- 172.21.15.47:0/826605645 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0390018070 con 0x7f03a80f8420 2024-02-23T00:42:59.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.549+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a8104160 con 0x7f03a80f8420 2024-02-23T00:42:59.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.550+0000 7f03acd6e700 10 client.24977.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:42:59.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.550+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a810e990 con 0x7f03a80f8420 2024-02-23T00:42:59.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.550+0000 7f03acd6e700 1 librados: init done 2024-02-23T00:42:59.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.551+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f03a805ee80 con 0x7f03a80f8420 2024-02-23T00:42:59.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.551+0000 7f039f7fe700 1 -- 172.21.15.47:0/826605645 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f0390009240 con 0x7f03a80f8420 2024-02-23T00:42:59.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.552+0000 7f039f7fe700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0394062cc0 0x7f0394065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:59.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.552+0000 7f03a5d9b700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0394062cc0 0x7f0394065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:59.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.552+0000 7f039f7fe700 1 -- 172.21.15.47:0/826605645 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(110..110 src has 48..110) v4 ==== 17032+0+0 (secure 0 0 0) 0x7f0390080680 con 0x7f03a80f8420 2024-02-23T00:42:59.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.552+0000 7f039f7fe700 10 client.24977.objecter ms_dispatch 0x7f03a8001550 osd_map(110..110 src has 48..110) v4 2024-02-23T00:42:59.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.552+0000 7f039f7fe700 3 client.24977.objecter handle_osd_map got epochs [110,110] > 0 2024-02-23T00:42:59.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.552+0000 7f039f7fe700 3 client.24977.objecter handle_osd_map decoding full epoch 110 2024-02-23T00:42:59.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.553+0000 7f039f7fe700 20 client.24977.objecter dump_active .. 0 homeless 2024-02-23T00:42:59.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.553+0000 7f03a5d9b700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0394062cc0 0x7f0394065180 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f03a81075b0 tx=0x7f039800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:59.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.553+0000 7f039f7fe700 10 client.24977.objecter ms_handle_connect 0x7f0394062cc0 2024-02-23T00:42:59.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.553+0000 7f039f7fe700 1 -- 172.21.15.47:0/826605645 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v110) v1 ==== 54+0+16 (secure 0 0 0) 0x7f0390087070 con 0x7f03a80f8420 2024-02-23T00:42:59.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.553+0000 7f03acd6e700 10 client.24977.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:59.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.553+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_assign -1 1 2024-02-23T00:42:59.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.553+0000 7f03acd6e700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f03a810f890 0x7f03a80273c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:59.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.553+0000 7f03acd6e700 20 client.24977.objecter _get_session s=0x7f03a8104b10 osd=0 3 2024-02-23T00:42:59.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.553+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a8104b10 osd=0 3 2024-02-23T00:42:59.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.553+0000 7f03acd6e700 20 client.24977.objecter _get_session s=0x7f03a8104b10 osd=0 3 2024-02-23T00:42:59.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.553+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_remove -1 1 2024-02-23T00:42:59.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.553+0000 7f03acd6e700 20 client.24977.objecter get_session s=0x7f03a8104b10 osd=0 3 2024-02-23T00:42:59.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.553+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_assign 0 1 2024-02-23T00:42:59.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.553+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a8104b10 osd=0 4 2024-02-23T00:42:59.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.553+0000 7f03acd6e700 10 client.24977.objecter _send_command 1 2024-02-23T00:42:59.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.553+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f03a8106e20 con 0x7f03a810f890 2024-02-23T00:42:59.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.553+0000 7f03a6d9d700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f03a810f890 0x7f03a80273c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:59.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.553+0000 7f03a6d9d700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f03a810f890 0x7f03a80273c0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:59.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.553+0000 7f039f7fe700 10 client.24977.objecter ms_handle_connect 0x7f03a810f890 2024-02-23T00:42:59.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.554+0000 7f039f7fe700 1 -- 172.21.15.47:0/826605645 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f03a8106e20 con 0x7f03a810f890 2024-02-23T00:42:59.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.554+0000 7f039f7fe700 10 client.24977.objecter ms_dispatch 0x7f03a8001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:42:59.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.554+0000 7f039f7fe700 10 client.24977.objecter _finish_command 1 = system:0 2024-02-23T00:42:59.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.554+0000 7f039f7fe700 20 client.24977.objecter put_session s=0x7f03a8104b10 osd=0 4 2024-02-23T00:42:59.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.554+0000 7f039f7fe700 15 client.24977.objecter _session_command_op_remove 0 1 2024-02-23T00:42:59.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.594+0000 7f03acd6e700 10 client.24977.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:42:59.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.594+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_assign -1 2 2024-02-23T00:42:59.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.594+0000 7f03acd6e700 20 client.24977.objecter _get_session s=0x7f03a8104b10 osd=0 3 2024-02-23T00:42:59.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.594+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a8104b10 osd=0 3 2024-02-23T00:42:59.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.594+0000 7f03acd6e700 20 client.24977.objecter _get_session s=0x7f03a8104b10 osd=0 3 2024-02-23T00:42:59.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.594+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_remove -1 2 2024-02-23T00:42:59.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.594+0000 7f03acd6e700 20 client.24977.objecter get_session s=0x7f03a8104b10 osd=0 3 2024-02-23T00:42:59.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.594+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_assign 0 2 2024-02-23T00:42:59.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.594+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a8104b10 osd=0 4 2024-02-23T00:42:59.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.594+0000 7f03acd6e700 10 client.24977.objecter _send_command 2 2024-02-23T00:42:59.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.594+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f03a810edf0 con 0x7f03a810f890 2024-02-23T00:42:59.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.596+0000 7f039f7fe700 1 -- 172.21.15.47:0/826605645 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f03a810edf0 con 0x7f03a810f890 2024-02-23T00:42:59.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.596+0000 7f039f7fe700 10 client.24977.objecter ms_dispatch 0x7f03a8001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:42:59.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.596+0000 7f039f7fe700 10 client.24977.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:42:59.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.596+0000 7f039f7fe700 20 client.24977.objecter put_session s=0x7f03a8104b10 osd=0 4 2024-02-23T00:42:59.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.596+0000 7f039f7fe700 15 client.24977.objecter _session_command_op_remove 0 2 2024-02-23T00:42:59.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.596+0000 7f03acd6e700 10 client.24977.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:59.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.596+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_assign -1 3 2024-02-23T00:42:59.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.596+0000 7f03acd6e700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f03a8028590 0x7f03a80faf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:59.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.596+0000 7f03acd6e700 20 client.24977.objecter _get_session s=0x7f03a810f200 osd=1 3 2024-02-23T00:42:59.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.596+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a810f200 osd=1 3 2024-02-23T00:42:59.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.596+0000 7f03acd6e700 20 client.24977.objecter _get_session s=0x7f03a810f200 osd=1 3 2024-02-23T00:42:59.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.596+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_remove -1 3 2024-02-23T00:42:59.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.596+0000 7f03acd6e700 20 client.24977.objecter get_session s=0x7f03a810f200 osd=1 3 2024-02-23T00:42:59.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.596+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_assign 1 3 2024-02-23T00:42:59.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.596+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a810f200 osd=1 4 2024-02-23T00:42:59.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.596+0000 7f03acd6e700 10 client.24977.objecter _send_command 3 2024-02-23T00:42:59.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.596+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f03a80fc580 con 0x7f03a8028590 2024-02-23T00:42:59.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.596+0000 7f03a659c700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f03a8028590 0x7f03a80faf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:59.878 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:42:59.878 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:42:59.878 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:42:59.878 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:42:59.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.597+0000 7f03a659c700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f03a8028590 0x7f03a80faf10 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:59.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.597+0000 7f039f7fe700 10 client.24977.objecter ms_handle_connect 0x7f03a8028590 2024-02-23T00:42:59.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.597+0000 7f039f7fe700 1 -- 172.21.15.47:0/826605645 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f03a80fc580 con 0x7f03a8028590 2024-02-23T00:42:59.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.597+0000 7f039f7fe700 10 client.24977.objecter ms_dispatch 0x7f03a8001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:42:59.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.597+0000 7f039f7fe700 10 client.24977.objecter _finish_command 3 = system:0 2024-02-23T00:42:59.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.597+0000 7f039f7fe700 20 client.24977.objecter put_session s=0x7f03a810f200 osd=1 4 2024-02-23T00:42:59.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.597+0000 7f039f7fe700 15 client.24977.objecter _session_command_op_remove 1 3 2024-02-23T00:42:59.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.636+0000 7f03acd6e700 10 client.24977.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:42:59.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.636+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_assign -1 4 2024-02-23T00:42:59.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.636+0000 7f03acd6e700 20 client.24977.objecter _get_session s=0x7f03a810f200 osd=1 3 2024-02-23T00:42:59.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.636+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a810f200 osd=1 3 2024-02-23T00:42:59.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.636+0000 7f03acd6e700 20 client.24977.objecter _get_session s=0x7f03a810f200 osd=1 3 2024-02-23T00:42:59.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.636+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_remove -1 4 2024-02-23T00:42:59.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.636+0000 7f03acd6e700 20 client.24977.objecter get_session s=0x7f03a810f200 osd=1 3 2024-02-23T00:42:59.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.636+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_assign 1 4 2024-02-23T00:42:59.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.636+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a810f200 osd=1 4 2024-02-23T00:42:59.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.636+0000 7f03acd6e700 10 client.24977.objecter _send_command 4 2024-02-23T00:42:59.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.636+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f03a8027af0 con 0x7f03a8028590 2024-02-23T00:42:59.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.640+0000 7f039f7fe700 1 -- 172.21.15.47:0/826605645 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f03a8027af0 con 0x7f03a8028590 2024-02-23T00:42:59.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.640+0000 7f039f7fe700 10 client.24977.objecter ms_dispatch 0x7f03a8001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:42:59.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.640+0000 7f039f7fe700 10 client.24977.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:42:59.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.640+0000 7f039f7fe700 20 client.24977.objecter put_session s=0x7f03a810f200 osd=1 4 2024-02-23T00:42:59.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.640+0000 7f039f7fe700 15 client.24977.objecter _session_command_op_remove 1 4 2024-02-23T00:42:59.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.640+0000 7f03acd6e700 10 client.24977.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:59.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.640+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_assign -1 5 2024-02-23T00:42:59.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.640+0000 7f03acd6e700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f03a8059e50 0x7f03a812ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:59.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.640+0000 7f03acd6e700 20 client.24977.objecter _get_session s=0x7f03a8027f00 osd=2 3 2024-02-23T00:42:59.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.640+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a8027f00 osd=2 3 2024-02-23T00:42:59.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.640+0000 7f03acd6e700 20 client.24977.objecter _get_session s=0x7f03a8027f00 osd=2 3 2024-02-23T00:42:59.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.640+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_remove -1 5 2024-02-23T00:42:59.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.640+0000 7f03acd6e700 20 client.24977.objecter get_session s=0x7f03a8027f00 osd=2 3 2024-02-23T00:42:59.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.640+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_assign 2 5 2024-02-23T00:42:59.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.640+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a8027f00 osd=2 4 2024-02-23T00:42:59.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.640+0000 7f03acd6e700 10 client.24977.objecter _send_command 5 2024-02-23T00:42:59.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.640+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f03a812dc00 con 0x7f03a8059e50 2024-02-23T00:42:59.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.641+0000 7f03a5d9b700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f03a8059e50 0x7f03a812ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:59.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.641+0000 7f039f7fe700 10 client.24977.objecter ms_handle_connect 0x7f03a8059e50 2024-02-23T00:42:59.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.641+0000 7f03a5d9b700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f03a8059e50 0x7f03a812ad70 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:59.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.642+0000 7f039f7fe700 1 -- 172.21.15.47:0/826605645 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f03a812dc00 con 0x7f03a8059e50 2024-02-23T00:42:59.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.642+0000 7f039f7fe700 10 client.24977.objecter ms_dispatch 0x7f03a8001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:42:59.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.642+0000 7f039f7fe700 10 client.24977.objecter _finish_command 5 = system:0 2024-02-23T00:42:59.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.642+0000 7f039f7fe700 20 client.24977.objecter put_session s=0x7f03a8027f00 osd=2 4 2024-02-23T00:42:59.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.642+0000 7f039f7fe700 15 client.24977.objecter _session_command_op_remove 2 5 2024-02-23T00:42:59.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.661+0000 7f03acd6e700 10 client.24977.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:42:59.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.661+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_assign -1 6 2024-02-23T00:42:59.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.661+0000 7f03acd6e700 20 client.24977.objecter _get_session s=0x7f03a8027f00 osd=2 3 2024-02-23T00:42:59.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.661+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a8027f00 osd=2 3 2024-02-23T00:42:59.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.661+0000 7f03acd6e700 20 client.24977.objecter _get_session s=0x7f03a8027f00 osd=2 3 2024-02-23T00:42:59.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.661+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_remove -1 6 2024-02-23T00:42:59.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.661+0000 7f03acd6e700 20 client.24977.objecter get_session s=0x7f03a8027f00 osd=2 3 2024-02-23T00:42:59.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.661+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_assign 2 6 2024-02-23T00:42:59.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.661+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a8027f00 osd=2 4 2024-02-23T00:42:59.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.661+0000 7f03acd6e700 10 client.24977.objecter _send_command 6 2024-02-23T00:42:59.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.661+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f03a8059360 con 0x7f03a8059e50 2024-02-23T00:42:59.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.662+0000 7f039f7fe700 1 -- 172.21.15.47:0/826605645 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f03a8059360 con 0x7f03a8059e50 2024-02-23T00:42:59.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.662+0000 7f039f7fe700 10 client.24977.objecter ms_dispatch 0x7f03a8001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:42:59.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.662+0000 7f039f7fe700 10 client.24977.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:42:59.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.662+0000 7f039f7fe700 20 client.24977.objecter put_session s=0x7f03a8027f00 osd=2 4 2024-02-23T00:42:59.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.662+0000 7f039f7fe700 15 client.24977.objecter _session_command_op_remove 2 6 2024-02-23T00:42:59.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.663+0000 7f03acd6e700 10 client.24977.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:59.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.663+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_assign -1 7 2024-02-23T00:42:59.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.663+0000 7f03acd6e700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f03a80fc130 0x7f03a812b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:59.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.663+0000 7f03acd6e700 20 client.24977.objecter _get_session s=0x7f03a8059760 osd=3 3 2024-02-23T00:42:59.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.663+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a8059760 osd=3 3 2024-02-23T00:42:59.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.663+0000 7f03acd6e700 20 client.24977.objecter _get_session s=0x7f03a8059760 osd=3 3 2024-02-23T00:42:59.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.663+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_remove -1 7 2024-02-23T00:42:59.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.663+0000 7f03acd6e700 20 client.24977.objecter get_session s=0x7f03a8059760 osd=3 3 2024-02-23T00:42:59.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.663+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_assign 3 7 2024-02-23T00:42:59.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.663+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a8059760 osd=3 4 2024-02-23T00:42:59.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.663+0000 7f03acd6e700 10 client.24977.objecter _send_command 7 2024-02-23T00:42:59.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.663+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f03a812ba20 con 0x7f03a80fc130 2024-02-23T00:42:59.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.663+0000 7f03a6d9d700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f03a80fc130 0x7f03a812b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:59.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.663+0000 7f039f7fe700 10 client.24977.objecter ms_handle_connect 0x7f03a80fc130 2024-02-23T00:42:59.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.663+0000 7f03a6d9d700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f03a80fc130 0x7f03a812b370 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:59.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.664+0000 7f039f7fe700 1 -- 172.21.15.47:0/826605645 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f03a812ba20 con 0x7f03a80fc130 2024-02-23T00:42:59.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.664+0000 7f039f7fe700 10 client.24977.objecter ms_dispatch 0x7f03a8001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:42:59.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.664+0000 7f039f7fe700 10 client.24977.objecter _finish_command 7 = system:0 2024-02-23T00:42:59.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.664+0000 7f039f7fe700 20 client.24977.objecter put_session s=0x7f03a8059760 osd=3 4 2024-02-23T00:42:59.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.664+0000 7f039f7fe700 15 client.24977.objecter _session_command_op_remove 3 7 2024-02-23T00:42:59.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.684+0000 7f03acd6e700 10 client.24977.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:42:59.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.684+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_assign -1 8 2024-02-23T00:42:59.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.684+0000 7f03acd6e700 20 client.24977.objecter _get_session s=0x7f03a8059760 osd=3 3 2024-02-23T00:42:59.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.684+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a8059760 osd=3 3 2024-02-23T00:42:59.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.684+0000 7f03acd6e700 20 client.24977.objecter _get_session s=0x7f03a8059760 osd=3 3 2024-02-23T00:42:59.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.684+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_remove -1 8 2024-02-23T00:42:59.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.684+0000 7f03acd6e700 20 client.24977.objecter get_session s=0x7f03a8059760 osd=3 3 2024-02-23T00:42:59.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.684+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_assign 3 8 2024-02-23T00:42:59.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.684+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a8059760 osd=3 4 2024-02-23T00:42:59.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.684+0000 7f03acd6e700 10 client.24977.objecter _send_command 8 2024-02-23T00:42:59.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.684+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f03a812c3f0 con 0x7f03a80fc130 2024-02-23T00:42:59.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.686+0000 7f039f7fe700 1 -- 172.21.15.47:0/826605645 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f03a812c3f0 con 0x7f03a80fc130 2024-02-23T00:42:59.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.686+0000 7f039f7fe700 10 client.24977.objecter ms_dispatch 0x7f03a8001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:42:59.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.686+0000 7f039f7fe700 10 client.24977.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:42:59.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.686+0000 7f039f7fe700 20 client.24977.objecter put_session s=0x7f03a8059760 osd=3 4 2024-02-23T00:42:59.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.686+0000 7f039f7fe700 15 client.24977.objecter _session_command_op_remove 3 8 2024-02-23T00:42:59.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.686+0000 7f03acd6e700 10 client.24977.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:59.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.686+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_assign -1 9 2024-02-23T00:42:59.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.686+0000 7f03acd6e700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f03a812fd80 0x7f03a8132170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:59.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.686+0000 7f03acd6e700 20 client.24977.objecter _get_session s=0x7f03a812bc20 osd=4 3 2024-02-23T00:42:59.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.686+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a812bc20 osd=4 3 2024-02-23T00:42:59.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.686+0000 7f03acd6e700 20 client.24977.objecter _get_session s=0x7f03a812bc20 osd=4 3 2024-02-23T00:42:59.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.686+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_remove -1 9 2024-02-23T00:42:59.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.686+0000 7f03acd6e700 20 client.24977.objecter get_session s=0x7f03a812bc20 osd=4 3 2024-02-23T00:42:59.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.686+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_assign 4 9 2024-02-23T00:42:59.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.686+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a812bc20 osd=4 4 2024-02-23T00:42:59.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.686+0000 7f03acd6e700 10 client.24977.objecter _send_command 9 2024-02-23T00:42:59.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.686+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f03a8134dc0 con 0x7f03a812fd80 2024-02-23T00:42:59.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.686+0000 7f03a659c700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f03a812fd80 0x7f03a8132170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:59.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.687+0000 7f03a659c700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f03a812fd80 0x7f03a8132170 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:59.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.687+0000 7f039f7fe700 10 client.24977.objecter ms_handle_connect 0x7f03a812fd80 2024-02-23T00:42:59.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.687+0000 7f039f7fe700 1 -- 172.21.15.47:0/826605645 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f03a8134dc0 con 0x7f03a812fd80 2024-02-23T00:42:59.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.687+0000 7f039f7fe700 10 client.24977.objecter ms_dispatch 0x7f03a8001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:42:59.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.687+0000 7f039f7fe700 10 client.24977.objecter _finish_command 9 = system:0 2024-02-23T00:42:59.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.687+0000 7f039f7fe700 20 client.24977.objecter put_session s=0x7f03a812bc20 osd=4 4 2024-02-23T00:42:59.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.687+0000 7f039f7fe700 15 client.24977.objecter _session_command_op_remove 4 9 2024-02-23T00:42:59.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.708+0000 7f03acd6e700 10 client.24977.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:42:59.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.708+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_assign -1 10 2024-02-23T00:42:59.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.708+0000 7f03acd6e700 20 client.24977.objecter _get_session s=0x7f03a812bc20 osd=4 3 2024-02-23T00:42:59.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.708+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a812bc20 osd=4 3 2024-02-23T00:42:59.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.708+0000 7f03acd6e700 20 client.24977.objecter _get_session s=0x7f03a812bc20 osd=4 3 2024-02-23T00:42:59.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.708+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_remove -1 10 2024-02-23T00:42:59.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.708+0000 7f03acd6e700 20 client.24977.objecter get_session s=0x7f03a812bc20 osd=4 3 2024-02-23T00:42:59.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.708+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_assign 4 10 2024-02-23T00:42:59.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.708+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a812bc20 osd=4 4 2024-02-23T00:42:59.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.708+0000 7f03acd6e700 10 client.24977.objecter _send_command 10 2024-02-23T00:42:59.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.708+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f03a8132660 con 0x7f03a812fd80 2024-02-23T00:42:59.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.742+0000 7f039f7fe700 1 -- 172.21.15.47:0/826605645 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f03a8132660 con 0x7f03a812fd80 2024-02-23T00:42:59.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.743+0000 7f039f7fe700 10 client.24977.objecter ms_dispatch 0x7f03a8001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:42:59.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.743+0000 7f039f7fe700 10 client.24977.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:42:59.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.743+0000 7f039f7fe700 20 client.24977.objecter put_session s=0x7f03a812bc20 osd=4 4 2024-02-23T00:42:59.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.743+0000 7f039f7fe700 15 client.24977.objecter _session_command_op_remove 4 10 2024-02-23T00:42:59.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.744+0000 7f03acd6e700 10 client.24977.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:59.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.744+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_assign -1 11 2024-02-23T00:42:59.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.744+0000 7f03acd6e700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f03a8132d10 0x7f03a8135100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:59.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.744+0000 7f03acd6e700 20 client.24977.objecter _get_session s=0x7f03a8132660 osd=5 3 2024-02-23T00:42:59.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.744+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a8132660 osd=5 3 2024-02-23T00:42:59.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.744+0000 7f03acd6e700 20 client.24977.objecter _get_session s=0x7f03a8132660 osd=5 3 2024-02-23T00:42:59.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.744+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_remove -1 11 2024-02-23T00:42:59.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.744+0000 7f03acd6e700 20 client.24977.objecter get_session s=0x7f03a8132660 osd=5 3 2024-02-23T00:42:59.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.744+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_assign 5 11 2024-02-23T00:42:59.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.744+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a8132660 osd=5 4 2024-02-23T00:42:59.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.744+0000 7f03acd6e700 10 client.24977.objecter _send_command 11 2024-02-23T00:42:59.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.744+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f03a8137ef0 con 0x7f03a8132d10 2024-02-23T00:42:59.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.750+0000 7f03a5d9b700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f03a8132d10 0x7f03a8135100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:59.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.762+0000 7f03a5d9b700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f03a8132d10 0x7f03a8135100 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:59.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.762+0000 7f039f7fe700 10 client.24977.objecter ms_handle_connect 0x7f03a8132d10 2024-02-23T00:42:59.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.766+0000 7f039f7fe700 1 -- 172.21.15.47:0/826605645 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f03a8137ef0 con 0x7f03a8132d10 2024-02-23T00:42:59.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.766+0000 7f039f7fe700 10 client.24977.objecter ms_dispatch 0x7f03a8001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:42:59.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.766+0000 7f039f7fe700 10 client.24977.objecter _finish_command 11 = system:0 2024-02-23T00:42:59.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.766+0000 7f039f7fe700 20 client.24977.objecter put_session s=0x7f03a8132660 osd=5 4 2024-02-23T00:42:59.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.766+0000 7f039f7fe700 15 client.24977.objecter _session_command_op_remove 5 11 2024-02-23T00:42:59.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.800+0000 7f03acd6e700 10 client.24977.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:42:59.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.800+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_assign -1 12 2024-02-23T00:42:59.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.800+0000 7f03acd6e700 20 client.24977.objecter _get_session s=0x7f03a8132660 osd=5 3 2024-02-23T00:42:59.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.800+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a8132660 osd=5 3 2024-02-23T00:42:59.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.800+0000 7f03acd6e700 20 client.24977.objecter _get_session s=0x7f03a8132660 osd=5 3 2024-02-23T00:42:59.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.800+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_remove -1 12 2024-02-23T00:42:59.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.800+0000 7f03acd6e700 20 client.24977.objecter get_session s=0x7f03a8132660 osd=5 3 2024-02-23T00:42:59.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.800+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_assign 5 12 2024-02-23T00:42:59.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.800+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a8132660 osd=5 4 2024-02-23T00:42:59.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.800+0000 7f03acd6e700 10 client.24977.objecter _send_command 12 2024-02-23T00:42:59.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.800+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f03a8137ef0 con 0x7f03a8132d10 2024-02-23T00:42:59.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.803+0000 7f039f7fe700 1 -- 172.21.15.47:0/826605645 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f03a8137ef0 con 0x7f03a8132d10 2024-02-23T00:42:59.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.803+0000 7f039f7fe700 10 client.24977.objecter ms_dispatch 0x7f03a8001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:42:59.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.803+0000 7f039f7fe700 10 client.24977.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:42:59.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.803+0000 7f039f7fe700 20 client.24977.objecter put_session s=0x7f03a8132660 osd=5 4 2024-02-23T00:42:59.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.803+0000 7f039f7fe700 15 client.24977.objecter _session_command_op_remove 5 12 2024-02-23T00:42:59.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.804+0000 7f03acd6e700 10 client.24977.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:59.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.804+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_assign -1 13 2024-02-23T00:42:59.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.804+0000 7f03acd6e700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f03a8135c40 0x7f03a81360a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:59.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.804+0000 7f03acd6e700 20 client.24977.objecter _get_session s=0x7f03a8137ef0 osd=6 3 2024-02-23T00:42:59.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.804+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a8137ef0 osd=6 3 2024-02-23T00:42:59.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.804+0000 7f03acd6e700 20 client.24977.objecter _get_session s=0x7f03a8137ef0 osd=6 3 2024-02-23T00:42:59.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.804+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_remove -1 13 2024-02-23T00:42:59.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.804+0000 7f03acd6e700 20 client.24977.objecter get_session s=0x7f03a8137ef0 osd=6 3 2024-02-23T00:42:59.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.804+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_assign 6 13 2024-02-23T00:42:59.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.804+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a8137ef0 osd=6 4 2024-02-23T00:42:59.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.804+0000 7f03acd6e700 10 client.24977.objecter _send_command 13 2024-02-23T00:42:59.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.804+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f03a813b2a0 con 0x7f03a8135c40 2024-02-23T00:42:59.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.804+0000 7f03a6d9d700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f03a8135c40 0x7f03a81360a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:59.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.804+0000 7f03a6d9d700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f03a8135c40 0x7f03a81360a0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:59.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.804+0000 7f039f7fe700 10 client.24977.objecter ms_handle_connect 0x7f03a8135c40 2024-02-23T00:42:59.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.805+0000 7f039f7fe700 1 -- 172.21.15.47:0/826605645 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f03a813b2a0 con 0x7f03a8135c40 2024-02-23T00:42:59.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.805+0000 7f039f7fe700 10 client.24977.objecter ms_dispatch 0x7f03a8001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:42:59.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.805+0000 7f039f7fe700 10 client.24977.objecter _finish_command 13 = system:0 2024-02-23T00:42:59.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.805+0000 7f039f7fe700 20 client.24977.objecter put_session s=0x7f03a8137ef0 osd=6 4 2024-02-23T00:42:59.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.805+0000 7f039f7fe700 15 client.24977.objecter _session_command_op_remove 6 13 2024-02-23T00:42:59.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.834+0000 7f03acd6e700 10 client.24977.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:42:59.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.834+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_assign -1 14 2024-02-23T00:42:59.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.834+0000 7f03acd6e700 20 client.24977.objecter _get_session s=0x7f03a8137ef0 osd=6 3 2024-02-23T00:42:59.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.834+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a8137ef0 osd=6 3 2024-02-23T00:42:59.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.834+0000 7f03acd6e700 20 client.24977.objecter _get_session s=0x7f03a8137ef0 osd=6 3 2024-02-23T00:42:59.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.834+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_remove -1 14 2024-02-23T00:42:59.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.834+0000 7f03acd6e700 20 client.24977.objecter get_session s=0x7f03a8137ef0 osd=6 3 2024-02-23T00:42:59.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.834+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_assign 6 14 2024-02-23T00:42:59.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.834+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a8137ef0 osd=6 4 2024-02-23T00:42:59.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.834+0000 7f03acd6e700 10 client.24977.objecter _send_command 14 2024-02-23T00:42:59.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.834+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f03a81366a0 con 0x7f03a8135c40 2024-02-23T00:42:59.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.836+0000 7f039f7fe700 1 -- 172.21.15.47:0/826605645 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f03a81366a0 con 0x7f03a8135c40 2024-02-23T00:42:59.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.836+0000 7f039f7fe700 10 client.24977.objecter ms_dispatch 0x7f03a8001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:42:59.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.836+0000 7f039f7fe700 10 client.24977.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:42:59.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.836+0000 7f039f7fe700 20 client.24977.objecter put_session s=0x7f03a8137ef0 osd=6 4 2024-02-23T00:42:59.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.836+0000 7f039f7fe700 15 client.24977.objecter _session_command_op_remove 6 14 2024-02-23T00:42:59.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.838+0000 7f03acd6e700 10 client.24977.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:42:59.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.838+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_assign -1 15 2024-02-23T00:42:59.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.838+0000 7f03acd6e700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f03a8136d20 0x7f03a8137180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:42:59.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.838+0000 7f03acd6e700 20 client.24977.objecter _get_session s=0x7f03a81366a0 osd=7 3 2024-02-23T00:42:59.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.838+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a81366a0 osd=7 3 2024-02-23T00:42:59.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.838+0000 7f03acd6e700 20 client.24977.objecter _get_session s=0x7f03a81366a0 osd=7 3 2024-02-23T00:42:59.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.838+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_remove -1 15 2024-02-23T00:42:59.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.838+0000 7f03acd6e700 20 client.24977.objecter get_session s=0x7f03a81366a0 osd=7 3 2024-02-23T00:42:59.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.838+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_assign 7 15 2024-02-23T00:42:59.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.838+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a81366a0 osd=7 4 2024-02-23T00:42:59.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.838+0000 7f03acd6e700 10 client.24977.objecter _send_command 15 2024-02-23T00:42:59.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.838+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f03a8137860 con 0x7f03a8136d20 2024-02-23T00:42:59.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.838+0000 7f03a659c700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f03a8136d20 0x7f03a8137180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:42:59.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.838+0000 7f039f7fe700 10 client.24977.objecter ms_handle_connect 0x7f03a8136d20 2024-02-23T00:42:59.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.838+0000 7f03a659c700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f03a8136d20 0x7f03a8137180 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:42:59.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.841+0000 7f039f7fe700 1 -- 172.21.15.47:0/826605645 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f03a8137860 con 0x7f03a8136d20 2024-02-23T00:42:59.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.841+0000 7f039f7fe700 10 client.24977.objecter ms_dispatch 0x7f03a8001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:42:59.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.841+0000 7f039f7fe700 10 client.24977.objecter _finish_command 15 = system:0 2024-02-23T00:42:59.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.841+0000 7f039f7fe700 20 client.24977.objecter put_session s=0x7f03a81366a0 osd=7 4 2024-02-23T00:42:59.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.841+0000 7f039f7fe700 15 client.24977.objecter _session_command_op_remove 7 15 2024-02-23T00:42:59.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.870+0000 7f03acd6e700 10 client.24977.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:42:59.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.870+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_assign -1 16 2024-02-23T00:42:59.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.870+0000 7f03acd6e700 20 client.24977.objecter _get_session s=0x7f03a81366a0 osd=7 3 2024-02-23T00:42:59.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.870+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a81366a0 osd=7 3 2024-02-23T00:42:59.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.870+0000 7f03acd6e700 20 client.24977.objecter _get_session s=0x7f03a81366a0 osd=7 3 2024-02-23T00:42:59.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.870+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_remove -1 16 2024-02-23T00:42:59.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.870+0000 7f03acd6e700 20 client.24977.objecter get_session s=0x7f03a81366a0 osd=7 3 2024-02-23T00:42:59.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.870+0000 7f03acd6e700 15 client.24977.objecter _session_command_op_assign 7 16 2024-02-23T00:42:59.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.870+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a81366a0 osd=7 4 2024-02-23T00:42:59.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.870+0000 7f03acd6e700 10 client.24977.objecter _send_command 16 2024-02-23T00:42:59.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.870+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f03a8137860 con 0x7f03a8136d20 2024-02-23T00:42:59.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f039f7fe700 1 -- 172.21.15.47:0/826605645 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f03a8137860 con 0x7f03a8136d20 2024-02-23T00:42:59.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f039f7fe700 10 client.24977.objecter ms_dispatch 0x7f03a8001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:42:59.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f039f7fe700 10 client.24977.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:42:59.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f039f7fe700 20 client.24977.objecter put_session s=0x7f03a81366a0 osd=7 4 2024-02-23T00:42:59.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f039f7fe700 15 client.24977.objecter _session_command_op_remove 7 16 2024-02-23T00:42:59.897 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:42:59.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 10 librados: watch_flush enter 2024-02-23T00:42:59.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 10 librados: watch_flush exit 2024-02-23T00:42:59.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 10 client.24977.objecter close_session for osd.0 2024-02-23T00:42:59.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f03a810f890 msgr2=0x7f03a80273c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:59.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f03a810f890 0x7f03a80273c0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:59.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a8104b10 osd=0 1 2024-02-23T00:42:59.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 10 client.24977.objecter close_session for osd.1 2024-02-23T00:42:59.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f03a8028590 msgr2=0x7f03a80faf10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:59.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f03a8028590 0x7f03a80faf10 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:59.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a810f200 osd=1 1 2024-02-23T00:42:59.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 10 client.24977.objecter close_session for osd.2 2024-02-23T00:42:59.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f03a8059e50 msgr2=0x7f03a812ad70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:59.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f03a8059e50 0x7f03a812ad70 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:59.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a8027f00 osd=2 1 2024-02-23T00:42:59.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 10 client.24977.objecter close_session for osd.3 2024-02-23T00:42:59.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f03a80fc130 msgr2=0x7f03a812b370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:59.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03a6d9d700 1 -- 172.21.15.47:0/826605645 reap_dead start 2024-02-23T00:42:59.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f03a80fc130 0x7f03a812b370 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:59.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a8059760 osd=3 1 2024-02-23T00:42:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 10 client.24977.objecter close_session for osd.4 2024-02-23T00:42:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f03a812fd80 msgr2=0x7f03a8132170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f03a812fd80 0x7f03a8132170 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a812bc20 osd=4 1 2024-02-23T00:42:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 10 client.24977.objecter close_session for osd.5 2024-02-23T00:42:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f03a8132d10 msgr2=0x7f03a8135100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f03a8132d10 0x7f03a8135100 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a8132660 osd=5 1 2024-02-23T00:42:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 10 client.24977.objecter close_session for osd.6 2024-02-23T00:42:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f03a8135c40 msgr2=0x7f03a81360a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f03a8135c40 0x7f03a81360a0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a8137ef0 osd=6 1 2024-02-23T00:42:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 10 client.24977.objecter close_session for osd.7 2024-02-23T00:42:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f03a8136d20 msgr2=0x7f03a8137180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f03a8136d20 0x7f03a8137180 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 20 client.24977.objecter put_session s=0x7f03a81366a0 osd=7 1 2024-02-23T00:42:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 20 client.24977.objecter shutdown clearing up homeless session... 2024-02-23T00:42:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 10 client.24977.objecter successfully canceled tick 2024-02-23T00:42:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03a6d9d700 1 -- 172.21.15.47:0/826605645 reap_dead start 2024-02-23T00:42:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0394062cc0 msgr2=0x7f0394065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0394062cc0 0x7f0394065180 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f03a81075b0 tx=0x7f039800b040).stop 2024-02-23T00:42:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f03a80f8420 msgr2=0x7f03a810ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:42:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.872+0000 7f03acd6e700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f03a80f8420 0x7f03a810ab70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f0390009da0 tx=0x7f0390009cb0).stop 2024-02-23T00:42:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.873+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 shutdown_connections 2024-02-23T00:42:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.873+0000 7f03acd6e700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f03a80f8420 0x7f03a810ab70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.873+0000 7f03acd6e700 1 --2- 172.21.15.47:0/826605645 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0394062cc0 0x7f0394065180 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:42:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.873+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 >> 172.21.15.47:0/826605645 conn(0x7f03a80f3800 msgr2=0x7f03a80f97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:42:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.873+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 shutdown_connections 2024-02-23T00:42:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.874+0000 7f03acd6e700 1 -- 172.21.15.47:0/826605645 wait complete. 2024-02-23T00:42:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:42:59.874+0000 7f03acd6e700 1 librados: shutdown 2024-02-23T00:42:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:42:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:42:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:42:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:42:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:42:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:42:59.903 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:42:59.903 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:42:59.923 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:42:59.925 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:42:59.925 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:43:00.026 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:43:00.073 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:00.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:58.583149+0000 mon.a (mon.0) 832 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:43:00.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:58.589953+0000 mon.a (mon.0) 833 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:00.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:58.590034+0000 mon.a (mon.0) 834 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi047-82428-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:00.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:58.590098+0000 mon.a (mon.0) 835 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:43:00.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:58.590156+0000 mon.a (mon.0) 836 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi047-82386-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:00.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:58.590233+0000 mon.a (mon.0) 837 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi047-82512-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:00.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:58.590300+0000 mon.a (mon.0) 838 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi047-82504-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:00.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:58.590363+0000 mon.a (mon.0) 839 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi047-82465-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:00.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:58.590427+0000 mon.a (mon.0) 840 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi047-82438-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:00.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:58.590509+0000 mon.a (mon.0) 841 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi047-82487-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:00.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:58.590577+0000 mon.a (mon.0) 842 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi047-82419-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:00.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:58.590640+0000 mon.a (mon.0) 843 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi047-82378-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:00.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:58.590742+0000 mon.a (mon.0) 844 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi047-82517-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:00.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:58.590823+0000 mon.a (mon.0) 845 : audit [INF] from='client.? 172.21.15.47:0/2649136881' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82544-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:00.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:58.590883+0000 mon.a (mon.0) 846 : audit [INF] from='client.? 172.21.15.47:0/1494681915' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi047-82496-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:00.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:58.590938+0000 mon.a (mon.0) 847 : audit [INF] from='client.? 172.21.15.47:0/2014338467' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi047-82576-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:00.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:58.590988+0000 mon.a (mon.0) 848 : audit [INF] from='client.? 172.21.15.47:0/446573282' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi047-82394-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:00.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:58.591045+0000 mon.a (mon.0) 849 : audit [INF] from='client.? 172.21.15.47:0/1911970904' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:00.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:59.131532+0000 mon.a (mon.0) 850 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi047-82457-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:00.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:59.131775+0000 mon.a (mon.0) 851 : cluster [DBG] osdmap e110: 8 total, 8 up, 8 in 2024-02-23T00:43:00.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:59.153675+0000 mon.a (mon.0) 852 : audit [INF] from='client.? 172.21.15.47:0/2832234163' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82536-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:00.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:59.162238+0000 mon.b (mon.1) 51 : audit [INF] from='client.? 172.21.15.47:0/1505701467' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:00.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:42:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:59.179138+0000 mon.a (mon.0) 853 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num_actual", "val": "22"}]: dispatch 2024-02-23T00:43:00.407 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : e list_parallel: process_1_[82604]: starting. 2024-02-23T00:43:00.407 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_1_[82604]: creating pool ceph_test_rados_list_parallel.smithi047-82584 2024-02-23T00:43:00.407 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_1_[82604]: created object 0... 2024-02-23T00:43:00.407 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_1_[82604]: created object 25... 2024-02-23T00:43:00.407 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_1_[82604]: created object 49... 2024-02-23T00:43:00.407 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_1_[82604]: finishing. 2024-02-23T00:43:00.407 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_1_[82604]: shutting down. 2024-02-23T00:43:00.407 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_2_[82605]: starting. 2024-02-23T00:43:00.408 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_2_[82605]: listing objects. 2024-02-23T00:43:00.408 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_2_[82605]: listed object 0... 2024-02-23T00:43:00.408 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_2_[82605]: listed object 25... 2024-02-23T00:43:00.408 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_2_[82605]: saw 50 objects 2024-02-23T00:43:00.408 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_2_[82605]: shutting down. 2024-02-23T00:43:00.408 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:00.408 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_3_[85130]: starting. 2024-02-23T00:43:00.408 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_3_[85130]: creating pool ceph_test_rados_list_parallel.smithi047-82584 2024-02-23T00:43:00.408 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_3_[85130]: created object 0... 2024-02-23T00:43:00.408 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_3_[85130]: created object 25... 2024-02-23T00:43:00.408 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_3_[85130]: created object 49... 2024-02-23T00:43:00.408 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_3_[85130]: finishing. 2024-02-23T00:43:00.408 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_3_[85130]: shutting down. 2024-02-23T00:43:00.408 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:00.409 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_4_[85131]: starting. 2024-02-23T00:43:00.409 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_4_[85131]: listing objects. 2024-02-23T00:43:00.409 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_4_[85131]: listed object 0... 2024-02-23T00:43:00.409 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_4_[85131]: listed object 25... 2024-02-23T00:43:00.409 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_4_[85131]: saw 50 objects 2024-02-23T00:43:00.409 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_4_[85131]: shutting down. 2024-02-23T00:43:00.409 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:00.409 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_5_[85132]: starting. 2024-02-23T00:43:00.409 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_5_[85132]: removed 25 objects... 2024-02-23T00:43:00.409 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_5_[85132]: removed half of the objects 2024-02-23T00:43:00.409 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_5_[85132]: removed 50 objects... 2024-02-23T00:43:00.409 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_5_[85132]: removed 50 objects 2024-02-23T00:43:00.409 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_5_[85132]: shutting down. 2024-02-23T00:43:00.409 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:00.410 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:00.410 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_6_[85231]: starting. 2024-02-23T00:43:00.410 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_6_[85231]: creating pool ceph_test_rados_list_parallel.smithi047-82584 2024-02-23T00:43:00.410 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_6_[85231]: created object 0... 2024-02-23T00:43:00.410 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_6_[85231]: created object 25... 2024-02-23T00:43:00.410 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_6_[85231]: created object 49... 2024-02-23T00:43:00.410 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_6_[85231]: finishing. 2024-02-23T00:43:00.410 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_6_[85231]: shutting down. 2024-02-23T00:43:00.410 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:00.410 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:00.410 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_7_[85232]: starting. 2024-02-23T00:43:00.410 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_7_[85232]: listing objects. 2024-02-23T00:43:00.410 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_7_[85232]: listed object 0... 2024-02-23T00:43:00.410 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_7_[85232]: listed object 25... 2024-02-23T00:43:00.411 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_7_[85232]: listed object 50... 2024-02-23T00:43:00.411 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_7_[85232]: saw 56 objects 2024-02-23T00:43:00.411 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_7_[85232]: shutting down. 2024-02-23T00:43:00.411 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:00.411 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:00.411 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_8_[85233]: starting. 2024-02-23T00:43:00.411 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_8_[85233]: added 25 objects... 2024-02-23T00:43:00.411 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_8_[85233]: added half of the objects 2024-02-23T00:43:00.411 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_8_[85233]: added 50 objects... 2024-02-23T00:43:00.411 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_8_[85233]: added 50 objects 2024-02-23T00:43:00.411 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_8_[85233]: shutting down. 2024-02-23T00:43:00.411 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:00.411 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:00.719 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:43:00] "GET /metrics HTTP/1.1" 200 229257 "" "Prometheus/2.33.4" 2024-02-23T00:43:01.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:58.012032+0000 osd.4 (osd.4) 73 : cluster [DBG] 4.13 scrub starts 2024-02-23T00:43:01.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:58.013684+0000 osd.4 (osd.4) 74 : cluster [DBG] 4.13 scrub ok 2024-02-23T00:43:01.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:59.175495+0000 mgr.y (mgr.14517) 147 : cluster [DBG] pgmap v139: 900 pgs: 384 unknown, 384 creating+peering, 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 847 B/s rd, 0 op/s 2024-02-23T00:43:01.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:59.300758+0000 osd.7 (osd.7) 77 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:43:01.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:59.300796+0000 osd.7 (osd.7) 78 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:43:01.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:59.461355+0000 mon.b (mon.1) 52 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:01.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:59.553005+0000 mon.c (mon.2) 32 : audit [DBG] from='client.? 172.21.15.47:0/826605645' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:43:01.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:59.710617+0000 mon.a (mon.0) 854 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi047-82528-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1"}]': finished 2024-02-23T00:43:01.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:59.710865+0000 mon.a (mon.0) 855 : audit [INF] from='client.? 172.21.15.47:0/2832234163' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82536-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:01.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:59.710954+0000 mon.a (mon.0) 856 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num_actual", "val": "22"}]': finished 2024-02-23T00:43:01.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:42:59.712409+0000 mon.a (mon.0) 857 : cluster [DBG] osdmap e111: 8 total, 8 up, 8 in 2024-02-23T00:43:01.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:59.718270+0000 mon.b (mon.1) 53 : audit [DBG] from='client.? 172.21.15.47:0/2469090228' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-23T00:43:01.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:59.732906+0000 mon.c (mon.2) 33 : audit [INF] from='client.? 172.21.15.47:0/1913712492' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:01.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:59.735048+0000 mon.a (mon.0) 858 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:01.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:59.739056+0000 mon.a (mon.0) 859 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:01.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:59.753103+0000 mon.b (mon.1) 54 : audit [INF] from='client.? 172.21.15.47:0/1505701467' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:01.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:59.760907+0000 mon.a (mon.0) 860 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:01.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:59.765188+0000 mon.b (mon.1) 55 : audit [INF] from='client.? 172.21.15.47:0/1505701467' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi047-82402-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:01.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:42:59.768024+0000 mon.a (mon.0) 861 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi047-82402-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:01.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:58.012032+0000 osd.4 (osd.4) 73 : cluster [DBG] 4.13 scrub starts 2024-02-23T00:43:01.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:58.013684+0000 osd.4 (osd.4) 74 : cluster [DBG] 4.13 scrub ok 2024-02-23T00:43:01.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:59.175495+0000 mgr.y (mgr.14517) 147 : cluster [DBG] pgmap v139: 900 pgs: 384 unknown, 384 creating+peering, 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 847 B/s rd, 0 op/s 2024-02-23T00:43:01.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:59.300758+0000 osd.7 (osd.7) 77 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:43:01.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:59.300796+0000 osd.7 (osd.7) 78 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:43:01.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:59.461355+0000 mon.b (mon.1) 52 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:01.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:59.553005+0000 mon.c (mon.2) 32 : audit [DBG] from='client.? 172.21.15.47:0/826605645' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:43:01.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:59.710617+0000 mon.a (mon.0) 854 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi047-82528-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1"}]': finished 2024-02-23T00:43:01.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:59.710865+0000 mon.a (mon.0) 855 : audit [INF] from='client.? 172.21.15.47:0/2832234163' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82536-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:01.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:59.710954+0000 mon.a (mon.0) 856 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num_actual", "val": "22"}]': finished 2024-02-23T00:43:01.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:42:59.712409+0000 mon.a (mon.0) 857 : cluster [DBG] osdmap e111: 8 total, 8 up, 8 in 2024-02-23T00:43:01.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:59.718270+0000 mon.b (mon.1) 53 : audit [DBG] from='client.? 172.21.15.47:0/2469090228' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-23T00:43:01.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:59.732906+0000 mon.c (mon.2) 33 : audit [INF] from='client.? 172.21.15.47:0/1913712492' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:01.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:59.735048+0000 mon.a (mon.0) 858 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:01.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:59.739056+0000 mon.a (mon.0) 859 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:01.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:59.753103+0000 mon.b (mon.1) 54 : audit [INF] from='client.? 172.21.15.47:0/1505701467' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:01.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:59.760907+0000 mon.a (mon.0) 860 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:01.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:59.765188+0000 mon.b (mon.1) 55 : audit [INF] from='client.? 172.21.15.47:0/1505701467' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi047-82402-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:01.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:42:59.768024+0000 mon.a (mon.0) 861 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi047-82402-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:01.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:58.012032+0000 osd.4 (osd.4) 73 : cluster [DBG] 4.13 scrub starts 2024-02-23T00:43:01.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:58.013684+0000 osd.4 (osd.4) 74 : cluster [DBG] 4.13 scrub ok 2024-02-23T00:43:01.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:59.175495+0000 mgr.y (mgr.14517) 147 : cluster [DBG] pgmap v139: 900 pgs: 384 unknown, 384 creating+peering, 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 847 B/s rd, 0 op/s 2024-02-23T00:43:01.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:59.300758+0000 osd.7 (osd.7) 77 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:43:01.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:59.300796+0000 osd.7 (osd.7) 78 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:43:01.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:59.461355+0000 mon.b (mon.1) 52 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:01.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:59.553005+0000 mon.c (mon.2) 32 : audit [DBG] from='client.? 172.21.15.47:0/826605645' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:43:01.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:59.710617+0000 mon.a (mon.0) 854 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi047-82528-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1"}]': finished 2024-02-23T00:43:01.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:59.710865+0000 mon.a (mon.0) 855 : audit [INF] from='client.? 172.21.15.47:0/2832234163' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82536-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:01.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:59.710954+0000 mon.a (mon.0) 856 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num_actual", "val": "22"}]': finished 2024-02-23T00:43:01.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:42:59.712409+0000 mon.a (mon.0) 857 : cluster [DBG] osdmap e111: 8 total, 8 up, 8 in 2024-02-23T00:43:01.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:59.718270+0000 mon.b (mon.1) 53 : audit [DBG] from='client.? 172.21.15.47:0/2469090228' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-23T00:43:01.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:59.732906+0000 mon.c (mon.2) 33 : audit [INF] from='client.? 172.21.15.47:0/1913712492' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:01.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:59.735048+0000 mon.a (mon.0) 858 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:01.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:59.739056+0000 mon.a (mon.0) 859 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:01.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:59.753103+0000 mon.b (mon.1) 54 : audit [INF] from='client.? 172.21.15.47:0/1505701467' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:01.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:59.760907+0000 mon.a (mon.0) 860 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:01.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:59.765188+0000 mon.b (mon.1) 55 : audit [INF] from='client.? 172.21.15.47:0/1505701467' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi047-82402-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:01.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:42:59.768024+0000 mon.a (mon.0) 861 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi047-82402-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:01.656 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:43:01.659 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:01.659 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:43:01.760 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:43:01.792 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:01.829 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:43:01.834 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:01.835 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:43:01.840 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ********************** api_c_read_operations: Running main() from gmock_main.cc 2024-02-23T00:43:01.840 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [==========] Running 16 tests from 1 test suite. 2024-02-23T00:43:01.840 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [----------] Global test environment set-up. 2024-02-23T00:43:01.840 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [----------] 16 tests from CReadOpsTest 2024-02-23T00:43:01.840 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2024-02-23T00:43:01.840 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2024-02-23T00:43:01.840 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2024-02-23T00:43:01.840 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (7 ms) 2024-02-23T00:43:01.840 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2024-02-23T00:43:01.841 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (6 ms) 2024-02-23T00:43:01.841 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2024-02-23T00:43:01.841 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (6 ms) 2024-02-23T00:43:01.841 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2024-02-23T00:43:01.841 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (33 ms) 2024-02-23T00:43:01.841 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2024-02-23T00:43:01.841 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (6 ms) 2024-02-23T00:43:01.841 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2024-02-23T00:43:01.841 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (13 ms) 2024-02-23T00:43:01.841 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2024-02-23T00:43:01.841 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (6 ms) 2024-02-23T00:43:01.841 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2024-02-23T00:43:01.841 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (6 ms) 2024-02-23T00:43:01.841 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2024-02-23T00:43:01.841 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (2 ms) 2024-02-23T00:43:01.842 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2024-02-23T00:43:01.842 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (8 ms) 2024-02-23T00:43:01.842 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2024-02-23T00:43:01.842 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (6 ms) 2024-02-23T00:43:01.842 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2024-02-23T00:43:01.842 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (15 ms) 2024-02-23T00:43:01.842 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2024-02-23T00:43:01.842 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (11 ms) 2024-02-23T00:43:01.842 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2024-02-23T00:43:01.842 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (11 ms) 2024-02-23T00:43:01.842 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2024-02-23T00:43:01.842 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (16 ms) 2024-02-23T00:43:01.842 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [----------] 16 tests from CReadOpsTest (153 ms total) 2024-02-23T00:43:01.842 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: 2024-02-23T00:43:01.842 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [----------] Global test environment tear-down 2024-02-23T00:43:01.843 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [==========] 16 tests from 1 test suite ran. (14618 ms total) 2024-02-23T00:43:01.843 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_read_operations: [ PASSED ] 16 tests. 2024-02-23T00:43:01.851 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd_pp: Running main() from gmock_main.cc 2024-02-23T00:43:01.851 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2024-02-23T00:43:01.851 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd_pp: [----------] Global test environment set-up. 2024-02-23T00:43:01.851 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2024-02-23T00:43:01.851 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2024-02-23T00:43:01.852 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (108 ms) 2024-02-23T00:43:01.852 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2024-02-23T00:43:01.852 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (468 ms) 2024-02-23T00:43:01.852 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2024-02-23T00:43:01.852 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (14058 ms) 2024-02-23T00:43:01.852 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (14634 ms total) 2024-02-23T00:43:01.852 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd_pp: 2024-02-23T00:43:01.852 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd_pp: [----------] Global test environment tear-down 2024-02-23T00:43:01.852 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (14634 ms total) 2024-02-23T00:43:01.852 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2024-02-23T00:43:01.936 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:43:01.979 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:02.007 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:43:02.011 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:02.011 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:43:02.050 INFO:tasks.workunit.client.0.smithi047.stdout:********* 2024-02-23T00:43:02.050 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_9_[85478]: starting. 2024-02-23T00:43:02.050 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_9_[85478]: creating pool ceph_test_rados_list_parallel.smithi047-82584 2024-02-23T00:43:02.050 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_9_[85478]: created object 0... 2024-02-23T00:43:02.050 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_9_[85478]: created object 25... 2024-02-23T00:43:02.050 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_9_[85478]: created object 49... 2024-02-23T00:43:02.050 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_9_[85478]: finishing. 2024-02-23T00:43:02.050 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_9_[85478]: shutting down. 2024-02-23T00:43:02.051 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:02.051 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:02.051 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:02.051 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_13_[85482]: starting. 2024-02-23T00:43:02.051 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_13_[85482]: removed 25 objects... 2024-02-23T00:43:02.051 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_13_[85482]: removed half of the objects 2024-02-23T00:43:02.051 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_13_[85482]: removed 50 objects... 2024-02-23T00:43:02.051 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_13_[85482]: removed 50 objects 2024-02-23T00:43:02.051 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_13_[85482]: shutting down. 2024-02-23T00:43:02.051 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:02.051 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:02.051 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:02.051 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_11_[85480]: starting. 2024-02-23T00:43:02.051 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_11_[85480]: added 25 objects... 2024-02-23T00:43:02.051 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_11_[85480]: added half of the objects 2024-02-23T00:43:02.052 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_11_[85480]: added 50 objects... 2024-02-23T00:43:02.052 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_11_[85480]: added 50 objects 2024-02-23T00:43:02.052 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_11_[85480]: shutting down. 2024-02-23T00:43:02.052 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:02.052 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:02.052 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:02.052 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_12_[85481]: starting. 2024-02-23T00:43:02.052 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_12_[85481]: added 25 objects... 2024-02-23T00:43:02.052 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_12_[85481]: added half of the objects 2024-02-23T00:43:02.052 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_12_[85481]: added 50 objects... 2024-02-23T00:43:02.052 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_12_[85481]: added 50 objects 2024-02-23T00:43:02.052 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_12_[85481]: shutting down. 2024-02-23T00:43:02.052 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:02.052 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:02.053 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:02.053 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_10_[85479]: starting. 2024-02-23T00:43:02.053 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_10_[85479]: listing objects. 2024-02-23T00:43:02.053 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_10_[85479]: listed object 0... 2024-02-23T00:43:02.053 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_10_[85479]: listed object 25... 2024-02-23T00:43:02.053 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_10_[85479]: listed object 50... 2024-02-23T00:43:02.053 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_10_[85479]: listed object 75... 2024-02-23T00:43:02.053 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_10_[85479]: saw 100 objects 2024-02-23T00:43:02.053 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_10_[85479]: shutting down. 2024-02-23T00:43:02.053 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:02.053 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:02.053 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:02.053 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:02.053 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_14_[85697]: starting. 2024-02-23T00:43:02.053 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_14_[85697]: creating pool ceph_test_rados_list_parallel.smithi047-82584 2024-02-23T00:43:02.054 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_14_[85697]: created object 0... 2024-02-23T00:43:02.054 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_14_[85697]: created object 25... 2024-02-23T00:43:02.054 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_14_[85697]: created object 49... 2024-02-23T00:43:02.054 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_14_[85697]: finishing. 2024-02-23T00:43:02.054 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_14_[85697]: shutting down. 2024-02-23T00:43:02.054 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:02.054 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:02.054 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:02.054 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:02.054 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_15_[85698]: starting. 2024-02-23T00:43:02.054 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_15_[85698]: listing objects. 2024-02-23T00:43:02.054 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_15_[85698]: listed object 0... 2024-02-23T00:43:02.112 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:43:02.141 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:02.161 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:43:02.165 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:02.165 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:43:02.266 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:43:02.283 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:43:02.283 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-02-23T00:43:02.284 INFO:teuthology.orchestra.run.smithi047.stdout: "num_ops": 0 2024-02-23T00:43:02.284 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:43:02.296 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:02.307 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_15_[85698]: listed object api_cmd: Running main() from gmock_main.cc 2024-02-23T00:43:02.307 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2024-02-23T00:43:02.307 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: [----------] Global test environment set-up. 2024-02-23T00:43:02.307 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2024-02-23T00:43:02.307 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2024-02-23T00:43:02.307 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (10416 ms) 2024-02-23T00:43:02.307 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2024-02-23T00:43:02.307 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (47 ms) 2024-02-23T00:43:02.307 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2024-02-23T00:43:02.307 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (4122 ms) 2024-02-23T00:43:02.307 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2024-02-23T00:43:02.308 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:42:59.710617+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi047-82528-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1"}]': finished 2024-02-23T00:43:02.308 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:42:59.710865+0000 mon.a [INF] from='client.? 172.21.15.47:0/2832234163' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82536-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:02.308 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:42:59.710954+0000 mon.a [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num_actual", "val": "22"}]': finished 2024-02-23T00:43:02.308 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:42:59.732906+0000 mon.c [INF] from='client.? 172.21.15.47:0/1913712492' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:02.308 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:42:59.735048+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:02.308 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:42:59.739056+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:02.308 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:42:59.753103+0000 mon.b [INF] from='client.? 172.21.15.47:0/1505701467' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:02.308 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:42:59.760907+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:02.308 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:42:59.765188+0000 mon.b [INF] from='client.? 172.21.15.47:0/1505701467' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi047-82402-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:02.308 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:42:59.768024+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi047-82402-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:02.308 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:00.826837+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:02.308 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:00.827023+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi047-82402-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:02.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:00.462741+0000 mon.b (mon.1) 56 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:02.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:00.826837+0000 mon.a (mon.0) 862 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:02.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:00.827023+0000 mon.a (mon.0) 863 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi047-82402-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:02.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:00.828086+0000 mon.b (mon.1) 57 : audit [INF] from='client.? 172.21.15.47:0/1505701467' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi047-82402-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:02.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:00.834069+0000 mon.b (mon.1) 58 : audit [INF] from='client.? 172.21.15.47:0/854895257' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi047-82386-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:02.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:01.176127+0000 mgr.y (mgr.14517) 148 : cluster [DBG] pgmap v142: 620 pgs: 296 unknown, 192 creating+peering, 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:43:02.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:01.465367+0000 mon.b (mon.1) 59 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:02.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:01.803189+0000 mon.a (mon.0) 864 : cluster [DBG] osdmap e112: 8 total, 8 up, 8 in 2024-02-23T00:43:02.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:01.809234+0000 mon.a (mon.0) 865 : audit [INF] from='client.? 172.21.15.47:0/3592004948' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi047-82378-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:02.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:01.813571+0000 mon.a (mon.0) 866 : audit [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi047-82504-6"}]: dispatch 2024-02-23T00:43:02.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:01.814314+0000 mon.a (mon.0) 867 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi047-82402-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:02.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:01.820342+0000 mon.a (mon.0) 868 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi047-82386-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:02.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:01.828285+0000 mon.b (mon.1) 60 : audit [INF] from='client.? 172.21.15.47:0/851664869' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:02.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:00.462741+0000 mon.b (mon.1) 56 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:02.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:00.826837+0000 mon.a (mon.0) 862 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:02.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:00.827023+0000 mon.a (mon.0) 863 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi047-82402-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:02.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:00.828086+0000 mon.b (mon.1) 57 : audit [INF] from='client.? 172.21.15.47:0/1505701467' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi047-82402-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:02.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:00.834069+0000 mon.b (mon.1) 58 : audit [INF] from='client.? 172.21.15.47:0/854895257' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi047-82386-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:02.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:01.176127+0000 mgr.y (mgr.14517) 148 : cluster [DBG] pgmap v142: 620 pgs: 296 unknown, 192 creating+peering, 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:43:02.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:01.465367+0000 mon.b (mon.1) 59 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:02.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:01.803189+0000 mon.a (mon.0) 864 : cluster [DBG] osdmap e112: 8 total, 8 up, 8 in 2024-02-23T00:43:02.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:01.809234+0000 mon.a (mon.0) 865 : audit [INF] from='client.? 172.21.15.47:0/3592004948' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi047-82378-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:02.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:01.813571+0000 mon.a (mon.0) 866 : audit [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi047-82504-6"}]: dispatch 2024-02-23T00:43:02.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:01.814314+0000 mon.a (mon.0) 867 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi047-82402-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:02.561 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:01.820342+0000 mon.a (mon.0) 868 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi047-82386-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:02.561 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:01.828285+0000 mon.b (mon.1) 60 : audit [INF] from='client.? 172.21.15.47:0/851664869' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:02.561 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:43:02.562 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:02.563 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:43:02.664 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:43:02.690 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:43:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:43:02.690 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:43:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:43:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:43:02.690 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:00.462741+0000 mon.b (mon.1) 56 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:02.690 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:00.826837+0000 mon.a (mon.0) 862 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:02.690 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:00.827023+0000 mon.a (mon.0) 863 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi047-82402-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:02.691 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:00.828086+0000 mon.b (mon.1) 57 : audit [INF] from='client.? 172.21.15.47:0/1505701467' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi047-82402-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:02.691 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:00.834069+0000 mon.b (mon.1) 58 : audit [INF] from='client.? 172.21.15.47:0/854895257' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi047-82386-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:02.691 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:01.176127+0000 mgr.y (mgr.14517) 148 : cluster [DBG] pgmap v142: 620 pgs: 296 unknown, 192 creating+peering, 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:43:02.691 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:01.465367+0000 mon.b (mon.1) 59 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:02.691 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:01.803189+0000 mon.a (mon.0) 864 : cluster [DBG] osdmap e112: 8 total, 8 up, 8 in 2024-02-23T00:43:02.691 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:01.809234+0000 mon.a (mon.0) 865 : audit [INF] from='client.? 172.21.15.47:0/3592004948' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi047-82378-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:02.691 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:01.813571+0000 mon.a (mon.0) 866 : audit [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi047-82504-6"}]: dispatch 2024-02-23T00:43:02.691 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:01.814314+0000 mon.a (mon.0) 867 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi047-82402-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:02.691 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:01.820342+0000 mon.a (mon.0) 868 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi047-82386-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:02.691 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:01.828285+0000 mon.b (mon.1) 60 : audit [INF] from='client.? 172.21.15.47:0/851664869' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:02.692 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:02.711 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:43:02.713 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:02.714 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:43:02.815 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:43:02.843 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:03.121 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:43:03.492 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:43:03.495 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:03.495 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:43:03.529 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.2 dump_blocked_ops 2024-02-23T00:43:03.551 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:43:03.597 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:43:03.629 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:03.651 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:43:03.653 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:03.654 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:43:03.677 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-23T00:43:03.692 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:00.828086+0000 mon.b [INF] from='client.? 172.21.15.47:0/1505701467' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi047-82402-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:03.692 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:00.834069+0000 mon.b [INF] from='client.? 172.21.15.47:0/854895257' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi047-82386-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:03.692 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:01.809234+0000 mon.a [INF] from='client.? 172.21.15.47:0/3592004948' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi047-82378-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:03.692 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:01.813571+0000 mon.a [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi047-82504-6"}]: dispatch 2024-02-23T00:43:03.692 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:01.814314+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi047-82402-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:03.692 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:01.820342+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi047-82386-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:03.692 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:01.828285+0000 mon.b [INF] from='client.? 172.21.15.47:0/851664869' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:03.693 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:01.860466+0000 mon.b [INF] from='client.? 172.21.15.47:0/3655582872' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-23T00:43:03.693 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:01.865131+0000 mon.b [INF] from='client.? 172.21.15.47:0/1728200087' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:03.693 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:02.064389+0000 mon.b [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.0"}]: dispatch 2024-02-23T00:43:03.693 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:02.065134+0000 mon.b [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.1"}]: dispatch 2024-02-23T00:43:03.693 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:02.065688+0000 mon.b [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.2"}]: dispatch 2024-02-23T00:43:03.693 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:02.066239+0000 mon.b [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.3"}]: dispatch 2024-02-23T00:43:03.693 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:02.066786+0000 mon.b [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.4"}]: dispatch 2024-02-23T00:43:03.693 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:02.067302+0000 mon.b [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.5"}]: dispatch 2024-02-23T00:43:03.693 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:02.067828+0000 mon.b [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.6"}]: dispatch 2024-02-23T00:43:03.693 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:02.068519+0000 mon.b [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.7"}]: dispatch 2024-02-23T00:43:03.755 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:43:03.787 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:03.805 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:43:03.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:01.860466+0000 mon.b (mon.1) 61 : audit [INF] from='client.? 172.21.15.47:0/3655582872' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-23T00:43:03.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:01.865131+0000 mon.b (mon.1) 62 : audit [INF] from='client.? 172.21.15.47:0/1728200087' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:03.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:02.064389+0000 mon.b (mon.1) 63 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.0"}]: dispatch 2024-02-23T00:43:03.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:02.064601+0000 mgr.y (mgr.14517) 149 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.0"}]: dispatch 2024-02-23T00:43:03.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:02.065134+0000 mon.b (mon.1) 64 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.1"}]: dispatch 2024-02-23T00:43:03.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:02.065254+0000 mgr.y (mgr.14517) 150 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.1"}]: dispatch 2024-02-23T00:43:03.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:02.065688+0000 mon.b (mon.1) 65 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.2"}]: dispatch 2024-02-23T00:43:03.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:02.065783+0000 mgr.y (mgr.14517) 151 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.2"}]: dispatch 2024-02-23T00:43:03.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:02.066239+0000 mon.b (mon.1) 66 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.3"}]: dispatch 2024-02-23T00:43:03.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:02.066343+0000 mgr.y (mgr.14517) 152 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.3"}]: dispatch 2024-02-23T00:43:03.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:02.066786+0000 mon.b (mon.1) 67 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.4"}]: dispatch 2024-02-23T00:43:03.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:02.066890+0000 mgr.y (mgr.14517) 153 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.4"}]: dispatch 2024-02-23T00:43:03.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:02.067302+0000 mon.b (mon.1) 68 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.5"}]: dispatch 2024-02-23T00:43:03.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:02.067405+0000 mgr.y (mgr.14517) 154 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.5"}]: dispatch 2024-02-23T00:43:03.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:02.067828+0000 mon.b (mon.1) 69 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.6"}]: dispatch 2024-02-23T00:43:03.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:02.067970+0000 mgr.y (mgr.14517) 155 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.6"}]: dispatch 2024-02-23T00:43:03.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:02.068519+0000 mon.b (mon.1) 70 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.7"}]: dispatch 2024-02-23T00:43:03.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:02.068642+0000 mgr.y (mgr.14517) 156 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.7"}]: dispatch 2024-02-23T00:43:03.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:02.069141+0000 mon.b (mon.1) 71 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.8"}]: dispatch 2024-02-23T00:43:03.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:02.069274+0000 mgr.y (mgr.14517) 157 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.8"}]: dispatch 2024-02-23T00:43:03.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:02.069753+0000 mon.b (mon.1) 72 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.9"}]: dispatch 2024-02-23T00:43:03.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:02.069880+0000 mgr.y (mgr.14517) 158 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.9"}]: dispatch 2024-02-23T00:43:03.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:02.294218+0000 mgr.y (mgr.14517) 159 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:43:03.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:02.310077+0000 mon.a (mon.0) 869 : audit [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi047-82504-6"}]: dispatch 2024-02-23T00:43:03.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:02.314747+0000 mon.a (mon.0) 870 : audit [INF] from='client.? 172.21.15.47:0/3592004948' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi047-82378-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:03.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:02.318571+0000 mon.c (mon.2) 34 : audit [INF] from='client.? 172.21.15.47:0/550771032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:03.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:02.467381+0000 mon.b (mon.1) 73 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:03.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:02.755248+0000 mon.a (mon.0) 871 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi047-82386-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:03.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:03.191982+0000 mon.a (mon.0) 872 : cluster [DBG] osdmap e113: 8 total, 8 up, 8 in 2024-02-23T00:43:03.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:03.201343+0000 mon.a (mon.0) 873 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi047-82402-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi047-82402-22"}]': finished 2024-02-23T00:43:03.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:03.201624+0000 mon.a (mon.0) 874 : cluster [DBG] osdmap e114: 8 total, 8 up, 8 in 2024-02-23T00:43:03.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:03.210614+0000 mon.c (mon.2) 35 : audit [INF] from='client.? 172.21.15.47:0/1913712492' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:03.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:03.212985+0000 mon.a (mon.0) 875 : audit [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi047-82504-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:03.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:03.214124+0000 mon.a (mon.0) 876 : audit [INF] from='client.? 172.21.15.47:0/30649521' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:03.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:03.217306+0000 mon.a (mon.0) 877 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-23T00:43:03.810 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:03.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:01.860466+0000 mon.b (mon.1) 61 : audit [INF] from='client.? 172.21.15.47:0/3655582872' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-23T00:43:03.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:01.865131+0000 mon.b (mon.1) 62 : audit [INF] from='client.? 172.21.15.47:0/1728200087' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:03.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:02.064389+0000 mon.b (mon.1) 63 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.0"}]: dispatch 2024-02-23T00:43:03.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:02.064601+0000 mgr.y (mgr.14517) 149 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.0"}]: dispatch 2024-02-23T00:43:03.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:02.065134+0000 mon.b (mon.1) 64 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.1"}]: dispatch 2024-02-23T00:43:03.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:02.065254+0000 mgr.y (mgr.14517) 150 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.1"}]: dispatch 2024-02-23T00:43:03.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:02.065688+0000 mon.b (mon.1) 65 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.2"}]: dispatch 2024-02-23T00:43:03.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:02.065783+0000 mgr.y (mgr.14517) 151 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.2"}]: dispatch 2024-02-23T00:43:03.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:02.066239+0000 mon.b (mon.1) 66 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.3"}]: dispatch 2024-02-23T00:43:03.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:02.066343+0000 mgr.y (mgr.14517) 152 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.3"}]: dispatch 2024-02-23T00:43:03.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:02.066786+0000 mon.b (mon.1) 67 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.4"}]: dispatch 2024-02-23T00:43:03.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:02.066890+0000 mgr.y (mgr.14517) 153 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.4"}]: dispatch 2024-02-23T00:43:03.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:02.067302+0000 mon.b (mon.1) 68 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.5"}]: dispatch 2024-02-23T00:43:03.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:02.067405+0000 mgr.y (mgr.14517) 154 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.5"}]: dispatch 2024-02-23T00:43:03.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:02.067828+0000 mon.b (mon.1) 69 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.6"}]: dispatch 2024-02-23T00:43:03.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:02.067970+0000 mgr.y (mgr.14517) 155 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.6"}]: dispatch 2024-02-23T00:43:03.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:02.068519+0000 mon.b (mon.1) 70 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.7"}]: dispatch 2024-02-23T00:43:03.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:02.068642+0000 mgr.y (mgr.14517) 156 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.7"}]: dispatch 2024-02-23T00:43:03.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:02.069141+0000 mon.b (mon.1) 71 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.8"}]: dispatch 2024-02-23T00:43:03.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:02.069274+0000 mgr.y (mgr.14517) 157 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.8"}]: dispatch 2024-02-23T00:43:03.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:02.069753+0000 mon.b (mon.1) 72 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.9"}]: dispatch 2024-02-23T00:43:03.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:02.069880+0000 mgr.y (mgr.14517) 158 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.9"}]: dispatch 2024-02-23T00:43:03.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:02.294218+0000 mgr.y (mgr.14517) 159 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:43:03.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:02.310077+0000 mon.a (mon.0) 869 : audit [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi047-82504-6"}]: dispatch 2024-02-23T00:43:03.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:02.314747+0000 mon.a (mon.0) 870 : audit [INF] from='client.? 172.21.15.47:0/3592004948' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi047-82378-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:03.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:02.318571+0000 mon.c (mon.2) 34 : audit [INF] from='client.? 172.21.15.47:0/550771032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:03.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:02.467381+0000 mon.b (mon.1) 73 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:03.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:02.755248+0000 mon.a (mon.0) 871 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi047-82386-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:03.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:03.191982+0000 mon.a (mon.0) 872 : cluster [DBG] osdmap e113: 8 total, 8 up, 8 in 2024-02-23T00:43:03.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:03.201343+0000 mon.a (mon.0) 873 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi047-82402-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi047-82402-22"}]': finished 2024-02-23T00:43:03.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:03.201624+0000 mon.a (mon.0) 874 : cluster [DBG] osdmap e114: 8 total, 8 up, 8 in 2024-02-23T00:43:03.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:03.210614+0000 mon.c (mon.2) 35 : audit [INF] from='client.? 172.21.15.47:0/1913712492' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:03.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:03.212985+0000 mon.a (mon.0) 875 : audit [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi047-82504-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:03.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:03.214124+0000 mon.a (mon.0) 876 : audit [INF] from='client.? 172.21.15.47:0/30649521' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:03.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:03.217306+0000 mon.a (mon.0) 877 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-23T00:43:03.813 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:43:03.914 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:43:03.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:01.860466+0000 mon.b (mon.1) 61 : audit [INF] from='client.? 172.21.15.47:0/3655582872' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-23T00:43:03.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:01.865131+0000 mon.b (mon.1) 62 : audit [INF] from='client.? 172.21.15.47:0/1728200087' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:03.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:02.064389+0000 mon.b (mon.1) 63 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.0"}]: dispatch 2024-02-23T00:43:03.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:02.064601+0000 mgr.y (mgr.14517) 149 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.0"}]: dispatch 2024-02-23T00:43:03.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:02.065134+0000 mon.b (mon.1) 64 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.1"}]: dispatch 2024-02-23T00:43:03.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:02.065254+0000 mgr.y (mgr.14517) 150 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.1"}]: dispatch 2024-02-23T00:43:03.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:02.065688+0000 mon.b (mon.1) 65 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.2"}]: dispatch 2024-02-23T00:43:03.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:02.065783+0000 mgr.y (mgr.14517) 151 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.2"}]: dispatch 2024-02-23T00:43:03.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:02.066239+0000 mon.b (mon.1) 66 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.3"}]: dispatch 2024-02-23T00:43:03.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:02.066343+0000 mgr.y (mgr.14517) 152 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.3"}]: dispatch 2024-02-23T00:43:03.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:02.066786+0000 mon.b (mon.1) 67 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.4"}]: dispatch 2024-02-23T00:43:03.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:02.066890+0000 mgr.y (mgr.14517) 153 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.4"}]: dispatch 2024-02-23T00:43:03.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:02.067302+0000 mon.b (mon.1) 68 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.5"}]: dispatch 2024-02-23T00:43:03.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:02.067405+0000 mgr.y (mgr.14517) 154 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.5"}]: dispatch 2024-02-23T00:43:03.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:02.067828+0000 mon.b (mon.1) 69 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.6"}]: dispatch 2024-02-23T00:43:03.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:02.067970+0000 mgr.y (mgr.14517) 155 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.6"}]: dispatch 2024-02-23T00:43:03.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:02.068519+0000 mon.b (mon.1) 70 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.7"}]: dispatch 2024-02-23T00:43:03.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:02.068642+0000 mgr.y (mgr.14517) 156 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.7"}]: dispatch 2024-02-23T00:43:03.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:02.069141+0000 mon.b (mon.1) 71 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.8"}]: dispatch 2024-02-23T00:43:03.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:02.069274+0000 mgr.y (mgr.14517) 157 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.8"}]: dispatch 2024-02-23T00:43:03.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:02.069753+0000 mon.b (mon.1) 72 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.9"}]: dispatch 2024-02-23T00:43:03.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:02.069880+0000 mgr.y (mgr.14517) 158 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.9"}]: dispatch 2024-02-23T00:43:03.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:02.294218+0000 mgr.y (mgr.14517) 159 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:43:03.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:02.310077+0000 mon.a (mon.0) 869 : audit [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi047-82504-6"}]: dispatch 2024-02-23T00:43:03.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:02.314747+0000 mon.a (mon.0) 870 : audit [INF] from='client.? 172.21.15.47:0/3592004948' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi047-82378-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:03.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:02.318571+0000 mon.c (mon.2) 34 : audit [INF] from='client.? 172.21.15.47:0/550771032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:03.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:02.467381+0000 mon.b (mon.1) 73 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:03.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:02.755248+0000 mon.a (mon.0) 871 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi047-82386-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:03.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:03.191982+0000 mon.a (mon.0) 872 : cluster [DBG] osdmap e113: 8 total, 8 up, 8 in 2024-02-23T00:43:03.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:03.201343+0000 mon.a (mon.0) 873 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi047-82402-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi047-82402-22"}]': finished 2024-02-23T00:43:03.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:03.201624+0000 mon.a (mon.0) 874 : cluster [DBG] osdmap e114: 8 total, 8 up, 8 in 2024-02-23T00:43:03.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:03.210614+0000 mon.c (mon.2) 35 : audit [INF] from='client.? 172.21.15.47:0/1913712492' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:03.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:03.212985+0000 mon.a (mon.0) 875 : audit [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi047-82504-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:03.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:03.214124+0000 mon.a (mon.0) 876 : audit [INF] from='client.? 172.21.15.47:0/30649521' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:03.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:03.217306+0000 mon.a (mon.0) 877 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-23T00:43:03.946 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:03.972 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:43:03.976 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:03.976 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:43:04.077 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:43:04.109 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:04.130 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:43:04.133 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:04.134 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:43:04.221 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:02.069141+0000 mon.b [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "p25... 2024-02-23T00:43:04.221 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_15_[85698]: listed object 50... 2024-02-23T00:43:04.221 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_15_[85698]: listed object 75... 2024-02-23T00:43:04.221 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_15_[85698]: listed object 100... 2024-02-23T00:43:04.221 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_15_[85698]: listed object 125... 2024-02-23T00:43:04.221 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_15_[85698]: saw 150 objects 2024-02-23T00:43:04.221 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_15_[85698]: shutting down. 2024-02-23T00:43:04.221 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:04.222 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:04.222 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:04.222 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:04.222 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_16_[85699]: starting. 2024-02-23T00:43:04.222 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_16_[85699]: added 25 objects... 2024-02-23T00:43:04.222 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_16_[85699]: added half of the objects 2024-02-23T00:43:04.222 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_16_[85699]: added 50 objects... 2024-02-23T00:43:04.222 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_16_[85699]: added 50 objects 2024-02-23T00:43:04.222 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: process_16_[85699]: shutting down. 2024-02-23T00:43:04.222 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:04.222 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:04.222 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:04.222 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:04.222 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******************************* 2024-02-23T00:43:04.222 INFO:tasks.workunit.client.0.smithi047.stdout: list_parallel: ******* SUCCESS ********** 2024-02-23T00:43:04.234 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:43:04.300 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:04.330 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:43:04.333 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:04.333 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:43:04.379 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: Running main() from gmock_main.cc 2024-02-23T00:43:04.380 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [==========] Running 37 tests from 2 test suites. 2024-02-23T00:43:04.380 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [----------] Global test environment set-up. 2024-02-23T00:43:04.380 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP 2024-02-23T00:43:04.380 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: seed 82402 2024-02-23T00:43:04.380 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2024-02-23T00:43:04.380 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-02-23T00:43:04.380 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2024-02-23T00:43:04.380 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (96 ms) 2024-02-23T00:43:04.380 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2024-02-23T00:43:04.380 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (9 ms) 2024-02-23T00:43:04.380 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2024-02-23T00:43:04.381 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (3 ms) 2024-02-23T00:43:04.381 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2024-02-23T00:43:04.381 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (2 ms) 2024-02-23T00:43:04.381 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2024-02-23T00:43:04.381 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (2 ms) 2024-02-23T00:43:04.382 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2024-02-23T00:43:04.382 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (3 ms) 2024-02-23T00:43:04.382 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-02-23T00:43:04.382 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (2 ms) 2024-02-23T00:43:04.382 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-02-23T00:43:04.382 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (4 ms) 2024-02-23T00:43:04.382 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-02-23T00:43:04.383 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (4 ms) 2024-02-23T00:43:04.383 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-02-23T00:43:04.383 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (1 ms) 2024-02-23T00:43:04.383 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-02-23T00:43:04.383 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (4 ms) 2024-02-23T00:43:04.383 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2024-02-23T00:43:04.383 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (3 ms) 2024-02-23T00:43:04.383 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2024-02-23T00:43:04.383 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (4 ms) 2024-02-23T00:43:04.383 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-02-23T00:43:04.383 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (4 ms) 2024-02-23T00:43:04.383 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2024-02-23T00:43:04.383 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (10 ms) 2024-02-23T00:43:04.384 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-02-23T00:43:04.384 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (3 ms) 2024-02-23T00:43:04.384 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2024-02-23T00:43:04.384 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (3 ms) 2024-02-23T00:43:04.384 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-02-23T00:43:04.384 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (4 ms) 2024-02-23T00:43:04.384 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-02-23T00:43:04.384 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (2 ms) 2024-02-23T00:43:04.384 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP (163 ms total) 2024-02-23T00:43:04.384 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: 2024-02-23T00:43:04.384 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP 2024-02-23T00:43:04.384 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2024-02-23T00:43:04.384 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (1137 ms) 2024-02-23T00:43:04.384 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2024-02-23T00:43:04.385 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (11 ms) 2024-02-23T00:43:04.385 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-02-23T00:43:04.385 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (3 ms) 2024-02-23T00:43:04.385 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2024-02-23T00:43:04.385 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (3 ms) 2024-02-23T00:43:04.385 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2024-02-23T00:43:04.385 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP2 (2 ms) 2024-02-23T00:43:04.434 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:43:04.469 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:04.481 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:43:04.492 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:43:04.770 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] Lgid": "20.8"}]: dispatch 2024-02-23T00:43:04.770 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:02.069753+0000 mon.b [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.9"}]: dispatch 2024-02-23T00:43:04.770 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:02.310077+0000 mon.a [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi047-82504-6"}]: dispatch 2024-02-23T00:43:04.771 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:02.314747+0000 mon.a [INF] from='client.? 172.21.15.47:0/3592004948' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi047-82378-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:04.771 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:02.318571+0000 mon.c [INF] from='client.? 172.21.15.47:0/550771032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:04.771 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:02.755248+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi047-82386-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:04.771 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:03.201343+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi047-82402-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi047-82402-22"}]': finished 2024-02-23T00:43:04.771 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:03.210614+0000 mon.c [INF] from='client.? 172.21.15.47:0/1913712492' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:04.771 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:03.212985+0000 mon.a [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi047-82504-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:04.771 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:03.214124+0000 mon.a [INF] from='client.? 172.21.15.47:0/30649521' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:04.771 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:03.217306+0000 mon.a [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-23T00:43:04.771 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:01.860588+0000 client.admin [INF] onexx 2024-02-23T00:43:04.771 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:03.228306+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:04.771 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:03.234738+0000 mon.b [INF] from='client.? 172.21.15.47:0/851664869' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:04.771 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:03.686115+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-23T00:43:04.771 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:03.693158+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:04.772 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:03.693868+0000 mon.b [INF] from='client.? 172.21.15.47:0/1728200087' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:04.772 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:03.694593+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:04.772 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:03.695590+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:04.772 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:04.157715+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:04.772 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:04.158020+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:04.772 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:04.159102+0000 mon.b [INF] from='client.? 172.21.15.47:0/851664869' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi047-82512-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:04.772 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:04.159418+0000 mon.b [INF] from='client.? 172.21.15.47:0/1728200087' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi047-82394-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:04.772 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:04.159679+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi047-82512-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:04.772 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:04.160058+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi047-82394-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:04.772 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:04.199001+0000 mon.a [WRN] Health check failed: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:43:04.772 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:04.203999+0000 mon.a [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi047-82504-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:04.772 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:04.204113+0000 mon.a [INF] from='client.? 172.21.15.47:0/30649521' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:04.772 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:04.204191+0000 mon.a [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-23T00:43:04.772 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:04.204508+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:04.773 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:04.204632+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:04.773 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:04.204840+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi047-82512-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:04.773 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:04.204927+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi047-82394-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:04.773 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:04.211271+0000 mon.b [INF] from='client.? 172.21.15.47:0/1728200087' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi047-82394-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:04.773 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:04.211948+0000 mon.c [INF] from='client.? 172.21.15.47:0/1913712492' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-4", "tierpool": "test-rados-api-smithi047-82465-4-cache"}]: dispatch 2024-02-23T00:43:04.773 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:04.212068+0000 mon.b [INF] from='client.? 172.21.15.47:0/851664869' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi047-82512-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:04.773 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:04.223744+0000 mon.b [INF] from='client.? 172.21.15.47:0/854895257' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi047-82386-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:04.773 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:04.226840+0000 mon.a [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi047-82504-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi047-82504-6"}]: dispatch 2024-02-23T00:43:04.773 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:04.230315+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi047-82394-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:04.773 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:04.230969+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi047-82512-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:04.773 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:04.231371+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-4", "tierpool": "test-rados-api-smithi047-82465-4-cache"}]: dispatch 2024-02-23T00:43:04.773 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:04.237093+0000 mon.b [INF] from='client.? 172.21.15.47:0/2670125835' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:43:04.773 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:04.240136+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi047-82386-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:04.774 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:04.240602+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:43:04.774 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:04.261489+0000 mon.c [INF] from='client.? 172.21.15.47:0/1119363424' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:04.774 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:04.263149+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:04.774 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:04.268606+0000 mon.c [INF] from='client.? 172.21.15.47:0/1119363424' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:04.816 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-23T00:43:04.816 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-23T00:43:04.817 INFO:tasks.thrashosds.thrasher:Shrinking pool device_health_metrics 2024-02-23T00:43:04.817 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool device_health_metrics by 10 min 8 2024-02-23T00:43:04.817 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph pg dump --format=json 2024-02-23T00:43:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:43:04.932 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:04.932 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:43:05.033 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:43:05.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:01.860588+0000 client.admin (client.?) 0 : cluster [INF] onexx 2024-02-23T00:43:05.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:02.448345+0000 osd.6 (osd.6) 59 : cluster [DBG] 20.1 deep-scrub starts 2024-02-23T00:43:05.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:02.449511+0000 osd.6 (osd.6) 60 : cluster [DBG] 20.1 deep-scrub ok 2024-02-23T00:43:05.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:02.832492+0000 osd.1 (osd.1) 81 : cluster [DBG] 20.4 deep-scrub starts 2024-02-23T00:43:05.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:02.835653+0000 osd.1 (osd.1) 82 : cluster [DBG] 20.4 deep-scrub ok 2024-02-23T00:43:05.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:03.114270+0000 osd.3 (osd.3) 87 : cluster [DBG] 20.0 deep-scrub starts 2024-02-23T00:43:05.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:03.117439+0000 osd.3 (osd.3) 88 : cluster [DBG] 20.0 deep-scrub ok 2024-02-23T00:43:05.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:03.176662+0000 mgr.y (mgr.14517) 160 : cluster [DBG] pgmap v144: 716 pgs: 96 unknown, 64 creating+peering, 556 active+clean; 144 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 10 MiB/s rd, 33 MiB/s wr, 613 op/s; 80 B/s, 0 objects/s recovering 2024-02-23T00:43:05.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:03.228306+0000 mon.a (mon.0) 878 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:05.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:03.234738+0000 mon.b (mon.1) 74 : audit [INF] from='client.? 172.21.15.47:0/851664869' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:05.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:03.468532+0000 mon.b (mon.1) 75 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:05.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:03.686115+0000 mon.a (mon.0) 879 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-23T00:43:05.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:03.693158+0000 mon.a (mon.0) 880 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:05.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:03.693868+0000 mon.b (mon.1) 76 : audit [INF] from='client.? 172.21.15.47:0/1728200087' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:05.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:03.694593+0000 mon.a (mon.0) 881 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:05.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:03.695590+0000 mon.a (mon.0) 882 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:05.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:01.860588+0000 client.admin (client.?) 0 : cluster [INF] onexx 2024-02-23T00:43:05.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:02.448345+0000 osd.6 (osd.6) 59 : cluster [DBG] 20.1 deep-scrub starts 2024-02-23T00:43:05.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:02.449511+0000 osd.6 (osd.6) 60 : cluster [DBG] 20.1 deep-scrub ok 2024-02-23T00:43:05.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:02.832492+0000 osd.1 (osd.1) 81 : cluster [DBG] 20.4 deep-scrub starts 2024-02-23T00:43:05.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:02.835653+0000 osd.1 (osd.1) 82 : cluster [DBG] 20.4 deep-scrub ok 2024-02-23T00:43:05.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:03.114270+0000 osd.3 (osd.3) 87 : cluster [DBG] 20.0 deep-scrub starts 2024-02-23T00:43:05.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:03.117439+0000 osd.3 (osd.3) 88 : cluster [DBG] 20.0 deep-scrub ok 2024-02-23T00:43:05.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:03.176662+0000 mgr.y (mgr.14517) 160 : cluster [DBG] pgmap v144: 716 pgs: 96 unknown, 64 creating+peering, 556 active+clean; 144 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 10 MiB/s rd, 33 MiB/s wr, 613 op/s; 80 B/s, 0 objects/s recovering 2024-02-23T00:43:05.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:03.228306+0000 mon.a (mon.0) 878 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:05.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:03.234738+0000 mon.b (mon.1) 74 : audit [INF] from='client.? 172.21.15.47:0/851664869' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:05.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:03.468532+0000 mon.b (mon.1) 75 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:05.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:03.686115+0000 mon.a (mon.0) 879 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-23T00:43:05.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:03.693158+0000 mon.a (mon.0) 880 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:05.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:03.693868+0000 mon.b (mon.1) 76 : audit [INF] from='client.? 172.21.15.47:0/1728200087' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:05.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:03.694593+0000 mon.a (mon.0) 881 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:05.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:03.695590+0000 mon.a (mon.0) 882 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:05.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.157715+0000 mon.a (mon.0) 883 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:05.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.158020+0000 mon.a (mon.0) 884 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:05.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.159102+0000 mon.b (mon.1) 77 : audit [INF] from='client.? 172.21.15.47:0/851664869' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi047-82512-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:05.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.159418+0000 mon.b (mon.1) 78 : audit [INF] from='client.? 172.21.15.47:0/1728200087' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi047-82394-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:05.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.159679+0000 mon.a (mon.0) 885 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi047-82512-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:05.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.160058+0000 mon.a (mon.0) 886 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi047-82394-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:05.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:04.199001+0000 mon.a (mon.0) 887 : cluster [WRN] Health check failed: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:43:05.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.203999+0000 mon.a (mon.0) 888 : audit [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi047-82504-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:05.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.204113+0000 mon.a (mon.0) 889 : audit [INF] from='client.? 172.21.15.47:0/30649521' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:05.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.204191+0000 mon.a (mon.0) 890 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-23T00:43:05.061 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.204508+0000 mon.a (mon.0) 891 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:05.061 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.204632+0000 mon.a (mon.0) 892 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:05.061 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.157715+0000 mon.a (mon.0) 883 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:05.061 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.158020+0000 mon.a (mon.0) 884 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:05.061 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.159102+0000 mon.b (mon.1) 77 : audit [INF] from='client.? 172.21.15.47:0/851664869' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi047-82512-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:05.061 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.159418+0000 mon.b (mon.1) 78 : audit [INF] from='client.? 172.21.15.47:0/1728200087' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi047-82394-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:05.061 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.159679+0000 mon.a (mon.0) 885 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi047-82512-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:05.061 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.160058+0000 mon.a (mon.0) 886 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi047-82394-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:05.061 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:04.199001+0000 mon.a (mon.0) 887 : cluster [WRN] Health check failed: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:43:05.061 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.203999+0000 mon.a (mon.0) 888 : audit [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi047-82504-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:05.062 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.204113+0000 mon.a (mon.0) 889 : audit [INF] from='client.? 172.21.15.47:0/30649521' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:05.062 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.204191+0000 mon.a (mon.0) 890 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-23T00:43:05.062 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.204508+0000 mon.a (mon.0) 891 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:05.062 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.204632+0000 mon.a (mon.0) 892 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:05.062 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.204840+0000 mon.a (mon.0) 893 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi047-82512-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:05.062 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.204927+0000 mon.a (mon.0) 894 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi047-82394-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:05.062 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:04.205006+0000 mon.a (mon.0) 895 : cluster [DBG] osdmap e115: 8 total, 8 up, 8 in 2024-02-23T00:43:05.062 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.211271+0000 mon.b (mon.1) 79 : audit [INF] from='client.? 172.21.15.47:0/1728200087' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi047-82394-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:05.062 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.211948+0000 mon.c (mon.2) 36 : audit [INF] from='client.? 172.21.15.47:0/1913712492' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-4", "tierpool": "test-rados-api-smithi047-82465-4-cache"}]: dispatch 2024-02-23T00:43:05.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.212068+0000 mon.b (mon.1) 80 : audit [INF] from='client.? 172.21.15.47:0/851664869' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi047-82512-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:05.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.223744+0000 mon.b (mon.1) 81 : audit [INF] from='client.? 172.21.15.47:0/854895257' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi047-82386-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:05.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.226840+0000 mon.a (mon.0) 896 : audit [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi047-82504-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi047-82504-6"}]: dispatch 2024-02-23T00:43:05.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.230315+0000 mon.a (mon.0) 897 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi047-82394-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:05.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.230969+0000 mon.a (mon.0) 898 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi047-82512-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:05.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.231371+0000 mon.a (mon.0) 899 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-4", "tierpool": "test-rados-api-smithi047-82465-4-cache"}]: dispatch 2024-02-23T00:43:05.063 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.204840+0000 mon.a (mon.0) 893 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi047-82512-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:05.063 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.204927+0000 mon.a (mon.0) 894 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi047-82394-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:05.063 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:04.205006+0000 mon.a (mon.0) 895 : cluster [DBG] osdmap e115: 8 total, 8 up, 8 in 2024-02-23T00:43:05.063 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.211271+0000 mon.b (mon.1) 79 : audit [INF] from='client.? 172.21.15.47:0/1728200087' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi047-82394-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:05.063 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.211948+0000 mon.c (mon.2) 36 : audit [INF] from='client.? 172.21.15.47:0/1913712492' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-4", "tierpool": "test-rados-api-smithi047-82465-4-cache"}]: dispatch 2024-02-23T00:43:05.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.212068+0000 mon.b (mon.1) 80 : audit [INF] from='client.? 172.21.15.47:0/851664869' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi047-82512-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:05.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.223744+0000 mon.b (mon.1) 81 : audit [INF] from='client.? 172.21.15.47:0/854895257' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi047-82386-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:05.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.226840+0000 mon.a (mon.0) 896 : audit [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi047-82504-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi047-82504-6"}]: dispatch 2024-02-23T00:43:05.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.230315+0000 mon.a (mon.0) 897 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi047-82394-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:05.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.230969+0000 mon.a (mon.0) 898 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi047-82512-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:05.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.231371+0000 mon.a (mon.0) 899 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-4", "tierpool": "test-rados-api-smithi047-82465-4-cache"}]: dispatch 2024-02-23T00:43:05.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.237093+0000 mon.b (mon.1) 82 : audit [INF] from='client.? 172.21.15.47:0/2670125835' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:43:05.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.240136+0000 mon.a (mon.0) 900 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi047-82386-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:05.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.240602+0000 mon.a (mon.0) 901 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:43:05.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.261489+0000 mon.c (mon.2) 37 : audit [INF] from='client.? 172.21.15.47:0/1119363424' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:05.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.263149+0000 mon.a (mon.0) 902 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:05.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.268606+0000 mon.c (mon.2) 38 : audit [INF] from='client.? 172.21.15.47:0/1119363424' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:05.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.269057+0000 mon.a (mon.0) 903 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:05.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.271561+0000 mon.c (mon.2) 39 : audit [INF] from='client.? 172.21.15.47:0/1119363424' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi047-82428-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:05.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.272262+0000 mon.a (mon.0) 904 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi047-82428-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:05.065 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.237093+0000 mon.b (mon.1) 82 : audit [INF] from='client.? 172.21.15.47:0/2670125835' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:43:05.065 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.240136+0000 mon.a (mon.0) 900 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi047-82386-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:05.065 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.240602+0000 mon.a (mon.0) 901 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:43:05.065 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.261489+0000 mon.c (mon.2) 37 : audit [INF] from='client.? 172.21.15.47:0/1119363424' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:05.065 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.263149+0000 mon.a (mon.0) 902 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:05.066 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.268606+0000 mon.c (mon.2) 38 : audit [INF] from='client.? 172.21.15.47:0/1119363424' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:05.066 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.269057+0000 mon.a (mon.0) 903 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:05.066 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.271561+0000 mon.c (mon.2) 39 : audit [INF] from='client.? 172.21.15.47:0/1119363424' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi047-82428-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:05.066 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.272262+0000 mon.a (mon.0) 904 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi047-82428-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:05.067 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:01.860588+0000 client.admin (client.?) 0 : cluster [INF] onexx 2024-02-23T00:43:05.067 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:02.448345+0000 osd.6 (osd.6) 59 : cluster [DBG] 20.1 deep-scrub starts 2024-02-23T00:43:05.067 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:02.449511+0000 osd.6 (osd.6) 60 : cluster [DBG] 20.1 deep-scrub ok 2024-02-23T00:43:05.067 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:02.832492+0000 osd.1 (osd.1) 81 : cluster [DBG] 20.4 deep-scrub starts 2024-02-23T00:43:05.067 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:02.835653+0000 osd.1 (osd.1) 82 : cluster [DBG] 20.4 deep-scrub ok 2024-02-23T00:43:05.067 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:03.114270+0000 osd.3 (osd.3) 87 : cluster [DBG] 20.0 deep-scrub starts 2024-02-23T00:43:05.067 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:03.117439+0000 osd.3 (osd.3) 88 : cluster [DBG] 20.0 deep-scrub ok 2024-02-23T00:43:05.067 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:03.176662+0000 mgr.y (mgr.14517) 160 : cluster [DBG] pgmap v144: 716 pgs: 96 unknown, 64 creating+peering, 556 active+clean; 144 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 10 MiB/s rd, 33 MiB/s wr, 613 op/s; 80 B/s, 0 objects/s recovering 2024-02-23T00:43:05.067 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:03.228306+0000 mon.a (mon.0) 878 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:05.067 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:03.234738+0000 mon.b (mon.1) 74 : audit [INF] from='client.? 172.21.15.47:0/851664869' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:05.067 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:03.468532+0000 mon.b (mon.1) 75 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:05.067 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:03.686115+0000 mon.a (mon.0) 879 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-23T00:43:05.067 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:03.693158+0000 mon.a (mon.0) 880 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:05.068 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:03.693868+0000 mon.b (mon.1) 76 : audit [INF] from='client.? 172.21.15.47:0/1728200087' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:05.068 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:03.694593+0000 mon.a (mon.0) 881 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:05.068 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:03.695590+0000 mon.a (mon.0) 882 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:05.068 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.157715+0000 mon.a (mon.0) 883 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:05.068 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.158020+0000 mon.a (mon.0) 884 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:05.068 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.159102+0000 mon.b (mon.1) 77 : audit [INF] from='client.? 172.21.15.47:0/851664869' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi047-82512-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:05.068 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.159418+0000 mon.b (mon.1) 78 : audit [INF] from='client.? 172.21.15.47:0/1728200087' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi047-82394-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:05.068 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.159679+0000 mon.a (mon.0) 885 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi047-82512-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:05.068 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.160058+0000 mon.a (mon.0) 886 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi047-82394-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:05.068 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:04.199001+0000 mon.a (mon.0) 887 : cluster [WRN] Health check failed: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:43:05.068 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.203999+0000 mon.a (mon.0) 888 : audit [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi047-82504-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:05.068 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.204113+0000 mon.a (mon.0) 889 : audit [INF] from='client.? 172.21.15.47:0/30649521' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:05.068 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.204191+0000 mon.a (mon.0) 890 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-23T00:43:05.069 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.204508+0000 mon.a (mon.0) 891 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:05.069 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.204632+0000 mon.a (mon.0) 892 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:05.069 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.204840+0000 mon.a (mon.0) 893 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi047-82512-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:05.069 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.204927+0000 mon.a (mon.0) 894 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi047-82394-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:05.069 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:04.205006+0000 mon.a (mon.0) 895 : cluster [DBG] osdmap e115: 8 total, 8 up, 8 in 2024-02-23T00:43:05.069 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.211271+0000 mon.b (mon.1) 79 : audit [INF] from='client.? 172.21.15.47:0/1728200087' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi047-82394-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:05.069 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.211948+0000 mon.c (mon.2) 36 : audit [INF] from='client.? 172.21.15.47:0/1913712492' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-4", "tierpool": "test-rados-api-smithi047-82465-4-cache"}]: dispatch 2024-02-23T00:43:05.069 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.212068+0000 mon.b (mon.1) 80 : audit [INF] from='client.? 172.21.15.47:0/851664869' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi047-82512-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:05.069 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.223744+0000 mon.b (mon.1) 81 : audit [INF] from='client.? 172.21.15.47:0/854895257' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi047-82386-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:05.069 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.226840+0000 mon.a (mon.0) 896 : audit [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi047-82504-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi047-82504-6"}]: dispatch 2024-02-23T00:43:05.069 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.230315+0000 mon.a (mon.0) 897 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi047-82394-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:05.069 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.230969+0000 mon.a (mon.0) 898 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi047-82512-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:05.069 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.231371+0000 mon.a (mon.0) 899 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-4", "tierpool": "test-rados-api-smithi047-82465-4-cache"}]: dispatch 2024-02-23T00:43:05.069 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.237093+0000 mon.b (mon.1) 82 : audit [INF] from='client.? 172.21.15.47:0/2670125835' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:43:05.070 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.240136+0000 mon.a (mon.0) 900 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi047-82386-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:05.070 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.240602+0000 mon.a (mon.0) 901 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:43:05.070 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.261489+0000 mon.c (mon.2) 37 : audit [INF] from='client.? 172.21.15.47:0/1119363424' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:05.070 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.263149+0000 mon.a (mon.0) 902 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:05.070 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.268606+0000 mon.c (mon.2) 38 : audit [INF] from='client.? 172.21.15.47:0/1119363424' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:05.070 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.269057+0000 mon.a (mon.0) 903 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:05.070 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.271561+0000 mon.c (mon.2) 39 : audit [INF] from='client.? 172.21.15.47:0/1119363424' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi047-82428-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:05.070 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.272262+0000 mon.a (mon.0) 904 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi047-82428-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:05.070 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:05.083 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:43:05.086 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:05.087 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:43:05.187 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:43:05.250 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:05.281 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:43:05.290 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:05.291 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:43:05.392 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:43:05.418 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:05.436 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:43:05.438 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:05.439 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:43:05.539 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:43:05.566 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:05.584 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:43:05.586 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:05.587 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:43:05.688 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:43:05.714 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:05.733 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:43:05.735 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:05.736 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:43:05.837 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:43:05.868 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:06.096 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:43:06.102 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:06.103 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:43:06.203 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:43:06.231 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:06.561 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:04.269057+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd delete_pools_parallel: process_1_[82642]: starting. 2024-02-23T00:43:06.561 INFO:tasks.workunit.client.0.smithi047.stdout: delete_pools_parallel: process_1_[82642]: creating pool ceph_test_rados_delete_pools_parallel.smithi047-82596 2024-02-23T00:43:06.561 INFO:tasks.workunit.client.0.smithi047.stdout: delete_pools_parallel: process_1_[82642]: created object 0... 2024-02-23T00:43:06.562 INFO:tasks.workunit.client.0.smithi047.stdout: delete_pools_parallel: process_1_[82642]: created object 25... 2024-02-23T00:43:06.562 INFO:tasks.workunit.client.0.smithi047.stdout: delete_pools_parallel: process_1_[82642]: created object 49... 2024-02-23T00:43:06.562 INFO:tasks.workunit.client.0.smithi047.stdout: delete_pools_parallel: process_1_[82642]: finishing. 2024-02-23T00:43:06.562 INFO:tasks.workunit.client.0.smithi047.stdout: delete_pools_parallel: process_1_[82642]: shutting down. 2024-02-23T00:43:06.562 INFO:tasks.workunit.client.0.smithi047.stdout: delete_pools_parallel: process_2_[82643]: starting. 2024-02-23T00:43:06.562 INFO:tasks.workunit.client.0.smithi047.stdout: delete_pools_parallel: process_2_[82643]: deleting pool ceph_test_rados_delete_pools_parallel.smithi047-82596 2024-02-23T00:43:06.562 INFO:tasks.workunit.client.0.smithi047.stdout: delete_pools_parallel: process_2_[82643]: shutting down. 2024-02-23T00:43:06.562 INFO:tasks.workunit.client.0.smithi047.stdout: delete_pools_parallel: ******************************* 2024-02-23T00:43:06.562 INFO:tasks.workunit.client.0.smithi047.stdout: delete_pools_parallel: process_3_[85615]: starting. 2024-02-23T00:43:06.562 INFO:tasks.workunit.client.0.smithi047.stdout: delete_pools_parallel: process_3_[85615]: creating pool ceph_test_rados_delete_pools_parallel.smithi047-82596 2024-02-23T00:43:06.562 INFO:tasks.workunit.client.0.smithi047.stdout: delete_pools_parallel: process_3_[85615]: created object 0... 2024-02-23T00:43:06.562 INFO:tasks.workunit.client.0.smithi047.stdout: delete_pools_parallel: process_3_[85615]: created object 25... 2024-02-23T00:43:06.562 INFO:tasks.workunit.client.0.smithi047.stdout: delete_pools_parallel: process_3_[85615]: created object 49... 2024-02-23T00:43:06.562 INFO:tasks.workunit.client.0.smithi047.stdout: delete_pools_parallel: process_3_[85615]: finishing. 2024-02-23T00:43:06.562 INFO:tasks.workunit.client.0.smithi047.stdout: delete_pools_parallel: process_3_[85615]: shutting down. 2024-02-23T00:43:06.563 INFO:tasks.workunit.client.0.smithi047.stdout: delete_pools_parallel: ******************************* 2024-02-23T00:43:06.563 INFO:tasks.workunit.client.0.smithi047.stdout: delete_pools_parallel: process_5_[85617]: starting. 2024-02-23T00:43:06.563 INFO:tasks.workunit.client.0.smithi047.stdout: delete_pools_parallel: process_5_[85617]: listing objects. 2024-02-23T00:43:06.563 INFO:tasks.workunit.client.0.smithi047.stdout: delete_pools_parallel: process_5_[85617]: listed object 0... 2024-02-23T00:43:06.563 INFO:tasks.workunit.client.0.smithi047.stdout: delete_pools_parallel: process_5_[85617]: listed object 25... 2024-02-23T00:43:06.563 INFO:tasks.workunit.client.0.smithi047.stdout: delete_pools_parallel: process_5_[85617]: saw 50 objects 2024-02-23T00:43:06.563 INFO:tasks.workunit.client.0.smithi047.stdout: delete_pools_parallel: process_5_[85617]: shutting down. 2024-02-23T00:43:06.563 INFO:tasks.workunit.client.0.smithi047.stdout: delete_pools_parallel: ******************************* 2024-02-23T00:43:06.563 INFO:tasks.workunit.client.0.smithi047.stdout: delete_pools_parallel: process_4_[85616]: starting. 2024-02-23T00:43:06.563 INFO:tasks.workunit.client.0.smithi047.stdout: delete_pools_parallel: process_4_[85616]: deleting pool ceph_test_rados_delete_pools_parallel.smithi047-82596 2024-02-23T00:43:06.564 INFO:tasks.workunit.client.0.smithi047.stdout: delete_pools_parallel: process_4_[85616]: shutting down. 2024-02-23T00:43:06.564 INFO:tasks.workunit.client.0.smithi047.stdout: delete_pools_parallel: ******************************* 2024-02-23T00:43:06.564 INFO:tasks.workunit.client.0.smithi047.stdout: delete_pools_parallel: ******************************* 2024-02-23T00:43:06.564 INFO:tasks.workunit.client.0.smithi047.stdout: delete_pools_parallel: ******* SUCCESS ********** 2024-02-23T00:43:06.794 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:43:06.797 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:06.797 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:43:06.833 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:04.399818+0000 osd.0 (osd.0) 75 : cluster [DBG] 20.3 deep-scrub starts 2024-02-23T00:43:06.834 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:04.401459+0000 osd.0 (osd.0) 76 : cluster [DBG] 20.3 deep-scrub ok 2024-02-23T00:43:06.834 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.469616+0000 mon.b (mon.1) 83 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:06.834 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.766320+0000 mon.a (mon.0) 905 : audit [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi047-82438-10"}]: dispatch 2024-02-23T00:43:06.834 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.774060+0000 mon.a (mon.0) 906 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-23T00:43:06.834 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.774855+0000 mon.b (mon.1) 84 : audit [INF] from='client.? 172.21.15.47:0/3655582872' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-23T00:43:06.835 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:04.774958+0000 client.admin (client.?) 0 : cluster [INF] twoxx 2024-02-23T00:43:06.835 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.775348+0000 mon.a (mon.0) 907 : audit [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi047-82438-10"}]: dispatch 2024-02-23T00:43:06.835 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.775884+0000 mon.a (mon.0) 908 : audit [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi047-82438-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:06.835 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:04.783533+0000 mon.a (mon.0) 909 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-23T00:43:06.835 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:06.550126+0000 mon.a (mon.0) 910 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-4", "tierpool": "test-rados-api-smithi047-82465-4-cache"}]': finished 2024-02-23T00:43:06.835 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:04.399818+0000 osd.0 (osd.0) 75 : cluster [DBG] 20.3 deep-scrub starts 2024-02-23T00:43:06.835 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:04.401459+0000 osd.0 (osd.0) 76 : cluster [DBG] 20.3 deep-scrub ok 2024-02-23T00:43:06.835 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.469616+0000 mon.b (mon.1) 83 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:06.835 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.766320+0000 mon.a (mon.0) 905 : audit [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi047-82438-10"}]: dispatch 2024-02-23T00:43:06.836 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.774060+0000 mon.a (mon.0) 906 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-23T00:43:06.836 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.774855+0000 mon.b (mon.1) 84 : audit [INF] from='client.? 172.21.15.47:0/3655582872' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-23T00:43:06.836 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:04.774958+0000 client.admin (client.?) 0 : cluster [INF] twoxx 2024-02-23T00:43:06.836 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.775348+0000 mon.a (mon.0) 907 : audit [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi047-82438-10"}]: dispatch 2024-02-23T00:43:06.836 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.775884+0000 mon.a (mon.0) 908 : audit [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi047-82438-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:06.836 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:04.783533+0000 mon.a (mon.0) 909 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-23T00:43:06.836 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:06.550126+0000 mon.a (mon.0) 910 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-4", "tierpool": "test-rados-api-smithi047-82465-4-cache"}]': finished 2024-02-23T00:43:06.898 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:43:06.929 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:06.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:04.399818+0000 osd.0 (osd.0) 75 : cluster [DBG] 20.3 deep-scrub starts 2024-02-23T00:43:06.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:04.401459+0000 osd.0 (osd.0) 76 : cluster [DBG] 20.3 deep-scrub ok 2024-02-23T00:43:06.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.469616+0000 mon.b (mon.1) 83 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:06.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.766320+0000 mon.a (mon.0) 905 : audit [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi047-82438-10"}]: dispatch 2024-02-23T00:43:06.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.774060+0000 mon.a (mon.0) 906 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-23T00:43:06.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.774855+0000 mon.b (mon.1) 84 : audit [INF] from='client.? 172.21.15.47:0/3655582872' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-23T00:43:06.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:04.774958+0000 client.admin (client.?) 0 : cluster [INF] twoxx 2024-02-23T00:43:06.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.775348+0000 mon.a (mon.0) 907 : audit [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi047-82438-10"}]: dispatch 2024-02-23T00:43:06.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.775884+0000 mon.a (mon.0) 908 : audit [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi047-82438-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:06.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:04.783533+0000 mon.a (mon.0) 909 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-23T00:43:06.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:06.550126+0000 mon.a (mon.0) 910 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-4", "tierpool": "test-rados-api-smithi047-82465-4-cache"}]': finished 2024-02-23T00:43:06.983 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:43:06.985 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:06.986 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:43:07.086 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:43:07.114 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:07.312 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:43:07.314 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:07.315 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:43:07.416 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:43:07.447 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:07.469 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:43:07.471 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:07.472 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:43:07.573 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:43:07.603 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:07.624 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:43:07.626 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:07.627 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:43:07.727 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:43:07.758 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:07.778 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:43:07.780 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:07.781 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:43:07.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.755+0000 7fe76a919700 1 -- 172.21.15.47:0/394203854 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe76406b110 msgr2=0x7fe76406b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:07.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.755+0000 7fe76a919700 1 --2- 172.21.15.47:0/394203854 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe76406b110 0x7fe76406b4f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fe754004870 tx=0x7fe754009cb0).stop 2024-02-23T00:43:07.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.756+0000 7fe76a919700 1 -- 172.21.15.47:0/394203854 shutdown_connections 2024-02-23T00:43:07.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.756+0000 7fe76a919700 1 --2- 172.21.15.47:0/394203854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7641044a0 0x7fe764106890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:07.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.756+0000 7fe76a919700 1 --2- 172.21.15.47:0/394203854 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe76406b110 0x7fe76406b4f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:07.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.756+0000 7fe76a919700 1 --2- 172.21.15.47:0/394203854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe76406b9e0 0x7fe764103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:07.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.756+0000 7fe76a919700 1 -- 172.21.15.47:0/394203854 >> 172.21.15.47:0/394203854 conn(0x7fe7640f3800 msgr2=0x7fe7640f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:43:07.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.756+0000 7fe76a919700 1 -- 172.21.15.47:0/394203854 shutdown_connections 2024-02-23T00:43:07.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.756+0000 7fe76a919700 1 -- 172.21.15.47:0/394203854 wait complete. 2024-02-23T00:43:07.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.756+0000 7fe76a919700 1 librados: starting msgr at 2024-02-23T00:43:07.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.756+0000 7fe76a919700 1 librados: starting objecter 2024-02-23T00:43:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.756+0000 7fe76a919700 1 Processor -- start 2024-02-23T00:43:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.757+0000 7fe76a919700 1 -- start start 2024-02-23T00:43:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.757+0000 7fe76a919700 1 librados: setting wanted keys 2024-02-23T00:43:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.757+0000 7fe76a919700 1 librados: calling monclient init 2024-02-23T00:43:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.757+0000 7fe76a919700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe76406b110 0x7fe76410ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.757+0000 7fe76a919700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe76406b9e0 0x7fe764110430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.757+0000 7fe76a919700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7641044a0 0x7fe76410a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.757+0000 7fe76a919700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76410a830 con 0x7fe7641044a0 2024-02-23T00:43:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.757+0000 7fe76a919700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76410a9a0 con 0x7fe76406b9e0 2024-02-23T00:43:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.757+0000 7fe76a919700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe76410ab60 con 0x7fe76406b110 2024-02-23T00:43:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.757+0000 7fe768eb6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7641044a0 0x7fe76410a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.757+0000 7fe763fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe76406b110 0x7fe76410ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.757+0000 7fe768eb6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7641044a0 0x7fe76410a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36398/0 (socket says 172.21.15.47:36398) 2024-02-23T00:43:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.757+0000 7fe768eb6700 1 -- 172.21.15.47:0/669469315 learned_addr learned my addr 172.21.15.47:0/669469315 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:43:07.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.757+0000 7fe7637fe700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe76406b9e0 0x7fe764110430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:07.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.758+0000 7fe768eb6700 1 -- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe76406b110 msgr2=0x7fe76410ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:07.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.758+0000 7fe768eb6700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe76406b110 0x7fe76410ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:07.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.758+0000 7fe768eb6700 1 -- 172.21.15.47:0/669469315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe76406b9e0 msgr2=0x7fe764110430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:07.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.758+0000 7fe768eb6700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe76406b9e0 0x7fe764110430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:07.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.758+0000 7fe768eb6700 1 -- 172.21.15.47:0/669469315 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe754005040 con 0x7fe7641044a0 2024-02-23T00:43:07.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.758+0000 7fe763fff700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe76406b110 0x7fe76410ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:43:07.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.758+0000 7fe7637fe700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe76406b9e0 0x7fe764110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:43:07.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.758+0000 7fe768eb6700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7641044a0 0x7fe76410a340 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7fe75c00eb70 tx=0x7fe75c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:07.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.758+0000 7fe7617fa700 10 client.?.objecter ms_handle_connect 0x7fe7641044a0 2024-02-23T00:43:07.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.758+0000 7fe7617fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:43:07.882 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:43:07.886 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:43:07.909 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:07.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.925+0000 7fe7617fa700 1 -- 172.21.15.47:0/669469315 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c009910 con 0x7fe7641044a0 2024-02-23T00:43:07.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.925+0000 7fe7617fa700 1 -- 172.21.15.47:0/669469315 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fe75c00f040 con 0x7fe7641044a0 2024-02-23T00:43:07.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.925+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe76410ade0 con 0x7fe7641044a0 2024-02-23T00:43:07.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.925+0000 7fe7617fa700 1 -- 172.21.15.47:0/669469315 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c007d50 con 0x7fe7641044a0 2024-02-23T00:43:07.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.925+0000 7fe76a919700 10 client.15225.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:43:07.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.925+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe764116d70 con 0x7fe7641044a0 2024-02-23T00:43:07.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.925+0000 7fe76a919700 1 librados: init done 2024-02-23T00:43:07.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.926+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fe750004f30 con 0x7fe7641044a0 2024-02-23T00:43:07.929 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:43:07.931 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:07.932 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:43:07.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.930+0000 7fe7617fa700 1 -- 172.21.15.47:0/669469315 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fe75c0092e0 con 0x7fe7641044a0 2024-02-23T00:43:07.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.930+0000 7fe7617fa700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe74c062cc0 0x7fe74c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:07.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.930+0000 7fe763fff700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe74c062cc0 0x7fe74c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:07.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.931+0000 7fe763fff700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe74c062cc0 0x7fe74c065180 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fe754004ee0 tx=0x7fe754015040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:07.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.931+0000 7fe7617fa700 10 client.15225.objecter ms_handle_connect 0x7fe74c062cc0 2024-02-23T00:43:07.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.932+0000 7fe7617fa700 1 -- 172.21.15.47:0/669469315 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(117..117 src has 48..117) v4 ==== 16147+0+0 (secure 0 0 0) 0x7fe75c050ba0 con 0x7fe7641044a0 2024-02-23T00:43:07.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.932+0000 7fe7617fa700 10 client.15225.objecter ms_dispatch 0x7fe764001550 osd_map(117..117 src has 48..117) v4 2024-02-23T00:43:07.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.932+0000 7fe7617fa700 3 client.15225.objecter handle_osd_map got epochs [117,117] > 0 2024-02-23T00:43:07.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.932+0000 7fe7617fa700 3 client.15225.objecter handle_osd_map decoding full epoch 117 2024-02-23T00:43:07.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.932+0000 7fe7617fa700 20 client.15225.objecter dump_active .. 0 homeless 2024-02-23T00:43:07.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.932+0000 7fe7617fa700 1 -- 172.21.15.47:0/669469315 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v117) v1 ==== 54+0+16 (secure 0 0 0) 0x7fe75c04cf40 con 0x7fe7641044a0 2024-02-23T00:43:07.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.933+0000 7fe76a919700 10 client.15225.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:07.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.933+0000 7fe76a919700 15 client.15225.objecter _session_command_op_assign -1 1 2024-02-23T00:43:07.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.933+0000 7fe76a919700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fe750004560 0x7fe750004980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:07.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.933+0000 7fe76a919700 20 client.15225.objecter _get_session s=0x7fe7500051d0 osd=0 3 2024-02-23T00:43:07.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.933+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe7500051d0 osd=0 3 2024-02-23T00:43:07.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.933+0000 7fe76a919700 20 client.15225.objecter _get_session s=0x7fe7500051d0 osd=0 3 2024-02-23T00:43:07.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.933+0000 7fe76a919700 15 client.15225.objecter _session_command_op_remove -1 1 2024-02-23T00:43:07.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.933+0000 7fe76a919700 20 client.15225.objecter get_session s=0x7fe7500051d0 osd=0 3 2024-02-23T00:43:07.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.933+0000 7fe76a919700 15 client.15225.objecter _session_command_op_assign 0 1 2024-02-23T00:43:07.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.933+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe7500051d0 osd=0 4 2024-02-23T00:43:07.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.933+0000 7fe76a919700 10 client.15225.objecter _send_command 1 2024-02-23T00:43:07.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.933+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe750008960 con 0x7fe750004560 2024-02-23T00:43:07.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.933+0000 7fe7637fe700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fe750004560 0x7fe750004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:07.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.933+0000 7fe7617fa700 10 client.15225.objecter ms_handle_connect 0x7fe750004560 2024-02-23T00:43:07.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.933+0000 7fe7637fe700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fe750004560 0x7fe750004980 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:07.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.934+0000 7fe7617fa700 1 -- 172.21.15.47:0/669469315 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe750008960 con 0x7fe750004560 2024-02-23T00:43:07.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.934+0000 7fe7617fa700 10 client.15225.objecter ms_dispatch 0x7fe764001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:43:07.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.934+0000 7fe7617fa700 10 client.15225.objecter _finish_command 1 = system:0 2024-02-23T00:43:07.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.934+0000 7fe7617fa700 20 client.15225.objecter put_session s=0x7fe7500051d0 osd=0 4 2024-02-23T00:43:07.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.934+0000 7fe7617fa700 15 client.15225.objecter _session_command_op_remove 0 1 2024-02-23T00:43:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.975+0000 7fe76a919700 10 client.15225.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:43:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.975+0000 7fe76a919700 15 client.15225.objecter _session_command_op_assign -1 2 2024-02-23T00:43:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.975+0000 7fe76a919700 20 client.15225.objecter _get_session s=0x7fe7500051d0 osd=0 3 2024-02-23T00:43:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.975+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe7500051d0 osd=0 3 2024-02-23T00:43:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.975+0000 7fe76a919700 20 client.15225.objecter _get_session s=0x7fe7500051d0 osd=0 3 2024-02-23T00:43:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.975+0000 7fe76a919700 15 client.15225.objecter _session_command_op_remove -1 2 2024-02-23T00:43:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.975+0000 7fe76a919700 20 client.15225.objecter get_session s=0x7fe7500051d0 osd=0 3 2024-02-23T00:43:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.975+0000 7fe76a919700 15 client.15225.objecter _session_command_op_assign 0 2 2024-02-23T00:43:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.975+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe7500051d0 osd=0 4 2024-02-23T00:43:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.975+0000 7fe76a919700 10 client.15225.objecter _send_command 2 2024-02-23T00:43:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.975+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fe750003fd0 con 0x7fe750004560 2024-02-23T00:43:08.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.999+0000 7fe7617fa700 1 -- 172.21.15.47:0/669469315 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fe750003fd0 con 0x7fe750004560 2024-02-23T00:43:08.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.999+0000 7fe7617fa700 10 client.15225.objecter ms_dispatch 0x7fe764001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:43:08.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.999+0000 7fe7617fa700 10 client.15225.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:43:08.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.999+0000 7fe7617fa700 20 client.15225.objecter put_session s=0x7fe7500051d0 osd=0 4 2024-02-23T00:43:08.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:07.999+0000 7fe7617fa700 15 client.15225.objecter _session_command_op_remove 0 2 2024-02-23T00:43:08.003 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:43:08.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.001+0000 7fe76a919700 10 client.15225.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:08.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.001+0000 7fe76a919700 15 client.15225.objecter _session_command_op_assign -1 3 2024-02-23T00:43:08.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.001+0000 7fe76a919700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fe750008510 0x7fe750002780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:08.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.001+0000 7fe76a919700 20 client.15225.objecter _get_session s=0x7fe7500043e0 osd=1 3 2024-02-23T00:43:08.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.001+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe7500043e0 osd=1 3 2024-02-23T00:43:08.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.001+0000 7fe76a919700 20 client.15225.objecter _get_session s=0x7fe7500043e0 osd=1 3 2024-02-23T00:43:08.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.001+0000 7fe76a919700 15 client.15225.objecter _session_command_op_remove -1 3 2024-02-23T00:43:08.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.001+0000 7fe76a919700 20 client.15225.objecter get_session s=0x7fe7500043e0 osd=1 3 2024-02-23T00:43:08.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.001+0000 7fe76a919700 15 client.15225.objecter _session_command_op_assign 1 3 2024-02-23T00:43:08.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.001+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe7500043e0 osd=1 4 2024-02-23T00:43:08.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.001+0000 7fe76a919700 10 client.15225.objecter _send_command 3 2024-02-23T00:43:08.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.001+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe750002ed0 con 0x7fe750008510 2024-02-23T00:43:08.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.001+0000 7fe763fff700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fe750008510 0x7fe750002780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:08.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.002+0000 7fe7617fa700 10 client.15225.objecter ms_handle_connect 0x7fe750008510 2024-02-23T00:43:08.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.002+0000 7fe763fff700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fe750008510 0x7fe750002780 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:08.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.005+0000 7fe7617fa700 1 -- 172.21.15.47:0/669469315 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe750002ed0 con 0x7fe750008510 2024-02-23T00:43:08.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.005+0000 7fe7617fa700 10 client.15225.objecter ms_dispatch 0x7fe764001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:43:08.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.005+0000 7fe7617fa700 10 client.15225.objecter _finish_command 3 = system:0 2024-02-23T00:43:08.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.005+0000 7fe7617fa700 20 client.15225.objecter put_session s=0x7fe7500043e0 osd=1 4 2024-02-23T00:43:08.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.005+0000 7fe7617fa700 15 client.15225.objecter _session_command_op_remove 1 3 2024-02-23T00:43:08.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.029+0000 7fe76a919700 10 client.15225.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:43:08.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.029+0000 7fe76a919700 15 client.15225.objecter _session_command_op_assign -1 4 2024-02-23T00:43:08.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.029+0000 7fe76a919700 20 client.15225.objecter _get_session s=0x7fe7500043e0 osd=1 3 2024-02-23T00:43:08.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.029+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe7500043e0 osd=1 3 2024-02-23T00:43:08.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.029+0000 7fe76a919700 20 client.15225.objecter _get_session s=0x7fe7500043e0 osd=1 3 2024-02-23T00:43:08.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.029+0000 7fe76a919700 15 client.15225.objecter _session_command_op_remove -1 4 2024-02-23T00:43:08.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.029+0000 7fe76a919700 20 client.15225.objecter get_session s=0x7fe7500043e0 osd=1 3 2024-02-23T00:43:08.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.029+0000 7fe76a919700 15 client.15225.objecter _session_command_op_assign 1 4 2024-02-23T00:43:08.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.029+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe7500043e0 osd=1 4 2024-02-23T00:43:08.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.029+0000 7fe76a919700 10 client.15225.objecter _send_command 4 2024-02-23T00:43:08.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.029+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fe7500038a0 con 0x7fe750008510 2024-02-23T00:43:08.032 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:43:08.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.031+0000 7fe7617fa700 1 -- 172.21.15.47:0/669469315 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fe7500038a0 con 0x7fe750008510 2024-02-23T00:43:08.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.031+0000 7fe7617fa700 10 client.15225.objecter ms_dispatch 0x7fe764001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:43:08.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.031+0000 7fe7617fa700 10 client.15225.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:43:08.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.031+0000 7fe7617fa700 20 client.15225.objecter put_session s=0x7fe7500043e0 osd=1 4 2024-02-23T00:43:08.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.031+0000 7fe7617fa700 15 client.15225.objecter _session_command_op_remove 1 4 2024-02-23T00:43:08.034 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:43:08.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.031+0000 7fe76a919700 10 client.15225.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:08.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.031+0000 7fe76a919700 15 client.15225.objecter _session_command_op_assign -1 5 2024-02-23T00:43:08.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.031+0000 7fe76a919700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fe750007b90 0x7fe75000d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:08.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.031+0000 7fe76a919700 20 client.15225.objecter _get_session s=0x7fe750007a20 osd=2 3 2024-02-23T00:43:08.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.031+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe750007a20 osd=2 3 2024-02-23T00:43:08.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.031+0000 7fe76a919700 20 client.15225.objecter _get_session s=0x7fe750007a20 osd=2 3 2024-02-23T00:43:08.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.031+0000 7fe76a919700 15 client.15225.objecter _session_command_op_remove -1 5 2024-02-23T00:43:08.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.031+0000 7fe76a919700 20 client.15225.objecter get_session s=0x7fe750007a20 osd=2 3 2024-02-23T00:43:08.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.031+0000 7fe76a919700 15 client.15225.objecter _session_command_op_assign 2 5 2024-02-23T00:43:08.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.031+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe750007a20 osd=2 4 2024-02-23T00:43:08.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.031+0000 7fe76a919700 10 client.15225.objecter _send_command 5 2024-02-23T00:43:08.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.031+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe75000feb0 con 0x7fe750007b90 2024-02-23T00:43:08.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.031+0000 7fe7637fe700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fe750007b90 0x7fe75000d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:08.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.031+0000 7fe7617fa700 10 client.15225.objecter ms_handle_connect 0x7fe750007b90 2024-02-23T00:43:08.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.031+0000 7fe7637fe700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fe750007b90 0x7fe75000d000 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:08.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.032+0000 7fe7617fa700 1 -- 172.21.15.47:0/669469315 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe75000feb0 con 0x7fe750007b90 2024-02-23T00:43:08.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.032+0000 7fe7617fa700 10 client.15225.objecter ms_dispatch 0x7fe764001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:43:08.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.032+0000 7fe7617fa700 10 client.15225.objecter _finish_command 5 = system:0 2024-02-23T00:43:08.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.032+0000 7fe7617fa700 20 client.15225.objecter put_session s=0x7fe750007a20 osd=2 4 2024-02-23T00:43:08.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.032+0000 7fe7617fa700 15 client.15225.objecter _session_command_op_remove 2 5 2024-02-23T00:43:08.058 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:05.177410+0000 mgr.y (mgr.14517) 161 : cluster [DBG] pgmap v147: 652 pgs: 200 unknown, 32 creating+peering, 420 active+clean; 144 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 11 MiB/s rd, 34 MiB/s wr, 503 op/s; 84 B/s, 0 objects/s recovering 2024-02-23T00:43:08.058 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:05.470721+0000 mon.b (mon.1) 85 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:08.058 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:05.857795+0000 osd.1 (osd.1) 83 : cluster [DBG] 20.5 deep-scrub starts 2024-02-23T00:43:08.058 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:05.860326+0000 osd.1 (osd.1) 84 : cluster [DBG] 20.5 deep-scrub ok 2024-02-23T00:43:08.058 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:06.010312+0000 osd.7 (osd.7) 79 : cluster [DBG] 20.2 deep-scrub starts 2024-02-23T00:43:08.058 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:06.013821+0000 osd.7 (osd.7) 80 : cluster [DBG] 20.2 deep-scrub ok 2024-02-23T00:43:08.058 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:06.471842+0000 mon.b (mon.1) 86 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:08.058 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:06.550257+0000 mon.a (mon.0) 911 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi047-82386-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:08.058 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:06.550385+0000 mon.a (mon.0) 912 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1"}]': finished 2024-02-23T00:43:08.058 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:06.550539+0000 mon.a (mon.0) 913 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi047-82428-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:08.058 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:06.550791+0000 mon.a (mon.0) 914 : audit [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi047-82438-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:08.058 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:06.550961+0000 mon.a (mon.0) 915 : cluster [DBG] osdmap e116: 8 total, 8 up, 8 in 2024-02-23T00:43:08.058 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:06.551100+0000 mon.c (mon.2) 40 : audit [INF] from='client.? 172.21.15.47:0/1913712492' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-4", "overlaypool": "test-rados-api-smithi047-82465-4-cache"}]: dispatch 2024-02-23T00:43:08.059 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:06.551162+0000 mon.b (mon.1) 87 : audit [INF] from='client.? 172.21.15.47:0/2670125835' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:43:08.059 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:06.551531+0000 mon.c (mon.2) 41 : audit [INF] from='client.? 172.21.15.47:0/1119363424' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi047-82428-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:08.059 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:06.552342+0000 mon.b (mon.1) 88 : audit [INF] from='client.? 172.21.15.47:0/1505701467' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:08.059 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:06.552414+0000 mon.c (mon.2) 42 : audit [INF] from='client.? 172.21.15.47:0/830645324' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi047-82378-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:08.059 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:06.552503+0000 mon.b (mon.1) 89 : audit [INF] from='client.? 172.21.15.47:0/854895257' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi047-82386-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-23T00:43:08.059 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:06.555720+0000 mon.a (mon.0) 916 : audit [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi047-82438-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi047-82438-10"}]: dispatch 2024-02-23T00:43:08.059 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:06.559950+0000 mon.a (mon.0) 917 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-4", "overlaypool": "test-rados-api-smithi047-82465-4-cache"}]: dispatch 2024-02-23T00:43:08.059 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:06.560274+0000 mon.a (mon.0) 918 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi047-82428-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:08.059 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:06.564384+0000 mon.a (mon.0) 919 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi047-82378-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:08.059 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:06.569567+0000 mon.a (mon.0) 920 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-23T00:43:08.059 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:06.570396+0000 mon.a (mon.0) 921 : audit [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi047-82504-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi047-82504-6"}]': finished 2024-02-23T00:43:08.059 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:06.570482+0000 mon.a (mon.0) 922 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi047-82394-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi047-82394-15"}]': finished 2024-02-23T00:43:08.060 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:06.570566+0000 mon.a (mon.0) 923 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi047-82512-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi047-82512-7"}]': finished 2024-02-23T00:43:08.060 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:06.890552+0000 osd.1 (osd.1) 85 : cluster [DBG] 20.6 deep-scrub starts 2024-02-23T00:43:08.060 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:06.922774+0000 osd.1 (osd.1) 86 : cluster [DBG] 20.6 deep-scrub ok 2024-02-23T00:43:08.060 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:07.028411+0000 mon.a (mon.0) 924 : cluster [DBG] osdmap e117: 8 total, 8 up, 8 in 2024-02-23T00:43:08.060 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:07.180147+0000 mgr.y (mgr.14517) 162 : cluster [DBG] pgmap v150: 572 pgs: 56 creating+peering, 2 clean+premerge+peered, 514 active+clean; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:43:08.060 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:07.472931+0000 mon.b (mon.1) 90 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:08.060 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:07.476344+0000 mon.a (mon.0) 925 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:43:08.060 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:07.476622+0000 mon.a (mon.0) 926 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:08.060 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:07.476856+0000 mon.a (mon.0) 927 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi047-82386-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-23T00:43:08.061 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:08.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.056+0000 7fe76a919700 10 client.15225.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:43:08.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.056+0000 7fe76a919700 15 client.15225.objecter _session_command_op_assign -1 6 2024-02-23T00:43:08.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.056+0000 7fe76a919700 20 client.15225.objecter _get_session s=0x7fe750007a20 osd=2 3 2024-02-23T00:43:08.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.056+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe750007a20 osd=2 3 2024-02-23T00:43:08.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.056+0000 7fe76a919700 20 client.15225.objecter _get_session s=0x7fe750007a20 osd=2 3 2024-02-23T00:43:08.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.056+0000 7fe76a919700 15 client.15225.objecter _session_command_op_remove -1 6 2024-02-23T00:43:08.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.056+0000 7fe76a919700 20 client.15225.objecter get_session s=0x7fe750007a20 osd=2 3 2024-02-23T00:43:08.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.056+0000 7fe76a919700 15 client.15225.objecter _session_command_op_assign 2 6 2024-02-23T00:43:08.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.056+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe750007a20 osd=2 4 2024-02-23T00:43:08.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.056+0000 7fe76a919700 10 client.15225.objecter _send_command 6 2024-02-23T00:43:08.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.056+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fe750003280 con 0x7fe750007b90 2024-02-23T00:43:08.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.058+0000 7fe7617fa700 1 -- 172.21.15.47:0/669469315 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fe750003280 con 0x7fe750007b90 2024-02-23T00:43:08.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.058+0000 7fe7617fa700 10 client.15225.objecter ms_dispatch 0x7fe764001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:43:08.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.058+0000 7fe7617fa700 10 client.15225.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:43:08.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.058+0000 7fe7617fa700 20 client.15225.objecter put_session s=0x7fe750007a20 osd=2 4 2024-02-23T00:43:08.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.058+0000 7fe7617fa700 15 client.15225.objecter _session_command_op_remove 2 6 2024-02-23T00:43:08.062 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:43:08.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.058+0000 7fe76a919700 10 client.15225.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:08.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.058+0000 7fe76a919700 15 client.15225.objecter _session_command_op_assign -1 7 2024-02-23T00:43:08.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.058+0000 7fe76a919700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fe750002080 0x7fe75000e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:08.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.058+0000 7fe76a919700 20 client.15225.objecter _get_session s=0x7fe750003630 osd=3 3 2024-02-23T00:43:08.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.058+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe750003630 osd=3 3 2024-02-23T00:43:08.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.058+0000 7fe76a919700 20 client.15225.objecter _get_session s=0x7fe750003630 osd=3 3 2024-02-23T00:43:08.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.058+0000 7fe76a919700 15 client.15225.objecter _session_command_op_remove -1 7 2024-02-23T00:43:08.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.058+0000 7fe76a919700 20 client.15225.objecter get_session s=0x7fe750003630 osd=3 3 2024-02-23T00:43:08.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.058+0000 7fe76a919700 15 client.15225.objecter _session_command_op_assign 3 7 2024-02-23T00:43:08.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.058+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe750003630 osd=3 4 2024-02-23T00:43:08.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.058+0000 7fe76a919700 10 client.15225.objecter _send_command 7 2024-02-23T00:43:08.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.058+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe75000ee40 con 0x7fe750002080 2024-02-23T00:43:08.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.059+0000 7fe768eb6700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fe750002080 0x7fe75000e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:08.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.059+0000 7fe7617fa700 10 client.15225.objecter ms_handle_connect 0x7fe750002080 2024-02-23T00:43:08.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.059+0000 7fe768eb6700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fe750002080 0x7fe75000e760 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:08.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.060+0000 7fe7617fa700 1 -- 172.21.15.47:0/669469315 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe75000ee40 con 0x7fe750002080 2024-02-23T00:43:08.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.060+0000 7fe7617fa700 10 client.15225.objecter ms_dispatch 0x7fe764001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:43:08.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.060+0000 7fe7617fa700 10 client.15225.objecter _finish_command 7 = system:0 2024-02-23T00:43:08.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.060+0000 7fe7617fa700 20 client.15225.objecter put_session s=0x7fe750003630 osd=3 4 2024-02-23T00:43:08.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.060+0000 7fe7617fa700 15 client.15225.objecter _session_command_op_remove 3 7 2024-02-23T00:43:08.078 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:43:08.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.077+0000 7fe76a919700 10 client.15225.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:43:08.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.077+0000 7fe76a919700 15 client.15225.objecter _session_command_op_assign -1 8 2024-02-23T00:43:08.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.077+0000 7fe76a919700 20 client.15225.objecter _get_session s=0x7fe750003630 osd=3 3 2024-02-23T00:43:08.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.077+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe750003630 osd=3 3 2024-02-23T00:43:08.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.077+0000 7fe76a919700 20 client.15225.objecter _get_session s=0x7fe750003630 osd=3 3 2024-02-23T00:43:08.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.077+0000 7fe76a919700 15 client.15225.objecter _session_command_op_remove -1 8 2024-02-23T00:43:08.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.077+0000 7fe76a919700 20 client.15225.objecter get_session s=0x7fe750003630 osd=3 3 2024-02-23T00:43:08.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.077+0000 7fe76a919700 15 client.15225.objecter _session_command_op_assign 3 8 2024-02-23T00:43:08.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.077+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe750003630 osd=3 4 2024-02-23T00:43:08.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.077+0000 7fe76a919700 10 client.15225.objecter _send_command 8 2024-02-23T00:43:08.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.077+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fe75000f810 con 0x7fe750002080 2024-02-23T00:43:08.081 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:08.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.078+0000 7fe7617fa700 1 -- 172.21.15.47:0/669469315 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fe75000f810 con 0x7fe750002080 2024-02-23T00:43:08.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.078+0000 7fe7617fa700 10 client.15225.objecter ms_dispatch 0x7fe764001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:43:08.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.078+0000 7fe7617fa700 10 client.15225.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:43:08.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.078+0000 7fe7617fa700 20 client.15225.objecter put_session s=0x7fe750003630 osd=3 4 2024-02-23T00:43:08.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.078+0000 7fe7617fa700 15 client.15225.objecter _session_command_op_remove 3 8 2024-02-23T00:43:08.082 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:43:08.082 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:43:08.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.078+0000 7fe76a919700 10 client.15225.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:08.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.078+0000 7fe76a919700 15 client.15225.objecter _session_command_op_assign -1 9 2024-02-23T00:43:08.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.079+0000 7fe76a919700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fe7500120c0 0x7fe7500144b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:08.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.079+0000 7fe76a919700 20 client.15225.objecter _get_session s=0x7fe75000f040 osd=4 3 2024-02-23T00:43:08.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.079+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe75000f040 osd=4 3 2024-02-23T00:43:08.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.079+0000 7fe76a919700 20 client.15225.objecter _get_session s=0x7fe75000f040 osd=4 3 2024-02-23T00:43:08.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.079+0000 7fe76a919700 15 client.15225.objecter _session_command_op_remove -1 9 2024-02-23T00:43:08.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.079+0000 7fe76a919700 20 client.15225.objecter get_session s=0x7fe75000f040 osd=4 3 2024-02-23T00:43:08.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.079+0000 7fe76a919700 15 client.15225.objecter _session_command_op_assign 4 9 2024-02-23T00:43:08.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.079+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe75000f040 osd=4 4 2024-02-23T00:43:08.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.079+0000 7fe76a919700 10 client.15225.objecter _send_command 9 2024-02-23T00:43:08.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.079+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe750017100 con 0x7fe7500120c0 2024-02-23T00:43:08.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.079+0000 7fe763fff700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fe7500120c0 0x7fe7500144b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:08.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.079+0000 7fe7617fa700 10 client.15225.objecter ms_handle_connect 0x7fe7500120c0 2024-02-23T00:43:08.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.079+0000 7fe763fff700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fe7500120c0 0x7fe7500144b0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:08.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.080+0000 7fe7617fa700 1 -- 172.21.15.47:0/669469315 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe750017100 con 0x7fe7500120c0 2024-02-23T00:43:08.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.080+0000 7fe7617fa700 10 client.15225.objecter ms_dispatch 0x7fe764001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:43:08.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.080+0000 7fe7617fa700 10 client.15225.objecter _finish_command 9 = system:0 2024-02-23T00:43:08.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.080+0000 7fe7617fa700 20 client.15225.objecter put_session s=0x7fe75000f040 osd=4 4 2024-02-23T00:43:08.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.080+0000 7fe7617fa700 15 client.15225.objecter _session_command_op_remove 4 9 2024-02-23T00:43:08.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.095+0000 7fe76a919700 10 client.15225.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:43:08.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.095+0000 7fe76a919700 15 client.15225.objecter _session_command_op_assign -1 10 2024-02-23T00:43:08.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.095+0000 7fe76a919700 20 client.15225.objecter _get_session s=0x7fe75000f040 osd=4 3 2024-02-23T00:43:08.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.095+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe75000f040 osd=4 3 2024-02-23T00:43:08.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.095+0000 7fe76a919700 20 client.15225.objecter _get_session s=0x7fe75000f040 osd=4 3 2024-02-23T00:43:08.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.095+0000 7fe76a919700 15 client.15225.objecter _session_command_op_remove -1 10 2024-02-23T00:43:08.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.095+0000 7fe76a919700 20 client.15225.objecter get_session s=0x7fe75000f040 osd=4 3 2024-02-23T00:43:08.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.095+0000 7fe76a919700 15 client.15225.objecter _session_command_op_assign 4 10 2024-02-23T00:43:08.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.095+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe75000f040 osd=4 4 2024-02-23T00:43:08.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.095+0000 7fe76a919700 10 client.15225.objecter _send_command 10 2024-02-23T00:43:08.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.095+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fe7500149a0 con 0x7fe7500120c0 2024-02-23T00:43:08.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.096+0000 7fe7617fa700 1 -- 172.21.15.47:0/669469315 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fe7500149a0 con 0x7fe7500120c0 2024-02-23T00:43:08.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.096+0000 7fe7617fa700 10 client.15225.objecter ms_dispatch 0x7fe764001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:43:08.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.096+0000 7fe7617fa700 10 client.15225.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:43:08.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.096+0000 7fe7617fa700 20 client.15225.objecter put_session s=0x7fe75000f040 osd=4 4 2024-02-23T00:43:08.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.096+0000 7fe7617fa700 15 client.15225.objecter _session_command_op_remove 4 10 2024-02-23T00:43:08.099 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:43:08.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.097+0000 7fe76a919700 10 client.15225.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:08.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.097+0000 7fe76a919700 15 client.15225.objecter _session_command_op_assign -1 11 2024-02-23T00:43:08.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.097+0000 7fe76a919700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fe750015020 0x7fe750017450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:08.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.097+0000 7fe76a919700 20 client.15225.objecter _get_session s=0x7fe7500149a0 osd=5 3 2024-02-23T00:43:08.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.097+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe7500149a0 osd=5 3 2024-02-23T00:43:08.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.097+0000 7fe76a919700 20 client.15225.objecter _get_session s=0x7fe7500149a0 osd=5 3 2024-02-23T00:43:08.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.097+0000 7fe76a919700 15 client.15225.objecter _session_command_op_remove -1 11 2024-02-23T00:43:08.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.097+0000 7fe76a919700 20 client.15225.objecter get_session s=0x7fe7500149a0 osd=5 3 2024-02-23T00:43:08.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.097+0000 7fe76a919700 15 client.15225.objecter _session_command_op_assign 5 11 2024-02-23T00:43:08.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.097+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe7500149a0 osd=5 4 2024-02-23T00:43:08.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.097+0000 7fe76a919700 10 client.15225.objecter _send_command 11 2024-02-23T00:43:08.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.097+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe75001a290 con 0x7fe750015020 2024-02-23T00:43:08.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.097+0000 7fe7637fe700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fe750015020 0x7fe750017450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:08.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.097+0000 7fe7617fa700 10 client.15225.objecter ms_handle_connect 0x7fe750015020 2024-02-23T00:43:08.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.097+0000 7fe7637fe700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fe750015020 0x7fe750017450 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:08.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.098+0000 7fe7617fa700 1 -- 172.21.15.47:0/669469315 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe75001a290 con 0x7fe750015020 2024-02-23T00:43:08.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.098+0000 7fe7617fa700 10 client.15225.objecter ms_dispatch 0x7fe764001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:43:08.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.098+0000 7fe7617fa700 10 client.15225.objecter _finish_command 11 = system:0 2024-02-23T00:43:08.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.098+0000 7fe7617fa700 20 client.15225.objecter put_session s=0x7fe7500149a0 osd=5 4 2024-02-23T00:43:08.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.098+0000 7fe7617fa700 15 client.15225.objecter _session_command_op_remove 5 11 2024-02-23T00:43:08.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.113+0000 7fe76a919700 10 client.15225.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:43:08.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.113+0000 7fe76a919700 15 client.15225.objecter _session_command_op_assign -1 12 2024-02-23T00:43:08.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.113+0000 7fe76a919700 20 client.15225.objecter _get_session s=0x7fe7500149a0 osd=5 3 2024-02-23T00:43:08.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.113+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe7500149a0 osd=5 3 2024-02-23T00:43:08.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.113+0000 7fe76a919700 20 client.15225.objecter _get_session s=0x7fe7500149a0 osd=5 3 2024-02-23T00:43:08.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.113+0000 7fe76a919700 15 client.15225.objecter _session_command_op_remove -1 12 2024-02-23T00:43:08.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.113+0000 7fe76a919700 20 client.15225.objecter get_session s=0x7fe7500149a0 osd=5 3 2024-02-23T00:43:08.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.113+0000 7fe76a919700 15 client.15225.objecter _session_command_op_assign 5 12 2024-02-23T00:43:08.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.113+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe7500149a0 osd=5 4 2024-02-23T00:43:08.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.113+0000 7fe76a919700 10 client.15225.objecter _send_command 12 2024-02-23T00:43:08.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.113+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fe75001a290 con 0x7fe750015020 2024-02-23T00:43:08.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.114+0000 7fe7617fa700 1 -- 172.21.15.47:0/669469315 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fe75001a290 con 0x7fe750015020 2024-02-23T00:43:08.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.114+0000 7fe7617fa700 10 client.15225.objecter ms_dispatch 0x7fe764001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:43:08.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.114+0000 7fe7617fa700 10 client.15225.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:43:08.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.114+0000 7fe7617fa700 20 client.15225.objecter put_session s=0x7fe7500149a0 osd=5 4 2024-02-23T00:43:08.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.114+0000 7fe7617fa700 15 client.15225.objecter _session_command_op_remove 5 12 2024-02-23T00:43:08.117 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:43:08.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.115+0000 7fe76a919700 10 client.15225.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:08.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.115+0000 7fe76a919700 15 client.15225.objecter _session_command_op_assign -1 13 2024-02-23T00:43:08.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.115+0000 7fe76a919700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fe7500195c0 0x7fe750019a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:08.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.115+0000 7fe76a919700 20 client.15225.objecter _get_session s=0x7fe75001a290 osd=6 3 2024-02-23T00:43:08.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.115+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe75001a290 osd=6 3 2024-02-23T00:43:08.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.115+0000 7fe76a919700 20 client.15225.objecter _get_session s=0x7fe75001a290 osd=6 3 2024-02-23T00:43:08.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.115+0000 7fe76a919700 15 client.15225.objecter _session_command_op_remove -1 13 2024-02-23T00:43:08.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.115+0000 7fe76a919700 20 client.15225.objecter get_session s=0x7fe75001a290 osd=6 3 2024-02-23T00:43:08.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.115+0000 7fe76a919700 15 client.15225.objecter _session_command_op_assign 6 13 2024-02-23T00:43:08.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.115+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe75001a290 osd=6 4 2024-02-23T00:43:08.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.115+0000 7fe76a919700 10 client.15225.objecter _send_command 13 2024-02-23T00:43:08.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.115+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe75001d5c0 con 0x7fe7500195c0 2024-02-23T00:43:08.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.115+0000 7fe768eb6700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fe7500195c0 0x7fe750019a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:08.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.115+0000 7fe7617fa700 10 client.15225.objecter ms_handle_connect 0x7fe7500195c0 2024-02-23T00:43:08.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.115+0000 7fe768eb6700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fe7500195c0 0x7fe750019a40 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:08.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.116+0000 7fe7617fa700 1 -- 172.21.15.47:0/669469315 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe75001d5c0 con 0x7fe7500195c0 2024-02-23T00:43:08.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.116+0000 7fe7617fa700 10 client.15225.objecter ms_dispatch 0x7fe764001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:43:08.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.116+0000 7fe7617fa700 10 client.15225.objecter _finish_command 13 = system:0 2024-02-23T00:43:08.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.116+0000 7fe7617fa700 20 client.15225.objecter put_session s=0x7fe75001a290 osd=6 4 2024-02-23T00:43:08.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.116+0000 7fe7617fa700 15 client.15225.objecter _session_command_op_remove 6 13 2024-02-23T00:43:08.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.130+0000 7fe76a919700 10 client.15225.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:43:08.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.130+0000 7fe76a919700 15 client.15225.objecter _session_command_op_assign -1 14 2024-02-23T00:43:08.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.130+0000 7fe76a919700 20 client.15225.objecter _get_session s=0x7fe75001a290 osd=6 3 2024-02-23T00:43:08.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.130+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe75001a290 osd=6 3 2024-02-23T00:43:08.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.130+0000 7fe76a919700 20 client.15225.objecter _get_session s=0x7fe75001a290 osd=6 3 2024-02-23T00:43:08.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.130+0000 7fe76a919700 15 client.15225.objecter _session_command_op_remove -1 14 2024-02-23T00:43:08.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.130+0000 7fe76a919700 20 client.15225.objecter get_session s=0x7fe75001a290 osd=6 3 2024-02-23T00:43:08.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.130+0000 7fe76a919700 15 client.15225.objecter _session_command_op_assign 6 14 2024-02-23T00:43:08.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.130+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe75001a290 osd=6 4 2024-02-23T00:43:08.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.130+0000 7fe76a919700 10 client.15225.objecter _send_command 14 2024-02-23T00:43:08.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.130+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fe750017a50 con 0x7fe7500195c0 2024-02-23T00:43:08.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.132+0000 7fe7617fa700 1 -- 172.21.15.47:0/669469315 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fe750017a50 con 0x7fe7500195c0 2024-02-23T00:43:08.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.132+0000 7fe7617fa700 10 client.15225.objecter ms_dispatch 0x7fe764001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:43:08.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.132+0000 7fe7617fa700 10 client.15225.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:43:08.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.132+0000 7fe7617fa700 20 client.15225.objecter put_session s=0x7fe75001a290 osd=6 4 2024-02-23T00:43:08.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.132+0000 7fe7617fa700 15 client.15225.objecter _session_command_op_remove 6 14 2024-02-23T00:43:08.135 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:43:08.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.132+0000 7fe76a919700 10 client.15225.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:08.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.132+0000 7fe76a919700 15 client.15225.objecter _session_command_op_assign -1 15 2024-02-23T00:43:08.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.132+0000 7fe76a919700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fe7500180d0 0x7fe750018530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:08.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.132+0000 7fe76a919700 20 client.15225.objecter _get_session s=0x7fe750017a50 osd=7 3 2024-02-23T00:43:08.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.132+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe750017a50 osd=7 3 2024-02-23T00:43:08.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.132+0000 7fe76a919700 20 client.15225.objecter _get_session s=0x7fe750017a50 osd=7 3 2024-02-23T00:43:08.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.132+0000 7fe76a919700 15 client.15225.objecter _session_command_op_remove -1 15 2024-02-23T00:43:08.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.132+0000 7fe76a919700 20 client.15225.objecter get_session s=0x7fe750017a50 osd=7 3 2024-02-23T00:43:08.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.132+0000 7fe76a919700 15 client.15225.objecter _session_command_op_assign 7 15 2024-02-23T00:43:08.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.132+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe750017a50 osd=7 4 2024-02-23T00:43:08.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.132+0000 7fe76a919700 10 client.15225.objecter _send_command 15 2024-02-23T00:43:08.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.132+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe750018bc0 con 0x7fe7500180d0 2024-02-23T00:43:08.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.133+0000 7fe763fff700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fe7500180d0 0x7fe750018530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:08.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.133+0000 7fe7617fa700 10 client.15225.objecter ms_handle_connect 0x7fe7500180d0 2024-02-23T00:43:08.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.133+0000 7fe763fff700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fe7500180d0 0x7fe750018530 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:08.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.134+0000 7fe7617fa700 1 -- 172.21.15.47:0/669469315 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe750018bc0 con 0x7fe7500180d0 2024-02-23T00:43:08.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.134+0000 7fe7617fa700 10 client.15225.objecter ms_dispatch 0x7fe764001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:43:08.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.134+0000 7fe7617fa700 10 client.15225.objecter _finish_command 15 = system:0 2024-02-23T00:43:08.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.134+0000 7fe7617fa700 20 client.15225.objecter put_session s=0x7fe750017a50 osd=7 4 2024-02-23T00:43:08.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.134+0000 7fe7617fa700 15 client.15225.objecter _session_command_op_remove 7 15 2024-02-23T00:43:08.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.148+0000 7fe76a919700 10 client.15225.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:43:08.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.148+0000 7fe76a919700 15 client.15225.objecter _session_command_op_assign -1 16 2024-02-23T00:43:08.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.148+0000 7fe76a919700 20 client.15225.objecter _get_session s=0x7fe750017a50 osd=7 3 2024-02-23T00:43:08.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.148+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe750017a50 osd=7 3 2024-02-23T00:43:08.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.148+0000 7fe76a919700 20 client.15225.objecter _get_session s=0x7fe750017a50 osd=7 3 2024-02-23T00:43:08.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.148+0000 7fe76a919700 15 client.15225.objecter _session_command_op_remove -1 16 2024-02-23T00:43:08.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.148+0000 7fe76a919700 20 client.15225.objecter get_session s=0x7fe750017a50 osd=7 3 2024-02-23T00:43:08.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.148+0000 7fe76a919700 15 client.15225.objecter _session_command_op_assign 7 16 2024-02-23T00:43:08.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.148+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe750017a50 osd=7 4 2024-02-23T00:43:08.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.148+0000 7fe76a919700 10 client.15225.objecter _send_command 16 2024-02-23T00:43:08.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.148+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fe750018bc0 con 0x7fe7500180d0 2024-02-23T00:43:08.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe7617fa700 1 -- 172.21.15.47:0/669469315 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fe750018bc0 con 0x7fe7500180d0 2024-02-23T00:43:08.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe7617fa700 10 client.15225.objecter ms_dispatch 0x7fe764001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:43:08.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe7617fa700 10 client.15225.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:43:08.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe7617fa700 20 client.15225.objecter put_session s=0x7fe750017a50 osd=7 4 2024-02-23T00:43:08.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe7617fa700 15 client.15225.objecter _session_command_op_remove 7 16 2024-02-23T00:43:08.153 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:43:08.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 10 librados: watch_flush enter 2024-02-23T00:43:08.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 10 librados: watch_flush exit 2024-02-23T00:43:08.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 10 client.15225.objecter close_session for osd.0 2024-02-23T00:43:08.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fe750004560 msgr2=0x7fe750004980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:08.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fe750004560 0x7fe750004980 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:08.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe7500051d0 osd=0 1 2024-02-23T00:43:08.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 10 client.15225.objecter close_session for osd.1 2024-02-23T00:43:08.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fe750008510 msgr2=0x7fe750002780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:08.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fe750008510 0x7fe750002780 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:08.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe7500043e0 osd=1 1 2024-02-23T00:43:08.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 10 client.15225.objecter close_session for osd.2 2024-02-23T00:43:08.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fe750007b90 msgr2=0x7fe75000d000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:08.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fe750007b90 0x7fe75000d000 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:08.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe750007a20 osd=2 1 2024-02-23T00:43:08.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 10 client.15225.objecter close_session for osd.3 2024-02-23T00:43:08.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fe750002080 msgr2=0x7fe75000e760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:08.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fe750002080 0x7fe75000e760 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:08.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe750003630 osd=3 1 2024-02-23T00:43:08.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 10 client.15225.objecter close_session for osd.4 2024-02-23T00:43:08.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fe7500120c0 msgr2=0x7fe7500144b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:08.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fe7500120c0 0x7fe7500144b0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:08.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe768eb6700 1 -- 172.21.15.47:0/669469315 reap_dead start 2024-02-23T00:43:08.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe75000f040 osd=4 1 2024-02-23T00:43:08.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 10 client.15225.objecter close_session for osd.5 2024-02-23T00:43:08.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fe750015020 msgr2=0x7fe750017450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:08.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fe750015020 0x7fe750017450 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:08.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe768eb6700 1 -- 172.21.15.47:0/669469315 reap_dead start 2024-02-23T00:43:08.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe7500149a0 osd=5 1 2024-02-23T00:43:08.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 10 client.15225.objecter close_session for osd.6 2024-02-23T00:43:08.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fe7500195c0 msgr2=0x7fe750019a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:08.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fe7500195c0 0x7fe750019a40 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:08.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe75001a290 osd=6 1 2024-02-23T00:43:08.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 10 client.15225.objecter close_session for osd.7 2024-02-23T00:43:08.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fe7500180d0 msgr2=0x7fe750018530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:08.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fe7500180d0 0x7fe750018530 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:08.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 20 client.15225.objecter put_session s=0x7fe750017a50 osd=7 1 2024-02-23T00:43:08.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 20 client.15225.objecter shutdown clearing up homeless session... 2024-02-23T00:43:08.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe768eb6700 1 -- 172.21.15.47:0/669469315 reap_dead start 2024-02-23T00:43:08.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 10 client.15225.objecter successfully canceled tick 2024-02-23T00:43:08.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe74c062cc0 msgr2=0x7fe74c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:08.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.150+0000 7fe76a919700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe74c062cc0 0x7fe74c065180 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fe754004ee0 tx=0x7fe754015040).stop 2024-02-23T00:43:08.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.151+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7641044a0 msgr2=0x7fe76410a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:08.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.151+0000 7fe76a919700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7641044a0 0x7fe76410a340 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7fe75c00eb70 tx=0x7fe75c00b040).stop 2024-02-23T00:43:08.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.151+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 shutdown_connections 2024-02-23T00:43:08.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.151+0000 7fe76a919700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe74c062cc0 0x7fe74c065180 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:08.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.151+0000 7fe76a919700 1 --2- 172.21.15.47:0/669469315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7641044a0 0x7fe76410a340 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:08.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.151+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 >> 172.21.15.47:0/669469315 conn(0x7fe7640f3800 msgr2=0x7fe7640f4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:43:08.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.151+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 shutdown_connections 2024-02-23T00:43:08.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.151+0000 7fe76a919700 1 -- 172.21.15.47:0/669469315 wait complete. 2024-02-23T00:43:08.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:08.151+0000 7fe76a919700 1 librados: shutdown 2024-02-23T00:43:08.157 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:43:08.157 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:43:08.158 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:43:08.158 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:43:08.158 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:43:08.158 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:43:08.158 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:43:08.158 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:43:08.179 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:05.177410+0000 mgr.y (mgr.14517) 161 : cluster [DBG] pgmap v147: 652 pgs: 200 unknown, 32 creating+peering, 420 active+clean; 144 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 11 MiB/s rd, 34 MiB/s wr, 503 op/s; 84 B/s, 0 objects/s recovering 2024-02-23T00:43:08.179 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:05.470721+0000 mon.b (mon.1) 85 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:08.179 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:05.857795+0000 osd.1 (osd.1) 83 : cluster [DBG] 20.5 deep-scrub starts 2024-02-23T00:43:08.179 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:05.860326+0000 osd.1 (osd.1) 84 : cluster [DBG] 20.5 deep-scrub ok 2024-02-23T00:43:08.179 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:06.010312+0000 osd.7 (osd.7) 79 : cluster [DBG] 20.2 deep-scrub starts 2024-02-23T00:43:08.179 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:06.013821+0000 osd.7 (osd.7) 80 : cluster [DBG] 20.2 deep-scrub ok 2024-02-23T00:43:08.179 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:06.471842+0000 mon.b (mon.1) 86 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:08.179 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:06.550257+0000 mon.a (mon.0) 911 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi047-82386-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:08.179 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:06.550385+0000 mon.a (mon.0) 912 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1"}]': finished 2024-02-23T00:43:08.179 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:06.550539+0000 mon.a (mon.0) 913 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi047-82428-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:08.180 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:06.550791+0000 mon.a (mon.0) 914 : audit [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi047-82438-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:08.180 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:06.550961+0000 mon.a (mon.0) 915 : cluster [DBG] osdmap e116: 8 total, 8 up, 8 in 2024-02-23T00:43:08.180 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:06.551100+0000 mon.c (mon.2) 40 : audit [INF] from='client.? 172.21.15.47:0/1913712492' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-4", "overlaypool": "test-rados-api-smithi047-82465-4-cache"}]: dispatch 2024-02-23T00:43:08.180 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:06.551162+0000 mon.b (mon.1) 87 : audit [INF] from='client.? 172.21.15.47:0/2670125835' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:43:08.180 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:06.551531+0000 mon.c (mon.2) 41 : audit [INF] from='client.? 172.21.15.47:0/1119363424' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi047-82428-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:08.180 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:06.552342+0000 mon.b (mon.1) 88 : audit [INF] from='client.? 172.21.15.47:0/1505701467' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:08.180 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:06.552414+0000 mon.c (mon.2) 42 : audit [INF] from='client.? 172.21.15.47:0/830645324' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi047-82378-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:08.180 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:06.552503+0000 mon.b (mon.1) 89 : audit [INF] from='client.? 172.21.15.47:0/854895257' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi047-82386-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-23T00:43:08.180 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:06.555720+0000 mon.a (mon.0) 916 : audit [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi047-82438-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi047-82438-10"}]: dispatch 2024-02-23T00:43:08.180 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:06.559950+0000 mon.a (mon.0) 917 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-4", "overlaypool": "test-rados-api-smithi047-82465-4-cache"}]: dispatch 2024-02-23T00:43:08.180 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:06.560274+0000 mon.a (mon.0) 918 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi047-82428-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:08.180 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:06.564384+0000 mon.a (mon.0) 919 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi047-82378-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:08.180 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:06.569567+0000 mon.a (mon.0) 920 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-23T00:43:08.181 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:06.570396+0000 mon.a (mon.0) 921 : audit [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi047-82504-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi047-82504-6"}]': finished 2024-02-23T00:43:08.181 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:06.570482+0000 mon.a (mon.0) 922 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi047-82394-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi047-82394-15"}]': finished 2024-02-23T00:43:08.181 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:05.177410+0000 mgr.y (mgr.14517) 161 : cluster [DBG] pgmap v147: 652 pgs: 200 unknown, 32 creating+peering, 420 active+clean; 144 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 11 MiB/s rd, 34 MiB/s wr, 503 op/s; 84 B/s, 0 objects/s recovering 2024-02-23T00:43:08.181 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:05.470721+0000 mon.b (mon.1) 85 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:08.181 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:05.857795+0000 osd.1 (osd.1) 83 : cluster [DBG] 20.5 deep-scrub starts 2024-02-23T00:43:08.181 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:05.860326+0000 osd.1 (osd.1) 84 : cluster [DBG] 20.5 deep-scrub ok 2024-02-23T00:43:08.181 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:06.010312+0000 osd.7 (osd.7) 79 : cluster [DBG] 20.2 deep-scrub starts 2024-02-23T00:43:08.181 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:06.013821+0000 osd.7 (osd.7) 80 : cluster [DBG] 20.2 deep-scrub ok 2024-02-23T00:43:08.181 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:06.471842+0000 mon.b (mon.1) 86 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:08.181 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:06.550257+0000 mon.a (mon.0) 911 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi047-82386-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:08.181 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:06.550385+0000 mon.a (mon.0) 912 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1"}]': finished 2024-02-23T00:43:08.182 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:06.550539+0000 mon.a (mon.0) 913 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi047-82428-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:08.182 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:06.550791+0000 mon.a (mon.0) 914 : audit [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi047-82438-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:08.182 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:06.550961+0000 mon.a (mon.0) 915 : cluster [DBG] osdmap e116: 8 total, 8 up, 8 in 2024-02-23T00:43:08.182 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:06.551100+0000 mon.c (mon.2) 40 : audit [INF] from='client.? 172.21.15.47:0/1913712492' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-4", "overlaypool": "test-rados-api-smithi047-82465-4-cache"}]: dispatch 2024-02-23T00:43:08.182 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:06.551162+0000 mon.b (mon.1) 87 : audit [INF] from='client.? 172.21.15.47:0/2670125835' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:43:08.182 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:06.570566+0000 mon.a (mon.0) 923 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi047-82512-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi047-82512-7"}]': finished 2024-02-23T00:43:08.182 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:06.890552+0000 osd.1 (osd.1) 85 : cluster [DBG] 20.6 deep-scrub starts 2024-02-23T00:43:08.182 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:06.922774+0000 osd.1 (osd.1) 86 : cluster [DBG] 20.6 deep-scrub ok 2024-02-23T00:43:08.182 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:07.028411+0000 mon.a (mon.0) 924 : cluster [DBG] osdmap e117: 8 total, 8 up, 8 in 2024-02-23T00:43:08.183 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:07.180147+0000 mgr.y (mgr.14517) 162 : cluster [DBG] pgmap v150: 572 pgs: 56 creating+peering, 2 clean+premerge+peered, 514 active+clean; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:43:08.183 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:07.472931+0000 mon.b (mon.1) 90 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:08.183 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:07.476344+0000 mon.a (mon.0) 925 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:43:08.183 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:07.476622+0000 mon.a (mon.0) 926 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:08.183 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:07.476856+0000 mon.a (mon.0) 927 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi047-82386-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-23T00:43:08.183 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:06.551531+0000 mon.c (mon.2) 41 : audit [INF] from='client.? 172.21.15.47:0/1119363424' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi047-82428-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:08.183 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:06.552342+0000 mon.b (mon.1) 88 : audit [INF] from='client.? 172.21.15.47:0/1505701467' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:08.183 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:06.552414+0000 mon.c (mon.2) 42 : audit [INF] from='client.? 172.21.15.47:0/830645324' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi047-82378-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:08.183 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:06.552503+0000 mon.b (mon.1) 89 : audit [INF] from='client.? 172.21.15.47:0/854895257' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi047-82386-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-23T00:43:08.184 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:06.555720+0000 mon.a (mon.0) 916 : audit [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi047-82438-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi047-82438-10"}]: dispatch 2024-02-23T00:43:08.184 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:06.559950+0000 mon.a (mon.0) 917 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-4", "overlaypool": "test-rados-api-smithi047-82465-4-cache"}]: dispatch 2024-02-23T00:43:08.184 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:06.560274+0000 mon.a (mon.0) 918 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi047-82428-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:08.184 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:06.564384+0000 mon.a (mon.0) 919 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi047-82378-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:08.184 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:06.569567+0000 mon.a (mon.0) 920 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-23T00:43:08.184 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:06.570396+0000 mon.a (mon.0) 921 : audit [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi047-82504-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi047-82504-6"}]': finished 2024-02-23T00:43:08.184 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:06.570482+0000 mon.a (mon.0) 922 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi047-82394-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi047-82394-15"}]': finished 2024-02-23T00:43:08.184 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:06.570566+0000 mon.a (mon.0) 923 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi047-82512-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi047-82512-7"}]': finished 2024-02-23T00:43:08.184 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:06.890552+0000 osd.1 (osd.1) 85 : cluster [DBG] 20.6 deep-scrub starts 2024-02-23T00:43:08.184 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:06.922774+0000 osd.1 (osd.1) 86 : cluster [DBG] 20.6 deep-scrub ok 2024-02-23T00:43:08.184 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:07.028411+0000 mon.a (mon.0) 924 : cluster [DBG] osdmap e117: 8 total, 8 up, 8 in 2024-02-23T00:43:08.184 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:07.180147+0000 mgr.y (mgr.14517) 162 : cluster [DBG] pgmap v150: 572 pgs: 56 creating+peering, 2 clean+premerge+peered, 514 active+clean; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:43:08.184 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:07.472931+0000 mon.b (mon.1) 90 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:08.185 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:07.476344+0000 mon.a (mon.0) 925 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:43:08.185 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:07.476622+0000 mon.a (mon.0) 926 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:08.185 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:07.476856+0000 mon.a (mon.0) 927 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi047-82386-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-23T00:43:08.185 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:43:08.211 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:08.229 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:43:08.231 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:08.231 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:43:08.332 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:43:08.358 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:08.377 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:43:08.379 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:08.379 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:43:08.480 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:43:08.511 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:08.587 INFO:tasks.workunit.client.0.smithi047.stdout:crush rule rm", "name":"LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:08.587 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:04.271561+0000 mon.c [INF] from='client.? 172.21.15.47:0/1119363424' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi047-82428-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:08.587 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:04.272262+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi047-82428-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:08.587 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:06.550257+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi047-82386-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:08.587 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:06.550385+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi047-82528-1"}]': finished 2024-02-23T00:43:08.587 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:06.550539+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi047-82428-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:08.587 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:06.550791+0000 mon.a [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi047-82438-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:08.587 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:06.551100+0000 mon.c [INF] from='client.? 172.21.15.47:0/1913712492' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-4", "overlaypool": "test-rados-api-smithi047-82465-4-cache"}]: dispatch 2024-02-23T00:43:08.587 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:06.551162+0000 mon.b [INF] from='client.? 172.21.15.47:0/2670125835' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:43:08.588 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:06.551531+0000 mon.c [INF] from='client.? 172.21.15.47:0/1119363424' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi047-82428-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:08.588 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:06.552342+0000 mon.b [INF] from='client.? 172.21.15.47:0/1505701467' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:08.588 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:06.552414+0000 mon.c [INF] from='client.? 172.21.15.47:0/830645324' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi047-82378-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:08.588 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:06.552503+0000 mon.b [INF] from='client.? 172.21.15.47:0/854895257' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi047-82386-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-23T00:43:08.588 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:06.555720+0000 mon.a [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi047-82438-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi047-82438-10"}]: dispatch 2024-02-23T00:43:08.588 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:06.559950+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-4", "overlaypool": "test-rados-api-smithi047-82465-4-cache"}]: dispatch 2024-02-23T00:43:08.832 INFO:tasks.workunit.client.0.smithi047.stdout: open_pools_parallel: process_1_[82608]: starting. 2024-02-23T00:43:08.832 INFO:tasks.workunit.client.0.smithi047.stdout: open_pools_parallel: process_1_[82608]: creating pool ceph_test_rados_open_pools_parallel.smithi047-82592 2024-02-23T00:43:08.832 INFO:tasks.workunit.client.0.smithi047.stdout: open_pools_parallel: process_1_[82608]: created object 0... 2024-02-23T00:43:08.833 INFO:tasks.workunit.client.0.smithi047.stdout: open_pools_parallel: process_1_[82608]: created object 25... 2024-02-23T00:43:08.833 INFO:tasks.workunit.client.0.smithi047.stdout: open_pools_parallel: process_1_[82608]: created object 49... 2024-02-23T00:43:08.833 INFO:tasks.workunit.client.0.smithi047.stdout: open_pools_parallel: process_1_[82608]: finishing. 2024-02-23T00:43:08.833 INFO:tasks.workunit.client.0.smithi047.stdout: open_pools_parallel: process_1_[82608]: shutting down. 2024-02-23T00:43:08.833 INFO:tasks.workunit.client.0.smithi047.stdout: open_pools_parallel: process_2_[82609]: starting. 2024-02-23T00:43:08.833 INFO:tasks.workunit.client.0.smithi047.stdout: open_pools_parallel: process_2_[82609]: rados_pool_create. 2024-02-23T00:43:08.833 INFO:tasks.workunit.client.0.smithi047.stdout: open_pools_parallel: process_2_[82609]: rados_ioctx_create. 2024-02-23T00:43:08.833 INFO:tasks.workunit.client.0.smithi047.stdout: open_pools_parallel: process_2_[82609]: shutting down. 2024-02-23T00:43:08.833 INFO:tasks.workunit.client.0.smithi047.stdout: open_pools_parallel: ******************************* 2024-02-23T00:43:08.833 INFO:tasks.workunit.client.0.smithi047.stdout: open_pools_parallel: process_3_[85646]: starting. 2024-02-23T00:43:08.833 INFO:tasks.workunit.client.0.smithi047.stdout: open_pools_parallel: process_3_[85646]: creating pool ceph_test_rados_open_pools_parallel.smithi047-82592 2024-02-23T00:43:08.833 INFO:tasks.workunit.client.0.smithi047.stdout: open_pools_parallel: process_3_[85646]: created object 0... 2024-02-23T00:43:08.833 INFO:tasks.workunit.client.0.smithi047.stdout: open_pools_parallel: process_3_[85646]: created object 25... 2024-02-23T00:43:08.833 INFO:tasks.workunit.client.0.smithi047.stdout: open_pools_parallel: process_3_[85646]: created object 49... 2024-02-23T00:43:08.834 INFO:tasks.workunit.client.0.smithi047.stdout: open_pools_parallel: process_3_[85646]: finishing. 2024-02-23T00:43:08.834 INFO:tasks.workunit.client.0.smithi047.stdout: open_pools_parallel: process_3_[85646]: shutting down. 2024-02-23T00:43:08.834 INFO:tasks.workunit.client.0.smithi047.stdout: open_pools_parallel: ******************************* 2024-02-23T00:43:08.834 INFO:tasks.workunit.client.0.smithi047.stdout: open_pools_parallel: process_4_[85647]: starting. 2024-02-23T00:43:08.834 INFO:tasks.workunit.client.0.smithi047.stdout: open_pools_parallel: process_4_[85647]: rados_pool_create. 2024-02-23T00:43:08.834 INFO:tasks.workunit.client.0.smithi047.stdout: open_pools_parallel: process_4_[85647]: rados_ioctx_create. 2024-02-23T00:43:08.834 INFO:tasks.workunit.client.0.smithi047.stdout: open_pools_parallel: process_4_[85647]: shutting down. 2024-02-23T00:43:08.834 INFO:tasks.workunit.client.0.smithi047.stdout: open_pools_parallel: ******************************* 2024-02-23T00:43:08.834 INFO:tasks.workunit.client.0.smithi047.stdout: open_pools_parallel: ******************************* 2024-02-23T00:43:08.834 INFO:tasks.workunit.client.0.smithi047.stdout: open_pools_parallel: ******* SUCCESS ********** 2024-02-23T00:43:09.315 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:06.560274+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi047-82428-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:09.315 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:06.564384+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi047-82378-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:09.315 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:06.569567+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-23T00:43:09.315 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:06.570396+0000 mon.a [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi047-82504-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi047-82504-6"}]': finished 2024-02-23T00:43:09.315 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:06.570482+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi047-82394-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi047-82394-15"}]': finished 2024-02-23T00:43:09.315 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:06.570566+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi047-82512-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi047-82512-7"}]': finished 2024-02-23T00:43:09.315 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:07.476344+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi047-82528-1"}]: dispatch 2024-02-23T00:43:09.315 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:07.476622+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:09.315 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:07.476856+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi047-82386-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-23T00:43:09.316 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:08.390216+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-4", "overlaypool": "test-rados-api-smithi047-82465-4-cache"}]': finished 2024-02-23T00:43:09.316 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:08.390796+0000 mon.c [INF] from='client.? 172.21.15.47:0/1913712492' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-4-cache", "mode": "writeback"}]: dispatch 2024-02-23T00:43:09.316 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:08.391053+0000 mon.c [INF] from='client.? 172.21.15.47:0/1006300819' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi047-82448-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:09.316 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:08.585886+0000 mon.b [INF] from='client.? 172.21.15.47:0/3655582872' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-23T00:43:09.316 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:08.586056+0000 client.admin [INF] threexx 2024-02-23T00:43:09.316 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:08.820636+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi047-82378-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:09.316 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:08.820818+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi047-82528-1"}]': finished 2024-02-23T00:43:09.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:07.933603+0000 mon.a (mon.0) 928 : audit [DBG] from='client.? 172.21.15.47:0/669469315' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:43:09.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:08.390216+0000 mon.a (mon.0) 929 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-4", "overlaypool": "test-rados-api-smithi047-82465-4-cache"}]': finished 2024-02-23T00:43:09.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:08.390796+0000 mon.c (mon.2) 43 : audit [INF] from='client.? 172.21.15.47:0/1913712492' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-4-cache", "mode": "writeback"}]: dispatch 2024-02-23T00:43:09.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:08.391053+0000 mon.c (mon.2) 44 : audit [INF] from='client.? 172.21.15.47:0/1006300819' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi047-82448-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:09.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:08.474022+0000 mon.b (mon.1) 91 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:09.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:08.585886+0000 mon.b (mon.1) 92 : audit [INF] from='client.? 172.21.15.47:0/3655582872' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-23T00:43:09.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:08.586056+0000 client.admin (client.?) 0 : cluster [INF] threexx 2024-02-23T00:43:09.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:08.820636+0000 mon.a (mon.0) 930 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi047-82378-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:09.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:08.820818+0000 mon.a (mon.0) 931 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi047-82528-1"}]': finished 2024-02-23T00:43:09.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:08.820902+0000 mon.a (mon.0) 932 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi047-82402-22"}]': finished 2024-02-23T00:43:09.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:08.821012+0000 mon.a (mon.0) 933 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi047-82386-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-23T00:43:09.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:08.821259+0000 mon.a (mon.0) 934 : cluster [DBG] osdmap e118: 8 total, 8 up, 8 in 2024-02-23T00:43:09.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:08.821568+0000 mon.b (mon.1) 93 : audit [INF] from='client.? 172.21.15.47:0/1505701467' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:09.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:07.933603+0000 mon.a (mon.0) 928 : audit [DBG] from='client.? 172.21.15.47:0/669469315' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:43:09.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:08.390216+0000 mon.a (mon.0) 929 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-4", "overlaypool": "test-rados-api-smithi047-82465-4-cache"}]': finished 2024-02-23T00:43:09.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:08.390796+0000 mon.c (mon.2) 43 : audit [INF] from='client.? 172.21.15.47:0/1913712492' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-4-cache", "mode": "writeback"}]: dispatch 2024-02-23T00:43:09.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:08.391053+0000 mon.c (mon.2) 44 : audit [INF] from='client.? 172.21.15.47:0/1006300819' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi047-82448-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:09.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:08.474022+0000 mon.b (mon.1) 91 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:09.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:08.585886+0000 mon.b (mon.1) 92 : audit [INF] from='client.? 172.21.15.47:0/3655582872' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-23T00:43:09.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:08.586056+0000 client.admin (client.?) 0 : cluster [INF] threexx 2024-02-23T00:43:09.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:08.820636+0000 mon.a (mon.0) 930 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi047-82378-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:09.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:08.820818+0000 mon.a (mon.0) 931 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi047-82528-1"}]': finished 2024-02-23T00:43:09.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:08.820902+0000 mon.a (mon.0) 932 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi047-82402-22"}]': finished 2024-02-23T00:43:09.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:08.821012+0000 mon.a (mon.0) 933 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi047-82386-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-23T00:43:09.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:08.821259+0000 mon.a (mon.0) 934 : cluster [DBG] osdmap e118: 8 total, 8 up, 8 in 2024-02-23T00:43:09.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:08.821568+0000 mon.b (mon.1) 93 : audit [INF] from='client.? 172.21.15.47:0/1505701467' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:09.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:08.822283+0000 mon.c (mon.2) 45 : audit [INF] from='client.? 172.21.15.47:0/2229270924' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:09.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:08.830580+0000 mon.a (mon.0) 935 : audit [INF] from='client.? 172.21.15.47:0/2049502571' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:09.560 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:08.833379+0000 mon.a (mon.0) 936 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-4-cache", "mode": "writeback"}]: dispatch 2024-02-23T00:43:09.560 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:08.833730+0000 mon.a (mon.0) 937 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi047-82448-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:09.560 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:08.838068+0000 mon.a (mon.0) 938 : audit [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi047-82438-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi047-82438-10"}]': finished 2024-02-23T00:43:09.560 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:08.838154+0000 mon.a (mon.0) 939 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi047-82428-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi047-82428-10"}]': finished 2024-02-23T00:43:09.560 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:08.840238+0000 mon.a (mon.0) 940 : cluster [DBG] osdmap e119: 8 total, 8 up, 8 in 2024-02-23T00:43:09.560 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:08.840910+0000 mon.a (mon.0) 941 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:09.560 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:08.842395+0000 mon.a (mon.0) 942 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-23T00:43:09.560 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:08.842573+0000 mon.a (mon.0) 943 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:09.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:08.822283+0000 mon.c (mon.2) 45 : audit [INF] from='client.? 172.21.15.47:0/2229270924' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:09.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:08.830580+0000 mon.a (mon.0) 935 : audit [INF] from='client.? 172.21.15.47:0/2049502571' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:09.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:08.833379+0000 mon.a (mon.0) 936 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-4-cache", "mode": "writeback"}]: dispatch 2024-02-23T00:43:09.561 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:08.833730+0000 mon.a (mon.0) 937 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi047-82448-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:09.561 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:08.838068+0000 mon.a (mon.0) 938 : audit [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi047-82438-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi047-82438-10"}]': finished 2024-02-23T00:43:09.561 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:08.838154+0000 mon.a (mon.0) 939 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi047-82428-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi047-82428-10"}]': finished 2024-02-23T00:43:09.561 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:08.840238+0000 mon.a (mon.0) 940 : cluster [DBG] osdmap e119: 8 total, 8 up, 8 in 2024-02-23T00:43:09.561 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:08.840910+0000 mon.a (mon.0) 941 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:09.561 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:08.842395+0000 mon.a (mon.0) 942 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-23T00:43:09.561 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:08.842573+0000 mon.a (mon.0) 943 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:09.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:07.933603+0000 mon.a (mon.0) 928 : audit [DBG] from='client.? 172.21.15.47:0/669469315' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:43:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:08.390216+0000 mon.a (mon.0) 929 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-4", "overlaypool": "test-rados-api-smithi047-82465-4-cache"}]': finished 2024-02-23T00:43:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:08.390796+0000 mon.c (mon.2) 43 : audit [INF] from='client.? 172.21.15.47:0/1913712492' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-4-cache", "mode": "writeback"}]: dispatch 2024-02-23T00:43:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:08.391053+0000 mon.c (mon.2) 44 : audit [INF] from='client.? 172.21.15.47:0/1006300819' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi047-82448-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:08.474022+0000 mon.b (mon.1) 91 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:08.585886+0000 mon.b (mon.1) 92 : audit [INF] from='client.? 172.21.15.47:0/3655582872' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-23T00:43:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:08.586056+0000 client.admin (client.?) 0 : cluster [INF] threexx 2024-02-23T00:43:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:08.820636+0000 mon.a (mon.0) 930 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi047-82378-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:08.820818+0000 mon.a (mon.0) 931 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi047-82528-1"}]': finished 2024-02-23T00:43:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:08.820902+0000 mon.a (mon.0) 932 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi047-82402-22"}]': finished 2024-02-23T00:43:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:08.821012+0000 mon.a (mon.0) 933 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi047-82386-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-23T00:43:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:08.821259+0000 mon.a (mon.0) 934 : cluster [DBG] osdmap e118: 8 total, 8 up, 8 in 2024-02-23T00:43:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:08.821568+0000 mon.b (mon.1) 93 : audit [INF] from='client.? 172.21.15.47:0/1505701467' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:08.822283+0000 mon.c (mon.2) 45 : audit [INF] from='client.? 172.21.15.47:0/2229270924' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:08.830580+0000 mon.a (mon.0) 935 : audit [INF] from='client.? 172.21.15.47:0/2049502571' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:09.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:08.833379+0000 mon.a (mon.0) 936 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-4-cache", "mode": "writeback"}]: dispatch 2024-02-23T00:43:09.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:08.833730+0000 mon.a (mon.0) 937 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi047-82448-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:09.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:08.838068+0000 mon.a (mon.0) 938 : audit [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi047-82438-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi047-82438-10"}]': finished 2024-02-23T00:43:09.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:08.838154+0000 mon.a (mon.0) 939 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi047-82428-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi047-82428-10"}]': finished 2024-02-23T00:43:09.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:08.840238+0000 mon.a (mon.0) 940 : cluster [DBG] osdmap e119: 8 total, 8 up, 8 in 2024-02-23T00:43:09.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:08.840910+0000 mon.a (mon.0) 941 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:09.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:08.842395+0000 mon.a (mon.0) 942 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-23T00:43:09.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:08.842573+0000 mon.a (mon.0) 943 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:09.834 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:43:09.834 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-02-23T00:43:09.834 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-02-23T00:43:09.834 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-02-23T00:43:09.834 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:43:09.888 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:08.820902+0000 mon.a [INF] from='client.?ibRadosIoECPP.OverlappingWriteRoundTripPP 2024-02-23T00:43:09.888 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (4 ms) 2024-02-23T00:43:09.889 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-02-23T00:43:09.889 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (4 ms) 2024-02-23T00:43:09.889 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-02-23T00:43:09.889 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (3 ms) 2024-02-23T00:43:09.889 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-02-23T00:43:09.889 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (6 ms) 2024-02-23T00:43:09.889 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2024-02-23T00:43:09.889 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (5 ms) 2024-02-23T00:43:09.889 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2024-02-23T00:43:09.889 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (4 ms) 2024-02-23T00:43:09.889 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-02-23T00:43:09.889 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (4 ms) 2024-02-23T00:43:09.889 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2024-02-23T00:43:09.889 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (9 ms) 2024-02-23T00:43:09.890 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2024-02-23T00:43:09.890 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (3 ms) 2024-02-23T00:43:09.890 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2024-02-23T00:43:09.890 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (6 ms) 2024-02-23T00:43:09.890 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-02-23T00:43:09.890 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (3 ms) 2024-02-23T00:43:09.890 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-02-23T00:43:09.890 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (4 ms) 2024-02-23T00:43:09.890 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP (1213 ms total) 2024-02-23T00:43:09.890 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: 2024-02-23T00:43:09.890 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [----------] Global test environment tear-down 2024-02-23T00:43:09.890 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [==========] 37 tests from 2 test suites ran. (22676 ms total) 2024-02-23T00:43:09.890 INFO:tasks.workunit.client.0.smithi047.stdout: api_io_pp: [ PASSED ] 37 tests. 2024-02-23T00:43:10.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:08.399359+0000 osd.6 (osd.6) 61 : cluster [DBG] 20.8 deep-scrub starts 2024-02-23T00:43:10.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:08.400202+0000 osd.6 (osd.6) 62 : cluster [DBG] 20.8 deep-scrub ok 2024-02-23T00:43:10.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:08.979058+0000 osd.7 (osd.7) 81 : cluster [DBG] 20.7 deep-scrub starts 2024-02-23T00:43:10.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:08.979786+0000 osd.7 (osd.7) 82 : cluster [DBG] 20.7 deep-scrub ok 2024-02-23T00:43:10.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:09.180709+0000 mgr.y (mgr.14517) 163 : cluster [DBG] pgmap v153: 651 pgs: 112 unknown, 56 creating+peering, 1 clean+premerge+peered, 482 active+clean; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-23T00:43:10.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:09.317521+0000 mon.a (mon.0) 944 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-23T00:43:10.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:09.317702+0000 mon.a (mon.0) 945 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-02-23T00:43:10.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:09.318100+0000 mon.b (mon.1) 94 : audit [INF] from='client.? 172.21.15.47:0/3655582872' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-23T00:43:10.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:09.318200+0000 client.admin (client.?) 0 : cluster [INF] fourxx 2024-02-23T00:43:10.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:09.321254+0000 mon.a (mon.0) 946 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-23T00:43:10.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:09.475164+0000 mon.b (mon.1) 95 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:10.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:09.835268+0000 mon.a (mon.0) 947 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:43:10.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:09.835723+0000 mon.a (mon.0) 948 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:43:10.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:09.840949+0000 mon.a (mon.0) 949 : audit [INF] from='client.? 172.21.15.47:0/2049502571' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:10.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:09.841140+0000 mon.a (mon.0) 950 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-4-cache", "mode": "writeback"}]': finished 2024-02-23T00:43:10.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:09.841225+0000 mon.a (mon.0) 951 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi047-82448-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:10.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:09.841304+0000 mon.a (mon.0) 952 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:10.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:09.841442+0000 mon.a (mon.0) 953 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi047-82402-22"}]': finished 2024-02-23T00:43:10.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:09.841651+0000 mon.a (mon.0) 954 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "30"}]': finished 2024-02-23T00:43:10.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:09.841710+0000 mon.a (mon.0) 955 : cluster [DBG] osdmap e120: 8 total, 8 up, 8 in 2024-02-23T00:43:10.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:09.846792+0000 mon.c (mon.2) 46 : audit [INF] from='client.? 172.21.15.47:0/4182485101' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:10.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:09.857915+0000 mon.a (mon.0) 956 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:10.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:09.859797+0000 mon.b (mon.1) 96 : audit [INF] from='client.? 172.21.15.47:0/1728200087' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:10.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:09.867300+0000 mon.b (mon.1) 97 : audit [INF] from='client.? 172.21.15.47:0/851664869' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:10.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:09.867350+0000 mon.a (mon.0) 957 : audit [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi047-82504-6"}]: dispatch 2024-02-23T00:43:10.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:09.883343+0000 mon.a (mon.0) 958 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:10.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:09.883703+0000 mon.a (mon.0) 959 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:10.806 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:43:10] "GET /metrics HTTP/1.1" 200 229257 "" "Prometheus/2.33.4" 2024-02-23T00:43:10.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:08.399359+0000 osd.6 (osd.6) 61 : cluster [DBG] 20.8 deep-scrub starts 2024-02-23T00:43:10.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:08.400202+0000 osd.6 (osd.6) 62 : cluster [DBG] 20.8 deep-scrub ok 2024-02-23T00:43:10.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:08.979058+0000 osd.7 (osd.7) 81 : cluster [DBG] 20.7 deep-scrub starts 2024-02-23T00:43:10.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:08.979786+0000 osd.7 (osd.7) 82 : cluster [DBG] 20.7 deep-scrub ok 2024-02-23T00:43:10.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:09.180709+0000 mgr.y (mgr.14517) 163 : cluster [DBG] pgmap v153: 651 pgs: 112 unknown, 56 creating+peering, 1 clean+premerge+peered, 482 active+clean; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-23T00:43:10.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:09.317521+0000 mon.a (mon.0) 944 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-23T00:43:10.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:09.317702+0000 mon.a (mon.0) 945 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-02-23T00:43:10.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:09.318100+0000 mon.b (mon.1) 94 : audit [INF] from='client.? 172.21.15.47:0/3655582872' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-23T00:43:10.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:09.318200+0000 client.admin (client.?) 0 : cluster [INF] fourxx 2024-02-23T00:43:10.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:09.321254+0000 mon.a (mon.0) 946 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-23T00:43:10.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:09.475164+0000 mon.b (mon.1) 95 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:10.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:09.835268+0000 mon.a (mon.0) 947 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:43:10.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:09.835723+0000 mon.a (mon.0) 948 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:43:10.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:09.840949+0000 mon.a (mon.0) 949 : audit [INF] from='client.? 172.21.15.47:0/2049502571' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:10.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:09.841140+0000 mon.a (mon.0) 950 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-4-cache", "mode": "writeback"}]': finished 2024-02-23T00:43:10.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:09.841225+0000 mon.a (mon.0) 951 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi047-82448-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:10.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:08.399359+0000 osd.6 (osd.6) 61 : cluster [DBG] 20.8 deep-scrub starts 2024-02-23T00:43:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:08.400202+0000 osd.6 (osd.6) 62 : cluster [DBG] 20.8 deep-scrub ok 2024-02-23T00:43:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:08.979058+0000 osd.7 (osd.7) 81 : cluster [DBG] 20.7 deep-scrub starts 2024-02-23T00:43:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:08.979786+0000 osd.7 (osd.7) 82 : cluster [DBG] 20.7 deep-scrub ok 2024-02-23T00:43:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:09.180709+0000 mgr.y (mgr.14517) 163 : cluster [DBG] pgmap v153: 651 pgs: 112 unknown, 56 creating+peering, 1 clean+premerge+peered, 482 active+clean; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-23T00:43:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:09.317521+0000 mon.a (mon.0) 944 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-23T00:43:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:09.317702+0000 mon.a (mon.0) 945 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-02-23T00:43:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:09.318100+0000 mon.b (mon.1) 94 : audit [INF] from='client.? 172.21.15.47:0/3655582872' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-23T00:43:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:09.318200+0000 client.admin (client.?) 0 : cluster [INF] fourxx 2024-02-23T00:43:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:09.321254+0000 mon.a (mon.0) 946 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-23T00:43:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:09.475164+0000 mon.b (mon.1) 95 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:09.835268+0000 mon.a (mon.0) 947 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:43:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:09.835723+0000 mon.a (mon.0) 948 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:43:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:09.840949+0000 mon.a (mon.0) 949 : audit [INF] from='client.? 172.21.15.47:0/2049502571' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:09.841140+0000 mon.a (mon.0) 950 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-4-cache", "mode": "writeback"}]': finished 2024-02-23T00:43:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:09.841225+0000 mon.a (mon.0) 951 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi047-82448-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:10.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:09.841304+0000 mon.a (mon.0) 952 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:10.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:09.841442+0000 mon.a (mon.0) 953 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi047-82402-22"}]': finished 2024-02-23T00:43:10.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:09.841651+0000 mon.a (mon.0) 954 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "30"}]': finished 2024-02-23T00:43:10.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:09.841710+0000 mon.a (mon.0) 955 : cluster [DBG] osdmap e120: 8 total, 8 up, 8 in 2024-02-23T00:43:10.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:09.846792+0000 mon.c (mon.2) 46 : audit [INF] from='client.? 172.21.15.47:0/4182485101' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:10.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:09.857915+0000 mon.a (mon.0) 956 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:10.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:09.859797+0000 mon.b (mon.1) 96 : audit [INF] from='client.? 172.21.15.47:0/1728200087' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:10.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:09.867300+0000 mon.b (mon.1) 97 : audit [INF] from='client.? 172.21.15.47:0/851664869' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:10.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:09.867350+0000 mon.a (mon.0) 957 : audit [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi047-82504-6"}]: dispatch 2024-02-23T00:43:10.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:09.883343+0000 mon.a (mon.0) 958 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:10.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:09.883703+0000 mon.a (mon.0) 959 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:10.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:09.841304+0000 mon.a (mon.0) 952 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:10.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:09.841442+0000 mon.a (mon.0) 953 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi047-82402-22"}]': finished 2024-02-23T00:43:10.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:09.841651+0000 mon.a (mon.0) 954 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "30"}]': finished 2024-02-23T00:43:10.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:09.841710+0000 mon.a (mon.0) 955 : cluster [DBG] osdmap e120: 8 total, 8 up, 8 in 2024-02-23T00:43:10.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:09.846792+0000 mon.c (mon.2) 46 : audit [INF] from='client.? 172.21.15.47:0/4182485101' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:10.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:09.857915+0000 mon.a (mon.0) 956 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:10.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:09.859797+0000 mon.b (mon.1) 96 : audit [INF] from='client.? 172.21.15.47:0/1728200087' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:10.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:09.867300+0000 mon.b (mon.1) 97 : audit [INF] from='client.? 172.21.15.47:0/851664869' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:10.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:09.867350+0000 mon.a (mon.0) 957 : audit [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi047-82504-6"}]: dispatch 2024-02-23T00:43:10.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:09.883343+0000 mon.a (mon.0) 958 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:10.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:09.883703+0000 mon.a (mon.0) 959 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:11.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:09.925638+0000 mon.c (mon.2) 47 : audit [INF] from='client.? 172.21.15.47:0/1913712492' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-4"}]: dispatch 2024-02-23T00:43:11.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:09.927250+0000 mon.c (mon.2) 48 : audit [DBG] from='client.? 172.21.15.47:0/1006300819' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-23T00:43:11.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:09.927802+0000 mon.c (mon.2) 49 : audit [INF] from='client.? 172.21.15.47:0/1006300819' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi047-82448-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:11.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:10.345325+0000 mon.a (mon.0) 960 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-23T00:43:11.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:10.347128+0000 mon.a (mon.0) 961 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-4"}]: dispatch 2024-02-23T00:43:11.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:10.347325+0000 mon.a (mon.0) 962 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi047-82448-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:11.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:10.476283+0000 mon.b (mon.1) 98 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:11.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:10.842709+0000 mon.a (mon.0) 963 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:11.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:10.842905+0000 mon.a (mon.0) 964 : audit [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi047-82504-6"}]': finished 2024-02-23T00:43:11.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:10.842982+0000 mon.a (mon.0) 965 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi047-82394-15"}]': finished 2024-02-23T00:43:11.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:10.843077+0000 mon.a (mon.0) 966 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi047-82512-7"}]': finished 2024-02-23T00:43:11.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:10.843275+0000 mon.a (mon.0) 967 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-4"}]': finished 2024-02-23T00:43:11.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:10.843359+0000 mon.a (mon.0) 968 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi047-82448-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:11.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:10.843441+0000 mon.a (mon.0) 969 : cluster [DBG] osdmap e121: 8 total, 8 up, 8 in 2024-02-23T00:43:11.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:10.847193+0000 mon.b (mon.1) 99 : audit [INF] from='client.? 172.21.15.47:0/851664869' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:11.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:10.847450+0000 mon.b (mon.1) 100 : audit [INF] from='client.? 172.21.15.47:0/1728200087' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:11.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:10.849202+0000 mon.c (mon.2) 50 : audit [INF] from='client.? 172.21.15.47:0/1913712492' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-4", "tierpool": "test-rados-api-smithi047-82465-4-cache"}]: dispatch 2024-02-23T00:43:11.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:10.850035+0000 mon.a (mon.0) 970 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-4", "tierpool": "test-rados-api-smithi047-82465-4-cache"}]: dispatch 2024-02-23T00:43:11.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:10.851458+0000 mon.a (mon.0) 971 : audit [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi047-82504-6"}]: dispatch 2024-02-23T00:43:11.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:10.852477+0000 mon.c (mon.2) 51 : audit [INF] from='client.? 172.21.15.47:0/3057594868' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi047-82378-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:11.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:10.852941+0000 mon.a (mon.0) 972 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi047-82378-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:11.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:10.855956+0000 mon.a (mon.0) 973 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:11.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:10.856257+0000 mon.a (mon.0) 974 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:11.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:10.869764+0000 mon.c (mon.2) 52 : audit [INF] from='client.? 172.21.15.47:0/1006300819' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi047-82448-1","app": "app2"}]: dispatch 2024-02-23T00:43:11.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:10.873362+0000 mon.c (mon.2) 53 : audit [INF] from='client.? 172.21.15.47:0/1006300819' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi047-82448-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:11.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:10.874187+0000 mon.a (mon.0) 975 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi047-82448-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:11.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:10.925430+0000 mon.b (mon.1) 101 : audit [INF] from='client.? 172.21.15.47:0/2137942802' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi047-82517-12"}]: dispatch 2024-02-23T00:43:11.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:10.926176+0000 mon.a (mon.0) 976 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi047-82517-12"}]: dispatch 2024-02-23T00:43:11.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:09.925638+0000 mon.c (mon.2) 47 : audit [INF] from='client.? 172.21.15.47:0/1913712492' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-4"}]: dispatch 2024-02-23T00:43:11.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:09.927250+0000 mon.c (mon.2) 48 : audit [DBG] from='client.? 172.21.15.47:0/1006300819' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-23T00:43:11.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:09.927802+0000 mon.c (mon.2) 49 : audit [INF] from='client.? 172.21.15.47:0/1006300819' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi047-82448-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:11.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:10.345325+0000 mon.a (mon.0) 960 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-23T00:43:11.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:10.347128+0000 mon.a (mon.0) 961 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-4"}]: dispatch 2024-02-23T00:43:11.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:10.347325+0000 mon.a (mon.0) 962 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi047-82448-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:11.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:10.476283+0000 mon.b (mon.1) 98 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:11.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:10.842709+0000 mon.a (mon.0) 963 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:11.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:10.842905+0000 mon.a (mon.0) 964 : audit [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi047-82504-6"}]': finished 2024-02-23T00:43:11.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:10.842982+0000 mon.a (mon.0) 965 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi047-82394-15"}]': finished 2024-02-23T00:43:11.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:10.843077+0000 mon.a (mon.0) 966 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi047-82512-7"}]': finished 2024-02-23T00:43:11.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:10.843275+0000 mon.a (mon.0) 967 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-4"}]': finished 2024-02-23T00:43:11.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:10.843359+0000 mon.a (mon.0) 968 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi047-82448-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:11.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:09.925638+0000 mon.c (mon.2) 47 : audit [INF] from='client.? 172.21.15.47:0/1913712492' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-4"}]: dispatch 2024-02-23T00:43:11.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:09.927250+0000 mon.c (mon.2) 48 : audit [DBG] from='client.? 172.21.15.47:0/1006300819' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-23T00:43:11.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:09.927802+0000 mon.c (mon.2) 49 : audit [INF] from='client.? 172.21.15.47:0/1006300819' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi047-82448-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:11.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:10.345325+0000 mon.a (mon.0) 960 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-23T00:43:11.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:10.347128+0000 mon.a (mon.0) 961 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-4"}]: dispatch 2024-02-23T00:43:11.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:10.347325+0000 mon.a (mon.0) 962 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi047-82448-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:11.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:10.476283+0000 mon.b (mon.1) 98 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:11.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:10.842709+0000 mon.a (mon.0) 963 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:11.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:10.842905+0000 mon.a (mon.0) 964 : audit [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi047-82504-6"}]': finished 2024-02-23T00:43:11.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:10.842982+0000 mon.a (mon.0) 965 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi047-82394-15"}]': finished 2024-02-23T00:43:11.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:10.843077+0000 mon.a (mon.0) 966 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi047-82512-7"}]': finished 2024-02-23T00:43:11.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:10.843275+0000 mon.a (mon.0) 967 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-4"}]': finished 2024-02-23T00:43:11.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:10.843359+0000 mon.a (mon.0) 968 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi047-82448-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:11.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:10.843441+0000 mon.a (mon.0) 969 : cluster [DBG] osdmap e121: 8 total, 8 up, 8 in 2024-02-23T00:43:11.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:10.847193+0000 mon.b (mon.1) 99 : audit [INF] from='client.? 172.21.15.47:0/851664869' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:11.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:10.847450+0000 mon.b (mon.1) 100 : audit [INF] from='client.? 172.21.15.47:0/1728200087' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:11.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:10.849202+0000 mon.c (mon.2) 50 : audit [INF] from='client.? 172.21.15.47:0/1913712492' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-4", "tierpool": "test-rados-api-smithi047-82465-4-cache"}]: dispatch 2024-02-23T00:43:11.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:10.850035+0000 mon.a (mon.0) 970 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-4", "tierpool": "test-rados-api-smithi047-82465-4-cache"}]: dispatch 2024-02-23T00:43:11.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:10.851458+0000 mon.a (mon.0) 971 : audit [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi047-82504-6"}]: dispatch 2024-02-23T00:43:11.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:10.852477+0000 mon.c (mon.2) 51 : audit [INF] from='client.? 172.21.15.47:0/3057594868' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi047-82378-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:11.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:10.852941+0000 mon.a (mon.0) 972 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi047-82378-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:11.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:10.855956+0000 mon.a (mon.0) 973 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:11.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:10.856257+0000 mon.a (mon.0) 974 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:11.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:10.869764+0000 mon.c (mon.2) 52 : audit [INF] from='client.? 172.21.15.47:0/1006300819' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi047-82448-1","app": "app2"}]: dispatch 2024-02-23T00:43:11.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:10.873362+0000 mon.c (mon.2) 53 : audit [INF] from='client.? 172.21.15.47:0/1006300819' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi047-82448-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:11.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:10.874187+0000 mon.a (mon.0) 975 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi047-82448-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:11.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:10.843441+0000 mon.a (mon.0) 969 : cluster [DBG] osdmap e121: 8 total, 8 up, 8 in 2024-02-23T00:43:11.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:10.847193+0000 mon.b (mon.1) 99 : audit [INF] from='client.? 172.21.15.47:0/851664869' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:11.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:10.847450+0000 mon.b (mon.1) 100 : audit [INF] from='client.? 172.21.15.47:0/1728200087' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:11.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:10.849202+0000 mon.c (mon.2) 50 : audit [INF] from='client.? 172.21.15.47:0/1913712492' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-4", "tierpool": "test-rados-api-smithi047-82465-4-cache"}]: dispatch 2024-02-23T00:43:11.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:10.850035+0000 mon.a (mon.0) 970 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-4", "tierpool": "test-rados-api-smithi047-82465-4-cache"}]: dispatch 2024-02-23T00:43:11.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:10.851458+0000 mon.a (mon.0) 971 : audit [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi047-82504-6"}]: dispatch 2024-02-23T00:43:11.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:10.852477+0000 mon.c (mon.2) 51 : audit [INF] from='client.? 172.21.15.47:0/3057594868' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi047-82378-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:11.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:10.852941+0000 mon.a (mon.0) 972 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi047-82378-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:11.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:10.855956+0000 mon.a (mon.0) 973 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi047-82512-7"}]: dispatch 2024-02-23T00:43:11.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:10.856257+0000 mon.a (mon.0) 974 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi047-82394-15"}]: dispatch 2024-02-23T00:43:11.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:10.869764+0000 mon.c (mon.2) 52 : audit [INF] from='client.? 172.21.15.47:0/1006300819' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi047-82448-1","app": "app2"}]: dispatch 2024-02-23T00:43:11.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:10.873362+0000 mon.c (mon.2) 53 : audit [INF] from='client.? 172.21.15.47:0/1006300819' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi047-82448-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:11.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:10.874187+0000 mon.a (mon.0) 975 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi047-82448-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:11.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:10.925430+0000 mon.b (mon.1) 101 : audit [INF] from='client.? 172.21.15.47:0/2137942802' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi047-82517-12"}]: dispatch 2024-02-23T00:43:11.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:10.926176+0000 mon.a (mon.0) 976 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi047-82517-12"}]: dispatch 2024-02-23T00:43:11.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:10.925430+0000 mon.b (mon.1) 101 : audit [INF] from='client.? 172.21.15.47:0/2137942802' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi047-82517-12"}]: dispatch 2024-02-23T00:43:11.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:10.926176+0000 mon.a (mon.0) 976 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi047-82517-12"}]: dispatch 2024-02-23T00:43:12.271 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: Running main() from gmock_main.cc 2024-02-23T00:43:12.271 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [==========] Running 23 tests from 2 test suites. 2024-02-23T00:43:12.271 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [----------] Global test environment set-up. 2024-02-23T00:43:12.271 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [----------] 13 tests from LibRadosIo 2024-02-23T00:43:12.271 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2024-02-23T00:43:12.272 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (8 ms) 2024-02-23T00:43:12.272 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2024-02-23T00:43:12.272 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2024-02-23T00:43:12.272 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2024-02-23T00:43:12.272 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: no timeout :/ 2024-02-23T00:43:12.272 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: no timeout :/ 2024-02-23T00:43:12.272 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: no timeout :/ 2024-02-23T00:43:12.272 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: no timeout :/ 2024-02-23T00:43:12.272 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: no timeout :/ 2024-02-23T00:43:12.272 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (44 ms) 2024-02-23T00:43:12.272 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2024-02-23T00:43:12.272 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (9 ms) 2024-02-23T00:43:12.272 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2024-02-23T00:43:12.272 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ OK ] LibRadosIo.Checksum (6 ms) 2024-02-23T00:43:12.273 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-02-23T00:43:12.273 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (8 ms) 2024-02-23T00:43:12.273 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2024-02-23T00:43:12.273 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (5 ms) 2024-02-23T00:43:12.273 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2024-02-23T00:43:12.273 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (6 ms) 2024-02-23T00:43:12.273 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2024-02-23T00:43:12.273 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ OK ] LibRadosIo.TruncTest (6 ms) 2024-02-23T00:43:12.273 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2024-02-23T00:43:12.273 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (5 ms) 2024-02-23T00:43:12.273 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2024-02-23T00:43:12.273 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (7 ms) 2024-02-23T00:43:12.273 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2024-02-23T00:43:12.273 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ OK ] LibRadosIo.RmXattr (11 ms) 2024-02-23T00:43:12.274 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2024-02-23T00:43:12.274 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ OK ] LibRadosIo.XattrIter (5 ms) 2024-02-23T00:43:12.274 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [----------] 13 tests from LibRadosIo (120 ms total) 2024-02-23T00:43:12.274 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: 2024-02-23T00:43:12.274 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2024-02-23T00:43:12.274 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2024-02-23T00:43:12.274 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (1422 ms) 2024-02-23T00:43:12.274 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2024-02-23T00:43:12.274 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (7 ms) 2024-02-23T00:43:12.274 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-02-23T00:43:12.274 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (8 ms) 2024-02-23T00:43:12.274 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-02-23T00:43:12.274 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (4 ms) 2024-02-23T00:43:12.274 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2024-02-23T00:43:12.275 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (8 ms) 2024-02-23T00:43:12.275 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2024-02-23T00:43:12.275 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (3 ms) 2024-02-23T00:43:12.275 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2024-02-23T00:43:12.275 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (3 ms) 2024-02-23T00:43:12.275 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-02-23T00:43:12.275 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (4 ms) 2024-02-23T00:43:12.275 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2024-02-23T00:43:12.275 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (12 ms) 2024-02-23T00:43:12.275 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2024-02-23T00:43:12.275 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (5 ms) 2024-02-23T00:43:12.275 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [----------] 10 tests from LibRadosIoEC (1476 ms total) 2024-02-23T00:43:12.276 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: 2024-02-23T00:43:12.276 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [----------] Global test environment tear-down 2024-02-23T00:43:12.276 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [==========] 23 tests from 2 test suites ran. (25021 ms total) 2024-02-23T00:43:12.276 INFO:tasks.workunit.client.0.smithi047.stdout: api_io: [ PASSED ] 23 tests. 2024-02-23T00:43:12.276 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat_pp: Running main() from gmock_main.cc 2024-02-23T00:43:12.277 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2024-02-23T00:43:12.277 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat_pp: [----------] Global test environment set-up. 2024-02-23T00:43:12.277 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2024-02-23T00:43:12.277 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat_pp: seed 82512 2024-02-23T00:43:12.277 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2024-02-23T00:43:12.277 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (243 ms) 2024-02-23T00:43:12.277 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-02-23T00:43:12.277 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (3 ms) 2024-02-23T00:43:12.277 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2024-02-23T00:43:12.277 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (0 ms) 2024-02-23T00:43:12.277 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2024-02-23T00:43:12.277 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (25 ms) 2024-02-23T00:43:12.277 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2024-02-23T00:43:12.278 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (15 ms) 2024-02-23T00:43:12.278 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (287 ms total) 2024-02-23T00:43:12.278 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat_pp: 2024-02-23T00:43:12.278 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2024-02-23T00:43:12.278 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2024-02-23T00:43:12.278 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (1428 ms) 2024-02-23T00:43:12.278 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2024-02-23T00:43:12.278 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (1 ms) 2024-02-23T00:43:12.278 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2024-02-23T00:43:12.278 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (3 ms) 2024-02-23T00:43:12.278 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2024-02-23T00:43:12.278 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (11 ms) 2024-02-23T00:43:12.278 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (1443 ms total) 2024-02-23T00:43:12.278 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat_pp: 2024-02-23T00:43:12.279 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat_pp: [----------] Global test environment tear-down 2024-02-23T00:43:12.279 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (25046 ms total) 2024-02-23T00:43:12.279 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat_pp: [ PASSED ] 9 tests. 2024-02-23T00:43:12.296 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat: Running main() from gmock_main.cc 2024-02-23T00:43:12.296 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat: [==========] Running 8 tests from 2 test suites. 2024-02-23T00:43:12.296 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat: [----------] Global test environment set-up. 2024-02-23T00:43:12.296 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat: [----------] 4 tests from LibRadosStat 2024-02-23T00:43:12.296 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2024-02-23T00:43:12.296 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat: [ OK ] LibRadosStat.Stat (306 ms) 2024-02-23T00:43:12.296 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2024-02-23T00:43:12.296 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat: [ OK ] LibRadosStat.StatNS (12 ms) 2024-02-23T00:43:12.297 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2024-02-23T00:43:12.297 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (1 ms) 2024-02-23T00:43:12.297 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2024-02-23T00:43:12.297 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (2 ms) 2024-02-23T00:43:12.297 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat: [----------] 4 tests from LibRadosStat (322 ms total) 2024-02-23T00:43:12.297 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat: 2024-02-23T00:43:12.297 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2024-02-23T00:43:12.297 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2024-02-23T00:43:12.297 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (530 ms) 2024-02-23T00:43:12.297 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2024-02-23T00:43:12.298 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (21 ms) 2024-02-23T00:43:12.298 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2024-02-23T00:43:12.298 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (801 ms) 2024-02-23T00:43:12.298 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2024-02-23T00:43:12.298 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (3 ms) 2024-02-23T00:43:12.298 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (1355 ms total) 2024-02-23T00:43:12.298 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat: 2024-02-23T00:43:12.298 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat: [----------] Global test environment tear-down 2024-02-23T00:43:12.298 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat: [==========] 8 tests from 2 test suites ran. (25089 ms total) 2024-02-23T00:43:12.298 INFO:tasks.workunit.client.0.smithi047.stdout: api_stat: [ PASSED ] 8 tests. 2024-02-23T00:43:12.349 INFO:tasks.workunit.client.0.smithi047.stdout: ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi047-82402-22"}]': finished 2024-02-23T00:43:12.349 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:08.821012+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi047-82386-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-23T00:43:12.349 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:08.821568+0000 mon.b [INF] from='client.? 172.21.15.47:0/1505701467' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:12.349 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:08.822283+0000 mon.c [INF] from='client.? 172.21.15.47:0/2229270924' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:12.349 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:08.830580+0000 mon.a [INF] from='client.? 172.21.15.47:0/2049502571' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:12.349 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:08.833379+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-4-cache", "mode": "writeback"}]: dispatch 2024-02-23T00:43:12.349 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:08.833730+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi047-82448-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:12.349 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:08.838068+0000 mon.a [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi047-82438-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi047-82438-10"}]': finished 2024-02-23T00:43:12.349 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:08.838154+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi047-82428-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi047-82428-10"}]': finished 2024-02-23T00:43:12.349 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:08.840910+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:12.350 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:08.842395+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-23T00:43:12.350 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: got: 2024-02-23T00:43:08.842573+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi047-82402-22"}]: dispatch 2024-02-23T00:43:12.350 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (10536 ms) 2024-02-23T00:43:12.350 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (25121 ms total) 2024-02-23T00:43:12.350 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: 2024-02-23T00:43:12.350 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: [----------] Global test environment tear-down 2024-02-23T00:43:12.350 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (25121 ms total) 2024-02-23T00:43:12.350 INFO:tasks.workunit.client.0.smithi047.stdout: api_cmd: [ PASSED ] 4 tests. 2024-02-23T00:43:12.693 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:43:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:43:12.693 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:43:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:10.928547+0000 mon.b (mon.1) 102 : audit [INF] from='client.? 172.21.15.47:0/2137942802' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi047-82517-12"}]: dispatch 2024-02-23T00:43:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:10.933887+0000 mon.c (mon.2) 54 : audit [INF] from='client.? 172.21.15.47:0/4182485101' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "pg_num": 4}]: dispatch 2024-02-23T00:43:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:11.032400+0000 osd.7 (osd.7) 83 : cluster [DBG] 20.9 deep-scrub starts 2024-02-23T00:43:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:11.035858+0000 osd.7 (osd.7) 84 : cluster [DBG] 20.9 deep-scrub ok 2024-02-23T00:43:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:11.181234+0000 mgr.y (mgr.14517) 164 : cluster [DBG] pgmap v156: 563 pgs: 112 unknown, 1 clean+premerge+peered, 450 active+clean; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:43:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:11.359925+0000 mon.a (mon.0) 977 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi047-82517-12"}]: dispatch 2024-02-23T00:43:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:11.364318+0000 mon.b (mon.1) 103 : audit [INF] from='client.? 172.21.15.47:0/2137942802' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi047-82517-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:12.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:11.366954+0000 mon.a (mon.0) 978 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi047-82517-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:12.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:11.367225+0000 mon.a (mon.0) 979 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "pg_num": 4}]: dispatch 2024-02-23T00:43:12.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:11.477205+0000 mon.b (mon.1) 104 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:12.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:11.577987+0000 osd.6 (osd.6) 63 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:43:12.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:11.578072+0000 osd.6 (osd.6) 64 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:43:12.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:12.236692+0000 mon.a (mon.0) 980 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:43:12.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:12.239656+0000 mon.a (mon.0) 981 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:43:12.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:12.244351+0000 mon.a (mon.0) 982 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-4", "tierpool": "test-rados-api-smithi047-82465-4-cache"}]': finished 2024-02-23T00:43:12.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:12.244422+0000 mon.a (mon.0) 983 : audit [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi047-82504-6"}]': finished 2024-02-23T00:43:12.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:12.244478+0000 mon.a (mon.0) 984 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi047-82378-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:12.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:12.244544+0000 mon.a (mon.0) 985 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi047-82512-7"}]': finished 2024-02-23T00:43:12.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:12.244609+0000 mon.a (mon.0) 986 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi047-82394-15"}]': finished 2024-02-23T00:43:12.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:12.244796+0000 mon.a (mon.0) 987 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi047-82448-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:12.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:12.244983+0000 mon.a (mon.0) 988 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi047-82517-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:12.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:12.245083+0000 mon.a (mon.0) 989 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "pg_num": 4}]': finished 2024-02-23T00:43:12.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:12.245357+0000 mon.a (mon.0) 990 : cluster [DBG] osdmap e122: 8 total, 8 up, 8 in 2024-02-23T00:43:12.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:12.246127+0000 mon.c (mon.2) 55 : audit [INF] from='client.? 172.21.15.47:0/1006300819' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi047-82448-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-23T00:43:12.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:12.246544+0000 mon.c (mon.2) 56 : audit [INF] from='client.? 172.21.15.47:0/2783513670' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:12.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:12.247272+0000 mon.c (mon.2) 57 : audit [INF] from='client.? 172.21.15.47:0/4182485101' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:43:12.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:12.247738+0000 mon.c (mon.2) 58 : audit [INF] from='client.? 172.21.15.47:0/1094397307' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:12.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:12.247948+0000 mon.c (mon.2) 59 : audit [INF] from='client.? 172.21.15.47:0/1006300819' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi047-82448-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-23T00:43:12.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:12.248903+0000 mon.b (mon.1) 105 : audit [INF] from='client.? 172.21.15.47:0/2137942802' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi047-82517-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi047-82517-12"}]: dispatch 2024-02-23T00:43:12.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:12.276030+0000 mon.a (mon.0) 991 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi047-82517-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi047-82517-12"}]: dispatch 2024-02-23T00:43:12.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:12.276597+0000 mon.a (mon.0) 992 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:12.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:12.276924+0000 mon.a (mon.0) 993 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:43:12.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:12.277248+0000 mon.a (mon.0) 994 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:12.697 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:12.277532+0000 mon.a (mon.0) 995 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi047-82448-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-23T00:43:12.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:10.928547+0000 mon.b (mon.1) 102 : audit [INF] from='client.? 172.21.15.47:0/2137942802' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi047-82517-12"}]: dispatch 2024-02-23T00:43:12.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:10.933887+0000 mon.c (mon.2) 54 : audit [INF] from='client.? 172.21.15.47:0/4182485101' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "pg_num": 4}]: dispatch 2024-02-23T00:43:12.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:11.032400+0000 osd.7 (osd.7) 83 : cluster [DBG] 20.9 deep-scrub starts 2024-02-23T00:43:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:11.035858+0000 osd.7 (osd.7) 84 : cluster [DBG] 20.9 deep-scrub ok 2024-02-23T00:43:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:11.181234+0000 mgr.y (mgr.14517) 164 : cluster [DBG] pgmap v156: 563 pgs: 112 unknown, 1 clean+premerge+peered, 450 active+clean; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:43:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:11.359925+0000 mon.a (mon.0) 977 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi047-82517-12"}]: dispatch 2024-02-23T00:43:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:11.364318+0000 mon.b (mon.1) 103 : audit [INF] from='client.? 172.21.15.47:0/2137942802' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi047-82517-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:11.366954+0000 mon.a (mon.0) 978 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi047-82517-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:11.367225+0000 mon.a (mon.0) 979 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "pg_num": 4}]: dispatch 2024-02-23T00:43:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:11.477205+0000 mon.b (mon.1) 104 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:12.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:11.577987+0000 osd.6 (osd.6) 63 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:43:12.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:11.578072+0000 osd.6 (osd.6) 64 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:43:12.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:12.236692+0000 mon.a (mon.0) 980 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:43:12.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:12.239656+0000 mon.a (mon.0) 981 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:43:12.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:10.928547+0000 mon.b (mon.1) 102 : audit [INF] from='client.? 172.21.15.47:0/2137942802' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi047-82517-12"}]: dispatch 2024-02-23T00:43:12.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:10.933887+0000 mon.c (mon.2) 54 : audit [INF] from='client.? 172.21.15.47:0/4182485101' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "pg_num": 4}]: dispatch 2024-02-23T00:43:12.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:11.032400+0000 osd.7 (osd.7) 83 : cluster [DBG] 20.9 deep-scrub starts 2024-02-23T00:43:12.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:11.035858+0000 osd.7 (osd.7) 84 : cluster [DBG] 20.9 deep-scrub ok 2024-02-23T00:43:12.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:11.181234+0000 mgr.y (mgr.14517) 164 : cluster [DBG] pgmap v156: 563 pgs: 112 unknown, 1 clean+premerge+peered, 450 active+clean; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:43:12.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:11.359925+0000 mon.a (mon.0) 977 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi047-82517-12"}]: dispatch 2024-02-23T00:43:12.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:11.364318+0000 mon.b (mon.1) 103 : audit [INF] from='client.? 172.21.15.47:0/2137942802' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi047-82517-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:12.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:11.366954+0000 mon.a (mon.0) 978 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi047-82517-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:12.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:11.367225+0000 mon.a (mon.0) 979 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "pg_num": 4}]: dispatch 2024-02-23T00:43:12.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:11.477205+0000 mon.b (mon.1) 104 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:12.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:11.577987+0000 osd.6 (osd.6) 63 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:43:12.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:11.578072+0000 osd.6 (osd.6) 64 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:43:12.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:12.236692+0000 mon.a (mon.0) 980 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:43:12.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:12.239656+0000 mon.a (mon.0) 981 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:43:12.811 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:12.244351+0000 mon.a (mon.0) 982 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-4", "tierpool": "test-rados-api-smithi047-82465-4-cache"}]': finished 2024-02-23T00:43:12.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:12.244422+0000 mon.a (mon.0) 983 : audit [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi047-82504-6"}]': finished 2024-02-23T00:43:12.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:12.244478+0000 mon.a (mon.0) 984 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi047-82378-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:12.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:12.244544+0000 mon.a (mon.0) 985 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi047-82512-7"}]': finished 2024-02-23T00:43:12.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:12.244609+0000 mon.a (mon.0) 986 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi047-82394-15"}]': finished 2024-02-23T00:43:12.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:12.244796+0000 mon.a (mon.0) 987 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi047-82448-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:12.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:12.244983+0000 mon.a (mon.0) 988 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi047-82517-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:12.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:12.245083+0000 mon.a (mon.0) 989 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "pg_num": 4}]': finished 2024-02-23T00:43:12.813 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:12.245357+0000 mon.a (mon.0) 990 : cluster [DBG] osdmap e122: 8 total, 8 up, 8 in 2024-02-23T00:43:12.813 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:12.246127+0000 mon.c (mon.2) 55 : audit [INF] from='client.? 172.21.15.47:0/1006300819' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi047-82448-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-23T00:43:12.813 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:12.246544+0000 mon.c (mon.2) 56 : audit [INF] from='client.? 172.21.15.47:0/2783513670' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:12.813 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:12.247272+0000 mon.c (mon.2) 57 : audit [INF] from='client.? 172.21.15.47:0/4182485101' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:43:12.813 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:12.247738+0000 mon.c (mon.2) 58 : audit [INF] from='client.? 172.21.15.47:0/1094397307' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:12.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:12.244351+0000 mon.a (mon.0) 982 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-4", "tierpool": "test-rados-api-smithi047-82465-4-cache"}]': finished 2024-02-23T00:43:12.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:12.244422+0000 mon.a (mon.0) 983 : audit [INF] from='client.? 172.21.15.47:0/640613617' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi047-82504-6"}]': finished 2024-02-23T00:43:12.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:12.244478+0000 mon.a (mon.0) 984 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi047-82378-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:12.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:12.244544+0000 mon.a (mon.0) 985 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi047-82512-7"}]': finished 2024-02-23T00:43:12.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:12.244609+0000 mon.a (mon.0) 986 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi047-82394-15"}]': finished 2024-02-23T00:43:12.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:12.244796+0000 mon.a (mon.0) 987 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi047-82448-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:12.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:12.244983+0000 mon.a (mon.0) 988 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi047-82517-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:12.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:12.245083+0000 mon.a (mon.0) 989 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "pg_num": 4}]': finished 2024-02-23T00:43:12.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:12.245357+0000 mon.a (mon.0) 990 : cluster [DBG] osdmap e122: 8 total, 8 up, 8 in 2024-02-23T00:43:12.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:12.246127+0000 mon.c (mon.2) 55 : audit [INF] from='client.? 172.21.15.47:0/1006300819' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi047-82448-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-23T00:43:12.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:12.246544+0000 mon.c (mon.2) 56 : audit [INF] from='client.? 172.21.15.47:0/2783513670' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:12.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:12.247272+0000 mon.c (mon.2) 57 : audit [INF] from='client.? 172.21.15.47:0/4182485101' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:43:12.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:12.247738+0000 mon.c (mon.2) 58 : audit [INF] from='client.? 172.21.15.47:0/1094397307' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:12.817 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:12.247948+0000 mon.c (mon.2) 59 : audit [INF] from='client.? 172.21.15.47:0/1006300819' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi047-82448-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-23T00:43:12.817 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:12.248903+0000 mon.b (mon.1) 105 : audit [INF] from='client.? 172.21.15.47:0/2137942802' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi047-82517-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi047-82517-12"}]: dispatch 2024-02-23T00:43:12.817 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:12.276030+0000 mon.a (mon.0) 991 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi047-82517-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi047-82517-12"}]: dispatch 2024-02-23T00:43:12.817 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:12.276597+0000 mon.a (mon.0) 992 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:12.817 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:12.276924+0000 mon.a (mon.0) 993 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:43:12.817 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:12.277248+0000 mon.a (mon.0) 994 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:12.818 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:12.277532+0000 mon.a (mon.0) 995 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi047-82448-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-23T00:43:12.818 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:12.247948+0000 mon.c (mon.2) 59 : audit [INF] from='client.? 172.21.15.47:0/1006300819' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi047-82448-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-23T00:43:12.818 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:12.248903+0000 mon.b (mon.1) 105 : audit [INF] from='client.? 172.21.15.47:0/2137942802' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi047-82517-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi047-82517-12"}]: dispatch 2024-02-23T00:43:12.818 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:12.276030+0000 mon.a (mon.0) 991 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi047-82517-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi047-82517-12"}]: dispatch 2024-02-23T00:43:12.818 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:12.276597+0000 mon.a (mon.0) 992 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:12.818 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:12.276924+0000 mon.a (mon.0) 993 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:43:12.819 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:12.277248+0000 mon.a (mon.0) 994 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:12.819 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:12.277532+0000 mon.a (mon.0) 995 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi047-82448-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-23T00:43:14.056 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:12.304706+0000 mgr.y (mgr.14517) 165 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:43:14.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:12.478441+0000 mon.b (mon.1) 106 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:14.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:13.183513+0000 mon.a (mon.0) 996 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "29"}]: dispatch 2024-02-23T00:43:14.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:13.259015+0000 mon.c (mon.2) 60 : audit [INF] from='client.? 172.21.15.47:0/1119363424' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:14.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:13.479600+0000 mon.b (mon.1) 107 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:14.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:13.699213+0000 mon.a (mon.0) 997 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:14.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:12.304706+0000 mgr.y (mgr.14517) 165 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:43:14.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:12.478441+0000 mon.b (mon.1) 106 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:14.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:13.183513+0000 mon.a (mon.0) 996 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "29"}]: dispatch 2024-02-23T00:43:14.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:13.259015+0000 mon.c (mon.2) 60 : audit [INF] from='client.? 172.21.15.47:0/1119363424' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:14.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:13.479600+0000 mon.b (mon.1) 107 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:14.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:13.699213+0000 mon.a (mon.0) 997 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:14.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:12.304706+0000 mgr.y (mgr.14517) 165 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:43:14.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:12.478441+0000 mon.b (mon.1) 106 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:14.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:13.183513+0000 mon.a (mon.0) 996 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "29"}]: dispatch 2024-02-23T00:43:14.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:13.259015+0000 mon.c (mon.2) 60 : audit [INF] from='client.? 172.21.15.47:0/1119363424' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:14.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:13.479600+0000 mon.b (mon.1) 107 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:14.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:13.699213+0000 mon.a (mon.0) 997 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:14.801 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.2 dump_historic_ops 2024-02-23T00:43:14.806 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:43:14.810 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:14.810 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:43:14.911 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:43:14.938 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:15.553 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: Running main() from gmock_main.cc 2024-02-23T00:43:15.553 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2024-02-23T00:43:15.553 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [----------] Global test environment set-up. 2024-02-23T00:43:15.553 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2024-02-23T00:43:15.553 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: seed 82438 2024-02-23T00:43:15.553 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2024-02-23T00:43:15.553 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (189 ms) 2024-02-23T00:43:15.554 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2024-02-23T00:43:15.554 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (45 ms) 2024-02-23T00:43:15.554 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-02-23T00:43:15.554 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (1106 ms) 2024-02-23T00:43:15.554 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2024-02-23T00:43:15.554 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (1138 ms) 2024-02-23T00:43:15.554 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2024-02-23T00:43:15.554 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (3 ms) 2024-02-23T00:43:15.554 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2024-02-23T00:43:15.554 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (4 ms) 2024-02-23T00:43:15.554 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2024-02-23T00:43:15.554 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (5 ms) 2024-02-23T00:43:15.554 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2024-02-23T00:43:15.554 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (6 ms) 2024-02-23T00:43:15.555 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (2496 ms total) 2024-02-23T00:43:15.555 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: 2024-02-23T00:43:15.555 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2024-02-23T00:43:15.555 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2024-02-23T00:43:15.555 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (1075 ms) 2024-02-23T00:43:15.555 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2024-02-23T00:43:15.555 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (4 ms) 2024-02-23T00:43:15.555 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-02-23T00:43:15.555 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1009 ms) 2024-02-23T00:43:15.555 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-02-23T00:43:15.555 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1005 ms) 2024-02-23T00:43:15.555 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-02-23T00:43:15.555 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (4 ms) 2024-02-23T00:43:15.555 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2024-02-23T00:43:15.555 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (5 ms) 2024-02-23T00:43:15.556 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2024-02-23T00:43:15.556 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (5 ms) 2024-02-23T00:43:15.556 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2024-02-23T00:43:15.556 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (3 ms) 2024-02-23T00:43:15.556 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (3110 ms total) 2024-02-23T00:43:15.556 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: 2024-02-23T00:43:15.556 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [----------] Global test environment tear-down 2024-02-23T00:43:15.556 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (28346 ms total) 2024-02-23T00:43:15.556 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock_pp: [ PASSED ] 16 tests. 2024-02-23T00:43:15.560 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: Running main() from gmock_main.cc 2024-02-23T00:43:15.560 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2024-02-23T00:43:15.560 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [----------] Global test environment set-up. 2024-02-23T00:43:15.560 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [----------] 8 tests from LibRadosLock 2024-02-23T00:43:15.560 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2024-02-23T00:43:15.560 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (242 ms) 2024-02-23T00:43:15.560 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2024-02-23T00:43:15.560 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ OK ] LibRadosLock.LockShared (7 ms) 2024-02-23T00:43:15.561 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2024-02-23T00:43:15.561 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (1006 ms) 2024-02-23T00:43:15.561 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2024-02-23T00:43:15.561 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1046 ms) 2024-02-23T00:43:15.561 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2024-02-23T00:43:15.561 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (42 ms) 2024-02-23T00:43:15.561 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2024-02-23T00:43:15.561 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ OK ] LibRadosLock.Unlock (20 ms) 2024-02-23T00:43:15.561 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2024-02-23T00:43:15.561 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (8 ms) 2024-02-23T00:43:15.561 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2024-02-23T00:43:15.561 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (3 ms) 2024-02-23T00:43:15.561 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [----------] 8 tests from LibRadosLock (2374 ms total) 2024-02-23T00:43:15.561 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: 2024-02-23T00:43:15.562 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2024-02-23T00:43:15.562 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2024-02-23T00:43:15.562 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (1063 ms) 2024-02-23T00:43:15.562 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2024-02-23T00:43:15.562 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (4 ms) 2024-02-23T00:43:15.562 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2024-02-23T00:43:15.562 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1009 ms) 2024-02-23T00:43:15.562 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2024-02-23T00:43:15.562 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1006 ms) 2024-02-23T00:43:15.562 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2024-02-23T00:43:15.562 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (5 ms) 2024-02-23T00:43:15.562 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2024-02-23T00:43:15.562 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (4 ms) 2024-02-23T00:43:15.562 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2024-02-23T00:43:15.563 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (6 ms) 2024-02-23T00:43:15.563 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2024-02-23T00:43:15.563 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (3 ms) 2024-02-23T00:43:15.563 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (3100 ms total) 2024-02-23T00:43:15.563 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: 2024-02-23T00:43:15.563 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [----------] Global test environment tear-down 2024-02-23T00:43:15.563 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (28331 ms total) 2024-02-23T00:43:15.563 INFO:tasks.workunit.client.0.smithi047.stdout: api_lock: [ PASSED ] 16 tests. 2024-02-23T00:43:15.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:13.181862+0000 mgr.y (mgr.14517) 166 : cluster [DBG] pgmap v158: 630 pgs: 68 unknown, 32 creating+peering, 1 clean+premerge+peered, 529 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 7.1 KiB/s rd, 5.1 KiB/s wr, 20 op/s 2024-02-23T00:43:15.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:13.699317+0000 mon.a (mon.0) 998 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:43:15.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:13.699415+0000 mon.a (mon.0) 999 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:15.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:13.699550+0000 mon.a (mon.0) 1000 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi047-82448-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-23T00:43:15.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:13.699912+0000 mon.a (mon.0) 1001 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "29"}]': finished 2024-02-23T00:43:15.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:13.699988+0000 mon.a (mon.0) 1002 : cluster [DBG] osdmap e123: 8 total, 8 up, 8 in 2024-02-23T00:43:15.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:13.700048+0000 mon.c (mon.2) 61 : audit [INF] from='client.? 172.21.15.47:0/4182485101' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6"}]: dispatch 2024-02-23T00:43:15.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:13.702019+0000 mon.c (mon.2) 62 : audit [INF] from='client.? 172.21.15.47:0/1006300819' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi047-82448-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-23T00:43:15.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:13.702212+0000 mon.a (mon.0) 1003 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:15.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:13.702596+0000 mon.a (mon.0) 1004 : audit [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi047-82438-10"}]: dispatch 2024-02-23T00:43:15.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:13.712989+0000 mon.a (mon.0) 1005 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6"}]: dispatch 2024-02-23T00:43:15.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:13.713295+0000 mon.a (mon.0) 1006 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi047-82448-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-23T00:43:15.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:14.161623+0000 mon.a (mon.0) 1007 : cluster [WRN] pool 'PoolQuotaPP_smithi047-82386-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-23T00:43:15.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:14.162543+0000 mon.a (mon.0) 1008 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-23T00:43:15.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:14.168768+0000 mon.a (mon.0) 1009 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi047-82517-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi047-82517-12"}]': finished 2024-02-23T00:43:15.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:14.168865+0000 mon.a (mon.0) 1010 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi047-82428-10"}]': finished 2024-02-23T00:43:15.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:14.168929+0000 mon.a (mon.0) 1011 : audit [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi047-82438-10"}]': finished 2024-02-23T00:43:15.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:14.169039+0000 mon.a (mon.0) 1012 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6"}]': finished 2024-02-23T00:43:15.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:14.169105+0000 mon.a (mon.0) 1013 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi047-82448-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-23T00:43:15.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:14.178682+0000 mon.c (mon.2) 63 : audit [INF] from='client.? 172.21.15.47:0/4182485101' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "mode": "writeback"}]: dispatch 2024-02-23T00:43:15.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:14.181641+0000 mon.c (mon.2) 64 : audit [INF] from='client.? 172.21.15.47:0/4112168838' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi047-82378-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:15.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:14.182892+0000 mon.c (mon.2) 65 : audit [INF] from='client.? 172.21.15.47:0/1119363424' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:15.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:14.189135+0000 mon.c (mon.2) 66 : audit [INF] from='client.? 172.21.15.47:0/1006300819' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi047-82448-1","app":"app1","key":"key1"}]: dispatch 2024-02-23T00:43:15.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:14.480642+0000 mon.b (mon.1) 108 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:15.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:14.617606+0000 mon.a (mon.0) 1014 : cluster [DBG] osdmap e124: 8 total, 8 up, 8 in 2024-02-23T00:43:15.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:14.619265+0000 mon.a (mon.0) 1015 : audit [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi047-82438-10"}]: dispatch 2024-02-23T00:43:15.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:14.623973+0000 mon.a (mon.0) 1016 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "mode": "writeback"}]: dispatch 2024-02-23T00:43:15.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:14.624552+0000 mon.a (mon.0) 1017 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi047-82378-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:15.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:14.625346+0000 mon.a (mon.0) 1018 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:15.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:14.625564+0000 mon.a (mon.0) 1019 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi047-82448-1","app":"app1","key":"key1"}]: dispatch 2024-02-23T00:43:15.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:13.181862+0000 mgr.y (mgr.14517) 166 : cluster [DBG] pgmap v158: 630 pgs: 68 unknown, 32 creating+peering, 1 clean+premerge+peered, 529 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 7.1 KiB/s rd, 5.1 KiB/s wr, 20 op/s 2024-02-23T00:43:15.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:13.699317+0000 mon.a (mon.0) 998 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:43:15.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:13.699415+0000 mon.a (mon.0) 999 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:15.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:13.699550+0000 mon.a (mon.0) 1000 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi047-82448-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-23T00:43:15.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:13.699912+0000 mon.a (mon.0) 1001 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "29"}]': finished 2024-02-23T00:43:15.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:13.699988+0000 mon.a (mon.0) 1002 : cluster [DBG] osdmap e123: 8 total, 8 up, 8 in 2024-02-23T00:43:15.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:13.700048+0000 mon.c (mon.2) 61 : audit [INF] from='client.? 172.21.15.47:0/4182485101' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6"}]: dispatch 2024-02-23T00:43:15.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:13.702019+0000 mon.c (mon.2) 62 : audit [INF] from='client.? 172.21.15.47:0/1006300819' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi047-82448-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-23T00:43:15.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:13.702212+0000 mon.a (mon.0) 1003 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:15.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:13.702596+0000 mon.a (mon.0) 1004 : audit [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi047-82438-10"}]: dispatch 2024-02-23T00:43:15.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:13.712989+0000 mon.a (mon.0) 1005 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6"}]: dispatch 2024-02-23T00:43:15.811 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:13.713295+0000 mon.a (mon.0) 1006 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi047-82448-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-23T00:43:15.811 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:14.161623+0000 mon.a (mon.0) 1007 : cluster [WRN] pool 'PoolQuotaPP_smithi047-82386-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-23T00:43:15.811 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:14.162543+0000 mon.a (mon.0) 1008 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-23T00:43:15.811 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:14.168768+0000 mon.a (mon.0) 1009 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi047-82517-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi047-82517-12"}]': finished 2024-02-23T00:43:15.811 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:14.168865+0000 mon.a (mon.0) 1010 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi047-82428-10"}]': finished 2024-02-23T00:43:15.811 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:14.168929+0000 mon.a (mon.0) 1011 : audit [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi047-82438-10"}]': finished 2024-02-23T00:43:15.811 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:14.169039+0000 mon.a (mon.0) 1012 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6"}]': finished 2024-02-23T00:43:15.811 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:14.169105+0000 mon.a (mon.0) 1013 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi047-82448-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-23T00:43:15.811 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:14.178682+0000 mon.c (mon.2) 63 : audit [INF] from='client.? 172.21.15.47:0/4182485101' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "mode": "writeback"}]: dispatch 2024-02-23T00:43:15.811 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:14.181641+0000 mon.c (mon.2) 64 : audit [INF] from='client.? 172.21.15.47:0/4112168838' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi047-82378-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:15.811 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:14.182892+0000 mon.c (mon.2) 65 : audit [INF] from='client.? 172.21.15.47:0/1119363424' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:15.811 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:14.189135+0000 mon.c (mon.2) 66 : audit [INF] from='client.? 172.21.15.47:0/1006300819' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi047-82448-1","app":"app1","key":"key1"}]: dispatch 2024-02-23T00:43:15.811 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:14.480642+0000 mon.b (mon.1) 108 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:15.811 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:14.617606+0000 mon.a (mon.0) 1014 : cluster [DBG] osdmap e124: 8 total, 8 up, 8 in 2024-02-23T00:43:15.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:14.619265+0000 mon.a (mon.0) 1015 : audit [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi047-82438-10"}]: dispatch 2024-02-23T00:43:15.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:14.623973+0000 mon.a (mon.0) 1016 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "mode": "writeback"}]: dispatch 2024-02-23T00:43:15.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:14.624552+0000 mon.a (mon.0) 1017 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi047-82378-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:15.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:14.625346+0000 mon.a (mon.0) 1018 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:15.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:14.625564+0000 mon.a (mon.0) 1019 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi047-82448-1","app":"app1","key":"key1"}]: dispatch 2024-02-23T00:43:15.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:13.181862+0000 mgr.y (mgr.14517) 166 : cluster [DBG] pgmap v158: 630 pgs: 68 unknown, 32 creating+peering, 1 clean+premerge+peered, 529 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 7.1 KiB/s rd, 5.1 KiB/s wr, 20 op/s 2024-02-23T00:43:15.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:13.699317+0000 mon.a (mon.0) 998 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:43:15.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:13.699415+0000 mon.a (mon.0) 999 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:15.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:13.699550+0000 mon.a (mon.0) 1000 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi047-82448-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-23T00:43:15.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:13.699912+0000 mon.a (mon.0) 1001 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "29"}]': finished 2024-02-23T00:43:15.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:13.699988+0000 mon.a (mon.0) 1002 : cluster [DBG] osdmap e123: 8 total, 8 up, 8 in 2024-02-23T00:43:15.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:13.700048+0000 mon.c (mon.2) 61 : audit [INF] from='client.? 172.21.15.47:0/4182485101' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6"}]: dispatch 2024-02-23T00:43:15.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:13.702019+0000 mon.c (mon.2) 62 : audit [INF] from='client.? 172.21.15.47:0/1006300819' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi047-82448-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-23T00:43:15.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:13.702212+0000 mon.a (mon.0) 1003 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:15.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:13.702596+0000 mon.a (mon.0) 1004 : audit [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi047-82438-10"}]: dispatch 2024-02-23T00:43:15.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:13.712989+0000 mon.a (mon.0) 1005 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6"}]: dispatch 2024-02-23T00:43:15.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:13.713295+0000 mon.a (mon.0) 1006 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi047-82448-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-23T00:43:15.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:14.161623+0000 mon.a (mon.0) 1007 : cluster [WRN] pool 'PoolQuotaPP_smithi047-82386-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-23T00:43:15.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:14.162543+0000 mon.a (mon.0) 1008 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-23T00:43:15.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:14.168768+0000 mon.a (mon.0) 1009 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi047-82517-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi047-82517-12"}]': finished 2024-02-23T00:43:15.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:14.168865+0000 mon.a (mon.0) 1010 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi047-82428-10"}]': finished 2024-02-23T00:43:15.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:14.168929+0000 mon.a (mon.0) 1011 : audit [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi047-82438-10"}]': finished 2024-02-23T00:43:15.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:14.169039+0000 mon.a (mon.0) 1012 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6"}]': finished 2024-02-23T00:43:15.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:14.169105+0000 mon.a (mon.0) 1013 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi047-82448-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-23T00:43:15.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:14.178682+0000 mon.c (mon.2) 63 : audit [INF] from='client.? 172.21.15.47:0/4182485101' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "mode": "writeback"}]: dispatch 2024-02-23T00:43:15.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:14.181641+0000 mon.c (mon.2) 64 : audit [INF] from='client.? 172.21.15.47:0/4112168838' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi047-82378-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:15.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:14.182892+0000 mon.c (mon.2) 65 : audit [INF] from='client.? 172.21.15.47:0/1119363424' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:15.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:14.189135+0000 mon.c (mon.2) 66 : audit [INF] from='client.? 172.21.15.47:0/1006300819' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi047-82448-1","app":"app1","key":"key1"}]: dispatch 2024-02-23T00:43:15.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:14.480642+0000 mon.b (mon.1) 108 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:15.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:14.617606+0000 mon.a (mon.0) 1014 : cluster [DBG] osdmap e124: 8 total, 8 up, 8 in 2024-02-23T00:43:15.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:14.619265+0000 mon.a (mon.0) 1015 : audit [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi047-82438-10"}]: dispatch 2024-02-23T00:43:15.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:14.623973+0000 mon.a (mon.0) 1016 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "mode": "writeback"}]: dispatch 2024-02-23T00:43:15.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:14.624552+0000 mon.a (mon.0) 1017 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi047-82378-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:15.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:14.625346+0000 mon.a (mon.0) 1018 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi047-82428-10"}]: dispatch 2024-02-23T00:43:15.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:14.625564+0000 mon.a (mon.0) 1019 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi047-82448-1","app":"app1","key":"key1"}]: dispatch 2024-02-23T00:43:16.536 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:15.182757+0000 mgr.y (mgr.14517) 167 : cluster [DBG] pgmap v161: 462 pgs: 44 unknown, 1 clean+premerge+peered, 417 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.6 KiB/s rd, 4.8 KiB/s wr, 3 op/s 2024-02-23T00:43:16.536 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:15.481608+0000 mon.b (mon.1) 109 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:16.536 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:15.523791+0000 mon.a (mon.0) 1020 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:43:16.536 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:15.536610+0000 mon.a (mon.0) 1021 : audit [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi047-82438-10"}]': finished 2024-02-23T00:43:16.536 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:15.536788+0000 mon.a (mon.0) 1022 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "mode": "writeback"}]': finished 2024-02-23T00:43:16.536 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:15.536952+0000 mon.a (mon.0) 1023 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi047-82378-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:16.536 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:15.537047+0000 mon.a (mon.0) 1024 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi047-82428-10"}]': finished 2024-02-23T00:43:16.536 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:15.537126+0000 mon.a (mon.0) 1025 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi047-82448-1","app":"app1","key":"key1"}]': finished 2024-02-23T00:43:16.537 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:15.537243+0000 mon.a (mon.0) 1026 : cluster [DBG] osdmap e125: 8 total, 8 up, 8 in 2024-02-23T00:43:16.537 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:15.555726+0000 mon.c (mon.2) 67 : audit [INF] from='client.? 172.21.15.47:0/3285306023' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:16.537 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:15.557793+0000 mon.c (mon.2) 68 : audit [INF] from='client.? 172.21.15.47:0/1978716317' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:16.537 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:15.558201+0000 mon.a (mon.0) 1027 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:16.537 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:15.558497+0000 mon.a (mon.0) 1028 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:16.537 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:15.569750+0000 mon.a (mon.0) 1029 : audit [INF] from='client.? 172.21.15.47:0/328451096' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi047-82487-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:16.537 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:15.620345+0000 mon.a (mon.0) 1030 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:43:16.537 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:16.482499+0000 mon.b (mon.1) 110 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:16.701 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:43:16.704 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:16.704 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:43:16.805 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:43:16.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:15.182757+0000 mgr.y (mgr.14517) 167 : cluster [DBG] pgmap v161: 462 pgs: 44 unknown, 1 clean+premerge+peered, 417 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.6 KiB/s rd, 4.8 KiB/s wr, 3 op/s 2024-02-23T00:43:16.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:15.481608+0000 mon.b (mon.1) 109 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:16.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:15.523791+0000 mon.a (mon.0) 1020 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:43:16.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:15.536610+0000 mon.a (mon.0) 1021 : audit [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi047-82438-10"}]': finished 2024-02-23T00:43:16.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:15.536788+0000 mon.a (mon.0) 1022 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "mode": "writeback"}]': finished 2024-02-23T00:43:16.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:15.536952+0000 mon.a (mon.0) 1023 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi047-82378-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:16.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:15.537047+0000 mon.a (mon.0) 1024 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi047-82428-10"}]': finished 2024-02-23T00:43:16.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:15.537126+0000 mon.a (mon.0) 1025 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi047-82448-1","app":"app1","key":"key1"}]': finished 2024-02-23T00:43:16.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:15.537243+0000 mon.a (mon.0) 1026 : cluster [DBG] osdmap e125: 8 total, 8 up, 8 in 2024-02-23T00:43:16.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:15.555726+0000 mon.c (mon.2) 67 : audit [INF] from='client.? 172.21.15.47:0/3285306023' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:16.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:15.557793+0000 mon.c (mon.2) 68 : audit [INF] from='client.? 172.21.15.47:0/1978716317' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:16.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:15.558201+0000 mon.a (mon.0) 1027 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:16.813 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:15.558497+0000 mon.a (mon.0) 1028 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:16.813 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:15.569750+0000 mon.a (mon.0) 1029 : audit [INF] from='client.? 172.21.15.47:0/328451096' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi047-82487-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:16.813 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:15.620345+0000 mon.a (mon.0) 1030 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:43:16.813 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:16.482499+0000 mon.b (mon.1) 110 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:16.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.846+0000 7f26981b7700 1 -- 172.21.15.47:0/4106907749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26900fcba0 msgr2=0x7f26900f6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:16.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.846+0000 7f26981b7700 1 --2- 172.21.15.47:0/4106907749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26900fcba0 0x7f26900f6450 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f26840048b0 tx=0x7f268400adc0).stop 2024-02-23T00:43:16.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.846+0000 7f26981b7700 1 -- 172.21.15.47:0/4106907749 shutdown_connections 2024-02-23T00:43:16.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.846+0000 7f26981b7700 1 --2- 172.21.15.47:0/4106907749 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f26900f6b10 0x7f26901030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:16.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.846+0000 7f26981b7700 1 --2- 172.21.15.47:0/4106907749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26900fc240 0x7f26900fc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:16.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.846+0000 7f26981b7700 1 --2- 172.21.15.47:0/4106907749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26900fcba0 0x7f26900f6450 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:16.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.846+0000 7f26981b7700 1 -- 172.21.15.47:0/4106907749 >> 172.21.15.47:0/4106907749 conn(0x7f269006c370 msgr2=0x7f269006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:43:16.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.855+0000 7f26981b7700 1 -- 172.21.15.47:0/4106907749 shutdown_connections 2024-02-23T00:43:16.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.855+0000 7f26981b7700 1 -- 172.21.15.47:0/4106907749 wait complete. 2024-02-23T00:43:16.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.856+0000 7f26981b7700 1 librados: starting msgr at 2024-02-23T00:43:16.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.856+0000 7f26981b7700 1 librados: starting objecter 2024-02-23T00:43:16.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.857+0000 7f26981b7700 1 Processor -- start 2024-02-23T00:43:16.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.857+0000 7f26981b7700 1 -- start start 2024-02-23T00:43:16.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.857+0000 7f26981b7700 1 librados: setting wanted keys 2024-02-23T00:43:16.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.857+0000 7f26981b7700 1 librados: calling monclient init 2024-02-23T00:43:16.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.857+0000 7f26981b7700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26900f6b10 0x7f26900fd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:16.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.857+0000 7f26981b7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26900fc240 0x7f26900fd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:16.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.857+0000 7f26981b7700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f26900fcba0 0x7f26900fdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:16.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.857+0000 7f26981b7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26901012d0 con 0x7f26900fc240 2024-02-23T00:43:16.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.857+0000 7f26981b7700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269011ab20 con 0x7f26900f6b10 2024-02-23T00:43:16.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.857+0000 7f26981b7700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f269011ace0 con 0x7f26900fcba0 2024-02-23T00:43:16.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.857+0000 7f2695752700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26900fc240 0x7f26900fd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:16.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.857+0000 7f2695752700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26900fc240 0x7f26900fd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35074/0 (socket says 172.21.15.47:35074) 2024-02-23T00:43:16.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.857+0000 7f2695752700 1 -- 172.21.15.47:0/3905711187 learned_addr learned my addr 172.21.15.47:0/3905711187 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:43:16.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.857+0000 7f2695f53700 1 --2- 172.21.15.47:0/3905711187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26900f6b10 0x7f26900fd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:16.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.857+0000 7f2696754700 1 --2- 172.21.15.47:0/3905711187 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f26900fcba0 0x7f26900fdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:16.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.858+0000 7f2695f53700 1 -- 172.21.15.47:0/3905711187 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f26900fcba0 msgr2=0x7f26900fdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:16.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.858+0000 7f2695f53700 1 --2- 172.21.15.47:0/3905711187 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f26900fcba0 0x7f26900fdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:16.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.858+0000 7f2695f53700 1 -- 172.21.15.47:0/3905711187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26900fc240 msgr2=0x7f26900fd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:16.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.858+0000 7f2695f53700 1 --2- 172.21.15.47:0/3905711187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26900fc240 0x7f26900fd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:16.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.858+0000 7f2695f53700 1 -- 172.21.15.47:0/3905711187 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268c005040 con 0x7f26900f6b10 2024-02-23T00:43:16.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.858+0000 7f2695f53700 1 --2- 172.21.15.47:0/3905711187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26900f6b10 0x7f26900fd200 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f268c009220 tx=0x7f268c009cb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:16.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.858+0000 7f2682ffd700 10 client.?.objecter ms_handle_connect 0x7f26900f6b10 2024-02-23T00:43:16.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.858+0000 7f2682ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:43:16.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.858+0000 7f2682ffd700 1 -- 172.21.15.47:0/3905711187 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c00e030 con 0x7f26900f6b10 2024-02-23T00:43:16.862 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:16.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.858+0000 7f2682ffd700 1 -- 172.21.15.47:0/3905711187 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f268c00f040 con 0x7f26900f6b10 2024-02-23T00:43:16.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.858+0000 7f2682ffd700 1 -- 172.21.15.47:0/3905711187 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c014720 con 0x7f26900f6b10 2024-02-23T00:43:16.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.858+0000 7f26981b7700 1 -- 172.21.15.47:0/3905711187 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2684005040 con 0x7f26900f6b10 2024-02-23T00:43:16.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.858+0000 7f26981b7700 10 client.25618.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:43:16.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.858+0000 7f26981b7700 1 -- 172.21.15.47:0/3905711187 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f269011b240 con 0x7f26900f6b10 2024-02-23T00:43:16.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.858+0000 7f26981b7700 1 librados: init done 2024-02-23T00:43:16.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.859+0000 7f26981b7700 1 -- 172.21.15.47:0/3905711187 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f269005fa40 con 0x7f26900f6b10 2024-02-23T00:43:16.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.861+0000 7f2682ffd700 1 -- 172.21.15.47:0/3905711187 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f268c014b50 con 0x7f26900f6b10 2024-02-23T00:43:16.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.861+0000 7f2682ffd700 1 --2- 172.21.15.47:0/3905711187 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f267c062d60 0x7f267c065220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:16.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.861+0000 7f2682ffd700 1 -- 172.21.15.47:0/3905711187 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(126..126 src has 48..126) v4 ==== 12691+0+0 (secure 0 0 0) 0x7f268c07e200 con 0x7f26900f6b10 2024-02-23T00:43:16.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.861+0000 7f2682ffd700 10 client.25618.objecter ms_dispatch 0x7f2690001550 osd_map(126..126 src has 48..126) v4 2024-02-23T00:43:16.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.861+0000 7f2682ffd700 3 client.25618.objecter handle_osd_map got epochs [126,126] > 0 2024-02-23T00:43:16.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.861+0000 7f2682ffd700 3 client.25618.objecter handle_osd_map decoding full epoch 126 2024-02-23T00:43:16.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.861+0000 7f2695752700 1 --2- 172.21.15.47:0/3905711187 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f267c062d60 0x7f267c065220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:16.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.861+0000 7f2682ffd700 10 client.25618.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:43:16.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.861+0000 7f2682ffd700 1 -- 172.21.15.47:0/3905711187 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=127}) v3 -- 0x7f267c076c30 con 0x7f26900f6b10 2024-02-23T00:43:16.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.861+0000 7f2682ffd700 20 client.25618.objecter dump_active .. 0 homeless 2024-02-23T00:43:16.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.861+0000 7f2682ffd700 10 client.25618.objecter ms_handle_connect 0x7f267c062d60 2024-02-23T00:43:16.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.861+0000 7f2695752700 1 --2- 172.21.15.47:0/3905711187 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f267c062d60 0x7f267c065220 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f2684005010 tx=0x7f2684017040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:16.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:16.865+0000 7f2682ffd700 1 -- 172.21.15.47:0/3905711187 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f268c04ca80 con 0x7f26900f6b10 2024-02-23T00:43:16.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:15.182757+0000 mgr.y (mgr.14517) 167 : cluster [DBG] pgmap v161: 462 pgs: 44 unknown, 1 clean+premerge+peered, 417 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.6 KiB/s rd, 4.8 KiB/s wr, 3 op/s 2024-02-23T00:43:16.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:15.481608+0000 mon.b (mon.1) 109 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:16.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:15.523791+0000 mon.a (mon.0) 1020 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:43:16.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:15.536610+0000 mon.a (mon.0) 1021 : audit [INF] from='client.? 172.21.15.47:0/4015522315' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi047-82438-10"}]': finished 2024-02-23T00:43:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:15.536788+0000 mon.a (mon.0) 1022 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "mode": "writeback"}]': finished 2024-02-23T00:43:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:15.536952+0000 mon.a (mon.0) 1023 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi047-82378-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:15.537047+0000 mon.a (mon.0) 1024 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi047-82428-10"}]': finished 2024-02-23T00:43:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:15.537126+0000 mon.a (mon.0) 1025 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi047-82448-1","app":"app1","key":"key1"}]': finished 2024-02-23T00:43:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:15.537243+0000 mon.a (mon.0) 1026 : cluster [DBG] osdmap e125: 8 total, 8 up, 8 in 2024-02-23T00:43:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:15.555726+0000 mon.c (mon.2) 67 : audit [INF] from='client.? 172.21.15.47:0/3285306023' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:15.557793+0000 mon.c (mon.2) 68 : audit [INF] from='client.? 172.21.15.47:0/1978716317' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:15.558201+0000 mon.a (mon.0) 1027 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:15.558497+0000 mon.a (mon.0) 1028 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:15.569750+0000 mon.a (mon.0) 1029 : audit [INF] from='client.? 172.21.15.47:0/328451096' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi047-82487-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:15.620345+0000 mon.a (mon.0) 1030 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:43:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:16.482499+0000 mon.b (mon.1) 110 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:17.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:17.150+0000 7f26981b7700 1 -- 172.21.15.47:0/3905711187 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7f269005ee80 con 0x7f26900f6b10 2024-02-23T00:43:17.267 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:43:17.275 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:17.275 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:43:17.289 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:43:17.376 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:43:17.442 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:17.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:17.765+0000 7f2682ffd700 1 -- 172.21.15.47:0/3905711187 <== mon.1 v2:172.21.15.196:3300/0 7 ==== osd_map(127..127 src has 48..127) v4 ==== 1850+0+0 (secure 0 0 0) 0x7f268c0506e0 con 0x7f26900f6b10 2024-02-23T00:43:17.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:17.765+0000 7f2682ffd700 10 client.25618.objecter ms_dispatch 0x7f2690001550 osd_map(127..127 src has 48..127) v4 2024-02-23T00:43:17.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:17.765+0000 7f2682ffd700 3 client.25618.objecter handle_osd_map got epochs [127,127] > 126 2024-02-23T00:43:17.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:17.765+0000 7f2682ffd700 3 client.25618.objecter handle_osd_map decoding incremental epoch 127 2024-02-23T00:43:17.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:17.765+0000 7f2682ffd700 10 client.25618.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:43:17.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:17.765+0000 7f2682ffd700 1 -- 172.21.15.47:0/3905711187 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=128}) v3 -- 0x7f267c06b7d0 con 0x7f26900f6b10 2024-02-23T00:43:17.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:17.765+0000 7f2682ffd700 20 client.25618.objecter dump_active .. 0 homeless 2024-02-23T00:43:17.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:17.770+0000 7f2682ffd700 1 -- 172.21.15.47:0/3905711187 <== mon.1 v2:172.21.15.196:3300/0 8 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v127) v1 ==== 97+0+0 (secure 0 0 0) 0x7f268c0500a0 con 0x7f26900f6b10 2024-02-23T00:43:17.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:17.781+0000 7f26981b7700 10 librados: watch_flush enter 2024-02-23T00:43:17.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:17.786+0000 7f26981b7700 10 librados: watch_flush exit 2024-02-23T00:43:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:17.786+0000 7f26981b7700 20 client.25618.objecter shutdown clearing up homeless session... 2024-02-23T00:43:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:17.786+0000 7f26981b7700 10 client.25618.objecter successfully canceled tick 2024-02-23T00:43:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:17.786+0000 7f26981b7700 1 -- 172.21.15.47:0/3905711187 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f267c062d60 msgr2=0x7f267c065220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:17.786+0000 7f26981b7700 1 --2- 172.21.15.47:0/3905711187 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f267c062d60 0x7f267c065220 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f2684005010 tx=0x7f2684017040).stop 2024-02-23T00:43:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:17.786+0000 7f26981b7700 1 -- 172.21.15.47:0/3905711187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26900f6b10 msgr2=0x7f26900fd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:17.786+0000 7f26981b7700 1 --2- 172.21.15.47:0/3905711187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26900f6b10 0x7f26900fd200 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f268c009220 tx=0x7f268c009cb0).stop 2024-02-23T00:43:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:43:17.804 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:17.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:17.800+0000 7f26981b7700 1 -- 172.21.15.47:0/3905711187 shutdown_connections 2024-02-23T00:43:17.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:17.800+0000 7f26981b7700 1 --2- 172.21.15.47:0/3905711187 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f267c062d60 0x7f267c065220 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:17.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:17.800+0000 7f26981b7700 1 --2- 172.21.15.47:0/3905711187 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f26900fcba0 0x7f26900fdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:17.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:17.800+0000 7f26981b7700 1 --2- 172.21.15.47:0/3905711187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26900f6b10 0x7f26900fd200 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:17.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:17.800+0000 7f26981b7700 1 --2- 172.21.15.47:0/3905711187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26900fc240 0x7f26900fd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:17.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:17.800+0000 7f26981b7700 1 -- 172.21.15.47:0/3905711187 >> 172.21.15.47:0/3905711187 conn(0x7f269006c370 msgr2=0x7f26900f52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:43:17.805 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:43:17.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:17.813+0000 7f26981b7700 1 -- 172.21.15.47:0/3905711187 shutdown_connections 2024-02-23T00:43:17.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:17.819+0000 7f26981b7700 1 -- 172.21.15.47:0/3905711187 wait complete. 2024-02-23T00:43:17.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:17.820+0000 7f26981b7700 1 librados: shutdown 2024-02-23T00:43:17.843 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is set 2024-02-23T00:43:17.906 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:43:17.932 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:17.951 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:43:17.953 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:17.954 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:43:18.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:16.547998+0000 mon.a (mon.0) 1031 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:18.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:16.548076+0000 mon.a (mon.0) 1032 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:18.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:16.548136+0000 mon.a (mon.0) 1033 : audit [INF] from='client.? 172.21.15.47:0/328451096' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi047-82487-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:18.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:16.548292+0000 mon.a (mon.0) 1034 : cluster [DBG] osdmap e126: 8 total, 8 up, 8 in 2024-02-23T00:43:18.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:16.551492+0000 mon.b (mon.1) 111 : audit [INF] from='client.? 172.21.15.47:0/2137942802' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi047-82517-12"}]: dispatch 2024-02-23T00:43:18.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:16.551883+0000 mon.a (mon.0) 1035 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi047-82517-12"}]: dispatch 2024-02-23T00:43:18.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:16.553018+0000 mon.a (mon.0) 1036 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:18.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:17.151764+0000 mon.b (mon.1) 112 : audit [INF] from='client.? 172.21.15.47:0/3905711187' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:43:18.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:17.180199+0000 mon.a (mon.0) 1037 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:43:18.043 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:16.547998+0000 mon.a (mon.0) 1031 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:18.043 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:16.548076+0000 mon.a (mon.0) 1032 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:18.043 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:16.548136+0000 mon.a (mon.0) 1033 : audit [INF] from='client.? 172.21.15.47:0/328451096' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi047-82487-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:18.043 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:16.548292+0000 mon.a (mon.0) 1034 : cluster [DBG] osdmap e126: 8 total, 8 up, 8 in 2024-02-23T00:43:18.043 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:16.551492+0000 mon.b (mon.1) 111 : audit [INF] from='client.? 172.21.15.47:0/2137942802' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi047-82517-12"}]: dispatch 2024-02-23T00:43:18.043 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:16.551883+0000 mon.a (mon.0) 1035 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi047-82517-12"}]: dispatch 2024-02-23T00:43:18.043 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:16.553018+0000 mon.a (mon.0) 1036 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:18.044 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:17.151764+0000 mon.b (mon.1) 112 : audit [INF] from='client.? 172.21.15.47:0/3905711187' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:43:18.044 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:17.180199+0000 mon.a (mon.0) 1037 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:43:18.054 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:43:18.107 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:18.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:16.547998+0000 mon.a (mon.0) 1031 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:18.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:16.548076+0000 mon.a (mon.0) 1032 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:18.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:16.548136+0000 mon.a (mon.0) 1033 : audit [INF] from='client.? 172.21.15.47:0/328451096' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi047-82487-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:18.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:16.548292+0000 mon.a (mon.0) 1034 : cluster [DBG] osdmap e126: 8 total, 8 up, 8 in 2024-02-23T00:43:18.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:16.551492+0000 mon.b (mon.1) 111 : audit [INF] from='client.? 172.21.15.47:0/2137942802' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi047-82517-12"}]: dispatch 2024-02-23T00:43:18.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:16.551883+0000 mon.a (mon.0) 1035 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi047-82517-12"}]: dispatch 2024-02-23T00:43:18.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:16.553018+0000 mon.a (mon.0) 1036 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:18.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:17.151764+0000 mon.b (mon.1) 112 : audit [INF] from='client.? 172.21.15.47:0/3905711187' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:43:18.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:17.180199+0000 mon.a (mon.0) 1037 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:43:18.386 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:43:18.391 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:18.391 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:43:18.492 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:43:18.518 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:18.537 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:43:18.539 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:18.540 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:43:18.622 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:43:18.640 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:43:18.698 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: Running main() from gmock_main.cc 2024-02-23T00:43:18.698 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2024-02-23T00:43:18.698 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2024-02-23T00:43:18.698 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2024-02-23T00:43:18.698 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-02-23T00:43:18.698 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: notify 2024-02-23T00:43:18.698 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (1281 ms) 2024-02-23T00:43:18.698 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-02-23T00:43:18.698 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (7 ms) 2024-02-23T00:43:18.698 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (1289 ms total) 2024-02-23T00:43:18.699 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: 2024-02-23T00:43:18.699 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-02-23T00:43:18.699 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-02-23T00:43:18.699 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: notify 2024-02-23T00:43:18.699 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (75 ms) 2024-02-23T00:43:18.699 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-02-23T00:43:18.699 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: notify 2024-02-23T00:43:18.699 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (4685 ms) 2024-02-23T00:43:18.699 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-02-23T00:43:18.699 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (7 ms) 2024-02-23T00:43:18.699 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-02-23T00:43:18.699 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (6 ms) 2024-02-23T00:43:18.699 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-02-23T00:43:18.699 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: handle_notify cookie 94873506994976 notify_id 536870912002 notifier_gid 25103 2024-02-23T00:43:18.699 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (6 ms) 2024-02-23T00:43:18.699 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-02-23T00:43:18.700 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: handle_notify cookie 94873507008128 notify_id 536870912003 notifier_gid 25103 2024-02-23T00:43:18.700 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (6 ms) 2024-02-23T00:43:18.700 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-02-23T00:43:18.700 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: handle_notify cookie 94873507012016 notify_id 536870912005 notifier_gid 25103 2024-02-23T00:43:18.700 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (11 ms) 2024-02-23T00:43:18.700 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-02-23T00:43:18.700 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: handle_notify cookie 94873507011984 notify_id 536870912003 notifier_gid 25103 2024-02-23T00:43:18.700 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (6 ms) 2024-02-23T00:43:18.700 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-02-23T00:43:18.700 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: handle_notify cookie 94873507011984 notify_id 536870912006 notifier_gid 25103 2024-02-23T00:43:18.700 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (11 ms) 2024-02-23T00:43:18.700 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-02-23T00:43:18.700 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: handle_notify cookie 94873507011984 notify_id 536870912004 notifier_gid 25103 2024-02-23T00:43:18.700 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (18 ms) 2024-02-23T00:43:18.700 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-02-23T00:43:18.701 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: trying... 2024-02-23T00:43:18.701 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: handle_notify cookie 94873507011984 notify_id 536870912005 notifier_gid 25103 2024-02-23T00:43:18.701 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: timed out 2024-02-23T00:43:18.701 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: flushing 2024-02-23T00:43:18.721 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:18.939 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:17.184602+0000 mgr.y (mgr.14517) 168 : cluster [DBG] pgmap v164: 485 pgs: 32 creating+peering, 453 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:43:18.940 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:17.487733+0000 mon.b (mon.1) 113 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:18.940 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:17.718557+0000 mon.a (mon.0) 1038 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:43:18.940 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:17.726506+0000 mon.a (mon.0) 1039 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "28"}]: dispatch 2024-02-23T00:43:18.940 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:17.765964+0000 mon.a (mon.0) 1040 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi047-82517-12"}]': finished 2024-02-23T00:43:18.940 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:17.766018+0000 mon.a (mon.0) 1041 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:18.941 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:17.184602+0000 mgr.y (mgr.14517) 168 : cluster [DBG] pgmap v164: 485 pgs: 32 creating+peering, 453 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:43:18.941 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:17.487733+0000 mon.b (mon.1) 113 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:18.941 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:17.718557+0000 mon.a (mon.0) 1038 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:43:18.941 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:17.726506+0000 mon.a (mon.0) 1039 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "28"}]: dispatch 2024-02-23T00:43:18.941 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:17.765964+0000 mon.a (mon.0) 1040 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi047-82517-12"}]': finished 2024-02-23T00:43:18.941 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:17.766018+0000 mon.a (mon.0) 1041 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:18.941 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:17.766143+0000 mon.a (mon.0) 1042 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:43:18.941 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:17.766934+0000 mon.a (mon.0) 1043 : cluster [DBG] osdmap e127: 8 total, 8 up, 8 in 2024-02-23T00:43:18.941 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:17.776150+0000 mon.b (mon.1) 114 : audit [INF] from='client.? 172.21.15.47:0/2137942802' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi047-82517-12"}]: dispatch 2024-02-23T00:43:18.941 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:17.777740+0000 mon.c (mon.2) 69 : audit [INF] from='client.? 172.21.15.47:0/2064770043' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi047-82496-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:18.941 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:17.789021+0000 mon.c (mon.2) 70 : audit [INF] from='client.? 172.21.15.47:0/2747693458' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi047-82378-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:18.942 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:18.318147+0000 mon.a (mon.0) 1044 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi047-82496-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:18.942 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:18.325115+0000 mon.a (mon.0) 1045 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi047-82378-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:18.942 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:18.326205+0000 mon.a (mon.0) 1046 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi047-82517-12"}]: dispatch 2024-02-23T00:43:18.981 INFO:tasks.workunit.client.0.smithi047.stdout: api_wa api_watch_notify: Running main() from gmock_main.cc 2024-02-23T00:43:18.981 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2024-02-23T00:43:18.982 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: [----------] Global test environment set-up. 2024-02-23T00:43:18.982 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2024-02-23T00:43:18.982 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2024-02-23T00:43:18.982 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: watch_notify_test_cb 2024-02-23T00:43:18.982 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (266 ms) 2024-02-23T00:43:18.982 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2024-02-23T00:43:18.982 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-02-23T00:43:18.982 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94456142948592 err -107 2024-02-23T00:43:18.983 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (1008 ms) 2024-02-23T00:43:18.983 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-02-23T00:43:18.983 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-02-23T00:43:18.983 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94456142960576 err -107 2024-02-23T00:43:18.983 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (1059 ms) 2024-02-23T00:43:18.983 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2024-02-23T00:43:18.983 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: watch_notify2_test_cb from 24832 notify_id 481036337152 cookie 94456142960576 2024-02-23T00:43:18.984 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (34 ms) 2024-02-23T00:43:18.984 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-02-23T00:43:18.984 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: watch_notify2_test_cb from 24832 notify_id 481036337153 cookie 94456143016864 2024-02-23T00:43:18.984 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (20 ms) 2024-02-23T00:43:18.984 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2024-02-23T00:43:18.984 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: watch_notify2_test_cb from 24832 notify_id 481036337154 cookie 94456143016864 2024-02-23T00:43:18.984 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (15 ms) 2024-02-23T00:43:18.985 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-02-23T00:43:18.985 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: watch_notify2_test_cb from 24832 notify_id 481036337152 cookie 94456143016864 2024-02-23T00:43:18.985 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: watch_notify2_test_cb from 24832 notify_id 481036337152 cookie 94456143036768 2024-02-23T00:43:18.985 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (15 ms) 2024-02-23T00:43:18.985 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-02-23T00:43:18.986 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: watch_notify2_test_cb from 24832 notify_id 481036337155 cookie 94456143036768 2024-02-23T00:43:18.986 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: watch_notify2_test_cb from 24832 notify_id 481036337156 cookie 94456143036768 2024-02-23T00:43:18.986 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3007 ms) 2024-02-23T00:43:18.986 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-02-23T00:43:18.986 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2024-02-23T00:43:18.986 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94456143036768 err -107 2024-02-23T00:43:18.987 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: watch_notify2_test_cb from 24832 notify_id 511101108225 cookie 94456143016864 2024-02-23T00:43:18.987 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (5007 ms) 2024-02-23T00:43:18.987 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-02-23T00:43:18.987 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2024-02-23T00:43:18.987 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94456143016864 err -107 2024-02-23T00:43:18.987 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (1005 ms) 2024-02-23T00:43:18.988 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (11436 ms total) 2024-02-23T00:43:18.988 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: 2024-02-23T00:43:18.988 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2024-02-23T00:43:18.988 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-02-23T00:43:18.988 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: watch_notify_test_cb 2024-02-23T00:43:18.989 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (1433 ms) 2024-02-23T00:43:18.989 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (1433 ms total) 2024-02-23T00:43:18.989 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: 2024-02-23T00:43:18.989 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: [----------] Global test environment tear-down 2024-02-23T00:43:18.989 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (31763 ms total) 2024-02-23T00:43:18.989 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify: [ PASSED ] 11 tests. 2024-02-23T00:43:19.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.559+0000 7fe2781d9700 1 -- 172.21.15.47:0/459453806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2700f6b10 msgr2=0x7fe2701030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:19.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.559+0000 7fe2781d9700 1 --2- 172.21.15.47:0/459453806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2700f6b10 0x7fe2701030b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7fe26800acf0 tx=0x7fe268004d10).stop 2024-02-23T00:43:19.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.559+0000 7fe2781d9700 1 -- 172.21.15.47:0/459453806 shutdown_connections 2024-02-23T00:43:19.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.559+0000 7fe2781d9700 1 --2- 172.21.15.47:0/459453806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2700f6b10 0x7fe2701030b0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:19.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.559+0000 7fe2781d9700 1 --2- 172.21.15.47:0/459453806 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe2700fc240 0x7fe2700fc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:43:19.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.559+0000 7fe2781d9700 1 --2- 172.21.15.47:0/459453806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2700fcba0 0x7fe2700f6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:43:19.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.559+0000 7fe2781d9700 1 -- 172.21.15.47:0/459453806 >> 172.21.15.47:0/459453806 conn(0x7fe27006c370 msgr2=0x7fe27006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:43:19.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.559+0000 7fe2781d9700 1 -- 172.21.15.47:0/459453806 shutdown_connections 2024-02-23T00:43:19.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.559+0000 7fe2781d9700 1 -- 172.21.15.47:0/459453806 wait complete. 2024-02-23T00:43:19.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.560+0000 7fe2781d9700 1 librados: starting msgr at 2024-02-23T00:43:19.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.560+0000 7fe2781d9700 1 librados: starting objecter 2024-02-23T00:43:19.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.560+0000 7fe2781d9700 1 Processor -- start 2024-02-23T00:43:19.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.560+0000 7fe2781d9700 1 -- start start 2024-02-23T00:43:19.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.560+0000 7fe2781d9700 1 librados: setting wanted keys 2024-02-23T00:43:19.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.560+0000 7fe2781d9700 1 librados: calling monclient init 2024-02-23T00:43:19.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.560+0000 7fe2781d9700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2700f6b10 0x7fe27010ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:19.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.560+0000 7fe2781d9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2700fc240 0x7fe27010b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:19.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.560+0000 7fe2781d9700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe2700fcba0 0x7fe270105e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:19.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.560+0000 7fe2781d9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe270106480 con 0x7fe2700fc240 2024-02-23T00:43:19.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.560+0000 7fe2781d9700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2701065f0 con 0x7fe2700f6b10 2024-02-23T00:43:19.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.560+0000 7fe2781d9700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2701067b0 con 0x7fe2700fcba0 2024-02-23T00:43:19.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.560+0000 7fe276776700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe2700fcba0 0x7fe270105e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:19.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.560+0000 7fe276776700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe2700fcba0 0x7fe270105e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:39580/0 (socket says 172.21.15.47:39580) 2024-02-23T00:43:19.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.560+0000 7fe276776700 1 -- 172.21.15.47:0/3183398453 learned_addr learned my addr 172.21.15.47:0/3183398453 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:43:19.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.561+0000 7fe275f75700 1 --2- 172.21.15.47:0/3183398453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2700f6b10 0x7fe27010ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:19.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.561+0000 7fe276776700 1 -- 172.21.15.47:0/3183398453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2700f6b10 msgr2=0x7fe27010ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:19.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.561+0000 7fe276776700 1 --2- 172.21.15.47:0/3183398453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2700f6b10 0x7fe27010ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:19.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.561+0000 7fe276776700 1 -- 172.21.15.47:0/3183398453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2700fc240 msgr2=0x7fe27010b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:19.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.561+0000 7fe276776700 1 --2- 172.21.15.47:0/3183398453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2700fc240 0x7fe27010b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:43:19.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.561+0000 7fe276776700 1 -- 172.21.15.47:0/3183398453 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe26800a8d0 con 0x7fe2700fcba0 2024-02-23T00:43:19.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.561+0000 7fe275f75700 1 --2- 172.21.15.47:0/3183398453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2700f6b10 0x7fe27010ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:43:19.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.561+0000 7fe276776700 1 --2- 172.21.15.47:0/3183398453 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe2700fcba0 0x7fe270105e70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7fe268004a30 tx=0x7fe268004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:19.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.561+0000 7fe266ffd700 10 client.?.objecter ms_handle_connect 0x7fe2700fcba0 2024-02-23T00:43:19.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.561+0000 7fe266ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:43:19.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.561+0000 7fe266ffd700 1 -- 172.21.15.47:0/3183398453 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26800f030 con 0x7fe2700fcba0 2024-02-23T00:43:19.140 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:43:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.561+0000 7fe266ffd700 1 -- 172.21.15.47:0/3183398453 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fe268007e40 con 0x7fe2700fcba0 2024-02-23T00:43:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.561+0000 7fe2781d9700 1 -- 172.21.15.47:0/3183398453 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe270106a30 con 0x7fe2700fcba0 2024-02-23T00:43:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.561+0000 7fe266ffd700 1 -- 172.21.15.47:0/3183398453 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe268019070 con 0x7fe2700fcba0 2024-02-23T00:43:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.561+0000 7fe2781d9700 10 client.26489.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:43:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.561+0000 7fe2781d9700 1 -- 172.21.15.47:0/3183398453 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe270110c20 con 0x7fe2700fcba0 2024-02-23T00:43:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.561+0000 7fe2781d9700 1 librados: init done 2024-02-23T00:43:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.563+0000 7fe266ffd700 1 -- 172.21.15.47:0/3183398453 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fe2680079b0 con 0x7fe2700fcba0 2024-02-23T00:43:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.564+0000 7fe266ffd700 1 --2- 172.21.15.47:0/3183398453 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe25c062cc0 0x7fe25c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.564+0000 7fe266ffd700 1 -- 172.21.15.47:0/3183398453 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(127..127 src has 48..127) v4 ==== 12393+0+0 (secure 0 0 0) 0x7fe26807fd00 con 0x7fe2700fcba0 2024-02-23T00:43:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.564+0000 7fe266ffd700 10 client.26489.objecter ms_dispatch 0x7fe270001550 osd_map(127..127 src has 48..127) v4 2024-02-23T00:43:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.564+0000 7fe266ffd700 3 client.26489.objecter handle_osd_map got epochs [127,127] > 0 2024-02-23T00:43:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.564+0000 7fe266ffd700 3 client.26489.objecter handle_osd_map decoding full epoch 127 2024-02-23T00:43:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.564+0000 7fe266ffd700 10 client.26489.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:43:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.564+0000 7fe275f75700 1 --2- 172.21.15.47:0/3183398453 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe25c062cc0 0x7fe25c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.564+0000 7fe266ffd700 1 -- 172.21.15.47:0/3183398453 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=128}) v3 -- 0x7fe25c076440 con 0x7fe2700fcba0 2024-02-23T00:43:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.564+0000 7fe266ffd700 20 client.26489.objecter dump_active .. 0 homeless 2024-02-23T00:43:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.564+0000 7fe266ffd700 10 client.26489.objecter ms_handle_connect 0x7fe25c062cc0 2024-02-23T00:43:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.564+0000 7fe275f75700 1 --2- 172.21.15.47:0/3183398453 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe25c062cc0 0x7fe25c065180 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7fe26c006fd0 tx=0x7fe26c009060).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.565+0000 7fe2781d9700 1 -- 172.21.15.47:0/3183398453 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2701077f0 con 0x7fe2700fcba0 2024-02-23T00:43:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.570+0000 7fe266ffd700 1 -- 172.21.15.47:0/3183398453 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe26804e650 con 0x7fe2700fcba0 2024-02-23T00:43:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.899+0000 7fe2781d9700 1 -- 172.21.15.47:0/3183398453 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe27005ee80 con 0x7fe25c062cc0 2024-02-23T00:43:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.950+0000 7fe266ffd700 1 -- 172.21.15.47:0/3183398453 <== mon.2 v2:172.21.15.47:3301/0 7 ==== osd_map(128..128 src has 48..128) v4 ==== 3766+0+0 (secure 0 0 0) 0x7fe26800a6f0 con 0x7fe2700fcba0 2024-02-23T00:43:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.950+0000 7fe266ffd700 10 client.26489.objecter ms_dispatch 0x7fe270001550 osd_map(128..128 src has 48..128) v4 2024-02-23T00:43:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.950+0000 7fe266ffd700 3 client.26489.objecter handle_osd_map got epochs [128,128] > 127 2024-02-23T00:43:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.950+0000 7fe266ffd700 3 client.26489.objecter handle_osd_map decoding incremental epoch 128 2024-02-23T00:43:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.951+0000 7fe266ffd700 10 client.26489.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:43:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.951+0000 7fe266ffd700 1 -- 172.21.15.47:0/3183398453 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=129}) v3 -- 0x7fe25c07a410 con 0x7fe2700fcba0 2024-02-23T00:43:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.951+0000 7fe266ffd700 20 client.26489.objecter dump_active .. 0 homeless 2024-02-23T00:43:19.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:18.963+0000 7fe266ffd700 1 -- 172.21.15.47:0/3183398453 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+1170548 (secure 0 0 0) 0x7fe27005ee80 con 0x7fe25c062cc0 2024-02-23T00:43:19.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:19.171+0000 7fe2781d9700 10 librados: watch_flush enter 2024-02-23T00:43:19.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:19.171+0000 7fe2781d9700 10 librados: watch_flush exit 2024-02-23T00:43:19.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:19.171+0000 7fe2781d9700 20 client.26489.objecter shutdown clearing up homeless session... 2024-02-23T00:43:19.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:19.171+0000 7fe2781d9700 10 client.26489.objecter successfully canceled tick 2024-02-23T00:43:19.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:19.171+0000 7fe2781d9700 1 -- 172.21.15.47:0/3183398453 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe25c062cc0 msgr2=0x7fe25c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:19.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:19.171+0000 7fe2781d9700 1 --2- 172.21.15.47:0/3183398453 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe25c062cc0 0x7fe25c065180 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7fe26c006fd0 tx=0x7fe26c009060).stop 2024-02-23T00:43:19.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:19.171+0000 7fe2781d9700 1 -- 172.21.15.47:0/3183398453 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe2700fcba0 msgr2=0x7fe270105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:19.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:19.171+0000 7fe2781d9700 1 --2- 172.21.15.47:0/3183398453 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe2700fcba0 0x7fe270105e70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7fe268004a30 tx=0x7fe268004d10).stop 2024-02-23T00:43:19.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:19.177+0000 7fe2781d9700 1 -- 172.21.15.47:0/3183398453 shutdown_connections 2024-02-23T00:43:19.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:19.177+0000 7fe2781d9700 1 --2- 172.21.15.47:0/3183398453 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe25c062cc0 0x7fe25c065180 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:19.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:19.177+0000 7fe2781d9700 1 --2- 172.21.15.47:0/3183398453 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe2700fcba0 0x7fe270105e70 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:19.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:19.177+0000 7fe2781d9700 1 --2- 172.21.15.47:0/3183398453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2700f6b10 0x7fe27010ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:19.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:19.177+0000 7fe2781d9700 1 --2- 172.21.15.47:0/3183398453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2700fc240 0x7fe27010b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:43:19.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:19.177+0000 7fe2781d9700 1 -- 172.21.15.47:0/3183398453 >> 172.21.15.47:0/3183398453 conn(0x7fe27006c370 msgr2=0x7fe2700f51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:43:19.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:19.178+0000 7fe2781d9700 1 -- 172.21.15.47:0/3183398453 shutdown_connections 2024-02-23T00:43:19.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:19.178+0000 7fe2781d9700 1 -- 172.21.15.47:0/3183398453 wait complete. 2024-02-23T00:43:19.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:19.179+0000 7fe2781d9700 1 librados: shutdown 2024-02-23T00:43:19.188 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-02-23T00:43:19.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:17.766143+0000 mon.a (mon.0) 1042 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:43:19.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:17.766934+0000 mon.a (mon.0) 1043 : cluster [DBG] osdmap e127: 8 total, 8 up, 8 in 2024-02-23T00:43:19.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:17.776150+0000 mon.b (mon.1) 114 : audit [INF] from='client.? 172.21.15.47:0/2137942802' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi047-82517-12"}]: dispatch 2024-02-23T00:43:19.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:17.777740+0000 mon.c (mon.2) 69 : audit [INF] from='client.? 172.21.15.47:0/2064770043' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi047-82496-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:19.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:17.789021+0000 mon.c (mon.2) 70 : audit [INF] from='client.? 172.21.15.47:0/2747693458' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi047-82378-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:19.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:18.318147+0000 mon.a (mon.0) 1044 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi047-82496-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:19.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:18.325115+0000 mon.a (mon.0) 1045 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi047-82378-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:19.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:18.326205+0000 mon.a (mon.0) 1046 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi047-82517-12"}]: dispatch 2024-02-23T00:43:19.224 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:17.184602+0000 mgr.y (mgr.14517) 168 : cluster [DBG] pgmap v164: 485 pgs: 32 creating+peering, 453 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:43:19.225 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:17.487733+0000 mon.b (mon.1) 113 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:19.225 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:17.718557+0000 mon.a (mon.0) 1038 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:43:19.225 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:17.726506+0000 mon.a (mon.0) 1039 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "28"}]: dispatch 2024-02-23T00:43:19.225 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:17.765964+0000 mon.a (mon.0) 1040 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi047-82517-12"}]': finished 2024-02-23T00:43:19.225 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:17.766018+0000 mon.a (mon.0) 1041 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:19.225 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:17.766143+0000 mon.a (mon.0) 1042 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:43:19.225 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:17.766934+0000 mon.a (mon.0) 1043 : cluster [DBG] osdmap e127: 8 total, 8 up, 8 in 2024-02-23T00:43:19.225 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:17.776150+0000 mon.b (mon.1) 114 : audit [INF] from='client.? 172.21.15.47:0/2137942802' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi047-82517-12"}]: dispatch 2024-02-23T00:43:19.225 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:17.777740+0000 mon.c (mon.2) 69 : audit [INF] from='client.? 172.21.15.47:0/2064770043' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi047-82496-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:19.226 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:17.789021+0000 mon.c (mon.2) 70 : audit [INF] from='client.? 172.21.15.47:0/2747693458' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi047-82378-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:19.226 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:18.318147+0000 mon.a (mon.0) 1044 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi047-82496-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:19.226 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:18.325115+0000 mon.a (mon.0) 1045 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi047-82378-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:19.226 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:18.326205+0000 mon.a (mon.0) 1046 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi047-82517-12"}]: dispatch 2024-02-23T00:43:20.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:18.492680+0000 mon.b (mon.1) 115 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:20.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:18.900506+0000 mgr.y (mgr.14517) 169 : audit [DBG] from='client.26489 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:43:20.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:18.939542+0000 mon.a (mon.0) 1047 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "28"}]': finished 2024-02-23T00:43:20.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:18.939671+0000 mon.a (mon.0) 1048 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi047-82496-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:20.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:18.939754+0000 mon.a (mon.0) 1049 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi047-82378-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:20.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:18.939817+0000 mon.a (mon.0) 1050 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi047-82517-12"}]': finished 2024-02-23T00:43:20.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:18.962103+0000 mon.b (mon.1) 116 : audit [INF] from='client.? 172.21.15.47:0/1104409358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:20.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:19.185137+0000 mgr.y (mgr.14517) 170 : cluster [DBG] pgmap v167: 549 pgs: 128 unknown, 32 creating+peering, 389 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 3.0 KiB/s wr, 3 op/s 2024-02-23T00:43:20.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:19.495473+0000 mon.a (mon.0) 1051 : cluster [DBG] osdmap e128: 8 total, 8 up, 8 in 2024-02-23T00:43:20.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:19.506470+0000 mon.b (mon.1) 117 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:20.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:19.511971+0000 mon.a (mon.0) 1052 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:20.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:19.514636+0000 mon.a (mon.0) 1053 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:20.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:19.527119+0000 mon.a (mon.0) 1054 : cluster [DBG] osdmap e129: 8 total, 8 up, 8 in 2024-02-23T00:43:20.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:19.540761+0000 mon.b (mon.1) 118 : audit [INF] from='client.? 172.21.15.47:0/752621479' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:20.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:19.542152+0000 mon.a (mon.0) 1055 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:20.311 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:18.492680+0000 mon.b (mon.1) 115 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:20.311 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:18.900506+0000 mgr.y (mgr.14517) 169 : audit [DBG] from='client.26489 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:43:20.311 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:18.939542+0000 mon.a (mon.0) 1047 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "28"}]': finished 2024-02-23T00:43:20.311 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:18.939671+0000 mon.a (mon.0) 1048 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi047-82496-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:20.311 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:18.939754+0000 mon.a (mon.0) 1049 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi047-82378-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:20.311 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:18.939817+0000 mon.a (mon.0) 1050 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi047-82517-12"}]': finished 2024-02-23T00:43:20.311 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:18.962103+0000 mon.b (mon.1) 116 : audit [INF] from='client.? 172.21.15.47:0/1104409358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:20.311 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:19.185137+0000 mgr.y (mgr.14517) 170 : cluster [DBG] pgmap v167: 549 pgs: 128 unknown, 32 creating+peering, 389 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 3.0 KiB/s wr, 3 op/s 2024-02-23T00:43:20.311 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:19.495473+0000 mon.a (mon.0) 1051 : cluster [DBG] osdmap e128: 8 total, 8 up, 8 in 2024-02-23T00:43:20.311 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:19.506470+0000 mon.b (mon.1) 117 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:20.311 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:19.511971+0000 mon.a (mon.0) 1052 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:20.311 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:19.514636+0000 mon.a (mon.0) 1053 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:20.311 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:19.527119+0000 mon.a (mon.0) 1054 : cluster [DBG] osdmap e129: 8 total, 8 up, 8 in 2024-02-23T00:43:20.312 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:19.540761+0000 mon.b (mon.1) 118 : audit [INF] from='client.? 172.21.15.47:0/752621479' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:20.312 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:19.542152+0000 mon.a (mon.0) 1055 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:20.373 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: [==========] Running 4 tests from 1 test suite. 2024-02-23T00:43:20.373 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: [----------] Global test environment set-up. 2024-02-23T00:43:20.373 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: [----------] 4 tests from LibRadosService 2024-02-23T00:43:20.373 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2024-02-23T00:43:20.373 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5072 ms) 2024-02-23T00:43:20.374 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2024-02-23T00:43:20.374 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: [ OK ] LibRadosService.RegisterLate (15 ms) 2024-02-23T00:43:20.374 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2024-02-23T00:43:20.374 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: cluster: 2024-02-23T00:43:20.374 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: id: fce2617e-d1e2-11ee-95c0-87774f69a715 2024-02-23T00:43:20.374 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: health: HEALTH_OK 2024-02-23T00:43:20.374 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: 2024-02-23T00:43:20.374 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: services: 2024-02-23T00:43:20.374 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 7m) 2024-02-23T00:43:20.374 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: mgr: y(active, since 3m), standbys: x 2024-02-23T00:43:20.374 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: osd: 8 osds: 8 up (since 3m), 8 in (since 4m) 2024-02-23T00:43:20.374 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-02-23T00:43:20.374 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-23T00:43:20.375 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: 2024-02-23T00:43:20.375 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: data: 2024-02-23T00:43:20.375 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: pools: 7 pools, 164 pgs 2024-02-23T00:43:20.375 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: objects: 224 objects, 5.7 KiB 2024-02-23T00:43:20.375 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: usage: 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:43:20.375 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: pgs: 19.512% pgs unknown 2024-02-23T00:43:20.375 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: 132 active+clean 2024-02-23T00:43:20.375 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: 32 unknown 2024-02-23T00:43:20.375 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: 2024-02-23T00:43:20.375 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: io: 2024-02-23T00:43:20.375 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: client: 401 B/s rd, 0 op/s rd, 0 op/s wr 2024-02-23T00:43:20.375 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: recovery: 0 B/s, 0 objects/s 2024-02-23T00:43:20.375 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: 2024-02-23T00:43:20.375 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: 2024-02-23T00:43:20.376 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: cluster: 2024-02-23T00:43:20.376 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: id: fce2617e-d1e2-11ee-95c0-87774f69a715 2024-02-23T00:43:20.376 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: health: HEALTH_WARN 2024-02-23T00:43:20.376 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: noscrub flag(s) set 2024-02-23T00:43:20.376 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: 2024-02-23T00:43:20.376 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: services: 2024-02-23T00:43:20.376 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 7m) 2024-02-23T00:43:20.376 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: mgr: y(active, since 3m), standbys: x 2024-02-23T00:43:20.376 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: osd: 8 osds: 8 up (since 3m), 8 in (since 4m) 2024-02-23T00:43:20.376 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: flags noscrub 2024-02-23T00:43:20.376 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2024-02-23T00:43:20.376 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-02-23T00:43:20.376 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-23T00:43:20.376 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: 2024-02-23T00:43:20.377 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: data: 2024-02-23T00:43:20.377 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: pools: 19 pools, 548 pgs 2024-02-23T00:43:20.377 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: objects: 224 objects, 5.7 KiB 2024-02-23T00:43:20.377 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: usage: 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:43:20.377 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: pgs: 75.912% pgs not active 2024-02-23T00:43:20.377 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: 416 creating+peering 2024-02-23T00:43:20.377 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: 132 active+clean 2024-02-23T00:43:20.377 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: 2024-02-23T00:43:20.377 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: io: 2024-02-23T00:43:20.377 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: client: 1.1 KiB/s rd, 1 op/s rd, 0 op/s wr 2024-02-23T00:43:20.377 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: 2024-02-23T00:43:20.377 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: 2024-02-23T00:43:20.377 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: [ OK ] LibRadosService.StatusFormat (7543 ms) 2024-02-23T00:43:20.377 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: [ RUN ] LibRadosService.Status 2024-02-23T00:43:20.378 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: [ OK ] LibRadosService.Status (20478 ms) 2024-02-23T00:43:20.378 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: [----------] 4 tests from LibRadosService (33109 ms total) 2024-02-23T00:43:20.378 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: 2024-02-23T00:43:20.378 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: [----------] Global test environment tear-down 2024-02-23T00:43:20.378 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: [==========] 4 tests from 1 test suite ran. (33110 ms total) 2024-02-23T00:43:20.378 INFO:tasks.workunit.client.0.smithi047.stdout: api_service: [ PASSED ] 4 tests. 2024-02-23T00:43:20.439 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:43:20.443 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:20.443 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:43:20.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:18.492680+0000 mon.b (mon.1) 115 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:20.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:18.900506+0000 mgr.y (mgr.14517) 169 : audit [DBG] from='client.26489 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:43:20.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:18.939542+0000 mon.a (mon.0) 1047 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "28"}]': finished 2024-02-23T00:43:20.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:18.939671+0000 mon.a (mon.0) 1048 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi047-82496-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:20.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:18.939754+0000 mon.a (mon.0) 1049 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi047-82378-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:20.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:18.939817+0000 mon.a (mon.0) 1050 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi047-82517-12"}]': finished 2024-02-23T00:43:20.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:18.962103+0000 mon.b (mon.1) 116 : audit [INF] from='client.? 172.21.15.47:0/1104409358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:20.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:19.185137+0000 mgr.y (mgr.14517) 170 : cluster [DBG] pgmap v167: 549 pgs: 128 unknown, 32 creating+peering, 389 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 3.0 KiB/s wr, 3 op/s 2024-02-23T00:43:20.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:19.495473+0000 mon.a (mon.0) 1051 : cluster [DBG] osdmap e128: 8 total, 8 up, 8 in 2024-02-23T00:43:20.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:19.506470+0000 mon.b (mon.1) 117 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:20.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:19.511971+0000 mon.a (mon.0) 1052 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:20.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:19.514636+0000 mon.a (mon.0) 1053 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:20.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:19.527119+0000 mon.a (mon.0) 1054 : cluster [DBG] osdmap e129: 8 total, 8 up, 8 in 2024-02-23T00:43:20.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:19.540761+0000 mon.b (mon.1) 118 : audit [INF] from='client.? 172.21.15.47:0/752621479' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:20.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:19.542152+0000 mon.a (mon.0) 1055 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:20.521 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:43:20.545 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:43:20.597 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:20.619 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:43:20.623 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:20.624 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:43:20.725 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:43:20.789 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:43:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:43:20] "GET /metrics HTTP/1.1" 200 217036 "" "Prometheus/2.33.4" 2024-02-23T00:43:20.950 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:21.091 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset noscrub 2024-02-23T00:43:21.236 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:43:21.251 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:21.252 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:43:21.352 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:43:21.378 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:21.397 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:43:21.400 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:21.400 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:43:21.422 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:20.508657+0000 mon.b (mon.1) 119 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:21.422 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:20.518502+0000 mon.a (mon.0) 1056 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-23T00:43:21.422 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:20.530623+0000 mon.a (mon.0) 1057 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:21.422 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:20.530761+0000 mon.a (mon.0) 1058 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:21.422 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:20.530904+0000 mon.a (mon.0) 1059 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:21.422 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:20.530995+0000 mon.a (mon.0) 1060 : cluster [DBG] osdmap e130: 8 total, 8 up, 8 in 2024-02-23T00:43:21.423 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:20.508657+0000 mon.b (mon.1) 119 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:21.423 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:20.518502+0000 mon.a (mon.0) 1056 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-23T00:43:21.423 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:20.530623+0000 mon.a (mon.0) 1057 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:21.423 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:20.530761+0000 mon.a (mon.0) 1058 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:21.423 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:20.530904+0000 mon.a (mon.0) 1059 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:21.423 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:20.530995+0000 mon.a (mon.0) 1060 : cluster [DBG] osdmap e130: 8 total, 8 up, 8 in 2024-02-23T00:43:21.500 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:43:21.571 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:21.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:20.508657+0000 mon.b (mon.1) 119 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:21.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:20.518502+0000 mon.a (mon.0) 1056 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-23T00:43:21.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:20.530623+0000 mon.a (mon.0) 1057 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:21.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:20.530761+0000 mon.a (mon.0) 1058 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:21.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:20.530904+0000 mon.a (mon.0) 1059 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:21.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:20.530995+0000 mon.a (mon.0) 1060 : cluster [DBG] osdmap e130: 8 total, 8 up, 8 in 2024-02-23T00:43:21.812 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:43:21.821 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:21.822 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:43:21.846 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2024-02-23T00:43:17.775390+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":151033462,"num_objects":290,"num_object_clones":0,"num_object_copies":870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":290,"num_whiteouts":0,"num_read":2364,"num_read_kb":50225,"num_write":1792,"num_write_kb":153124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":947,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":10,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2800000,"num_omap_keys":1000,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1928,"ondisk_log_size":1928,"up":1263,"acting":1263,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":1361,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2908792,"kb_used_data":454200,"kb_used_omap":0,"kb_used_meta":2454528,"kb_avail":747052424,"statfs":{"total":767960285184,"available":764981682176,"internally_reserved":0,"allocated":465100800,"data_stored":459580276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2513436672},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8949,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":9,"num_read_kb":4,"num_write":3,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":10,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.530818"},"pg_stats":[{"pgid":"64.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"64.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"64.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"64.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"64.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"64.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"64.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"64.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"64.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"64.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"64.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"64.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"64.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"64.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"64.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"64.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"64.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"64.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"64.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"64.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"64.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"64.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"64.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"64.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"64.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"64.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"64.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"64.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"64.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"64.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"64.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"64.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"63.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:16.533032+0000","last_change":"2024-02-23T00:43:16.570608+0000","last_active":"2024-02-23T00:43:16.533032+0000","last_peered":"2024-02-23T00:43:16.533032+0000","last_clean":"2024-02-23T00:43:16.533032+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:16.533032+0000","last_undegraded":"2024-02-23T00:43:16.533032+0000","last_fullsized":"2024-02-23T00:43:16.533032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"59.5","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.635588+0000","last_change":"2024-02-23T00:43:16.635588+0000","last_active":"2024-02-23T00:43:16.635588+0000","last_peered":"2024-02-23T00:43:16.635588+0000","last_clean":"2024-02-23T00:43:16.635588+0000","last_became_active":"2024-02-23T00:43:16.635301+0000","last_became_peered":"2024-02-23T00:43:16.635301+0000","last_unstale":"2024-02-23T00:43:16.635588+0000","last_undegraded":"2024-02-23T00:43:16.635588+0000","last_fullsized":"2024-02-23T00:43:16.635588+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_clean_scrub_stamp":"2024-02-23T00:43:15.522490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.a","version":"0'0","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.537249+0000","last_change":"2024-02-23T00:43:10.896978+0000","last_active":"2024-02-23T00:43:15.537249+0000","last_peered":"2024-02-23T00:43:15.537249+0000","last_clean":"2024-02-23T00:43:15.537249+0000","last_became_active":"2024-02-23T00:43:10.896660+0000","last_became_peered":"2024-02-23T00:43:10.896660+0000","last_unstale":"2024-02-23T00:43:15.537249+0000","last_undegraded":"2024-02-23T00:43:15.537249+0000","last_fullsized":"2024-02-23T00:43:15.537249+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_clean_scrub_stamp":"2024-02-23T00:43:09.833908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,0],"acting":[5,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"47.11","version":"0'0","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.546140+0000","last_change":"2024-02-23T00:43:08.855287+0000","last_active":"2024-02-23T00:43:15.546140+0000","last_peered":"2024-02-23T00:43:15.546140+0000","last_clean":"2024-02-23T00:43:15.546140+0000","last_became_active":"2024-02-23T00:43:08.854959+0000","last_became_peered":"2024-02-23T00:43:08.854959+0000","last_unstale":"2024-02-23T00:43:15.546140+0000","last_undegraded":"2024-02-23T00:43:15.546140+0000","last_fullsized":"2024-02-23T00:43:15.546140+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,7],"acting":[4,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.14","version":"0'0","reported_seq":19,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.538496+0000","last_change":"2024-02-23T00:43:05.248382+0000","last_active":"2024-02-23T00:43:15.538496+0000","last_peered":"2024-02-23T00:43:15.538496+0000","last_clean":"2024-02-23T00:43:15.538496+0000","last_became_active":"2024-02-23T00:43:05.247655+0000","last_became_peered":"2024-02-23T00:43:05.247655+0000","last_unstale":"2024-02-23T00:43:15.538496+0000","last_undegraded":"2024-02-23T00:43:15.538496+0000","last_fullsized":"2024-02-23T00:43:15.538496+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.197452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.1d","version":"0'0","reported_seq":22,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.545797+0000","last_change":"2024-02-23T00:43:02.367988+0000","last_active":"2024-02-23T00:43:15.545797+0000","last_peered":"2024-02-23T00:43:15.545797+0000","last_clean":"2024-02-23T00:43:15.545797+0000","last_became_active":"2024-02-23T00:43:02.367750+0000","last_became_peered":"2024-02-23T00:43:02.367750+0000","last_unstale":"2024-02-23T00:43:15.545797+0000","last_undegraded":"2024-02-23T00:43:15.545797+0000","last_fullsized":"2024-02-23T00:43:15.545797+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_clean_scrub_stamp":"2024-02-23T00:43:00.812733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"63.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:16.533032+0000","last_change":"2024-02-23T00:43:16.570394+0000","last_active":"2024-02-23T00:43:16.533032+0000","last_peered":"2024-02-23T00:43:16.533032+0000","last_clean":"2024-02-23T00:43:16.533032+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:16.533032+0000","last_undegraded":"2024-02-23T00:43:16.533032+0000","last_fullsized":"2024-02-23T00:43:16.533032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"59.4","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.625532+0000","last_change":"2024-02-23T00:43:16.625532+0000","last_active":"2024-02-23T00:43:16.625532+0000","last_peered":"2024-02-23T00:43:16.625532+0000","last_clean":"2024-02-23T00:43:16.625532+0000","last_became_active":"2024-02-23T00:43:16.625207+0000","last_became_peered":"2024-02-23T00:43:16.625207+0000","last_unstale":"2024-02-23T00:43:16.625532+0000","last_undegraded":"2024-02-23T00:43:16.625532+0000","last_fullsized":"2024-02-23T00:43:16.625532+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_clean_scrub_stamp":"2024-02-23T00:43:15.522490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.b","version":"0'0","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.548177+0000","last_change":"2024-02-23T00:43:10.896773+0000","last_active":"2024-02-23T00:43:15.548177+0000","last_peered":"2024-02-23T00:43:15.548177+0000","last_clean":"2024-02-23T00:43:15.548177+0000","last_became_active":"2024-02-23T00:43:10.896464+0000","last_became_peered":"2024-02-23T00:43:10.896464+0000","last_unstale":"2024-02-23T00:43:15.548177+0000","last_undegraded":"2024-02-23T00:43:15.548177+0000","last_fullsized":"2024-02-23T00:43:15.548177+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_clean_scrub_stamp":"2024-02-23T00:43:09.833908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"47.10","version":"0'0","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.543663+0000","last_change":"2024-02-23T00:43:08.889966+0000","last_active":"2024-02-23T00:43:15.543663+0000","last_peered":"2024-02-23T00:43:15.543663+0000","last_clean":"2024-02-23T00:43:15.543663+0000","last_became_active":"2024-02-23T00:43:08.889086+0000","last_became_peered":"2024-02-23T00:43:08.889086+0000","last_unstale":"2024-02-23T00:43:15.543663+0000","last_undegraded":"2024-02-23T00:43:15.543663+0000","last_fullsized":"2024-02-23T00:43:15.543663+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.15","version":"0'0","reported_seq":19,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.555088+0000","last_change":"2024-02-23T00:43:05.282540+0000","last_active":"2024-02-23T00:43:15.555088+0000","last_peered":"2024-02-23T00:43:15.555088+0000","last_clean":"2024-02-23T00:43:15.555088+0000","last_became_active":"2024-02-23T00:43:05.282031+0000","last_became_peered":"2024-02-23T00:43:05.282031+0000","last_unstale":"2024-02-23T00:43:15.555088+0000","last_undegraded":"2024-02-23T00:43:15.555088+0000","last_fullsized":"2024-02-23T00:43:15.555088+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.197452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.1c","version":"0'0","reported_seq":22,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.541068+0000","last_change":"2024-02-23T00:43:02.367341+0000","last_active":"2024-02-23T00:43:15.541068+0000","last_peered":"2024-02-23T00:43:15.541068+0000","last_clean":"2024-02-23T00:43:15.541068+0000","last_became_active":"2024-02-23T00:43:02.367009+0000","last_became_peered":"2024-02-23T00:43:02.367009+0000","last_unstale":"2024-02-23T00:43:15.541068+0000","last_undegraded":"2024-02-23T00:43:15.541068+0000","last_fullsized":"2024-02-23T00:43:15.541068+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_clean_scrub_stamp":"2024-02-23T00:43:00.812733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"63.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:16.533032+0000","last_change":"2024-02-23T00:43:16.550294+0000","last_active":"2024-02-23T00:43:16.533032+0000","last_peered":"2024-02-23T00:43:16.533032+0000","last_clean":"2024-02-23T00:43:16.533032+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:16.533032+0000","last_undegraded":"2024-02-23T00:43:16.533032+0000","last_fullsized":"2024-02-23T00:43:16.533032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"59.7","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.666836+0000","last_change":"2024-02-23T00:43:16.666836+0000","last_active":"2024-02-23T00:43:16.666836+0000","last_peered":"2024-02-23T00:43:16.666836+0000","last_clean":"2024-02-23T00:43:16.666836+0000","last_became_active":"2024-02-23T00:43:16.666392+0000","last_became_peered":"2024-02-23T00:43:16.666392+0000","last_unstale":"2024-02-23T00:43:16.666836+0000","last_undegraded":"2024-02-23T00:43:16.666836+0000","last_fullsized":"2024-02-23T00:43:16.666836+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_clean_scrub_stamp":"2024-02-23T00:43:15.522490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"52.8","version":"0'0","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.542390+0000","last_change":"2024-02-23T00:43:10.903766+0000","last_active":"2024-02-23T00:43:15.542390+0000","last_peered":"2024-02-23T00:43:15.542390+0000","last_clean":"2024-02-23T00:43:15.542390+0000","last_became_active":"2024-02-23T00:43:10.903560+0000","last_became_peered":"2024-02-23T00:43:10.903560+0000","last_unstale":"2024-02-23T00:43:15.542390+0000","last_undegraded":"2024-02-23T00:43:15.542390+0000","last_fullsized":"2024-02-23T00:43:15.542390+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_clean_scrub_stamp":"2024-02-23T00:43:09.833908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"47.13","version":"0'0","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.549858+0000","last_change":"2024-02-23T00:43:08.886490+0000","last_active":"2024-02-23T00:43:15.549858+0000","last_peered":"2024-02-23T00:43:15.549858+0000","last_clean":"2024-02-23T00:43:15.549858+0000","last_became_active":"2024-02-23T00:43:08.886200+0000","last_became_peered":"2024-02-23T00:43:08.886200+0000","last_unstale":"2024-02-23T00:43:15.549858+0000","last_undegraded":"2024-02-23T00:43:15.549858+0000","last_fullsized":"2024-02-23T00:43:15.549858+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"42.16","version":"0'0","reported_seq":19,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.571614+0000","last_change":"2024-02-23T00:43:05.291856+0000","last_active":"2024-02-23T00:43:15.571614+0000","last_peered":"2024-02-23T00:43:15.571614+0000","last_clean":"2024-02-23T00:43:15.571614+0000","last_became_active":"2024-02-23T00:43:05.291576+0000","last_became_peered":"2024-02-23T00:43:05.291576+0000","last_unstale":"2024-02-23T00:43:15.571614+0000","last_undegraded":"2024-02-23T00:43:15.571614+0000","last_fullsized":"2024-02-23T00:43:15.571614+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.197452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.1f","version":"0'0","reported_seq":22,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.554674+0000","last_change":"2024-02-23T00:43:02.353765+0000","last_active":"2024-02-23T00:43:15.554674+0000","last_peered":"2024-02-23T00:43:15.554674+0000","last_clean":"2024-02-23T00:43:15.554674+0000","last_became_active":"2024-02-23T00:43:02.353422+0000","last_became_peered":"2024-02-23T00:43:02.353422+0000","last_unstale":"2024-02-23T00:43:15.554674+0000","last_undegraded":"2024-02-23T00:43:15.554674+0000","last_fullsized":"2024-02-23T00:43:15.554674+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_clean_scrub_stamp":"2024-02-23T00:43:00.812733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"63.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:16.533032+0000","last_change":"2024-02-23T00:43:16.629498+0000","last_active":"2024-02-23T00:43:16.533032+0000","last_peered":"2024-02-23T00:43:16.533032+0000","last_clean":"2024-02-23T00:43:16.533032+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:16.533032+0000","last_undegraded":"2024-02-23T00:43:16.533032+0000","last_fullsized":"2024-02-23T00:43:16.533032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"59.6","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.633873+0000","last_change":"2024-02-23T00:43:16.633873+0000","last_active":"2024-02-23T00:43:16.633873+0000","last_peered":"2024-02-23T00:43:16.633873+0000","last_clean":"2024-02-23T00:43:16.633873+0000","last_became_active":"2024-02-23T00:43:16.633572+0000","last_became_peered":"2024-02-23T00:43:16.633572+0000","last_unstale":"2024-02-23T00:43:16.633873+0000","last_undegraded":"2024-02-23T00:43:16.633873+0000","last_fullsized":"2024-02-23T00:43:16.633873+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_clean_scrub_stamp":"2024-02-23T00:43:15.522490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.9","version":"0'0","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.545479+0000","last_change":"2024-02-23T00:43:10.901780+0000","last_active":"2024-02-23T00:43:15.545479+0000","last_peered":"2024-02-23T00:43:15.545479+0000","last_clean":"2024-02-23T00:43:15.545479+0000","last_became_active":"2024-02-23T00:43:10.901521+0000","last_became_peered":"2024-02-23T00:43:10.901521+0000","last_unstale":"2024-02-23T00:43:15.545479+0000","last_undegraded":"2024-02-23T00:43:15.545479+0000","last_fullsized":"2024-02-23T00:43:15.545479+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_clean_scrub_stamp":"2024-02-23T00:43:09.833908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"47.12","version":"0'0","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.557317+0000","last_change":"2024-02-23T00:43:08.887138+0000","last_active":"2024-02-23T00:43:15.557317+0000","last_peered":"2024-02-23T00:43:15.557317+0000","last_clean":"2024-02-23T00:43:15.557317+0000","last_became_active":"2024-02-23T00:43:08.886831+0000","last_became_peered":"2024-02-23T00:43:08.886831+0000","last_unstale":"2024-02-23T00:43:15.557317+0000","last_undegraded":"2024-02-23T00:43:15.557317+0000","last_fullsized":"2024-02-23T00:43:15.557317+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"42.17","version":"122'1","reported_seq":20,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.541528+0000","last_change":"2024-02-23T00:43:05.256925+0000","last_active":"2024-02-23T00:43:15.541528+0000","last_peered":"2024-02-23T00:43:15.541528+0000","last_clean":"2024-02-23T00:43:15.541528+0000","last_became_active":"2024-02-23T00:43:05.256626+0000","last_became_peered":"2024-02-23T00:43:05.256626+0000","last_unstale":"2024-02-23T00:43:15.541528+0000","last_undegraded":"2024-02-23T00:43:15.541528+0000","last_fullsized":"2024-02-23T00:43:15.541528+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.197452+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4096,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"35.1e","version":"0'0","reported_seq":22,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.543558+0000","last_change":"2024-02-23T00:43:03.241979+0000","last_active":"2024-02-23T00:43:15.543558+0000","last_peered":"2024-02-23T00:43:15.543558+0000","last_clean":"2024-02-23T00:43:15.543558+0000","last_became_active":"2024-02-23T00:43:03.241699+0000","last_became_peered":"2024-02-23T00:43:03.241699+0000","last_unstale":"2024-02-23T00:43:15.543558+0000","last_undegraded":"2024-02-23T00:43:15.543558+0000","last_fullsized":"2024-02-23T00:43:15.543558+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_clean_scrub_stamp":"2024-02-23T00:43:00.812733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,4],"acting":[0,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"63.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:16.533032+0000","last_change":"2024-02-23T00:43:16.607322+0000","last_active":"2024-02-23T00:43:16.533032+0000","last_peered":"2024-02-23T00:43:16.533032+0000","last_clean":"2024-02-23T00:43:16.533032+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:16.533032+0000","last_undegraded":"2024-02-23T00:43:16.533032+0000","last_fullsized":"2024-02-23T00:43:16.533032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6,7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"59.1","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.620473+0000","last_change":"2024-02-23T00:43:16.620473+0000","last_active":"2024-02-23T00:43:16.620473+0000","last_peered":"2024-02-23T00:43:16.620473+0000","last_clean":"2024-02-23T00:43:16.620473+0000","last_became_active":"2024-02-23T00:43:16.620196+0000","last_became_peered":"2024-02-23T00:43:16.620196+0000","last_unstale":"2024-02-23T00:43:16.620473+0000","last_undegraded":"2024-02-23T00:43:16.620473+0000","last_fullsized":"2024-02-23T00:43:16.620473+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_clean_scrub_stamp":"2024-02-23T00:43:15.522490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.e","version":"0'0","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.551029+0000","last_change":"2024-02-23T00:43:10.891364+0000","last_active":"2024-02-23T00:43:15.551029+0000","last_peered":"2024-02-23T00:43:15.551029+0000","last_clean":"2024-02-23T00:43:15.551029+0000","last_became_active":"2024-02-23T00:43:10.890705+0000","last_became_peered":"2024-02-23T00:43:10.890705+0000","last_unstale":"2024-02-23T00:43:15.551029+0000","last_undegraded":"2024-02-23T00:43:15.551029+0000","last_fullsized":"2024-02-23T00:43:15.551029+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_clean_scrub_stamp":"2024-02-23T00:43:09.833908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"47.15","version":"0'0","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.554937+0000","last_change":"2024-02-23T00:43:08.880115+0000","last_active":"2024-02-23T00:43:15.554937+0000","last_peered":"2024-02-23T00:43:15.554937+0000","last_clean":"2024-02-23T00:43:15.554937+0000","last_became_active":"2024-02-23T00:43:08.879758+0000","last_became_peered":"2024-02-23T00:43:08.879758+0000","last_unstale":"2024-02-23T00:43:15.554937+0000","last_undegraded":"2024-02-23T00:43:15.554937+0000","last_fullsized":"2024-02-23T00:43:15.554937+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"42.10","version":"0'0","reported_seq":19,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.555373+0000","last_change":"2024-02-23T00:43:05.297803+0000","last_active":"2024-02-23T00:43:15.555373+0000","last_peered":"2024-02-23T00:43:15.555373+0000","last_clean":"2024-02-23T00:43:15.555373+0000","last_became_active":"2024-02-23T00:43:05.297652+0000","last_became_peered":"2024-02-23T00:43:05.297652+0000","last_unstale":"2024-02-23T00:43:15.555373+0000","last_undegraded":"2024-02-23T00:43:15.555373+0000","last_fullsized":"2024-02-23T00:43:15.555373+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.197452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.19","version":"0'0","reported_seq":22,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.537074+0000","last_change":"2024-02-23T00:43:02.355942+0000","last_active":"2024-02-23T00:43:15.537074+0000","last_peered":"2024-02-23T00:43:15.537074+0000","last_clean":"2024-02-23T00:43:15.537074+0000","last_became_active":"2024-02-23T00:43:02.355678+0000","last_became_peered":"2024-02-23T00:43:02.355678+0000","last_unstale":"2024-02-23T00:43:15.537074+0000","last_undegraded":"2024-02-23T00:43:15.537074+0000","last_fullsized":"2024-02-23T00:43:15.537074+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_clean_scrub_stamp":"2024-02-23T00:43:00.812733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"63.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:16.533032+0000","last_change":"2024-02-23T00:43:16.550227+0000","last_active":"2024-02-23T00:43:16.533032+0000","last_peered":"2024-02-23T00:43:16.533032+0000","last_clean":"2024-02-23T00:43:16.533032+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:16.533032+0000","last_undegraded":"2024-02-23T00:43:16.533032+0000","last_fullsized":"2024-02-23T00:43:16.533032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"59.0","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.621647+0000","last_change":"2024-02-23T00:43:16.621647+0000","last_active":"2024-02-23T00:43:16.621647+0000","last_peered":"2024-02-23T00:43:16.621647+0000","last_clean":"2024-02-23T00:43:16.621647+0000","last_became_active":"2024-02-23T00:43:16.621374+0000","last_became_peered":"2024-02-23T00:43:16.621374+0000","last_unstale":"2024-02-23T00:43:16.621647+0000","last_undegraded":"2024-02-23T00:43:16.621647+0000","last_fullsized":"2024-02-23T00:43:16.621647+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_clean_scrub_stamp":"2024-02-23T00:43:15.522490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"52.f","version":"0'0","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.541854+0000","last_change":"2024-02-23T00:43:10.892414+0000","last_active":"2024-02-23T00:43:15.541854+0000","last_peered":"2024-02-23T00:43:15.541854+0000","last_clean":"2024-02-23T00:43:15.541854+0000","last_became_active":"2024-02-23T00:43:10.892122+0000","last_became_peered":"2024-02-23T00:43:10.892122+0000","last_unstale":"2024-02-23T00:43:15.541854+0000","last_undegraded":"2024-02-23T00:43:15.541854+0000","last_fullsized":"2024-02-23T00:43:15.541854+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_clean_scrub_stamp":"2024-02-23T00:43:09.833908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"47.14","version":"120'1","reported_seq":18,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.540964+0000","last_change":"2024-02-23T00:43:09.896503+0000","last_active":"2024-02-23T00:43:15.540964+0000","last_peered":"2024-02-23T00:43:15.540964+0000","last_clean":"2024-02-23T00:43:15.540964+0000","last_became_active":"2024-02-23T00:43:09.895720+0000","last_became_peered":"2024-02-23T00:43:09.895720+0000","last_unstale":"2024-02-23T00:43:15.540964+0000","last_undegraded":"2024-02-23T00:43:15.540964+0000","last_fullsized":"2024-02-23T00:43:15.540964+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"42.11","version":"0'0","reported_seq":19,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.540576+0000","last_change":"2024-02-23T00:43:05.302437+0000","last_active":"2024-02-23T00:43:15.540576+0000","last_peered":"2024-02-23T00:43:15.540576+0000","last_clean":"2024-02-23T00:43:15.540576+0000","last_became_active":"2024-02-23T00:43:05.300207+0000","last_became_peered":"2024-02-23T00:43:05.300207+0000","last_unstale":"2024-02-23T00:43:15.540576+0000","last_undegraded":"2024-02-23T00:43:15.540576+0000","last_fullsized":"2024-02-23T00:43:15.540576+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.197452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.18","version":"0'0","reported_seq":22,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.555457+0000","last_change":"2024-02-23T00:43:03.235090+0000","last_active":"2024-02-23T00:43:15.555457+0000","last_peered":"2024-02-23T00:43:15.555457+0000","last_clean":"2024-02-23T00:43:15.555457+0000","last_became_active":"2024-02-23T00:43:03.232244+0000","last_became_peered":"2024-02-23T00:43:03.232244+0000","last_unstale":"2024-02-23T00:43:15.555457+0000","last_undegraded":"2024-02-23T00:43:15.555457+0000","last_fullsized":"2024-02-23T00:43:15.555457+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_clean_scrub_stamp":"2024-02-23T00:43:00.812733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"63.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:16.533032+0000","last_change":"2024-02-23T00:43:16.582157+0000","last_active":"2024-02-23T00:43:16.533032+0000","last_peered":"2024-02-23T00:43:16.533032+0000","last_clean":"2024-02-23T00:43:16.533032+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:16.533032+0000","last_undegraded":"2024-02-23T00:43:16.533032+0000","last_fullsized":"2024-02-23T00:43:16.533032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"59.3","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.642457+0000","last_change":"2024-02-23T00:43:16.642457+0000","last_active":"2024-02-23T00:43:16.642457+0000","last_peered":"2024-02-23T00:43:16.642457+0000","last_clean":"2024-02-23T00:43:16.642457+0000","last_became_active":"2024-02-23T00:43:16.641948+0000","last_became_peered":"2024-02-23T00:43:16.641948+0000","last_unstale":"2024-02-23T00:43:16.642457+0000","last_undegraded":"2024-02-23T00:43:16.642457+0000","last_fullsized":"2024-02-23T00:43:16.642457+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_clean_scrub_stamp":"2024-02-23T00:43:15.522490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.c","version":"0'0","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.540859+0000","last_change":"2024-02-23T00:43:10.884269+0000","last_active":"2024-02-23T00:43:15.540859+0000","last_peered":"2024-02-23T00:43:15.540859+0000","last_clean":"2024-02-23T00:43:15.540859+0000","last_became_active":"2024-02-23T00:43:10.883951+0000","last_became_peered":"2024-02-23T00:43:10.883951+0000","last_unstale":"2024-02-23T00:43:15.540859+0000","last_undegraded":"2024-02-23T00:43:15.540859+0000","last_fullsized":"2024-02-23T00:43:15.540859+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_clean_scrub_stamp":"2024-02-23T00:43:09.833908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"47.17","version":"0'0","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.540663+0000","last_change":"2024-02-23T00:43:09.916378+0000","last_active":"2024-02-23T00:43:15.540663+0000","last_peered":"2024-02-23T00:43:15.540663+0000","last_clean":"2024-02-23T00:43:15.540663+0000","last_became_active":"2024-02-23T00:43:09.916022+0000","last_became_peered":"2024-02-23T00:43:09.916022+0000","last_unstale":"2024-02-23T00:43:15.540663+0000","last_undegraded":"2024-02-23T00:43:15.540663+0000","last_fullsized":"2024-02-23T00:43:15.540663+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.12","version":"0'0","reported_seq":19,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.548915+0000","last_change":"2024-02-23T00:43:05.292908+0000","last_active":"2024-02-23T00:43:15.548915+0000","last_peered":"2024-02-23T00:43:15.548915+0000","last_clean":"2024-02-23T00:43:15.548915+0000","last_became_active":"2024-02-23T00:43:05.292604+0000","last_became_peered":"2024-02-23T00:43:05.292604+0000","last_unstale":"2024-02-23T00:43:15.548915+0000","last_undegraded":"2024-02-23T00:43:15.548915+0000","last_fullsized":"2024-02-23T00:43:15.548915+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.197452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.1b","version":"0'0","reported_seq":22,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.543381+0000","last_change":"2024-02-23T00:43:02.362259+0000","last_active":"2024-02-23T00:43:15.543381+0000","last_peered":"2024-02-23T00:43:15.543381+0000","last_clean":"2024-02-23T00:43:15.543381+0000","last_became_active":"2024-02-23T00:43:02.361791+0000","last_became_peered":"2024-02-23T00:43:02.361791+0000","last_unstale":"2024-02-23T00:43:15.543381+0000","last_undegraded":"2024-02-23T00:43:15.543381+0000","last_fullsized":"2024-02-23T00:43:15.543381+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_clean_scrub_stamp":"2024-02-23T00:43:00.812733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"63.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:16.533032+0000","last_change":"2024-02-23T00:43:16.629957+0000","last_active":"2024-02-23T00:43:16.533032+0000","last_peered":"2024-02-23T00:43:16.533032+0000","last_clean":"2024-02-23T00:43:16.533032+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:16.533032+0000","last_undegraded":"2024-02-23T00:43:16.533032+0000","last_fullsized":"2024-02-23T00:43:16.533032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"59.2","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.661302+0000","last_change":"2024-02-23T00:43:16.661302+0000","last_active":"2024-02-23T00:43:16.661302+0000","last_peered":"2024-02-23T00:43:16.661302+0000","last_clean":"2024-02-23T00:43:16.661302+0000","last_became_active":"2024-02-23T00:43:16.660954+0000","last_became_peered":"2024-02-23T00:43:16.660954+0000","last_unstale":"2024-02-23T00:43:16.661302+0000","last_undegraded":"2024-02-23T00:43:16.661302+0000","last_fullsized":"2024-02-23T00:43:16.661302+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_clean_scrub_stamp":"2024-02-23T00:43:15.522490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.d","version":"0'0","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.561034+0000","last_change":"2024-02-23T00:43:10.905012+0000","last_active":"2024-02-23T00:43:15.561034+0000","last_peered":"2024-02-23T00:43:15.561034+0000","last_clean":"2024-02-23T00:43:15.561034+0000","last_became_active":"2024-02-23T00:43:10.904630+0000","last_became_peered":"2024-02-23T00:43:10.904630+0000","last_unstale":"2024-02-23T00:43:15.561034+0000","last_undegraded":"2024-02-23T00:43:15.561034+0000","last_fullsized":"2024-02-23T00:43:15.561034+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_clean_scrub_stamp":"2024-02-23T00:43:09.833908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"47.16","version":"0'0","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.542293+0000","last_change":"2024-02-23T00:43:08.887758+0000","last_active":"2024-02-23T00:43:15.542293+0000","last_peered":"2024-02-23T00:43:15.542293+0000","last_clean":"2024-02-23T00:43:15.542293+0000","last_became_active":"2024-02-23T00:43:08.887509+0000","last_became_peered":"2024-02-23T00:43:08.887509+0000","last_unstale":"2024-02-23T00:43:15.542293+0000","last_undegraded":"2024-02-23T00:43:15.542293+0000","last_fullsized":"2024-02-23T00:43:15.542293+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.13","version":"120'2","reported_seq":21,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.548996+0000","last_change":"2024-02-23T00:43:05.269578+0000","last_active":"2024-02-23T00:43:15.548996+0000","last_peered":"2024-02-23T00:43:15.548996+0000","last_clean":"2024-02-23T00:43:15.548996+0000","last_became_active":"2024-02-23T00:43:05.269369+0000","last_became_peered":"2024-02-23T00:43:05.269369+0000","last_unstale":"2024-02-23T00:43:15.548996+0000","last_undegraded":"2024-02-23T00:43:15.548996+0000","last_fullsized":"2024-02-23T00:43:15.548996+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.197452+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8192,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"35.1a","version":"0'0","reported_seq":22,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.553025+0000","last_change":"2024-02-23T00:43:03.238767+0000","last_active":"2024-02-23T00:43:15.553025+0000","last_peered":"2024-02-23T00:43:15.553025+0000","last_clean":"2024-02-23T00:43:15.553025+0000","last_became_active":"2024-02-23T00:43:03.238038+0000","last_became_peered":"2024-02-23T00:43:03.238038+0000","last_unstale":"2024-02-23T00:43:15.553025+0000","last_undegraded":"2024-02-23T00:43:15.553025+0000","last_fullsized":"2024-02-23T00:43:15.553025+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_clean_scrub_stamp":"2024-02-23T00:43:00.812733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"63.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:16.533032+0000","last_change":"2024-02-23T00:43:16.581163+0000","last_active":"2024-02-23T00:43:16.533032+0000","last_peered":"2024-02-23T00:43:16.533032+0000","last_clean":"2024-02-23T00:43:16.533032+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:16.533032+0000","last_undegraded":"2024-02-23T00:43:16.533032+0000","last_fullsized":"2024-02-23T00:43:16.533032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"59.d","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.658097+0000","last_change":"2024-02-23T00:43:16.658097+0000","last_active":"2024-02-23T00:43:16.658097+0000","last_peered":"2024-02-23T00:43:16.658097+0000","last_clean":"2024-02-23T00:43:16.658097+0000","last_became_active":"2024-02-23T00:43:16.657827+0000","last_became_peered":"2024-02-23T00:43:16.657827+0000","last_unstale":"2024-02-23T00:43:16.658097+0000","last_undegraded":"2024-02-23T00:43:16.658097+0000","last_fullsized":"2024-02-23T00:43:16.658097+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_clean_scrub_stamp":"2024-02-23T00:43:15.522490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,0],"acting":[2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"55.1","version":"125'8","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.642954+0000","last_change":"2024-02-23T00:43:13.722878+0000","last_active":"2024-02-23T00:43:15.642954+0000","last_peered":"2024-02-23T00:43:15.642954+0000","last_clean":"2024-02-23T00:43:15.642954+0000","last_became_active":"2024-02-23T00:43:13.722440+0000","last_became_peered":"2024-02-23T00:43:13.722440+0000","last_unstale":"2024-02-23T00:43:15.642954+0000","last_undegraded":"2024-02-23T00:43:15.642954+0000","last_fullsized":"2024-02-23T00:43:15.642954+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:12.235460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:12.235460+0000","last_clean_scrub_stamp":"2024-02-23T00:43:12.235460+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.2","version":"0'0","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.552459+0000","last_change":"2024-02-23T00:43:10.901850+0000","last_active":"2024-02-23T00:43:15.552459+0000","last_peered":"2024-02-23T00:43:15.552459+0000","last_clean":"2024-02-23T00:43:15.552459+0000","last_became_active":"2024-02-23T00:43:10.901531+0000","last_became_peered":"2024-02-23T00:43:10.901531+0000","last_unstale":"2024-02-23T00:43:15.552459+0000","last_undegraded":"2024-02-23T00:43:15.552459+0000","last_fullsized":"2024-02-23T00:43:15.552459+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_clean_scrub_stamp":"2024-02-23T00:43:09.833908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"47.19","version":"0'0","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.541049+0000","last_change":"2024-02-23T00:43:09.914057+0000","last_active":"2024-02-23T00:43:15.541049+0000","last_peered":"2024-02-23T00:43:15.541049+0000","last_clean":"2024-02-23T00:43:15.541049+0000","last_became_active":"2024-02-23T00:43:09.913739+0000","last_became_peered":"2024-02-23T00:43:09.913739+0000","last_unstale":"2024-02-23T00:43:15.541049+0000","last_undegraded":"2024-02-23T00:43:15.541049+0000","last_fullsized":"2024-02-23T00:43:15.541049+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.1c","version":"0'0","reported_seq":19,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.571649+0000","last_change":"2024-02-23T00:43:05.292907+0000","last_active":"2024-02-23T00:43:15.571649+0000","last_peered":"2024-02-23T00:43:15.571649+0000","last_clean":"2024-02-23T00:43:15.571649+0000","last_became_active":"2024-02-23T00:43:05.292596+0000","last_became_peered":"2024-02-23T00:43:05.292596+0000","last_unstale":"2024-02-23T00:43:15.571649+0000","last_undegraded":"2024-02-23T00:43:15.571649+0000","last_fullsized":"2024-02-23T00:43:15.571649+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.197452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.15","version":"0'0","reported_seq":22,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.542919+0000","last_change":"2024-02-23T00:43:02.345843+0000","last_active":"2024-02-23T00:43:15.542919+0000","last_peered":"2024-02-23T00:43:15.542919+0000","last_clean":"2024-02-23T00:43:15.542919+0000","last_became_active":"2024-02-23T00:43:02.345577+0000","last_became_peered":"2024-02-23T00:43:02.345577+0000","last_unstale":"2024-02-23T00:43:15.542919+0000","last_undegraded":"2024-02-23T00:43:15.542919+0000","last_fullsized":"2024-02-23T00:43:15.542919+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_clean_scrub_stamp":"2024-02-23T00:43:00.812733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"63.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:16.533032+0000","last_change":"2024-02-23T00:43:16.551706+0000","last_active":"2024-02-23T00:43:16.533032+0000","last_peered":"2024-02-23T00:43:16.533032+0000","last_clean":"2024-02-23T00:43:16.533032+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:16.533032+0000","last_undegraded":"2024-02-23T00:43:16.533032+0000","last_fullsized":"2024-02-23T00:43:16.533032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"59.c","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.654674+0000","last_change":"2024-02-23T00:43:16.654674+0000","last_active":"2024-02-23T00:43:16.654674+0000","last_peered":"2024-02-23T00:43:16.654674+0000","last_clean":"2024-02-23T00:43:16.654674+0000","last_became_active":"2024-02-23T00:43:16.654332+0000","last_became_peered":"2024-02-23T00:43:16.654332+0000","last_unstale":"2024-02-23T00:43:16.654674+0000","last_undegraded":"2024-02-23T00:43:16.654674+0000","last_fullsized":"2024-02-23T00:43:16.654674+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_clean_scrub_stamp":"2024-02-23T00:43:15.522490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"55.0","version":"125'12","reported_seq":30,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.671012+0000","last_change":"2024-02-23T00:43:13.714977+0000","last_active":"2024-02-23T00:43:15.671012+0000","last_peered":"2024-02-23T00:43:15.671012+0000","last_clean":"2024-02-23T00:43:15.671012+0000","last_became_active":"2024-02-23T00:43:13.714334+0000","last_became_peered":"2024-02-23T00:43:13.714334+0000","last_unstale":"2024-02-23T00:43:15.671012+0000","last_undegraded":"2024-02-23T00:43:15.671012+0000","last_fullsized":"2024-02-23T00:43:15.671012+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:12.235460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:12.235460+0000","last_clean_scrub_stamp":"2024-02-23T00:43:12.235460+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":384,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.3","version":"0'0","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.548251+0000","last_change":"2024-02-23T00:43:10.886896+0000","last_active":"2024-02-23T00:43:15.548251+0000","last_peered":"2024-02-23T00:43:15.548251+0000","last_clean":"2024-02-23T00:43:15.548251+0000","last_became_active":"2024-02-23T00:43:10.886548+0000","last_became_peered":"2024-02-23T00:43:10.886548+0000","last_unstale":"2024-02-23T00:43:15.548251+0000","last_undegraded":"2024-02-23T00:43:15.548251+0000","last_fullsized":"2024-02-23T00:43:15.548251+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_clean_scrub_stamp":"2024-02-23T00:43:09.833908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"47.18","version":"0'0","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.544542+0000","last_change":"2024-02-23T00:43:08.861765+0000","last_active":"2024-02-23T00:43:15.544542+0000","last_peered":"2024-02-23T00:43:15.544542+0000","last_clean":"2024-02-23T00:43:15.544542+0000","last_became_active":"2024-02-23T00:43:08.861457+0000","last_became_peered":"2024-02-23T00:43:08.861457+0000","last_unstale":"2024-02-23T00:43:15.544542+0000","last_undegraded":"2024-02-23T00:43:15.544542+0000","last_fullsized":"2024-02-23T00:43:15.544542+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"42.1d","version":"0'0","reported_seq":19,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.545791+0000","last_change":"2024-02-23T00:43:05.301365+0000","last_active":"2024-02-23T00:43:15.545791+0000","last_peered":"2024-02-23T00:43:15.545791+0000","last_clean":"2024-02-23T00:43:15.545791+0000","last_became_active":"2024-02-23T00:43:05.301047+0000","last_became_peered":"2024-02-23T00:43:05.301047+0000","last_unstale":"2024-02-23T00:43:15.545791+0000","last_undegraded":"2024-02-23T00:43:15.545791+0000","last_fullsized":"2024-02-23T00:43:15.545791+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.197452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.14","version":"0'0","reported_seq":22,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.552325+0000","last_change":"2024-02-23T00:43:02.344258+0000","last_active":"2024-02-23T00:43:15.552325+0000","last_peered":"2024-02-23T00:43:15.552325+0000","last_clean":"2024-02-23T00:43:15.552325+0000","last_became_active":"2024-02-23T00:43:02.343935+0000","last_became_peered":"2024-02-23T00:43:02.343935+0000","last_unstale":"2024-02-23T00:43:15.552325+0000","last_undegraded":"2024-02-23T00:43:15.552325+0000","last_fullsized":"2024-02-23T00:43:15.552325+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_clean_scrub_stamp":"2024-02-23T00:43:00.812733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"63.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:16.533032+0000","last_change":"2024-02-23T00:43:16.577210+0000","last_active":"2024-02-23T00:43:16.533032+0000","last_peered":"2024-02-23T00:43:16.533032+0000","last_clean":"2024-02-23T00:43:16.533032+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:16.533032+0000","last_undegraded":"2024-02-23T00:43:16.533032+0000","last_fullsized":"2024-02-23T00:43:16.533032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"59.f","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.632824+0000","last_change":"2024-02-23T00:43:16.632824+0000","last_active":"2024-02-23T00:43:16.632824+0000","last_peered":"2024-02-23T00:43:16.632824+0000","last_clean":"2024-02-23T00:43:16.632824+0000","last_became_active":"2024-02-23T00:43:16.629061+0000","last_became_peered":"2024-02-23T00:43:16.629061+0000","last_unstale":"2024-02-23T00:43:16.632824+0000","last_undegraded":"2024-02-23T00:43:16.632824+0000","last_fullsized":"2024-02-23T00:43:16.632824+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_clean_scrub_stamp":"2024-02-23T00:43:15.522490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"55.3","version":"125'8","reported_seq":23,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.660228+0000","last_change":"2024-02-23T00:43:13.708097+0000","last_active":"2024-02-23T00:43:15.660228+0000","last_peered":"2024-02-23T00:43:15.660228+0000","last_clean":"2024-02-23T00:43:15.660228+0000","last_became_active":"2024-02-23T00:43:13.707759+0000","last_became_peered":"2024-02-23T00:43:13.707759+0000","last_unstale":"2024-02-23T00:43:15.660228+0000","last_undegraded":"2024-02-23T00:43:15.660228+0000","last_fullsized":"2024-02-23T00:43:15.660228+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:12.235460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:12.235460+0000","last_clean_scrub_stamp":"2024-02-23T00:43:12.235460+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.0","version":"0'0","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.537162+0000","last_change":"2024-02-23T00:43:10.885152+0000","last_active":"2024-02-23T00:43:15.537162+0000","last_peered":"2024-02-23T00:43:15.537162+0000","last_clean":"2024-02-23T00:43:15.537162+0000","last_became_active":"2024-02-23T00:43:10.884259+0000","last_became_peered":"2024-02-23T00:43:10.884259+0000","last_unstale":"2024-02-23T00:43:15.537162+0000","last_undegraded":"2024-02-23T00:43:15.537162+0000","last_fullsized":"2024-02-23T00:43:15.537162+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_clean_scrub_stamp":"2024-02-23T00:43:09.833908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"47.1b","version":"0'0","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.560497+0000","last_change":"2024-02-23T00:43:08.889561+0000","last_active":"2024-02-23T00:43:15.560497+0000","last_peered":"2024-02-23T00:43:15.560497+0000","last_clean":"2024-02-23T00:43:15.560497+0000","last_became_active":"2024-02-23T00:43:08.889268+0000","last_became_peered":"2024-02-23T00:43:08.889268+0000","last_unstale":"2024-02-23T00:43:15.560497+0000","last_undegraded":"2024-02-23T00:43:15.560497+0000","last_fullsized":"2024-02-23T00:43:15.560497+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.1e","version":"0'0","reported_seq":19,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.575795+0000","last_change":"2024-02-23T00:43:05.297351+0000","last_active":"2024-02-23T00:43:15.575795+0000","last_peered":"2024-02-23T00:43:15.575795+0000","last_clean":"2024-02-23T00:43:15.575795+0000","last_became_active":"2024-02-23T00:43:05.297046+0000","last_became_peered":"2024-02-23T00:43:05.297046+0000","last_unstale":"2024-02-23T00:43:15.575795+0000","last_undegraded":"2024-02-23T00:43:15.575795+0000","last_fullsized":"2024-02-23T00:43:15.575795+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.197452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.17","version":"0'0","reported_seq":22,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.561866+0000","last_change":"2024-02-23T00:43:02.346123+0000","last_active":"2024-02-23T00:43:15.561866+0000","last_peered":"2024-02-23T00:43:15.561866+0000","last_clean":"2024-02-23T00:43:15.561866+0000","last_became_active":"2024-02-23T00:43:02.345808+0000","last_became_peered":"2024-02-23T00:43:02.345808+0000","last_unstale":"2024-02-23T00:43:15.561866+0000","last_undegraded":"2024-02-23T00:43:15.561866+0000","last_fullsized":"2024-02-23T00:43:15.561866+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_clean_scrub_stamp":"2024-02-23T00:43:00.812733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"63.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:16.533032+0000","last_change":"2024-02-23T00:43:16.585464+0000","last_active":"2024-02-23T00:43:16.533032+0000","last_peered":"2024-02-23T00:43:16.533032+0000","last_clean":"2024-02-23T00:43:16.533032+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:16.533032+0000","last_undegraded":"2024-02-23T00:43:16.533032+0000","last_fullsized":"2024-02-23T00:43:16.533032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"59.e","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.632253+0000","last_change":"2024-02-23T00:43:16.632253+0000","last_active":"2024-02-23T00:43:16.632253+0000","last_peered":"2024-02-23T00:43:16.632253+0000","last_clean":"2024-02-23T00:43:16.632253+0000","last_became_active":"2024-02-23T00:43:16.631953+0000","last_became_peered":"2024-02-23T00:43:16.631953+0000","last_unstale":"2024-02-23T00:43:16.632253+0000","last_undegraded":"2024-02-23T00:43:16.632253+0000","last_fullsized":"2024-02-23T00:43:16.632253+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_clean_scrub_stamp":"2024-02-23T00:43:15.522490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"55.2","version":"126'12","reported_seq":32,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.583812+0000","last_change":"2024-02-23T00:43:13.485504+0000","last_active":"2024-02-23T00:43:16.583812+0000","last_peered":"2024-02-23T00:43:16.583812+0000","last_clean":"2024-02-23T00:43:16.583812+0000","last_became_active":"2024-02-23T00:43:13.485182+0000","last_became_peered":"2024-02-23T00:43:13.485182+0000","last_unstale":"2024-02-23T00:43:16.583812+0000","last_undegraded":"2024-02-23T00:43:16.583812+0000","last_fullsized":"2024-02-23T00:43:16.583812+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:12.235460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:12.235460+0000","last_clean_scrub_stamp":"2024-02-23T00:43:12.235460+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":384,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.1","version":"0'0","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.549972+0000","last_change":"2024-02-23T00:43:10.902812+0000","last_active":"2024-02-23T00:43:15.549972+0000","last_peered":"2024-02-23T00:43:15.549972+0000","last_clean":"2024-02-23T00:43:15.549972+0000","last_became_active":"2024-02-23T00:43:10.902502+0000","last_became_peered":"2024-02-23T00:43:10.902502+0000","last_unstale":"2024-02-23T00:43:15.549972+0000","last_undegraded":"2024-02-23T00:43:15.549972+0000","last_fullsized":"2024-02-23T00:43:15.549972+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_clean_scrub_stamp":"2024-02-23T00:43:09.833908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"47.1a","version":"0'0","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.537352+0000","last_change":"2024-02-23T00:43:08.879558+0000","last_active":"2024-02-23T00:43:15.537352+0000","last_peered":"2024-02-23T00:43:15.537352+0000","last_clean":"2024-02-23T00:43:15.537352+0000","last_became_active":"2024-02-23T00:43:08.879255+0000","last_became_peered":"2024-02-23T00:43:08.879255+0000","last_unstale":"2024-02-23T00:43:15.537352+0000","last_undegraded":"2024-02-23T00:43:15.537352+0000","last_fullsized":"2024-02-23T00:43:15.537352+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.1f","version":"0'0","reported_seq":19,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.541423+0000","last_change":"2024-02-23T00:43:05.265088+0000","last_active":"2024-02-23T00:43:15.541423+0000","last_peered":"2024-02-23T00:43:15.541423+0000","last_clean":"2024-02-23T00:43:15.541423+0000","last_became_active":"2024-02-23T00:43:05.264779+0000","last_became_peered":"2024-02-23T00:43:05.264779+0000","last_unstale":"2024-02-23T00:43:15.541423+0000","last_undegraded":"2024-02-23T00:43:15.541423+0000","last_fullsized":"2024-02-23T00:43:15.541423+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.197452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"35.16","version":"0'0","reported_seq":22,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.551253+0000","last_change":"2024-02-23T00:43:02.338283+0000","last_active":"2024-02-23T00:43:15.551253+0000","last_peered":"2024-02-23T00:43:15.551253+0000","last_clean":"2024-02-23T00:43:15.551253+0000","last_became_active":"2024-02-23T00:43:02.337992+0000","last_became_peered":"2024-02-23T00:43:02.337992+0000","last_unstale":"2024-02-23T00:43:15.551253+0000","last_undegraded":"2024-02-23T00:43:15.551253+0000","last_fullsized":"2024-02-23T00:43:15.551253+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_clean_scrub_stamp":"2024-02-23T00:43:00.812733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"63.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:16.533032+0000","last_change":"2024-02-23T00:43:16.559022+0000","last_active":"2024-02-23T00:43:16.533032+0000","last_peered":"2024-02-23T00:43:16.533032+0000","last_clean":"2024-02-23T00:43:16.533032+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:16.533032+0000","last_undegraded":"2024-02-23T00:43:16.533032+0000","last_fullsized":"2024-02-23T00:43:16.533032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"59.9","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.635741+0000","last_change":"2024-02-23T00:43:16.635741+0000","last_active":"2024-02-23T00:43:16.635741+0000","last_peered":"2024-02-23T00:43:16.635741+0000","last_clean":"2024-02-23T00:43:16.635741+0000","last_became_active":"2024-02-23T00:43:16.634876+0000","last_became_peered":"2024-02-23T00:43:16.634876+0000","last_unstale":"2024-02-23T00:43:16.635741+0000","last_undegraded":"2024-02-23T00:43:16.635741+0000","last_fullsized":"2024-02-23T00:43:16.635741+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_clean_scrub_stamp":"2024-02-23T00:43:15.522490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.6","version":"0'0","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.557136+0000","last_change":"2024-02-23T00:43:10.896109+0000","last_active":"2024-02-23T00:43:15.557136+0000","last_peered":"2024-02-23T00:43:15.557136+0000","last_clean":"2024-02-23T00:43:15.557136+0000","last_became_active":"2024-02-23T00:43:10.895809+0000","last_became_peered":"2024-02-23T00:43:10.895809+0000","last_unstale":"2024-02-23T00:43:15.557136+0000","last_undegraded":"2024-02-23T00:43:15.557136+0000","last_fullsized":"2024-02-23T00:43:15.557136+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_clean_scrub_stamp":"2024-02-23T00:43:09.833908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"47.1d","version":"0'0","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.542408+0000","last_change":"2024-02-23T00:43:08.876647+0000","last_active":"2024-02-23T00:43:15.542408+0000","last_peered":"2024-02-23T00:43:15.542408+0000","last_clean":"2024-02-23T00:43:15.542408+0000","last_became_active":"2024-02-23T00:43:08.876321+0000","last_became_peered":"2024-02-23T00:43:08.876321+0000","last_unstale":"2024-02-23T00:43:15.542408+0000","last_undegraded":"2024-02-23T00:43:15.542408+0000","last_fullsized":"2024-02-23T00:43:15.542408+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.18","version":"0'0","reported_seq":19,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.543810+0000","last_change":"2024-02-23T00:43:05.262703+0000","last_active":"2024-02-23T00:43:15.543810+0000","last_peered":"2024-02-23T00:43:15.543810+0000","last_clean":"2024-02-23T00:43:15.543810+0000","last_became_active":"2024-02-23T00:43:05.262412+0000","last_became_peered":"2024-02-23T00:43:05.262412+0000","last_unstale":"2024-02-23T00:43:15.543810+0000","last_undegraded":"2024-02-23T00:43:15.543810+0000","last_fullsized":"2024-02-23T00:43:15.543810+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.197452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.11","version":"0'0","reported_seq":22,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.541837+0000","last_change":"2024-02-23T00:43:02.368641+0000","last_active":"2024-02-23T00:43:15.541837+0000","last_peered":"2024-02-23T00:43:15.541837+0000","last_clean":"2024-02-23T00:43:15.541837+0000","last_became_active":"2024-02-23T00:43:02.368348+0000","last_became_peered":"2024-02-23T00:43:02.368348+0000","last_unstale":"2024-02-23T00:43:15.541837+0000","last_undegraded":"2024-02-23T00:43:15.541837+0000","last_fullsized":"2024-02-23T00:43:15.541837+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_clean_scrub_stamp":"2024-02-23T00:43:00.812733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"63.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:16.533032+0000","last_change":"2024-02-23T00:43:16.550557+0000","last_active":"2024-02-23T00:43:16.533032+0000","last_peered":"2024-02-23T00:43:16.533032+0000","last_clean":"2024-02-23T00:43:16.533032+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:16.533032+0000","last_undegraded":"2024-02-23T00:43:16.533032+0000","last_fullsized":"2024-02-23T00:43:16.533032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"59.8","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.672343+0000","last_change":"2024-02-23T00:43:16.672343+0000","last_active":"2024-02-23T00:43:16.672343+0000","last_peered":"2024-02-23T00:43:16.672343+0000","last_clean":"2024-02-23T00:43:16.672343+0000","last_became_active":"2024-02-23T00:43:16.671502+0000","last_became_peered":"2024-02-23T00:43:16.671502+0000","last_unstale":"2024-02-23T00:43:16.672343+0000","last_undegraded":"2024-02-23T00:43:16.672343+0000","last_fullsized":"2024-02-23T00:43:16.672343+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_clean_scrub_stamp":"2024-02-23T00:43:15.522490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.7","version":"0'0","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.554279+0000","last_change":"2024-02-23T00:43:10.908496+0000","last_active":"2024-02-23T00:43:15.554279+0000","last_peered":"2024-02-23T00:43:15.554279+0000","last_clean":"2024-02-23T00:43:15.554279+0000","last_became_active":"2024-02-23T00:43:10.908227+0000","last_became_peered":"2024-02-23T00:43:10.908227+0000","last_unstale":"2024-02-23T00:43:15.554279+0000","last_undegraded":"2024-02-23T00:43:15.554279+0000","last_fullsized":"2024-02-23T00:43:15.554279+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_clean_scrub_stamp":"2024-02-23T00:43:09.833908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"47.1c","version":"0'0","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.571487+0000","last_change":"2024-02-23T00:43:08.891941+0000","last_active":"2024-02-23T00:43:15.571487+0000","last_peered":"2024-02-23T00:43:15.571487+0000","last_clean":"2024-02-23T00:43:15.571487+0000","last_became_active":"2024-02-23T00:43:08.891349+0000","last_became_peered":"2024-02-23T00:43:08.891349+0000","last_unstale":"2024-02-23T00:43:15.571487+0000","last_undegraded":"2024-02-23T00:43:15.571487+0000","last_fullsized":"2024-02-23T00:43:15.571487+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.19","version":"0'0","reported_seq":19,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.536995+0000","last_change":"2024-02-23T00:43:05.269052+0000","last_active":"2024-02-23T00:43:15.536995+0000","last_peered":"2024-02-23T00:43:15.536995+0000","last_clean":"2024-02-23T00:43:15.536995+0000","last_became_active":"2024-02-23T00:43:05.268770+0000","last_became_peered":"2024-02-23T00:43:05.268770+0000","last_unstale":"2024-02-23T00:43:15.536995+0000","last_undegraded":"2024-02-23T00:43:15.536995+0000","last_fullsized":"2024-02-23T00:43:15.536995+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.197452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"35.10","version":"0'0","reported_seq":22,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.545196+0000","last_change":"2024-02-23T00:43:02.341762+0000","last_active":"2024-02-23T00:43:15.545196+0000","last_peered":"2024-02-23T00:43:15.545196+0000","last_clean":"2024-02-23T00:43:15.545196+0000","last_became_active":"2024-02-23T00:43:02.341479+0000","last_became_peered":"2024-02-23T00:43:02.341479+0000","last_unstale":"2024-02-23T00:43:15.545196+0000","last_undegraded":"2024-02-23T00:43:15.545196+0000","last_fullsized":"2024-02-23T00:43:15.545196+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_clean_scrub_stamp":"2024-02-23T00:43:00.812733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"63.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:16.533032+0000","last_change":"2024-02-23T00:43:16.603843+0000","last_active":"2024-02-23T00:43:16.533032+0000","last_peered":"2024-02-23T00:43:16.533032+0000","last_clean":"2024-02-23T00:43:16.533032+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:16.533032+0000","last_undegraded":"2024-02-23T00:43:16.533032+0000","last_fullsized":"2024-02-23T00:43:16.533032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"59.b","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.634669+0000","last_change":"2024-02-23T00:43:16.634669+0000","last_active":"2024-02-23T00:43:16.634669+0000","last_peered":"2024-02-23T00:43:16.634669+0000","last_clean":"2024-02-23T00:43:16.634669+0000","last_became_active":"2024-02-23T00:43:16.634241+0000","last_became_peered":"2024-02-23T00:43:16.634241+0000","last_unstale":"2024-02-23T00:43:16.634669+0000","last_undegraded":"2024-02-23T00:43:16.634669+0000","last_fullsized":"2024-02-23T00:43:16.634669+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_clean_scrub_stamp":"2024-02-23T00:43:15.522490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.4","version":"0'0","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.537463+0000","last_change":"2024-02-23T00:43:10.910904+0000","last_active":"2024-02-23T00:43:15.537463+0000","last_peered":"2024-02-23T00:43:15.537463+0000","last_clean":"2024-02-23T00:43:15.537463+0000","last_became_active":"2024-02-23T00:43:10.910544+0000","last_became_peered":"2024-02-23T00:43:10.910544+0000","last_unstale":"2024-02-23T00:43:15.537463+0000","last_undegraded":"2024-02-23T00:43:15.537463+0000","last_fullsized":"2024-02-23T00:43:15.537463+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_clean_scrub_stamp":"2024-02-23T00:43:09.833908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,1],"acting":[6,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"47.1f","version":"0'0","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.554381+0000","last_change":"2024-02-23T00:43:08.895615+0000","last_active":"2024-02-23T00:43:15.554381+0000","last_peered":"2024-02-23T00:43:15.554381+0000","last_clean":"2024-02-23T00:43:15.554381+0000","last_became_active":"2024-02-23T00:43:08.895310+0000","last_became_peered":"2024-02-23T00:43:08.895310+0000","last_unstale":"2024-02-23T00:43:15.554381+0000","last_undegraded":"2024-02-23T00:43:15.554381+0000","last_fullsized":"2024-02-23T00:43:15.554381+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"42.1a","version":"0'0","reported_seq":19,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.557362+0000","last_change":"2024-02-23T00:43:05.293987+0000","last_active":"2024-02-23T00:43:15.557362+0000","last_peered":"2024-02-23T00:43:15.557362+0000","last_clean":"2024-02-23T00:43:15.557362+0000","last_became_active":"2024-02-23T00:43:05.293614+0000","last_became_peered":"2024-02-23T00:43:05.293614+0000","last_unstale":"2024-02-23T00:43:15.557362+0000","last_undegraded":"2024-02-23T00:43:15.557362+0000","last_fullsized":"2024-02-23T00:43:15.557362+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.197452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.13","version":"0'0","reported_seq":22,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.550353+0000","last_change":"2024-02-23T00:43:02.354760+0000","last_active":"2024-02-23T00:43:15.550353+0000","last_peered":"2024-02-23T00:43:15.550353+0000","last_clean":"2024-02-23T00:43:15.550353+0000","last_became_active":"2024-02-23T00:43:02.354504+0000","last_became_peered":"2024-02-23T00:43:02.354504+0000","last_unstale":"2024-02-23T00:43:15.550353+0000","last_undegraded":"2024-02-23T00:43:15.550353+0000","last_fullsized":"2024-02-23T00:43:15.550353+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_clean_scrub_stamp":"2024-02-23T00:43:00.812733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"63.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:16.533032+0000","last_change":"2024-02-23T00:43:16.549486+0000","last_active":"2024-02-23T00:43:16.533032+0000","last_peered":"2024-02-23T00:43:16.533032+0000","last_clean":"2024-02-23T00:43:16.533032+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:16.533032+0000","last_undegraded":"2024-02-23T00:43:16.533032+0000","last_fullsized":"2024-02-23T00:43:16.533032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"59.a","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.631086+0000","last_change":"2024-02-23T00:43:16.631086+0000","last_active":"2024-02-23T00:43:16.631086+0000","last_peered":"2024-02-23T00:43:16.631086+0000","last_clean":"2024-02-23T00:43:16.631086+0000","last_became_active":"2024-02-23T00:43:16.630799+0000","last_became_peered":"2024-02-23T00:43:16.630799+0000","last_unstale":"2024-02-23T00:43:16.631086+0000","last_undegraded":"2024-02-23T00:43:16.631086+0000","last_fullsized":"2024-02-23T00:43:16.631086+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_clean_scrub_stamp":"2024-02-23T00:43:15.522490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.5","version":"0'0","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.545692+0000","last_change":"2024-02-23T00:43:10.899416+0000","last_active":"2024-02-23T00:43:15.545692+0000","last_peered":"2024-02-23T00:43:15.545692+0000","last_clean":"2024-02-23T00:43:15.545692+0000","last_became_active":"2024-02-23T00:43:10.899101+0000","last_became_peered":"2024-02-23T00:43:10.899101+0000","last_unstale":"2024-02-23T00:43:15.545692+0000","last_undegraded":"2024-02-23T00:43:15.545692+0000","last_fullsized":"2024-02-23T00:43:15.545692+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_clean_scrub_stamp":"2024-02-23T00:43:09.833908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"47.1e","version":"0'0","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.543232+0000","last_change":"2024-02-23T00:43:09.908968+0000","last_active":"2024-02-23T00:43:15.543232+0000","last_peered":"2024-02-23T00:43:15.543232+0000","last_clean":"2024-02-23T00:43:15.543232+0000","last_became_active":"2024-02-23T00:43:09.908444+0000","last_became_peered":"2024-02-23T00:43:09.908444+0000","last_unstale":"2024-02-23T00:43:15.543232+0000","last_undegraded":"2024-02-23T00:43:15.543232+0000","last_fullsized":"2024-02-23T00:43:15.543232+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.1b","version":"0'0","reported_seq":19,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.558557+0000","last_change":"2024-02-23T00:43:05.288120+0000","last_active":"2024-02-23T00:43:15.558557+0000","last_peered":"2024-02-23T00:43:15.558557+0000","last_clean":"2024-02-23T00:43:15.558557+0000","last_became_active":"2024-02-23T00:43:05.287582+0000","last_became_peered":"2024-02-23T00:43:05.287582+0000","last_unstale":"2024-02-23T00:43:15.558557+0000","last_undegraded":"2024-02-23T00:43:15.558557+0000","last_fullsized":"2024-02-23T00:43:15.558557+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.197452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,4],"acting":[2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.12","version":"0'0","reported_seq":22,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.543257+0000","last_change":"2024-02-23T00:43:02.368721+0000","last_active":"2024-02-23T00:43:15.543257+0000","last_peered":"2024-02-23T00:43:15.543257+0000","last_clean":"2024-02-23T00:43:15.543257+0000","last_became_active":"2024-02-23T00:43:02.368449+0000","last_became_peered":"2024-02-23T00:43:02.368449+0000","last_unstale":"2024-02-23T00:43:15.543257+0000","last_undegraded":"2024-02-23T00:43:15.543257+0000","last_fullsized":"2024-02-23T00:43:15.543257+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_clean_scrub_stamp":"2024-02-23T00:43:00.812733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1],"acting":[6,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"63.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:16.533032+0000","last_change":"2024-02-23T00:43:16.568538+0000","last_active":"2024-02-23T00:43:16.533032+0000","last_peered":"2024-02-23T00:43:16.533032+0000","last_clean":"2024-02-23T00:43:16.533032+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:16.533032+0000","last_undegraded":"2024-02-23T00:43:16.533032+0000","last_fullsized":"2024-02-23T00:43:16.533032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"59.15","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.588407+0000","last_change":"2024-02-23T00:43:16.588407+0000","last_active":"2024-02-23T00:43:16.588407+0000","last_peered":"2024-02-23T00:43:16.588407+0000","last_clean":"2024-02-23T00:43:16.588407+0000","last_became_active":"2024-02-23T00:43:16.588029+0000","last_became_peered":"2024-02-23T00:43:16.588029+0000","last_unstale":"2024-02-23T00:43:16.588407+0000","last_undegraded":"2024-02-23T00:43:16.588407+0000","last_fullsized":"2024-02-23T00:43:16.588407+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_clean_scrub_stamp":"2024-02-23T00:43:15.522490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.1a","version":"0'0","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.546108+0000","last_change":"2024-02-23T00:43:10.888840+0000","last_active":"2024-02-23T00:43:15.546108+0000","last_peered":"2024-02-23T00:43:15.546108+0000","last_clean":"2024-02-23T00:43:15.546108+0000","last_became_active":"2024-02-23T00:43:10.888510+0000","last_became_peered":"2024-02-23T00:43:10.888510+0000","last_unstale":"2024-02-23T00:43:15.546108+0000","last_undegraded":"2024-02-23T00:43:15.546108+0000","last_fullsized":"2024-02-23T00:43:15.546108+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_clean_scrub_stamp":"2024-02-23T00:43:09.833908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"47.1","version":"0'0","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.536905+0000","last_change":"2024-02-23T00:43:08.882233+0000","last_active":"2024-02-23T00:43:15.536905+0000","last_peered":"2024-02-23T00:43:15.536905+0000","last_clean":"2024-02-23T00:43:15.536905+0000","last_became_active":"2024-02-23T00:43:08.881844+0000","last_became_peered":"2024-02-23T00:43:08.881844+0000","last_unstale":"2024-02-23T00:43:15.536905+0000","last_undegraded":"2024-02-23T00:43:15.536905+0000","last_fullsized":"2024-02-23T00:43:15.536905+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.4","version":"0'0","reported_seq":19,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.553694+0000","last_change":"2024-02-23T00:43:05.258211+0000","last_active":"2024-02-23T00:43:15.553694+0000","last_peered":"2024-02-23T00:43:15.553694+0000","last_clean":"2024-02-23T00:43:15.553694+0000","last_became_active":"2024-02-23T00:43:05.257966+0000","last_became_peered":"2024-02-23T00:43:05.257966+0000","last_unstale":"2024-02-23T00:43:15.553694+0000","last_undegraded":"2024-02-23T00:43:15.553694+0000","last_fullsized":"2024-02-23T00:43:15.553694+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.197452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.d","version":"0'0","reported_seq":22,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.545887+0000","last_change":"2024-02-23T00:43:02.367356+0000","last_active":"2024-02-23T00:43:15.545887+0000","last_peered":"2024-02-23T00:43:15.545887+0000","last_clean":"2024-02-23T00:43:15.545887+0000","last_became_active":"2024-02-23T00:43:02.366962+0000","last_became_peered":"2024-02-23T00:43:02.366962+0000","last_unstale":"2024-02-23T00:43:15.545887+0000","last_undegraded":"2024-02-23T00:43:15.545887+0000","last_fullsized":"2024-02-23T00:43:15.545887+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_clean_scrub_stamp":"2024-02-23T00:43:00.812733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"63.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:16.533032+0000","last_change":"2024-02-23T00:43:16.570754+0000","last_active":"2024-02-23T00:43:16.533032+0000","last_peered":"2024-02-23T00:43:16.533032+0000","last_clean":"2024-02-23T00:43:16.533032+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:16.533032+0000","last_undegraded":"2024-02-23T00:43:16.533032+0000","last_fullsized":"2024-02-23T00:43:16.533032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"59.14","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.608267+0000","last_change":"2024-02-23T00:43:16.608267+0000","last_active":"2024-02-23T00:43:16.608267+0000","last_peered":"2024-02-23T00:43:16.608267+0000","last_clean":"2024-02-23T00:43:16.608267+0000","last_became_active":"2024-02-23T00:43:16.607980+0000","last_became_peered":"2024-02-23T00:43:16.607980+0000","last_unstale":"2024-02-23T00:43:16.608267+0000","last_undegraded":"2024-02-23T00:43:16.608267+0000","last_fullsized":"2024-02-23T00:43:16.608267+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_clean_scrub_stamp":"2024-02-23T00:43:15.522490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7],"acting":[6,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"52.1b","version":"0'0","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.560213+0000","last_change":"2024-02-23T00:43:10.907187+0000","last_active":"2024-02-23T00:43:15.560213+0000","last_peered":"2024-02-23T00:43:15.560213+0000","last_clean":"2024-02-23T00:43:15.560213+0000","last_became_active":"2024-02-23T00:43:10.906869+0000","last_became_peered":"2024-02-23T00:43:10.906869+0000","last_unstale":"2024-02-23T00:43:15.560213+0000","last_undegraded":"2024-02-23T00:43:15.560213+0000","last_fullsized":"2024-02-23T00:43:15.560213+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_clean_scrub_stamp":"2024-02-23T00:43:09.833908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"47.0","version":"0'0","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.562889+0000","last_change":"2024-02-23T00:43:08.872662+0000","last_active":"2024-02-23T00:43:15.562889+0000","last_peered":"2024-02-23T00:43:15.562889+0000","last_clean":"2024-02-23T00:43:15.562889+0000","last_became_active":"2024-02-23T00:43:08.872188+0000","last_became_peered":"2024-02-23T00:43:08.872188+0000","last_unstale":"2024-02-23T00:43:15.562889+0000","last_undegraded":"2024-02-23T00:43:15.562889+0000","last_fullsized":"2024-02-23T00:43:15.562889+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.5","version":"123'2","reported_seq":21,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.545714+0000","last_change":"2024-02-23T00:43:05.297879+0000","last_active":"2024-02-23T00:43:15.545714+0000","last_peered":"2024-02-23T00:43:15.545714+0000","last_clean":"2024-02-23T00:43:15.545714+0000","last_became_active":"2024-02-23T00:43:05.297551+0000","last_became_peered":"2024-02-23T00:43:05.297551+0000","last_unstale":"2024-02-23T00:43:15.545714+0000","last_undegraded":"2024-02-23T00:43:15.545714+0000","last_fullsized":"2024-02-23T00:43:15.545714+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.197452+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8192,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"35.c","version":"0'0","reported_seq":22,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.540636+0000","last_change":"2024-02-23T00:43:02.331179+0000","last_active":"2024-02-23T00:43:15.540636+0000","last_peered":"2024-02-23T00:43:15.540636+0000","last_clean":"2024-02-23T00:43:15.540636+0000","last_became_active":"2024-02-23T00:43:02.330893+0000","last_became_peered":"2024-02-23T00:43:02.330893+0000","last_unstale":"2024-02-23T00:43:15.540636+0000","last_undegraded":"2024-02-23T00:43:15.540636+0000","last_fullsized":"2024-02-23T00:43:15.540636+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_clean_scrub_stamp":"2024-02-23T00:43:00.812733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"63.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:16.533032+0000","last_change":"2024-02-23T00:43:16.629045+0000","last_active":"2024-02-23T00:43:16.533032+0000","last_peered":"2024-02-23T00:43:16.533032+0000","last_clean":"2024-02-23T00:43:16.533032+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:16.533032+0000","last_undegraded":"2024-02-23T00:43:16.533032+0000","last_fullsized":"2024-02-23T00:43:16.533032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"59.17","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.653021+0000","last_change":"2024-02-23T00:43:16.653021+0000","last_active":"2024-02-23T00:43:16.653021+0000","last_peered":"2024-02-23T00:43:16.653021+0000","last_clean":"2024-02-23T00:43:16.653021+0000","last_became_active":"2024-02-23T00:43:16.652752+0000","last_became_peered":"2024-02-23T00:43:16.652752+0000","last_unstale":"2024-02-23T00:43:16.653021+0000","last_undegraded":"2024-02-23T00:43:16.653021+0000","last_fullsized":"2024-02-23T00:43:16.653021+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_clean_scrub_stamp":"2024-02-23T00:43:15.522490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"52.18","version":"0'0","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.542271+0000","last_change":"2024-02-23T00:43:10.920778+0000","last_active":"2024-02-23T00:43:15.542271+0000","last_peered":"2024-02-23T00:43:15.542271+0000","last_clean":"2024-02-23T00:43:15.542271+0000","last_became_active":"2024-02-23T00:43:10.920495+0000","last_became_peered":"2024-02-23T00:43:10.920495+0000","last_unstale":"2024-02-23T00:43:15.542271+0000","last_undegraded":"2024-02-23T00:43:15.542271+0000","last_fullsized":"2024-02-23T00:43:15.542271+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_clean_scrub_stamp":"2024-02-23T00:43:09.833908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"47.3","version":"0'0","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.536290+0000","last_change":"2024-02-23T00:43:09.904602+0000","last_active":"2024-02-23T00:43:15.536290+0000","last_peered":"2024-02-23T00:43:15.536290+0000","last_clean":"2024-02-23T00:43:15.536290+0000","last_became_active":"2024-02-23T00:43:09.904062+0000","last_became_peered":"2024-02-23T00:43:09.904062+0000","last_unstale":"2024-02-23T00:43:15.536290+0000","last_undegraded":"2024-02-23T00:43:15.536290+0000","last_fullsized":"2024-02-23T00:43:15.536290+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.6","version":"0'0","reported_seq":19,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.551110+0000","last_change":"2024-02-23T00:43:05.286801+0000","last_active":"2024-02-23T00:43:15.551110+0000","last_peered":"2024-02-23T00:43:15.551110+0000","last_clean":"2024-02-23T00:43:15.551110+0000","last_became_active":"2024-02-23T00:43:05.286511+0000","last_became_peered":"2024-02-23T00:43:05.286511+0000","last_unstale":"2024-02-23T00:43:15.551110+0000","last_undegraded":"2024-02-23T00:43:15.551110+0000","last_fullsized":"2024-02-23T00:43:15.551110+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.197452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"35.f","version":"0'0","reported_seq":22,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.541126+0000","last_change":"2024-02-23T00:43:02.340143+0000","last_active":"2024-02-23T00:43:15.541126+0000","last_peered":"2024-02-23T00:43:15.541126+0000","last_clean":"2024-02-23T00:43:15.541126+0000","last_became_active":"2024-02-23T00:43:02.339845+0000","last_became_peered":"2024-02-23T00:43:02.339845+0000","last_unstale":"2024-02-23T00:43:15.541126+0000","last_undegraded":"2024-02-23T00:43:15.541126+0000","last_fullsized":"2024-02-23T00:43:15.541126+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_clean_scrub_stamp":"2024-02-23T00:43:00.812733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"63.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:16.533032+0000","last_change":"2024-02-23T00:43:16.605144+0000","last_active":"2024-02-23T00:43:16.533032+0000","last_peered":"2024-02-23T00:43:16.533032+0000","last_clean":"2024-02-23T00:43:16.533032+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:16.533032+0000","last_undegraded":"2024-02-23T00:43:16.533032+0000","last_fullsized":"2024-02-23T00:43:16.533032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"59.16","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.617383+0000","last_change":"2024-02-23T00:43:16.617383+0000","last_active":"2024-02-23T00:43:16.617383+0000","last_peered":"2024-02-23T00:43:16.617383+0000","last_clean":"2024-02-23T00:43:16.617383+0000","last_became_active":"2024-02-23T00:43:16.617065+0000","last_became_peered":"2024-02-23T00:43:16.617065+0000","last_unstale":"2024-02-23T00:43:16.617383+0000","last_undegraded":"2024-02-23T00:43:16.617383+0000","last_fullsized":"2024-02-23T00:43:16.617383+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_clean_scrub_stamp":"2024-02-23T00:43:15.522490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.19","version":"0'0","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.540944+0000","last_change":"2024-02-23T00:43:10.913657+0000","last_active":"2024-02-23T00:43:15.540944+0000","last_peered":"2024-02-23T00:43:15.540944+0000","last_clean":"2024-02-23T00:43:15.540944+0000","last_became_active":"2024-02-23T00:43:10.913318+0000","last_became_peered":"2024-02-23T00:43:10.913318+0000","last_unstale":"2024-02-23T00:43:15.540944+0000","last_undegraded":"2024-02-23T00:43:15.540944+0000","last_fullsized":"2024-02-23T00:43:15.540944+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_clean_scrub_stamp":"2024-02-23T00:43:09.833908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"47.2","version":"0'0","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.552690+0000","last_change":"2024-02-23T00:43:08.889781+0000","last_active":"2024-02-23T00:43:15.552690+0000","last_peered":"2024-02-23T00:43:15.552690+0000","last_clean":"2024-02-23T00:43:15.552690+0000","last_became_active":"2024-02-23T00:43:08.885309+0000","last_became_peered":"2024-02-23T00:43:08.885309+0000","last_unstale":"2024-02-23T00:43:15.552690+0000","last_undegraded":"2024-02-23T00:43:15.552690+0000","last_fullsized":"2024-02-23T00:43:15.552690+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.7","version":"0'0","reported_seq":19,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.554589+0000","last_change":"2024-02-23T00:43:05.309921+0000","last_active":"2024-02-23T00:43:15.554589+0000","last_peered":"2024-02-23T00:43:15.554589+0000","last_clean":"2024-02-23T00:43:15.554589+0000","last_became_active":"2024-02-23T00:43:05.309679+0000","last_became_peered":"2024-02-23T00:43:05.309679+0000","last_unstale":"2024-02-23T00:43:15.554589+0000","last_undegraded":"2024-02-23T00:43:15.554589+0000","last_fullsized":"2024-02-23T00:43:15.554589+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.197452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.e","version":"0'0","reported_seq":22,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.571786+0000","last_change":"2024-02-23T00:43:03.235867+0000","last_active":"2024-02-23T00:43:15.571786+0000","last_peered":"2024-02-23T00:43:15.571786+0000","last_clean":"2024-02-23T00:43:15.571786+0000","last_became_active":"2024-02-23T00:43:03.235582+0000","last_became_peered":"2024-02-23T00:43:03.235582+0000","last_unstale":"2024-02-23T00:43:15.571786+0000","last_undegraded":"2024-02-23T00:43:15.571786+0000","last_fullsized":"2024-02-23T00:43:15.571786+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_clean_scrub_stamp":"2024-02-23T00:43:00.812733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"63.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:16.533032+0000","last_change":"2024-02-23T00:43:16.577571+0000","last_active":"2024-02-23T00:43:16.533032+0000","last_peered":"2024-02-23T00:43:16.533032+0000","last_clean":"2024-02-23T00:43:16.533032+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:16.533032+0000","last_undegraded":"2024-02-23T00:43:16.533032+0000","last_fullsized":"2024-02-23T00:43:16.533032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"59.11","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.620845+0000","last_change":"2024-02-23T00:43:16.620845+0000","last_active":"2024-02-23T00:43:16.620845+0000","last_peered":"2024-02-23T00:43:16.620845+0000","last_clean":"2024-02-23T00:43:16.620845+0000","last_became_active":"2024-02-23T00:43:16.620530+0000","last_became_peered":"2024-02-23T00:43:16.620530+0000","last_unstale":"2024-02-23T00:43:16.620845+0000","last_undegraded":"2024-02-23T00:43:16.620845+0000","last_fullsized":"2024-02-23T00:43:16.620845+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_clean_scrub_stamp":"2024-02-23T00:43:15.522490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"52.1e","version":"0'0","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.542720+0000","last_change":"2024-02-23T00:43:10.884058+0000","last_active":"2024-02-23T00:43:15.542720+0000","last_peered":"2024-02-23T00:43:15.542720+0000","last_clean":"2024-02-23T00:43:15.542720+0000","last_became_active":"2024-02-23T00:43:10.883748+0000","last_became_peered":"2024-02-23T00:43:10.883748+0000","last_unstale":"2024-02-23T00:43:15.542720+0000","last_undegraded":"2024-02-23T00:43:15.542720+0000","last_fullsized":"2024-02-23T00:43:15.542720+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_clean_scrub_stamp":"2024-02-23T00:43:09.833908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"47.5","version":"0'0","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.545711+0000","last_change":"2024-02-23T00:43:09.906559+0000","last_active":"2024-02-23T00:43:15.545711+0000","last_peered":"2024-02-23T00:43:15.545711+0000","last_clean":"2024-02-23T00:43:15.545711+0000","last_became_active":"2024-02-23T00:43:09.906396+0000","last_became_peered":"2024-02-23T00:43:09.906396+0000","last_unstale":"2024-02-23T00:43:15.545711+0000","last_undegraded":"2024-02-23T00:43:15.545711+0000","last_fullsized":"2024-02-23T00:43:15.545711+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"42.0","version":"0'0","reported_seq":19,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.563029+0000","last_change":"2024-02-23T00:43:05.282157+0000","last_active":"2024-02-23T00:43:15.563029+0000","last_peered":"2024-02-23T00:43:15.563029+0000","last_clean":"2024-02-23T00:43:15.563029+0000","last_became_active":"2024-02-23T00:43:05.281873+0000","last_became_peered":"2024-02-23T00:43:05.281873+0000","last_unstale":"2024-02-23T00:43:15.563029+0000","last_undegraded":"2024-02-23T00:43:15.563029+0000","last_fullsized":"2024-02-23T00:43:15.563029+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.197452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"35.9","version":"0'0","reported_seq":22,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.540492+0000","last_change":"2024-02-23T00:43:03.241691+0000","last_active":"2024-02-23T00:43:15.540492+0000","last_peered":"2024-02-23T00:43:15.540492+0000","last_clean":"2024-02-23T00:43:15.540492+0000","last_became_active":"2024-02-23T00:43:03.241237+0000","last_became_peered":"2024-02-23T00:43:03.241237+0000","last_unstale":"2024-02-23T00:43:15.540492+0000","last_undegraded":"2024-02-23T00:43:15.540492+0000","last_fullsized":"2024-02-23T00:43:15.540492+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_clean_scrub_stamp":"2024-02-23T00:43:00.812733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"63.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:16.533032+0000","last_change":"2024-02-23T00:43:16.589194+0000","last_active":"2024-02-23T00:43:16.533032+0000","last_peered":"2024-02-23T00:43:16.533032+0000","last_clean":"2024-02-23T00:43:16.533032+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:16.533032+0000","last_undegraded":"2024-02-23T00:43:16.533032+0000","last_fullsized":"2024-02-23T00:43:16.533032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"59.10","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.660418+0000","last_change":"2024-02-23T00:43:16.660418+0000","last_active":"2024-02-23T00:43:16.660418+0000","last_peered":"2024-02-23T00:43:16.660418+0000","last_clean":"2024-02-23T00:43:16.660418+0000","last_became_active":"2024-02-23T00:43:16.660132+0000","last_became_peered":"2024-02-23T00:43:16.660132+0000","last_unstale":"2024-02-23T00:43:16.660418+0000","last_undegraded":"2024-02-23T00:43:16.660418+0000","last_fullsized":"2024-02-23T00:43:16.660418+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_clean_scrub_stamp":"2024-02-23T00:43:15.522490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.1f","version":"0'0","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.578758+0000","last_change":"2024-02-23T00:43:10.908335+0000","last_active":"2024-02-23T00:43:15.578758+0000","last_peered":"2024-02-23T00:43:15.578758+0000","last_clean":"2024-02-23T00:43:15.578758+0000","last_became_active":"2024-02-23T00:43:10.908009+0000","last_became_peered":"2024-02-23T00:43:10.908009+0000","last_unstale":"2024-02-23T00:43:15.578758+0000","last_undegraded":"2024-02-23T00:43:15.578758+0000","last_fullsized":"2024-02-23T00:43:15.578758+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_clean_scrub_stamp":"2024-02-23T00:43:09.833908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"47.4","version":"0'0","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.540984+0000","last_change":"2024-02-23T00:43:09.895723+0000","last_active":"2024-02-23T00:43:15.540984+0000","last_peered":"2024-02-23T00:43:15.540984+0000","last_clean":"2024-02-23T00:43:15.540984+0000","last_became_active":"2024-02-23T00:43:09.895484+0000","last_became_peered":"2024-02-23T00:43:09.895484+0000","last_unstale":"2024-02-23T00:43:15.540984+0000","last_undegraded":"2024-02-23T00:43:15.540984+0000","last_fullsized":"2024-02-23T00:43:15.540984+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"42.1","version":"121'1","reported_seq":20,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.552195+0000","last_change":"2024-02-23T00:43:05.284326+0000","last_active":"2024-02-23T00:43:15.552195+0000","last_peered":"2024-02-23T00:43:15.552195+0000","last_clean":"2024-02-23T00:43:15.552195+0000","last_became_active":"2024-02-23T00:43:05.283961+0000","last_became_peered":"2024-02-23T00:43:05.283961+0000","last_unstale":"2024-02-23T00:43:15.552195+0000","last_undegraded":"2024-02-23T00:43:15.552195+0000","last_fullsized":"2024-02-23T00:43:15.552195+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.197452+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4096,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.8","version":"0'0","reported_seq":22,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.563163+0000","last_change":"2024-02-23T00:43:02.353395+0000","last_active":"2024-02-23T00:43:15.563163+0000","last_peered":"2024-02-23T00:43:15.563163+0000","last_clean":"2024-02-23T00:43:15.563163+0000","last_became_active":"2024-02-23T00:43:02.352994+0000","last_became_peered":"2024-02-23T00:43:02.352994+0000","last_unstale":"2024-02-23T00:43:15.563163+0000","last_undegraded":"2024-02-23T00:43:15.563163+0000","last_fullsized":"2024-02-23T00:43:15.563163+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_clean_scrub_stamp":"2024-02-23T00:43:00.812733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"63.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:16.533032+0000","last_change":"2024-02-23T00:43:16.551263+0000","last_active":"2024-02-23T00:43:16.533032+0000","last_peered":"2024-02-23T00:43:16.533032+0000","last_clean":"2024-02-23T00:43:16.533032+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:16.533032+0000","last_undegraded":"2024-02-23T00:43:16.533032+0000","last_fullsized":"2024-02-23T00:43:16.533032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"59.13","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.626020+0000","last_change":"2024-02-23T00:43:16.626020+0000","last_active":"2024-02-23T00:43:16.626020+0000","last_peered":"2024-02-23T00:43:16.626020+0000","last_clean":"2024-02-23T00:43:16.626020+0000","last_became_active":"2024-02-23T00:43:16.625671+0000","last_became_peered":"2024-02-23T00:43:16.625671+0000","last_unstale":"2024-02-23T00:43:16.626020+0000","last_undegraded":"2024-02-23T00:43:16.626020+0000","last_fullsized":"2024-02-23T00:43:16.626020+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_clean_scrub_stamp":"2024-02-23T00:43:15.522490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.1c","version":"0'0","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.571393+0000","last_change":"2024-02-23T00:43:10.907592+0000","last_active":"2024-02-23T00:43:15.571393+0000","last_peered":"2024-02-23T00:43:15.571393+0000","last_clean":"2024-02-23T00:43:15.571393+0000","last_became_active":"2024-02-23T00:43:10.907094+0000","last_became_peered":"2024-02-23T00:43:10.907094+0000","last_unstale":"2024-02-23T00:43:15.571393+0000","last_undegraded":"2024-02-23T00:43:15.571393+0000","last_fullsized":"2024-02-23T00:43:15.571393+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_clean_scrub_stamp":"2024-02-23T00:43:09.833908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"47.7","version":"120'1","reported_seq":18,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.585689+0000","last_change":"2024-02-23T00:43:08.893084+0000","last_active":"2024-02-23T00:43:15.585689+0000","last_peered":"2024-02-23T00:43:15.585689+0000","last_clean":"2024-02-23T00:43:15.585689+0000","last_became_active":"2024-02-23T00:43:08.892646+0000","last_became_peered":"2024-02-23T00:43:08.892646+0000","last_unstale":"2024-02-23T00:43:15.585689+0000","last_undegraded":"2024-02-23T00:43:15.585689+0000","last_fullsized":"2024-02-23T00:43:15.585689+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.2","version":"0'0","reported_seq":19,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.557431+0000","last_change":"2024-02-23T00:43:05.298909+0000","last_active":"2024-02-23T00:43:15.557431+0000","last_peered":"2024-02-23T00:43:15.557431+0000","last_clean":"2024-02-23T00:43:15.557431+0000","last_became_active":"2024-02-23T00:43:05.298440+0000","last_became_peered":"2024-02-23T00:43:05.298440+0000","last_unstale":"2024-02-23T00:43:15.557431+0000","last_undegraded":"2024-02-23T00:43:15.557431+0000","last_fullsized":"2024-02-23T00:43:15.557431+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.197452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.b","version":"0'0","reported_seq":22,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.549330+0000","last_change":"2024-02-23T00:43:02.355928+0000","last_active":"2024-02-23T00:43:15.549330+0000","last_peered":"2024-02-23T00:43:15.549330+0000","last_clean":"2024-02-23T00:43:15.549330+0000","last_became_active":"2024-02-23T00:43:02.355603+0000","last_became_peered":"2024-02-23T00:43:02.355603+0000","last_unstale":"2024-02-23T00:43:15.549330+0000","last_undegraded":"2024-02-23T00:43:15.549330+0000","last_fullsized":"2024-02-23T00:43:15.549330+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_clean_scrub_stamp":"2024-02-23T00:43:00.812733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"63.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:16.533032+0000","last_change":"2024-02-23T00:43:16.557672+0000","last_active":"2024-02-23T00:43:16.533032+0000","last_peered":"2024-02-23T00:43:16.533032+0000","last_clean":"2024-02-23T00:43:16.533032+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:16.533032+0000","last_undegraded":"2024-02-23T00:43:16.533032+0000","last_fullsized":"2024-02-23T00:43:16.533032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"59.12","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.621081+0000","last_change":"2024-02-23T00:43:16.621081+0000","last_active":"2024-02-23T00:43:16.621081+0000","last_peered":"2024-02-23T00:43:16.621081+0000","last_clean":"2024-02-23T00:43:16.621081+0000","last_became_active":"2024-02-23T00:43:16.620905+0000","last_became_peered":"2024-02-23T00:43:16.620905+0000","last_unstale":"2024-02-23T00:43:16.621081+0000","last_undegraded":"2024-02-23T00:43:16.621081+0000","last_fullsized":"2024-02-23T00:43:16.621081+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_clean_scrub_stamp":"2024-02-23T00:43:15.522490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"52.1d","version":"121'3","reported_seq":20,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.537747+0000","last_change":"2024-02-23T00:43:10.912037+0000","last_active":"2024-02-23T00:43:15.537747+0000","last_peered":"2024-02-23T00:43:15.537747+0000","last_clean":"2024-02-23T00:43:15.537747+0000","last_became_active":"2024-02-23T00:43:10.911743+0000","last_became_peered":"2024-02-23T00:43:10.911743+0000","last_unstale":"2024-02-23T00:43:15.537747+0000","last_undegraded":"2024-02-23T00:43:15.537747+0000","last_fullsized":"2024-02-23T00:43:15.537747+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_clean_scrub_stamp":"2024-02-23T00:43:09.833908+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"47.6","version":"0'0","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.543320+0000","last_change":"2024-02-23T00:43:09.911196+0000","last_active":"2024-02-23T00:43:15.543320+0000","last_peered":"2024-02-23T00:43:15.543320+0000","last_clean":"2024-02-23T00:43:15.543320+0000","last_became_active":"2024-02-23T00:43:09.910649+0000","last_became_peered":"2024-02-23T00:43:09.910649+0000","last_unstale":"2024-02-23T00:43:15.543320+0000","last_undegraded":"2024-02-23T00:43:15.543320+0000","last_fullsized":"2024-02-23T00:43:15.543320+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.3","version":"0'0","reported_seq":19,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.543226+0000","last_change":"2024-02-23T00:43:05.301933+0000","last_active":"2024-02-23T00:43:15.543226+0000","last_peered":"2024-02-23T00:43:15.543226+0000","last_clean":"2024-02-23T00:43:15.543226+0000","last_became_active":"2024-02-23T00:43:05.301639+0000","last_became_peered":"2024-02-23T00:43:05.301639+0000","last_unstale":"2024-02-23T00:43:15.543226+0000","last_undegraded":"2024-02-23T00:43:15.543226+0000","last_fullsized":"2024-02-23T00:43:15.543226+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.197452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.a","version":"0'0","reported_seq":22,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.557509+0000","last_change":"2024-02-23T00:43:03.230424+0000","last_active":"2024-02-23T00:43:15.557509+0000","last_peered":"2024-02-23T00:43:15.557509+0000","last_clean":"2024-02-23T00:43:15.557509+0000","last_became_active":"2024-02-23T00:43:03.229694+0000","last_became_peered":"2024-02-23T00:43:03.229694+0000","last_unstale":"2024-02-23T00:43:15.557509+0000","last_undegraded":"2024-02-23T00:43:15.557509+0000","last_fullsized":"2024-02-23T00:43:15.557509+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_clean_scrub_stamp":"2024-02-23T00:43:00.812733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"63.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:16.533032+0000","last_change":"2024-02-23T00:43:16.549638+0000","last_active":"2024-02-23T00:43:16.533032+0000","last_peered":"2024-02-23T00:43:16.533032+0000","last_clean":"2024-02-23T00:43:16.533032+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:16.533032+0000","last_undegraded":"2024-02-23T00:43:16.533032+0000","last_fullsized":"2024-02-23T00:43:16.533032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"59.1c","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.638771+0000","last_change":"2024-02-23T00:43:16.638771+0000","last_active":"2024-02-23T00:43:16.638771+0000","last_peered":"2024-02-23T00:43:16.638771+0000","last_clean":"2024-02-23T00:43:16.638771+0000","last_became_active":"2024-02-23T00:43:16.638331+0000","last_became_peered":"2024-02-23T00:43:16.638331+0000","last_unstale":"2024-02-23T00:43:16.638771+0000","last_undegraded":"2024-02-23T00:43:16.638771+0000","last_fullsized":"2024-02-23T00:43:16.638771+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_clean_scrub_stamp":"2024-02-23T00:43:15.522490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.13","version":"0'0","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.542966+0000","last_change":"2024-02-23T00:43:10.906960+0000","last_active":"2024-02-23T00:43:15.542966+0000","last_peered":"2024-02-23T00:43:15.542966+0000","last_clean":"2024-02-23T00:43:15.542966+0000","last_became_active":"2024-02-23T00:43:10.906659+0000","last_became_peered":"2024-02-23T00:43:10.906659+0000","last_unstale":"2024-02-23T00:43:15.542966+0000","last_undegraded":"2024-02-23T00:43:15.542966+0000","last_fullsized":"2024-02-23T00:43:15.542966+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_clean_scrub_stamp":"2024-02-23T00:43:09.833908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"47.8","version":"120'3","reported_seq":20,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.537473+0000","last_change":"2024-02-23T00:43:08.869192+0000","last_active":"2024-02-23T00:43:15.537473+0000","last_peered":"2024-02-23T00:43:15.537473+0000","last_clean":"2024-02-23T00:43:15.537473+0000","last_became_active":"2024-02-23T00:43:08.868897+0000","last_became_peered":"2024-02-23T00:43:08.868897+0000","last_unstale":"2024-02-23T00:43:15.537473+0000","last_undegraded":"2024-02-23T00:43:15.537473+0000","last_fullsized":"2024-02-23T00:43:15.537473+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":512,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.d","version":"0'0","reported_seq":19,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.540893+0000","last_change":"2024-02-23T00:43:05.282085+0000","last_active":"2024-02-23T00:43:15.540893+0000","last_peered":"2024-02-23T00:43:15.540893+0000","last_clean":"2024-02-23T00:43:15.540893+0000","last_became_active":"2024-02-23T00:43:05.281782+0000","last_became_peered":"2024-02-23T00:43:05.281782+0000","last_unstale":"2024-02-23T00:43:15.540893+0000","last_undegraded":"2024-02-23T00:43:15.540893+0000","last_fullsized":"2024-02-23T00:43:15.540893+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.197452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.4","version":"0'0","reported_seq":22,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.538012+0000","last_change":"2024-02-23T00:43:02.345350+0000","last_active":"2024-02-23T00:43:15.538012+0000","last_peered":"2024-02-23T00:43:15.538012+0000","last_clean":"2024-02-23T00:43:15.538012+0000","last_became_active":"2024-02-23T00:43:02.345084+0000","last_became_peered":"2024-02-23T00:43:02.345084+0000","last_unstale":"2024-02-23T00:43:15.538012+0000","last_undegraded":"2024-02-23T00:43:15.538012+0000","last_fullsized":"2024-02-23T00:43:15.538012+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_clean_scrub_stamp":"2024-02-23T00:43:00.812733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"63.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:16.533032+0000","last_change":"2024-02-23T00:43:16.601607+0000","last_active":"2024-02-23T00:43:16.533032+0000","last_peered":"2024-02-23T00:43:16.533032+0000","last_clean":"2024-02-23T00:43:16.533032+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:16.533032+0000","last_undegraded":"2024-02-23T00:43:16.533032+0000","last_fullsized":"2024-02-23T00:43:16.533032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"59.1d","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.613829+0000","last_change":"2024-02-23T00:43:16.613829+0000","last_active":"2024-02-23T00:43:16.613829+0000","last_peered":"2024-02-23T00:43:16.613829+0000","last_clean":"2024-02-23T00:43:16.613829+0000","last_became_active":"2024-02-23T00:43:16.613595+0000","last_became_peered":"2024-02-23T00:43:16.613595+0000","last_unstale":"2024-02-23T00:43:16.613829+0000","last_undegraded":"2024-02-23T00:43:16.613829+0000","last_fullsized":"2024-02-23T00:43:16.613829+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_clean_scrub_stamp":"2024-02-23T00:43:15.522490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.12","version":"0'0","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.552383+0000","last_change":"2024-02-23T00:43:10.927033+0000","last_active":"2024-02-23T00:43:15.552383+0000","last_peered":"2024-02-23T00:43:15.552383+0000","last_clean":"2024-02-23T00:43:15.552383+0000","last_became_active":"2024-02-23T00:43:10.926723+0000","last_became_peered":"2024-02-23T00:43:10.926723+0000","last_unstale":"2024-02-23T00:43:15.552383+0000","last_undegraded":"2024-02-23T00:43:15.552383+0000","last_fullsized":"2024-02-23T00:43:15.552383+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_clean_scrub_stamp":"2024-02-23T00:43:09.833908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"47.9","version":"0'0","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.541142+0000","last_change":"2024-02-23T00:43:09.914042+0000","last_active":"2024-02-23T00:43:15.541142+0000","last_peered":"2024-02-23T00:43:15.541142+0000","last_clean":"2024-02-23T00:43:15.541142+0000","last_became_active":"2024-02-23T00:43:09.913719+0000","last_became_peered":"2024-02-23T00:43:09.913719+0000","last_unstale":"2024-02-23T00:43:15.541142+0000","last_undegraded":"2024-02-23T00:43:15.541142+0000","last_fullsized":"2024-02-23T00:43:15.541142+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.c","version":"0'0","reported_seq":19,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.581575+0000","last_change":"2024-02-23T00:43:05.283046+0000","last_active":"2024-02-23T00:43:15.581575+0000","last_peered":"2024-02-23T00:43:15.581575+0000","last_clean":"2024-02-23T00:43:15.581575+0000","last_became_active":"2024-02-23T00:43:05.282751+0000","last_became_peered":"2024-02-23T00:43:05.282751+0000","last_unstale":"2024-02-23T00:43:15.581575+0000","last_undegraded":"2024-02-23T00:43:15.581575+0000","last_fullsized":"2024-02-23T00:43:15.581575+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.197452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.5","version":"0'0","reported_seq":22,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.545891+0000","last_change":"2024-02-23T00:43:02.367489+0000","last_active":"2024-02-23T00:43:15.545891+0000","last_peered":"2024-02-23T00:43:15.545891+0000","last_clean":"2024-02-23T00:43:15.545891+0000","last_became_active":"2024-02-23T00:43:02.367211+0000","last_became_peered":"2024-02-23T00:43:02.367211+0000","last_unstale":"2024-02-23T00:43:15.545891+0000","last_undegraded":"2024-02-23T00:43:15.545891+0000","last_fullsized":"2024-02-23T00:43:15.545891+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_clean_scrub_stamp":"2024-02-23T00:43:00.812733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"63.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:16.533032+0000","last_change":"2024-02-23T00:43:16.553115+0000","last_active":"2024-02-23T00:43:16.533032+0000","last_peered":"2024-02-23T00:43:16.533032+0000","last_clean":"2024-02-23T00:43:16.533032+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:16.533032+0000","last_undegraded":"2024-02-23T00:43:16.533032+0000","last_fullsized":"2024-02-23T00:43:16.533032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"59.1e","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.650433+0000","last_change":"2024-02-23T00:43:16.650433+0000","last_active":"2024-02-23T00:43:16.650433+0000","last_peered":"2024-02-23T00:43:16.650433+0000","last_clean":"2024-02-23T00:43:16.650433+0000","last_became_active":"2024-02-23T00:43:16.650135+0000","last_became_peered":"2024-02-23T00:43:16.650135+0000","last_unstale":"2024-02-23T00:43:16.650433+0000","last_undegraded":"2024-02-23T00:43:16.650433+0000","last_fullsized":"2024-02-23T00:43:16.650433+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_clean_scrub_stamp":"2024-02-23T00:43:15.522490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.11","version":"0'0","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.550175+0000","last_change":"2024-02-23T00:43:10.863286+0000","last_active":"2024-02-23T00:43:15.550175+0000","last_peered":"2024-02-23T00:43:15.550175+0000","last_clean":"2024-02-23T00:43:15.550175+0000","last_became_active":"2024-02-23T00:43:10.862691+0000","last_became_peered":"2024-02-23T00:43:10.862691+0000","last_unstale":"2024-02-23T00:43:15.550175+0000","last_undegraded":"2024-02-23T00:43:15.550175+0000","last_fullsized":"2024-02-23T00:43:15.550175+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_clean_scrub_stamp":"2024-02-23T00:43:09.833908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"47.a","version":"0'0","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.552608+0000","last_change":"2024-02-23T00:43:08.890324+0000","last_active":"2024-02-23T00:43:15.552608+0000","last_peered":"2024-02-23T00:43:15.552608+0000","last_clean":"2024-02-23T00:43:15.552608+0000","last_became_active":"2024-02-23T00:43:08.890143+0000","last_became_peered":"2024-02-23T00:43:08.890143+0000","last_unstale":"2024-02-23T00:43:15.552608+0000","last_undegraded":"2024-02-23T00:43:15.552608+0000","last_fullsized":"2024-02-23T00:43:15.552608+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.f","version":"0'0","reported_seq":19,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.554754+0000","last_change":"2024-02-23T00:43:05.291596+0000","last_active":"2024-02-23T00:43:15.554754+0000","last_peered":"2024-02-23T00:43:15.554754+0000","last_clean":"2024-02-23T00:43:15.554754+0000","last_became_active":"2024-02-23T00:43:05.291323+0000","last_became_peered":"2024-02-23T00:43:05.291323+0000","last_unstale":"2024-02-23T00:43:15.554754+0000","last_undegraded":"2024-02-23T00:43:15.554754+0000","last_fullsized":"2024-02-23T00:43:15.554754+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.197452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.6","version":"0'0","reported_seq":22,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.550459+0000","last_change":"2024-02-23T00:43:02.368673+0000","last_active":"2024-02-23T00:43:15.550459+0000","last_peered":"2024-02-23T00:43:15.550459+0000","last_clean":"2024-02-23T00:43:15.550459+0000","last_became_active":"2024-02-23T00:43:02.368413+0000","last_became_peered":"2024-02-23T00:43:02.368413+0000","last_unstale":"2024-02-23T00:43:15.550459+0000","last_undegraded":"2024-02-23T00:43:15.550459+0000","last_fullsized":"2024-02-23T00:43:15.550459+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_clean_scrub_stamp":"2024-02-23T00:43:00.812733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1],"acting":[6,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"63.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:16.533032+0000","last_change":"2024-02-23T00:43:16.550649+0000","last_active":"2024-02-23T00:43:16.533032+0000","last_peered":"2024-02-23T00:43:16.533032+0000","last_clean":"2024-02-23T00:43:16.533032+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:16.533032+0000","last_undegraded":"2024-02-23T00:43:16.533032+0000","last_fullsized":"2024-02-23T00:43:16.533032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"59.1f","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.630998+0000","last_change":"2024-02-23T00:43:16.630998+0000","last_active":"2024-02-23T00:43:16.630998+0000","last_peered":"2024-02-23T00:43:16.630998+0000","last_clean":"2024-02-23T00:43:16.630998+0000","last_became_active":"2024-02-23T00:43:16.630693+0000","last_became_peered":"2024-02-23T00:43:16.630693+0000","last_unstale":"2024-02-23T00:43:16.630998+0000","last_undegraded":"2024-02-23T00:43:16.630998+0000","last_fullsized":"2024-02-23T00:43:16.630998+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_clean_scrub_stamp":"2024-02-23T00:43:15.522490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.10","version":"0'0","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.562657+0000","last_change":"2024-02-23T00:43:10.890469+0000","last_active":"2024-02-23T00:43:15.562657+0000","last_peered":"2024-02-23T00:43:15.562657+0000","last_clean":"2024-02-23T00:43:15.562657+0000","last_became_active":"2024-02-23T00:43:10.890148+0000","last_became_peered":"2024-02-23T00:43:10.890148+0000","last_unstale":"2024-02-23T00:43:15.562657+0000","last_undegraded":"2024-02-23T00:43:15.562657+0000","last_fullsized":"2024-02-23T00:43:15.562657+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_clean_scrub_stamp":"2024-02-23T00:43:09.833908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"47.b","version":"0'0","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.549949+0000","last_change":"2024-02-23T00:43:08.871891+0000","last_active":"2024-02-23T00:43:15.549949+0000","last_peered":"2024-02-23T00:43:15.549949+0000","last_clean":"2024-02-23T00:43:15.549949+0000","last_became_active":"2024-02-23T00:43:08.871529+0000","last_became_peered":"2024-02-23T00:43:08.871529+0000","last_unstale":"2024-02-23T00:43:15.549949+0000","last_undegraded":"2024-02-23T00:43:15.549949+0000","last_fullsized":"2024-02-23T00:43:15.549949+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"42.e","version":"0'0","reported_seq":19,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.545997+0000","last_change":"2024-02-23T00:43:05.282738+0000","last_active":"2024-02-23T00:43:15.545997+0000","last_peered":"2024-02-23T00:43:15.545997+0000","last_clean":"2024-02-23T00:43:15.545997+0000","last_became_active":"2024-02-23T00:43:05.282563+0000","last_became_peered":"2024-02-23T00:43:05.282563+0000","last_unstale":"2024-02-23T00:43:15.545997+0000","last_undegraded":"2024-02-23T00:43:15.545997+0000","last_fullsized":"2024-02-23T00:43:15.545997+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.197452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.7","version":"0'0","reported_seq":22,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.541668+0000","last_change":"2024-02-23T00:43:02.343556+0000","last_active":"2024-02-23T00:43:15.541668+0000","last_peered":"2024-02-23T00:43:15.541668+0000","last_clean":"2024-02-23T00:43:15.541668+0000","last_became_active":"2024-02-23T00:43:02.343244+0000","last_became_peered":"2024-02-23T00:43:02.343244+0000","last_unstale":"2024-02-23T00:43:15.541668+0000","last_undegraded":"2024-02-23T00:43:15.541668+0000","last_fullsized":"2024-02-23T00:43:15.541668+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_clean_scrub_stamp":"2024-02-23T00:43:00.812733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"63.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:16.533032+0000","last_change":"2024-02-23T00:43:16.552683+0000","last_active":"2024-02-23T00:43:16.533032+0000","last_peered":"2024-02-23T00:43:16.533032+0000","last_clean":"2024-02-23T00:43:16.533032+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:16.533032+0000","last_undegraded":"2024-02-23T00:43:16.533032+0000","last_fullsized":"2024-02-23T00:43:16.533032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"59.18","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.639402+0000","last_change":"2024-02-23T00:43:16.639402+0000","last_active":"2024-02-23T00:43:16.639402+0000","last_peered":"2024-02-23T00:43:16.639402+0000","last_clean":"2024-02-23T00:43:16.639402+0000","last_became_active":"2024-02-23T00:43:16.638772+0000","last_became_peered":"2024-02-23T00:43:16.638772+0000","last_unstale":"2024-02-23T00:43:16.639402+0000","last_undegraded":"2024-02-23T00:43:16.639402+0000","last_fullsized":"2024-02-23T00:43:16.639402+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_clean_scrub_stamp":"2024-02-23T00:43:15.522490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.17","version":"0'0","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.541266+0000","last_change":"2024-02-23T00:43:10.910932+0000","last_active":"2024-02-23T00:43:15.541266+0000","last_peered":"2024-02-23T00:43:15.541266+0000","last_clean":"2024-02-23T00:43:15.541266+0000","last_became_active":"2024-02-23T00:43:10.910584+0000","last_became_peered":"2024-02-23T00:43:10.910584+0000","last_unstale":"2024-02-23T00:43:15.541266+0000","last_undegraded":"2024-02-23T00:43:15.541266+0000","last_fullsized":"2024-02-23T00:43:15.541266+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_clean_scrub_stamp":"2024-02-23T00:43:09.833908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"47.c","version":"0'0","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.571573+0000","last_change":"2024-02-23T00:43:08.891783+0000","last_active":"2024-02-23T00:43:15.571573+0000","last_peered":"2024-02-23T00:43:15.571573+0000","last_clean":"2024-02-23T00:43:15.571573+0000","last_became_active":"2024-02-23T00:43:08.890962+0000","last_became_peered":"2024-02-23T00:43:08.890962+0000","last_unstale":"2024-02-23T00:43:15.571573+0000","last_undegraded":"2024-02-23T00:43:15.571573+0000","last_fullsized":"2024-02-23T00:43:15.571573+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.9","version":"0'0","reported_seq":19,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.541684+0000","last_change":"2024-02-23T00:43:05.248302+0000","last_active":"2024-02-23T00:43:15.541684+0000","last_peered":"2024-02-23T00:43:15.541684+0000","last_clean":"2024-02-23T00:43:15.541684+0000","last_became_active":"2024-02-23T00:43:05.247954+0000","last_became_peered":"2024-02-23T00:43:05.247954+0000","last_unstale":"2024-02-23T00:43:15.541684+0000","last_undegraded":"2024-02-23T00:43:15.541684+0000","last_fullsized":"2024-02-23T00:43:15.541684+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.197452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.0","version":"0'0","reported_seq":22,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.567265+0000","last_change":"2024-02-23T00:43:02.355845+0000","last_active":"2024-02-23T00:43:15.567265+0000","last_peered":"2024-02-23T00:43:15.567265+0000","last_clean":"2024-02-23T00:43:15.567265+0000","last_became_active":"2024-02-23T00:43:02.355559+0000","last_became_peered":"2024-02-23T00:43:02.355559+0000","last_unstale":"2024-02-23T00:43:15.567265+0000","last_undegraded":"2024-02-23T00:43:15.567265+0000","last_fullsized":"2024-02-23T00:43:15.567265+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_clean_scrub_stamp":"2024-02-23T00:43:00.812733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"63.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:16.533032+0000","last_change":"2024-02-23T00:43:16.556944+0000","last_active":"2024-02-23T00:43:16.533032+0000","last_peered":"2024-02-23T00:43:16.533032+0000","last_clean":"2024-02-23T00:43:16.533032+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:16.533032+0000","last_undegraded":"2024-02-23T00:43:16.533032+0000","last_fullsized":"2024-02-23T00:43:16.533032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"59.19","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.639162+0000","last_change":"2024-02-23T00:43:16.639162+0000","last_active":"2024-02-23T00:43:16.639162+0000","last_peered":"2024-02-23T00:43:16.639162+0000","last_clean":"2024-02-23T00:43:16.639162+0000","last_became_active":"2024-02-23T00:43:16.638666+0000","last_became_peered":"2024-02-23T00:43:16.638666+0000","last_unstale":"2024-02-23T00:43:16.639162+0000","last_undegraded":"2024-02-23T00:43:16.639162+0000","last_fullsized":"2024-02-23T00:43:16.639162+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_clean_scrub_stamp":"2024-02-23T00:43:15.522490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.16","version":"0'0","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.545784+0000","last_change":"2024-02-23T00:43:10.884272+0000","last_active":"2024-02-23T00:43:15.545784+0000","last_peered":"2024-02-23T00:43:15.545784+0000","last_clean":"2024-02-23T00:43:15.545784+0000","last_became_active":"2024-02-23T00:43:10.883986+0000","last_became_peered":"2024-02-23T00:43:10.883986+0000","last_unstale":"2024-02-23T00:43:15.545784+0000","last_undegraded":"2024-02-23T00:43:15.545784+0000","last_fullsized":"2024-02-23T00:43:15.545784+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_clean_scrub_stamp":"2024-02-23T00:43:09.833908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"47.d","version":"0'0","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.542532+0000","last_change":"2024-02-23T00:43:08.870250+0000","last_active":"2024-02-23T00:43:15.542532+0000","last_peered":"2024-02-23T00:43:15.542532+0000","last_clean":"2024-02-23T00:43:15.542532+0000","last_became_active":"2024-02-23T00:43:08.869935+0000","last_became_peered":"2024-02-23T00:43:08.869935+0000","last_unstale":"2024-02-23T00:43:15.542532+0000","last_undegraded":"2024-02-23T00:43:15.542532+0000","last_fullsized":"2024-02-23T00:43:15.542532+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.8","version":"0'0","reported_seq":19,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.545109+0000","last_change":"2024-02-23T00:43:05.307066+0000","last_active":"2024-02-23T00:43:15.545109+0000","last_peered":"2024-02-23T00:43:15.545109+0000","last_clean":"2024-02-23T00:43:15.545109+0000","last_became_active":"2024-02-23T00:43:05.306799+0000","last_became_peered":"2024-02-23T00:43:05.306799+0000","last_unstale":"2024-02-23T00:43:15.545109+0000","last_undegraded":"2024-02-23T00:43:15.545109+0000","last_fullsized":"2024-02-23T00:43:15.545109+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.197452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.1","version":"0'0","reported_seq":22,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.546402+0000","last_change":"2024-02-23T00:43:02.336984+0000","last_active":"2024-02-23T00:43:15.546402+0000","last_peered":"2024-02-23T00:43:15.546402+0000","last_clean":"2024-02-23T00:43:15.546402+0000","last_became_active":"2024-02-23T00:43:02.336685+0000","last_became_peered":"2024-02-23T00:43:02.336685+0000","last_unstale":"2024-02-23T00:43:15.546402+0000","last_undegraded":"2024-02-23T00:43:15.546402+0000","last_fullsized":"2024-02-23T00:43:15.546402+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_clean_scrub_stamp":"2024-02-23T00:43:00.812733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"63.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:16.533032+0000","last_change":"2024-02-23T00:43:16.639668+0000","last_active":"2024-02-23T00:43:16.533032+0000","last_peered":"2024-02-23T00:43:16.533032+0000","last_clean":"2024-02-23T00:43:16.533032+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:16.533032+0000","last_undegraded":"2024-02-23T00:43:16.533032+0000","last_fullsized":"2024-02-23T00:43:16.533032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6,7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"59.1a","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.650618+0000","last_change":"2024-02-23T00:43:16.650618+0000","last_active":"2024-02-23T00:43:16.650618+0000","last_peered":"2024-02-23T00:43:16.650618+0000","last_clean":"2024-02-23T00:43:16.650618+0000","last_became_active":"2024-02-23T00:43:16.650173+0000","last_became_peered":"2024-02-23T00:43:16.650173+0000","last_unstale":"2024-02-23T00:43:16.650618+0000","last_undegraded":"2024-02-23T00:43:16.650618+0000","last_fullsized":"2024-02-23T00:43:16.650618+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_clean_scrub_stamp":"2024-02-23T00:43:15.522490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.15","version":"0'0","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.554765+0000","last_change":"2024-02-23T00:43:10.895067+0000","last_active":"2024-02-23T00:43:15.554765+0000","last_peered":"2024-02-23T00:43:15.554765+0000","last_clean":"2024-02-23T00:43:15.554765+0000","last_became_active":"2024-02-23T00:43:10.894747+0000","last_became_peered":"2024-02-23T00:43:10.894747+0000","last_unstale":"2024-02-23T00:43:15.554765+0000","last_undegraded":"2024-02-23T00:43:15.554765+0000","last_fullsized":"2024-02-23T00:43:15.554765+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_clean_scrub_stamp":"2024-02-23T00:43:09.833908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"47.e","version":"0'0","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.557240+0000","last_change":"2024-02-23T00:43:08.881765+0000","last_active":"2024-02-23T00:43:15.557240+0000","last_peered":"2024-02-23T00:43:15.557240+0000","last_clean":"2024-02-23T00:43:15.557240+0000","last_became_active":"2024-02-23T00:43:08.881454+0000","last_became_peered":"2024-02-23T00:43:08.881454+0000","last_unstale":"2024-02-23T00:43:15.557240+0000","last_undegraded":"2024-02-23T00:43:15.557240+0000","last_fullsized":"2024-02-23T00:43:15.557240+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"42.b","version":"0'0","reported_seq":19,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.539131+0000","last_change":"2024-02-23T00:43:05.283300+0000","last_active":"2024-02-23T00:43:15.539131+0000","last_peered":"2024-02-23T00:43:15.539131+0000","last_clean":"2024-02-23T00:43:15.539131+0000","last_became_active":"2024-02-23T00:43:05.283014+0000","last_became_peered":"2024-02-23T00:43:05.283014+0000","last_unstale":"2024-02-23T00:43:15.539131+0000","last_undegraded":"2024-02-23T00:43:15.539131+0000","last_fullsized":"2024-02-23T00:43:15.539131+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.197452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.2","version":"0'0","reported_seq":22,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.557587+0000","last_change":"2024-02-23T00:43:03.230584+0000","last_active":"2024-02-23T00:43:15.557587+0000","last_peered":"2024-02-23T00:43:15.557587+0000","last_clean":"2024-02-23T00:43:15.557587+0000","last_became_active":"2024-02-23T00:43:03.230108+0000","last_became_peered":"2024-02-23T00:43:03.230108+0000","last_unstale":"2024-02-23T00:43:15.557587+0000","last_undegraded":"2024-02-23T00:43:15.557587+0000","last_fullsized":"2024-02-23T00:43:15.557587+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_clean_scrub_stamp":"2024-02-23T00:43:00.812733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"63.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:17.717661+0000","last_change":"2024-02-23T00:43:17.717661+0000","last_active":"2024-02-23T00:43:17.717661+0000","last_peered":"2024-02-23T00:43:17.717661+0000","last_clean":"2024-02-23T00:43:17.717661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:17.717661+0000","last_undegraded":"2024-02-23T00:43:17.717661+0000","last_fullsized":"2024-02-23T00:43:17.717661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:16.533032+0000","last_change":"2024-02-23T00:43:16.587344+0000","last_active":"2024-02-23T00:43:16.533032+0000","last_peered":"2024-02-23T00:43:16.533032+0000","last_clean":"2024-02-23T00:43:16.533032+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:16.533032+0000","last_undegraded":"2024-02-23T00:43:16.533032+0000","last_fullsized":"2024-02-23T00:43:16.533032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"59.1b","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.631188+0000","last_change":"2024-02-23T00:43:16.631188+0000","last_active":"2024-02-23T00:43:16.631188+0000","last_peered":"2024-02-23T00:43:16.631188+0000","last_clean":"2024-02-23T00:43:16.631188+0000","last_became_active":"2024-02-23T00:43:16.630905+0000","last_became_peered":"2024-02-23T00:43:16.630905+0000","last_unstale":"2024-02-23T00:43:16.631188+0000","last_undegraded":"2024-02-23T00:43:16.631188+0000","last_fullsized":"2024-02-23T00:43:16.631188+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:15.522490+0000","last_clean_scrub_stamp":"2024-02-23T00:43:15.522490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.14","version":"0'0","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.540820+0000","last_change":"2024-02-23T00:43:10.898031+0000","last_active":"2024-02-23T00:43:15.540820+0000","last_peered":"2024-02-23T00:43:15.540820+0000","last_clean":"2024-02-23T00:43:15.540820+0000","last_became_active":"2024-02-23T00:43:10.897653+0000","last_became_peered":"2024-02-23T00:43:10.897653+0000","last_unstale":"2024-02-23T00:43:15.540820+0000","last_undegraded":"2024-02-23T00:43:15.540820+0000","last_fullsized":"2024-02-23T00:43:15.540820+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:09.833908+0000","last_clean_scrub_stamp":"2024-02-23T00:43:09.833908+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"47.f","version":"0'0","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.541947+0000","last_change":"2024-02-23T00:43:09.905858+0000","last_active":"2024-02-23T00:43:15.541947+0000","last_peered":"2024-02-23T00:43:15.541947+0000","last_clean":"2024-02-23T00:43:15.541947+0000","last_became_active":"2024-02-23T00:43:09.905531+0000","last_became_peered":"2024-02-23T00:43:09.905531+0000","last_unstale":"2024-02-23T00:43:15.541947+0000","last_undegraded":"2024-02-23T00:43:15.541947+0000","last_fullsized":"2024-02-23T00:43:15.541947+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.a","version":"0'0","reported_seq":19,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.537431+0000","last_change":"2024-02-23T00:43:05.286640+0000","last_active":"2024-02-23T00:43:15.537431+0000","last_peered":"2024-02-23T00:43:15.537431+0000","last_clean":"2024-02-23T00:43:15.537431+0000","last_became_active":"2024-02-23T00:43:05.286352+0000","last_became_peered":"2024-02-23T00:43:05.286352+0000","last_unstale":"2024-02-23T00:43:15.537431+0000","last_undegraded":"2024-02-23T00:43:15.537431+0000","last_fullsized":"2024-02-23T00:43:15.537431+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:04.197452+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.197452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"35.3","version":"0'0","reported_seq":22,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.549093+0000","last_change":"2024-02-23T00:43:02.340363+0000","last_active":"2024-02-23T00:43:15.549093+0000","last_peered":"2024-02-23T00:43:15.549093+0000","last_clean":"2024-02-23T00:43:15.549093+0000","last_became_active":"2024-02-23T00:43:02.340113+0000","last_became_peered":"2024-02-23T00:43:02.340113+0000","last_unstale":"2024-02-23T00:43:15.549093+0000","last_undegraded":"2024-02-23T00:43:15.549093+0000","last_fullsized":"2024-02-23T00:43:15.549093+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:00.812733+0000","last_clean_scrub_stamp":"2024-02-23T00:43:00.812733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"67'8","reported_seq":138,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.546878+0000","last_change":"2024-02-23T00:42:47.985647+0000","last_active":"2024-02-23T00:43:15.546878+0000","last_peered":"2024-02-23T00:43:15.546878+0000","last_clean":"2024-02-23T00:43:15.546878+0000","last_became_active":"2024-02-23T00:39:18.793414+0000","last_became_peered":"2024-02-23T00:39:18.793414+0000","last_unstale":"2024-02-23T00:43:15.546878+0000","last_undegraded":"2024-02-23T00:43:15.546878+0000","last_fullsized":"2024-02-23T00:43:15.546878+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:42:47.985573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:47.985573+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"68'27","reported_seq":138,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.563260+0000","last_change":"2024-02-23T00:42:49.925777+0000","last_active":"2024-02-23T00:43:15.563260+0000","last_peered":"2024-02-23T00:43:15.563260+0000","last_clean":"2024-02-23T00:43:15.563260+0000","last_became_active":"2024-02-23T00:39:16.859401+0000","last_became_peered":"2024-02-23T00:39:16.859401+0000","last_unstale":"2024-02-23T00:43:15.563260+0000","last_undegraded":"2024-02-23T00:43:15.563260+0000","last_fullsized":"2024-02-23T00:43:15.563260+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'27","last_scrub_stamp":"2024-02-23T00:42:49.925673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:49.925673+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":84,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.576291+0000","last_change":"2024-02-23T00:42:04.330493+0000","last_active":"2024-02-23T00:43:15.576291+0000","last_peered":"2024-02-23T00:43:15.576291+0000","last_clean":"2024-02-23T00:43:15.576291+0000","last_became_active":"2024-02-23T00:39:14.784436+0000","last_became_peered":"2024-02-23T00:39:14.784436+0000","last_unstale":"2024-02-23T00:43:15.576291+0000","last_undegraded":"2024-02-23T00:43:15.576291+0000","last_fullsized":"2024-02-23T00:43:15.576291+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:04.330426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:04.330426+0000","last_clean_scrub_stamp":"2024-02-23T00:42:04.330426+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":79,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.537830+0000","last_change":"2024-02-23T00:42:05.725881+0000","last_active":"2024-02-23T00:43:15.537830+0000","last_peered":"2024-02-23T00:43:15.537830+0000","last_clean":"2024-02-23T00:43:15.537830+0000","last_became_active":"2024-02-23T00:39:21.160429+0000","last_became_peered":"2024-02-23T00:39:21.160429+0000","last_unstale":"2024-02-23T00:43:15.537830+0000","last_undegraded":"2024-02-23T00:43:15.537830+0000","last_fullsized":"2024-02-23T00:43:15.537830+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:05.725796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:05.725796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.3","version":"110'5","reported_seq":31,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.539689+0000","last_change":"2024-02-23T00:42:58.836257+0000","last_active":"2024-02-23T00:43:15.539689+0000","last_peered":"2024-02-23T00:43:15.539689+0000","last_clean":"2024-02-23T00:43:15.539689+0000","last_became_active":"2024-02-23T00:42:58.835612+0000","last_became_peered":"2024-02-23T00:42:58.835612+0000","last_unstale":"2024-02-23T00:43:15.539689+0000","last_undegraded":"2024-02-23T00:43:15.539689+0000","last_fullsized":"2024-02-23T00:43:15.539689+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.551335+0000","last_change":"2024-02-23T00:42:58.832536+0000","last_active":"2024-02-23T00:43:15.551335+0000","last_peered":"2024-02-23T00:43:15.551335+0000","last_clean":"2024-02-23T00:43:15.551335+0000","last_became_active":"2024-02-23T00:42:58.828846+0000","last_became_peered":"2024-02-23T00:42:58.828846+0000","last_unstale":"2024-02-23T00:43:15.551335+0000","last_undegraded":"2024-02-23T00:43:15.551335+0000","last_fullsized":"2024-02-23T00:43:15.551335+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"112'2","reported_seq":29,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.558953+0000","last_change":"2024-02-23T00:42:58.732299+0000","last_active":"2024-02-23T00:43:15.558953+0000","last_peered":"2024-02-23T00:43:15.558953+0000","last_clean":"2024-02-23T00:43:15.558953+0000","last_became_active":"2024-02-23T00:42:58.731849+0000","last_became_peered":"2024-02-23T00:42:58.731849+0000","last_unstale":"2024-02-23T00:43:15.558953+0000","last_undegraded":"2024-02-23T00:43:15.558953+0000","last_fullsized":"2024-02-23T00:43:15.558953+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"68'15","reported_seq":114,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.540324+0000","last_change":"2024-02-23T00:42:08.357114+0000","last_active":"2024-02-23T00:43:15.540324+0000","last_peered":"2024-02-23T00:43:15.540324+0000","last_clean":"2024-02-23T00:43:15.540324+0000","last_became_active":"2024-02-23T00:39:16.783701+0000","last_became_peered":"2024-02-23T00:39:16.783701+0000","last_unstale":"2024-02-23T00:43:15.540324+0000","last_undegraded":"2024-02-23T00:43:15.540324+0000","last_fullsized":"2024-02-23T00:43:15.540324+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:42:08.357019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:08.357019+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":84,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.575052+0000","last_change":"2024-02-23T00:42:48.591600+0000","last_active":"2024-02-23T00:43:15.575052+0000","last_peered":"2024-02-23T00:43:15.575052+0000","last_clean":"2024-02-23T00:43:15.575052+0000","last_became_active":"2024-02-23T00:39:14.772493+0000","last_became_peered":"2024-02-23T00:39:14.772493+0000","last_unstale":"2024-02-23T00:43:15.575052+0000","last_undegraded":"2024-02-23T00:43:15.575052+0000","last_fullsized":"2024-02-23T00:43:15.575052+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:48.591486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:48.591486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":80,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.558267+0000","last_change":"2024-02-23T00:42:08.335396+0000","last_active":"2024-02-23T00:43:15.558267+0000","last_peered":"2024-02-23T00:43:15.558267+0000","last_clean":"2024-02-23T00:43:15.558267+0000","last_became_active":"2024-02-23T00:39:18.794476+0000","last_became_peered":"2024-02-23T00:39:18.794476+0000","last_unstale":"2024-02-23T00:43:15.558267+0000","last_undegraded":"2024-02-23T00:43:15.558267+0000","last_fullsized":"2024-02-23T00:43:15.558267+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:08.335330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:08.335330+0000","last_clean_scrub_stamp":"2024-02-23T00:42:08.335330+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":79,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.542111+0000","last_change":"2024-02-23T00:42:46.359743+0000","last_active":"2024-02-23T00:43:15.542111+0000","last_peered":"2024-02-23T00:43:15.542111+0000","last_clean":"2024-02-23T00:43:15.542111+0000","last_became_active":"2024-02-23T00:39:20.802938+0000","last_became_peered":"2024-02-23T00:39:20.802938+0000","last_unstale":"2024-02-23T00:43:15.542111+0000","last_undegraded":"2024-02-23T00:43:15.542111+0000","last_fullsized":"2024-02-23T00:43:15.542111+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:46.359646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:46.359646+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"110'5","reported_seq":31,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.557013+0000","last_change":"2024-02-23T00:42:58.742014+0000","last_active":"2024-02-23T00:43:15.557013+0000","last_peered":"2024-02-23T00:43:15.557013+0000","last_clean":"2024-02-23T00:43:15.557013+0000","last_became_active":"2024-02-23T00:42:58.741558+0000","last_became_peered":"2024-02-23T00:42:58.741558+0000","last_unstale":"2024-02-23T00:43:15.557013+0000","last_undegraded":"2024-02-23T00:43:15.557013+0000","last_fullsized":"2024-02-23T00:43:15.557013+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.546097+0000","last_change":"2024-02-23T00:42:58.871003+0000","last_active":"2024-02-23T00:43:15.546097+0000","last_peered":"2024-02-23T00:43:15.546097+0000","last_clean":"2024-02-23T00:43:15.546097+0000","last_became_active":"2024-02-23T00:42:58.867305+0000","last_became_peered":"2024-02-23T00:42:58.867305+0000","last_unstale":"2024-02-23T00:43:15.546097+0000","last_undegraded":"2024-02-23T00:43:15.546097+0000","last_fullsized":"2024-02-23T00:43:15.546097+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.1b","version":"110'1","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.566293+0000","last_change":"2024-02-23T00:42:58.744995+0000","last_active":"2024-02-23T00:43:15.566293+0000","last_peered":"2024-02-23T00:43:15.566293+0000","last_clean":"2024-02-23T00:43:15.566293+0000","last_became_active":"2024-02-23T00:42:58.744766+0000","last_became_peered":"2024-02-23T00:42:58.744766+0000","last_unstale":"2024-02-23T00:43:15.566293+0000","last_undegraded":"2024-02-23T00:43:15.566293+0000","last_fullsized":"2024-02-23T00:43:15.566293+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"88'35","reported_seq":138,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.546325+0000","last_change":"2024-02-23T00:41:58.120657+0000","last_active":"2024-02-23T00:43:15.546325+0000","last_peered":"2024-02-23T00:43:15.546325+0000","last_clean":"2024-02-23T00:43:15.546325+0000","last_became_active":"2024-02-23T00:39:16.857413+0000","last_became_peered":"2024-02-23T00:39:16.857413+0000","last_unstale":"2024-02-23T00:43:15.546325+0000","last_undegraded":"2024-02-23T00:43:15.546325+0000","last_fullsized":"2024-02-23T00:43:15.546325+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'35","last_scrub_stamp":"2024-02-23T00:41:58.120561+0000","last_deep_scrub":"88'35","last_deep_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_clean_scrub_stamp":"2024-02-23T00:41:58.120561+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":84,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.567298+0000","last_change":"2024-02-23T00:41:54.303858+0000","last_active":"2024-02-23T00:43:15.567298+0000","last_peered":"2024-02-23T00:43:15.567298+0000","last_clean":"2024-02-23T00:43:15.567298+0000","last_became_active":"2024-02-23T00:39:14.776705+0000","last_became_peered":"2024-02-23T00:39:14.776705+0000","last_unstale":"2024-02-23T00:43:15.567298+0000","last_undegraded":"2024-02-23T00:43:15.567298+0000","last_fullsized":"2024-02-23T00:43:15.567298+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:54.303790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_clean_scrub_stamp":"2024-02-23T00:41:54.303790+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":80,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.565461+0000","last_change":"2024-02-23T00:42:03.291538+0000","last_active":"2024-02-23T00:43:15.565461+0000","last_peered":"2024-02-23T00:43:15.565461+0000","last_clean":"2024-02-23T00:43:15.565461+0000","last_became_active":"2024-02-23T00:39:18.792718+0000","last_became_peered":"2024-02-23T00:39:18.792718+0000","last_unstale":"2024-02-23T00:43:15.565461+0000","last_undegraded":"2024-02-23T00:43:15.565461+0000","last_fullsized":"2024-02-23T00:43:15.565461+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:03.291436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:03.291436+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":77,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.545398+0000","last_change":"2024-02-23T00:42:43.027150+0000","last_active":"2024-02-23T00:43:15.545398+0000","last_peered":"2024-02-23T00:43:15.545398+0000","last_clean":"2024-02-23T00:43:15.545398+0000","last_became_active":"2024-02-23T00:39:20.798989+0000","last_became_peered":"2024-02-23T00:39:20.798989+0000","last_unstale":"2024-02-23T00:43:15.545398+0000","last_undegraded":"2024-02-23T00:43:15.545398+0000","last_fullsized":"2024-02-23T00:43:15.545398+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:43.027029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:43.027029+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"110'5","reported_seq":31,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.558260+0000","last_change":"2024-02-23T00:42:58.688988+0000","last_active":"2024-02-23T00:43:15.558260+0000","last_peered":"2024-02-23T00:43:15.558260+0000","last_clean":"2024-02-23T00:43:15.558260+0000","last_became_active":"2024-02-23T00:42:58.684001+0000","last_became_peered":"2024-02-23T00:42:58.684001+0000","last_unstale":"2024-02-23T00:43:15.558260+0000","last_undegraded":"2024-02-23T00:43:15.558260+0000","last_fullsized":"2024-02-23T00:43:15.558260+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.581778+0000","last_change":"2024-02-23T00:42:57.641727+0000","last_active":"2024-02-23T00:43:15.581778+0000","last_peered":"2024-02-23T00:43:15.581778+0000","last_clean":"2024-02-23T00:43:15.581778+0000","last_became_active":"2024-02-23T00:42:57.639013+0000","last_became_peered":"2024-02-23T00:42:57.639013+0000","last_unstale":"2024-02-23T00:43:15.581778+0000","last_undegraded":"2024-02-23T00:43:15.581778+0000","last_fullsized":"2024-02-23T00:43:15.581778+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1a","version":"112'5","reported_seq":30,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.538972+0000","last_change":"2024-02-23T00:42:58.827911+0000","last_active":"2024-02-23T00:43:15.538972+0000","last_peered":"2024-02-23T00:43:15.538972+0000","last_clean":"2024-02-23T00:43:15.538972+0000","last_became_active":"2024-02-23T00:42:58.827630+0000","last_became_peered":"2024-02-23T00:42:58.827630+0000","last_unstale":"2024-02-23T00:43:15.538972+0000","last_undegraded":"2024-02-23T00:43:15.538972+0000","last_fullsized":"2024-02-23T00:43:15.538972+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25168628,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":27328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0],"acting":[5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"67'17","reported_seq":132,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.549615+0000","last_change":"2024-02-23T00:41:02.038717+0000","last_active":"2024-02-23T00:43:15.549615+0000","last_peered":"2024-02-23T00:43:15.549615+0000","last_clean":"2024-02-23T00:43:15.549615+0000","last_became_active":"2024-02-23T00:40:42.933806+0000","last_became_peered":"2024-02-23T00:40:42.933806+0000","last_unstale":"2024-02-23T00:43:15.549615+0000","last_undegraded":"2024-02-23T00:43:15.549615+0000","last_fullsized":"2024-02-23T00:43:15.549615+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'17","last_scrub_stamp":"2024-02-23T00:41:02.038627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:02.038627+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":55,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.566209+0000","last_change":"2024-02-23T00:41:40.128467+0000","last_active":"2024-02-23T00:43:15.566209+0000","last_peered":"2024-02-23T00:43:15.566209+0000","last_clean":"2024-02-23T00:43:15.566209+0000","last_became_active":"2024-02-23T00:40:42.931041+0000","last_became_peered":"2024-02-23T00:40:42.931041+0000","last_unstale":"2024-02-23T00:43:15.566209+0000","last_undegraded":"2024-02-23T00:43:15.566209+0000","last_fullsized":"2024-02-23T00:43:15.566209+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:40.128397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:40.128397+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":80,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.590290+0000","last_change":"2024-02-23T00:41:53.692312+0000","last_active":"2024-02-23T00:43:15.590290+0000","last_peered":"2024-02-23T00:43:15.590290+0000","last_clean":"2024-02-23T00:43:15.590290+0000","last_became_active":"2024-02-23T00:39:18.799875+0000","last_became_peered":"2024-02-23T00:39:18.799875+0000","last_unstale":"2024-02-23T00:43:15.590290+0000","last_undegraded":"2024-02-23T00:43:15.590290+0000","last_fullsized":"2024-02-23T00:43:15.590290+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:53.692185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:53.692185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":96,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.583601+0000","last_change":"2024-02-23T00:43:08.885549+0000","last_active":"2024-02-23T00:43:15.583601+0000","last_peered":"2024-02-23T00:43:15.583601+0000","last_clean":"2024-02-23T00:43:15.583601+0000","last_became_active":"2024-02-23T00:43:08.885257+0000","last_became_peered":"2024-02-23T00:43:08.885257+0000","last_unstale":"2024-02-23T00:43:15.583601+0000","last_undegraded":"2024-02-23T00:43:15.583601+0000","last_fullsized":"2024-02-23T00:43:15.583601+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:37.180472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:37.180472+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.6","version":"110'9","reported_seq":35,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.545218+0000","last_change":"2024-02-23T00:42:58.871934+0000","last_active":"2024-02-23T00:43:15.545218+0000","last_peered":"2024-02-23T00:43:15.545218+0000","last_clean":"2024-02-23T00:43:15.545218+0000","last_became_active":"2024-02-23T00:42:58.870147+0000","last_became_peered":"2024-02-23T00:42:58.870147+0000","last_unstale":"2024-02-23T00:43:15.545218+0000","last_undegraded":"2024-02-23T00:43:15.545218+0000","last_fullsized":"2024-02-23T00:43:15.545218+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.560650+0000","last_change":"2024-02-23T00:42:57.683188+0000","last_active":"2024-02-23T00:43:15.560650+0000","last_peered":"2024-02-23T00:43:15.560650+0000","last_clean":"2024-02-23T00:43:15.560650+0000","last_became_active":"2024-02-23T00:42:57.675306+0000","last_became_peered":"2024-02-23T00:42:57.675306+0000","last_unstale":"2024-02-23T00:43:15.560650+0000","last_undegraded":"2024-02-23T00:43:15.560650+0000","last_fullsized":"2024-02-23T00:43:15.560650+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":25,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.549701+0000","last_change":"2024-02-23T00:42:58.827765+0000","last_active":"2024-02-23T00:43:15.549701+0000","last_peered":"2024-02-23T00:43:15.549701+0000","last_clean":"2024-02-23T00:43:15.549701+0000","last_became_active":"2024-02-23T00:42:58.826869+0000","last_became_peered":"2024-02-23T00:42:58.826869+0000","last_unstale":"2024-02-23T00:43:15.549701+0000","last_undegraded":"2024-02-23T00:43:15.549701+0000","last_fullsized":"2024-02-23T00:43:15.549701+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.c","version":"67'10","reported_seq":97,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.554529+0000","last_change":"2024-02-23T00:42:50.078150+0000","last_active":"2024-02-23T00:43:15.554529+0000","last_peered":"2024-02-23T00:43:15.554529+0000","last_clean":"2024-02-23T00:43:15.554529+0000","last_became_active":"2024-02-23T00:39:16.777530+0000","last_became_peered":"2024-02-23T00:39:16.777530+0000","last_unstale":"2024-02-23T00:43:15.554529+0000","last_undegraded":"2024-02-23T00:43:15.554529+0000","last_fullsized":"2024-02-23T00:43:15.554529+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'10","last_scrub_stamp":"2024-02-23T00:42:50.078042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:50.078042+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":84,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.553776+0000","last_change":"2024-02-23T00:42:06.575346+0000","last_active":"2024-02-23T00:43:15.553776+0000","last_peered":"2024-02-23T00:43:15.553776+0000","last_clean":"2024-02-23T00:43:15.553776+0000","last_became_active":"2024-02-23T00:39:14.780379+0000","last_became_peered":"2024-02-23T00:39:14.780379+0000","last_unstale":"2024-02-23T00:43:15.553776+0000","last_undegraded":"2024-02-23T00:43:15.553776+0000","last_fullsized":"2024-02-23T00:43:15.553776+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:06.575242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.575242+0000","last_clean_scrub_stamp":"2024-02-23T00:42:06.575242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"67'8","reported_seq":136,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.561322+0000","last_change":"2024-02-23T00:42:36.214941+0000","last_active":"2024-02-23T00:43:15.561322+0000","last_peered":"2024-02-23T00:43:15.561322+0000","last_clean":"2024-02-23T00:43:15.561322+0000","last_became_active":"2024-02-23T00:39:18.799544+0000","last_became_peered":"2024-02-23T00:39:18.799544+0000","last_unstale":"2024-02-23T00:43:15.561322+0000","last_undegraded":"2024-02-23T00:43:15.561322+0000","last_fullsized":"2024-02-23T00:43:15.561322+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:42:36.214826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:36.214826+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":91,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.553472+0000","last_change":"2024-02-23T00:43:13.339003+0000","last_active":"2024-02-23T00:43:15.553472+0000","last_peered":"2024-02-23T00:43:15.553472+0000","last_clean":"2024-02-23T00:43:15.553472+0000","last_became_active":"2024-02-23T00:43:13.338697+0000","last_became_peered":"2024-02-23T00:43:13.338697+0000","last_unstale":"2024-02-23T00:43:15.553472+0000","last_undegraded":"2024-02-23T00:43:15.553472+0000","last_fullsized":"2024-02-23T00:43:15.553472+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:15.081153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:15.081153+0000","last_clean_scrub_stamp":"2024-02-23T00:41:15.081153+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.586346+0000","last_change":"2024-02-23T00:42:57.686234+0000","last_active":"2024-02-23T00:43:15.586346+0000","last_peered":"2024-02-23T00:43:15.586346+0000","last_clean":"2024-02-23T00:43:15.586346+0000","last_became_active":"2024-02-23T00:42:57.685579+0000","last_became_peered":"2024-02-23T00:42:57.685579+0000","last_unstale":"2024-02-23T00:43:15.586346+0000","last_undegraded":"2024-02-23T00:43:15.586346+0000","last_fullsized":"2024-02-23T00:43:15.586346+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.550171+0000","last_change":"2024-02-23T00:42:58.772582+0000","last_active":"2024-02-23T00:43:15.550171+0000","last_peered":"2024-02-23T00:43:15.550171+0000","last_clean":"2024-02-23T00:43:15.550171+0000","last_became_active":"2024-02-23T00:42:58.765106+0000","last_became_peered":"2024-02-23T00:42:58.765106+0000","last_unstale":"2024-02-23T00:43:15.550171+0000","last_undegraded":"2024-02-23T00:43:15.550171+0000","last_fullsized":"2024-02-23T00:43:15.550171+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1c","version":"110'1","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.554630+0000","last_change":"2024-02-23T00:42:58.731884+0000","last_active":"2024-02-23T00:43:15.554630+0000","last_peered":"2024-02-23T00:43:15.554630+0000","last_clean":"2024-02-23T00:43:15.554630+0000","last_became_active":"2024-02-23T00:42:58.727707+0000","last_became_peered":"2024-02-23T00:42:58.727707+0000","last_unstale":"2024-02-23T00:43:15.554630+0000","last_undegraded":"2024-02-23T00:43:15.554630+0000","last_fullsized":"2024-02-23T00:43:15.554630+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"67'15","reported_seq":107,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.586441+0000","last_change":"2024-02-23T00:41:39.682283+0000","last_active":"2024-02-23T00:43:15.586441+0000","last_peered":"2024-02-23T00:43:15.586441+0000","last_clean":"2024-02-23T00:43:15.586441+0000","last_became_active":"2024-02-23T00:39:16.789757+0000","last_became_peered":"2024-02-23T00:39:16.789757+0000","last_unstale":"2024-02-23T00:43:15.586441+0000","last_undegraded":"2024-02-23T00:43:15.586441+0000","last_fullsized":"2024-02-23T00:43:15.586441+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:41:39.682182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:39.682182+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":84,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.553106+0000","last_change":"2024-02-23T00:41:56.728679+0000","last_active":"2024-02-23T00:43:15.553106+0000","last_peered":"2024-02-23T00:43:15.553106+0000","last_clean":"2024-02-23T00:43:15.553106+0000","last_became_active":"2024-02-23T00:39:14.780526+0000","last_became_peered":"2024-02-23T00:39:14.780526+0000","last_unstale":"2024-02-23T00:43:15.553106+0000","last_undegraded":"2024-02-23T00:43:15.553106+0000","last_fullsized":"2024-02-23T00:43:15.553106+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:56.728577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:56.728577+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":136,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.553357+0000","last_change":"2024-02-23T00:41:49.927409+0000","last_active":"2024-02-23T00:43:15.553357+0000","last_peered":"2024-02-23T00:43:15.553357+0000","last_clean":"2024-02-23T00:43:15.553357+0000","last_became_active":"2024-02-23T00:39:18.800440+0000","last_became_peered":"2024-02-23T00:39:18.800440+0000","last_unstale":"2024-02-23T00:43:15.553357+0000","last_undegraded":"2024-02-23T00:43:15.553357+0000","last_fullsized":"2024-02-23T00:43:15.553357+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:41:49.927326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:49.927326+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":97,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.650829+0000","last_change":"2024-02-23T00:43:16.650829+0000","last_active":"2024-02-23T00:43:16.650829+0000","last_peered":"2024-02-23T00:43:16.650829+0000","last_clean":"2024-02-23T00:43:16.650829+0000","last_became_active":"2024-02-23T00:43:16.650527+0000","last_became_peered":"2024-02-23T00:43:16.650527+0000","last_unstale":"2024-02-23T00:43:16.650829+0000","last_undegraded":"2024-02-23T00:43:16.650829+0000","last_fullsized":"2024-02-23T00:43:16.650829+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:10.666299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:10.666299+0000","last_clean_scrub_stamp":"2024-02-23T00:42:10.666299+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.581851+0000","last_change":"2024-02-23T00:42:57.671776+0000","last_active":"2024-02-23T00:43:15.581851+0000","last_peered":"2024-02-23T00:43:15.581851+0000","last_clean":"2024-02-23T00:43:15.581851+0000","last_became_active":"2024-02-23T00:42:57.671472+0000","last_became_peered":"2024-02-23T00:42:57.671472+0000","last_unstale":"2024-02-23T00:43:15.581851+0000","last_undegraded":"2024-02-23T00:43:15.581851+0000","last_fullsized":"2024-02-23T00:43:15.581851+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,3],"acting":[1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.546618+0000","last_change":"2024-02-23T00:42:58.834770+0000","last_active":"2024-02-23T00:43:15.546618+0000","last_peered":"2024-02-23T00:43:15.546618+0000","last_clean":"2024-02-23T00:43:15.546618+0000","last_became_active":"2024-02-23T00:42:58.833924+0000","last_became_peered":"2024-02-23T00:42:58.833924+0000","last_unstale":"2024-02-23T00:43:15.546618+0000","last_undegraded":"2024-02-23T00:43:15.546618+0000","last_fullsized":"2024-02-23T00:43:15.546618+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":25,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.556427+0000","last_change":"2024-02-23T00:42:58.749425+0000","last_active":"2024-02-23T00:43:15.556427+0000","last_peered":"2024-02-23T00:43:15.556427+0000","last_clean":"2024-02-23T00:43:15.556427+0000","last_became_active":"2024-02-23T00:42:58.744735+0000","last_became_peered":"2024-02-23T00:42:58.744735+0000","last_unstale":"2024-02-23T00:43:15.556427+0000","last_undegraded":"2024-02-23T00:43:15.556427+0000","last_fullsized":"2024-02-23T00:43:15.556427+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"68'17","reported_seq":134,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.542727+0000","last_change":"2024-02-23T00:41:13.017122+0000","last_active":"2024-02-23T00:43:15.542727+0000","last_peered":"2024-02-23T00:43:15.542727+0000","last_clean":"2024-02-23T00:43:15.542727+0000","last_became_active":"2024-02-23T00:40:42.933759+0000","last_became_peered":"2024-02-23T00:40:42.933759+0000","last_unstale":"2024-02-23T00:43:15.542727+0000","last_undegraded":"2024-02-23T00:43:15.542727+0000","last_fullsized":"2024-02-23T00:43:15.542727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:41:13.017043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:13.017043+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":83,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.546290+0000","last_change":"2024-02-23T00:41:42.978987+0000","last_active":"2024-02-23T00:43:15.546290+0000","last_peered":"2024-02-23T00:43:15.546290+0000","last_clean":"2024-02-23T00:43:15.546290+0000","last_became_active":"2024-02-23T00:39:14.784121+0000","last_became_peered":"2024-02-23T00:39:14.784121+0000","last_unstale":"2024-02-23T00:43:15.546290+0000","last_undegraded":"2024-02-23T00:43:15.546290+0000","last_fullsized":"2024-02-23T00:43:15.546290+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:42.978905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:42.978905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":81,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.542861+0000","last_change":"2024-02-23T00:42:53.965005+0000","last_active":"2024-02-23T00:43:15.542861+0000","last_peered":"2024-02-23T00:43:15.542861+0000","last_clean":"2024-02-23T00:43:15.542861+0000","last_became_active":"2024-02-23T00:39:18.800777+0000","last_became_peered":"2024-02-23T00:39:18.800777+0000","last_unstale":"2024-02-23T00:43:15.542861+0000","last_undegraded":"2024-02-23T00:43:15.542861+0000","last_fullsized":"2024-02-23T00:43:15.542861+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.964934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.964934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":79,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.541905+0000","last_change":"2024-02-23T00:42:09.080023+0000","last_active":"2024-02-23T00:43:15.541905+0000","last_peered":"2024-02-23T00:43:15.541905+0000","last_clean":"2024-02-23T00:43:15.541905+0000","last_became_active":"2024-02-23T00:39:21.160443+0000","last_became_peered":"2024-02-23T00:39:21.160443+0000","last_unstale":"2024-02-23T00:43:15.541905+0000","last_undegraded":"2024-02-23T00:43:15.541905+0000","last_fullsized":"2024-02-23T00:43:15.541905+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:09.079918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:09.079918+0000","last_clean_scrub_stamp":"2024-02-23T00:42:09.079918+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.541869+0000","last_change":"2024-02-23T00:42:58.719381+0000","last_active":"2024-02-23T00:43:15.541869+0000","last_peered":"2024-02-23T00:43:15.541869+0000","last_clean":"2024-02-23T00:43:15.541869+0000","last_became_active":"2024-02-23T00:42:58.718615+0000","last_became_peered":"2024-02-23T00:42:58.718615+0000","last_unstale":"2024-02-23T00:43:15.541869+0000","last_undegraded":"2024-02-23T00:43:15.541869+0000","last_fullsized":"2024-02-23T00:43:15.541869+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.547265+0000","last_change":"2024-02-23T00:42:57.655252+0000","last_active":"2024-02-23T00:43:15.547265+0000","last_peered":"2024-02-23T00:43:15.547265+0000","last_clean":"2024-02-23T00:43:15.547265+0000","last_became_active":"2024-02-23T00:42:57.653279+0000","last_became_peered":"2024-02-23T00:42:57.653279+0000","last_unstale":"2024-02-23T00:43:15.547265+0000","last_undegraded":"2024-02-23T00:43:15.547265+0000","last_fullsized":"2024-02-23T00:43:15.547265+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1e","version":"112'1004","reported_seq":2033,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.547524+0000","last_change":"2024-02-23T00:42:58.808262+0000","last_active":"2024-02-23T00:43:15.547524+0000","last_peered":"2024-02-23T00:43:15.547524+0000","last_clean":"2024-02-23T00:43:15.547524+0000","last_became_active":"2024-02-23T00:42:58.807886+0000","last_became_peered":"2024-02-23T00:42:58.807886+0000","last_unstale":"2024-02-23T00:43:15.547524+0000","last_undegraded":"2024-02-23T00:43:15.547524+0000","last_fullsized":"2024-02-23T00:43:15.547524+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":1004,"ondisk_log_size":1004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25165850,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1001,"num_read_kb":24577,"num_write":1003,"num_write_kb":24579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":137,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.539113+0000","last_change":"2024-02-23T00:42:53.978054+0000","last_active":"2024-02-23T00:43:15.539113+0000","last_peered":"2024-02-23T00:43:15.539113+0000","last_clean":"2024-02-23T00:43:15.539113+0000","last_became_active":"2024-02-23T00:39:18.798536+0000","last_became_peered":"2024-02-23T00:39:18.798536+0000","last_unstale":"2024-02-23T00:43:15.539113+0000","last_undegraded":"2024-02-23T00:43:15.539113+0000","last_fullsized":"2024-02-23T00:43:15.539113+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:42:53.977950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.977950+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"68'20","reported_seq":118,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.547433+0000","last_change":"2024-02-23T00:42:42.204029+0000","last_active":"2024-02-23T00:43:15.547433+0000","last_peered":"2024-02-23T00:43:15.547433+0000","last_clean":"2024-02-23T00:43:15.547433+0000","last_became_active":"2024-02-23T00:39:16.780130+0000","last_became_peered":"2024-02-23T00:39:16.780130+0000","last_unstale":"2024-02-23T00:43:15.547433+0000","last_undegraded":"2024-02-23T00:43:15.547433+0000","last_fullsized":"2024-02-23T00:43:15.547433+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'20","last_scrub_stamp":"2024-02-23T00:42:42.203911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:42.203911+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":86,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.551493+0000","last_change":"2024-02-23T00:42:04.191017+0000","last_active":"2024-02-23T00:43:15.551493+0000","last_peered":"2024-02-23T00:43:15.551493+0000","last_clean":"2024-02-23T00:43:15.551493+0000","last_became_active":"2024-02-23T00:39:14.781999+0000","last_became_peered":"2024-02-23T00:39:14.781999+0000","last_unstale":"2024-02-23T00:43:15.551493+0000","last_undegraded":"2024-02-23T00:43:15.551493+0000","last_fullsized":"2024-02-23T00:43:15.551493+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:04.190935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:04.190935+0000","last_clean_scrub_stamp":"2024-02-23T00:42:04.190935+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":77,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.552460+0000","last_change":"2024-02-23T00:42:06.945763+0000","last_active":"2024-02-23T00:43:15.552460+0000","last_peered":"2024-02-23T00:43:15.552460+0000","last_clean":"2024-02-23T00:43:15.552460+0000","last_became_active":"2024-02-23T00:39:21.161928+0000","last_became_peered":"2024-02-23T00:39:21.161928+0000","last_unstale":"2024-02-23T00:43:15.552460+0000","last_undegraded":"2024-02-23T00:43:15.552460+0000","last_fullsized":"2024-02-23T00:43:15.552460+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:06.945642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.945642+0000","last_clean_scrub_stamp":"2024-02-23T00:42:06.945642+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"110'10","reported_seq":36,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.558501+0000","last_change":"2024-02-23T00:42:58.696579+0000","last_active":"2024-02-23T00:43:15.558501+0000","last_peered":"2024-02-23T00:43:15.558501+0000","last_clean":"2024-02-23T00:43:15.558501+0000","last_became_active":"2024-02-23T00:42:58.696292+0000","last_became_peered":"2024-02-23T00:42:58.696292+0000","last_unstale":"2024-02-23T00:43:15.558501+0000","last_undegraded":"2024-02-23T00:43:15.558501+0000","last_fullsized":"2024-02-23T00:43:15.558501+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.543116+0000","last_change":"2024-02-23T00:42:58.717788+0000","last_active":"2024-02-23T00:43:15.543116+0000","last_peered":"2024-02-23T00:43:15.543116+0000","last_clean":"2024-02-23T00:43:15.543116+0000","last_became_active":"2024-02-23T00:42:58.717371+0000","last_became_peered":"2024-02-23T00:42:58.717371+0000","last_unstale":"2024-02-23T00:43:15.543116+0000","last_undegraded":"2024-02-23T00:43:15.543116+0000","last_fullsized":"2024-02-23T00:43:15.543116+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"112'6","reported_seq":34,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.553297+0000","last_change":"2024-02-23T00:42:58.762595+0000","last_active":"2024-02-23T00:43:15.553297+0000","last_peered":"2024-02-23T00:43:15.553297+0000","last_clean":"2024-02-23T00:43:15.553297+0000","last_became_active":"2024-02-23T00:42:58.757008+0000","last_became_peered":"2024-02-23T00:42:58.757008+0000","last_unstale":"2024-02-23T00:43:15.553297+0000","last_undegraded":"2024-02-23T00:43:15.553297+0000","last_fullsized":"2024-02-23T00:43:15.553297+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":80,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.548302+0000","last_change":"2024-02-23T00:41:52.965337+0000","last_active":"2024-02-23T00:43:15.548302+0000","last_peered":"2024-02-23T00:43:15.548302+0000","last_clean":"2024-02-23T00:43:15.548302+0000","last_became_active":"2024-02-23T00:39:18.798829+0000","last_became_peered":"2024-02-23T00:39:18.798829+0000","last_unstale":"2024-02-23T00:43:15.548302+0000","last_undegraded":"2024-02-23T00:43:15.548302+0000","last_fullsized":"2024-02-23T00:43:15.548302+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:52.965253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:52.965253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"68'30","reported_seq":145,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.539027+0000","last_change":"2024-02-23T00:42:49.970825+0000","last_active":"2024-02-23T00:43:15.539027+0000","last_peered":"2024-02-23T00:43:15.539027+0000","last_clean":"2024-02-23T00:43:15.539027+0000","last_became_active":"2024-02-23T00:39:16.859002+0000","last_became_peered":"2024-02-23T00:39:16.859002+0000","last_unstale":"2024-02-23T00:43:15.539027+0000","last_undegraded":"2024-02-23T00:43:15.539027+0000","last_fullsized":"2024-02-23T00:43:15.539027+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'30","last_scrub_stamp":"2024-02-23T00:42:49.970752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:49.970752+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":84,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.572015+0000","last_change":"2024-02-23T00:42:40.534390+0000","last_active":"2024-02-23T00:43:15.572015+0000","last_peered":"2024-02-23T00:43:15.572015+0000","last_clean":"2024-02-23T00:43:15.572015+0000","last_became_active":"2024-02-23T00:39:14.779193+0000","last_became_peered":"2024-02-23T00:39:14.779193+0000","last_unstale":"2024-02-23T00:43:15.572015+0000","last_undegraded":"2024-02-23T00:43:15.572015+0000","last_fullsized":"2024-02-23T00:43:15.572015+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:40.534284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:40.534284+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":78,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.558475+0000","last_change":"2024-02-23T00:42:36.113546+0000","last_active":"2024-02-23T00:43:15.558475+0000","last_peered":"2024-02-23T00:43:15.558475+0000","last_clean":"2024-02-23T00:43:15.558475+0000","last_became_active":"2024-02-23T00:39:20.795271+0000","last_became_peered":"2024-02-23T00:39:20.795271+0000","last_unstale":"2024-02-23T00:43:15.558475+0000","last_undegraded":"2024-02-23T00:43:15.558475+0000","last_fullsized":"2024-02-23T00:43:15.558475+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:36.113469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:36.113469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"110'2","reported_seq":28,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.565385+0000","last_change":"2024-02-23T00:42:58.816302+0000","last_active":"2024-02-23T00:43:15.565385+0000","last_peered":"2024-02-23T00:43:15.565385+0000","last_clean":"2024-02-23T00:43:15.565385+0000","last_became_active":"2024-02-23T00:42:58.814093+0000","last_became_peered":"2024-02-23T00:42:58.814093+0000","last_unstale":"2024-02-23T00:43:15.565385+0000","last_undegraded":"2024-02-23T00:43:15.565385+0000","last_fullsized":"2024-02-23T00:43:15.565385+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.547377+0000","last_change":"2024-02-23T00:42:58.809948+0000","last_active":"2024-02-23T00:43:15.547377+0000","last_peered":"2024-02-23T00:43:15.547377+0000","last_clean":"2024-02-23T00:43:15.547377+0000","last_became_active":"2024-02-23T00:42:58.809283+0000","last_became_peered":"2024-02-23T00:42:58.809283+0000","last_unstale":"2024-02-23T00:43:15.547377+0000","last_undegraded":"2024-02-23T00:43:15.547377+0000","last_fullsized":"2024-02-23T00:43:15.547377+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.10","version":"112'3","reported_seq":36,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.557097+0000","last_change":"2024-02-23T00:42:58.700115+0000","last_active":"2024-02-23T00:43:15.557097+0000","last_peered":"2024-02-23T00:43:15.557097+0000","last_clean":"2024-02-23T00:43:15.557097+0000","last_became_active":"2024-02-23T00:42:58.699664+0000","last_became_peered":"2024-02-23T00:42:58.699664+0000","last_unstale":"2024-02-23T00:43:15.557097+0000","last_undegraded":"2024-02-23T00:43:15.557097+0000","last_fullsized":"2024-02-23T00:43:15.557097+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25165843,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":24578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":80,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.545970+0000","last_change":"2024-02-23T00:42:36.454764+0000","last_active":"2024-02-23T00:43:15.545970+0000","last_peered":"2024-02-23T00:43:15.545970+0000","last_clean":"2024-02-23T00:43:15.545970+0000","last_became_active":"2024-02-23T00:39:18.796664+0000","last_became_peered":"2024-02-23T00:39:18.796664+0000","last_unstale":"2024-02-23T00:43:15.545970+0000","last_undegraded":"2024-02-23T00:43:15.545970+0000","last_fullsized":"2024-02-23T00:43:15.545970+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:36.454660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:36.454660+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"68'25","reported_seq":134,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.537313+0000","last_change":"2024-02-23T00:42:34.248481+0000","last_active":"2024-02-23T00:43:15.537313+0000","last_peered":"2024-02-23T00:43:15.537313+0000","last_clean":"2024-02-23T00:43:15.537313+0000","last_became_active":"2024-02-23T00:39:16.857278+0000","last_became_peered":"2024-02-23T00:39:16.857278+0000","last_unstale":"2024-02-23T00:43:15.537313+0000","last_undegraded":"2024-02-23T00:43:15.537313+0000","last_fullsized":"2024-02-23T00:43:15.537313+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'25","last_scrub_stamp":"2024-02-23T00:42:34.248371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:34.248371+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":83,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.556017+0000","last_change":"2024-02-23T00:42:43.990569+0000","last_active":"2024-02-23T00:43:15.556017+0000","last_peered":"2024-02-23T00:43:15.556017+0000","last_clean":"2024-02-23T00:43:15.556017+0000","last_became_active":"2024-02-23T00:39:14.783662+0000","last_became_peered":"2024-02-23T00:39:14.783662+0000","last_unstale":"2024-02-23T00:43:15.556017+0000","last_undegraded":"2024-02-23T00:43:15.556017+0000","last_fullsized":"2024-02-23T00:43:15.556017+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:43.990486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:43.990486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":78,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.553209+0000","last_change":"2024-02-23T00:41:50.656785+0000","last_active":"2024-02-23T00:43:15.553209+0000","last_peered":"2024-02-23T00:43:15.553209+0000","last_clean":"2024-02-23T00:43:15.553209+0000","last_became_active":"2024-02-23T00:39:21.160060+0000","last_became_peered":"2024-02-23T00:39:21.160060+0000","last_unstale":"2024-02-23T00:43:15.553209+0000","last_undegraded":"2024-02-23T00:43:15.553209+0000","last_fullsized":"2024-02-23T00:43:15.553209+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:50.656709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:50.656709+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.8","version":"110'2","reported_seq":28,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.556063+0000","last_change":"2024-02-23T00:42:58.810425+0000","last_active":"2024-02-23T00:43:15.556063+0000","last_peered":"2024-02-23T00:43:15.556063+0000","last_clean":"2024-02-23T00:43:15.556063+0000","last_became_active":"2024-02-23T00:42:58.809584+0000","last_became_peered":"2024-02-23T00:42:58.809584+0000","last_unstale":"2024-02-23T00:43:15.556063+0000","last_undegraded":"2024-02-23T00:43:15.556063+0000","last_fullsized":"2024-02-23T00:43:15.556063+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.545797+0000","last_change":"2024-02-23T00:42:58.726955+0000","last_active":"2024-02-23T00:43:15.545797+0000","last_peered":"2024-02-23T00:43:15.545797+0000","last_clean":"2024-02-23T00:43:15.545797+0000","last_became_active":"2024-02-23T00:42:58.726652+0000","last_became_peered":"2024-02-23T00:42:58.726652+0000","last_unstale":"2024-02-23T00:43:15.545797+0000","last_undegraded":"2024-02-23T00:43:15.545797+0000","last_fullsized":"2024-02-23T00:43:15.545797+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.13","version":"112'1","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.543937+0000","last_change":"2024-02-23T00:42:58.804930+0000","last_active":"2024-02-23T00:43:15.543937+0000","last_peered":"2024-02-23T00:43:15.543937+0000","last_clean":"2024-02-23T00:43:15.543937+0000","last_became_active":"2024-02-23T00:42:58.804240+0000","last_became_peered":"2024-02-23T00:42:58.804240+0000","last_unstale":"2024-02-23T00:43:15.543937+0000","last_undegraded":"2024-02-23T00:43:15.543937+0000","last_fullsized":"2024-02-23T00:43:15.543937+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25165828,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":24577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7],"acting":[6,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":81,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.542940+0000","last_change":"2024-02-23T00:41:59.694132+0000","last_active":"2024-02-23T00:43:15.542940+0000","last_peered":"2024-02-23T00:43:15.542940+0000","last_clean":"2024-02-23T00:43:15.542940+0000","last_became_active":"2024-02-23T00:39:18.800378+0000","last_became_peered":"2024-02-23T00:39:18.800378+0000","last_unstale":"2024-02-23T00:43:15.542940+0000","last_undegraded":"2024-02-23T00:43:15.542940+0000","last_fullsized":"2024-02-23T00:43:15.542940+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:59.694033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:59.694033+0000","last_clean_scrub_stamp":"2024-02-23T00:41:59.694033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"95'18","reported_seq":107,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.544839+0000","last_change":"2024-02-23T00:42:01.195610+0000","last_active":"2024-02-23T00:43:15.544839+0000","last_peered":"2024-02-23T00:43:15.544839+0000","last_clean":"2024-02-23T00:43:15.544839+0000","last_became_active":"2024-02-23T00:39:16.787143+0000","last_became_peered":"2024-02-23T00:39:16.787143+0000","last_unstale":"2024-02-23T00:43:15.544839+0000","last_undegraded":"2024-02-23T00:43:15.544839+0000","last_fullsized":"2024-02-23T00:43:15.544839+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'18","last_scrub_stamp":"2024-02-23T00:42:01.195545+0000","last_deep_scrub":"95'18","last_deep_scrub_stamp":"2024-02-23T00:42:01.195545+0000","last_clean_scrub_stamp":"2024-02-23T00:42:01.195545+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":88,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.546040+0000","last_change":"2024-02-23T00:42:50.947420+0000","last_active":"2024-02-23T00:43:15.546040+0000","last_peered":"2024-02-23T00:43:15.546040+0000","last_clean":"2024-02-23T00:43:15.546040+0000","last_became_active":"2024-02-23T00:39:14.784578+0000","last_became_peered":"2024-02-23T00:39:14.784578+0000","last_unstale":"2024-02-23T00:43:15.546040+0000","last_undegraded":"2024-02-23T00:43:15.546040+0000","last_fullsized":"2024-02-23T00:43:15.546040+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:50.947349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:50.947349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":85,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.541831+0000","last_change":"2024-02-23T00:41:49.197029+0000","last_active":"2024-02-23T00:43:15.541831+0000","last_peered":"2024-02-23T00:43:15.541831+0000","last_clean":"2024-02-23T00:43:15.541831+0000","last_became_active":"2024-02-23T00:39:14.775446+0000","last_became_peered":"2024-02-23T00:39:14.775446+0000","last_unstale":"2024-02-23T00:43:15.541831+0000","last_undegraded":"2024-02-23T00:43:15.541831+0000","last_fullsized":"2024-02-23T00:43:15.541831+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:49.196953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:49.196953+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":73,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.590478+0000","last_change":"2024-02-23T00:41:02.714085+0000","last_active":"2024-02-23T00:43:15.590478+0000","last_peered":"2024-02-23T00:43:15.590478+0000","last_clean":"2024-02-23T00:43:15.590478+0000","last_became_active":"2024-02-23T00:39:20.797242+0000","last_became_peered":"2024-02-23T00:39:20.797242+0000","last_unstale":"2024-02-23T00:43:15.590478+0000","last_undegraded":"2024-02-23T00:43:15.590478+0000","last_fullsized":"2024-02-23T00:43:15.590478+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:02.714020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:02.714020+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"110'5","reported_seq":31,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.555554+0000","last_change":"2024-02-23T00:42:58.743470+0000","last_active":"2024-02-23T00:43:15.555554+0000","last_peered":"2024-02-23T00:43:15.555554+0000","last_clean":"2024-02-23T00:43:15.555554+0000","last_became_active":"2024-02-23T00:42:58.742998+0000","last_became_peered":"2024-02-23T00:42:58.742998+0000","last_unstale":"2024-02-23T00:43:15.555554+0000","last_undegraded":"2024-02-23T00:43:15.555554+0000","last_fullsized":"2024-02-23T00:43:15.555554+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.542102+0000","last_change":"2024-02-23T00:42:58.691842+0000","last_active":"2024-02-23T00:43:15.542102+0000","last_peered":"2024-02-23T00:43:15.542102+0000","last_clean":"2024-02-23T00:43:15.542102+0000","last_became_active":"2024-02-23T00:42:58.690858+0000","last_became_peered":"2024-02-23T00:42:58.690858+0000","last_unstale":"2024-02-23T00:43:15.542102+0000","last_undegraded":"2024-02-23T00:43:15.542102+0000","last_fullsized":"2024-02-23T00:43:15.542102+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.16","version":"110'1","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.558044+0000","last_change":"2024-02-23T00:42:58.772995+0000","last_active":"2024-02-23T00:43:15.558044+0000","last_peered":"2024-02-23T00:43:15.558044+0000","last_clean":"2024-02-23T00:43:15.558044+0000","last_became_active":"2024-02-23T00:42:58.772347+0000","last_became_peered":"2024-02-23T00:42:58.772347+0000","last_unstale":"2024-02-23T00:43:15.558044+0000","last_undegraded":"2024-02-23T00:43:15.558044+0000","last_fullsized":"2024-02-23T00:43:15.558044+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":79,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.553559+0000","last_change":"2024-02-23T00:42:42.029769+0000","last_active":"2024-02-23T00:43:15.553559+0000","last_peered":"2024-02-23T00:43:15.553559+0000","last_clean":"2024-02-23T00:43:15.553559+0000","last_became_active":"2024-02-23T00:39:18.793801+0000","last_became_peered":"2024-02-23T00:39:18.793801+0000","last_unstale":"2024-02-23T00:43:15.553559+0000","last_undegraded":"2024-02-23T00:43:15.553559+0000","last_fullsized":"2024-02-23T00:43:15.553559+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:42.029652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:42.029652+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"67'14","reported_seq":104,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.541539+0000","last_change":"2024-02-23T00:41:48.041191+0000","last_active":"2024-02-23T00:43:15.541539+0000","last_peered":"2024-02-23T00:43:15.541539+0000","last_clean":"2024-02-23T00:43:15.541539+0000","last_became_active":"2024-02-23T00:39:16.858880+0000","last_became_peered":"2024-02-23T00:39:16.858880+0000","last_unstale":"2024-02-23T00:43:15.541539+0000","last_undegraded":"2024-02-23T00:43:15.541539+0000","last_fullsized":"2024-02-23T00:43:15.541539+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'14","last_scrub_stamp":"2024-02-23T00:41:48.041106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:48.041106+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.546201+0000","last_change":"2024-02-23T00:42:49.977357+0000","last_active":"2024-02-23T00:43:15.546201+0000","last_peered":"2024-02-23T00:43:15.546201+0000","last_clean":"2024-02-23T00:43:15.546201+0000","last_became_active":"2024-02-23T00:39:08.248249+0000","last_became_peered":"2024-02-23T00:39:08.248249+0000","last_unstale":"2024-02-23T00:43:15.546201+0000","last_undegraded":"2024-02-23T00:43:15.546201+0000","last_fullsized":"2024-02-23T00:43:15.546201+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:49.977248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:49.977248+0000","last_clean_scrub_stamp":"2024-02-23T00:42:49.977248+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":85,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.549662+0000","last_change":"2024-02-23T00:42:46.945425+0000","last_active":"2024-02-23T00:43:15.549662+0000","last_peered":"2024-02-23T00:43:15.549662+0000","last_clean":"2024-02-23T00:43:15.549662+0000","last_became_active":"2024-02-23T00:39:14.788888+0000","last_became_peered":"2024-02-23T00:39:14.788888+0000","last_unstale":"2024-02-23T00:43:15.549662+0000","last_undegraded":"2024-02-23T00:43:15.549662+0000","last_fullsized":"2024-02-23T00:43:15.549662+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:46.945358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:46.945358+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":85,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.547762+0000","last_change":"2024-02-23T00:42:34.984785+0000","last_active":"2024-02-23T00:43:15.547762+0000","last_peered":"2024-02-23T00:43:15.547762+0000","last_clean":"2024-02-23T00:43:15.547762+0000","last_became_active":"2024-02-23T00:39:14.783489+0000","last_became_peered":"2024-02-23T00:39:14.783489+0000","last_unstale":"2024-02-23T00:43:15.547762+0000","last_undegraded":"2024-02-23T00:43:15.547762+0000","last_fullsized":"2024-02-23T00:43:15.547762+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:34.984682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:34.984682+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":79,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.543037+0000","last_change":"2024-02-23T00:41:55.726397+0000","last_active":"2024-02-23T00:43:15.543037+0000","last_peered":"2024-02-23T00:43:15.543037+0000","last_clean":"2024-02-23T00:43:15.543037+0000","last_became_active":"2024-02-23T00:39:20.797226+0000","last_became_peered":"2024-02-23T00:39:20.797226+0000","last_unstale":"2024-02-23T00:43:15.543037+0000","last_undegraded":"2024-02-23T00:43:15.543037+0000","last_fullsized":"2024-02-23T00:43:15.543037+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:55.726321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:55.726321+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.576091+0000","last_change":"2024-02-23T00:42:58.721944+0000","last_active":"2024-02-23T00:43:15.576091+0000","last_peered":"2024-02-23T00:43:15.576091+0000","last_clean":"2024-02-23T00:43:15.576091+0000","last_became_active":"2024-02-23T00:42:58.721439+0000","last_became_peered":"2024-02-23T00:42:58.721439+0000","last_unstale":"2024-02-23T00:43:15.576091+0000","last_undegraded":"2024-02-23T00:43:15.576091+0000","last_fullsized":"2024-02-23T00:43:15.576091+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.551134+0000","last_change":"2024-02-23T00:42:58.701934+0000","last_active":"2024-02-23T00:43:15.551134+0000","last_peered":"2024-02-23T00:43:15.551134+0000","last_clean":"2024-02-23T00:43:15.551134+0000","last_became_active":"2024-02-23T00:42:58.701502+0000","last_became_peered":"2024-02-23T00:42:58.701502+0000","last_unstale":"2024-02-23T00:43:15.551134+0000","last_undegraded":"2024-02-23T00:43:15.551134+0000","last_fullsized":"2024-02-23T00:43:15.551134+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.15","version":"110'2","reported_seq":28,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.541624+0000","last_change":"2024-02-23T00:42:58.672263+0000","last_active":"2024-02-23T00:43:15.541624+0000","last_peered":"2024-02-23T00:43:15.541624+0000","last_clean":"2024-02-23T00:43:15.541624+0000","last_became_active":"2024-02-23T00:42:58.670377+0000","last_became_peered":"2024-02-23T00:42:58.670377+0000","last_unstale":"2024-02-23T00:43:15.541624+0000","last_undegraded":"2024-02-23T00:43:15.541624+0000","last_fullsized":"2024-02-23T00:43:15.541624+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":80,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.576932+0000","last_change":"2024-02-23T00:41:52.264308+0000","last_active":"2024-02-23T00:43:15.576932+0000","last_peered":"2024-02-23T00:43:15.576932+0000","last_clean":"2024-02-23T00:43:15.576932+0000","last_became_active":"2024-02-23T00:39:18.789330+0000","last_became_peered":"2024-02-23T00:39:18.789330+0000","last_unstale":"2024-02-23T00:43:15.576932+0000","last_undegraded":"2024-02-23T00:43:15.576932+0000","last_fullsized":"2024-02-23T00:43:15.576932+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:52.264240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.264240+0000","last_clean_scrub_stamp":"2024-02-23T00:41:52.264240+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"68'19","reported_seq":124,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.586626+0000","last_change":"2024-02-23T00:42:04.625118+0000","last_active":"2024-02-23T00:43:15.586626+0000","last_peered":"2024-02-23T00:43:15.586626+0000","last_clean":"2024-02-23T00:43:15.586626+0000","last_became_active":"2024-02-23T00:39:16.787484+0000","last_became_peered":"2024-02-23T00:39:16.787484+0000","last_unstale":"2024-02-23T00:43:15.586626+0000","last_undegraded":"2024-02-23T00:43:15.586626+0000","last_fullsized":"2024-02-23T00:43:15.586626+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'19","last_scrub_stamp":"2024-02-23T00:42:04.625006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:04.625006+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":84,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.559677+0000","last_change":"2024-02-23T00:42:05.329858+0000","last_active":"2024-02-23T00:43:15.559677+0000","last_peered":"2024-02-23T00:43:15.559677+0000","last_clean":"2024-02-23T00:43:15.559677+0000","last_became_active":"2024-02-23T00:39:14.772411+0000","last_became_peered":"2024-02-23T00:39:14.772411+0000","last_unstale":"2024-02-23T00:43:15.559677+0000","last_undegraded":"2024-02-23T00:43:15.559677+0000","last_fullsized":"2024-02-23T00:43:15.559677+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:05.329757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:05.329757+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"68'5","reported_seq":316,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:16.322783+0000","last_change":"2024-02-23T00:41:00.648420+0000","last_active":"2024-02-23T00:43:16.322783+0000","last_peered":"2024-02-23T00:43:16.322783+0000","last_clean":"2024-02-23T00:43:16.322783+0000","last_became_active":"2024-02-23T00:39:14.779109+0000","last_became_peered":"2024-02-23T00:39:14.779109+0000","last_unstale":"2024-02-23T00:43:16.322783+0000","last_undegraded":"2024-02-23T00:43:16.322783+0000","last_fullsized":"2024-02-23T00:43:16.322783+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'5","last_scrub_stamp":"2024-02-23T00:41:00.648353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:00.648353+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":234,"num_read_kb":229,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":77,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.555927+0000","last_change":"2024-02-23T00:41:44.001469+0000","last_active":"2024-02-23T00:43:15.555927+0000","last_peered":"2024-02-23T00:43:15.555927+0000","last_clean":"2024-02-23T00:43:15.555927+0000","last_became_active":"2024-02-23T00:39:21.160457+0000","last_became_peered":"2024-02-23T00:39:21.160457+0000","last_unstale":"2024-02-23T00:43:15.555927+0000","last_undegraded":"2024-02-23T00:43:15.555927+0000","last_fullsized":"2024-02-23T00:43:15.555927+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:44.001408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:44.001408+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.539884+0000","last_change":"2024-02-23T00:42:58.831449+0000","last_active":"2024-02-23T00:43:15.539884+0000","last_peered":"2024-02-23T00:43:15.539884+0000","last_clean":"2024-02-23T00:43:15.539884+0000","last_became_active":"2024-02-23T00:42:58.830815+0000","last_became_peered":"2024-02-23T00:42:58.830815+0000","last_unstale":"2024-02-23T00:43:15.539884+0000","last_undegraded":"2024-02-23T00:43:15.539884+0000","last_fullsized":"2024-02-23T00:43:15.539884+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.552344+0000","last_change":"2024-02-23T00:42:57.653669+0000","last_active":"2024-02-23T00:43:15.552344+0000","last_peered":"2024-02-23T00:43:15.552344+0000","last_clean":"2024-02-23T00:43:15.552344+0000","last_became_active":"2024-02-23T00:42:57.653355+0000","last_became_peered":"2024-02-23T00:42:57.653355+0000","last_unstale":"2024-02-23T00:43:15.552344+0000","last_undegraded":"2024-02-23T00:43:15.552344+0000","last_fullsized":"2024-02-23T00:43:15.552344+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.17","version":"110'8","reported_seq":34,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.541984+0000","last_change":"2024-02-23T00:42:58.729148+0000","last_active":"2024-02-23T00:43:15.541984+0000","last_peered":"2024-02-23T00:43:15.541984+0000","last_clean":"2024-02-23T00:43:15.541984+0000","last_became_active":"2024-02-23T00:42:58.726659+0000","last_became_peered":"2024-02-23T00:42:58.726659+0000","last_unstale":"2024-02-23T00:43:15.541984+0000","last_undegraded":"2024-02-23T00:43:15.541984+0000","last_fullsized":"2024-02-23T00:43:15.541984+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":81,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.542186+0000","last_change":"2024-02-23T00:42:47.349912+0000","last_active":"2024-02-23T00:43:15.542186+0000","last_peered":"2024-02-23T00:43:15.542186+0000","last_clean":"2024-02-23T00:43:15.542186+0000","last_became_active":"2024-02-23T00:39:18.802134+0000","last_became_peered":"2024-02-23T00:39:18.802134+0000","last_unstale":"2024-02-23T00:43:15.542186+0000","last_undegraded":"2024-02-23T00:43:15.542186+0000","last_fullsized":"2024-02-23T00:43:15.542186+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:47.349847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:47.349847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"86'35","reported_seq":115,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.539976+0000","last_change":"2024-02-23T00:42:47.988747+0000","last_active":"2024-02-23T00:43:15.539976+0000","last_peered":"2024-02-23T00:43:15.539976+0000","last_clean":"2024-02-23T00:43:15.539976+0000","last_became_active":"2024-02-23T00:40:42.933322+0000","last_became_peered":"2024-02-23T00:40:42.933322+0000","last_unstale":"2024-02-23T00:43:15.539976+0000","last_undegraded":"2024-02-23T00:43:15.539976+0000","last_fullsized":"2024-02-23T00:43:15.539976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'35","last_scrub_stamp":"2024-02-23T00:42:47.988676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:47.988676+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":85,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.538490+0000","last_change":"2024-02-23T00:42:04.708358+0000","last_active":"2024-02-23T00:43:15.538490+0000","last_peered":"2024-02-23T00:43:15.538490+0000","last_clean":"2024-02-23T00:43:15.538490+0000","last_became_active":"2024-02-23T00:39:14.784015+0000","last_became_peered":"2024-02-23T00:39:14.784015+0000","last_unstale":"2024-02-23T00:43:15.538490+0000","last_undegraded":"2024-02-23T00:43:15.538490+0000","last_fullsized":"2024-02-23T00:43:15.538490+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:04.708280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:04.708280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":74,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.547267+0000","last_change":"2024-02-23T00:41:18.910682+0000","last_active":"2024-02-23T00:43:15.547267+0000","last_peered":"2024-02-23T00:43:15.547267+0000","last_clean":"2024-02-23T00:43:15.547267+0000","last_became_active":"2024-02-23T00:39:20.790053+0000","last_became_peered":"2024-02-23T00:39:20.790053+0000","last_unstale":"2024-02-23T00:43:15.547267+0000","last_undegraded":"2024-02-23T00:43:15.547267+0000","last_fullsized":"2024-02-23T00:43:15.547267+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:18.910605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:18.910605+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.550766+0000","last_change":"2024-02-23T00:42:58.755124+0000","last_active":"2024-02-23T00:43:15.550766+0000","last_peered":"2024-02-23T00:43:15.550766+0000","last_clean":"2024-02-23T00:43:15.550766+0000","last_became_active":"2024-02-23T00:42:58.754813+0000","last_became_peered":"2024-02-23T00:42:58.754813+0000","last_unstale":"2024-02-23T00:43:15.550766+0000","last_undegraded":"2024-02-23T00:43:15.550766+0000","last_fullsized":"2024-02-23T00:43:15.550766+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.558111+0000","last_change":"2024-02-23T00:42:58.767404+0000","last_active":"2024-02-23T00:43:15.558111+0000","last_peered":"2024-02-23T00:43:15.558111+0000","last_clean":"2024-02-23T00:43:15.558111+0000","last_became_active":"2024-02-23T00:42:58.766579+0000","last_became_peered":"2024-02-23T00:42:58.766579+0000","last_unstale":"2024-02-23T00:43:15.558111+0000","last_undegraded":"2024-02-23T00:43:15.558111+0000","last_fullsized":"2024-02-23T00:43:15.558111+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":25,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.538662+0000","last_change":"2024-02-23T00:42:58.874626+0000","last_active":"2024-02-23T00:43:15.538662+0000","last_peered":"2024-02-23T00:43:15.538662+0000","last_clean":"2024-02-23T00:43:15.538662+0000","last_became_active":"2024-02-23T00:42:58.872796+0000","last_became_peered":"2024-02-23T00:42:58.872796+0000","last_unstale":"2024-02-23T00:43:15.538662+0000","last_undegraded":"2024-02-23T00:43:15.538662+0000","last_fullsized":"2024-02-23T00:43:15.538662+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"91'9","reported_seq":138,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.542511+0000","last_change":"2024-02-23T00:42:49.410329+0000","last_active":"2024-02-23T00:43:15.542511+0000","last_peered":"2024-02-23T00:43:15.542511+0000","last_clean":"2024-02-23T00:43:15.542511+0000","last_became_active":"2024-02-23T00:39:18.801297+0000","last_became_peered":"2024-02-23T00:39:18.801297+0000","last_unstale":"2024-02-23T00:43:15.542511+0000","last_undegraded":"2024-02-23T00:43:15.542511+0000","last_fullsized":"2024-02-23T00:43:15.542511+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'9","last_scrub_stamp":"2024-02-23T00:42:49.410259+0000","last_deep_scrub":"91'9","last_deep_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_clean_scrub_stamp":"2024-02-23T00:42:49.410259+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"68'16","reported_seq":112,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.554150+0000","last_change":"2024-02-23T00:42:34.633620+0000","last_active":"2024-02-23T00:43:15.554150+0000","last_peered":"2024-02-23T00:43:15.554150+0000","last_clean":"2024-02-23T00:43:15.554150+0000","last_became_active":"2024-02-23T00:39:16.787468+0000","last_became_peered":"2024-02-23T00:39:16.787468+0000","last_unstale":"2024-02-23T00:43:15.554150+0000","last_undegraded":"2024-02-23T00:43:15.554150+0000","last_fullsized":"2024-02-23T00:43:15.554150+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:42:34.633549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:34.633549+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":84,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.589987+0000","last_change":"2024-02-23T00:42:46.606249+0000","last_active":"2024-02-23T00:43:15.589987+0000","last_peered":"2024-02-23T00:43:15.589987+0000","last_clean":"2024-02-23T00:43:15.589987+0000","last_became_active":"2024-02-23T00:39:14.779952+0000","last_became_peered":"2024-02-23T00:39:14.779952+0000","last_unstale":"2024-02-23T00:43:15.589987+0000","last_undegraded":"2024-02-23T00:43:15.589987+0000","last_fullsized":"2024-02-23T00:43:15.589987+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:46.606185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:46.606185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"107'2","reported_seq":81,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.542951+0000","last_change":"2024-02-23T00:42:48.362807+0000","last_active":"2024-02-23T00:43:15.542951+0000","last_peered":"2024-02-23T00:43:15.542951+0000","last_clean":"2024-02-23T00:43:15.542951+0000","last_became_active":"2024-02-23T00:39:20.800858+0000","last_became_peered":"2024-02-23T00:39:20.800858+0000","last_unstale":"2024-02-23T00:43:15.542951+0000","last_undegraded":"2024-02-23T00:43:15.542951+0000","last_fullsized":"2024-02-23T00:43:15.542951+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'2","last_scrub_stamp":"2024-02-23T00:42:48.362739+0000","last_deep_scrub":"107'2","last_deep_scrub_stamp":"2024-02-23T00:42:48.362739+0000","last_clean_scrub_stamp":"2024-02-23T00:42:48.362739+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.547629+0000","last_change":"2024-02-23T00:42:58.727288+0000","last_active":"2024-02-23T00:43:15.547629+0000","last_peered":"2024-02-23T00:43:15.547629+0000","last_clean":"2024-02-23T00:43:15.547629+0000","last_became_active":"2024-02-23T00:42:58.726861+0000","last_became_peered":"2024-02-23T00:42:58.726861+0000","last_unstale":"2024-02-23T00:43:15.547629+0000","last_undegraded":"2024-02-23T00:43:15.547629+0000","last_fullsized":"2024-02-23T00:43:15.547629+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.539563+0000","last_change":"2024-02-23T00:42:58.664797+0000","last_active":"2024-02-23T00:43:15.539563+0000","last_peered":"2024-02-23T00:43:15.539563+0000","last_clean":"2024-02-23T00:43:15.539563+0000","last_became_active":"2024-02-23T00:42:58.664282+0000","last_became_peered":"2024-02-23T00:42:58.664282+0000","last_unstale":"2024-02-23T00:43:15.539563+0000","last_undegraded":"2024-02-23T00:43:15.539563+0000","last_fullsized":"2024-02-23T00:43:15.539563+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.12","version":"112'1","reported_seq":30,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.546058+0000","last_change":"2024-02-23T00:42:58.724372+0000","last_active":"2024-02-23T00:43:15.546058+0000","last_peered":"2024-02-23T00:43:15.546058+0000","last_clean":"2024-02-23T00:43:15.546058+0000","last_became_active":"2024-02-23T00:42:58.723546+0000","last_became_peered":"2024-02-23T00:42:58.723546+0000","last_unstale":"2024-02-23T00:43:15.546058+0000","last_undegraded":"2024-02-23T00:43:15.546058+0000","last_fullsized":"2024-02-23T00:43:15.546058+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25165836,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":24577,"num_write":1,"num_write_kb":24577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":80,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.556780+0000","last_change":"2024-02-23T00:41:57.376669+0000","last_active":"2024-02-23T00:43:15.556780+0000","last_peered":"2024-02-23T00:43:15.556780+0000","last_clean":"2024-02-23T00:43:15.556780+0000","last_became_active":"2024-02-23T00:39:18.796323+0000","last_became_peered":"2024-02-23T00:39:18.796323+0000","last_unstale":"2024-02-23T00:43:15.556780+0000","last_undegraded":"2024-02-23T00:43:15.556780+0000","last_fullsized":"2024-02-23T00:43:15.556780+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:57.376576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_clean_scrub_stamp":"2024-02-23T00:41:57.376576+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"68'25","reported_seq":131,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.548147+0000","last_change":"2024-02-23T00:41:08.004573+0000","last_active":"2024-02-23T00:43:15.548147+0000","last_peered":"2024-02-23T00:43:15.548147+0000","last_clean":"2024-02-23T00:43:15.548147+0000","last_became_active":"2024-02-23T00:39:16.783926+0000","last_became_peered":"2024-02-23T00:39:16.783926+0000","last_unstale":"2024-02-23T00:43:15.548147+0000","last_undegraded":"2024-02-23T00:43:15.548147+0000","last_fullsized":"2024-02-23T00:43:15.548147+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'25","last_scrub_stamp":"2024-02-23T00:41:08.004489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:08.004489+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"95'4","reported_seq":99,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.562521+0000","last_change":"2024-02-23T00:41:43.874460+0000","last_active":"2024-02-23T00:43:15.562521+0000","last_peered":"2024-02-23T00:43:15.562521+0000","last_clean":"2024-02-23T00:43:15.562521+0000","last_became_active":"2024-02-23T00:39:14.779205+0000","last_became_peered":"2024-02-23T00:39:14.779205+0000","last_unstale":"2024-02-23T00:43:15.562521+0000","last_undegraded":"2024-02-23T00:43:15.562521+0000","last_fullsized":"2024-02-23T00:43:15.562521+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'4","last_scrub_stamp":"2024-02-23T00:41:43.874397+0000","last_deep_scrub":"95'4","last_deep_scrub_stamp":"2024-02-23T00:41:43.874397+0000","last_clean_scrub_stamp":"2024-02-23T00:41:43.874397+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":79,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.550322+0000","last_change":"2024-02-23T00:41:48.031707+0000","last_active":"2024-02-23T00:43:15.550322+0000","last_peered":"2024-02-23T00:43:15.550322+0000","last_clean":"2024-02-23T00:43:15.550322+0000","last_became_active":"2024-02-23T00:39:20.793710+0000","last_became_peered":"2024-02-23T00:39:20.793710+0000","last_unstale":"2024-02-23T00:43:15.550322+0000","last_undegraded":"2024-02-23T00:43:15.550322+0000","last_fullsized":"2024-02-23T00:43:15.550322+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:48.031633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:48.031633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.558196+0000","last_change":"2024-02-23T00:42:58.810468+0000","last_active":"2024-02-23T00:43:15.558196+0000","last_peered":"2024-02-23T00:43:15.558196+0000","last_clean":"2024-02-23T00:43:15.558196+0000","last_became_active":"2024-02-23T00:42:58.809926+0000","last_became_peered":"2024-02-23T00:42:58.809926+0000","last_unstale":"2024-02-23T00:43:15.558196+0000","last_undegraded":"2024-02-23T00:43:15.558196+0000","last_fullsized":"2024-02-23T00:43:15.558196+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.555795+0000","last_change":"2024-02-23T00:42:58.755160+0000","last_active":"2024-02-23T00:43:15.555795+0000","last_peered":"2024-02-23T00:43:15.555795+0000","last_clean":"2024-02-23T00:43:15.555795+0000","last_became_active":"2024-02-23T00:42:58.744983+0000","last_became_peered":"2024-02-23T00:42:58.744983+0000","last_unstale":"2024-02-23T00:43:15.555795+0000","last_undegraded":"2024-02-23T00:43:15.555795+0000","last_fullsized":"2024-02-23T00:43:15.555795+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":25,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.560362+0000","last_change":"2024-02-23T00:42:58.741573+0000","last_active":"2024-02-23T00:43:15.560362+0000","last_peered":"2024-02-23T00:43:15.560362+0000","last_clean":"2024-02-23T00:43:15.560362+0000","last_became_active":"2024-02-23T00:42:58.741284+0000","last_became_peered":"2024-02-23T00:42:58.741284+0000","last_unstale":"2024-02-23T00:43:15.560362+0000","last_undegraded":"2024-02-23T00:43:15.560362+0000","last_fullsized":"2024-02-23T00:43:15.560362+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"68'21","reported_seq":124,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.544925+0000","last_change":"2024-02-23T00:41:48.151380+0000","last_active":"2024-02-23T00:43:15.544925+0000","last_peered":"2024-02-23T00:43:15.544925+0000","last_clean":"2024-02-23T00:43:15.544925+0000","last_became_active":"2024-02-23T00:39:16.858760+0000","last_became_peered":"2024-02-23T00:39:16.858760+0000","last_unstale":"2024-02-23T00:43:15.544925+0000","last_undegraded":"2024-02-23T00:43:15.544925+0000","last_fullsized":"2024-02-23T00:43:15.544925+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'21","last_scrub_stamp":"2024-02-23T00:41:48.151312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:48.151312+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":89,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.556703+0000","last_change":"2024-02-23T00:42:52.317469+0000","last_active":"2024-02-23T00:43:15.556703+0000","last_peered":"2024-02-23T00:43:15.556703+0000","last_clean":"2024-02-23T00:43:15.556703+0000","last_became_active":"2024-02-23T00:39:14.774244+0000","last_became_peered":"2024-02-23T00:39:14.774244+0000","last_unstale":"2024-02-23T00:43:15.556703+0000","last_undegraded":"2024-02-23T00:43:15.556703+0000","last_fullsized":"2024-02-23T00:43:15.556703+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:52.317403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:52.317403+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":81,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.541738+0000","last_change":"2024-02-23T00:41:46.998945+0000","last_active":"2024-02-23T00:43:15.541738+0000","last_peered":"2024-02-23T00:43:15.541738+0000","last_clean":"2024-02-23T00:43:15.541738+0000","last_became_active":"2024-02-23T00:39:18.798624+0000","last_became_peered":"2024-02-23T00:39:18.798624+0000","last_unstale":"2024-02-23T00:43:15.541738+0000","last_undegraded":"2024-02-23T00:43:15.541738+0000","last_fullsized":"2024-02-23T00:43:15.541738+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:46.998875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:46.998875+0000","last_clean_scrub_stamp":"2024-02-23T00:41:46.998875+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":78,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.560066+0000","last_change":"2024-02-23T00:42:35.226401+0000","last_active":"2024-02-23T00:43:15.560066+0000","last_peered":"2024-02-23T00:43:15.560066+0000","last_clean":"2024-02-23T00:43:15.560066+0000","last_became_active":"2024-02-23T00:39:20.796041+0000","last_became_peered":"2024-02-23T00:39:20.796041+0000","last_unstale":"2024-02-23T00:43:15.560066+0000","last_undegraded":"2024-02-23T00:43:15.560066+0000","last_fullsized":"2024-02-23T00:43:15.560066+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:35.226337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:35.226337+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1d","version":"110'7","reported_seq":33,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.549449+0000","last_change":"2024-02-23T00:42:58.755048+0000","last_active":"2024-02-23T00:43:15.549449+0000","last_peered":"2024-02-23T00:43:15.549449+0000","last_clean":"2024-02-23T00:43:15.549449+0000","last_became_active":"2024-02-23T00:42:58.754763+0000","last_became_peered":"2024-02-23T00:42:58.754763+0000","last_unstale":"2024-02-23T00:43:15.549449+0000","last_undegraded":"2024-02-23T00:43:15.549449+0000","last_fullsized":"2024-02-23T00:43:15.549449+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.543292+0000","last_change":"2024-02-23T00:42:58.829459+0000","last_active":"2024-02-23T00:43:15.543292+0000","last_peered":"2024-02-23T00:43:15.543292+0000","last_clean":"2024-02-23T00:43:15.543292+0000","last_became_active":"2024-02-23T00:42:58.825015+0000","last_became_peered":"2024-02-23T00:42:58.825015+0000","last_unstale":"2024-02-23T00:43:15.543292+0000","last_undegraded":"2024-02-23T00:43:15.543292+0000","last_fullsized":"2024-02-23T00:43:15.543292+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.6","version":"112'7","reported_seq":37,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.580417+0000","last_change":"2024-02-23T00:43:06.922950+0000","last_active":"2024-02-23T00:43:15.580417+0000","last_peered":"2024-02-23T00:43:15.580417+0000","last_clean":"2024-02-23T00:43:15.580417+0000","last_became_active":"2024-02-23T00:42:58.769223+0000","last_became_peered":"2024-02-23T00:42:58.769223+0000","last_unstale":"2024-02-23T00:43:15.580417+0000","last_undegraded":"2024-02-23T00:43:15.580417+0000","last_fullsized":"2024-02-23T00:43:15.580417+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"112'7","last_scrub_stamp":"2024-02-23T00:43:06.922785+0000","last_deep_scrub":"112'7","last_deep_scrub_stamp":"2024-02-23T00:43:06.922785+0000","last_clean_scrub_stamp":"2024-02-23T00:43:06.922785+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25165828,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":27325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2800000,"num_omap_keys":1000,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"107'16","reported_seq":111,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.541677+0000","last_change":"2024-02-23T00:42:52.964576+0000","last_active":"2024-02-23T00:43:15.541677+0000","last_peered":"2024-02-23T00:43:15.541677+0000","last_clean":"2024-02-23T00:43:15.541677+0000","last_became_active":"2024-02-23T00:39:16.858449+0000","last_became_peered":"2024-02-23T00:39:16.858449+0000","last_unstale":"2024-02-23T00:43:15.541677+0000","last_undegraded":"2024-02-23T00:43:15.541677+0000","last_fullsized":"2024-02-23T00:43:15.541677+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'16","last_scrub_stamp":"2024-02-23T00:42:52.964510+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-02-23T00:42:52.964510+0000","last_clean_scrub_stamp":"2024-02-23T00:42:52.964510+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":84,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.544135+0000","last_change":"2024-02-23T00:42:41.429191+0000","last_active":"2024-02-23T00:43:15.544135+0000","last_peered":"2024-02-23T00:43:15.544135+0000","last_clean":"2024-02-23T00:43:15.544135+0000","last_became_active":"2024-02-23T00:39:14.783164+0000","last_became_peered":"2024-02-23T00:39:14.783164+0000","last_unstale":"2024-02-23T00:43:15.544135+0000","last_undegraded":"2024-02-23T00:43:15.544135+0000","last_fullsized":"2024-02-23T00:43:15.544135+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:41.429079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:41.429079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":81,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.556460+0000","last_change":"2024-02-23T00:41:35.655313+0000","last_active":"2024-02-23T00:43:15.556460+0000","last_peered":"2024-02-23T00:43:15.556460+0000","last_clean":"2024-02-23T00:43:15.556460+0000","last_became_active":"2024-02-23T00:39:18.800297+0000","last_became_peered":"2024-02-23T00:39:18.800297+0000","last_unstale":"2024-02-23T00:43:15.556460+0000","last_undegraded":"2024-02-23T00:43:15.556460+0000","last_fullsized":"2024-02-23T00:43:15.556460+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:35.655209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:35.655209+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":77,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.555667+0000","last_change":"2024-02-23T00:41:45.928004+0000","last_active":"2024-02-23T00:43:15.555667+0000","last_peered":"2024-02-23T00:43:15.555667+0000","last_clean":"2024-02-23T00:43:15.555667+0000","last_became_active":"2024-02-23T00:39:21.162036+0000","last_became_peered":"2024-02-23T00:39:21.162036+0000","last_unstale":"2024-02-23T00:43:15.555667+0000","last_undegraded":"2024-02-23T00:43:15.555667+0000","last_fullsized":"2024-02-23T00:43:15.555667+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:45.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:45.927925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.590377+0000","last_change":"2024-02-23T00:42:57.649841+0000","last_active":"2024-02-23T00:43:15.590377+0000","last_peered":"2024-02-23T00:43:15.590377+0000","last_clean":"2024-02-23T00:43:15.590377+0000","last_became_active":"2024-02-23T00:42:57.648267+0000","last_became_peered":"2024-02-23T00:42:57.648267+0000","last_unstale":"2024-02-23T00:43:15.590377+0000","last_undegraded":"2024-02-23T00:43:15.590377+0000","last_fullsized":"2024-02-23T00:43:15.590377+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1","version":"110'7","reported_seq":39,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.550765+0000","last_change":"2024-02-23T00:42:58.782380+0000","last_active":"2024-02-23T00:43:15.550765+0000","last_peered":"2024-02-23T00:43:15.550765+0000","last_clean":"2024-02-23T00:43:15.550765+0000","last_became_active":"2024-02-23T00:42:58.777251+0000","last_became_peered":"2024-02-23T00:42:58.777251+0000","last_unstale":"2024-02-23T00:43:15.550765+0000","last_undegraded":"2024-02-23T00:43:15.550765+0000","last_fullsized":"2024-02-23T00:43:15.550765+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":30,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.556168+0000","last_change":"2024-02-23T00:43:08.979886+0000","last_active":"2024-02-23T00:43:15.556168+0000","last_peered":"2024-02-23T00:43:15.556168+0000","last_clean":"2024-02-23T00:43:15.556168+0000","last_became_active":"2024-02-23T00:42:58.713766+0000","last_became_peered":"2024-02-23T00:42:58.713766+0000","last_unstale":"2024-02-23T00:43:15.556168+0000","last_undegraded":"2024-02-23T00:43:15.556168+0000","last_fullsized":"2024-02-23T00:43:15.556168+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:08.979794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:08.979794+0000","last_clean_scrub_stamp":"2024-02-23T00:43:08.979794+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"68'22","reported_seq":52,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.559830+0000","last_change":"2024-02-23T00:42:44.344558+0000","last_active":"2024-02-23T00:43:15.559830+0000","last_peered":"2024-02-23T00:43:15.559830+0000","last_clean":"2024-02-23T00:43:15.559830+0000","last_became_active":"2024-02-23T00:42:43.521189+0000","last_became_peered":"2024-02-23T00:42:43.521189+0000","last_unstale":"2024-02-23T00:43:15.559830+0000","last_undegraded":"2024-02-23T00:43:15.559830+0000","last_fullsized":"2024-02-23T00:43:15.559830+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'22","last_scrub_stamp":"2024-02-23T00:42:44.344443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:44.344443+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":37,"num_read_kb":30,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,7],"acting":[2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":85,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.537892+0000","last_change":"2024-02-23T00:42:02.694103+0000","last_active":"2024-02-23T00:43:15.537892+0000","last_peered":"2024-02-23T00:43:15.537892+0000","last_clean":"2024-02-23T00:43:15.537892+0000","last_became_active":"2024-02-23T00:39:14.785590+0000","last_became_peered":"2024-02-23T00:39:14.785590+0000","last_unstale":"2024-02-23T00:43:15.537892+0000","last_undegraded":"2024-02-23T00:43:15.537892+0000","last_fullsized":"2024-02-23T00:43:15.537892+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:02.694022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_clean_scrub_stamp":"2024-02-23T00:42:02.694022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"67'8","reported_seq":137,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.559672+0000","last_change":"2024-02-23T00:42:40.872975+0000","last_active":"2024-02-23T00:43:15.559672+0000","last_peered":"2024-02-23T00:43:15.559672+0000","last_clean":"2024-02-23T00:43:15.559672+0000","last_became_active":"2024-02-23T00:39:18.803566+0000","last_became_peered":"2024-02-23T00:39:18.803566+0000","last_unstale":"2024-02-23T00:43:15.559672+0000","last_undegraded":"2024-02-23T00:43:15.559672+0000","last_fullsized":"2024-02-23T00:43:15.559672+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:42:40.872876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:40.872876+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.546300+0000","last_change":"2024-02-23T00:43:00.930610+0000","last_active":"2024-02-23T00:43:15.546300+0000","last_peered":"2024-02-23T00:43:15.546300+0000","last_clean":"2024-02-23T00:43:15.546300+0000","last_became_active":"2024-02-23T00:43:00.930360+0000","last_became_peered":"2024-02-23T00:43:00.930360+0000","last_unstale":"2024-02-23T00:43:15.546300+0000","last_undegraded":"2024-02-23T00:43:15.546300+0000","last_fullsized":"2024-02-23T00:43:15.546300+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:56.456815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:56.456815+0000","last_clean_scrub_stamp":"2024-02-23T00:42:56.456815+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1f","version":"110'5","reported_seq":31,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.541886+0000","last_change":"2024-02-23T00:42:58.770053+0000","last_active":"2024-02-23T00:43:15.541886+0000","last_peered":"2024-02-23T00:43:15.541886+0000","last_clean":"2024-02-23T00:43:15.541886+0000","last_became_active":"2024-02-23T00:42:58.758349+0000","last_became_peered":"2024-02-23T00:42:58.758349+0000","last_unstale":"2024-02-23T00:43:15.541886+0000","last_undegraded":"2024-02-23T00:43:15.541886+0000","last_fullsized":"2024-02-23T00:43:15.541886+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.549942+0000","last_change":"2024-02-23T00:42:58.766295+0000","last_active":"2024-02-23T00:43:15.549942+0000","last_peered":"2024-02-23T00:43:15.549942+0000","last_clean":"2024-02-23T00:43:15.549942+0000","last_became_active":"2024-02-23T00:42:58.757424+0000","last_became_peered":"2024-02-23T00:42:58.757424+0000","last_unstale":"2024-02-23T00:43:15.549942+0000","last_undegraded":"2024-02-23T00:43:15.549942+0000","last_fullsized":"2024-02-23T00:43:15.549942+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":30,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.590207+0000","last_change":"2024-02-23T00:43:02.835731+0000","last_active":"2024-02-23T00:43:15.590207+0000","last_peered":"2024-02-23T00:43:15.590207+0000","last_clean":"2024-02-23T00:43:15.590207+0000","last_became_active":"2024-02-23T00:42:58.736416+0000","last_became_peered":"2024-02-23T00:42:58.736416+0000","last_unstale":"2024-02-23T00:43:15.590207+0000","last_undegraded":"2024-02-23T00:43:15.590207+0000","last_fullsized":"2024-02-23T00:43:15.590207+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:02.835657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:02.835657+0000","last_clean_scrub_stamp":"2024-02-23T00:43:02.835657+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"68'15","reported_seq":114,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.546098+0000","last_change":"2024-02-23T00:42:34.829123+0000","last_active":"2024-02-23T00:43:15.546098+0000","last_peered":"2024-02-23T00:43:15.546098+0000","last_clean":"2024-02-23T00:43:15.546098+0000","last_became_active":"2024-02-23T00:39:16.858953+0000","last_became_peered":"2024-02-23T00:39:16.858953+0000","last_unstale":"2024-02-23T00:43:15.546098+0000","last_undegraded":"2024-02-23T00:43:15.546098+0000","last_fullsized":"2024-02-23T00:43:15.546098+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:42:34.828996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:34.828996+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":85,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.537453+0000","last_change":"2024-02-23T00:41:52.151201+0000","last_active":"2024-02-23T00:43:15.537453+0000","last_peered":"2024-02-23T00:43:15.537453+0000","last_clean":"2024-02-23T00:43:15.537453+0000","last_became_active":"2024-02-23T00:39:14.773772+0000","last_became_peered":"2024-02-23T00:39:14.773772+0000","last_unstale":"2024-02-23T00:43:15.537453+0000","last_undegraded":"2024-02-23T00:43:15.537453+0000","last_fullsized":"2024-02-23T00:43:15.537453+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:52.151119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:52.151119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"67'8","reported_seq":132,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.542425+0000","last_change":"2024-02-23T00:41:25.942914+0000","last_active":"2024-02-23T00:43:15.542425+0000","last_peered":"2024-02-23T00:43:15.542425+0000","last_clean":"2024-02-23T00:43:15.542425+0000","last_became_active":"2024-02-23T00:39:18.787313+0000","last_became_peered":"2024-02-23T00:39:18.787313+0000","last_unstale":"2024-02-23T00:43:15.542425+0000","last_undegraded":"2024-02-23T00:43:15.542425+0000","last_fullsized":"2024-02-23T00:43:15.542425+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:41:25.942822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:25.942822+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.542414+0000","last_change":"2024-02-23T00:43:00.935992+0000","last_active":"2024-02-23T00:43:15.542414+0000","last_peered":"2024-02-23T00:43:15.542414+0000","last_clean":"2024-02-23T00:43:15.542414+0000","last_became_active":"2024-02-23T00:43:00.935724+0000","last_became_peered":"2024-02-23T00:43:00.935724+0000","last_unstale":"2024-02-23T00:43:15.542414+0000","last_undegraded":"2024-02-23T00:43:15.542414+0000","last_fullsized":"2024-02-23T00:43:15.542414+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:48.118585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:54.938209+0000","last_clean_scrub_stamp":"2024-02-23T00:42:48.118585+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.576016+0000","last_change":"2024-02-23T00:42:58.746104+0000","last_active":"2024-02-23T00:43:15.576016+0000","last_peered":"2024-02-23T00:43:15.576016+0000","last_clean":"2024-02-23T00:43:15.576016+0000","last_became_active":"2024-02-23T00:42:58.745471+0000","last_became_peered":"2024-02-23T00:42:58.745471+0000","last_unstale":"2024-02-23T00:43:15.576016+0000","last_undegraded":"2024-02-23T00:43:15.576016+0000","last_fullsized":"2024-02-23T00:43:15.576016+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.560690+0000","last_change":"2024-02-23T00:42:57.652853+0000","last_active":"2024-02-23T00:43:15.560690+0000","last_peered":"2024-02-23T00:43:15.560690+0000","last_clean":"2024-02-23T00:43:15.560690+0000","last_became_active":"2024-02-23T00:42:57.652488+0000","last_became_peered":"2024-02-23T00:42:57.652488+0000","last_unstale":"2024-02-23T00:43:15.560690+0000","last_undegraded":"2024-02-23T00:43:15.560690+0000","last_fullsized":"2024-02-23T00:43:15.560690+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.5","version":"116'2","reported_seq":35,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.575859+0000","last_change":"2024-02-23T00:43:05.860442+0000","last_active":"2024-02-23T00:43:15.575859+0000","last_peered":"2024-02-23T00:43:15.575859+0000","last_clean":"2024-02-23T00:43:15.575859+0000","last_became_active":"2024-02-23T00:42:58.728978+0000","last_became_peered":"2024-02-23T00:42:58.728978+0000","last_unstale":"2024-02-23T00:43:15.575859+0000","last_undegraded":"2024-02-23T00:43:15.575859+0000","last_fullsized":"2024-02-23T00:43:15.575859+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"116'2","last_scrub_stamp":"2024-02-23T00:43:05.860338+0000","last_deep_scrub":"116'2","last_deep_scrub_stamp":"2024-02-23T00:43:05.860338+0000","last_clean_scrub_stamp":"2024-02-23T00:43:05.860338+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"67'9","reported_seq":98,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.553977+0000","last_change":"2024-02-23T00:42:57.705378+0000","last_active":"2024-02-23T00:43:15.553977+0000","last_peered":"2024-02-23T00:43:15.553977+0000","last_clean":"2024-02-23T00:43:15.553977+0000","last_became_active":"2024-02-23T00:39:16.786936+0000","last_became_peered":"2024-02-23T00:39:16.786936+0000","last_unstale":"2024-02-23T00:43:15.553977+0000","last_undegraded":"2024-02-23T00:43:15.553977+0000","last_fullsized":"2024-02-23T00:43:15.553977+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'9","last_scrub_stamp":"2024-02-23T00:42:57.705261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.705261+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":89,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.538647+0000","last_change":"2024-02-23T00:42:42.461098+0000","last_active":"2024-02-23T00:43:15.538647+0000","last_peered":"2024-02-23T00:43:15.538647+0000","last_clean":"2024-02-23T00:43:15.538647+0000","last_became_active":"2024-02-23T00:39:14.778455+0000","last_became_peered":"2024-02-23T00:39:14.778455+0000","last_unstale":"2024-02-23T00:43:15.538647+0000","last_undegraded":"2024-02-23T00:43:15.538647+0000","last_fullsized":"2024-02-23T00:43:15.538647+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:42.461020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:42.461020+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":81,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.550224+0000","last_change":"2024-02-23T00:42:40.896368+0000","last_active":"2024-02-23T00:43:15.550224+0000","last_peered":"2024-02-23T00:43:15.550224+0000","last_clean":"2024-02-23T00:43:15.550224+0000","last_became_active":"2024-02-23T00:39:18.799052+0000","last_became_peered":"2024-02-23T00:39:18.799052+0000","last_unstale":"2024-02-23T00:43:15.550224+0000","last_undegraded":"2024-02-23T00:43:15.550224+0000","last_fullsized":"2024-02-23T00:43:15.550224+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:40.896249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_clean_scrub_stamp":"2024-02-23T00:42:40.896249+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":79,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.542843+0000","last_change":"2024-02-23T00:42:50.425748+0000","last_active":"2024-02-23T00:43:15.542843+0000","last_peered":"2024-02-23T00:43:15.542843+0000","last_clean":"2024-02-23T00:43:15.542843+0000","last_became_active":"2024-02-23T00:39:20.802648+0000","last_became_peered":"2024-02-23T00:39:20.802648+0000","last_unstale":"2024-02-23T00:43:15.542843+0000","last_undegraded":"2024-02-23T00:43:15.542843+0000","last_fullsized":"2024-02-23T00:43:15.542843+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:50.425641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:50.425641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.547342+0000","last_change":"2024-02-23T00:42:58.834611+0000","last_active":"2024-02-23T00:43:15.547342+0000","last_peered":"2024-02-23T00:43:15.547342+0000","last_clean":"2024-02-23T00:43:15.547342+0000","last_became_active":"2024-02-23T00:42:58.833531+0000","last_became_peered":"2024-02-23T00:42:58.833531+0000","last_unstale":"2024-02-23T00:43:15.547342+0000","last_undegraded":"2024-02-23T00:43:15.547342+0000","last_fullsized":"2024-02-23T00:43:15.547342+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.538582+0000","last_change":"2024-02-23T00:42:58.841292+0000","last_active":"2024-02-23T00:43:15.538582+0000","last_peered":"2024-02-23T00:43:15.538582+0000","last_clean":"2024-02-23T00:43:15.538582+0000","last_became_active":"2024-02-23T00:42:58.835862+0000","last_became_peered":"2024-02-23T00:42:58.835862+0000","last_unstale":"2024-02-23T00:43:15.538582+0000","last_undegraded":"2024-02-23T00:43:15.538582+0000","last_fullsized":"2024-02-23T00:43:15.538582+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.2","version":"110'6","reported_seq":37,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.549227+0000","last_change":"2024-02-23T00:43:06.013905+0000","last_active":"2024-02-23T00:43:15.549227+0000","last_peered":"2024-02-23T00:43:15.549227+0000","last_clean":"2024-02-23T00:43:15.549227+0000","last_became_active":"2024-02-23T00:42:58.718984+0000","last_became_peered":"2024-02-23T00:42:58.718984+0000","last_unstale":"2024-02-23T00:43:15.549227+0000","last_undegraded":"2024-02-23T00:43:15.549227+0000","last_fullsized":"2024-02-23T00:43:15.549227+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'6","last_scrub_stamp":"2024-02-23T00:43:06.013828+0000","last_deep_scrub":"110'6","last_deep_scrub_stamp":"2024-02-23T00:43:06.013828+0000","last_clean_scrub_stamp":"2024-02-23T00:43:06.013828+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":64,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"67'11","reported_seq":101,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.539983+0000","last_change":"2024-02-23T00:42:58.013765+0000","last_active":"2024-02-23T00:43:15.539983+0000","last_peered":"2024-02-23T00:43:15.539983+0000","last_clean":"2024-02-23T00:43:15.539983+0000","last_became_active":"2024-02-23T00:39:16.783896+0000","last_became_peered":"2024-02-23T00:39:16.783896+0000","last_unstale":"2024-02-23T00:43:15.539983+0000","last_undegraded":"2024-02-23T00:43:15.539983+0000","last_fullsized":"2024-02-23T00:43:15.539983+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'11","last_scrub_stamp":"2024-02-23T00:42:58.013688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:58.013688+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"60'1","reported_seq":91,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.574974+0000","last_change":"2024-02-23T00:41:57.687046+0000","last_active":"2024-02-23T00:43:15.574974+0000","last_peered":"2024-02-23T00:43:15.574974+0000","last_clean":"2024-02-23T00:43:15.574974+0000","last_became_active":"2024-02-23T00:39:14.775804+0000","last_became_peered":"2024-02-23T00:39:14.775804+0000","last_unstale":"2024-02-23T00:43:15.574974+0000","last_undegraded":"2024-02-23T00:43:15.574974+0000","last_fullsized":"2024-02-23T00:43:15.574974+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'1","last_scrub_stamp":"2024-02-23T00:41:57.686958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:57.686958+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":75,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.554025+0000","last_change":"2024-02-23T00:40:59.669880+0000","last_active":"2024-02-23T00:43:15.554025+0000","last_peered":"2024-02-23T00:43:15.554025+0000","last_clean":"2024-02-23T00:43:15.554025+0000","last_became_active":"2024-02-23T00:39:18.800382+0000","last_became_peered":"2024-02-23T00:39:18.800382+0000","last_unstale":"2024-02-23T00:43:15.554025+0000","last_undegraded":"2024-02-23T00:43:15.554025+0000","last_fullsized":"2024-02-23T00:43:15.554025+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:40:59.669810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:40:59.669810+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":79,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.550342+0000","last_change":"2024-02-23T00:42:08.062268+0000","last_active":"2024-02-23T00:43:15.550342+0000","last_peered":"2024-02-23T00:43:15.550342+0000","last_clean":"2024-02-23T00:43:15.550342+0000","last_became_active":"2024-02-23T00:39:20.799053+0000","last_became_peered":"2024-02-23T00:39:20.799053+0000","last_unstale":"2024-02-23T00:43:15.550342+0000","last_undegraded":"2024-02-23T00:43:15.550342+0000","last_fullsized":"2024-02-23T00:43:15.550342+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:08.062156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:08.062156+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"110'2","reported_seq":28,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.538858+0000","last_change":"2024-02-23T00:42:58.709534+0000","last_active":"2024-02-23T00:43:15.538858+0000","last_peered":"2024-02-23T00:43:15.538858+0000","last_clean":"2024-02-23T00:43:15.538858+0000","last_became_active":"2024-02-23T00:42:58.698725+0000","last_became_peered":"2024-02-23T00:42:58.698725+0000","last_unstale":"2024-02-23T00:43:15.538858+0000","last_undegraded":"2024-02-23T00:43:15.538858+0000","last_fullsized":"2024-02-23T00:43:15.538858+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.579851+0000","last_change":"2024-02-23T00:42:57.652742+0000","last_active":"2024-02-23T00:43:15.579851+0000","last_peered":"2024-02-23T00:43:15.579851+0000","last_clean":"2024-02-23T00:43:15.579851+0000","last_became_active":"2024-02-23T00:42:57.652438+0000","last_became_peered":"2024-02-23T00:42:57.652438+0000","last_unstale":"2024-02-23T00:43:15.579851+0000","last_undegraded":"2024-02-23T00:43:15.579851+0000","last_fullsized":"2024-02-23T00:43:15.579851+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.3","version":"110'1","reported_seq":32,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.542761+0000","last_change":"2024-02-23T00:43:04.401584+0000","last_active":"2024-02-23T00:43:15.542761+0000","last_peered":"2024-02-23T00:43:15.542761+0000","last_clean":"2024-02-23T00:43:15.542761+0000","last_became_active":"2024-02-23T00:42:58.823366+0000","last_became_peered":"2024-02-23T00:42:58.823366+0000","last_unstale":"2024-02-23T00:43:15.542761+0000","last_undegraded":"2024-02-23T00:43:15.542761+0000","last_fullsized":"2024-02-23T00:43:15.542761+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'1","last_scrub_stamp":"2024-02-23T00:43:04.401467+0000","last_deep_scrub":"110'1","last_deep_scrub_stamp":"2024-02-23T00:43:04.401467+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.401467+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1024,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"67'4","reported_seq":89,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.538689+0000","last_change":"2024-02-23T00:42:42.934024+0000","last_active":"2024-02-23T00:43:15.538689+0000","last_peered":"2024-02-23T00:43:15.538689+0000","last_clean":"2024-02-23T00:43:15.538689+0000","last_became_active":"2024-02-23T00:39:16.785830+0000","last_became_peered":"2024-02-23T00:39:16.785830+0000","last_unstale":"2024-02-23T00:43:15.538689+0000","last_undegraded":"2024-02-23T00:43:15.538689+0000","last_fullsized":"2024-02-23T00:43:15.538689+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'4","last_scrub_stamp":"2024-02-23T00:42:42.933958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:42.933958+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":85,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.555004+0000","last_change":"2024-02-23T00:41:41.650513+0000","last_active":"2024-02-23T00:43:15.555004+0000","last_peered":"2024-02-23T00:43:15.555004+0000","last_clean":"2024-02-23T00:43:15.555004+0000","last_became_active":"2024-02-23T00:39:14.783325+0000","last_became_peered":"2024-02-23T00:39:14.783325+0000","last_unstale":"2024-02-23T00:43:15.555004+0000","last_undegraded":"2024-02-23T00:43:15.555004+0000","last_fullsized":"2024-02-23T00:43:15.555004+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:41.650434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:41.650434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":85,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.543963+0000","last_change":"2024-02-23T00:42:57.574460+0000","last_active":"2024-02-23T00:43:15.543963+0000","last_peered":"2024-02-23T00:43:15.543963+0000","last_clean":"2024-02-23T00:43:15.543963+0000","last_became_active":"2024-02-23T00:39:18.797287+0000","last_became_peered":"2024-02-23T00:39:18.797287+0000","last_unstale":"2024-02-23T00:43:15.543963+0000","last_undegraded":"2024-02-23T00:43:15.543963+0000","last_fullsized":"2024-02-23T00:43:15.543963+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.574369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.574369+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":77,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.549414+0000","last_change":"2024-02-23T00:42:44.983880+0000","last_active":"2024-02-23T00:43:15.549414+0000","last_peered":"2024-02-23T00:43:15.549414+0000","last_clean":"2024-02-23T00:43:15.549414+0000","last_became_active":"2024-02-23T00:39:20.795351+0000","last_became_peered":"2024-02-23T00:39:20.795351+0000","last_unstale":"2024-02-23T00:43:15.549414+0000","last_undegraded":"2024-02-23T00:43:15.549414+0000","last_fullsized":"2024-02-23T00:43:15.549414+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:44.983813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:44.983813+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1b","version":"110'10","reported_seq":36,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.550284+0000","last_change":"2024-02-23T00:42:58.749680+0000","last_active":"2024-02-23T00:43:15.550284+0000","last_peered":"2024-02-23T00:43:15.550284+0000","last_clean":"2024-02-23T00:43:15.550284+0000","last_became_active":"2024-02-23T00:42:58.749126+0000","last_became_peered":"2024-02-23T00:42:58.749126+0000","last_unstale":"2024-02-23T00:43:15.550284+0000","last_undegraded":"2024-02-23T00:43:15.550284+0000","last_fullsized":"2024-02-23T00:43:15.550284+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.557757+0000","last_change":"2024-02-23T00:42:58.741674+0000","last_active":"2024-02-23T00:43:15.557757+0000","last_peered":"2024-02-23T00:43:15.557757+0000","last_clean":"2024-02-23T00:43:15.557757+0000","last_became_active":"2024-02-23T00:42:58.741365+0000","last_became_peered":"2024-02-23T00:42:58.741365+0000","last_unstale":"2024-02-23T00:43:15.557757+0000","last_undegraded":"2024-02-23T00:43:15.557757+0000","last_fullsized":"2024-02-23T00:43:15.557757+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":30,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.538772+0000","last_change":"2024-02-23T00:43:03.117508+0000","last_active":"2024-02-23T00:43:15.538772+0000","last_peered":"2024-02-23T00:43:15.538772+0000","last_clean":"2024-02-23T00:43:15.538772+0000","last_became_active":"2024-02-23T00:42:58.804980+0000","last_became_peered":"2024-02-23T00:42:58.804980+0000","last_unstale":"2024-02-23T00:43:15.538772+0000","last_undegraded":"2024-02-23T00:43:15.538772+0000","last_fullsized":"2024-02-23T00:43:15.538772+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:03.117444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:03.117444+0000","last_clean_scrub_stamp":"2024-02-23T00:43:03.117444+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"68'17","reported_seq":117,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.552569+0000","last_change":"2024-02-23T00:41:45.979896+0000","last_active":"2024-02-23T00:43:15.552569+0000","last_peered":"2024-02-23T00:43:15.552569+0000","last_clean":"2024-02-23T00:43:15.552569+0000","last_became_active":"2024-02-23T00:39:16.857232+0000","last_became_peered":"2024-02-23T00:39:16.857232+0000","last_unstale":"2024-02-23T00:43:15.552569+0000","last_undegraded":"2024-02-23T00:43:15.552569+0000","last_fullsized":"2024-02-23T00:43:15.552569+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:41:45.979819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:45.979819+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":84,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.550856+0000","last_change":"2024-02-23T00:41:47.034138+0000","last_active":"2024-02-23T00:43:15.550856+0000","last_peered":"2024-02-23T00:43:15.550856+0000","last_clean":"2024-02-23T00:43:15.550856+0000","last_became_active":"2024-02-23T00:39:14.784092+0000","last_became_peered":"2024-02-23T00:39:14.784092+0000","last_unstale":"2024-02-23T00:43:15.550856+0000","last_undegraded":"2024-02-23T00:43:15.550856+0000","last_fullsized":"2024-02-23T00:43:15.550856+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:47.034064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:47.034064+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":79,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.547086+0000","last_change":"2024-02-23T00:42:02.965536+0000","last_active":"2024-02-23T00:43:15.547086+0000","last_peered":"2024-02-23T00:43:15.547086+0000","last_clean":"2024-02-23T00:43:15.547086+0000","last_became_active":"2024-02-23T00:39:18.791457+0000","last_became_peered":"2024-02-23T00:39:18.791457+0000","last_unstale":"2024-02-23T00:43:15.547086+0000","last_undegraded":"2024-02-23T00:43:15.547086+0000","last_fullsized":"2024-02-23T00:43:15.547086+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:02.965455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:02.965455+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":74,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.550316+0000","last_change":"2024-02-23T00:41:40.941453+0000","last_active":"2024-02-23T00:43:15.550316+0000","last_peered":"2024-02-23T00:43:15.550316+0000","last_clean":"2024-02-23T00:43:15.550316+0000","last_became_active":"2024-02-23T00:39:21.160376+0000","last_became_peered":"2024-02-23T00:39:21.160376+0000","last_unstale":"2024-02-23T00:43:15.550316+0000","last_undegraded":"2024-02-23T00:43:15.550316+0000","last_fullsized":"2024-02-23T00:43:15.550316+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:40.941380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:40.941380+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1a","version":"110'5","reported_seq":31,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.537567+0000","last_change":"2024-02-23T00:42:58.738588+0000","last_active":"2024-02-23T00:43:15.537567+0000","last_peered":"2024-02-23T00:43:15.537567+0000","last_clean":"2024-02-23T00:43:15.537567+0000","last_became_active":"2024-02-23T00:42:58.736706+0000","last_became_peered":"2024-02-23T00:42:58.736706+0000","last_unstale":"2024-02-23T00:43:15.537567+0000","last_undegraded":"2024-02-23T00:43:15.537567+0000","last_fullsized":"2024-02-23T00:43:15.537567+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.582958+0000","last_change":"2024-02-23T00:42:58.705557+0000","last_active":"2024-02-23T00:43:15.582958+0000","last_peered":"2024-02-23T00:43:15.582958+0000","last_clean":"2024-02-23T00:43:15.582958+0000","last_became_active":"2024-02-23T00:42:58.702580+0000","last_became_peered":"2024-02-23T00:42:58.702580+0000","last_unstale":"2024-02-23T00:43:15.582958+0000","last_undegraded":"2024-02-23T00:43:15.582958+0000","last_fullsized":"2024-02-23T00:43:15.582958+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"110'1","reported_seq":31,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.544480+0000","last_change":"2024-02-23T00:43:02.449579+0000","last_active":"2024-02-23T00:43:15.544480+0000","last_peered":"2024-02-23T00:43:15.544480+0000","last_clean":"2024-02-23T00:43:15.544480+0000","last_became_active":"2024-02-23T00:42:58.804939+0000","last_became_peered":"2024-02-23T00:42:58.804939+0000","last_unstale":"2024-02-23T00:43:15.544480+0000","last_undegraded":"2024-02-23T00:43:15.544480+0000","last_fullsized":"2024-02-23T00:43:15.544480+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'1","last_scrub_stamp":"2024-02-23T00:43:02.449516+0000","last_deep_scrub":"110'1","last_deep_scrub_stamp":"2024-02-23T00:43:02.449516+0000","last_clean_scrub_stamp":"2024-02-23T00:43:02.449516+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,4],"acting":[6,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"68'16","reported_seq":113,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.545301+0000","last_change":"2024-02-23T00:42:53.444587+0000","last_active":"2024-02-23T00:43:15.545301+0000","last_peered":"2024-02-23T00:43:15.545301+0000","last_clean":"2024-02-23T00:43:15.545301+0000","last_became_active":"2024-02-23T00:39:16.858945+0000","last_became_peered":"2024-02-23T00:39:16.858945+0000","last_unstale":"2024-02-23T00:43:15.545301+0000","last_undegraded":"2024-02-23T00:43:15.545301+0000","last_fullsized":"2024-02-23T00:43:15.545301+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:42:53.444512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.444512+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":92,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.563350+0000","last_change":"2024-02-23T00:41:32.702659+0000","last_active":"2024-02-23T00:43:15.563350+0000","last_peered":"2024-02-23T00:43:15.563350+0000","last_clean":"2024-02-23T00:43:15.563350+0000","last_became_active":"2024-02-23T00:39:14.785363+0000","last_became_peered":"2024-02-23T00:39:14.785363+0000","last_unstale":"2024-02-23T00:43:15.563350+0000","last_undegraded":"2024-02-23T00:43:15.563350+0000","last_fullsized":"2024-02-23T00:43:15.563350+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"80'2","last_scrub_stamp":"2024-02-23T00:41:32.702574+0000","last_deep_scrub":"80'2","last_deep_scrub_stamp":"2024-02-23T00:40:15.942725+0000","last_clean_scrub_stamp":"2024-02-23T00:41:32.702574+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"93'9","reported_seq":140,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.551599+0000","last_change":"2024-02-23T00:41:38.881071+0000","last_active":"2024-02-23T00:43:15.551599+0000","last_peered":"2024-02-23T00:43:15.551599+0000","last_clean":"2024-02-23T00:43:15.551599+0000","last_became_active":"2024-02-23T00:39:18.796680+0000","last_became_peered":"2024-02-23T00:39:18.796680+0000","last_unstale":"2024-02-23T00:43:15.551599+0000","last_undegraded":"2024-02-23T00:43:15.551599+0000","last_fullsized":"2024-02-23T00:43:15.551599+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-02-23T00:41:38.880989+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-02-23T00:41:38.880989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:38.880989+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"67'1","reported_seq":29,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.541732+0000","last_change":"2024-02-23T00:43:00.972717+0000","last_active":"2024-02-23T00:43:15.541732+0000","last_peered":"2024-02-23T00:43:15.541732+0000","last_clean":"2024-02-23T00:43:15.541732+0000","last_became_active":"2024-02-23T00:43:00.933797+0000","last_became_peered":"2024-02-23T00:43:00.933797+0000","last_unstale":"2024-02-23T00:43:15.541732+0000","last_undegraded":"2024-02-23T00:43:15.541732+0000","last_fullsized":"2024-02-23T00:43:15.541732+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:41:46.926552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:46.926552+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.556332+0000","last_change":"2024-02-23T00:42:58.805923+0000","last_active":"2024-02-23T00:43:15.556332+0000","last_peered":"2024-02-23T00:43:15.556332+0000","last_clean":"2024-02-23T00:43:15.556332+0000","last_became_active":"2024-02-23T00:42:58.803900+0000","last_became_peered":"2024-02-23T00:42:58.803900+0000","last_unstale":"2024-02-23T00:43:15.556332+0000","last_undegraded":"2024-02-23T00:43:15.556332+0000","last_fullsized":"2024-02-23T00:43:15.556332+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.544208+0000","last_change":"2024-02-23T00:42:58.850530+0000","last_active":"2024-02-23T00:43:15.544208+0000","last_peered":"2024-02-23T00:43:15.544208+0000","last_clean":"2024-02-23T00:43:15.544208+0000","last_became_active":"2024-02-23T00:42:58.850072+0000","last_became_peered":"2024-02-23T00:42:58.850072+0000","last_unstale":"2024-02-23T00:43:15.544208+0000","last_undegraded":"2024-02-23T00:43:15.544208+0000","last_fullsized":"2024-02-23T00:43:15.544208+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":25,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.546512+0000","last_change":"2024-02-23T00:42:58.743439+0000","last_active":"2024-02-23T00:43:15.546512+0000","last_peered":"2024-02-23T00:43:15.546512+0000","last_clean":"2024-02-23T00:43:15.546512+0000","last_became_active":"2024-02-23T00:42:58.734965+0000","last_became_peered":"2024-02-23T00:42:58.734965+0000","last_unstale":"2024-02-23T00:43:15.546512+0000","last_undegraded":"2024-02-23T00:43:15.546512+0000","last_fullsized":"2024-02-23T00:43:15.546512+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"93'23","reported_seq":127,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.551835+0000","last_change":"2024-02-23T00:42:48.524850+0000","last_active":"2024-02-23T00:43:15.551835+0000","last_peered":"2024-02-23T00:43:15.551835+0000","last_clean":"2024-02-23T00:43:15.551835+0000","last_became_active":"2024-02-23T00:39:16.782660+0000","last_became_peered":"2024-02-23T00:39:16.782660+0000","last_unstale":"2024-02-23T00:43:15.551835+0000","last_undegraded":"2024-02-23T00:43:15.551835+0000","last_fullsized":"2024-02-23T00:43:15.551835+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'23","last_scrub_stamp":"2024-02-23T00:42:48.524772+0000","last_deep_scrub":"93'23","last_deep_scrub_stamp":"2024-02-23T00:41:35.954516+0000","last_clean_scrub_stamp":"2024-02-23T00:42:48.524772+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":85,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.550487+0000","last_change":"2024-02-23T00:41:57.053372+0000","last_active":"2024-02-23T00:43:15.550487+0000","last_peered":"2024-02-23T00:43:15.550487+0000","last_clean":"2024-02-23T00:43:15.550487+0000","last_became_active":"2024-02-23T00:39:14.781560+0000","last_became_peered":"2024-02-23T00:39:14.781560+0000","last_unstale":"2024-02-23T00:43:15.550487+0000","last_undegraded":"2024-02-23T00:43:15.550487+0000","last_fullsized":"2024-02-23T00:43:15.550487+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:57.053295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:57.053295+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":81,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.545381+0000","last_change":"2024-02-23T00:41:53.189947+0000","last_active":"2024-02-23T00:43:15.545381+0000","last_peered":"2024-02-23T00:43:15.545381+0000","last_clean":"2024-02-23T00:43:15.545381+0000","last_became_active":"2024-02-23T00:39:18.790602+0000","last_became_peered":"2024-02-23T00:39:18.790602+0000","last_unstale":"2024-02-23T00:43:15.545381+0000","last_undegraded":"2024-02-23T00:43:15.545381+0000","last_fullsized":"2024-02-23T00:43:15.545381+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:53.189881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:53.189881+0000","last_clean_scrub_stamp":"2024-02-23T00:41:53.189881+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.14","version":"110'2","reported_seq":28,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.539474+0000","last_change":"2024-02-23T00:42:58.804637+0000","last_active":"2024-02-23T00:43:15.539474+0000","last_peered":"2024-02-23T00:43:15.539474+0000","last_clean":"2024-02-23T00:43:15.539474+0000","last_became_active":"2024-02-23T00:42:58.804344+0000","last_became_peered":"2024-02-23T00:42:58.804344+0000","last_unstale":"2024-02-23T00:43:15.539474+0000","last_undegraded":"2024-02-23T00:43:15.539474+0000","last_fullsized":"2024-02-23T00:43:15.539474+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.550563+0000","last_change":"2024-02-23T00:42:58.778321+0000","last_active":"2024-02-23T00:43:15.550563+0000","last_peered":"2024-02-23T00:43:15.550563+0000","last_clean":"2024-02-23T00:43:15.550563+0000","last_became_active":"2024-02-23T00:42:58.774720+0000","last_became_peered":"2024-02-23T00:42:58.774720+0000","last_unstale":"2024-02-23T00:43:15.550563+0000","last_undegraded":"2024-02-23T00:43:15.550563+0000","last_fullsized":"2024-02-23T00:43:15.550563+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":25,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.542778+0000","last_change":"2024-02-23T00:42:58.766655+0000","last_active":"2024-02-23T00:43:15.542778+0000","last_peered":"2024-02-23T00:43:15.542778+0000","last_clean":"2024-02-23T00:43:15.542778+0000","last_became_active":"2024-02-23T00:42:58.764146+0000","last_became_peered":"2024-02-23T00:42:58.764146+0000","last_unstale":"2024-02-23T00:43:15.542778+0000","last_undegraded":"2024-02-23T00:43:15.542778+0000","last_fullsized":"2024-02-23T00:43:15.542778+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"67'15","reported_seq":107,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.567378+0000","last_change":"2024-02-23T00:41:41.121005+0000","last_active":"2024-02-23T00:43:15.567378+0000","last_peered":"2024-02-23T00:43:15.567378+0000","last_clean":"2024-02-23T00:43:15.567378+0000","last_became_active":"2024-02-23T00:39:16.787138+0000","last_became_peered":"2024-02-23T00:39:16.787138+0000","last_unstale":"2024-02-23T00:43:15.567378+0000","last_undegraded":"2024-02-23T00:43:15.567378+0000","last_fullsized":"2024-02-23T00:43:15.567378+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:41:41.120939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:41.120939+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":84,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.545834+0000","last_change":"2024-02-23T00:41:57.138591+0000","last_active":"2024-02-23T00:43:15.545834+0000","last_peered":"2024-02-23T00:43:15.545834+0000","last_clean":"2024-02-23T00:43:15.545834+0000","last_became_active":"2024-02-23T00:39:14.782494+0000","last_became_peered":"2024-02-23T00:39:14.782494+0000","last_unstale":"2024-02-23T00:43:15.545834+0000","last_undegraded":"2024-02-23T00:43:15.545834+0000","last_fullsized":"2024-02-23T00:43:15.545834+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:57.138515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:57.138515+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":80,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.575255+0000","last_change":"2024-02-23T00:41:52.641797+0000","last_active":"2024-02-23T00:43:15.575255+0000","last_peered":"2024-02-23T00:43:15.575255+0000","last_clean":"2024-02-23T00:43:15.575255+0000","last_became_active":"2024-02-23T00:39:18.801039+0000","last_became_peered":"2024-02-23T00:39:18.801039+0000","last_unstale":"2024-02-23T00:43:15.575255+0000","last_undegraded":"2024-02-23T00:43:15.575255+0000","last_fullsized":"2024-02-23T00:43:15.575255+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:52.641689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.641689+0000","last_clean_scrub_stamp":"2024-02-23T00:41:52.641689+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"110'5","reported_seq":31,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.561960+0000","last_change":"2024-02-23T00:42:58.783591+0000","last_active":"2024-02-23T00:43:15.561960+0000","last_peered":"2024-02-23T00:43:15.561960+0000","last_clean":"2024-02-23T00:43:15.561960+0000","last_became_active":"2024-02-23T00:42:58.783343+0000","last_became_peered":"2024-02-23T00:42:58.783343+0000","last_unstale":"2024-02-23T00:43:15.561960+0000","last_undegraded":"2024-02-23T00:43:15.561960+0000","last_fullsized":"2024-02-23T00:43:15.561960+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.550466+0000","last_change":"2024-02-23T00:42:58.773110+0000","last_active":"2024-02-23T00:43:15.550466+0000","last_peered":"2024-02-23T00:43:15.550466+0000","last_clean":"2024-02-23T00:43:15.550466+0000","last_became_active":"2024-02-23T00:42:58.770759+0000","last_became_peered":"2024-02-23T00:42:58.770759+0000","last_unstale":"2024-02-23T00:43:15.550466+0000","last_undegraded":"2024-02-23T00:43:15.550466+0000","last_fullsized":"2024-02-23T00:43:15.550466+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":25,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.542224+0000","last_change":"2024-02-23T00:42:58.676904+0000","last_active":"2024-02-23T00:43:15.542224+0000","last_peered":"2024-02-23T00:43:15.542224+0000","last_clean":"2024-02-23T00:43:15.542224+0000","last_became_active":"2024-02-23T00:42:58.671953+0000","last_became_peered":"2024-02-23T00:42:58.671953+0000","last_unstale":"2024-02-23T00:43:15.542224+0000","last_undegraded":"2024-02-23T00:43:15.542224+0000","last_fullsized":"2024-02-23T00:43:15.542224+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"68'18","reported_seq":121,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.546522+0000","last_change":"2024-02-23T00:41:54.082061+0000","last_active":"2024-02-23T00:43:15.546522+0000","last_peered":"2024-02-23T00:43:15.546522+0000","last_clean":"2024-02-23T00:43:15.546522+0000","last_became_active":"2024-02-23T00:39:16.782803+0000","last_became_peered":"2024-02-23T00:39:16.782803+0000","last_unstale":"2024-02-23T00:43:15.546522+0000","last_undegraded":"2024-02-23T00:43:15.546522+0000","last_fullsized":"2024-02-23T00:43:15.546522+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'18","last_scrub_stamp":"2024-02-23T00:41:54.081977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:54.081977+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":85,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.550142+0000","last_change":"2024-02-23T00:41:44.937852+0000","last_active":"2024-02-23T00:43:15.550142+0000","last_peered":"2024-02-23T00:43:15.550142+0000","last_clean":"2024-02-23T00:43:15.550142+0000","last_became_active":"2024-02-23T00:39:14.782163+0000","last_became_peered":"2024-02-23T00:39:14.782163+0000","last_unstale":"2024-02-23T00:43:15.550142+0000","last_undegraded":"2024-02-23T00:43:15.550142+0000","last_fullsized":"2024-02-23T00:43:15.550142+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:44.937776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:44.937776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":80,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.554723+0000","last_change":"2024-02-23T00:42:07.939130+0000","last_active":"2024-02-23T00:43:15.554723+0000","last_peered":"2024-02-23T00:43:15.554723+0000","last_clean":"2024-02-23T00:43:15.554723+0000","last_became_active":"2024-02-23T00:39:18.794903+0000","last_became_peered":"2024-02-23T00:39:18.794903+0000","last_unstale":"2024-02-23T00:43:15.554723+0000","last_undegraded":"2024-02-23T00:43:15.554723+0000","last_fullsized":"2024-02-23T00:43:15.554723+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:07.939058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_clean_scrub_stamp":"2024-02-23T00:42:07.939058+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"110'5","reported_seq":31,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.545464+0000","last_change":"2024-02-23T00:42:58.839927+0000","last_active":"2024-02-23T00:43:15.545464+0000","last_peered":"2024-02-23T00:43:15.545464+0000","last_clean":"2024-02-23T00:43:15.545464+0000","last_became_active":"2024-02-23T00:42:58.838833+0000","last_became_peered":"2024-02-23T00:42:58.838833+0000","last_unstale":"2024-02-23T00:43:15.545464+0000","last_undegraded":"2024-02-23T00:43:15.545464+0000","last_fullsized":"2024-02-23T00:43:15.545464+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.550434+0000","last_change":"2024-02-23T00:42:58.698567+0000","last_active":"2024-02-23T00:43:15.550434+0000","last_peered":"2024-02-23T00:43:15.550434+0000","last_clean":"2024-02-23T00:43:15.550434+0000","last_became_active":"2024-02-23T00:42:58.698258+0000","last_became_peered":"2024-02-23T00:42:58.698258+0000","last_unstale":"2024-02-23T00:43:15.550434+0000","last_undegraded":"2024-02-23T00:43:15.550434+0000","last_fullsized":"2024-02-23T00:43:15.550434+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":25,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.561952+0000","last_change":"2024-02-23T00:42:58.716683+0000","last_active":"2024-02-23T00:43:15.561952+0000","last_peered":"2024-02-23T00:43:15.561952+0000","last_clean":"2024-02-23T00:43:15.561952+0000","last_became_active":"2024-02-23T00:42:58.716194+0000","last_became_peered":"2024-02-23T00:42:58.716194+0000","last_unstale":"2024-02-23T00:43:15.561952+0000","last_undegraded":"2024-02-23T00:43:15.561952+0000","last_fullsized":"2024-02-23T00:43:15.561952+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,1],"acting":[2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"95'27","reported_seq":134,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.555003+0000","last_change":"2024-02-23T00:41:55.920001+0000","last_active":"2024-02-23T00:43:15.555003+0000","last_peered":"2024-02-23T00:43:15.555003+0000","last_clean":"2024-02-23T00:43:15.555003+0000","last_became_active":"2024-02-23T00:39:16.786791+0000","last_became_peered":"2024-02-23T00:39:16.786791+0000","last_unstale":"2024-02-23T00:43:15.555003+0000","last_undegraded":"2024-02-23T00:43:15.555003+0000","last_fullsized":"2024-02-23T00:43:15.555003+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'27","last_scrub_stamp":"2024-02-23T00:41:55.919895+0000","last_deep_scrub":"95'27","last_deep_scrub_stamp":"2024-02-23T00:41:55.919895+0000","last_clean_scrub_stamp":"2024-02-23T00:41:55.919895+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":83,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.553319+0000","last_change":"2024-02-23T00:42:40.046742+0000","last_active":"2024-02-23T00:43:15.553319+0000","last_peered":"2024-02-23T00:43:15.553319+0000","last_clean":"2024-02-23T00:43:15.553319+0000","last_became_active":"2024-02-23T00:39:14.787148+0000","last_became_peered":"2024-02-23T00:39:14.787148+0000","last_unstale":"2024-02-23T00:43:15.553319+0000","last_undegraded":"2024-02-23T00:43:15.553319+0000","last_fullsized":"2024-02-23T00:43:15.553319+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:40.046647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:40.046647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":81,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.549746+0000","last_change":"2024-02-23T00:42:51.967597+0000","last_active":"2024-02-23T00:43:15.549746+0000","last_peered":"2024-02-23T00:43:15.549746+0000","last_clean":"2024-02-23T00:43:15.549746+0000","last_became_active":"2024-02-23T00:39:18.796431+0000","last_became_peered":"2024-02-23T00:39:18.796431+0000","last_unstale":"2024-02-23T00:43:15.549746+0000","last_undegraded":"2024-02-23T00:43:15.549746+0000","last_fullsized":"2024-02-23T00:43:15.549746+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:51.967526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:51.967526+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.548717+0000","last_change":"2024-02-23T00:43:00.945436+0000","last_active":"2024-02-23T00:43:15.548717+0000","last_peered":"2024-02-23T00:43:15.548717+0000","last_clean":"2024-02-23T00:43:15.548717+0000","last_became_active":"2024-02-23T00:43:00.945181+0000","last_became_peered":"2024-02-23T00:43:00.945181+0000","last_unstale":"2024-02-23T00:43:15.548717+0000","last_undegraded":"2024-02-23T00:43:15.548717+0000","last_fullsized":"2024-02-23T00:43:15.548717+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:42.283249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:42.283249+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"110'7","reported_seq":33,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.559396+0000","last_change":"2024-02-23T00:42:58.812691+0000","last_active":"2024-02-23T00:43:15.559396+0000","last_peered":"2024-02-23T00:43:15.559396+0000","last_clean":"2024-02-23T00:43:15.559396+0000","last_became_active":"2024-02-23T00:42:58.810765+0000","last_became_peered":"2024-02-23T00:42:58.810765+0000","last_unstale":"2024-02-23T00:43:15.559396+0000","last_undegraded":"2024-02-23T00:43:15.559396+0000","last_fullsized":"2024-02-23T00:43:15.559396+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.540406+0000","last_change":"2024-02-23T00:42:58.833221+0000","last_active":"2024-02-23T00:43:15.540406+0000","last_peered":"2024-02-23T00:43:15.540406+0000","last_clean":"2024-02-23T00:43:15.540406+0000","last_became_active":"2024-02-23T00:42:58.832885+0000","last_became_peered":"2024-02-23T00:42:58.832885+0000","last_unstale":"2024-02-23T00:43:15.540406+0000","last_undegraded":"2024-02-23T00:43:15.540406+0000","last_fullsized":"2024-02-23T00:43:15.540406+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.a","version":"110'1","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.549788+0000","last_change":"2024-02-23T00:42:58.720623+0000","last_active":"2024-02-23T00:43:15.549788+0000","last_peered":"2024-02-23T00:43:15.549788+0000","last_clean":"2024-02-23T00:43:15.549788+0000","last_became_active":"2024-02-23T00:42:58.718587+0000","last_became_peered":"2024-02-23T00:42:58.718587+0000","last_unstale":"2024-02-23T00:43:15.549788+0000","last_undegraded":"2024-02-23T00:43:15.549788+0000","last_fullsized":"2024-02-23T00:43:15.549788+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"82'15","reported_seq":111,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.540666+0000","last_change":"2024-02-23T00:41:50.975772+0000","last_active":"2024-02-23T00:43:15.540666+0000","last_peered":"2024-02-23T00:43:15.540666+0000","last_clean":"2024-02-23T00:43:15.540666+0000","last_became_active":"2024-02-23T00:39:16.782762+0000","last_became_peered":"2024-02-23T00:39:16.782762+0000","last_unstale":"2024-02-23T00:43:15.540666+0000","last_undegraded":"2024-02-23T00:43:15.540666+0000","last_fullsized":"2024-02-23T00:43:15.540666+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'15","last_scrub_stamp":"2024-02-23T00:41:50.975683+0000","last_deep_scrub":"82'15","last_deep_scrub_stamp":"2024-02-23T00:41:50.975683+0000","last_clean_scrub_stamp":"2024-02-23T00:41:50.975683+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":83,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.555755+0000","last_change":"2024-02-23T00:41:40.926914+0000","last_active":"2024-02-23T00:43:15.555755+0000","last_peered":"2024-02-23T00:43:15.555755+0000","last_clean":"2024-02-23T00:43:15.555755+0000","last_became_active":"2024-02-23T00:39:14.783469+0000","last_became_peered":"2024-02-23T00:39:14.783469+0000","last_unstale":"2024-02-23T00:43:15.555755+0000","last_undegraded":"2024-02-23T00:43:15.555755+0000","last_fullsized":"2024-02-23T00:43:15.555755+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:40.926851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:40.926851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":81,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.549497+0000","last_change":"2024-02-23T00:42:00.998947+0000","last_active":"2024-02-23T00:43:15.549497+0000","last_peered":"2024-02-23T00:43:15.549497+0000","last_clean":"2024-02-23T00:43:15.549497+0000","last_became_active":"2024-02-23T00:39:18.796952+0000","last_became_peered":"2024-02-23T00:39:18.796952+0000","last_unstale":"2024-02-23T00:43:15.549497+0000","last_undegraded":"2024-02-23T00:43:15.549497+0000","last_fullsized":"2024-02-23T00:43:15.549497+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:00.998885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:00.998885+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":24,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.540680+0000","last_change":"2024-02-23T00:43:00.961280+0000","last_active":"2024-02-23T00:43:15.540680+0000","last_peered":"2024-02-23T00:43:15.540680+0000","last_clean":"2024-02-23T00:43:15.540680+0000","last_became_active":"2024-02-23T00:43:00.960896+0000","last_became_peered":"2024-02-23T00:43:00.960896+0000","last_unstale":"2024-02-23T00:43:15.540680+0000","last_undegraded":"2024-02-23T00:43:15.540680+0000","last_fullsized":"2024-02-23T00:43:15.540680+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:08.695270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:55.854964+0000","last_clean_scrub_stamp":"2024-02-23T00:42:08.695270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.549293+0000","last_change":"2024-02-23T00:42:58.731558+0000","last_active":"2024-02-23T00:43:15.549293+0000","last_peered":"2024-02-23T00:43:15.549293+0000","last_clean":"2024-02-23T00:43:15.549293+0000","last_became_active":"2024-02-23T00:42:58.726021+0000","last_became_peered":"2024-02-23T00:42:58.726021+0000","last_unstale":"2024-02-23T00:43:15.549293+0000","last_undegraded":"2024-02-23T00:43:15.549293+0000","last_fullsized":"2024-02-23T00:43:15.549293+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.543001+0000","last_change":"2024-02-23T00:42:58.759026+0000","last_active":"2024-02-23T00:43:15.543001+0000","last_peered":"2024-02-23T00:43:15.543001+0000","last_clean":"2024-02-23T00:43:15.543001+0000","last_became_active":"2024-02-23T00:42:58.758147+0000","last_became_peered":"2024-02-23T00:42:58.758147+0000","last_unstale":"2024-02-23T00:43:15.543001+0000","last_undegraded":"2024-02-23T00:43:15.543001+0000","last_fullsized":"2024-02-23T00:43:15.543001+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":25,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.542614+0000","last_change":"2024-02-23T00:42:58.829447+0000","last_active":"2024-02-23T00:43:15.542614+0000","last_peered":"2024-02-23T00:43:15.542614+0000","last_clean":"2024-02-23T00:43:15.542614+0000","last_became_active":"2024-02-23T00:42:58.824135+0000","last_became_peered":"2024-02-23T00:42:58.824135+0000","last_unstale":"2024-02-23T00:43:15.542614+0000","last_undegraded":"2024-02-23T00:43:15.542614+0000","last_fullsized":"2024-02-23T00:43:15.542614+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"68'15","reported_seq":113,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.549421+0000","last_change":"2024-02-23T00:41:33.064109+0000","last_active":"2024-02-23T00:43:15.549421+0000","last_peered":"2024-02-23T00:43:15.549421+0000","last_clean":"2024-02-23T00:43:15.549421+0000","last_became_active":"2024-02-23T00:39:16.784606+0000","last_became_peered":"2024-02-23T00:39:16.784606+0000","last_unstale":"2024-02-23T00:43:15.549421+0000","last_undegraded":"2024-02-23T00:43:15.549421+0000","last_fullsized":"2024-02-23T00:43:15.549421+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:41:33.063992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:33.063992+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":85,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.547214+0000","last_change":"2024-02-23T00:42:37.949147+0000","last_active":"2024-02-23T00:43:15.547214+0000","last_peered":"2024-02-23T00:43:15.547214+0000","last_clean":"2024-02-23T00:43:15.547214+0000","last_became_active":"2024-02-23T00:39:14.785362+0000","last_became_peered":"2024-02-23T00:39:14.785362+0000","last_unstale":"2024-02-23T00:43:15.547214+0000","last_undegraded":"2024-02-23T00:43:15.547214+0000","last_fullsized":"2024-02-23T00:43:15.547214+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:37.949044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:37.949044+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":80,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.553033+0000","last_change":"2024-02-23T00:42:44.553722+0000","last_active":"2024-02-23T00:43:15.553033+0000","last_peered":"2024-02-23T00:43:15.553033+0000","last_clean":"2024-02-23T00:43:15.553033+0000","last_became_active":"2024-02-23T00:39:18.796312+0000","last_became_peered":"2024-02-23T00:39:18.796312+0000","last_unstale":"2024-02-23T00:43:15.553033+0000","last_undegraded":"2024-02-23T00:43:15.553033+0000","last_fullsized":"2024-02-23T00:43:15.553033+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:44.553621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:44.553621+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.537740+0000","last_change":"2024-02-23T00:43:00.941642+0000","last_active":"2024-02-23T00:43:15.537740+0000","last_peered":"2024-02-23T00:43:15.537740+0000","last_clean":"2024-02-23T00:43:15.537740+0000","last_became_active":"2024-02-23T00:43:00.941350+0000","last_became_peered":"2024-02-23T00:43:00.941350+0000","last_unstale":"2024-02-23T00:43:15.537740+0000","last_undegraded":"2024-02-23T00:43:15.537740+0000","last_fullsized":"2024-02-23T00:43:15.537740+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:00.004521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:00.004521+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"110'9","reported_seq":35,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.561277+0000","last_change":"2024-02-23T00:42:57.656016+0000","last_active":"2024-02-23T00:43:15.561277+0000","last_peered":"2024-02-23T00:43:15.561277+0000","last_clean":"2024-02-23T00:43:15.561277+0000","last_became_active":"2024-02-23T00:42:57.655617+0000","last_became_peered":"2024-02-23T00:42:57.655617+0000","last_unstale":"2024-02-23T00:43:15.561277+0000","last_undegraded":"2024-02-23T00:43:15.561277+0000","last_fullsized":"2024-02-23T00:43:15.561277+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.545052+0000","last_change":"2024-02-23T00:42:58.839750+0000","last_active":"2024-02-23T00:43:15.545052+0000","last_peered":"2024-02-23T00:43:15.545052+0000","last_clean":"2024-02-23T00:43:15.545052+0000","last_became_active":"2024-02-23T00:42:58.838257+0000","last_became_peered":"2024-02-23T00:42:58.838257+0000","last_unstale":"2024-02-23T00:43:15.545052+0000","last_undegraded":"2024-02-23T00:43:15.545052+0000","last_fullsized":"2024-02-23T00:43:15.545052+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":30,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.553625+0000","last_change":"2024-02-23T00:43:08.400281+0000","last_active":"2024-02-23T00:43:15.553625+0000","last_peered":"2024-02-23T00:43:15.553625+0000","last_clean":"2024-02-23T00:43:15.553625+0000","last_became_active":"2024-02-23T00:42:58.691686+0000","last_became_peered":"2024-02-23T00:42:58.691686+0000","last_unstale":"2024-02-23T00:43:15.553625+0000","last_undegraded":"2024-02-23T00:43:15.553625+0000","last_fullsized":"2024-02-23T00:43:15.553625+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:08.400207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:08.400207+0000","last_clean_scrub_stamp":"2024-02-23T00:43:08.400207+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"88'50","reported_seq":179,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.552657+0000","last_change":"2024-02-23T00:42:39.939941+0000","last_active":"2024-02-23T00:43:15.552657+0000","last_peered":"2024-02-23T00:43:15.552657+0000","last_clean":"2024-02-23T00:43:15.552657+0000","last_became_active":"2024-02-23T00:39:16.780144+0000","last_became_peered":"2024-02-23T00:39:16.780144+0000","last_unstale":"2024-02-23T00:43:15.552657+0000","last_undegraded":"2024-02-23T00:43:15.552657+0000","last_fullsized":"2024-02-23T00:43:15.552657+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'50","last_scrub_stamp":"2024-02-23T00:42:39.939829+0000","last_deep_scrub":"88'50","last_deep_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_clean_scrub_stamp":"2024-02-23T00:42:39.939829+0000","log_size":50,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":85,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.541244+0000","last_change":"2024-02-23T00:42:38.204365+0000","last_active":"2024-02-23T00:43:15.541244+0000","last_peered":"2024-02-23T00:43:15.541244+0000","last_clean":"2024-02-23T00:43:15.541244+0000","last_became_active":"2024-02-23T00:39:14.782000+0000","last_became_peered":"2024-02-23T00:39:14.782000+0000","last_unstale":"2024-02-23T00:43:15.541244+0000","last_undegraded":"2024-02-23T00:43:15.541244+0000","last_fullsized":"2024-02-23T00:43:15.541244+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:38.204283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_clean_scrub_stamp":"2024-02-23T00:42:38.204283+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":75,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.559039+0000","last_change":"2024-02-23T00:41:05.965631+0000","last_active":"2024-02-23T00:43:15.559039+0000","last_peered":"2024-02-23T00:43:15.559039+0000","last_clean":"2024-02-23T00:43:15.559039+0000","last_became_active":"2024-02-23T00:39:18.799723+0000","last_became_peered":"2024-02-23T00:39:18.799723+0000","last_unstale":"2024-02-23T00:43:15.559039+0000","last_undegraded":"2024-02-23T00:43:15.559039+0000","last_fullsized":"2024-02-23T00:43:15.559039+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:05.965531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:05.965531+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":83,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.549611+0000","last_change":"2024-02-23T00:43:00.988729+0000","last_active":"2024-02-23T00:43:15.549611+0000","last_peered":"2024-02-23T00:43:15.549611+0000","last_clean":"2024-02-23T00:43:15.549611+0000","last_became_active":"2024-02-23T00:43:00.987582+0000","last_became_peered":"2024-02-23T00:43:00.987582+0000","last_unstale":"2024-02-23T00:43:15.549611+0000","last_undegraded":"2024-02-23T00:43:15.549611+0000","last_fullsized":"2024-02-23T00:43:15.549611+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:34.000844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:34.000844+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.550315+0000","last_change":"2024-02-23T00:42:58.772753+0000","last_active":"2024-02-23T00:43:15.550315+0000","last_peered":"2024-02-23T00:43:15.550315+0000","last_clean":"2024-02-23T00:43:15.550315+0000","last_became_active":"2024-02-23T00:42:58.765643+0000","last_became_peered":"2024-02-23T00:42:58.765643+0000","last_unstale":"2024-02-23T00:43:15.550315+0000","last_undegraded":"2024-02-23T00:43:15.550315+0000","last_fullsized":"2024-02-23T00:43:15.550315+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":26,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.541745+0000","last_change":"2024-02-23T00:42:58.825262+0000","last_active":"2024-02-23T00:43:15.541745+0000","last_peered":"2024-02-23T00:43:15.541745+0000","last_clean":"2024-02-23T00:43:15.541745+0000","last_became_active":"2024-02-23T00:42:58.824573+0000","last_became_peered":"2024-02-23T00:42:58.824573+0000","last_unstale":"2024-02-23T00:43:15.541745+0000","last_undegraded":"2024-02-23T00:43:15.541745+0000","last_fullsized":"2024-02-23T00:43:15.541745+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0],"acting":[5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"110'1","reported_seq":31,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-02-23T00:43:15.547043+0000","last_change":"2024-02-23T00:43:11.035928+0000","last_active":"2024-02-23T00:43:15.547043+0000","last_peered":"2024-02-23T00:43:15.547043+0000","last_clean":"2024-02-23T00:43:15.547043+0000","last_became_active":"2024-02-23T00:42:58.677646+0000","last_became_peered":"2024-02-23T00:42:58.677646+0000","last_unstale":"2024-02-23T00:43:15.547043+0000","last_undegraded":"2024-02-23T00:43:15.547043+0000","last_fullsized":"2024-02-23T00:43:15.547043+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'1","last_scrub_stamp":"2024-02-23T00:43:11.035864+0000","last_deep_scrub":"110'1","last_deep_scrub_stamp":"2024-02-23T00:43:11.035864+0000","last_clean_scrub_stamp":"2024-02-23T00:43:11.035864+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":64,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":63,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":59,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":8},{"poolid":35,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":8},{"poolid":62,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":150998978,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1011,"num_read_kb":49161,"num_write":1043,"num_write_kb":152978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2800000,"num_omap_keys":1000,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":453267456,"data_stored":452996934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1054,"ondisk_log_size":1054,"up":96,"acting":96,"num_store_stats":8},{"poolid":55,"num_pg":4,"stat_sum":{"num_bytes":1280,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":8,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":10,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":3840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":40,"ondisk_log_size":40,"up":12,"acting":12,"num_store_stats":7},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":29,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":87,"acting":87,"num_store_stats":8},{"poolid":42,"num_pg":32,"stat_sum":{"num_bytes":24576,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":73728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1080,"num_read_kb":805,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":632,"ondisk_log_size":632,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":234,"num_read_kb":229,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":47,"num_pg":32,"stat_sum":{"num_bytes":651,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":52,"num_pg":32,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2048,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":196608,"data_stored":6144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234036,"num_pgs":179,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":409220,"kb_used_data":99836,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":93335932,"statfs":{"total":95995035648,"available":95575994368,"internally_reserved":0,"allocated":102232064,"data_stored":101511738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.53500000000000003,"15min":0.54300000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72199999999999998,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.50600000000000001,"15min":0.50800000000000001},"min":{"1min":0.36899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.68100000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.44700000000000001}]},{"osd":1,"last update":"Fri Feb 23 00:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.53000000000000003,"15min":0.54500000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.60399999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.53700000000000003,"15min":0.54600000000000004},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.64100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.39400000000000002}]},{"osd":2,"last update":"Fri Feb 23 00:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.52400000000000002,"15min":0.53600000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.73399999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.433},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53900000000000003,"15min":0.53600000000000003},"min":{"1min":0.42699999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.69099999999999995,"5min":1.024,"15min":1.024},"last":0.47699999999999998}]},{"osd":3,"last update":"Fri Feb 23 00:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.53900000000000003,"15min":0.53100000000000003},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70499999999999996,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57699999999999996,"15min":0.57099999999999995},"min":{"1min":0.41699999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.41699999999999998}]},{"osd":4,"last update":"Fri Feb 23 00:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.438,"15min":0.434},"min":{"1min":0.314,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.63700000000000001,"5min":0.754,"15min":0.754},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.46899999999999997,"15min":0.45900000000000002},"min":{"1min":0.38100000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.70099999999999996,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.45900000000000002}]},{"osd":5,"last update":"Fri Feb 23 00:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.48899999999999999,"15min":0.48199999999999998},"min":{"1min":0.379,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.70699999999999996,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.50600000000000001,"15min":0.48599999999999999},"min":{"1min":0.40600000000000003,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.72299999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.495}]},{"osd":6,"last update":"Fri Feb 23 00:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52100000000000002,"15min":0.51100000000000001},"min":{"1min":0.34599999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.69199999999999995,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.51700000000000002,"15min":0.51000000000000001},"min":{"1min":0.36599999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.67200000000000004,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.51400000000000001}]}]},{"osd":6,"up_from":47,"seq":201863462971,"num_pgs":183,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":380456,"kb_used_data":75168,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":93364696,"statfs":{"total":95995035648,"available":95605448704,"internally_reserved":0,"allocated":76972032,"data_stored":76302634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55200000000000005,"15min":0.56599999999999995},"min":{"1min":0.379,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.77700000000000002,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.55000000000000004,"15min":0.57399999999999995},"min":{"1min":0.36199999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.69499999999999995,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.498}]},{"osd":1,"last update":"Fri Feb 23 00:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54500000000000004,"15min":0.55100000000000005},"min":{"1min":0.34499999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.55900000000000005,"15min":0.57799999999999996},"min":{"1min":0.40400000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76400000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.46200000000000002}]},{"osd":2,"last update":"Fri Feb 23 00:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.58999999999999997,"15min":0.58699999999999997},"min":{"1min":0.40000000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.8439999999999999,"5min":2.8439999999999999,"15min":2.8439999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.58599999999999997,"15min":0.56899999999999995},"min":{"1min":0.441,"5min":0.38,"15min":0.38},"max":{"1min":2.8530000000000002,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":0.44500000000000001}]},{"osd":3,"last update":"Fri Feb 23 00:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.39200000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77600000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.59999999999999998,"15min":0.59699999999999998},"min":{"1min":0.39700000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.51600000000000001}]},{"osd":4,"last update":"Fri Feb 23 00:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.436,"15min":0.41299999999999998},"min":{"1min":0.36699999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.61699999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.46000000000000002,"15min":0.436},"min":{"1min":0.32100000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.64600000000000002,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.47899999999999998}]},{"osd":5,"last update":"Fri Feb 23 00:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.44900000000000001,"15min":0.42199999999999999},"min":{"1min":0.35799999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.628,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.505},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.48199999999999998,"15min":0.44900000000000001},"min":{"1min":0.34499999999999997,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.68700000000000006,"5min":0.877,"15min":0.877},"last":0.62}]},{"osd":7,"last update":"Fri Feb 23 00:43:13 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.58099999999999996,"15min":0.51900000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":8.548,"5min":8.548,"15min":8.548},"last":0.54500000000000004},{"interface":"front","average":{"1min":1.5660000000000001,"5min":0.72999999999999998,"15min":0.58599999999999997},"min":{"1min":0.36499999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":15.618,"5min":15.618,"15min":15.618},"last":0.59099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":181,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":355948,"kb_used_data":50660,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":93389204,"statfs":{"total":95995035648,"available":95630544896,"internally_reserved":0,"allocated":51875840,"data_stored":51165420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:42:38 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.39900000000000002,"15min":0.35499999999999998},"min":{"1min":0.254,"5min":0.214,"15min":0.214},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.442,"15min":0.378},"min":{"1min":0.252,"5min":0.191,"15min":0.191},"max":{"1min":2.194,"5min":2.194,"15min":2.194},"last":0.56399999999999995}]},{"osd":2,"last update":"Fri Feb 23 00:43:00 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.41099999999999998,"15min":0.39900000000000002},"min":{"1min":0.219,"5min":0.188,"15min":0.188},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.46400000000000002,"15min":0.443},"min":{"1min":0.29299999999999998,"5min":0.249,"15min":0.249},"max":{"1min":2.2330000000000001,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":0.61199999999999999}]},{"osd":3,"last update":"Fri Feb 23 00:42:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.48899999999999999,"15min":0.49099999999999999},"min":{"1min":0.29699999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":2.4350000000000001,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.45300000000000001,"15min":0.45900000000000002},"min":{"1min":0.27700000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":1.0409999999999999}]},{"osd":4,"last update":"Fri Feb 23 00:42:49 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":0.70099999999999996,"15min":0.64300000000000002},"min":{"1min":0.44500000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":8.9359999999999999,"5min":8.9359999999999999,"15min":8.9359999999999999},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.87,"5min":0.67100000000000004,"15min":0.63300000000000001},"min":{"1min":0.46500000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":3.9079999999999999,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":0.68600000000000005}]},{"osd":5,"last update":"Fri Feb 23 00:42:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.624,"15min":0.65300000000000002},"min":{"1min":0.42599999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68600000000000005,"15min":0.71199999999999997},"min":{"1min":0.39500000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.997,"5min":1.008,"15min":1.008},"last":0.89700000000000002}]},{"osd":6,"last update":"Fri Feb 23 00:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.64300000000000002,"15min":0.64400000000000002},"min":{"1min":0.52700000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.67000000000000004,"15min":0.66000000000000003},"min":{"1min":0.58199999999999996,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":2.4079999999999999,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":0.93400000000000005}]},{"osd":7,"last update":"Fri Feb 23 00:43:15 2024","interfaces":[{"interface":"back","average":{"1min":1.597,"5min":0.83399999999999996,"15min":0.69399999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":20.225000000000001,"5min":20.225000000000001,"15min":20.225000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.756,"5min":0.84799999999999998,"15min":0.69699999999999995},"min":{"1min":0.48599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":20.254999999999999,"5min":20.254999999999999,"15min":20.254999999999999},"last":0.66600000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":166,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":384552,"kb_used_data":75168,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":93360600,"statfs":{"total":95995035648,"available":95601254400,"internally_reserved":0,"allocated":76972032,"data_stored":76309590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 23 00:42:35 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.44600000000000001,"15min":0.36099999999999999},"min":{"1min":0.26700000000000002,"5min":0.223,"15min":0.19500000000000001},"max":{"1min":6.5919999999999996,"5min":6.5919999999999996,"15min":6.5919999999999996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42099999999999999,"15min":0.35899999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.20799999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":3.4630000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:43:03 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.49099999999999999,"15min":0.41699999999999998},"min":{"1min":0.26000000000000001,"5min":0.24199999999999999,"15min":0.20499999999999999},"max":{"1min":6.3440000000000003,"5min":6.3440000000000003,"15min":6.3440000000000003},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.48799999999999999,"15min":0.41199999999999998},"min":{"1min":0.27300000000000002,"5min":0.24299999999999999,"15min":0.19800000000000001},"max":{"1min":6.407,"5min":6.407,"15min":6.407},"last":0.47799999999999998}]},{"osd":3,"last update":"Fri Feb 23 00:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.52000000000000002,"15min":0.47199999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":7.7110000000000003,"5min":7.7110000000000003,"15min":7.7110000000000003},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.54300000000000004,"15min":0.51800000000000002},"min":{"1min":0.33400000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":6.6929999999999996,"5min":6.6929999999999996,"15min":6.6929999999999996},"last":0.64800000000000002}]},{"osd":4,"last update":"Fri Feb 23 00:42:51 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":0.70099999999999996,"15min":0.70699999999999996},"min":{"1min":0.46000000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":6.4589999999999996,"5min":6.4589999999999996,"15min":6.4589999999999996},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.64600000000000002,"15min":0.72399999999999998},"min":{"1min":0.503,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.85699999999999998,"5min":3.319,"15min":3.319},"last":0.501}]},{"osd":5,"last update":"Fri Feb 23 00:43:16 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":0.67200000000000004,"15min":0.63400000000000001},"min":{"1min":0.375,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":6.4909999999999997,"5min":6.4909999999999997,"15min":6.4909999999999997},"last":0.375},{"interface":"front","average":{"1min":1.0349999999999999,"5min":0.71899999999999997,"15min":0.68899999999999995},"min":{"1min":0.437,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":6.6680000000000001,"5min":6.6680000000000001,"15min":6.6680000000000001},"last":0.437}]},{"osd":6,"last update":"Fri Feb 23 00:42:51 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":0.71599999999999997,"15min":0.66900000000000004},"min":{"1min":0.45300000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":6.633,"5min":6.633,"15min":6.633},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.63300000000000001,"15min":0.63500000000000001},"min":{"1min":0.49099999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.53900000000000003}]},{"osd":7,"last update":"Fri Feb 23 00:43:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":0.72199999999999998,"15min":0.69299999999999995},"min":{"1min":0.47099999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":6.6989999999999998,"5min":6.6989999999999998,"15min":6.6989999999999998},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.63500000000000001,"15min":0.64600000000000002},"min":{"1min":0.48399999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.56899999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":133,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":331292,"kb_used_data":26004,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":93413860,"statfs":{"total":95995035648,"available":95655792640,"internally_reserved":0,"allocated":26628096,"data_stored":25981897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:42:58 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38200000000000001,"15min":0.378},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.48499999999999999,"15min":0.41499999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.23599999999999999},"max":{"1min":6.8879999999999999,"5min":6.8879999999999999,"15min":6.8879999999999999},"last":0.28799999999999998}]},{"osd":1,"last update":"Fri Feb 23 00:42:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.40899999999999997,"15min":0.38300000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.42999999999999999,"15min":0.40400000000000003},"min":{"1min":0.30199999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.30299999999999999}]},{"osd":3,"last update":"Fri Feb 23 00:42:26 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.51400000000000001,"15min":0.501},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":7.0289999999999999,"5min":7.0289999999999999,"15min":7.0289999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.46400000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.71899999999999997,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.38100000000000001}]},{"osd":4,"last update":"Fri Feb 23 00:42:58 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.626,"15min":0.60699999999999998},"min":{"1min":0.48899999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.224,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.629,"15min":0.59399999999999997},"min":{"1min":0.38400000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.878,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.63100000000000001}]},{"osd":5,"last update":"Fri Feb 23 00:42:18 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.59299999999999997,"15min":0.58699999999999997},"min":{"1min":0.41899999999999998,"5min":0.378,"15min":0.378},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.63600000000000001,"15min":0.621},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.67000000000000004}]},{"osd":6,"last update":"Fri Feb 23 00:42:48 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":0.72599999999999998,"15min":0.65800000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":6.9390000000000001,"5min":6.9390000000000001,"15min":6.9390000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.629,"15min":0.60699999999999998},"min":{"1min":0.442,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.95499999999999996,"5min":1.165,"15min":1.165},"last":0.58999999999999997}]},{"osd":7,"last update":"Fri Feb 23 00:43:10 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.74199999999999999,"15min":0.69699999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":6.9740000000000002,"5min":6.9740000000000002,"15min":6.9740000000000002},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.433,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90200000000000002,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.55700000000000005}]}]},{"osd":3,"up_from":26,"seq":111669149769,"num_pgs":191,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":360052,"kb_used_data":50668,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":93385100,"statfs":{"total":95995035648,"available":95626342400,"internally_reserved":0,"allocated":51884032,"data_stored":51152869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.39700000000000002,"15min":0.42899999999999999},"min":{"1min":0.23200000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42299999999999999,"15min":0.45000000000000001},"min":{"1min":0.29399999999999998,"5min":0.248,"15min":0.248},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":2.3380000000000001}]},{"osd":1,"last update":"Fri Feb 23 00:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.432,"15min":0.46700000000000003},"min":{"1min":0.25900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.69499999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":2.173},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.438,"15min":0.45100000000000001},"min":{"1min":0.26200000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.1619999999999999}]},{"osd":2,"last update":"Fri Feb 23 00:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.48499999999999999,"15min":0.495},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":2.698,"5min":2.698,"15min":2.698},"last":2.9849999999999999},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.45400000000000001,"15min":0.47999999999999998},"min":{"1min":0.30399999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.49299999999999999}]},{"osd":4,"last update":"Fri Feb 23 00:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":2.2200000000000002},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.63600000000000001,"15min":0.60799999999999998},"min":{"1min":0.39500000000000002,"5min":0.373,"15min":0.373},"max":{"1min":4.4329999999999998,"5min":4.4329999999999998,"15min":4.4329999999999998},"last":1.292}]},{"osd":5,"last update":"Fri Feb 23 00:42:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.59799999999999998,"15min":0.60499999999999998},"min":{"1min":0.41099999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.61599999999999999,"15min":0.624},"min":{"1min":0.45800000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.84099999999999997,"5min":1.024,"15min":1.024},"last":2.2999999999999998}]},{"osd":6,"last update":"Fri Feb 23 00:42:41 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.64900000000000002,"15min":0.625},"min":{"1min":0.44400000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":4.46,"5min":4.46,"15min":4.46},"last":1.248},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63300000000000001,"15min":0.629},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.58199999999999996}]},{"osd":7,"last update":"Fri Feb 23 00:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.66800000000000004,"15min":0.64800000000000002},"min":{"1min":0.441,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":2.2589999999999999,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":2.2589999999999999},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.70799999999999996,"15min":0.67500000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":4.4850000000000003,"5min":4.4850000000000003,"15min":4.4850000000000003},"last":1.2090000000000001}]}]},{"osd":4,"up_from":32,"seq":137438953541,"num_pgs":178,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":306756,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":93438396,"statfs":{"total":95995035648,"available":95680917504,"internally_reserved":0,"allocated":1503232,"data_stored":814444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53800000000000003,"15min":0.57399999999999995},"min":{"1min":0.29999999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.57599999999999996,"15min":0.58599999999999997},"min":{"1min":0.32100000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.8660000000000001,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":0.42099999999999999}]},{"osd":1,"last update":"Fri Feb 23 00:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.52700000000000002,"15min":0.57099999999999995},"min":{"1min":0.33500000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69199999999999995,"5min":1.175,"15min":1.175},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.55900000000000005,"15min":0.57199999999999995},"min":{"1min":0.34300000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.96199999999999997,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.40600000000000003}]},{"osd":2,"last update":"Fri Feb 23 00:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.58799999999999997,"15min":0.58299999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.0790000000000002,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.56000000000000005,"15min":0.58299999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.82399999999999995,"5min":1.113,"15min":1.113},"last":0.51800000000000002}]},{"osd":3,"last update":"Fri Feb 23 00:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.59099999999999997,"15min":0.58599999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.9239999999999999,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.627,"15min":0.59999999999999998},"min":{"1min":0.36299999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":4.1909999999999998,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":0.42699999999999999}]},{"osd":5,"last update":"Fri Feb 23 00:42:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.42199999999999999,"15min":0.38},"min":{"1min":0.32500000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.47799999999999998,"15min":0.42699999999999999},"min":{"1min":0.36599999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.46999999999999997}]},{"osd":6,"last update":"Fri Feb 23 00:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.45700000000000002,"15min":0.41499999999999998},"min":{"1min":0.35499999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.45900000000000002,"15min":0.41499999999999998},"min":{"1min":0.38,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70499999999999996,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.375}]},{"osd":7,"last update":"Fri Feb 23 00:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.50900000000000001,"15min":0.49099999999999999},"min":{"1min":0.28599999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.56000000000000005,"15min":0.54300000000000004},"min":{"1min":0.40899999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.75700000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.48999999999999999}]}]},{"osd":5,"up_from":39,"seq":167503724608,"num_pgs":150,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":380516,"kb_used_data":75228,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":93364636,"statfs":{"total":95995035648,"available":95605387264,"internally_reserved":0,"allocated":77033472,"data_stored":76341684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:42:17 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.50600000000000001,"15min":0.50800000000000001},"min":{"1min":0.315,"5min":0.313,"15min":0.313},"max":{"1min":0.67800000000000005,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.252},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.51800000000000002,"15min":0.52300000000000002},"min":{"1min":0.40600000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64900000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.377}]},{"osd":1,"last update":"Fri Feb 23 00:42:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52500000000000002,"15min":0.52600000000000002},"min":{"1min":0.36499999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.64400000000000002,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.52600000000000002,"15min":0.53800000000000003},"min":{"1min":0.35199999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.61599999999999999,"5min":1.004,"15min":1.004},"last":0.44400000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:42:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54100000000000004,"15min":0.54800000000000004},"min":{"1min":0.371,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.72699999999999998,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.70999999999999996,"5min":1.026,"15min":1.026},"last":0.33100000000000002}]},{"osd":3,"last update":"Fri Feb 23 00:42:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52900000000000003,"15min":0.53100000000000003},"min":{"1min":0.39000000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.75700000000000001,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55700000000000005,"15min":0.55900000000000005},"min":{"1min":0.39300000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80000000000000004,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.433}]},{"osd":4,"last update":"Fri Feb 23 00:42:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.41799999999999998,"15min":0.39000000000000001},"min":{"1min":0.34599999999999997,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.628,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.44400000000000001,"15min":0.40000000000000002},"min":{"1min":0.39700000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.36399999999999999}]},{"osd":6,"last update":"Fri Feb 23 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.46800000000000003,"15min":0.438},"min":{"1min":0.41699999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73099999999999998,"5min":0.754,"15min":0.754},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.49099999999999999,"15min":0.45500000000000002},"min":{"1min":0.41699999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.748,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.36699999999999999}]},{"osd":7,"last update":"Fri Feb 23 00:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.59799999999999998,"15min":0.53800000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":7.0300000000000002,"5min":7.0300000000000002,"15min":7.0300000000000002},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.59799999999999998,"15min":0.53800000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":9.0579999999999998,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":0.46400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":75534336,"data_stored":75501345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":50372608,"data_stored":50331777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":25186304,"data_stored":25165866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":50364416,"data_stored":50331703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":75534336,"data_stored":75500336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":75522048,"data_stored":75497578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":100732928,"data_stored":100667276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":20480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":24576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":47,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":47,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":47,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":11,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":47,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":47,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":47,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":47,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":47,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":55,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":55,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":55,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":55,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":55,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":55,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":55,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":59,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":59,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":59,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":59,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":59,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":59,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":59,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":59,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-23T00:43:21.867 INFO:tasks.cephadm.ceph_manager.ceph:too many creating 2024-02-23T00:43:21.922 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:43:22.022 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:22.071 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:43:22.071 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:43:22.693 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:43:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:43:22.693 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:43:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:43:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:43:22.792 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:21.185778+0000 mgr.y (mgr.14517) 171 : cluster [DBG] pgmap v170: 517 pgs: 128 unknown, 32 creating+peering, 357 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:43:22.793 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:21.229726+0000 mon.a (mon.0) 1061 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:43:22.793 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:21.519377+0000 mon.b (mon.1) 120 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:22.793 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:21.539868+0000 mon.a (mon.0) 1062 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:43:22.793 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:21.539935+0000 mon.a (mon.0) 1063 : cluster [DBG] osdmap e131: 8 total, 8 up, 8 in 2024-02-23T00:43:22.793 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:21.578135+0000 mon.a (mon.0) 1064 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-7"}]: dispatch 2024-02-23T00:43:22.793 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:21.619982+0000 mon.c (mon.2) 71 : audit [INF] from='client.? 172.21.15.47:0/1135893813' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi047-82378-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:22.793 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:21.185778+0000 mgr.y (mgr.14517) 171 : cluster [DBG] pgmap v170: 517 pgs: 128 unknown, 32 creating+peering, 357 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:43:22.793 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:21.229726+0000 mon.a (mon.0) 1061 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:43:22.793 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:21.519377+0000 mon.b (mon.1) 120 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:22.793 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:21.539868+0000 mon.a (mon.0) 1062 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:43:22.794 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:21.539935+0000 mon.a (mon.0) 1063 : cluster [DBG] osdmap e131: 8 total, 8 up, 8 in 2024-02-23T00:43:22.794 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:21.578135+0000 mon.a (mon.0) 1064 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-7"}]: dispatch 2024-02-23T00:43:22.794 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:21.619982+0000 mon.c (mon.2) 71 : audit [INF] from='client.? 172.21.15.47:0/1135893813' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi047-82378-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:22.851 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:43:22.855 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:22.855 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:43:22.902 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.3 dump_ops_in_flight 2024-02-23T00:43:22.956 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:43:22.982 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:23.001 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:43:23.003 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:23.004 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:43:23.104 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:43:23.129 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:21.185778+0000 mgr.y (mgr.14517) 171 : cluster [DBG] pgmap v170: 517 pgs: 128 unknown, 32 creating+peering, 357 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:43:23.129 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:21.229726+0000 mon.a (mon.0) 1061 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:43:23.129 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:21.519377+0000 mon.b (mon.1) 120 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:23.129 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:21.539868+0000 mon.a (mon.0) 1062 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:43:23.129 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:21.539935+0000 mon.a (mon.0) 1063 : cluster [DBG] osdmap e131: 8 total, 8 up, 8 in 2024-02-23T00:43:23.129 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:21.578135+0000 mon.a (mon.0) 1064 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-7"}]: dispatch 2024-02-23T00:43:23.129 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:21.619982+0000 mon.c (mon.2) 71 : audit [INF] from='client.? 172.21.15.47:0/1135893813' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi047-82378-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:23.131 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:23.150 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:43:23.152 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:23.153 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:43:23.254 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:43:23.391 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:23.412 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: [==========] Running 4 tests from 1 test suite. 2024-02-23T00:43:23.412 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: [----------] Global test environment set-up. 2024-02-23T00:43:23.412 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2024-02-23T00:43:23.412 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2024-02-23T00:43:23.412 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (10866 ms) 2024-02-23T00:43:23.412 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2024-02-23T00:43:23.412 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (35 ms) 2024-02-23T00:43:23.412 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2024-02-23T00:43:23.412 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (20028 ms) 2024-02-23T00:43:23.412 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2024-02-23T00:43:23.413 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: attempt 0 of 20 2024-02-23T00:43:23.413 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: failed to deregister: 2024-02-23T00:43:23.413 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: { 2024-02-23T00:43:23.413 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "epoch": 20, 2024-02-23T00:43:23.413 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "modified": "2024-02-23T00:43:21.185388+0000", 2024-02-23T00:43:23.413 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "services": { 2024-02-23T00:43:23.413 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "laundry": { 2024-02-23T00:43:23.413 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "daemons": { 2024-02-23T00:43:23.413 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "summary": "", 2024-02-23T00:43:23.413 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "close-test-pid82560": { 2024-02-23T00:43:23.413 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "start_epoch": 19, 2024-02-23T00:43:23.413 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "start_stamp": "2024-02-23T00:43:18.354907+0000", 2024-02-23T00:43:23.413 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "gid": 26368, 2024-02-23T00:43:23.414 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "addr": "172.21.15.47:0/2259011143", 2024-02-23T00:43:23.414 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "metadata": { 2024-02-23T00:43:23.414 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "arch": "x86_64", 2024-02-23T00:43:23.414 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "ceph_release": "pacific", 2024-02-23T00:43:23.414 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "ceph_version": "ceph version 16.2.14-550-geb66ed92 (eb66ed921e744301e6863be6353618d63967ea59) pacific (stable)", 2024-02-23T00:43:23.414 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "ceph_version_short": "16.2.14-550-geb66ed92", 2024-02-23T00:43:23.414 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2024-02-23T00:43:23.414 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "distro": "centos", 2024-02-23T00:43:23.414 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "distro_description": "CentOS Stream 8", 2024-02-23T00:43:23.414 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "distro_version": "8", 2024-02-23T00:43:23.414 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "foo": "bar", 2024-02-23T00:43:23.414 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "hostname": "smithi047", 2024-02-23T00:43:23.414 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "kernel_description": "#1 SMP Fri Feb 9 14:58:00 UTC 2024", 2024-02-23T00:43:23.414 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "kernel_version": "4.18.0-540.el8.x86_64", 2024-02-23T00:43:23.414 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "mem_swap_kb": "0", 2024-02-23T00:43:23.415 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "mem_total_kb": "32543848", 2024-02-23T00:43:23.415 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "os": "Linux", 2024-02-23T00:43:23.415 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "this": "that" 2024-02-23T00:43:23.415 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: }, 2024-02-23T00:43:23.415 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "task_status": {} 2024-02-23T00:43:23.415 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: } 2024-02-23T00:43:23.415 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: } 2024-02-23T00:43:23.415 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: }, 2024-02-23T00:43:23.415 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "rgw": { 2024-02-23T00:43:23.415 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "daemons": { 2024-02-23T00:43:23.415 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "summary": "", 2024-02-23T00:43:23.415 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "14460": { 2024-02-23T00:43:23.415 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "start_epoch": 12, 2024-02-23T00:43:23.415 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "start_stamp": "2024-02-23T00:39:21.905359+0000", 2024-02-23T00:43:23.416 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "gid": 14460, 2024-02-23T00:43:23.416 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "addr": "172.21.15.47:0/1573276039", 2024-02-23T00:43:23.416 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "metadata": { 2024-02-23T00:43:23.416 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "arch": "x86_64", 2024-02-23T00:43:23.416 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "ceph_release": "pacific", 2024-02-23T00:43:23.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:22.315038+0000 mgr.y (mgr.14517) 172 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:43:23.909 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:43:23.916 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:23.917 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:43:23.923 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:43:24.017 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:43:24.043 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:24.062 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:43:24.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:22.541464+0000 mon.b (mon.1) 121 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:24.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:22.778118+0000 mon.a (mon.0) 1065 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi047-82378-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:24.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:22.801688+0000 mon.a (mon.0) 1066 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-7"}]': finished 2024-02-23T00:43:24.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:22.810766+0000 mon.b (mon.1) 122 : audit [INF] from='client.? 172.21.15.47:0/1191227590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi047-82386-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:24.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:23.186668+0000 mgr.y (mgr.14517) 173 : cluster [DBG] pgmap v173: 516 pgs: 64 unknown, 32 creating+peering, 1 active+clean+snaptrim, 1 peering, 418 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-23T00:43:24.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:23.393048+0000 mon.b (mon.1) 123 : audit [DBG] from='client.? 172.21.15.47:0/3676364753' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-23T00:43:24.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:23.395186+0000 mgr.y (mgr.14517) 174 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-23T00:43:24.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:23.447507+0000 mon.a (mon.0) 1067 : cluster [DBG] osdmap e132: 8 total, 8 up, 8 in 2024-02-23T00:43:24.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:23.461050+0000 mon.a (mon.0) 1068 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "27"}]: dispatch 2024-02-23T00:43:24.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:23.461829+0000 mon.a (mon.0) 1069 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi047-82386-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:24.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:23.543510+0000 mon.b (mon.1) 124 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:24.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:23.629578+0000 mon.a (mon.0) 1070 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:43:24.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:22.315038+0000 mgr.y (mgr.14517) 172 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:43:24.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:22.541464+0000 mon.b (mon.1) 121 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:24.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:22.778118+0000 mon.a (mon.0) 1065 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi047-82378-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:24.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:22.801688+0000 mon.a (mon.0) 1066 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-7"}]': finished 2024-02-23T00:43:24.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:22.810766+0000 mon.b (mon.1) 122 : audit [INF] from='client.? 172.21.15.47:0/1191227590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi047-82386-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:24.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:23.186668+0000 mgr.y (mgr.14517) 173 : cluster [DBG] pgmap v173: 516 pgs: 64 unknown, 32 creating+peering, 1 active+clean+snaptrim, 1 peering, 418 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-23T00:43:24.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:23.393048+0000 mon.b (mon.1) 123 : audit [DBG] from='client.? 172.21.15.47:0/3676364753' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-23T00:43:24.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:23.395186+0000 mgr.y (mgr.14517) 174 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-23T00:43:24.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:23.447507+0000 mon.a (mon.0) 1067 : cluster [DBG] osdmap e132: 8 total, 8 up, 8 in 2024-02-23T00:43:24.065 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:23.461050+0000 mon.a (mon.0) 1068 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "27"}]: dispatch 2024-02-23T00:43:24.065 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:23.461829+0000 mon.a (mon.0) 1069 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi047-82386-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:24.065 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:23.543510+0000 mon.b (mon.1) 124 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:24.065 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:23.629578+0000 mon.a (mon.0) 1070 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:43:24.065 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:22.315038+0000 mgr.y (mgr.14517) 172 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:43:24.065 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:22.541464+0000 mon.b (mon.1) 121 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:24.065 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:22.778118+0000 mon.a (mon.0) 1065 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi047-82378-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:24.065 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:22.801688+0000 mon.a (mon.0) 1066 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-7"}]': finished 2024-02-23T00:43:24.066 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:22.810766+0000 mon.b (mon.1) 122 : audit [INF] from='client.? 172.21.15.47:0/1191227590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi047-82386-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:24.066 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:23.186668+0000 mgr.y (mgr.14517) 173 : cluster [DBG] pgmap v173: 516 pgs: 64 unknown, 32 creating+peering, 1 active+clean+snaptrim, 1 peering, 418 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-23T00:43:24.066 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:23.393048+0000 mon.b (mon.1) 123 : audit [DBG] from='client.? 172.21.15.47:0/3676364753' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-23T00:43:24.066 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:23.395186+0000 mgr.y (mgr.14517) 174 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-23T00:43:24.066 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:23.447507+0000 mon.a (mon.0) 1067 : cluster [DBG] osdmap e132: 8 total, 8 up, 8 in 2024-02-23T00:43:24.066 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:23.461050+0000 mon.a (mon.0) 1068 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "27"}]: dispatch 2024-02-23T00:43:24.066 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:23.461829+0000 mon.a (mon.0) 1069 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi047-82386-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:24.066 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:23.543510+0000 mon.b (mon.1) 124 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:24.066 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:23.629578+0000 mon.a (mon.0) 1070 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:43:24.067 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:24.067 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:43:24.168 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:43:24.214 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:24.233 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:43:24.235 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:24.235 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:43:24.336 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:43:24.476 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:24.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.916+0000 7f677c095700 1 -- 172.21.15.47:0/4009658835 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f67740692d0 msgr2=0x7f67740696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:24.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.916+0000 7f677c095700 1 --2- 172.21.15.47:0/4009658835 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f67740692d0 0x7f67740696b0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7f67700048a0 tx=0x7f6770009cb0).stop 2024-02-23T00:43:24.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.933+0000 7f677c095700 1 -- 172.21.15.47:0/4009658835 shutdown_connections 2024-02-23T00:43:24.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.933+0000 7f677c095700 1 --2- 172.21.15.47:0/4009658835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67741044a0 0x7f6774104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:43:24.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.933+0000 7f677c095700 1 --2- 172.21.15.47:0/4009658835 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f67740692d0 0x7f67740696b0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:24.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.933+0000 7f677c095700 1 --2- 172.21.15.47:0/4009658835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6774069ba0 0x7f6774103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:24.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.933+0000 7f677c095700 1 -- 172.21.15.47:0/4009658835 >> 172.21.15.47:0/4009658835 conn(0x7f6774063db0 msgr2=0x7f6774067560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:43:24.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.934+0000 7f677c095700 1 -- 172.21.15.47:0/4009658835 shutdown_connections 2024-02-23T00:43:24.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.945+0000 7f677c095700 1 -- 172.21.15.47:0/4009658835 wait complete. 2024-02-23T00:43:24.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.945+0000 7f677c095700 1 librados: starting msgr at 2024-02-23T00:43:24.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.945+0000 7f677c095700 1 librados: starting objecter 2024-02-23T00:43:24.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.945+0000 7f677c095700 1 Processor -- start 2024-02-23T00:43:24.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.946+0000 7f677c095700 1 -- start start 2024-02-23T00:43:24.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.946+0000 7f677c095700 1 librados: setting wanted keys 2024-02-23T00:43:24.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.946+0000 7f677c095700 1 librados: calling monclient init 2024-02-23T00:43:24.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.946+0000 7f677c095700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67740692d0 0x7f677411cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:24.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.946+0000 7f677c095700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6774069ba0 0x7f677411d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:24.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.946+0000 7f677c095700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67741044a0 0x7f6774117040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:24.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.946+0000 7f677c095700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6774117530 con 0x7f67740692d0 2024-02-23T00:43:24.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.946+0000 7f677c095700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67741176a0 con 0x7f67741044a0 2024-02-23T00:43:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.946+0000 7f677c095700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6774117860 con 0x7f6774069ba0 2024-02-23T00:43:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.949+0000 7f6779630700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6774069ba0 0x7f677411d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.949+0000 7f6779630700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6774069ba0 0x7f677411d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:54314/0 (socket says 172.21.15.47:54314) 2024-02-23T00:43:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.949+0000 7f6779630700 1 -- 172.21.15.47:0/3214221007 learned_addr learned my addr 172.21.15.47:0/3214221007 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:43:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.949+0000 7f677a632700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67741044a0 0x7f6774117040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.950+0000 7f677a632700 1 -- 172.21.15.47:0/3214221007 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6774069ba0 msgr2=0x7f677411d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.950+0000 7f677a632700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6774069ba0 0x7f677411d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.950+0000 7f677a632700 1 -- 172.21.15.47:0/3214221007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67740692d0 msgr2=0x7f677411cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.950+0000 7f677a632700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67740692d0 0x7f677411cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:43:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.950+0000 7f677a632700 1 -- 172.21.15.47:0/3214221007 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6770005040 con 0x7f67741044a0 2024-02-23T00:43:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.950+0000 7f677a632700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67741044a0 0x7f6774117040 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 rx=0x7f676c00bad0 tx=0x7f676c009780).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:24.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.957+0000 7f676affd700 10 client.?.objecter ms_handle_connect 0x7f67741044a0 2024-02-23T00:43:24.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.957+0000 7f676affd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:43:24.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.957+0000 7f676affd700 1 -- 172.21.15.47:0/3214221007 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c00c840 con 0x7f67741044a0 2024-02-23T00:43:24.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.957+0000 7f676affd700 1 -- 172.21.15.47:0/3214221007 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f676c00ce30 con 0x7f67741044a0 2024-02-23T00:43:24.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.957+0000 7f676affd700 1 -- 172.21.15.47:0/3214221007 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c015b70 con 0x7f67741044a0 2024-02-23T00:43:24.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.957+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6774117b40 con 0x7f67741044a0 2024-02-23T00:43:24.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.957+0000 7f677c095700 10 client.30124.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:43:24.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.957+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6774070f40 con 0x7f67741044a0 2024-02-23T00:43:24.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.957+0000 7f677c095700 1 librados: init done 2024-02-23T00:43:24.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.962+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f6774060280 con 0x7f67741044a0 2024-02-23T00:43:24.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.967+0000 7f676affd700 1 -- 172.21.15.47:0/3214221007 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f676c012020 con 0x7f67741044a0 2024-02-23T00:43:24.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.968+0000 7f676affd700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6760062d60 0x7f6760065220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:24.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.968+0000 7f676affd700 1 -- 172.21.15.47:0/3214221007 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(134..134 src has 48..134) v4 ==== 12428+0+0 (secure 0 0 0) 0x7f676c07e350 con 0x7f67741044a0 2024-02-23T00:43:24.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.968+0000 7f676affd700 10 client.30124.objecter ms_dispatch 0x7f6774001550 osd_map(134..134 src has 48..134) v4 2024-02-23T00:43:24.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.968+0000 7f676affd700 3 client.30124.objecter handle_osd_map got epochs [134,134] > 0 2024-02-23T00:43:24.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.968+0000 7f676affd700 3 client.30124.objecter handle_osd_map decoding full epoch 134 2024-02-23T00:43:24.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.968+0000 7f676affd700 20 client.30124.objecter dump_active .. 0 homeless 2024-02-23T00:43:24.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.968+0000 7f676affd700 1 -- 172.21.15.47:0/3214221007 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v134) v1 ==== 54+0+16 (secure 0 0 0) 0x7f676c085070 con 0x7f67741044a0 2024-02-23T00:43:24.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.973+0000 7f677c095700 10 client.30124.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:24.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.973+0000 7f677c095700 15 client.30124.objecter _session_command_op_assign -1 1 2024-02-23T00:43:24.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.973+0000 7f677c095700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f6774112140 0x7f6774112520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:24.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.973+0000 7f677c095700 20 client.30124.objecter _get_session s=0x7f677405ee80 osd=0 3 2024-02-23T00:43:24.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.973+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f677405ee80 osd=0 3 2024-02-23T00:43:24.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.973+0000 7f677c095700 20 client.30124.objecter _get_session s=0x7f677405ee80 osd=0 3 2024-02-23T00:43:24.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.973+0000 7f677c095700 15 client.30124.objecter _session_command_op_remove -1 1 2024-02-23T00:43:24.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.973+0000 7f677c095700 20 client.30124.objecter get_session s=0x7f677405ee80 osd=0 3 2024-02-23T00:43:24.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.973+0000 7f677c095700 15 client.30124.objecter _session_command_op_assign 0 1 2024-02-23T00:43:24.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.973+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f677405ee80 osd=0 4 2024-02-23T00:43:24.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.973+0000 7f677c095700 10 client.30124.objecter _send_command 1 2024-02-23T00:43:24.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.973+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6774115dc0 con 0x7f6774112140 2024-02-23T00:43:24.982 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:43:24.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.975+0000 7f6779e31700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6760062d60 0x7f6760065220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:24.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.978+0000 7f6779e31700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6760062d60 0x7f6760065220 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7f6770004710 tx=0x7f6770009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:24.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.978+0000 7f676affd700 10 client.30124.objecter ms_handle_connect 0x7f6760062d60 2024-02-23T00:43:24.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.984+0000 7f6779630700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f6774112140 0x7f6774112520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:24.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.992+0000 7f6779630700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f6774112140 0x7f6774112520 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:24.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.992+0000 7f676affd700 10 client.30124.objecter ms_handle_connect 0x7f6774112140 2024-02-23T00:43:24.997 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:24.997 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:43:24.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.995+0000 7f676affd700 1 -- 172.21.15.47:0/3214221007 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f6774115dc0 con 0x7f6774112140 2024-02-23T00:43:24.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.995+0000 7f676affd700 10 client.30124.objecter ms_dispatch 0x7f6774001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:43:24.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.995+0000 7f676affd700 10 client.30124.objecter _finish_command 1 = system:0 2024-02-23T00:43:24.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.995+0000 7f676affd700 20 client.30124.objecter put_session s=0x7f677405ee80 osd=0 4 2024-02-23T00:43:24.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:24.995+0000 7f676affd700 15 client.30124.objecter _session_command_op_remove 0 1 2024-02-23T00:43:25.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.030+0000 7f677c095700 10 client.30124.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:43:25.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.030+0000 7f677c095700 15 client.30124.objecter _session_command_op_assign -1 2 2024-02-23T00:43:25.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.030+0000 7f677c095700 20 client.30124.objecter _get_session s=0x7f677405ee80 osd=0 3 2024-02-23T00:43:25.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.030+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f677405ee80 osd=0 3 2024-02-23T00:43:25.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.030+0000 7f677c095700 20 client.30124.objecter _get_session s=0x7f677405ee80 osd=0 3 2024-02-23T00:43:25.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.030+0000 7f677c095700 15 client.30124.objecter _session_command_op_remove -1 2 2024-02-23T00:43:25.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.030+0000 7f677c095700 20 client.30124.objecter get_session s=0x7f677405ee80 osd=0 3 2024-02-23T00:43:25.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.030+0000 7f677c095700 15 client.30124.objecter _session_command_op_assign 0 2 2024-02-23T00:43:25.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.030+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f677405ee80 osd=0 4 2024-02-23T00:43:25.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.030+0000 7f677c095700 10 client.30124.objecter _send_command 2 2024-02-23T00:43:25.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.030+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f67740273c0 con 0x7f6774112140 2024-02-23T00:43:25.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.040+0000 7f676affd700 1 -- 172.21.15.47:0/3214221007 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f67740273c0 con 0x7f6774112140 2024-02-23T00:43:25.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.040+0000 7f676affd700 10 client.30124.objecter ms_dispatch 0x7f6774001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:43:25.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.040+0000 7f676affd700 10 client.30124.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:43:25.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.040+0000 7f676affd700 20 client.30124.objecter put_session s=0x7f677405ee80 osd=0 4 2024-02-23T00:43:25.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.040+0000 7f676affd700 15 client.30124.objecter _session_command_op_remove 0 2 2024-02-23T00:43:25.045 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:43:25.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.041+0000 7f677c095700 10 client.30124.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:25.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.041+0000 7f677c095700 15 client.30124.objecter _session_command_op_assign -1 3 2024-02-23T00:43:25.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.041+0000 7f677c095700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f67740598f0 0x7f6774059cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:25.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.041+0000 7f677c095700 20 client.30124.objecter _get_session s=0x7f6774028710 osd=1 3 2024-02-23T00:43:25.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.041+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f6774028710 osd=1 3 2024-02-23T00:43:25.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.041+0000 7f677c095700 20 client.30124.objecter _get_session s=0x7f6774028710 osd=1 3 2024-02-23T00:43:25.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.041+0000 7f677c095700 15 client.30124.objecter _session_command_op_remove -1 3 2024-02-23T00:43:25.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.041+0000 7f677c095700 20 client.30124.objecter get_session s=0x7f6774028710 osd=1 3 2024-02-23T00:43:25.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.041+0000 7f677c095700 15 client.30124.objecter _session_command_op_assign 1 3 2024-02-23T00:43:25.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.041+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f6774028710 osd=1 4 2024-02-23T00:43:25.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.041+0000 7f677c095700 10 client.30124.objecter _send_command 3 2024-02-23T00:43:25.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.041+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f67740779a0 con 0x7f67740598f0 2024-02-23T00:43:25.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.043+0000 7f6779e31700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f67740598f0 0x7f6774059cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:25.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.043+0000 7f6779e31700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f67740598f0 0x7f6774059cf0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:25.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.046+0000 7f676affd700 10 client.30124.objecter ms_handle_connect 0x7f67740598f0 2024-02-23T00:43:25.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.046+0000 7f676affd700 1 -- 172.21.15.47:0/3214221007 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f67740779a0 con 0x7f67740598f0 2024-02-23T00:43:25.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.046+0000 7f676affd700 10 client.30124.objecter ms_dispatch 0x7f6774001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:43:25.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.046+0000 7f676affd700 10 client.30124.objecter _finish_command 3 = system:0 2024-02-23T00:43:25.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.046+0000 7f676affd700 20 client.30124.objecter put_session s=0x7f6774028710 osd=1 4 2024-02-23T00:43:25.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.046+0000 7f676affd700 15 client.30124.objecter _session_command_op_remove 1 3 2024-02-23T00:43:25.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:23.812738+0000 mon.a (mon.0) 1071 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi047-82378-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:25.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:23.812910+0000 mon.a (mon.0) 1072 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "27"}]': finished 2024-02-23T00:43:25.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:23.813016+0000 mon.a (mon.0) 1073 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi047-82386-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:25.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:23.813141+0000 mon.a (mon.0) 1074 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:43:25.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:23.813207+0000 mon.a (mon.0) 1075 : cluster [DBG] osdmap e133: 8 total, 8 up, 8 in 2024-02-23T00:43:25.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:23.849587+0000 mon.c (mon.2) 72 : audit [INF] from='client.? 172.21.15.47:0/1267702494' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:25.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:23.868564+0000 mon.a (mon.0) 1076 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-7"}]: dispatch 2024-02-23T00:43:25.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:23.934788+0000 mon.a (mon.0) 1077 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:25.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:24.552631+0000 mon.b (mon.1) 125 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:25.062 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:23.812738+0000 mon.a (mon.0) 1071 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi047-82378-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:25.062 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:23.812910+0000 mon.a (mon.0) 1072 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "27"}]': finished 2024-02-23T00:43:25.062 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:23.813016+0000 mon.a (mon.0) 1073 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi047-82386-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:25.062 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:23.813141+0000 mon.a (mon.0) 1074 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:43:25.062 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:23.813207+0000 mon.a (mon.0) 1075 : cluster [DBG] osdmap e133: 8 total, 8 up, 8 in 2024-02-23T00:43:25.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:23.849587+0000 mon.c (mon.2) 72 : audit [INF] from='client.? 172.21.15.47:0/1267702494' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:25.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:23.868564+0000 mon.a (mon.0) 1076 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-7"}]: dispatch 2024-02-23T00:43:25.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:23.934788+0000 mon.a (mon.0) 1077 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:25.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:24.552631+0000 mon.b (mon.1) 125 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:25.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.072+0000 7f677c095700 10 client.30124.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:43:25.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.072+0000 7f677c095700 15 client.30124.objecter _session_command_op_assign -1 4 2024-02-23T00:43:25.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.072+0000 7f677c095700 20 client.30124.objecter _get_session s=0x7f6774028710 osd=1 3 2024-02-23T00:43:25.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.072+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f6774028710 osd=1 3 2024-02-23T00:43:25.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.072+0000 7f677c095700 20 client.30124.objecter _get_session s=0x7f6774028710 osd=1 3 2024-02-23T00:43:25.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.072+0000 7f677c095700 15 client.30124.objecter _session_command_op_remove -1 4 2024-02-23T00:43:25.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.072+0000 7f677c095700 20 client.30124.objecter get_session s=0x7f6774028710 osd=1 3 2024-02-23T00:43:25.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.072+0000 7f677c095700 15 client.30124.objecter _session_command_op_assign 1 4 2024-02-23T00:43:25.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.072+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f6774028710 osd=1 4 2024-02-23T00:43:25.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.072+0000 7f677c095700 10 client.30124.objecter _send_command 4 2024-02-23T00:43:25.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.072+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f6774059360 con 0x7f67740598f0 2024-02-23T00:43:25.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.075+0000 7f676affd700 1 -- 172.21.15.47:0/3214221007 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f6774059360 con 0x7f67740598f0 2024-02-23T00:43:25.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.075+0000 7f676affd700 10 client.30124.objecter ms_dispatch 0x7f6774001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:43:25.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.075+0000 7f676affd700 10 client.30124.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:43:25.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.075+0000 7f676affd700 20 client.30124.objecter put_session s=0x7f6774028710 osd=1 4 2024-02-23T00:43:25.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.075+0000 7f676affd700 15 client.30124.objecter _session_command_op_remove 1 4 2024-02-23T00:43:25.080 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:43:25.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.075+0000 7f677c095700 10 client.30124.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:25.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.075+0000 7f677c095700 15 client.30124.objecter _session_command_op_assign -1 5 2024-02-23T00:43:25.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.075+0000 7f677c095700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f6774028270 0x7f67740763c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:25.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.075+0000 7f677c095700 20 client.30124.objecter _get_session s=0x7f6774059770 osd=2 3 2024-02-23T00:43:25.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.075+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f6774059770 osd=2 3 2024-02-23T00:43:25.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.075+0000 7f677c095700 20 client.30124.objecter _get_session s=0x7f6774059770 osd=2 3 2024-02-23T00:43:25.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.075+0000 7f677c095700 15 client.30124.objecter _session_command_op_remove -1 5 2024-02-23T00:43:25.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.075+0000 7f677c095700 20 client.30124.objecter get_session s=0x7f6774059770 osd=2 3 2024-02-23T00:43:25.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.075+0000 7f677c095700 15 client.30124.objecter _session_command_op_assign 2 5 2024-02-23T00:43:25.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.075+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f6774059770 osd=2 4 2024-02-23T00:43:25.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.075+0000 7f677c095700 10 client.30124.objecter _send_command 5 2024-02-23T00:43:25.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.075+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f677407b130 con 0x7f6774028270 2024-02-23T00:43:25.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.076+0000 7f6779630700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f6774028270 0x7f67740763c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:25.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.076+0000 7f6779630700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f6774028270 0x7f67740763c0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:25.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.077+0000 7f676affd700 10 client.30124.objecter ms_handle_connect 0x7f6774028270 2024-02-23T00:43:25.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.084+0000 7f676affd700 1 -- 172.21.15.47:0/3214221007 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f677407b130 con 0x7f6774028270 2024-02-23T00:43:25.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.084+0000 7f676affd700 10 client.30124.objecter ms_dispatch 0x7f6774001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:43:25.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.084+0000 7f676affd700 10 client.30124.objecter _finish_command 5 = system:0 2024-02-23T00:43:25.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.084+0000 7f676affd700 20 client.30124.objecter put_session s=0x7f6774059770 osd=2 4 2024-02-23T00:43:25.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.084+0000 7f676affd700 15 client.30124.objecter _session_command_op_remove 2 5 2024-02-23T00:43:25.098 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:43:25.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.119+0000 7f677c095700 10 client.30124.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:43:25.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.119+0000 7f677c095700 15 client.30124.objecter _session_command_op_assign -1 6 2024-02-23T00:43:25.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.119+0000 7f677c095700 20 client.30124.objecter _get_session s=0x7f6774059770 osd=2 3 2024-02-23T00:43:25.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.119+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f6774059770 osd=2 3 2024-02-23T00:43:25.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.119+0000 7f677c095700 20 client.30124.objecter _get_session s=0x7f6774059770 osd=2 3 2024-02-23T00:43:25.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.119+0000 7f677c095700 15 client.30124.objecter _session_command_op_remove -1 6 2024-02-23T00:43:25.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.119+0000 7f677c095700 20 client.30124.objecter get_session s=0x7f6774059770 osd=2 3 2024-02-23T00:43:25.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.119+0000 7f677c095700 15 client.30124.objecter _session_command_op_assign 2 6 2024-02-23T00:43:25.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.119+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f6774059770 osd=2 4 2024-02-23T00:43:25.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.119+0000 7f677c095700 10 client.30124.objecter _send_command 6 2024-02-23T00:43:25.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.119+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f677407bad0 con 0x7f6774028270 2024-02-23T00:43:25.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.122+0000 7f676affd700 1 -- 172.21.15.47:0/3214221007 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f677407bad0 con 0x7f6774028270 2024-02-23T00:43:25.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.122+0000 7f676affd700 10 client.30124.objecter ms_dispatch 0x7f6774001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:43:25.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.122+0000 7f676affd700 10 client.30124.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:43:25.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.122+0000 7f676affd700 20 client.30124.objecter put_session s=0x7f6774059770 osd=2 4 2024-02-23T00:43:25.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.122+0000 7f676affd700 15 client.30124.objecter _session_command_op_remove 2 6 2024-02-23T00:43:25.134 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:43:25.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.139+0000 7f677c095700 10 client.30124.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.139+0000 7f677c095700 15 client.30124.objecter _session_command_op_assign -1 7 2024-02-23T00:43:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.139+0000 7f677c095700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f6774027940 0x7f6774076dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.139+0000 7f677c095700 20 client.30124.objecter _get_session s=0x7f67740277d0 osd=3 3 2024-02-23T00:43:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.139+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f67740277d0 osd=3 3 2024-02-23T00:43:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.139+0000 7f677c095700 20 client.30124.objecter _get_session s=0x7f67740277d0 osd=3 3 2024-02-23T00:43:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.139+0000 7f677c095700 15 client.30124.objecter _session_command_op_remove -1 7 2024-02-23T00:43:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.139+0000 7f677c095700 20 client.30124.objecter get_session s=0x7f67740277d0 osd=3 3 2024-02-23T00:43:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.139+0000 7f677c095700 15 client.30124.objecter _session_command_op_assign 3 7 2024-02-23T00:43:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.139+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f67740277d0 osd=3 4 2024-02-23T00:43:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.139+0000 7f677c095700 10 client.30124.objecter _send_command 7 2024-02-23T00:43:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.139+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f67740773c0 con 0x7f6774027940 2024-02-23T00:43:25.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.140+0000 7f677a632700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f6774027940 0x7f6774076dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:25.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.141+0000 7f677a632700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f6774027940 0x7f6774076dc0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:25.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.144+0000 7f676affd700 10 client.30124.objecter ms_handle_connect 0x7f6774027940 2024-02-23T00:43:25.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.144+0000 7f676affd700 1 -- 172.21.15.47:0/3214221007 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f67740773c0 con 0x7f6774027940 2024-02-23T00:43:25.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.144+0000 7f676affd700 10 client.30124.objecter ms_dispatch 0x7f6774001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:43:25.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.144+0000 7f676affd700 10 client.30124.objecter _finish_command 7 = system:0 2024-02-23T00:43:25.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.144+0000 7f676affd700 20 client.30124.objecter put_session s=0x7f67740277d0 osd=3 4 2024-02-23T00:43:25.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.144+0000 7f676affd700 15 client.30124.objecter _session_command_op_remove 3 7 2024-02-23T00:43:25.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.176+0000 7f677c095700 10 client.30124.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:43:25.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.176+0000 7f677c095700 15 client.30124.objecter _session_command_op_assign -1 8 2024-02-23T00:43:25.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.176+0000 7f677c095700 20 client.30124.objecter _get_session s=0x7f67740277d0 osd=3 3 2024-02-23T00:43:25.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.176+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f67740277d0 osd=3 3 2024-02-23T00:43:25.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.176+0000 7f677c095700 20 client.30124.objecter _get_session s=0x7f67740277d0 osd=3 3 2024-02-23T00:43:25.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.176+0000 7f677c095700 15 client.30124.objecter _session_command_op_remove -1 8 2024-02-23T00:43:25.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.176+0000 7f677c095700 20 client.30124.objecter get_session s=0x7f67740277d0 osd=3 3 2024-02-23T00:43:25.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.176+0000 7f677c095700 15 client.30124.objecter _session_command_op_assign 3 8 2024-02-23T00:43:25.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.176+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f67740277d0 osd=3 4 2024-02-23T00:43:25.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.176+0000 7f677c095700 10 client.30124.objecter _send_command 8 2024-02-23T00:43:25.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.176+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f67740777a0 con 0x7f6774027940 2024-02-23T00:43:25.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:23.812738+0000 mon.a (mon.0) 1071 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi047-82378-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:25.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:23.812910+0000 mon.a (mon.0) 1072 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "27"}]': finished 2024-02-23T00:43:25.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:23.813016+0000 mon.a (mon.0) 1073 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi047-82386-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:25.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:23.813141+0000 mon.a (mon.0) 1074 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:43:25.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:23.813207+0000 mon.a (mon.0) 1075 : cluster [DBG] osdmap e133: 8 total, 8 up, 8 in 2024-02-23T00:43:25.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:23.849587+0000 mon.c (mon.2) 72 : audit [INF] from='client.? 172.21.15.47:0/1267702494' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:25.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:23.868564+0000 mon.a (mon.0) 1076 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-7"}]: dispatch 2024-02-23T00:43:25.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:23.934788+0000 mon.a (mon.0) 1077 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:25.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:24.552631+0000 mon.b (mon.1) 125 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:25.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.191+0000 7f676affd700 1 -- 172.21.15.47:0/3214221007 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f67740777a0 con 0x7f6774027940 2024-02-23T00:43:25.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.191+0000 7f676affd700 10 client.30124.objecter ms_dispatch 0x7f6774001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:43:25.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.191+0000 7f676affd700 10 client.30124.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:43:25.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.191+0000 7f676affd700 20 client.30124.objecter put_session s=0x7f67740277d0 osd=3 4 2024-02-23T00:43:25.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.191+0000 7f676affd700 15 client.30124.objecter _session_command_op_remove 3 8 2024-02-23T00:43:25.204 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:43:25.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.204+0000 7f677c095700 10 client.30124.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:25.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.204+0000 7f677c095700 15 client.30124.objecter _session_command_op_assign -1 9 2024-02-23T00:43:25.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.204+0000 7f677c095700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f677407a0b0 0x7f677407a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:25.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.204+0000 7f677c095700 20 client.30124.objecter _get_session s=0x7f67740777a0 osd=4 3 2024-02-23T00:43:25.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.204+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f67740777a0 osd=4 3 2024-02-23T00:43:25.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.204+0000 7f677c095700 20 client.30124.objecter _get_session s=0x7f67740777a0 osd=4 3 2024-02-23T00:43:25.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.204+0000 7f677c095700 15 client.30124.objecter _session_command_op_remove -1 9 2024-02-23T00:43:25.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.204+0000 7f677c095700 20 client.30124.objecter get_session s=0x7f67740777a0 osd=4 3 2024-02-23T00:43:25.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.204+0000 7f677c095700 15 client.30124.objecter _session_command_op_assign 4 9 2024-02-23T00:43:25.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.204+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f67740777a0 osd=4 4 2024-02-23T00:43:25.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.204+0000 7f677c095700 10 client.30124.objecter _send_command 9 2024-02-23T00:43:25.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.204+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f677407abd0 con 0x7f677407a0b0 2024-02-23T00:43:25.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.209+0000 7f6779e31700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f677407a0b0 0x7f677407a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:25.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.224+0000 7f6779e31700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f677407a0b0 0x7f677407a4f0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:25.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.225+0000 7f676affd700 10 client.30124.objecter ms_handle_connect 0x7f677407a0b0 2024-02-23T00:43:25.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.226+0000 7f676affd700 1 -- 172.21.15.47:0/3214221007 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f677407abd0 con 0x7f677407a0b0 2024-02-23T00:43:25.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.226+0000 7f676affd700 10 client.30124.objecter ms_dispatch 0x7f6774001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:43:25.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.226+0000 7f676affd700 10 client.30124.objecter _finish_command 9 = system:0 2024-02-23T00:43:25.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.226+0000 7f676affd700 20 client.30124.objecter put_session s=0x7f67740777a0 osd=4 4 2024-02-23T00:43:25.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.226+0000 7f676affd700 15 client.30124.objecter _session_command_op_remove 4 9 2024-02-23T00:43:25.257 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:25.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.265+0000 7f677c095700 10 client.30124.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:43:25.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.265+0000 7f677c095700 15 client.30124.objecter _session_command_op_assign -1 10 2024-02-23T00:43:25.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.265+0000 7f677c095700 20 client.30124.objecter _get_session s=0x7f67740777a0 osd=4 3 2024-02-23T00:43:25.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.265+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f67740777a0 osd=4 3 2024-02-23T00:43:25.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.265+0000 7f677c095700 20 client.30124.objecter _get_session s=0x7f67740777a0 osd=4 3 2024-02-23T00:43:25.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.265+0000 7f677c095700 15 client.30124.objecter _session_command_op_remove -1 10 2024-02-23T00:43:25.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.265+0000 7f677c095700 20 client.30124.objecter get_session s=0x7f67740777a0 osd=4 3 2024-02-23T00:43:25.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.265+0000 7f677c095700 15 client.30124.objecter _session_command_op_assign 4 10 2024-02-23T00:43:25.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.265+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f67740777a0 osd=4 4 2024-02-23T00:43:25.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.265+0000 7f677c095700 10 client.30124.objecter _send_command 10 2024-02-23T00:43:25.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.265+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f677407abd0 con 0x7f677407a0b0 2024-02-23T00:43:25.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.286+0000 7f676affd700 1 -- 172.21.15.47:0/3214221007 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f677407abd0 con 0x7f677407a0b0 2024-02-23T00:43:25.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.286+0000 7f676affd700 10 client.30124.objecter ms_dispatch 0x7f6774001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:43:25.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.286+0000 7f676affd700 10 client.30124.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:43:25.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.286+0000 7f676affd700 20 client.30124.objecter put_session s=0x7f67740777a0 osd=4 4 2024-02-23T00:43:25.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.286+0000 7f676affd700 15 client.30124.objecter _session_command_op_remove 4 10 2024-02-23T00:43:25.299 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:43:25.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.304+0000 7f677c095700 10 client.30124.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:25.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.304+0000 7f677c095700 15 client.30124.objecter _session_command_op_assign -1 11 2024-02-23T00:43:25.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.304+0000 7f677c095700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f67741441e0 0x7f67741465d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:25.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.304+0000 7f677c095700 20 client.30124.objecter _get_session s=0x7f677407abd0 osd=5 3 2024-02-23T00:43:25.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.304+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f677407abd0 osd=5 3 2024-02-23T00:43:25.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.304+0000 7f677c095700 20 client.30124.objecter _get_session s=0x7f677407abd0 osd=5 3 2024-02-23T00:43:25.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.304+0000 7f677c095700 15 client.30124.objecter _session_command_op_remove -1 11 2024-02-23T00:43:25.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.304+0000 7f677c095700 20 client.30124.objecter get_session s=0x7f677407abd0 osd=5 3 2024-02-23T00:43:25.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.304+0000 7f677c095700 15 client.30124.objecter _session_command_op_assign 5 11 2024-02-23T00:43:25.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.304+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f677407abd0 osd=5 4 2024-02-23T00:43:25.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.304+0000 7f677c095700 10 client.30124.objecter _send_command 11 2024-02-23T00:43:25.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.304+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6774149220 con 0x7f67741441e0 2024-02-23T00:43:25.316 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:43:25.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.315+0000 7f6779630700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f67741441e0 0x7f67741465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:25.320 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:25.321 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:43:25.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.326+0000 7f6779630700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f67741441e0 0x7f67741465d0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:25.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.327+0000 7f676affd700 10 client.30124.objecter ms_handle_connect 0x7f67741441e0 2024-02-23T00:43:25.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.337+0000 7f676affd700 1 -- 172.21.15.47:0/3214221007 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f6774149220 con 0x7f67741441e0 2024-02-23T00:43:25.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.337+0000 7f676affd700 10 client.30124.objecter ms_dispatch 0x7f6774001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:43:25.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.337+0000 7f676affd700 10 client.30124.objecter _finish_command 11 = system:0 2024-02-23T00:43:25.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.337+0000 7f676affd700 20 client.30124.objecter put_session s=0x7f677407abd0 osd=5 4 2024-02-23T00:43:25.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.337+0000 7f676affd700 15 client.30124.objecter _session_command_op_remove 5 11 2024-02-23T00:43:25.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.364+0000 7f677c095700 10 client.30124.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:43:25.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.364+0000 7f677c095700 15 client.30124.objecter _session_command_op_assign -1 12 2024-02-23T00:43:25.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.364+0000 7f677c095700 20 client.30124.objecter _get_session s=0x7f677407abd0 osd=5 3 2024-02-23T00:43:25.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.364+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f677407abd0 osd=5 3 2024-02-23T00:43:25.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.364+0000 7f677c095700 20 client.30124.objecter _get_session s=0x7f677407abd0 osd=5 3 2024-02-23T00:43:25.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.364+0000 7f677c095700 15 client.30124.objecter _session_command_op_remove -1 12 2024-02-23T00:43:25.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.364+0000 7f677c095700 20 client.30124.objecter get_session s=0x7f677407abd0 osd=5 3 2024-02-23T00:43:25.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.364+0000 7f677c095700 15 client.30124.objecter _session_command_op_assign 5 12 2024-02-23T00:43:25.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.364+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f677407abd0 osd=5 4 2024-02-23T00:43:25.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.364+0000 7f677c095700 10 client.30124.objecter _send_command 12 2024-02-23T00:43:25.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.364+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f6774146ac0 con 0x7f67741441e0 2024-02-23T00:43:25.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.369+0000 7f676affd700 1 -- 172.21.15.47:0/3214221007 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f6774146ac0 con 0x7f67741441e0 2024-02-23T00:43:25.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.369+0000 7f676affd700 10 client.30124.objecter ms_dispatch 0x7f6774001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:43:25.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.369+0000 7f676affd700 10 client.30124.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:43:25.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.369+0000 7f676affd700 20 client.30124.objecter put_session s=0x7f677407abd0 osd=5 4 2024-02-23T00:43:25.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.369+0000 7f676affd700 15 client.30124.objecter _session_command_op_remove 5 12 2024-02-23T00:43:25.372 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:43:25.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.369+0000 7f677c095700 10 client.30124.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:25.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.369+0000 7f677c095700 15 client.30124.objecter _session_command_op_assign -1 13 2024-02-23T00:43:25.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.369+0000 7f677c095700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f6774147140 0x7f6774149570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:25.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.369+0000 7f677c095700 20 client.30124.objecter _get_session s=0x7f6774146ac0 osd=6 3 2024-02-23T00:43:25.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.369+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f6774146ac0 osd=6 3 2024-02-23T00:43:25.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.369+0000 7f677c095700 20 client.30124.objecter _get_session s=0x7f6774146ac0 osd=6 3 2024-02-23T00:43:25.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.369+0000 7f677c095700 15 client.30124.objecter _session_command_op_remove -1 13 2024-02-23T00:43:25.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.369+0000 7f677c095700 20 client.30124.objecter get_session s=0x7f6774146ac0 osd=6 3 2024-02-23T00:43:25.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.369+0000 7f677c095700 15 client.30124.objecter _session_command_op_assign 6 13 2024-02-23T00:43:25.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.369+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f6774146ac0 osd=6 4 2024-02-23T00:43:25.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.369+0000 7f677c095700 10 client.30124.objecter _send_command 13 2024-02-23T00:43:25.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.369+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f677414c3b0 con 0x7f6774147140 2024-02-23T00:43:25.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.370+0000 7f677a632700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f6774147140 0x7f6774149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:25.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.370+0000 7f677a632700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f6774147140 0x7f6774149570 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:25.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.370+0000 7f676affd700 10 client.30124.objecter ms_handle_connect 0x7f6774147140 2024-02-23T00:43:25.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.370+0000 7f676affd700 1 -- 172.21.15.47:0/3214221007 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f677414c3b0 con 0x7f6774147140 2024-02-23T00:43:25.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.371+0000 7f676affd700 10 client.30124.objecter ms_dispatch 0x7f6774001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:43:25.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.371+0000 7f676affd700 10 client.30124.objecter _finish_command 13 = system:0 2024-02-23T00:43:25.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.371+0000 7f676affd700 20 client.30124.objecter put_session s=0x7f6774146ac0 osd=6 4 2024-02-23T00:43:25.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.371+0000 7f676affd700 15 client.30124.objecter _session_command_op_remove 6 13 2024-02-23T00:43:25.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.393+0000 7f677c095700 10 client.30124.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:43:25.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.394+0000 7f677c095700 15 client.30124.objecter _session_command_op_assign -1 14 2024-02-23T00:43:25.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.394+0000 7f677c095700 20 client.30124.objecter _get_session s=0x7f6774146ac0 osd=6 3 2024-02-23T00:43:25.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.394+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f6774146ac0 osd=6 3 2024-02-23T00:43:25.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.394+0000 7f677c095700 20 client.30124.objecter _get_session s=0x7f6774146ac0 osd=6 3 2024-02-23T00:43:25.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.394+0000 7f677c095700 15 client.30124.objecter _session_command_op_remove -1 14 2024-02-23T00:43:25.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.394+0000 7f677c095700 20 client.30124.objecter get_session s=0x7f6774146ac0 osd=6 3 2024-02-23T00:43:25.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.394+0000 7f677c095700 15 client.30124.objecter _session_command_op_assign 6 14 2024-02-23T00:43:25.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.394+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f6774146ac0 osd=6 4 2024-02-23T00:43:25.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.394+0000 7f677c095700 10 client.30124.objecter _send_command 14 2024-02-23T00:43:25.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.394+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f677414c3b0 con 0x7f6774147140 2024-02-23T00:43:25.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.396+0000 7f676affd700 1 -- 172.21.15.47:0/3214221007 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f677414c3b0 con 0x7f6774147140 2024-02-23T00:43:25.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.396+0000 7f676affd700 10 client.30124.objecter ms_dispatch 0x7f6774001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:43:25.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.396+0000 7f676affd700 10 client.30124.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:43:25.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.396+0000 7f676affd700 20 client.30124.objecter put_session s=0x7f6774146ac0 osd=6 4 2024-02-23T00:43:25.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.396+0000 7f676affd700 15 client.30124.objecter _session_command_op_remove 6 14 2024-02-23T00:43:25.404 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:43:25.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.403+0000 7f677c095700 10 client.30124.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:25.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.403+0000 7f677c095700 15 client.30124.objecter _session_command_op_assign -1 15 2024-02-23T00:43:25.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.403+0000 7f677c095700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f677414b720 0x7f677414bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:25.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.403+0000 7f677c095700 20 client.30124.objecter _get_session s=0x7f677414c3b0 osd=7 3 2024-02-23T00:43:25.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.403+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f677414c3b0 osd=7 3 2024-02-23T00:43:25.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.403+0000 7f677c095700 20 client.30124.objecter _get_session s=0x7f677414c3b0 osd=7 3 2024-02-23T00:43:25.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.403+0000 7f677c095700 15 client.30124.objecter _session_command_op_remove -1 15 2024-02-23T00:43:25.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.403+0000 7f677c095700 20 client.30124.objecter get_session s=0x7f677414c3b0 osd=7 3 2024-02-23T00:43:25.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.403+0000 7f677c095700 15 client.30124.objecter _session_command_op_assign 7 15 2024-02-23T00:43:25.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.403+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f677414c3b0 osd=7 4 2024-02-23T00:43:25.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.403+0000 7f677c095700 10 client.30124.objecter _send_command 15 2024-02-23T00:43:25.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.403+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f677414f6e0 con 0x7f677414b720 2024-02-23T00:43:25.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.406+0000 7f6779e31700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f677414b720 0x7f677414bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:25.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.406+0000 7f6779e31700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f677414b720 0x7f677414bb80 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:25.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.406+0000 7f676affd700 10 client.30124.objecter ms_handle_connect 0x7f677414b720 2024-02-23T00:43:25.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.407+0000 7f676affd700 1 -- 172.21.15.47:0/3214221007 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f677414f6e0 con 0x7f677414b720 2024-02-23T00:43:25.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.407+0000 7f676affd700 10 client.30124.objecter ms_dispatch 0x7f6774001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:43:25.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.407+0000 7f676affd700 10 client.30124.objecter _finish_command 15 = system:0 2024-02-23T00:43:25.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.407+0000 7f676affd700 20 client.30124.objecter put_session s=0x7f677414c3b0 osd=7 4 2024-02-23T00:43:25.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.407+0000 7f676affd700 15 client.30124.objecter _session_command_op_remove 7 15 2024-02-23T00:43:25.422 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:43:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.430+0000 7f677c095700 10 client.30124.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:43:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.430+0000 7f677c095700 15 client.30124.objecter _session_command_op_assign -1 16 2024-02-23T00:43:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.430+0000 7f677c095700 20 client.30124.objecter _get_session s=0x7f677414c3b0 osd=7 3 2024-02-23T00:43:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.430+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f677414c3b0 osd=7 3 2024-02-23T00:43:25.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.430+0000 7f677c095700 20 client.30124.objecter _get_session s=0x7f677414c3b0 osd=7 3 2024-02-23T00:43:25.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.430+0000 7f677c095700 15 client.30124.objecter _session_command_op_remove -1 16 2024-02-23T00:43:25.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.430+0000 7f677c095700 20 client.30124.objecter get_session s=0x7f677414c3b0 osd=7 3 2024-02-23T00:43:25.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.430+0000 7f677c095700 15 client.30124.objecter _session_command_op_assign 7 16 2024-02-23T00:43:25.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.430+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f677414c3b0 osd=7 4 2024-02-23T00:43:25.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.430+0000 7f677c095700 10 client.30124.objecter _send_command 16 2024-02-23T00:43:25.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.430+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f6774149b70 con 0x7f677414b720 2024-02-23T00:43:25.438 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:43:25.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.435+0000 7f676affd700 1 -- 172.21.15.47:0/3214221007 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f6774149b70 con 0x7f677414b720 2024-02-23T00:43:25.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.435+0000 7f676affd700 10 client.30124.objecter ms_dispatch 0x7f6774001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:43:25.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.435+0000 7f676affd700 10 client.30124.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:43:25.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.435+0000 7f676affd700 20 client.30124.objecter put_session s=0x7f677414c3b0 osd=7 4 2024-02-23T00:43:25.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.435+0000 7f676affd700 15 client.30124.objecter _session_command_op_remove 7 16 2024-02-23T00:43:25.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 10 librados: watch_flush enter 2024-02-23T00:43:25.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 10 librados: watch_flush exit 2024-02-23T00:43:25.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 10 client.30124.objecter close_session for osd.0 2024-02-23T00:43:25.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f6774112140 msgr2=0x7f6774112520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:25.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f6774112140 0x7f6774112520 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:25.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f677405ee80 osd=0 1 2024-02-23T00:43:25.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 10 client.30124.objecter close_session for osd.1 2024-02-23T00:43:25.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f67740598f0 msgr2=0x7f6774059cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:25.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f67740598f0 0x7f6774059cf0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:25.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f6774028710 osd=1 1 2024-02-23T00:43:25.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 10 client.30124.objecter close_session for osd.2 2024-02-23T00:43:25.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f6774028270 msgr2=0x7f67740763c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:25.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f6774028270 0x7f67740763c0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:25.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f6774059770 osd=2 1 2024-02-23T00:43:25.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 10 client.30124.objecter close_session for osd.3 2024-02-23T00:43:25.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f6774027940 msgr2=0x7f6774076dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:25.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f6774027940 0x7f6774076dc0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:25.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f67740277d0 osd=3 1 2024-02-23T00:43:25.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 10 client.30124.objecter close_session for osd.4 2024-02-23T00:43:25.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f677407a0b0 msgr2=0x7f677407a4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:25.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f677407a0b0 0x7f677407a4f0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:25.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f67740777a0 osd=4 1 2024-02-23T00:43:25.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 10 client.30124.objecter close_session for osd.5 2024-02-23T00:43:25.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f67741441e0 msgr2=0x7f67741465d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:25.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f67741441e0 0x7f67741465d0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:25.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f677407abd0 osd=5 1 2024-02-23T00:43:25.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 10 client.30124.objecter close_session for osd.6 2024-02-23T00:43:25.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f6774147140 msgr2=0x7f6774149570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:25.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f6774147140 0x7f6774149570 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:25.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f6774146ac0 osd=6 1 2024-02-23T00:43:25.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 10 client.30124.objecter close_session for osd.7 2024-02-23T00:43:25.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f677414b720 msgr2=0x7f677414bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:25.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f677414b720 0x7f677414bb80 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:25.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 20 client.30124.objecter put_session s=0x7f677414c3b0 osd=7 1 2024-02-23T00:43:25.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 20 client.30124.objecter shutdown clearing up homeless session... 2024-02-23T00:43:25.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 10 client.30124.objecter successfully canceled tick 2024-02-23T00:43:25.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677a632700 1 -- 172.21.15.47:0/3214221007 reap_dead start 2024-02-23T00:43:25.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677a632700 1 -- 172.21.15.47:0/3214221007 reap_dead start 2024-02-23T00:43:25.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677a632700 1 -- 172.21.15.47:0/3214221007 reap_dead start 2024-02-23T00:43:25.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677a632700 1 -- 172.21.15.47:0/3214221007 reap_dead start 2024-02-23T00:43:25.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677a632700 1 -- 172.21.15.47:0/3214221007 reap_dead start 2024-02-23T00:43:25.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6760062d60 msgr2=0x7f6760065220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:25.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6760062d60 0x7f6760065220 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7f6770004710 tx=0x7f6770009cb0).stop 2024-02-23T00:43:25.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67741044a0 msgr2=0x7f6774117040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:25.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.436+0000 7f677c095700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67741044a0 0x7f6774117040 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 rx=0x7f676c00bad0 tx=0x7f676c009780).stop 2024-02-23T00:43:25.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.437+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 shutdown_connections 2024-02-23T00:43:25.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.437+0000 7f677c095700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6760062d60 0x7f6760065220 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:25.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.437+0000 7f677c095700 1 --2- 172.21.15.47:0/3214221007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67741044a0 0x7f6774117040 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:25.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.437+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 >> 172.21.15.47:0/3214221007 conn(0x7f6774063db0 msgr2=0x7f6774105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:43:25.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.441+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 shutdown_connections 2024-02-23T00:43:25.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.441+0000 7f677c095700 1 -- 172.21.15.47:0/3214221007 wait complete. 2024-02-23T00:43:25.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:25.441+0000 7f677c095700 1 librados: shutdown 2024-02-23T00:43:25.448 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:43:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:43:25.452 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:43:25.452 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:43:25.452 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:43:25.452 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:43:25.452 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:43:25.452 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:43:25.467 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:43:25.470 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:25.470 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:43:25.571 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:43:25.653 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:25.956 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:43:25.981 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:43:25.997 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:25.998 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:43:26.099 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:43:26.124 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:26.143 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:43:26.146 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:26.146 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:43:26.235 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "ceph_version": "ceph version 16.2.14-550-geb66ed92 (eb66ed921e744301e6863be6353618d63967ea5 api_c_write_operations: Running main() from gmock_main.cc 2024-02-23T00:43:26.235 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2024-02-23T00:43:26.235 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_write_operations: [----------] Global test environment set-up. 2024-02-23T00:43:26.235 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2024-02-23T00:43:26.235 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2024-02-23T00:43:26.235 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-02-23T00:43:26.235 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2024-02-23T00:43:26.235 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_write_operations: 2024-02-23T00:43:26.235 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2024-02-23T00:43:26.235 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2024-02-23T00:43:26.235 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (12535 ms) 2024-02-23T00:43:26.236 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-02-23T00:43:26.236 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (6804 ms) 2024-02-23T00:43:26.236 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2024-02-23T00:43:26.236 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (4299 ms) 2024-02-23T00:43:26.236 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2024-02-23T00:43:26.236 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (3350 ms) 2024-02-23T00:43:26.236 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2024-02-23T00:43:26.236 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (3583 ms) 2024-02-23T00:43:26.236 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2024-02-23T00:43:26.236 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (3774 ms) 2024-02-23T00:43:26.236 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2024-02-23T00:43:26.236 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (4671 ms) 2024-02-23T00:43:26.236 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (39017 ms total) 2024-02-23T00:43:26.237 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_write_operations: 2024-02-23T00:43:26.237 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_write_operations: [----------] Global test environment tear-down 2024-02-23T00:43:26.237 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (39017 ms total) 2024-02-23T00:43:26.237 INFO:tasks.workunit.client.0.smithi047.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2024-02-23T00:43:26.247 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:43:26.274 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:24.822066+0000 mon.a (mon.0) 1078 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-7"}]': finished 2024-02-23T00:43:26.274 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:24.822283+0000 mon.a (mon.0) 1079 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:26.274 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:24.822420+0000 mon.a (mon.0) 1080 : cluster [DBG] osdmap e134: 8 total, 8 up, 8 in 2024-02-23T00:43:26.274 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:24.847332+0000 mon.b (mon.1) 126 : audit [INF] from='client.? 172.21.15.47:0/3883540601' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:26.274 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:24.931873+0000 mon.a (mon.0) 1081 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:26.274 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:24.969147+0000 mon.b (mon.1) 127 : audit [DBG] from='client.? 172.21.15.47:0/3214221007' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:43:26.274 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:25.192636+0000 mgr.y (mgr.14517) 175 : cluster [DBG] pgmap v176: 484 pgs: 64 unknown, 1 active+clean+snaptrim, 1 peering, 418 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-02-23T00:43:26.274 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:25.564541+0000 mon.b (mon.1) 128 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:26.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:24.822066+0000 mon.a (mon.0) 1078 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-7"}]': finished 2024-02-23T00:43:26.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:24.822283+0000 mon.a (mon.0) 1079 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:26.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:24.822420+0000 mon.a (mon.0) 1080 : cluster [DBG] osdmap e134: 8 total, 8 up, 8 in 2024-02-23T00:43:26.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:24.847332+0000 mon.b (mon.1) 126 : audit [INF] from='client.? 172.21.15.47:0/3883540601' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:26.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:24.931873+0000 mon.a (mon.0) 1081 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:26.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:24.969147+0000 mon.b (mon.1) 127 : audit [DBG] from='client.? 172.21.15.47:0/3214221007' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:43:26.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:25.192636+0000 mgr.y (mgr.14517) 175 : cluster [DBG] pgmap v176: 484 pgs: 64 unknown, 1 active+clean+snaptrim, 1 peering, 418 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-02-23T00:43:26.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:25.564541+0000 mon.b (mon.1) 128 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:26.338 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:26.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:24.822066+0000 mon.a (mon.0) 1078 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-7"}]': finished 2024-02-23T00:43:26.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:24.822283+0000 mon.a (mon.0) 1079 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82568-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:26.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:24.822420+0000 mon.a (mon.0) 1080 : cluster [DBG] osdmap e134: 8 total, 8 up, 8 in 2024-02-23T00:43:26.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:24.847332+0000 mon.b (mon.1) 126 : audit [INF] from='client.? 172.21.15.47:0/3883540601' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:26.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:24.931873+0000 mon.a (mon.0) 1081 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:26.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:24.969147+0000 mon.b (mon.1) 127 : audit [DBG] from='client.? 172.21.15.47:0/3214221007' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:43:26.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:25.192636+0000 mgr.y (mgr.14517) 175 : cluster [DBG] pgmap v176: 484 pgs: 64 unknown, 1 active+clean+snaptrim, 1 peering, 418 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-02-23T00:43:26.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:25.564541+0000 mon.b (mon.1) 128 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:26.647 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:43:26.650 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:26.650 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:43:26.751 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:43:26.777 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:26.795 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:43:26.798 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:26.798 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:43:26.869 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-23T00:43:26.869 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-23T00:43:26.869 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph pg dump --format=json 2024-02-23T00:43:26.899 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:43:26.925 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:26.944 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:43:26.947 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:26.947 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:43:27.048 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:43:27.074 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:27.093 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:43:27.096 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:27.097 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:43:27.198 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:43:27.633 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:26.169051+0000 mon.a (mon.0) 1082 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:27.633 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:26.169206+0000 mon.a (mon.0) 1083 : cluster [DBG] osdmap e135: 8 total, 8 up, 8 in 2024-02-23T00:43:27.633 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:26.192645+0000 mon.c (mon.2) 73 : audit [INF] from='client.? 172.21.15.47:0/3806507986' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi047-82386-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:27.633 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:26.201052+0000 mon.b (mon.1) 129 : audit [INF] from='client.? 172.21.15.47:0/843347886' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi047-82378-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:27.634 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:26.567962+0000 mon.b (mon.1) 130 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:27.634 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:26.819308+0000 mon.a (mon.0) 1084 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi047-82386-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:27.634 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:26.820435+0000 mon.a (mon.0) 1085 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi047-82378-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:27.634 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:26.169051+0000 mon.a (mon.0) 1082 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:27.634 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:26.169206+0000 mon.a (mon.0) 1083 : cluster [DBG] osdmap e135: 8 total, 8 up, 8 in 2024-02-23T00:43:27.634 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:26.192645+0000 mon.c (mon.2) 73 : audit [INF] from='client.? 172.21.15.47:0/3806507986' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi047-82386-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:27.634 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:26.201052+0000 mon.b (mon.1) 129 : audit [INF] from='client.? 172.21.15.47:0/843347886' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi047-82378-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:27.634 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:26.567962+0000 mon.b (mon.1) 130 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:27.635 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:26.819308+0000 mon.a (mon.0) 1084 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi047-82386-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:27.635 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:26.820435+0000 mon.a (mon.0) 1085 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi047-82378-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:27.641 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:27.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:26.169051+0000 mon.a (mon.0) 1082 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:27.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:26.169206+0000 mon.a (mon.0) 1083 : cluster [DBG] osdmap e135: 8 total, 8 up, 8 in 2024-02-23T00:43:27.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:26.192645+0000 mon.c (mon.2) 73 : audit [INF] from='client.? 172.21.15.47:0/3806507986' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi047-82386-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:27.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:26.201052+0000 mon.b (mon.1) 129 : audit [INF] from='client.? 172.21.15.47:0/843347886' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi047-82378-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:27.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:26.567962+0000 mon.b (mon.1) 130 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:27.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:26.819308+0000 mon.a (mon.0) 1084 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi047-82386-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:27.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:26.820435+0000 mon.a (mon.0) 1085 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi047-82378-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:28.311 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:43:28.320 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:28.320 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:43:28.421 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:43:28.447 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:28.466 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:43:28.468 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:28.469 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:43:28.518 INFO:tasks.workunit.client.0.smithi047.stdout:9) pacific (stable)", 2024-02-23T00:43:28.518 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "ceph_version_short": "16.2.14-550-geb66ed92", 2024-02-23T00:43:28.518 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "container_hostname": "smithi047", 2024-02-23T00:43:28.518 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:44277403522cf37edeee63b04643fd6c6e9416f13c3731a771e3cc8e53b7d010", 2024-02-23T00:43:28.518 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2024-02-23T00:43:28.518 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "distro": "centos", 2024-02-23T00:43:28.518 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "distro_description": "CentOS Stream 8", 2024-02-23T00:43:28.519 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "distro_version": "8", 2024-02-23T00:43:28.519 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "frontend_config#0": "beast port=80", 2024-02-23T00:43:28.519 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "frontend_type#0": "beast", 2024-02-23T00:43:28.519 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "hostname": "smithi047", 2024-02-23T00:43:28.519 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "id": "foo.a", 2024-02-23T00:43:28.519 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "kernel_description": "#1 SMP Fri Feb 9 14:58:00 UTC 2024", 2024-02-23T00:43:28.519 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "kernel_version": "4.18.0-540.el8.x86_64", 2024-02-23T00:43:28.519 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "mem_swap_kb": "0", 2024-02-23T00:43:28.519 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "mem_total_kb": "32543848", 2024-02-23T00:43:28.519 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "num_handles": "1", 2024-02-23T00:43:28.519 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "os": "Linux", 2024-02-23T00:43:28.519 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "pid": "7", 2024-02-23T00:43:28.519 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "realm_id": "", 2024-02-23T00:43:28.519 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "realm_name": "", 2024-02-23T00:43:28.520 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "zone_id": "52970d38-d172-4ec0-ad23-3afdfeba31e4", 2024-02-23T00:43:28.520 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "zone_name": "default", 2024-02-23T00:43:28.520 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "zonegroup_id": "343d5407-1492-440c-b380-09e4750e9af7", 2024-02-23T00:43:28.520 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "zonegroup_name": "default" 2024-02-23T00:43:28.520 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: }, 2024-02-23T00:43:28.520 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: "task_status": {} 2024-02-23T00:43:28.520 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: } 2024-02-23T00:43:28.520 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: } 2024-02-23T00:43:28.520 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: } 2024-02-23T00:43:28.520 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: } 2024-02-23T00:43:28.520 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: } 2024-02-23T00:43:28.520 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: 2024-02-23T00:43:28.520 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: attempt 1 of 20 2024-02-23T00:43:28.520 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (10327 ms) 2024-02-23T00:43:28.520 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (41256 ms total) 2024-02-23T00:43:28.521 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: 2024-02-23T00:43:28.521 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: [----------] Global test environment tear-down 2024-02-23T00:43:28.521 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (41256 ms total) 2024-02-23T00:43:28.521 INFO:tasks.workunit.client.0.smithi047.stdout: api_service_pp: [ PASSED ] 4 tests. 2024-02-23T00:43:28.569 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:43:28.603 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:28.624 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:43:28.627 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:28.627 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:43:28.728 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:43:28.905 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:27.194632+0000 mgr.y (mgr.14517) 176 : cluster [DBG] pgmap v178: 483 pgs: 64 creating+peering, 1 peering, 418 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-02-23T00:43:28.906 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:27.497335+0000 mon.a (mon.0) 1086 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-02-23T00:43:28.906 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:27.526488+0000 mon.a (mon.0) 1087 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi047-82386-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:28.906 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:27.526571+0000 mon.a (mon.0) 1088 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi047-82378-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:28.906 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:27.526923+0000 mon.a (mon.0) 1089 : cluster [DBG] osdmap e136: 8 total, 8 up, 8 in 2024-02-23T00:43:28.906 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:27.536804+0000 mon.a (mon.0) 1090 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:28.906 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:27.577750+0000 mon.b (mon.1) 131 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:28.906 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:27.600915+0000 mon.b (mon.1) 132 : audit [INF] from='client.? 172.21.15.47:0/1361361966' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi047-82487-9"}]: dispatch 2024-02-23T00:43:28.906 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:27.610466+0000 mon.a (mon.0) 1091 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi047-82487-9"}]: dispatch 2024-02-23T00:43:28.906 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:27.612273+0000 mon.b (mon.1) 133 : audit [INF] from='client.? 172.21.15.47:0/1361361966' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi047-82487-9"}]: dispatch 2024-02-23T00:43:28.906 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:27.621878+0000 mon.a (mon.0) 1092 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi047-82487-9"}]: dispatch 2024-02-23T00:43:28.906 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:27.624198+0000 mon.b (mon.1) 134 : audit [INF] from='client.? 172.21.15.47:0/1361361966' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi047-82487-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:28.906 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:27.624695+0000 mon.a (mon.0) 1093 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi047-82487-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:28.906 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:27.762434+0000 mon.c (mon.2) 74 : audit [INF] from='client.? 172.21.15.47:0/4182485101' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4"}]: dispatch 2024-02-23T00:43:28.907 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:27.763375+0000 mon.a (mon.0) 1094 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4"}]: dispatch 2024-02-23T00:43:28.907 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:28.513072+0000 mon.a (mon.0) 1095 : audit [DBG] from='client.? 172.21.15.47:0/695991716' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-23T00:43:28.907 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:28.543041+0000 mon.a (mon.0) 1096 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "26"}]': finished 2024-02-23T00:43:28.907 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:28.543119+0000 mon.a (mon.0) 1097 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:28.907 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:28.543326+0000 mon.a (mon.0) 1098 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi047-82487-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:28.907 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:28.543449+0000 mon.a (mon.0) 1099 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4"}]': finished 2024-02-23T00:43:28.907 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:28.543500+0000 mon.a (mon.0) 1100 : cluster [DBG] osdmap e137: 8 total, 8 up, 8 in 2024-02-23T00:43:28.907 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:28.550653+0000 mon.b (mon.1) 135 : audit [INF] from='client.? 172.21.15.47:0/1361361966' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi047-82487-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi047-82487-9"}]: dispatch 2024-02-23T00:43:28.907 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:28.551124+0000 mon.c (mon.2) 75 : audit [INF] from='client.? 172.21.15.47:0/4182485101' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6"}]: dispatch 2024-02-23T00:43:28.907 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:28.551899+0000 mon.a (mon.0) 1101 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6"}]: dispatch 2024-02-23T00:43:28.907 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:28.576716+0000 mon.a (mon.0) 1102 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi047-82487-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi047-82487-9"}]: dispatch 2024-02-23T00:43:28.908 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:27.194632+0000 mgr.y (mgr.14517) 176 : cluster [DBG] pgmap v178: 483 pgs: 64 creating+peering, 1 peering, 418 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-02-23T00:43:28.908 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:27.497335+0000 mon.a (mon.0) 1086 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-02-23T00:43:28.908 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:27.526488+0000 mon.a (mon.0) 1087 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi047-82386-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:28.908 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:27.526571+0000 mon.a (mon.0) 1088 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi047-82378-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:28.908 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:27.526923+0000 mon.a (mon.0) 1089 : cluster [DBG] osdmap e136: 8 total, 8 up, 8 in 2024-02-23T00:43:28.908 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:27.536804+0000 mon.a (mon.0) 1090 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:28.908 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:27.577750+0000 mon.b (mon.1) 131 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:28.908 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:27.600915+0000 mon.b (mon.1) 132 : audit [INF] from='client.? 172.21.15.47:0/1361361966' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi047-82487-9"}]: dispatch 2024-02-23T00:43:28.909 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:27.610466+0000 mon.a (mon.0) 1091 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi047-82487-9"}]: dispatch 2024-02-23T00:43:28.909 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:27.612273+0000 mon.b (mon.1) 133 : audit [INF] from='client.? 172.21.15.47:0/1361361966' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi047-82487-9"}]: dispatch 2024-02-23T00:43:28.909 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:27.621878+0000 mon.a (mon.0) 1092 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi047-82487-9"}]: dispatch 2024-02-23T00:43:28.909 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:27.624198+0000 mon.b (mon.1) 134 : audit [INF] from='client.? 172.21.15.47:0/1361361966' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi047-82487-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:28.909 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:27.624695+0000 mon.a (mon.0) 1093 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi047-82487-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:28.909 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:27.762434+0000 mon.c (mon.2) 74 : audit [INF] from='client.? 172.21.15.47:0/4182485101' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4"}]: dispatch 2024-02-23T00:43:28.909 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:27.763375+0000 mon.a (mon.0) 1094 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4"}]: dispatch 2024-02-23T00:43:28.909 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:28.513072+0000 mon.a (mon.0) 1095 : audit [DBG] from='client.? 172.21.15.47:0/695991716' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-23T00:43:28.909 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:28.543041+0000 mon.a (mon.0) 1096 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "26"}]': finished 2024-02-23T00:43:28.909 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:28.543119+0000 mon.a (mon.0) 1097 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:28.909 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:28.543326+0000 mon.a (mon.0) 1098 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi047-82487-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:28.909 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:28.543449+0000 mon.a (mon.0) 1099 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4"}]': finished 2024-02-23T00:43:28.909 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:28.543500+0000 mon.a (mon.0) 1100 : cluster [DBG] osdmap e137: 8 total, 8 up, 8 in 2024-02-23T00:43:28.909 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:28.550653+0000 mon.b (mon.1) 135 : audit [INF] from='client.? 172.21.15.47:0/1361361966' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi047-82487-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi047-82487-9"}]: dispatch 2024-02-23T00:43:28.909 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:28.551124+0000 mon.c (mon.2) 75 : audit [INF] from='client.? 172.21.15.47:0/4182485101' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6"}]: dispatch 2024-02-23T00:43:28.910 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:28.551899+0000 mon.a (mon.0) 1101 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6"}]: dispatch 2024-02-23T00:43:28.910 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:28.576716+0000 mon.a (mon.0) 1102 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi047-82487-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi047-82487-9"}]: dispatch 2024-02-23T00:43:28.911 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:28.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:27.194632+0000 mgr.y (mgr.14517) 176 : cluster [DBG] pgmap v178: 483 pgs: 64 creating+peering, 1 peering, 418 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-02-23T00:43:28.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:27.497335+0000 mon.a (mon.0) 1086 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-02-23T00:43:28.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:27.526488+0000 mon.a (mon.0) 1087 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi047-82386-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:28.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:27.526571+0000 mon.a (mon.0) 1088 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi047-82378-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:27.526923+0000 mon.a (mon.0) 1089 : cluster [DBG] osdmap e136: 8 total, 8 up, 8 in 2024-02-23T00:43:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:27.536804+0000 mon.a (mon.0) 1090 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:27.577750+0000 mon.b (mon.1) 131 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:27.600915+0000 mon.b (mon.1) 132 : audit [INF] from='client.? 172.21.15.47:0/1361361966' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi047-82487-9"}]: dispatch 2024-02-23T00:43:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:27.610466+0000 mon.a (mon.0) 1091 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi047-82487-9"}]: dispatch 2024-02-23T00:43:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:27.612273+0000 mon.b (mon.1) 133 : audit [INF] from='client.? 172.21.15.47:0/1361361966' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi047-82487-9"}]: dispatch 2024-02-23T00:43:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:27.621878+0000 mon.a (mon.0) 1092 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi047-82487-9"}]: dispatch 2024-02-23T00:43:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:27.624198+0000 mon.b (mon.1) 134 : audit [INF] from='client.? 172.21.15.47:0/1361361966' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi047-82487-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:27.624695+0000 mon.a (mon.0) 1093 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi047-82487-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:27.762434+0000 mon.c (mon.2) 74 : audit [INF] from='client.? 172.21.15.47:0/4182485101' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4"}]: dispatch 2024-02-23T00:43:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:27.763375+0000 mon.a (mon.0) 1094 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4"}]: dispatch 2024-02-23T00:43:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:28.513072+0000 mon.a (mon.0) 1095 : audit [DBG] from='client.? 172.21.15.47:0/695991716' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-23T00:43:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:28.543041+0000 mon.a (mon.0) 1096 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "26"}]': finished 2024-02-23T00:43:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:28.543119+0000 mon.a (mon.0) 1097 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:28.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:28.543326+0000 mon.a (mon.0) 1098 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi047-82487-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:28.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:28.543449+0000 mon.a (mon.0) 1099 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4"}]': finished 2024-02-23T00:43:28.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:28.543500+0000 mon.a (mon.0) 1100 : cluster [DBG] osdmap e137: 8 total, 8 up, 8 in 2024-02-23T00:43:28.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:28.550653+0000 mon.b (mon.1) 135 : audit [INF] from='client.? 172.21.15.47:0/1361361966' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi047-82487-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi047-82487-9"}]: dispatch 2024-02-23T00:43:28.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:28.551124+0000 mon.c (mon.2) 75 : audit [INF] from='client.? 172.21.15.47:0/4182485101' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6"}]: dispatch 2024-02-23T00:43:28.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:28.551899+0000 mon.a (mon.0) 1101 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6"}]: dispatch 2024-02-23T00:43:28.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:28.576716+0000 mon.a (mon.0) 1102 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi047-82487-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi047-82487-9"}]: dispatch 2024-02-23T00:43:29.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.133+0000 7f40574ba700 1 -- 172.21.15.47:0/3276341019 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40501001a0 msgr2=0x7f4050100580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.133+0000 7f40574ba700 1 --2- 172.21.15.47:0/3276341019 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40501001a0 0x7f4050100580 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 rx=0x7f404c004870 tx=0x7f404c009cb0).stop 2024-02-23T00:43:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.140+0000 7f40574ba700 1 -- 172.21.15.47:0/3276341019 shutdown_connections 2024-02-23T00:43:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.140+0000 7f40574ba700 1 --2- 172.21.15.47:0/3276341019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4050069c40 0x7f405010d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.140+0000 7f40574ba700 1 --2- 172.21.15.47:0/3276341019 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40501001a0 0x7f4050100580 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.140+0000 7f40574ba700 1 --2- 172.21.15.47:0/3276341019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40500692d0 0x7f4050069750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:43:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.140+0000 7f40574ba700 1 -- 172.21.15.47:0/3276341019 >> 172.21.15.47:0/3276341019 conn(0x7f4050063db0 msgr2=0x7f4050067560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:43:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.140+0000 7f40574ba700 1 -- 172.21.15.47:0/3276341019 shutdown_connections 2024-02-23T00:43:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.140+0000 7f40574ba700 1 -- 172.21.15.47:0/3276341019 wait complete. 2024-02-23T00:43:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.141+0000 7f40574ba700 1 librados: starting msgr at 2024-02-23T00:43:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.141+0000 7f40574ba700 1 librados: starting objecter 2024-02-23T00:43:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.141+0000 7f40574ba700 1 Processor -- start 2024-02-23T00:43:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.141+0000 7f40574ba700 1 -- start start 2024-02-23T00:43:29.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.141+0000 7f40574ba700 1 librados: setting wanted keys 2024-02-23T00:43:29.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.141+0000 7f40574ba700 1 librados: calling monclient init 2024-02-23T00:43:29.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.142+0000 7f40574ba700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40500692d0 0x7f4050116920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:29.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.142+0000 7f40574ba700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4050069c40 0x7f4050116e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:29.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.142+0000 7f40574ba700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40501001a0 0x7f4050110ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:29.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.142+0000 7f40574ba700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4050110fb0 con 0x7f4050069c40 2024-02-23T00:43:29.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.142+0000 7f40574ba700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40501110f0 con 0x7f40500692d0 2024-02-23T00:43:29.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.142+0000 7f40574ba700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40501112b0 con 0x7f40501001a0 2024-02-23T00:43:29.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.142+0000 7f4054a55700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4050069c40 0x7f4050116e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:29.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.142+0000 7f4054a55700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4050069c40 0x7f4050116e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42952/0 (socket says 172.21.15.47:42952) 2024-02-23T00:43:29.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.142+0000 7f4054a55700 1 -- 172.21.15.47:0/2685779481 learned_addr learned my addr 172.21.15.47:0/2685779481 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:43:29.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.142+0000 7f4055a57700 1 --2- 172.21.15.47:0/2685779481 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40501001a0 0x7f4050110ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:29.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.142+0000 7f4054a55700 1 -- 172.21.15.47:0/2685779481 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40501001a0 msgr2=0x7f4050110ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:29.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.142+0000 7f4054a55700 1 --2- 172.21.15.47:0/2685779481 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40501001a0 0x7f4050110ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:29.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.142+0000 7f4054a55700 1 -- 172.21.15.47:0/2685779481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40500692d0 msgr2=0x7f4050116920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:29.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.142+0000 7f4054a55700 1 --2- 172.21.15.47:0/2685779481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40500692d0 0x7f4050116920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:43:29.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.142+0000 7f4054a55700 1 -- 172.21.15.47:0/2685779481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404c005040 con 0x7f4050069c40 2024-02-23T00:43:29.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.143+0000 7f4054a55700 1 --2- 172.21.15.47:0/2685779481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4050069c40 0x7f4050116e10 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 rx=0x7f404000b530 tx=0x7f40400090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:29.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.143+0000 7f40467fc700 10 client.?.objecter ms_handle_connect 0x7f4050069c40 2024-02-23T00:43:29.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.143+0000 7f40467fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:43:29.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.239+0000 7f40467fc700 1 -- 172.21.15.47:0/2685779481 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4040002320 con 0x7f4050069c40 2024-02-23T00:43:29.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.240+0000 7f40574ba700 1 -- 172.21.15.47:0/2685779481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4050111590 con 0x7f4050069c40 2024-02-23T00:43:29.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.240+0000 7f40574ba700 10 client.21570.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:43:29.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.240+0000 7f40574ba700 1 -- 172.21.15.47:0/2685779481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f405011d720 con 0x7f4050069c40 2024-02-23T00:43:29.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.240+0000 7f40574ba700 1 librados: init done 2024-02-23T00:43:29.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.241+0000 7f40574ba700 1 -- 172.21.15.47:0/2685779481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4050060280 con 0x7f4050069c40 2024-02-23T00:43:29.411 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:43:29.415 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:29.416 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:43:29.516 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:43:29.542 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:29.561 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:43:29.563 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:29.564 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:43:29.665 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:43:29.853 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:43:29.853 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-02-23T00:43:29.853 INFO:teuthology.orchestra.run.smithi047.stdout: "num_ops": 0 2024-02-23T00:43:29.853 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:43:29.876 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:43:29.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.870+0000 7f40467fc700 1 -- 172.21.15.47:0/2685779481 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f4040009160 con 0x7f4050069c40 2024-02-23T00:43:29.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.870+0000 7f40467fc700 1 -- 172.21.15.47:0/2685779481 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4040012720 con 0x7f4050069c40 2024-02-23T00:43:29.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.870+0000 7f40467fc700 1 -- 172.21.15.47:0/2685779481 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f4040012960 con 0x7f4050069c40 2024-02-23T00:43:29.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.871+0000 7f40467fc700 1 --2- 172.21.15.47:0/2685779481 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f403c062d10 0x7f403c0651d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:29.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.871+0000 7f40467fc700 1 -- 172.21.15.47:0/2685779481 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(137..137 src has 48..137) v4 ==== 11941+0+0 (secure 0 0 0) 0x7f404007f590 con 0x7f4050069c40 2024-02-23T00:43:29.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.871+0000 7f40467fc700 10 client.21570.objecter ms_dispatch 0x7f4050001550 osd_map(137..137 src has 48..137) v4 2024-02-23T00:43:29.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.871+0000 7f40467fc700 3 client.21570.objecter handle_osd_map got epochs [137,137] > 0 2024-02-23T00:43:29.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.871+0000 7f40467fc700 3 client.21570.objecter handle_osd_map decoding full epoch 137 2024-02-23T00:43:29.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.871+0000 7f40467fc700 20 client.21570.objecter dump_active .. 0 homeless 2024-02-23T00:43:29.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.877+0000 7f40467fc700 1 -- 172.21.15.47:0/2685779481 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f404004e0a0 con 0x7f4050069c40 2024-02-23T00:43:29.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.877+0000 7f4055256700 1 --2- 172.21.15.47:0/2685779481 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f403c062d10 0x7f403c0651d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:29.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.882+0000 7f4055256700 1 --2- 172.21.15.47:0/2685779481 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f403c062d10 0x7f403c0651d0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 rx=0x7f404c004540 tx=0x7f404c009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:29.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:29.884+0000 7f40467fc700 10 client.21570.objecter ms_handle_connect 0x7f403c062d10 2024-02-23T00:43:29.936 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:30.129 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:43:30.145 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:30.145 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:43:30.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:30.197+0000 7f40574ba700 1 -- 172.21.15.47:0/2685779481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7f405010f7d0 con 0x7f4050069c40 2024-02-23T00:43:30.246 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:43:30.273 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:30.292 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:43:30.294 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:30.295 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:43:30.334 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:43:30.396 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:43:30.422 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:30.441 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:43:30.444 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:30.444 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:43:30.545 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:43:30.635 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:43:30.680 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:28.513229+0000 mgr.y (mgr.14517) 177 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-23T00:43:30.680 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:28.608593+0000 mon.b (mon.1) 136 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:30.680 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:29.195856+0000 mgr.y (mgr.14517) 178 : cluster [DBG] pgmap v181: 419 pgs: 32 unknown, 1 peering, 386 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 234 B/s wr, 1 op/s 2024-02-23T00:43:30.680 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:29.236538+0000 mon.a (mon.0) 1103 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:43:30.680 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:29.622372+0000 mon.b (mon.1) 137 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:30.681 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:43:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:43:30] "GET /metrics HTTP/1.1" 200 206383 "" "Prometheus/2.33.4" 2024-02-23T00:43:30.681 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:28.513229+0000 mgr.y (mgr.14517) 177 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-23T00:43:30.681 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:28.608593+0000 mon.b (mon.1) 136 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:30.681 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:29.195856+0000 mgr.y (mgr.14517) 178 : cluster [DBG] pgmap v181: 419 pgs: 32 unknown, 1 peering, 386 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 234 B/s wr, 1 op/s 2024-02-23T00:43:30.681 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:29.236538+0000 mon.a (mon.0) 1103 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:43:30.681 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:29.622372+0000 mon.b (mon.1) 137 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:30.682 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:30.682 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:43:30.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:28.513229+0000 mgr.y (mgr.14517) 177 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-23T00:43:30.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:28.608593+0000 mon.b (mon.1) 136 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:30.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:29.195856+0000 mgr.y (mgr.14517) 178 : cluster [DBG] pgmap v181: 419 pgs: 32 unknown, 1 peering, 386 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 234 B/s wr, 1 op/s 2024-02-23T00:43:30.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:29.236538+0000 mon.a (mon.0) 1103 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:43:30.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:29.622372+0000 mon.b (mon.1) 137 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:30.739 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.3 dump_blocked_ops 2024-02-23T00:43:30.783 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:43:31.064 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:31.136 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:43:31.141 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:31.141 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:43:31.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:31.151+0000 7f40467fc700 1 -- 172.21.15.47:0/2685779481 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v139) v1 ==== 91+0+0 (secure 0 0 0) 0x7f40400516c0 con 0x7f4050069c40 2024-02-23T00:43:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:31.169+0000 7f40574ba700 10 librados: watch_flush enter 2024-02-23T00:43:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:31.170+0000 7f40574ba700 10 librados: watch_flush exit 2024-02-23T00:43:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:31.170+0000 7f40574ba700 20 client.21570.objecter shutdown clearing up homeless session... 2024-02-23T00:43:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:31.170+0000 7f40574ba700 10 client.21570.objecter successfully canceled tick 2024-02-23T00:43:31.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:31.177+0000 7f40574ba700 1 -- 172.21.15.47:0/2685779481 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f403c062d10 msgr2=0x7f403c0651d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:31.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:31.177+0000 7f40574ba700 1 --2- 172.21.15.47:0/2685779481 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f403c062d10 0x7f403c0651d0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 rx=0x7f404c004540 tx=0x7f404c009cb0).stop 2024-02-23T00:43:31.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:31.177+0000 7f40574ba700 1 -- 172.21.15.47:0/2685779481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4050069c40 msgr2=0x7f4050116e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:31.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:31.177+0000 7f40574ba700 1 --2- 172.21.15.47:0/2685779481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4050069c40 0x7f4050116e10 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 rx=0x7f404000b530 tx=0x7f40400090f0).stop 2024-02-23T00:43:31.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:31.191+0000 7f40574ba700 1 -- 172.21.15.47:0/2685779481 shutdown_connections 2024-02-23T00:43:31.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:31.191+0000 7f40574ba700 1 --2- 172.21.15.47:0/2685779481 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f403c062d10 0x7f403c0651d0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:31.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:31.191+0000 7f40574ba700 1 --2- 172.21.15.47:0/2685779481 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40501001a0 0x7f4050110ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:31.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:31.191+0000 7f40574ba700 1 --2- 172.21.15.47:0/2685779481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40500692d0 0x7f4050116920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:43:31.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:31.191+0000 7f40574ba700 1 --2- 172.21.15.47:0/2685779481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4050069c40 0x7f4050116e10 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:31.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:31.191+0000 7f40574ba700 1 -- 172.21.15.47:0/2685779481 >> 172.21.15.47:0/2685779481 conn(0x7f4050063db0 msgr2=0x7f4050105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:43:31.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:31.201+0000 7f40574ba700 1 -- 172.21.15.47:0/2685779481 shutdown_connections 2024-02-23T00:43:31.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:31.203+0000 7f40574ba700 1 -- 172.21.15.47:0/2685779481 wait complete. 2024-02-23T00:43:31.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:31.203+0000 7f40574ba700 1 librados: shutdown 2024-02-23T00:43:31.215 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is unset 2024-02-23T00:43:31.242 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:43:31.361 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:31.694 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:29.877675+0000 mon.a (mon.0) 1104 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:43:31.694 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:29.892135+0000 mon.a (mon.0) 1105 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6"}]': finished 2024-02-23T00:43:31.694 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:29.911006+0000 mon.c (mon.2) 76 : audit [INF] from='client.? 172.21.15.47:0/4182485101' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-23T00:43:31.694 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:30.442756+0000 mon.a (mon.0) 1106 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:43:31.694 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:30.442856+0000 mon.a (mon.0) 1107 : cluster [DBG] osdmap e138: 8 total, 8 up, 8 in 2024-02-23T00:43:31.694 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:30.446371+0000 mon.b (mon.1) 138 : audit [INF] from='client.? 172.21.15.47:0/1813864525' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi047-82378-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:31.694 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:30.454866+0000 mon.a (mon.0) 1108 : audit [INF] from='client.? 172.21.15.47:0/2685779481' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:43:31.694 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:30.461758+0000 mon.a (mon.0) 1109 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-9"}]: dispatch 2024-02-23T00:43:31.695 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:30.462131+0000 mon.a (mon.0) 1110 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-23T00:43:31.695 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:30.462797+0000 mon.a (mon.0) 1111 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi047-82378-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:31.695 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:30.467259+0000 mon.a (mon.0) 1112 : audit [INF] from='client.? 172.21.15.47:0/3936817760' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi047-82386-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:31.695 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:30.623262+0000 mon.b (mon.1) 139 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:31.695 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:31.123978+0000 mon.a (mon.0) 1113 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:43:31.695 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:31.150050+0000 mon.a (mon.0) 1114 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi047-82487-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi047-82487-9"}]': finished 2024-02-23T00:43:31.695 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:31.150154+0000 mon.a (mon.0) 1115 : audit [INF] from='client.? 172.21.15.47:0/2685779481' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:43:31.695 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:31.150306+0000 mon.a (mon.0) 1116 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-9"}]': finished 2024-02-23T00:43:31.695 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:31.150399+0000 mon.a (mon.0) 1117 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-23T00:43:31.695 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:31.150555+0000 mon.a (mon.0) 1118 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi047-82378-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:31.695 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:31.150634+0000 mon.a (mon.0) 1119 : audit [INF] from='client.? 172.21.15.47:0/3936817760' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi047-82386-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:31.695 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:31.150703+0000 mon.a (mon.0) 1120 : cluster [DBG] osdmap e139: 8 total, 8 up, 8 in 2024-02-23T00:43:31.695 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:31.183349+0000 mon.a (mon.0) 1121 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-9", "mode": "writeback"}]: dispatch 2024-02-23T00:43:31.695 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:31.189557+0000 mon.a (mon.0) 1122 : audit [INF] from='client.? 172.21.15.47:0/880866486' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:31.696 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:31.200296+0000 mon.a (mon.0) 1123 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "25"}]: dispatch 2024-02-23T00:43:31.696 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:29.877675+0000 mon.a (mon.0) 1104 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:43:31.696 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:29.892135+0000 mon.a (mon.0) 1105 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6"}]': finished 2024-02-23T00:43:31.696 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:29.911006+0000 mon.c (mon.2) 76 : audit [INF] from='client.? 172.21.15.47:0/4182485101' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-23T00:43:31.696 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:30.442756+0000 mon.a (mon.0) 1106 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:43:31.696 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:30.442856+0000 mon.a (mon.0) 1107 : cluster [DBG] osdmap e138: 8 total, 8 up, 8 in 2024-02-23T00:43:31.696 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:30.446371+0000 mon.b (mon.1) 138 : audit [INF] from='client.? 172.21.15.47:0/1813864525' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi047-82378-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:31.696 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:30.454866+0000 mon.a (mon.0) 1108 : audit [INF] from='client.? 172.21.15.47:0/2685779481' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:43:31.696 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:30.461758+0000 mon.a (mon.0) 1109 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-9"}]: dispatch 2024-02-23T00:43:31.696 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:30.462131+0000 mon.a (mon.0) 1110 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-23T00:43:31.697 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:30.462797+0000 mon.a (mon.0) 1111 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi047-82378-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:31.697 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:30.467259+0000 mon.a (mon.0) 1112 : audit [INF] from='client.? 172.21.15.47:0/3936817760' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi047-82386-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:31.697 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:30.623262+0000 mon.b (mon.1) 139 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:31.697 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:31.123978+0000 mon.a (mon.0) 1113 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:43:31.697 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:31.150050+0000 mon.a (mon.0) 1114 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi047-82487-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi047-82487-9"}]': finished 2024-02-23T00:43:31.697 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:31.150154+0000 mon.a (mon.0) 1115 : audit [INF] from='client.? 172.21.15.47:0/2685779481' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:43:31.697 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:31.150306+0000 mon.a (mon.0) 1116 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-9"}]': finished 2024-02-23T00:43:31.697 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:31.150399+0000 mon.a (mon.0) 1117 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-23T00:43:31.697 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:31.150555+0000 mon.a (mon.0) 1118 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi047-82378-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:31.697 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:31.150634+0000 mon.a (mon.0) 1119 : audit [INF] from='client.? 172.21.15.47:0/3936817760' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi047-82386-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:31.697 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:31.150703+0000 mon.a (mon.0) 1120 : cluster [DBG] osdmap e139: 8 total, 8 up, 8 in 2024-02-23T00:43:31.697 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:31.183349+0000 mon.a (mon.0) 1121 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-9", "mode": "writeback"}]: dispatch 2024-02-23T00:43:31.697 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:31.189557+0000 mon.a (mon.0) 1122 : audit [INF] from='client.? 172.21.15.47:0/880866486' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:31.698 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:31.200296+0000 mon.a (mon.0) 1123 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "25"}]: dispatch 2024-02-23T00:43:31.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:29.877675+0000 mon.a (mon.0) 1104 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:43:31.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:29.892135+0000 mon.a (mon.0) 1105 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6"}]': finished 2024-02-23T00:43:31.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:29.911006+0000 mon.c (mon.2) 76 : audit [INF] from='client.? 172.21.15.47:0/4182485101' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-23T00:43:31.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:30.442756+0000 mon.a (mon.0) 1106 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:43:31.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:30.442856+0000 mon.a (mon.0) 1107 : cluster [DBG] osdmap e138: 8 total, 8 up, 8 in 2024-02-23T00:43:31.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:30.446371+0000 mon.b (mon.1) 138 : audit [INF] from='client.? 172.21.15.47:0/1813864525' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi047-82378-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:31.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:30.454866+0000 mon.a (mon.0) 1108 : audit [INF] from='client.? 172.21.15.47:0/2685779481' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:43:31.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:30.461758+0000 mon.a (mon.0) 1109 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-9"}]: dispatch 2024-02-23T00:43:31.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:30.462131+0000 mon.a (mon.0) 1110 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-23T00:43:31.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:30.462797+0000 mon.a (mon.0) 1111 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi047-82378-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:31.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:30.467259+0000 mon.a (mon.0) 1112 : audit [INF] from='client.? 172.21.15.47:0/3936817760' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi047-82386-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:31.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:30.623262+0000 mon.b (mon.1) 139 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:31.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:31.123978+0000 mon.a (mon.0) 1113 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:43:31.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:31.150050+0000 mon.a (mon.0) 1114 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi047-82487-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi047-82487-9"}]': finished 2024-02-23T00:43:31.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:31.150154+0000 mon.a (mon.0) 1115 : audit [INF] from='client.? 172.21.15.47:0/2685779481' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:43:31.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:31.150306+0000 mon.a (mon.0) 1116 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-9"}]': finished 2024-02-23T00:43:31.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:31.150399+0000 mon.a (mon.0) 1117 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi047-82528-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-23T00:43:31.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:31.150555+0000 mon.a (mon.0) 1118 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi047-82378-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:31.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:31.150634+0000 mon.a (mon.0) 1119 : audit [INF] from='client.? 172.21.15.47:0/3936817760' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi047-82386-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:31.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:31.150703+0000 mon.a (mon.0) 1120 : cluster [DBG] osdmap e139: 8 total, 8 up, 8 in 2024-02-23T00:43:31.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:31.183349+0000 mon.a (mon.0) 1121 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-9", "mode": "writeback"}]: dispatch 2024-02-23T00:43:31.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:31.189557+0000 mon.a (mon.0) 1122 : audit [INF] from='client.? 172.21.15.47:0/880866486' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:31.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:31.200296+0000 mon.a (mon.0) 1123 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "25"}]: dispatch 2024-02-23T00:43:32.217 INFO:tasks.workunit.client.0.smithi047.stdout:tch_notify_pp: flushed 2024-02-23T00:43:32.217 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3008 ms) 2024-02-23T00:43:32.217 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-02-23T00:43:32.217 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: trying... 2024-02-23T00:43:32.217 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: handle_notify cookie 94873507011984 notify_id 545460846599 notifier_gid 25103 2024-02-23T00:43:32.217 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: timed out 2024-02-23T00:43:32.217 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: flushing 2024-02-23T00:43:32.217 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: flushed 2024-02-23T00:43:32.218 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3014 ms) 2024-02-23T00:43:32.218 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-02-23T00:43:32.218 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: List watches 2024-02-23T00:43:32.218 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: notify2 2024-02-23T00:43:32.218 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: handle_notify cookie 94873507011984 notify_id 562640715782 notifier_gid 25103 2024-02-23T00:43:32.218 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: notify2 done 2024-02-23T00:43:32.218 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: watch_check 2024-02-23T00:43:32.218 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: unwatch2 2024-02-23T00:43:32.218 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: flushing 2024-02-23T00:43:32.218 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: done 2024-02-23T00:43:32.218 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3037 ms) 2024-02-23T00:43:32.218 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-02-23T00:43:32.218 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: List watches 2024-02-23T00:43:32.219 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: notify2 2024-02-23T00:43:32.219 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: handle_notify cookie 94873507011984 notify_id 571230650375 notifier_gid 25103 2024-02-23T00:43:32.219 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: notify2 done 2024-02-23T00:43:32.219 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: watch_check 2024-02-23T00:43:32.219 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: unwatch2 2024-02-23T00:43:32.219 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: flushing 2024-02-23T00:43:32.219 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: done 2024-02-23T00:43:32.219 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3014 ms) 2024-02-23T00:43:32.219 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (16905 ms total) 2024-02-23T00:43:32.219 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: 2024-02-23T00:43:32.219 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2024-02-23T00:43:32.219 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (44988 ms total) 2024-02-23T00:43:32.219 INFO:tasks.workunit.client.0.smithi047.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2024-02-23T00:43:32.266 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:43:32.289 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:32.290 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:43:32.391 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:43:32.419 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:32.439 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:43:32.442 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:32.442 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:43:32.543 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:43:32.572 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:32.592 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:43:32.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:43:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:43:32.594 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:43:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:43:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:43:32.594 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:32.595 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:43:32.696 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:43:32.722 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:32.741 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:43:32.743 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:32.744 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:43:32.844 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:43:32.871 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:32.892 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:43:32.894 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:32.895 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:43:32.996 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:43:33.076 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:33.214 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:43:33.225 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:33.225 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:43:33.280 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:43:33.326 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:43:33.389 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:43:33.431 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:31.197444+0000 mgr.y (mgr.14517) 179 : cluster [DBG] pgmap v184: 486 pgs: 136 unknown, 1 peering, 349 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:43:33.432 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:31.626785+0000 mon.b (mon.1) 140 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:33.432 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:31.899468+0000 osd.1 (osd.1) 87 : cluster [DBG] 5.12 scrub starts 2024-02-23T00:43:33.432 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:31.903023+0000 osd.1 (osd.1) 88 : cluster [DBG] 5.12 scrub ok 2024-02-23T00:43:33.432 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:32.148697+0000 mon.a (mon.0) 1124 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:43:33.432 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:32.170899+0000 mon.a (mon.0) 1125 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-9", "mode": "writeback"}]': finished 2024-02-23T00:43:33.432 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:32.171032+0000 mon.a (mon.0) 1126 : audit [INF] from='client.? 172.21.15.47:0/880866486' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:33.432 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:32.171091+0000 mon.a (mon.0) 1127 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "25"}]': finished 2024-02-23T00:43:33.432 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:32.171238+0000 mon.a (mon.0) 1128 : cluster [DBG] osdmap e140: 8 total, 8 up, 8 in 2024-02-23T00:43:33.432 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:32.333248+0000 mgr.y (mgr.14517) 180 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:43:33.432 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:32.628805+0000 mon.b (mon.1) 141 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:33.432 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:32.706031+0000 mon.b (mon.1) 142 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.0"}]: dispatch 2024-02-23T00:43:33.433 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:32.723670+0000 mon.b (mon.1) 143 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.1"}]: dispatch 2024-02-23T00:43:33.433 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:32.746610+0000 mon.b (mon.1) 144 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.2"}]: dispatch 2024-02-23T00:43:33.433 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:32.759903+0000 mon.b (mon.1) 145 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.3"}]: dispatch 2024-02-23T00:43:33.433 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:32.769315+0000 mon.b (mon.1) 146 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.4"}]: dispatch 2024-02-23T00:43:33.433 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:32.777493+0000 mon.b (mon.1) 147 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.5"}]: dispatch 2024-02-23T00:43:33.433 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:32.785110+0000 mon.b (mon.1) 148 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.6"}]: dispatch 2024-02-23T00:43:33.433 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:32.787746+0000 mon.b (mon.1) 149 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.7"}]: dispatch 2024-02-23T00:43:33.433 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:32.789620+0000 mon.b (mon.1) 150 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.8"}]: dispatch 2024-02-23T00:43:33.433 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:32.790320+0000 mon.b (mon.1) 151 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.9"}]: dispatch 2024-02-23T00:43:33.435 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:31.197444+0000 mgr.y (mgr.14517) 179 : cluster [DBG] pgmap v184: 486 pgs: 136 unknown, 1 peering, 349 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:43:33.435 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:31.626785+0000 mon.b (mon.1) 140 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:33.435 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:31.899468+0000 osd.1 (osd.1) 87 : cluster [DBG] 5.12 scrub starts 2024-02-23T00:43:33.435 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:31.903023+0000 osd.1 (osd.1) 88 : cluster [DBG] 5.12 scrub ok 2024-02-23T00:43:33.435 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:32.148697+0000 mon.a (mon.0) 1124 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:43:33.435 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:32.170899+0000 mon.a (mon.0) 1125 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-9", "mode": "writeback"}]': finished 2024-02-23T00:43:33.435 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:32.171032+0000 mon.a (mon.0) 1126 : audit [INF] from='client.? 172.21.15.47:0/880866486' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:33.435 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:32.171091+0000 mon.a (mon.0) 1127 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "25"}]': finished 2024-02-23T00:43:33.435 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:32.171238+0000 mon.a (mon.0) 1128 : cluster [DBG] osdmap e140: 8 total, 8 up, 8 in 2024-02-23T00:43:33.435 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:32.333248+0000 mgr.y (mgr.14517) 180 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:43:33.436 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:32.628805+0000 mon.b (mon.1) 141 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:33.436 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:32.706031+0000 mon.b (mon.1) 142 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.0"}]: dispatch 2024-02-23T00:43:33.436 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:32.723670+0000 mon.b (mon.1) 143 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.1"}]: dispatch 2024-02-23T00:43:33.436 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:32.746610+0000 mon.b (mon.1) 144 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.2"}]: dispatch 2024-02-23T00:43:33.436 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:32.759903+0000 mon.b (mon.1) 145 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.3"}]: dispatch 2024-02-23T00:43:33.436 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:32.769315+0000 mon.b (mon.1) 146 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.4"}]: dispatch 2024-02-23T00:43:33.436 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:32.777493+0000 mon.b (mon.1) 147 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.5"}]: dispatch 2024-02-23T00:43:33.436 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:32.785110+0000 mon.b (mon.1) 148 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.6"}]: dispatch 2024-02-23T00:43:33.436 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:32.787746+0000 mon.b (mon.1) 149 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.7"}]: dispatch 2024-02-23T00:43:33.436 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:32.789620+0000 mon.b (mon.1) 150 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.8"}]: dispatch 2024-02-23T00:43:33.436 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:32.790320+0000 mon.b (mon.1) 151 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.9"}]: dispatch 2024-02-23T00:43:33.513 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:33.661 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:43:33.664 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:33.664 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:43:33.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:31.197444+0000 mgr.y (mgr.14517) 179 : cluster [DBG] pgmap v184: 486 pgs: 136 unknown, 1 peering, 349 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:43:33.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:31.626785+0000 mon.b (mon.1) 140 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:33.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:31.899468+0000 osd.1 (osd.1) 87 : cluster [DBG] 5.12 scrub starts 2024-02-23T00:43:33.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:31.903023+0000 osd.1 (osd.1) 88 : cluster [DBG] 5.12 scrub ok 2024-02-23T00:43:33.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:32.148697+0000 mon.a (mon.0) 1124 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:43:33.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:32.170899+0000 mon.a (mon.0) 1125 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-9", "mode": "writeback"}]': finished 2024-02-23T00:43:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:32.171032+0000 mon.a (mon.0) 1126 : audit [INF] from='client.? 172.21.15.47:0/880866486' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:32.171091+0000 mon.a (mon.0) 1127 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "25"}]': finished 2024-02-23T00:43:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:32.171238+0000 mon.a (mon.0) 1128 : cluster [DBG] osdmap e140: 8 total, 8 up, 8 in 2024-02-23T00:43:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:32.333248+0000 mgr.y (mgr.14517) 180 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:43:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:32.628805+0000 mon.b (mon.1) 141 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:32.706031+0000 mon.b (mon.1) 142 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.0"}]: dispatch 2024-02-23T00:43:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:32.723670+0000 mon.b (mon.1) 143 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.1"}]: dispatch 2024-02-23T00:43:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:32.746610+0000 mon.b (mon.1) 144 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.2"}]: dispatch 2024-02-23T00:43:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:32.759903+0000 mon.b (mon.1) 145 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.3"}]: dispatch 2024-02-23T00:43:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:32.769315+0000 mon.b (mon.1) 146 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.4"}]: dispatch 2024-02-23T00:43:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:32.777493+0000 mon.b (mon.1) 147 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.5"}]: dispatch 2024-02-23T00:43:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:32.785110+0000 mon.b (mon.1) 148 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.6"}]: dispatch 2024-02-23T00:43:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:32.787746+0000 mon.b (mon.1) 149 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.7"}]: dispatch 2024-02-23T00:43:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:32.789620+0000 mon.b (mon.1) 150 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.8"}]: dispatch 2024-02-23T00:43:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:32.790320+0000 mon.b (mon.1) 151 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.9"}]: dispatch 2024-02-23T00:43:33.765 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:43:34.046 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:34.452 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:43:34.455 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:34.456 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:43:34.557 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:43:34.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:31.360047+0000 osd.2 (osd.2) 67 : cluster [DBG] 2.b scrub starts 2024-02-23T00:43:34.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:31.362504+0000 osd.2 (osd.2) 68 : cluster [DBG] 2.b scrub ok 2024-02-23T00:43:34.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:32.711269+0000 mgr.y (mgr.14517) 181 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.0"}]: dispatch 2024-02-23T00:43:34.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:32.729344+0000 mgr.y (mgr.14517) 182 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.1"}]: dispatch 2024-02-23T00:43:34.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:32.747350+0000 mgr.y (mgr.14517) 183 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.2"}]: dispatch 2024-02-23T00:43:34.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:32.765416+0000 mgr.y (mgr.14517) 184 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.3"}]: dispatch 2024-02-23T00:43:34.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:32.773604+0000 mgr.y (mgr.14517) 185 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.4"}]: dispatch 2024-02-23T00:43:34.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:32.779183+0000 mgr.y (mgr.14517) 186 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.5"}]: dispatch 2024-02-23T00:43:34.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:32.785680+0000 mgr.y (mgr.14517) 187 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.6"}]: dispatch 2024-02-23T00:43:34.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:32.788069+0000 mgr.y (mgr.14517) 188 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.7"}]: dispatch 2024-02-23T00:43:34.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:32.789786+0000 mgr.y (mgr.14517) 189 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.8"}]: dispatch 2024-02-23T00:43:34.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:32.790448+0000 mgr.y (mgr.14517) 190 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.9"}]: dispatch 2024-02-23T00:43:34.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:32.822625+0000 mon.a (mon.0) 1129 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:43:34.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:33.018907+0000 osd.3 (osd.3) 89 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:43:34.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:33.019007+0000 osd.3 (osd.3) 90 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:43:34.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:33.150130+0000 osd.4 (osd.4) 75 : cluster [DBG] 4.e scrub starts 2024-02-23T00:43:34.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:33.151396+0000 osd.4 (osd.4) 76 : cluster [DBG] 4.e scrub ok 2024-02-23T00:43:34.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:33.198246+0000 mgr.y (mgr.14517) 191 : cluster [DBG] pgmap v186: 390 pgs: 40 creating+peering, 1 peering, 349 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:43:34.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:33.447126+0000 mon.a (mon.0) 1130 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:43:34.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:33.450276+0000 mon.b (mon.1) 152 : audit [INF] from='client.? 172.21.15.47:0/1503027630' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi047-82386-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:34.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:33.639998+0000 mon.b (mon.1) 153 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:34.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:34.081790+0000 mon.a (mon.0) 1131 : cluster [DBG] osdmap e141: 8 total, 8 up, 8 in 2024-02-23T00:43:34.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:34.086515+0000 mon.a (mon.0) 1132 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-9"}]: dispatch 2024-02-23T00:43:34.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:34.087304+0000 mon.a (mon.0) 1133 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi047-82386-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:34.566 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:31.360047+0000 osd.2 (osd.2) 67 : cluster [DBG] 2.b scrub starts 2024-02-23T00:43:34.566 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:31.362504+0000 osd.2 (osd.2) 68 : cluster [DBG] 2.b scrub ok 2024-02-23T00:43:34.566 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:32.711269+0000 mgr.y (mgr.14517) 181 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.0"}]: dispatch 2024-02-23T00:43:34.567 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:32.729344+0000 mgr.y (mgr.14517) 182 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.1"}]: dispatch 2024-02-23T00:43:34.567 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:32.747350+0000 mgr.y (mgr.14517) 183 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.2"}]: dispatch 2024-02-23T00:43:34.567 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:32.765416+0000 mgr.y (mgr.14517) 184 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.3"}]: dispatch 2024-02-23T00:43:34.567 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:32.773604+0000 mgr.y (mgr.14517) 185 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.4"}]: dispatch 2024-02-23T00:43:34.567 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:32.779183+0000 mgr.y (mgr.14517) 186 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.5"}]: dispatch 2024-02-23T00:43:34.567 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:32.785680+0000 mgr.y (mgr.14517) 187 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.6"}]: dispatch 2024-02-23T00:43:34.567 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:32.788069+0000 mgr.y (mgr.14517) 188 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.7"}]: dispatch 2024-02-23T00:43:34.567 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:32.789786+0000 mgr.y (mgr.14517) 189 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.8"}]: dispatch 2024-02-23T00:43:34.567 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:32.790448+0000 mgr.y (mgr.14517) 190 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.9"}]: dispatch 2024-02-23T00:43:34.567 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:32.822625+0000 mon.a (mon.0) 1129 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:43:34.567 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:33.018907+0000 osd.3 (osd.3) 89 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:43:34.567 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:33.019007+0000 osd.3 (osd.3) 90 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:43:34.567 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:33.150130+0000 osd.4 (osd.4) 75 : cluster [DBG] 4.e scrub starts 2024-02-23T00:43:34.567 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:33.151396+0000 osd.4 (osd.4) 76 : cluster [DBG] 4.e scrub ok 2024-02-23T00:43:34.568 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:33.198246+0000 mgr.y (mgr.14517) 191 : cluster [DBG] pgmap v186: 390 pgs: 40 creating+peering, 1 peering, 349 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:43:34.568 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:33.447126+0000 mon.a (mon.0) 1130 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:43:34.568 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:33.450276+0000 mon.b (mon.1) 152 : audit [INF] from='client.? 172.21.15.47:0/1503027630' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi047-82386-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:34.568 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:33.639998+0000 mon.b (mon.1) 153 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:34.568 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:34.081790+0000 mon.a (mon.0) 1131 : cluster [DBG] osdmap e141: 8 total, 8 up, 8 in 2024-02-23T00:43:34.568 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:34.086515+0000 mon.a (mon.0) 1132 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-9"}]: dispatch 2024-02-23T00:43:34.568 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:34.087304+0000 mon.a (mon.0) 1133 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi047-82386-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:34.692 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:34.768 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:43:34.792 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:34.793 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:43:34.893 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:43:34.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:31.360047+0000 osd.2 (osd.2) 67 : cluster [DBG] 2.b scrub starts 2024-02-23T00:43:34.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:31.362504+0000 osd.2 (osd.2) 68 : cluster [DBG] 2.b scrub ok 2024-02-23T00:43:34.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:32.711269+0000 mgr.y (mgr.14517) 181 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.0"}]: dispatch 2024-02-23T00:43:34.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:32.729344+0000 mgr.y (mgr.14517) 182 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.1"}]: dispatch 2024-02-23T00:43:34.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:32.747350+0000 mgr.y (mgr.14517) 183 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.2"}]: dispatch 2024-02-23T00:43:34.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:32.765416+0000 mgr.y (mgr.14517) 184 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.3"}]: dispatch 2024-02-23T00:43:34.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:32.773604+0000 mgr.y (mgr.14517) 185 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.4"}]: dispatch 2024-02-23T00:43:34.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:32.779183+0000 mgr.y (mgr.14517) 186 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.5"}]: dispatch 2024-02-23T00:43:34.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:32.785680+0000 mgr.y (mgr.14517) 187 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.6"}]: dispatch 2024-02-23T00:43:34.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:32.788069+0000 mgr.y (mgr.14517) 188 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.7"}]: dispatch 2024-02-23T00:43:34.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:32.789786+0000 mgr.y (mgr.14517) 189 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.8"}]: dispatch 2024-02-23T00:43:34.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:32.790448+0000 mgr.y (mgr.14517) 190 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "20.9"}]: dispatch 2024-02-23T00:43:34.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:32.822625+0000 mon.a (mon.0) 1129 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:43:34.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:33.018907+0000 osd.3 (osd.3) 89 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:43:34.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:33.019007+0000 osd.3 (osd.3) 90 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:43:34.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:33.150130+0000 osd.4 (osd.4) 75 : cluster [DBG] 4.e scrub starts 2024-02-23T00:43:34.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:33.151396+0000 osd.4 (osd.4) 76 : cluster [DBG] 4.e scrub ok 2024-02-23T00:43:34.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:33.198246+0000 mgr.y (mgr.14517) 191 : cluster [DBG] pgmap v186: 390 pgs: 40 creating+peering, 1 peering, 349 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:43:34.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:33.447126+0000 mon.a (mon.0) 1130 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:43:34.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:33.450276+0000 mon.b (mon.1) 152 : audit [INF] from='client.? 172.21.15.47:0/1503027630' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi047-82386-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:34.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:33.639998+0000 mon.b (mon.1) 153 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:34.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:34.081790+0000 mon.a (mon.0) 1131 : cluster [DBG] osdmap e141: 8 total, 8 up, 8 in 2024-02-23T00:43:34.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:34.086515+0000 mon.a (mon.0) 1132 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-9"}]: dispatch 2024-02-23T00:43:34.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:34.087304+0000 mon.a (mon.0) 1133 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi047-82386-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:35.140 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:35.261 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-23T00:43:35.695 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:43:35.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:34.972+0000 7f2d99651700 1 -- 172.21.15.47:0/2532663410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d941044a0 msgr2=0x7f2d94104920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:35.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:34.972+0000 7f2d99651700 1 --2- 172.21.15.47:0/2532663410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d941044a0 0x7f2d94104920 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 rx=0x7f2d8c009be0 tx=0x7f2d8c004d10).stop 2024-02-23T00:43:35.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:34.994+0000 7f2d99651700 1 -- 172.21.15.47:0/2532663410 shutdown_connections 2024-02-23T00:43:35.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:34.994+0000 7f2d99651700 1 --2- 172.21.15.47:0/2532663410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d941044a0 0x7f2d94104920 unknown :-1 s=CLOSED pgs=2456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:35.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:34.994+0000 7f2d99651700 1 --2- 172.21.15.47:0/2532663410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d940692d0 0x7f2d940696b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:43:35.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:34.994+0000 7f2d99651700 1 --2- 172.21.15.47:0/2532663410 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2d94069ba0 0x7f2d94103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:43:35.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:34.994+0000 7f2d99651700 1 -- 172.21.15.47:0/2532663410 >> 172.21.15.47:0/2532663410 conn(0x7f2d94063db0 msgr2=0x7f2d94067560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:43:35.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.016+0000 7f2d99651700 1 -- 172.21.15.47:0/2532663410 shutdown_connections 2024-02-23T00:43:35.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.016+0000 7f2d99651700 1 -- 172.21.15.47:0/2532663410 wait complete. 2024-02-23T00:43:35.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.017+0000 7f2d99651700 1 librados: starting msgr at 2024-02-23T00:43:35.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.017+0000 7f2d99651700 1 librados: starting objecter 2024-02-23T00:43:35.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.017+0000 7f2d99651700 1 Processor -- start 2024-02-23T00:43:35.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.018+0000 7f2d99651700 1 -- start start 2024-02-23T00:43:35.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.018+0000 7f2d99651700 1 librados: setting wanted keys 2024-02-23T00:43:35.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.018+0000 7f2d99651700 1 librados: calling monclient init 2024-02-23T00:43:35.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.018+0000 7f2d99651700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2d940692d0 0x7f2d9410bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:35.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.018+0000 7f2d99651700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d94069ba0 0x7f2d9410c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:35.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.018+0000 7f2d99651700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d941044a0 0x7f2d9411a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:35.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.018+0000 7f2d99651700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9410cc80 con 0x7f2d94069ba0 2024-02-23T00:43:35.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.018+0000 7f2d99651700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9411aa10 con 0x7f2d941044a0 2024-02-23T00:43:35.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.018+0000 7f2d99651700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9411abd0 con 0x7f2d940692d0 2024-02-23T00:43:35.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.018+0000 7f2d937fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d941044a0 0x7f2d9411a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:35.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.019+0000 7f2d937fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d941044a0 0x7f2d9411a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.47:36686/0 (socket says 172.21.15.47:36686) 2024-02-23T00:43:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.019+0000 7f2d937fe700 1 -- 172.21.15.47:0/1678994535 learned_addr learned my addr 172.21.15.47:0/1678994535 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:43:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.020+0000 7f2d937fe700 1 -- 172.21.15.47:0/1678994535 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2d940692d0 msgr2=0x7f2d9410bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.020+0000 7f2d937fe700 1 --2- 172.21.15.47:0/1678994535 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2d940692d0 0x7f2d9410bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:43:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.020+0000 7f2d937fe700 1 -- 172.21.15.47:0/1678994535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d94069ba0 msgr2=0x7f2d9410c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.020+0000 7f2d937fe700 1 --2- 172.21.15.47:0/1678994535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d94069ba0 0x7f2d9410c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:43:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.021+0000 7f2d937fe700 1 -- 172.21.15.47:0/1678994535 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8c0097c0 con 0x7f2d941044a0 2024-02-23T00:43:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.021+0000 7f2d937fe700 1 --2- 172.21.15.47:0/1678994535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d941044a0 0x7f2d9411a4f0 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 rx=0x7f2d8c004b70 tx=0x7f2d8c004d10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.021+0000 7f2d7bfff700 10 client.?.objecter ms_handle_connect 0x7f2d941044a0 2024-02-23T00:43:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.021+0000 7f2d7bfff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:43:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.021+0000 7f2d7bfff700 1 -- 172.21.15.47:0/1678994535 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c00e040 con 0x7f2d941044a0 2024-02-23T00:43:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.021+0000 7f2d99651700 1 -- 172.21.15.47:0/1678994535 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d9411adf0 con 0x7f2d941044a0 2024-02-23T00:43:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.021+0000 7f2d99651700 10 client.35746.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:43:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.021+0000 7f2d99651700 1 -- 172.21.15.47:0/1678994535 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d9411b230 con 0x7f2d941044a0 2024-02-23T00:43:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.021+0000 7f2d99651700 1 librados: init done 2024-02-23T00:43:35.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.021+0000 7f2d7bfff700 1 -- 172.21.15.47:0/1678994535 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f2d8c018040 con 0x7f2d941044a0 2024-02-23T00:43:35.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.021+0000 7f2d7bfff700 1 -- 172.21.15.47:0/1678994535 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c01d400 con 0x7f2d941044a0 2024-02-23T00:43:35.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.031+0000 7f2d7bfff700 1 -- 172.21.15.47:0/1678994535 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f2d8c013420 con 0x7f2d941044a0 2024-02-23T00:43:35.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.031+0000 7f2d7bfff700 1 --2- 172.21.15.47:0/1678994535 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2d7c062d10 0x7f2d7c0651d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:35.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.031+0000 7f2d7bfff700 1 -- 172.21.15.47:0/1678994535 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(143..143 src has 48..143) v4 ==== 11329+0+0 (secure 0 0 0) 0x7f2d8c07fc40 con 0x7f2d941044a0 2024-02-23T00:43:35.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.031+0000 7f2d7bfff700 10 client.35746.objecter ms_dispatch 0x7f2d94001550 osd_map(143..143 src has 48..143) v4 2024-02-23T00:43:35.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.031+0000 7f2d7bfff700 3 client.35746.objecter handle_osd_map got epochs [143,143] > 0 2024-02-23T00:43:35.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.031+0000 7f2d7bfff700 3 client.35746.objecter handle_osd_map decoding full epoch 143 2024-02-23T00:43:35.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.032+0000 7f2d7bfff700 20 client.35746.objecter dump_active .. 0 homeless 2024-02-23T00:43:35.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.035+0000 7f2d99651700 1 -- 172.21.15.47:0/1678994535 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d80004f50 con 0x7f2d941044a0 2024-02-23T00:43:35.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.040+0000 7f2d92ffd700 1 --2- 172.21.15.47:0/1678994535 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2d7c062d10 0x7f2d7c0651d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:35.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.042+0000 7f2d92ffd700 1 --2- 172.21.15.47:0/1678994535 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2d7c062d10 0x7f2d7c0651d0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 rx=0x7f2d88009630 tx=0x7f2d88006b50).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:35.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.042+0000 7f2d7bfff700 10 client.35746.objecter ms_handle_connect 0x7f2d7c062d10 2024-02-23T00:43:35.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.045+0000 7f2d7bfff700 1 -- 172.21.15.47:0/1678994535 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f2d8c04ea10 con 0x7f2d941044a0 2024-02-23T00:43:35.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.367+0000 7f2d99651700 1 -- 172.21.15.47:0/1678994535 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2d80000bc0 con 0x7f2d7c062d10 2024-02-23T00:43:35.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.444+0000 7f2d7bfff700 1 -- 172.21.15.47:0/1678994535 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+879684 (secure 0 0 0) 0x7f2d80000bc0 con 0x7f2d7c062d10 2024-02-23T00:43:35.720 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:43:35.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.703+0000 7f2d99651700 10 librados: watch_flush enter 2024-02-23T00:43:35.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.703+0000 7f2d99651700 10 librados: watch_flush exit 2024-02-23T00:43:35.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.703+0000 7f2d99651700 20 client.35746.objecter shutdown clearing up homeless session... 2024-02-23T00:43:35.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.703+0000 7f2d99651700 10 client.35746.objecter successfully canceled tick 2024-02-23T00:43:35.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.705+0000 7f2d99651700 1 -- 172.21.15.47:0/1678994535 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2d7c062d10 msgr2=0x7f2d7c0651d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:35.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.705+0000 7f2d99651700 1 --2- 172.21.15.47:0/1678994535 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2d7c062d10 0x7f2d7c0651d0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 rx=0x7f2d88009630 tx=0x7f2d88006b50).stop 2024-02-23T00:43:35.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.705+0000 7f2d99651700 1 -- 172.21.15.47:0/1678994535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d941044a0 msgr2=0x7f2d9411a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:35.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.705+0000 7f2d99651700 1 --2- 172.21.15.47:0/1678994535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d941044a0 0x7f2d9411a4f0 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 rx=0x7f2d8c004b70 tx=0x7f2d8c004d10).stop 2024-02-23T00:43:35.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.707+0000 7f2d99651700 1 -- 172.21.15.47:0/1678994535 shutdown_connections 2024-02-23T00:43:35.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.707+0000 7f2d99651700 1 --2- 172.21.15.47:0/1678994535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d941044a0 0x7f2d9411a4f0 unknown :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:35.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.707+0000 7f2d99651700 1 --2- 172.21.15.47:0/1678994535 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2d7c062d10 0x7f2d7c0651d0 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:35.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.707+0000 7f2d99651700 1 --2- 172.21.15.47:0/1678994535 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2d940692d0 0x7f2d9410bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:43:35.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.707+0000 7f2d99651700 1 --2- 172.21.15.47:0/1678994535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d94069ba0 0x7f2d9410c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:43:35.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.707+0000 7f2d99651700 1 -- 172.21.15.47:0/1678994535 >> 172.21.15.47:0/1678994535 conn(0x7f2d94063db0 msgr2=0x7f2d94064fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:43:35.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.718+0000 7f2d99651700 1 -- 172.21.15.47:0/1678994535 shutdown_connections 2024-02-23T00:43:35.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.722+0000 7f2d99651700 1 -- 172.21.15.47:0/1678994535 wait complete. 2024-02-23T00:43:35.728 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:35.728 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:43:35.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:35.725+0000 7f2d99651700 1 librados: shutdown 2024-02-23T00:43:35.739 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-02-23T00:43:35.829 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:43:35.899 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:36.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:33.755566+0000 osd.7 (osd.7) 85 : cluster [DBG] 20.2 deep-scrub starts 2024-02-23T00:43:36.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:33.773071+0000 osd.7 (osd.7) 86 : cluster [DBG] 20.2 deep-scrub ok 2024-02-23T00:43:36.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:34.443587+0000 mon.a (mon.0) 1134 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:43:36.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:34.488845+0000 mon.a (mon.0) 1135 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-9"}]': finished 2024-02-23T00:43:36.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:34.488910+0000 mon.a (mon.0) 1136 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi047-82386-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:36.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:34.489070+0000 mon.a (mon.0) 1137 : cluster [DBG] osdmap e142: 8 total, 8 up, 8 in 2024-02-23T00:43:36.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:34.497093+0000 mon.b (mon.1) 154 : audit [INF] from='client.? 172.21.15.47:0/1819343708' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi047-82378-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:36.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:34.499440+0000 mon.a (mon.0) 1138 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi047-82378-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:36.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:34.652889+0000 mon.b (mon.1) 155 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:36.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:34.894811+0000 mon.a (mon.0) 1139 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi047-82378-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:36.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:34.895017+0000 mon.a (mon.0) 1140 : cluster [DBG] osdmap e143: 8 total, 8 up, 8 in 2024-02-23T00:43:36.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:35.199413+0000 mgr.y (mgr.14517) 192 : cluster [DBG] pgmap v190: 357 pgs: 32 unknown, 8 creating+peering, 1 peering, 316 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2024-02-23T00:43:36.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:35.390381+0000 mgr.y (mgr.14517) 193 : audit [DBG] from='client.35746 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:43:36.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:35.591680+0000 mon.a (mon.0) 1141 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "24"}]: dispatch 2024-02-23T00:43:36.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:35.664249+0000 mon.b (mon.1) 156 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:36.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:36.027564+0000 osd.3 (osd.3) 91 : cluster [DBG] 20.0 deep-scrub starts 2024-02-23T00:43:36.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:36.071269+0000 osd.3 (osd.3) 92 : cluster [DBG] 20.0 deep-scrub ok 2024-02-23T00:43:36.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:33.755566+0000 osd.7 (osd.7) 85 : cluster [DBG] 20.2 deep-scrub starts 2024-02-23T00:43:36.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:33.773071+0000 osd.7 (osd.7) 86 : cluster [DBG] 20.2 deep-scrub ok 2024-02-23T00:43:36.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:34.443587+0000 mon.a (mon.0) 1134 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:43:36.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:34.488845+0000 mon.a (mon.0) 1135 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-9"}]': finished 2024-02-23T00:43:36.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:34.488910+0000 mon.a (mon.0) 1136 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi047-82386-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:36.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:34.489070+0000 mon.a (mon.0) 1137 : cluster [DBG] osdmap e142: 8 total, 8 up, 8 in 2024-02-23T00:43:36.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:34.497093+0000 mon.b (mon.1) 154 : audit [INF] from='client.? 172.21.15.47:0/1819343708' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi047-82378-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:36.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:34.499440+0000 mon.a (mon.0) 1138 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi047-82378-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:36.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:34.652889+0000 mon.b (mon.1) 155 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:36.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:34.894811+0000 mon.a (mon.0) 1139 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi047-82378-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:36.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:34.895017+0000 mon.a (mon.0) 1140 : cluster [DBG] osdmap e143: 8 total, 8 up, 8 in 2024-02-23T00:43:36.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:35.199413+0000 mgr.y (mgr.14517) 192 : cluster [DBG] pgmap v190: 357 pgs: 32 unknown, 8 creating+peering, 1 peering, 316 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2024-02-23T00:43:36.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:35.390381+0000 mgr.y (mgr.14517) 193 : audit [DBG] from='client.35746 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:43:36.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:35.591680+0000 mon.a (mon.0) 1141 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "24"}]: dispatch 2024-02-23T00:43:36.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:35.664249+0000 mon.b (mon.1) 156 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:36.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:36.027564+0000 osd.3 (osd.3) 91 : cluster [DBG] 20.0 deep-scrub starts 2024-02-23T00:43:36.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:36.071269+0000 osd.3 (osd.3) 92 : cluster [DBG] 20.0 deep-scrub ok 2024-02-23T00:43:36.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:33.755566+0000 osd.7 (osd.7) 85 : cluster [DBG] 20.2 deep-scrub starts 2024-02-23T00:43:36.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:33.773071+0000 osd.7 (osd.7) 86 : cluster [DBG] 20.2 deep-scrub ok 2024-02-23T00:43:36.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:34.443587+0000 mon.a (mon.0) 1134 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:43:36.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:34.488845+0000 mon.a (mon.0) 1135 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-9"}]': finished 2024-02-23T00:43:36.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:34.488910+0000 mon.a (mon.0) 1136 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi047-82386-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:36.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:34.489070+0000 mon.a (mon.0) 1137 : cluster [DBG] osdmap e142: 8 total, 8 up, 8 in 2024-02-23T00:43:36.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:34.497093+0000 mon.b (mon.1) 154 : audit [INF] from='client.? 172.21.15.47:0/1819343708' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi047-82378-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:36.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:34.499440+0000 mon.a (mon.0) 1138 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi047-82378-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:36.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:34.652889+0000 mon.b (mon.1) 155 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:36.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:34.894811+0000 mon.a (mon.0) 1139 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi047-82378-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:36.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:34.895017+0000 mon.a (mon.0) 1140 : cluster [DBG] osdmap e143: 8 total, 8 up, 8 in 2024-02-23T00:43:36.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:35.199413+0000 mgr.y (mgr.14517) 192 : cluster [DBG] pgmap v190: 357 pgs: 32 unknown, 8 creating+peering, 1 peering, 316 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2024-02-23T00:43:36.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:35.390381+0000 mgr.y (mgr.14517) 193 : audit [DBG] from='client.35746 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:43:36.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:35.591680+0000 mon.a (mon.0) 1141 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "24"}]: dispatch 2024-02-23T00:43:36.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:35.664249+0000 mon.b (mon.1) 156 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:36.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:36.027564+0000 osd.3 (osd.3) 91 : cluster [DBG] 20.0 deep-scrub starts 2024-02-23T00:43:36.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:36.071269+0000 osd.3 (osd.3) 92 : cluster [DBG] 20.0 deep-scrub ok 2024-02-23T00:43:37.437 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:35.877614+0000 osd.1 (osd.1) 89 : cluster [DBG] 20.4 deep-scrub starts 2024-02-23T00:43:37.437 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:35.881919+0000 osd.1 (osd.1) 90 : cluster [DBG] 20.4 deep-scrub ok 2024-02-23T00:43:37.437 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:36.170788+0000 mon.a (mon.0) 1142 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "24"}]': finished 2024-02-23T00:43:37.437 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:36.174563+0000 osd.4 (osd.4) 77 : cluster [DBG] 6.e scrub starts 2024-02-23T00:43:37.437 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:36.183074+0000 mon.b (mon.1) 157 : audit [INF] from='client.? 172.21.15.47:0/2559316827' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:37.437 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:36.205874+0000 osd.4 (osd.4) 78 : cluster [DBG] 6.e scrub ok 2024-02-23T00:43:37.437 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:36.670703+0000 mon.b (mon.1) 158 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:37.438 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:36.739369+0000 mon.a (mon.0) 1143 : cluster [DBG] osdmap e144: 8 total, 8 up, 8 in 2024-02-23T00:43:37.438 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:36.745569+0000 mon.a (mon.0) 1144 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:37.438 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:36.767262+0000 mon.a (mon.0) 1145 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:37.438 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:35.877614+0000 osd.1 (osd.1) 89 : cluster [DBG] 20.4 deep-scrub starts 2024-02-23T00:43:37.438 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:35.881919+0000 osd.1 (osd.1) 90 : cluster [DBG] 20.4 deep-scrub ok 2024-02-23T00:43:37.438 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:36.170788+0000 mon.a (mon.0) 1142 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "24"}]': finished 2024-02-23T00:43:37.438 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:36.174563+0000 osd.4 (osd.4) 77 : cluster [DBG] 6.e scrub starts 2024-02-23T00:43:37.438 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:36.183074+0000 mon.b (mon.1) 157 : audit [INF] from='client.? 172.21.15.47:0/2559316827' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:37.438 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:36.205874+0000 osd.4 (osd.4) 78 : cluster [DBG] 6.e scrub ok 2024-02-23T00:43:37.438 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:36.670703+0000 mon.b (mon.1) 158 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:37.438 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:36.739369+0000 mon.a (mon.0) 1143 : cluster [DBG] osdmap e144: 8 total, 8 up, 8 in 2024-02-23T00:43:37.438 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:36.745569+0000 mon.a (mon.0) 1144 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:37.439 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:36.767262+0000 mon.a (mon.0) 1145 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:37.475 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2024-02-23T00:43:35.199051+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":151011826,"num_objects":278,"num_object_clones":0,"num_object_copies":834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":278,"num_whiteouts":0,"num_read":2379,"num_read_kb":50245,"num_write":1791,"num_write_kb":153106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":947,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2800000,"num_omap_keys":1000,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1901,"ondisk_log_size":1901,"up":975,"acting":975,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":1319,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2927728,"kb_used_data":456752,"kb_used_omap":0,"kb_used_meta":2470912,"kb_avail":747033488,"statfs":{"total":767960285184,"available":764962291712,"internally_reserved":0,"allocated":467714048,"data_stored":461186180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2530213888},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3832,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":8,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.002060"},"pg_stats":[{"pgid":"83.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:34.442775+0000","last_change":"2024-02-23T00:43:34.442775+0000","last_active":"2024-02-23T00:43:34.442775+0000","last_peered":"2024-02-23T00:43:34.442775+0000","last_clean":"2024-02-23T00:43:34.442775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:34.442775+0000","last_undegraded":"2024-02-23T00:43:34.442775+0000","last_fullsized":"2024-02-23T00:43:34.442775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_clean_scrub_stamp":"2024-02-23T00:43:34.442775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"83.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:34.442775+0000","last_change":"2024-02-23T00:43:34.442775+0000","last_active":"2024-02-23T00:43:34.442775+0000","last_peered":"2024-02-23T00:43:34.442775+0000","last_clean":"2024-02-23T00:43:34.442775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:34.442775+0000","last_undegraded":"2024-02-23T00:43:34.442775+0000","last_fullsized":"2024-02-23T00:43:34.442775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_clean_scrub_stamp":"2024-02-23T00:43:34.442775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"83.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:34.442775+0000","last_change":"2024-02-23T00:43:34.442775+0000","last_active":"2024-02-23T00:43:34.442775+0000","last_peered":"2024-02-23T00:43:34.442775+0000","last_clean":"2024-02-23T00:43:34.442775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:34.442775+0000","last_undegraded":"2024-02-23T00:43:34.442775+0000","last_fullsized":"2024-02-23T00:43:34.442775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_clean_scrub_stamp":"2024-02-23T00:43:34.442775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"83.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:34.442775+0000","last_change":"2024-02-23T00:43:34.442775+0000","last_active":"2024-02-23T00:43:34.442775+0000","last_peered":"2024-02-23T00:43:34.442775+0000","last_clean":"2024-02-23T00:43:34.442775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:34.442775+0000","last_undegraded":"2024-02-23T00:43:34.442775+0000","last_fullsized":"2024-02-23T00:43:34.442775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_clean_scrub_stamp":"2024-02-23T00:43:34.442775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"83.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:34.442775+0000","last_change":"2024-02-23T00:43:34.442775+0000","last_active":"2024-02-23T00:43:34.442775+0000","last_peered":"2024-02-23T00:43:34.442775+0000","last_clean":"2024-02-23T00:43:34.442775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:34.442775+0000","last_undegraded":"2024-02-23T00:43:34.442775+0000","last_fullsized":"2024-02-23T00:43:34.442775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_clean_scrub_stamp":"2024-02-23T00:43:34.442775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"83.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:34.442775+0000","last_change":"2024-02-23T00:43:34.442775+0000","last_active":"2024-02-23T00:43:34.442775+0000","last_peered":"2024-02-23T00:43:34.442775+0000","last_clean":"2024-02-23T00:43:34.442775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:34.442775+0000","last_undegraded":"2024-02-23T00:43:34.442775+0000","last_fullsized":"2024-02-23T00:43:34.442775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_clean_scrub_stamp":"2024-02-23T00:43:34.442775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"83.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:34.442775+0000","last_change":"2024-02-23T00:43:34.442775+0000","last_active":"2024-02-23T00:43:34.442775+0000","last_peered":"2024-02-23T00:43:34.442775+0000","last_clean":"2024-02-23T00:43:34.442775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:34.442775+0000","last_undegraded":"2024-02-23T00:43:34.442775+0000","last_fullsized":"2024-02-23T00:43:34.442775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_clean_scrub_stamp":"2024-02-23T00:43:34.442775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"83.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:34.442775+0000","last_change":"2024-02-23T00:43:34.442775+0000","last_active":"2024-02-23T00:43:34.442775+0000","last_peered":"2024-02-23T00:43:34.442775+0000","last_clean":"2024-02-23T00:43:34.442775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:34.442775+0000","last_undegraded":"2024-02-23T00:43:34.442775+0000","last_fullsized":"2024-02-23T00:43:34.442775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_clean_scrub_stamp":"2024-02-23T00:43:34.442775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"83.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:34.442775+0000","last_change":"2024-02-23T00:43:34.442775+0000","last_active":"2024-02-23T00:43:34.442775+0000","last_peered":"2024-02-23T00:43:34.442775+0000","last_clean":"2024-02-23T00:43:34.442775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:34.442775+0000","last_undegraded":"2024-02-23T00:43:34.442775+0000","last_fullsized":"2024-02-23T00:43:34.442775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_clean_scrub_stamp":"2024-02-23T00:43:34.442775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"83.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:34.442775+0000","last_change":"2024-02-23T00:43:34.442775+0000","last_active":"2024-02-23T00:43:34.442775+0000","last_peered":"2024-02-23T00:43:34.442775+0000","last_clean":"2024-02-23T00:43:34.442775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:34.442775+0000","last_undegraded":"2024-02-23T00:43:34.442775+0000","last_fullsized":"2024-02-23T00:43:34.442775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_clean_scrub_stamp":"2024-02-23T00:43:34.442775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"83.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:34.442775+0000","last_change":"2024-02-23T00:43:34.442775+0000","last_active":"2024-02-23T00:43:34.442775+0000","last_peered":"2024-02-23T00:43:34.442775+0000","last_clean":"2024-02-23T00:43:34.442775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:34.442775+0000","last_undegraded":"2024-02-23T00:43:34.442775+0000","last_fullsized":"2024-02-23T00:43:34.442775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_clean_scrub_stamp":"2024-02-23T00:43:34.442775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"83.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:34.442775+0000","last_change":"2024-02-23T00:43:34.442775+0000","last_active":"2024-02-23T00:43:34.442775+0000","last_peered":"2024-02-23T00:43:34.442775+0000","last_clean":"2024-02-23T00:43:34.442775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:34.442775+0000","last_undegraded":"2024-02-23T00:43:34.442775+0000","last_fullsized":"2024-02-23T00:43:34.442775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_clean_scrub_stamp":"2024-02-23T00:43:34.442775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"83.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:34.442775+0000","last_change":"2024-02-23T00:43:34.442775+0000","last_active":"2024-02-23T00:43:34.442775+0000","last_peered":"2024-02-23T00:43:34.442775+0000","last_clean":"2024-02-23T00:43:34.442775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:34.442775+0000","last_undegraded":"2024-02-23T00:43:34.442775+0000","last_fullsized":"2024-02-23T00:43:34.442775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_clean_scrub_stamp":"2024-02-23T00:43:34.442775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"83.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:34.442775+0000","last_change":"2024-02-23T00:43:34.442775+0000","last_active":"2024-02-23T00:43:34.442775+0000","last_peered":"2024-02-23T00:43:34.442775+0000","last_clean":"2024-02-23T00:43:34.442775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:34.442775+0000","last_undegraded":"2024-02-23T00:43:34.442775+0000","last_fullsized":"2024-02-23T00:43:34.442775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_clean_scrub_stamp":"2024-02-23T00:43:34.442775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"83.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:34.442775+0000","last_change":"2024-02-23T00:43:34.442775+0000","last_active":"2024-02-23T00:43:34.442775+0000","last_peered":"2024-02-23T00:43:34.442775+0000","last_clean":"2024-02-23T00:43:34.442775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:34.442775+0000","last_undegraded":"2024-02-23T00:43:34.442775+0000","last_fullsized":"2024-02-23T00:43:34.442775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_clean_scrub_stamp":"2024-02-23T00:43:34.442775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"83.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:34.442775+0000","last_change":"2024-02-23T00:43:34.442775+0000","last_active":"2024-02-23T00:43:34.442775+0000","last_peered":"2024-02-23T00:43:34.442775+0000","last_clean":"2024-02-23T00:43:34.442775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:34.442775+0000","last_undegraded":"2024-02-23T00:43:34.442775+0000","last_fullsized":"2024-02-23T00:43:34.442775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_clean_scrub_stamp":"2024-02-23T00:43:34.442775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"83.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:34.442775+0000","last_change":"2024-02-23T00:43:34.442775+0000","last_active":"2024-02-23T00:43:34.442775+0000","last_peered":"2024-02-23T00:43:34.442775+0000","last_clean":"2024-02-23T00:43:34.442775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:34.442775+0000","last_undegraded":"2024-02-23T00:43:34.442775+0000","last_fullsized":"2024-02-23T00:43:34.442775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_clean_scrub_stamp":"2024-02-23T00:43:34.442775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:31.122849+0000","last_change":"2024-02-23T00:43:31.170920+0000","last_active":"2024-02-23T00:43:31.122849+0000","last_peered":"2024-02-23T00:43:31.122849+0000","last_clean":"2024-02-23T00:43:31.122849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:31.122849+0000","last_undegraded":"2024-02-23T00:43:31.122849+0000","last_fullsized":"2024-02-23T00:43:31.122849+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:31.122849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:31.122849+0000","last_clean_scrub_stamp":"2024-02-23T00:43:31.122849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"83.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:34.442775+0000","last_change":"2024-02-23T00:43:34.442775+0000","last_active":"2024-02-23T00:43:34.442775+0000","last_peered":"2024-02-23T00:43:34.442775+0000","last_clean":"2024-02-23T00:43:34.442775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:34.442775+0000","last_undegraded":"2024-02-23T00:43:34.442775+0000","last_fullsized":"2024-02-23T00:43:34.442775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_clean_scrub_stamp":"2024-02-23T00:43:34.442775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:31.122849+0000","last_change":"2024-02-23T00:43:31.206653+0000","last_active":"2024-02-23T00:43:31.122849+0000","last_peered":"2024-02-23T00:43:31.122849+0000","last_clean":"2024-02-23T00:43:31.122849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:31.122849+0000","last_undegraded":"2024-02-23T00:43:31.122849+0000","last_fullsized":"2024-02-23T00:43:31.122849+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:31.122849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:31.122849+0000","last_clean_scrub_stamp":"2024-02-23T00:43:31.122849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"83.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:34.442775+0000","last_change":"2024-02-23T00:43:34.442775+0000","last_active":"2024-02-23T00:43:34.442775+0000","last_peered":"2024-02-23T00:43:34.442775+0000","last_clean":"2024-02-23T00:43:34.442775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:34.442775+0000","last_undegraded":"2024-02-23T00:43:34.442775+0000","last_fullsized":"2024-02-23T00:43:34.442775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_clean_scrub_stamp":"2024-02-23T00:43:34.442775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:31.122849+0000","last_change":"2024-02-23T00:43:31.152140+0000","last_active":"2024-02-23T00:43:31.122849+0000","last_peered":"2024-02-23T00:43:31.122849+0000","last_clean":"2024-02-23T00:43:31.122849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:31.122849+0000","last_undegraded":"2024-02-23T00:43:31.122849+0000","last_fullsized":"2024-02-23T00:43:31.122849+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:31.122849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:31.122849+0000","last_clean_scrub_stamp":"2024-02-23T00:43:31.122849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"83.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:34.442775+0000","last_change":"2024-02-23T00:43:34.442775+0000","last_active":"2024-02-23T00:43:34.442775+0000","last_peered":"2024-02-23T00:43:34.442775+0000","last_clean":"2024-02-23T00:43:34.442775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:34.442775+0000","last_undegraded":"2024-02-23T00:43:34.442775+0000","last_fullsized":"2024-02-23T00:43:34.442775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_clean_scrub_stamp":"2024-02-23T00:43:34.442775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:31.122849+0000","last_change":"2024-02-23T00:43:31.156168+0000","last_active":"2024-02-23T00:43:31.122849+0000","last_peered":"2024-02-23T00:43:31.122849+0000","last_clean":"2024-02-23T00:43:31.122849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:31.122849+0000","last_undegraded":"2024-02-23T00:43:31.122849+0000","last_fullsized":"2024-02-23T00:43:31.122849+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:31.122849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:31.122849+0000","last_clean_scrub_stamp":"2024-02-23T00:43:31.122849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"83.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:34.442775+0000","last_change":"2024-02-23T00:43:34.442775+0000","last_active":"2024-02-23T00:43:34.442775+0000","last_peered":"2024-02-23T00:43:34.442775+0000","last_clean":"2024-02-23T00:43:34.442775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:34.442775+0000","last_undegraded":"2024-02-23T00:43:34.442775+0000","last_fullsized":"2024-02-23T00:43:34.442775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_clean_scrub_stamp":"2024-02-23T00:43:34.442775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:31.122849+0000","last_change":"2024-02-23T00:43:31.160430+0000","last_active":"2024-02-23T00:43:31.122849+0000","last_peered":"2024-02-23T00:43:31.122849+0000","last_clean":"2024-02-23T00:43:31.122849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:31.122849+0000","last_undegraded":"2024-02-23T00:43:31.122849+0000","last_fullsized":"2024-02-23T00:43:31.122849+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:31.122849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:31.122849+0000","last_clean_scrub_stamp":"2024-02-23T00:43:31.122849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"83.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:34.442775+0000","last_change":"2024-02-23T00:43:34.442775+0000","last_active":"2024-02-23T00:43:34.442775+0000","last_peered":"2024-02-23T00:43:34.442775+0000","last_clean":"2024-02-23T00:43:34.442775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:34.442775+0000","last_undegraded":"2024-02-23T00:43:34.442775+0000","last_fullsized":"2024-02-23T00:43:34.442775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_clean_scrub_stamp":"2024-02-23T00:43:34.442775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:31.122849+0000","last_change":"2024-02-23T00:43:31.167853+0000","last_active":"2024-02-23T00:43:31.122849+0000","last_peered":"2024-02-23T00:43:31.122849+0000","last_clean":"2024-02-23T00:43:31.122849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:31.122849+0000","last_undegraded":"2024-02-23T00:43:31.122849+0000","last_fullsized":"2024-02-23T00:43:31.122849+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:31.122849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:31.122849+0000","last_clean_scrub_stamp":"2024-02-23T00:43:31.122849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"83.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:34.442775+0000","last_change":"2024-02-23T00:43:34.442775+0000","last_active":"2024-02-23T00:43:34.442775+0000","last_peered":"2024-02-23T00:43:34.442775+0000","last_clean":"2024-02-23T00:43:34.442775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:34.442775+0000","last_undegraded":"2024-02-23T00:43:34.442775+0000","last_fullsized":"2024-02-23T00:43:34.442775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_clean_scrub_stamp":"2024-02-23T00:43:34.442775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:31.122849+0000","last_change":"2024-02-23T00:43:31.156847+0000","last_active":"2024-02-23T00:43:31.122849+0000","last_peered":"2024-02-23T00:43:31.122849+0000","last_clean":"2024-02-23T00:43:31.122849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:31.122849+0000","last_undegraded":"2024-02-23T00:43:31.122849+0000","last_fullsized":"2024-02-23T00:43:31.122849+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:31.122849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:31.122849+0000","last_clean_scrub_stamp":"2024-02-23T00:43:31.122849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"83.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:34.442775+0000","last_change":"2024-02-23T00:43:34.442775+0000","last_active":"2024-02-23T00:43:34.442775+0000","last_peered":"2024-02-23T00:43:34.442775+0000","last_clean":"2024-02-23T00:43:34.442775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:34.442775+0000","last_undegraded":"2024-02-23T00:43:34.442775+0000","last_fullsized":"2024-02-23T00:43:34.442775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_clean_scrub_stamp":"2024-02-23T00:43:34.442775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:43:31.122849+0000","last_change":"2024-02-23T00:43:31.163281+0000","last_active":"2024-02-23T00:43:31.122849+0000","last_peered":"2024-02-23T00:43:31.122849+0000","last_clean":"2024-02-23T00:43:31.122849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:31.122849+0000","last_undegraded":"2024-02-23T00:43:31.122849+0000","last_fullsized":"2024-02-23T00:43:31.122849+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:31.122849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:31.122849+0000","last_clean_scrub_stamp":"2024-02-23T00:43:31.122849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"83.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:34.442775+0000","last_change":"2024-02-23T00:43:34.442775+0000","last_active":"2024-02-23T00:43:34.442775+0000","last_peered":"2024-02-23T00:43:34.442775+0000","last_clean":"2024-02-23T00:43:34.442775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:34.442775+0000","last_undegraded":"2024-02-23T00:43:34.442775+0000","last_fullsized":"2024-02-23T00:43:34.442775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_clean_scrub_stamp":"2024-02-23T00:43:34.442775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"83.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:34.442775+0000","last_change":"2024-02-23T00:43:34.442775+0000","last_active":"2024-02-23T00:43:34.442775+0000","last_peered":"2024-02-23T00:43:34.442775+0000","last_clean":"2024-02-23T00:43:34.442775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:34.442775+0000","last_undegraded":"2024-02-23T00:43:34.442775+0000","last_fullsized":"2024-02-23T00:43:34.442775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_clean_scrub_stamp":"2024-02-23T00:43:34.442775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"83.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:34.442775+0000","last_change":"2024-02-23T00:43:34.442775+0000","last_active":"2024-02-23T00:43:34.442775+0000","last_peered":"2024-02-23T00:43:34.442775+0000","last_clean":"2024-02-23T00:43:34.442775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:34.442775+0000","last_undegraded":"2024-02-23T00:43:34.442775+0000","last_fullsized":"2024-02-23T00:43:34.442775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_clean_scrub_stamp":"2024-02-23T00:43:34.442775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"83.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:34.442775+0000","last_change":"2024-02-23T00:43:34.442775+0000","last_active":"2024-02-23T00:43:34.442775+0000","last_peered":"2024-02-23T00:43:34.442775+0000","last_clean":"2024-02-23T00:43:34.442775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:34.442775+0000","last_undegraded":"2024-02-23T00:43:34.442775+0000","last_fullsized":"2024-02-23T00:43:34.442775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_clean_scrub_stamp":"2024-02-23T00:43:34.442775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"83.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:34.442775+0000","last_change":"2024-02-23T00:43:34.442775+0000","last_active":"2024-02-23T00:43:34.442775+0000","last_peered":"2024-02-23T00:43:34.442775+0000","last_clean":"2024-02-23T00:43:34.442775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:34.442775+0000","last_undegraded":"2024-02-23T00:43:34.442775+0000","last_fullsized":"2024-02-23T00:43:34.442775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_clean_scrub_stamp":"2024-02-23T00:43:34.442775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"83.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:34.442775+0000","last_change":"2024-02-23T00:43:34.442775+0000","last_active":"2024-02-23T00:43:34.442775+0000","last_peered":"2024-02-23T00:43:34.442775+0000","last_clean":"2024-02-23T00:43:34.442775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:34.442775+0000","last_undegraded":"2024-02-23T00:43:34.442775+0000","last_fullsized":"2024-02-23T00:43:34.442775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_clean_scrub_stamp":"2024-02-23T00:43:34.442775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"83.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:34.442775+0000","last_change":"2024-02-23T00:43:34.442775+0000","last_active":"2024-02-23T00:43:34.442775+0000","last_peered":"2024-02-23T00:43:34.442775+0000","last_clean":"2024-02-23T00:43:34.442775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:34.442775+0000","last_undegraded":"2024-02-23T00:43:34.442775+0000","last_fullsized":"2024-02-23T00:43:34.442775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_clean_scrub_stamp":"2024-02-23T00:43:34.442775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"83.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:43:34.442775+0000","last_change":"2024-02-23T00:43:34.442775+0000","last_active":"2024-02-23T00:43:34.442775+0000","last_peered":"2024-02-23T00:43:34.442775+0000","last_clean":"2024-02-23T00:43:34.442775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:43:34.442775+0000","last_undegraded":"2024-02-23T00:43:34.442775+0000","last_fullsized":"2024-02-23T00:43:34.442775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:34.442775+0000","last_clean_scrub_stamp":"2024-02-23T00:43:34.442775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"63.1","version":"0'0","reported_seq":36,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.911403+0000","last_change":"2024-02-23T00:43:29.911403+0000","last_active":"2024-02-23T00:43:29.911403+0000","last_peered":"2024-02-23T00:43:29.911403+0000","last_clean":"2024-02-23T00:43:29.911403+0000","last_became_active":"2024-02-23T00:43:18.993940+0000","last_became_peered":"2024-02-23T00:43:18.993940+0000","last_unstale":"2024-02-23T00:43:29.911403+0000","last_undegraded":"2024-02-23T00:43:29.911403+0000","last_fullsized":"2024-02-23T00:43:29.911403+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1],"acting":[4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"62.0","version":"130'1","reported_seq":23,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.913435+0000","last_change":"2024-02-23T00:43:17.855674+0000","last_active":"2024-02-23T00:43:29.913435+0000","last_peered":"2024-02-23T00:43:29.913435+0000","last_clean":"2024-02-23T00:43:29.913435+0000","last_became_active":"2024-02-23T00:43:17.855316+0000","last_became_peered":"2024-02-23T00:43:17.855316+0000","last_unstale":"2024-02-23T00:43:29.913435+0000","last_undegraded":"2024-02-23T00:43:29.913435+0000","last_fullsized":"2024-02-23T00:43:29.913435+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"47.11","version":"0'0","reported_seq":29,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.895498+0000","last_change":"2024-02-23T00:43:08.855287+0000","last_active":"2024-02-23T00:43:29.895498+0000","last_peered":"2024-02-23T00:43:29.895498+0000","last_clean":"2024-02-23T00:43:29.895498+0000","last_became_active":"2024-02-23T00:43:08.854959+0000","last_became_peered":"2024-02-23T00:43:08.854959+0000","last_unstale":"2024-02-23T00:43:29.895498+0000","last_undegraded":"2024-02-23T00:43:29.895498+0000","last_fullsized":"2024-02-23T00:43:29.895498+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,7],"acting":[4,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"63.0","version":"137'9","reported_seq":50,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:30.648074+0000","last_change":"2024-02-23T00:43:30.648074+0000","last_active":"2024-02-23T00:43:30.648074+0000","last_peered":"2024-02-23T00:43:30.648074+0000","last_clean":"2024-02-23T00:43:30.648074+0000","last_became_active":"2024-02-23T00:43:19.009369+0000","last_became_peered":"2024-02-23T00:43:19.009369+0000","last_unstale":"2024-02-23T00:43:30.648074+0000","last_undegraded":"2024-02-23T00:43:30.648074+0000","last_fullsized":"2024-02-23T00:43:30.648074+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":384,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"62.1","version":"0'0","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.926522+0000","last_change":"2024-02-23T00:43:17.852026+0000","last_active":"2024-02-23T00:43:29.926522+0000","last_peered":"2024-02-23T00:43:29.926522+0000","last_clean":"2024-02-23T00:43:29.926522+0000","last_became_active":"2024-02-23T00:43:17.851866+0000","last_became_peered":"2024-02-23T00:43:17.851866+0000","last_unstale":"2024-02-23T00:43:29.926522+0000","last_undegraded":"2024-02-23T00:43:29.926522+0000","last_fullsized":"2024-02-23T00:43:29.926522+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"47.10","version":"0'0","reported_seq":29,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.944990+0000","last_change":"2024-02-23T00:43:08.889966+0000","last_active":"2024-02-23T00:43:29.944990+0000","last_peered":"2024-02-23T00:43:29.944990+0000","last_clean":"2024-02-23T00:43:29.944990+0000","last_became_active":"2024-02-23T00:43:08.889086+0000","last_became_peered":"2024-02-23T00:43:08.889086+0000","last_unstale":"2024-02-23T00:43:29.944990+0000","last_undegraded":"2024-02-23T00:43:29.944990+0000","last_fullsized":"2024-02-23T00:43:29.944990+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"63.3","version":"0'0","reported_seq":36,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.906109+0000","last_change":"2024-02-23T00:43:29.906109+0000","last_active":"2024-02-23T00:43:29.906109+0000","last_peered":"2024-02-23T00:43:29.906109+0000","last_clean":"2024-02-23T00:43:29.906109+0000","last_became_active":"2024-02-23T00:43:19.004347+0000","last_became_peered":"2024-02-23T00:43:19.004347+0000","last_unstale":"2024-02-23T00:43:29.906109+0000","last_undegraded":"2024-02-23T00:43:29.906109+0000","last_fullsized":"2024-02-23T00:43:29.906109+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"62.2","version":"0'0","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.907420+0000","last_change":"2024-02-23T00:43:17.805103+0000","last_active":"2024-02-23T00:43:29.907420+0000","last_peered":"2024-02-23T00:43:29.907420+0000","last_clean":"2024-02-23T00:43:29.907420+0000","last_became_active":"2024-02-23T00:43:17.804792+0000","last_became_peered":"2024-02-23T00:43:17.804792+0000","last_unstale":"2024-02-23T00:43:29.907420+0000","last_undegraded":"2024-02-23T00:43:29.907420+0000","last_fullsized":"2024-02-23T00:43:29.907420+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"47.13","version":"0'0","reported_seq":29,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.915814+0000","last_change":"2024-02-23T00:43:08.886490+0000","last_active":"2024-02-23T00:43:29.915814+0000","last_peered":"2024-02-23T00:43:29.915814+0000","last_clean":"2024-02-23T00:43:29.915814+0000","last_became_active":"2024-02-23T00:43:08.886200+0000","last_became_peered":"2024-02-23T00:43:08.886200+0000","last_unstale":"2024-02-23T00:43:29.915814+0000","last_undegraded":"2024-02-23T00:43:29.915814+0000","last_fullsized":"2024-02-23T00:43:29.915814+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"63.2","version":"0'0","reported_seq":36,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.924443+0000","last_change":"2024-02-23T00:43:29.924443+0000","last_active":"2024-02-23T00:43:29.924443+0000","last_peered":"2024-02-23T00:43:29.924443+0000","last_clean":"2024-02-23T00:43:29.924443+0000","last_became_active":"2024-02-23T00:43:18.997818+0000","last_became_peered":"2024-02-23T00:43:18.997818+0000","last_unstale":"2024-02-23T00:43:29.924443+0000","last_undegraded":"2024-02-23T00:43:29.924443+0000","last_fullsized":"2024-02-23T00:43:29.924443+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"62.3","version":"0'0","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.927125+0000","last_change":"2024-02-23T00:43:17.834974+0000","last_active":"2024-02-23T00:43:29.927125+0000","last_peered":"2024-02-23T00:43:29.927125+0000","last_clean":"2024-02-23T00:43:29.927125+0000","last_became_active":"2024-02-23T00:43:17.834686+0000","last_became_peered":"2024-02-23T00:43:17.834686+0000","last_unstale":"2024-02-23T00:43:29.927125+0000","last_undegraded":"2024-02-23T00:43:29.927125+0000","last_fullsized":"2024-02-23T00:43:29.927125+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"47.12","version":"0'0","reported_seq":29,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.934988+0000","last_change":"2024-02-23T00:43:08.887138+0000","last_active":"2024-02-23T00:43:29.934988+0000","last_peered":"2024-02-23T00:43:29.934988+0000","last_clean":"2024-02-23T00:43:29.934988+0000","last_became_active":"2024-02-23T00:43:08.886831+0000","last_became_peered":"2024-02-23T00:43:08.886831+0000","last_unstale":"2024-02-23T00:43:29.934988+0000","last_undegraded":"2024-02-23T00:43:29.934988+0000","last_fullsized":"2024-02-23T00:43:29.934988+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"63.5","version":"0'0","reported_seq":36,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.911769+0000","last_change":"2024-02-23T00:43:29.911769+0000","last_active":"2024-02-23T00:43:29.911769+0000","last_peered":"2024-02-23T00:43:29.911769+0000","last_clean":"2024-02-23T00:43:29.911769+0000","last_became_active":"2024-02-23T00:43:18.964049+0000","last_became_peered":"2024-02-23T00:43:18.964049+0000","last_unstale":"2024-02-23T00:43:29.911769+0000","last_undegraded":"2024-02-23T00:43:29.911769+0000","last_fullsized":"2024-02-23T00:43:29.911769+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"62.4","version":"0'0","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.933222+0000","last_change":"2024-02-23T00:43:17.810705+0000","last_active":"2024-02-23T00:43:29.933222+0000","last_peered":"2024-02-23T00:43:29.933222+0000","last_clean":"2024-02-23T00:43:29.933222+0000","last_became_active":"2024-02-23T00:43:17.810343+0000","last_became_peered":"2024-02-23T00:43:17.810343+0000","last_unstale":"2024-02-23T00:43:29.933222+0000","last_undegraded":"2024-02-23T00:43:29.933222+0000","last_fullsized":"2024-02-23T00:43:29.933222+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"47.15","version":"0'0","reported_seq":29,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.904155+0000","last_change":"2024-02-23T00:43:08.880115+0000","last_active":"2024-02-23T00:43:29.904155+0000","last_peered":"2024-02-23T00:43:29.904155+0000","last_clean":"2024-02-23T00:43:29.904155+0000","last_became_active":"2024-02-23T00:43:08.879758+0000","last_became_peered":"2024-02-23T00:43:08.879758+0000","last_unstale":"2024-02-23T00:43:29.904155+0000","last_undegraded":"2024-02-23T00:43:29.904155+0000","last_fullsized":"2024-02-23T00:43:29.904155+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"63.4","version":"0'0","reported_seq":36,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.910402+0000","last_change":"2024-02-23T00:43:29.910402+0000","last_active":"2024-02-23T00:43:29.910402+0000","last_peered":"2024-02-23T00:43:29.910402+0000","last_clean":"2024-02-23T00:43:29.910402+0000","last_became_active":"2024-02-23T00:43:18.955783+0000","last_became_peered":"2024-02-23T00:43:18.955783+0000","last_unstale":"2024-02-23T00:43:29.910402+0000","last_undegraded":"2024-02-23T00:43:29.910402+0000","last_fullsized":"2024-02-23T00:43:29.910402+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"62.5","version":"0'0","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.900411+0000","last_change":"2024-02-23T00:43:17.855648+0000","last_active":"2024-02-23T00:43:29.900411+0000","last_peered":"2024-02-23T00:43:29.900411+0000","last_clean":"2024-02-23T00:43:29.900411+0000","last_became_active":"2024-02-23T00:43:17.855339+0000","last_became_peered":"2024-02-23T00:43:17.855339+0000","last_unstale":"2024-02-23T00:43:29.900411+0000","last_undegraded":"2024-02-23T00:43:29.900411+0000","last_fullsized":"2024-02-23T00:43:29.900411+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"47.14","version":"120'1","reported_seq":31,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.896815+0000","last_change":"2024-02-23T00:43:09.896503+0000","last_active":"2024-02-23T00:43:29.896815+0000","last_peered":"2024-02-23T00:43:29.896815+0000","last_clean":"2024-02-23T00:43:29.896815+0000","last_became_active":"2024-02-23T00:43:09.895720+0000","last_became_peered":"2024-02-23T00:43:09.895720+0000","last_unstale":"2024-02-23T00:43:29.896815+0000","last_undegraded":"2024-02-23T00:43:29.896815+0000","last_fullsized":"2024-02-23T00:43:29.896815+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"63.7","version":"0'0","reported_seq":36,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.966772+0000","last_change":"2024-02-23T00:43:29.966772+0000","last_active":"2024-02-23T00:43:29.966772+0000","last_peered":"2024-02-23T00:43:29.966772+0000","last_clean":"2024-02-23T00:43:29.966772+0000","last_became_active":"2024-02-23T00:43:18.985022+0000","last_became_peered":"2024-02-23T00:43:18.985022+0000","last_unstale":"2024-02-23T00:43:29.966772+0000","last_undegraded":"2024-02-23T00:43:29.966772+0000","last_fullsized":"2024-02-23T00:43:29.966772+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"62.6","version":"0'0","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.907299+0000","last_change":"2024-02-23T00:43:17.839129+0000","last_active":"2024-02-23T00:43:29.907299+0000","last_peered":"2024-02-23T00:43:29.907299+0000","last_clean":"2024-02-23T00:43:29.907299+0000","last_became_active":"2024-02-23T00:43:17.838838+0000","last_became_peered":"2024-02-23T00:43:17.838838+0000","last_unstale":"2024-02-23T00:43:29.907299+0000","last_undegraded":"2024-02-23T00:43:29.907299+0000","last_fullsized":"2024-02-23T00:43:29.907299+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"47.17","version":"0'0","reported_seq":29,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.912369+0000","last_change":"2024-02-23T00:43:09.916378+0000","last_active":"2024-02-23T00:43:29.912369+0000","last_peered":"2024-02-23T00:43:29.912369+0000","last_clean":"2024-02-23T00:43:29.912369+0000","last_became_active":"2024-02-23T00:43:09.916022+0000","last_became_peered":"2024-02-23T00:43:09.916022+0000","last_unstale":"2024-02-23T00:43:29.912369+0000","last_undegraded":"2024-02-23T00:43:29.912369+0000","last_fullsized":"2024-02-23T00:43:29.912369+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"63.6","version":"0'0","reported_seq":36,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.958893+0000","last_change":"2024-02-23T00:43:29.958893+0000","last_active":"2024-02-23T00:43:29.958893+0000","last_peered":"2024-02-23T00:43:29.958893+0000","last_clean":"2024-02-23T00:43:29.958893+0000","last_became_active":"2024-02-23T00:43:19.000234+0000","last_became_peered":"2024-02-23T00:43:19.000234+0000","last_unstale":"2024-02-23T00:43:29.958893+0000","last_undegraded":"2024-02-23T00:43:29.958893+0000","last_fullsized":"2024-02-23T00:43:29.958893+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"62.7","version":"0'0","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.955513+0000","last_change":"2024-02-23T00:43:17.847092+0000","last_active":"2024-02-23T00:43:29.955513+0000","last_peered":"2024-02-23T00:43:29.955513+0000","last_clean":"2024-02-23T00:43:29.955513+0000","last_became_active":"2024-02-23T00:43:17.846548+0000","last_became_peered":"2024-02-23T00:43:17.846548+0000","last_unstale":"2024-02-23T00:43:29.955513+0000","last_undegraded":"2024-02-23T00:43:29.955513+0000","last_fullsized":"2024-02-23T00:43:29.955513+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"47.16","version":"0'0","reported_seq":29,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.898046+0000","last_change":"2024-02-23T00:43:08.887758+0000","last_active":"2024-02-23T00:43:29.898046+0000","last_peered":"2024-02-23T00:43:29.898046+0000","last_clean":"2024-02-23T00:43:29.898046+0000","last_became_active":"2024-02-23T00:43:08.887509+0000","last_became_peered":"2024-02-23T00:43:08.887509+0000","last_unstale":"2024-02-23T00:43:29.898046+0000","last_undegraded":"2024-02-23T00:43:29.898046+0000","last_fullsized":"2024-02-23T00:43:29.898046+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"63.9","version":"0'0","reported_seq":36,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.954642+0000","last_change":"2024-02-23T00:43:29.954642+0000","last_active":"2024-02-23T00:43:29.954642+0000","last_peered":"2024-02-23T00:43:29.954642+0000","last_clean":"2024-02-23T00:43:29.954642+0000","last_became_active":"2024-02-23T00:43:19.008756+0000","last_became_peered":"2024-02-23T00:43:19.008756+0000","last_unstale":"2024-02-23T00:43:29.954642+0000","last_undegraded":"2024-02-23T00:43:29.954642+0000","last_fullsized":"2024-02-23T00:43:29.954642+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"62.8","version":"0'0","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.913338+0000","last_change":"2024-02-23T00:43:17.842841+0000","last_active":"2024-02-23T00:43:29.913338+0000","last_peered":"2024-02-23T00:43:29.913338+0000","last_clean":"2024-02-23T00:43:29.913338+0000","last_became_active":"2024-02-23T00:43:17.842613+0000","last_became_peered":"2024-02-23T00:43:17.842613+0000","last_unstale":"2024-02-23T00:43:29.913338+0000","last_undegraded":"2024-02-23T00:43:29.913338+0000","last_fullsized":"2024-02-23T00:43:29.913338+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"47.19","version":"0'0","reported_seq":29,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.905884+0000","last_change":"2024-02-23T00:43:09.914057+0000","last_active":"2024-02-23T00:43:29.905884+0000","last_peered":"2024-02-23T00:43:29.905884+0000","last_clean":"2024-02-23T00:43:29.905884+0000","last_became_active":"2024-02-23T00:43:09.913739+0000","last_became_peered":"2024-02-23T00:43:09.913739+0000","last_unstale":"2024-02-23T00:43:29.905884+0000","last_undegraded":"2024-02-23T00:43:29.905884+0000","last_fullsized":"2024-02-23T00:43:29.905884+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"63.8","version":"0'0","reported_seq":36,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.961958+0000","last_change":"2024-02-23T00:43:29.961958+0000","last_active":"2024-02-23T00:43:29.961958+0000","last_peered":"2024-02-23T00:43:29.961958+0000","last_clean":"2024-02-23T00:43:29.961958+0000","last_became_active":"2024-02-23T00:43:19.031721+0000","last_became_peered":"2024-02-23T00:43:19.031721+0000","last_unstale":"2024-02-23T00:43:29.961958+0000","last_undegraded":"2024-02-23T00:43:29.961958+0000","last_fullsized":"2024-02-23T00:43:29.961958+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"62.9","version":"0'0","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.895340+0000","last_change":"2024-02-23T00:43:17.834630+0000","last_active":"2024-02-23T00:43:29.895340+0000","last_peered":"2024-02-23T00:43:29.895340+0000","last_clean":"2024-02-23T00:43:29.895340+0000","last_became_active":"2024-02-23T00:43:17.834297+0000","last_became_peered":"2024-02-23T00:43:17.834297+0000","last_unstale":"2024-02-23T00:43:29.895340+0000","last_undegraded":"2024-02-23T00:43:29.895340+0000","last_fullsized":"2024-02-23T00:43:29.895340+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"47.18","version":"0'0","reported_seq":29,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.913709+0000","last_change":"2024-02-23T00:43:08.861765+0000","last_active":"2024-02-23T00:43:29.913709+0000","last_peered":"2024-02-23T00:43:29.913709+0000","last_clean":"2024-02-23T00:43:29.913709+0000","last_became_active":"2024-02-23T00:43:08.861457+0000","last_became_peered":"2024-02-23T00:43:08.861457+0000","last_unstale":"2024-02-23T00:43:29.913709+0000","last_undegraded":"2024-02-23T00:43:29.913709+0000","last_fullsized":"2024-02-23T00:43:29.913709+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"63.b","version":"0'0","reported_seq":36,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.916717+0000","last_change":"2024-02-23T00:43:29.916717+0000","last_active":"2024-02-23T00:43:29.916717+0000","last_peered":"2024-02-23T00:43:29.916717+0000","last_clean":"2024-02-23T00:43:29.916717+0000","last_became_active":"2024-02-23T00:43:19.004699+0000","last_became_peered":"2024-02-23T00:43:19.004699+0000","last_unstale":"2024-02-23T00:43:29.916717+0000","last_undegraded":"2024-02-23T00:43:29.916717+0000","last_fullsized":"2024-02-23T00:43:29.916717+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"62.a","version":"0'0","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.910538+0000","last_change":"2024-02-23T00:43:17.839451+0000","last_active":"2024-02-23T00:43:29.910538+0000","last_peered":"2024-02-23T00:43:29.910538+0000","last_clean":"2024-02-23T00:43:29.910538+0000","last_became_active":"2024-02-23T00:43:17.839142+0000","last_became_peered":"2024-02-23T00:43:17.839142+0000","last_unstale":"2024-02-23T00:43:29.910538+0000","last_undegraded":"2024-02-23T00:43:29.910538+0000","last_fullsized":"2024-02-23T00:43:29.910538+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"47.1b","version":"0'0","reported_seq":29,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.922384+0000","last_change":"2024-02-23T00:43:08.889561+0000","last_active":"2024-02-23T00:43:29.922384+0000","last_peered":"2024-02-23T00:43:29.922384+0000","last_clean":"2024-02-23T00:43:29.922384+0000","last_became_active":"2024-02-23T00:43:08.889268+0000","last_became_peered":"2024-02-23T00:43:08.889268+0000","last_unstale":"2024-02-23T00:43:29.922384+0000","last_undegraded":"2024-02-23T00:43:29.922384+0000","last_fullsized":"2024-02-23T00:43:29.922384+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"63.a","version":"0'0","reported_seq":36,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.905170+0000","last_change":"2024-02-23T00:43:29.905170+0000","last_active":"2024-02-23T00:43:29.905170+0000","last_peered":"2024-02-23T00:43:29.905170+0000","last_clean":"2024-02-23T00:43:29.905170+0000","last_became_active":"2024-02-23T00:43:18.994201+0000","last_became_peered":"2024-02-23T00:43:18.994201+0000","last_unstale":"2024-02-23T00:43:29.905170+0000","last_undegraded":"2024-02-23T00:43:29.905170+0000","last_fullsized":"2024-02-23T00:43:29.905170+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"62.b","version":"0'0","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.915212+0000","last_change":"2024-02-23T00:43:17.855906+0000","last_active":"2024-02-23T00:43:29.915212+0000","last_peered":"2024-02-23T00:43:29.915212+0000","last_clean":"2024-02-23T00:43:29.915212+0000","last_became_active":"2024-02-23T00:43:17.855321+0000","last_became_peered":"2024-02-23T00:43:17.855321+0000","last_unstale":"2024-02-23T00:43:29.915212+0000","last_undegraded":"2024-02-23T00:43:29.915212+0000","last_fullsized":"2024-02-23T00:43:29.915212+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"47.1a","version":"0'0","reported_seq":29,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.897818+0000","last_change":"2024-02-23T00:43:08.879558+0000","last_active":"2024-02-23T00:43:29.897818+0000","last_peered":"2024-02-23T00:43:29.897818+0000","last_clean":"2024-02-23T00:43:29.897818+0000","last_became_active":"2024-02-23T00:43:08.879255+0000","last_became_peered":"2024-02-23T00:43:08.879255+0000","last_unstale":"2024-02-23T00:43:29.897818+0000","last_undegraded":"2024-02-23T00:43:29.897818+0000","last_fullsized":"2024-02-23T00:43:29.897818+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"63.d","version":"0'0","reported_seq":36,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.916693+0000","last_change":"2024-02-23T00:43:29.916693+0000","last_active":"2024-02-23T00:43:29.916693+0000","last_peered":"2024-02-23T00:43:29.916693+0000","last_clean":"2024-02-23T00:43:29.916693+0000","last_became_active":"2024-02-23T00:43:19.004404+0000","last_became_peered":"2024-02-23T00:43:19.004404+0000","last_unstale":"2024-02-23T00:43:29.916693+0000","last_undegraded":"2024-02-23T00:43:29.916693+0000","last_fullsized":"2024-02-23T00:43:29.916693+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"62.c","version":"0'0","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.894894+0000","last_change":"2024-02-23T00:43:17.829651+0000","last_active":"2024-02-23T00:43:29.894894+0000","last_peered":"2024-02-23T00:43:29.894894+0000","last_clean":"2024-02-23T00:43:29.894894+0000","last_became_active":"2024-02-23T00:43:17.829044+0000","last_became_peered":"2024-02-23T00:43:17.829044+0000","last_unstale":"2024-02-23T00:43:29.894894+0000","last_undegraded":"2024-02-23T00:43:29.894894+0000","last_fullsized":"2024-02-23T00:43:29.894894+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"47.1d","version":"0'0","reported_seq":29,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.900804+0000","last_change":"2024-02-23T00:43:08.876647+0000","last_active":"2024-02-23T00:43:29.900804+0000","last_peered":"2024-02-23T00:43:29.900804+0000","last_clean":"2024-02-23T00:43:29.900804+0000","last_became_active":"2024-02-23T00:43:08.876321+0000","last_became_peered":"2024-02-23T00:43:08.876321+0000","last_unstale":"2024-02-23T00:43:29.900804+0000","last_undegraded":"2024-02-23T00:43:29.900804+0000","last_fullsized":"2024-02-23T00:43:29.900804+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"63.c","version":"0'0","reported_seq":36,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.910970+0000","last_change":"2024-02-23T00:43:29.910970+0000","last_active":"2024-02-23T00:43:29.910970+0000","last_peered":"2024-02-23T00:43:29.910970+0000","last_clean":"2024-02-23T00:43:29.910970+0000","last_became_active":"2024-02-23T00:43:18.963782+0000","last_became_peered":"2024-02-23T00:43:18.963782+0000","last_unstale":"2024-02-23T00:43:29.910970+0000","last_undegraded":"2024-02-23T00:43:29.910970+0000","last_fullsized":"2024-02-23T00:43:29.910970+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,4],"acting":[6,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"62.d","version":"0'0","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.900326+0000","last_change":"2024-02-23T00:43:17.841913+0000","last_active":"2024-02-23T00:43:29.900326+0000","last_peered":"2024-02-23T00:43:29.900326+0000","last_clean":"2024-02-23T00:43:29.900326+0000","last_became_active":"2024-02-23T00:43:17.841614+0000","last_became_peered":"2024-02-23T00:43:17.841614+0000","last_unstale":"2024-02-23T00:43:29.900326+0000","last_undegraded":"2024-02-23T00:43:29.900326+0000","last_fullsized":"2024-02-23T00:43:29.900326+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"47.1c","version":"0'0","reported_seq":29,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.931874+0000","last_change":"2024-02-23T00:43:08.891941+0000","last_active":"2024-02-23T00:43:29.931874+0000","last_peered":"2024-02-23T00:43:29.931874+0000","last_clean":"2024-02-23T00:43:29.931874+0000","last_became_active":"2024-02-23T00:43:08.891349+0000","last_became_peered":"2024-02-23T00:43:08.891349+0000","last_unstale":"2024-02-23T00:43:29.931874+0000","last_undegraded":"2024-02-23T00:43:29.931874+0000","last_fullsized":"2024-02-23T00:43:29.931874+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"63.f","version":"0'0","reported_seq":36,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.970050+0000","last_change":"2024-02-23T00:43:29.970050+0000","last_active":"2024-02-23T00:43:29.970050+0000","last_peered":"2024-02-23T00:43:29.970050+0000","last_clean":"2024-02-23T00:43:29.970050+0000","last_became_active":"2024-02-23T00:43:19.012581+0000","last_became_peered":"2024-02-23T00:43:19.012581+0000","last_unstale":"2024-02-23T00:43:29.970050+0000","last_undegraded":"2024-02-23T00:43:29.970050+0000","last_fullsized":"2024-02-23T00:43:29.970050+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"62.e","version":"0'0","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.930869+0000","last_change":"2024-02-23T00:43:17.846111+0000","last_active":"2024-02-23T00:43:29.930869+0000","last_peered":"2024-02-23T00:43:29.930869+0000","last_clean":"2024-02-23T00:43:29.930869+0000","last_became_active":"2024-02-23T00:43:17.845956+0000","last_became_peered":"2024-02-23T00:43:17.845956+0000","last_unstale":"2024-02-23T00:43:29.930869+0000","last_undegraded":"2024-02-23T00:43:29.930869+0000","last_fullsized":"2024-02-23T00:43:29.930869+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"47.1f","version":"0'0","reported_seq":29,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.904247+0000","last_change":"2024-02-23T00:43:08.895615+0000","last_active":"2024-02-23T00:43:29.904247+0000","last_peered":"2024-02-23T00:43:29.904247+0000","last_clean":"2024-02-23T00:43:29.904247+0000","last_became_active":"2024-02-23T00:43:08.895310+0000","last_became_peered":"2024-02-23T00:43:08.895310+0000","last_unstale":"2024-02-23T00:43:29.904247+0000","last_undegraded":"2024-02-23T00:43:29.904247+0000","last_fullsized":"2024-02-23T00:43:29.904247+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"63.e","version":"139'5","reported_seq":42,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-02-23T00:43:31.263321+0000","last_change":"2024-02-23T00:43:29.947027+0000","last_active":"2024-02-23T00:43:31.263321+0000","last_peered":"2024-02-23T00:43:31.263321+0000","last_clean":"2024-02-23T00:43:31.263321+0000","last_became_active":"2024-02-23T00:43:19.008061+0000","last_became_peered":"2024-02-23T00:43:19.008061+0000","last_unstale":"2024-02-23T00:43:31.263321+0000","last_undegraded":"2024-02-23T00:43:31.263321+0000","last_fullsized":"2024-02-23T00:43:31.263321+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1152,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"62.f","version":"0'0","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.895932+0000","last_change":"2024-02-23T00:43:17.855629+0000","last_active":"2024-02-23T00:43:29.895932+0000","last_peered":"2024-02-23T00:43:29.895932+0000","last_clean":"2024-02-23T00:43:29.895932+0000","last_became_active":"2024-02-23T00:43:17.855331+0000","last_became_peered":"2024-02-23T00:43:17.855331+0000","last_unstale":"2024-02-23T00:43:29.895932+0000","last_undegraded":"2024-02-23T00:43:29.895932+0000","last_fullsized":"2024-02-23T00:43:29.895932+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"47.1e","version":"0'0","reported_seq":29,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.930948+0000","last_change":"2024-02-23T00:43:09.908968+0000","last_active":"2024-02-23T00:43:29.930948+0000","last_peered":"2024-02-23T00:43:29.930948+0000","last_clean":"2024-02-23T00:43:29.930948+0000","last_became_active":"2024-02-23T00:43:09.908444+0000","last_became_peered":"2024-02-23T00:43:09.908444+0000","last_unstale":"2024-02-23T00:43:29.930948+0000","last_undegraded":"2024-02-23T00:43:29.930948+0000","last_fullsized":"2024-02-23T00:43:29.930948+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"63.11","version":"0'0","reported_seq":36,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.910289+0000","last_change":"2024-02-23T00:43:29.910289+0000","last_active":"2024-02-23T00:43:29.910289+0000","last_peered":"2024-02-23T00:43:29.910289+0000","last_clean":"2024-02-23T00:43:29.910289+0000","last_became_active":"2024-02-23T00:43:18.993969+0000","last_became_peered":"2024-02-23T00:43:18.993969+0000","last_unstale":"2024-02-23T00:43:29.910289+0000","last_undegraded":"2024-02-23T00:43:29.910289+0000","last_fullsized":"2024-02-23T00:43:29.910289+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"62.10","version":"0'0","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.944646+0000","last_change":"2024-02-23T00:43:17.833050+0000","last_active":"2024-02-23T00:43:29.944646+0000","last_peered":"2024-02-23T00:43:29.944646+0000","last_clean":"2024-02-23T00:43:29.944646+0000","last_became_active":"2024-02-23T00:43:17.819865+0000","last_became_peered":"2024-02-23T00:43:17.819865+0000","last_unstale":"2024-02-23T00:43:29.944646+0000","last_undegraded":"2024-02-23T00:43:29.944646+0000","last_fullsized":"2024-02-23T00:43:29.944646+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"47.1","version":"0'0","reported_seq":29,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.897089+0000","last_change":"2024-02-23T00:43:08.882233+0000","last_active":"2024-02-23T00:43:29.897089+0000","last_peered":"2024-02-23T00:43:29.897089+0000","last_clean":"2024-02-23T00:43:29.897089+0000","last_became_active":"2024-02-23T00:43:08.881844+0000","last_became_peered":"2024-02-23T00:43:08.881844+0000","last_unstale":"2024-02-23T00:43:29.897089+0000","last_undegraded":"2024-02-23T00:43:29.897089+0000","last_fullsized":"2024-02-23T00:43:29.897089+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"63.10","version":"0'0","reported_seq":36,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.965756+0000","last_change":"2024-02-23T00:43:29.965756+0000","last_active":"2024-02-23T00:43:29.965756+0000","last_peered":"2024-02-23T00:43:29.965756+0000","last_clean":"2024-02-23T00:43:29.965756+0000","last_became_active":"2024-02-23T00:43:19.002791+0000","last_became_peered":"2024-02-23T00:43:19.002791+0000","last_unstale":"2024-02-23T00:43:29.965756+0000","last_undegraded":"2024-02-23T00:43:29.965756+0000","last_fullsized":"2024-02-23T00:43:29.965756+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"62.11","version":"0'0","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.926403+0000","last_change":"2024-02-23T00:43:17.853226+0000","last_active":"2024-02-23T00:43:29.926403+0000","last_peered":"2024-02-23T00:43:29.926403+0000","last_clean":"2024-02-23T00:43:29.926403+0000","last_became_active":"2024-02-23T00:43:17.852697+0000","last_became_peered":"2024-02-23T00:43:17.852697+0000","last_unstale":"2024-02-23T00:43:29.926403+0000","last_undegraded":"2024-02-23T00:43:29.926403+0000","last_fullsized":"2024-02-23T00:43:29.926403+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"47.0","version":"0'0","reported_seq":29,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.896999+0000","last_change":"2024-02-23T00:43:08.872662+0000","last_active":"2024-02-23T00:43:29.896999+0000","last_peered":"2024-02-23T00:43:29.896999+0000","last_clean":"2024-02-23T00:43:29.896999+0000","last_became_active":"2024-02-23T00:43:08.872188+0000","last_became_peered":"2024-02-23T00:43:08.872188+0000","last_unstale":"2024-02-23T00:43:29.896999+0000","last_undegraded":"2024-02-23T00:43:29.896999+0000","last_fullsized":"2024-02-23T00:43:29.896999+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"63.13","version":"0'0","reported_seq":36,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.916908+0000","last_change":"2024-02-23T00:43:29.916908+0000","last_active":"2024-02-23T00:43:29.916908+0000","last_peered":"2024-02-23T00:43:29.916908+0000","last_clean":"2024-02-23T00:43:29.916908+0000","last_became_active":"2024-02-23T00:43:18.968944+0000","last_became_peered":"2024-02-23T00:43:18.968944+0000","last_unstale":"2024-02-23T00:43:29.916908+0000","last_undegraded":"2024-02-23T00:43:29.916908+0000","last_fullsized":"2024-02-23T00:43:29.916908+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"62.12","version":"0'0","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.934461+0000","last_change":"2024-02-23T00:43:17.856865+0000","last_active":"2024-02-23T00:43:29.934461+0000","last_peered":"2024-02-23T00:43:29.934461+0000","last_clean":"2024-02-23T00:43:29.934461+0000","last_became_active":"2024-02-23T00:43:17.856118+0000","last_became_peered":"2024-02-23T00:43:17.856118+0000","last_unstale":"2024-02-23T00:43:29.934461+0000","last_undegraded":"2024-02-23T00:43:29.934461+0000","last_fullsized":"2024-02-23T00:43:29.934461+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"47.3","version":"0'0","reported_seq":29,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.915686+0000","last_change":"2024-02-23T00:43:09.904602+0000","last_active":"2024-02-23T00:43:29.915686+0000","last_peered":"2024-02-23T00:43:29.915686+0000","last_clean":"2024-02-23T00:43:29.915686+0000","last_became_active":"2024-02-23T00:43:09.904062+0000","last_became_peered":"2024-02-23T00:43:09.904062+0000","last_unstale":"2024-02-23T00:43:29.915686+0000","last_undegraded":"2024-02-23T00:43:29.915686+0000","last_fullsized":"2024-02-23T00:43:29.915686+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"63.12","version":"0'0","reported_seq":36,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.911424+0000","last_change":"2024-02-23T00:43:29.911424+0000","last_active":"2024-02-23T00:43:29.911424+0000","last_peered":"2024-02-23T00:43:29.911424+0000","last_clean":"2024-02-23T00:43:29.911424+0000","last_became_active":"2024-02-23T00:43:18.975156+0000","last_became_peered":"2024-02-23T00:43:18.975156+0000","last_unstale":"2024-02-23T00:43:29.911424+0000","last_undegraded":"2024-02-23T00:43:29.911424+0000","last_fullsized":"2024-02-23T00:43:29.911424+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"62.13","version":"0'0","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.915254+0000","last_change":"2024-02-23T00:43:17.858129+0000","last_active":"2024-02-23T00:43:29.915254+0000","last_peered":"2024-02-23T00:43:29.915254+0000","last_clean":"2024-02-23T00:43:29.915254+0000","last_became_active":"2024-02-23T00:43:17.857823+0000","last_became_peered":"2024-02-23T00:43:17.857823+0000","last_unstale":"2024-02-23T00:43:29.915254+0000","last_undegraded":"2024-02-23T00:43:29.915254+0000","last_fullsized":"2024-02-23T00:43:29.915254+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"47.2","version":"0'0","reported_seq":29,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.901875+0000","last_change":"2024-02-23T00:43:08.889781+0000","last_active":"2024-02-23T00:43:29.901875+0000","last_peered":"2024-02-23T00:43:29.901875+0000","last_clean":"2024-02-23T00:43:29.901875+0000","last_became_active":"2024-02-23T00:43:08.885309+0000","last_became_peered":"2024-02-23T00:43:08.885309+0000","last_unstale":"2024-02-23T00:43:29.901875+0000","last_undegraded":"2024-02-23T00:43:29.901875+0000","last_fullsized":"2024-02-23T00:43:29.901875+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"63.15","version":"0'0","reported_seq":36,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.911606+0000","last_change":"2024-02-23T00:43:29.911606+0000","last_active":"2024-02-23T00:43:29.911606+0000","last_peered":"2024-02-23T00:43:29.911606+0000","last_clean":"2024-02-23T00:43:29.911606+0000","last_became_active":"2024-02-23T00:43:18.993849+0000","last_became_peered":"2024-02-23T00:43:18.993849+0000","last_unstale":"2024-02-23T00:43:29.911606+0000","last_undegraded":"2024-02-23T00:43:29.911606+0000","last_fullsized":"2024-02-23T00:43:29.911606+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"62.14","version":"0'0","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.916752+0000","last_change":"2024-02-23T00:43:17.816440+0000","last_active":"2024-02-23T00:43:29.916752+0000","last_peered":"2024-02-23T00:43:29.916752+0000","last_clean":"2024-02-23T00:43:29.916752+0000","last_became_active":"2024-02-23T00:43:17.816138+0000","last_became_peered":"2024-02-23T00:43:17.816138+0000","last_unstale":"2024-02-23T00:43:29.916752+0000","last_undegraded":"2024-02-23T00:43:29.916752+0000","last_fullsized":"2024-02-23T00:43:29.916752+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"47.5","version":"0'0","reported_seq":29,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.939578+0000","last_change":"2024-02-23T00:43:09.906559+0000","last_active":"2024-02-23T00:43:29.939578+0000","last_peered":"2024-02-23T00:43:29.939578+0000","last_clean":"2024-02-23T00:43:29.939578+0000","last_became_active":"2024-02-23T00:43:09.906396+0000","last_became_peered":"2024-02-23T00:43:09.906396+0000","last_unstale":"2024-02-23T00:43:29.939578+0000","last_undegraded":"2024-02-23T00:43:29.939578+0000","last_fullsized":"2024-02-23T00:43:29.939578+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"63.14","version":"0'0","reported_seq":36,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.911280+0000","last_change":"2024-02-23T00:43:29.911280+0000","last_active":"2024-02-23T00:43:29.911280+0000","last_peered":"2024-02-23T00:43:29.911280+0000","last_clean":"2024-02-23T00:43:29.911280+0000","last_became_active":"2024-02-23T00:43:18.963795+0000","last_became_peered":"2024-02-23T00:43:18.963795+0000","last_unstale":"2024-02-23T00:43:29.911280+0000","last_undegraded":"2024-02-23T00:43:29.911280+0000","last_fullsized":"2024-02-23T00:43:29.911280+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7],"acting":[6,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"62.15","version":"0'0","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.939413+0000","last_change":"2024-02-23T00:43:17.844116+0000","last_active":"2024-02-23T00:43:29.939413+0000","last_peered":"2024-02-23T00:43:29.939413+0000","last_clean":"2024-02-23T00:43:29.939413+0000","last_became_active":"2024-02-23T00:43:17.843834+0000","last_became_peered":"2024-02-23T00:43:17.843834+0000","last_unstale":"2024-02-23T00:43:29.939413+0000","last_undegraded":"2024-02-23T00:43:29.939413+0000","last_fullsized":"2024-02-23T00:43:29.939413+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"47.4","version":"0'0","reported_seq":29,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.896923+0000","last_change":"2024-02-23T00:43:09.895723+0000","last_active":"2024-02-23T00:43:29.896923+0000","last_peered":"2024-02-23T00:43:29.896923+0000","last_clean":"2024-02-23T00:43:29.896923+0000","last_became_active":"2024-02-23T00:43:09.895484+0000","last_became_peered":"2024-02-23T00:43:09.895484+0000","last_unstale":"2024-02-23T00:43:29.896923+0000","last_undegraded":"2024-02-23T00:43:29.896923+0000","last_fullsized":"2024-02-23T00:43:29.896923+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"63.17","version":"0'0","reported_seq":36,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.908277+0000","last_change":"2024-02-23T00:43:29.908277+0000","last_active":"2024-02-23T00:43:29.908277+0000","last_peered":"2024-02-23T00:43:29.908277+0000","last_clean":"2024-02-23T00:43:29.908277+0000","last_became_active":"2024-02-23T00:43:18.986520+0000","last_became_peered":"2024-02-23T00:43:18.986520+0000","last_unstale":"2024-02-23T00:43:29.908277+0000","last_undegraded":"2024-02-23T00:43:29.908277+0000","last_fullsized":"2024-02-23T00:43:29.908277+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"62.16","version":"0'0","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.895998+0000","last_change":"2024-02-23T00:43:17.834599+0000","last_active":"2024-02-23T00:43:29.895998+0000","last_peered":"2024-02-23T00:43:29.895998+0000","last_clean":"2024-02-23T00:43:29.895998+0000","last_became_active":"2024-02-23T00:43:17.834291+0000","last_became_peered":"2024-02-23T00:43:17.834291+0000","last_unstale":"2024-02-23T00:43:29.895998+0000","last_undegraded":"2024-02-23T00:43:29.895998+0000","last_fullsized":"2024-02-23T00:43:29.895998+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"47.7","version":"120'1","reported_seq":31,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.949242+0000","last_change":"2024-02-23T00:43:08.893084+0000","last_active":"2024-02-23T00:43:29.949242+0000","last_peered":"2024-02-23T00:43:29.949242+0000","last_clean":"2024-02-23T00:43:29.949242+0000","last_became_active":"2024-02-23T00:43:08.892646+0000","last_became_peered":"2024-02-23T00:43:08.892646+0000","last_unstale":"2024-02-23T00:43:29.949242+0000","last_undegraded":"2024-02-23T00:43:29.949242+0000","last_fullsized":"2024-02-23T00:43:29.949242+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"63.16","version":"0'0","reported_seq":36,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.959049+0000","last_change":"2024-02-23T00:43:29.959049+0000","last_active":"2024-02-23T00:43:29.959049+0000","last_peered":"2024-02-23T00:43:29.959049+0000","last_clean":"2024-02-23T00:43:29.959049+0000","last_became_active":"2024-02-23T00:43:19.008319+0000","last_became_peered":"2024-02-23T00:43:19.008319+0000","last_unstale":"2024-02-23T00:43:29.959049+0000","last_undegraded":"2024-02-23T00:43:29.959049+0000","last_fullsized":"2024-02-23T00:43:29.959049+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"62.17","version":"137'1","reported_seq":22,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.906708+0000","last_change":"2024-02-23T00:43:17.843798+0000","last_active":"2024-02-23T00:43:29.906708+0000","last_peered":"2024-02-23T00:43:29.906708+0000","last_clean":"2024-02-23T00:43:29.906708+0000","last_became_active":"2024-02-23T00:43:17.843525+0000","last_became_peered":"2024-02-23T00:43:17.843525+0000","last_unstale":"2024-02-23T00:43:29.906708+0000","last_undegraded":"2024-02-23T00:43:29.906708+0000","last_fullsized":"2024-02-23T00:43:29.906708+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"47.6","version":"0'0","reported_seq":29,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.931089+0000","last_change":"2024-02-23T00:43:09.911196+0000","last_active":"2024-02-23T00:43:29.931089+0000","last_peered":"2024-02-23T00:43:29.931089+0000","last_clean":"2024-02-23T00:43:29.931089+0000","last_became_active":"2024-02-23T00:43:09.910649+0000","last_became_peered":"2024-02-23T00:43:09.910649+0000","last_unstale":"2024-02-23T00:43:29.931089+0000","last_undegraded":"2024-02-23T00:43:29.931089+0000","last_fullsized":"2024-02-23T00:43:29.931089+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"63.18","version":"134'6","reported_seq":42,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.915125+0000","last_change":"2024-02-23T00:43:29.915125+0000","last_active":"2024-02-23T00:43:29.915125+0000","last_peered":"2024-02-23T00:43:29.915125+0000","last_clean":"2024-02-23T00:43:29.915125+0000","last_became_active":"2024-02-23T00:43:18.980503+0000","last_became_peered":"2024-02-23T00:43:18.980503+0000","last_unstale":"2024-02-23T00:43:29.915125+0000","last_undegraded":"2024-02-23T00:43:29.915125+0000","last_fullsized":"2024-02-23T00:43:29.915125+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"62.19","version":"0'0","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.910156+0000","last_change":"2024-02-23T00:43:17.816183+0000","last_active":"2024-02-23T00:43:29.910156+0000","last_peered":"2024-02-23T00:43:29.910156+0000","last_clean":"2024-02-23T00:43:29.910156+0000","last_became_active":"2024-02-23T00:43:17.815830+0000","last_became_peered":"2024-02-23T00:43:17.815830+0000","last_unstale":"2024-02-23T00:43:29.910156+0000","last_undegraded":"2024-02-23T00:43:29.910156+0000","last_fullsized":"2024-02-23T00:43:29.910156+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"47.8","version":"120'3","reported_seq":33,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.898798+0000","last_change":"2024-02-23T00:43:08.869192+0000","last_active":"2024-02-23T00:43:29.898798+0000","last_peered":"2024-02-23T00:43:29.898798+0000","last_clean":"2024-02-23T00:43:29.898798+0000","last_became_active":"2024-02-23T00:43:08.868897+0000","last_became_peered":"2024-02-23T00:43:08.868897+0000","last_unstale":"2024-02-23T00:43:29.898798+0000","last_undegraded":"2024-02-23T00:43:29.898798+0000","last_fullsized":"2024-02-23T00:43:29.898798+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":512,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"63.19","version":"0'0","reported_seq":36,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.910151+0000","last_change":"2024-02-23T00:43:29.910151+0000","last_active":"2024-02-23T00:43:29.910151+0000","last_peered":"2024-02-23T00:43:29.910151+0000","last_clean":"2024-02-23T00:43:29.910151+0000","last_became_active":"2024-02-23T00:43:18.973620+0000","last_became_peered":"2024-02-23T00:43:18.973620+0000","last_unstale":"2024-02-23T00:43:29.910151+0000","last_undegraded":"2024-02-23T00:43:29.910151+0000","last_fullsized":"2024-02-23T00:43:29.910151+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"62.18","version":"0'0","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.899777+0000","last_change":"2024-02-23T00:43:17.840527+0000","last_active":"2024-02-23T00:43:29.899777+0000","last_peered":"2024-02-23T00:43:29.899777+0000","last_clean":"2024-02-23T00:43:29.899777+0000","last_became_active":"2024-02-23T00:43:17.840351+0000","last_became_peered":"2024-02-23T00:43:17.840351+0000","last_unstale":"2024-02-23T00:43:29.899777+0000","last_undegraded":"2024-02-23T00:43:29.899777+0000","last_fullsized":"2024-02-23T00:43:29.899777+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"47.9","version":"0'0","reported_seq":29,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.906037+0000","last_change":"2024-02-23T00:43:09.914042+0000","last_active":"2024-02-23T00:43:29.906037+0000","last_peered":"2024-02-23T00:43:29.906037+0000","last_clean":"2024-02-23T00:43:29.906037+0000","last_became_active":"2024-02-23T00:43:09.913719+0000","last_became_peered":"2024-02-23T00:43:09.913719+0000","last_unstale":"2024-02-23T00:43:29.906037+0000","last_undegraded":"2024-02-23T00:43:29.906037+0000","last_fullsized":"2024-02-23T00:43:29.906037+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"63.1a","version":"0'0","reported_seq":36,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.950805+0000","last_change":"2024-02-23T00:43:29.950805+0000","last_active":"2024-02-23T00:43:29.950805+0000","last_peered":"2024-02-23T00:43:29.950805+0000","last_clean":"2024-02-23T00:43:29.950805+0000","last_became_active":"2024-02-23T00:43:19.003371+0000","last_became_peered":"2024-02-23T00:43:19.003371+0000","last_unstale":"2024-02-23T00:43:29.950805+0000","last_undegraded":"2024-02-23T00:43:29.950805+0000","last_fullsized":"2024-02-23T00:43:29.950805+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"62.1b","version":"0'0","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.894931+0000","last_change":"2024-02-23T00:43:17.803389+0000","last_active":"2024-02-23T00:43:29.894931+0000","last_peered":"2024-02-23T00:43:29.894931+0000","last_clean":"2024-02-23T00:43:29.894931+0000","last_became_active":"2024-02-23T00:43:17.803043+0000","last_became_peered":"2024-02-23T00:43:17.803043+0000","last_unstale":"2024-02-23T00:43:29.894931+0000","last_undegraded":"2024-02-23T00:43:29.894931+0000","last_fullsized":"2024-02-23T00:43:29.894931+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"47.a","version":"0'0","reported_seq":29,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.901769+0000","last_change":"2024-02-23T00:43:08.890324+0000","last_active":"2024-02-23T00:43:29.901769+0000","last_peered":"2024-02-23T00:43:29.901769+0000","last_clean":"2024-02-23T00:43:29.901769+0000","last_became_active":"2024-02-23T00:43:08.890143+0000","last_became_peered":"2024-02-23T00:43:08.890143+0000","last_unstale":"2024-02-23T00:43:29.901769+0000","last_undegraded":"2024-02-23T00:43:29.901769+0000","last_fullsized":"2024-02-23T00:43:29.901769+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"63.1b","version":"0'0","reported_seq":36,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.916532+0000","last_change":"2024-02-23T00:43:29.916532+0000","last_active":"2024-02-23T00:43:29.916532+0000","last_peered":"2024-02-23T00:43:29.916532+0000","last_clean":"2024-02-23T00:43:29.916532+0000","last_became_active":"2024-02-23T00:43:18.986455+0000","last_became_peered":"2024-02-23T00:43:18.986455+0000","last_unstale":"2024-02-23T00:43:29.916532+0000","last_undegraded":"2024-02-23T00:43:29.916532+0000","last_fullsized":"2024-02-23T00:43:29.916532+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"62.1a","version":"0'0","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.907333+0000","last_change":"2024-02-23T00:43:17.798571+0000","last_active":"2024-02-23T00:43:29.907333+0000","last_peered":"2024-02-23T00:43:29.907333+0000","last_clean":"2024-02-23T00:43:29.907333+0000","last_became_active":"2024-02-23T00:43:17.798285+0000","last_became_peered":"2024-02-23T00:43:17.798285+0000","last_unstale":"2024-02-23T00:43:29.907333+0000","last_undegraded":"2024-02-23T00:43:29.907333+0000","last_fullsized":"2024-02-23T00:43:29.907333+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"47.b","version":"0'0","reported_seq":29,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.915890+0000","last_change":"2024-02-23T00:43:08.871891+0000","last_active":"2024-02-23T00:43:29.915890+0000","last_peered":"2024-02-23T00:43:29.915890+0000","last_clean":"2024-02-23T00:43:29.915890+0000","last_became_active":"2024-02-23T00:43:08.871529+0000","last_became_peered":"2024-02-23T00:43:08.871529+0000","last_unstale":"2024-02-23T00:43:29.915890+0000","last_undegraded":"2024-02-23T00:43:29.915890+0000","last_fullsized":"2024-02-23T00:43:29.915890+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"63.1c","version":"0'0","reported_seq":36,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.955089+0000","last_change":"2024-02-23T00:43:29.955089+0000","last_active":"2024-02-23T00:43:29.955089+0000","last_peered":"2024-02-23T00:43:29.955089+0000","last_clean":"2024-02-23T00:43:29.955089+0000","last_became_active":"2024-02-23T00:43:19.012643+0000","last_became_peered":"2024-02-23T00:43:19.012643+0000","last_unstale":"2024-02-23T00:43:29.955089+0000","last_undegraded":"2024-02-23T00:43:29.955089+0000","last_fullsized":"2024-02-23T00:43:29.955089+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"62.1d","version":"0'0","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.904977+0000","last_change":"2024-02-23T00:43:17.828524+0000","last_active":"2024-02-23T00:43:29.904977+0000","last_peered":"2024-02-23T00:43:29.904977+0000","last_clean":"2024-02-23T00:43:29.904977+0000","last_became_active":"2024-02-23T00:43:17.828194+0000","last_became_peered":"2024-02-23T00:43:17.828194+0000","last_unstale":"2024-02-23T00:43:29.904977+0000","last_undegraded":"2024-02-23T00:43:29.904977+0000","last_fullsized":"2024-02-23T00:43:29.904977+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"47.c","version":"0'0","reported_seq":29,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.932198+0000","last_change":"2024-02-23T00:43:08.891783+0000","last_active":"2024-02-23T00:43:29.932198+0000","last_peered":"2024-02-23T00:43:29.932198+0000","last_clean":"2024-02-23T00:43:29.932198+0000","last_became_active":"2024-02-23T00:43:08.890962+0000","last_became_peered":"2024-02-23T00:43:08.890962+0000","last_unstale":"2024-02-23T00:43:29.932198+0000","last_undegraded":"2024-02-23T00:43:29.932198+0000","last_fullsized":"2024-02-23T00:43:29.932198+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"63.1d","version":"0'0","reported_seq":36,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.965425+0000","last_change":"2024-02-23T00:43:29.965425+0000","last_active":"2024-02-23T00:43:29.965425+0000","last_peered":"2024-02-23T00:43:29.965425+0000","last_clean":"2024-02-23T00:43:29.965425+0000","last_became_active":"2024-02-23T00:43:18.994831+0000","last_became_peered":"2024-02-23T00:43:18.994831+0000","last_unstale":"2024-02-23T00:43:29.965425+0000","last_undegraded":"2024-02-23T00:43:29.965425+0000","last_fullsized":"2024-02-23T00:43:29.965425+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"62.1c","version":"0'0","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.901186+0000","last_change":"2024-02-23T00:43:17.805004+0000","last_active":"2024-02-23T00:43:29.901186+0000","last_peered":"2024-02-23T00:43:29.901186+0000","last_clean":"2024-02-23T00:43:29.901186+0000","last_became_active":"2024-02-23T00:43:17.804718+0000","last_became_peered":"2024-02-23T00:43:17.804718+0000","last_unstale":"2024-02-23T00:43:29.901186+0000","last_undegraded":"2024-02-23T00:43:29.901186+0000","last_fullsized":"2024-02-23T00:43:29.901186+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"47.d","version":"0'0","reported_seq":29,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.900879+0000","last_change":"2024-02-23T00:43:08.870250+0000","last_active":"2024-02-23T00:43:29.900879+0000","last_peered":"2024-02-23T00:43:29.900879+0000","last_clean":"2024-02-23T00:43:29.900879+0000","last_became_active":"2024-02-23T00:43:08.869935+0000","last_became_peered":"2024-02-23T00:43:08.869935+0000","last_unstale":"2024-02-23T00:43:29.900879+0000","last_undegraded":"2024-02-23T00:43:29.900879+0000","last_fullsized":"2024-02-23T00:43:29.900879+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"63.1e","version":"0'0","reported_seq":36,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.916996+0000","last_change":"2024-02-23T00:43:29.916996+0000","last_active":"2024-02-23T00:43:29.916996+0000","last_peered":"2024-02-23T00:43:29.916996+0000","last_clean":"2024-02-23T00:43:29.916996+0000","last_became_active":"2024-02-23T00:43:18.960132+0000","last_became_peered":"2024-02-23T00:43:18.960132+0000","last_unstale":"2024-02-23T00:43:29.916996+0000","last_undegraded":"2024-02-23T00:43:29.916996+0000","last_fullsized":"2024-02-23T00:43:29.916996+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"62.1f","version":"0'0","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.955408+0000","last_change":"2024-02-23T00:43:17.840259+0000","last_active":"2024-02-23T00:43:29.955408+0000","last_peered":"2024-02-23T00:43:29.955408+0000","last_clean":"2024-02-23T00:43:29.955408+0000","last_became_active":"2024-02-23T00:43:17.839914+0000","last_became_peered":"2024-02-23T00:43:17.839914+0000","last_unstale":"2024-02-23T00:43:29.955408+0000","last_undegraded":"2024-02-23T00:43:29.955408+0000","last_fullsized":"2024-02-23T00:43:29.955408+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"47.e","version":"0'0","reported_seq":29,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.907525+0000","last_change":"2024-02-23T00:43:08.881765+0000","last_active":"2024-02-23T00:43:29.907525+0000","last_peered":"2024-02-23T00:43:29.907525+0000","last_clean":"2024-02-23T00:43:29.907525+0000","last_became_active":"2024-02-23T00:43:08.881454+0000","last_became_peered":"2024-02-23T00:43:08.881454+0000","last_unstale":"2024-02-23T00:43:29.907525+0000","last_undegraded":"2024-02-23T00:43:29.907525+0000","last_fullsized":"2024-02-23T00:43:29.907525+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"63.1f","version":"0'0","reported_seq":36,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.908118+0000","last_change":"2024-02-23T00:43:29.908118+0000","last_active":"2024-02-23T00:43:29.908118+0000","last_peered":"2024-02-23T00:43:29.908118+0000","last_clean":"2024-02-23T00:43:29.908118+0000","last_became_active":"2024-02-23T00:43:18.986617+0000","last_became_peered":"2024-02-23T00:43:18.986617+0000","last_unstale":"2024-02-23T00:43:29.908118+0000","last_undegraded":"2024-02-23T00:43:29.908118+0000","last_fullsized":"2024-02-23T00:43:29.908118+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:17.717661+0000","last_clean_scrub_stamp":"2024-02-23T00:43:17.717661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"62.1e","version":"0'0","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.907191+0000","last_change":"2024-02-23T00:43:17.838947+0000","last_active":"2024-02-23T00:43:29.907191+0000","last_peered":"2024-02-23T00:43:29.907191+0000","last_clean":"2024-02-23T00:43:29.907191+0000","last_became_active":"2024-02-23T00:43:17.838682+0000","last_became_peered":"2024-02-23T00:43:17.838682+0000","last_unstale":"2024-02-23T00:43:29.907191+0000","last_undegraded":"2024-02-23T00:43:29.907191+0000","last_fullsized":"2024-02-23T00:43:29.907191+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"47.f","version":"0'0","reported_seq":29,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.906881+0000","last_change":"2024-02-23T00:43:09.905858+0000","last_active":"2024-02-23T00:43:29.906881+0000","last_peered":"2024-02-23T00:43:29.906881+0000","last_clean":"2024-02-23T00:43:29.906881+0000","last_became_active":"2024-02-23T00:43:09.905531+0000","last_became_peered":"2024-02-23T00:43:09.905531+0000","last_unstale":"2024-02-23T00:43:29.906881+0000","last_undegraded":"2024-02-23T00:43:29.906881+0000","last_fullsized":"2024-02-23T00:43:29.906881+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:07.927111+0000","last_clean_scrub_stamp":"2024-02-23T00:43:07.927111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.9","version":"67'8","reported_seq":154,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.909471+0000","last_change":"2024-02-23T00:42:47.985647+0000","last_active":"2024-02-23T00:43:29.909471+0000","last_peered":"2024-02-23T00:43:29.909471+0000","last_clean":"2024-02-23T00:43:29.909471+0000","last_became_active":"2024-02-23T00:39:18.793414+0000","last_became_peered":"2024-02-23T00:39:18.793414+0000","last_unstale":"2024-02-23T00:43:29.909471+0000","last_undegraded":"2024-02-23T00:43:29.909471+0000","last_fullsized":"2024-02-23T00:43:29.909471+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:42:47.985573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:47.985573+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"68'27","reported_seq":151,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.897170+0000","last_change":"2024-02-23T00:42:49.925777+0000","last_active":"2024-02-23T00:43:29.897170+0000","last_peered":"2024-02-23T00:43:29.897170+0000","last_clean":"2024-02-23T00:43:29.897170+0000","last_became_active":"2024-02-23T00:39:16.859401+0000","last_became_peered":"2024-02-23T00:39:16.859401+0000","last_unstale":"2024-02-23T00:43:29.897170+0000","last_undegraded":"2024-02-23T00:43:29.897170+0000","last_fullsized":"2024-02-23T00:43:29.897170+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'27","last_scrub_stamp":"2024-02-23T00:42:49.925673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:49.925673+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":97,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.956628+0000","last_change":"2024-02-23T00:42:04.330493+0000","last_active":"2024-02-23T00:43:29.956628+0000","last_peered":"2024-02-23T00:43:29.956628+0000","last_clean":"2024-02-23T00:43:29.956628+0000","last_became_active":"2024-02-23T00:39:14.784436+0000","last_became_peered":"2024-02-23T00:39:14.784436+0000","last_unstale":"2024-02-23T00:43:29.956628+0000","last_undegraded":"2024-02-23T00:43:29.956628+0000","last_fullsized":"2024-02-23T00:43:29.956628+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:04.330426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:04.330426+0000","last_clean_scrub_stamp":"2024-02-23T00:42:04.330426+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":102,"reported_epoch":139,"state":"peering","last_fresh":"2024-02-23T00:43:31.158937+0000","last_change":"2024-02-23T00:43:31.159142+0000","last_active":"2024-02-23T00:43:28.545001+0000","last_peered":"2024-02-23T00:43:29.940636+0000","last_clean":"2024-02-23T00:43:29.940636+0000","last_became_active":"2024-02-23T00:39:21.160429+0000","last_became_peered":"2024-02-23T00:43:29.940173+0000","last_unstale":"2024-02-23T00:43:31.158937+0000","last_undegraded":"2024-02-23T00:43:31.158937+0000","last_fullsized":"2024-02-23T00:43:31.158937+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:05.725796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:05.725796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.3","version":"110'5","reported_seq":44,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.895368+0000","last_change":"2024-02-23T00:42:58.836257+0000","last_active":"2024-02-23T00:43:29.895368+0000","last_peered":"2024-02-23T00:43:29.895368+0000","last_clean":"2024-02-23T00:43:29.895368+0000","last_became_active":"2024-02-23T00:42:58.835612+0000","last_became_peered":"2024-02-23T00:42:58.835612+0000","last_unstale":"2024-02-23T00:43:29.895368+0000","last_undegraded":"2024-02-23T00:43:29.895368+0000","last_fullsized":"2024-02-23T00:43:29.895368+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.906928+0000","last_change":"2024-02-23T00:42:58.832536+0000","last_active":"2024-02-23T00:43:29.906928+0000","last_peered":"2024-02-23T00:43:29.906928+0000","last_clean":"2024-02-23T00:43:29.906928+0000","last_became_active":"2024-02-23T00:42:58.828846+0000","last_became_peered":"2024-02-23T00:42:58.828846+0000","last_unstale":"2024-02-23T00:43:29.906928+0000","last_undegraded":"2024-02-23T00:43:29.906928+0000","last_fullsized":"2024-02-23T00:43:29.906928+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"112'2","reported_seq":42,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.895742+0000","last_change":"2024-02-23T00:42:58.732299+0000","last_active":"2024-02-23T00:43:29.895742+0000","last_peered":"2024-02-23T00:43:29.895742+0000","last_clean":"2024-02-23T00:43:29.895742+0000","last_became_active":"2024-02-23T00:42:58.731849+0000","last_became_peered":"2024-02-23T00:42:58.731849+0000","last_unstale":"2024-02-23T00:43:29.895742+0000","last_undegraded":"2024-02-23T00:43:29.895742+0000","last_fullsized":"2024-02-23T00:43:29.895742+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"68'15","reported_seq":127,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.933482+0000","last_change":"2024-02-23T00:42:08.357114+0000","last_active":"2024-02-23T00:43:29.933482+0000","last_peered":"2024-02-23T00:43:29.933482+0000","last_clean":"2024-02-23T00:43:29.933482+0000","last_became_active":"2024-02-23T00:39:16.783701+0000","last_became_peered":"2024-02-23T00:39:16.783701+0000","last_unstale":"2024-02-23T00:43:29.933482+0000","last_undegraded":"2024-02-23T00:43:29.933482+0000","last_fullsized":"2024-02-23T00:43:29.933482+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:42:08.357019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:08.357019+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":97,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.946232+0000","last_change":"2024-02-23T00:42:48.591600+0000","last_active":"2024-02-23T00:43:29.946232+0000","last_peered":"2024-02-23T00:43:29.946232+0000","last_clean":"2024-02-23T00:43:29.946232+0000","last_became_active":"2024-02-23T00:39:14.772493+0000","last_became_peered":"2024-02-23T00:39:14.772493+0000","last_unstale":"2024-02-23T00:43:29.946232+0000","last_undegraded":"2024-02-23T00:43:29.946232+0000","last_fullsized":"2024-02-23T00:43:29.946232+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:48.591486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:48.591486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":93,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.935233+0000","last_change":"2024-02-23T00:42:08.335396+0000","last_active":"2024-02-23T00:43:29.935233+0000","last_peered":"2024-02-23T00:43:29.935233+0000","last_clean":"2024-02-23T00:43:29.935233+0000","last_became_active":"2024-02-23T00:39:18.794476+0000","last_became_peered":"2024-02-23T00:39:18.794476+0000","last_unstale":"2024-02-23T00:43:29.935233+0000","last_undegraded":"2024-02-23T00:43:29.935233+0000","last_fullsized":"2024-02-23T00:43:29.935233+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:08.335330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:08.335330+0000","last_clean_scrub_stamp":"2024-02-23T00:42:08.335330+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":92,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.935557+0000","last_change":"2024-02-23T00:42:46.359743+0000","last_active":"2024-02-23T00:43:29.935557+0000","last_peered":"2024-02-23T00:43:29.935557+0000","last_clean":"2024-02-23T00:43:29.935557+0000","last_became_active":"2024-02-23T00:39:20.802938+0000","last_became_peered":"2024-02-23T00:39:20.802938+0000","last_unstale":"2024-02-23T00:43:29.935557+0000","last_undegraded":"2024-02-23T00:43:29.935557+0000","last_fullsized":"2024-02-23T00:43:29.935557+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:46.359646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:46.359646+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"110'5","reported_seq":44,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.928479+0000","last_change":"2024-02-23T00:42:58.742014+0000","last_active":"2024-02-23T00:43:29.928479+0000","last_peered":"2024-02-23T00:43:29.928479+0000","last_clean":"2024-02-23T00:43:29.928479+0000","last_became_active":"2024-02-23T00:42:58.741558+0000","last_became_peered":"2024-02-23T00:42:58.741558+0000","last_unstale":"2024-02-23T00:43:29.928479+0000","last_undegraded":"2024-02-23T00:43:29.928479+0000","last_fullsized":"2024-02-23T00:43:29.928479+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.914406+0000","last_change":"2024-02-23T00:42:58.871003+0000","last_active":"2024-02-23T00:43:29.914406+0000","last_peered":"2024-02-23T00:43:29.914406+0000","last_clean":"2024-02-23T00:43:29.914406+0000","last_became_active":"2024-02-23T00:42:58.867305+0000","last_became_peered":"2024-02-23T00:42:58.867305+0000","last_unstale":"2024-02-23T00:43:29.914406+0000","last_undegraded":"2024-02-23T00:43:29.914406+0000","last_fullsized":"2024-02-23T00:43:29.914406+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.1b","version":"110'1","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.927882+0000","last_change":"2024-02-23T00:42:58.744995+0000","last_active":"2024-02-23T00:43:29.927882+0000","last_peered":"2024-02-23T00:43:29.927882+0000","last_clean":"2024-02-23T00:43:29.927882+0000","last_became_active":"2024-02-23T00:42:58.744766+0000","last_became_peered":"2024-02-23T00:42:58.744766+0000","last_unstale":"2024-02-23T00:43:29.927882+0000","last_undegraded":"2024-02-23T00:43:29.927882+0000","last_fullsized":"2024-02-23T00:43:29.927882+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"88'35","reported_seq":151,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.909430+0000","last_change":"2024-02-23T00:41:58.120657+0000","last_active":"2024-02-23T00:43:29.909430+0000","last_peered":"2024-02-23T00:43:29.909430+0000","last_clean":"2024-02-23T00:43:29.909430+0000","last_became_active":"2024-02-23T00:39:16.857413+0000","last_became_peered":"2024-02-23T00:39:16.857413+0000","last_unstale":"2024-02-23T00:43:29.909430+0000","last_undegraded":"2024-02-23T00:43:29.909430+0000","last_fullsized":"2024-02-23T00:43:29.909430+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'35","last_scrub_stamp":"2024-02-23T00:41:58.120561+0000","last_deep_scrub":"88'35","last_deep_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_clean_scrub_stamp":"2024-02-23T00:41:58.120561+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":97,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.927939+0000","last_change":"2024-02-23T00:41:54.303858+0000","last_active":"2024-02-23T00:43:29.927939+0000","last_peered":"2024-02-23T00:43:29.927939+0000","last_clean":"2024-02-23T00:43:29.927939+0000","last_became_active":"2024-02-23T00:39:14.776705+0000","last_became_peered":"2024-02-23T00:39:14.776705+0000","last_unstale":"2024-02-23T00:43:29.927939+0000","last_undegraded":"2024-02-23T00:43:29.927939+0000","last_fullsized":"2024-02-23T00:43:29.927939+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:54.303790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_clean_scrub_stamp":"2024-02-23T00:41:54.303790+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":93,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.927528+0000","last_change":"2024-02-23T00:42:03.291538+0000","last_active":"2024-02-23T00:43:29.927528+0000","last_peered":"2024-02-23T00:43:29.927528+0000","last_clean":"2024-02-23T00:43:29.927528+0000","last_became_active":"2024-02-23T00:39:18.792718+0000","last_became_peered":"2024-02-23T00:39:18.792718+0000","last_unstale":"2024-02-23T00:43:29.927528+0000","last_undegraded":"2024-02-23T00:43:29.927528+0000","last_fullsized":"2024-02-23T00:43:29.927528+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:03.291436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:03.291436+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":90,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.913921+0000","last_change":"2024-02-23T00:42:43.027150+0000","last_active":"2024-02-23T00:43:29.913921+0000","last_peered":"2024-02-23T00:43:29.913921+0000","last_clean":"2024-02-23T00:43:29.913921+0000","last_became_active":"2024-02-23T00:39:20.798989+0000","last_became_peered":"2024-02-23T00:39:20.798989+0000","last_unstale":"2024-02-23T00:43:29.913921+0000","last_undegraded":"2024-02-23T00:43:29.913921+0000","last_fullsized":"2024-02-23T00:43:29.913921+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:43.027029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:43.027029+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"110'5","reported_seq":44,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.906780+0000","last_change":"2024-02-23T00:42:58.688988+0000","last_active":"2024-02-23T00:43:29.906780+0000","last_peered":"2024-02-23T00:43:29.906780+0000","last_clean":"2024-02-23T00:43:29.906780+0000","last_became_active":"2024-02-23T00:42:58.684001+0000","last_became_peered":"2024-02-23T00:42:58.684001+0000","last_unstale":"2024-02-23T00:43:29.906780+0000","last_undegraded":"2024-02-23T00:43:29.906780+0000","last_fullsized":"2024-02-23T00:43:29.906780+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.932306+0000","last_change":"2024-02-23T00:42:57.641727+0000","last_active":"2024-02-23T00:43:29.932306+0000","last_peered":"2024-02-23T00:43:29.932306+0000","last_clean":"2024-02-23T00:43:29.932306+0000","last_became_active":"2024-02-23T00:42:57.639013+0000","last_became_peered":"2024-02-23T00:42:57.639013+0000","last_unstale":"2024-02-23T00:43:29.932306+0000","last_undegraded":"2024-02-23T00:43:29.932306+0000","last_fullsized":"2024-02-23T00:43:29.932306+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1a","version":"112'5","reported_seq":44,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-02-23T00:43:31.158418+0000","last_change":"2024-02-23T00:42:58.827911+0000","last_active":"2024-02-23T00:43:31.158418+0000","last_peered":"2024-02-23T00:43:31.158418+0000","last_clean":"2024-02-23T00:43:31.158418+0000","last_became_active":"2024-02-23T00:42:58.827630+0000","last_became_peered":"2024-02-23T00:42:58.827630+0000","last_unstale":"2024-02-23T00:43:31.158418+0000","last_undegraded":"2024-02-23T00:43:31.158418+0000","last_fullsized":"2024-02-23T00:43:31.158418+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25168628,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":27328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0],"acting":[5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"67'17","reported_seq":145,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.901456+0000","last_change":"2024-02-23T00:41:02.038717+0000","last_active":"2024-02-23T00:43:29.901456+0000","last_peered":"2024-02-23T00:43:29.901456+0000","last_clean":"2024-02-23T00:43:29.901456+0000","last_became_active":"2024-02-23T00:40:42.933806+0000","last_became_peered":"2024-02-23T00:40:42.933806+0000","last_unstale":"2024-02-23T00:43:29.901456+0000","last_undegraded":"2024-02-23T00:43:29.901456+0000","last_fullsized":"2024-02-23T00:43:29.901456+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'17","last_scrub_stamp":"2024-02-23T00:41:02.038627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:02.038627+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":73,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-02-23T00:43:31.362599+0000","last_change":"2024-02-23T00:43:31.362599+0000","last_active":"2024-02-23T00:43:31.362599+0000","last_peered":"2024-02-23T00:43:31.362599+0000","last_clean":"2024-02-23T00:43:31.362599+0000","last_became_active":"2024-02-23T00:40:42.931041+0000","last_became_peered":"2024-02-23T00:40:42.931041+0000","last_unstale":"2024-02-23T00:43:31.362599+0000","last_undegraded":"2024-02-23T00:43:31.362599+0000","last_fullsized":"2024-02-23T00:43:31.362599+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:31.362510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:31.362510+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":93,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.933031+0000","last_change":"2024-02-23T00:41:53.692312+0000","last_active":"2024-02-23T00:43:29.933031+0000","last_peered":"2024-02-23T00:43:29.933031+0000","last_clean":"2024-02-23T00:43:29.933031+0000","last_became_active":"2024-02-23T00:39:18.799875+0000","last_became_peered":"2024-02-23T00:39:18.799875+0000","last_unstale":"2024-02-23T00:43:29.933031+0000","last_undegraded":"2024-02-23T00:43:29.933031+0000","last_fullsized":"2024-02-23T00:43:29.933031+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:53.692185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:53.692185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":109,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.956138+0000","last_change":"2024-02-23T00:43:08.885549+0000","last_active":"2024-02-23T00:43:29.956138+0000","last_peered":"2024-02-23T00:43:29.956138+0000","last_clean":"2024-02-23T00:43:29.956138+0000","last_became_active":"2024-02-23T00:43:08.885257+0000","last_became_peered":"2024-02-23T00:43:08.885257+0000","last_unstale":"2024-02-23T00:43:29.956138+0000","last_undegraded":"2024-02-23T00:43:29.956138+0000","last_fullsized":"2024-02-23T00:43:29.956138+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:37.180472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:37.180472+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.6","version":"110'9","reported_seq":48,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.931593+0000","last_change":"2024-02-23T00:42:58.871934+0000","last_active":"2024-02-23T00:43:29.931593+0000","last_peered":"2024-02-23T00:43:29.931593+0000","last_clean":"2024-02-23T00:43:29.931593+0000","last_became_active":"2024-02-23T00:42:58.870147+0000","last_became_peered":"2024-02-23T00:42:58.870147+0000","last_unstale":"2024-02-23T00:43:29.931593+0000","last_undegraded":"2024-02-23T00:43:29.931593+0000","last_fullsized":"2024-02-23T00:43:29.931593+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.922493+0000","last_change":"2024-02-23T00:42:57.683188+0000","last_active":"2024-02-23T00:43:29.922493+0000","last_peered":"2024-02-23T00:43:29.922493+0000","last_clean":"2024-02-23T00:43:29.922493+0000","last_became_active":"2024-02-23T00:42:57.675306+0000","last_became_peered":"2024-02-23T00:42:57.675306+0000","last_unstale":"2024-02-23T00:43:29.922493+0000","last_undegraded":"2024-02-23T00:43:29.922493+0000","last_fullsized":"2024-02-23T00:43:29.922493+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"139'1","reported_seq":39,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-02-23T00:43:31.164476+0000","last_change":"2024-02-23T00:42:58.827765+0000","last_active":"2024-02-23T00:43:31.164476+0000","last_peered":"2024-02-23T00:43:31.164476+0000","last_clean":"2024-02-23T00:43:31.164476+0000","last_became_active":"2024-02-23T00:42:58.826869+0000","last_became_peered":"2024-02-23T00:42:58.826869+0000","last_unstale":"2024-02-23T00:43:31.164476+0000","last_undegraded":"2024-02-23T00:43:31.164476+0000","last_fullsized":"2024-02-23T00:43:31.164476+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2800,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.c","version":"67'10","reported_seq":110,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.895454+0000","last_change":"2024-02-23T00:42:50.078150+0000","last_active":"2024-02-23T00:43:29.895454+0000","last_peered":"2024-02-23T00:43:29.895454+0000","last_clean":"2024-02-23T00:43:29.895454+0000","last_became_active":"2024-02-23T00:39:16.777530+0000","last_became_peered":"2024-02-23T00:39:16.777530+0000","last_unstale":"2024-02-23T00:43:29.895454+0000","last_undegraded":"2024-02-23T00:43:29.895454+0000","last_fullsized":"2024-02-23T00:43:29.895454+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'10","last_scrub_stamp":"2024-02-23T00:42:50.078042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:50.078042+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":97,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.902223+0000","last_change":"2024-02-23T00:42:06.575346+0000","last_active":"2024-02-23T00:43:29.902223+0000","last_peered":"2024-02-23T00:43:29.902223+0000","last_clean":"2024-02-23T00:43:29.902223+0000","last_became_active":"2024-02-23T00:39:14.780379+0000","last_became_peered":"2024-02-23T00:39:14.780379+0000","last_unstale":"2024-02-23T00:43:29.902223+0000","last_undegraded":"2024-02-23T00:43:29.902223+0000","last_fullsized":"2024-02-23T00:43:29.902223+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:06.575242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.575242+0000","last_clean_scrub_stamp":"2024-02-23T00:42:06.575242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"67'8","reported_seq":152,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.932823+0000","last_change":"2024-02-23T00:42:36.214941+0000","last_active":"2024-02-23T00:43:29.932823+0000","last_peered":"2024-02-23T00:43:29.932823+0000","last_clean":"2024-02-23T00:43:29.932823+0000","last_became_active":"2024-02-23T00:39:18.799544+0000","last_became_peered":"2024-02-23T00:39:18.799544+0000","last_unstale":"2024-02-23T00:43:29.932823+0000","last_undegraded":"2024-02-23T00:43:29.932823+0000","last_fullsized":"2024-02-23T00:43:29.932823+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:42:36.214826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:36.214826+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":104,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.898425+0000","last_change":"2024-02-23T00:43:13.339003+0000","last_active":"2024-02-23T00:43:29.898425+0000","last_peered":"2024-02-23T00:43:29.898425+0000","last_clean":"2024-02-23T00:43:29.898425+0000","last_became_active":"2024-02-23T00:43:13.338697+0000","last_became_peered":"2024-02-23T00:43:13.338697+0000","last_unstale":"2024-02-23T00:43:29.898425+0000","last_undegraded":"2024-02-23T00:43:29.898425+0000","last_fullsized":"2024-02-23T00:43:29.898425+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:15.081153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:15.081153+0000","last_clean_scrub_stamp":"2024-02-23T00:41:15.081153+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.949437+0000","last_change":"2024-02-23T00:42:57.686234+0000","last_active":"2024-02-23T00:43:29.949437+0000","last_peered":"2024-02-23T00:43:29.949437+0000","last_clean":"2024-02-23T00:43:29.949437+0000","last_became_active":"2024-02-23T00:42:57.685579+0000","last_became_peered":"2024-02-23T00:42:57.685579+0000","last_unstale":"2024-02-23T00:43:29.949437+0000","last_undegraded":"2024-02-23T00:43:29.949437+0000","last_fullsized":"2024-02-23T00:43:29.949437+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.961042+0000","last_change":"2024-02-23T00:42:58.772582+0000","last_active":"2024-02-23T00:43:29.961042+0000","last_peered":"2024-02-23T00:43:29.961042+0000","last_clean":"2024-02-23T00:43:29.961042+0000","last_became_active":"2024-02-23T00:42:58.765106+0000","last_became_peered":"2024-02-23T00:42:58.765106+0000","last_unstale":"2024-02-23T00:43:29.961042+0000","last_undegraded":"2024-02-23T00:43:29.961042+0000","last_fullsized":"2024-02-23T00:43:29.961042+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1c","version":"110'1","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.895410+0000","last_change":"2024-02-23T00:42:58.731884+0000","last_active":"2024-02-23T00:43:29.895410+0000","last_peered":"2024-02-23T00:43:29.895410+0000","last_clean":"2024-02-23T00:43:29.895410+0000","last_became_active":"2024-02-23T00:42:58.727707+0000","last_became_peered":"2024-02-23T00:42:58.727707+0000","last_unstale":"2024-02-23T00:43:29.895410+0000","last_undegraded":"2024-02-23T00:43:29.895410+0000","last_fullsized":"2024-02-23T00:43:29.895410+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"67'15","reported_seq":120,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.949519+0000","last_change":"2024-02-23T00:41:39.682283+0000","last_active":"2024-02-23T00:43:29.949519+0000","last_peered":"2024-02-23T00:43:29.949519+0000","last_clean":"2024-02-23T00:43:29.949519+0000","last_became_active":"2024-02-23T00:39:16.789757+0000","last_became_peered":"2024-02-23T00:39:16.789757+0000","last_unstale":"2024-02-23T00:43:29.949519+0000","last_undegraded":"2024-02-23T00:43:29.949519+0000","last_fullsized":"2024-02-23T00:43:29.949519+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:41:39.682182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:39.682182+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":97,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.927480+0000","last_change":"2024-02-23T00:41:56.728679+0000","last_active":"2024-02-23T00:43:29.927480+0000","last_peered":"2024-02-23T00:43:29.927480+0000","last_clean":"2024-02-23T00:43:29.927480+0000","last_became_active":"2024-02-23T00:39:14.780526+0000","last_became_peered":"2024-02-23T00:39:14.780526+0000","last_unstale":"2024-02-23T00:43:29.927480+0000","last_undegraded":"2024-02-23T00:43:29.927480+0000","last_fullsized":"2024-02-23T00:43:29.927480+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:56.728577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:56.728577+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":152,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.898343+0000","last_change":"2024-02-23T00:41:49.927409+0000","last_active":"2024-02-23T00:43:29.898343+0000","last_peered":"2024-02-23T00:43:29.898343+0000","last_clean":"2024-02-23T00:43:29.898343+0000","last_became_active":"2024-02-23T00:39:18.800440+0000","last_became_peered":"2024-02-23T00:39:18.800440+0000","last_unstale":"2024-02-23T00:43:29.898343+0000","last_undegraded":"2024-02-23T00:43:29.898343+0000","last_fullsized":"2024-02-23T00:43:29.898343+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:41:49.927326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:49.927326+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":110,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.905419+0000","last_change":"2024-02-23T00:43:16.650829+0000","last_active":"2024-02-23T00:43:29.905419+0000","last_peered":"2024-02-23T00:43:29.905419+0000","last_clean":"2024-02-23T00:43:29.905419+0000","last_became_active":"2024-02-23T00:43:16.650527+0000","last_became_peered":"2024-02-23T00:43:16.650527+0000","last_unstale":"2024-02-23T00:43:29.905419+0000","last_undegraded":"2024-02-23T00:43:29.905419+0000","last_fullsized":"2024-02-23T00:43:29.905419+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:10.666299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:10.666299+0000","last_clean_scrub_stamp":"2024-02-23T00:42:10.666299+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.932406+0000","last_change":"2024-02-23T00:42:57.671776+0000","last_active":"2024-02-23T00:43:29.932406+0000","last_peered":"2024-02-23T00:43:29.932406+0000","last_clean":"2024-02-23T00:43:29.932406+0000","last_became_active":"2024-02-23T00:42:57.671472+0000","last_became_peered":"2024-02-23T00:42:57.671472+0000","last_unstale":"2024-02-23T00:43:29.932406+0000","last_undegraded":"2024-02-23T00:43:29.932406+0000","last_fullsized":"2024-02-23T00:43:29.932406+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,3],"acting":[1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.895769+0000","last_change":"2024-02-23T00:42:58.834770+0000","last_active":"2024-02-23T00:43:29.895769+0000","last_peered":"2024-02-23T00:43:29.895769+0000","last_clean":"2024-02-23T00:43:29.895769+0000","last_became_active":"2024-02-23T00:42:58.833924+0000","last_became_peered":"2024-02-23T00:42:58.833924+0000","last_unstale":"2024-02-23T00:43:29.895769+0000","last_undegraded":"2024-02-23T00:43:29.895769+0000","last_fullsized":"2024-02-23T00:43:29.895769+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":38,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.908621+0000","last_change":"2024-02-23T00:42:58.749425+0000","last_active":"2024-02-23T00:43:29.908621+0000","last_peered":"2024-02-23T00:43:29.908621+0000","last_clean":"2024-02-23T00:43:29.908621+0000","last_became_active":"2024-02-23T00:42:58.744735+0000","last_became_peered":"2024-02-23T00:42:58.744735+0000","last_unstale":"2024-02-23T00:43:29.908621+0000","last_undegraded":"2024-02-23T00:43:29.908621+0000","last_fullsized":"2024-02-23T00:43:29.908621+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"68'17","reported_seq":147,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.898199+0000","last_change":"2024-02-23T00:41:13.017122+0000","last_active":"2024-02-23T00:43:29.898199+0000","last_peered":"2024-02-23T00:43:29.898199+0000","last_clean":"2024-02-23T00:43:29.898199+0000","last_became_active":"2024-02-23T00:40:42.933759+0000","last_became_peered":"2024-02-23T00:40:42.933759+0000","last_unstale":"2024-02-23T00:43:29.898199+0000","last_undegraded":"2024-02-23T00:43:29.898199+0000","last_fullsized":"2024-02-23T00:43:29.898199+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:41:13.017043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:13.017043+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":96,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.914203+0000","last_change":"2024-02-23T00:41:42.978987+0000","last_active":"2024-02-23T00:43:29.914203+0000","last_peered":"2024-02-23T00:43:29.914203+0000","last_clean":"2024-02-23T00:43:29.914203+0000","last_became_active":"2024-02-23T00:39:14.784121+0000","last_became_peered":"2024-02-23T00:39:14.784121+0000","last_unstale":"2024-02-23T00:43:29.914203+0000","last_undegraded":"2024-02-23T00:43:29.914203+0000","last_fullsized":"2024-02-23T00:43:29.914203+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:42.978905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:42.978905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":94,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.904626+0000","last_change":"2024-02-23T00:42:53.965005+0000","last_active":"2024-02-23T00:43:29.904626+0000","last_peered":"2024-02-23T00:43:29.904626+0000","last_clean":"2024-02-23T00:43:29.904626+0000","last_became_active":"2024-02-23T00:39:18.800777+0000","last_became_peered":"2024-02-23T00:39:18.800777+0000","last_unstale":"2024-02-23T00:43:29.904626+0000","last_undegraded":"2024-02-23T00:43:29.904626+0000","last_fullsized":"2024-02-23T00:43:29.904626+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.964934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.964934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"67'1","reported_seq":110,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.897124+0000","last_change":"2024-02-23T00:43:22.866715+0000","last_active":"2024-02-23T00:43:29.897124+0000","last_peered":"2024-02-23T00:43:29.897124+0000","last_clean":"2024-02-23T00:43:29.897124+0000","last_became_active":"2024-02-23T00:43:22.866277+0000","last_became_peered":"2024-02-23T00:43:22.866277+0000","last_unstale":"2024-02-23T00:43:29.897124+0000","last_undegraded":"2024-02-23T00:43:29.897124+0000","last_fullsized":"2024-02-23T00:43:29.897124+0000","mapping_epoch":65,"log_start":"67'1","ondisk_log_start":"67'1","created":65,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:09.079918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:09.079918+0000","last_clean_scrub_stamp":"2024-02-23T00:42:09.079918+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.910095+0000","last_change":"2024-02-23T00:42:58.719381+0000","last_active":"2024-02-23T00:43:29.910095+0000","last_peered":"2024-02-23T00:43:29.910095+0000","last_clean":"2024-02-23T00:43:29.910095+0000","last_became_active":"2024-02-23T00:42:58.718615+0000","last_became_peered":"2024-02-23T00:42:58.718615+0000","last_unstale":"2024-02-23T00:43:29.910095+0000","last_undegraded":"2024-02-23T00:43:29.910095+0000","last_fullsized":"2024-02-23T00:43:29.910095+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.945358+0000","last_change":"2024-02-23T00:42:57.655252+0000","last_active":"2024-02-23T00:43:29.945358+0000","last_peered":"2024-02-23T00:43:29.945358+0000","last_clean":"2024-02-23T00:43:29.945358+0000","last_became_active":"2024-02-23T00:42:57.653279+0000","last_became_peered":"2024-02-23T00:42:57.653279+0000","last_unstale":"2024-02-23T00:43:29.945358+0000","last_undegraded":"2024-02-23T00:43:29.945358+0000","last_fullsized":"2024-02-23T00:43:29.945358+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1e","version":"112'1004","reported_seq":2046,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.960252+0000","last_change":"2024-02-23T00:42:58.808262+0000","last_active":"2024-02-23T00:43:29.960252+0000","last_peered":"2024-02-23T00:43:29.960252+0000","last_clean":"2024-02-23T00:43:29.960252+0000","last_became_active":"2024-02-23T00:42:58.807886+0000","last_became_peered":"2024-02-23T00:42:58.807886+0000","last_unstale":"2024-02-23T00:43:29.960252+0000","last_undegraded":"2024-02-23T00:43:29.960252+0000","last_fullsized":"2024-02-23T00:43:29.960252+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":1004,"ondisk_log_size":1004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25165850,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1001,"num_read_kb":24577,"num_write":1003,"num_write_kb":24579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":153,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.921509+0000","last_change":"2024-02-23T00:42:53.978054+0000","last_active":"2024-02-23T00:43:29.921509+0000","last_peered":"2024-02-23T00:43:29.921509+0000","last_clean":"2024-02-23T00:43:29.921509+0000","last_became_active":"2024-02-23T00:39:18.798536+0000","last_became_peered":"2024-02-23T00:39:18.798536+0000","last_unstale":"2024-02-23T00:43:29.921509+0000","last_undegraded":"2024-02-23T00:43:29.921509+0000","last_fullsized":"2024-02-23T00:43:29.921509+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:42:53.977950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.977950+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"68'20","reported_seq":131,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.895986+0000","last_change":"2024-02-23T00:42:42.204029+0000","last_active":"2024-02-23T00:43:29.895986+0000","last_peered":"2024-02-23T00:43:29.895986+0000","last_clean":"2024-02-23T00:43:29.895986+0000","last_became_active":"2024-02-23T00:39:16.780130+0000","last_became_peered":"2024-02-23T00:39:16.780130+0000","last_unstale":"2024-02-23T00:43:29.895986+0000","last_undegraded":"2024-02-23T00:43:29.895986+0000","last_fullsized":"2024-02-23T00:43:29.895986+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'20","last_scrub_stamp":"2024-02-23T00:42:42.203911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:42.203911+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":99,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.909651+0000","last_change":"2024-02-23T00:42:04.191017+0000","last_active":"2024-02-23T00:43:29.909651+0000","last_peered":"2024-02-23T00:43:29.909651+0000","last_clean":"2024-02-23T00:43:29.909651+0000","last_became_active":"2024-02-23T00:39:14.781999+0000","last_became_peered":"2024-02-23T00:39:14.781999+0000","last_unstale":"2024-02-23T00:43:29.909651+0000","last_undegraded":"2024-02-23T00:43:29.909651+0000","last_fullsized":"2024-02-23T00:43:29.909651+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:04.190935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:04.190935+0000","last_clean_scrub_stamp":"2024-02-23T00:42:04.190935+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":90,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.916325+0000","last_change":"2024-02-23T00:42:06.945763+0000","last_active":"2024-02-23T00:43:29.916325+0000","last_peered":"2024-02-23T00:43:29.916325+0000","last_clean":"2024-02-23T00:43:29.916325+0000","last_became_active":"2024-02-23T00:39:21.161928+0000","last_became_peered":"2024-02-23T00:39:21.161928+0000","last_unstale":"2024-02-23T00:43:29.916325+0000","last_undegraded":"2024-02-23T00:43:29.916325+0000","last_fullsized":"2024-02-23T00:43:29.916325+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:06.945642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.945642+0000","last_clean_scrub_stamp":"2024-02-23T00:42:06.945642+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"110'10","reported_seq":49,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.935440+0000","last_change":"2024-02-23T00:42:58.696579+0000","last_active":"2024-02-23T00:43:29.935440+0000","last_peered":"2024-02-23T00:43:29.935440+0000","last_clean":"2024-02-23T00:43:29.935440+0000","last_became_active":"2024-02-23T00:42:58.696292+0000","last_became_peered":"2024-02-23T00:42:58.696292+0000","last_unstale":"2024-02-23T00:43:29.935440+0000","last_undegraded":"2024-02-23T00:43:29.935440+0000","last_fullsized":"2024-02-23T00:43:29.935440+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.904860+0000","last_change":"2024-02-23T00:42:58.717788+0000","last_active":"2024-02-23T00:43:29.904860+0000","last_peered":"2024-02-23T00:43:29.904860+0000","last_clean":"2024-02-23T00:43:29.904860+0000","last_became_active":"2024-02-23T00:42:58.717371+0000","last_became_peered":"2024-02-23T00:42:58.717371+0000","last_unstale":"2024-02-23T00:43:29.904860+0000","last_undegraded":"2024-02-23T00:43:29.904860+0000","last_fullsized":"2024-02-23T00:43:29.904860+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"112'6","reported_seq":47,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.927711+0000","last_change":"2024-02-23T00:42:58.762595+0000","last_active":"2024-02-23T00:43:29.927711+0000","last_peered":"2024-02-23T00:43:29.927711+0000","last_clean":"2024-02-23T00:43:29.927711+0000","last_became_active":"2024-02-23T00:42:58.757008+0000","last_became_peered":"2024-02-23T00:42:58.757008+0000","last_unstale":"2024-02-23T00:43:29.927711+0000","last_undegraded":"2024-02-23T00:43:29.927711+0000","last_fullsized":"2024-02-23T00:43:29.927711+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":93,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.897705+0000","last_change":"2024-02-23T00:41:52.965337+0000","last_active":"2024-02-23T00:43:29.897705+0000","last_peered":"2024-02-23T00:43:29.897705+0000","last_clean":"2024-02-23T00:43:29.897705+0000","last_became_active":"2024-02-23T00:39:18.798829+0000","last_became_peered":"2024-02-23T00:39:18.798829+0000","last_unstale":"2024-02-23T00:43:29.897705+0000","last_undegraded":"2024-02-23T00:43:29.897705+0000","last_fullsized":"2024-02-23T00:43:29.897705+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:52.965253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:52.965253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"68'30","reported_seq":158,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.921429+0000","last_change":"2024-02-23T00:42:49.970825+0000","last_active":"2024-02-23T00:43:29.921429+0000","last_peered":"2024-02-23T00:43:29.921429+0000","last_clean":"2024-02-23T00:43:29.921429+0000","last_became_active":"2024-02-23T00:39:16.859002+0000","last_became_peered":"2024-02-23T00:39:16.859002+0000","last_unstale":"2024-02-23T00:43:29.921429+0000","last_undegraded":"2024-02-23T00:43:29.921429+0000","last_fullsized":"2024-02-23T00:43:29.921429+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'30","last_scrub_stamp":"2024-02-23T00:42:49.970752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:49.970752+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":97,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.919736+0000","last_change":"2024-02-23T00:42:40.534390+0000","last_active":"2024-02-23T00:43:29.919736+0000","last_peered":"2024-02-23T00:43:29.919736+0000","last_clean":"2024-02-23T00:43:29.919736+0000","last_became_active":"2024-02-23T00:39:14.779193+0000","last_became_peered":"2024-02-23T00:39:14.779193+0000","last_unstale":"2024-02-23T00:43:29.919736+0000","last_undegraded":"2024-02-23T00:43:29.919736+0000","last_fullsized":"2024-02-23T00:43:29.919736+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:40.534284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:40.534284+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":91,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.902348+0000","last_change":"2024-02-23T00:42:36.113546+0000","last_active":"2024-02-23T00:43:29.902348+0000","last_peered":"2024-02-23T00:43:29.902348+0000","last_clean":"2024-02-23T00:43:29.902348+0000","last_became_active":"2024-02-23T00:39:20.795271+0000","last_became_peered":"2024-02-23T00:39:20.795271+0000","last_unstale":"2024-02-23T00:43:29.902348+0000","last_undegraded":"2024-02-23T00:43:29.902348+0000","last_fullsized":"2024-02-23T00:43:29.902348+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:36.113469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:36.113469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"110'2","reported_seq":41,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.927443+0000","last_change":"2024-02-23T00:42:58.816302+0000","last_active":"2024-02-23T00:43:29.927443+0000","last_peered":"2024-02-23T00:43:29.927443+0000","last_clean":"2024-02-23T00:43:29.927443+0000","last_became_active":"2024-02-23T00:42:58.814093+0000","last_became_peered":"2024-02-23T00:42:58.814093+0000","last_unstale":"2024-02-23T00:43:29.927443+0000","last_undegraded":"2024-02-23T00:43:29.927443+0000","last_fullsized":"2024-02-23T00:43:29.927443+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.960084+0000","last_change":"2024-02-23T00:42:58.809948+0000","last_active":"2024-02-23T00:43:29.960084+0000","last_peered":"2024-02-23T00:43:29.960084+0000","last_clean":"2024-02-23T00:43:29.960084+0000","last_became_active":"2024-02-23T00:42:58.809283+0000","last_became_peered":"2024-02-23T00:42:58.809283+0000","last_unstale":"2024-02-23T00:43:29.960084+0000","last_undegraded":"2024-02-23T00:43:29.960084+0000","last_fullsized":"2024-02-23T00:43:29.960084+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.10","version":"112'3","reported_seq":49,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.928566+0000","last_change":"2024-02-23T00:42:58.700115+0000","last_active":"2024-02-23T00:43:29.928566+0000","last_peered":"2024-02-23T00:43:29.928566+0000","last_clean":"2024-02-23T00:43:29.928566+0000","last_became_active":"2024-02-23T00:42:58.699664+0000","last_became_peered":"2024-02-23T00:42:58.699664+0000","last_unstale":"2024-02-23T00:43:29.928566+0000","last_undegraded":"2024-02-23T00:43:29.928566+0000","last_fullsized":"2024-02-23T00:43:29.928566+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25165843,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":24578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":93,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.909134+0000","last_change":"2024-02-23T00:42:36.454764+0000","last_active":"2024-02-23T00:43:29.909134+0000","last_peered":"2024-02-23T00:43:29.909134+0000","last_clean":"2024-02-23T00:43:29.909134+0000","last_became_active":"2024-02-23T00:39:18.796664+0000","last_became_peered":"2024-02-23T00:39:18.796664+0000","last_unstale":"2024-02-23T00:43:29.909134+0000","last_undegraded":"2024-02-23T00:43:29.909134+0000","last_fullsized":"2024-02-23T00:43:29.909134+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:36.454660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:36.454660+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"68'25","reported_seq":147,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.933258+0000","last_change":"2024-02-23T00:42:34.248481+0000","last_active":"2024-02-23T00:43:29.933258+0000","last_peered":"2024-02-23T00:43:29.933258+0000","last_clean":"2024-02-23T00:43:29.933258+0000","last_became_active":"2024-02-23T00:39:16.857278+0000","last_became_peered":"2024-02-23T00:39:16.857278+0000","last_unstale":"2024-02-23T00:43:29.933258+0000","last_undegraded":"2024-02-23T00:43:29.933258+0000","last_fullsized":"2024-02-23T00:43:29.933258+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'25","last_scrub_stamp":"2024-02-23T00:42:34.248371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:34.248371+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":96,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.904758+0000","last_change":"2024-02-23T00:42:43.990569+0000","last_active":"2024-02-23T00:43:29.904758+0000","last_peered":"2024-02-23T00:43:29.904758+0000","last_clean":"2024-02-23T00:43:29.904758+0000","last_became_active":"2024-02-23T00:39:14.783662+0000","last_became_peered":"2024-02-23T00:39:14.783662+0000","last_unstale":"2024-02-23T00:43:29.904758+0000","last_undegraded":"2024-02-23T00:43:29.904758+0000","last_fullsized":"2024-02-23T00:43:29.904758+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:43.990486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:43.990486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":91,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.927599+0000","last_change":"2024-02-23T00:41:50.656785+0000","last_active":"2024-02-23T00:43:29.927599+0000","last_peered":"2024-02-23T00:43:29.927599+0000","last_clean":"2024-02-23T00:43:29.927599+0000","last_became_active":"2024-02-23T00:39:21.160060+0000","last_became_peered":"2024-02-23T00:39:21.160060+0000","last_unstale":"2024-02-23T00:43:29.927599+0000","last_undegraded":"2024-02-23T00:43:29.927599+0000","last_fullsized":"2024-02-23T00:43:29.927599+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:50.656709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:50.656709+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.8","version":"110'2","reported_seq":41,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.933130+0000","last_change":"2024-02-23T00:42:58.810425+0000","last_active":"2024-02-23T00:43:29.933130+0000","last_peered":"2024-02-23T00:43:29.933130+0000","last_clean":"2024-02-23T00:43:29.933130+0000","last_became_active":"2024-02-23T00:42:58.809584+0000","last_became_peered":"2024-02-23T00:42:58.809584+0000","last_unstale":"2024-02-23T00:43:29.933130+0000","last_undegraded":"2024-02-23T00:43:29.933130+0000","last_fullsized":"2024-02-23T00:43:29.933130+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.905017+0000","last_change":"2024-02-23T00:42:58.726955+0000","last_active":"2024-02-23T00:43:29.905017+0000","last_peered":"2024-02-23T00:43:29.905017+0000","last_clean":"2024-02-23T00:43:29.905017+0000","last_became_active":"2024-02-23T00:42:58.726652+0000","last_became_peered":"2024-02-23T00:42:58.726652+0000","last_unstale":"2024-02-23T00:43:29.905017+0000","last_undegraded":"2024-02-23T00:43:29.905017+0000","last_fullsized":"2024-02-23T00:43:29.905017+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.13","version":"112'1","reported_seq":41,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-02-23T00:43:32.088186+0000","last_change":"2024-02-23T00:42:58.804930+0000","last_active":"2024-02-23T00:43:32.088186+0000","last_peered":"2024-02-23T00:43:32.088186+0000","last_clean":"2024-02-23T00:43:32.088186+0000","last_became_active":"2024-02-23T00:42:58.804240+0000","last_became_peered":"2024-02-23T00:42:58.804240+0000","last_unstale":"2024-02-23T00:43:32.088186+0000","last_undegraded":"2024-02-23T00:43:32.088186+0000","last_fullsized":"2024-02-23T00:43:32.088186+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25165828,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":24577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7],"acting":[6,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":94,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.904701+0000","last_change":"2024-02-23T00:41:59.694132+0000","last_active":"2024-02-23T00:43:29.904701+0000","last_peered":"2024-02-23T00:43:29.904701+0000","last_clean":"2024-02-23T00:43:29.904701+0000","last_became_active":"2024-02-23T00:39:18.800378+0000","last_became_peered":"2024-02-23T00:39:18.800378+0000","last_unstale":"2024-02-23T00:43:29.904701+0000","last_undegraded":"2024-02-23T00:43:29.904701+0000","last_fullsized":"2024-02-23T00:43:29.904701+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:59.694033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:59.694033+0000","last_clean_scrub_stamp":"2024-02-23T00:41:59.694033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"95'18","reported_seq":120,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.931240+0000","last_change":"2024-02-23T00:42:01.195610+0000","last_active":"2024-02-23T00:43:29.931240+0000","last_peered":"2024-02-23T00:43:29.931240+0000","last_clean":"2024-02-23T00:43:29.931240+0000","last_became_active":"2024-02-23T00:39:16.787143+0000","last_became_peered":"2024-02-23T00:39:16.787143+0000","last_unstale":"2024-02-23T00:43:29.931240+0000","last_undegraded":"2024-02-23T00:43:29.931240+0000","last_fullsized":"2024-02-23T00:43:29.931240+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'18","last_scrub_stamp":"2024-02-23T00:42:01.195545+0000","last_deep_scrub":"95'18","last_deep_scrub_stamp":"2024-02-23T00:42:01.195545+0000","last_clean_scrub_stamp":"2024-02-23T00:42:01.195545+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":101,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.914052+0000","last_change":"2024-02-23T00:42:50.947420+0000","last_active":"2024-02-23T00:43:29.914052+0000","last_peered":"2024-02-23T00:43:29.914052+0000","last_clean":"2024-02-23T00:43:29.914052+0000","last_became_active":"2024-02-23T00:39:14.784578+0000","last_became_peered":"2024-02-23T00:39:14.784578+0000","last_unstale":"2024-02-23T00:43:29.914052+0000","last_undegraded":"2024-02-23T00:43:29.914052+0000","last_fullsized":"2024-02-23T00:43:29.914052+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:50.947349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:50.947349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":98,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.935310+0000","last_change":"2024-02-23T00:41:49.197029+0000","last_active":"2024-02-23T00:43:29.935310+0000","last_peered":"2024-02-23T00:43:29.935310+0000","last_clean":"2024-02-23T00:43:29.935310+0000","last_became_active":"2024-02-23T00:39:14.775446+0000","last_became_peered":"2024-02-23T00:39:14.775446+0000","last_unstale":"2024-02-23T00:43:29.935310+0000","last_undegraded":"2024-02-23T00:43:29.935310+0000","last_fullsized":"2024-02-23T00:43:29.935310+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:49.196953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:49.196953+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":86,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.933260+0000","last_change":"2024-02-23T00:41:02.714085+0000","last_active":"2024-02-23T00:43:29.933260+0000","last_peered":"2024-02-23T00:43:29.933260+0000","last_clean":"2024-02-23T00:43:29.933260+0000","last_became_active":"2024-02-23T00:39:20.797242+0000","last_became_peered":"2024-02-23T00:39:20.797242+0000","last_unstale":"2024-02-23T00:43:29.933260+0000","last_undegraded":"2024-02-23T00:43:29.933260+0000","last_fullsized":"2024-02-23T00:43:29.933260+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:02.714020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:02.714020+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"110'5","reported_seq":44,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.904368+0000","last_change":"2024-02-23T00:42:58.743470+0000","last_active":"2024-02-23T00:43:29.904368+0000","last_peered":"2024-02-23T00:43:29.904368+0000","last_clean":"2024-02-23T00:43:29.904368+0000","last_became_active":"2024-02-23T00:42:58.742998+0000","last_became_peered":"2024-02-23T00:42:58.742998+0000","last_unstale":"2024-02-23T00:43:29.904368+0000","last_undegraded":"2024-02-23T00:43:29.904368+0000","last_fullsized":"2024-02-23T00:43:29.904368+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.900540+0000","last_change":"2024-02-23T00:42:58.691842+0000","last_active":"2024-02-23T00:43:29.900540+0000","last_peered":"2024-02-23T00:43:29.900540+0000","last_clean":"2024-02-23T00:43:29.900540+0000","last_became_active":"2024-02-23T00:42:58.690858+0000","last_became_peered":"2024-02-23T00:42:58.690858+0000","last_unstale":"2024-02-23T00:43:29.900540+0000","last_undegraded":"2024-02-23T00:43:29.900540+0000","last_fullsized":"2024-02-23T00:43:29.900540+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.16","version":"110'1","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.908027+0000","last_change":"2024-02-23T00:42:58.772995+0000","last_active":"2024-02-23T00:43:29.908027+0000","last_peered":"2024-02-23T00:43:29.908027+0000","last_clean":"2024-02-23T00:43:29.908027+0000","last_became_active":"2024-02-23T00:42:58.772347+0000","last_became_peered":"2024-02-23T00:42:58.772347+0000","last_unstale":"2024-02-23T00:43:29.908027+0000","last_undegraded":"2024-02-23T00:43:29.908027+0000","last_fullsized":"2024-02-23T00:43:29.908027+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":92,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.917124+0000","last_change":"2024-02-23T00:42:42.029769+0000","last_active":"2024-02-23T00:43:29.917124+0000","last_peered":"2024-02-23T00:43:29.917124+0000","last_clean":"2024-02-23T00:43:29.917124+0000","last_became_active":"2024-02-23T00:39:18.793801+0000","last_became_peered":"2024-02-23T00:39:18.793801+0000","last_unstale":"2024-02-23T00:43:29.917124+0000","last_undegraded":"2024-02-23T00:43:29.917124+0000","last_fullsized":"2024-02-23T00:43:29.917124+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:42.029652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:42.029652+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"67'14","reported_seq":117,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.906562+0000","last_change":"2024-02-23T00:41:48.041191+0000","last_active":"2024-02-23T00:43:29.906562+0000","last_peered":"2024-02-23T00:43:29.906562+0000","last_clean":"2024-02-23T00:43:29.906562+0000","last_became_active":"2024-02-23T00:39:16.858880+0000","last_became_peered":"2024-02-23T00:39:16.858880+0000","last_unstale":"2024-02-23T00:43:29.906562+0000","last_undegraded":"2024-02-23T00:43:29.906562+0000","last_fullsized":"2024-02-23T00:43:29.906562+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'14","last_scrub_stamp":"2024-02-23T00:41:48.041106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:48.041106+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.914129+0000","last_change":"2024-02-23T00:42:49.977357+0000","last_active":"2024-02-23T00:43:29.914129+0000","last_peered":"2024-02-23T00:43:29.914129+0000","last_clean":"2024-02-23T00:43:29.914129+0000","last_became_active":"2024-02-23T00:39:08.248249+0000","last_became_peered":"2024-02-23T00:39:08.248249+0000","last_unstale":"2024-02-23T00:43:29.914129+0000","last_undegraded":"2024-02-23T00:43:29.914129+0000","last_fullsized":"2024-02-23T00:43:29.914129+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:49.977248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:49.977248+0000","last_clean_scrub_stamp":"2024-02-23T00:42:49.977248+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":98,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.900686+0000","last_change":"2024-02-23T00:42:46.945425+0000","last_active":"2024-02-23T00:43:29.900686+0000","last_peered":"2024-02-23T00:43:29.900686+0000","last_clean":"2024-02-23T00:43:29.900686+0000","last_became_active":"2024-02-23T00:39:14.788888+0000","last_became_peered":"2024-02-23T00:39:14.788888+0000","last_unstale":"2024-02-23T00:43:29.900686+0000","last_undegraded":"2024-02-23T00:43:29.900686+0000","last_fullsized":"2024-02-23T00:43:29.900686+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:46.945358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:46.945358+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":98,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.908092+0000","last_change":"2024-02-23T00:42:34.984785+0000","last_active":"2024-02-23T00:43:29.908092+0000","last_peered":"2024-02-23T00:43:29.908092+0000","last_clean":"2024-02-23T00:43:29.908092+0000","last_became_active":"2024-02-23T00:39:14.783489+0000","last_became_peered":"2024-02-23T00:39:14.783489+0000","last_unstale":"2024-02-23T00:43:29.908092+0000","last_undegraded":"2024-02-23T00:43:29.908092+0000","last_fullsized":"2024-02-23T00:43:29.908092+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:34.984682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:34.984682+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":92,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.904782+0000","last_change":"2024-02-23T00:41:55.726397+0000","last_active":"2024-02-23T00:43:29.904782+0000","last_peered":"2024-02-23T00:43:29.904782+0000","last_clean":"2024-02-23T00:43:29.904782+0000","last_became_active":"2024-02-23T00:39:20.797226+0000","last_became_peered":"2024-02-23T00:39:20.797226+0000","last_unstale":"2024-02-23T00:43:29.904782+0000","last_undegraded":"2024-02-23T00:43:29.904782+0000","last_fullsized":"2024-02-23T00:43:29.904782+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:55.726321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:55.726321+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.956508+0000","last_change":"2024-02-23T00:42:58.721944+0000","last_active":"2024-02-23T00:43:29.956508+0000","last_peered":"2024-02-23T00:43:29.956508+0000","last_clean":"2024-02-23T00:43:29.956508+0000","last_became_active":"2024-02-23T00:42:58.721439+0000","last_became_peered":"2024-02-23T00:42:58.721439+0000","last_unstale":"2024-02-23T00:43:29.956508+0000","last_undegraded":"2024-02-23T00:43:29.956508+0000","last_fullsized":"2024-02-23T00:43:29.956508+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.916168+0000","last_change":"2024-02-23T00:42:58.701934+0000","last_active":"2024-02-23T00:43:29.916168+0000","last_peered":"2024-02-23T00:43:29.916168+0000","last_clean":"2024-02-23T00:43:29.916168+0000","last_became_active":"2024-02-23T00:42:58.701502+0000","last_became_peered":"2024-02-23T00:42:58.701502+0000","last_unstale":"2024-02-23T00:43:29.916168+0000","last_undegraded":"2024-02-23T00:43:29.916168+0000","last_fullsized":"2024-02-23T00:43:29.916168+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.15","version":"110'2","reported_seq":41,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.906654+0000","last_change":"2024-02-23T00:42:58.672263+0000","last_active":"2024-02-23T00:43:29.906654+0000","last_peered":"2024-02-23T00:43:29.906654+0000","last_clean":"2024-02-23T00:43:29.906654+0000","last_became_active":"2024-02-23T00:42:58.670377+0000","last_became_peered":"2024-02-23T00:42:58.670377+0000","last_unstale":"2024-02-23T00:43:29.906654+0000","last_undegraded":"2024-02-23T00:43:29.906654+0000","last_fullsized":"2024-02-23T00:43:29.906654+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":93,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.956739+0000","last_change":"2024-02-23T00:41:52.264308+0000","last_active":"2024-02-23T00:43:29.956739+0000","last_peered":"2024-02-23T00:43:29.956739+0000","last_clean":"2024-02-23T00:43:29.956739+0000","last_became_active":"2024-02-23T00:39:18.789330+0000","last_became_peered":"2024-02-23T00:39:18.789330+0000","last_unstale":"2024-02-23T00:43:29.956739+0000","last_undegraded":"2024-02-23T00:43:29.956739+0000","last_fullsized":"2024-02-23T00:43:29.956739+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:52.264240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.264240+0000","last_clean_scrub_stamp":"2024-02-23T00:41:52.264240+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"68'19","reported_seq":137,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.949705+0000","last_change":"2024-02-23T00:42:04.625118+0000","last_active":"2024-02-23T00:43:29.949705+0000","last_peered":"2024-02-23T00:43:29.949705+0000","last_clean":"2024-02-23T00:43:29.949705+0000","last_became_active":"2024-02-23T00:39:16.787484+0000","last_became_peered":"2024-02-23T00:39:16.787484+0000","last_unstale":"2024-02-23T00:43:29.949705+0000","last_undegraded":"2024-02-23T00:43:29.949705+0000","last_fullsized":"2024-02-23T00:43:29.949705+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'19","last_scrub_stamp":"2024-02-23T00:42:04.625006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:04.625006+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":97,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.923094+0000","last_change":"2024-02-23T00:42:05.329858+0000","last_active":"2024-02-23T00:43:29.923094+0000","last_peered":"2024-02-23T00:43:29.923094+0000","last_clean":"2024-02-23T00:43:29.923094+0000","last_became_active":"2024-02-23T00:39:14.772411+0000","last_became_peered":"2024-02-23T00:39:14.772411+0000","last_unstale":"2024-02-23T00:43:29.923094+0000","last_undegraded":"2024-02-23T00:43:29.923094+0000","last_fullsized":"2024-02-23T00:43:29.923094+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:05.329757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:05.329757+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"68'5","reported_seq":344,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-02-23T00:43:31.193340+0000","last_change":"2024-02-23T00:41:00.648420+0000","last_active":"2024-02-23T00:43:31.193340+0000","last_peered":"2024-02-23T00:43:31.193340+0000","last_clean":"2024-02-23T00:43:31.193340+0000","last_became_active":"2024-02-23T00:39:14.779109+0000","last_became_peered":"2024-02-23T00:39:14.779109+0000","last_unstale":"2024-02-23T00:43:31.193340+0000","last_undegraded":"2024-02-23T00:43:31.193340+0000","last_fullsized":"2024-02-23T00:43:31.193340+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'5","last_scrub_stamp":"2024-02-23T00:41:00.648353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:00.648353+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":249,"num_read_kb":244,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":90,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.904682+0000","last_change":"2024-02-23T00:41:44.001469+0000","last_active":"2024-02-23T00:43:29.904682+0000","last_peered":"2024-02-23T00:43:29.904682+0000","last_clean":"2024-02-23T00:43:29.904682+0000","last_became_active":"2024-02-23T00:39:21.160457+0000","last_became_peered":"2024-02-23T00:39:21.160457+0000","last_unstale":"2024-02-23T00:43:29.904682+0000","last_undegraded":"2024-02-23T00:43:29.904682+0000","last_fullsized":"2024-02-23T00:43:29.904682+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:44.001408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:44.001408+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.903186+0000","last_change":"2024-02-23T00:42:58.831449+0000","last_active":"2024-02-23T00:43:29.903186+0000","last_peered":"2024-02-23T00:43:29.903186+0000","last_clean":"2024-02-23T00:43:29.903186+0000","last_became_active":"2024-02-23T00:42:58.830815+0000","last_became_peered":"2024-02-23T00:42:58.830815+0000","last_unstale":"2024-02-23T00:43:29.903186+0000","last_undegraded":"2024-02-23T00:43:29.903186+0000","last_fullsized":"2024-02-23T00:43:29.903186+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.927054+0000","last_change":"2024-02-23T00:42:57.653669+0000","last_active":"2024-02-23T00:43:29.927054+0000","last_peered":"2024-02-23T00:43:29.927054+0000","last_clean":"2024-02-23T00:43:29.927054+0000","last_became_active":"2024-02-23T00:42:57.653355+0000","last_became_peered":"2024-02-23T00:42:57.653355+0000","last_unstale":"2024-02-23T00:43:29.927054+0000","last_undegraded":"2024-02-23T00:43:29.927054+0000","last_fullsized":"2024-02-23T00:43:29.927054+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.17","version":"110'8","reported_seq":47,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.960559+0000","last_change":"2024-02-23T00:42:58.729148+0000","last_active":"2024-02-23T00:43:29.960559+0000","last_peered":"2024-02-23T00:43:29.960559+0000","last_clean":"2024-02-23T00:43:29.960559+0000","last_became_active":"2024-02-23T00:42:58.726659+0000","last_became_peered":"2024-02-23T00:42:58.726659+0000","last_unstale":"2024-02-23T00:43:29.960559+0000","last_undegraded":"2024-02-23T00:43:29.960559+0000","last_fullsized":"2024-02-23T00:43:29.960559+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":94,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.914236+0000","last_change":"2024-02-23T00:42:47.349912+0000","last_active":"2024-02-23T00:43:29.914236+0000","last_peered":"2024-02-23T00:43:29.914236+0000","last_clean":"2024-02-23T00:43:29.914236+0000","last_became_active":"2024-02-23T00:39:18.802134+0000","last_became_peered":"2024-02-23T00:39:18.802134+0000","last_unstale":"2024-02-23T00:43:29.914236+0000","last_undegraded":"2024-02-23T00:43:29.914236+0000","last_fullsized":"2024-02-23T00:43:29.914236+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:47.349847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:47.349847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"86'35","reported_seq":131,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.903368+0000","last_change":"2024-02-23T00:42:47.988747+0000","last_active":"2024-02-23T00:43:29.903368+0000","last_peered":"2024-02-23T00:43:29.903368+0000","last_clean":"2024-02-23T00:43:29.903368+0000","last_became_active":"2024-02-23T00:40:42.933322+0000","last_became_peered":"2024-02-23T00:40:42.933322+0000","last_unstale":"2024-02-23T00:43:29.903368+0000","last_undegraded":"2024-02-23T00:43:29.903368+0000","last_fullsized":"2024-02-23T00:43:29.903368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'35","last_scrub_stamp":"2024-02-23T00:42:47.988676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:47.988676+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":98,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.904481+0000","last_change":"2024-02-23T00:42:04.708358+0000","last_active":"2024-02-23T00:43:29.904481+0000","last_peered":"2024-02-23T00:43:29.904481+0000","last_clean":"2024-02-23T00:43:29.904481+0000","last_became_active":"2024-02-23T00:39:14.784015+0000","last_became_peered":"2024-02-23T00:39:14.784015+0000","last_unstale":"2024-02-23T00:43:29.904481+0000","last_undegraded":"2024-02-23T00:43:29.904481+0000","last_fullsized":"2024-02-23T00:43:29.904481+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:04.708280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:04.708280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":87,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.959813+0000","last_change":"2024-02-23T00:41:18.910682+0000","last_active":"2024-02-23T00:43:29.959813+0000","last_peered":"2024-02-23T00:43:29.959813+0000","last_clean":"2024-02-23T00:43:29.959813+0000","last_became_active":"2024-02-23T00:39:20.790053+0000","last_became_peered":"2024-02-23T00:39:20.790053+0000","last_unstale":"2024-02-23T00:43:29.959813+0000","last_undegraded":"2024-02-23T00:43:29.959813+0000","last_fullsized":"2024-02-23T00:43:29.959813+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:18.910605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:18.910605+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.909379+0000","last_change":"2024-02-23T00:42:58.755124+0000","last_active":"2024-02-23T00:43:29.909379+0000","last_peered":"2024-02-23T00:43:29.909379+0000","last_clean":"2024-02-23T00:43:29.909379+0000","last_became_active":"2024-02-23T00:42:58.754813+0000","last_became_peered":"2024-02-23T00:42:58.754813+0000","last_unstale":"2024-02-23T00:43:29.909379+0000","last_undegraded":"2024-02-23T00:43:29.909379+0000","last_fullsized":"2024-02-23T00:43:29.909379+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.935067+0000","last_change":"2024-02-23T00:42:58.767404+0000","last_active":"2024-02-23T00:43:29.935067+0000","last_peered":"2024-02-23T00:43:29.935067+0000","last_clean":"2024-02-23T00:43:29.935067+0000","last_became_active":"2024-02-23T00:42:58.766579+0000","last_became_peered":"2024-02-23T00:42:58.766579+0000","last_unstale":"2024-02-23T00:43:29.935067+0000","last_undegraded":"2024-02-23T00:43:29.935067+0000","last_fullsized":"2024-02-23T00:43:29.935067+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":38,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.933684+0000","last_change":"2024-02-23T00:42:58.874626+0000","last_active":"2024-02-23T00:43:29.933684+0000","last_peered":"2024-02-23T00:43:29.933684+0000","last_clean":"2024-02-23T00:43:29.933684+0000","last_became_active":"2024-02-23T00:42:58.872796+0000","last_became_peered":"2024-02-23T00:42:58.872796+0000","last_unstale":"2024-02-23T00:43:29.933684+0000","last_undegraded":"2024-02-23T00:43:29.933684+0000","last_fullsized":"2024-02-23T00:43:29.933684+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"91'9","reported_seq":154,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.933621+0000","last_change":"2024-02-23T00:42:49.410329+0000","last_active":"2024-02-23T00:43:29.933621+0000","last_peered":"2024-02-23T00:43:29.933621+0000","last_clean":"2024-02-23T00:43:29.933621+0000","last_became_active":"2024-02-23T00:39:18.801297+0000","last_became_peered":"2024-02-23T00:39:18.801297+0000","last_unstale":"2024-02-23T00:43:29.933621+0000","last_undegraded":"2024-02-23T00:43:29.933621+0000","last_fullsized":"2024-02-23T00:43:29.933621+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'9","last_scrub_stamp":"2024-02-23T00:42:49.410259+0000","last_deep_scrub":"91'9","last_deep_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_clean_scrub_stamp":"2024-02-23T00:42:49.410259+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"68'16","reported_seq":125,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.902811+0000","last_change":"2024-02-23T00:42:34.633620+0000","last_active":"2024-02-23T00:43:29.902811+0000","last_peered":"2024-02-23T00:43:29.902811+0000","last_clean":"2024-02-23T00:43:29.902811+0000","last_became_active":"2024-02-23T00:39:16.787468+0000","last_became_peered":"2024-02-23T00:39:16.787468+0000","last_unstale":"2024-02-23T00:43:29.902811+0000","last_undegraded":"2024-02-23T00:43:29.902811+0000","last_fullsized":"2024-02-23T00:43:29.902811+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:42:34.633549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:34.633549+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":97,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.932579+0000","last_change":"2024-02-23T00:42:46.606249+0000","last_active":"2024-02-23T00:43:29.932579+0000","last_peered":"2024-02-23T00:43:29.932579+0000","last_clean":"2024-02-23T00:43:29.932579+0000","last_became_active":"2024-02-23T00:39:14.779952+0000","last_became_peered":"2024-02-23T00:39:14.779952+0000","last_unstale":"2024-02-23T00:43:29.932579+0000","last_undegraded":"2024-02-23T00:43:29.932579+0000","last_fullsized":"2024-02-23T00:43:29.932579+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:46.606185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:46.606185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"107'2","reported_seq":94,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.905568+0000","last_change":"2024-02-23T00:42:48.362807+0000","last_active":"2024-02-23T00:43:29.905568+0000","last_peered":"2024-02-23T00:43:29.905568+0000","last_clean":"2024-02-23T00:43:29.905568+0000","last_became_active":"2024-02-23T00:39:20.800858+0000","last_became_peered":"2024-02-23T00:39:20.800858+0000","last_unstale":"2024-02-23T00:43:29.905568+0000","last_undegraded":"2024-02-23T00:43:29.905568+0000","last_fullsized":"2024-02-23T00:43:29.905568+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'2","last_scrub_stamp":"2024-02-23T00:42:48.362739+0000","last_deep_scrub":"107'2","last_deep_scrub_stamp":"2024-02-23T00:42:48.362739+0000","last_clean_scrub_stamp":"2024-02-23T00:42:48.362739+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.906865+0000","last_change":"2024-02-23T00:42:58.727288+0000","last_active":"2024-02-23T00:43:29.906865+0000","last_peered":"2024-02-23T00:43:29.906865+0000","last_clean":"2024-02-23T00:43:29.906865+0000","last_became_active":"2024-02-23T00:42:58.726861+0000","last_became_peered":"2024-02-23T00:42:58.726861+0000","last_unstale":"2024-02-23T00:43:29.906865+0000","last_undegraded":"2024-02-23T00:43:29.906865+0000","last_fullsized":"2024-02-23T00:43:29.906865+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.901600+0000","last_change":"2024-02-23T00:42:58.664797+0000","last_active":"2024-02-23T00:43:29.901600+0000","last_peered":"2024-02-23T00:43:29.901600+0000","last_clean":"2024-02-23T00:43:29.901600+0000","last_became_active":"2024-02-23T00:42:58.664282+0000","last_became_peered":"2024-02-23T00:42:58.664282+0000","last_unstale":"2024-02-23T00:43:29.901600+0000","last_undegraded":"2024-02-23T00:43:29.901600+0000","last_fullsized":"2024-02-23T00:43:29.901600+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.12","version":"112'1","reported_seq":43,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.909223+0000","last_change":"2024-02-23T00:42:58.724372+0000","last_active":"2024-02-23T00:43:29.909223+0000","last_peered":"2024-02-23T00:43:29.909223+0000","last_clean":"2024-02-23T00:43:29.909223+0000","last_became_active":"2024-02-23T00:42:58.723546+0000","last_became_peered":"2024-02-23T00:42:58.723546+0000","last_unstale":"2024-02-23T00:43:29.909223+0000","last_undegraded":"2024-02-23T00:43:29.909223+0000","last_fullsized":"2024-02-23T00:43:29.909223+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25165836,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":24577,"num_write":1,"num_write_kb":24577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":93,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.928269+0000","last_change":"2024-02-23T00:41:57.376669+0000","last_active":"2024-02-23T00:43:29.928269+0000","last_peered":"2024-02-23T00:43:29.928269+0000","last_clean":"2024-02-23T00:43:29.928269+0000","last_became_active":"2024-02-23T00:39:18.796323+0000","last_became_peered":"2024-02-23T00:39:18.796323+0000","last_unstale":"2024-02-23T00:43:29.928269+0000","last_undegraded":"2024-02-23T00:43:29.928269+0000","last_fullsized":"2024-02-23T00:43:29.928269+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:57.376576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_clean_scrub_stamp":"2024-02-23T00:41:57.376576+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"68'25","reported_seq":144,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.897573+0000","last_change":"2024-02-23T00:41:08.004573+0000","last_active":"2024-02-23T00:43:29.897573+0000","last_peered":"2024-02-23T00:43:29.897573+0000","last_clean":"2024-02-23T00:43:29.897573+0000","last_became_active":"2024-02-23T00:39:16.783926+0000","last_became_peered":"2024-02-23T00:39:16.783926+0000","last_unstale":"2024-02-23T00:43:29.897573+0000","last_undegraded":"2024-02-23T00:43:29.897573+0000","last_fullsized":"2024-02-23T00:43:29.897573+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'25","last_scrub_stamp":"2024-02-23T00:41:08.004489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:08.004489+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"95'4","reported_seq":112,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.899058+0000","last_change":"2024-02-23T00:41:43.874460+0000","last_active":"2024-02-23T00:43:29.899058+0000","last_peered":"2024-02-23T00:43:29.899058+0000","last_clean":"2024-02-23T00:43:29.899058+0000","last_became_active":"2024-02-23T00:39:14.779205+0000","last_became_peered":"2024-02-23T00:39:14.779205+0000","last_unstale":"2024-02-23T00:43:29.899058+0000","last_undegraded":"2024-02-23T00:43:29.899058+0000","last_fullsized":"2024-02-23T00:43:29.899058+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'4","last_scrub_stamp":"2024-02-23T00:41:43.874397+0000","last_deep_scrub":"95'4","last_deep_scrub_stamp":"2024-02-23T00:41:43.874397+0000","last_clean_scrub_stamp":"2024-02-23T00:41:43.874397+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":110,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.906903+0000","last_change":"2024-02-23T00:43:27.612380+0000","last_active":"2024-02-23T00:43:29.906903+0000","last_peered":"2024-02-23T00:43:29.906903+0000","last_clean":"2024-02-23T00:43:29.906903+0000","last_became_active":"2024-02-23T00:43:27.611660+0000","last_became_peered":"2024-02-23T00:43:27.611660+0000","last_unstale":"2024-02-23T00:43:29.906903+0000","last_undegraded":"2024-02-23T00:43:29.906903+0000","last_fullsized":"2024-02-23T00:43:29.906903+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:48.031633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:48.031633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.935148+0000","last_change":"2024-02-23T00:42:58.810468+0000","last_active":"2024-02-23T00:43:29.935148+0000","last_peered":"2024-02-23T00:43:29.935148+0000","last_clean":"2024-02-23T00:43:29.935148+0000","last_became_active":"2024-02-23T00:42:58.809926+0000","last_became_peered":"2024-02-23T00:42:58.809926+0000","last_unstale":"2024-02-23T00:43:29.935148+0000","last_undegraded":"2024-02-23T00:43:29.935148+0000","last_fullsized":"2024-02-23T00:43:29.935148+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.904435+0000","last_change":"2024-02-23T00:42:58.755160+0000","last_active":"2024-02-23T00:43:29.904435+0000","last_peered":"2024-02-23T00:43:29.904435+0000","last_clean":"2024-02-23T00:43:29.904435+0000","last_became_active":"2024-02-23T00:42:58.744983+0000","last_became_peered":"2024-02-23T00:42:58.744983+0000","last_unstale":"2024-02-23T00:43:29.904435+0000","last_undegraded":"2024-02-23T00:43:29.904435+0000","last_fullsized":"2024-02-23T00:43:29.904435+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":38,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.923309+0000","last_change":"2024-02-23T00:42:58.741573+0000","last_active":"2024-02-23T00:43:29.923309+0000","last_peered":"2024-02-23T00:43:29.923309+0000","last_clean":"2024-02-23T00:43:29.923309+0000","last_became_active":"2024-02-23T00:42:58.741284+0000","last_became_peered":"2024-02-23T00:42:58.741284+0000","last_unstale":"2024-02-23T00:43:29.923309+0000","last_undegraded":"2024-02-23T00:43:29.923309+0000","last_fullsized":"2024-02-23T00:43:29.923309+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"68'21","reported_seq":137,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.931325+0000","last_change":"2024-02-23T00:41:48.151380+0000","last_active":"2024-02-23T00:43:29.931325+0000","last_peered":"2024-02-23T00:43:29.931325+0000","last_clean":"2024-02-23T00:43:29.931325+0000","last_became_active":"2024-02-23T00:39:16.858760+0000","last_became_peered":"2024-02-23T00:39:16.858760+0000","last_unstale":"2024-02-23T00:43:29.931325+0000","last_undegraded":"2024-02-23T00:43:29.931325+0000","last_fullsized":"2024-02-23T00:43:29.931325+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'21","last_scrub_stamp":"2024-02-23T00:41:48.151312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:48.151312+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":102,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.928192+0000","last_change":"2024-02-23T00:42:52.317469+0000","last_active":"2024-02-23T00:43:29.928192+0000","last_peered":"2024-02-23T00:43:29.928192+0000","last_clean":"2024-02-23T00:43:29.928192+0000","last_became_active":"2024-02-23T00:39:14.774244+0000","last_became_peered":"2024-02-23T00:39:14.774244+0000","last_unstale":"2024-02-23T00:43:29.928192+0000","last_undegraded":"2024-02-23T00:43:29.928192+0000","last_fullsized":"2024-02-23T00:43:29.928192+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:52.317403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:52.317403+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":94,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.943818+0000","last_change":"2024-02-23T00:41:46.998945+0000","last_active":"2024-02-23T00:43:29.943818+0000","last_peered":"2024-02-23T00:43:29.943818+0000","last_clean":"2024-02-23T00:43:29.943818+0000","last_became_active":"2024-02-23T00:39:18.798624+0000","last_became_peered":"2024-02-23T00:39:18.798624+0000","last_unstale":"2024-02-23T00:43:29.943818+0000","last_undegraded":"2024-02-23T00:43:29.943818+0000","last_fullsized":"2024-02-23T00:43:29.943818+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:46.998875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:46.998875+0000","last_clean_scrub_stamp":"2024-02-23T00:41:46.998875+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":91,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.927868+0000","last_change":"2024-02-23T00:42:35.226401+0000","last_active":"2024-02-23T00:43:29.927868+0000","last_peered":"2024-02-23T00:43:29.927868+0000","last_clean":"2024-02-23T00:43:29.927868+0000","last_became_active":"2024-02-23T00:39:20.796041+0000","last_became_peered":"2024-02-23T00:39:20.796041+0000","last_unstale":"2024-02-23T00:43:29.927868+0000","last_undegraded":"2024-02-23T00:43:29.927868+0000","last_fullsized":"2024-02-23T00:43:29.927868+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:35.226337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:35.226337+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1d","version":"110'7","reported_seq":46,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.901311+0000","last_change":"2024-02-23T00:42:58.755048+0000","last_active":"2024-02-23T00:43:29.901311+0000","last_peered":"2024-02-23T00:43:29.901311+0000","last_clean":"2024-02-23T00:43:29.901311+0000","last_became_active":"2024-02-23T00:42:58.754763+0000","last_became_peered":"2024-02-23T00:42:58.754763+0000","last_unstale":"2024-02-23T00:43:29.901311+0000","last_undegraded":"2024-02-23T00:43:29.901311+0000","last_fullsized":"2024-02-23T00:43:29.901311+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.905746+0000","last_change":"2024-02-23T00:42:58.829459+0000","last_active":"2024-02-23T00:43:29.905746+0000","last_peered":"2024-02-23T00:43:29.905746+0000","last_clean":"2024-02-23T00:43:29.905746+0000","last_became_active":"2024-02-23T00:42:58.825015+0000","last_became_peered":"2024-02-23T00:42:58.825015+0000","last_unstale":"2024-02-23T00:43:29.905746+0000","last_undegraded":"2024-02-23T00:43:29.905746+0000","last_fullsized":"2024-02-23T00:43:29.905746+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.6","version":"112'7","reported_seq":50,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.920071+0000","last_change":"2024-02-23T00:43:06.922950+0000","last_active":"2024-02-23T00:43:29.920071+0000","last_peered":"2024-02-23T00:43:29.920071+0000","last_clean":"2024-02-23T00:43:29.920071+0000","last_became_active":"2024-02-23T00:42:58.769223+0000","last_became_peered":"2024-02-23T00:42:58.769223+0000","last_unstale":"2024-02-23T00:43:29.920071+0000","last_undegraded":"2024-02-23T00:43:29.920071+0000","last_fullsized":"2024-02-23T00:43:29.920071+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"112'7","last_scrub_stamp":"2024-02-23T00:43:06.922785+0000","last_deep_scrub":"112'7","last_deep_scrub_stamp":"2024-02-23T00:43:06.922785+0000","last_clean_scrub_stamp":"2024-02-23T00:43:06.922785+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25165828,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":27325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2800000,"num_omap_keys":1000,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"107'16","reported_seq":124,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.943736+0000","last_change":"2024-02-23T00:42:52.964576+0000","last_active":"2024-02-23T00:43:29.943736+0000","last_peered":"2024-02-23T00:43:29.943736+0000","last_clean":"2024-02-23T00:43:29.943736+0000","last_became_active":"2024-02-23T00:39:16.858449+0000","last_became_peered":"2024-02-23T00:39:16.858449+0000","last_unstale":"2024-02-23T00:43:29.943736+0000","last_undegraded":"2024-02-23T00:43:29.943736+0000","last_fullsized":"2024-02-23T00:43:29.943736+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'16","last_scrub_stamp":"2024-02-23T00:42:52.964510+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-02-23T00:42:52.964510+0000","last_clean_scrub_stamp":"2024-02-23T00:42:52.964510+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":97,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.906420+0000","last_change":"2024-02-23T00:42:41.429191+0000","last_active":"2024-02-23T00:43:29.906420+0000","last_peered":"2024-02-23T00:43:29.906420+0000","last_clean":"2024-02-23T00:43:29.906420+0000","last_became_active":"2024-02-23T00:39:14.783164+0000","last_became_peered":"2024-02-23T00:39:14.783164+0000","last_unstale":"2024-02-23T00:43:29.906420+0000","last_undegraded":"2024-02-23T00:43:29.906420+0000","last_fullsized":"2024-02-23T00:43:29.906420+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:41.429079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:41.429079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":94,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.907122+0000","last_change":"2024-02-23T00:41:35.655313+0000","last_active":"2024-02-23T00:43:29.907122+0000","last_peered":"2024-02-23T00:43:29.907122+0000","last_clean":"2024-02-23T00:43:29.907122+0000","last_became_active":"2024-02-23T00:39:18.800297+0000","last_became_peered":"2024-02-23T00:39:18.800297+0000","last_unstale":"2024-02-23T00:43:29.907122+0000","last_undegraded":"2024-02-23T00:43:29.907122+0000","last_fullsized":"2024-02-23T00:43:29.907122+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:35.655209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:35.655209+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":90,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.904458+0000","last_change":"2024-02-23T00:41:45.928004+0000","last_active":"2024-02-23T00:43:29.904458+0000","last_peered":"2024-02-23T00:43:29.904458+0000","last_clean":"2024-02-23T00:43:29.904458+0000","last_became_active":"2024-02-23T00:39:21.162036+0000","last_became_peered":"2024-02-23T00:39:21.162036+0000","last_unstale":"2024-02-23T00:43:29.904458+0000","last_undegraded":"2024-02-23T00:43:29.904458+0000","last_fullsized":"2024-02-23T00:43:29.904458+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:45.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:45.927925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.933132+0000","last_change":"2024-02-23T00:42:57.649841+0000","last_active":"2024-02-23T00:43:29.933132+0000","last_peered":"2024-02-23T00:43:29.933132+0000","last_clean":"2024-02-23T00:43:29.933132+0000","last_became_active":"2024-02-23T00:42:57.648267+0000","last_became_peered":"2024-02-23T00:42:57.648267+0000","last_unstale":"2024-02-23T00:43:29.933132+0000","last_undegraded":"2024-02-23T00:43:29.933132+0000","last_fullsized":"2024-02-23T00:43:29.933132+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1","version":"110'7","reported_seq":52,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.949589+0000","last_change":"2024-02-23T00:42:58.782380+0000","last_active":"2024-02-23T00:43:29.949589+0000","last_peered":"2024-02-23T00:43:29.949589+0000","last_clean":"2024-02-23T00:43:29.949589+0000","last_became_active":"2024-02-23T00:42:58.777251+0000","last_became_peered":"2024-02-23T00:42:58.777251+0000","last_unstale":"2024-02-23T00:43:29.949589+0000","last_undegraded":"2024-02-23T00:43:29.949589+0000","last_fullsized":"2024-02-23T00:43:29.949589+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":43,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.904717+0000","last_change":"2024-02-23T00:43:08.979886+0000","last_active":"2024-02-23T00:43:29.904717+0000","last_peered":"2024-02-23T00:43:29.904717+0000","last_clean":"2024-02-23T00:43:29.904717+0000","last_became_active":"2024-02-23T00:42:58.713766+0000","last_became_peered":"2024-02-23T00:42:58.713766+0000","last_unstale":"2024-02-23T00:43:29.904717+0000","last_undegraded":"2024-02-23T00:43:29.904717+0000","last_fullsized":"2024-02-23T00:43:29.904717+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:08.979794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:08.979794+0000","last_clean_scrub_stamp":"2024-02-23T00:43:08.979794+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"68'22","reported_seq":65,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.927718+0000","last_change":"2024-02-23T00:42:44.344558+0000","last_active":"2024-02-23T00:43:29.927718+0000","last_peered":"2024-02-23T00:43:29.927718+0000","last_clean":"2024-02-23T00:43:29.927718+0000","last_became_active":"2024-02-23T00:42:43.521189+0000","last_became_peered":"2024-02-23T00:42:43.521189+0000","last_unstale":"2024-02-23T00:43:29.927718+0000","last_undegraded":"2024-02-23T00:43:29.927718+0000","last_fullsized":"2024-02-23T00:43:29.927718+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'22","last_scrub_stamp":"2024-02-23T00:42:44.344443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:44.344443+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":37,"num_read_kb":30,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,7],"acting":[2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":98,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.904402+0000","last_change":"2024-02-23T00:42:02.694103+0000","last_active":"2024-02-23T00:43:29.904402+0000","last_peered":"2024-02-23T00:43:29.904402+0000","last_clean":"2024-02-23T00:43:29.904402+0000","last_became_active":"2024-02-23T00:39:14.785590+0000","last_became_peered":"2024-02-23T00:39:14.785590+0000","last_unstale":"2024-02-23T00:43:29.904402+0000","last_undegraded":"2024-02-23T00:43:29.904402+0000","last_fullsized":"2024-02-23T00:43:29.904402+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:02.694022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_clean_scrub_stamp":"2024-02-23T00:42:02.694022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"67'8","reported_seq":153,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.905269+0000","last_change":"2024-02-23T00:42:40.872975+0000","last_active":"2024-02-23T00:43:29.905269+0000","last_peered":"2024-02-23T00:43:29.905269+0000","last_clean":"2024-02-23T00:43:29.905269+0000","last_became_active":"2024-02-23T00:39:18.803566+0000","last_became_peered":"2024-02-23T00:39:18.803566+0000","last_unstale":"2024-02-23T00:43:29.905269+0000","last_undegraded":"2024-02-23T00:43:29.905269+0000","last_fullsized":"2024-02-23T00:43:29.905269+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:42:40.872876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:40.872876+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.959248+0000","last_change":"2024-02-23T00:43:00.930610+0000","last_active":"2024-02-23T00:43:29.959248+0000","last_peered":"2024-02-23T00:43:29.959248+0000","last_clean":"2024-02-23T00:43:29.959248+0000","last_became_active":"2024-02-23T00:43:00.930360+0000","last_became_peered":"2024-02-23T00:43:00.930360+0000","last_unstale":"2024-02-23T00:43:29.959248+0000","last_undegraded":"2024-02-23T00:43:29.959248+0000","last_fullsized":"2024-02-23T00:43:29.959248+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:56.456815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:56.456815+0000","last_clean_scrub_stamp":"2024-02-23T00:42:56.456815+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1f","version":"110'5","reported_seq":44,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.960418+0000","last_change":"2024-02-23T00:42:58.770053+0000","last_active":"2024-02-23T00:43:29.960418+0000","last_peered":"2024-02-23T00:43:29.960418+0000","last_clean":"2024-02-23T00:43:29.960418+0000","last_became_active":"2024-02-23T00:42:58.758349+0000","last_became_peered":"2024-02-23T00:42:58.758349+0000","last_unstale":"2024-02-23T00:43:29.960418+0000","last_undegraded":"2024-02-23T00:43:29.960418+0000","last_fullsized":"2024-02-23T00:43:29.960418+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.911613+0000","last_change":"2024-02-23T00:42:58.766295+0000","last_active":"2024-02-23T00:43:29.911613+0000","last_peered":"2024-02-23T00:43:29.911613+0000","last_clean":"2024-02-23T00:43:29.911613+0000","last_became_active":"2024-02-23T00:42:58.757424+0000","last_became_peered":"2024-02-23T00:42:58.757424+0000","last_unstale":"2024-02-23T00:43:29.911613+0000","last_undegraded":"2024-02-23T00:43:29.911613+0000","last_fullsized":"2024-02-23T00:43:29.911613+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":43,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.932928+0000","last_change":"2024-02-23T00:43:02.835731+0000","last_active":"2024-02-23T00:43:29.932928+0000","last_peered":"2024-02-23T00:43:29.932928+0000","last_clean":"2024-02-23T00:43:29.932928+0000","last_became_active":"2024-02-23T00:42:58.736416+0000","last_became_peered":"2024-02-23T00:42:58.736416+0000","last_unstale":"2024-02-23T00:43:29.932928+0000","last_undegraded":"2024-02-23T00:43:29.932928+0000","last_fullsized":"2024-02-23T00:43:29.932928+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:02.835657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:02.835657+0000","last_clean_scrub_stamp":"2024-02-23T00:43:02.835657+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"68'15","reported_seq":127,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.905104+0000","last_change":"2024-02-23T00:42:34.829123+0000","last_active":"2024-02-23T00:43:29.905104+0000","last_peered":"2024-02-23T00:43:29.905104+0000","last_clean":"2024-02-23T00:43:29.905104+0000","last_became_active":"2024-02-23T00:39:16.858953+0000","last_became_peered":"2024-02-23T00:39:16.858953+0000","last_unstale":"2024-02-23T00:43:29.905104+0000","last_undegraded":"2024-02-23T00:43:29.905104+0000","last_fullsized":"2024-02-23T00:43:29.905104+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:42:34.828996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:34.828996+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":98,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.933398+0000","last_change":"2024-02-23T00:41:52.151201+0000","last_active":"2024-02-23T00:43:29.933398+0000","last_peered":"2024-02-23T00:43:29.933398+0000","last_clean":"2024-02-23T00:43:29.933398+0000","last_became_active":"2024-02-23T00:39:14.773772+0000","last_became_peered":"2024-02-23T00:39:14.773772+0000","last_unstale":"2024-02-23T00:43:29.933398+0000","last_undegraded":"2024-02-23T00:43:29.933398+0000","last_fullsized":"2024-02-23T00:43:29.933398+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:52.151119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:52.151119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"67'8","reported_seq":148,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.897947+0000","last_change":"2024-02-23T00:41:25.942914+0000","last_active":"2024-02-23T00:43:29.897947+0000","last_peered":"2024-02-23T00:43:29.897947+0000","last_clean":"2024-02-23T00:43:29.897947+0000","last_became_active":"2024-02-23T00:39:18.787313+0000","last_became_peered":"2024-02-23T00:39:18.787313+0000","last_unstale":"2024-02-23T00:43:29.897947+0000","last_undegraded":"2024-02-23T00:43:29.897947+0000","last_fullsized":"2024-02-23T00:43:29.897947+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:41:25.942822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:25.942822+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.904211+0000","last_change":"2024-02-23T00:43:00.935992+0000","last_active":"2024-02-23T00:43:29.904211+0000","last_peered":"2024-02-23T00:43:29.904211+0000","last_clean":"2024-02-23T00:43:29.904211+0000","last_became_active":"2024-02-23T00:43:00.935724+0000","last_became_peered":"2024-02-23T00:43:00.935724+0000","last_unstale":"2024-02-23T00:43:29.904211+0000","last_undegraded":"2024-02-23T00:43:29.904211+0000","last_fullsized":"2024-02-23T00:43:29.904211+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:48.118585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:54.938209+0000","last_clean_scrub_stamp":"2024-02-23T00:42:48.118585+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.956463+0000","last_change":"2024-02-23T00:42:58.746104+0000","last_active":"2024-02-23T00:43:29.956463+0000","last_peered":"2024-02-23T00:43:29.956463+0000","last_clean":"2024-02-23T00:43:29.956463+0000","last_became_active":"2024-02-23T00:42:58.745471+0000","last_became_peered":"2024-02-23T00:42:58.745471+0000","last_unstale":"2024-02-23T00:43:29.956463+0000","last_undegraded":"2024-02-23T00:43:29.956463+0000","last_fullsized":"2024-02-23T00:43:29.956463+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.922600+0000","last_change":"2024-02-23T00:42:57.652853+0000","last_active":"2024-02-23T00:43:29.922600+0000","last_peered":"2024-02-23T00:43:29.922600+0000","last_clean":"2024-02-23T00:43:29.922600+0000","last_became_active":"2024-02-23T00:42:57.652488+0000","last_became_peered":"2024-02-23T00:42:57.652488+0000","last_unstale":"2024-02-23T00:43:29.922600+0000","last_undegraded":"2024-02-23T00:43:29.922600+0000","last_fullsized":"2024-02-23T00:43:29.922600+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.5","version":"116'2","reported_seq":48,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.941547+0000","last_change":"2024-02-23T00:43:05.860442+0000","last_active":"2024-02-23T00:43:29.941547+0000","last_peered":"2024-02-23T00:43:29.941547+0000","last_clean":"2024-02-23T00:43:29.941547+0000","last_became_active":"2024-02-23T00:42:58.728978+0000","last_became_peered":"2024-02-23T00:42:58.728978+0000","last_unstale":"2024-02-23T00:43:29.941547+0000","last_undegraded":"2024-02-23T00:43:29.941547+0000","last_fullsized":"2024-02-23T00:43:29.941547+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"116'2","last_scrub_stamp":"2024-02-23T00:43:05.860338+0000","last_deep_scrub":"116'2","last_deep_scrub_stamp":"2024-02-23T00:43:05.860338+0000","last_clean_scrub_stamp":"2024-02-23T00:43:05.860338+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"67'9","reported_seq":111,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.902501+0000","last_change":"2024-02-23T00:42:57.705378+0000","last_active":"2024-02-23T00:43:29.902501+0000","last_peered":"2024-02-23T00:43:29.902501+0000","last_clean":"2024-02-23T00:43:29.902501+0000","last_became_active":"2024-02-23T00:39:16.786936+0000","last_became_peered":"2024-02-23T00:39:16.786936+0000","last_unstale":"2024-02-23T00:43:29.902501+0000","last_undegraded":"2024-02-23T00:43:29.902501+0000","last_fullsized":"2024-02-23T00:43:29.902501+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'9","last_scrub_stamp":"2024-02-23T00:42:57.705261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.705261+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":102,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.901490+0000","last_change":"2024-02-23T00:42:42.461098+0000","last_active":"2024-02-23T00:43:29.901490+0000","last_peered":"2024-02-23T00:43:29.901490+0000","last_clean":"2024-02-23T00:43:29.901490+0000","last_became_active":"2024-02-23T00:39:14.778455+0000","last_became_peered":"2024-02-23T00:39:14.778455+0000","last_unstale":"2024-02-23T00:43:29.901490+0000","last_undegraded":"2024-02-23T00:43:29.901490+0000","last_fullsized":"2024-02-23T00:43:29.901490+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:42.461020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:42.461020+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":94,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.906773+0000","last_change":"2024-02-23T00:42:40.896368+0000","last_active":"2024-02-23T00:43:29.906773+0000","last_peered":"2024-02-23T00:43:29.906773+0000","last_clean":"2024-02-23T00:43:29.906773+0000","last_became_active":"2024-02-23T00:39:18.799052+0000","last_became_peered":"2024-02-23T00:39:18.799052+0000","last_unstale":"2024-02-23T00:43:29.906773+0000","last_undegraded":"2024-02-23T00:43:29.906773+0000","last_fullsized":"2024-02-23T00:43:29.906773+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:40.896249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_clean_scrub_stamp":"2024-02-23T00:42:40.896249+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":92,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.905485+0000","last_change":"2024-02-23T00:42:50.425748+0000","last_active":"2024-02-23T00:43:29.905485+0000","last_peered":"2024-02-23T00:43:29.905485+0000","last_clean":"2024-02-23T00:43:29.905485+0000","last_became_active":"2024-02-23T00:39:20.802648+0000","last_became_peered":"2024-02-23T00:39:20.802648+0000","last_unstale":"2024-02-23T00:43:29.905485+0000","last_undegraded":"2024-02-23T00:43:29.905485+0000","last_fullsized":"2024-02-23T00:43:29.905485+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:50.425641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:50.425641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.901839+0000","last_change":"2024-02-23T00:42:58.834611+0000","last_active":"2024-02-23T00:43:29.901839+0000","last_peered":"2024-02-23T00:43:29.901839+0000","last_clean":"2024-02-23T00:43:29.901839+0000","last_became_active":"2024-02-23T00:42:58.833531+0000","last_became_peered":"2024-02-23T00:42:58.833531+0000","last_unstale":"2024-02-23T00:43:29.901839+0000","last_undegraded":"2024-02-23T00:43:29.901839+0000","last_fullsized":"2024-02-23T00:43:29.901839+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.933545+0000","last_change":"2024-02-23T00:42:58.841292+0000","last_active":"2024-02-23T00:43:29.933545+0000","last_peered":"2024-02-23T00:43:29.933545+0000","last_clean":"2024-02-23T00:43:29.933545+0000","last_became_active":"2024-02-23T00:42:58.835862+0000","last_became_peered":"2024-02-23T00:42:58.835862+0000","last_unstale":"2024-02-23T00:43:29.933545+0000","last_undegraded":"2024-02-23T00:43:29.933545+0000","last_fullsized":"2024-02-23T00:43:29.933545+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.2","version":"110'6","reported_seq":50,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.911123+0000","last_change":"2024-02-23T00:43:06.013905+0000","last_active":"2024-02-23T00:43:29.911123+0000","last_peered":"2024-02-23T00:43:29.911123+0000","last_clean":"2024-02-23T00:43:29.911123+0000","last_became_active":"2024-02-23T00:42:58.718984+0000","last_became_peered":"2024-02-23T00:42:58.718984+0000","last_unstale":"2024-02-23T00:43:29.911123+0000","last_undegraded":"2024-02-23T00:43:29.911123+0000","last_fullsized":"2024-02-23T00:43:29.911123+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'6","last_scrub_stamp":"2024-02-23T00:43:06.013828+0000","last_deep_scrub":"110'6","last_deep_scrub_stamp":"2024-02-23T00:43:06.013828+0000","last_clean_scrub_stamp":"2024-02-23T00:43:06.013828+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":64,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"67'11","reported_seq":114,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.895636+0000","last_change":"2024-02-23T00:42:58.013765+0000","last_active":"2024-02-23T00:43:29.895636+0000","last_peered":"2024-02-23T00:43:29.895636+0000","last_clean":"2024-02-23T00:43:29.895636+0000","last_became_active":"2024-02-23T00:39:16.783896+0000","last_became_peered":"2024-02-23T00:39:16.783896+0000","last_unstale":"2024-02-23T00:43:29.895636+0000","last_undegraded":"2024-02-23T00:43:29.895636+0000","last_fullsized":"2024-02-23T00:43:29.895636+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'11","last_scrub_stamp":"2024-02-23T00:42:58.013688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:58.013688+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"60'1","reported_seq":104,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.936076+0000","last_change":"2024-02-23T00:41:57.687046+0000","last_active":"2024-02-23T00:43:29.936076+0000","last_peered":"2024-02-23T00:43:29.936076+0000","last_clean":"2024-02-23T00:43:29.936076+0000","last_became_active":"2024-02-23T00:39:14.775804+0000","last_became_peered":"2024-02-23T00:39:14.775804+0000","last_unstale":"2024-02-23T00:43:29.936076+0000","last_undegraded":"2024-02-23T00:43:29.936076+0000","last_fullsized":"2024-02-23T00:43:29.936076+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'1","last_scrub_stamp":"2024-02-23T00:41:57.686958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:57.686958+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":93,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-02-23T00:43:31.903157+0000","last_change":"2024-02-23T00:43:31.903157+0000","last_active":"2024-02-23T00:43:31.903157+0000","last_peered":"2024-02-23T00:43:31.903157+0000","last_clean":"2024-02-23T00:43:31.903157+0000","last_became_active":"2024-02-23T00:39:18.800382+0000","last_became_peered":"2024-02-23T00:39:18.800382+0000","last_unstale":"2024-02-23T00:43:31.903157+0000","last_undegraded":"2024-02-23T00:43:31.903157+0000","last_fullsized":"2024-02-23T00:43:31.903157+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:31.903031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:31.903031+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":92,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.899051+0000","last_change":"2024-02-23T00:42:08.062268+0000","last_active":"2024-02-23T00:43:29.899051+0000","last_peered":"2024-02-23T00:43:29.899051+0000","last_clean":"2024-02-23T00:43:29.899051+0000","last_became_active":"2024-02-23T00:39:20.799053+0000","last_became_peered":"2024-02-23T00:39:20.799053+0000","last_unstale":"2024-02-23T00:43:29.899051+0000","last_undegraded":"2024-02-23T00:43:29.899051+0000","last_fullsized":"2024-02-23T00:43:29.899051+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:08.062156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:08.062156+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"110'2","reported_seq":41,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.921282+0000","last_change":"2024-02-23T00:42:58.709534+0000","last_active":"2024-02-23T00:43:29.921282+0000","last_peered":"2024-02-23T00:43:29.921282+0000","last_clean":"2024-02-23T00:43:29.921282+0000","last_became_active":"2024-02-23T00:42:58.698725+0000","last_became_peered":"2024-02-23T00:42:58.698725+0000","last_unstale":"2024-02-23T00:43:29.921282+0000","last_undegraded":"2024-02-23T00:43:29.921282+0000","last_fullsized":"2024-02-23T00:43:29.921282+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.935959+0000","last_change":"2024-02-23T00:42:57.652742+0000","last_active":"2024-02-23T00:43:29.935959+0000","last_peered":"2024-02-23T00:43:29.935959+0000","last_clean":"2024-02-23T00:43:29.935959+0000","last_became_active":"2024-02-23T00:42:57.652438+0000","last_became_peered":"2024-02-23T00:42:57.652438+0000","last_unstale":"2024-02-23T00:43:29.935959+0000","last_undegraded":"2024-02-23T00:43:29.935959+0000","last_fullsized":"2024-02-23T00:43:29.935959+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.3","version":"110'1","reported_seq":45,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.933848+0000","last_change":"2024-02-23T00:43:04.401584+0000","last_active":"2024-02-23T00:43:29.933848+0000","last_peered":"2024-02-23T00:43:29.933848+0000","last_clean":"2024-02-23T00:43:29.933848+0000","last_became_active":"2024-02-23T00:42:58.823366+0000","last_became_peered":"2024-02-23T00:42:58.823366+0000","last_unstale":"2024-02-23T00:43:29.933848+0000","last_undegraded":"2024-02-23T00:43:29.933848+0000","last_fullsized":"2024-02-23T00:43:29.933848+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'1","last_scrub_stamp":"2024-02-23T00:43:04.401467+0000","last_deep_scrub":"110'1","last_deep_scrub_stamp":"2024-02-23T00:43:04.401467+0000","last_clean_scrub_stamp":"2024-02-23T00:43:04.401467+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1024,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"67'4","reported_seq":102,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.921104+0000","last_change":"2024-02-23T00:42:42.934024+0000","last_active":"2024-02-23T00:43:29.921104+0000","last_peered":"2024-02-23T00:43:29.921104+0000","last_clean":"2024-02-23T00:43:29.921104+0000","last_became_active":"2024-02-23T00:39:16.785830+0000","last_became_peered":"2024-02-23T00:39:16.785830+0000","last_unstale":"2024-02-23T00:43:29.921104+0000","last_undegraded":"2024-02-23T00:43:29.921104+0000","last_fullsized":"2024-02-23T00:43:29.921104+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'4","last_scrub_stamp":"2024-02-23T00:42:42.933958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:42.933958+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":98,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.907000+0000","last_change":"2024-02-23T00:41:41.650513+0000","last_active":"2024-02-23T00:43:29.907000+0000","last_peered":"2024-02-23T00:43:29.907000+0000","last_clean":"2024-02-23T00:43:29.907000+0000","last_became_active":"2024-02-23T00:39:14.783325+0000","last_became_peered":"2024-02-23T00:39:14.783325+0000","last_unstale":"2024-02-23T00:43:29.907000+0000","last_undegraded":"2024-02-23T00:43:29.907000+0000","last_fullsized":"2024-02-23T00:43:29.907000+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:41.650434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:41.650434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":98,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.906242+0000","last_change":"2024-02-23T00:42:57.574460+0000","last_active":"2024-02-23T00:43:29.906242+0000","last_peered":"2024-02-23T00:43:29.906242+0000","last_clean":"2024-02-23T00:43:29.906242+0000","last_became_active":"2024-02-23T00:39:18.797287+0000","last_became_peered":"2024-02-23T00:39:18.797287+0000","last_unstale":"2024-02-23T00:43:29.906242+0000","last_undegraded":"2024-02-23T00:43:29.906242+0000","last_fullsized":"2024-02-23T00:43:29.906242+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.574369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.574369+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":90,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.911208+0000","last_change":"2024-02-23T00:42:44.983880+0000","last_active":"2024-02-23T00:43:29.911208+0000","last_peered":"2024-02-23T00:43:29.911208+0000","last_clean":"2024-02-23T00:43:29.911208+0000","last_became_active":"2024-02-23T00:39:20.795351+0000","last_became_peered":"2024-02-23T00:39:20.795351+0000","last_unstale":"2024-02-23T00:43:29.911208+0000","last_undegraded":"2024-02-23T00:43:29.911208+0000","last_fullsized":"2024-02-23T00:43:29.911208+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:44.983813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:44.983813+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1b","version":"110'10","reported_seq":49,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.915958+0000","last_change":"2024-02-23T00:42:58.749680+0000","last_active":"2024-02-23T00:43:29.915958+0000","last_peered":"2024-02-23T00:43:29.915958+0000","last_clean":"2024-02-23T00:43:29.915958+0000","last_became_active":"2024-02-23T00:42:58.749126+0000","last_became_peered":"2024-02-23T00:42:58.749126+0000","last_unstale":"2024-02-23T00:43:29.915958+0000","last_undegraded":"2024-02-23T00:43:29.915958+0000","last_fullsized":"2024-02-23T00:43:29.915958+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.907871+0000","last_change":"2024-02-23T00:42:58.741674+0000","last_active":"2024-02-23T00:43:29.907871+0000","last_peered":"2024-02-23T00:43:29.907871+0000","last_clean":"2024-02-23T00:43:29.907871+0000","last_became_active":"2024-02-23T00:42:58.741365+0000","last_became_peered":"2024-02-23T00:42:58.741365+0000","last_unstale":"2024-02-23T00:43:29.907871+0000","last_undegraded":"2024-02-23T00:43:29.907871+0000","last_fullsized":"2024-02-23T00:43:29.907871+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":43,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.921201+0000","last_change":"2024-02-23T00:43:03.117508+0000","last_active":"2024-02-23T00:43:29.921201+0000","last_peered":"2024-02-23T00:43:29.921201+0000","last_clean":"2024-02-23T00:43:29.921201+0000","last_became_active":"2024-02-23T00:42:58.804980+0000","last_became_peered":"2024-02-23T00:42:58.804980+0000","last_unstale":"2024-02-23T00:43:29.921201+0000","last_undegraded":"2024-02-23T00:43:29.921201+0000","last_fullsized":"2024-02-23T00:43:29.921201+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:03.117444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:03.117444+0000","last_clean_scrub_stamp":"2024-02-23T00:43:03.117444+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"68'17","reported_seq":130,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.950532+0000","last_change":"2024-02-23T00:41:45.979896+0000","last_active":"2024-02-23T00:43:29.950532+0000","last_peered":"2024-02-23T00:43:29.950532+0000","last_clean":"2024-02-23T00:43:29.950532+0000","last_became_active":"2024-02-23T00:39:16.857232+0000","last_became_peered":"2024-02-23T00:39:16.857232+0000","last_unstale":"2024-02-23T00:43:29.950532+0000","last_undegraded":"2024-02-23T00:43:29.950532+0000","last_fullsized":"2024-02-23T00:43:29.950532+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:41:45.979819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:45.979819+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":97,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.909462+0000","last_change":"2024-02-23T00:41:47.034138+0000","last_active":"2024-02-23T00:43:29.909462+0000","last_peered":"2024-02-23T00:43:29.909462+0000","last_clean":"2024-02-23T00:43:29.909462+0000","last_became_active":"2024-02-23T00:39:14.784092+0000","last_became_peered":"2024-02-23T00:39:14.784092+0000","last_unstale":"2024-02-23T00:43:29.909462+0000","last_undegraded":"2024-02-23T00:43:29.909462+0000","last_fullsized":"2024-02-23T00:43:29.909462+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:47.034064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:47.034064+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":92,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.914520+0000","last_change":"2024-02-23T00:42:02.965536+0000","last_active":"2024-02-23T00:43:29.914520+0000","last_peered":"2024-02-23T00:43:29.914520+0000","last_clean":"2024-02-23T00:43:29.914520+0000","last_became_active":"2024-02-23T00:39:18.791457+0000","last_became_peered":"2024-02-23T00:39:18.791457+0000","last_unstale":"2024-02-23T00:43:29.914520+0000","last_undegraded":"2024-02-23T00:43:29.914520+0000","last_fullsized":"2024-02-23T00:43:29.914520+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:02.965455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:02.965455+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":87,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.907020+0000","last_change":"2024-02-23T00:41:40.941453+0000","last_active":"2024-02-23T00:43:29.907020+0000","last_peered":"2024-02-23T00:43:29.907020+0000","last_clean":"2024-02-23T00:43:29.907020+0000","last_became_active":"2024-02-23T00:39:21.160376+0000","last_became_peered":"2024-02-23T00:39:21.160376+0000","last_unstale":"2024-02-23T00:43:29.907020+0000","last_undegraded":"2024-02-23T00:43:29.907020+0000","last_fullsized":"2024-02-23T00:43:29.907020+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:40.941380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:40.941380+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1a","version":"110'5","reported_seq":44,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.897901+0000","last_change":"2024-02-23T00:42:58.738588+0000","last_active":"2024-02-23T00:43:29.897901+0000","last_peered":"2024-02-23T00:43:29.897901+0000","last_clean":"2024-02-23T00:43:29.897901+0000","last_became_active":"2024-02-23T00:42:58.736706+0000","last_became_peered":"2024-02-23T00:42:58.736706+0000","last_unstale":"2024-02-23T00:43:29.897901+0000","last_undegraded":"2024-02-23T00:43:29.897901+0000","last_fullsized":"2024-02-23T00:43:29.897901+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.955826+0000","last_change":"2024-02-23T00:42:58.705557+0000","last_active":"2024-02-23T00:43:29.955826+0000","last_peered":"2024-02-23T00:43:29.955826+0000","last_clean":"2024-02-23T00:43:29.955826+0000","last_became_active":"2024-02-23T00:42:58.702580+0000","last_became_peered":"2024-02-23T00:42:58.702580+0000","last_unstale":"2024-02-23T00:43:29.955826+0000","last_undegraded":"2024-02-23T00:43:29.955826+0000","last_fullsized":"2024-02-23T00:43:29.955826+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"110'1","reported_seq":44,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.906684+0000","last_change":"2024-02-23T00:43:02.449579+0000","last_active":"2024-02-23T00:43:29.906684+0000","last_peered":"2024-02-23T00:43:29.906684+0000","last_clean":"2024-02-23T00:43:29.906684+0000","last_became_active":"2024-02-23T00:42:58.804939+0000","last_became_peered":"2024-02-23T00:42:58.804939+0000","last_unstale":"2024-02-23T00:43:29.906684+0000","last_undegraded":"2024-02-23T00:43:29.906684+0000","last_fullsized":"2024-02-23T00:43:29.906684+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'1","last_scrub_stamp":"2024-02-23T00:43:02.449516+0000","last_deep_scrub":"110'1","last_deep_scrub_stamp":"2024-02-23T00:43:02.449516+0000","last_clean_scrub_stamp":"2024-02-23T00:43:02.449516+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,4],"acting":[6,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"68'16","reported_seq":126,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.931675+0000","last_change":"2024-02-23T00:42:53.444587+0000","last_active":"2024-02-23T00:43:29.931675+0000","last_peered":"2024-02-23T00:43:29.931675+0000","last_clean":"2024-02-23T00:43:29.931675+0000","last_became_active":"2024-02-23T00:39:16.858945+0000","last_became_peered":"2024-02-23T00:39:16.858945+0000","last_unstale":"2024-02-23T00:43:29.931675+0000","last_undegraded":"2024-02-23T00:43:29.931675+0000","last_fullsized":"2024-02-23T00:43:29.931675+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:42:53.444512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.444512+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":105,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.897265+0000","last_change":"2024-02-23T00:41:32.702659+0000","last_active":"2024-02-23T00:43:29.897265+0000","last_peered":"2024-02-23T00:43:29.897265+0000","last_clean":"2024-02-23T00:43:29.897265+0000","last_became_active":"2024-02-23T00:39:14.785363+0000","last_became_peered":"2024-02-23T00:39:14.785363+0000","last_unstale":"2024-02-23T00:43:29.897265+0000","last_undegraded":"2024-02-23T00:43:29.897265+0000","last_fullsized":"2024-02-23T00:43:29.897265+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"80'2","last_scrub_stamp":"2024-02-23T00:41:32.702574+0000","last_deep_scrub":"80'2","last_deep_scrub_stamp":"2024-02-23T00:40:15.942725+0000","last_clean_scrub_stamp":"2024-02-23T00:41:32.702574+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"93'9","reported_seq":156,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.909749+0000","last_change":"2024-02-23T00:41:38.881071+0000","last_active":"2024-02-23T00:43:29.909749+0000","last_peered":"2024-02-23T00:43:29.909749+0000","last_clean":"2024-02-23T00:43:29.909749+0000","last_became_active":"2024-02-23T00:39:18.796680+0000","last_became_peered":"2024-02-23T00:39:18.796680+0000","last_unstale":"2024-02-23T00:43:29.909749+0000","last_undegraded":"2024-02-23T00:43:29.909749+0000","last_fullsized":"2024-02-23T00:43:29.909749+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-02-23T00:41:38.880989+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-02-23T00:41:38.880989+0000","last_clean_scrub_stamp":"2024-02-23T00:41:38.880989+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.908768+0000","last_change":"2024-02-23T00:42:58.805923+0000","last_active":"2024-02-23T00:43:29.908768+0000","last_peered":"2024-02-23T00:43:29.908768+0000","last_clean":"2024-02-23T00:43:29.908768+0000","last_became_active":"2024-02-23T00:42:58.803900+0000","last_became_peered":"2024-02-23T00:42:58.803900+0000","last_unstale":"2024-02-23T00:43:29.908768+0000","last_undegraded":"2024-02-23T00:43:29.908768+0000","last_fullsized":"2024-02-23T00:43:29.908768+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.906014+0000","last_change":"2024-02-23T00:42:58.850530+0000","last_active":"2024-02-23T00:43:29.906014+0000","last_peered":"2024-02-23T00:43:29.906014+0000","last_clean":"2024-02-23T00:43:29.906014+0000","last_became_active":"2024-02-23T00:42:58.850072+0000","last_became_peered":"2024-02-23T00:42:58.850072+0000","last_unstale":"2024-02-23T00:43:29.906014+0000","last_undegraded":"2024-02-23T00:43:29.906014+0000","last_fullsized":"2024-02-23T00:43:29.906014+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":38,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.959523+0000","last_change":"2024-02-23T00:42:58.743439+0000","last_active":"2024-02-23T00:43:29.959523+0000","last_peered":"2024-02-23T00:43:29.959523+0000","last_clean":"2024-02-23T00:43:29.959523+0000","last_became_active":"2024-02-23T00:42:58.734965+0000","last_became_peered":"2024-02-23T00:42:58.734965+0000","last_unstale":"2024-02-23T00:43:29.959523+0000","last_undegraded":"2024-02-23T00:43:29.959523+0000","last_fullsized":"2024-02-23T00:43:29.959523+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"93'23","reported_seq":140,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.909897+0000","last_change":"2024-02-23T00:42:48.524850+0000","last_active":"2024-02-23T00:43:29.909897+0000","last_peered":"2024-02-23T00:43:29.909897+0000","last_clean":"2024-02-23T00:43:29.909897+0000","last_became_active":"2024-02-23T00:39:16.782660+0000","last_became_peered":"2024-02-23T00:39:16.782660+0000","last_unstale":"2024-02-23T00:43:29.909897+0000","last_undegraded":"2024-02-23T00:43:29.909897+0000","last_fullsized":"2024-02-23T00:43:29.909897+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'23","last_scrub_stamp":"2024-02-23T00:42:48.524772+0000","last_deep_scrub":"93'23","last_deep_scrub_stamp":"2024-02-23T00:41:35.954516+0000","last_clean_scrub_stamp":"2024-02-23T00:42:48.524772+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":98,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.899196+0000","last_change":"2024-02-23T00:41:57.053372+0000","last_active":"2024-02-23T00:43:29.899196+0000","last_peered":"2024-02-23T00:43:29.899196+0000","last_clean":"2024-02-23T00:43:29.899196+0000","last_became_active":"2024-02-23T00:39:14.781560+0000","last_became_peered":"2024-02-23T00:39:14.781560+0000","last_unstale":"2024-02-23T00:43:29.899196+0000","last_undegraded":"2024-02-23T00:43:29.899196+0000","last_fullsized":"2024-02-23T00:43:29.899196+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:57.053295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:57.053295+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":94,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.931751+0000","last_change":"2024-02-23T00:41:53.189947+0000","last_active":"2024-02-23T00:43:29.931751+0000","last_peered":"2024-02-23T00:43:29.931751+0000","last_clean":"2024-02-23T00:43:29.931751+0000","last_became_active":"2024-02-23T00:39:18.790602+0000","last_became_peered":"2024-02-23T00:39:18.790602+0000","last_unstale":"2024-02-23T00:43:29.931751+0000","last_undegraded":"2024-02-23T00:43:29.931751+0000","last_fullsized":"2024-02-23T00:43:29.931751+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:53.189881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:53.189881+0000","last_clean_scrub_stamp":"2024-02-23T00:41:53.189881+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.14","version":"110'2","reported_seq":41,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.901607+0000","last_change":"2024-02-23T00:42:58.804637+0000","last_active":"2024-02-23T00:43:29.901607+0000","last_peered":"2024-02-23T00:43:29.901607+0000","last_clean":"2024-02-23T00:43:29.901607+0000","last_became_active":"2024-02-23T00:42:58.804344+0000","last_became_peered":"2024-02-23T00:42:58.804344+0000","last_unstale":"2024-02-23T00:43:29.901607+0000","last_undegraded":"2024-02-23T00:43:29.901607+0000","last_fullsized":"2024-02-23T00:43:29.901607+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.899267+0000","last_change":"2024-02-23T00:42:58.778321+0000","last_active":"2024-02-23T00:43:29.899267+0000","last_peered":"2024-02-23T00:43:29.899267+0000","last_clean":"2024-02-23T00:43:29.899267+0000","last_became_active":"2024-02-23T00:42:58.774720+0000","last_became_peered":"2024-02-23T00:42:58.774720+0000","last_unstale":"2024-02-23T00:43:29.899267+0000","last_undegraded":"2024-02-23T00:43:29.899267+0000","last_fullsized":"2024-02-23T00:43:29.899267+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":38,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.904548+0000","last_change":"2024-02-23T00:42:58.766655+0000","last_active":"2024-02-23T00:43:29.904548+0000","last_peered":"2024-02-23T00:43:29.904548+0000","last_clean":"2024-02-23T00:43:29.904548+0000","last_became_active":"2024-02-23T00:42:58.764146+0000","last_became_peered":"2024-02-23T00:42:58.764146+0000","last_unstale":"2024-02-23T00:43:29.904548+0000","last_undegraded":"2024-02-23T00:43:29.904548+0000","last_fullsized":"2024-02-23T00:43:29.904548+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"67'15","reported_seq":120,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.927985+0000","last_change":"2024-02-23T00:41:41.121005+0000","last_active":"2024-02-23T00:43:29.927985+0000","last_peered":"2024-02-23T00:43:29.927985+0000","last_clean":"2024-02-23T00:43:29.927985+0000","last_became_active":"2024-02-23T00:39:16.787138+0000","last_became_peered":"2024-02-23T00:39:16.787138+0000","last_unstale":"2024-02-23T00:43:29.927985+0000","last_undegraded":"2024-02-23T00:43:29.927985+0000","last_fullsized":"2024-02-23T00:43:29.927985+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:41:41.120939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:41.120939+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":97,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.909006+0000","last_change":"2024-02-23T00:41:57.138591+0000","last_active":"2024-02-23T00:43:29.909006+0000","last_peered":"2024-02-23T00:43:29.909006+0000","last_clean":"2024-02-23T00:43:29.909006+0000","last_became_active":"2024-02-23T00:39:14.782494+0000","last_became_peered":"2024-02-23T00:39:14.782494+0000","last_unstale":"2024-02-23T00:43:29.909006+0000","last_undegraded":"2024-02-23T00:43:29.909006+0000","last_fullsized":"2024-02-23T00:43:29.909006+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:57.138515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:57.138515+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":93,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.941410+0000","last_change":"2024-02-23T00:41:52.641797+0000","last_active":"2024-02-23T00:43:29.941410+0000","last_peered":"2024-02-23T00:43:29.941410+0000","last_clean":"2024-02-23T00:43:29.941410+0000","last_became_active":"2024-02-23T00:39:18.801039+0000","last_became_peered":"2024-02-23T00:39:18.801039+0000","last_unstale":"2024-02-23T00:43:29.941410+0000","last_undegraded":"2024-02-23T00:43:29.941410+0000","last_fullsized":"2024-02-23T00:43:29.941410+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:52.641689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.641689+0000","last_clean_scrub_stamp":"2024-02-23T00:41:52.641689+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"110'5","reported_seq":44,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.898847+0000","last_change":"2024-02-23T00:42:58.783591+0000","last_active":"2024-02-23T00:43:29.898847+0000","last_peered":"2024-02-23T00:43:29.898847+0000","last_clean":"2024-02-23T00:43:29.898847+0000","last_became_active":"2024-02-23T00:42:58.783343+0000","last_became_peered":"2024-02-23T00:42:58.783343+0000","last_unstale":"2024-02-23T00:43:29.898847+0000","last_undegraded":"2024-02-23T00:43:29.898847+0000","last_fullsized":"2024-02-23T00:43:29.898847+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.961562+0000","last_change":"2024-02-23T00:42:58.773110+0000","last_active":"2024-02-23T00:43:29.961562+0000","last_peered":"2024-02-23T00:43:29.961562+0000","last_clean":"2024-02-23T00:43:29.961562+0000","last_became_active":"2024-02-23T00:42:58.770759+0000","last_became_peered":"2024-02-23T00:42:58.770759+0000","last_unstale":"2024-02-23T00:43:29.961562+0000","last_undegraded":"2024-02-23T00:43:29.961562+0000","last_fullsized":"2024-02-23T00:43:29.961562+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":38,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.897681+0000","last_change":"2024-02-23T00:42:58.676904+0000","last_active":"2024-02-23T00:43:29.897681+0000","last_peered":"2024-02-23T00:43:29.897681+0000","last_clean":"2024-02-23T00:43:29.897681+0000","last_became_active":"2024-02-23T00:42:58.671953+0000","last_became_peered":"2024-02-23T00:42:58.671953+0000","last_unstale":"2024-02-23T00:43:29.897681+0000","last_undegraded":"2024-02-23T00:43:29.897681+0000","last_fullsized":"2024-02-23T00:43:29.897681+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"68'18","reported_seq":134,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.940232+0000","last_change":"2024-02-23T00:41:54.082061+0000","last_active":"2024-02-23T00:43:29.940232+0000","last_peered":"2024-02-23T00:43:29.940232+0000","last_clean":"2024-02-23T00:43:29.940232+0000","last_became_active":"2024-02-23T00:39:16.782803+0000","last_became_peered":"2024-02-23T00:39:16.782803+0000","last_unstale":"2024-02-23T00:43:29.940232+0000","last_undegraded":"2024-02-23T00:43:29.940232+0000","last_fullsized":"2024-02-23T00:43:29.940232+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'18","last_scrub_stamp":"2024-02-23T00:41:54.081977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:54.081977+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":98,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.906651+0000","last_change":"2024-02-23T00:41:44.937852+0000","last_active":"2024-02-23T00:43:29.906651+0000","last_peered":"2024-02-23T00:43:29.906651+0000","last_clean":"2024-02-23T00:43:29.906651+0000","last_became_active":"2024-02-23T00:39:14.782163+0000","last_became_peered":"2024-02-23T00:39:14.782163+0000","last_unstale":"2024-02-23T00:43:29.906651+0000","last_undegraded":"2024-02-23T00:43:29.906651+0000","last_fullsized":"2024-02-23T00:43:29.906651+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:44.937776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:44.937776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":93,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.895537+0000","last_change":"2024-02-23T00:42:07.939130+0000","last_active":"2024-02-23T00:43:29.895537+0000","last_peered":"2024-02-23T00:43:29.895537+0000","last_clean":"2024-02-23T00:43:29.895537+0000","last_became_active":"2024-02-23T00:39:18.794903+0000","last_became_peered":"2024-02-23T00:39:18.794903+0000","last_unstale":"2024-02-23T00:43:29.895537+0000","last_undegraded":"2024-02-23T00:43:29.895537+0000","last_fullsized":"2024-02-23T00:43:29.895537+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:07.939058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_clean_scrub_stamp":"2024-02-23T00:42:07.939058+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"110'5","reported_seq":44,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.931832+0000","last_change":"2024-02-23T00:42:58.839927+0000","last_active":"2024-02-23T00:43:29.931832+0000","last_peered":"2024-02-23T00:43:29.931832+0000","last_clean":"2024-02-23T00:43:29.931832+0000","last_became_active":"2024-02-23T00:42:58.838833+0000","last_became_peered":"2024-02-23T00:42:58.838833+0000","last_unstale":"2024-02-23T00:43:29.931832+0000","last_undegraded":"2024-02-23T00:43:29.931832+0000","last_fullsized":"2024-02-23T00:43:29.931832+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.900909+0000","last_change":"2024-02-23T00:42:58.698567+0000","last_active":"2024-02-23T00:43:29.900909+0000","last_peered":"2024-02-23T00:43:29.900909+0000","last_clean":"2024-02-23T00:43:29.900909+0000","last_became_active":"2024-02-23T00:42:58.698258+0000","last_became_peered":"2024-02-23T00:42:58.698258+0000","last_unstale":"2024-02-23T00:43:29.900909+0000","last_undegraded":"2024-02-23T00:43:29.900909+0000","last_fullsized":"2024-02-23T00:43:29.900909+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":38,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.952048+0000","last_change":"2024-02-23T00:42:58.716683+0000","last_active":"2024-02-23T00:43:29.952048+0000","last_peered":"2024-02-23T00:43:29.952048+0000","last_clean":"2024-02-23T00:43:29.952048+0000","last_became_active":"2024-02-23T00:42:58.716194+0000","last_became_peered":"2024-02-23T00:42:58.716194+0000","last_unstale":"2024-02-23T00:43:29.952048+0000","last_undegraded":"2024-02-23T00:43:29.952048+0000","last_fullsized":"2024-02-23T00:43:29.952048+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,1],"acting":[2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"95'27","reported_seq":147,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.902071+0000","last_change":"2024-02-23T00:41:55.920001+0000","last_active":"2024-02-23T00:43:29.902071+0000","last_peered":"2024-02-23T00:43:29.902071+0000","last_clean":"2024-02-23T00:43:29.902071+0000","last_became_active":"2024-02-23T00:39:16.786791+0000","last_became_peered":"2024-02-23T00:39:16.786791+0000","last_unstale":"2024-02-23T00:43:29.902071+0000","last_undegraded":"2024-02-23T00:43:29.902071+0000","last_fullsized":"2024-02-23T00:43:29.902071+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'27","last_scrub_stamp":"2024-02-23T00:41:55.919895+0000","last_deep_scrub":"95'27","last_deep_scrub_stamp":"2024-02-23T00:41:55.919895+0000","last_clean_scrub_stamp":"2024-02-23T00:41:55.919895+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":96,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.916935+0000","last_change":"2024-02-23T00:42:40.046742+0000","last_active":"2024-02-23T00:43:29.916935+0000","last_peered":"2024-02-23T00:43:29.916935+0000","last_clean":"2024-02-23T00:43:29.916935+0000","last_became_active":"2024-02-23T00:39:14.787148+0000","last_became_peered":"2024-02-23T00:39:14.787148+0000","last_unstale":"2024-02-23T00:43:29.916935+0000","last_undegraded":"2024-02-23T00:43:29.916935+0000","last_fullsized":"2024-02-23T00:43:29.916935+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:40.046647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:40.046647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":94,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.900773+0000","last_change":"2024-02-23T00:42:51.967597+0000","last_active":"2024-02-23T00:43:29.900773+0000","last_peered":"2024-02-23T00:43:29.900773+0000","last_clean":"2024-02-23T00:43:29.900773+0000","last_became_active":"2024-02-23T00:39:18.796431+0000","last_became_peered":"2024-02-23T00:39:18.796431+0000","last_unstale":"2024-02-23T00:43:29.900773+0000","last_undegraded":"2024-02-23T00:43:29.900773+0000","last_fullsized":"2024-02-23T00:43:29.900773+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:51.967526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:51.967526+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.913839+0000","last_change":"2024-02-23T00:43:00.945436+0000","last_active":"2024-02-23T00:43:29.913839+0000","last_peered":"2024-02-23T00:43:29.913839+0000","last_clean":"2024-02-23T00:43:29.913839+0000","last_became_active":"2024-02-23T00:43:00.945181+0000","last_became_peered":"2024-02-23T00:43:00.945181+0000","last_unstale":"2024-02-23T00:43:29.913839+0000","last_undegraded":"2024-02-23T00:43:29.913839+0000","last_fullsized":"2024-02-23T00:43:29.913839+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:42.283249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:42.283249+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"110'7","reported_seq":46,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.922741+0000","last_change":"2024-02-23T00:42:58.812691+0000","last_active":"2024-02-23T00:43:29.922741+0000","last_peered":"2024-02-23T00:43:29.922741+0000","last_clean":"2024-02-23T00:43:29.922741+0000","last_became_active":"2024-02-23T00:42:58.810765+0000","last_became_peered":"2024-02-23T00:42:58.810765+0000","last_unstale":"2024-02-23T00:43:29.922741+0000","last_undegraded":"2024-02-23T00:43:29.922741+0000","last_fullsized":"2024-02-23T00:43:29.922741+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.903648+0000","last_change":"2024-02-23T00:42:58.833221+0000","last_active":"2024-02-23T00:43:29.903648+0000","last_peered":"2024-02-23T00:43:29.903648+0000","last_clean":"2024-02-23T00:43:29.903648+0000","last_became_active":"2024-02-23T00:42:58.832885+0000","last_became_peered":"2024-02-23T00:42:58.832885+0000","last_unstale":"2024-02-23T00:43:29.903648+0000","last_undegraded":"2024-02-23T00:43:29.903648+0000","last_fullsized":"2024-02-23T00:43:29.903648+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.a","version":"110'1","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.911039+0000","last_change":"2024-02-23T00:42:58.720623+0000","last_active":"2024-02-23T00:43:29.911039+0000","last_peered":"2024-02-23T00:43:29.911039+0000","last_clean":"2024-02-23T00:43:29.911039+0000","last_became_active":"2024-02-23T00:42:58.718587+0000","last_became_peered":"2024-02-23T00:42:58.718587+0000","last_unstale":"2024-02-23T00:43:29.911039+0000","last_undegraded":"2024-02-23T00:43:29.911039+0000","last_fullsized":"2024-02-23T00:43:29.911039+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"82'15","reported_seq":124,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.895931+0000","last_change":"2024-02-23T00:41:50.975772+0000","last_active":"2024-02-23T00:43:29.895931+0000","last_peered":"2024-02-23T00:43:29.895931+0000","last_clean":"2024-02-23T00:43:29.895931+0000","last_became_active":"2024-02-23T00:39:16.782762+0000","last_became_peered":"2024-02-23T00:39:16.782762+0000","last_unstale":"2024-02-23T00:43:29.895931+0000","last_undegraded":"2024-02-23T00:43:29.895931+0000","last_fullsized":"2024-02-23T00:43:29.895931+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'15","last_scrub_stamp":"2024-02-23T00:41:50.975683+0000","last_deep_scrub":"82'15","last_deep_scrub_stamp":"2024-02-23T00:41:50.975683+0000","last_clean_scrub_stamp":"2024-02-23T00:41:50.975683+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":96,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.904540+0000","last_change":"2024-02-23T00:41:40.926914+0000","last_active":"2024-02-23T00:43:29.904540+0000","last_peered":"2024-02-23T00:43:29.904540+0000","last_clean":"2024-02-23T00:43:29.904540+0000","last_became_active":"2024-02-23T00:39:14.783469+0000","last_became_peered":"2024-02-23T00:39:14.783469+0000","last_unstale":"2024-02-23T00:43:29.904540+0000","last_undegraded":"2024-02-23T00:43:29.904540+0000","last_fullsized":"2024-02-23T00:43:29.904540+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:40.926851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:41:40.926851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":94,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.911289+0000","last_change":"2024-02-23T00:42:00.998947+0000","last_active":"2024-02-23T00:43:29.911289+0000","last_peered":"2024-02-23T00:43:29.911289+0000","last_clean":"2024-02-23T00:43:29.911289+0000","last_became_active":"2024-02-23T00:39:18.796952+0000","last_became_peered":"2024-02-23T00:39:18.796952+0000","last_unstale":"2024-02-23T00:43:29.911289+0000","last_undegraded":"2024-02-23T00:43:29.911289+0000","last_fullsized":"2024-02-23T00:43:29.911289+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:00.998885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:00.998885+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.895138+0000","last_change":"2024-02-23T00:42:58.731558+0000","last_active":"2024-02-23T00:43:29.895138+0000","last_peered":"2024-02-23T00:43:29.895138+0000","last_clean":"2024-02-23T00:43:29.895138+0000","last_became_active":"2024-02-23T00:42:58.726021+0000","last_became_peered":"2024-02-23T00:42:58.726021+0000","last_unstale":"2024-02-23T00:43:29.895138+0000","last_undegraded":"2024-02-23T00:43:29.895138+0000","last_fullsized":"2024-02-23T00:43:29.895138+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.900956+0000","last_change":"2024-02-23T00:42:58.759026+0000","last_active":"2024-02-23T00:43:29.900956+0000","last_peered":"2024-02-23T00:43:29.900956+0000","last_clean":"2024-02-23T00:43:29.900956+0000","last_became_active":"2024-02-23T00:42:58.758147+0000","last_became_peered":"2024-02-23T00:42:58.758147+0000","last_unstale":"2024-02-23T00:43:29.900956+0000","last_undegraded":"2024-02-23T00:43:29.900956+0000","last_fullsized":"2024-02-23T00:43:29.900956+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":38,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.933697+0000","last_change":"2024-02-23T00:42:58.829447+0000","last_active":"2024-02-23T00:43:29.933697+0000","last_peered":"2024-02-23T00:43:29.933697+0000","last_clean":"2024-02-23T00:43:29.933697+0000","last_became_active":"2024-02-23T00:42:58.824135+0000","last_became_peered":"2024-02-23T00:42:58.824135+0000","last_unstale":"2024-02-23T00:43:29.933697+0000","last_undegraded":"2024-02-23T00:43:29.933697+0000","last_fullsized":"2024-02-23T00:43:29.933697+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:57.565761+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.565761+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"68'15","reported_seq":126,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.895235+0000","last_change":"2024-02-23T00:41:33.064109+0000","last_active":"2024-02-23T00:43:29.895235+0000","last_peered":"2024-02-23T00:43:29.895235+0000","last_clean":"2024-02-23T00:43:29.895235+0000","last_became_active":"2024-02-23T00:39:16.784606+0000","last_became_peered":"2024-02-23T00:39:16.784606+0000","last_unstale":"2024-02-23T00:43:29.895235+0000","last_undegraded":"2024-02-23T00:43:29.895235+0000","last_fullsized":"2024-02-23T00:43:29.895235+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:41:33.063992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:41:33.063992+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":98,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.959699+0000","last_change":"2024-02-23T00:42:37.949147+0000","last_active":"2024-02-23T00:43:29.959699+0000","last_peered":"2024-02-23T00:43:29.959699+0000","last_clean":"2024-02-23T00:43:29.959699+0000","last_became_active":"2024-02-23T00:39:14.785362+0000","last_became_peered":"2024-02-23T00:39:14.785362+0000","last_unstale":"2024-02-23T00:43:29.959699+0000","last_undegraded":"2024-02-23T00:43:29.959699+0000","last_fullsized":"2024-02-23T00:43:29.959699+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:37.949044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:37.949044+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":93,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.927312+0000","last_change":"2024-02-23T00:42:44.553722+0000","last_active":"2024-02-23T00:43:29.927312+0000","last_peered":"2024-02-23T00:43:29.927312+0000","last_clean":"2024-02-23T00:43:29.927312+0000","last_became_active":"2024-02-23T00:39:18.796312+0000","last_became_peered":"2024-02-23T00:39:18.796312+0000","last_unstale":"2024-02-23T00:43:29.927312+0000","last_undegraded":"2024-02-23T00:43:29.927312+0000","last_fullsized":"2024-02-23T00:43:29.927312+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:44.553621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:44.553621+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.13","version":"110'9","reported_seq":48,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.922717+0000","last_change":"2024-02-23T00:42:57.656016+0000","last_active":"2024-02-23T00:43:29.922717+0000","last_peered":"2024-02-23T00:43:29.922717+0000","last_clean":"2024-02-23T00:43:29.922717+0000","last_became_active":"2024-02-23T00:42:57.655617+0000","last_became_peered":"2024-02-23T00:42:57.655617+0000","last_unstale":"2024-02-23T00:43:29.922717+0000","last_undegraded":"2024-02-23T00:43:29.922717+0000","last_fullsized":"2024-02-23T00:43:29.922717+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.931454+0000","last_change":"2024-02-23T00:42:58.839750+0000","last_active":"2024-02-23T00:43:29.931454+0000","last_peered":"2024-02-23T00:43:29.931454+0000","last_clean":"2024-02-23T00:43:29.931454+0000","last_became_active":"2024-02-23T00:42:58.838257+0000","last_became_peered":"2024-02-23T00:42:58.838257+0000","last_unstale":"2024-02-23T00:43:29.931454+0000","last_undegraded":"2024-02-23T00:43:29.931454+0000","last_fullsized":"2024-02-23T00:43:29.931454+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":43,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.901235+0000","last_change":"2024-02-23T00:43:08.400281+0000","last_active":"2024-02-23T00:43:29.901235+0000","last_peered":"2024-02-23T00:43:29.901235+0000","last_clean":"2024-02-23T00:43:29.901235+0000","last_became_active":"2024-02-23T00:42:58.691686+0000","last_became_peered":"2024-02-23T00:42:58.691686+0000","last_unstale":"2024-02-23T00:43:29.901235+0000","last_undegraded":"2024-02-23T00:43:29.901235+0000","last_fullsized":"2024-02-23T00:43:29.901235+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:08.400207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:08.400207+0000","last_clean_scrub_stamp":"2024-02-23T00:43:08.400207+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"88'50","reported_seq":192,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.950648+0000","last_change":"2024-02-23T00:42:39.939941+0000","last_active":"2024-02-23T00:43:29.950648+0000","last_peered":"2024-02-23T00:43:29.950648+0000","last_clean":"2024-02-23T00:43:29.950648+0000","last_became_active":"2024-02-23T00:39:16.780144+0000","last_became_peered":"2024-02-23T00:39:16.780144+0000","last_unstale":"2024-02-23T00:43:29.950648+0000","last_undegraded":"2024-02-23T00:43:29.950648+0000","last_fullsized":"2024-02-23T00:43:29.950648+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'50","last_scrub_stamp":"2024-02-23T00:42:39.939829+0000","last_deep_scrub":"88'50","last_deep_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_clean_scrub_stamp":"2024-02-23T00:42:39.939829+0000","log_size":50,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":98,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.912521+0000","last_change":"2024-02-23T00:42:38.204365+0000","last_active":"2024-02-23T00:43:29.912521+0000","last_peered":"2024-02-23T00:43:29.912521+0000","last_clean":"2024-02-23T00:43:29.912521+0000","last_became_active":"2024-02-23T00:39:14.782000+0000","last_became_peered":"2024-02-23T00:39:14.782000+0000","last_unstale":"2024-02-23T00:43:29.912521+0000","last_undegraded":"2024-02-23T00:43:29.912521+0000","last_fullsized":"2024-02-23T00:43:29.912521+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:38.204283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_clean_scrub_stamp":"2024-02-23T00:42:38.204283+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":88,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.895822+0000","last_change":"2024-02-23T00:41:05.965631+0000","last_active":"2024-02-23T00:43:29.895822+0000","last_peered":"2024-02-23T00:43:29.895822+0000","last_clean":"2024-02-23T00:43:29.895822+0000","last_became_active":"2024-02-23T00:39:18.799723+0000","last_became_peered":"2024-02-23T00:39:18.799723+0000","last_unstale":"2024-02-23T00:43:29.895822+0000","last_undegraded":"2024-02-23T00:43:29.895822+0000","last_fullsized":"2024-02-23T00:43:29.895822+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:41:05.965531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:41:05.965531+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.961455+0000","last_change":"2024-02-23T00:42:58.772753+0000","last_active":"2024-02-23T00:43:29.961455+0000","last_peered":"2024-02-23T00:43:29.961455+0000","last_clean":"2024-02-23T00:43:29.961455+0000","last_became_active":"2024-02-23T00:42:58.765643+0000","last_became_peered":"2024-02-23T00:42:58.765643+0000","last_unstale":"2024-02-23T00:43:29.961455+0000","last_undegraded":"2024-02-23T00:43:29.961455+0000","last_fullsized":"2024-02-23T00:43:29.961455+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":39,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.904115+0000","last_change":"2024-02-23T00:42:58.825262+0000","last_active":"2024-02-23T00:43:29.904115+0000","last_peered":"2024-02-23T00:43:29.904115+0000","last_clean":"2024-02-23T00:43:29.904115+0000","last_became_active":"2024-02-23T00:42:58.824573+0000","last_became_peered":"2024-02-23T00:42:58.824573+0000","last_unstale":"2024-02-23T00:43:29.904115+0000","last_undegraded":"2024-02-23T00:43:29.904115+0000","last_fullsized":"2024-02-23T00:43:29.904115+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0],"acting":[5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"110'1","reported_seq":44,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-02-23T00:43:29.909607+0000","last_change":"2024-02-23T00:43:11.035928+0000","last_active":"2024-02-23T00:43:29.909607+0000","last_peered":"2024-02-23T00:43:29.909607+0000","last_clean":"2024-02-23T00:43:29.909607+0000","last_became_active":"2024-02-23T00:42:58.677646+0000","last_became_peered":"2024-02-23T00:42:58.677646+0000","last_unstale":"2024-02-23T00:43:29.909607+0000","last_undegraded":"2024-02-23T00:43:29.909607+0000","last_fullsized":"2024-02-23T00:43:29.909607+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'1","last_scrub_stamp":"2024-02-23T00:43:11.035864+0000","last_deep_scrub":"110'1","last_deep_scrub_stamp":"2024-02-23T00:43:11.035864+0000","last_clean_scrub_stamp":"2024-02-23T00:43:11.035864+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":83,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":63,"num_pg":32,"stat_sum":{"num_bytes":1536,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":4,"num_write":13,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20,"ondisk_log_size":20,"up":96,"acting":96,"num_store_stats":8},{"poolid":62,"num_pg":32,"stat_sum":{"num_bytes":12,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":36,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":151001778,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1011,"num_read_kb":49161,"num_write":1044,"num_write_kb":152981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2800000,"num_omap_keys":1000,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":478445568,"data_stored":478171158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1055,"ondisk_log_size":1055,"up":96,"acting":96,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":25,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":75,"acting":75,"num_store_stats":8},{"poolid":79,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":5},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1080,"num_read_kb":805,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":632,"ondisk_log_size":632,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":249,"num_read_kb":244,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":47,"num_pg":32,"stat_sum":{"num_bytes":651,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2048,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":196608,"data_stored":6144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234039,"num_pgs":190,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":413600,"kb_used_data":100120,"kb_used_omap":0,"kb_used_meta":313472,"kb_avail":93331552,"statfs":{"total":95995035648,"available":95571509248,"internally_reserved":0,"allocated":102522880,"data_stored":101689267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":320995328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.53500000000000003,"15min":0.54300000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72199999999999998,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.50600000000000001,"15min":0.50800000000000001},"min":{"1min":0.36899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.68100000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.39700000000000002}]},{"osd":1,"last update":"Fri Feb 23 00:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.53000000000000003,"15min":0.54500000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.60399999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":5.1820000000000004},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.53700000000000003,"15min":0.54600000000000004},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.64100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.52900000000000003}]},{"osd":2,"last update":"Fri Feb 23 00:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.52400000000000002,"15min":0.53600000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.73399999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53900000000000003,"15min":0.53600000000000003},"min":{"1min":0.42699999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.69099999999999995,"5min":1.024,"15min":1.024},"last":0.38800000000000001}]},{"osd":3,"last update":"Fri Feb 23 00:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.53900000000000003,"15min":0.53100000000000003},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70499999999999996,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57699999999999996,"15min":0.57099999999999995},"min":{"1min":0.41699999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.54500000000000004}]},{"osd":4,"last update":"Fri Feb 23 00:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.438,"15min":0.434},"min":{"1min":0.314,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.63700000000000001,"5min":0.754,"15min":0.754},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.46899999999999997,"15min":0.45900000000000002},"min":{"1min":0.38100000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.70099999999999996,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.29599999999999999}]},{"osd":5,"last update":"Fri Feb 23 00:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.48899999999999999,"15min":0.48199999999999998},"min":{"1min":0.379,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.70699999999999996,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.50600000000000001,"15min":0.48599999999999999},"min":{"1min":0.40600000000000003,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.72299999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.34300000000000003}]},{"osd":6,"last update":"Fri Feb 23 00:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52100000000000002,"15min":0.51100000000000001},"min":{"1min":0.34599999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.69199999999999995,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.373},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.51700000000000002,"15min":0.51000000000000001},"min":{"1min":0.36599999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.67200000000000004,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.36599999999999999}]}]},{"osd":6,"up_from":47,"seq":201863462974,"num_pgs":174,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":384888,"kb_used_data":75504,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":93360264,"statfs":{"total":95995035648,"available":95600910336,"internally_reserved":0,"allocated":77316096,"data_stored":76520810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55200000000000005,"15min":0.56599999999999995},"min":{"1min":0.379,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.77700000000000002,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.55000000000000004,"15min":0.57399999999999995},"min":{"1min":0.36199999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.69499999999999995,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.83299999999999996}]},{"osd":1,"last update":"Fri Feb 23 00:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54500000000000004,"15min":0.55100000000000005},"min":{"1min":0.34499999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.499},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.55900000000000005,"15min":0.57799999999999996},"min":{"1min":0.40400000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76400000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.63100000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.58999999999999997,"15min":0.58699999999999997},"min":{"1min":0.40000000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.8439999999999999,"5min":2.8439999999999999,"15min":2.8439999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.58599999999999997,"15min":0.56899999999999995},"min":{"1min":0.441,"5min":0.38,"15min":0.38},"max":{"1min":2.8530000000000002,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":0.871}]},{"osd":3,"last update":"Fri Feb 23 00:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.39200000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77600000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.59999999999999998,"15min":0.59699999999999998},"min":{"1min":0.39700000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.96999999999999997}]},{"osd":4,"last update":"Fri Feb 23 00:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.436,"15min":0.41299999999999998},"min":{"1min":0.36699999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.61699999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.46000000000000002,"15min":0.436},"min":{"1min":0.32100000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.64600000000000002,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.52100000000000002}]},{"osd":5,"last update":"Fri Feb 23 00:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.44900000000000001,"15min":0.42199999999999999},"min":{"1min":0.35799999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.628,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.48199999999999998,"15min":0.44900000000000001},"min":{"1min":0.34499999999999997,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.68700000000000006,"5min":0.877,"15min":0.877},"last":0.46400000000000002}]},{"osd":7,"last update":"Fri Feb 23 00:43:13 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.58099999999999996,"15min":0.51900000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":8.548,"5min":8.548,"15min":8.548},"last":0.51400000000000001},{"interface":"front","average":{"1min":1.5660000000000001,"5min":0.72999999999999998,"15min":0.58599999999999997},"min":{"1min":0.36499999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":15.618,"5min":15.618,"15min":15.618},"last":0.54100000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":191,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":356244,"kb_used_data":50956,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":93388908,"statfs":{"total":95995035648,"available":95630241792,"internally_reserved":0,"allocated":52178944,"data_stored":51356365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:42:38 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.39900000000000002,"15min":0.35499999999999998},"min":{"1min":0.254,"5min":0.214,"15min":0.214},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.442,"15min":0.378},"min":{"1min":0.252,"5min":0.191,"15min":0.191},"max":{"1min":2.194,"5min":2.194,"15min":2.194},"last":2.0590000000000002}]},{"osd":2,"last update":"Fri Feb 23 00:43:00 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.41099999999999998,"15min":0.39900000000000002},"min":{"1min":0.219,"5min":0.188,"15min":0.188},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.46400000000000002,"15min":0.443},"min":{"1min":0.29299999999999998,"5min":0.249,"15min":0.249},"max":{"1min":2.2330000000000001,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":2.2160000000000002}]},{"osd":3,"last update":"Fri Feb 23 00:42:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.48899999999999999,"15min":0.49099999999999999},"min":{"1min":0.29699999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":2.4350000000000001,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":2.1440000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.45300000000000001,"15min":0.45900000000000002},"min":{"1min":0.27700000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.45100000000000001}]},{"osd":4,"last update":"Fri Feb 23 00:42:49 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":0.70099999999999996,"15min":0.64300000000000002},"min":{"1min":0.44500000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":8.9359999999999999,"5min":8.9359999999999999,"15min":8.9359999999999999},"last":0.624},{"interface":"front","average":{"1min":0.87,"5min":0.67100000000000004,"15min":0.63300000000000001},"min":{"1min":0.46500000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":3.9079999999999999,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":0.72099999999999997}]},{"osd":5,"last update":"Fri Feb 23 00:43:22 2024","interfaces":[{"interface":"back","average":{"1min":3.4279999999999999,"5min":1.1759999999999999,"15min":0.83699999999999997},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":48.792000000000002,"5min":48.792000000000002,"15min":48.792000000000002},"last":0.69799999999999995},{"interface":"front","average":{"1min":1.2250000000000001,"5min":0.78600000000000003,"15min":0.746},"min":{"1min":0.38100000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":11.872999999999999,"5min":11.872999999999999,"15min":11.872999999999999},"last":2.1669999999999998}]},{"osd":6,"last update":"Fri Feb 23 00:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.64300000000000002,"15min":0.64400000000000002},"min":{"1min":0.52700000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.67000000000000004,"15min":0.66000000000000003},"min":{"1min":0.58199999999999996,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":2.4079999999999999,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":2.1840000000000002}]},{"osd":7,"last update":"Fri Feb 23 00:43:15 2024","interfaces":[{"interface":"back","average":{"1min":1.597,"5min":0.83399999999999996,"15min":0.69399999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":20.225000000000001,"5min":20.225000000000001,"15min":20.225000000000001},"last":2.2000000000000002},{"interface":"front","average":{"1min":1.756,"5min":0.84799999999999998,"15min":0.69699999999999995},"min":{"1min":0.48599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":20.254999999999999,"5min":20.254999999999999,"15min":20.254999999999999},"last":0.68400000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":165,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":384896,"kb_used_data":75512,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":93360256,"statfs":{"total":95995035648,"available":95600902144,"internally_reserved":0,"allocated":77324288,"data_stored":76524174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 23 00:42:35 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.44600000000000001,"15min":0.36099999999999999},"min":{"1min":0.26700000000000002,"5min":0.223,"15min":0.19500000000000001},"max":{"1min":6.5919999999999996,"5min":6.5919999999999996,"15min":6.5919999999999996},"last":28.856999999999999},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42099999999999999,"15min":0.35899999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.20799999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":20.577000000000002}]},{"osd":2,"last update":"Fri Feb 23 00:43:03 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.49099999999999999,"15min":0.41699999999999998},"min":{"1min":0.26000000000000001,"5min":0.24199999999999999,"15min":0.20499999999999999},"max":{"1min":6.3440000000000003,"5min":6.3440000000000003,"15min":6.3440000000000003},"last":3.9540000000000002},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.48799999999999999,"15min":0.41199999999999998},"min":{"1min":0.27300000000000002,"5min":0.24299999999999999,"15min":0.19800000000000001},"max":{"1min":6.407,"5min":6.407,"15min":6.407},"last":3.4580000000000002}]},{"osd":3,"last update":"Fri Feb 23 00:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.52000000000000002,"15min":0.47199999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":7.7110000000000003,"5min":7.7110000000000003,"15min":7.7110000000000003},"last":35.566000000000003},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.54300000000000004,"15min":0.51800000000000002},"min":{"1min":0.33400000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":6.6929999999999996,"5min":6.6929999999999996,"15min":6.6929999999999996},"last":20.702999999999999}]},{"osd":4,"last update":"Fri Feb 23 00:42:51 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":0.70099999999999996,"15min":0.70699999999999996},"min":{"1min":0.46000000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":6.4589999999999996,"5min":6.4589999999999996,"15min":6.4589999999999996},"last":3.1890000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.64600000000000002,"15min":0.72399999999999998},"min":{"1min":0.503,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.85699999999999998,"5min":3.319,"15min":3.319},"last":3.8999999999999999}]},{"osd":5,"last update":"Fri Feb 23 00:43:16 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":0.67200000000000004,"15min":0.63400000000000001},"min":{"1min":0.375,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":6.4909999999999997,"5min":6.4909999999999997,"15min":6.4909999999999997},"last":3.1469999999999998},{"interface":"front","average":{"1min":1.0349999999999999,"5min":0.71899999999999997,"15min":0.68899999999999995},"min":{"1min":0.437,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":6.6680000000000001,"5min":6.6680000000000001,"15min":6.6680000000000001},"last":1.8080000000000001}]},{"osd":6,"last update":"Fri Feb 23 00:42:51 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":0.71599999999999997,"15min":0.66900000000000004},"min":{"1min":0.45300000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":6.633,"5min":6.633,"15min":6.633},"last":1.7809999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.63300000000000001,"15min":0.63500000000000001},"min":{"1min":0.49099999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":3.867}]},{"osd":7,"last update":"Fri Feb 23 00:43:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":0.72199999999999998,"15min":0.69299999999999995},"min":{"1min":0.47099999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":6.6989999999999998,"5min":6.6989999999999998,"15min":6.6989999999999998},"last":1.8240000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.63500000000000001,"15min":0.64600000000000002},"min":{"1min":0.48399999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":3.927}]}]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":139,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":331624,"kb_used_data":26336,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":93413528,"statfs":{"total":95995035648,"available":95655452672,"internally_reserved":0,"allocated":26968064,"data_stored":26188293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:42:58 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38200000000000001,"15min":0.378},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":1.165},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.48499999999999999,"15min":0.41499999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.23599999999999999},"max":{"1min":6.8879999999999999,"5min":6.8879999999999999,"15min":6.8879999999999999},"last":3.9950000000000001}]},{"osd":1,"last update":"Fri Feb 23 00:42:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.40899999999999997,"15min":0.38300000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":2.2549999999999999},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.42999999999999999,"15min":0.40400000000000003},"min":{"1min":0.30199999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":1.2589999999999999}]},{"osd":3,"last update":"Fri Feb 23 00:43:30 2024","interfaces":[{"interface":"back","average":{"1min":1.9079999999999999,"5min":0.79700000000000004,"15min":0.59499999999999997},"min":{"1min":0.28299999999999997,"5min":0.25,"15min":0.25},"max":{"1min":22.588000000000001,"5min":22.588000000000001,"15min":22.588000000000001},"last":4.0140000000000002},{"interface":"front","average":{"1min":2.2519999999999998,"5min":0.79800000000000004,"15min":0.58199999999999996},"min":{"1min":0.307,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":19.960000000000001,"5min":19.960000000000001,"15min":19.960000000000001},"last":2.23}]},{"osd":4,"last update":"Fri Feb 23 00:42:58 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.626,"15min":0.60699999999999998},"min":{"1min":0.48899999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.224,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":1.202},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.629,"15min":0.59399999999999997},"min":{"1min":0.38400000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.878,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":2.1309999999999998}]},{"osd":5,"last update":"Fri Feb 23 00:43:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.63300000000000001,"15min":0.60099999999999998},"min":{"1min":0.40600000000000003,"5min":0.378,"15min":0.378},"max":{"1min":3.137,"5min":3.137,"15min":3.137},"last":1.2310000000000001},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.70599999999999996,"15min":0.64400000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":7.0019999999999998,"5min":7.0019999999999998,"15min":7.0019999999999998},"last":3.9380000000000002}]},{"osd":6,"last update":"Fri Feb 23 00:42:48 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":0.72599999999999998,"15min":0.65800000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":6.9390000000000001,"5min":6.9390000000000001,"15min":6.9390000000000001},"last":3.8929999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.629,"15min":0.60699999999999998},"min":{"1min":0.442,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.95499999999999996,"5min":1.165,"15min":1.165},"last":2.1760000000000002}]},{"osd":7,"last update":"Fri Feb 23 00:43:10 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.74199999999999999,"15min":0.69699999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":6.9740000000000002,"5min":6.9740000000000002,"15min":6.9740000000000002},"last":3.9689999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.433,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90200000000000002,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":2.202}]}]},{"osd":3,"up_from":26,"seq":111669149772,"num_pgs":160,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":360376,"kb_used_data":50992,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":93384776,"statfs":{"total":95995035648,"available":95626010624,"internally_reserved":0,"allocated":52215808,"data_stored":51355998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:43:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":0.52800000000000002,"15min":0.47199999999999998},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":5.9039999999999999,"5min":5.9039999999999999,"15min":5.9039999999999999},"last":3.0419999999999998},{"interface":"front","average":{"1min":1.498,"5min":0.63,"15min":0.51900000000000002},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":8.9710000000000001,"5min":8.9710000000000001,"15min":8.9710000000000001},"last":3.4580000000000002}]},{"osd":1,"last update":"Fri Feb 23 00:43:30 2024","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":0.64800000000000002,"15min":0.53900000000000003},"min":{"1min":0.24399999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":9,"5min":9,"15min":9},"last":3.9750000000000001},{"interface":"front","average":{"1min":1.292,"5min":0.60499999999999998,"15min":0.50700000000000001},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":4.3419999999999996,"5min":4.3419999999999996,"15min":4.3419999999999996},"last":3.75}]},{"osd":2,"last update":"Fri Feb 23 00:43:30 2024","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":0.67200000000000004,"15min":0.55800000000000005},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":6.6630000000000003,"5min":6.6630000000000003,"15min":6.6630000000000003},"last":1.032},{"interface":"front","average":{"1min":1.3340000000000001,"5min":0.622,"15min":0.53600000000000003},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":5.9329999999999998,"5min":5.9329999999999998,"15min":5.9329999999999998},"last":4.5819999999999999}]},{"osd":4,"last update":"Fri Feb 23 00:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.63600000000000001,"15min":0.60799999999999998},"min":{"1min":0.39500000000000002,"5min":0.373,"15min":0.373},"max":{"1min":4.4329999999999998,"5min":4.4329999999999998,"15min":4.4329999999999998},"last":0.46700000000000003}]},{"osd":5,"last update":"Fri Feb 23 00:43:18 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.60399999999999998},"min":{"1min":0.38100000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.65000000000000002,"15min":0.63600000000000001},"min":{"1min":0.45100000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.2999999999999998,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":0.35499999999999998}]},{"osd":6,"last update":"Fri Feb 23 00:42:41 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.64900000000000002,"15min":0.625},"min":{"1min":0.44400000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":4.46,"5min":4.46,"15min":4.46},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63300000000000001,"15min":0.629},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.52000000000000002}]},{"osd":7,"last update":"Fri Feb 23 00:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.66800000000000004,"15min":0.64800000000000002},"min":{"1min":0.441,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":2.2589999999999999,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.70799999999999996,"15min":0.67500000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":4.4850000000000003,"5min":4.4850000000000003,"15min":4.4850000000000003},"last":0.50600000000000001}]}]},{"osd":4,"up_from":32,"seq":137438953544,"num_pgs":162,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":311192,"kb_used_data":1808,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":93433960,"statfs":{"total":95995035648,"available":95676375040,"internally_reserved":0,"allocated":1851392,"data_stored":1027348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53800000000000003,"15min":0.57399999999999995},"min":{"1min":0.29999999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.57599999999999996,"15min":0.58599999999999997},"min":{"1min":0.32100000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.8660000000000001,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":0.66400000000000003}]},{"osd":1,"last update":"Fri Feb 23 00:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.52700000000000002,"15min":0.57099999999999995},"min":{"1min":0.33500000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69199999999999995,"5min":1.175,"15min":1.175},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.55900000000000005,"15min":0.57199999999999995},"min":{"1min":0.34300000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.96199999999999997,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.76000000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.58799999999999997,"15min":0.58299999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.0790000000000002,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.56000000000000005,"15min":0.58299999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.82399999999999995,"5min":1.113,"15min":1.113},"last":0.86399999999999999}]},{"osd":3,"last update":"Fri Feb 23 00:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.59099999999999997,"15min":0.58599999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.9239999999999999,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.627,"15min":0.59999999999999998},"min":{"1min":0.36299999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":4.1909999999999998,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":0.76400000000000001}]},{"osd":5,"last update":"Fri Feb 23 00:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.42799999999999999,"15min":0.38200000000000001},"min":{"1min":0.252,"5min":0.246,"15min":0.246},"max":{"1min":0.66300000000000003,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.37},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.48899999999999999,"15min":0.43099999999999999},"min":{"1min":0.32500000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.79700000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.34799999999999998}]},{"osd":6,"last update":"Fri Feb 23 00:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.45700000000000002,"15min":0.41499999999999998},"min":{"1min":0.35499999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.45900000000000002,"15min":0.41499999999999998},"min":{"1min":0.38,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70499999999999996,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.314}]},{"osd":7,"last update":"Fri Feb 23 00:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.50900000000000001,"15min":0.49099999999999999},"min":{"1min":0.28599999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.56000000000000005,"15min":0.54300000000000004},"min":{"1min":0.40899999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.75700000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.39200000000000002}]}]},{"osd":5,"up_from":39,"seq":167503724611,"num_pgs":138,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":384908,"kb_used_data":75524,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":93360244,"statfs":{"total":95995035648,"available":95600889856,"internally_reserved":0,"allocated":77336576,"data_stored":76523925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:43:18 2024","interfaces":[{"interface":"back","average":{"1min":1.5760000000000001,"5min":0.71899999999999997,"15min":0.57899999999999996},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":23.881,"5min":23.881,"15min":23.881},"last":0.879},{"interface":"front","average":{"1min":1.054,"5min":0.623,"15min":0.55800000000000005},"min":{"1min":0.34599999999999997,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":5.4649999999999999,"5min":5.4649999999999999,"15min":5.4649999999999999},"last":2.9239999999999999}]},{"osd":1,"last update":"Fri Feb 23 00:43:18 2024","interfaces":[{"interface":"back","average":{"1min":1.571,"5min":0.73399999999999999,"15min":0.59599999999999997},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":18.175999999999998,"5min":18.175999999999998,"15min":18.175999999999998},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.2110000000000001,"5min":0.65900000000000003,"15min":0.58299999999999996},"min":{"1min":0.35599999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":9.0950000000000006,"5min":9.0950000000000006,"15min":9.0950000000000006},"last":3.0539999999999998}]},{"osd":2,"last update":"Fri Feb 23 00:43:18 2024","interfaces":[{"interface":"back","average":{"1min":1.728,"5min":0.77600000000000002,"15min":0.626},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":22.282,"5min":22.282,"15min":22.282},"last":1.8420000000000001},{"interface":"front","average":{"1min":2.141,"5min":0.86399999999999999,"15min":0.64500000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":23.925000000000001,"5min":23.925000000000001,"15min":23.925000000000001},"last":1.081}]},{"osd":3,"last update":"Fri Feb 23 00:43:18 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.57699999999999996,"15min":0.54700000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":6.2190000000000003,"5min":6.2190000000000003,"15min":6.2190000000000003},"last":3.2629999999999999},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.63500000000000001,"15min":0.58499999999999996},"min":{"1min":0.40200000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":8.1760000000000002,"5min":8.1760000000000002,"15min":8.1760000000000002},"last":6.7910000000000004}]},{"osd":4,"last update":"Fri Feb 23 00:43:18 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.432,"15min":0.39500000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.63400000000000001,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.46899999999999997,"15min":0.40899999999999997},"min":{"1min":0.27500000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":3.879}]},{"osd":6,"last update":"Fri Feb 23 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.46800000000000003,"15min":0.438},"min":{"1min":0.41699999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73099999999999998,"5min":0.754,"15min":0.754},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.49099999999999999,"15min":0.45500000000000002},"min":{"1min":0.41699999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.748,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.379}]},{"osd":7,"last update":"Fri Feb 23 00:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.59799999999999998,"15min":0.53800000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":7.0300000000000002,"5min":7.0300000000000002,"15min":7.0300000000000002},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.59799999999999998,"15min":0.53800000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":9.0579999999999998,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":0.35899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":83927040,"data_stored":83892753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":50372608,"data_stored":50331777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":25186304,"data_stored":25165866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":58753024,"data_stored":58720311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":75534336,"data_stored":75500336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":75522048,"data_stored":75497578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":109125632,"data_stored":109058684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":47,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":47,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":47,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":11,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":47,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":47,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":47,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":47,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":47,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":12,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":12,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":63,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":63,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":63,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":63,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":63,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":63,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":63,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":63,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":79,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":79,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":79,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":79,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":79,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-23T00:43:37.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:35.877614+0000 osd.1 (osd.1) 89 : cluster [DBG] 20.4 deep-scrub starts 2024-02-23T00:43:37.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:35.881919+0000 osd.1 (osd.1) 90 : cluster [DBG] 20.4 deep-scrub ok 2024-02-23T00:43:37.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:36.170788+0000 mon.a (mon.0) 1142 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "24"}]': finished 2024-02-23T00:43:37.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:36.174563+0000 osd.4 (osd.4) 77 : cluster [DBG] 6.e scrub starts 2024-02-23T00:43:37.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:36.183074+0000 mon.b (mon.1) 157 : audit [INF] from='client.? 172.21.15.47:0/2559316827' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:37.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:36.205874+0000 osd.4 (osd.4) 78 : cluster [DBG] 6.e scrub ok 2024-02-23T00:43:37.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:36.670703+0000 mon.b (mon.1) 158 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:37.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:36.739369+0000 mon.a (mon.0) 1143 : cluster [DBG] osdmap e144: 8 total, 8 up, 8 in 2024-02-23T00:43:37.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:36.745569+0000 mon.a (mon.0) 1144 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:37.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:36.767262+0000 mon.a (mon.0) 1145 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:37.875 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:43:37.889 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:37.889 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:43:37.990 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:43:38.016 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:38.035 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:43:38.037 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:38.038 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:43:38.139 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:43:38.185 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:43:38.310 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:38.405 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:36.443711+0000 osd.2 (osd.2) 69 : cluster [DBG] 4.1c scrub starts 2024-02-23T00:43:38.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:36.443711+0000 osd.2 (osd.2) 69 : cluster [DBG] 4.1c scrub starts 2024-02-23T00:43:38.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:36.449680+0000 osd.2 (osd.2) 70 : cluster [DBG] 4.1c scrub ok 2024-02-23T00:43:38.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:37.174707+0000 mon.a (mon.0) 1146 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:38.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:37.174882+0000 mon.a (mon.0) 1147 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:38.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:37.174947+0000 mon.a (mon.0) 1148 : cluster [DBG] osdmap e145: 8 total, 8 up, 8 in 2024-02-23T00:43:38.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:37.181333+0000 mon.b (mon.1) 159 : audit [INF] from='client.? 172.21.15.47:0/2655493420' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi047-82386-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:38.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:37.201653+0000 mgr.y (mgr.14517) 194 : cluster [DBG] pgmap v193: 421 pgs: 64 creating+peering, 1 active+clean+snaptrim, 32 unknown, 2 peering, 322 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.3 KiB/s rd, 19 MiB/s wr, 5 op/s 2024-02-23T00:43:38.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:37.205606+0000 mon.a (mon.0) 1149 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi047-82386-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:38.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:37.237542+0000 osd.0 (osd.0) 77 : cluster [DBG] 20.3 deep-scrub starts 2024-02-23T00:43:38.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:37.249085+0000 osd.0 (osd.0) 78 : cluster [DBG] 20.3 deep-scrub ok 2024-02-23T00:43:38.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:37.385730+0000 osd.6 (osd.6) 65 : cluster [DBG] 20.1 deep-scrub starts 2024-02-23T00:43:38.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:37.387863+0000 osd.6 (osd.6) 66 : cluster [DBG] 20.1 deep-scrub ok 2024-02-23T00:43:38.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:37.672867+0000 mon.b (mon.1) 160 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:38.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:36.443711+0000 osd.2 (osd.2) 69 : cluster [DBG] 4.1c scrub starts 2024-02-23T00:43:38.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:36.449680+0000 osd.2 (osd.2) 70 : cluster [DBG] 4.1c scrub ok 2024-02-23T00:43:38.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:37.174707+0000 mon.a (mon.0) 1146 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:38.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:37.174882+0000 mon.a (mon.0) 1147 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:38.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:37.174947+0000 mon.a (mon.0) 1148 : cluster [DBG] osdmap e145: 8 total, 8 up, 8 in 2024-02-23T00:43:38.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:37.181333+0000 mon.b (mon.1) 159 : audit [INF] from='client.? 172.21.15.47:0/2655493420' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi047-82386-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:38.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:37.201653+0000 mgr.y (mgr.14517) 194 : cluster [DBG] pgmap v193: 421 pgs: 64 creating+peering, 1 active+clean+snaptrim, 32 unknown, 2 peering, 322 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.3 KiB/s rd, 19 MiB/s wr, 5 op/s 2024-02-23T00:43:38.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:37.205606+0000 mon.a (mon.0) 1149 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi047-82386-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:38.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:37.237542+0000 osd.0 (osd.0) 77 : cluster [DBG] 20.3 deep-scrub starts 2024-02-23T00:43:38.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:37.249085+0000 osd.0 (osd.0) 78 : cluster [DBG] 20.3 deep-scrub ok 2024-02-23T00:43:38.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:37.385730+0000 osd.6 (osd.6) 65 : cluster [DBG] 20.1 deep-scrub starts 2024-02-23T00:43:38.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:37.387863+0000 osd.6 (osd.6) 66 : cluster [DBG] 20.1 deep-scrub ok 2024-02-23T00:43:38.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:37.672867+0000 mon.b (mon.1) 160 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:38.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:36.449680+0000 osd.2 (osd.2) 70 : cluster [DBG] 4.1c scrub ok 2024-02-23T00:43:38.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:37.174707+0000 mon.a (mon.0) 1146 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82478-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:38.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:37.174882+0000 mon.a (mon.0) 1147 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:38.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:37.174947+0000 mon.a (mon.0) 1148 : cluster [DBG] osdmap e145: 8 total, 8 up, 8 in 2024-02-23T00:43:38.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:37.181333+0000 mon.b (mon.1) 159 : audit [INF] from='client.? 172.21.15.47:0/2655493420' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi047-82386-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:38.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:37.201653+0000 mgr.y (mgr.14517) 194 : cluster [DBG] pgmap v193: 421 pgs: 64 creating+peering, 1 active+clean+snaptrim, 32 unknown, 2 peering, 322 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.3 KiB/s rd, 19 MiB/s wr, 5 op/s 2024-02-23T00:43:38.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:37.205606+0000 mon.a (mon.0) 1149 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi047-82386-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:38.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:37.237542+0000 osd.0 (osd.0) 77 : cluster [DBG] 20.3 deep-scrub starts 2024-02-23T00:43:38.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:37.249085+0000 osd.0 (osd.0) 78 : cluster [DBG] 20.3 deep-scrub ok 2024-02-23T00:43:38.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:37.385730+0000 osd.6 (osd.6) 65 : cluster [DBG] 20.1 deep-scrub starts 2024-02-23T00:43:38.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:37.387863+0000 osd.6 (osd.6) 66 : cluster [DBG] 20.1 deep-scrub ok 2024-02-23T00:43:38.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:37.672867+0000 mon.b (mon.1) 160 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:38.776 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:43:38.776 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-02-23T00:43:38.776 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-02-23T00:43:38.776 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-02-23T00:43:38.776 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:43:39.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.105+0000 7f2877660700 1 -- 172.21.15.47:0/3498938815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28700fe510 msgr2=0x7f28700f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:39.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.105+0000 7f2877660700 1 --2- 172.21.15.47:0/3498938815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28700fe510 0x7f28700f7d40 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 rx=0x7f286c0047b0 tx=0x7f286c009cb0).stop 2024-02-23T00:43:39.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.106+0000 7f2877660700 1 -- 172.21.15.47:0/3498938815 shutdown_connections 2024-02-23T00:43:39.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.106+0000 7f2877660700 1 --2- 172.21.15.47:0/3498938815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28700f8420 0x7f2870104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:43:39.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.106+0000 7f2877660700 1 --2- 172.21.15.47:0/3498938815 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f28700fdbb0 0x7f28700fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:39.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.106+0000 7f2877660700 1 --2- 172.21.15.47:0/3498938815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28700fe510 0x7f28700f7d40 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.106+0000 7f2877660700 1 -- 172.21.15.47:0/3498938815 >> 172.21.15.47:0/3498938815 conn(0x7f28700f3800 msgr2=0x7f28700f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:43:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.106+0000 7f2877660700 1 -- 172.21.15.47:0/3498938815 shutdown_connections 2024-02-23T00:43:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.106+0000 7f2877660700 1 -- 172.21.15.47:0/3498938815 wait complete. 2024-02-23T00:43:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.107+0000 7f2877660700 1 librados: starting msgr at 2024-02-23T00:43:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.107+0000 7f2877660700 1 librados: starting objecter 2024-02-23T00:43:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.107+0000 7f2877660700 1 Processor -- start 2024-02-23T00:43:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.107+0000 7f2877660700 1 -- start start 2024-02-23T00:43:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.107+0000 7f2877660700 1 librados: setting wanted keys 2024-02-23T00:43:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.107+0000 7f2877660700 1 librados: calling monclient init 2024-02-23T00:43:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.107+0000 7f2877660700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f28700f8420 0x7f2870068060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.107+0000 7f2877660700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28700fdbb0 0x7f2870068590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.107+0000 7f2877660700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28700fe510 0x7f28700632a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.107+0000 7f2877660700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2870063820 con 0x7f28700fe510 2024-02-23T00:43:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.107+0000 7f2877660700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2870063990 con 0x7f28700fdbb0 2024-02-23T00:43:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.107+0000 7f2877660700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2870063b50 con 0x7f28700f8420 2024-02-23T00:43:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.107+0000 7f2875bfd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28700fe510 0x7f28700632a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.108+0000 7f2875bfd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28700fe510 0x7f28700632a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37068/0 (socket says 172.21.15.47:37068) 2024-02-23T00:43:39.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.108+0000 7f2875bfd700 1 -- 172.21.15.47:0/1296951668 learned_addr learned my addr 172.21.15.47:0/1296951668 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:43:39.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.108+0000 7f2874bfb700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28700fdbb0 0x7f2870068590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:39.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.108+0000 7f2875bfd700 1 -- 172.21.15.47:0/1296951668 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f28700f8420 msgr2=0x7f2870068060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-23T00:43:39.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.108+0000 7f2875bfd700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f28700f8420 0x7f2870068060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:43:39.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.108+0000 7f2875bfd700 1 -- 172.21.15.47:0/1296951668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28700fdbb0 msgr2=0x7f2870068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:39.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.108+0000 7f2875bfd700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28700fdbb0 0x7f2870068590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:39.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.108+0000 7f2875bfd700 1 -- 172.21.15.47:0/1296951668 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28680097c0 con 0x7f28700fe510 2024-02-23T00:43:39.338 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:43:39.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.108+0000 7f2875bfd700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28700fe510 0x7f28700632a0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 rx=0x7f286000b460 tx=0x7f2860006f20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:39.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.109+0000 7f28667fc700 10 client.?.objecter ms_handle_connect 0x7f28700fe510 2024-02-23T00:43:39.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.109+0000 7f28667fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:43:39.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.109+0000 7f28667fc700 1 -- 172.21.15.47:0/1296951668 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286000be60 con 0x7f28700fe510 2024-02-23T00:43:39.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.109+0000 7f28667fc700 1 -- 172.21.15.47:0/1296951668 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f28600128a0 con 0x7f28700fe510 2024-02-23T00:43:39.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.109+0000 7f28667fc700 1 -- 172.21.15.47:0/1296951668 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286000d8d0 con 0x7f28700fe510 2024-02-23T00:43:39.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.109+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f286c005040 con 0x7f28700fe510 2024-02-23T00:43:39.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.109+0000 7f2877660700 10 client.26418.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:43:39.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.109+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f287006be80 con 0x7f28700fe510 2024-02-23T00:43:39.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.109+0000 7f2877660700 1 librados: init done 2024-02-23T00:43:39.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.110+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f2870064c20 con 0x7f28700fe510 2024-02-23T00:43:39.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.114+0000 7f28667fc700 1 -- 172.21.15.47:0/1296951668 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f286000dd00 con 0x7f28700fe510 2024-02-23T00:43:39.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.114+0000 7f28667fc700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f285c062d10 0x7f285c0651d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:39.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.114+0000 7f28667fc700 1 -- 172.21.15.47:0/1296951668 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(146..146 src has 48..146) v4 ==== 12803+0+0 (secure 0 0 0) 0x7f286007e6b0 con 0x7f28700fe510 2024-02-23T00:43:39.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.114+0000 7f28667fc700 10 client.26418.objecter ms_dispatch 0x7f2870001550 osd_map(146..146 src has 48..146) v4 2024-02-23T00:43:39.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.114+0000 7f28667fc700 3 client.26418.objecter handle_osd_map got epochs [146,146] > 0 2024-02-23T00:43:39.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.114+0000 7f28667fc700 3 client.26418.objecter handle_osd_map decoding full epoch 146 2024-02-23T00:43:39.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.114+0000 7f28667fc700 20 client.26418.objecter dump_active .. 0 homeless 2024-02-23T00:43:39.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.115+0000 7f28667fc700 1 -- 172.21.15.47:0/1296951668 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v146) v1 ==== 54+0+16 (secure 0 0 0) 0x7f286007eb10 con 0x7f28700fe510 2024-02-23T00:43:39.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.115+0000 7f2877660700 10 client.26418.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:39.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.115+0000 7f2877660700 15 client.26418.objecter _session_command_op_assign -1 1 2024-02-23T00:43:39.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.115+0000 7f2877660700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f28700273c0 0x7f28700277a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:39.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.115+0000 7f2877660700 20 client.26418.objecter _get_session s=0x7f2870060280 osd=0 3 2024-02-23T00:43:39.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.115+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f2870060280 osd=0 3 2024-02-23T00:43:39.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.115+0000 7f2877660700 20 client.26418.objecter _get_session s=0x7f2870060280 osd=0 3 2024-02-23T00:43:39.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.115+0000 7f2877660700 15 client.26418.objecter _session_command_op_remove -1 1 2024-02-23T00:43:39.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.115+0000 7f2877660700 20 client.26418.objecter get_session s=0x7f2870060280 osd=0 3 2024-02-23T00:43:39.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.115+0000 7f2877660700 15 client.26418.objecter _session_command_op_assign 0 1 2024-02-23T00:43:39.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.115+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f2870060280 osd=0 4 2024-02-23T00:43:39.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.115+0000 7f2877660700 10 client.26418.objecter _send_command 1 2024-02-23T00:43:39.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.115+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2870064140 con 0x7f28700273c0 2024-02-23T00:43:39.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.115+0000 7f28753fc700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f285c062d10 0x7f285c0651d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:39.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.115+0000 7f2874bfb700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f28700273c0 0x7f28700277a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:39.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.116+0000 7f28753fc700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f285c062d10 0x7f285c0651d0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 rx=0x7f28680012a0 tx=0x7f2868004d10).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:39.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.116+0000 7f28667fc700 10 client.26418.objecter ms_handle_connect 0x7f285c062d10 2024-02-23T00:43:39.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.116+0000 7f28667fc700 10 client.26418.objecter ms_handle_connect 0x7f28700273c0 2024-02-23T00:43:39.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.116+0000 7f2874bfb700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f28700273c0 0x7f28700277a0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:39.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.121+0000 7f28667fc700 1 -- 172.21.15.47:0/1296951668 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2870064140 con 0x7f28700273c0 2024-02-23T00:43:39.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.121+0000 7f28667fc700 10 client.26418.objecter ms_dispatch 0x7f2870001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:43:39.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.121+0000 7f28667fc700 10 client.26418.objecter _finish_command 1 = system:0 2024-02-23T00:43:39.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.121+0000 7f28667fc700 20 client.26418.objecter put_session s=0x7f2870060280 osd=0 4 2024-02-23T00:43:39.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.121+0000 7f28667fc700 15 client.26418.objecter _session_command_op_remove 0 1 2024-02-23T00:43:39.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.143+0000 7f2877660700 10 client.26418.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:43:39.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.143+0000 7f2877660700 15 client.26418.objecter _session_command_op_assign -1 2 2024-02-23T00:43:39.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.143+0000 7f2877660700 20 client.26418.objecter _get_session s=0x7f2870060280 osd=0 3 2024-02-23T00:43:39.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.143+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f2870060280 osd=0 3 2024-02-23T00:43:39.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.143+0000 7f2877660700 20 client.26418.objecter _get_session s=0x7f2870060280 osd=0 3 2024-02-23T00:43:39.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.144+0000 7f2877660700 15 client.26418.objecter _session_command_op_remove -1 2 2024-02-23T00:43:39.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.144+0000 7f2877660700 20 client.26418.objecter get_session s=0x7f2870060280 osd=0 3 2024-02-23T00:43:39.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.144+0000 7f2877660700 15 client.26418.objecter _session_command_op_assign 0 2 2024-02-23T00:43:39.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.144+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f2870060280 osd=0 4 2024-02-23T00:43:39.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.144+0000 7f2877660700 10 client.26418.objecter _send_command 2 2024-02-23T00:43:39.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.144+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f287005ee80 con 0x7f28700273c0 2024-02-23T00:43:39.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.149+0000 7f28667fc700 1 -- 172.21.15.47:0/1296951668 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f287005ee80 con 0x7f28700273c0 2024-02-23T00:43:39.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.149+0000 7f28667fc700 10 client.26418.objecter ms_dispatch 0x7f2870001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:43:39.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.149+0000 7f28667fc700 10 client.26418.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:43:39.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.149+0000 7f28667fc700 20 client.26418.objecter put_session s=0x7f2870060280 osd=0 4 2024-02-23T00:43:39.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.149+0000 7f28667fc700 15 client.26418.objecter _session_command_op_remove 0 2 2024-02-23T00:43:39.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.152+0000 7f2877660700 10 client.26418.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:39.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.152+0000 7f2877660700 15 client.26418.objecter _session_command_op_assign -1 3 2024-02-23T00:43:39.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.152+0000 7f2877660700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f287006c2e0 0x7f28700fe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:39.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.152+0000 7f2877660700 20 client.26418.objecter _get_session s=0x7f2870106ed0 osd=1 3 2024-02-23T00:43:39.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.152+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f2870106ed0 osd=1 3 2024-02-23T00:43:39.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.152+0000 7f2877660700 20 client.26418.objecter _get_session s=0x7f2870106ed0 osd=1 3 2024-02-23T00:43:39.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.152+0000 7f2877660700 15 client.26418.objecter _session_command_op_remove -1 3 2024-02-23T00:43:39.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.152+0000 7f2877660700 20 client.26418.objecter get_session s=0x7f2870106ed0 osd=1 3 2024-02-23T00:43:39.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.152+0000 7f2877660700 15 client.26418.objecter _session_command_op_assign 1 3 2024-02-23T00:43:39.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.152+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f2870106ed0 osd=1 4 2024-02-23T00:43:39.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.152+0000 7f2877660700 10 client.26418.objecter _send_command 3 2024-02-23T00:43:39.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.152+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f287006c6c0 con 0x7f287006c2e0 2024-02-23T00:43:39.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.152+0000 7f28753fc700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f287006c2e0 0x7f28700fe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:39.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.153+0000 7f28753fc700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f287006c2e0 0x7f28700fe8f0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:39.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.153+0000 7f28667fc700 10 client.26418.objecter ms_handle_connect 0x7f287006c2e0 2024-02-23T00:43:39.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.154+0000 7f28667fc700 1 -- 172.21.15.47:0/1296951668 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f287006c6c0 con 0x7f287006c2e0 2024-02-23T00:43:39.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.154+0000 7f28667fc700 10 client.26418.objecter ms_dispatch 0x7f2870001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:43:39.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.154+0000 7f28667fc700 10 client.26418.objecter _finish_command 3 = system:0 2024-02-23T00:43:39.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.154+0000 7f28667fc700 20 client.26418.objecter put_session s=0x7f2870106ed0 osd=1 4 2024-02-23T00:43:39.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.154+0000 7f28667fc700 15 client.26418.objecter _session_command_op_remove 1 3 2024-02-23T00:43:39.344 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:43:39.344 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:43:39.344 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:43:39.345 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:43:39.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.173+0000 7f2877660700 10 client.26418.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:43:39.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.173+0000 7f2877660700 15 client.26418.objecter _session_command_op_assign -1 4 2024-02-23T00:43:39.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.173+0000 7f2877660700 20 client.26418.objecter _get_session s=0x7f2870106ed0 osd=1 3 2024-02-23T00:43:39.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.173+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f2870106ed0 osd=1 3 2024-02-23T00:43:39.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.173+0000 7f2877660700 20 client.26418.objecter _get_session s=0x7f2870106ed0 osd=1 3 2024-02-23T00:43:39.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.173+0000 7f2877660700 15 client.26418.objecter _session_command_op_remove -1 4 2024-02-23T00:43:39.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.173+0000 7f2877660700 20 client.26418.objecter get_session s=0x7f2870106ed0 osd=1 3 2024-02-23T00:43:39.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.173+0000 7f2877660700 15 client.26418.objecter _session_command_op_assign 1 4 2024-02-23T00:43:39.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.173+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f2870106ed0 osd=1 4 2024-02-23T00:43:39.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.173+0000 7f2877660700 10 client.26418.objecter _send_command 4 2024-02-23T00:43:39.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.173+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f28700fff60 con 0x7f287006c2e0 2024-02-23T00:43:39.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.184+0000 7f28667fc700 1 -- 172.21.15.47:0/1296951668 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f28700fff60 con 0x7f287006c2e0 2024-02-23T00:43:39.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.184+0000 7f28667fc700 10 client.26418.objecter ms_dispatch 0x7f2870001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:43:39.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.184+0000 7f28667fc700 10 client.26418.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:43:39.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.184+0000 7f28667fc700 20 client.26418.objecter put_session s=0x7f2870106ed0 osd=1 4 2024-02-23T00:43:39.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.184+0000 7f28667fc700 15 client.26418.objecter _session_command_op_remove 1 4 2024-02-23T00:43:39.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.190+0000 7f2877660700 10 client.26418.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:39.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.190+0000 7f2877660700 15 client.26418.objecter _session_command_op_assign -1 5 2024-02-23T00:43:39.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.190+0000 7f2877660700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f2870059e00 0x7f287011e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:39.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.190+0000 7f2877660700 20 client.26418.objecter _get_session s=0x7f2870028110 osd=2 3 2024-02-23T00:43:39.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.190+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f2870028110 osd=2 3 2024-02-23T00:43:39.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.190+0000 7f2877660700 20 client.26418.objecter _get_session s=0x7f2870028110 osd=2 3 2024-02-23T00:43:39.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.190+0000 7f2877660700 15 client.26418.objecter _session_command_op_remove -1 5 2024-02-23T00:43:39.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.190+0000 7f2877660700 20 client.26418.objecter get_session s=0x7f2870028110 osd=2 3 2024-02-23T00:43:39.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.190+0000 7f2877660700 15 client.26418.objecter _session_command_op_assign 2 5 2024-02-23T00:43:39.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.190+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f2870028110 osd=2 4 2024-02-23T00:43:39.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.190+0000 7f2877660700 10 client.26418.objecter _send_command 5 2024-02-23T00:43:39.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.190+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f28701200e0 con 0x7f2870059e00 2024-02-23T00:43:39.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.190+0000 7f2874bfb700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f2870059e00 0x7f287011e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:39.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.191+0000 7f2874bfb700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f2870059e00 0x7f287011e610 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:39.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.192+0000 7f28667fc700 10 client.26418.objecter ms_handle_connect 0x7f2870059e00 2024-02-23T00:43:39.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.194+0000 7f28667fc700 1 -- 172.21.15.47:0/1296951668 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f28701200e0 con 0x7f2870059e00 2024-02-23T00:43:39.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.194+0000 7f28667fc700 10 client.26418.objecter ms_dispatch 0x7f2870001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:43:39.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.194+0000 7f28667fc700 10 client.26418.objecter _finish_command 5 = system:0 2024-02-23T00:43:39.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.194+0000 7f28667fc700 20 client.26418.objecter put_session s=0x7f2870028110 osd=2 4 2024-02-23T00:43:39.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.194+0000 7f28667fc700 15 client.26418.objecter _session_command_op_remove 2 5 2024-02-23T00:43:39.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.220+0000 7f2877660700 10 client.26418.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:43:39.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.220+0000 7f2877660700 15 client.26418.objecter _session_command_op_assign -1 6 2024-02-23T00:43:39.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.220+0000 7f2877660700 20 client.26418.objecter _get_session s=0x7f2870028110 osd=2 3 2024-02-23T00:43:39.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.220+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f2870028110 osd=2 3 2024-02-23T00:43:39.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.220+0000 7f2877660700 20 client.26418.objecter _get_session s=0x7f2870028110 osd=2 3 2024-02-23T00:43:39.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.220+0000 7f2877660700 15 client.26418.objecter _session_command_op_remove -1 6 2024-02-23T00:43:39.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.220+0000 7f2877660700 20 client.26418.objecter get_session s=0x7f2870028110 osd=2 3 2024-02-23T00:43:39.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.220+0000 7f2877660700 15 client.26418.objecter _session_command_op_assign 2 6 2024-02-23T00:43:39.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.220+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f2870028110 osd=2 4 2024-02-23T00:43:39.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.220+0000 7f2877660700 10 client.26418.objecter _send_command 6 2024-02-23T00:43:39.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.220+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f2870120a80 con 0x7f2870059e00 2024-02-23T00:43:39.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.222+0000 7f28667fc700 1 -- 172.21.15.47:0/1296951668 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f2870120a80 con 0x7f2870059e00 2024-02-23T00:43:39.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.222+0000 7f28667fc700 10 client.26418.objecter ms_dispatch 0x7f2870001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:43:39.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.222+0000 7f28667fc700 10 client.26418.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:43:39.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.222+0000 7f28667fc700 20 client.26418.objecter put_session s=0x7f2870028110 osd=2 4 2024-02-23T00:43:39.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.222+0000 7f28667fc700 15 client.26418.objecter _session_command_op_remove 2 6 2024-02-23T00:43:39.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.224+0000 7f2877660700 10 client.26418.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:39.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.224+0000 7f2877660700 15 client.26418.objecter _session_command_op_assign -1 7 2024-02-23T00:43:39.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.224+0000 7f2877660700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f287011f010 0x7f287011f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:39.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.224+0000 7f2877660700 20 client.26418.objecter _get_session s=0x7f2870120c50 osd=3 3 2024-02-23T00:43:39.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.224+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f2870120c50 osd=3 3 2024-02-23T00:43:39.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.224+0000 7f2877660700 20 client.26418.objecter _get_session s=0x7f2870120c50 osd=3 3 2024-02-23T00:43:39.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.224+0000 7f2877660700 15 client.26418.objecter _session_command_op_remove -1 7 2024-02-23T00:43:39.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.224+0000 7f2877660700 20 client.26418.objecter get_session s=0x7f2870120c50 osd=3 3 2024-02-23T00:43:39.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.224+0000 7f2877660700 15 client.26418.objecter _session_command_op_assign 3 7 2024-02-23T00:43:39.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.224+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f2870120c50 osd=3 4 2024-02-23T00:43:39.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.224+0000 7f2877660700 10 client.26418.objecter _send_command 7 2024-02-23T00:43:39.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.224+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f28700596d0 con 0x7f287011f010 2024-02-23T00:43:39.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.231+0000 7f2875bfd700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f287011f010 0x7f287011f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:39.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.232+0000 7f2875bfd700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f287011f010 0x7f287011f450 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:39.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.232+0000 7f28667fc700 10 client.26418.objecter ms_handle_connect 0x7f287011f010 2024-02-23T00:43:39.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.233+0000 7f28667fc700 1 -- 172.21.15.47:0/1296951668 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f28700596d0 con 0x7f287011f010 2024-02-23T00:43:39.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.233+0000 7f28667fc700 10 client.26418.objecter ms_dispatch 0x7f2870001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:43:39.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.233+0000 7f28667fc700 10 client.26418.objecter _finish_command 7 = system:0 2024-02-23T00:43:39.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.233+0000 7f28667fc700 20 client.26418.objecter put_session s=0x7f2870120c50 osd=3 4 2024-02-23T00:43:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.233+0000 7f28667fc700 15 client.26418.objecter _session_command_op_remove 3 7 2024-02-23T00:43:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.258+0000 7f2877660700 10 client.26418.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:43:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.258+0000 7f2877660700 15 client.26418.objecter _session_command_op_assign -1 8 2024-02-23T00:43:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.258+0000 7f2877660700 20 client.26418.objecter _get_session s=0x7f2870120c50 osd=3 3 2024-02-23T00:43:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.258+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f2870120c50 osd=3 3 2024-02-23T00:43:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.258+0000 7f2877660700 20 client.26418.objecter _get_session s=0x7f2870120c50 osd=3 3 2024-02-23T00:43:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.258+0000 7f2877660700 15 client.26418.objecter _session_command_op_remove -1 8 2024-02-23T00:43:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.258+0000 7f2877660700 20 client.26418.objecter get_session s=0x7f2870120c50 osd=3 3 2024-02-23T00:43:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.258+0000 7f2877660700 15 client.26418.objecter _session_command_op_assign 3 8 2024-02-23T00:43:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.258+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f2870120c50 osd=3 4 2024-02-23T00:43:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.258+0000 7f2877660700 10 client.26418.objecter _send_command 8 2024-02-23T00:43:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.258+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f287011fb80 con 0x7f287011f010 2024-02-23T00:43:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.262+0000 7f28667fc700 1 -- 172.21.15.47:0/1296951668 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f287011fb80 con 0x7f287011f010 2024-02-23T00:43:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.262+0000 7f28667fc700 10 client.26418.objecter ms_dispatch 0x7f2870001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:43:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.262+0000 7f28667fc700 10 client.26418.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:43:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.262+0000 7f28667fc700 20 client.26418.objecter put_session s=0x7f2870120c50 osd=3 4 2024-02-23T00:43:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.262+0000 7f28667fc700 15 client.26418.objecter _session_command_op_remove 3 8 2024-02-23T00:43:39.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.271+0000 7f2877660700 10 client.26418.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:39.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.271+0000 7f2877660700 15 client.26418.objecter _session_command_op_assign -1 9 2024-02-23T00:43:39.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.271+0000 7f2877660700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2870138340 0x7f287013a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:39.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.271+0000 7f2877660700 20 client.26418.objecter _get_session s=0x7f287011fb80 osd=4 3 2024-02-23T00:43:39.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.271+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f287011fb80 osd=4 3 2024-02-23T00:43:39.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.271+0000 7f2877660700 20 client.26418.objecter _get_session s=0x7f287011fb80 osd=4 3 2024-02-23T00:43:39.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.271+0000 7f2877660700 15 client.26418.objecter _session_command_op_remove -1 9 2024-02-23T00:43:39.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.271+0000 7f2877660700 20 client.26418.objecter get_session s=0x7f287011fb80 osd=4 3 2024-02-23T00:43:39.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.271+0000 7f2877660700 15 client.26418.objecter _session_command_op_assign 4 9 2024-02-23T00:43:39.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.271+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f287011fb80 osd=4 4 2024-02-23T00:43:39.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.271+0000 7f2877660700 10 client.26418.objecter _send_command 9 2024-02-23T00:43:39.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.271+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f287013d380 con 0x7f2870138340 2024-02-23T00:43:39.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.271+0000 7f28753fc700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2870138340 0x7f287013a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:39.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.271+0000 7f28667fc700 10 client.26418.objecter ms_handle_connect 0x7f2870138340 2024-02-23T00:43:39.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.271+0000 7f28753fc700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2870138340 0x7f287013a730 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:39.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.277+0000 7f28667fc700 1 -- 172.21.15.47:0/1296951668 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f287013d380 con 0x7f2870138340 2024-02-23T00:43:39.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.277+0000 7f28667fc700 10 client.26418.objecter ms_dispatch 0x7f2870001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:43:39.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.277+0000 7f28667fc700 10 client.26418.objecter _finish_command 9 = system:0 2024-02-23T00:43:39.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.277+0000 7f28667fc700 20 client.26418.objecter put_session s=0x7f287011fb80 osd=4 4 2024-02-23T00:43:39.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.277+0000 7f28667fc700 15 client.26418.objecter _session_command_op_remove 4 9 2024-02-23T00:43:39.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.306+0000 7f2877660700 10 client.26418.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:43:39.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.306+0000 7f2877660700 15 client.26418.objecter _session_command_op_assign -1 10 2024-02-23T00:43:39.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.306+0000 7f2877660700 20 client.26418.objecter _get_session s=0x7f287011fb80 osd=4 3 2024-02-23T00:43:39.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.306+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f287011fb80 osd=4 3 2024-02-23T00:43:39.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.306+0000 7f2877660700 20 client.26418.objecter _get_session s=0x7f287011fb80 osd=4 3 2024-02-23T00:43:39.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.306+0000 7f2877660700 15 client.26418.objecter _session_command_op_remove -1 10 2024-02-23T00:43:39.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.306+0000 7f2877660700 20 client.26418.objecter get_session s=0x7f287011fb80 osd=4 3 2024-02-23T00:43:39.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.306+0000 7f2877660700 15 client.26418.objecter _session_command_op_assign 4 10 2024-02-23T00:43:39.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.306+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f287011fb80 osd=4 4 2024-02-23T00:43:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.306+0000 7f2877660700 10 client.26418.objecter _send_command 10 2024-02-23T00:43:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.306+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f287013ac20 con 0x7f2870138340 2024-02-23T00:43:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.307+0000 7f28667fc700 1 -- 172.21.15.47:0/1296951668 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f287013ac20 con 0x7f2870138340 2024-02-23T00:43:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.307+0000 7f28667fc700 10 client.26418.objecter ms_dispatch 0x7f2870001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:43:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.307+0000 7f28667fc700 10 client.26418.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:43:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.307+0000 7f28667fc700 20 client.26418.objecter put_session s=0x7f287011fb80 osd=4 4 2024-02-23T00:43:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.307+0000 7f28667fc700 15 client.26418.objecter _session_command_op_remove 4 10 2024-02-23T00:43:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.311+0000 7f2877660700 10 client.26418.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.311+0000 7f2877660700 15 client.26418.objecter _session_command_op_assign -1 11 2024-02-23T00:43:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.311+0000 7f2877660700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f287013b2a0 0x7f287013d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.311+0000 7f2877660700 20 client.26418.objecter _get_session s=0x7f287013ac20 osd=5 3 2024-02-23T00:43:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.311+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f287013ac20 osd=5 3 2024-02-23T00:43:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.311+0000 7f2877660700 20 client.26418.objecter _get_session s=0x7f287013ac20 osd=5 3 2024-02-23T00:43:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.311+0000 7f2877660700 15 client.26418.objecter _session_command_op_remove -1 11 2024-02-23T00:43:39.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.311+0000 7f2877660700 20 client.26418.objecter get_session s=0x7f287013ac20 osd=5 3 2024-02-23T00:43:39.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.311+0000 7f2877660700 15 client.26418.objecter _session_command_op_assign 5 11 2024-02-23T00:43:39.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.311+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f287013ac20 osd=5 4 2024-02-23T00:43:39.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.311+0000 7f2877660700 10 client.26418.objecter _send_command 11 2024-02-23T00:43:39.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.311+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2870140490 con 0x7f287013b2a0 2024-02-23T00:43:39.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.311+0000 7f2874bfb700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f287013b2a0 0x7f287013d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:39.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.311+0000 7f2874bfb700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f287013b2a0 0x7f287013d690 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:39.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.311+0000 7f28667fc700 10 client.26418.objecter ms_handle_connect 0x7f287013b2a0 2024-02-23T00:43:39.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.312+0000 7f28667fc700 1 -- 172.21.15.47:0/1296951668 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2870140490 con 0x7f287013b2a0 2024-02-23T00:43:39.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.312+0000 7f28667fc700 10 client.26418.objecter ms_dispatch 0x7f2870001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:43:39.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.312+0000 7f28667fc700 10 client.26418.objecter _finish_command 11 = system:0 2024-02-23T00:43:39.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.312+0000 7f28667fc700 20 client.26418.objecter put_session s=0x7f287013ac20 osd=5 4 2024-02-23T00:43:39.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.313+0000 7f28667fc700 15 client.26418.objecter _session_command_op_remove 5 11 2024-02-23T00:43:39.355 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:43:39.355 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:43:39.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.341+0000 7f2877660700 10 client.26418.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:43:39.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.341+0000 7f2877660700 15 client.26418.objecter _session_command_op_assign -1 12 2024-02-23T00:43:39.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.341+0000 7f2877660700 20 client.26418.objecter _get_session s=0x7f287013ac20 osd=5 3 2024-02-23T00:43:39.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.341+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f287013ac20 osd=5 3 2024-02-23T00:43:39.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.341+0000 7f2877660700 20 client.26418.objecter _get_session s=0x7f287013ac20 osd=5 3 2024-02-23T00:43:39.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.341+0000 7f2877660700 15 client.26418.objecter _session_command_op_remove -1 12 2024-02-23T00:43:39.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.341+0000 7f2877660700 20 client.26418.objecter get_session s=0x7f287013ac20 osd=5 3 2024-02-23T00:43:39.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.341+0000 7f2877660700 15 client.26418.objecter _session_command_op_assign 5 12 2024-02-23T00:43:39.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.341+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f287013ac20 osd=5 4 2024-02-23T00:43:39.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.341+0000 7f2877660700 10 client.26418.objecter _send_command 12 2024-02-23T00:43:39.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.341+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f2870140490 con 0x7f287013b2a0 2024-02-23T00:43:39.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.344+0000 7f28667fc700 1 -- 172.21.15.47:0/1296951668 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f2870140490 con 0x7f287013b2a0 2024-02-23T00:43:39.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.344+0000 7f28667fc700 10 client.26418.objecter ms_dispatch 0x7f2870001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:43:39.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.344+0000 7f28667fc700 10 client.26418.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:43:39.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.344+0000 7f28667fc700 20 client.26418.objecter put_session s=0x7f287013ac20 osd=5 4 2024-02-23T00:43:39.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.344+0000 7f28667fc700 15 client.26418.objecter _session_command_op_remove 5 12 2024-02-23T00:43:39.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.346+0000 7f2877660700 10 client.26418.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:39.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.346+0000 7f2877660700 15 client.26418.objecter _session_command_op_assign -1 13 2024-02-23T00:43:39.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.346+0000 7f2877660700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f287013f7e0 0x7f287013fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:39.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.346+0000 7f2877660700 20 client.26418.objecter _get_session s=0x7f2870140490 osd=6 3 2024-02-23T00:43:39.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.346+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f2870140490 osd=6 3 2024-02-23T00:43:39.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.346+0000 7f2877660700 20 client.26418.objecter _get_session s=0x7f2870140490 osd=6 3 2024-02-23T00:43:39.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.346+0000 7f2877660700 15 client.26418.objecter _session_command_op_remove -1 13 2024-02-23T00:43:39.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.346+0000 7f2877660700 20 client.26418.objecter get_session s=0x7f2870140490 osd=6 3 2024-02-23T00:43:39.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.346+0000 7f2877660700 15 client.26418.objecter _session_command_op_assign 6 13 2024-02-23T00:43:39.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.346+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f2870140490 osd=6 4 2024-02-23T00:43:39.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.346+0000 7f2877660700 10 client.26418.objecter _send_command 13 2024-02-23T00:43:39.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.346+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f28701437c0 con 0x7f287013f7e0 2024-02-23T00:43:39.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.346+0000 7f2875bfd700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f287013f7e0 0x7f287013fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:39.357 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:39.358 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:43:39.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.347+0000 7f2875bfd700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f287013f7e0 0x7f287013fc40 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:39.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.347+0000 7f28667fc700 10 client.26418.objecter ms_handle_connect 0x7f287013f7e0 2024-02-23T00:43:39.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.347+0000 7f28667fc700 1 -- 172.21.15.47:0/1296951668 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f28701437c0 con 0x7f287013f7e0 2024-02-23T00:43:39.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.347+0000 7f28667fc700 10 client.26418.objecter ms_dispatch 0x7f2870001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:43:39.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.347+0000 7f28667fc700 10 client.26418.objecter _finish_command 13 = system:0 2024-02-23T00:43:39.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.347+0000 7f28667fc700 20 client.26418.objecter put_session s=0x7f2870140490 osd=6 4 2024-02-23T00:43:39.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.347+0000 7f28667fc700 15 client.26418.objecter _session_command_op_remove 6 13 2024-02-23T00:43:39.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.369+0000 7f2877660700 10 client.26418.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:43:39.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.369+0000 7f2877660700 15 client.26418.objecter _session_command_op_assign -1 14 2024-02-23T00:43:39.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.369+0000 7f2877660700 20 client.26418.objecter _get_session s=0x7f2870140490 osd=6 3 2024-02-23T00:43:39.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.369+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f2870140490 osd=6 3 2024-02-23T00:43:39.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.369+0000 7f2877660700 20 client.26418.objecter _get_session s=0x7f2870140490 osd=6 3 2024-02-23T00:43:39.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.369+0000 7f2877660700 15 client.26418.objecter _session_command_op_remove -1 14 2024-02-23T00:43:39.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.369+0000 7f2877660700 20 client.26418.objecter get_session s=0x7f2870140490 osd=6 3 2024-02-23T00:43:39.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.369+0000 7f2877660700 15 client.26418.objecter _session_command_op_assign 6 14 2024-02-23T00:43:39.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.369+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f2870140490 osd=6 4 2024-02-23T00:43:39.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.369+0000 7f2877660700 10 client.26418.objecter _send_command 14 2024-02-23T00:43:39.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.369+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f287013dc70 con 0x7f287013f7e0 2024-02-23T00:43:39.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.374+0000 7f28667fc700 1 -- 172.21.15.47:0/1296951668 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f287013dc70 con 0x7f287013f7e0 2024-02-23T00:43:39.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.374+0000 7f28667fc700 10 client.26418.objecter ms_dispatch 0x7f2870001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:43:39.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.374+0000 7f28667fc700 10 client.26418.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:43:39.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.374+0000 7f28667fc700 20 client.26418.objecter put_session s=0x7f2870140490 osd=6 4 2024-02-23T00:43:39.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.374+0000 7f28667fc700 15 client.26418.objecter _session_command_op_remove 6 14 2024-02-23T00:43:39.382 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:43:39.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.380+0000 7f2877660700 10 client.26418.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:39.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.380+0000 7f2877660700 15 client.26418.objecter _session_command_op_assign -1 15 2024-02-23T00:43:39.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.380+0000 7f2877660700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f287013e2f0 0x7f287013e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:39.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.380+0000 7f2877660700 20 client.26418.objecter _get_session s=0x7f287013dc70 osd=7 3 2024-02-23T00:43:39.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.380+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f287013dc70 osd=7 3 2024-02-23T00:43:39.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.380+0000 7f2877660700 20 client.26418.objecter _get_session s=0x7f287013dc70 osd=7 3 2024-02-23T00:43:39.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.380+0000 7f2877660700 15 client.26418.objecter _session_command_op_remove -1 15 2024-02-23T00:43:39.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.380+0000 7f2877660700 20 client.26418.objecter get_session s=0x7f287013dc70 osd=7 3 2024-02-23T00:43:39.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.380+0000 7f2877660700 15 client.26418.objecter _session_command_op_assign 7 15 2024-02-23T00:43:39.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.380+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f287013dc70 osd=7 4 2024-02-23T00:43:39.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.380+0000 7f2877660700 10 client.26418.objecter _send_command 15 2024-02-23T00:43:39.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.380+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f287013ee10 con 0x7f287013e2f0 2024-02-23T00:43:39.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.380+0000 7f28753fc700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f287013e2f0 0x7f287013e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:39.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.381+0000 7f28753fc700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f287013e2f0 0x7f287013e730 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:39.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.381+0000 7f28667fc700 10 client.26418.objecter ms_handle_connect 0x7f287013e2f0 2024-02-23T00:43:39.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.383+0000 7f28667fc700 1 -- 172.21.15.47:0/1296951668 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f287013ee10 con 0x7f287013e2f0 2024-02-23T00:43:39.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.383+0000 7f28667fc700 10 client.26418.objecter ms_dispatch 0x7f2870001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:43:39.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.383+0000 7f28667fc700 10 client.26418.objecter _finish_command 15 = system:0 2024-02-23T00:43:39.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.383+0000 7f28667fc700 20 client.26418.objecter put_session s=0x7f287013dc70 osd=7 4 2024-02-23T00:43:39.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.383+0000 7f28667fc700 15 client.26418.objecter _session_command_op_remove 7 15 2024-02-23T00:43:39.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.419+0000 7f2877660700 10 client.26418.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:43:39.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.419+0000 7f2877660700 15 client.26418.objecter _session_command_op_assign -1 16 2024-02-23T00:43:39.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.419+0000 7f2877660700 20 client.26418.objecter _get_session s=0x7f287013dc70 osd=7 3 2024-02-23T00:43:39.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.419+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f287013dc70 osd=7 3 2024-02-23T00:43:39.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.419+0000 7f2877660700 20 client.26418.objecter _get_session s=0x7f287013dc70 osd=7 3 2024-02-23T00:43:39.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.419+0000 7f2877660700 15 client.26418.objecter _session_command_op_remove -1 16 2024-02-23T00:43:39.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.419+0000 7f2877660700 20 client.26418.objecter get_session s=0x7f287013dc70 osd=7 3 2024-02-23T00:43:39.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.419+0000 7f2877660700 15 client.26418.objecter _session_command_op_assign 7 16 2024-02-23T00:43:39.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.419+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f287013dc70 osd=7 4 2024-02-23T00:43:39.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.419+0000 7f2877660700 10 client.26418.objecter _send_command 16 2024-02-23T00:43:39.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.419+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f287013ee10 con 0x7f287013e2f0 2024-02-23T00:43:39.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.423+0000 7f28667fc700 1 -- 172.21.15.47:0/1296951668 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f287013ee10 con 0x7f287013e2f0 2024-02-23T00:43:39.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.423+0000 7f28667fc700 10 client.26418.objecter ms_dispatch 0x7f2870001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:43:39.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.423+0000 7f28667fc700 10 client.26418.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:43:39.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.423+0000 7f28667fc700 20 client.26418.objecter put_session s=0x7f287013dc70 osd=7 4 2024-02-23T00:43:39.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.423+0000 7f28667fc700 15 client.26418.objecter _session_command_op_remove 7 16 2024-02-23T00:43:39.441 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:43:39.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.423+0000 7f2877660700 10 librados: watch_flush enter 2024-02-23T00:43:39.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 10 librados: watch_flush exit 2024-02-23T00:43:39.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 10 client.26418.objecter close_session for osd.0 2024-02-23T00:43:39.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f28700273c0 msgr2=0x7f28700277a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:39.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f28700273c0 0x7f28700277a0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:39.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f2870060280 osd=0 1 2024-02-23T00:43:39.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 10 client.26418.objecter close_session for osd.1 2024-02-23T00:43:39.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f287006c2e0 msgr2=0x7f28700fe8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:39.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f287006c2e0 0x7f28700fe8f0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:39.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f2870106ed0 osd=1 1 2024-02-23T00:43:39.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 10 client.26418.objecter close_session for osd.2 2024-02-23T00:43:39.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f2870059e00 msgr2=0x7f287011e610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:39.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f2870059e00 0x7f287011e610 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:39.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f2870028110 osd=2 1 2024-02-23T00:43:39.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 10 client.26418.objecter close_session for osd.3 2024-02-23T00:43:39.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f287011f010 msgr2=0x7f287011f450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:39.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f287011f010 0x7f287011f450 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:39.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f2870120c50 osd=3 1 2024-02-23T00:43:39.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 10 client.26418.objecter close_session for osd.4 2024-02-23T00:43:39.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2870138340 msgr2=0x7f287013a730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:39.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2870138340 0x7f287013a730 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:39.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f287011fb80 osd=4 1 2024-02-23T00:43:39.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 10 client.26418.objecter close_session for osd.5 2024-02-23T00:43:39.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f287013b2a0 msgr2=0x7f287013d690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:39.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f287013b2a0 0x7f287013d690 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:39.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f287013ac20 osd=5 1 2024-02-23T00:43:39.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 10 client.26418.objecter close_session for osd.6 2024-02-23T00:43:39.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f287013f7e0 msgr2=0x7f287013fc40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:39.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f287013f7e0 0x7f287013fc40 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:39.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f2870140490 osd=6 1 2024-02-23T00:43:39.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 10 client.26418.objecter close_session for osd.7 2024-02-23T00:43:39.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f287013e2f0 msgr2=0x7f287013e730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:39.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f287013e2f0 0x7f287013e730 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:39.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 20 client.26418.objecter put_session s=0x7f287013dc70 osd=7 1 2024-02-23T00:43:39.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 20 client.26418.objecter shutdown clearing up homeless session... 2024-02-23T00:43:39.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 10 client.26418.objecter successfully canceled tick 2024-02-23T00:43:39.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f285c062d10 msgr2=0x7f285c0651d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:39.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f285c062d10 0x7f285c0651d0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 rx=0x7f28680012a0 tx=0x7f2868004d10).stop 2024-02-23T00:43:39.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28700fe510 msgr2=0x7f28700632a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:39.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.440+0000 7f2877660700 1 --2- 172.21.15.47:0/1296951668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28700fe510 0x7f28700632a0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 rx=0x7f286000b460 tx=0x7f2860006f20).stop 2024-02-23T00:43:39.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.444+0000 7f2875bfd700 1 -- 172.21.15.47:0/1296951668 reap_dead start 2024-02-23T00:43:39.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.444+0000 7f2875bfd700 1 -- 172.21.15.47:0/1296951668 reap_dead start 2024-02-23T00:43:39.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.444+0000 7f2875bfd700 1 -- 172.21.15.47:0/1296951668 reap_dead start 2024-02-23T00:43:39.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.444+0000 7f2875bfd700 1 -- 172.21.15.47:0/1296951668 reap_dead start 2024-02-23T00:43:39.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.444+0000 7f2875bfd700 1 -- 172.21.15.47:0/1296951668 reap_dead start 2024-02-23T00:43:39.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.444+0000 7f2875bfd700 1 -- 172.21.15.47:0/1296951668 reap_dead start 2024-02-23T00:43:39.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.444+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 shutdown_connections 2024-02-23T00:43:39.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.444+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 >> 172.21.15.47:0/1296951668 conn(0x7f28700f3800 msgr2=0x7f28700fc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:43:39.459 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:43:39.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.458+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 shutdown_connections 2024-02-23T00:43:39.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.459+0000 7f2877660700 1 -- 172.21.15.47:0/1296951668 wait complete. 2024-02-23T00:43:39.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:39.465+0000 7f2877660700 1 librados: shutdown 2024-02-23T00:43:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:43:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:43:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:43:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:43:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:43:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:43:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:43:39.474 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:43:39.599 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:39.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:37.803377+0000 osd.7 (osd.7) 87 : cluster [DBG] 20.7 deep-scrub starts 2024-02-23T00:43:39.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:37.810164+0000 osd.7 (osd.7) 88 : cluster [DBG] 20.7 deep-scrub ok 2024-02-23T00:43:39.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:38.379848+0000 mon.a (mon.0) 1150 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:43:39.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:38.387470+0000 mon.a (mon.0) 1151 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi047-82386-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:39.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:38.387755+0000 mon.a (mon.0) 1152 : cluster [DBG] osdmap e146: 8 total, 8 up, 8 in 2024-02-23T00:43:39.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:38.397990+0000 mon.b (mon.1) 161 : audit [INF] from='client.? 172.21.15.47:0/2559316827' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82478-10", "tierpool":"test-rados-api-smithi047-82478-10-cache", "force_nonempty":""}]: dispatch 2024-02-23T00:43:39.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:38.407683+0000 mon.b (mon.1) 162 : audit [INF] from='client.? 172.21.15.47:0/1110018035' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi047-82378-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:39.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:38.413887+0000 mon.a (mon.0) 1153 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82478-10", "tierpool":"test-rados-api-smithi047-82478-10-cache", "force_nonempty":""}]: dispatch 2024-02-23T00:43:39.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:38.414195+0000 mon.a (mon.0) 1154 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi047-82378-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:39.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:38.490378+0000 mon.a (mon.0) 1155 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:43:39.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:38.673894+0000 mon.b (mon.1) 163 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:39.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:38.935005+0000 osd.5 (osd.5) 79 : cluster [DBG] 2.16 scrub starts 2024-02-23T00:43:39.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:38.947671+0000 osd.5 (osd.5) 80 : cluster [DBG] 2.16 scrub ok 2024-02-23T00:43:39.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:39.084179+0000 osd.3 (osd.3) 93 : cluster [DBG] 5.14 scrub starts 2024-02-23T00:43:39.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:39.086103+0000 osd.3 (osd.3) 94 : cluster [DBG] 5.14 scrub ok 2024-02-23T00:43:39.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:39.110784+0000 mon.a (mon.0) 1156 : audit [DBG] from='client.? 172.21.15.47:0/1296951668' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:43:39.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:39.205567+0000 mon.a (mon.0) 1157 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "23"}]: dispatch 2024-02-23T00:43:39.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:39.392650+0000 mon.a (mon.0) 1158 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82478-10", "tierpool":"test-rados-api-smithi047-82478-10-cache", "force_nonempty":""}]': finished 2024-02-23T00:43:39.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:39.392730+0000 mon.a (mon.0) 1159 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi047-82378-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:39.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:39.392903+0000 mon.a (mon.0) 1160 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:43:39.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:39.392962+0000 mon.a (mon.0) 1161 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "23"}]': finished 2024-02-23T00:43:39.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:39.393002+0000 mon.a (mon.0) 1162 : cluster [DBG] osdmap e147: 8 total, 8 up, 8 in 2024-02-23T00:43:39.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:37.803377+0000 osd.7 (osd.7) 87 : cluster [DBG] 20.7 deep-scrub starts 2024-02-23T00:43:39.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:37.810164+0000 osd.7 (osd.7) 88 : cluster [DBG] 20.7 deep-scrub ok 2024-02-23T00:43:39.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:38.379848+0000 mon.a (mon.0) 1150 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:43:39.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:38.387470+0000 mon.a (mon.0) 1151 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi047-82386-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:39.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:38.387755+0000 mon.a (mon.0) 1152 : cluster [DBG] osdmap e146: 8 total, 8 up, 8 in 2024-02-23T00:43:39.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:38.397990+0000 mon.b (mon.1) 161 : audit [INF] from='client.? 172.21.15.47:0/2559316827' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82478-10", "tierpool":"test-rados-api-smithi047-82478-10-cache", "force_nonempty":""}]: dispatch 2024-02-23T00:43:39.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:38.407683+0000 mon.b (mon.1) 162 : audit [INF] from='client.? 172.21.15.47:0/1110018035' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi047-82378-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:39.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:38.413887+0000 mon.a (mon.0) 1153 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82478-10", "tierpool":"test-rados-api-smithi047-82478-10-cache", "force_nonempty":""}]: dispatch 2024-02-23T00:43:39.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:38.414195+0000 mon.a (mon.0) 1154 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi047-82378-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:39.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:38.490378+0000 mon.a (mon.0) 1155 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:43:39.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:38.673894+0000 mon.b (mon.1) 163 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:39.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:38.935005+0000 osd.5 (osd.5) 79 : cluster [DBG] 2.16 scrub starts 2024-02-23T00:43:39.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:38.947671+0000 osd.5 (osd.5) 80 : cluster [DBG] 2.16 scrub ok 2024-02-23T00:43:39.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:39.084179+0000 osd.3 (osd.3) 93 : cluster [DBG] 5.14 scrub starts 2024-02-23T00:43:39.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:39.086103+0000 osd.3 (osd.3) 94 : cluster [DBG] 5.14 scrub ok 2024-02-23T00:43:39.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:39.110784+0000 mon.a (mon.0) 1156 : audit [DBG] from='client.? 172.21.15.47:0/1296951668' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:43:39.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:39.205567+0000 mon.a (mon.0) 1157 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "23"}]: dispatch 2024-02-23T00:43:39.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:39.392650+0000 mon.a (mon.0) 1158 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82478-10", "tierpool":"test-rados-api-smithi047-82478-10-cache", "force_nonempty":""}]': finished 2024-02-23T00:43:39.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:39.392730+0000 mon.a (mon.0) 1159 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi047-82378-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:39.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:39.392903+0000 mon.a (mon.0) 1160 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:43:39.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:39.392962+0000 mon.a (mon.0) 1161 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "23"}]': finished 2024-02-23T00:43:39.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:39.393002+0000 mon.a (mon.0) 1162 : cluster [DBG] osdmap e147: 8 total, 8 up, 8 in 2024-02-23T00:43:39.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:37.803377+0000 osd.7 (osd.7) 87 : cluster [DBG] 20.7 deep-scrub starts 2024-02-23T00:43:39.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:37.810164+0000 osd.7 (osd.7) 88 : cluster [DBG] 20.7 deep-scrub ok 2024-02-23T00:43:39.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:38.379848+0000 mon.a (mon.0) 1150 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:43:39.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:38.387470+0000 mon.a (mon.0) 1151 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi047-82386-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:39.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:38.387755+0000 mon.a (mon.0) 1152 : cluster [DBG] osdmap e146: 8 total, 8 up, 8 in 2024-02-23T00:43:39.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:38.397990+0000 mon.b (mon.1) 161 : audit [INF] from='client.? 172.21.15.47:0/2559316827' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82478-10", "tierpool":"test-rados-api-smithi047-82478-10-cache", "force_nonempty":""}]: dispatch 2024-02-23T00:43:39.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:38.407683+0000 mon.b (mon.1) 162 : audit [INF] from='client.? 172.21.15.47:0/1110018035' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi047-82378-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:39.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:38.413887+0000 mon.a (mon.0) 1153 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82478-10", "tierpool":"test-rados-api-smithi047-82478-10-cache", "force_nonempty":""}]: dispatch 2024-02-23T00:43:39.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:38.414195+0000 mon.a (mon.0) 1154 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi047-82378-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:39.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:38.490378+0000 mon.a (mon.0) 1155 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:43:39.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:38.673894+0000 mon.b (mon.1) 163 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:39.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:38.935005+0000 osd.5 (osd.5) 79 : cluster [DBG] 2.16 scrub starts 2024-02-23T00:43:39.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:38.947671+0000 osd.5 (osd.5) 80 : cluster [DBG] 2.16 scrub ok 2024-02-23T00:43:39.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:39.084179+0000 osd.3 (osd.3) 93 : cluster [DBG] 5.14 scrub starts 2024-02-23T00:43:39.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:39.086103+0000 osd.3 (osd.3) 94 : cluster [DBG] 5.14 scrub ok 2024-02-23T00:43:39.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:39.110784+0000 mon.a (mon.0) 1156 : audit [DBG] from='client.? 172.21.15.47:0/1296951668' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:43:39.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:39.205567+0000 mon.a (mon.0) 1157 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "23"}]: dispatch 2024-02-23T00:43:39.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:39.392650+0000 mon.a (mon.0) 1158 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82478-10", "tierpool":"test-rados-api-smithi047-82478-10-cache", "force_nonempty":""}]': finished 2024-02-23T00:43:39.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:39.392730+0000 mon.a (mon.0) 1159 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi047-82378-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:39.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:39.392903+0000 mon.a (mon.0) 1160 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:43:39.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:39.392962+0000 mon.a (mon.0) 1161 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "23"}]': finished 2024-02-23T00:43:39.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:39.393002+0000 mon.a (mon.0) 1162 : cluster [DBG] osdmap e147: 8 total, 8 up, 8 in 2024-02-23T00:43:40.322 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:43:40.326 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:40.326 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:43:40.367 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.3 dump_historic_ops 2024-02-23T00:43:40.428 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:43:40.454 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:40.475 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:43:40.477 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:40.477 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:43:40.578 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:43:40.605 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:39.203662+0000 mgr.y (mgr.14517) 195 : cluster [DBG] pgmap v195: 484 pgs: 64 creating+peering, 1 active+clean+snaptrim, 96 unknown, 1 peering, 322 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 17 MiB/s wr, 5 op/s 2024-02-23T00:43:40.605 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:39.279257+0000 osd.0 (osd.0) 79 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:43:40.605 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:39.279353+0000 osd.0 (osd.0) 80 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:43:40.605 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:39.401328+0000 mon.b (mon.1) 164 : audit [INF] from='client.? 172.21.15.47:0/2559316827' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82478-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:40.605 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:39.403522+0000 mon.a (mon.0) 1163 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82478-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:40.605 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:39.403803+0000 mon.a (mon.0) 1164 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-11"}]: dispatch 2024-02-23T00:43:40.605 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:39.675406+0000 mon.b (mon.1) 165 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:40.605 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:40.060121+0000 osd.3 (osd.3) 95 : cluster [DBG] 6.6 scrub starts 2024-02-23T00:43:40.605 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:40.062486+0000 osd.3 (osd.3) 96 : cluster [DBG] 6.6 scrub ok 2024-02-23T00:43:40.605 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:40.098536+0000 mon.a (mon.0) 1165 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82478-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:40.605 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:40.098622+0000 mon.a (mon.0) 1166 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-11"}]': finished 2024-02-23T00:43:40.605 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:40.099652+0000 mon.a (mon.0) 1167 : cluster [DBG] osdmap e148: 8 total, 8 up, 8 in 2024-02-23T00:43:40.606 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:40.101537+0000 mon.a (mon.0) 1168 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-11", "mode": "writeback"}]: dispatch 2024-02-23T00:43:40.606 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:40.109588+0000 mon.b (mon.1) 166 : audit [INF] from='client.? 172.21.15.47:0/288292886' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi047-82386-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:40.606 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:40.110674+0000 mon.a (mon.0) 1169 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi047-82386-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:40.606 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:40.114323+0000 mon.b (mon.1) 167 : audit [INF] from='client.? 172.21.15.47:0/2559316827' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82478-10", "tierpool":"test-rados-api-smithi047-82478-10-cache"}]: dispatch 2024-02-23T00:43:40.606 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:40.115760+0000 mon.a (mon.0) 1170 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82478-10", "tierpool":"test-rados-api-smithi047-82478-10-cache"}]: dispatch 2024-02-23T00:43:40.606 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:43:40] "GET /metrics HTTP/1.1" 200 206383 "" "Prometheus/2.33.4" 2024-02-23T00:43:40.606 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:39.203662+0000 mgr.y (mgr.14517) 195 : cluster [DBG] pgmap v195: 484 pgs: 64 creating+peering, 1 active+clean+snaptrim, 96 unknown, 1 peering, 322 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 17 MiB/s wr, 5 op/s 2024-02-23T00:43:40.607 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:39.279257+0000 osd.0 (osd.0) 79 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:43:40.607 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:39.279353+0000 osd.0 (osd.0) 80 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:43:40.607 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:39.401328+0000 mon.b (mon.1) 164 : audit [INF] from='client.? 172.21.15.47:0/2559316827' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82478-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:40.607 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:39.403522+0000 mon.a (mon.0) 1163 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82478-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:40.607 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:39.403803+0000 mon.a (mon.0) 1164 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-11"}]: dispatch 2024-02-23T00:43:40.607 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:39.675406+0000 mon.b (mon.1) 165 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:40.607 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:40.060121+0000 osd.3 (osd.3) 95 : cluster [DBG] 6.6 scrub starts 2024-02-23T00:43:40.607 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:40.062486+0000 osd.3 (osd.3) 96 : cluster [DBG] 6.6 scrub ok 2024-02-23T00:43:40.607 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:40.098536+0000 mon.a (mon.0) 1165 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82478-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:40.607 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:40.098622+0000 mon.a (mon.0) 1166 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-11"}]': finished 2024-02-23T00:43:40.607 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:40.099652+0000 mon.a (mon.0) 1167 : cluster [DBG] osdmap e148: 8 total, 8 up, 8 in 2024-02-23T00:43:40.607 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:40.101537+0000 mon.a (mon.0) 1168 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-11", "mode": "writeback"}]: dispatch 2024-02-23T00:43:40.607 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:40.109588+0000 mon.b (mon.1) 166 : audit [INF] from='client.? 172.21.15.47:0/288292886' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi047-82386-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:40.607 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:40.110674+0000 mon.a (mon.0) 1169 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi047-82386-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:40.608 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:40.114323+0000 mon.b (mon.1) 167 : audit [INF] from='client.? 172.21.15.47:0/2559316827' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82478-10", "tierpool":"test-rados-api-smithi047-82478-10-cache"}]: dispatch 2024-02-23T00:43:40.608 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:40.115760+0000 mon.a (mon.0) 1170 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82478-10", "tierpool":"test-rados-api-smithi047-82478-10-cache"}]: dispatch 2024-02-23T00:43:40.647 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:40.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:39.203662+0000 mgr.y (mgr.14517) 195 : cluster [DBG] pgmap v195: 484 pgs: 64 creating+peering, 1 active+clean+snaptrim, 96 unknown, 1 peering, 322 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 17 MiB/s wr, 5 op/s 2024-02-23T00:43:40.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:39.279257+0000 osd.0 (osd.0) 79 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:43:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:39.279353+0000 osd.0 (osd.0) 80 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:43:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:39.401328+0000 mon.b (mon.1) 164 : audit [INF] from='client.? 172.21.15.47:0/2559316827' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82478-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:39.403522+0000 mon.a (mon.0) 1163 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82478-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:39.403803+0000 mon.a (mon.0) 1164 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-11"}]: dispatch 2024-02-23T00:43:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:39.675406+0000 mon.b (mon.1) 165 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:40.060121+0000 osd.3 (osd.3) 95 : cluster [DBG] 6.6 scrub starts 2024-02-23T00:43:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:40.062486+0000 osd.3 (osd.3) 96 : cluster [DBG] 6.6 scrub ok 2024-02-23T00:43:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:40.098536+0000 mon.a (mon.0) 1165 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82478-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:40.098622+0000 mon.a (mon.0) 1166 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-11"}]': finished 2024-02-23T00:43:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:40.099652+0000 mon.a (mon.0) 1167 : cluster [DBG] osdmap e148: 8 total, 8 up, 8 in 2024-02-23T00:43:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:40.101537+0000 mon.a (mon.0) 1168 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-11", "mode": "writeback"}]: dispatch 2024-02-23T00:43:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:40.109588+0000 mon.b (mon.1) 166 : audit [INF] from='client.? 172.21.15.47:0/288292886' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi047-82386-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:40.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:40.110674+0000 mon.a (mon.0) 1169 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi047-82386-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:40.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:40.114323+0000 mon.b (mon.1) 167 : audit [INF] from='client.? 172.21.15.47:0/2559316827' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82478-10", "tierpool":"test-rados-api-smithi047-82478-10-cache"}]: dispatch 2024-02-23T00:43:40.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:40.115760+0000 mon.a (mon.0) 1170 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82478-10", "tierpool":"test-rados-api-smithi047-82478-10-cache"}]: dispatch 2024-02-23T00:43:41.437 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:43:41.441 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:41.442 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:43:41.543 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:43:41.715 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:42.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:40.687382+0000 mon.b (mon.1) 168 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:42.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:40.958373+0000 osd.5 (osd.5) 81 : cluster [DBG] 5.16 scrub starts 2024-02-23T00:43:42.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:40.971240+0000 osd.5 (osd.5) 82 : cluster [DBG] 5.16 scrub ok 2024-02-23T00:43:42.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:41.095859+0000 mon.a (mon.0) 1171 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:43:42.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:41.103648+0000 mon.a (mon.0) 1172 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-11", "mode": "writeback"}]': finished 2024-02-23T00:43:42.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:41.103784+0000 mon.a (mon.0) 1173 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi047-82386-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:42.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:41.103855+0000 mon.a (mon.0) 1174 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82478-10", "tierpool":"test-rados-api-smithi047-82478-10-cache"}]': finished 2024-02-23T00:43:42.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:41.103954+0000 mon.a (mon.0) 1175 : cluster [DBG] osdmap e149: 8 total, 8 up, 8 in 2024-02-23T00:43:42.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:40.687382+0000 mon.b (mon.1) 168 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:42.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:40.958373+0000 osd.5 (osd.5) 81 : cluster [DBG] 5.16 scrub starts 2024-02-23T00:43:42.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:40.971240+0000 osd.5 (osd.5) 82 : cluster [DBG] 5.16 scrub ok 2024-02-23T00:43:42.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:41.095859+0000 mon.a (mon.0) 1171 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:43:42.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:41.103648+0000 mon.a (mon.0) 1172 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-11", "mode": "writeback"}]': finished 2024-02-23T00:43:42.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:41.103784+0000 mon.a (mon.0) 1173 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi047-82386-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:42.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:41.103855+0000 mon.a (mon.0) 1174 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82478-10", "tierpool":"test-rados-api-smithi047-82478-10-cache"}]': finished 2024-02-23T00:43:42.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:41.103954+0000 mon.a (mon.0) 1175 : cluster [DBG] osdmap e149: 8 total, 8 up, 8 in 2024-02-23T00:43:42.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:40.687382+0000 mon.b (mon.1) 168 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:42.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:40.958373+0000 osd.5 (osd.5) 81 : cluster [DBG] 5.16 scrub starts 2024-02-23T00:43:42.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:40.971240+0000 osd.5 (osd.5) 82 : cluster [DBG] 5.16 scrub ok 2024-02-23T00:43:42.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:41.095859+0000 mon.a (mon.0) 1171 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:43:42.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:41.103648+0000 mon.a (mon.0) 1172 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-11", "mode": "writeback"}]': finished 2024-02-23T00:43:42.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:41.103784+0000 mon.a (mon.0) 1173 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi047-82386-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:42.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:41.103855+0000 mon.a (mon.0) 1174 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82478-10", "tierpool":"test-rados-api-smithi047-82478-10-cache"}]': finished 2024-02-23T00:43:42.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:41.103954+0000 mon.a (mon.0) 1175 : cluster [DBG] osdmap e149: 8 total, 8 up, 8 in 2024-02-23T00:43:42.489 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-23T00:43:42.489 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight 0 1 2024-02-23T00:43:42.693 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:43:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:43:42.693 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:43:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:43:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:43:42.832 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:43:42.840 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:42.840 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:43:42.941 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:43:42.967 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:42.987 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:43:42.990 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:42.990 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:43:43.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:40.431691+0000 osd.6 (osd.6) 67 : cluster [DBG] 20.8 deep-scrub starts 2024-02-23T00:43:43.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:40.432734+0000 osd.6 (osd.6) 68 : cluster [DBG] 20.8 deep-scrub ok 2024-02-23T00:43:43.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:40.771228+0000 osd.7 (osd.7) 89 : cluster [DBG] 20.9 deep-scrub starts 2024-02-23T00:43:43.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:40.777584+0000 osd.7 (osd.7) 90 : cluster [DBG] 20.9 deep-scrub ok 2024-02-23T00:43:43.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:41.207671+0000 mgr.y (mgr.14517) 196 : cluster [DBG] pgmap v199: 451 pgs: 64 creating+peering, 1 active+clean+snaptrim, 64 unknown, 1 peering, 321 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:43:43.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:41.689303+0000 mon.b (mon.1) 169 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:43.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:41.778105+0000 mon.a (mon.0) 1176 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "22"}]: dispatch 2024-02-23T00:43:43.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:41.814495+0000 osd.7 (osd.7) 91 : cluster [DBG] 5.1a scrub starts 2024-02-23T00:43:43.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:41.817383+0000 osd.7 (osd.7) 92 : cluster [DBG] 5.1a scrub ok 2024-02-23T00:43:43.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:42.118114+0000 mon.a (mon.0) 1177 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "22"}]': finished 2024-02-23T00:43:43.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:42.118182+0000 mon.a (mon.0) 1178 : cluster [DBG] osdmap e150: 8 total, 8 up, 8 in 2024-02-23T00:43:43.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:42.120318+0000 mon.b (mon.1) 170 : audit [INF] from='client.? 172.21.15.47:0/1361361966' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi047-82487-9"}]: dispatch 2024-02-23T00:43:43.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:42.132715+0000 mon.a (mon.0) 1179 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi047-82487-9"}]: dispatch 2024-02-23T00:43:43.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:42.156882+0000 mon.b (mon.1) 171 : audit [INF] from='client.? 172.21.15.47:0/2824343318' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi047-82378-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:43.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:42.186434+0000 mon.a (mon.0) 1180 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi047-82378-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:43.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:42.318352+0000 osd.2 (osd.2) 71 : cluster [DBG] 2.e scrub starts 2024-02-23T00:43:43.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:42.320340+0000 osd.2 (osd.2) 72 : cluster [DBG] 2.e scrub ok 2024-02-23T00:43:43.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:42.339513+0000 mgr.y (mgr.14517) 197 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:43:43.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:42.692803+0000 mon.b (mon.1) 172 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:43.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:42.826037+0000 mon.a (mon.0) 1181 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:43:43.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:40.431691+0000 osd.6 (osd.6) 67 : cluster [DBG] 20.8 deep-scrub starts 2024-02-23T00:43:43.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:40.432734+0000 osd.6 (osd.6) 68 : cluster [DBG] 20.8 deep-scrub ok 2024-02-23T00:43:43.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:40.771228+0000 osd.7 (osd.7) 89 : cluster [DBG] 20.9 deep-scrub starts 2024-02-23T00:43:43.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:40.777584+0000 osd.7 (osd.7) 90 : cluster [DBG] 20.9 deep-scrub ok 2024-02-23T00:43:43.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:41.207671+0000 mgr.y (mgr.14517) 196 : cluster [DBG] pgmap v199: 451 pgs: 64 creating+peering, 1 active+clean+snaptrim, 64 unknown, 1 peering, 321 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:43:43.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:41.689303+0000 mon.b (mon.1) 169 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:43.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:41.778105+0000 mon.a (mon.0) 1176 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "22"}]: dispatch 2024-02-23T00:43:43.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:41.814495+0000 osd.7 (osd.7) 91 : cluster [DBG] 5.1a scrub starts 2024-02-23T00:43:43.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:41.817383+0000 osd.7 (osd.7) 92 : cluster [DBG] 5.1a scrub ok 2024-02-23T00:43:43.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:42.118114+0000 mon.a (mon.0) 1177 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "22"}]': finished 2024-02-23T00:43:43.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:42.118182+0000 mon.a (mon.0) 1178 : cluster [DBG] osdmap e150: 8 total, 8 up, 8 in 2024-02-23T00:43:43.061 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:42.120318+0000 mon.b (mon.1) 170 : audit [INF] from='client.? 172.21.15.47:0/1361361966' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi047-82487-9"}]: dispatch 2024-02-23T00:43:43.061 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:42.132715+0000 mon.a (mon.0) 1179 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi047-82487-9"}]: dispatch 2024-02-23T00:43:43.061 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:42.156882+0000 mon.b (mon.1) 171 : audit [INF] from='client.? 172.21.15.47:0/2824343318' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi047-82378-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:43.061 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:42.186434+0000 mon.a (mon.0) 1180 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi047-82378-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:43.061 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:42.318352+0000 osd.2 (osd.2) 71 : cluster [DBG] 2.e scrub starts 2024-02-23T00:43:43.061 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:42.320340+0000 osd.2 (osd.2) 72 : cluster [DBG] 2.e scrub ok 2024-02-23T00:43:43.061 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:42.339513+0000 mgr.y (mgr.14517) 197 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:43:43.061 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:42.692803+0000 mon.b (mon.1) 172 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:43.061 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:42.826037+0000 mon.a (mon.0) 1181 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:43:43.091 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:43:43.178 INFO:tasks.workunit.client.0.smithi047.stdout: api_pool: Running main() from gmock_main.cc 2024-02-23T00:43:43.178 INFO:tasks.workunit.client.0.smithi047.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2024-02-23T00:43:43.178 INFO:tasks.workunit.client.0.smithi047.stdout: api_pool: [----------] Global test environment set-up. 2024-02-23T00:43:43.178 INFO:tasks.workunit.client.0.smithi047.stdout: api_pool: [----------] 9 tests from LibRadosPools 2024-02-23T00:43:43.178 INFO:tasks.workunit.client.0.smithi047.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2024-02-23T00:43:43.178 INFO:tasks.workunit.client.0.smithi047.stdout: api_pool: [ OK ] LibRadosPools.PoolList (12538 ms) 2024-02-23T00:43:43.178 INFO:tasks.workunit.client.0.smithi047.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2024-02-23T00:43:43.179 INFO:tasks.workunit.client.0.smithi047.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (6814 ms) 2024-02-23T00:43:43.179 INFO:tasks.workunit.client.0.smithi047.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2024-02-23T00:43:43.179 INFO:tasks.workunit.client.0.smithi047.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (4315 ms) 2024-02-23T00:43:43.179 INFO:tasks.workunit.client.0.smithi047.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-02-23T00:43:43.179 INFO:tasks.workunit.client.0.smithi047.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (3350 ms) 2024-02-23T00:43:43.179 INFO:tasks.workunit.client.0.smithi047.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-02-23T00:43:43.179 INFO:tasks.workunit.client.0.smithi047.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (3581 ms) 2024-02-23T00:43:43.179 INFO:tasks.workunit.client.0.smithi047.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2024-02-23T00:43:43.179 INFO:tasks.workunit.client.0.smithi047.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (6124 ms) 2024-02-23T00:43:43.179 INFO:tasks.workunit.client.0.smithi047.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2024-02-23T00:43:43.179 INFO:tasks.workunit.client.0.smithi047.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (6530 ms) 2024-02-23T00:43:43.179 INFO:tasks.workunit.client.0.smithi047.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-02-23T00:43:43.179 INFO:tasks.workunit.client.0.smithi047.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (4475 ms) 2024-02-23T00:43:43.179 INFO:tasks.workunit.client.0.smithi047.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2024-02-23T00:43:43.179 INFO:tasks.workunit.client.0.smithi047.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (8241 ms) 2024-02-23T00:43:43.180 INFO:tasks.workunit.client.0.smithi047.stdout: api_pool: [----------] 9 tests from LibRadosPools (55969 ms total) 2024-02-23T00:43:43.180 INFO:tasks.workunit.client.0.smithi047.stdout: api_pool: 2024-02-23T00:43:43.180 INFO:tasks.workunit.client.0.smithi047.stdout: api_pool: [----------] Global test environment tear-down 2024-02-23T00:43:43.180 INFO:tasks.workunit.client.0.smithi047.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (55970 ms total) 2024-02-23T00:43:43.180 INFO:tasks.workunit.client.0.smithi047.stdout: api_pool: [ PASSED ] 9 tests. 2024-02-23T00:43:43.184 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:43.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:40.431691+0000 osd.6 (osd.6) 67 : cluster [DBG] 20.8 deep-scrub starts 2024-02-23T00:43:43.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:40.432734+0000 osd.6 (osd.6) 68 : cluster [DBG] 20.8 deep-scrub ok 2024-02-23T00:43:43.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:40.771228+0000 osd.7 (osd.7) 89 : cluster [DBG] 20.9 deep-scrub starts 2024-02-23T00:43:43.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:40.777584+0000 osd.7 (osd.7) 90 : cluster [DBG] 20.9 deep-scrub ok 2024-02-23T00:43:43.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:41.207671+0000 mgr.y (mgr.14517) 196 : cluster [DBG] pgmap v199: 451 pgs: 64 creating+peering, 1 active+clean+snaptrim, 64 unknown, 1 peering, 321 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:43:43.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:41.689303+0000 mon.b (mon.1) 169 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:43.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:41.778105+0000 mon.a (mon.0) 1176 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "22"}]: dispatch 2024-02-23T00:43:43.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:41.814495+0000 osd.7 (osd.7) 91 : cluster [DBG] 5.1a scrub starts 2024-02-23T00:43:43.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:41.817383+0000 osd.7 (osd.7) 92 : cluster [DBG] 5.1a scrub ok 2024-02-23T00:43:43.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:42.118114+0000 mon.a (mon.0) 1177 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "22"}]': finished 2024-02-23T00:43:43.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:42.118182+0000 mon.a (mon.0) 1178 : cluster [DBG] osdmap e150: 8 total, 8 up, 8 in 2024-02-23T00:43:43.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:42.120318+0000 mon.b (mon.1) 170 : audit [INF] from='client.? 172.21.15.47:0/1361361966' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi047-82487-9"}]: dispatch 2024-02-23T00:43:43.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:42.132715+0000 mon.a (mon.0) 1179 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi047-82487-9"}]: dispatch 2024-02-23T00:43:43.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:42.156882+0000 mon.b (mon.1) 171 : audit [INF] from='client.? 172.21.15.47:0/2824343318' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi047-82378-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:43.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:42.186434+0000 mon.a (mon.0) 1180 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi047-82378-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:43.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:42.318352+0000 osd.2 (osd.2) 71 : cluster [DBG] 2.e scrub starts 2024-02-23T00:43:43.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:42.320340+0000 osd.2 (osd.2) 72 : cluster [DBG] 2.e scrub ok 2024-02-23T00:43:43.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:42.339513+0000 mgr.y (mgr.14517) 197 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:43:43.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:42.692803+0000 mon.b (mon.1) 172 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:43.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:42.826037+0000 mon.a (mon.0) 1181 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:43:43.453 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:43:44.127 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:43:44.131 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:44.131 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:43:44.232 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:43:44.259 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:44.277 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:43:44.279 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:44.280 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:43:44.381 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:43:44.407 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:44.426 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:43:44.428 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:43.132361+0000 mon.a (mon.0) 1182 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi047-82487-9"}]': finished 2024-02-23T00:43:44.428 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:43.132558+0000 mon.a (mon.0) 1183 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi047-82378-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:44.428 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:43.132701+0000 mon.a (mon.0) 1184 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:43:44.428 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:43.132745+0000 mon.a (mon.0) 1185 : cluster [DBG] osdmap e151: 8 total, 8 up, 8 in 2024-02-23T00:43:44.428 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:43.147595+0000 mon.b (mon.1) 173 : audit [INF] from='client.? 172.21.15.47:0/3335353598' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi047-82386-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:44.428 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:43.151509+0000 mon.a (mon.0) 1186 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi047-82386-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:44.428 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:43.153799+0000 mon.b (mon.1) 174 : audit [INF] from='client.? 172.21.15.47:0/1361361966' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi047-82487-9"}]: dispatch 2024-02-23T00:43:44.428 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:43.164142+0000 mon.a (mon.0) 1187 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi047-82487-9"}]: dispatch 2024-02-23T00:43:44.428 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:43.164440+0000 mon.a (mon.0) 1188 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-11"}]: dispatch 2024-02-23T00:43:44.428 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:43.210655+0000 mgr.y (mgr.14517) 198 : cluster [DBG] pgmap v202: 411 pgs: 22 active+clean+snaptrim_wait, 313 active+clean, 2 peering, 1 activating, 9 active+clean+snaptrim, 38 unknown, 26 creating+peering; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.1 KiB/s rd, 270 KiB/s wr, 6 op/s 2024-02-23T00:43:44.428 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:43.696833+0000 mon.b (mon.1) 175 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:44.429 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:44.429 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:43:44.530 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:43:44.556 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:44.563 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:43.132361+0000 mon.a (mon.0) 1182 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi047-82487-9"}]': finished 2024-02-23T00:43:44.563 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:43.132558+0000 mon.a (mon.0) 1183 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi047-82378-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:44.563 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:43.132701+0000 mon.a (mon.0) 1184 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:43:44.563 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:43.132745+0000 mon.a (mon.0) 1185 : cluster [DBG] osdmap e151: 8 total, 8 up, 8 in 2024-02-23T00:43:44.563 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:43.147595+0000 mon.b (mon.1) 173 : audit [INF] from='client.? 172.21.15.47:0/3335353598' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi047-82386-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:44.563 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:43.151509+0000 mon.a (mon.0) 1186 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi047-82386-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:44.563 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:43.153799+0000 mon.b (mon.1) 174 : audit [INF] from='client.? 172.21.15.47:0/1361361966' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi047-82487-9"}]: dispatch 2024-02-23T00:43:44.563 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:43.164142+0000 mon.a (mon.0) 1187 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi047-82487-9"}]: dispatch 2024-02-23T00:43:44.563 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:43.164440+0000 mon.a (mon.0) 1188 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-11"}]: dispatch 2024-02-23T00:43:44.563 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:43.210655+0000 mgr.y (mgr.14517) 198 : cluster [DBG] pgmap v202: 411 pgs: 22 active+clean+snaptrim_wait, 313 active+clean, 2 peering, 1 activating, 9 active+clean+snaptrim, 38 unknown, 26 creating+peering; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.1 KiB/s rd, 270 KiB/s wr, 6 op/s 2024-02-23T00:43:44.563 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:43.696833+0000 mon.b (mon.1) 175 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:44.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:43.132361+0000 mon.a (mon.0) 1182 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi047-82487-9"}]': finished 2024-02-23T00:43:44.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:43.132558+0000 mon.a (mon.0) 1183 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi047-82378-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:44.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:43.132701+0000 mon.a (mon.0) 1184 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:43:44.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:43.132745+0000 mon.a (mon.0) 1185 : cluster [DBG] osdmap e151: 8 total, 8 up, 8 in 2024-02-23T00:43:44.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:43.147595+0000 mon.b (mon.1) 173 : audit [INF] from='client.? 172.21.15.47:0/3335353598' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi047-82386-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:44.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:43.151509+0000 mon.a (mon.0) 1186 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi047-82386-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:44.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:43.153799+0000 mon.b (mon.1) 174 : audit [INF] from='client.? 172.21.15.47:0/1361361966' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi047-82487-9"}]: dispatch 2024-02-23T00:43:44.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:43.164142+0000 mon.a (mon.0) 1187 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi047-82487-9"}]: dispatch 2024-02-23T00:43:44.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:43.164440+0000 mon.a (mon.0) 1188 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-11"}]: dispatch 2024-02-23T00:43:44.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:43.210655+0000 mgr.y (mgr.14517) 198 : cluster [DBG] pgmap v202: 411 pgs: 22 active+clean+snaptrim_wait, 313 active+clean, 2 peering, 1 activating, 9 active+clean+snaptrim, 38 unknown, 26 creating+peering; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.1 KiB/s rd, 270 KiB/s wr, 6 op/s 2024-02-23T00:43:44.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:43.696833+0000 mon.b (mon.1) 175 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:44.575 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:43:44.577 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:44.577 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:43:44.758 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:43:44.787 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:43:44.790 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:43:44.906 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:45.086 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:43:45.092 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:45.093 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:43:45.153 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:43:45.194 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:43:45.290 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:45.421 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:43:45.466 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:45.467 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:43:45.567 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:43:45.622 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:45.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:43.874446+0000 osd.1 (osd.1) 91 : cluster [DBG] 20.5 deep-scrub starts 2024-02-23T00:43:45.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:43.919111+0000 osd.1 (osd.1) 92 : cluster [DBG] 20.5 deep-scrub ok 2024-02-23T00:43:45.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:43.992374+0000 osd.3 (osd.3) 97 : cluster [DBG] 4.11 scrub starts 2024-02-23T00:43:45.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:43.997444+0000 osd.3 (osd.3) 98 : cluster [DBG] 4.11 scrub ok 2024-02-23T00:43:45.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:44.030403+0000 osd.5 (osd.5) 83 : cluster [DBG] 2.18 scrub starts 2024-02-23T00:43:45.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:44.031788+0000 osd.5 (osd.5) 84 : cluster [DBG] 2.18 scrub ok 2024-02-23T00:43:45.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:44.131952+0000 mon.a (mon.0) 1189 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:43:45.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:44.135183+0000 mon.a (mon.0) 1190 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:43:45.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:44.151939+0000 mon.a (mon.0) 1191 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi047-82386-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:45.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:44.151999+0000 mon.a (mon.0) 1192 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi047-82487-9"}]': finished 2024-02-23T00:43:45.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:44.152035+0000 mon.a (mon.0) 1193 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-11"}]': finished 2024-02-23T00:43:45.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:44.152117+0000 mon.a (mon.0) 1194 : cluster [DBG] osdmap e152: 8 total, 8 up, 8 in 2024-02-23T00:43:45.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:44.250979+0000 mon.c (mon.2) 77 : audit [INF] from='client.? 172.21.15.47:0/2338570187' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]: dispatch 2024-02-23T00:43:45.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:44.253944+0000 mon.a (mon.0) 1195 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]: dispatch 2024-02-23T00:43:45.817 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:44.254600+0000 mon.c (mon.2) 78 : audit [INF] from='client.? 172.21.15.47:0/2338570187' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]: dispatch 2024-02-23T00:43:45.817 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:44.255471+0000 mon.a (mon.0) 1196 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]: dispatch 2024-02-23T00:43:45.817 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:44.258147+0000 mon.c (mon.2) 79 : audit [INF] from='client.? 172.21.15.47:0/2338570187' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:45.817 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:44.258443+0000 mon.a (mon.0) 1197 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:45.817 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:44.309634+0000 osd.2 (osd.2) 73 : cluster [DBG] 5.b scrub starts 2024-02-23T00:43:45.817 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:44.313491+0000 osd.2 (osd.2) 74 : cluster [DBG] 5.b scrub ok 2024-02-23T00:43:45.817 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:44.711741+0000 mon.b (mon.1) 176 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:45.817 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:45.102237+0000 mon.a (mon.0) 1198 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:45.817 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:45.102368+0000 mon.a (mon.0) 1199 : cluster [DBG] osdmap e153: 8 total, 8 up, 8 in 2024-02-23T00:43:45.817 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:45.109229+0000 mon.c (mon.2) 80 : audit [INF] from='client.? 172.21.15.47:0/2338570187' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi047-82487-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]: dispatch 2024-02-23T00:43:45.817 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:45.111072+0000 mon.c (mon.2) 81 : audit [INF] from='client.? 172.21.15.47:0/1354776715' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi047-82378-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:45.817 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:45.119146+0000 mon.a (mon.0) 1200 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi047-82487-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]: dispatch 2024-02-23T00:43:45.818 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:45.119873+0000 mon.a (mon.0) 1201 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi047-82378-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:45.818 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:43.874446+0000 osd.1 (osd.1) 91 : cluster [DBG] 20.5 deep-scrub starts 2024-02-23T00:43:45.818 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:43.919111+0000 osd.1 (osd.1) 92 : cluster [DBG] 20.5 deep-scrub ok 2024-02-23T00:43:45.818 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:43.992374+0000 osd.3 (osd.3) 97 : cluster [DBG] 4.11 scrub starts 2024-02-23T00:43:45.818 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:43.997444+0000 osd.3 (osd.3) 98 : cluster [DBG] 4.11 scrub ok 2024-02-23T00:43:45.818 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:44.030403+0000 osd.5 (osd.5) 83 : cluster [DBG] 2.18 scrub starts 2024-02-23T00:43:45.818 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:44.031788+0000 osd.5 (osd.5) 84 : cluster [DBG] 2.18 scrub ok 2024-02-23T00:43:45.818 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:44.131952+0000 mon.a (mon.0) 1189 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:43:45.818 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:44.135183+0000 mon.a (mon.0) 1190 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:43:45.818 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:44.151939+0000 mon.a (mon.0) 1191 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi047-82386-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:45.818 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:44.151999+0000 mon.a (mon.0) 1192 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi047-82487-9"}]': finished 2024-02-23T00:43:45.819 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:44.152035+0000 mon.a (mon.0) 1193 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-11"}]': finished 2024-02-23T00:43:45.819 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:44.152117+0000 mon.a (mon.0) 1194 : cluster [DBG] osdmap e152: 8 total, 8 up, 8 in 2024-02-23T00:43:45.819 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:44.250979+0000 mon.c (mon.2) 77 : audit [INF] from='client.? 172.21.15.47:0/2338570187' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]: dispatch 2024-02-23T00:43:45.819 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:44.253944+0000 mon.a (mon.0) 1195 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]: dispatch 2024-02-23T00:43:45.819 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:44.254600+0000 mon.c (mon.2) 78 : audit [INF] from='client.? 172.21.15.47:0/2338570187' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]: dispatch 2024-02-23T00:43:45.819 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:44.255471+0000 mon.a (mon.0) 1196 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]: dispatch 2024-02-23T00:43:45.819 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:44.258147+0000 mon.c (mon.2) 79 : audit [INF] from='client.? 172.21.15.47:0/2338570187' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:45.819 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:44.258443+0000 mon.a (mon.0) 1197 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:45.819 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:44.309634+0000 osd.2 (osd.2) 73 : cluster [DBG] 5.b scrub starts 2024-02-23T00:43:45.819 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:44.313491+0000 osd.2 (osd.2) 74 : cluster [DBG] 5.b scrub ok 2024-02-23T00:43:45.819 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:44.711741+0000 mon.b (mon.1) 176 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:45.819 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:45.102237+0000 mon.a (mon.0) 1198 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:45.819 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:45.102368+0000 mon.a (mon.0) 1199 : cluster [DBG] osdmap e153: 8 total, 8 up, 8 in 2024-02-23T00:43:45.820 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:45.109229+0000 mon.c (mon.2) 80 : audit [INF] from='client.? 172.21.15.47:0/2338570187' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi047-82487-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]: dispatch 2024-02-23T00:43:45.820 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:45.111072+0000 mon.c (mon.2) 81 : audit [INF] from='client.? 172.21.15.47:0/1354776715' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi047-82378-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:45.820 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:45.119146+0000 mon.a (mon.0) 1200 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi047-82487-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]: dispatch 2024-02-23T00:43:45.820 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:45.119873+0000 mon.a (mon.0) 1201 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi047-82378-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:45.884 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:43:45.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:45.886+0000 7fb754944700 1 -- 172.21.15.47:0/1487987878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7500f7ba0 msgr2=0x7fb750103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:45.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:45.886+0000 7fb754944700 1 --2- 172.21.15.47:0/1487987878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7500f7ba0 0x7fb750103950 secure :-1 s=READY pgs=3911 cs=0 l=1 rev1=1 rx=0x7fb744004870 tx=0x7fb744009cb0).stop 2024-02-23T00:43:45.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:45.887+0000 7fb754944700 1 -- 172.21.15.47:0/1487987878 shutdown_connections 2024-02-23T00:43:45.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:45.887+0000 7fb754944700 1 --2- 172.21.15.47:0/1487987878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7500f7ba0 0x7fb750103950 unknown :-1 s=CLOSED pgs=3911 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:45.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:45.887+0000 7fb754944700 1 --2- 172.21.15.47:0/1487987878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7500fcb40 0x7fb7500fcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:45.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:45.887+0000 7fb754944700 1 --2- 172.21.15.47:0/1487987878 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb7500fd4a0 0x7fb7500f74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:45.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:45.887+0000 7fb754944700 1 -- 172.21.15.47:0/1487987878 >> 172.21.15.47:0/1487987878 conn(0x7fb75006c370 msgr2=0x7fb75006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:43:45.890 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:45.890 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:43:45.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:45.887+0000 7fb754944700 1 -- 172.21.15.47:0/1487987878 shutdown_connections 2024-02-23T00:43:45.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:45.887+0000 7fb754944700 1 -- 172.21.15.47:0/1487987878 wait complete. 2024-02-23T00:43:45.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:45.888+0000 7fb754944700 1 librados: starting msgr at 2024-02-23T00:43:45.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:45.888+0000 7fb754944700 1 librados: starting objecter 2024-02-23T00:43:45.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:45.888+0000 7fb754944700 1 Processor -- start 2024-02-23T00:43:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:45.890+0000 7fb754944700 1 -- start start 2024-02-23T00:43:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:45.890+0000 7fb754944700 1 librados: setting wanted keys 2024-02-23T00:43:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:45.890+0000 7fb754944700 1 librados: calling monclient init 2024-02-23T00:43:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:45.891+0000 7fb754944700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7500f7ba0 0x7fb7500feec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:45.891+0000 7fb754944700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7500fcb40 0x7fb7500ff3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:45.891+0000 7fb754944700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb7500fd4a0 0x7fb7500ff970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:45.891+0000 7fb754944700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75011a5d0 con 0x7fb7500f7ba0 2024-02-23T00:43:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:45.891+0000 7fb754944700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb750062700 con 0x7fb7500fcb40 2024-02-23T00:43:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:45.891+0000 7fb754944700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7500628c0 con 0x7fb7500fd4a0 2024-02-23T00:43:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:45.892+0000 7fb74e59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7500f7ba0 0x7fb7500feec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:45.892+0000 7fb74e59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7500f7ba0 0x7fb7500feec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54270/0 (socket says 172.21.15.47:54270) 2024-02-23T00:43:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:45.892+0000 7fb74e59c700 1 -- 172.21.15.47:0/3531352093 learned_addr learned my addr 172.21.15.47:0/3531352093 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:43:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:45.892+0000 7fb74e59c700 1 -- 172.21.15.47:0/3531352093 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb7500fd4a0 msgr2=0x7fb7500ff970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:45.892+0000 7fb74e59c700 1 --2- 172.21.15.47:0/3531352093 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb7500fd4a0 0x7fb7500ff970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:43:45.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:45.892+0000 7fb74e59c700 1 -- 172.21.15.47:0/3531352093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7500fcb40 msgr2=0x7fb7500ff3b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-23T00:43:45.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:45.892+0000 7fb74e59c700 1 --2- 172.21.15.47:0/3531352093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7500fcb40 0x7fb7500ff3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:43:45.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:45.892+0000 7fb74e59c700 1 -- 172.21.15.47:0/3531352093 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb744005040 con 0x7fb7500f7ba0 2024-02-23T00:43:45.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:45.893+0000 7fb74e59c700 1 --2- 172.21.15.47:0/3531352093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7500f7ba0 0x7fb7500feec0 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 rx=0x7fb7400041d0 tx=0x7fb740009f60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:45.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:45.893+0000 7fb73f7fe700 10 client.?.objecter ms_handle_connect 0x7fb7500f7ba0 2024-02-23T00:43:45.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:45.893+0000 7fb73f7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:43:45.991 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:43:46.020 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:46.144 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:43:46.148 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:46.148 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:43:46.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:43.874446+0000 osd.1 (osd.1) 91 : cluster [DBG] 20.5 deep-scrub starts 2024-02-23T00:43:46.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:43.919111+0000 osd.1 (osd.1) 92 : cluster [DBG] 20.5 deep-scrub ok 2024-02-23T00:43:46.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:43.992374+0000 osd.3 (osd.3) 97 : cluster [DBG] 4.11 scrub starts 2024-02-23T00:43:46.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:43.997444+0000 osd.3 (osd.3) 98 : cluster [DBG] 4.11 scrub ok 2024-02-23T00:43:46.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:44.030403+0000 osd.5 (osd.5) 83 : cluster [DBG] 2.18 scrub starts 2024-02-23T00:43:46.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:44.031788+0000 osd.5 (osd.5) 84 : cluster [DBG] 2.18 scrub ok 2024-02-23T00:43:46.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:44.131952+0000 mon.a (mon.0) 1189 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:43:46.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:44.135183+0000 mon.a (mon.0) 1190 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:43:46.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:44.151939+0000 mon.a (mon.0) 1191 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi047-82386-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:46.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:44.151999+0000 mon.a (mon.0) 1192 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi047-82487-9"}]': finished 2024-02-23T00:43:46.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:44.152035+0000 mon.a (mon.0) 1193 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-11"}]': finished 2024-02-23T00:43:46.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:44.152117+0000 mon.a (mon.0) 1194 : cluster [DBG] osdmap e152: 8 total, 8 up, 8 in 2024-02-23T00:43:46.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:44.250979+0000 mon.c (mon.2) 77 : audit [INF] from='client.? 172.21.15.47:0/2338570187' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]: dispatch 2024-02-23T00:43:46.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:44.253944+0000 mon.a (mon.0) 1195 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]: dispatch 2024-02-23T00:43:46.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:44.254600+0000 mon.c (mon.2) 78 : audit [INF] from='client.? 172.21.15.47:0/2338570187' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]: dispatch 2024-02-23T00:43:46.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:44.255471+0000 mon.a (mon.0) 1196 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]: dispatch 2024-02-23T00:43:46.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:44.258147+0000 mon.c (mon.2) 79 : audit [INF] from='client.? 172.21.15.47:0/2338570187' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:46.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:44.258443+0000 mon.a (mon.0) 1197 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:43:46.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:44.309634+0000 osd.2 (osd.2) 73 : cluster [DBG] 5.b scrub starts 2024-02-23T00:43:46.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:44.313491+0000 osd.2 (osd.2) 74 : cluster [DBG] 5.b scrub ok 2024-02-23T00:43:46.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:44.711741+0000 mon.b (mon.1) 176 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:46.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:45.102237+0000 mon.a (mon.0) 1198 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:43:46.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:45.102368+0000 mon.a (mon.0) 1199 : cluster [DBG] osdmap e153: 8 total, 8 up, 8 in 2024-02-23T00:43:46.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:45.109229+0000 mon.c (mon.2) 80 : audit [INF] from='client.? 172.21.15.47:0/2338570187' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi047-82487-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]: dispatch 2024-02-23T00:43:46.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:45.111072+0000 mon.c (mon.2) 81 : audit [INF] from='client.? 172.21.15.47:0/1354776715' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi047-82378-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:46.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:45.119146+0000 mon.a (mon.0) 1200 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi047-82487-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]: dispatch 2024-02-23T00:43:46.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:45.119873+0000 mon.a (mon.0) 1201 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi047-82378-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:46.249 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:43:46.278 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:46.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:46.657+0000 7fb73f7fe700 1 -- 172.21.15.47:0/3531352093 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb740007b40 con 0x7fb7500f7ba0 2024-02-23T00:43:46.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:46.658+0000 7fb754944700 1 -- 172.21.15.47:0/3531352093 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb750062ba0 con 0x7fb7500f7ba0 2024-02-23T00:43:46.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:46.658+0000 7fb754944700 10 client.29940.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:43:46.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:46.658+0000 7fb754944700 1 -- 172.21.15.47:0/3531352093 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7500630e0 con 0x7fb7500f7ba0 2024-02-23T00:43:46.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:46.658+0000 7fb754944700 1 librados: init done 2024-02-23T00:43:46.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:46.660+0000 7fb754944700 1 -- 172.21.15.47:0/3531352093 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb75005fa40 con 0x7fb7500f7ba0 2024-02-23T00:43:46.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:46.671+0000 7fb73f7fe700 1 -- 172.21.15.47:0/3531352093 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fb74000f040 con 0x7fb7500f7ba0 2024-02-23T00:43:46.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:46.671+0000 7fb73f7fe700 1 -- 172.21.15.47:0/3531352093 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74000d5c0 con 0x7fb7500f7ba0 2024-02-23T00:43:46.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:46.671+0000 7fb73f7fe700 1 -- 172.21.15.47:0/3531352093 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fb74000d7e0 con 0x7fb7500f7ba0 2024-02-23T00:43:46.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:46.672+0000 7fb73f7fe700 1 --2- 172.21.15.47:0/3531352093 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb738062cc0 0x7fb738065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:46.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:46.672+0000 7fb73f7fe700 1 -- 172.21.15.47:0/3531352093 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(154..154 src has 48..154) v4 ==== 11657+0+0 (secure 0 0 0) 0x7fb74007eef0 con 0x7fb7500f7ba0 2024-02-23T00:43:46.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:46.672+0000 7fb73f7fe700 10 client.29940.objecter ms_dispatch 0x7fb750001550 osd_map(154..154 src has 48..154) v4 2024-02-23T00:43:46.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:46.672+0000 7fb73f7fe700 3 client.29940.objecter handle_osd_map got epochs [154,154] > 0 2024-02-23T00:43:46.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:46.672+0000 7fb73f7fe700 3 client.29940.objecter handle_osd_map decoding full epoch 154 2024-02-23T00:43:46.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:46.672+0000 7fb73f7fe700 20 client.29940.objecter dump_active .. 0 homeless 2024-02-23T00:43:46.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:46.682+0000 7fb74dd9b700 1 --2- 172.21.15.47:0/3531352093 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb738062cc0 0x7fb738065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:46.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:46.685+0000 7fb73f7fe700 1 -- 172.21.15.47:0/3531352093 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb74004db80 con 0x7fb7500f7ba0 2024-02-23T00:43:46.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:46.685+0000 7fb74dd9b700 1 --2- 172.21.15.47:0/3531352093 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb738062cc0 0x7fb738065180 secure :-1 s=READY pgs=2892 cs=0 l=1 rev1=1 rx=0x7fb750100660 tx=0x7fb748004f70).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:46.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:46.685+0000 7fb73f7fe700 10 client.29940.objecter ms_handle_connect 0x7fb738062cc0 2024-02-23T00:43:46.729 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:44.368014+0000 osd.6 (osd.6) 69 : cluster [DBG] 4.5 scrub starts 2024-02-23T00:43:46.729 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:44.369788+0000 osd.6 (osd.6) 70 : cluster [DBG] 4.5 scrub ok 2024-02-23T00:43:46.729 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:44.791055+0000 osd.7 (osd.7) 93 : cluster [DBG] 2.1d scrub starts 2024-02-23T00:43:46.729 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:44.794924+0000 osd.7 (osd.7) 94 : cluster [DBG] 2.1d scrub ok 2024-02-23T00:43:46.729 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:44.950001+0000 osd.3 (osd.3) 99 : cluster [DBG] 5.17 scrub starts 2024-02-23T00:43:46.729 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:44.951689+0000 osd.3 (osd.3) 100 : cluster [DBG] 5.17 scrub ok 2024-02-23T00:43:46.729 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:45.050191+0000 osd.5 (osd.5) 85 : cluster [DBG] 5.7 scrub starts 2024-02-23T00:43:46.729 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:45.054111+0000 osd.5 (osd.5) 86 : cluster [DBG] 5.7 scrub ok 2024-02-23T00:43:46.730 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:45.192352+0000 osd.4 (osd.4) 79 : cluster [DBG] 4.d scrub starts 2024-02-23T00:43:46.730 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:45.194135+0000 osd.4 (osd.4) 80 : cluster [DBG] 4.d scrub ok 2024-02-23T00:43:46.730 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:45.211331+0000 mgr.y (mgr.14517) 199 : cluster [DBG] pgmap v205: 346 pgs: 17 active+clean+snaptrim_wait, 288 active+clean, 1 peering, 1 activating, 7 active+clean+snaptrim, 32 unknown; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 257 KiB/s wr, 4 op/s 2024-02-23T00:43:46.730 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:45.307832+0000 osd.0 (osd.0) 81 : cluster [DBG] 2.13 scrub starts 2024-02-23T00:43:46.730 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:45.309077+0000 osd.0 (osd.0) 82 : cluster [DBG] 2.13 scrub ok 2024-02-23T00:43:46.730 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:45.361186+0000 osd.6 (osd.6) 71 : cluster [DBG] 2.1a scrub starts 2024-02-23T00:43:46.730 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:45.361971+0000 osd.6 (osd.6) 72 : cluster [DBG] 2.1a scrub ok 2024-02-23T00:43:46.730 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:45.713342+0000 mon.b (mon.1) 177 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:46.730 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:45.964484+0000 osd.3 (osd.3) 101 : cluster [DBG] 6.13 scrub starts 2024-02-23T00:43:46.730 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:45.965564+0000 osd.3 (osd.3) 102 : cluster [DBG] 6.13 scrub ok 2024-02-23T00:43:46.730 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:46.292916+0000 osd.2 (osd.2) 75 : cluster [DBG] 5.6 scrub starts 2024-02-23T00:43:46.730 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:46.293854+0000 osd.2 (osd.2) 76 : cluster [DBG] 5.6 scrub ok 2024-02-23T00:43:46.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:46.935+0000 7fb754944700 1 -- 172.21.15.47:0/3531352093 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7fb75005ee80 con 0x7fb7500f7ba0 2024-02-23T00:43:47.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:46.656059+0000 mon.a (mon.0) 1202 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi047-82378-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:47.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:46.656348+0000 mon.a (mon.0) 1203 : cluster [DBG] osdmap e154: 8 total, 8 up, 8 in 2024-02-23T00:43:47.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:46.657940+0000 mon.b (mon.1) 178 : audit [INF] from='client.? 172.21.15.47:0/1599782321' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi047-82386-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:47.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:46.659928+0000 mon.a (mon.0) 1204 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:43:47.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:46.662634+0000 mon.a (mon.0) 1205 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:47.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:46.666692+0000 mon.a (mon.0) 1206 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi047-82386-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:47.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:44.368014+0000 osd.6 (osd.6) 69 : cluster [DBG] 4.5 scrub starts 2024-02-23T00:43:47.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:44.369788+0000 osd.6 (osd.6) 70 : cluster [DBG] 4.5 scrub ok 2024-02-23T00:43:47.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:44.791055+0000 osd.7 (osd.7) 93 : cluster [DBG] 2.1d scrub starts 2024-02-23T00:43:47.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:44.794924+0000 osd.7 (osd.7) 94 : cluster [DBG] 2.1d scrub ok 2024-02-23T00:43:47.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:44.950001+0000 osd.3 (osd.3) 99 : cluster [DBG] 5.17 scrub starts 2024-02-23T00:43:47.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:44.951689+0000 osd.3 (osd.3) 100 : cluster [DBG] 5.17 scrub ok 2024-02-23T00:43:47.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:45.050191+0000 osd.5 (osd.5) 85 : cluster [DBG] 5.7 scrub starts 2024-02-23T00:43:47.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:45.054111+0000 osd.5 (osd.5) 86 : cluster [DBG] 5.7 scrub ok 2024-02-23T00:43:47.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:45.192352+0000 osd.4 (osd.4) 79 : cluster [DBG] 4.d scrub starts 2024-02-23T00:43:47.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:45.194135+0000 osd.4 (osd.4) 80 : cluster [DBG] 4.d scrub ok 2024-02-23T00:43:47.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:45.211331+0000 mgr.y (mgr.14517) 199 : cluster [DBG] pgmap v205: 346 pgs: 17 active+clean+snaptrim_wait, 288 active+clean, 1 peering, 1 activating, 7 active+clean+snaptrim, 32 unknown; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 257 KiB/s wr, 4 op/s 2024-02-23T00:43:47.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:45.307832+0000 osd.0 (osd.0) 81 : cluster [DBG] 2.13 scrub starts 2024-02-23T00:43:47.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:45.309077+0000 osd.0 (osd.0) 82 : cluster [DBG] 2.13 scrub ok 2024-02-23T00:43:47.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:45.361186+0000 osd.6 (osd.6) 71 : cluster [DBG] 2.1a scrub starts 2024-02-23T00:43:47.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:45.361971+0000 osd.6 (osd.6) 72 : cluster [DBG] 2.1a scrub ok 2024-02-23T00:43:47.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:45.713342+0000 mon.b (mon.1) 177 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:47.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:45.964484+0000 osd.3 (osd.3) 101 : cluster [DBG] 6.13 scrub starts 2024-02-23T00:43:47.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:45.965564+0000 osd.3 (osd.3) 102 : cluster [DBG] 6.13 scrub ok 2024-02-23T00:43:47.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:46.292916+0000 osd.2 (osd.2) 75 : cluster [DBG] 5.6 scrub starts 2024-02-23T00:43:47.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:46.293854+0000 osd.2 (osd.2) 76 : cluster [DBG] 5.6 scrub ok 2024-02-23T00:43:47.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:46.656059+0000 mon.a (mon.0) 1202 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi047-82378-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:47.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:46.656348+0000 mon.a (mon.0) 1203 : cluster [DBG] osdmap e154: 8 total, 8 up, 8 in 2024-02-23T00:43:47.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:46.657940+0000 mon.b (mon.1) 178 : audit [INF] from='client.? 172.21.15.47:0/1599782321' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi047-82386-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:47.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:46.659928+0000 mon.a (mon.0) 1204 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:43:47.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:46.662634+0000 mon.a (mon.0) 1205 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:47.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:46.666692+0000 mon.a (mon.0) 1206 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi047-82386-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:47.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:44.368014+0000 osd.6 (osd.6) 69 : cluster [DBG] 4.5 scrub starts 2024-02-23T00:43:47.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:44.369788+0000 osd.6 (osd.6) 70 : cluster [DBG] 4.5 scrub ok 2024-02-23T00:43:47.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:44.791055+0000 osd.7 (osd.7) 93 : cluster [DBG] 2.1d scrub starts 2024-02-23T00:43:47.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:44.794924+0000 osd.7 (osd.7) 94 : cluster [DBG] 2.1d scrub ok 2024-02-23T00:43:47.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:44.950001+0000 osd.3 (osd.3) 99 : cluster [DBG] 5.17 scrub starts 2024-02-23T00:43:47.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:44.951689+0000 osd.3 (osd.3) 100 : cluster [DBG] 5.17 scrub ok 2024-02-23T00:43:47.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:45.050191+0000 osd.5 (osd.5) 85 : cluster [DBG] 5.7 scrub starts 2024-02-23T00:43:47.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:45.054111+0000 osd.5 (osd.5) 86 : cluster [DBG] 5.7 scrub ok 2024-02-23T00:43:47.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:45.192352+0000 osd.4 (osd.4) 79 : cluster [DBG] 4.d scrub starts 2024-02-23T00:43:47.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:45.194135+0000 osd.4 (osd.4) 80 : cluster [DBG] 4.d scrub ok 2024-02-23T00:43:47.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:45.211331+0000 mgr.y (mgr.14517) 199 : cluster [DBG] pgmap v205: 346 pgs: 17 active+clean+snaptrim_wait, 288 active+clean, 1 peering, 1 activating, 7 active+clean+snaptrim, 32 unknown; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 257 KiB/s wr, 4 op/s 2024-02-23T00:43:47.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:45.307832+0000 osd.0 (osd.0) 81 : cluster [DBG] 2.13 scrub starts 2024-02-23T00:43:47.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:45.309077+0000 osd.0 (osd.0) 82 : cluster [DBG] 2.13 scrub ok 2024-02-23T00:43:47.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:45.361186+0000 osd.6 (osd.6) 71 : cluster [DBG] 2.1a scrub starts 2024-02-23T00:43:47.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:45.361971+0000 osd.6 (osd.6) 72 : cluster [DBG] 2.1a scrub ok 2024-02-23T00:43:47.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:45.713342+0000 mon.b (mon.1) 177 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:47.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:45.964484+0000 osd.3 (osd.3) 101 : cluster [DBG] 6.13 scrub starts 2024-02-23T00:43:47.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:45.965564+0000 osd.3 (osd.3) 102 : cluster [DBG] 6.13 scrub ok 2024-02-23T00:43:47.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:46.292916+0000 osd.2 (osd.2) 75 : cluster [DBG] 5.6 scrub starts 2024-02-23T00:43:47.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:46.293854+0000 osd.2 (osd.2) 76 : cluster [DBG] 5.6 scrub ok 2024-02-23T00:43:47.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:46.656059+0000 mon.a (mon.0) 1202 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi047-82378-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:47.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:46.656348+0000 mon.a (mon.0) 1203 : cluster [DBG] osdmap e154: 8 total, 8 up, 8 in 2024-02-23T00:43:47.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:46.657940+0000 mon.b (mon.1) 178 : audit [INF] from='client.? 172.21.15.47:0/1599782321' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi047-82386-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:47.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:46.659928+0000 mon.a (mon.0) 1204 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:43:47.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:46.662634+0000 mon.a (mon.0) 1205 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:47.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:46.666692+0000 mon.a (mon.0) 1206 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi047-82386-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:47.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:47.215+0000 7fb73f7fe700 1 -- 172.21.15.47:0/3531352093 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v155) v1 ==== 101+0+0 (secure 0 0 0) 0x7fb75005ee80 con 0x7fb7500f7ba0 2024-02-23T00:43:47.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:47.220+0000 7fb754944700 10 librados: watch_flush enter 2024-02-23T00:43:47.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:47.220+0000 7fb754944700 10 librados: watch_flush exit 2024-02-23T00:43:47.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:47.220+0000 7fb754944700 20 client.29940.objecter shutdown clearing up homeless session... 2024-02-23T00:43:47.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:47.220+0000 7fb754944700 10 client.29940.objecter successfully canceled tick 2024-02-23T00:43:47.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:47.221+0000 7fb754944700 1 -- 172.21.15.47:0/3531352093 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb738062cc0 msgr2=0x7fb738065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:47.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:47.221+0000 7fb754944700 1 --2- 172.21.15.47:0/3531352093 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb738062cc0 0x7fb738065180 secure :-1 s=READY pgs=2892 cs=0 l=1 rev1=1 rx=0x7fb750100660 tx=0x7fb748004f70).stop 2024-02-23T00:43:47.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:47.221+0000 7fb754944700 1 -- 172.21.15.47:0/3531352093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7500f7ba0 msgr2=0x7fb7500feec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:47.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:47.221+0000 7fb754944700 1 --2- 172.21.15.47:0/3531352093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7500f7ba0 0x7fb7500feec0 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 rx=0x7fb7400041d0 tx=0x7fb740009f60).stop 2024-02-23T00:43:47.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:47.224+0000 7fb754944700 1 -- 172.21.15.47:0/3531352093 shutdown_connections 2024-02-23T00:43:47.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:47.224+0000 7fb754944700 1 --2- 172.21.15.47:0/3531352093 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb738062cc0 0x7fb738065180 unknown :-1 s=CLOSED pgs=2892 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:47.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:47.224+0000 7fb754944700 1 --2- 172.21.15.47:0/3531352093 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb7500fd4a0 0x7fb7500ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:43:47.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:47.224+0000 7fb754944700 1 --2- 172.21.15.47:0/3531352093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7500f7ba0 0x7fb7500feec0 unknown :-1 s=CLOSED pgs=2412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:47.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:47.224+0000 7fb754944700 1 --2- 172.21.15.47:0/3531352093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7500fcb40 0x7fb7500ff3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:43:47.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:47.224+0000 7fb754944700 1 -- 172.21.15.47:0/3531352093 >> 172.21.15.47:0/3531352093 conn(0x7fb75006c370 msgr2=0x7fb7500f51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:43:47.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:47.225+0000 7fb754944700 1 -- 172.21.15.47:0/3531352093 shutdown_connections 2024-02-23T00:43:47.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:47.228+0000 7fb754944700 1 -- 172.21.15.47:0/3531352093 wait complete. 2024-02-23T00:43:47.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:47.228+0000 7fb754944700 1 librados: shutdown 2024-02-23T00:43:47.232 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is unset 2024-02-23T00:43:47.247 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2024-02-23T00:43:47.248 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [----------] Global test environment set-up. 2024-02-23T00:43:47.248 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2024-02-23T00:43:47.248 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2024-02-23T00:43:47.248 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2024-02-23T00:43:47.248 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-02-23T00:43:47.248 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: 2024-02-23T00:43:47.248 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2024-02-23T00:43:47.249 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-02-23T00:43:47.249 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2024-02-23T00:43:47.249 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: 2024-02-23T00:42:47.790+0000 7f53d5e38700 -1 failed for service _ceph-mon._tcp 2024-02-23T00:43:47.249 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: 2024-02-23T00:42:47.790+0000 7f53d5e38700 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-02-23T00:43:47.249 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (552 ms) 2024-02-23T00:43:47.249 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-02-23T00:43:47.249 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (2005 ms) 2024-02-23T00:43:47.249 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (2557 ms total) 2024-02-23T00:43:47.249 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: 2024-02-23T00:43:47.249 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2024-02-23T00:43:47.249 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2024-02-23T00:43:47.249 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: started 0x555b0b235f70 2024-02-23T00:43:47.249 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: started 0x555b0b23f450 2024-02-23T00:43:47.249 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: started 2 aios 2024-02-23T00:43:47.249 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: waiting 0x555b0b235f70 2024-02-23T00:43:47.250 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: waiting 0x555b0b23f450 2024-02-23T00:43:47.250 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: done. 2024-02-23T00:43:47.250 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (16317 ms) 2024-02-23T00:43:47.250 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (16317 ms total) 2024-02-23T00:43:47.250 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: 2024-02-23T00:43:47.250 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2024-02-23T00:43:47.250 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2024-02-23T00:43:47.250 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-02-23T00:43:47.250 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2024-02-23T00:43:47.250 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [ OK ] LibRadosMisc.Exec (57 ms) 2024-02-23T00:43:47.250 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2024-02-23T00:43:47.250 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (13 ms) 2024-02-23T00:43:47.250 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2024-02-23T00:43:47.250 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (7 ms) 2024-02-23T00:43:47.250 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2024-02-23T00:43:47.251 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [ OK ] LibRadosMisc.Applications (5631 ms) 2024-02-23T00:43:47.251 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2024-02-23T00:43:47.251 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-02-23T00:43:47.251 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2024-02-23T00:43:47.251 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-02-23T00:43:47.251 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2024-02-23T00:43:47.251 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (31265 ms) 2024-02-23T00:43:47.251 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [----------] 8 tests from LibRadosMisc (36973 ms total) 2024-02-23T00:43:47.251 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: 2024-02-23T00:43:47.251 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [----------] Global test environment tear-down 2024-02-23T00:43:47.251 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (59993 ms total) 2024-02-23T00:43:47.251 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc: [ PASSED ] 12 tests. 2024-02-23T00:43:48.056 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:46.156249+0000 osd.4 (osd.4) 81 : cluster [DBG] 4.9 scrub starts 2024-02-23T00:43:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:46.158143+0000 osd.4 (osd.4) 82 : cluster [DBG] 4.9 scrub ok 2024-02-23T00:43:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:46.714180+0000 mon.b (mon.1) 179 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:46.868516+0000 osd.1 (osd.1) 93 : cluster [DBG] 20.6 deep-scrub starts 2024-02-23T00:43:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:46.925734+0000 osd.1 (osd.1) 94 : cluster [DBG] 20.6 deep-scrub ok 2024-02-23T00:43:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:46.936856+0000 mon.a (mon.0) 1207 : audit [INF] from='client.? 172.21.15.47:0/3531352093' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:43:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:47.204394+0000 mon.a (mon.0) 1208 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:43:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:47.208913+0000 mon.a (mon.0) 1209 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi047-82487-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]': finished 2024-02-23T00:43:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:47.209099+0000 mon.a (mon.0) 1210 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:47.209251+0000 mon.a (mon.0) 1211 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi047-82386-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:47.209357+0000 mon.a (mon.0) 1212 : audit [INF] from='client.? 172.21.15.47:0/3531352093' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:43:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:47.209402+0000 mon.a (mon.0) 1213 : cluster [DBG] osdmap e155: 8 total, 8 up, 8 in 2024-02-23T00:43:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:47.211842+0000 mgr.y (mgr.14517) 200 : cluster [DBG] pgmap v208: 354 pgs: 282 active+clean, 64 creating+peering, 8 unknown; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:43:48.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:47.715281+0000 mon.b (mon.1) 180 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:48.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:47.723487+0000 mon.a (mon.0) 1214 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:43:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:46.156249+0000 osd.4 (osd.4) 81 : cluster [DBG] 4.9 scrub starts 2024-02-23T00:43:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:46.158143+0000 osd.4 (osd.4) 82 : cluster [DBG] 4.9 scrub ok 2024-02-23T00:43:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:46.714180+0000 mon.b (mon.1) 179 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:46.868516+0000 osd.1 (osd.1) 93 : cluster [DBG] 20.6 deep-scrub starts 2024-02-23T00:43:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:46.925734+0000 osd.1 (osd.1) 94 : cluster [DBG] 20.6 deep-scrub ok 2024-02-23T00:43:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:46.936856+0000 mon.a (mon.0) 1207 : audit [INF] from='client.? 172.21.15.47:0/3531352093' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:43:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:47.204394+0000 mon.a (mon.0) 1208 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:43:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:47.208913+0000 mon.a (mon.0) 1209 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi047-82487-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]': finished 2024-02-23T00:43:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:47.209099+0000 mon.a (mon.0) 1210 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:47.209251+0000 mon.a (mon.0) 1211 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi047-82386-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:47.209357+0000 mon.a (mon.0) 1212 : audit [INF] from='client.? 172.21.15.47:0/3531352093' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:43:48.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:47.209402+0000 mon.a (mon.0) 1213 : cluster [DBG] osdmap e155: 8 total, 8 up, 8 in 2024-02-23T00:43:48.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:47.211842+0000 mgr.y (mgr.14517) 200 : cluster [DBG] pgmap v208: 354 pgs: 282 active+clean, 64 creating+peering, 8 unknown; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:43:48.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:47.715281+0000 mon.b (mon.1) 180 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:48.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:47.723487+0000 mon.a (mon.0) 1214 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:43:48.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:46.156249+0000 osd.4 (osd.4) 81 : cluster [DBG] 4.9 scrub starts 2024-02-23T00:43:48.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:46.158143+0000 osd.4 (osd.4) 82 : cluster [DBG] 4.9 scrub ok 2024-02-23T00:43:48.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:46.714180+0000 mon.b (mon.1) 179 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:48.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:46.868516+0000 osd.1 (osd.1) 93 : cluster [DBG] 20.6 deep-scrub starts 2024-02-23T00:43:48.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:46.925734+0000 osd.1 (osd.1) 94 : cluster [DBG] 20.6 deep-scrub ok 2024-02-23T00:43:48.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:46.936856+0000 mon.a (mon.0) 1207 : audit [INF] from='client.? 172.21.15.47:0/3531352093' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:43:48.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:47.204394+0000 mon.a (mon.0) 1208 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:43:48.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:47.208913+0000 mon.a (mon.0) 1209 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi047-82487-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]': finished 2024-02-23T00:43:48.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:47.209099+0000 mon.a (mon.0) 1210 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:48.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:47.209251+0000 mon.a (mon.0) 1211 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi047-82386-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:48.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:47.209357+0000 mon.a (mon.0) 1212 : audit [INF] from='client.? 172.21.15.47:0/3531352093' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:43:48.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:47.209402+0000 mon.a (mon.0) 1213 : cluster [DBG] osdmap e155: 8 total, 8 up, 8 in 2024-02-23T00:43:48.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:47.211842+0000 mgr.y (mgr.14517) 200 : cluster [DBG] pgmap v208: 354 pgs: 282 active+clean, 64 creating+peering, 8 unknown; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:43:48.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:47.715281+0000 mon.b (mon.1) 180 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:48.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:47.723487+0000 mon.a (mon.0) 1214 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:43:48.562 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:43:48.566 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:48.567 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:43:48.667 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:43:48.694 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:48.713 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:43:48.715 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:48.716 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:43:48.817 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:43:48.843 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:48.861 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:43:48.863 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:48.864 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:43:48.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:46.757441+0000 osd.7 (osd.7) 95 : cluster [DBG] 6.15 scrub starts 2024-02-23T00:43:48.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:46.760339+0000 osd.7 (osd.7) 96 : cluster [DBG] 6.15 scrub ok 2024-02-23T00:43:48.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:47.212355+0000 osd.1 (osd.1) 95 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:43:48.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:47.212464+0000 osd.1 (osd.1) 96 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:43:48.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:47.902427+0000 osd.1 (osd.1) 97 : cluster [DBG] 6.4 scrub starts 2024-02-23T00:43:48.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:47.905490+0000 osd.1 (osd.1) 98 : cluster [DBG] 6.4 scrub ok 2024-02-23T00:43:48.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:48.029983+0000 osd.3 (osd.3) 103 : cluster [DBG] 2.1b scrub starts 2024-02-23T00:43:48.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:48.030842+0000 osd.3 (osd.3) 104 : cluster [DBG] 2.1b scrub ok 2024-02-23T00:43:48.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:48.077988+0000 mon.a (mon.0) 1215 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:43:48.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:48.078752+0000 mon.a (mon.0) 1216 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:43:48.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:48.083357+0000 mon.a (mon.0) 1217 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:43:48.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:48.210488+0000 mon.a (mon.0) 1218 : cluster [DBG] osdmap e156: 8 total, 8 up, 8 in 2024-02-23T00:43:48.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:48.233179+0000 mon.b (mon.1) 181 : audit [INF] from='client.? 172.21.15.47:0/3255081840' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi047-82378-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:48.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:48.234335+0000 mon.a (mon.0) 1219 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi047-82378-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:48.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:48.257256+0000 osd.0 (osd.0) 83 : cluster [DBG] 3.1 scrub starts 2024-02-23T00:43:48.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:48.258265+0000 osd.0 (osd.0) 84 : cluster [DBG] 3.1 scrub ok 2024-02-23T00:43:48.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:48.716342+0000 mon.b (mon.1) 182 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:48.964 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:43:48.973 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:46.757441+0000 osd.7 (osd.7) 95 : cluster [DBG] 6.15 scrub starts 2024-02-23T00:43:48.974 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:46.760339+0000 osd.7 (osd.7) 96 : cluster [DBG] 6.15 scrub ok 2024-02-23T00:43:48.974 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:47.212355+0000 osd.1 (osd.1) 95 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:43:48.974 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:47.212464+0000 osd.1 (osd.1) 96 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:43:48.974 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:47.902427+0000 osd.1 (osd.1) 97 : cluster [DBG] 6.4 scrub starts 2024-02-23T00:43:48.974 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:47.905490+0000 osd.1 (osd.1) 98 : cluster [DBG] 6.4 scrub ok 2024-02-23T00:43:48.974 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:48.029983+0000 osd.3 (osd.3) 103 : cluster [DBG] 2.1b scrub starts 2024-02-23T00:43:48.974 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:48.030842+0000 osd.3 (osd.3) 104 : cluster [DBG] 2.1b scrub ok 2024-02-23T00:43:48.974 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:48.077988+0000 mon.a (mon.0) 1215 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:43:48.975 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:48.078752+0000 mon.a (mon.0) 1216 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:43:48.975 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:48.083357+0000 mon.a (mon.0) 1217 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:43:48.975 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:48.210488+0000 mon.a (mon.0) 1218 : cluster [DBG] osdmap e156: 8 total, 8 up, 8 in 2024-02-23T00:43:48.975 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:48.233179+0000 mon.b (mon.1) 181 : audit [INF] from='client.? 172.21.15.47:0/3255081840' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi047-82378-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:48.975 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:48.234335+0000 mon.a (mon.0) 1219 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi047-82378-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:48.975 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:48.257256+0000 osd.0 (osd.0) 83 : cluster [DBG] 3.1 scrub starts 2024-02-23T00:43:48.975 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:48.258265+0000 osd.0 (osd.0) 84 : cluster [DBG] 3.1 scrub ok 2024-02-23T00:43:48.975 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:48.716342+0000 mon.b (mon.1) 182 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:48.976 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:46.757441+0000 osd.7 (osd.7) 95 : cluster [DBG] 6.15 scrub starts 2024-02-23T00:43:48.976 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:46.760339+0000 osd.7 (osd.7) 96 : cluster [DBG] 6.15 scrub ok 2024-02-23T00:43:48.976 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:47.212355+0000 osd.1 (osd.1) 95 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:43:48.976 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:47.212464+0000 osd.1 (osd.1) 96 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:43:48.976 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:47.902427+0000 osd.1 (osd.1) 97 : cluster [DBG] 6.4 scrub starts 2024-02-23T00:43:48.976 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:47.905490+0000 osd.1 (osd.1) 98 : cluster [DBG] 6.4 scrub ok 2024-02-23T00:43:48.976 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:48.029983+0000 osd.3 (osd.3) 103 : cluster [DBG] 2.1b scrub starts 2024-02-23T00:43:48.976 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:48.030842+0000 osd.3 (osd.3) 104 : cluster [DBG] 2.1b scrub ok 2024-02-23T00:43:48.976 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:48.077988+0000 mon.a (mon.0) 1215 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:43:48.976 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:48.078752+0000 mon.a (mon.0) 1216 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:43:48.976 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:48.083357+0000 mon.a (mon.0) 1217 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:43:48.977 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:48.210488+0000 mon.a (mon.0) 1218 : cluster [DBG] osdmap e156: 8 total, 8 up, 8 in 2024-02-23T00:43:48.977 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:48.233179+0000 mon.b (mon.1) 181 : audit [INF] from='client.? 172.21.15.47:0/3255081840' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi047-82378-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:48.977 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:48.234335+0000 mon.a (mon.0) 1219 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi047-82378-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:48.977 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:48.257256+0000 osd.0 (osd.0) 83 : cluster [DBG] 3.1 scrub starts 2024-02-23T00:43:48.977 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:48.258265+0000 osd.0 (osd.0) 84 : cluster [DBG] 3.1 scrub ok 2024-02-23T00:43:48.977 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:48.716342+0000 mon.b (mon.1) 182 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:48.996 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:49.018 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:43:49.021 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:49.022 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:43:49.122 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:43:49.151 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:49.803 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:43:49.806 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:49.807 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:43:49.908 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:43:49.938 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:50.028 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:47.791639+0000 osd.7 (osd.7) 97 : cluster [DBG] 6.5 scrub starts 2024-02-23T00:43:50.029 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:47.794530+0000 osd.7 (osd.7) 98 : cluster [DBG] 6.5 scrub ok 2024-02-23T00:43:50.029 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:48.125571+0000 osd.4 (osd.4) 83 : cluster [DBG] 5.18 scrub starts 2024-02-23T00:43:50.029 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:48.126546+0000 osd.4 (osd.4) 84 : cluster [DBG] 5.18 scrub ok 2024-02-23T00:43:50.029 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:48.875324+0000 osd.1 (osd.1) 99 : cluster [DBG] 3.0 scrub starts 2024-02-23T00:43:50.029 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:48.876352+0000 osd.1 (osd.1) 100 : cluster [DBG] 3.0 scrub ok 2024-02-23T00:43:50.029 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:49.212188+0000 mon.a (mon.0) 1220 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi047-82378-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:50.029 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:49.212350+0000 mon.a (mon.0) 1221 : cluster [DBG] osdmap e157: 8 total, 8 up, 8 in 2024-02-23T00:43:50.029 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:49.213292+0000 mgr.y (mgr.14517) 201 : cluster [DBG] pgmap v211: 386 pgs: 282 active+clean, 32 creating+peering, 72 unknown; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-23T00:43:50.029 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:49.219978+0000 mon.c (mon.2) 82 : audit [INF] from='client.? 172.21.15.47:0/3016630746' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi047-82386-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:50.029 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:49.221354+0000 mon.a (mon.0) 1222 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi047-82386-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:50.029 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:49.228307+0000 osd.0 (osd.0) 85 : cluster [DBG] 5.1e scrub starts 2024-02-23T00:43:50.030 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:49.238700+0000 osd.0 (osd.0) 86 : cluster [DBG] 5.1e scrub ok 2024-02-23T00:43:50.030 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:49.249884+0000 osd.2 (osd.2) 77 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:43:50.030 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:49.250126+0000 osd.2 (osd.2) 78 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:43:50.030 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:49.717718+0000 mon.b (mon.1) 183 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:50.030 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:47.791639+0000 osd.7 (osd.7) 97 : cluster [DBG] 6.5 scrub starts 2024-02-23T00:43:50.030 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:47.794530+0000 osd.7 (osd.7) 98 : cluster [DBG] 6.5 scrub ok 2024-02-23T00:43:50.030 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:48.125571+0000 osd.4 (osd.4) 83 : cluster [DBG] 5.18 scrub starts 2024-02-23T00:43:50.030 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:48.126546+0000 osd.4 (osd.4) 84 : cluster [DBG] 5.18 scrub ok 2024-02-23T00:43:50.030 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:48.875324+0000 osd.1 (osd.1) 99 : cluster [DBG] 3.0 scrub starts 2024-02-23T00:43:50.030 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:48.876352+0000 osd.1 (osd.1) 100 : cluster [DBG] 3.0 scrub ok 2024-02-23T00:43:50.030 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:49.212188+0000 mon.a (mon.0) 1220 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi047-82378-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:50.031 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:49.212350+0000 mon.a (mon.0) 1221 : cluster [DBG] osdmap e157: 8 total, 8 up, 8 in 2024-02-23T00:43:50.031 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:49.213292+0000 mgr.y (mgr.14517) 201 : cluster [DBG] pgmap v211: 386 pgs: 282 active+clean, 32 creating+peering, 72 unknown; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-23T00:43:50.031 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:49.219978+0000 mon.c (mon.2) 82 : audit [INF] from='client.? 172.21.15.47:0/3016630746' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi047-82386-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:50.031 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:49.221354+0000 mon.a (mon.0) 1222 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi047-82386-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:50.031 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:49.228307+0000 osd.0 (osd.0) 85 : cluster [DBG] 5.1e scrub starts 2024-02-23T00:43:50.031 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:49.238700+0000 osd.0 (osd.0) 86 : cluster [DBG] 5.1e scrub ok 2024-02-23T00:43:50.031 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:49.249884+0000 osd.2 (osd.2) 77 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:43:50.031 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:49.250126+0000 osd.2 (osd.2) 78 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:43:50.031 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:49.717718+0000 mon.b (mon.1) 183 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:50.036 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:43:50.039 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:50.040 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:43:50.140 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:43:50.173 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:50.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:47.791639+0000 osd.7 (osd.7) 97 : cluster [DBG] 6.5 scrub starts 2024-02-23T00:43:50.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:47.794530+0000 osd.7 (osd.7) 98 : cluster [DBG] 6.5 scrub ok 2024-02-23T00:43:50.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:48.125571+0000 osd.4 (osd.4) 83 : cluster [DBG] 5.18 scrub starts 2024-02-23T00:43:50.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:48.126546+0000 osd.4 (osd.4) 84 : cluster [DBG] 5.18 scrub ok 2024-02-23T00:43:50.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:48.875324+0000 osd.1 (osd.1) 99 : cluster [DBG] 3.0 scrub starts 2024-02-23T00:43:50.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:48.876352+0000 osd.1 (osd.1) 100 : cluster [DBG] 3.0 scrub ok 2024-02-23T00:43:50.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:49.212188+0000 mon.a (mon.0) 1220 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi047-82378-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:50.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:49.212350+0000 mon.a (mon.0) 1221 : cluster [DBG] osdmap e157: 8 total, 8 up, 8 in 2024-02-23T00:43:50.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:49.213292+0000 mgr.y (mgr.14517) 201 : cluster [DBG] pgmap v211: 386 pgs: 282 active+clean, 32 creating+peering, 72 unknown; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-23T00:43:50.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:49.219978+0000 mon.c (mon.2) 82 : audit [INF] from='client.? 172.21.15.47:0/3016630746' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi047-82386-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:50.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:49.221354+0000 mon.a (mon.0) 1222 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi047-82386-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:50.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:49.228307+0000 osd.0 (osd.0) 85 : cluster [DBG] 5.1e scrub starts 2024-02-23T00:43:50.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:49.238700+0000 osd.0 (osd.0) 86 : cluster [DBG] 5.1e scrub ok 2024-02-23T00:43:50.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:49.249884+0000 osd.2 (osd.2) 77 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:43:50.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:49.250126+0000 osd.2 (osd.2) 78 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:43:50.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:49.717718+0000 mon.b (mon.1) 183 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:50.262 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set noscrub 2024-02-23T00:43:50.399 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:43:50.400 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:43:50.404 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:43:50.406 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:50.407 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:43:50.508 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:43:50.534 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:50.552 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:43:50.554 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:50.555 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:43:50.556 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:43:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:43:50] "GET /metrics HTTP/1.1" 200 204555 "" "Prometheus/2.33.4" 2024-02-23T00:43:50.656 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:43:50.682 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:50.701 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:43:50.703 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:50.704 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:43:50.804 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:43:50.836 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:50.900 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:43:50.900 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:43:50.900 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:43:50.901 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:43:50.901 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:43:50.901 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:43:50.901 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:43:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.387+0000 7fa04fc5e700 1 -- 172.21.15.47:0/4129611109 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa0480f8420 msgr2=0x7fa048104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.387+0000 7fa04fc5e700 1 --2- 172.21.15.47:0/4129611109 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa0480f8420 0x7fa048104a20 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 rx=0x7fa040009be0 tx=0x7fa040004d10).stop 2024-02-23T00:43:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.387+0000 7fa04fc5e700 1 -- 172.21.15.47:0/4129611109 shutdown_connections 2024-02-23T00:43:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.387+0000 7fa04fc5e700 1 --2- 172.21.15.47:0/4129611109 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa0480f8420 0x7fa048104a20 unknown :-1 s=CLOSED pgs=2467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.387+0000 7fa04fc5e700 1 --2- 172.21.15.47:0/4129611109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0480fdbb0 0x7fa0480fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.387+0000 7fa04fc5e700 1 --2- 172.21.15.47:0/4129611109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0480fe510 0x7fa0480f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.387+0000 7fa04fc5e700 1 -- 172.21.15.47:0/4129611109 >> 172.21.15.47:0/4129611109 conn(0x7fa0480f3800 msgr2=0x7fa0480f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:43:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.387+0000 7fa04fc5e700 1 -- 172.21.15.47:0/4129611109 shutdown_connections 2024-02-23T00:43:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.388+0000 7fa04fc5e700 1 -- 172.21.15.47:0/4129611109 wait complete. 2024-02-23T00:43:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.388+0000 7fa04fc5e700 1 librados: starting msgr at 2024-02-23T00:43:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.388+0000 7fa04fc5e700 1 librados: starting objecter 2024-02-23T00:43:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.388+0000 7fa04fc5e700 1 Processor -- start 2024-02-23T00:43:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.388+0000 7fa04fc5e700 1 -- start start 2024-02-23T00:43:50.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.388+0000 7fa04fc5e700 1 librados: setting wanted keys 2024-02-23T00:43:50.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.388+0000 7fa04fc5e700 1 librados: calling monclient init 2024-02-23T00:43:50.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.388+0000 7fa04fc5e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0480f8420 0x7fa04806b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:50.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.388+0000 7fa04fc5e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0480fdbb0 0x7fa04806b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:50.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.388+0000 7fa04fc5e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa0480fe510 0x7fa04806bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:50.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.388+0000 7fa04fc5e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0481161b0 con 0x7fa0480fdbb0 2024-02-23T00:43:50.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.388+0000 7fa04fc5e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa048116320 con 0x7fa0480f8420 2024-02-23T00:43:50.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.388+0000 7fa04fc5e700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0481164e0 con 0x7fa0480fe510 2024-02-23T00:43:50.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.389+0000 7fa04e1fb700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa0480fe510 0x7fa04806bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:50.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.389+0000 7fa04d9fa700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0480f8420 0x7fa04806b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:50.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.389+0000 7fa04e1fb700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa0480fe510 0x7fa04806bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:35320/0 (socket says 172.21.15.47:35320) 2024-02-23T00:43:50.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.389+0000 7fa04d9fa700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0480f8420 0x7fa04806b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.47:33002/0 (socket says 172.21.15.47:33002) 2024-02-23T00:43:50.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.389+0000 7fa04e1fb700 1 -- 172.21.15.47:0/1097988865 learned_addr learned my addr 172.21.15.47:0/1097988865 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:43:50.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.389+0000 7fa04d1f9700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0480fdbb0 0x7fa04806b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.389+0000 7fa04d9fa700 1 -- 172.21.15.47:0/1097988865 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa0480fe510 msgr2=0x7fa04806bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.389+0000 7fa04d9fa700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa0480fe510 0x7fa04806bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.389+0000 7fa04d9fa700 1 -- 172.21.15.47:0/1097988865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0480fdbb0 msgr2=0x7fa04806b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.389+0000 7fa04d9fa700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0480fdbb0 0x7fa04806b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.389+0000 7fa04d9fa700 1 -- 172.21.15.47:0/1097988865 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0400097c0 con 0x7fa0480f8420 2024-02-23T00:43:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.389+0000 7fa04e1fb700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa0480fe510 0x7fa04806bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:43:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.389+0000 7fa04d9fa700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0480f8420 0x7fa04806b0d0 secure :-1 s=READY pgs=3939 cs=0 l=1 rev1=1 rx=0x7fa044002780 tx=0x7fa044009e40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.389+0000 7fa03effd700 10 client.?.objecter ms_handle_connect 0x7fa0480f8420 2024-02-23T00:43:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.389+0000 7fa03effd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:43:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.389+0000 7fa03effd700 1 -- 172.21.15.47:0/1097988865 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa044007b40 con 0x7fa0480f8420 2024-02-23T00:43:50.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.389+0000 7fa03effd700 1 -- 172.21.15.47:0/1097988865 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fa04400f040 con 0x7fa0480f8420 2024-02-23T00:43:50.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.389+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0481167c0 con 0x7fa0480f8420 2024-02-23T00:43:50.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.389+0000 7fa03effd700 1 -- 172.21.15.47:0/1097988865 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa044014670 con 0x7fa0480f8420 2024-02-23T00:43:50.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.390+0000 7fa04fc5e700 10 client.42502.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:43:50.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.390+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa048116cb0 con 0x7fa0480f8420 2024-02-23T00:43:50.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.390+0000 7fa04fc5e700 1 librados: init done 2024-02-23T00:43:50.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.390+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fa048060280 con 0x7fa0480f8420 2024-02-23T00:43:50.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.392+0000 7fa03effd700 1 -- 172.21.15.47:0/1097988865 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fa044009110 con 0x7fa0480f8420 2024-02-23T00:43:50.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.392+0000 7fa03effd700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa034062cc0 0x7fa034065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:50.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.392+0000 7fa04d1f9700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa034062cc0 0x7fa034065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:50.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.392+0000 7fa03effd700 1 -- 172.21.15.47:0/1097988865 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(158..158 src has 48..158) v4 ==== 11411+0+0 (secure 0 0 0) 0x7fa04407ddf0 con 0x7fa0480f8420 2024-02-23T00:43:50.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.392+0000 7fa03effd700 10 client.42502.objecter ms_dispatch 0x7fa048001550 osd_map(158..158 src has 48..158) v4 2024-02-23T00:43:50.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.392+0000 7fa03effd700 3 client.42502.objecter handle_osd_map got epochs [158,158] > 0 2024-02-23T00:43:50.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.392+0000 7fa03effd700 3 client.42502.objecter handle_osd_map decoding full epoch 158 2024-02-23T00:43:50.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.392+0000 7fa03effd700 20 client.42502.objecter dump_active .. 0 homeless 2024-02-23T00:43:50.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.392+0000 7fa03effd700 1 -- 172.21.15.47:0/1097988865 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v158) v1 ==== 54+0+16 (secure 0 0 0) 0x7fa04404c440 con 0x7fa0480f8420 2024-02-23T00:43:50.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.393+0000 7fa03effd700 10 client.42502.objecter ms_handle_connect 0x7fa034062cc0 2024-02-23T00:43:50.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.393+0000 7fa04d1f9700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa034062cc0 0x7fa034065180 secure :-1 s=READY pgs=2899 cs=0 l=1 rev1=1 rx=0x7fa04806c820 tx=0x7fa03800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:50.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.393+0000 7fa04fc5e700 10 client.42502.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:50.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.393+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_assign -1 1 2024-02-23T00:43:50.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.393+0000 7fa04fc5e700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fa0480fee80 0x7fa0480ff260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:50.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.393+0000 7fa04fc5e700 20 client.42502.objecter _get_session s=0x7fa04805ee80 osd=0 3 2024-02-23T00:43:50.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.393+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa04805ee80 osd=0 3 2024-02-23T00:43:50.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.393+0000 7fa04fc5e700 20 client.42502.objecter _get_session s=0x7fa04805ee80 osd=0 3 2024-02-23T00:43:50.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.393+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_remove -1 1 2024-02-23T00:43:50.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.393+0000 7fa04fc5e700 20 client.42502.objecter get_session s=0x7fa04805ee80 osd=0 3 2024-02-23T00:43:50.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.393+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_assign 0 1 2024-02-23T00:43:50.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.393+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa04805ee80 osd=0 4 2024-02-23T00:43:50.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.393+0000 7fa04fc5e700 10 client.42502.objecter _send_command 1 2024-02-23T00:43:50.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.393+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa048106f70 con 0x7fa0480fee80 2024-02-23T00:43:50.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.393+0000 7fa04e1fb700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fa0480fee80 0x7fa0480ff260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:50.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.393+0000 7fa03effd700 10 client.42502.objecter ms_handle_connect 0x7fa0480fee80 2024-02-23T00:43:50.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.393+0000 7fa04e1fb700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fa0480fee80 0x7fa0480ff260 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:50.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.394+0000 7fa03effd700 1 -- 172.21.15.47:0/1097988865 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa048106f70 con 0x7fa0480fee80 2024-02-23T00:43:50.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.394+0000 7fa03effd700 10 client.42502.objecter ms_dispatch 0x7fa048001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:43:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.394+0000 7fa03effd700 10 client.42502.objecter _finish_command 1 = system:0 2024-02-23T00:43:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.394+0000 7fa03effd700 20 client.42502.objecter put_session s=0x7fa04805ee80 osd=0 4 2024-02-23T00:43:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.394+0000 7fa03effd700 15 client.42502.objecter _session_command_op_remove 0 1 2024-02-23T00:43:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.427+0000 7fa04fc5e700 10 client.42502.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:43:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.427+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_assign -1 2 2024-02-23T00:43:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.427+0000 7fa04fc5e700 20 client.42502.objecter _get_session s=0x7fa04805ee80 osd=0 3 2024-02-23T00:43:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.427+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa04805ee80 osd=0 3 2024-02-23T00:43:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.427+0000 7fa04fc5e700 20 client.42502.objecter _get_session s=0x7fa04805ee80 osd=0 3 2024-02-23T00:43:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.427+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_remove -1 2 2024-02-23T00:43:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.427+0000 7fa04fc5e700 20 client.42502.objecter get_session s=0x7fa04805ee80 osd=0 3 2024-02-23T00:43:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.427+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_assign 0 2 2024-02-23T00:43:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.427+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa04805ee80 osd=0 4 2024-02-23T00:43:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.427+0000 7fa04fc5e700 10 client.42502.objecter _send_command 2 2024-02-23T00:43:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.427+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa0480273c0 con 0x7fa0480fee80 2024-02-23T00:43:50.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.430+0000 7fa03effd700 1 -- 172.21.15.47:0/1097988865 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa0480273c0 con 0x7fa0480fee80 2024-02-23T00:43:50.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.430+0000 7fa03effd700 10 client.42502.objecter ms_dispatch 0x7fa048001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:43:50.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.430+0000 7fa03effd700 10 client.42502.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:43:50.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.430+0000 7fa03effd700 20 client.42502.objecter put_session s=0x7fa04805ee80 osd=0 4 2024-02-23T00:43:50.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.430+0000 7fa03effd700 15 client.42502.objecter _session_command_op_remove 0 2 2024-02-23T00:43:50.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.430+0000 7fa04fc5e700 10 client.42502.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:50.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.430+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_assign -1 3 2024-02-23T00:43:50.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.430+0000 7fa04fc5e700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fa0480598f0 0x7fa048059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:50.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.430+0000 7fa04fc5e700 20 client.42502.objecter _get_session s=0x7fa048028710 osd=1 3 2024-02-23T00:43:50.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.430+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa048028710 osd=1 3 2024-02-23T00:43:50.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.430+0000 7fa04fc5e700 20 client.42502.objecter _get_session s=0x7fa048028710 osd=1 3 2024-02-23T00:43:50.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.430+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_remove -1 3 2024-02-23T00:43:50.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.430+0000 7fa04fc5e700 20 client.42502.objecter get_session s=0x7fa048028710 osd=1 3 2024-02-23T00:43:50.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.430+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_assign 1 3 2024-02-23T00:43:50.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.430+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa048028710 osd=1 4 2024-02-23T00:43:50.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.430+0000 7fa04fc5e700 10 client.42502.objecter _send_command 3 2024-02-23T00:43:50.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.430+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa04811d6b0 con 0x7fa0480598f0 2024-02-23T00:43:50.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.430+0000 7fa04d9fa700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fa0480598f0 0x7fa048059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:50.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.431+0000 7fa03effd700 10 client.42502.objecter ms_handle_connect 0x7fa0480598f0 2024-02-23T00:43:50.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.431+0000 7fa04d9fa700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fa0480598f0 0x7fa048059cd0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:50.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.431+0000 7fa03effd700 1 -- 172.21.15.47:0/1097988865 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa04811d6b0 con 0x7fa0480598f0 2024-02-23T00:43:50.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.431+0000 7fa03effd700 10 client.42502.objecter ms_dispatch 0x7fa048001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:43:50.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.431+0000 7fa03effd700 10 client.42502.objecter _finish_command 3 = system:0 2024-02-23T00:43:50.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.431+0000 7fa03effd700 20 client.42502.objecter put_session s=0x7fa048028710 osd=1 4 2024-02-23T00:43:50.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.431+0000 7fa03effd700 15 client.42502.objecter _session_command_op_remove 1 3 2024-02-23T00:43:50.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.464+0000 7fa04fc5e700 10 client.42502.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:43:50.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.464+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_assign -1 4 2024-02-23T00:43:50.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.464+0000 7fa04fc5e700 20 client.42502.objecter _get_session s=0x7fa048028710 osd=1 3 2024-02-23T00:43:50.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.464+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa048028710 osd=1 3 2024-02-23T00:43:50.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.464+0000 7fa04fc5e700 20 client.42502.objecter _get_session s=0x7fa048028710 osd=1 3 2024-02-23T00:43:50.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.464+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_remove -1 4 2024-02-23T00:43:50.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.464+0000 7fa04fc5e700 20 client.42502.objecter get_session s=0x7fa048028710 osd=1 3 2024-02-23T00:43:50.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.464+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_assign 1 4 2024-02-23T00:43:50.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.464+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa048028710 osd=1 4 2024-02-23T00:43:50.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.464+0000 7fa04fc5e700 10 client.42502.objecter _send_command 4 2024-02-23T00:43:50.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.464+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa048059360 con 0x7fa0480598f0 2024-02-23T00:43:50.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.466+0000 7fa03effd700 1 -- 172.21.15.47:0/1097988865 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa048059360 con 0x7fa0480598f0 2024-02-23T00:43:50.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.466+0000 7fa03effd700 10 client.42502.objecter ms_dispatch 0x7fa048001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:43:50.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.466+0000 7fa03effd700 10 client.42502.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:43:50.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.466+0000 7fa03effd700 20 client.42502.objecter put_session s=0x7fa048028710 osd=1 4 2024-02-23T00:43:50.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.466+0000 7fa03effd700 15 client.42502.objecter _session_command_op_remove 1 4 2024-02-23T00:43:50.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.466+0000 7fa04fc5e700 10 client.42502.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:50.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.466+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_assign -1 5 2024-02-23T00:43:50.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.466+0000 7fa04fc5e700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa048028270 0x7fa04811c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:50.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.466+0000 7fa04fc5e700 20 client.42502.objecter _get_session s=0x7fa048059770 osd=2 3 2024-02-23T00:43:50.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.466+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa048059770 osd=2 3 2024-02-23T00:43:50.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.466+0000 7fa04fc5e700 20 client.42502.objecter _get_session s=0x7fa048059770 osd=2 3 2024-02-23T00:43:50.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.466+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_remove -1 5 2024-02-23T00:43:50.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.466+0000 7fa04fc5e700 20 client.42502.objecter get_session s=0x7fa048059770 osd=2 3 2024-02-23T00:43:50.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.466+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_assign 2 5 2024-02-23T00:43:50.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.466+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa048059770 osd=2 4 2024-02-23T00:43:50.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.466+0000 7fa04fc5e700 10 client.42502.objecter _send_command 5 2024-02-23T00:43:50.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.466+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa04811c5c0 con 0x7fa048028270 2024-02-23T00:43:50.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.466+0000 7fa04d1f9700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa048028270 0x7fa04811c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:50.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.466+0000 7fa03effd700 10 client.42502.objecter ms_handle_connect 0x7fa048028270 2024-02-23T00:43:50.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.466+0000 7fa04d1f9700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa048028270 0x7fa04811c0d0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:50.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.467+0000 7fa03effd700 1 -- 172.21.15.47:0/1097988865 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa04811c5c0 con 0x7fa048028270 2024-02-23T00:43:50.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.467+0000 7fa03effd700 10 client.42502.objecter ms_dispatch 0x7fa048001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:43:50.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.467+0000 7fa03effd700 10 client.42502.objecter _finish_command 5 = system:0 2024-02-23T00:43:50.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.467+0000 7fa03effd700 20 client.42502.objecter put_session s=0x7fa048059770 osd=2 4 2024-02-23T00:43:50.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.467+0000 7fa03effd700 15 client.42502.objecter _session_command_op_remove 2 5 2024-02-23T00:43:50.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.487+0000 7fa04fc5e700 10 client.42502.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:43:50.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.487+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_assign -1 6 2024-02-23T00:43:50.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.487+0000 7fa04fc5e700 20 client.42502.objecter _get_session s=0x7fa048059770 osd=2 3 2024-02-23T00:43:50.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.487+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa048059770 osd=2 3 2024-02-23T00:43:50.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.487+0000 7fa04fc5e700 20 client.42502.objecter _get_session s=0x7fa048059770 osd=2 3 2024-02-23T00:43:50.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.487+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_remove -1 6 2024-02-23T00:43:50.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.487+0000 7fa04fc5e700 20 client.42502.objecter get_session s=0x7fa048059770 osd=2 3 2024-02-23T00:43:50.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.487+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_assign 2 6 2024-02-23T00:43:50.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.487+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa048059770 osd=2 4 2024-02-23T00:43:50.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.487+0000 7fa04fc5e700 10 client.42502.objecter _send_command 6 2024-02-23T00:43:50.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.487+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa048027830 con 0x7fa048028270 2024-02-23T00:43:50.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.490+0000 7fa03effd700 1 -- 172.21.15.47:0/1097988865 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa048027830 con 0x7fa048028270 2024-02-23T00:43:50.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.490+0000 7fa03effd700 10 client.42502.objecter ms_dispatch 0x7fa048001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:43:50.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.490+0000 7fa03effd700 10 client.42502.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:43:50.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.490+0000 7fa03effd700 20 client.42502.objecter put_session s=0x7fa048059770 osd=2 4 2024-02-23T00:43:50.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.490+0000 7fa03effd700 15 client.42502.objecter _session_command_op_remove 2 6 2024-02-23T00:43:50.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.490+0000 7fa04fc5e700 10 client.42502.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:50.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.490+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_assign -1 7 2024-02-23T00:43:50.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.490+0000 7fa04fc5e700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fa04811f830 0x7fa048121c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:50.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.490+0000 7fa04fc5e700 20 client.42502.objecter _get_session s=0x7fa04811cf00 osd=3 3 2024-02-23T00:43:50.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.490+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa04811cf00 osd=3 3 2024-02-23T00:43:50.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.490+0000 7fa04fc5e700 20 client.42502.objecter _get_session s=0x7fa04811cf00 osd=3 3 2024-02-23T00:43:50.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.490+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_remove -1 7 2024-02-23T00:43:50.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.490+0000 7fa04fc5e700 20 client.42502.objecter get_session s=0x7fa04811cf00 osd=3 3 2024-02-23T00:43:50.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.490+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_assign 3 7 2024-02-23T00:43:50.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.490+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa04811cf00 osd=3 4 2024-02-23T00:43:50.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.490+0000 7fa04fc5e700 10 client.42502.objecter _send_command 7 2024-02-23T00:43:50.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.490+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa048124910 con 0x7fa04811f830 2024-02-23T00:43:50.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.490+0000 7fa04e1fb700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fa04811f830 0x7fa048121c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:50.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.491+0000 7fa03effd700 10 client.42502.objecter ms_handle_connect 0x7fa04811f830 2024-02-23T00:43:50.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.491+0000 7fa04e1fb700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fa04811f830 0x7fa048121c20 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:50.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.492+0000 7fa03effd700 1 -- 172.21.15.47:0/1097988865 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa048124910 con 0x7fa04811f830 2024-02-23T00:43:50.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.492+0000 7fa03effd700 10 client.42502.objecter ms_dispatch 0x7fa048001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:43:50.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.492+0000 7fa03effd700 10 client.42502.objecter _finish_command 7 = system:0 2024-02-23T00:43:50.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.492+0000 7fa03effd700 20 client.42502.objecter put_session s=0x7fa04811cf00 osd=3 4 2024-02-23T00:43:50.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.492+0000 7fa03effd700 15 client.42502.objecter _session_command_op_remove 3 7 2024-02-23T00:43:50.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.522+0000 7fa04fc5e700 10 client.42502.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:43:50.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.522+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_assign -1 8 2024-02-23T00:43:50.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.522+0000 7fa04fc5e700 20 client.42502.objecter _get_session s=0x7fa04811cf00 osd=3 3 2024-02-23T00:43:50.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.522+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa04811cf00 osd=3 3 2024-02-23T00:43:50.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.522+0000 7fa04fc5e700 20 client.42502.objecter _get_session s=0x7fa04811cf00 osd=3 3 2024-02-23T00:43:50.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.522+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_remove -1 8 2024-02-23T00:43:50.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.522+0000 7fa04fc5e700 20 client.42502.objecter get_session s=0x7fa04811cf00 osd=3 3 2024-02-23T00:43:50.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.522+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_assign 3 8 2024-02-23T00:43:50.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.522+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa04811cf00 osd=3 4 2024-02-23T00:43:50.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.522+0000 7fa04fc5e700 10 client.42502.objecter _send_command 8 2024-02-23T00:43:50.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.522+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa04811c970 con 0x7fa04811f830 2024-02-23T00:43:50.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.524+0000 7fa03effd700 1 -- 172.21.15.47:0/1097988865 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa04811c970 con 0x7fa04811f830 2024-02-23T00:43:50.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.524+0000 7fa03effd700 10 client.42502.objecter ms_dispatch 0x7fa048001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:43:50.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.524+0000 7fa03effd700 10 client.42502.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:43:50.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.524+0000 7fa03effd700 20 client.42502.objecter put_session s=0x7fa04811cf00 osd=3 4 2024-02-23T00:43:50.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.524+0000 7fa03effd700 15 client.42502.objecter _session_command_op_remove 3 8 2024-02-23T00:43:50.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.524+0000 7fa04fc5e700 10 client.42502.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:50.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.524+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_assign -1 9 2024-02-23T00:43:50.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.524+0000 7fa04fc5e700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fa04811bc30 0x7fa048122110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:50.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.524+0000 7fa04fc5e700 20 client.42502.objecter _get_session s=0x7fa04811cb10 osd=4 3 2024-02-23T00:43:50.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.524+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa04811cb10 osd=4 3 2024-02-23T00:43:50.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.524+0000 7fa04fc5e700 20 client.42502.objecter _get_session s=0x7fa04811cb10 osd=4 3 2024-02-23T00:43:50.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.524+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_remove -1 9 2024-02-23T00:43:50.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.524+0000 7fa04fc5e700 20 client.42502.objecter get_session s=0x7fa04811cb10 osd=4 3 2024-02-23T00:43:50.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.524+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_assign 4 9 2024-02-23T00:43:50.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.524+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa04811cb10 osd=4 4 2024-02-23T00:43:50.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.524+0000 7fa04fc5e700 10 client.42502.objecter _send_command 9 2024-02-23T00:43:50.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.524+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa04811cd30 con 0x7fa04811bc30 2024-02-23T00:43:50.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.525+0000 7fa04d9fa700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fa04811bc30 0x7fa048122110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:50.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.525+0000 7fa03effd700 10 client.42502.objecter ms_handle_connect 0x7fa04811bc30 2024-02-23T00:43:50.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.525+0000 7fa04d9fa700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fa04811bc30 0x7fa048122110 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:50.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.526+0000 7fa03effd700 1 -- 172.21.15.47:0/1097988865 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa04811cd30 con 0x7fa04811bc30 2024-02-23T00:43:50.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.526+0000 7fa03effd700 10 client.42502.objecter ms_dispatch 0x7fa048001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:43:50.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.526+0000 7fa03effd700 10 client.42502.objecter _finish_command 9 = system:0 2024-02-23T00:43:50.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.526+0000 7fa03effd700 20 client.42502.objecter put_session s=0x7fa04811cb10 osd=4 4 2024-02-23T00:43:50.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.526+0000 7fa03effd700 15 client.42502.objecter _session_command_op_remove 4 9 2024-02-23T00:43:50.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.564+0000 7fa04fc5e700 10 client.42502.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:43:50.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.564+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_assign -1 10 2024-02-23T00:43:50.919 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:43:50.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.564+0000 7fa04fc5e700 20 client.42502.objecter _get_session s=0x7fa04811cb10 osd=4 3 2024-02-23T00:43:50.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.564+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa04811cb10 osd=4 3 2024-02-23T00:43:50.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.564+0000 7fa04fc5e700 20 client.42502.objecter _get_session s=0x7fa04811cb10 osd=4 3 2024-02-23T00:43:50.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.564+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_remove -1 10 2024-02-23T00:43:50.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.564+0000 7fa04fc5e700 20 client.42502.objecter get_session s=0x7fa04811cb10 osd=4 3 2024-02-23T00:43:50.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.564+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_assign 4 10 2024-02-23T00:43:50.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.564+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa04811cb10 osd=4 4 2024-02-23T00:43:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.564+0000 7fa04fc5e700 10 client.42502.objecter _send_command 10 2024-02-23T00:43:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.564+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa04811cd30 con 0x7fa04811bc30 2024-02-23T00:43:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.566+0000 7fa03effd700 1 -- 172.21.15.47:0/1097988865 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa04811cd30 con 0x7fa04811bc30 2024-02-23T00:43:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.566+0000 7fa03effd700 10 client.42502.objecter ms_dispatch 0x7fa048001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:43:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.566+0000 7fa03effd700 10 client.42502.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:43:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.566+0000 7fa03effd700 20 client.42502.objecter put_session s=0x7fa04811cb10 osd=4 4 2024-02-23T00:43:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.566+0000 7fa03effd700 15 client.42502.objecter _session_command_op_remove 4 10 2024-02-23T00:43:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.566+0000 7fa04fc5e700 10 client.42502.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.566+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_assign -1 11 2024-02-23T00:43:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.566+0000 7fa04fc5e700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fa048122b90 0x7fa048122ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.566+0000 7fa04fc5e700 20 client.42502.objecter _get_session s=0x7fa04811cd30 osd=5 3 2024-02-23T00:43:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.566+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa04811cd30 osd=5 3 2024-02-23T00:43:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.566+0000 7fa04fc5e700 20 client.42502.objecter _get_session s=0x7fa04811cd30 osd=5 3 2024-02-23T00:43:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.566+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_remove -1 11 2024-02-23T00:43:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.566+0000 7fa04fc5e700 20 client.42502.objecter get_session s=0x7fa04811cd30 osd=5 3 2024-02-23T00:43:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.566+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_assign 5 11 2024-02-23T00:43:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.566+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa04811cd30 osd=5 4 2024-02-23T00:43:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.566+0000 7fa04fc5e700 10 client.42502.objecter _send_command 11 2024-02-23T00:43:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.566+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa04812b290 con 0x7fa048122b90 2024-02-23T00:43:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.567+0000 7fa04d1f9700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fa048122b90 0x7fa048122ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.567+0000 7fa03effd700 10 client.42502.objecter ms_handle_connect 0x7fa048122b90 2024-02-23T00:43:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.567+0000 7fa04d1f9700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fa048122b90 0x7fa048122ff0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.568+0000 7fa03effd700 1 -- 172.21.15.47:0/1097988865 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa04812b290 con 0x7fa048122b90 2024-02-23T00:43:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.568+0000 7fa03effd700 10 client.42502.objecter ms_dispatch 0x7fa048001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:43:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.568+0000 7fa03effd700 10 client.42502.objecter _finish_command 11 = system:0 2024-02-23T00:43:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.568+0000 7fa03effd700 20 client.42502.objecter put_session s=0x7fa04811cd30 osd=5 4 2024-02-23T00:43:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.568+0000 7fa03effd700 15 client.42502.objecter _session_command_op_remove 5 11 2024-02-23T00:43:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.582+0000 7fa04fc5e700 10 client.42502.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:43:50.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.582+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_assign -1 12 2024-02-23T00:43:50.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.582+0000 7fa04fc5e700 20 client.42502.objecter _get_session s=0x7fa04811cd30 osd=5 3 2024-02-23T00:43:50.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.582+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa04811cd30 osd=5 3 2024-02-23T00:43:50.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.582+0000 7fa04fc5e700 20 client.42502.objecter _get_session s=0x7fa04811cd30 osd=5 3 2024-02-23T00:43:50.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.582+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_remove -1 12 2024-02-23T00:43:50.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.582+0000 7fa04fc5e700 20 client.42502.objecter get_session s=0x7fa04811cd30 osd=5 3 2024-02-23T00:43:50.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.582+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_assign 5 12 2024-02-23T00:43:50.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.582+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa04811cd30 osd=5 4 2024-02-23T00:43:50.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.582+0000 7fa04fc5e700 10 client.42502.objecter _send_command 12 2024-02-23T00:43:50.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.582+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa048128b30 con 0x7fa048122b90 2024-02-23T00:43:50.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.584+0000 7fa03effd700 1 -- 172.21.15.47:0/1097988865 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa048128b30 con 0x7fa048122b90 2024-02-23T00:43:50.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.584+0000 7fa03effd700 10 client.42502.objecter ms_dispatch 0x7fa048001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:43:50.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.584+0000 7fa03effd700 10 client.42502.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:43:50.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.584+0000 7fa03effd700 20 client.42502.objecter put_session s=0x7fa04811cd30 osd=5 4 2024-02-23T00:43:50.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.584+0000 7fa03effd700 15 client.42502.objecter _session_command_op_remove 5 12 2024-02-23T00:43:50.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.584+0000 7fa04fc5e700 10 client.42502.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:50.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.584+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_assign -1 13 2024-02-23T00:43:50.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.584+0000 7fa04fc5e700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fa048129260 0x7fa04812b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:50.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.584+0000 7fa04fc5e700 20 client.42502.objecter _get_session s=0x7fa048128b30 osd=6 3 2024-02-23T00:43:50.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.584+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa048128b30 osd=6 3 2024-02-23T00:43:50.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.584+0000 7fa04fc5e700 20 client.42502.objecter _get_session s=0x7fa048128b30 osd=6 3 2024-02-23T00:43:50.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.584+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_remove -1 13 2024-02-23T00:43:50.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.584+0000 7fa04fc5e700 20 client.42502.objecter get_session s=0x7fa048128b30 osd=6 3 2024-02-23T00:43:50.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.584+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_assign 6 13 2024-02-23T00:43:50.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.584+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa048128b30 osd=6 4 2024-02-23T00:43:50.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.584+0000 7fa04fc5e700 10 client.42502.objecter _send_command 13 2024-02-23T00:43:50.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.584+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa04812e4f0 con 0x7fa048129260 2024-02-23T00:43:50.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.585+0000 7fa04e1fb700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fa048129260 0x7fa04812b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:50.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.585+0000 7fa04e1fb700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fa048129260 0x7fa04812b6b0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:50.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.585+0000 7fa03effd700 10 client.42502.objecter ms_handle_connect 0x7fa048129260 2024-02-23T00:43:50.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.586+0000 7fa03effd700 1 -- 172.21.15.47:0/1097988865 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa04812e4f0 con 0x7fa048129260 2024-02-23T00:43:50.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.586+0000 7fa03effd700 10 client.42502.objecter ms_dispatch 0x7fa048001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:43:50.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.586+0000 7fa03effd700 10 client.42502.objecter _finish_command 13 = system:0 2024-02-23T00:43:50.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.586+0000 7fa03effd700 20 client.42502.objecter put_session s=0x7fa048128b30 osd=6 4 2024-02-23T00:43:50.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.586+0000 7fa03effd700 15 client.42502.objecter _session_command_op_remove 6 13 2024-02-23T00:43:50.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.600+0000 7fa04fc5e700 10 client.42502.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:43:50.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.600+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_assign -1 14 2024-02-23T00:43:50.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.600+0000 7fa04fc5e700 20 client.42502.objecter _get_session s=0x7fa048128b30 osd=6 3 2024-02-23T00:43:50.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.600+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa048128b30 osd=6 3 2024-02-23T00:43:50.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.600+0000 7fa04fc5e700 20 client.42502.objecter _get_session s=0x7fa048128b30 osd=6 3 2024-02-23T00:43:50.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.600+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_remove -1 14 2024-02-23T00:43:50.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.600+0000 7fa04fc5e700 20 client.42502.objecter get_session s=0x7fa048128b30 osd=6 3 2024-02-23T00:43:50.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.600+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_assign 6 14 2024-02-23T00:43:50.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.600+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa048128b30 osd=6 4 2024-02-23T00:43:50.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.600+0000 7fa04fc5e700 10 client.42502.objecter _send_command 14 2024-02-23T00:43:50.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.600+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa04812e4f0 con 0x7fa048129260 2024-02-23T00:43:50.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.602+0000 7fa03effd700 1 -- 172.21.15.47:0/1097988865 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa04812e4f0 con 0x7fa048129260 2024-02-23T00:43:50.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.602+0000 7fa03effd700 10 client.42502.objecter ms_dispatch 0x7fa048001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:43:50.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.602+0000 7fa03effd700 10 client.42502.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:43:50.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.602+0000 7fa03effd700 20 client.42502.objecter put_session s=0x7fa048128b30 osd=6 4 2024-02-23T00:43:50.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.602+0000 7fa03effd700 15 client.42502.objecter _session_command_op_remove 6 14 2024-02-23T00:43:50.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.602+0000 7fa04fc5e700 10 client.42502.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:50.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.602+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_assign -1 15 2024-02-23T00:43:50.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.602+0000 7fa04fc5e700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fa04812d820 0x7fa04812dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:50.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.602+0000 7fa04fc5e700 20 client.42502.objecter _get_session s=0x7fa04812e4f0 osd=7 3 2024-02-23T00:43:50.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.602+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa04812e4f0 osd=7 3 2024-02-23T00:43:50.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.602+0000 7fa04fc5e700 20 client.42502.objecter _get_session s=0x7fa04812e4f0 osd=7 3 2024-02-23T00:43:50.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.602+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_remove -1 15 2024-02-23T00:43:50.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.602+0000 7fa04fc5e700 20 client.42502.objecter get_session s=0x7fa04812e4f0 osd=7 3 2024-02-23T00:43:50.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.602+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_assign 7 15 2024-02-23T00:43:50.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.602+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa04812e4f0 osd=7 4 2024-02-23T00:43:50.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.602+0000 7fa04fc5e700 10 client.42502.objecter _send_command 15 2024-02-23T00:43:50.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.602+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa048131870 con 0x7fa04812d820 2024-02-23T00:43:50.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.602+0000 7fa04d9fa700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fa04812d820 0x7fa04812dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:50.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.603+0000 7fa03effd700 10 client.42502.objecter ms_handle_connect 0x7fa04812d820 2024-02-23T00:43:50.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.603+0000 7fa04d9fa700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fa04812d820 0x7fa04812dc80 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:50.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.603+0000 7fa03effd700 1 -- 172.21.15.47:0/1097988865 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa048131870 con 0x7fa04812d820 2024-02-23T00:43:50.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.603+0000 7fa03effd700 10 client.42502.objecter ms_dispatch 0x7fa048001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:43:50.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.603+0000 7fa03effd700 10 client.42502.objecter _finish_command 15 = system:0 2024-02-23T00:43:50.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.603+0000 7fa03effd700 20 client.42502.objecter put_session s=0x7fa04812e4f0 osd=7 4 2024-02-23T00:43:50.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.603+0000 7fa03effd700 15 client.42502.objecter _session_command_op_remove 7 15 2024-02-23T00:43:50.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.620+0000 7fa04fc5e700 10 client.42502.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:43:50.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.620+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_assign -1 16 2024-02-23T00:43:50.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.620+0000 7fa04fc5e700 20 client.42502.objecter _get_session s=0x7fa04812e4f0 osd=7 3 2024-02-23T00:43:50.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.620+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa04812e4f0 osd=7 3 2024-02-23T00:43:50.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.620+0000 7fa04fc5e700 20 client.42502.objecter _get_session s=0x7fa04812e4f0 osd=7 3 2024-02-23T00:43:50.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.620+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_remove -1 16 2024-02-23T00:43:50.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.620+0000 7fa04fc5e700 20 client.42502.objecter get_session s=0x7fa04812e4f0 osd=7 3 2024-02-23T00:43:50.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.620+0000 7fa04fc5e700 15 client.42502.objecter _session_command_op_assign 7 16 2024-02-23T00:43:50.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.620+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa04812e4f0 osd=7 4 2024-02-23T00:43:50.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.620+0000 7fa04fc5e700 10 client.42502.objecter _send_command 16 2024-02-23T00:43:50.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.620+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa04812bcb0 con 0x7fa04812d820 2024-02-23T00:43:50.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.621+0000 7fa03effd700 1 -- 172.21.15.47:0/1097988865 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa04812bcb0 con 0x7fa04812d820 2024-02-23T00:43:50.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.621+0000 7fa03effd700 10 client.42502.objecter ms_dispatch 0x7fa048001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:43:50.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.621+0000 7fa03effd700 10 client.42502.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:43:50.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.621+0000 7fa03effd700 20 client.42502.objecter put_session s=0x7fa04812e4f0 osd=7 4 2024-02-23T00:43:50.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.621+0000 7fa03effd700 15 client.42502.objecter _session_command_op_remove 7 16 2024-02-23T00:43:50.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 10 librados: watch_flush enter 2024-02-23T00:43:50.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 10 librados: watch_flush exit 2024-02-23T00:43:50.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 10 client.42502.objecter close_session for osd.0 2024-02-23T00:43:50.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fa0480fee80 msgr2=0x7fa0480ff260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:50.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fa0480fee80 0x7fa0480ff260 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:50.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa04805ee80 osd=0 1 2024-02-23T00:43:50.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 10 client.42502.objecter close_session for osd.1 2024-02-23T00:43:50.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fa0480598f0 msgr2=0x7fa048059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:50.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fa0480598f0 0x7fa048059cd0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:50.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa048028710 osd=1 1 2024-02-23T00:43:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 10 client.42502.objecter close_session for osd.2 2024-02-23T00:43:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa048028270 msgr2=0x7fa04811c0d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa048028270 0x7fa04811c0d0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa048059770 osd=2 1 2024-02-23T00:43:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 10 client.42502.objecter close_session for osd.3 2024-02-23T00:43:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fa04811f830 msgr2=0x7fa048121c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fa04811f830 0x7fa048121c20 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04e1fb700 1 -- 172.21.15.47:0/1097988865 reap_dead start 2024-02-23T00:43:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa04811cf00 osd=3 1 2024-02-23T00:43:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 10 client.42502.objecter close_session for osd.4 2024-02-23T00:43:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fa04811bc30 msgr2=0x7fa048122110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fa04811bc30 0x7fa048122110 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04e1fb700 1 -- 172.21.15.47:0/1097988865 reap_dead start 2024-02-23T00:43:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa04811cb10 osd=4 1 2024-02-23T00:43:50.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 10 client.42502.objecter close_session for osd.5 2024-02-23T00:43:50.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fa048122b90 msgr2=0x7fa048122ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:50.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fa048122b90 0x7fa048122ff0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:50.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa04811cd30 osd=5 1 2024-02-23T00:43:50.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 10 client.42502.objecter close_session for osd.6 2024-02-23T00:43:50.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fa048129260 msgr2=0x7fa04812b6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:50.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fa048129260 0x7fa04812b6b0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:50.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa048128b30 osd=6 1 2024-02-23T00:43:50.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 10 client.42502.objecter close_session for osd.7 2024-02-23T00:43:50.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fa04812d820 msgr2=0x7fa04812dc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:50.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fa04812d820 0x7fa04812dc80 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:50.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 20 client.42502.objecter put_session s=0x7fa04812e4f0 osd=7 1 2024-02-23T00:43:50.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 20 client.42502.objecter shutdown clearing up homeless session... 2024-02-23T00:43:50.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 10 client.42502.objecter successfully canceled tick 2024-02-23T00:43:50.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa034062cc0 msgr2=0x7fa034065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:50.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa034062cc0 0x7fa034065180 secure :-1 s=READY pgs=2899 cs=0 l=1 rev1=1 rx=0x7fa04806c820 tx=0x7fa03800b040).stop 2024-02-23T00:43:50.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0480f8420 msgr2=0x7fa04806b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:50.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04e1fb700 1 -- 172.21.15.47:0/1097988865 reap_dead start 2024-02-23T00:43:50.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0480f8420 0x7fa04806b0d0 secure :-1 s=READY pgs=3939 cs=0 l=1 rev1=1 rx=0x7fa044002780 tx=0x7fa044009e40).stop 2024-02-23T00:43:50.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 shutdown_connections 2024-02-23T00:43:50.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 1 --2- 172.21.15.47:0/1097988865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0480f8420 0x7fa04806b0d0 unknown :-1 s=CLOSED pgs=3939 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:50.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 >> 172.21.15.47:0/1097988865 conn(0x7fa0480f3800 msgr2=0x7fa0480f5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:43:50.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 shutdown_connections 2024-02-23T00:43:50.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 1 -- 172.21.15.47:0/1097988865 wait complete. 2024-02-23T00:43:50.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:50.622+0000 7fa04fc5e700 1 librados: shutdown 2024-02-23T00:43:50.931 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:43:50.931 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:43:50.931 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:43:50.932 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:43:50.932 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:43:50.932 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:43:50.932 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:43:50.932 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:43:50.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:49.137825+0000 osd.4 (osd.4) 85 : cluster [DBG] 4.18 scrub starts 2024-02-23T00:43:50.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:49.139151+0000 osd.4 (osd.4) 86 : cluster [DBG] 4.18 scrub ok 2024-02-23T00:43:50.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:50.093665+0000 mon.a (mon.0) 1223 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi047-82386-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:50.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:50.093784+0000 mon.a (mon.0) 1224 : cluster [DBG] osdmap e158: 8 total, 8 up, 8 in 2024-02-23T00:43:50.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:50.393560+0000 mon.b (mon.1) 184 : audit [DBG] from='client.? 172.21.15.47:0/1097988865' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:43:50.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:50.718686+0000 mon.b (mon.1) 185 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:51.056 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:49.137825+0000 osd.4 (osd.4) 85 : cluster [DBG] 4.18 scrub starts 2024-02-23T00:43:51.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:49.139151+0000 osd.4 (osd.4) 86 : cluster [DBG] 4.18 scrub ok 2024-02-23T00:43:51.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:50.093665+0000 mon.a (mon.0) 1223 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi047-82386-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:51.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:50.093784+0000 mon.a (mon.0) 1224 : cluster [DBG] osdmap e158: 8 total, 8 up, 8 in 2024-02-23T00:43:51.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:50.393560+0000 mon.b (mon.1) 184 : audit [DBG] from='client.? 172.21.15.47:0/1097988865' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:43:51.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:50.718686+0000 mon.b (mon.1) 185 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:51.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:49.137825+0000 osd.4 (osd.4) 85 : cluster [DBG] 4.18 scrub starts 2024-02-23T00:43:51.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:49.139151+0000 osd.4 (osd.4) 86 : cluster [DBG] 4.18 scrub ok 2024-02-23T00:43:51.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:50.093665+0000 mon.a (mon.0) 1223 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi047-82386-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:51.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:50.093784+0000 mon.a (mon.0) 1224 : cluster [DBG] osdmap e158: 8 total, 8 up, 8 in 2024-02-23T00:43:51.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:50.393560+0000 mon.b (mon.1) 184 : audit [DBG] from='client.? 172.21.15.47:0/1097988865' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:43:51.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:50.718686+0000 mon.b (mon.1) 185 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:51.485 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:43:51.492 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:51.492 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:43:51.593 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:43:51.621 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:51.639 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:43:51.642 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:51.642 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:43:51.743 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:43:51.772 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:51.845 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:43:51.849 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:51.850 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:43:51.951 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:43:51.981 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:52.141 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:43:52.145 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:52.145 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:43:52.246 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:43:52.273 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:52.291 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:43:52.294 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:52.294 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:43:52.395 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:43:52.419 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:43:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:43:52.419 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:50.392625+0000 osd.6 (osd.6) 73 : cluster [DBG] 5.1f deep-scrub starts 2024-02-23T00:43:52.419 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:50.393704+0000 osd.6 (osd.6) 74 : cluster [DBG] 5.1f deep-scrub ok 2024-02-23T00:43:52.419 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:51.078007+0000 osd.3 (osd.3) 105 : cluster [DBG] 4.1d scrub starts 2024-02-23T00:43:52.420 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:51.079752+0000 osd.3 (osd.3) 106 : cluster [DBG] 4.1d scrub ok 2024-02-23T00:43:52.420 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:51.095925+0000 mon.a (mon.0) 1225 : cluster [DBG] osdmap e159: 8 total, 8 up, 8 in 2024-02-23T00:43:52.420 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:51.109067+0000 mon.a (mon.0) 1226 : audit [INF] from='client.? 172.21.15.47:0/3670462155' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi047-82378-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:52.420 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:51.214042+0000 mgr.y (mgr.14517) 202 : cluster [DBG] pgmap v214: 354 pgs: 282 active+clean, 32 creating+peering, 40 unknown; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:43:52.420 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:51.260630+0000 osd.2 (osd.2) 79 : cluster [DBG] 5.8 scrub starts 2024-02-23T00:43:52.420 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:51.261363+0000 osd.2 (osd.2) 80 : cluster [DBG] 5.8 scrub ok 2024-02-23T00:43:52.420 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:51.578299+0000 mon.a (mon.0) 1227 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:43:52.420 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:51.719731+0000 mon.b (mon.1) 186 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:52.421 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:52.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.420+0000 7fcd15fec700 1 -- 172.21.15.47:0/103092359 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcd100fc240 msgr2=0x7fcd100fc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:52.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.420+0000 7fcd15fec700 1 --2- 172.21.15.47:0/103092359 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcd100fc240 0x7fcd100fc620 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 rx=0x7fcd00004970 tx=0x7fcd0000adc0).stop 2024-02-23T00:43:52.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.420+0000 7fcd15fec700 1 -- 172.21.15.47:0/103092359 shutdown_connections 2024-02-23T00:43:52.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.420+0000 7fcd15fec700 1 --2- 172.21.15.47:0/103092359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd100f6b10 0x7fcd101030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:52.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.420+0000 7fcd15fec700 1 --2- 172.21.15.47:0/103092359 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcd100fc240 0x7fcd100fc620 unknown :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:52.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.420+0000 7fcd15fec700 1 --2- 172.21.15.47:0/103092359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd100fcba0 0x7fcd100f6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:52.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.420+0000 7fcd15fec700 1 -- 172.21.15.47:0/103092359 >> 172.21.15.47:0/103092359 conn(0x7fcd1006c370 msgr2=0x7fcd1006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:43:52.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.420+0000 7fcd15fec700 1 -- 172.21.15.47:0/103092359 shutdown_connections 2024-02-23T00:43:52.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.420+0000 7fcd15fec700 1 -- 172.21.15.47:0/103092359 wait complete. 2024-02-23T00:43:52.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.421+0000 7fcd15fec700 1 librados: starting msgr at 2024-02-23T00:43:52.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.421+0000 7fcd15fec700 1 librados: starting objecter 2024-02-23T00:43:52.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.421+0000 7fcd15fec700 1 Processor -- start 2024-02-23T00:43:52.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.421+0000 7fcd15fec700 1 -- start start 2024-02-23T00:43:52.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.421+0000 7fcd15fec700 1 librados: setting wanted keys 2024-02-23T00:43:52.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.421+0000 7fcd15fec700 1 librados: calling monclient init 2024-02-23T00:43:52.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.421+0000 7fcd15fec700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcd100f6b10 0x7fcd1010be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:52.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.421+0000 7fcd15fec700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd100fc240 0x7fcd1010c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:52.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.421+0000 7fcd15fec700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd100fcba0 0x7fcd10106000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:52.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.421+0000 7fcd15fec700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd101064f0 con 0x7fcd100fcba0 2024-02-23T00:43:52.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.421+0000 7fcd15fec700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd10106660 con 0x7fcd100fc240 2024-02-23T00:43:52.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.421+0000 7fcd15fec700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd10106820 con 0x7fcd100f6b10 2024-02-23T00:43:52.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.421+0000 7fcd0f7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcd100f6b10 0x7fcd1010be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:52.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.421+0000 7fcd0f7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcd100f6b10 0x7fcd1010be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:35362/0 (socket says 172.21.15.47:35362) 2024-02-23T00:43:52.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.421+0000 7fcd0f7fe700 1 -- 172.21.15.47:0/2968758094 learned_addr learned my addr 172.21.15.47:0/2968758094 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:43:52.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.421+0000 7fcd0ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd100fcba0 0x7fcd10106000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:52.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.421+0000 7fcd0effd700 1 --2- 172.21.15.47:0/2968758094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd100fc240 0x7fcd1010c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:52.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.422+0000 7fcd0f7fe700 1 -- 172.21.15.47:0/2968758094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd100fc240 msgr2=0x7fcd1010c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:52.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.422+0000 7fcd0f7fe700 1 --2- 172.21.15.47:0/2968758094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd100fc240 0x7fcd1010c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:52.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.422+0000 7fcd0f7fe700 1 -- 172.21.15.47:0/2968758094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd100fcba0 msgr2=0x7fcd10106000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:52.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.422+0000 7fcd0f7fe700 1 --2- 172.21.15.47:0/2968758094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd100fcba0 0x7fcd10106000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:52.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.422+0000 7fcd0f7fe700 1 -- 172.21.15.47:0/2968758094 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd00005040 con 0x7fcd100f6b10 2024-02-23T00:43:52.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.422+0000 7fcd0ffff700 1 --2- 172.21.15.47:0/2968758094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd100fcba0 0x7fcd10106000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:43:52.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.422+0000 7fcd0f7fe700 1 --2- 172.21.15.47:0/2968758094 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcd100f6b10 0x7fcd1010be10 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 rx=0x7fcd000043a0 tx=0x7fcd0000adc0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:52.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.422+0000 7fcd0cff9700 10 client.?.objecter ms_handle_connect 0x7fcd100f6b10 2024-02-23T00:43:52.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.422+0000 7fcd0cff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:43:52.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.422+0000 7fcd0cff9700 1 -- 172.21.15.47:0/2968758094 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0000f030 con 0x7fcd100f6b10 2024-02-23T00:43:52.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.422+0000 7fcd0cff9700 1 -- 172.21.15.47:0/2968758094 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fcd00008dc0 con 0x7fcd100f6b10 2024-02-23T00:43:52.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.422+0000 7fcd15fec700 1 -- 172.21.15.47:0/2968758094 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd10106aa0 con 0x7fcd100f6b10 2024-02-23T00:43:52.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.422+0000 7fcd0cff9700 1 -- 172.21.15.47:0/2968758094 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd00019070 con 0x7fcd100f6b10 2024-02-23T00:43:52.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.422+0000 7fcd15fec700 10 client.40412.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:43:52.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.422+0000 7fcd15fec700 1 -- 172.21.15.47:0/2968758094 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd10112c40 con 0x7fcd100f6b10 2024-02-23T00:43:52.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.422+0000 7fcd15fec700 1 librados: init done 2024-02-23T00:43:52.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.423+0000 7fcd15fec700 1 -- 172.21.15.47:0/2968758094 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd1005fa40 con 0x7fcd100f6b10 2024-02-23T00:43:52.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.423+0000 7fcd0cff9700 1 -- 172.21.15.47:0/2968758094 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fcd0000a2e0 con 0x7fcd100f6b10 2024-02-23T00:43:52.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.424+0000 7fcd0cff9700 1 --2- 172.21.15.47:0/2968758094 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fccfc062cc0 0x7fccfc065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:52.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.424+0000 7fcd0cff9700 1 -- 172.21.15.47:0/2968758094 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(160..160 src has 48..160) v4 ==== 11703+0+0 (secure 0 0 0) 0x7fcd0007fd50 con 0x7fcd100f6b10 2024-02-23T00:43:52.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.424+0000 7fcd0cff9700 10 client.40412.objecter ms_dispatch 0x7fcd10001550 osd_map(160..160 src has 48..160) v4 2024-02-23T00:43:52.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.424+0000 7fcd0cff9700 3 client.40412.objecter handle_osd_map got epochs [160,160] > 0 2024-02-23T00:43:52.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.424+0000 7fcd0cff9700 3 client.40412.objecter handle_osd_map decoding full epoch 160 2024-02-23T00:43:52.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.424+0000 7fcd0cff9700 20 client.40412.objecter dump_active .. 0 homeless 2024-02-23T00:43:52.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.424+0000 7fcd0effd700 1 --2- 172.21.15.47:0/2968758094 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fccfc062cc0 0x7fccfc065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:52.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.425+0000 7fcd0cff9700 10 client.40412.objecter ms_handle_connect 0x7fccfc062cc0 2024-02-23T00:43:52.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.425+0000 7fcd0effd700 1 --2- 172.21.15.47:0/2968758094 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fccfc062cc0 0x7fccfc065180 secure :-1 s=READY pgs=2901 cs=0 l=1 rev1=1 rx=0x7fcd10107750 tx=0x7fccf800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:52.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.429+0000 7fcd0cff9700 1 -- 172.21.15.47:0/2968758094 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fcd00083030 con 0x7fcd100f6b10 2024-02-23T00:43:52.439 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:43:52.442 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:52.442 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:43:52.475 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:43:52.543 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:43:52.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:50.392625+0000 osd.6 (osd.6) 73 : cluster [DBG] 5.1f deep-scrub starts 2024-02-23T00:43:52.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:50.393704+0000 osd.6 (osd.6) 74 : cluster [DBG] 5.1f deep-scrub ok 2024-02-23T00:43:52.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:51.078007+0000 osd.3 (osd.3) 105 : cluster [DBG] 4.1d scrub starts 2024-02-23T00:43:52.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:51.079752+0000 osd.3 (osd.3) 106 : cluster [DBG] 4.1d scrub ok 2024-02-23T00:43:52.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:51.095925+0000 mon.a (mon.0) 1225 : cluster [DBG] osdmap e159: 8 total, 8 up, 8 in 2024-02-23T00:43:52.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:51.109067+0000 mon.a (mon.0) 1226 : audit [INF] from='client.? 172.21.15.47:0/3670462155' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi047-82378-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:52.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:51.214042+0000 mgr.y (mgr.14517) 202 : cluster [DBG] pgmap v214: 354 pgs: 282 active+clean, 32 creating+peering, 40 unknown; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:43:52.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:51.260630+0000 osd.2 (osd.2) 79 : cluster [DBG] 5.8 scrub starts 2024-02-23T00:43:52.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:51.261363+0000 osd.2 (osd.2) 80 : cluster [DBG] 5.8 scrub ok 2024-02-23T00:43:52.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:51.578299+0000 mon.a (mon.0) 1227 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:43:52.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:51.719731+0000 mon.b (mon.1) 186 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:52.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:50.392625+0000 osd.6 (osd.6) 73 : cluster [DBG] 5.1f deep-scrub starts 2024-02-23T00:43:52.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:50.393704+0000 osd.6 (osd.6) 74 : cluster [DBG] 5.1f deep-scrub ok 2024-02-23T00:43:52.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:51.078007+0000 osd.3 (osd.3) 105 : cluster [DBG] 4.1d scrub starts 2024-02-23T00:43:52.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:51.079752+0000 osd.3 (osd.3) 106 : cluster [DBG] 4.1d scrub ok 2024-02-23T00:43:52.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:51.095925+0000 mon.a (mon.0) 1225 : cluster [DBG] osdmap e159: 8 total, 8 up, 8 in 2024-02-23T00:43:52.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:51.109067+0000 mon.a (mon.0) 1226 : audit [INF] from='client.? 172.21.15.47:0/3670462155' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi047-82378-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:52.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:51.214042+0000 mgr.y (mgr.14517) 202 : cluster [DBG] pgmap v214: 354 pgs: 282 active+clean, 32 creating+peering, 40 unknown; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:43:52.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:51.260630+0000 osd.2 (osd.2) 79 : cluster [DBG] 5.8 scrub starts 2024-02-23T00:43:52.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:51.261363+0000 osd.2 (osd.2) 80 : cluster [DBG] 5.8 scrub ok 2024-02-23T00:43:52.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:51.578299+0000 mon.a (mon.0) 1227 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:43:52.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:51.719731+0000 mon.b (mon.1) 186 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:52.569 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:52.587 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:43:52.590 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:52.590 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:43:52.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:52.621+0000 7fcd15fec700 1 -- 172.21.15.47:0/2968758094 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd reweight", "id": 0, "weight": 1.0} v 0) v1 -- 0x7fcd10104ce0 con 0x7fcd100f6b10 2024-02-23T00:43:52.690 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:43:52.693 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:43:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:43:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:43:52.720 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:53.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:53.103+0000 7fcd0cff9700 1 -- 172.21.15.47:0/2968758094 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 0, "weight": 1.0}]=0 reweighted osd.0 to 1 (10000) v161) v1 ==== 113+0+0 (secure 0 0 0) 0x7fcd0004ea30 con 0x7fcd100f6b10 2024-02-23T00:43:53.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:53.106+0000 7fcd15fec700 10 librados: watch_flush enter 2024-02-23T00:43:53.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:53.106+0000 7fcd15fec700 10 librados: watch_flush exit 2024-02-23T00:43:53.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:53.106+0000 7fcd15fec700 20 client.40412.objecter shutdown clearing up homeless session... 2024-02-23T00:43:53.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:53.106+0000 7fcd15fec700 10 client.40412.objecter successfully canceled tick 2024-02-23T00:43:53.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:53.107+0000 7fcd15fec700 1 -- 172.21.15.47:0/2968758094 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fccfc062cc0 msgr2=0x7fccfc065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:53.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:53.107+0000 7fcd15fec700 1 --2- 172.21.15.47:0/2968758094 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fccfc062cc0 0x7fccfc065180 secure :-1 s=READY pgs=2901 cs=0 l=1 rev1=1 rx=0x7fcd10107750 tx=0x7fccf800b040).stop 2024-02-23T00:43:53.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:53.107+0000 7fcd15fec700 1 -- 172.21.15.47:0/2968758094 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcd100f6b10 msgr2=0x7fcd1010be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:53.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:53.107+0000 7fcd15fec700 1 --2- 172.21.15.47:0/2968758094 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcd100f6b10 0x7fcd1010be10 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 rx=0x7fcd000043a0 tx=0x7fcd0000adc0).stop 2024-02-23T00:43:53.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:53.110+0000 7fcd15fec700 1 -- 172.21.15.47:0/2968758094 shutdown_connections 2024-02-23T00:43:53.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:53.110+0000 7fcd15fec700 1 --2- 172.21.15.47:0/2968758094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd100fcba0 0x7fcd10106000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:53.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:53.110+0000 7fcd15fec700 1 --2- 172.21.15.47:0/2968758094 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fccfc062cc0 0x7fccfc065180 unknown :-1 s=CLOSED pgs=2901 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:53.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:53.110+0000 7fcd15fec700 1 --2- 172.21.15.47:0/2968758094 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcd100f6b10 0x7fcd1010be10 unknown :-1 s=CLOSED pgs=2469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:53.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:53.110+0000 7fcd15fec700 1 --2- 172.21.15.47:0/2968758094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd100fc240 0x7fcd1010c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:53.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:53.110+0000 7fcd15fec700 1 -- 172.21.15.47:0/2968758094 >> 172.21.15.47:0/2968758094 conn(0x7fcd1006c370 msgr2=0x7fcd100fa070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:43:53.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:53.119+0000 7fcd15fec700 1 -- 172.21.15.47:0/2968758094 shutdown_connections 2024-02-23T00:43:53.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:53.120+0000 7fcd15fec700 1 -- 172.21.15.47:0/2968758094 wait complete. 2024-02-23T00:43:53.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:53.120+0000 7fcd15fec700 1 librados: shutdown 2024-02-23T00:43:53.127 INFO:teuthology.orchestra.run.smithi047.stderr:reweighted osd.0 to 1 (10000) 2024-02-23T00:43:53.201 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:51.391182+0000 osd.6 (osd.6) 75 : cluster [DBG] 2.7 deep-scrub starts 2024-02-23T00:43:53.202 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:51.391934+0000 osd.6 (osd.6) 76 : cluster [DBG] 2.7 deep-scrub ok 2024-02-23T00:43:53.202 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:52.035222+0000 osd.3 (osd.3) 107 : cluster [DBG] 6.b scrub starts 2024-02-23T00:43:53.202 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:52.036246+0000 osd.3 (osd.3) 108 : cluster [DBG] 6.b scrub ok 2024-02-23T00:43:53.202 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:52.104291+0000 mon.a (mon.0) 1228 : audit [INF] from='client.? 172.21.15.47:0/3670462155' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi047-82378-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:53.202 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:52.104516+0000 mon.a (mon.0) 1229 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:43:53.202 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:52.104583+0000 mon.a (mon.0) 1230 : cluster [DBG] osdmap e160: 8 total, 8 up, 8 in 2024-02-23T00:43:53.202 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:52.110520+0000 mon.a (mon.0) 1231 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-13"}]: dispatch 2024-02-23T00:43:53.202 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:52.119091+0000 osd.4 (osd.4) 87 : cluster [DBG] 4.1a scrub starts 2024-02-23T00:43:53.202 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:52.121869+0000 mon.b (mon.1) 187 : audit [INF] from='client.? 172.21.15.47:0/2817654023' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi047-82386-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:53.202 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:52.123621+0000 mon.a (mon.0) 1232 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi047-82386-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:53.202 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:52.124239+0000 osd.4 (osd.4) 88 : cluster [DBG] 4.1a scrub ok 2024-02-23T00:43:53.203 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:52.308144+0000 osd.0 (osd.0) 87 : cluster [DBG] 4.16 scrub starts 2024-02-23T00:43:53.203 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:52.309528+0000 osd.0 (osd.0) 88 : cluster [DBG] 4.16 scrub ok 2024-02-23T00:43:53.203 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:52.349023+0000 mgr.y (mgr.14517) 203 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:43:53.203 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:52.623389+0000 mon.c (mon.2) 83 : audit [INF] from='client.? 172.21.15.47:0/2968758094' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-23T00:43:53.203 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:52.623685+0000 mon.a (mon.0) 1233 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-23T00:43:53.203 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:52.720977+0000 mon.b (mon.1) 188 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:53.203 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:51.391182+0000 osd.6 (osd.6) 75 : cluster [DBG] 2.7 deep-scrub starts 2024-02-23T00:43:53.203 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:51.391934+0000 osd.6 (osd.6) 76 : cluster [DBG] 2.7 deep-scrub ok 2024-02-23T00:43:53.203 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:52.035222+0000 osd.3 (osd.3) 107 : cluster [DBG] 6.b scrub starts 2024-02-23T00:43:53.203 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:52.036246+0000 osd.3 (osd.3) 108 : cluster [DBG] 6.b scrub ok 2024-02-23T00:43:53.203 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:52.104291+0000 mon.a (mon.0) 1228 : audit [INF] from='client.? 172.21.15.47:0/3670462155' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi047-82378-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:53.203 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:52.104516+0000 mon.a (mon.0) 1229 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:43:53.204 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:52.104583+0000 mon.a (mon.0) 1230 : cluster [DBG] osdmap e160: 8 total, 8 up, 8 in 2024-02-23T00:43:53.204 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:52.110520+0000 mon.a (mon.0) 1231 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-13"}]: dispatch 2024-02-23T00:43:53.204 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:52.119091+0000 osd.4 (osd.4) 87 : cluster [DBG] 4.1a scrub starts 2024-02-23T00:43:53.204 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:52.121869+0000 mon.b (mon.1) 187 : audit [INF] from='client.? 172.21.15.47:0/2817654023' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi047-82386-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:53.204 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:52.123621+0000 mon.a (mon.0) 1232 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi047-82386-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:53.204 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:52.124239+0000 osd.4 (osd.4) 88 : cluster [DBG] 4.1a scrub ok 2024-02-23T00:43:53.204 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:52.308144+0000 osd.0 (osd.0) 87 : cluster [DBG] 4.16 scrub starts 2024-02-23T00:43:53.204 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:52.309528+0000 osd.0 (osd.0) 88 : cluster [DBG] 4.16 scrub ok 2024-02-23T00:43:53.204 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:52.349023+0000 mgr.y (mgr.14517) 203 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:43:53.204 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:52.623389+0000 mon.c (mon.2) 83 : audit [INF] from='client.? 172.21.15.47:0/2968758094' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-23T00:43:53.204 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:52.623685+0000 mon.a (mon.0) 1233 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-23T00:43:53.204 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:52.720977+0000 mon.b (mon.1) 188 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:53.336 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.4 dump_ops_in_flight 2024-02-23T00:43:53.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:51.391182+0000 osd.6 (osd.6) 75 : cluster [DBG] 2.7 deep-scrub starts 2024-02-23T00:43:53.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:51.391934+0000 osd.6 (osd.6) 76 : cluster [DBG] 2.7 deep-scrub ok 2024-02-23T00:43:53.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:52.035222+0000 osd.3 (osd.3) 107 : cluster [DBG] 6.b scrub starts 2024-02-23T00:43:53.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:52.036246+0000 osd.3 (osd.3) 108 : cluster [DBG] 6.b scrub ok 2024-02-23T00:43:53.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:52.104291+0000 mon.a (mon.0) 1228 : audit [INF] from='client.? 172.21.15.47:0/3670462155' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi047-82378-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:53.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:52.104516+0000 mon.a (mon.0) 1229 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:43:53.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:52.104583+0000 mon.a (mon.0) 1230 : cluster [DBG] osdmap e160: 8 total, 8 up, 8 in 2024-02-23T00:43:53.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:52.110520+0000 mon.a (mon.0) 1231 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-13"}]: dispatch 2024-02-23T00:43:53.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:52.119091+0000 osd.4 (osd.4) 87 : cluster [DBG] 4.1a scrub starts 2024-02-23T00:43:53.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:52.121869+0000 mon.b (mon.1) 187 : audit [INF] from='client.? 172.21.15.47:0/2817654023' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi047-82386-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:53.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:52.123621+0000 mon.a (mon.0) 1232 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi047-82386-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:53.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:52.124239+0000 osd.4 (osd.4) 88 : cluster [DBG] 4.1a scrub ok 2024-02-23T00:43:53.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:52.308144+0000 osd.0 (osd.0) 87 : cluster [DBG] 4.16 scrub starts 2024-02-23T00:43:53.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:52.309528+0000 osd.0 (osd.0) 88 : cluster [DBG] 4.16 scrub ok 2024-02-23T00:43:53.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:52.349023+0000 mgr.y (mgr.14517) 203 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:43:53.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:52.623389+0000 mon.c (mon.2) 83 : audit [INF] from='client.? 172.21.15.47:0/2968758094' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-23T00:43:53.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:52.623685+0000 mon.a (mon.0) 1233 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-23T00:43:53.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:52.720977+0000 mon.b (mon.1) 188 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:53.601 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:43:53.866 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:43:54.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:52.418096+0000 osd.6 (osd.6) 77 : cluster [DBG] 2.17 scrub starts 2024-02-23T00:43:54.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:52.419092+0000 osd.6 (osd.6) 78 : cluster [DBG] 2.17 scrub ok 2024-02-23T00:43:54.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:53.103197+0000 mon.a (mon.0) 1234 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-13"}]': finished 2024-02-23T00:43:54.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:53.103325+0000 mon.a (mon.0) 1235 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi047-82386-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:54.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:53.103439+0000 mon.a (mon.0) 1236 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-02-23T00:43:54.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:53.103497+0000 mon.a (mon.0) 1237 : cluster [DBG] osdmap e161: 8 total, 8 up, 8 in 2024-02-23T00:43:54.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:53.105250+0000 mon.a (mon.0) 1238 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-13", "mode": "writeback"}]: dispatch 2024-02-23T00:43:54.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:53.214491+0000 mgr.y (mgr.14517) 204 : cluster [DBG] pgmap v217: 354 pgs: 322 active+clean, 32 creating+peering; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 131 KiB/s wr, 127 op/s 2024-02-23T00:43:54.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:53.315350+0000 osd.2 (osd.2) 81 : cluster [DBG] 2.c scrub starts 2024-02-23T00:43:54.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:53.317196+0000 osd.2 (osd.2) 82 : cluster [DBG] 2.c scrub ok 2024-02-23T00:43:54.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:53.721892+0000 mon.b (mon.1) 189 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:54.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:54.100883+0000 mon.a (mon.0) 1239 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:43:54.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:54.106123+0000 mon.a (mon.0) 1240 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-13", "mode": "writeback"}]': finished 2024-02-23T00:43:54.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:54.106278+0000 mon.a (mon.0) 1241 : cluster [DBG] osdmap e162: 8 total, 8 up, 8 in 2024-02-23T00:43:54.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:54.107753+0000 mon.c (mon.2) 84 : audit [INF] from='client.? 172.21.15.47:0/3013997716' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi047-82378-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:54.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:52.418096+0000 osd.6 (osd.6) 77 : cluster [DBG] 2.17 scrub starts 2024-02-23T00:43:54.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:52.419092+0000 osd.6 (osd.6) 78 : cluster [DBG] 2.17 scrub ok 2024-02-23T00:43:54.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:53.103197+0000 mon.a (mon.0) 1234 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-13"}]': finished 2024-02-23T00:43:54.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:53.103325+0000 mon.a (mon.0) 1235 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi047-82386-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:54.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:53.103439+0000 mon.a (mon.0) 1236 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-02-23T00:43:54.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:53.103497+0000 mon.a (mon.0) 1237 : cluster [DBG] osdmap e161: 8 total, 8 up, 8 in 2024-02-23T00:43:54.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:53.105250+0000 mon.a (mon.0) 1238 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-13", "mode": "writeback"}]: dispatch 2024-02-23T00:43:54.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:53.214491+0000 mgr.y (mgr.14517) 204 : cluster [DBG] pgmap v217: 354 pgs: 322 active+clean, 32 creating+peering; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 131 KiB/s wr, 127 op/s 2024-02-23T00:43:54.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:53.315350+0000 osd.2 (osd.2) 81 : cluster [DBG] 2.c scrub starts 2024-02-23T00:43:54.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:53.317196+0000 osd.2 (osd.2) 82 : cluster [DBG] 2.c scrub ok 2024-02-23T00:43:54.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:53.721892+0000 mon.b (mon.1) 189 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:54.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:54.100883+0000 mon.a (mon.0) 1239 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:43:54.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:54.106123+0000 mon.a (mon.0) 1240 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-13", "mode": "writeback"}]': finished 2024-02-23T00:43:54.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:54.106278+0000 mon.a (mon.0) 1241 : cluster [DBG] osdmap e162: 8 total, 8 up, 8 in 2024-02-23T00:43:54.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:54.107753+0000 mon.c (mon.2) 84 : audit [INF] from='client.? 172.21.15.47:0/3013997716' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi047-82378-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:54.343 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight 1 1 2024-02-23T00:43:54.372 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:43:54.375 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:54.376 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:43:54.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:52.418096+0000 osd.6 (osd.6) 77 : cluster [DBG] 2.17 scrub starts 2024-02-23T00:43:54.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:52.419092+0000 osd.6 (osd.6) 78 : cluster [DBG] 2.17 scrub ok 2024-02-23T00:43:54.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:53.103197+0000 mon.a (mon.0) 1234 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-13"}]': finished 2024-02-23T00:43:54.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:53.103325+0000 mon.a (mon.0) 1235 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi047-82386-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:54.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:53.103439+0000 mon.a (mon.0) 1236 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-02-23T00:43:54.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:53.103497+0000 mon.a (mon.0) 1237 : cluster [DBG] osdmap e161: 8 total, 8 up, 8 in 2024-02-23T00:43:54.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:53.105250+0000 mon.a (mon.0) 1238 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-13", "mode": "writeback"}]: dispatch 2024-02-23T00:43:54.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:53.214491+0000 mgr.y (mgr.14517) 204 : cluster [DBG] pgmap v217: 354 pgs: 322 active+clean, 32 creating+peering; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 131 KiB/s wr, 127 op/s 2024-02-23T00:43:54.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:53.315350+0000 osd.2 (osd.2) 81 : cluster [DBG] 2.c scrub starts 2024-02-23T00:43:54.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:53.317196+0000 osd.2 (osd.2) 82 : cluster [DBG] 2.c scrub ok 2024-02-23T00:43:54.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:53.721892+0000 mon.b (mon.1) 189 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:54.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:54.100883+0000 mon.a (mon.0) 1239 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:43:54.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:54.106123+0000 mon.a (mon.0) 1240 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-13", "mode": "writeback"}]': finished 2024-02-23T00:43:54.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:54.106278+0000 mon.a (mon.0) 1241 : cluster [DBG] osdmap e162: 8 total, 8 up, 8 in 2024-02-23T00:43:54.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:54.107753+0000 mon.c (mon.2) 84 : audit [INF] from='client.? 172.21.15.47:0/3013997716' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi047-82378-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:54.476 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:43:54.510 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:55.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.431+0000 7f963a894700 1 -- 172.21.15.47:0/2957989598 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f96340f9ab0 msgr2=0x7f96340f9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:55.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.431+0000 7f963a894700 1 --2- 172.21.15.47:0/2957989598 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f96340f9ab0 0x7f96340f9e90 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 rx=0x7f9624004870 tx=0x7f9624009cb0).stop 2024-02-23T00:43:55.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.431+0000 7f963a894700 1 -- 172.21.15.47:0/2957989598 shutdown_connections 2024-02-23T00:43:55.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.431+0000 7f963a894700 1 --2- 172.21.15.47:0/2957989598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96340f8440 0x7f9634104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:55.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.431+0000 7f963a894700 1 --2- 172.21.15.47:0/2957989598 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f96340f9ab0 0x7f96340f9e90 unknown :-1 s=CLOSED pgs=2473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:55.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.431+0000 7f963a894700 1 --2- 172.21.15.47:0/2957989598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96340fa380 0x7f96340f7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:55.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.431+0000 7f963a894700 1 -- 172.21.15.47:0/2957989598 >> 172.21.15.47:0/2957989598 conn(0x7f96340f3800 msgr2=0x7f96340f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:43:55.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.431+0000 7f963a894700 1 -- 172.21.15.47:0/2957989598 shutdown_connections 2024-02-23T00:43:55.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.431+0000 7f963a894700 1 -- 172.21.15.47:0/2957989598 wait complete. 2024-02-23T00:43:55.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.432+0000 7f963a894700 1 librados: starting msgr at 2024-02-23T00:43:55.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.432+0000 7f963a894700 1 librados: starting objecter 2024-02-23T00:43:55.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.432+0000 7f963a894700 1 Processor -- start 2024-02-23T00:43:55.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.432+0000 7f963a894700 1 -- start start 2024-02-23T00:43:55.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.432+0000 7f963a894700 1 librados: setting wanted keys 2024-02-23T00:43:55.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.432+0000 7f963a894700 1 librados: calling monclient init 2024-02-23T00:43:55.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.432+0000 7f963a894700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f96340f8440 0x7f963410e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:55.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.432+0000 7f963a894700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96340f9ab0 0x7f963410e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:55.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.432+0000 7f963a894700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96340fa380 0x7f96341081e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:55.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.432+0000 7f963a894700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96341086d0 con 0x7f96340fa380 2024-02-23T00:43:55.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.432+0000 7f963a894700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9634108840 con 0x7f96340f9ab0 2024-02-23T00:43:55.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.432+0000 7f963a894700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9634108a00 con 0x7f96340f8440 2024-02-23T00:43:55.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.432+0000 7f9633fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f96340f8440 0x7f963410e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:55.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.432+0000 7f9638e31700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96340fa380 0x7f96341081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:55.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.433+0000 7f9633fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f96340f8440 0x7f963410e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:35418/0 (socket says 172.21.15.47:35418) 2024-02-23T00:43:55.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.433+0000 7f9638e31700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96340fa380 0x7f96341081e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35150/0 (socket says 172.21.15.47:35150) 2024-02-23T00:43:55.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.433+0000 7f9633fff700 1 -- 172.21.15.47:0/762052867 learned_addr learned my addr 172.21.15.47:0/762052867 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:43:55.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.433+0000 7f96337fe700 1 --2- 172.21.15.47:0/762052867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96340f9ab0 0x7f963410e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:55.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.433+0000 7f9633fff700 1 -- 172.21.15.47:0/762052867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96340f9ab0 msgr2=0x7f963410e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:55.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.433+0000 7f9633fff700 1 --2- 172.21.15.47:0/762052867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96340f9ab0 0x7f963410e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:55.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.433+0000 7f9633fff700 1 -- 172.21.15.47:0/762052867 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96340fa380 msgr2=0x7f96341081e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:55.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.433+0000 7f9633fff700 1 --2- 172.21.15.47:0/762052867 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96340fa380 0x7f96341081e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:55.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.433+0000 7f9633fff700 1 -- 172.21.15.47:0/762052867 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9628005040 con 0x7f96340f8440 2024-02-23T00:43:55.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.433+0000 7f96337fe700 1 --2- 172.21.15.47:0/762052867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96340f9ab0 0x7f963410e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:43:55.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.433+0000 7f9638e31700 1 --2- 172.21.15.47:0/762052867 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96340fa380 0x7f96341081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:43:55.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.433+0000 7f9633fff700 1 --2- 172.21.15.47:0/762052867 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f96340f8440 0x7f963410e040 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 rx=0x7f9624004320 tx=0x7f9624009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:55.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.433+0000 7f96317fa700 10 client.?.objecter ms_handle_connect 0x7f96340f8440 2024-02-23T00:43:55.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.433+0000 7f96317fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:43:55.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.433+0000 7f96317fa700 1 -- 172.21.15.47:0/762052867 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962400e030 con 0x7f96340f8440 2024-02-23T00:43:55.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.433+0000 7f96317fa700 1 -- 172.21.15.47:0/762052867 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f9624013df0 con 0x7f96340f8440 2024-02-23T00:43:55.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.433+0000 7f963a894700 1 -- 172.21.15.47:0/762052867 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9624005040 con 0x7f96340f8440 2024-02-23T00:43:55.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.433+0000 7f96317fa700 1 -- 172.21.15.47:0/762052867 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9624018070 con 0x7f96340f8440 2024-02-23T00:43:55.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.433+0000 7f963a894700 10 client.40427.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:43:55.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.433+0000 7f963a894700 1 -- 172.21.15.47:0/762052867 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9634108f70 con 0x7f96340f8440 2024-02-23T00:43:55.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.433+0000 7f963a894700 1 librados: init done 2024-02-23T00:43:55.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.434+0000 7f963a894700 1 -- 172.21.15.47:0/762052867 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f963405ee80 con 0x7f96340f8440 2024-02-23T00:43:55.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.434+0000 7f96317fa700 1 -- 172.21.15.47:0/762052867 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f9624009240 con 0x7f96340f8440 2024-02-23T00:43:55.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.435+0000 7f96317fa700 1 --2- 172.21.15.47:0/762052867 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f961c062cc0 0x7f961c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:55.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.435+0000 7f96317fa700 1 -- 172.21.15.47:0/762052867 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(163..163 src has 48..163) v4 ==== 11758+0+0 (secure 0 0 0) 0x7f962407f170 con 0x7f96340f8440 2024-02-23T00:43:55.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.435+0000 7f96317fa700 10 client.40427.objecter ms_dispatch 0x7f9634001550 osd_map(163..163 src has 48..163) v4 2024-02-23T00:43:55.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.435+0000 7f96317fa700 3 client.40427.objecter handle_osd_map got epochs [163,163] > 0 2024-02-23T00:43:55.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.435+0000 7f96317fa700 3 client.40427.objecter handle_osd_map decoding full epoch 163 2024-02-23T00:43:55.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.435+0000 7f96337fe700 1 --2- 172.21.15.47:0/762052867 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f961c062cc0 0x7f961c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:55.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.435+0000 7f96317fa700 20 client.40427.objecter dump_active .. 0 homeless 2024-02-23T00:43:55.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.435+0000 7f96317fa700 10 client.40427.objecter ms_handle_connect 0x7f961c062cc0 2024-02-23T00:43:55.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.435+0000 7f96337fe700 1 --2- 172.21.15.47:0/762052867 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f961c062cc0 0x7f961c065180 secure :-1 s=READY pgs=2904 cs=0 l=1 rev1=1 rx=0x7f9634109980 tx=0x7f9628009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:55.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.439+0000 7f96317fa700 1 -- 172.21.15.47:0/762052867 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f962404ded0 con 0x7f96340f8440 2024-02-23T00:43:55.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:54.108264+0000 mon.a (mon.0) 1242 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi047-82378-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:55.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:54.115402+0000 mon.a (mon.0) 1243 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:43:55.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:54.303985+0000 osd.2 (osd.2) 83 : cluster [DBG] 2.1 scrub starts 2024-02-23T00:43:55.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:54.304944+0000 osd.2 (osd.2) 84 : cluster [DBG] 2.1 scrub ok 2024-02-23T00:43:55.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:54.352866+0000 osd.0 (osd.0) 89 : cluster [DBG] 4.b scrub starts 2024-02-23T00:43:55.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:54.354969+0000 osd.0 (osd.0) 90 : cluster [DBG] 4.b scrub ok 2024-02-23T00:43:55.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:54.690671+0000 mon.a (mon.0) 1244 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "93.0"}]: dispatch 2024-02-23T00:43:55.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:54.691310+0000 mon.a (mon.0) 1245 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "93.1"}]: dispatch 2024-02-23T00:43:55.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:54.691806+0000 mon.a (mon.0) 1246 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "93.2"}]: dispatch 2024-02-23T00:43:55.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:54.692274+0000 mon.a (mon.0) 1247 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "93.3"}]: dispatch 2024-02-23T00:43:55.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:54.692803+0000 mon.a (mon.0) 1248 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "93.4"}]: dispatch 2024-02-23T00:43:55.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:54.693254+0000 mon.a (mon.0) 1249 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "93.5"}]: dispatch 2024-02-23T00:43:55.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:54.693685+0000 mon.a (mon.0) 1250 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "93.6"}]: dispatch 2024-02-23T00:43:55.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:54.694118+0000 mon.a (mon.0) 1251 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "93.7"}]: dispatch 2024-02-23T00:43:55.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:54.694577+0000 mon.a (mon.0) 1252 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "93.8"}]: dispatch 2024-02-23T00:43:55.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:54.695031+0000 mon.a (mon.0) 1253 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "93.9"}]: dispatch 2024-02-23T00:43:55.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:54.722903+0000 mon.b (mon.1) 190 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:55.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:55.097254+0000 mon.a (mon.0) 1254 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi047-82378-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:55.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:55.097553+0000 mon.a (mon.0) 1255 : cluster [DBG] osdmap e163: 8 total, 8 up, 8 in 2024-02-23T00:43:55.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:55.106679+0000 mon.c (mon.2) 85 : audit [INF] from='client.? 172.21.15.47:0/353737300' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi047-82386-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:55.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:55.116973+0000 mon.a (mon.0) 1256 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi047-82386-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:55.468 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:43:55.472 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:54.108264+0000 mon.a (mon.0) 1242 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi047-82378-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:55.472 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:54.115402+0000 mon.a (mon.0) 1243 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:43:55.472 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:54.303985+0000 osd.2 (osd.2) 83 : cluster [DBG] 2.1 scrub starts 2024-02-23T00:43:55.472 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:54.304944+0000 osd.2 (osd.2) 84 : cluster [DBG] 2.1 scrub ok 2024-02-23T00:43:55.472 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:54.352866+0000 osd.0 (osd.0) 89 : cluster [DBG] 4.b scrub starts 2024-02-23T00:43:55.472 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:54.354969+0000 osd.0 (osd.0) 90 : cluster [DBG] 4.b scrub ok 2024-02-23T00:43:55.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:54.690671+0000 mon.a (mon.0) 1244 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "93.0"}]: dispatch 2024-02-23T00:43:55.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:54.691310+0000 mon.a (mon.0) 1245 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "93.1"}]: dispatch 2024-02-23T00:43:55.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:54.691806+0000 mon.a (mon.0) 1246 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "93.2"}]: dispatch 2024-02-23T00:43:55.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:54.692274+0000 mon.a (mon.0) 1247 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "93.3"}]: dispatch 2024-02-23T00:43:55.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:54.692803+0000 mon.a (mon.0) 1248 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "93.4"}]: dispatch 2024-02-23T00:43:55.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:54.693254+0000 mon.a (mon.0) 1249 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "93.5"}]: dispatch 2024-02-23T00:43:55.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:54.693685+0000 mon.a (mon.0) 1250 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "93.6"}]: dispatch 2024-02-23T00:43:55.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:54.694118+0000 mon.a (mon.0) 1251 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "93.7"}]: dispatch 2024-02-23T00:43:55.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:54.694577+0000 mon.a (mon.0) 1252 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "93.8"}]: dispatch 2024-02-23T00:43:55.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:54.695031+0000 mon.a (mon.0) 1253 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "93.9"}]: dispatch 2024-02-23T00:43:55.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:54.722903+0000 mon.b (mon.1) 190 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:55.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:55.097254+0000 mon.a (mon.0) 1254 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi047-82378-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:55.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:55.097553+0000 mon.a (mon.0) 1255 : cluster [DBG] osdmap e163: 8 total, 8 up, 8 in 2024-02-23T00:43:55.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:55.106679+0000 mon.c (mon.2) 85 : audit [INF] from='client.? 172.21.15.47:0/353737300' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi047-82386-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:55.474 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:55.116973+0000 mon.a (mon.0) 1256 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi047-82386-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:55.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:54.108264+0000 mon.a (mon.0) 1242 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi047-82378-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:55.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:54.115402+0000 mon.a (mon.0) 1243 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:43:55.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:54.303985+0000 osd.2 (osd.2) 83 : cluster [DBG] 2.1 scrub starts 2024-02-23T00:43:55.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:54.304944+0000 osd.2 (osd.2) 84 : cluster [DBG] 2.1 scrub ok 2024-02-23T00:43:55.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:54.352866+0000 osd.0 (osd.0) 89 : cluster [DBG] 4.b scrub starts 2024-02-23T00:43:55.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:54.354969+0000 osd.0 (osd.0) 90 : cluster [DBG] 4.b scrub ok 2024-02-23T00:43:55.475 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:54.690671+0000 mon.a (mon.0) 1244 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "93.0"}]: dispatch 2024-02-23T00:43:55.475 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:54.691310+0000 mon.a (mon.0) 1245 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "93.1"}]: dispatch 2024-02-23T00:43:55.475 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:54.691806+0000 mon.a (mon.0) 1246 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "93.2"}]: dispatch 2024-02-23T00:43:55.475 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:54.692274+0000 mon.a (mon.0) 1247 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "93.3"}]: dispatch 2024-02-23T00:43:55.475 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:54.692803+0000 mon.a (mon.0) 1248 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "93.4"}]: dispatch 2024-02-23T00:43:55.475 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:54.693254+0000 mon.a (mon.0) 1249 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "93.5"}]: dispatch 2024-02-23T00:43:55.475 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:54.693685+0000 mon.a (mon.0) 1250 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "93.6"}]: dispatch 2024-02-23T00:43:55.475 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:54.694118+0000 mon.a (mon.0) 1251 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "93.7"}]: dispatch 2024-02-23T00:43:55.475 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:54.694577+0000 mon.a (mon.0) 1252 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "93.8"}]: dispatch 2024-02-23T00:43:55.475 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:54.695031+0000 mon.a (mon.0) 1253 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "93.9"}]: dispatch 2024-02-23T00:43:55.475 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:54.722903+0000 mon.b (mon.1) 190 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:55.475 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:55.097254+0000 mon.a (mon.0) 1254 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi047-82378-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:55.475 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:55.097553+0000 mon.a (mon.0) 1255 : cluster [DBG] osdmap e163: 8 total, 8 up, 8 in 2024-02-23T00:43:55.476 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:55.106679+0000 mon.c (mon.2) 85 : audit [INF] from='client.? 172.21.15.47:0/353737300' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi047-82386-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:55.476 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:55.116973+0000 mon.a (mon.0) 1256 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi047-82386-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:55.476 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:55.476 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:43:55.485 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:43:55.516 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:43:55.578 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:43:55.608 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:55.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:55.632+0000 7f963a894700 1 -- 172.21.15.47:0/762052867 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7f9634106f60 con 0x7f96340f8440 2024-02-23T00:43:55.645 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:43:55.647 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:55.648 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:43:55.748 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:43:55.772 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:43:55.772 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-02-23T00:43:55.772 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 0 2024-02-23T00:43:55.772 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:43:55.776 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:55.796 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:43:55.798 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:55.798 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:43:55.899 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:43:55.927 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:56.049 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:43:56.052 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:56.052 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:43:56.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:56.099+0000 7f96317fa700 1 -- 172.21.15.47:0/762052867 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v164) v1 ==== 87+0+0 (secure 0 0 0) 0x7f96240514f0 con 0x7f96340f8440 2024-02-23T00:43:56.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:56.102+0000 7f963a894700 10 librados: watch_flush enter 2024-02-23T00:43:56.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:56.104+0000 7f963a894700 10 librados: watch_flush exit 2024-02-23T00:43:56.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:56.104+0000 7f963a894700 20 client.40427.objecter shutdown clearing up homeless session... 2024-02-23T00:43:56.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:56.104+0000 7f963a894700 10 client.40427.objecter successfully canceled tick 2024-02-23T00:43:56.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:56.104+0000 7f963a894700 1 -- 172.21.15.47:0/762052867 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f961c062cc0 msgr2=0x7f961c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:56.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:56.104+0000 7f963a894700 1 --2- 172.21.15.47:0/762052867 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f961c062cc0 0x7f961c065180 secure :-1 s=READY pgs=2904 cs=0 l=1 rev1=1 rx=0x7f9634109980 tx=0x7f9628009cb0).stop 2024-02-23T00:43:56.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:56.104+0000 7f963a894700 1 -- 172.21.15.47:0/762052867 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f96340f8440 msgr2=0x7f963410e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:56.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:56.104+0000 7f963a894700 1 --2- 172.21.15.47:0/762052867 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f96340f8440 0x7f963410e040 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 rx=0x7f9624004320 tx=0x7f9624009cb0).stop 2024-02-23T00:43:56.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:56.107+0000 7f963a894700 1 -- 172.21.15.47:0/762052867 shutdown_connections 2024-02-23T00:43:56.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:56.107+0000 7f963a894700 1 --2- 172.21.15.47:0/762052867 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96340fa380 0x7f96341081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:56.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:56.107+0000 7f963a894700 1 --2- 172.21.15.47:0/762052867 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f961c062cc0 0x7f961c065180 unknown :-1 s=CLOSED pgs=2904 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:56.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:56.107+0000 7f963a894700 1 --2- 172.21.15.47:0/762052867 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f96340f8440 0x7f963410e040 unknown :-1 s=CLOSED pgs=2474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:56.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:56.107+0000 7f963a894700 1 --2- 172.21.15.47:0/762052867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96340f9ab0 0x7f963410e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:56.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:56.107+0000 7f963a894700 1 -- 172.21.15.47:0/762052867 >> 172.21.15.47:0/762052867 conn(0x7f96340f3800 msgr2=0x7f96340fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:43:56.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:56.116+0000 7f963a894700 1 -- 172.21.15.47:0/762052867 shutdown_connections 2024-02-23T00:43:56.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:56.124+0000 7f963a894700 1 -- 172.21.15.47:0/762052867 wait complete. 2024-02-23T00:43:56.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:56.129+0000 7f963a894700 1 librados: shutdown 2024-02-23T00:43:56.140 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is set 2024-02-23T00:43:56.153 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:43:56.188 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:56.261 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:43:56.265 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:56.266 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:43:56.367 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:43:56.395 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:54.435678+0000 osd.6 (osd.6) 79 : cluster [DBG] 4.a scrub starts 2024-02-23T00:43:56.395 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:54.438160+0000 osd.6 (osd.6) 80 : cluster [DBG] 4.a scrub ok 2024-02-23T00:43:56.395 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:54.690854+0000 mgr.y (mgr.14517) 205 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "93.0"}]: dispatch 2024-02-23T00:43:56.395 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:54.691433+0000 mgr.y (mgr.14517) 206 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "93.1"}]: dispatch 2024-02-23T00:43:56.395 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:54.691931+0000 mgr.y (mgr.14517) 207 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "93.2"}]: dispatch 2024-02-23T00:43:56.395 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:54.692393+0000 mgr.y (mgr.14517) 208 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "93.3"}]: dispatch 2024-02-23T00:43:56.395 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:54.692922+0000 mgr.y (mgr.14517) 209 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "93.4"}]: dispatch 2024-02-23T00:43:56.395 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:54.693364+0000 mgr.y (mgr.14517) 210 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "93.5"}]: dispatch 2024-02-23T00:43:56.396 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:54.693795+0000 mgr.y (mgr.14517) 211 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "93.6"}]: dispatch 2024-02-23T00:43:56.396 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:54.694232+0000 mgr.y (mgr.14517) 212 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "93.7"}]: dispatch 2024-02-23T00:43:56.396 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:54.694684+0000 mgr.y (mgr.14517) 213 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "93.8"}]: dispatch 2024-02-23T00:43:56.396 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:54.695135+0000 mgr.y (mgr.14517) 214 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "93.9"}]: dispatch 2024-02-23T00:43:56.396 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:54.815854+0000 osd.1 (osd.1) 101 : cluster [DBG] 93.4 scrub starts 2024-02-23T00:43:56.396 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:54.818742+0000 osd.1 (osd.1) 102 : cluster [DBG] 93.4 scrub ok 2024-02-23T00:43:56.396 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:54.976440+0000 osd.5 (osd.5) 87 : cluster [DBG] 93.6 scrub starts 2024-02-23T00:43:56.396 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:54.979915+0000 osd.5 (osd.5) 88 : cluster [DBG] 93.6 scrub ok 2024-02-23T00:43:56.396 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:55.214925+0000 mgr.y (mgr.14517) 215 : cluster [DBG] pgmap v220: 386 pgs: 322 active+clean, 64 unknown; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 131 KiB/s wr, 127 op/s 2024-02-23T00:43:56.396 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:55.634077+0000 mon.c (mon.2) 86 : audit [INF] from='client.? 172.21.15.47:0/762052867' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:43:56.396 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:55.634355+0000 mon.a (mon.0) 1257 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:43:56.396 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:55.723998+0000 mon.b (mon.1) 191 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:56.396 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:56.093621+0000 mon.a (mon.0) 1258 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:43:56.397 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:56.098241+0000 mon.a (mon.0) 1259 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi047-82386-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:56.397 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:56.098422+0000 mon.a (mon.0) 1260 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:43:56.397 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:56.098489+0000 mon.a (mon.0) 1261 : cluster [DBG] osdmap e164: 8 total, 8 up, 8 in 2024-02-23T00:43:56.397 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:56.103417+0000 mon.c (mon.2) 87 : audit [INF] from='client.? 172.21.15.47:0/2338570187' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]: dispatch 2024-02-23T00:43:56.397 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:56.106132+0000 mon.a (mon.0) 1262 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]: dispatch 2024-02-23T00:43:56.397 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:54.435678+0000 osd.6 (osd.6) 79 : cluster [DBG] 4.a scrub starts 2024-02-23T00:43:56.397 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:54.438160+0000 osd.6 (osd.6) 80 : cluster [DBG] 4.a scrub ok 2024-02-23T00:43:56.397 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:54.690854+0000 mgr.y (mgr.14517) 205 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "93.0"}]: dispatch 2024-02-23T00:43:56.398 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:54.691433+0000 mgr.y (mgr.14517) 206 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "93.1"}]: dispatch 2024-02-23T00:43:56.398 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:54.691931+0000 mgr.y (mgr.14517) 207 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "93.2"}]: dispatch 2024-02-23T00:43:56.398 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:54.692393+0000 mgr.y (mgr.14517) 208 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "93.3"}]: dispatch 2024-02-23T00:43:56.398 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:54.692922+0000 mgr.y (mgr.14517) 209 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "93.4"}]: dispatch 2024-02-23T00:43:56.398 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:54.693364+0000 mgr.y (mgr.14517) 210 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "93.5"}]: dispatch 2024-02-23T00:43:56.398 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:54.693795+0000 mgr.y (mgr.14517) 211 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "93.6"}]: dispatch 2024-02-23T00:43:56.398 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:54.694232+0000 mgr.y (mgr.14517) 212 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "93.7"}]: dispatch 2024-02-23T00:43:56.398 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:54.694684+0000 mgr.y (mgr.14517) 213 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "93.8"}]: dispatch 2024-02-23T00:43:56.398 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:54.695135+0000 mgr.y (mgr.14517) 214 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "93.9"}]: dispatch 2024-02-23T00:43:56.398 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:54.815854+0000 osd.1 (osd.1) 101 : cluster [DBG] 93.4 scrub starts 2024-02-23T00:43:56.398 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:54.818742+0000 osd.1 (osd.1) 102 : cluster [DBG] 93.4 scrub ok 2024-02-23T00:43:56.398 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:54.976440+0000 osd.5 (osd.5) 87 : cluster [DBG] 93.6 scrub starts 2024-02-23T00:43:56.398 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:54.979915+0000 osd.5 (osd.5) 88 : cluster [DBG] 93.6 scrub ok 2024-02-23T00:43:56.398 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:55.214925+0000 mgr.y (mgr.14517) 215 : cluster [DBG] pgmap v220: 386 pgs: 322 active+clean, 64 unknown; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 131 KiB/s wr, 127 op/s 2024-02-23T00:43:56.398 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:55.634077+0000 mon.c (mon.2) 86 : audit [INF] from='client.? 172.21.15.47:0/762052867' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:43:56.399 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:55.634355+0000 mon.a (mon.0) 1257 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:43:56.399 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:55.723998+0000 mon.b (mon.1) 191 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:56.399 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:56.093621+0000 mon.a (mon.0) 1258 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:43:56.399 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:56.098241+0000 mon.a (mon.0) 1259 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi047-82386-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:56.399 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:56.098422+0000 mon.a (mon.0) 1260 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:43:56.399 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:56.098489+0000 mon.a (mon.0) 1261 : cluster [DBG] osdmap e164: 8 total, 8 up, 8 in 2024-02-23T00:43:56.399 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:56.103417+0000 mon.c (mon.2) 87 : audit [INF] from='client.? 172.21.15.47:0/2338570187' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]: dispatch 2024-02-23T00:43:56.399 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:56.106132+0000 mon.a (mon.0) 1262 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]: dispatch 2024-02-23T00:43:56.399 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:56.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:54.435678+0000 osd.6 (osd.6) 79 : cluster [DBG] 4.a scrub starts 2024-02-23T00:43:56.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:54.438160+0000 osd.6 (osd.6) 80 : cluster [DBG] 4.a scrub ok 2024-02-23T00:43:56.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:54.690854+0000 mgr.y (mgr.14517) 205 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "93.0"}]: dispatch 2024-02-23T00:43:56.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:54.691433+0000 mgr.y (mgr.14517) 206 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "93.1"}]: dispatch 2024-02-23T00:43:56.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:54.691931+0000 mgr.y (mgr.14517) 207 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "93.2"}]: dispatch 2024-02-23T00:43:56.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:54.692393+0000 mgr.y (mgr.14517) 208 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "93.3"}]: dispatch 2024-02-23T00:43:56.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:54.692922+0000 mgr.y (mgr.14517) 209 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "93.4"}]: dispatch 2024-02-23T00:43:56.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:54.693364+0000 mgr.y (mgr.14517) 210 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "93.5"}]: dispatch 2024-02-23T00:43:56.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:54.693795+0000 mgr.y (mgr.14517) 211 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "93.6"}]: dispatch 2024-02-23T00:43:56.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:54.694232+0000 mgr.y (mgr.14517) 212 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "93.7"}]: dispatch 2024-02-23T00:43:56.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:54.694684+0000 mgr.y (mgr.14517) 213 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "93.8"}]: dispatch 2024-02-23T00:43:56.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:54.695135+0000 mgr.y (mgr.14517) 214 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "93.9"}]: dispatch 2024-02-23T00:43:56.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:54.815854+0000 osd.1 (osd.1) 101 : cluster [DBG] 93.4 scrub starts 2024-02-23T00:43:56.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:54.818742+0000 osd.1 (osd.1) 102 : cluster [DBG] 93.4 scrub ok 2024-02-23T00:43:56.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:54.976440+0000 osd.5 (osd.5) 87 : cluster [DBG] 93.6 scrub starts 2024-02-23T00:43:56.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:54.979915+0000 osd.5 (osd.5) 88 : cluster [DBG] 93.6 scrub ok 2024-02-23T00:43:56.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:55.214925+0000 mgr.y (mgr.14517) 215 : cluster [DBG] pgmap v220: 386 pgs: 322 active+clean, 64 unknown; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 131 KiB/s wr, 127 op/s 2024-02-23T00:43:56.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:55.634077+0000 mon.c (mon.2) 86 : audit [INF] from='client.? 172.21.15.47:0/762052867' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:43:56.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:55.634355+0000 mon.a (mon.0) 1257 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:43:56.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:55.723998+0000 mon.b (mon.1) 191 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:56.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:56.093621+0000 mon.a (mon.0) 1258 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:43:56.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:56.098241+0000 mon.a (mon.0) 1259 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi047-82386-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:56.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:56.098422+0000 mon.a (mon.0) 1260 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:43:56.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:56.098489+0000 mon.a (mon.0) 1261 : cluster [DBG] osdmap e164: 8 total, 8 up, 8 in 2024-02-23T00:43:56.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:56.103417+0000 mon.c (mon.2) 87 : audit [INF] from='client.? 172.21.15.47:0/2338570187' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]: dispatch 2024-02-23T00:43:56.446 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:56.106132+0000 mon.a (mon.0) 1262 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]: dispatch 2024-02-23T00:43:56.659 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.4 dump_blocked_ops 2024-02-23T00:43:56.931 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:43:57.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:55.784405+0000 osd.1 (osd.1) 103 : cluster [DBG] 93.5 scrub starts 2024-02-23T00:43:57.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:55.786110+0000 osd.1 (osd.1) 104 : cluster [DBG] 93.5 scrub ok 2024-02-23T00:43:57.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:56.089486+0000 osd.3 (osd.3) 109 : cluster [DBG] 6.c scrub starts 2024-02-23T00:43:57.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:56.090978+0000 osd.3 (osd.3) 110 : cluster [DBG] 6.c scrub ok 2024-02-23T00:43:57.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:56.725072+0000 mon.b (mon.1) 192 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:57.101202+0000 mon.a (mon.0) 1263 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]': finished 2024-02-23T00:43:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:57.101343+0000 mon.a (mon.0) 1264 : cluster [DBG] osdmap e165: 8 total, 8 up, 8 in 2024-02-23T00:43:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:57.103603+0000 mon.c (mon.2) 88 : audit [INF] from='client.? 172.21.15.47:0/2338570187' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]: dispatch 2024-02-23T00:43:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:57.105888+0000 mon.a (mon.0) 1265 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]: dispatch 2024-02-23T00:43:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:57.118203+0000 mon.b (mon.1) 193 : audit [INF] from='client.? 172.21.15.47:0/2259505769' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi047-82378-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:57.119108+0000 mon.a (mon.0) 1266 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi047-82378-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:57.993 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:55.784405+0000 osd.1 (osd.1) 103 : cluster [DBG] 93.5 scrub starts 2024-02-23T00:43:57.993 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:55.786110+0000 osd.1 (osd.1) 104 : cluster [DBG] 93.5 scrub ok 2024-02-23T00:43:57.993 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:56.089486+0000 osd.3 (osd.3) 109 : cluster [DBG] 6.c scrub starts 2024-02-23T00:43:57.993 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:56.090978+0000 osd.3 (osd.3) 110 : cluster [DBG] 6.c scrub ok 2024-02-23T00:43:57.993 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:56.725072+0000 mon.b (mon.1) 192 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:57.993 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:57.101202+0000 mon.a (mon.0) 1263 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]': finished 2024-02-23T00:43:57.993 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:57.101343+0000 mon.a (mon.0) 1264 : cluster [DBG] osdmap e165: 8 total, 8 up, 8 in 2024-02-23T00:43:57.993 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:57.103603+0000 mon.c (mon.2) 88 : audit [INF] from='client.? 172.21.15.47:0/2338570187' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]: dispatch 2024-02-23T00:43:57.994 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:57.105888+0000 mon.a (mon.0) 1265 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]: dispatch 2024-02-23T00:43:57.994 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:57.118203+0000 mon.b (mon.1) 193 : audit [INF] from='client.? 172.21.15.47:0/2259505769' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi047-82378-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:57.994 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:57.119108+0000 mon.a (mon.0) 1266 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi047-82378-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:57.994 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:55.784405+0000 osd.1 (osd.1) 103 : cluster [DBG] 93.5 scrub starts 2024-02-23T00:43:57.994 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:55.786110+0000 osd.1 (osd.1) 104 : cluster [DBG] 93.5 scrub ok 2024-02-23T00:43:57.994 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:56.089486+0000 osd.3 (osd.3) 109 : cluster [DBG] 6.c scrub starts 2024-02-23T00:43:57.994 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:56.090978+0000 osd.3 (osd.3) 110 : cluster [DBG] 6.c scrub ok 2024-02-23T00:43:57.994 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:56.725072+0000 mon.b (mon.1) 192 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:57.994 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:57.101202+0000 mon.a (mon.0) 1263 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]': finished 2024-02-23T00:43:57.994 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:57.101343+0000 mon.a (mon.0) 1264 : cluster [DBG] osdmap e165: 8 total, 8 up, 8 in 2024-02-23T00:43:57.994 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:57.103603+0000 mon.c (mon.2) 88 : audit [INF] from='client.? 172.21.15.47:0/2338570187' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]: dispatch 2024-02-23T00:43:57.995 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:57.105888+0000 mon.a (mon.0) 1265 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]: dispatch 2024-02-23T00:43:57.995 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:57.118203+0000 mon.b (mon.1) 193 : audit [INF] from='client.? 172.21.15.47:0/2259505769' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi047-82378-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:57.995 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:57.119108+0000 mon.a (mon.0) 1266 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi047-82378-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:58.121 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: Running main() from gmock_main.cc 2024-02-23T00:43:58.121 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [==========] Running 12 tests from 4 test suites. 2024-02-23T00:43:58.121 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [----------] Global test environment set-up. 2024-02-23T00:43:58.121 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2024-02-23T00:43:58.121 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2024-02-23T00:43:58.122 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (4617 ms) 2024-02-23T00:43:58.122 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2024-02-23T00:43:58.122 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (3332 ms) 2024-02-23T00:43:58.122 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2024-02-23T00:43:58.122 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (3307 ms) 2024-02-23T00:43:58.122 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2024-02-23T00:43:58.122 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (2391 ms) 2024-02-23T00:43:58.122 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (13647 ms total) 2024-02-23T00:43:58.122 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: 2024-02-23T00:43:58.122 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManaged 2024-02-23T00:43:58.122 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-02-23T00:43:58.122 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (5064 ms) 2024-02-23T00:43:58.122 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-02-23T00:43:58.123 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (4566 ms) 2024-02-23T00:43:58.123 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManaged (9630 ms total) 2024-02-23T00:43:58.123 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: 2024-02-23T00:43:58.123 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2024-02-23T00:43:58.123 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2024-02-23T00:43:58.123 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (3725 ms) 2024-02-23T00:43:58.123 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-02-23T00:43:58.123 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (2269 ms) 2024-02-23T00:43:58.123 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2024-02-23T00:43:58.123 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (2234 ms) 2024-02-23T00:43:58.123 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-02-23T00:43:58.123 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (1740 ms) 2024-02-23T00:43:58.123 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (9968 ms total) 2024-02-23T00:43:58.123 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: 2024-02-23T00:43:58.123 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-02-23T00:43:58.124 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-02-23T00:43:58.124 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (3870 ms) 2024-02-23T00:43:58.124 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-02-23T00:43:58.124 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (4016 ms) 2024-02-23T00:43:58.124 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (7886 ms total) 2024-02-23T00:43:58.124 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: 2024-02-23T00:43:58.124 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [----------] Global test environment tear-down 2024-02-23T00:43:58.124 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [==========] 12 tests from 4 test suites ran. (70892 ms total) 2024-02-23T00:43:58.124 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots: [ PASSED ] 12 tests. 2024-02-23T00:43:58.125 INFO:tasks.workunit.client.0.smithi047.stderr:+ for t in "${!pids[@]}" 2024-02-23T00:43:58.125 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82563 2024-02-23T00:43:58.125 INFO:tasks.workunit.client.0.smithi047.stderr:+ wait 82563 2024-02-23T00:43:58.125 INFO:tasks.workunit.client.0.smithi047.stderr:+ for t in "${!pids[@]}" 2024-02-23T00:43:58.125 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82405 2024-02-23T00:43:58.125 INFO:tasks.workunit.client.0.smithi047.stderr:+ wait 82405 2024-02-23T00:43:58.126 INFO:tasks.workunit.client.0.smithi047.stderr:+ for t in "${!pids[@]}" 2024-02-23T00:43:58.126 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82523 2024-02-23T00:43:58.126 INFO:tasks.workunit.client.0.smithi047.stderr:+ wait 82523 2024-02-23T00:43:58.126 INFO:tasks.workunit.client.0.smithi047.stderr:+ for t in "${!pids[@]}" 2024-02-23T00:43:58.126 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82491 2024-02-23T00:43:58.126 INFO:tasks.workunit.client.0.smithi047.stderr:+ wait 82491 2024-02-23T00:43:58.773 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:57.216141+0000 mgr.y (mgr.14517) 216 : cluster [DBG] pgmap v223: 346 pgs: 314 active+clean, 32 creating+peering; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 76 op/s 2024-02-23T00:43:58.773 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:57.410765+0000 osd.6 (osd.6) 81 : cluster [DBG] 93.2 scrub starts 2024-02-23T00:43:58.773 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:57.412764+0000 osd.6 (osd.6) 82 : cluster [DBG] 93.2 scrub ok 2024-02-23T00:43:58.773 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:57.726232+0000 mon.b (mon.1) 194 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:58.773 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:58.104697+0000 mon.a (mon.0) 1267 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]': finished 2024-02-23T00:43:58.773 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:58.104899+0000 mon.a (mon.0) 1268 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi047-82378-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:58.773 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:58.105006+0000 mon.a (mon.0) 1269 : cluster [DBG] osdmap e166: 8 total, 8 up, 8 in 2024-02-23T00:43:58.773 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:58.125181+0000 mon.b (mon.1) 195 : audit [INF] from='client.? 172.21.15.47:0/2356880053' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi047-82386-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:58.773 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:58.129834+0000 mon.a (mon.0) 1270 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi047-82386-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:58.774 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:57.216141+0000 mgr.y (mgr.14517) 216 : cluster [DBG] pgmap v223: 346 pgs: 314 active+clean, 32 creating+peering; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 76 op/s 2024-02-23T00:43:58.774 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:57.410765+0000 osd.6 (osd.6) 81 : cluster [DBG] 93.2 scrub starts 2024-02-23T00:43:58.774 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:57.412764+0000 osd.6 (osd.6) 82 : cluster [DBG] 93.2 scrub ok 2024-02-23T00:43:58.774 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:57.726232+0000 mon.b (mon.1) 194 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:58.774 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:58.104697+0000 mon.a (mon.0) 1267 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]': finished 2024-02-23T00:43:58.774 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:58.104899+0000 mon.a (mon.0) 1268 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi047-82378-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:58.774 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:58.105006+0000 mon.a (mon.0) 1269 : cluster [DBG] osdmap e166: 8 total, 8 up, 8 in 2024-02-23T00:43:58.774 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:58.125181+0000 mon.b (mon.1) 195 : audit [INF] from='client.? 172.21.15.47:0/2356880053' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi047-82386-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:58.774 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:58.129834+0000 mon.a (mon.0) 1270 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi047-82386-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:58.897 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:57.216141+0000 mgr.y (mgr.14517) 216 : cluster [DBG] pgmap v223: 346 pgs: 314 active+clean, 32 creating+peering; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 76 op/s 2024-02-23T00:43:58.898 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:57.410765+0000 osd.6 (osd.6) 81 : cluster [DBG] 93.2 scrub starts 2024-02-23T00:43:58.898 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:57.412764+0000 osd.6 (osd.6) 82 : cluster [DBG] 93.2 scrub ok 2024-02-23T00:43:58.898 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:57.726232+0000 mon.b (mon.1) 194 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:58.898 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:58.104697+0000 mon.a (mon.0) 1267 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi047-82487-14"}]': finished 2024-02-23T00:43:58.898 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:58.104899+0000 mon.a (mon.0) 1268 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi047-82378-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:58.898 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:58.105006+0000 mon.a (mon.0) 1269 : cluster [DBG] osdmap e166: 8 total, 8 up, 8 in 2024-02-23T00:43:58.898 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:58.125181+0000 mon.b (mon.1) 195 : audit [INF] from='client.? 172.21.15.47:0/2356880053' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi047-82386-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:58.898 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:58.129834+0000 mon.a (mon.0) 1270 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi047-82386-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:43:59.246 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:43:59.246 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-02-23T00:43:59.246 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-02-23T00:43:59.246 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-02-23T00:43:59.246 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:43:59.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.439+0000 7f3d06f54700 1 -- 172.21.15.47:0/1292019850 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3d000fcb00 msgr2=0x7f3d000fcee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:59.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.439+0000 7f3d06f54700 1 --2- 172.21.15.47:0/1292019850 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3d000fcb00 0x7f3d000fcee0 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 rx=0x7f3cf40047b0 tx=0x7f3cf4009cb0).stop 2024-02-23T00:43:59.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.439+0000 7f3d06f54700 1 -- 172.21.15.47:0/1292019850 shutdown_connections 2024-02-23T00:43:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.439+0000 7f3d06f54700 1 --2- 172.21.15.47:0/1292019850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d000f7970 0x7f3d00103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.439+0000 7f3d06f54700 1 --2- 172.21.15.47:0/1292019850 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3d000fcb00 0x7f3d000fcee0 unknown :-1 s=CLOSED pgs=2477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.439+0000 7f3d06f54700 1 --2- 172.21.15.47:0/1292019850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d000fd460 0x7f3d000f72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.439+0000 7f3d06f54700 1 -- 172.21.15.47:0/1292019850 >> 172.21.15.47:0/1292019850 conn(0x7f3d0006ffc0 msgr2=0x7f3d000f43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:43:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.440+0000 7f3d06f54700 1 -- 172.21.15.47:0/1292019850 shutdown_connections 2024-02-23T00:43:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.440+0000 7f3d06f54700 1 -- 172.21.15.47:0/1292019850 wait complete. 2024-02-23T00:43:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.440+0000 7f3d06f54700 1 librados: starting msgr at 2024-02-23T00:43:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.440+0000 7f3d06f54700 1 librados: starting objecter 2024-02-23T00:43:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.440+0000 7f3d06f54700 1 Processor -- start 2024-02-23T00:43:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.440+0000 7f3d06f54700 1 -- start start 2024-02-23T00:43:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.440+0000 7f3d06f54700 1 librados: setting wanted keys 2024-02-23T00:43:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.440+0000 7f3d06f54700 1 librados: calling monclient init 2024-02-23T00:43:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.441+0000 7f3d06f54700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d000f7970 0x7f3d00104140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.441+0000 7f3d06f54700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3d000fcb00 0x7f3d00104630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.441+0000 7f3d06f54700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d000fd460 0x7f3d00104b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.441+0000 7f3d06f54700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d00116540 con 0x7f3d000f7970 2024-02-23T00:43:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.441+0000 7f3d06f54700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d001166b0 con 0x7f3d000fd460 2024-02-23T00:43:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.441+0000 7f3d06f54700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d00116870 con 0x7f3d000fcb00 2024-02-23T00:43:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.441+0000 7f3cfffff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3d000fcb00 0x7f3d00104630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.441+0000 7f3cfffff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3d000fcb00 0x7f3d00104630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:48298/0 (socket says 172.21.15.47:48298) 2024-02-23T00:43:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.441+0000 7f3cfffff700 1 -- 172.21.15.47:0/4229644801 learned_addr learned my addr 172.21.15.47:0/4229644801 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:43:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.441+0000 7f3d054f1700 1 --2- 172.21.15.47:0/4229644801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d000fd460 0x7f3d00104b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.441+0000 7f3d04cf0700 1 --2- 172.21.15.47:0/4229644801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d000f7970 0x7f3d00104140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.441+0000 7f3cfffff700 1 -- 172.21.15.47:0/4229644801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d000fd460 msgr2=0x7f3d00104b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.441+0000 7f3cfffff700 1 --2- 172.21.15.47:0/4229644801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d000fd460 0x7f3d00104b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.441+0000 7f3cfffff700 1 -- 172.21.15.47:0/4229644801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d000f7970 msgr2=0x7f3d00104140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.441+0000 7f3cfffff700 1 --2- 172.21.15.47:0/4229644801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d000f7970 0x7f3d00104140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.441+0000 7f3cfffff700 1 -- 172.21.15.47:0/4229644801 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf80097c0 con 0x7f3d000fcb00 2024-02-23T00:43:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.441+0000 7f3d04cf0700 1 --2- 172.21.15.47:0/4229644801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d000f7970 0x7f3d00104140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:43:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.441+0000 7f3cfffff700 1 --2- 172.21.15.47:0/4229644801 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3d000fcb00 0x7f3d00104630 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 rx=0x7f3cec009400 tx=0x7f3cec009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.441+0000 7f3cfdffb700 10 client.?.objecter ms_handle_connect 0x7f3d000fcb00 2024-02-23T00:43:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.441+0000 7f3cfdffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:43:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.442+0000 7f3cfdffb700 1 -- 172.21.15.47:0/4229644801 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec007a40 con 0x7f3d000fcb00 2024-02-23T00:43:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.442+0000 7f3cfdffb700 1 -- 172.21.15.47:0/4229644801 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f3cec00f040 con 0x7f3d000fcb00 2024-02-23T00:43:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.442+0000 7f3d06f54700 1 -- 172.21.15.47:0/4229644801 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cec005040 con 0x7f3d000fcb00 2024-02-23T00:43:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.442+0000 7f3cfdffb700 1 -- 172.21.15.47:0/4229644801 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec014990 con 0x7f3d000fcb00 2024-02-23T00:43:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.442+0000 7f3d06f54700 10 client.40445.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:43:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.442+0000 7f3d06f54700 1 -- 172.21.15.47:0/4229644801 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cf4005040 con 0x7f3d000fcb00 2024-02-23T00:43:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.442+0000 7f3d06f54700 1 librados: init done 2024-02-23T00:43:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.443+0000 7f3d06f54700 1 -- 172.21.15.47:0/4229644801 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d0005f9b0 con 0x7f3d000fcb00 2024-02-23T00:43:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.443+0000 7f3cfdffb700 1 -- 172.21.15.47:0/4229644801 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f3cec00e630 con 0x7f3d000fcb00 2024-02-23T00:43:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.444+0000 7f3cfdffb700 1 --2- 172.21.15.47:0/4229644801 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3cf0062cc0 0x7f3cf0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.444+0000 7f3cfdffb700 1 -- 172.21.15.47:0/4229644801 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(167..167 src has 48..167) v4 ==== 10431+0+0 (secure 0 0 0) 0x7f3cec07d7a0 con 0x7f3d000fcb00 2024-02-23T00:43:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.444+0000 7f3cfdffb700 10 client.40445.objecter ms_dispatch 0x7f3d00001550 osd_map(167..167 src has 48..167) v4 2024-02-23T00:43:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.444+0000 7f3cfdffb700 3 client.40445.objecter handle_osd_map got epochs [167,167] > 0 2024-02-23T00:43:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.444+0000 7f3cfdffb700 3 client.40445.objecter handle_osd_map decoding full epoch 167 2024-02-23T00:43:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.444+0000 7f3cfdffb700 20 client.40445.objecter dump_active .. 0 homeless 2024-02-23T00:43:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.444+0000 7f3d04cf0700 1 --2- 172.21.15.47:0/4229644801 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3cf0062cc0 0x7f3cf0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.444+0000 7f3cfdffb700 10 client.40445.objecter ms_handle_connect 0x7f3cf0062cc0 2024-02-23T00:43:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.444+0000 7f3d04cf0700 1 --2- 172.21.15.47:0/4229644801 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3cf0062cc0 0x7f3cf0065180 secure :-1 s=READY pgs=2908 cs=0 l=1 rev1=1 rx=0x7f3cf4009bc0 tx=0x7f3cf4009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.448+0000 7f3cfdffb700 1 -- 172.21.15.47:0/4229644801 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f3cec049f60 con 0x7f3d000fcb00 2024-02-23T00:43:59.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.635+0000 7f3d06f54700 1 -- 172.21.15.47:0/4229644801 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd reweight", "id": 1, "weight": 1.0} v 0) v1 -- 0x7f3d00106ee0 con 0x7f3d000fcb00 2024-02-23T00:43:59.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.723+0000 7f54f9d63700 1 -- 172.21.15.47:0/2576679592 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f54f40fe510 msgr2=0x7f54f40f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:59.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.723+0000 7f54f9d63700 1 --2- 172.21.15.47:0/2576679592 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f54f40fe510 0x7f54f40f7d40 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 rx=0x7f54e8004870 tx=0x7f54e8009cb0).stop 2024-02-23T00:43:59.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.723+0000 7f54f9d63700 1 -- 172.21.15.47:0/2576679592 shutdown_connections 2024-02-23T00:43:59.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.723+0000 7f54f9d63700 1 --2- 172.21.15.47:0/2576679592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54f40f8420 0x7f54f4104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:59.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.723+0000 7f54f9d63700 1 --2- 172.21.15.47:0/2576679592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54f40fdbb0 0x7f54f40fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:59.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.723+0000 7f54f9d63700 1 --2- 172.21.15.47:0/2576679592 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f54f40fe510 0x7f54f40f7d40 unknown :-1 s=CLOSED pgs=2479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:59.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.723+0000 7f54f9d63700 1 -- 172.21.15.47:0/2576679592 >> 172.21.15.47:0/2576679592 conn(0x7f54f40f3800 msgr2=0x7f54f40f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:43:59.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.724+0000 7f54f9d63700 1 -- 172.21.15.47:0/2576679592 shutdown_connections 2024-02-23T00:43:59.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.724+0000 7f54f9d63700 1 -- 172.21.15.47:0/2576679592 wait complete. 2024-02-23T00:43:59.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.724+0000 7f54f9d63700 1 librados: starting msgr at 2024-02-23T00:43:59.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.724+0000 7f54f9d63700 1 librados: starting objecter 2024-02-23T00:43:59.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.724+0000 7f54f9d63700 1 Processor -- start 2024-02-23T00:43:59.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.724+0000 7f54f9d63700 1 -- start start 2024-02-23T00:43:59.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.724+0000 7f54f9d63700 1 librados: setting wanted keys 2024-02-23T00:43:59.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.724+0000 7f54f9d63700 1 librados: calling monclient init 2024-02-23T00:43:59.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.724+0000 7f54f9d63700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54f40f8420 0x7f54f410df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:59.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.725+0000 7f54f9d63700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54f40fdbb0 0x7f54f410e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:59.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.725+0000 7f54f9d63700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f54f40fe510 0x7f54f41080a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:59.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.725+0000 7f54f9d63700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f4108590 con 0x7f54f40fdbb0 2024-02-23T00:43:59.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.725+0000 7f54f9d63700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f4108700 con 0x7f54f40f8420 2024-02-23T00:43:59.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.725+0000 7f54f9d63700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f41088c0 con 0x7f54f40fe510 2024-02-23T00:43:59.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.725+0000 7f54f3fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f54f40fe510 0x7f54f41080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:59.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.725+0000 7f54f2ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54f40fdbb0 0x7f54f410e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:59.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.725+0000 7f54f37fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54f40f8420 0x7f54f410df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:59.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.725+0000 7f54f3fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f54f40fe510 0x7f54f41080a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:48318/0 (socket says 172.21.15.47:48318) 2024-02-23T00:43:59.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.725+0000 7f54f3fff700 1 -- 172.21.15.47:0/3807262655 learned_addr learned my addr 172.21.15.47:0/3807262655 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:43:59.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.725+0000 7f54f2ffd700 1 -- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f54f40fe510 msgr2=0x7f54f41080a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:59.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.725+0000 7f54f2ffd700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f54f40fe510 0x7f54f41080a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:59.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.725+0000 7f54f2ffd700 1 -- 172.21.15.47:0/3807262655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54f40f8420 msgr2=0x7f54f410df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:59.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.725+0000 7f54f2ffd700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54f40f8420 0x7f54f410df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:59.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.725+0000 7f54f2ffd700 1 -- 172.21.15.47:0/3807262655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e8005040 con 0x7f54f40fdbb0 2024-02-23T00:43:59.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.725+0000 7f54f3fff700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f54f40fe510 0x7f54f41080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:43:59.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.725+0000 7f54f37fe700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54f40f8420 0x7f54f410df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:43:59.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.725+0000 7f54f2ffd700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54f40fdbb0 0x7f54f410e3f0 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 rx=0x7f54e8004710 tx=0x7f54e8009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:59.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.725+0000 7f54f0ff9700 10 client.?.objecter ms_handle_connect 0x7f54f40fdbb0 2024-02-23T00:43:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.725+0000 7f54f0ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:43:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.725+0000 7f54f0ff9700 1 -- 172.21.15.47:0/3807262655 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e800e030 con 0x7f54f40fdbb0 2024-02-23T00:43:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.725+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54f4108b40 con 0x7f54f40fdbb0 2024-02-23T00:43:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.725+0000 7f54f0ff9700 1 -- 172.21.15.47:0/3807262655 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f54e8013d70 con 0x7f54f40fdbb0 2024-02-23T00:43:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.725+0000 7f54f0ff9700 1 -- 172.21.15.47:0/3807262655 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e8018070 con 0x7f54f40fdbb0 2024-02-23T00:43:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.725+0000 7f54f9d63700 10 client.30120.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:43:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.725+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54f4114d30 con 0x7f54f40fdbb0 2024-02-23T00:43:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.725+0000 7f54f9d63700 1 librados: init done 2024-02-23T00:43:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.726+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f54f405ee80 con 0x7f54f40fdbb0 2024-02-23T00:43:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.727+0000 7f54f0ff9700 1 -- 172.21.15.47:0/3807262655 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f54e8009240 con 0x7f54f40fdbb0 2024-02-23T00:43:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.727+0000 7f54f0ff9700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f54dc062cc0 0x7f54dc065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.727+0000 7f54f0ff9700 1 -- 172.21.15.47:0/3807262655 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(167..167 src has 48..167) v4 ==== 10431+0+0 (secure 0 0 0) 0x7f54e807f110 con 0x7f54f40fdbb0 2024-02-23T00:43:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.727+0000 7f54f0ff9700 10 client.30120.objecter ms_dispatch 0x7f54f4001550 osd_map(167..167 src has 48..167) v4 2024-02-23T00:43:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.727+0000 7f54f0ff9700 3 client.30120.objecter handle_osd_map got epochs [167,167] > 0 2024-02-23T00:43:59.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.727+0000 7f54f0ff9700 3 client.30120.objecter handle_osd_map decoding full epoch 167 2024-02-23T00:43:59.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.727+0000 7f54f37fe700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f54dc062cc0 0x7f54dc065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:59.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.727+0000 7f54f0ff9700 20 client.30120.objecter dump_active .. 0 homeless 2024-02-23T00:43:59.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.727+0000 7f54f0ff9700 1 -- 172.21.15.47:0/3807262655 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v167) v1 ==== 54+0+16 (secure 0 0 0) 0x7f54e807f5b0 con 0x7f54f40fdbb0 2024-02-23T00:43:59.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.728+0000 7f54f0ff9700 10 client.30120.objecter ms_handle_connect 0x7f54dc062cc0 2024-02-23T00:43:59.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.728+0000 7f54f37fe700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f54dc062cc0 0x7f54dc065180 secure :-1 s=READY pgs=2909 cs=0 l=1 rev1=1 rx=0x7f54e4011510 tx=0x7f54e400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:59.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.728+0000 7f54f9d63700 10 client.30120.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:59.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.728+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_assign -1 1 2024-02-23T00:43:59.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.728+0000 7f54f9d63700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f54f40fee20 0x7f54f40ff200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:59.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.728+0000 7f54f9d63700 20 client.30120.objecter _get_session s=0x7f54f4106e60 osd=0 3 2024-02-23T00:43:59.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.728+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f4106e60 osd=0 3 2024-02-23T00:43:59.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.728+0000 7f54f9d63700 20 client.30120.objecter _get_session s=0x7f54f4106e60 osd=0 3 2024-02-23T00:43:59.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.728+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_remove -1 1 2024-02-23T00:43:59.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.728+0000 7f54f9d63700 20 client.30120.objecter get_session s=0x7f54f4106e60 osd=0 3 2024-02-23T00:43:59.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.728+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_assign 0 1 2024-02-23T00:43:59.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.728+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f4106e60 osd=0 4 2024-02-23T00:43:59.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.728+0000 7f54f9d63700 10 client.30120.objecter _send_command 1 2024-02-23T00:43:59.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.728+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54f4109580 con 0x7f54f40fee20 2024-02-23T00:43:59.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.728+0000 7f54f3fff700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f54f40fee20 0x7f54f40ff200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:59.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.728+0000 7f54f0ff9700 10 client.30120.objecter ms_handle_connect 0x7f54f40fee20 2024-02-23T00:43:59.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.728+0000 7f54f3fff700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f54f40fee20 0x7f54f40ff200 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:59.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.729+0000 7f54f0ff9700 1 -- 172.21.15.47:0/3807262655 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f54f4109580 con 0x7f54f40fee20 2024-02-23T00:43:59.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.729+0000 7f54f0ff9700 10 client.30120.objecter ms_dispatch 0x7f54f4001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:43:59.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.729+0000 7f54f0ff9700 10 client.30120.objecter _finish_command 1 = system:0 2024-02-23T00:43:59.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.729+0000 7f54f0ff9700 20 client.30120.objecter put_session s=0x7f54f4106e60 osd=0 4 2024-02-23T00:43:59.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.729+0000 7f54f0ff9700 15 client.30120.objecter _session_command_op_remove 0 1 2024-02-23T00:43:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.744+0000 7f54f9d63700 10 client.30120.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:43:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.744+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_assign -1 2 2024-02-23T00:43:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.744+0000 7f54f9d63700 20 client.30120.objecter _get_session s=0x7f54f4106e60 osd=0 3 2024-02-23T00:43:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.744+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f4106e60 osd=0 3 2024-02-23T00:43:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.744+0000 7f54f9d63700 20 client.30120.objecter _get_session s=0x7f54f4106e60 osd=0 3 2024-02-23T00:43:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.744+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_remove -1 2 2024-02-23T00:43:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.744+0000 7f54f9d63700 20 client.30120.objecter get_session s=0x7f54f4106e60 osd=0 3 2024-02-23T00:43:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.744+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_assign 0 2 2024-02-23T00:43:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.744+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f4106e60 osd=0 4 2024-02-23T00:43:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.744+0000 7f54f9d63700 10 client.30120.objecter _send_command 2 2024-02-23T00:43:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.744+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f54f40273c0 con 0x7f54f40fee20 2024-02-23T00:43:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.746+0000 7f54f0ff9700 1 -- 172.21.15.47:0/3807262655 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f54f40273c0 con 0x7f54f40fee20 2024-02-23T00:43:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.746+0000 7f54f0ff9700 10 client.30120.objecter ms_dispatch 0x7f54f4001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:43:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.746+0000 7f54f0ff9700 10 client.30120.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:43:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.746+0000 7f54f0ff9700 20 client.30120.objecter put_session s=0x7f54f4106e60 osd=0 4 2024-02-23T00:43:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.746+0000 7f54f0ff9700 15 client.30120.objecter _session_command_op_remove 0 2 2024-02-23T00:43:59.748 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:43:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.746+0000 7f54f9d63700 10 client.30120.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.746+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_assign -1 3 2024-02-23T00:43:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.746+0000 7f54f9d63700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f54f40598f0 0x7f54f4059d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.746+0000 7f54f9d63700 20 client.30120.objecter _get_session s=0x7f54f4028710 osd=1 3 2024-02-23T00:43:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.746+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f4028710 osd=1 3 2024-02-23T00:43:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.746+0000 7f54f9d63700 20 client.30120.objecter _get_session s=0x7f54f4028710 osd=1 3 2024-02-23T00:43:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.746+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_remove -1 3 2024-02-23T00:43:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.746+0000 7f54f9d63700 20 client.30120.objecter get_session s=0x7f54f4028710 osd=1 3 2024-02-23T00:43:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.746+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_assign 1 3 2024-02-23T00:43:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.746+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f4028710 osd=1 4 2024-02-23T00:43:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.746+0000 7f54f9d63700 10 client.30120.objecter _send_command 3 2024-02-23T00:43:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.746+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54f411b640 con 0x7f54f40598f0 2024-02-23T00:43:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.746+0000 7f54f37fe700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f54f40598f0 0x7f54f4059d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.746+0000 7f54f0ff9700 10 client.30120.objecter ms_handle_connect 0x7f54f40598f0 2024-02-23T00:43:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.746+0000 7f54f37fe700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f54f40598f0 0x7f54f4059d30 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:43:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.747+0000 7f54f0ff9700 1 -- 172.21.15.47:0/3807262655 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f54f411b640 con 0x7f54f40598f0 2024-02-23T00:43:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.747+0000 7f54f0ff9700 10 client.30120.objecter ms_dispatch 0x7f54f4001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:43:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.747+0000 7f54f0ff9700 10 client.30120.objecter _finish_command 3 = system:0 2024-02-23T00:43:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.747+0000 7f54f0ff9700 20 client.30120.objecter put_session s=0x7f54f4028710 osd=1 4 2024-02-23T00:43:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.747+0000 7f54f0ff9700 15 client.30120.objecter _session_command_op_remove 1 3 2024-02-23T00:43:59.751 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:58.727485+0000 mon.b (mon.1) 196 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:59.751 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:59.106782+0000 mon.a (mon.0) 1271 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi047-82386-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:59.751 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:43:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:59.106886+0000 mon.a (mon.0) 1272 : cluster [DBG] osdmap e167: 8 total, 8 up, 8 in 2024-02-23T00:43:59.752 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:58.727485+0000 mon.b (mon.1) 196 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:59.752 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:59.106782+0000 mon.a (mon.0) 1271 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi047-82386-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:59.752 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:43:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:59.106886+0000 mon.a (mon.0) 1272 : cluster [DBG] osdmap e167: 8 total, 8 up, 8 in 2024-02-23T00:43:59.752 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:59.753 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:43:59.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.762+0000 7f54f9d63700 10 client.30120.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:43:59.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.762+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_assign -1 4 2024-02-23T00:43:59.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.762+0000 7f54f9d63700 20 client.30120.objecter _get_session s=0x7f54f4028710 osd=1 3 2024-02-23T00:43:59.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.762+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f4028710 osd=1 3 2024-02-23T00:43:59.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.762+0000 7f54f9d63700 20 client.30120.objecter _get_session s=0x7f54f4028710 osd=1 3 2024-02-23T00:43:59.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.762+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_remove -1 4 2024-02-23T00:43:59.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.762+0000 7f54f9d63700 20 client.30120.objecter get_session s=0x7f54f4028710 osd=1 3 2024-02-23T00:43:59.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.762+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_assign 1 4 2024-02-23T00:43:59.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.762+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f4028710 osd=1 4 2024-02-23T00:43:59.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.762+0000 7f54f9d63700 10 client.30120.objecter _send_command 4 2024-02-23T00:43:59.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.762+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f54f4059360 con 0x7f54f40598f0 2024-02-23T00:43:59.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.764+0000 7f54f0ff9700 1 -- 172.21.15.47:0/3807262655 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f54f4059360 con 0x7f54f40598f0 2024-02-23T00:43:59.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.764+0000 7f54f0ff9700 10 client.30120.objecter ms_dispatch 0x7f54f4001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:43:59.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.764+0000 7f54f0ff9700 10 client.30120.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:43:59.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.764+0000 7f54f0ff9700 20 client.30120.objecter put_session s=0x7f54f4028710 osd=1 4 2024-02-23T00:43:59.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.764+0000 7f54f0ff9700 15 client.30120.objecter _session_command_op_remove 1 4 2024-02-23T00:43:59.767 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:43:59.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.764+0000 7f54f9d63700 10 client.30120.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:59.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.764+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_assign -1 5 2024-02-23T00:43:59.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.764+0000 7f54f9d63700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f54f4028270 0x7f54f4118ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:59.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.764+0000 7f54f9d63700 20 client.30120.objecter _get_session s=0x7f54f4059770 osd=2 3 2024-02-23T00:43:59.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.764+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f4059770 osd=2 3 2024-02-23T00:43:59.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.764+0000 7f54f9d63700 20 client.30120.objecter _get_session s=0x7f54f4059770 osd=2 3 2024-02-23T00:43:59.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.764+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_remove -1 5 2024-02-23T00:43:59.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.764+0000 7f54f9d63700 20 client.30120.objecter get_session s=0x7f54f4059770 osd=2 3 2024-02-23T00:43:59.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.764+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_assign 2 5 2024-02-23T00:43:59.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.764+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f4059770 osd=2 4 2024-02-23T00:43:59.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.764+0000 7f54f9d63700 10 client.30120.objecter _send_command 5 2024-02-23T00:43:59.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.764+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54f406cf20 con 0x7f54f4028270 2024-02-23T00:43:59.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.764+0000 7f54f2ffd700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f54f4028270 0x7f54f4118ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:59.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.764+0000 7f54f0ff9700 10 client.30120.objecter ms_handle_connect 0x7f54f4028270 2024-02-23T00:43:59.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.764+0000 7f54f2ffd700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f54f4028270 0x7f54f4118ee0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:59.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.765+0000 7f54f0ff9700 1 -- 172.21.15.47:0/3807262655 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f54f406cf20 con 0x7f54f4028270 2024-02-23T00:43:59.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.765+0000 7f54f0ff9700 10 client.30120.objecter ms_dispatch 0x7f54f4001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:43:59.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.765+0000 7f54f0ff9700 10 client.30120.objecter _finish_command 5 = system:0 2024-02-23T00:43:59.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.765+0000 7f54f0ff9700 20 client.30120.objecter put_session s=0x7f54f4059770 osd=2 4 2024-02-23T00:43:59.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.765+0000 7f54f0ff9700 15 client.30120.objecter _session_command_op_remove 2 5 2024-02-23T00:43:59.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.780+0000 7f54f9d63700 10 client.30120.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:43:59.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.780+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_assign -1 6 2024-02-23T00:43:59.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.780+0000 7f54f9d63700 20 client.30120.objecter _get_session s=0x7f54f4059770 osd=2 3 2024-02-23T00:43:59.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.780+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f4059770 osd=2 3 2024-02-23T00:43:59.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.780+0000 7f54f9d63700 20 client.30120.objecter _get_session s=0x7f54f4059770 osd=2 3 2024-02-23T00:43:59.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.780+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_remove -1 6 2024-02-23T00:43:59.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.780+0000 7f54f9d63700 20 client.30120.objecter get_session s=0x7f54f4059770 osd=2 3 2024-02-23T00:43:59.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.780+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_assign 2 6 2024-02-23T00:43:59.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.780+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f4059770 osd=2 4 2024-02-23T00:43:59.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.780+0000 7f54f9d63700 10 client.30120.objecter _send_command 6 2024-02-23T00:43:59.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.780+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f54f40277d0 con 0x7f54f4028270 2024-02-23T00:43:59.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.781+0000 7f54f0ff9700 1 -- 172.21.15.47:0/3807262655 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f54f40277d0 con 0x7f54f4028270 2024-02-23T00:43:59.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.781+0000 7f54f0ff9700 10 client.30120.objecter ms_dispatch 0x7f54f4001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:43:59.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.781+0000 7f54f0ff9700 10 client.30120.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:43:59.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.781+0000 7f54f0ff9700 20 client.30120.objecter put_session s=0x7f54f4059770 osd=2 4 2024-02-23T00:43:59.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.781+0000 7f54f0ff9700 15 client.30120.objecter _session_command_op_remove 2 6 2024-02-23T00:43:59.784 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:43:59.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.781+0000 7f54f9d63700 10 client.30120.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:59.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.781+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_assign -1 7 2024-02-23T00:43:59.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.781+0000 7f54f9d63700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f54f411a550 0x7f54f411a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:59.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.781+0000 7f54f9d63700 20 client.30120.objecter _get_session s=0x7f54f411bf80 osd=3 3 2024-02-23T00:43:59.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.781+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f411bf80 osd=3 3 2024-02-23T00:43:59.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.781+0000 7f54f9d63700 20 client.30120.objecter _get_session s=0x7f54f411bf80 osd=3 3 2024-02-23T00:43:59.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.781+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_remove -1 7 2024-02-23T00:43:59.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.781+0000 7f54f9d63700 20 client.30120.objecter get_session s=0x7f54f411bf80 osd=3 3 2024-02-23T00:43:59.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.781+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_assign 3 7 2024-02-23T00:43:59.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.781+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f411bf80 osd=3 4 2024-02-23T00:43:59.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.781+0000 7f54f9d63700 10 client.30120.objecter _send_command 7 2024-02-23T00:43:59.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.781+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54f411c680 con 0x7f54f411a550 2024-02-23T00:43:59.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.781+0000 7f54f3fff700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f54f411a550 0x7f54f411a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:59.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.782+0000 7f54f0ff9700 10 client.30120.objecter ms_handle_connect 0x7f54f411a550 2024-02-23T00:43:59.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.782+0000 7f54f3fff700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f54f411a550 0x7f54f411a990 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:59.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.782+0000 7f54f0ff9700 1 -- 172.21.15.47:0/3807262655 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f54f411c680 con 0x7f54f411a550 2024-02-23T00:43:59.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.782+0000 7f54f0ff9700 10 client.30120.objecter ms_dispatch 0x7f54f4001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:43:59.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.782+0000 7f54f0ff9700 10 client.30120.objecter _finish_command 7 = system:0 2024-02-23T00:43:59.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.782+0000 7f54f0ff9700 20 client.30120.objecter put_session s=0x7f54f411bf80 osd=3 4 2024-02-23T00:43:59.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.782+0000 7f54f0ff9700 15 client.30120.objecter _session_command_op_remove 3 7 2024-02-23T00:43:59.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.797+0000 7f54f9d63700 10 client.30120.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:43:59.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.797+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_assign -1 8 2024-02-23T00:43:59.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.797+0000 7f54f9d63700 20 client.30120.objecter _get_session s=0x7f54f411bf80 osd=3 3 2024-02-23T00:43:59.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.797+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f411bf80 osd=3 3 2024-02-23T00:43:59.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.797+0000 7f54f9d63700 20 client.30120.objecter _get_session s=0x7f54f411bf80 osd=3 3 2024-02-23T00:43:59.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.797+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_remove -1 8 2024-02-23T00:43:59.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.797+0000 7f54f9d63700 20 client.30120.objecter get_session s=0x7f54f411bf80 osd=3 3 2024-02-23T00:43:59.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.797+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_assign 3 8 2024-02-23T00:43:59.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.797+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f411bf80 osd=3 4 2024-02-23T00:43:59.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.797+0000 7f54f9d63700 10 client.30120.objecter _send_command 8 2024-02-23T00:43:59.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.797+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f54f411b0c0 con 0x7f54f411a550 2024-02-23T00:43:59.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.798+0000 7f54f0ff9700 1 -- 172.21.15.47:0/3807262655 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f54f411b0c0 con 0x7f54f411a550 2024-02-23T00:43:59.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.798+0000 7f54f0ff9700 10 client.30120.objecter ms_dispatch 0x7f54f4001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:43:59.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.798+0000 7f54f0ff9700 10 client.30120.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:43:59.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.798+0000 7f54f0ff9700 20 client.30120.objecter put_session s=0x7f54f411bf80 osd=3 4 2024-02-23T00:43:59.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.798+0000 7f54f0ff9700 15 client.30120.objecter _session_command_op_remove 3 8 2024-02-23T00:43:59.801 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:43:59.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.798+0000 7f54f9d63700 10 client.30120.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:59.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.798+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_assign -1 9 2024-02-23T00:43:59.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.798+0000 7f54f9d63700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f54f411b230 0x7f54f4125270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.798+0000 7f54f9d63700 20 client.30120.objecter _get_session s=0x7f54f411b0c0 osd=4 3 2024-02-23T00:43:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.798+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f411b0c0 osd=4 3 2024-02-23T00:43:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.798+0000 7f54f9d63700 20 client.30120.objecter _get_session s=0x7f54f411b0c0 osd=4 3 2024-02-23T00:43:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.798+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_remove -1 9 2024-02-23T00:43:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.798+0000 7f54f9d63700 20 client.30120.objecter get_session s=0x7f54f411b0c0 osd=4 3 2024-02-23T00:43:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.798+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_assign 4 9 2024-02-23T00:43:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.798+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f411b0c0 osd=4 4 2024-02-23T00:43:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.798+0000 7f54f9d63700 10 client.30120.objecter _send_command 9 2024-02-23T00:43:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.798+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54f4128040 con 0x7f54f411b230 2024-02-23T00:43:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.799+0000 7f54f37fe700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f54f411b230 0x7f54f4125270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.799+0000 7f54f0ff9700 10 client.30120.objecter ms_handle_connect 0x7f54f411b230 2024-02-23T00:43:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.799+0000 7f54f37fe700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f54f411b230 0x7f54f4125270 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:59.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.800+0000 7f54f0ff9700 1 -- 172.21.15.47:0/3807262655 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f54f4128040 con 0x7f54f411b230 2024-02-23T00:43:59.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.800+0000 7f54f0ff9700 10 client.30120.objecter ms_dispatch 0x7f54f4001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:43:59.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.800+0000 7f54f0ff9700 10 client.30120.objecter _finish_command 9 = system:0 2024-02-23T00:43:59.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.800+0000 7f54f0ff9700 20 client.30120.objecter put_session s=0x7f54f411b0c0 osd=4 4 2024-02-23T00:43:59.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.800+0000 7f54f0ff9700 15 client.30120.objecter _session_command_op_remove 4 9 2024-02-23T00:43:59.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.814+0000 7f54f9d63700 10 client.30120.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:43:59.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.814+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_assign -1 10 2024-02-23T00:43:59.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.814+0000 7f54f9d63700 20 client.30120.objecter _get_session s=0x7f54f411b0c0 osd=4 3 2024-02-23T00:43:59.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.814+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f411b0c0 osd=4 3 2024-02-23T00:43:59.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.814+0000 7f54f9d63700 20 client.30120.objecter _get_session s=0x7f54f411b0c0 osd=4 3 2024-02-23T00:43:59.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.814+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_remove -1 10 2024-02-23T00:43:59.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.814+0000 7f54f9d63700 20 client.30120.objecter get_session s=0x7f54f411b0c0 osd=4 3 2024-02-23T00:43:59.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.814+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_assign 4 10 2024-02-23T00:43:59.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.814+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f411b0c0 osd=4 4 2024-02-23T00:43:59.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.814+0000 7f54f9d63700 10 client.30120.objecter _send_command 10 2024-02-23T00:43:59.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.814+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f54f4128040 con 0x7f54f411b230 2024-02-23T00:43:59.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.816+0000 7f54f0ff9700 1 -- 172.21.15.47:0/3807262655 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f54f4128040 con 0x7f54f411b230 2024-02-23T00:43:59.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.816+0000 7f54f0ff9700 10 client.30120.objecter ms_dispatch 0x7f54f4001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:43:59.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.816+0000 7f54f0ff9700 10 client.30120.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:43:59.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.816+0000 7f54f0ff9700 20 client.30120.objecter put_session s=0x7f54f411b0c0 osd=4 4 2024-02-23T00:43:59.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.816+0000 7f54f0ff9700 15 client.30120.objecter _session_command_op_remove 4 10 2024-02-23T00:43:59.819 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:43:59.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.816+0000 7f54f9d63700 10 client.30120.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:59.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.816+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_assign -1 11 2024-02-23T00:43:59.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.816+0000 7f54f9d63700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f54f4127370 0x7f54f41277f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:59.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.816+0000 7f54f9d63700 20 client.30120.objecter _get_session s=0x7f54f4128040 osd=5 3 2024-02-23T00:43:59.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.816+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f4128040 osd=5 3 2024-02-23T00:43:59.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.816+0000 7f54f9d63700 20 client.30120.objecter _get_session s=0x7f54f4128040 osd=5 3 2024-02-23T00:43:59.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.816+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_remove -1 11 2024-02-23T00:43:59.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.816+0000 7f54f9d63700 20 client.30120.objecter get_session s=0x7f54f4128040 osd=5 3 2024-02-23T00:43:59.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.816+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_assign 5 11 2024-02-23T00:43:59.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.816+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f4128040 osd=5 4 2024-02-23T00:43:59.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.816+0000 7f54f9d63700 10 client.30120.objecter _send_command 11 2024-02-23T00:43:59.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.816+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54f412b3a0 con 0x7f54f4127370 2024-02-23T00:43:59.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.817+0000 7f54f2ffd700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f54f4127370 0x7f54f41277f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:59.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.817+0000 7f54f0ff9700 10 client.30120.objecter ms_handle_connect 0x7f54f4127370 2024-02-23T00:43:59.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.817+0000 7f54f2ffd700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f54f4127370 0x7f54f41277f0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:59.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.818+0000 7f54f0ff9700 1 -- 172.21.15.47:0/3807262655 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f54f412b3a0 con 0x7f54f4127370 2024-02-23T00:43:59.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.818+0000 7f54f0ff9700 10 client.30120.objecter ms_dispatch 0x7f54f4001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:43:59.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.818+0000 7f54f0ff9700 10 client.30120.objecter _finish_command 11 = system:0 2024-02-23T00:43:59.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.818+0000 7f54f0ff9700 20 client.30120.objecter put_session s=0x7f54f4128040 osd=5 4 2024-02-23T00:43:59.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.818+0000 7f54f0ff9700 15 client.30120.objecter _session_command_op_remove 5 11 2024-02-23T00:43:59.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.832+0000 7f54f9d63700 10 client.30120.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:43:59.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.832+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_assign -1 12 2024-02-23T00:43:59.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.832+0000 7f54f9d63700 20 client.30120.objecter _get_session s=0x7f54f4128040 osd=5 3 2024-02-23T00:43:59.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.832+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f4128040 osd=5 3 2024-02-23T00:43:59.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.832+0000 7f54f9d63700 20 client.30120.objecter _get_session s=0x7f54f4128040 osd=5 3 2024-02-23T00:43:59.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.832+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_remove -1 12 2024-02-23T00:43:59.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.832+0000 7f54f9d63700 20 client.30120.objecter get_session s=0x7f54f4128040 osd=5 3 2024-02-23T00:43:59.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.832+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_assign 5 12 2024-02-23T00:43:59.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.832+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f4128040 osd=5 4 2024-02-23T00:43:59.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.832+0000 7f54f9d63700 10 client.30120.objecter _send_command 12 2024-02-23T00:43:59.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.832+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f54f4125800 con 0x7f54f4127370 2024-02-23T00:43:59.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.834+0000 7f54f0ff9700 1 -- 172.21.15.47:0/3807262655 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f54f4125800 con 0x7f54f4127370 2024-02-23T00:43:59.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.834+0000 7f54f0ff9700 10 client.30120.objecter ms_dispatch 0x7f54f4001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:43:59.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.834+0000 7f54f0ff9700 10 client.30120.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:43:59.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.834+0000 7f54f0ff9700 20 client.30120.objecter put_session s=0x7f54f4128040 osd=5 4 2024-02-23T00:43:59.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.834+0000 7f54f0ff9700 15 client.30120.objecter _session_command_op_remove 5 12 2024-02-23T00:43:59.836 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:43:59.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.834+0000 7f54f9d63700 10 client.30120.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:59.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.834+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_assign -1 13 2024-02-23T00:43:59.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.834+0000 7f54f9d63700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f54f4125ee0 0x7f54f4126340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:59.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.834+0000 7f54f9d63700 20 client.30120.objecter _get_session s=0x7f54f4125800 osd=6 3 2024-02-23T00:43:59.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.834+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f4125800 osd=6 3 2024-02-23T00:43:59.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.834+0000 7f54f9d63700 20 client.30120.objecter _get_session s=0x7f54f4125800 osd=6 3 2024-02-23T00:43:59.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.834+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_remove -1 13 2024-02-23T00:43:59.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.834+0000 7f54f9d63700 20 client.30120.objecter get_session s=0x7f54f4125800 osd=6 3 2024-02-23T00:43:59.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.834+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_assign 6 13 2024-02-23T00:43:59.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.834+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f4125800 osd=6 4 2024-02-23T00:43:59.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.834+0000 7f54f9d63700 10 client.30120.objecter _send_command 13 2024-02-23T00:43:59.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.834+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54f4126980 con 0x7f54f4125ee0 2024-02-23T00:43:59.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.834+0000 7f54f3fff700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f54f4125ee0 0x7f54f4126340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:59.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.834+0000 7f54f3fff700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f54f4125ee0 0x7f54f4126340 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:59.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.834+0000 7f54f0ff9700 10 client.30120.objecter ms_handle_connect 0x7f54f4125ee0 2024-02-23T00:43:59.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.835+0000 7f54f0ff9700 1 -- 172.21.15.47:0/3807262655 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f54f4126980 con 0x7f54f4125ee0 2024-02-23T00:43:59.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.835+0000 7f54f0ff9700 10 client.30120.objecter ms_dispatch 0x7f54f4001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:43:59.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.835+0000 7f54f0ff9700 10 client.30120.objecter _finish_command 13 = system:0 2024-02-23T00:43:59.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.835+0000 7f54f0ff9700 20 client.30120.objecter put_session s=0x7f54f4125800 osd=6 4 2024-02-23T00:43:59.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.835+0000 7f54f0ff9700 15 client.30120.objecter _session_command_op_remove 6 13 2024-02-23T00:43:59.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.850+0000 7f54f9d63700 10 client.30120.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:43:59.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.850+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_assign -1 14 2024-02-23T00:43:59.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.850+0000 7f54f9d63700 20 client.30120.objecter _get_session s=0x7f54f4125800 osd=6 3 2024-02-23T00:43:59.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.850+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f4125800 osd=6 3 2024-02-23T00:43:59.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.850+0000 7f54f9d63700 20 client.30120.objecter _get_session s=0x7f54f4125800 osd=6 3 2024-02-23T00:43:59.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.850+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_remove -1 14 2024-02-23T00:43:59.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.850+0000 7f54f9d63700 20 client.30120.objecter get_session s=0x7f54f4125800 osd=6 3 2024-02-23T00:43:59.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.850+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_assign 6 14 2024-02-23T00:43:59.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.850+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f4125800 osd=6 4 2024-02-23T00:43:59.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.850+0000 7f54f9d63700 10 client.30120.objecter _send_command 14 2024-02-23T00:43:59.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.850+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f54f4126980 con 0x7f54f4125ee0 2024-02-23T00:43:59.853 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:43:59.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.851+0000 7f54f0ff9700 1 -- 172.21.15.47:0/3807262655 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f54f4126980 con 0x7f54f4125ee0 2024-02-23T00:43:59.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.851+0000 7f54f0ff9700 10 client.30120.objecter ms_dispatch 0x7f54f4001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:43:59.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.851+0000 7f54f0ff9700 10 client.30120.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:43:59.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.851+0000 7f54f0ff9700 20 client.30120.objecter put_session s=0x7f54f4125800 osd=6 4 2024-02-23T00:43:59.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.851+0000 7f54f0ff9700 15 client.30120.objecter _session_command_op_remove 6 14 2024-02-23T00:43:59.854 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:43:59.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.851+0000 7f54f9d63700 10 client.30120.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:43:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.851+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_assign -1 15 2024-02-23T00:43:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.851+0000 7f54f9d63700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f54f412c740 0x7f54f412eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:43:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.851+0000 7f54f9d63700 20 client.30120.objecter _get_session s=0x7f54f4126980 osd=7 3 2024-02-23T00:43:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.851+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f4126980 osd=7 3 2024-02-23T00:43:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.851+0000 7f54f9d63700 20 client.30120.objecter _get_session s=0x7f54f4126980 osd=7 3 2024-02-23T00:43:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.851+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_remove -1 15 2024-02-23T00:43:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.851+0000 7f54f9d63700 20 client.30120.objecter get_session s=0x7f54f4126980 osd=7 3 2024-02-23T00:43:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.851+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_assign 7 15 2024-02-23T00:43:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.851+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f4126980 osd=7 4 2024-02-23T00:43:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.851+0000 7f54f9d63700 10 client.30120.objecter _send_command 15 2024-02-23T00:43:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.851+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54f41317d0 con 0x7f54f412c740 2024-02-23T00:43:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.852+0000 7f54f37fe700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f54f412c740 0x7f54f412eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:43:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.852+0000 7f54f37fe700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f54f412c740 0x7f54f412eb30 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:43:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.852+0000 7f54f0ff9700 10 client.30120.objecter ms_handle_connect 0x7f54f412c740 2024-02-23T00:43:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.853+0000 7f54f0ff9700 1 -- 172.21.15.47:0/3807262655 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f54f41317d0 con 0x7f54f412c740 2024-02-23T00:43:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.853+0000 7f54f0ff9700 10 client.30120.objecter ms_dispatch 0x7f54f4001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:43:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.853+0000 7f54f0ff9700 10 client.30120.objecter _finish_command 15 = system:0 2024-02-23T00:43:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.853+0000 7f54f0ff9700 20 client.30120.objecter put_session s=0x7f54f4126980 osd=7 4 2024-02-23T00:43:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.853+0000 7f54f0ff9700 15 client.30120.objecter _session_command_op_remove 7 15 2024-02-23T00:43:59.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.868+0000 7f54f9d63700 10 client.30120.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:43:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.869+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_assign -1 16 2024-02-23T00:43:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.869+0000 7f54f9d63700 20 client.30120.objecter _get_session s=0x7f54f4126980 osd=7 3 2024-02-23T00:43:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.869+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f4126980 osd=7 3 2024-02-23T00:43:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.869+0000 7f54f9d63700 20 client.30120.objecter _get_session s=0x7f54f4126980 osd=7 3 2024-02-23T00:43:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.869+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_remove -1 16 2024-02-23T00:43:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.869+0000 7f54f9d63700 20 client.30120.objecter get_session s=0x7f54f4126980 osd=7 3 2024-02-23T00:43:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.869+0000 7f54f9d63700 15 client.30120.objecter _session_command_op_assign 7 16 2024-02-23T00:43:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.869+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f4126980 osd=7 4 2024-02-23T00:43:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.869+0000 7f54f9d63700 10 client.30120.objecter _send_command 16 2024-02-23T00:43:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.869+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f54f41317d0 con 0x7f54f412c740 2024-02-23T00:43:59.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.870+0000 7f54f0ff9700 1 -- 172.21.15.47:0/3807262655 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f54f41317d0 con 0x7f54f412c740 2024-02-23T00:43:59.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.870+0000 7f54f0ff9700 10 client.30120.objecter ms_dispatch 0x7f54f4001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:43:59.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.870+0000 7f54f0ff9700 10 client.30120.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:43:59.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.870+0000 7f54f0ff9700 20 client.30120.objecter put_session s=0x7f54f4126980 osd=7 4 2024-02-23T00:43:59.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.870+0000 7f54f0ff9700 15 client.30120.objecter _session_command_op_remove 7 16 2024-02-23T00:43:59.873 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:43:59.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 10 librados: watch_flush enter 2024-02-23T00:43:59.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 10 librados: watch_flush exit 2024-02-23T00:43:59.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 10 client.30120.objecter close_session for osd.0 2024-02-23T00:43:59.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f54f40fee20 msgr2=0x7f54f40ff200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:59.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f54f40fee20 0x7f54f40ff200 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:59.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f4106e60 osd=0 1 2024-02-23T00:43:59.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 10 client.30120.objecter close_session for osd.1 2024-02-23T00:43:59.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f54f40598f0 msgr2=0x7f54f4059d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:59.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f54f40598f0 0x7f54f4059d30 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:59.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f4028710 osd=1 1 2024-02-23T00:43:59.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 10 client.30120.objecter close_session for osd.2 2024-02-23T00:43:59.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f54f4028270 msgr2=0x7f54f4118ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:59.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f54f4028270 0x7f54f4118ee0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:59.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f4059770 osd=2 1 2024-02-23T00:43:59.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 10 client.30120.objecter close_session for osd.3 2024-02-23T00:43:59.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f54f411a550 msgr2=0x7f54f411a990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:59.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f54f411a550 0x7f54f411a990 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:59.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f411bf80 osd=3 1 2024-02-23T00:43:59.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 10 client.30120.objecter close_session for osd.4 2024-02-23T00:43:59.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f54f411b230 msgr2=0x7f54f4125270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:59.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f54f411b230 0x7f54f4125270 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:59.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f411b0c0 osd=4 1 2024-02-23T00:43:59.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 10 client.30120.objecter close_session for osd.5 2024-02-23T00:43:59.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f54f4127370 msgr2=0x7f54f41277f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:59.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f54f4127370 0x7f54f41277f0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:59.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f4128040 osd=5 1 2024-02-23T00:43:59.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 10 client.30120.objecter close_session for osd.6 2024-02-23T00:43:59.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f54f4125ee0 msgr2=0x7f54f4126340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:59.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f54f4125ee0 0x7f54f4126340 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:59.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f4125800 osd=6 1 2024-02-23T00:43:59.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 10 client.30120.objecter close_session for osd.7 2024-02-23T00:43:59.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f54f412c740 msgr2=0x7f54f412eb30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:59.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f54f412c740 0x7f54f412eb30 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:59.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 20 client.30120.objecter put_session s=0x7f54f4126980 osd=7 1 2024-02-23T00:43:59.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 20 client.30120.objecter shutdown clearing up homeless session... 2024-02-23T00:43:59.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 10 client.30120.objecter successfully canceled tick 2024-02-23T00:43:59.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f3fff700 1 -- 172.21.15.47:0/3807262655 reap_dead start 2024-02-23T00:43:59.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f3fff700 1 -- 172.21.15.47:0/3807262655 reap_dead start 2024-02-23T00:43:59.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f3fff700 1 -- 172.21.15.47:0/3807262655 reap_dead start 2024-02-23T00:43:59.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f3fff700 1 -- 172.21.15.47:0/3807262655 reap_dead start 2024-02-23T00:43:59.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f54dc062cc0 msgr2=0x7f54dc065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:59.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f54dc062cc0 0x7f54dc065180 secure :-1 s=READY pgs=2909 cs=0 l=1 rev1=1 rx=0x7f54e4011510 tx=0x7f54e400b040).stop 2024-02-23T00:43:59.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f3fff700 1 -- 172.21.15.47:0/3807262655 reap_dead start 2024-02-23T00:43:59.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54f40fdbb0 msgr2=0x7f54f410e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:43:59.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54f40fdbb0 0x7f54f410e3f0 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 rx=0x7f54e8004710 tx=0x7f54e8009cb0).stop 2024-02-23T00:43:59.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 shutdown_connections 2024-02-23T00:43:59.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54f40fdbb0 0x7f54f410e3f0 unknown :-1 s=CLOSED pgs=2431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:59.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 1 --2- 172.21.15.47:0/3807262655 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f54dc062cc0 0x7f54dc065180 unknown :-1 s=CLOSED pgs=2909 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:43:59.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 >> 172.21.15.47:0/3807262655 conn(0x7f54f40f3800 msgr2=0x7f54f40fb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:43:59.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 shutdown_connections 2024-02-23T00:43:59.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 1 -- 172.21.15.47:0/3807262655 wait complete. 2024-02-23T00:43:59.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:43:59.871+0000 7f54f9d63700 1 librados: shutdown 2024-02-23T00:43:59.878 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:43:59.878 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:43:59.878 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:43:59.878 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:43:59.878 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:43:59.878 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:43:59.878 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:43:59.878 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:43:59.887 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:43:59.905 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:43:59.907 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:43:59.908 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:43:59.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:58.727485+0000 mon.b (mon.1) 196 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:43:59.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:59.106782+0000 mon.a (mon.0) 1271 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi047-82386-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:43:59.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:43:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:59.106886+0000 mon.a (mon.0) 1272 : cluster [DBG] osdmap e167: 8 total, 8 up, 8 in 2024-02-23T00:44:00.009 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:44:00.012 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-23T00:44:00.040 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:00.079 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:44:00.082 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:00.082 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:44:00.093 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.4 dump_historic_ops 2024-02-23T00:44:00.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:00.113+0000 7f3cfdffb700 1 -- 172.21.15.47:0/4229644801 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 1, "weight": 1.0}]=0 reweighted osd.1 to 1 (10000) v168) v1 ==== 113+0+0 (secure 0 0 0) 0x7f3cec04c980 con 0x7f3d000fcb00 2024-02-23T00:44:00.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:00.116+0000 7f3d06f54700 10 librados: watch_flush enter 2024-02-23T00:44:00.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:00.119+0000 7f3d06f54700 10 librados: watch_flush exit 2024-02-23T00:44:00.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:00.119+0000 7f3d06f54700 20 client.40445.objecter shutdown clearing up homeless session... 2024-02-23T00:44:00.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:00.119+0000 7f3d06f54700 10 client.40445.objecter successfully canceled tick 2024-02-23T00:44:00.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:00.122+0000 7f3d06f54700 1 -- 172.21.15.47:0/4229644801 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3cf0062cc0 msgr2=0x7f3cf0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:00.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:00.122+0000 7f3d06f54700 1 --2- 172.21.15.47:0/4229644801 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3cf0062cc0 0x7f3cf0065180 secure :-1 s=READY pgs=2908 cs=0 l=1 rev1=1 rx=0x7f3cf4009bc0 tx=0x7f3cf4009cb0).stop 2024-02-23T00:44:00.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:00.122+0000 7f3d06f54700 1 -- 172.21.15.47:0/4229644801 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3d000fcb00 msgr2=0x7f3d00104630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:00.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:00.122+0000 7f3d06f54700 1 --2- 172.21.15.47:0/4229644801 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3d000fcb00 0x7f3d00104630 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 rx=0x7f3cec009400 tx=0x7f3cec009cb0).stop 2024-02-23T00:44:00.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:00.124+0000 7f3d06f54700 1 -- 172.21.15.47:0/4229644801 shutdown_connections 2024-02-23T00:44:00.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:00.124+0000 7f3d06f54700 1 --2- 172.21.15.47:0/4229644801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d000fd460 0x7f3d00104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:00.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:00.124+0000 7f3d06f54700 1 --2- 172.21.15.47:0/4229644801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d000f7970 0x7f3d00104140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:00.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:00.124+0000 7f3d06f54700 1 --2- 172.21.15.47:0/4229644801 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3cf0062cc0 0x7f3cf0065180 unknown :-1 s=CLOSED pgs=2908 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:00.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:00.124+0000 7f3d06f54700 1 --2- 172.21.15.47:0/4229644801 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3d000fcb00 0x7f3d00104630 unknown :-1 s=CLOSED pgs=2478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:00.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:00.124+0000 7f3d06f54700 1 -- 172.21.15.47:0/4229644801 >> 172.21.15.47:0/4229644801 conn(0x7f3d0006ffc0 msgr2=0x7f3d000faf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:00.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:00.135+0000 7f3d06f54700 1 -- 172.21.15.47:0/4229644801 shutdown_connections 2024-02-23T00:44:00.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:00.136+0000 7f3d06f54700 1 -- 172.21.15.47:0/4229644801 wait complete. 2024-02-23T00:44:00.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:00.138+0000 7f3d06f54700 1 librados: shutdown 2024-02-23T00:44:00.142 INFO:teuthology.orchestra.run.smithi047.stderr:reweighted osd.1 to 1 (10000) 2024-02-23T00:44:00.183 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:44:00.210 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:00.234 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:44:00.237 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:00.237 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:44:00.338 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:44:00.367 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:00.388 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:44:00.394 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:00.394 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:44:00.417 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:44:00.495 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:44:00.521 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:00.655 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:43:59.216520+0000 mgr.y (mgr.14517) 217 : cluster [DBG] pgmap v226: 346 pgs: 314 active+clean, 32 unknown; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 76 op/s 2024-02-23T00:44:00.655 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:59.636594+0000 mon.c (mon.2) 89 : audit [INF] from='client.? 172.21.15.47:0/4229644801' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-23T00:44:00.655 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:59.636908+0000 mon.a (mon.0) 1273 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-23T00:44:00.655 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:59.728142+0000 mon.a (mon.0) 1274 : audit [DBG] from='client.? 172.21.15.47:0/3807262655' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:44:00.655 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:43:59.728428+0000 mon.b (mon.1) 197 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:00.655 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:00.093047+0000 mon.a (mon.0) 1275 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:44:00.655 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:00.111517+0000 mon.a (mon.0) 1276 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-23T00:44:00.655 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:00.111596+0000 mon.a (mon.0) 1277 : cluster [DBG] osdmap e168: 8 total, 8 up, 8 in 2024-02-23T00:44:00.655 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:00.115093+0000 mon.c (mon.2) 90 : audit [INF] from='client.? 172.21.15.47:0/2205441042' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi047-82378-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:00.656 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:00.115640+0000 mon.a (mon.0) 1278 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi047-82378-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:00.669 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:44:00.672 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:00.672 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:44:00.773 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:44:00.800 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:44:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:44:00] "GET /metrics HTTP/1.1" 200 195393 "" "Prometheus/2.33.4" 2024-02-23T00:44:00.800 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:43:59.216520+0000 mgr.y (mgr.14517) 217 : cluster [DBG] pgmap v226: 346 pgs: 314 active+clean, 32 unknown; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 76 op/s 2024-02-23T00:44:00.800 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:59.636594+0000 mon.c (mon.2) 89 : audit [INF] from='client.? 172.21.15.47:0/4229644801' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-23T00:44:00.800 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:59.636908+0000 mon.a (mon.0) 1273 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-23T00:44:00.800 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:59.728142+0000 mon.a (mon.0) 1274 : audit [DBG] from='client.? 172.21.15.47:0/3807262655' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:44:00.801 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:43:59.728428+0000 mon.b (mon.1) 197 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:00.801 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:00.093047+0000 mon.a (mon.0) 1275 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:44:00.801 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:00.111517+0000 mon.a (mon.0) 1276 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-23T00:44:00.801 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:00.111596+0000 mon.a (mon.0) 1277 : cluster [DBG] osdmap e168: 8 total, 8 up, 8 in 2024-02-23T00:44:00.801 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:00.115093+0000 mon.c (mon.2) 90 : audit [INF] from='client.? 172.21.15.47:0/2205441042' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi047-82378-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:00.801 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:00.115640+0000 mon.a (mon.0) 1278 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi047-82378-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:00.801 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:43:59.216520+0000 mgr.y (mgr.14517) 217 : cluster [DBG] pgmap v226: 346 pgs: 314 active+clean, 32 unknown; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 76 op/s 2024-02-23T00:44:00.801 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:59.636594+0000 mon.c (mon.2) 89 : audit [INF] from='client.? 172.21.15.47:0/4229644801' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-23T00:44:00.801 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:59.636908+0000 mon.a (mon.0) 1273 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-23T00:44:00.801 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:59.728142+0000 mon.a (mon.0) 1274 : audit [DBG] from='client.? 172.21.15.47:0/3807262655' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:44:00.801 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:43:59.728428+0000 mon.b (mon.1) 197 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:00.801 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:00.093047+0000 mon.a (mon.0) 1275 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:44:00.802 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:00.111517+0000 mon.a (mon.0) 1276 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-23T00:44:00.802 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:00.111596+0000 mon.a (mon.0) 1277 : cluster [DBG] osdmap e168: 8 total, 8 up, 8 in 2024-02-23T00:44:00.802 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:00.115093+0000 mon.c (mon.2) 90 : audit [INF] from='client.? 172.21.15.47:0/2205441042' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi047-82378-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:00.802 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:00.115640+0000 mon.a (mon.0) 1278 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi047-82378-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:00.803 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:01.401 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight 2 1 2024-02-23T00:44:01.453 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:44:01.456 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:01.456 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:44:01.478 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:44:01.557 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:44:01.584 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:01.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:00.729381+0000 mon.b (mon.1) 198 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:01.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:00.752361+0000 osd.1 (osd.1) 105 : cluster [DBG] 93.8 scrub starts 2024-02-23T00:44:01.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:00.753956+0000 osd.1 (osd.1) 106 : cluster [DBG] 93.8 scrub ok 2024-02-23T00:44:01.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:01.113977+0000 mon.a (mon.0) 1279 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi047-82378-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:01.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:01.114232+0000 mon.a (mon.0) 1280 : cluster [DBG] osdmap e169: 8 total, 8 up, 8 in 2024-02-23T00:44:01.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:01.129552+0000 mon.c (mon.2) 91 : audit [INF] from='client.? 172.21.15.47:0/2662596439' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi047-82386-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:01.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:01.130107+0000 mon.a (mon.0) 1281 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi047-82386-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:01.779 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:00.729381+0000 mon.b (mon.1) 198 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:01.779 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:00.752361+0000 osd.1 (osd.1) 105 : cluster [DBG] 93.8 scrub starts 2024-02-23T00:44:01.779 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:00.753956+0000 osd.1 (osd.1) 106 : cluster [DBG] 93.8 scrub ok 2024-02-23T00:44:01.780 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:01.113977+0000 mon.a (mon.0) 1279 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi047-82378-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:01.780 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:01.114232+0000 mon.a (mon.0) 1280 : cluster [DBG] osdmap e169: 8 total, 8 up, 8 in 2024-02-23T00:44:01.780 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:01.129552+0000 mon.c (mon.2) 91 : audit [INF] from='client.? 172.21.15.47:0/2662596439' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi047-82386-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:01.780 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:01.130107+0000 mon.a (mon.0) 1281 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi047-82386-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:01.780 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:00.729381+0000 mon.b (mon.1) 198 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:01.780 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:00.752361+0000 osd.1 (osd.1) 105 : cluster [DBG] 93.8 scrub starts 2024-02-23T00:44:01.780 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:00.753956+0000 osd.1 (osd.1) 106 : cluster [DBG] 93.8 scrub ok 2024-02-23T00:44:01.780 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:01.113977+0000 mon.a (mon.0) 1279 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi047-82378-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:01.781 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:01.114232+0000 mon.a (mon.0) 1280 : cluster [DBG] osdmap e169: 8 total, 8 up, 8 in 2024-02-23T00:44:01.781 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:01.129552+0000 mon.c (mon.2) 91 : audit [INF] from='client.? 172.21.15.47:0/2662596439' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi047-82386-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:01.781 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:01.130107+0000 mon.a (mon.0) 1281 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi047-82386-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:01.933 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:44:01.936 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:01.936 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:44:02.037 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:44:02.065 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:02.267 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:44:02.270 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:02.270 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:44:02.371 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:44:02.400 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:02.455 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:44:02.527 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:44:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:44:02.564 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:44:02.565 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:44:02.657 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:44:02.848 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:44:02.850 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:44:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:44:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:44:02.851 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:01.216972+0000 mgr.y (mgr.14517) 218 : cluster [DBG] pgmap v229: 378 pgs: 314 active+clean, 64 unknown; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:44:02.851 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:01.399720+0000 osd.0 (osd.0) 91 : cluster [DBG] 93.3 scrub starts 2024-02-23T00:44:02.851 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:01.401358+0000 osd.0 (osd.0) 92 : cluster [DBG] 93.3 scrub ok 2024-02-23T00:44:02.851 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:01.452853+0000 osd.7 (osd.7) 99 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:44:02.851 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:01.453815+0000 osd.7 (osd.7) 100 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:44:02.851 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:01.730525+0000 mon.b (mon.1) 199 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:02.851 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:02.118404+0000 mon.a (mon.0) 1282 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi047-82386-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:02.851 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:02.118550+0000 mon.a (mon.0) 1283 : cluster [DBG] osdmap e170: 8 total, 8 up, 8 in 2024-02-23T00:44:02.852 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:02.852 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:44:02.953 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:44:02.969 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:01.216972+0000 mgr.y (mgr.14517) 218 : cluster [DBG] pgmap v229: 378 pgs: 314 active+clean, 64 unknown; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:44:02.969 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:01.399720+0000 osd.0 (osd.0) 91 : cluster [DBG] 93.3 scrub starts 2024-02-23T00:44:02.969 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:01.401358+0000 osd.0 (osd.0) 92 : cluster [DBG] 93.3 scrub ok 2024-02-23T00:44:02.969 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:01.452853+0000 osd.7 (osd.7) 99 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:44:02.969 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:01.453815+0000 osd.7 (osd.7) 100 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:44:02.969 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:01.730525+0000 mon.b (mon.1) 199 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:02.969 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:02.118404+0000 mon.a (mon.0) 1282 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi047-82386-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:02.969 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:02.118550+0000 mon.a (mon.0) 1283 : cluster [DBG] osdmap e170: 8 total, 8 up, 8 in 2024-02-23T00:44:02.970 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:01.216972+0000 mgr.y (mgr.14517) 218 : cluster [DBG] pgmap v229: 378 pgs: 314 active+clean, 64 unknown; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:44:02.970 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:01.399720+0000 osd.0 (osd.0) 91 : cluster [DBG] 93.3 scrub starts 2024-02-23T00:44:02.970 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:01.401358+0000 osd.0 (osd.0) 92 : cluster [DBG] 93.3 scrub ok 2024-02-23T00:44:02.970 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:01.452853+0000 osd.7 (osd.7) 99 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:44:02.970 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:01.453815+0000 osd.7 (osd.7) 100 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:44:02.970 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:01.730525+0000 mon.b (mon.1) 199 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:02.970 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:02.118404+0000 mon.a (mon.0) 1282 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi047-82386-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:02.970 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:02.118550+0000 mon.a (mon.0) 1283 : cluster [DBG] osdmap e170: 8 total, 8 up, 8 in 2024-02-23T00:44:02.985 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:03.366 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:44:03.368 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:03.369 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:44:03.470 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:44:03.498 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:03.521 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:44:03.523 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:03.524 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:44:03.529 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.5 dump_ops_in_flight 2024-02-23T00:44:03.625 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:44:03.653 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:03.831 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:44:03.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:01.922423+0000 osd.7 (osd.7) 101 : cluster [DBG] 93.0 scrub starts 2024-02-23T00:44:03.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:01.925905+0000 osd.7 (osd.7) 102 : cluster [DBG] 93.0 scrub ok 2024-02-23T00:44:03.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:02.357664+0000 mgr.y (mgr.14517) 219 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:44:03.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:02.731424+0000 mon.b (mon.1) 200 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:03.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:02.799283+0000 mon.b (mon.1) 201 : audit [DBG] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-23T00:44:03.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:02.799871+0000 mon.b (mon.1) 202 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi047-82457-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:03.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:02.800139+0000 mon.a (mon.0) 1284 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi047-82457-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:03.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:03.120095+0000 mon.a (mon.0) 1285 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi047-82457-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:03.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:03.120185+0000 mon.a (mon.0) 1286 : cluster [DBG] osdmap e171: 8 total, 8 up, 8 in 2024-02-23T00:44:03.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:03.135761+0000 mon.b (mon.1) 203 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi047-82457-1","app": "app2"}]: dispatch 2024-02-23T00:44:03.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:03.137194+0000 mon.b (mon.1) 204 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi047-82457-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:03.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:03.137711+0000 mon.b (mon.1) 205 : audit [INF] from='client.? 172.21.15.47:0/1379424711' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi047-82378-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:03.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:03.137854+0000 mon.a (mon.0) 1287 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi047-82457-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:03.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:03.139254+0000 mon.a (mon.0) 1288 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi047-82378-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:03.967 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:01.922423+0000 osd.7 (osd.7) 101 : cluster [DBG] 93.0 scrub starts 2024-02-23T00:44:03.967 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:01.925905+0000 osd.7 (osd.7) 102 : cluster [DBG] 93.0 scrub ok 2024-02-23T00:44:03.967 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:02.357664+0000 mgr.y (mgr.14517) 219 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:44:03.967 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:02.731424+0000 mon.b (mon.1) 200 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:03.967 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:02.799283+0000 mon.b (mon.1) 201 : audit [DBG] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-23T00:44:03.967 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:02.799871+0000 mon.b (mon.1) 202 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi047-82457-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:03.967 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:02.800139+0000 mon.a (mon.0) 1284 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi047-82457-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:03.968 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:03.120095+0000 mon.a (mon.0) 1285 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi047-82457-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:03.968 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:03.120185+0000 mon.a (mon.0) 1286 : cluster [DBG] osdmap e171: 8 total, 8 up, 8 in 2024-02-23T00:44:03.968 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:03.135761+0000 mon.b (mon.1) 203 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi047-82457-1","app": "app2"}]: dispatch 2024-02-23T00:44:03.968 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:03.137194+0000 mon.b (mon.1) 204 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi047-82457-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:03.968 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:03.137711+0000 mon.b (mon.1) 205 : audit [INF] from='client.? 172.21.15.47:0/1379424711' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi047-82378-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:03.968 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:03.137854+0000 mon.a (mon.0) 1287 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi047-82457-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:03.968 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:03.139254+0000 mon.a (mon.0) 1288 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi047-82378-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:03.968 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:01.922423+0000 osd.7 (osd.7) 101 : cluster [DBG] 93.0 scrub starts 2024-02-23T00:44:03.968 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:01.925905+0000 osd.7 (osd.7) 102 : cluster [DBG] 93.0 scrub ok 2024-02-23T00:44:03.968 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:02.357664+0000 mgr.y (mgr.14517) 219 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:44:03.968 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:02.731424+0000 mon.b (mon.1) 200 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:03.969 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:02.799283+0000 mon.b (mon.1) 201 : audit [DBG] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-23T00:44:03.969 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:02.799871+0000 mon.b (mon.1) 202 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi047-82457-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:03.969 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:02.800139+0000 mon.a (mon.0) 1284 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi047-82457-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:03.969 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:03.120095+0000 mon.a (mon.0) 1285 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi047-82457-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:03.969 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:03.120185+0000 mon.a (mon.0) 1286 : cluster [DBG] osdmap e171: 8 total, 8 up, 8 in 2024-02-23T00:44:03.969 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:03.135761+0000 mon.b (mon.1) 203 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi047-82457-1","app": "app2"}]: dispatch 2024-02-23T00:44:03.969 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:03.137194+0000 mon.b (mon.1) 204 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi047-82457-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:03.969 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:03.137711+0000 mon.b (mon.1) 205 : audit [INF] from='client.? 172.21.15.47:0/1379424711' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi047-82378-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:03.969 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:03.137854+0000 mon.a (mon.0) 1287 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi047-82457-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:03.969 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:03.139254+0000 mon.a (mon.0) 1288 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi047-82378-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:04.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:02.946721+0000 osd.7 (osd.7) 103 : cluster [DBG] 93.1 scrub starts 2024-02-23T00:44:04.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:02.950498+0000 osd.7 (osd.7) 104 : cluster [DBG] 93.1 scrub ok 2024-02-23T00:44:04.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:03.217387+0000 mgr.y (mgr.14517) 220 : cluster [DBG] pgmap v232: 346 pgs: 314 active+clean, 32 unknown; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:44:04.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:03.732310+0000 mon.b (mon.1) 206 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:04.124511+0000 mon.a (mon.0) 1289 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi047-82457-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:04.124600+0000 mon.a (mon.0) 1290 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi047-82378-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:04.124713+0000 mon.a (mon.0) 1291 : cluster [DBG] osdmap e172: 8 total, 8 up, 8 in 2024-02-23T00:44:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:04.138263+0000 mon.c (mon.2) 92 : audit [INF] from='client.? 172.21.15.47:0/3485668706' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi047-82386-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:04.139292+0000 mon.b (mon.1) 207 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi047-82457-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-23T00:44:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:04.139552+0000 mon.a (mon.0) 1292 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi047-82386-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:04.155411+0000 mon.b (mon.1) 208 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi047-82457-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-23T00:44:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:04.156984+0000 mon.a (mon.0) 1293 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi047-82457-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-23T00:44:05.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.033+0000 7f3103ea3700 1 -- 172.21.15.47:0/2400232450 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f30fc0fcbb0 msgr2=0x7f30fc0fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:05.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.033+0000 7f3103ea3700 1 --2- 172.21.15.47:0/2400232450 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f30fc0fcbb0 0x7f30fc0fcf90 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 rx=0x7f30f00047b0 tx=0x7f30f0009cb0).stop 2024-02-23T00:44:05.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.034+0000 7f3103ea3700 1 -- 172.21.15.47:0/2400232450 shutdown_connections 2024-02-23T00:44:05.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.034+0000 7f3103ea3700 1 --2- 172.21.15.47:0/2400232450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30fc0f7400 0x7f30fc103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:05.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.034+0000 7f3103ea3700 1 --2- 172.21.15.47:0/2400232450 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f30fc0fcbb0 0x7f30fc0fcf90 unknown :-1 s=CLOSED pgs=2484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:05.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.034+0000 7f3103ea3700 1 --2- 172.21.15.47:0/2400232450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30fc0fd510 0x7f30fc0f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:05.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.034+0000 7f3103ea3700 1 -- 172.21.15.47:0/2400232450 >> 172.21.15.47:0/2400232450 conn(0x7f30fc0f2820 msgr2=0x7f30fc0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:05.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.034+0000 7f3103ea3700 1 -- 172.21.15.47:0/2400232450 shutdown_connections 2024-02-23T00:44:05.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.034+0000 7f3103ea3700 1 -- 172.21.15.47:0/2400232450 wait complete. 2024-02-23T00:44:05.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.034+0000 7f3103ea3700 1 librados: starting msgr at 2024-02-23T00:44:05.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.034+0000 7f3103ea3700 1 librados: starting objecter 2024-02-23T00:44:05.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.034+0000 7f3103ea3700 1 Processor -- start 2024-02-23T00:44:05.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.034+0000 7f3103ea3700 1 -- start start 2024-02-23T00:44:05.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.034+0000 7f3103ea3700 1 librados: setting wanted keys 2024-02-23T00:44:05.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.034+0000 7f3103ea3700 1 librados: calling monclient init 2024-02-23T00:44:05.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.035+0000 7f3103ea3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30fc0f7400 0x7f30fc0f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:05.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.035+0000 7f3103ea3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30fc0fcbb0 0x7f30fc0f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:05.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.035+0000 7f3103ea3700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f30fc0fd510 0x7f30fc0f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:05.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.035+0000 7f3103ea3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc116460 con 0x7f30fc0fcbb0 2024-02-23T00:44:05.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.035+0000 7f3103ea3700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc1165d0 con 0x7f30fc0f7400 2024-02-23T00:44:05.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.035+0000 7f3103ea3700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc116790 con 0x7f30fc0fd510 2024-02-23T00:44:05.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.035+0000 7f310143e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30fc0fcbb0 0x7f30fc0f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:05.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.035+0000 7f310143e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30fc0fcbb0 0x7f30fc0f8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56666/0 (socket says 172.21.15.47:56666) 2024-02-23T00:44:05.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.035+0000 7f310143e700 1 -- 172.21.15.47:0/984423723 learned_addr learned my addr 172.21.15.47:0/984423723 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:44:05.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.035+0000 7f3102440700 1 --2- 172.21.15.47:0/984423723 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f30fc0fd510 0x7f30fc0f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:05.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.035+0000 7f3101c3f700 1 --2- 172.21.15.47:0/984423723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30fc0f7400 0x7f30fc0f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:05.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.035+0000 7f3102440700 1 -- 172.21.15.47:0/984423723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30fc0f7400 msgr2=0x7f30fc0f89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:05.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.035+0000 7f3102440700 1 --2- 172.21.15.47:0/984423723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30fc0f7400 0x7f30fc0f89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:05.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.035+0000 7f3102440700 1 -- 172.21.15.47:0/984423723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30fc0fcbb0 msgr2=0x7f30fc0f8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:05.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.035+0000 7f3102440700 1 --2- 172.21.15.47:0/984423723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30fc0fcbb0 0x7f30fc0f8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:05.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.035+0000 7f3102440700 1 -- 172.21.15.47:0/984423723 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f40097c0 con 0x7f30fc0fd510 2024-02-23T00:44:05.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.035+0000 7f310143e700 1 --2- 172.21.15.47:0/984423723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30fc0fcbb0 0x7f30fc0f8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:44:05.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.035+0000 7f3102440700 1 --2- 172.21.15.47:0/984423723 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f30fc0fd510 0x7f30fc0f9390 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 rx=0x7f30f4001df0 tx=0x7f30f4004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:05.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.035+0000 7f30eeffd700 10 client.?.objecter ms_handle_connect 0x7f30fc0fd510 2024-02-23T00:44:05.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.035+0000 7f30eeffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:44:05.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.035+0000 7f30eeffd700 1 -- 172.21.15.47:0/984423723 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f40099f0 con 0x7f30fc0fd510 2024-02-23T00:44:05.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.035+0000 7f3103ea3700 1 -- 172.21.15.47:0/984423723 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30f0005040 con 0x7f30fc0fd510 2024-02-23T00:44:05.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.036+0000 7f30eeffd700 1 -- 172.21.15.47:0/984423723 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f30f40090d0 con 0x7f30fc0fd510 2024-02-23T00:44:05.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.036+0000 7f3103ea3700 10 client.40475.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:44:05.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.036+0000 7f30eeffd700 1 -- 172.21.15.47:0/984423723 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f4007b20 con 0x7f30fc0fd510 2024-02-23T00:44:05.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.036+0000 7f3103ea3700 1 -- 172.21.15.47:0/984423723 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30fc116e20 con 0x7f30fc0fd510 2024-02-23T00:44:05.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.036+0000 7f3103ea3700 1 librados: init done 2024-02-23T00:44:05.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.036+0000 7f3103ea3700 1 -- 172.21.15.47:0/984423723 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30fc05fa60 con 0x7f30fc0fd510 2024-02-23T00:44:05.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.037+0000 7f30eeffd700 1 -- 172.21.15.47:0/984423723 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f30f400fad0 con 0x7f30fc0fd510 2024-02-23T00:44:05.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.037+0000 7f30eeffd700 1 --2- 172.21.15.47:0/984423723 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f30e8062cc0 0x7f30e8065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:05.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.037+0000 7f30eeffd700 1 -- 172.21.15.47:0/984423723 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(172..172 src has 48..172) v4 ==== 10432+0+0 (secure 0 0 0) 0x7f30f407e330 con 0x7f30fc0fd510 2024-02-23T00:44:05.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.037+0000 7f30eeffd700 10 client.40475.objecter ms_dispatch 0x7f30fc001550 osd_map(172..172 src has 48..172) v4 2024-02-23T00:44:05.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.037+0000 7f30eeffd700 3 client.40475.objecter handle_osd_map got epochs [172,172] > 0 2024-02-23T00:44:05.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.037+0000 7f30eeffd700 3 client.40475.objecter handle_osd_map decoding full epoch 172 2024-02-23T00:44:05.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.037+0000 7f3101c3f700 1 --2- 172.21.15.47:0/984423723 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f30e8062cc0 0x7f30e8065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:05.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.037+0000 7f30eeffd700 20 client.40475.objecter dump_active .. 0 homeless 2024-02-23T00:44:05.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.038+0000 7f30eeffd700 10 client.40475.objecter ms_handle_connect 0x7f30e8062cc0 2024-02-23T00:44:05.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.038+0000 7f3101c3f700 1 --2- 172.21.15.47:0/984423723 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f30e8062cc0 0x7f30e8065180 secure :-1 s=READY pgs=2914 cs=0 l=1 rev1=1 rx=0x7f30f0004e20 tx=0x7f30f0009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:05.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.042+0000 7f30eeffd700 1 -- 172.21.15.47:0/984423723 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f30f404d490 con 0x7f30fc0fd510 2024-02-23T00:44:05.056 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:02.946721+0000 osd.7 (osd.7) 103 : cluster [DBG] 93.1 scrub starts 2024-02-23T00:44:05.056 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:02.950498+0000 osd.7 (osd.7) 104 : cluster [DBG] 93.1 scrub ok 2024-02-23T00:44:05.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:03.217387+0000 mgr.y (mgr.14517) 220 : cluster [DBG] pgmap v232: 346 pgs: 314 active+clean, 32 unknown; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:44:05.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:03.732310+0000 mon.b (mon.1) 206 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:05.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:04.124511+0000 mon.a (mon.0) 1289 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi047-82457-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:05.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:04.124600+0000 mon.a (mon.0) 1290 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi047-82378-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:05.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:04.124713+0000 mon.a (mon.0) 1291 : cluster [DBG] osdmap e172: 8 total, 8 up, 8 in 2024-02-23T00:44:05.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:04.138263+0000 mon.c (mon.2) 92 : audit [INF] from='client.? 172.21.15.47:0/3485668706' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi047-82386-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:05.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:04.139292+0000 mon.b (mon.1) 207 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi047-82457-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-23T00:44:05.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:04.139552+0000 mon.a (mon.0) 1292 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi047-82386-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:05.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:04.155411+0000 mon.b (mon.1) 208 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi047-82457-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-23T00:44:05.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:04.156984+0000 mon.a (mon.0) 1293 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi047-82457-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-23T00:44:05.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:02.946721+0000 osd.7 (osd.7) 103 : cluster [DBG] 93.1 scrub starts 2024-02-23T00:44:05.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:02.950498+0000 osd.7 (osd.7) 104 : cluster [DBG] 93.1 scrub ok 2024-02-23T00:44:05.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:03.217387+0000 mgr.y (mgr.14517) 220 : cluster [DBG] pgmap v232: 346 pgs: 314 active+clean, 32 unknown; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:44:05.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:03.732310+0000 mon.b (mon.1) 206 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:05.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:04.124511+0000 mon.a (mon.0) 1289 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi047-82457-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:05.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:04.124600+0000 mon.a (mon.0) 1290 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi047-82378-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:05.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:04.124713+0000 mon.a (mon.0) 1291 : cluster [DBG] osdmap e172: 8 total, 8 up, 8 in 2024-02-23T00:44:05.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:04.138263+0000 mon.c (mon.2) 92 : audit [INF] from='client.? 172.21.15.47:0/3485668706' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi047-82386-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:05.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:04.139292+0000 mon.b (mon.1) 207 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi047-82457-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-23T00:44:05.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:04.139552+0000 mon.a (mon.0) 1292 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi047-82386-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:05.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:04.155411+0000 mon.b (mon.1) 208 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi047-82457-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-23T00:44:05.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:04.156984+0000 mon.a (mon.0) 1293 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi047-82457-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-23T00:44:05.105 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:44:05.114 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:05.114 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:44:05.215 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:44:05.245 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:05.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.256+0000 7f3103ea3700 1 -- 172.21.15.47:0/984423723 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7f30fc05ee80 con 0x7f30fc0fd510 2024-02-23T00:44:05.351 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:44:05.354 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:05.354 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:44:05.455 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:44:05.484 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:05.538 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:44:05.540 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:05.541 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:44:05.641 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:44:05.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.640+0000 7fdd84bda700 1 -- 172.21.15.47:0/44124522 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdd800fdbb0 msgr2=0x7fdd800fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:05.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.640+0000 7fdd84bda700 1 --2- 172.21.15.47:0/44124522 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdd800fdbb0 0x7fdd800fdf90 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 rx=0x7fdd70004870 tx=0x7fdd70009cb0).stop 2024-02-23T00:44:05.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.640+0000 7fdd84bda700 1 -- 172.21.15.47:0/44124522 shutdown_connections 2024-02-23T00:44:05.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.640+0000 7fdd84bda700 1 --2- 172.21.15.47:0/44124522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd800f8420 0x7fdd80104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:05.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.640+0000 7fdd84bda700 1 --2- 172.21.15.47:0/44124522 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdd800fdbb0 0x7fdd800fdf90 unknown :-1 s=CLOSED pgs=2487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:05.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.640+0000 7fdd84bda700 1 --2- 172.21.15.47:0/44124522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd800fe510 0x7fdd800f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:05.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.640+0000 7fdd84bda700 1 -- 172.21.15.47:0/44124522 >> 172.21.15.47:0/44124522 conn(0x7fdd800f3800 msgr2=0x7fdd800f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:05.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.640+0000 7fdd84bda700 1 -- 172.21.15.47:0/44124522 shutdown_connections 2024-02-23T00:44:05.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.640+0000 7fdd84bda700 1 -- 172.21.15.47:0/44124522 wait complete. 2024-02-23T00:44:05.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.641+0000 7fdd84bda700 1 librados: starting msgr at 2024-02-23T00:44:05.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.641+0000 7fdd84bda700 1 librados: starting objecter 2024-02-23T00:44:05.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.641+0000 7fdd84bda700 1 Processor -- start 2024-02-23T00:44:05.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.641+0000 7fdd84bda700 1 -- start start 2024-02-23T00:44:05.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.641+0000 7fdd84bda700 1 librados: setting wanted keys 2024-02-23T00:44:05.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.641+0000 7fdd84bda700 1 librados: calling monclient init 2024-02-23T00:44:05.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.641+0000 7fdd84bda700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd800f8420 0x7fdd8006b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:05.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.641+0000 7fdd84bda700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd800fdbb0 0x7fdd8006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:05.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.641+0000 7fdd84bda700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdd800fe510 0x7fdd8006bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:05.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.641+0000 7fdd84bda700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd801163e0 con 0x7fdd800f8420 2024-02-23T00:44:05.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.641+0000 7fdd84bda700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd80116550 con 0x7fdd800fdbb0 2024-02-23T00:44:05.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.641+0000 7fdd84bda700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd80116710 con 0x7fdd800fe510 2024-02-23T00:44:05.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.641+0000 7fdd7ed9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdd800fe510 0x7fdd8006bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:05.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.641+0000 7fdd7ed9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdd800fe510 0x7fdd8006bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:48440/0 (socket says 172.21.15.47:48440) 2024-02-23T00:44:05.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.641+0000 7fdd7ed9d700 1 -- 172.21.15.47:0/228618125 learned_addr learned my addr 172.21.15.47:0/228618125 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:44:05.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.641+0000 7fdd7e59c700 1 --2- 172.21.15.47:0/228618125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd800f8420 0x7fdd8006b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:05.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.641+0000 7fdd7dd9b700 1 --2- 172.21.15.47:0/228618125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd800fdbb0 0x7fdd8006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:05.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.642+0000 7fdd7ed9d700 1 -- 172.21.15.47:0/228618125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd800fdbb0 msgr2=0x7fdd8006b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:05.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.642+0000 7fdd7ed9d700 1 --2- 172.21.15.47:0/228618125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd800fdbb0 0x7fdd8006b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:05.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.642+0000 7fdd7ed9d700 1 -- 172.21.15.47:0/228618125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd800f8420 msgr2=0x7fdd8006b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:05.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.642+0000 7fdd7ed9d700 1 --2- 172.21.15.47:0/228618125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd800f8420 0x7fdd8006b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:05.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.642+0000 7fdd7ed9d700 1 -- 172.21.15.47:0/228618125 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd68005040 con 0x7fdd800fe510 2024-02-23T00:44:05.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.642+0000 7fdd7ed9d700 1 --2- 172.21.15.47:0/228618125 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdd800fe510 0x7fdd8006bab0 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 rx=0x7fdd7800ec20 tx=0x7fdd7800b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:05.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.642+0000 7fdd777fe700 10 client.?.objecter ms_handle_connect 0x7fdd800fe510 2024-02-23T00:44:05.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.642+0000 7fdd777fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:44:05.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.642+0000 7fdd777fe700 1 -- 172.21.15.47:0/228618125 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd78009950 con 0x7fdd800fe510 2024-02-23T00:44:05.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.642+0000 7fdd777fe700 1 -- 172.21.15.47:0/228618125 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fdd7800f040 con 0x7fdd800fe510 2024-02-23T00:44:05.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.642+0000 7fdd777fe700 1 -- 172.21.15.47:0/228618125 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd78007d10 con 0x7fdd800fe510 2024-02-23T00:44:05.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.643+0000 7fdd84bda700 1 -- 172.21.15.47:0/228618125 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd70005040 con 0x7fdd800fe510 2024-02-23T00:44:05.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.643+0000 7fdd84bda700 10 client.40487.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:44:05.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.643+0000 7fdd84bda700 1 -- 172.21.15.47:0/228618125 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd80116d20 con 0x7fdd800fe510 2024-02-23T00:44:05.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.643+0000 7fdd84bda700 1 librados: init done 2024-02-23T00:44:05.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.644+0000 7fdd84bda700 1 -- 172.21.15.47:0/228618125 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd80106e60 con 0x7fdd800fe510 2024-02-23T00:44:05.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.644+0000 7fdd777fe700 1 -- 172.21.15.47:0/228618125 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fdd780092e0 con 0x7fdd800fe510 2024-02-23T00:44:05.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.645+0000 7fdd777fe700 1 --2- 172.21.15.47:0/228618125 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdd6c062cc0 0x7fdd6c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:05.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.645+0000 7fdd777fe700 1 -- 172.21.15.47:0/228618125 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(173..173 src has 48..173) v4 ==== 10454+0+0 (secure 0 0 0) 0x7fdd7807db20 con 0x7fdd800fe510 2024-02-23T00:44:05.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.645+0000 7fdd777fe700 10 client.40487.objecter ms_dispatch 0x7fdd80001550 osd_map(173..173 src has 48..173) v4 2024-02-23T00:44:05.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.645+0000 7fdd777fe700 3 client.40487.objecter handle_osd_map got epochs [173,173] > 0 2024-02-23T00:44:05.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.645+0000 7fdd777fe700 3 client.40487.objecter handle_osd_map decoding full epoch 173 2024-02-23T00:44:05.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.645+0000 7fdd7e59c700 1 --2- 172.21.15.47:0/228618125 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdd6c062cc0 0x7fdd6c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:05.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.645+0000 7fdd777fe700 20 client.40487.objecter dump_active .. 0 homeless 2024-02-23T00:44:05.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.645+0000 7fdd777fe700 10 client.40487.objecter ms_handle_connect 0x7fdd6c062cc0 2024-02-23T00:44:05.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.645+0000 7fdd7e59c700 1 --2- 172.21.15.47:0/228618125 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdd6c062cc0 0x7fdd6c065180 secure :-1 s=READY pgs=2916 cs=0 l=1 rev1=1 rx=0x7fdd70004ee0 tx=0x7fdd7000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:05.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.649+0000 7fdd777fe700 1 -- 172.21.15.47:0/228618125 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fdd7804cce0 con 0x7fdd800fe510 2024-02-23T00:44:05.669 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:04.733531+0000 mon.b (mon.1) 209 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:05.670 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:04.807194+0000 osd.4 (osd.4) 89 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:44:05.670 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:04.807748+0000 osd.4 (osd.4) 90 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:44:05.670 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:05.099246+0000 mon.a (mon.0) 1294 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi047-82386-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:05.670 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:05.099374+0000 mon.a (mon.0) 1295 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi047-82457-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-23T00:44:05.670 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:05.099497+0000 mon.a (mon.0) 1296 : cluster [DBG] osdmap e173: 8 total, 8 up, 8 in 2024-02-23T00:44:05.670 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:05.117617+0000 mon.a (mon.0) 1297 : audit [INF] from='client.? 172.21.15.47:0/346485915' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82496-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:05.670 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:05.117881+0000 mon.b (mon.1) 210 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi047-82457-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-23T00:44:05.670 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:05.124014+0000 mon.a (mon.0) 1298 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi047-82457-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-23T00:44:05.670 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:05.257915+0000 mon.c (mon.2) 93 : audit [INF] from='client.? 172.21.15.47:0/984423723' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:44:05.671 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:05.591683+0000 mon.a (mon.0) 1299 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:44:05.671 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:04.733531+0000 mon.b (mon.1) 209 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:05.671 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:04.807194+0000 osd.4 (osd.4) 89 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:44:05.671 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:04.807748+0000 osd.4 (osd.4) 90 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:44:05.671 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:05.099246+0000 mon.a (mon.0) 1294 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi047-82386-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:05.671 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:05.099374+0000 mon.a (mon.0) 1295 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi047-82457-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-23T00:44:05.671 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:05.099497+0000 mon.a (mon.0) 1296 : cluster [DBG] osdmap e173: 8 total, 8 up, 8 in 2024-02-23T00:44:05.671 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:05.117617+0000 mon.a (mon.0) 1297 : audit [INF] from='client.? 172.21.15.47:0/346485915' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82496-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:05.671 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:05.117881+0000 mon.b (mon.1) 210 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi047-82457-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-23T00:44:05.672 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:05.124014+0000 mon.a (mon.0) 1298 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi047-82457-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-23T00:44:05.672 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:05.257915+0000 mon.c (mon.2) 93 : audit [INF] from='client.? 172.21.15.47:0/984423723' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:44:05.672 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:05.591683+0000 mon.a (mon.0) 1299 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:44:05.672 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:05.692 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:44:05.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:04.733531+0000 mon.b (mon.1) 209 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:05.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:04.807194+0000 osd.4 (osd.4) 89 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:44:05.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:04.807748+0000 osd.4 (osd.4) 90 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:44:05.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:05.099246+0000 mon.a (mon.0) 1294 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi047-82386-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:05.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:05.099374+0000 mon.a (mon.0) 1295 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi047-82457-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-23T00:44:05.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:05.099497+0000 mon.a (mon.0) 1296 : cluster [DBG] osdmap e173: 8 total, 8 up, 8 in 2024-02-23T00:44:05.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:05.117617+0000 mon.a (mon.0) 1297 : audit [INF] from='client.? 172.21.15.47:0/346485915' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82496-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:05.117881+0000 mon.b (mon.1) 210 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi047-82457-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-23T00:44:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:05.124014+0000 mon.a (mon.0) 1298 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi047-82457-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-23T00:44:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:05.257915+0000 mon.c (mon.2) 93 : audit [INF] from='client.? 172.21.15.47:0/984423723' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:44:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:05.591683+0000 mon.a (mon.0) 1299 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:44:05.694 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:05.695 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:44:05.723 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:44:05.796 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:44:05.825 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:05.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:05.854+0000 7fdd84bda700 1 -- 172.21.15.47:0/228618125 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd reweight", "id": 2, "weight": 1.0} v 0) v1 -- 0x7fdd8005fa60 con 0x7fdd800fe510 2024-02-23T00:44:06.055 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:44:06.055 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:44:06.075 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:44:06.078 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:06.078 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:44:06.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.113+0000 7f30eeffd700 1 -- 172.21.15.47:0/984423723 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v174) v1 ==== 97+0+0 (secure 0 0 0) 0x7f30f40191b0 con 0x7f30fc0fd510 2024-02-23T00:44:06.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.118+0000 7f3103ea3700 10 librados: watch_flush enter 2024-02-23T00:44:06.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.118+0000 7fdd777fe700 1 -- 172.21.15.47:0/228618125 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 2, "weight": 1.0}]=0 reweighted osd.2 to 1 (10000) v174) v1 ==== 113+0+0 (secure 0 0 0) 0x7fdd78050300 con 0x7fdd800fe510 2024-02-23T00:44:06.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.123+0000 7f3103ea3700 10 librados: watch_flush exit 2024-02-23T00:44:06.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.123+0000 7f3103ea3700 20 client.40475.objecter shutdown clearing up homeless session... 2024-02-23T00:44:06.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.123+0000 7f3103ea3700 10 client.40475.objecter successfully canceled tick 2024-02-23T00:44:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.123+0000 7f3103ea3700 1 -- 172.21.15.47:0/984423723 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f30e8062cc0 msgr2=0x7f30e8065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.123+0000 7f3103ea3700 1 --2- 172.21.15.47:0/984423723 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f30e8062cc0 0x7f30e8065180 secure :-1 s=READY pgs=2914 cs=0 l=1 rev1=1 rx=0x7f30f0004e20 tx=0x7f30f0009cb0).stop 2024-02-23T00:44:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.123+0000 7f3103ea3700 1 -- 172.21.15.47:0/984423723 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f30fc0fd510 msgr2=0x7f30fc0f9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.123+0000 7f3103ea3700 1 --2- 172.21.15.47:0/984423723 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f30fc0fd510 0x7f30fc0f9390 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 rx=0x7f30f4001df0 tx=0x7f30f4004d10).stop 2024-02-23T00:44:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.126+0000 7f3103ea3700 1 -- 172.21.15.47:0/984423723 shutdown_connections 2024-02-23T00:44:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.126+0000 7f3103ea3700 1 --2- 172.21.15.47:0/984423723 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f30e8062cc0 0x7f30e8065180 unknown :-1 s=CLOSED pgs=2914 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.126+0000 7f3103ea3700 1 --2- 172.21.15.47:0/984423723 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f30fc0fd510 0x7f30fc0f9390 unknown :-1 s=CLOSED pgs=2485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.126+0000 7f3103ea3700 1 --2- 172.21.15.47:0/984423723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30fc0f7400 0x7f30fc0f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.126+0000 7f3103ea3700 1 --2- 172.21.15.47:0/984423723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30fc0fcbb0 0x7f30fc0f8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.126+0000 7f3103ea3700 1 -- 172.21.15.47:0/984423723 >> 172.21.15.47:0/984423723 conn(0x7f30fc0f2820 msgr2=0x7f30fc060e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:06.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.126+0000 7fdd84bda700 10 librados: watch_flush enter 2024-02-23T00:44:06.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.126+0000 7fdd84bda700 10 librados: watch_flush exit 2024-02-23T00:44:06.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.126+0000 7fdd84bda700 20 client.40487.objecter shutdown clearing up homeless session... 2024-02-23T00:44:06.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.126+0000 7fdd84bda700 10 client.40487.objecter successfully canceled tick 2024-02-23T00:44:06.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.139+0000 7fdd84bda700 1 -- 172.21.15.47:0/228618125 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdd6c062cc0 msgr2=0x7fdd6c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:06.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.139+0000 7fdd84bda700 1 --2- 172.21.15.47:0/228618125 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdd6c062cc0 0x7fdd6c065180 secure :-1 s=READY pgs=2916 cs=0 l=1 rev1=1 rx=0x7fdd70004ee0 tx=0x7fdd7000b040).stop 2024-02-23T00:44:06.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.139+0000 7fdd84bda700 1 -- 172.21.15.47:0/228618125 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdd800fe510 msgr2=0x7fdd8006bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:06.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.139+0000 7fdd84bda700 1 --2- 172.21.15.47:0/228618125 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdd800fe510 0x7fdd8006bab0 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 rx=0x7fdd7800ec20 tx=0x7fdd7800b040).stop 2024-02-23T00:44:06.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.141+0000 7f3103ea3700 1 -- 172.21.15.47:0/984423723 shutdown_connections 2024-02-23T00:44:06.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.141+0000 7f3103ea3700 1 -- 172.21.15.47:0/984423723 wait complete. 2024-02-23T00:44:06.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.144+0000 7f3103ea3700 1 librados: shutdown 2024-02-23T00:44:06.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.145+0000 7fdd84bda700 1 -- 172.21.15.47:0/228618125 shutdown_connections 2024-02-23T00:44:06.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.145+0000 7fdd84bda700 1 --2- 172.21.15.47:0/228618125 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdd6c062cc0 0x7fdd6c065180 unknown :-1 s=CLOSED pgs=2916 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:06.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.145+0000 7fdd84bda700 1 --2- 172.21.15.47:0/228618125 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdd800fe510 0x7fdd8006bab0 unknown :-1 s=CLOSED pgs=2488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:06.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.145+0000 7fdd84bda700 1 --2- 172.21.15.47:0/228618125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd800f8420 0x7fdd8006b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:06.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.145+0000 7fdd84bda700 1 --2- 172.21.15.47:0/228618125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd800fdbb0 0x7fdd8006b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:06.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.145+0000 7fdd84bda700 1 -- 172.21.15.47:0/228618125 >> 172.21.15.47:0/228618125 conn(0x7fdd800f3800 msgr2=0x7fdd800fb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:06.148 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is set 2024-02-23T00:44:06.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.148+0000 7fdd84bda700 1 -- 172.21.15.47:0/228618125 shutdown_connections 2024-02-23T00:44:06.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.149+0000 7fdd84bda700 1 -- 172.21.15.47:0/228618125 wait complete. 2024-02-23T00:44:06.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:06.150+0000 7fdd84bda700 1 librados: shutdown 2024-02-23T00:44:06.154 INFO:teuthology.orchestra.run.smithi047.stderr:reweighted osd.2 to 1 (10000) 2024-02-23T00:44:06.180 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:44:06.208 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:06.331 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:44:06.334 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:06.334 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:44:06.435 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:44:06.464 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:06.518 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:44:06.521 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:06.522 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:44:06.623 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:44:06.650 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:06.873 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:05.217830+0000 mgr.y (mgr.14517) 221 : cluster [DBG] pgmap v235: 346 pgs: 282 active+clean, 64 unknown; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:44:06.874 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:05.734495+0000 mon.b (mon.1) 211 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:06.874 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:05.855945+0000 mon.c (mon.2) 94 : audit [INF] from='client.? 172.21.15.47:0/228618125' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-23T00:44:06.874 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:05.856279+0000 mon.a (mon.0) 1300 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-23T00:44:06.874 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:06.097150+0000 mon.a (mon.0) 1301 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:44:06.874 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:06.104855+0000 mon.a (mon.0) 1302 : audit [INF] from='client.? 172.21.15.47:0/346485915' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82496-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:06.874 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:06.104975+0000 mon.a (mon.0) 1303 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi047-82457-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-23T00:44:06.874 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:06.105121+0000 mon.a (mon.0) 1304 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:44:06.874 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:06.105195+0000 mon.a (mon.0) 1305 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-02-23T00:44:06.874 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:06.105256+0000 mon.a (mon.0) 1306 : cluster [DBG] osdmap e174: 8 total, 8 up, 8 in 2024-02-23T00:44:06.874 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:06.128031+0000 mon.b (mon.1) 212 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi047-82457-1","app":"app1","key":"key1"}]: dispatch 2024-02-23T00:44:06.874 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:06.131050+0000 mon.a (mon.0) 1307 : audit [INF] from='client.? 172.21.15.47:0/283101316' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi047-82378-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:06.874 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:06.131349+0000 mon.a (mon.0) 1308 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi047-82457-1","app":"app1","key":"key1"}]: dispatch 2024-02-23T00:44:06.883 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:05.217830+0000 mgr.y (mgr.14517) 221 : cluster [DBG] pgmap v235: 346 pgs: 282 active+clean, 64 unknown; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:44:06.883 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:05.734495+0000 mon.b (mon.1) 211 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:06.883 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:05.855945+0000 mon.c (mon.2) 94 : audit [INF] from='client.? 172.21.15.47:0/228618125' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-23T00:44:06.883 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:05.856279+0000 mon.a (mon.0) 1300 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-23T00:44:06.883 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:06.097150+0000 mon.a (mon.0) 1301 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:44:06.883 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:06.104855+0000 mon.a (mon.0) 1302 : audit [INF] from='client.? 172.21.15.47:0/346485915' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82496-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:06.883 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:06.104975+0000 mon.a (mon.0) 1303 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi047-82457-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-23T00:44:06.883 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:06.105121+0000 mon.a (mon.0) 1304 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:44:06.883 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:06.105195+0000 mon.a (mon.0) 1305 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-02-23T00:44:06.883 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:06.105256+0000 mon.a (mon.0) 1306 : cluster [DBG] osdmap e174: 8 total, 8 up, 8 in 2024-02-23T00:44:06.883 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:06.128031+0000 mon.b (mon.1) 212 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi047-82457-1","app":"app1","key":"key1"}]: dispatch 2024-02-23T00:44:06.884 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:06.131050+0000 mon.a (mon.0) 1307 : audit [INF] from='client.? 172.21.15.47:0/283101316' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi047-82378-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:06.884 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:06.131349+0000 mon.a (mon.0) 1308 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi047-82457-1","app":"app1","key":"key1"}]: dispatch 2024-02-23T00:44:06.884 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:05.217830+0000 mgr.y (mgr.14517) 221 : cluster [DBG] pgmap v235: 346 pgs: 282 active+clean, 64 unknown; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:44:06.884 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:05.734495+0000 mon.b (mon.1) 211 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:06.884 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:05.855945+0000 mon.c (mon.2) 94 : audit [INF] from='client.? 172.21.15.47:0/228618125' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-23T00:44:06.884 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:05.856279+0000 mon.a (mon.0) 1300 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-23T00:44:06.884 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:06.097150+0000 mon.a (mon.0) 1301 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:44:06.884 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:06.104855+0000 mon.a (mon.0) 1302 : audit [INF] from='client.? 172.21.15.47:0/346485915' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82496-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:06.884 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:06.104975+0000 mon.a (mon.0) 1303 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi047-82457-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-23T00:44:06.884 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:06.105121+0000 mon.a (mon.0) 1304 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:44:06.884 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:06.105195+0000 mon.a (mon.0) 1305 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-02-23T00:44:06.885 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:06.105256+0000 mon.a (mon.0) 1306 : cluster [DBG] osdmap e174: 8 total, 8 up, 8 in 2024-02-23T00:44:06.885 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:06.128031+0000 mon.b (mon.1) 212 : audit [INF] from='client.? 172.21.15.47:0/1191937413' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi047-82457-1","app":"app1","key":"key1"}]: dispatch 2024-02-23T00:44:06.885 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:06.131050+0000 mon.a (mon.0) 1307 : audit [INF] from='client.? 172.21.15.47:0/283101316' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi047-82378-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:06.885 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:06.131349+0000 mon.a (mon.0) 1308 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi047-82457-1","app":"app1","key":"key1"}]: dispatch 2024-02-23T00:44:06.891 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.5 dump_blocked_ops 2024-02-23T00:44:06.899 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:44:06.901 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:06.902 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:44:07.002 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:44:07.031 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:07.179 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:44:07.383 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:44:07.385 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:07.386 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:44:07.401 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight 3 1 2024-02-23T00:44:07.487 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:44:07.515 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:07.797 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:05.874452+0000 osd.7 (osd.7) 105 : cluster [DBG] 93.7 scrub starts 2024-02-23T00:44:07.797 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:05.876161+0000 osd.7 (osd.7) 106 : cluster [DBG] 93.7 scrub ok 2024-02-23T00:44:07.797 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:06.735608+0000 mon.b (mon.1) 213 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:07.797 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:07.106899+0000 mon.a (mon.0) 1309 : audit [INF] from='client.? 172.21.15.47:0/283101316' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi047-82378-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:07.797 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:07.106985+0000 mon.a (mon.0) 1310 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi047-82457-1","app":"app1","key":"key1"}]': finished 2024-02-23T00:44:07.797 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:07.107129+0000 mon.a (mon.0) 1311 : cluster [DBG] osdmap e175: 8 total, 8 up, 8 in 2024-02-23T00:44:07.797 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:07.119203+0000 mon.b (mon.1) 214 : audit [INF] from='client.? 172.21.15.47:0/409438768' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi047-82386-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:07.797 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:07.120441+0000 mon.a (mon.0) 1312 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi047-82386-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:07.798 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:05.874452+0000 osd.7 (osd.7) 105 : cluster [DBG] 93.7 scrub starts 2024-02-23T00:44:07.798 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:05.876161+0000 osd.7 (osd.7) 106 : cluster [DBG] 93.7 scrub ok 2024-02-23T00:44:07.798 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:06.735608+0000 mon.b (mon.1) 213 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:07.798 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:07.106899+0000 mon.a (mon.0) 1309 : audit [INF] from='client.? 172.21.15.47:0/283101316' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi047-82378-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:07.798 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:07.106985+0000 mon.a (mon.0) 1310 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi047-82457-1","app":"app1","key":"key1"}]': finished 2024-02-23T00:44:07.798 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:07.107129+0000 mon.a (mon.0) 1311 : cluster [DBG] osdmap e175: 8 total, 8 up, 8 in 2024-02-23T00:44:07.798 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:07.119203+0000 mon.b (mon.1) 214 : audit [INF] from='client.? 172.21.15.47:0/409438768' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi047-82386-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:07.798 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:07.120441+0000 mon.a (mon.0) 1312 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi047-82386-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:07.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:05.874452+0000 osd.7 (osd.7) 105 : cluster [DBG] 93.7 scrub starts 2024-02-23T00:44:07.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:05.876161+0000 osd.7 (osd.7) 106 : cluster [DBG] 93.7 scrub ok 2024-02-23T00:44:07.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:06.735608+0000 mon.b (mon.1) 213 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:07.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:07.106899+0000 mon.a (mon.0) 1309 : audit [INF] from='client.? 172.21.15.47:0/283101316' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi047-82378-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:07.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:07.106985+0000 mon.a (mon.0) 1310 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi047-82457-1","app":"app1","key":"key1"}]': finished 2024-02-23T00:44:07.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:07.107129+0000 mon.a (mon.0) 1311 : cluster [DBG] osdmap e175: 8 total, 8 up, 8 in 2024-02-23T00:44:07.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:07.119203+0000 mon.b (mon.1) 214 : audit [INF] from='client.? 172.21.15.47:0/409438768' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi047-82386-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:07.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:07.120441+0000 mon.a (mon.0) 1312 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi047-82386-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:08.169 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [==========] Running 31 tests from 7 test suites. 2024-02-23T00:44:08.169 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [----------] Global test environment set-up. 2024-02-23T00:44:08.169 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2024-02-23T00:44:08.169 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2024-02-23T00:44:08.169 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-02-23T00:44:08.169 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-02-23T00:44:08.169 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: 2024-02-23T00:44:08.169 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2024-02-23T00:44:08.169 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: seed 82457 2024-02-23T00:44:08.169 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-02-23T00:44:08.170 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (0 ms) 2024-02-23T00:44:08.170 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2024-02-23T00:44:08.170 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (18 ms) 2024-02-23T00:44:08.170 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2024-02-23T00:44:08.170 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (12 ms) 2024-02-23T00:44:08.170 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2024-02-23T00:44:08.170 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (19 ms) 2024-02-23T00:44:08.170 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-02-23T00:44:08.170 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (23 ms) 2024-02-23T00:44:08.170 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2024-02-23T00:44:08.170 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (3 ms) 2024-02-23T00:44:08.170 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2024-02-23T00:44:08.170 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (2 ms) 2024-02-23T00:44:08.170 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2024-02-23T00:44:08.170 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (8 ms) 2024-02-23T00:44:08.170 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2024-02-23T00:44:08.171 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (3 ms) 2024-02-23T00:44:08.171 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2024-02-23T00:44:08.171 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (19 ms) 2024-02-23T00:44:08.171 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2024-02-23T00:44:08.171 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (4 ms) 2024-02-23T00:44:08.171 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2024-02-23T00:44:08.171 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (7 ms) 2024-02-23T00:44:08.171 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2024-02-23T00:44:08.171 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (6 ms) 2024-02-23T00:44:08.171 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2024-02-23T00:44:08.171 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: osd_max_attr_size = 0 2024-02-23T00:44:08.171 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2024-02-23T00:44:08.171 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (2038 ms) 2024-02-23T00:44:08.171 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2024-02-23T00:44:08.171 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (364 ms) 2024-02-23T00:44:08.171 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2024-02-23T00:44:08.172 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: waiting for initial deep scrubs... 2024-02-23T00:44:08.172 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: done waiting, doing copies 2024-02-23T00:44:08.172 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: waiting for final deep scrubs... 2024-02-23T00:44:08.172 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: done waiting 2024-02-23T00:44:08.172 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (61131 ms) 2024-02-23T00:44:08.172 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2024-02-23T00:44:08.172 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (4 ms) 2024-02-23T00:44:08.172 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2024-02-23T00:44:08.172 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (5 ms) 2024-02-23T00:44:08.172 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2024-02-23T00:44:08.172 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (4332 ms) 2024-02-23T00:44:08.172 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2024-02-23T00:44:08.172 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-02-23T00:44:08.173 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2024-02-23T00:44:08.173 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-02-23T00:44:08.173 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2024-02-23T00:44:08.173 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2024-02-23T00:44:08.746 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:07.218867+0000 mgr.y (mgr.14517) 222 : cluster [DBG] pgmap v238: 378 pgs: 346 active+clean, 32 creating+peering; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:44:08.746 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:07.737096+0000 mon.b (mon.1) 215 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:08.746 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:08.110995+0000 mon.a (mon.0) 1313 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi047-82386-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:08.746 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:08.111153+0000 mon.a (mon.0) 1314 : cluster [DBG] osdmap e176: 8 total, 8 up, 8 in 2024-02-23T00:44:08.746 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:08.179653+0000 mon.c (mon.2) 95 : audit [INF] from='client.? 172.21.15.47:0/3108721855' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82457-24"}]: dispatch 2024-02-23T00:44:08.746 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:08.606575+0000 mon.a (mon.0) 1315 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82457-24"}]: dispatch 2024-02-23T00:44:08.746 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:08.607152+0000 mon.c (mon.2) 96 : audit [INF] from='client.? 172.21.15.47:0/3108721855' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82457-24"}]: dispatch 2024-02-23T00:44:08.747 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:08.607457+0000 mon.a (mon.0) 1316 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82457-24"}]: dispatch 2024-02-23T00:44:08.747 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:08.607876+0000 mon.c (mon.2) 97 : audit [INF] from='client.? 172.21.15.47:0/3108721855' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi047-82457-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:08.747 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:08.608202+0000 mon.a (mon.0) 1317 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi047-82457-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:08.747 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:07.218867+0000 mgr.y (mgr.14517) 222 : cluster [DBG] pgmap v238: 378 pgs: 346 active+clean, 32 creating+peering; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:44:08.747 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:07.737096+0000 mon.b (mon.1) 215 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:08.747 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:08.110995+0000 mon.a (mon.0) 1313 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi047-82386-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:08.747 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:08.111153+0000 mon.a (mon.0) 1314 : cluster [DBG] osdmap e176: 8 total, 8 up, 8 in 2024-02-23T00:44:08.747 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:08.179653+0000 mon.c (mon.2) 95 : audit [INF] from='client.? 172.21.15.47:0/3108721855' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82457-24"}]: dispatch 2024-02-23T00:44:08.747 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:08.606575+0000 mon.a (mon.0) 1315 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82457-24"}]: dispatch 2024-02-23T00:44:08.747 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:08.607152+0000 mon.c (mon.2) 96 : audit [INF] from='client.? 172.21.15.47:0/3108721855' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82457-24"}]: dispatch 2024-02-23T00:44:08.747 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:08.607457+0000 mon.a (mon.0) 1316 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82457-24"}]: dispatch 2024-02-23T00:44:08.748 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:08.607876+0000 mon.c (mon.2) 97 : audit [INF] from='client.? 172.21.15.47:0/3108721855' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi047-82457-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:08.748 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:08.608202+0000 mon.a (mon.0) 1317 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi047-82457-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:08.768 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:44:08.772 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:08.773 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:44:08.784 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:44:08.874 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:44:08.903 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:08.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:07.218867+0000 mgr.y (mgr.14517) 222 : cluster [DBG] pgmap v238: 378 pgs: 346 active+clean, 32 creating+peering; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:44:08.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:07.737096+0000 mon.b (mon.1) 215 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:08.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:08.110995+0000 mon.a (mon.0) 1313 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi047-82386-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:08.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:08.111153+0000 mon.a (mon.0) 1314 : cluster [DBG] osdmap e176: 8 total, 8 up, 8 in 2024-02-23T00:44:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:08.179653+0000 mon.c (mon.2) 95 : audit [INF] from='client.? 172.21.15.47:0/3108721855' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82457-24"}]: dispatch 2024-02-23T00:44:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:08.606575+0000 mon.a (mon.0) 1315 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82457-24"}]: dispatch 2024-02-23T00:44:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:08.607152+0000 mon.c (mon.2) 96 : audit [INF] from='client.? 172.21.15.47:0/3108721855' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82457-24"}]: dispatch 2024-02-23T00:44:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:08.607457+0000 mon.a (mon.0) 1316 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82457-24"}]: dispatch 2024-02-23T00:44:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:08.607876+0000 mon.c (mon.2) 97 : audit [INF] from='client.? 172.21.15.47:0/3108721855' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi047-82457-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:08.608202+0000 mon.a (mon.0) 1317 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi047-82457-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:09.027 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:44:09.030 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:09.030 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:44:09.131 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:44:09.167 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:09.416 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:44:09.419 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:09.419 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:44:09.520 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:44:09.552 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:09.575 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:44:09.578 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:09.579 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:44:09.680 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:44:09.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:08.507112+0000 osd.5 (osd.5) 89 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:44:09.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:08.507968+0000 osd.5 (osd.5) 90 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:44:09.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:08.738333+0000 mon.b (mon.1) 216 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:09.113938+0000 mon.a (mon.0) 1318 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi047-82457-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:44:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:09.114022+0000 mon.a (mon.0) 1319 : cluster [DBG] osdmap e177: 8 total, 8 up, 8 in 2024-02-23T00:44:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:09.124430+0000 mon.b (mon.1) 217 : audit [INF] from='client.? 172.21.15.47:0/1576989539' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi047-82378-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:09.127002+0000 mon.c (mon.2) 98 : audit [INF] from='client.? 172.21.15.47:0/3108721855' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi047-82457-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi047-82457-24"}]: dispatch 2024-02-23T00:44:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:09.128696+0000 mon.a (mon.0) 1320 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi047-82457-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi047-82457-24"}]: dispatch 2024-02-23T00:44:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:09.129374+0000 mon.a (mon.0) 1321 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi047-82378-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:09.178063+0000 mon.c (mon.2) 99 : audit [INF] from='client.? 172.21.15.47:0/3414618047' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi047-82496-15"}]: dispatch 2024-02-23T00:44:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:09.601350+0000 mon.a (mon.0) 1322 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi047-82496-15"}]: dispatch 2024-02-23T00:44:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:09.601922+0000 mon.c (mon.2) 100 : audit [INF] from='client.? 172.21.15.47:0/3414618047' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi047-82496-15"}]: dispatch 2024-02-23T00:44:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:09.602443+0000 mon.a (mon.0) 1323 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi047-82496-15"}]: dispatch 2024-02-23T00:44:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:09.602840+0000 mon.c (mon.2) 101 : audit [INF] from='client.? 172.21.15.47:0/3414618047' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi047-82496-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:09.603084+0000 mon.a (mon.0) 1324 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi047-82496-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:09.708 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:09.728 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:44:09.731 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:09.731 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:44:09.754 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:44:09.754 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:44:09.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:08.507112+0000 osd.5 (osd.5) 89 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:44:09.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:08.507968+0000 osd.5 (osd.5) 90 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:44:09.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:08.738333+0000 mon.b (mon.1) 216 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:09.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:09.113938+0000 mon.a (mon.0) 1318 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi047-82457-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:44:09.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:09.114022+0000 mon.a (mon.0) 1319 : cluster [DBG] osdmap e177: 8 total, 8 up, 8 in 2024-02-23T00:44:09.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:09.124430+0000 mon.b (mon.1) 217 : audit [INF] from='client.? 172.21.15.47:0/1576989539' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi047-82378-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:09.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:09.127002+0000 mon.c (mon.2) 98 : audit [INF] from='client.? 172.21.15.47:0/3108721855' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi047-82457-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi047-82457-24"}]: dispatch 2024-02-23T00:44:09.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:09.128696+0000 mon.a (mon.0) 1320 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi047-82457-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi047-82457-24"}]: dispatch 2024-02-23T00:44:09.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:09.129374+0000 mon.a (mon.0) 1321 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi047-82378-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:09.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:09.178063+0000 mon.c (mon.2) 99 : audit [INF] from='client.? 172.21.15.47:0/3414618047' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi047-82496-15"}]: dispatch 2024-02-23T00:44:09.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:09.601350+0000 mon.a (mon.0) 1322 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi047-82496-15"}]: dispatch 2024-02-23T00:44:09.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:09.601922+0000 mon.c (mon.2) 100 : audit [INF] from='client.? 172.21.15.47:0/3414618047' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi047-82496-15"}]: dispatch 2024-02-23T00:44:09.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:09.602443+0000 mon.a (mon.0) 1323 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi047-82496-15"}]: dispatch 2024-02-23T00:44:09.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:09.602840+0000 mon.c (mon.2) 101 : audit [INF] from='client.? 172.21.15.47:0/3414618047' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi047-82496-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:09.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:09.603084+0000 mon.a (mon.0) 1324 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi047-82496-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:09.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:08.507112+0000 osd.5 (osd.5) 89 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:44:09.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:08.507968+0000 osd.5 (osd.5) 90 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:44:09.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:08.738333+0000 mon.b (mon.1) 216 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:09.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:09.113938+0000 mon.a (mon.0) 1318 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi047-82457-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:44:09.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:09.114022+0000 mon.a (mon.0) 1319 : cluster [DBG] osdmap e177: 8 total, 8 up, 8 in 2024-02-23T00:44:09.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:09.124430+0000 mon.b (mon.1) 217 : audit [INF] from='client.? 172.21.15.47:0/1576989539' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi047-82378-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:09.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:09.127002+0000 mon.c (mon.2) 98 : audit [INF] from='client.? 172.21.15.47:0/3108721855' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi047-82457-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi047-82457-24"}]: dispatch 2024-02-23T00:44:09.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:09.128696+0000 mon.a (mon.0) 1320 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi047-82457-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi047-82457-24"}]: dispatch 2024-02-23T00:44:09.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:09.129374+0000 mon.a (mon.0) 1321 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi047-82378-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:09.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:09.178063+0000 mon.c (mon.2) 99 : audit [INF] from='client.? 172.21.15.47:0/3414618047' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi047-82496-15"}]: dispatch 2024-02-23T00:44:09.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:09.601350+0000 mon.a (mon.0) 1322 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi047-82496-15"}]: dispatch 2024-02-23T00:44:09.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:09.601922+0000 mon.c (mon.2) 100 : audit [INF] from='client.? 172.21.15.47:0/3414618047' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi047-82496-15"}]: dispatch 2024-02-23T00:44:09.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:09.602443+0000 mon.a (mon.0) 1323 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi047-82496-15"}]: dispatch 2024-02-23T00:44:09.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:09.602840+0000 mon.c (mon.2) 101 : audit [INF] from='client.? 172.21.15.47:0/3414618047' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi047-82496-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:09.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:09.603084+0000 mon.a (mon.0) 1324 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi047-82496-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:09.833 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:44:09.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.837+0000 7fa47da17700 1 -- 172.21.15.47:0/759390066 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4780fcab0 msgr2=0x7fa4780fce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:09.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.837+0000 7fa47da17700 1 --2- 172.21.15.47:0/759390066 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4780fcab0 0x7fa4780fce90 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 rx=0x7fa468004870 tx=0x7fa468009cb0).stop 2024-02-23T00:44:09.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.837+0000 7fa47da17700 1 -- 172.21.15.47:0/759390066 shutdown_connections 2024-02-23T00:44:09.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.837+0000 7fa47da17700 1 --2- 172.21.15.47:0/759390066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4780f7320 0x7fa478103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:09.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.837+0000 7fa47da17700 1 --2- 172.21.15.47:0/759390066 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4780fcab0 0x7fa4780fce90 unknown :-1 s=CLOSED pgs=2441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:09.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.837+0000 7fa47da17700 1 --2- 172.21.15.47:0/759390066 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4780fd410 0x7fa4780f6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:09.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.837+0000 7fa47da17700 1 -- 172.21.15.47:0/759390066 >> 172.21.15.47:0/759390066 conn(0x7fa4780f26e0 msgr2=0x7fa4780f4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:09.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.838+0000 7fa47da17700 1 -- 172.21.15.47:0/759390066 shutdown_connections 2024-02-23T00:44:09.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.838+0000 7fa47da17700 1 -- 172.21.15.47:0/759390066 wait complete. 2024-02-23T00:44:09.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.838+0000 7fa47da17700 1 librados: starting msgr at 2024-02-23T00:44:09.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.838+0000 7fa47da17700 1 librados: starting objecter 2024-02-23T00:44:09.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.838+0000 7fa47da17700 1 Processor -- start 2024-02-23T00:44:09.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.838+0000 7fa47da17700 1 -- start start 2024-02-23T00:44:09.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.838+0000 7fa47da17700 1 librados: setting wanted keys 2024-02-23T00:44:09.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.838+0000 7fa47da17700 1 librados: calling monclient init 2024-02-23T00:44:09.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.839+0000 7fa47da17700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4780f7320 0x7fa4780f88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:09.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.839+0000 7fa47da17700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4780fcab0 0x7fa4780f8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:09.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.839+0000 7fa47da17700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4780fd410 0x7fa4780f9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:09.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.839+0000 7fa47da17700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa478116390 con 0x7fa4780f7320 2024-02-23T00:44:09.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.839+0000 7fa47da17700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa478116500 con 0x7fa4780fd410 2024-02-23T00:44:09.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.839+0000 7fa47da17700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4781166c0 con 0x7fa4780fcab0 2024-02-23T00:44:09.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.839+0000 7fa4767fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4780fcab0 0x7fa4780f8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:09.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.839+0000 7fa476ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4780f7320 0x7fa4780f88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:09.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.839+0000 7fa4767fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4780fcab0 0x7fa4780f8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:50188/0 (socket says 172.21.15.47:50188) 2024-02-23T00:44:09.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.839+0000 7fa4767fc700 1 -- 172.21.15.47:0/3631802115 learned_addr learned my addr 172.21.15.47:0/3631802115 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:44:09.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.839+0000 7fa4777fe700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4780fd410 0x7fa4780f9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:09.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.839+0000 7fa4767fc700 1 -- 172.21.15.47:0/3631802115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4780fd410 msgr2=0x7fa4780f9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:09.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.839+0000 7fa4767fc700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4780fd410 0x7fa4780f9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:09.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.839+0000 7fa4767fc700 1 -- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4780f7320 msgr2=0x7fa4780f88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:09.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.839+0000 7fa4767fc700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4780f7320 0x7fa4780f88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:09.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.839+0000 7fa4767fc700 1 -- 172.21.15.47:0/3631802115 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa46c005040 con 0x7fa4780fcab0 2024-02-23T00:44:09.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.839+0000 7fa4777fe700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4780fd410 0x7fa4780f9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:44:09.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.839+0000 7fa476ffd700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4780f7320 0x7fa4780f88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:44:09.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.839+0000 7fa4767fc700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4780fcab0 0x7fa4780f8da0 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 rx=0x7fa46c0048f0 tx=0x7fa46c009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:09.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.839+0000 7fa45ffff700 10 client.?.objecter ms_handle_connect 0x7fa4780fcab0 2024-02-23T00:44:09.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.839+0000 7fa45ffff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:44:09.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.839+0000 7fa45ffff700 1 -- 172.21.15.47:0/3631802115 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c007b00 con 0x7fa4780fcab0 2024-02-23T00:44:09.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.840+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa468005040 con 0x7fa4780fcab0 2024-02-23T00:44:09.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.840+0000 7fa45ffff700 1 -- 172.21.15.47:0/3631802115 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fa46c00f040 con 0x7fa4780fcab0 2024-02-23T00:44:09.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.840+0000 7fa45ffff700 1 -- 172.21.15.47:0/3631802115 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c014890 con 0x7fa4780fcab0 2024-02-23T00:44:09.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.840+0000 7fa47da17700 10 client.40535.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:44:09.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.840+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa478116d80 con 0x7fa4780fcab0 2024-02-23T00:44:09.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.840+0000 7fa47da17700 1 librados: init done 2024-02-23T00:44:09.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.840+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fa47805f9d0 con 0x7fa4780fcab0 2024-02-23T00:44:09.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.841+0000 7fa45ffff700 1 -- 172.21.15.47:0/3631802115 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fa46c014cc0 con 0x7fa4780fcab0 2024-02-23T00:44:09.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.842+0000 7fa45ffff700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa460062cc0 0x7fa460065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:09.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.842+0000 7fa45ffff700 1 -- 172.21.15.47:0/3631802115 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(177..177 src has 48..177) v4 ==== 9952+0+0 (secure 0 0 0) 0x7fa46c07d5b0 con 0x7fa4780fcab0 2024-02-23T00:44:09.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.842+0000 7fa45ffff700 10 client.40535.objecter ms_dispatch 0x7fa478001550 osd_map(177..177 src has 48..177) v4 2024-02-23T00:44:09.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.842+0000 7fa476ffd700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa460062cc0 0x7fa460065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:09.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.842+0000 7fa45ffff700 3 client.40535.objecter handle_osd_map got epochs [177,177] > 0 2024-02-23T00:44:09.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.842+0000 7fa45ffff700 3 client.40535.objecter handle_osd_map decoding full epoch 177 2024-02-23T00:44:09.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.842+0000 7fa45ffff700 20 client.40535.objecter dump_active .. 0 homeless 2024-02-23T00:44:09.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.842+0000 7fa45ffff700 1 -- 172.21.15.47:0/3631802115 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v177) v1 ==== 54+0+16 (secure 0 0 0) 0x7fa46c084070 con 0x7fa4780fcab0 2024-02-23T00:44:09.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.842+0000 7fa45ffff700 10 client.40535.objecter ms_handle_connect 0x7fa460062cc0 2024-02-23T00:44:09.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.842+0000 7fa476ffd700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa460062cc0 0x7fa460065180 secure :-1 s=READY pgs=2923 cs=0 l=1 rev1=1 rx=0x7fa468004ee0 tx=0x7fa46800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:09.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.842+0000 7fa47da17700 10 client.40535.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:09.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.842+0000 7fa47da17700 15 client.40535.objecter _session_command_op_assign -1 1 2024-02-23T00:44:09.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.842+0000 7fa47da17700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fa4780ffd70 0x7fa478100150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:09.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.842+0000 7fa47da17700 20 client.40535.objecter _get_session s=0x7fa478105d20 osd=0 3 2024-02-23T00:44:09.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.842+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa478105d20 osd=0 3 2024-02-23T00:44:09.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.842+0000 7fa47da17700 20 client.40535.objecter _get_session s=0x7fa478105d20 osd=0 3 2024-02-23T00:44:09.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.842+0000 7fa47da17700 15 client.40535.objecter _session_command_op_remove -1 1 2024-02-23T00:44:09.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.842+0000 7fa47da17700 20 client.40535.objecter get_session s=0x7fa478105d20 osd=0 3 2024-02-23T00:44:09.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.842+0000 7fa47da17700 15 client.40535.objecter _session_command_op_assign 0 1 2024-02-23T00:44:09.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.842+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa478105d20 osd=0 4 2024-02-23T00:44:09.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.842+0000 7fa47da17700 10 client.40535.objecter _send_command 1 2024-02-23T00:44:09.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.842+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa47805ee80 con 0x7fa4780ffd70 2024-02-23T00:44:09.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.842+0000 7fa4777fe700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fa4780ffd70 0x7fa478100150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:09.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.843+0000 7fa4777fe700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fa4780ffd70 0x7fa478100150 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:09.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.843+0000 7fa45ffff700 10 client.40535.objecter ms_handle_connect 0x7fa4780ffd70 2024-02-23T00:44:09.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.843+0000 7fa45ffff700 1 -- 172.21.15.47:0/3631802115 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa47805ee80 con 0x7fa4780ffd70 2024-02-23T00:44:09.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.843+0000 7fa45ffff700 10 client.40535.objecter ms_dispatch 0x7fa478001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:44:09.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.843+0000 7fa45ffff700 10 client.40535.objecter _finish_command 1 = system:0 2024-02-23T00:44:09.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.843+0000 7fa45ffff700 20 client.40535.objecter put_session s=0x7fa478105d20 osd=0 4 2024-02-23T00:44:09.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.843+0000 7fa45ffff700 15 client.40535.objecter _session_command_op_remove 0 1 2024-02-23T00:44:09.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.859+0000 7fa47da17700 10 client.40535.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:44:09.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.859+0000 7fa47da17700 15 client.40535.objecter _session_command_op_assign -1 2 2024-02-23T00:44:09.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.859+0000 7fa47da17700 20 client.40535.objecter _get_session s=0x7fa478105d20 osd=0 3 2024-02-23T00:44:09.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.859+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa478105d20 osd=0 3 2024-02-23T00:44:09.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.859+0000 7fa47da17700 20 client.40535.objecter _get_session s=0x7fa478105d20 osd=0 3 2024-02-23T00:44:09.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.859+0000 7fa47da17700 15 client.40535.objecter _session_command_op_remove -1 2 2024-02-23T00:44:09.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.859+0000 7fa47da17700 20 client.40535.objecter get_session s=0x7fa478105d20 osd=0 3 2024-02-23T00:44:09.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.859+0000 7fa47da17700 15 client.40535.objecter _session_command_op_assign 0 2 2024-02-23T00:44:09.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.859+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa478105d20 osd=0 4 2024-02-23T00:44:09.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.859+0000 7fa47da17700 10 client.40535.objecter _send_command 2 2024-02-23T00:44:09.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.859+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa4780273c0 con 0x7fa4780ffd70 2024-02-23T00:44:09.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.860+0000 7fa45ffff700 1 -- 172.21.15.47:0/3631802115 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa4780273c0 con 0x7fa4780ffd70 2024-02-23T00:44:09.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.860+0000 7fa45ffff700 10 client.40535.objecter ms_dispatch 0x7fa478001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:44:09.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.860+0000 7fa45ffff700 10 client.40535.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:44:09.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.860+0000 7fa45ffff700 20 client.40535.objecter put_session s=0x7fa478105d20 osd=0 4 2024-02-23T00:44:09.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.860+0000 7fa45ffff700 15 client.40535.objecter _session_command_op_remove 0 2 2024-02-23T00:44:09.863 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:44:09.863 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:09.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.861+0000 7fa47da17700 10 client.40535.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:09.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.861+0000 7fa47da17700 15 client.40535.objecter _session_command_op_assign -1 3 2024-02-23T00:44:09.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.861+0000 7fa47da17700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fa4780598f0 0x7fa478059d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:09.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.861+0000 7fa47da17700 20 client.40535.objecter _get_session s=0x7fa478028710 osd=1 3 2024-02-23T00:44:09.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.861+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa478028710 osd=1 3 2024-02-23T00:44:09.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.861+0000 7fa47da17700 20 client.40535.objecter _get_session s=0x7fa478028710 osd=1 3 2024-02-23T00:44:09.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.861+0000 7fa47da17700 15 client.40535.objecter _session_command_op_remove -1 3 2024-02-23T00:44:09.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.861+0000 7fa47da17700 20 client.40535.objecter get_session s=0x7fa478028710 osd=1 3 2024-02-23T00:44:09.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.861+0000 7fa47da17700 15 client.40535.objecter _session_command_op_assign 1 3 2024-02-23T00:44:09.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.861+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa478028710 osd=1 4 2024-02-23T00:44:09.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.861+0000 7fa47da17700 10 client.40535.objecter _send_command 3 2024-02-23T00:44:09.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.861+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa47811d720 con 0x7fa4780598f0 2024-02-23T00:44:09.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.861+0000 7fa476ffd700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fa4780598f0 0x7fa478059d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:09.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.861+0000 7fa45ffff700 10 client.40535.objecter ms_handle_connect 0x7fa4780598f0 2024-02-23T00:44:09.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.861+0000 7fa476ffd700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fa4780598f0 0x7fa478059d10 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:09.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.862+0000 7fa45ffff700 1 -- 172.21.15.47:0/3631802115 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa47811d720 con 0x7fa4780598f0 2024-02-23T00:44:09.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.862+0000 7fa45ffff700 10 client.40535.objecter ms_dispatch 0x7fa478001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:44:09.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.862+0000 7fa45ffff700 10 client.40535.objecter _finish_command 3 = system:0 2024-02-23T00:44:09.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.862+0000 7fa45ffff700 20 client.40535.objecter put_session s=0x7fa478028710 osd=1 4 2024-02-23T00:44:09.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.862+0000 7fa45ffff700 15 client.40535.objecter _session_command_op_remove 1 3 2024-02-23T00:44:09.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.878+0000 7fa47da17700 10 client.40535.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:44:09.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.878+0000 7fa47da17700 15 client.40535.objecter _session_command_op_assign -1 4 2024-02-23T00:44:09.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.878+0000 7fa47da17700 20 client.40535.objecter _get_session s=0x7fa478028710 osd=1 3 2024-02-23T00:44:09.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.878+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa478028710 osd=1 3 2024-02-23T00:44:09.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.878+0000 7fa47da17700 20 client.40535.objecter _get_session s=0x7fa478028710 osd=1 3 2024-02-23T00:44:09.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.878+0000 7fa47da17700 15 client.40535.objecter _session_command_op_remove -1 4 2024-02-23T00:44:09.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.878+0000 7fa47da17700 20 client.40535.objecter get_session s=0x7fa478028710 osd=1 3 2024-02-23T00:44:09.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.878+0000 7fa47da17700 15 client.40535.objecter _session_command_op_assign 1 4 2024-02-23T00:44:09.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.878+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa478028710 osd=1 4 2024-02-23T00:44:09.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.878+0000 7fa47da17700 10 client.40535.objecter _send_command 4 2024-02-23T00:44:09.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.878+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa478059360 con 0x7fa4780598f0 2024-02-23T00:44:09.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.879+0000 7fa45ffff700 1 -- 172.21.15.47:0/3631802115 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa478059360 con 0x7fa4780598f0 2024-02-23T00:44:09.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.879+0000 7fa45ffff700 10 client.40535.objecter ms_dispatch 0x7fa478001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:44:09.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.879+0000 7fa45ffff700 10 client.40535.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:44:09.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.879+0000 7fa45ffff700 20 client.40535.objecter put_session s=0x7fa478028710 osd=1 4 2024-02-23T00:44:09.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.879+0000 7fa45ffff700 15 client.40535.objecter _session_command_op_remove 1 4 2024-02-23T00:44:09.882 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:44:09.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.880+0000 7fa47da17700 10 client.40535.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:09.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.880+0000 7fa47da17700 15 client.40535.objecter _session_command_op_assign -1 5 2024-02-23T00:44:09.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.880+0000 7fa47da17700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa478028270 0x7fa47811c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:09.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.880+0000 7fa47da17700 20 client.40535.objecter _get_session s=0x7fa478059770 osd=2 3 2024-02-23T00:44:09.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.880+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa478059770 osd=2 3 2024-02-23T00:44:09.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.880+0000 7fa47da17700 20 client.40535.objecter _get_session s=0x7fa478059770 osd=2 3 2024-02-23T00:44:09.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.880+0000 7fa47da17700 15 client.40535.objecter _session_command_op_remove -1 5 2024-02-23T00:44:09.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.880+0000 7fa47da17700 20 client.40535.objecter get_session s=0x7fa478059770 osd=2 3 2024-02-23T00:44:09.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.880+0000 7fa47da17700 15 client.40535.objecter _session_command_op_assign 2 5 2024-02-23T00:44:09.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.880+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa478059770 osd=2 4 2024-02-23T00:44:09.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.880+0000 7fa47da17700 10 client.40535.objecter _send_command 5 2024-02-23T00:44:09.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.880+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa47811c6c0 con 0x7fa478028270 2024-02-23T00:44:09.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.880+0000 7fa4767fc700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa478028270 0x7fa47811c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:09.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.880+0000 7fa45ffff700 10 client.40535.objecter ms_handle_connect 0x7fa478028270 2024-02-23T00:44:09.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.880+0000 7fa4767fc700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa478028270 0x7fa47811c140 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:09.884 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:44:09.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.881+0000 7fa45ffff700 1 -- 172.21.15.47:0/3631802115 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa47811c6c0 con 0x7fa478028270 2024-02-23T00:44:09.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.881+0000 7fa45ffff700 10 client.40535.objecter ms_dispatch 0x7fa478001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:44:09.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.881+0000 7fa45ffff700 10 client.40535.objecter _finish_command 5 = system:0 2024-02-23T00:44:09.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.881+0000 7fa45ffff700 20 client.40535.objecter put_session s=0x7fa478059770 osd=2 4 2024-02-23T00:44:09.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.881+0000 7fa45ffff700 15 client.40535.objecter _session_command_op_remove 2 5 2024-02-23T00:44:09.888 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:09.888 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:44:09.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.896+0000 7fa47da17700 10 client.40535.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:44:09.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.896+0000 7fa47da17700 15 client.40535.objecter _session_command_op_assign -1 6 2024-02-23T00:44:09.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.896+0000 7fa47da17700 20 client.40535.objecter _get_session s=0x7fa478059770 osd=2 3 2024-02-23T00:44:09.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.896+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa478059770 osd=2 3 2024-02-23T00:44:09.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.896+0000 7fa47da17700 20 client.40535.objecter _get_session s=0x7fa478059770 osd=2 3 2024-02-23T00:44:09.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.896+0000 7fa47da17700 15 client.40535.objecter _session_command_op_remove -1 6 2024-02-23T00:44:09.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.896+0000 7fa47da17700 20 client.40535.objecter get_session s=0x7fa478059770 osd=2 3 2024-02-23T00:44:09.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.896+0000 7fa47da17700 15 client.40535.objecter _session_command_op_assign 2 6 2024-02-23T00:44:09.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.896+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa478059770 osd=2 4 2024-02-23T00:44:09.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.896+0000 7fa47da17700 10 client.40535.objecter _send_command 6 2024-02-23T00:44:09.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.896+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa478027830 con 0x7fa478028270 2024-02-23T00:44:09.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.898+0000 7fa45ffff700 1 -- 172.21.15.47:0/3631802115 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa478027830 con 0x7fa478028270 2024-02-23T00:44:09.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.898+0000 7fa45ffff700 10 client.40535.objecter ms_dispatch 0x7fa478001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:44:09.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.898+0000 7fa45ffff700 10 client.40535.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:44:09.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.898+0000 7fa45ffff700 20 client.40535.objecter put_session s=0x7fa478059770 osd=2 4 2024-02-23T00:44:09.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.898+0000 7fa45ffff700 15 client.40535.objecter _session_command_op_remove 2 6 2024-02-23T00:44:09.901 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:44:09.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.898+0000 7fa47da17700 10 client.40535.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:09.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.898+0000 7fa47da17700 15 client.40535.objecter _session_command_op_assign -1 7 2024-02-23T00:44:09.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.898+0000 7fa47da17700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fa47811f8a0 0x7fa478121c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:09.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.898+0000 7fa47da17700 20 client.40535.objecter _get_session s=0x7fa47811d030 osd=3 3 2024-02-23T00:44:09.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.898+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa47811d030 osd=3 3 2024-02-23T00:44:09.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.898+0000 7fa47da17700 20 client.40535.objecter _get_session s=0x7fa47811d030 osd=3 3 2024-02-23T00:44:09.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.898+0000 7fa47da17700 15 client.40535.objecter _session_command_op_remove -1 7 2024-02-23T00:44:09.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.898+0000 7fa47da17700 20 client.40535.objecter get_session s=0x7fa47811d030 osd=3 3 2024-02-23T00:44:09.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.898+0000 7fa47da17700 15 client.40535.objecter _session_command_op_assign 3 7 2024-02-23T00:44:09.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.898+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa47811d030 osd=3 4 2024-02-23T00:44:09.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.898+0000 7fa47da17700 10 client.40535.objecter _send_command 7 2024-02-23T00:44:09.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.898+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa4781249b0 con 0x7fa47811f8a0 2024-02-23T00:44:09.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.898+0000 7fa4777fe700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fa47811f8a0 0x7fa478121c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:09.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.899+0000 7fa45ffff700 10 client.40535.objecter ms_handle_connect 0x7fa47811f8a0 2024-02-23T00:44:09.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.899+0000 7fa4777fe700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fa47811f8a0 0x7fa478121c90 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:09.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.899+0000 7fa45ffff700 1 -- 172.21.15.47:0/3631802115 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa4781249b0 con 0x7fa47811f8a0 2024-02-23T00:44:09.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.899+0000 7fa45ffff700 10 client.40535.objecter ms_dispatch 0x7fa478001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:44:09.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.899+0000 7fa45ffff700 10 client.40535.objecter _finish_command 7 = system:0 2024-02-23T00:44:09.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.899+0000 7fa45ffff700 20 client.40535.objecter put_session s=0x7fa47811d030 osd=3 4 2024-02-23T00:44:09.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.899+0000 7fa45ffff700 15 client.40535.objecter _session_command_op_remove 3 7 2024-02-23T00:44:09.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.914+0000 7fa47da17700 10 client.40535.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:44:09.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.914+0000 7fa47da17700 15 client.40535.objecter _session_command_op_assign -1 8 2024-02-23T00:44:09.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.914+0000 7fa47da17700 20 client.40535.objecter _get_session s=0x7fa47811d030 osd=3 3 2024-02-23T00:44:09.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.914+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa47811d030 osd=3 3 2024-02-23T00:44:09.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.914+0000 7fa47da17700 20 client.40535.objecter _get_session s=0x7fa47811d030 osd=3 3 2024-02-23T00:44:09.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.914+0000 7fa47da17700 15 client.40535.objecter _session_command_op_remove -1 8 2024-02-23T00:44:09.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.914+0000 7fa47da17700 20 client.40535.objecter get_session s=0x7fa47811d030 osd=3 3 2024-02-23T00:44:09.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.914+0000 7fa47da17700 15 client.40535.objecter _session_command_op_assign 3 8 2024-02-23T00:44:09.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.914+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa47811d030 osd=3 4 2024-02-23T00:44:09.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.914+0000 7fa47da17700 10 client.40535.objecter _send_command 8 2024-02-23T00:44:09.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.914+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa47811caa0 con 0x7fa47811f8a0 2024-02-23T00:44:09.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.915+0000 7fa45ffff700 1 -- 172.21.15.47:0/3631802115 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa47811caa0 con 0x7fa47811f8a0 2024-02-23T00:44:09.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.915+0000 7fa45ffff700 10 client.40535.objecter ms_dispatch 0x7fa478001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:44:09.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.915+0000 7fa45ffff700 10 client.40535.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:44:09.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.915+0000 7fa45ffff700 20 client.40535.objecter put_session s=0x7fa47811d030 osd=3 4 2024-02-23T00:44:09.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.915+0000 7fa45ffff700 15 client.40535.objecter _session_command_op_remove 3 8 2024-02-23T00:44:09.919 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:44:09.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.915+0000 7fa47da17700 10 client.40535.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:09.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.915+0000 7fa47da17700 15 client.40535.objecter _session_command_op_assign -1 9 2024-02-23T00:44:09.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.915+0000 7fa47da17700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fa47811bca0 0x7fa4781221d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:09.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.915+0000 7fa47da17700 20 client.40535.objecter _get_session s=0x7fa47811cc40 osd=4 3 2024-02-23T00:44:09.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.915+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa47811cc40 osd=4 3 2024-02-23T00:44:09.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.915+0000 7fa47da17700 20 client.40535.objecter _get_session s=0x7fa47811cc40 osd=4 3 2024-02-23T00:44:09.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.915+0000 7fa47da17700 15 client.40535.objecter _session_command_op_remove -1 9 2024-02-23T00:44:09.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.915+0000 7fa47da17700 20 client.40535.objecter get_session s=0x7fa47811cc40 osd=4 3 2024-02-23T00:44:09.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.915+0000 7fa47da17700 15 client.40535.objecter _session_command_op_assign 4 9 2024-02-23T00:44:09.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.915+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa47811cc40 osd=4 4 2024-02-23T00:44:09.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.915+0000 7fa47da17700 10 client.40535.objecter _send_command 9 2024-02-23T00:44:09.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.915+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa47811ce00 con 0x7fa47811bca0 2024-02-23T00:44:09.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.916+0000 7fa476ffd700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fa47811bca0 0x7fa4781221d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:09.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.916+0000 7fa45ffff700 10 client.40535.objecter ms_handle_connect 0x7fa47811bca0 2024-02-23T00:44:09.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.916+0000 7fa476ffd700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fa47811bca0 0x7fa4781221d0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:09.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.917+0000 7fa45ffff700 1 -- 172.21.15.47:0/3631802115 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa47811ce00 con 0x7fa47811bca0 2024-02-23T00:44:09.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.917+0000 7fa45ffff700 10 client.40535.objecter ms_dispatch 0x7fa478001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:44:09.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.917+0000 7fa45ffff700 10 client.40535.objecter _finish_command 9 = system:0 2024-02-23T00:44:09.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.917+0000 7fa45ffff700 20 client.40535.objecter put_session s=0x7fa47811cc40 osd=4 4 2024-02-23T00:44:09.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.917+0000 7fa45ffff700 15 client.40535.objecter _session_command_op_remove 4 9 2024-02-23T00:44:09.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.932+0000 7fa47da17700 10 client.40535.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:44:09.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.932+0000 7fa47da17700 15 client.40535.objecter _session_command_op_assign -1 10 2024-02-23T00:44:09.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.932+0000 7fa47da17700 20 client.40535.objecter _get_session s=0x7fa47811cc40 osd=4 3 2024-02-23T00:44:09.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.932+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa47811cc40 osd=4 3 2024-02-23T00:44:09.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.932+0000 7fa47da17700 20 client.40535.objecter _get_session s=0x7fa47811cc40 osd=4 3 2024-02-23T00:44:09.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.932+0000 7fa47da17700 15 client.40535.objecter _session_command_op_remove -1 10 2024-02-23T00:44:09.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.932+0000 7fa47da17700 20 client.40535.objecter get_session s=0x7fa47811cc40 osd=4 3 2024-02-23T00:44:09.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.932+0000 7fa47da17700 15 client.40535.objecter _session_command_op_assign 4 10 2024-02-23T00:44:09.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.932+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa47811cc40 osd=4 4 2024-02-23T00:44:09.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.932+0000 7fa47da17700 10 client.40535.objecter _send_command 10 2024-02-23T00:44:09.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.932+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa47811ce00 con 0x7fa47811bca0 2024-02-23T00:44:09.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.933+0000 7fa45ffff700 1 -- 172.21.15.47:0/3631802115 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa47811ce00 con 0x7fa47811bca0 2024-02-23T00:44:09.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.933+0000 7fa45ffff700 10 client.40535.objecter ms_dispatch 0x7fa478001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:44:09.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.933+0000 7fa45ffff700 10 client.40535.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:44:09.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.933+0000 7fa45ffff700 20 client.40535.objecter put_session s=0x7fa47811cc40 osd=4 4 2024-02-23T00:44:09.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.933+0000 7fa45ffff700 15 client.40535.objecter _session_command_op_remove 4 10 2024-02-23T00:44:09.936 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:44:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.934+0000 7fa47da17700 10 client.40535.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.934+0000 7fa47da17700 15 client.40535.objecter _session_command_op_assign -1 11 2024-02-23T00:44:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.934+0000 7fa47da17700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fa478123dd0 0x7fa478124210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.934+0000 7fa47da17700 20 client.40535.objecter _get_session s=0x7fa47811ce00 osd=5 3 2024-02-23T00:44:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.934+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa47811ce00 osd=5 3 2024-02-23T00:44:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.934+0000 7fa47da17700 20 client.40535.objecter _get_session s=0x7fa47811ce00 osd=5 3 2024-02-23T00:44:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.934+0000 7fa47da17700 15 client.40535.objecter _session_command_op_remove -1 11 2024-02-23T00:44:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.934+0000 7fa47da17700 20 client.40535.objecter get_session s=0x7fa47811ce00 osd=5 3 2024-02-23T00:44:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.934+0000 7fa47da17700 15 client.40535.objecter _session_command_op_assign 5 11 2024-02-23T00:44:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.934+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa47811ce00 osd=5 4 2024-02-23T00:44:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.934+0000 7fa47da17700 10 client.40535.objecter _send_command 11 2024-02-23T00:44:09.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.934+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa47812b2d0 con 0x7fa478123dd0 2024-02-23T00:44:09.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.934+0000 7fa4767fc700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fa478123dd0 0x7fa478124210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:09.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.934+0000 7fa45ffff700 10 client.40535.objecter ms_handle_connect 0x7fa478123dd0 2024-02-23T00:44:09.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.934+0000 7fa4767fc700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fa478123dd0 0x7fa478124210 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:09.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.935+0000 7fa45ffff700 1 -- 172.21.15.47:0/3631802115 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa47812b2d0 con 0x7fa478123dd0 2024-02-23T00:44:09.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.935+0000 7fa45ffff700 10 client.40535.objecter ms_dispatch 0x7fa478001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:44:09.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.935+0000 7fa45ffff700 10 client.40535.objecter _finish_command 11 = system:0 2024-02-23T00:44:09.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.935+0000 7fa45ffff700 20 client.40535.objecter put_session s=0x7fa47811ce00 osd=5 4 2024-02-23T00:44:09.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.935+0000 7fa45ffff700 15 client.40535.objecter _session_command_op_remove 5 11 2024-02-23T00:44:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.949+0000 7fa47da17700 10 client.40535.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:44:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.949+0000 7fa47da17700 15 client.40535.objecter _session_command_op_assign -1 12 2024-02-23T00:44:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.949+0000 7fa47da17700 20 client.40535.objecter _get_session s=0x7fa47811ce00 osd=5 3 2024-02-23T00:44:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.949+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa47811ce00 osd=5 3 2024-02-23T00:44:09.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.949+0000 7fa47da17700 20 client.40535.objecter _get_session s=0x7fa47811ce00 osd=5 3 2024-02-23T00:44:09.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.949+0000 7fa47da17700 15 client.40535.objecter _session_command_op_remove -1 12 2024-02-23T00:44:09.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.949+0000 7fa47da17700 20 client.40535.objecter get_session s=0x7fa47811ce00 osd=5 3 2024-02-23T00:44:09.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.949+0000 7fa47da17700 15 client.40535.objecter _session_command_op_assign 5 12 2024-02-23T00:44:09.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.949+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa47811ce00 osd=5 4 2024-02-23T00:44:09.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.949+0000 7fa47da17700 10 client.40535.objecter _send_command 12 2024-02-23T00:44:09.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.949+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa478128b70 con 0x7fa478123dd0 2024-02-23T00:44:09.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.951+0000 7fa45ffff700 1 -- 172.21.15.47:0/3631802115 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa478128b70 con 0x7fa478123dd0 2024-02-23T00:44:09.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.951+0000 7fa45ffff700 10 client.40535.objecter ms_dispatch 0x7fa478001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:44:09.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.951+0000 7fa45ffff700 10 client.40535.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:44:09.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.951+0000 7fa45ffff700 20 client.40535.objecter put_session s=0x7fa47811ce00 osd=5 4 2024-02-23T00:44:09.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.951+0000 7fa45ffff700 15 client.40535.objecter _session_command_op_remove 5 12 2024-02-23T00:44:09.954 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:44:09.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.951+0000 7fa47da17700 10 client.40535.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:09.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.951+0000 7fa47da17700 15 client.40535.objecter _session_command_op_assign -1 13 2024-02-23T00:44:09.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.951+0000 7fa47da17700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fa4781291f0 0x7fa47812b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:09.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.951+0000 7fa47da17700 20 client.40535.objecter _get_session s=0x7fa478128b70 osd=6 3 2024-02-23T00:44:09.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.951+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa478128b70 osd=6 3 2024-02-23T00:44:09.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.951+0000 7fa47da17700 20 client.40535.objecter _get_session s=0x7fa478128b70 osd=6 3 2024-02-23T00:44:09.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.951+0000 7fa47da17700 15 client.40535.objecter _session_command_op_remove -1 13 2024-02-23T00:44:09.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.951+0000 7fa47da17700 20 client.40535.objecter get_session s=0x7fa478128b70 osd=6 3 2024-02-23T00:44:09.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.951+0000 7fa47da17700 15 client.40535.objecter _session_command_op_assign 6 13 2024-02-23T00:44:09.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.951+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa478128b70 osd=6 4 2024-02-23T00:44:09.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.951+0000 7fa47da17700 10 client.40535.objecter _send_command 13 2024-02-23T00:44:09.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.951+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa47812e4a0 con 0x7fa4781291f0 2024-02-23T00:44:09.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.952+0000 7fa4777fe700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fa4781291f0 0x7fa47812b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:09.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.952+0000 7fa45ffff700 10 client.40535.objecter ms_handle_connect 0x7fa4781291f0 2024-02-23T00:44:09.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.952+0000 7fa4777fe700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fa4781291f0 0x7fa47812b660 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:09.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.953+0000 7fa45ffff700 1 -- 172.21.15.47:0/3631802115 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa47812e4a0 con 0x7fa4781291f0 2024-02-23T00:44:09.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.953+0000 7fa45ffff700 10 client.40535.objecter ms_dispatch 0x7fa478001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:44:09.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.953+0000 7fa45ffff700 10 client.40535.objecter _finish_command 13 = system:0 2024-02-23T00:44:09.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.953+0000 7fa45ffff700 20 client.40535.objecter put_session s=0x7fa478128b70 osd=6 4 2024-02-23T00:44:09.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.953+0000 7fa45ffff700 15 client.40535.objecter _session_command_op_remove 6 13 2024-02-23T00:44:09.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.967+0000 7fa47da17700 10 client.40535.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:44:09.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.967+0000 7fa47da17700 15 client.40535.objecter _session_command_op_assign -1 14 2024-02-23T00:44:09.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.967+0000 7fa47da17700 20 client.40535.objecter _get_session s=0x7fa478128b70 osd=6 3 2024-02-23T00:44:09.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.967+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa478128b70 osd=6 3 2024-02-23T00:44:09.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.967+0000 7fa47da17700 20 client.40535.objecter _get_session s=0x7fa478128b70 osd=6 3 2024-02-23T00:44:09.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.967+0000 7fa47da17700 15 client.40535.objecter _session_command_op_remove -1 14 2024-02-23T00:44:09.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.967+0000 7fa47da17700 20 client.40535.objecter get_session s=0x7fa478128b70 osd=6 3 2024-02-23T00:44:09.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.967+0000 7fa47da17700 15 client.40535.objecter _session_command_op_assign 6 14 2024-02-23T00:44:09.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.967+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa478128b70 osd=6 4 2024-02-23T00:44:09.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.967+0000 7fa47da17700 10 client.40535.objecter _send_command 14 2024-02-23T00:44:09.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.967+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa47812e4a0 con 0x7fa4781291f0 2024-02-23T00:44:09.971 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:44:09.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.969+0000 7fa45ffff700 1 -- 172.21.15.47:0/3631802115 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa47812e4a0 con 0x7fa4781291f0 2024-02-23T00:44:09.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.969+0000 7fa45ffff700 10 client.40535.objecter ms_dispatch 0x7fa478001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:44:09.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.969+0000 7fa45ffff700 10 client.40535.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:44:09.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.969+0000 7fa45ffff700 20 client.40535.objecter put_session s=0x7fa478128b70 osd=6 4 2024-02-23T00:44:09.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.969+0000 7fa45ffff700 15 client.40535.objecter _session_command_op_remove 6 14 2024-02-23T00:44:09.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.969+0000 7fa47da17700 10 client.40535.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:09.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.969+0000 7fa47da17700 15 client.40535.objecter _session_command_op_assign -1 15 2024-02-23T00:44:09.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.969+0000 7fa47da17700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fa47812d7d0 0x7fa47812dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:09.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.969+0000 7fa47da17700 20 client.40535.objecter _get_session s=0x7fa47812e4a0 osd=7 3 2024-02-23T00:44:09.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.969+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa47812e4a0 osd=7 3 2024-02-23T00:44:09.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.969+0000 7fa47da17700 20 client.40535.objecter _get_session s=0x7fa47812e4a0 osd=7 3 2024-02-23T00:44:09.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.969+0000 7fa47da17700 15 client.40535.objecter _session_command_op_remove -1 15 2024-02-23T00:44:09.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.969+0000 7fa47da17700 20 client.40535.objecter get_session s=0x7fa47812e4a0 osd=7 3 2024-02-23T00:44:09.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.969+0000 7fa47da17700 15 client.40535.objecter _session_command_op_assign 7 15 2024-02-23T00:44:09.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.969+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa47812e4a0 osd=7 4 2024-02-23T00:44:09.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.969+0000 7fa47da17700 10 client.40535.objecter _send_command 15 2024-02-23T00:44:09.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.969+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa478131830 con 0x7fa47812d7d0 2024-02-23T00:44:09.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.969+0000 7fa476ffd700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fa47812d7d0 0x7fa47812dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:09.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.970+0000 7fa45ffff700 10 client.40535.objecter ms_handle_connect 0x7fa47812d7d0 2024-02-23T00:44:09.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.970+0000 7fa476ffd700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fa47812d7d0 0x7fa47812dc30 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:09.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.970+0000 7fa45ffff700 1 -- 172.21.15.47:0/3631802115 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa478131830 con 0x7fa47812d7d0 2024-02-23T00:44:09.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.970+0000 7fa45ffff700 10 client.40535.objecter ms_dispatch 0x7fa478001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:44:09.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.970+0000 7fa45ffff700 10 client.40535.objecter _finish_command 15 = system:0 2024-02-23T00:44:09.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.970+0000 7fa45ffff700 20 client.40535.objecter put_session s=0x7fa47812e4a0 osd=7 4 2024-02-23T00:44:09.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.970+0000 7fa45ffff700 15 client.40535.objecter _session_command_op_remove 7 15 2024-02-23T00:44:09.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.985+0000 7fa47da17700 10 client.40535.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:44:09.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.985+0000 7fa47da17700 15 client.40535.objecter _session_command_op_assign -1 16 2024-02-23T00:44:09.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.985+0000 7fa47da17700 20 client.40535.objecter _get_session s=0x7fa47812e4a0 osd=7 3 2024-02-23T00:44:09.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.985+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa47812e4a0 osd=7 3 2024-02-23T00:44:09.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.985+0000 7fa47da17700 20 client.40535.objecter _get_session s=0x7fa47812e4a0 osd=7 3 2024-02-23T00:44:09.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.985+0000 7fa47da17700 15 client.40535.objecter _session_command_op_remove -1 16 2024-02-23T00:44:09.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.985+0000 7fa47da17700 20 client.40535.objecter get_session s=0x7fa47812e4a0 osd=7 3 2024-02-23T00:44:09.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.985+0000 7fa47da17700 15 client.40535.objecter _session_command_op_assign 7 16 2024-02-23T00:44:09.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.985+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa47812e4a0 osd=7 4 2024-02-23T00:44:09.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.985+0000 7fa47da17700 10 client.40535.objecter _send_command 16 2024-02-23T00:44:09.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.985+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa47812bc60 con 0x7fa47812d7d0 2024-02-23T00:44:09.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.986+0000 7fa45ffff700 1 -- 172.21.15.47:0/3631802115 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa47812bc60 con 0x7fa47812d7d0 2024-02-23T00:44:09.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.986+0000 7fa45ffff700 10 client.40535.objecter ms_dispatch 0x7fa478001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:44:09.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.986+0000 7fa45ffff700 10 client.40535.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:44:09.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.986+0000 7fa45ffff700 20 client.40535.objecter put_session s=0x7fa47812e4a0 osd=7 4 2024-02-23T00:44:09.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.986+0000 7fa45ffff700 15 client.40535.objecter _session_command_op_remove 7 16 2024-02-23T00:44:09.989 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:44:09.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 10 librados: watch_flush enter 2024-02-23T00:44:09.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 10 librados: watch_flush exit 2024-02-23T00:44:09.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 10 client.40535.objecter close_session for osd.0 2024-02-23T00:44:09.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fa4780ffd70 msgr2=0x7fa478100150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:09.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fa4780ffd70 0x7fa478100150 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:09.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa478105d20 osd=0 1 2024-02-23T00:44:09.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 10 client.40535.objecter close_session for osd.1 2024-02-23T00:44:09.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fa4780598f0 msgr2=0x7fa478059d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:09.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fa4780598f0 0x7fa478059d10 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:09.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa478028710 osd=1 1 2024-02-23T00:44:09.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 10 client.40535.objecter close_session for osd.2 2024-02-23T00:44:09.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa478028270 msgr2=0x7fa47811c140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:09.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa478028270 0x7fa47811c140 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:09.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa478059770 osd=2 1 2024-02-23T00:44:09.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 10 client.40535.objecter close_session for osd.3 2024-02-23T00:44:09.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fa47811f8a0 msgr2=0x7fa478121c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:09.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa4777fe700 1 -- 172.21.15.47:0/3631802115 reap_dead start 2024-02-23T00:44:09.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fa47811f8a0 0x7fa478121c90 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:09.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa47811d030 osd=3 1 2024-02-23T00:44:09.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 10 client.40535.objecter close_session for osd.4 2024-02-23T00:44:09.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fa47811bca0 msgr2=0x7fa4781221d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:09.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fa47811bca0 0x7fa4781221d0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:09.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa47811cc40 osd=4 1 2024-02-23T00:44:09.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 10 client.40535.objecter close_session for osd.5 2024-02-23T00:44:09.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fa478123dd0 msgr2=0x7fa478124210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:09.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fa478123dd0 0x7fa478124210 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:09.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa47811ce00 osd=5 1 2024-02-23T00:44:09.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 10 client.40535.objecter close_session for osd.6 2024-02-23T00:44:09.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fa4781291f0 msgr2=0x7fa47812b660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:09.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fa4781291f0 0x7fa47812b660 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:09.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa478128b70 osd=6 1 2024-02-23T00:44:09.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 10 client.40535.objecter close_session for osd.7 2024-02-23T00:44:09.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fa47812d7d0 msgr2=0x7fa47812dc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:09.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fa47812d7d0 0x7fa47812dc30 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:09.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 20 client.40535.objecter put_session s=0x7fa47812e4a0 osd=7 1 2024-02-23T00:44:09.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 20 client.40535.objecter shutdown clearing up homeless session... 2024-02-23T00:44:09.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 10 client.40535.objecter successfully canceled tick 2024-02-23T00:44:09.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa4777fe700 1 -- 172.21.15.47:0/3631802115 reap_dead start 2024-02-23T00:44:09.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa460062cc0 msgr2=0x7fa460065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:09.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa460062cc0 0x7fa460065180 secure :-1 s=READY pgs=2923 cs=0 l=1 rev1=1 rx=0x7fa468004ee0 tx=0x7fa46800b040).stop 2024-02-23T00:44:09.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4780fcab0 msgr2=0x7fa4780f8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:09.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4780fcab0 0x7fa4780f8da0 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 rx=0x7fa46c0048f0 tx=0x7fa46c009cb0).stop 2024-02-23T00:44:09.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 shutdown_connections 2024-02-23T00:44:09.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4780fcab0 0x7fa4780f8da0 unknown :-1 s=CLOSED pgs=2494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:09.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 1 --2- 172.21.15.47:0/3631802115 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa460062cc0 0x7fa460065180 unknown :-1 s=CLOSED pgs=2923 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:09.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 >> 172.21.15.47:0/3631802115 conn(0x7fa4780f26e0 msgr2=0x7fa478060ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:09.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 shutdown_connections 2024-02-23T00:44:09.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 1 -- 172.21.15.47:0/3631802115 wait complete. 2024-02-23T00:44:09.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:09.987+0000 7fa47da17700 1 librados: shutdown 2024-02-23T00:44:09.994 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:44:09.994 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:44:09.994 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:44:09.994 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:44:09.994 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:44:09.995 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:44:09.995 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:44:09.995 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:44:09.995 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:44:10.026 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:10.045 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:44:10.047 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:10.048 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:44:10.149 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:44:10.180 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:10.378 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:44:10.381 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:10.381 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:44:10.395 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset noscrub 2024-02-23T00:44:10.482 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:44:10.516 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:10.606 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.5 dump_historic_ops 2024-02-23T00:44:10.632 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:44:10.635 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:10.635 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:44:10.736 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:44:10.747 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:44:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:44:10] "GET /metrics HTTP/1.1" 200 195393 "" "Prometheus/2.33.4" 2024-02-23T00:44:10.747 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:09.219230+0000 mgr.y (mgr.14517) 223 : cluster [DBG] pgmap v241: 282 pgs: 250 active+clean, 32 unknown; 15 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:44:10.747 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:09.739302+0000 mon.b (mon.1) 218 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:10.747 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:09.842882+0000 mon.c (mon.2) 102 : audit [DBG] from='client.? 172.21.15.47:0/3631802115' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:44:10.747 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:10.561448+0000 mon.a (mon.0) 1325 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi047-82378-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:10.747 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:10.561612+0000 mon.a (mon.0) 1326 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi047-82496-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:44:10.748 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:10.561712+0000 mon.a (mon.0) 1327 : cluster [DBG] osdmap e178: 8 total, 8 up, 8 in 2024-02-23T00:44:10.748 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:10.562476+0000 mon.c (mon.2) 103 : audit [INF] from='client.? 172.21.15.47:0/3414618047' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi047-82496-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi047-82496-15"}]: dispatch 2024-02-23T00:44:10.748 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:10.563649+0000 mon.a (mon.0) 1328 : audit [INF] from='client.? 172.21.15.47:0/1047954873' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi047-82386-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:10.748 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:10.565527+0000 mon.a (mon.0) 1329 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi047-82496-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi047-82496-15"}]: dispatch 2024-02-23T00:44:10.748 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:09.219230+0000 mgr.y (mgr.14517) 223 : cluster [DBG] pgmap v241: 282 pgs: 250 active+clean, 32 unknown; 15 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:44:10.748 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:09.739302+0000 mon.b (mon.1) 218 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:10.748 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:09.842882+0000 mon.c (mon.2) 102 : audit [DBG] from='client.? 172.21.15.47:0/3631802115' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:44:10.748 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:10.561448+0000 mon.a (mon.0) 1325 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi047-82378-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:10.748 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:10.561612+0000 mon.a (mon.0) 1326 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi047-82496-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:44:10.748 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:10.561712+0000 mon.a (mon.0) 1327 : cluster [DBG] osdmap e178: 8 total, 8 up, 8 in 2024-02-23T00:44:10.748 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:10.562476+0000 mon.c (mon.2) 103 : audit [INF] from='client.? 172.21.15.47:0/3414618047' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi047-82496-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi047-82496-15"}]: dispatch 2024-02-23T00:44:10.749 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:10.563649+0000 mon.a (mon.0) 1328 : audit [INF] from='client.? 172.21.15.47:0/1047954873' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi047-82386-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:10.749 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:10.565527+0000 mon.a (mon.0) 1329 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi047-82496-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi047-82496-15"}]: dispatch 2024-02-23T00:44:10.764 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:10.783 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:44:10.786 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:10.787 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:44:10.884 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:44:10.887 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:44:10.914 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:10.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:09.219230+0000 mgr.y (mgr.14517) 223 : cluster [DBG] pgmap v241: 282 pgs: 250 active+clean, 32 unknown; 15 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:44:10.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:09.739302+0000 mon.b (mon.1) 218 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:10.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:09.842882+0000 mon.c (mon.2) 102 : audit [DBG] from='client.? 172.21.15.47:0/3631802115' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:44:10.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:10.561448+0000 mon.a (mon.0) 1325 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi047-82378-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:10.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:10.561612+0000 mon.a (mon.0) 1326 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi047-82496-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:44:10.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:10.561712+0000 mon.a (mon.0) 1327 : cluster [DBG] osdmap e178: 8 total, 8 up, 8 in 2024-02-23T00:44:10.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:10.562476+0000 mon.c (mon.2) 103 : audit [INF] from='client.? 172.21.15.47:0/3414618047' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi047-82496-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi047-82496-15"}]: dispatch 2024-02-23T00:44:10.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:10.563649+0000 mon.a (mon.0) 1328 : audit [INF] from='client.? 172.21.15.47:0/1047954873' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi047-82386-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:10.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:10.565527+0000 mon.a (mon.0) 1329 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi047-82496-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi047-82496-15"}]: dispatch 2024-02-23T00:44:11.963 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:44:11.966 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:11.967 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:44:12.067 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:44:12.097 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:12.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.287+0000 7f24d176a700 1 -- 172.21.15.47:0/1962144740 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f24cc0f96e0 msgr2=0x7f24cc101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:12.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.287+0000 7f24d176a700 1 --2- 172.21.15.47:0/1962144740 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f24cc0f96e0 0x7f24cc101c30 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 rx=0x7f24c0004870 tx=0x7f24c0009cb0).stop 2024-02-23T00:44:12.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.287+0000 7f24d176a700 1 -- 172.21.15.47:0/1962144740 shutdown_connections 2024-02-23T00:44:12.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.287+0000 7f24d176a700 1 --2- 172.21.15.47:0/1962144740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24cc102250 0x7f24cc104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:12.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.287+0000 7f24d176a700 1 --2- 172.21.15.47:0/1962144740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24cc0f8d80 0x7f24cc0f9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:12.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.287+0000 7f24d176a700 1 --2- 172.21.15.47:0/1962144740 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f24cc0f96e0 0x7f24cc101c30 unknown :-1 s=CLOSED pgs=2497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:12.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.287+0000 7f24d176a700 1 -- 172.21.15.47:0/1962144740 >> 172.21.15.47:0/1962144740 conn(0x7f24cc0f26f0 msgr2=0x7f24cc0f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:12.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.287+0000 7f24d176a700 1 -- 172.21.15.47:0/1962144740 shutdown_connections 2024-02-23T00:44:12.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.287+0000 7f24d176a700 1 -- 172.21.15.47:0/1962144740 wait complete. 2024-02-23T00:44:12.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.288+0000 7f24d176a700 1 librados: starting msgr at 2024-02-23T00:44:12.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.288+0000 7f24d176a700 1 librados: starting objecter 2024-02-23T00:44:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.288+0000 7f24d176a700 1 Processor -- start 2024-02-23T00:44:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.288+0000 7f24d176a700 1 -- start start 2024-02-23T00:44:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.288+0000 7f24d176a700 1 librados: setting wanted keys 2024-02-23T00:44:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.288+0000 7f24d176a700 1 librados: calling monclient init 2024-02-23T00:44:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.288+0000 7f24d176a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24cc0f8d80 0x7f24cc0f85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.288+0000 7f24d176a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24cc0f96e0 0x7f24cc0f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.288+0000 7f24d176a700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f24cc102250 0x7f24cc0f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.288+0000 7f24d176a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc0f7860 con 0x7f24cc0f96e0 2024-02-23T00:44:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.288+0000 7f24d176a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc0f79d0 con 0x7f24cc0f8d80 2024-02-23T00:44:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.288+0000 7f24d176a700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc11ad10 con 0x7f24cc102250 2024-02-23T00:44:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.289+0000 7f24cb7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f24cc102250 0x7f24cc0f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.289+0000 7f24cb7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f24cc102250 0x7f24cc0f71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:50216/0 (socket says 172.21.15.47:50216) 2024-02-23T00:44:12.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.289+0000 7f24cb7fe700 1 -- 172.21.15.47:0/3662285077 learned_addr learned my addr 172.21.15.47:0/3662285077 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:44:12.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.289+0000 7f24ca7fc700 1 --2- 172.21.15.47:0/3662285077 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24cc0f96e0 0x7f24cc0f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:12.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.289+0000 7f24caffd700 1 --2- 172.21.15.47:0/3662285077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24cc0f8d80 0x7f24cc0f85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:12.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.289+0000 7f24cb7fe700 1 -- 172.21.15.47:0/3662285077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24cc0f8d80 msgr2=0x7f24cc0f85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:12.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.289+0000 7f24cb7fe700 1 --2- 172.21.15.47:0/3662285077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24cc0f8d80 0x7f24cc0f85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:12.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.289+0000 7f24cb7fe700 1 -- 172.21.15.47:0/3662285077 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24cc0f96e0 msgr2=0x7f24cc0f6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:12.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.289+0000 7f24cb7fe700 1 --2- 172.21.15.47:0/3662285077 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24cc0f96e0 0x7f24cc0f6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:12.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.289+0000 7f24cb7fe700 1 -- 172.21.15.47:0/3662285077 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24bc005040 con 0x7f24cc102250 2024-02-23T00:44:12.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.289+0000 7f24cb7fe700 1 --2- 172.21.15.47:0/3662285077 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f24cc102250 0x7f24cc0f71a0 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 rx=0x7f24c4001df0 tx=0x7f24c4004c50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:12.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.289+0000 7f24b3fff700 10 client.?.objecter ms_handle_connect 0x7f24cc102250 2024-02-23T00:44:12.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.289+0000 7f24b3fff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:44:12.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.289+0000 7f24b3fff700 1 -- 172.21.15.47:0/3662285077 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c4009a10 con 0x7f24cc102250 2024-02-23T00:44:12.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.289+0000 7f24d176a700 1 -- 172.21.15.47:0/3662285077 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24c40097c0 con 0x7f24cc102250 2024-02-23T00:44:12.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.289+0000 7f24b3fff700 1 -- 172.21.15.47:0/3662285077 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f24c4009560 con 0x7f24cc102250 2024-02-23T00:44:12.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.289+0000 7f24b3fff700 1 -- 172.21.15.47:0/3662285077 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c4007c30 con 0x7f24cc102250 2024-02-23T00:44:12.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.289+0000 7f24d176a700 10 client.40547.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:44:12.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.290+0000 7f24d176a700 1 -- 172.21.15.47:0/3662285077 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24c0005040 con 0x7f24cc102250 2024-02-23T00:44:12.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.290+0000 7f24d176a700 1 librados: init done 2024-02-23T00:44:12.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.290+0000 7f24d176a700 1 -- 172.21.15.47:0/3662285077 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24cc05ee80 con 0x7f24cc102250 2024-02-23T00:44:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.292+0000 7f24b3fff700 1 -- 172.21.15.47:0/3662285077 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f24c40090d0 con 0x7f24cc102250 2024-02-23T00:44:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.292+0000 7f24b3fff700 1 --2- 172.21.15.47:0/3662285077 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f24b4062cc0 0x7f24b4065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.293+0000 7f24caffd700 1 --2- 172.21.15.47:0/3662285077 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f24b4062cc0 0x7f24b4065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.293+0000 7f24b3fff700 1 -- 172.21.15.47:0/3662285077 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(179..179 src has 48..179) v4 ==== 10913+0+0 (secure 0 0 0) 0x7f24c407eb00 con 0x7f24cc102250 2024-02-23T00:44:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.293+0000 7f24b3fff700 10 client.40547.objecter ms_dispatch 0x7f24cc001550 osd_map(179..179 src has 48..179) v4 2024-02-23T00:44:12.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.293+0000 7f24b3fff700 3 client.40547.objecter handle_osd_map got epochs [179,179] > 0 2024-02-23T00:44:12.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.293+0000 7f24b3fff700 3 client.40547.objecter handle_osd_map decoding full epoch 179 2024-02-23T00:44:12.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.293+0000 7f24b3fff700 20 client.40547.objecter dump_active .. 0 homeless 2024-02-23T00:44:12.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.293+0000 7f24caffd700 1 --2- 172.21.15.47:0/3662285077 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f24b4062cc0 0x7f24b4065180 secure :-1 s=READY pgs=2925 cs=0 l=1 rev1=1 rx=0x7f24bc009350 tx=0x7f24bc00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:12.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.293+0000 7f24b3fff700 10 client.40547.objecter ms_handle_connect 0x7f24b4062cc0 2024-02-23T00:44:12.297 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:44:12.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.296+0000 7f24b3fff700 1 -- 172.21.15.47:0/3662285077 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f24c404da90 con 0x7f24cc102250 2024-02-23T00:44:12.301 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:12.302 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:44:12.314 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:44:12.403 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:44:12.433 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:12.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:12.513+0000 7f24d176a700 1 -- 172.21.15.47:0/3662285077 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd reweight", "id": 3, "weight": 1.0} v 0) v1 -- 0x7f24cc107010 con 0x7f24cc102250 2024-02-23T00:44:12.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:10.740323+0000 mon.b (mon.1) 219 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:12.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:11.120421+0000 mon.a (mon.0) 1330 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi047-82457-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi047-82457-24"}]': finished 2024-02-23T00:44:12.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:11.120642+0000 mon.a (mon.0) 1331 : audit [INF] from='client.? 172.21.15.47:0/1047954873' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi047-82386-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:12.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:11.553328+0000 mon.a (mon.0) 1332 : cluster [DBG] osdmap e179: 8 total, 8 up, 8 in 2024-02-23T00:44:12.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:10.740323+0000 mon.b (mon.1) 219 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:12.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:11.120421+0000 mon.a (mon.0) 1330 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi047-82457-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi047-82457-24"}]': finished 2024-02-23T00:44:12.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:11.120642+0000 mon.a (mon.0) 1331 : audit [INF] from='client.? 172.21.15.47:0/1047954873' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi047-82386-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:12.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:11.553328+0000 mon.a (mon.0) 1332 : cluster [DBG] osdmap e179: 8 total, 8 up, 8 in 2024-02-23T00:44:12.662 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:44:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:44:12.662 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:10.740323+0000 mon.b (mon.1) 219 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:12.662 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:11.120421+0000 mon.a (mon.0) 1330 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi047-82457-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi047-82457-24"}]': finished 2024-02-23T00:44:12.662 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:11.120642+0000 mon.a (mon.0) 1331 : audit [INF] from='client.? 172.21.15.47:0/1047954873' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi047-82386-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:12.662 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:11.553328+0000 mon.a (mon.0) 1332 : cluster [DBG] osdmap e179: 8 total, 8 up, 8 in 2024-02-23T00:44:12.663 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:44:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:44:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:44:12.682 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:44:12.686 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:12.687 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:44:12.788 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:44:12.816 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:12.835 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:44:12.838 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:12.838 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:44:12.939 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:44:12.955 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:44:12.955 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-02-23T00:44:12.955 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-02-23T00:44:12.955 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-02-23T00:44:12.955 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.955 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.14427.0:1 2.12 2:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected e59)", 2024-02-23T00:44:12.955 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:39:14.739762+0000", 2024-02-23T00:44:12.955 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 298.211484504, 2024-02-23T00:44:12.955 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.046258505999999998, 2024-02-23T00:44:12.955 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:44:12.955 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "reached pg", 2024-02-23T00:44:12.955 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:44:12.955 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.14427", 2024-02-23T00:44:12.955 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1045795476", 2024-02-23T00:44:12.956 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1 2024-02-23T00:44:12.956 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.956 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:44:12.956 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.956 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:44:12.956 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:14.739762+0000", 2024-02-23T00:44:12.956 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:44:12.956 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.956 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.956 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:44:12.956 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:14.739762+0000", 2024-02-23T00:44:12.956 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:44:12.956 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.957 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.957 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:44:12.957 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:14.739760+0000", 2024-02-23T00:44:12.957 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6000000000000003e-06 2024-02-23T00:44:12.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.957 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.957 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:44:12.957 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:14.739767+0000", 2024-02-23T00:44:12.957 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5310000000000007e-06 2024-02-23T00:44:12.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.957 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.957 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:44:12.957 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:14.739777+0000", 2024-02-23T00:44:12.957 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1076e-05 2024-02-23T00:44:12.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.958 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.958 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:44:12.958 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:14.739788+0000", 2024-02-23T00:44:12.958 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6112999999999997e-05 2024-02-23T00:44:12.958 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.958 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.958 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:44:12.958 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:14.739834+0000", 2024-02-23T00:44:12.958 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3006e-05 2024-02-23T00:44:12.958 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.958 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.958 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:44:12.958 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:14.739847+0000", 2024-02-23T00:44:12.958 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.046047980000000002 2024-02-23T00:44:12.958 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.959 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.959 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:44:12.959 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:14.785895+0000", 2024-02-23T00:44:12.959 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000125983 2024-02-23T00:44:12.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.959 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.959 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:44:12.959 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:14.786021+0000", 2024-02-23T00:44:12.959 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.046258505999999998 2024-02-23T00:44:12.959 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:12.959 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:44:12.959 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:12.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.959 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.960 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.24311.0:2 2.12 2:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected e60)", 2024-02-23T00:44:12.960 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:39:14.772432+0000", 2024-02-23T00:44:12.960 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 298.17881481400002, 2024-02-23T00:44:12.960 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013874579, 2024-02-23T00:44:12.960 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:44:12.960 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "reached pg", 2024-02-23T00:44:12.960 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:44:12.960 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.24311", 2024-02-23T00:44:12.960 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/3723202188", 2024-02-23T00:44:12.960 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 2 2024-02-23T00:44:12.960 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.960 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:44:12.960 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.961 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:44:12.961 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:14.772432+0000", 2024-02-23T00:44:12.961 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:44:12.961 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.961 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.961 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:44:12.961 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:14.772432+0000", 2024-02-23T00:44:12.961 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:44:12.961 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.961 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.961 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:44:12.961 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:14.772431+0000", 2024-02-23T00:44:12.961 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.118e-06 2024-02-23T00:44:12.961 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.961 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.961 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:44:12.962 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:14.772436+0000", 2024-02-23T00:44:12.962 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1180000000000003e-06 2024-02-23T00:44:12.962 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.962 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.962 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:44:12.962 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:14.772443+0000", 2024-02-23T00:44:12.962 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0469999999999999e-06 2024-02-23T00:44:12.962 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.962 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.962 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:44:12.962 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:14.772449+0000", 2024-02-23T00:44:12.962 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8992999999999997e-05 2024-02-23T00:44:12.962 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.962 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.962 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:44:12.963 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:14.772498+0000", 2024-02-23T00:44:12.963 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.345e-06 2024-02-23T00:44:12.963 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.963 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.963 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op must wait for map", 2024-02-23T00:44:12.963 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:14.772505+0000", 2024-02-23T00:44:12.963 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0025976290000000002 2024-02-23T00:44:12.963 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.963 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.963 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:44:12.963 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:14.775103+0000", 2024-02-23T00:44:12.963 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6030000000000001e-06 2024-02-23T00:44:12.963 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.963 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.964 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:44:12.964 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:14.775110+0000", 2024-02-23T00:44:12.964 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011121541 2024-02-23T00:44:12.964 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.964 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.964 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:44:12.964 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:14.786231+0000", 2024-02-23T00:44:12.964 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5291999999999997e-05 2024-02-23T00:44:12.964 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.964 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.964 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:44:12.964 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:14.786307+0000", 2024-02-23T00:44:12.964 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013874579 2024-02-23T00:44:12.964 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:12.965 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:44:12.965 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:12.965 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.965 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.965 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.24311.0:555 4.4 4:22d26bf9:::data_loggenerations_metadata:head [watch reconnect cookie 94587070406528 gen 1] snapc 0=[] ondisk+write+known_if_redirected e85)", 2024-02-23T00:44:12.965 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:40:42.859283+0000", 2024-02-23T00:44:12.965 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 210.091964123, 2024-02-23T00:44:12.965 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.079133090000000003, 2024-02-23T00:44:12.965 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:44:12.965 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:44:12.965 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:44:12.965 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.24311", 2024-02-23T00:44:12.965 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/3723202188", 2024-02-23T00:44:12.965 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 555 2024-02-23T00:44:12.965 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.966 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:44:12.966 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.966 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:44:12.966 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:40:42.859283+0000", 2024-02-23T00:44:12.966 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:44:12.966 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.966 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.966 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:44:12.966 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:40:42.859283+0000", 2024-02-23T00:44:12.966 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:44:12.966 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.966 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.966 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:44:12.966 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:40:42.859281+0000", 2024-02-23T00:44:12.966 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5530000000000004e-06 2024-02-23T00:44:12.967 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.967 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.967 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:44:12.967 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:40:42.859289+0000", 2024-02-23T00:44:12.967 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3624e-05 2024-02-23T00:44:12.967 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.967 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.967 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:44:12.967 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:40:42.859303+0000", 2024-02-23T00:44:12.967 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1999999999999999e-06 2024-02-23T00:44:12.967 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.967 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.967 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:44:12.967 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:40:42.859309+0000", 2024-02-23T00:44:12.967 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6791e-05 2024-02-23T00:44:12.967 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.968 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.968 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:44:12.968 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:40:42.859356+0000", 2024-02-23T00:44:12.968 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.5350000000000004e-06 2024-02-23T00:44:12.968 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.968 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.968 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:44:12.968 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:40:42.859364+0000", 2024-02-23T00:44:12.968 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.075947903999999997 2024-02-23T00:44:12.968 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.968 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.968 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:44:12.968 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:40:42.935312+0000", 2024-02-23T00:44:12.969 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000192991 2024-02-23T00:44:12.969 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.969 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.969 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for missing object", 2024-02-23T00:44:12.969 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:40:42.935505+0000", 2024-02-23T00:44:12.969 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001097071 2024-02-23T00:44:12.969 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.969 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.969 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:44:12.969 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:40:42.936602+0000", 2024-02-23T00:44:12.969 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011565000000000001 2024-02-23T00:44:12.969 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.969 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.969 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for rw locks", 2024-02-23T00:44:12.970 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:40:42.936718+0000", 2024-02-23T00:44:12.970 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00042773600000000001 2024-02-23T00:44:12.970 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.970 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.970 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:44:12.970 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:40:42.937146+0000", 2024-02-23T00:44:12.970 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100151 2024-02-23T00:44:12.970 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.970 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.970 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:44:12.970 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:40:42.937246+0000", 2024-02-23T00:44:12.970 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0011700479999999999 2024-02-23T00:44:12.970 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.970 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.971 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:44:12.971 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:40:42.938416+0000", 2024-02-23T00:44:12.971 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.079133090000000003 2024-02-23T00:44:12.971 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:12.971 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:44:12.971 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:12.971 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.971 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.971 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.24769.0:11 8.d 8:b44cf66e:test-rados-api-smithi047-82428-5::foo:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected e111)", 2024-02-23T00:44:12.971 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:00.902153+0000", 2024-02-23T00:44:12.971 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 72.049093687999999, 2024-02-23T00:44:12.971 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.040007200999999999, 2024-02-23T00:44:12.971 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:44:12.971 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:44:12.972 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:44:12.972 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.24769", 2024-02-23T00:44:12.972 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/820656190", 2024-02-23T00:44:12.972 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 11 2024-02-23T00:44:12.972 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.972 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:44:12.972 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.972 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:44:12.972 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.902153+0000", 2024-02-23T00:44:12.972 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:44:12.972 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.972 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.972 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:44:12.972 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.902153+0000", 2024-02-23T00:44:12.973 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:44:12.973 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.973 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.973 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:44:12.973 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.902152+0000", 2024-02-23T00:44:12.973 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6229999999999999e-06 2024-02-23T00:44:12.973 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.973 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.973 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:44:12.973 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.902158+0000", 2024-02-23T00:44:12.973 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0861e-05 2024-02-23T00:44:12.973 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.973 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.973 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:44:12.974 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.902169+0000", 2024-02-23T00:44:12.974 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6389999999999997e-06 2024-02-23T00:44:12.974 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.974 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.974 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:44:12.974 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.902175+0000", 2024-02-23T00:44:12.974 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.002966186 2024-02-23T00:44:12.974 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.974 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.974 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:44:12.974 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.905141+0000", 2024-02-23T00:44:12.974 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000110677 2024-02-23T00:44:12.974 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.974 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.975 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:44:12.975 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.905252+0000", 2024-02-23T00:44:12.975 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00019720999999999999 2024-02-23T00:44:12.975 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.975 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.975 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 2,7", 2024-02-23T00:44:12.975 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.905449+0000", 2024-02-23T00:44:12.975 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0024371100000000001 2024-02-23T00:44:12.975 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.975 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.975 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:44:12.975 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.907886+0000", 2024-02-23T00:44:12.975 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0042338150000000001 2024-02-23T00:44:12.975 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.975 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.976 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:44:12.976 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.912120+0000", 2024-02-23T00:44:12.976 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029932031000000001 2024-02-23T00:44:12.976 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.976 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.976 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:44:12.976 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.942052+0000", 2024-02-23T00:44:12.976 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.2205000000000005e-05 2024-02-23T00:44:12.976 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.976 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.976 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:44:12.976 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.942114+0000", 2024-02-23T00:44:12.976 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6199999999999998e-05 2024-02-23T00:44:12.977 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.977 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.977 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:44:12.977 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.942160+0000", 2024-02-23T00:44:12.977 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.040007200999999999 2024-02-23T00:44:12.977 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:12.977 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:44:12.977 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:12.977 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.977 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.977 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.24775.0:2059 20.10 e112/109 20:0c23f028:test-rados-api-smithi047-82457-16::big:head v 112'2, mlcod=0'0)", 2024-02-23T00:44:12.977 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:01.336044+0000", 2024-02-23T00:44:12.977 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 71.615203245999993, 2024-02-23T00:44:12.977 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.070215984999999995, 2024-02-23T00:44:12.978 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:44:12.978 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:44:12.978 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:44:12.978 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.978 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:44:12.978 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.336044+0000", 2024-02-23T00:44:12.978 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:44:12.978 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.978 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.978 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:44:12.978 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.336044+0000", 2024-02-23T00:44:12.978 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:44:12.978 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.978 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.978 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:44:12.979 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.336043+0000", 2024-02-23T00:44:12.979 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.031567871999999997 2024-02-23T00:44:12.979 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.979 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.979 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:44:12.979 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.367610+0000", 2024-02-23T00:44:12.979 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2143000000000001e-05 2024-02-23T00:44:12.979 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.979 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.979 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:44:12.979 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.367633+0000", 2024-02-23T00:44:12.979 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9549999999999998e-06 2024-02-23T00:44:12.979 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.979 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.980 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:44:12.980 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.367641+0000", 2024-02-23T00:44:12.980 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.5167999999999998e-05 2024-02-23T00:44:12.980 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.980 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.980 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:44:12.980 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.367697+0000", 2024-02-23T00:44:12.980 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7206e-05 2024-02-23T00:44:12.980 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.980 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.980 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:44:12.980 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.367724+0000", 2024-02-23T00:44:12.980 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038475098999999999 2024-02-23T00:44:12.980 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.981 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.981 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:44:12.981 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.406199+0000", 2024-02-23T00:44:12.981 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0538999999999998e-05 2024-02-23T00:44:12.981 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.981 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.981 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:44:12.981 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.406260+0000", 2024-02-23T00:44:12.981 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.070215984999999995 2024-02-23T00:44:12.981 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:12.981 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:44:12.981 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:12.981 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.981 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.982 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(osd.6.0:245 20.12 e112/109 -22:4d27b8a4:::temp_20.12_0_24277_1:head v 0'0, mlcod=0'0)", 2024-02-23T00:44:12.982 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:01.452184+0000", 2024-02-23T00:44:12.982 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 71.499062835999993, 2024-02-23T00:44:12.982 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.016245961, 2024-02-23T00:44:12.982 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:44:12.982 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:44:12.982 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:44:12.982 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.982 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:44:12.982 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.452184+0000", 2024-02-23T00:44:12.982 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:44:12.982 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.982 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.982 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:44:12.983 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.452184+0000", 2024-02-23T00:44:12.983 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999976 2024-02-23T00:44:12.983 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.983 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.983 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:44:12.983 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.452182+0000", 2024-02-23T00:44:12.983 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.002515387 2024-02-23T00:44:12.983 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.983 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.983 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:44:12.983 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.454697+0000", 2024-02-23T00:44:12.983 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4111999999999999e-05 2024-02-23T00:44:12.983 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.983 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.984 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:44:12.984 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.454721+0000", 2024-02-23T00:44:12.984 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.5599999999999994e-06 2024-02-23T00:44:12.984 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.984 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.984 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:44:12.984 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.454730+0000", 2024-02-23T00:44:12.984 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1328000000000001e-05 2024-02-23T00:44:12.984 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.984 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.984 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:44:12.984 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.454781+0000", 2024-02-23T00:44:12.984 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9772999999999999e-05 2024-02-23T00:44:12.985 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.985 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.985 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:44:12.985 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.454801+0000", 2024-02-23T00:44:12.985 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013588922 2024-02-23T00:44:12.985 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.985 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.985 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:44:12.985 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.468390+0000", 2024-02-23T00:44:12.985 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0067000000000001e-05 2024-02-23T00:44:12.985 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.985 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.985 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:44:12.985 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.468430+0000", 2024-02-23T00:44:12.985 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.016245961 2024-02-23T00:44:12.986 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:12.986 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:44:12.986 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:12.986 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.986 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.986 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(osd.6.0:246 20.12 e112/109 -22:4d27b8a4:::temp_20.12_0_24277_1:head v 0'0, mlcod=0'0)", 2024-02-23T00:44:12.986 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:01.470926+0000", 2024-02-23T00:44:12.986 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 71.480320348000006, 2024-02-23T00:44:12.986 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0201228, 2024-02-23T00:44:12.986 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:44:12.986 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:44:12.986 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:44:12.986 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.986 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:44:12.986 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.470926+0000", 2024-02-23T00:44:12.987 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:44:12.987 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.987 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.987 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:44:12.987 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.470926+0000", 2024-02-23T00:44:12.987 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:44:12.987 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.987 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.987 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:44:12.987 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.470925+0000", 2024-02-23T00:44:12.987 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0031871600000000001 2024-02-23T00:44:12.987 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.987 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.987 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:44:12.987 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.474112+0000", 2024-02-23T00:44:12.987 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.6032000000000001e-05 2024-02-23T00:44:12.988 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.988 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.988 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:44:12.988 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.474138+0000", 2024-02-23T00:44:12.988 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.5290000000000008e-06 2024-02-23T00:44:12.988 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.988 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.988 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:44:12.988 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.474146+0000", 2024-02-23T00:44:12.988 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1718000000000002e-05 2024-02-23T00:44:12.988 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.988 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.988 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:44:12.988 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.474198+0000", 2024-02-23T00:44:12.989 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7005999999999998e-05 2024-02-23T00:44:12.989 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.989 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.989 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:44:12.989 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.474235+0000", 2024-02-23T00:44:12.989 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.016765140000000001 2024-02-23T00:44:12.989 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.989 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.989 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:44:12.989 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.491000+0000", 2024-02-23T00:44:12.989 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8930000000000001e-05 2024-02-23T00:44:12.989 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.989 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.989 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:44:12.989 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.491049+0000", 2024-02-23T00:44:12.989 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0201228 2024-02-23T00:44:12.990 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:12.990 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:44:12.990 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:12.990 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.990 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.990 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(osd.6.0:247 20.12 e112/109 -22:4d27b8a4:::temp_20.12_0_24277_1:head v 0'0, mlcod=0'0)", 2024-02-23T00:44:12.990 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:01.492074+0000", 2024-02-23T00:44:12.990 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 71.459172468999995, 2024-02-23T00:44:12.990 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017188332000000001, 2024-02-23T00:44:12.990 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:44:12.990 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:44:12.990 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:44:12.990 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.990 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:44:12.990 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.492074+0000", 2024-02-23T00:44:12.990 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:44:12.990 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.991 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.991 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:44:12.991 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.492074+0000", 2024-02-23T00:44:12.991 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:44:12.991 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.991 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.991 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:44:12.991 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.492073+0000", 2024-02-23T00:44:12.991 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00353272 2024-02-23T00:44:12.991 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.991 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.991 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:44:12.991 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.495606+0000", 2024-02-23T00:44:12.991 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2286999999999999e-05 2024-02-23T00:44:12.991 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.991 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.992 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:44:12.992 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.495628+0000", 2024-02-23T00:44:12.992 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0412e-05 2024-02-23T00:44:12.992 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.992 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.992 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:44:12.992 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.495638+0000", 2024-02-23T00:44:12.992 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.6077999999999998e-05 2024-02-23T00:44:12.992 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.992 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.992 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:44:12.992 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.495724+0000", 2024-02-23T00:44:12.992 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.8677999999999999e-05 2024-02-23T00:44:12.992 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.993 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.993 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:44:12.993 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.495753+0000", 2024-02-23T00:44:12.993 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013480469 2024-02-23T00:44:12.993 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.993 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.993 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:44:12.993 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.509234+0000", 2024-02-23T00:44:12.993 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.9009e-05 2024-02-23T00:44:12.993 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.993 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.993 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:44:12.993 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.509263+0000", 2024-02-23T00:44:12.993 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017188332000000001 2024-02-23T00:44:12.993 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:12.993 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:44:12.994 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:12.994 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.994 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.994 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.24775.0:2061 20.12 e112/109 20:4d27b8a4:test-rados-api-smithi047-82457-16::big.copy:head v 112'1, mlcod=0'0)", 2024-02-23T00:44:12.994 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:01.497370+0000", 2024-02-23T00:44:12.994 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 71.453877120000001, 2024-02-23T00:44:12.994 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011938891, 2024-02-23T00:44:12.994 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:44:12.994 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:44:12.994 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:44:12.994 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.994 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:44:12.994 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.497370+0000", 2024-02-23T00:44:12.994 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:44:12.994 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.995 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.995 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:44:12.995 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.497370+0000", 2024-02-23T00:44:12.995 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:44:12.995 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.995 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.995 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:44:12.995 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.497368+0000", 2024-02-23T00:44:12.995 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1321999999999999e-05 2024-02-23T00:44:12.995 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.995 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.995 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:44:12.995 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.497380+0000", 2024-02-23T00:44:12.995 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6954999999999999e-05 2024-02-23T00:44:12.996 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.996 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.996 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:44:12.996 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.497397+0000", 2024-02-23T00:44:12.996 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.8340000000000005e-06 2024-02-23T00:44:12.996 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.996 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.996 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:44:12.996 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.497406+0000", 2024-02-23T00:44:12.996 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001880655 2024-02-23T00:44:12.996 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.996 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.996 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:44:12.996 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.499286+0000", 2024-02-23T00:44:12.997 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8538000000000001e-05 2024-02-23T00:44:12.997 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.997 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.997 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:44:12.997 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.499305+0000", 2024-02-23T00:44:12.997 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0099917470000000005 2024-02-23T00:44:12.997 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.997 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.997 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:44:12.997 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.509297+0000", 2024-02-23T00:44:12.997 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2002e-05 2024-02-23T00:44:12.997 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.997 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.997 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:44:12.997 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.509309+0000", 2024-02-23T00:44:12.997 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011938891 2024-02-23T00:44:12.998 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:12.998 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:44:12.998 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:12.998 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.998 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.998 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.25117.0:5 29.d e112/109 29:b3c717c4:::4.obj:head v 112'24, mlcod=111'22)", 2024-02-23T00:44:12.998 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:01.872497+0000", 2024-02-23T00:44:12.998 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 71.078750014999997, 2024-02-23T00:44:12.998 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015898065999999999, 2024-02-23T00:44:12.998 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:44:12.998 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:44:12.998 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:44:12.998 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.998 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:44:12.998 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.872497+0000", 2024-02-23T00:44:12.999 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:44:12.999 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.999 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.999 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:44:12.999 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.872497+0000", 2024-02-23T00:44:12.999 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:44:12.999 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.999 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.999 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:44:12.999 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.872495+0000", 2024-02-23T00:44:12.999 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3344e-05 2024-02-23T00:44:12.999 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:12.999 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:12.999 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:44:12.999 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.872509+0000", 2024-02-23T00:44:13.000 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9233999999999999e-05 2024-02-23T00:44:13.000 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.000 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.000 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:44:13.000 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.872528+0000", 2024-02-23T00:44:13.000 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5700000000000004e-06 2024-02-23T00:44:13.000 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.000 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.000 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:44:13.000 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.872535+0000", 2024-02-23T00:44:13.000 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3579999999999999e-05 2024-02-23T00:44:13.000 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.000 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.000 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:44:13.001 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.872579+0000", 2024-02-23T00:44:13.001 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8539999999999999e-05 2024-02-23T00:44:13.001 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.001 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.001 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:44:13.001 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.872598+0000", 2024-02-23T00:44:13.001 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015771070000000002 2024-02-23T00:44:13.001 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.001 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.001 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:44:13.001 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.888369+0000", 2024-02-23T00:44:13.001 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.6197999999999999e-05 2024-02-23T00:44:13.001 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.001 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.002 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:44:13.002 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.888395+0000", 2024-02-23T00:44:13.002 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015898065999999999 2024-02-23T00:44:13.002 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:13.002 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:44:13.002 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:13.002 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.002 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.002 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.24775.0:2074 20.1a 20:5de0a470:test-rados-api-smithi047-82457-17::big3:head [writefull 0~25165828] snapc 0=[] ondisk+write+known_if_redirected e112)", 2024-02-23T00:44:13.002 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:01.911456+0000", 2024-02-23T00:44:13.002 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 71.039790949999997, 2024-02-23T00:44:13.002 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.135054068, 2024-02-23T00:44:13.002 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:44:13.002 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:44:13.003 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:44:13.003 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.24775", 2024-02-23T00:44:13.003 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1191937413", 2024-02-23T00:44:13.003 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 2074 2024-02-23T00:44:13.003 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.003 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:44:13.003 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.003 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:44:13.003 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.911456+0000", 2024-02-23T00:44:13.003 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:44:13.003 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.003 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.003 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:44:13.003 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.911456+0000", 2024-02-23T00:44:13.004 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:44:13.004 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.004 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.004 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:44:13.004 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.911454+0000", 2024-02-23T00:44:13.004 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.045409616999999999 2024-02-23T00:44:13.004 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.004 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.004 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:44:13.004 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.956864+0000", 2024-02-23T00:44:13.004 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8623000000000001e-05 2024-02-23T00:44:13.004 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.004 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.005 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:44:13.005 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.956882+0000", 2024-02-23T00:44:13.005 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6969999999999998e-06 2024-02-23T00:44:13.005 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.005 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.005 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:44:13.005 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.956888+0000", 2024-02-23T00:44:13.005 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.6355e-05 2024-02-23T00:44:13.005 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.005 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.005 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:44:13.005 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.956924+0000", 2024-02-23T00:44:13.005 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.2887999999999996e-05 2024-02-23T00:44:13.005 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.006 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.006 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:44:13.006 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.957017+0000", 2024-02-23T00:44:13.006 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7796999999999999e-05 2024-02-23T00:44:13.006 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.006 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.006 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0,7", 2024-02-23T00:44:13.006 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.957115+0000", 2024-02-23T00:44:13.006 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.060621657000000002 2024-02-23T00:44:13.006 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.006 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.006 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:44:13.006 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.017737+0000", 2024-02-23T00:44:13.006 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.468e-05 2024-02-23T00:44:13.007 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.007 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.007 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:44:13.007 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.017771+0000", 2024-02-23T00:44:13.007 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028684866 2024-02-23T00:44:13.007 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.007 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.007 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:44:13.007 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.046456+0000", 2024-02-23T00:44:13.007 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.1053000000000002e-05 2024-02-23T00:44:13.007 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.007 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.007 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:44:13.007 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.046487+0000", 2024-02-23T00:44:13.008 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2582000000000002e-05 2024-02-23T00:44:13.008 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.008 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.008 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:44:13.008 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.046510+0000", 2024-02-23T00:44:13.008 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.135054068 2024-02-23T00:44:13.008 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:13.008 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:44:13.008 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:13.008 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.008 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.008 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.24775.0:2076 20.1a 20:5de0a470:test-rados-api-smithi047-82457-17::big3:head [omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected e112)", 2024-02-23T00:44:13.008 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:02.049332+0000", 2024-02-23T00:44:13.009 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 70.901915187, 2024-02-23T00:44:13.009 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014523405, 2024-02-23T00:44:13.009 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:44:13.009 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:44:13.009 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:44:13.009 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.24775", 2024-02-23T00:44:13.009 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1191937413", 2024-02-23T00:44:13.009 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 2076 2024-02-23T00:44:13.009 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.009 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:44:13.009 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.009 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:44:13.009 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.049332+0000", 2024-02-23T00:44:13.009 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:44:13.010 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.010 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.010 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:44:13.010 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.049332+0000", 2024-02-23T00:44:13.010 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:44:13.010 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.010 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.010 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:44:13.010 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.049331+0000", 2024-02-23T00:44:13.010 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0026349609999999999 2024-02-23T00:44:13.010 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.010 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.010 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:44:13.010 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.051966+0000", 2024-02-23T00:44:13.011 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2293000000000001e-05 2024-02-23T00:44:13.011 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.011 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.011 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:44:13.011 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.051978+0000", 2024-02-23T00:44:13.011 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0080000000000002e-06 2024-02-23T00:44:13.011 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.011 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.011 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:44:13.011 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.051986+0000", 2024-02-23T00:44:13.011 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.5505e-05 2024-02-23T00:44:13.011 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.011 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.011 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:44:13.011 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.052021+0000", 2024-02-23T00:44:13.012 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3334000000000001e-05 2024-02-23T00:44:13.012 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.012 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.012 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:44:13.012 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.052075+0000", 2024-02-23T00:44:13.012 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0030188070000000001 2024-02-23T00:44:13.012 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.012 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.012 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0,7", 2024-02-23T00:44:13.012 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.055094+0000", 2024-02-23T00:44:13.012 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0080902790000000006 2024-02-23T00:44:13.012 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.012 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.012 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:44:13.013 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.063184+0000", 2024-02-23T00:44:13.013 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00051940000000000005 2024-02-23T00:44:13.013 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.013 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.013 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:44:13.013 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.063703+0000", 2024-02-23T00:44:13.013 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9100000000000002e-05 2024-02-23T00:44:13.013 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.013 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.013 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:44:13.013 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.063742+0000", 2024-02-23T00:44:13.013 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.8199999999999998e-05 2024-02-23T00:44:13.013 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.013 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.014 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:44:13.014 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.063801+0000", 2024-02-23T00:44:13.014 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4428999999999997e-05 2024-02-23T00:44:13.014 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.014 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.014 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:44:13.014 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.063855+0000", 2024-02-23T00:44:13.014 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014523405 2024-02-23T00:44:13.014 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:13.014 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:44:13.014 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:13.014 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.014 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.014 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.15099.0:2 34.6 34:602f83fe:::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e114)", 2024-02-23T00:44:13.014 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:03.223645+0000", 2024-02-23T00:44:13.015 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 69.727601755999999, 2024-02-23T00:44:13.015 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015291755000000001, 2024-02-23T00:44:13.015 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:44:13.015 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:44:13.015 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:44:13.015 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.15099", 2024-02-23T00:44:13.015 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/3592004948", 2024-02-23T00:44:13.015 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 2 2024-02-23T00:44:13.015 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.015 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:44:13.015 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.015 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:44:13.015 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.223645+0000", 2024-02-23T00:44:13.015 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:44:13.015 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.016 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.016 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:44:13.016 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.223645+0000", 2024-02-23T00:44:13.016 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:44:13.016 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.016 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.016 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:44:13.016 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.223643+0000", 2024-02-23T00:44:13.016 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5359999999999998e-06 2024-02-23T00:44:13.016 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.016 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.016 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:44:13.016 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.223653+0000", 2024-02-23T00:44:13.016 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9113000000000001e-05 2024-02-23T00:44:13.017 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.017 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.017 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:44:13.017 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.223672+0000", 2024-02-23T00:44:13.017 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.4780000000000008e-06 2024-02-23T00:44:13.017 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.017 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.017 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:44:13.017 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.223680+0000", 2024-02-23T00:44:13.017 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1610000000000001e-05 2024-02-23T00:44:13.017 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.017 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.017 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:44:13.017 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.223742+0000", 2024-02-23T00:44:13.018 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4283999999999994e-05 2024-02-23T00:44:13.018 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.018 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.018 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:44:13.018 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.223836+0000", 2024-02-23T00:44:13.018 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012437900000000001 2024-02-23T00:44:13.018 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.018 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.018 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1,2", 2024-02-23T00:44:13.018 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.223960+0000", 2024-02-23T00:44:13.018 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037544500000000002 2024-02-23T00:44:13.018 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.018 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.018 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:44:13.019 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.224336+0000", 2024-02-23T00:44:13.019 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013498499000000001 2024-02-23T00:44:13.019 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.019 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.019 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:44:13.019 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.237834+0000", 2024-02-23T00:44:13.019 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0010338649999999999 2024-02-23T00:44:13.019 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.019 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.019 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:44:13.019 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.238868+0000", 2024-02-23T00:44:13.019 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7762999999999997e-05 2024-02-23T00:44:13.019 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.019 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.020 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:44:13.020 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.238906+0000", 2024-02-23T00:44:13.020 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.0822999999999999e-05 2024-02-23T00:44:13.020 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.020 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.020 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:44:13.020 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.238937+0000", 2024-02-23T00:44:13.020 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015291755000000001 2024-02-23T00:44:13.020 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:13.020 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:44:13.020 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:13.020 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.020 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.020 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.24997.0:1 39.0s0 39:1e3e2c6e:test-rados-api-smithi047-82402-23::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e114)", 2024-02-23T00:44:13.021 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:03.229886+0000", 2024-02-23T00:44:13.021 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 69.721361040999994, 2024-02-23T00:44:13.021 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.126777119, 2024-02-23T00:44:13.021 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:44:13.021 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:44:13.021 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:44:13.021 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.24997", 2024-02-23T00:44:13.021 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1505701467", 2024-02-23T00:44:13.021 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1 2024-02-23T00:44:13.021 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.021 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:44:13.021 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.021 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:44:13.021 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.229886+0000", 2024-02-23T00:44:13.021 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:44:13.021 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.022 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.022 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:44:13.022 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.229886+0000", 2024-02-23T00:44:13.022 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:44:13.022 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.022 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.022 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:44:13.022 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.229884+0000", 2024-02-23T00:44:13.022 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4189999999999999e-06 2024-02-23T00:44:13.022 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.022 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.022 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:44:13.022 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.229891+0000", 2024-02-23T00:44:13.023 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.9709999999999999e-06 2024-02-23T00:44:13.023 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.023 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.023 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:44:13.023 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.229901+0000", 2024-02-23T00:44:13.023 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0509999999999997e-06 2024-02-23T00:44:13.023 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.023 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.023 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:44:13.023 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.229909+0000", 2024-02-23T00:44:13.023 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.2765e-05 2024-02-23T00:44:13.023 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.023 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.023 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:44:13.024 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.229961+0000", 2024-02-23T00:44:13.024 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.625e-06 2024-02-23T00:44:13.024 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.024 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.024 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:44:13.024 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.229965+0000", 2024-02-23T00:44:13.024 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1077668439999999 2024-02-23T00:44:13.024 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.024 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.024 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:44:13.024 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.337732+0000", 2024-02-23T00:44:13.024 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012659300000000001 2024-02-23T00:44:13.024 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.024 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.025 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:44:13.025 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.337859+0000", 2024-02-23T00:44:13.025 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0010284910000000001 2024-02-23T00:44:13.025 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.025 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.025 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:44:13.025 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.338887+0000", 2024-02-23T00:44:13.025 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00067548200000000004 2024-02-23T00:44:13.025 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.025 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.025 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:44:13.025 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.339563+0000", 2024-02-23T00:44:13.025 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017058336 2024-02-23T00:44:13.025 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.026 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.026 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:44:13.026 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.356621+0000", 2024-02-23T00:44:13.026 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1999999999999998e-05 2024-02-23T00:44:13.026 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.026 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.026 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:44:13.026 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.356663+0000", 2024-02-23T00:44:13.026 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.126777119 2024-02-23T00:44:13.026 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:13.026 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:44:13.026 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:13.026 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.026 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.027 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(osd.1.0:297 20.5 e140/109 -22:a70ec860:::temp_20.5_0_24170_1:head v 0'0, mlcod=112'1)", 2024-02-23T00:44:13.027 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:32.555598+0000", 2024-02-23T00:44:13.027 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 40.395648325000003, 2024-02-23T00:44:13.027 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.049529112, 2024-02-23T00:44:13.027 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:44:13.027 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:44:13.027 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:44:13.027 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.027 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:44:13.027 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.555598+0000", 2024-02-23T00:44:13.027 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:44:13.027 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.027 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.027 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:44:13.028 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.555598+0000", 2024-02-23T00:44:13.028 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:44:13.028 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.028 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.028 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:44:13.028 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.555597+0000", 2024-02-23T00:44:13.028 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.039391554000000002 2024-02-23T00:44:13.028 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.028 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.028 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:44:13.028 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.594989+0000", 2024-02-23T00:44:13.028 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9025000000000002e-05 2024-02-23T00:44:13.028 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.029 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.029 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:44:13.029 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.595008+0000", 2024-02-23T00:44:13.029 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7115000000000001e-05 2024-02-23T00:44:13.029 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.029 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.029 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:44:13.029 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.595025+0000", 2024-02-23T00:44:13.029 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.2793000000000001e-05 2024-02-23T00:44:13.029 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.029 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.029 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:44:13.029 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.595078+0000", 2024-02-23T00:44:13.030 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1537999999999999e-05 2024-02-23T00:44:13.030 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.030 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.030 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:44:13.030 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.595099+0000", 2024-02-23T00:44:13.030 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0099891800000000003 2024-02-23T00:44:13.030 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.030 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.030 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:44:13.030 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.605088+0000", 2024-02-23T00:44:13.030 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9180000000000001e-05 2024-02-23T00:44:13.030 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.030 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.030 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:44:13.031 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.605128+0000", 2024-02-23T00:44:13.031 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.049529112 2024-02-23T00:44:13.031 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:13.031 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:44:13.031 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:13.031 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.031 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.031 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(osd.1.0:298 20.5 e140/109 -22:a70ec860:::temp_20.5_0_24170_1:head v 0'0, mlcod=112'1)", 2024-02-23T00:44:13.031 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:32.595037+0000", 2024-02-23T00:44:13.031 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 40.356210193000003, 2024-02-23T00:44:13.031 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.052500485999999999, 2024-02-23T00:44:13.031 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:44:13.031 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:44:13.031 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:44:13.032 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.032 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:44:13.032 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.595037+0000", 2024-02-23T00:44:13.032 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:44:13.032 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.032 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.032 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:44:13.032 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.595037+0000", 2024-02-23T00:44:13.032 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:44:13.032 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.032 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.032 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:44:13.032 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.595036+0000", 2024-02-23T00:44:13.033 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.042091349 2024-02-23T00:44:13.033 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.033 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.033 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:44:13.033 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.637127+0000", 2024-02-23T00:44:13.033 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6313000000000001e-05 2024-02-23T00:44:13.033 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.033 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.033 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:44:13.033 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.637143+0000", 2024-02-23T00:44:13.033 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4218000000000001e-05 2024-02-23T00:44:13.033 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.033 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.033 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:44:13.034 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.637167+0000", 2024-02-23T00:44:13.034 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2020999999999994e-05 2024-02-23T00:44:13.034 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.034 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.034 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:44:13.034 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.637240+0000", 2024-02-23T00:44:13.034 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0267000000000001e-05 2024-02-23T00:44:13.034 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.034 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.034 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:44:13.034 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.637260+0000", 2024-02-23T00:44:13.034 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010234715 2024-02-23T00:44:13.034 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.034 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.035 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:44:13.035 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.647494+0000", 2024-02-23T00:44:13.035 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.2564000000000001e-05 2024-02-23T00:44:13.035 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.035 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.035 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:44:13.035 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.647537+0000", 2024-02-23T00:44:13.035 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.052500485999999999 2024-02-23T00:44:13.035 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:13.035 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:44:13.035 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:13.035 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.035 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.035 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(osd.1.0:299 20.5 e140/109 -22:a70ec860:::temp_20.5_0_24170_1:head v 0'0, mlcod=112'1)", 2024-02-23T00:44:13.036 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:32.637181+0000", 2024-02-23T00:44:13.036 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 40.314065749999997, 2024-02-23T00:44:13.036 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.039894883999999999, 2024-02-23T00:44:13.036 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:44:13.036 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:44:13.036 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:44:13.036 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.036 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:44:13.036 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.637181+0000", 2024-02-23T00:44:13.036 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:44:13.036 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.036 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.037 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:44:13.037 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.637181+0000", 2024-02-23T00:44:13.037 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:44:13.037 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.037 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.037 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:44:13.037 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.637180+0000", 2024-02-23T00:44:13.037 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029888125000000001 2024-02-23T00:44:13.037 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.037 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.037 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:44:13.037 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.667068+0000", 2024-02-23T00:44:13.037 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9562999999999999e-05 2024-02-23T00:44:13.037 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.038 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.038 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:44:13.038 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.667088+0000", 2024-02-23T00:44:13.038 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.0469999999999999e-06 2024-02-23T00:44:13.038 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.038 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.038 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:44:13.038 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.667097+0000", 2024-02-23T00:44:13.038 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2268000000000007e-05 2024-02-23T00:44:13.038 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.038 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.038 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:44:13.038 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.667169+0000", 2024-02-23T00:44:13.038 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7490999999999999e-05 2024-02-23T00:44:13.039 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.039 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.039 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:44:13.039 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.667187+0000", 2024-02-23T00:44:13.039 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0098505959999999997 2024-02-23T00:44:13.039 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.039 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.039 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:44:13.039 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.677037+0000", 2024-02-23T00:44:13.039 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.8726999999999997e-05 2024-02-23T00:44:13.039 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.039 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.039 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:44:13.039 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.677076+0000", 2024-02-23T00:44:13.039 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.039894883999999999 2024-02-23T00:44:13.040 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:13.040 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:44:13.040 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:13.040 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.040 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.040 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.24775.0:2080 20.5 e140/109 20:a70ec860:test-rados-api-smithi047-82457-17::big3.copy:head v 140'3, mlcod=112'1)", 2024-02-23T00:44:13.040 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:32.667110+0000", 2024-02-23T00:44:13.040 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 40.284136898, 2024-02-23T00:44:13.040 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.024788509, 2024-02-23T00:44:13.040 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:44:13.040 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:44:13.040 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:44:13.040 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.041 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:44:13.041 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.667110+0000", 2024-02-23T00:44:13.041 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:44:13.041 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.041 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.041 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:44:13.041 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.667110+0000", 2024-02-23T00:44:13.041 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:44:13.041 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.041 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.041 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:44:13.041 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.667109+0000", 2024-02-23T00:44:13.041 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.018720831 2024-02-23T00:44:13.041 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.042 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.042 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:44:13.042 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.685829+0000", 2024-02-23T00:44:13.042 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0949e-05 2024-02-23T00:44:13.042 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.042 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.042 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:44:13.042 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.685840+0000", 2024-02-23T00:44:13.042 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9130000000000003e-06 2024-02-23T00:44:13.042 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.042 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.042 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:44:13.042 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.685848+0000", 2024-02-23T00:44:13.042 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3827999999999999e-05 2024-02-23T00:44:13.043 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.043 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.043 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:44:13.043 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.685912+0000", 2024-02-23T00:44:13.043 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.12e-05 2024-02-23T00:44:13.043 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.043 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.043 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:44:13.043 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.685933+0000", 2024-02-23T00:44:13.043 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0059192359999999996 2024-02-23T00:44:13.043 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.043 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.043 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:44:13.043 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.691852+0000", 2024-02-23T00:44:13.043 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5884000000000003e-05 2024-02-23T00:44:13.044 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.044 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.044 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:44:13.044 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.691898+0000", 2024-02-23T00:44:13.044 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.024788509 2024-02-23T00:44:13.044 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:13.044 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:44:13.044 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:13.044 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.044 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.044 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.31603.0:7 79.6s0 79:6ddaff94:test-rados-api-smithi047-82487-12::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e145)", 2024-02-23T00:44:13.044 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:37.207111+0000", 2024-02-23T00:44:13.044 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 35.744135888000002, 2024-02-23T00:44:13.045 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.050316774000000002, 2024-02-23T00:44:13.045 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:44:13.045 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:44:13.045 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:44:13.045 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.31603", 2024-02-23T00:44:13.045 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1361361966", 2024-02-23T00:44:13.045 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 7 2024-02-23T00:44:13.045 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.045 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:44:13.045 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.045 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:44:13.045 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.207111+0000", 2024-02-23T00:44:13.045 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:44:13.045 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.046 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.046 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:44:13.046 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.207111+0000", 2024-02-23T00:44:13.046 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:44:13.046 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.046 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.046 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:44:13.046 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.207109+0000", 2024-02-23T00:44:13.046 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.571e-06 2024-02-23T00:44:13.046 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.046 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.046 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:44:13.046 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.207117+0000", 2024-02-23T00:44:13.046 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0654e-05 2024-02-23T00:44:13.046 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.047 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.047 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:44:13.047 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.207127+0000", 2024-02-23T00:44:13.047 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0709999999999997e-06 2024-02-23T00:44:13.047 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.047 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.047 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:44:13.047 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.207134+0000", 2024-02-23T00:44:13.047 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010019800000000001 2024-02-23T00:44:13.047 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.047 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.047 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:44:13.047 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.207235+0000", 2024-02-23T00:44:13.047 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013806100000000001 2024-02-23T00:44:13.048 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.048 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.048 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:44:13.048 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.207373+0000", 2024-02-23T00:44:13.048 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00043899 2024-02-23T00:44:13.048 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.048 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.048 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:44:13.048 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.207812+0000", 2024-02-23T00:44:13.048 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00089506700000000004 2024-02-23T00:44:13.048 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.048 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.048 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:44:13.048 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.208707+0000", 2024-02-23T00:44:13.049 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.048651597999999997 2024-02-23T00:44:13.049 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.049 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.049 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:44:13.049 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.257358+0000", 2024-02-23T00:44:13.049 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.9276000000000006e-05 2024-02-23T00:44:13.049 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.049 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.049 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:44:13.049 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.257428+0000", 2024-02-23T00:44:13.049 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.050316774000000002 2024-02-23T00:44:13.049 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:13.049 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:44:13.049 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:13.050 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.050 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.050 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.31603.0:9 79.6s0 79:6ddaff94:test-rados-api-smithi047-82487-12::foo:head [writefull 0~128] snapc 0=[] ondisk+write+known_if_redirected e146)", 2024-02-23T00:44:13.050 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:38.402504+0000", 2024-02-23T00:44:13.050 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 34.548742353000002, 2024-02-23T00:44:13.050 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036230714999999997, 2024-02-23T00:44:13.050 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:44:13.050 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:44:13.050 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:44:13.050 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.31603", 2024-02-23T00:44:13.050 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1361361966", 2024-02-23T00:44:13.050 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 9 2024-02-23T00:44:13.050 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.050 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:44:13.050 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.051 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:44:13.051 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.402504+0000", 2024-02-23T00:44:13.051 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:44:13.051 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.051 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.051 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:44:13.051 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.402504+0000", 2024-02-23T00:44:13.051 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:44:13.051 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.051 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.051 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:44:13.051 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.402503+0000", 2024-02-23T00:44:13.051 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.6512e-05 2024-02-23T00:44:13.051 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.052 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.052 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:44:13.052 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.402529+0000", 2024-02-23T00:44:13.052 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7428999999999999e-05 2024-02-23T00:44:13.052 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.052 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.052 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:44:13.052 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.402547+0000", 2024-02-23T00:44:13.052 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2077e-05 2024-02-23T00:44:13.052 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.052 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.052 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:44:13.052 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.402559+0000", 2024-02-23T00:44:13.052 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000108826 2024-02-23T00:44:13.052 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.053 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.053 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:44:13.053 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.402668+0000", 2024-02-23T00:44:13.053 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010388299999999999 2024-02-23T00:44:13.053 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.053 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.053 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:44:13.053 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.402771+0000", 2024-02-23T00:44:13.053 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00051973999999999998 2024-02-23T00:44:13.053 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.053 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.053 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:44:13.053 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.403291+0000", 2024-02-23T00:44:13.054 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00049441100000000005 2024-02-23T00:44:13.054 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.054 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.054 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:44:13.054 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.403786+0000", 2024-02-23T00:44:13.054 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034898881 2024-02-23T00:44:13.054 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.054 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.054 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:44:13.054 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.438685+0000", 2024-02-23T00:44:13.054 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0618000000000002e-05 2024-02-23T00:44:13.054 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:44:13.054 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:44:13.054 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:44:13.054 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.438735+0000", 2024-02-23T00:44:13.055 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036230714999999997 2024-02-23T00:44:13.055 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:13.055 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:44:13.055 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:13.055 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:44:13.055 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:44:13.055 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:44:13.056 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:13.056 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:13.056 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:44:13.057 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:44:13.157 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:44:13.189 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:13.208 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:44:13.211 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:13.211 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:44:13.312 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:44:13.340 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:13.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:13.464+0000 7f24b3fff700 1 -- 172.21.15.47:0/3662285077 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 3, "weight": 1.0}]=0 reweighted osd.3 to 1 (10000) v181) v1 ==== 113+0+0 (secure 0 0 0) 0x7f24c40510b0 con 0x7f24cc102250 2024-02-23T00:44:13.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:13.468+0000 7f24d176a700 10 librados: watch_flush enter 2024-02-23T00:44:13.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:13.469+0000 7f24d176a700 10 librados: watch_flush exit 2024-02-23T00:44:13.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:13.469+0000 7f24d176a700 20 client.40547.objecter shutdown clearing up homeless session... 2024-02-23T00:44:13.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:13.469+0000 7f24d176a700 10 client.40547.objecter successfully canceled tick 2024-02-23T00:44:13.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:13.470+0000 7f24d176a700 1 -- 172.21.15.47:0/3662285077 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f24b4062cc0 msgr2=0x7f24b4065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:13.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:13.470+0000 7f24d176a700 1 --2- 172.21.15.47:0/3662285077 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f24b4062cc0 0x7f24b4065180 secure :-1 s=READY pgs=2925 cs=0 l=1 rev1=1 rx=0x7f24bc009350 tx=0x7f24bc00b040).stop 2024-02-23T00:44:13.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:13.470+0000 7f24d176a700 1 -- 172.21.15.47:0/3662285077 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f24cc102250 msgr2=0x7f24cc0f71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:13.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:13.470+0000 7f24d176a700 1 --2- 172.21.15.47:0/3662285077 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f24cc102250 0x7f24cc0f71a0 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 rx=0x7f24c4001df0 tx=0x7f24c4004c50).stop 2024-02-23T00:44:13.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:13.471+0000 7f24d176a700 1 -- 172.21.15.47:0/3662285077 shutdown_connections 2024-02-23T00:44:13.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:13.471+0000 7f24d176a700 1 --2- 172.21.15.47:0/3662285077 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f24b4062cc0 0x7f24b4065180 secure :-1 s=CLOSED pgs=2925 cs=0 l=1 rev1=1 rx=0x7f24bc009350 tx=0x7f24bc00b040).stop 2024-02-23T00:44:13.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:13.471+0000 7f24d176a700 1 --2- 172.21.15.47:0/3662285077 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f24cc102250 0x7f24cc0f71a0 unknown :-1 s=CLOSED pgs=2498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:13.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:13.471+0000 7f24d176a700 1 --2- 172.21.15.47:0/3662285077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24cc0f8d80 0x7f24cc0f85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:13.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:13.471+0000 7f24d176a700 1 --2- 172.21.15.47:0/3662285077 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24cc0f96e0 0x7f24cc0f6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:13.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:13.471+0000 7f24d176a700 1 -- 172.21.15.47:0/3662285077 >> 172.21.15.47:0/3662285077 conn(0x7f24cc0f26f0 msgr2=0x7f24cc0f3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:13.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:13.473+0000 7f24d176a700 1 -- 172.21.15.47:0/3662285077 shutdown_connections 2024-02-23T00:44:13.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:13.474+0000 7f24d176a700 1 -- 172.21.15.47:0/3662285077 wait complete. 2024-02-23T00:44:13.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:13.474+0000 7f24d176a700 1 librados: shutdown 2024-02-23T00:44:13.485 INFO:teuthology.orchestra.run.smithi047.stderr:reweighted osd.3 to 1 (10000) 2024-02-23T00:44:13.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:11.219584+0000 mgr.y (mgr.14517) 224 : cluster [DBG] pgmap v244: 290 pgs: 250 active+clean, 40 unknown; 15 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:44:13.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:11.741758+0000 mon.b (mon.1) 220 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:13.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:11.815830+0000 osd.7 (osd.7) 107 : cluster [DBG] 93.9 scrub starts 2024-02-23T00:44:13.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:11.819484+0000 osd.7 (osd.7) 108 : cluster [DBG] 93.9 scrub ok 2024-02-23T00:44:13.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:12.367740+0000 mgr.y (mgr.14517) 225 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:44:13.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:12.457589+0000 mon.a (mon.0) 1333 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi047-82496-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi047-82496-15"}]': finished 2024-02-23T00:44:13.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:12.458034+0000 mon.a (mon.0) 1334 : cluster [DBG] osdmap e180: 8 total, 8 up, 8 in 2024-02-23T00:44:13.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:12.463079+0000 mon.c (mon.2) 104 : audit [INF] from='client.? 172.21.15.47:0/865084253' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi047-82378-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:13.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:12.463846+0000 mon.a (mon.0) 1335 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi047-82378-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:13.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:12.466248+0000 mon.c (mon.2) 105 : audit [INF] from='client.? 172.21.15.47:0/3108721855' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82457-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:13.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:12.466664+0000 mon.a (mon.0) 1336 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82457-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:13.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:12.514641+0000 mon.c (mon.2) 106 : audit [INF] from='client.? 172.21.15.47:0/3662285077' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-23T00:44:13.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:12.514932+0000 mon.a (mon.0) 1337 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-23T00:44:13.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:12.742881+0000 mon.b (mon.1) 221 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:13.727 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:44:13.746 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:11.219584+0000 mgr.y (mgr.14517) 224 : cluster [DBG] pgmap v244: 290 pgs: 250 active+clean, 40 unknown; 15 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:44:13.746 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:11.741758+0000 mon.b (mon.1) 220 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:13.746 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:11.815830+0000 osd.7 (osd.7) 107 : cluster [DBG] 93.9 scrub starts 2024-02-23T00:44:13.746 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:11.819484+0000 osd.7 (osd.7) 108 : cluster [DBG] 93.9 scrub ok 2024-02-23T00:44:13.746 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:12.367740+0000 mgr.y (mgr.14517) 225 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:44:13.746 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:12.457589+0000 mon.a (mon.0) 1333 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi047-82496-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi047-82496-15"}]': finished 2024-02-23T00:44:13.746 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:12.458034+0000 mon.a (mon.0) 1334 : cluster [DBG] osdmap e180: 8 total, 8 up, 8 in 2024-02-23T00:44:13.746 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:12.463079+0000 mon.c (mon.2) 104 : audit [INF] from='client.? 172.21.15.47:0/865084253' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi047-82378-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:13.747 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:12.463846+0000 mon.a (mon.0) 1335 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi047-82378-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:13.747 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:12.466248+0000 mon.c (mon.2) 105 : audit [INF] from='client.? 172.21.15.47:0/3108721855' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82457-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:13.747 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:12.466664+0000 mon.a (mon.0) 1336 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82457-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:13.747 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:12.514641+0000 mon.c (mon.2) 106 : audit [INF] from='client.? 172.21.15.47:0/3662285077' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-23T00:44:13.747 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:12.514932+0000 mon.a (mon.0) 1337 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-23T00:44:13.747 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:12.742881+0000 mon.b (mon.1) 221 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:13.747 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:11.219584+0000 mgr.y (mgr.14517) 224 : cluster [DBG] pgmap v244: 290 pgs: 250 active+clean, 40 unknown; 15 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-23T00:44:13.747 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:11.741758+0000 mon.b (mon.1) 220 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:13.747 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:11.815830+0000 osd.7 (osd.7) 107 : cluster [DBG] 93.9 scrub starts 2024-02-23T00:44:13.747 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:11.819484+0000 osd.7 (osd.7) 108 : cluster [DBG] 93.9 scrub ok 2024-02-23T00:44:13.747 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:12.367740+0000 mgr.y (mgr.14517) 225 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:44:13.748 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:12.457589+0000 mon.a (mon.0) 1333 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi047-82496-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi047-82496-15"}]': finished 2024-02-23T00:44:13.748 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:12.458034+0000 mon.a (mon.0) 1334 : cluster [DBG] osdmap e180: 8 total, 8 up, 8 in 2024-02-23T00:44:13.748 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:12.463079+0000 mon.c (mon.2) 104 : audit [INF] from='client.? 172.21.15.47:0/865084253' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi047-82378-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:13.748 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:12.463846+0000 mon.a (mon.0) 1335 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi047-82378-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:13.748 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:12.466248+0000 mon.c (mon.2) 105 : audit [INF] from='client.? 172.21.15.47:0/3108721855' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82457-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:13.748 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:12.466664+0000 mon.a (mon.0) 1336 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82457-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:13.748 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:12.514641+0000 mon.c (mon.2) 106 : audit [INF] from='client.? 172.21.15.47:0/3662285077' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-23T00:44:13.748 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:12.514932+0000 mon.a (mon.0) 1337 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-23T00:44:13.748 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:12.742881+0000 mon.b (mon.1) 221 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:14.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:13.220020+0000 mgr.y (mgr.14517) 226 : cluster [DBG] pgmap v246: 362 pgs: 282 active+clean, 72 unknown, 8 creating+peering; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 249 B/s wr, 1 op/s 2024-02-23T00:44:14.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:13.461074+0000 mon.a (mon.0) 1338 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi047-82378-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:14.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:13.461186+0000 mon.a (mon.0) 1339 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82457-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:14.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:13.461333+0000 mon.a (mon.0) 1340 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-23T00:44:14.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:13.461388+0000 mon.a (mon.0) 1341 : cluster [DBG] osdmap e181: 8 total, 8 up, 8 in 2024-02-23T00:44:14.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:13.467904+0000 mon.c (mon.2) 107 : audit [INF] from='client.? 172.21.15.47:0/3108721855' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82457-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:14.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:13.468402+0000 mon.a (mon.0) 1342 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82457-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:14.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:13.743959+0000 mon.b (mon.1) 222 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:14.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:13.220020+0000 mgr.y (mgr.14517) 226 : cluster [DBG] pgmap v246: 362 pgs: 282 active+clean, 72 unknown, 8 creating+peering; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 249 B/s wr, 1 op/s 2024-02-23T00:44:14.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:13.461074+0000 mon.a (mon.0) 1338 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi047-82378-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:14.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:13.461186+0000 mon.a (mon.0) 1339 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82457-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:14.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:13.461333+0000 mon.a (mon.0) 1340 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-23T00:44:14.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:13.461388+0000 mon.a (mon.0) 1341 : cluster [DBG] osdmap e181: 8 total, 8 up, 8 in 2024-02-23T00:44:14.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:13.467904+0000 mon.c (mon.2) 107 : audit [INF] from='client.? 172.21.15.47:0/3108721855' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82457-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:14.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:13.468402+0000 mon.a (mon.0) 1342 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82457-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:14.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:13.743959+0000 mon.b (mon.1) 222 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:14.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:13.220020+0000 mgr.y (mgr.14517) 226 : cluster [DBG] pgmap v246: 362 pgs: 282 active+clean, 72 unknown, 8 creating+peering; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 249 B/s wr, 1 op/s 2024-02-23T00:44:14.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:13.461074+0000 mon.a (mon.0) 1338 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi047-82378-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:14.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:13.461186+0000 mon.a (mon.0) 1339 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82457-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:14.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:13.461333+0000 mon.a (mon.0) 1340 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-23T00:44:14.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:13.461388+0000 mon.a (mon.0) 1341 : cluster [DBG] osdmap e181: 8 total, 8 up, 8 in 2024-02-23T00:44:14.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:13.467904+0000 mon.c (mon.2) 107 : audit [INF] from='client.? 172.21.15.47:0/3108721855' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82457-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:14.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:13.468402+0000 mon.a (mon.0) 1342 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82457-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:14.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:13.743959+0000 mon.b (mon.1) 222 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:15.269 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight 4 1 2024-02-23T00:44:15.340 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:44:15.397 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:15.397 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:44:15.397 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:44:15.498 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:44:15.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:14.464454+0000 mon.a (mon.0) 1343 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82457-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:15.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:14.464622+0000 mon.a (mon.0) 1344 : cluster [DBG] osdmap e182: 8 total, 8 up, 8 in 2024-02-23T00:44:15.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:14.470452+0000 mon.c (mon.2) 108 : audit [INF] from='client.? 172.21.15.47:0/1915740554' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi047-82386-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:15.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:14.493050+0000 mon.a (mon.0) 1345 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi047-82386-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:15.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:14.745070+0000 mon.b (mon.1) 223 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:15.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:15.103747+0000 mon.a (mon.0) 1346 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi047-82386-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:15.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:15.103934+0000 mon.a (mon.0) 1347 : cluster [DBG] osdmap e183: 8 total, 8 up, 8 in 2024-02-23T00:44:15.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:15.115532+0000 mon.b (mon.1) 224 : audit [INF] from='client.? 172.21.15.47:0/101635949' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi047-82378-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:15.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:15.124581+0000 mon.a (mon.0) 1348 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi047-82378-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:15.526 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:14.464454+0000 mon.a (mon.0) 1343 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82457-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:15.526 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:14.464622+0000 mon.a (mon.0) 1344 : cluster [DBG] osdmap e182: 8 total, 8 up, 8 in 2024-02-23T00:44:15.526 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:14.470452+0000 mon.c (mon.2) 108 : audit [INF] from='client.? 172.21.15.47:0/1915740554' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi047-82386-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:15.526 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:14.493050+0000 mon.a (mon.0) 1345 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi047-82386-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:15.526 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:14.745070+0000 mon.b (mon.1) 223 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:15.526 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:15.103747+0000 mon.a (mon.0) 1346 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi047-82386-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:15.526 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:15.103934+0000 mon.a (mon.0) 1347 : cluster [DBG] osdmap e183: 8 total, 8 up, 8 in 2024-02-23T00:44:15.526 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:15.115532+0000 mon.b (mon.1) 224 : audit [INF] from='client.? 172.21.15.47:0/101635949' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi047-82378-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:15.526 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:15.124581+0000 mon.a (mon.0) 1348 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi047-82378-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:15.528 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:15.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:14.464454+0000 mon.a (mon.0) 1343 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82457-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:15.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:14.464622+0000 mon.a (mon.0) 1344 : cluster [DBG] osdmap e182: 8 total, 8 up, 8 in 2024-02-23T00:44:15.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:14.470452+0000 mon.c (mon.2) 108 : audit [INF] from='client.? 172.21.15.47:0/1915740554' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi047-82386-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:15.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:14.493050+0000 mon.a (mon.0) 1345 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi047-82386-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:15.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:14.745070+0000 mon.b (mon.1) 223 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:15.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:15.103747+0000 mon.a (mon.0) 1346 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi047-82386-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:15.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:15.103934+0000 mon.a (mon.0) 1347 : cluster [DBG] osdmap e183: 8 total, 8 up, 8 in 2024-02-23T00:44:15.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:15.115532+0000 mon.b (mon.1) 224 : audit [INF] from='client.? 172.21.15.47:0/101635949' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi047-82378-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:15.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:15.124581+0000 mon.a (mon.0) 1348 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi047-82378-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:16.040 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.6 dump_ops_in_flight 2024-02-23T00:44:16.333 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:44:16.843 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:15.220441+0000 mgr.y (mgr.14517) 227 : cluster [DBG] pgmap v250: 330 pgs: 250 active+clean, 72 unknown, 8 creating+peering; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:44:16.843 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:15.746198+0000 mon.b (mon.1) 225 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:16.843 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:16.104716+0000 mon.a (mon.0) 1349 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi047-82378-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:16.843 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:16.104934+0000 mon.a (mon.0) 1350 : cluster [DBG] osdmap e184: 8 total, 8 up, 8 in 2024-02-23T00:44:16.844 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:16.106971+0000 mon.c (mon.2) 109 : audit [INF] from='client.? 172.21.15.47:0/3108721855' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82457-24"}]: dispatch 2024-02-23T00:44:16.844 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:16.107644+0000 mon.a (mon.0) 1351 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82457-24"}]: dispatch 2024-02-23T00:44:16.844 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:15.220441+0000 mgr.y (mgr.14517) 227 : cluster [DBG] pgmap v250: 330 pgs: 250 active+clean, 72 unknown, 8 creating+peering; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:44:16.844 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:15.746198+0000 mon.b (mon.1) 225 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:16.844 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:16.104716+0000 mon.a (mon.0) 1349 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi047-82378-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:16.844 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:16.104934+0000 mon.a (mon.0) 1350 : cluster [DBG] osdmap e184: 8 total, 8 up, 8 in 2024-02-23T00:44:16.844 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:16.106971+0000 mon.c (mon.2) 109 : audit [INF] from='client.? 172.21.15.47:0/3108721855' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82457-24"}]: dispatch 2024-02-23T00:44:16.844 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:16.107644+0000 mon.a (mon.0) 1351 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82457-24"}]: dispatch 2024-02-23T00:44:16.926 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:15.220441+0000 mgr.y (mgr.14517) 227 : cluster [DBG] pgmap v250: 330 pgs: 250 active+clean, 72 unknown, 8 creating+peering; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:44:16.926 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:15.746198+0000 mon.b (mon.1) 225 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:16.926 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:16.104716+0000 mon.a (mon.0) 1349 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi047-82378-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:16.926 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:16.104934+0000 mon.a (mon.0) 1350 : cluster [DBG] osdmap e184: 8 total, 8 up, 8 in 2024-02-23T00:44:16.926 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:16.106971+0000 mon.c (mon.2) 109 : audit [INF] from='client.? 172.21.15.47:0/3108721855' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82457-24"}]: dispatch 2024-02-23T00:44:16.926 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:16.107644+0000 mon.a (mon.0) 1351 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82457-24"}]: dispatch 2024-02-23T00:44:17.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.018+0000 7fce76418700 1 -- 172.21.15.47:0/4056477507 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fce700fcbb0 msgr2=0x7fce700fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:17.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.018+0000 7fce76418700 1 --2- 172.21.15.47:0/4056477507 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fce700fcbb0 0x7fce700fcf90 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 rx=0x7fce60004870 tx=0x7fce60009cb0).stop 2024-02-23T00:44:17.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.019+0000 7fce76418700 1 -- 172.21.15.47:0/4056477507 shutdown_connections 2024-02-23T00:44:17.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.019+0000 7fce76418700 1 --2- 172.21.15.47:0/4056477507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce700f7400 0x7fce70103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:17.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.019+0000 7fce76418700 1 --2- 172.21.15.47:0/4056477507 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fce700fcbb0 0x7fce700fcf90 unknown :-1 s=CLOSED pgs=2504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:17.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.019+0000 7fce76418700 1 --2- 172.21.15.47:0/4056477507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce700fd510 0x7fce700f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:17.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.019+0000 7fce76418700 1 -- 172.21.15.47:0/4056477507 >> 172.21.15.47:0/4056477507 conn(0x7fce700f2820 msgr2=0x7fce700f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:17.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.019+0000 7fce76418700 1 -- 172.21.15.47:0/4056477507 shutdown_connections 2024-02-23T00:44:17.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.019+0000 7fce76418700 1 -- 172.21.15.47:0/4056477507 wait complete. 2024-02-23T00:44:17.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.019+0000 7fce76418700 1 librados: starting msgr at 2024-02-23T00:44:17.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.019+0000 7fce76418700 1 librados: starting objecter 2024-02-23T00:44:17.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.019+0000 7fce76418700 1 Processor -- start 2024-02-23T00:44:17.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.019+0000 7fce76418700 1 -- start start 2024-02-23T00:44:17.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.019+0000 7fce76418700 1 librados: setting wanted keys 2024-02-23T00:44:17.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.019+0000 7fce76418700 1 librados: calling monclient init 2024-02-23T00:44:17.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.020+0000 7fce76418700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fce700f7400 0x7fce70109170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:17.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.020+0000 7fce76418700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce700fcbb0 0x7fce70109660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:17.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.020+0000 7fce76418700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce700fd510 0x7fce70109c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:17.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.020+0000 7fce76418700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce70114350 con 0x7fce700fd510 2024-02-23T00:44:17.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.020+0000 7fce76418700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce701144c0 con 0x7fce700fcbb0 2024-02-23T00:44:17.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.020+0000 7fce76418700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce70114680 con 0x7fce700f7400 2024-02-23T00:44:17.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.020+0000 7fce6ffff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fce700f7400 0x7fce70109170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:17.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.020+0000 7fce6ffff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fce700f7400 0x7fce70109170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52594/0 (socket says 172.21.15.47:52594) 2024-02-23T00:44:17.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.020+0000 7fce6ffff700 1 -- 172.21.15.47:0/782549266 learned_addr learned my addr 172.21.15.47:0/782549266 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:44:17.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.020+0000 7fce6f7fe700 1 --2- 172.21.15.47:0/782549266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce700fcbb0 0x7fce70109660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:17.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.020+0000 7fce749b5700 1 --2- 172.21.15.47:0/782549266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce700fd510 0x7fce70109c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:17.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.020+0000 7fce6ffff700 1 -- 172.21.15.47:0/782549266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce700fcbb0 msgr2=0x7fce70109660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:17.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.020+0000 7fce6ffff700 1 --2- 172.21.15.47:0/782549266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce700fcbb0 0x7fce70109660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:17.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.020+0000 7fce6ffff700 1 -- 172.21.15.47:0/782549266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce700fd510 msgr2=0x7fce70109c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:17.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.020+0000 7fce6ffff700 1 --2- 172.21.15.47:0/782549266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce700fd510 0x7fce70109c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:17.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.020+0000 7fce6ffff700 1 -- 172.21.15.47:0/782549266 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce680097c0 con 0x7fce700f7400 2024-02-23T00:44:17.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.021+0000 7fce6ffff700 1 --2- 172.21.15.47:0/782549266 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fce700f7400 0x7fce70109170 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 rx=0x7fce60000e50 tx=0x7fce60009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:17.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.021+0000 7fce6d7fa700 10 client.?.objecter ms_handle_connect 0x7fce700f7400 2024-02-23T00:44:17.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.021+0000 7fce6d7fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:44:17.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.021+0000 7fce6d7fa700 1 -- 172.21.15.47:0/782549266 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6000e030 con 0x7fce700f7400 2024-02-23T00:44:17.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.021+0000 7fce6d7fa700 1 -- 172.21.15.47:0/782549266 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fce60013d70 con 0x7fce700f7400 2024-02-23T00:44:17.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.021+0000 7fce76418700 1 -- 172.21.15.47:0/782549266 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce60005040 con 0x7fce700f7400 2024-02-23T00:44:17.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.021+0000 7fce6d7fa700 1 -- 172.21.15.47:0/782549266 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce60018070 con 0x7fce700f7400 2024-02-23T00:44:17.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.021+0000 7fce76418700 10 client.40568.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:44:17.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.021+0000 7fce76418700 1 -- 172.21.15.47:0/782549266 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce70114b70 con 0x7fce700f7400 2024-02-23T00:44:17.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.021+0000 7fce76418700 1 librados: init done 2024-02-23T00:44:17.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.022+0000 7fce76418700 1 -- 172.21.15.47:0/782549266 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce7005fa60 con 0x7fce700f7400 2024-02-23T00:44:17.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.022+0000 7fce6d7fa700 1 -- 172.21.15.47:0/782549266 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fce60009240 con 0x7fce700f7400 2024-02-23T00:44:17.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.023+0000 7fce6d7fa700 1 --2- 172.21.15.47:0/782549266 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fce58062cc0 0x7fce58065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:17.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.023+0000 7fce6d7fa700 1 -- 172.21.15.47:0/782549266 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(184..184 src has 48..184) v4 ==== 10985+0+0 (secure 0 0 0) 0x7fce6007ee70 con 0x7fce700f7400 2024-02-23T00:44:17.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.023+0000 7fce6d7fa700 10 client.40568.objecter ms_dispatch 0x7fce70001550 osd_map(184..184 src has 48..184) v4 2024-02-23T00:44:17.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.023+0000 7fce6d7fa700 3 client.40568.objecter handle_osd_map got epochs [184,184] > 0 2024-02-23T00:44:17.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.023+0000 7fce6d7fa700 3 client.40568.objecter handle_osd_map decoding full epoch 184 2024-02-23T00:44:17.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.023+0000 7fce6f7fe700 1 --2- 172.21.15.47:0/782549266 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fce58062cc0 0x7fce58065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:17.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.023+0000 7fce6d7fa700 20 client.40568.objecter dump_active .. 0 homeless 2024-02-23T00:44:17.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.023+0000 7fce6d7fa700 10 client.40568.objecter ms_handle_connect 0x7fce58062cc0 2024-02-23T00:44:17.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.023+0000 7fce6f7fe700 1 --2- 172.21.15.47:0/782549266 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fce58062cc0 0x7fce58065180 secure :-1 s=READY pgs=2929 cs=0 l=1 rev1=1 rx=0x7fce70061700 tx=0x7fce6400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:17.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.027+0000 7fce6d7fa700 1 -- 172.21.15.47:0/782549266 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fce6004de20 con 0x7fce700f7400 2024-02-23T00:44:17.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:17.237+0000 7fce76418700 1 -- 172.21.15.47:0/782549266 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7fce70105f00 con 0x7fce700f7400 2024-02-23T00:44:17.378 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:44:17.383 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:17.383 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:44:17.484 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:44:17.514 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:17.882 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:16.747464+0000 mon.b (mon.1) 226 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:17.882 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:17.105537+0000 mon.a (mon.0) 1352 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82457-24"}]': finished 2024-02-23T00:44:17.883 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:17.105645+0000 mon.a (mon.0) 1353 : cluster [DBG] osdmap e185: 8 total, 8 up, 8 in 2024-02-23T00:44:17.883 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:17.108296+0000 mon.c (mon.2) 110 : audit [INF] from='client.? 172.21.15.47:0/3108721855' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82457-24"}]: dispatch 2024-02-23T00:44:17.883 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:17.109007+0000 mon.a (mon.0) 1354 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82457-24"}]: dispatch 2024-02-23T00:44:17.883 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:17.117771+0000 mon.c (mon.2) 111 : audit [INF] from='client.? 172.21.15.47:0/2164499477' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi047-82386-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:17.883 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:17.118198+0000 mon.a (mon.0) 1355 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi047-82386-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:17.883 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:16.747464+0000 mon.b (mon.1) 226 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:17.883 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:17.105537+0000 mon.a (mon.0) 1352 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82457-24"}]': finished 2024-02-23T00:44:17.883 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:17.105645+0000 mon.a (mon.0) 1353 : cluster [DBG] osdmap e185: 8 total, 8 up, 8 in 2024-02-23T00:44:17.883 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:17.108296+0000 mon.c (mon.2) 110 : audit [INF] from='client.? 172.21.15.47:0/3108721855' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82457-24"}]: dispatch 2024-02-23T00:44:17.883 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:17.109007+0000 mon.a (mon.0) 1354 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82457-24"}]: dispatch 2024-02-23T00:44:17.883 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:17.117771+0000 mon.c (mon.2) 111 : audit [INF] from='client.? 172.21.15.47:0/2164499477' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi047-82386-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:17.883 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:17.118198+0000 mon.a (mon.0) 1355 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi047-82386-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:17.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:16.747464+0000 mon.b (mon.1) 226 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:17.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:17.105537+0000 mon.a (mon.0) 1352 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82457-24"}]': finished 2024-02-23T00:44:17.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:17.105645+0000 mon.a (mon.0) 1353 : cluster [DBG] osdmap e185: 8 total, 8 up, 8 in 2024-02-23T00:44:17.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:17.108296+0000 mon.c (mon.2) 110 : audit [INF] from='client.? 172.21.15.47:0/3108721855' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82457-24"}]: dispatch 2024-02-23T00:44:17.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:17.109007+0000 mon.a (mon.0) 1354 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82457-24"}]: dispatch 2024-02-23T00:44:17.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:17.117771+0000 mon.c (mon.2) 111 : audit [INF] from='client.? 172.21.15.47:0/2164499477' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi047-82386-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:17.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:17.118198+0000 mon.a (mon.0) 1355 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi047-82386-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:18.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:18.111+0000 7fce6d7fa700 1 -- 172.21.15.47:0/782549266 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v186) v1 ==== 91+0+0 (secure 0 0 0) 0x7fce60051440 con 0x7fce700f7400 2024-02-23T00:44:18.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:18.116+0000 7fce76418700 10 librados: watch_flush enter 2024-02-23T00:44:18.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:18.116+0000 7fce76418700 10 librados: watch_flush exit 2024-02-23T00:44:18.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:18.116+0000 7fce76418700 20 client.40568.objecter shutdown clearing up homeless session... 2024-02-23T00:44:18.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:18.116+0000 7fce76418700 10 client.40568.objecter successfully canceled tick 2024-02-23T00:44:18.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:18.122+0000 7fce76418700 1 -- 172.21.15.47:0/782549266 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fce58062cc0 msgr2=0x7fce58065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:18.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:18.122+0000 7fce76418700 1 --2- 172.21.15.47:0/782549266 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fce58062cc0 0x7fce58065180 secure :-1 s=READY pgs=2929 cs=0 l=1 rev1=1 rx=0x7fce70061700 tx=0x7fce6400b040).stop 2024-02-23T00:44:18.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:18.122+0000 7fce76418700 1 -- 172.21.15.47:0/782549266 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fce700f7400 msgr2=0x7fce70109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:18.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:18.122+0000 7fce76418700 1 --2- 172.21.15.47:0/782549266 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fce700f7400 0x7fce70109170 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 rx=0x7fce60000e50 tx=0x7fce60009cb0).stop 2024-02-23T00:44:18.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:18.122+0000 7fce76418700 1 -- 172.21.15.47:0/782549266 shutdown_connections 2024-02-23T00:44:18.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:18.122+0000 7fce76418700 1 --2- 172.21.15.47:0/782549266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce700fd510 0x7fce70109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:18.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:18.122+0000 7fce76418700 1 --2- 172.21.15.47:0/782549266 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fce58062cc0 0x7fce58065180 secure :-1 s=CLOSED pgs=2929 cs=0 l=1 rev1=1 rx=0x7fce70061700 tx=0x7fce6400b040).stop 2024-02-23T00:44:18.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:18.122+0000 7fce76418700 1 --2- 172.21.15.47:0/782549266 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fce700f7400 0x7fce70109170 secure :-1 s=CLOSED pgs=2505 cs=0 l=1 rev1=1 rx=0x7fce60000e50 tx=0x7fce60009cb0).stop 2024-02-23T00:44:18.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:18.122+0000 7fce76418700 1 --2- 172.21.15.47:0/782549266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce700fcbb0 0x7fce70109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:18.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:18.122+0000 7fce76418700 1 -- 172.21.15.47:0/782549266 >> 172.21.15.47:0/782549266 conn(0x7fce700f2820 msgr2=0x7fce700fb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:18.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:18.132+0000 7fce76418700 1 -- 172.21.15.47:0/782549266 shutdown_connections 2024-02-23T00:44:18.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:18.133+0000 7fce76418700 1 -- 172.21.15.47:0/782549266 wait complete. 2024-02-23T00:44:18.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:18.133+0000 7fce76418700 1 librados: shutdown 2024-02-23T00:44:18.144 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is unset 2024-02-23T00:44:18.264 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:44:18.266 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:18.267 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:44:18.368 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:44:18.398 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:18.480 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:44:18.484 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:18.484 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:44:18.544 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:44:18.544 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-02-23T00:44:18.544 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 0 2024-02-23T00:44:18.544 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:44:18.585 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:44:18.616 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:18.756 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:17.221380+0000 mgr.y (mgr.14517) 228 : cluster [DBG] pgmap v253: 290 pgs: 258 active+clean, 32 creating+peering; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 816 B/s wr, 2 op/s 2024-02-23T00:44:18.756 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:17.239445+0000 mon.c (mon.2) 112 : audit [INF] from='client.? 172.21.15.47:0/782549266' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:44:18.756 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:17.589026+0000 mon.a (mon.0) 1356 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:44:18.757 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:17.748391+0000 mon.b (mon.1) 227 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:18.757 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:18.102577+0000 mon.a (mon.0) 1357 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:44:18.757 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:18.103625+0000 mon.a (mon.0) 1358 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:44:18.757 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:18.111478+0000 mon.a (mon.0) 1359 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82457-24"}]': finished 2024-02-23T00:44:18.757 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:18.111686+0000 mon.a (mon.0) 1360 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi047-82386-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:18.757 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:18.111849+0000 mon.a (mon.0) 1361 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:44:18.757 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:18.111910+0000 mon.a (mon.0) 1362 : cluster [DBG] osdmap e186: 8 total, 8 up, 8 in 2024-02-23T00:44:18.757 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:18.122132+0000 mon.a (mon.0) 1363 : audit [INF] from='client.? 172.21.15.47:0/1351400035' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi047-82378-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:18.757 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:17.221380+0000 mgr.y (mgr.14517) 228 : cluster [DBG] pgmap v253: 290 pgs: 258 active+clean, 32 creating+peering; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 816 B/s wr, 2 op/s 2024-02-23T00:44:18.757 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:17.239445+0000 mon.c (mon.2) 112 : audit [INF] from='client.? 172.21.15.47:0/782549266' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:44:18.757 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:17.589026+0000 mon.a (mon.0) 1356 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:44:18.758 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:17.748391+0000 mon.b (mon.1) 227 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:18.758 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:18.102577+0000 mon.a (mon.0) 1357 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:44:18.758 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:18.103625+0000 mon.a (mon.0) 1358 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:44:18.758 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:18.111478+0000 mon.a (mon.0) 1359 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82457-24"}]': finished 2024-02-23T00:44:18.758 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:18.111686+0000 mon.a (mon.0) 1360 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi047-82386-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:18.758 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:18.111849+0000 mon.a (mon.0) 1361 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:44:18.758 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:18.111910+0000 mon.a (mon.0) 1362 : cluster [DBG] osdmap e186: 8 total, 8 up, 8 in 2024-02-23T00:44:18.758 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:18.122132+0000 mon.a (mon.0) 1363 : audit [INF] from='client.? 172.21.15.47:0/1351400035' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi047-82378-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:18.774 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:44:18.816 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:44:18.818 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:17.221380+0000 mgr.y (mgr.14517) 228 : cluster [DBG] pgmap v253: 290 pgs: 258 active+clean, 32 creating+peering; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 816 B/s wr, 2 op/s 2024-02-23T00:44:18.818 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:17.239445+0000 mon.c (mon.2) 112 : audit [INF] from='client.? 172.21.15.47:0/782549266' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:44:18.818 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:17.589026+0000 mon.a (mon.0) 1356 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:44:18.818 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:17.748391+0000 mon.b (mon.1) 227 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:18.818 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:18.102577+0000 mon.a (mon.0) 1357 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:44:18.818 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:18.103625+0000 mon.a (mon.0) 1358 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:44:18.819 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:18.111478+0000 mon.a (mon.0) 1359 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82457-24"}]': finished 2024-02-23T00:44:18.819 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:18.111686+0000 mon.a (mon.0) 1360 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi047-82386-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:18.819 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:18.111849+0000 mon.a (mon.0) 1361 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:44:18.819 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:18.111910+0000 mon.a (mon.0) 1362 : cluster [DBG] osdmap e186: 8 total, 8 up, 8 in 2024-02-23T00:44:18.819 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:18.122132+0000 mon.a (mon.0) 1363 : audit [INF] from='client.? 172.21.15.47:0/1351400035' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi047-82378-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:18.819 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:18.820 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:44:18.921 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:44:18.947 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:18.966 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:44:18.984 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:18.984 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:44:19.085 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:44:19.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.095+0000 7fe749824700 1 -- 172.21.15.47:0/2301152564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7440fd510 msgr2=0x7fe7440f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:19.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.095+0000 7fe749824700 1 --2- 172.21.15.47:0/2301152564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7440fd510 0x7fe7440f6d40 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 rx=0x7fe7340047b0 tx=0x7fe734009cb0).stop 2024-02-23T00:44:19.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.095+0000 7fe749824700 1 -- 172.21.15.47:0/2301152564 shutdown_connections 2024-02-23T00:44:19.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.095+0000 7fe749824700 1 --2- 172.21.15.47:0/2301152564 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe7440f7400 0x7fe744103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:19.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.095+0000 7fe749824700 1 --2- 172.21.15.47:0/2301152564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7440fcbb0 0x7fe7440fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:19.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.095+0000 7fe749824700 1 --2- 172.21.15.47:0/2301152564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7440fd510 0x7fe7440f6d40 unknown :-1 s=CLOSED pgs=2444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:19.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.095+0000 7fe749824700 1 -- 172.21.15.47:0/2301152564 >> 172.21.15.47:0/2301152564 conn(0x7fe7440f2820 msgr2=0x7fe7440f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:19.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.095+0000 7fe749824700 1 -- 172.21.15.47:0/2301152564 shutdown_connections 2024-02-23T00:44:19.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.095+0000 7fe749824700 1 -- 172.21.15.47:0/2301152564 wait complete. 2024-02-23T00:44:19.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.096+0000 7fe749824700 1 librados: starting msgr at 2024-02-23T00:44:19.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.096+0000 7fe749824700 1 librados: starting objecter 2024-02-23T00:44:19.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.096+0000 7fe749824700 1 Processor -- start 2024-02-23T00:44:19.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.096+0000 7fe749824700 1 -- start start 2024-02-23T00:44:19.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.096+0000 7fe749824700 1 librados: setting wanted keys 2024-02-23T00:44:19.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.096+0000 7fe749824700 1 librados: calling monclient init 2024-02-23T00:44:19.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.096+0000 7fe749824700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7440f7400 0x7fe74410e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:19.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.096+0000 7fe749824700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe7440fcbb0 0x7fe74410e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:19.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.096+0000 7fe749824700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7440fd510 0x7fe744108200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:19.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.096+0000 7fe749824700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7441086f0 con 0x7fe7440fd510 2024-02-23T00:44:19.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.096+0000 7fe749824700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe744108860 con 0x7fe7440f7400 2024-02-23T00:44:19.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.096+0000 7fe749824700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe744108a20 con 0x7fe7440fcbb0 2024-02-23T00:44:19.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.097+0000 7fe7427fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe7440fcbb0 0x7fe74410e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:19.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.097+0000 7fe7427fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe7440fcbb0 0x7fe74410e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52632/0 (socket says 172.21.15.47:52632) 2024-02-23T00:44:19.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.097+0000 7fe7427fc700 1 -- 172.21.15.47:0/4214806690 learned_addr learned my addr 172.21.15.47:0/4214806690 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:44:19.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.097+0000 7fe7437fe700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7440fd510 0x7fe744108200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:19.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.097+0000 7fe742ffd700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7440f7400 0x7fe74410e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:19.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.097+0000 7fe7427fc700 1 -- 172.21.15.47:0/4214806690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7440f7400 msgr2=0x7fe74410e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:19.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.097+0000 7fe7427fc700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7440f7400 0x7fe74410e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:19.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.097+0000 7fe7427fc700 1 -- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7440fd510 msgr2=0x7fe744108200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:19.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.097+0000 7fe7427fc700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7440fd510 0x7fe744108200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:19.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.097+0000 7fe7427fc700 1 -- 172.21.15.47:0/4214806690 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe73c0097c0 con 0x7fe7440fcbb0 2024-02-23T00:44:19.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.097+0000 7fe7437fe700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7440fd510 0x7fe744108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:44:19.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.097+0000 7fe7427fc700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe7440fcbb0 0x7fe74410e550 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 rx=0x7fe734009f20 tx=0x7fe734009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:19.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.097+0000 7fe73bfff700 10 client.?.objecter ms_handle_connect 0x7fe7440fcbb0 2024-02-23T00:44:19.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.097+0000 7fe73bfff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:44:19.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.097+0000 7fe73bfff700 1 -- 172.21.15.47:0/4214806690 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe734018070 con 0x7fe7440fcbb0 2024-02-23T00:44:19.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.097+0000 7fe73bfff700 1 -- 172.21.15.47:0/4214806690 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fe734013d50 con 0x7fe7440fcbb0 2024-02-23T00:44:19.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.097+0000 7fe73bfff700 1 -- 172.21.15.47:0/4214806690 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73401d9d0 con 0x7fe7440fcbb0 2024-02-23T00:44:19.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.097+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe734005040 con 0x7fe7440fcbb0 2024-02-23T00:44:19.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.097+0000 7fe749824700 10 client.40583.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:44:19.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.097+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe744108ec0 con 0x7fe7440fcbb0 2024-02-23T00:44:19.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.097+0000 7fe749824700 1 librados: init done 2024-02-23T00:44:19.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.099+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fe7441090b0 con 0x7fe7440fcbb0 2024-02-23T00:44:19.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.099+0000 7fe73bfff700 1 -- 172.21.15.47:0/4214806690 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fe734009240 con 0x7fe7440fcbb0 2024-02-23T00:44:19.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.099+0000 7fe73bfff700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe730062cc0 0x7fe730065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:19.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.099+0000 7fe73bfff700 1 -- 172.21.15.47:0/4214806690 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(186..186 src has 48..186) v4 ==== 10970+0+0 (secure 0 0 0) 0x7fe73407eed0 con 0x7fe7440fcbb0 2024-02-23T00:44:19.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.099+0000 7fe73bfff700 10 client.40583.objecter ms_dispatch 0x7fe744001550 osd_map(186..186 src has 48..186) v4 2024-02-23T00:44:19.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.099+0000 7fe73bfff700 3 client.40583.objecter handle_osd_map got epochs [186,186] > 0 2024-02-23T00:44:19.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.099+0000 7fe73bfff700 3 client.40583.objecter handle_osd_map decoding full epoch 186 2024-02-23T00:44:19.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.099+0000 7fe742ffd700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe730062cc0 0x7fe730065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:19.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.099+0000 7fe73bfff700 20 client.40583.objecter dump_active .. 0 homeless 2024-02-23T00:44:19.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.099+0000 7fe73bfff700 1 -- 172.21.15.47:0/4214806690 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v186) v1 ==== 54+0+16 (secure 0 0 0) 0x7fe7340815a0 con 0x7fe7440fcbb0 2024-02-23T00:44:19.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.100+0000 7fe73bfff700 10 client.40583.objecter ms_handle_connect 0x7fe730062cc0 2024-02-23T00:44:19.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.100+0000 7fe742ffd700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe730062cc0 0x7fe730065180 secure :-1 s=READY pgs=2932 cs=0 l=1 rev1=1 rx=0x7fe72c009a00 tx=0x7fe72c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:19.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.100+0000 7fe749824700 10 client.40583.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:19.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.100+0000 7fe749824700 15 client.40583.objecter _session_command_op_assign -1 1 2024-02-23T00:44:19.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.100+0000 7fe749824700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fe7440ffed0 0x7fe7441002b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:19.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.100+0000 7fe749824700 20 client.40583.objecter _get_session s=0x7fe74405fa60 osd=0 3 2024-02-23T00:44:19.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.100+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe74405fa60 osd=0 3 2024-02-23T00:44:19.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.100+0000 7fe749824700 20 client.40583.objecter _get_session s=0x7fe74405fa60 osd=0 3 2024-02-23T00:44:19.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.100+0000 7fe749824700 15 client.40583.objecter _session_command_op_remove -1 1 2024-02-23T00:44:19.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.100+0000 7fe749824700 20 client.40583.objecter get_session s=0x7fe74405fa60 osd=0 3 2024-02-23T00:44:19.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.100+0000 7fe749824700 15 client.40583.objecter _session_command_op_assign 0 1 2024-02-23T00:44:19.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.100+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe74405fa60 osd=0 4 2024-02-23T00:44:19.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.100+0000 7fe749824700 10 client.40583.objecter _send_command 1 2024-02-23T00:44:19.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.100+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe74405ee80 con 0x7fe7440ffed0 2024-02-23T00:44:19.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.100+0000 7fe7437fe700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fe7440ffed0 0x7fe7441002b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:19.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.100+0000 7fe7437fe700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fe7440ffed0 0x7fe7441002b0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:19.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.100+0000 7fe73bfff700 10 client.40583.objecter ms_handle_connect 0x7fe7440ffed0 2024-02-23T00:44:19.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.101+0000 7fe73bfff700 1 -- 172.21.15.47:0/4214806690 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe74405ee80 con 0x7fe7440ffed0 2024-02-23T00:44:19.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.101+0000 7fe73bfff700 10 client.40583.objecter ms_dispatch 0x7fe744001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:44:19.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.101+0000 7fe73bfff700 10 client.40583.objecter _finish_command 1 = system:0 2024-02-23T00:44:19.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.101+0000 7fe73bfff700 20 client.40583.objecter put_session s=0x7fe74405fa60 osd=0 4 2024-02-23T00:44:19.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.101+0000 7fe73bfff700 15 client.40583.objecter _session_command_op_remove 0 1 2024-02-23T00:44:19.112 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:19.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.123+0000 7fe749824700 10 client.40583.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:44:19.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.123+0000 7fe749824700 15 client.40583.objecter _session_command_op_assign -1 2 2024-02-23T00:44:19.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.123+0000 7fe749824700 20 client.40583.objecter _get_session s=0x7fe74405fa60 osd=0 3 2024-02-23T00:44:19.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.123+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe74405fa60 osd=0 3 2024-02-23T00:44:19.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.123+0000 7fe749824700 20 client.40583.objecter _get_session s=0x7fe74405fa60 osd=0 3 2024-02-23T00:44:19.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.123+0000 7fe749824700 15 client.40583.objecter _session_command_op_remove -1 2 2024-02-23T00:44:19.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.123+0000 7fe749824700 20 client.40583.objecter get_session s=0x7fe74405fa60 osd=0 3 2024-02-23T00:44:19.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.123+0000 7fe749824700 15 client.40583.objecter _session_command_op_assign 0 2 2024-02-23T00:44:19.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.123+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe74405fa60 osd=0 4 2024-02-23T00:44:19.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.123+0000 7fe749824700 10 client.40583.objecter _send_command 2 2024-02-23T00:44:19.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.123+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fe744105ee0 con 0x7fe7440ffed0 2024-02-23T00:44:19.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.139+0000 7fe73bfff700 1 -- 172.21.15.47:0/4214806690 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fe744105ee0 con 0x7fe7440ffed0 2024-02-23T00:44:19.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.139+0000 7fe73bfff700 10 client.40583.objecter ms_dispatch 0x7fe744001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:44:19.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.139+0000 7fe73bfff700 10 client.40583.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:44:19.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.139+0000 7fe73bfff700 20 client.40583.objecter put_session s=0x7fe74405fa60 osd=0 4 2024-02-23T00:44:19.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.139+0000 7fe73bfff700 15 client.40583.objecter _session_command_op_remove 0 2 2024-02-23T00:44:19.141 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:44:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.139+0000 7fe749824700 10 client.40583.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.139+0000 7fe749824700 15 client.40583.objecter _session_command_op_assign -1 3 2024-02-23T00:44:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.139+0000 7fe749824700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fe7440598f0 0x7fe744059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.139+0000 7fe749824700 20 client.40583.objecter _get_session s=0x7fe744028540 osd=1 3 2024-02-23T00:44:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.139+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe744028540 osd=1 3 2024-02-23T00:44:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.139+0000 7fe749824700 20 client.40583.objecter _get_session s=0x7fe744028540 osd=1 3 2024-02-23T00:44:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.139+0000 7fe749824700 15 client.40583.objecter _session_command_op_remove -1 3 2024-02-23T00:44:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.139+0000 7fe749824700 20 client.40583.objecter get_session s=0x7fe744028540 osd=1 3 2024-02-23T00:44:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.139+0000 7fe749824700 15 client.40583.objecter _session_command_op_assign 1 3 2024-02-23T00:44:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.139+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe744028540 osd=1 4 2024-02-23T00:44:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.139+0000 7fe749824700 10 client.40583.objecter _send_command 3 2024-02-23T00:44:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.139+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe7440286b0 con 0x7fe7440598f0 2024-02-23T00:44:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.139+0000 7fe742ffd700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fe7440598f0 0x7fe744059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:19.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.141+0000 7fe742ffd700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fe7440598f0 0x7fe744059cd0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:19.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.141+0000 7fe73bfff700 10 client.40583.objecter ms_handle_connect 0x7fe7440598f0 2024-02-23T00:44:19.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.144+0000 7fe73bfff700 1 -- 172.21.15.47:0/4214806690 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe7440286b0 con 0x7fe7440598f0 2024-02-23T00:44:19.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.144+0000 7fe73bfff700 10 client.40583.objecter ms_dispatch 0x7fe744001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:44:19.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.144+0000 7fe73bfff700 10 client.40583.objecter _finish_command 3 = system:0 2024-02-23T00:44:19.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.144+0000 7fe73bfff700 20 client.40583.objecter put_session s=0x7fe744028540 osd=1 4 2024-02-23T00:44:19.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.144+0000 7fe73bfff700 15 client.40583.objecter _session_command_op_remove 1 3 2024-02-23T00:44:19.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.165+0000 7fe749824700 10 client.40583.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:44:19.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.165+0000 7fe749824700 15 client.40583.objecter _session_command_op_assign -1 4 2024-02-23T00:44:19.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.165+0000 7fe749824700 20 client.40583.objecter _get_session s=0x7fe744028540 osd=1 3 2024-02-23T00:44:19.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.165+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe744028540 osd=1 3 2024-02-23T00:44:19.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.165+0000 7fe749824700 20 client.40583.objecter _get_session s=0x7fe744028540 osd=1 3 2024-02-23T00:44:19.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.165+0000 7fe749824700 15 client.40583.objecter _session_command_op_remove -1 4 2024-02-23T00:44:19.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.165+0000 7fe749824700 20 client.40583.objecter get_session s=0x7fe744028540 osd=1 3 2024-02-23T00:44:19.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.165+0000 7fe749824700 15 client.40583.objecter _session_command_op_assign 1 4 2024-02-23T00:44:19.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.165+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe744028540 osd=1 4 2024-02-23T00:44:19.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.165+0000 7fe749824700 10 client.40583.objecter _send_command 4 2024-02-23T00:44:19.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.165+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fe744059360 con 0x7fe7440598f0 2024-02-23T00:44:19.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.167+0000 7fe73bfff700 1 -- 172.21.15.47:0/4214806690 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fe744059360 con 0x7fe7440598f0 2024-02-23T00:44:19.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.167+0000 7fe73bfff700 10 client.40583.objecter ms_dispatch 0x7fe744001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:44:19.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.167+0000 7fe73bfff700 10 client.40583.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:44:19.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.167+0000 7fe73bfff700 20 client.40583.objecter put_session s=0x7fe744028540 osd=1 4 2024-02-23T00:44:19.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.167+0000 7fe73bfff700 15 client.40583.objecter _session_command_op_remove 1 4 2024-02-23T00:44:19.169 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:44:19.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.167+0000 7fe749824700 10 client.40583.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:19.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.167+0000 7fe749824700 15 client.40583.objecter _session_command_op_assign -1 5 2024-02-23T00:44:19.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.167+0000 7fe749824700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fe74411a030 0x7fe74411c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:19.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.167+0000 7fe749824700 20 client.40583.objecter _get_session s=0x7fe744027b90 osd=2 3 2024-02-23T00:44:19.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.167+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe744027b90 osd=2 3 2024-02-23T00:44:19.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.167+0000 7fe749824700 20 client.40583.objecter _get_session s=0x7fe744027b90 osd=2 3 2024-02-23T00:44:19.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.167+0000 7fe749824700 15 client.40583.objecter _session_command_op_remove -1 5 2024-02-23T00:44:19.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.167+0000 7fe749824700 20 client.40583.objecter get_session s=0x7fe744027b90 osd=2 3 2024-02-23T00:44:19.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.167+0000 7fe749824700 15 client.40583.objecter _session_command_op_assign 2 5 2024-02-23T00:44:19.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.167+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe744027b90 osd=2 4 2024-02-23T00:44:19.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.167+0000 7fe749824700 10 client.40583.objecter _send_command 5 2024-02-23T00:44:19.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.167+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe744028210 con 0x7fe74411a030 2024-02-23T00:44:19.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.167+0000 7fe7427fc700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fe74411a030 0x7fe74411c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:19.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.167+0000 7fe7427fc700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fe74411a030 0x7fe74411c420 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:19.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.168+0000 7fe73bfff700 10 client.40583.objecter ms_handle_connect 0x7fe74411a030 2024-02-23T00:44:19.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.168+0000 7fe73bfff700 1 -- 172.21.15.47:0/4214806690 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe744028210 con 0x7fe74411a030 2024-02-23T00:44:19.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.168+0000 7fe73bfff700 10 client.40583.objecter ms_dispatch 0x7fe744001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:44:19.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.168+0000 7fe73bfff700 10 client.40583.objecter _finish_command 5 = system:0 2024-02-23T00:44:19.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.168+0000 7fe73bfff700 20 client.40583.objecter put_session s=0x7fe744027b90 osd=2 4 2024-02-23T00:44:19.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.168+0000 7fe73bfff700 15 client.40583.objecter _session_command_op_remove 2 5 2024-02-23T00:44:19.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.183+0000 7fe749824700 10 client.40583.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:44:19.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.183+0000 7fe749824700 15 client.40583.objecter _session_command_op_assign -1 6 2024-02-23T00:44:19.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.183+0000 7fe749824700 20 client.40583.objecter _get_session s=0x7fe744027b90 osd=2 3 2024-02-23T00:44:19.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.183+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe744027b90 osd=2 3 2024-02-23T00:44:19.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.183+0000 7fe749824700 20 client.40583.objecter _get_session s=0x7fe744027b90 osd=2 3 2024-02-23T00:44:19.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.183+0000 7fe749824700 15 client.40583.objecter _session_command_op_remove -1 6 2024-02-23T00:44:19.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.183+0000 7fe749824700 20 client.40583.objecter get_session s=0x7fe744027b90 osd=2 3 2024-02-23T00:44:19.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.183+0000 7fe749824700 15 client.40583.objecter _session_command_op_assign 2 6 2024-02-23T00:44:19.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.183+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe744027b90 osd=2 4 2024-02-23T00:44:19.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.183+0000 7fe749824700 10 client.40583.objecter _send_command 6 2024-02-23T00:44:19.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.183+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fe744027600 con 0x7fe74411a030 2024-02-23T00:44:19.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.184+0000 7fe73bfff700 1 -- 172.21.15.47:0/4214806690 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fe744027600 con 0x7fe74411a030 2024-02-23T00:44:19.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.184+0000 7fe73bfff700 10 client.40583.objecter ms_dispatch 0x7fe744001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:44:19.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.184+0000 7fe73bfff700 10 client.40583.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:44:19.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.184+0000 7fe73bfff700 20 client.40583.objecter put_session s=0x7fe744027b90 osd=2 4 2024-02-23T00:44:19.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.184+0000 7fe73bfff700 15 client.40583.objecter _session_command_op_remove 2 6 2024-02-23T00:44:19.186 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:44:19.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.185+0000 7fe749824700 10 client.40583.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:19.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.185+0000 7fe749824700 15 client.40583.objecter _session_command_op_assign -1 7 2024-02-23T00:44:19.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.185+0000 7fe749824700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fe744119b90 0x7fe744121ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:19.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.185+0000 7fe749824700 20 client.40583.objecter _get_session s=0x7fe7440279b0 osd=3 3 2024-02-23T00:44:19.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.185+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe7440279b0 osd=3 3 2024-02-23T00:44:19.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.185+0000 7fe749824700 20 client.40583.objecter _get_session s=0x7fe7440279b0 osd=3 3 2024-02-23T00:44:19.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.185+0000 7fe749824700 15 client.40583.objecter _session_command_op_remove -1 7 2024-02-23T00:44:19.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.185+0000 7fe749824700 20 client.40583.objecter get_session s=0x7fe7440279b0 osd=3 3 2024-02-23T00:44:19.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.185+0000 7fe749824700 15 client.40583.objecter _session_command_op_assign 3 7 2024-02-23T00:44:19.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.185+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe7440279b0 osd=3 4 2024-02-23T00:44:19.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.185+0000 7fe749824700 10 client.40583.objecter _send_command 7 2024-02-23T00:44:19.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.185+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe7440fa870 con 0x7fe744119b90 2024-02-23T00:44:19.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.185+0000 7fe7437fe700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fe744119b90 0x7fe744121ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:19.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.185+0000 7fe73bfff700 10 client.40583.objecter ms_handle_connect 0x7fe744119b90 2024-02-23T00:44:19.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.185+0000 7fe7437fe700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fe744119b90 0x7fe744121ea0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:19.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.186+0000 7fe73bfff700 1 -- 172.21.15.47:0/4214806690 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe7440fa870 con 0x7fe744119b90 2024-02-23T00:44:19.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.186+0000 7fe73bfff700 10 client.40583.objecter ms_dispatch 0x7fe744001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:44:19.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.186+0000 7fe73bfff700 10 client.40583.objecter _finish_command 7 = system:0 2024-02-23T00:44:19.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.186+0000 7fe73bfff700 20 client.40583.objecter put_session s=0x7fe7440279b0 osd=3 4 2024-02-23T00:44:19.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.186+0000 7fe73bfff700 15 client.40583.objecter _session_command_op_remove 3 7 2024-02-23T00:44:19.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.200+0000 7fe749824700 10 client.40583.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:44:19.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.200+0000 7fe749824700 15 client.40583.objecter _session_command_op_assign -1 8 2024-02-23T00:44:19.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.200+0000 7fe749824700 20 client.40583.objecter _get_session s=0x7fe7440279b0 osd=3 3 2024-02-23T00:44:19.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.200+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe7440279b0 osd=3 3 2024-02-23T00:44:19.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.200+0000 7fe749824700 20 client.40583.objecter _get_session s=0x7fe7440279b0 osd=3 3 2024-02-23T00:44:19.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.200+0000 7fe749824700 15 client.40583.objecter _session_command_op_remove -1 8 2024-02-23T00:44:19.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.200+0000 7fe749824700 20 client.40583.objecter get_session s=0x7fe7440279b0 osd=3 3 2024-02-23T00:44:19.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.200+0000 7fe749824700 15 client.40583.objecter _session_command_op_assign 3 8 2024-02-23T00:44:19.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.200+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe7440279b0 osd=3 4 2024-02-23T00:44:19.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.200+0000 7fe749824700 10 client.40583.objecter _send_command 8 2024-02-23T00:44:19.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.200+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fe7441190f0 con 0x7fe744119b90 2024-02-23T00:44:19.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.202+0000 7fe73bfff700 1 -- 172.21.15.47:0/4214806690 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fe7441190f0 con 0x7fe744119b90 2024-02-23T00:44:19.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.202+0000 7fe73bfff700 10 client.40583.objecter ms_dispatch 0x7fe744001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:44:19.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.202+0000 7fe73bfff700 10 client.40583.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:44:19.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.202+0000 7fe73bfff700 20 client.40583.objecter put_session s=0x7fe7440279b0 osd=3 4 2024-02-23T00:44:19.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.202+0000 7fe73bfff700 15 client.40583.objecter _session_command_op_remove 3 8 2024-02-23T00:44:19.204 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:44:19.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.202+0000 7fe749824700 10 client.40583.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:19.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.202+0000 7fe749824700 15 client.40583.objecter _session_command_op_assign -1 9 2024-02-23T00:44:19.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.202+0000 7fe749824700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fe744119260 0x7fe744123510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:19.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.202+0000 7fe749824700 20 client.40583.objecter _get_session s=0x7fe7441190f0 osd=4 3 2024-02-23T00:44:19.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.202+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe7441190f0 osd=4 3 2024-02-23T00:44:19.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.202+0000 7fe749824700 20 client.40583.objecter _get_session s=0x7fe7441190f0 osd=4 3 2024-02-23T00:44:19.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.202+0000 7fe749824700 15 client.40583.objecter _session_command_op_remove -1 9 2024-02-23T00:44:19.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.202+0000 7fe749824700 20 client.40583.objecter get_session s=0x7fe7441190f0 osd=4 3 2024-02-23T00:44:19.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.202+0000 7fe749824700 15 client.40583.objecter _session_command_op_assign 4 9 2024-02-23T00:44:19.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.202+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe7441190f0 osd=4 4 2024-02-23T00:44:19.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.202+0000 7fe749824700 10 client.40583.objecter _send_command 9 2024-02-23T00:44:19.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.202+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe74411d7e0 con 0x7fe744119260 2024-02-23T00:44:19.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.202+0000 7fe742ffd700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fe744119260 0x7fe744123510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:19.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.203+0000 7fe73bfff700 10 client.40583.objecter ms_handle_connect 0x7fe744119260 2024-02-23T00:44:19.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.203+0000 7fe742ffd700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fe744119260 0x7fe744123510 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:19.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.203+0000 7fe73bfff700 1 -- 172.21.15.47:0/4214806690 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe74411d7e0 con 0x7fe744119260 2024-02-23T00:44:19.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.203+0000 7fe73bfff700 10 client.40583.objecter ms_dispatch 0x7fe744001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:44:19.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.203+0000 7fe73bfff700 10 client.40583.objecter _finish_command 9 = system:0 2024-02-23T00:44:19.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.203+0000 7fe73bfff700 20 client.40583.objecter put_session s=0x7fe7441190f0 osd=4 4 2024-02-23T00:44:19.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.203+0000 7fe73bfff700 15 client.40583.objecter _session_command_op_remove 4 9 2024-02-23T00:44:19.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.218+0000 7fe749824700 10 client.40583.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:44:19.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.218+0000 7fe749824700 15 client.40583.objecter _session_command_op_assign -1 10 2024-02-23T00:44:19.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.218+0000 7fe749824700 20 client.40583.objecter _get_session s=0x7fe7441190f0 osd=4 3 2024-02-23T00:44:19.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.218+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe7441190f0 osd=4 3 2024-02-23T00:44:19.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.218+0000 7fe749824700 20 client.40583.objecter _get_session s=0x7fe7441190f0 osd=4 3 2024-02-23T00:44:19.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.218+0000 7fe749824700 15 client.40583.objecter _session_command_op_remove -1 10 2024-02-23T00:44:19.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.218+0000 7fe749824700 20 client.40583.objecter get_session s=0x7fe7441190f0 osd=4 3 2024-02-23T00:44:19.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.218+0000 7fe749824700 15 client.40583.objecter _session_command_op_assign 4 10 2024-02-23T00:44:19.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.218+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe7441190f0 osd=4 4 2024-02-23T00:44:19.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.218+0000 7fe749824700 10 client.40583.objecter _send_command 10 2024-02-23T00:44:19.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.218+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fe74411d7e0 con 0x7fe744119260 2024-02-23T00:44:19.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.220+0000 7fe73bfff700 1 -- 172.21.15.47:0/4214806690 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fe74411d7e0 con 0x7fe744119260 2024-02-23T00:44:19.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.220+0000 7fe73bfff700 10 client.40583.objecter ms_dispatch 0x7fe744001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:44:19.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.220+0000 7fe73bfff700 10 client.40583.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:44:19.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.220+0000 7fe73bfff700 20 client.40583.objecter put_session s=0x7fe7441190f0 osd=4 4 2024-02-23T00:44:19.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.220+0000 7fe73bfff700 15 client.40583.objecter _session_command_op_remove 4 10 2024-02-23T00:44:19.222 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:44:19.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.220+0000 7fe749824700 10 client.40583.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:19.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.220+0000 7fe749824700 15 client.40583.objecter _session_command_op_assign -1 11 2024-02-23T00:44:19.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.220+0000 7fe749824700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fe744123f10 0x7fe744124370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:19.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.220+0000 7fe749824700 20 client.40583.objecter _get_session s=0x7fe74411d7e0 osd=5 3 2024-02-23T00:44:19.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.220+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe74411d7e0 osd=5 3 2024-02-23T00:44:19.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.220+0000 7fe749824700 20 client.40583.objecter _get_session s=0x7fe74411d7e0 osd=5 3 2024-02-23T00:44:19.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.220+0000 7fe749824700 15 client.40583.objecter _session_command_op_remove -1 11 2024-02-23T00:44:19.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.220+0000 7fe749824700 20 client.40583.objecter get_session s=0x7fe74411d7e0 osd=5 3 2024-02-23T00:44:19.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.220+0000 7fe749824700 15 client.40583.objecter _session_command_op_assign 5 11 2024-02-23T00:44:19.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.220+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe74411d7e0 osd=5 4 2024-02-23T00:44:19.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.220+0000 7fe749824700 10 client.40583.objecter _send_command 11 2024-02-23T00:44:19.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.220+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe74412b2d0 con 0x7fe744123f10 2024-02-23T00:44:19.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.220+0000 7fe7427fc700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fe744123f10 0x7fe744124370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:19.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.221+0000 7fe73bfff700 10 client.40583.objecter ms_handle_connect 0x7fe744123f10 2024-02-23T00:44:19.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.221+0000 7fe7427fc700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fe744123f10 0x7fe744124370 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:19.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.221+0000 7fe73bfff700 1 -- 172.21.15.47:0/4214806690 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe74412b2d0 con 0x7fe744123f10 2024-02-23T00:44:19.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.221+0000 7fe73bfff700 10 client.40583.objecter ms_dispatch 0x7fe744001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:44:19.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.221+0000 7fe73bfff700 10 client.40583.objecter _finish_command 11 = system:0 2024-02-23T00:44:19.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.221+0000 7fe73bfff700 20 client.40583.objecter put_session s=0x7fe74411d7e0 osd=5 4 2024-02-23T00:44:19.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.221+0000 7fe73bfff700 15 client.40583.objecter _session_command_op_remove 5 11 2024-02-23T00:44:19.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.236+0000 7fe749824700 10 client.40583.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:44:19.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.236+0000 7fe749824700 15 client.40583.objecter _session_command_op_assign -1 12 2024-02-23T00:44:19.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.236+0000 7fe749824700 20 client.40583.objecter _get_session s=0x7fe74411d7e0 osd=5 3 2024-02-23T00:44:19.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.236+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe74411d7e0 osd=5 3 2024-02-23T00:44:19.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.236+0000 7fe749824700 20 client.40583.objecter _get_session s=0x7fe74411d7e0 osd=5 3 2024-02-23T00:44:19.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.236+0000 7fe749824700 15 client.40583.objecter _session_command_op_remove -1 12 2024-02-23T00:44:19.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.236+0000 7fe749824700 20 client.40583.objecter get_session s=0x7fe74411d7e0 osd=5 3 2024-02-23T00:44:19.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.236+0000 7fe749824700 15 client.40583.objecter _session_command_op_assign 5 12 2024-02-23T00:44:19.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.236+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe74411d7e0 osd=5 4 2024-02-23T00:44:19.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.236+0000 7fe749824700 10 client.40583.objecter _send_command 12 2024-02-23T00:44:19.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.236+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fe744128b70 con 0x7fe744123f10 2024-02-23T00:44:19.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.238+0000 7fe73bfff700 1 -- 172.21.15.47:0/4214806690 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fe744128b70 con 0x7fe744123f10 2024-02-23T00:44:19.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.238+0000 7fe73bfff700 10 client.40583.objecter ms_dispatch 0x7fe744001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:44:19.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.238+0000 7fe73bfff700 10 client.40583.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:44:19.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.238+0000 7fe73bfff700 20 client.40583.objecter put_session s=0x7fe74411d7e0 osd=5 4 2024-02-23T00:44:19.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.238+0000 7fe73bfff700 15 client.40583.objecter _session_command_op_remove 5 12 2024-02-23T00:44:19.240 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:44:19.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.238+0000 7fe749824700 10 client.40583.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.238+0000 7fe749824700 15 client.40583.objecter _session_command_op_assign -1 13 2024-02-23T00:44:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.238+0000 7fe749824700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fe7441291f0 0x7fe74412b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.238+0000 7fe749824700 20 client.40583.objecter _get_session s=0x7fe744128b70 osd=6 3 2024-02-23T00:44:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.238+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe744128b70 osd=6 3 2024-02-23T00:44:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.238+0000 7fe749824700 20 client.40583.objecter _get_session s=0x7fe744128b70 osd=6 3 2024-02-23T00:44:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.238+0000 7fe749824700 15 client.40583.objecter _session_command_op_remove -1 13 2024-02-23T00:44:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.238+0000 7fe749824700 20 client.40583.objecter get_session s=0x7fe744128b70 osd=6 3 2024-02-23T00:44:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.238+0000 7fe749824700 15 client.40583.objecter _session_command_op_assign 6 13 2024-02-23T00:44:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.238+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe744128b70 osd=6 4 2024-02-23T00:44:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.238+0000 7fe749824700 10 client.40583.objecter _send_command 13 2024-02-23T00:44:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.238+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe74412e420 con 0x7fe7441291f0 2024-02-23T00:44:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.238+0000 7fe7437fe700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fe7441291f0 0x7fe74412b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:19.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.238+0000 7fe73bfff700 10 client.40583.objecter ms_handle_connect 0x7fe7441291f0 2024-02-23T00:44:19.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.238+0000 7fe7437fe700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fe7441291f0 0x7fe74412b5e0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:19.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.239+0000 7fe73bfff700 1 -- 172.21.15.47:0/4214806690 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe74412e420 con 0x7fe7441291f0 2024-02-23T00:44:19.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.239+0000 7fe73bfff700 10 client.40583.objecter ms_dispatch 0x7fe744001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:44:19.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.239+0000 7fe73bfff700 10 client.40583.objecter _finish_command 13 = system:0 2024-02-23T00:44:19.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.239+0000 7fe73bfff700 20 client.40583.objecter put_session s=0x7fe744128b70 osd=6 4 2024-02-23T00:44:19.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.239+0000 7fe73bfff700 15 client.40583.objecter _session_command_op_remove 6 13 2024-02-23T00:44:19.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.255+0000 7fe749824700 10 client.40583.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:44:19.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.255+0000 7fe749824700 15 client.40583.objecter _session_command_op_assign -1 14 2024-02-23T00:44:19.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.255+0000 7fe749824700 20 client.40583.objecter _get_session s=0x7fe744128b70 osd=6 3 2024-02-23T00:44:19.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.255+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe744128b70 osd=6 3 2024-02-23T00:44:19.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.255+0000 7fe749824700 20 client.40583.objecter _get_session s=0x7fe744128b70 osd=6 3 2024-02-23T00:44:19.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.255+0000 7fe749824700 15 client.40583.objecter _session_command_op_remove -1 14 2024-02-23T00:44:19.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.255+0000 7fe749824700 20 client.40583.objecter get_session s=0x7fe744128b70 osd=6 3 2024-02-23T00:44:19.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.255+0000 7fe749824700 15 client.40583.objecter _session_command_op_assign 6 14 2024-02-23T00:44:19.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.255+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe744128b70 osd=6 4 2024-02-23T00:44:19.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.255+0000 7fe749824700 10 client.40583.objecter _send_command 14 2024-02-23T00:44:19.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.255+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fe74412e420 con 0x7fe7441291f0 2024-02-23T00:44:19.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.256+0000 7fe73bfff700 1 -- 172.21.15.47:0/4214806690 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fe74412e420 con 0x7fe7441291f0 2024-02-23T00:44:19.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.256+0000 7fe73bfff700 10 client.40583.objecter ms_dispatch 0x7fe744001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:44:19.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.256+0000 7fe73bfff700 10 client.40583.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:44:19.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.256+0000 7fe73bfff700 20 client.40583.objecter put_session s=0x7fe744128b70 osd=6 4 2024-02-23T00:44:19.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.256+0000 7fe73bfff700 15 client.40583.objecter _session_command_op_remove 6 14 2024-02-23T00:44:19.259 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:44:19.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.257+0000 7fe749824700 10 client.40583.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:19.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.257+0000 7fe749824700 15 client.40583.objecter _session_command_op_assign -1 15 2024-02-23T00:44:19.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.257+0000 7fe749824700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fe74412d700 0x7fe74412db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:19.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.257+0000 7fe749824700 20 client.40583.objecter _get_session s=0x7fe74412e420 osd=7 3 2024-02-23T00:44:19.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.257+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe74412e420 osd=7 3 2024-02-23T00:44:19.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.257+0000 7fe749824700 20 client.40583.objecter _get_session s=0x7fe74412e420 osd=7 3 2024-02-23T00:44:19.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.257+0000 7fe749824700 15 client.40583.objecter _session_command_op_remove -1 15 2024-02-23T00:44:19.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.257+0000 7fe749824700 20 client.40583.objecter get_session s=0x7fe74412e420 osd=7 3 2024-02-23T00:44:19.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.257+0000 7fe749824700 15 client.40583.objecter _session_command_op_assign 7 15 2024-02-23T00:44:19.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.257+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe74412e420 osd=7 4 2024-02-23T00:44:19.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.257+0000 7fe749824700 10 client.40583.objecter _send_command 15 2024-02-23T00:44:19.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.257+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe744131750 con 0x7fe74412d700 2024-02-23T00:44:19.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.257+0000 7fe742ffd700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fe74412d700 0x7fe74412db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:19.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.257+0000 7fe73bfff700 10 client.40583.objecter ms_handle_connect 0x7fe74412d700 2024-02-23T00:44:19.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.257+0000 7fe742ffd700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fe74412d700 0x7fe74412db80 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:19.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.258+0000 7fe73bfff700 1 -- 172.21.15.47:0/4214806690 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe744131750 con 0x7fe74412d700 2024-02-23T00:44:19.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.258+0000 7fe73bfff700 10 client.40583.objecter ms_dispatch 0x7fe744001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:44:19.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.258+0000 7fe73bfff700 10 client.40583.objecter _finish_command 15 = system:0 2024-02-23T00:44:19.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.258+0000 7fe73bfff700 20 client.40583.objecter put_session s=0x7fe74412e420 osd=7 4 2024-02-23T00:44:19.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.258+0000 7fe73bfff700 15 client.40583.objecter _session_command_op_remove 7 15 2024-02-23T00:44:19.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.274+0000 7fe749824700 10 client.40583.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:44:19.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.274+0000 7fe749824700 15 client.40583.objecter _session_command_op_assign -1 16 2024-02-23T00:44:19.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.274+0000 7fe749824700 20 client.40583.objecter _get_session s=0x7fe74412e420 osd=7 3 2024-02-23T00:44:19.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.274+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe74412e420 osd=7 3 2024-02-23T00:44:19.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.274+0000 7fe749824700 20 client.40583.objecter _get_session s=0x7fe74412e420 osd=7 3 2024-02-23T00:44:19.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.274+0000 7fe749824700 15 client.40583.objecter _session_command_op_remove -1 16 2024-02-23T00:44:19.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.274+0000 7fe749824700 20 client.40583.objecter get_session s=0x7fe74412e420 osd=7 3 2024-02-23T00:44:19.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.274+0000 7fe749824700 15 client.40583.objecter _session_command_op_assign 7 16 2024-02-23T00:44:19.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.274+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe74412e420 osd=7 4 2024-02-23T00:44:19.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.274+0000 7fe749824700 10 client.40583.objecter _send_command 16 2024-02-23T00:44:19.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.274+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fe74412bbe0 con 0x7fe74412d700 2024-02-23T00:44:19.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe73bfff700 1 -- 172.21.15.47:0/4214806690 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fe74412bbe0 con 0x7fe74412d700 2024-02-23T00:44:19.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe73bfff700 10 client.40583.objecter ms_dispatch 0x7fe744001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:44:19.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe73bfff700 10 client.40583.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:44:19.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe73bfff700 20 client.40583.objecter put_session s=0x7fe74412e420 osd=7 4 2024-02-23T00:44:19.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe73bfff700 15 client.40583.objecter _session_command_op_remove 7 16 2024-02-23T00:44:19.278 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:44:19.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 10 librados: watch_flush enter 2024-02-23T00:44:19.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 10 librados: watch_flush exit 2024-02-23T00:44:19.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 10 client.40583.objecter close_session for osd.0 2024-02-23T00:44:19.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fe7440ffed0 msgr2=0x7fe7441002b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:19.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fe7440ffed0 0x7fe7441002b0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:19.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe74405fa60 osd=0 1 2024-02-23T00:44:19.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 10 client.40583.objecter close_session for osd.1 2024-02-23T00:44:19.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fe7440598f0 msgr2=0x7fe744059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:19.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fe7440598f0 0x7fe744059cd0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:19.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe744028540 osd=1 1 2024-02-23T00:44:19.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 10 client.40583.objecter close_session for osd.2 2024-02-23T00:44:19.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fe74411a030 msgr2=0x7fe74411c420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:19.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fe74411a030 0x7fe74411c420 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:19.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe744027b90 osd=2 1 2024-02-23T00:44:19.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 10 client.40583.objecter close_session for osd.3 2024-02-23T00:44:19.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fe744119b90 msgr2=0x7fe744121ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:19.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fe744119b90 0x7fe744121ea0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:19.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe7440279b0 osd=3 1 2024-02-23T00:44:19.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 10 client.40583.objecter close_session for osd.4 2024-02-23T00:44:19.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fe744119260 msgr2=0x7fe744123510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:19.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fe744119260 0x7fe744123510 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:19.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe7441190f0 osd=4 1 2024-02-23T00:44:19.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 10 client.40583.objecter close_session for osd.5 2024-02-23T00:44:19.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fe744123f10 msgr2=0x7fe744124370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:19.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fe744123f10 0x7fe744124370 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:19.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe74411d7e0 osd=5 1 2024-02-23T00:44:19.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 10 client.40583.objecter close_session for osd.6 2024-02-23T00:44:19.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fe7441291f0 msgr2=0x7fe74412b5e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:19.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fe7441291f0 0x7fe74412b5e0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:19.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe744128b70 osd=6 1 2024-02-23T00:44:19.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 10 client.40583.objecter close_session for osd.7 2024-02-23T00:44:19.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fe74412d700 msgr2=0x7fe74412db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:19.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fe74412d700 0x7fe74412db80 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:19.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 20 client.40583.objecter put_session s=0x7fe74412e420 osd=7 1 2024-02-23T00:44:19.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 20 client.40583.objecter shutdown clearing up homeless session... 2024-02-23T00:44:19.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 10 client.40583.objecter successfully canceled tick 2024-02-23T00:44:19.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe7437fe700 1 -- 172.21.15.47:0/4214806690 reap_dead start 2024-02-23T00:44:19.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe7437fe700 1 -- 172.21.15.47:0/4214806690 reap_dead start 2024-02-23T00:44:19.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe7437fe700 1 -- 172.21.15.47:0/4214806690 reap_dead start 2024-02-23T00:44:19.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe7437fe700 1 -- 172.21.15.47:0/4214806690 reap_dead start 2024-02-23T00:44:19.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe7437fe700 1 -- 172.21.15.47:0/4214806690 reap_dead start 2024-02-23T00:44:19.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe730062cc0 msgr2=0x7fe730065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:19.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe730062cc0 0x7fe730065180 secure :-1 s=READY pgs=2932 cs=0 l=1 rev1=1 rx=0x7fe72c009a00 tx=0x7fe72c00b040).stop 2024-02-23T00:44:19.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe7440fcbb0 msgr2=0x7fe74410e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:19.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe7440fcbb0 0x7fe74410e550 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 rx=0x7fe734009f20 tx=0x7fe734009cb0).stop 2024-02-23T00:44:19.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 shutdown_connections 2024-02-23T00:44:19.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe7440fcbb0 0x7fe74410e550 unknown :-1 s=CLOSED pgs=2506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:19.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 1 --2- 172.21.15.47:0/4214806690 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe730062cc0 0x7fe730065180 unknown :-1 s=CLOSED pgs=2932 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:19.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.276+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 >> 172.21.15.47:0/4214806690 conn(0x7fe7440f2820 msgr2=0x7fe744060e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:19.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.277+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 shutdown_connections 2024-02-23T00:44:19.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.277+0000 7fe749824700 1 -- 172.21.15.47:0/4214806690 wait complete. 2024-02-23T00:44:19.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:19.277+0000 7fe749824700 1 librados: shutdown 2024-02-23T00:44:19.284 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:44:19.284 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:44:19.284 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:44:19.284 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:44:19.284 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:44:19.284 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:44:19.285 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:44:19.285 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:44:19.326 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:44:19.332 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:19.332 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:44:19.433 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:44:19.459 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:19.532 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:44:19.535 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:19.536 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:44:19.542 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.6 dump_blocked_ops 2024-02-23T00:44:19.637 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:44:19.667 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:19.845 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:44:19.888 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:17.360019+0000 osd.6 (osd.6) 83 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:44:19.888 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:17.360683+0000 osd.6 (osd.6) 84 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:44:19.888 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:18.629018+0000 osd.0 (osd.0) 93 : cluster [DBG] 4.6 scrub starts 2024-02-23T00:44:19.888 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:18.630359+0000 osd.0 (osd.0) 94 : cluster [DBG] 4.6 scrub ok 2024-02-23T00:44:19.889 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:18.749334+0000 mon.b (mon.1) 228 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:19.889 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:19.099544+0000 mon.c (mon.2) 113 : audit [DBG] from='client.? 172.21.15.47:0/4214806690' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:44:19.889 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:19.112933+0000 mon.a (mon.0) 1364 : audit [INF] from='client.? 172.21.15.47:0/1351400035' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi047-82378-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:19.889 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:19.113103+0000 mon.a (mon.0) 1365 : cluster [DBG] osdmap e187: 8 total, 8 up, 8 in 2024-02-23T00:44:19.889 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:19.126390+0000 mon.b (mon.1) 229 : audit [INF] from='client.? 172.21.15.47:0/2650583472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi047-82457-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:19.889 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:19.133890+0000 mon.a (mon.0) 1366 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi047-82457-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:19.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:17.360019+0000 osd.6 (osd.6) 83 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:44:19.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:17.360683+0000 osd.6 (osd.6) 84 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:44:19.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:18.629018+0000 osd.0 (osd.0) 93 : cluster [DBG] 4.6 scrub starts 2024-02-23T00:44:19.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:18.630359+0000 osd.0 (osd.0) 94 : cluster [DBG] 4.6 scrub ok 2024-02-23T00:44:19.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:18.749334+0000 mon.b (mon.1) 228 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:19.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:19.099544+0000 mon.c (mon.2) 113 : audit [DBG] from='client.? 172.21.15.47:0/4214806690' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:44:19.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:19.112933+0000 mon.a (mon.0) 1364 : audit [INF] from='client.? 172.21.15.47:0/1351400035' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi047-82378-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:19.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:19.113103+0000 mon.a (mon.0) 1365 : cluster [DBG] osdmap e187: 8 total, 8 up, 8 in 2024-02-23T00:44:19.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:19.126390+0000 mon.b (mon.1) 229 : audit [INF] from='client.? 172.21.15.47:0/2650583472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi047-82457-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:19.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:19.133890+0000 mon.a (mon.0) 1366 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi047-82457-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:19.917 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:44:19.921 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:19.921 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:44:19.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:17.360019+0000 osd.6 (osd.6) 83 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:44:19.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:17.360683+0000 osd.6 (osd.6) 84 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:44:19.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:18.629018+0000 osd.0 (osd.0) 93 : cluster [DBG] 4.6 scrub starts 2024-02-23T00:44:19.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:18.630359+0000 osd.0 (osd.0) 94 : cluster [DBG] 4.6 scrub ok 2024-02-23T00:44:19.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:18.749334+0000 mon.b (mon.1) 228 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:19.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:19.099544+0000 mon.c (mon.2) 113 : audit [DBG] from='client.? 172.21.15.47:0/4214806690' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:44:19.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:19.112933+0000 mon.a (mon.0) 1364 : audit [INF] from='client.? 172.21.15.47:0/1351400035' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi047-82378-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:19.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:19.113103+0000 mon.a (mon.0) 1365 : cluster [DBG] osdmap e187: 8 total, 8 up, 8 in 2024-02-23T00:44:19.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:19.126390+0000 mon.b (mon.1) 229 : audit [INF] from='client.? 172.21.15.47:0/2650583472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi047-82457-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:19.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:19.133890+0000 mon.a (mon.0) 1366 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi047-82457-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:20.022 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:44:20.052 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:20.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:18.883574+0000 osd.7 (osd.7) 109 : cluster [DBG] 2.8 scrub starts 2024-02-23T00:44:20.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:18.886473+0000 osd.7 (osd.7) 110 : cluster [DBG] 2.8 scrub ok 2024-02-23T00:44:20.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:18.945723+0000 osd.5 (osd.5) 91 : cluster [DBG] 2.12 scrub starts 2024-02-23T00:44:20.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:18.946556+0000 osd.5 (osd.5) 92 : cluster [DBG] 2.12 scrub ok 2024-02-23T00:44:20.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:19.221772+0000 mgr.y (mgr.14517) 229 : cluster [DBG] pgmap v256: 322 pgs: 258 active+clean, 64 unknown; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-23T00:44:20.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:19.311569+0000 osd.6 (osd.6) 85 : cluster [DBG] 5.2 scrub starts 2024-02-23T00:44:20.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:19.312410+0000 osd.6 (osd.6) 86 : cluster [DBG] 5.2 scrub ok 2024-02-23T00:44:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:19.750129+0000 mon.b (mon.1) 230 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:19.828696+0000 osd.1 (osd.1) 107 : cluster [DBG] 4.f scrub starts 2024-02-23T00:44:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:19.830304+0000 osd.1 (osd.1) 108 : cluster [DBG] 4.f scrub ok 2024-02-23T00:44:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:20.104634+0000 mon.a (mon.0) 1367 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi047-82457-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:20.104779+0000 mon.a (mon.0) 1368 : cluster [DBG] osdmap e188: 8 total, 8 up, 8 in 2024-02-23T00:44:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:20.121540+0000 mon.c (mon.2) 114 : audit [INF] from='client.? 172.21.15.47:0/2162750798' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi047-82386-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:20.129581+0000 mon.a (mon.0) 1369 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi047-82386-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:20.705 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:44:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:44:20] "GET /metrics HTTP/1.1" 200 193327 "" "Prometheus/2.33.4" 2024-02-23T00:44:20.705 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:18.883574+0000 osd.7 (osd.7) 109 : cluster [DBG] 2.8 scrub starts 2024-02-23T00:44:20.705 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:18.886473+0000 osd.7 (osd.7) 110 : cluster [DBG] 2.8 scrub ok 2024-02-23T00:44:20.705 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:18.945723+0000 osd.5 (osd.5) 91 : cluster [DBG] 2.12 scrub starts 2024-02-23T00:44:20.705 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:18.946556+0000 osd.5 (osd.5) 92 : cluster [DBG] 2.12 scrub ok 2024-02-23T00:44:20.705 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:19.221772+0000 mgr.y (mgr.14517) 229 : cluster [DBG] pgmap v256: 322 pgs: 258 active+clean, 64 unknown; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-23T00:44:20.705 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:19.311569+0000 osd.6 (osd.6) 85 : cluster [DBG] 5.2 scrub starts 2024-02-23T00:44:20.705 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:19.312410+0000 osd.6 (osd.6) 86 : cluster [DBG] 5.2 scrub ok 2024-02-23T00:44:20.705 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:19.750129+0000 mon.b (mon.1) 230 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:20.705 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:19.828696+0000 osd.1 (osd.1) 107 : cluster [DBG] 4.f scrub starts 2024-02-23T00:44:20.705 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:19.830304+0000 osd.1 (osd.1) 108 : cluster [DBG] 4.f scrub ok 2024-02-23T00:44:20.706 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:20.104634+0000 mon.a (mon.0) 1367 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi047-82457-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:20.706 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:20.104779+0000 mon.a (mon.0) 1368 : cluster [DBG] osdmap e188: 8 total, 8 up, 8 in 2024-02-23T00:44:20.706 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:20.121540+0000 mon.c (mon.2) 114 : audit [INF] from='client.? 172.21.15.47:0/2162750798' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi047-82386-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:20.706 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:20.129581+0000 mon.a (mon.0) 1369 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi047-82386-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:20.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:18.883574+0000 osd.7 (osd.7) 109 : cluster [DBG] 2.8 scrub starts 2024-02-23T00:44:20.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:18.886473+0000 osd.7 (osd.7) 110 : cluster [DBG] 2.8 scrub ok 2024-02-23T00:44:20.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:18.945723+0000 osd.5 (osd.5) 91 : cluster [DBG] 2.12 scrub starts 2024-02-23T00:44:20.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:18.946556+0000 osd.5 (osd.5) 92 : cluster [DBG] 2.12 scrub ok 2024-02-23T00:44:20.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:19.221772+0000 mgr.y (mgr.14517) 229 : cluster [DBG] pgmap v256: 322 pgs: 258 active+clean, 64 unknown; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-23T00:44:20.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:19.311569+0000 osd.6 (osd.6) 85 : cluster [DBG] 5.2 scrub starts 2024-02-23T00:44:20.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:19.312410+0000 osd.6 (osd.6) 86 : cluster [DBG] 5.2 scrub ok 2024-02-23T00:44:20.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:19.750129+0000 mon.b (mon.1) 230 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:20.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:19.828696+0000 osd.1 (osd.1) 107 : cluster [DBG] 4.f scrub starts 2024-02-23T00:44:20.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:19.830304+0000 osd.1 (osd.1) 108 : cluster [DBG] 4.f scrub ok 2024-02-23T00:44:20.707 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:20.104634+0000 mon.a (mon.0) 1367 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi047-82457-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:20.707 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:20.104779+0000 mon.a (mon.0) 1368 : cluster [DBG] osdmap e188: 8 total, 8 up, 8 in 2024-02-23T00:44:20.707 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:20.121540+0000 mon.c (mon.2) 114 : audit [INF] from='client.? 172.21.15.47:0/2162750798' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi047-82386-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:20.707 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:20.129581+0000 mon.a (mon.0) 1369 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi047-82386-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:21.102 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:44:21.105 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:21.106 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:44:21.207 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:44:21.235 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:21.383 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:44:21.386 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:21.386 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:44:21.487 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:44:21.513 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:21.635 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:44:21.639 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:21.639 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:44:21.740 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:44:21.768 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:21.892 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:44:21.894 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:20.625144+0000 osd.0 (osd.0) 95 : cluster [DBG] 5.5 scrub starts 2024-02-23T00:44:21.895 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:20.626048+0000 osd.0 (osd.0) 96 : cluster [DBG] 5.5 scrub ok 2024-02-23T00:44:21.895 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:20.750978+0000 mon.b (mon.1) 231 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:21.895 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:20.955795+0000 osd.5 (osd.5) 93 : cluster [DBG] 6.7 scrub starts 2024-02-23T00:44:21.895 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:20.958776+0000 osd.5 (osd.5) 94 : cluster [DBG] 6.7 scrub ok 2024-02-23T00:44:21.895 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:21.107100+0000 mon.a (mon.0) 1370 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi047-82386-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:21.895 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:21.107276+0000 mon.a (mon.0) 1371 : cluster [DBG] osdmap e189: 8 total, 8 up, 8 in 2024-02-23T00:44:21.895 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:21.116413+0000 mon.b (mon.1) 232 : audit [INF] from='client.? 172.21.15.47:0/2906348713' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi047-82378-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:21.895 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:21.119212+0000 mon.a (mon.0) 1372 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi047-82378-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:21.895 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:20.625144+0000 osd.0 (osd.0) 95 : cluster [DBG] 5.5 scrub starts 2024-02-23T00:44:21.896 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:20.626048+0000 osd.0 (osd.0) 96 : cluster [DBG] 5.5 scrub ok 2024-02-23T00:44:21.896 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:20.750978+0000 mon.b (mon.1) 231 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:21.896 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:20.955795+0000 osd.5 (osd.5) 93 : cluster [DBG] 6.7 scrub starts 2024-02-23T00:44:21.896 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:20.958776+0000 osd.5 (osd.5) 94 : cluster [DBG] 6.7 scrub ok 2024-02-23T00:44:21.896 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:21.107100+0000 mon.a (mon.0) 1370 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi047-82386-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:21.896 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:21.107276+0000 mon.a (mon.0) 1371 : cluster [DBG] osdmap e189: 8 total, 8 up, 8 in 2024-02-23T00:44:21.896 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:21.116413+0000 mon.b (mon.1) 232 : audit [INF] from='client.? 172.21.15.47:0/2906348713' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi047-82378-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:21.896 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:21.119212+0000 mon.a (mon.0) 1372 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi047-82378-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:21.897 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:21.897 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:44:21.914 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:44:21.914 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:44:21.935 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:20.625144+0000 osd.0 (osd.0) 95 : cluster [DBG] 5.5 scrub starts 2024-02-23T00:44:21.935 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:20.626048+0000 osd.0 (osd.0) 96 : cluster [DBG] 5.5 scrub ok 2024-02-23T00:44:21.935 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:20.750978+0000 mon.b (mon.1) 231 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:21.935 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:20.955795+0000 osd.5 (osd.5) 93 : cluster [DBG] 6.7 scrub starts 2024-02-23T00:44:21.935 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:20.958776+0000 osd.5 (osd.5) 94 : cluster [DBG] 6.7 scrub ok 2024-02-23T00:44:21.935 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:21.107100+0000 mon.a (mon.0) 1370 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi047-82386-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:21.935 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:21.107276+0000 mon.a (mon.0) 1371 : cluster [DBG] osdmap e189: 8 total, 8 up, 8 in 2024-02-23T00:44:21.935 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:21.116413+0000 mon.b (mon.1) 232 : audit [INF] from='client.? 172.21.15.47:0/2906348713' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi047-82378-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:21.935 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:21.119212+0000 mon.a (mon.0) 1372 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi047-82378-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:21.998 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:44:22.027 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:22.225 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:44:22.228 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:22.228 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:44:22.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.229+0000 7f47c49f3700 1 -- 172.21.15.47:0/611770251 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47c00f63b0 msgr2=0x7f47c00f6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:22.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.229+0000 7f47c49f3700 1 --2- 172.21.15.47:0/611770251 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47c00f63b0 0x7f47c00f6830 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 rx=0x7f47b40048a0 tx=0x7f47b4009cb0).stop 2024-02-23T00:44:22.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.229+0000 7f47c49f3700 1 -- 172.21.15.47:0/611770251 shutdown_connections 2024-02-23T00:44:22.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.229+0000 7f47c49f3700 1 --2- 172.21.15.47:0/611770251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47c00f6d20 0x7f47c0106b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:22.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.229+0000 7f47c49f3700 1 --2- 172.21.15.47:0/611770251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47c00f89b0 0x7f47c00f8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:22.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.229+0000 7f47c49f3700 1 --2- 172.21.15.47:0/611770251 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47c00f63b0 0x7f47c00f6830 unknown :-1 s=CLOSED pgs=2512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:22.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.229+0000 7f47c49f3700 1 -- 172.21.15.47:0/611770251 >> 172.21.15.47:0/611770251 conn(0x7f47c006c370 msgr2=0x7f47c006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:22.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.229+0000 7f47c49f3700 1 -- 172.21.15.47:0/611770251 shutdown_connections 2024-02-23T00:44:22.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.229+0000 7f47c49f3700 1 -- 172.21.15.47:0/611770251 wait complete. 2024-02-23T00:44:22.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.230+0000 7f47c49f3700 1 librados: starting msgr at 2024-02-23T00:44:22.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.230+0000 7f47c49f3700 1 librados: starting objecter 2024-02-23T00:44:22.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.230+0000 7f47c49f3700 1 Processor -- start 2024-02-23T00:44:22.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.230+0000 7f47c49f3700 1 -- start start 2024-02-23T00:44:22.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.230+0000 7f47c49f3700 1 librados: setting wanted keys 2024-02-23T00:44:22.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.230+0000 7f47c49f3700 1 librados: calling monclient init 2024-02-23T00:44:22.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.230+0000 7f47c49f3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47c00f63b0 0x7f47c01101f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:22.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.230+0000 7f47c49f3700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47c00f6d20 0x7f47c01106e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:22.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.230+0000 7f47c49f3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47c00f89b0 0x7f47c010a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:22.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.230+0000 7f47c49f3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c010a880 con 0x7f47c00f63b0 2024-02-23T00:44:22.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.230+0000 7f47c49f3700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c010a9f0 con 0x7f47c00f89b0 2024-02-23T00:44:22.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.230+0000 7f47c49f3700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c010abb0 con 0x7f47c00f6d20 2024-02-23T00:44:22.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.230+0000 7f47bdd9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47c00f6d20 0x7f47c01106e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:22.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.230+0000 7f47be59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47c00f63b0 0x7f47c01101f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:22.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.230+0000 7f47bdd9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47c00f6d20 0x7f47c01106e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52732/0 (socket says 172.21.15.47:52732) 2024-02-23T00:44:22.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.230+0000 7f47bdd9b700 1 -- 172.21.15.47:0/3851130027 learned_addr learned my addr 172.21.15.47:0/3851130027 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:44:22.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.231+0000 7f47bed9d700 1 --2- 172.21.15.47:0/3851130027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47c00f89b0 0x7f47c010a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:22.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.231+0000 7f47bdd9b700 1 -- 172.21.15.47:0/3851130027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47c00f89b0 msgr2=0x7f47c010a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:22.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.231+0000 7f47bdd9b700 1 --2- 172.21.15.47:0/3851130027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47c00f89b0 0x7f47c010a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:22.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.231+0000 7f47bdd9b700 1 -- 172.21.15.47:0/3851130027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47c00f63b0 msgr2=0x7f47c01101f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:22.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.231+0000 7f47bdd9b700 1 --2- 172.21.15.47:0/3851130027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47c00f63b0 0x7f47c01101f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:22.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.231+0000 7f47bdd9b700 1 -- 172.21.15.47:0/3851130027 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b4005040 con 0x7f47c00f6d20 2024-02-23T00:44:22.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.231+0000 7f47be59c700 1 --2- 172.21.15.47:0/3851130027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47c00f63b0 0x7f47c01101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:44:22.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.231+0000 7f47af7fe700 10 client.?.objecter ms_handle_connect 0x7f47c00f6d20 2024-02-23T00:44:22.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.231+0000 7f47af7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:44:22.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.231+0000 7f47bdd9b700 1 --2- 172.21.15.47:0/3851130027 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47c00f6d20 0x7f47c01106e0 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 rx=0x7f47b4007fd0 tx=0x7f47b4009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:22.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.231+0000 7f47af7fe700 1 -- 172.21.15.47:0/3851130027 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b400e030 con 0x7f47c00f6d20 2024-02-23T00:44:22.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.231+0000 7f47af7fe700 1 -- 172.21.15.47:0/3851130027 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f47b40096d0 con 0x7f47c00f6d20 2024-02-23T00:44:22.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.231+0000 7f47af7fe700 1 -- 172.21.15.47:0/3851130027 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b4018070 con 0x7f47c00f6d20 2024-02-23T00:44:22.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.231+0000 7f47c49f3700 1 -- 172.21.15.47:0/3851130027 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c010ae30 con 0x7f47c00f6d20 2024-02-23T00:44:22.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.231+0000 7f47c49f3700 10 client.40613.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:44:22.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.231+0000 7f47c49f3700 1 -- 172.21.15.47:0/3851130027 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c0117020 con 0x7f47c00f6d20 2024-02-23T00:44:22.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.231+0000 7f47c49f3700 1 librados: init done 2024-02-23T00:44:22.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.233+0000 7f47c49f3700 1 -- 172.21.15.47:0/3851130027 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c010b270 con 0x7f47c00f6d20 2024-02-23T00:44:22.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.233+0000 7f47af7fe700 1 -- 172.21.15.47:0/3851130027 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f47b4009240 con 0x7f47c00f6d20 2024-02-23T00:44:22.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.233+0000 7f47af7fe700 1 --2- 172.21.15.47:0/3851130027 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f47a8062cc0 0x7f47a8065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:22.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.234+0000 7f47be59c700 1 --2- 172.21.15.47:0/3851130027 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f47a8062cc0 0x7f47a8065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:22.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.234+0000 7f47af7fe700 1 -- 172.21.15.47:0/3851130027 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(190..190 src has 48..190) v4 ==== 10505+0+0 (secure 0 0 0) 0x7f47b407ec90 con 0x7f47c00f6d20 2024-02-23T00:44:22.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.234+0000 7f47af7fe700 10 client.40613.objecter ms_dispatch 0x7f47c0001550 osd_map(190..190 src has 48..190) v4 2024-02-23T00:44:22.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.234+0000 7f47af7fe700 3 client.40613.objecter handle_osd_map got epochs [190,190] > 0 2024-02-23T00:44:22.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.234+0000 7f47af7fe700 3 client.40613.objecter handle_osd_map decoding full epoch 190 2024-02-23T00:44:22.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.234+0000 7f47af7fe700 20 client.40613.objecter dump_active .. 0 homeless 2024-02-23T00:44:22.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.234+0000 7f47af7fe700 10 client.40613.objecter ms_handle_connect 0x7f47a8062cc0 2024-02-23T00:44:22.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.234+0000 7f47be59c700 1 --2- 172.21.15.47:0/3851130027 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f47a8062cc0 0x7f47a8065180 secure :-1 s=READY pgs=2937 cs=0 l=1 rev1=1 rx=0x7f47b0007bf0 tx=0x7f47b000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:22.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.238+0000 7f47af7fe700 1 -- 172.21.15.47:0/3851130027 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f47b404de20 con 0x7f47c00f6d20 2024-02-23T00:44:22.330 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:44:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:22.395 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:44:22.399 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-23T00:44:22.400 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:22.404 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:44:22.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:22.431+0000 7f47c49f3700 1 -- 172.21.15.47:0/3851130027 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd reweight", "id": 4, "weight": 1.0} v 0) v1 -- 0x7f47c0109110 con 0x7f47c00f6d20 2024-02-23T00:44:22.505 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:44:22.536 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:22.541 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:44:22.563 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:44:22.567 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:22.567 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:44:22.617 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:44:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:44:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:44:22.617 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:44:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:44:22.668 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:44:22.689 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.6 dump_historic_ops 2024-02-23T00:44:22.698 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:22.724 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:44:22.725 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:44:22.727 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:22.728 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:44:22.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:21.156299+0000 osd.3 (osd.3) 111 : cluster [DBG] 2.19 scrub starts 2024-02-23T00:44:22.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:21.157312+0000 osd.3 (osd.3) 112 : cluster [DBG] 2.19 scrub ok 2024-02-23T00:44:22.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:21.222179+0000 mgr.y (mgr.14517) 230 : cluster [DBG] pgmap v259: 322 pgs: 258 active+clean, 64 unknown; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:44:22.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:21.751950+0000 mon.b (mon.1) 233 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:22.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:22.111445+0000 mon.a (mon.0) 1373 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi047-82378-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:22.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:22.111645+0000 mon.a (mon.0) 1374 : cluster [DBG] osdmap e190: 8 total, 8 up, 8 in 2024-02-23T00:44:22.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:22.130670+0000 mon.c (mon.2) 115 : audit [INF] from='client.? 172.21.15.47:0/288219773' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi047-82457-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:22.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:22.131387+0000 mon.a (mon.0) 1375 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi047-82457-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:22.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:22.138816+0000 mon.c (mon.2) 116 : audit [INF] from='client.? 172.21.15.47:0/3414618047' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi047-82496-15"}]: dispatch 2024-02-23T00:44:22.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:22.139417+0000 mon.a (mon.0) 1376 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi047-82496-15"}]: dispatch 2024-02-23T00:44:22.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:21.156299+0000 osd.3 (osd.3) 111 : cluster [DBG] 2.19 scrub starts 2024-02-23T00:44:22.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:21.157312+0000 osd.3 (osd.3) 112 : cluster [DBG] 2.19 scrub ok 2024-02-23T00:44:22.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:21.222179+0000 mgr.y (mgr.14517) 230 : cluster [DBG] pgmap v259: 322 pgs: 258 active+clean, 64 unknown; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:44:22.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:21.751950+0000 mon.b (mon.1) 233 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:22.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:22.111445+0000 mon.a (mon.0) 1373 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi047-82378-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:22.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:22.111645+0000 mon.a (mon.0) 1374 : cluster [DBG] osdmap e190: 8 total, 8 up, 8 in 2024-02-23T00:44:22.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:22.130670+0000 mon.c (mon.2) 115 : audit [INF] from='client.? 172.21.15.47:0/288219773' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi047-82457-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:22.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:22.131387+0000 mon.a (mon.0) 1375 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi047-82457-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:22.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:22.138816+0000 mon.c (mon.2) 116 : audit [INF] from='client.? 172.21.15.47:0/3414618047' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi047-82496-15"}]: dispatch 2024-02-23T00:44:22.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:22.139417+0000 mon.a (mon.0) 1376 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi047-82496-15"}]: dispatch 2024-02-23T00:44:22.828 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:44:22.858 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:22.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:21.156299+0000 osd.3 (osd.3) 111 : cluster [DBG] 2.19 scrub starts 2024-02-23T00:44:22.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:21.157312+0000 osd.3 (osd.3) 112 : cluster [DBG] 2.19 scrub ok 2024-02-23T00:44:22.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:21.222179+0000 mgr.y (mgr.14517) 230 : cluster [DBG] pgmap v259: 322 pgs: 258 active+clean, 64 unknown; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:44:22.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:21.751950+0000 mon.b (mon.1) 233 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:22.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:22.111445+0000 mon.a (mon.0) 1373 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi047-82378-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:22.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:22.111645+0000 mon.a (mon.0) 1374 : cluster [DBG] osdmap e190: 8 total, 8 up, 8 in 2024-02-23T00:44:22.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:22.130670+0000 mon.c (mon.2) 115 : audit [INF] from='client.? 172.21.15.47:0/288219773' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi047-82457-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:22.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:22.131387+0000 mon.a (mon.0) 1375 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi047-82457-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:22.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:22.138816+0000 mon.c (mon.2) 116 : audit [INF] from='client.? 172.21.15.47:0/3414618047' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi047-82496-15"}]: dispatch 2024-02-23T00:44:22.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:22.139417+0000 mon.a (mon.0) 1376 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi047-82496-15"}]: dispatch 2024-02-23T00:44:22.957 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:44:22.960 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:44:22.960 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:22.961 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:44:23.062 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:44:23.088 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:23.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:23.119+0000 7f47af7fe700 1 -- 172.21.15.47:0/3851130027 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 4, "weight": 1.0}]=0 reweighted osd.4 to 1 (10000) v191) v1 ==== 113+0+0 (secure 0 0 0) 0x7f47b401de60 con 0x7f47c00f6d20 2024-02-23T00:44:23.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:23.122+0000 7f47c49f3700 10 librados: watch_flush enter 2024-02-23T00:44:23.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:23.122+0000 7f47c49f3700 10 librados: watch_flush exit 2024-02-23T00:44:23.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:23.122+0000 7f47c49f3700 20 client.40613.objecter shutdown clearing up homeless session... 2024-02-23T00:44:23.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:23.122+0000 7f47c49f3700 10 client.40613.objecter successfully canceled tick 2024-02-23T00:44:23.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:23.131+0000 7f47c49f3700 1 -- 172.21.15.47:0/3851130027 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f47a8062cc0 msgr2=0x7f47a8065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:23.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:23.131+0000 7f47c49f3700 1 --2- 172.21.15.47:0/3851130027 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f47a8062cc0 0x7f47a8065180 secure :-1 s=READY pgs=2937 cs=0 l=1 rev1=1 rx=0x7f47b0007bf0 tx=0x7f47b000b040).stop 2024-02-23T00:44:23.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:23.131+0000 7f47c49f3700 1 -- 172.21.15.47:0/3851130027 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47c00f6d20 msgr2=0x7f47c01106e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:23.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:23.131+0000 7f47c49f3700 1 --2- 172.21.15.47:0/3851130027 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47c00f6d20 0x7f47c01106e0 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 rx=0x7f47b4007fd0 tx=0x7f47b4009cb0).stop 2024-02-23T00:44:23.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:23.132+0000 7f47c49f3700 1 -- 172.21.15.47:0/3851130027 shutdown_connections 2024-02-23T00:44:23.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:23.132+0000 7f47c49f3700 1 --2- 172.21.15.47:0/3851130027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47c00f89b0 0x7f47c010a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:23.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:23.132+0000 7f47c49f3700 1 --2- 172.21.15.47:0/3851130027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47c00f63b0 0x7f47c01101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:23.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:23.132+0000 7f47c49f3700 1 --2- 172.21.15.47:0/3851130027 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f47a8062cc0 0x7f47a8065180 secure :-1 s=CLOSED pgs=2937 cs=0 l=1 rev1=1 rx=0x7f47b0007bf0 tx=0x7f47b000b040).stop 2024-02-23T00:44:23.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:23.132+0000 7f47c49f3700 1 --2- 172.21.15.47:0/3851130027 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47c00f6d20 0x7f47c01106e0 unknown :-1 s=CLOSED pgs=2513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:23.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:23.132+0000 7f47c49f3700 1 -- 172.21.15.47:0/3851130027 >> 172.21.15.47:0/3851130027 conn(0x7f47c006c370 msgr2=0x7f47c00f44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:23.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:23.144+0000 7f47c49f3700 1 -- 172.21.15.47:0/3851130027 shutdown_connections 2024-02-23T00:44:23.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:23.145+0000 7f47c49f3700 1 -- 172.21.15.47:0/3851130027 wait complete. 2024-02-23T00:44:23.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:23.146+0000 7f47c49f3700 1 librados: shutdown 2024-02-23T00:44:23.151 INFO:teuthology.orchestra.run.smithi047.stderr:reweighted osd.4 to 1 (10000) 2024-02-23T00:44:23.286 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:44:23.290 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:23.290 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:44:23.391 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:44:23.417 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:23.542 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:44:23.544 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:23.545 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:44:23.646 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:44:23.678 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:23.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:21.971569+0000 osd.5 (osd.5) 95 : cluster [DBG] 6.a scrub starts 2024-02-23T00:44:23.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:21.972379+0000 osd.5 (osd.5) 96 : cluster [DBG] 6.a scrub ok 2024-02-23T00:44:23.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:22.379191+0000 mgr.y (mgr.14517) 231 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:44:23.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:22.379789+0000 osd.2 (osd.2) 85 : cluster [DBG] 5.a scrub starts 2024-02-23T00:44:23.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:22.380833+0000 osd.2 (osd.2) 86 : cluster [DBG] 5.a scrub ok 2024-02-23T00:44:23.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:22.431637+0000 mon.c (mon.2) 117 : audit [INF] from='client.? 172.21.15.47:0/3851130027' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-23T00:44:23.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:22.615369+0000 mon.a (mon.0) 1377 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-23T00:44:23.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:22.752953+0000 mon.b (mon.1) 234 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:23.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:22.801374+0000 osd.1 (osd.1) 109 : cluster [DBG] 5.c scrub starts 2024-02-23T00:44:23.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:22.804273+0000 osd.1 (osd.1) 110 : cluster [DBG] 5.c scrub ok 2024-02-23T00:44:23.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:22.973528+0000 osd.5 (osd.5) 97 : cluster [DBG] 6.d scrub starts 2024-02-23T00:44:23.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:22.976604+0000 osd.5 (osd.5) 98 : cluster [DBG] 6.d scrub ok 2024-02-23T00:44:23.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:23.113513+0000 mon.a (mon.0) 1378 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi047-82457-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:23.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:23.113623+0000 mon.a (mon.0) 1379 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi047-82496-15"}]': finished 2024-02-23T00:44:23.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:23.113773+0000 mon.a (mon.0) 1380 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-02-23T00:44:23.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:23.113837+0000 mon.a (mon.0) 1381 : cluster [DBG] osdmap e191: 8 total, 8 up, 8 in 2024-02-23T00:44:23.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:23.120187+0000 mon.c (mon.2) 118 : audit [INF] from='client.? 172.21.15.47:0/3414618047' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi047-82496-15"}]: dispatch 2024-02-23T00:44:23.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:23.127284+0000 mon.a (mon.0) 1382 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi047-82496-15"}]: dispatch 2024-02-23T00:44:23.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:23.129150+0000 mon.a (mon.0) 1383 : audit [INF] from='client.? 172.21.15.47:0/4069652202' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi047-82386-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:23.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:23.151571+0000 osd.3 (osd.3) 113 : cluster [DBG] 6.11 scrub starts 2024-02-23T00:44:23.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:23.152598+0000 osd.3 (osd.3) 114 : cluster [DBG] 6.11 scrub ok 2024-02-23T00:44:23.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:23.159706+0000 mon.a (mon.0) 1384 : audit [INF] from='client.? 172.21.15.47:0/2361769855' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi047-82378-26"}]: dispatch 2024-02-23T00:44:23.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:23.161182+0000 mon.a (mon.0) 1385 : audit [INF] from='client.? 172.21.15.47:0/2361769855' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi047-82378-26"}]: dispatch 2024-02-23T00:44:23.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:23.161536+0000 mon.a (mon.0) 1386 : audit [INF] from='client.? 172.21.15.47:0/2361769855' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi047-82378-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:23.857 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:44:23.860 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:23.860 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:44:23.961 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:44:23.990 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:21.971569+0000 osd.5 (osd.5) 95 : cluster [DBG] 6.a scrub starts 2024-02-23T00:44:23.990 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:21.972379+0000 osd.5 (osd.5) 96 : cluster [DBG] 6.a scrub ok 2024-02-23T00:44:23.990 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:22.379191+0000 mgr.y (mgr.14517) 231 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:44:23.990 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:22.379789+0000 osd.2 (osd.2) 85 : cluster [DBG] 5.a scrub starts 2024-02-23T00:44:23.990 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:22.380833+0000 osd.2 (osd.2) 86 : cluster [DBG] 5.a scrub ok 2024-02-23T00:44:23.990 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:22.431637+0000 mon.c (mon.2) 117 : audit [INF] from='client.? 172.21.15.47:0/3851130027' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-23T00:44:23.990 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:22.615369+0000 mon.a (mon.0) 1377 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-23T00:44:23.990 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:22.752953+0000 mon.b (mon.1) 234 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:23.990 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:22.801374+0000 osd.1 (osd.1) 109 : cluster [DBG] 5.c scrub starts 2024-02-23T00:44:23.990 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:22.804273+0000 osd.1 (osd.1) 110 : cluster [DBG] 5.c scrub ok 2024-02-23T00:44:23.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:22.973528+0000 osd.5 (osd.5) 97 : cluster [DBG] 6.d scrub starts 2024-02-23T00:44:23.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:22.976604+0000 osd.5 (osd.5) 98 : cluster [DBG] 6.d scrub ok 2024-02-23T00:44:23.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:23.113513+0000 mon.a (mon.0) 1378 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi047-82457-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:23.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:23.113623+0000 mon.a (mon.0) 1379 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi047-82496-15"}]': finished 2024-02-23T00:44:23.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:23.113773+0000 mon.a (mon.0) 1380 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-02-23T00:44:23.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:23.113837+0000 mon.a (mon.0) 1381 : cluster [DBG] osdmap e191: 8 total, 8 up, 8 in 2024-02-23T00:44:23.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:23.120187+0000 mon.c (mon.2) 118 : audit [INF] from='client.? 172.21.15.47:0/3414618047' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi047-82496-15"}]: dispatch 2024-02-23T00:44:23.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:23.127284+0000 mon.a (mon.0) 1382 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi047-82496-15"}]: dispatch 2024-02-23T00:44:23.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:23.129150+0000 mon.a (mon.0) 1383 : audit [INF] from='client.? 172.21.15.47:0/4069652202' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi047-82386-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:23.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:23.151571+0000 osd.3 (osd.3) 113 : cluster [DBG] 6.11 scrub starts 2024-02-23T00:44:23.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:23.152598+0000 osd.3 (osd.3) 114 : cluster [DBG] 6.11 scrub ok 2024-02-23T00:44:23.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:23.159706+0000 mon.a (mon.0) 1384 : audit [INF] from='client.? 172.21.15.47:0/2361769855' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi047-82378-26"}]: dispatch 2024-02-23T00:44:23.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:23.161182+0000 mon.a (mon.0) 1385 : audit [INF] from='client.? 172.21.15.47:0/2361769855' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi047-82378-26"}]: dispatch 2024-02-23T00:44:23.992 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:23.161536+0000 mon.a (mon.0) 1386 : audit [INF] from='client.? 172.21.15.47:0/2361769855' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi047-82378-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:23.992 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:21.971569+0000 osd.5 (osd.5) 95 : cluster [DBG] 6.a scrub starts 2024-02-23T00:44:23.992 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:21.972379+0000 osd.5 (osd.5) 96 : cluster [DBG] 6.a scrub ok 2024-02-23T00:44:23.992 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:22.379191+0000 mgr.y (mgr.14517) 231 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:44:23.992 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:22.379789+0000 osd.2 (osd.2) 85 : cluster [DBG] 5.a scrub starts 2024-02-23T00:44:23.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:22.380833+0000 osd.2 (osd.2) 86 : cluster [DBG] 5.a scrub ok 2024-02-23T00:44:23.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:22.431637+0000 mon.c (mon.2) 117 : audit [INF] from='client.? 172.21.15.47:0/3851130027' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-23T00:44:23.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:22.615369+0000 mon.a (mon.0) 1377 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-23T00:44:23.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:22.752953+0000 mon.b (mon.1) 234 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:23.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:22.801374+0000 osd.1 (osd.1) 109 : cluster [DBG] 5.c scrub starts 2024-02-23T00:44:23.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:22.804273+0000 osd.1 (osd.1) 110 : cluster [DBG] 5.c scrub ok 2024-02-23T00:44:23.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:22.973528+0000 osd.5 (osd.5) 97 : cluster [DBG] 6.d scrub starts 2024-02-23T00:44:23.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:22.976604+0000 osd.5 (osd.5) 98 : cluster [DBG] 6.d scrub ok 2024-02-23T00:44:23.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:23.113513+0000 mon.a (mon.0) 1378 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi047-82457-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:23.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:23.113623+0000 mon.a (mon.0) 1379 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi047-82496-15"}]': finished 2024-02-23T00:44:23.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:23.113773+0000 mon.a (mon.0) 1380 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-02-23T00:44:23.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:23.113837+0000 mon.a (mon.0) 1381 : cluster [DBG] osdmap e191: 8 total, 8 up, 8 in 2024-02-23T00:44:23.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:23.120187+0000 mon.c (mon.2) 118 : audit [INF] from='client.? 172.21.15.47:0/3414618047' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi047-82496-15"}]: dispatch 2024-02-23T00:44:23.994 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:23.127284+0000 mon.a (mon.0) 1382 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi047-82496-15"}]: dispatch 2024-02-23T00:44:23.994 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:23.129150+0000 mon.a (mon.0) 1383 : audit [INF] from='client.? 172.21.15.47:0/4069652202' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi047-82386-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:23.994 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:23.151571+0000 osd.3 (osd.3) 113 : cluster [DBG] 6.11 scrub starts 2024-02-23T00:44:23.994 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:23.152598+0000 osd.3 (osd.3) 114 : cluster [DBG] 6.11 scrub ok 2024-02-23T00:44:23.994 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:23.159706+0000 mon.a (mon.0) 1384 : audit [INF] from='client.? 172.21.15.47:0/2361769855' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi047-82378-26"}]: dispatch 2024-02-23T00:44:23.994 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:23.161182+0000 mon.a (mon.0) 1385 : audit [INF] from='client.? 172.21.15.47:0/2361769855' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi047-82378-26"}]: dispatch 2024-02-23T00:44:23.994 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:23.161536+0000 mon.a (mon.0) 1386 : audit [INF] from='client.? 172.21.15.47:0/2361769855' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi047-82378-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:23.994 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:24.152 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP api_snapshots_pp: Running main() from gmock_main.cc 2024-02-23T00:44:24.152 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [==========] Running 20 tests from 5 test suites. 2024-02-23T00:44:24.152 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2024-02-23T00:44:24.152 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2024-02-23T00:44:24.153 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: seed 82496 2024-02-23T00:44:24.153 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-02-23T00:44:24.153 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (2664 ms) 2024-02-23T00:44:24.153 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-02-23T00:44:24.153 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (2433 ms) 2024-02-23T00:44:24.153 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-02-23T00:44:24.153 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (4586 ms) 2024-02-23T00:44:24.153 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-02-23T00:44:24.153 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (2020 ms) 2024-02-23T00:44:24.153 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-02-23T00:44:24.153 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (4693 ms) 2024-02-23T00:44:24.153 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (16396 ms total) 2024-02-23T00:44:24.153 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: 2024-02-23T00:44:24.153 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [----------] 6 tests from LibRadosSnapshotsSelfManagedPP 2024-02-23T00:44:24.154 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-02-23T00:44:24.154 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (5918 ms) 2024-02-23T00:44:24.154 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-02-23T00:44:24.154 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (5561 ms) 2024-02-23T00:44:24.154 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-02-23T00:44:24.154 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (6734 ms) 2024-02-23T00:44:24.154 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-02-23T00:44:24.154 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (5000 ms) 2024-02-23T00:44:24.154 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-02-23T00:44:24.154 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2081 ms) 2024-02-23T00:44:24.154 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-02-23T00:44:24.154 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: deleting snap 23 in pool LibRadosSnapshotsSelfManagedPP_smithi047-82496-7 2024-02-23T00:44:24.154 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: waiting for snaps to purge 2024-02-23T00:44:24.154 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (18961 ms) 2024-02-23T00:44:24.155 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [----------] 6 tests from LibRadosSnapshotsSelfManagedPP (44256 ms total) 2024-02-23T00:44:24.155 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: 2024-02-23T00:44:24.155 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-02-23T00:44:24.155 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-02-23T00:44:24.155 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (8 ms) 2024-02-23T00:44:24.155 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-02-23T00:44:24.155 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5014 ms) 2024-02-23T00:44:24.155 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5022 ms total) 2024-02-23T00:44:24.155 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: 2024-02-23T00:44:24.155 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2024-02-23T00:44:24.155 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-02-23T00:44:24.155 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (2640 ms) 2024-02-23T00:44:24.155 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-02-23T00:44:24.155 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapRemovePP (1996 ms) 2024-02-23T00:44:24.156 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-02-23T00:44:24.156 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (2018 ms) 2024-02-23T00:44:24.156 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-02-23T00:44:24.156 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (1987 ms) 2024-02-23T00:44:24.156 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (8641 ms total) 2024-02-23T00:44:24.156 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: 2024-02-23T00:44:24.279 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight 5 1 2024-02-23T00:44:24.295 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:44:24.719 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:23.108064+0000 osd.4 (osd.4) 91 : cluster [DBG] 5.e scrub starts 2024-02-23T00:44:24.719 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:23.109434+0000 osd.4 (osd.4) 92 : cluster [DBG] 5.e scrub ok 2024-02-23T00:44:24.720 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:23.222575+0000 mgr.y (mgr.14517) 232 : cluster [DBG] pgmap v262: 314 pgs: 250 active+clean, 32 creating+peering, 32 unknown; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:44:24.720 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:23.542015+0000 osd.0 (osd.0) 97 : cluster [DBG] 18.4 scrub starts 2024-02-23T00:44:24.720 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:23.542971+0000 osd.0 (osd.0) 98 : cluster [DBG] 18.4 scrub ok 2024-02-23T00:44:24.720 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:23.754063+0000 mon.b (mon.1) 235 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:24.720 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:23.964156+0000 osd.5 (osd.5) 99 : cluster [DBG] 2.2 scrub starts 2024-02-23T00:44:24.720 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:23.965117+0000 osd.5 (osd.5) 100 : cluster [DBG] 2.2 scrub ok 2024-02-23T00:44:24.720 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:24.111219+0000 mon.a (mon.0) 1387 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:44:24.720 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:24.115276+0000 mon.a (mon.0) 1388 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi047-82496-15"}]': finished 2024-02-23T00:44:24.720 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:24.115390+0000 mon.a (mon.0) 1389 : audit [INF] from='client.? 172.21.15.47:0/4069652202' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi047-82386-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:24.720 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:24.115585+0000 mon.a (mon.0) 1390 : audit [INF] from='client.? 172.21.15.47:0/2361769855' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi047-82378-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:44:24.720 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:24.115644+0000 mon.a (mon.0) 1391 : cluster [DBG] osdmap e192: 8 total, 8 up, 8 in 2024-02-23T00:44:24.720 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:24.122811+0000 mon.a (mon.0) 1392 : audit [INF] from='client.? 172.21.15.47:0/2361769855' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi047-82378-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi047-82378-26"}]: dispatch 2024-02-23T00:44:24.721 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:24.167924+0000 mon.b (mon.1) 236 : audit [INF] from='client.? 172.21.15.47:0/1089067269' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]: dispatch 2024-02-23T00:44:24.721 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:24.604186+0000 mon.a (mon.0) 1393 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]: dispatch 2024-02-23T00:44:24.721 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:24.604950+0000 mon.b (mon.1) 237 : audit [INF] from='client.? 172.21.15.47:0/1089067269' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]: dispatch 2024-02-23T00:44:24.721 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:24.605242+0000 mon.a (mon.0) 1394 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]: dispatch 2024-02-23T00:44:24.721 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:24.605702+0000 mon.b (mon.1) 238 : audit [INF] from='client.? 172.21.15.47:0/1089067269' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:24.721 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:24.605953+0000 mon.a (mon.0) 1395 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:24.998 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:23.108064+0000 osd.4 (osd.4) 91 : cluster [DBG] 5.e scrub starts 2024-02-23T00:44:24.998 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:23.109434+0000 osd.4 (osd.4) 92 : cluster [DBG] 5.e scrub ok 2024-02-23T00:44:24.998 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:23.222575+0000 mgr.y (mgr.14517) 232 : cluster [DBG] pgmap v262: 314 pgs: 250 active+clean, 32 creating+peering, 32 unknown; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:44:24.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:23.542015+0000 osd.0 (osd.0) 97 : cluster [DBG] 18.4 scrub starts 2024-02-23T00:44:24.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:23.542971+0000 osd.0 (osd.0) 98 : cluster [DBG] 18.4 scrub ok 2024-02-23T00:44:24.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:23.754063+0000 mon.b (mon.1) 235 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:24.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:23.964156+0000 osd.5 (osd.5) 99 : cluster [DBG] 2.2 scrub starts 2024-02-23T00:44:24.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:23.965117+0000 osd.5 (osd.5) 100 : cluster [DBG] 2.2 scrub ok 2024-02-23T00:44:24.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:24.111219+0000 mon.a (mon.0) 1387 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:44:24.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:24.115276+0000 mon.a (mon.0) 1388 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi047-82496-15"}]': finished 2024-02-23T00:44:24.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:24.115390+0000 mon.a (mon.0) 1389 : audit [INF] from='client.? 172.21.15.47:0/4069652202' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi047-82386-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:24.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:24.115585+0000 mon.a (mon.0) 1390 : audit [INF] from='client.? 172.21.15.47:0/2361769855' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi047-82378-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:44:24.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:24.115644+0000 mon.a (mon.0) 1391 : cluster [DBG] osdmap e192: 8 total, 8 up, 8 in 2024-02-23T00:44:24.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:24.122811+0000 mon.a (mon.0) 1392 : audit [INF] from='client.? 172.21.15.47:0/2361769855' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi047-82378-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi047-82378-26"}]: dispatch 2024-02-23T00:44:24.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:24.167924+0000 mon.b (mon.1) 236 : audit [INF] from='client.? 172.21.15.47:0/1089067269' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]: dispatch 2024-02-23T00:44:25.000 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:23.108064+0000 osd.4 (osd.4) 91 : cluster [DBG] 5.e scrub starts 2024-02-23T00:44:25.000 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:23.109434+0000 osd.4 (osd.4) 92 : cluster [DBG] 5.e scrub ok 2024-02-23T00:44:25.000 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:23.222575+0000 mgr.y (mgr.14517) 232 : cluster [DBG] pgmap v262: 314 pgs: 250 active+clean, 32 creating+peering, 32 unknown; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:44:25.000 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:23.542015+0000 osd.0 (osd.0) 97 : cluster [DBG] 18.4 scrub starts 2024-02-23T00:44:25.000 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:23.542971+0000 osd.0 (osd.0) 98 : cluster [DBG] 18.4 scrub ok 2024-02-23T00:44:25.000 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:23.754063+0000 mon.b (mon.1) 235 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:25.000 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:23.964156+0000 osd.5 (osd.5) 99 : cluster [DBG] 2.2 scrub starts 2024-02-23T00:44:25.000 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:23.965117+0000 osd.5 (osd.5) 100 : cluster [DBG] 2.2 scrub ok 2024-02-23T00:44:25.000 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:24.111219+0000 mon.a (mon.0) 1387 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:44:25.000 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:24.115276+0000 mon.a (mon.0) 1388 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi047-82496-15"}]': finished 2024-02-23T00:44:25.000 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:24.115390+0000 mon.a (mon.0) 1389 : audit [INF] from='client.? 172.21.15.47:0/4069652202' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi047-82386-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:25.001 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:24.115585+0000 mon.a (mon.0) 1390 : audit [INF] from='client.? 172.21.15.47:0/2361769855' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi047-82378-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:44:25.001 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:24.115644+0000 mon.a (mon.0) 1391 : cluster [DBG] osdmap e192: 8 total, 8 up, 8 in 2024-02-23T00:44:25.001 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:24.122811+0000 mon.a (mon.0) 1392 : audit [INF] from='client.? 172.21.15.47:0/2361769855' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi047-82378-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi047-82378-26"}]: dispatch 2024-02-23T00:44:25.001 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:24.167924+0000 mon.b (mon.1) 236 : audit [INF] from='client.? 172.21.15.47:0/1089067269' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]: dispatch 2024-02-23T00:44:25.001 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:24.604186+0000 mon.a (mon.0) 1393 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]: dispatch 2024-02-23T00:44:25.001 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:24.604950+0000 mon.b (mon.1) 237 : audit [INF] from='client.? 172.21.15.47:0/1089067269' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]: dispatch 2024-02-23T00:44:25.001 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:24.605242+0000 mon.a (mon.0) 1394 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]: dispatch 2024-02-23T00:44:25.001 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:24.605702+0000 mon.b (mon.1) 238 : audit [INF] from='client.? 172.21.15.47:0/1089067269' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:25.001 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:24.605953+0000 mon.a (mon.0) 1395 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:25.002 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:24.604186+0000 mon.a (mon.0) 1393 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]: dispatch 2024-02-23T00:44:25.002 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:24.604950+0000 mon.b (mon.1) 237 : audit [INF] from='client.? 172.21.15.47:0/1089067269' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]: dispatch 2024-02-23T00:44:25.002 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:24.605242+0000 mon.a (mon.0) 1394 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]: dispatch 2024-02-23T00:44:25.002 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:24.605702+0000 mon.b (mon.1) 238 : audit [INF] from='client.? 172.21.15.47:0/1089067269' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:25.002 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:24.605953+0000 mon.a (mon.0) 1395 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:25.012 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:44:25.012 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:44:25.447 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:44:25.454 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:25.454 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:44:25.555 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:44:25.591 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:25.683 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:44:25.684 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:25.685 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:44:25.785 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:44:25.816 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:25.939 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:44:25.941 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:25.942 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:44:25.950 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.7 dump_ops_in_flight 2024-02-23T00:44:26.042 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:44:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:26.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:24.349959+0000 osd.6 (osd.6) 87 : cluster [DBG] 2.14 scrub starts 2024-02-23T00:44:26.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:24.350793+0000 osd.6 (osd.6) 88 : cluster [DBG] 2.14 scrub ok 2024-02-23T00:44:26.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:24.755281+0000 mon.b (mon.1) 239 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:26.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:25.554983+0000 mon.a (mon.0) 1396 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:44:26.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:25.555117+0000 mon.a (mon.0) 1397 : cluster [DBG] osdmap e193: 8 total, 8 up, 8 in 2024-02-23T00:44:26.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:25.555926+0000 mon.c (mon.2) 119 : audit [INF] from='client.? 172.21.15.47:0/2732424952' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi047-82457-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:26.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:25.555995+0000 mon.b (mon.1) 240 : audit [INF] from='client.? 172.21.15.47:0/1089067269' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]: dispatch 2024-02-23T00:44:26.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:25.562806+0000 mon.a (mon.0) 1398 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi047-82457-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:26.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:25.563587+0000 mon.a (mon.0) 1399 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]: dispatch 2024-02-23T00:44:26.273 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:44:26.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:24.349959+0000 osd.6 (osd.6) 87 : cluster [DBG] 2.14 scrub starts 2024-02-23T00:44:26.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:24.350793+0000 osd.6 (osd.6) 88 : cluster [DBG] 2.14 scrub ok 2024-02-23T00:44:26.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:24.755281+0000 mon.b (mon.1) 239 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:26.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:25.554983+0000 mon.a (mon.0) 1396 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:44:26.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:25.555117+0000 mon.a (mon.0) 1397 : cluster [DBG] osdmap e193: 8 total, 8 up, 8 in 2024-02-23T00:44:26.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:25.555926+0000 mon.c (mon.2) 119 : audit [INF] from='client.? 172.21.15.47:0/2732424952' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi047-82457-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:26.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:25.555995+0000 mon.b (mon.1) 240 : audit [INF] from='client.? 172.21.15.47:0/1089067269' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]: dispatch 2024-02-23T00:44:26.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:25.562806+0000 mon.a (mon.0) 1398 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi047-82457-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:26.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:25.563587+0000 mon.a (mon.0) 1399 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]: dispatch 2024-02-23T00:44:26.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:24.349959+0000 osd.6 (osd.6) 87 : cluster [DBG] 2.14 scrub starts 2024-02-23T00:44:26.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:24.350793+0000 osd.6 (osd.6) 88 : cluster [DBG] 2.14 scrub ok 2024-02-23T00:44:26.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:24.755281+0000 mon.b (mon.1) 239 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:26.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:25.554983+0000 mon.a (mon.0) 1396 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:44:26.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:25.555117+0000 mon.a (mon.0) 1397 : cluster [DBG] osdmap e193: 8 total, 8 up, 8 in 2024-02-23T00:44:26.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:25.555926+0000 mon.c (mon.2) 119 : audit [INF] from='client.? 172.21.15.47:0/2732424952' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi047-82457-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:26.292 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:25.555995+0000 mon.b (mon.1) 240 : audit [INF] from='client.? 172.21.15.47:0/1089067269' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]: dispatch 2024-02-23T00:44:26.292 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:25.562806+0000 mon.a (mon.0) 1398 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi047-82457-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:26.292 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:25.563587+0000 mon.a (mon.0) 1399 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]: dispatch 2024-02-23T00:44:26.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.057+0000 7f631e99f700 1 -- 172.21.15.47:0/3957717034 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f63180f8420 msgr2=0x7f6318104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:26.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.057+0000 7f631e99f700 1 --2- 172.21.15.47:0/3957717034 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f63180f8420 0x7f6318104a20 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 rx=0x7f6310009be0 tx=0x7f6310004d10).stop 2024-02-23T00:44:26.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.057+0000 7f631e99f700 1 -- 172.21.15.47:0/3957717034 shutdown_connections 2024-02-23T00:44:26.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.057+0000 7f631e99f700 1 --2- 172.21.15.47:0/3957717034 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f63180f8420 0x7f6318104a20 unknown :-1 s=CLOSED pgs=2518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:26.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.057+0000 7f631e99f700 1 --2- 172.21.15.47:0/3957717034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63180fdbb0 0x7f63180fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:26.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.057+0000 7f631e99f700 1 --2- 172.21.15.47:0/3957717034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63180fe510 0x7f63180f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:26.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.057+0000 7f631e99f700 1 -- 172.21.15.47:0/3957717034 >> 172.21.15.47:0/3957717034 conn(0x7f63180f3800 msgr2=0x7f63180f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:26.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.057+0000 7f631e99f700 1 -- 172.21.15.47:0/3957717034 shutdown_connections 2024-02-23T00:44:26.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.057+0000 7f631e99f700 1 -- 172.21.15.47:0/3957717034 wait complete. 2024-02-23T00:44:26.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.057+0000 7f631e99f700 1 librados: starting msgr at 2024-02-23T00:44:26.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.057+0000 7f631e99f700 1 librados: starting objecter 2024-02-23T00:44:26.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.057+0000 7f631e99f700 1 Processor -- start 2024-02-23T00:44:26.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.057+0000 7f631e99f700 1 -- start start 2024-02-23T00:44:26.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.057+0000 7f631e99f700 1 librados: setting wanted keys 2024-02-23T00:44:26.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.057+0000 7f631e99f700 1 librados: calling monclient init 2024-02-23T00:44:26.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.058+0000 7f631e99f700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f63180f8420 0x7f631810dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:26.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.058+0000 7f631e99f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63180fdbb0 0x7f631810e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:26.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.058+0000 7f631e99f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63180fe510 0x7f6318107f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:26.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.058+0000 7f631e99f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6318108420 con 0x7f63180fe510 2024-02-23T00:44:26.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.058+0000 7f631e99f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6318108590 con 0x7f63180fdbb0 2024-02-23T00:44:26.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.058+0000 7f631e99f700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6318108750 con 0x7f63180f8420 2024-02-23T00:44:26.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.058+0000 7f6317fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f63180f8420 0x7f631810dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:26.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.058+0000 7f6317fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f63180f8420 0x7f631810dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52814/0 (socket says 172.21.15.47:52814) 2024-02-23T00:44:26.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.058+0000 7f6317fff700 1 -- 172.21.15.47:0/3360978148 learned_addr learned my addr 172.21.15.47:0/3360978148 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:44:26.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.058+0000 7f63177fe700 1 --2- 172.21.15.47:0/3360978148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63180fdbb0 0x7f631810e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:26.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.058+0000 7f631cf3c700 1 --2- 172.21.15.47:0/3360978148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63180fe510 0x7f6318107f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:26.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.058+0000 7f6317fff700 1 -- 172.21.15.47:0/3360978148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63180fdbb0 msgr2=0x7f631810e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:26.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.058+0000 7f6317fff700 1 --2- 172.21.15.47:0/3360978148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63180fdbb0 0x7f631810e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:26.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.058+0000 7f6317fff700 1 -- 172.21.15.47:0/3360978148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63180fe510 msgr2=0x7f6318107f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:26.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.058+0000 7f6317fff700 1 --2- 172.21.15.47:0/3360978148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63180fe510 0x7f6318107f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:26.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.058+0000 7f6317fff700 1 -- 172.21.15.47:0/3360978148 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630c005040 con 0x7f63180f8420 2024-02-23T00:44:26.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.058+0000 7f6317fff700 1 --2- 172.21.15.47:0/3360978148 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f63180f8420 0x7f631810dd90 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 rx=0x7f630800e970 tx=0x7f630800b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:26.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.058+0000 7f63157fa700 10 client.?.objecter ms_handle_connect 0x7f63180f8420 2024-02-23T00:44:26.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.058+0000 7f63157fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:44:26.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.058+0000 7f63157fa700 1 -- 172.21.15.47:0/3360978148 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6308004660 con 0x7f63180f8420 2024-02-23T00:44:26.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.059+0000 7f631e99f700 1 -- 172.21.15.47:0/3360978148 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63100097c0 con 0x7f63180f8420 2024-02-23T00:44:26.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.059+0000 7f63157fa700 1 -- 172.21.15.47:0/3360978148 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f630800f040 con 0x7f63180f8420 2024-02-23T00:44:26.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.059+0000 7f63157fa700 1 -- 172.21.15.47:0/3360978148 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63080146f0 con 0x7f63180f8420 2024-02-23T00:44:26.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.059+0000 7f631e99f700 10 client.40640.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:44:26.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.059+0000 7f631e99f700 1 -- 172.21.15.47:0/3360978148 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6318114bc0 con 0x7f63180f8420 2024-02-23T00:44:26.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.059+0000 7f631e99f700 1 librados: init done 2024-02-23T00:44:26.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.060+0000 7f631e99f700 1 -- 172.21.15.47:0/3360978148 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6318106ed0 con 0x7f63180f8420 2024-02-23T00:44:26.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.060+0000 7f63157fa700 1 -- 172.21.15.47:0/3360978148 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f6308007b40 con 0x7f63180f8420 2024-02-23T00:44:26.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.060+0000 7f63157fa700 1 --2- 172.21.15.47:0/3360978148 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6300062cc0 0x7f6300065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:26.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.061+0000 7f63157fa700 1 -- 172.21.15.47:0/3360978148 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(193..193 src has 48..193) v4 ==== 10525+0+0 (secure 0 0 0) 0x7f630807d620 con 0x7f63180f8420 2024-02-23T00:44:26.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.061+0000 7f63157fa700 10 client.40640.objecter ms_dispatch 0x7f6318001550 osd_map(193..193 src has 48..193) v4 2024-02-23T00:44:26.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.061+0000 7f63157fa700 3 client.40640.objecter handle_osd_map got epochs [193,193] > 0 2024-02-23T00:44:26.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.061+0000 7f63157fa700 3 client.40640.objecter handle_osd_map decoding full epoch 193 2024-02-23T00:44:26.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.061+0000 7f63177fe700 1 --2- 172.21.15.47:0/3360978148 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6300062cc0 0x7f6300065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:26.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.061+0000 7f63157fa700 20 client.40640.objecter dump_active .. 0 homeless 2024-02-23T00:44:26.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.061+0000 7f63157fa700 10 client.40640.objecter ms_handle_connect 0x7f6300062cc0 2024-02-23T00:44:26.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.061+0000 7f63177fe700 1 --2- 172.21.15.47:0/3360978148 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6300062cc0 0x7f6300065180 secure :-1 s=READY pgs=2942 cs=0 l=1 rev1=1 rx=0x7f63181096d0 tx=0x7f630c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:26.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.065+0000 7f63157fa700 1 -- 172.21.15.47:0/3360978148 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f630804c7d0 con 0x7f63180f8420 2024-02-23T00:44:26.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.273+0000 7f631e99f700 1 -- 172.21.15.47:0/3360978148 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7f631805fa60 con 0x7f63180f8420 2024-02-23T00:44:26.384 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:44:26.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.941+0000 7f1d4d709700 1 -- 172.21.15.47:0/83560849 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1d480f7330 msgr2=0x7f1d48103930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:26.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.941+0000 7f1d4d709700 1 --2- 172.21.15.47:0/83560849 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1d480f7330 0x7f1d48103930 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 rx=0x7f1d40009be0 tx=0x7f1d40004d10).stop 2024-02-23T00:44:26.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.941+0000 7f1d4d709700 1 -- 172.21.15.47:0/83560849 shutdown_connections 2024-02-23T00:44:26.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.941+0000 7f1d4d709700 1 --2- 172.21.15.47:0/83560849 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1d480f7330 0x7f1d48103930 unknown :-1 s=CLOSED pgs=2520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:26.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.941+0000 7f1d4d709700 1 --2- 172.21.15.47:0/83560849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d480fcac0 0x7f1d480fcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:26.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.941+0000 7f1d4d709700 1 --2- 172.21.15.47:0/83560849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d480fd420 0x7f1d480f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:26.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.941+0000 7f1d4d709700 1 -- 172.21.15.47:0/83560849 >> 172.21.15.47:0/83560849 conn(0x7f1d480f2710 msgr2=0x7f1d480f4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:26.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.941+0000 7f1d4d709700 1 -- 172.21.15.47:0/83560849 shutdown_connections 2024-02-23T00:44:26.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.941+0000 7f1d4d709700 1 -- 172.21.15.47:0/83560849 wait complete. 2024-02-23T00:44:26.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.942+0000 7f1d4d709700 1 librados: starting msgr at 2024-02-23T00:44:26.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.942+0000 7f1d4d709700 1 librados: starting objecter 2024-02-23T00:44:26.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.942+0000 7f1d4d709700 1 Processor -- start 2024-02-23T00:44:26.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.942+0000 7f1d4d709700 1 -- start start 2024-02-23T00:44:26.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.942+0000 7f1d4d709700 1 librados: setting wanted keys 2024-02-23T00:44:26.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.942+0000 7f1d4d709700 1 librados: calling monclient init 2024-02-23T00:44:26.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.942+0000 7f1d4d709700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1d480f7330 0x7f1d48108eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:26.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.942+0000 7f1d4d709700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d480fcac0 0x7f1d481093a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:26.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.942+0000 7f1d4d709700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d480fd420 0x7f1d48109980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:26.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.942+0000 7f1d4d709700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d48114090 con 0x7f1d480fcac0 2024-02-23T00:44:26.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.942+0000 7f1d4d709700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d48114200 con 0x7f1d480fd420 2024-02-23T00:44:26.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.942+0000 7f1d4d709700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d481143c0 con 0x7f1d480f7330 2024-02-23T00:44:26.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.942+0000 7f1d467fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d480fcac0 0x7f1d481093a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:26.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.942+0000 7f1d467fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d480fcac0 0x7f1d481093a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40098/0 (socket says 172.21.15.47:40098) 2024-02-23T00:44:26.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.942+0000 7f1d46ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1d480f7330 0x7f1d48108eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:26.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.942+0000 7f1d467fc700 1 -- 172.21.15.47:0/3943936498 learned_addr learned my addr 172.21.15.47:0/3943936498 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:44:26.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.942+0000 7f1d477fe700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d480fd420 0x7f1d48109980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:26.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.943+0000 7f1d467fc700 1 -- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1d480f7330 msgr2=0x7f1d48108eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:26.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.943+0000 7f1d467fc700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1d480f7330 0x7f1d48108eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:26.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.943+0000 7f1d467fc700 1 -- 172.21.15.47:0/3943936498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d480fd420 msgr2=0x7f1d48109980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:26.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.943+0000 7f1d467fc700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d480fd420 0x7f1d48109980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:26.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.943+0000 7f1d467fc700 1 -- 172.21.15.47:0/3943936498 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d38005040 con 0x7f1d480fcac0 2024-02-23T00:44:26.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.943+0000 7f1d46ffd700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1d480f7330 0x7f1d48108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:44:26.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.943+0000 7f1d467fc700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d480fcac0 0x7f1d481093a0 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 rx=0x7f1d3c00e970 tx=0x7f1d3c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:26.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.943+0000 7f1d2ffff700 10 client.?.objecter ms_handle_connect 0x7f1d480fcac0 2024-02-23T00:44:26.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.943+0000 7f1d2ffff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:44:26.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.943+0000 7f1d2ffff700 1 -- 172.21.15.47:0/3943936498 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c004660 con 0x7f1d480fcac0 2024-02-23T00:44:26.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.943+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d400097c0 con 0x7f1d480fcac0 2024-02-23T00:44:26.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.943+0000 7f1d2ffff700 1 -- 172.21.15.47:0/3943936498 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f1d3c00f040 con 0x7f1d480fcac0 2024-02-23T00:44:26.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.943+0000 7f1d2ffff700 1 -- 172.21.15.47:0/3943936498 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c0146f0 con 0x7f1d480fcac0 2024-02-23T00:44:26.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.943+0000 7f1d4d709700 10 client.30303.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:44:26.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.943+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d48114ad0 con 0x7f1d480fcac0 2024-02-23T00:44:26.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.943+0000 7f1d4d709700 1 librados: init done 2024-02-23T00:44:26.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.944+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f1d480606d0 con 0x7f1d480fcac0 2024-02-23T00:44:26.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.945+0000 7f1d2ffff700 1 -- 172.21.15.47:0/3943936498 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f1d3c007b40 con 0x7f1d480fcac0 2024-02-23T00:44:26.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.945+0000 7f1d2ffff700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1d30062cc0 0x7f1d30065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:26.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.945+0000 7f1d2ffff700 1 -- 172.21.15.47:0/3943936498 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(194..194 src has 48..194) v4 ==== 11446+0+0 (secure 0 0 0) 0x7f1d3c07def0 con 0x7f1d480fcac0 2024-02-23T00:44:26.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.945+0000 7f1d2ffff700 10 client.30303.objecter ms_dispatch 0x7f1d48001550 osd_map(194..194 src has 48..194) v4 2024-02-23T00:44:26.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.945+0000 7f1d2ffff700 3 client.30303.objecter handle_osd_map got epochs [194,194] > 0 2024-02-23T00:44:26.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.945+0000 7f1d2ffff700 3 client.30303.objecter handle_osd_map decoding full epoch 194 2024-02-23T00:44:26.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.945+0000 7f1d46ffd700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1d30062cc0 0x7f1d30065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:26.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.945+0000 7f1d2ffff700 20 client.30303.objecter dump_active .. 0 homeless 2024-02-23T00:44:26.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.945+0000 7f1d2ffff700 1 -- 172.21.15.47:0/3943936498 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v194) v1 ==== 54+0+16 (secure 0 0 0) 0x7f1d3c07e390 con 0x7f1d480fcac0 2024-02-23T00:44:26.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.946+0000 7f1d4d709700 10 client.30303.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:26.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.946+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_assign -1 1 2024-02-23T00:44:26.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.946+0000 7f1d4d709700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f1d480fdd90 0x7f1d480fe170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:26.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.946+0000 7f1d2ffff700 10 client.30303.objecter ms_handle_connect 0x7f1d30062cc0 2024-02-23T00:44:26.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.946+0000 7f1d4d709700 20 client.30303.objecter _get_session s=0x7f1d4805ee80 osd=0 3 2024-02-23T00:44:26.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.946+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d4805ee80 osd=0 3 2024-02-23T00:44:26.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.946+0000 7f1d46ffd700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1d30062cc0 0x7f1d30065180 secure :-1 s=READY pgs=2943 cs=0 l=1 rev1=1 rx=0x7f1d38009760 tx=0x7f1d3800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:26.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.946+0000 7f1d4d709700 20 client.30303.objecter _get_session s=0x7f1d4805ee80 osd=0 3 2024-02-23T00:44:26.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.946+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_remove -1 1 2024-02-23T00:44:26.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.946+0000 7f1d4d709700 20 client.30303.objecter get_session s=0x7f1d4805ee80 osd=0 3 2024-02-23T00:44:26.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.946+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_assign 0 1 2024-02-23T00:44:26.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.946+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d4805ee80 osd=0 4 2024-02-23T00:44:26.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.946+0000 7f1d4d709700 10 client.30303.objecter _send_command 1 2024-02-23T00:44:26.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.946+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1d48105de0 con 0x7f1d480fdd90 2024-02-23T00:44:26.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.946+0000 7f1d477fe700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f1d480fdd90 0x7f1d480fe170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:26.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.946+0000 7f1d2ffff700 10 client.30303.objecter ms_handle_connect 0x7f1d480fdd90 2024-02-23T00:44:26.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.946+0000 7f1d477fe700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f1d480fdd90 0x7f1d480fe170 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:26.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.947+0000 7f1d2ffff700 1 -- 172.21.15.47:0/3943936498 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1d48105de0 con 0x7f1d480fdd90 2024-02-23T00:44:26.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.947+0000 7f1d2ffff700 10 client.30303.objecter ms_dispatch 0x7f1d48001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:44:26.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.947+0000 7f1d2ffff700 10 client.30303.objecter _finish_command 1 = system:0 2024-02-23T00:44:26.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.947+0000 7f1d2ffff700 20 client.30303.objecter put_session s=0x7f1d4805ee80 osd=0 4 2024-02-23T00:44:26.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.947+0000 7f1d2ffff700 15 client.30303.objecter _session_command_op_remove 0 1 2024-02-23T00:44:26.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.962+0000 7f1d4d709700 10 client.30303.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:44:26.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.962+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_assign -1 2 2024-02-23T00:44:26.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.962+0000 7f1d4d709700 20 client.30303.objecter _get_session s=0x7f1d4805ee80 osd=0 3 2024-02-23T00:44:26.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.962+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d4805ee80 osd=0 3 2024-02-23T00:44:26.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.962+0000 7f1d4d709700 20 client.30303.objecter _get_session s=0x7f1d4805ee80 osd=0 3 2024-02-23T00:44:26.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.962+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_remove -1 2 2024-02-23T00:44:26.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.962+0000 7f1d4d709700 20 client.30303.objecter get_session s=0x7f1d4805ee80 osd=0 3 2024-02-23T00:44:26.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.962+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_assign 0 2 2024-02-23T00:44:26.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.962+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d4805ee80 osd=0 4 2024-02-23T00:44:26.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.962+0000 7f1d4d709700 10 client.30303.objecter _send_command 2 2024-02-23T00:44:26.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.962+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f1d480273c0 con 0x7f1d480fdd90 2024-02-23T00:44:26.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.963+0000 7f1d2ffff700 1 -- 172.21.15.47:0/3943936498 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f1d480273c0 con 0x7f1d480fdd90 2024-02-23T00:44:26.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.963+0000 7f1d2ffff700 10 client.30303.objecter ms_dispatch 0x7f1d48001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:44:26.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.963+0000 7f1d2ffff700 10 client.30303.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:44:26.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.963+0000 7f1d2ffff700 20 client.30303.objecter put_session s=0x7f1d4805ee80 osd=0 4 2024-02-23T00:44:26.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.963+0000 7f1d2ffff700 15 client.30303.objecter _session_command_op_remove 0 2 2024-02-23T00:44:26.966 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:44:26.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.963+0000 7f1d4d709700 10 client.30303.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:26.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.963+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_assign -1 3 2024-02-23T00:44:26.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.963+0000 7f1d4d709700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f1d480598f0 0x7f1d48059cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:26.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.963+0000 7f1d4d709700 20 client.30303.objecter _get_session s=0x7f1d48028710 osd=1 3 2024-02-23T00:44:26.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.963+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d48028710 osd=1 3 2024-02-23T00:44:26.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.963+0000 7f1d4d709700 20 client.30303.objecter _get_session s=0x7f1d48028710 osd=1 3 2024-02-23T00:44:26.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.963+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_remove -1 3 2024-02-23T00:44:26.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.963+0000 7f1d4d709700 20 client.30303.objecter get_session s=0x7f1d48028710 osd=1 3 2024-02-23T00:44:26.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.963+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_assign 1 3 2024-02-23T00:44:26.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.963+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d48028710 osd=1 4 2024-02-23T00:44:26.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.963+0000 7f1d4d709700 10 client.30303.objecter _send_command 3 2024-02-23T00:44:26.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.963+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1d4811b4d0 con 0x7f1d480598f0 2024-02-23T00:44:26.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.964+0000 7f1d46ffd700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f1d480598f0 0x7f1d48059cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:26.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.964+0000 7f1d46ffd700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f1d480598f0 0x7f1d48059cf0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:26.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.964+0000 7f1d2ffff700 10 client.30303.objecter ms_handle_connect 0x7f1d480598f0 2024-02-23T00:44:26.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.964+0000 7f1d2ffff700 1 -- 172.21.15.47:0/3943936498 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1d4811b4d0 con 0x7f1d480598f0 2024-02-23T00:44:26.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.964+0000 7f1d2ffff700 10 client.30303.objecter ms_dispatch 0x7f1d48001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:44:26.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.964+0000 7f1d2ffff700 10 client.30303.objecter _finish_command 3 = system:0 2024-02-23T00:44:26.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.964+0000 7f1d2ffff700 20 client.30303.objecter put_session s=0x7f1d48028710 osd=1 4 2024-02-23T00:44:26.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.964+0000 7f1d2ffff700 15 client.30303.objecter _session_command_op_remove 1 3 2024-02-23T00:44:26.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.979+0000 7f1d4d709700 10 client.30303.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:44:26.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.979+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_assign -1 4 2024-02-23T00:44:26.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.979+0000 7f1d4d709700 20 client.30303.objecter _get_session s=0x7f1d48028710 osd=1 3 2024-02-23T00:44:26.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.979+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d48028710 osd=1 3 2024-02-23T00:44:26.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.979+0000 7f1d4d709700 20 client.30303.objecter _get_session s=0x7f1d48028710 osd=1 3 2024-02-23T00:44:26.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.979+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_remove -1 4 2024-02-23T00:44:26.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.979+0000 7f1d4d709700 20 client.30303.objecter get_session s=0x7f1d48028710 osd=1 3 2024-02-23T00:44:26.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.979+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_assign 1 4 2024-02-23T00:44:26.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.979+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d48028710 osd=1 4 2024-02-23T00:44:26.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.979+0000 7f1d4d709700 10 client.30303.objecter _send_command 4 2024-02-23T00:44:26.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.979+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f1d48059360 con 0x7f1d480598f0 2024-02-23T00:44:26.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.981+0000 7f1d2ffff700 1 -- 172.21.15.47:0/3943936498 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f1d48059360 con 0x7f1d480598f0 2024-02-23T00:44:26.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.981+0000 7f1d2ffff700 10 client.30303.objecter ms_dispatch 0x7f1d48001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:44:26.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.981+0000 7f1d2ffff700 10 client.30303.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:44:26.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.981+0000 7f1d2ffff700 20 client.30303.objecter put_session s=0x7f1d48028710 osd=1 4 2024-02-23T00:44:26.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.981+0000 7f1d2ffff700 15 client.30303.objecter _session_command_op_remove 1 4 2024-02-23T00:44:26.983 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:44:26.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.981+0000 7f1d4d709700 10 client.30303.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:26.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.981+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_assign -1 5 2024-02-23T00:44:26.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.981+0000 7f1d4d709700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f1d48028270 0x7f1d48119ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:26.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.981+0000 7f1d4d709700 20 client.30303.objecter _get_session s=0x7f1d48059710 osd=2 3 2024-02-23T00:44:26.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.981+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d48059710 osd=2 3 2024-02-23T00:44:26.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.981+0000 7f1d4d709700 20 client.30303.objecter _get_session s=0x7f1d48059710 osd=2 3 2024-02-23T00:44:26.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.981+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_remove -1 5 2024-02-23T00:44:26.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.981+0000 7f1d4d709700 20 client.30303.objecter get_session s=0x7f1d48059710 osd=2 3 2024-02-23T00:44:26.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.981+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_assign 2 5 2024-02-23T00:44:26.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.981+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d48059710 osd=2 4 2024-02-23T00:44:26.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.981+0000 7f1d4d709700 10 client.30303.objecter _send_command 5 2024-02-23T00:44:26.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.981+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1d4811a3e0 con 0x7f1d48028270 2024-02-23T00:44:26.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.981+0000 7f1d467fc700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f1d48028270 0x7f1d48119ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:26.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.981+0000 7f1d2ffff700 10 client.30303.objecter ms_handle_connect 0x7f1d48028270 2024-02-23T00:44:26.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.981+0000 7f1d467fc700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f1d48028270 0x7f1d48119ef0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:26.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.982+0000 7f1d2ffff700 1 -- 172.21.15.47:0/3943936498 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1d4811a3e0 con 0x7f1d48028270 2024-02-23T00:44:26.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.982+0000 7f1d2ffff700 10 client.30303.objecter ms_dispatch 0x7f1d48001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:44:26.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.982+0000 7f1d2ffff700 10 client.30303.objecter _finish_command 5 = system:0 2024-02-23T00:44:26.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.982+0000 7f1d2ffff700 20 client.30303.objecter put_session s=0x7f1d48059710 osd=2 4 2024-02-23T00:44:26.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.982+0000 7f1d2ffff700 15 client.30303.objecter _session_command_op_remove 2 5 2024-02-23T00:44:26.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.996+0000 7f1d4d709700 10 client.30303.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:44:26.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.996+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_assign -1 6 2024-02-23T00:44:26.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.996+0000 7f1d4d709700 20 client.30303.objecter _get_session s=0x7f1d48059710 osd=2 3 2024-02-23T00:44:26.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.996+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d48059710 osd=2 3 2024-02-23T00:44:26.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.996+0000 7f1d4d709700 20 client.30303.objecter _get_session s=0x7f1d48059710 osd=2 3 2024-02-23T00:44:26.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.996+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_remove -1 6 2024-02-23T00:44:26.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.996+0000 7f1d4d709700 20 client.30303.objecter get_session s=0x7f1d48059710 osd=2 3 2024-02-23T00:44:26.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.996+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_assign 2 6 2024-02-23T00:44:26.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.996+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d48059710 osd=2 4 2024-02-23T00:44:26.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.996+0000 7f1d4d709700 10 client.30303.objecter _send_command 6 2024-02-23T00:44:26.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.996+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f1d480277d0 con 0x7f1d48028270 2024-02-23T00:44:27.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.998+0000 7f1d2ffff700 1 -- 172.21.15.47:0/3943936498 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f1d480277d0 con 0x7f1d48028270 2024-02-23T00:44:27.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.998+0000 7f1d2ffff700 10 client.30303.objecter ms_dispatch 0x7f1d48001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:44:27.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.998+0000 7f1d2ffff700 10 client.30303.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:44:27.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.998+0000 7f1d2ffff700 20 client.30303.objecter put_session s=0x7f1d48059710 osd=2 4 2024-02-23T00:44:27.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.998+0000 7f1d2ffff700 15 client.30303.objecter _session_command_op_remove 2 6 2024-02-23T00:44:27.000 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:44:27.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.998+0000 7f1d4d709700 10 client.30303.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:27.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.998+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_assign -1 7 2024-02-23T00:44:27.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.998+0000 7f1d4d709700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f1d4811d650 0x7f1d4811fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.998+0000 7f1d4d709700 20 client.30303.objecter _get_session s=0x7f1d48027be0 osd=3 3 2024-02-23T00:44:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.998+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d48027be0 osd=3 3 2024-02-23T00:44:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.998+0000 7f1d4d709700 20 client.30303.objecter _get_session s=0x7f1d48027be0 osd=3 3 2024-02-23T00:44:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.998+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_remove -1 7 2024-02-23T00:44:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.998+0000 7f1d4d709700 20 client.30303.objecter get_session s=0x7f1d48027be0 osd=3 3 2024-02-23T00:44:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.998+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_assign 3 7 2024-02-23T00:44:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.998+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d48027be0 osd=3 4 2024-02-23T00:44:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.998+0000 7f1d4d709700 10 client.30303.objecter _send_command 7 2024-02-23T00:44:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.998+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1d48122690 con 0x7f1d4811d650 2024-02-23T00:44:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.998+0000 7f1d477fe700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f1d4811d650 0x7f1d4811fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.998+0000 7f1d2ffff700 10 client.30303.objecter ms_handle_connect 0x7f1d4811d650 2024-02-23T00:44:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.998+0000 7f1d477fe700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f1d4811d650 0x7f1d4811fa40 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.999+0000 7f1d2ffff700 1 -- 172.21.15.47:0/3943936498 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1d48122690 con 0x7f1d4811d650 2024-02-23T00:44:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.999+0000 7f1d2ffff700 10 client.30303.objecter ms_dispatch 0x7f1d48001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:44:27.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.999+0000 7f1d2ffff700 10 client.30303.objecter _finish_command 7 = system:0 2024-02-23T00:44:27.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.999+0000 7f1d2ffff700 20 client.30303.objecter put_session s=0x7f1d48027be0 osd=3 4 2024-02-23T00:44:27.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:26.999+0000 7f1d2ffff700 15 client.30303.objecter _session_command_op_remove 3 7 2024-02-23T00:44:27.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.014+0000 7f1d4d709700 10 client.30303.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:44:27.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.014+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_assign -1 8 2024-02-23T00:44:27.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.014+0000 7f1d4d709700 20 client.30303.objecter _get_session s=0x7f1d48027be0 osd=3 3 2024-02-23T00:44:27.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.014+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d48027be0 osd=3 3 2024-02-23T00:44:27.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.014+0000 7f1d4d709700 20 client.30303.objecter _get_session s=0x7f1d48027be0 osd=3 3 2024-02-23T00:44:27.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.014+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_remove -1 8 2024-02-23T00:44:27.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.014+0000 7f1d4d709700 20 client.30303.objecter get_session s=0x7f1d48027be0 osd=3 3 2024-02-23T00:44:27.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.014+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_assign 3 8 2024-02-23T00:44:27.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.014+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d48027be0 osd=3 4 2024-02-23T00:44:27.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.014+0000 7f1d4d709700 10 client.30303.objecter _send_command 8 2024-02-23T00:44:27.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.014+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f1d4811a790 con 0x7f1d4811d650 2024-02-23T00:44:27.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.015+0000 7f1d2ffff700 1 -- 172.21.15.47:0/3943936498 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f1d4811a790 con 0x7f1d4811d650 2024-02-23T00:44:27.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.015+0000 7f1d2ffff700 10 client.30303.objecter ms_dispatch 0x7f1d48001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:44:27.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.015+0000 7f1d2ffff700 10 client.30303.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:44:27.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.015+0000 7f1d2ffff700 20 client.30303.objecter put_session s=0x7f1d48027be0 osd=3 4 2024-02-23T00:44:27.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.015+0000 7f1d2ffff700 15 client.30303.objecter _session_command_op_remove 3 8 2024-02-23T00:44:27.018 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:44:27.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.015+0000 7f1d4d709700 10 client.30303.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:27.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.015+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_assign -1 9 2024-02-23T00:44:27.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.015+0000 7f1d4d709700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f1d4811a900 0x7f1d481210b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:27.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.015+0000 7f1d4d709700 20 client.30303.objecter _get_session s=0x7f1d4811a790 osd=4 3 2024-02-23T00:44:27.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.015+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d4811a790 osd=4 3 2024-02-23T00:44:27.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.015+0000 7f1d4d709700 20 client.30303.objecter _get_session s=0x7f1d4811a790 osd=4 3 2024-02-23T00:44:27.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.015+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_remove -1 9 2024-02-23T00:44:27.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.015+0000 7f1d4d709700 20 client.30303.objecter get_session s=0x7f1d4811a790 osd=4 3 2024-02-23T00:44:27.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.015+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_assign 4 9 2024-02-23T00:44:27.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.015+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d4811a790 osd=4 4 2024-02-23T00:44:27.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.015+0000 7f1d4d709700 10 client.30303.objecter _send_command 9 2024-02-23T00:44:27.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.015+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1d48119b00 con 0x7f1d4811a900 2024-02-23T00:44:27.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.016+0000 7f1d46ffd700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f1d4811a900 0x7f1d481210b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:27.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.016+0000 7f1d2ffff700 10 client.30303.objecter ms_handle_connect 0x7f1d4811a900 2024-02-23T00:44:27.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.016+0000 7f1d46ffd700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f1d4811a900 0x7f1d481210b0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:27.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.017+0000 7f1d2ffff700 1 -- 172.21.15.47:0/3943936498 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1d48119b00 con 0x7f1d4811a900 2024-02-23T00:44:27.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.017+0000 7f1d2ffff700 10 client.30303.objecter ms_dispatch 0x7f1d48001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:44:27.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.017+0000 7f1d2ffff700 10 client.30303.objecter _finish_command 9 = system:0 2024-02-23T00:44:27.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.017+0000 7f1d2ffff700 20 client.30303.objecter put_session s=0x7f1d4811a790 osd=4 4 2024-02-23T00:44:27.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.017+0000 7f1d2ffff700 15 client.30303.objecter _session_command_op_remove 4 9 2024-02-23T00:44:27.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.032+0000 7f1d4d709700 10 client.30303.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:44:27.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.032+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_assign -1 10 2024-02-23T00:44:27.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.032+0000 7f1d4d709700 20 client.30303.objecter _get_session s=0x7f1d4811a790 osd=4 3 2024-02-23T00:44:27.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.032+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d4811a790 osd=4 3 2024-02-23T00:44:27.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.032+0000 7f1d4d709700 20 client.30303.objecter _get_session s=0x7f1d4811a790 osd=4 3 2024-02-23T00:44:27.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.032+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_remove -1 10 2024-02-23T00:44:27.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.032+0000 7f1d4d709700 20 client.30303.objecter get_session s=0x7f1d4811a790 osd=4 3 2024-02-23T00:44:27.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.032+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_assign 4 10 2024-02-23T00:44:27.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.032+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d4811a790 osd=4 4 2024-02-23T00:44:27.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.032+0000 7f1d4d709700 10 client.30303.objecter _send_command 10 2024-02-23T00:44:27.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.032+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f1d48119b00 con 0x7f1d4811a900 2024-02-23T00:44:27.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.033+0000 7f1d2ffff700 1 -- 172.21.15.47:0/3943936498 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f1d48119b00 con 0x7f1d4811a900 2024-02-23T00:44:27.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.033+0000 7f1d2ffff700 10 client.30303.objecter ms_dispatch 0x7f1d48001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:44:27.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.033+0000 7f1d2ffff700 10 client.30303.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:44:27.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.033+0000 7f1d2ffff700 20 client.30303.objecter put_session s=0x7f1d4811a790 osd=4 4 2024-02-23T00:44:27.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.033+0000 7f1d2ffff700 15 client.30303.objecter _session_command_op_remove 4 10 2024-02-23T00:44:27.036 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:44:27.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.033+0000 7f1d4d709700 10 client.30303.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:27.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.033+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_assign -1 11 2024-02-23T00:44:27.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.033+0000 7f1d4d709700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f1d48121ab0 0x7f1d48121ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:27.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.033+0000 7f1d4d709700 20 client.30303.objecter _get_session s=0x7f1d48119b00 osd=5 3 2024-02-23T00:44:27.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.033+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d48119b00 osd=5 3 2024-02-23T00:44:27.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.033+0000 7f1d4d709700 20 client.30303.objecter _get_session s=0x7f1d48119b00 osd=5 3 2024-02-23T00:44:27.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.033+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_remove -1 11 2024-02-23T00:44:27.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.033+0000 7f1d4d709700 20 client.30303.objecter get_session s=0x7f1d48119b00 osd=5 3 2024-02-23T00:44:27.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.033+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_assign 5 11 2024-02-23T00:44:27.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.033+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d48119b00 osd=5 4 2024-02-23T00:44:27.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.033+0000 7f1d4d709700 10 client.30303.objecter _send_command 11 2024-02-23T00:44:27.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.033+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1d48122430 con 0x7f1d48121ab0 2024-02-23T00:44:27.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.034+0000 7f1d467fc700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f1d48121ab0 0x7f1d48121ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:27.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.034+0000 7f1d2ffff700 10 client.30303.objecter ms_handle_connect 0x7f1d48121ab0 2024-02-23T00:44:27.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.034+0000 7f1d467fc700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f1d48121ab0 0x7f1d48121ef0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:27.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.035+0000 7f1d2ffff700 1 -- 172.21.15.47:0/3943936498 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1d48122430 con 0x7f1d48121ab0 2024-02-23T00:44:27.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.035+0000 7f1d2ffff700 10 client.30303.objecter ms_dispatch 0x7f1d48001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:44:27.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.035+0000 7f1d2ffff700 10 client.30303.objecter _finish_command 11 = system:0 2024-02-23T00:44:27.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.035+0000 7f1d2ffff700 20 client.30303.objecter put_session s=0x7f1d48119b00 osd=5 4 2024-02-23T00:44:27.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.035+0000 7f1d2ffff700 15 client.30303.objecter _session_command_op_remove 5 11 2024-02-23T00:44:27.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:24.756660+0000 osd.7 (osd.7) 111 : cluster [DBG] 6.3 scrub starts 2024-02-23T00:44:27.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:24.759624+0000 osd.7 (osd.7) 112 : cluster [DBG] 6.3 scrub ok 2024-02-23T00:44:27.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:25.223231+0000 mgr.y (mgr.14517) 233 : cluster [DBG] pgmap v265: 282 pgs: 250 active+clean, 32 unknown; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:44:27.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:25.468636+0000 osd.2 (osd.2) 87 : cluster [DBG] 5.d scrub starts 2024-02-23T00:44:27.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:25.470170+0000 osd.2 (osd.2) 88 : cluster [DBG] 5.d scrub ok 2024-02-23T00:44:27.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:25.756459+0000 mon.b (mon.1) 241 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:27.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:25.794290+0000 osd.7 (osd.7) 113 : cluster [DBG] 5.10 scrub starts 2024-02-23T00:44:27.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:25.797180+0000 osd.7 (osd.7) 114 : cluster [DBG] 5.10 scrub ok 2024-02-23T00:44:27.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:25.995308+0000 osd.5 (osd.5) 101 : cluster [DBG] 4.15 scrub starts 2024-02-23T00:44:27.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:25.998798+0000 osd.5 (osd.5) 102 : cluster [DBG] 4.15 scrub ok 2024-02-23T00:44:27.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:26.110413+0000 mon.a (mon.0) 1400 : audit [INF] from='client.? 172.21.15.47:0/2361769855' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi047-82378-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi047-82378-26"}]': finished 2024-02-23T00:44:27.043 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:26.110638+0000 mon.a (mon.0) 1401 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi047-82457-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:27.043 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:26.273772+0000 mon.c (mon.2) 120 : audit [INF] from='client.? 172.21.15.47:0/3360978148' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:44:27.043 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:26.557004+0000 mon.a (mon.0) 1402 : cluster [DBG] osdmap e194: 8 total, 8 up, 8 in 2024-02-23T00:44:27.043 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:26.558071+0000 mon.b (mon.1) 242 : audit [INF] from='client.? 172.21.15.47:0/2464885804' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi047-82386-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:27.043 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:26.561635+0000 mon.a (mon.0) 1403 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:44:27.043 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:26.562156+0000 mon.a (mon.0) 1404 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi047-82386-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:27.043 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:26.594531+0000 osd.0 (osd.0) 99 : cluster [DBG] 18.0 scrub starts 2024-02-23T00:44:27.043 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:26.595340+0000 osd.0 (osd.0) 100 : cluster [DBG] 18.0 scrub ok 2024-02-23T00:44:27.043 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:26.757411+0000 mon.b (mon.1) 243 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:27.043 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:26.945880+0000 mon.a (mon.0) 1405 : audit [DBG] from='client.? 172.21.15.47:0/3943936498' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:44:27.044 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:24.756660+0000 osd.7 (osd.7) 111 : cluster [DBG] 6.3 scrub starts 2024-02-23T00:44:27.044 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:24.759624+0000 osd.7 (osd.7) 112 : cluster [DBG] 6.3 scrub ok 2024-02-23T00:44:27.044 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:25.223231+0000 mgr.y (mgr.14517) 233 : cluster [DBG] pgmap v265: 282 pgs: 250 active+clean, 32 unknown; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:44:27.044 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:25.468636+0000 osd.2 (osd.2) 87 : cluster [DBG] 5.d scrub starts 2024-02-23T00:44:27.044 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:25.470170+0000 osd.2 (osd.2) 88 : cluster [DBG] 5.d scrub ok 2024-02-23T00:44:27.044 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:25.756459+0000 mon.b (mon.1) 241 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:27.044 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:25.794290+0000 osd.7 (osd.7) 113 : cluster [DBG] 5.10 scrub starts 2024-02-23T00:44:27.044 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:25.797180+0000 osd.7 (osd.7) 114 : cluster [DBG] 5.10 scrub ok 2024-02-23T00:44:27.044 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:25.995308+0000 osd.5 (osd.5) 101 : cluster [DBG] 4.15 scrub starts 2024-02-23T00:44:27.044 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:25.998798+0000 osd.5 (osd.5) 102 : cluster [DBG] 4.15 scrub ok 2024-02-23T00:44:27.045 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:26.110413+0000 mon.a (mon.0) 1400 : audit [INF] from='client.? 172.21.15.47:0/2361769855' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi047-82378-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi047-82378-26"}]': finished 2024-02-23T00:44:27.045 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:26.110638+0000 mon.a (mon.0) 1401 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi047-82457-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:27.045 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:26.273772+0000 mon.c (mon.2) 120 : audit [INF] from='client.? 172.21.15.47:0/3360978148' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:44:27.045 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:26.557004+0000 mon.a (mon.0) 1402 : cluster [DBG] osdmap e194: 8 total, 8 up, 8 in 2024-02-23T00:44:27.045 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:26.558071+0000 mon.b (mon.1) 242 : audit [INF] from='client.? 172.21.15.47:0/2464885804' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi047-82386-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:27.045 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:26.561635+0000 mon.a (mon.0) 1403 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:44:27.045 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:26.562156+0000 mon.a (mon.0) 1404 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi047-82386-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:27.045 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:26.594531+0000 osd.0 (osd.0) 99 : cluster [DBG] 18.0 scrub starts 2024-02-23T00:44:27.045 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:26.595340+0000 osd.0 (osd.0) 100 : cluster [DBG] 18.0 scrub ok 2024-02-23T00:44:27.045 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:26.757411+0000 mon.b (mon.1) 243 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:27.045 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:26.945880+0000 mon.a (mon.0) 1405 : audit [DBG] from='client.? 172.21.15.47:0/3943936498' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:44:27.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.051+0000 7f1d4d709700 10 client.30303.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:44:27.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.051+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_assign -1 12 2024-02-23T00:44:27.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.051+0000 7f1d4d709700 20 client.30303.objecter _get_session s=0x7f1d48119b00 osd=5 3 2024-02-23T00:44:27.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.051+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d48119b00 osd=5 3 2024-02-23T00:44:27.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.051+0000 7f1d4d709700 20 client.30303.objecter _get_session s=0x7f1d48119b00 osd=5 3 2024-02-23T00:44:27.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.051+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_remove -1 12 2024-02-23T00:44:27.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.051+0000 7f1d4d709700 20 client.30303.objecter get_session s=0x7f1d48119b00 osd=5 3 2024-02-23T00:44:27.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.051+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_assign 5 12 2024-02-23T00:44:27.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.051+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d48119b00 osd=5 4 2024-02-23T00:44:27.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.051+0000 7f1d4d709700 10 client.30303.objecter _send_command 12 2024-02-23T00:44:27.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.051+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f1d48122430 con 0x7f1d48121ab0 2024-02-23T00:44:27.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.053+0000 7f1d2ffff700 1 -- 172.21.15.47:0/3943936498 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f1d48122430 con 0x7f1d48121ab0 2024-02-23T00:44:27.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.053+0000 7f1d2ffff700 10 client.30303.objecter ms_dispatch 0x7f1d48001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:44:27.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.053+0000 7f1d2ffff700 10 client.30303.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:44:27.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.053+0000 7f1d2ffff700 20 client.30303.objecter put_session s=0x7f1d48119b00 osd=5 4 2024-02-23T00:44:27.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.053+0000 7f1d2ffff700 15 client.30303.objecter _session_command_op_remove 5 12 2024-02-23T00:44:27.056 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:44:27.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.053+0000 7f1d4d709700 10 client.30303.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:27.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.053+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_assign -1 13 2024-02-23T00:44:27.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.053+0000 7f1d4d709700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f1d48126d30 0x7f1d48129180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:27.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.053+0000 7f1d4d709700 20 client.30303.objecter _get_session s=0x7f1d48122430 osd=6 3 2024-02-23T00:44:27.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.053+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d48122430 osd=6 3 2024-02-23T00:44:27.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.053+0000 7f1d4d709700 20 client.30303.objecter _get_session s=0x7f1d48122430 osd=6 3 2024-02-23T00:44:27.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.053+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_remove -1 13 2024-02-23T00:44:27.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.053+0000 7f1d4d709700 20 client.30303.objecter get_session s=0x7f1d48122430 osd=6 3 2024-02-23T00:44:27.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.053+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_assign 6 13 2024-02-23T00:44:27.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.053+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d48122430 osd=6 4 2024-02-23T00:44:27.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.053+0000 7f1d4d709700 10 client.30303.objecter _send_command 13 2024-02-23T00:44:27.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.053+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1d4812bfc0 con 0x7f1d48126d30 2024-02-23T00:44:27.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.053+0000 7f1d477fe700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f1d48126d30 0x7f1d48129180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:27.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.054+0000 7f1d2ffff700 10 client.30303.objecter ms_handle_connect 0x7f1d48126d30 2024-02-23T00:44:27.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.054+0000 7f1d477fe700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f1d48126d30 0x7f1d48129180 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:27.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.055+0000 7f1d2ffff700 1 -- 172.21.15.47:0/3943936498 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1d4812bfc0 con 0x7f1d48126d30 2024-02-23T00:44:27.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.055+0000 7f1d2ffff700 10 client.30303.objecter ms_dispatch 0x7f1d48001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:44:27.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.055+0000 7f1d2ffff700 10 client.30303.objecter _finish_command 13 = system:0 2024-02-23T00:44:27.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.055+0000 7f1d2ffff700 20 client.30303.objecter put_session s=0x7f1d48122430 osd=6 4 2024-02-23T00:44:27.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.055+0000 7f1d2ffff700 15 client.30303.objecter _session_command_op_remove 6 13 2024-02-23T00:44:27.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.070+0000 7f1d4d709700 10 client.30303.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:44:27.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.070+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_assign -1 14 2024-02-23T00:44:27.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.070+0000 7f1d4d709700 20 client.30303.objecter _get_session s=0x7f1d48122430 osd=6 3 2024-02-23T00:44:27.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.070+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d48122430 osd=6 3 2024-02-23T00:44:27.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.070+0000 7f1d4d709700 20 client.30303.objecter _get_session s=0x7f1d48122430 osd=6 3 2024-02-23T00:44:27.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.070+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_remove -1 14 2024-02-23T00:44:27.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.070+0000 7f1d4d709700 20 client.30303.objecter get_session s=0x7f1d48122430 osd=6 3 2024-02-23T00:44:27.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.070+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_assign 6 14 2024-02-23T00:44:27.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.070+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d48122430 osd=6 4 2024-02-23T00:44:27.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.070+0000 7f1d4d709700 10 client.30303.objecter _send_command 14 2024-02-23T00:44:27.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.070+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f1d4812bfc0 con 0x7f1d48126d30 2024-02-23T00:44:27.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.071+0000 7f1d2ffff700 1 -- 172.21.15.47:0/3943936498 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f1d4812bfc0 con 0x7f1d48126d30 2024-02-23T00:44:27.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.071+0000 7f1d2ffff700 10 client.30303.objecter ms_dispatch 0x7f1d48001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:44:27.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.071+0000 7f1d2ffff700 10 client.30303.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:44:27.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.071+0000 7f1d2ffff700 20 client.30303.objecter put_session s=0x7f1d48122430 osd=6 4 2024-02-23T00:44:27.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.071+0000 7f1d2ffff700 15 client.30303.objecter _session_command_op_remove 6 14 2024-02-23T00:44:27.074 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:44:27.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.072+0000 7f1d4d709700 10 client.30303.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:27.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.072+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_assign -1 15 2024-02-23T00:44:27.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.072+0000 7f1d4d709700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f1d4812b2f0 0x7f1d4812b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:27.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.072+0000 7f1d4d709700 20 client.30303.objecter _get_session s=0x7f1d4812bfc0 osd=7 3 2024-02-23T00:44:27.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.072+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d4812bfc0 osd=7 3 2024-02-23T00:44:27.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.072+0000 7f1d4d709700 20 client.30303.objecter _get_session s=0x7f1d4812bfc0 osd=7 3 2024-02-23T00:44:27.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.072+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_remove -1 15 2024-02-23T00:44:27.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.072+0000 7f1d4d709700 20 client.30303.objecter get_session s=0x7f1d4812bfc0 osd=7 3 2024-02-23T00:44:27.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.072+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_assign 7 15 2024-02-23T00:44:27.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.072+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d4812bfc0 osd=7 4 2024-02-23T00:44:27.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.072+0000 7f1d4d709700 10 client.30303.objecter _send_command 15 2024-02-23T00:44:27.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.072+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1d4812f2f0 con 0x7f1d4812b2f0 2024-02-23T00:44:27.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.072+0000 7f1d46ffd700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f1d4812b2f0 0x7f1d4812b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:27.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.072+0000 7f1d2ffff700 10 client.30303.objecter ms_handle_connect 0x7f1d4812b2f0 2024-02-23T00:44:27.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.072+0000 7f1d46ffd700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f1d4812b2f0 0x7f1d4812b770 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:27.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.073+0000 7f1d2ffff700 1 -- 172.21.15.47:0/3943936498 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1d4812f2f0 con 0x7f1d4812b2f0 2024-02-23T00:44:27.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.073+0000 7f1d2ffff700 10 client.30303.objecter ms_dispatch 0x7f1d48001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:44:27.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.073+0000 7f1d2ffff700 10 client.30303.objecter _finish_command 15 = system:0 2024-02-23T00:44:27.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.073+0000 7f1d2ffff700 20 client.30303.objecter put_session s=0x7f1d4812bfc0 osd=7 4 2024-02-23T00:44:27.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.073+0000 7f1d2ffff700 15 client.30303.objecter _session_command_op_remove 7 15 2024-02-23T00:44:27.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.088+0000 7f1d4d709700 10 client.30303.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:44:27.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.088+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_assign -1 16 2024-02-23T00:44:27.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.088+0000 7f1d4d709700 20 client.30303.objecter _get_session s=0x7f1d4812bfc0 osd=7 3 2024-02-23T00:44:27.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.088+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d4812bfc0 osd=7 3 2024-02-23T00:44:27.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.088+0000 7f1d4d709700 20 client.30303.objecter _get_session s=0x7f1d4812bfc0 osd=7 3 2024-02-23T00:44:27.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.088+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_remove -1 16 2024-02-23T00:44:27.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.088+0000 7f1d4d709700 20 client.30303.objecter get_session s=0x7f1d4812bfc0 osd=7 3 2024-02-23T00:44:27.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.088+0000 7f1d4d709700 15 client.30303.objecter _session_command_op_assign 7 16 2024-02-23T00:44:27.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.088+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d4812bfc0 osd=7 4 2024-02-23T00:44:27.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.088+0000 7f1d4d709700 10 client.30303.objecter _send_command 16 2024-02-23T00:44:27.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.088+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f1d48129780 con 0x7f1d4812b2f0 2024-02-23T00:44:27.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.090+0000 7f1d2ffff700 1 -- 172.21.15.47:0/3943936498 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f1d48129780 con 0x7f1d4812b2f0 2024-02-23T00:44:27.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.090+0000 7f1d2ffff700 10 client.30303.objecter ms_dispatch 0x7f1d48001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:44:27.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.090+0000 7f1d2ffff700 10 client.30303.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:44:27.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.090+0000 7f1d2ffff700 20 client.30303.objecter put_session s=0x7f1d4812bfc0 osd=7 4 2024-02-23T00:44:27.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.090+0000 7f1d2ffff700 15 client.30303.objecter _session_command_op_remove 7 16 2024-02-23T00:44:27.093 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:44:27.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.090+0000 7f1d4d709700 10 librados: watch_flush enter 2024-02-23T00:44:27.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.090+0000 7f1d4d709700 10 librados: watch_flush exit 2024-02-23T00:44:27.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.090+0000 7f1d4d709700 10 client.30303.objecter close_session for osd.0 2024-02-23T00:44:27.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.090+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f1d480fdd90 msgr2=0x7f1d480fe170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:27.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.090+0000 7f1d4d709700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f1d480fdd90 0x7f1d480fe170 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:27.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.090+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d4805ee80 osd=0 1 2024-02-23T00:44:27.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.090+0000 7f1d4d709700 10 client.30303.objecter close_session for osd.1 2024-02-23T00:44:27.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f1d480598f0 msgr2=0x7f1d48059cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:27.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f1d480598f0 0x7f1d48059cf0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:27.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d48028710 osd=1 1 2024-02-23T00:44:27.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 10 client.30303.objecter close_session for osd.2 2024-02-23T00:44:27.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f1d48028270 msgr2=0x7f1d48119ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:27.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f1d48028270 0x7f1d48119ef0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:27.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d48059710 osd=2 1 2024-02-23T00:44:27.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 10 client.30303.objecter close_session for osd.3 2024-02-23T00:44:27.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f1d4811d650 msgr2=0x7f1d4811fa40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:27.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f1d4811d650 0x7f1d4811fa40 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:27.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d48027be0 osd=3 1 2024-02-23T00:44:27.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 10 client.30303.objecter close_session for osd.4 2024-02-23T00:44:27.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f1d4811a900 msgr2=0x7f1d481210b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:27.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f1d4811a900 0x7f1d481210b0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:27.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d477fe700 1 -- 172.21.15.47:0/3943936498 reap_dead start 2024-02-23T00:44:27.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d4811a790 osd=4 1 2024-02-23T00:44:27.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 10 client.30303.objecter close_session for osd.5 2024-02-23T00:44:27.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f1d48121ab0 msgr2=0x7f1d48121ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:27.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f1d48121ab0 0x7f1d48121ef0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:27.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d477fe700 1 -- 172.21.15.47:0/3943936498 reap_dead start 2024-02-23T00:44:27.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d48119b00 osd=5 1 2024-02-23T00:44:27.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 10 client.30303.objecter close_session for osd.6 2024-02-23T00:44:27.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f1d48126d30 msgr2=0x7f1d48129180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:27.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f1d48126d30 0x7f1d48129180 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:27.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d48122430 osd=6 1 2024-02-23T00:44:27.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 10 client.30303.objecter close_session for osd.7 2024-02-23T00:44:27.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d477fe700 1 -- 172.21.15.47:0/3943936498 reap_dead start 2024-02-23T00:44:27.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f1d4812b2f0 msgr2=0x7f1d4812b770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:27.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f1d4812b2f0 0x7f1d4812b770 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:27.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 20 client.30303.objecter put_session s=0x7f1d4812bfc0 osd=7 1 2024-02-23T00:44:27.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 20 client.30303.objecter shutdown clearing up homeless session... 2024-02-23T00:44:27.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 10 client.30303.objecter successfully canceled tick 2024-02-23T00:44:27.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1d30062cc0 msgr2=0x7f1d30065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:27.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1d30062cc0 0x7f1d30065180 secure :-1 s=READY pgs=2943 cs=0 l=1 rev1=1 rx=0x7f1d38009760 tx=0x7f1d3800b040).stop 2024-02-23T00:44:27.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d480fcac0 msgr2=0x7f1d481093a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:27.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d480fcac0 0x7f1d481093a0 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 rx=0x7f1d3c00e970 tx=0x7f1d3c00b040).stop 2024-02-23T00:44:27.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 shutdown_connections 2024-02-23T00:44:27.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d480fcac0 0x7f1d481093a0 unknown :-1 s=CLOSED pgs=2449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:27.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1d30062cc0 0x7f1d30065180 unknown :-1 s=CLOSED pgs=2943 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:27.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 1 --2- 172.21.15.47:0/3943936498 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f1d4812b2f0 0x7f1d4812b770 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:27.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 >> 172.21.15.47:0/3943936498 conn(0x7f1d480f2710 msgr2=0x7f1d480fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:27.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 shutdown_connections 2024-02-23T00:44:27.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 1 -- 172.21.15.47:0/3943936498 wait complete. 2024-02-23T00:44:27.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.091+0000 7f1d4d709700 1 librados: shutdown 2024-02-23T00:44:27.098 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:44:27.098 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:44:27.098 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:44:27.098 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:44:27.098 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:44:27.099 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:44:27.099 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:44:27.099 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:44:27.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.118+0000 7f63157fa700 1 -- 172.21.15.47:0/3360978148 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v195) v1 ==== 101+0+0 (secure 0 0 0) 0x7f6308019070 con 0x7f63180f8420 2024-02-23T00:44:27.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.126+0000 7f631e99f700 10 librados: watch_flush enter 2024-02-23T00:44:27.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.133+0000 7f631e99f700 10 librados: watch_flush exit 2024-02-23T00:44:27.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.133+0000 7f631e99f700 20 client.40640.objecter shutdown clearing up homeless session... 2024-02-23T00:44:27.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.133+0000 7f631e99f700 10 client.40640.objecter successfully canceled tick 2024-02-23T00:44:27.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.133+0000 7f631e99f700 1 -- 172.21.15.47:0/3360978148 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6300062cc0 msgr2=0x7f6300065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:27.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.133+0000 7f631e99f700 1 --2- 172.21.15.47:0/3360978148 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6300062cc0 0x7f6300065180 secure :-1 s=READY pgs=2942 cs=0 l=1 rev1=1 rx=0x7f63181096d0 tx=0x7f630c00b040).stop 2024-02-23T00:44:27.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.133+0000 7f631e99f700 1 -- 172.21.15.47:0/3360978148 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f63180f8420 msgr2=0x7f631810dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:27.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.133+0000 7f631e99f700 1 --2- 172.21.15.47:0/3360978148 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f63180f8420 0x7f631810dd90 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 rx=0x7f630800e970 tx=0x7f630800b040).stop 2024-02-23T00:44:27.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.134+0000 7f631e99f700 1 -- 172.21.15.47:0/3360978148 shutdown_connections 2024-02-23T00:44:27.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.134+0000 7f631e99f700 1 --2- 172.21.15.47:0/3360978148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63180fe510 0x7f6318107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:27.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.134+0000 7f631e99f700 1 --2- 172.21.15.47:0/3360978148 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6300062cc0 0x7f6300065180 unknown :-1 s=CLOSED pgs=2942 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:27.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.134+0000 7f631e99f700 1 --2- 172.21.15.47:0/3360978148 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f63180f8420 0x7f631810dd90 unknown :-1 s=CLOSED pgs=2519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:27.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.134+0000 7f631e99f700 1 --2- 172.21.15.47:0/3360978148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63180fdbb0 0x7f631810e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:27.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.134+0000 7f631e99f700 1 -- 172.21.15.47:0/3360978148 >> 172.21.15.47:0/3360978148 conn(0x7f63180f3800 msgr2=0x7f63180f5170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:27.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.137+0000 7f631e99f700 1 -- 172.21.15.47:0/3360978148 shutdown_connections 2024-02-23T00:44:27.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.148+0000 7f631e99f700 1 -- 172.21.15.47:0/3360978148 wait complete. 2024-02-23T00:44:27.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:27.153+0000 7f631e99f700 1 librados: shutdown 2024-02-23T00:44:27.156 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is unset 2024-02-23T00:44:27.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:24.756660+0000 osd.7 (osd.7) 111 : cluster [DBG] 6.3 scrub starts 2024-02-23T00:44:27.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:24.759624+0000 osd.7 (osd.7) 112 : cluster [DBG] 6.3 scrub ok 2024-02-23T00:44:27.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:25.223231+0000 mgr.y (mgr.14517) 233 : cluster [DBG] pgmap v265: 282 pgs: 250 active+clean, 32 unknown; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:44:27.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:25.468636+0000 osd.2 (osd.2) 87 : cluster [DBG] 5.d scrub starts 2024-02-23T00:44:27.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:25.470170+0000 osd.2 (osd.2) 88 : cluster [DBG] 5.d scrub ok 2024-02-23T00:44:27.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:25.756459+0000 mon.b (mon.1) 241 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:27.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:25.794290+0000 osd.7 (osd.7) 113 : cluster [DBG] 5.10 scrub starts 2024-02-23T00:44:27.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:25.797180+0000 osd.7 (osd.7) 114 : cluster [DBG] 5.10 scrub ok 2024-02-23T00:44:27.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:25.995308+0000 osd.5 (osd.5) 101 : cluster [DBG] 4.15 scrub starts 2024-02-23T00:44:27.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:25.998798+0000 osd.5 (osd.5) 102 : cluster [DBG] 4.15 scrub ok 2024-02-23T00:44:27.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:26.110413+0000 mon.a (mon.0) 1400 : audit [INF] from='client.? 172.21.15.47:0/2361769855' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi047-82378-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi047-82378-26"}]': finished 2024-02-23T00:44:27.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:26.110638+0000 mon.a (mon.0) 1401 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi047-82457-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:27.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:26.273772+0000 mon.c (mon.2) 120 : audit [INF] from='client.? 172.21.15.47:0/3360978148' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:44:27.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:26.557004+0000 mon.a (mon.0) 1402 : cluster [DBG] osdmap e194: 8 total, 8 up, 8 in 2024-02-23T00:44:27.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:26.558071+0000 mon.b (mon.1) 242 : audit [INF] from='client.? 172.21.15.47:0/2464885804' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi047-82386-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:27.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:26.561635+0000 mon.a (mon.0) 1403 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:44:27.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:26.562156+0000 mon.a (mon.0) 1404 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi047-82386-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:27.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:26.594531+0000 osd.0 (osd.0) 99 : cluster [DBG] 18.0 scrub starts 2024-02-23T00:44:27.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:26.595340+0000 osd.0 (osd.0) 100 : cluster [DBG] 18.0 scrub ok 2024-02-23T00:44:27.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:26.757411+0000 mon.b (mon.1) 243 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:27.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:26.945880+0000 mon.a (mon.0) 1405 : audit [DBG] from='client.? 172.21.15.47:0/3943936498' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:44:27.925 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:44:27.928 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:27.929 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:44:28.029 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:44:28.062 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:28.151 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:26.431785+0000 osd.2 (osd.2) 89 : cluster [DBG] 6.14 scrub starts 2024-02-23T00:44:28.151 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:26.432699+0000 osd.2 (osd.2) 90 : cluster [DBG] 6.14 scrub ok 2024-02-23T00:44:28.151 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:27.108667+0000 mon.a (mon.0) 1406 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:44:28.151 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:27.113084+0000 mon.a (mon.0) 1407 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]': finished 2024-02-23T00:44:28.151 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:27.113357+0000 mon.a (mon.0) 1408 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:44:28.151 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:27.113467+0000 mon.a (mon.0) 1409 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi047-82386-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:28.151 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:27.113531+0000 mon.a (mon.0) 1410 : cluster [DBG] osdmap e195: 8 total, 8 up, 8 in 2024-02-23T00:44:28.151 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:27.164926+0000 mon.c (mon.2) 121 : audit [INF] from='client.? 172.21.15.47:0/2935214693' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi047-82457-36"}]: dispatch 2024-02-23T00:44:28.151 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:27.165290+0000 mon.a (mon.0) 1411 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi047-82457-36"}]: dispatch 2024-02-23T00:44:28.151 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:27.166247+0000 mon.c (mon.2) 122 : audit [INF] from='client.? 172.21.15.47:0/2935214693' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi047-82457-36"}]: dispatch 2024-02-23T00:44:28.152 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:27.166554+0000 mon.a (mon.0) 1412 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi047-82457-36"}]: dispatch 2024-02-23T00:44:28.152 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:27.167068+0000 mon.c (mon.2) 123 : audit [INF] from='client.? 172.21.15.47:0/2935214693' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi047-82457-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:28.152 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:27.167380+0000 mon.a (mon.0) 1413 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi047-82457-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:28.152 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:27.224242+0000 mgr.y (mgr.14517) 234 : cluster [DBG] pgmap v268: 298 pgs: 4 creating+activating, 16 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 222 active+clean, 8 creating+peering; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-23T00:44:28.152 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:27.758425+0000 mon.b (mon.1) 244 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:28.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:26.431785+0000 osd.2 (osd.2) 89 : cluster [DBG] 6.14 scrub starts 2024-02-23T00:44:28.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:26.432699+0000 osd.2 (osd.2) 90 : cluster [DBG] 6.14 scrub ok 2024-02-23T00:44:28.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:27.108667+0000 mon.a (mon.0) 1406 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:44:28.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:27.113084+0000 mon.a (mon.0) 1407 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]': finished 2024-02-23T00:44:28.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:27.113357+0000 mon.a (mon.0) 1408 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:44:28.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:27.113467+0000 mon.a (mon.0) 1409 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi047-82386-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:28.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:27.113531+0000 mon.a (mon.0) 1410 : cluster [DBG] osdmap e195: 8 total, 8 up, 8 in 2024-02-23T00:44:28.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:27.164926+0000 mon.c (mon.2) 121 : audit [INF] from='client.? 172.21.15.47:0/2935214693' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi047-82457-36"}]: dispatch 2024-02-23T00:44:28.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:27.165290+0000 mon.a (mon.0) 1411 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi047-82457-36"}]: dispatch 2024-02-23T00:44:28.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:27.166247+0000 mon.c (mon.2) 122 : audit [INF] from='client.? 172.21.15.47:0/2935214693' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi047-82457-36"}]: dispatch 2024-02-23T00:44:28.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:27.166554+0000 mon.a (mon.0) 1412 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi047-82457-36"}]: dispatch 2024-02-23T00:44:28.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:27.167068+0000 mon.c (mon.2) 123 : audit [INF] from='client.? 172.21.15.47:0/2935214693' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi047-82457-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:28.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:27.167380+0000 mon.a (mon.0) 1413 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi047-82457-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:28.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:27.224242+0000 mgr.y (mgr.14517) 234 : cluster [DBG] pgmap v268: 298 pgs: 4 creating+activating, 16 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 222 active+clean, 8 creating+peering; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-23T00:44:28.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:27.758425+0000 mon.b (mon.1) 244 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:28.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:26.431785+0000 osd.2 (osd.2) 89 : cluster [DBG] 6.14 scrub starts 2024-02-23T00:44:28.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:26.432699+0000 osd.2 (osd.2) 90 : cluster [DBG] 6.14 scrub ok 2024-02-23T00:44:28.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:27.108667+0000 mon.a (mon.0) 1406 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:44:28.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:27.113084+0000 mon.a (mon.0) 1407 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]': finished 2024-02-23T00:44:28.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:27.113357+0000 mon.a (mon.0) 1408 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:44:28.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:27.113467+0000 mon.a (mon.0) 1409 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi047-82386-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:28.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:27.113531+0000 mon.a (mon.0) 1410 : cluster [DBG] osdmap e195: 8 total, 8 up, 8 in 2024-02-23T00:44:28.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:27.164926+0000 mon.c (mon.2) 121 : audit [INF] from='client.? 172.21.15.47:0/2935214693' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi047-82457-36"}]: dispatch 2024-02-23T00:44:28.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:27.165290+0000 mon.a (mon.0) 1411 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi047-82457-36"}]: dispatch 2024-02-23T00:44:28.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:27.166247+0000 mon.c (mon.2) 122 : audit [INF] from='client.? 172.21.15.47:0/2935214693' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi047-82457-36"}]: dispatch 2024-02-23T00:44:28.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:27.166554+0000 mon.a (mon.0) 1412 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi047-82457-36"}]: dispatch 2024-02-23T00:44:28.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:27.167068+0000 mon.c (mon.2) 123 : audit [INF] from='client.? 172.21.15.47:0/2935214693' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi047-82457-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:28.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:27.167380+0000 mon.a (mon.0) 1413 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi047-82457-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:28.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:27.224242+0000 mgr.y (mgr.14517) 234 : cluster [DBG] pgmap v268: 298 pgs: 4 creating+activating, 16 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 222 active+clean, 8 creating+peering; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-23T00:44:28.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:27.758425+0000 mon.b (mon.1) 244 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:28.437 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:44:28.437 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-02-23T00:44:28.437 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 0 2024-02-23T00:44:28.437 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:44:28.712 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:44:28.714 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:28.714 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:44:28.815 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:44:28.846 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:28.866 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:44:28.873 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:28.874 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:44:28.975 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:44:29.002 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:29.237 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.7 dump_blocked_ops 2024-02-23T00:44:29.250 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:44:29.253 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:29.253 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:44:29.354 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:44:29.379 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:27.756350+0000 osd.7 (osd.7) 115 : cluster [DBG] 2.1c scrub starts 2024-02-23T00:44:29.379 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:27.759494+0000 osd.7 (osd.7) 116 : cluster [DBG] 2.1c scrub ok 2024-02-23T00:44:29.379 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:28.117940+0000 mon.a (mon.0) 1414 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi047-82457-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:44:29.379 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:28.118026+0000 mon.a (mon.0) 1415 : cluster [DBG] osdmap e196: 8 total, 8 up, 8 in 2024-02-23T00:44:29.379 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:28.128871+0000 mon.c (mon.2) 124 : audit [INF] from='client.? 172.21.15.47:0/2935214693' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi047-82457-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi047-82457-36"}]: dispatch 2024-02-23T00:44:29.379 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:28.129184+0000 mon.a (mon.0) 1416 : audit [INF] from='client.? 172.21.15.47:0/2361769855' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi047-82378-26"}]: dispatch 2024-02-23T00:44:29.379 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:28.129591+0000 mon.a (mon.0) 1417 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi047-82457-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi047-82457-36"}]: dispatch 2024-02-23T00:44:29.379 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:28.595403+0000 mon.a (mon.0) 1418 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:44:29.379 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:28.617418+0000 osd.0 (osd.0) 101 : cluster [DBG] 5.3 scrub starts 2024-02-23T00:44:29.379 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:28.618914+0000 osd.0 (osd.0) 102 : cluster [DBG] 5.3 scrub ok 2024-02-23T00:44:29.379 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:28.759492+0000 mon.b (mon.1) 245 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:29.381 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:29.400 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:44:29.402 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:29.403 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:44:29.504 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:44:29.519 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:44:29.530 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:29.550 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:44:29.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:27.756350+0000 osd.7 (osd.7) 115 : cluster [DBG] 2.1c scrub starts 2024-02-23T00:44:29.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:27.759494+0000 osd.7 (osd.7) 116 : cluster [DBG] 2.1c scrub ok 2024-02-23T00:44:29.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:28.117940+0000 mon.a (mon.0) 1414 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi047-82457-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:44:29.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:28.118026+0000 mon.a (mon.0) 1415 : cluster [DBG] osdmap e196: 8 total, 8 up, 8 in 2024-02-23T00:44:29.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:28.128871+0000 mon.c (mon.2) 124 : audit [INF] from='client.? 172.21.15.47:0/2935214693' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi047-82457-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi047-82457-36"}]: dispatch 2024-02-23T00:44:29.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:28.129184+0000 mon.a (mon.0) 1416 : audit [INF] from='client.? 172.21.15.47:0/2361769855' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi047-82378-26"}]: dispatch 2024-02-23T00:44:29.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:28.129591+0000 mon.a (mon.0) 1417 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi047-82457-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi047-82457-36"}]: dispatch 2024-02-23T00:44:29.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:28.595403+0000 mon.a (mon.0) 1418 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:44:29.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:28.617418+0000 osd.0 (osd.0) 101 : cluster [DBG] 5.3 scrub starts 2024-02-23T00:44:29.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:28.618914+0000 osd.0 (osd.0) 102 : cluster [DBG] 5.3 scrub ok 2024-02-23T00:44:29.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:28.759492+0000 mon.b (mon.1) 245 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:29.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:27.756350+0000 osd.7 (osd.7) 115 : cluster [DBG] 2.1c scrub starts 2024-02-23T00:44:29.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:27.759494+0000 osd.7 (osd.7) 116 : cluster [DBG] 2.1c scrub ok 2024-02-23T00:44:29.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:28.117940+0000 mon.a (mon.0) 1414 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi047-82457-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:44:29.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:28.118026+0000 mon.a (mon.0) 1415 : cluster [DBG] osdmap e196: 8 total, 8 up, 8 in 2024-02-23T00:44:29.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:28.128871+0000 mon.c (mon.2) 124 : audit [INF] from='client.? 172.21.15.47:0/2935214693' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi047-82457-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi047-82457-36"}]: dispatch 2024-02-23T00:44:29.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:28.129184+0000 mon.a (mon.0) 1416 : audit [INF] from='client.? 172.21.15.47:0/2361769855' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi047-82378-26"}]: dispatch 2024-02-23T00:44:29.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:28.129591+0000 mon.a (mon.0) 1417 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi047-82457-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi047-82457-36"}]: dispatch 2024-02-23T00:44:29.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:28.595403+0000 mon.a (mon.0) 1418 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:44:29.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:28.617418+0000 osd.0 (osd.0) 101 : cluster [DBG] 5.3 scrub starts 2024-02-23T00:44:29.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:28.618914+0000 osd.0 (osd.0) 102 : cluster [DBG] 5.3 scrub ok 2024-02-23T00:44:29.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:28.759492+0000 mon.b (mon.1) 245 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:29.560 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:29.561 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:44:29.661 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:44:29.688 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:29.811 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:44:29.814 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:29.814 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:44:29.915 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:44:29.942 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:30.065 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:44:30.068 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:30.068 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:44:30.154 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp api_aio: Running main() from gmock_main.cc 2024-02-23T00:44:30.154 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [==========] Running 41 tests from 2 test suites. 2024-02-23T00:44:30.154 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [----------] Global test environment set-up. 2024-02-23T00:44:30.154 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [----------] 25 tests from LibRadosAio 2024-02-23T00:44:30.154 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2024-02-23T00:44:30.154 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAio.TooBig (12517 ms) 2024-02-23T00:44:30.154 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2024-02-23T00:44:30.154 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (4492 ms) 2024-02-23T00:44:30.154 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2024-02-23T00:44:30.155 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (5629 ms) 2024-02-23T00:44:30.155 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2024-02-23T00:44:30.155 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (3840 ms) 2024-02-23T00:44:30.155 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2024-02-23T00:44:30.155 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (2867 ms) 2024-02-23T00:44:30.155 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2024-02-23T00:44:30.155 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (4005 ms) 2024-02-23T00:44:30.155 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2024-02-23T00:44:30.155 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (4262 ms) 2024-02-23T00:44:30.155 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2024-02-23T00:44:30.155 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (3767 ms) 2024-02-23T00:44:30.155 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2024-02-23T00:44:30.155 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (3609 ms) 2024-02-23T00:44:30.155 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2024-02-23T00:44:30.155 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (4978 ms) 2024-02-23T00:44:30.156 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2024-02-23T00:44:30.156 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (3936 ms) 2024-02-23T00:44:30.156 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2024-02-23T00:44:30.156 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (3085 ms) 2024-02-23T00:44:30.156 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2024-02-23T00:44:30.156 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (3045 ms) 2024-02-23T00:44:30.156 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2024-02-23T00:44:30.156 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (2854 ms) 2024-02-23T00:44:30.156 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2024-02-23T00:44:30.156 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAio.Flush (3015 ms) 2024-02-23T00:44:30.156 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2024-02-23T00:44:30.156 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (2989 ms) 2024-02-23T00:44:30.156 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2024-02-23T00:44:30.156 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (3004 ms) 2024-02-23T00:44:30.156 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2024-02-23T00:44:30.157 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (3011 ms) 2024-02-23T00:44:30.157 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2024-02-23T00:44:30.157 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (2994 ms) 2024-02-23T00:44:30.157 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2024-02-23T00:44:30.157 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (3008 ms) 2024-02-23T00:44:30.157 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2024-02-23T00:44:30.157 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (3421 ms) 2024-02-23T00:44:30.157 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2024-02-23T00:44:30.157 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (2917 ms) 2024-02-23T00:44:30.157 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2024-02-23T00:44:30.157 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (2645 ms) 2024-02-23T00:44:30.157 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2024-02-23T00:44:30.157 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (2998 ms) 2024-02-23T00:44:30.157 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2024-02-23T00:44:30.157 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (3020 ms) 2024-02-23T00:44:30.158 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [----------] 25 tests from LibRadosAio (95909 ms total) 2024-02-23T00:44:30.158 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: 2024-02-23T00:44:30.158 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2024-02-23T00:44:30.158 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleWrite 2024-02-23T00:44:30.158 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleWrite (7001 ms) 2024-02-23T00:44:30.169 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:44:30.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:28.385501+0000 osd.6 (osd.6) 89 : cluster [DBG] 4.1f scrub starts 2024-02-23T00:44:30.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:28.387151+0000 osd.6 (osd.6) 90 : cluster [DBG] 4.1f scrub ok 2024-02-23T00:44:30.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:29.034678+0000 osd.5 (osd.5) 103 : cluster [DBG] 5.15 scrub starts 2024-02-23T00:44:30.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:29.038130+0000 osd.5 (osd.5) 104 : cluster [DBG] 5.15 scrub ok 2024-02-23T00:44:30.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:29.126223+0000 mon.a (mon.0) 1419 : audit [INF] from='client.? 172.21.15.47:0/2361769855' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi047-82378-26"}]': finished 2024-02-23T00:44:30.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:29.224643+0000 mgr.y (mgr.14517) 235 : cluster [DBG] pgmap v271: 290 pgs: 16 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 186 active+clean, 32 unknown, 8 creating+peering; 14 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:44:30.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:29.562593+0000 mon.a (mon.0) 1420 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:44:30.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:29.562673+0000 mon.a (mon.0) 1421 : cluster [DBG] osdmap e197: 8 total, 8 up, 8 in 2024-02-23T00:44:30.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:29.563475+0000 mon.a (mon.0) 1422 : audit [INF] from='client.? 172.21.15.47:0/2361769855' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi047-82378-26"}]: dispatch 2024-02-23T00:44:30.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:29.563586+0000 mon.c (mon.2) 125 : audit [INF] from='client.? 172.21.15.47:0/187838867' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi047-82386-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:30.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:29.565253+0000 mon.a (mon.0) 1423 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-13"}]: dispatch 2024-02-23T00:44:30.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:29.565869+0000 mon.a (mon.0) 1424 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi047-82386-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:30.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:29.760594+0000 mon.b (mon.1) 246 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:30.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:30.103137+0000 mon.a (mon.0) 1425 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:44:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:30.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.270+0000 7fba95494700 1 -- 172.21.15.47:0/497382597 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba900fe510 msgr2=0x7fba900f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:30.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.270+0000 7fba95494700 1 --2- 172.21.15.47:0/497382597 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba900fe510 0x7fba900f7d40 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 rx=0x7fba800047b0 tx=0x7fba80009cb0).stop 2024-02-23T00:44:30.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.270+0000 7fba95494700 1 -- 172.21.15.47:0/497382597 shutdown_connections 2024-02-23T00:44:30.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.270+0000 7fba95494700 1 --2- 172.21.15.47:0/497382597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba900f8420 0x7fba90104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:30.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.270+0000 7fba95494700 1 --2- 172.21.15.47:0/497382597 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba900fdbb0 0x7fba900fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:30.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.270+0000 7fba95494700 1 --2- 172.21.15.47:0/497382597 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba900fe510 0x7fba900f7d40 unknown :-1 s=CLOSED pgs=2524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:30.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.270+0000 7fba95494700 1 -- 172.21.15.47:0/497382597 >> 172.21.15.47:0/497382597 conn(0x7fba900f3800 msgr2=0x7fba900f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:30.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.270+0000 7fba95494700 1 -- 172.21.15.47:0/497382597 shutdown_connections 2024-02-23T00:44:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.270+0000 7fba95494700 1 -- 172.21.15.47:0/497382597 wait complete. 2024-02-23T00:44:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.271+0000 7fba95494700 1 librados: starting msgr at 2024-02-23T00:44:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.271+0000 7fba95494700 1 librados: starting objecter 2024-02-23T00:44:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.271+0000 7fba95494700 1 Processor -- start 2024-02-23T00:44:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.271+0000 7fba95494700 1 -- start start 2024-02-23T00:44:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.271+0000 7fba95494700 1 librados: setting wanted keys 2024-02-23T00:44:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.271+0000 7fba95494700 1 librados: calling monclient init 2024-02-23T00:44:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.271+0000 7fba95494700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba900f8420 0x7fba9006b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.271+0000 7fba95494700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba900fdbb0 0x7fba9006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.271+0000 7fba95494700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba900fe510 0x7fba9006bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.271+0000 7fba95494700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba90116330 con 0x7fba900f8420 2024-02-23T00:44:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.271+0000 7fba95494700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba901164a0 con 0x7fba900fe510 2024-02-23T00:44:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.271+0000 7fba95494700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba90116660 con 0x7fba900fdbb0 2024-02-23T00:44:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.271+0000 7fba8e7fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba900fdbb0 0x7fba9006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.272+0000 7fba8effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba900f8420 0x7fba9006b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:30.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.272+0000 7fba8e7fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba900fdbb0 0x7fba9006b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:34260/0 (socket says 172.21.15.47:34260) 2024-02-23T00:44:30.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.272+0000 7fba8e7fc700 1 -- 172.21.15.47:0/3611721675 learned_addr learned my addr 172.21.15.47:0/3611721675 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:44:30.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.272+0000 7fba8f7fe700 1 --2- 172.21.15.47:0/3611721675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba900fe510 0x7fba9006bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:30.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.272+0000 7fba8e7fc700 1 -- 172.21.15.47:0/3611721675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba900fe510 msgr2=0x7fba9006bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:30.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.272+0000 7fba8e7fc700 1 --2- 172.21.15.47:0/3611721675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba900fe510 0x7fba9006bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:30.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.272+0000 7fba8e7fc700 1 -- 172.21.15.47:0/3611721675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba900f8420 msgr2=0x7fba9006b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:30.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.272+0000 7fba8e7fc700 1 --2- 172.21.15.47:0/3611721675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba900f8420 0x7fba9006b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:30.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.272+0000 7fba8e7fc700 1 -- 172.21.15.47:0/3611721675 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba880097c0 con 0x7fba900fdbb0 2024-02-23T00:44:30.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.272+0000 7fba8f7fe700 1 --2- 172.21.15.47:0/3611721675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba900fe510 0x7fba9006bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-23T00:44:30.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.272+0000 7fba8effd700 1 --2- 172.21.15.47:0/3611721675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba900f8420 0x7fba9006b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:44:30.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.272+0000 7fba77fff700 10 client.?.objecter ms_handle_connect 0x7fba900fdbb0 2024-02-23T00:44:30.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.272+0000 7fba77fff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:44:30.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.272+0000 7fba8e7fc700 1 --2- 172.21.15.47:0/3611721675 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba900fdbb0 0x7fba9006b5c0 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 rx=0x7fba800044e0 tx=0x7fba80009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:30.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.272+0000 7fba77fff700 1 -- 172.21.15.47:0/3611721675 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba80018070 con 0x7fba900fdbb0 2024-02-23T00:44:30.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.272+0000 7fba77fff700 1 -- 172.21.15.47:0/3611721675 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fba800096d0 con 0x7fba900fdbb0 2024-02-23T00:44:30.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.272+0000 7fba95494700 1 -- 172.21.15.47:0/3611721675 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba80005040 con 0x7fba900fdbb0 2024-02-23T00:44:30.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.272+0000 7fba77fff700 1 -- 172.21.15.47:0/3611721675 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8001dc10 con 0x7fba900fdbb0 2024-02-23T00:44:30.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.272+0000 7fba95494700 10 client.40667.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:44:30.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.272+0000 7fba95494700 1 -- 172.21.15.47:0/3611721675 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba90116bd0 con 0x7fba900fdbb0 2024-02-23T00:44:30.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.272+0000 7fba95494700 1 librados: init done 2024-02-23T00:44:30.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.273+0000 7fba95494700 1 -- 172.21.15.47:0/3611721675 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba9005ee80 con 0x7fba900fdbb0 2024-02-23T00:44:30.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.273+0000 7fba77fff700 1 -- 172.21.15.47:0/3611721675 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fba8001c5d0 con 0x7fba900fdbb0 2024-02-23T00:44:30.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.274+0000 7fba77fff700 1 --2- 172.21.15.47:0/3611721675 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fba78062cc0 0x7fba78065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:30.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.274+0000 7fba8effd700 1 --2- 172.21.15.47:0/3611721675 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fba78062cc0 0x7fba78065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:30.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.274+0000 7fba77fff700 1 -- 172.21.15.47:0/3611721675 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(198..198 src has 48..198) v4 ==== 11537+0+0 (secure 0 0 0) 0x7fba8007f490 con 0x7fba900fdbb0 2024-02-23T00:44:30.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.274+0000 7fba77fff700 10 client.40667.objecter ms_dispatch 0x7fba90001550 osd_map(198..198 src has 48..198) v4 2024-02-23T00:44:30.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.274+0000 7fba77fff700 3 client.40667.objecter handle_osd_map got epochs [198,198] > 0 2024-02-23T00:44:30.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.274+0000 7fba77fff700 3 client.40667.objecter handle_osd_map decoding full epoch 198 2024-02-23T00:44:30.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.274+0000 7fba77fff700 20 client.40667.objecter dump_active .. 0 homeless 2024-02-23T00:44:30.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.274+0000 7fba77fff700 10 client.40667.objecter ms_handle_connect 0x7fba78062cc0 2024-02-23T00:44:30.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.274+0000 7fba8effd700 1 --2- 172.21.15.47:0/3611721675 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fba78062cc0 0x7fba78065180 secure :-1 s=READY pgs=2947 cs=0 l=1 rev1=1 rx=0x7fba84010420 tx=0x7fba8400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:30.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.278+0000 7fba77fff700 1 -- 172.21.15.47:0/3611721675 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fba8004e210 con 0x7fba900fdbb0 2024-02-23T00:44:30.306 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:28.385501+0000 osd.6 (osd.6) 89 : cluster [DBG] 4.1f scrub starts 2024-02-23T00:44:30.306 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:28.387151+0000 osd.6 (osd.6) 90 : cluster [DBG] 4.1f scrub ok 2024-02-23T00:44:30.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:29.034678+0000 osd.5 (osd.5) 103 : cluster [DBG] 5.15 scrub starts 2024-02-23T00:44:30.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:29.038130+0000 osd.5 (osd.5) 104 : cluster [DBG] 5.15 scrub ok 2024-02-23T00:44:30.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:29.126223+0000 mon.a (mon.0) 1419 : audit [INF] from='client.? 172.21.15.47:0/2361769855' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi047-82378-26"}]': finished 2024-02-23T00:44:30.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:29.224643+0000 mgr.y (mgr.14517) 235 : cluster [DBG] pgmap v271: 290 pgs: 16 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 186 active+clean, 32 unknown, 8 creating+peering; 14 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:44:30.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:29.562593+0000 mon.a (mon.0) 1420 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:44:30.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:29.562673+0000 mon.a (mon.0) 1421 : cluster [DBG] osdmap e197: 8 total, 8 up, 8 in 2024-02-23T00:44:30.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:29.563475+0000 mon.a (mon.0) 1422 : audit [INF] from='client.? 172.21.15.47:0/2361769855' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi047-82378-26"}]: dispatch 2024-02-23T00:44:30.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:29.563586+0000 mon.c (mon.2) 125 : audit [INF] from='client.? 172.21.15.47:0/187838867' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi047-82386-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:30.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:29.565253+0000 mon.a (mon.0) 1423 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-13"}]: dispatch 2024-02-23T00:44:30.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:29.565869+0000 mon.a (mon.0) 1424 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi047-82386-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:30.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:29.760594+0000 mon.b (mon.1) 246 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:30.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:30.103137+0000 mon.a (mon.0) 1425 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:44:30.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:28.385501+0000 osd.6 (osd.6) 89 : cluster [DBG] 4.1f scrub starts 2024-02-23T00:44:30.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:28.387151+0000 osd.6 (osd.6) 90 : cluster [DBG] 4.1f scrub ok 2024-02-23T00:44:30.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:29.034678+0000 osd.5 (osd.5) 103 : cluster [DBG] 5.15 scrub starts 2024-02-23T00:44:30.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:29.038130+0000 osd.5 (osd.5) 104 : cluster [DBG] 5.15 scrub ok 2024-02-23T00:44:30.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:29.126223+0000 mon.a (mon.0) 1419 : audit [INF] from='client.? 172.21.15.47:0/2361769855' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi047-82378-26"}]': finished 2024-02-23T00:44:30.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:29.224643+0000 mgr.y (mgr.14517) 235 : cluster [DBG] pgmap v271: 290 pgs: 16 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 186 active+clean, 32 unknown, 8 creating+peering; 14 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:44:30.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:29.562593+0000 mon.a (mon.0) 1420 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:44:30.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:29.562673+0000 mon.a (mon.0) 1421 : cluster [DBG] osdmap e197: 8 total, 8 up, 8 in 2024-02-23T00:44:30.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:29.563475+0000 mon.a (mon.0) 1422 : audit [INF] from='client.? 172.21.15.47:0/2361769855' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi047-82378-26"}]: dispatch 2024-02-23T00:44:30.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:29.563586+0000 mon.c (mon.2) 125 : audit [INF] from='client.? 172.21.15.47:0/187838867' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi047-82386-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:30.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:29.565253+0000 mon.a (mon.0) 1423 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-13"}]: dispatch 2024-02-23T00:44:30.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:29.565869+0000 mon.a (mon.0) 1424 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi047-82386-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:30.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:29.760594+0000 mon.b (mon.1) 246 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:30.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:30.103137+0000 mon.a (mon.0) 1425 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:44:30.309 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:44:30.310 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:30.310 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:44:30.411 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:44:30.438 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:30.456 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:44:30.459 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:30.459 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:44:30.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:30.465+0000 7fba95494700 1 -- 172.21.15.47:0/3611721675 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd reweight", "id": 5, "weight": 1.0} v 0) v1 -- 0x7fba9005fa60 con 0x7fba900fdbb0 2024-02-23T00:44:30.560 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:44:30.587 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:44:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:44:30] "GET /metrics HTTP/1.1" 200 196117 "" "Prometheus/2.33.4" 2024-02-23T00:44:30.589 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:30.609 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:44:30.611 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:30.612 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:44:30.708 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:44:30.712 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:44:30.739 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:30.984 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:44:31.086 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:44:31.089 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:31.089 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:44:31.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:31.143+0000 7fba77fff700 1 -- 172.21.15.47:0/3611721675 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 5, "weight": 1.0}]=0 reweighted osd.5 to 1 (10000) v199) v1 ==== 113+0+0 (secure 0 0 0) 0x7fba80051830 con 0x7fba900fdbb0 2024-02-23T00:44:31.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:31.146+0000 7fba95494700 10 librados: watch_flush enter 2024-02-23T00:44:31.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:31.147+0000 7fba95494700 10 librados: watch_flush exit 2024-02-23T00:44:31.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:31.147+0000 7fba95494700 20 client.40667.objecter shutdown clearing up homeless session... 2024-02-23T00:44:31.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:31.147+0000 7fba95494700 10 client.40667.objecter successfully canceled tick 2024-02-23T00:44:31.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:31.147+0000 7fba95494700 1 -- 172.21.15.47:0/3611721675 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fba78062cc0 msgr2=0x7fba78065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:31.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:31.147+0000 7fba95494700 1 --2- 172.21.15.47:0/3611721675 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fba78062cc0 0x7fba78065180 secure :-1 s=READY pgs=2947 cs=0 l=1 rev1=1 rx=0x7fba84010420 tx=0x7fba8400b040).stop 2024-02-23T00:44:31.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:31.147+0000 7fba95494700 1 -- 172.21.15.47:0/3611721675 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba900fdbb0 msgr2=0x7fba9006b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:31.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:31.147+0000 7fba95494700 1 --2- 172.21.15.47:0/3611721675 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba900fdbb0 0x7fba9006b5c0 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 rx=0x7fba800044e0 tx=0x7fba80009cb0).stop 2024-02-23T00:44:31.161 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set noscrub 2024-02-23T00:44:31.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:31.163+0000 7fba95494700 1 -- 172.21.15.47:0/3611721675 shutdown_connections 2024-02-23T00:44:31.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:31.163+0000 7fba95494700 1 --2- 172.21.15.47:0/3611721675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba900fe510 0x7fba9006bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:31.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:31.163+0000 7fba95494700 1 --2- 172.21.15.47:0/3611721675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba900f8420 0x7fba9006b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:31.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:31.163+0000 7fba95494700 1 --2- 172.21.15.47:0/3611721675 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fba78062cc0 0x7fba78065180 unknown :-1 s=CLOSED pgs=2947 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:31.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:31.163+0000 7fba95494700 1 --2- 172.21.15.47:0/3611721675 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba900fdbb0 0x7fba9006b5c0 unknown :-1 s=CLOSED pgs=2525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:31.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:31.163+0000 7fba95494700 1 -- 172.21.15.47:0/3611721675 >> 172.21.15.47:0/3611721675 conn(0x7fba900f3800 msgr2=0x7fba900fb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:31.173+0000 7fba95494700 1 -- 172.21.15.47:0/3611721675 shutdown_connections 2024-02-23T00:44:31.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:31.178+0000 7fba95494700 1 -- 172.21.15.47:0/3611721675 wait complete. 2024-02-23T00:44:31.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:31.180+0000 7fba95494700 1 librados: shutdown 2024-02-23T00:44:31.187 INFO:teuthology.orchestra.run.smithi047.stderr:reweighted osd.5 to 1 (10000) 2024-02-23T00:44:31.190 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:44:31.221 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:31.442 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:29.769868+0000 osd.1 (osd.1) 111 : cluster [DBG] 2.9 scrub starts 2024-02-23T00:44:31.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:29.772930+0000 osd.1 (osd.1) 112 : cluster [DBG] 2.9 scrub ok 2024-02-23T00:44:31.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:30.123717+0000 mon.a (mon.0) 1426 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:44:31.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:30.132711+0000 mon.a (mon.0) 1427 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi047-82457-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi047-82457-36"}]': finished 2024-02-23T00:44:31.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:30.132798+0000 mon.a (mon.0) 1428 : audit [INF] from='client.? 172.21.15.47:0/2361769855' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi047-82378-26"}]': finished 2024-02-23T00:44:31.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:30.132918+0000 mon.a (mon.0) 1429 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-13"}]': finished 2024-02-23T00:44:31.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:30.133094+0000 mon.a (mon.0) 1430 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi047-82386-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:31.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:30.133183+0000 mon.a (mon.0) 1431 : cluster [DBG] osdmap e198: 8 total, 8 up, 8 in 2024-02-23T00:44:31.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:30.165010+0000 mon.b (mon.1) 247 : audit [INF] from='client.? 172.21.15.47:0/1459394068' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi047-82378-27"}]: dispatch 2024-02-23T00:44:31.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:30.165311+0000 mon.a (mon.0) 1432 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi047-82378-27"}]: dispatch 2024-02-23T00:44:31.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:30.166069+0000 mon.b (mon.1) 248 : audit [INF] from='client.? 172.21.15.47:0/1459394068' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi047-82378-27"}]: dispatch 2024-02-23T00:44:31.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:30.166299+0000 mon.a (mon.0) 1433 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi047-82378-27"}]: dispatch 2024-02-23T00:44:31.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:30.166771+0000 mon.b (mon.1) 249 : audit [INF] from='client.? 172.21.15.47:0/1459394068' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi047-82378-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:31.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:30.167120+0000 mon.a (mon.0) 1434 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi047-82378-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:31.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:30.365053+0000 osd.2 (osd.2) 91 : cluster [DBG] 15.11 scrub starts 2024-02-23T00:44:31.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:30.366387+0000 osd.2 (osd.2) 92 : cluster [DBG] 15.11 scrub ok 2024-02-23T00:44:31.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:30.466426+0000 mon.c (mon.2) 126 : audit [INF] from='client.? 172.21.15.47:0/3611721675' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-23T00:44:31.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:30.466711+0000 mon.a (mon.0) 1435 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-23T00:44:31.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:30.761966+0000 mon.b (mon.1) 250 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:31.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:29.769868+0000 osd.1 (osd.1) 111 : cluster [DBG] 2.9 scrub starts 2024-02-23T00:44:31.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:29.772930+0000 osd.1 (osd.1) 112 : cluster [DBG] 2.9 scrub ok 2024-02-23T00:44:31.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:30.123717+0000 mon.a (mon.0) 1426 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:44:31.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:30.132711+0000 mon.a (mon.0) 1427 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi047-82457-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi047-82457-36"}]': finished 2024-02-23T00:44:31.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:30.132798+0000 mon.a (mon.0) 1428 : audit [INF] from='client.? 172.21.15.47:0/2361769855' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi047-82378-26"}]': finished 2024-02-23T00:44:31.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:30.132918+0000 mon.a (mon.0) 1429 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-13"}]': finished 2024-02-23T00:44:31.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:30.133094+0000 mon.a (mon.0) 1430 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi047-82386-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:31.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:30.133183+0000 mon.a (mon.0) 1431 : cluster [DBG] osdmap e198: 8 total, 8 up, 8 in 2024-02-23T00:44:31.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:30.165010+0000 mon.b (mon.1) 247 : audit [INF] from='client.? 172.21.15.47:0/1459394068' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi047-82378-27"}]: dispatch 2024-02-23T00:44:31.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:30.165311+0000 mon.a (mon.0) 1432 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi047-82378-27"}]: dispatch 2024-02-23T00:44:31.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:30.166069+0000 mon.b (mon.1) 248 : audit [INF] from='client.? 172.21.15.47:0/1459394068' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi047-82378-27"}]: dispatch 2024-02-23T00:44:31.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:30.166299+0000 mon.a (mon.0) 1433 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi047-82378-27"}]: dispatch 2024-02-23T00:44:31.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:30.166771+0000 mon.b (mon.1) 249 : audit [INF] from='client.? 172.21.15.47:0/1459394068' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi047-82378-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:31.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:30.167120+0000 mon.a (mon.0) 1434 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi047-82378-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:31.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:30.365053+0000 osd.2 (osd.2) 91 : cluster [DBG] 15.11 scrub starts 2024-02-23T00:44:31.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:30.366387+0000 osd.2 (osd.2) 92 : cluster [DBG] 15.11 scrub ok 2024-02-23T00:44:31.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:30.466426+0000 mon.c (mon.2) 126 : audit [INF] from='client.? 172.21.15.47:0/3611721675' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-23T00:44:31.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:30.466711+0000 mon.a (mon.0) 1435 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-23T00:44:31.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:30.761966+0000 mon.b (mon.1) 250 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:31.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:29.769868+0000 osd.1 (osd.1) 111 : cluster [DBG] 2.9 scrub starts 2024-02-23T00:44:31.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:29.772930+0000 osd.1 (osd.1) 112 : cluster [DBG] 2.9 scrub ok 2024-02-23T00:44:31.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:30.123717+0000 mon.a (mon.0) 1426 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:44:31.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:30.132711+0000 mon.a (mon.0) 1427 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi047-82457-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi047-82457-36"}]': finished 2024-02-23T00:44:31.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:30.132798+0000 mon.a (mon.0) 1428 : audit [INF] from='client.? 172.21.15.47:0/2361769855' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi047-82378-26"}]': finished 2024-02-23T00:44:31.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:30.132918+0000 mon.a (mon.0) 1429 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-13"}]': finished 2024-02-23T00:44:31.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:30.133094+0000 mon.a (mon.0) 1430 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi047-82386-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:31.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:30.133183+0000 mon.a (mon.0) 1431 : cluster [DBG] osdmap e198: 8 total, 8 up, 8 in 2024-02-23T00:44:31.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:30.165010+0000 mon.b (mon.1) 247 : audit [INF] from='client.? 172.21.15.47:0/1459394068' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi047-82378-27"}]: dispatch 2024-02-23T00:44:31.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:30.165311+0000 mon.a (mon.0) 1432 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi047-82378-27"}]: dispatch 2024-02-23T00:44:31.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:30.166069+0000 mon.b (mon.1) 248 : audit [INF] from='client.? 172.21.15.47:0/1459394068' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi047-82378-27"}]: dispatch 2024-02-23T00:44:31.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:30.166299+0000 mon.a (mon.0) 1433 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi047-82378-27"}]: dispatch 2024-02-23T00:44:31.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:30.166771+0000 mon.b (mon.1) 249 : audit [INF] from='client.? 172.21.15.47:0/1459394068' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi047-82378-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:31.560 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:30.167120+0000 mon.a (mon.0) 1434 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi047-82378-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:31.560 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:30.365053+0000 osd.2 (osd.2) 91 : cluster [DBG] 15.11 scrub starts 2024-02-23T00:44:31.560 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:30.366387+0000 osd.2 (osd.2) 92 : cluster [DBG] 15.11 scrub ok 2024-02-23T00:44:31.560 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:30.466426+0000 mon.c (mon.2) 126 : audit [INF] from='client.? 172.21.15.47:0/3611721675' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-23T00:44:31.560 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:30.466711+0000 mon.a (mon.0) 1435 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-23T00:44:31.560 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:30.761966+0000 mon.b (mon.1) 250 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:31.750 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:44:31.750 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-02-23T00:44:31.750 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-02-23T00:44:31.750 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-02-23T00:44:31.750 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:44:31.878 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:44:31.882 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:31.882 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:44:31.983 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:44:32.013 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:32.145 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:31.104875+0000 osd.4 (osd.4) 93 : cluster [DBG] 2.f scrub starts 2024-02-23T00:44:32.145 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:31.106007+0000 osd.4 (osd.4) 94 : cluster [DBG] 2.f scrub ok 2024-02-23T00:44:32.146 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:31.137347+0000 mon.a (mon.0) 1436 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi047-82378-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:44:32.146 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:31.137466+0000 mon.a (mon.0) 1437 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-23T00:44:32.146 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:31.137530+0000 mon.a (mon.0) 1438 : cluster [DBG] osdmap e199: 8 total, 8 up, 8 in 2024-02-23T00:44:32.146 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:31.142458+0000 mon.b (mon.1) 251 : audit [INF] from='client.? 172.21.15.47:0/1459394068' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi047-82378-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi047-82378-27"}]: dispatch 2024-02-23T00:44:32.146 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:31.225317+0000 mgr.y (mgr.14517) 236 : cluster [DBG] pgmap v274: 234 pgs: 12 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 186 active+clean, 8 unknown, 8 creating+peering; 12 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:44:32.146 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:31.602230+0000 mon.a (mon.0) 1439 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi047-82378-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi047-82378-27"}]: dispatch 2024-02-23T00:44:32.146 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:31.762908+0000 mon.b (mon.1) 252 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:32.146 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:31.104875+0000 osd.4 (osd.4) 93 : cluster [DBG] 2.f scrub starts 2024-02-23T00:44:32.146 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:31.106007+0000 osd.4 (osd.4) 94 : cluster [DBG] 2.f scrub ok 2024-02-23T00:44:32.146 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:31.137347+0000 mon.a (mon.0) 1436 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi047-82378-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:44:32.146 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:31.137466+0000 mon.a (mon.0) 1437 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-23T00:44:32.147 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:31.137530+0000 mon.a (mon.0) 1438 : cluster [DBG] osdmap e199: 8 total, 8 up, 8 in 2024-02-23T00:44:32.147 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:31.142458+0000 mon.b (mon.1) 251 : audit [INF] from='client.? 172.21.15.47:0/1459394068' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi047-82378-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi047-82378-27"}]: dispatch 2024-02-23T00:44:32.147 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:31.225317+0000 mgr.y (mgr.14517) 236 : cluster [DBG] pgmap v274: 234 pgs: 12 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 186 active+clean, 8 unknown, 8 creating+peering; 12 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:44:32.147 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:31.602230+0000 mon.a (mon.0) 1439 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi047-82378-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi047-82378-27"}]: dispatch 2024-02-23T00:44:32.147 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:31.762908+0000 mon.b (mon.1) 252 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:32.362 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:44:32.365 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:32.366 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:44:32.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:31.104875+0000 osd.4 (osd.4) 93 : cluster [DBG] 2.f scrub starts 2024-02-23T00:44:32.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:31.106007+0000 osd.4 (osd.4) 94 : cluster [DBG] 2.f scrub ok 2024-02-23T00:44:32.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:31.137347+0000 mon.a (mon.0) 1436 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi047-82378-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:44:32.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:31.137466+0000 mon.a (mon.0) 1437 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-23T00:44:32.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:31.137530+0000 mon.a (mon.0) 1438 : cluster [DBG] osdmap e199: 8 total, 8 up, 8 in 2024-02-23T00:44:32.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:31.142458+0000 mon.b (mon.1) 251 : audit [INF] from='client.? 172.21.15.47:0/1459394068' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi047-82378-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi047-82378-27"}]: dispatch 2024-02-23T00:44:32.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:31.225317+0000 mgr.y (mgr.14517) 236 : cluster [DBG] pgmap v274: 234 pgs: 12 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 186 active+clean, 8 unknown, 8 creating+peering; 12 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:44:32.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:31.602230+0000 mon.a (mon.0) 1439 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi047-82378-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi047-82378-27"}]: dispatch 2024-02-23T00:44:32.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:31.762908+0000 mon.b (mon.1) 252 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:32.467 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:44:32.496 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:32.646 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:44:32.647 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:44:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:44:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:44:32.647 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:44:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:44:32.648 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:32.648 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:44:32.653 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.7 dump_historic_ops 2024-02-23T00:44:32.749 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:44:32.776 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:32.794 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:44:32.797 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:32.797 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:44:32.898 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:44:32.927 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:32.948 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:44:32.952 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:32.952 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:44:32.955 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:44:32.973 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:44:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.981+0000 7f6328eab700 1 -- 172.21.15.47:0/2133968808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63240f8cb0 msgr2=0x7f63240f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.981+0000 7f6328eab700 1 --2- 172.21.15.47:0/2133968808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63240f8cb0 0x7f63240f9090 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 rx=0x7f6314004870 tx=0x7f6314009cb0).stop 2024-02-23T00:44:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.981+0000 7f6328eab700 1 -- 172.21.15.47:0/2133968808 shutdown_connections 2024-02-23T00:44:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.981+0000 7f6328eab700 1 --2- 172.21.15.47:0/2133968808 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6324102210 0x7f6324104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.981+0000 7f6328eab700 1 --2- 172.21.15.47:0/2133968808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63240f8cb0 0x7f63240f9090 unknown :-1 s=CLOSED pgs=2452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.981+0000 7f6328eab700 1 --2- 172.21.15.47:0/2133968808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63240f9610 0x7f6324101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.981+0000 7f6328eab700 1 -- 172.21.15.47:0/2133968808 >> 172.21.15.47:0/2133968808 conn(0x7f63240f2820 msgr2=0x7f63240f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.982+0000 7f6328eab700 1 -- 172.21.15.47:0/2133968808 shutdown_connections 2024-02-23T00:44:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.982+0000 7f6328eab700 1 -- 172.21.15.47:0/2133968808 wait complete. 2024-02-23T00:44:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.982+0000 7f6328eab700 1 librados: starting msgr at 2024-02-23T00:44:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.982+0000 7f6328eab700 1 librados: starting objecter 2024-02-23T00:44:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.982+0000 7f6328eab700 1 Processor -- start 2024-02-23T00:44:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.982+0000 7f6328eab700 1 -- start start 2024-02-23T00:44:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.982+0000 7f6328eab700 1 librados: setting wanted keys 2024-02-23T00:44:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.982+0000 7f6328eab700 1 librados: calling monclient init 2024-02-23T00:44:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.983+0000 7f6328eab700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f63240f8cb0 0x7f6324110220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.983+0000 7f6328eab700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63240f9610 0x7f6324110710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.983+0000 7f6328eab700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6324102210 0x7f632410a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.983+0000 7f6328eab700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632410a8b0 con 0x7f6324102210 2024-02-23T00:44:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.983+0000 7f6328eab700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632410aa20 con 0x7f63240f9610 2024-02-23T00:44:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.983+0000 7f6328eab700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f632410abe0 con 0x7f63240f8cb0 2024-02-23T00:44:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.983+0000 7f632259c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f63240f8cb0 0x7f6324110220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.983+0000 7f632259c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f63240f8cb0 0x7f6324110220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:34308/0 (socket says 172.21.15.47:34308) 2024-02-23T00:44:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.983+0000 7f6321d9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63240f9610 0x7f6324110710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.983+0000 7f632259c700 1 -- 172.21.15.47:0/2457830450 learned_addr learned my addr 172.21.15.47:0/2457830450 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:44:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.983+0000 7f6322d9d700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6324102210 0x7f632410a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.983+0000 7f6322d9d700 1 -- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f63240f8cb0 msgr2=0x7f6324110220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.983+0000 7f6322d9d700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f63240f8cb0 0x7f6324110220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.983+0000 7f6322d9d700 1 -- 172.21.15.47:0/2457830450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63240f9610 msgr2=0x7f6324110710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.983+0000 7f6322d9d700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63240f9610 0x7f6324110710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.983+0000 7f6322d9d700 1 -- 172.21.15.47:0/2457830450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f631c0097c0 con 0x7f6324102210 2024-02-23T00:44:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.984+0000 7f6322d9d700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6324102210 0x7f632410a3c0 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 rx=0x7f6324060200 tx=0x7f631c004c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.984+0000 7f63137fe700 10 client.?.objecter ms_handle_connect 0x7f6324102210 2024-02-23T00:44:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.984+0000 7f63137fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:44:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.984+0000 7f63137fe700 1 -- 172.21.15.47:0/2457830450 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c0079b0 con 0x7f6324102210 2024-02-23T00:44:32.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.984+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6314005040 con 0x7f6324102210 2024-02-23T00:44:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.984+0000 7f63137fe700 1 -- 172.21.15.47:0/2457830450 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f631c0090d0 con 0x7f6324102210 2024-02-23T00:44:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.984+0000 7f63137fe700 1 -- 172.21.15.47:0/2457830450 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c00fb80 con 0x7f6324102210 2024-02-23T00:44:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.984+0000 7f6328eab700 10 client.30336.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:44:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.984+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f632410b1d0 con 0x7f6324102210 2024-02-23T00:44:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.984+0000 7f6328eab700 1 librados: init done 2024-02-23T00:44:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.985+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f632405fa60 con 0x7f6324102210 2024-02-23T00:44:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.986+0000 7f63137fe700 1 -- 172.21.15.47:0/2457830450 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f631c007b10 con 0x7f6324102210 2024-02-23T00:44:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.986+0000 7f63137fe700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f630c062cc0 0x7f630c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.986+0000 7f63137fe700 1 -- 172.21.15.47:0/2457830450 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(200..200 src has 48..200) v4 ==== 11455+0+0 (secure 0 0 0) 0x7f631c07e730 con 0x7f6324102210 2024-02-23T00:44:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.986+0000 7f63137fe700 10 client.30336.objecter ms_dispatch 0x7f6324001550 osd_map(200..200 src has 48..200) v4 2024-02-23T00:44:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.986+0000 7f63137fe700 3 client.30336.objecter handle_osd_map got epochs [200,200] > 0 2024-02-23T00:44:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.986+0000 7f63137fe700 3 client.30336.objecter handle_osd_map decoding full epoch 200 2024-02-23T00:44:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.986+0000 7f632259c700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f630c062cc0 0x7f630c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.987+0000 7f63137fe700 20 client.30336.objecter dump_active .. 0 homeless 2024-02-23T00:44:32.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.987+0000 7f63137fe700 1 -- 172.21.15.47:0/2457830450 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v200) v1 ==== 54+0+16 (secure 0 0 0) 0x7f631c085070 con 0x7f6324102210 2024-02-23T00:44:32.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.987+0000 7f6328eab700 10 client.30336.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:32.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.987+0000 7f6328eab700 15 client.30336.objecter _session_command_op_assign -1 1 2024-02-23T00:44:32.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.987+0000 7f6328eab700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f63240fe5e0 0x7f63240fe9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:32.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.987+0000 7f63137fe700 10 client.30336.objecter ms_handle_connect 0x7f630c062cc0 2024-02-23T00:44:32.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.987+0000 7f6328eab700 20 client.30336.objecter _get_session s=0x7f632405ee80 osd=0 3 2024-02-23T00:44:32.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.987+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f632405ee80 osd=0 3 2024-02-23T00:44:32.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.987+0000 7f632259c700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f630c062cc0 0x7f630c065180 secure :-1 s=READY pgs=2949 cs=0 l=1 rev1=1 rx=0x7f6314004ee0 tx=0x7f631400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:32.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.987+0000 7f6328eab700 20 client.30336.objecter _get_session s=0x7f632405ee80 osd=0 3 2024-02-23T00:44:32.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.987+0000 7f6328eab700 15 client.30336.objecter _session_command_op_remove -1 1 2024-02-23T00:44:32.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.987+0000 7f6328eab700 20 client.30336.objecter get_session s=0x7f632405ee80 osd=0 3 2024-02-23T00:44:32.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.987+0000 7f6328eab700 15 client.30336.objecter _session_command_op_assign 0 1 2024-02-23T00:44:32.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.987+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f632405ee80 osd=0 4 2024-02-23T00:44:32.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.987+0000 7f6328eab700 10 client.30336.objecter _send_command 1 2024-02-23T00:44:32.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.987+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6324107050 con 0x7f63240fe5e0 2024-02-23T00:44:32.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.987+0000 7f6321d9b700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f63240fe5e0 0x7f63240fe9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:32.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.987+0000 7f63137fe700 10 client.30336.objecter ms_handle_connect 0x7f63240fe5e0 2024-02-23T00:44:32.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.987+0000 7f6321d9b700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f63240fe5e0 0x7f63240fe9c0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:32.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.988+0000 7f63137fe700 1 -- 172.21.15.47:0/2457830450 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f6324107050 con 0x7f63240fe5e0 2024-02-23T00:44:32.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.988+0000 7f63137fe700 10 client.30336.objecter ms_dispatch 0x7f6324001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:44:32.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.988+0000 7f63137fe700 10 client.30336.objecter _finish_command 1 = system:0 2024-02-23T00:44:32.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.988+0000 7f63137fe700 20 client.30336.objecter put_session s=0x7f632405ee80 osd=0 4 2024-02-23T00:44:32.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:32.988+0000 7f63137fe700 15 client.30336.objecter _session_command_op_remove 0 1 2024-02-23T00:44:33.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.003+0000 7f6328eab700 10 client.30336.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:44:33.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.003+0000 7f6328eab700 15 client.30336.objecter _session_command_op_assign -1 2 2024-02-23T00:44:33.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.003+0000 7f6328eab700 20 client.30336.objecter _get_session s=0x7f632405ee80 osd=0 3 2024-02-23T00:44:33.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.003+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f632405ee80 osd=0 3 2024-02-23T00:44:33.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.003+0000 7f6328eab700 20 client.30336.objecter _get_session s=0x7f632405ee80 osd=0 3 2024-02-23T00:44:33.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.003+0000 7f6328eab700 15 client.30336.objecter _session_command_op_remove -1 2 2024-02-23T00:44:33.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.003+0000 7f6328eab700 20 client.30336.objecter get_session s=0x7f632405ee80 osd=0 3 2024-02-23T00:44:33.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.003+0000 7f6328eab700 15 client.30336.objecter _session_command_op_assign 0 2 2024-02-23T00:44:33.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.003+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f632405ee80 osd=0 4 2024-02-23T00:44:33.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.003+0000 7f6328eab700 10 client.30336.objecter _send_command 2 2024-02-23T00:44:33.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.003+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f63240273c0 con 0x7f63240fe5e0 2024-02-23T00:44:33.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.004+0000 7f63137fe700 1 -- 172.21.15.47:0/2457830450 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f63240273c0 con 0x7f63240fe5e0 2024-02-23T00:44:33.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.004+0000 7f63137fe700 10 client.30336.objecter ms_dispatch 0x7f6324001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:44:33.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.004+0000 7f63137fe700 10 client.30336.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:44:33.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.004+0000 7f63137fe700 20 client.30336.objecter put_session s=0x7f632405ee80 osd=0 4 2024-02-23T00:44:33.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.004+0000 7f63137fe700 15 client.30336.objecter _session_command_op_remove 0 2 2024-02-23T00:44:33.007 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:44:33.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.005+0000 7f6328eab700 10 client.30336.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:33.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.005+0000 7f6328eab700 15 client.30336.objecter _session_command_op_assign -1 3 2024-02-23T00:44:33.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.005+0000 7f6328eab700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f63240598f0 0x7f6324059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:33.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.005+0000 7f6328eab700 20 client.30336.objecter _get_session s=0x7f6324028710 osd=1 3 2024-02-23T00:44:33.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.005+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f6324028710 osd=1 3 2024-02-23T00:44:33.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.005+0000 7f6328eab700 20 client.30336.objecter _get_session s=0x7f6324028710 osd=1 3 2024-02-23T00:44:33.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.005+0000 7f6328eab700 15 client.30336.objecter _session_command_op_remove -1 3 2024-02-23T00:44:33.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.005+0000 7f6328eab700 20 client.30336.objecter get_session s=0x7f6324028710 osd=1 3 2024-02-23T00:44:33.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.005+0000 7f6328eab700 15 client.30336.objecter _session_command_op_assign 1 3 2024-02-23T00:44:33.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.005+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f6324028710 osd=1 4 2024-02-23T00:44:33.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.005+0000 7f6328eab700 10 client.30336.objecter _send_command 3 2024-02-23T00:44:33.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.005+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f632411d830 con 0x7f63240598f0 2024-02-23T00:44:33.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.005+0000 7f632259c700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f63240598f0 0x7f6324059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:33.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.005+0000 7f63137fe700 10 client.30336.objecter ms_handle_connect 0x7f63240598f0 2024-02-23T00:44:33.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.005+0000 7f632259c700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f63240598f0 0x7f6324059cd0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:33.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.006+0000 7f63137fe700 1 -- 172.21.15.47:0/2457830450 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f632411d830 con 0x7f63240598f0 2024-02-23T00:44:33.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.006+0000 7f63137fe700 10 client.30336.objecter ms_dispatch 0x7f6324001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:44:33.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.006+0000 7f63137fe700 10 client.30336.objecter _finish_command 3 = system:0 2024-02-23T00:44:33.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.006+0000 7f63137fe700 20 client.30336.objecter put_session s=0x7f6324028710 osd=1 4 2024-02-23T00:44:33.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.006+0000 7f63137fe700 15 client.30336.objecter _session_command_op_remove 1 3 2024-02-23T00:44:33.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.020+0000 7f6328eab700 10 client.30336.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:44:33.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.020+0000 7f6328eab700 15 client.30336.objecter _session_command_op_assign -1 4 2024-02-23T00:44:33.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.020+0000 7f6328eab700 20 client.30336.objecter _get_session s=0x7f6324028710 osd=1 3 2024-02-23T00:44:33.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.020+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f6324028710 osd=1 3 2024-02-23T00:44:33.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.020+0000 7f6328eab700 20 client.30336.objecter _get_session s=0x7f6324028710 osd=1 3 2024-02-23T00:44:33.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.020+0000 7f6328eab700 15 client.30336.objecter _session_command_op_remove -1 4 2024-02-23T00:44:33.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.020+0000 7f6328eab700 20 client.30336.objecter get_session s=0x7f6324028710 osd=1 3 2024-02-23T00:44:33.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.020+0000 7f6328eab700 15 client.30336.objecter _session_command_op_assign 1 4 2024-02-23T00:44:33.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.020+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f6324028710 osd=1 4 2024-02-23T00:44:33.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.020+0000 7f6328eab700 10 client.30336.objecter _send_command 4 2024-02-23T00:44:33.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.020+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f6324059360 con 0x7f63240598f0 2024-02-23T00:44:33.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.022+0000 7f63137fe700 1 -- 172.21.15.47:0/2457830450 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f6324059360 con 0x7f63240598f0 2024-02-23T00:44:33.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.022+0000 7f63137fe700 10 client.30336.objecter ms_dispatch 0x7f6324001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:44:33.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.022+0000 7f63137fe700 10 client.30336.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:44:33.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.022+0000 7f63137fe700 20 client.30336.objecter put_session s=0x7f6324028710 osd=1 4 2024-02-23T00:44:33.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.022+0000 7f63137fe700 15 client.30336.objecter _session_command_op_remove 1 4 2024-02-23T00:44:33.025 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:44:33.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.022+0000 7f6328eab700 10 client.30336.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:33.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.022+0000 7f6328eab700 15 client.30336.objecter _session_command_op_assign -1 5 2024-02-23T00:44:33.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.022+0000 7f6328eab700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f6324028270 0x7f632411c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:33.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.022+0000 7f6328eab700 20 client.30336.objecter _get_session s=0x7f6324059770 osd=2 3 2024-02-23T00:44:33.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.022+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f6324059770 osd=2 3 2024-02-23T00:44:33.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.022+0000 7f6328eab700 20 client.30336.objecter _get_session s=0x7f6324059770 osd=2 3 2024-02-23T00:44:33.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.022+0000 7f6328eab700 15 client.30336.objecter _session_command_op_remove -1 5 2024-02-23T00:44:33.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.022+0000 7f6328eab700 20 client.30336.objecter get_session s=0x7f6324059770 osd=2 3 2024-02-23T00:44:33.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.022+0000 7f6328eab700 15 client.30336.objecter _session_command_op_assign 2 5 2024-02-23T00:44:33.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.022+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f6324059770 osd=2 4 2024-02-23T00:44:33.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.022+0000 7f6328eab700 10 client.30336.objecter _send_command 5 2024-02-23T00:44:33.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.022+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f632411c740 con 0x7f6324028270 2024-02-23T00:44:33.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.022+0000 7f6321d9b700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f6324028270 0x7f632411c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:33.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.022+0000 7f63137fe700 10 client.30336.objecter ms_handle_connect 0x7f6324028270 2024-02-23T00:44:33.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.022+0000 7f6321d9b700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f6324028270 0x7f632411c250 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:33.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.023+0000 7f63137fe700 1 -- 172.21.15.47:0/2457830450 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f632411c740 con 0x7f6324028270 2024-02-23T00:44:33.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.023+0000 7f63137fe700 10 client.30336.objecter ms_dispatch 0x7f6324001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:44:33.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.023+0000 7f63137fe700 10 client.30336.objecter _finish_command 5 = system:0 2024-02-23T00:44:33.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.023+0000 7f63137fe700 20 client.30336.objecter put_session s=0x7f6324059770 osd=2 4 2024-02-23T00:44:33.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.023+0000 7f63137fe700 15 client.30336.objecter _session_command_op_remove 2 5 2024-02-23T00:44:33.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.037+0000 7f6328eab700 10 client.30336.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:44:33.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.037+0000 7f6328eab700 15 client.30336.objecter _session_command_op_assign -1 6 2024-02-23T00:44:33.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.037+0000 7f6328eab700 20 client.30336.objecter _get_session s=0x7f6324059770 osd=2 3 2024-02-23T00:44:33.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.037+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f6324059770 osd=2 3 2024-02-23T00:44:33.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.037+0000 7f6328eab700 20 client.30336.objecter _get_session s=0x7f6324059770 osd=2 3 2024-02-23T00:44:33.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.037+0000 7f6328eab700 15 client.30336.objecter _session_command_op_remove -1 6 2024-02-23T00:44:33.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.037+0000 7f6328eab700 20 client.30336.objecter get_session s=0x7f6324059770 osd=2 3 2024-02-23T00:44:33.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.037+0000 7f6328eab700 15 client.30336.objecter _session_command_op_assign 2 6 2024-02-23T00:44:33.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.037+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f6324059770 osd=2 4 2024-02-23T00:44:33.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.037+0000 7f6328eab700 10 client.30336.objecter _send_command 6 2024-02-23T00:44:33.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.037+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f63240277d0 con 0x7f6324028270 2024-02-23T00:44:33.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.039+0000 7f63137fe700 1 -- 172.21.15.47:0/2457830450 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f63240277d0 con 0x7f6324028270 2024-02-23T00:44:33.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.039+0000 7f63137fe700 10 client.30336.objecter ms_dispatch 0x7f6324001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:44:33.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.039+0000 7f63137fe700 10 client.30336.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:44:33.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.039+0000 7f63137fe700 20 client.30336.objecter put_session s=0x7f6324059770 osd=2 4 2024-02-23T00:44:33.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.039+0000 7f63137fe700 15 client.30336.objecter _session_command_op_remove 2 6 2024-02-23T00:44:33.043 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:44:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.039+0000 7f6328eab700 10 client.30336.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.039+0000 7f6328eab700 15 client.30336.objecter _session_command_op_assign -1 7 2024-02-23T00:44:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.039+0000 7f6328eab700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f632411f9b0 0x7f6324121da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.039+0000 7f6328eab700 20 client.30336.objecter _get_session s=0x7f6324027bb0 osd=3 3 2024-02-23T00:44:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.039+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f6324027bb0 osd=3 3 2024-02-23T00:44:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.039+0000 7f6328eab700 20 client.30336.objecter _get_session s=0x7f6324027bb0 osd=3 3 2024-02-23T00:44:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.039+0000 7f6328eab700 15 client.30336.objecter _session_command_op_remove -1 7 2024-02-23T00:44:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.039+0000 7f6328eab700 20 client.30336.objecter get_session s=0x7f6324027bb0 osd=3 3 2024-02-23T00:44:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.039+0000 7f6328eab700 15 client.30336.objecter _session_command_op_assign 3 7 2024-02-23T00:44:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.039+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f6324027bb0 osd=3 4 2024-02-23T00:44:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.039+0000 7f6328eab700 10 client.30336.objecter _send_command 7 2024-02-23T00:44:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.039+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f63241249f0 con 0x7f632411f9b0 2024-02-23T00:44:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.039+0000 7f6322d9d700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f632411f9b0 0x7f6324121da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.039+0000 7f63137fe700 10 client.30336.objecter ms_handle_connect 0x7f632411f9b0 2024-02-23T00:44:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.039+0000 7f6322d9d700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f632411f9b0 0x7f6324121da0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.040+0000 7f63137fe700 1 -- 172.21.15.47:0/2457830450 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f63241249f0 con 0x7f632411f9b0 2024-02-23T00:44:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.040+0000 7f63137fe700 10 client.30336.objecter ms_dispatch 0x7f6324001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:44:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.040+0000 7f63137fe700 10 client.30336.objecter _finish_command 7 = system:0 2024-02-23T00:44:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.040+0000 7f63137fe700 20 client.30336.objecter put_session s=0x7f6324027bb0 osd=3 4 2024-02-23T00:44:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.040+0000 7f63137fe700 15 client.30336.objecter _session_command_op_remove 3 7 2024-02-23T00:44:33.053 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:44:33.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.055+0000 7f6328eab700 10 client.30336.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:44:33.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.055+0000 7f6328eab700 15 client.30336.objecter _session_command_op_assign -1 8 2024-02-23T00:44:33.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.055+0000 7f6328eab700 20 client.30336.objecter _get_session s=0x7f6324027bb0 osd=3 3 2024-02-23T00:44:33.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.055+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f6324027bb0 osd=3 3 2024-02-23T00:44:33.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.055+0000 7f6328eab700 20 client.30336.objecter _get_session s=0x7f6324027bb0 osd=3 3 2024-02-23T00:44:33.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.055+0000 7f6328eab700 15 client.30336.objecter _session_command_op_remove -1 8 2024-02-23T00:44:33.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.055+0000 7f6328eab700 20 client.30336.objecter get_session s=0x7f6324027bb0 osd=3 3 2024-02-23T00:44:33.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.055+0000 7f6328eab700 15 client.30336.objecter _session_command_op_assign 3 8 2024-02-23T00:44:33.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.055+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f6324027bb0 osd=3 4 2024-02-23T00:44:33.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.055+0000 7f6328eab700 10 client.30336.objecter _send_command 8 2024-02-23T00:44:33.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.055+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f632411cb80 con 0x7f632411f9b0 2024-02-23T00:44:33.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.057+0000 7f63137fe700 1 -- 172.21.15.47:0/2457830450 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f632411cb80 con 0x7f632411f9b0 2024-02-23T00:44:33.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.057+0000 7f63137fe700 10 client.30336.objecter ms_dispatch 0x7f6324001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:44:33.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.057+0000 7f63137fe700 10 client.30336.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:44:33.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.057+0000 7f63137fe700 20 client.30336.objecter put_session s=0x7f6324027bb0 osd=3 4 2024-02-23T00:44:33.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.057+0000 7f63137fe700 15 client.30336.objecter _session_command_op_remove 3 8 2024-02-23T00:44:33.059 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:44:33.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.057+0000 7f6328eab700 10 client.30336.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:33.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.057+0000 7f6328eab700 15 client.30336.objecter _session_command_op_assign -1 9 2024-02-23T00:44:33.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.057+0000 7f6328eab700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f632411ccf0 0x7f6324123410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:33.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.057+0000 7f6328eab700 20 client.30336.objecter _get_session s=0x7f632411cb80 osd=4 3 2024-02-23T00:44:33.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.057+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f632411cb80 osd=4 3 2024-02-23T00:44:33.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.057+0000 7f6328eab700 20 client.30336.objecter _get_session s=0x7f632411cb80 osd=4 3 2024-02-23T00:44:33.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.057+0000 7f6328eab700 15 client.30336.objecter _session_command_op_remove -1 9 2024-02-23T00:44:33.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.057+0000 7f6328eab700 20 client.30336.objecter get_session s=0x7f632411cb80 osd=4 3 2024-02-23T00:44:33.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.057+0000 7f6328eab700 15 client.30336.objecter _session_command_op_assign 4 9 2024-02-23T00:44:33.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.057+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f632411cb80 osd=4 4 2024-02-23T00:44:33.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.057+0000 7f6328eab700 10 client.30336.objecter _send_command 9 2024-02-23T00:44:33.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.057+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f632411bfe0 con 0x7f632411ccf0 2024-02-23T00:44:33.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.057+0000 7f632259c700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f632411ccf0 0x7f6324123410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:33.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.058+0000 7f63137fe700 10 client.30336.objecter ms_handle_connect 0x7f632411ccf0 2024-02-23T00:44:33.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.058+0000 7f632259c700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f632411ccf0 0x7f6324123410 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:33.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.058+0000 7f63137fe700 1 -- 172.21.15.47:0/2457830450 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f632411bfe0 con 0x7f632411ccf0 2024-02-23T00:44:33.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.058+0000 7f63137fe700 10 client.30336.objecter ms_dispatch 0x7f6324001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:44:33.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.058+0000 7f63137fe700 10 client.30336.objecter _finish_command 9 = system:0 2024-02-23T00:44:33.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.058+0000 7f63137fe700 20 client.30336.objecter put_session s=0x7f632411cb80 osd=4 4 2024-02-23T00:44:33.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.058+0000 7f63137fe700 15 client.30336.objecter _session_command_op_remove 4 9 2024-02-23T00:44:33.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.075+0000 7f6328eab700 10 client.30336.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:44:33.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.075+0000 7f6328eab700 15 client.30336.objecter _session_command_op_assign -1 10 2024-02-23T00:44:33.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.075+0000 7f6328eab700 20 client.30336.objecter _get_session s=0x7f632411cb80 osd=4 3 2024-02-23T00:44:33.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.075+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f632411cb80 osd=4 3 2024-02-23T00:44:33.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.075+0000 7f6328eab700 20 client.30336.objecter _get_session s=0x7f632411cb80 osd=4 3 2024-02-23T00:44:33.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.075+0000 7f6328eab700 15 client.30336.objecter _session_command_op_remove -1 10 2024-02-23T00:44:33.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.075+0000 7f6328eab700 20 client.30336.objecter get_session s=0x7f632411cb80 osd=4 3 2024-02-23T00:44:33.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.075+0000 7f6328eab700 15 client.30336.objecter _session_command_op_assign 4 10 2024-02-23T00:44:33.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.075+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f632411cb80 osd=4 4 2024-02-23T00:44:33.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.075+0000 7f6328eab700 10 client.30336.objecter _send_command 10 2024-02-23T00:44:33.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.075+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f632411bfe0 con 0x7f632411ccf0 2024-02-23T00:44:33.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.076+0000 7f63137fe700 1 -- 172.21.15.47:0/2457830450 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f632411bfe0 con 0x7f632411ccf0 2024-02-23T00:44:33.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.076+0000 7f63137fe700 10 client.30336.objecter ms_dispatch 0x7f6324001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:44:33.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.076+0000 7f63137fe700 10 client.30336.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:44:33.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.076+0000 7f63137fe700 20 client.30336.objecter put_session s=0x7f632411cb80 osd=4 4 2024-02-23T00:44:33.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.076+0000 7f63137fe700 15 client.30336.objecter _session_command_op_remove 4 10 2024-02-23T00:44:33.079 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:44:33.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.077+0000 7f6328eab700 10 client.30336.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:33.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.077+0000 7f6328eab700 15 client.30336.objecter _session_command_op_assign -1 11 2024-02-23T00:44:33.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.077+0000 7f6328eab700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f6324123e10 0x7f6324124270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:33.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.077+0000 7f6328eab700 20 client.30336.objecter _get_session s=0x7f632411bfe0 osd=5 3 2024-02-23T00:44:33.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.077+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f632411bfe0 osd=5 3 2024-02-23T00:44:33.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.077+0000 7f6328eab700 20 client.30336.objecter _get_session s=0x7f632411bfe0 osd=5 3 2024-02-23T00:44:33.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.077+0000 7f6328eab700 15 client.30336.objecter _session_command_op_remove -1 11 2024-02-23T00:44:33.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.077+0000 7f6328eab700 20 client.30336.objecter get_session s=0x7f632411bfe0 osd=5 3 2024-02-23T00:44:33.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.077+0000 7f6328eab700 15 client.30336.objecter _session_command_op_assign 5 11 2024-02-23T00:44:33.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.077+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f632411bfe0 osd=5 4 2024-02-23T00:44:33.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.077+0000 7f6328eab700 10 client.30336.objecter _send_command 11 2024-02-23T00:44:33.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.077+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f632412b2e0 con 0x7f6324123e10 2024-02-23T00:44:33.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.077+0000 7f6321d9b700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f6324123e10 0x7f6324124270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:33.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.077+0000 7f63137fe700 10 client.30336.objecter ms_handle_connect 0x7f6324123e10 2024-02-23T00:44:33.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.077+0000 7f6321d9b700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f6324123e10 0x7f6324124270 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:33.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.078+0000 7f63137fe700 1 -- 172.21.15.47:0/2457830450 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f632412b2e0 con 0x7f6324123e10 2024-02-23T00:44:33.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.078+0000 7f63137fe700 10 client.30336.objecter ms_dispatch 0x7f6324001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:44:33.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.078+0000 7f63137fe700 10 client.30336.objecter _finish_command 11 = system:0 2024-02-23T00:44:33.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.078+0000 7f63137fe700 20 client.30336.objecter put_session s=0x7f632411bfe0 osd=5 4 2024-02-23T00:44:33.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.078+0000 7f63137fe700 15 client.30336.objecter _session_command_op_remove 5 11 2024-02-23T00:44:33.083 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:33.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.094+0000 7f6328eab700 10 client.30336.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:44:33.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.094+0000 7f6328eab700 15 client.30336.objecter _session_command_op_assign -1 12 2024-02-23T00:44:33.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.094+0000 7f6328eab700 20 client.30336.objecter _get_session s=0x7f632411bfe0 osd=5 3 2024-02-23T00:44:33.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.094+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f632411bfe0 osd=5 3 2024-02-23T00:44:33.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.094+0000 7f6328eab700 20 client.30336.objecter _get_session s=0x7f632411bfe0 osd=5 3 2024-02-23T00:44:33.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.094+0000 7f6328eab700 15 client.30336.objecter _session_command_op_remove -1 12 2024-02-23T00:44:33.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.094+0000 7f6328eab700 20 client.30336.objecter get_session s=0x7f632411bfe0 osd=5 3 2024-02-23T00:44:33.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.094+0000 7f6328eab700 15 client.30336.objecter _session_command_op_assign 5 12 2024-02-23T00:44:33.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.094+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f632411bfe0 osd=5 4 2024-02-23T00:44:33.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.094+0000 7f6328eab700 10 client.30336.objecter _send_command 12 2024-02-23T00:44:33.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.094+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f6324128b80 con 0x7f6324123e10 2024-02-23T00:44:33.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.095+0000 7f63137fe700 1 -- 172.21.15.47:0/2457830450 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f6324128b80 con 0x7f6324123e10 2024-02-23T00:44:33.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.095+0000 7f63137fe700 10 client.30336.objecter ms_dispatch 0x7f6324001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:44:33.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.095+0000 7f63137fe700 10 client.30336.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:44:33.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.095+0000 7f63137fe700 20 client.30336.objecter put_session s=0x7f632411bfe0 osd=5 4 2024-02-23T00:44:33.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.095+0000 7f63137fe700 15 client.30336.objecter _session_command_op_remove 5 12 2024-02-23T00:44:33.098 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:44:33.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.096+0000 7f6328eab700 10 client.30336.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:33.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.096+0000 7f6328eab700 15 client.30336.objecter _session_command_op_assign -1 13 2024-02-23T00:44:33.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.096+0000 7f6328eab700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f6324129200 0x7f632412b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:33.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.096+0000 7f6328eab700 20 client.30336.objecter _get_session s=0x7f6324128b80 osd=6 3 2024-02-23T00:44:33.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.096+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f6324128b80 osd=6 3 2024-02-23T00:44:33.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.096+0000 7f6328eab700 20 client.30336.objecter _get_session s=0x7f6324128b80 osd=6 3 2024-02-23T00:44:33.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.096+0000 7f6328eab700 15 client.30336.objecter _session_command_op_remove -1 13 2024-02-23T00:44:33.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.096+0000 7f6328eab700 20 client.30336.objecter get_session s=0x7f6324128b80 osd=6 3 2024-02-23T00:44:33.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.096+0000 7f6328eab700 15 client.30336.objecter _session_command_op_assign 6 13 2024-02-23T00:44:33.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.096+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f6324128b80 osd=6 4 2024-02-23T00:44:33.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.096+0000 7f6328eab700 10 client.30336.objecter _send_command 13 2024-02-23T00:44:33.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.096+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f632412e410 con 0x7f6324129200 2024-02-23T00:44:33.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.096+0000 7f6322d9d700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f6324129200 0x7f632412b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:33.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.096+0000 7f63137fe700 10 client.30336.objecter ms_handle_connect 0x7f6324129200 2024-02-23T00:44:33.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.096+0000 7f6322d9d700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f6324129200 0x7f632412b5f0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:33.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.097+0000 7f63137fe700 1 -- 172.21.15.47:0/2457830450 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f632412e410 con 0x7f6324129200 2024-02-23T00:44:33.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.097+0000 7f63137fe700 10 client.30336.objecter ms_dispatch 0x7f6324001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:44:33.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.097+0000 7f63137fe700 10 client.30336.objecter _finish_command 13 = system:0 2024-02-23T00:44:33.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.097+0000 7f63137fe700 20 client.30336.objecter put_session s=0x7f6324128b80 osd=6 4 2024-02-23T00:44:33.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.097+0000 7f63137fe700 15 client.30336.objecter _session_command_op_remove 6 13 2024-02-23T00:44:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.112+0000 7f6328eab700 10 client.30336.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:44:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.112+0000 7f6328eab700 15 client.30336.objecter _session_command_op_assign -1 14 2024-02-23T00:44:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.112+0000 7f6328eab700 20 client.30336.objecter _get_session s=0x7f6324128b80 osd=6 3 2024-02-23T00:44:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.112+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f6324128b80 osd=6 3 2024-02-23T00:44:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.112+0000 7f6328eab700 20 client.30336.objecter _get_session s=0x7f6324128b80 osd=6 3 2024-02-23T00:44:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.112+0000 7f6328eab700 15 client.30336.objecter _session_command_op_remove -1 14 2024-02-23T00:44:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.112+0000 7f6328eab700 20 client.30336.objecter get_session s=0x7f6324128b80 osd=6 3 2024-02-23T00:44:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.112+0000 7f6328eab700 15 client.30336.objecter _session_command_op_assign 6 14 2024-02-23T00:44:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.112+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f6324128b80 osd=6 4 2024-02-23T00:44:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.112+0000 7f6328eab700 10 client.30336.objecter _send_command 14 2024-02-23T00:44:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.112+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f632412e410 con 0x7f6324129200 2024-02-23T00:44:33.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.113+0000 7f63137fe700 1 -- 172.21.15.47:0/2457830450 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f632412e410 con 0x7f6324129200 2024-02-23T00:44:33.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.113+0000 7f63137fe700 10 client.30336.objecter ms_dispatch 0x7f6324001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:44:33.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.113+0000 7f63137fe700 10 client.30336.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:44:33.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.113+0000 7f63137fe700 20 client.30336.objecter put_session s=0x7f6324128b80 osd=6 4 2024-02-23T00:44:33.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.113+0000 7f63137fe700 15 client.30336.objecter _session_command_op_remove 6 14 2024-02-23T00:44:33.116 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:44:33.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.113+0000 7f6328eab700 10 client.30336.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:33.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.113+0000 7f6328eab700 15 client.30336.objecter _session_command_op_assign -1 15 2024-02-23T00:44:33.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.113+0000 7f6328eab700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f632412d6f0 0x7f632412db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:33.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.113+0000 7f6328eab700 20 client.30336.objecter _get_session s=0x7f632412e410 osd=7 3 2024-02-23T00:44:33.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.113+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f632412e410 osd=7 3 2024-02-23T00:44:33.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.113+0000 7f6328eab700 20 client.30336.objecter _get_session s=0x7f632412e410 osd=7 3 2024-02-23T00:44:33.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.113+0000 7f6328eab700 15 client.30336.objecter _session_command_op_remove -1 15 2024-02-23T00:44:33.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.113+0000 7f6328eab700 20 client.30336.objecter get_session s=0x7f632412e410 osd=7 3 2024-02-23T00:44:33.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.113+0000 7f6328eab700 15 client.30336.objecter _session_command_op_assign 7 15 2024-02-23T00:44:33.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.113+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f632412e410 osd=7 4 2024-02-23T00:44:33.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.113+0000 7f6328eab700 10 client.30336.objecter _send_command 15 2024-02-23T00:44:33.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.113+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6324131790 con 0x7f632412d6f0 2024-02-23T00:44:33.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.114+0000 7f632259c700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f632412d6f0 0x7f632412db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:33.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.114+0000 7f63137fe700 10 client.30336.objecter ms_handle_connect 0x7f632412d6f0 2024-02-23T00:44:33.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.114+0000 7f632259c700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f632412d6f0 0x7f632412db70 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:33.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.115+0000 7f63137fe700 1 -- 172.21.15.47:0/2457830450 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f6324131790 con 0x7f632412d6f0 2024-02-23T00:44:33.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.115+0000 7f63137fe700 10 client.30336.objecter ms_dispatch 0x7f6324001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:44:33.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.115+0000 7f63137fe700 10 client.30336.objecter _finish_command 15 = system:0 2024-02-23T00:44:33.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.115+0000 7f63137fe700 20 client.30336.objecter put_session s=0x7f632412e410 osd=7 4 2024-02-23T00:44:33.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.115+0000 7f63137fe700 15 client.30336.objecter _session_command_op_remove 7 15 2024-02-23T00:44:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.129+0000 7f6328eab700 10 client.30336.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:44:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.129+0000 7f6328eab700 15 client.30336.objecter _session_command_op_assign -1 16 2024-02-23T00:44:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.129+0000 7f6328eab700 20 client.30336.objecter _get_session s=0x7f632412e410 osd=7 3 2024-02-23T00:44:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.129+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f632412e410 osd=7 3 2024-02-23T00:44:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.129+0000 7f6328eab700 20 client.30336.objecter _get_session s=0x7f632412e410 osd=7 3 2024-02-23T00:44:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.129+0000 7f6328eab700 15 client.30336.objecter _session_command_op_remove -1 16 2024-02-23T00:44:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.129+0000 7f6328eab700 20 client.30336.objecter get_session s=0x7f632412e410 osd=7 3 2024-02-23T00:44:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.129+0000 7f6328eab700 15 client.30336.objecter _session_command_op_assign 7 16 2024-02-23T00:44:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.129+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f632412e410 osd=7 4 2024-02-23T00:44:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.129+0000 7f6328eab700 10 client.30336.objecter _send_command 16 2024-02-23T00:44:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.129+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f632412bbd0 con 0x7f632412d6f0 2024-02-23T00:44:33.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f63137fe700 1 -- 172.21.15.47:0/2457830450 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f632412bbd0 con 0x7f632412d6f0 2024-02-23T00:44:33.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f63137fe700 10 client.30336.objecter ms_dispatch 0x7f6324001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:44:33.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f63137fe700 10 client.30336.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:44:33.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f63137fe700 20 client.30336.objecter put_session s=0x7f632412e410 osd=7 4 2024-02-23T00:44:33.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f63137fe700 15 client.30336.objecter _session_command_op_remove 7 16 2024-02-23T00:44:33.133 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:44:33.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 10 librados: watch_flush enter 2024-02-23T00:44:33.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 10 librados: watch_flush exit 2024-02-23T00:44:33.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 10 client.30336.objecter close_session for osd.0 2024-02-23T00:44:33.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f63240fe5e0 msgr2=0x7f63240fe9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:33.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f63240fe5e0 0x7f63240fe9c0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:33.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f632405ee80 osd=0 1 2024-02-23T00:44:33.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 10 client.30336.objecter close_session for osd.1 2024-02-23T00:44:33.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f63240598f0 msgr2=0x7f6324059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:33.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f63240598f0 0x7f6324059cd0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:33.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f6324028710 osd=1 1 2024-02-23T00:44:33.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 10 client.30336.objecter close_session for osd.2 2024-02-23T00:44:33.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f6324028270 msgr2=0x7f632411c250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:33.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f6324028270 0x7f632411c250 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:33.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f6324059770 osd=2 1 2024-02-23T00:44:33.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 10 client.30336.objecter close_session for osd.3 2024-02-23T00:44:33.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f632411f9b0 msgr2=0x7f6324121da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:33.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f632411f9b0 0x7f6324121da0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:33.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f6324027bb0 osd=3 1 2024-02-23T00:44:33.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 10 client.30336.objecter close_session for osd.4 2024-02-23T00:44:33.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f632411ccf0 msgr2=0x7f6324123410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:33.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f632411ccf0 0x7f6324123410 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:33.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f632411cb80 osd=4 1 2024-02-23T00:44:33.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 10 client.30336.objecter close_session for osd.5 2024-02-23T00:44:33.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f6324123e10 msgr2=0x7f6324124270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:33.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f6324123e10 0x7f6324124270 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:33.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f632411bfe0 osd=5 1 2024-02-23T00:44:33.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 10 client.30336.objecter close_session for osd.6 2024-02-23T00:44:33.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f6324129200 msgr2=0x7f632412b5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:33.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f6324129200 0x7f632412b5f0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:33.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f6324128b80 osd=6 1 2024-02-23T00:44:33.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 10 client.30336.objecter close_session for osd.7 2024-02-23T00:44:33.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f632412d6f0 msgr2=0x7f632412db70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:33.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f632412d6f0 0x7f632412db70 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:33.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 20 client.30336.objecter put_session s=0x7f632412e410 osd=7 1 2024-02-23T00:44:33.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 20 client.30336.objecter shutdown clearing up homeless session... 2024-02-23T00:44:33.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 10 client.30336.objecter successfully canceled tick 2024-02-23T00:44:33.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6322d9d700 1 -- 172.21.15.47:0/2457830450 reap_dead start 2024-02-23T00:44:33.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6322d9d700 1 -- 172.21.15.47:0/2457830450 reap_dead start 2024-02-23T00:44:33.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6322d9d700 1 -- 172.21.15.47:0/2457830450 reap_dead start 2024-02-23T00:44:33.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6322d9d700 1 -- 172.21.15.47:0/2457830450 reap_dead start 2024-02-23T00:44:33.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6322d9d700 1 -- 172.21.15.47:0/2457830450 reap_dead start 2024-02-23T00:44:33.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f630c062cc0 msgr2=0x7f630c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:33.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f630c062cc0 0x7f630c065180 secure :-1 s=READY pgs=2949 cs=0 l=1 rev1=1 rx=0x7f6314004ee0 tx=0x7f631400b040).stop 2024-02-23T00:44:33.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6324102210 msgr2=0x7f632410a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:33.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.131+0000 7f6328eab700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6324102210 0x7f632410a3c0 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 rx=0x7f6324060200 tx=0x7f631c004c50).stop 2024-02-23T00:44:33.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.132+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 shutdown_connections 2024-02-23T00:44:33.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.132+0000 7f6328eab700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f630c062cc0 0x7f630c065180 unknown :-1 s=CLOSED pgs=2949 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:33.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.132+0000 7f6328eab700 1 --2- 172.21.15.47:0/2457830450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6324102210 0x7f632410a3c0 unknown :-1 s=CLOSED pgs=2453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:33.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.132+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 >> 172.21.15.47:0/2457830450 conn(0x7f63240f2820 msgr2=0x7f63240f4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:33.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.132+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 shutdown_connections 2024-02-23T00:44:33.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.132+0000 7f6328eab700 1 -- 172.21.15.47:0/2457830450 wait complete. 2024-02-23T00:44:33.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:33.132+0000 7f6328eab700 1 librados: shutdown 2024-02-23T00:44:33.139 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:44:33.139 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:44:33.139 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:44:33.139 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:44:33.139 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:44:33.139 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:44:33.139 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:44:33.139 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:44:33.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:31.389908+0000 osd.2 (osd.2) 93 : cluster [DBG] 4.14 scrub starts 2024-02-23T00:44:33.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:31.391810+0000 osd.2 (osd.2) 94 : cluster [DBG] 4.14 scrub ok 2024-02-23T00:44:33.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:31.409892+0000 osd.6 (osd.6) 91 : cluster [DBG] 15.f deep-scrub starts 2024-02-23T00:44:33.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:31.410665+0000 osd.6 (osd.6) 92 : cluster [DBG] 15.f deep-scrub ok 2024-02-23T00:44:33.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:32.390084+0000 mgr.y (mgr.14517) 237 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:44:33.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:32.589226+0000 mon.a (mon.0) 1440 : cluster [DBG] osdmap e200: 8 total, 8 up, 8 in 2024-02-23T00:44:33.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:32.590227+0000 mon.c (mon.2) 127 : audit [INF] from='client.? 172.21.15.47:0/2935214693' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi047-82457-36"}]: dispatch 2024-02-23T00:44:33.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:32.590745+0000 mon.a (mon.0) 1441 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:33.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:32.591378+0000 mon.a (mon.0) 1442 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi047-82457-36"}]: dispatch 2024-02-23T00:44:33.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:32.591522+0000 mon.c (mon.2) 128 : audit [INF] from='client.? 172.21.15.47:0/1865083625' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi047-82386-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:33.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:32.591921+0000 mon.a (mon.0) 1443 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi047-82386-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:33.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:32.763846+0000 mon.b (mon.1) 253 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:33.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:32.987722+0000 mon.a (mon.0) 1444 : audit [DBG] from='client.? 172.21.15.47:0/2457830450' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:44:33.472 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:44:33.475 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:31.389908+0000 osd.2 (osd.2) 93 : cluster [DBG] 4.14 scrub starts 2024-02-23T00:44:33.475 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:31.391810+0000 osd.2 (osd.2) 94 : cluster [DBG] 4.14 scrub ok 2024-02-23T00:44:33.475 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:31.409892+0000 osd.6 (osd.6) 91 : cluster [DBG] 15.f deep-scrub starts 2024-02-23T00:44:33.475 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:31.410665+0000 osd.6 (osd.6) 92 : cluster [DBG] 15.f deep-scrub ok 2024-02-23T00:44:33.475 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:32.390084+0000 mgr.y (mgr.14517) 237 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:44:33.475 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:32.589226+0000 mon.a (mon.0) 1440 : cluster [DBG] osdmap e200: 8 total, 8 up, 8 in 2024-02-23T00:44:33.475 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:32.590227+0000 mon.c (mon.2) 127 : audit [INF] from='client.? 172.21.15.47:0/2935214693' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi047-82457-36"}]: dispatch 2024-02-23T00:44:33.476 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:32.590745+0000 mon.a (mon.0) 1441 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:33.476 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:32.591378+0000 mon.a (mon.0) 1442 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi047-82457-36"}]: dispatch 2024-02-23T00:44:33.476 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:32.591522+0000 mon.c (mon.2) 128 : audit [INF] from='client.? 172.21.15.47:0/1865083625' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi047-82386-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:33.476 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:32.591921+0000 mon.a (mon.0) 1443 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi047-82386-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:33.476 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:32.763846+0000 mon.b (mon.1) 253 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:33.476 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:32.987722+0000 mon.a (mon.0) 1444 : audit [DBG] from='client.? 172.21.15.47:0/2457830450' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:44:33.476 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:31.389908+0000 osd.2 (osd.2) 93 : cluster [DBG] 4.14 scrub starts 2024-02-23T00:44:33.476 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:31.391810+0000 osd.2 (osd.2) 94 : cluster [DBG] 4.14 scrub ok 2024-02-23T00:44:33.477 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:31.409892+0000 osd.6 (osd.6) 91 : cluster [DBG] 15.f deep-scrub starts 2024-02-23T00:44:33.477 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:31.410665+0000 osd.6 (osd.6) 92 : cluster [DBG] 15.f deep-scrub ok 2024-02-23T00:44:33.477 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:32.390084+0000 mgr.y (mgr.14517) 237 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:44:33.477 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:32.589226+0000 mon.a (mon.0) 1440 : cluster [DBG] osdmap e200: 8 total, 8 up, 8 in 2024-02-23T00:44:33.477 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:32.590227+0000 mon.c (mon.2) 127 : audit [INF] from='client.? 172.21.15.47:0/2935214693' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi047-82457-36"}]: dispatch 2024-02-23T00:44:33.477 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:32.590745+0000 mon.a (mon.0) 1441 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:33.477 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:32.591378+0000 mon.a (mon.0) 1442 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi047-82457-36"}]: dispatch 2024-02-23T00:44:33.477 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:32.591522+0000 mon.c (mon.2) 128 : audit [INF] from='client.? 172.21.15.47:0/1865083625' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi047-82386-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:33.477 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:32.591921+0000 mon.a (mon.0) 1443 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi047-82386-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:33.477 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:32.763846+0000 mon.b (mon.1) 253 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:33.477 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:32.987722+0000 mon.a (mon.0) 1444 : audit [DBG] from='client.? 172.21.15.47:0/2457830450' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:44:33.478 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:33.478 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:44:33.579 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:44:33.605 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:33.954 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:44:33.956 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:33.957 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:44:34.057 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:44:34.084 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:34.136 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:44:34.140 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:34.141 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:44:34.193 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForCompl(67998 ms total) 2024-02-23T00:44:34.193 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: 2024-02-23T00:44:34.193 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2024-02-23T00:44:34.193 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-02-23T00:44:34.193 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (104 ms) 2024-02-23T00:44:34.193 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (104 ms total) 2024-02-23T00:44:34.193 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: 2024-02-23T00:44:34.194 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-02-23T00:44:34.194 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2024-02-23T00:44:34.194 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (25 ms) 2024-02-23T00:44:34.194 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2024-02-23T00:44:34.194 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (4 ms) 2024-02-23T00:44:34.194 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (29 ms total) 2024-02-23T00:44:34.194 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: 2024-02-23T00:44:34.194 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-02-23T00:44:34.194 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2024-02-23T00:44:34.194 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (23 ms) 2024-02-23T00:44:34.194 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2024-02-23T00:44:34.194 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (2 ms) 2024-02-23T00:44:34.194 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (25 ms total) 2024-02-23T00:44:34.194 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: 2024-02-23T00:44:34.195 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-02-23T00:44:34.195 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2024-02-23T00:44:34.195 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (33 ms) 2024-02-23T00:44:34.195 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2024-02-23T00:44:34.195 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (3 ms) 2024-02-23T00:44:34.195 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (36 ms total) 2024-02-23T00:44:34.195 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: 2024-02-23T00:44:34.195 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2024-02-23T00:44:34.195 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-02-23T00:44:34.195 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (1072 ms) 2024-02-23T00:44:34.195 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (1072 ms total) 2024-02-23T00:44:34.195 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: 2024-02-23T00:44:34.195 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [----------] Global test environment tear-down 2024-02-23T00:44:34.195 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (106948 ms total) 2024-02-23T00:44:34.195 INFO:tasks.workunit.client.0.smithi047.stdout: api_misc_pp: [ PASSED ] 31 tests. 2024-02-23T00:44:34.241 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:44:34.265 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:32.169651+0000 osd.3 (osd.3) 115 : cluster [DBG] 2.1e scrub starts 2024-02-23T00:44:34.265 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:32.170546+0000 osd.3 (osd.3) 116 : cluster [DBG] 2.1e scrub ok 2024-02-23T00:44:34.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:33.142509+0000 osd.3 (osd.3) 117 : cluster [DBG] 3.2 scrub starts 2024-02-23T00:44:34.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:33.143588+0000 osd.3 (osd.3) 118 : cluster [DBG] 3.2 scrub ok 2024-02-23T00:44:34.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:33.150641+0000 mon.a (mon.0) 1445 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi047-82378-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi047-82378-27"}]': finished 2024-02-23T00:44:34.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:33.150809+0000 mon.a (mon.0) 1446 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:34.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:33.150926+0000 mon.a (mon.0) 1447 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi047-82457-36"}]': finished 2024-02-23T00:44:34.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:33.151002+0000 mon.a (mon.0) 1448 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi047-82386-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:34.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:33.151089+0000 mon.a (mon.0) 1449 : cluster [DBG] osdmap e201: 8 total, 8 up, 8 in 2024-02-23T00:44:34.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:33.157892+0000 mon.c (mon.2) 129 : audit [INF] from='client.? 172.21.15.47:0/2935214693' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi047-82457-36"}]: dispatch 2024-02-23T00:44:34.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:33.160128+0000 mon.a (mon.0) 1450 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi047-82457-36"}]: dispatch 2024-02-23T00:44:34.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:33.225815+0000 mgr.y (mgr.14517) 238 : cluster [DBG] pgmap v277: 298 pgs: 15 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 188 active+clean, 60 creating+peering, 12 unknown; 20 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s wr, 0 op/s 2024-02-23T00:44:34.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:33.388595+0000 osd.2 (osd.2) 95 : cluster [DBG] 15.8 deep-scrub starts 2024-02-23T00:44:34.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:33.765022+0000 mon.b (mon.1) 254 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:34.267 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:34.479 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:32.169651+0000 osd.3 (osd.3) 115 : cluster [DBG] 2.1e scrub starts 2024-02-23T00:44:34.480 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:32.170546+0000 osd.3 (osd.3) 116 : cluster [DBG] 2.1e scrub ok 2024-02-23T00:44:34.480 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:33.142509+0000 osd.3 (osd.3) 117 : cluster [DBG] 3.2 scrub starts 2024-02-23T00:44:34.480 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:33.143588+0000 osd.3 (osd.3) 118 : cluster [DBG] 3.2 scrub ok 2024-02-23T00:44:34.480 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:33.150641+0000 mon.a (mon.0) 1445 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi047-82378-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi047-82378-27"}]': finished 2024-02-23T00:44:34.480 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:33.150809+0000 mon.a (mon.0) 1446 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:34.480 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:33.150926+0000 mon.a (mon.0) 1447 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi047-82457-36"}]': finished 2024-02-23T00:44:34.480 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:33.151002+0000 mon.a (mon.0) 1448 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi047-82386-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:34.480 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:33.151089+0000 mon.a (mon.0) 1449 : cluster [DBG] osdmap e201: 8 total, 8 up, 8 in 2024-02-23T00:44:34.480 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:33.157892+0000 mon.c (mon.2) 129 : audit [INF] from='client.? 172.21.15.47:0/2935214693' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi047-82457-36"}]: dispatch 2024-02-23T00:44:34.480 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:33.160128+0000 mon.a (mon.0) 1450 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi047-82457-36"}]: dispatch 2024-02-23T00:44:34.480 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:33.225815+0000 mgr.y (mgr.14517) 238 : cluster [DBG] pgmap v277: 298 pgs: 15 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 188 active+clean, 60 creating+peering, 12 unknown; 20 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s wr, 0 op/s 2024-02-23T00:44:34.481 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:33.388595+0000 osd.2 (osd.2) 95 : cluster [DBG] 15.8 deep-scrub starts 2024-02-23T00:44:34.481 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:33.765022+0000 mon.b (mon.1) 254 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:34.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:32.169651+0000 osd.3 (osd.3) 115 : cluster [DBG] 2.1e scrub starts 2024-02-23T00:44:34.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:32.170546+0000 osd.3 (osd.3) 116 : cluster [DBG] 2.1e scrub ok 2024-02-23T00:44:34.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:33.142509+0000 osd.3 (osd.3) 117 : cluster [DBG] 3.2 scrub starts 2024-02-23T00:44:34.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:33.143588+0000 osd.3 (osd.3) 118 : cluster [DBG] 3.2 scrub ok 2024-02-23T00:44:34.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:33.150641+0000 mon.a (mon.0) 1445 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi047-82378-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi047-82378-27"}]': finished 2024-02-23T00:44:34.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:33.150809+0000 mon.a (mon.0) 1446 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:34.484 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:33.150926+0000 mon.a (mon.0) 1447 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi047-82457-36"}]': finished 2024-02-23T00:44:34.484 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:33.151002+0000 mon.a (mon.0) 1448 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi047-82386-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:34.484 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:33.151089+0000 mon.a (mon.0) 1449 : cluster [DBG] osdmap e201: 8 total, 8 up, 8 in 2024-02-23T00:44:34.484 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:33.157892+0000 mon.c (mon.2) 129 : audit [INF] from='client.? 172.21.15.47:0/2935214693' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi047-82457-36"}]: dispatch 2024-02-23T00:44:34.484 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:33.160128+0000 mon.a (mon.0) 1450 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi047-82457-36"}]: dispatch 2024-02-23T00:44:34.484 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:33.225815+0000 mgr.y (mgr.14517) 238 : cluster [DBG] pgmap v277: 298 pgs: 15 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 188 active+clean, 60 creating+peering, 12 unknown; 20 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s wr, 0 op/s 2024-02-23T00:44:34.484 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:33.388595+0000 osd.2 (osd.2) 95 : cluster [DBG] 15.8 deep-scrub starts 2024-02-23T00:44:34.484 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:33.765022+0000 mon.b (mon.1) 254 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:34.615 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:44:34.617 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:34.617 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:44:34.718 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:44:34.745 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:34.944 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:44:34.946 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:34.947 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:44:35.018 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight 6 1 2024-02-23T00:44:35.047 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:44:35.089 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:35.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:33.389600+0000 osd.2 (osd.2) 96 : cluster [DBG] 15.8 deep-scrub ok 2024-02-23T00:44:35.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:34.058193+0000 osd.4 (osd.4) 95 : cluster [DBG] 5.1 scrub starts 2024-02-23T00:44:35.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:34.059125+0000 osd.4 (osd.4) 96 : cluster [DBG] 5.1 scrub ok 2024-02-23T00:44:35.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:34.161255+0000 mon.a (mon.0) 1451 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi047-82457-36"}]': finished 2024-02-23T00:44:35.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:34.161433+0000 mon.a (mon.0) 1452 : cluster [DBG] osdmap e202: 8 total, 8 up, 8 in 2024-02-23T00:44:35.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:34.178115+0000 mon.a (mon.0) 1453 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:44:35.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:34.766196+0000 mon.b (mon.1) 255 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:35.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:35.110854+0000 mon.a (mon.0) 1454 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:44:35.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:35.110970+0000 mon.a (mon.0) 1455 : cluster [DBG] osdmap e203: 8 total, 8 up, 8 in 2024-02-23T00:44:35.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:35.115555+0000 mon.b (mon.1) 256 : audit [INF] from='client.? 172.21.15.47:0/1929334916' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi047-82386-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:35.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:35.119810+0000 mon.a (mon.0) 1456 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-15"}]: dispatch 2024-02-23T00:44:35.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:35.120387+0000 mon.a (mon.0) 1457 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi047-82386-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:35.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:35.122061+0000 mon.b (mon.1) 257 : audit [INF] from='client.? 172.21.15.47:0/1459394068' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi047-82378-27"}]: dispatch 2024-02-23T00:44:35.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:35.129887+0000 mon.a (mon.0) 1458 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi047-82378-27"}]: dispatch 2024-02-23T00:44:35.255 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:33.389600+0000 osd.2 (osd.2) 96 : cluster [DBG] 15.8 deep-scrub ok 2024-02-23T00:44:35.255 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:34.058193+0000 osd.4 (osd.4) 95 : cluster [DBG] 5.1 scrub starts 2024-02-23T00:44:35.256 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:34.059125+0000 osd.4 (osd.4) 96 : cluster [DBG] 5.1 scrub ok 2024-02-23T00:44:35.256 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:34.161255+0000 mon.a (mon.0) 1451 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi047-82457-36"}]': finished 2024-02-23T00:44:35.256 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:34.161433+0000 mon.a (mon.0) 1452 : cluster [DBG] osdmap e202: 8 total, 8 up, 8 in 2024-02-23T00:44:35.256 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:34.178115+0000 mon.a (mon.0) 1453 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:44:35.256 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:34.766196+0000 mon.b (mon.1) 255 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:35.256 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:35.110854+0000 mon.a (mon.0) 1454 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:44:35.256 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:35.110970+0000 mon.a (mon.0) 1455 : cluster [DBG] osdmap e203: 8 total, 8 up, 8 in 2024-02-23T00:44:35.256 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:35.115555+0000 mon.b (mon.1) 256 : audit [INF] from='client.? 172.21.15.47:0/1929334916' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi047-82386-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:35.256 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:35.119810+0000 mon.a (mon.0) 1456 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-15"}]: dispatch 2024-02-23T00:44:35.256 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:35.120387+0000 mon.a (mon.0) 1457 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi047-82386-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:35.256 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:35.122061+0000 mon.b (mon.1) 257 : audit [INF] from='client.? 172.21.15.47:0/1459394068' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi047-82378-27"}]: dispatch 2024-02-23T00:44:35.256 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:35.129887+0000 mon.a (mon.0) 1458 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi047-82378-27"}]: dispatch 2024-02-23T00:44:35.257 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:33.389600+0000 osd.2 (osd.2) 96 : cluster [DBG] 15.8 deep-scrub ok 2024-02-23T00:44:35.257 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:34.058193+0000 osd.4 (osd.4) 95 : cluster [DBG] 5.1 scrub starts 2024-02-23T00:44:35.257 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:34.059125+0000 osd.4 (osd.4) 96 : cluster [DBG] 5.1 scrub ok 2024-02-23T00:44:35.257 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:34.161255+0000 mon.a (mon.0) 1451 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi047-82457-36"}]': finished 2024-02-23T00:44:35.257 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:34.161433+0000 mon.a (mon.0) 1452 : cluster [DBG] osdmap e202: 8 total, 8 up, 8 in 2024-02-23T00:44:35.257 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:34.178115+0000 mon.a (mon.0) 1453 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:44:35.257 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:34.766196+0000 mon.b (mon.1) 255 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:35.257 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:35.110854+0000 mon.a (mon.0) 1454 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:44:35.257 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:35.110970+0000 mon.a (mon.0) 1455 : cluster [DBG] osdmap e203: 8 total, 8 up, 8 in 2024-02-23T00:44:35.257 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:35.115555+0000 mon.b (mon.1) 256 : audit [INF] from='client.? 172.21.15.47:0/1929334916' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi047-82386-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:35.257 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:35.119810+0000 mon.a (mon.0) 1456 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-15"}]: dispatch 2024-02-23T00:44:35.257 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:35.120387+0000 mon.a (mon.0) 1457 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi047-82386-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:35.257 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:35.122061+0000 mon.b (mon.1) 257 : audit [INF] from='client.? 172.21.15.47:0/1459394068' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi047-82378-27"}]: dispatch 2024-02-23T00:44:35.258 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:35.129887+0000 mon.a (mon.0) 1458 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi047-82378-27"}]: dispatch 2024-02-23T00:44:35.259 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:44:35.259 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:44:35.648 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:44:35.650 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:35.651 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:44:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:44:35.752 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:44:35.781 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:36.161 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.0 dump_ops_in_flight 2024-02-23T00:44:36.231 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:44:36.233 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:36.234 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:44:36.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.303+0000 7f404739d700 1 -- 172.21.15.47:0/3873803852 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f404006b110 msgr2=0x7f404006b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:36.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.303+0000 7f404739d700 1 --2- 172.21.15.47:0/3873803852 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f404006b110 0x7f404006b4f0 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 rx=0x7f403c004870 tx=0x7f403c009cb0).stop 2024-02-23T00:44:36.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.303+0000 7f404739d700 1 -- 172.21.15.47:0/3873803852 shutdown_connections 2024-02-23T00:44:36.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.303+0000 7f404739d700 1 --2- 172.21.15.47:0/3873803852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40401044a0 0x7f4040106890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:36.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.303+0000 7f404739d700 1 --2- 172.21.15.47:0/3873803852 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f404006b110 0x7f404006b4f0 unknown :-1 s=CLOSED pgs=2527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:36.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.303+0000 7f404739d700 1 --2- 172.21.15.47:0/3873803852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f404006b9e0 0x7f4040103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:36.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.303+0000 7f404739d700 1 -- 172.21.15.47:0/3873803852 >> 172.21.15.47:0/3873803852 conn(0x7f40400f3800 msgr2=0x7f40400f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:36.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.303+0000 7f404739d700 1 -- 172.21.15.47:0/3873803852 shutdown_connections 2024-02-23T00:44:36.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.303+0000 7f404739d700 1 -- 172.21.15.47:0/3873803852 wait complete. 2024-02-23T00:44:36.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.303+0000 7f404739d700 1 librados: starting msgr at 2024-02-23T00:44:36.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.303+0000 7f404739d700 1 librados: starting objecter 2024-02-23T00:44:36.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.304+0000 7f404739d700 1 Processor -- start 2024-02-23T00:44:36.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.304+0000 7f404739d700 1 -- start start 2024-02-23T00:44:36.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.304+0000 7f404739d700 1 librados: setting wanted keys 2024-02-23T00:44:36.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.304+0000 7f404739d700 1 librados: calling monclient init 2024-02-23T00:44:36.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.304+0000 7f404739d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f404006b110 0x7f4040110210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:36.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.304+0000 7f404739d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f404006b9e0 0x7f4040110700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:36.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.304+0000 7f404739d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40401044a0 0x7f404010a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:36.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:34.436572+0000 osd.2 (osd.2) 97 : cluster [DBG] 15.1e deep-scrub starts 2024-02-23T00:44:36.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:34.437777+0000 osd.2 (osd.2) 98 : cluster [DBG] 15.1e deep-scrub ok 2024-02-23T00:44:36.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:34.718221+0000 osd.1 (osd.1) 113 : cluster [DBG] 2.15 deep-scrub starts 2024-02-23T00:44:36.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:34.720635+0000 osd.1 (osd.1) 114 : cluster [DBG] 2.15 deep-scrub ok 2024-02-23T00:44:36.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:35.093770+0000 osd.4 (osd.4) 97 : cluster [DBG] 4.1b scrub starts 2024-02-23T00:44:36.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:35.095404+0000 osd.4 (osd.4) 98 : cluster [DBG] 4.1b scrub ok 2024-02-23T00:44:36.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:35.226230+0000 mgr.y (mgr.14517) 239 : cluster [DBG] pgmap v280: 290 pgs: 15 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 188 active+clean, 31 creating+peering, 33 unknown; 20 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 2 op/s 2024-02-23T00:44:36.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:35.767113+0000 mon.b (mon.1) 258 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:36.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:36.111415+0000 mon.a (mon.0) 1459 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-15"}]': finished 2024-02-23T00:44:36.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:36.111541+0000 mon.a (mon.0) 1460 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi047-82386-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:36.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:36.111625+0000 mon.a (mon.0) 1461 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi047-82378-27"}]': finished 2024-02-23T00:44:36.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:36.111733+0000 mon.a (mon.0) 1462 : cluster [DBG] osdmap e204: 8 total, 8 up, 8 in 2024-02-23T00:44:36.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:36.114685+0000 mon.a (mon.0) 1463 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-15", "mode": "writeback"}]: dispatch 2024-02-23T00:44:36.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:36.118072+0000 mon.b (mon.1) 259 : audit [INF] from='client.? 172.21.15.47:0/1459394068' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi047-82378-27"}]: dispatch 2024-02-23T00:44:36.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:36.119429+0000 mon.a (mon.0) 1464 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi047-82378-27"}]: dispatch 2024-02-23T00:44:36.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.304+0000 7f404739d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404010a910 con 0x7f404006b9e0 2024-02-23T00:44:36.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.304+0000 7f404739d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404010aa80 con 0x7f40401044a0 2024-02-23T00:44:36.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.304+0000 7f404739d700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f404010ac40 con 0x7f404006b110 2024-02-23T00:44:36.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.304+0000 7f4045139700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f404006b110 0x7f4040110210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:36.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.304+0000 7f4045139700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f404006b110 0x7f4040110210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:34364/0 (socket says 172.21.15.47:34364) 2024-02-23T00:44:36.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.304+0000 7f4045139700 1 -- 172.21.15.47:0/1430177368 learned_addr learned my addr 172.21.15.47:0/1430177368 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:44:36.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.304+0000 7f404593a700 1 --2- 172.21.15.47:0/1430177368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40401044a0 0x7f404010a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:36.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.304+0000 7f4045139700 1 -- 172.21.15.47:0/1430177368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40401044a0 msgr2=0x7f404010a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:36.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.304+0000 7f4045139700 1 --2- 172.21.15.47:0/1430177368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40401044a0 0x7f404010a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:36.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.304+0000 7f4045139700 1 -- 172.21.15.47:0/1430177368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f404006b9e0 msgr2=0x7f4040110700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:36.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.304+0000 7f4045139700 1 --2- 172.21.15.47:0/1430177368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f404006b9e0 0x7f4040110700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:44:36.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.304+0000 7f4045139700 1 -- 172.21.15.47:0/1430177368 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4030005040 con 0x7f404006b110 2024-02-23T00:44:36.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.304+0000 7f4045139700 1 --2- 172.21.15.47:0/1430177368 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f404006b110 0x7f4040110210 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 rx=0x7f403c009da0 tx=0x7f403c009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:36.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.304+0000 7f40367fc700 10 client.?.objecter ms_handle_connect 0x7f404006b110 2024-02-23T00:44:36.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.304+0000 7f40367fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:44:36.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.305+0000 7f40367fc700 1 -- 172.21.15.47:0/1430177368 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c00e030 con 0x7f404006b110 2024-02-23T00:44:36.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.305+0000 7f404739d700 1 -- 172.21.15.47:0/1430177368 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f403c005040 con 0x7f404006b110 2024-02-23T00:44:36.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.305+0000 7f40367fc700 1 -- 172.21.15.47:0/1430177368 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f403c029070 con 0x7f404006b110 2024-02-23T00:44:36.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.305+0000 7f40367fc700 1 -- 172.21.15.47:0/1430177368 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c02e7e0 con 0x7f404006b110 2024-02-23T00:44:36.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.305+0000 7f404739d700 10 client.40688.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:44:36.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.305+0000 7f404739d700 1 -- 172.21.15.47:0/1430177368 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f404010b0e0 con 0x7f404006b110 2024-02-23T00:44:36.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.305+0000 7f404739d700 1 librados: init done 2024-02-23T00:44:36.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.306+0000 7f404739d700 1 -- 172.21.15.47:0/1430177368 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f404005ee80 con 0x7f404006b110 2024-02-23T00:44:36.312 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:34.436572+0000 osd.2 (osd.2) 97 : cluster [DBG] 15.1e deep-scrub starts 2024-02-23T00:44:36.312 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:34.437777+0000 osd.2 (osd.2) 98 : cluster [DBG] 15.1e deep-scrub ok 2024-02-23T00:44:36.312 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:34.718221+0000 osd.1 (osd.1) 113 : cluster [DBG] 2.15 deep-scrub starts 2024-02-23T00:44:36.312 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:34.720635+0000 osd.1 (osd.1) 114 : cluster [DBG] 2.15 deep-scrub ok 2024-02-23T00:44:36.312 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:35.093770+0000 osd.4 (osd.4) 97 : cluster [DBG] 4.1b scrub starts 2024-02-23T00:44:36.312 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:35.095404+0000 osd.4 (osd.4) 98 : cluster [DBG] 4.1b scrub ok 2024-02-23T00:44:36.312 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:35.226230+0000 mgr.y (mgr.14517) 239 : cluster [DBG] pgmap v280: 290 pgs: 15 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 188 active+clean, 31 creating+peering, 33 unknown; 20 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 2 op/s 2024-02-23T00:44:36.312 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:35.767113+0000 mon.b (mon.1) 258 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:36.312 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:36.111415+0000 mon.a (mon.0) 1459 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-15"}]': finished 2024-02-23T00:44:36.312 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:36.111541+0000 mon.a (mon.0) 1460 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi047-82386-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:36.312 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:36.111625+0000 mon.a (mon.0) 1461 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi047-82378-27"}]': finished 2024-02-23T00:44:36.312 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:36.111733+0000 mon.a (mon.0) 1462 : cluster [DBG] osdmap e204: 8 total, 8 up, 8 in 2024-02-23T00:44:36.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:36.114685+0000 mon.a (mon.0) 1463 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-15", "mode": "writeback"}]: dispatch 2024-02-23T00:44:36.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:36.118072+0000 mon.b (mon.1) 259 : audit [INF] from='client.? 172.21.15.47:0/1459394068' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi047-82378-27"}]: dispatch 2024-02-23T00:44:36.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:36.119429+0000 mon.a (mon.0) 1464 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi047-82378-27"}]: dispatch 2024-02-23T00:44:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.307+0000 7f40367fc700 1 -- 172.21.15.47:0/1430177368 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f403c02b030 con 0x7f404006b110 2024-02-23T00:44:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.307+0000 7f40367fc700 1 --2- 172.21.15.47:0/1430177368 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f402c062cc0 0x7f402c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.307+0000 7f40367fc700 1 -- 172.21.15.47:0/1430177368 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(204..204 src has 48..204) v4 ==== 10817+0+0 (secure 0 0 0) 0x7f403c090230 con 0x7f404006b110 2024-02-23T00:44:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.307+0000 7f40367fc700 10 client.40688.objecter ms_dispatch 0x7f4040001550 osd_map(204..204 src has 48..204) v4 2024-02-23T00:44:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.307+0000 7f40367fc700 3 client.40688.objecter handle_osd_map got epochs [204,204] > 0 2024-02-23T00:44:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.307+0000 7f40367fc700 3 client.40688.objecter handle_osd_map decoding full epoch 204 2024-02-23T00:44:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.308+0000 7f4044938700 1 --2- 172.21.15.47:0/1430177368 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f402c062cc0 0x7f402c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.308+0000 7f40367fc700 20 client.40688.objecter dump_active .. 0 homeless 2024-02-23T00:44:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.308+0000 7f40367fc700 10 client.40688.objecter ms_handle_connect 0x7f402c062cc0 2024-02-23T00:44:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.308+0000 7f4044938700 1 --2- 172.21.15.47:0/1430177368 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f402c062cc0 0x7f402c065180 secure :-1 s=READY pgs=2951 cs=0 l=1 rev1=1 rx=0x7f404010bb50 tx=0x7f4030009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.311+0000 7f40367fc700 1 -- 172.21.15.47:0/1430177368 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f403c05f280 con 0x7f404006b110 2024-02-23T00:44:36.335 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:44:36.362 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:36.380 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:44:36.382 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:36.383 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:44:36.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:34.436572+0000 osd.2 (osd.2) 97 : cluster [DBG] 15.1e deep-scrub starts 2024-02-23T00:44:36.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:34.437777+0000 osd.2 (osd.2) 98 : cluster [DBG] 15.1e deep-scrub ok 2024-02-23T00:44:36.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:34.718221+0000 osd.1 (osd.1) 113 : cluster [DBG] 2.15 deep-scrub starts 2024-02-23T00:44:36.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:34.720635+0000 osd.1 (osd.1) 114 : cluster [DBG] 2.15 deep-scrub ok 2024-02-23T00:44:36.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:35.093770+0000 osd.4 (osd.4) 97 : cluster [DBG] 4.1b scrub starts 2024-02-23T00:44:36.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:35.095404+0000 osd.4 (osd.4) 98 : cluster [DBG] 4.1b scrub ok 2024-02-23T00:44:36.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:35.226230+0000 mgr.y (mgr.14517) 239 : cluster [DBG] pgmap v280: 290 pgs: 15 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 188 active+clean, 31 creating+peering, 33 unknown; 20 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 2 op/s 2024-02-23T00:44:36.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:35.767113+0000 mon.b (mon.1) 258 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:36.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:36.111415+0000 mon.a (mon.0) 1459 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-15"}]': finished 2024-02-23T00:44:36.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:36.111541+0000 mon.a (mon.0) 1460 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi047-82386-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:36.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:36.111625+0000 mon.a (mon.0) 1461 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi047-82378-27"}]': finished 2024-02-23T00:44:36.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:36.111733+0000 mon.a (mon.0) 1462 : cluster [DBG] osdmap e204: 8 total, 8 up, 8 in 2024-02-23T00:44:36.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:36.114685+0000 mon.a (mon.0) 1463 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-15", "mode": "writeback"}]: dispatch 2024-02-23T00:44:36.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:36.118072+0000 mon.b (mon.1) 259 : audit [INF] from='client.? 172.21.15.47:0/1459394068' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi047-82378-27"}]: dispatch 2024-02-23T00:44:36.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:36.119429+0000 mon.a (mon.0) 1464 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi047-82378-27"}]: dispatch 2024-02-23T00:44:36.484 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:44:36.498 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:44:36.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:36.499+0000 7f404739d700 1 -- 172.21.15.47:0/1430177368 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7f404010b900 con 0x7f404006b110 2024-02-23T00:44:36.512 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:36.616 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:44:36.621 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:36.621 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:44:36.722 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:44:36.750 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:36.769 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:44:36.772 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:36.772 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:44:36.873 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:44:36.900 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:36.919 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:44:36.921 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:36.922 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:44:37.023 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:44:37.051 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:37.113+0000 7f40367fc700 1 -- 172.21.15.47:0/1430177368 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v205) v1 ==== 87+0+0 (secure 0 0 0) 0x7f403c064030 con 0x7f404006b110 2024-02-23T00:44:37.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:37.117+0000 7f404739d700 10 librados: watch_flush enter 2024-02-23T00:44:37.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:37.117+0000 7f404739d700 10 librados: watch_flush exit 2024-02-23T00:44:37.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:37.117+0000 7f404739d700 20 client.40688.objecter shutdown clearing up homeless session... 2024-02-23T00:44:37.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:37.117+0000 7f404739d700 10 client.40688.objecter successfully canceled tick 2024-02-23T00:44:37.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:37.118+0000 7f404739d700 1 -- 172.21.15.47:0/1430177368 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f402c062cc0 msgr2=0x7f402c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:37.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:37.118+0000 7f404739d700 1 --2- 172.21.15.47:0/1430177368 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f402c062cc0 0x7f402c065180 secure :-1 s=READY pgs=2951 cs=0 l=1 rev1=1 rx=0x7f404010bb50 tx=0x7f4030009cb0).stop 2024-02-23T00:44:37.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:37.118+0000 7f404739d700 1 -- 172.21.15.47:0/1430177368 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f404006b110 msgr2=0x7f4040110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:37.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:37.118+0000 7f404739d700 1 --2- 172.21.15.47:0/1430177368 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f404006b110 0x7f4040110210 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 rx=0x7f403c009da0 tx=0x7f403c009cb0).stop 2024-02-23T00:44:37.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:37.119+0000 7f404739d700 1 -- 172.21.15.47:0/1430177368 shutdown_connections 2024-02-23T00:44:37.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:37.119+0000 7f404739d700 1 --2- 172.21.15.47:0/1430177368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40401044a0 0x7f404010a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:37.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:37.119+0000 7f404739d700 1 --2- 172.21.15.47:0/1430177368 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f402c062cc0 0x7f402c065180 unknown :-1 s=CLOSED pgs=2951 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:37.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:37.119+0000 7f404739d700 1 --2- 172.21.15.47:0/1430177368 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f404006b110 0x7f4040110210 secure :-1 s=CLOSED pgs=2528 cs=0 l=1 rev1=1 rx=0x7f403c009da0 tx=0x7f403c009cb0).stop 2024-02-23T00:44:37.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:37.119+0000 7f404739d700 1 --2- 172.21.15.47:0/1430177368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f404006b9e0 0x7f4040110700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:44:37.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:37.119+0000 7f404739d700 1 -- 172.21.15.47:0/1430177368 >> 172.21.15.47:0/1430177368 conn(0x7f40400f3800 msgr2=0x7f40400f44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:37.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:37.120+0000 7f404739d700 1 -- 172.21.15.47:0/1430177368 shutdown_connections 2024-02-23T00:44:37.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:37.120+0000 7f404739d700 1 -- 172.21.15.47:0/1430177368 wait complete. 2024-02-23T00:44:37.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:37.120+0000 7f404739d700 1 librados: shutdown 2024-02-23T00:44:37.123 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is set 2024-02-23T00:44:37.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:35.359228+0000 osd.6 (osd.6) 93 : cluster [DBG] 18.10 deep-scrub starts 2024-02-23T00:44:37.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:35.360049+0000 osd.6 (osd.6) 94 : cluster [DBG] 18.10 deep-scrub ok 2024-02-23T00:44:37.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:35.637346+0000 osd.0 (osd.0) 103 : cluster [DBG] 4.3 deep-scrub starts 2024-02-23T00:44:37.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:35.643924+0000 osd.0 (osd.0) 104 : cluster [DBG] 4.3 deep-scrub ok 2024-02-23T00:44:37.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:36.501290+0000 mon.c (mon.2) 130 : audit [INF] from='client.? 172.21.15.47:0/1430177368' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:44:37.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:36.501626+0000 mon.a (mon.0) 1465 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:44:37.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:36.768150+0000 mon.b (mon.1) 260 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:37.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:37.108882+0000 mon.a (mon.0) 1466 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:44:37.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:37.108906+0000 mon.a (mon.0) 1467 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:44:37.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:37.112896+0000 mon.a (mon.0) 1468 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-15", "mode": "writeback"}]': finished 2024-02-23T00:44:37.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:37.112968+0000 mon.a (mon.0) 1469 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi047-82378-27"}]': finished 2024-02-23T00:44:37.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:37.113068+0000 mon.a (mon.0) 1470 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:44:37.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:37.113120+0000 mon.a (mon.0) 1471 : cluster [DBG] osdmap e205: 8 total, 8 up, 8 in 2024-02-23T00:44:37.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:37.143878+0000 mon.a (mon.0) 1472 : audit [INF] from='client.? 172.21.15.47:0/1013032236' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi047-82378-28"}]: dispatch 2024-02-23T00:44:37.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:37.145536+0000 mon.a (mon.0) 1473 : audit [INF] from='client.? 172.21.15.47:0/1013032236' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi047-82378-28"}]: dispatch 2024-02-23T00:44:37.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:37.146110+0000 mon.a (mon.0) 1474 : audit [INF] from='client.? 172.21.15.47:0/1013032236' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi047-82378-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:37.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:35.359228+0000 osd.6 (osd.6) 93 : cluster [DBG] 18.10 deep-scrub starts 2024-02-23T00:44:37.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:35.360049+0000 osd.6 (osd.6) 94 : cluster [DBG] 18.10 deep-scrub ok 2024-02-23T00:44:37.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:35.637346+0000 osd.0 (osd.0) 103 : cluster [DBG] 4.3 deep-scrub starts 2024-02-23T00:44:37.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:35.643924+0000 osd.0 (osd.0) 104 : cluster [DBG] 4.3 deep-scrub ok 2024-02-23T00:44:37.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:36.501290+0000 mon.c (mon.2) 130 : audit [INF] from='client.? 172.21.15.47:0/1430177368' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:44:37.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:36.501626+0000 mon.a (mon.0) 1465 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:44:37.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:36.768150+0000 mon.b (mon.1) 260 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:37.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:37.108882+0000 mon.a (mon.0) 1466 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:44:37.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:37.108906+0000 mon.a (mon.0) 1467 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:44:37.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:37.112896+0000 mon.a (mon.0) 1468 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-15", "mode": "writeback"}]': finished 2024-02-23T00:44:37.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:37.112968+0000 mon.a (mon.0) 1469 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi047-82378-27"}]': finished 2024-02-23T00:44:37.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:37.113068+0000 mon.a (mon.0) 1470 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:44:37.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:37.113120+0000 mon.a (mon.0) 1471 : cluster [DBG] osdmap e205: 8 total, 8 up, 8 in 2024-02-23T00:44:37.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:37.143878+0000 mon.a (mon.0) 1472 : audit [INF] from='client.? 172.21.15.47:0/1013032236' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi047-82378-28"}]: dispatch 2024-02-23T00:44:37.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:37.145536+0000 mon.a (mon.0) 1473 : audit [INF] from='client.? 172.21.15.47:0/1013032236' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi047-82378-28"}]: dispatch 2024-02-23T00:44:37.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:37.146110+0000 mon.a (mon.0) 1474 : audit [INF] from='client.? 172.21.15.47:0/1013032236' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi047-82378-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:37.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:35.359228+0000 osd.6 (osd.6) 93 : cluster [DBG] 18.10 deep-scrub starts 2024-02-23T00:44:37.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:35.360049+0000 osd.6 (osd.6) 94 : cluster [DBG] 18.10 deep-scrub ok 2024-02-23T00:44:37.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:35.637346+0000 osd.0 (osd.0) 103 : cluster [DBG] 4.3 deep-scrub starts 2024-02-23T00:44:37.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:35.643924+0000 osd.0 (osd.0) 104 : cluster [DBG] 4.3 deep-scrub ok 2024-02-23T00:44:37.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:36.501290+0000 mon.c (mon.2) 130 : audit [INF] from='client.? 172.21.15.47:0/1430177368' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:44:37.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:36.501626+0000 mon.a (mon.0) 1465 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:44:37.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:36.768150+0000 mon.b (mon.1) 260 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:37.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:37.108882+0000 mon.a (mon.0) 1466 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:44:37.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:37.108906+0000 mon.a (mon.0) 1467 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:44:37.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:37.112896+0000 mon.a (mon.0) 1468 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-15", "mode": "writeback"}]': finished 2024-02-23T00:44:37.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:37.112968+0000 mon.a (mon.0) 1469 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi047-82378-27"}]': finished 2024-02-23T00:44:37.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:37.113068+0000 mon.a (mon.0) 1470 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:44:37.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:37.113120+0000 mon.a (mon.0) 1471 : cluster [DBG] osdmap e205: 8 total, 8 up, 8 in 2024-02-23T00:44:37.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:37.143878+0000 mon.a (mon.0) 1472 : audit [INF] from='client.? 172.21.15.47:0/1013032236' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi047-82378-28"}]: dispatch 2024-02-23T00:44:37.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:37.145536+0000 mon.a (mon.0) 1473 : audit [INF] from='client.? 172.21.15.47:0/1013032236' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi047-82378-28"}]: dispatch 2024-02-23T00:44:37.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:37.146110+0000 mon.a (mon.0) 1474 : audit [INF] from='client.? 172.21.15.47:0/1013032236' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi047-82378-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:38.000 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:44:38.002 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:38.003 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:44:38.011 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:44:38.012 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:44:38.104 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:44:38.144 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:38.165 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:44:38.170 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:38.171 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:44:38.272 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:44:38.300 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:38.319 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:44:38.322 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:38.323 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:44:38.423 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:44:38.453 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:38.526 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:44:38.529 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:38.530 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:44:38.630 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:44:38.659 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:38.782 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:44:38.786 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:36.480308+0000 osd.2 (osd.2) 99 : cluster [DBG] 6.f deep-scrub starts 2024-02-23T00:44:38.786 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:36.481251+0000 osd.2 (osd.2) 100 : cluster [DBG] 6.f deep-scrub ok 2024-02-23T00:44:38.786 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:36.755061+0000 osd.7 (osd.7) 117 : cluster [DBG] 6.8 scrub starts 2024-02-23T00:44:38.786 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:36.757955+0000 osd.7 (osd.7) 118 : cluster [DBG] 6.8 scrub ok 2024-02-23T00:44:38.786 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:37.068084+0000 osd.4 (osd.4) 99 : cluster [DBG] 5.1c scrub starts 2024-02-23T00:44:38.786 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:37.068830+0000 osd.4 (osd.4) 100 : cluster [DBG] 5.1c scrub ok 2024-02-23T00:44:38.787 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:37.104294+0000 osd.3 (osd.3) 119 : cluster [DBG] 5.13 scrub starts 2024-02-23T00:44:38.787 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:37.105008+0000 osd.3 (osd.3) 120 : cluster [DBG] 5.13 scrub ok 2024-02-23T00:44:38.787 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:37.227099+0000 mgr.y (mgr.14517) 240 : cluster [DBG] pgmap v283: 258 pgs: 13 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 225 active+clean; 43 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 7.2 KiB/s rd, 10 KiB/s wr, 4 op/s 2024-02-23T00:44:38.787 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:37.769432+0000 mon.b (mon.1) 261 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:38.787 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:38.117712+0000 mon.a (mon.0) 1475 : audit [INF] from='client.? 172.21.15.47:0/1013032236' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi047-82378-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:44:38.787 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:38.117813+0000 mon.a (mon.0) 1476 : cluster [DBG] osdmap e206: 8 total, 8 up, 8 in 2024-02-23T00:44:38.787 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:38.121849+0000 mon.a (mon.0) 1477 : audit [INF] from='client.? 172.21.15.47:0/1013032236' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi047-82378-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi047-82378-28"}]: dispatch 2024-02-23T00:44:38.787 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:38.129478+0000 mon.b (mon.1) 262 : audit [INF] from='client.? 172.21.15.47:0/1594396895' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi047-82386-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:38.787 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:38.137000+0000 mon.a (mon.0) 1478 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi047-82386-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:38.787 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:38.149498+0000 mon.a (mon.0) 1479 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:44:38.788 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:36.480308+0000 osd.2 (osd.2) 99 : cluster [DBG] 6.f deep-scrub starts 2024-02-23T00:44:38.788 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:36.481251+0000 osd.2 (osd.2) 100 : cluster [DBG] 6.f deep-scrub ok 2024-02-23T00:44:38.788 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:36.755061+0000 osd.7 (osd.7) 117 : cluster [DBG] 6.8 scrub starts 2024-02-23T00:44:38.788 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:36.757955+0000 osd.7 (osd.7) 118 : cluster [DBG] 6.8 scrub ok 2024-02-23T00:44:38.788 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:37.068084+0000 osd.4 (osd.4) 99 : cluster [DBG] 5.1c scrub starts 2024-02-23T00:44:38.788 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:37.068830+0000 osd.4 (osd.4) 100 : cluster [DBG] 5.1c scrub ok 2024-02-23T00:44:38.788 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:37.104294+0000 osd.3 (osd.3) 119 : cluster [DBG] 5.13 scrub starts 2024-02-23T00:44:38.788 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:37.105008+0000 osd.3 (osd.3) 120 : cluster [DBG] 5.13 scrub ok 2024-02-23T00:44:38.788 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:37.227099+0000 mgr.y (mgr.14517) 240 : cluster [DBG] pgmap v283: 258 pgs: 13 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 225 active+clean; 43 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 7.2 KiB/s rd, 10 KiB/s wr, 4 op/s 2024-02-23T00:44:38.788 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:37.769432+0000 mon.b (mon.1) 261 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:38.788 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:38.117712+0000 mon.a (mon.0) 1475 : audit [INF] from='client.? 172.21.15.47:0/1013032236' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi047-82378-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:44:38.788 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:38.117813+0000 mon.a (mon.0) 1476 : cluster [DBG] osdmap e206: 8 total, 8 up, 8 in 2024-02-23T00:44:38.788 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:38.121849+0000 mon.a (mon.0) 1477 : audit [INF] from='client.? 172.21.15.47:0/1013032236' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi047-82378-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi047-82378-28"}]: dispatch 2024-02-23T00:44:38.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:38.129478+0000 mon.b (mon.1) 262 : audit [INF] from='client.? 172.21.15.47:0/1594396895' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi047-82386-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:38.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:38.137000+0000 mon.a (mon.0) 1478 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi047-82386-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:38.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:38.149498+0000 mon.a (mon.0) 1479 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:44:38.789 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:38.790 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:44:38.891 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:44:38.921 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:36.480308+0000 osd.2 (osd.2) 99 : cluster [DBG] 6.f deep-scrub starts 2024-02-23T00:44:38.921 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:36.481251+0000 osd.2 (osd.2) 100 : cluster [DBG] 6.f deep-scrub ok 2024-02-23T00:44:38.921 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:36.755061+0000 osd.7 (osd.7) 117 : cluster [DBG] 6.8 scrub starts 2024-02-23T00:44:38.921 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:36.757955+0000 osd.7 (osd.7) 118 : cluster [DBG] 6.8 scrub ok 2024-02-23T00:44:38.921 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:37.068084+0000 osd.4 (osd.4) 99 : cluster [DBG] 5.1c scrub starts 2024-02-23T00:44:38.921 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:37.068830+0000 osd.4 (osd.4) 100 : cluster [DBG] 5.1c scrub ok 2024-02-23T00:44:38.921 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:37.104294+0000 osd.3 (osd.3) 119 : cluster [DBG] 5.13 scrub starts 2024-02-23T00:44:38.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:37.105008+0000 osd.3 (osd.3) 120 : cluster [DBG] 5.13 scrub ok 2024-02-23T00:44:38.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:37.227099+0000 mgr.y (mgr.14517) 240 : cluster [DBG] pgmap v283: 258 pgs: 13 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 225 active+clean; 43 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 7.2 KiB/s rd, 10 KiB/s wr, 4 op/s 2024-02-23T00:44:38.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:37.769432+0000 mon.b (mon.1) 261 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:38.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:38.117712+0000 mon.a (mon.0) 1475 : audit [INF] from='client.? 172.21.15.47:0/1013032236' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi047-82378-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:44:38.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:38.117813+0000 mon.a (mon.0) 1476 : cluster [DBG] osdmap e206: 8 total, 8 up, 8 in 2024-02-23T00:44:38.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:38.121849+0000 mon.a (mon.0) 1477 : audit [INF] from='client.? 172.21.15.47:0/1013032236' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi047-82378-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi047-82378-28"}]: dispatch 2024-02-23T00:44:38.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:38.129478+0000 mon.b (mon.1) 262 : audit [INF] from='client.? 172.21.15.47:0/1594396895' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi047-82386-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:38.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:38.137000+0000 mon.a (mon.0) 1478 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi047-82386-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:38.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:38.149498+0000 mon.a (mon.0) 1479 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:44:38.923 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:38.945 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:44:38.947 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:38.948 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:44:39.049 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:44:39.079 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:39.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.083+0000 7fcfc8cb6700 1 -- 172.21.15.47:0/392317031 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcfc40f8420 msgr2=0x7fcfc4104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:39.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.083+0000 7fcfc8cb6700 1 --2- 172.21.15.47:0/392317031 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcfc40f8420 0x7fcfc4104a20 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 rx=0x7fcfbc004870 tx=0x7fcfbc009cb0).stop 2024-02-23T00:44:39.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.083+0000 7fcfc8cb6700 1 -- 172.21.15.47:0/392317031 shutdown_connections 2024-02-23T00:44:39.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.083+0000 7fcfc8cb6700 1 --2- 172.21.15.47:0/392317031 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcfc40f8420 0x7fcfc4104a20 unknown :-1 s=CLOSED pgs=2530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:39.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.083+0000 7fcfc8cb6700 1 --2- 172.21.15.47:0/392317031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcfc40fdbb0 0x7fcfc40fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:39.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.083+0000 7fcfc8cb6700 1 --2- 172.21.15.47:0/392317031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfc40fe510 0x7fcfc40f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:39.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.083+0000 7fcfc8cb6700 1 -- 172.21.15.47:0/392317031 >> 172.21.15.47:0/392317031 conn(0x7fcfc40f3800 msgr2=0x7fcfc40f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:39.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.083+0000 7fcfc8cb6700 1 -- 172.21.15.47:0/392317031 shutdown_connections 2024-02-23T00:44:39.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.083+0000 7fcfc8cb6700 1 -- 172.21.15.47:0/392317031 wait complete. 2024-02-23T00:44:39.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.084+0000 7fcfc8cb6700 1 librados: starting msgr at 2024-02-23T00:44:39.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.084+0000 7fcfc8cb6700 1 librados: starting objecter 2024-02-23T00:44:39.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.084+0000 7fcfc8cb6700 1 Processor -- start 2024-02-23T00:44:39.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.084+0000 7fcfc8cb6700 1 -- start start 2024-02-23T00:44:39.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.084+0000 7fcfc8cb6700 1 librados: setting wanted keys 2024-02-23T00:44:39.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.084+0000 7fcfc8cb6700 1 librados: calling monclient init 2024-02-23T00:44:39.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.084+0000 7fcfc8cb6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcfc40f8420 0x7fcfc410dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:39.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.084+0000 7fcfc8cb6700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcfc40fdbb0 0x7fcfc410e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:39.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.084+0000 7fcfc8cb6700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfc40fe510 0x7fcfc4107e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:39.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.084+0000 7fcfc8cb6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc4108330 con 0x7fcfc40f8420 2024-02-23T00:44:39.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.084+0000 7fcfc8cb6700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc41084a0 con 0x7fcfc40fe510 2024-02-23T00:44:39.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.084+0000 7fcfc8cb6700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc4108660 con 0x7fcfc40fdbb0 2024-02-23T00:44:39.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.084+0000 7fcfc259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcfc40f8420 0x7fcfc410dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:39.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.084+0000 7fcfc1d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcfc40fdbb0 0x7fcfc410e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:39.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.084+0000 7fcfc1d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcfc40fdbb0 0x7fcfc410e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:35580/0 (socket says 172.21.15.47:35580) 2024-02-23T00:44:39.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.084+0000 7fcfc1d9b700 1 -- 172.21.15.47:0/1693257906 learned_addr learned my addr 172.21.15.47:0/1693257906 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:44:39.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.084+0000 7fcfc2d9d700 1 --2- 172.21.15.47:0/1693257906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfc40fe510 0x7fcfc4107e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:39.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.085+0000 7fcfc1d9b700 1 -- 172.21.15.47:0/1693257906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfc40fe510 msgr2=0x7fcfc4107e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:39.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.085+0000 7fcfc1d9b700 1 --2- 172.21.15.47:0/1693257906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfc40fe510 0x7fcfc4107e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:39.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.085+0000 7fcfc1d9b700 1 -- 172.21.15.47:0/1693257906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcfc40f8420 msgr2=0x7fcfc410dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:39.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.085+0000 7fcfc1d9b700 1 --2- 172.21.15.47:0/1693257906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcfc40f8420 0x7fcfc410dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:39.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.085+0000 7fcfc1d9b700 1 -- 172.21.15.47:0/1693257906 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb40097c0 con 0x7fcfc40fdbb0 2024-02-23T00:44:39.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.085+0000 7fcfc1d9b700 1 --2- 172.21.15.47:0/1693257906 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcfc40fdbb0 0x7fcfc410e190 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 rx=0x7fcfac00e970 tx=0x7fcfac00b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:39.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.085+0000 7fcfbb7fe700 10 client.?.objecter ms_handle_connect 0x7fcfc40fdbb0 2024-02-23T00:44:39.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.085+0000 7fcfbb7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:44:39.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.085+0000 7fcfbb7fe700 1 -- 172.21.15.47:0/1693257906 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac004660 con 0x7fcfc40fdbb0 2024-02-23T00:44:39.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.085+0000 7fcfbb7fe700 1 -- 172.21.15.47:0/1693257906 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fcfac00f040 con 0x7fcfc40fdbb0 2024-02-23T00:44:39.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.085+0000 7fcfc8cb6700 1 -- 172.21.15.47:0/1693257906 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfbc005040 con 0x7fcfc40fdbb0 2024-02-23T00:44:39.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.085+0000 7fcfbb7fe700 1 -- 172.21.15.47:0/1693257906 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac0146f0 con 0x7fcfc40fdbb0 2024-02-23T00:44:39.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.085+0000 7fcfc8cb6700 10 client.40706.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:44:39.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.085+0000 7fcfc8cb6700 1 -- 172.21.15.47:0/1693257906 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfc4114ad0 con 0x7fcfc40fdbb0 2024-02-23T00:44:39.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.085+0000 7fcfc8cb6700 1 librados: init done 2024-02-23T00:44:39.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.086+0000 7fcfc8cb6700 1 -- 172.21.15.47:0/1693257906 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfc405ee80 con 0x7fcfc40fdbb0 2024-02-23T00:44:39.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.086+0000 7fcfbb7fe700 1 -- 172.21.15.47:0/1693257906 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fcfac007b40 con 0x7fcfc40fdbb0 2024-02-23T00:44:39.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.087+0000 7fcfbb7fe700 1 --2- 172.21.15.47:0/1693257906 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcfb0062cc0 0x7fcfb0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:39.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.087+0000 7fcfbb7fe700 1 -- 172.21.15.47:0/1693257906 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(206..206 src has 48..206) v4 ==== 11122+0+0 (secure 0 0 0) 0x7fcfac07d810 con 0x7fcfc40fdbb0 2024-02-23T00:44:39.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.087+0000 7fcfbb7fe700 10 client.40706.objecter ms_dispatch 0x7fcfc4001550 osd_map(206..206 src has 48..206) v4 2024-02-23T00:44:39.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.087+0000 7fcfbb7fe700 3 client.40706.objecter handle_osd_map got epochs [206,206] > 0 2024-02-23T00:44:39.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.087+0000 7fcfbb7fe700 3 client.40706.objecter handle_osd_map decoding full epoch 206 2024-02-23T00:44:39.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.087+0000 7fcfc259c700 1 --2- 172.21.15.47:0/1693257906 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcfb0062cc0 0x7fcfb0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:39.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.087+0000 7fcfbb7fe700 20 client.40706.objecter dump_active .. 0 homeless 2024-02-23T00:44:39.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.087+0000 7fcfbb7fe700 10 client.40706.objecter ms_handle_connect 0x7fcfb0062cc0 2024-02-23T00:44:39.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.087+0000 7fcfc259c700 1 --2- 172.21.15.47:0/1693257906 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcfb0062cc0 0x7fcfb0065180 secure :-1 s=READY pgs=2954 cs=0 l=1 rev1=1 rx=0x7fcfb4001530 tx=0x7fcfb4004c50).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:39.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.091+0000 7fcfbb7fe700 1 -- 172.21.15.47:0/1693257906 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fcfac04c7e0 con 0x7fcfc40fdbb0 2024-02-23T00:44:39.100 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:44:39.103 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:39.103 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:44:39.204 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:44:39.246 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:39.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:39.325+0000 7fcfc8cb6700 1 -- 172.21.15.47:0/1693257906 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd reweight", "id": 6, "weight": 1.0} v 0) v1 -- 0x7fcfc41093c0 con 0x7fcfc40fdbb0 2024-02-23T00:44:39.941 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-23T00:44:39.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:38.770688+0000 mon.b (mon.1) 263 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:39.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:39.326580+0000 mon.c (mon.2) 131 : audit [INF] from='client.? 172.21.15.47:0/1693257906' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-23T00:44:39.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:39.576676+0000 mon.a (mon.0) 1480 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi047-82386-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:39.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:39.576774+0000 mon.a (mon.0) 1481 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:44:39.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:39.576836+0000 mon.a (mon.0) 1482 : cluster [DBG] osdmap e207: 8 total, 8 up, 8 in 2024-02-23T00:44:39.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:39.578781+0000 mon.a (mon.0) 1483 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-15"}]: dispatch 2024-02-23T00:44:39.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:39.580506+0000 mon.a (mon.0) 1484 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-23T00:44:39.968 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:38.770688+0000 mon.b (mon.1) 263 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:39.968 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:39.326580+0000 mon.c (mon.2) 131 : audit [INF] from='client.? 172.21.15.47:0/1693257906' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-23T00:44:39.968 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:39.576676+0000 mon.a (mon.0) 1480 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi047-82386-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:39.968 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:39.576774+0000 mon.a (mon.0) 1481 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:44:39.969 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:39.576836+0000 mon.a (mon.0) 1482 : cluster [DBG] osdmap e207: 8 total, 8 up, 8 in 2024-02-23T00:44:39.969 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:39.578781+0000 mon.a (mon.0) 1483 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-15"}]: dispatch 2024-02-23T00:44:39.969 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:39.580506+0000 mon.a (mon.0) 1484 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-23T00:44:39.969 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:38.770688+0000 mon.b (mon.1) 263 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:39.969 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:39.326580+0000 mon.c (mon.2) 131 : audit [INF] from='client.? 172.21.15.47:0/1693257906' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-23T00:44:39.969 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:39.576676+0000 mon.a (mon.0) 1480 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi047-82386-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:39.969 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:39.576774+0000 mon.a (mon.0) 1481 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:44:39.969 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:39.576836+0000 mon.a (mon.0) 1482 : cluster [DBG] osdmap e207: 8 total, 8 up, 8 in 2024-02-23T00:44:39.969 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:39.578781+0000 mon.a (mon.0) 1483 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-15"}]: dispatch 2024-02-23T00:44:39.969 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:39.580506+0000 mon.a (mon.0) 1484 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-23T00:44:40.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:40.131+0000 7fcfbb7fe700 1 -- 172.21.15.47:0/1693257906 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 6, "weight": 1.0}]=0 reweighted osd.6 to 1 (10000) v208) v1 ==== 113+0+0 (secure 0 0 0) 0x7fcfac019070 con 0x7fcfc40fdbb0 2024-02-23T00:44:40.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:40.135+0000 7fcfc8cb6700 10 librados: watch_flush enter 2024-02-23T00:44:40.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:40.135+0000 7fcfc8cb6700 10 librados: watch_flush exit 2024-02-23T00:44:40.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:40.135+0000 7fcfc8cb6700 20 client.40706.objecter shutdown clearing up homeless session... 2024-02-23T00:44:40.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:40.135+0000 7fcfc8cb6700 10 client.40706.objecter successfully canceled tick 2024-02-23T00:44:40.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:40.136+0000 7fcfc8cb6700 1 -- 172.21.15.47:0/1693257906 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcfb0062cc0 msgr2=0x7fcfb0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:40.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:40.136+0000 7fcfc8cb6700 1 --2- 172.21.15.47:0/1693257906 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcfb0062cc0 0x7fcfb0065180 secure :-1 s=READY pgs=2954 cs=0 l=1 rev1=1 rx=0x7fcfb4001530 tx=0x7fcfb4004c50).stop 2024-02-23T00:44:40.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:40.136+0000 7fcfc8cb6700 1 -- 172.21.15.47:0/1693257906 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcfc40fdbb0 msgr2=0x7fcfc410e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:40.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:40.136+0000 7fcfc8cb6700 1 --2- 172.21.15.47:0/1693257906 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcfc40fdbb0 0x7fcfc410e190 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 rx=0x7fcfac00e970 tx=0x7fcfac00b040).stop 2024-02-23T00:44:40.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:40.141+0000 7fcfc8cb6700 1 -- 172.21.15.47:0/1693257906 shutdown_connections 2024-02-23T00:44:40.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:40.141+0000 7fcfc8cb6700 1 --2- 172.21.15.47:0/1693257906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfc40fe510 0x7fcfc4107e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:40.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:40.141+0000 7fcfc8cb6700 1 --2- 172.21.15.47:0/1693257906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcfc40f8420 0x7fcfc410dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:40.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:40.141+0000 7fcfc8cb6700 1 --2- 172.21.15.47:0/1693257906 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcfb0062cc0 0x7fcfb0065180 unknown :-1 s=CLOSED pgs=2954 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:40.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:40.141+0000 7fcfc8cb6700 1 --2- 172.21.15.47:0/1693257906 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcfc40fdbb0 0x7fcfc410e190 unknown :-1 s=CLOSED pgs=2531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:40.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:40.141+0000 7fcfc8cb6700 1 -- 172.21.15.47:0/1693257906 >> 172.21.15.47:0/1693257906 conn(0x7fcfc40f3800 msgr2=0x7fcfc40f5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:40.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:40.144+0000 7fcfc8cb6700 1 -- 172.21.15.47:0/1693257906 shutdown_connections 2024-02-23T00:44:40.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:40.145+0000 7fcfc8cb6700 1 -- 172.21.15.47:0/1693257906 wait complete. 2024-02-23T00:44:40.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:40.145+0000 7fcfc8cb6700 1 librados: shutdown 2024-02-23T00:44:40.149 INFO:teuthology.orchestra.run.smithi047.stderr:reweighted osd.6 to 1 (10000) 2024-02-23T00:44:40.616 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:44:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:44:40] "GET /metrics HTTP/1.1" 200 196117 "" "Prometheus/2.33.4" 2024-02-23T00:44:40.618 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:39.227587+0000 mgr.y (mgr.14517) 241 : cluster [DBG] pgmap v286: 290 pgs: 32 unknown, 13 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 225 active+clean; 43 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 7.2 KiB/s rd, 10 KiB/s wr, 4 op/s 2024-02-23T00:44:40.618 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:39.771836+0000 mon.b (mon.1) 264 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:40.618 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:40.118764+0000 mon.a (mon.0) 1485 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:44:40.618 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:40.127745+0000 mon.a (mon.0) 1486 : audit [INF] from='client.? 172.21.15.47:0/1013032236' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi047-82378-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi047-82378-28"}]': finished 2024-02-23T00:44:40.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:40.127873+0000 mon.a (mon.0) 1487 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-15"}]': finished 2024-02-23T00:44:40.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:40.127941+0000 mon.a (mon.0) 1488 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-02-23T00:44:40.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:40.128062+0000 mon.a (mon.0) 1489 : cluster [DBG] osdmap e208: 8 total, 8 up, 8 in 2024-02-23T00:44:40.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:40.148807+0000 mon.b (mon.1) 265 : audit [INF] from='client.? 172.21.15.47:0/1089067269' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]: dispatch 2024-02-23T00:44:40.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:40.149524+0000 mon.a (mon.0) 1490 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]: dispatch 2024-02-23T00:44:40.795 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:44:40.798 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:40.798 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:44:40.899 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:44:40.932 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:41.097 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:39.227587+0000 mgr.y (mgr.14517) 241 : cluster [DBG] pgmap v286: 290 pgs: 32 unknown, 13 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 225 active+clean; 43 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 7.2 KiB/s rd, 10 KiB/s wr, 4 op/s 2024-02-23T00:44:41.097 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:39.771836+0000 mon.b (mon.1) 264 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:41.097 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:40.118764+0000 mon.a (mon.0) 1485 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:44:41.097 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:40.127745+0000 mon.a (mon.0) 1486 : audit [INF] from='client.? 172.21.15.47:0/1013032236' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi047-82378-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi047-82378-28"}]': finished 2024-02-23T00:44:41.097 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:40.127873+0000 mon.a (mon.0) 1487 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-15"}]': finished 2024-02-23T00:44:41.097 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:40.127941+0000 mon.a (mon.0) 1488 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-02-23T00:44:41.097 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:40.128062+0000 mon.a (mon.0) 1489 : cluster [DBG] osdmap e208: 8 total, 8 up, 8 in 2024-02-23T00:44:41.097 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:40.148807+0000 mon.b (mon.1) 265 : audit [INF] from='client.? 172.21.15.47:0/1089067269' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]: dispatch 2024-02-23T00:44:41.098 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:40.149524+0000 mon.a (mon.0) 1490 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]: dispatch 2024-02-23T00:44:41.098 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:41.098 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:39.227587+0000 mgr.y (mgr.14517) 241 : cluster [DBG] pgmap v286: 290 pgs: 32 unknown, 13 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 225 active+clean; 43 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 7.2 KiB/s rd, 10 KiB/s wr, 4 op/s 2024-02-23T00:44:41.098 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:39.771836+0000 mon.b (mon.1) 264 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:41.098 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:40.118764+0000 mon.a (mon.0) 1485 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:44:41.098 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:40.127745+0000 mon.a (mon.0) 1486 : audit [INF] from='client.? 172.21.15.47:0/1013032236' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi047-82378-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi047-82378-28"}]': finished 2024-02-23T00:44:41.098 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:40.127873+0000 mon.a (mon.0) 1487 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-15"}]': finished 2024-02-23T00:44:41.098 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:40.127941+0000 mon.a (mon.0) 1488 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-02-23T00:44:41.098 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:40.128062+0000 mon.a (mon.0) 1489 : cluster [DBG] osdmap e208: 8 total, 8 up, 8 in 2024-02-23T00:44:41.098 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:40.148807+0000 mon.b (mon.1) 265 : audit [INF] from='client.? 172.21.15.47:0/1089067269' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]: dispatch 2024-02-23T00:44:41.098 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:40.149524+0000 mon.a (mon.0) 1490 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]: dispatch 2024-02-23T00:44:41.099 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:44:41.099 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:44:41.200 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:44:41.229 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:41.248 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:44:41.251 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:41.251 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:44:41.332 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight 7 1 2024-02-23T00:44:41.352 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:44:41.379 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:41.398 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:44:41.401 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:41.401 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:44:41.502 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:44:41.528 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:41.546 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:44:41.549 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:41.549 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:44:41.623 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:40.772878+0000 mon.b (mon.1) 266 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:41.623 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:41.128895+0000 mon.a (mon.0) 1491 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]': finished 2024-02-23T00:44:41.623 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:41.129009+0000 mon.a (mon.0) 1492 : cluster [DBG] osdmap e209: 8 total, 8 up, 8 in 2024-02-23T00:44:41.623 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:41.133524+0000 mon.b (mon.1) 267 : audit [INF] from='client.? 172.21.15.47:0/1089067269' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]: dispatch 2024-02-23T00:44:41.650 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:44:41.676 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:41.695 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:44:41.697 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:41.698 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:44:41.798 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:44:41.822 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:40.772878+0000 mon.b (mon.1) 266 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:41.822 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:41.128895+0000 mon.a (mon.0) 1491 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]': finished 2024-02-23T00:44:41.822 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:41.129009+0000 mon.a (mon.0) 1492 : cluster [DBG] osdmap e209: 8 total, 8 up, 8 in 2024-02-23T00:44:41.822 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:41.133524+0000 mon.b (mon.1) 267 : audit [INF] from='client.? 172.21.15.47:0/1089067269' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]: dispatch 2024-02-23T00:44:41.822 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:41.145664+0000 mon.b (mon.1) 268 : audit [INF] from='client.? 172.21.15.47:0/1542833380' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi047-82386-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:41.822 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:41.581925+0000 mon.a (mon.0) 1493 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]: dispatch 2024-02-23T00:44:41.823 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:41.582911+0000 mon.a (mon.0) 1494 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi047-82386-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:41.823 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:41.585992+0000 mon.a (mon.0) 1495 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:44:41.824 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:41.842 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:44:41.845 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:41.845 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:44:41.868 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:44:41.945 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:44:41.971 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:41.990 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:44:41.993 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:41.993 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:44:42.034 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:41.145664+0000 mon.b (mon.1) 268 : audit [INF] from='client.? 172.21.15.47:0/1542833380' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi047-82386-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:42.034 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:41.581925+0000 mon.a (mon.0) 1493 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]: dispatch 2024-02-23T00:44:42.034 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:41.582911+0000 mon.a (mon.0) 1494 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi047-82386-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:42.034 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:41.585992+0000 mon.a (mon.0) 1495 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:44:42.034 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:40.772878+0000 mon.b (mon.1) 266 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:42.034 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:41.128895+0000 mon.a (mon.0) 1491 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]': finished 2024-02-23T00:44:42.034 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:41.129009+0000 mon.a (mon.0) 1492 : cluster [DBG] osdmap e209: 8 total, 8 up, 8 in 2024-02-23T00:44:42.034 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:41.133524+0000 mon.b (mon.1) 267 : audit [INF] from='client.? 172.21.15.47:0/1089067269' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]: dispatch 2024-02-23T00:44:42.035 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:41.145664+0000 mon.b (mon.1) 268 : audit [INF] from='client.? 172.21.15.47:0/1542833380' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi047-82386-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:42.035 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:41.581925+0000 mon.a (mon.0) 1493 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]: dispatch 2024-02-23T00:44:42.035 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:41.582911+0000 mon.a (mon.0) 1494 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi047-82386-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:42.035 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:41.585992+0000 mon.a (mon.0) 1495 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:44:42.094 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:44:42.120 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:42.144 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:44:42.149 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:42.149 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:44:42.162 INFO:tasks.workunit.client.0.smithi047.stdout:: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-02-23T00:44:42.162 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-02-23T00:44:42.162 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (4056 ms) 2024-02-23T00:44:42.162 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-02-23T00:44:42.162 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (3937 ms) 2024-02-23T00:44:42.162 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-02-23T00:44:42.163 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (4452 ms) 2024-02-23T00:44:42.163 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (12445 ms total) 2024-02-23T00:44:42.163 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: 2024-02-23T00:44:42.163 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2024-02-23T00:44:42.163 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [==========] 20 tests from 5 test suites ran. (114954 ms total) 2024-02-23T00:44:42.163 INFO:tasks.workunit.client.0.smithi047.stdout: api_snapshots_pp: [ PASSED ] 20 tests. 2024-02-23T00:44:42.164 INFO:tasks.workunit.client.0.smithi047.stderr:+ for t in "${!pids[@]}" 2024-02-23T00:44:42.164 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82442 2024-02-23T00:44:42.164 INFO:tasks.workunit.client.0.smithi047.stderr:+ wait 82442 2024-02-23T00:44:42.164 INFO:tasks.workunit.client.0.smithi047.stderr:+ for t in "${!pids[@]}" 2024-02-23T00:44:42.164 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82433 2024-02-23T00:44:42.164 INFO:tasks.workunit.client.0.smithi047.stderr:+ wait 82433 2024-02-23T00:44:42.164 INFO:tasks.workunit.client.0.smithi047.stderr:+ for t in "${!pids[@]}" 2024-02-23T00:44:42.164 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82413 2024-02-23T00:44:42.164 INFO:tasks.workunit.client.0.smithi047.stderr:+ wait 82413 2024-02-23T00:44:42.250 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:44:42.276 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:42.295 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:44:42.297 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:42.298 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:44:42.398 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:44:42.432 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:42.459 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:44:42.481 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:44:42.483 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:42.484 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:44:42.584 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:44:42.611 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:42.630 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:44:42.632 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:42.633 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:44:42.693 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:44:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:44:42.693 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:44:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:44:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:44:42.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:41.228245+0000 mgr.y (mgr.14517) 242 : cluster [DBG] pgmap v289: 258 pgs: 40 unknown, 13 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 189 active+clean; 11 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:44:42.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:41.774295+0000 mon.b (mon.1) 269 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:42.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:42.125548+0000 mon.a (mon.0) 1496 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:44:42.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:42.132801+0000 mon.a (mon.0) 1497 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]': finished 2024-02-23T00:44:42.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:42.132963+0000 mon.a (mon.0) 1498 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi047-82386-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:42.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:42.133036+0000 mon.a (mon.0) 1499 : cluster [DBG] osdmap e210: 8 total, 8 up, 8 in 2024-02-23T00:44:42.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:42.134990+0000 mon.a (mon.0) 1500 : audit [INF] from='client.? 172.21.15.47:0/1013032236' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi047-82378-28"}]: dispatch 2024-02-23T00:44:42.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:42.143860+0000 mon.a (mon.0) 1501 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:42.733 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:44:42.759 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:42.778 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:44:42.781 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:42.781 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:44:42.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:41.228245+0000 mgr.y (mgr.14517) 242 : cluster [DBG] pgmap v289: 258 pgs: 40 unknown, 13 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 189 active+clean; 11 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:44:42.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:41.774295+0000 mon.b (mon.1) 269 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:42.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:42.125548+0000 mon.a (mon.0) 1496 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:44:42.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:42.132801+0000 mon.a (mon.0) 1497 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]': finished 2024-02-23T00:44:42.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:42.132963+0000 mon.a (mon.0) 1498 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi047-82386-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:42.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:42.133036+0000 mon.a (mon.0) 1499 : cluster [DBG] osdmap e210: 8 total, 8 up, 8 in 2024-02-23T00:44:42.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:42.134990+0000 mon.a (mon.0) 1500 : audit [INF] from='client.? 172.21.15.47:0/1013032236' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi047-82378-28"}]: dispatch 2024-02-23T00:44:42.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:42.143860+0000 mon.a (mon.0) 1501 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:42.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:41.228245+0000 mgr.y (mgr.14517) 242 : cluster [DBG] pgmap v289: 258 pgs: 40 unknown, 13 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 189 active+clean; 11 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:44:42.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:41.774295+0000 mon.b (mon.1) 269 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:42.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:42.125548+0000 mon.a (mon.0) 1496 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:44:42.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:42.132801+0000 mon.a (mon.0) 1497 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi047-82496-20"}]': finished 2024-02-23T00:44:42.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:42.132963+0000 mon.a (mon.0) 1498 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi047-82386-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:42.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:42.133036+0000 mon.a (mon.0) 1499 : cluster [DBG] osdmap e210: 8 total, 8 up, 8 in 2024-02-23T00:44:42.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:42.134990+0000 mon.a (mon.0) 1500 : audit [INF] from='client.? 172.21.15.47:0/1013032236' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi047-82378-28"}]: dispatch 2024-02-23T00:44:42.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:42.143860+0000 mon.a (mon.0) 1501 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:42.882 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:44:42.911 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:43.165 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: Running main() from gmock_main.cc 2024-02-23T00:44:43.165 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [==========] Running 56 tests from 4 test suites. 2024-02-23T00:44:43.165 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [----------] Global test environment set-up. 2024-02-23T00:44:43.165 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [----------] 31 tests from LibRadosAio 2024-02-23T00:44:43.165 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2024-02-23T00:44:43.166 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (12503 ms) 2024-02-23T00:44:43.166 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2024-02-23T00:44:43.166 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (21838 ms) 2024-02-23T00:44:43.166 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2024-02-23T00:44:43.166 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (7039 ms) 2024-02-23T00:44:43.166 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2024-02-23T00:44:43.166 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (3579 ms) 2024-02-23T00:44:43.166 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2024-02-23T00:44:43.166 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (2739 ms) 2024-02-23T00:44:43.166 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2024-02-23T00:44:43.166 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (4486 ms) 2024-02-23T00:44:43.166 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2024-02-23T00:44:43.166 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (2744 ms) 2024-02-23T00:44:43.167 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-02-23T00:44:43.167 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (2968 ms) 2024-02-23T00:44:43.167 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2024-02-23T00:44:43.167 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (3129 ms) 2024-02-23T00:44:43.167 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2024-02-23T00:44:43.167 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (2868 ms) 2024-02-23T00:44:43.167 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2024-02-23T00:44:43.167 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (3007 ms) 2024-02-23T00:44:43.167 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2024-02-23T00:44:43.167 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (3000 ms) 2024-02-23T00:44:43.167 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2024-02-23T00:44:43.167 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (3022 ms) 2024-02-23T00:44:43.167 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-02-23T00:44:43.167 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (2994 ms) 2024-02-23T00:44:43.168 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-02-23T00:44:43.168 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (2979 ms) 2024-02-23T00:44:43.168 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-02-23T00:44:43.168 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (3025 ms) 2024-02-23T00:44:43.168 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-02-23T00:44:43.168 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (4332 ms) 2024-02-23T00:44:43.168 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2024-02-23T00:44:43.168 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (2636 ms) 2024-02-23T00:44:43.168 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2024-02-23T00:44:43.168 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (3025 ms) 2024-02-23T00:44:43.169 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2024-02-23T00:44:43.169 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (3000 ms) 2024-02-23T00:44:43.169 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2024-02-23T00:44:43.169 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (3431 ms) 2024-02-23T00:44:43.169 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2024-02-23T00:44:43.169 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (2567 ms) 2024-02-23T00:44:43.169 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2024-02-23T00:44:43.169 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (3019 ms) 2024-02-23T00:44:43.169 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2024-02-23T00:44:43.169 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (3018 ms) 2024-02-23T00:44:43.169 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2024-02-23T00:44:43.169 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (2949 ms) 2024-02-23T00:44:43.169 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2024-02-23T00:44:43.169 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (3030 ms) 2024-02-23T00:44:43.170 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2024-02-23T00:44:43.667 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:44:43.668 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:44:43.862 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:44:43.865 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:42.396821+0000 mgr.y (mgr.14517) 243 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:44:43.865 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:42.775176+0000 mon.b (mon.1) 270 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:43.865 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:43.134597+0000 mon.a (mon.0) 1502 : audit [INF] from='client.? 172.21.15.47:0/1013032236' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi047-82378-28"}]': finished 2024-02-23T00:44:43.865 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:43.134746+0000 mon.a (mon.0) 1503 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:43.865 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:43.134868+0000 mon.a (mon.0) 1504 : cluster [DBG] osdmap e211: 8 total, 8 up, 8 in 2024-02-23T00:44:43.865 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:43.138242+0000 mon.a (mon.0) 1505 : audit [INF] from='client.? 172.21.15.47:0/1013032236' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi047-82378-28"}]: dispatch 2024-02-23T00:44:43.865 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:43.156970+0000 mon.a (mon.0) 1506 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:44:43.866 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:42.396821+0000 mgr.y (mgr.14517) 243 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:44:43.866 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:42.775176+0000 mon.b (mon.1) 270 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:43.866 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:43.134597+0000 mon.a (mon.0) 1502 : audit [INF] from='client.? 172.21.15.47:0/1013032236' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi047-82378-28"}]': finished 2024-02-23T00:44:43.866 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:43.134746+0000 mon.a (mon.0) 1503 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:43.866 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:43.134868+0000 mon.a (mon.0) 1504 : cluster [DBG] osdmap e211: 8 total, 8 up, 8 in 2024-02-23T00:44:43.866 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:43.138242+0000 mon.a (mon.0) 1505 : audit [INF] from='client.? 172.21.15.47:0/1013032236' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi047-82378-28"}]: dispatch 2024-02-23T00:44:43.866 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:43.156970+0000 mon.a (mon.0) 1506 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:44:43.866 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:43.867 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:44:43.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:42.396821+0000 mgr.y (mgr.14517) 243 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:44:43.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:42.775176+0000 mon.b (mon.1) 270 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:43.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:43.134597+0000 mon.a (mon.0) 1502 : audit [INF] from='client.? 172.21.15.47:0/1013032236' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi047-82378-28"}]': finished 2024-02-23T00:44:43.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:43.134746+0000 mon.a (mon.0) 1503 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:43.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:43.134868+0000 mon.a (mon.0) 1504 : cluster [DBG] osdmap e211: 8 total, 8 up, 8 in 2024-02-23T00:44:43.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:43.138242+0000 mon.a (mon.0) 1505 : audit [INF] from='client.? 172.21.15.47:0/1013032236' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi047-82378-28"}]: dispatch 2024-02-23T00:44:43.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:43.156970+0000 mon.a (mon.0) 1506 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:44:43.967 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:44:43.997 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:44.072 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:44:44.078 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:44.078 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:44:44.179 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:44:44.209 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:44.228 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:44:44.231 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:44.231 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:44:44.332 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:44:44.363 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:44.872 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:43.228675+0000 mgr.y (mgr.14517) 244 : cluster [DBG] pgmap v292: 250 pgs: 32 creating+peering, 12 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 191 active+clean; 11 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:44:44.872 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:43.776324+0000 mon.b (mon.1) 271 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:44.872 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:44.138874+0000 mon.a (mon.0) 1507 : audit [INF] from='client.? 172.21.15.47:0/1013032236' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi047-82378-28"}]': finished 2024-02-23T00:44:44.872 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:44.138958+0000 mon.a (mon.0) 1508 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:44:44.872 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:44.139058+0000 mon.a (mon.0) 1509 : cluster [DBG] osdmap e212: 8 total, 8 up, 8 in 2024-02-23T00:44:44.872 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:44.146830+0000 mon.a (mon.0) 1510 : audit [INF] from='client.? 172.21.15.47:0/3233510788' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi047-82386-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:44.872 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:44.149061+0000 mon.a (mon.0) 1511 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-17"}]: dispatch 2024-02-23T00:44:44.872 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:44.166437+0000 mon.c (mon.2) 132 : audit [INF] from='client.? 172.21.15.47:0/2198787178' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi047-82378-29"}]: dispatch 2024-02-23T00:44:44.872 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:44.166780+0000 mon.a (mon.0) 1512 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi047-82378-29"}]: dispatch 2024-02-23T00:44:44.872 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:44.167294+0000 mon.c (mon.2) 133 : audit [INF] from='client.? 172.21.15.47:0/2198787178' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi047-82378-29"}]: dispatch 2024-02-23T00:44:44.872 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:44.167530+0000 mon.a (mon.0) 1513 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi047-82378-29"}]: dispatch 2024-02-23T00:44:44.873 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:44.167930+0000 mon.c (mon.2) 134 : audit [INF] from='client.? 172.21.15.47:0/2198787178' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi047-82378-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:44.873 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:44.168209+0000 mon.a (mon.0) 1514 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi047-82378-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:44.873 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:44.444416+0000 mon.a (mon.0) 1515 : audit [DBG] from='client.? 172.21.15.47:0/3694818479' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:44:44.873 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:43.228675+0000 mgr.y (mgr.14517) 244 : cluster [DBG] pgmap v292: 250 pgs: 32 creating+peering, 12 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 191 active+clean; 11 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:44:44.873 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:43.776324+0000 mon.b (mon.1) 271 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:44.873 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:44.138874+0000 mon.a (mon.0) 1507 : audit [INF] from='client.? 172.21.15.47:0/1013032236' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi047-82378-28"}]': finished 2024-02-23T00:44:44.873 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:44.138958+0000 mon.a (mon.0) 1508 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:44:44.873 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:44.139058+0000 mon.a (mon.0) 1509 : cluster [DBG] osdmap e212: 8 total, 8 up, 8 in 2024-02-23T00:44:44.873 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:44.146830+0000 mon.a (mon.0) 1510 : audit [INF] from='client.? 172.21.15.47:0/3233510788' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi047-82386-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:44.873 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:44.149061+0000 mon.a (mon.0) 1511 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-17"}]: dispatch 2024-02-23T00:44:44.873 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:44.166437+0000 mon.c (mon.2) 132 : audit [INF] from='client.? 172.21.15.47:0/2198787178' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi047-82378-29"}]: dispatch 2024-02-23T00:44:44.873 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:44.166780+0000 mon.a (mon.0) 1512 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi047-82378-29"}]: dispatch 2024-02-23T00:44:44.874 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:44.167294+0000 mon.c (mon.2) 133 : audit [INF] from='client.? 172.21.15.47:0/2198787178' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi047-82378-29"}]: dispatch 2024-02-23T00:44:44.874 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:44.167530+0000 mon.a (mon.0) 1513 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi047-82378-29"}]: dispatch 2024-02-23T00:44:44.874 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:44.167930+0000 mon.c (mon.2) 134 : audit [INF] from='client.? 172.21.15.47:0/2198787178' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi047-82378-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:44.874 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:44.168209+0000 mon.a (mon.0) 1514 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi047-82378-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:44.874 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:44.444416+0000 mon.a (mon.0) 1515 : audit [DBG] from='client.? 172.21.15.47:0/3694818479' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:44:44.875 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.0 dump_blocked_ops 2024-02-23T00:44:44.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:43.228675+0000 mgr.y (mgr.14517) 244 : cluster [DBG] pgmap v292: 250 pgs: 32 creating+peering, 12 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 191 active+clean; 11 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:44:44.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:43.776324+0000 mon.b (mon.1) 271 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:44.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:44.138874+0000 mon.a (mon.0) 1507 : audit [INF] from='client.? 172.21.15.47:0/1013032236' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi047-82378-28"}]': finished 2024-02-23T00:44:44.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:44.138958+0000 mon.a (mon.0) 1508 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:44:44.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:44.139058+0000 mon.a (mon.0) 1509 : cluster [DBG] osdmap e212: 8 total, 8 up, 8 in 2024-02-23T00:44:44.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:44.146830+0000 mon.a (mon.0) 1510 : audit [INF] from='client.? 172.21.15.47:0/3233510788' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi047-82386-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:44.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:44.149061+0000 mon.a (mon.0) 1511 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-17"}]: dispatch 2024-02-23T00:44:44.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:44.166437+0000 mon.c (mon.2) 132 : audit [INF] from='client.? 172.21.15.47:0/2198787178' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi047-82378-29"}]: dispatch 2024-02-23T00:44:44.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:44.166780+0000 mon.a (mon.0) 1512 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi047-82378-29"}]: dispatch 2024-02-23T00:44:44.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:44.167294+0000 mon.c (mon.2) 133 : audit [INF] from='client.? 172.21.15.47:0/2198787178' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi047-82378-29"}]: dispatch 2024-02-23T00:44:44.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:44.167530+0000 mon.a (mon.0) 1513 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi047-82378-29"}]: dispatch 2024-02-23T00:44:44.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:44.167930+0000 mon.c (mon.2) 134 : audit [INF] from='client.? 172.21.15.47:0/2198787178' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi047-82378-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:44.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:44.168209+0000 mon.a (mon.0) 1514 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi047-82378-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:44.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:44.444416+0000 mon.a (mon.0) 1515 : audit [DBG] from='client.? 172.21.15.47:0/3694818479' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:44:44.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.439+0000 7f461098a700 1 -- 172.21.15.47:0/1188723778 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f460c0fcbb0 msgr2=0x7f460c0fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:44.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.439+0000 7f461098a700 1 --2- 172.21.15.47:0/1188723778 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f460c0fcbb0 0x7f460c0fcf90 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 rx=0x7f45fc004870 tx=0x7f45fc009cb0).stop 2024-02-23T00:44:44.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.439+0000 7f461098a700 1 -- 172.21.15.47:0/1188723778 shutdown_connections 2024-02-23T00:44:44.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.439+0000 7f461098a700 1 --2- 172.21.15.47:0/1188723778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f460c0f7400 0x7f460c103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:44.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.439+0000 7f461098a700 1 --2- 172.21.15.47:0/1188723778 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f460c0fcbb0 0x7f460c0fcf90 unknown :-1 s=CLOSED pgs=2535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:44.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.439+0000 7f461098a700 1 --2- 172.21.15.47:0/1188723778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f460c0fd510 0x7f460c0f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:44.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.439+0000 7f461098a700 1 -- 172.21.15.47:0/1188723778 >> 172.21.15.47:0/1188723778 conn(0x7f460c0f2820 msgr2=0x7f460c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:44.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.439+0000 7f461098a700 1 -- 172.21.15.47:0/1188723778 shutdown_connections 2024-02-23T00:44:44.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.439+0000 7f461098a700 1 -- 172.21.15.47:0/1188723778 wait complete. 2024-02-23T00:44:44.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.440+0000 7f461098a700 1 librados: starting msgr at 2024-02-23T00:44:44.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.440+0000 7f461098a700 1 librados: starting objecter 2024-02-23T00:44:44.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.440+0000 7f461098a700 1 Processor -- start 2024-02-23T00:44:44.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.440+0000 7f461098a700 1 -- start start 2024-02-23T00:44:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.440+0000 7f461098a700 1 librados: setting wanted keys 2024-02-23T00:44:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.440+0000 7f461098a700 1 librados: calling monclient init 2024-02-23T00:44:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.440+0000 7f461098a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f460c0f7400 0x7f460c10dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.440+0000 7f461098a700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f460c0fcbb0 0x7f460c10e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.440+0000 7f461098a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f460c0fd510 0x7f460c108160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.440+0000 7f461098a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c108650 con 0x7f460c0fd510 2024-02-23T00:44:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.440+0000 7f461098a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c1087c0 con 0x7f460c0f7400 2024-02-23T00:44:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.440+0000 7f461098a700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f460c108980 con 0x7f460c0fcbb0 2024-02-23T00:44:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.441+0000 7f4609d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f460c0fcbb0 0x7f460c10e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.441+0000 7f460ad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f460c0fd510 0x7f460c108160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.441+0000 7f4609d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f460c0fcbb0 0x7f460c10e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:35622/0 (socket says 172.21.15.47:35622) 2024-02-23T00:44:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.441+0000 7f4609d9b700 1 -- 172.21.15.47:0/3694818479 learned_addr learned my addr 172.21.15.47:0/3694818479 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:44:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.441+0000 7f460a59c700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f460c0f7400 0x7f460c10dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.441+0000 7f460ad9d700 1 -- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f460c0fcbb0 msgr2=0x7f460c10e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.441+0000 7f460ad9d700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f460c0fcbb0 0x7f460c10e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:44.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.441+0000 7f460ad9d700 1 -- 172.21.15.47:0/3694818479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f460c0f7400 msgr2=0x7f460c10dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:44.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.441+0000 7f460ad9d700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f460c0f7400 0x7f460c10dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:44.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.441+0000 7f460ad9d700 1 -- 172.21.15.47:0/3694818479 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46040097c0 con 0x7f460c0fd510 2024-02-23T00:44:44.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.441+0000 7f4609d9b700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f460c0fcbb0 0x7f460c10e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:44:44.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.441+0000 7f460ad9d700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f460c0fd510 0x7f460c108160 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 rx=0x7f4604001cb0 tx=0x7f4604004c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:44.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.441+0000 7f46037fe700 10 client.?.objecter ms_handle_connect 0x7f460c0fd510 2024-02-23T00:44:44.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.441+0000 7f46037fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:44:44.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.441+0000 7f46037fe700 1 -- 172.21.15.47:0/3694818479 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46040079b0 con 0x7f460c0fd510 2024-02-23T00:44:44.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.441+0000 7f46037fe700 1 -- 172.21.15.47:0/3694818479 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f46040090d0 con 0x7f460c0fd510 2024-02-23T00:44:44.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.441+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45fc005040 con 0x7f460c0fd510 2024-02-23T00:44:44.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.441+0000 7f46037fe700 1 -- 172.21.15.47:0/3694818479 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460400fba0 con 0x7f460c0fd510 2024-02-23T00:44:44.970 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:44:44.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.441+0000 7f461098a700 10 client.30381.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:44:44.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.441+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f460c108f70 con 0x7f460c0fd510 2024-02-23T00:44:44.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.441+0000 7f461098a700 1 librados: init done 2024-02-23T00:44:44.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.442+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f460c05fa60 con 0x7f460c0fd510 2024-02-23T00:44:44.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.443+0000 7f46037fe700 1 -- 172.21.15.47:0/3694818479 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f4604007b10 con 0x7f460c0fd510 2024-02-23T00:44:44.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.443+0000 7f46037fe700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f45f8062cc0 0x7f45f8065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:44.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.443+0000 7f46037fe700 1 -- 172.21.15.47:0/3694818479 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(212..212 src has 48..212) v4 ==== 9868+0+0 (secure 0 0 0) 0x7f460407e120 con 0x7f460c0fd510 2024-02-23T00:44:44.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.443+0000 7f46037fe700 10 client.30381.objecter ms_dispatch 0x7f460c001550 osd_map(212..212 src has 48..212) v4 2024-02-23T00:44:44.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.443+0000 7f46037fe700 3 client.30381.objecter handle_osd_map got epochs [212,212] > 0 2024-02-23T00:44:44.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.443+0000 7f46037fe700 3 client.30381.objecter handle_osd_map decoding full epoch 212 2024-02-23T00:44:44.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.443+0000 7f460a59c700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f45f8062cc0 0x7f45f8065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:44.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.443+0000 7f46037fe700 20 client.30381.objecter dump_active .. 0 homeless 2024-02-23T00:44:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.443+0000 7f46037fe700 1 -- 172.21.15.47:0/3694818479 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v212) v1 ==== 54+0+16 (secure 0 0 0) 0x7f4604085070 con 0x7f460c0fd510 2024-02-23T00:44:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.443+0000 7f460a59c700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f45f8062cc0 0x7f45f8065180 secure :-1 s=READY pgs=2958 cs=0 l=1 rev1=1 rx=0x7f45fc004ee0 tx=0x7f45fc00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.443+0000 7f46037fe700 10 client.30381.objecter ms_handle_connect 0x7f45f8062cc0 2024-02-23T00:44:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.443+0000 7f461098a700 10 client.30381.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.443+0000 7f461098a700 15 client.30381.objecter _session_command_op_assign -1 1 2024-02-23T00:44:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.443+0000 7f461098a700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f460c0ffed0 0x7f460c1002b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.443+0000 7f461098a700 20 client.30381.objecter _get_session s=0x7f460c105ea0 osd=0 3 2024-02-23T00:44:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.443+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c105ea0 osd=0 3 2024-02-23T00:44:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.443+0000 7f461098a700 20 client.30381.objecter _get_session s=0x7f460c105ea0 osd=0 3 2024-02-23T00:44:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.443+0000 7f461098a700 15 client.30381.objecter _session_command_op_remove -1 1 2024-02-23T00:44:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.443+0000 7f461098a700 20 client.30381.objecter get_session s=0x7f460c105ea0 osd=0 3 2024-02-23T00:44:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.443+0000 7f461098a700 15 client.30381.objecter _session_command_op_assign 0 1 2024-02-23T00:44:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.443+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c105ea0 osd=0 4 2024-02-23T00:44:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.443+0000 7f461098a700 10 client.30381.objecter _send_command 1 2024-02-23T00:44:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.443+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f460c1007a0 con 0x7f460c0ffed0 2024-02-23T00:44:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.444+0000 7f4609d9b700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f460c0ffed0 0x7f460c1002b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.444+0000 7f46037fe700 10 client.30381.objecter ms_handle_connect 0x7f460c0ffed0 2024-02-23T00:44:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.444+0000 7f4609d9b700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f460c0ffed0 0x7f460c1002b0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.444+0000 7f46037fe700 1 -- 172.21.15.47:0/3694818479 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f460c1007a0 con 0x7f460c0ffed0 2024-02-23T00:44:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.444+0000 7f46037fe700 10 client.30381.objecter ms_dispatch 0x7f460c001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:44:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.444+0000 7f46037fe700 10 client.30381.objecter _finish_command 1 = system:0 2024-02-23T00:44:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.444+0000 7f46037fe700 20 client.30381.objecter put_session s=0x7f460c105ea0 osd=0 4 2024-02-23T00:44:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.444+0000 7f46037fe700 15 client.30381.objecter _session_command_op_remove 0 1 2024-02-23T00:44:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.459+0000 7f461098a700 10 client.30381.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:44:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.459+0000 7f461098a700 15 client.30381.objecter _session_command_op_assign -1 2 2024-02-23T00:44:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.459+0000 7f461098a700 20 client.30381.objecter _get_session s=0x7f460c105ea0 osd=0 3 2024-02-23T00:44:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.459+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c105ea0 osd=0 3 2024-02-23T00:44:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.459+0000 7f461098a700 20 client.30381.objecter _get_session s=0x7f460c105ea0 osd=0 3 2024-02-23T00:44:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.459+0000 7f461098a700 15 client.30381.objecter _session_command_op_remove -1 2 2024-02-23T00:44:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.459+0000 7f461098a700 20 client.30381.objecter get_session s=0x7f460c105ea0 osd=0 3 2024-02-23T00:44:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.459+0000 7f461098a700 15 client.30381.objecter _session_command_op_assign 0 2 2024-02-23T00:44:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.459+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c105ea0 osd=0 4 2024-02-23T00:44:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.459+0000 7f461098a700 10 client.30381.objecter _send_command 2 2024-02-23T00:44:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.459+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f460c100910 con 0x7f460c0ffed0 2024-02-23T00:44:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.461+0000 7f46037fe700 1 -- 172.21.15.47:0/3694818479 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f460c100910 con 0x7f460c0ffed0 2024-02-23T00:44:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.461+0000 7f46037fe700 10 client.30381.objecter ms_dispatch 0x7f460c001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:44:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.461+0000 7f46037fe700 10 client.30381.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:44:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.461+0000 7f46037fe700 20 client.30381.objecter put_session s=0x7f460c105ea0 osd=0 4 2024-02-23T00:44:44.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.461+0000 7f46037fe700 15 client.30381.objecter _session_command_op_remove 0 2 2024-02-23T00:44:44.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.461+0000 7f461098a700 10 client.30381.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:44.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.461+0000 7f461098a700 15 client.30381.objecter _session_command_op_assign -1 3 2024-02-23T00:44:44.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.461+0000 7f461098a700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f460c028540 0x7f460c118e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:44.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.461+0000 7f461098a700 20 client.30381.objecter _get_session s=0x7f460c0fbe90 osd=1 3 2024-02-23T00:44:44.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.461+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c0fbe90 osd=1 3 2024-02-23T00:44:44.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.461+0000 7f461098a700 20 client.30381.objecter _get_session s=0x7f460c0fbe90 osd=1 3 2024-02-23T00:44:44.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.461+0000 7f461098a700 15 client.30381.objecter _session_command_op_remove -1 3 2024-02-23T00:44:44.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.461+0000 7f461098a700 20 client.30381.objecter get_session s=0x7f460c0fbe90 osd=1 3 2024-02-23T00:44:44.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.461+0000 7f461098a700 15 client.30381.objecter _session_command_op_assign 1 3 2024-02-23T00:44:44.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.461+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c0fbe90 osd=1 4 2024-02-23T00:44:44.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.461+0000 7f461098a700 10 client.30381.objecter _send_command 3 2024-02-23T00:44:44.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.461+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f460c0fc000 con 0x7f460c028540 2024-02-23T00:44:44.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.461+0000 7f460a59c700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f460c028540 0x7f460c118e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:44.976 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:44:44.976 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:44:44.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.462+0000 7f46037fe700 10 client.30381.objecter ms_handle_connect 0x7f460c028540 2024-02-23T00:44:44.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.462+0000 7f460a59c700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f460c028540 0x7f460c118e70 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:44.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.462+0000 7f46037fe700 1 -- 172.21.15.47:0/3694818479 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f460c0fc000 con 0x7f460c028540 2024-02-23T00:44:44.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.462+0000 7f46037fe700 10 client.30381.objecter ms_dispatch 0x7f460c001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:44:44.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.462+0000 7f46037fe700 10 client.30381.objecter _finish_command 3 = system:0 2024-02-23T00:44:44.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.462+0000 7f46037fe700 20 client.30381.objecter put_session s=0x7f460c0fbe90 osd=1 4 2024-02-23T00:44:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.462+0000 7f46037fe700 15 client.30381.objecter _session_command_op_remove 1 3 2024-02-23T00:44:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.477+0000 7f461098a700 10 client.30381.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:44:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.477+0000 7f461098a700 15 client.30381.objecter _session_command_op_assign -1 4 2024-02-23T00:44:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.477+0000 7f461098a700 20 client.30381.objecter _get_session s=0x7f460c0fbe90 osd=1 3 2024-02-23T00:44:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.477+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c0fbe90 osd=1 3 2024-02-23T00:44:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.477+0000 7f461098a700 20 client.30381.objecter _get_session s=0x7f460c0fbe90 osd=1 3 2024-02-23T00:44:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.477+0000 7f461098a700 15 client.30381.objecter _session_command_op_remove -1 4 2024-02-23T00:44:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.477+0000 7f461098a700 20 client.30381.objecter get_session s=0x7f460c0fbe90 osd=1 3 2024-02-23T00:44:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.477+0000 7f461098a700 15 client.30381.objecter _session_command_op_assign 1 4 2024-02-23T00:44:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.477+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c0fbe90 osd=1 4 2024-02-23T00:44:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.477+0000 7f461098a700 10 client.30381.objecter _send_command 4 2024-02-23T00:44:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.477+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f460c05a040 con 0x7f460c028540 2024-02-23T00:44:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.478+0000 7f46037fe700 1 -- 172.21.15.47:0/3694818479 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f460c05a040 con 0x7f460c028540 2024-02-23T00:44:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.478+0000 7f46037fe700 10 client.30381.objecter ms_dispatch 0x7f460c001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:44:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.478+0000 7f46037fe700 10 client.30381.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:44:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.478+0000 7f46037fe700 20 client.30381.objecter put_session s=0x7f460c0fbe90 osd=1 4 2024-02-23T00:44:44.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.478+0000 7f46037fe700 15 client.30381.objecter _session_command_op_remove 1 4 2024-02-23T00:44:44.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.479+0000 7f461098a700 10 client.30381.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:44.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.479+0000 7f461098a700 15 client.30381.objecter _session_command_op_assign -1 5 2024-02-23T00:44:44.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.479+0000 7f461098a700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f460c0594d0 0x7f460c11c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:44.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.479+0000 7f461098a700 20 client.30381.objecter _get_session s=0x7f460c059360 osd=2 3 2024-02-23T00:44:44.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.479+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c059360 osd=2 3 2024-02-23T00:44:44.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.479+0000 7f461098a700 20 client.30381.objecter _get_session s=0x7f460c059360 osd=2 3 2024-02-23T00:44:44.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.479+0000 7f461098a700 15 client.30381.objecter _session_command_op_remove -1 5 2024-02-23T00:44:44.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.479+0000 7f461098a700 20 client.30381.objecter get_session s=0x7f460c059360 osd=2 3 2024-02-23T00:44:44.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.479+0000 7f461098a700 15 client.30381.objecter _session_command_op_assign 2 5 2024-02-23T00:44:44.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.479+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c059360 osd=2 4 2024-02-23T00:44:44.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.479+0000 7f461098a700 10 client.30381.objecter _send_command 5 2024-02-23T00:44:44.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.479+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f460c0280a0 con 0x7f460c0594d0 2024-02-23T00:44:44.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.479+0000 7f4609d9b700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f460c0594d0 0x7f460c11c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:44.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.479+0000 7f46037fe700 10 client.30381.objecter ms_handle_connect 0x7f460c0594d0 2024-02-23T00:44:44.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.479+0000 7f4609d9b700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f460c0594d0 0x7f460c11c4f0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:44.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.480+0000 7f46037fe700 1 -- 172.21.15.47:0/3694818479 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f460c0280a0 con 0x7f460c0594d0 2024-02-23T00:44:44.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.480+0000 7f46037fe700 10 client.30381.objecter ms_dispatch 0x7f460c001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:44:44.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.480+0000 7f46037fe700 10 client.30381.objecter _finish_command 5 = system:0 2024-02-23T00:44:44.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.480+0000 7f46037fe700 20 client.30381.objecter put_session s=0x7f460c059360 osd=2 4 2024-02-23T00:44:44.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.480+0000 7f46037fe700 15 client.30381.objecter _session_command_op_remove 2 5 2024-02-23T00:44:44.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.494+0000 7f461098a700 10 client.30381.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:44:44.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.494+0000 7f461098a700 15 client.30381.objecter _session_command_op_assign -1 6 2024-02-23T00:44:44.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.494+0000 7f461098a700 20 client.30381.objecter _get_session s=0x7f460c059360 osd=2 3 2024-02-23T00:44:44.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.494+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c059360 osd=2 3 2024-02-23T00:44:44.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.494+0000 7f461098a700 20 client.30381.objecter _get_session s=0x7f460c059360 osd=2 3 2024-02-23T00:44:44.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.494+0000 7f461098a700 15 client.30381.objecter _session_command_op_remove -1 6 2024-02-23T00:44:44.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.494+0000 7f461098a700 20 client.30381.objecter get_session s=0x7f460c059360 osd=2 3 2024-02-23T00:44:44.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.494+0000 7f461098a700 15 client.30381.objecter _session_command_op_assign 2 6 2024-02-23T00:44:44.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.494+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c059360 osd=2 4 2024-02-23T00:44:44.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.494+0000 7f461098a700 10 client.30381.objecter _send_command 6 2024-02-23T00:44:44.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.494+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f460c11c9e0 con 0x7f460c0594d0 2024-02-23T00:44:44.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.496+0000 7f46037fe700 1 -- 172.21.15.47:0/3694818479 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f460c11c9e0 con 0x7f460c0594d0 2024-02-23T00:44:44.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.496+0000 7f46037fe700 10 client.30381.objecter ms_dispatch 0x7f460c001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:44:44.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.496+0000 7f46037fe700 10 client.30381.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:44:44.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.496+0000 7f46037fe700 20 client.30381.objecter put_session s=0x7f460c059360 osd=2 4 2024-02-23T00:44:44.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.496+0000 7f46037fe700 15 client.30381.objecter _session_command_op_remove 2 6 2024-02-23T00:44:44.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.496+0000 7f461098a700 10 client.30381.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:44.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.496+0000 7f461098a700 15 client.30381.objecter _session_command_op_assign -1 7 2024-02-23T00:44:44.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.496+0000 7f461098a700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f460c027770 0x7f460c11dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:44.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.496+0000 7f461098a700 20 client.30381.objecter _get_session s=0x7f460c027600 osd=3 3 2024-02-23T00:44:44.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.496+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c027600 osd=3 3 2024-02-23T00:44:44.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.496+0000 7f461098a700 20 client.30381.objecter _get_session s=0x7f460c027600 osd=3 3 2024-02-23T00:44:44.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.496+0000 7f461098a700 15 client.30381.objecter _session_command_op_remove -1 7 2024-02-23T00:44:44.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.496+0000 7f461098a700 20 client.30381.objecter get_session s=0x7f460c027600 osd=3 3 2024-02-23T00:44:44.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.496+0000 7f461098a700 15 client.30381.objecter _session_command_op_assign 3 7 2024-02-23T00:44:44.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.496+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c027600 osd=3 4 2024-02-23T00:44:44.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.496+0000 7f461098a700 10 client.30381.objecter _send_command 7 2024-02-23T00:44:44.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.496+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f460c11a210 con 0x7f460c027770 2024-02-23T00:44:44.981 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:44:44.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.496+0000 7f460ad9d700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f460c027770 0x7f460c11dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:44.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.496+0000 7f46037fe700 10 client.30381.objecter ms_handle_connect 0x7f460c027770 2024-02-23T00:44:44.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.496+0000 7f460ad9d700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f460c027770 0x7f460c11dd30 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:44.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.497+0000 7f46037fe700 1 -- 172.21.15.47:0/3694818479 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f460c11a210 con 0x7f460c027770 2024-02-23T00:44:44.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.497+0000 7f46037fe700 10 client.30381.objecter ms_dispatch 0x7f460c001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:44:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.497+0000 7f46037fe700 10 client.30381.objecter _finish_command 7 = system:0 2024-02-23T00:44:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.497+0000 7f46037fe700 20 client.30381.objecter put_session s=0x7f460c027600 osd=3 4 2024-02-23T00:44:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.497+0000 7f46037fe700 15 client.30381.objecter _session_command_op_remove 3 7 2024-02-23T00:44:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.511+0000 7f461098a700 10 client.30381.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:44:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.511+0000 7f461098a700 15 client.30381.objecter _session_command_op_assign -1 8 2024-02-23T00:44:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.511+0000 7f461098a700 20 client.30381.objecter _get_session s=0x7f460c027600 osd=3 3 2024-02-23T00:44:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.511+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c027600 osd=3 3 2024-02-23T00:44:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.511+0000 7f461098a700 20 client.30381.objecter _get_session s=0x7f460c027600 osd=3 3 2024-02-23T00:44:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.511+0000 7f461098a700 15 client.30381.objecter _session_command_op_remove -1 8 2024-02-23T00:44:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.511+0000 7f461098a700 20 client.30381.objecter get_session s=0x7f460c027600 osd=3 3 2024-02-23T00:44:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.511+0000 7f461098a700 15 client.30381.objecter _session_command_op_assign 3 8 2024-02-23T00:44:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.511+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c027600 osd=3 4 2024-02-23T00:44:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.511+0000 7f461098a700 10 client.30381.objecter _send_command 8 2024-02-23T00:44:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.511+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f460c1195a0 con 0x7f460c027770 2024-02-23T00:44:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.513+0000 7f46037fe700 1 -- 172.21.15.47:0/3694818479 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f460c1195a0 con 0x7f460c027770 2024-02-23T00:44:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.513+0000 7f46037fe700 10 client.30381.objecter ms_dispatch 0x7f460c001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:44:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.513+0000 7f46037fe700 10 client.30381.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:44:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.513+0000 7f46037fe700 20 client.30381.objecter put_session s=0x7f460c027600 osd=3 4 2024-02-23T00:44:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.513+0000 7f46037fe700 15 client.30381.objecter _session_command_op_remove 3 8 2024-02-23T00:44:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.513+0000 7f461098a700 10 client.30381.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.513+0000 7f461098a700 15 client.30381.objecter _session_command_op_assign -1 9 2024-02-23T00:44:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.513+0000 7f461098a700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f460c11ef00 0x7f460c123380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.513+0000 7f461098a700 20 client.30381.objecter _get_session s=0x7f460c1195a0 osd=4 3 2024-02-23T00:44:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.513+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c1195a0 osd=4 3 2024-02-23T00:44:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.513+0000 7f461098a700 20 client.30381.objecter _get_session s=0x7f460c1195a0 osd=4 3 2024-02-23T00:44:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.513+0000 7f461098a700 15 client.30381.objecter _session_command_op_remove -1 9 2024-02-23T00:44:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.513+0000 7f461098a700 20 client.30381.objecter get_session s=0x7f460c1195a0 osd=4 3 2024-02-23T00:44:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.513+0000 7f461098a700 15 client.30381.objecter _session_command_op_assign 4 9 2024-02-23T00:44:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.513+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c1195a0 osd=4 4 2024-02-23T00:44:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.513+0000 7f461098a700 10 client.30381.objecter _send_command 9 2024-02-23T00:44:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.513+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f460c1198b0 con 0x7f460c11ef00 2024-02-23T00:44:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.513+0000 7f460a59c700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f460c11ef00 0x7f460c123380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.514+0000 7f46037fe700 10 client.30381.objecter ms_handle_connect 0x7f460c11ef00 2024-02-23T00:44:44.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.514+0000 7f460a59c700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f460c11ef00 0x7f460c123380 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:44.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.514+0000 7f46037fe700 1 -- 172.21.15.47:0/3694818479 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f460c1198b0 con 0x7f460c11ef00 2024-02-23T00:44:44.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.514+0000 7f46037fe700 10 client.30381.objecter ms_dispatch 0x7f460c001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:44:44.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.514+0000 7f46037fe700 10 client.30381.objecter _finish_command 9 = system:0 2024-02-23T00:44:44.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.514+0000 7f46037fe700 20 client.30381.objecter put_session s=0x7f460c1195a0 osd=4 4 2024-02-23T00:44:44.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.514+0000 7f46037fe700 15 client.30381.objecter _session_command_op_remove 4 9 2024-02-23T00:44:44.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.529+0000 7f461098a700 10 client.30381.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:44:44.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.529+0000 7f461098a700 15 client.30381.objecter _session_command_op_assign -1 10 2024-02-23T00:44:44.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.529+0000 7f461098a700 20 client.30381.objecter _get_session s=0x7f460c1195a0 osd=4 3 2024-02-23T00:44:44.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.529+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c1195a0 osd=4 3 2024-02-23T00:44:44.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.529+0000 7f461098a700 20 client.30381.objecter _get_session s=0x7f460c1195a0 osd=4 3 2024-02-23T00:44:44.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.529+0000 7f461098a700 15 client.30381.objecter _session_command_op_remove -1 10 2024-02-23T00:44:44.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.529+0000 7f461098a700 20 client.30381.objecter get_session s=0x7f460c1195a0 osd=4 3 2024-02-23T00:44:44.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.529+0000 7f461098a700 15 client.30381.objecter _session_command_op_assign 4 10 2024-02-23T00:44:44.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.529+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c1195a0 osd=4 4 2024-02-23T00:44:44.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.529+0000 7f461098a700 10 client.30381.objecter _send_command 10 2024-02-23T00:44:44.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.529+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f460c1198b0 con 0x7f460c11ef00 2024-02-23T00:44:44.985 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:44:44.985 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:44:44.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.531+0000 7f46037fe700 1 -- 172.21.15.47:0/3694818479 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f460c1198b0 con 0x7f460c11ef00 2024-02-23T00:44:44.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.531+0000 7f46037fe700 10 client.30381.objecter ms_dispatch 0x7f460c001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:44:44.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.531+0000 7f46037fe700 10 client.30381.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:44:44.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.531+0000 7f46037fe700 20 client.30381.objecter put_session s=0x7f460c1195a0 osd=4 4 2024-02-23T00:44:44.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.531+0000 7f46037fe700 15 client.30381.objecter _session_command_op_remove 4 10 2024-02-23T00:44:44.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.531+0000 7f461098a700 10 client.30381.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:44.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.531+0000 7f461098a700 15 client.30381.objecter _session_command_op_assign -1 11 2024-02-23T00:44:44.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.531+0000 7f461098a700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f460c123d80 0x7f460c126190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:44.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.531+0000 7f461098a700 20 client.30381.objecter _get_session s=0x7f460c1198b0 osd=5 3 2024-02-23T00:44:44.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.531+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c1198b0 osd=5 3 2024-02-23T00:44:44.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.531+0000 7f461098a700 20 client.30381.objecter _get_session s=0x7f460c1198b0 osd=5 3 2024-02-23T00:44:44.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.531+0000 7f461098a700 15 client.30381.objecter _session_command_op_remove -1 11 2024-02-23T00:44:44.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.531+0000 7f461098a700 20 client.30381.objecter get_session s=0x7f460c1198b0 osd=5 3 2024-02-23T00:44:44.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.531+0000 7f461098a700 15 client.30381.objecter _session_command_op_assign 5 11 2024-02-23T00:44:44.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.531+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c1198b0 osd=5 4 2024-02-23T00:44:44.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.531+0000 7f461098a700 10 client.30381.objecter _send_command 11 2024-02-23T00:44:44.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.531+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f460c128fd0 con 0x7f460c123d80 2024-02-23T00:44:44.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.532+0000 7f4609d9b700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f460c123d80 0x7f460c126190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:44.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.532+0000 7f4609d9b700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f460c123d80 0x7f460c126190 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:44.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.532+0000 7f46037fe700 10 client.30381.objecter ms_handle_connect 0x7f460c123d80 2024-02-23T00:44:44.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.533+0000 7f46037fe700 1 -- 172.21.15.47:0/3694818479 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f460c128fd0 con 0x7f460c123d80 2024-02-23T00:44:44.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.533+0000 7f46037fe700 10 client.30381.objecter ms_dispatch 0x7f460c001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:44:44.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.533+0000 7f46037fe700 10 client.30381.objecter _finish_command 11 = system:0 2024-02-23T00:44:44.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.533+0000 7f46037fe700 20 client.30381.objecter put_session s=0x7f460c1198b0 osd=5 4 2024-02-23T00:44:44.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.533+0000 7f46037fe700 15 client.30381.objecter _session_command_op_remove 5 11 2024-02-23T00:44:44.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.547+0000 7f461098a700 10 client.30381.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:44:44.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.547+0000 7f461098a700 15 client.30381.objecter _session_command_op_assign -1 12 2024-02-23T00:44:44.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.547+0000 7f461098a700 20 client.30381.objecter _get_session s=0x7f460c1198b0 osd=5 3 2024-02-23T00:44:44.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.547+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c1198b0 osd=5 3 2024-02-23T00:44:44.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.547+0000 7f461098a700 20 client.30381.objecter _get_session s=0x7f460c1198b0 osd=5 3 2024-02-23T00:44:44.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.547+0000 7f461098a700 15 client.30381.objecter _session_command_op_remove -1 12 2024-02-23T00:44:44.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.547+0000 7f461098a700 20 client.30381.objecter get_session s=0x7f460c1198b0 osd=5 3 2024-02-23T00:44:44.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.547+0000 7f461098a700 15 client.30381.objecter _session_command_op_assign 5 12 2024-02-23T00:44:44.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.547+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c1198b0 osd=5 4 2024-02-23T00:44:44.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.547+0000 7f461098a700 10 client.30381.objecter _send_command 12 2024-02-23T00:44:44.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.547+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f460c11cb50 con 0x7f460c123d80 2024-02-23T00:44:44.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.549+0000 7f46037fe700 1 -- 172.21.15.47:0/3694818479 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f460c11cb50 con 0x7f460c123d80 2024-02-23T00:44:44.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.549+0000 7f46037fe700 10 client.30381.objecter ms_dispatch 0x7f460c001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:44:44.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.549+0000 7f46037fe700 10 client.30381.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:44:44.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.549+0000 7f46037fe700 20 client.30381.objecter put_session s=0x7f460c1198b0 osd=5 4 2024-02-23T00:44:44.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.549+0000 7f46037fe700 15 client.30381.objecter _session_command_op_remove 5 12 2024-02-23T00:44:44.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.550+0000 7f461098a700 10 client.30381.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:44.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.550+0000 7f461098a700 15 client.30381.objecter _session_command_op_assign -1 13 2024-02-23T00:44:44.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.550+0000 7f461098a700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f460c11d280 0x7f460c11d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:44.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.550+0000 7f461098a700 20 client.30381.objecter _get_session s=0x7f460c11cb50 osd=6 3 2024-02-23T00:44:44.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.550+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c11cb50 osd=6 3 2024-02-23T00:44:44.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.550+0000 7f461098a700 20 client.30381.objecter _get_session s=0x7f460c11cb50 osd=6 3 2024-02-23T00:44:44.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.550+0000 7f461098a700 15 client.30381.objecter _session_command_op_remove -1 13 2024-02-23T00:44:44.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.550+0000 7f461098a700 20 client.30381.objecter get_session s=0x7f460c11cb50 osd=6 3 2024-02-23T00:44:44.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.550+0000 7f461098a700 15 client.30381.objecter _session_command_op_assign 6 13 2024-02-23T00:44:44.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.550+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c11cb50 osd=6 4 2024-02-23T00:44:44.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.550+0000 7f461098a700 10 client.30381.objecter _send_command 13 2024-02-23T00:44:44.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.550+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f460c12c1f0 con 0x7f460c11d280 2024-02-23T00:44:44.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.550+0000 7f460ad9d700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f460c11d280 0x7f460c11d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:44.989 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:44:44.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.550+0000 7f460ad9d700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f460c11d280 0x7f460c11d6e0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:44.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.550+0000 7f46037fe700 10 client.30381.objecter ms_handle_connect 0x7f460c11d280 2024-02-23T00:44:44.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.551+0000 7f46037fe700 1 -- 172.21.15.47:0/3694818479 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f460c12c1f0 con 0x7f460c11d280 2024-02-23T00:44:44.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.551+0000 7f46037fe700 10 client.30381.objecter ms_dispatch 0x7f460c001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:44:44.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.551+0000 7f46037fe700 10 client.30381.objecter _finish_command 13 = system:0 2024-02-23T00:44:44.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.551+0000 7f46037fe700 20 client.30381.objecter put_session s=0x7f460c11cb50 osd=6 4 2024-02-23T00:44:44.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.551+0000 7f46037fe700 15 client.30381.objecter _session_command_op_remove 6 13 2024-02-23T00:44:44.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.565+0000 7f461098a700 10 client.30381.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:44:44.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.565+0000 7f461098a700 15 client.30381.objecter _session_command_op_assign -1 14 2024-02-23T00:44:44.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.565+0000 7f461098a700 20 client.30381.objecter _get_session s=0x7f460c11cb50 osd=6 3 2024-02-23T00:44:44.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.565+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c11cb50 osd=6 3 2024-02-23T00:44:44.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.565+0000 7f461098a700 20 client.30381.objecter _get_session s=0x7f460c11cb50 osd=6 3 2024-02-23T00:44:44.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.566+0000 7f461098a700 15 client.30381.objecter _session_command_op_remove -1 14 2024-02-23T00:44:44.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.566+0000 7f461098a700 20 client.30381.objecter get_session s=0x7f460c11cb50 osd=6 3 2024-02-23T00:44:44.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.566+0000 7f461098a700 15 client.30381.objecter _session_command_op_assign 6 14 2024-02-23T00:44:44.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.566+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c11cb50 osd=6 4 2024-02-23T00:44:44.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.566+0000 7f461098a700 10 client.30381.objecter _send_command 14 2024-02-23T00:44:44.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.566+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f460c126790 con 0x7f460c11d280 2024-02-23T00:44:44.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.567+0000 7f46037fe700 1 -- 172.21.15.47:0/3694818479 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f460c126790 con 0x7f460c11d280 2024-02-23T00:44:44.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.568+0000 7f46037fe700 10 client.30381.objecter ms_dispatch 0x7f460c001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:44:44.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.568+0000 7f46037fe700 10 client.30381.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:44:44.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.568+0000 7f46037fe700 20 client.30381.objecter put_session s=0x7f460c11cb50 osd=6 4 2024-02-23T00:44:44.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.568+0000 7f46037fe700 15 client.30381.objecter _session_command_op_remove 6 14 2024-02-23T00:44:44.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.568+0000 7f461098a700 10 client.30381.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:44.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.568+0000 7f461098a700 15 client.30381.objecter _session_command_op_assign -1 15 2024-02-23T00:44:44.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.568+0000 7f461098a700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f460c128600 0x7f460c126790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:44.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.568+0000 7f461098a700 20 client.30381.objecter _get_session s=0x7f460c127f30 osd=7 3 2024-02-23T00:44:44.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.568+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c127f30 osd=7 3 2024-02-23T00:44:44.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.568+0000 7f461098a700 20 client.30381.objecter _get_session s=0x7f460c127f30 osd=7 3 2024-02-23T00:44:44.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.568+0000 7f461098a700 15 client.30381.objecter _session_command_op_remove -1 15 2024-02-23T00:44:44.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.568+0000 7f461098a700 20 client.30381.objecter get_session s=0x7f460c127f30 osd=7 3 2024-02-23T00:44:44.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.568+0000 7f461098a700 15 client.30381.objecter _session_command_op_assign 7 15 2024-02-23T00:44:44.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.568+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c127f30 osd=7 4 2024-02-23T00:44:44.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.568+0000 7f461098a700 10 client.30381.objecter _send_command 15 2024-02-23T00:44:44.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.568+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f460c128b80 con 0x7f460c128600 2024-02-23T00:44:44.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.568+0000 7f460a59c700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f460c128600 0x7f460c126790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:44.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.569+0000 7f460a59c700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f460c128600 0x7f460c126790 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:44.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.569+0000 7f46037fe700 10 client.30381.objecter ms_handle_connect 0x7f460c128600 2024-02-23T00:44:44.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.569+0000 7f46037fe700 1 -- 172.21.15.47:0/3694818479 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f460c128b80 con 0x7f460c128600 2024-02-23T00:44:44.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.569+0000 7f46037fe700 10 client.30381.objecter ms_dispatch 0x7f460c001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:44:44.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.569+0000 7f46037fe700 10 client.30381.objecter _finish_command 15 = system:0 2024-02-23T00:44:44.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.569+0000 7f46037fe700 20 client.30381.objecter put_session s=0x7f460c127f30 osd=7 4 2024-02-23T00:44:44.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.569+0000 7f46037fe700 15 client.30381.objecter _session_command_op_remove 7 15 2024-02-23T00:44:44.992 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:44:44.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.584+0000 7f461098a700 10 client.30381.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:44:44.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.584+0000 7f461098a700 15 client.30381.objecter _session_command_op_assign -1 16 2024-02-23T00:44:44.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.584+0000 7f461098a700 20 client.30381.objecter _get_session s=0x7f460c127f30 osd=7 3 2024-02-23T00:44:44.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.584+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c127f30 osd=7 3 2024-02-23T00:44:44.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.584+0000 7f461098a700 20 client.30381.objecter _get_session s=0x7f460c127f30 osd=7 3 2024-02-23T00:44:44.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.584+0000 7f461098a700 15 client.30381.objecter _session_command_op_remove -1 16 2024-02-23T00:44:44.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.584+0000 7f461098a700 20 client.30381.objecter get_session s=0x7f460c127f30 osd=7 3 2024-02-23T00:44:44.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.584+0000 7f461098a700 15 client.30381.objecter _session_command_op_assign 7 16 2024-02-23T00:44:44.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.584+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c127f30 osd=7 4 2024-02-23T00:44:44.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.584+0000 7f461098a700 10 client.30381.objecter _send_command 16 2024-02-23T00:44:44.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.584+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f460c128b80 con 0x7f460c128600 2024-02-23T00:44:44.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f46037fe700 1 -- 172.21.15.47:0/3694818479 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f460c128b80 con 0x7f460c128600 2024-02-23T00:44:44.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f46037fe700 10 client.30381.objecter ms_dispatch 0x7f460c001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:44:44.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f46037fe700 10 client.30381.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:44:44.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f46037fe700 20 client.30381.objecter put_session s=0x7f460c127f30 osd=7 4 2024-02-23T00:44:44.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f46037fe700 15 client.30381.objecter _session_command_op_remove 7 16 2024-02-23T00:44:44.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 10 librados: watch_flush enter 2024-02-23T00:44:44.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 10 librados: watch_flush exit 2024-02-23T00:44:44.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 10 client.30381.objecter close_session for osd.0 2024-02-23T00:44:44.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f460c0ffed0 msgr2=0x7f460c1002b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:44.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f460c0ffed0 0x7f460c1002b0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:44.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c105ea0 osd=0 1 2024-02-23T00:44:44.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 10 client.30381.objecter close_session for osd.1 2024-02-23T00:44:44.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f460c028540 msgr2=0x7f460c118e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:44.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f460c028540 0x7f460c118e70 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:44.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c0fbe90 osd=1 1 2024-02-23T00:44:44.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 10 client.30381.objecter close_session for osd.2 2024-02-23T00:44:44.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f460c0594d0 msgr2=0x7f460c11c4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:44.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f460c0594d0 0x7f460c11c4f0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:44.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c059360 osd=2 1 2024-02-23T00:44:44.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 10 client.30381.objecter close_session for osd.3 2024-02-23T00:44:44.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f460c027770 msgr2=0x7f460c11dd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:44.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f460c027770 0x7f460c11dd30 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:44.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f460ad9d700 1 -- 172.21.15.47:0/3694818479 reap_dead start 2024-02-23T00:44:44.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c027600 osd=3 1 2024-02-23T00:44:44.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 10 client.30381.objecter close_session for osd.4 2024-02-23T00:44:44.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f460c11ef00 msgr2=0x7f460c123380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:44.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f460c11ef00 0x7f460c123380 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:44.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c1195a0 osd=4 1 2024-02-23T00:44:44.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 10 client.30381.objecter close_session for osd.5 2024-02-23T00:44:44.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f460c123d80 msgr2=0x7f460c126190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:44.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f460c123d80 0x7f460c126190 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:44.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c1198b0 osd=5 1 2024-02-23T00:44:44.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 10 client.30381.objecter close_session for osd.6 2024-02-23T00:44:44.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f460ad9d700 1 -- 172.21.15.47:0/3694818479 reap_dead start 2024-02-23T00:44:44.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f460c11d280 msgr2=0x7f460c11d6e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:44.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f460c11d280 0x7f460c11d6e0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:44.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c11cb50 osd=6 1 2024-02-23T00:44:44.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 10 client.30381.objecter close_session for osd.7 2024-02-23T00:44:44.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f460c128600 msgr2=0x7f460c126790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:44.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f460c128600 0x7f460c126790 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:44.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 20 client.30381.objecter put_session s=0x7f460c127f30 osd=7 1 2024-02-23T00:44:44.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 20 client.30381.objecter shutdown clearing up homeless session... 2024-02-23T00:44:44.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 10 client.30381.objecter successfully canceled tick 2024-02-23T00:44:44.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f460ad9d700 1 -- 172.21.15.47:0/3694818479 reap_dead start 2024-02-23T00:44:44.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f45f8062cc0 msgr2=0x7f45f8065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:44.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f45f8062cc0 0x7f45f8065180 secure :-1 s=READY pgs=2958 cs=0 l=1 rev1=1 rx=0x7f45fc004ee0 tx=0x7f45fc00b040).stop 2024-02-23T00:44:44.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f460c0fd510 msgr2=0x7f460c108160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:44.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f460c0fd510 0x7f460c108160 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 rx=0x7f4604001cb0 tx=0x7f4604004c50).stop 2024-02-23T00:44:44.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 shutdown_connections 2024-02-23T00:44:44.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f45f8062cc0 0x7f45f8065180 unknown :-1 s=CLOSED pgs=2958 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:44.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 1 --2- 172.21.15.47:0/3694818479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f460c0fd510 0x7f460c108160 unknown :-1 s=CLOSED pgs=2458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:44.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 >> 172.21.15.47:0/3694818479 conn(0x7f460c0f2820 msgr2=0x7f460c060ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:44.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 shutdown_connections 2024-02-23T00:44:44.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 1 -- 172.21.15.47:0/3694818479 wait complete. 2024-02-23T00:44:44.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:44.586+0000 7f461098a700 1 librados: shutdown 2024-02-23T00:44:44.997 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:44:44.997 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:44:44.997 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:44:44.998 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:44:44.998 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:44:44.998 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:44:44.998 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:44:44.998 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:44:45.213 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:44:45.216 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:45.216 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:44:45.317 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:44:45.345 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:45.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:44.777401+0000 mon.b (mon.1) 272 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:45.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:45.113296+0000 mon.a (mon.0) 1516 : audit [INF] from='client.? 172.21.15.47:0/3233510788' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi047-82386-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:45.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:45.113436+0000 mon.a (mon.0) 1517 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-17"}]': finished 2024-02-23T00:44:45.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:45.113523+0000 mon.a (mon.0) 1518 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi047-82378-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:44:45.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:45.113598+0000 mon.a (mon.0) 1519 : cluster [DBG] osdmap e213: 8 total, 8 up, 8 in 2024-02-23T00:44:45.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:45.121267+0000 mon.c (mon.2) 135 : audit [INF] from='client.? 172.21.15.47:0/2198787178' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi047-82378-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi047-82378-29"}]: dispatch 2024-02-23T00:44:45.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:45.122541+0000 mon.a (mon.0) 1520 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi047-82378-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi047-82378-29"}]: dispatch 2024-02-23T00:44:45.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:45.123097+0000 mon.a (mon.0) 1521 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-17", "mode": "writeback"}]: dispatch 2024-02-23T00:44:46.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:44.777401+0000 mon.b (mon.1) 272 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:46.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:45.113296+0000 mon.a (mon.0) 1516 : audit [INF] from='client.? 172.21.15.47:0/3233510788' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi047-82386-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:46.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:45.113436+0000 mon.a (mon.0) 1517 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-17"}]': finished 2024-02-23T00:44:46.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:45.113523+0000 mon.a (mon.0) 1518 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi047-82378-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:44:46.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:45.113598+0000 mon.a (mon.0) 1519 : cluster [DBG] osdmap e213: 8 total, 8 up, 8 in 2024-02-23T00:44:46.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:45.121267+0000 mon.c (mon.2) 135 : audit [INF] from='client.? 172.21.15.47:0/2198787178' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi047-82378-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi047-82378-29"}]: dispatch 2024-02-23T00:44:46.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:45.122541+0000 mon.a (mon.0) 1520 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi047-82378-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi047-82378-29"}]: dispatch 2024-02-23T00:44:46.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:45.123097+0000 mon.a (mon.0) 1521 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-17", "mode": "writeback"}]: dispatch 2024-02-23T00:44:46.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:44.777401+0000 mon.b (mon.1) 272 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:46.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:45.113296+0000 mon.a (mon.0) 1516 : audit [INF] from='client.? 172.21.15.47:0/3233510788' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi047-82386-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:46.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:45.113436+0000 mon.a (mon.0) 1517 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-17"}]': finished 2024-02-23T00:44:46.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:45.113523+0000 mon.a (mon.0) 1518 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi047-82378-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:44:46.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:45.113598+0000 mon.a (mon.0) 1519 : cluster [DBG] osdmap e213: 8 total, 8 up, 8 in 2024-02-23T00:44:46.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:45.121267+0000 mon.c (mon.2) 135 : audit [INF] from='client.? 172.21.15.47:0/2198787178' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi047-82378-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi047-82378-29"}]: dispatch 2024-02-23T00:44:46.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:45.122541+0000 mon.a (mon.0) 1520 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi047-82378-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi047-82378-29"}]: dispatch 2024-02-23T00:44:46.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:45.123097+0000 mon.a (mon.0) 1521 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-17", "mode": "writeback"}]: dispatch 2024-02-23T00:44:46.494 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:44:46.497 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:46.498 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:44:46.598 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:44:46.627 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:46.647 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:44:46.649 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:46.650 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:44:46.750 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:44:46.778 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:46.798 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:44:46.801 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:46.801 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:44:46.902 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:44:46.930 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:47.145 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:45.229143+0000 mgr.y (mgr.14517) 245 : cluster [DBG] pgmap v295: 282 pgs: 32 unknown, 32 creating+peering, 12 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 191 active+clean; 11 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:44:47.145 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:45.778429+0000 mon.b (mon.1) 273 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:47.145 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:46.111016+0000 mon.a (mon.0) 1522 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:44:47.145 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:46.545487+0000 mon.a (mon.0) 1523 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-17", "mode": "writeback"}]': finished 2024-02-23T00:44:47.145 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:46.545570+0000 mon.a (mon.0) 1524 : cluster [DBG] osdmap e214: 8 total, 8 up, 8 in 2024-02-23T00:44:47.145 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:46.779553+0000 mon.b (mon.1) 274 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:47.146 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:45.229143+0000 mgr.y (mgr.14517) 245 : cluster [DBG] pgmap v295: 282 pgs: 32 unknown, 32 creating+peering, 12 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 191 active+clean; 11 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:44:47.146 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:45.778429+0000 mon.b (mon.1) 273 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:47.146 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:46.111016+0000 mon.a (mon.0) 1522 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:44:47.146 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:46.545487+0000 mon.a (mon.0) 1523 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-17", "mode": "writeback"}]': finished 2024-02-23T00:44:47.146 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:46.545570+0000 mon.a (mon.0) 1524 : cluster [DBG] osdmap e214: 8 total, 8 up, 8 in 2024-02-23T00:44:47.146 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:46.779553+0000 mon.b (mon.1) 274 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:47.180 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:44:47.183 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:47.184 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:44:47.285 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:44:47.311 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:45.229143+0000 mgr.y (mgr.14517) 245 : cluster [DBG] pgmap v295: 282 pgs: 32 unknown, 32 creating+peering, 12 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 191 active+clean; 11 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:44:47.311 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:45.778429+0000 mon.b (mon.1) 273 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:47.311 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:46.111016+0000 mon.a (mon.0) 1522 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:44:47.311 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:46.545487+0000 mon.a (mon.0) 1523 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-17", "mode": "writeback"}]': finished 2024-02-23T00:44:47.311 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:46.545570+0000 mon.a (mon.0) 1524 : cluster [DBG] osdmap e214: 8 total, 8 up, 8 in 2024-02-23T00:44:47.311 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:46.779553+0000 mon.b (mon.1) 274 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:47.312 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:47.332 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:44:47.334 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:47.335 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:44:47.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.393+0000 7fdd8565d700 1 -- 172.21.15.47:0/234264653 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdd800fdbb0 msgr2=0x7fdd800fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:47.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.393+0000 7fdd8565d700 1 --2- 172.21.15.47:0/234264653 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdd800fdbb0 0x7fdd800fdf90 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 rx=0x7fdd70004870 tx=0x7fdd70009cb0).stop 2024-02-23T00:44:47.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.393+0000 7fdd8565d700 1 -- 172.21.15.47:0/234264653 shutdown_connections 2024-02-23T00:44:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.393+0000 7fdd8565d700 1 --2- 172.21.15.47:0/234264653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd800f8420 0x7fdd80104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.393+0000 7fdd8565d700 1 --2- 172.21.15.47:0/234264653 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdd800fdbb0 0x7fdd800fdf90 unknown :-1 s=CLOSED pgs=2538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.393+0000 7fdd8565d700 1 --2- 172.21.15.47:0/234264653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd800fe510 0x7fdd800f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.393+0000 7fdd8565d700 1 -- 172.21.15.47:0/234264653 >> 172.21.15.47:0/234264653 conn(0x7fdd800f3800 msgr2=0x7fdd800f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.393+0000 7fdd8565d700 1 -- 172.21.15.47:0/234264653 shutdown_connections 2024-02-23T00:44:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.393+0000 7fdd8565d700 1 -- 172.21.15.47:0/234264653 wait complete. 2024-02-23T00:44:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.394+0000 7fdd8565d700 1 librados: starting msgr at 2024-02-23T00:44:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.394+0000 7fdd8565d700 1 librados: starting objecter 2024-02-23T00:44:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.394+0000 7fdd8565d700 1 Processor -- start 2024-02-23T00:44:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.394+0000 7fdd8565d700 1 -- start start 2024-02-23T00:44:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.394+0000 7fdd8565d700 1 librados: setting wanted keys 2024-02-23T00:44:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.394+0000 7fdd8565d700 1 librados: calling monclient init 2024-02-23T00:44:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.394+0000 7fdd8565d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd800f8420 0x7fdd8010de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.394+0000 7fdd8565d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdd800fdbb0 0x7fdd8010e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.394+0000 7fdd8565d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd800fe510 0x7fdd80107ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.394+0000 7fdd8565d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd801084e0 con 0x7fdd800f8420 2024-02-23T00:44:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.394+0000 7fdd8565d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd80108650 con 0x7fdd800fe510 2024-02-23T00:44:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.394+0000 7fdd8565d700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd80108810 con 0x7fdd800fdbb0 2024-02-23T00:44:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.394+0000 7fdd7effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd800f8420 0x7fdd8010de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.395+0000 7fdd7effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd800f8420 0x7fdd8010de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39426/0 (socket says 172.21.15.47:39426) 2024-02-23T00:44:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.395+0000 7fdd7e7fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdd800fdbb0 0x7fdd8010e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.395+0000 7fdd7effd700 1 -- 172.21.15.47:0/1688144355 learned_addr learned my addr 172.21.15.47:0/1688144355 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:44:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.395+0000 7fdd7f7fe700 1 --2- 172.21.15.47:0/1688144355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd800fe510 0x7fdd80107ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.395+0000 7fdd7e7fc700 1 -- 172.21.15.47:0/1688144355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd800fe510 msgr2=0x7fdd80107ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.395+0000 7fdd7e7fc700 1 --2- 172.21.15.47:0/1688144355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd800fe510 0x7fdd80107ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.395+0000 7fdd7e7fc700 1 -- 172.21.15.47:0/1688144355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd800f8420 msgr2=0x7fdd8010de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.395+0000 7fdd7e7fc700 1 --2- 172.21.15.47:0/1688144355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd800f8420 0x7fdd8010de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.395+0000 7fdd7e7fc700 1 -- 172.21.15.47:0/1688144355 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd780097c0 con 0x7fdd800fdbb0 2024-02-23T00:44:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.395+0000 7fdd7effd700 1 --2- 172.21.15.47:0/1688144355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd800f8420 0x7fdd8010de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:44:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.395+0000 7fdd7e7fc700 1 --2- 172.21.15.47:0/1688144355 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdd800fdbb0 0x7fdd8010e340 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 rx=0x7fdd68002780 tx=0x7fdd68009e40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.395+0000 7fdd77fff700 10 client.?.objecter ms_handle_connect 0x7fdd800fdbb0 2024-02-23T00:44:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.395+0000 7fdd77fff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:44:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.395+0000 7fdd77fff700 1 -- 172.21.15.47:0/1688144355 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd68007b40 con 0x7fdd800fdbb0 2024-02-23T00:44:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.395+0000 7fdd77fff700 1 -- 172.21.15.47:0/1688144355 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fdd6800f040 con 0x7fdd800fdbb0 2024-02-23T00:44:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.395+0000 7fdd8565d700 1 -- 172.21.15.47:0/1688144355 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd70005040 con 0x7fdd800fdbb0 2024-02-23T00:44:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.395+0000 7fdd77fff700 1 -- 172.21.15.47:0/1688144355 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd68014630 con 0x7fdd800fdbb0 2024-02-23T00:44:47.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.395+0000 7fdd8565d700 10 client.40736.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:44:47.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.395+0000 7fdd8565d700 1 -- 172.21.15.47:0/1688144355 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd80114c80 con 0x7fdd800fdbb0 2024-02-23T00:44:47.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.395+0000 7fdd8565d700 1 librados: init done 2024-02-23T00:44:47.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.396+0000 7fdd8565d700 1 -- 172.21.15.47:0/1688144355 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd800606d0 con 0x7fdd800fdbb0 2024-02-23T00:44:47.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.397+0000 7fdd77fff700 1 -- 172.21.15.47:0/1688144355 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fdd68007ca0 con 0x7fdd800fdbb0 2024-02-23T00:44:47.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.397+0000 7fdd77fff700 1 --2- 172.21.15.47:0/1688144355 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdd6c062cc0 0x7fdd6c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:47.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.397+0000 7fdd77fff700 1 -- 172.21.15.47:0/1688144355 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(215..215 src has 48..215) v4 ==== 10671+0+0 (secure 0 0 0) 0x7fdd6807db80 con 0x7fdd800fdbb0 2024-02-23T00:44:47.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.397+0000 7fdd77fff700 10 client.40736.objecter ms_dispatch 0x7fdd80001550 osd_map(215..215 src has 48..215) v4 2024-02-23T00:44:47.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.397+0000 7fdd77fff700 3 client.40736.objecter handle_osd_map got epochs [215,215] > 0 2024-02-23T00:44:47.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.397+0000 7fdd77fff700 3 client.40736.objecter handle_osd_map decoding full epoch 215 2024-02-23T00:44:47.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.397+0000 7fdd7effd700 1 --2- 172.21.15.47:0/1688144355 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdd6c062cc0 0x7fdd6c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:47.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.397+0000 7fdd77fff700 20 client.40736.objecter dump_active .. 0 homeless 2024-02-23T00:44:47.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.397+0000 7fdd77fff700 10 client.40736.objecter ms_handle_connect 0x7fdd6c062cc0 2024-02-23T00:44:47.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.397+0000 7fdd7effd700 1 --2- 172.21.15.47:0/1688144355 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdd6c062cc0 0x7fdd6c065180 secure :-1 s=READY pgs=2960 cs=0 l=1 rev1=1 rx=0x7fdd70004ee0 tx=0x7fdd7000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:47.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.401+0000 7fdd77fff700 1 -- 172.21.15.47:0/1688144355 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fdd6804cc70 con 0x7fdd800fdbb0 2024-02-23T00:44:47.435 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:44:47.464 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:47.488 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:44:47.490 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:47.491 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:44:47.591 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:44:47.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:47.597+0000 7fdd8565d700 1 -- 172.21.15.47:0/1688144355 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7fdd80106e60 con 0x7fdd800fdbb0 2024-02-23T00:44:47.618 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:47.637 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:44:47.639 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:47.639 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:44:47.740 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:44:47.766 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:47.784 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:44:47.787 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:47.787 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:44:47.888 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:44:47.915 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:47.933 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:44:47.935 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:47.936 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:44:48.036 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:44:48.063 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:48.081 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:44:48.083 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:48.084 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:44:48.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:48.127+0000 7fdd77fff700 1 -- 172.21.15.47:0/1688144355 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v216) v1 ==== 97+0+0 (secure 0 0 0) 0x7fdd68050290 con 0x7fdd800fdbb0 2024-02-23T00:44:48.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:48.133+0000 7fdd8565d700 10 librados: watch_flush enter 2024-02-23T00:44:48.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:48.133+0000 7fdd8565d700 10 librados: watch_flush exit 2024-02-23T00:44:48.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:48.133+0000 7fdd8565d700 20 client.40736.objecter shutdown clearing up homeless session... 2024-02-23T00:44:48.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:48.133+0000 7fdd8565d700 10 client.40736.objecter successfully canceled tick 2024-02-23T00:44:48.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:48.133+0000 7fdd8565d700 1 -- 172.21.15.47:0/1688144355 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdd6c062cc0 msgr2=0x7fdd6c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:48.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:48.133+0000 7fdd8565d700 1 --2- 172.21.15.47:0/1688144355 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdd6c062cc0 0x7fdd6c065180 secure :-1 s=READY pgs=2960 cs=0 l=1 rev1=1 rx=0x7fdd70004ee0 tx=0x7fdd7000b040).stop 2024-02-23T00:44:48.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:48.133+0000 7fdd8565d700 1 -- 172.21.15.47:0/1688144355 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdd800fdbb0 msgr2=0x7fdd8010e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:48.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:48.133+0000 7fdd8565d700 1 --2- 172.21.15.47:0/1688144355 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdd800fdbb0 0x7fdd8010e340 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 rx=0x7fdd68002780 tx=0x7fdd68009e40).stop 2024-02-23T00:44:48.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:48.137+0000 7fdd8565d700 1 -- 172.21.15.47:0/1688144355 shutdown_connections 2024-02-23T00:44:48.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:48.137+0000 7fdd8565d700 1 --2- 172.21.15.47:0/1688144355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd800fe510 0x7fdd80107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:48.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:48.137+0000 7fdd8565d700 1 --2- 172.21.15.47:0/1688144355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd800f8420 0x7fdd8010de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:48.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:48.137+0000 7fdd8565d700 1 --2- 172.21.15.47:0/1688144355 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdd6c062cc0 0x7fdd6c065180 secure :-1 s=CLOSED pgs=2960 cs=0 l=1 rev1=1 rx=0x7fdd70004ee0 tx=0x7fdd7000b040).stop 2024-02-23T00:44:48.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:48.137+0000 7fdd8565d700 1 --2- 172.21.15.47:0/1688144355 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdd800fdbb0 0x7fdd8010e340 unknown :-1 s=CLOSED pgs=2539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:48.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:48.137+0000 7fdd8565d700 1 -- 172.21.15.47:0/1688144355 >> 172.21.15.47:0/1688144355 conn(0x7fdd800f3800 msgr2=0x7fdd800f5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:48.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:48.141+0000 7fdd8565d700 1 -- 172.21.15.47:0/1688144355 shutdown_connections 2024-02-23T00:44:48.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:48.146+0000 7fdd8565d700 1 -- 172.21.15.47:0/1688144355 wait complete. 2024-02-23T00:44:48.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:48.146+0000 7fdd8565d700 1 librados: shutdown 2024-02-23T00:44:48.155 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is set 2024-02-23T00:44:48.176 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:47.015886+0000 mon.a (mon.0) 1525 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:44:48.177 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:47.117252+0000 mon.a (mon.0) 1526 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi047-82378-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi047-82378-29"}]': finished 2024-02-23T00:44:48.177 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:47.117386+0000 mon.a (mon.0) 1527 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:44:48.177 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:47.117437+0000 mon.a (mon.0) 1528 : cluster [DBG] osdmap e215: 8 total, 8 up, 8 in 2024-02-23T00:44:48.177 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:47.122332+0000 mon.a (mon.0) 1529 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-17"}]: dispatch 2024-02-23T00:44:48.177 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:47.122885+0000 mon.c (mon.2) 136 : audit [INF] from='client.? 172.21.15.47:0/1331945109' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi047-82386-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:48.177 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:47.123283+0000 mon.a (mon.0) 1530 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi047-82386-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:48.177 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:47.230084+0000 mgr.y (mgr.14517) 246 : cluster [DBG] pgmap v298: 290 pgs: 40 creating+peering, 11 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 224 active+clean; 10 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-23T00:44:48.177 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:47.598718+0000 mon.c (mon.2) 137 : audit [INF] from='client.? 172.21.15.47:0/1688144355' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:44:48.177 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:47.599048+0000 mon.a (mon.0) 1531 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:44:48.177 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:47.781234+0000 mon.b (mon.1) 275 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:48.177 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:47.015886+0000 mon.a (mon.0) 1525 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:44:48.177 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:47.117252+0000 mon.a (mon.0) 1526 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi047-82378-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi047-82378-29"}]': finished 2024-02-23T00:44:48.178 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:47.117386+0000 mon.a (mon.0) 1527 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:44:48.178 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:47.117437+0000 mon.a (mon.0) 1528 : cluster [DBG] osdmap e215: 8 total, 8 up, 8 in 2024-02-23T00:44:48.178 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:47.122332+0000 mon.a (mon.0) 1529 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-17"}]: dispatch 2024-02-23T00:44:48.178 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:47.122885+0000 mon.c (mon.2) 136 : audit [INF] from='client.? 172.21.15.47:0/1331945109' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi047-82386-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:48.178 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:47.123283+0000 mon.a (mon.0) 1530 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi047-82386-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:48.178 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:47.230084+0000 mgr.y (mgr.14517) 246 : cluster [DBG] pgmap v298: 290 pgs: 40 creating+peering, 11 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 224 active+clean; 10 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-23T00:44:48.178 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:47.598718+0000 mon.c (mon.2) 137 : audit [INF] from='client.? 172.21.15.47:0/1688144355' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:44:48.178 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:47.599048+0000 mon.a (mon.0) 1531 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:44:48.178 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:47.781234+0000 mon.b (mon.1) 275 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:48.184 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:44:48.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:47.015886+0000 mon.a (mon.0) 1525 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:44:48.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:47.117252+0000 mon.a (mon.0) 1526 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi047-82378-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi047-82378-29"}]': finished 2024-02-23T00:44:48.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:47.117386+0000 mon.a (mon.0) 1527 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:44:48.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:47.117437+0000 mon.a (mon.0) 1528 : cluster [DBG] osdmap e215: 8 total, 8 up, 8 in 2024-02-23T00:44:48.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:47.122332+0000 mon.a (mon.0) 1529 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-17"}]: dispatch 2024-02-23T00:44:48.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:47.122885+0000 mon.c (mon.2) 136 : audit [INF] from='client.? 172.21.15.47:0/1331945109' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi047-82386-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:48.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:47.123283+0000 mon.a (mon.0) 1530 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi047-82386-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:48.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:47.230084+0000 mgr.y (mgr.14517) 246 : cluster [DBG] pgmap v298: 290 pgs: 40 creating+peering, 11 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 224 active+clean; 10 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-23T00:44:48.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:47.598718+0000 mon.c (mon.2) 137 : audit [INF] from='client.? 172.21.15.47:0/1688144355' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:44:48.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:47.599048+0000 mon.a (mon.0) 1531 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:44:48.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:47.781234+0000 mon.b (mon.1) 275 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:48.210 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:48.228 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:44:48.230 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:48.231 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:44:48.331 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:44:48.362 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:48.493 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:44:49.306 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:48.087851+0000 mon.a (mon.0) 1532 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:44:49.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:48.114770+0000 mon.a (mon.0) 1533 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:44:49.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:48.114793+0000 mon.a (mon.0) 1534 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:44:49.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:48.122666+0000 mon.a (mon.0) 1535 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-17"}]': finished 2024-02-23T00:44:49.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:48.122766+0000 mon.a (mon.0) 1536 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi047-82386-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:49.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:48.122932+0000 mon.a (mon.0) 1537 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:44:49.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:48.122986+0000 mon.a (mon.0) 1538 : cluster [DBG] osdmap e216: 8 total, 8 up, 8 in 2024-02-23T00:44:49.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:48.342095+0000 mon.a (mon.0) 1539 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:44:49.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:48.782646+0000 mon.b (mon.1) 276 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:49.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:48.087851+0000 mon.a (mon.0) 1532 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:44:49.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:48.114770+0000 mon.a (mon.0) 1533 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:44:49.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:48.114793+0000 mon.a (mon.0) 1534 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:44:49.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:48.122666+0000 mon.a (mon.0) 1535 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-17"}]': finished 2024-02-23T00:44:49.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:48.122766+0000 mon.a (mon.0) 1536 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi047-82386-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:49.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:48.122932+0000 mon.a (mon.0) 1537 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:44:49.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:48.122986+0000 mon.a (mon.0) 1538 : cluster [DBG] osdmap e216: 8 total, 8 up, 8 in 2024-02-23T00:44:49.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:48.342095+0000 mon.a (mon.0) 1539 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:44:49.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:48.782646+0000 mon.b (mon.1) 276 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:49.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:48.087851+0000 mon.a (mon.0) 1532 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:44:49.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:48.114770+0000 mon.a (mon.0) 1533 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:44:49.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:48.114793+0000 mon.a (mon.0) 1534 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:44:49.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:48.122666+0000 mon.a (mon.0) 1535 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-17"}]': finished 2024-02-23T00:44:49.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:48.122766+0000 mon.a (mon.0) 1536 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi047-82386-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:49.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:48.122932+0000 mon.a (mon.0) 1537 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:44:49.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:48.122986+0000 mon.a (mon.0) 1538 : cluster [DBG] osdmap e216: 8 total, 8 up, 8 in 2024-02-23T00:44:49.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:48.342095+0000 mon.a (mon.0) 1539 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:44:49.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:48.782646+0000 mon.b (mon.1) 276 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:49.658 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:44:49.912 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:44:49.915 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:49.916 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:44:50.017 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:44:50.044 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:50.168 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:49.124940+0000 mon.a (mon.0) 1540 : cluster [DBG] osdmap e217: 8 total, 8 up, 8 in 2024-02-23T00:44:50.168 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:49.140872+0000 mon.c (mon.2) 138 : audit [INF] from='client.? 172.21.15.47:0/2198787178' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi047-82378-29"}]: dispatch 2024-02-23T00:44:50.168 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:49.230525+0000 mgr.y (mgr.14517) 247 : cluster [DBG] pgmap v301: 218 pgs: 11 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 192 active+clean; 10 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-23T00:44:50.168 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:49.577896+0000 mon.a (mon.0) 1541 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi047-82378-29"}]: dispatch 2024-02-23T00:44:50.168 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:49.783918+0000 mon.b (mon.1) 277 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:50.168 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:44:50.169 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:49.124940+0000 mon.a (mon.0) 1540 : cluster [DBG] osdmap e217: 8 total, 8 up, 8 in 2024-02-23T00:44:50.169 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:49.140872+0000 mon.c (mon.2) 138 : audit [INF] from='client.? 172.21.15.47:0/2198787178' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi047-82378-29"}]: dispatch 2024-02-23T00:44:50.169 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:49.230525+0000 mgr.y (mgr.14517) 247 : cluster [DBG] pgmap v301: 218 pgs: 11 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 192 active+clean; 10 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-23T00:44:50.169 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:49.577896+0000 mon.a (mon.0) 1541 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi047-82378-29"}]: dispatch 2024-02-23T00:44:50.169 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:49.783918+0000 mon.b (mon.1) 277 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:50.170 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:50.171 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:44:50.272 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:44:50.305 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:50.327 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:44:50.330 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:50.330 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:44:50.431 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:44:50.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:49.124940+0000 mon.a (mon.0) 1540 : cluster [DBG] osdmap e217: 8 total, 8 up, 8 in 2024-02-23T00:44:50.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:49.140872+0000 mon.c (mon.2) 138 : audit [INF] from='client.? 172.21.15.47:0/2198787178' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi047-82378-29"}]: dispatch 2024-02-23T00:44:50.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:49.230525+0000 mgr.y (mgr.14517) 247 : cluster [DBG] pgmap v301: 218 pgs: 11 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 192 active+clean; 10 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-23T00:44:50.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:49.577896+0000 mon.a (mon.0) 1541 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi047-82378-29"}]: dispatch 2024-02-23T00:44:50.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:49.783918+0000 mon.b (mon.1) 277 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:50.461 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:50.482 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:44:50.484 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:50.485 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:44:50.586 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:44:50.615 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:50.806 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:44:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:44:50] "GET /metrics HTTP/1.1" 200 191526 "" "Prometheus/2.33.4" 2024-02-23T00:44:51.164 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:44:51.182 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:51.183 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:44:51.284 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:44:51.312 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:51.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:50.131425+0000 mon.a (mon.0) 1542 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi047-82378-29"}]': finished 2024-02-23T00:44:51.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:50.131490+0000 mon.a (mon.0) 1543 : cluster [DBG] osdmap e218: 8 total, 8 up, 8 in 2024-02-23T00:44:51.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:50.132473+0000 mon.c (mon.2) 139 : audit [INF] from='client.? 172.21.15.47:0/2198787178' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi047-82378-29"}]: dispatch 2024-02-23T00:44:51.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:50.132881+0000 mon.a (mon.0) 1544 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:51.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:50.132994+0000 mon.c (mon.2) 140 : audit [INF] from='client.? 172.21.15.47:0/3868307806' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi047-82386-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:51.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:50.133185+0000 mon.a (mon.0) 1545 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi047-82378-29"}]: dispatch 2024-02-23T00:44:51.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:50.136662+0000 mon.a (mon.0) 1546 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi047-82386-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:51.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:50.351494+0000 mon.a (mon.0) 1547 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:44:51.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:50.352198+0000 mon.a (mon.0) 1548 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:44:51.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:50.356866+0000 mon.a (mon.0) 1549 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:44:51.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:50.785128+0000 mon.b (mon.1) 278 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:51.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:50.131425+0000 mon.a (mon.0) 1542 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi047-82378-29"}]': finished 2024-02-23T00:44:51.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:50.131490+0000 mon.a (mon.0) 1543 : cluster [DBG] osdmap e218: 8 total, 8 up, 8 in 2024-02-23T00:44:51.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:50.132473+0000 mon.c (mon.2) 139 : audit [INF] from='client.? 172.21.15.47:0/2198787178' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi047-82378-29"}]: dispatch 2024-02-23T00:44:51.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:50.132881+0000 mon.a (mon.0) 1544 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:51.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:50.132994+0000 mon.c (mon.2) 140 : audit [INF] from='client.? 172.21.15.47:0/3868307806' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi047-82386-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:51.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:50.133185+0000 mon.a (mon.0) 1545 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi047-82378-29"}]: dispatch 2024-02-23T00:44:51.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:50.136662+0000 mon.a (mon.0) 1546 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi047-82386-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:51.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:50.351494+0000 mon.a (mon.0) 1547 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:44:51.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:50.352198+0000 mon.a (mon.0) 1548 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:44:51.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:50.356866+0000 mon.a (mon.0) 1549 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:44:51.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:50.785128+0000 mon.b (mon.1) 278 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:51.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:50.131425+0000 mon.a (mon.0) 1542 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi047-82378-29"}]': finished 2024-02-23T00:44:51.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:50.131490+0000 mon.a (mon.0) 1543 : cluster [DBG] osdmap e218: 8 total, 8 up, 8 in 2024-02-23T00:44:51.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:50.132473+0000 mon.c (mon.2) 139 : audit [INF] from='client.? 172.21.15.47:0/2198787178' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi047-82378-29"}]: dispatch 2024-02-23T00:44:51.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:50.132881+0000 mon.a (mon.0) 1544 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:51.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:50.132994+0000 mon.c (mon.2) 140 : audit [INF] from='client.? 172.21.15.47:0/3868307806' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi047-82386-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:51.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:50.133185+0000 mon.a (mon.0) 1545 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi047-82378-29"}]: dispatch 2024-02-23T00:44:51.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:50.136662+0000 mon.a (mon.0) 1546 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi047-82386-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:51.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:50.351494+0000 mon.a (mon.0) 1547 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:44:51.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:50.352198+0000 mon.a (mon.0) 1548 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:44:51.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:50.356866+0000 mon.a (mon.0) 1549 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:44:51.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:50.785128+0000 mon.b (mon.1) 278 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:52.161 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:44:52.182 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:52.182 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:44:52.283 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:44:52.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:51.133030+0000 mon.a (mon.0) 1550 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:52.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:51.133106+0000 mon.a (mon.0) 1551 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi047-82378-29"}]': finished 2024-02-23T00:44:52.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:51.133231+0000 mon.a (mon.0) 1552 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi047-82386-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:52.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:51.133364+0000 mon.a (mon.0) 1553 : cluster [DBG] osdmap e219: 8 total, 8 up, 8 in 2024-02-23T00:44:52.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:51.151505+0000 mon.a (mon.0) 1554 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:44:52.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:51.187326+0000 mon.c (mon.2) 141 : audit [INF] from='client.? 172.21.15.47:0/2606886447' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi047-82378-30"}]: dispatch 2024-02-23T00:44:52.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:51.187645+0000 mon.a (mon.0) 1555 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi047-82378-30"}]: dispatch 2024-02-23T00:44:52.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:51.188156+0000 mon.c (mon.2) 142 : audit [INF] from='client.? 172.21.15.47:0/2606886447' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi047-82378-30"}]: dispatch 2024-02-23T00:44:52.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:51.188383+0000 mon.a (mon.0) 1556 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi047-82378-30"}]: dispatch 2024-02-23T00:44:52.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:51.188808+0000 mon.c (mon.2) 143 : audit [INF] from='client.? 172.21.15.47:0/2606886447' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi047-82378-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:52.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:51.189019+0000 mon.a (mon.0) 1557 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi047-82378-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:52.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:51.230929+0000 mgr.y (mgr.14517) 248 : cluster [DBG] pgmap v304: 282 pgs: 64 unknown, 11 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 192 active+clean; 10 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:44:52.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:51.133030+0000 mon.a (mon.0) 1550 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:52.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:51.133106+0000 mon.a (mon.0) 1551 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi047-82378-29"}]': finished 2024-02-23T00:44:52.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:51.133231+0000 mon.a (mon.0) 1552 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi047-82386-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:52.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:51.133364+0000 mon.a (mon.0) 1553 : cluster [DBG] osdmap e219: 8 total, 8 up, 8 in 2024-02-23T00:44:52.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:51.151505+0000 mon.a (mon.0) 1554 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:44:52.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:51.187326+0000 mon.c (mon.2) 141 : audit [INF] from='client.? 172.21.15.47:0/2606886447' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi047-82378-30"}]: dispatch 2024-02-23T00:44:52.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:51.187645+0000 mon.a (mon.0) 1555 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi047-82378-30"}]: dispatch 2024-02-23T00:44:52.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:51.188156+0000 mon.c (mon.2) 142 : audit [INF] from='client.? 172.21.15.47:0/2606886447' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi047-82378-30"}]: dispatch 2024-02-23T00:44:52.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:51.188383+0000 mon.a (mon.0) 1556 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi047-82378-30"}]: dispatch 2024-02-23T00:44:52.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:51.188808+0000 mon.c (mon.2) 143 : audit [INF] from='client.? 172.21.15.47:0/2606886447' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi047-82378-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:52.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:51.189019+0000 mon.a (mon.0) 1557 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi047-82378-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:52.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:51.230929+0000 mgr.y (mgr.14517) 248 : cluster [DBG] pgmap v304: 282 pgs: 64 unknown, 11 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 192 active+clean; 10 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:44:52.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:51.786190+0000 mon.b (mon.1) 279 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:52.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:52.134270+0000 mon.a (mon.0) 1558 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:44:52.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:52.134374+0000 mon.a (mon.0) 1559 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi047-82378-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:44:52.310 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:52.134448+0000 mon.a (mon.0) 1560 : cluster [DBG] osdmap e220: 8 total, 8 up, 8 in 2024-02-23T00:44:52.310 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:52.139590+0000 mon.c (mon.2) 144 : audit [INF] from='client.? 172.21.15.47:0/2606886447' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi047-82378-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi047-82378-30"}]: dispatch 2024-02-23T00:44:52.310 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:52.139998+0000 mon.a (mon.0) 1561 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi047-82378-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi047-82378-30"}]: dispatch 2024-02-23T00:44:52.310 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:52.141247+0000 mon.a (mon.0) 1562 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-19"}]: dispatch 2024-02-23T00:44:52.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:51.786190+0000 mon.b (mon.1) 279 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:52.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:52.134270+0000 mon.a (mon.0) 1558 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:44:52.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:52.134374+0000 mon.a (mon.0) 1559 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi047-82378-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:44:52.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:52.134448+0000 mon.a (mon.0) 1560 : cluster [DBG] osdmap e220: 8 total, 8 up, 8 in 2024-02-23T00:44:52.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:52.139590+0000 mon.c (mon.2) 144 : audit [INF] from='client.? 172.21.15.47:0/2606886447' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi047-82378-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi047-82378-30"}]: dispatch 2024-02-23T00:44:52.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:52.139998+0000 mon.a (mon.0) 1561 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi047-82378-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi047-82378-30"}]: dispatch 2024-02-23T00:44:52.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:52.141247+0000 mon.a (mon.0) 1562 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-19"}]: dispatch 2024-02-23T00:44:52.313 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:52.409 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:51.133030+0000 mon.a (mon.0) 1550 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:52.410 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:51.133106+0000 mon.a (mon.0) 1551 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi047-82378-29"}]': finished 2024-02-23T00:44:52.410 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:51.133231+0000 mon.a (mon.0) 1552 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi047-82386-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:52.410 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:51.133364+0000 mon.a (mon.0) 1553 : cluster [DBG] osdmap e219: 8 total, 8 up, 8 in 2024-02-23T00:44:52.410 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:51.151505+0000 mon.a (mon.0) 1554 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:44:52.410 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:51.187326+0000 mon.c (mon.2) 141 : audit [INF] from='client.? 172.21.15.47:0/2606886447' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi047-82378-30"}]: dispatch 2024-02-23T00:44:52.410 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:51.187645+0000 mon.a (mon.0) 1555 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi047-82378-30"}]: dispatch 2024-02-23T00:44:52.410 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:51.188156+0000 mon.c (mon.2) 142 : audit [INF] from='client.? 172.21.15.47:0/2606886447' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi047-82378-30"}]: dispatch 2024-02-23T00:44:52.410 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:51.188383+0000 mon.a (mon.0) 1556 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi047-82378-30"}]: dispatch 2024-02-23T00:44:52.410 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:51.188808+0000 mon.c (mon.2) 143 : audit [INF] from='client.? 172.21.15.47:0/2606886447' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi047-82378-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:52.410 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:51.189019+0000 mon.a (mon.0) 1557 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi047-82378-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:52.411 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:51.230929+0000 mgr.y (mgr.14517) 248 : cluster [DBG] pgmap v304: 282 pgs: 64 unknown, 11 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 192 active+clean; 10 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:44:52.411 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:51.786190+0000 mon.b (mon.1) 279 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:52.411 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:52.134270+0000 mon.a (mon.0) 1558 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:44:52.411 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:52.134374+0000 mon.a (mon.0) 1559 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi047-82378-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:44:52.411 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:52.134448+0000 mon.a (mon.0) 1560 : cluster [DBG] osdmap e220: 8 total, 8 up, 8 in 2024-02-23T00:44:52.411 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:52.139590+0000 mon.c (mon.2) 144 : audit [INF] from='client.? 172.21.15.47:0/2606886447' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi047-82378-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi047-82378-30"}]: dispatch 2024-02-23T00:44:52.411 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:52.139998+0000 mon.a (mon.0) 1561 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi047-82378-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi047-82378-30"}]: dispatch 2024-02-23T00:44:52.411 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:52.141247+0000 mon.a (mon.0) 1562 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-19"}]: dispatch 2024-02-23T00:44:52.564 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:44:52.567 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:52.567 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:44:52.668 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:44:52.695 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:44:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:44:52.697 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:44:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:44:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:44:52.706 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:52.728 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:44:52.730 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:52.731 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:44:52.831 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:44:52.860 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:52.879 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:44:52.882 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:52.882 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:44:52.983 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:44:53.011 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:53.030 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:44:53.033 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:53.033 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:44:53.134 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:44:53.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.014+0000 7f8589571700 1 -- 172.21.15.47:0/2132700516 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f85840f9ab0 msgr2=0x7f85840f9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:53.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.014+0000 7f8589571700 1 --2- 172.21.15.47:0/2132700516 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f85840f9ab0 0x7f85840f9e90 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 rx=0x7f85740047b0 tx=0x7f8574009cb0).stop 2024-02-23T00:44:53.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.015+0000 7f8589571700 1 -- 172.21.15.47:0/2132700516 shutdown_connections 2024-02-23T00:44:53.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.015+0000 7f8589571700 1 --2- 172.21.15.47:0/2132700516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85840f8440 0x7f8584104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:53.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.015+0000 7f8589571700 1 --2- 172.21.15.47:0/2132700516 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f85840f9ab0 0x7f85840f9e90 unknown :-1 s=CLOSED pgs=2545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:53.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.015+0000 7f8589571700 1 --2- 172.21.15.47:0/2132700516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85840fa380 0x7f85840f7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:53.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.015+0000 7f8589571700 1 -- 172.21.15.47:0/2132700516 >> 172.21.15.47:0/2132700516 conn(0x7f85840f3800 msgr2=0x7f85840f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:53.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.015+0000 7f8589571700 1 -- 172.21.15.47:0/2132700516 shutdown_connections 2024-02-23T00:44:53.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.015+0000 7f8589571700 1 -- 172.21.15.47:0/2132700516 wait complete. 2024-02-23T00:44:53.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.015+0000 7f8589571700 1 librados: starting msgr at 2024-02-23T00:44:53.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.015+0000 7f8589571700 1 librados: starting objecter 2024-02-23T00:44:53.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.015+0000 7f8589571700 1 Processor -- start 2024-02-23T00:44:53.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.016+0000 7f8589571700 1 -- start start 2024-02-23T00:44:53.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.016+0000 7f8589571700 1 librados: setting wanted keys 2024-02-23T00:44:53.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.016+0000 7f8589571700 1 librados: calling monclient init 2024-02-23T00:44:53.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.016+0000 7f8589571700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85840f8440 0x7f858410e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:53.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.016+0000 7f8589571700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f85840f9ab0 0x7f858410e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.016+0000 7f8589571700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85840fa380 0x7f85841081a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.016+0000 7f8589571700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8584108690 con 0x7f85840f8440 2024-02-23T00:44:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.016+0000 7f8589571700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8584108800 con 0x7f85840fa380 2024-02-23T00:44:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.016+0000 7f8589571700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85841089c0 con 0x7f85840f9ab0 2024-02-23T00:44:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.016+0000 7f8582ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85840f8440 0x7f858410e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.016+0000 7f85827fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f85840f9ab0 0x7f858410e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.016+0000 7f8582ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85840f8440 0x7f858410e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39504/0 (socket says 172.21.15.47:39504) 2024-02-23T00:44:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.016+0000 7f8582ffd700 1 -- 172.21.15.47:0/4255540357 learned_addr learned my addr 172.21.15.47:0/4255540357 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:44:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.016+0000 7f85837fe700 1 --2- 172.21.15.47:0/4255540357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85840fa380 0x7f85841081a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.016+0000 7f8582ffd700 1 -- 172.21.15.47:0/4255540357 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f85840f9ab0 msgr2=0x7f858410e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.016+0000 7f8582ffd700 1 --2- 172.21.15.47:0/4255540357 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f85840f9ab0 0x7f858410e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.016+0000 7f8582ffd700 1 -- 172.21.15.47:0/4255540357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85840fa380 msgr2=0x7f85841081a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.016+0000 7f8582ffd700 1 --2- 172.21.15.47:0/4255540357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85840fa380 0x7f85841081a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.016+0000 7f8582ffd700 1 -- 172.21.15.47:0/4255540357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f857c0097c0 con 0x7f85840f8440 2024-02-23T00:44:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.016+0000 7f85827fc700 1 --2- 172.21.15.47:0/4255540357 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f85840f9ab0 0x7f858410e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:44:53.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.017+0000 7f8582ffd700 1 --2- 172.21.15.47:0/4255540357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85840f8440 0x7f858410e000 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 rx=0x7f85740044e0 tx=0x7f8574009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:53.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.017+0000 7f856bfff700 10 client.?.objecter ms_handle_connect 0x7f85840f8440 2024-02-23T00:44:53.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.017+0000 7f856bfff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:44:53.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.017+0000 7f856bfff700 1 -- 172.21.15.47:0/4255540357 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8574018070 con 0x7f85840f8440 2024-02-23T00:44:53.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.017+0000 7f8589571700 1 -- 172.21.15.47:0/4255540357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8574005040 con 0x7f85840f8440 2024-02-23T00:44:53.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.017+0000 7f856bfff700 1 -- 172.21.15.47:0/4255540357 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f8574013e50 con 0x7f85840f8440 2024-02-23T00:44:53.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.017+0000 7f856bfff700 1 -- 172.21.15.47:0/4255540357 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857401db60 con 0x7f85840f8440 2024-02-23T00:44:53.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.017+0000 7f8589571700 10 client.30417.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:44:53.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.017+0000 7f8589571700 1 -- 172.21.15.47:0/4255540357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8584108e60 con 0x7f85840f8440 2024-02-23T00:44:53.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.017+0000 7f8589571700 1 librados: init done 2024-02-23T00:44:53.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.018+0000 7f8589571700 1 -- 172.21.15.47:0/4255540357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8584109050 con 0x7f85840f8440 2024-02-23T00:44:53.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.018+0000 7f856bfff700 1 -- 172.21.15.47:0/4255540357 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f8574009240 con 0x7f85840f8440 2024-02-23T00:44:53.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.018+0000 7f856bfff700 1 --2- 172.21.15.47:0/4255540357 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f856c062cc0 0x7f856c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:53.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.019+0000 7f856bfff700 1 -- 172.21.15.47:0/4255540357 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(220..220 src has 48..220) v4 ==== 9787+0+0 (secure 0 0 0) 0x7f857407ece0 con 0x7f85840f8440 2024-02-23T00:44:53.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.019+0000 7f856bfff700 10 client.30417.objecter ms_dispatch 0x7f8584001550 osd_map(220..220 src has 48..220) v4 2024-02-23T00:44:53.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.019+0000 7f856bfff700 3 client.30417.objecter handle_osd_map got epochs [220,220] > 0 2024-02-23T00:44:53.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.019+0000 7f856bfff700 3 client.30417.objecter handle_osd_map decoding full epoch 220 2024-02-23T00:44:53.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.019+0000 7f85827fc700 1 --2- 172.21.15.47:0/4255540357 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f856c062cc0 0x7f856c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:53.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.019+0000 7f856bfff700 20 client.30417.objecter dump_active .. 0 homeless 2024-02-23T00:44:53.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.019+0000 7f856bfff700 10 client.30417.objecter ms_handle_connect 0x7f856c062cc0 2024-02-23T00:44:53.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.019+0000 7f85827fc700 1 --2- 172.21.15.47:0/4255540357 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f856c062cc0 0x7f856c065180 secure :-1 s=READY pgs=2964 cs=0 l=1 rev1=1 rx=0x7f8584109940 tx=0x7f8578011040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:53.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.023+0000 7f856bfff700 1 -- 172.21.15.47:0/4255540357 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f857404e0c0 con 0x7f85840f8440 2024-02-23T00:44:53.180 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:53.186 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset noscrub 2024-02-23T00:44:53.203 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:44:53.207 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:53.208 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:44:53.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:53.239+0000 7f8589571700 1 -- 172.21.15.47:0/4255540357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd reweight", "id": 7, "weight": 1.0} v 0) v1 -- 0x7f8584106fc0 con 0x7f85840f8440 2024-02-23T00:44:53.303 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:44:53.303 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-02-23T00:44:53.304 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-02-23T00:44:53.304 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-02-23T00:44:53.304 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:44:53.309 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:44:53.337 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:53.357 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:44:53.359 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:53.359 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:44:53.460 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:44:53.487 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:53.508 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:44:53.509 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:53.510 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:44:53.513 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:44:53.610 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:44:53.638 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:53.837 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:44:53.839 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:53.840 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:44:53.849 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:52.407645+0000 mgr.y (mgr.14517) 249 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:44:53.849 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:52.787154+0000 mon.b (mon.1) 280 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:53.849 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:52.407645+0000 mgr.y (mgr.14517) 249 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:44:53.849 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:52.787154+0000 mon.b (mon.1) 280 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:53.941 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:44:53.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:52.407645+0000 mgr.y (mgr.14517) 249 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:44:53.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:52.787154+0000 mon.b (mon.1) 280 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:53.969 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:54.143+0000 7f856bfff700 1 -- 172.21.15.47:0/4255540357 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 7, "weight": 1.0}]=0 reweighted osd.7 to 1 (10000) v222) v1 ==== 113+0+0 (secure 0 0 0) 0x7f85740516e0 con 0x7f85840f8440 2024-02-23T00:44:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:54.146+0000 7f8589571700 10 librados: watch_flush enter 2024-02-23T00:44:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:54.148+0000 7f8589571700 10 librados: watch_flush exit 2024-02-23T00:44:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:54.148+0000 7f8589571700 20 client.30417.objecter shutdown clearing up homeless session... 2024-02-23T00:44:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:54.148+0000 7f8589571700 10 client.30417.objecter successfully canceled tick 2024-02-23T00:44:54.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:54.148+0000 7f8589571700 1 -- 172.21.15.47:0/4255540357 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f856c062cc0 msgr2=0x7f856c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:54.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:54.149+0000 7f8589571700 1 --2- 172.21.15.47:0/4255540357 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f856c062cc0 0x7f856c065180 secure :-1 s=READY pgs=2964 cs=0 l=1 rev1=1 rx=0x7f8584109940 tx=0x7f8578011040).stop 2024-02-23T00:44:54.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:54.149+0000 7f8589571700 1 -- 172.21.15.47:0/4255540357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85840f8440 msgr2=0x7f858410e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:54.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:54.149+0000 7f8589571700 1 --2- 172.21.15.47:0/4255540357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85840f8440 0x7f858410e000 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 rx=0x7f85740044e0 tx=0x7f8574009cb0).stop 2024-02-23T00:44:54.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:54.149+0000 7f8589571700 1 -- 172.21.15.47:0/4255540357 shutdown_connections 2024-02-23T00:44:54.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:54.149+0000 7f8589571700 1 --2- 172.21.15.47:0/4255540357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85840fa380 0x7f85841081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:54.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:54.149+0000 7f8589571700 1 --2- 172.21.15.47:0/4255540357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85840f8440 0x7f858410e000 unknown :-1 s=CLOSED pgs=2460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:54.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:54.149+0000 7f8589571700 1 --2- 172.21.15.47:0/4255540357 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f856c062cc0 0x7f856c065180 secure :-1 s=CLOSED pgs=2964 cs=0 l=1 rev1=1 rx=0x7f8584109940 tx=0x7f8578011040).stop 2024-02-23T00:44:54.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:54.149+0000 7f8589571700 1 --2- 172.21.15.47:0/4255540357 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f85840f9ab0 0x7f858410e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:54.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:54.149+0000 7f8589571700 1 -- 172.21.15.47:0/4255540357 >> 172.21.15.47:0/4255540357 conn(0x7f85840f3800 msgr2=0x7f85840fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:54.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:54.151+0000 7f8589571700 1 -- 172.21.15.47:0/4255540357 shutdown_connections 2024-02-23T00:44:54.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:54.153+0000 7f8589571700 1 -- 172.21.15.47:0/4255540357 wait complete. 2024-02-23T00:44:54.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:54.155+0000 7f8589571700 1 librados: shutdown 2024-02-23T00:44:54.165 INFO:teuthology.orchestra.run.smithi047.stderr:reweighted osd.7 to 1 (10000) 2024-02-23T00:44:54.570 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.0 dump_historic_ops 2024-02-23T00:44:54.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:53.231453+0000 mgr.y (mgr.14517) 250 : cluster [DBG] pgmap v307: 282 pgs: 32 unknown, 8 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 230 active+clean; 10 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:44:54.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:53.587871+0000 mon.a (mon.0) 1563 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-19"}]': finished 2024-02-23T00:44:54.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:53.587987+0000 mon.a (mon.0) 1564 : cluster [DBG] osdmap e221: 8 total, 8 up, 8 in 2024-02-23T00:44:54.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:53.589112+0000 mon.a (mon.0) 1565 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-19", "mode": "writeback"}]: dispatch 2024-02-23T00:44:54.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:53.589370+0000 mon.a (mon.0) 1566 : audit [INF] from='client.? 172.21.15.47:0/4255540357' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-23T00:44:54.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:53.589709+0000 mon.c (mon.2) 145 : audit [INF] from='client.? 172.21.15.47:0/2235208660' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi047-82386-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:54.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:53.595488+0000 mon.a (mon.0) 1567 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi047-82386-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:54.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:53.788098+0000 mon.b (mon.1) 281 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:54.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:54.134399+0000 mon.a (mon.0) 1568 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:44:54.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:54.141015+0000 mon.a (mon.0) 1569 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi047-82378-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi047-82378-30"}]': finished 2024-02-23T00:44:54.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:54.141115+0000 mon.a (mon.0) 1570 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-19", "mode": "writeback"}]': finished 2024-02-23T00:44:54.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:54.141178+0000 mon.a (mon.0) 1571 : audit [INF] from='client.? 172.21.15.47:0/4255540357' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-02-23T00:44:54.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:54.141357+0000 mon.a (mon.0) 1572 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi047-82386-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:54.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:54.141429+0000 mon.a (mon.0) 1573 : cluster [DBG] osdmap e222: 8 total, 8 up, 8 in 2024-02-23T00:44:54.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:54.188041+0000 mon.a (mon.0) 1574 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:44:54.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:53.231453+0000 mgr.y (mgr.14517) 250 : cluster [DBG] pgmap v307: 282 pgs: 32 unknown, 8 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 230 active+clean; 10 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:44:54.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:53.587871+0000 mon.a (mon.0) 1563 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-19"}]': finished 2024-02-23T00:44:54.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:53.587987+0000 mon.a (mon.0) 1564 : cluster [DBG] osdmap e221: 8 total, 8 up, 8 in 2024-02-23T00:44:54.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:53.589112+0000 mon.a (mon.0) 1565 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-19", "mode": "writeback"}]: dispatch 2024-02-23T00:44:54.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:53.589370+0000 mon.a (mon.0) 1566 : audit [INF] from='client.? 172.21.15.47:0/4255540357' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-23T00:44:54.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:53.589709+0000 mon.c (mon.2) 145 : audit [INF] from='client.? 172.21.15.47:0/2235208660' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi047-82386-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:54.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:53.595488+0000 mon.a (mon.0) 1567 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi047-82386-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:54.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:53.788098+0000 mon.b (mon.1) 281 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:54.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:54.134399+0000 mon.a (mon.0) 1568 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:44:54.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:54.141015+0000 mon.a (mon.0) 1569 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi047-82378-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi047-82378-30"}]': finished 2024-02-23T00:44:54.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:54.141115+0000 mon.a (mon.0) 1570 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-19", "mode": "writeback"}]': finished 2024-02-23T00:44:54.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:54.141178+0000 mon.a (mon.0) 1571 : audit [INF] from='client.? 172.21.15.47:0/4255540357' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-02-23T00:44:54.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:54.141357+0000 mon.a (mon.0) 1572 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi047-82386-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:54.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:54.141429+0000 mon.a (mon.0) 1573 : cluster [DBG] osdmap e222: 8 total, 8 up, 8 in 2024-02-23T00:44:54.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:54.188041+0000 mon.a (mon.0) 1574 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:44:54.820 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:44:54.822 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:54.823 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:44:54.923 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:44:54.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:53.231453+0000 mgr.y (mgr.14517) 250 : cluster [DBG] pgmap v307: 282 pgs: 32 unknown, 8 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 230 active+clean; 10 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:44:54.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:53.587871+0000 mon.a (mon.0) 1563 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-19"}]': finished 2024-02-23T00:44:54.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:53.587987+0000 mon.a (mon.0) 1564 : cluster [DBG] osdmap e221: 8 total, 8 up, 8 in 2024-02-23T00:44:54.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:53.589112+0000 mon.a (mon.0) 1565 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-19", "mode": "writeback"}]: dispatch 2024-02-23T00:44:54.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:53.589370+0000 mon.a (mon.0) 1566 : audit [INF] from='client.? 172.21.15.47:0/4255540357' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-23T00:44:54.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:53.589709+0000 mon.c (mon.2) 145 : audit [INF] from='client.? 172.21.15.47:0/2235208660' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi047-82386-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:54.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:53.595488+0000 mon.a (mon.0) 1567 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi047-82386-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:54.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:53.788098+0000 mon.b (mon.1) 281 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:54.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:54.134399+0000 mon.a (mon.0) 1568 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:44:54.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:54.141015+0000 mon.a (mon.0) 1569 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi047-82378-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi047-82378-30"}]': finished 2024-02-23T00:44:54.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:54.141115+0000 mon.a (mon.0) 1570 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-19", "mode": "writeback"}]': finished 2024-02-23T00:44:54.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:54.141178+0000 mon.a (mon.0) 1571 : audit [INF] from='client.? 172.21.15.47:0/4255540357' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-02-23T00:44:54.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:54.141357+0000 mon.a (mon.0) 1572 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi047-82386-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:54.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:54.141429+0000 mon.a (mon.0) 1573 : cluster [DBG] osdmap e222: 8 total, 8 up, 8 in 2024-02-23T00:44:54.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:54.188041+0000 mon.a (mon.0) 1574 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:44:54.950 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:55.599 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:44:55.602 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:55.603 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:44:55.703 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:44:55.732 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:55.753 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:44:55.757 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:55.757 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:44:55.858 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:44:55.859 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:54.789030+0000 mon.b (mon.1) 282 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:55.859 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:55.116374+0000 mon.a (mon.0) 1575 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:44:55.860 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:55.116432+0000 mon.a (mon.0) 1576 : cluster [DBG] osdmap e223: 8 total, 8 up, 8 in 2024-02-23T00:44:55.860 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:55.122273+0000 mon.a (mon.0) 1577 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-19"}]: dispatch 2024-02-23T00:44:55.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:54.789030+0000 mon.b (mon.1) 282 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:55.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:55.116374+0000 mon.a (mon.0) 1575 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:44:55.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:55.116432+0000 mon.a (mon.0) 1576 : cluster [DBG] osdmap e223: 8 total, 8 up, 8 in 2024-02-23T00:44:55.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:55.122273+0000 mon.a (mon.0) 1577 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-19"}]: dispatch 2024-02-23T00:44:55.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:54.789030+0000 mon.b (mon.1) 282 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:55.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:55.116374+0000 mon.a (mon.0) 1575 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:44:55.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:55.116432+0000 mon.a (mon.0) 1576 : cluster [DBG] osdmap e223: 8 total, 8 up, 8 in 2024-02-23T00:44:55.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:55.122273+0000 mon.a (mon.0) 1577 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-19"}]: dispatch 2024-02-23T00:44:55.891 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:55.911 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:44:55.914 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:55.915 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:44:56.016 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:44:56.045 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:56.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.131+0000 7fb51e29c700 1 -- 172.21.15.47:0/3039361074 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb5180f6b70 msgr2=0x7fb5180f6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:56.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.131+0000 7fb51e29c700 1 --2- 172.21.15.47:0/3039361074 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb5180f6b70 0x7fb5180f6f50 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 rx=0x7fb508004870 tx=0x7fb508009cb0).stop 2024-02-23T00:44:56.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.131+0000 7fb51e29c700 1 -- 172.21.15.47:0/3039361074 shutdown_connections 2024-02-23T00:44:56.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.131+0000 7fb51e29c700 1 --2- 172.21.15.47:0/3039361074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5180fc0e0 0x7fb5180fc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:56.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.131+0000 7fb51e29c700 1 --2- 172.21.15.47:0/3039361074 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb5180f6b70 0x7fb5180f6f50 unknown :-1 s=CLOSED pgs=2546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:56.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.131+0000 7fb51e29c700 1 --2- 172.21.15.47:0/3039361074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5180f74d0 0x7fb5180fba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:56.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.131+0000 7fb51e29c700 1 -- 172.21.15.47:0/3039361074 >> 172.21.15.47:0/3039361074 conn(0x7fb5180f26f0 msgr2=0x7fb5180f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:56.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.132+0000 7fb51e29c700 1 -- 172.21.15.47:0/3039361074 shutdown_connections 2024-02-23T00:44:56.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.132+0000 7fb51e29c700 1 -- 172.21.15.47:0/3039361074 wait complete. 2024-02-23T00:44:56.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.132+0000 7fb51e29c700 1 librados: starting msgr at 2024-02-23T00:44:56.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.132+0000 7fb51e29c700 1 librados: starting objecter 2024-02-23T00:44:56.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.133+0000 7fb51e29c700 1 Processor -- start 2024-02-23T00:44:56.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.133+0000 7fb51e29c700 1 -- start start 2024-02-23T00:44:56.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.133+0000 7fb51e29c700 1 librados: setting wanted keys 2024-02-23T00:44:56.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.133+0000 7fb51e29c700 1 librados: calling monclient init 2024-02-23T00:44:56.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.133+0000 7fb51e29c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5180f6b70 0x7fb5181146a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:56.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.133+0000 7fb51e29c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5180f74d0 0x7fb518114b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:56.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.133+0000 7fb51e29c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb5180fc0e0 0x7fb51810e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:56.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.133+0000 7fb51e29c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51810ed30 con 0x7fb5180f6b70 2024-02-23T00:44:56.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.133+0000 7fb51e29c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51810ee70 con 0x7fb5180f74d0 2024-02-23T00:44:56.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.133+0000 7fb51e29c700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51810f030 con 0x7fb5180fc0e0 2024-02-23T00:44:56.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.133+0000 7fb517fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5180f6b70 0x7fb5181146a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:56.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.133+0000 7fb517fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5180f6b70 0x7fb5181146a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39538/0 (socket says 172.21.15.47:39538) 2024-02-23T00:44:56.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.133+0000 7fb517fff700 1 -- 172.21.15.47:0/2165388759 learned_addr learned my addr 172.21.15.47:0/2165388759 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:44:56.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.134+0000 7fb5177fe700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5180f74d0 0x7fb518114b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:56.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.134+0000 7fb51c839700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb5180fc0e0 0x7fb51810e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:56.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.134+0000 7fb517fff700 1 -- 172.21.15.47:0/2165388759 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb5180fc0e0 msgr2=0x7fb51810e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:56.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.134+0000 7fb517fff700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb5180fc0e0 0x7fb51810e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:56.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.134+0000 7fb517fff700 1 -- 172.21.15.47:0/2165388759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5180f74d0 msgr2=0x7fb518114b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:56.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.134+0000 7fb517fff700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5180f74d0 0x7fb518114b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:56.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.134+0000 7fb517fff700 1 -- 172.21.15.47:0/2165388759 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb508005040 con 0x7fb5180f6b70 2024-02-23T00:44:56.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.134+0000 7fb517fff700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5180f6b70 0x7fb5181146a0 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 rx=0x7fb5080043d0 tx=0x7fb508009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:56.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.134+0000 7fb515775700 10 client.?.objecter ms_handle_connect 0x7fb5180f6b70 2024-02-23T00:44:56.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.134+0000 7fb515775700 10 client.?.objecter resend_mon_ops 2024-02-23T00:44:56.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.136+0000 7fb515775700 1 -- 172.21.15.47:0/2165388759 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50800e030 con 0x7fb5180f6b70 2024-02-23T00:44:56.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.136+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb51810f2b0 con 0x7fb5180f6b70 2024-02-23T00:44:56.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.137+0000 7fb51e29c700 10 client.30429.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:44:56.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.137+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb51811b4a0 con 0x7fb5180f6b70 2024-02-23T00:44:56.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.137+0000 7fb51e29c700 1 librados: init done 2024-02-23T00:44:56.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.137+0000 7fb515775700 1 -- 172.21.15.47:0/2165388759 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fb508013df0 con 0x7fb5180f6b70 2024-02-23T00:44:56.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.137+0000 7fb515775700 1 -- 172.21.15.47:0/2165388759 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb508018070 con 0x7fb5180f6b70 2024-02-23T00:44:56.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.138+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fb504004f30 con 0x7fb5180f6b70 2024-02-23T00:44:56.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.138+0000 7fb515775700 1 -- 172.21.15.47:0/2165388759 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fb508009240 con 0x7fb5180f6b70 2024-02-23T00:44:56.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.139+0000 7fb515775700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb500062cc0 0x7fb500065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:56.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.139+0000 7fb515775700 1 -- 172.21.15.47:0/2165388759 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(224..224 src has 48..224) v4 ==== 10265+0+0 (secure 0 0 0) 0x7fb50807f510 con 0x7fb5180f6b70 2024-02-23T00:44:56.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.139+0000 7fb515775700 10 client.30429.objecter ms_dispatch 0x7fb518001550 osd_map(224..224 src has 48..224) v4 2024-02-23T00:44:56.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.139+0000 7fb515775700 3 client.30429.objecter handle_osd_map got epochs [224,224] > 0 2024-02-23T00:44:56.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.139+0000 7fb515775700 3 client.30429.objecter handle_osd_map decoding full epoch 224 2024-02-23T00:44:56.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.139+0000 7fb5177fe700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb500062cc0 0x7fb500065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:56.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.139+0000 7fb515775700 20 client.30429.objecter dump_active .. 0 homeless 2024-02-23T00:44:56.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.139+0000 7fb515775700 1 -- 172.21.15.47:0/2165388759 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v224) v1 ==== 54+0+16 (secure 0 0 0) 0x7fb50807fa10 con 0x7fb5180f6b70 2024-02-23T00:44:56.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.139+0000 7fb515775700 10 client.30429.objecter ms_handle_connect 0x7fb500062cc0 2024-02-23T00:44:56.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.139+0000 7fb5177fe700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb500062cc0 0x7fb500065180 secure :-1 s=READY pgs=2966 cs=0 l=1 rev1=1 rx=0x7fb51810fa20 tx=0x7fb50c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:56.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.139+0000 7fb51e29c700 10 client.30429.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:56.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.139+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_assign -1 1 2024-02-23T00:44:56.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.139+0000 7fb51e29c700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fb504004560 0x7fb504004980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:56.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.139+0000 7fb51e29c700 20 client.30429.objecter _get_session s=0x7fb5040051d0 osd=0 3 2024-02-23T00:44:56.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.139+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb5040051d0 osd=0 3 2024-02-23T00:44:56.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.139+0000 7fb51e29c700 20 client.30429.objecter _get_session s=0x7fb5040051d0 osd=0 3 2024-02-23T00:44:56.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.139+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_remove -1 1 2024-02-23T00:44:56.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.139+0000 7fb51e29c700 20 client.30429.objecter get_session s=0x7fb5040051d0 osd=0 3 2024-02-23T00:44:56.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.139+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_assign 0 1 2024-02-23T00:44:56.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.139+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb5040051d0 osd=0 4 2024-02-23T00:44:56.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.139+0000 7fb51e29c700 10 client.30429.objecter _send_command 1 2024-02-23T00:44:56.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.139+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb504008960 con 0x7fb504004560 2024-02-23T00:44:56.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.139+0000 7fb51c839700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fb504004560 0x7fb504004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:56.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.140+0000 7fb515775700 10 client.30429.objecter ms_handle_connect 0x7fb504004560 2024-02-23T00:44:56.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.140+0000 7fb51c839700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fb504004560 0x7fb504004980 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:56.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.140+0000 7fb515775700 1 -- 172.21.15.47:0/2165388759 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb504008960 con 0x7fb504004560 2024-02-23T00:44:56.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.140+0000 7fb515775700 10 client.30429.objecter ms_dispatch 0x7fb518001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:44:56.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.140+0000 7fb515775700 10 client.30429.objecter _finish_command 1 = system:0 2024-02-23T00:44:56.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.140+0000 7fb515775700 20 client.30429.objecter put_session s=0x7fb5040051d0 osd=0 4 2024-02-23T00:44:56.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.140+0000 7fb515775700 15 client.30429.objecter _session_command_op_remove 0 1 2024-02-23T00:44:56.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.157+0000 7fb51e29c700 10 client.30429.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:44:56.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.157+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_assign -1 2 2024-02-23T00:44:56.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.157+0000 7fb51e29c700 20 client.30429.objecter _get_session s=0x7fb5040051d0 osd=0 3 2024-02-23T00:44:56.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.157+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb5040051d0 osd=0 3 2024-02-23T00:44:56.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.157+0000 7fb51e29c700 20 client.30429.objecter _get_session s=0x7fb5040051d0 osd=0 3 2024-02-23T00:44:56.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.157+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_remove -1 2 2024-02-23T00:44:56.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.157+0000 7fb51e29c700 20 client.30429.objecter get_session s=0x7fb5040051d0 osd=0 3 2024-02-23T00:44:56.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.157+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_assign 0 2 2024-02-23T00:44:56.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.157+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb5040051d0 osd=0 4 2024-02-23T00:44:56.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.157+0000 7fb51e29c700 10 client.30429.objecter _send_command 2 2024-02-23T00:44:56.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.157+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fb504003fd0 con 0x7fb504004560 2024-02-23T00:44:56.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.158+0000 7fb515775700 1 -- 172.21.15.47:0/2165388759 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fb504003fd0 con 0x7fb504004560 2024-02-23T00:44:56.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.158+0000 7fb515775700 10 client.30429.objecter ms_dispatch 0x7fb518001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:44:56.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.158+0000 7fb515775700 10 client.30429.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:44:56.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.158+0000 7fb515775700 20 client.30429.objecter put_session s=0x7fb5040051d0 osd=0 4 2024-02-23T00:44:56.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.158+0000 7fb515775700 15 client.30429.objecter _session_command_op_remove 0 2 2024-02-23T00:44:56.164 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:44:56.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.159+0000 7fb51e29c700 10 client.30429.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:56.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.159+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_assign -1 3 2024-02-23T00:44:56.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.159+0000 7fb51e29c700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fb504008510 0x7fb504002760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:56.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.159+0000 7fb51e29c700 20 client.30429.objecter _get_session s=0x7fb504004380 osd=1 3 2024-02-23T00:44:56.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.159+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb504004380 osd=1 3 2024-02-23T00:44:56.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.159+0000 7fb51e29c700 20 client.30429.objecter _get_session s=0x7fb504004380 osd=1 3 2024-02-23T00:44:56.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.159+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_remove -1 3 2024-02-23T00:44:56.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.159+0000 7fb51e29c700 20 client.30429.objecter get_session s=0x7fb504004380 osd=1 3 2024-02-23T00:44:56.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.159+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_assign 1 3 2024-02-23T00:44:56.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.159+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb504004380 osd=1 4 2024-02-23T00:44:56.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.159+0000 7fb51e29c700 10 client.30429.objecter _send_command 3 2024-02-23T00:44:56.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.159+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb504002e40 con 0x7fb504008510 2024-02-23T00:44:56.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.159+0000 7fb517fff700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fb504008510 0x7fb504002760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:56.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.159+0000 7fb515775700 10 client.30429.objecter ms_handle_connect 0x7fb504008510 2024-02-23T00:44:56.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.159+0000 7fb517fff700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fb504008510 0x7fb504002760 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:56.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.160+0000 7fb515775700 1 -- 172.21.15.47:0/2165388759 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb504002e40 con 0x7fb504008510 2024-02-23T00:44:56.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.160+0000 7fb515775700 10 client.30429.objecter ms_dispatch 0x7fb518001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:44:56.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.160+0000 7fb515775700 10 client.30429.objecter _finish_command 3 = system:0 2024-02-23T00:44:56.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.160+0000 7fb515775700 20 client.30429.objecter put_session s=0x7fb504004380 osd=1 4 2024-02-23T00:44:56.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.160+0000 7fb515775700 15 client.30429.objecter _session_command_op_remove 1 3 2024-02-23T00:44:56.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.175+0000 7fb51e29c700 10 client.30429.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:44:56.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.175+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_assign -1 4 2024-02-23T00:44:56.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.175+0000 7fb51e29c700 20 client.30429.objecter _get_session s=0x7fb504004380 osd=1 3 2024-02-23T00:44:56.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.175+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb504004380 osd=1 3 2024-02-23T00:44:56.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.175+0000 7fb51e29c700 20 client.30429.objecter _get_session s=0x7fb504004380 osd=1 3 2024-02-23T00:44:56.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.175+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_remove -1 4 2024-02-23T00:44:56.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.175+0000 7fb51e29c700 20 client.30429.objecter get_session s=0x7fb504004380 osd=1 3 2024-02-23T00:44:56.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.175+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_assign 1 4 2024-02-23T00:44:56.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.175+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb504004380 osd=1 4 2024-02-23T00:44:56.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.175+0000 7fb51e29c700 10 client.30429.objecter _send_command 4 2024-02-23T00:44:56.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.175+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fb504003810 con 0x7fb504008510 2024-02-23T00:44:56.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.176+0000 7fb515775700 1 -- 172.21.15.47:0/2165388759 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fb504003810 con 0x7fb504008510 2024-02-23T00:44:56.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.176+0000 7fb515775700 10 client.30429.objecter ms_dispatch 0x7fb518001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:44:56.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.176+0000 7fb515775700 10 client.30429.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:44:56.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.176+0000 7fb515775700 20 client.30429.objecter put_session s=0x7fb504004380 osd=1 4 2024-02-23T00:44:56.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.176+0000 7fb515775700 15 client.30429.objecter _session_command_op_remove 1 4 2024-02-23T00:44:56.179 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:44:56.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.176+0000 7fb51e29c700 10 client.30429.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:56.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.176+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_assign -1 5 2024-02-23T00:44:56.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.176+0000 7fb51e29c700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fb504007b90 0x7fb50400d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:56.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.176+0000 7fb51e29c700 20 client.30429.objecter _get_session s=0x7fb504007a20 osd=2 3 2024-02-23T00:44:56.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.176+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb504007a20 osd=2 3 2024-02-23T00:44:56.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.176+0000 7fb51e29c700 20 client.30429.objecter _get_session s=0x7fb504007a20 osd=2 3 2024-02-23T00:44:56.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.176+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_remove -1 5 2024-02-23T00:44:56.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.176+0000 7fb51e29c700 20 client.30429.objecter get_session s=0x7fb504007a20 osd=2 3 2024-02-23T00:44:56.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.176+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_assign 2 5 2024-02-23T00:44:56.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.176+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb504007a20 osd=2 4 2024-02-23T00:44:56.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.176+0000 7fb51e29c700 10 client.30429.objecter _send_command 5 2024-02-23T00:44:56.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.176+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb50400fe10 con 0x7fb504007b90 2024-02-23T00:44:56.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.176+0000 7fb5177fe700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fb504007b90 0x7fb50400d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:56.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.177+0000 7fb515775700 10 client.30429.objecter ms_handle_connect 0x7fb504007b90 2024-02-23T00:44:56.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.177+0000 7fb5177fe700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fb504007b90 0x7fb50400d000 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:56.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.177+0000 7fb515775700 1 -- 172.21.15.47:0/2165388759 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb50400fe10 con 0x7fb504007b90 2024-02-23T00:44:56.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.177+0000 7fb515775700 10 client.30429.objecter ms_dispatch 0x7fb518001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:44:56.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.177+0000 7fb515775700 10 client.30429.objecter _finish_command 5 = system:0 2024-02-23T00:44:56.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.177+0000 7fb515775700 20 client.30429.objecter put_session s=0x7fb504007a20 osd=2 4 2024-02-23T00:44:56.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.177+0000 7fb515775700 15 client.30429.objecter _session_command_op_remove 2 5 2024-02-23T00:44:56.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.192+0000 7fb51e29c700 10 client.30429.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:44:56.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.192+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_assign -1 6 2024-02-23T00:44:56.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.192+0000 7fb51e29c700 20 client.30429.objecter _get_session s=0x7fb504007a20 osd=2 3 2024-02-23T00:44:56.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.192+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb504007a20 osd=2 3 2024-02-23T00:44:56.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.192+0000 7fb51e29c700 20 client.30429.objecter _get_session s=0x7fb504007a20 osd=2 3 2024-02-23T00:44:56.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.192+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_remove -1 6 2024-02-23T00:44:56.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.192+0000 7fb51e29c700 20 client.30429.objecter get_session s=0x7fb504007a20 osd=2 3 2024-02-23T00:44:56.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.192+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_assign 2 6 2024-02-23T00:44:56.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.192+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb504007a20 osd=2 4 2024-02-23T00:44:56.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.192+0000 7fb51e29c700 10 client.30429.objecter _send_command 6 2024-02-23T00:44:56.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.192+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fb504003250 con 0x7fb504007b90 2024-02-23T00:44:56.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.193+0000 7fb515775700 1 -- 172.21.15.47:0/2165388759 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fb504003250 con 0x7fb504007b90 2024-02-23T00:44:56.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.193+0000 7fb515775700 10 client.30429.objecter ms_dispatch 0x7fb518001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:44:56.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.193+0000 7fb515775700 10 client.30429.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:44:56.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.193+0000 7fb515775700 20 client.30429.objecter put_session s=0x7fb504007a20 osd=2 4 2024-02-23T00:44:56.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.193+0000 7fb515775700 15 client.30429.objecter _session_command_op_remove 2 6 2024-02-23T00:44:56.196 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:44:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.193+0000 7fb51e29c700 10 client.30429.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.193+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_assign -1 7 2024-02-23T00:44:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.193+0000 7fb51e29c700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fb50400e6c0 0x7fb50400eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.193+0000 7fb51e29c700 20 client.30429.objecter _get_session s=0x7fb504001b70 osd=3 3 2024-02-23T00:44:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.193+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb504001b70 osd=3 3 2024-02-23T00:44:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.193+0000 7fb51e29c700 20 client.30429.objecter _get_session s=0x7fb504001b70 osd=3 3 2024-02-23T00:44:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.193+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_remove -1 7 2024-02-23T00:44:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.193+0000 7fb51e29c700 20 client.30429.objecter get_session s=0x7fb504001b70 osd=3 3 2024-02-23T00:44:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.193+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_assign 3 7 2024-02-23T00:44:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.193+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb504001b70 osd=3 4 2024-02-23T00:44:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.193+0000 7fb51e29c700 10 client.30429.objecter _send_command 7 2024-02-23T00:44:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.193+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb504002350 con 0x7fb50400e6c0 2024-02-23T00:44:56.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.194+0000 7fb51c839700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fb50400e6c0 0x7fb50400eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:56.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.194+0000 7fb515775700 10 client.30429.objecter ms_handle_connect 0x7fb50400e6c0 2024-02-23T00:44:56.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.194+0000 7fb51c839700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fb50400e6c0 0x7fb50400eb70 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:56.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.194+0000 7fb515775700 1 -- 172.21.15.47:0/2165388759 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb504002350 con 0x7fb50400e6c0 2024-02-23T00:44:56.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.194+0000 7fb515775700 10 client.30429.objecter ms_dispatch 0x7fb518001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:44:56.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.194+0000 7fb515775700 10 client.30429.objecter _finish_command 7 = system:0 2024-02-23T00:44:56.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.194+0000 7fb515775700 20 client.30429.objecter put_session s=0x7fb504001b70 osd=3 4 2024-02-23T00:44:56.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.194+0000 7fb515775700 15 client.30429.objecter _session_command_op_remove 3 7 2024-02-23T00:44:56.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.209+0000 7fb51e29c700 10 client.30429.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:44:56.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.209+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_assign -1 8 2024-02-23T00:44:56.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.209+0000 7fb51e29c700 20 client.30429.objecter _get_session s=0x7fb504001b70 osd=3 3 2024-02-23T00:44:56.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.209+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb504001b70 osd=3 3 2024-02-23T00:44:56.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.209+0000 7fb51e29c700 20 client.30429.objecter _get_session s=0x7fb504001b70 osd=3 3 2024-02-23T00:44:56.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.209+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_remove -1 8 2024-02-23T00:44:56.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.209+0000 7fb51e29c700 20 client.30429.objecter get_session s=0x7fb504001b70 osd=3 3 2024-02-23T00:44:56.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.209+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_assign 3 8 2024-02-23T00:44:56.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.209+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb504001b70 osd=3 4 2024-02-23T00:44:56.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.209+0000 7fb51e29c700 10 client.30429.objecter _send_command 8 2024-02-23T00:44:56.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.209+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fb50400f8b0 con 0x7fb50400e6c0 2024-02-23T00:44:56.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.211+0000 7fb515775700 1 -- 172.21.15.47:0/2165388759 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fb50400f8b0 con 0x7fb50400e6c0 2024-02-23T00:44:56.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.211+0000 7fb515775700 10 client.30429.objecter ms_dispatch 0x7fb518001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:44:56.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.211+0000 7fb515775700 10 client.30429.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:44:56.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.211+0000 7fb515775700 20 client.30429.objecter put_session s=0x7fb504001b70 osd=3 4 2024-02-23T00:44:56.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.211+0000 7fb515775700 15 client.30429.objecter _session_command_op_remove 3 8 2024-02-23T00:44:56.213 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:44:56.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.211+0000 7fb51e29c700 10 client.30429.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:56.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.211+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_assign -1 9 2024-02-23T00:44:56.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.211+0000 7fb51e29c700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fb504012020 0x7fb504014410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:56.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.211+0000 7fb51e29c700 20 client.30429.objecter _get_session s=0x7fb50400f8b0 osd=4 3 2024-02-23T00:44:56.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.211+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb50400f8b0 osd=4 3 2024-02-23T00:44:56.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.211+0000 7fb51e29c700 20 client.30429.objecter _get_session s=0x7fb50400f8b0 osd=4 3 2024-02-23T00:44:56.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.211+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_remove -1 9 2024-02-23T00:44:56.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.211+0000 7fb51e29c700 20 client.30429.objecter get_session s=0x7fb50400f8b0 osd=4 3 2024-02-23T00:44:56.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.211+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_assign 4 9 2024-02-23T00:44:56.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.211+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb50400f8b0 osd=4 4 2024-02-23T00:44:56.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.211+0000 7fb51e29c700 10 client.30429.objecter _send_command 9 2024-02-23T00:44:56.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.211+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb5040170b0 con 0x7fb504012020 2024-02-23T00:44:56.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.211+0000 7fb517fff700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fb504012020 0x7fb504014410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:56.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.212+0000 7fb515775700 10 client.30429.objecter ms_handle_connect 0x7fb504012020 2024-02-23T00:44:56.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.212+0000 7fb517fff700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fb504012020 0x7fb504014410 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:56.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.212+0000 7fb515775700 1 -- 172.21.15.47:0/2165388759 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb5040170b0 con 0x7fb504012020 2024-02-23T00:44:56.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.212+0000 7fb515775700 10 client.30429.objecter ms_dispatch 0x7fb518001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:44:56.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.212+0000 7fb515775700 10 client.30429.objecter _finish_command 9 = system:0 2024-02-23T00:44:56.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.212+0000 7fb515775700 20 client.30429.objecter put_session s=0x7fb50400f8b0 osd=4 4 2024-02-23T00:44:56.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.212+0000 7fb515775700 15 client.30429.objecter _session_command_op_remove 4 9 2024-02-23T00:44:56.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.227+0000 7fb51e29c700 10 client.30429.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:44:56.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.227+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_assign -1 10 2024-02-23T00:44:56.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.227+0000 7fb51e29c700 20 client.30429.objecter _get_session s=0x7fb50400f8b0 osd=4 3 2024-02-23T00:44:56.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.227+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb50400f8b0 osd=4 3 2024-02-23T00:44:56.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.227+0000 7fb51e29c700 20 client.30429.objecter _get_session s=0x7fb50400f8b0 osd=4 3 2024-02-23T00:44:56.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.227+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_remove -1 10 2024-02-23T00:44:56.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.227+0000 7fb51e29c700 20 client.30429.objecter get_session s=0x7fb50400f8b0 osd=4 3 2024-02-23T00:44:56.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.227+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_assign 4 10 2024-02-23T00:44:56.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.227+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb50400f8b0 osd=4 4 2024-02-23T00:44:56.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.227+0000 7fb51e29c700 10 client.30429.objecter _send_command 10 2024-02-23T00:44:56.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.227+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fb504014900 con 0x7fb504012020 2024-02-23T00:44:56.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.229+0000 7fb515775700 1 -- 172.21.15.47:0/2165388759 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fb504014900 con 0x7fb504012020 2024-02-23T00:44:56.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.229+0000 7fb515775700 10 client.30429.objecter ms_dispatch 0x7fb518001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:44:56.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.229+0000 7fb515775700 10 client.30429.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:44:56.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.229+0000 7fb515775700 20 client.30429.objecter put_session s=0x7fb50400f8b0 osd=4 4 2024-02-23T00:44:56.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.229+0000 7fb515775700 15 client.30429.objecter _session_command_op_remove 4 10 2024-02-23T00:44:56.232 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:44:56.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.229+0000 7fb51e29c700 10 client.30429.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:56.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.229+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_assign -1 11 2024-02-23T00:44:56.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.229+0000 7fb51e29c700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fb504016700 0x7fb504016b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:56.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.229+0000 7fb51e29c700 20 client.30429.objecter _get_session s=0x7fb504016080 osd=5 3 2024-02-23T00:44:56.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.229+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb504016080 osd=5 3 2024-02-23T00:44:56.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.229+0000 7fb51e29c700 20 client.30429.objecter _get_session s=0x7fb504016080 osd=5 3 2024-02-23T00:44:56.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.229+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_remove -1 11 2024-02-23T00:44:56.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.229+0000 7fb51e29c700 20 client.30429.objecter get_session s=0x7fb504016080 osd=5 3 2024-02-23T00:44:56.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.229+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_assign 5 11 2024-02-23T00:44:56.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.229+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb504016080 osd=5 4 2024-02-23T00:44:56.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.229+0000 7fb51e29c700 10 client.30429.objecter _send_command 11 2024-02-23T00:44:56.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.229+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb50401a2d0 con 0x7fb504016700 2024-02-23T00:44:56.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.230+0000 7fb5177fe700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fb504016700 0x7fb504016b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:56.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.230+0000 7fb515775700 10 client.30429.objecter ms_handle_connect 0x7fb504016700 2024-02-23T00:44:56.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.230+0000 7fb5177fe700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fb504016700 0x7fb504016b60 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:56.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.230+0000 7fb515775700 1 -- 172.21.15.47:0/2165388759 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb50401a2d0 con 0x7fb504016700 2024-02-23T00:44:56.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.230+0000 7fb515775700 10 client.30429.objecter ms_dispatch 0x7fb518001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:44:56.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.230+0000 7fb515775700 10 client.30429.objecter _finish_command 11 = system:0 2024-02-23T00:44:56.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.230+0000 7fb515775700 20 client.30429.objecter put_session s=0x7fb504016080 osd=5 4 2024-02-23T00:44:56.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.230+0000 7fb515775700 15 client.30429.objecter _session_command_op_remove 5 11 2024-02-23T00:44:56.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.245+0000 7fb51e29c700 10 client.30429.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:44:56.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.245+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_assign -1 12 2024-02-23T00:44:56.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.245+0000 7fb51e29c700 20 client.30429.objecter _get_session s=0x7fb504016080 osd=5 3 2024-02-23T00:44:56.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.245+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb504016080 osd=5 3 2024-02-23T00:44:56.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.245+0000 7fb51e29c700 20 client.30429.objecter _get_session s=0x7fb504016080 osd=5 3 2024-02-23T00:44:56.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.245+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_remove -1 12 2024-02-23T00:44:56.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.245+0000 7fb51e29c700 20 client.30429.objecter get_session s=0x7fb504016080 osd=5 3 2024-02-23T00:44:56.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.245+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_assign 5 12 2024-02-23T00:44:56.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.245+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb504016080 osd=5 4 2024-02-23T00:44:56.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.245+0000 7fb51e29c700 10 client.30429.objecter _send_command 12 2024-02-23T00:44:56.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.245+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fb50401a2d0 con 0x7fb504016700 2024-02-23T00:44:56.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.247+0000 7fb515775700 1 -- 172.21.15.47:0/2165388759 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fb50401a2d0 con 0x7fb504016700 2024-02-23T00:44:56.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.247+0000 7fb515775700 10 client.30429.objecter ms_dispatch 0x7fb518001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:44:56.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.247+0000 7fb515775700 10 client.30429.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:44:56.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.247+0000 7fb515775700 20 client.30429.objecter put_session s=0x7fb504016080 osd=5 4 2024-02-23T00:44:56.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.247+0000 7fb515775700 15 client.30429.objecter _session_command_op_remove 5 12 2024-02-23T00:44:56.250 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:44:56.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.247+0000 7fb51e29c700 10 client.30429.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.247+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_assign -1 13 2024-02-23T00:44:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.247+0000 7fb51e29c700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fb504019620 0x7fb504019aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.247+0000 7fb51e29c700 20 client.30429.objecter _get_session s=0x7fb50401a2d0 osd=6 3 2024-02-23T00:44:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.247+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb50401a2d0 osd=6 3 2024-02-23T00:44:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.247+0000 7fb51e29c700 20 client.30429.objecter _get_session s=0x7fb50401a2d0 osd=6 3 2024-02-23T00:44:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.247+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_remove -1 13 2024-02-23T00:44:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.247+0000 7fb51e29c700 20 client.30429.objecter get_session s=0x7fb50401a2d0 osd=6 3 2024-02-23T00:44:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.247+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_assign 6 13 2024-02-23T00:44:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.247+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb50401a2d0 osd=6 4 2024-02-23T00:44:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.247+0000 7fb51e29c700 10 client.30429.objecter _send_command 13 2024-02-23T00:44:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.247+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb50401d600 con 0x7fb504019620 2024-02-23T00:44:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.247+0000 7fb51c839700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fb504019620 0x7fb504019aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.248+0000 7fb515775700 10 client.30429.objecter ms_handle_connect 0x7fb504019620 2024-02-23T00:44:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.248+0000 7fb51c839700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fb504019620 0x7fb504019aa0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:56.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.248+0000 7fb515775700 1 -- 172.21.15.47:0/2165388759 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb50401d600 con 0x7fb504019620 2024-02-23T00:44:56.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.248+0000 7fb515775700 10 client.30429.objecter ms_dispatch 0x7fb518001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:44:56.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.248+0000 7fb515775700 10 client.30429.objecter _finish_command 13 = system:0 2024-02-23T00:44:56.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.248+0000 7fb515775700 20 client.30429.objecter put_session s=0x7fb50401a2d0 osd=6 4 2024-02-23T00:44:56.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.248+0000 7fb515775700 15 client.30429.objecter _session_command_op_remove 6 13 2024-02-23T00:44:56.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.263+0000 7fb51e29c700 10 client.30429.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:44:56.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.263+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_assign -1 14 2024-02-23T00:44:56.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.263+0000 7fb51e29c700 20 client.30429.objecter _get_session s=0x7fb50401a2d0 osd=6 3 2024-02-23T00:44:56.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.263+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb50401a2d0 osd=6 3 2024-02-23T00:44:56.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.263+0000 7fb51e29c700 20 client.30429.objecter _get_session s=0x7fb50401a2d0 osd=6 3 2024-02-23T00:44:56.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.263+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_remove -1 14 2024-02-23T00:44:56.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.263+0000 7fb51e29c700 20 client.30429.objecter get_session s=0x7fb50401a2d0 osd=6 3 2024-02-23T00:44:56.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.263+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_assign 6 14 2024-02-23T00:44:56.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.263+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb50401a2d0 osd=6 4 2024-02-23T00:44:56.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.263+0000 7fb51e29c700 10 client.30429.objecter _send_command 14 2024-02-23T00:44:56.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.263+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fb504014a10 con 0x7fb504019620 2024-02-23T00:44:56.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.265+0000 7fb515775700 1 -- 172.21.15.47:0/2165388759 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fb504014a10 con 0x7fb504019620 2024-02-23T00:44:56.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.265+0000 7fb515775700 10 client.30429.objecter ms_dispatch 0x7fb518001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:44:56.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.265+0000 7fb515775700 10 client.30429.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:44:56.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.265+0000 7fb515775700 20 client.30429.objecter put_session s=0x7fb50401a2d0 osd=6 4 2024-02-23T00:44:56.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.265+0000 7fb515775700 15 client.30429.objecter _session_command_op_remove 6 14 2024-02-23T00:44:56.268 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:44:56.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.265+0000 7fb51e29c700 10 client.30429.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:44:56.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.265+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_assign -1 15 2024-02-23T00:44:56.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.265+0000 7fb51e29c700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fb5040150c0 0x7fb504015520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:44:56.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.265+0000 7fb51e29c700 20 client.30429.objecter _get_session s=0x7fb504014a10 osd=7 3 2024-02-23T00:44:56.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.265+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb504014a10 osd=7 3 2024-02-23T00:44:56.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.265+0000 7fb51e29c700 20 client.30429.objecter _get_session s=0x7fb504014a10 osd=7 3 2024-02-23T00:44:56.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.265+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_remove -1 15 2024-02-23T00:44:56.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.265+0000 7fb51e29c700 20 client.30429.objecter get_session s=0x7fb504014a10 osd=7 3 2024-02-23T00:44:56.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.265+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_assign 7 15 2024-02-23T00:44:56.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.265+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb504014a10 osd=7 4 2024-02-23T00:44:56.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.265+0000 7fb51e29c700 10 client.30429.objecter _send_command 15 2024-02-23T00:44:56.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.265+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb504015b20 con 0x7fb5040150c0 2024-02-23T00:44:56.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.266+0000 7fb517fff700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fb5040150c0 0x7fb504015520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:44:56.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.266+0000 7fb515775700 10 client.30429.objecter ms_handle_connect 0x7fb5040150c0 2024-02-23T00:44:56.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.266+0000 7fb517fff700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fb5040150c0 0x7fb504015520 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:44:56.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.267+0000 7fb515775700 1 -- 172.21.15.47:0/2165388759 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb504015b20 con 0x7fb5040150c0 2024-02-23T00:44:56.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.267+0000 7fb515775700 10 client.30429.objecter ms_dispatch 0x7fb518001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:44:56.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.267+0000 7fb515775700 10 client.30429.objecter _finish_command 15 = system:0 2024-02-23T00:44:56.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.267+0000 7fb515775700 20 client.30429.objecter put_session s=0x7fb504014a10 osd=7 4 2024-02-23T00:44:56.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.267+0000 7fb515775700 15 client.30429.objecter _session_command_op_remove 7 15 2024-02-23T00:44:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.281+0000 7fb51e29c700 10 client.30429.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:44:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.281+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_assign -1 16 2024-02-23T00:44:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.281+0000 7fb51e29c700 20 client.30429.objecter _get_session s=0x7fb504014a10 osd=7 3 2024-02-23T00:44:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.281+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb504014a10 osd=7 3 2024-02-23T00:44:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.281+0000 7fb51e29c700 20 client.30429.objecter _get_session s=0x7fb504014a10 osd=7 3 2024-02-23T00:44:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.281+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_remove -1 16 2024-02-23T00:44:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.281+0000 7fb51e29c700 20 client.30429.objecter get_session s=0x7fb504014a10 osd=7 3 2024-02-23T00:44:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.281+0000 7fb51e29c700 15 client.30429.objecter _session_command_op_assign 7 16 2024-02-23T00:44:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.281+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb504014a10 osd=7 4 2024-02-23T00:44:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.281+0000 7fb51e29c700 10 client.30429.objecter _send_command 16 2024-02-23T00:44:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.281+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fb504015b20 con 0x7fb5040150c0 2024-02-23T00:44:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.283+0000 7fb515775700 1 -- 172.21.15.47:0/2165388759 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fb504015b20 con 0x7fb5040150c0 2024-02-23T00:44:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.283+0000 7fb515775700 10 client.30429.objecter ms_dispatch 0x7fb518001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:44:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.283+0000 7fb515775700 10 client.30429.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:44:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.283+0000 7fb515775700 20 client.30429.objecter put_session s=0x7fb504014a10 osd=7 4 2024-02-23T00:44:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.283+0000 7fb515775700 15 client.30429.objecter _session_command_op_remove 7 16 2024-02-23T00:44:56.286 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:44:56.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 10 librados: watch_flush enter 2024-02-23T00:44:56.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 10 librados: watch_flush exit 2024-02-23T00:44:56.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 10 client.30429.objecter close_session for osd.0 2024-02-23T00:44:56.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fb504004560 msgr2=0x7fb504004980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:56.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fb504004560 0x7fb504004980 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:56.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb5040051d0 osd=0 1 2024-02-23T00:44:56.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 10 client.30429.objecter close_session for osd.1 2024-02-23T00:44:56.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fb504008510 msgr2=0x7fb504002760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:56.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fb504008510 0x7fb504002760 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:56.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb504004380 osd=1 1 2024-02-23T00:44:56.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 10 client.30429.objecter close_session for osd.2 2024-02-23T00:44:56.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fb504007b90 msgr2=0x7fb50400d000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:56.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fb504007b90 0x7fb50400d000 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:56.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb504007a20 osd=2 1 2024-02-23T00:44:56.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 10 client.30429.objecter close_session for osd.3 2024-02-23T00:44:56.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fb50400e6c0 msgr2=0x7fb50400eb70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:56.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fb50400e6c0 0x7fb50400eb70 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:56.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb504001b70 osd=3 1 2024-02-23T00:44:56.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 10 client.30429.objecter close_session for osd.4 2024-02-23T00:44:56.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fb504012020 msgr2=0x7fb504014410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:56.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fb504012020 0x7fb504014410 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:56.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb50400f8b0 osd=4 1 2024-02-23T00:44:56.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 10 client.30429.objecter close_session for osd.5 2024-02-23T00:44:56.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51c839700 1 -- 172.21.15.47:0/2165388759 reap_dead start 2024-02-23T00:44:56.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fb504016700 msgr2=0x7fb504016b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:56.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fb504016700 0x7fb504016b60 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:56.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51c839700 1 -- 172.21.15.47:0/2165388759 reap_dead start 2024-02-23T00:44:56.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51c839700 1 -- 172.21.15.47:0/2165388759 reap_dead start 2024-02-23T00:44:56.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb504016080 osd=5 1 2024-02-23T00:44:56.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 10 client.30429.objecter close_session for osd.6 2024-02-23T00:44:56.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fb504019620 msgr2=0x7fb504019aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:56.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fb504019620 0x7fb504019aa0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:56.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb50401a2d0 osd=6 1 2024-02-23T00:44:56.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 10 client.30429.objecter close_session for osd.7 2024-02-23T00:44:56.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fb5040150c0 msgr2=0x7fb504015520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:56.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fb5040150c0 0x7fb504015520 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:44:56.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 20 client.30429.objecter put_session s=0x7fb504014a10 osd=7 1 2024-02-23T00:44:56.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 20 client.30429.objecter shutdown clearing up homeless session... 2024-02-23T00:44:56.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 10 client.30429.objecter successfully canceled tick 2024-02-23T00:44:56.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb500062cc0 msgr2=0x7fb500065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:56.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb500062cc0 0x7fb500065180 secure :-1 s=READY pgs=2966 cs=0 l=1 rev1=1 rx=0x7fb51810fa20 tx=0x7fb50c00b040).stop 2024-02-23T00:44:56.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5180f6b70 msgr2=0x7fb5181146a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:44:56.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 1 --2- 172.21.15.47:0/2165388759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5180f6b70 0x7fb5181146a0 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 rx=0x7fb5080043d0 tx=0x7fb508009cb0).stop 2024-02-23T00:44:56.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51c839700 1 -- 172.21.15.47:0/2165388759 reap_dead start 2024-02-23T00:44:56.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 shutdown_connections 2024-02-23T00:44:56.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 >> 172.21.15.47:0/2165388759 conn(0x7fb5180f26f0 msgr2=0x7fb5180f99b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:44:56.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 shutdown_connections 2024-02-23T00:44:56.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 1 -- 172.21.15.47:0/2165388759 wait complete. 2024-02-23T00:44:56.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:44:56.284+0000 7fb51e29c700 1 librados: shutdown 2024-02-23T00:44:56.292 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:44:56.292 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:44:56.292 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:44:56.292 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:44:56.292 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:44:56.292 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:44:56.292 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:44:56.292 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:44:56.735 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-02-23T00:44:56.735 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph pg dump --format=json 2024-02-23T00:44:56.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:55.211658+0000 osd.3 (osd.3) 121 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:44:56.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:55.211863+0000 osd.3 (osd.3) 122 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:44:56.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:55.231857+0000 mgr.y (mgr.14517) 251 : cluster [DBG] pgmap v310: 258 pgs: 8 unknown, 8 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 230 active+clean; 10 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:44:56.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:55.790118+0000 mon.b (mon.1) 283 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:56.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:56.113808+0000 mon.a (mon.0) 1578 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:44:56.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:56.118393+0000 mon.a (mon.0) 1579 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-19"}]': finished 2024-02-23T00:44:56.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:56.118509+0000 mon.a (mon.0) 1580 : cluster [DBG] osdmap e224: 8 total, 8 up, 8 in 2024-02-23T00:44:56.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:56.127004+0000 mon.c (mon.2) 146 : audit [INF] from='client.? 172.21.15.47:0/2606886447' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi047-82378-30"}]: dispatch 2024-02-23T00:44:56.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:56.137729+0000 mon.a (mon.0) 1581 : audit [INF] from='client.? 172.21.15.47:0/862585224' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi047-82386-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:56.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:56.138034+0000 mon.a (mon.0) 1582 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi047-82378-30"}]: dispatch 2024-02-23T00:44:56.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:56.139614+0000 mon.a (mon.0) 1583 : audit [DBG] from='client.? 172.21.15.47:0/2165388759' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:44:56.976 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:44:56.980 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:44:56.996 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:44:56.999 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:55.211658+0000 osd.3 (osd.3) 121 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:44:57.000 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:55.211863+0000 osd.3 (osd.3) 122 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:44:57.000 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:55.231857+0000 mgr.y (mgr.14517) 251 : cluster [DBG] pgmap v310: 258 pgs: 8 unknown, 8 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 230 active+clean; 10 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:44:57.000 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:55.790118+0000 mon.b (mon.1) 283 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:57.000 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:56.113808+0000 mon.a (mon.0) 1578 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:44:57.000 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:56.118393+0000 mon.a (mon.0) 1579 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-19"}]': finished 2024-02-23T00:44:57.000 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:56.118509+0000 mon.a (mon.0) 1580 : cluster [DBG] osdmap e224: 8 total, 8 up, 8 in 2024-02-23T00:44:57.000 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:56.127004+0000 mon.c (mon.2) 146 : audit [INF] from='client.? 172.21.15.47:0/2606886447' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi047-82378-30"}]: dispatch 2024-02-23T00:44:57.000 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:56.137729+0000 mon.a (mon.0) 1581 : audit [INF] from='client.? 172.21.15.47:0/862585224' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi047-82386-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:57.000 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:56.138034+0000 mon.a (mon.0) 1582 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi047-82378-30"}]: dispatch 2024-02-23T00:44:57.000 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:56.139614+0000 mon.a (mon.0) 1583 : audit [DBG] from='client.? 172.21.15.47:0/2165388759' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:44:57.001 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:57.001 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:44:57.001 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:55.211658+0000 osd.3 (osd.3) 121 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:44:57.001 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:55.211863+0000 osd.3 (osd.3) 122 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:44:57.002 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:55.231857+0000 mgr.y (mgr.14517) 251 : cluster [DBG] pgmap v310: 258 pgs: 8 unknown, 8 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 230 active+clean; 10 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:44:57.002 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:55.790118+0000 mon.b (mon.1) 283 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:57.002 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:56.113808+0000 mon.a (mon.0) 1578 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:44:57.002 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:56.118393+0000 mon.a (mon.0) 1579 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-19"}]': finished 2024-02-23T00:44:57.002 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:56.118509+0000 mon.a (mon.0) 1580 : cluster [DBG] osdmap e224: 8 total, 8 up, 8 in 2024-02-23T00:44:57.002 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:56.127004+0000 mon.c (mon.2) 146 : audit [INF] from='client.? 172.21.15.47:0/2606886447' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi047-82378-30"}]: dispatch 2024-02-23T00:44:57.002 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:56.137729+0000 mon.a (mon.0) 1581 : audit [INF] from='client.? 172.21.15.47:0/862585224' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi047-82386-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:57.002 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:56.138034+0000 mon.a (mon.0) 1582 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi047-82378-30"}]: dispatch 2024-02-23T00:44:57.002 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:56.139614+0000 mon.a (mon.0) 1583 : audit [DBG] from='client.? 172.21.15.47:0/2165388759' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:44:57.102 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:44:57.148 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:57.171 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:44:57.174 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:57.174 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:44:57.275 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:44:57.303 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:57.651 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:44:57.657 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:57.658 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:44:57.759 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:44:57.787 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:57.899 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:56.791021+0000 mon.b (mon.1) 284 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:57.899 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:57.119558+0000 mon.a (mon.0) 1584 : audit [INF] from='client.? 172.21.15.47:0/862585224' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi047-82386-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:57.899 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:57.119649+0000 mon.a (mon.0) 1585 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi047-82378-30"}]': finished 2024-02-23T00:44:57.899 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:57.119723+0000 mon.a (mon.0) 1586 : cluster [DBG] osdmap e225: 8 total, 8 up, 8 in 2024-02-23T00:44:57.899 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:57.126044+0000 mon.c (mon.2) 147 : audit [INF] from='client.? 172.21.15.47:0/2606886447' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi047-82378-30"}]: dispatch 2024-02-23T00:44:57.899 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:57.129528+0000 mon.a (mon.0) 1587 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi047-82378-30"}]: dispatch 2024-02-23T00:44:57.900 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:56.791021+0000 mon.b (mon.1) 284 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:57.900 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:57.119558+0000 mon.a (mon.0) 1584 : audit [INF] from='client.? 172.21.15.47:0/862585224' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi047-82386-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:57.900 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:57.119649+0000 mon.a (mon.0) 1585 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi047-82378-30"}]': finished 2024-02-23T00:44:57.900 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:57.119723+0000 mon.a (mon.0) 1586 : cluster [DBG] osdmap e225: 8 total, 8 up, 8 in 2024-02-23T00:44:57.900 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:57.126044+0000 mon.c (mon.2) 147 : audit [INF] from='client.? 172.21.15.47:0/2606886447' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi047-82378-30"}]: dispatch 2024-02-23T00:44:57.900 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:57.129528+0000 mon.a (mon.0) 1587 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi047-82378-30"}]: dispatch 2024-02-23T00:44:57.910 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:44:57.913 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:57.914 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:44:57.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:56.791021+0000 mon.b (mon.1) 284 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:57.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:57.119558+0000 mon.a (mon.0) 1584 : audit [INF] from='client.? 172.21.15.47:0/862585224' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi047-82386-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:57.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:57.119649+0000 mon.a (mon.0) 1585 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi047-82378-30"}]': finished 2024-02-23T00:44:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:57.119723+0000 mon.a (mon.0) 1586 : cluster [DBG] osdmap e225: 8 total, 8 up, 8 in 2024-02-23T00:44:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:57.126044+0000 mon.c (mon.2) 147 : audit [INF] from='client.? 172.21.15.47:0/2606886447' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi047-82378-30"}]: dispatch 2024-02-23T00:44:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:57.129528+0000 mon.a (mon.0) 1587 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi047-82378-30"}]: dispatch 2024-02-23T00:44:58.014 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:44:58.046 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:58.067 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:44:58.070 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:58.070 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:44:58.171 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:44:58.198 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:58.217 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:44:58.219 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:58.220 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:44:58.320 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:44:58.347 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:58.366 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:44:58.368 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:58.369 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:44:58.470 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:44:58.500 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:58.522 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:44:58.525 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:58.526 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:44:58.626 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:44:58.654 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:58.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:57.232649+0000 mgr.y (mgr.14517) 252 : cluster [DBG] pgmap v313: 250 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 232 active+clean; 9.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-23T00:44:58.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:57.792105+0000 mon.b (mon.1) 285 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:58.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:58.123731+0000 mon.a (mon.0) 1588 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi047-82378-30"}]': finished 2024-02-23T00:44:58.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:58.123844+0000 mon.a (mon.0) 1589 : cluster [DBG] osdmap e226: 8 total, 8 up, 8 in 2024-02-23T00:44:58.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:58.126868+0000 mon.a (mon.0) 1590 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:58.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:58.154482+0000 mon.a (mon.0) 1591 : audit [INF] from='client.? 172.21.15.47:0/4291660050' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi047-82378-31"}]: dispatch 2024-02-23T00:44:58.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:58.155689+0000 mon.a (mon.0) 1592 : audit [INF] from='client.? 172.21.15.47:0/4291660050' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi047-82378-31"}]: dispatch 2024-02-23T00:44:58.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:58.156148+0000 mon.a (mon.0) 1593 : audit [INF] from='client.? 172.21.15.47:0/4291660050' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi047-82378-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:58.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:57.232649+0000 mgr.y (mgr.14517) 252 : cluster [DBG] pgmap v313: 250 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 232 active+clean; 9.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-23T00:44:58.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:57.792105+0000 mon.b (mon.1) 285 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:58.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:58.123731+0000 mon.a (mon.0) 1588 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi047-82378-30"}]': finished 2024-02-23T00:44:58.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:58.123844+0000 mon.a (mon.0) 1589 : cluster [DBG] osdmap e226: 8 total, 8 up, 8 in 2024-02-23T00:44:58.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:58.126868+0000 mon.a (mon.0) 1590 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:58.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:58.154482+0000 mon.a (mon.0) 1591 : audit [INF] from='client.? 172.21.15.47:0/4291660050' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi047-82378-31"}]: dispatch 2024-02-23T00:44:58.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:58.155689+0000 mon.a (mon.0) 1592 : audit [INF] from='client.? 172.21.15.47:0/4291660050' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi047-82378-31"}]: dispatch 2024-02-23T00:44:58.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:58.156148+0000 mon.a (mon.0) 1593 : audit [INF] from='client.? 172.21.15.47:0/4291660050' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi047-82378-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:58.902 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:44:58.905 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:58.905 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:44:58.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:57.232649+0000 mgr.y (mgr.14517) 252 : cluster [DBG] pgmap v313: 250 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 232 active+clean; 9.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-23T00:44:58.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:57.792105+0000 mon.b (mon.1) 285 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:58.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:58.123731+0000 mon.a (mon.0) 1588 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi047-82378-30"}]': finished 2024-02-23T00:44:58.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:58.123844+0000 mon.a (mon.0) 1589 : cluster [DBG] osdmap e226: 8 total, 8 up, 8 in 2024-02-23T00:44:58.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:58.126868+0000 mon.a (mon.0) 1590 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:58.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:58.154482+0000 mon.a (mon.0) 1591 : audit [INF] from='client.? 172.21.15.47:0/4291660050' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi047-82378-31"}]: dispatch 2024-02-23T00:44:58.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:58.155689+0000 mon.a (mon.0) 1592 : audit [INF] from='client.? 172.21.15.47:0/4291660050' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi047-82378-31"}]: dispatch 2024-02-23T00:44:58.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:58.156148+0000 mon.a (mon.0) 1593 : audit [INF] from='client.? 172.21.15.47:0/4291660050' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi047-82378-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:44:59.006 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:44:59.032 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:59.052 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:44:59.054 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:59.055 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:44:59.155 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:44:59.182 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:59.220 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:44:59.222 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:59.223 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:44:59.324 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:44:59.352 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:59.630 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:44:59.801 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:44:59.804 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:44:59.804 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:44:59.905 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:44:59.931 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:58.792990+0000 mon.b (mon.1) 286 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:59.931 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:59.125955+0000 mon.a (mon.0) 1594 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:59.931 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:59.126196+0000 mon.a (mon.0) 1595 : audit [INF] from='client.? 172.21.15.47:0/4291660050' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi047-82378-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:44:59.931 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:59.126283+0000 mon.a (mon.0) 1596 : cluster [DBG] osdmap e227: 8 total, 8 up, 8 in 2024-02-23T00:44:59.931 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:59.132082+0000 mon.a (mon.0) 1597 : audit [INF] from='client.? 172.21.15.47:0/4291660050' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi047-82378-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi047-82378-31"}]: dispatch 2024-02-23T00:44:59.931 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:59.138951+0000 mon.a (mon.0) 1598 : audit [INF] from='client.? 172.21.15.47:0/1546714245' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi047-82386-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:59.931 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:44:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:59.150586+0000 mon.a (mon.0) 1599 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:44:59.931 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:58.792990+0000 mon.b (mon.1) 286 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:59.931 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:59.125955+0000 mon.a (mon.0) 1594 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:59.931 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:59.126196+0000 mon.a (mon.0) 1595 : audit [INF] from='client.? 172.21.15.47:0/4291660050' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi047-82378-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:44:59.931 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:59.126283+0000 mon.a (mon.0) 1596 : cluster [DBG] osdmap e227: 8 total, 8 up, 8 in 2024-02-23T00:44:59.932 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:59.132082+0000 mon.a (mon.0) 1597 : audit [INF] from='client.? 172.21.15.47:0/4291660050' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi047-82378-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi047-82378-31"}]: dispatch 2024-02-23T00:44:59.932 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:59.138951+0000 mon.a (mon.0) 1598 : audit [INF] from='client.? 172.21.15.47:0/1546714245' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi047-82386-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:59.932 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:44:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:59.150586+0000 mon.a (mon.0) 1599 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:44:59.933 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:44:59.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:58.792990+0000 mon.b (mon.1) 286 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:44:59.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:59.125955+0000 mon.a (mon.0) 1594 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:44:59.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:59.126196+0000 mon.a (mon.0) 1595 : audit [INF] from='client.? 172.21.15.47:0/4291660050' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi047-82378-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:44:59.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:59.126283+0000 mon.a (mon.0) 1596 : cluster [DBG] osdmap e227: 8 total, 8 up, 8 in 2024-02-23T00:44:59.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:59.132082+0000 mon.a (mon.0) 1597 : audit [INF] from='client.? 172.21.15.47:0/4291660050' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi047-82378-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi047-82378-31"}]: dispatch 2024-02-23T00:44:59.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:59.138951+0000 mon.a (mon.0) 1598 : audit [INF] from='client.? 172.21.15.47:0/1546714245' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi047-82386-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:44:59.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:44:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:59.150586+0000 mon.a (mon.0) 1599 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:45:00.084 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:45:00.091 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:00.092 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:45:00.192 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:45:00.224 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:00.245 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:45:00.248 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:00.248 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:45:00.349 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:45:00.377 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:00.384 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:45:00.384 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:45:00.397 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:45:00.399 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:00.400 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:45:00.410 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:45:00.500 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:45:00.530 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:00.654 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:45:00.657 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:00.658 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:45:00.758 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:45:00.790 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:00.806 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:45:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:45:00] "GET /metrics HTTP/1.1" 200 191767 "" "Prometheus/2.33.4" 2024-02-23T00:45:00.811 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:45:00.813 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:00.814 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:45:00.915 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:45:00.945 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:01.021 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:45:01.024 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:01.024 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:45:01.125 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:45:01.162 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:01.182 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:45:01.184 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:01.185 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:45:01.285 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:45:01.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:44:59.233035+0000 mgr.y (mgr.14517) 253 : cluster [DBG] pgmap v316: 282 pgs: 64 unknown, 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 200 active+clean; 9.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:45:01.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:44:59.794010+0000 mon.b (mon.1) 287 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:01.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:00.582080+0000 mon.a (mon.0) 1600 : audit [INF] from='client.? 172.21.15.47:0/1546714245' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi047-82386-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:01.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:00.582275+0000 mon.a (mon.0) 1601 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:45:01.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:00.582383+0000 mon.a (mon.0) 1602 : cluster [DBG] osdmap e228: 8 total, 8 up, 8 in 2024-02-23T00:45:01.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:00.583172+0000 mon.a (mon.0) 1603 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-21"}]: dispatch 2024-02-23T00:45:01.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:00.794935+0000 mon.b (mon.1) 288 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:01.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:44:59.233035+0000 mgr.y (mgr.14517) 253 : cluster [DBG] pgmap v316: 282 pgs: 64 unknown, 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 200 active+clean; 9.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:45:01.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:44:59.794010+0000 mon.b (mon.1) 287 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:01.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:00.582080+0000 mon.a (mon.0) 1600 : audit [INF] from='client.? 172.21.15.47:0/1546714245' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi047-82386-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:01.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:00.582275+0000 mon.a (mon.0) 1601 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:45:01.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:00.582383+0000 mon.a (mon.0) 1602 : cluster [DBG] osdmap e228: 8 total, 8 up, 8 in 2024-02-23T00:45:01.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:00.583172+0000 mon.a (mon.0) 1603 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-21"}]: dispatch 2024-02-23T00:45:01.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:00.794935+0000 mon.b (mon.1) 288 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:01.311 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:44:59.233035+0000 mgr.y (mgr.14517) 253 : cluster [DBG] pgmap v316: 282 pgs: 64 unknown, 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 200 active+clean; 9.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:45:01.311 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:44:59.794010+0000 mon.b (mon.1) 287 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:01.311 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:00.582080+0000 mon.a (mon.0) 1600 : audit [INF] from='client.? 172.21.15.47:0/1546714245' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi047-82386-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:01.311 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:00.582275+0000 mon.a (mon.0) 1601 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:45:01.311 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:00.582383+0000 mon.a (mon.0) 1602 : cluster [DBG] osdmap e228: 8 total, 8 up, 8 in 2024-02-23T00:45:01.311 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:00.583172+0000 mon.a (mon.0) 1603 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-21"}]: dispatch 2024-02-23T00:45:01.312 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:00.794935+0000 mon.b (mon.1) 288 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:01.313 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:01.332 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:45:01.334 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:01.335 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:45:01.436 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:45:01.462 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:01.484 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:01.485 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:45:01.485 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:45:01.586 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:45:01.614 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:01.814 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:45:01.816 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:01.817 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:45:01.918 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:45:01.947 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:02.046 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:45:02.060 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:02.061 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:45:02.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.073+0000 7fba0bf85700 1 -- 172.21.15.47:0/2777610501 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba040fcbb0 msgr2=0x7fba040fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:02.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.073+0000 7fba0bf85700 1 --2- 172.21.15.47:0/2777610501 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba040fcbb0 0x7fba040fcf90 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 rx=0x7fba000047b0 tx=0x7fba00009cb0).stop 2024-02-23T00:45:02.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.074+0000 7fba0bf85700 1 -- 172.21.15.47:0/2777610501 shutdown_connections 2024-02-23T00:45:02.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.074+0000 7fba0bf85700 1 --2- 172.21.15.47:0/2777610501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba040f7400 0x7fba04103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:02.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.074+0000 7fba0bf85700 1 --2- 172.21.15.47:0/2777610501 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba040fcbb0 0x7fba040fcf90 unknown :-1 s=CLOSED pgs=2549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:02.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.074+0000 7fba0bf85700 1 --2- 172.21.15.47:0/2777610501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba040fd510 0x7fba040f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:02.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.074+0000 7fba0bf85700 1 -- 172.21.15.47:0/2777610501 >> 172.21.15.47:0/2777610501 conn(0x7fba040f2820 msgr2=0x7fba040f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:45:02.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.074+0000 7fba0bf85700 1 -- 172.21.15.47:0/2777610501 shutdown_connections 2024-02-23T00:45:02.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.074+0000 7fba0bf85700 1 -- 172.21.15.47:0/2777610501 wait complete. 2024-02-23T00:45:02.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.074+0000 7fba0bf85700 1 librados: starting msgr at 2024-02-23T00:45:02.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.074+0000 7fba0bf85700 1 librados: starting objecter 2024-02-23T00:45:02.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.074+0000 7fba0bf85700 1 Processor -- start 2024-02-23T00:45:02.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.074+0000 7fba0bf85700 1 -- start start 2024-02-23T00:45:02.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.074+0000 7fba0bf85700 1 librados: setting wanted keys 2024-02-23T00:45:02.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.074+0000 7fba0bf85700 1 librados: calling monclient init 2024-02-23T00:45:02.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.075+0000 7fba0bf85700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba040f7400 0x7fba0410e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:02.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.075+0000 7fba0bf85700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba040fcbb0 0x7fba0410e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:02.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.075+0000 7fba0bf85700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba040fd510 0x7fba04108230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:02.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.075+0000 7fba0bf85700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba04108720 con 0x7fba040fd510 2024-02-23T00:45:02.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.075+0000 7fba0bf85700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba04108890 con 0x7fba040fcbb0 2024-02-23T00:45:02.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.075+0000 7fba0bf85700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba04108a50 con 0x7fba040f7400 2024-02-23T00:45:02.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.075+0000 7fba09d21700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba040f7400 0x7fba0410e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:02.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.075+0000 7fba09d21700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba040f7400 0x7fba0410e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:56786/0 (socket says 172.21.15.47:56786) 2024-02-23T00:45:02.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.075+0000 7fba09d21700 1 -- 172.21.15.47:0/2590580039 learned_addr learned my addr 172.21.15.47:0/2590580039 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:45:02.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.075+0000 7fba0a522700 1 --2- 172.21.15.47:0/2590580039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba040fd510 0x7fba04108230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:02.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.075+0000 7fba09520700 1 --2- 172.21.15.47:0/2590580039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba040fcbb0 0x7fba0410e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:02.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.075+0000 7fba09d21700 1 -- 172.21.15.47:0/2590580039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba040fcbb0 msgr2=0x7fba0410e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:02.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.075+0000 7fba09d21700 1 --2- 172.21.15.47:0/2590580039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba040fcbb0 0x7fba0410e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:02.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.075+0000 7fba09d21700 1 -- 172.21.15.47:0/2590580039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba040fd510 msgr2=0x7fba04108230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:02.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.075+0000 7fba09d21700 1 --2- 172.21.15.47:0/2590580039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba040fd510 0x7fba04108230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:02.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.075+0000 7fba09d21700 1 -- 172.21.15.47:0/2590580039 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f4005040 con 0x7fba040f7400 2024-02-23T00:45:02.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.075+0000 7fba0a522700 1 --2- 172.21.15.47:0/2590580039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba040fd510 0x7fba04108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:45:02.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.075+0000 7fba09d21700 1 --2- 172.21.15.47:0/2590580039 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba040f7400 0x7fba0410e090 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 rx=0x7fba00009f20 tx=0x7fba00009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:02.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.075+0000 7fb9faffd700 10 client.?.objecter ms_handle_connect 0x7fba040f7400 2024-02-23T00:45:02.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.075+0000 7fb9faffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:45:02.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.075+0000 7fb9faffd700 1 -- 172.21.15.47:0/2590580039 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba00018070 con 0x7fba040f7400 2024-02-23T00:45:02.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.075+0000 7fb9faffd700 1 -- 172.21.15.47:0/2590580039 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fba00013dd0 con 0x7fba040f7400 2024-02-23T00:45:02.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.075+0000 7fba0bf85700 1 -- 172.21.15.47:0/2590580039 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba00005040 con 0x7fba040f7400 2024-02-23T00:45:02.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.075+0000 7fb9faffd700 1 -- 172.21.15.47:0/2590580039 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0001da10 con 0x7fba040f7400 2024-02-23T00:45:02.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.075+0000 7fba0bf85700 10 client.40796.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:45:02.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.075+0000 7fba0bf85700 1 -- 172.21.15.47:0/2590580039 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba04108ef0 con 0x7fba040f7400 2024-02-23T00:45:02.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.075+0000 7fba0bf85700 1 librados: init done 2024-02-23T00:45:02.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.076+0000 7fba0bf85700 1 -- 172.21.15.47:0/2590580039 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba041090e0 con 0x7fba040f7400 2024-02-23T00:45:02.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.077+0000 7fb9faffd700 1 -- 172.21.15.47:0/2590580039 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fba00009240 con 0x7fba040f7400 2024-02-23T00:45:02.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.077+0000 7fb9faffd700 1 --2- 172.21.15.47:0/2590580039 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb9f0062cc0 0x7fb9f0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:02.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.077+0000 7fb9faffd700 1 -- 172.21.15.47:0/2590580039 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(229..229 src has 48..229) v4 ==== 11059+0+0 (secure 0 0 0) 0x7fba0007f000 con 0x7fba040f7400 2024-02-23T00:45:02.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.077+0000 7fba09520700 1 --2- 172.21.15.47:0/2590580039 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb9f0062cc0 0x7fb9f0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:02.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.077+0000 7fb9faffd700 10 client.40796.objecter ms_dispatch 0x7fba04001550 osd_map(229..229 src has 48..229) v4 2024-02-23T00:45:02.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.077+0000 7fb9faffd700 3 client.40796.objecter handle_osd_map got epochs [229,229] > 0 2024-02-23T00:45:02.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.077+0000 7fb9faffd700 3 client.40796.objecter handle_osd_map decoding full epoch 229 2024-02-23T00:45:02.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.078+0000 7fb9faffd700 20 client.40796.objecter dump_active .. 0 homeless 2024-02-23T00:45:02.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.078+0000 7fb9faffd700 10 client.40796.objecter ms_handle_connect 0x7fb9f0062cc0 2024-02-23T00:45:02.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.078+0000 7fba09520700 1 --2- 172.21.15.47:0/2590580039 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb9f0062cc0 0x7fb9f0065180 secure :-1 s=READY pgs=2970 cs=0 l=1 rev1=1 rx=0x7fba041099d0 tx=0x7fb9f400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:02.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.082+0000 7fb9faffd700 1 -- 172.21.15.47:0/2590580039 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fba000adf60 con 0x7fba040f7400 2024-02-23T00:45:02.162 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:45:02.193 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:02.212 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:45:02.215 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:02.215 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:45:02.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:02.293+0000 7fba0bf85700 1 -- 172.21.15.47:0/2590580039 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7fba04105ea0 con 0x7fba040f7400 2024-02-23T00:45:02.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:01.133628+0000 mon.a (mon.0) 1604 : audit [INF] from='client.? 172.21.15.47:0/4291660050' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi047-82378-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi047-82378-31"}]': finished 2024-02-23T00:45:02.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:01.133735+0000 mon.a (mon.0) 1605 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-21"}]': finished 2024-02-23T00:45:02.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:01.133830+0000 mon.a (mon.0) 1606 : cluster [DBG] osdmap e229: 8 total, 8 up, 8 in 2024-02-23T00:45:02.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:01.139404+0000 mon.c (mon.2) 148 : audit [INF] from='client.? 172.21.15.47:0/471315850' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi047-82386-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:02.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:01.142332+0000 mon.a (mon.0) 1607 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi047-82386-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:02.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:01.142574+0000 mon.a (mon.0) 1608 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-21", "mode": "writeback"}]: dispatch 2024-02-23T00:45:02.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:01.233516+0000 mgr.y (mgr.14517) 254 : cluster [DBG] pgmap v319: 322 pgs: 104 unknown, 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 200 active+clean; 9.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:45:02.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:01.795841+0000 mon.b (mon.1) 289 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:02.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:01.133628+0000 mon.a (mon.0) 1604 : audit [INF] from='client.? 172.21.15.47:0/4291660050' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi047-82378-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi047-82378-31"}]': finished 2024-02-23T00:45:02.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:01.133735+0000 mon.a (mon.0) 1605 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-21"}]': finished 2024-02-23T00:45:02.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:01.133830+0000 mon.a (mon.0) 1606 : cluster [DBG] osdmap e229: 8 total, 8 up, 8 in 2024-02-23T00:45:02.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:01.139404+0000 mon.c (mon.2) 148 : audit [INF] from='client.? 172.21.15.47:0/471315850' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi047-82386-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:02.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:01.142332+0000 mon.a (mon.0) 1607 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi047-82386-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:02.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:01.142574+0000 mon.a (mon.0) 1608 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-21", "mode": "writeback"}]: dispatch 2024-02-23T00:45:02.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:01.233516+0000 mgr.y (mgr.14517) 254 : cluster [DBG] pgmap v319: 322 pgs: 104 unknown, 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 200 active+clean; 9.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:45:02.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:01.795841+0000 mon.b (mon.1) 289 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:02.316 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:45:02.342 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:02.361 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:45:02.363 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:02.363 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:45:02.420 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:01.133628+0000 mon.a (mon.0) 1604 : audit [INF] from='client.? 172.21.15.47:0/4291660050' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi047-82378-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi047-82378-31"}]': finished 2024-02-23T00:45:02.420 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:01.133735+0000 mon.a (mon.0) 1605 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-21"}]': finished 2024-02-23T00:45:02.420 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:01.133830+0000 mon.a (mon.0) 1606 : cluster [DBG] osdmap e229: 8 total, 8 up, 8 in 2024-02-23T00:45:02.420 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:01.139404+0000 mon.c (mon.2) 148 : audit [INF] from='client.? 172.21.15.47:0/471315850' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi047-82386-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:02.420 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:01.142332+0000 mon.a (mon.0) 1607 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi047-82386-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:02.420 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:01.142574+0000 mon.a (mon.0) 1608 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-21", "mode": "writeback"}]: dispatch 2024-02-23T00:45:02.421 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:01.233516+0000 mgr.y (mgr.14517) 254 : cluster [DBG] pgmap v319: 322 pgs: 104 unknown, 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 200 active+clean; 9.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:45:02.421 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:01.795841+0000 mon.b (mon.1) 289 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:02.464 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:45:02.490 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:02.509 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:45:02.511 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:02.511 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:45:02.601 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:45:02.612 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:45:02.623 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.1 dump_ops_in_flight 2024-02-23T00:45:02.642 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:02.693 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:45:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:45:02.693 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:45:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:45:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:45:03.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:03.144+0000 7fb9faffd700 1 -- 172.21.15.47:0/2590580039 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v231) v1 ==== 91+0+0 (secure 0 0 0) 0x7fba0004dee0 con 0x7fba040f7400 2024-02-23T00:45:03.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:03.147+0000 7fba0bf85700 10 librados: watch_flush enter 2024-02-23T00:45:03.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:03.148+0000 7fba0bf85700 10 librados: watch_flush exit 2024-02-23T00:45:03.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:03.148+0000 7fba0bf85700 20 client.40796.objecter shutdown clearing up homeless session... 2024-02-23T00:45:03.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:03.148+0000 7fba0bf85700 10 client.40796.objecter successfully canceled tick 2024-02-23T00:45:03.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:03.153+0000 7fba0bf85700 1 -- 172.21.15.47:0/2590580039 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb9f0062cc0 msgr2=0x7fb9f0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:03.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:03.153+0000 7fba0bf85700 1 --2- 172.21.15.47:0/2590580039 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb9f0062cc0 0x7fb9f0065180 secure :-1 s=READY pgs=2970 cs=0 l=1 rev1=1 rx=0x7fba041099d0 tx=0x7fb9f400b040).stop 2024-02-23T00:45:03.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:03.153+0000 7fba0bf85700 1 -- 172.21.15.47:0/2590580039 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba040f7400 msgr2=0x7fba0410e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:03.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:03.153+0000 7fba0bf85700 1 --2- 172.21.15.47:0/2590580039 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba040f7400 0x7fba0410e090 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 rx=0x7fba00009f20 tx=0x7fba00009cb0).stop 2024-02-23T00:45:03.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:03.159+0000 7fba0bf85700 1 -- 172.21.15.47:0/2590580039 shutdown_connections 2024-02-23T00:45:03.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:03.159+0000 7fba0bf85700 1 --2- 172.21.15.47:0/2590580039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba040fd510 0x7fba04108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:03.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:03.159+0000 7fba0bf85700 1 --2- 172.21.15.47:0/2590580039 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb9f0062cc0 0x7fb9f0065180 unknown :-1 s=CLOSED pgs=2970 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:03.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:03.159+0000 7fba0bf85700 1 --2- 172.21.15.47:0/2590580039 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba040f7400 0x7fba0410e090 unknown :-1 s=CLOSED pgs=2550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:03.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:03.159+0000 7fba0bf85700 1 --2- 172.21.15.47:0/2590580039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba040fcbb0 0x7fba0410e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:03.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:03.159+0000 7fba0bf85700 1 -- 172.21.15.47:0/2590580039 >> 172.21.15.47:0/2590580039 conn(0x7fba040f2820 msgr2=0x7fba04060eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:45:03.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:03.160+0000 7fba0bf85700 1 -- 172.21.15.47:0/2590580039 shutdown_connections 2024-02-23T00:45:03.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:03.160+0000 7fba0bf85700 1 -- 172.21.15.47:0/2590580039 wait complete. 2024-02-23T00:45:03.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:03.160+0000 7fba0bf85700 1 librados: shutdown 2024-02-23T00:45:03.163 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is unset 2024-02-23T00:45:03.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:02.131152+0000 mon.a (mon.0) 1609 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:45:03.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:02.138659+0000 mon.a (mon.0) 1610 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi047-82386-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:03.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:02.138767+0000 mon.a (mon.0) 1611 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-21", "mode": "writeback"}]': finished 2024-02-23T00:45:03.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:02.138894+0000 mon.a (mon.0) 1612 : cluster [DBG] osdmap e230: 8 total, 8 up, 8 in 2024-02-23T00:45:03.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:02.295344+0000 mon.c (mon.2) 149 : audit [INF] from='client.? 172.21.15.47:0/2590580039' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:45:03.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:02.418659+0000 mgr.y (mgr.14517) 255 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:45:03.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:02.626173+0000 mon.a (mon.0) 1613 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:45:03.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:02.626439+0000 mon.a (mon.0) 1614 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:45:03.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:02.796832+0000 mon.b (mon.1) 290 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:03.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:02.131152+0000 mon.a (mon.0) 1609 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:45:03.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:02.138659+0000 mon.a (mon.0) 1610 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi047-82386-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:03.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:02.138767+0000 mon.a (mon.0) 1611 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-21", "mode": "writeback"}]': finished 2024-02-23T00:45:03.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:02.138894+0000 mon.a (mon.0) 1612 : cluster [DBG] osdmap e230: 8 total, 8 up, 8 in 2024-02-23T00:45:03.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:02.295344+0000 mon.c (mon.2) 149 : audit [INF] from='client.? 172.21.15.47:0/2590580039' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:45:03.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:02.418659+0000 mgr.y (mgr.14517) 255 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:45:03.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:02.626173+0000 mon.a (mon.0) 1613 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:45:03.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:02.626439+0000 mon.a (mon.0) 1614 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:45:03.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:02.796832+0000 mon.b (mon.1) 290 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:03.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:02.131152+0000 mon.a (mon.0) 1609 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:45:03.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:02.138659+0000 mon.a (mon.0) 1610 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi047-82386-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:03.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:02.138767+0000 mon.a (mon.0) 1611 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-21", "mode": "writeback"}]': finished 2024-02-23T00:45:03.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:02.138894+0000 mon.a (mon.0) 1612 : cluster [DBG] osdmap e230: 8 total, 8 up, 8 in 2024-02-23T00:45:03.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:02.295344+0000 mon.c (mon.2) 149 : audit [INF] from='client.? 172.21.15.47:0/2590580039' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:45:03.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:02.418659+0000 mgr.y (mgr.14517) 255 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:45:03.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:02.626173+0000 mon.a (mon.0) 1613 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:45:03.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:02.626439+0000 mon.a (mon.0) 1614 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:45:03.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:02.796832+0000 mon.b (mon.1) 290 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:04.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:03.135870+0000 mon.a (mon.0) 1615 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:45:04.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:03.144144+0000 mon.a (mon.0) 1616 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:45:04.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:03.144225+0000 mon.a (mon.0) 1617 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:45:04.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:03.144294+0000 mon.a (mon.0) 1618 : cluster [DBG] osdmap e231: 8 total, 8 up, 8 in 2024-02-23T00:45:04.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:03.149075+0000 mon.a (mon.0) 1619 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-21"}]: dispatch 2024-02-23T00:45:04.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:03.150581+0000 mon.c (mon.2) 150 : audit [INF] from='client.? 172.21.15.47:0/1045433648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi047-82386-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:04.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:03.150865+0000 mon.a (mon.0) 1620 : audit [INF] from='client.? 172.21.15.47:0/4291660050' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi047-82378-31"}]: dispatch 2024-02-23T00:45:04.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:03.151150+0000 mon.a (mon.0) 1621 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi047-82386-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:04.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:03.227170+0000 osd.4 (osd.4) 101 : cluster [DBG] 6.2 scrub starts 2024-02-23T00:45:04.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:03.227988+0000 osd.4 (osd.4) 102 : cluster [DBG] 6.2 scrub ok 2024-02-23T00:45:04.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:03.233964+0000 mgr.y (mgr.14517) 256 : cluster [DBG] pgmap v322: 346 pgs: 8 creating+activating, 32 unknown, 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 288 active+clean; 9.5 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-23T00:45:04.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:03.798050+0000 mon.b (mon.1) 291 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:04.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:03.135870+0000 mon.a (mon.0) 1615 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:45:04.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:03.144144+0000 mon.a (mon.0) 1616 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:45:04.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:03.144225+0000 mon.a (mon.0) 1617 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:45:04.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:03.144294+0000 mon.a (mon.0) 1618 : cluster [DBG] osdmap e231: 8 total, 8 up, 8 in 2024-02-23T00:45:04.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:03.149075+0000 mon.a (mon.0) 1619 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-21"}]: dispatch 2024-02-23T00:45:04.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:03.150581+0000 mon.c (mon.2) 150 : audit [INF] from='client.? 172.21.15.47:0/1045433648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi047-82386-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:04.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:03.150865+0000 mon.a (mon.0) 1620 : audit [INF] from='client.? 172.21.15.47:0/4291660050' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi047-82378-31"}]: dispatch 2024-02-23T00:45:04.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:03.151150+0000 mon.a (mon.0) 1621 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi047-82386-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:04.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:03.227170+0000 osd.4 (osd.4) 101 : cluster [DBG] 6.2 scrub starts 2024-02-23T00:45:04.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:03.227988+0000 osd.4 (osd.4) 102 : cluster [DBG] 6.2 scrub ok 2024-02-23T00:45:04.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:03.233964+0000 mgr.y (mgr.14517) 256 : cluster [DBG] pgmap v322: 346 pgs: 8 creating+activating, 32 unknown, 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 288 active+clean; 9.5 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-23T00:45:04.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:03.798050+0000 mon.b (mon.1) 291 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:04.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:03.135870+0000 mon.a (mon.0) 1615 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:45:04.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:03.144144+0000 mon.a (mon.0) 1616 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:45:04.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:03.144225+0000 mon.a (mon.0) 1617 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:45:04.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:03.144294+0000 mon.a (mon.0) 1618 : cluster [DBG] osdmap e231: 8 total, 8 up, 8 in 2024-02-23T00:45:04.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:03.149075+0000 mon.a (mon.0) 1619 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-21"}]: dispatch 2024-02-23T00:45:04.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:03.150581+0000 mon.c (mon.2) 150 : audit [INF] from='client.? 172.21.15.47:0/1045433648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi047-82386-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:04.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:03.150865+0000 mon.a (mon.0) 1620 : audit [INF] from='client.? 172.21.15.47:0/4291660050' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi047-82378-31"}]: dispatch 2024-02-23T00:45:04.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:03.151150+0000 mon.a (mon.0) 1621 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi047-82386-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:04.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:03.227170+0000 osd.4 (osd.4) 101 : cluster [DBG] 6.2 scrub starts 2024-02-23T00:45:04.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:03.227988+0000 osd.4 (osd.4) 102 : cluster [DBG] 6.2 scrub ok 2024-02-23T00:45:04.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:03.233964+0000 mgr.y (mgr.14517) 256 : cluster [DBG] pgmap v322: 346 pgs: 8 creating+activating, 32 unknown, 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 288 active+clean; 9.5 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-23T00:45:04.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:03.798050+0000 mon.b (mon.1) 291 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:04.495 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:45:04.502 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:04.503 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:45:04.604 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:45:04.638 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:05.574 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:45:05.590 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:45:05.593 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:05.594 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:45:05.694 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:45:05.726 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:05.747 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:45:05.750 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:05.750 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:45:05.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:03.614615+0000 osd.1 (osd.1) 115 : cluster [DBG] 5.1d scrub starts 2024-02-23T00:45:05.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:03.615688+0000 osd.1 (osd.1) 116 : cluster [DBG] 5.1d scrub ok 2024-02-23T00:45:05.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:03.715650+0000 osd.1 (osd.1) 117 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:45:05.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:03.715794+0000 osd.1 (osd.1) 118 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:45:05.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:04.136079+0000 osd.5 (osd.5) 105 : cluster [DBG] 4.8 scrub starts 2024-02-23T00:45:05.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:04.138741+0000 osd.5 (osd.5) 106 : cluster [DBG] 4.8 scrub ok 2024-02-23T00:45:05.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:04.143132+0000 mon.a (mon.0) 1622 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:45:05.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:04.149588+0000 mon.a (mon.0) 1623 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-21"}]': finished 2024-02-23T00:45:05.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:04.149714+0000 mon.a (mon.0) 1624 : audit [INF] from='client.? 172.21.15.47:0/4291660050' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi047-82378-31"}]': finished 2024-02-23T00:45:05.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:04.149781+0000 mon.a (mon.0) 1625 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi047-82386-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:05.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:04.149958+0000 mon.a (mon.0) 1626 : cluster [DBG] osdmap e232: 8 total, 8 up, 8 in 2024-02-23T00:45:05.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:04.161071+0000 mon.a (mon.0) 1627 : audit [INF] from='client.? 172.21.15.47:0/4291660050' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi047-82378-31"}]: dispatch 2024-02-23T00:45:05.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:04.522006+0000 osd.0 (osd.0) 105 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:45:05.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:04.522190+0000 osd.0 (osd.0) 106 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:45:05.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:04.799087+0000 mon.b (mon.1) 292 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:05.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:05.120523+0000 mon.a (mon.0) 1628 : audit [INF] from='client.? 172.21.15.47:0/4291660050' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi047-82378-31"}]': finished 2024-02-23T00:45:05.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:05.120640+0000 mon.a (mon.0) 1629 : cluster [DBG] osdmap e233: 8 total, 8 up, 8 in 2024-02-23T00:45:05.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:03.614615+0000 osd.1 (osd.1) 115 : cluster [DBG] 5.1d scrub starts 2024-02-23T00:45:05.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:03.615688+0000 osd.1 (osd.1) 116 : cluster [DBG] 5.1d scrub ok 2024-02-23T00:45:05.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:03.715650+0000 osd.1 (osd.1) 117 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:45:05.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:03.715794+0000 osd.1 (osd.1) 118 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:45:05.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:04.136079+0000 osd.5 (osd.5) 105 : cluster [DBG] 4.8 scrub starts 2024-02-23T00:45:05.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:04.138741+0000 osd.5 (osd.5) 106 : cluster [DBG] 4.8 scrub ok 2024-02-23T00:45:05.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:04.143132+0000 mon.a (mon.0) 1622 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:45:05.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:04.149588+0000 mon.a (mon.0) 1623 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-21"}]': finished 2024-02-23T00:45:05.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:04.149714+0000 mon.a (mon.0) 1624 : audit [INF] from='client.? 172.21.15.47:0/4291660050' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi047-82378-31"}]': finished 2024-02-23T00:45:05.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:04.149781+0000 mon.a (mon.0) 1625 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi047-82386-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:05.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:04.149958+0000 mon.a (mon.0) 1626 : cluster [DBG] osdmap e232: 8 total, 8 up, 8 in 2024-02-23T00:45:05.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:04.161071+0000 mon.a (mon.0) 1627 : audit [INF] from='client.? 172.21.15.47:0/4291660050' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi047-82378-31"}]: dispatch 2024-02-23T00:45:05.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:04.522006+0000 osd.0 (osd.0) 105 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:45:05.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:04.522190+0000 osd.0 (osd.0) 106 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:45:05.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:04.799087+0000 mon.b (mon.1) 292 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:05.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:05.120523+0000 mon.a (mon.0) 1628 : audit [INF] from='client.? 172.21.15.47:0/4291660050' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi047-82378-31"}]': finished 2024-02-23T00:45:05.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:05.120640+0000 mon.a (mon.0) 1629 : cluster [DBG] osdmap e233: 8 total, 8 up, 8 in 2024-02-23T00:45:05.851 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:45:05.880 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:05.908 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:45:05.911 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:05.911 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:45:05.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:03.614615+0000 osd.1 (osd.1) 115 : cluster [DBG] 5.1d scrub starts 2024-02-23T00:45:05.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:03.615688+0000 osd.1 (osd.1) 116 : cluster [DBG] 5.1d scrub ok 2024-02-23T00:45:05.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:03.715650+0000 osd.1 (osd.1) 117 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:45:05.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:03.715794+0000 osd.1 (osd.1) 118 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:45:05.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:04.136079+0000 osd.5 (osd.5) 105 : cluster [DBG] 4.8 scrub starts 2024-02-23T00:45:05.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:04.138741+0000 osd.5 (osd.5) 106 : cluster [DBG] 4.8 scrub ok 2024-02-23T00:45:05.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:04.143132+0000 mon.a (mon.0) 1622 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:45:05.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:04.149588+0000 mon.a (mon.0) 1623 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-21"}]': finished 2024-02-23T00:45:05.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:04.149714+0000 mon.a (mon.0) 1624 : audit [INF] from='client.? 172.21.15.47:0/4291660050' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi047-82378-31"}]': finished 2024-02-23T00:45:05.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:04.149781+0000 mon.a (mon.0) 1625 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi047-82386-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:05.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:04.149958+0000 mon.a (mon.0) 1626 : cluster [DBG] osdmap e232: 8 total, 8 up, 8 in 2024-02-23T00:45:05.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:04.161071+0000 mon.a (mon.0) 1627 : audit [INF] from='client.? 172.21.15.47:0/4291660050' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi047-82378-31"}]: dispatch 2024-02-23T00:45:05.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:04.522006+0000 osd.0 (osd.0) 105 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:45:05.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:04.522190+0000 osd.0 (osd.0) 106 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:45:05.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:04.799087+0000 mon.b (mon.1) 292 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:05.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:05.120523+0000 mon.a (mon.0) 1628 : audit [INF] from='client.? 172.21.15.47:0/4291660050' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi047-82378-31"}]': finished 2024-02-23T00:45:05.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:05.120640+0000 mon.a (mon.0) 1629 : cluster [DBG] osdmap e233: 8 total, 8 up, 8 in 2024-02-23T00:45:06.012 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:45:06.043 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:06.065 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:45:06.067 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:06.068 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:45:06.169 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:45:06.199 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:06.519 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-23T00:45:06.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.640+0000 7fa921cd4700 1 -- 172.21.15.47:0/1053103370 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa91c0fe510 msgr2=0x7fa91c0f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:06.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.640+0000 7fa921cd4700 1 --2- 172.21.15.47:0/1053103370 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa91c0fe510 0x7fa91c0f7d40 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 rx=0x7fa910004870 tx=0x7fa910009cb0).stop 2024-02-23T00:45:06.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.640+0000 7fa921cd4700 1 -- 172.21.15.47:0/1053103370 shutdown_connections 2024-02-23T00:45:06.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.640+0000 7fa921cd4700 1 --2- 172.21.15.47:0/1053103370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa91c0f8420 0x7fa91c104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:06.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.640+0000 7fa921cd4700 1 --2- 172.21.15.47:0/1053103370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa91c0fdbb0 0x7fa91c0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:06.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.640+0000 7fa921cd4700 1 --2- 172.21.15.47:0/1053103370 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa91c0fe510 0x7fa91c0f7d40 unknown :-1 s=CLOSED pgs=2555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:06.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.640+0000 7fa921cd4700 1 -- 172.21.15.47:0/1053103370 >> 172.21.15.47:0/1053103370 conn(0x7fa91c0f3800 msgr2=0x7fa91c0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:45:06.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.640+0000 7fa921cd4700 1 -- 172.21.15.47:0/1053103370 shutdown_connections 2024-02-23T00:45:06.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.640+0000 7fa921cd4700 1 -- 172.21.15.47:0/1053103370 wait complete. 2024-02-23T00:45:06.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.641+0000 7fa921cd4700 1 librados: starting msgr at 2024-02-23T00:45:06.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.641+0000 7fa921cd4700 1 librados: starting objecter 2024-02-23T00:45:06.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.641+0000 7fa921cd4700 1 Processor -- start 2024-02-23T00:45:06.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.641+0000 7fa921cd4700 1 -- start start 2024-02-23T00:45:06.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.641+0000 7fa921cd4700 1 librados: setting wanted keys 2024-02-23T00:45:06.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.641+0000 7fa921cd4700 1 librados: calling monclient init 2024-02-23T00:45:06.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.641+0000 7fa921cd4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa91c0f8420 0x7fa91c10dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:06.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.641+0000 7fa921cd4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa91c0fdbb0 0x7fa91c10e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.641+0000 7fa921cd4700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa91c0fe510 0x7fa91c108180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.641+0000 7fa921cd4700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c108670 con 0x7fa91c0fdbb0 2024-02-23T00:45:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.641+0000 7fa921cd4700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c1087e0 con 0x7fa91c0f8420 2024-02-23T00:45:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.641+0000 7fa921cd4700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c1089a0 con 0x7fa91c0fe510 2024-02-23T00:45:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.641+0000 7fa91bfff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa91c0fe510 0x7fa91c108180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.641+0000 7fa91affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa91c0fdbb0 0x7fa91c10e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.641+0000 7fa91bfff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa91c0fe510 0x7fa91c108180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52210/0 (socket says 172.21.15.47:52210) 2024-02-23T00:45:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.641+0000 7fa91bfff700 1 -- 172.21.15.47:0/2316729014 learned_addr learned my addr 172.21.15.47:0/2316729014 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:45:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.641+0000 7fa91b7fe700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa91c0f8420 0x7fa91c10dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.642+0000 7fa91affd700 1 -- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa91c0fe510 msgr2=0x7fa91c108180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.642+0000 7fa91affd700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa91c0fe510 0x7fa91c108180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.642+0000 7fa91affd700 1 -- 172.21.15.47:0/2316729014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa91c0f8420 msgr2=0x7fa91c10dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.642+0000 7fa91affd700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa91c0f8420 0x7fa91c10dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.642+0000 7fa91affd700 1 -- 172.21.15.47:0/2316729014 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa910005040 con 0x7fa91c0fdbb0 2024-02-23T00:45:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.642+0000 7fa91bfff700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa91c0fe510 0x7fa91c108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:45:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.642+0000 7fa91b7fe700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa91c0f8420 0x7fa91c10dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:45:06.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.642+0000 7fa91affd700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa91c0fdbb0 0x7fa91c10e4d0 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 rx=0x7fa910009da0 tx=0x7fa910009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:06.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.642+0000 7fa918ff9700 10 client.?.objecter ms_handle_connect 0x7fa91c0fdbb0 2024-02-23T00:45:06.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.642+0000 7fa918ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:45:06.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.642+0000 7fa918ff9700 1 -- 172.21.15.47:0/2316729014 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91000e030 con 0x7fa91c0fdbb0 2024-02-23T00:45:06.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.642+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa91c108c20 con 0x7fa91c0fdbb0 2024-02-23T00:45:06.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.642+0000 7fa918ff9700 1 -- 172.21.15.47:0/2316729014 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fa910013d70 con 0x7fa91c0fdbb0 2024-02-23T00:45:06.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.642+0000 7fa918ff9700 1 -- 172.21.15.47:0/2316729014 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa910018070 con 0x7fa91c0fdbb0 2024-02-23T00:45:06.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.642+0000 7fa921cd4700 10 client.30471.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:45:06.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.642+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa91c114e10 con 0x7fa91c0fdbb0 2024-02-23T00:45:06.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.642+0000 7fa921cd4700 1 librados: init done 2024-02-23T00:45:06.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.643+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fa91c05ee80 con 0x7fa91c0fdbb0 2024-02-23T00:45:06.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.644+0000 7fa918ff9700 1 -- 172.21.15.47:0/2316729014 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fa910009240 con 0x7fa91c0fdbb0 2024-02-23T00:45:06.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.644+0000 7fa918ff9700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa904062cc0 0x7fa904065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:06.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.644+0000 7fa918ff9700 1 -- 172.21.15.47:0/2316729014 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(234..234 src has 48..234) v4 ==== 10145+0+0 (secure 0 0 0) 0x7fa91007f470 con 0x7fa91c0fdbb0 2024-02-23T00:45:06.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.644+0000 7fa918ff9700 10 client.30471.objecter ms_dispatch 0x7fa91c001550 osd_map(234..234 src has 48..234) v4 2024-02-23T00:45:06.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.644+0000 7fa918ff9700 3 client.30471.objecter handle_osd_map got epochs [234,234] > 0 2024-02-23T00:45:06.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.644+0000 7fa918ff9700 3 client.30471.objecter handle_osd_map decoding full epoch 234 2024-02-23T00:45:06.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.644+0000 7fa91b7fe700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa904062cc0 0x7fa904065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:06.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.644+0000 7fa918ff9700 20 client.30471.objecter dump_active .. 0 homeless 2024-02-23T00:45:06.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.644+0000 7fa918ff9700 1 -- 172.21.15.47:0/2316729014 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v234) v1 ==== 54+0+16 (secure 0 0 0) 0x7fa91007f910 con 0x7fa91c0fdbb0 2024-02-23T00:45:06.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.645+0000 7fa921cd4700 10 client.30471.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:06.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.645+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_assign -1 1 2024-02-23T00:45:06.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.645+0000 7fa921cd4700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fa91c028540 0x7fa91c0fe910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:06.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.645+0000 7fa918ff9700 10 client.30471.objecter ms_handle_connect 0x7fa904062cc0 2024-02-23T00:45:06.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.645+0000 7fa91b7fe700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa904062cc0 0x7fa904065180 secure :-1 s=READY pgs=2973 cs=0 l=1 rev1=1 rx=0x7fa90c007b70 tx=0x7fa90c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:06.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.645+0000 7fa921cd4700 20 client.30471.objecter _get_session s=0x7fa91c106e60 osd=0 3 2024-02-23T00:45:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.645+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c106e60 osd=0 3 2024-02-23T00:45:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.645+0000 7fa921cd4700 20 client.30471.objecter _get_session s=0x7fa91c106e60 osd=0 3 2024-02-23T00:45:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.645+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_remove -1 1 2024-02-23T00:45:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.645+0000 7fa921cd4700 20 client.30471.objecter get_session s=0x7fa91c106e60 osd=0 3 2024-02-23T00:45:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.645+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_assign 0 1 2024-02-23T00:45:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.645+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c106e60 osd=0 4 2024-02-23T00:45:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.645+0000 7fa921cd4700 10 client.30471.objecter _send_command 1 2024-02-23T00:45:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.645+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa91c05fa60 con 0x7fa91c028540 2024-02-23T00:45:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.645+0000 7fa91bfff700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fa91c028540 0x7fa91c0fe910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:06.793 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:45:06.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.645+0000 7fa918ff9700 10 client.30471.objecter ms_handle_connect 0x7fa91c028540 2024-02-23T00:45:06.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.645+0000 7fa91bfff700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fa91c028540 0x7fa91c0fe910 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:06.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.646+0000 7fa918ff9700 1 -- 172.21.15.47:0/2316729014 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa91c05fa60 con 0x7fa91c028540 2024-02-23T00:45:06.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.646+0000 7fa918ff9700 10 client.30471.objecter ms_dispatch 0x7fa91c001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:45:06.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.646+0000 7fa918ff9700 10 client.30471.objecter _finish_command 1 = system:0 2024-02-23T00:45:06.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.646+0000 7fa918ff9700 20 client.30471.objecter put_session s=0x7fa91c106e60 osd=0 4 2024-02-23T00:45:06.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.646+0000 7fa918ff9700 15 client.30471.objecter _session_command_op_remove 0 1 2024-02-23T00:45:06.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.661+0000 7fa921cd4700 10 client.30471.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:45:06.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.661+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_assign -1 2 2024-02-23T00:45:06.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.661+0000 7fa921cd4700 20 client.30471.objecter _get_session s=0x7fa91c106e60 osd=0 3 2024-02-23T00:45:06.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.661+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c106e60 osd=0 3 2024-02-23T00:45:06.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.661+0000 7fa921cd4700 20 client.30471.objecter _get_session s=0x7fa91c106e60 osd=0 3 2024-02-23T00:45:06.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.661+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_remove -1 2 2024-02-23T00:45:06.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.661+0000 7fa921cd4700 20 client.30471.objecter get_session s=0x7fa91c106e60 osd=0 3 2024-02-23T00:45:06.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.661+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_assign 0 2 2024-02-23T00:45:06.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.661+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c106e60 osd=0 4 2024-02-23T00:45:06.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.661+0000 7fa921cd4700 10 client.30471.objecter _send_command 2 2024-02-23T00:45:06.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.661+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa91c05fbd0 con 0x7fa91c028540 2024-02-23T00:45:06.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.663+0000 7fa918ff9700 1 -- 172.21.15.47:0/2316729014 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa91c05fbd0 con 0x7fa91c028540 2024-02-23T00:45:06.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.663+0000 7fa918ff9700 10 client.30471.objecter ms_dispatch 0x7fa91c001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:45:06.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.663+0000 7fa918ff9700 10 client.30471.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:45:06.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.663+0000 7fa918ff9700 20 client.30471.objecter put_session s=0x7fa91c106e60 osd=0 4 2024-02-23T00:45:06.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.663+0000 7fa918ff9700 15 client.30471.objecter _session_command_op_remove 0 2 2024-02-23T00:45:06.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.663+0000 7fa921cd4700 10 client.30471.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:06.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.663+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_assign -1 3 2024-02-23T00:45:06.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.663+0000 7fa921cd4700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fa91c0ff480 0x7fa91c0ff860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:06.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.664+0000 7fa921cd4700 20 client.30471.objecter _get_session s=0x7fa91c0fee00 osd=1 3 2024-02-23T00:45:06.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.664+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c0fee00 osd=1 3 2024-02-23T00:45:06.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.664+0000 7fa921cd4700 20 client.30471.objecter _get_session s=0x7fa91c0fee00 osd=1 3 2024-02-23T00:45:06.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.664+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_remove -1 3 2024-02-23T00:45:06.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.664+0000 7fa921cd4700 20 client.30471.objecter get_session s=0x7fa91c0fee00 osd=1 3 2024-02-23T00:45:06.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.664+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_assign 1 3 2024-02-23T00:45:06.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.664+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c0fee00 osd=1 4 2024-02-23T00:45:06.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.664+0000 7fa921cd4700 10 client.30471.objecter _send_command 3 2024-02-23T00:45:06.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.664+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa91c0ffd50 con 0x7fa91c0ff480 2024-02-23T00:45:06.797 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:45:06.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.664+0000 7fa91b7fe700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fa91c0ff480 0x7fa91c0ff860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:06.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.664+0000 7fa91b7fe700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fa91c0ff480 0x7fa91c0ff860 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:06.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.664+0000 7fa918ff9700 10 client.30471.objecter ms_handle_connect 0x7fa91c0ff480 2024-02-23T00:45:06.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.665+0000 7fa918ff9700 1 -- 172.21.15.47:0/2316729014 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa91c0ffd50 con 0x7fa91c0ff480 2024-02-23T00:45:06.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.665+0000 7fa918ff9700 10 client.30471.objecter ms_dispatch 0x7fa91c001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:45:06.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.665+0000 7fa918ff9700 10 client.30471.objecter _finish_command 3 = system:0 2024-02-23T00:45:06.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.665+0000 7fa918ff9700 20 client.30471.objecter put_session s=0x7fa91c0fee00 osd=1 4 2024-02-23T00:45:06.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.665+0000 7fa918ff9700 15 client.30471.objecter _session_command_op_remove 1 3 2024-02-23T00:45:06.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.680+0000 7fa921cd4700 10 client.30471.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:45:06.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.680+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_assign -1 4 2024-02-23T00:45:06.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.680+0000 7fa921cd4700 20 client.30471.objecter _get_session s=0x7fa91c0fee00 osd=1 3 2024-02-23T00:45:06.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.680+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c0fee00 osd=1 3 2024-02-23T00:45:06.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.680+0000 7fa921cd4700 20 client.30471.objecter _get_session s=0x7fa91c0fee00 osd=1 3 2024-02-23T00:45:06.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.680+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_remove -1 4 2024-02-23T00:45:06.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.680+0000 7fa921cd4700 20 client.30471.objecter get_session s=0x7fa91c0fee00 osd=1 3 2024-02-23T00:45:06.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.680+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_assign 1 4 2024-02-23T00:45:06.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.680+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c0fee00 osd=1 4 2024-02-23T00:45:06.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.680+0000 7fa921cd4700 10 client.30471.objecter _send_command 4 2024-02-23T00:45:06.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.680+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa91c118fc0 con 0x7fa91c0ff480 2024-02-23T00:45:06.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.682+0000 7fa918ff9700 1 -- 172.21.15.47:0/2316729014 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa91c118fc0 con 0x7fa91c0ff480 2024-02-23T00:45:06.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.682+0000 7fa918ff9700 10 client.30471.objecter ms_dispatch 0x7fa91c001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:45:06.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.682+0000 7fa918ff9700 10 client.30471.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:45:06.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.682+0000 7fa918ff9700 20 client.30471.objecter put_session s=0x7fa91c0fee00 osd=1 4 2024-02-23T00:45:06.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.682+0000 7fa918ff9700 15 client.30471.objecter _session_command_op_remove 1 4 2024-02-23T00:45:06.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.682+0000 7fa921cd4700 10 client.30471.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:06.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.682+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_assign -1 5 2024-02-23T00:45:06.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.682+0000 7fa921cd4700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa91c0594d0 0x7fa91c11c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:06.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.682+0000 7fa921cd4700 20 client.30471.objecter _get_session s=0x7fa91c059360 osd=2 3 2024-02-23T00:45:06.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.682+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c059360 osd=2 3 2024-02-23T00:45:06.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.682+0000 7fa921cd4700 20 client.30471.objecter _get_session s=0x7fa91c059360 osd=2 3 2024-02-23T00:45:06.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.682+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_remove -1 5 2024-02-23T00:45:06.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.682+0000 7fa921cd4700 20 client.30471.objecter get_session s=0x7fa91c059360 osd=2 3 2024-02-23T00:45:06.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.682+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_assign 2 5 2024-02-23T00:45:06.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.682+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c059360 osd=2 4 2024-02-23T00:45:06.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.682+0000 7fa921cd4700 10 client.30471.objecter _send_command 5 2024-02-23T00:45:06.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.682+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa91c028270 con 0x7fa91c0594d0 2024-02-23T00:45:06.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.682+0000 7fa91affd700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa91c0594d0 0x7fa91c11c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:06.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.682+0000 7fa918ff9700 10 client.30471.objecter ms_handle_connect 0x7fa91c0594d0 2024-02-23T00:45:06.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.682+0000 7fa91affd700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa91c0594d0 0x7fa91c11c320 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:06.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.683+0000 7fa918ff9700 1 -- 172.21.15.47:0/2316729014 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa91c028270 con 0x7fa91c0594d0 2024-02-23T00:45:06.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.683+0000 7fa918ff9700 10 client.30471.objecter ms_dispatch 0x7fa91c001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:45:06.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.683+0000 7fa918ff9700 10 client.30471.objecter _finish_command 5 = system:0 2024-02-23T00:45:06.801 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:45:06.801 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:45:06.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.683+0000 7fa918ff9700 20 client.30471.objecter put_session s=0x7fa91c059360 osd=2 4 2024-02-23T00:45:06.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.683+0000 7fa918ff9700 15 client.30471.objecter _session_command_op_remove 2 5 2024-02-23T00:45:06.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.699+0000 7fa921cd4700 10 client.30471.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:45:06.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.699+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_assign -1 6 2024-02-23T00:45:06.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.699+0000 7fa921cd4700 20 client.30471.objecter _get_session s=0x7fa91c059360 osd=2 3 2024-02-23T00:45:06.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.699+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c059360 osd=2 3 2024-02-23T00:45:06.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.699+0000 7fa921cd4700 20 client.30471.objecter _get_session s=0x7fa91c059360 osd=2 3 2024-02-23T00:45:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.699+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_remove -1 6 2024-02-23T00:45:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.699+0000 7fa921cd4700 20 client.30471.objecter get_session s=0x7fa91c059360 osd=2 3 2024-02-23T00:45:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.699+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_assign 2 6 2024-02-23T00:45:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.699+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c059360 osd=2 4 2024-02-23T00:45:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.699+0000 7fa921cd4700 10 client.30471.objecter _send_command 6 2024-02-23T00:45:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.699+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa91c027600 con 0x7fa91c0594d0 2024-02-23T00:45:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.700+0000 7fa918ff9700 1 -- 172.21.15.47:0/2316729014 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa91c027600 con 0x7fa91c0594d0 2024-02-23T00:45:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.700+0000 7fa918ff9700 10 client.30471.objecter ms_dispatch 0x7fa91c001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:45:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.700+0000 7fa918ff9700 10 client.30471.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:45:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.700+0000 7fa918ff9700 20 client.30471.objecter put_session s=0x7fa91c059360 osd=2 4 2024-02-23T00:45:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.700+0000 7fa918ff9700 15 client.30471.objecter _session_command_op_remove 2 6 2024-02-23T00:45:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.700+0000 7fa921cd4700 10 client.30471.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.700+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_assign -1 7 2024-02-23T00:45:06.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.700+0000 7fa921cd4700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fa91c11d990 0x7fa91c11fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:06.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.700+0000 7fa921cd4700 20 client.30471.objecter _get_session s=0x7fa91c119960 osd=3 3 2024-02-23T00:45:06.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.700+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c119960 osd=3 3 2024-02-23T00:45:06.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.700+0000 7fa921cd4700 20 client.30471.objecter _get_session s=0x7fa91c119960 osd=3 3 2024-02-23T00:45:06.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.700+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_remove -1 7 2024-02-23T00:45:06.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.700+0000 7fa921cd4700 20 client.30471.objecter get_session s=0x7fa91c119960 osd=3 3 2024-02-23T00:45:06.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.700+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_assign 3 7 2024-02-23T00:45:06.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.700+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c119960 osd=3 4 2024-02-23T00:45:06.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.700+0000 7fa921cd4700 10 client.30471.objecter _send_command 7 2024-02-23T00:45:06.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.700+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa91c11a110 con 0x7fa91c11d990 2024-02-23T00:45:06.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.701+0000 7fa91bfff700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fa91c11d990 0x7fa91c11fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:06.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.701+0000 7fa918ff9700 10 client.30471.objecter ms_handle_connect 0x7fa91c11d990 2024-02-23T00:45:06.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.701+0000 7fa91bfff700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fa91c11d990 0x7fa91c11fd80 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:06.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.701+0000 7fa918ff9700 1 -- 172.21.15.47:0/2316729014 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa91c11a110 con 0x7fa91c11d990 2024-02-23T00:45:06.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.701+0000 7fa918ff9700 10 client.30471.objecter ms_dispatch 0x7fa91c001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:45:06.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.701+0000 7fa918ff9700 10 client.30471.objecter _finish_command 7 = system:0 2024-02-23T00:45:06.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.701+0000 7fa918ff9700 20 client.30471.objecter put_session s=0x7fa91c119960 osd=3 4 2024-02-23T00:45:06.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.701+0000 7fa918ff9700 15 client.30471.objecter _session_command_op_remove 3 7 2024-02-23T00:45:06.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.717+0000 7fa921cd4700 10 client.30471.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:45:06.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.717+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_assign -1 8 2024-02-23T00:45:06.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.717+0000 7fa921cd4700 20 client.30471.objecter _get_session s=0x7fa91c119960 osd=3 3 2024-02-23T00:45:06.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.717+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c119960 osd=3 3 2024-02-23T00:45:06.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.717+0000 7fa921cd4700 20 client.30471.objecter _get_session s=0x7fa91c119960 osd=3 3 2024-02-23T00:45:06.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.717+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_remove -1 8 2024-02-23T00:45:06.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.717+0000 7fa921cd4700 20 client.30471.objecter get_session s=0x7fa91c119960 osd=3 3 2024-02-23T00:45:06.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.717+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_assign 3 8 2024-02-23T00:45:06.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.717+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c119960 osd=3 4 2024-02-23T00:45:06.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.717+0000 7fa921cd4700 10 client.30471.objecter _send_command 8 2024-02-23T00:45:06.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.717+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa91c1193d0 con 0x7fa91c11d990 2024-02-23T00:45:06.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.718+0000 7fa918ff9700 1 -- 172.21.15.47:0/2316729014 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa91c1193d0 con 0x7fa91c11d990 2024-02-23T00:45:06.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.718+0000 7fa918ff9700 10 client.30471.objecter ms_dispatch 0x7fa91c001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:45:06.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.718+0000 7fa918ff9700 10 client.30471.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:45:06.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.718+0000 7fa918ff9700 20 client.30471.objecter put_session s=0x7fa91c119960 osd=3 4 2024-02-23T00:45:06.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.718+0000 7fa918ff9700 15 client.30471.objecter _session_command_op_remove 3 8 2024-02-23T00:45:06.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.719+0000 7fa921cd4700 10 client.30471.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:06.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.719+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_assign -1 9 2024-02-23T00:45:06.806 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:45:06.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.719+0000 7fa921cd4700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fa91c11d4f0 0x7fa91c123400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:06.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.719+0000 7fa921cd4700 20 client.30471.objecter _get_session s=0x7fa91c119190 osd=4 3 2024-02-23T00:45:06.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.719+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c119190 osd=4 3 2024-02-23T00:45:06.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.719+0000 7fa921cd4700 20 client.30471.objecter _get_session s=0x7fa91c119190 osd=4 3 2024-02-23T00:45:06.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.719+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_remove -1 9 2024-02-23T00:45:06.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.719+0000 7fa921cd4700 20 client.30471.objecter get_session s=0x7fa91c119190 osd=4 3 2024-02-23T00:45:06.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.719+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_assign 4 9 2024-02-23T00:45:06.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.719+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c119190 osd=4 4 2024-02-23T00:45:06.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.719+0000 7fa921cd4700 10 client.30471.objecter _send_command 9 2024-02-23T00:45:06.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.719+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa91c119390 con 0x7fa91c11d4f0 2024-02-23T00:45:06.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.719+0000 7fa91b7fe700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fa91c11d4f0 0x7fa91c123400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:06.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.719+0000 7fa91b7fe700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fa91c11d4f0 0x7fa91c123400 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:06.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.719+0000 7fa918ff9700 10 client.30471.objecter ms_handle_connect 0x7fa91c11d4f0 2024-02-23T00:45:06.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.720+0000 7fa918ff9700 1 -- 172.21.15.47:0/2316729014 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa91c119390 con 0x7fa91c11d4f0 2024-02-23T00:45:06.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.720+0000 7fa918ff9700 10 client.30471.objecter ms_dispatch 0x7fa91c001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:45:06.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.720+0000 7fa918ff9700 10 client.30471.objecter _finish_command 9 = system:0 2024-02-23T00:45:06.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.720+0000 7fa918ff9700 20 client.30471.objecter put_session s=0x7fa91c119190 osd=4 4 2024-02-23T00:45:06.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.720+0000 7fa918ff9700 15 client.30471.objecter _session_command_op_remove 4 9 2024-02-23T00:45:06.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.736+0000 7fa921cd4700 10 client.30471.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:45:06.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.736+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_assign -1 10 2024-02-23T00:45:06.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.736+0000 7fa921cd4700 20 client.30471.objecter _get_session s=0x7fa91c119190 osd=4 3 2024-02-23T00:45:06.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.736+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c119190 osd=4 3 2024-02-23T00:45:06.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.736+0000 7fa921cd4700 20 client.30471.objecter _get_session s=0x7fa91c119190 osd=4 3 2024-02-23T00:45:06.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.736+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_remove -1 10 2024-02-23T00:45:06.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.736+0000 7fa921cd4700 20 client.30471.objecter get_session s=0x7fa91c119190 osd=4 3 2024-02-23T00:45:06.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.736+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_assign 4 10 2024-02-23T00:45:06.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.736+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c119190 osd=4 4 2024-02-23T00:45:06.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.736+0000 7fa921cd4700 10 client.30471.objecter _send_command 10 2024-02-23T00:45:06.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.736+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa91c119390 con 0x7fa91c11d4f0 2024-02-23T00:45:06.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.738+0000 7fa918ff9700 1 -- 172.21.15.47:0/2316729014 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa91c119390 con 0x7fa91c11d4f0 2024-02-23T00:45:06.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.738+0000 7fa918ff9700 10 client.30471.objecter ms_dispatch 0x7fa91c001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:45:06.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.738+0000 7fa918ff9700 10 client.30471.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:45:06.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.738+0000 7fa918ff9700 20 client.30471.objecter put_session s=0x7fa91c119190 osd=4 4 2024-02-23T00:45:06.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.738+0000 7fa918ff9700 15 client.30471.objecter _session_command_op_remove 4 10 2024-02-23T00:45:06.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.738+0000 7fa921cd4700 10 client.30471.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:06.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.739+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_assign -1 11 2024-02-23T00:45:06.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.739+0000 7fa921cd4700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fa91c123e00 0x7fa91c124280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:06.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.739+0000 7fa921cd4700 20 client.30471.objecter _get_session s=0x7fa91c119390 osd=5 3 2024-02-23T00:45:06.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.739+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c119390 osd=5 3 2024-02-23T00:45:06.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.739+0000 7fa921cd4700 20 client.30471.objecter _get_session s=0x7fa91c119390 osd=5 3 2024-02-23T00:45:06.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.739+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_remove -1 11 2024-02-23T00:45:06.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.739+0000 7fa921cd4700 20 client.30471.objecter get_session s=0x7fa91c119390 osd=5 3 2024-02-23T00:45:06.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.739+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_assign 5 11 2024-02-23T00:45:06.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.739+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c119390 osd=5 4 2024-02-23T00:45:06.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.739+0000 7fa921cd4700 10 client.30471.objecter _send_command 11 2024-02-23T00:45:06.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.739+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa91c129210 con 0x7fa91c123e00 2024-02-23T00:45:06.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.739+0000 7fa91affd700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fa91c123e00 0x7fa91c124280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:06.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.739+0000 7fa91affd700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fa91c123e00 0x7fa91c124280 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:06.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.739+0000 7fa918ff9700 10 client.30471.objecter ms_handle_connect 0x7fa91c123e00 2024-02-23T00:45:06.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.740+0000 7fa918ff9700 1 -- 172.21.15.47:0/2316729014 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa91c129210 con 0x7fa91c123e00 2024-02-23T00:45:06.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.740+0000 7fa918ff9700 10 client.30471.objecter ms_dispatch 0x7fa91c001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:45:06.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.740+0000 7fa918ff9700 10 client.30471.objecter _finish_command 11 = system:0 2024-02-23T00:45:06.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.740+0000 7fa918ff9700 20 client.30471.objecter put_session s=0x7fa91c119390 osd=5 4 2024-02-23T00:45:06.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.740+0000 7fa918ff9700 15 client.30471.objecter _session_command_op_remove 5 11 2024-02-23T00:45:06.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.755+0000 7fa921cd4700 10 client.30471.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:45:06.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.755+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_assign -1 12 2024-02-23T00:45:06.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.755+0000 7fa921cd4700 20 client.30471.objecter _get_session s=0x7fa91c119390 osd=5 3 2024-02-23T00:45:06.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.755+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c119390 osd=5 3 2024-02-23T00:45:06.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.755+0000 7fa921cd4700 20 client.30471.objecter _get_session s=0x7fa91c119390 osd=5 3 2024-02-23T00:45:06.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.756+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_remove -1 12 2024-02-23T00:45:06.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.756+0000 7fa921cd4700 20 client.30471.objecter get_session s=0x7fa91c119390 osd=5 3 2024-02-23T00:45:06.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.756+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_assign 5 12 2024-02-23T00:45:06.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.756+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c119390 osd=5 4 2024-02-23T00:45:06.811 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:45:06.812 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:45:06.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.756+0000 7fa921cd4700 10 client.30471.objecter _send_command 12 2024-02-23T00:45:06.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.756+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa91c124880 con 0x7fa91c123e00 2024-02-23T00:45:06.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.757+0000 7fa918ff9700 1 -- 172.21.15.47:0/2316729014 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa91c124880 con 0x7fa91c123e00 2024-02-23T00:45:06.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.757+0000 7fa918ff9700 10 client.30471.objecter ms_dispatch 0x7fa91c001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:45:06.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.757+0000 7fa918ff9700 10 client.30471.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:45:06.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.757+0000 7fa918ff9700 20 client.30471.objecter put_session s=0x7fa91c119390 osd=5 4 2024-02-23T00:45:06.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.757+0000 7fa918ff9700 15 client.30471.objecter _session_command_op_remove 5 12 2024-02-23T00:45:06.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.758+0000 7fa921cd4700 10 client.30471.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:06.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.758+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_assign -1 13 2024-02-23T00:45:06.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.758+0000 7fa921cd4700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fa91c124f00 0x7fa91c125360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:06.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.758+0000 7fa921cd4700 20 client.30471.objecter _get_session s=0x7fa91c124880 osd=6 3 2024-02-23T00:45:06.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.758+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c124880 osd=6 3 2024-02-23T00:45:06.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.758+0000 7fa921cd4700 20 client.30471.objecter _get_session s=0x7fa91c124880 osd=6 3 2024-02-23T00:45:06.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.758+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_remove -1 13 2024-02-23T00:45:06.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.758+0000 7fa921cd4700 20 client.30471.objecter get_session s=0x7fa91c124880 osd=6 3 2024-02-23T00:45:06.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.758+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_assign 6 13 2024-02-23T00:45:06.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.758+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c124880 osd=6 4 2024-02-23T00:45:06.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.758+0000 7fa921cd4700 10 client.30471.objecter _send_command 13 2024-02-23T00:45:06.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.758+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa91c125a10 con 0x7fa91c124f00 2024-02-23T00:45:06.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.758+0000 7fa91bfff700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fa91c124f00 0x7fa91c125360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:06.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.758+0000 7fa918ff9700 10 client.30471.objecter ms_handle_connect 0x7fa91c124f00 2024-02-23T00:45:06.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.758+0000 7fa91bfff700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fa91c124f00 0x7fa91c125360 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:06.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.759+0000 7fa918ff9700 1 -- 172.21.15.47:0/2316729014 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa91c125a10 con 0x7fa91c124f00 2024-02-23T00:45:06.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.759+0000 7fa918ff9700 10 client.30471.objecter ms_dispatch 0x7fa91c001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:45:06.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.759+0000 7fa918ff9700 10 client.30471.objecter _finish_command 13 = system:0 2024-02-23T00:45:06.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.759+0000 7fa918ff9700 20 client.30471.objecter put_session s=0x7fa91c124880 osd=6 4 2024-02-23T00:45:06.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.759+0000 7fa918ff9700 15 client.30471.objecter _session_command_op_remove 6 13 2024-02-23T00:45:06.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.774+0000 7fa921cd4700 10 client.30471.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:45:06.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.774+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_assign -1 14 2024-02-23T00:45:06.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.774+0000 7fa921cd4700 20 client.30471.objecter _get_session s=0x7fa91c124880 osd=6 3 2024-02-23T00:45:06.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.774+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c124880 osd=6 3 2024-02-23T00:45:06.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.774+0000 7fa921cd4700 20 client.30471.objecter _get_session s=0x7fa91c124880 osd=6 3 2024-02-23T00:45:06.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.774+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_remove -1 14 2024-02-23T00:45:06.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.774+0000 7fa921cd4700 20 client.30471.objecter get_session s=0x7fa91c124880 osd=6 3 2024-02-23T00:45:06.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.774+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_assign 6 14 2024-02-23T00:45:06.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.774+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c124880 osd=6 4 2024-02-23T00:45:06.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.774+0000 7fa921cd4700 10 client.30471.objecter _send_command 14 2024-02-23T00:45:06.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.774+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa91c125a10 con 0x7fa91c124f00 2024-02-23T00:45:06.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.775+0000 7fa918ff9700 1 -- 172.21.15.47:0/2316729014 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa91c125a10 con 0x7fa91c124f00 2024-02-23T00:45:06.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.775+0000 7fa918ff9700 10 client.30471.objecter ms_dispatch 0x7fa91c001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:45:06.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.775+0000 7fa918ff9700 10 client.30471.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:45:06.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.775+0000 7fa918ff9700 20 client.30471.objecter put_session s=0x7fa91c124880 osd=6 4 2024-02-23T00:45:06.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.775+0000 7fa918ff9700 15 client.30471.objecter _session_command_op_remove 6 14 2024-02-23T00:45:06.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.776+0000 7fa921cd4700 10 client.30471.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:06.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.776+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_assign -1 15 2024-02-23T00:45:06.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.776+0000 7fa921cd4700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fa91c12a5b0 0x7fa91c12c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:06.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.776+0000 7fa921cd4700 20 client.30471.objecter _get_session s=0x7fa91c125a10 osd=7 3 2024-02-23T00:45:06.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.776+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c125a10 osd=7 3 2024-02-23T00:45:06.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.776+0000 7fa921cd4700 20 client.30471.objecter _get_session s=0x7fa91c125a10 osd=7 3 2024-02-23T00:45:06.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.776+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_remove -1 15 2024-02-23T00:45:06.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.776+0000 7fa921cd4700 20 client.30471.objecter get_session s=0x7fa91c125a10 osd=7 3 2024-02-23T00:45:06.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.776+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_assign 7 15 2024-02-23T00:45:06.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.776+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c125a10 osd=7 4 2024-02-23T00:45:06.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.776+0000 7fa921cd4700 10 client.30471.objecter _send_command 15 2024-02-23T00:45:06.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.776+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa91c12f5f0 con 0x7fa91c12a5b0 2024-02-23T00:45:06.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.776+0000 7fa91b7fe700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fa91c12a5b0 0x7fa91c12c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:06.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.776+0000 7fa918ff9700 10 client.30471.objecter ms_handle_connect 0x7fa91c12a5b0 2024-02-23T00:45:06.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.776+0000 7fa91b7fe700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fa91c12a5b0 0x7fa91c12c9a0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:06.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.777+0000 7fa918ff9700 1 -- 172.21.15.47:0/2316729014 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa91c12f5f0 con 0x7fa91c12a5b0 2024-02-23T00:45:06.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.777+0000 7fa918ff9700 10 client.30471.objecter ms_dispatch 0x7fa91c001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:45:06.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.777+0000 7fa918ff9700 10 client.30471.objecter _finish_command 15 = system:0 2024-02-23T00:45:06.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.777+0000 7fa918ff9700 20 client.30471.objecter put_session s=0x7fa91c125a10 osd=7 4 2024-02-23T00:45:06.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.777+0000 7fa918ff9700 15 client.30471.objecter _session_command_op_remove 7 15 2024-02-23T00:45:06.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.793+0000 7fa921cd4700 10 client.30471.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:45:06.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.793+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_assign -1 16 2024-02-23T00:45:06.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.793+0000 7fa921cd4700 20 client.30471.objecter _get_session s=0x7fa91c125a10 osd=7 3 2024-02-23T00:45:06.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.793+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c125a10 osd=7 3 2024-02-23T00:45:06.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.793+0000 7fa921cd4700 20 client.30471.objecter _get_session s=0x7fa91c125a10 osd=7 3 2024-02-23T00:45:06.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.793+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_remove -1 16 2024-02-23T00:45:06.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.793+0000 7fa921cd4700 20 client.30471.objecter get_session s=0x7fa91c125a10 osd=7 3 2024-02-23T00:45:06.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.793+0000 7fa921cd4700 15 client.30471.objecter _session_command_op_assign 7 16 2024-02-23T00:45:06.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.793+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c125a10 osd=7 4 2024-02-23T00:45:06.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.793+0000 7fa921cd4700 10 client.30471.objecter _send_command 16 2024-02-23T00:45:06.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.793+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa91c12ce90 con 0x7fa91c12a5b0 2024-02-23T00:45:06.820 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:45:06.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.795+0000 7fa918ff9700 1 -- 172.21.15.47:0/2316729014 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa91c12ce90 con 0x7fa91c12a5b0 2024-02-23T00:45:06.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.795+0000 7fa918ff9700 10 client.30471.objecter ms_dispatch 0x7fa91c001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:45:06.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.795+0000 7fa918ff9700 10 client.30471.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:45:06.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.795+0000 7fa918ff9700 20 client.30471.objecter put_session s=0x7fa91c125a10 osd=7 4 2024-02-23T00:45:06.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.795+0000 7fa918ff9700 15 client.30471.objecter _session_command_op_remove 7 16 2024-02-23T00:45:06.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.795+0000 7fa921cd4700 10 librados: watch_flush enter 2024-02-23T00:45:06.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.795+0000 7fa921cd4700 10 librados: watch_flush exit 2024-02-23T00:45:06.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 10 client.30471.objecter close_session for osd.0 2024-02-23T00:45:06.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fa91c028540 msgr2=0x7fa91c0fe910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:06.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fa91c028540 0x7fa91c0fe910 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:06.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c106e60 osd=0 1 2024-02-23T00:45:06.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 10 client.30471.objecter close_session for osd.1 2024-02-23T00:45:06.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fa91c0ff480 msgr2=0x7fa91c0ff860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:06.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fa91c0ff480 0x7fa91c0ff860 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:06.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c0fee00 osd=1 1 2024-02-23T00:45:06.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 10 client.30471.objecter close_session for osd.2 2024-02-23T00:45:06.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa91c0594d0 msgr2=0x7fa91c11c320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:06.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa91c0594d0 0x7fa91c11c320 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:06.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c059360 osd=2 1 2024-02-23T00:45:06.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 10 client.30471.objecter close_session for osd.3 2024-02-23T00:45:06.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fa91c11d990 msgr2=0x7fa91c11fd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:06.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fa91c11d990 0x7fa91c11fd80 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:06.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c119960 osd=3 1 2024-02-23T00:45:06.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 10 client.30471.objecter close_session for osd.4 2024-02-23T00:45:06.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fa91c11d4f0 msgr2=0x7fa91c123400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:06.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fa91c11d4f0 0x7fa91c123400 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:06.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c119190 osd=4 1 2024-02-23T00:45:06.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 10 client.30471.objecter close_session for osd.5 2024-02-23T00:45:06.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fa91c123e00 msgr2=0x7fa91c124280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:06.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fa91c123e00 0x7fa91c124280 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:06.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c119390 osd=5 1 2024-02-23T00:45:06.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 10 client.30471.objecter close_session for osd.6 2024-02-23T00:45:06.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fa91c124f00 msgr2=0x7fa91c125360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fa91c124f00 0x7fa91c125360 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c124880 osd=6 1 2024-02-23T00:45:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 10 client.30471.objecter close_session for osd.7 2024-02-23T00:45:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fa91c12a5b0 msgr2=0x7fa91c12c9a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fa91c12a5b0 0x7fa91c12c9a0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 20 client.30471.objecter put_session s=0x7fa91c125a10 osd=7 1 2024-02-23T00:45:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 20 client.30471.objecter shutdown clearing up homeless session... 2024-02-23T00:45:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa921cd4700 10 client.30471.objecter successfully canceled tick 2024-02-23T00:45:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa91bfff700 1 -- 172.21.15.47:0/2316729014 reap_dead start 2024-02-23T00:45:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa91bfff700 1 -- 172.21.15.47:0/2316729014 reap_dead start 2024-02-23T00:45:06.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa91bfff700 1 -- 172.21.15.47:0/2316729014 reap_dead start 2024-02-23T00:45:06.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa91bfff700 1 -- 172.21.15.47:0/2316729014 reap_dead start 2024-02-23T00:45:06.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.796+0000 7fa91bfff700 1 -- 172.21.15.47:0/2316729014 reap_dead start 2024-02-23T00:45:06.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.797+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa904062cc0 msgr2=0x7fa904065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:06.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.797+0000 7fa921cd4700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa904062cc0 0x7fa904065180 secure :-1 s=READY pgs=2973 cs=0 l=1 rev1=1 rx=0x7fa90c007b70 tx=0x7fa90c00b040).stop 2024-02-23T00:45:06.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.797+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa91c0fdbb0 msgr2=0x7fa91c10e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:06.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.797+0000 7fa921cd4700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa91c0fdbb0 0x7fa91c10e4d0 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 rx=0x7fa910009da0 tx=0x7fa910009cb0).stop 2024-02-23T00:45:06.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.800+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 shutdown_connections 2024-02-23T00:45:06.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.800+0000 7fa921cd4700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa91c0fdbb0 0x7fa91c10e4d0 unknown :-1 s=CLOSED pgs=2467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:06.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.800+0000 7fa921cd4700 1 --2- 172.21.15.47:0/2316729014 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa904062cc0 0x7fa904065180 unknown :-1 s=CLOSED pgs=2973 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:06.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.801+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 >> 172.21.15.47:0/2316729014 conn(0x7fa91c0f3800 msgr2=0x7fa91c0fc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:45:06.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.801+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 shutdown_connections 2024-02-23T00:45:06.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.801+0000 7fa921cd4700 1 -- 172.21.15.47:0/2316729014 wait complete. 2024-02-23T00:45:06.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:06.801+0000 7fa921cd4700 1 librados: shutdown 2024-02-23T00:45:06.826 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:45:06.826 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:45:06.826 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:45:06.826 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:45:06.826 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:45:06.826 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:45:06.826 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:45:06.826 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:45:06.826 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:04.581799+0000 osd.1 (osd.1) 119 : cluster [DBG] 6.1 scrub starts 2024-02-23T00:45:06.827 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:04.585170+0000 osd.1 (osd.1) 120 : cluster [DBG] 6.1 scrub ok 2024-02-23T00:45:06.827 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:04.620755+0000 osd.6 (osd.6) 95 : cluster [DBG] 18.14 scrub starts 2024-02-23T00:45:06.827 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:04.621710+0000 osd.6 (osd.6) 96 : cluster [DBG] 18.14 scrub ok 2024-02-23T00:45:06.827 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:04.670959+0000 osd.0 (osd.0) 107 : cluster [DBG] 4.1e scrub starts 2024-02-23T00:45:06.827 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:04.672532+0000 osd.0 (osd.0) 108 : cluster [DBG] 4.1e scrub ok 2024-02-23T00:45:06.827 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:05.152531+0000 mon.c (mon.2) 151 : audit [INF] from='client.? 172.21.15.47:0/2440014932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi047-82378-32"}]: dispatch 2024-02-23T00:45:06.827 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:05.234351+0000 mgr.y (mgr.14517) 257 : cluster [DBG] pgmap v325: 282 pgs: 8 creating+activating, 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 256 active+clean; 9.5 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-23T00:45:06.827 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:05.290360+0000 osd.4 (osd.4) 103 : cluster [DBG] 4.1 scrub starts 2024-02-23T00:45:06.827 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:05.291828+0000 osd.4 (osd.4) 104 : cluster [DBG] 4.1 scrub ok 2024-02-23T00:45:06.827 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:05.489566+0000 osd.2 (osd.2) 101 : cluster [DBG] 18.12 scrub starts 2024-02-23T00:45:06.827 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:05.490527+0000 osd.2 (osd.2) 102 : cluster [DBG] 18.12 scrub ok 2024-02-23T00:45:06.827 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:05.575661+0000 mon.a (mon.0) 1630 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi047-82378-32"}]: dispatch 2024-02-23T00:45:06.827 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:05.580381+0000 mon.c (mon.2) 152 : audit [INF] from='client.? 172.21.15.47:0/2440014932' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi047-82378-32"}]: dispatch 2024-02-23T00:45:06.828 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:05.582217+0000 mon.a (mon.0) 1631 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi047-82378-32"}]: dispatch 2024-02-23T00:45:06.828 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:05.582734+0000 mon.c (mon.2) 153 : audit [INF] from='client.? 172.21.15.47:0/2440014932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi047-82378-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:06.828 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:05.582997+0000 mon.a (mon.0) 1632 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi047-82378-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:06.828 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:04.581799+0000 osd.1 (osd.1) 119 : cluster [DBG] 6.1 scrub starts 2024-02-23T00:45:06.828 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:04.585170+0000 osd.1 (osd.1) 120 : cluster [DBG] 6.1 scrub ok 2024-02-23T00:45:06.828 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:04.620755+0000 osd.6 (osd.6) 95 : cluster [DBG] 18.14 scrub starts 2024-02-23T00:45:06.828 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:04.621710+0000 osd.6 (osd.6) 96 : cluster [DBG] 18.14 scrub ok 2024-02-23T00:45:06.828 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:04.670959+0000 osd.0 (osd.0) 107 : cluster [DBG] 4.1e scrub starts 2024-02-23T00:45:06.829 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:04.672532+0000 osd.0 (osd.0) 108 : cluster [DBG] 4.1e scrub ok 2024-02-23T00:45:06.829 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:05.152531+0000 mon.c (mon.2) 151 : audit [INF] from='client.? 172.21.15.47:0/2440014932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi047-82378-32"}]: dispatch 2024-02-23T00:45:06.829 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:05.234351+0000 mgr.y (mgr.14517) 257 : cluster [DBG] pgmap v325: 282 pgs: 8 creating+activating, 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 256 active+clean; 9.5 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-23T00:45:06.829 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:05.290360+0000 osd.4 (osd.4) 103 : cluster [DBG] 4.1 scrub starts 2024-02-23T00:45:06.829 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:05.291828+0000 osd.4 (osd.4) 104 : cluster [DBG] 4.1 scrub ok 2024-02-23T00:45:06.829 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:05.489566+0000 osd.2 (osd.2) 101 : cluster [DBG] 18.12 scrub starts 2024-02-23T00:45:06.829 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:05.490527+0000 osd.2 (osd.2) 102 : cluster [DBG] 18.12 scrub ok 2024-02-23T00:45:06.829 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:05.575661+0000 mon.a (mon.0) 1630 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi047-82378-32"}]: dispatch 2024-02-23T00:45:06.829 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:05.580381+0000 mon.c (mon.2) 152 : audit [INF] from='client.? 172.21.15.47:0/2440014932' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi047-82378-32"}]: dispatch 2024-02-23T00:45:06.829 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:05.582217+0000 mon.a (mon.0) 1631 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi047-82378-32"}]: dispatch 2024-02-23T00:45:06.829 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:05.582734+0000 mon.c (mon.2) 153 : audit [INF] from='client.? 172.21.15.47:0/2440014932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi047-82378-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:06.830 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:05.582997+0000 mon.a (mon.0) 1632 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi047-82378-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:06.830 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:05.800135+0000 mon.b (mon.1) 293 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:06.830 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:06.036632+0000 osd.3 (osd.3) 123 : cluster [DBG] 4.19 scrub starts 2024-02-23T00:45:06.830 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:06.038366+0000 osd.3 (osd.3) 124 : cluster [DBG] 4.19 scrub ok 2024-02-23T00:45:06.830 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:06.125182+0000 mon.a (mon.0) 1633 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi047-82378-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:06.830 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:06.125270+0000 mon.a (mon.0) 1634 : cluster [DBG] osdmap e234: 8 total, 8 up, 8 in 2024-02-23T00:45:06.830 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:06.134020+0000 mon.c (mon.2) 154 : audit [INF] from='client.? 172.21.15.47:0/2440014932' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi047-82378-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi047-82378-32"}]: dispatch 2024-02-23T00:45:06.830 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:06.134672+0000 mon.a (mon.0) 1635 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi047-82378-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi047-82378-32"}]: dispatch 2024-02-23T00:45:06.830 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:06.138188+0000 mon.a (mon.0) 1636 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:06.831 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:05.800135+0000 mon.b (mon.1) 293 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:06.831 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:06.036632+0000 osd.3 (osd.3) 123 : cluster [DBG] 4.19 scrub starts 2024-02-23T00:45:06.831 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:06.038366+0000 osd.3 (osd.3) 124 : cluster [DBG] 4.19 scrub ok 2024-02-23T00:45:06.831 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:06.125182+0000 mon.a (mon.0) 1633 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi047-82378-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:06.831 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:06.125270+0000 mon.a (mon.0) 1634 : cluster [DBG] osdmap e234: 8 total, 8 up, 8 in 2024-02-23T00:45:06.831 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:06.134020+0000 mon.c (mon.2) 154 : audit [INF] from='client.? 172.21.15.47:0/2440014932' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi047-82378-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi047-82378-32"}]: dispatch 2024-02-23T00:45:06.831 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:06.134672+0000 mon.a (mon.0) 1635 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi047-82378-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi047-82378-32"}]: dispatch 2024-02-23T00:45:06.831 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:06.138188+0000 mon.a (mon.0) 1636 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:06.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:04.581799+0000 osd.1 (osd.1) 119 : cluster [DBG] 6.1 scrub starts 2024-02-23T00:45:06.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:04.585170+0000 osd.1 (osd.1) 120 : cluster [DBG] 6.1 scrub ok 2024-02-23T00:45:06.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:04.620755+0000 osd.6 (osd.6) 95 : cluster [DBG] 18.14 scrub starts 2024-02-23T00:45:06.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:04.621710+0000 osd.6 (osd.6) 96 : cluster [DBG] 18.14 scrub ok 2024-02-23T00:45:06.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:04.670959+0000 osd.0 (osd.0) 107 : cluster [DBG] 4.1e scrub starts 2024-02-23T00:45:06.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:04.672532+0000 osd.0 (osd.0) 108 : cluster [DBG] 4.1e scrub ok 2024-02-23T00:45:06.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:05.152531+0000 mon.c (mon.2) 151 : audit [INF] from='client.? 172.21.15.47:0/2440014932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi047-82378-32"}]: dispatch 2024-02-23T00:45:06.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:05.234351+0000 mgr.y (mgr.14517) 257 : cluster [DBG] pgmap v325: 282 pgs: 8 creating+activating, 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 256 active+clean; 9.5 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-23T00:45:06.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:05.290360+0000 osd.4 (osd.4) 103 : cluster [DBG] 4.1 scrub starts 2024-02-23T00:45:06.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:05.291828+0000 osd.4 (osd.4) 104 : cluster [DBG] 4.1 scrub ok 2024-02-23T00:45:06.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:05.489566+0000 osd.2 (osd.2) 101 : cluster [DBG] 18.12 scrub starts 2024-02-23T00:45:06.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:05.490527+0000 osd.2 (osd.2) 102 : cluster [DBG] 18.12 scrub ok 2024-02-23T00:45:06.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:05.575661+0000 mon.a (mon.0) 1630 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi047-82378-32"}]: dispatch 2024-02-23T00:45:06.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:05.580381+0000 mon.c (mon.2) 152 : audit [INF] from='client.? 172.21.15.47:0/2440014932' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi047-82378-32"}]: dispatch 2024-02-23T00:45:06.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:05.582217+0000 mon.a (mon.0) 1631 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi047-82378-32"}]: dispatch 2024-02-23T00:45:06.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:05.582734+0000 mon.c (mon.2) 153 : audit [INF] from='client.? 172.21.15.47:0/2440014932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi047-82378-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:06.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:05.582997+0000 mon.a (mon.0) 1632 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi047-82378-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:06.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:05.800135+0000 mon.b (mon.1) 293 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:06.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:06.036632+0000 osd.3 (osd.3) 123 : cluster [DBG] 4.19 scrub starts 2024-02-23T00:45:06.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:06.038366+0000 osd.3 (osd.3) 124 : cluster [DBG] 4.19 scrub ok 2024-02-23T00:45:06.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:06.125182+0000 mon.a (mon.0) 1633 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi047-82378-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:06.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:06.125270+0000 mon.a (mon.0) 1634 : cluster [DBG] osdmap e234: 8 total, 8 up, 8 in 2024-02-23T00:45:06.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:06.134020+0000 mon.c (mon.2) 154 : audit [INF] from='client.? 172.21.15.47:0/2440014932' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi047-82378-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi047-82378-32"}]: dispatch 2024-02-23T00:45:06.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:06.134672+0000 mon.a (mon.0) 1635 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi047-82378-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi047-82378-32"}]: dispatch 2024-02-23T00:45:06.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:06.138188+0000 mon.a (mon.0) 1636 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:07.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.069+0000 7f8d28226700 1 -- 172.21.15.47:0/110285017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d200f8420 msgr2=0x7f8d20104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:07.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.069+0000 7f8d28226700 1 --2- 172.21.15.47:0/110285017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d200f8420 0x7f8d20104a20 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 rx=0x7f8d18009be0 tx=0x7f8d18004d10).stop 2024-02-23T00:45:07.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.069+0000 7f8d28226700 1 -- 172.21.15.47:0/110285017 shutdown_connections 2024-02-23T00:45:07.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.069+0000 7f8d28226700 1 --2- 172.21.15.47:0/110285017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d200f8420 0x7f8d20104a20 unknown :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:07.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.069+0000 7f8d28226700 1 --2- 172.21.15.47:0/110285017 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8d200fdbb0 0x7f8d200fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:07.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.069+0000 7f8d28226700 1 --2- 172.21.15.47:0/110285017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d200fe510 0x7f8d200f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:07.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.069+0000 7f8d28226700 1 -- 172.21.15.47:0/110285017 >> 172.21.15.47:0/110285017 conn(0x7f8d200f3800 msgr2=0x7f8d200f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:45:07.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.069+0000 7f8d28226700 1 -- 172.21.15.47:0/110285017 shutdown_connections 2024-02-23T00:45:07.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.069+0000 7f8d28226700 1 -- 172.21.15.47:0/110285017 wait complete. 2024-02-23T00:45:07.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.070+0000 7f8d28226700 1 librados: starting msgr at 2024-02-23T00:45:07.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.070+0000 7f8d28226700 1 librados: starting objecter 2024-02-23T00:45:07.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.070+0000 7f8d28226700 1 Processor -- start 2024-02-23T00:45:07.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.070+0000 7f8d28226700 1 -- start start 2024-02-23T00:45:07.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.070+0000 7f8d28226700 1 librados: setting wanted keys 2024-02-23T00:45:07.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.070+0000 7f8d28226700 1 librados: calling monclient init 2024-02-23T00:45:07.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.070+0000 7f8d28226700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8d200f8420 0x7f8d2006b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:07.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.070+0000 7f8d28226700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d200fdbb0 0x7f8d2006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:07.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.070+0000 7f8d28226700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d200fe510 0x7f8d2006bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:07.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.070+0000 7f8d28226700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d20116100 con 0x7f8d200fe510 2024-02-23T00:45:07.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.070+0000 7f8d28226700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d20116270 con 0x7f8d200fdbb0 2024-02-23T00:45:07.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.070+0000 7f8d28226700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d20116430 con 0x7f8d200f8420 2024-02-23T00:45:07.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.070+0000 7f8d25fc2700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8d200f8420 0x7f8d2006b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:07.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.070+0000 7f8d25fc2700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8d200f8420 0x7f8d2006b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52226/0 (socket says 172.21.15.47:52226) 2024-02-23T00:45:07.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.070+0000 7f8d25fc2700 1 -- 172.21.15.47:0/6704064 learned_addr learned my addr 172.21.15.47:0/6704064 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:45:07.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.070+0000 7f8d267c3700 1 --2- 172.21.15.47:0/6704064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d200fe510 0x7f8d2006bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:07.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.070+0000 7f8d257c1700 1 --2- 172.21.15.47:0/6704064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d200fdbb0 0x7f8d2006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:07.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.070+0000 7f8d25fc2700 1 -- 172.21.15.47:0/6704064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d200fdbb0 msgr2=0x7f8d2006b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:07.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.070+0000 7f8d25fc2700 1 --2- 172.21.15.47:0/6704064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d200fdbb0 0x7f8d2006b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:07.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.070+0000 7f8d25fc2700 1 -- 172.21.15.47:0/6704064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d200fe510 msgr2=0x7f8d2006bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:07.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.070+0000 7f8d25fc2700 1 --2- 172.21.15.47:0/6704064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d200fe510 0x7f8d2006bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:07.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.070+0000 7f8d25fc2700 1 -- 172.21.15.47:0/6704064 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d14005040 con 0x7f8d200f8420 2024-02-23T00:45:07.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.070+0000 7f8d267c3700 1 --2- 172.21.15.47:0/6704064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d200fe510 0x7f8d2006bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:45:07.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.071+0000 7f8d25fc2700 1 --2- 172.21.15.47:0/6704064 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8d200f8420 0x7f8d2006b0d0 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 rx=0x7f8d140048f0 tx=0x7f8d14009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:07.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.071+0000 7f8d12ffd700 10 client.?.objecter ms_handle_connect 0x7f8d200f8420 2024-02-23T00:45:07.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.071+0000 7f8d12ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:45:07.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.071+0000 7f8d12ffd700 1 -- 172.21.15.47:0/6704064 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d14007b00 con 0x7f8d200f8420 2024-02-23T00:45:07.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.071+0000 7f8d12ffd700 1 -- 172.21.15.47:0/6704064 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f8d1400f040 con 0x7f8d200f8420 2024-02-23T00:45:07.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.071+0000 7f8d28226700 1 -- 172.21.15.47:0/6704064 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d180097c0 con 0x7f8d200f8420 2024-02-23T00:45:07.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.071+0000 7f8d12ffd700 1 -- 172.21.15.47:0/6704064 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d14014890 con 0x7f8d200f8420 2024-02-23T00:45:07.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.071+0000 7f8d28226700 10 client.40820.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:45:07.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.071+0000 7f8d28226700 1 -- 172.21.15.47:0/6704064 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d20116af0 con 0x7f8d200f8420 2024-02-23T00:45:07.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.071+0000 7f8d28226700 1 librados: init done 2024-02-23T00:45:07.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.072+0000 7f8d28226700 1 -- 172.21.15.47:0/6704064 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d20106ed0 con 0x7f8d200f8420 2024-02-23T00:45:07.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.072+0000 7f8d12ffd700 1 -- 172.21.15.47:0/6704064 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f8d14014cc0 con 0x7f8d200f8420 2024-02-23T00:45:07.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.073+0000 7f8d12ffd700 1 --2- 172.21.15.47:0/6704064 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8d0c062cc0 0x7f8d0c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:07.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.073+0000 7f8d12ffd700 1 -- 172.21.15.47:0/6704064 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(234..234 src has 48..234) v4 ==== 10145+0+0 (secure 0 0 0) 0x7f8d1407d620 con 0x7f8d200f8420 2024-02-23T00:45:07.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.073+0000 7f8d12ffd700 10 client.40820.objecter ms_dispatch 0x7f8d20001550 osd_map(234..234 src has 48..234) v4 2024-02-23T00:45:07.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.073+0000 7f8d12ffd700 3 client.40820.objecter handle_osd_map got epochs [234,234] > 0 2024-02-23T00:45:07.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.073+0000 7f8d12ffd700 3 client.40820.objecter handle_osd_map decoding full epoch 234 2024-02-23T00:45:07.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.073+0000 7f8d257c1700 1 --2- 172.21.15.47:0/6704064 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8d0c062cc0 0x7f8d0c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:07.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.073+0000 7f8d12ffd700 20 client.40820.objecter dump_active .. 0 homeless 2024-02-23T00:45:07.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.073+0000 7f8d12ffd700 10 client.40820.objecter ms_handle_connect 0x7f8d0c062cc0 2024-02-23T00:45:07.344 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:45:07.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.073+0000 7f8d257c1700 1 --2- 172.21.15.47:0/6704064 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8d0c062cc0 0x7f8d0c065180 secure :-1 s=READY pgs=2974 cs=0 l=1 rev1=1 rx=0x7f8d2006c870 tx=0x7f8d1c009d60).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:07.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.077+0000 7f8d12ffd700 1 -- 172.21.15.47:0/6704064 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f8d14049f60 con 0x7f8d200f8420 2024-02-23T00:45:07.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.267+0000 7f8d28226700 1 -- 172.21.15.47:0/6704064 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8d2005fb60 con 0x7f8d0c062cc0 2024-02-23T00:45:07.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.285+0000 7f8d12ffd700 1 -- 172.21.15.47:0/6704064 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+632475 (secure 0 0 0) 0x7f8d2005fb60 con 0x7f8d0c062cc0 2024-02-23T00:45:07.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.341+0000 7f8d28226700 10 librados: watch_flush enter 2024-02-23T00:45:07.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.341+0000 7f8d28226700 10 librados: watch_flush exit 2024-02-23T00:45:07.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.341+0000 7f8d28226700 20 client.40820.objecter shutdown clearing up homeless session... 2024-02-23T00:45:07.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.341+0000 7f8d28226700 10 client.40820.objecter successfully canceled tick 2024-02-23T00:45:07.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.342+0000 7f8d28226700 1 -- 172.21.15.47:0/6704064 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8d0c062cc0 msgr2=0x7f8d0c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:07.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.342+0000 7f8d28226700 1 --2- 172.21.15.47:0/6704064 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8d0c062cc0 0x7f8d0c065180 secure :-1 s=READY pgs=2974 cs=0 l=1 rev1=1 rx=0x7f8d2006c870 tx=0x7f8d1c009d60).stop 2024-02-23T00:45:07.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.342+0000 7f8d28226700 1 -- 172.21.15.47:0/6704064 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8d200f8420 msgr2=0x7f8d2006b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:07.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.342+0000 7f8d28226700 1 --2- 172.21.15.47:0/6704064 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8d200f8420 0x7f8d2006b0d0 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 rx=0x7f8d140048f0 tx=0x7f8d14009cb0).stop 2024-02-23T00:45:07.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.342+0000 7f8d28226700 1 -- 172.21.15.47:0/6704064 shutdown_connections 2024-02-23T00:45:07.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.342+0000 7f8d28226700 1 --2- 172.21.15.47:0/6704064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d200fe510 0x7f8d2006bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:07.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.342+0000 7f8d28226700 1 --2- 172.21.15.47:0/6704064 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8d0c062cc0 0x7f8d0c065180 unknown :-1 s=CLOSED pgs=2974 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:07.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.342+0000 7f8d28226700 1 --2- 172.21.15.47:0/6704064 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8d200f8420 0x7f8d2006b0d0 unknown :-1 s=CLOSED pgs=2556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:07.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.342+0000 7f8d28226700 1 --2- 172.21.15.47:0/6704064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d200fdbb0 0x7f8d2006b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:07.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.342+0000 7f8d28226700 1 -- 172.21.15.47:0/6704064 >> 172.21.15.47:0/6704064 conn(0x7f8d200f3800 msgr2=0x7f8d200f50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:45:07.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.342+0000 7f8d28226700 1 -- 172.21.15.47:0/6704064 shutdown_connections 2024-02-23T00:45:07.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.342+0000 7f8d28226700 1 -- 172.21.15.47:0/6704064 wait complete. 2024-02-23T00:45:07.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:07.342+0000 7f8d28226700 1 librados: shutdown 2024-02-23T00:45:07.357 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-02-23T00:45:07.360 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:07.361 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:45:07.361 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:45:07.462 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:45:07.490 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:07.642 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:45:07.645 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:06.481859+0000 osd.2 (osd.2) 103 : cluster [DBG] 15.0 scrub starts 2024-02-23T00:45:07.645 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:06.483248+0000 osd.2 (osd.2) 104 : cluster [DBG] 15.0 scrub ok 2024-02-23T00:45:07.645 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:06.644053+0000 mon.a (mon.0) 1637 : audit [DBG] from='client.? 172.21.15.47:0/2316729014' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:45:07.645 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:06.648921+0000 osd.6 (osd.6) 97 : cluster [DBG] 15.5 scrub starts 2024-02-23T00:45:07.645 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:06.651703+0000 osd.6 (osd.6) 98 : cluster [DBG] 15.5 scrub ok 2024-02-23T00:45:07.645 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:06.805272+0000 mon.b (mon.1) 294 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:07.645 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:07.010419+0000 osd.3 (osd.3) 125 : cluster [DBG] 4.17 scrub starts 2024-02-23T00:45:07.645 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:07.011947+0000 osd.3 (osd.3) 126 : cluster [DBG] 4.17 scrub ok 2024-02-23T00:45:07.645 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:07.201725+0000 osd.2 (osd.2) 105 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:45:07.645 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:07.201949+0000 osd.2 (osd.2) 106 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:45:07.645 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:07.565472+0000 mon.a (mon.0) 1638 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:07.645 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:07.565637+0000 mon.a (mon.0) 1639 : cluster [DBG] osdmap e235: 8 total, 8 up, 8 in 2024-02-23T00:45:07.646 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:06.481859+0000 osd.2 (osd.2) 103 : cluster [DBG] 15.0 scrub starts 2024-02-23T00:45:07.646 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:06.483248+0000 osd.2 (osd.2) 104 : cluster [DBG] 15.0 scrub ok 2024-02-23T00:45:07.646 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:06.644053+0000 mon.a (mon.0) 1637 : audit [DBG] from='client.? 172.21.15.47:0/2316729014' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:45:07.646 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:06.648921+0000 osd.6 (osd.6) 97 : cluster [DBG] 15.5 scrub starts 2024-02-23T00:45:07.646 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:06.651703+0000 osd.6 (osd.6) 98 : cluster [DBG] 15.5 scrub ok 2024-02-23T00:45:07.646 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:06.805272+0000 mon.b (mon.1) 294 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:07.646 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:07.010419+0000 osd.3 (osd.3) 125 : cluster [DBG] 4.17 scrub starts 2024-02-23T00:45:07.646 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:07.011947+0000 osd.3 (osd.3) 126 : cluster [DBG] 4.17 scrub ok 2024-02-23T00:45:07.646 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:07.201725+0000 osd.2 (osd.2) 105 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:45:07.646 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:07.201949+0000 osd.2 (osd.2) 106 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:45:07.646 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:07.565472+0000 mon.a (mon.0) 1638 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:07.646 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:07.565637+0000 mon.a (mon.0) 1639 : cluster [DBG] osdmap e235: 8 total, 8 up, 8 in 2024-02-23T00:45:07.647 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:07.648 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:45:07.749 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:45:07.780 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:07.801 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:45:07.804 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:07.804 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:45:07.905 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:45:07.939 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:07.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:06.481859+0000 osd.2 (osd.2) 103 : cluster [DBG] 15.0 scrub starts 2024-02-23T00:45:07.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:06.483248+0000 osd.2 (osd.2) 104 : cluster [DBG] 15.0 scrub ok 2024-02-23T00:45:07.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:06.644053+0000 mon.a (mon.0) 1637 : audit [DBG] from='client.? 172.21.15.47:0/2316729014' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:45:07.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:06.648921+0000 osd.6 (osd.6) 97 : cluster [DBG] 15.5 scrub starts 2024-02-23T00:45:07.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:06.651703+0000 osd.6 (osd.6) 98 : cluster [DBG] 15.5 scrub ok 2024-02-23T00:45:07.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:06.805272+0000 mon.b (mon.1) 294 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:07.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:07.010419+0000 osd.3 (osd.3) 125 : cluster [DBG] 4.17 scrub starts 2024-02-23T00:45:07.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:07.011947+0000 osd.3 (osd.3) 126 : cluster [DBG] 4.17 scrub ok 2024-02-23T00:45:07.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:07.201725+0000 osd.2 (osd.2) 105 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:45:07.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:07.201949+0000 osd.2 (osd.2) 106 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:45:07.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:07.565472+0000 mon.a (mon.0) 1638 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:07.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:07.565637+0000 mon.a (mon.0) 1639 : cluster [DBG] osdmap e235: 8 total, 8 up, 8 in 2024-02-23T00:45:08.627 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-02-23T00:45:07.234499+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":9426,"num_objects":390,"num_object_clones":39,"num_object_copies":1170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":390,"num_whiteouts":0,"num_read":1541,"num_read_kb":1173,"num_write":1245,"num_write_kb":622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":947,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2484,"ondisk_log_size":2484,"up":750,"acting":750,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":908,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2470020,"kb_used_data":30716,"kb_used_omap":0,"kb_used_meta":2439168,"kb_avail":747491196,"statfs":{"total":767960285184,"available":765430984704,"internally_reserved":0,"allocated":31453184,"data_stored":18032044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2497708032},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-334,"num_objects":-19,"num_object_clones":-18,"num_object_copies":-57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-18,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":-1,"num_write_kb":-1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":-1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000826"},"pg_stats":[{"pgid":"153.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:06.117379+0000","last_change":"2024-02-23T00:45:06.134631+0000","last_active":"2024-02-23T00:45:06.117379+0000","last_peered":"2024-02-23T00:45:06.117379+0000","last_clean":"2024-02-23T00:45:06.117379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:06.117379+0000","last_undegraded":"2024-02-23T00:45:06.117379+0000","last_fullsized":"2024-02-23T00:45:06.117379+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.117379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"153.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:06.117379+0000","last_change":"2024-02-23T00:45:06.130221+0000","last_active":"2024-02-23T00:45:06.117379+0000","last_peered":"2024-02-23T00:45:06.117379+0000","last_clean":"2024-02-23T00:45:06.117379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:06.117379+0000","last_undegraded":"2024-02-23T00:45:06.117379+0000","last_fullsized":"2024-02-23T00:45:06.117379+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.117379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"153.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:06.117379+0000","last_change":"2024-02-23T00:45:06.130432+0000","last_active":"2024-02-23T00:45:06.117379+0000","last_peered":"2024-02-23T00:45:06.117379+0000","last_clean":"2024-02-23T00:45:06.117379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:06.117379+0000","last_undegraded":"2024-02-23T00:45:06.117379+0000","last_fullsized":"2024-02-23T00:45:06.117379+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.117379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"153.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:06.117379+0000","last_change":"2024-02-23T00:45:06.133551+0000","last_active":"2024-02-23T00:45:06.117379+0000","last_peered":"2024-02-23T00:45:06.117379+0000","last_clean":"2024-02-23T00:45:06.117379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:06.117379+0000","last_undegraded":"2024-02-23T00:45:06.117379+0000","last_fullsized":"2024-02-23T00:45:06.117379+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.117379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"153.d","version":"0'0","reported_seq":3,"reported_epoch":235,"state":"creating+activating","last_fresh":"2024-02-23T00:45:07.135209+0000","last_change":"2024-02-23T00:45:07.135209+0000","last_active":"2024-02-23T00:45:06.117379+0000","last_peered":"2024-02-23T00:45:06.117379+0000","last_clean":"2024-02-23T00:45:06.117379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:07.135209+0000","last_undegraded":"2024-02-23T00:45:07.135209+0000","last_fullsized":"2024-02-23T00:45:07.135209+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.117379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":["4","1","3"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"153.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:06.117379+0000","last_change":"2024-02-23T00:45:06.129994+0000","last_active":"2024-02-23T00:45:06.117379+0000","last_peered":"2024-02-23T00:45:06.117379+0000","last_clean":"2024-02-23T00:45:06.117379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:06.117379+0000","last_undegraded":"2024-02-23T00:45:06.117379+0000","last_fullsized":"2024-02-23T00:45:06.117379+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.117379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"153.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:06.117379+0000","last_change":"2024-02-23T00:45:06.133391+0000","last_active":"2024-02-23T00:45:06.117379+0000","last_peered":"2024-02-23T00:45:06.117379+0000","last_clean":"2024-02-23T00:45:06.117379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:06.117379+0000","last_undegraded":"2024-02-23T00:45:06.117379+0000","last_fullsized":"2024-02-23T00:45:06.117379+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.117379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"153.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:06.117379+0000","last_change":"2024-02-23T00:45:06.138983+0000","last_active":"2024-02-23T00:45:06.117379+0000","last_peered":"2024-02-23T00:45:06.117379+0000","last_clean":"2024-02-23T00:45:06.117379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:06.117379+0000","last_undegraded":"2024-02-23T00:45:06.117379+0000","last_fullsized":"2024-02-23T00:45:06.117379+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.117379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"153.1","version":"0'0","reported_seq":3,"reported_epoch":235,"state":"creating+activating","last_fresh":"2024-02-23T00:45:07.131183+0000","last_change":"2024-02-23T00:45:07.131183+0000","last_active":"2024-02-23T00:45:06.117379+0000","last_peered":"2024-02-23T00:45:06.117379+0000","last_clean":"2024-02-23T00:45:06.117379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:07.131183+0000","last_undegraded":"2024-02-23T00:45:07.131183+0000","last_fullsized":"2024-02-23T00:45:07.131183+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.117379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":["2","0","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"153.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:06.117379+0000","last_change":"2024-02-23T00:45:06.129810+0000","last_active":"2024-02-23T00:45:06.117379+0000","last_peered":"2024-02-23T00:45:06.117379+0000","last_clean":"2024-02-23T00:45:06.117379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:06.117379+0000","last_undegraded":"2024-02-23T00:45:06.117379+0000","last_fullsized":"2024-02-23T00:45:06.117379+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.117379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"153.3","version":"0'0","reported_seq":2,"reported_epoch":235,"state":"creating+activating","last_fresh":"2024-02-23T00:45:07.134774+0000","last_change":"2024-02-23T00:45:07.135069+0000","last_active":"2024-02-23T00:45:06.117379+0000","last_peered":"2024-02-23T00:45:06.117379+0000","last_clean":"2024-02-23T00:45:06.117379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:07.134774+0000","last_undegraded":"2024-02-23T00:45:07.134774+0000","last_fullsized":"2024-02-23T00:45:07.134774+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.117379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":["4","0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"153.2","version":"0'0","reported_seq":3,"reported_epoch":235,"state":"creating+activating","last_fresh":"2024-02-23T00:45:07.126215+0000","last_change":"2024-02-23T00:45:07.126215+0000","last_active":"2024-02-23T00:45:06.117379+0000","last_peered":"2024-02-23T00:45:06.117379+0000","last_clean":"2024-02-23T00:45:06.117379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:07.126215+0000","last_undegraded":"2024-02-23T00:45:07.126215+0000","last_fullsized":"2024-02-23T00:45:07.126215+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.117379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":["3","1","6"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"153.5","version":"0'0","reported_seq":9,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-02-23T00:45:07.144153+0000","last_change":"2024-02-23T00:45:07.144153+0000","last_active":"2024-02-23T00:45:07.144153+0000","last_peered":"2024-02-23T00:45:07.144153+0000","last_clean":"2024-02-23T00:45:07.144153+0000","last_became_active":"2024-02-23T00:45:07.143803+0000","last_became_peered":"2024-02-23T00:45:07.143803+0000","last_unstale":"2024-02-23T00:45:07.144153+0000","last_undegraded":"2024-02-23T00:45:07.144153+0000","last_fullsized":"2024-02-23T00:45:07.144153+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.117379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"153.4","version":"0'0","reported_seq":3,"reported_epoch":235,"state":"creating+activating","last_fresh":"2024-02-23T00:45:07.128419+0000","last_change":"2024-02-23T00:45:07.128419+0000","last_active":"2024-02-23T00:45:06.117379+0000","last_peered":"2024-02-23T00:45:06.117379+0000","last_clean":"2024-02-23T00:45:06.117379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:07.128419+0000","last_undegraded":"2024-02-23T00:45:07.128419+0000","last_fullsized":"2024-02-23T00:45:07.128419+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.117379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":["3","0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"153.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:06.117379+0000","last_change":"2024-02-23T00:45:06.129919+0000","last_active":"2024-02-23T00:45:06.117379+0000","last_peered":"2024-02-23T00:45:06.117379+0000","last_clean":"2024-02-23T00:45:06.117379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:06.117379+0000","last_undegraded":"2024-02-23T00:45:06.117379+0000","last_fullsized":"2024-02-23T00:45:06.117379+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.117379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"153.6","version":"0'0","reported_seq":2,"reported_epoch":235,"state":"creating+activating","last_fresh":"2024-02-23T00:45:07.134869+0000","last_change":"2024-02-23T00:45:07.135308+0000","last_active":"2024-02-23T00:45:06.117379+0000","last_peered":"2024-02-23T00:45:06.117379+0000","last_clean":"2024-02-23T00:45:06.117379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:07.134869+0000","last_undegraded":"2024-02-23T00:45:07.134869+0000","last_fullsized":"2024-02-23T00:45:07.134869+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.117379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":["4","1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"153.19","version":"0'0","reported_seq":2,"reported_epoch":235,"state":"creating+activating","last_fresh":"2024-02-23T00:45:07.125883+0000","last_change":"2024-02-23T00:45:07.126820+0000","last_active":"2024-02-23T00:45:06.117379+0000","last_peered":"2024-02-23T00:45:06.117379+0000","last_clean":"2024-02-23T00:45:06.117379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:07.125883+0000","last_undegraded":"2024-02-23T00:45:07.125883+0000","last_fullsized":"2024-02-23T00:45:07.125883+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.117379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":["3","5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"153.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:06.117379+0000","last_change":"2024-02-23T00:45:06.130562+0000","last_active":"2024-02-23T00:45:06.117379+0000","last_peered":"2024-02-23T00:45:06.117379+0000","last_clean":"2024-02-23T00:45:06.117379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:06.117379+0000","last_undegraded":"2024-02-23T00:45:06.117379+0000","last_fullsized":"2024-02-23T00:45:06.117379+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.117379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"153.1b","version":"0'0","reported_seq":3,"reported_epoch":235,"state":"creating+activating","last_fresh":"2024-02-23T00:45:07.129428+0000","last_change":"2024-02-23T00:45:07.129428+0000","last_active":"2024-02-23T00:45:06.117379+0000","last_peered":"2024-02-23T00:45:06.117379+0000","last_clean":"2024-02-23T00:45:06.117379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:07.129428+0000","last_undegraded":"2024-02-23T00:45:07.129428+0000","last_fullsized":"2024-02-23T00:45:07.129428+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.117379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":["6","0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"153.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:06.117379+0000","last_change":"2024-02-23T00:45:06.129981+0000","last_active":"2024-02-23T00:45:06.117379+0000","last_peered":"2024-02-23T00:45:06.117379+0000","last_clean":"2024-02-23T00:45:06.117379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:06.117379+0000","last_undegraded":"2024-02-23T00:45:06.117379+0000","last_fullsized":"2024-02-23T00:45:06.117379+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.117379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"153.1d","version":"0'0","reported_seq":3,"reported_epoch":235,"state":"creating+activating","last_fresh":"2024-02-23T00:45:07.140948+0000","last_change":"2024-02-23T00:45:07.140948+0000","last_active":"2024-02-23T00:45:06.117379+0000","last_peered":"2024-02-23T00:45:06.117379+0000","last_clean":"2024-02-23T00:45:06.117379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:07.140948+0000","last_undegraded":"2024-02-23T00:45:07.140948+0000","last_fullsized":"2024-02-23T00:45:07.140948+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.117379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":["2","0","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"153.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:06.117379+0000","last_change":"2024-02-23T00:45:06.130808+0000","last_active":"2024-02-23T00:45:06.117379+0000","last_peered":"2024-02-23T00:45:06.117379+0000","last_clean":"2024-02-23T00:45:06.117379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:06.117379+0000","last_undegraded":"2024-02-23T00:45:06.117379+0000","last_fullsized":"2024-02-23T00:45:06.117379+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.117379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,0],"acting":[5,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"153.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:06.117379+0000","last_change":"2024-02-23T00:45:06.130287+0000","last_active":"2024-02-23T00:45:06.117379+0000","last_peered":"2024-02-23T00:45:06.117379+0000","last_clean":"2024-02-23T00:45:06.117379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:06.117379+0000","last_undegraded":"2024-02-23T00:45:06.117379+0000","last_fullsized":"2024-02-23T00:45:06.117379+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.117379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"153.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:06.117379+0000","last_change":"2024-02-23T00:45:06.130166+0000","last_active":"2024-02-23T00:45:06.117379+0000","last_peered":"2024-02-23T00:45:06.117379+0000","last_clean":"2024-02-23T00:45:06.117379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:06.117379+0000","last_undegraded":"2024-02-23T00:45:06.117379+0000","last_fullsized":"2024-02-23T00:45:06.117379+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.117379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"153.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:06.117379+0000","last_change":"2024-02-23T00:45:06.132796+0000","last_active":"2024-02-23T00:45:06.117379+0000","last_peered":"2024-02-23T00:45:06.117379+0000","last_clean":"2024-02-23T00:45:06.117379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:06.117379+0000","last_undegraded":"2024-02-23T00:45:06.117379+0000","last_fullsized":"2024-02-23T00:45:06.117379+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.117379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"153.10","version":"0'0","reported_seq":3,"reported_epoch":235,"state":"creating+activating","last_fresh":"2024-02-23T00:45:07.129443+0000","last_change":"2024-02-23T00:45:07.129443+0000","last_active":"2024-02-23T00:45:06.117379+0000","last_peered":"2024-02-23T00:45:06.117379+0000","last_clean":"2024-02-23T00:45:06.117379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:07.129443+0000","last_undegraded":"2024-02-23T00:45:07.129443+0000","last_fullsized":"2024-02-23T00:45:07.129443+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.117379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":["6","3","5"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"153.13","version":"0'0","reported_seq":3,"reported_epoch":235,"state":"creating+activating","last_fresh":"2024-02-23T00:45:07.126390+0000","last_change":"2024-02-23T00:45:07.126390+0000","last_active":"2024-02-23T00:45:06.117379+0000","last_peered":"2024-02-23T00:45:06.117379+0000","last_clean":"2024-02-23T00:45:06.117379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:07.126390+0000","last_undegraded":"2024-02-23T00:45:07.126390+0000","last_fullsized":"2024-02-23T00:45:07.126390+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.117379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":["3","2","7"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"153.12","version":"0'0","reported_seq":3,"reported_epoch":235,"state":"creating+activating","last_fresh":"2024-02-23T00:45:07.136041+0000","last_change":"2024-02-23T00:45:07.136041+0000","last_active":"2024-02-23T00:45:06.117379+0000","last_peered":"2024-02-23T00:45:06.117379+0000","last_clean":"2024-02-23T00:45:06.117379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:07.136041+0000","last_undegraded":"2024-02-23T00:45:07.136041+0000","last_fullsized":"2024-02-23T00:45:07.136041+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.117379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":["2","1","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"153.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:06.117379+0000","last_change":"2024-02-23T00:45:06.130776+0000","last_active":"2024-02-23T00:45:06.117379+0000","last_peered":"2024-02-23T00:45:06.117379+0000","last_clean":"2024-02-23T00:45:06.117379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:06.117379+0000","last_undegraded":"2024-02-23T00:45:06.117379+0000","last_fullsized":"2024-02-23T00:45:06.117379+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.117379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"153.14","version":"0'0","reported_seq":2,"reported_epoch":235,"state":"creating+activating","last_fresh":"2024-02-23T00:45:07.134831+0000","last_change":"2024-02-23T00:45:07.135042+0000","last_active":"2024-02-23T00:45:06.117379+0000","last_peered":"2024-02-23T00:45:06.117379+0000","last_clean":"2024-02-23T00:45:06.117379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:07.134831+0000","last_undegraded":"2024-02-23T00:45:07.134831+0000","last_fullsized":"2024-02-23T00:45:07.134831+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.117379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,5],"acting":[4,7,5],"avail_no_missing":["4","5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"153.17","version":"0'0","reported_seq":3,"reported_epoch":235,"state":"creating+activating","last_fresh":"2024-02-23T00:45:07.128724+0000","last_change":"2024-02-23T00:45:07.128724+0000","last_active":"2024-02-23T00:45:06.117379+0000","last_peered":"2024-02-23T00:45:06.117379+0000","last_clean":"2024-02-23T00:45:06.117379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:07.128724+0000","last_undegraded":"2024-02-23T00:45:07.128724+0000","last_fullsized":"2024-02-23T00:45:07.128724+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.117379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":["3","1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"153.16","version":"0'0","reported_seq":2,"reported_epoch":235,"state":"creating+activating","last_fresh":"2024-02-23T00:45:07.129298+0000","last_change":"2024-02-23T00:45:07.129476+0000","last_active":"2024-02-23T00:45:06.117379+0000","last_peered":"2024-02-23T00:45:06.117379+0000","last_clean":"2024-02-23T00:45:06.117379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:07.129298+0000","last_undegraded":"2024-02-23T00:45:07.129298+0000","last_fullsized":"2024-02-23T00:45:07.129298+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:06.117379+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.117379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5],"acting":[6,7,5],"avail_no_missing":["6","5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.0","version":"231'35","reported_seq":172,"reported_epoch":233,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-23T00:45:05.235365+0000","last_change":"2024-02-23T00:45:05.232069+0000","last_active":"2024-02-23T00:45:05.235365+0000","last_peered":"2024-02-23T00:45:05.235365+0000","last_clean":"2024-02-23T00:45:05.235365+0000","last_became_active":"2024-02-23T00:43:17.855316+0000","last_became_peered":"2024-02-23T00:43:17.855316+0000","last_unstale":"2024-02-23T00:45:05.235365+0000","last_undegraded":"2024-02-23T00:45:05.235365+0000","last_fullsized":"2024-02-23T00:45:05.235365+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":26,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":11,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"6","length":"2"}]},{"pgid":"62.1","version":"230'87","reported_seq":220,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.315052+0000","last_change":"2024-02-23T00:44:55.242749+0000","last_active":"2024-02-23T00:45:05.315052+0000","last_peered":"2024-02-23T00:45:05.315052+0000","last_clean":"2024-02-23T00:45:05.315052+0000","last_became_active":"2024-02-23T00:43:17.851866+0000","last_became_peered":"2024-02-23T00:43:17.851866+0000","last_unstale":"2024-02-23T00:45:05.315052+0000","last_undegraded":"2024-02-23T00:45:05.315052+0000","last_fullsized":"2024-02-23T00:45:05.315052+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":87,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":63,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"}]},{"pgid":"62.2","version":"229'68","reported_seq":202,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.139583+0000","last_change":"2024-02-23T00:45:03.247120+0000","last_active":"2024-02-23T00:45:06.139583+0000","last_peered":"2024-02-23T00:45:06.139583+0000","last_clean":"2024-02-23T00:45:06.139583+0000","last_became_active":"2024-02-23T00:43:17.804792+0000","last_became_peered":"2024-02-23T00:43:17.804792+0000","last_unstale":"2024-02-23T00:45:06.139583+0000","last_undegraded":"2024-02-23T00:45:06.139583+0000","last_fullsized":"2024-02-23T00:45:06.139583+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"}]},{"pgid":"62.3","version":"217'17","reported_seq":161,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.128101+0000","last_change":"2024-02-23T00:44:51.235164+0000","last_active":"2024-02-23T00:45:06.128101+0000","last_peered":"2024-02-23T00:45:06.128101+0000","last_clean":"2024-02-23T00:45:06.128101+0000","last_became_active":"2024-02-23T00:43:17.834686+0000","last_became_peered":"2024-02-23T00:43:17.834686+0000","last_unstale":"2024-02-23T00:45:06.128101+0000","last_undegraded":"2024-02-23T00:45:06.128101+0000","last_fullsized":"2024-02-23T00:45:06.128101+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"}]},{"pgid":"62.4","version":"207'17","reported_seq":156,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.317874+0000","last_change":"2024-02-23T00:44:41.227996+0000","last_active":"2024-02-23T00:45:05.317874+0000","last_peered":"2024-02-23T00:45:05.317874+0000","last_clean":"2024-02-23T00:45:05.317874+0000","last_became_active":"2024-02-23T00:43:17.810343+0000","last_became_peered":"2024-02-23T00:43:17.810343+0000","last_unstale":"2024-02-23T00:45:05.317874+0000","last_undegraded":"2024-02-23T00:45:05.317874+0000","last_fullsized":"2024-02-23T00:45:05.317874+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"}]},{"pgid":"62.5","version":"223'45","reported_seq":179,"reported_epoch":234,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-23T00:45:06.129951+0000","last_change":"2024-02-23T00:44:57.235757+0000","last_active":"2024-02-23T00:45:06.129951+0000","last_peered":"2024-02-23T00:45:06.129951+0000","last_clean":"2024-02-23T00:45:06.129951+0000","last_became_active":"2024-02-23T00:43:17.855339+0000","last_became_peered":"2024-02-23T00:43:17.855339+0000","last_unstale":"2024-02-23T00:45:06.129951+0000","last_undegraded":"2024-02-23T00:45:06.129951+0000","last_fullsized":"2024-02-23T00:45:06.129951+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":45,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":66,"num_objects":6,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6","length":"1"}]},{"pgid":"62.6","version":"232'43","reported_seq":175,"reported_epoch":233,"state":"active+clean+snaptrim","last_fresh":"2024-02-23T00:45:05.240029+0000","last_change":"2024-02-23T00:45:05.232110+0000","last_active":"2024-02-23T00:45:05.240029+0000","last_peered":"2024-02-23T00:45:05.240029+0000","last_clean":"2024-02-23T00:45:05.240029+0000","last_became_active":"2024-02-23T00:43:17.838838+0000","last_became_peered":"2024-02-23T00:43:17.838838+0000","last_unstale":"2024-02-23T00:45:05.240029+0000","last_undegraded":"2024-02-23T00:45:05.240029+0000","last_fullsized":"2024-02-23T00:45:05.240029+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":43,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"stat_sum":{"num_bytes":77,"num_objects":7,"num_object_clones":4,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.7","version":"215'68","reported_seq":195,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.126018+0000","last_change":"2024-02-23T00:44:49.229509+0000","last_active":"2024-02-23T00:45:06.126018+0000","last_peered":"2024-02-23T00:45:06.126018+0000","last_clean":"2024-02-23T00:45:06.126018+0000","last_became_active":"2024-02-23T00:43:17.846548+0000","last_became_peered":"2024-02-23T00:43:17.846548+0000","last_unstale":"2024-02-23T00:45:06.126018+0000","last_undegraded":"2024-02-23T00:45:06.126018+0000","last_fullsized":"2024-02-23T00:45:06.126018+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"}]},{"pgid":"62.8","version":"225'75","reported_seq":199,"reported_epoch":233,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-23T00:45:05.235203+0000","last_change":"2024-02-23T00:44:59.239581+0000","last_active":"2024-02-23T00:45:05.235203+0000","last_peered":"2024-02-23T00:45:05.235203+0000","last_clean":"2024-02-23T00:45:05.235203+0000","last_became_active":"2024-02-23T00:43:17.842613+0000","last_became_peered":"2024-02-23T00:43:17.842613+0000","last_unstale":"2024-02-23T00:45:05.235203+0000","last_undegraded":"2024-02-23T00:45:05.235203+0000","last_fullsized":"2024-02-23T00:45:05.235203+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":75,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":5,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":25,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"7","length":"1"}]},{"pgid":"62.9","version":"225'34","reported_seq":172,"reported_epoch":234,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-23T00:45:06.127911+0000","last_change":"2024-02-23T00:44:59.216449+0000","last_active":"2024-02-23T00:45:06.127911+0000","last_peered":"2024-02-23T00:45:06.127911+0000","last_clean":"2024-02-23T00:45:06.127911+0000","last_became_active":"2024-02-23T00:43:17.834297+0000","last_became_peered":"2024-02-23T00:43:17.834297+0000","last_unstale":"2024-02-23T00:45:06.127911+0000","last_undegraded":"2024-02-23T00:45:06.127911+0000","last_fullsized":"2024-02-23T00:45:06.127911+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6","length":"2"}]},{"pgid":"62.a","version":"233'72","reported_seq":194,"reported_epoch":233,"state":"active+clean+snaptrim","last_fresh":"2024-02-23T00:45:05.245299+0000","last_change":"2024-02-23T00:45:05.243867+0000","last_active":"2024-02-23T00:45:05.245299+0000","last_peered":"2024-02-23T00:45:05.245299+0000","last_clean":"2024-02-23T00:45:05.245299+0000","last_became_active":"2024-02-23T00:43:17.839142+0000","last_became_peered":"2024-02-23T00:43:17.839142+0000","last_unstale":"2024-02-23T00:45:05.245299+0000","last_undegraded":"2024-02-23T00:45:05.245299+0000","last_fullsized":"2024-02-23T00:45:05.245299+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":72,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"stat_sum":{"num_bytes":156,"num_objects":15,"num_object_clones":9,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.b","version":"229'60","reported_seq":184,"reported_epoch":233,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-23T00:45:05.236166+0000","last_change":"2024-02-23T00:45:03.230227+0000","last_active":"2024-02-23T00:45:05.236166+0000","last_peered":"2024-02-23T00:45:05.236166+0000","last_clean":"2024-02-23T00:45:05.236166+0000","last_became_active":"2024-02-23T00:43:17.855321+0000","last_became_peered":"2024-02-23T00:43:17.855321+0000","last_unstale":"2024-02-23T00:45:05.236166+0000","last_undegraded":"2024-02-23T00:45:05.236166+0000","last_fullsized":"2024-02-23T00:45:05.236166+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":60,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":88,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"6","length":"1"}]},{"pgid":"62.c","version":"203'34","reported_seq":173,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.129398+0000","last_change":"2024-02-23T00:44:38.121782+0000","last_active":"2024-02-23T00:45:06.129398+0000","last_peered":"2024-02-23T00:45:06.129398+0000","last_clean":"2024-02-23T00:45:06.129398+0000","last_became_active":"2024-02-23T00:43:17.829044+0000","last_became_peered":"2024-02-23T00:43:17.829044+0000","last_unstale":"2024-02-23T00:45:06.129398+0000","last_undegraded":"2024-02-23T00:45:06.129398+0000","last_fullsized":"2024-02-23T00:45:06.129398+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"}]},{"pgid":"62.d","version":"233'51","reported_seq":178,"reported_epoch":234,"state":"active+clean+snaptrim","last_fresh":"2024-02-23T00:45:06.129839+0000","last_change":"2024-02-23T00:45:03.240781+0000","last_active":"2024-02-23T00:45:06.129839+0000","last_peered":"2024-02-23T00:45:06.129839+0000","last_clean":"2024-02-23T00:45:06.129839+0000","last_became_active":"2024-02-23T00:43:17.841614+0000","last_became_peered":"2024-02-23T00:43:17.841614+0000","last_unstale":"2024-02-23T00:45:06.129839+0000","last_undegraded":"2024-02-23T00:45:06.129839+0000","last_fullsized":"2024-02-23T00:45:06.129839+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6","length":"1"}]},{"pgid":"62.e","version":"211'51","reported_seq":183,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.320596+0000","last_change":"2024-02-23T00:44:45.201500+0000","last_active":"2024-02-23T00:45:05.320596+0000","last_peered":"2024-02-23T00:45:05.320596+0000","last_clean":"2024-02-23T00:45:05.320596+0000","last_became_active":"2024-02-23T00:43:17.845956+0000","last_became_peered":"2024-02-23T00:43:17.845956+0000","last_unstale":"2024-02-23T00:45:05.320596+0000","last_undegraded":"2024-02-23T00:45:05.320596+0000","last_fullsized":"2024-02-23T00:45:05.320596+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"}]},{"pgid":"62.f","version":"232'120","reported_seq":233,"reported_epoch":234,"state":"active+clean+snaptrim","last_fresh":"2024-02-23T00:45:06.132210+0000","last_change":"2024-02-23T00:44:59.216497+0000","last_active":"2024-02-23T00:45:06.132210+0000","last_peered":"2024-02-23T00:45:06.132210+0000","last_clean":"2024-02-23T00:45:06.132210+0000","last_became_active":"2024-02-23T00:43:17.855331+0000","last_became_peered":"2024-02-23T00:43:17.855331+0000","last_unstale":"2024-02-23T00:45:06.132210+0000","last_undegraded":"2024-02-23T00:45:06.132210+0000","last_fullsized":"2024-02-23T00:45:06.132210+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":120,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"stat_sum":{"num_bytes":176,"num_objects":16,"num_object_clones":8,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":40,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.10","version":"219'68","reported_seq":201,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.313057+0000","last_change":"2024-02-23T00:44:53.240096+0000","last_active":"2024-02-23T00:45:05.313057+0000","last_peered":"2024-02-23T00:45:05.313057+0000","last_clean":"2024-02-23T00:45:05.313057+0000","last_became_active":"2024-02-23T00:43:17.819865+0000","last_became_peered":"2024-02-23T00:43:17.819865+0000","last_unstale":"2024-02-23T00:45:05.313057+0000","last_undegraded":"2024-02-23T00:45:05.313057+0000","last_fullsized":"2024-02-23T00:45:05.313057+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"}]},{"pgid":"62.11","version":"215'51","reported_seq":181,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.314841+0000","last_change":"2024-02-23T00:44:49.235143+0000","last_active":"2024-02-23T00:45:05.314841+0000","last_peered":"2024-02-23T00:45:05.314841+0000","last_clean":"2024-02-23T00:45:05.314841+0000","last_became_active":"2024-02-23T00:43:17.852697+0000","last_became_peered":"2024-02-23T00:43:17.852697+0000","last_unstale":"2024-02-23T00:45:05.314841+0000","last_undegraded":"2024-02-23T00:45:05.314841+0000","last_fullsized":"2024-02-23T00:45:05.314841+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"}]},{"pgid":"62.12","version":"215'68","reported_seq":203,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.126879+0000","last_change":"2024-02-23T00:44:49.229268+0000","last_active":"2024-02-23T00:45:06.126879+0000","last_peered":"2024-02-23T00:45:06.126879+0000","last_clean":"2024-02-23T00:45:06.126879+0000","last_became_active":"2024-02-23T00:43:17.856118+0000","last_became_peered":"2024-02-23T00:43:17.856118+0000","last_unstale":"2024-02-23T00:45:06.126879+0000","last_undegraded":"2024-02-23T00:45:06.126879+0000","last_fullsized":"2024-02-23T00:45:06.126879+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"}]},{"pgid":"62.13","version":"205'34","reported_seq":173,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.318136+0000","last_change":"2024-02-23T00:44:41.197548+0000","last_active":"2024-02-23T00:45:05.318136+0000","last_peered":"2024-02-23T00:45:05.318136+0000","last_clean":"2024-02-23T00:45:05.318136+0000","last_became_active":"2024-02-23T00:43:17.857823+0000","last_became_peered":"2024-02-23T00:43:17.857823+0000","last_unstale":"2024-02-23T00:45:05.318136+0000","last_undegraded":"2024-02-23T00:45:05.318136+0000","last_fullsized":"2024-02-23T00:45:05.318136+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"}]},{"pgid":"62.14","version":"227'35","reported_seq":165,"reported_epoch":233,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-23T00:45:05.236946+0000","last_change":"2024-02-23T00:45:01.241006+0000","last_active":"2024-02-23T00:45:05.236946+0000","last_peered":"2024-02-23T00:45:05.236946+0000","last_clean":"2024-02-23T00:45:05.236946+0000","last_became_active":"2024-02-23T00:43:17.816138+0000","last_became_peered":"2024-02-23T00:43:17.816138+0000","last_unstale":"2024-02-23T00:45:05.236946+0000","last_undegraded":"2024-02-23T00:45:05.236946+0000","last_fullsized":"2024-02-23T00:45:05.236946+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":49,"num_objects":5,"num_object_clones":2,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":12,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"6","length":"1"}]},{"pgid":"62.15","version":"201'17","reported_seq":160,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.130409+0000","last_change":"2024-02-23T00:44:38.120612+0000","last_active":"2024-02-23T00:45:06.130409+0000","last_peered":"2024-02-23T00:45:06.130409+0000","last_clean":"2024-02-23T00:45:06.130409+0000","last_became_active":"2024-02-23T00:43:17.843834+0000","last_became_peered":"2024-02-23T00:43:17.843834+0000","last_unstale":"2024-02-23T00:45:06.130409+0000","last_undegraded":"2024-02-23T00:45:06.130409+0000","last_fullsized":"2024-02-23T00:45:06.130409+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"}]},{"pgid":"62.16","version":"225'18","reported_seq":159,"reported_epoch":234,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-23T00:45:06.131006+0000","last_change":"2024-02-23T00:44:59.237287+0000","last_active":"2024-02-23T00:45:06.131006+0000","last_peered":"2024-02-23T00:45:06.131006+0000","last_clean":"2024-02-23T00:45:06.131006+0000","last_became_active":"2024-02-23T00:43:17.834291+0000","last_became_peered":"2024-02-23T00:43:17.834291+0000","last_unstale":"2024-02-23T00:45:06.131006+0000","last_undegraded":"2024-02-23T00:45:06.131006+0000","last_fullsized":"2024-02-23T00:45:06.131006+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":14,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6","length":"2"}]},{"pgid":"62.17","version":"229'57","reported_seq":200,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.141031+0000","last_change":"2024-02-23T00:45:03.246907+0000","last_active":"2024-02-23T00:45:06.141031+0000","last_peered":"2024-02-23T00:45:06.141031+0000","last_clean":"2024-02-23T00:45:06.141031+0000","last_became_active":"2024-02-23T00:43:17.843525+0000","last_became_peered":"2024-02-23T00:43:17.843525+0000","last_unstale":"2024-02-23T00:45:06.141031+0000","last_undegraded":"2024-02-23T00:45:06.141031+0000","last_fullsized":"2024-02-23T00:45:06.141031+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":57,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":18,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"}]},{"pgid":"62.19","version":"232'88","reported_seq":218,"reported_epoch":234,"state":"active+clean+snaptrim","last_fresh":"2024-02-23T00:45:06.137497+0000","last_change":"2024-02-23T00:45:03.244446+0000","last_active":"2024-02-23T00:45:06.137497+0000","last_peered":"2024-02-23T00:45:06.137497+0000","last_clean":"2024-02-23T00:45:06.137497+0000","last_became_active":"2024-02-23T00:43:17.815830+0000","last_became_peered":"2024-02-23T00:43:17.815830+0000","last_unstale":"2024-02-23T00:45:06.137497+0000","last_undegraded":"2024-02-23T00:45:06.137497+0000","last_fullsized":"2024-02-23T00:45:06.137497+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":88,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":71,"num_objects":7,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6","length":"1"}]},{"pgid":"62.18","version":"150'5","reported_seq":151,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.124801+0000","last_change":"2024-02-23T00:44:38.117270+0000","last_active":"2024-02-23T00:45:06.124801+0000","last_peered":"2024-02-23T00:45:06.124801+0000","last_clean":"2024-02-23T00:45:06.124801+0000","last_became_active":"2024-02-23T00:43:17.840351+0000","last_became_peered":"2024-02-23T00:43:17.840351+0000","last_unstale":"2024-02-23T00:45:06.124801+0000","last_undegraded":"2024-02-23T00:45:06.124801+0000","last_fullsized":"2024-02-23T00:45:06.124801+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"}]},{"pgid":"62.1b","version":"229'68","reported_seq":198,"reported_epoch":234,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-23T00:45:06.128895+0000","last_change":"2024-02-23T00:45:03.240741+0000","last_active":"2024-02-23T00:45:06.128895+0000","last_peered":"2024-02-23T00:45:06.128895+0000","last_clean":"2024-02-23T00:45:06.128895+0000","last_became_active":"2024-02-23T00:43:17.803043+0000","last_became_peered":"2024-02-23T00:43:17.803043+0000","last_unstale":"2024-02-23T00:45:06.128895+0000","last_undegraded":"2024-02-23T00:45:06.128895+0000","last_fullsized":"2024-02-23T00:45:06.128895+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6","length":"2"}]},{"pgid":"62.1a","version":"225'34","reported_seq":171,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.138777+0000","last_change":"2024-02-23T00:45:03.246692+0000","last_active":"2024-02-23T00:45:06.138777+0000","last_peered":"2024-02-23T00:45:06.138777+0000","last_clean":"2024-02-23T00:45:06.138777+0000","last_became_active":"2024-02-23T00:43:17.798285+0000","last_became_peered":"2024-02-23T00:43:17.798285+0000","last_unstale":"2024-02-23T00:45:06.138777+0000","last_undegraded":"2024-02-23T00:45:06.138777+0000","last_fullsized":"2024-02-23T00:45:06.138777+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"}]},{"pgid":"62.1d","version":"215'104","reported_seq":234,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.126720+0000","last_change":"2024-02-23T00:44:49.205689+0000","last_active":"2024-02-23T00:45:05.126720+0000","last_peered":"2024-02-23T00:45:05.126720+0000","last_clean":"2024-02-23T00:45:05.126720+0000","last_became_active":"2024-02-23T00:43:17.828194+0000","last_became_peered":"2024-02-23T00:43:17.828194+0000","last_unstale":"2024-02-23T00:45:05.126720+0000","last_undegraded":"2024-02-23T00:45:05.126720+0000","last_fullsized":"2024-02-23T00:45:05.126720+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":104,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":74,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":32,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"}]},{"pgid":"62.1c","version":"223'51","reported_seq":185,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.137659+0000","last_change":"2024-02-23T00:45:03.244386+0000","last_active":"2024-02-23T00:45:06.137659+0000","last_peered":"2024-02-23T00:45:06.137659+0000","last_clean":"2024-02-23T00:45:06.137659+0000","last_became_active":"2024-02-23T00:43:17.804718+0000","last_became_peered":"2024-02-23T00:43:17.804718+0000","last_unstale":"2024-02-23T00:45:06.137659+0000","last_undegraded":"2024-02-23T00:45:06.137659+0000","last_fullsized":"2024-02-23T00:45:06.137659+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"}]},{"pgid":"62.1f","version":"230'35","reported_seq":175,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.125546+0000","last_change":"2024-02-23T00:44:49.229045+0000","last_active":"2024-02-23T00:45:06.125546+0000","last_peered":"2024-02-23T00:45:06.125546+0000","last_clean":"2024-02-23T00:45:06.125546+0000","last_became_active":"2024-02-23T00:43:17.839914+0000","last_became_peered":"2024-02-23T00:43:17.839914+0000","last_unstale":"2024-02-23T00:45:06.125546+0000","last_undegraded":"2024-02-23T00:45:06.125546+0000","last_fullsized":"2024-02-23T00:45:06.125546+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":11,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"}]},{"pgid":"62.1e","version":"231'45","reported_seq":173,"reported_epoch":233,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-23T00:45:05.243868+0000","last_change":"2024-02-23T00:45:05.243868+0000","last_active":"2024-02-23T00:45:05.243868+0000","last_peered":"2024-02-23T00:45:05.243868+0000","last_clean":"2024-02-23T00:45:05.243868+0000","last_became_active":"2024-02-23T00:43:17.838682+0000","last_became_peered":"2024-02-23T00:43:17.838682+0000","last_unstale":"2024-02-23T00:45:05.243868+0000","last_undegraded":"2024-02-23T00:45:05.243868+0000","last_fullsized":"2024-02-23T00:45:05.243868+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":45,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":66,"num_objects":6,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"6","length":"1"}]},{"pgid":"5.9","version":"67'8","reported_seq":268,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.236420+0000","last_change":"2024-02-23T00:42:47.985647+0000","last_active":"2024-02-23T00:45:05.236420+0000","last_peered":"2024-02-23T00:45:05.236420+0000","last_clean":"2024-02-23T00:45:05.236420+0000","last_became_active":"2024-02-23T00:39:18.793414+0000","last_became_peered":"2024-02-23T00:39:18.793414+0000","last_unstale":"2024-02-23T00:45:05.236420+0000","last_undegraded":"2024-02-23T00:45:05.236420+0000","last_fullsized":"2024-02-23T00:45:05.236420+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:42:47.985573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:47.985573+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"68'27","reported_seq":251,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.122735+0000","last_change":"2024-02-23T00:45:04.138883+0000","last_active":"2024-02-23T00:45:05.122735+0000","last_peered":"2024-02-23T00:45:05.122735+0000","last_clean":"2024-02-23T00:45:05.122735+0000","last_became_active":"2024-02-23T00:39:16.859401+0000","last_became_peered":"2024-02-23T00:39:16.859401+0000","last_unstale":"2024-02-23T00:45:05.122735+0000","last_undegraded":"2024-02-23T00:45:05.122735+0000","last_fullsized":"2024-02-23T00:45:05.122735+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'27","last_scrub_stamp":"2024-02-23T00:45:04.138751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.138751+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":198,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.129791+0000","last_change":"2024-02-23T00:43:42.320439+0000","last_active":"2024-02-23T00:45:06.129791+0000","last_peered":"2024-02-23T00:45:06.129791+0000","last_clean":"2024-02-23T00:45:06.129791+0000","last_became_active":"2024-02-23T00:39:14.784436+0000","last_became_peered":"2024-02-23T00:39:14.784436+0000","last_unstale":"2024-02-23T00:45:06.129791+0000","last_undegraded":"2024-02-23T00:45:06.129791+0000","last_fullsized":"2024-02-23T00:45:06.129791+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:42.320346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:04.330426+0000","last_clean_scrub_stamp":"2024-02-23T00:43:42.320346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":210,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.124618+0000","last_change":"2024-02-23T00:44:21.972446+0000","last_active":"2024-02-23T00:45:05.124618+0000","last_peered":"2024-02-23T00:45:05.124618+0000","last_clean":"2024-02-23T00:45:05.124618+0000","last_became_active":"2024-02-23T00:43:32.282545+0000","last_became_peered":"2024-02-23T00:43:32.282545+0000","last_unstale":"2024-02-23T00:45:05.124618+0000","last_undegraded":"2024-02-23T00:45:05.124618+0000","last_fullsized":"2024-02-23T00:45:05.124618+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:21.972383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:21.972383+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.3","version":"110'5","reported_seq":140,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.129030+0000","last_change":"2024-02-23T00:42:58.836257+0000","last_active":"2024-02-23T00:45:06.129030+0000","last_peered":"2024-02-23T00:45:06.129030+0000","last_clean":"2024-02-23T00:45:06.129030+0000","last_became_active":"2024-02-23T00:42:58.835612+0000","last_became_peered":"2024-02-23T00:42:58.835612+0000","last_unstale":"2024-02-23T00:45:06.129030+0000","last_undegraded":"2024-02-23T00:45:06.129030+0000","last_fullsized":"2024-02-23T00:45:06.129030+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":134,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.125965+0000","last_change":"2024-02-23T00:42:58.832536+0000","last_active":"2024-02-23T00:45:05.125965+0000","last_peered":"2024-02-23T00:45:05.125965+0000","last_clean":"2024-02-23T00:45:05.125965+0000","last_became_active":"2024-02-23T00:42:58.828846+0000","last_became_peered":"2024-02-23T00:42:58.828846+0000","last_unstale":"2024-02-23T00:45:05.125965+0000","last_undegraded":"2024-02-23T00:45:05.125965+0000","last_fullsized":"2024-02-23T00:45:05.125965+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"68'15","reported_seq":227,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.318419+0000","last_change":"2024-02-23T00:43:54.355050+0000","last_active":"2024-02-23T00:45:05.318419+0000","last_peered":"2024-02-23T00:45:05.318419+0000","last_clean":"2024-02-23T00:45:05.318419+0000","last_became_active":"2024-02-23T00:39:16.783701+0000","last_became_peered":"2024-02-23T00:39:16.783701+0000","last_unstale":"2024-02-23T00:45:05.318419+0000","last_undegraded":"2024-02-23T00:45:05.318419+0000","last_fullsized":"2024-02-23T00:45:05.318419+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:43:54.354974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:54.354974+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":192,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.319043+0000","last_change":"2024-02-23T00:42:48.591600+0000","last_active":"2024-02-23T00:45:05.319043+0000","last_peered":"2024-02-23T00:45:05.319043+0000","last_clean":"2024-02-23T00:45:05.319043+0000","last_became_active":"2024-02-23T00:39:14.772493+0000","last_became_peered":"2024-02-23T00:39:14.772493+0000","last_unstale":"2024-02-23T00:45:05.319043+0000","last_undegraded":"2024-02-23T00:45:05.319043+0000","last_fullsized":"2024-02-23T00:45:05.319043+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:48.591486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:48.591486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":194,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.127237+0000","last_change":"2024-02-23T00:44:22.380954+0000","last_active":"2024-02-23T00:45:06.127237+0000","last_peered":"2024-02-23T00:45:06.127237+0000","last_clean":"2024-02-23T00:45:06.127237+0000","last_became_active":"2024-02-23T00:39:18.794476+0000","last_became_peered":"2024-02-23T00:39:18.794476+0000","last_unstale":"2024-02-23T00:45:06.127237+0000","last_undegraded":"2024-02-23T00:45:06.127237+0000","last_fullsized":"2024-02-23T00:45:06.127237+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:22.380841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:08.335330+0000","last_clean_scrub_stamp":"2024-02-23T00:44:22.380841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":205,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.313377+0000","last_change":"2024-02-23T00:43:35.079353+0000","last_active":"2024-02-23T00:45:05.313377+0000","last_peered":"2024-02-23T00:45:05.313377+0000","last_clean":"2024-02-23T00:45:05.313377+0000","last_became_active":"2024-02-23T00:43:35.079004+0000","last_became_peered":"2024-02-23T00:43:35.079004+0000","last_unstale":"2024-02-23T00:45:05.313377+0000","last_undegraded":"2024-02-23T00:45:05.313377+0000","last_fullsized":"2024-02-23T00:45:05.313377+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:46.359646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:46.359646+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"110'5","reported_seq":145,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.483348+0000","last_change":"2024-02-23T00:45:06.483322+0000","last_active":"2024-02-23T00:45:06.483348+0000","last_peered":"2024-02-23T00:45:06.483348+0000","last_clean":"2024-02-23T00:45:06.483348+0000","last_became_active":"2024-02-23T00:42:58.741558+0000","last_became_peered":"2024-02-23T00:42:58.741558+0000","last_unstale":"2024-02-23T00:45:06.483348+0000","last_undegraded":"2024-02-23T00:45:06.483348+0000","last_fullsized":"2024-02-23T00:45:06.483348+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'5","last_scrub_stamp":"2024-02-23T00:45:06.483252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.483252+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":134,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.321146+0000","last_change":"2024-02-23T00:42:58.871003+0000","last_active":"2024-02-23T00:45:05.321146+0000","last_peered":"2024-02-23T00:45:05.321146+0000","last_clean":"2024-02-23T00:45:05.321146+0000","last_became_active":"2024-02-23T00:42:58.867305+0000","last_became_peered":"2024-02-23T00:42:58.867305+0000","last_unstale":"2024-02-23T00:45:05.321146+0000","last_undegraded":"2024-02-23T00:45:05.321146+0000","last_fullsized":"2024-02-23T00:45:05.321146+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"88'35","reported_seq":252,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.139934+0000","last_change":"2024-02-23T00:43:54.438282+0000","last_active":"2024-02-23T00:45:06.139934+0000","last_peered":"2024-02-23T00:45:06.139934+0000","last_clean":"2024-02-23T00:45:06.139934+0000","last_became_active":"2024-02-23T00:39:16.857413+0000","last_became_peered":"2024-02-23T00:39:16.857413+0000","last_unstale":"2024-02-23T00:45:06.139934+0000","last_undegraded":"2024-02-23T00:45:06.139934+0000","last_fullsized":"2024-02-23T00:45:06.139934+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'35","last_scrub_stamp":"2024-02-23T00:43:54.438168+0000","last_deep_scrub":"88'35","last_deep_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_clean_scrub_stamp":"2024-02-23T00:43:54.438168+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":198,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.126547+0000","last_change":"2024-02-23T00:43:53.317283+0000","last_active":"2024-02-23T00:45:06.126547+0000","last_peered":"2024-02-23T00:45:06.126547+0000","last_clean":"2024-02-23T00:45:06.126547+0000","last_became_active":"2024-02-23T00:39:14.776705+0000","last_became_peered":"2024-02-23T00:39:14.776705+0000","last_unstale":"2024-02-23T00:45:06.126547+0000","last_undegraded":"2024-02-23T00:45:06.126547+0000","last_fullsized":"2024-02-23T00:45:06.126547+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:53.317202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_clean_scrub_stamp":"2024-02-23T00:43:53.317202+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":194,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.128314+0000","last_change":"2024-02-23T00:43:44.313672+0000","last_active":"2024-02-23T00:45:06.128314+0000","last_peered":"2024-02-23T00:45:06.128314+0000","last_clean":"2024-02-23T00:45:06.128314+0000","last_became_active":"2024-02-23T00:39:18.792718+0000","last_became_peered":"2024-02-23T00:39:18.792718+0000","last_unstale":"2024-02-23T00:45:06.128314+0000","last_undegraded":"2024-02-23T00:45:06.128314+0000","last_fullsized":"2024-02-23T00:45:06.128314+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:44.313501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:44.313501+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":208,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.235371+0000","last_change":"2024-02-23T00:44:36.758043+0000","last_active":"2024-02-23T00:45:05.235371+0000","last_peered":"2024-02-23T00:45:05.235371+0000","last_clean":"2024-02-23T00:45:05.235371+0000","last_became_active":"2024-02-23T00:43:39.470999+0000","last_became_peered":"2024-02-23T00:43:39.470999+0000","last_unstale":"2024-02-23T00:45:05.235371+0000","last_undegraded":"2024-02-23T00:45:05.235371+0000","last_fullsized":"2024-02-23T00:45:05.235371+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:36.757960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:36.757960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"110'5","reported_seq":140,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.138579+0000","last_change":"2024-02-23T00:42:58.688988+0000","last_active":"2024-02-23T00:45:06.138579+0000","last_peered":"2024-02-23T00:45:06.138579+0000","last_clean":"2024-02-23T00:45:06.138579+0000","last_became_active":"2024-02-23T00:42:58.684001+0000","last_became_peered":"2024-02-23T00:42:58.684001+0000","last_unstale":"2024-02-23T00:45:06.138579+0000","last_undegraded":"2024-02-23T00:45:06.138579+0000","last_fullsized":"2024-02-23T00:45:06.138579+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":134,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.316610+0000","last_change":"2024-02-23T00:42:57.641727+0000","last_active":"2024-02-23T00:45:05.316610+0000","last_peered":"2024-02-23T00:45:05.316610+0000","last_clean":"2024-02-23T00:45:05.316610+0000","last_became_active":"2024-02-23T00:42:57.639013+0000","last_became_peered":"2024-02-23T00:42:57.639013+0000","last_unstale":"2024-02-23T00:45:05.316610+0000","last_undegraded":"2024-02-23T00:45:05.316610+0000","last_fullsized":"2024-02-23T00:45:05.316610+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"67'17","reported_seq":246,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.130329+0000","last_change":"2024-02-23T00:43:45.194207+0000","last_active":"2024-02-23T00:45:06.130329+0000","last_peered":"2024-02-23T00:45:06.130329+0000","last_clean":"2024-02-23T00:45:06.130329+0000","last_became_active":"2024-02-23T00:40:42.933806+0000","last_became_peered":"2024-02-23T00:40:42.933806+0000","last_unstale":"2024-02-23T00:45:06.130329+0000","last_undegraded":"2024-02-23T00:45:06.130329+0000","last_fullsized":"2024-02-23T00:45:06.130329+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'17","last_scrub_stamp":"2024-02-23T00:43:45.194139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.194139+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":169,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.128454+0000","last_change":"2024-02-23T00:43:31.362599+0000","last_active":"2024-02-23T00:45:06.128454+0000","last_peered":"2024-02-23T00:45:06.128454+0000","last_clean":"2024-02-23T00:45:06.128454+0000","last_became_active":"2024-02-23T00:40:42.931041+0000","last_became_peered":"2024-02-23T00:40:42.931041+0000","last_unstale":"2024-02-23T00:45:06.128454+0000","last_undegraded":"2024-02-23T00:45:06.128454+0000","last_fullsized":"2024-02-23T00:45:06.128454+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:31.362510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:31.362510+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":193,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.319172+0000","last_change":"2024-02-23T00:44:22.804352+0000","last_active":"2024-02-23T00:45:05.319172+0000","last_peered":"2024-02-23T00:45:05.319172+0000","last_clean":"2024-02-23T00:45:05.319172+0000","last_became_active":"2024-02-23T00:39:18.799875+0000","last_became_peered":"2024-02-23T00:39:18.799875+0000","last_unstale":"2024-02-23T00:45:05.319172+0000","last_undegraded":"2024-02-23T00:45:05.319172+0000","last_fullsized":"2024-02-23T00:45:05.319172+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:22.804279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:22.804279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":210,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.126387+0000","last_change":"2024-02-23T00:44:36.481327+0000","last_active":"2024-02-23T00:45:06.126387+0000","last_peered":"2024-02-23T00:45:06.126387+0000","last_clean":"2024-02-23T00:45:06.126387+0000","last_became_active":"2024-02-23T00:43:08.885257+0000","last_became_peered":"2024-02-23T00:43:08.885257+0000","last_unstale":"2024-02-23T00:45:06.126387+0000","last_undegraded":"2024-02-23T00:45:06.126387+0000","last_fullsized":"2024-02-23T00:45:06.126387+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:36.481256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:36.481256+0000","last_clean_scrub_stamp":"2024-02-23T00:44:36.481256+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.6","version":"110'9","reported_seq":143,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.321228+0000","last_change":"2024-02-23T00:42:58.871934+0000","last_active":"2024-02-23T00:45:05.321228+0000","last_peered":"2024-02-23T00:45:05.321228+0000","last_clean":"2024-02-23T00:45:05.321228+0000","last_became_active":"2024-02-23T00:42:58.870147+0000","last_became_peered":"2024-02-23T00:42:58.870147+0000","last_unstale":"2024-02-23T00:45:05.321228+0000","last_undegraded":"2024-02-23T00:45:05.321228+0000","last_fullsized":"2024-02-23T00:45:05.321228+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":134,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.319337+0000","last_change":"2024-02-23T00:42:57.683188+0000","last_active":"2024-02-23T00:45:05.319337+0000","last_peered":"2024-02-23T00:45:05.319337+0000","last_clean":"2024-02-23T00:45:05.319337+0000","last_became_active":"2024-02-23T00:42:57.675306+0000","last_became_peered":"2024-02-23T00:42:57.675306+0000","last_unstale":"2024-02-23T00:45:05.319337+0000","last_undegraded":"2024-02-23T00:45:05.319337+0000","last_fullsized":"2024-02-23T00:45:05.319337+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"67'10","reported_seq":206,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.134021+0000","last_change":"2024-02-23T00:42:50.078150+0000","last_active":"2024-02-23T00:45:06.134021+0000","last_peered":"2024-02-23T00:45:06.134021+0000","last_clean":"2024-02-23T00:45:06.134021+0000","last_became_active":"2024-02-23T00:39:16.777530+0000","last_became_peered":"2024-02-23T00:39:16.777530+0000","last_unstale":"2024-02-23T00:45:06.134021+0000","last_undegraded":"2024-02-23T00:45:06.134021+0000","last_fullsized":"2024-02-23T00:45:06.134021+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'10","last_scrub_stamp":"2024-02-23T00:42:50.078042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:50.078042+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":192,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.315207+0000","last_change":"2024-02-23T00:42:06.575346+0000","last_active":"2024-02-23T00:45:05.315207+0000","last_peered":"2024-02-23T00:45:05.315207+0000","last_clean":"2024-02-23T00:45:05.315207+0000","last_became_active":"2024-02-23T00:39:14.780379+0000","last_became_peered":"2024-02-23T00:39:14.780379+0000","last_unstale":"2024-02-23T00:45:05.315207+0000","last_undegraded":"2024-02-23T00:45:05.315207+0000","last_fullsized":"2024-02-23T00:45:05.315207+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:06.575242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.575242+0000","last_clean_scrub_stamp":"2024-02-23T00:42:06.575242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"67'8","reported_seq":272,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.132152+0000","last_change":"2024-02-23T00:44:25.470245+0000","last_active":"2024-02-23T00:45:06.132152+0000","last_peered":"2024-02-23T00:45:06.132152+0000","last_clean":"2024-02-23T00:45:06.132152+0000","last_became_active":"2024-02-23T00:39:18.799544+0000","last_became_peered":"2024-02-23T00:39:18.799544+0000","last_unstale":"2024-02-23T00:45:06.132152+0000","last_undegraded":"2024-02-23T00:45:06.132152+0000","last_fullsized":"2024-02-23T00:45:06.132152+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:44:25.470174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:25.470174+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":207,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.131426+0000","last_change":"2024-02-23T00:43:36.205958+0000","last_active":"2024-02-23T00:45:06.131426+0000","last_peered":"2024-02-23T00:45:06.131426+0000","last_clean":"2024-02-23T00:45:06.131426+0000","last_became_active":"2024-02-23T00:43:13.338697+0000","last_became_peered":"2024-02-23T00:43:13.338697+0000","last_unstale":"2024-02-23T00:45:06.131426+0000","last_undegraded":"2024-02-23T00:45:06.131426+0000","last_fullsized":"2024-02-23T00:45:06.131426+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:36.205886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:15.081153+0000","last_clean_scrub_stamp":"2024-02-23T00:43:36.205886+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":134,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.316817+0000","last_change":"2024-02-23T00:42:57.686234+0000","last_active":"2024-02-23T00:45:05.316817+0000","last_peered":"2024-02-23T00:45:05.316817+0000","last_clean":"2024-02-23T00:45:05.316817+0000","last_became_active":"2024-02-23T00:42:57.685579+0000","last_became_peered":"2024-02-23T00:42:57.685579+0000","last_unstale":"2024-02-23T00:45:05.316817+0000","last_undegraded":"2024-02-23T00:45:05.316817+0000","last_fullsized":"2024-02-23T00:45:05.316817+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":135,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.124950+0000","last_change":"2024-02-23T00:42:58.772582+0000","last_active":"2024-02-23T00:45:06.124950+0000","last_peered":"2024-02-23T00:45:06.124950+0000","last_clean":"2024-02-23T00:45:06.124950+0000","last_became_active":"2024-02-23T00:42:58.765106+0000","last_became_peered":"2024-02-23T00:42:58.765106+0000","last_unstale":"2024-02-23T00:45:06.124950+0000","last_undegraded":"2024-02-23T00:45:06.124950+0000","last_fullsized":"2024-02-23T00:45:06.124950+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"67'15","reported_seq":220,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.318559+0000","last_change":"2024-02-23T00:44:19.830383+0000","last_active":"2024-02-23T00:45:05.318559+0000","last_peered":"2024-02-23T00:45:05.318559+0000","last_clean":"2024-02-23T00:45:05.318559+0000","last_became_active":"2024-02-23T00:39:16.789757+0000","last_became_peered":"2024-02-23T00:39:16.789757+0000","last_unstale":"2024-02-23T00:45:05.318559+0000","last_undegraded":"2024-02-23T00:45:05.318559+0000","last_fullsized":"2024-02-23T00:45:05.318559+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:44:19.830309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:44:19.830309+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":197,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.315565+0000","last_change":"2024-02-23T00:44:29.773000+0000","last_active":"2024-02-23T00:45:05.315565+0000","last_peered":"2024-02-23T00:45:05.315565+0000","last_clean":"2024-02-23T00:45:05.315565+0000","last_became_active":"2024-02-23T00:39:14.780526+0000","last_became_peered":"2024-02-23T00:39:14.780526+0000","last_unstale":"2024-02-23T00:45:05.315565+0000","last_undegraded":"2024-02-23T00:45:05.315565+0000","last_fullsized":"2024-02-23T00:45:05.315565+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:29.772933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:29.772933+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":272,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.131336+0000","last_change":"2024-02-23T00:44:23.109508+0000","last_active":"2024-02-23T00:45:06.131336+0000","last_peered":"2024-02-23T00:45:06.131336+0000","last_clean":"2024-02-23T00:45:06.131336+0000","last_became_active":"2024-02-23T00:39:18.800440+0000","last_became_peered":"2024-02-23T00:39:18.800440+0000","last_unstale":"2024-02-23T00:45:06.131336+0000","last_undegraded":"2024-02-23T00:45:06.131336+0000","last_fullsized":"2024-02-23T00:45:06.131336+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:44:23.109439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.109439+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":210,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.127858+0000","last_change":"2024-02-23T00:44:22.976713+0000","last_active":"2024-02-23T00:45:05.127858+0000","last_peered":"2024-02-23T00:45:05.127858+0000","last_clean":"2024-02-23T00:45:05.127858+0000","last_became_active":"2024-02-23T00:43:16.650527+0000","last_became_peered":"2024-02-23T00:43:16.650527+0000","last_unstale":"2024-02-23T00:45:05.127858+0000","last_undegraded":"2024-02-23T00:45:05.127858+0000","last_fullsized":"2024-02-23T00:45:05.127858+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:22.976611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:10.666299+0000","last_clean_scrub_stamp":"2024-02-23T00:44:22.976611+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":134,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.316700+0000","last_change":"2024-02-23T00:42:57.671776+0000","last_active":"2024-02-23T00:45:05.316700+0000","last_peered":"2024-02-23T00:45:05.316700+0000","last_clean":"2024-02-23T00:45:05.316700+0000","last_became_active":"2024-02-23T00:42:57.671472+0000","last_became_peered":"2024-02-23T00:42:57.671472+0000","last_unstale":"2024-02-23T00:45:05.316700+0000","last_undegraded":"2024-02-23T00:45:05.316700+0000","last_fullsized":"2024-02-23T00:45:05.316700+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,3],"acting":[1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":135,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.128016+0000","last_change":"2024-02-23T00:42:58.834770+0000","last_active":"2024-02-23T00:45:06.128016+0000","last_peered":"2024-02-23T00:45:06.128016+0000","last_clean":"2024-02-23T00:45:06.128016+0000","last_became_active":"2024-02-23T00:42:58.833924+0000","last_became_peered":"2024-02-23T00:42:58.833924+0000","last_unstale":"2024-02-23T00:45:06.128016+0000","last_undegraded":"2024-02-23T00:45:06.128016+0000","last_fullsized":"2024-02-23T00:45:06.128016+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"68'17","reported_seq":248,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.131177+0000","last_change":"2024-02-23T00:43:33.151471+0000","last_active":"2024-02-23T00:45:06.131177+0000","last_peered":"2024-02-23T00:45:06.131177+0000","last_clean":"2024-02-23T00:45:06.131177+0000","last_became_active":"2024-02-23T00:40:42.933759+0000","last_became_peered":"2024-02-23T00:40:42.933759+0000","last_unstale":"2024-02-23T00:45:06.131177+0000","last_undegraded":"2024-02-23T00:45:06.131177+0000","last_fullsized":"2024-02-23T00:45:06.131177+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:43:33.151401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:33.151401+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":196,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.235681+0000","last_change":"2024-02-23T00:44:18.886545+0000","last_active":"2024-02-23T00:45:05.235681+0000","last_peered":"2024-02-23T00:45:05.235681+0000","last_clean":"2024-02-23T00:45:05.235681+0000","last_became_active":"2024-02-23T00:39:14.784121+0000","last_became_peered":"2024-02-23T00:39:14.784121+0000","last_unstale":"2024-02-23T00:45:05.235681+0000","last_undegraded":"2024-02-23T00:45:05.235681+0000","last_fullsized":"2024-02-23T00:45:05.235681+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:18.886477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:18.886477+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":189,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.124979+0000","last_change":"2024-02-23T00:42:53.965005+0000","last_active":"2024-02-23T00:45:05.124979+0000","last_peered":"2024-02-23T00:45:05.124979+0000","last_clean":"2024-02-23T00:45:05.124979+0000","last_became_active":"2024-02-23T00:39:18.800777+0000","last_became_peered":"2024-02-23T00:39:18.800777+0000","last_unstale":"2024-02-23T00:45:05.124979+0000","last_undegraded":"2024-02-23T00:45:05.124979+0000","last_fullsized":"2024-02-23T00:45:05.124979+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.964934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.964934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"67'1","reported_seq":211,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.129549+0000","last_change":"2024-02-23T00:43:56.091057+0000","last_active":"2024-02-23T00:45:06.129549+0000","last_peered":"2024-02-23T00:45:06.129549+0000","last_clean":"2024-02-23T00:45:06.129549+0000","last_became_active":"2024-02-23T00:43:22.866277+0000","last_became_peered":"2024-02-23T00:43:22.866277+0000","last_unstale":"2024-02-23T00:45:06.129549+0000","last_undegraded":"2024-02-23T00:45:06.129549+0000","last_fullsized":"2024-02-23T00:45:06.129549+0000","mapping_epoch":65,"log_start":"67'1","ondisk_log_start":"67'1","created":65,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:43:56.090982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:09.079918+0000","last_clean_scrub_stamp":"2024-02-23T00:43:56.090982+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":140,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.651895+0000","last_change":"2024-02-23T00:45:06.651852+0000","last_active":"2024-02-23T00:45:06.651895+0000","last_peered":"2024-02-23T00:45:06.651895+0000","last_clean":"2024-02-23T00:45:06.651895+0000","last_became_active":"2024-02-23T00:42:58.718615+0000","last_became_peered":"2024-02-23T00:42:58.718615+0000","last_unstale":"2024-02-23T00:45:06.651895+0000","last_undegraded":"2024-02-23T00:45:06.651895+0000","last_fullsized":"2024-02-23T00:45:06.651895+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.651711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.651711+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":134,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.313446+0000","last_change":"2024-02-23T00:42:57.655252+0000","last_active":"2024-02-23T00:45:05.313446+0000","last_peered":"2024-02-23T00:45:05.313446+0000","last_clean":"2024-02-23T00:45:05.313446+0000","last_became_active":"2024-02-23T00:42:57.653279+0000","last_became_peered":"2024-02-23T00:42:57.653279+0000","last_unstale":"2024-02-23T00:45:05.313446+0000","last_undegraded":"2024-02-23T00:45:05.313446+0000","last_fullsized":"2024-02-23T00:45:05.313446+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":268,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.125239+0000","last_change":"2024-02-23T00:42:53.978054+0000","last_active":"2024-02-23T00:45:06.125239+0000","last_peered":"2024-02-23T00:45:06.125239+0000","last_clean":"2024-02-23T00:45:06.125239+0000","last_became_active":"2024-02-23T00:39:18.798536+0000","last_became_peered":"2024-02-23T00:39:18.798536+0000","last_unstale":"2024-02-23T00:45:06.125239+0000","last_undegraded":"2024-02-23T00:45:06.125239+0000","last_fullsized":"2024-02-23T00:45:06.125239+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:42:53.977950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.977950+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"68'20","reported_seq":232,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.128296+0000","last_change":"2024-02-23T00:45:05.291909+0000","last_active":"2024-02-23T00:45:06.128296+0000","last_peered":"2024-02-23T00:45:06.128296+0000","last_clean":"2024-02-23T00:45:06.128296+0000","last_became_active":"2024-02-23T00:39:16.780130+0000","last_became_peered":"2024-02-23T00:39:16.780130+0000","last_unstale":"2024-02-23T00:45:06.128296+0000","last_undegraded":"2024-02-23T00:45:06.128296+0000","last_fullsized":"2024-02-23T00:45:06.128296+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'20","last_scrub_stamp":"2024-02-23T00:45:05.291836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:05.291836+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":200,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.141348+0000","last_change":"2024-02-23T00:43:51.391999+0000","last_active":"2024-02-23T00:45:06.141348+0000","last_peered":"2024-02-23T00:45:06.141348+0000","last_clean":"2024-02-23T00:45:06.141348+0000","last_became_active":"2024-02-23T00:39:14.781999+0000","last_became_peered":"2024-02-23T00:39:14.781999+0000","last_unstale":"2024-02-23T00:45:06.141348+0000","last_undegraded":"2024-02-23T00:45:06.141348+0000","last_fullsized":"2024-02-23T00:45:06.141348+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:51.391939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:51.391939+0000","last_clean_scrub_stamp":"2024-02-23T00:43:51.391939+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":190,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.234298+0000","last_change":"2024-02-23T00:44:24.759694+0000","last_active":"2024-02-23T00:45:05.234298+0000","last_peered":"2024-02-23T00:45:05.234298+0000","last_clean":"2024-02-23T00:45:05.234298+0000","last_became_active":"2024-02-23T00:39:21.161928+0000","last_became_peered":"2024-02-23T00:39:21.161928+0000","last_unstale":"2024-02-23T00:45:05.234298+0000","last_undegraded":"2024-02-23T00:45:05.234298+0000","last_fullsized":"2024-02-23T00:45:05.234298+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:24.759627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.945642+0000","last_clean_scrub_stamp":"2024-02-23T00:44:24.759627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"110'10","reported_seq":145,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.127426+0000","last_change":"2024-02-23T00:42:58.696579+0000","last_active":"2024-02-23T00:45:06.127426+0000","last_peered":"2024-02-23T00:45:06.127426+0000","last_clean":"2024-02-23T00:45:06.127426+0000","last_became_active":"2024-02-23T00:42:58.696292+0000","last_became_peered":"2024-02-23T00:42:58.696292+0000","last_unstale":"2024-02-23T00:45:06.127426+0000","last_undegraded":"2024-02-23T00:45:06.127426+0000","last_fullsized":"2024-02-23T00:45:06.127426+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":134,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.126488+0000","last_change":"2024-02-23T00:42:58.717788+0000","last_active":"2024-02-23T00:45:05.126488+0000","last_peered":"2024-02-23T00:45:05.126488+0000","last_clean":"2024-02-23T00:45:05.126488+0000","last_became_active":"2024-02-23T00:42:58.717371+0000","last_became_peered":"2024-02-23T00:42:58.717371+0000","last_unstale":"2024-02-23T00:45:05.126488+0000","last_undegraded":"2024-02-23T00:45:05.126488+0000","last_fullsized":"2024-02-23T00:45:05.126488+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":194,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.128536+0000","last_change":"2024-02-23T00:44:34.059194+0000","last_active":"2024-02-23T00:45:06.128536+0000","last_peered":"2024-02-23T00:45:06.128536+0000","last_clean":"2024-02-23T00:45:06.128536+0000","last_became_active":"2024-02-23T00:39:18.798829+0000","last_became_peered":"2024-02-23T00:39:18.798829+0000","last_unstale":"2024-02-23T00:45:06.128536+0000","last_undegraded":"2024-02-23T00:45:06.128536+0000","last_fullsized":"2024-02-23T00:45:06.128536+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:34.059129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:34.059129+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"68'30","reported_seq":254,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.125145+0000","last_change":"2024-02-23T00:42:49.970825+0000","last_active":"2024-02-23T00:45:06.125145+0000","last_peered":"2024-02-23T00:45:06.125145+0000","last_clean":"2024-02-23T00:45:06.125145+0000","last_became_active":"2024-02-23T00:39:16.859002+0000","last_became_peered":"2024-02-23T00:39:16.859002+0000","last_unstale":"2024-02-23T00:45:06.125145+0000","last_undegraded":"2024-02-23T00:45:06.125145+0000","last_fullsized":"2024-02-23T00:45:06.125145+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'30","last_scrub_stamp":"2024-02-23T00:42:49.970752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:49.970752+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":192,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.316837+0000","last_change":"2024-02-23T00:42:40.534390+0000","last_active":"2024-02-23T00:45:05.316837+0000","last_peered":"2024-02-23T00:45:05.316837+0000","last_clean":"2024-02-23T00:45:05.316837+0000","last_became_active":"2024-02-23T00:39:14.779193+0000","last_became_peered":"2024-02-23T00:39:14.779193+0000","last_unstale":"2024-02-23T00:45:05.316837+0000","last_undegraded":"2024-02-23T00:45:05.316837+0000","last_fullsized":"2024-02-23T00:45:05.316837+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:40.534284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:40.534284+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":192,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.130449+0000","last_change":"2024-02-23T00:45:03.228087+0000","last_active":"2024-02-23T00:45:06.130449+0000","last_peered":"2024-02-23T00:45:06.130449+0000","last_clean":"2024-02-23T00:45:06.130449+0000","last_became_active":"2024-02-23T00:39:20.795271+0000","last_became_peered":"2024-02-23T00:39:20.795271+0000","last_unstale":"2024-02-23T00:45:06.130449+0000","last_undegraded":"2024-02-23T00:45:06.130449+0000","last_fullsized":"2024-02-23T00:45:06.130449+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:03.227995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:45:03.227995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"110'2","reported_seq":137,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.128254+0000","last_change":"2024-02-23T00:42:58.816302+0000","last_active":"2024-02-23T00:45:06.128254+0000","last_peered":"2024-02-23T00:45:06.128254+0000","last_clean":"2024-02-23T00:45:06.128254+0000","last_became_active":"2024-02-23T00:42:58.814093+0000","last_became_peered":"2024-02-23T00:42:58.814093+0000","last_unstale":"2024-02-23T00:45:06.128254+0000","last_undegraded":"2024-02-23T00:45:06.128254+0000","last_fullsized":"2024-02-23T00:45:06.128254+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":135,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.133419+0000","last_change":"2024-02-23T00:42:58.809948+0000","last_active":"2024-02-23T00:45:06.133419+0000","last_peered":"2024-02-23T00:45:06.133419+0000","last_clean":"2024-02-23T00:45:06.133419+0000","last_became_active":"2024-02-23T00:42:58.809283+0000","last_became_peered":"2024-02-23T00:42:58.809283+0000","last_unstale":"2024-02-23T00:45:06.133419+0000","last_undegraded":"2024-02-23T00:45:06.133419+0000","last_fullsized":"2024-02-23T00:45:06.133419+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":194,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.139837+0000","last_change":"2024-02-23T00:44:19.312509+0000","last_active":"2024-02-23T00:45:06.139837+0000","last_peered":"2024-02-23T00:45:06.139837+0000","last_clean":"2024-02-23T00:45:06.139837+0000","last_became_active":"2024-02-23T00:39:18.796664+0000","last_became_peered":"2024-02-23T00:39:18.796664+0000","last_unstale":"2024-02-23T00:45:06.139837+0000","last_undegraded":"2024-02-23T00:45:06.139837+0000","last_fullsized":"2024-02-23T00:45:06.139837+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:19.312418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:19.312418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"203'34","reported_seq":256,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.318323+0000","last_change":"2024-02-23T00:44:35.644033+0000","last_active":"2024-02-23T00:45:05.318323+0000","last_peered":"2024-02-23T00:45:05.318323+0000","last_clean":"2024-02-23T00:45:05.318323+0000","last_became_active":"2024-02-23T00:39:16.857278+0000","last_became_peered":"2024-02-23T00:39:16.857278+0000","last_unstale":"2024-02-23T00:45:05.318323+0000","last_undegraded":"2024-02-23T00:45:05.318323+0000","last_fullsized":"2024-02-23T00:45:05.318323+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'34","last_scrub_stamp":"2024-02-23T00:44:35.643931+0000","last_deep_scrub":"203'34","last_deep_scrub_stamp":"2024-02-23T00:44:35.643931+0000","last_clean_scrub_stamp":"2024-02-23T00:44:35.643931+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":191,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.239346+0000","last_change":"2024-02-23T00:42:43.990569+0000","last_active":"2024-02-23T00:45:05.239346+0000","last_peered":"2024-02-23T00:45:05.239346+0000","last_clean":"2024-02-23T00:45:05.239346+0000","last_became_active":"2024-02-23T00:39:14.783662+0000","last_became_peered":"2024-02-23T00:39:14.783662+0000","last_unstale":"2024-02-23T00:45:05.239346+0000","last_undegraded":"2024-02-23T00:45:05.239346+0000","last_fullsized":"2024-02-23T00:45:05.239346+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:43.990486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:43.990486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":191,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.310188+0000","last_change":"2024-02-23T00:45:04.585266+0000","last_active":"2024-02-23T00:45:05.310188+0000","last_peered":"2024-02-23T00:45:05.310188+0000","last_clean":"2024-02-23T00:45:05.310188+0000","last_became_active":"2024-02-23T00:39:21.160060+0000","last_became_peered":"2024-02-23T00:39:21.160060+0000","last_unstale":"2024-02-23T00:45:05.310188+0000","last_undegraded":"2024-02-23T00:45:05.310188+0000","last_fullsized":"2024-02-23T00:45:05.310188+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:04.585179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.585179+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.8","version":"110'2","reported_seq":142,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.128544+0000","last_change":"2024-02-23T00:44:33.389680+0000","last_active":"2024-02-23T00:45:06.128544+0000","last_peered":"2024-02-23T00:45:06.128544+0000","last_clean":"2024-02-23T00:45:06.128544+0000","last_became_active":"2024-02-23T00:42:58.809584+0000","last_became_peered":"2024-02-23T00:42:58.809584+0000","last_unstale":"2024-02-23T00:45:06.128544+0000","last_undegraded":"2024-02-23T00:45:06.128544+0000","last_fullsized":"2024-02-23T00:45:06.128544+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'2","last_scrub_stamp":"2024-02-23T00:44:33.389605+0000","last_deep_scrub":"110'2","last_deep_scrub_stamp":"2024-02-23T00:44:33.389605+0000","last_clean_scrub_stamp":"2024-02-23T00:44:33.389605+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":134,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.126872+0000","last_change":"2024-02-23T00:42:58.726955+0000","last_active":"2024-02-23T00:45:05.126872+0000","last_peered":"2024-02-23T00:45:05.126872+0000","last_clean":"2024-02-23T00:45:05.126872+0000","last_became_active":"2024-02-23T00:42:58.726652+0000","last_became_peered":"2024-02-23T00:42:58.726652+0000","last_unstale":"2024-02-23T00:45:05.126872+0000","last_undegraded":"2024-02-23T00:45:05.126872+0000","last_fullsized":"2024-02-23T00:45:05.126872+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":194,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.125095+0000","last_change":"2024-02-23T00:43:45.054194+0000","last_active":"2024-02-23T00:45:05.125095+0000","last_peered":"2024-02-23T00:45:05.125095+0000","last_clean":"2024-02-23T00:45:05.125095+0000","last_became_active":"2024-02-23T00:39:18.800378+0000","last_became_peered":"2024-02-23T00:39:18.800378+0000","last_unstale":"2024-02-23T00:45:05.125095+0000","last_undegraded":"2024-02-23T00:45:05.125095+0000","last_fullsized":"2024-02-23T00:45:05.125095+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:45.054115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:59.694033+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.054115+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"95'18","reported_seq":220,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.320800+0000","last_change":"2024-02-23T00:44:18.630435+0000","last_active":"2024-02-23T00:45:05.320800+0000","last_peered":"2024-02-23T00:45:05.320800+0000","last_clean":"2024-02-23T00:45:05.320800+0000","last_became_active":"2024-02-23T00:39:16.787143+0000","last_became_peered":"2024-02-23T00:39:16.787143+0000","last_unstale":"2024-02-23T00:45:05.320800+0000","last_undegraded":"2024-02-23T00:45:05.320800+0000","last_fullsized":"2024-02-23T00:45:05.320800+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'18","last_scrub_stamp":"2024-02-23T00:44:18.630363+0000","last_deep_scrub":"95'18","last_deep_scrub_stamp":"2024-02-23T00:42:01.195545+0000","last_clean_scrub_stamp":"2024-02-23T00:44:18.630363+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":196,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.235569+0000","last_change":"2024-02-23T00:42:50.947420+0000","last_active":"2024-02-23T00:45:05.235569+0000","last_peered":"2024-02-23T00:45:05.235569+0000","last_clean":"2024-02-23T00:45:05.235569+0000","last_became_active":"2024-02-23T00:39:14.784578+0000","last_became_peered":"2024-02-23T00:39:14.784578+0000","last_unstale":"2024-02-23T00:45:05.235569+0000","last_undegraded":"2024-02-23T00:45:05.235569+0000","last_fullsized":"2024-02-23T00:45:05.235569+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:50.947349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:50.947349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":198,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.312986+0000","last_change":"2024-02-23T00:43:48.258338+0000","last_active":"2024-02-23T00:45:05.312986+0000","last_peered":"2024-02-23T00:45:05.312986+0000","last_clean":"2024-02-23T00:45:05.312986+0000","last_became_active":"2024-02-23T00:39:14.775446+0000","last_became_peered":"2024-02-23T00:39:14.775446+0000","last_unstale":"2024-02-23T00:45:05.312986+0000","last_undegraded":"2024-02-23T00:45:05.312986+0000","last_fullsized":"2024-02-23T00:45:05.312986+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:48.258269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.258269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":186,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.319096+0000","last_change":"2024-02-23T00:43:47.905617+0000","last_active":"2024-02-23T00:45:05.319096+0000","last_peered":"2024-02-23T00:45:05.319096+0000","last_clean":"2024-02-23T00:45:05.319096+0000","last_became_active":"2024-02-23T00:39:20.797242+0000","last_became_peered":"2024-02-23T00:39:20.797242+0000","last_unstale":"2024-02-23T00:45:05.319096+0000","last_undegraded":"2024-02-23T00:45:05.319096+0000","last_fullsized":"2024-02-23T00:45:05.319096+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:47.905501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:47.905501+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"110'5","reported_seq":139,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.238282+0000","last_change":"2024-02-23T00:42:58.743470+0000","last_active":"2024-02-23T00:45:05.238282+0000","last_peered":"2024-02-23T00:45:05.238282+0000","last_clean":"2024-02-23T00:45:05.238282+0000","last_became_active":"2024-02-23T00:42:58.742998+0000","last_became_peered":"2024-02-23T00:42:58.742998+0000","last_unstale":"2024-02-23T00:45:05.238282+0000","last_undegraded":"2024-02-23T00:45:05.238282+0000","last_fullsized":"2024-02-23T00:45:05.238282+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":140,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.136016+0000","last_change":"2024-02-23T00:44:35.360128+0000","last_active":"2024-02-23T00:45:06.136016+0000","last_peered":"2024-02-23T00:45:06.136016+0000","last_clean":"2024-02-23T00:45:06.136016+0000","last_became_active":"2024-02-23T00:42:58.690858+0000","last_became_peered":"2024-02-23T00:42:58.690858+0000","last_unstale":"2024-02-23T00:45:06.136016+0000","last_undegraded":"2024-02-23T00:45:06.136016+0000","last_fullsized":"2024-02-23T00:45:06.136016+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:35.360054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:35.360054+0000","last_clean_scrub_stamp":"2024-02-23T00:44:35.360054+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":187,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.238270+0000","last_change":"2024-02-23T00:42:42.029769+0000","last_active":"2024-02-23T00:45:05.238270+0000","last_peered":"2024-02-23T00:45:05.238270+0000","last_clean":"2024-02-23T00:45:05.238270+0000","last_became_active":"2024-02-23T00:39:18.793801+0000","last_became_peered":"2024-02-23T00:39:18.793801+0000","last_unstale":"2024-02-23T00:45:05.238270+0000","last_undegraded":"2024-02-23T00:45:05.238270+0000","last_fullsized":"2024-02-23T00:45:05.238270+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:42.029652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:42.029652+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"67'14","reported_seq":218,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.137637+0000","last_change":"2024-02-23T00:43:44.369862+0000","last_active":"2024-02-23T00:45:06.137637+0000","last_peered":"2024-02-23T00:45:06.137637+0000","last_clean":"2024-02-23T00:45:06.137637+0000","last_became_active":"2024-02-23T00:39:16.858880+0000","last_became_peered":"2024-02-23T00:39:16.858880+0000","last_unstale":"2024-02-23T00:45:06.137637+0000","last_undegraded":"2024-02-23T00:45:06.137637+0000","last_fullsized":"2024-02-23T00:45:06.137637+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'14","last_scrub_stamp":"2024-02-23T00:43:44.369792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:44.369792+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":201,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.235567+0000","last_change":"2024-02-23T00:42:49.977357+0000","last_active":"2024-02-23T00:45:05.235567+0000","last_peered":"2024-02-23T00:45:05.235567+0000","last_clean":"2024-02-23T00:45:05.235567+0000","last_became_active":"2024-02-23T00:39:08.248249+0000","last_became_peered":"2024-02-23T00:39:08.248249+0000","last_unstale":"2024-02-23T00:45:05.235567+0000","last_undegraded":"2024-02-23T00:45:05.235567+0000","last_fullsized":"2024-02-23T00:45:05.235567+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:49.977248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:49.977248+0000","last_clean_scrub_stamp":"2024-02-23T00:42:49.977248+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":193,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.126013+0000","last_change":"2024-02-23T00:42:46.945425+0000","last_active":"2024-02-23T00:45:05.126013+0000","last_peered":"2024-02-23T00:45:05.126013+0000","last_clean":"2024-02-23T00:45:05.126013+0000","last_became_active":"2024-02-23T00:39:14.788888+0000","last_became_peered":"2024-02-23T00:39:14.788888+0000","last_unstale":"2024-02-23T00:45:05.126013+0000","last_undegraded":"2024-02-23T00:45:05.126013+0000","last_fullsized":"2024-02-23T00:45:05.126013+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:46.945358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:46.945358+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":199,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.124818+0000","last_change":"2024-02-23T00:44:33.143663+0000","last_active":"2024-02-23T00:45:06.124818+0000","last_peered":"2024-02-23T00:45:06.124818+0000","last_clean":"2024-02-23T00:45:06.124818+0000","last_became_active":"2024-02-23T00:39:14.783489+0000","last_became_peered":"2024-02-23T00:39:14.783489+0000","last_unstale":"2024-02-23T00:45:06.124818+0000","last_undegraded":"2024-02-23T00:45:06.124818+0000","last_fullsized":"2024-02-23T00:45:06.124818+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:33.143592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:33.143592+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":210,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.125206+0000","last_change":"2024-02-23T00:44:20.958888+0000","last_active":"2024-02-23T00:45:05.125206+0000","last_peered":"2024-02-23T00:45:05.125206+0000","last_clean":"2024-02-23T00:45:05.125206+0000","last_became_active":"2024-02-23T00:43:42.168435+0000","last_became_peered":"2024-02-23T00:43:42.168435+0000","last_unstale":"2024-02-23T00:45:05.125206+0000","last_undegraded":"2024-02-23T00:45:05.125206+0000","last_fullsized":"2024-02-23T00:45:05.125206+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:20.958797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:20.958797+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":135,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.129602+0000","last_change":"2024-02-23T00:42:58.721944+0000","last_active":"2024-02-23T00:45:06.129602+0000","last_peered":"2024-02-23T00:45:06.129602+0000","last_clean":"2024-02-23T00:45:06.129602+0000","last_became_active":"2024-02-23T00:42:58.721439+0000","last_became_peered":"2024-02-23T00:42:58.721439+0000","last_unstale":"2024-02-23T00:45:06.129602+0000","last_undegraded":"2024-02-23T00:45:06.129602+0000","last_fullsized":"2024-02-23T00:45:06.129602+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":134,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.238475+0000","last_change":"2024-02-23T00:42:58.701934+0000","last_active":"2024-02-23T00:45:05.238475+0000","last_peered":"2024-02-23T00:45:05.238475+0000","last_clean":"2024-02-23T00:45:05.238475+0000","last_became_active":"2024-02-23T00:42:58.701502+0000","last_became_peered":"2024-02-23T00:42:58.701502+0000","last_unstale":"2024-02-23T00:45:05.238475+0000","last_undegraded":"2024-02-23T00:45:05.238475+0000","last_fullsized":"2024-02-23T00:45:05.238475+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":194,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.129975+0000","last_change":"2024-02-23T00:43:46.293962+0000","last_active":"2024-02-23T00:45:06.129975+0000","last_peered":"2024-02-23T00:45:06.129975+0000","last_clean":"2024-02-23T00:45:06.129975+0000","last_became_active":"2024-02-23T00:39:18.789330+0000","last_became_peered":"2024-02-23T00:39:18.789330+0000","last_unstale":"2024-02-23T00:45:06.129975+0000","last_undegraded":"2024-02-23T00:45:06.129975+0000","last_fullsized":"2024-02-23T00:45:06.129975+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:46.293864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.264240+0000","last_clean_scrub_stamp":"2024-02-23T00:43:46.293864+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"68'19","reported_seq":232,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.319375+0000","last_change":"2024-02-23T00:42:04.625118+0000","last_active":"2024-02-23T00:45:05.319375+0000","last_peered":"2024-02-23T00:45:05.319375+0000","last_clean":"2024-02-23T00:45:05.319375+0000","last_became_active":"2024-02-23T00:39:16.787484+0000","last_became_peered":"2024-02-23T00:39:16.787484+0000","last_unstale":"2024-02-23T00:45:05.319375+0000","last_undegraded":"2024-02-23T00:45:05.319375+0000","last_fullsized":"2024-02-23T00:45:05.319375+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'19","last_scrub_stamp":"2024-02-23T00:42:04.625006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:04.625006+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":198,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.131656+0000","last_change":"2024-02-23T00:43:54.305009+0000","last_active":"2024-02-23T00:45:06.131656+0000","last_peered":"2024-02-23T00:45:06.131656+0000","last_clean":"2024-02-23T00:45:06.131656+0000","last_became_active":"2024-02-23T00:39:14.772411+0000","last_became_peered":"2024-02-23T00:39:14.772411+0000","last_unstale":"2024-02-23T00:45:06.131656+0000","last_undegraded":"2024-02-23T00:45:06.131656+0000","last_fullsized":"2024-02-23T00:45:06.131656+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:54.304948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:54.304948+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"68'5","reported_seq":539,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.143647+0000","last_change":"2024-02-23T00:43:48.876428+0000","last_active":"2024-02-23T00:45:06.143647+0000","last_peered":"2024-02-23T00:45:06.143647+0000","last_clean":"2024-02-23T00:45:06.143647+0000","last_became_active":"2024-02-23T00:39:14.779109+0000","last_became_peered":"2024-02-23T00:39:14.779109+0000","last_unstale":"2024-02-23T00:45:06.143647+0000","last_undegraded":"2024-02-23T00:45:06.143647+0000","last_fullsized":"2024-02-23T00:45:06.143647+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'5","last_scrub_stamp":"2024-02-23T00:43:48.876356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.876356+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":344,"num_read_kb":339,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":190,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.234376+0000","last_change":"2024-02-23T00:43:47.794616+0000","last_active":"2024-02-23T00:45:05.234376+0000","last_peered":"2024-02-23T00:45:05.234376+0000","last_clean":"2024-02-23T00:45:05.234376+0000","last_became_active":"2024-02-23T00:39:21.160457+0000","last_became_peered":"2024-02-23T00:39:21.160457+0000","last_unstale":"2024-02-23T00:45:05.234376+0000","last_undegraded":"2024-02-23T00:45:05.234376+0000","last_fullsized":"2024-02-23T00:45:05.234376+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:47.794534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:47.794534+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":134,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.124921+0000","last_change":"2024-02-23T00:42:58.831449+0000","last_active":"2024-02-23T00:45:05.124921+0000","last_peered":"2024-02-23T00:45:05.124921+0000","last_clean":"2024-02-23T00:45:05.124921+0000","last_became_active":"2024-02-23T00:42:58.830815+0000","last_became_peered":"2024-02-23T00:42:58.830815+0000","last_unstale":"2024-02-23T00:45:05.124921+0000","last_undegraded":"2024-02-23T00:45:05.124921+0000","last_fullsized":"2024-02-23T00:45:05.124921+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":134,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.315191+0000","last_change":"2024-02-23T00:42:57.653669+0000","last_active":"2024-02-23T00:45:05.315191+0000","last_peered":"2024-02-23T00:45:05.315191+0000","last_clean":"2024-02-23T00:45:05.315191+0000","last_became_active":"2024-02-23T00:42:57.653355+0000","last_became_peered":"2024-02-23T00:42:57.653355+0000","last_unstale":"2024-02-23T00:45:05.315191+0000","last_undegraded":"2024-02-23T00:45:05.315191+0000","last_fullsized":"2024-02-23T00:45:05.315191+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":194,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.321037+0000","last_change":"2024-02-23T00:44:20.626116+0000","last_active":"2024-02-23T00:45:05.321037+0000","last_peered":"2024-02-23T00:45:05.321037+0000","last_clean":"2024-02-23T00:45:05.321037+0000","last_became_active":"2024-02-23T00:39:18.802134+0000","last_became_peered":"2024-02-23T00:39:18.802134+0000","last_unstale":"2024-02-23T00:45:05.321037+0000","last_undegraded":"2024-02-23T00:45:05.321037+0000","last_fullsized":"2024-02-23T00:45:05.321037+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:20.626052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:20.626052+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"86'35","reported_seq":245,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.124672+0000","last_change":"2024-02-23T00:42:47.988747+0000","last_active":"2024-02-23T00:45:05.124672+0000","last_peered":"2024-02-23T00:45:05.124672+0000","last_clean":"2024-02-23T00:45:05.124672+0000","last_became_active":"2024-02-23T00:40:42.933322+0000","last_became_peered":"2024-02-23T00:40:42.933322+0000","last_unstale":"2024-02-23T00:45:05.124672+0000","last_undegraded":"2024-02-23T00:45:05.124672+0000","last_fullsized":"2024-02-23T00:45:05.124672+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'35","last_scrub_stamp":"2024-02-23T00:42:47.988676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:47.988676+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":198,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.125178+0000","last_change":"2024-02-23T00:44:23.965228+0000","last_active":"2024-02-23T00:45:05.125178+0000","last_peered":"2024-02-23T00:45:05.125178+0000","last_clean":"2024-02-23T00:45:05.125178+0000","last_became_active":"2024-02-23T00:39:14.784015+0000","last_became_peered":"2024-02-23T00:39:14.784015+0000","last_unstale":"2024-02-23T00:45:05.125178+0000","last_undegraded":"2024-02-23T00:45:05.125178+0000","last_fullsized":"2024-02-23T00:45:05.125178+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:23.965125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.965125+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":206,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.133275+0000","last_change":"2024-02-23T00:43:45.144499+0000","last_active":"2024-02-23T00:45:06.133275+0000","last_peered":"2024-02-23T00:45:06.133275+0000","last_clean":"2024-02-23T00:45:06.133275+0000","last_became_active":"2024-02-23T00:43:45.144043+0000","last_became_peered":"2024-02-23T00:43:45.144043+0000","last_unstale":"2024-02-23T00:45:06.133275+0000","last_undegraded":"2024-02-23T00:45:06.133275+0000","last_fullsized":"2024-02-23T00:45:06.133275+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:40.062493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:40.062493+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":140,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.141185+0000","last_change":"2024-02-23T00:44:31.410733+0000","last_active":"2024-02-23T00:45:06.141185+0000","last_peered":"2024-02-23T00:45:06.141185+0000","last_clean":"2024-02-23T00:45:06.141185+0000","last_became_active":"2024-02-23T00:42:58.754813+0000","last_became_peered":"2024-02-23T00:42:58.754813+0000","last_unstale":"2024-02-23T00:45:06.141185+0000","last_undegraded":"2024-02-23T00:45:06.141185+0000","last_fullsized":"2024-02-23T00:45:06.141185+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:31.410670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:31.410670+0000","last_clean_scrub_stamp":"2024-02-23T00:44:31.410670+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":140,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.126983+0000","last_change":"2024-02-23T00:45:05.490595+0000","last_active":"2024-02-23T00:45:06.126983+0000","last_peered":"2024-02-23T00:45:06.126983+0000","last_clean":"2024-02-23T00:45:06.126983+0000","last_became_active":"2024-02-23T00:42:58.766579+0000","last_became_peered":"2024-02-23T00:42:58.766579+0000","last_unstale":"2024-02-23T00:45:06.126983+0000","last_undegraded":"2024-02-23T00:45:06.126983+0000","last_fullsized":"2024-02-23T00:45:06.126983+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:05.490531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:05.490531+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"91'9","reported_seq":273,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.318519+0000","last_change":"2024-02-23T00:44:28.619036+0000","last_active":"2024-02-23T00:45:05.318519+0000","last_peered":"2024-02-23T00:45:05.318519+0000","last_clean":"2024-02-23T00:45:05.318519+0000","last_became_active":"2024-02-23T00:39:18.801297+0000","last_became_peered":"2024-02-23T00:39:18.801297+0000","last_unstale":"2024-02-23T00:45:05.318519+0000","last_undegraded":"2024-02-23T00:45:05.318519+0000","last_fullsized":"2024-02-23T00:45:05.318519+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'9","last_scrub_stamp":"2024-02-23T00:44:28.618923+0000","last_deep_scrub":"91'9","last_deep_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_clean_scrub_stamp":"2024-02-23T00:44:28.618923+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"68'16","reported_seq":220,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.317086+0000","last_change":"2024-02-23T00:42:34.633620+0000","last_active":"2024-02-23T00:45:05.317086+0000","last_peered":"2024-02-23T00:45:05.317086+0000","last_clean":"2024-02-23T00:45:05.317086+0000","last_became_active":"2024-02-23T00:39:16.787468+0000","last_became_peered":"2024-02-23T00:39:16.787468+0000","last_unstale":"2024-02-23T00:45:05.317086+0000","last_undegraded":"2024-02-23T00:45:05.317086+0000","last_fullsized":"2024-02-23T00:45:05.317086+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:42:34.633549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:34.633549+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":192,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.316877+0000","last_change":"2024-02-23T00:42:46.606249+0000","last_active":"2024-02-23T00:45:05.316877+0000","last_peered":"2024-02-23T00:45:05.316877+0000","last_clean":"2024-02-23T00:45:05.316877+0000","last_became_active":"2024-02-23T00:39:14.779952+0000","last_became_peered":"2024-02-23T00:39:14.779952+0000","last_unstale":"2024-02-23T00:45:05.316877+0000","last_undegraded":"2024-02-23T00:45:05.316877+0000","last_fullsized":"2024-02-23T00:45:05.316877+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:46.606185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:46.606185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"107'2","reported_seq":189,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.311997+0000","last_change":"2024-02-23T00:42:48.362807+0000","last_active":"2024-02-23T00:45:05.311997+0000","last_peered":"2024-02-23T00:45:05.311997+0000","last_clean":"2024-02-23T00:45:05.311997+0000","last_became_active":"2024-02-23T00:39:20.800858+0000","last_became_peered":"2024-02-23T00:39:20.800858+0000","last_unstale":"2024-02-23T00:45:05.311997+0000","last_undegraded":"2024-02-23T00:45:05.311997+0000","last_fullsized":"2024-02-23T00:45:05.311997+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'2","last_scrub_stamp":"2024-02-23T00:42:48.362739+0000","last_deep_scrub":"107'2","last_deep_scrub_stamp":"2024-02-23T00:42:48.362739+0000","last_clean_scrub_stamp":"2024-02-23T00:42:48.362739+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":135,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.139146+0000","last_change":"2024-02-23T00:42:58.727288+0000","last_active":"2024-02-23T00:45:06.139146+0000","last_peered":"2024-02-23T00:45:06.139146+0000","last_clean":"2024-02-23T00:45:06.139146+0000","last_became_active":"2024-02-23T00:42:58.726861+0000","last_became_peered":"2024-02-23T00:42:58.726861+0000","last_unstale":"2024-02-23T00:45:06.139146+0000","last_undegraded":"2024-02-23T00:45:06.139146+0000","last_fullsized":"2024-02-23T00:45:06.139146+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":140,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.138386+0000","last_change":"2024-02-23T00:45:04.621794+0000","last_active":"2024-02-23T00:45:06.138386+0000","last_peered":"2024-02-23T00:45:06.138386+0000","last_clean":"2024-02-23T00:45:06.138386+0000","last_became_active":"2024-02-23T00:42:58.664282+0000","last_became_peered":"2024-02-23T00:42:58.664282+0000","last_unstale":"2024-02-23T00:45:06.138386+0000","last_undegraded":"2024-02-23T00:45:06.138386+0000","last_fullsized":"2024-02-23T00:45:06.138386+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:04.621714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.621714+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":194,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.128765+0000","last_change":"2024-02-23T00:43:51.261426+0000","last_active":"2024-02-23T00:45:06.128765+0000","last_peered":"2024-02-23T00:45:06.128765+0000","last_clean":"2024-02-23T00:45:06.128765+0000","last_became_active":"2024-02-23T00:39:18.796323+0000","last_became_peered":"2024-02-23T00:39:18.796323+0000","last_unstale":"2024-02-23T00:45:06.128765+0000","last_undegraded":"2024-02-23T00:45:06.128765+0000","last_fullsized":"2024-02-23T00:45:06.128765+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:51.261367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_clean_scrub_stamp":"2024-02-23T00:43:51.261367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"68'25","reported_seq":245,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.128538+0000","last_change":"2024-02-23T00:43:46.158231+0000","last_active":"2024-02-23T00:45:06.128538+0000","last_peered":"2024-02-23T00:45:06.128538+0000","last_clean":"2024-02-23T00:45:06.128538+0000","last_became_active":"2024-02-23T00:39:16.783926+0000","last_became_peered":"2024-02-23T00:39:16.783926+0000","last_unstale":"2024-02-23T00:45:06.128538+0000","last_undegraded":"2024-02-23T00:45:06.128538+0000","last_fullsized":"2024-02-23T00:45:06.128538+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'25","last_scrub_stamp":"2024-02-23T00:43:46.158149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:46.158149+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"95'4","reported_seq":213,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.132519+0000","last_change":"2024-02-23T00:44:31.106105+0000","last_active":"2024-02-23T00:45:06.132519+0000","last_peered":"2024-02-23T00:45:06.132519+0000","last_clean":"2024-02-23T00:45:06.132519+0000","last_became_active":"2024-02-23T00:39:14.779205+0000","last_became_peered":"2024-02-23T00:39:14.779205+0000","last_unstale":"2024-02-23T00:45:06.132519+0000","last_undegraded":"2024-02-23T00:45:06.132519+0000","last_fullsized":"2024-02-23T00:45:06.132519+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'4","last_scrub_stamp":"2024-02-23T00:44:31.106012+0000","last_deep_scrub":"95'4","last_deep_scrub_stamp":"2024-02-23T00:41:43.874397+0000","last_clean_scrub_stamp":"2024-02-23T00:44:31.106012+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":211,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.126549+0000","last_change":"2024-02-23T00:43:52.036315+0000","last_active":"2024-02-23T00:45:06.126549+0000","last_peered":"2024-02-23T00:45:06.126549+0000","last_clean":"2024-02-23T00:45:06.126549+0000","last_became_active":"2024-02-23T00:43:27.611660+0000","last_became_peered":"2024-02-23T00:43:27.611660+0000","last_unstale":"2024-02-23T00:45:06.126549+0000","last_undegraded":"2024-02-23T00:45:06.126549+0000","last_fullsized":"2024-02-23T00:45:06.126549+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:52.036249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:52.036249+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":135,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.127064+0000","last_change":"2024-02-23T00:42:58.810468+0000","last_active":"2024-02-23T00:45:06.127064+0000","last_peered":"2024-02-23T00:45:06.127064+0000","last_clean":"2024-02-23T00:45:06.127064+0000","last_became_active":"2024-02-23T00:42:58.809926+0000","last_became_peered":"2024-02-23T00:42:58.809926+0000","last_unstale":"2024-02-23T00:45:06.127064+0000","last_undegraded":"2024-02-23T00:45:06.127064+0000","last_fullsized":"2024-02-23T00:45:06.127064+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":134,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.239439+0000","last_change":"2024-02-23T00:42:58.755160+0000","last_active":"2024-02-23T00:45:05.239439+0000","last_peered":"2024-02-23T00:45:05.239439+0000","last_clean":"2024-02-23T00:45:05.239439+0000","last_became_active":"2024-02-23T00:42:58.744983+0000","last_became_peered":"2024-02-23T00:42:58.744983+0000","last_unstale":"2024-02-23T00:45:05.239439+0000","last_undegraded":"2024-02-23T00:45:05.239439+0000","last_fullsized":"2024-02-23T00:45:05.239439+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"68'21","reported_seq":237,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.320889+0000","last_change":"2024-02-23T00:43:52.309599+0000","last_active":"2024-02-23T00:45:05.320889+0000","last_peered":"2024-02-23T00:45:05.320889+0000","last_clean":"2024-02-23T00:45:05.320889+0000","last_became_active":"2024-02-23T00:39:16.858760+0000","last_became_peered":"2024-02-23T00:39:16.858760+0000","last_unstale":"2024-02-23T00:45:05.320889+0000","last_undegraded":"2024-02-23T00:45:05.320889+0000","last_fullsized":"2024-02-23T00:45:05.320889+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'21","last_scrub_stamp":"2024-02-23T00:43:52.309532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:52.309532+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":198,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.128689+0000","last_change":"2024-02-23T00:42:52.317469+0000","last_active":"2024-02-23T00:45:06.128689+0000","last_peered":"2024-02-23T00:45:06.128689+0000","last_clean":"2024-02-23T00:45:06.128689+0000","last_became_active":"2024-02-23T00:39:14.774244+0000","last_became_peered":"2024-02-23T00:39:14.774244+0000","last_unstale":"2024-02-23T00:45:06.128689+0000","last_undegraded":"2024-02-23T00:45:06.128689+0000","last_fullsized":"2024-02-23T00:45:06.128689+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:52.317403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:52.317403+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":195,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.136420+0000","last_change":"2024-02-23T00:43:44.951800+0000","last_active":"2024-02-23T00:45:06.136420+0000","last_peered":"2024-02-23T00:45:06.136420+0000","last_clean":"2024-02-23T00:45:06.136420+0000","last_became_active":"2024-02-23T00:39:18.798624+0000","last_became_peered":"2024-02-23T00:39:18.798624+0000","last_unstale":"2024-02-23T00:45:06.136420+0000","last_undegraded":"2024-02-23T00:45:06.136420+0000","last_fullsized":"2024-02-23T00:45:06.136420+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:44.951696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:46.998875+0000","last_clean_scrub_stamp":"2024-02-23T00:43:44.951696+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":192,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.126108+0000","last_change":"2024-02-23T00:44:26.432766+0000","last_active":"2024-02-23T00:45:06.126108+0000","last_peered":"2024-02-23T00:45:06.126108+0000","last_clean":"2024-02-23T00:45:06.126108+0000","last_became_active":"2024-02-23T00:39:20.796041+0000","last_became_peered":"2024-02-23T00:39:20.796041+0000","last_unstale":"2024-02-23T00:45:06.126108+0000","last_undegraded":"2024-02-23T00:45:06.126108+0000","last_fullsized":"2024-02-23T00:45:06.126108+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:26.432703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:26.432703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1d","version":"110'7","reported_seq":142,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.130236+0000","last_change":"2024-02-23T00:42:58.755048+0000","last_active":"2024-02-23T00:45:06.130236+0000","last_peered":"2024-02-23T00:45:06.130236+0000","last_clean":"2024-02-23T00:45:06.130236+0000","last_became_active":"2024-02-23T00:42:58.754763+0000","last_became_peered":"2024-02-23T00:42:58.754763+0000","last_unstale":"2024-02-23T00:45:06.130236+0000","last_undegraded":"2024-02-23T00:45:06.130236+0000","last_fullsized":"2024-02-23T00:45:06.130236+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":139,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.312199+0000","last_change":"2024-02-23T00:44:26.595443+0000","last_active":"2024-02-23T00:45:05.312199+0000","last_peered":"2024-02-23T00:45:05.312199+0000","last_clean":"2024-02-23T00:45:05.312199+0000","last_became_active":"2024-02-23T00:42:58.825015+0000","last_became_peered":"2024-02-23T00:42:58.825015+0000","last_unstale":"2024-02-23T00:45:05.312199+0000","last_undegraded":"2024-02-23T00:45:05.312199+0000","last_fullsized":"2024-02-23T00:45:05.312199+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:26.595346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:44:26.595346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"107'16","reported_seq":225,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:07.012139+0000","last_change":"2024-02-23T00:45:07.012086+0000","last_active":"2024-02-23T00:45:07.012139+0000","last_peered":"2024-02-23T00:45:07.012139+0000","last_clean":"2024-02-23T00:45:07.012139+0000","last_became_active":"2024-02-23T00:39:16.858449+0000","last_became_peered":"2024-02-23T00:39:16.858449+0000","last_unstale":"2024-02-23T00:45:07.012139+0000","last_undegraded":"2024-02-23T00:45:07.012139+0000","last_fullsized":"2024-02-23T00:45:07.012139+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'16","last_scrub_stamp":"2024-02-23T00:45:07.011956+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-02-23T00:42:52.964510+0000","last_clean_scrub_stamp":"2024-02-23T00:45:07.011956+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":193,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.138257+0000","last_change":"2024-02-23T00:42:41.429191+0000","last_active":"2024-02-23T00:45:06.138257+0000","last_peered":"2024-02-23T00:45:06.138257+0000","last_clean":"2024-02-23T00:45:06.138257+0000","last_became_active":"2024-02-23T00:39:14.783164+0000","last_became_peered":"2024-02-23T00:39:14.783164+0000","last_unstale":"2024-02-23T00:45:06.138257+0000","last_undegraded":"2024-02-23T00:45:06.138257+0000","last_fullsized":"2024-02-23T00:45:06.138257+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:41.429079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:41.429079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":194,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.126141+0000","last_change":"2024-02-23T00:43:40.971316+0000","last_active":"2024-02-23T00:45:05.126141+0000","last_peered":"2024-02-23T00:45:05.126141+0000","last_clean":"2024-02-23T00:45:05.126141+0000","last_became_active":"2024-02-23T00:39:18.800297+0000","last_became_peered":"2024-02-23T00:39:18.800297+0000","last_unstale":"2024-02-23T00:45:05.126141+0000","last_undegraded":"2024-02-23T00:45:05.126141+0000","last_fullsized":"2024-02-23T00:45:05.126141+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:40.971243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:40.971243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":190,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.238341+0000","last_change":"2024-02-23T00:43:46.760431+0000","last_active":"2024-02-23T00:45:05.238341+0000","last_peered":"2024-02-23T00:45:05.238341+0000","last_clean":"2024-02-23T00:45:05.238341+0000","last_became_active":"2024-02-23T00:39:21.162036+0000","last_became_peered":"2024-02-23T00:39:21.162036+0000","last_unstale":"2024-02-23T00:45:05.238341+0000","last_undegraded":"2024-02-23T00:45:05.238341+0000","last_fullsized":"2024-02-23T00:45:05.238341+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:46.760348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:46.760348+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":134,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.319008+0000","last_change":"2024-02-23T00:42:57.649841+0000","last_active":"2024-02-23T00:45:05.319008+0000","last_peered":"2024-02-23T00:45:05.319008+0000","last_clean":"2024-02-23T00:45:05.319008+0000","last_became_active":"2024-02-23T00:42:57.648267+0000","last_became_peered":"2024-02-23T00:42:57.648267+0000","last_unstale":"2024-02-23T00:45:05.319008+0000","last_undegraded":"2024-02-23T00:45:05.319008+0000","last_fullsized":"2024-02-23T00:45:05.319008+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1","version":"110'7","reported_seq":148,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.127369+0000","last_change":"2024-02-23T00:42:58.782380+0000","last_active":"2024-02-23T00:45:06.127369+0000","last_peered":"2024-02-23T00:45:06.127369+0000","last_clean":"2024-02-23T00:45:06.127369+0000","last_became_active":"2024-02-23T00:42:58.777251+0000","last_became_peered":"2024-02-23T00:42:58.777251+0000","last_unstale":"2024-02-23T00:45:06.127369+0000","last_undegraded":"2024-02-23T00:45:06.127369+0000","last_fullsized":"2024-02-23T00:45:06.127369+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"68'22","reported_seq":166,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.125907+0000","last_change":"2024-02-23T00:44:31.391929+0000","last_active":"2024-02-23T00:45:06.125907+0000","last_peered":"2024-02-23T00:45:06.125907+0000","last_clean":"2024-02-23T00:45:06.125907+0000","last_became_active":"2024-02-23T00:42:43.521189+0000","last_became_peered":"2024-02-23T00:42:43.521189+0000","last_unstale":"2024-02-23T00:45:06.125907+0000","last_undegraded":"2024-02-23T00:45:06.125907+0000","last_fullsized":"2024-02-23T00:45:06.125907+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'22","last_scrub_stamp":"2024-02-23T00:44:31.391819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:44:31.391819+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":37,"num_read_kb":30,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,7],"acting":[2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":198,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.124826+0000","last_change":"2024-02-23T00:44:18.946624+0000","last_active":"2024-02-23T00:45:05.124826+0000","last_peered":"2024-02-23T00:45:05.124826+0000","last_clean":"2024-02-23T00:45:05.124826+0000","last_became_active":"2024-02-23T00:39:14.785590+0000","last_became_peered":"2024-02-23T00:39:14.785590+0000","last_unstale":"2024-02-23T00:45:05.124826+0000","last_undegraded":"2024-02-23T00:45:05.124826+0000","last_fullsized":"2024-02-23T00:45:05.124826+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:18.946560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_clean_scrub_stamp":"2024-02-23T00:44:18.946560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"67'8","reported_seq":272,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.127134+0000","last_change":"2024-02-23T00:44:29.038226+0000","last_active":"2024-02-23T00:45:05.127134+0000","last_peered":"2024-02-23T00:45:05.127134+0000","last_clean":"2024-02-23T00:45:05.127134+0000","last_became_active":"2024-02-23T00:39:18.803566+0000","last_became_peered":"2024-02-23T00:39:18.803566+0000","last_unstale":"2024-02-23T00:45:05.127134+0000","last_undegraded":"2024-02-23T00:45:05.127134+0000","last_fullsized":"2024-02-23T00:45:05.127134+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:44:29.038134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:29.038134+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"110'5","reported_seq":140,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.136576+0000","last_change":"2024-02-23T00:42:58.770053+0000","last_active":"2024-02-23T00:45:06.136576+0000","last_peered":"2024-02-23T00:45:06.136576+0000","last_clean":"2024-02-23T00:45:06.136576+0000","last_became_active":"2024-02-23T00:42:58.758349+0000","last_became_peered":"2024-02-23T00:42:58.758349+0000","last_unstale":"2024-02-23T00:45:06.136576+0000","last_undegraded":"2024-02-23T00:45:06.136576+0000","last_fullsized":"2024-02-23T00:45:06.136576+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":134,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.239483+0000","last_change":"2024-02-23T00:42:58.766295+0000","last_active":"2024-02-23T00:45:05.239483+0000","last_peered":"2024-02-23T00:45:05.239483+0000","last_clean":"2024-02-23T00:45:05.239483+0000","last_became_active":"2024-02-23T00:42:58.757424+0000","last_became_peered":"2024-02-23T00:42:58.757424+0000","last_unstale":"2024-02-23T00:45:05.239483+0000","last_undegraded":"2024-02-23T00:45:05.239483+0000","last_fullsized":"2024-02-23T00:45:05.239483+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"68'15","reported_seq":227,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.127191+0000","last_change":"2024-02-23T00:44:25.998879+0000","last_active":"2024-02-23T00:45:05.127191+0000","last_peered":"2024-02-23T00:45:05.127191+0000","last_clean":"2024-02-23T00:45:05.127191+0000","last_became_active":"2024-02-23T00:39:16.858953+0000","last_became_peered":"2024-02-23T00:39:16.858953+0000","last_unstale":"2024-02-23T00:45:05.127191+0000","last_undegraded":"2024-02-23T00:45:05.127191+0000","last_fullsized":"2024-02-23T00:45:05.127191+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:44:25.998804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:44:25.998804+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":198,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.318401+0000","last_change":"2024-02-23T00:43:45.309194+0000","last_active":"2024-02-23T00:45:05.318401+0000","last_peered":"2024-02-23T00:45:05.318401+0000","last_clean":"2024-02-23T00:45:05.318401+0000","last_became_active":"2024-02-23T00:39:14.773772+0000","last_became_peered":"2024-02-23T00:39:14.773772+0000","last_unstale":"2024-02-23T00:45:05.318401+0000","last_undegraded":"2024-02-23T00:45:05.318401+0000","last_fullsized":"2024-02-23T00:45:05.318401+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:45.309083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.309083+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"67'8","reported_seq":268,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.129922+0000","last_change":"2024-02-23T00:43:39.086199+0000","last_active":"2024-02-23T00:45:06.129922+0000","last_peered":"2024-02-23T00:45:06.129922+0000","last_clean":"2024-02-23T00:45:06.129922+0000","last_became_active":"2024-02-23T00:39:18.787313+0000","last_became_peered":"2024-02-23T00:39:18.787313+0000","last_unstale":"2024-02-23T00:45:06.129922+0000","last_undegraded":"2024-02-23T00:45:06.129922+0000","last_fullsized":"2024-02-23T00:45:06.129922+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:43:39.086107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:39.086107+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":140,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.129520+0000","last_change":"2024-02-23T00:44:34.437892+0000","last_active":"2024-02-23T00:45:06.129520+0000","last_peered":"2024-02-23T00:45:06.129520+0000","last_clean":"2024-02-23T00:45:06.129520+0000","last_became_active":"2024-02-23T00:42:58.745471+0000","last_became_peered":"2024-02-23T00:42:58.745471+0000","last_unstale":"2024-02-23T00:45:06.129520+0000","last_undegraded":"2024-02-23T00:45:06.129520+0000","last_fullsized":"2024-02-23T00:45:06.129520+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:34.437785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:34.437785+0000","last_clean_scrub_stamp":"2024-02-23T00:44:34.437785+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":134,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.319758+0000","last_change":"2024-02-23T00:42:57.652853+0000","last_active":"2024-02-23T00:45:05.319758+0000","last_peered":"2024-02-23T00:45:05.319758+0000","last_clean":"2024-02-23T00:45:05.319758+0000","last_became_active":"2024-02-23T00:42:57.652488+0000","last_became_peered":"2024-02-23T00:42:57.652488+0000","last_unstale":"2024-02-23T00:45:05.319758+0000","last_undegraded":"2024-02-23T00:45:05.319758+0000","last_fullsized":"2024-02-23T00:45:05.319758+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"67'9","reported_seq":206,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.315393+0000","last_change":"2024-02-23T00:42:57.705378+0000","last_active":"2024-02-23T00:45:05.315393+0000","last_peered":"2024-02-23T00:45:05.315393+0000","last_clean":"2024-02-23T00:45:05.315393+0000","last_became_active":"2024-02-23T00:39:16.786936+0000","last_became_peered":"2024-02-23T00:39:16.786936+0000","last_unstale":"2024-02-23T00:45:05.315393+0000","last_undegraded":"2024-02-23T00:45:05.315393+0000","last_fullsized":"2024-02-23T00:45:05.315393+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'9","last_scrub_stamp":"2024-02-23T00:42:57.705261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.705261+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":203,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.137932+0000","last_change":"2024-02-23T00:44:24.350861+0000","last_active":"2024-02-23T00:45:06.137932+0000","last_peered":"2024-02-23T00:45:06.137932+0000","last_clean":"2024-02-23T00:45:06.137932+0000","last_became_active":"2024-02-23T00:39:14.778455+0000","last_became_peered":"2024-02-23T00:39:14.778455+0000","last_unstale":"2024-02-23T00:45:06.137932+0000","last_undegraded":"2024-02-23T00:45:06.137932+0000","last_fullsized":"2024-02-23T00:45:06.137932+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:24.350797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:24.350797+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":195,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.126463+0000","last_change":"2024-02-23T00:44:37.105079+0000","last_active":"2024-02-23T00:45:06.126463+0000","last_peered":"2024-02-23T00:45:06.126463+0000","last_clean":"2024-02-23T00:45:06.126463+0000","last_became_active":"2024-02-23T00:39:18.799052+0000","last_became_peered":"2024-02-23T00:39:18.799052+0000","last_unstale":"2024-02-23T00:45:06.126463+0000","last_undegraded":"2024-02-23T00:45:06.126463+0000","last_fullsized":"2024-02-23T00:45:06.126463+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:37.105013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_clean_scrub_stamp":"2024-02-23T00:44:37.105013+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":187,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.311929+0000","last_change":"2024-02-23T00:42:50.425748+0000","last_active":"2024-02-23T00:45:05.311929+0000","last_peered":"2024-02-23T00:45:05.311929+0000","last_clean":"2024-02-23T00:45:05.311929+0000","last_became_active":"2024-02-23T00:39:20.802648+0000","last_became_peered":"2024-02-23T00:39:20.802648+0000","last_unstale":"2024-02-23T00:45:05.311929+0000","last_undegraded":"2024-02-23T00:45:05.311929+0000","last_fullsized":"2024-02-23T00:45:05.311929+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:50.425641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:50.425641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":135,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.128229+0000","last_change":"2024-02-23T00:42:58.834611+0000","last_active":"2024-02-23T00:45:06.128229+0000","last_peered":"2024-02-23T00:45:06.128229+0000","last_clean":"2024-02-23T00:45:06.128229+0000","last_became_active":"2024-02-23T00:42:58.833531+0000","last_became_peered":"2024-02-23T00:42:58.833531+0000","last_unstale":"2024-02-23T00:45:06.128229+0000","last_undegraded":"2024-02-23T00:45:06.128229+0000","last_fullsized":"2024-02-23T00:45:06.128229+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":139,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.318039+0000","last_change":"2024-02-23T00:44:23.543049+0000","last_active":"2024-02-23T00:45:05.318039+0000","last_peered":"2024-02-23T00:45:05.318039+0000","last_clean":"2024-02-23T00:45:05.318039+0000","last_became_active":"2024-02-23T00:42:58.835862+0000","last_became_peered":"2024-02-23T00:42:58.835862+0000","last_unstale":"2024-02-23T00:45:05.318039+0000","last_undegraded":"2024-02-23T00:45:05.318039+0000","last_fullsized":"2024-02-23T00:45:05.318039+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:23.542975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.542975+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"67'11","reported_seq":210,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.129136+0000","last_change":"2024-02-23T00:42:58.013765+0000","last_active":"2024-02-23T00:45:06.129136+0000","last_peered":"2024-02-23T00:45:06.129136+0000","last_clean":"2024-02-23T00:45:06.129136+0000","last_became_active":"2024-02-23T00:39:16.783896+0000","last_became_peered":"2024-02-23T00:39:16.783896+0000","last_unstale":"2024-02-23T00:45:06.129136+0000","last_undegraded":"2024-02-23T00:45:06.129136+0000","last_fullsized":"2024-02-23T00:45:06.129136+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'11","last_scrub_stamp":"2024-02-23T00:42:58.013688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:58.013688+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"202'2","reported_seq":205,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.318964+0000","last_change":"2024-02-23T00:44:34.720705+0000","last_active":"2024-02-23T00:45:05.318964+0000","last_peered":"2024-02-23T00:45:05.318964+0000","last_clean":"2024-02-23T00:45:05.318964+0000","last_became_active":"2024-02-23T00:39:14.775804+0000","last_became_peered":"2024-02-23T00:39:14.775804+0000","last_unstale":"2024-02-23T00:45:05.318964+0000","last_undegraded":"2024-02-23T00:45:05.318964+0000","last_fullsized":"2024-02-23T00:45:05.318964+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"202'2","last_scrub_stamp":"2024-02-23T00:44:34.720640+0000","last_deep_scrub":"202'2","last_deep_scrub_stamp":"2024-02-23T00:44:34.720640+0000","last_clean_scrub_stamp":"2024-02-23T00:44:34.720640+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":188,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.315484+0000","last_change":"2024-02-23T00:43:31.903157+0000","last_active":"2024-02-23T00:45:05.315484+0000","last_peered":"2024-02-23T00:45:05.315484+0000","last_clean":"2024-02-23T00:45:05.315484+0000","last_became_active":"2024-02-23T00:39:18.800382+0000","last_became_peered":"2024-02-23T00:39:18.800382+0000","last_unstale":"2024-02-23T00:45:05.315484+0000","last_undegraded":"2024-02-23T00:45:05.315484+0000","last_fullsized":"2024-02-23T00:45:05.315484+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:31.903031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:31.903031+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":193,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.126877+0000","last_change":"2024-02-23T00:44:23.152677+0000","last_active":"2024-02-23T00:45:06.126877+0000","last_peered":"2024-02-23T00:45:06.126877+0000","last_clean":"2024-02-23T00:45:06.126877+0000","last_became_active":"2024-02-23T00:39:20.799053+0000","last_became_peered":"2024-02-23T00:39:20.799053+0000","last_unstale":"2024-02-23T00:45:06.126877+0000","last_undegraded":"2024-02-23T00:45:06.126877+0000","last_fullsized":"2024-02-23T00:45:06.126877+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:23.152602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.152602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"110'2","reported_seq":137,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.124980+0000","last_change":"2024-02-23T00:42:58.709534+0000","last_active":"2024-02-23T00:45:06.124980+0000","last_peered":"2024-02-23T00:45:06.124980+0000","last_clean":"2024-02-23T00:45:06.124980+0000","last_became_active":"2024-02-23T00:42:58.698725+0000","last_became_peered":"2024-02-23T00:42:58.698725+0000","last_unstale":"2024-02-23T00:45:06.124980+0000","last_undegraded":"2024-02-23T00:45:06.124980+0000","last_fullsized":"2024-02-23T00:45:06.124980+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":134,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.318874+0000","last_change":"2024-02-23T00:42:57.652742+0000","last_active":"2024-02-23T00:45:05.318874+0000","last_peered":"2024-02-23T00:45:05.318874+0000","last_clean":"2024-02-23T00:45:05.318874+0000","last_became_active":"2024-02-23T00:42:57.652438+0000","last_became_peered":"2024-02-23T00:42:57.652438+0000","last_unstale":"2024-02-23T00:45:05.318874+0000","last_undegraded":"2024-02-23T00:45:05.318874+0000","last_fullsized":"2024-02-23T00:45:05.318874+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"67'4","reported_seq":198,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.124892+0000","last_change":"2024-02-23T00:42:42.934024+0000","last_active":"2024-02-23T00:45:06.124892+0000","last_peered":"2024-02-23T00:45:06.124892+0000","last_clean":"2024-02-23T00:45:06.124892+0000","last_became_active":"2024-02-23T00:39:16.785830+0000","last_became_peered":"2024-02-23T00:39:16.785830+0000","last_unstale":"2024-02-23T00:45:06.124892+0000","last_undegraded":"2024-02-23T00:45:06.124892+0000","last_fullsized":"2024-02-23T00:45:06.124892+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'4","last_scrub_stamp":"2024-02-23T00:42:42.933958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:42.933958+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":198,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.126015+0000","last_change":"2024-02-23T00:43:38.947741+0000","last_active":"2024-02-23T00:45:05.126015+0000","last_peered":"2024-02-23T00:45:05.126015+0000","last_clean":"2024-02-23T00:45:05.126015+0000","last_became_active":"2024-02-23T00:39:14.783325+0000","last_became_peered":"2024-02-23T00:39:14.783325+0000","last_unstale":"2024-02-23T00:45:05.126015+0000","last_undegraded":"2024-02-23T00:45:05.126015+0000","last_fullsized":"2024-02-23T00:45:05.126015+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:38.947675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:38.947675+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":194,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.138052+0000","last_change":"2024-02-23T00:42:57.574460+0000","last_active":"2024-02-23T00:45:06.138052+0000","last_peered":"2024-02-23T00:45:06.138052+0000","last_clean":"2024-02-23T00:45:06.138052+0000","last_became_active":"2024-02-23T00:39:18.797287+0000","last_became_peered":"2024-02-23T00:39:18.797287+0000","last_unstale":"2024-02-23T00:45:06.138052+0000","last_undegraded":"2024-02-23T00:45:06.138052+0000","last_fullsized":"2024-02-23T00:45:06.138052+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.574369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.574369+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":185,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.238399+0000","last_change":"2024-02-23T00:42:44.983880+0000","last_active":"2024-02-23T00:45:05.238399+0000","last_peered":"2024-02-23T00:45:05.238399+0000","last_clean":"2024-02-23T00:45:05.238399+0000","last_became_active":"2024-02-23T00:39:20.795351+0000","last_became_peered":"2024-02-23T00:39:20.795351+0000","last_unstale":"2024-02-23T00:45:05.238399+0000","last_undegraded":"2024-02-23T00:45:05.238399+0000","last_fullsized":"2024-02-23T00:45:05.238399+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:44.983813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:44.983813+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1b","version":"110'10","reported_seq":144,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.236111+0000","last_change":"2024-02-23T00:42:58.749680+0000","last_active":"2024-02-23T00:45:05.236111+0000","last_peered":"2024-02-23T00:45:05.236111+0000","last_clean":"2024-02-23T00:45:05.236111+0000","last_became_active":"2024-02-23T00:42:58.749126+0000","last_became_peered":"2024-02-23T00:42:58.749126+0000","last_unstale":"2024-02-23T00:45:05.236111+0000","last_undegraded":"2024-02-23T00:45:05.236111+0000","last_fullsized":"2024-02-23T00:45:05.236111+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":134,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.240366+0000","last_change":"2024-02-23T00:42:58.741674+0000","last_active":"2024-02-23T00:45:05.240366+0000","last_peered":"2024-02-23T00:45:05.240366+0000","last_clean":"2024-02-23T00:45:05.240366+0000","last_became_active":"2024-02-23T00:42:58.741365+0000","last_became_peered":"2024-02-23T00:42:58.741365+0000","last_unstale":"2024-02-23T00:45:05.240366+0000","last_undegraded":"2024-02-23T00:45:05.240366+0000","last_fullsized":"2024-02-23T00:45:05.240366+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"68'17","reported_seq":231,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.127561+0000","last_change":"2024-02-23T00:43:43.997579+0000","last_active":"2024-02-23T00:45:06.127561+0000","last_peered":"2024-02-23T00:45:06.127561+0000","last_clean":"2024-02-23T00:45:06.127561+0000","last_became_active":"2024-02-23T00:39:16.857232+0000","last_became_peered":"2024-02-23T00:39:16.857232+0000","last_unstale":"2024-02-23T00:45:06.127561+0000","last_undegraded":"2024-02-23T00:45:06.127561+0000","last_fullsized":"2024-02-23T00:45:06.127561+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:43:43.997452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:43.997452+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":198,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.141271+0000","last_change":"2024-02-23T00:43:52.419206+0000","last_active":"2024-02-23T00:45:06.141271+0000","last_peered":"2024-02-23T00:45:06.141271+0000","last_clean":"2024-02-23T00:45:06.141271+0000","last_became_active":"2024-02-23T00:39:14.784092+0000","last_became_peered":"2024-02-23T00:39:14.784092+0000","last_unstale":"2024-02-23T00:45:06.141271+0000","last_undegraded":"2024-02-23T00:45:06.141271+0000","last_fullsized":"2024-02-23T00:45:06.141271+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:52.419100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:52.419100+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":192,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.235875+0000","last_change":"2024-02-23T00:44:25.797288+0000","last_active":"2024-02-23T00:45:05.235875+0000","last_peered":"2024-02-23T00:45:05.235875+0000","last_clean":"2024-02-23T00:45:05.235875+0000","last_became_active":"2024-02-23T00:39:18.791457+0000","last_became_peered":"2024-02-23T00:39:18.791457+0000","last_unstale":"2024-02-23T00:45:05.235875+0000","last_undegraded":"2024-02-23T00:45:05.235875+0000","last_fullsized":"2024-02-23T00:45:05.235875+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:25.797184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:25.797184+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":188,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.126633+0000","last_change":"2024-02-23T00:43:45.965636+0000","last_active":"2024-02-23T00:45:06.126633+0000","last_peered":"2024-02-23T00:45:06.126633+0000","last_clean":"2024-02-23T00:45:06.126633+0000","last_became_active":"2024-02-23T00:39:21.160376+0000","last_became_peered":"2024-02-23T00:39:21.160376+0000","last_unstale":"2024-02-23T00:45:06.126633+0000","last_undegraded":"2024-02-23T00:45:06.126633+0000","last_fullsized":"2024-02-23T00:45:06.126633+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:45.965570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.965570+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1a","version":"110'5","reported_seq":139,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.124079+0000","last_change":"2024-02-23T00:42:58.738588+0000","last_active":"2024-02-23T00:45:05.124079+0000","last_peered":"2024-02-23T00:45:05.124079+0000","last_clean":"2024-02-23T00:45:05.124079+0000","last_became_active":"2024-02-23T00:42:58.736706+0000","last_became_peered":"2024-02-23T00:42:58.736706+0000","last_unstale":"2024-02-23T00:45:05.124079+0000","last_undegraded":"2024-02-23T00:45:05.124079+0000","last_fullsized":"2024-02-23T00:45:05.124079+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":135,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.126144+0000","last_change":"2024-02-23T00:42:58.705557+0000","last_active":"2024-02-23T00:45:06.126144+0000","last_peered":"2024-02-23T00:45:06.126144+0000","last_clean":"2024-02-23T00:45:06.126144+0000","last_became_active":"2024-02-23T00:42:58.702580+0000","last_became_peered":"2024-02-23T00:42:58.702580+0000","last_unstale":"2024-02-23T00:45:06.126144+0000","last_undegraded":"2024-02-23T00:45:06.126144+0000","last_fullsized":"2024-02-23T00:45:06.126144+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"68'16","reported_seq":226,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.321347+0000","last_change":"2024-02-23T00:45:04.672612+0000","last_active":"2024-02-23T00:45:05.321347+0000","last_peered":"2024-02-23T00:45:05.321347+0000","last_clean":"2024-02-23T00:45:05.321347+0000","last_became_active":"2024-02-23T00:39:16.858945+0000","last_became_peered":"2024-02-23T00:39:16.858945+0000","last_unstale":"2024-02-23T00:45:05.321347+0000","last_undegraded":"2024-02-23T00:45:05.321347+0000","last_fullsized":"2024-02-23T00:45:05.321347+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:45:04.672537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.672537+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":205,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.123454+0000","last_change":"2024-02-23T00:43:44.031862+0000","last_active":"2024-02-23T00:45:05.123454+0000","last_peered":"2024-02-23T00:45:05.123454+0000","last_clean":"2024-02-23T00:45:05.123454+0000","last_became_active":"2024-02-23T00:39:14.785363+0000","last_became_peered":"2024-02-23T00:39:14.785363+0000","last_unstale":"2024-02-23T00:45:05.123454+0000","last_undegraded":"2024-02-23T00:45:05.123454+0000","last_fullsized":"2024-02-23T00:45:05.123454+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"80'2","last_scrub_stamp":"2024-02-23T00:43:44.031793+0000","last_deep_scrub":"80'2","last_deep_scrub_stamp":"2024-02-23T00:40:15.942725+0000","last_clean_scrub_stamp":"2024-02-23T00:43:44.031793+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"93'9","reported_seq":276,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.141434+0000","last_change":"2024-02-23T00:43:50.393789+0000","last_active":"2024-02-23T00:45:06.141434+0000","last_peered":"2024-02-23T00:45:06.141434+0000","last_clean":"2024-02-23T00:45:06.141434+0000","last_became_active":"2024-02-23T00:39:18.796680+0000","last_became_peered":"2024-02-23T00:39:18.796680+0000","last_unstale":"2024-02-23T00:45:06.141434+0000","last_undegraded":"2024-02-23T00:45:06.141434+0000","last_fullsized":"2024-02-23T00:45:06.141434+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-02-23T00:43:50.393713+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-02-23T00:43:50.393713+0000","last_clean_scrub_stamp":"2024-02-23T00:43:50.393713+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":134,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.239459+0000","last_change":"2024-02-23T00:42:58.805923+0000","last_active":"2024-02-23T00:45:05.239459+0000","last_peered":"2024-02-23T00:45:05.239459+0000","last_clean":"2024-02-23T00:45:05.239459+0000","last_became_active":"2024-02-23T00:42:58.803900+0000","last_became_peered":"2024-02-23T00:42:58.803900+0000","last_unstale":"2024-02-23T00:45:05.239459+0000","last_undegraded":"2024-02-23T00:45:05.239459+0000","last_fullsized":"2024-02-23T00:45:05.239459+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":134,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.312425+0000","last_change":"2024-02-23T00:42:58.850530+0000","last_active":"2024-02-23T00:45:05.312425+0000","last_peered":"2024-02-23T00:45:05.312425+0000","last_clean":"2024-02-23T00:45:05.312425+0000","last_became_active":"2024-02-23T00:42:58.850072+0000","last_became_peered":"2024-02-23T00:42:58.850072+0000","last_unstale":"2024-02-23T00:45:05.312425+0000","last_undegraded":"2024-02-23T00:45:05.312425+0000","last_fullsized":"2024-02-23T00:45:05.312425+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"93'23","reported_seq":241,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.141528+0000","last_change":"2024-02-23T00:44:28.387225+0000","last_active":"2024-02-23T00:45:06.141528+0000","last_peered":"2024-02-23T00:45:06.141528+0000","last_clean":"2024-02-23T00:45:06.141528+0000","last_became_active":"2024-02-23T00:39:16.782660+0000","last_became_peered":"2024-02-23T00:39:16.782660+0000","last_unstale":"2024-02-23T00:45:06.141528+0000","last_undegraded":"2024-02-23T00:45:06.141528+0000","last_fullsized":"2024-02-23T00:45:06.141528+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'23","last_scrub_stamp":"2024-02-23T00:44:28.387155+0000","last_deep_scrub":"93'23","last_deep_scrub_stamp":"2024-02-23T00:41:35.954516+0000","last_clean_scrub_stamp":"2024-02-23T00:44:28.387155+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":199,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.127018+0000","last_change":"2024-02-23T00:44:21.157372+0000","last_active":"2024-02-23T00:45:06.127018+0000","last_peered":"2024-02-23T00:45:06.127018+0000","last_clean":"2024-02-23T00:45:06.127018+0000","last_became_active":"2024-02-23T00:39:14.781560+0000","last_became_peered":"2024-02-23T00:39:14.781560+0000","last_unstale":"2024-02-23T00:45:06.127018+0000","last_undegraded":"2024-02-23T00:45:06.127018+0000","last_fullsized":"2024-02-23T00:45:06.127018+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:21.157316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:21.157316+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":194,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.321429+0000","last_change":"2024-02-23T00:43:49.238770+0000","last_active":"2024-02-23T00:45:05.321429+0000","last_peered":"2024-02-23T00:45:05.321429+0000","last_clean":"2024-02-23T00:45:05.321429+0000","last_became_active":"2024-02-23T00:39:18.790602+0000","last_became_peered":"2024-02-23T00:39:18.790602+0000","last_unstale":"2024-02-23T00:45:05.321429+0000","last_undegraded":"2024-02-23T00:45:05.321429+0000","last_fullsized":"2024-02-23T00:45:05.321429+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:49.238704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:53.189881+0000","last_clean_scrub_stamp":"2024-02-23T00:43:49.238704+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.14","version":"110'2","reported_seq":137,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.138174+0000","last_change":"2024-02-23T00:42:58.804637+0000","last_active":"2024-02-23T00:45:06.138174+0000","last_peered":"2024-02-23T00:45:06.138174+0000","last_clean":"2024-02-23T00:45:06.138174+0000","last_became_active":"2024-02-23T00:42:58.804344+0000","last_became_peered":"2024-02-23T00:42:58.804344+0000","last_unstale":"2024-02-23T00:45:06.138174+0000","last_undegraded":"2024-02-23T00:45:06.138174+0000","last_fullsized":"2024-02-23T00:45:06.138174+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":135,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.127097+0000","last_change":"2024-02-23T00:42:58.778321+0000","last_active":"2024-02-23T00:45:06.127097+0000","last_peered":"2024-02-23T00:45:06.127097+0000","last_clean":"2024-02-23T00:45:06.127097+0000","last_became_active":"2024-02-23T00:42:58.774720+0000","last_became_peered":"2024-02-23T00:42:58.774720+0000","last_unstale":"2024-02-23T00:45:06.127097+0000","last_undegraded":"2024-02-23T00:45:06.127097+0000","last_fullsized":"2024-02-23T00:45:06.127097+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"67'15","reported_seq":221,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.126644+0000","last_change":"2024-02-23T00:43:36.449809+0000","last_active":"2024-02-23T00:45:06.126644+0000","last_peered":"2024-02-23T00:45:06.126644+0000","last_clean":"2024-02-23T00:45:06.126644+0000","last_became_active":"2024-02-23T00:39:16.787138+0000","last_became_peered":"2024-02-23T00:39:16.787138+0000","last_unstale":"2024-02-23T00:45:06.126644+0000","last_undegraded":"2024-02-23T00:45:06.126644+0000","last_fullsized":"2024-02-23T00:45:06.126644+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:43:36.449687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:36.449687+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":198,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.139690+0000","last_change":"2024-02-23T00:43:45.362071+0000","last_active":"2024-02-23T00:45:06.139690+0000","last_peered":"2024-02-23T00:45:06.139690+0000","last_clean":"2024-02-23T00:45:06.139690+0000","last_became_active":"2024-02-23T00:39:14.782494+0000","last_became_peered":"2024-02-23T00:39:14.782494+0000","last_unstale":"2024-02-23T00:45:06.139690+0000","last_undegraded":"2024-02-23T00:45:06.139690+0000","last_fullsized":"2024-02-23T00:45:06.139690+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:45.361978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.361978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":193,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.319186+0000","last_change":"2024-02-23T00:45:03.615758+0000","last_active":"2024-02-23T00:45:05.319186+0000","last_peered":"2024-02-23T00:45:05.319186+0000","last_clean":"2024-02-23T00:45:05.319186+0000","last_became_active":"2024-02-23T00:39:18.801039+0000","last_became_peered":"2024-02-23T00:39:18.801039+0000","last_unstale":"2024-02-23T00:45:05.319186+0000","last_undegraded":"2024-02-23T00:45:05.319186+0000","last_fullsized":"2024-02-23T00:45:05.319186+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:03.615692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.641689+0000","last_clean_scrub_stamp":"2024-02-23T00:45:03.615692+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"110'5","reported_seq":140,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.132316+0000","last_change":"2024-02-23T00:42:58.783591+0000","last_active":"2024-02-23T00:45:06.132316+0000","last_peered":"2024-02-23T00:45:06.132316+0000","last_clean":"2024-02-23T00:45:06.132316+0000","last_became_active":"2024-02-23T00:42:58.783343+0000","last_became_peered":"2024-02-23T00:42:58.783343+0000","last_unstale":"2024-02-23T00:45:06.132316+0000","last_undegraded":"2024-02-23T00:45:06.132316+0000","last_fullsized":"2024-02-23T00:45:06.132316+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":135,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.125341+0000","last_change":"2024-02-23T00:42:58.773110+0000","last_active":"2024-02-23T00:45:06.125341+0000","last_peered":"2024-02-23T00:45:06.125341+0000","last_clean":"2024-02-23T00:45:06.125341+0000","last_became_active":"2024-02-23T00:42:58.770759+0000","last_became_peered":"2024-02-23T00:42:58.770759+0000","last_unstale":"2024-02-23T00:45:06.125341+0000","last_undegraded":"2024-02-23T00:45:06.125341+0000","last_fullsized":"2024-02-23T00:45:06.125341+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"68'18","reported_seq":235,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.130671+0000","last_change":"2024-02-23T00:43:51.079824+0000","last_active":"2024-02-23T00:45:06.130671+0000","last_peered":"2024-02-23T00:45:06.130671+0000","last_clean":"2024-02-23T00:45:06.130671+0000","last_became_active":"2024-02-23T00:39:16.782803+0000","last_became_peered":"2024-02-23T00:39:16.782803+0000","last_unstale":"2024-02-23T00:45:06.130671+0000","last_undegraded":"2024-02-23T00:45:06.130671+0000","last_fullsized":"2024-02-23T00:45:06.130671+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'18","last_scrub_stamp":"2024-02-23T00:43:51.079756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:51.079756+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":199,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.126382+0000","last_change":"2024-02-23T00:43:48.030906+0000","last_active":"2024-02-23T00:45:06.126382+0000","last_peered":"2024-02-23T00:45:06.126382+0000","last_clean":"2024-02-23T00:45:06.126382+0000","last_became_active":"2024-02-23T00:39:14.782163+0000","last_became_peered":"2024-02-23T00:39:14.782163+0000","last_unstale":"2024-02-23T00:45:06.126382+0000","last_undegraded":"2024-02-23T00:45:06.126382+0000","last_fullsized":"2024-02-23T00:45:06.126382+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:48.030846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.030846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":194,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.134094+0000","last_change":"2024-02-23T00:44:37.068899+0000","last_active":"2024-02-23T00:45:06.134094+0000","last_peered":"2024-02-23T00:45:06.134094+0000","last_clean":"2024-02-23T00:45:06.134094+0000","last_became_active":"2024-02-23T00:39:18.794903+0000","last_became_peered":"2024-02-23T00:39:18.794903+0000","last_unstale":"2024-02-23T00:45:06.134094+0000","last_undegraded":"2024-02-23T00:45:06.134094+0000","last_fullsized":"2024-02-23T00:45:06.134094+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:37.068834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_clean_scrub_stamp":"2024-02-23T00:44:37.068834+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"110'5","reported_seq":139,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.321520+0000","last_change":"2024-02-23T00:42:58.839927+0000","last_active":"2024-02-23T00:45:05.321520+0000","last_peered":"2024-02-23T00:45:05.321520+0000","last_clean":"2024-02-23T00:45:05.321520+0000","last_became_active":"2024-02-23T00:42:58.838833+0000","last_became_peered":"2024-02-23T00:42:58.838833+0000","last_unstale":"2024-02-23T00:45:05.321520+0000","last_undegraded":"2024-02-23T00:45:05.321520+0000","last_fullsized":"2024-02-23T00:45:05.321520+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":134,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.126482+0000","last_change":"2024-02-23T00:42:58.698567+0000","last_active":"2024-02-23T00:45:05.126482+0000","last_peered":"2024-02-23T00:45:05.126482+0000","last_clean":"2024-02-23T00:45:05.126482+0000","last_became_active":"2024-02-23T00:42:58.698258+0000","last_became_peered":"2024-02-23T00:42:58.698258+0000","last_unstale":"2024-02-23T00:45:05.126482+0000","last_undegraded":"2024-02-23T00:45:05.126482+0000","last_fullsized":"2024-02-23T00:45:05.126482+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"95'27","reported_seq":248,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.128741+0000","last_change":"2024-02-23T00:43:52.124336+0000","last_active":"2024-02-23T00:45:06.128741+0000","last_peered":"2024-02-23T00:45:06.128741+0000","last_clean":"2024-02-23T00:45:06.128741+0000","last_became_active":"2024-02-23T00:39:16.786791+0000","last_became_peered":"2024-02-23T00:39:16.786791+0000","last_unstale":"2024-02-23T00:45:06.128741+0000","last_undegraded":"2024-02-23T00:45:06.128741+0000","last_fullsized":"2024-02-23T00:45:06.128741+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'27","last_scrub_stamp":"2024-02-23T00:43:52.124247+0000","last_deep_scrub":"95'27","last_deep_scrub_stamp":"2024-02-23T00:41:55.919895+0000","last_clean_scrub_stamp":"2024-02-23T00:43:52.124247+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":196,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.238017+0000","last_change":"2024-02-23T00:44:27.759571+0000","last_active":"2024-02-23T00:45:05.238017+0000","last_peered":"2024-02-23T00:45:05.238017+0000","last_clean":"2024-02-23T00:45:05.238017+0000","last_became_active":"2024-02-23T00:39:14.787148+0000","last_became_peered":"2024-02-23T00:39:14.787148+0000","last_unstale":"2024-02-23T00:45:05.238017+0000","last_undegraded":"2024-02-23T00:45:05.238017+0000","last_fullsized":"2024-02-23T00:45:05.238017+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:27.759499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:27.759499+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":189,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.126303+0000","last_change":"2024-02-23T00:42:51.967597+0000","last_active":"2024-02-23T00:45:05.126303+0000","last_peered":"2024-02-23T00:45:05.126303+0000","last_clean":"2024-02-23T00:45:05.126303+0000","last_became_active":"2024-02-23T00:39:18.796431+0000","last_became_peered":"2024-02-23T00:39:18.796431+0000","last_unstale":"2024-02-23T00:45:05.126303+0000","last_undegraded":"2024-02-23T00:45:05.126303+0000","last_fullsized":"2024-02-23T00:45:05.126303+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:51.967526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:51.967526+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.11","version":"110'7","reported_seq":147,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.131450+0000","last_change":"2024-02-23T00:44:30.366468+0000","last_active":"2024-02-23T00:45:06.131450+0000","last_peered":"2024-02-23T00:45:06.131450+0000","last_clean":"2024-02-23T00:45:06.131450+0000","last_became_active":"2024-02-23T00:42:58.810765+0000","last_became_peered":"2024-02-23T00:42:58.810765+0000","last_unstale":"2024-02-23T00:45:06.131450+0000","last_undegraded":"2024-02-23T00:45:06.131450+0000","last_fullsized":"2024-02-23T00:45:06.131450+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'7","last_scrub_stamp":"2024-02-23T00:44:30.366391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:44:30.366391+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":134,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.125950+0000","last_change":"2024-02-23T00:42:58.833221+0000","last_active":"2024-02-23T00:45:05.125950+0000","last_peered":"2024-02-23T00:45:05.125950+0000","last_clean":"2024-02-23T00:45:05.125950+0000","last_became_active":"2024-02-23T00:42:58.832885+0000","last_became_peered":"2024-02-23T00:42:58.832885+0000","last_unstale":"2024-02-23T00:45:05.125950+0000","last_undegraded":"2024-02-23T00:45:05.125950+0000","last_fullsized":"2024-02-23T00:45:05.125950+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"82'15","reported_seq":225,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.133475+0000","last_change":"2024-02-23T00:44:35.095478+0000","last_active":"2024-02-23T00:45:06.133475+0000","last_peered":"2024-02-23T00:45:06.133475+0000","last_clean":"2024-02-23T00:45:06.133475+0000","last_became_active":"2024-02-23T00:39:16.782762+0000","last_became_peered":"2024-02-23T00:39:16.782762+0000","last_unstale":"2024-02-23T00:45:06.133475+0000","last_undegraded":"2024-02-23T00:45:06.133475+0000","last_fullsized":"2024-02-23T00:45:06.133475+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'15","last_scrub_stamp":"2024-02-23T00:44:35.095409+0000","last_deep_scrub":"82'15","last_deep_scrub_stamp":"2024-02-23T00:41:50.975683+0000","last_clean_scrub_stamp":"2024-02-23T00:44:35.095409+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":196,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.238397+0000","last_change":"2024-02-23T00:43:44.794995+0000","last_active":"2024-02-23T00:45:05.238397+0000","last_peered":"2024-02-23T00:45:05.238397+0000","last_clean":"2024-02-23T00:45:05.238397+0000","last_became_active":"2024-02-23T00:39:14.783469+0000","last_became_peered":"2024-02-23T00:39:14.783469+0000","last_unstale":"2024-02-23T00:45:05.238397+0000","last_undegraded":"2024-02-23T00:45:05.238397+0000","last_fullsized":"2024-02-23T00:45:05.238397+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:44.794928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:44.794928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":194,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.238496+0000","last_change":"2024-02-23T00:43:41.817455+0000","last_active":"2024-02-23T00:45:05.238496+0000","last_peered":"2024-02-23T00:45:05.238496+0000","last_clean":"2024-02-23T00:45:05.238496+0000","last_became_active":"2024-02-23T00:39:18.796952+0000","last_became_peered":"2024-02-23T00:39:18.796952+0000","last_unstale":"2024-02-23T00:45:05.238496+0000","last_undegraded":"2024-02-23T00:45:05.238496+0000","last_fullsized":"2024-02-23T00:45:05.238496+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:41.817387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:41.817387+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":135,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.133897+0000","last_change":"2024-02-23T00:42:58.731558+0000","last_active":"2024-02-23T00:45:06.133897+0000","last_peered":"2024-02-23T00:45:06.133897+0000","last_clean":"2024-02-23T00:45:06.133897+0000","last_became_active":"2024-02-23T00:42:58.726021+0000","last_became_peered":"2024-02-23T00:42:58.726021+0000","last_unstale":"2024-02-23T00:45:06.133897+0000","last_undegraded":"2024-02-23T00:45:06.133897+0000","last_fullsized":"2024-02-23T00:45:06.133897+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":135,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.130037+0000","last_change":"2024-02-23T00:42:58.759026+0000","last_active":"2024-02-23T00:45:06.130037+0000","last_peered":"2024-02-23T00:45:06.130037+0000","last_clean":"2024-02-23T00:45:06.130037+0000","last_became_active":"2024-02-23T00:42:58.758147+0000","last_became_peered":"2024-02-23T00:42:58.758147+0000","last_unstale":"2024-02-23T00:45:06.130037+0000","last_undegraded":"2024-02-23T00:45:06.130037+0000","last_fullsized":"2024-02-23T00:45:06.130037+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"68'15","reported_seq":227,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.129313+0000","last_change":"2024-02-23T00:43:49.139235+0000","last_active":"2024-02-23T00:45:06.129313+0000","last_peered":"2024-02-23T00:45:06.129313+0000","last_clean":"2024-02-23T00:45:06.129313+0000","last_became_active":"2024-02-23T00:39:16.784606+0000","last_became_peered":"2024-02-23T00:39:16.784606+0000","last_unstale":"2024-02-23T00:45:06.129313+0000","last_undegraded":"2024-02-23T00:45:06.129313+0000","last_fullsized":"2024-02-23T00:45:06.129313+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:43:49.139156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:49.139156+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":199,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.133192+0000","last_change":"2024-02-23T00:44:32.170717+0000","last_active":"2024-02-23T00:45:06.133192+0000","last_peered":"2024-02-23T00:45:06.133192+0000","last_clean":"2024-02-23T00:45:06.133192+0000","last_became_active":"2024-02-23T00:39:14.785362+0000","last_became_peered":"2024-02-23T00:39:14.785362+0000","last_unstale":"2024-02-23T00:45:06.133192+0000","last_undegraded":"2024-02-23T00:45:06.133192+0000","last_fullsized":"2024-02-23T00:45:06.133192+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:32.170554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:32.170554+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":188,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.315459+0000","last_change":"2024-02-23T00:42:44.553722+0000","last_active":"2024-02-23T00:45:05.315459+0000","last_peered":"2024-02-23T00:45:05.315459+0000","last_clean":"2024-02-23T00:45:05.315459+0000","last_became_active":"2024-02-23T00:39:18.796312+0000","last_became_peered":"2024-02-23T00:39:18.796312+0000","last_unstale":"2024-02-23T00:45:05.315459+0000","last_undegraded":"2024-02-23T00:45:05.315459+0000","last_fullsized":"2024-02-23T00:45:05.315459+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:44.553621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:44.553621+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.13","version":"110'9","reported_seq":143,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.319854+0000","last_change":"2024-02-23T00:42:57.656016+0000","last_active":"2024-02-23T00:45:05.319854+0000","last_peered":"2024-02-23T00:45:05.319854+0000","last_clean":"2024-02-23T00:45:05.319854+0000","last_became_active":"2024-02-23T00:42:57.655617+0000","last_became_peered":"2024-02-23T00:42:57.655617+0000","last_unstale":"2024-02-23T00:45:05.319854+0000","last_undegraded":"2024-02-23T00:45:05.319854+0000","last_fullsized":"2024-02-23T00:45:05.319854+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":134,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.321031+0000","last_change":"2024-02-23T00:42:58.839750+0000","last_active":"2024-02-23T00:45:05.321031+0000","last_peered":"2024-02-23T00:45:05.321031+0000","last_clean":"2024-02-23T00:45:05.321031+0000","last_became_active":"2024-02-23T00:42:58.838257+0000","last_became_peered":"2024-02-23T00:42:58.838257+0000","last_unstale":"2024-02-23T00:45:05.321031+0000","last_undegraded":"2024-02-23T00:45:05.321031+0000","last_fullsized":"2024-02-23T00:45:05.321031+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"88'50","reported_seq":293,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.127647+0000","last_change":"2024-02-23T00:45:06.038488+0000","last_active":"2024-02-23T00:45:06.127647+0000","last_peered":"2024-02-23T00:45:06.127647+0000","last_clean":"2024-02-23T00:45:06.127647+0000","last_became_active":"2024-02-23T00:39:16.780144+0000","last_became_peered":"2024-02-23T00:39:16.780144+0000","last_unstale":"2024-02-23T00:45:06.127647+0000","last_undegraded":"2024-02-23T00:45:06.127647+0000","last_fullsized":"2024-02-23T00:45:06.127647+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'50","last_scrub_stamp":"2024-02-23T00:45:06.038374+0000","last_deep_scrub":"88'50","last_deep_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.038374+0000","log_size":50,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":193,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.324903+0000","last_change":"2024-02-23T00:42:38.204365+0000","last_active":"2024-02-23T00:45:05.324903+0000","last_peered":"2024-02-23T00:45:05.324903+0000","last_clean":"2024-02-23T00:45:05.324903+0000","last_became_active":"2024-02-23T00:39:14.782000+0000","last_became_peered":"2024-02-23T00:39:14.782000+0000","last_unstale":"2024-02-23T00:45:05.324903+0000","last_undegraded":"2024-02-23T00:45:05.324903+0000","last_fullsized":"2024-02-23T00:45:05.324903+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:38.204283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_clean_scrub_stamp":"2024-02-23T00:42:38.204283+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":189,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.134090+0000","last_change":"2024-02-23T00:43:48.126631+0000","last_active":"2024-02-23T00:45:06.134090+0000","last_peered":"2024-02-23T00:45:06.134090+0000","last_clean":"2024-02-23T00:45:06.134090+0000","last_became_active":"2024-02-23T00:39:18.799723+0000","last_became_peered":"2024-02-23T00:39:18.799723+0000","last_unstale":"2024-02-23T00:45:06.134090+0000","last_undegraded":"2024-02-23T00:45:06.134090+0000","last_fullsized":"2024-02-23T00:45:06.134090+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:48.126552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.126552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":135,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-02-23T00:45:06.125267+0000","last_change":"2024-02-23T00:42:58.772753+0000","last_active":"2024-02-23T00:45:06.125267+0000","last_peered":"2024-02-23T00:45:06.125267+0000","last_clean":"2024-02-23T00:45:06.125267+0000","last_became_active":"2024-02-23T00:42:58.765643+0000","last_became_peered":"2024-02-23T00:42:58.765643+0000","last_unstale":"2024-02-23T00:45:06.125267+0000","last_undegraded":"2024-02-23T00:45:06.125267+0000","last_fullsized":"2024-02-23T00:45:06.125267+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":134,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-02-23T00:45:05.124077+0000","last_change":"2024-02-23T00:42:58.825262+0000","last_active":"2024-02-23T00:45:05.124077+0000","last_peered":"2024-02-23T00:45:05.124077+0000","last_clean":"2024-02-23T00:45:05.124077+0000","last_became_active":"2024-02-23T00:42:58.824573+0000","last_became_peered":"2024-02-23T00:42:58.824573+0000","last_unstale":"2024-02-23T00:45:05.124077+0000","last_undegraded":"2024-02-23T00:45:05.124077+0000","last_fullsized":"2024-02-23T00:45:05.124077+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0],"acting":[5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":62,"num_pg":32,"stat_sum":{"num_bytes":1577,"num_objects":149,"num_object_clones":39,"num_object_copies":447,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":89,"num_read_kb":1,"num_write":518,"num_write_kb":516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1818624,"data_stored":4731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1655,"ondisk_log_size":1655,"up":96,"acting":96,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":22,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":66,"acting":66,"num_store_stats":8},{"poolid":153,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1080,"num_read_kb":805,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":641,"ondisk_log_size":641,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":344,"num_read_kb":339,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2048,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":196608,"data_stored":6144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234058,"num_pgs":118,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":313436,"kb_used_data":3920,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93431716,"statfs":{"total":95995035648,"available":95674077184,"internally_reserved":0,"allocated":4014080,"data_stored":2253409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.56100000000000005,"15min":0.55100000000000005},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":4.9420000000000002,"5min":4.9420000000000002,"15min":4.9420000000000002},"last":0.503},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.54100000000000004,"15min":0.51900000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":4.9729999999999999,"5min":4.9729999999999999,"15min":4.9729999999999999},"last":0.40300000000000002}]},{"osd":1,"last update":"Fri Feb 23 00:44:19 2024","interfaces":[{"interface":"back","average":{"1min":1.6519999999999999,"5min":0.75,"15min":0.61799999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":10.786,"5min":10.786,"15min":10.786},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.3640000000000001,"5min":0.69999999999999996,"15min":0.59999999999999998},"min":{"1min":0.34399999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":11.433999999999999,"5min":11.433999999999999,"15min":11.433999999999999},"last":0.46999999999999997}]},{"osd":2,"last update":"Fri Feb 23 00:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.57999999999999996,"15min":0.55400000000000005},"min":{"1min":0.315,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":5.0789999999999997,"5min":5.0789999999999997,"15min":5.0789999999999997},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.57999999999999996,"15min":0.55000000000000004},"min":{"1min":0.35699999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":5.1280000000000001,"5min":5.1280000000000001,"15min":5.1280000000000001},"last":0.441}]},{"osd":3,"last update":"Fri Feb 23 00:44:19 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":0.67400000000000004,"15min":0.57599999999999996},"min":{"1min":0.34799999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":5.7469999999999999,"5min":5.7469999999999999,"15min":5.7469999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":1.3109999999999999,"5min":0.72599999999999998,"15min":0.621},"min":{"1min":0.39000000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":9.4090000000000007,"5min":9.4090000000000007,"15min":9.4090000000000007},"last":0.56699999999999995}]},{"osd":4,"last update":"Fri Feb 23 00:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.42799999999999999,"15min":0.42999999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70099999999999996,"5min":0.754,"15min":0.754},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.45500000000000002,"15min":0.45400000000000001},"min":{"1min":0.29599999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.60599999999999998,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.48799999999999999}]},{"osd":5,"last update":"Fri Feb 23 00:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.47199999999999998,"15min":0.47599999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.63200000000000001,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.499,"15min":0.48299999999999998},"min":{"1min":0.29299999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.80500000000000005,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.58299999999999996}]},{"osd":6,"last update":"Fri Feb 23 00:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.50800000000000001,"15min":0.50600000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76200000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.501,"15min":0.505},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64600000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.59499999999999997}]}]},{"osd":6,"up_from":47,"seq":201863462993,"num_pgs":89,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309224,"kb_used_data":3792,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93435928,"statfs":{"total":95995035648,"available":95678390272,"internally_reserved":0,"allocated":3883008,"data_stored":2253542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:44:49 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.98799999999999999,"15min":0.71199999999999997},"min":{"1min":0.29699999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61399999999999999,"5min":30.641999999999999,"15min":30.641999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.98199999999999998,"15min":0.71799999999999997},"min":{"1min":0.311,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1,"5min":29.292000000000002,"15min":29.292000000000002},"last":0.46899999999999997}]},{"osd":1,"last update":"Fri Feb 23 00:44:49 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.98599999999999999,"15min":0.69799999999999995},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.59799999999999998,"5min":28.186,"15min":28.186},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.48899999999999999,"5min":1.4910000000000001,"15min":0.88900000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.97599999999999998,"5min":33.104999999999997,"15min":33.104999999999997},"last":0.40899999999999997}]},{"osd":2,"last update":"Fri Feb 23 00:44:49 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":1.0449999999999999,"15min":0.73899999999999999},"min":{"1min":0.32800000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.1200000000000001,"5min":19.960000000000001,"15min":19.960000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.84799999999999998,"15min":0.65600000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0549999999999999,"5min":8.5860000000000003,"15min":8.5860000000000003},"last":0.47799999999999998}]},{"osd":3,"last update":"Fri Feb 23 00:44:49 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.76000000000000001,"15min":0.64200000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.0069999999999999,"5min":19.562000000000001,"15min":19.562000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.85599999999999998,"15min":0.68200000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.94299999999999995,"5min":14.199,"15min":14.199},"last":0.55300000000000005}]},{"osd":4,"last update":"Fri Feb 23 00:44:49 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.57899999999999996,"15min":0.46100000000000002},"min":{"1min":0.25900000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.55800000000000005,"5min":11.109999999999999,"15min":11.109999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.751,"15min":0.53300000000000003},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.65600000000000003,"5min":16.734000000000002,"15min":16.734000000000002},"last":0.497}]},{"osd":5,"last update":"Fri Feb 23 00:44:49 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.629,"15min":0.48199999999999998},"min":{"1min":0.29299999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.59699999999999998,"5min":14.154999999999999,"15min":14.154999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.63500000000000001,"15min":0.5},"min":{"1min":0.312,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.67800000000000005,"5min":6.6100000000000003,"15min":6.6100000000000003},"last":0.39700000000000002}]},{"osd":7,"last update":"Fri Feb 23 00:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.58799999999999997,"15min":0.52100000000000002},"min":{"1min":0.30199999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.6949999999999998,"5min":8.548,"15min":8.548},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.75600000000000001,"15min":0.59499999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":4.6680000000000001,"5min":15.618,"15min":15.618},"last":0.59199999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574955,"num_pgs":123,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":305080,"kb_used_data":3752,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93440072,"statfs":{"total":95995035648,"available":95682633728,"internally_reserved":0,"allocated":3842048,"data_stored":2254241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:44:43 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":1.298,"15min":0.66500000000000004},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":1.637,"5min":78.361000000000004,"15min":78.361000000000004},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.57299999999999995,"15min":0.437},"min":{"1min":0.218,"5min":0.191,"15min":0.191},"max":{"1min":1.284,"5min":5.8849999999999998,"15min":5.8849999999999998},"last":0.39500000000000002}]},{"osd":2,"last update":"Fri Feb 23 00:45:03 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":1.1850000000000001,"15min":0.65900000000000003},"min":{"1min":0.247,"5min":0.219,"15min":0.188},"max":{"1min":0.63,"5min":48.606000000000002,"15min":48.606000000000002},"last":0.374},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.88300000000000001,"15min":0.58599999999999997},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.249},"max":{"1min":0.82599999999999996,"5min":23.911000000000001,"15min":23.911000000000001},"last":0.45400000000000001}]},{"osd":3,"last update":"Fri Feb 23 00:44:38 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.89800000000000002,"15min":0.63},"min":{"1min":0.29899999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":2.1949999999999998,"5min":35.244999999999997,"15min":35.244999999999997},"last":0.432},{"interface":"front","average":{"1min":0.70799999999999996,"5min":1.135,"15min":0.68799999999999994},"min":{"1min":0.27100000000000002,"5min":0.253,"15min":0.248},"max":{"1min":6.0510000000000002,"5min":56.853999999999999,"15min":56.853999999999999},"last":0.41299999999999998}]},{"osd":4,"last update":"Fri Feb 23 00:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":1.2350000000000001,"15min":0.81999999999999995},"min":{"1min":0.33700000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72899999999999998,"5min":37.646999999999998,"15min":37.646999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.875,"15min":0.69999999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83499999999999996,"5min":9.6859999999999999,"15min":9.6859999999999999},"last":0.57499999999999996}]},{"osd":5,"last update":"Fri Feb 23 00:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":1.222,"15min":0.85199999999999998},"min":{"1min":0.45100000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":5.9500000000000002,"5min":48.792000000000002,"15min":48.792000000000002},"last":0.626},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.83899999999999997,"15min":0.76300000000000001},"min":{"1min":0.41099999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":3.593,"5min":11.872999999999999,"15min":11.872999999999999},"last":0.51800000000000002}]},{"osd":6,"last update":"Fri Feb 23 00:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":1.2,"15min":0.82999999999999996},"min":{"1min":0.39200000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.90200000000000002,"5min":40.741,"15min":40.741},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.77800000000000002,"15min":0.69599999999999995},"min":{"1min":0.46100000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77200000000000002,"5min":6.0069999999999997,"15min":6.0069999999999997},"last":0.61399999999999999}]},{"osd":7,"last update":"Fri Feb 23 00:44:15 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":0.92000000000000004,"15min":0.72199999999999998},"min":{"1min":0.47599999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":3.6230000000000002,"5min":20.225000000000001,"15min":20.225000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":1.137,"5min":0.95099999999999996,"15min":0.73099999999999998},"min":{"1min":0.45000000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":5.9169999999999998,"5min":20.254999999999999,"15min":20.254999999999999},"last":0.63600000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738481,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309192,"kb_used_data":3772,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93435960,"statfs":{"total":95995035648,"available":95678423040,"internally_reserved":0,"allocated":3862528,"data_stored":2253602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 23 00:44:42 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":1.0980000000000001,"15min":0.59099999999999997},"min":{"1min":0.25900000000000001,"5min":0.223,"15min":0.19500000000000001},"max":{"1min":0.82799999999999996,"5min":28.856999999999999,"15min":28.856999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.3480000000000001,"5min":0.97799999999999998,"15min":0.56100000000000005},"min":{"1min":0.22700000000000001,"5min":0.218,"15min":0.20799999999999999},"max":{"1min":12.923,"5min":20.577000000000002,"15min":20.577000000000002},"last":13.162000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:45:06 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.026,"15min":0.60399999999999998},"min":{"1min":0.308,"5min":0.26000000000000001,"15min":0.20499999999999999},"max":{"1min":13.67,"5min":13.863,"15min":13.863},"last":13.67},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.88400000000000001,"15min":0.54900000000000004},"min":{"1min":0.25700000000000001,"5min":0.24299999999999999,"15min":0.19800000000000001},"max":{"1min":0.60099999999999998,"5min":10.455,"15min":10.455},"last":0.35799999999999998}]},{"osd":3,"last update":"Fri Feb 23 00:44:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.389,"15min":0.76400000000000001},"min":{"1min":0.28699999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":13.266,"5min":35.566000000000003,"15min":35.566000000000003},"last":12.566000000000001},{"interface":"front","average":{"1min":1.6859999999999999,"5min":1.24,"15min":0.751},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29399999999999998},"max":{"1min":16.931999999999999,"5min":20.702999999999999,"15min":20.702999999999999},"last":12.727}]},{"osd":4,"last update":"Fri Feb 23 00:44:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.038,"15min":0.81299999999999994},"min":{"1min":0.36199999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":10.584,"5min":10.584,"15min":10.584},"last":0.63700000000000001},{"interface":"front","average":{"1min":1.306,"5min":0.89600000000000002,"15min":0.79700000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":13.029999999999999,"5min":13.029999999999999,"15min":13.029999999999999},"last":13.198}]},{"osd":5,"last update":"Fri Feb 23 00:44:21 2024","interfaces":[{"interface":"back","average":{"1min":2.3239999999999998,"5min":1.014,"15min":0.748},"min":{"1min":0.379,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":10.397,"5min":10.397,"15min":10.397},"last":0.88200000000000001},{"interface":"front","average":{"1min":2.5339999999999998,"5min":1.091,"15min":0.81299999999999994},"min":{"1min":0.44800000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":17.297000000000001,"5min":17.297000000000001,"15min":17.297000000000001},"last":0.78000000000000003}]},{"osd":6,"last update":"Fri Feb 23 00:44:57 2024","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.1160000000000001,"15min":0.80200000000000005},"min":{"1min":0.434,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":13.237,"5min":17.373999999999999,"15min":17.373999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.3029999999999999,"5min":0.90200000000000002,"15min":0.72399999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":13.086,"5min":13.086,"15min":13.086},"last":13.973000000000001}]},{"osd":7,"last update":"Fri Feb 23 00:44:16 2024","interfaces":[{"interface":"back","average":{"1min":2.5449999999999999,"5min":1.095,"15min":0.81699999999999995},"min":{"1min":0.41799999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":17.399999999999999,"5min":17.399999999999999,"15min":17.399999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":1.958,"5min":0.89600000000000002,"15min":0.73299999999999998},"min":{"1min":0.433,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":13.061999999999999,"5min":13.061999999999999,"15min":13.061999999999999},"last":16.427}]}]},{"osd":2,"up_from":18,"seq":77309411430,"num_pgs":116,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":305288,"kb_used_data":3956,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93439864,"statfs":{"total":95995035648,"available":95682420736,"internally_reserved":0,"allocated":4050944,"data_stored":2253674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:45:02 2024","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":0.70199999999999996,"15min":0.48699999999999999},"min":{"1min":0.20499999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":13.138999999999999,"5min":13.657,"15min":13.657},"last":0.33500000000000002},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2649999999999999,"15min":0.68000000000000005},"min":{"1min":0.25700000000000001,"5min":0.246,"15min":0.23599999999999999},"max":{"1min":12.705,"5min":42.343000000000004,"15min":42.343000000000004},"last":0.38500000000000001}]},{"osd":1,"last update":"Fri Feb 23 00:45:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":0.96999999999999997,"15min":0.57499999999999996},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":8.8569999999999993,"5min":18.952000000000002,"15min":18.952000000000002},"last":0.32000000000000001},{"interface":"front","average":{"1min":1.6419999999999999,"5min":0.76800000000000002,"15min":0.52300000000000002},"min":{"1min":0.26100000000000001,"5min":0.23999999999999999,"15min":0.23599999999999999},"max":{"1min":15.391,"5min":15.391,"15min":15.391},"last":0.372}]},{"osd":3,"last update":"Fri Feb 23 00:44:34 2024","interfaces":[{"interface":"back","average":{"1min":2.3879999999999999,"5min":1.1879999999999999,"15min":0.72199999999999998},"min":{"1min":0.251,"5min":0.25,"15min":0.25},"max":{"1min":25.635999999999999,"5min":25.635999999999999,"15min":25.635999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.008,"15min":0.65000000000000002},"min":{"1min":0.27600000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":14.472,"5min":19.960000000000001,"15min":19.960000000000001},"last":0.35999999999999999}]},{"osd":4,"last update":"Fri Feb 23 00:45:02 2024","interfaces":[{"interface":"back","average":{"1min":2.1920000000000002,"5min":0.998,"15min":0.73099999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":13.673,"5min":13.673,"15min":13.673},"last":0.55100000000000005},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.044,"15min":0.73499999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.33900000000000002},"max":{"1min":8.7870000000000008,"5min":10.103999999999999,"15min":10.103999999999999},"last":0.45600000000000002}]},{"osd":5,"last update":"Fri Feb 23 00:44:20 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.66300000000000003,"15min":0.61099999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":2.3730000000000002,"5min":3.137,"15min":3.137},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.73599999999999999,"15min":0.65400000000000003},"min":{"1min":0.39700000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":3.9380000000000002,"5min":7.0019999999999998,"15min":7.0019999999999998},"last":0.48499999999999999}]},{"osd":6,"last update":"Fri Feb 23 00:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":0.872,"15min":0.70599999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":11.544,"5min":11.544,"15min":11.544},"last":0.56000000000000005},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0029999999999999,"15min":0.73199999999999998},"min":{"1min":0.38800000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":8.8290000000000006,"5min":10.132999999999999,"15min":10.132999999999999},"last":0.59899999999999998}]},{"osd":7,"last update":"Fri Feb 23 00:44:13 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.76500000000000001,"15min":0.70499999999999996},"min":{"1min":0.46500000000000002,"5min":0.377,"15min":0.377},"max":{"1min":3.9689999999999999,"5min":6.9740000000000002,"15min":6.9740000000000002},"last":0.51300000000000001},{"interface":"front","average":{"1min":2.0350000000000001,"5min":0.92900000000000005,"15min":0.74399999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":10.148999999999999,"5min":10.148999999999999,"15min":10.148999999999999},"last":0.56899999999999995}]}]},{"osd":3,"up_from":26,"seq":111669149791,"num_pgs":135,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309296,"kb_used_data":3868,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93435856,"statfs":{"total":95995035648,"available":95678316544,"internally_reserved":0,"allocated":3960832,"data_stored":2254671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:44:32 2024","interfaces":[{"interface":"back","average":{"1min":1.5489999999999999,"5min":0.754,"15min":0.54600000000000004},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":12.212999999999999,"5min":12.212999999999999,"15min":12.212999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":1.802,"5min":0.90400000000000003,"15min":0.60799999999999998},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":11.714,"5min":11.714,"15min":11.714},"last":0.55600000000000005}]},{"osd":1,"last update":"Fri Feb 23 00:44:32 2024","interfaces":[{"interface":"back","average":{"1min":1.627,"5min":0.88500000000000001,"15min":0.61499999999999999},"min":{"1min":0.23599999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":11.742000000000001,"5min":11.742000000000001,"15min":11.742000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":1.278,"5min":0.77300000000000002,"15min":0.56100000000000005},"min":{"1min":0.24199999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":6.383,"5min":6.383,"15min":6.383},"last":1.8300000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:44:32 2024","interfaces":[{"interface":"back","average":{"1min":1.8939999999999999,"5min":0.95399999999999996,"15min":0.65100000000000002},"min":{"1min":0.28899999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":13.359999999999999,"5min":13.359999999999999,"15min":13.359999999999999},"last":1.869},{"interface":"front","average":{"1min":2.403,"5min":1.01,"15min":0.66300000000000003},"min":{"1min":0.26200000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":21.122,"5min":21.122,"15min":21.122},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Feb 23 00:44:57 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.97399999999999998,"15min":0.72499999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":8.125,"5min":11.644,"15min":11.644},"last":0.628},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.85399999999999998,"15min":0.68000000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":6.2549999999999999,"5min":6.2549999999999999,"15min":6.2549999999999999},"last":2.2429999999999999}]},{"osd":5,"last update":"Fri Feb 23 00:44:20 2024","interfaces":[{"interface":"back","average":{"1min":1.5169999999999999,"5min":0.77600000000000002,"15min":0.66400000000000003},"min":{"1min":0.40200000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":5.6360000000000001,"5min":5.6360000000000001,"15min":5.6360000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":2.492,"5min":1.0229999999999999,"15min":0.76000000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":11.597,"5min":11.597,"15min":11.597},"last":0.65900000000000003}]},{"osd":6,"last update":"Fri Feb 23 00:44:46 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88100000000000001,"15min":0.70199999999999996},"min":{"1min":0.40899999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":6.343,"5min":6.343,"15min":6.343},"last":2.0539999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.80200000000000005,"15min":0.68600000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":2.8260000000000001,"5min":5.6680000000000001,"15min":5.6680000000000001},"last":0.60999999999999999}]},{"osd":7,"last update":"Fri Feb 23 00:44:16 2024","interfaces":[{"interface":"back","average":{"1min":2.5569999999999999,"5min":1.052,"15min":0.77600000000000002},"min":{"1min":0.434,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":11.67,"5min":11.67,"15min":11.67},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.821,"5min":0.93999999999999995,"15min":0.752},"min":{"1min":0.46999999999999997,"5min":0.433,"15min":0.433},"max":{"1min":6.3010000000000002,"5min":6.3010000000000002,"15min":6.3010000000000002},"last":2.1360000000000001}]}]},{"osd":4,"up_from":32,"seq":137438953563,"num_pgs":108,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309304,"kb_used_data":3884,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93435848,"statfs":{"total":95995035648,"available":95678308352,"internally_reserved":0,"allocated":3977216,"data_stored":2254335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:44:59 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.52100000000000002,"15min":0.56100000000000005},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.59499999999999997,"5min":3.5219999999999998,"15min":3.5219999999999998},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.66900000000000004,"15min":0.61399999999999999},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.64100000000000001,"5min":8.6140000000000008,"15min":8.6140000000000008},"last":0.44800000000000001}]},{"osd":1,"last update":"Fri Feb 23 00:44:59 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.66700000000000004,"15min":0.61299999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.27800000000000002},"max":{"1min":0.69799999999999995,"5min":9.6110000000000007,"15min":9.6110000000000007},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.66300000000000003,"15min":0.60499999999999998},"min":{"1min":0.33500000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.67400000000000004,"5min":6.6950000000000003,"15min":6.6950000000000003},"last":0.41499999999999998}]},{"osd":2,"last update":"Fri Feb 23 00:44:59 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.624,"15min":0.59399999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64800000000000002,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":0.379},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68700000000000006,"5min":5.8440000000000003,"15min":5.8440000000000003},"last":0.57099999999999995}]},{"osd":3,"last update":"Fri Feb 23 00:44:59 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.65800000000000003,"15min":0.60599999999999998},"min":{"1min":0.34399999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65900000000000003,"5min":6.5540000000000003,"15min":6.5540000000000003},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.66200000000000003,"15min":0.60899999999999999},"min":{"1min":0.32700000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73699999999999999,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":0.53300000000000003}]},{"osd":5,"last update":"Fri Feb 23 00:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.42499999999999999,"15min":0.38100000000000001},"min":{"1min":0.27000000000000002,"5min":0.251,"15min":0.246},"max":{"1min":0.47999999999999998,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.49099999999999999,"15min":0.43099999999999999},"min":{"1min":0.33400000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.56499999999999995,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.47599999999999998}]},{"osd":6,"last update":"Fri Feb 23 00:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.45000000000000001,"15min":0.41299999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.68100000000000005,"5min":0.75,"15min":0.75},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.45400000000000001,"15min":0.41299999999999998},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.66100000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.55600000000000005}]},{"osd":7,"last update":"Fri Feb 23 00:44:16 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.48899999999999999,"15min":0.48399999999999999},"min":{"1min":0.27300000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.53800000000000003,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.34599999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.57599999999999996,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.59299999999999997}]}]},{"osd":5,"up_from":39,"seq":167503724630,"num_pgs":114,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309200,"kb_used_data":3772,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93435952,"statfs":{"total":95995035648,"available":95678414848,"internally_reserved":0,"allocated":3862528,"data_stored":2254570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.79800000000000004,"15min":0.60599999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":3.7480000000000002,"5min":23.881,"15min":23.881},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.69799999999999995,"15min":0.58299999999999996},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":2.9239999999999999,"5min":5.4649999999999999,"15min":5.4649999999999999},"last":0.505}]},{"osd":1,"last update":"Fri Feb 23 00:44:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":0.90000000000000002,"15min":0.65100000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":5.6509999999999998,"5min":18.175999999999998,"15min":18.175999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":1.248,"5min":0.79900000000000004,"15min":0.629},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":4.2240000000000002,"5min":9.0950000000000006,"15min":9.0950000000000006},"last":0.46700000000000003}]},{"osd":2,"last update":"Fri Feb 23 00:44:19 2024","interfaces":[{"interface":"back","average":{"1min":1.831,"5min":1.032,"15min":0.71199999999999997},"min":{"1min":0.36399999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":10.590999999999999,"5min":22.282,"15min":22.282},"last":0.47599999999999998},{"interface":"front","average":{"1min":1.536,"5min":1.0640000000000001,"15min":0.71199999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":11.196999999999999,"5min":23.925000000000001,"15min":23.925000000000001},"last":0.45600000000000002}]},{"osd":3,"last update":"Fri Feb 23 00:44:19 2024","interfaces":[{"interface":"back","average":{"1min":1.5860000000000001,"5min":0.78700000000000003,"15min":0.61699999999999999},"min":{"1min":0.36499999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":6.5730000000000004,"5min":6.5730000000000004,"15min":6.5730000000000004},"last":0.52800000000000002},{"interface":"front","average":{"1min":1.4419999999999999,"5min":0.81100000000000005,"15min":0.64300000000000002},"min":{"1min":0.39100000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":6.7910000000000004,"5min":8.1760000000000002,"15min":8.1760000000000002},"last":0.56399999999999995}]},{"osd":4,"last update":"Fri Feb 23 00:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.432,"15min":0.39500000000000002},"min":{"1min":0.26000000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.68999999999999995,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.626,"5min":0.51900000000000002,"15min":0.42499999999999999},"min":{"1min":0.28199999999999997,"5min":0.27500000000000002,"15min":0.27300000000000002},"max":{"1min":3.879,"5min":3.879,"15min":3.879},"last":0.44}]},{"osd":6,"last update":"Fri Feb 23 00:44:44 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.48399999999999999,"15min":0.44400000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.98099999999999998,"5min":1.736,"15min":1.736},"last":0.379},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.49299999999999999,"15min":0.45600000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.40000000000000002}]},{"osd":7,"last update":"Fri Feb 23 00:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.58299999999999996,"15min":0.53300000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.83999999999999997,"5min":7.0300000000000002,"15min":7.0300000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.58599999999999997,"15min":0.53400000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.038,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":0.42699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":364544,"data_stored":937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":221184,"data_stored":575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":249856,"data_stored":654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":192512,"data_stored":505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":153,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":153,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":153,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":153,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":153,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":153,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":153,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":153,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-23T00:45:08.639 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph pg dump --format=json 2024-02-23T00:45:08.692 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:07.235364+0000 mgr.y (mgr.14517) 258 : cluster [DBG] pgmap v328: 250 pgs: 15 creating+activating, 16 creating+peering, 5 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 205 active+clean; 9.2 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:45:08.693 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:07.268336+0000 mgr.y (mgr.14517) 259 : audit [DBG] from='client.40820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:45:08.693 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:07.695280+0000 osd.1 (osd.1) 121 : cluster [DBG] 4.7 scrub starts 2024-02-23T00:45:08.693 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:07.697391+0000 osd.1 (osd.1) 122 : cluster [DBG] 4.7 scrub ok 2024-02-23T00:45:08.693 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:07.806300+0000 mon.b (mon.1) 295 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:08.693 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:08.115327+0000 osd.5 (osd.5) 107 : cluster [DBG] 5.f scrub starts 2024-02-23T00:45:08.693 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:08.118298+0000 osd.5 (osd.5) 108 : cluster [DBG] 5.f scrub ok 2024-02-23T00:45:08.693 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:08.132010+0000 mon.a (mon.0) 1640 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi047-82378-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi047-82378-32"}]': finished 2024-02-23T00:45:08.693 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:08.132168+0000 mon.a (mon.0) 1641 : cluster [DBG] osdmap e236: 8 total, 8 up, 8 in 2024-02-23T00:45:08.693 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:08.141281+0000 mon.a (mon.0) 1642 : audit [INF] from='client.? 172.21.15.47:0/3044143263' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:08.693 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:08.152511+0000 mon.a (mon.0) 1643 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:45:08.694 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:07.235364+0000 mgr.y (mgr.14517) 258 : cluster [DBG] pgmap v328: 250 pgs: 15 creating+activating, 16 creating+peering, 5 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 205 active+clean; 9.2 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:45:08.694 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:07.268336+0000 mgr.y (mgr.14517) 259 : audit [DBG] from='client.40820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:45:08.694 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:07.695280+0000 osd.1 (osd.1) 121 : cluster [DBG] 4.7 scrub starts 2024-02-23T00:45:08.694 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:07.697391+0000 osd.1 (osd.1) 122 : cluster [DBG] 4.7 scrub ok 2024-02-23T00:45:08.694 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:07.806300+0000 mon.b (mon.1) 295 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:08.694 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:08.115327+0000 osd.5 (osd.5) 107 : cluster [DBG] 5.f scrub starts 2024-02-23T00:45:08.694 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:08.118298+0000 osd.5 (osd.5) 108 : cluster [DBG] 5.f scrub ok 2024-02-23T00:45:08.694 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:08.132010+0000 mon.a (mon.0) 1640 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi047-82378-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi047-82378-32"}]': finished 2024-02-23T00:45:08.694 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:08.132168+0000 mon.a (mon.0) 1641 : cluster [DBG] osdmap e236: 8 total, 8 up, 8 in 2024-02-23T00:45:08.694 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:08.141281+0000 mon.a (mon.0) 1642 : audit [INF] from='client.? 172.21.15.47:0/3044143263' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:08.694 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:08.152511+0000 mon.a (mon.0) 1643 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:45:08.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:07.235364+0000 mgr.y (mgr.14517) 258 : cluster [DBG] pgmap v328: 250 pgs: 15 creating+activating, 16 creating+peering, 5 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 205 active+clean; 9.2 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:45:08.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:07.268336+0000 mgr.y (mgr.14517) 259 : audit [DBG] from='client.40820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:45:08.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:07.695280+0000 osd.1 (osd.1) 121 : cluster [DBG] 4.7 scrub starts 2024-02-23T00:45:08.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:07.697391+0000 osd.1 (osd.1) 122 : cluster [DBG] 4.7 scrub ok 2024-02-23T00:45:08.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:07.806300+0000 mon.b (mon.1) 295 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:08.115327+0000 osd.5 (osd.5) 107 : cluster [DBG] 5.f scrub starts 2024-02-23T00:45:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:08.118298+0000 osd.5 (osd.5) 108 : cluster [DBG] 5.f scrub ok 2024-02-23T00:45:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:08.132010+0000 mon.a (mon.0) 1640 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi047-82378-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi047-82378-32"}]': finished 2024-02-23T00:45:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:08.132168+0000 mon.a (mon.0) 1641 : cluster [DBG] osdmap e236: 8 total, 8 up, 8 in 2024-02-23T00:45:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:08.141281+0000 mon.a (mon.0) 1642 : audit [INF] from='client.? 172.21.15.47:0/3044143263' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:08.152511+0000 mon.a (mon.0) 1643 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:45:09.587 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:45:09.591 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:45:09.596 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:09.596 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:45:09.631 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:45:09.697 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:45:09.725 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:09.876 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:45:09.877 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:08.214104+0000 osd.4 (osd.4) 105 : cluster [DBG] 15.10 scrub starts 2024-02-23T00:45:09.877 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:08.215057+0000 osd.4 (osd.4) 106 : cluster [DBG] 15.10 scrub ok 2024-02-23T00:45:09.877 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:08.807380+0000 mon.b (mon.1) 296 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:09.877 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:09.134584+0000 mon.a (mon.0) 1644 : audit [INF] from='client.? 172.21.15.47:0/3044143263' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:09.877 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:09.134703+0000 mon.a (mon.0) 1645 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:45:09.877 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:09.134795+0000 mon.a (mon.0) 1646 : cluster [DBG] osdmap e237: 8 total, 8 up, 8 in 2024-02-23T00:45:09.877 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:09.136317+0000 mon.a (mon.0) 1647 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-23"}]: dispatch 2024-02-23T00:45:09.878 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:08.214104+0000 osd.4 (osd.4) 105 : cluster [DBG] 15.10 scrub starts 2024-02-23T00:45:09.878 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:08.215057+0000 osd.4 (osd.4) 106 : cluster [DBG] 15.10 scrub ok 2024-02-23T00:45:09.878 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:08.807380+0000 mon.b (mon.1) 296 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:09.878 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:09.134584+0000 mon.a (mon.0) 1644 : audit [INF] from='client.? 172.21.15.47:0/3044143263' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:09.878 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:09.134703+0000 mon.a (mon.0) 1645 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:45:09.878 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:09.134795+0000 mon.a (mon.0) 1646 : cluster [DBG] osdmap e237: 8 total, 8 up, 8 in 2024-02-23T00:45:09.878 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:09.136317+0000 mon.a (mon.0) 1647 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-23"}]: dispatch 2024-02-23T00:45:09.879 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:09.879 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:45:09.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:08.214104+0000 osd.4 (osd.4) 105 : cluster [DBG] 15.10 scrub starts 2024-02-23T00:45:09.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:08.215057+0000 osd.4 (osd.4) 106 : cluster [DBG] 15.10 scrub ok 2024-02-23T00:45:09.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:08.807380+0000 mon.b (mon.1) 296 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:09.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:09.134584+0000 mon.a (mon.0) 1644 : audit [INF] from='client.? 172.21.15.47:0/3044143263' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:09.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:09.134703+0000 mon.a (mon.0) 1645 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:45:09.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:09.134795+0000 mon.a (mon.0) 1646 : cluster [DBG] osdmap e237: 8 total, 8 up, 8 in 2024-02-23T00:45:09.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:09.136317+0000 mon.a (mon.0) 1647 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-23"}]: dispatch 2024-02-23T00:45:09.980 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:45:10.011 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:10.136 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:45:10.144 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:10.145 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:45:10.245 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:45:10.273 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:10.284 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:45:10.284 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-02-23T00:45:10.284 INFO:teuthology.orchestra.run.smithi047.stdout: "num_ops": 0 2024-02-23T00:45:10.285 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:45:10.292 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:45:10.294 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:10.295 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:45:10.396 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:45:10.422 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:10.441 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:45:10.444 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:10.444 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:45:10.545 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:45:10.576 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:10.807 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:45:10] "GET /metrics HTTP/1.1" 200 191767 "" "Prometheus/2.33.4" 2024-02-23T00:45:10.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:08.661219+0000 osd.0 (osd.0) 109 : cluster [DBG] 6.0 scrub starts 2024-02-23T00:45:10.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:08.662663+0000 osd.0 (osd.0) 110 : cluster [DBG] 6.0 scrub ok 2024-02-23T00:45:10.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:08.793145+0000 osd.7 (osd.7) 119 : cluster [DBG] 1.0 scrub starts 2024-02-23T00:45:10.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:08.796072+0000 osd.7 (osd.7) 120 : cluster [DBG] 1.0 scrub ok 2024-02-23T00:45:10.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:09.235765+0000 mgr.y (mgr.14517) 260 : cluster [DBG] pgmap v331: 290 pgs: 15 creating+activating, 40 unknown, 16 creating+peering, 5 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 205 active+clean; 9.2 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:45:10.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:09.639572+0000 osd.0 (osd.0) 111 : cluster [DBG] 18.e scrub starts 2024-02-23T00:45:10.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:09.808392+0000 mon.b (mon.1) 297 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:10.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:09.914213+0000 osd.3 (osd.3) 127 : cluster [DBG] 15.18 scrub starts 2024-02-23T00:45:10.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:09.917409+0000 osd.3 (osd.3) 128 : cluster [DBG] 15.18 scrub ok 2024-02-23T00:45:10.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:10.089307+0000 osd.5 (osd.5) 109 : cluster [DBG] 5.1b scrub starts 2024-02-23T00:45:10.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:10.090264+0000 osd.5 (osd.5) 110 : cluster [DBG] 5.1b scrub ok 2024-02-23T00:45:10.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:10.141073+0000 mon.a (mon.0) 1648 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-23"}]': finished 2024-02-23T00:45:10.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:10.141183+0000 mon.a (mon.0) 1649 : cluster [DBG] osdmap e238: 8 total, 8 up, 8 in 2024-02-23T00:45:10.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:10.143843+0000 mon.a (mon.0) 1650 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-23", "mode": "writeback"}]: dispatch 2024-02-23T00:45:10.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:10.144199+0000 mon.a (mon.0) 1651 : audit [INF] from='client.? 172.21.15.47:0/1125425092' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:10.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:10.156070+0000 mon.c (mon.2) 155 : audit [INF] from='client.? 172.21.15.47:0/2440014932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi047-82378-32"}]: dispatch 2024-02-23T00:45:10.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:10.156516+0000 mon.a (mon.0) 1652 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi047-82378-32"}]: dispatch 2024-02-23T00:45:10.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:08.661219+0000 osd.0 (osd.0) 109 : cluster [DBG] 6.0 scrub starts 2024-02-23T00:45:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:08.662663+0000 osd.0 (osd.0) 110 : cluster [DBG] 6.0 scrub ok 2024-02-23T00:45:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:08.793145+0000 osd.7 (osd.7) 119 : cluster [DBG] 1.0 scrub starts 2024-02-23T00:45:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:08.796072+0000 osd.7 (osd.7) 120 : cluster [DBG] 1.0 scrub ok 2024-02-23T00:45:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:09.235765+0000 mgr.y (mgr.14517) 260 : cluster [DBG] pgmap v331: 290 pgs: 15 creating+activating, 40 unknown, 16 creating+peering, 5 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 205 active+clean; 9.2 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:45:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:09.639572+0000 osd.0 (osd.0) 111 : cluster [DBG] 18.e scrub starts 2024-02-23T00:45:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:09.808392+0000 mon.b (mon.1) 297 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:09.914213+0000 osd.3 (osd.3) 127 : cluster [DBG] 15.18 scrub starts 2024-02-23T00:45:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:09.917409+0000 osd.3 (osd.3) 128 : cluster [DBG] 15.18 scrub ok 2024-02-23T00:45:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:10.089307+0000 osd.5 (osd.5) 109 : cluster [DBG] 5.1b scrub starts 2024-02-23T00:45:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:10.090264+0000 osd.5 (osd.5) 110 : cluster [DBG] 5.1b scrub ok 2024-02-23T00:45:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:10.141073+0000 mon.a (mon.0) 1648 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-23"}]': finished 2024-02-23T00:45:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:10.141183+0000 mon.a (mon.0) 1649 : cluster [DBG] osdmap e238: 8 total, 8 up, 8 in 2024-02-23T00:45:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:10.143843+0000 mon.a (mon.0) 1650 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-23", "mode": "writeback"}]: dispatch 2024-02-23T00:45:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:10.144199+0000 mon.a (mon.0) 1651 : audit [INF] from='client.? 172.21.15.47:0/1125425092' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:10.156070+0000 mon.c (mon.2) 155 : audit [INF] from='client.? 172.21.15.47:0/2440014932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi047-82378-32"}]: dispatch 2024-02-23T00:45:10.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:10.156516+0000 mon.a (mon.0) 1652 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi047-82378-32"}]: dispatch 2024-02-23T00:45:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:45:10.827 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:10.828 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:45:10.929 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:45:10.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:08.661219+0000 osd.0 (osd.0) 109 : cluster [DBG] 6.0 scrub starts 2024-02-23T00:45:10.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:08.662663+0000 osd.0 (osd.0) 110 : cluster [DBG] 6.0 scrub ok 2024-02-23T00:45:10.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:08.793145+0000 osd.7 (osd.7) 119 : cluster [DBG] 1.0 scrub starts 2024-02-23T00:45:10.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:08.796072+0000 osd.7 (osd.7) 120 : cluster [DBG] 1.0 scrub ok 2024-02-23T00:45:10.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:09.235765+0000 mgr.y (mgr.14517) 260 : cluster [DBG] pgmap v331: 290 pgs: 15 creating+activating, 40 unknown, 16 creating+peering, 5 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 205 active+clean; 9.2 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:45:10.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:09.639572+0000 osd.0 (osd.0) 111 : cluster [DBG] 18.e scrub starts 2024-02-23T00:45:10.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:09.808392+0000 mon.b (mon.1) 297 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:10.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:09.914213+0000 osd.3 (osd.3) 127 : cluster [DBG] 15.18 scrub starts 2024-02-23T00:45:10.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:09.917409+0000 osd.3 (osd.3) 128 : cluster [DBG] 15.18 scrub ok 2024-02-23T00:45:10.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:10.089307+0000 osd.5 (osd.5) 109 : cluster [DBG] 5.1b scrub starts 2024-02-23T00:45:10.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:10.090264+0000 osd.5 (osd.5) 110 : cluster [DBG] 5.1b scrub ok 2024-02-23T00:45:10.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:10.141073+0000 mon.a (mon.0) 1648 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-23"}]': finished 2024-02-23T00:45:10.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:10.141183+0000 mon.a (mon.0) 1649 : cluster [DBG] osdmap e238: 8 total, 8 up, 8 in 2024-02-23T00:45:10.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:10.143843+0000 mon.a (mon.0) 1650 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-23", "mode": "writeback"}]: dispatch 2024-02-23T00:45:10.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:10.144199+0000 mon.a (mon.0) 1651 : audit [INF] from='client.? 172.21.15.47:0/1125425092' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:10.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:10.156070+0000 mon.c (mon.2) 155 : audit [INF] from='client.? 172.21.15.47:0/2440014932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi047-82378-32"}]: dispatch 2024-02-23T00:45:10.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:10.156516+0000 mon.a (mon.0) 1652 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi047-82378-32"}]: dispatch 2024-02-23T00:45:10.958 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:10.977 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:45:10.980 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:10.980 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:45:11.081 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:45:11.109 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:11.147 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:45:11.157 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:11.157 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:45:11.258 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:45:11.286 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:11.304 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:45:11.307 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:11.307 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:45:11.408 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:45:11.435 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:11.454 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:45:11.456 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:11.457 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:45:11.557 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:45:11.584 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:11.602 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:45:11.604 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:11.605 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:45:11.614 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:45:11.684 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.1 dump_blocked_ops 2024-02-23T00:45:11.705 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:45:11.731 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:11.749 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:45:11.751 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:11.751 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:45:11.852 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:45:11.877 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:09.640932+0000 osd.0 (osd.0) 112 : cluster [DBG] 18.e scrub ok 2024-02-23T00:45:11.877 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:10.464210+0000 osd.2 (osd.2) 107 : cluster [DBG] 15.b scrub starts 2024-02-23T00:45:11.877 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:10.465102+0000 osd.2 (osd.2) 108 : cluster [DBG] 15.b scrub ok 2024-02-23T00:45:11.877 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:10.690120+0000 osd.1 (osd.1) 123 : cluster [DBG] 2.a scrub starts 2024-02-23T00:45:11.877 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:10.693269+0000 osd.1 (osd.1) 124 : cluster [DBG] 2.a scrub ok 2024-02-23T00:45:11.877 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:10.738786+0000 osd.7 (osd.7) 121 : cluster [DBG] 6.12 scrub starts 2024-02-23T00:45:11.877 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:10.741691+0000 osd.7 (osd.7) 122 : cluster [DBG] 6.12 scrub ok 2024-02-23T00:45:11.877 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:10.809158+0000 mon.b (mon.1) 298 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:11.878 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:11.138439+0000 mon.a (mon.0) 1653 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:45:11.878 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:11.143152+0000 mon.a (mon.0) 1654 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-23", "mode": "writeback"}]': finished 2024-02-23T00:45:11.878 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:11.143231+0000 mon.a (mon.0) 1655 : audit [INF] from='client.? 172.21.15.47:0/1125425092' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:11.878 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:11.143377+0000 mon.a (mon.0) 1656 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi047-82378-32"}]': finished 2024-02-23T00:45:11.878 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:11.143463+0000 mon.a (mon.0) 1657 : cluster [DBG] osdmap e239: 8 total, 8 up, 8 in 2024-02-23T00:45:11.878 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:11.146637+0000 mon.c (mon.2) 156 : audit [INF] from='client.? 172.21.15.47:0/2440014932' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi047-82378-32"}]: dispatch 2024-02-23T00:45:11.878 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:11.148470+0000 mon.a (mon.0) 1658 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi047-82378-32"}]: dispatch 2024-02-23T00:45:11.878 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:11.630714+0000 mon.a (mon.0) 1659 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:45:11.878 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:11.897 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:45:11.900 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:11.900 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:45:12.001 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:45:12.027 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:09.640932+0000 osd.0 (osd.0) 112 : cluster [DBG] 18.e scrub ok 2024-02-23T00:45:12.028 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:10.464210+0000 osd.2 (osd.2) 107 : cluster [DBG] 15.b scrub starts 2024-02-23T00:45:12.028 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:10.465102+0000 osd.2 (osd.2) 108 : cluster [DBG] 15.b scrub ok 2024-02-23T00:45:12.028 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:10.690120+0000 osd.1 (osd.1) 123 : cluster [DBG] 2.a scrub starts 2024-02-23T00:45:12.028 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:10.693269+0000 osd.1 (osd.1) 124 : cluster [DBG] 2.a scrub ok 2024-02-23T00:45:12.028 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:10.738786+0000 osd.7 (osd.7) 121 : cluster [DBG] 6.12 scrub starts 2024-02-23T00:45:12.028 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:10.741691+0000 osd.7 (osd.7) 122 : cluster [DBG] 6.12 scrub ok 2024-02-23T00:45:12.028 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:10.809158+0000 mon.b (mon.1) 298 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:12.028 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:11.138439+0000 mon.a (mon.0) 1653 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:45:12.028 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:11.143152+0000 mon.a (mon.0) 1654 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-23", "mode": "writeback"}]': finished 2024-02-23T00:45:12.028 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:11.143231+0000 mon.a (mon.0) 1655 : audit [INF] from='client.? 172.21.15.47:0/1125425092' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:12.028 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:11.143377+0000 mon.a (mon.0) 1656 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi047-82378-32"}]': finished 2024-02-23T00:45:12.028 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:11.143463+0000 mon.a (mon.0) 1657 : cluster [DBG] osdmap e239: 8 total, 8 up, 8 in 2024-02-23T00:45:12.028 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:11.146637+0000 mon.c (mon.2) 156 : audit [INF] from='client.? 172.21.15.47:0/2440014932' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi047-82378-32"}]: dispatch 2024-02-23T00:45:12.029 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:11.148470+0000 mon.a (mon.0) 1658 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi047-82378-32"}]: dispatch 2024-02-23T00:45:12.029 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:11.630714+0000 mon.a (mon.0) 1659 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:45:12.029 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:09.640932+0000 osd.0 (osd.0) 112 : cluster [DBG] 18.e scrub ok 2024-02-23T00:45:12.029 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:10.464210+0000 osd.2 (osd.2) 107 : cluster [DBG] 15.b scrub starts 2024-02-23T00:45:12.029 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:10.465102+0000 osd.2 (osd.2) 108 : cluster [DBG] 15.b scrub ok 2024-02-23T00:45:12.029 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:10.690120+0000 osd.1 (osd.1) 123 : cluster [DBG] 2.a scrub starts 2024-02-23T00:45:12.029 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:10.693269+0000 osd.1 (osd.1) 124 : cluster [DBG] 2.a scrub ok 2024-02-23T00:45:12.029 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:10.738786+0000 osd.7 (osd.7) 121 : cluster [DBG] 6.12 scrub starts 2024-02-23T00:45:12.029 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:10.741691+0000 osd.7 (osd.7) 122 : cluster [DBG] 6.12 scrub ok 2024-02-23T00:45:12.029 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:10.809158+0000 mon.b (mon.1) 298 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:12.029 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:11.138439+0000 mon.a (mon.0) 1653 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:45:12.029 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:11.143152+0000 mon.a (mon.0) 1654 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-23", "mode": "writeback"}]': finished 2024-02-23T00:45:12.030 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:11.143231+0000 mon.a (mon.0) 1655 : audit [INF] from='client.? 172.21.15.47:0/1125425092' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:12.030 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:11.143377+0000 mon.a (mon.0) 1656 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi047-82378-32"}]': finished 2024-02-23T00:45:12.030 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:11.143463+0000 mon.a (mon.0) 1657 : cluster [DBG] osdmap e239: 8 total, 8 up, 8 in 2024-02-23T00:45:12.030 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:11.146637+0000 mon.c (mon.2) 156 : audit [INF] from='client.? 172.21.15.47:0/2440014932' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi047-82378-32"}]: dispatch 2024-02-23T00:45:12.030 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:11.148470+0000 mon.a (mon.0) 1658 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi047-82378-32"}]: dispatch 2024-02-23T00:45:12.030 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:11.630714+0000 mon.a (mon.0) 1659 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:45:12.030 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:12.179 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:45:12.182 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:12.182 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:45:12.283 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:45:12.312 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:45:12.333 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:12.333 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:45:12.434 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:45:12.461 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:12.585 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:45:12.588 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:12.588 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:45:12.689 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:45:12.693 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:45:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:45:12.693 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:45:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:45:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:45:12.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:11.236179+0000 mgr.y (mgr.14517) 261 : cluster [DBG] pgmap v334: 314 pgs: 15 creating+activating, 64 unknown, 16 creating+peering, 5 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 205 active+clean; 9.2 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:45:12.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:11.710259+0000 osd.1 (osd.1) 125 : cluster [DBG] 4.2 scrub starts 2024-02-23T00:45:12.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:11.713727+0000 osd.1 (osd.1) 126 : cluster [DBG] 4.2 scrub ok 2024-02-23T00:45:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:11.810173+0000 mon.b (mon.1) 299 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:12.146816+0000 mon.a (mon.0) 1660 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi047-82378-32"}]': finished 2024-02-23T00:45:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:12.146946+0000 mon.a (mon.0) 1661 : cluster [DBG] osdmap e240: 8 total, 8 up, 8 in 2024-02-23T00:45:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:12.158023+0000 mon.c (mon.2) 157 : audit [INF] from='client.? 172.21.15.47:0/2166779162' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:12.158400+0000 mon.a (mon.0) 1662 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:12.175712+0000 mon.c (mon.2) 158 : audit [INF] from='client.? 172.21.15.47:0/1302226360' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi047-82378-33"}]: dispatch 2024-02-23T00:45:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:12.176114+0000 mon.a (mon.0) 1663 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi047-82378-33"}]: dispatch 2024-02-23T00:45:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:12.177347+0000 mon.c (mon.2) 159 : audit [INF] from='client.? 172.21.15.47:0/1302226360' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi047-82378-33"}]: dispatch 2024-02-23T00:45:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:12.177730+0000 mon.a (mon.0) 1664 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi047-82378-33"}]: dispatch 2024-02-23T00:45:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:12.178232+0000 mon.c (mon.2) 160 : audit [INF] from='client.? 172.21.15.47:0/1302226360' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi047-82378-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:12.178906+0000 mon.a (mon.0) 1665 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi047-82378-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:12.184249+0000 mon.a (mon.0) 1666 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:45:12.718 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:12.739 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:45:12.741 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:12.742 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:45:12.843 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:45:12.869 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:11.236179+0000 mgr.y (mgr.14517) 261 : cluster [DBG] pgmap v334: 314 pgs: 15 creating+activating, 64 unknown, 16 creating+peering, 5 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 205 active+clean; 9.2 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:45:12.870 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:11.710259+0000 osd.1 (osd.1) 125 : cluster [DBG] 4.2 scrub starts 2024-02-23T00:45:12.870 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:11.713727+0000 osd.1 (osd.1) 126 : cluster [DBG] 4.2 scrub ok 2024-02-23T00:45:12.870 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:11.810173+0000 mon.b (mon.1) 299 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:12.870 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:12.146816+0000 mon.a (mon.0) 1660 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi047-82378-32"}]': finished 2024-02-23T00:45:12.870 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:12.146946+0000 mon.a (mon.0) 1661 : cluster [DBG] osdmap e240: 8 total, 8 up, 8 in 2024-02-23T00:45:12.870 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:12.158023+0000 mon.c (mon.2) 157 : audit [INF] from='client.? 172.21.15.47:0/2166779162' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:12.870 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:12.158400+0000 mon.a (mon.0) 1662 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:12.870 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:12.175712+0000 mon.c (mon.2) 158 : audit [INF] from='client.? 172.21.15.47:0/1302226360' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi047-82378-33"}]: dispatch 2024-02-23T00:45:12.870 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:12.176114+0000 mon.a (mon.0) 1663 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi047-82378-33"}]: dispatch 2024-02-23T00:45:12.870 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:12.177347+0000 mon.c (mon.2) 159 : audit [INF] from='client.? 172.21.15.47:0/1302226360' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi047-82378-33"}]: dispatch 2024-02-23T00:45:12.870 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:12.177730+0000 mon.a (mon.0) 1664 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi047-82378-33"}]: dispatch 2024-02-23T00:45:12.870 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:12.178232+0000 mon.c (mon.2) 160 : audit [INF] from='client.? 172.21.15.47:0/1302226360' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi047-82378-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:12.870 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:12.178906+0000 mon.a (mon.0) 1665 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi047-82378-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:12.871 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:12.184249+0000 mon.a (mon.0) 1666 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:45:12.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:11.236179+0000 mgr.y (mgr.14517) 261 : cluster [DBG] pgmap v334: 314 pgs: 15 creating+activating, 64 unknown, 16 creating+peering, 5 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 205 active+clean; 9.2 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:45:12.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:11.710259+0000 osd.1 (osd.1) 125 : cluster [DBG] 4.2 scrub starts 2024-02-23T00:45:12.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:11.713727+0000 osd.1 (osd.1) 126 : cluster [DBG] 4.2 scrub ok 2024-02-23T00:45:12.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:11.810173+0000 mon.b (mon.1) 299 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:12.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:12.146816+0000 mon.a (mon.0) 1660 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi047-82378-32"}]': finished 2024-02-23T00:45:12.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:12.146946+0000 mon.a (mon.0) 1661 : cluster [DBG] osdmap e240: 8 total, 8 up, 8 in 2024-02-23T00:45:12.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:12.158023+0000 mon.c (mon.2) 157 : audit [INF] from='client.? 172.21.15.47:0/2166779162' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:12.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:12.158400+0000 mon.a (mon.0) 1662 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:12.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:12.175712+0000 mon.c (mon.2) 158 : audit [INF] from='client.? 172.21.15.47:0/1302226360' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi047-82378-33"}]: dispatch 2024-02-23T00:45:12.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:12.176114+0000 mon.a (mon.0) 1663 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi047-82378-33"}]: dispatch 2024-02-23T00:45:12.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:12.177347+0000 mon.c (mon.2) 159 : audit [INF] from='client.? 172.21.15.47:0/1302226360' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi047-82378-33"}]: dispatch 2024-02-23T00:45:12.872 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:12.177730+0000 mon.a (mon.0) 1664 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi047-82378-33"}]: dispatch 2024-02-23T00:45:12.872 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:12.178232+0000 mon.c (mon.2) 160 : audit [INF] from='client.? 172.21.15.47:0/1302226360' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi047-82378-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:12.872 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:12.178906+0000 mon.a (mon.0) 1665 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi047-82378-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:12.872 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:12.184249+0000 mon.a (mon.0) 1666 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:45:12.872 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:13.778 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:45:13.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:12.306117+0000 osd.4 (osd.4) 107 : cluster [DBG] 15.1d scrub starts 2024-02-23T00:45:13.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:12.307526+0000 osd.4 (osd.4) 108 : cluster [DBG] 15.1d scrub ok 2024-02-23T00:45:13.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:12.429621+0000 mgr.y (mgr.14517) 262 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:45:13.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:12.727499+0000 osd.7 (osd.7) 123 : cluster [DBG] 2.5 scrub starts 2024-02-23T00:45:13.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:12.730530+0000 osd.7 (osd.7) 124 : cluster [DBG] 2.5 scrub ok 2024-02-23T00:45:13.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:12.811197+0000 mon.b (mon.1) 300 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:13.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:13.148440+0000 mon.a (mon.0) 1667 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:13.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:13.148577+0000 mon.a (mon.0) 1668 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi047-82378-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:13.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:13.148652+0000 mon.a (mon.0) 1669 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:45:13.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:13.148703+0000 mon.a (mon.0) 1670 : cluster [DBG] osdmap e241: 8 total, 8 up, 8 in 2024-02-23T00:45:13.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:13.154216+0000 mon.c (mon.2) 161 : audit [INF] from='client.? 172.21.15.47:0/1302226360' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi047-82378-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi047-82378-33"}]: dispatch 2024-02-23T00:45:13.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:13.155751+0000 mon.a (mon.0) 1671 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi047-82378-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi047-82378-33"}]: dispatch 2024-02-23T00:45:13.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:13.157125+0000 mon.a (mon.0) 1672 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-23"}]: dispatch 2024-02-23T00:45:14.049 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:12.306117+0000 osd.4 (osd.4) 107 : cluster [DBG] 15.1d scrub starts 2024-02-23T00:45:14.049 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:12.307526+0000 osd.4 (osd.4) 108 : cluster [DBG] 15.1d scrub ok 2024-02-23T00:45:14.049 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:12.429621+0000 mgr.y (mgr.14517) 262 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:45:14.049 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:12.727499+0000 osd.7 (osd.7) 123 : cluster [DBG] 2.5 scrub starts 2024-02-23T00:45:14.049 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:12.730530+0000 osd.7 (osd.7) 124 : cluster [DBG] 2.5 scrub ok 2024-02-23T00:45:14.049 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:12.811197+0000 mon.b (mon.1) 300 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:14.049 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:13.148440+0000 mon.a (mon.0) 1667 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:14.049 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:13.148577+0000 mon.a (mon.0) 1668 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi047-82378-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:14.050 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:13.148652+0000 mon.a (mon.0) 1669 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:45:14.050 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:13.148703+0000 mon.a (mon.0) 1670 : cluster [DBG] osdmap e241: 8 total, 8 up, 8 in 2024-02-23T00:45:14.050 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:13.154216+0000 mon.c (mon.2) 161 : audit [INF] from='client.? 172.21.15.47:0/1302226360' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi047-82378-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi047-82378-33"}]: dispatch 2024-02-23T00:45:14.050 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:13.155751+0000 mon.a (mon.0) 1671 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi047-82378-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi047-82378-33"}]: dispatch 2024-02-23T00:45:14.050 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:13.157125+0000 mon.a (mon.0) 1672 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-23"}]: dispatch 2024-02-23T00:45:14.050 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:12.306117+0000 osd.4 (osd.4) 107 : cluster [DBG] 15.1d scrub starts 2024-02-23T00:45:14.050 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:12.307526+0000 osd.4 (osd.4) 108 : cluster [DBG] 15.1d scrub ok 2024-02-23T00:45:14.050 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:12.429621+0000 mgr.y (mgr.14517) 262 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:45:14.050 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:12.727499+0000 osd.7 (osd.7) 123 : cluster [DBG] 2.5 scrub starts 2024-02-23T00:45:14.050 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:12.730530+0000 osd.7 (osd.7) 124 : cluster [DBG] 2.5 scrub ok 2024-02-23T00:45:14.050 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:12.811197+0000 mon.b (mon.1) 300 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:14.050 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:13.148440+0000 mon.a (mon.0) 1667 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:14.051 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:13.148577+0000 mon.a (mon.0) 1668 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi047-82378-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:14.051 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:13.148652+0000 mon.a (mon.0) 1669 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:45:14.051 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:13.148703+0000 mon.a (mon.0) 1670 : cluster [DBG] osdmap e241: 8 total, 8 up, 8 in 2024-02-23T00:45:14.051 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:13.154216+0000 mon.c (mon.2) 161 : audit [INF] from='client.? 172.21.15.47:0/1302226360' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi047-82378-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi047-82378-33"}]: dispatch 2024-02-23T00:45:14.051 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:13.155751+0000 mon.a (mon.0) 1671 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi047-82378-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi047-82378-33"}]: dispatch 2024-02-23T00:45:14.051 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:13.157125+0000 mon.a (mon.0) 1672 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-23"}]: dispatch 2024-02-23T00:45:14.221 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:45:14.224 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:14.225 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:45:14.325 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:45:14.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.326+0000 7f83970a7700 1 -- 172.21.15.47:0/1776225910 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f83900fdbb0 msgr2=0x7f83900fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:14.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.326+0000 7f83970a7700 1 --2- 172.21.15.47:0/1776225910 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f83900fdbb0 0x7f83900fdf90 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 rx=0x7f8384004870 tx=0x7f8384009cb0).stop 2024-02-23T00:45:14.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.326+0000 7f83970a7700 1 -- 172.21.15.47:0/1776225910 shutdown_connections 2024-02-23T00:45:14.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.326+0000 7f83970a7700 1 --2- 172.21.15.47:0/1776225910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83900f8420 0x7f8390104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:14.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.326+0000 7f83970a7700 1 --2- 172.21.15.47:0/1776225910 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f83900fdbb0 0x7f83900fdf90 unknown :-1 s=CLOSED pgs=2563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:14.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.326+0000 7f83970a7700 1 --2- 172.21.15.47:0/1776225910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83900fe510 0x7f83900f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:14.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.326+0000 7f83970a7700 1 -- 172.21.15.47:0/1776225910 >> 172.21.15.47:0/1776225910 conn(0x7f83900f3800 msgr2=0x7f83900f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:45:14.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.326+0000 7f83970a7700 1 -- 172.21.15.47:0/1776225910 shutdown_connections 2024-02-23T00:45:14.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.326+0000 7f83970a7700 1 -- 172.21.15.47:0/1776225910 wait complete. 2024-02-23T00:45:14.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.327+0000 7f83970a7700 1 librados: starting msgr at 2024-02-23T00:45:14.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.327+0000 7f83970a7700 1 librados: starting objecter 2024-02-23T00:45:14.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.327+0000 7f83970a7700 1 Processor -- start 2024-02-23T00:45:14.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.327+0000 7f83970a7700 1 -- start start 2024-02-23T00:45:14.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.327+0000 7f83970a7700 1 librados: setting wanted keys 2024-02-23T00:45:14.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.327+0000 7f83970a7700 1 librados: calling monclient init 2024-02-23T00:45:14.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.327+0000 7f83970a7700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83900f8420 0x7f839010de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:14.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.327+0000 7f83970a7700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f83900fdbb0 0x7f839010e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:14.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.327+0000 7f83970a7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83900fe510 0x7f8390108070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:14.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.327+0000 7f83970a7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8390108560 con 0x7f83900fe510 2024-02-23T00:45:14.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.327+0000 7f83970a7700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83901086d0 con 0x7f83900f8420 2024-02-23T00:45:14.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.327+0000 7f83970a7700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8390108890 con 0x7f83900fdbb0 2024-02-23T00:45:14.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.327+0000 7f838ffff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f83900fdbb0 0x7f839010e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:14.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.327+0000 7f838ffff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f83900fdbb0 0x7f839010e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52364/0 (socket says 172.21.15.47:52364) 2024-02-23T00:45:14.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.327+0000 7f838ffff700 1 -- 172.21.15.47:0/1085564695 learned_addr learned my addr 172.21.15.47:0/1085564695 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:45:14.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.327+0000 7f8395644700 1 --2- 172.21.15.47:0/1085564695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83900fe510 0x7f8390108070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:14.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.327+0000 7f8394e43700 1 --2- 172.21.15.47:0/1085564695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83900f8420 0x7f839010de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:14.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.328+0000 7f838ffff700 1 -- 172.21.15.47:0/1085564695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83900f8420 msgr2=0x7f839010de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:14.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.328+0000 7f838ffff700 1 --2- 172.21.15.47:0/1085564695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83900f8420 0x7f839010de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:14.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.328+0000 7f838ffff700 1 -- 172.21.15.47:0/1085564695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83900fe510 msgr2=0x7f8390108070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:14.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.328+0000 7f838ffff700 1 --2- 172.21.15.47:0/1085564695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83900fe510 0x7f8390108070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:14.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.328+0000 7f838ffff700 1 -- 172.21.15.47:0/1085564695 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8384005040 con 0x7f83900fdbb0 2024-02-23T00:45:14.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.328+0000 7f838ffff700 1 --2- 172.21.15.47:0/1085564695 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f83900fdbb0 0x7f839010e370 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 rx=0x7f837c0041d0 tx=0x7f837c009f60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:14.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.328+0000 7f838dffb700 10 client.?.objecter ms_handle_connect 0x7f83900fdbb0 2024-02-23T00:45:14.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.328+0000 7f838dffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:45:14.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.328+0000 7f838dffb700 1 -- 172.21.15.47:0/1085564695 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837c007b40 con 0x7f83900fdbb0 2024-02-23T00:45:14.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.328+0000 7f838dffb700 1 -- 172.21.15.47:0/1085564695 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f837c00f040 con 0x7f83900fdbb0 2024-02-23T00:45:14.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.328+0000 7f83970a7700 1 -- 172.21.15.47:0/1085564695 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8390108b70 con 0x7f83900fdbb0 2024-02-23T00:45:14.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.328+0000 7f838dffb700 1 -- 172.21.15.47:0/1085564695 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837c014730 con 0x7f83900fdbb0 2024-02-23T00:45:14.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.328+0000 7f83970a7700 10 client.40853.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:45:14.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.328+0000 7f83970a7700 1 -- 172.21.15.47:0/1085564695 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8390108ef0 con 0x7f83900fdbb0 2024-02-23T00:45:14.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.328+0000 7f83970a7700 1 librados: init done 2024-02-23T00:45:14.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.329+0000 7f83970a7700 1 -- 172.21.15.47:0/1085564695 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f839005ee80 con 0x7f83900fdbb0 2024-02-23T00:45:14.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.330+0000 7f838dffb700 1 -- 172.21.15.47:0/1085564695 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f837c014ae0 con 0x7f83900fdbb0 2024-02-23T00:45:14.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.331+0000 7f838dffb700 1 --2- 172.21.15.47:0/1085564695 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8380062cc0 0x7f8380065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:14.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.331+0000 7f8394e43700 1 --2- 172.21.15.47:0/1085564695 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8380062cc0 0x7f8380065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:14.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.331+0000 7f838dffb700 1 -- 172.21.15.47:0/1085564695 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(242..242 src has 48..242) v4 ==== 11432+0+0 (secure 0 0 0) 0x7f837c07e270 con 0x7f83900fdbb0 2024-02-23T00:45:14.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.331+0000 7f838dffb700 10 client.40853.objecter ms_dispatch 0x7f8390001550 osd_map(242..242 src has 48..242) v4 2024-02-23T00:45:14.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.331+0000 7f838dffb700 3 client.40853.objecter handle_osd_map got epochs [242,242] > 0 2024-02-23T00:45:14.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.331+0000 7f838dffb700 3 client.40853.objecter handle_osd_map decoding full epoch 242 2024-02-23T00:45:14.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.331+0000 7f8394e43700 1 --2- 172.21.15.47:0/1085564695 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8380062cc0 0x7f8380065180 secure :-1 s=READY pgs=2980 cs=0 l=1 rev1=1 rx=0x7f8384009da0 tx=0x7f8384009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:14.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.331+0000 7f838dffb700 20 client.40853.objecter dump_active .. 0 homeless 2024-02-23T00:45:14.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.331+0000 7f838dffb700 10 client.40853.objecter ms_handle_connect 0x7f8380062cc0 2024-02-23T00:45:14.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.334+0000 7f838dffb700 1 -- 172.21.15.47:0/1085564695 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f837c04d040 con 0x7f83900fdbb0 2024-02-23T00:45:14.357 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:14.378 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:45:14.381 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:14.381 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:45:14.386 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:45:14.482 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:45:14.510 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:14.530 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:45:14.532 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:14.533 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:45:14.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:14.531+0000 7f83970a7700 1 -- 172.21.15.47:0/1085564695 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7f8390106e90 con 0x7f83900fdbb0 2024-02-23T00:45:14.634 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:45:14.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:13.236585+0000 mgr.y (mgr.14517) 263 : cluster [DBG] pgmap v337: 346 pgs: 32 creating+peering, 7 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 294 active+clean; 9.2 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-02-23T00:45:14.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:13.313721+0000 osd.4 (osd.4) 109 : cluster [DBG] 4.13 scrub starts 2024-02-23T00:45:14.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:13.315429+0000 osd.4 (osd.4) 110 : cluster [DBG] 4.13 scrub ok 2024-02-23T00:45:14.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:13.612670+0000 osd.6 (osd.6) 99 : cluster [DBG] 2.11 scrub starts 2024-02-23T00:45:14.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:13.613649+0000 osd.6 (osd.6) 100 : cluster [DBG] 2.11 scrub ok 2024-02-23T00:45:14.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:13.812426+0000 mon.b (mon.1) 301 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:14.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:13.873414+0000 osd.3 (osd.3) 129 : cluster [DBG] 18.16 scrub starts 2024-02-23T00:45:14.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:13.874369+0000 osd.3 (osd.3) 130 : cluster [DBG] 18.16 scrub ok 2024-02-23T00:45:14.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:14.146792+0000 mon.a (mon.0) 1673 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:45:14.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:14.532570+0000 mon.c (mon.2) 162 : audit [INF] from='client.? 172.21.15.47:0/1085564695' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:45:14.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:14.596154+0000 mon.a (mon.0) 1674 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-23"}]': finished 2024-02-23T00:45:14.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:14.596275+0000 mon.a (mon.0) 1675 : cluster [DBG] osdmap e242: 8 total, 8 up, 8 in 2024-02-23T00:45:14.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:14.597988+0000 mon.c (mon.2) 163 : audit [INF] from='client.? 172.21.15.47:0/1304175216' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:14.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:14.598029+0000 mon.a (mon.0) 1676 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:45:14.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:14.598569+0000 mon.a (mon.0) 1677 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:14.666 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:14.687 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:45:14.689 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:14.690 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:45:14.790 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:45:14.821 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:15.015 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:13.236585+0000 mgr.y (mgr.14517) 263 : cluster [DBG] pgmap v337: 346 pgs: 32 creating+peering, 7 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 294 active+clean; 9.2 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-02-23T00:45:15.015 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:13.313721+0000 osd.4 (osd.4) 109 : cluster [DBG] 4.13 scrub starts 2024-02-23T00:45:15.015 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:13.315429+0000 osd.4 (osd.4) 110 : cluster [DBG] 4.13 scrub ok 2024-02-23T00:45:15.015 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:13.612670+0000 osd.6 (osd.6) 99 : cluster [DBG] 2.11 scrub starts 2024-02-23T00:45:15.015 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:13.613649+0000 osd.6 (osd.6) 100 : cluster [DBG] 2.11 scrub ok 2024-02-23T00:45:15.015 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:13.812426+0000 mon.b (mon.1) 301 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:15.015 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:13.873414+0000 osd.3 (osd.3) 129 : cluster [DBG] 18.16 scrub starts 2024-02-23T00:45:15.015 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:13.874369+0000 osd.3 (osd.3) 130 : cluster [DBG] 18.16 scrub ok 2024-02-23T00:45:15.015 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:14.146792+0000 mon.a (mon.0) 1673 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:45:15.015 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:14.532570+0000 mon.c (mon.2) 162 : audit [INF] from='client.? 172.21.15.47:0/1085564695' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:45:15.016 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:14.596154+0000 mon.a (mon.0) 1674 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-23"}]': finished 2024-02-23T00:45:15.016 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:14.596275+0000 mon.a (mon.0) 1675 : cluster [DBG] osdmap e242: 8 total, 8 up, 8 in 2024-02-23T00:45:15.016 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:14.597988+0000 mon.c (mon.2) 163 : audit [INF] from='client.? 172.21.15.47:0/1304175216' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:15.016 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:14.598029+0000 mon.a (mon.0) 1676 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:45:15.016 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:14.598569+0000 mon.a (mon.0) 1677 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:15.016 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:13.236585+0000 mgr.y (mgr.14517) 263 : cluster [DBG] pgmap v337: 346 pgs: 32 creating+peering, 7 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 294 active+clean; 9.2 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-02-23T00:45:15.016 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:13.313721+0000 osd.4 (osd.4) 109 : cluster [DBG] 4.13 scrub starts 2024-02-23T00:45:15.016 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:13.315429+0000 osd.4 (osd.4) 110 : cluster [DBG] 4.13 scrub ok 2024-02-23T00:45:15.016 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:13.612670+0000 osd.6 (osd.6) 99 : cluster [DBG] 2.11 scrub starts 2024-02-23T00:45:15.016 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:13.613649+0000 osd.6 (osd.6) 100 : cluster [DBG] 2.11 scrub ok 2024-02-23T00:45:15.016 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:13.812426+0000 mon.b (mon.1) 301 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:15.016 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:13.873414+0000 osd.3 (osd.3) 129 : cluster [DBG] 18.16 scrub starts 2024-02-23T00:45:15.017 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:13.874369+0000 osd.3 (osd.3) 130 : cluster [DBG] 18.16 scrub ok 2024-02-23T00:45:15.017 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:14.146792+0000 mon.a (mon.0) 1673 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:45:15.017 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:14.532570+0000 mon.c (mon.2) 162 : audit [INF] from='client.? 172.21.15.47:0/1085564695' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:45:15.017 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:14.596154+0000 mon.a (mon.0) 1674 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-23"}]': finished 2024-02-23T00:45:15.017 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:14.596275+0000 mon.a (mon.0) 1675 : cluster [DBG] osdmap e242: 8 total, 8 up, 8 in 2024-02-23T00:45:15.017 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:14.597988+0000 mon.c (mon.2) 163 : audit [INF] from='client.? 172.21.15.47:0/1304175216' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:15.017 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:14.598029+0000 mon.a (mon.0) 1676 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:45:15.017 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:14.598569+0000 mon.a (mon.0) 1677 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:15.020 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:45:15.023 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:15.023 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:45:15.124 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:45:15.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:15.136+0000 7f838dffb700 1 -- 172.21.15.47:0/1085564695 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v243) v1 ==== 101+0+0 (secure 0 0 0) 0x7f837c050660 con 0x7f83900fdbb0 2024-02-23T00:45:15.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:15.139+0000 7f83970a7700 10 librados: watch_flush enter 2024-02-23T00:45:15.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:15.139+0000 7f83970a7700 10 librados: watch_flush exit 2024-02-23T00:45:15.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:15.139+0000 7f83970a7700 20 client.40853.objecter shutdown clearing up homeless session... 2024-02-23T00:45:15.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:15.139+0000 7f83970a7700 10 client.40853.objecter successfully canceled tick 2024-02-23T00:45:15.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:15.140+0000 7f83970a7700 1 -- 172.21.15.47:0/1085564695 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8380062cc0 msgr2=0x7f8380065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:15.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:15.140+0000 7f83970a7700 1 --2- 172.21.15.47:0/1085564695 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8380062cc0 0x7f8380065180 secure :-1 s=READY pgs=2980 cs=0 l=1 rev1=1 rx=0x7f8384009da0 tx=0x7f8384009cb0).stop 2024-02-23T00:45:15.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:15.140+0000 7f83970a7700 1 -- 172.21.15.47:0/1085564695 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f83900fdbb0 msgr2=0x7f839010e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:15.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:15.140+0000 7f83970a7700 1 --2- 172.21.15.47:0/1085564695 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f83900fdbb0 0x7f839010e370 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 rx=0x7f837c0041d0 tx=0x7f837c009f60).stop 2024-02-23T00:45:15.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:15.140+0000 7f83970a7700 1 -- 172.21.15.47:0/1085564695 shutdown_connections 2024-02-23T00:45:15.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:15.140+0000 7f83970a7700 1 --2- 172.21.15.47:0/1085564695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83900fe510 0x7f8390108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:15.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:15.140+0000 7f83970a7700 1 --2- 172.21.15.47:0/1085564695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83900f8420 0x7f839010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:15.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:15.140+0000 7f83970a7700 1 --2- 172.21.15.47:0/1085564695 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8380062cc0 0x7f8380065180 secure :-1 s=CLOSED pgs=2980 cs=0 l=1 rev1=1 rx=0x7f8384009da0 tx=0x7f8384009cb0).stop 2024-02-23T00:45:15.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:15.140+0000 7f83970a7700 1 --2- 172.21.15.47:0/1085564695 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f83900fdbb0 0x7f839010e370 secure :-1 s=CLOSED pgs=2564 cs=0 l=1 rev1=1 rx=0x7f837c0041d0 tx=0x7f837c009f60).stop 2024-02-23T00:45:15.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:15.140+0000 7f83970a7700 1 -- 172.21.15.47:0/1085564695 >> 172.21.15.47:0/1085564695 conn(0x7f83900f3800 msgr2=0x7f83900fb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:45:15.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:15.142+0000 7f83970a7700 1 -- 172.21.15.47:0/1085564695 shutdown_connections 2024-02-23T00:45:15.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:15.142+0000 7f83970a7700 1 -- 172.21.15.47:0/1085564695 wait complete. 2024-02-23T00:45:15.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:15.142+0000 7f83970a7700 1 librados: shutdown 2024-02-23T00:45:15.152 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is unset 2024-02-23T00:45:15.174 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:16.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.075+0000 7fbd5fa6e700 1 -- 172.21.15.47:0/3841520682 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbd580f9620 msgr2=0x7fbd58101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:16.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.075+0000 7fbd5fa6e700 1 --2- 172.21.15.47:0/3841520682 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbd580f9620 0x7fbd58101dc0 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 rx=0x7fbd48004870 tx=0x7fbd48009cb0).stop 2024-02-23T00:45:16.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.075+0000 7fbd5fa6e700 1 -- 172.21.15.47:0/3841520682 shutdown_connections 2024-02-23T00:45:16.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.075+0000 7fbd5fa6e700 1 --2- 172.21.15.47:0/3841520682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd581022b0 0x7fbd581046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:16.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.075+0000 7fbd5fa6e700 1 --2- 172.21.15.47:0/3841520682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd580f8d50 0x7fbd580f9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:45:16.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.075+0000 7fbd5fa6e700 1 --2- 172.21.15.47:0/3841520682 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbd580f9620 0x7fbd58101dc0 unknown :-1 s=CLOSED pgs=2566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:16.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.075+0000 7fbd5fa6e700 1 -- 172.21.15.47:0/3841520682 >> 172.21.15.47:0/3841520682 conn(0x7fbd580f2820 msgr2=0x7fbd580f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:45:16.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.076+0000 7fbd5fa6e700 1 -- 172.21.15.47:0/3841520682 shutdown_connections 2024-02-23T00:45:16.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.076+0000 7fbd5fa6e700 1 -- 172.21.15.47:0/3841520682 wait complete. 2024-02-23T00:45:16.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.076+0000 7fbd5fa6e700 1 librados: starting msgr at 2024-02-23T00:45:16.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.076+0000 7fbd5fa6e700 1 librados: starting objecter 2024-02-23T00:45:16.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.076+0000 7fbd5fa6e700 1 Processor -- start 2024-02-23T00:45:16.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.076+0000 7fbd5fa6e700 1 -- start start 2024-02-23T00:45:16.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.076+0000 7fbd5fa6e700 1 librados: setting wanted keys 2024-02-23T00:45:16.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.076+0000 7fbd5fa6e700 1 librados: calling monclient init 2024-02-23T00:45:16.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.077+0000 7fbd5fa6e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd580f8d50 0x7fbd5810e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:16.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.077+0000 7fbd5fa6e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd580f9620 0x7fbd5810e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:16.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.077+0000 7fbd5fa6e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbd581022b0 0x7fbd58108290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:16.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.077+0000 7fbd5fa6e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd58108780 con 0x7fbd580f8d50 2024-02-23T00:45:16.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.077+0000 7fbd5fa6e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd581088f0 con 0x7fbd580f9620 2024-02-23T00:45:16.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.077+0000 7fbd5fa6e700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd58108ab0 con 0x7fbd581022b0 2024-02-23T00:45:16.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.077+0000 7fbd5e00b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbd581022b0 0x7fbd58108290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:16.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.077+0000 7fbd5d80a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd580f8d50 0x7fbd5810e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:16.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.077+0000 7fbd5e00b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbd581022b0 0x7fbd58108290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52410/0 (socket says 172.21.15.47:52410) 2024-02-23T00:45:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.077+0000 7fbd5e00b700 1 -- 172.21.15.47:0/4184832147 learned_addr learned my addr 172.21.15.47:0/4184832147 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:45:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.077+0000 7fbd5d009700 1 --2- 172.21.15.47:0/4184832147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd580f9620 0x7fbd5810e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.077+0000 7fbd5e00b700 1 -- 172.21.15.47:0/4184832147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd580f9620 msgr2=0x7fbd5810e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.077+0000 7fbd5e00b700 1 --2- 172.21.15.47:0/4184832147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd580f9620 0x7fbd5810e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.077+0000 7fbd5e00b700 1 -- 172.21.15.47:0/4184832147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd580f8d50 msgr2=0x7fbd5810e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.077+0000 7fbd5e00b700 1 --2- 172.21.15.47:0/4184832147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd580f8d50 0x7fbd5810e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.077+0000 7fbd5e00b700 1 -- 172.21.15.47:0/4184832147 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd48005040 con 0x7fbd581022b0 2024-02-23T00:45:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.077+0000 7fbd5e00b700 1 --2- 172.21.15.47:0/4184832147 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbd581022b0 0x7fbd58108290 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 rx=0x7fbd5000fb50 tx=0x7fbd50004c50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.077+0000 7fbd4ef78700 10 client.?.objecter ms_handle_connect 0x7fbd581022b0 2024-02-23T00:45:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.077+0000 7fbd4ef78700 10 client.?.objecter resend_mon_ops 2024-02-23T00:45:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.077+0000 7fbd4ef78700 1 -- 172.21.15.47:0/4184832147 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd50007840 con 0x7fbd581022b0 2024-02-23T00:45:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.077+0000 7fbd4ef78700 1 -- 172.21.15.47:0/4184832147 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fbd5000dc60 con 0x7fbd581022b0 2024-02-23T00:45:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.077+0000 7fbd5fa6e700 1 -- 172.21.15.47:0/4184832147 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd58108d90 con 0x7fbd581022b0 2024-02-23T00:45:16.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.077+0000 7fbd4ef78700 1 -- 172.21.15.47:0/4184832147 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd50015840 con 0x7fbd581022b0 2024-02-23T00:45:16.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.077+0000 7fbd5fa6e700 10 client.40865.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:45:16.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.078+0000 7fbd5fa6e700 1 -- 172.21.15.47:0/4184832147 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd58114f20 con 0x7fbd581022b0 2024-02-23T00:45:16.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.078+0000 7fbd5fa6e700 1 librados: init done 2024-02-23T00:45:16.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.079+0000 7fbd5fa6e700 1 -- 172.21.15.47:0/4184832147 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd5805fa60 con 0x7fbd581022b0 2024-02-23T00:45:16.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.079+0000 7fbd4ef78700 1 -- 172.21.15.47:0/4184832147 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fbd50015c70 con 0x7fbd581022b0 2024-02-23T00:45:16.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.079+0000 7fbd4ef78700 1 --2- 172.21.15.47:0/4184832147 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbd44062cc0 0x7fbd44065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:16.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.079+0000 7fbd4ef78700 1 -- 172.21.15.47:0/4184832147 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(243..243 src has 48..243) v4 ==== 11471+0+0 (secure 0 0 0) 0x7fbd5007e470 con 0x7fbd581022b0 2024-02-23T00:45:16.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.079+0000 7fbd4ef78700 10 client.40865.objecter ms_dispatch 0x7fbd58001550 osd_map(243..243 src has 48..243) v4 2024-02-23T00:45:16.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.079+0000 7fbd4ef78700 3 client.40865.objecter handle_osd_map got epochs [243,243] > 0 2024-02-23T00:45:16.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.079+0000 7fbd4ef78700 3 client.40865.objecter handle_osd_map decoding full epoch 243 2024-02-23T00:45:16.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.079+0000 7fbd5d80a700 1 --2- 172.21.15.47:0/4184832147 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbd44062cc0 0x7fbd44065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:16.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.079+0000 7fbd4ef78700 20 client.40865.objecter dump_active .. 0 homeless 2024-02-23T00:45:16.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.080+0000 7fbd4ef78700 10 client.40865.objecter ms_handle_connect 0x7fbd44062cc0 2024-02-23T00:45:16.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.080+0000 7fbd5d80a700 1 --2- 172.21.15.47:0/4184832147 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbd44062cc0 0x7fbd44065180 secure :-1 s=READY pgs=2982 cs=0 l=1 rev1=1 rx=0x7fbd54006fd0 tx=0x7fbd54009060).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:16.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.084+0000 7fbd4ef78700 1 -- 172.21.15.47:0/4184832147 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fbd5004d1c0 con 0x7fbd581022b0 2024-02-23T00:45:16.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.267+0000 7fbd5fa6e700 1 -- 172.21.15.47:0/4184832147 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbd58107050 con 0x7fbd44062cc0 2024-02-23T00:45:16.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.297+0000 7fbd4ef78700 1 -- 172.21.15.47:0/4184832147 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+1011163 (secure 0 0 0) 0x7fbd58107050 con 0x7fbd44062cc0 2024-02-23T00:45:16.312 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:45:16.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.304+0000 7fbd5fa6e700 10 librados: watch_flush enter 2024-02-23T00:45:16.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.304+0000 7fbd5fa6e700 10 librados: watch_flush exit 2024-02-23T00:45:16.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.304+0000 7fbd5fa6e700 20 client.40865.objecter shutdown clearing up homeless session... 2024-02-23T00:45:16.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.304+0000 7fbd5fa6e700 10 client.40865.objecter successfully canceled tick 2024-02-23T00:45:16.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.304+0000 7fbd5fa6e700 1 -- 172.21.15.47:0/4184832147 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbd44062cc0 msgr2=0x7fbd44065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:16.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.304+0000 7fbd5fa6e700 1 --2- 172.21.15.47:0/4184832147 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbd44062cc0 0x7fbd44065180 secure :-1 s=READY pgs=2982 cs=0 l=1 rev1=1 rx=0x7fbd54006fd0 tx=0x7fbd54009060).stop 2024-02-23T00:45:16.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.304+0000 7fbd5fa6e700 1 -- 172.21.15.47:0/4184832147 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbd581022b0 msgr2=0x7fbd58108290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:16.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.304+0000 7fbd5fa6e700 1 --2- 172.21.15.47:0/4184832147 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbd581022b0 0x7fbd58108290 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 rx=0x7fbd5000fb50 tx=0x7fbd50004c50).stop 2024-02-23T00:45:16.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.304+0000 7fbd5fa6e700 1 -- 172.21.15.47:0/4184832147 shutdown_connections 2024-02-23T00:45:16.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.304+0000 7fbd5fa6e700 1 --2- 172.21.15.47:0/4184832147 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbd44062cc0 0x7fbd44065180 unknown :-1 s=CLOSED pgs=2982 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:16.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.304+0000 7fbd5fa6e700 1 --2- 172.21.15.47:0/4184832147 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbd581022b0 0x7fbd58108290 unknown :-1 s=CLOSED pgs=2567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:16.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.304+0000 7fbd5fa6e700 1 --2- 172.21.15.47:0/4184832147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd580f8d50 0x7fbd5810e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:16.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.304+0000 7fbd5fa6e700 1 --2- 172.21.15.47:0/4184832147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd580f9620 0x7fbd5810e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:16.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.304+0000 7fbd5fa6e700 1 -- 172.21.15.47:0/4184832147 >> 172.21.15.47:0/4184832147 conn(0x7fbd580f2820 msgr2=0x7fbd580f34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:45:16.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.304+0000 7fbd5fa6e700 1 -- 172.21.15.47:0/4184832147 shutdown_connections 2024-02-23T00:45:16.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.304+0000 7fbd5fa6e700 1 -- 172.21.15.47:0/4184832147 wait complete. 2024-02-23T00:45:16.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:16.304+0000 7fbd5fa6e700 1 librados: shutdown 2024-02-23T00:45:16.326 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-02-23T00:45:16.329 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:14.206130+0000 osd.5 (osd.5) 111 : cluster [DBG] 18.f scrub starts 2024-02-23T00:45:16.329 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:14.209300+0000 osd.5 (osd.5) 112 : cluster [DBG] 18.f scrub ok 2024-02-23T00:45:16.330 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:14.315975+0000 osd.4 (osd.4) 111 : cluster [DBG] 15.17 scrub starts 2024-02-23T00:45:16.330 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:14.317568+0000 osd.4 (osd.4) 112 : cluster [DBG] 15.17 scrub ok 2024-02-23T00:45:16.330 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:14.706604+0000 osd.7 (osd.7) 125 : cluster [DBG] 15.1b scrub starts 2024-02-23T00:45:16.330 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:14.710070+0000 osd.7 (osd.7) 126 : cluster [DBG] 15.1b scrub ok 2024-02-23T00:45:16.330 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:14.813494+0000 mon.b (mon.1) 302 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:16.330 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:15.119661+0000 mon.a (mon.0) 1678 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:45:16.330 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:15.124260+0000 mon.a (mon.0) 1679 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi047-82378-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi047-82378-33"}]': finished 2024-02-23T00:45:16.330 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:15.124456+0000 mon.a (mon.0) 1680 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:45:16.330 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:15.124563+0000 mon.a (mon.0) 1681 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:16.330 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:15.124652+0000 mon.a (mon.0) 1682 : cluster [DBG] osdmap e243: 8 total, 8 up, 8 in 2024-02-23T00:45:16.330 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:14.206130+0000 osd.5 (osd.5) 111 : cluster [DBG] 18.f scrub starts 2024-02-23T00:45:16.330 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:14.209300+0000 osd.5 (osd.5) 112 : cluster [DBG] 18.f scrub ok 2024-02-23T00:45:16.331 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:14.315975+0000 osd.4 (osd.4) 111 : cluster [DBG] 15.17 scrub starts 2024-02-23T00:45:16.331 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:14.317568+0000 osd.4 (osd.4) 112 : cluster [DBG] 15.17 scrub ok 2024-02-23T00:45:16.331 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:14.706604+0000 osd.7 (osd.7) 125 : cluster [DBG] 15.1b scrub starts 2024-02-23T00:45:16.331 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:14.710070+0000 osd.7 (osd.7) 126 : cluster [DBG] 15.1b scrub ok 2024-02-23T00:45:16.331 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:14.813494+0000 mon.b (mon.1) 302 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:16.331 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:15.119661+0000 mon.a (mon.0) 1678 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:45:16.331 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:15.124260+0000 mon.a (mon.0) 1679 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi047-82378-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi047-82378-33"}]': finished 2024-02-23T00:45:16.331 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:15.124456+0000 mon.a (mon.0) 1680 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:45:16.331 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:15.124563+0000 mon.a (mon.0) 1681 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:16.331 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:15.124652+0000 mon.a (mon.0) 1682 : cluster [DBG] osdmap e243: 8 total, 8 up, 8 in 2024-02-23T00:45:16.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:14.206130+0000 osd.5 (osd.5) 111 : cluster [DBG] 18.f scrub starts 2024-02-23T00:45:16.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:14.209300+0000 osd.5 (osd.5) 112 : cluster [DBG] 18.f scrub ok 2024-02-23T00:45:16.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:14.315975+0000 osd.4 (osd.4) 111 : cluster [DBG] 15.17 scrub starts 2024-02-23T00:45:16.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:14.317568+0000 osd.4 (osd.4) 112 : cluster [DBG] 15.17 scrub ok 2024-02-23T00:45:16.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:14.706604+0000 osd.7 (osd.7) 125 : cluster [DBG] 15.1b scrub starts 2024-02-23T00:45:16.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:14.710070+0000 osd.7 (osd.7) 126 : cluster [DBG] 15.1b scrub ok 2024-02-23T00:45:16.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:14.813494+0000 mon.b (mon.1) 302 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:16.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:15.119661+0000 mon.a (mon.0) 1678 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:45:16.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:15.124260+0000 mon.a (mon.0) 1679 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi047-82378-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi047-82378-33"}]': finished 2024-02-23T00:45:16.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:15.124456+0000 mon.a (mon.0) 1680 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:45:16.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:15.124563+0000 mon.a (mon.0) 1681 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:16.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:15.124652+0000 mon.a (mon.0) 1682 : cluster [DBG] osdmap e243: 8 total, 8 up, 8 in 2024-02-23T00:45:16.631 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:45:16.634 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:16.634 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:45:16.735 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:45:16.767 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:16.788 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:45:16.791 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:16.791 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:45:16.892 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:45:16.924 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:17.175 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:45:17.179 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:15.236969+0000 mgr.y (mgr.14517) 264 : cluster [DBG] pgmap v340: 354 pgs: 40 unknown, 32 creating+peering, 7 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 267 active+clean; 9.2 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-02-23T00:45:17.179 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:15.351257+0000 osd.4 (osd.4) 113 : cluster [DBG] 15.19 deep-scrub starts 2024-02-23T00:45:17.179 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:15.352022+0000 osd.4 (osd.4) 114 : cluster [DBG] 15.19 deep-scrub ok 2024-02-23T00:45:17.179 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:15.814623+0000 mon.b (mon.1) 303 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:17.179 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:15.932356+0000 osd.3 (osd.3) 131 : cluster [DBG] 18.9 scrub starts 2024-02-23T00:45:17.179 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:15.933434+0000 osd.3 (osd.3) 132 : cluster [DBG] 18.9 scrub ok 2024-02-23T00:45:17.179 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:16.129279+0000 mon.a (mon.0) 1683 : cluster [DBG] osdmap e244: 8 total, 8 up, 8 in 2024-02-23T00:45:17.179 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:16.141949+0000 mon.a (mon.0) 1684 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:17.180 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:16.144701+0000 mon.a (mon.0) 1685 : audit [INF] from='client.? 172.21.15.47:0/3951871746' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:17.180 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:16.268875+0000 mgr.y (mgr.14517) 265 : audit [DBG] from='client.40865 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:45:17.180 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:16.387756+0000 osd.2 (osd.2) 109 : cluster [DBG] 15.2 scrub starts 2024-02-23T00:45:17.180 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:16.388983+0000 osd.2 (osd.2) 110 : cluster [DBG] 15.2 scrub ok 2024-02-23T00:45:17.180 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:16.742888+0000 osd.7 (osd.7) 127 : cluster [DBG] 5.4 scrub starts 2024-02-23T00:45:17.180 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:16.815759+0000 mon.b (mon.1) 304 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:17.181 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:17.181 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:45:17.182 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:15.236969+0000 mgr.y (mgr.14517) 264 : cluster [DBG] pgmap v340: 354 pgs: 40 unknown, 32 creating+peering, 7 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 267 active+clean; 9.2 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-02-23T00:45:17.182 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:15.351257+0000 osd.4 (osd.4) 113 : cluster [DBG] 15.19 deep-scrub starts 2024-02-23T00:45:17.182 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:15.352022+0000 osd.4 (osd.4) 114 : cluster [DBG] 15.19 deep-scrub ok 2024-02-23T00:45:17.182 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:15.814623+0000 mon.b (mon.1) 303 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:17.182 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:15.932356+0000 osd.3 (osd.3) 131 : cluster [DBG] 18.9 scrub starts 2024-02-23T00:45:17.182 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:15.933434+0000 osd.3 (osd.3) 132 : cluster [DBG] 18.9 scrub ok 2024-02-23T00:45:17.182 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:16.129279+0000 mon.a (mon.0) 1683 : cluster [DBG] osdmap e244: 8 total, 8 up, 8 in 2024-02-23T00:45:17.182 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:16.141949+0000 mon.a (mon.0) 1684 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:17.182 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:16.144701+0000 mon.a (mon.0) 1685 : audit [INF] from='client.? 172.21.15.47:0/3951871746' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:17.182 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:16.268875+0000 mgr.y (mgr.14517) 265 : audit [DBG] from='client.40865 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:45:17.182 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:16.387756+0000 osd.2 (osd.2) 109 : cluster [DBG] 15.2 scrub starts 2024-02-23T00:45:17.182 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:16.388983+0000 osd.2 (osd.2) 110 : cluster [DBG] 15.2 scrub ok 2024-02-23T00:45:17.182 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:16.742888+0000 osd.7 (osd.7) 127 : cluster [DBG] 5.4 scrub starts 2024-02-23T00:45:17.183 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:16.815759+0000 mon.b (mon.1) 304 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:17.282 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:45:17.311 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:17.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:15.236969+0000 mgr.y (mgr.14517) 264 : cluster [DBG] pgmap v340: 354 pgs: 40 unknown, 32 creating+peering, 7 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 267 active+clean; 9.2 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-02-23T00:45:17.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:15.351257+0000 osd.4 (osd.4) 113 : cluster [DBG] 15.19 deep-scrub starts 2024-02-23T00:45:17.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:15.352022+0000 osd.4 (osd.4) 114 : cluster [DBG] 15.19 deep-scrub ok 2024-02-23T00:45:17.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:15.814623+0000 mon.b (mon.1) 303 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:17.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:15.932356+0000 osd.3 (osd.3) 131 : cluster [DBG] 18.9 scrub starts 2024-02-23T00:45:17.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:15.933434+0000 osd.3 (osd.3) 132 : cluster [DBG] 18.9 scrub ok 2024-02-23T00:45:17.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:16.129279+0000 mon.a (mon.0) 1683 : cluster [DBG] osdmap e244: 8 total, 8 up, 8 in 2024-02-23T00:45:17.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:16.141949+0000 mon.a (mon.0) 1684 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:17.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:16.144701+0000 mon.a (mon.0) 1685 : audit [INF] from='client.? 172.21.15.47:0/3951871746' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:17.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:16.268875+0000 mgr.y (mgr.14517) 265 : audit [DBG] from='client.40865 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:45:17.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:16.387756+0000 osd.2 (osd.2) 109 : cluster [DBG] 15.2 scrub starts 2024-02-23T00:45:17.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:16.388983+0000 osd.2 (osd.2) 110 : cluster [DBG] 15.2 scrub ok 2024-02-23T00:45:17.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:16.742888+0000 osd.7 (osd.7) 127 : cluster [DBG] 5.4 scrub starts 2024-02-23T00:45:17.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:16.815759+0000 mon.b (mon.1) 304 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:17.717 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-02-23T00:45:16.126733+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":9373,"num_objects":378,"num_object_clones":22,"num_object_copies":1134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":378,"num_whiteouts":1,"num_read":1549,"num_read_kb":1178,"num_write":1256,"num_write_kb":630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":947,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2541,"ondisk_log_size":2541,"up":942,"acting":942,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":849,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2470576,"kb_used_data":31276,"kb_used_omap":0,"kb_used_meta":2439168,"kb_avail":747490640,"statfs":{"total":767960285184,"available":765430415360,"internally_reserved":0,"allocated":32026624,"data_stored":18515188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2497708032},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-53,"num_objects":-12,"num_object_clones":-17,"num_object_copies":-36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-12,"num_whiteouts":1,"num_read":8,"num_read_kb":5,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.982453"},"pg_stats":[{"pgid":"161.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"160.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"161.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"160.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"161.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"160.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"161.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"160.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"161.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"160.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"161.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"160.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"161.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"160.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"161.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"160.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"161.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"160.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"161.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"160.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"161.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"160.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"161.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"160.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"161.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"160.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"161.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"160.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"161.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"160.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"161.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"160.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"161.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"160.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"161.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"160.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"161.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"160.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"161.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"160.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"161.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"160.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"161.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"160.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"161.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"160.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"161.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"160.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"161.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"160.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"161.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"160.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"161.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"160.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"161.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"160.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"161.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"160.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"161.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"160.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"161.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"160.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"161.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"160.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:16.121494+0000","last_change":"2024-02-23T00:45:16.121494+0000","last_active":"2024-02-23T00:45:16.121494+0000","last_peered":"2024-02-23T00:45:16.121494+0000","last_clean":"2024-02-23T00:45:16.121494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:16.121494+0000","last_undegraded":"2024-02-23T00:45:16.121494+0000","last_fullsized":"2024-02-23T00:45:16.121494+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:16.121494+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.121494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"158.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:14.145983+0000","last_change":"2024-02-23T00:45:14.145983+0000","last_active":"2024-02-23T00:45:14.145983+0000","last_peered":"2024-02-23T00:45:14.145983+0000","last_clean":"2024-02-23T00:45:14.145983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:14.145983+0000","last_undegraded":"2024-02-23T00:45:14.145983+0000","last_fullsized":"2024-02-23T00:45:14.145983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.145983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"157.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:12.139981+0000","last_change":"2024-02-23T00:45:12.150133+0000","last_active":"2024-02-23T00:45:12.139981+0000","last_peered":"2024-02-23T00:45:12.139981+0000","last_clean":"2024-02-23T00:45:12.139981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:12.139981+0000","last_undegraded":"2024-02-23T00:45:12.139981+0000","last_fullsized":"2024-02-23T00:45:12.139981+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.139981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"156.c","version":"0'0","reported_seq":10,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.162636+0000","last_change":"2024-02-23T00:45:11.161920+0000","last_active":"2024-02-23T00:45:11.162636+0000","last_peered":"2024-02-23T00:45:11.162636+0000","last_clean":"2024-02-23T00:45:11.162636+0000","last_became_active":"2024-02-23T00:45:11.161618+0000","last_became_peered":"2024-02-23T00:45:11.161618+0000","last_unstale":"2024-02-23T00:45:11.162636+0000","last_undegraded":"2024-02-23T00:45:11.162636+0000","last_fullsized":"2024-02-23T00:45:11.162636+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.131641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"155.b","version":"0'0","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.149277+0000","last_change":"2024-02-23T00:45:09.159131+0000","last_active":"2024-02-23T00:45:11.149277+0000","last_peered":"2024-02-23T00:45:11.149277+0000","last_clean":"2024-02-23T00:45:11.149277+0000","last_became_active":"2024-02-23T00:45:09.158709+0000","last_became_peered":"2024-02-23T00:45:09.158709+0000","last_unstale":"2024-02-23T00:45:11.149277+0000","last_undegraded":"2024-02-23T00:45:11.149277+0000","last_fullsized":"2024-02-23T00:45:11.149277+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.124500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"158.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:14.145983+0000","last_change":"2024-02-23T00:45:14.145983+0000","last_active":"2024-02-23T00:45:14.145983+0000","last_peered":"2024-02-23T00:45:14.145983+0000","last_clean":"2024-02-23T00:45:14.145983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:14.145983+0000","last_undegraded":"2024-02-23T00:45:14.145983+0000","last_fullsized":"2024-02-23T00:45:14.145983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.145983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"157.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:12.139981+0000","last_change":"2024-02-23T00:45:12.155434+0000","last_active":"2024-02-23T00:45:12.139981+0000","last_peered":"2024-02-23T00:45:12.139981+0000","last_clean":"2024-02-23T00:45:12.139981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:12.139981+0000","last_undegraded":"2024-02-23T00:45:12.139981+0000","last_fullsized":"2024-02-23T00:45:12.139981+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.139981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"156.d","version":"0'0","reported_seq":10,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.169323+0000","last_change":"2024-02-23T00:45:11.168755+0000","last_active":"2024-02-23T00:45:11.169323+0000","last_peered":"2024-02-23T00:45:11.169323+0000","last_clean":"2024-02-23T00:45:11.169323+0000","last_became_active":"2024-02-23T00:45:11.168391+0000","last_became_peered":"2024-02-23T00:45:11.168391+0000","last_unstale":"2024-02-23T00:45:11.169323+0000","last_undegraded":"2024-02-23T00:45:11.169323+0000","last_fullsized":"2024-02-23T00:45:11.169323+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.131641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"155.a","version":"0'0","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.142568+0000","last_change":"2024-02-23T00:45:09.161752+0000","last_active":"2024-02-23T00:45:11.142568+0000","last_peered":"2024-02-23T00:45:11.142568+0000","last_clean":"2024-02-23T00:45:11.142568+0000","last_became_active":"2024-02-23T00:45:09.161355+0000","last_became_peered":"2024-02-23T00:45:09.161355+0000","last_unstale":"2024-02-23T00:45:11.142568+0000","last_undegraded":"2024-02-23T00:45:11.142568+0000","last_fullsized":"2024-02-23T00:45:11.142568+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.124500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"158.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:14.145983+0000","last_change":"2024-02-23T00:45:14.145983+0000","last_active":"2024-02-23T00:45:14.145983+0000","last_peered":"2024-02-23T00:45:14.145983+0000","last_clean":"2024-02-23T00:45:14.145983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:14.145983+0000","last_undegraded":"2024-02-23T00:45:14.145983+0000","last_fullsized":"2024-02-23T00:45:14.145983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.145983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"157.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:12.139981+0000","last_change":"2024-02-23T00:45:12.151807+0000","last_active":"2024-02-23T00:45:12.139981+0000","last_peered":"2024-02-23T00:45:12.139981+0000","last_clean":"2024-02-23T00:45:12.139981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:12.139981+0000","last_undegraded":"2024-02-23T00:45:12.139981+0000","last_fullsized":"2024-02-23T00:45:12.139981+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.139981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"156.e","version":"0'0","reported_seq":10,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.164277+0000","last_change":"2024-02-23T00:45:11.163304+0000","last_active":"2024-02-23T00:45:11.164277+0000","last_peered":"2024-02-23T00:45:11.164277+0000","last_clean":"2024-02-23T00:45:11.164277+0000","last_became_active":"2024-02-23T00:45:11.163000+0000","last_became_peered":"2024-02-23T00:45:11.163000+0000","last_unstale":"2024-02-23T00:45:11.164277+0000","last_undegraded":"2024-02-23T00:45:11.164277+0000","last_fullsized":"2024-02-23T00:45:11.164277+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.131641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"155.9","version":"0'0","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.160602+0000","last_change":"2024-02-23T00:45:09.146730+0000","last_active":"2024-02-23T00:45:11.160602+0000","last_peered":"2024-02-23T00:45:11.160602+0000","last_clean":"2024-02-23T00:45:11.160602+0000","last_became_active":"2024-02-23T00:45:09.146383+0000","last_became_peered":"2024-02-23T00:45:09.146383+0000","last_unstale":"2024-02-23T00:45:11.160602+0000","last_undegraded":"2024-02-23T00:45:11.160602+0000","last_fullsized":"2024-02-23T00:45:11.160602+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.124500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"158.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:14.145983+0000","last_change":"2024-02-23T00:45:14.145983+0000","last_active":"2024-02-23T00:45:14.145983+0000","last_peered":"2024-02-23T00:45:14.145983+0000","last_clean":"2024-02-23T00:45:14.145983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:14.145983+0000","last_undegraded":"2024-02-23T00:45:14.145983+0000","last_fullsized":"2024-02-23T00:45:14.145983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.145983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"157.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:12.139981+0000","last_change":"2024-02-23T00:45:12.148904+0000","last_active":"2024-02-23T00:45:12.139981+0000","last_peered":"2024-02-23T00:45:12.139981+0000","last_clean":"2024-02-23T00:45:12.139981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:12.139981+0000","last_undegraded":"2024-02-23T00:45:12.139981+0000","last_fullsized":"2024-02-23T00:45:12.139981+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.139981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"156.f","version":"0'0","reported_seq":10,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.167628+0000","last_change":"2024-02-23T00:45:11.164272+0000","last_active":"2024-02-23T00:45:11.167628+0000","last_peered":"2024-02-23T00:45:11.167628+0000","last_clean":"2024-02-23T00:45:11.167628+0000","last_became_active":"2024-02-23T00:45:11.163946+0000","last_became_peered":"2024-02-23T00:45:11.163946+0000","last_unstale":"2024-02-23T00:45:11.167628+0000","last_undegraded":"2024-02-23T00:45:11.167628+0000","last_fullsized":"2024-02-23T00:45:11.167628+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.131641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"155.8","version":"0'0","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.146257+0000","last_change":"2024-02-23T00:45:09.145619+0000","last_active":"2024-02-23T00:45:11.146257+0000","last_peered":"2024-02-23T00:45:11.146257+0000","last_clean":"2024-02-23T00:45:11.146257+0000","last_became_active":"2024-02-23T00:45:09.145320+0000","last_became_peered":"2024-02-23T00:45:09.145320+0000","last_unstale":"2024-02-23T00:45:11.146257+0000","last_undegraded":"2024-02-23T00:45:11.146257+0000","last_fullsized":"2024-02-23T00:45:11.146257+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.124500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"158.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:14.145983+0000","last_change":"2024-02-23T00:45:14.145983+0000","last_active":"2024-02-23T00:45:14.145983+0000","last_peered":"2024-02-23T00:45:14.145983+0000","last_clean":"2024-02-23T00:45:14.145983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:14.145983+0000","last_undegraded":"2024-02-23T00:45:14.145983+0000","last_fullsized":"2024-02-23T00:45:14.145983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.145983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"157.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:12.139981+0000","last_change":"2024-02-23T00:45:12.149566+0000","last_active":"2024-02-23T00:45:12.139981+0000","last_peered":"2024-02-23T00:45:12.139981+0000","last_clean":"2024-02-23T00:45:12.139981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:12.139981+0000","last_undegraded":"2024-02-23T00:45:12.139981+0000","last_fullsized":"2024-02-23T00:45:12.139981+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.139981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7],"acting":[6,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"156.8","version":"0'0","reported_seq":10,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.167632+0000","last_change":"2024-02-23T00:45:11.164269+0000","last_active":"2024-02-23T00:45:11.167632+0000","last_peered":"2024-02-23T00:45:11.167632+0000","last_clean":"2024-02-23T00:45:11.167632+0000","last_became_active":"2024-02-23T00:45:11.163664+0000","last_became_peered":"2024-02-23T00:45:11.163664+0000","last_unstale":"2024-02-23T00:45:11.167632+0000","last_undegraded":"2024-02-23T00:45:11.167632+0000","last_fullsized":"2024-02-23T00:45:11.167632+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.131641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"155.f","version":"0'0","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.161360+0000","last_change":"2024-02-23T00:45:09.152988+0000","last_active":"2024-02-23T00:45:11.161360+0000","last_peered":"2024-02-23T00:45:11.161360+0000","last_clean":"2024-02-23T00:45:11.161360+0000","last_became_active":"2024-02-23T00:45:09.152153+0000","last_became_peered":"2024-02-23T00:45:09.152153+0000","last_unstale":"2024-02-23T00:45:11.161360+0000","last_undegraded":"2024-02-23T00:45:11.161360+0000","last_fullsized":"2024-02-23T00:45:11.161360+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.124500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"158.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:14.145983+0000","last_change":"2024-02-23T00:45:14.145983+0000","last_active":"2024-02-23T00:45:14.145983+0000","last_peered":"2024-02-23T00:45:14.145983+0000","last_clean":"2024-02-23T00:45:14.145983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:14.145983+0000","last_undegraded":"2024-02-23T00:45:14.145983+0000","last_fullsized":"2024-02-23T00:45:14.145983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.145983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"157.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:12.139981+0000","last_change":"2024-02-23T00:45:12.149275+0000","last_active":"2024-02-23T00:45:12.139981+0000","last_peered":"2024-02-23T00:45:12.139981+0000","last_clean":"2024-02-23T00:45:12.139981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:12.139981+0000","last_undegraded":"2024-02-23T00:45:12.139981+0000","last_fullsized":"2024-02-23T00:45:12.139981+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.139981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"156.9","version":"0'0","reported_seq":10,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.151888+0000","last_change":"2024-02-23T00:45:11.151632+0000","last_active":"2024-02-23T00:45:11.151888+0000","last_peered":"2024-02-23T00:45:11.151888+0000","last_clean":"2024-02-23T00:45:11.151888+0000","last_became_active":"2024-02-23T00:45:11.151371+0000","last_became_peered":"2024-02-23T00:45:11.151371+0000","last_unstale":"2024-02-23T00:45:11.151888+0000","last_undegraded":"2024-02-23T00:45:11.151888+0000","last_fullsized":"2024-02-23T00:45:11.151888+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.131641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"155.e","version":"0'0","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.157850+0000","last_change":"2024-02-23T00:45:09.154908+0000","last_active":"2024-02-23T00:45:11.157850+0000","last_peered":"2024-02-23T00:45:11.157850+0000","last_clean":"2024-02-23T00:45:11.157850+0000","last_became_active":"2024-02-23T00:45:09.154501+0000","last_became_peered":"2024-02-23T00:45:09.154501+0000","last_unstale":"2024-02-23T00:45:11.157850+0000","last_undegraded":"2024-02-23T00:45:11.157850+0000","last_fullsized":"2024-02-23T00:45:11.157850+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.124500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"158.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:14.145983+0000","last_change":"2024-02-23T00:45:14.145983+0000","last_active":"2024-02-23T00:45:14.145983+0000","last_peered":"2024-02-23T00:45:14.145983+0000","last_clean":"2024-02-23T00:45:14.145983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:14.145983+0000","last_undegraded":"2024-02-23T00:45:14.145983+0000","last_fullsized":"2024-02-23T00:45:14.145983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.145983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"157.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:12.139981+0000","last_change":"2024-02-23T00:45:12.151818+0000","last_active":"2024-02-23T00:45:12.139981+0000","last_peered":"2024-02-23T00:45:12.139981+0000","last_clean":"2024-02-23T00:45:12.139981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:12.139981+0000","last_undegraded":"2024-02-23T00:45:12.139981+0000","last_fullsized":"2024-02-23T00:45:12.139981+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.139981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"156.a","version":"0'0","reported_seq":10,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.169986+0000","last_change":"2024-02-23T00:45:11.169282+0000","last_active":"2024-02-23T00:45:11.169986+0000","last_peered":"2024-02-23T00:45:11.169986+0000","last_clean":"2024-02-23T00:45:11.169986+0000","last_became_active":"2024-02-23T00:45:11.168955+0000","last_became_peered":"2024-02-23T00:45:11.168955+0000","last_unstale":"2024-02-23T00:45:11.169986+0000","last_undegraded":"2024-02-23T00:45:11.169986+0000","last_fullsized":"2024-02-23T00:45:11.169986+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.131641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2],"acting":[6,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"155.d","version":"0'0","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.159183+0000","last_change":"2024-02-23T00:45:09.161472+0000","last_active":"2024-02-23T00:45:11.159183+0000","last_peered":"2024-02-23T00:45:11.159183+0000","last_clean":"2024-02-23T00:45:11.159183+0000","last_became_active":"2024-02-23T00:45:09.160984+0000","last_became_peered":"2024-02-23T00:45:09.160984+0000","last_unstale":"2024-02-23T00:45:11.159183+0000","last_undegraded":"2024-02-23T00:45:11.159183+0000","last_fullsized":"2024-02-23T00:45:11.159183+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.124500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"158.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:14.145983+0000","last_change":"2024-02-23T00:45:14.145983+0000","last_active":"2024-02-23T00:45:14.145983+0000","last_peered":"2024-02-23T00:45:14.145983+0000","last_clean":"2024-02-23T00:45:14.145983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:14.145983+0000","last_undegraded":"2024-02-23T00:45:14.145983+0000","last_fullsized":"2024-02-23T00:45:14.145983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.145983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"157.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:12.139981+0000","last_change":"2024-02-23T00:45:12.145969+0000","last_active":"2024-02-23T00:45:12.139981+0000","last_peered":"2024-02-23T00:45:12.139981+0000","last_clean":"2024-02-23T00:45:12.139981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:12.139981+0000","last_undegraded":"2024-02-23T00:45:12.139981+0000","last_fullsized":"2024-02-23T00:45:12.139981+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.139981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"156.b","version":"0'0","reported_seq":10,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.170109+0000","last_change":"2024-02-23T00:45:11.169911+0000","last_active":"2024-02-23T00:45:11.170109+0000","last_peered":"2024-02-23T00:45:11.170109+0000","last_clean":"2024-02-23T00:45:11.170109+0000","last_became_active":"2024-02-23T00:45:11.169550+0000","last_became_peered":"2024-02-23T00:45:11.169550+0000","last_unstale":"2024-02-23T00:45:11.170109+0000","last_undegraded":"2024-02-23T00:45:11.170109+0000","last_fullsized":"2024-02-23T00:45:11.170109+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.131641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"155.c","version":"0'0","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.149510+0000","last_change":"2024-02-23T00:45:09.170554+0000","last_active":"2024-02-23T00:45:11.149510+0000","last_peered":"2024-02-23T00:45:11.149510+0000","last_clean":"2024-02-23T00:45:11.149510+0000","last_became_active":"2024-02-23T00:45:09.170257+0000","last_became_peered":"2024-02-23T00:45:09.170257+0000","last_unstale":"2024-02-23T00:45:11.149510+0000","last_undegraded":"2024-02-23T00:45:11.149510+0000","last_fullsized":"2024-02-23T00:45:11.149510+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.124500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"159.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:15.119257+0000","last_change":"2024-02-23T00:45:15.119257+0000","last_active":"2024-02-23T00:45:15.119257+0000","last_peered":"2024-02-23T00:45:15.119257+0000","last_clean":"2024-02-23T00:45:15.119257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:15.119257+0000","last_undegraded":"2024-02-23T00:45:15.119257+0000","last_fullsized":"2024-02-23T00:45:15.119257+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:15.119257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:15.119257+0000","last_clean_scrub_stamp":"2024-02-23T00:45:15.119257+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"158.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:14.145983+0000","last_change":"2024-02-23T00:45:14.145983+0000","last_active":"2024-02-23T00:45:14.145983+0000","last_peered":"2024-02-23T00:45:14.145983+0000","last_clean":"2024-02-23T00:45:14.145983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:14.145983+0000","last_undegraded":"2024-02-23T00:45:14.145983+0000","last_fullsized":"2024-02-23T00:45:14.145983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.145983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"157.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:12.139981+0000","last_change":"2024-02-23T00:45:12.146033+0000","last_active":"2024-02-23T00:45:12.139981+0000","last_peered":"2024-02-23T00:45:12.139981+0000","last_clean":"2024-02-23T00:45:12.139981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:12.139981+0000","last_undegraded":"2024-02-23T00:45:12.139981+0000","last_fullsized":"2024-02-23T00:45:12.139981+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.139981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"156.4","version":"0'0","reported_seq":10,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.172605+0000","last_change":"2024-02-23T00:45:11.170809+0000","last_active":"2024-02-23T00:45:11.172605+0000","last_peered":"2024-02-23T00:45:11.172605+0000","last_clean":"2024-02-23T00:45:11.172605+0000","last_became_active":"2024-02-23T00:45:11.170486+0000","last_became_peered":"2024-02-23T00:45:11.170486+0000","last_unstale":"2024-02-23T00:45:11.172605+0000","last_undegraded":"2024-02-23T00:45:11.172605+0000","last_fullsized":"2024-02-23T00:45:11.172605+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.131641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"155.3","version":"0'0","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.156285+0000","last_change":"2024-02-23T00:45:09.157783+0000","last_active":"2024-02-23T00:45:11.156285+0000","last_peered":"2024-02-23T00:45:11.156285+0000","last_clean":"2024-02-23T00:45:11.156285+0000","last_became_active":"2024-02-23T00:45:09.157484+0000","last_became_peered":"2024-02-23T00:45:09.157484+0000","last_unstale":"2024-02-23T00:45:11.156285+0000","last_undegraded":"2024-02-23T00:45:11.156285+0000","last_fullsized":"2024-02-23T00:45:11.156285+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.124500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"159.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:15.119257+0000","last_change":"2024-02-23T00:45:15.119257+0000","last_active":"2024-02-23T00:45:15.119257+0000","last_peered":"2024-02-23T00:45:15.119257+0000","last_clean":"2024-02-23T00:45:15.119257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:15.119257+0000","last_undegraded":"2024-02-23T00:45:15.119257+0000","last_fullsized":"2024-02-23T00:45:15.119257+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:15.119257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:15.119257+0000","last_clean_scrub_stamp":"2024-02-23T00:45:15.119257+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"158.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:14.145983+0000","last_change":"2024-02-23T00:45:14.145983+0000","last_active":"2024-02-23T00:45:14.145983+0000","last_peered":"2024-02-23T00:45:14.145983+0000","last_clean":"2024-02-23T00:45:14.145983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:14.145983+0000","last_undegraded":"2024-02-23T00:45:14.145983+0000","last_fullsized":"2024-02-23T00:45:14.145983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.145983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"157.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:12.139981+0000","last_change":"2024-02-23T00:45:12.155029+0000","last_active":"2024-02-23T00:45:12.139981+0000","last_peered":"2024-02-23T00:45:12.139981+0000","last_clean":"2024-02-23T00:45:12.139981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:12.139981+0000","last_undegraded":"2024-02-23T00:45:12.139981+0000","last_fullsized":"2024-02-23T00:45:12.139981+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.139981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"156.5","version":"0'0","reported_seq":10,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.167582+0000","last_change":"2024-02-23T00:45:11.167362+0000","last_active":"2024-02-23T00:45:11.167582+0000","last_peered":"2024-02-23T00:45:11.167582+0000","last_clean":"2024-02-23T00:45:11.167582+0000","last_became_active":"2024-02-23T00:45:11.167069+0000","last_became_peered":"2024-02-23T00:45:11.167069+0000","last_unstale":"2024-02-23T00:45:11.167582+0000","last_undegraded":"2024-02-23T00:45:11.167582+0000","last_fullsized":"2024-02-23T00:45:11.167582+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.131641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"155.2","version":"0'0","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.158879+0000","last_change":"2024-02-23T00:45:09.152990+0000","last_active":"2024-02-23T00:45:11.158879+0000","last_peered":"2024-02-23T00:45:11.158879+0000","last_clean":"2024-02-23T00:45:11.158879+0000","last_became_active":"2024-02-23T00:45:09.152685+0000","last_became_peered":"2024-02-23T00:45:09.152685+0000","last_unstale":"2024-02-23T00:45:11.158879+0000","last_undegraded":"2024-02-23T00:45:11.158879+0000","last_fullsized":"2024-02-23T00:45:11.158879+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.124500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"159.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:15.119257+0000","last_change":"2024-02-23T00:45:15.119257+0000","last_active":"2024-02-23T00:45:15.119257+0000","last_peered":"2024-02-23T00:45:15.119257+0000","last_clean":"2024-02-23T00:45:15.119257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:15.119257+0000","last_undegraded":"2024-02-23T00:45:15.119257+0000","last_fullsized":"2024-02-23T00:45:15.119257+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:15.119257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:15.119257+0000","last_clean_scrub_stamp":"2024-02-23T00:45:15.119257+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"158.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:14.145983+0000","last_change":"2024-02-23T00:45:14.145983+0000","last_active":"2024-02-23T00:45:14.145983+0000","last_peered":"2024-02-23T00:45:14.145983+0000","last_clean":"2024-02-23T00:45:14.145983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:14.145983+0000","last_undegraded":"2024-02-23T00:45:14.145983+0000","last_fullsized":"2024-02-23T00:45:14.145983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.145983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"157.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:12.139981+0000","last_change":"2024-02-23T00:45:12.146571+0000","last_active":"2024-02-23T00:45:12.139981+0000","last_peered":"2024-02-23T00:45:12.139981+0000","last_clean":"2024-02-23T00:45:12.139981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:12.139981+0000","last_undegraded":"2024-02-23T00:45:12.139981+0000","last_fullsized":"2024-02-23T00:45:12.139981+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.139981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"156.6","version":"0'0","reported_seq":10,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.162646+0000","last_change":"2024-02-23T00:45:11.161676+0000","last_active":"2024-02-23T00:45:11.162646+0000","last_peered":"2024-02-23T00:45:11.162646+0000","last_clean":"2024-02-23T00:45:11.162646+0000","last_became_active":"2024-02-23T00:45:11.161386+0000","last_became_peered":"2024-02-23T00:45:11.161386+0000","last_unstale":"2024-02-23T00:45:11.162646+0000","last_undegraded":"2024-02-23T00:45:11.162646+0000","last_fullsized":"2024-02-23T00:45:11.162646+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.131641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"155.1","version":"0'0","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.144237+0000","last_change":"2024-02-23T00:45:09.167500+0000","last_active":"2024-02-23T00:45:11.144237+0000","last_peered":"2024-02-23T00:45:11.144237+0000","last_clean":"2024-02-23T00:45:11.144237+0000","last_became_active":"2024-02-23T00:45:09.167239+0000","last_became_peered":"2024-02-23T00:45:09.167239+0000","last_unstale":"2024-02-23T00:45:11.144237+0000","last_undegraded":"2024-02-23T00:45:11.144237+0000","last_fullsized":"2024-02-23T00:45:11.144237+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.124500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"159.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:15.119257+0000","last_change":"2024-02-23T00:45:15.119257+0000","last_active":"2024-02-23T00:45:15.119257+0000","last_peered":"2024-02-23T00:45:15.119257+0000","last_clean":"2024-02-23T00:45:15.119257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:15.119257+0000","last_undegraded":"2024-02-23T00:45:15.119257+0000","last_fullsized":"2024-02-23T00:45:15.119257+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:15.119257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:15.119257+0000","last_clean_scrub_stamp":"2024-02-23T00:45:15.119257+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"158.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:14.145983+0000","last_change":"2024-02-23T00:45:14.145983+0000","last_active":"2024-02-23T00:45:14.145983+0000","last_peered":"2024-02-23T00:45:14.145983+0000","last_clean":"2024-02-23T00:45:14.145983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:14.145983+0000","last_undegraded":"2024-02-23T00:45:14.145983+0000","last_fullsized":"2024-02-23T00:45:14.145983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.145983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"157.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:12.139981+0000","last_change":"2024-02-23T00:45:12.149147+0000","last_active":"2024-02-23T00:45:12.139981+0000","last_peered":"2024-02-23T00:45:12.139981+0000","last_clean":"2024-02-23T00:45:12.139981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:12.139981+0000","last_undegraded":"2024-02-23T00:45:12.139981+0000","last_fullsized":"2024-02-23T00:45:12.139981+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.139981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"156.7","version":"0'0","reported_seq":10,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.169817+0000","last_change":"2024-02-23T00:45:11.169343+0000","last_active":"2024-02-23T00:45:11.169817+0000","last_peered":"2024-02-23T00:45:11.169817+0000","last_clean":"2024-02-23T00:45:11.169817+0000","last_became_active":"2024-02-23T00:45:11.169007+0000","last_became_peered":"2024-02-23T00:45:11.169007+0000","last_unstale":"2024-02-23T00:45:11.169817+0000","last_undegraded":"2024-02-23T00:45:11.169817+0000","last_fullsized":"2024-02-23T00:45:11.169817+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.131641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"155.0","version":"0'0","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.160922+0000","last_change":"2024-02-23T00:45:09.149721+0000","last_active":"2024-02-23T00:45:11.160922+0000","last_peered":"2024-02-23T00:45:11.160922+0000","last_clean":"2024-02-23T00:45:11.160922+0000","last_became_active":"2024-02-23T00:45:09.149445+0000","last_became_peered":"2024-02-23T00:45:09.149445+0000","last_unstale":"2024-02-23T00:45:11.160922+0000","last_undegraded":"2024-02-23T00:45:11.160922+0000","last_fullsized":"2024-02-23T00:45:11.160922+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.124500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"159.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:15.119257+0000","last_change":"2024-02-23T00:45:15.119257+0000","last_active":"2024-02-23T00:45:15.119257+0000","last_peered":"2024-02-23T00:45:15.119257+0000","last_clean":"2024-02-23T00:45:15.119257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:15.119257+0000","last_undegraded":"2024-02-23T00:45:15.119257+0000","last_fullsized":"2024-02-23T00:45:15.119257+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:15.119257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:15.119257+0000","last_clean_scrub_stamp":"2024-02-23T00:45:15.119257+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"158.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:14.145983+0000","last_change":"2024-02-23T00:45:14.145983+0000","last_active":"2024-02-23T00:45:14.145983+0000","last_peered":"2024-02-23T00:45:14.145983+0000","last_clean":"2024-02-23T00:45:14.145983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:14.145983+0000","last_undegraded":"2024-02-23T00:45:14.145983+0000","last_fullsized":"2024-02-23T00:45:14.145983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.145983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"157.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:12.139981+0000","last_change":"2024-02-23T00:45:12.149196+0000","last_active":"2024-02-23T00:45:12.139981+0000","last_peered":"2024-02-23T00:45:12.139981+0000","last_clean":"2024-02-23T00:45:12.139981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:12.139981+0000","last_undegraded":"2024-02-23T00:45:12.139981+0000","last_fullsized":"2024-02-23T00:45:12.139981+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.139981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"156.0","version":"0'0","reported_seq":10,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.165025+0000","last_change":"2024-02-23T00:45:11.164653+0000","last_active":"2024-02-23T00:45:11.165025+0000","last_peered":"2024-02-23T00:45:11.165025+0000","last_clean":"2024-02-23T00:45:11.165025+0000","last_became_active":"2024-02-23T00:45:11.164329+0000","last_became_peered":"2024-02-23T00:45:11.164329+0000","last_unstale":"2024-02-23T00:45:11.165025+0000","last_undegraded":"2024-02-23T00:45:11.165025+0000","last_fullsized":"2024-02-23T00:45:11.165025+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.131641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"155.7","version":"239'3","reported_seq":15,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.166947+0000","last_change":"2024-02-23T00:45:09.161923+0000","last_active":"2024-02-23T00:45:11.166947+0000","last_peered":"2024-02-23T00:45:11.166947+0000","last_clean":"2024-02-23T00:45:11.166947+0000","last_became_active":"2024-02-23T00:45:09.161426+0000","last_became_peered":"2024-02-23T00:45:09.161426+0000","last_unstale":"2024-02-23T00:45:11.166947+0000","last_undegraded":"2024-02-23T00:45:11.166947+0000","last_fullsized":"2024-02-23T00:45:11.166947+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.124500+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2],"acting":[4,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"159.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:15.119257+0000","last_change":"2024-02-23T00:45:15.119257+0000","last_active":"2024-02-23T00:45:15.119257+0000","last_peered":"2024-02-23T00:45:15.119257+0000","last_clean":"2024-02-23T00:45:15.119257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:15.119257+0000","last_undegraded":"2024-02-23T00:45:15.119257+0000","last_fullsized":"2024-02-23T00:45:15.119257+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:15.119257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:15.119257+0000","last_clean_scrub_stamp":"2024-02-23T00:45:15.119257+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"158.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:14.145983+0000","last_change":"2024-02-23T00:45:14.145983+0000","last_active":"2024-02-23T00:45:14.145983+0000","last_peered":"2024-02-23T00:45:14.145983+0000","last_clean":"2024-02-23T00:45:14.145983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:14.145983+0000","last_undegraded":"2024-02-23T00:45:14.145983+0000","last_fullsized":"2024-02-23T00:45:14.145983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.145983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"157.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:12.139981+0000","last_change":"2024-02-23T00:45:12.146374+0000","last_active":"2024-02-23T00:45:12.139981+0000","last_peered":"2024-02-23T00:45:12.139981+0000","last_clean":"2024-02-23T00:45:12.139981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:12.139981+0000","last_undegraded":"2024-02-23T00:45:12.139981+0000","last_fullsized":"2024-02-23T00:45:12.139981+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.139981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"156.1","version":"0'0","reported_seq":10,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.163707+0000","last_change":"2024-02-23T00:45:11.154258+0000","last_active":"2024-02-23T00:45:11.163707+0000","last_peered":"2024-02-23T00:45:11.163707+0000","last_clean":"2024-02-23T00:45:11.163707+0000","last_became_active":"2024-02-23T00:45:11.153782+0000","last_became_peered":"2024-02-23T00:45:11.153782+0000","last_unstale":"2024-02-23T00:45:11.163707+0000","last_undegraded":"2024-02-23T00:45:11.163707+0000","last_fullsized":"2024-02-23T00:45:11.163707+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.131641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"155.6","version":"0'0","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.158124+0000","last_change":"2024-02-23T00:45:09.157224+0000","last_active":"2024-02-23T00:45:11.158124+0000","last_peered":"2024-02-23T00:45:11.158124+0000","last_clean":"2024-02-23T00:45:11.158124+0000","last_became_active":"2024-02-23T00:45:09.156944+0000","last_became_peered":"2024-02-23T00:45:09.156944+0000","last_unstale":"2024-02-23T00:45:11.158124+0000","last_undegraded":"2024-02-23T00:45:11.158124+0000","last_fullsized":"2024-02-23T00:45:11.158124+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.124500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,3],"acting":[1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"159.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:15.119257+0000","last_change":"2024-02-23T00:45:15.119257+0000","last_active":"2024-02-23T00:45:15.119257+0000","last_peered":"2024-02-23T00:45:15.119257+0000","last_clean":"2024-02-23T00:45:15.119257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:15.119257+0000","last_undegraded":"2024-02-23T00:45:15.119257+0000","last_fullsized":"2024-02-23T00:45:15.119257+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:15.119257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:15.119257+0000","last_clean_scrub_stamp":"2024-02-23T00:45:15.119257+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"158.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:14.145983+0000","last_change":"2024-02-23T00:45:14.145983+0000","last_active":"2024-02-23T00:45:14.145983+0000","last_peered":"2024-02-23T00:45:14.145983+0000","last_clean":"2024-02-23T00:45:14.145983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:14.145983+0000","last_undegraded":"2024-02-23T00:45:14.145983+0000","last_fullsized":"2024-02-23T00:45:14.145983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.145983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"157.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:12.139981+0000","last_change":"2024-02-23T00:45:12.146416+0000","last_active":"2024-02-23T00:45:12.139981+0000","last_peered":"2024-02-23T00:45:12.139981+0000","last_clean":"2024-02-23T00:45:12.139981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:12.139981+0000","last_undegraded":"2024-02-23T00:45:12.139981+0000","last_fullsized":"2024-02-23T00:45:12.139981+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.139981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"156.2","version":"0'0","reported_seq":10,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.172442+0000","last_change":"2024-02-23T00:45:11.172024+0000","last_active":"2024-02-23T00:45:11.172442+0000","last_peered":"2024-02-23T00:45:11.172442+0000","last_clean":"2024-02-23T00:45:11.172442+0000","last_became_active":"2024-02-23T00:45:11.171736+0000","last_became_peered":"2024-02-23T00:45:11.171736+0000","last_unstale":"2024-02-23T00:45:11.172442+0000","last_undegraded":"2024-02-23T00:45:11.172442+0000","last_fullsized":"2024-02-23T00:45:11.172442+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.131641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"155.5","version":"0'0","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.145516+0000","last_change":"2024-02-23T00:45:09.153597+0000","last_active":"2024-02-23T00:45:11.145516+0000","last_peered":"2024-02-23T00:45:11.145516+0000","last_clean":"2024-02-23T00:45:11.145516+0000","last_became_active":"2024-02-23T00:45:09.153177+0000","last_became_peered":"2024-02-23T00:45:09.153177+0000","last_unstale":"2024-02-23T00:45:11.145516+0000","last_undegraded":"2024-02-23T00:45:11.145516+0000","last_fullsized":"2024-02-23T00:45:11.145516+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.124500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"159.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:15.119257+0000","last_change":"2024-02-23T00:45:15.119257+0000","last_active":"2024-02-23T00:45:15.119257+0000","last_peered":"2024-02-23T00:45:15.119257+0000","last_clean":"2024-02-23T00:45:15.119257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:15.119257+0000","last_undegraded":"2024-02-23T00:45:15.119257+0000","last_fullsized":"2024-02-23T00:45:15.119257+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:15.119257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:15.119257+0000","last_clean_scrub_stamp":"2024-02-23T00:45:15.119257+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"158.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:14.145983+0000","last_change":"2024-02-23T00:45:14.145983+0000","last_active":"2024-02-23T00:45:14.145983+0000","last_peered":"2024-02-23T00:45:14.145983+0000","last_clean":"2024-02-23T00:45:14.145983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:14.145983+0000","last_undegraded":"2024-02-23T00:45:14.145983+0000","last_fullsized":"2024-02-23T00:45:14.145983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.145983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"157.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:12.139981+0000","last_change":"2024-02-23T00:45:12.154463+0000","last_active":"2024-02-23T00:45:12.139981+0000","last_peered":"2024-02-23T00:45:12.139981+0000","last_clean":"2024-02-23T00:45:12.139981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:12.139981+0000","last_undegraded":"2024-02-23T00:45:12.139981+0000","last_fullsized":"2024-02-23T00:45:12.139981+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.139981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"156.3","version":"0'0","reported_seq":10,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.164319+0000","last_change":"2024-02-23T00:45:11.162328+0000","last_active":"2024-02-23T00:45:11.164319+0000","last_peered":"2024-02-23T00:45:11.164319+0000","last_clean":"2024-02-23T00:45:11.164319+0000","last_became_active":"2024-02-23T00:45:11.161983+0000","last_became_peered":"2024-02-23T00:45:11.161983+0000","last_unstale":"2024-02-23T00:45:11.164319+0000","last_undegraded":"2024-02-23T00:45:11.164319+0000","last_fullsized":"2024-02-23T00:45:11.164319+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.131641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"155.4","version":"0'0","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.153100+0000","last_change":"2024-02-23T00:45:09.165915+0000","last_active":"2024-02-23T00:45:11.153100+0000","last_peered":"2024-02-23T00:45:11.153100+0000","last_clean":"2024-02-23T00:45:11.153100+0000","last_became_active":"2024-02-23T00:45:09.165536+0000","last_became_peered":"2024-02-23T00:45:09.165536+0000","last_unstale":"2024-02-23T00:45:11.153100+0000","last_undegraded":"2024-02-23T00:45:11.153100+0000","last_fullsized":"2024-02-23T00:45:11.153100+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.124500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"158.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:14.145983+0000","last_change":"2024-02-23T00:45:14.145983+0000","last_active":"2024-02-23T00:45:14.145983+0000","last_peered":"2024-02-23T00:45:14.145983+0000","last_clean":"2024-02-23T00:45:14.145983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:14.145983+0000","last_undegraded":"2024-02-23T00:45:14.145983+0000","last_fullsized":"2024-02-23T00:45:14.145983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.145983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"157.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:12.139981+0000","last_change":"2024-02-23T00:45:12.146396+0000","last_active":"2024-02-23T00:45:12.139981+0000","last_peered":"2024-02-23T00:45:12.139981+0000","last_clean":"2024-02-23T00:45:12.139981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:12.139981+0000","last_undegraded":"2024-02-23T00:45:12.139981+0000","last_fullsized":"2024-02-23T00:45:12.139981+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.139981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,2],"acting":[4,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"156.1c","version":"0'0","reported_seq":10,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.159353+0000","last_change":"2024-02-23T00:45:11.155301+0000","last_active":"2024-02-23T00:45:11.159353+0000","last_peered":"2024-02-23T00:45:11.159353+0000","last_clean":"2024-02-23T00:45:11.159353+0000","last_became_active":"2024-02-23T00:45:11.154943+0000","last_became_peered":"2024-02-23T00:45:11.154943+0000","last_unstale":"2024-02-23T00:45:11.159353+0000","last_undegraded":"2024-02-23T00:45:11.159353+0000","last_fullsized":"2024-02-23T00:45:11.159353+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.131641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"155.1b","version":"0'0","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.164165+0000","last_change":"2024-02-23T00:45:09.148065+0000","last_active":"2024-02-23T00:45:11.164165+0000","last_peered":"2024-02-23T00:45:11.164165+0000","last_clean":"2024-02-23T00:45:11.164165+0000","last_became_active":"2024-02-23T00:45:09.147717+0000","last_became_peered":"2024-02-23T00:45:09.147717+0000","last_unstale":"2024-02-23T00:45:11.164165+0000","last_undegraded":"2024-02-23T00:45:11.164165+0000","last_fullsized":"2024-02-23T00:45:11.164165+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.124500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1],"acting":[6,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"158.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:14.145983+0000","last_change":"2024-02-23T00:45:14.145983+0000","last_active":"2024-02-23T00:45:14.145983+0000","last_peered":"2024-02-23T00:45:14.145983+0000","last_clean":"2024-02-23T00:45:14.145983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:14.145983+0000","last_undegraded":"2024-02-23T00:45:14.145983+0000","last_fullsized":"2024-02-23T00:45:14.145983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.145983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"157.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:12.139981+0000","last_change":"2024-02-23T00:45:12.148980+0000","last_active":"2024-02-23T00:45:12.139981+0000","last_peered":"2024-02-23T00:45:12.139981+0000","last_clean":"2024-02-23T00:45:12.139981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:12.139981+0000","last_undegraded":"2024-02-23T00:45:12.139981+0000","last_fullsized":"2024-02-23T00:45:12.139981+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.139981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"156.1d","version":"0'0","reported_seq":10,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.172750+0000","last_change":"2024-02-23T00:45:11.171013+0000","last_active":"2024-02-23T00:45:11.172750+0000","last_peered":"2024-02-23T00:45:11.172750+0000","last_clean":"2024-02-23T00:45:11.172750+0000","last_became_active":"2024-02-23T00:45:11.170720+0000","last_became_peered":"2024-02-23T00:45:11.170720+0000","last_unstale":"2024-02-23T00:45:11.172750+0000","last_undegraded":"2024-02-23T00:45:11.172750+0000","last_fullsized":"2024-02-23T00:45:11.172750+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.131641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"155.1a","version":"0'0","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.142686+0000","last_change":"2024-02-23T00:45:09.163754+0000","last_active":"2024-02-23T00:45:11.142686+0000","last_peered":"2024-02-23T00:45:11.142686+0000","last_clean":"2024-02-23T00:45:11.142686+0000","last_became_active":"2024-02-23T00:45:09.163454+0000","last_became_peered":"2024-02-23T00:45:09.163454+0000","last_unstale":"2024-02-23T00:45:11.142686+0000","last_undegraded":"2024-02-23T00:45:11.142686+0000","last_fullsized":"2024-02-23T00:45:11.142686+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.124500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"158.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:14.145983+0000","last_change":"2024-02-23T00:45:14.145983+0000","last_active":"2024-02-23T00:45:14.145983+0000","last_peered":"2024-02-23T00:45:14.145983+0000","last_clean":"2024-02-23T00:45:14.145983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:14.145983+0000","last_undegraded":"2024-02-23T00:45:14.145983+0000","last_fullsized":"2024-02-23T00:45:14.145983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.145983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"157.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:12.139981+0000","last_change":"2024-02-23T00:45:12.152271+0000","last_active":"2024-02-23T00:45:12.139981+0000","last_peered":"2024-02-23T00:45:12.139981+0000","last_clean":"2024-02-23T00:45:12.139981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:12.139981+0000","last_undegraded":"2024-02-23T00:45:12.139981+0000","last_fullsized":"2024-02-23T00:45:12.139981+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.139981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"156.1e","version":"0'0","reported_seq":10,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.172436+0000","last_change":"2024-02-23T00:45:11.171885+0000","last_active":"2024-02-23T00:45:11.172436+0000","last_peered":"2024-02-23T00:45:11.172436+0000","last_clean":"2024-02-23T00:45:11.172436+0000","last_became_active":"2024-02-23T00:45:11.171561+0000","last_became_peered":"2024-02-23T00:45:11.171561+0000","last_unstale":"2024-02-23T00:45:11.172436+0000","last_undegraded":"2024-02-23T00:45:11.172436+0000","last_fullsized":"2024-02-23T00:45:11.172436+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.131641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"155.19","version":"0'0","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.148846+0000","last_change":"2024-02-23T00:45:09.167377+0000","last_active":"2024-02-23T00:45:11.148846+0000","last_peered":"2024-02-23T00:45:11.148846+0000","last_clean":"2024-02-23T00:45:11.148846+0000","last_became_active":"2024-02-23T00:45:09.167074+0000","last_became_peered":"2024-02-23T00:45:09.167074+0000","last_unstale":"2024-02-23T00:45:11.148846+0000","last_undegraded":"2024-02-23T00:45:11.148846+0000","last_fullsized":"2024-02-23T00:45:11.148846+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.124500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"158.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:14.145983+0000","last_change":"2024-02-23T00:45:14.145983+0000","last_active":"2024-02-23T00:45:14.145983+0000","last_peered":"2024-02-23T00:45:14.145983+0000","last_clean":"2024-02-23T00:45:14.145983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:14.145983+0000","last_undegraded":"2024-02-23T00:45:14.145983+0000","last_fullsized":"2024-02-23T00:45:14.145983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.145983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"157.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:12.139981+0000","last_change":"2024-02-23T00:45:12.148549+0000","last_active":"2024-02-23T00:45:12.139981+0000","last_peered":"2024-02-23T00:45:12.139981+0000","last_clean":"2024-02-23T00:45:12.139981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:12.139981+0000","last_undegraded":"2024-02-23T00:45:12.139981+0000","last_fullsized":"2024-02-23T00:45:12.139981+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.139981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5],"acting":[6,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"156.1f","version":"0'0","reported_seq":10,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.161251+0000","last_change":"2024-02-23T00:45:11.160966+0000","last_active":"2024-02-23T00:45:11.161251+0000","last_peered":"2024-02-23T00:45:11.161251+0000","last_clean":"2024-02-23T00:45:11.161251+0000","last_became_active":"2024-02-23T00:45:11.160629+0000","last_became_peered":"2024-02-23T00:45:11.160629+0000","last_unstale":"2024-02-23T00:45:11.161251+0000","last_undegraded":"2024-02-23T00:45:11.161251+0000","last_fullsized":"2024-02-23T00:45:11.161251+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.131641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"155.18","version":"0'0","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.153923+0000","last_change":"2024-02-23T00:45:09.154956+0000","last_active":"2024-02-23T00:45:11.153923+0000","last_peered":"2024-02-23T00:45:11.153923+0000","last_clean":"2024-02-23T00:45:11.153923+0000","last_became_active":"2024-02-23T00:45:09.154596+0000","last_became_peered":"2024-02-23T00:45:09.154596+0000","last_unstale":"2024-02-23T00:45:11.153923+0000","last_undegraded":"2024-02-23T00:45:11.153923+0000","last_fullsized":"2024-02-23T00:45:11.153923+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.124500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"158.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:14.145983+0000","last_change":"2024-02-23T00:45:14.145983+0000","last_active":"2024-02-23T00:45:14.145983+0000","last_peered":"2024-02-23T00:45:14.145983+0000","last_clean":"2024-02-23T00:45:14.145983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:14.145983+0000","last_undegraded":"2024-02-23T00:45:14.145983+0000","last_fullsized":"2024-02-23T00:45:14.145983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.145983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"157.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:12.139981+0000","last_change":"2024-02-23T00:45:12.155069+0000","last_active":"2024-02-23T00:45:12.139981+0000","last_peered":"2024-02-23T00:45:12.139981+0000","last_clean":"2024-02-23T00:45:12.139981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:12.139981+0000","last_undegraded":"2024-02-23T00:45:12.139981+0000","last_fullsized":"2024-02-23T00:45:12.139981+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.139981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"156.18","version":"0'0","reported_seq":10,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.167826+0000","last_change":"2024-02-23T00:45:11.166758+0000","last_active":"2024-02-23T00:45:11.167826+0000","last_peered":"2024-02-23T00:45:11.167826+0000","last_clean":"2024-02-23T00:45:11.167826+0000","last_became_active":"2024-02-23T00:45:11.166435+0000","last_became_peered":"2024-02-23T00:45:11.166435+0000","last_unstale":"2024-02-23T00:45:11.167826+0000","last_undegraded":"2024-02-23T00:45:11.167826+0000","last_fullsized":"2024-02-23T00:45:11.167826+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.131641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"155.1f","version":"0'0","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.164356+0000","last_change":"2024-02-23T00:45:09.147443+0000","last_active":"2024-02-23T00:45:11.164356+0000","last_peered":"2024-02-23T00:45:11.164356+0000","last_clean":"2024-02-23T00:45:11.164356+0000","last_became_active":"2024-02-23T00:45:09.147107+0000","last_became_peered":"2024-02-23T00:45:09.147107+0000","last_unstale":"2024-02-23T00:45:11.164356+0000","last_undegraded":"2024-02-23T00:45:11.164356+0000","last_fullsized":"2024-02-23T00:45:11.164356+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.124500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"158.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:14.145983+0000","last_change":"2024-02-23T00:45:14.145983+0000","last_active":"2024-02-23T00:45:14.145983+0000","last_peered":"2024-02-23T00:45:14.145983+0000","last_clean":"2024-02-23T00:45:14.145983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:14.145983+0000","last_undegraded":"2024-02-23T00:45:14.145983+0000","last_fullsized":"2024-02-23T00:45:14.145983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.145983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"157.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:12.139981+0000","last_change":"2024-02-23T00:45:12.153950+0000","last_active":"2024-02-23T00:45:12.139981+0000","last_peered":"2024-02-23T00:45:12.139981+0000","last_clean":"2024-02-23T00:45:12.139981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:12.139981+0000","last_undegraded":"2024-02-23T00:45:12.139981+0000","last_fullsized":"2024-02-23T00:45:12.139981+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.139981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"156.19","version":"0'0","reported_seq":10,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.172414+0000","last_change":"2024-02-23T00:45:11.172034+0000","last_active":"2024-02-23T00:45:11.172414+0000","last_peered":"2024-02-23T00:45:11.172414+0000","last_clean":"2024-02-23T00:45:11.172414+0000","last_became_active":"2024-02-23T00:45:11.171721+0000","last_became_peered":"2024-02-23T00:45:11.171721+0000","last_unstale":"2024-02-23T00:45:11.172414+0000","last_undegraded":"2024-02-23T00:45:11.172414+0000","last_fullsized":"2024-02-23T00:45:11.172414+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.131641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"155.1e","version":"0'0","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.143400+0000","last_change":"2024-02-23T00:45:09.162019+0000","last_active":"2024-02-23T00:45:11.143400+0000","last_peered":"2024-02-23T00:45:11.143400+0000","last_clean":"2024-02-23T00:45:11.143400+0000","last_became_active":"2024-02-23T00:45:09.161637+0000","last_became_peered":"2024-02-23T00:45:09.161637+0000","last_unstale":"2024-02-23T00:45:11.143400+0000","last_undegraded":"2024-02-23T00:45:11.143400+0000","last_fullsized":"2024-02-23T00:45:11.143400+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.124500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"158.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:14.145983+0000","last_change":"2024-02-23T00:45:14.145983+0000","last_active":"2024-02-23T00:45:14.145983+0000","last_peered":"2024-02-23T00:45:14.145983+0000","last_clean":"2024-02-23T00:45:14.145983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:14.145983+0000","last_undegraded":"2024-02-23T00:45:14.145983+0000","last_fullsized":"2024-02-23T00:45:14.145983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.145983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"157.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:12.139981+0000","last_change":"2024-02-23T00:45:12.154518+0000","last_active":"2024-02-23T00:45:12.139981+0000","last_peered":"2024-02-23T00:45:12.139981+0000","last_clean":"2024-02-23T00:45:12.139981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:12.139981+0000","last_undegraded":"2024-02-23T00:45:12.139981+0000","last_fullsized":"2024-02-23T00:45:12.139981+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.139981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"156.1a","version":"0'0","reported_seq":10,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.170616+0000","last_change":"2024-02-23T00:45:11.170155+0000","last_active":"2024-02-23T00:45:11.170616+0000","last_peered":"2024-02-23T00:45:11.170616+0000","last_clean":"2024-02-23T00:45:11.170616+0000","last_became_active":"2024-02-23T00:45:11.169872+0000","last_became_peered":"2024-02-23T00:45:11.169872+0000","last_unstale":"2024-02-23T00:45:11.170616+0000","last_undegraded":"2024-02-23T00:45:11.170616+0000","last_fullsized":"2024-02-23T00:45:11.170616+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.131641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"155.1d","version":"0'0","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.161265+0000","last_change":"2024-02-23T00:45:09.153407+0000","last_active":"2024-02-23T00:45:11.161265+0000","last_peered":"2024-02-23T00:45:11.161265+0000","last_clean":"2024-02-23T00:45:11.161265+0000","last_became_active":"2024-02-23T00:45:09.152651+0000","last_became_peered":"2024-02-23T00:45:09.152651+0000","last_unstale":"2024-02-23T00:45:11.161265+0000","last_undegraded":"2024-02-23T00:45:11.161265+0000","last_fullsized":"2024-02-23T00:45:11.161265+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.124500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"158.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:14.145983+0000","last_change":"2024-02-23T00:45:14.145983+0000","last_active":"2024-02-23T00:45:14.145983+0000","last_peered":"2024-02-23T00:45:14.145983+0000","last_clean":"2024-02-23T00:45:14.145983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:14.145983+0000","last_undegraded":"2024-02-23T00:45:14.145983+0000","last_fullsized":"2024-02-23T00:45:14.145983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.145983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"157.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:12.139981+0000","last_change":"2024-02-23T00:45:12.146397+0000","last_active":"2024-02-23T00:45:12.139981+0000","last_peered":"2024-02-23T00:45:12.139981+0000","last_clean":"2024-02-23T00:45:12.139981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:12.139981+0000","last_undegraded":"2024-02-23T00:45:12.139981+0000","last_fullsized":"2024-02-23T00:45:12.139981+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.139981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"156.1b","version":"0'0","reported_seq":10,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.173912+0000","last_change":"2024-02-23T00:45:11.173515+0000","last_active":"2024-02-23T00:45:11.173912+0000","last_peered":"2024-02-23T00:45:11.173912+0000","last_clean":"2024-02-23T00:45:11.173912+0000","last_became_active":"2024-02-23T00:45:11.173155+0000","last_became_peered":"2024-02-23T00:45:11.173155+0000","last_unstale":"2024-02-23T00:45:11.173912+0000","last_undegraded":"2024-02-23T00:45:11.173912+0000","last_fullsized":"2024-02-23T00:45:11.173912+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.131641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,7],"acting":[6,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"155.1c","version":"0'0","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.156074+0000","last_change":"2024-02-23T00:45:09.152705+0000","last_active":"2024-02-23T00:45:11.156074+0000","last_peered":"2024-02-23T00:45:11.156074+0000","last_clean":"2024-02-23T00:45:11.156074+0000","last_became_active":"2024-02-23T00:45:09.152308+0000","last_became_peered":"2024-02-23T00:45:09.152308+0000","last_unstale":"2024-02-23T00:45:11.156074+0000","last_undegraded":"2024-02-23T00:45:11.156074+0000","last_fullsized":"2024-02-23T00:45:11.156074+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.124500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"158.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:14.145983+0000","last_change":"2024-02-23T00:45:14.145983+0000","last_active":"2024-02-23T00:45:14.145983+0000","last_peered":"2024-02-23T00:45:14.145983+0000","last_clean":"2024-02-23T00:45:14.145983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:14.145983+0000","last_undegraded":"2024-02-23T00:45:14.145983+0000","last_fullsized":"2024-02-23T00:45:14.145983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.145983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"157.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:12.139981+0000","last_change":"2024-02-23T00:45:12.148565+0000","last_active":"2024-02-23T00:45:12.139981+0000","last_peered":"2024-02-23T00:45:12.139981+0000","last_clean":"2024-02-23T00:45:12.139981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:12.139981+0000","last_undegraded":"2024-02-23T00:45:12.139981+0000","last_fullsized":"2024-02-23T00:45:12.139981+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.139981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"156.14","version":"0'0","reported_seq":10,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.161789+0000","last_change":"2024-02-23T00:45:11.161076+0000","last_active":"2024-02-23T00:45:11.161789+0000","last_peered":"2024-02-23T00:45:11.161789+0000","last_clean":"2024-02-23T00:45:11.161789+0000","last_became_active":"2024-02-23T00:45:11.160763+0000","last_became_peered":"2024-02-23T00:45:11.160763+0000","last_unstale":"2024-02-23T00:45:11.161789+0000","last_undegraded":"2024-02-23T00:45:11.161789+0000","last_fullsized":"2024-02-23T00:45:11.161789+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.131641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"155.13","version":"0'0","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.149154+0000","last_change":"2024-02-23T00:45:09.158977+0000","last_active":"2024-02-23T00:45:11.149154+0000","last_peered":"2024-02-23T00:45:11.149154+0000","last_clean":"2024-02-23T00:45:11.149154+0000","last_became_active":"2024-02-23T00:45:09.158338+0000","last_became_peered":"2024-02-23T00:45:09.158338+0000","last_unstale":"2024-02-23T00:45:11.149154+0000","last_undegraded":"2024-02-23T00:45:11.149154+0000","last_fullsized":"2024-02-23T00:45:11.149154+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.124500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,0],"acting":[2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"158.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:14.145983+0000","last_change":"2024-02-23T00:45:14.145983+0000","last_active":"2024-02-23T00:45:14.145983+0000","last_peered":"2024-02-23T00:45:14.145983+0000","last_clean":"2024-02-23T00:45:14.145983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:14.145983+0000","last_undegraded":"2024-02-23T00:45:14.145983+0000","last_fullsized":"2024-02-23T00:45:14.145983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.145983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"157.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:12.139981+0000","last_change":"2024-02-23T00:45:12.152760+0000","last_active":"2024-02-23T00:45:12.139981+0000","last_peered":"2024-02-23T00:45:12.139981+0000","last_clean":"2024-02-23T00:45:12.139981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:12.139981+0000","last_undegraded":"2024-02-23T00:45:12.139981+0000","last_fullsized":"2024-02-23T00:45:12.139981+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.139981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"156.15","version":"0'0","reported_seq":10,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.169303+0000","last_change":"2024-02-23T00:45:11.167996+0000","last_active":"2024-02-23T00:45:11.169303+0000","last_peered":"2024-02-23T00:45:11.169303+0000","last_clean":"2024-02-23T00:45:11.169303+0000","last_became_active":"2024-02-23T00:45:11.167577+0000","last_became_peered":"2024-02-23T00:45:11.167577+0000","last_unstale":"2024-02-23T00:45:11.169303+0000","last_undegraded":"2024-02-23T00:45:11.169303+0000","last_fullsized":"2024-02-23T00:45:11.169303+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.131641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"155.12","version":"0'0","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.154693+0000","last_change":"2024-02-23T00:45:09.163170+0000","last_active":"2024-02-23T00:45:11.154693+0000","last_peered":"2024-02-23T00:45:11.154693+0000","last_clean":"2024-02-23T00:45:11.154693+0000","last_became_active":"2024-02-23T00:45:09.162837+0000","last_became_peered":"2024-02-23T00:45:09.162837+0000","last_unstale":"2024-02-23T00:45:11.154693+0000","last_undegraded":"2024-02-23T00:45:11.154693+0000","last_fullsized":"2024-02-23T00:45:11.154693+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.124500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"158.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:14.145983+0000","last_change":"2024-02-23T00:45:14.145983+0000","last_active":"2024-02-23T00:45:14.145983+0000","last_peered":"2024-02-23T00:45:14.145983+0000","last_clean":"2024-02-23T00:45:14.145983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:14.145983+0000","last_undegraded":"2024-02-23T00:45:14.145983+0000","last_fullsized":"2024-02-23T00:45:14.145983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.145983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"157.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:12.139981+0000","last_change":"2024-02-23T00:45:12.150092+0000","last_active":"2024-02-23T00:45:12.139981+0000","last_peered":"2024-02-23T00:45:12.139981+0000","last_clean":"2024-02-23T00:45:12.139981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:12.139981+0000","last_undegraded":"2024-02-23T00:45:12.139981+0000","last_fullsized":"2024-02-23T00:45:12.139981+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.139981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,7],"acting":[5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6,7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"156.16","version":"0'0","reported_seq":10,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.173921+0000","last_change":"2024-02-23T00:45:11.173571+0000","last_active":"2024-02-23T00:45:11.173921+0000","last_peered":"2024-02-23T00:45:11.173921+0000","last_clean":"2024-02-23T00:45:11.173921+0000","last_became_active":"2024-02-23T00:45:11.173150+0000","last_became_peered":"2024-02-23T00:45:11.173150+0000","last_unstale":"2024-02-23T00:45:11.173921+0000","last_undegraded":"2024-02-23T00:45:11.173921+0000","last_fullsized":"2024-02-23T00:45:11.173921+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.131641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"155.11","version":"0'0","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.143900+0000","last_change":"2024-02-23T00:45:09.151756+0000","last_active":"2024-02-23T00:45:11.143900+0000","last_peered":"2024-02-23T00:45:11.143900+0000","last_clean":"2024-02-23T00:45:11.143900+0000","last_became_active":"2024-02-23T00:45:09.151402+0000","last_became_peered":"2024-02-23T00:45:09.151402+0000","last_unstale":"2024-02-23T00:45:11.143900+0000","last_undegraded":"2024-02-23T00:45:11.143900+0000","last_fullsized":"2024-02-23T00:45:11.143900+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.124500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"158.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:14.145983+0000","last_change":"2024-02-23T00:45:14.145983+0000","last_active":"2024-02-23T00:45:14.145983+0000","last_peered":"2024-02-23T00:45:14.145983+0000","last_clean":"2024-02-23T00:45:14.145983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:14.145983+0000","last_undegraded":"2024-02-23T00:45:14.145983+0000","last_fullsized":"2024-02-23T00:45:14.145983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.145983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"157.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:12.139981+0000","last_change":"2024-02-23T00:45:12.146933+0000","last_active":"2024-02-23T00:45:12.139981+0000","last_peered":"2024-02-23T00:45:12.139981+0000","last_clean":"2024-02-23T00:45:12.139981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:12.139981+0000","last_undegraded":"2024-02-23T00:45:12.139981+0000","last_fullsized":"2024-02-23T00:45:12.139981+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.139981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"156.17","version":"0'0","reported_seq":10,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.166984+0000","last_change":"2024-02-23T00:45:11.166695+0000","last_active":"2024-02-23T00:45:11.166984+0000","last_peered":"2024-02-23T00:45:11.166984+0000","last_clean":"2024-02-23T00:45:11.166984+0000","last_became_active":"2024-02-23T00:45:11.166395+0000","last_became_peered":"2024-02-23T00:45:11.166395+0000","last_unstale":"2024-02-23T00:45:11.166984+0000","last_undegraded":"2024-02-23T00:45:11.166984+0000","last_fullsized":"2024-02-23T00:45:11.166984+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.131641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"155.10","version":"0'0","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.153930+0000","last_change":"2024-02-23T00:45:09.155586+0000","last_active":"2024-02-23T00:45:11.153930+0000","last_peered":"2024-02-23T00:45:11.153930+0000","last_clean":"2024-02-23T00:45:11.153930+0000","last_became_active":"2024-02-23T00:45:09.155059+0000","last_became_peered":"2024-02-23T00:45:09.155059+0000","last_unstale":"2024-02-23T00:45:11.153930+0000","last_undegraded":"2024-02-23T00:45:11.153930+0000","last_fullsized":"2024-02-23T00:45:11.153930+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.124500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"158.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:14.145983+0000","last_change":"2024-02-23T00:45:14.145983+0000","last_active":"2024-02-23T00:45:14.145983+0000","last_peered":"2024-02-23T00:45:14.145983+0000","last_clean":"2024-02-23T00:45:14.145983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:14.145983+0000","last_undegraded":"2024-02-23T00:45:14.145983+0000","last_fullsized":"2024-02-23T00:45:14.145983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.145983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"157.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:12.139981+0000","last_change":"2024-02-23T00:45:12.149608+0000","last_active":"2024-02-23T00:45:12.139981+0000","last_peered":"2024-02-23T00:45:12.139981+0000","last_clean":"2024-02-23T00:45:12.139981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:12.139981+0000","last_undegraded":"2024-02-23T00:45:12.139981+0000","last_fullsized":"2024-02-23T00:45:12.139981+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.139981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"156.10","version":"0'0","reported_seq":10,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.156812+0000","last_change":"2024-02-23T00:45:11.151697+0000","last_active":"2024-02-23T00:45:11.156812+0000","last_peered":"2024-02-23T00:45:11.156812+0000","last_clean":"2024-02-23T00:45:11.156812+0000","last_became_active":"2024-02-23T00:45:11.151526+0000","last_became_peered":"2024-02-23T00:45:11.151526+0000","last_unstale":"2024-02-23T00:45:11.156812+0000","last_undegraded":"2024-02-23T00:45:11.156812+0000","last_fullsized":"2024-02-23T00:45:11.156812+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.131641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"155.17","version":"0'0","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.158426+0000","last_change":"2024-02-23T00:45:09.163534+0000","last_active":"2024-02-23T00:45:11.158426+0000","last_peered":"2024-02-23T00:45:11.158426+0000","last_clean":"2024-02-23T00:45:11.158426+0000","last_became_active":"2024-02-23T00:45:09.163278+0000","last_became_peered":"2024-02-23T00:45:09.163278+0000","last_unstale":"2024-02-23T00:45:11.158426+0000","last_undegraded":"2024-02-23T00:45:11.158426+0000","last_fullsized":"2024-02-23T00:45:11.158426+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.124500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"158.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:14.145983+0000","last_change":"2024-02-23T00:45:14.145983+0000","last_active":"2024-02-23T00:45:14.145983+0000","last_peered":"2024-02-23T00:45:14.145983+0000","last_clean":"2024-02-23T00:45:14.145983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:14.145983+0000","last_undegraded":"2024-02-23T00:45:14.145983+0000","last_fullsized":"2024-02-23T00:45:14.145983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.145983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"157.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:12.139981+0000","last_change":"2024-02-23T00:45:12.145994+0000","last_active":"2024-02-23T00:45:12.139981+0000","last_peered":"2024-02-23T00:45:12.139981+0000","last_clean":"2024-02-23T00:45:12.139981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:12.139981+0000","last_undegraded":"2024-02-23T00:45:12.139981+0000","last_fullsized":"2024-02-23T00:45:12.139981+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.139981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6,7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"156.11","version":"0'0","reported_seq":10,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.158447+0000","last_change":"2024-02-23T00:45:11.157800+0000","last_active":"2024-02-23T00:45:11.158447+0000","last_peered":"2024-02-23T00:45:11.158447+0000","last_clean":"2024-02-23T00:45:11.158447+0000","last_became_active":"2024-02-23T00:45:11.157457+0000","last_became_peered":"2024-02-23T00:45:11.157457+0000","last_unstale":"2024-02-23T00:45:11.158447+0000","last_undegraded":"2024-02-23T00:45:11.158447+0000","last_fullsized":"2024-02-23T00:45:11.158447+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.131641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"155.16","version":"0'0","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.153979+0000","last_change":"2024-02-23T00:45:09.166471+0000","last_active":"2024-02-23T00:45:11.153979+0000","last_peered":"2024-02-23T00:45:11.153979+0000","last_clean":"2024-02-23T00:45:11.153979+0000","last_became_active":"2024-02-23T00:45:09.166188+0000","last_became_peered":"2024-02-23T00:45:09.166188+0000","last_unstale":"2024-02-23T00:45:11.153979+0000","last_undegraded":"2024-02-23T00:45:11.153979+0000","last_fullsized":"2024-02-23T00:45:11.153979+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.124500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"158.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:14.145983+0000","last_change":"2024-02-23T00:45:14.145983+0000","last_active":"2024-02-23T00:45:14.145983+0000","last_peered":"2024-02-23T00:45:14.145983+0000","last_clean":"2024-02-23T00:45:14.145983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:14.145983+0000","last_undegraded":"2024-02-23T00:45:14.145983+0000","last_fullsized":"2024-02-23T00:45:14.145983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.145983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"157.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:12.139981+0000","last_change":"2024-02-23T00:45:12.148073+0000","last_active":"2024-02-23T00:45:12.139981+0000","last_peered":"2024-02-23T00:45:12.139981+0000","last_clean":"2024-02-23T00:45:12.139981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:12.139981+0000","last_undegraded":"2024-02-23T00:45:12.139981+0000","last_fullsized":"2024-02-23T00:45:12.139981+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.139981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,7],"acting":[6,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"156.12","version":"0'0","reported_seq":10,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.161815+0000","last_change":"2024-02-23T00:45:11.161196+0000","last_active":"2024-02-23T00:45:11.161815+0000","last_peered":"2024-02-23T00:45:11.161815+0000","last_clean":"2024-02-23T00:45:11.161815+0000","last_became_active":"2024-02-23T00:45:11.160894+0000","last_became_peered":"2024-02-23T00:45:11.160894+0000","last_unstale":"2024-02-23T00:45:11.161815+0000","last_undegraded":"2024-02-23T00:45:11.161815+0000","last_fullsized":"2024-02-23T00:45:11.161815+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.131641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"155.15","version":"0'0","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.159141+0000","last_change":"2024-02-23T00:45:09.152005+0000","last_active":"2024-02-23T00:45:11.159141+0000","last_peered":"2024-02-23T00:45:11.159141+0000","last_clean":"2024-02-23T00:45:11.159141+0000","last_became_active":"2024-02-23T00:45:09.151796+0000","last_became_peered":"2024-02-23T00:45:09.151796+0000","last_unstale":"2024-02-23T00:45:11.159141+0000","last_undegraded":"2024-02-23T00:45:11.159141+0000","last_fullsized":"2024-02-23T00:45:11.159141+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.124500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"158.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:14.145983+0000","last_change":"2024-02-23T00:45:14.145983+0000","last_active":"2024-02-23T00:45:14.145983+0000","last_peered":"2024-02-23T00:45:14.145983+0000","last_clean":"2024-02-23T00:45:14.145983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:14.145983+0000","last_undegraded":"2024-02-23T00:45:14.145983+0000","last_fullsized":"2024-02-23T00:45:14.145983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:14.145983+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.145983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"157.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:45:12.139981+0000","last_change":"2024-02-23T00:45:12.148040+0000","last_active":"2024-02-23T00:45:12.139981+0000","last_peered":"2024-02-23T00:45:12.139981+0000","last_clean":"2024-02-23T00:45:12.139981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:12.139981+0000","last_undegraded":"2024-02-23T00:45:12.139981+0000","last_fullsized":"2024-02-23T00:45:12.139981+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:12.139981+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.139981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"156.13","version":"0'0","reported_seq":10,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.163895+0000","last_change":"2024-02-23T00:45:11.161349+0000","last_active":"2024-02-23T00:45:11.163895+0000","last_peered":"2024-02-23T00:45:11.163895+0000","last_clean":"2024-02-23T00:45:11.163895+0000","last_became_active":"2024-02-23T00:45:11.159861+0000","last_became_peered":"2024-02-23T00:45:11.159861+0000","last_unstale":"2024-02-23T00:45:11.163895+0000","last_undegraded":"2024-02-23T00:45:11.163895+0000","last_fullsized":"2024-02-23T00:45:11.163895+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:10.131641+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.131641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"155.14","version":"0'0","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.144599+0000","last_change":"2024-02-23T00:45:09.160429+0000","last_active":"2024-02-23T00:45:11.144599+0000","last_peered":"2024-02-23T00:45:11.144599+0000","last_clean":"2024-02-23T00:45:11.144599+0000","last_became_active":"2024-02-23T00:45:09.160048+0000","last_became_peered":"2024-02-23T00:45:09.160048+0000","last_unstale":"2024-02-23T00:45:11.144599+0000","last_undegraded":"2024-02-23T00:45:11.144599+0000","last_fullsized":"2024-02-23T00:45:11.144599+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:08.124500+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.124500+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.0","version":"231'35","reported_seq":178,"reported_epoch":239,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-23T00:45:11.154206+0000","last_change":"2024-02-23T00:45:05.232069+0000","last_active":"2024-02-23T00:45:11.154206+0000","last_peered":"2024-02-23T00:45:11.154206+0000","last_clean":"2024-02-23T00:45:11.154206+0000","last_became_active":"2024-02-23T00:43:17.855316+0000","last_became_peered":"2024-02-23T00:43:17.855316+0000","last_unstale":"2024-02-23T00:45:11.154206+0000","last_undegraded":"2024-02-23T00:45:11.154206+0000","last_fullsized":"2024-02-23T00:45:11.154206+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":26,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":11,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"6","length":"2"}]},{"pgid":"62.1","version":"230'87","reported_seq":230,"reported_epoch":240,"state":"active+clean","last_fresh":"2024-02-23T00:45:12.167834+0000","last_change":"2024-02-23T00:45:12.168178+0000","last_active":"2024-02-23T00:45:12.167834+0000","last_peered":"2024-02-23T00:45:12.167834+0000","last_clean":"2024-02-23T00:45:12.167834+0000","last_became_active":"2024-02-23T00:43:17.851866+0000","last_became_peered":"2024-02-23T00:43:17.851866+0000","last_unstale":"2024-02-23T00:45:12.167834+0000","last_undegraded":"2024-02-23T00:45:12.167834+0000","last_fullsized":"2024-02-23T00:45:12.167834+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":87,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":63,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.2","version":"229'68","reported_seq":211,"reported_epoch":240,"state":"active+clean","last_fresh":"2024-02-23T00:45:12.163761+0000","last_change":"2024-02-23T00:45:12.163975+0000","last_active":"2024-02-23T00:45:12.163761+0000","last_peered":"2024-02-23T00:45:12.163761+0000","last_clean":"2024-02-23T00:45:12.163761+0000","last_became_active":"2024-02-23T00:43:17.804792+0000","last_became_peered":"2024-02-23T00:43:17.804792+0000","last_unstale":"2024-02-23T00:45:12.163761+0000","last_undegraded":"2024-02-23T00:45:12.163761+0000","last_fullsized":"2024-02-23T00:45:12.163761+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.3","version":"217'17","reported_seq":170,"reported_epoch":240,"state":"active+clean","last_fresh":"2024-02-23T00:45:12.152114+0000","last_change":"2024-02-23T00:45:12.152491+0000","last_active":"2024-02-23T00:45:12.152114+0000","last_peered":"2024-02-23T00:45:12.152114+0000","last_clean":"2024-02-23T00:45:12.152114+0000","last_became_active":"2024-02-23T00:43:17.834686+0000","last_became_peered":"2024-02-23T00:43:17.834686+0000","last_unstale":"2024-02-23T00:45:12.152114+0000","last_undegraded":"2024-02-23T00:45:12.152114+0000","last_fullsized":"2024-02-23T00:45:12.152114+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.4","version":"207'17","reported_seq":166,"reported_epoch":240,"state":"active+clean","last_fresh":"2024-02-23T00:45:12.166565+0000","last_change":"2024-02-23T00:45:12.166829+0000","last_active":"2024-02-23T00:45:12.166565+0000","last_peered":"2024-02-23T00:45:12.166565+0000","last_clean":"2024-02-23T00:45:12.166565+0000","last_became_active":"2024-02-23T00:43:17.810343+0000","last_became_peered":"2024-02-23T00:43:17.810343+0000","last_unstale":"2024-02-23T00:45:12.166565+0000","last_undegraded":"2024-02-23T00:45:12.166565+0000","last_fullsized":"2024-02-23T00:45:12.166565+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.5","version":"237'51","reported_seq":197,"reported_epoch":240,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-23T00:45:12.159301+0000","last_change":"2024-02-23T00:45:12.159301+0000","last_active":"2024-02-23T00:45:12.159301+0000","last_peered":"2024-02-23T00:45:12.159301+0000","last_clean":"2024-02-23T00:45:12.159301+0000","last_became_active":"2024-02-23T00:43:17.855339+0000","last_became_peered":"2024-02-23T00:43:17.855339+0000","last_unstale":"2024-02-23T00:45:12.159301+0000","last_undegraded":"2024-02-23T00:45:12.159301+0000","last_fullsized":"2024-02-23T00:45:12.159301+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"}]},{"pgid":"62.6","version":"234'45","reported_seq":186,"reported_epoch":239,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-23T00:45:11.158281+0000","last_change":"2024-02-23T00:45:09.236819+0000","last_active":"2024-02-23T00:45:11.158281+0000","last_peered":"2024-02-23T00:45:11.158281+0000","last_clean":"2024-02-23T00:45:11.158281+0000","last_became_active":"2024-02-23T00:43:17.838838+0000","last_became_peered":"2024-02-23T00:43:17.838838+0000","last_unstale":"2024-02-23T00:45:11.158281+0000","last_undegraded":"2024-02-23T00:45:11.158281+0000","last_fullsized":"2024-02-23T00:45:11.158281+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":45,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"stat_sum":{"num_bytes":66,"num_objects":6,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"6","length":"1"}]},{"pgid":"62.7","version":"215'68","reported_seq":201,"reported_epoch":240,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-23T00:45:12.150442+0000","last_change":"2024-02-23T00:45:12.150442+0000","last_active":"2024-02-23T00:45:12.150442+0000","last_peered":"2024-02-23T00:45:12.150442+0000","last_clean":"2024-02-23T00:45:12.150442+0000","last_became_active":"2024-02-23T00:43:17.846548+0000","last_became_peered":"2024-02-23T00:43:17.846548+0000","last_unstale":"2024-02-23T00:45:12.150442+0000","last_undegraded":"2024-02-23T00:45:12.150442+0000","last_fullsized":"2024-02-23T00:45:12.150442+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"}]},{"pgid":"62.8","version":"239'83","reported_seq":212,"reported_epoch":239,"state":"active+clean+snaptrim","last_fresh":"2024-02-23T00:45:11.240158+0000","last_change":"2024-02-23T00:45:09.236848+0000","last_active":"2024-02-23T00:45:11.240158+0000","last_peered":"2024-02-23T00:45:11.240158+0000","last_clean":"2024-02-23T00:45:11.240158+0000","last_became_active":"2024-02-23T00:43:17.842613+0000","last_became_peered":"2024-02-23T00:43:17.842613+0000","last_unstale":"2024-02-23T00:45:11.240158+0000","last_undegraded":"2024-02-23T00:45:11.240158+0000","last_fullsized":"2024-02-23T00:45:11.240158+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":83,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"stat_sum":{"num_bytes":66,"num_objects":6,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":25,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"7","length":"1"}]},{"pgid":"62.9","version":"225'34","reported_seq":184,"reported_epoch":240,"state":"active+clean","last_fresh":"2024-02-23T00:45:12.148060+0000","last_change":"2024-02-23T00:45:12.148368+0000","last_active":"2024-02-23T00:45:12.148060+0000","last_peered":"2024-02-23T00:45:12.148060+0000","last_clean":"2024-02-23T00:45:12.148060+0000","last_became_active":"2024-02-23T00:43:17.834297+0000","last_became_peered":"2024-02-23T00:43:17.834297+0000","last_unstale":"2024-02-23T00:45:12.148060+0000","last_undegraded":"2024-02-23T00:45:12.148060+0000","last_fullsized":"2024-02-23T00:45:12.148060+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.a","version":"237'78","reported_seq":208,"reported_epoch":239,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-23T00:45:11.254174+0000","last_change":"2024-02-23T00:45:11.254174+0000","last_active":"2024-02-23T00:45:11.254174+0000","last_peered":"2024-02-23T00:45:11.254174+0000","last_clean":"2024-02-23T00:45:11.254174+0000","last_became_active":"2024-02-23T00:43:17.839142+0000","last_became_peered":"2024-02-23T00:43:17.839142+0000","last_unstale":"2024-02-23T00:45:11.254174+0000","last_undegraded":"2024-02-23T00:45:11.254174+0000","last_fullsized":"2024-02-23T00:45:11.254174+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":78,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"stat_sum":{"num_bytes":123,"num_objects":12,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"6","length":"1"}]},{"pgid":"62.b","version":"229'60","reported_seq":190,"reported_epoch":239,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-23T00:45:11.155906+0000","last_change":"2024-02-23T00:45:03.230227+0000","last_active":"2024-02-23T00:45:11.155906+0000","last_peered":"2024-02-23T00:45:11.155906+0000","last_clean":"2024-02-23T00:45:11.155906+0000","last_became_active":"2024-02-23T00:43:17.855321+0000","last_became_peered":"2024-02-23T00:43:17.855321+0000","last_unstale":"2024-02-23T00:45:11.155906+0000","last_undegraded":"2024-02-23T00:45:11.155906+0000","last_fullsized":"2024-02-23T00:45:11.155906+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":60,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"stat_sum":{"num_bytes":88,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"6","length":"1"}]},{"pgid":"62.c","version":"203'34","reported_seq":182,"reported_epoch":240,"state":"active+clean","last_fresh":"2024-02-23T00:45:12.156375+0000","last_change":"2024-02-23T00:45:12.157091+0000","last_active":"2024-02-23T00:45:12.156375+0000","last_peered":"2024-02-23T00:45:12.156375+0000","last_clean":"2024-02-23T00:45:12.156375+0000","last_became_active":"2024-02-23T00:43:17.829044+0000","last_became_peered":"2024-02-23T00:43:17.829044+0000","last_unstale":"2024-02-23T00:45:12.156375+0000","last_undegraded":"2024-02-23T00:45:12.156375+0000","last_fullsized":"2024-02-23T00:45:12.156375+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.d","version":"233'51","reported_seq":190,"reported_epoch":240,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-23T00:45:12.159141+0000","last_change":"2024-02-23T00:45:12.159141+0000","last_active":"2024-02-23T00:45:12.159141+0000","last_peered":"2024-02-23T00:45:12.159141+0000","last_clean":"2024-02-23T00:45:12.159141+0000","last_became_active":"2024-02-23T00:43:17.841614+0000","last_became_peered":"2024-02-23T00:43:17.841614+0000","last_unstale":"2024-02-23T00:45:12.159141+0000","last_undegraded":"2024-02-23T00:45:12.159141+0000","last_fullsized":"2024-02-23T00:45:12.159141+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"}]},{"pgid":"62.e","version":"211'51","reported_seq":193,"reported_epoch":240,"state":"active+clean","last_fresh":"2024-02-23T00:45:12.165816+0000","last_change":"2024-02-23T00:45:12.166782+0000","last_active":"2024-02-23T00:45:12.165816+0000","last_peered":"2024-02-23T00:45:12.165816+0000","last_clean":"2024-02-23T00:45:12.165816+0000","last_became_active":"2024-02-23T00:43:17.845956+0000","last_became_peered":"2024-02-23T00:43:17.845956+0000","last_unstale":"2024-02-23T00:45:12.165816+0000","last_undegraded":"2024-02-23T00:45:12.165816+0000","last_fullsized":"2024-02-23T00:45:12.165816+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.f","version":"239'132","reported_seq":251,"reported_epoch":239,"state":"active+clean+snaptrim","last_fresh":"2024-02-23T00:45:11.250938+0000","last_change":"2024-02-23T00:45:07.245123+0000","last_active":"2024-02-23T00:45:11.250938+0000","last_peered":"2024-02-23T00:45:11.250938+0000","last_clean":"2024-02-23T00:45:11.250938+0000","last_became_active":"2024-02-23T00:43:17.855331+0000","last_became_peered":"2024-02-23T00:43:17.855331+0000","last_unstale":"2024-02-23T00:45:11.250938+0000","last_undegraded":"2024-02-23T00:45:11.250938+0000","last_fullsized":"2024-02-23T00:45:11.250938+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":132,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":40,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6","length":"1"}]},{"pgid":"62.10","version":"219'68","reported_seq":211,"reported_epoch":240,"state":"active+clean","last_fresh":"2024-02-23T00:45:12.168448+0000","last_change":"2024-02-23T00:45:12.168642+0000","last_active":"2024-02-23T00:45:12.168448+0000","last_peered":"2024-02-23T00:45:12.168448+0000","last_clean":"2024-02-23T00:45:12.168448+0000","last_became_active":"2024-02-23T00:43:17.819865+0000","last_became_peered":"2024-02-23T00:43:17.819865+0000","last_unstale":"2024-02-23T00:45:12.168448+0000","last_undegraded":"2024-02-23T00:45:12.168448+0000","last_fullsized":"2024-02-23T00:45:12.168448+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.11","version":"215'51","reported_seq":191,"reported_epoch":240,"state":"active+clean","last_fresh":"2024-02-23T00:45:12.167308+0000","last_change":"2024-02-23T00:45:12.167995+0000","last_active":"2024-02-23T00:45:12.167308+0000","last_peered":"2024-02-23T00:45:12.167308+0000","last_clean":"2024-02-23T00:45:12.167308+0000","last_became_active":"2024-02-23T00:43:17.852697+0000","last_became_peered":"2024-02-23T00:43:17.852697+0000","last_unstale":"2024-02-23T00:45:12.167308+0000","last_undegraded":"2024-02-23T00:45:12.167308+0000","last_fullsized":"2024-02-23T00:45:12.167308+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.12","version":"240'73","reported_seq":216,"reported_epoch":240,"state":"active+clean+snaptrim","last_fresh":"2024-02-23T00:45:12.152269+0000","last_change":"2024-02-23T00:45:12.152269+0000","last_active":"2024-02-23T00:45:12.152269+0000","last_peered":"2024-02-23T00:45:12.152269+0000","last_clean":"2024-02-23T00:45:12.152269+0000","last_became_active":"2024-02-23T00:43:17.856118+0000","last_became_peered":"2024-02-23T00:43:17.856118+0000","last_unstale":"2024-02-23T00:45:12.152269+0000","last_undegraded":"2024-02-23T00:45:12.152269+0000","last_fullsized":"2024-02-23T00:45:12.152269+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":73,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":49,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"}]},{"pgid":"62.13","version":"205'34","reported_seq":183,"reported_epoch":240,"state":"active+clean","last_fresh":"2024-02-23T00:45:12.161869+0000","last_change":"2024-02-23T00:45:12.162228+0000","last_active":"2024-02-23T00:45:12.161869+0000","last_peered":"2024-02-23T00:45:12.161869+0000","last_clean":"2024-02-23T00:45:12.161869+0000","last_became_active":"2024-02-23T00:43:17.857823+0000","last_became_peered":"2024-02-23T00:43:17.857823+0000","last_unstale":"2024-02-23T00:45:12.161869+0000","last_undegraded":"2024-02-23T00:45:12.161869+0000","last_fullsized":"2024-02-23T00:45:12.161869+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.14","version":"239'39","reported_seq":175,"reported_epoch":239,"state":"active+clean+snaptrim","last_fresh":"2024-02-23T00:45:11.255746+0000","last_change":"2024-02-23T00:45:11.254206+0000","last_active":"2024-02-23T00:45:11.255746+0000","last_peered":"2024-02-23T00:45:11.255746+0000","last_clean":"2024-02-23T00:45:11.255746+0000","last_became_active":"2024-02-23T00:43:17.816138+0000","last_became_peered":"2024-02-23T00:43:17.816138+0000","last_unstale":"2024-02-23T00:45:11.255746+0000","last_undegraded":"2024-02-23T00:45:11.255746+0000","last_fullsized":"2024-02-23T00:45:11.255746+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"stat_sum":{"num_bytes":27,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":12,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"6","length":"1"}]},{"pgid":"62.15","version":"201'17","reported_seq":169,"reported_epoch":240,"state":"active+clean","last_fresh":"2024-02-23T00:45:12.157182+0000","last_change":"2024-02-23T00:45:12.157376+0000","last_active":"2024-02-23T00:45:12.157182+0000","last_peered":"2024-02-23T00:45:12.157182+0000","last_clean":"2024-02-23T00:45:12.157182+0000","last_became_active":"2024-02-23T00:43:17.843834+0000","last_became_peered":"2024-02-23T00:43:17.843834+0000","last_unstale":"2024-02-23T00:45:12.157182+0000","last_undegraded":"2024-02-23T00:45:12.157182+0000","last_fullsized":"2024-02-23T00:45:12.157182+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.16","version":"236'21","reported_seq":172,"reported_epoch":240,"state":"active+clean+snaptrim","last_fresh":"2024-02-23T00:45:12.158747+0000","last_change":"2024-02-23T00:45:12.161103+0000","last_active":"2024-02-23T00:45:12.158747+0000","last_peered":"2024-02-23T00:45:12.158747+0000","last_clean":"2024-02-23T00:45:12.158747+0000","last_became_active":"2024-02-23T00:43:17.834291+0000","last_became_peered":"2024-02-23T00:43:17.834291+0000","last_unstale":"2024-02-23T00:45:12.158747+0000","last_undegraded":"2024-02-23T00:45:12.158747+0000","last_fullsized":"2024-02-23T00:45:12.158747+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"}]},{"pgid":"62.17","version":"229'57","reported_seq":209,"reported_epoch":240,"state":"active+clean","last_fresh":"2024-02-23T00:45:12.165157+0000","last_change":"2024-02-23T00:45:12.165364+0000","last_active":"2024-02-23T00:45:12.165157+0000","last_peered":"2024-02-23T00:45:12.165157+0000","last_clean":"2024-02-23T00:45:12.165157+0000","last_became_active":"2024-02-23T00:43:17.843525+0000","last_became_peered":"2024-02-23T00:43:17.843525+0000","last_unstale":"2024-02-23T00:45:12.165157+0000","last_undegraded":"2024-02-23T00:45:12.165157+0000","last_fullsized":"2024-02-23T00:45:12.165157+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":57,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":18,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.19","version":"236'93","reported_seq":239,"reported_epoch":240,"state":"active+clean+snaptrim","last_fresh":"2024-02-23T00:45:12.161517+0000","last_change":"2024-02-23T00:45:12.161551+0000","last_active":"2024-02-23T00:45:12.161517+0000","last_peered":"2024-02-23T00:45:12.161517+0000","last_clean":"2024-02-23T00:45:12.161517+0000","last_became_active":"2024-02-23T00:43:17.815830+0000","last_became_peered":"2024-02-23T00:43:17.815830+0000","last_unstale":"2024-02-23T00:45:12.161517+0000","last_undegraded":"2024-02-23T00:45:12.161517+0000","last_fullsized":"2024-02-23T00:45:12.161517+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":93,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":73,"num_objects":8,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":7,"num_read_kb":0,"num_write":29,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"}]},{"pgid":"62.18","version":"150'5","reported_seq":160,"reported_epoch":240,"state":"active+clean","last_fresh":"2024-02-23T00:45:12.154839+0000","last_change":"2024-02-23T00:45:12.155143+0000","last_active":"2024-02-23T00:45:12.154839+0000","last_peered":"2024-02-23T00:45:12.154839+0000","last_clean":"2024-02-23T00:45:12.154839+0000","last_became_active":"2024-02-23T00:43:17.840351+0000","last_became_peered":"2024-02-23T00:43:17.840351+0000","last_unstale":"2024-02-23T00:45:12.154839+0000","last_undegraded":"2024-02-23T00:45:12.154839+0000","last_fullsized":"2024-02-23T00:45:12.154839+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.1b","version":"229'68","reported_seq":210,"reported_epoch":240,"state":"active+clean","last_fresh":"2024-02-23T00:45:12.157856+0000","last_change":"2024-02-23T00:45:12.158035+0000","last_active":"2024-02-23T00:45:12.157856+0000","last_peered":"2024-02-23T00:45:12.157856+0000","last_clean":"2024-02-23T00:45:12.157856+0000","last_became_active":"2024-02-23T00:43:17.803043+0000","last_became_peered":"2024-02-23T00:43:17.803043+0000","last_unstale":"2024-02-23T00:45:12.157856+0000","last_undegraded":"2024-02-23T00:45:12.157856+0000","last_fullsized":"2024-02-23T00:45:12.157856+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.1a","version":"225'34","reported_seq":180,"reported_epoch":240,"state":"active+clean","last_fresh":"2024-02-23T00:45:12.162835+0000","last_change":"2024-02-23T00:45:12.163192+0000","last_active":"2024-02-23T00:45:12.162835+0000","last_peered":"2024-02-23T00:45:12.162835+0000","last_clean":"2024-02-23T00:45:12.162835+0000","last_became_active":"2024-02-23T00:43:17.798285+0000","last_became_peered":"2024-02-23T00:43:17.798285+0000","last_unstale":"2024-02-23T00:45:12.162835+0000","last_undegraded":"2024-02-23T00:45:12.162835+0000","last_fullsized":"2024-02-23T00:45:12.162835+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.1d","version":"215'104","reported_seq":244,"reported_epoch":240,"state":"active+clean","last_fresh":"2024-02-23T00:45:12.161000+0000","last_change":"2024-02-23T00:45:12.161200+0000","last_active":"2024-02-23T00:45:12.161000+0000","last_peered":"2024-02-23T00:45:12.161000+0000","last_clean":"2024-02-23T00:45:12.161000+0000","last_became_active":"2024-02-23T00:43:17.828194+0000","last_became_peered":"2024-02-23T00:43:17.828194+0000","last_unstale":"2024-02-23T00:45:12.161000+0000","last_undegraded":"2024-02-23T00:45:12.161000+0000","last_fullsized":"2024-02-23T00:45:12.161000+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":104,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":74,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":32,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.1c","version":"223'51","reported_seq":194,"reported_epoch":240,"state":"active+clean","last_fresh":"2024-02-23T00:45:12.164017+0000","last_change":"2024-02-23T00:45:12.164182+0000","last_active":"2024-02-23T00:45:12.164017+0000","last_peered":"2024-02-23T00:45:12.164017+0000","last_clean":"2024-02-23T00:45:12.164017+0000","last_became_active":"2024-02-23T00:43:17.804718+0000","last_became_peered":"2024-02-23T00:43:17.804718+0000","last_unstale":"2024-02-23T00:45:12.164017+0000","last_undegraded":"2024-02-23T00:45:12.164017+0000","last_fullsized":"2024-02-23T00:45:12.164017+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.1f","version":"236'38","reported_seq":188,"reported_epoch":240,"state":"active+clean+snaptrim","last_fresh":"2024-02-23T00:45:12.154028+0000","last_change":"2024-02-23T00:45:12.154060+0000","last_active":"2024-02-23T00:45:12.154028+0000","last_peered":"2024-02-23T00:45:12.154028+0000","last_clean":"2024-02-23T00:45:12.154028+0000","last_became_active":"2024-02-23T00:43:17.839914+0000","last_became_peered":"2024-02-23T00:43:17.839914+0000","last_unstale":"2024-02-23T00:45:12.154028+0000","last_undegraded":"2024-02-23T00:45:12.154028+0000","last_fullsized":"2024-02-23T00:45:12.154028+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":38,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":35,"num_objects":5,"num_object_clones":1,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":13,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"}]},{"pgid":"62.1e","version":"231'45","reported_seq":179,"reported_epoch":239,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-23T00:45:11.158031+0000","last_change":"2024-02-23T00:45:05.243868+0000","last_active":"2024-02-23T00:45:11.158031+0000","last_peered":"2024-02-23T00:45:11.158031+0000","last_clean":"2024-02-23T00:45:11.158031+0000","last_became_active":"2024-02-23T00:43:17.838682+0000","last_became_peered":"2024-02-23T00:43:17.838682+0000","last_unstale":"2024-02-23T00:45:11.158031+0000","last_undegraded":"2024-02-23T00:45:11.158031+0000","last_fullsized":"2024-02-23T00:45:11.158031+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":45,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"stat_sum":{"num_bytes":66,"num_objects":6,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"6","length":"1"}]},{"pgid":"5.9","version":"67'8","reported_seq":275,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.155039+0000","last_change":"2024-02-23T00:42:47.985647+0000","last_active":"2024-02-23T00:45:11.155039+0000","last_peered":"2024-02-23T00:45:11.155039+0000","last_clean":"2024-02-23T00:45:11.155039+0000","last_became_active":"2024-02-23T00:39:18.793414+0000","last_became_peered":"2024-02-23T00:39:18.793414+0000","last_unstale":"2024-02-23T00:45:11.155039+0000","last_undegraded":"2024-02-23T00:45:11.155039+0000","last_fullsized":"2024-02-23T00:45:11.155039+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:42:47.985573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:47.985573+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"68'27","reported_seq":257,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.160010+0000","last_change":"2024-02-23T00:45:04.138883+0000","last_active":"2024-02-23T00:45:11.160010+0000","last_peered":"2024-02-23T00:45:11.160010+0000","last_clean":"2024-02-23T00:45:11.160010+0000","last_became_active":"2024-02-23T00:39:16.859401+0000","last_became_peered":"2024-02-23T00:39:16.859401+0000","last_unstale":"2024-02-23T00:45:11.160010+0000","last_undegraded":"2024-02-23T00:45:11.160010+0000","last_fullsized":"2024-02-23T00:45:11.160010+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'27","last_scrub_stamp":"2024-02-23T00:45:04.138751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.138751+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":203,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.154697+0000","last_change":"2024-02-23T00:43:42.320439+0000","last_active":"2024-02-23T00:45:11.154697+0000","last_peered":"2024-02-23T00:45:11.154697+0000","last_clean":"2024-02-23T00:45:11.154697+0000","last_became_active":"2024-02-23T00:39:14.784436+0000","last_became_peered":"2024-02-23T00:39:14.784436+0000","last_unstale":"2024-02-23T00:45:11.154697+0000","last_undegraded":"2024-02-23T00:45:11.154697+0000","last_fullsized":"2024-02-23T00:45:11.154697+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:42.320346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:04.330426+0000","last_clean_scrub_stamp":"2024-02-23T00:43:42.320346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":216,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.159285+0000","last_change":"2024-02-23T00:44:21.972446+0000","last_active":"2024-02-23T00:45:11.159285+0000","last_peered":"2024-02-23T00:45:11.159285+0000","last_clean":"2024-02-23T00:45:11.159285+0000","last_became_active":"2024-02-23T00:43:32.282545+0000","last_became_peered":"2024-02-23T00:43:32.282545+0000","last_unstale":"2024-02-23T00:45:11.159285+0000","last_undegraded":"2024-02-23T00:45:11.159285+0000","last_fullsized":"2024-02-23T00:45:11.159285+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:21.972383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:21.972383+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.3","version":"110'5","reported_seq":145,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.142763+0000","last_change":"2024-02-23T00:42:58.836257+0000","last_active":"2024-02-23T00:45:11.142763+0000","last_peered":"2024-02-23T00:45:11.142763+0000","last_clean":"2024-02-23T00:45:11.142763+0000","last_became_active":"2024-02-23T00:42:58.835612+0000","last_became_peered":"2024-02-23T00:42:58.835612+0000","last_unstale":"2024-02-23T00:45:11.142763+0000","last_undegraded":"2024-02-23T00:45:11.142763+0000","last_fullsized":"2024-02-23T00:45:11.142763+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.160762+0000","last_change":"2024-02-23T00:42:58.832536+0000","last_active":"2024-02-23T00:45:11.160762+0000","last_peered":"2024-02-23T00:45:11.160762+0000","last_clean":"2024-02-23T00:45:11.160762+0000","last_became_active":"2024-02-23T00:42:58.828846+0000","last_became_peered":"2024-02-23T00:42:58.828846+0000","last_unstale":"2024-02-23T00:45:11.160762+0000","last_undegraded":"2024-02-23T00:45:11.160762+0000","last_fullsized":"2024-02-23T00:45:11.160762+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"68'15","reported_seq":233,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.146008+0000","last_change":"2024-02-23T00:43:54.355050+0000","last_active":"2024-02-23T00:45:11.146008+0000","last_peered":"2024-02-23T00:45:11.146008+0000","last_clean":"2024-02-23T00:45:11.146008+0000","last_became_active":"2024-02-23T00:39:16.783701+0000","last_became_peered":"2024-02-23T00:39:16.783701+0000","last_unstale":"2024-02-23T00:45:11.146008+0000","last_undegraded":"2024-02-23T00:45:11.146008+0000","last_fullsized":"2024-02-23T00:45:11.146008+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:43:54.354974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:54.354974+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":198,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.159639+0000","last_change":"2024-02-23T00:42:48.591600+0000","last_active":"2024-02-23T00:45:11.159639+0000","last_peered":"2024-02-23T00:45:11.159639+0000","last_clean":"2024-02-23T00:45:11.159639+0000","last_became_active":"2024-02-23T00:39:14.772493+0000","last_became_peered":"2024-02-23T00:39:14.772493+0000","last_unstale":"2024-02-23T00:45:11.159639+0000","last_undegraded":"2024-02-23T00:45:11.159639+0000","last_fullsized":"2024-02-23T00:45:11.159639+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:48.591486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:48.591486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":199,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.149558+0000","last_change":"2024-02-23T00:44:22.380954+0000","last_active":"2024-02-23T00:45:11.149558+0000","last_peered":"2024-02-23T00:45:11.149558+0000","last_clean":"2024-02-23T00:45:11.149558+0000","last_became_active":"2024-02-23T00:39:18.794476+0000","last_became_peered":"2024-02-23T00:39:18.794476+0000","last_unstale":"2024-02-23T00:45:11.149558+0000","last_undegraded":"2024-02-23T00:45:11.149558+0000","last_fullsized":"2024-02-23T00:45:11.149558+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:22.380841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:08.335330+0000","last_clean_scrub_stamp":"2024-02-23T00:44:22.380841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":211,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.144800+0000","last_change":"2024-02-23T00:43:35.079353+0000","last_active":"2024-02-23T00:45:11.144800+0000","last_peered":"2024-02-23T00:45:11.144800+0000","last_clean":"2024-02-23T00:45:11.144800+0000","last_became_active":"2024-02-23T00:43:35.079004+0000","last_became_peered":"2024-02-23T00:43:35.079004+0000","last_unstale":"2024-02-23T00:45:11.144800+0000","last_undegraded":"2024-02-23T00:45:11.144800+0000","last_fullsized":"2024-02-23T00:45:11.144800+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:46.359646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:46.359646+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"110'5","reported_seq":150,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.149035+0000","last_change":"2024-02-23T00:45:06.483322+0000","last_active":"2024-02-23T00:45:11.149035+0000","last_peered":"2024-02-23T00:45:11.149035+0000","last_clean":"2024-02-23T00:45:11.149035+0000","last_became_active":"2024-02-23T00:42:58.741558+0000","last_became_peered":"2024-02-23T00:42:58.741558+0000","last_unstale":"2024-02-23T00:45:11.149035+0000","last_undegraded":"2024-02-23T00:45:11.149035+0000","last_fullsized":"2024-02-23T00:45:11.149035+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'5","last_scrub_stamp":"2024-02-23T00:45:06.483252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.483252+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.145680+0000","last_change":"2024-02-23T00:42:58.871003+0000","last_active":"2024-02-23T00:45:11.145680+0000","last_peered":"2024-02-23T00:45:11.145680+0000","last_clean":"2024-02-23T00:45:11.145680+0000","last_became_active":"2024-02-23T00:42:58.867305+0000","last_became_peered":"2024-02-23T00:42:58.867305+0000","last_unstale":"2024-02-23T00:45:11.145680+0000","last_undegraded":"2024-02-23T00:45:11.145680+0000","last_fullsized":"2024-02-23T00:45:11.145680+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"88'35","reported_seq":257,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.166663+0000","last_change":"2024-02-23T00:43:54.438282+0000","last_active":"2024-02-23T00:45:11.166663+0000","last_peered":"2024-02-23T00:45:11.166663+0000","last_clean":"2024-02-23T00:45:11.166663+0000","last_became_active":"2024-02-23T00:39:16.857413+0000","last_became_peered":"2024-02-23T00:39:16.857413+0000","last_unstale":"2024-02-23T00:45:11.166663+0000","last_undegraded":"2024-02-23T00:45:11.166663+0000","last_fullsized":"2024-02-23T00:45:11.166663+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'35","last_scrub_stamp":"2024-02-23T00:43:54.438168+0000","last_deep_scrub":"88'35","last_deep_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_clean_scrub_stamp":"2024-02-23T00:43:54.438168+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":203,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.153294+0000","last_change":"2024-02-23T00:43:53.317283+0000","last_active":"2024-02-23T00:45:11.153294+0000","last_peered":"2024-02-23T00:45:11.153294+0000","last_clean":"2024-02-23T00:45:11.153294+0000","last_became_active":"2024-02-23T00:39:14.776705+0000","last_became_peered":"2024-02-23T00:39:14.776705+0000","last_unstale":"2024-02-23T00:45:11.153294+0000","last_undegraded":"2024-02-23T00:45:11.153294+0000","last_fullsized":"2024-02-23T00:45:11.153294+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:53.317202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_clean_scrub_stamp":"2024-02-23T00:43:53.317202+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":199,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.153447+0000","last_change":"2024-02-23T00:43:44.313672+0000","last_active":"2024-02-23T00:45:11.153447+0000","last_peered":"2024-02-23T00:45:11.153447+0000","last_clean":"2024-02-23T00:45:11.153447+0000","last_became_active":"2024-02-23T00:39:18.792718+0000","last_became_peered":"2024-02-23T00:39:18.792718+0000","last_unstale":"2024-02-23T00:45:11.153447+0000","last_undegraded":"2024-02-23T00:45:11.153447+0000","last_fullsized":"2024-02-23T00:45:11.153447+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:44.313501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:44.313501+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":214,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.154261+0000","last_change":"2024-02-23T00:44:36.758043+0000","last_active":"2024-02-23T00:45:11.154261+0000","last_peered":"2024-02-23T00:45:11.154261+0000","last_clean":"2024-02-23T00:45:11.154261+0000","last_became_active":"2024-02-23T00:43:39.470999+0000","last_became_peered":"2024-02-23T00:43:39.470999+0000","last_unstale":"2024-02-23T00:45:11.154261+0000","last_undegraded":"2024-02-23T00:45:11.154261+0000","last_fullsized":"2024-02-23T00:45:11.154261+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:36.757960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:36.757960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"110'5","reported_seq":145,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.161470+0000","last_change":"2024-02-23T00:42:58.688988+0000","last_active":"2024-02-23T00:45:11.161470+0000","last_peered":"2024-02-23T00:45:11.161470+0000","last_clean":"2024-02-23T00:45:11.161470+0000","last_became_active":"2024-02-23T00:42:58.684001+0000","last_became_peered":"2024-02-23T00:42:58.684001+0000","last_unstale":"2024-02-23T00:45:11.161470+0000","last_undegraded":"2024-02-23T00:45:11.161470+0000","last_fullsized":"2024-02-23T00:45:11.161470+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.157935+0000","last_change":"2024-02-23T00:42:57.641727+0000","last_active":"2024-02-23T00:45:11.157935+0000","last_peered":"2024-02-23T00:45:11.157935+0000","last_clean":"2024-02-23T00:45:11.157935+0000","last_became_active":"2024-02-23T00:42:57.639013+0000","last_became_peered":"2024-02-23T00:42:57.639013+0000","last_unstale":"2024-02-23T00:45:11.157935+0000","last_undegraded":"2024-02-23T00:45:11.157935+0000","last_fullsized":"2024-02-23T00:45:11.157935+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"67'17","reported_seq":251,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.144143+0000","last_change":"2024-02-23T00:43:45.194207+0000","last_active":"2024-02-23T00:45:11.144143+0000","last_peered":"2024-02-23T00:45:11.144143+0000","last_clean":"2024-02-23T00:45:11.144143+0000","last_became_active":"2024-02-23T00:40:42.933806+0000","last_became_peered":"2024-02-23T00:40:42.933806+0000","last_unstale":"2024-02-23T00:45:11.144143+0000","last_undegraded":"2024-02-23T00:45:11.144143+0000","last_fullsized":"2024-02-23T00:45:11.144143+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'17","last_scrub_stamp":"2024-02-23T00:43:45.194139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.194139+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":174,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.151336+0000","last_change":"2024-02-23T00:43:31.362599+0000","last_active":"2024-02-23T00:45:11.151336+0000","last_peered":"2024-02-23T00:45:11.151336+0000","last_clean":"2024-02-23T00:45:11.151336+0000","last_became_active":"2024-02-23T00:40:42.931041+0000","last_became_peered":"2024-02-23T00:40:42.931041+0000","last_unstale":"2024-02-23T00:45:11.151336+0000","last_undegraded":"2024-02-23T00:45:11.151336+0000","last_fullsized":"2024-02-23T00:45:11.151336+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:31.362510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:31.362510+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":199,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.158438+0000","last_change":"2024-02-23T00:44:22.804352+0000","last_active":"2024-02-23T00:45:11.158438+0000","last_peered":"2024-02-23T00:45:11.158438+0000","last_clean":"2024-02-23T00:45:11.158438+0000","last_became_active":"2024-02-23T00:39:18.799875+0000","last_became_peered":"2024-02-23T00:39:18.799875+0000","last_unstale":"2024-02-23T00:45:11.158438+0000","last_undegraded":"2024-02-23T00:45:11.158438+0000","last_fullsized":"2024-02-23T00:45:11.158438+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:22.804279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:22.804279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":215,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.155037+0000","last_change":"2024-02-23T00:44:36.481327+0000","last_active":"2024-02-23T00:45:11.155037+0000","last_peered":"2024-02-23T00:45:11.155037+0000","last_clean":"2024-02-23T00:45:11.155037+0000","last_became_active":"2024-02-23T00:43:08.885257+0000","last_became_peered":"2024-02-23T00:43:08.885257+0000","last_unstale":"2024-02-23T00:45:11.155037+0000","last_undegraded":"2024-02-23T00:45:11.155037+0000","last_fullsized":"2024-02-23T00:45:11.155037+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:36.481256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:36.481256+0000","last_clean_scrub_stamp":"2024-02-23T00:44:36.481256+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.6","version":"110'9","reported_seq":149,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.150214+0000","last_change":"2024-02-23T00:42:58.871934+0000","last_active":"2024-02-23T00:45:11.150214+0000","last_peered":"2024-02-23T00:45:11.150214+0000","last_clean":"2024-02-23T00:45:11.150214+0000","last_became_active":"2024-02-23T00:42:58.870147+0000","last_became_peered":"2024-02-23T00:42:58.870147+0000","last_unstale":"2024-02-23T00:45:11.150214+0000","last_undegraded":"2024-02-23T00:45:11.150214+0000","last_fullsized":"2024-02-23T00:45:11.150214+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.156564+0000","last_change":"2024-02-23T00:42:57.683188+0000","last_active":"2024-02-23T00:45:11.156564+0000","last_peered":"2024-02-23T00:45:11.156564+0000","last_clean":"2024-02-23T00:45:11.156564+0000","last_became_active":"2024-02-23T00:42:57.675306+0000","last_became_peered":"2024-02-23T00:42:57.675306+0000","last_unstale":"2024-02-23T00:45:11.156564+0000","last_undegraded":"2024-02-23T00:45:11.156564+0000","last_fullsized":"2024-02-23T00:45:11.156564+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"67'10","reported_seq":211,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.143914+0000","last_change":"2024-02-23T00:42:50.078150+0000","last_active":"2024-02-23T00:45:11.143914+0000","last_peered":"2024-02-23T00:45:11.143914+0000","last_clean":"2024-02-23T00:45:11.143914+0000","last_became_active":"2024-02-23T00:39:16.777530+0000","last_became_peered":"2024-02-23T00:39:16.777530+0000","last_unstale":"2024-02-23T00:45:11.143914+0000","last_undegraded":"2024-02-23T00:45:11.143914+0000","last_fullsized":"2024-02-23T00:45:11.143914+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'10","last_scrub_stamp":"2024-02-23T00:42:50.078042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:50.078042+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":203,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.159847+0000","last_change":"2024-02-23T00:45:10.693332+0000","last_active":"2024-02-23T00:45:11.159847+0000","last_peered":"2024-02-23T00:45:11.159847+0000","last_clean":"2024-02-23T00:45:11.159847+0000","last_became_active":"2024-02-23T00:39:14.780379+0000","last_became_peered":"2024-02-23T00:39:14.780379+0000","last_unstale":"2024-02-23T00:45:11.159847+0000","last_undegraded":"2024-02-23T00:45:11.159847+0000","last_fullsized":"2024-02-23T00:45:11.159847+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.693273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.575242+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.693273+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"67'8","reported_seq":278,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.153731+0000","last_change":"2024-02-23T00:44:25.470245+0000","last_active":"2024-02-23T00:45:11.153731+0000","last_peered":"2024-02-23T00:45:11.153731+0000","last_clean":"2024-02-23T00:45:11.153731+0000","last_became_active":"2024-02-23T00:39:18.799544+0000","last_became_peered":"2024-02-23T00:39:18.799544+0000","last_unstale":"2024-02-23T00:45:11.153731+0000","last_undegraded":"2024-02-23T00:45:11.153731+0000","last_fullsized":"2024-02-23T00:45:11.153731+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:44:25.470174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:25.470174+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":212,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.144313+0000","last_change":"2024-02-23T00:43:36.205958+0000","last_active":"2024-02-23T00:45:11.144313+0000","last_peered":"2024-02-23T00:45:11.144313+0000","last_clean":"2024-02-23T00:45:11.144313+0000","last_became_active":"2024-02-23T00:43:13.338697+0000","last_became_peered":"2024-02-23T00:43:13.338697+0000","last_unstale":"2024-02-23T00:45:11.144313+0000","last_undegraded":"2024-02-23T00:45:11.144313+0000","last_fullsized":"2024-02-23T00:45:11.144313+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:36.205886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:15.081153+0000","last_clean_scrub_stamp":"2024-02-23T00:43:36.205886+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.158737+0000","last_change":"2024-02-23T00:42:57.686234+0000","last_active":"2024-02-23T00:45:11.158737+0000","last_peered":"2024-02-23T00:45:11.158737+0000","last_clean":"2024-02-23T00:45:11.158737+0000","last_became_active":"2024-02-23T00:42:57.685579+0000","last_became_peered":"2024-02-23T00:42:57.685579+0000","last_unstale":"2024-02-23T00:45:11.158737+0000","last_undegraded":"2024-02-23T00:45:11.158737+0000","last_fullsized":"2024-02-23T00:45:11.158737+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.145258+0000","last_change":"2024-02-23T00:42:58.772582+0000","last_active":"2024-02-23T00:45:11.145258+0000","last_peered":"2024-02-23T00:45:11.145258+0000","last_clean":"2024-02-23T00:45:11.145258+0000","last_became_active":"2024-02-23T00:42:58.765106+0000","last_became_peered":"2024-02-23T00:42:58.765106+0000","last_unstale":"2024-02-23T00:45:11.145258+0000","last_undegraded":"2024-02-23T00:45:11.145258+0000","last_fullsized":"2024-02-23T00:45:11.145258+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"67'15","reported_seq":226,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.158943+0000","last_change":"2024-02-23T00:44:19.830383+0000","last_active":"2024-02-23T00:45:11.158943+0000","last_peered":"2024-02-23T00:45:11.158943+0000","last_clean":"2024-02-23T00:45:11.158943+0000","last_became_active":"2024-02-23T00:39:16.789757+0000","last_became_peered":"2024-02-23T00:39:16.789757+0000","last_unstale":"2024-02-23T00:45:11.158943+0000","last_undegraded":"2024-02-23T00:45:11.158943+0000","last_fullsized":"2024-02-23T00:45:11.158943+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:44:19.830309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:44:19.830309+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":203,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.157387+0000","last_change":"2024-02-23T00:44:29.773000+0000","last_active":"2024-02-23T00:45:11.157387+0000","last_peered":"2024-02-23T00:45:11.157387+0000","last_clean":"2024-02-23T00:45:11.157387+0000","last_became_active":"2024-02-23T00:39:14.780526+0000","last_became_peered":"2024-02-23T00:39:14.780526+0000","last_unstale":"2024-02-23T00:45:11.157387+0000","last_undegraded":"2024-02-23T00:45:11.157387+0000","last_fullsized":"2024-02-23T00:45:11.157387+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:29.772933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:29.772933+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":278,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.144288+0000","last_change":"2024-02-23T00:44:23.109508+0000","last_active":"2024-02-23T00:45:11.144288+0000","last_peered":"2024-02-23T00:45:11.144288+0000","last_clean":"2024-02-23T00:45:11.144288+0000","last_became_active":"2024-02-23T00:39:18.800440+0000","last_became_peered":"2024-02-23T00:39:18.800440+0000","last_unstale":"2024-02-23T00:45:11.144288+0000","last_undegraded":"2024-02-23T00:45:11.144288+0000","last_fullsized":"2024-02-23T00:45:11.144288+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:44:23.109439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.109439+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":216,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.162031+0000","last_change":"2024-02-23T00:44:22.976713+0000","last_active":"2024-02-23T00:45:11.162031+0000","last_peered":"2024-02-23T00:45:11.162031+0000","last_clean":"2024-02-23T00:45:11.162031+0000","last_became_active":"2024-02-23T00:43:16.650527+0000","last_became_peered":"2024-02-23T00:43:16.650527+0000","last_unstale":"2024-02-23T00:45:11.162031+0000","last_undegraded":"2024-02-23T00:45:11.162031+0000","last_fullsized":"2024-02-23T00:45:11.162031+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:22.976611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:10.666299+0000","last_clean_scrub_stamp":"2024-02-23T00:44:22.976611+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.158017+0000","last_change":"2024-02-23T00:42:57.671776+0000","last_active":"2024-02-23T00:45:11.158017+0000","last_peered":"2024-02-23T00:45:11.158017+0000","last_clean":"2024-02-23T00:45:11.158017+0000","last_became_active":"2024-02-23T00:42:57.671472+0000","last_became_peered":"2024-02-23T00:42:57.671472+0000","last_unstale":"2024-02-23T00:45:11.158017+0000","last_undegraded":"2024-02-23T00:45:11.158017+0000","last_fullsized":"2024-02-23T00:45:11.158017+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,3],"acting":[1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.143085+0000","last_change":"2024-02-23T00:42:58.834770+0000","last_active":"2024-02-23T00:45:11.143085+0000","last_peered":"2024-02-23T00:45:11.143085+0000","last_clean":"2024-02-23T00:45:11.143085+0000","last_became_active":"2024-02-23T00:42:58.833924+0000","last_became_peered":"2024-02-23T00:42:58.833924+0000","last_unstale":"2024-02-23T00:45:11.143085+0000","last_undegraded":"2024-02-23T00:45:11.143085+0000","last_fullsized":"2024-02-23T00:45:11.143085+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"68'17","reported_seq":253,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.144210+0000","last_change":"2024-02-23T00:43:33.151471+0000","last_active":"2024-02-23T00:45:11.144210+0000","last_peered":"2024-02-23T00:45:11.144210+0000","last_clean":"2024-02-23T00:45:11.144210+0000","last_became_active":"2024-02-23T00:40:42.933759+0000","last_became_peered":"2024-02-23T00:40:42.933759+0000","last_unstale":"2024-02-23T00:45:11.144210+0000","last_undegraded":"2024-02-23T00:45:11.144210+0000","last_fullsized":"2024-02-23T00:45:11.144210+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:43:33.151401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:33.151401+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":202,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.154453+0000","last_change":"2024-02-23T00:44:18.886545+0000","last_active":"2024-02-23T00:45:11.154453+0000","last_peered":"2024-02-23T00:45:11.154453+0000","last_clean":"2024-02-23T00:45:11.154453+0000","last_became_active":"2024-02-23T00:39:14.784121+0000","last_became_peered":"2024-02-23T00:39:14.784121+0000","last_unstale":"2024-02-23T00:45:11.154453+0000","last_undegraded":"2024-02-23T00:45:11.154453+0000","last_fullsized":"2024-02-23T00:45:11.154453+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:18.886477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:18.886477+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":200,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.162008+0000","last_change":"2024-02-23T00:45:08.118391+0000","last_active":"2024-02-23T00:45:11.162008+0000","last_peered":"2024-02-23T00:45:11.162008+0000","last_clean":"2024-02-23T00:45:11.162008+0000","last_became_active":"2024-02-23T00:39:18.800777+0000","last_became_peered":"2024-02-23T00:39:18.800777+0000","last_unstale":"2024-02-23T00:45:11.162008+0000","last_undegraded":"2024-02-23T00:45:11.162008+0000","last_fullsized":"2024-02-23T00:45:11.162008+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.118302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.118302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"67'1","reported_seq":216,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.145232+0000","last_change":"2024-02-23T00:43:56.091057+0000","last_active":"2024-02-23T00:45:11.145232+0000","last_peered":"2024-02-23T00:45:11.145232+0000","last_clean":"2024-02-23T00:45:11.145232+0000","last_became_active":"2024-02-23T00:43:22.866277+0000","last_became_peered":"2024-02-23T00:43:22.866277+0000","last_unstale":"2024-02-23T00:45:11.145232+0000","last_undegraded":"2024-02-23T00:45:11.145232+0000","last_fullsized":"2024-02-23T00:45:11.145232+0000","mapping_epoch":65,"log_start":"67'1","ondisk_log_start":"67'1","created":65,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:43:56.090982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:09.079918+0000","last_clean_scrub_stamp":"2024-02-23T00:43:56.090982+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":145,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.163200+0000","last_change":"2024-02-23T00:45:06.651852+0000","last_active":"2024-02-23T00:45:11.163200+0000","last_peered":"2024-02-23T00:45:11.163200+0000","last_clean":"2024-02-23T00:45:11.163200+0000","last_became_active":"2024-02-23T00:42:58.718615+0000","last_became_peered":"2024-02-23T00:42:58.718615+0000","last_unstale":"2024-02-23T00:45:11.163200+0000","last_undegraded":"2024-02-23T00:45:11.163200+0000","last_fullsized":"2024-02-23T00:45:11.163200+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.651711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.651711+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.161009+0000","last_change":"2024-02-23T00:42:57.655252+0000","last_active":"2024-02-23T00:45:11.161009+0000","last_peered":"2024-02-23T00:45:11.161009+0000","last_clean":"2024-02-23T00:45:11.161009+0000","last_became_active":"2024-02-23T00:42:57.653279+0000","last_became_peered":"2024-02-23T00:42:57.653279+0000","last_unstale":"2024-02-23T00:45:11.161009+0000","last_undegraded":"2024-02-23T00:45:11.161009+0000","last_fullsized":"2024-02-23T00:45:11.161009+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":274,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.146817+0000","last_change":"2024-02-23T00:42:53.978054+0000","last_active":"2024-02-23T00:45:11.146817+0000","last_peered":"2024-02-23T00:45:11.146817+0000","last_clean":"2024-02-23T00:45:11.146817+0000","last_became_active":"2024-02-23T00:39:18.798536+0000","last_became_peered":"2024-02-23T00:39:18.798536+0000","last_unstale":"2024-02-23T00:45:11.146817+0000","last_undegraded":"2024-02-23T00:45:11.146817+0000","last_fullsized":"2024-02-23T00:45:11.146817+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:42:53.977950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.977950+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"68'20","reported_seq":237,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.143400+0000","last_change":"2024-02-23T00:45:05.291909+0000","last_active":"2024-02-23T00:45:11.143400+0000","last_peered":"2024-02-23T00:45:11.143400+0000","last_clean":"2024-02-23T00:45:11.143400+0000","last_became_active":"2024-02-23T00:39:16.780130+0000","last_became_peered":"2024-02-23T00:39:16.780130+0000","last_unstale":"2024-02-23T00:45:11.143400+0000","last_undegraded":"2024-02-23T00:45:11.143400+0000","last_fullsized":"2024-02-23T00:45:11.143400+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'20","last_scrub_stamp":"2024-02-23T00:45:05.291836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:05.291836+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":205,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.164902+0000","last_change":"2024-02-23T00:43:51.391999+0000","last_active":"2024-02-23T00:45:11.164902+0000","last_peered":"2024-02-23T00:45:11.164902+0000","last_clean":"2024-02-23T00:45:11.164902+0000","last_became_active":"2024-02-23T00:39:14.781999+0000","last_became_peered":"2024-02-23T00:39:14.781999+0000","last_unstale":"2024-02-23T00:45:11.164902+0000","last_undegraded":"2024-02-23T00:45:11.164902+0000","last_fullsized":"2024-02-23T00:45:11.164902+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:51.391939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:51.391939+0000","last_clean_scrub_stamp":"2024-02-23T00:43:51.391939+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":196,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.156281+0000","last_change":"2024-02-23T00:44:24.759694+0000","last_active":"2024-02-23T00:45:11.156281+0000","last_peered":"2024-02-23T00:45:11.156281+0000","last_clean":"2024-02-23T00:45:11.156281+0000","last_became_active":"2024-02-23T00:39:21.161928+0000","last_became_peered":"2024-02-23T00:39:21.161928+0000","last_unstale":"2024-02-23T00:45:11.156281+0000","last_undegraded":"2024-02-23T00:45:11.156281+0000","last_fullsized":"2024-02-23T00:45:11.156281+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:24.759627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.945642+0000","last_clean_scrub_stamp":"2024-02-23T00:44:24.759627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"110'10","reported_seq":150,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.151492+0000","last_change":"2024-02-23T00:42:58.696579+0000","last_active":"2024-02-23T00:45:11.151492+0000","last_peered":"2024-02-23T00:45:11.151492+0000","last_clean":"2024-02-23T00:45:11.151492+0000","last_became_active":"2024-02-23T00:42:58.696292+0000","last_became_peered":"2024-02-23T00:42:58.696292+0000","last_unstale":"2024-02-23T00:45:11.151492+0000","last_undegraded":"2024-02-23T00:45:11.151492+0000","last_fullsized":"2024-02-23T00:45:11.151492+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.162292+0000","last_change":"2024-02-23T00:42:58.717788+0000","last_active":"2024-02-23T00:45:11.162292+0000","last_peered":"2024-02-23T00:45:11.162292+0000","last_clean":"2024-02-23T00:45:11.162292+0000","last_became_active":"2024-02-23T00:42:58.717371+0000","last_became_peered":"2024-02-23T00:42:58.717371+0000","last_unstale":"2024-02-23T00:45:11.162292+0000","last_undegraded":"2024-02-23T00:45:11.162292+0000","last_fullsized":"2024-02-23T00:45:11.162292+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":199,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.144030+0000","last_change":"2024-02-23T00:44:34.059194+0000","last_active":"2024-02-23T00:45:11.144030+0000","last_peered":"2024-02-23T00:45:11.144030+0000","last_clean":"2024-02-23T00:45:11.144030+0000","last_became_active":"2024-02-23T00:39:18.798829+0000","last_became_peered":"2024-02-23T00:39:18.798829+0000","last_unstale":"2024-02-23T00:45:11.144030+0000","last_undegraded":"2024-02-23T00:45:11.144030+0000","last_fullsized":"2024-02-23T00:45:11.144030+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:34.059129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:34.059129+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"68'30","reported_seq":259,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.146736+0000","last_change":"2024-02-23T00:42:49.970825+0000","last_active":"2024-02-23T00:45:11.146736+0000","last_peered":"2024-02-23T00:45:11.146736+0000","last_clean":"2024-02-23T00:45:11.146736+0000","last_became_active":"2024-02-23T00:39:16.859002+0000","last_became_peered":"2024-02-23T00:39:16.859002+0000","last_unstale":"2024-02-23T00:45:11.146736+0000","last_undegraded":"2024-02-23T00:45:11.146736+0000","last_fullsized":"2024-02-23T00:45:11.146736+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'30","last_scrub_stamp":"2024-02-23T00:42:49.970752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:49.970752+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":198,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.158340+0000","last_change":"2024-02-23T00:42:40.534390+0000","last_active":"2024-02-23T00:45:11.158340+0000","last_peered":"2024-02-23T00:45:11.158340+0000","last_clean":"2024-02-23T00:45:11.158340+0000","last_became_active":"2024-02-23T00:39:14.779193+0000","last_became_peered":"2024-02-23T00:39:14.779193+0000","last_unstale":"2024-02-23T00:45:11.158340+0000","last_undegraded":"2024-02-23T00:45:11.158340+0000","last_fullsized":"2024-02-23T00:45:11.158340+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:40.534284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:40.534284+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":197,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.143493+0000","last_change":"2024-02-23T00:45:03.228087+0000","last_active":"2024-02-23T00:45:11.143493+0000","last_peered":"2024-02-23T00:45:11.143493+0000","last_clean":"2024-02-23T00:45:11.143493+0000","last_became_active":"2024-02-23T00:39:20.795271+0000","last_became_peered":"2024-02-23T00:39:20.795271+0000","last_unstale":"2024-02-23T00:45:11.143493+0000","last_undegraded":"2024-02-23T00:45:11.143493+0000","last_fullsized":"2024-02-23T00:45:11.143493+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:03.227995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:45:03.227995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"110'2","reported_seq":147,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.151045+0000","last_change":"2024-02-23T00:45:10.465187+0000","last_active":"2024-02-23T00:45:11.151045+0000","last_peered":"2024-02-23T00:45:11.151045+0000","last_clean":"2024-02-23T00:45:11.151045+0000","last_became_active":"2024-02-23T00:42:58.814093+0000","last_became_peered":"2024-02-23T00:42:58.814093+0000","last_unstale":"2024-02-23T00:45:11.151045+0000","last_undegraded":"2024-02-23T00:45:11.151045+0000","last_fullsized":"2024-02-23T00:45:11.151045+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'2","last_scrub_stamp":"2024-02-23T00:45:10.465108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.465108+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.144172+0000","last_change":"2024-02-23T00:42:58.809948+0000","last_active":"2024-02-23T00:45:11.144172+0000","last_peered":"2024-02-23T00:45:11.144172+0000","last_clean":"2024-02-23T00:45:11.144172+0000","last_became_active":"2024-02-23T00:42:58.809283+0000","last_became_peered":"2024-02-23T00:42:58.809283+0000","last_unstale":"2024-02-23T00:45:11.144172+0000","last_undegraded":"2024-02-23T00:45:11.144172+0000","last_fullsized":"2024-02-23T00:45:11.144172+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":199,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.166580+0000","last_change":"2024-02-23T00:44:19.312509+0000","last_active":"2024-02-23T00:45:11.166580+0000","last_peered":"2024-02-23T00:45:11.166580+0000","last_clean":"2024-02-23T00:45:11.166580+0000","last_became_active":"2024-02-23T00:39:18.796664+0000","last_became_peered":"2024-02-23T00:39:18.796664+0000","last_unstale":"2024-02-23T00:45:11.166580+0000","last_undegraded":"2024-02-23T00:45:11.166580+0000","last_fullsized":"2024-02-23T00:45:11.166580+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:19.312418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:19.312418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"203'34","reported_seq":262,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.145803+0000","last_change":"2024-02-23T00:44:35.644033+0000","last_active":"2024-02-23T00:45:11.145803+0000","last_peered":"2024-02-23T00:45:11.145803+0000","last_clean":"2024-02-23T00:45:11.145803+0000","last_became_active":"2024-02-23T00:39:16.857278+0000","last_became_peered":"2024-02-23T00:39:16.857278+0000","last_unstale":"2024-02-23T00:45:11.145803+0000","last_undegraded":"2024-02-23T00:45:11.145803+0000","last_fullsized":"2024-02-23T00:45:11.145803+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'34","last_scrub_stamp":"2024-02-23T00:44:35.643931+0000","last_deep_scrub":"203'34","last_deep_scrub_stamp":"2024-02-23T00:44:35.643931+0000","last_clean_scrub_stamp":"2024-02-23T00:44:35.643931+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":197,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.156955+0000","last_change":"2024-02-23T00:42:43.990569+0000","last_active":"2024-02-23T00:45:11.156955+0000","last_peered":"2024-02-23T00:45:11.156955+0000","last_clean":"2024-02-23T00:45:11.156955+0000","last_became_active":"2024-02-23T00:39:14.783662+0000","last_became_peered":"2024-02-23T00:39:14.783662+0000","last_unstale":"2024-02-23T00:45:11.156955+0000","last_undegraded":"2024-02-23T00:45:11.156955+0000","last_fullsized":"2024-02-23T00:45:11.156955+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:43.990486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:43.990486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":197,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.157471+0000","last_change":"2024-02-23T00:45:04.585266+0000","last_active":"2024-02-23T00:45:11.157471+0000","last_peered":"2024-02-23T00:45:11.157471+0000","last_clean":"2024-02-23T00:45:11.157471+0000","last_became_active":"2024-02-23T00:39:21.160060+0000","last_became_peered":"2024-02-23T00:39:21.160060+0000","last_unstale":"2024-02-23T00:45:11.157471+0000","last_undegraded":"2024-02-23T00:45:11.157471+0000","last_fullsized":"2024-02-23T00:45:11.157471+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:04.585179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.585179+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.8","version":"110'2","reported_seq":147,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.148799+0000","last_change":"2024-02-23T00:44:33.389680+0000","last_active":"2024-02-23T00:45:11.148799+0000","last_peered":"2024-02-23T00:45:11.148799+0000","last_clean":"2024-02-23T00:45:11.148799+0000","last_became_active":"2024-02-23T00:42:58.809584+0000","last_became_peered":"2024-02-23T00:42:58.809584+0000","last_unstale":"2024-02-23T00:45:11.148799+0000","last_undegraded":"2024-02-23T00:45:11.148799+0000","last_fullsized":"2024-02-23T00:45:11.148799+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'2","last_scrub_stamp":"2024-02-23T00:44:33.389605+0000","last_deep_scrub":"110'2","last_deep_scrub_stamp":"2024-02-23T00:44:33.389605+0000","last_clean_scrub_stamp":"2024-02-23T00:44:33.389605+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.161565+0000","last_change":"2024-02-23T00:42:58.726955+0000","last_active":"2024-02-23T00:45:11.161565+0000","last_peered":"2024-02-23T00:45:11.161565+0000","last_clean":"2024-02-23T00:45:11.161565+0000","last_became_active":"2024-02-23T00:42:58.726652+0000","last_became_peered":"2024-02-23T00:42:58.726652+0000","last_unstale":"2024-02-23T00:45:11.161565+0000","last_undegraded":"2024-02-23T00:45:11.161565+0000","last_fullsized":"2024-02-23T00:45:11.161565+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":200,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.162091+0000","last_change":"2024-02-23T00:43:45.054194+0000","last_active":"2024-02-23T00:45:11.162091+0000","last_peered":"2024-02-23T00:45:11.162091+0000","last_clean":"2024-02-23T00:45:11.162091+0000","last_became_active":"2024-02-23T00:39:18.800378+0000","last_became_peered":"2024-02-23T00:39:18.800378+0000","last_unstale":"2024-02-23T00:45:11.162091+0000","last_undegraded":"2024-02-23T00:45:11.162091+0000","last_fullsized":"2024-02-23T00:45:11.162091+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:45.054115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:59.694033+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.054115+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"95'18","reported_seq":226,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.149943+0000","last_change":"2024-02-23T00:44:18.630435+0000","last_active":"2024-02-23T00:45:11.149943+0000","last_peered":"2024-02-23T00:45:11.149943+0000","last_clean":"2024-02-23T00:45:11.149943+0000","last_became_active":"2024-02-23T00:39:16.787143+0000","last_became_peered":"2024-02-23T00:39:16.787143+0000","last_unstale":"2024-02-23T00:45:11.149943+0000","last_undegraded":"2024-02-23T00:45:11.149943+0000","last_fullsized":"2024-02-23T00:45:11.149943+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'18","last_scrub_stamp":"2024-02-23T00:44:18.630363+0000","last_deep_scrub":"95'18","last_deep_scrub_stamp":"2024-02-23T00:42:01.195545+0000","last_clean_scrub_stamp":"2024-02-23T00:44:18.630363+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":202,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.154350+0000","last_change":"2024-02-23T00:42:50.947420+0000","last_active":"2024-02-23T00:45:11.154350+0000","last_peered":"2024-02-23T00:45:11.154350+0000","last_clean":"2024-02-23T00:45:11.154350+0000","last_became_active":"2024-02-23T00:39:14.784578+0000","last_became_peered":"2024-02-23T00:39:14.784578+0000","last_unstale":"2024-02-23T00:45:11.154350+0000","last_undegraded":"2024-02-23T00:45:11.154350+0000","last_fullsized":"2024-02-23T00:45:11.154350+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:50.947349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:50.947349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":204,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.144563+0000","last_change":"2024-02-23T00:43:48.258338+0000","last_active":"2024-02-23T00:45:11.144563+0000","last_peered":"2024-02-23T00:45:11.144563+0000","last_clean":"2024-02-23T00:45:11.144563+0000","last_became_active":"2024-02-23T00:39:14.775446+0000","last_became_peered":"2024-02-23T00:39:14.775446+0000","last_unstale":"2024-02-23T00:45:11.144563+0000","last_undegraded":"2024-02-23T00:45:11.144563+0000","last_fullsized":"2024-02-23T00:45:11.144563+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:48.258269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.258269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":192,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.158858+0000","last_change":"2024-02-23T00:43:47.905617+0000","last_active":"2024-02-23T00:45:11.158858+0000","last_peered":"2024-02-23T00:45:11.158858+0000","last_clean":"2024-02-23T00:45:11.158858+0000","last_became_active":"2024-02-23T00:39:20.797242+0000","last_became_peered":"2024-02-23T00:39:20.797242+0000","last_unstale":"2024-02-23T00:45:11.158858+0000","last_undegraded":"2024-02-23T00:45:11.158858+0000","last_fullsized":"2024-02-23T00:45:11.158858+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:47.905501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:47.905501+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"110'5","reported_seq":145,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.156554+0000","last_change":"2024-02-23T00:42:58.743470+0000","last_active":"2024-02-23T00:45:11.156554+0000","last_peered":"2024-02-23T00:45:11.156554+0000","last_clean":"2024-02-23T00:45:11.156554+0000","last_became_active":"2024-02-23T00:42:58.742998+0000","last_became_peered":"2024-02-23T00:42:58.742998+0000","last_unstale":"2024-02-23T00:45:11.156554+0000","last_undegraded":"2024-02-23T00:45:11.156554+0000","last_fullsized":"2024-02-23T00:45:11.156554+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":145,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.161169+0000","last_change":"2024-02-23T00:44:35.360128+0000","last_active":"2024-02-23T00:45:11.161169+0000","last_peered":"2024-02-23T00:45:11.161169+0000","last_clean":"2024-02-23T00:45:11.161169+0000","last_became_active":"2024-02-23T00:42:58.690858+0000","last_became_peered":"2024-02-23T00:42:58.690858+0000","last_unstale":"2024-02-23T00:45:11.161169+0000","last_undegraded":"2024-02-23T00:45:11.161169+0000","last_fullsized":"2024-02-23T00:45:11.161169+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:35.360054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:35.360054+0000","last_clean_scrub_stamp":"2024-02-23T00:44:35.360054+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":193,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.156706+0000","last_change":"2024-02-23T00:42:42.029769+0000","last_active":"2024-02-23T00:45:11.156706+0000","last_peered":"2024-02-23T00:45:11.156706+0000","last_clean":"2024-02-23T00:45:11.156706+0000","last_became_active":"2024-02-23T00:39:18.793801+0000","last_became_peered":"2024-02-23T00:39:18.793801+0000","last_unstale":"2024-02-23T00:45:11.156706+0000","last_undegraded":"2024-02-23T00:45:11.156706+0000","last_fullsized":"2024-02-23T00:45:11.156706+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:42.029652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:42.029652+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"67'14","reported_seq":223,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.163004+0000","last_change":"2024-02-23T00:43:44.369862+0000","last_active":"2024-02-23T00:45:11.163004+0000","last_peered":"2024-02-23T00:45:11.163004+0000","last_clean":"2024-02-23T00:45:11.163004+0000","last_became_active":"2024-02-23T00:39:16.858880+0000","last_became_peered":"2024-02-23T00:39:16.858880+0000","last_unstale":"2024-02-23T00:45:11.163004+0000","last_undegraded":"2024-02-23T00:45:11.163004+0000","last_fullsized":"2024-02-23T00:45:11.163004+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'14","last_scrub_stamp":"2024-02-23T00:43:44.369792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:44.369792+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":212,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.154396+0000","last_change":"2024-02-23T00:45:08.796153+0000","last_active":"2024-02-23T00:45:11.154396+0000","last_peered":"2024-02-23T00:45:11.154396+0000","last_clean":"2024-02-23T00:45:11.154396+0000","last_became_active":"2024-02-23T00:39:08.248249+0000","last_became_peered":"2024-02-23T00:39:08.248249+0000","last_unstale":"2024-02-23T00:45:11.154396+0000","last_undegraded":"2024-02-23T00:45:11.154396+0000","last_fullsized":"2024-02-23T00:45:11.154396+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.796076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:49.977248+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.796076+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":199,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.160960+0000","last_change":"2024-02-23T00:42:46.945425+0000","last_active":"2024-02-23T00:45:11.160960+0000","last_peered":"2024-02-23T00:45:11.160960+0000","last_clean":"2024-02-23T00:45:11.160960+0000","last_became_active":"2024-02-23T00:39:14.788888+0000","last_became_peered":"2024-02-23T00:39:14.788888+0000","last_unstale":"2024-02-23T00:45:11.160960+0000","last_undegraded":"2024-02-23T00:45:11.160960+0000","last_fullsized":"2024-02-23T00:45:11.160960+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:46.945358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:46.945358+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":204,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.145122+0000","last_change":"2024-02-23T00:44:33.143663+0000","last_active":"2024-02-23T00:45:11.145122+0000","last_peered":"2024-02-23T00:45:11.145122+0000","last_clean":"2024-02-23T00:45:11.145122+0000","last_became_active":"2024-02-23T00:39:14.783489+0000","last_became_peered":"2024-02-23T00:39:14.783489+0000","last_unstale":"2024-02-23T00:45:11.145122+0000","last_undegraded":"2024-02-23T00:45:11.145122+0000","last_fullsized":"2024-02-23T00:45:11.145122+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:33.143592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:33.143592+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":216,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.162201+0000","last_change":"2024-02-23T00:44:20.958888+0000","last_active":"2024-02-23T00:45:11.162201+0000","last_peered":"2024-02-23T00:45:11.162201+0000","last_clean":"2024-02-23T00:45:11.162201+0000","last_became_active":"2024-02-23T00:43:42.168435+0000","last_became_peered":"2024-02-23T00:43:42.168435+0000","last_unstale":"2024-02-23T00:45:11.162201+0000","last_undegraded":"2024-02-23T00:45:11.162201+0000","last_fullsized":"2024-02-23T00:45:11.162201+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:20.958797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:20.958797+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.154494+0000","last_change":"2024-02-23T00:42:58.721944+0000","last_active":"2024-02-23T00:45:11.154494+0000","last_peered":"2024-02-23T00:45:11.154494+0000","last_clean":"2024-02-23T00:45:11.154494+0000","last_became_active":"2024-02-23T00:42:58.721439+0000","last_became_peered":"2024-02-23T00:42:58.721439+0000","last_unstale":"2024-02-23T00:45:11.154494+0000","last_undegraded":"2024-02-23T00:45:11.154494+0000","last_fullsized":"2024-02-23T00:45:11.154494+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.156158+0000","last_change":"2024-02-23T00:42:58.701934+0000","last_active":"2024-02-23T00:45:11.156158+0000","last_peered":"2024-02-23T00:45:11.156158+0000","last_clean":"2024-02-23T00:45:11.156158+0000","last_became_active":"2024-02-23T00:42:58.701502+0000","last_became_peered":"2024-02-23T00:42:58.701502+0000","last_unstale":"2024-02-23T00:45:11.156158+0000","last_undegraded":"2024-02-23T00:45:11.156158+0000","last_fullsized":"2024-02-23T00:45:11.156158+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":199,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.154875+0000","last_change":"2024-02-23T00:43:46.293962+0000","last_active":"2024-02-23T00:45:11.154875+0000","last_peered":"2024-02-23T00:45:11.154875+0000","last_clean":"2024-02-23T00:45:11.154875+0000","last_became_active":"2024-02-23T00:39:18.789330+0000","last_became_peered":"2024-02-23T00:39:18.789330+0000","last_unstale":"2024-02-23T00:45:11.154875+0000","last_undegraded":"2024-02-23T00:45:11.154875+0000","last_fullsized":"2024-02-23T00:45:11.154875+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:46.293864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.264240+0000","last_clean_scrub_stamp":"2024-02-23T00:43:46.293864+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"68'19","reported_seq":243,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.158894+0000","last_change":"2024-02-23T00:45:07.697475+0000","last_active":"2024-02-23T00:45:11.158894+0000","last_peered":"2024-02-23T00:45:11.158894+0000","last_clean":"2024-02-23T00:45:11.158894+0000","last_became_active":"2024-02-23T00:39:16.787484+0000","last_became_peered":"2024-02-23T00:39:16.787484+0000","last_unstale":"2024-02-23T00:45:11.158894+0000","last_undegraded":"2024-02-23T00:45:11.158894+0000","last_fullsized":"2024-02-23T00:45:11.158894+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'19","last_scrub_stamp":"2024-02-23T00:45:07.697394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:07.697394+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":203,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.149549+0000","last_change":"2024-02-23T00:43:54.305009+0000","last_active":"2024-02-23T00:45:11.149549+0000","last_peered":"2024-02-23T00:45:11.149549+0000","last_clean":"2024-02-23T00:45:11.149549+0000","last_became_active":"2024-02-23T00:39:14.772411+0000","last_became_peered":"2024-02-23T00:39:14.772411+0000","last_unstale":"2024-02-23T00:45:11.149549+0000","last_undegraded":"2024-02-23T00:45:11.149549+0000","last_fullsized":"2024-02-23T00:45:11.149549+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:54.304948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:54.304948+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"68'5","reported_seq":550,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.623729+0000","last_change":"2024-02-23T00:43:48.876428+0000","last_active":"2024-02-23T00:45:11.623729+0000","last_peered":"2024-02-23T00:45:11.623729+0000","last_clean":"2024-02-23T00:45:11.623729+0000","last_became_active":"2024-02-23T00:39:14.779109+0000","last_became_peered":"2024-02-23T00:39:14.779109+0000","last_unstale":"2024-02-23T00:45:11.623729+0000","last_undegraded":"2024-02-23T00:45:11.623729+0000","last_fullsized":"2024-02-23T00:45:11.623729+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'5","last_scrub_stamp":"2024-02-23T00:43:48.876356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.876356+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":349,"num_read_kb":344,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":196,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.156882+0000","last_change":"2024-02-23T00:43:47.794616+0000","last_active":"2024-02-23T00:45:11.156882+0000","last_peered":"2024-02-23T00:45:11.156882+0000","last_clean":"2024-02-23T00:45:11.156882+0000","last_became_active":"2024-02-23T00:39:21.160457+0000","last_became_peered":"2024-02-23T00:39:21.160457+0000","last_unstale":"2024-02-23T00:45:11.156882+0000","last_undegraded":"2024-02-23T00:45:11.156882+0000","last_fullsized":"2024-02-23T00:45:11.156882+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:47.794534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:47.794534+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.159074+0000","last_change":"2024-02-23T00:42:58.831449+0000","last_active":"2024-02-23T00:45:11.159074+0000","last_peered":"2024-02-23T00:45:11.159074+0000","last_clean":"2024-02-23T00:45:11.159074+0000","last_became_active":"2024-02-23T00:42:58.830815+0000","last_became_peered":"2024-02-23T00:42:58.830815+0000","last_unstale":"2024-02-23T00:45:11.159074+0000","last_undegraded":"2024-02-23T00:45:11.159074+0000","last_fullsized":"2024-02-23T00:45:11.159074+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.157214+0000","last_change":"2024-02-23T00:42:57.653669+0000","last_active":"2024-02-23T00:45:11.157214+0000","last_peered":"2024-02-23T00:45:11.157214+0000","last_clean":"2024-02-23T00:45:11.157214+0000","last_became_active":"2024-02-23T00:42:57.653355+0000","last_became_peered":"2024-02-23T00:42:57.653355+0000","last_unstale":"2024-02-23T00:45:11.157214+0000","last_undegraded":"2024-02-23T00:45:11.157214+0000","last_fullsized":"2024-02-23T00:45:11.157214+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":200,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.145599+0000","last_change":"2024-02-23T00:44:20.626116+0000","last_active":"2024-02-23T00:45:11.145599+0000","last_peered":"2024-02-23T00:45:11.145599+0000","last_clean":"2024-02-23T00:45:11.145599+0000","last_became_active":"2024-02-23T00:39:18.802134+0000","last_became_peered":"2024-02-23T00:39:18.802134+0000","last_unstale":"2024-02-23T00:45:11.145599+0000","last_undegraded":"2024-02-23T00:45:11.145599+0000","last_fullsized":"2024-02-23T00:45:11.145599+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:20.626052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:20.626052+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"86'35","reported_seq":252,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.159153+0000","last_change":"2024-02-23T00:42:47.988747+0000","last_active":"2024-02-23T00:45:11.159153+0000","last_peered":"2024-02-23T00:45:11.159153+0000","last_clean":"2024-02-23T00:45:11.159153+0000","last_became_active":"2024-02-23T00:40:42.933322+0000","last_became_peered":"2024-02-23T00:40:42.933322+0000","last_unstale":"2024-02-23T00:45:11.159153+0000","last_undegraded":"2024-02-23T00:45:11.159153+0000","last_fullsized":"2024-02-23T00:45:11.159153+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'35","last_scrub_stamp":"2024-02-23T00:42:47.988676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:47.988676+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":204,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.159427+0000","last_change":"2024-02-23T00:44:23.965228+0000","last_active":"2024-02-23T00:45:11.159427+0000","last_peered":"2024-02-23T00:45:11.159427+0000","last_clean":"2024-02-23T00:45:11.159427+0000","last_became_active":"2024-02-23T00:39:14.784015+0000","last_became_peered":"2024-02-23T00:39:14.784015+0000","last_unstale":"2024-02-23T00:45:11.159427+0000","last_undegraded":"2024-02-23T00:45:11.159427+0000","last_fullsized":"2024-02-23T00:45:11.159427+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:23.965125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.965125+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":211,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.144044+0000","last_change":"2024-02-23T00:43:45.144499+0000","last_active":"2024-02-23T00:45:11.144044+0000","last_peered":"2024-02-23T00:45:11.144044+0000","last_clean":"2024-02-23T00:45:11.144044+0000","last_became_active":"2024-02-23T00:43:45.144043+0000","last_became_peered":"2024-02-23T00:43:45.144043+0000","last_unstale":"2024-02-23T00:45:11.144044+0000","last_undegraded":"2024-02-23T00:45:11.144044+0000","last_fullsized":"2024-02-23T00:45:11.144044+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:40.062493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:40.062493+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":145,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.164759+0000","last_change":"2024-02-23T00:44:31.410733+0000","last_active":"2024-02-23T00:45:11.164759+0000","last_peered":"2024-02-23T00:45:11.164759+0000","last_clean":"2024-02-23T00:45:11.164759+0000","last_became_active":"2024-02-23T00:42:58.754813+0000","last_became_peered":"2024-02-23T00:42:58.754813+0000","last_unstale":"2024-02-23T00:45:11.164759+0000","last_undegraded":"2024-02-23T00:45:11.164759+0000","last_fullsized":"2024-02-23T00:45:11.164759+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:31.410670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:31.410670+0000","last_clean_scrub_stamp":"2024-02-23T00:44:31.410670+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":145,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.151285+0000","last_change":"2024-02-23T00:45:05.490595+0000","last_active":"2024-02-23T00:45:11.151285+0000","last_peered":"2024-02-23T00:45:11.151285+0000","last_clean":"2024-02-23T00:45:11.151285+0000","last_became_active":"2024-02-23T00:42:58.766579+0000","last_became_peered":"2024-02-23T00:42:58.766579+0000","last_unstale":"2024-02-23T00:45:11.151285+0000","last_undegraded":"2024-02-23T00:45:11.151285+0000","last_fullsized":"2024-02-23T00:45:11.151285+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:05.490531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:05.490531+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"91'9","reported_seq":280,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.146143+0000","last_change":"2024-02-23T00:44:28.619036+0000","last_active":"2024-02-23T00:45:11.146143+0000","last_peered":"2024-02-23T00:45:11.146143+0000","last_clean":"2024-02-23T00:45:11.146143+0000","last_became_active":"2024-02-23T00:39:18.801297+0000","last_became_peered":"2024-02-23T00:39:18.801297+0000","last_unstale":"2024-02-23T00:45:11.146143+0000","last_undegraded":"2024-02-23T00:45:11.146143+0000","last_fullsized":"2024-02-23T00:45:11.146143+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'9","last_scrub_stamp":"2024-02-23T00:44:28.618923+0000","last_deep_scrub":"91'9","last_deep_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_clean_scrub_stamp":"2024-02-23T00:44:28.618923+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"68'16","reported_seq":231,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.713826+0000","last_change":"2024-02-23T00:45:11.713800+0000","last_active":"2024-02-23T00:45:11.713826+0000","last_peered":"2024-02-23T00:45:11.713826+0000","last_clean":"2024-02-23T00:45:11.713826+0000","last_became_active":"2024-02-23T00:39:16.787468+0000","last_became_peered":"2024-02-23T00:39:16.787468+0000","last_unstale":"2024-02-23T00:45:11.713826+0000","last_undegraded":"2024-02-23T00:45:11.713826+0000","last_fullsized":"2024-02-23T00:45:11.713826+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:45:11.713731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:11.713731+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":198,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.158149+0000","last_change":"2024-02-23T00:42:46.606249+0000","last_active":"2024-02-23T00:45:11.158149+0000","last_peered":"2024-02-23T00:45:11.158149+0000","last_clean":"2024-02-23T00:45:11.158149+0000","last_became_active":"2024-02-23T00:39:14.779952+0000","last_became_peered":"2024-02-23T00:39:14.779952+0000","last_unstale":"2024-02-23T00:45:11.158149+0000","last_undegraded":"2024-02-23T00:45:11.158149+0000","last_fullsized":"2024-02-23T00:45:11.158149+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:46.606185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:46.606185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"107'2","reported_seq":200,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.143084+0000","last_change":"2024-02-23T00:45:08.662754+0000","last_active":"2024-02-23T00:45:11.143084+0000","last_peered":"2024-02-23T00:45:11.143084+0000","last_clean":"2024-02-23T00:45:11.143084+0000","last_became_active":"2024-02-23T00:39:20.800858+0000","last_became_peered":"2024-02-23T00:39:20.800858+0000","last_unstale":"2024-02-23T00:45:11.143084+0000","last_undegraded":"2024-02-23T00:45:11.143084+0000","last_fullsized":"2024-02-23T00:45:11.143084+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'2","last_scrub_stamp":"2024-02-23T00:45:08.662667+0000","last_deep_scrub":"107'2","last_deep_scrub_stamp":"2024-02-23T00:42:48.362739+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.662667+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.161557+0000","last_change":"2024-02-23T00:42:58.727288+0000","last_active":"2024-02-23T00:45:11.161557+0000","last_peered":"2024-02-23T00:45:11.161557+0000","last_clean":"2024-02-23T00:45:11.161557+0000","last_became_active":"2024-02-23T00:42:58.726861+0000","last_became_peered":"2024-02-23T00:42:58.726861+0000","last_unstale":"2024-02-23T00:45:11.161557+0000","last_undegraded":"2024-02-23T00:45:11.161557+0000","last_fullsized":"2024-02-23T00:45:11.161557+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":145,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.165741+0000","last_change":"2024-02-23T00:45:04.621794+0000","last_active":"2024-02-23T00:45:11.165741+0000","last_peered":"2024-02-23T00:45:11.165741+0000","last_clean":"2024-02-23T00:45:11.165741+0000","last_became_active":"2024-02-23T00:42:58.664282+0000","last_became_peered":"2024-02-23T00:42:58.664282+0000","last_unstale":"2024-02-23T00:45:11.165741+0000","last_undegraded":"2024-02-23T00:45:11.165741+0000","last_fullsized":"2024-02-23T00:45:11.165741+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:04.621714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.621714+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":199,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.148897+0000","last_change":"2024-02-23T00:43:51.261426+0000","last_active":"2024-02-23T00:45:11.148897+0000","last_peered":"2024-02-23T00:45:11.148897+0000","last_clean":"2024-02-23T00:45:11.148897+0000","last_became_active":"2024-02-23T00:39:18.796323+0000","last_became_peered":"2024-02-23T00:39:18.796323+0000","last_unstale":"2024-02-23T00:45:11.148897+0000","last_undegraded":"2024-02-23T00:45:11.148897+0000","last_fullsized":"2024-02-23T00:45:11.148897+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:51.261367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_clean_scrub_stamp":"2024-02-23T00:43:51.261367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"68'25","reported_seq":250,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.143867+0000","last_change":"2024-02-23T00:43:46.158231+0000","last_active":"2024-02-23T00:45:11.143867+0000","last_peered":"2024-02-23T00:45:11.143867+0000","last_clean":"2024-02-23T00:45:11.143867+0000","last_became_active":"2024-02-23T00:39:16.783926+0000","last_became_peered":"2024-02-23T00:39:16.783926+0000","last_unstale":"2024-02-23T00:45:11.143867+0000","last_undegraded":"2024-02-23T00:45:11.143867+0000","last_fullsized":"2024-02-23T00:45:11.143867+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'25","last_scrub_stamp":"2024-02-23T00:43:46.158149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:46.158149+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"95'4","reported_seq":218,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.143421+0000","last_change":"2024-02-23T00:44:31.106105+0000","last_active":"2024-02-23T00:45:11.143421+0000","last_peered":"2024-02-23T00:45:11.143421+0000","last_clean":"2024-02-23T00:45:11.143421+0000","last_became_active":"2024-02-23T00:39:14.779205+0000","last_became_peered":"2024-02-23T00:39:14.779205+0000","last_unstale":"2024-02-23T00:45:11.143421+0000","last_undegraded":"2024-02-23T00:45:11.143421+0000","last_fullsized":"2024-02-23T00:45:11.143421+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'4","last_scrub_stamp":"2024-02-23T00:44:31.106012+0000","last_deep_scrub":"95'4","last_deep_scrub_stamp":"2024-02-23T00:41:43.874397+0000","last_clean_scrub_stamp":"2024-02-23T00:44:31.106012+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":216,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.142972+0000","last_change":"2024-02-23T00:43:52.036315+0000","last_active":"2024-02-23T00:45:11.142972+0000","last_peered":"2024-02-23T00:45:11.142972+0000","last_clean":"2024-02-23T00:45:11.142972+0000","last_became_active":"2024-02-23T00:43:27.611660+0000","last_became_peered":"2024-02-23T00:43:27.611660+0000","last_unstale":"2024-02-23T00:45:11.142972+0000","last_undegraded":"2024-02-23T00:45:11.142972+0000","last_fullsized":"2024-02-23T00:45:11.142972+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:52.036249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:52.036249+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.149475+0000","last_change":"2024-02-23T00:42:58.810468+0000","last_active":"2024-02-23T00:45:11.149475+0000","last_peered":"2024-02-23T00:45:11.149475+0000","last_clean":"2024-02-23T00:45:11.149475+0000","last_became_active":"2024-02-23T00:42:58.809926+0000","last_became_peered":"2024-02-23T00:42:58.809926+0000","last_unstale":"2024-02-23T00:45:11.149475+0000","last_undegraded":"2024-02-23T00:45:11.149475+0000","last_fullsized":"2024-02-23T00:45:11.149475+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.158081+0000","last_change":"2024-02-23T00:42:58.755160+0000","last_active":"2024-02-23T00:45:11.158081+0000","last_peered":"2024-02-23T00:45:11.158081+0000","last_clean":"2024-02-23T00:45:11.158081+0000","last_became_active":"2024-02-23T00:42:58.744983+0000","last_became_peered":"2024-02-23T00:42:58.744983+0000","last_unstale":"2024-02-23T00:45:11.158081+0000","last_undegraded":"2024-02-23T00:45:11.158081+0000","last_fullsized":"2024-02-23T00:45:11.158081+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"68'21","reported_seq":243,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.150020+0000","last_change":"2024-02-23T00:43:52.309599+0000","last_active":"2024-02-23T00:45:11.150020+0000","last_peered":"2024-02-23T00:45:11.150020+0000","last_clean":"2024-02-23T00:45:11.150020+0000","last_became_active":"2024-02-23T00:39:16.858760+0000","last_became_peered":"2024-02-23T00:39:16.858760+0000","last_unstale":"2024-02-23T00:45:11.150020+0000","last_undegraded":"2024-02-23T00:45:11.150020+0000","last_fullsized":"2024-02-23T00:45:11.150020+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'21","last_scrub_stamp":"2024-02-23T00:43:52.309532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:52.309532+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":203,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.148860+0000","last_change":"2024-02-23T00:42:52.317469+0000","last_active":"2024-02-23T00:45:11.148860+0000","last_peered":"2024-02-23T00:45:11.148860+0000","last_clean":"2024-02-23T00:45:11.148860+0000","last_became_active":"2024-02-23T00:39:14.774244+0000","last_became_peered":"2024-02-23T00:39:14.774244+0000","last_unstale":"2024-02-23T00:45:11.148860+0000","last_undegraded":"2024-02-23T00:45:11.148860+0000","last_fullsized":"2024-02-23T00:45:11.148860+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:52.317403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:52.317403+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":200,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.143033+0000","last_change":"2024-02-23T00:43:44.951800+0000","last_active":"2024-02-23T00:45:11.143033+0000","last_peered":"2024-02-23T00:45:11.143033+0000","last_clean":"2024-02-23T00:45:11.143033+0000","last_became_active":"2024-02-23T00:39:18.798624+0000","last_became_peered":"2024-02-23T00:39:18.798624+0000","last_unstale":"2024-02-23T00:45:11.143033+0000","last_undegraded":"2024-02-23T00:45:11.143033+0000","last_fullsized":"2024-02-23T00:45:11.143033+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:44.951696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:46.998875+0000","last_clean_scrub_stamp":"2024-02-23T00:43:44.951696+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":197,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.153254+0000","last_change":"2024-02-23T00:44:26.432766+0000","last_active":"2024-02-23T00:45:11.153254+0000","last_peered":"2024-02-23T00:45:11.153254+0000","last_clean":"2024-02-23T00:45:11.153254+0000","last_became_active":"2024-02-23T00:39:20.796041+0000","last_became_peered":"2024-02-23T00:39:20.796041+0000","last_unstale":"2024-02-23T00:45:11.153254+0000","last_undegraded":"2024-02-23T00:45:11.153254+0000","last_fullsized":"2024-02-23T00:45:11.153254+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:26.432703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:26.432703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1d","version":"110'7","reported_seq":147,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.144029+0000","last_change":"2024-02-23T00:42:58.755048+0000","last_active":"2024-02-23T00:45:11.144029+0000","last_peered":"2024-02-23T00:45:11.144029+0000","last_clean":"2024-02-23T00:45:11.144029+0000","last_became_active":"2024-02-23T00:42:58.754763+0000","last_became_peered":"2024-02-23T00:42:58.754763+0000","last_unstale":"2024-02-23T00:45:11.144029+0000","last_undegraded":"2024-02-23T00:45:11.144029+0000","last_fullsized":"2024-02-23T00:45:11.144029+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":145,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.143288+0000","last_change":"2024-02-23T00:44:26.595443+0000","last_active":"2024-02-23T00:45:11.143288+0000","last_peered":"2024-02-23T00:45:11.143288+0000","last_clean":"2024-02-23T00:45:11.143288+0000","last_became_active":"2024-02-23T00:42:58.825015+0000","last_became_peered":"2024-02-23T00:42:58.825015+0000","last_unstale":"2024-02-23T00:45:11.143288+0000","last_undegraded":"2024-02-23T00:45:11.143288+0000","last_fullsized":"2024-02-23T00:45:11.143288+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:26.595346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:44:26.595346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"107'16","reported_seq":230,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.142951+0000","last_change":"2024-02-23T00:45:07.012086+0000","last_active":"2024-02-23T00:45:11.142951+0000","last_peered":"2024-02-23T00:45:11.142951+0000","last_clean":"2024-02-23T00:45:11.142951+0000","last_became_active":"2024-02-23T00:39:16.858449+0000","last_became_peered":"2024-02-23T00:39:16.858449+0000","last_unstale":"2024-02-23T00:45:11.142951+0000","last_undegraded":"2024-02-23T00:45:11.142951+0000","last_fullsized":"2024-02-23T00:45:11.142951+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'16","last_scrub_stamp":"2024-02-23T00:45:07.011956+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-02-23T00:42:52.964510+0000","last_clean_scrub_stamp":"2024-02-23T00:45:07.011956+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":198,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.161154+0000","last_change":"2024-02-23T00:42:41.429191+0000","last_active":"2024-02-23T00:45:11.161154+0000","last_peered":"2024-02-23T00:45:11.161154+0000","last_clean":"2024-02-23T00:45:11.161154+0000","last_became_active":"2024-02-23T00:39:14.783164+0000","last_became_peered":"2024-02-23T00:39:14.783164+0000","last_unstale":"2024-02-23T00:45:11.161154+0000","last_undegraded":"2024-02-23T00:45:11.161154+0000","last_fullsized":"2024-02-23T00:45:11.161154+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:41.429079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:41.429079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":200,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.160988+0000","last_change":"2024-02-23T00:43:40.971316+0000","last_active":"2024-02-23T00:45:11.160988+0000","last_peered":"2024-02-23T00:45:11.160988+0000","last_clean":"2024-02-23T00:45:11.160988+0000","last_became_active":"2024-02-23T00:39:18.800297+0000","last_became_peered":"2024-02-23T00:39:18.800297+0000","last_unstale":"2024-02-23T00:45:11.160988+0000","last_undegraded":"2024-02-23T00:45:11.160988+0000","last_fullsized":"2024-02-23T00:45:11.160988+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:40.971243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:40.971243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":196,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.156648+0000","last_change":"2024-02-23T00:43:46.760431+0000","last_active":"2024-02-23T00:45:11.156648+0000","last_peered":"2024-02-23T00:45:11.156648+0000","last_clean":"2024-02-23T00:45:11.156648+0000","last_became_active":"2024-02-23T00:39:21.162036+0000","last_became_peered":"2024-02-23T00:39:21.162036+0000","last_unstale":"2024-02-23T00:45:11.156648+0000","last_undegraded":"2024-02-23T00:45:11.156648+0000","last_fullsized":"2024-02-23T00:45:11.156648+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:46.760348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:46.760348+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.158780+0000","last_change":"2024-02-23T00:42:57.649841+0000","last_active":"2024-02-23T00:45:11.158780+0000","last_peered":"2024-02-23T00:45:11.158780+0000","last_clean":"2024-02-23T00:45:11.158780+0000","last_became_active":"2024-02-23T00:42:57.648267+0000","last_became_peered":"2024-02-23T00:42:57.648267+0000","last_unstale":"2024-02-23T00:45:11.158780+0000","last_undegraded":"2024-02-23T00:45:11.158780+0000","last_fullsized":"2024-02-23T00:45:11.158780+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1","version":"110'7","reported_seq":153,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.144710+0000","last_change":"2024-02-23T00:42:58.782380+0000","last_active":"2024-02-23T00:45:11.144710+0000","last_peered":"2024-02-23T00:45:11.144710+0000","last_clean":"2024-02-23T00:45:11.144710+0000","last_became_active":"2024-02-23T00:42:58.777251+0000","last_became_peered":"2024-02-23T00:42:58.777251+0000","last_unstale":"2024-02-23T00:45:11.144710+0000","last_undegraded":"2024-02-23T00:45:11.144710+0000","last_fullsized":"2024-02-23T00:45:11.144710+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"68'22","reported_seq":171,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.152948+0000","last_change":"2024-02-23T00:44:31.391929+0000","last_active":"2024-02-23T00:45:11.152948+0000","last_peered":"2024-02-23T00:45:11.152948+0000","last_clean":"2024-02-23T00:45:11.152948+0000","last_became_active":"2024-02-23T00:42:43.521189+0000","last_became_peered":"2024-02-23T00:42:43.521189+0000","last_unstale":"2024-02-23T00:45:11.152948+0000","last_undegraded":"2024-02-23T00:45:11.152948+0000","last_fullsized":"2024-02-23T00:45:11.152948+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'22","last_scrub_stamp":"2024-02-23T00:44:31.391819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:44:31.391819+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":37,"num_read_kb":30,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,7],"acting":[2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":204,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.159352+0000","last_change":"2024-02-23T00:44:18.946624+0000","last_active":"2024-02-23T00:45:11.159352+0000","last_peered":"2024-02-23T00:45:11.159352+0000","last_clean":"2024-02-23T00:45:11.159352+0000","last_became_active":"2024-02-23T00:39:14.785590+0000","last_became_peered":"2024-02-23T00:39:14.785590+0000","last_unstale":"2024-02-23T00:45:11.159352+0000","last_undegraded":"2024-02-23T00:45:11.159352+0000","last_fullsized":"2024-02-23T00:45:11.159352+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:18.946560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_clean_scrub_stamp":"2024-02-23T00:44:18.946560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"67'8","reported_seq":279,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.161870+0000","last_change":"2024-02-23T00:44:29.038226+0000","last_active":"2024-02-23T00:45:11.161870+0000","last_peered":"2024-02-23T00:45:11.161870+0000","last_clean":"2024-02-23T00:45:11.161870+0000","last_became_active":"2024-02-23T00:39:18.803566+0000","last_became_peered":"2024-02-23T00:39:18.803566+0000","last_unstale":"2024-02-23T00:45:11.161870+0000","last_undegraded":"2024-02-23T00:45:11.161870+0000","last_fullsized":"2024-02-23T00:45:11.161870+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:44:29.038134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:29.038134+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"110'5","reported_seq":145,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.143236+0000","last_change":"2024-02-23T00:42:58.770053+0000","last_active":"2024-02-23T00:45:11.143236+0000","last_peered":"2024-02-23T00:45:11.143236+0000","last_clean":"2024-02-23T00:45:11.143236+0000","last_became_active":"2024-02-23T00:42:58.758349+0000","last_became_peered":"2024-02-23T00:42:58.758349+0000","last_unstale":"2024-02-23T00:45:11.143236+0000","last_undegraded":"2024-02-23T00:45:11.143236+0000","last_fullsized":"2024-02-23T00:45:11.143236+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.155622+0000","last_change":"2024-02-23T00:42:58.766295+0000","last_active":"2024-02-23T00:45:11.155622+0000","last_peered":"2024-02-23T00:45:11.155622+0000","last_clean":"2024-02-23T00:45:11.155622+0000","last_became_active":"2024-02-23T00:42:58.757424+0000","last_became_peered":"2024-02-23T00:42:58.757424+0000","last_unstale":"2024-02-23T00:45:11.155622+0000","last_undegraded":"2024-02-23T00:45:11.155622+0000","last_fullsized":"2024-02-23T00:45:11.155622+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"68'15","reported_seq":233,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.161715+0000","last_change":"2024-02-23T00:44:25.998879+0000","last_active":"2024-02-23T00:45:11.161715+0000","last_peered":"2024-02-23T00:45:11.161715+0000","last_clean":"2024-02-23T00:45:11.161715+0000","last_became_active":"2024-02-23T00:39:16.858953+0000","last_became_peered":"2024-02-23T00:39:16.858953+0000","last_unstale":"2024-02-23T00:45:11.161715+0000","last_undegraded":"2024-02-23T00:45:11.161715+0000","last_fullsized":"2024-02-23T00:45:11.161715+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:44:25.998804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:44:25.998804+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":204,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.145932+0000","last_change":"2024-02-23T00:43:45.309194+0000","last_active":"2024-02-23T00:45:11.145932+0000","last_peered":"2024-02-23T00:45:11.145932+0000","last_clean":"2024-02-23T00:45:11.145932+0000","last_became_active":"2024-02-23T00:39:14.773772+0000","last_became_peered":"2024-02-23T00:39:14.773772+0000","last_unstale":"2024-02-23T00:45:11.145932+0000","last_undegraded":"2024-02-23T00:45:11.145932+0000","last_fullsized":"2024-02-23T00:45:11.145932+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:45.309083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.309083+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"67'8","reported_seq":274,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.145616+0000","last_change":"2024-02-23T00:43:39.086199+0000","last_active":"2024-02-23T00:45:11.145616+0000","last_peered":"2024-02-23T00:45:11.145616+0000","last_clean":"2024-02-23T00:45:11.145616+0000","last_became_active":"2024-02-23T00:39:18.787313+0000","last_became_peered":"2024-02-23T00:39:18.787313+0000","last_unstale":"2024-02-23T00:45:11.145616+0000","last_undegraded":"2024-02-23T00:45:11.145616+0000","last_fullsized":"2024-02-23T00:45:11.145616+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:43:39.086107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:39.086107+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":145,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.154410+0000","last_change":"2024-02-23T00:44:34.437892+0000","last_active":"2024-02-23T00:45:11.154410+0000","last_peered":"2024-02-23T00:45:11.154410+0000","last_clean":"2024-02-23T00:45:11.154410+0000","last_became_active":"2024-02-23T00:42:58.745471+0000","last_became_peered":"2024-02-23T00:42:58.745471+0000","last_unstale":"2024-02-23T00:45:11.154410+0000","last_undegraded":"2024-02-23T00:45:11.154410+0000","last_fullsized":"2024-02-23T00:45:11.154410+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:34.437785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:34.437785+0000","last_clean_scrub_stamp":"2024-02-23T00:44:34.437785+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.156638+0000","last_change":"2024-02-23T00:42:57.652853+0000","last_active":"2024-02-23T00:45:11.156638+0000","last_peered":"2024-02-23T00:45:11.156638+0000","last_clean":"2024-02-23T00:45:11.156638+0000","last_became_active":"2024-02-23T00:42:57.652488+0000","last_became_peered":"2024-02-23T00:42:57.652488+0000","last_unstale":"2024-02-23T00:45:11.156638+0000","last_undegraded":"2024-02-23T00:45:11.156638+0000","last_fullsized":"2024-02-23T00:45:11.156638+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"67'9","reported_seq":212,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.160003+0000","last_change":"2024-02-23T00:42:57.705378+0000","last_active":"2024-02-23T00:45:11.160003+0000","last_peered":"2024-02-23T00:45:11.160003+0000","last_clean":"2024-02-23T00:45:11.160003+0000","last_became_active":"2024-02-23T00:39:16.786936+0000","last_became_peered":"2024-02-23T00:39:16.786936+0000","last_unstale":"2024-02-23T00:45:11.160003+0000","last_undegraded":"2024-02-23T00:45:11.160003+0000","last_fullsized":"2024-02-23T00:45:11.160003+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'9","last_scrub_stamp":"2024-02-23T00:42:57.705261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.705261+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":208,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.165602+0000","last_change":"2024-02-23T00:44:24.350861+0000","last_active":"2024-02-23T00:45:11.165602+0000","last_peered":"2024-02-23T00:45:11.165602+0000","last_clean":"2024-02-23T00:45:11.165602+0000","last_became_active":"2024-02-23T00:39:14.778455+0000","last_became_peered":"2024-02-23T00:39:14.778455+0000","last_unstale":"2024-02-23T00:45:11.165602+0000","last_undegraded":"2024-02-23T00:45:11.165602+0000","last_fullsized":"2024-02-23T00:45:11.165602+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:24.350797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:24.350797+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":200,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.142884+0000","last_change":"2024-02-23T00:44:37.105079+0000","last_active":"2024-02-23T00:45:11.142884+0000","last_peered":"2024-02-23T00:45:11.142884+0000","last_clean":"2024-02-23T00:45:11.142884+0000","last_became_active":"2024-02-23T00:39:18.799052+0000","last_became_peered":"2024-02-23T00:39:18.799052+0000","last_unstale":"2024-02-23T00:45:11.142884+0000","last_undegraded":"2024-02-23T00:45:11.142884+0000","last_fullsized":"2024-02-23T00:45:11.142884+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:37.105013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_clean_scrub_stamp":"2024-02-23T00:44:37.105013+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":193,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.142994+0000","last_change":"2024-02-23T00:42:50.425748+0000","last_active":"2024-02-23T00:45:11.142994+0000","last_peered":"2024-02-23T00:45:11.142994+0000","last_clean":"2024-02-23T00:45:11.142994+0000","last_became_active":"2024-02-23T00:39:20.802648+0000","last_became_peered":"2024-02-23T00:39:20.802648+0000","last_unstale":"2024-02-23T00:45:11.142994+0000","last_undegraded":"2024-02-23T00:45:11.142994+0000","last_fullsized":"2024-02-23T00:45:11.142994+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:50.425641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:50.425641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.143285+0000","last_change":"2024-02-23T00:42:58.834611+0000","last_active":"2024-02-23T00:45:11.143285+0000","last_peered":"2024-02-23T00:45:11.143285+0000","last_clean":"2024-02-23T00:45:11.143285+0000","last_became_active":"2024-02-23T00:42:58.833531+0000","last_became_peered":"2024-02-23T00:42:58.833531+0000","last_unstale":"2024-02-23T00:45:11.143285+0000","last_undegraded":"2024-02-23T00:45:11.143285+0000","last_fullsized":"2024-02-23T00:45:11.143285+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":145,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.143782+0000","last_change":"2024-02-23T00:44:23.543049+0000","last_active":"2024-02-23T00:45:11.143782+0000","last_peered":"2024-02-23T00:45:11.143782+0000","last_clean":"2024-02-23T00:45:11.143782+0000","last_became_active":"2024-02-23T00:42:58.835862+0000","last_became_peered":"2024-02-23T00:42:58.835862+0000","last_unstale":"2024-02-23T00:45:11.143782+0000","last_undegraded":"2024-02-23T00:45:11.143782+0000","last_fullsized":"2024-02-23T00:45:11.143782+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:23.542975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.542975+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"67'11","reported_seq":215,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.143037+0000","last_change":"2024-02-23T00:42:58.013765+0000","last_active":"2024-02-23T00:45:11.143037+0000","last_peered":"2024-02-23T00:45:11.143037+0000","last_clean":"2024-02-23T00:45:11.143037+0000","last_became_active":"2024-02-23T00:39:16.783896+0000","last_became_peered":"2024-02-23T00:39:16.783896+0000","last_unstale":"2024-02-23T00:45:11.143037+0000","last_undegraded":"2024-02-23T00:45:11.143037+0000","last_fullsized":"2024-02-23T00:45:11.143037+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'11","last_scrub_stamp":"2024-02-23T00:42:58.013688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:58.013688+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"202'2","reported_seq":211,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.159561+0000","last_change":"2024-02-23T00:44:34.720705+0000","last_active":"2024-02-23T00:45:11.159561+0000","last_peered":"2024-02-23T00:45:11.159561+0000","last_clean":"2024-02-23T00:45:11.159561+0000","last_became_active":"2024-02-23T00:39:14.775804+0000","last_became_peered":"2024-02-23T00:39:14.775804+0000","last_unstale":"2024-02-23T00:45:11.159561+0000","last_undegraded":"2024-02-23T00:45:11.159561+0000","last_fullsized":"2024-02-23T00:45:11.159561+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"202'2","last_scrub_stamp":"2024-02-23T00:44:34.720640+0000","last_deep_scrub":"202'2","last_deep_scrub_stamp":"2024-02-23T00:44:34.720640+0000","last_clean_scrub_stamp":"2024-02-23T00:44:34.720640+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":194,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.163631+0000","last_change":"2024-02-23T00:43:31.903157+0000","last_active":"2024-02-23T00:45:11.163631+0000","last_peered":"2024-02-23T00:45:11.163631+0000","last_clean":"2024-02-23T00:45:11.163631+0000","last_became_active":"2024-02-23T00:39:18.800382+0000","last_became_peered":"2024-02-23T00:39:18.800382+0000","last_unstale":"2024-02-23T00:45:11.163631+0000","last_undegraded":"2024-02-23T00:45:11.163631+0000","last_fullsized":"2024-02-23T00:45:11.163631+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:31.903031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:31.903031+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":198,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.144262+0000","last_change":"2024-02-23T00:44:23.152677+0000","last_active":"2024-02-23T00:45:11.144262+0000","last_peered":"2024-02-23T00:45:11.144262+0000","last_clean":"2024-02-23T00:45:11.144262+0000","last_became_active":"2024-02-23T00:39:20.799053+0000","last_became_peered":"2024-02-23T00:39:20.799053+0000","last_unstale":"2024-02-23T00:45:11.144262+0000","last_undegraded":"2024-02-23T00:45:11.144262+0000","last_fullsized":"2024-02-23T00:45:11.144262+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:23.152602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.152602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"110'2","reported_seq":147,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.146590+0000","last_change":"2024-02-23T00:45:09.917568+0000","last_active":"2024-02-23T00:45:11.146590+0000","last_peered":"2024-02-23T00:45:11.146590+0000","last_clean":"2024-02-23T00:45:11.146590+0000","last_became_active":"2024-02-23T00:42:58.698725+0000","last_became_peered":"2024-02-23T00:42:58.698725+0000","last_unstale":"2024-02-23T00:45:11.146590+0000","last_undegraded":"2024-02-23T00:45:11.146590+0000","last_fullsized":"2024-02-23T00:45:11.146590+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'2","last_scrub_stamp":"2024-02-23T00:45:09.917420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:09.917420+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.159549+0000","last_change":"2024-02-23T00:42:57.652742+0000","last_active":"2024-02-23T00:45:11.159549+0000","last_peered":"2024-02-23T00:45:11.159549+0000","last_clean":"2024-02-23T00:45:11.159549+0000","last_became_active":"2024-02-23T00:42:57.652438+0000","last_became_peered":"2024-02-23T00:42:57.652438+0000","last_unstale":"2024-02-23T00:45:11.159549+0000","last_undegraded":"2024-02-23T00:45:11.159549+0000","last_fullsized":"2024-02-23T00:45:11.159549+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"67'4","reported_seq":203,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.146510+0000","last_change":"2024-02-23T00:42:42.934024+0000","last_active":"2024-02-23T00:45:11.146510+0000","last_peered":"2024-02-23T00:45:11.146510+0000","last_clean":"2024-02-23T00:45:11.146510+0000","last_became_active":"2024-02-23T00:39:16.785830+0000","last_became_peered":"2024-02-23T00:39:16.785830+0000","last_unstale":"2024-02-23T00:45:11.146510+0000","last_undegraded":"2024-02-23T00:45:11.146510+0000","last_fullsized":"2024-02-23T00:45:11.146510+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'4","last_scrub_stamp":"2024-02-23T00:42:42.933958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:42.933958+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":204,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.160847+0000","last_change":"2024-02-23T00:43:38.947741+0000","last_active":"2024-02-23T00:45:11.160847+0000","last_peered":"2024-02-23T00:45:11.160847+0000","last_clean":"2024-02-23T00:45:11.160847+0000","last_became_active":"2024-02-23T00:39:14.783325+0000","last_became_peered":"2024-02-23T00:39:14.783325+0000","last_unstale":"2024-02-23T00:45:11.160847+0000","last_undegraded":"2024-02-23T00:45:11.160847+0000","last_fullsized":"2024-02-23T00:45:11.160847+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:38.947675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:38.947675+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":199,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.161012+0000","last_change":"2024-02-23T00:42:57.574460+0000","last_active":"2024-02-23T00:45:11.161012+0000","last_peered":"2024-02-23T00:45:11.161012+0000","last_clean":"2024-02-23T00:45:11.161012+0000","last_became_active":"2024-02-23T00:39:18.797287+0000","last_became_peered":"2024-02-23T00:39:18.797287+0000","last_unstale":"2024-02-23T00:45:11.161012+0000","last_undegraded":"2024-02-23T00:45:11.161012+0000","last_fullsized":"2024-02-23T00:45:11.161012+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:57.574369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.574369+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":196,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.155282+0000","last_change":"2024-02-23T00:45:10.741762+0000","last_active":"2024-02-23T00:45:11.155282+0000","last_peered":"2024-02-23T00:45:11.155282+0000","last_clean":"2024-02-23T00:45:11.155282+0000","last_became_active":"2024-02-23T00:39:20.795351+0000","last_became_peered":"2024-02-23T00:39:20.795351+0000","last_unstale":"2024-02-23T00:45:11.155282+0000","last_undegraded":"2024-02-23T00:45:11.155282+0000","last_fullsized":"2024-02-23T00:45:11.155282+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.741694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.741694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1b","version":"110'10","reported_seq":150,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.155947+0000","last_change":"2024-02-23T00:42:58.749680+0000","last_active":"2024-02-23T00:45:11.155947+0000","last_peered":"2024-02-23T00:45:11.155947+0000","last_clean":"2024-02-23T00:45:11.155947+0000","last_became_active":"2024-02-23T00:42:58.749126+0000","last_became_peered":"2024-02-23T00:42:58.749126+0000","last_unstale":"2024-02-23T00:45:11.155947+0000","last_undegraded":"2024-02-23T00:45:11.155947+0000","last_fullsized":"2024-02-23T00:45:11.155947+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.159420+0000","last_change":"2024-02-23T00:42:58.741674+0000","last_active":"2024-02-23T00:45:11.159420+0000","last_peered":"2024-02-23T00:45:11.159420+0000","last_clean":"2024-02-23T00:45:11.159420+0000","last_became_active":"2024-02-23T00:42:58.741365+0000","last_became_peered":"2024-02-23T00:42:58.741365+0000","last_unstale":"2024-02-23T00:45:11.159420+0000","last_undegraded":"2024-02-23T00:45:11.159420+0000","last_fullsized":"2024-02-23T00:45:11.159420+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"68'17","reported_seq":236,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.144914+0000","last_change":"2024-02-23T00:43:43.997579+0000","last_active":"2024-02-23T00:45:11.144914+0000","last_peered":"2024-02-23T00:45:11.144914+0000","last_clean":"2024-02-23T00:45:11.144914+0000","last_became_active":"2024-02-23T00:39:16.857232+0000","last_became_peered":"2024-02-23T00:39:16.857232+0000","last_unstale":"2024-02-23T00:45:11.144914+0000","last_undegraded":"2024-02-23T00:45:11.144914+0000","last_fullsized":"2024-02-23T00:45:11.144914+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:43:43.997452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:43.997452+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":203,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.164833+0000","last_change":"2024-02-23T00:43:52.419206+0000","last_active":"2024-02-23T00:45:11.164833+0000","last_peered":"2024-02-23T00:45:11.164833+0000","last_clean":"2024-02-23T00:45:11.164833+0000","last_became_active":"2024-02-23T00:39:14.784092+0000","last_became_peered":"2024-02-23T00:39:14.784092+0000","last_unstale":"2024-02-23T00:45:11.164833+0000","last_undegraded":"2024-02-23T00:45:11.164833+0000","last_fullsized":"2024-02-23T00:45:11.164833+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:52.419100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:52.419100+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":198,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.154624+0000","last_change":"2024-02-23T00:44:25.797288+0000","last_active":"2024-02-23T00:45:11.154624+0000","last_peered":"2024-02-23T00:45:11.154624+0000","last_clean":"2024-02-23T00:45:11.154624+0000","last_became_active":"2024-02-23T00:39:18.791457+0000","last_became_peered":"2024-02-23T00:39:18.791457+0000","last_unstale":"2024-02-23T00:45:11.154624+0000","last_undegraded":"2024-02-23T00:45:11.154624+0000","last_fullsized":"2024-02-23T00:45:11.154624+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:25.797184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:25.797184+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":193,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.143190+0000","last_change":"2024-02-23T00:43:45.965636+0000","last_active":"2024-02-23T00:45:11.143190+0000","last_peered":"2024-02-23T00:45:11.143190+0000","last_clean":"2024-02-23T00:45:11.143190+0000","last_became_active":"2024-02-23T00:39:21.160376+0000","last_became_peered":"2024-02-23T00:39:21.160376+0000","last_unstale":"2024-02-23T00:45:11.143190+0000","last_undegraded":"2024-02-23T00:45:11.143190+0000","last_fullsized":"2024-02-23T00:45:11.143190+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:45.965570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.965570+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1a","version":"110'5","reported_seq":145,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.159198+0000","last_change":"2024-02-23T00:42:58.738588+0000","last_active":"2024-02-23T00:45:11.159198+0000","last_peered":"2024-02-23T00:45:11.159198+0000","last_clean":"2024-02-23T00:45:11.159198+0000","last_became_active":"2024-02-23T00:42:58.736706+0000","last_became_peered":"2024-02-23T00:42:58.736706+0000","last_unstale":"2024-02-23T00:45:11.159198+0000","last_undegraded":"2024-02-23T00:45:11.159198+0000","last_fullsized":"2024-02-23T00:45:11.159198+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.154942+0000","last_change":"2024-02-23T00:42:58.705557+0000","last_active":"2024-02-23T00:45:11.154942+0000","last_peered":"2024-02-23T00:45:11.154942+0000","last_clean":"2024-02-23T00:45:11.154942+0000","last_became_active":"2024-02-23T00:42:58.702580+0000","last_became_peered":"2024-02-23T00:42:58.702580+0000","last_unstale":"2024-02-23T00:45:11.154942+0000","last_undegraded":"2024-02-23T00:45:11.154942+0000","last_fullsized":"2024-02-23T00:45:11.154942+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"68'16","reported_seq":232,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.150252+0000","last_change":"2024-02-23T00:45:04.672612+0000","last_active":"2024-02-23T00:45:11.150252+0000","last_peered":"2024-02-23T00:45:11.150252+0000","last_clean":"2024-02-23T00:45:11.150252+0000","last_became_active":"2024-02-23T00:39:16.858945+0000","last_became_peered":"2024-02-23T00:39:16.858945+0000","last_unstale":"2024-02-23T00:45:11.150252+0000","last_undegraded":"2024-02-23T00:45:11.150252+0000","last_fullsized":"2024-02-23T00:45:11.150252+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:45:04.672537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.672537+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":211,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.160102+0000","last_change":"2024-02-23T00:43:44.031862+0000","last_active":"2024-02-23T00:45:11.160102+0000","last_peered":"2024-02-23T00:45:11.160102+0000","last_clean":"2024-02-23T00:45:11.160102+0000","last_became_active":"2024-02-23T00:39:14.785363+0000","last_became_peered":"2024-02-23T00:39:14.785363+0000","last_unstale":"2024-02-23T00:45:11.160102+0000","last_undegraded":"2024-02-23T00:45:11.160102+0000","last_fullsized":"2024-02-23T00:45:11.160102+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"80'2","last_scrub_stamp":"2024-02-23T00:43:44.031793+0000","last_deep_scrub":"80'2","last_deep_scrub_stamp":"2024-02-23T00:40:15.942725+0000","last_clean_scrub_stamp":"2024-02-23T00:43:44.031793+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"93'9","reported_seq":282,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.165010+0000","last_change":"2024-02-23T00:43:50.393789+0000","last_active":"2024-02-23T00:45:11.165010+0000","last_peered":"2024-02-23T00:45:11.165010+0000","last_clean":"2024-02-23T00:45:11.165010+0000","last_became_active":"2024-02-23T00:39:18.796680+0000","last_became_peered":"2024-02-23T00:39:18.796680+0000","last_unstale":"2024-02-23T00:45:11.165010+0000","last_undegraded":"2024-02-23T00:45:11.165010+0000","last_fullsized":"2024-02-23T00:45:11.165010+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-02-23T00:43:50.393713+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-02-23T00:43:50.393713+0000","last_clean_scrub_stamp":"2024-02-23T00:43:50.393713+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.157145+0000","last_change":"2024-02-23T00:42:58.805923+0000","last_active":"2024-02-23T00:45:11.157145+0000","last_peered":"2024-02-23T00:45:11.157145+0000","last_clean":"2024-02-23T00:45:11.157145+0000","last_became_active":"2024-02-23T00:42:58.803900+0000","last_became_peered":"2024-02-23T00:42:58.803900+0000","last_unstale":"2024-02-23T00:45:11.157145+0000","last_undegraded":"2024-02-23T00:45:11.157145+0000","last_fullsized":"2024-02-23T00:45:11.157145+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.143597+0000","last_change":"2024-02-23T00:42:58.850530+0000","last_active":"2024-02-23T00:45:11.143597+0000","last_peered":"2024-02-23T00:45:11.143597+0000","last_clean":"2024-02-23T00:45:11.143597+0000","last_became_active":"2024-02-23T00:42:58.850072+0000","last_became_peered":"2024-02-23T00:42:58.850072+0000","last_unstale":"2024-02-23T00:45:11.143597+0000","last_undegraded":"2024-02-23T00:45:11.143597+0000","last_fullsized":"2024-02-23T00:45:11.143597+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"93'23","reported_seq":246,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.165130+0000","last_change":"2024-02-23T00:44:28.387225+0000","last_active":"2024-02-23T00:45:11.165130+0000","last_peered":"2024-02-23T00:45:11.165130+0000","last_clean":"2024-02-23T00:45:11.165130+0000","last_became_active":"2024-02-23T00:39:16.782660+0000","last_became_peered":"2024-02-23T00:39:16.782660+0000","last_unstale":"2024-02-23T00:45:11.165130+0000","last_undegraded":"2024-02-23T00:45:11.165130+0000","last_fullsized":"2024-02-23T00:45:11.165130+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'23","last_scrub_stamp":"2024-02-23T00:44:28.387155+0000","last_deep_scrub":"93'23","last_deep_scrub_stamp":"2024-02-23T00:41:35.954516+0000","last_clean_scrub_stamp":"2024-02-23T00:44:28.387155+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":204,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.144386+0000","last_change":"2024-02-23T00:44:21.157372+0000","last_active":"2024-02-23T00:45:11.144386+0000","last_peered":"2024-02-23T00:45:11.144386+0000","last_clean":"2024-02-23T00:45:11.144386+0000","last_became_active":"2024-02-23T00:39:14.781560+0000","last_became_peered":"2024-02-23T00:39:14.781560+0000","last_unstale":"2024-02-23T00:45:11.144386+0000","last_undegraded":"2024-02-23T00:45:11.144386+0000","last_fullsized":"2024-02-23T00:45:11.144386+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:21.157316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:21.157316+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":200,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.150292+0000","last_change":"2024-02-23T00:43:49.238770+0000","last_active":"2024-02-23T00:45:11.150292+0000","last_peered":"2024-02-23T00:45:11.150292+0000","last_clean":"2024-02-23T00:45:11.150292+0000","last_became_active":"2024-02-23T00:39:18.790602+0000","last_became_peered":"2024-02-23T00:39:18.790602+0000","last_unstale":"2024-02-23T00:45:11.150292+0000","last_undegraded":"2024-02-23T00:45:11.150292+0000","last_fullsized":"2024-02-23T00:45:11.150292+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:49.238704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:53.189881+0000","last_clean_scrub_stamp":"2024-02-23T00:43:49.238704+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.14","version":"110'2","reported_seq":142,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.165710+0000","last_change":"2024-02-23T00:42:58.804637+0000","last_active":"2024-02-23T00:45:11.165710+0000","last_peered":"2024-02-23T00:45:11.165710+0000","last_clean":"2024-02-23T00:45:11.165710+0000","last_became_active":"2024-02-23T00:42:58.804344+0000","last_became_peered":"2024-02-23T00:42:58.804344+0000","last_unstale":"2024-02-23T00:45:11.165710+0000","last_undegraded":"2024-02-23T00:45:11.165710+0000","last_fullsized":"2024-02-23T00:45:11.165710+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.144461+0000","last_change":"2024-02-23T00:42:58.778321+0000","last_active":"2024-02-23T00:45:11.144461+0000","last_peered":"2024-02-23T00:45:11.144461+0000","last_clean":"2024-02-23T00:45:11.144461+0000","last_became_active":"2024-02-23T00:42:58.774720+0000","last_became_peered":"2024-02-23T00:42:58.774720+0000","last_unstale":"2024-02-23T00:45:11.144461+0000","last_undegraded":"2024-02-23T00:45:11.144461+0000","last_fullsized":"2024-02-23T00:45:11.144461+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"67'15","reported_seq":226,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.153343+0000","last_change":"2024-02-23T00:43:36.449809+0000","last_active":"2024-02-23T00:45:11.153343+0000","last_peered":"2024-02-23T00:45:11.153343+0000","last_clean":"2024-02-23T00:45:11.153343+0000","last_became_active":"2024-02-23T00:39:16.787138+0000","last_became_peered":"2024-02-23T00:39:16.787138+0000","last_unstale":"2024-02-23T00:45:11.153343+0000","last_undegraded":"2024-02-23T00:45:11.153343+0000","last_fullsized":"2024-02-23T00:45:11.153343+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:43:36.449687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:36.449687+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":203,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.166436+0000","last_change":"2024-02-23T00:43:45.362071+0000","last_active":"2024-02-23T00:45:11.166436+0000","last_peered":"2024-02-23T00:45:11.166436+0000","last_clean":"2024-02-23T00:45:11.166436+0000","last_became_active":"2024-02-23T00:39:14.782494+0000","last_became_peered":"2024-02-23T00:39:14.782494+0000","last_unstale":"2024-02-23T00:45:11.166436+0000","last_undegraded":"2024-02-23T00:45:11.166436+0000","last_fullsized":"2024-02-23T00:45:11.166436+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:45.361978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.361978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":199,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.159738+0000","last_change":"2024-02-23T00:45:03.615758+0000","last_active":"2024-02-23T00:45:11.159738+0000","last_peered":"2024-02-23T00:45:11.159738+0000","last_clean":"2024-02-23T00:45:11.159738+0000","last_became_active":"2024-02-23T00:39:18.801039+0000","last_became_peered":"2024-02-23T00:39:18.801039+0000","last_unstale":"2024-02-23T00:45:11.159738+0000","last_undegraded":"2024-02-23T00:45:11.159738+0000","last_fullsized":"2024-02-23T00:45:11.159738+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:03.615692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.641689+0000","last_clean_scrub_stamp":"2024-02-23T00:45:03.615692+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"110'5","reported_seq":145,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.143289+0000","last_change":"2024-02-23T00:42:58.783591+0000","last_active":"2024-02-23T00:45:11.143289+0000","last_peered":"2024-02-23T00:45:11.143289+0000","last_clean":"2024-02-23T00:45:11.143289+0000","last_became_active":"2024-02-23T00:42:58.783343+0000","last_became_peered":"2024-02-23T00:42:58.783343+0000","last_unstale":"2024-02-23T00:45:11.143289+0000","last_undegraded":"2024-02-23T00:45:11.143289+0000","last_fullsized":"2024-02-23T00:45:11.143289+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.145635+0000","last_change":"2024-02-23T00:42:58.773110+0000","last_active":"2024-02-23T00:45:11.145635+0000","last_peered":"2024-02-23T00:45:11.145635+0000","last_clean":"2024-02-23T00:45:11.145635+0000","last_became_active":"2024-02-23T00:42:58.770759+0000","last_became_peered":"2024-02-23T00:42:58.770759+0000","last_unstale":"2024-02-23T00:45:11.145635+0000","last_undegraded":"2024-02-23T00:45:11.145635+0000","last_fullsized":"2024-02-23T00:45:11.145635+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"68'18","reported_seq":240,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.143937+0000","last_change":"2024-02-23T00:43:51.079824+0000","last_active":"2024-02-23T00:45:11.143937+0000","last_peered":"2024-02-23T00:45:11.143937+0000","last_clean":"2024-02-23T00:45:11.143937+0000","last_became_active":"2024-02-23T00:39:16.782803+0000","last_became_peered":"2024-02-23T00:39:16.782803+0000","last_unstale":"2024-02-23T00:45:11.143937+0000","last_undegraded":"2024-02-23T00:45:11.143937+0000","last_fullsized":"2024-02-23T00:45:11.143937+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'18","last_scrub_stamp":"2024-02-23T00:43:51.079756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:51.079756+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":204,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.142808+0000","last_change":"2024-02-23T00:43:48.030906+0000","last_active":"2024-02-23T00:45:11.142808+0000","last_peered":"2024-02-23T00:45:11.142808+0000","last_clean":"2024-02-23T00:45:11.142808+0000","last_became_active":"2024-02-23T00:39:14.782163+0000","last_became_peered":"2024-02-23T00:39:14.782163+0000","last_unstale":"2024-02-23T00:45:11.142808+0000","last_undegraded":"2024-02-23T00:45:11.142808+0000","last_fullsized":"2024-02-23T00:45:11.142808+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:48.030846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.030846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":199,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.143984+0000","last_change":"2024-02-23T00:44:37.068899+0000","last_active":"2024-02-23T00:45:11.143984+0000","last_peered":"2024-02-23T00:45:11.143984+0000","last_clean":"2024-02-23T00:45:11.143984+0000","last_became_active":"2024-02-23T00:39:18.794903+0000","last_became_peered":"2024-02-23T00:39:18.794903+0000","last_unstale":"2024-02-23T00:45:11.143984+0000","last_undegraded":"2024-02-23T00:45:11.143984+0000","last_fullsized":"2024-02-23T00:45:11.143984+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:37.068834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_clean_scrub_stamp":"2024-02-23T00:44:37.068834+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"110'5","reported_seq":145,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.150348+0000","last_change":"2024-02-23T00:42:58.839927+0000","last_active":"2024-02-23T00:45:11.150348+0000","last_peered":"2024-02-23T00:45:11.150348+0000","last_clean":"2024-02-23T00:45:11.150348+0000","last_became_active":"2024-02-23T00:42:58.838833+0000","last_became_peered":"2024-02-23T00:42:58.838833+0000","last_unstale":"2024-02-23T00:45:11.150348+0000","last_undegraded":"2024-02-23T00:45:11.150348+0000","last_fullsized":"2024-02-23T00:45:11.150348+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.161203+0000","last_change":"2024-02-23T00:42:58.698567+0000","last_active":"2024-02-23T00:45:11.161203+0000","last_peered":"2024-02-23T00:45:11.161203+0000","last_clean":"2024-02-23T00:45:11.161203+0000","last_became_active":"2024-02-23T00:42:58.698258+0000","last_became_peered":"2024-02-23T00:42:58.698258+0000","last_unstale":"2024-02-23T00:45:11.161203+0000","last_undegraded":"2024-02-23T00:45:11.161203+0000","last_fullsized":"2024-02-23T00:45:11.161203+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"95'27","reported_seq":253,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.142839+0000","last_change":"2024-02-23T00:43:52.124336+0000","last_active":"2024-02-23T00:45:11.142839+0000","last_peered":"2024-02-23T00:45:11.142839+0000","last_clean":"2024-02-23T00:45:11.142839+0000","last_became_active":"2024-02-23T00:39:16.786791+0000","last_became_peered":"2024-02-23T00:39:16.786791+0000","last_unstale":"2024-02-23T00:45:11.142839+0000","last_undegraded":"2024-02-23T00:45:11.142839+0000","last_fullsized":"2024-02-23T00:45:11.142839+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'27","last_scrub_stamp":"2024-02-23T00:43:52.124247+0000","last_deep_scrub":"95'27","last_deep_scrub_stamp":"2024-02-23T00:41:55.919895+0000","last_clean_scrub_stamp":"2024-02-23T00:43:52.124247+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":202,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.156514+0000","last_change":"2024-02-23T00:44:27.759571+0000","last_active":"2024-02-23T00:45:11.156514+0000","last_peered":"2024-02-23T00:45:11.156514+0000","last_clean":"2024-02-23T00:45:11.156514+0000","last_became_active":"2024-02-23T00:39:14.787148+0000","last_became_peered":"2024-02-23T00:39:14.787148+0000","last_unstale":"2024-02-23T00:45:11.156514+0000","last_undegraded":"2024-02-23T00:45:11.156514+0000","last_fullsized":"2024-02-23T00:45:11.156514+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:27.759499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:27.759499+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":200,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.161056+0000","last_change":"2024-02-23T00:45:10.090353+0000","last_active":"2024-02-23T00:45:11.161056+0000","last_peered":"2024-02-23T00:45:11.161056+0000","last_clean":"2024-02-23T00:45:11.161056+0000","last_became_active":"2024-02-23T00:39:18.796431+0000","last_became_peered":"2024-02-23T00:39:18.796431+0000","last_unstale":"2024-02-23T00:45:11.161056+0000","last_undegraded":"2024-02-23T00:45:11.161056+0000","last_fullsized":"2024-02-23T00:45:11.161056+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.090270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.090270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.11","version":"110'7","reported_seq":152,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.149198+0000","last_change":"2024-02-23T00:44:30.366468+0000","last_active":"2024-02-23T00:45:11.149198+0000","last_peered":"2024-02-23T00:45:11.149198+0000","last_clean":"2024-02-23T00:45:11.149198+0000","last_became_active":"2024-02-23T00:42:58.810765+0000","last_became_peered":"2024-02-23T00:42:58.810765+0000","last_unstale":"2024-02-23T00:45:11.149198+0000","last_undegraded":"2024-02-23T00:45:11.149198+0000","last_fullsized":"2024-02-23T00:45:11.149198+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'7","last_scrub_stamp":"2024-02-23T00:44:30.366391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:44:30.366391+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.159437+0000","last_change":"2024-02-23T00:42:58.833221+0000","last_active":"2024-02-23T00:45:11.159437+0000","last_peered":"2024-02-23T00:45:11.159437+0000","last_clean":"2024-02-23T00:45:11.159437+0000","last_became_active":"2024-02-23T00:42:58.832885+0000","last_became_peered":"2024-02-23T00:42:58.832885+0000","last_unstale":"2024-02-23T00:45:11.159437+0000","last_undegraded":"2024-02-23T00:45:11.159437+0000","last_fullsized":"2024-02-23T00:45:11.159437+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"82'15","reported_seq":230,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.143153+0000","last_change":"2024-02-23T00:44:35.095478+0000","last_active":"2024-02-23T00:45:11.143153+0000","last_peered":"2024-02-23T00:45:11.143153+0000","last_clean":"2024-02-23T00:45:11.143153+0000","last_became_active":"2024-02-23T00:39:16.782762+0000","last_became_peered":"2024-02-23T00:39:16.782762+0000","last_unstale":"2024-02-23T00:45:11.143153+0000","last_undegraded":"2024-02-23T00:45:11.143153+0000","last_fullsized":"2024-02-23T00:45:11.143153+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'15","last_scrub_stamp":"2024-02-23T00:44:35.095409+0000","last_deep_scrub":"82'15","last_deep_scrub_stamp":"2024-02-23T00:41:50.975683+0000","last_clean_scrub_stamp":"2024-02-23T00:44:35.095409+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":202,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.156737+0000","last_change":"2024-02-23T00:43:44.794995+0000","last_active":"2024-02-23T00:45:11.156737+0000","last_peered":"2024-02-23T00:45:11.156737+0000","last_clean":"2024-02-23T00:45:11.156737+0000","last_became_active":"2024-02-23T00:39:14.783469+0000","last_became_peered":"2024-02-23T00:39:14.783469+0000","last_unstale":"2024-02-23T00:45:11.156737+0000","last_undegraded":"2024-02-23T00:45:11.156737+0000","last_fullsized":"2024-02-23T00:45:11.156737+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:44.794928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:44.794928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":200,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.155352+0000","last_change":"2024-02-23T00:43:41.817455+0000","last_active":"2024-02-23T00:45:11.155352+0000","last_peered":"2024-02-23T00:45:11.155352+0000","last_clean":"2024-02-23T00:45:11.155352+0000","last_became_active":"2024-02-23T00:39:18.796952+0000","last_became_peered":"2024-02-23T00:39:18.796952+0000","last_unstale":"2024-02-23T00:45:11.155352+0000","last_undegraded":"2024-02-23T00:45:11.155352+0000","last_fullsized":"2024-02-23T00:45:11.155352+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:41.817387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:41.817387+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":145,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.142511+0000","last_change":"2024-02-23T00:45:08.215139+0000","last_active":"2024-02-23T00:45:11.142511+0000","last_peered":"2024-02-23T00:45:11.142511+0000","last_clean":"2024-02-23T00:45:11.142511+0000","last_became_active":"2024-02-23T00:42:58.726021+0000","last_became_peered":"2024-02-23T00:42:58.726021+0000","last_unstale":"2024-02-23T00:45:11.142511+0000","last_undegraded":"2024-02-23T00:45:11.142511+0000","last_fullsized":"2024-02-23T00:45:11.142511+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.215063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.215063+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.143315+0000","last_change":"2024-02-23T00:42:58.759026+0000","last_active":"2024-02-23T00:45:11.143315+0000","last_peered":"2024-02-23T00:45:11.143315+0000","last_clean":"2024-02-23T00:45:11.143315+0000","last_became_active":"2024-02-23T00:42:58.758147+0000","last_became_peered":"2024-02-23T00:42:58.758147+0000","last_unstale":"2024-02-23T00:45:11.143315+0000","last_undegraded":"2024-02-23T00:45:11.143315+0000","last_fullsized":"2024-02-23T00:45:11.143315+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"68'15","reported_seq":232,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.142613+0000","last_change":"2024-02-23T00:43:49.139235+0000","last_active":"2024-02-23T00:45:11.142613+0000","last_peered":"2024-02-23T00:45:11.142613+0000","last_clean":"2024-02-23T00:45:11.142613+0000","last_became_active":"2024-02-23T00:39:16.784606+0000","last_became_peered":"2024-02-23T00:39:16.784606+0000","last_unstale":"2024-02-23T00:45:11.142613+0000","last_undegraded":"2024-02-23T00:45:11.142613+0000","last_fullsized":"2024-02-23T00:45:11.142613+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:43:49.139156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:49.139156+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":204,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.143971+0000","last_change":"2024-02-23T00:44:32.170717+0000","last_active":"2024-02-23T00:45:11.143971+0000","last_peered":"2024-02-23T00:45:11.143971+0000","last_clean":"2024-02-23T00:45:11.143971+0000","last_became_active":"2024-02-23T00:39:14.785362+0000","last_became_peered":"2024-02-23T00:39:14.785362+0000","last_unstale":"2024-02-23T00:45:11.143971+0000","last_undegraded":"2024-02-23T00:45:11.143971+0000","last_fullsized":"2024-02-23T00:45:11.143971+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:32.170554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:32.170554+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":194,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.157317+0000","last_change":"2024-02-23T00:42:44.553722+0000","last_active":"2024-02-23T00:45:11.157317+0000","last_peered":"2024-02-23T00:45:11.157317+0000","last_clean":"2024-02-23T00:45:11.157317+0000","last_became_active":"2024-02-23T00:39:18.796312+0000","last_became_peered":"2024-02-23T00:39:18.796312+0000","last_unstale":"2024-02-23T00:45:11.157317+0000","last_undegraded":"2024-02-23T00:45:11.157317+0000","last_fullsized":"2024-02-23T00:45:11.157317+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:44.553621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:44.553621+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.13","version":"110'9","reported_seq":149,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.156714+0000","last_change":"2024-02-23T00:42:57.656016+0000","last_active":"2024-02-23T00:45:11.156714+0000","last_peered":"2024-02-23T00:45:11.156714+0000","last_clean":"2024-02-23T00:45:11.156714+0000","last_became_active":"2024-02-23T00:42:57.655617+0000","last_became_peered":"2024-02-23T00:42:57.655617+0000","last_unstale":"2024-02-23T00:45:11.156714+0000","last_undegraded":"2024-02-23T00:45:11.156714+0000","last_fullsized":"2024-02-23T00:45:11.156714+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":145,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.150112+0000","last_change":"2024-02-23T00:45:09.641006+0000","last_active":"2024-02-23T00:45:11.150112+0000","last_peered":"2024-02-23T00:45:11.150112+0000","last_clean":"2024-02-23T00:45:11.150112+0000","last_became_active":"2024-02-23T00:42:58.838257+0000","last_became_peered":"2024-02-23T00:42:58.838257+0000","last_unstale":"2024-02-23T00:45:11.150112+0000","last_undegraded":"2024-02-23T00:45:11.150112+0000","last_fullsized":"2024-02-23T00:45:11.150112+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:09.640936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:09.640936+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"88'50","reported_seq":298,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.145051+0000","last_change":"2024-02-23T00:45:06.038488+0000","last_active":"2024-02-23T00:45:11.145051+0000","last_peered":"2024-02-23T00:45:11.145051+0000","last_clean":"2024-02-23T00:45:11.145051+0000","last_became_active":"2024-02-23T00:39:16.780144+0000","last_became_peered":"2024-02-23T00:39:16.780144+0000","last_unstale":"2024-02-23T00:45:11.145051+0000","last_undegraded":"2024-02-23T00:45:11.145051+0000","last_fullsized":"2024-02-23T00:45:11.145051+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'50","last_scrub_stamp":"2024-02-23T00:45:06.038374+0000","last_deep_scrub":"88'50","last_deep_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.038374+0000","log_size":50,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":199,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.150454+0000","last_change":"2024-02-23T00:42:38.204365+0000","last_active":"2024-02-23T00:45:11.150454+0000","last_peered":"2024-02-23T00:45:11.150454+0000","last_clean":"2024-02-23T00:45:11.150454+0000","last_became_active":"2024-02-23T00:39:14.782000+0000","last_became_peered":"2024-02-23T00:39:14.782000+0000","last_unstale":"2024-02-23T00:45:11.150454+0000","last_undegraded":"2024-02-23T00:45:11.150454+0000","last_fullsized":"2024-02-23T00:45:11.150454+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:38.204283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_clean_scrub_stamp":"2024-02-23T00:42:38.204283+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":194,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.142967+0000","last_change":"2024-02-23T00:43:48.126631+0000","last_active":"2024-02-23T00:45:11.142967+0000","last_peered":"2024-02-23T00:45:11.142967+0000","last_clean":"2024-02-23T00:45:11.142967+0000","last_became_active":"2024-02-23T00:39:18.799723+0000","last_became_peered":"2024-02-23T00:39:18.799723+0000","last_unstale":"2024-02-23T00:45:11.142967+0000","last_undegraded":"2024-02-23T00:45:11.142967+0000","last_fullsized":"2024-02-23T00:45:11.142967+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:48.126552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.126552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.145560+0000","last_change":"2024-02-23T00:42:58.772753+0000","last_active":"2024-02-23T00:45:11.145560+0000","last_peered":"2024-02-23T00:45:11.145560+0000","last_clean":"2024-02-23T00:45:11.145560+0000","last_became_active":"2024-02-23T00:42:58.765643+0000","last_became_peered":"2024-02-23T00:42:58.765643+0000","last_unstale":"2024-02-23T00:45:11.145560+0000","last_undegraded":"2024-02-23T00:45:11.145560+0000","last_fullsized":"2024-02-23T00:45:11.145560+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":140,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-02-23T00:45:11.161696+0000","last_change":"2024-02-23T00:42:58.825262+0000","last_active":"2024-02-23T00:45:11.161696+0000","last_peered":"2024-02-23T00:45:11.161696+0000","last_clean":"2024-02-23T00:45:11.161696+0000","last_became_active":"2024-02-23T00:42:58.824573+0000","last_became_peered":"2024-02-23T00:42:58.824573+0000","last_unstale":"2024-02-23T00:45:11.161696+0000","last_undegraded":"2024-02-23T00:45:11.161696+0000","last_fullsized":"2024-02-23T00:45:11.161696+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0],"acting":[5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":161,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":160,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":159,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":158,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":157,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":8},{"poolid":156,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":8},{"poolid":155,"num_pg":32,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":62,"num_pg":32,"stat_sum":{"num_bytes":1396,"num_objects":136,"num_object_clones":22,"num_object_copies":408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136,"num_whiteouts":1,"num_read":92,"num_read_kb":1,"num_write":526,"num_write_kb":523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1613824,"data_stored":4124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1709,"ondisk_log_size":1709,"up":96,"acting":96,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":22,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":66,"acting":66,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1080,"num_read_kb":805,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":641,"ondisk_log_size":641,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":349,"num_read_kb":344,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2048,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":196608,"data_stored":6144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234059,"num_pgs":121,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":313504,"kb_used_data":3980,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93431648,"statfs":{"total":95995035648,"available":95674007552,"internally_reserved":0,"allocated":4075520,"data_stored":2313773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.56100000000000005,"15min":0.55100000000000005},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":4.9420000000000002,"5min":4.9420000000000002,"15min":4.9420000000000002},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.54100000000000004,"15min":0.51900000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":4.9729999999999999,"5min":4.9729999999999999,"15min":4.9729999999999999},"last":0.32600000000000001}]},{"osd":1,"last update":"Fri Feb 23 00:44:19 2024","interfaces":[{"interface":"back","average":{"1min":1.6519999999999999,"5min":0.75,"15min":0.61799999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":10.786,"5min":10.786,"15min":10.786},"last":0.47999999999999998},{"interface":"front","average":{"1min":1.3640000000000001,"5min":0.69999999999999996,"15min":0.59999999999999998},"min":{"1min":0.34399999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":11.433999999999999,"5min":11.433999999999999,"15min":11.433999999999999},"last":0.38}]},{"osd":2,"last update":"Fri Feb 23 00:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.57999999999999996,"15min":0.55400000000000005},"min":{"1min":0.315,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":5.0789999999999997,"5min":5.0789999999999997,"15min":5.0789999999999997},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.57999999999999996,"15min":0.55000000000000004},"min":{"1min":0.35699999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":5.1280000000000001,"5min":5.1280000000000001,"15min":5.1280000000000001},"last":0.54000000000000004}]},{"osd":3,"last update":"Fri Feb 23 00:44:19 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":0.67400000000000004,"15min":0.57599999999999996},"min":{"1min":0.34799999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":5.7469999999999999,"5min":5.7469999999999999,"15min":5.7469999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":1.3109999999999999,"5min":0.72599999999999998,"15min":0.621},"min":{"1min":0.39000000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":9.4090000000000007,"5min":9.4090000000000007,"15min":9.4090000000000007},"last":0.497}]},{"osd":4,"last update":"Fri Feb 23 00:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.42799999999999999,"15min":0.42999999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70099999999999996,"5min":0.754,"15min":0.754},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.45500000000000002,"15min":0.45400000000000001},"min":{"1min":0.29599999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.60599999999999998,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.42799999999999999}]},{"osd":5,"last update":"Fri Feb 23 00:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.47199999999999998,"15min":0.47599999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.63200000000000001,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.499,"15min":0.48299999999999998},"min":{"1min":0.29299999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.80500000000000005,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.439}]},{"osd":6,"last update":"Fri Feb 23 00:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.50800000000000001,"15min":0.50600000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76200000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.501,"15min":0.505},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64600000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.48599999999999999}]}]},{"osd":6,"up_from":47,"seq":201863462994,"num_pgs":86,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309320,"kb_used_data":3896,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93435832,"statfs":{"total":95995035648,"available":95678291968,"internally_reserved":0,"allocated":3989504,"data_stored":2313898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:44:49 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.98799999999999999,"15min":0.71199999999999997},"min":{"1min":0.29699999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61399999999999999,"5min":30.641999999999999,"15min":30.641999999999999},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.98199999999999998,"15min":0.71799999999999997},"min":{"1min":0.311,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1,"5min":29.292000000000002,"15min":29.292000000000002},"last":0.42999999999999999}]},{"osd":1,"last update":"Fri Feb 23 00:44:49 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.98599999999999999,"15min":0.69799999999999995},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.59799999999999998,"5min":28.186,"15min":28.186},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":1.4910000000000001,"15min":0.88900000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.97599999999999998,"5min":33.104999999999997,"15min":33.104999999999997},"last":0.47399999999999998}]},{"osd":2,"last update":"Fri Feb 23 00:44:49 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":1.0449999999999999,"15min":0.73899999999999999},"min":{"1min":0.32800000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.1200000000000001,"5min":19.960000000000001,"15min":19.960000000000001},"last":0.505},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.84799999999999998,"15min":0.65600000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0549999999999999,"5min":8.5860000000000003,"15min":8.5860000000000003},"last":0.40799999999999997}]},{"osd":3,"last update":"Fri Feb 23 00:44:49 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.76000000000000001,"15min":0.64200000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.0069999999999999,"5min":19.562000000000001,"15min":19.562000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.85599999999999998,"15min":0.68200000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.94299999999999995,"5min":14.199,"15min":14.199},"last":0.53400000000000003}]},{"osd":4,"last update":"Fri Feb 23 00:44:49 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.57899999999999996,"15min":0.46100000000000002},"min":{"1min":0.25900000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.55800000000000005,"5min":11.109999999999999,"15min":11.109999999999999},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.751,"15min":0.53300000000000003},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.65600000000000003,"5min":16.734000000000002,"15min":16.734000000000002},"last":0.51500000000000001}]},{"osd":5,"last update":"Fri Feb 23 00:44:49 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.629,"15min":0.48199999999999998},"min":{"1min":0.29299999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.59699999999999998,"5min":14.154999999999999,"15min":14.154999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.63500000000000001,"15min":0.5},"min":{"1min":0.312,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.67800000000000005,"5min":6.6100000000000003,"15min":6.6100000000000003},"last":0.48999999999999999}]},{"osd":7,"last update":"Fri Feb 23 00:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.58799999999999997,"15min":0.52100000000000002},"min":{"1min":0.30199999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.6949999999999998,"5min":8.548,"15min":8.548},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.75600000000000001,"15min":0.59499999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":4.6680000000000001,"5min":15.618,"15min":15.618},"last":0.54200000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574956,"num_pgs":118,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":305156,"kb_used_data":3832,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93439996,"statfs":{"total":95995035648,"available":95682555904,"internally_reserved":0,"allocated":3923968,"data_stored":2314666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:44:43 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":1.298,"15min":0.66500000000000004},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":1.637,"5min":78.361000000000004,"15min":78.361000000000004},"last":0.249},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.57299999999999995,"15min":0.437},"min":{"1min":0.218,"5min":0.191,"15min":0.191},"max":{"1min":1.284,"5min":5.8849999999999998,"15min":5.8849999999999998},"last":0.38300000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:45:03 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":1.1850000000000001,"15min":0.65900000000000003},"min":{"1min":0.247,"5min":0.219,"15min":0.188},"max":{"1min":0.63,"5min":48.606000000000002,"15min":48.606000000000002},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.88300000000000001,"15min":0.58599999999999997},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.249},"max":{"1min":0.82599999999999996,"5min":23.911000000000001,"15min":23.911000000000001},"last":0.28599999999999998}]},{"osd":3,"last update":"Fri Feb 23 00:44:38 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.89800000000000002,"15min":0.63},"min":{"1min":0.29899999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":2.1949999999999998,"5min":35.244999999999997,"15min":35.244999999999997},"last":0.316},{"interface":"front","average":{"1min":0.70799999999999996,"5min":1.135,"15min":0.68799999999999994},"min":{"1min":0.27100000000000002,"5min":0.253,"15min":0.248},"max":{"1min":6.0510000000000002,"5min":56.853999999999999,"15min":56.853999999999999},"last":0.32900000000000001}]},{"osd":4,"last update":"Fri Feb 23 00:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":1.2350000000000001,"15min":0.81999999999999995},"min":{"1min":0.33700000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72899999999999998,"5min":37.646999999999998,"15min":37.646999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.875,"15min":0.69999999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83499999999999996,"5min":9.6859999999999999,"15min":9.6859999999999999},"last":0.53400000000000003}]},{"osd":5,"last update":"Fri Feb 23 00:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":1.222,"15min":0.85199999999999998},"min":{"1min":0.45100000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":5.9500000000000002,"5min":48.792000000000002,"15min":48.792000000000002},"last":0.5},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.83899999999999997,"15min":0.76300000000000001},"min":{"1min":0.41099999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":3.593,"5min":11.872999999999999,"15min":11.872999999999999},"last":0.56299999999999994}]},{"osd":6,"last update":"Fri Feb 23 00:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":1.2,"15min":0.82999999999999996},"min":{"1min":0.39200000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.90200000000000002,"5min":40.741,"15min":40.741},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.77800000000000002,"15min":0.69599999999999995},"min":{"1min":0.46100000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77200000000000002,"5min":6.0069999999999997,"15min":6.0069999999999997},"last":0.60099999999999998}]},{"osd":7,"last update":"Fri Feb 23 00:44:15 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":0.92000000000000004,"15min":0.72199999999999998},"min":{"1min":0.47599999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":3.6230000000000002,"5min":20.225000000000001,"15min":20.225000000000001},"last":0.628},{"interface":"front","average":{"1min":1.137,"5min":0.95099999999999996,"15min":0.73099999999999998},"min":{"1min":0.45000000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":5.9169999999999998,"5min":20.254999999999999,"15min":20.254999999999999},"last":0.49199999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738482,"num_pgs":91,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309284,"kb_used_data":3856,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93435868,"statfs":{"total":95995035648,"available":95678328832,"internally_reserved":0,"allocated":3948544,"data_stored":2314038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 23 00:44:42 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":1.0980000000000001,"15min":0.59099999999999997},"min":{"1min":0.25900000000000001,"5min":0.223,"15min":0.19500000000000001},"max":{"1min":0.82799999999999996,"5min":28.856999999999999,"15min":28.856999999999999},"last":0.34799999999999998},{"interface":"front","average":{"1min":1.3480000000000001,"5min":0.97799999999999998,"15min":0.56100000000000005},"min":{"1min":0.22700000000000001,"5min":0.218,"15min":0.20799999999999999},"max":{"1min":12.923,"5min":20.577000000000002,"15min":20.577000000000002},"last":0.28100000000000003}]},{"osd":2,"last update":"Fri Feb 23 00:45:06 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.026,"15min":0.60399999999999998},"min":{"1min":0.308,"5min":0.26000000000000001,"15min":0.20499999999999999},"max":{"1min":13.67,"5min":13.863,"15min":13.863},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.88400000000000001,"15min":0.54900000000000004},"min":{"1min":0.25700000000000001,"5min":0.24299999999999999,"15min":0.19800000000000001},"max":{"1min":0.60099999999999998,"5min":10.455,"15min":10.455},"last":0.33400000000000002}]},{"osd":3,"last update":"Fri Feb 23 00:44:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.389,"15min":0.76400000000000001},"min":{"1min":0.28699999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":13.266,"5min":35.566000000000003,"15min":35.566000000000003},"last":0.313},{"interface":"front","average":{"1min":1.6859999999999999,"5min":1.24,"15min":0.751},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29399999999999998},"max":{"1min":16.931999999999999,"5min":20.702999999999999,"15min":20.702999999999999},"last":0.38400000000000001}]},{"osd":4,"last update":"Fri Feb 23 00:44:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.038,"15min":0.81299999999999994},"min":{"1min":0.36199999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":10.584,"5min":10.584,"15min":10.584},"last":0.49299999999999999},{"interface":"front","average":{"1min":1.306,"5min":0.89600000000000002,"15min":0.79700000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":13.029999999999999,"5min":13.029999999999999,"15min":13.029999999999999},"last":0.48299999999999998}]},{"osd":5,"last update":"Fri Feb 23 00:44:21 2024","interfaces":[{"interface":"back","average":{"1min":2.3239999999999998,"5min":1.014,"15min":0.748},"min":{"1min":0.379,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":10.397,"5min":10.397,"15min":10.397},"last":0.433},{"interface":"front","average":{"1min":2.5339999999999998,"5min":1.091,"15min":0.81299999999999994},"min":{"1min":0.44800000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":17.297000000000001,"5min":17.297000000000001,"15min":17.297000000000001},"last":0.57499999999999996}]},{"osd":6,"last update":"Fri Feb 23 00:44:57 2024","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.1160000000000001,"15min":0.80200000000000005},"min":{"1min":0.434,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":13.237,"5min":17.373999999999999,"15min":17.373999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":1.3029999999999999,"5min":0.90200000000000002,"15min":0.72399999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":13.086,"5min":13.086,"15min":13.086},"last":0.52600000000000002}]},{"osd":7,"last update":"Fri Feb 23 00:44:16 2024","interfaces":[{"interface":"back","average":{"1min":2.5449999999999999,"5min":1.095,"15min":0.81699999999999995},"min":{"1min":0.41799999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":17.399999999999999,"5min":17.399999999999999,"15min":17.399999999999999},"last":0.505},{"interface":"front","average":{"1min":1.958,"5min":0.89600000000000002,"15min":0.73299999999999998},"min":{"1min":0.433,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":13.061999999999999,"5min":13.061999999999999,"15min":13.061999999999999},"last":0.56299999999999994}]}]},{"osd":2,"up_from":18,"seq":77309411431,"num_pgs":90,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":305332,"kb_used_data":4004,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93439820,"statfs":{"total":95995035648,"available":95682375680,"internally_reserved":0,"allocated":4100096,"data_stored":2313990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:45:02 2024","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":0.70199999999999996,"15min":0.48699999999999999},"min":{"1min":0.20499999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":13.138999999999999,"5min":13.657,"15min":13.657},"last":0.29499999999999998},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2649999999999999,"15min":0.68000000000000005},"min":{"1min":0.25700000000000001,"5min":0.246,"15min":0.23599999999999999},"max":{"1min":12.705,"5min":42.343000000000004,"15min":42.343000000000004},"last":0.40400000000000003}]},{"osd":1,"last update":"Fri Feb 23 00:45:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":0.96999999999999997,"15min":0.57499999999999996},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":8.8569999999999993,"5min":18.952000000000002,"15min":18.952000000000002},"last":0.35299999999999998},{"interface":"front","average":{"1min":1.6419999999999999,"5min":0.76800000000000002,"15min":0.52300000000000002},"min":{"1min":0.26100000000000001,"5min":0.23999999999999999,"15min":0.23599999999999999},"max":{"1min":15.391,"5min":15.391,"15min":15.391},"last":0.39500000000000002}]},{"osd":3,"last update":"Fri Feb 23 00:44:34 2024","interfaces":[{"interface":"back","average":{"1min":2.3879999999999999,"5min":1.1879999999999999,"15min":0.72199999999999998},"min":{"1min":0.251,"5min":0.25,"15min":0.25},"max":{"1min":25.635999999999999,"5min":25.635999999999999,"15min":25.635999999999999},"last":0.33000000000000002},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.008,"15min":0.65000000000000002},"min":{"1min":0.27600000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":14.472,"5min":19.960000000000001,"15min":19.960000000000001},"last":0.316}]},{"osd":4,"last update":"Fri Feb 23 00:45:02 2024","interfaces":[{"interface":"back","average":{"1min":2.1920000000000002,"5min":0.998,"15min":0.73099999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":13.673,"5min":13.673,"15min":13.673},"last":0.58999999999999997},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.044,"15min":0.73499999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.33900000000000002},"max":{"1min":8.7870000000000008,"5min":10.103999999999999,"15min":10.103999999999999},"last":0.58299999999999996}]},{"osd":5,"last update":"Fri Feb 23 00:44:20 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.66300000000000003,"15min":0.61099999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":2.3730000000000002,"5min":3.137,"15min":3.137},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.73599999999999999,"15min":0.65400000000000003},"min":{"1min":0.39700000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":3.9380000000000002,"5min":7.0019999999999998,"15min":7.0019999999999998},"last":0.57299999999999995}]},{"osd":6,"last update":"Fri Feb 23 00:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":0.872,"15min":0.70599999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":11.544,"5min":11.544,"15min":11.544},"last":0.61099999999999999},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0029999999999999,"15min":0.73199999999999998},"min":{"1min":0.38800000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":8.8290000000000006,"5min":10.132999999999999,"15min":10.132999999999999},"last":0.503}]},{"osd":7,"last update":"Fri Feb 23 00:44:13 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.76500000000000001,"15min":0.70499999999999996},"min":{"1min":0.46500000000000002,"5min":0.377,"15min":0.377},"max":{"1min":3.9689999999999999,"5min":6.9740000000000002,"15min":6.9740000000000002},"last":0.64000000000000001},{"interface":"front","average":{"1min":2.0350000000000001,"5min":0.92900000000000005,"15min":0.74399999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":10.148999999999999,"5min":10.148999999999999,"15min":10.148999999999999},"last":0.52100000000000002}]}]},{"osd":3,"up_from":26,"seq":111669149792,"num_pgs":127,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309344,"kb_used_data":3920,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93435808,"statfs":{"total":95995035648,"available":95678267392,"internally_reserved":0,"allocated":4014080,"data_stored":2315028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:44:32 2024","interfaces":[{"interface":"back","average":{"1min":1.5489999999999999,"5min":0.754,"15min":0.54600000000000004},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":12.212999999999999,"5min":12.212999999999999,"15min":12.212999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":1.802,"5min":0.90400000000000003,"15min":0.60799999999999998},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":11.714,"5min":11.714,"15min":11.714},"last":0.33800000000000002}]},{"osd":1,"last update":"Fri Feb 23 00:44:32 2024","interfaces":[{"interface":"back","average":{"1min":1.627,"5min":0.88500000000000001,"15min":0.61499999999999999},"min":{"1min":0.23599999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":11.742000000000001,"5min":11.742000000000001,"15min":11.742000000000001},"last":0.379},{"interface":"front","average":{"1min":1.278,"5min":0.77300000000000002,"15min":0.56100000000000005},"min":{"1min":0.24199999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":6.383,"5min":6.383,"15min":6.383},"last":0.41899999999999998}]},{"osd":2,"last update":"Fri Feb 23 00:44:32 2024","interfaces":[{"interface":"back","average":{"1min":1.8939999999999999,"5min":0.95399999999999996,"15min":0.65100000000000002},"min":{"1min":0.28899999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":13.359999999999999,"5min":13.359999999999999,"15min":13.359999999999999},"last":0.30499999999999999},{"interface":"front","average":{"1min":2.403,"5min":1.01,"15min":0.66300000000000003},"min":{"1min":0.26200000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":21.122,"5min":21.122,"15min":21.122},"last":0.35299999999999998}]},{"osd":4,"last update":"Fri Feb 23 00:44:57 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.97399999999999998,"15min":0.72499999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":8.125,"5min":11.644,"15min":11.644},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.85399999999999998,"15min":0.68000000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":6.2549999999999999,"5min":6.2549999999999999,"15min":6.2549999999999999},"last":0.66600000000000004}]},{"osd":5,"last update":"Fri Feb 23 00:44:20 2024","interfaces":[{"interface":"back","average":{"1min":1.5169999999999999,"5min":0.77600000000000002,"15min":0.66400000000000003},"min":{"1min":0.40200000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":5.6360000000000001,"5min":5.6360000000000001,"15min":5.6360000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":2.492,"5min":1.0229999999999999,"15min":0.76000000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":11.597,"5min":11.597,"15min":11.597},"last":0.79300000000000004}]},{"osd":6,"last update":"Fri Feb 23 00:44:46 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88100000000000001,"15min":0.70199999999999996},"min":{"1min":0.40899999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":6.343,"5min":6.343,"15min":6.343},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.80200000000000005,"15min":0.68600000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":2.8260000000000001,"5min":5.6680000000000001,"15min":5.6680000000000001},"last":0.71699999999999997}]},{"osd":7,"last update":"Fri Feb 23 00:44:16 2024","interfaces":[{"interface":"back","average":{"1min":2.5569999999999999,"5min":1.052,"15min":0.77600000000000002},"min":{"1min":0.434,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":11.67,"5min":11.67,"15min":11.67},"last":0.68400000000000005},{"interface":"front","average":{"1min":1.821,"5min":0.93999999999999995,"15min":0.752},"min":{"1min":0.46999999999999997,"5min":0.433,"15min":0.433},"max":{"1min":6.3010000000000002,"5min":6.3010000000000002,"15min":6.3010000000000002},"last":0.72799999999999998}]}]},{"osd":4,"up_from":32,"seq":137438953564,"num_pgs":115,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309360,"kb_used_data":3932,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93435792,"statfs":{"total":95995035648,"available":95678251008,"internally_reserved":0,"allocated":4026368,"data_stored":2314795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:44:59 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.52100000000000002,"15min":0.56100000000000005},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.59499999999999997,"5min":3.5219999999999998,"15min":3.5219999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.66900000000000004,"15min":0.61399999999999999},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.64100000000000001,"5min":8.6140000000000008,"15min":8.6140000000000008},"last":0.48599999999999999}]},{"osd":1,"last update":"Fri Feb 23 00:44:59 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.66700000000000004,"15min":0.61299999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.27800000000000002},"max":{"1min":0.69799999999999995,"5min":9.6110000000000007,"15min":9.6110000000000007},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.66300000000000003,"15min":0.60499999999999998},"min":{"1min":0.33500000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.67400000000000004,"5min":6.6950000000000003,"15min":6.6950000000000003},"last":0.497}]},{"osd":2,"last update":"Fri Feb 23 00:44:59 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.624,"15min":0.59399999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64800000000000002,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68700000000000006,"5min":5.8440000000000003,"15min":5.8440000000000003},"last":0.46200000000000002}]},{"osd":3,"last update":"Fri Feb 23 00:44:59 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.65800000000000003,"15min":0.60599999999999998},"min":{"1min":0.34399999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65900000000000003,"5min":6.5540000000000003,"15min":6.5540000000000003},"last":0.375},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.66200000000000003,"15min":0.60899999999999999},"min":{"1min":0.32700000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73699999999999999,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":0.39600000000000002}]},{"osd":5,"last update":"Fri Feb 23 00:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.42499999999999999,"15min":0.38100000000000001},"min":{"1min":0.27000000000000002,"5min":0.251,"15min":0.246},"max":{"1min":0.47999999999999998,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.314},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.49099999999999999,"15min":0.43099999999999999},"min":{"1min":0.33400000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.56499999999999995,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.63}]},{"osd":6,"last update":"Fri Feb 23 00:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.45000000000000001,"15min":0.41299999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.68100000000000005,"5min":0.75,"15min":0.75},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.45400000000000001,"15min":0.41299999999999998},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.66100000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.33500000000000002}]},{"osd":7,"last update":"Fri Feb 23 00:44:16 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.48899999999999999,"15min":0.48399999999999999},"min":{"1min":0.27300000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.53800000000000003,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.34599999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.57599999999999996,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.52400000000000002}]}]},{"osd":5,"up_from":39,"seq":167503724631,"num_pgs":101,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309276,"kb_used_data":3856,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93435876,"statfs":{"total":95995035648,"available":95678337024,"internally_reserved":0,"allocated":3948544,"data_stored":2315000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.79800000000000004,"15min":0.60599999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":3.7480000000000002,"5min":23.881,"15min":23.881},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.69799999999999995,"15min":0.58299999999999996},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":2.9239999999999999,"5min":5.4649999999999999,"15min":5.4649999999999999},"last":0.57499999999999996}]},{"osd":1,"last update":"Fri Feb 23 00:44:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":0.90000000000000002,"15min":0.65100000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":5.6509999999999998,"5min":18.175999999999998,"15min":18.175999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":1.248,"5min":0.79900000000000004,"15min":0.629},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":4.2240000000000002,"5min":9.0950000000000006,"15min":9.0950000000000006},"last":0.60299999999999998}]},{"osd":2,"last update":"Fri Feb 23 00:44:19 2024","interfaces":[{"interface":"back","average":{"1min":1.831,"5min":1.032,"15min":0.71199999999999997},"min":{"1min":0.36399999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":10.590999999999999,"5min":22.282,"15min":22.282},"last":0.56299999999999994},{"interface":"front","average":{"1min":1.536,"5min":1.0640000000000001,"15min":0.71199999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":11.196999999999999,"5min":23.925000000000001,"15min":23.925000000000001},"last":0.61199999999999999}]},{"osd":3,"last update":"Fri Feb 23 00:44:19 2024","interfaces":[{"interface":"back","average":{"1min":1.5860000000000001,"5min":0.78700000000000003,"15min":0.61699999999999999},"min":{"1min":0.36499999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":6.5730000000000004,"5min":6.5730000000000004,"15min":6.5730000000000004},"last":0.59199999999999997},{"interface":"front","average":{"1min":1.4419999999999999,"5min":0.81100000000000005,"15min":0.64300000000000002},"min":{"1min":0.39100000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":6.7910000000000004,"5min":8.1760000000000002,"15min":8.1760000000000002},"last":0.55500000000000005}]},{"osd":4,"last update":"Fri Feb 23 00:44:19 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.432,"15min":0.39500000000000002},"min":{"1min":0.26000000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.68999999999999995,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.51900000000000002,"15min":0.42499999999999999},"min":{"1min":0.28199999999999997,"5min":0.27500000000000002,"15min":0.27300000000000002},"max":{"1min":3.879,"5min":3.879,"15min":3.879},"last":0.42599999999999999}]},{"osd":6,"last update":"Fri Feb 23 00:44:44 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.48399999999999999,"15min":0.44400000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.98099999999999998,"5min":1.736,"15min":1.736},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.49299999999999999,"15min":0.45600000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.41199999999999998}]},{"osd":7,"last update":"Fri Feb 23 00:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.58299999999999996,"15min":0.53300000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.83999999999999997,"5min":7.0300000000000002,"15min":7.0300000000000002},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.58599999999999997,"15min":0.53400000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.038,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":0.46600000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":159744,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":184320,"data_stored":476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":200704,"data_stored":522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":200704,"data_stored":509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":290816,"data_stored":744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":155,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":155,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":155,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":155,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":155,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":155,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":155,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":155,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":156,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":156,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":156,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":156,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":156,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":156,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":156,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":156,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":157,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":157,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":157,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":157,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":157,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":157,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":157,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":157,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-23T00:45:17.734 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph status --format=json 2024-02-23T00:45:18.231 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:16.663743+0000 osd.1 (osd.1) 127 : cluster [DBG] 2.6 scrub starts 2024-02-23T00:45:18.231 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:16.664673+0000 osd.1 (osd.1) 128 : cluster [DBG] 2.6 scrub ok 2024-02-23T00:45:18.231 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:16.746009+0000 osd.7 (osd.7) 128 : cluster [DBG] 5.4 scrub ok 2024-02-23T00:45:18.231 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:17.131428+0000 mon.a (mon.0) 1686 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:18.231 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:17.131540+0000 mon.a (mon.0) 1687 : audit [INF] from='client.? 172.21.15.47:0/3951871746' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:18.231 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:17.131628+0000 mon.a (mon.0) 1688 : cluster [DBG] osdmap e245: 8 total, 8 up, 8 in 2024-02-23T00:45:18.231 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:17.142296+0000 mon.c (mon.2) 164 : audit [INF] from='client.? 172.21.15.47:0/1302226360' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi047-82378-33"}]: dispatch 2024-02-23T00:45:18.231 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:17.144772+0000 mon.a (mon.0) 1689 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi047-82378-33"}]: dispatch 2024-02-23T00:45:18.232 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:17.174446+0000 osd.5 (osd.5) 113 : cluster [DBG] 18.1e scrub starts 2024-02-23T00:45:18.232 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:17.181331+0000 osd.5 (osd.5) 114 : cluster [DBG] 18.1e scrub ok 2024-02-23T00:45:18.232 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:17.238290+0000 mgr.y (mgr.14517) 266 : cluster [DBG] pgmap v343: 410 pgs: 16 creating+activating, 1 active, 1 active+clean+scrubbing, 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 385 active+clean; 9.0 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:45:18.232 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:17.355071+0000 osd.4 (osd.4) 115 : cluster [DBG] 15.3 scrub starts 2024-02-23T00:45:18.232 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:17.356401+0000 osd.4 (osd.4) 116 : cluster [DBG] 15.3 scrub ok 2024-02-23T00:45:18.232 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:17.816923+0000 mon.b (mon.1) 305 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:18.232 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:16.663743+0000 osd.1 (osd.1) 127 : cluster [DBG] 2.6 scrub starts 2024-02-23T00:45:18.232 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:16.664673+0000 osd.1 (osd.1) 128 : cluster [DBG] 2.6 scrub ok 2024-02-23T00:45:18.232 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:16.746009+0000 osd.7 (osd.7) 128 : cluster [DBG] 5.4 scrub ok 2024-02-23T00:45:18.232 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:17.131428+0000 mon.a (mon.0) 1686 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:18.232 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:17.131540+0000 mon.a (mon.0) 1687 : audit [INF] from='client.? 172.21.15.47:0/3951871746' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:18.233 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:17.131628+0000 mon.a (mon.0) 1688 : cluster [DBG] osdmap e245: 8 total, 8 up, 8 in 2024-02-23T00:45:18.233 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:17.142296+0000 mon.c (mon.2) 164 : audit [INF] from='client.? 172.21.15.47:0/1302226360' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi047-82378-33"}]: dispatch 2024-02-23T00:45:18.233 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:17.144772+0000 mon.a (mon.0) 1689 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi047-82378-33"}]: dispatch 2024-02-23T00:45:18.233 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:17.174446+0000 osd.5 (osd.5) 113 : cluster [DBG] 18.1e scrub starts 2024-02-23T00:45:18.233 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:17.181331+0000 osd.5 (osd.5) 114 : cluster [DBG] 18.1e scrub ok 2024-02-23T00:45:18.233 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:17.238290+0000 mgr.y (mgr.14517) 266 : cluster [DBG] pgmap v343: 410 pgs: 16 creating+activating, 1 active, 1 active+clean+scrubbing, 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 385 active+clean; 9.0 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:45:18.233 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:17.355071+0000 osd.4 (osd.4) 115 : cluster [DBG] 15.3 scrub starts 2024-02-23T00:45:18.233 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:17.356401+0000 osd.4 (osd.4) 116 : cluster [DBG] 15.3 scrub ok 2024-02-23T00:45:18.233 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:17.816923+0000 mon.b (mon.1) 305 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:18.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:16.663743+0000 osd.1 (osd.1) 127 : cluster [DBG] 2.6 scrub starts 2024-02-23T00:45:18.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:16.664673+0000 osd.1 (osd.1) 128 : cluster [DBG] 2.6 scrub ok 2024-02-23T00:45:18.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:16.746009+0000 osd.7 (osd.7) 128 : cluster [DBG] 5.4 scrub ok 2024-02-23T00:45:18.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:17.131428+0000 mon.a (mon.0) 1686 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:18.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:17.131540+0000 mon.a (mon.0) 1687 : audit [INF] from='client.? 172.21.15.47:0/3951871746' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi047-82386-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:18.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:17.131628+0000 mon.a (mon.0) 1688 : cluster [DBG] osdmap e245: 8 total, 8 up, 8 in 2024-02-23T00:45:18.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:17.142296+0000 mon.c (mon.2) 164 : audit [INF] from='client.? 172.21.15.47:0/1302226360' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi047-82378-33"}]: dispatch 2024-02-23T00:45:18.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:17.144772+0000 mon.a (mon.0) 1689 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi047-82378-33"}]: dispatch 2024-02-23T00:45:18.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:17.174446+0000 osd.5 (osd.5) 113 : cluster [DBG] 18.1e scrub starts 2024-02-23T00:45:18.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:17.181331+0000 osd.5 (osd.5) 114 : cluster [DBG] 18.1e scrub ok 2024-02-23T00:45:18.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:17.238290+0000 mgr.y (mgr.14517) 266 : cluster [DBG] pgmap v343: 410 pgs: 16 creating+activating, 1 active, 1 active+clean+scrubbing, 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 385 active+clean; 9.0 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:45:18.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:17.355071+0000 osd.4 (osd.4) 115 : cluster [DBG] 15.3 scrub starts 2024-02-23T00:45:18.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:17.356401+0000 osd.4 (osd.4) 116 : cluster [DBG] 15.3 scrub ok 2024-02-23T00:45:18.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:17.816923+0000 mon.b (mon.1) 305 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:18.690 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:45:18.694 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:18.694 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:45:18.795 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:45:18.825 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:19.184 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set noscrub 2024-02-23T00:45:19.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:17.760394+0000 osd.7 (osd.7) 129 : cluster [DBG] 18.13 scrub starts 2024-02-23T00:45:19.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:17.763471+0000 osd.7 (osd.7) 130 : cluster [DBG] 18.13 scrub ok 2024-02-23T00:45:19.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:18.136796+0000 mon.a (mon.0) 1690 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi047-82378-33"}]': finished 2024-02-23T00:45:19.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:18.136894+0000 mon.a (mon.0) 1691 : cluster [DBG] osdmap e246: 8 total, 8 up, 8 in 2024-02-23T00:45:19.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:18.138926+0000 mon.c (mon.2) 165 : audit [INF] from='client.? 172.21.15.47:0/1302226360' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi047-82378-33"}]: dispatch 2024-02-23T00:45:19.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:18.139339+0000 mon.a (mon.0) 1692 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi047-82378-33"}]: dispatch 2024-02-23T00:45:19.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:18.148015+0000 mon.a (mon.0) 1693 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:45:19.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:18.179197+0000 osd.5 (osd.5) 115 : cluster [DBG] 18.15 scrub starts 2024-02-23T00:45:19.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:18.817905+0000 mon.b (mon.1) 306 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:19.492 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:17.760394+0000 osd.7 (osd.7) 129 : cluster [DBG] 18.13 scrub starts 2024-02-23T00:45:19.492 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:17.763471+0000 osd.7 (osd.7) 130 : cluster [DBG] 18.13 scrub ok 2024-02-23T00:45:19.492 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:18.136796+0000 mon.a (mon.0) 1690 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi047-82378-33"}]': finished 2024-02-23T00:45:19.492 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:18.136894+0000 mon.a (mon.0) 1691 : cluster [DBG] osdmap e246: 8 total, 8 up, 8 in 2024-02-23T00:45:19.492 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:18.138926+0000 mon.c (mon.2) 165 : audit [INF] from='client.? 172.21.15.47:0/1302226360' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi047-82378-33"}]: dispatch 2024-02-23T00:45:19.492 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:18.139339+0000 mon.a (mon.0) 1692 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi047-82378-33"}]: dispatch 2024-02-23T00:45:19.492 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:18.148015+0000 mon.a (mon.0) 1693 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:45:19.492 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:18.179197+0000 osd.5 (osd.5) 115 : cluster [DBG] 18.15 scrub starts 2024-02-23T00:45:19.492 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:18.817905+0000 mon.b (mon.1) 306 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:19.492 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:17.760394+0000 osd.7 (osd.7) 129 : cluster [DBG] 18.13 scrub starts 2024-02-23T00:45:19.493 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:17.763471+0000 osd.7 (osd.7) 130 : cluster [DBG] 18.13 scrub ok 2024-02-23T00:45:19.493 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:18.136796+0000 mon.a (mon.0) 1690 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi047-82378-33"}]': finished 2024-02-23T00:45:19.493 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:18.136894+0000 mon.a (mon.0) 1691 : cluster [DBG] osdmap e246: 8 total, 8 up, 8 in 2024-02-23T00:45:19.493 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:18.138926+0000 mon.c (mon.2) 165 : audit [INF] from='client.? 172.21.15.47:0/1302226360' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi047-82378-33"}]: dispatch 2024-02-23T00:45:19.493 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:18.139339+0000 mon.a (mon.0) 1692 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi047-82378-33"}]: dispatch 2024-02-23T00:45:19.493 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:18.148015+0000 mon.a (mon.0) 1693 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:45:19.493 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:18.179197+0000 osd.5 (osd.5) 115 : cluster [DBG] 18.15 scrub starts 2024-02-23T00:45:19.493 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:18.817905+0000 mon.b (mon.1) 306 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:45:19.779 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:19.779 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:45:19.880 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:45:19.912 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:19.933 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:45:19.935 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:19.936 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:45:20.037 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:45:20.068 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:20.091 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:45:20.092 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:20.093 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:45:20.193 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:45:20.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:18.184985+0000 osd.5 (osd.5) 116 : cluster [DBG] 18.15 scrub ok 2024-02-23T00:45:20.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:18.641441+0000 osd.1 (osd.1) 129 : cluster [DBG] 15.1c scrub starts 2024-02-23T00:45:20.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:18.644530+0000 osd.1 (osd.1) 130 : cluster [DBG] 15.1c scrub ok 2024-02-23T00:45:20.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:18.946419+0000 osd.3 (osd.3) 133 : cluster [DBG] 18.1 scrub starts 2024-02-23T00:45:20.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:18.947738+0000 osd.3 (osd.3) 134 : cluster [DBG] 18.1 scrub ok 2024-02-23T00:45:20.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:19.138252+0000 mon.a (mon.0) 1694 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi047-82378-33"}]': finished 2024-02-23T00:45:20.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:19.138356+0000 mon.a (mon.0) 1695 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:45:20.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:19.138411+0000 mon.a (mon.0) 1696 : cluster [DBG] osdmap e247: 8 total, 8 up, 8 in 2024-02-23T00:45:20.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:19.143416+0000 mon.a (mon.0) 1697 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-25"}]: dispatch 2024-02-23T00:45:20.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:19.169527+0000 mon.a (mon.0) 1698 : audit [INF] from='client.? 172.21.15.47:0/290258726' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi047-82378-34"}]: dispatch 2024-02-23T00:45:20.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:19.170479+0000 mon.a (mon.0) 1699 : audit [INF] from='client.? 172.21.15.47:0/290258726' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi047-82378-34"}]: dispatch 2024-02-23T00:45:20.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:19.170909+0000 mon.a (mon.0) 1700 : audit [INF] from='client.? 172.21.15.47:0/290258726' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi047-82378-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:20.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:19.238739+0000 mgr.y (mgr.14517) 267 : cluster [DBG] pgmap v346: 346 pgs: 6 creating+activating, 1 active, 1 active+clean+scrubbing, 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 331 active+clean; 9.0 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:45:20.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:19.819302+0000 mon.b (mon.1) 307 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:20.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:20.126305+0000 mon.a (mon.0) 1701 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-25"}]': finished 2024-02-23T00:45:20.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:20.126414+0000 mon.a (mon.0) 1702 : audit [INF] from='client.? 172.21.15.47:0/290258726' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi047-82378-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:20.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:20.126470+0000 mon.a (mon.0) 1703 : cluster [DBG] osdmap e248: 8 total, 8 up, 8 in 2024-02-23T00:45:20.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:20.130522+0000 mon.a (mon.0) 1704 : audit [INF] from='client.? 172.21.15.47:0/290258726' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi047-82378-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi047-82378-34"}]: dispatch 2024-02-23T00:45:20.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:20.130957+0000 mon.a (mon.0) 1705 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-25", "mode": "writeback"}]: dispatch 2024-02-23T00:45:20.224 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:20.244 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:45:20.246 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:20.247 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:45:20.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.224+0000 7f0d8eee6700 1 -- 172.21.15.47:0/1739092326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d880f9670 msgr2=0x7f0d88101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:20.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.224+0000 7f0d8eee6700 1 --2- 172.21.15.47:0/1739092326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d880f9670 0x7f0d88101d80 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 rx=0x7f0d74004870 tx=0x7f0d74009cb0).stop 2024-02-23T00:45:20.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.224+0000 7f0d8eee6700 1 -- 172.21.15.47:0/1739092326 shutdown_connections 2024-02-23T00:45:20.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.224+0000 7f0d8eee6700 1 --2- 172.21.15.47:0/1739092326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d88102270 0x7f0d88104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:20.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.224+0000 7f0d8eee6700 1 --2- 172.21.15.47:0/1739092326 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0d880f8d10 0x7f0d880f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:20.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.224+0000 7f0d8eee6700 1 --2- 172.21.15.47:0/1739092326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d880f9670 0x7f0d88101d80 unknown :-1 s=CLOSED pgs=2475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:20.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.224+0000 7f0d8eee6700 1 -- 172.21.15.47:0/1739092326 >> 172.21.15.47:0/1739092326 conn(0x7f0d880f2820 msgr2=0x7f0d880f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:45:20.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.224+0000 7f0d8eee6700 1 -- 172.21.15.47:0/1739092326 shutdown_connections 2024-02-23T00:45:20.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.224+0000 7f0d8eee6700 1 -- 172.21.15.47:0/1739092326 wait complete. 2024-02-23T00:45:20.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.225+0000 7f0d8eee6700 1 librados: starting msgr at 2024-02-23T00:45:20.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.225+0000 7f0d8eee6700 1 librados: starting objecter 2024-02-23T00:45:20.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.225+0000 7f0d8eee6700 1 Processor -- start 2024-02-23T00:45:20.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.225+0000 7f0d8eee6700 1 -- start start 2024-02-23T00:45:20.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.225+0000 7f0d8eee6700 1 librados: setting wanted keys 2024-02-23T00:45:20.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.225+0000 7f0d8eee6700 1 librados: calling monclient init 2024-02-23T00:45:20.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.225+0000 7f0d8eee6700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0d880f8d10 0x7f0d8810e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:20.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.225+0000 7f0d8eee6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d880f9670 0x7f0d8810e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:20.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.225+0000 7f0d8eee6700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d88102270 0x7f0d88108220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:20.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.225+0000 7f0d8eee6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d88108710 con 0x7f0d880f9670 2024-02-23T00:45:20.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.225+0000 7f0d8eee6700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d88108880 con 0x7f0d88102270 2024-02-23T00:45:20.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.225+0000 7f0d8eee6700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d88108a40 con 0x7f0d880f8d10 2024-02-23T00:45:20.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.225+0000 7f0d8cc82700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0d880f8d10 0x7f0d8810e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:20.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.225+0000 7f0d8cc82700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0d880f8d10 0x7f0d8810e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:41778/0 (socket says 172.21.15.47:41778) 2024-02-23T00:45:20.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.225+0000 7f0d8cc82700 1 -- 172.21.15.47:0/4147048992 learned_addr learned my addr 172.21.15.47:0/4147048992 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:45:20.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.225+0000 7f0d87fff700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d880f9670 0x7f0d8810e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:20.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.225+0000 7f0d8d483700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d88102270 0x7f0d88108220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:20.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.226+0000 7f0d8cc82700 1 -- 172.21.15.47:0/4147048992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d88102270 msgr2=0x7f0d88108220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:20.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.226+0000 7f0d8cc82700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d88102270 0x7f0d88108220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:20.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.226+0000 7f0d8cc82700 1 -- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d880f9670 msgr2=0x7f0d8810e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:20.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.226+0000 7f0d8cc82700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d880f9670 0x7f0d8810e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:20.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.226+0000 7f0d8cc82700 1 -- 172.21.15.47:0/4147048992 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c005040 con 0x7f0d880f8d10 2024-02-23T00:45:20.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.226+0000 7f0d8cc82700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0d880f8d10 0x7f0d8810e080 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 rx=0x7f0d7c0048f0 tx=0x7f0d7c009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:20.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.226+0000 7f0d85ffb700 10 client.?.objecter ms_handle_connect 0x7f0d880f8d10 2024-02-23T00:45:20.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.226+0000 7f0d85ffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:45:20.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.226+0000 7f0d85ffb700 1 -- 172.21.15.47:0/4147048992 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c007b00 con 0x7f0d880f8d10 2024-02-23T00:45:20.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.226+0000 7f0d85ffb700 1 -- 172.21.15.47:0/4147048992 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f0d7c00f040 con 0x7f0d880f8d10 2024-02-23T00:45:20.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.226+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d74005040 con 0x7f0d880f8d10 2024-02-23T00:45:20.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.226+0000 7f0d85ffb700 1 -- 172.21.15.47:0/4147048992 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c014890 con 0x7f0d880f8d10 2024-02-23T00:45:20.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.226+0000 7f0d8eee6700 10 client.40877.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:45:20.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.226+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d88109050 con 0x7f0d880f8d10 2024-02-23T00:45:20.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.226+0000 7f0d8eee6700 1 librados: init done 2024-02-23T00:45:20.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.227+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f0d8805fa60 con 0x7f0d880f8d10 2024-02-23T00:45:20.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.227+0000 7f0d85ffb700 1 -- 172.21.15.47:0/4147048992 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f0d7c014cc0 con 0x7f0d880f8d10 2024-02-23T00:45:20.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.228+0000 7f0d85ffb700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0d78062cc0 0x7f0d78065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:20.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.228+0000 7f0d85ffb700 1 -- 172.21.15.47:0/4147048992 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(248..248 src has 48..248) v4 ==== 10609+0+0 (secure 0 0 0) 0x7f0d7c07d7f0 con 0x7f0d880f8d10 2024-02-23T00:45:20.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.228+0000 7f0d85ffb700 10 client.40877.objecter ms_dispatch 0x7f0d88001550 osd_map(248..248 src has 48..248) v4 2024-02-23T00:45:20.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.228+0000 7f0d85ffb700 3 client.40877.objecter handle_osd_map got epochs [248,248] > 0 2024-02-23T00:45:20.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.228+0000 7f0d85ffb700 3 client.40877.objecter handle_osd_map decoding full epoch 248 2024-02-23T00:45:20.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.228+0000 7f0d87fff700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0d78062cc0 0x7f0d78065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:20.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.228+0000 7f0d85ffb700 20 client.40877.objecter dump_active .. 0 homeless 2024-02-23T00:45:20.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.228+0000 7f0d85ffb700 1 -- 172.21.15.47:0/4147048992 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v248) v1 ==== 54+0+16 (secure 0 0 0) 0x7f0d7c084070 con 0x7f0d880f8d10 2024-02-23T00:45:20.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.228+0000 7f0d85ffb700 10 client.40877.objecter ms_handle_connect 0x7f0d78062cc0 2024-02-23T00:45:20.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.228+0000 7f0d8eee6700 10 client.40877.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:20.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.228+0000 7f0d87fff700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0d78062cc0 0x7f0d78065180 secure :-1 s=READY pgs=2984 cs=0 l=1 rev1=1 rx=0x7f0d74004ee0 tx=0x7f0d7400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:20.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.228+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_assign -1 1 2024-02-23T00:45:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.228+0000 7f0d8eee6700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f0d88028540 0x7f0d880fbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.228+0000 7f0d8eee6700 20 client.40877.objecter _get_session s=0x7f0d88107060 osd=0 3 2024-02-23T00:45:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.228+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d88107060 osd=0 3 2024-02-23T00:45:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.228+0000 7f0d8eee6700 20 client.40877.objecter _get_session s=0x7f0d88107060 osd=0 3 2024-02-23T00:45:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.228+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_remove -1 1 2024-02-23T00:45:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.228+0000 7f0d8eee6700 20 client.40877.objecter get_session s=0x7f0d88107060 osd=0 3 2024-02-23T00:45:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.228+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_assign 0 1 2024-02-23T00:45:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.228+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d88107060 osd=0 4 2024-02-23T00:45:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.228+0000 7f0d8eee6700 10 client.40877.objecter _send_command 1 2024-02-23T00:45:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.228+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0d880fd310 con 0x7f0d88028540 2024-02-23T00:45:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.228+0000 7f0d8d483700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f0d88028540 0x7f0d880fbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.229+0000 7f0d85ffb700 10 client.40877.objecter ms_handle_connect 0x7f0d88028540 2024-02-23T00:45:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.229+0000 7f0d8d483700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f0d88028540 0x7f0d880fbca0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:20.256 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:45:20.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.229+0000 7f0d85ffb700 1 -- 172.21.15.47:0/4147048992 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0d880fd310 con 0x7f0d88028540 2024-02-23T00:45:20.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.229+0000 7f0d85ffb700 10 client.40877.objecter ms_dispatch 0x7f0d88001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:45:20.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.229+0000 7f0d85ffb700 10 client.40877.objecter _finish_command 1 = system:0 2024-02-23T00:45:20.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.229+0000 7f0d85ffb700 20 client.40877.objecter put_session s=0x7f0d88107060 osd=0 4 2024-02-23T00:45:20.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.229+0000 7f0d85ffb700 15 client.40877.objecter _session_command_op_remove 0 1 2024-02-23T00:45:20.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.244+0000 7f0d8eee6700 10 client.40877.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:45:20.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.244+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_assign -1 2 2024-02-23T00:45:20.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.244+0000 7f0d8eee6700 20 client.40877.objecter _get_session s=0x7f0d88107060 osd=0 3 2024-02-23T00:45:20.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.244+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d88107060 osd=0 3 2024-02-23T00:45:20.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.244+0000 7f0d8eee6700 20 client.40877.objecter _get_session s=0x7f0d88107060 osd=0 3 2024-02-23T00:45:20.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.244+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_remove -1 2 2024-02-23T00:45:20.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.244+0000 7f0d8eee6700 20 client.40877.objecter get_session s=0x7f0d88107060 osd=0 3 2024-02-23T00:45:20.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.244+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_assign 0 2 2024-02-23T00:45:20.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.244+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d88107060 osd=0 4 2024-02-23T00:45:20.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.244+0000 7f0d8eee6700 10 client.40877.objecter _send_command 2 2024-02-23T00:45:20.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.244+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0d880fd480 con 0x7f0d88028540 2024-02-23T00:45:20.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.246+0000 7f0d85ffb700 1 -- 172.21.15.47:0/4147048992 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0d880fd480 con 0x7f0d88028540 2024-02-23T00:45:20.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.246+0000 7f0d85ffb700 10 client.40877.objecter ms_dispatch 0x7f0d88001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:45:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.246+0000 7f0d85ffb700 10 client.40877.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:45:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.246+0000 7f0d85ffb700 20 client.40877.objecter put_session s=0x7f0d88107060 osd=0 4 2024-02-23T00:45:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.246+0000 7f0d85ffb700 15 client.40877.objecter _session_command_op_remove 0 2 2024-02-23T00:45:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.246+0000 7f0d8eee6700 10 client.40877.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.246+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_assign -1 3 2024-02-23T00:45:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.246+0000 7f0d8eee6700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f0d880fc810 0x7f0d880fcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.246+0000 7f0d8eee6700 20 client.40877.objecter _get_session s=0x7f0d880fc190 osd=1 3 2024-02-23T00:45:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.246+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d880fc190 osd=1 3 2024-02-23T00:45:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.246+0000 7f0d8eee6700 20 client.40877.objecter _get_session s=0x7f0d880fc190 osd=1 3 2024-02-23T00:45:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.246+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_remove -1 3 2024-02-23T00:45:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.246+0000 7f0d8eee6700 20 client.40877.objecter get_session s=0x7f0d880fc190 osd=1 3 2024-02-23T00:45:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.246+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_assign 1 3 2024-02-23T00:45:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.246+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d880fc190 osd=1 4 2024-02-23T00:45:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.246+0000 7f0d8eee6700 10 client.40877.objecter _send_command 3 2024-02-23T00:45:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.246+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0d880f8b10 con 0x7f0d880fc810 2024-02-23T00:45:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.246+0000 7f0d8cc82700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f0d880fc810 0x7f0d880fcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:20.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.246+0000 7f0d85ffb700 10 client.40877.objecter ms_handle_connect 0x7f0d880fc810 2024-02-23T00:45:20.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.246+0000 7f0d8cc82700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f0d880fc810 0x7f0d880fcbf0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:20.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.247+0000 7f0d85ffb700 1 -- 172.21.15.47:0/4147048992 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0d880f8b10 con 0x7f0d880fc810 2024-02-23T00:45:20.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.247+0000 7f0d85ffb700 10 client.40877.objecter ms_dispatch 0x7f0d88001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:45:20.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.247+0000 7f0d85ffb700 10 client.40877.objecter _finish_command 3 = system:0 2024-02-23T00:45:20.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.247+0000 7f0d85ffb700 20 client.40877.objecter put_session s=0x7f0d880fc190 osd=1 4 2024-02-23T00:45:20.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.247+0000 7f0d85ffb700 15 client.40877.objecter _session_command_op_remove 1 3 2024-02-23T00:45:20.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.263+0000 7f0d8eee6700 10 client.40877.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:45:20.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.263+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_assign -1 4 2024-02-23T00:45:20.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.263+0000 7f0d8eee6700 20 client.40877.objecter _get_session s=0x7f0d880fc190 osd=1 3 2024-02-23T00:45:20.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.263+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d880fc190 osd=1 3 2024-02-23T00:45:20.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.263+0000 7f0d8eee6700 20 client.40877.objecter _get_session s=0x7f0d880fc190 osd=1 3 2024-02-23T00:45:20.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.263+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_remove -1 4 2024-02-23T00:45:20.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.263+0000 7f0d8eee6700 20 client.40877.objecter get_session s=0x7f0d880fc190 osd=1 3 2024-02-23T00:45:20.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.263+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_assign 1 4 2024-02-23T00:45:20.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.263+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d880fc190 osd=1 4 2024-02-23T00:45:20.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.263+0000 7f0d8eee6700 10 client.40877.objecter _send_command 4 2024-02-23T00:45:20.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.263+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0d880fd0e0 con 0x7f0d880fc810 2024-02-23T00:45:20.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.264+0000 7f0d85ffb700 1 -- 172.21.15.47:0/4147048992 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0d880fd0e0 con 0x7f0d880fc810 2024-02-23T00:45:20.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.264+0000 7f0d85ffb700 10 client.40877.objecter ms_dispatch 0x7f0d88001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:45:20.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.264+0000 7f0d85ffb700 10 client.40877.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:45:20.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.264+0000 7f0d85ffb700 20 client.40877.objecter put_session s=0x7f0d880fc190 osd=1 4 2024-02-23T00:45:20.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.264+0000 7f0d85ffb700 15 client.40877.objecter _session_command_op_remove 1 4 2024-02-23T00:45:20.267 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:45:20.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.265+0000 7f0d8eee6700 10 client.40877.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:20.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.265+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_assign -1 5 2024-02-23T00:45:20.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.265+0000 7f0d8eee6700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f0d880280a0 0x7f0d8811e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:20.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.265+0000 7f0d8eee6700 20 client.40877.objecter _get_session s=0x7f0d880595a0 osd=2 3 2024-02-23T00:45:20.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.265+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d880595a0 osd=2 3 2024-02-23T00:45:20.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.265+0000 7f0d8eee6700 20 client.40877.objecter _get_session s=0x7f0d880595a0 osd=2 3 2024-02-23T00:45:20.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.265+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_remove -1 5 2024-02-23T00:45:20.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.265+0000 7f0d8eee6700 20 client.40877.objecter get_session s=0x7f0d880595a0 osd=2 3 2024-02-23T00:45:20.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.265+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_assign 2 5 2024-02-23T00:45:20.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.265+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d880595a0 osd=2 4 2024-02-23T00:45:20.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.265+0000 7f0d8eee6700 10 client.40877.objecter _send_command 5 2024-02-23T00:45:20.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.265+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0d88121110 con 0x7f0d880280a0 2024-02-23T00:45:20.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.265+0000 7f0d87fff700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f0d880280a0 0x7f0d8811e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:20.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.265+0000 7f0d85ffb700 10 client.40877.objecter ms_handle_connect 0x7f0d880280a0 2024-02-23T00:45:20.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.265+0000 7f0d87fff700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f0d880280a0 0x7f0d8811e4c0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:20.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.266+0000 7f0d85ffb700 1 -- 172.21.15.47:0/4147048992 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0d88121110 con 0x7f0d880280a0 2024-02-23T00:45:20.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.266+0000 7f0d85ffb700 10 client.40877.objecter ms_dispatch 0x7f0d88001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:45:20.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.266+0000 7f0d85ffb700 10 client.40877.objecter _finish_command 5 = system:0 2024-02-23T00:45:20.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.266+0000 7f0d85ffb700 20 client.40877.objecter put_session s=0x7f0d880595a0 osd=2 4 2024-02-23T00:45:20.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.266+0000 7f0d85ffb700 15 client.40877.objecter _session_command_op_remove 2 5 2024-02-23T00:45:20.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.287+0000 7f0d8eee6700 10 client.40877.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:45:20.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.287+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_assign -1 6 2024-02-23T00:45:20.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.287+0000 7f0d8eee6700 20 client.40877.objecter _get_session s=0x7f0d880595a0 osd=2 3 2024-02-23T00:45:20.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.287+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d880595a0 osd=2 3 2024-02-23T00:45:20.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.287+0000 7f0d8eee6700 20 client.40877.objecter _get_session s=0x7f0d880595a0 osd=2 3 2024-02-23T00:45:20.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.287+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_remove -1 6 2024-02-23T00:45:20.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.287+0000 7f0d8eee6700 20 client.40877.objecter get_session s=0x7f0d880595a0 osd=2 3 2024-02-23T00:45:20.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.287+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_assign 2 6 2024-02-23T00:45:20.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.287+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d880595a0 osd=2 4 2024-02-23T00:45:20.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.287+0000 7f0d8eee6700 10 client.40877.objecter _send_command 6 2024-02-23T00:45:20.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.287+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0d88027600 con 0x7f0d880280a0 2024-02-23T00:45:20.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.289+0000 7f0d85ffb700 1 -- 172.21.15.47:0/4147048992 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0d88027600 con 0x7f0d880280a0 2024-02-23T00:45:20.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.289+0000 7f0d85ffb700 10 client.40877.objecter ms_dispatch 0x7f0d88001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:45:20.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.289+0000 7f0d85ffb700 10 client.40877.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:45:20.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.289+0000 7f0d85ffb700 20 client.40877.objecter put_session s=0x7f0d880595a0 osd=2 4 2024-02-23T00:45:20.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.289+0000 7f0d85ffb700 15 client.40877.objecter _session_command_op_remove 2 6 2024-02-23T00:45:20.291 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:45:20.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.289+0000 7f0d8eee6700 10 client.40877.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:20.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.289+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_assign -1 7 2024-02-23T00:45:20.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.289+0000 7f0d8eee6700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f0d8811c010 0x7f0d8811fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:20.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.289+0000 7f0d8eee6700 20 client.40877.objecter _get_session s=0x7f0d88027a10 osd=3 3 2024-02-23T00:45:20.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.289+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d88027a10 osd=3 3 2024-02-23T00:45:20.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.289+0000 7f0d8eee6700 20 client.40877.objecter _get_session s=0x7f0d88027a10 osd=3 3 2024-02-23T00:45:20.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.289+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_remove -1 7 2024-02-23T00:45:20.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.289+0000 7f0d8eee6700 20 client.40877.objecter get_session s=0x7f0d88027a10 osd=3 3 2024-02-23T00:45:20.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.289+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_assign 3 7 2024-02-23T00:45:20.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.289+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d88027a10 osd=3 4 2024-02-23T00:45:20.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.289+0000 7f0d8eee6700 10 client.40877.objecter _send_command 7 2024-02-23T00:45:20.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.289+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0d881200b0 con 0x7f0d8811c010 2024-02-23T00:45:20.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.289+0000 7f0d8d483700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f0d8811c010 0x7f0d8811fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:20.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.289+0000 7f0d85ffb700 10 client.40877.objecter ms_handle_connect 0x7f0d8811c010 2024-02-23T00:45:20.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.289+0000 7f0d8d483700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f0d8811c010 0x7f0d8811fb30 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:20.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.290+0000 7f0d85ffb700 1 -- 172.21.15.47:0/4147048992 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0d881200b0 con 0x7f0d8811c010 2024-02-23T00:45:20.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.290+0000 7f0d85ffb700 10 client.40877.objecter ms_dispatch 0x7f0d88001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:45:20.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.290+0000 7f0d85ffb700 10 client.40877.objecter _finish_command 7 = system:0 2024-02-23T00:45:20.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.290+0000 7f0d85ffb700 20 client.40877.objecter put_session s=0x7f0d88027a10 osd=3 4 2024-02-23T00:45:20.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.290+0000 7f0d85ffb700 15 client.40877.objecter _session_command_op_remove 3 7 2024-02-23T00:45:20.306 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:18.184985+0000 osd.5 (osd.5) 116 : cluster [DBG] 18.15 scrub ok 2024-02-23T00:45:20.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:18.641441+0000 osd.1 (osd.1) 129 : cluster [DBG] 15.1c scrub starts 2024-02-23T00:45:20.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:18.644530+0000 osd.1 (osd.1) 130 : cluster [DBG] 15.1c scrub ok 2024-02-23T00:45:20.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:18.946419+0000 osd.3 (osd.3) 133 : cluster [DBG] 18.1 scrub starts 2024-02-23T00:45:20.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:18.947738+0000 osd.3 (osd.3) 134 : cluster [DBG] 18.1 scrub ok 2024-02-23T00:45:20.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:19.138252+0000 mon.a (mon.0) 1694 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi047-82378-33"}]': finished 2024-02-23T00:45:20.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:19.138356+0000 mon.a (mon.0) 1695 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:45:20.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:19.138411+0000 mon.a (mon.0) 1696 : cluster [DBG] osdmap e247: 8 total, 8 up, 8 in 2024-02-23T00:45:20.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:19.143416+0000 mon.a (mon.0) 1697 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-25"}]: dispatch 2024-02-23T00:45:20.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:19.169527+0000 mon.a (mon.0) 1698 : audit [INF] from='client.? 172.21.15.47:0/290258726' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi047-82378-34"}]: dispatch 2024-02-23T00:45:20.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:19.170479+0000 mon.a (mon.0) 1699 : audit [INF] from='client.? 172.21.15.47:0/290258726' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi047-82378-34"}]: dispatch 2024-02-23T00:45:20.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:19.170909+0000 mon.a (mon.0) 1700 : audit [INF] from='client.? 172.21.15.47:0/290258726' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi047-82378-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:20.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:19.238739+0000 mgr.y (mgr.14517) 267 : cluster [DBG] pgmap v346: 346 pgs: 6 creating+activating, 1 active, 1 active+clean+scrubbing, 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 331 active+clean; 9.0 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:45:20.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:19.819302+0000 mon.b (mon.1) 307 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:20.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:18.184985+0000 osd.5 (osd.5) 116 : cluster [DBG] 18.15 scrub ok 2024-02-23T00:45:20.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:18.641441+0000 osd.1 (osd.1) 129 : cluster [DBG] 15.1c scrub starts 2024-02-23T00:45:20.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:18.644530+0000 osd.1 (osd.1) 130 : cluster [DBG] 15.1c scrub ok 2024-02-23T00:45:20.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:18.946419+0000 osd.3 (osd.3) 133 : cluster [DBG] 18.1 scrub starts 2024-02-23T00:45:20.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:18.947738+0000 osd.3 (osd.3) 134 : cluster [DBG] 18.1 scrub ok 2024-02-23T00:45:20.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:19.138252+0000 mon.a (mon.0) 1694 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi047-82378-33"}]': finished 2024-02-23T00:45:20.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:19.138356+0000 mon.a (mon.0) 1695 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:45:20.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:19.138411+0000 mon.a (mon.0) 1696 : cluster [DBG] osdmap e247: 8 total, 8 up, 8 in 2024-02-23T00:45:20.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:19.143416+0000 mon.a (mon.0) 1697 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-25"}]: dispatch 2024-02-23T00:45:20.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:19.169527+0000 mon.a (mon.0) 1698 : audit [INF] from='client.? 172.21.15.47:0/290258726' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi047-82378-34"}]: dispatch 2024-02-23T00:45:20.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:19.170479+0000 mon.a (mon.0) 1699 : audit [INF] from='client.? 172.21.15.47:0/290258726' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi047-82378-34"}]: dispatch 2024-02-23T00:45:20.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:19.170909+0000 mon.a (mon.0) 1700 : audit [INF] from='client.? 172.21.15.47:0/290258726' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi047-82378-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:20.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:19.238739+0000 mgr.y (mgr.14517) 267 : cluster [DBG] pgmap v346: 346 pgs: 6 creating+activating, 1 active, 1 active+clean+scrubbing, 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 331 active+clean; 9.0 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:45:20.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:19.819302+0000 mon.b (mon.1) 307 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:20.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:20.126305+0000 mon.a (mon.0) 1701 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-25"}]': finished 2024-02-23T00:45:20.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:20.126414+0000 mon.a (mon.0) 1702 : audit [INF] from='client.? 172.21.15.47:0/290258726' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi047-82378-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:20.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:20.126470+0000 mon.a (mon.0) 1703 : cluster [DBG] osdmap e248: 8 total, 8 up, 8 in 2024-02-23T00:45:20.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:20.130522+0000 mon.a (mon.0) 1704 : audit [INF] from='client.? 172.21.15.47:0/290258726' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi047-82378-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi047-82378-34"}]: dispatch 2024-02-23T00:45:20.310 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:20.130957+0000 mon.a (mon.0) 1705 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-25", "mode": "writeback"}]: dispatch 2024-02-23T00:45:20.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:20.126305+0000 mon.a (mon.0) 1701 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-25"}]': finished 2024-02-23T00:45:20.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:20.126414+0000 mon.a (mon.0) 1702 : audit [INF] from='client.? 172.21.15.47:0/290258726' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi047-82378-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:20.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:20.126470+0000 mon.a (mon.0) 1703 : cluster [DBG] osdmap e248: 8 total, 8 up, 8 in 2024-02-23T00:45:20.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:20.130522+0000 mon.a (mon.0) 1704 : audit [INF] from='client.? 172.21.15.47:0/290258726' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi047-82378-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi047-82378-34"}]: dispatch 2024-02-23T00:45:20.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:20.130957+0000 mon.a (mon.0) 1705 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-25", "mode": "writeback"}]: dispatch 2024-02-23T00:45:20.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.305+0000 7f0d8eee6700 10 client.40877.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:45:20.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.305+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_assign -1 8 2024-02-23T00:45:20.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.305+0000 7f0d8eee6700 20 client.40877.objecter _get_session s=0x7f0d88027a10 osd=3 3 2024-02-23T00:45:20.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.305+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d88027a10 osd=3 3 2024-02-23T00:45:20.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.305+0000 7f0d8eee6700 20 client.40877.objecter _get_session s=0x7f0d88027a10 osd=3 3 2024-02-23T00:45:20.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.305+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_remove -1 8 2024-02-23T00:45:20.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.305+0000 7f0d8eee6700 20 client.40877.objecter get_session s=0x7f0d88027a10 osd=3 3 2024-02-23T00:45:20.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.305+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_assign 3 8 2024-02-23T00:45:20.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.305+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d88027a10 osd=3 4 2024-02-23T00:45:20.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.305+0000 7f0d8eee6700 10 client.40877.objecter _send_command 8 2024-02-23T00:45:20.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.305+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0d8811b570 con 0x7f0d8811c010 2024-02-23T00:45:20.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.307+0000 7f0d85ffb700 1 -- 172.21.15.47:0/4147048992 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0d8811b570 con 0x7f0d8811c010 2024-02-23T00:45:20.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.307+0000 7f0d85ffb700 10 client.40877.objecter ms_dispatch 0x7f0d88001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:45:20.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.307+0000 7f0d85ffb700 10 client.40877.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:45:20.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.307+0000 7f0d85ffb700 20 client.40877.objecter put_session s=0x7f0d88027a10 osd=3 4 2024-02-23T00:45:20.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.307+0000 7f0d85ffb700 15 client.40877.objecter _session_command_op_remove 3 8 2024-02-23T00:45:20.312 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:45:20.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.307+0000 7f0d8eee6700 10 client.40877.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:20.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.307+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_assign -1 9 2024-02-23T00:45:20.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.307+0000 7f0d8eee6700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f0d8811b710 0x7f0d881252a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:20.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.307+0000 7f0d8eee6700 20 client.40877.objecter _get_session s=0x7f0d8811b570 osd=4 3 2024-02-23T00:45:20.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.307+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d8811b570 osd=4 3 2024-02-23T00:45:20.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.307+0000 7f0d8eee6700 20 client.40877.objecter _get_session s=0x7f0d8811b570 osd=4 3 2024-02-23T00:45:20.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.307+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_remove -1 9 2024-02-23T00:45:20.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.307+0000 7f0d8eee6700 20 client.40877.objecter get_session s=0x7f0d8811b570 osd=4 3 2024-02-23T00:45:20.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.307+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_assign 4 9 2024-02-23T00:45:20.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.307+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d8811b570 osd=4 4 2024-02-23T00:45:20.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.307+0000 7f0d8eee6700 10 client.40877.objecter _send_command 9 2024-02-23T00:45:20.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.307+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0d88127fa0 con 0x7f0d8811b710 2024-02-23T00:45:20.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.307+0000 7f0d8cc82700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f0d8811b710 0x7f0d881252a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:20.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.307+0000 7f0d85ffb700 10 client.40877.objecter ms_handle_connect 0x7f0d8811b710 2024-02-23T00:45:20.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.307+0000 7f0d8cc82700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f0d8811b710 0x7f0d881252a0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:20.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.308+0000 7f0d85ffb700 1 -- 172.21.15.47:0/4147048992 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0d88127fa0 con 0x7f0d8811b710 2024-02-23T00:45:20.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.308+0000 7f0d85ffb700 10 client.40877.objecter ms_dispatch 0x7f0d88001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:45:20.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.308+0000 7f0d85ffb700 10 client.40877.objecter _finish_command 9 = system:0 2024-02-23T00:45:20.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.308+0000 7f0d85ffb700 20 client.40877.objecter put_session s=0x7f0d8811b570 osd=4 4 2024-02-23T00:45:20.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.308+0000 7f0d85ffb700 15 client.40877.objecter _session_command_op_remove 4 9 2024-02-23T00:45:20.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.323+0000 7f0d8eee6700 10 client.40877.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:45:20.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.323+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_assign -1 10 2024-02-23T00:45:20.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.323+0000 7f0d8eee6700 20 client.40877.objecter _get_session s=0x7f0d8811b570 osd=4 3 2024-02-23T00:45:20.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.323+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d8811b570 osd=4 3 2024-02-23T00:45:20.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.323+0000 7f0d8eee6700 20 client.40877.objecter _get_session s=0x7f0d8811b570 osd=4 3 2024-02-23T00:45:20.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.323+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_remove -1 10 2024-02-23T00:45:20.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.323+0000 7f0d8eee6700 20 client.40877.objecter get_session s=0x7f0d8811b570 osd=4 3 2024-02-23T00:45:20.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.323+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_assign 4 10 2024-02-23T00:45:20.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.323+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d8811b570 osd=4 4 2024-02-23T00:45:20.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.323+0000 7f0d8eee6700 10 client.40877.objecter _send_command 10 2024-02-23T00:45:20.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.323+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0d88127fa0 con 0x7f0d8811b710 2024-02-23T00:45:20.327 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:45:20.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.325+0000 7f0d85ffb700 1 -- 172.21.15.47:0/4147048992 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0d88127fa0 con 0x7f0d8811b710 2024-02-23T00:45:20.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.325+0000 7f0d85ffb700 10 client.40877.objecter ms_dispatch 0x7f0d88001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:45:20.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.325+0000 7f0d85ffb700 10 client.40877.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:45:20.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.325+0000 7f0d85ffb700 20 client.40877.objecter put_session s=0x7f0d8811b570 osd=4 4 2024-02-23T00:45:20.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.325+0000 7f0d85ffb700 15 client.40877.objecter _session_command_op_remove 4 10 2024-02-23T00:45:20.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.325+0000 7f0d8eee6700 10 client.40877.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:20.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.325+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_assign -1 11 2024-02-23T00:45:20.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.325+0000 7f0d8eee6700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f0d88125cf0 0x7f0d88126170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:20.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.325+0000 7f0d8eee6700 20 client.40877.objecter _get_session s=0x7f0d88127fa0 osd=5 3 2024-02-23T00:45:20.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.325+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d88127fa0 osd=5 3 2024-02-23T00:45:20.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.325+0000 7f0d8eee6700 20 client.40877.objecter _get_session s=0x7f0d88127fa0 osd=5 3 2024-02-23T00:45:20.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.325+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_remove -1 11 2024-02-23T00:45:20.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.325+0000 7f0d8eee6700 20 client.40877.objecter get_session s=0x7f0d88127fa0 osd=5 3 2024-02-23T00:45:20.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.325+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_assign 5 11 2024-02-23T00:45:20.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.325+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d88127fa0 osd=5 4 2024-02-23T00:45:20.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.325+0000 7f0d8eee6700 10 client.40877.objecter _send_command 11 2024-02-23T00:45:20.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.325+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0d8812b2d0 con 0x7f0d88125cf0 2024-02-23T00:45:20.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.325+0000 7f0d87fff700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f0d88125cf0 0x7f0d88126170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:20.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.325+0000 7f0d85ffb700 10 client.40877.objecter ms_handle_connect 0x7f0d88125cf0 2024-02-23T00:45:20.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.325+0000 7f0d87fff700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f0d88125cf0 0x7f0d88126170 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:20.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.326+0000 7f0d85ffb700 1 -- 172.21.15.47:0/4147048992 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0d8812b2d0 con 0x7f0d88125cf0 2024-02-23T00:45:20.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.326+0000 7f0d85ffb700 10 client.40877.objecter ms_dispatch 0x7f0d88001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:45:20.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.326+0000 7f0d85ffb700 10 client.40877.objecter _finish_command 11 = system:0 2024-02-23T00:45:20.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.326+0000 7f0d85ffb700 20 client.40877.objecter put_session s=0x7f0d88127fa0 osd=5 4 2024-02-23T00:45:20.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.326+0000 7f0d85ffb700 15 client.40877.objecter _session_command_op_remove 5 11 2024-02-23T00:45:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.341+0000 7f0d8eee6700 10 client.40877.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:45:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.341+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_assign -1 12 2024-02-23T00:45:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.341+0000 7f0d8eee6700 20 client.40877.objecter _get_session s=0x7f0d88127fa0 osd=5 3 2024-02-23T00:45:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.341+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d88127fa0 osd=5 3 2024-02-23T00:45:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.341+0000 7f0d8eee6700 20 client.40877.objecter _get_session s=0x7f0d88127fa0 osd=5 3 2024-02-23T00:45:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.341+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_remove -1 12 2024-02-23T00:45:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.341+0000 7f0d8eee6700 20 client.40877.objecter get_session s=0x7f0d88127fa0 osd=5 3 2024-02-23T00:45:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.341+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_assign 5 12 2024-02-23T00:45:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.341+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d88127fa0 osd=5 4 2024-02-23T00:45:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.341+0000 7f0d8eee6700 10 client.40877.objecter _send_command 12 2024-02-23T00:45:20.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.341+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0d88126770 con 0x7f0d88125cf0 2024-02-23T00:45:20.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.342+0000 7f0d85ffb700 1 -- 172.21.15.47:0/4147048992 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0d88126770 con 0x7f0d88125cf0 2024-02-23T00:45:20.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.342+0000 7f0d85ffb700 10 client.40877.objecter ms_dispatch 0x7f0d88001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:45:20.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.342+0000 7f0d85ffb700 10 client.40877.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:45:20.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.342+0000 7f0d85ffb700 20 client.40877.objecter put_session s=0x7f0d88127fa0 osd=5 4 2024-02-23T00:45:20.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.342+0000 7f0d85ffb700 15 client.40877.objecter _session_command_op_remove 5 12 2024-02-23T00:45:20.345 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:45:20.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.343+0000 7f0d8eee6700 10 client.40877.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:20.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.343+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_assign -1 13 2024-02-23T00:45:20.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.343+0000 7f0d8eee6700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f0d88126df0 0x7f0d88127250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:20.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.343+0000 7f0d8eee6700 20 client.40877.objecter _get_session s=0x7f0d88126770 osd=6 3 2024-02-23T00:45:20.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.343+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d88126770 osd=6 3 2024-02-23T00:45:20.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.343+0000 7f0d8eee6700 20 client.40877.objecter _get_session s=0x7f0d88126770 osd=6 3 2024-02-23T00:45:20.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.343+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_remove -1 13 2024-02-23T00:45:20.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.343+0000 7f0d8eee6700 20 client.40877.objecter get_session s=0x7f0d88126770 osd=6 3 2024-02-23T00:45:20.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.343+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_assign 6 13 2024-02-23T00:45:20.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.343+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d88126770 osd=6 4 2024-02-23T00:45:20.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.343+0000 7f0d8eee6700 10 client.40877.objecter _send_command 13 2024-02-23T00:45:20.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.343+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0d881278a0 con 0x7f0d88126df0 2024-02-23T00:45:20.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.343+0000 7f0d8d483700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f0d88126df0 0x7f0d88127250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:20.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.343+0000 7f0d85ffb700 10 client.40877.objecter ms_handle_connect 0x7f0d88126df0 2024-02-23T00:45:20.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.343+0000 7f0d8d483700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f0d88126df0 0x7f0d88127250 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:20.347 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:45:20.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.344+0000 7f0d85ffb700 1 -- 172.21.15.47:0/4147048992 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0d881278a0 con 0x7f0d88126df0 2024-02-23T00:45:20.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.344+0000 7f0d85ffb700 10 client.40877.objecter ms_dispatch 0x7f0d88001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:45:20.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.344+0000 7f0d85ffb700 10 client.40877.objecter _finish_command 13 = system:0 2024-02-23T00:45:20.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.344+0000 7f0d85ffb700 20 client.40877.objecter put_session s=0x7f0d88126770 osd=6 4 2024-02-23T00:45:20.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.344+0000 7f0d85ffb700 15 client.40877.objecter _session_command_op_remove 6 13 2024-02-23T00:45:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.360+0000 7f0d8eee6700 10 client.40877.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:45:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.361+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_assign -1 14 2024-02-23T00:45:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.361+0000 7f0d8eee6700 20 client.40877.objecter _get_session s=0x7f0d88126770 osd=6 3 2024-02-23T00:45:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.361+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d88126770 osd=6 3 2024-02-23T00:45:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.361+0000 7f0d8eee6700 20 client.40877.objecter _get_session s=0x7f0d88126770 osd=6 3 2024-02-23T00:45:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.361+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_remove -1 14 2024-02-23T00:45:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.361+0000 7f0d8eee6700 20 client.40877.objecter get_session s=0x7f0d88126770 osd=6 3 2024-02-23T00:45:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.361+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_assign 6 14 2024-02-23T00:45:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.361+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d88126770 osd=6 4 2024-02-23T00:45:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.361+0000 7f0d8eee6700 10 client.40877.objecter _send_command 14 2024-02-23T00:45:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.361+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0d881278a0 con 0x7f0d88126df0 2024-02-23T00:45:20.365 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:45:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.362+0000 7f0d85ffb700 1 -- 172.21.15.47:0/4147048992 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0d881278a0 con 0x7f0d88126df0 2024-02-23T00:45:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.362+0000 7f0d85ffb700 10 client.40877.objecter ms_dispatch 0x7f0d88001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:45:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.362+0000 7f0d85ffb700 10 client.40877.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:45:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.362+0000 7f0d85ffb700 20 client.40877.objecter put_session s=0x7f0d88126770 osd=6 4 2024-02-23T00:45:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.362+0000 7f0d85ffb700 15 client.40877.objecter _session_command_op_remove 6 14 2024-02-23T00:45:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.363+0000 7f0d8eee6700 10 client.40877.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.363+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_assign -1 15 2024-02-23T00:45:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.363+0000 7f0d8eee6700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f0d8812c670 0x7f0d8812ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.363+0000 7f0d8eee6700 20 client.40877.objecter _get_session s=0x7f0d881278a0 osd=7 3 2024-02-23T00:45:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.363+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d881278a0 osd=7 3 2024-02-23T00:45:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.363+0000 7f0d8eee6700 20 client.40877.objecter _get_session s=0x7f0d881278a0 osd=7 3 2024-02-23T00:45:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.363+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_remove -1 15 2024-02-23T00:45:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.363+0000 7f0d8eee6700 20 client.40877.objecter get_session s=0x7f0d881278a0 osd=7 3 2024-02-23T00:45:20.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.363+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_assign 7 15 2024-02-23T00:45:20.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.363+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d881278a0 osd=7 4 2024-02-23T00:45:20.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.363+0000 7f0d8eee6700 10 client.40877.objecter _send_command 15 2024-02-23T00:45:20.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.363+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0d881316b0 con 0x7f0d8812c670 2024-02-23T00:45:20.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.363+0000 7f0d8cc82700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f0d8812c670 0x7f0d8812ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:20.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.364+0000 7f0d85ffb700 10 client.40877.objecter ms_handle_connect 0x7f0d8812c670 2024-02-23T00:45:20.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.364+0000 7f0d8cc82700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f0d8812c670 0x7f0d8812ea60 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:20.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.365+0000 7f0d85ffb700 1 -- 172.21.15.47:0/4147048992 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0d881316b0 con 0x7f0d8812c670 2024-02-23T00:45:20.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.365+0000 7f0d85ffb700 10 client.40877.objecter ms_dispatch 0x7f0d88001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:45:20.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.365+0000 7f0d85ffb700 10 client.40877.objecter _finish_command 15 = system:0 2024-02-23T00:45:20.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.365+0000 7f0d85ffb700 20 client.40877.objecter put_session s=0x7f0d881278a0 osd=7 4 2024-02-23T00:45:20.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.365+0000 7f0d85ffb700 15 client.40877.objecter _session_command_op_remove 7 15 2024-02-23T00:45:20.384 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:20.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.385+0000 7f0d8eee6700 10 client.40877.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:45:20.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.385+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_assign -1 16 2024-02-23T00:45:20.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.385+0000 7f0d8eee6700 20 client.40877.objecter _get_session s=0x7f0d881278a0 osd=7 3 2024-02-23T00:45:20.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.385+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d881278a0 osd=7 3 2024-02-23T00:45:20.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.385+0000 7f0d8eee6700 20 client.40877.objecter _get_session s=0x7f0d881278a0 osd=7 3 2024-02-23T00:45:20.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.385+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_remove -1 16 2024-02-23T00:45:20.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.385+0000 7f0d8eee6700 20 client.40877.objecter get_session s=0x7f0d881278a0 osd=7 3 2024-02-23T00:45:20.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.385+0000 7f0d8eee6700 15 client.40877.objecter _session_command_op_assign 7 16 2024-02-23T00:45:20.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.385+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d881278a0 osd=7 4 2024-02-23T00:45:20.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.385+0000 7f0d8eee6700 10 client.40877.objecter _send_command 16 2024-02-23T00:45:20.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.385+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0d8812ef50 con 0x7f0d8812c670 2024-02-23T00:45:20.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.387+0000 7f0d85ffb700 1 -- 172.21.15.47:0/4147048992 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0d8812ef50 con 0x7f0d8812c670 2024-02-23T00:45:20.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.387+0000 7f0d85ffb700 10 client.40877.objecter ms_dispatch 0x7f0d88001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:45:20.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.387+0000 7f0d85ffb700 10 client.40877.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:45:20.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.387+0000 7f0d85ffb700 20 client.40877.objecter put_session s=0x7f0d881278a0 osd=7 4 2024-02-23T00:45:20.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.387+0000 7f0d85ffb700 15 client.40877.objecter _session_command_op_remove 7 16 2024-02-23T00:45:20.390 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:45:20.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.387+0000 7f0d8eee6700 10 librados: watch_flush enter 2024-02-23T00:45:20.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.387+0000 7f0d8eee6700 10 librados: watch_flush exit 2024-02-23T00:45:20.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.387+0000 7f0d8eee6700 10 client.40877.objecter close_session for osd.0 2024-02-23T00:45:20.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.387+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f0d88028540 msgr2=0x7f0d880fbca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:20.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.387+0000 7f0d8eee6700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f0d88028540 0x7f0d880fbca0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:20.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.387+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d88107060 osd=0 1 2024-02-23T00:45:20.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.387+0000 7f0d8eee6700 10 client.40877.objecter close_session for osd.1 2024-02-23T00:45:20.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.387+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f0d880fc810 msgr2=0x7f0d880fcbf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:20.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.387+0000 7f0d8eee6700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f0d880fc810 0x7f0d880fcbf0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:20.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.387+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d880fc190 osd=1 1 2024-02-23T00:45:20.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.387+0000 7f0d8eee6700 10 client.40877.objecter close_session for osd.2 2024-02-23T00:45:20.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.387+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f0d880280a0 msgr2=0x7f0d8811e4c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:20.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.387+0000 7f0d8eee6700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f0d880280a0 0x7f0d8811e4c0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:20.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.387+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d880595a0 osd=2 1 2024-02-23T00:45:20.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.387+0000 7f0d8eee6700 10 client.40877.objecter close_session for osd.3 2024-02-23T00:45:20.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.387+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f0d8811c010 msgr2=0x7f0d8811fb30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:20.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.387+0000 7f0d8eee6700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f0d8811c010 0x7f0d8811fb30 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:20.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.387+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d88027a10 osd=3 1 2024-02-23T00:45:20.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.387+0000 7f0d8eee6700 10 client.40877.objecter close_session for osd.4 2024-02-23T00:45:20.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.387+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f0d8811b710 msgr2=0x7f0d881252a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:20.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.387+0000 7f0d8eee6700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f0d8811b710 0x7f0d881252a0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:20.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.387+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d8811b570 osd=4 1 2024-02-23T00:45:20.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.387+0000 7f0d8eee6700 10 client.40877.objecter close_session for osd.5 2024-02-23T00:45:20.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.387+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f0d88125cf0 msgr2=0x7f0d88126170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:20.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.388+0000 7f0d8eee6700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f0d88125cf0 0x7f0d88126170 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:20.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.388+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d88127fa0 osd=5 1 2024-02-23T00:45:20.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.388+0000 7f0d8eee6700 10 client.40877.objecter close_session for osd.6 2024-02-23T00:45:20.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.388+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f0d88126df0 msgr2=0x7f0d88127250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:20.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.388+0000 7f0d8eee6700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f0d88126df0 0x7f0d88127250 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:20.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.388+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d88126770 osd=6 1 2024-02-23T00:45:20.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.388+0000 7f0d8eee6700 10 client.40877.objecter close_session for osd.7 2024-02-23T00:45:20.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.388+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f0d8812c670 msgr2=0x7f0d8812ea60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:20.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.388+0000 7f0d8eee6700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f0d8812c670 0x7f0d8812ea60 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:20.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.388+0000 7f0d8eee6700 20 client.40877.objecter put_session s=0x7f0d881278a0 osd=7 1 2024-02-23T00:45:20.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.388+0000 7f0d8eee6700 20 client.40877.objecter shutdown clearing up homeless session... 2024-02-23T00:45:20.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.388+0000 7f0d8eee6700 10 client.40877.objecter successfully canceled tick 2024-02-23T00:45:20.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.388+0000 7f0d8d483700 1 -- 172.21.15.47:0/4147048992 reap_dead start 2024-02-23T00:45:20.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.388+0000 7f0d8d483700 1 -- 172.21.15.47:0/4147048992 reap_dead start 2024-02-23T00:45:20.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.388+0000 7f0d8d483700 1 -- 172.21.15.47:0/4147048992 reap_dead start 2024-02-23T00:45:20.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.388+0000 7f0d8d483700 1 -- 172.21.15.47:0/4147048992 reap_dead start 2024-02-23T00:45:20.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.388+0000 7f0d8d483700 1 -- 172.21.15.47:0/4147048992 reap_dead start 2024-02-23T00:45:20.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.388+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0d78062cc0 msgr2=0x7f0d78065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:20.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.388+0000 7f0d8eee6700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0d78062cc0 0x7f0d78065180 secure :-1 s=READY pgs=2984 cs=0 l=1 rev1=1 rx=0x7f0d74004ee0 tx=0x7f0d7400b040).stop 2024-02-23T00:45:20.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.388+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0d880f8d10 msgr2=0x7f0d8810e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:20.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.388+0000 7f0d8eee6700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0d880f8d10 0x7f0d8810e080 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 rx=0x7f0d7c0048f0 tx=0x7f0d7c009cb0).stop 2024-02-23T00:45:20.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.388+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 shutdown_connections 2024-02-23T00:45:20.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.388+0000 7f0d8eee6700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0d880f8d10 0x7f0d8810e080 unknown :-1 s=CLOSED pgs=2569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:20.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.388+0000 7f0d8eee6700 1 --2- 172.21.15.47:0/4147048992 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0d78062cc0 0x7f0d78065180 unknown :-1 s=CLOSED pgs=2984 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:20.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.388+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 >> 172.21.15.47:0/4147048992 conn(0x7f0d880f2820 msgr2=0x7f0d880f3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:45:20.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.388+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 shutdown_connections 2024-02-23T00:45:20.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.388+0000 7f0d8eee6700 1 -- 172.21.15.47:0/4147048992 wait complete. 2024-02-23T00:45:20.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:20.388+0000 7f0d8eee6700 1 librados: shutdown 2024-02-23T00:45:20.397 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:45:20.397 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:45:20.397 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:45:20.397 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:45:20.397 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:45:20.397 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:45:20.397 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:45:20.397 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:45:20.523 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:45:20.524 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:45:20.803 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:45:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:45:20] "GET /metrics HTTP/1.1" 200 194547 "" "Prometheus/2.33.4" 2024-02-23T00:45:21.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:19.631802+0000 osd.1 (osd.1) 131 : cluster [DBG] 2.4 scrub starts 2024-02-23T00:45:21.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:19.634999+0000 osd.1 (osd.1) 132 : cluster [DBG] 2.4 scrub ok 2024-02-23T00:45:21.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:19.710140+0000 osd.6 (osd.6) 101 : cluster [DBG] 15.9 scrub starts 2024-02-23T00:45:21.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:19.711111+0000 osd.6 (osd.6) 102 : cluster [DBG] 15.9 scrub ok 2024-02-23T00:45:21.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:20.229127+0000 mon.c (mon.2) 166 : audit [DBG] from='client.? 172.21.15.47:0/4147048992' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:45:21.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:20.572078+0000 osd.0 (osd.0) 113 : cluster [DBG] 2.1f scrub starts 2024-02-23T00:45:21.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:20.573154+0000 osd.0 (osd.0) 114 : cluster [DBG] 2.1f scrub ok 2024-02-23T00:45:21.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:20.820405+0000 mon.b (mon.1) 308 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:21.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:21.123654+0000 mon.a (mon.0) 1706 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:45:22.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:19.631802+0000 osd.1 (osd.1) 131 : cluster [DBG] 2.4 scrub starts 2024-02-23T00:45:22.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:19.634999+0000 osd.1 (osd.1) 132 : cluster [DBG] 2.4 scrub ok 2024-02-23T00:45:22.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:19.710140+0000 osd.6 (osd.6) 101 : cluster [DBG] 15.9 scrub starts 2024-02-23T00:45:22.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:19.711111+0000 osd.6 (osd.6) 102 : cluster [DBG] 15.9 scrub ok 2024-02-23T00:45:22.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:20.229127+0000 mon.c (mon.2) 166 : audit [DBG] from='client.? 172.21.15.47:0/4147048992' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:45:22.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:20.572078+0000 osd.0 (osd.0) 113 : cluster [DBG] 2.1f scrub starts 2024-02-23T00:45:22.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:20.573154+0000 osd.0 (osd.0) 114 : cluster [DBG] 2.1f scrub ok 2024-02-23T00:45:22.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:20.820405+0000 mon.b (mon.1) 308 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:22.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:21.123654+0000 mon.a (mon.0) 1706 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:45:22.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:19.631802+0000 osd.1 (osd.1) 131 : cluster [DBG] 2.4 scrub starts 2024-02-23T00:45:22.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:19.634999+0000 osd.1 (osd.1) 132 : cluster [DBG] 2.4 scrub ok 2024-02-23T00:45:22.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:19.710140+0000 osd.6 (osd.6) 101 : cluster [DBG] 15.9 scrub starts 2024-02-23T00:45:22.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:19.711111+0000 osd.6 (osd.6) 102 : cluster [DBG] 15.9 scrub ok 2024-02-23T00:45:22.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:20.229127+0000 mon.c (mon.2) 166 : audit [DBG] from='client.? 172.21.15.47:0/4147048992' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:45:22.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:20.572078+0000 osd.0 (osd.0) 113 : cluster [DBG] 2.1f scrub starts 2024-02-23T00:45:22.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:20.573154+0000 osd.0 (osd.0) 114 : cluster [DBG] 2.1f scrub ok 2024-02-23T00:45:22.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:20.820405+0000 mon.b (mon.1) 308 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:22.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:21.123654+0000 mon.a (mon.0) 1706 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:45:22.693 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:45:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:45:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:45:22.693 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:45:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:45:22.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:20.760367+0000 osd.6 (osd.6) 103 : cluster [DBG] 5.11 scrub starts 2024-02-23T00:45:22.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:20.761435+0000 osd.6 (osd.6) 104 : cluster [DBG] 5.11 scrub ok 2024-02-23T00:45:22.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:20.827594+0000 osd.7 (osd.7) 131 : cluster [DBG] 18.1f scrub starts 2024-02-23T00:45:22.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:20.829047+0000 osd.7 (osd.7) 132 : cluster [DBG] 18.1f scrub ok 2024-02-23T00:45:22.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:20.963617+0000 osd.3 (osd.3) 135 : cluster [DBG] 4.10 scrub starts 2024-02-23T00:45:22.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:20.967085+0000 osd.3 (osd.3) 136 : cluster [DBG] 4.10 scrub ok 2024-02-23T00:45:22.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:21.239136+0000 mgr.y (mgr.14517) 268 : cluster [DBG] pgmap v349: 282 pgs: 6 creating+activating, 1 active, 1 active+clean+scrubbing, 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 267 active+clean; 9.0 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:45:22.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:21.382736+0000 osd.2 (osd.2) 111 : cluster [DBG] 15.e scrub starts 2024-02-23T00:45:22.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:21.383618+0000 osd.2 (osd.2) 112 : cluster [DBG] 15.e scrub ok 2024-02-23T00:45:22.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:21.563793+0000 mon.a (mon.0) 1707 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-25", "mode": "writeback"}]': finished 2024-02-23T00:45:22.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:21.563875+0000 mon.a (mon.0) 1708 : cluster [DBG] osdmap e249: 8 total, 8 up, 8 in 2024-02-23T00:45:22.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:21.613371+0000 mon.a (mon.0) 1709 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:45:22.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:21.821979+0000 mon.b (mon.1) 309 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:22.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:22.133477+0000 mon.a (mon.0) 1710 : audit [INF] from='client.? 172.21.15.47:0/290258726' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi047-82378-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi047-82378-34"}]': finished 2024-02-23T00:45:22.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:22.133613+0000 mon.a (mon.0) 1711 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:45:22.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:22.133674+0000 mon.a (mon.0) 1712 : cluster [DBG] osdmap e250: 8 total, 8 up, 8 in 2024-02-23T00:45:22.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:22.134545+0000 mon.a (mon.0) 1713 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-25"}]: dispatch 2024-02-23T00:45:22.892 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:45:22.893 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:45:23.007 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:20.760367+0000 osd.6 (osd.6) 103 : cluster [DBG] 5.11 scrub starts 2024-02-23T00:45:23.007 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:20.761435+0000 osd.6 (osd.6) 104 : cluster [DBG] 5.11 scrub ok 2024-02-23T00:45:23.007 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:20.827594+0000 osd.7 (osd.7) 131 : cluster [DBG] 18.1f scrub starts 2024-02-23T00:45:23.007 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:20.829047+0000 osd.7 (osd.7) 132 : cluster [DBG] 18.1f scrub ok 2024-02-23T00:45:23.007 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:20.963617+0000 osd.3 (osd.3) 135 : cluster [DBG] 4.10 scrub starts 2024-02-23T00:45:23.007 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:20.967085+0000 osd.3 (osd.3) 136 : cluster [DBG] 4.10 scrub ok 2024-02-23T00:45:23.007 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:21.239136+0000 mgr.y (mgr.14517) 268 : cluster [DBG] pgmap v349: 282 pgs: 6 creating+activating, 1 active, 1 active+clean+scrubbing, 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 267 active+clean; 9.0 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:45:23.007 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:21.382736+0000 osd.2 (osd.2) 111 : cluster [DBG] 15.e scrub starts 2024-02-23T00:45:23.007 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:21.383618+0000 osd.2 (osd.2) 112 : cluster [DBG] 15.e scrub ok 2024-02-23T00:45:23.008 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:21.563793+0000 mon.a (mon.0) 1707 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-25", "mode": "writeback"}]': finished 2024-02-23T00:45:23.008 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:21.563875+0000 mon.a (mon.0) 1708 : cluster [DBG] osdmap e249: 8 total, 8 up, 8 in 2024-02-23T00:45:23.008 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:21.613371+0000 mon.a (mon.0) 1709 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:45:23.008 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:21.821979+0000 mon.b (mon.1) 309 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:23.008 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:22.133477+0000 mon.a (mon.0) 1710 : audit [INF] from='client.? 172.21.15.47:0/290258726' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi047-82378-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi047-82378-34"}]': finished 2024-02-23T00:45:23.008 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:22.133613+0000 mon.a (mon.0) 1711 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:45:23.008 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:22.133674+0000 mon.a (mon.0) 1712 : cluster [DBG] osdmap e250: 8 total, 8 up, 8 in 2024-02-23T00:45:23.008 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:22.134545+0000 mon.a (mon.0) 1713 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-25"}]: dispatch 2024-02-23T00:45:23.008 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:20.760367+0000 osd.6 (osd.6) 103 : cluster [DBG] 5.11 scrub starts 2024-02-23T00:45:23.008 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:20.761435+0000 osd.6 (osd.6) 104 : cluster [DBG] 5.11 scrub ok 2024-02-23T00:45:23.008 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:20.827594+0000 osd.7 (osd.7) 131 : cluster [DBG] 18.1f scrub starts 2024-02-23T00:45:23.009 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:20.829047+0000 osd.7 (osd.7) 132 : cluster [DBG] 18.1f scrub ok 2024-02-23T00:45:23.009 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:20.963617+0000 osd.3 (osd.3) 135 : cluster [DBG] 4.10 scrub starts 2024-02-23T00:45:23.009 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:20.967085+0000 osd.3 (osd.3) 136 : cluster [DBG] 4.10 scrub ok 2024-02-23T00:45:23.009 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:21.239136+0000 mgr.y (mgr.14517) 268 : cluster [DBG] pgmap v349: 282 pgs: 6 creating+activating, 1 active, 1 active+clean+scrubbing, 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 267 active+clean; 9.0 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:45:23.009 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:21.382736+0000 osd.2 (osd.2) 111 : cluster [DBG] 15.e scrub starts 2024-02-23T00:45:23.009 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:21.383618+0000 osd.2 (osd.2) 112 : cluster [DBG] 15.e scrub ok 2024-02-23T00:45:23.009 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:21.563793+0000 mon.a (mon.0) 1707 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-25", "mode": "writeback"}]': finished 2024-02-23T00:45:23.009 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:21.563875+0000 mon.a (mon.0) 1708 : cluster [DBG] osdmap e249: 8 total, 8 up, 8 in 2024-02-23T00:45:23.009 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:21.613371+0000 mon.a (mon.0) 1709 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:45:23.009 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:21.821979+0000 mon.b (mon.1) 309 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:23.009 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:22.133477+0000 mon.a (mon.0) 1710 : audit [INF] from='client.? 172.21.15.47:0/290258726' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi047-82378-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi047-82378-34"}]': finished 2024-02-23T00:45:23.009 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:22.133613+0000 mon.a (mon.0) 1711 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:45:23.009 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:22.133674+0000 mon.a (mon.0) 1712 : cluster [DBG] osdmap e250: 8 total, 8 up, 8 in 2024-02-23T00:45:23.009 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:22.134545+0000 mon.a (mon.0) 1713 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-25"}]: dispatch 2024-02-23T00:45:23.438 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:45:23.441 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:23.441 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:45:23.446 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.1 dump_historic_ops 2024-02-23T00:45:23.542 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:45:23.574 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:23.595 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:45:23.598 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:23.598 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:45:23.699 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:45:23.724 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:21.716431+0000 osd.1 (osd.1) 133 : cluster [DBG] 15.13 scrub starts 2024-02-23T00:45:23.725 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:21.719896+0000 osd.1 (osd.1) 134 : cluster [DBG] 15.13 scrub ok 2024-02-23T00:45:23.725 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:21.916098+0000 osd.3 (osd.3) 137 : cluster [DBG] 18.a scrub starts 2024-02-23T00:45:23.725 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:21.916992+0000 osd.3 (osd.3) 138 : cluster [DBG] 18.a scrub ok 2024-02-23T00:45:23.725 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:22.271096+0000 osd.5 (osd.5) 117 : cluster [DBG] 2.3 scrub starts 2024-02-23T00:45:23.725 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:22.271999+0000 osd.5 (osd.5) 118 : cluster [DBG] 2.3 scrub ok 2024-02-23T00:45:23.725 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:22.375296+0000 osd.2 (osd.2) 113 : cluster [DBG] 2.10 scrub starts 2024-02-23T00:45:23.725 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:22.376059+0000 osd.2 (osd.2) 114 : cluster [DBG] 2.10 scrub ok 2024-02-23T00:45:23.725 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:22.440621+0000 mgr.y (mgr.14517) 269 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:45:23.725 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:22.595969+0000 osd.0 (osd.0) 115 : cluster [DBG] 18.1d scrub starts 2024-02-23T00:45:23.725 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:22.597427+0000 osd.0 (osd.0) 116 : cluster [DBG] 18.1d scrub ok 2024-02-23T00:45:23.725 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:22.823297+0000 mon.b (mon.1) 310 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:23.725 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:22.873778+0000 osd.3 (osd.3) 139 : cluster [DBG] 18.1a scrub starts 2024-02-23T00:45:23.726 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:22.874692+0000 osd.3 (osd.3) 140 : cluster [DBG] 18.1a scrub ok 2024-02-23T00:45:23.726 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:23.130618+0000 mon.a (mon.0) 1714 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:45:23.726 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:23.135256+0000 mon.a (mon.0) 1715 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-25"}]': finished 2024-02-23T00:45:23.726 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:23.135403+0000 mon.a (mon.0) 1716 : cluster [DBG] osdmap e251: 8 total, 8 up, 8 in 2024-02-23T00:45:23.726 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:23.137196+0000 mon.a (mon.0) 1717 : audit [INF] from='client.? 172.21.15.47:0/954374882' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi047-82386-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:23.726 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:23.377037+0000 osd.2 (osd.2) 115 : cluster [DBG] 18.7 scrub starts 2024-02-23T00:45:23.726 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:23.378266+0000 osd.2 (osd.2) 116 : cluster [DBG] 18.7 scrub ok 2024-02-23T00:45:23.726 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:21.716431+0000 osd.1 (osd.1) 133 : cluster [DBG] 15.13 scrub starts 2024-02-23T00:45:23.726 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:21.719896+0000 osd.1 (osd.1) 134 : cluster [DBG] 15.13 scrub ok 2024-02-23T00:45:23.726 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:21.916098+0000 osd.3 (osd.3) 137 : cluster [DBG] 18.a scrub starts 2024-02-23T00:45:23.726 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:21.916992+0000 osd.3 (osd.3) 138 : cluster [DBG] 18.a scrub ok 2024-02-23T00:45:23.726 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:22.271096+0000 osd.5 (osd.5) 117 : cluster [DBG] 2.3 scrub starts 2024-02-23T00:45:23.726 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:22.271999+0000 osd.5 (osd.5) 118 : cluster [DBG] 2.3 scrub ok 2024-02-23T00:45:23.727 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:22.375296+0000 osd.2 (osd.2) 113 : cluster [DBG] 2.10 scrub starts 2024-02-23T00:45:23.727 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:22.376059+0000 osd.2 (osd.2) 114 : cluster [DBG] 2.10 scrub ok 2024-02-23T00:45:23.727 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:22.440621+0000 mgr.y (mgr.14517) 269 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:45:23.727 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:22.595969+0000 osd.0 (osd.0) 115 : cluster [DBG] 18.1d scrub starts 2024-02-23T00:45:23.727 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:22.597427+0000 osd.0 (osd.0) 116 : cluster [DBG] 18.1d scrub ok 2024-02-23T00:45:23.727 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:22.823297+0000 mon.b (mon.1) 310 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:23.727 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:22.873778+0000 osd.3 (osd.3) 139 : cluster [DBG] 18.1a scrub starts 2024-02-23T00:45:23.727 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:22.874692+0000 osd.3 (osd.3) 140 : cluster [DBG] 18.1a scrub ok 2024-02-23T00:45:23.727 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:23.130618+0000 mon.a (mon.0) 1714 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:45:23.727 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:23.135256+0000 mon.a (mon.0) 1715 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-25"}]': finished 2024-02-23T00:45:23.727 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:23.135403+0000 mon.a (mon.0) 1716 : cluster [DBG] osdmap e251: 8 total, 8 up, 8 in 2024-02-23T00:45:23.727 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:23.137196+0000 mon.a (mon.0) 1717 : audit [INF] from='client.? 172.21.15.47:0/954374882' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi047-82386-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:23.727 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:23.377037+0000 osd.2 (osd.2) 115 : cluster [DBG] 18.7 scrub starts 2024-02-23T00:45:23.727 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:23.378266+0000 osd.2 (osd.2) 116 : cluster [DBG] 18.7 scrub ok 2024-02-23T00:45:23.730 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:23.751 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:45:23.752 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:23.753 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:45:23.853 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:45:23.883 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:21.716431+0000 osd.1 (osd.1) 133 : cluster [DBG] 15.13 scrub starts 2024-02-23T00:45:23.883 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:21.719896+0000 osd.1 (osd.1) 134 : cluster [DBG] 15.13 scrub ok 2024-02-23T00:45:23.883 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:21.916098+0000 osd.3 (osd.3) 137 : cluster [DBG] 18.a scrub starts 2024-02-23T00:45:23.884 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:21.916992+0000 osd.3 (osd.3) 138 : cluster [DBG] 18.a scrub ok 2024-02-23T00:45:23.884 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:22.271096+0000 osd.5 (osd.5) 117 : cluster [DBG] 2.3 scrub starts 2024-02-23T00:45:23.884 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:22.271999+0000 osd.5 (osd.5) 118 : cluster [DBG] 2.3 scrub ok 2024-02-23T00:45:23.884 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:22.375296+0000 osd.2 (osd.2) 113 : cluster [DBG] 2.10 scrub starts 2024-02-23T00:45:23.884 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:22.376059+0000 osd.2 (osd.2) 114 : cluster [DBG] 2.10 scrub ok 2024-02-23T00:45:23.884 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:22.440621+0000 mgr.y (mgr.14517) 269 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:45:23.884 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:22.595969+0000 osd.0 (osd.0) 115 : cluster [DBG] 18.1d scrub starts 2024-02-23T00:45:23.884 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:22.597427+0000 osd.0 (osd.0) 116 : cluster [DBG] 18.1d scrub ok 2024-02-23T00:45:23.884 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:22.823297+0000 mon.b (mon.1) 310 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:23.884 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:22.873778+0000 osd.3 (osd.3) 139 : cluster [DBG] 18.1a scrub starts 2024-02-23T00:45:23.884 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:22.874692+0000 osd.3 (osd.3) 140 : cluster [DBG] 18.1a scrub ok 2024-02-23T00:45:23.884 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:23.130618+0000 mon.a (mon.0) 1714 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:45:23.884 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:23.135256+0000 mon.a (mon.0) 1715 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-25"}]': finished 2024-02-23T00:45:23.885 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:23.135403+0000 mon.a (mon.0) 1716 : cluster [DBG] osdmap e251: 8 total, 8 up, 8 in 2024-02-23T00:45:23.885 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:23.137196+0000 mon.a (mon.0) 1717 : audit [INF] from='client.? 172.21.15.47:0/954374882' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi047-82386-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:23.885 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:23.377037+0000 osd.2 (osd.2) 115 : cluster [DBG] 18.7 scrub starts 2024-02-23T00:45:23.885 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:23.378266+0000 osd.2 (osd.2) 116 : cluster [DBG] 18.7 scrub ok 2024-02-23T00:45:23.885 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:23.907 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:45:23.910 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:23.910 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:45:24.011 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:45:24.040 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:24.078 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:45:24.081 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:24.082 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:45:24.182 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:45:24.211 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:24.231 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:45:24.234 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:24.234 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:45:24.335 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:45:24.366 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:24.516 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:45:24.518 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:24.519 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:45:24.620 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:45:24.646 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:23.223602+0000 osd.4 (osd.4) 117 : cluster [DBG] 18.d deep-scrub starts 2024-02-23T00:45:24.647 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:23.224655+0000 osd.4 (osd.4) 118 : cluster [DBG] 18.d deep-scrub ok 2024-02-23T00:45:24.647 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:23.239539+0000 mgr.y (mgr.14517) 270 : cluster [DBG] pgmap v352: 290 pgs: 3 creating+peering, 37 unknown, 1 active+clean+snaptrim, 249 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-23T00:45:24.647 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:23.741534+0000 osd.1 (osd.1) 135 : cluster [DBG] 18.1b scrub starts 2024-02-23T00:45:24.647 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:23.744837+0000 osd.1 (osd.1) 136 : cluster [DBG] 18.1b scrub ok 2024-02-23T00:45:24.647 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:23.824926+0000 mon.b (mon.1) 311 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:24.647 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:23.830777+0000 osd.6 (osd.6) 105 : cluster [DBG] 15.1 scrub starts 2024-02-23T00:45:24.647 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:23.832285+0000 osd.6 (osd.6) 106 : cluster [DBG] 15.1 scrub ok 2024-02-23T00:45:24.647 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:23.902742+0000 osd.3 (osd.3) 141 : cluster [DBG] 4.0 deep-scrub starts 2024-02-23T00:45:24.647 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:23.910542+0000 osd.3 (osd.3) 142 : cluster [DBG] 4.0 deep-scrub ok 2024-02-23T00:45:24.647 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:24.140407+0000 mon.a (mon.0) 1718 : audit [INF] from='client.? 172.21.15.47:0/954374882' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi047-82386-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:24.647 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:24.140636+0000 mon.a (mon.0) 1719 : cluster [DBG] osdmap e252: 8 total, 8 up, 8 in 2024-02-23T00:45:24.647 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:24.375985+0000 osd.2 (osd.2) 117 : cluster [DBG] 15.a scrub starts 2024-02-23T00:45:24.647 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:24.377319+0000 osd.2 (osd.2) 118 : cluster [DBG] 15.a scrub ok 2024-02-23T00:45:24.648 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:24.590685+0000 mon.a (mon.0) 1720 : audit [INF] from='client.? 172.21.15.47:0/290258726' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi047-82378-34"}]: dispatch 2024-02-23T00:45:24.648 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:23.223602+0000 osd.4 (osd.4) 117 : cluster [DBG] 18.d deep-scrub starts 2024-02-23T00:45:24.648 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:23.224655+0000 osd.4 (osd.4) 118 : cluster [DBG] 18.d deep-scrub ok 2024-02-23T00:45:24.648 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:23.239539+0000 mgr.y (mgr.14517) 270 : cluster [DBG] pgmap v352: 290 pgs: 3 creating+peering, 37 unknown, 1 active+clean+snaptrim, 249 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-23T00:45:24.648 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:23.741534+0000 osd.1 (osd.1) 135 : cluster [DBG] 18.1b scrub starts 2024-02-23T00:45:24.648 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:23.744837+0000 osd.1 (osd.1) 136 : cluster [DBG] 18.1b scrub ok 2024-02-23T00:45:24.648 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:23.824926+0000 mon.b (mon.1) 311 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:24.648 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:23.830777+0000 osd.6 (osd.6) 105 : cluster [DBG] 15.1 scrub starts 2024-02-23T00:45:24.648 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:23.832285+0000 osd.6 (osd.6) 106 : cluster [DBG] 15.1 scrub ok 2024-02-23T00:45:24.648 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:23.902742+0000 osd.3 (osd.3) 141 : cluster [DBG] 4.0 deep-scrub starts 2024-02-23T00:45:24.649 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:23.910542+0000 osd.3 (osd.3) 142 : cluster [DBG] 4.0 deep-scrub ok 2024-02-23T00:45:24.649 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:24.140407+0000 mon.a (mon.0) 1718 : audit [INF] from='client.? 172.21.15.47:0/954374882' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi047-82386-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:24.649 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:24.140636+0000 mon.a (mon.0) 1719 : cluster [DBG] osdmap e252: 8 total, 8 up, 8 in 2024-02-23T00:45:24.649 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:24.375985+0000 osd.2 (osd.2) 117 : cluster [DBG] 15.a scrub starts 2024-02-23T00:45:24.649 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:24.377319+0000 osd.2 (osd.2) 118 : cluster [DBG] 15.a scrub ok 2024-02-23T00:45:24.649 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:24.590685+0000 mon.a (mon.0) 1720 : audit [INF] from='client.? 172.21.15.47:0/290258726' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi047-82378-34"}]: dispatch 2024-02-23T00:45:24.649 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:24.666 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:45:24.905 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:45:24.908 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:24.909 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:45:24.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:23.223602+0000 osd.4 (osd.4) 117 : cluster [DBG] 18.d deep-scrub starts 2024-02-23T00:45:24.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:23.224655+0000 osd.4 (osd.4) 118 : cluster [DBG] 18.d deep-scrub ok 2024-02-23T00:45:24.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:23.239539+0000 mgr.y (mgr.14517) 270 : cluster [DBG] pgmap v352: 290 pgs: 3 creating+peering, 37 unknown, 1 active+clean+snaptrim, 249 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-23T00:45:24.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:23.741534+0000 osd.1 (osd.1) 135 : cluster [DBG] 18.1b scrub starts 2024-02-23T00:45:24.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:23.744837+0000 osd.1 (osd.1) 136 : cluster [DBG] 18.1b scrub ok 2024-02-23T00:45:24.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:23.824926+0000 mon.b (mon.1) 311 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:24.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:23.830777+0000 osd.6 (osd.6) 105 : cluster [DBG] 15.1 scrub starts 2024-02-23T00:45:24.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:23.832285+0000 osd.6 (osd.6) 106 : cluster [DBG] 15.1 scrub ok 2024-02-23T00:45:24.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:23.902742+0000 osd.3 (osd.3) 141 : cluster [DBG] 4.0 deep-scrub starts 2024-02-23T00:45:24.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:23.910542+0000 osd.3 (osd.3) 142 : cluster [DBG] 4.0 deep-scrub ok 2024-02-23T00:45:24.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:24.140407+0000 mon.a (mon.0) 1718 : audit [INF] from='client.? 172.21.15.47:0/954374882' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi047-82386-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:24.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:24.140636+0000 mon.a (mon.0) 1719 : cluster [DBG] osdmap e252: 8 total, 8 up, 8 in 2024-02-23T00:45:24.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:24.375985+0000 osd.2 (osd.2) 117 : cluster [DBG] 15.a scrub starts 2024-02-23T00:45:24.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:24.377319+0000 osd.2 (osd.2) 118 : cluster [DBG] 15.a scrub ok 2024-02-23T00:45:24.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:24.590685+0000 mon.a (mon.0) 1720 : audit [INF] from='client.? 172.21.15.47:0/290258726' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi047-82378-34"}]: dispatch 2024-02-23T00:45:25.010 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:45:25.037 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:25.074 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:45:25.077 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:25.078 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:45:25.179 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:45:25.207 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:25.464 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:45:25.466 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:25.467 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:45:25.568 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:45:25.600 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:25.622 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:45:25.624 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:25.625 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:45:25.725 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:45:25.752 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:24.825997+0000 mon.b (mon.1) 312 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:25.752 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:25.127780+0000 mon.a (mon.0) 1721 : audit [INF] from='client.? 172.21.15.47:0/290258726' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi047-82378-34"}]': finished 2024-02-23T00:45:25.753 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:25.127881+0000 mon.a (mon.0) 1722 : cluster [DBG] osdmap e253: 8 total, 8 up, 8 in 2024-02-23T00:45:25.753 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:25.130742+0000 mon.a (mon.0) 1723 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:25.753 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:25.142496+0000 mon.a (mon.0) 1724 : audit [INF] from='client.? 172.21.15.47:0/290258726' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi047-82378-34"}]: dispatch 2024-02-23T00:45:25.753 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:24.825997+0000 mon.b (mon.1) 312 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:25.753 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:25.127780+0000 mon.a (mon.0) 1721 : audit [INF] from='client.? 172.21.15.47:0/290258726' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi047-82378-34"}]': finished 2024-02-23T00:45:25.754 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:25.127881+0000 mon.a (mon.0) 1722 : cluster [DBG] osdmap e253: 8 total, 8 up, 8 in 2024-02-23T00:45:25.754 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:25.130742+0000 mon.a (mon.0) 1723 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:25.754 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:25.142496+0000 mon.a (mon.0) 1724 : audit [INF] from='client.? 172.21.15.47:0/290258726' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi047-82378-34"}]: dispatch 2024-02-23T00:45:25.754 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:25.922 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:45:25.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:24.825997+0000 mon.b (mon.1) 312 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:25.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:25.127780+0000 mon.a (mon.0) 1721 : audit [INF] from='client.? 172.21.15.47:0/290258726' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi047-82378-34"}]': finished 2024-02-23T00:45:25.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:25.127881+0000 mon.a (mon.0) 1722 : cluster [DBG] osdmap e253: 8 total, 8 up, 8 in 2024-02-23T00:45:25.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:25.130742+0000 mon.a (mon.0) 1723 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:25.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:25.142496+0000 mon.a (mon.0) 1724 : audit [INF] from='client.? 172.21.15.47:0/290258726' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi047-82378-34"}]: dispatch 2024-02-23T00:45:26.205 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:45:26.208 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:26.208 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:45:26.309 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:45:26.341 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:26.363 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:45:26.366 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:26.366 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:45:26.467 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:45:26.496 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:26.515 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:45:26.518 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:26.518 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:45:26.619 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:45:26.649 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:26.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.688+0000 7f9896e72700 1 -- 172.21.15.47:0/4183622012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9890102210 msgr2=0x7f9890104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:26.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.688+0000 7f9896e72700 1 --2- 172.21.15.47:0/4183622012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9890102210 0x7f9890104600 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 rx=0x7f9888009be0 tx=0x7f9888004d10).stop 2024-02-23T00:45:26.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.688+0000 7f9896e72700 1 -- 172.21.15.47:0/4183622012 shutdown_connections 2024-02-23T00:45:26.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.688+0000 7f9896e72700 1 --2- 172.21.15.47:0/4183622012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9890102210 0x7f9890104600 unknown :-1 s=CLOSED pgs=2479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:26.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.688+0000 7f9896e72700 1 --2- 172.21.15.47:0/4183622012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98900f8cb0 0x7f98900f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:26.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.688+0000 7f9896e72700 1 --2- 172.21.15.47:0/4183622012 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f98900f9610 0x7f9890101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:26.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.688+0000 7f9896e72700 1 -- 172.21.15.47:0/4183622012 >> 172.21.15.47:0/4183622012 conn(0x7f98900f2820 msgr2=0x7f98900f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:45:26.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.688+0000 7f9896e72700 1 -- 172.21.15.47:0/4183622012 shutdown_connections 2024-02-23T00:45:26.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.688+0000 7f9896e72700 1 -- 172.21.15.47:0/4183622012 wait complete. 2024-02-23T00:45:26.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.688+0000 7f9896e72700 1 librados: starting msgr at 2024-02-23T00:45:26.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.688+0000 7f9896e72700 1 librados: starting objecter 2024-02-23T00:45:26.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.689+0000 7f9896e72700 1 Processor -- start 2024-02-23T00:45:26.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.689+0000 7f9896e72700 1 -- start start 2024-02-23T00:45:26.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.689+0000 7f9896e72700 1 librados: setting wanted keys 2024-02-23T00:45:26.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.689+0000 7f9896e72700 1 librados: calling monclient init 2024-02-23T00:45:26.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.689+0000 7f9896e72700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98900f8cb0 0x7f989010de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:26.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.689+0000 7f9896e72700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98900f9610 0x7f989010e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:26.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.689+0000 7f9896e72700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9890102210 0x7f9890107fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:26.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.689+0000 7f9896e72700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98901084a0 con 0x7f98900f8cb0 2024-02-23T00:45:26.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.689+0000 7f9896e72700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9890108610 con 0x7f98900f9610 2024-02-23T00:45:26.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.689+0000 7f9896e72700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98901087d0 con 0x7f9890102210 2024-02-23T00:45:26.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.689+0000 7f9894c0e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98900f8cb0 0x7f989010de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:26.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.689+0000 7f989540f700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9890102210 0x7f9890107fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:26.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.689+0000 7f9894c0e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98900f8cb0 0x7f989010de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52032/0 (socket says 172.21.15.47:52032) 2024-02-23T00:45:26.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.689+0000 7f989540f700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9890102210 0x7f9890107fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:48358/0 (socket says 172.21.15.47:48358) 2024-02-23T00:45:26.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.689+0000 7f9894c0e700 1 -- 172.21.15.47:0/2827751117 learned_addr learned my addr 172.21.15.47:0/2827751117 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:45:26.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.689+0000 7f988ffff700 1 --2- 172.21.15.47:0/2827751117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98900f9610 0x7f989010e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:26.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.689+0000 7f9894c0e700 1 -- 172.21.15.47:0/2827751117 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9890102210 msgr2=0x7f9890107fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:26.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.689+0000 7f9894c0e700 1 --2- 172.21.15.47:0/2827751117 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9890102210 0x7f9890107fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:26.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.689+0000 7f9894c0e700 1 -- 172.21.15.47:0/2827751117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98900f9610 msgr2=0x7f989010e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:26.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.689+0000 7f9894c0e700 1 --2- 172.21.15.47:0/2827751117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98900f9610 0x7f989010e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:26.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.690+0000 7f9894c0e700 1 -- 172.21.15.47:0/2827751117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f987c005040 con 0x7f98900f8cb0 2024-02-23T00:45:26.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.690+0000 7f989540f700 1 --2- 172.21.15.47:0/2827751117 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9890102210 0x7f9890107fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:45:26.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.690+0000 7f9894c0e700 1 --2- 172.21.15.47:0/2827751117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98900f8cb0 0x7f989010de10 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 rx=0x7f98840041d0 tx=0x7f9884009f60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:26.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.690+0000 7f988dffb700 10 client.?.objecter ms_handle_connect 0x7f98900f8cb0 2024-02-23T00:45:26.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.690+0000 7f988dffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:45:26.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.690+0000 7f988dffb700 1 -- 172.21.15.47:0/2827751117 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9884007b40 con 0x7f98900f8cb0 2024-02-23T00:45:26.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.690+0000 7f988dffb700 1 -- 172.21.15.47:0/2827751117 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f988400f040 con 0x7f98900f8cb0 2024-02-23T00:45:26.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.690+0000 7f988dffb700 1 -- 172.21.15.47:0/2827751117 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9884014730 con 0x7f98900f8cb0 2024-02-23T00:45:26.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.690+0000 7f9896e72700 1 -- 172.21.15.47:0/2827751117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98880097c0 con 0x7f98900f8cb0 2024-02-23T00:45:26.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.690+0000 7f9896e72700 10 client.30561.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:45:26.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.690+0000 7f9896e72700 1 -- 172.21.15.47:0/2827751117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9890108e10 con 0x7f98900f8cb0 2024-02-23T00:45:26.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.690+0000 7f9896e72700 1 librados: init done 2024-02-23T00:45:26.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.691+0000 7f9896e72700 1 -- 172.21.15.47:0/2827751117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f989005fa60 con 0x7f98900f8cb0 2024-02-23T00:45:26.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.691+0000 7f988dffb700 1 -- 172.21.15.47:0/2827751117 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f9884009110 con 0x7f98900f8cb0 2024-02-23T00:45:26.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.692+0000 7f988dffb700 1 --2- 172.21.15.47:0/2827751117 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9880062cc0 0x7f9880065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:26.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.692+0000 7f988dffb700 1 -- 172.21.15.47:0/2827751117 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(254..254 src has 48..254) v4 ==== 9901+0+0 (secure 0 0 0) 0x7f988407d8e0 con 0x7f98900f8cb0 2024-02-23T00:45:26.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.692+0000 7f988dffb700 10 client.30561.objecter ms_dispatch 0x7f9890001550 osd_map(254..254 src has 48..254) v4 2024-02-23T00:45:26.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.692+0000 7f988dffb700 3 client.30561.objecter handle_osd_map got epochs [254,254] > 0 2024-02-23T00:45:26.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.692+0000 7f988dffb700 3 client.30561.objecter handle_osd_map decoding full epoch 254 2024-02-23T00:45:26.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.692+0000 7f988ffff700 1 --2- 172.21.15.47:0/2827751117 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9880062cc0 0x7f9880065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:26.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.692+0000 7f988dffb700 20 client.30561.objecter dump_active .. 0 homeless 2024-02-23T00:45:26.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.692+0000 7f988dffb700 10 client.30561.objecter ms_handle_connect 0x7f9880062cc0 2024-02-23T00:45:26.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.692+0000 7f988ffff700 1 --2- 172.21.15.47:0/2827751117 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9880062cc0 0x7f9880065180 secure :-1 s=READY pgs=2988 cs=0 l=1 rev1=1 rx=0x7f9890109750 tx=0x7f987c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:26.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.697+0000 7f988dffb700 1 -- 172.21.15.47:0/2827751117 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f988404ccd0 con 0x7f98900f8cb0 2024-02-23T00:45:26.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:24.402156+0000 osd.5 (osd.5) 119 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:45:26.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:24.402373+0000 osd.5 (osd.5) 120 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:45:26.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:24.941546+0000 osd.3 (osd.3) 143 : cluster [DBG] 15.12 deep-scrub starts 2024-02-23T00:45:26.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:24.942531+0000 osd.3 (osd.3) 144 : cluster [DBG] 15.12 deep-scrub ok 2024-02-23T00:45:26.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:25.173684+0000 osd.4 (osd.4) 119 : cluster [DBG] 4.c scrub starts 2024-02-23T00:45:26.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:25.175168+0000 osd.4 (osd.4) 120 : cluster [DBG] 4.c scrub ok 2024-02-23T00:45:26.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:25.239950+0000 mgr.y (mgr.14517) 271 : cluster [DBG] pgmap v355: 250 pgs: 32 unknown, 1 active+clean+snaptrim, 217 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:45:26.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:25.349639+0000 osd.2 (osd.2) 119 : cluster [DBG] 62.7 scrub starts 2024-02-23T00:45:26.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:25.351409+0000 osd.2 (osd.2) 120 : cluster [DBG] 62.7 scrub ok 2024-02-23T00:45:26.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:25.827120+0000 mon.b (mon.1) 313 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:26.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:26.130395+0000 mon.a (mon.0) 1725 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:26.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:26.130548+0000 mon.a (mon.0) 1726 : audit [INF] from='client.? 172.21.15.47:0/290258726' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi047-82378-34"}]': finished 2024-02-23T00:45:26.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:26.130629+0000 mon.a (mon.0) 1727 : cluster [DBG] osdmap e254: 8 total, 8 up, 8 in 2024-02-23T00:45:26.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:26.152493+0000 mon.a (mon.0) 1728 : audit [INF] from='client.? 172.21.15.47:0/3327007590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi047-82386-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:26.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:26.165447+0000 mon.c (mon.2) 167 : audit [INF] from='client.? 172.21.15.47:0/538221556' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi047-82378-35"}]: dispatch 2024-02-23T00:45:26.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:26.165728+0000 mon.a (mon.0) 1729 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi047-82378-35"}]: dispatch 2024-02-23T00:45:26.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:26.166349+0000 mon.c (mon.2) 168 : audit [INF] from='client.? 172.21.15.47:0/538221556' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi047-82378-35"}]: dispatch 2024-02-23T00:45:26.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:26.166604+0000 mon.a (mon.0) 1730 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi047-82378-35"}]: dispatch 2024-02-23T00:45:26.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:26.166990+0000 mon.c (mon.2) 169 : audit [INF] from='client.? 172.21.15.47:0/538221556' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi047-82378-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:26.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:26.167247+0000 mon.a (mon.0) 1731 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi047-82378-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:26.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:26.299903+0000 mon.a (mon.0) 1732 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:45:26.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:24.402156+0000 osd.5 (osd.5) 119 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:45:26.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:24.402373+0000 osd.5 (osd.5) 120 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:45:26.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:24.941546+0000 osd.3 (osd.3) 143 : cluster [DBG] 15.12 deep-scrub starts 2024-02-23T00:45:26.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:24.942531+0000 osd.3 (osd.3) 144 : cluster [DBG] 15.12 deep-scrub ok 2024-02-23T00:45:26.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:25.173684+0000 osd.4 (osd.4) 119 : cluster [DBG] 4.c scrub starts 2024-02-23T00:45:26.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:25.175168+0000 osd.4 (osd.4) 120 : cluster [DBG] 4.c scrub ok 2024-02-23T00:45:26.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:25.239950+0000 mgr.y (mgr.14517) 271 : cluster [DBG] pgmap v355: 250 pgs: 32 unknown, 1 active+clean+snaptrim, 217 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:45:26.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:25.349639+0000 osd.2 (osd.2) 119 : cluster [DBG] 62.7 scrub starts 2024-02-23T00:45:26.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:25.351409+0000 osd.2 (osd.2) 120 : cluster [DBG] 62.7 scrub ok 2024-02-23T00:45:26.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:25.827120+0000 mon.b (mon.1) 313 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:26.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:26.130395+0000 mon.a (mon.0) 1725 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:26.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:26.130548+0000 mon.a (mon.0) 1726 : audit [INF] from='client.? 172.21.15.47:0/290258726' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi047-82378-34"}]': finished 2024-02-23T00:45:26.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:26.130629+0000 mon.a (mon.0) 1727 : cluster [DBG] osdmap e254: 8 total, 8 up, 8 in 2024-02-23T00:45:26.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:26.152493+0000 mon.a (mon.0) 1728 : audit [INF] from='client.? 172.21.15.47:0/3327007590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi047-82386-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:26.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:26.165447+0000 mon.c (mon.2) 167 : audit [INF] from='client.? 172.21.15.47:0/538221556' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi047-82378-35"}]: dispatch 2024-02-23T00:45:26.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:26.165728+0000 mon.a (mon.0) 1729 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi047-82378-35"}]: dispatch 2024-02-23T00:45:26.811 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:26.166349+0000 mon.c (mon.2) 168 : audit [INF] from='client.? 172.21.15.47:0/538221556' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi047-82378-35"}]: dispatch 2024-02-23T00:45:26.811 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:26.166604+0000 mon.a (mon.0) 1730 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi047-82378-35"}]: dispatch 2024-02-23T00:45:26.811 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:26.166990+0000 mon.c (mon.2) 169 : audit [INF] from='client.? 172.21.15.47:0/538221556' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi047-82378-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:26.811 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:26.167247+0000 mon.a (mon.0) 1731 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi047-82378-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:26.811 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:26.299903+0000 mon.a (mon.0) 1732 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:45:26.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:26.882+0000 7f9896e72700 1 -- 172.21.15.47:0/2827751117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7f989005ee80 con 0x7f98900f8cb0 2024-02-23T00:45:26.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:24.402156+0000 osd.5 (osd.5) 119 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:45:26.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:24.402373+0000 osd.5 (osd.5) 120 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:45:26.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:24.941546+0000 osd.3 (osd.3) 143 : cluster [DBG] 15.12 deep-scrub starts 2024-02-23T00:45:26.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:24.942531+0000 osd.3 (osd.3) 144 : cluster [DBG] 15.12 deep-scrub ok 2024-02-23T00:45:26.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:25.173684+0000 osd.4 (osd.4) 119 : cluster [DBG] 4.c scrub starts 2024-02-23T00:45:26.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:25.175168+0000 osd.4 (osd.4) 120 : cluster [DBG] 4.c scrub ok 2024-02-23T00:45:26.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:25.239950+0000 mgr.y (mgr.14517) 271 : cluster [DBG] pgmap v355: 250 pgs: 32 unknown, 1 active+clean+snaptrim, 217 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:45:26.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:25.349639+0000 osd.2 (osd.2) 119 : cluster [DBG] 62.7 scrub starts 2024-02-23T00:45:26.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:25.351409+0000 osd.2 (osd.2) 120 : cluster [DBG] 62.7 scrub ok 2024-02-23T00:45:26.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:25.827120+0000 mon.b (mon.1) 313 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:26.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:26.130395+0000 mon.a (mon.0) 1725 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:26.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:26.130548+0000 mon.a (mon.0) 1726 : audit [INF] from='client.? 172.21.15.47:0/290258726' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi047-82378-34"}]': finished 2024-02-23T00:45:26.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:26.130629+0000 mon.a (mon.0) 1727 : cluster [DBG] osdmap e254: 8 total, 8 up, 8 in 2024-02-23T00:45:26.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:26.152493+0000 mon.a (mon.0) 1728 : audit [INF] from='client.? 172.21.15.47:0/3327007590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi047-82386-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:26.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:26.165447+0000 mon.c (mon.2) 167 : audit [INF] from='client.? 172.21.15.47:0/538221556' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi047-82378-35"}]: dispatch 2024-02-23T00:45:26.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:26.165728+0000 mon.a (mon.0) 1729 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi047-82378-35"}]: dispatch 2024-02-23T00:45:26.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:26.166349+0000 mon.c (mon.2) 168 : audit [INF] from='client.? 172.21.15.47:0/538221556' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi047-82378-35"}]: dispatch 2024-02-23T00:45:26.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:26.166604+0000 mon.a (mon.0) 1730 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi047-82378-35"}]: dispatch 2024-02-23T00:45:26.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:26.166990+0000 mon.c (mon.2) 169 : audit [INF] from='client.? 172.21.15.47:0/538221556' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi047-82378-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:26.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:26.167247+0000 mon.a (mon.0) 1731 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi047-82378-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:26.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:26.299903+0000 mon.a (mon.0) 1732 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:45:27.102 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:45:27.105 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:27.105 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:45:27.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.132+0000 7f988dffb700 1 -- 172.21.15.47:0/2827751117 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v255) v1 ==== 87+0+0 (secure 0 0 0) 0x7f989005ee80 con 0x7f98900f8cb0 2024-02-23T00:45:27.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.135+0000 7f9896e72700 10 librados: watch_flush enter 2024-02-23T00:45:27.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.136+0000 7f9896e72700 10 librados: watch_flush exit 2024-02-23T00:45:27.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.136+0000 7f9896e72700 20 client.30561.objecter shutdown clearing up homeless session... 2024-02-23T00:45:27.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.136+0000 7f9896e72700 10 client.30561.objecter successfully canceled tick 2024-02-23T00:45:27.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.136+0000 7f9896e72700 1 -- 172.21.15.47:0/2827751117 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9880062cc0 msgr2=0x7f9880065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:27.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.136+0000 7f9896e72700 1 --2- 172.21.15.47:0/2827751117 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9880062cc0 0x7f9880065180 secure :-1 s=READY pgs=2988 cs=0 l=1 rev1=1 rx=0x7f9890109750 tx=0x7f987c00b040).stop 2024-02-23T00:45:27.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.136+0000 7f9896e72700 1 -- 172.21.15.47:0/2827751117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98900f8cb0 msgr2=0x7f989010de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:27.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.137+0000 7f9896e72700 1 --2- 172.21.15.47:0/2827751117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98900f8cb0 0x7f989010de10 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 rx=0x7f98840041d0 tx=0x7f9884009f60).stop 2024-02-23T00:45:27.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.139+0000 7f9896e72700 1 -- 172.21.15.47:0/2827751117 shutdown_connections 2024-02-23T00:45:27.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.139+0000 7f9896e72700 1 --2- 172.21.15.47:0/2827751117 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9880062cc0 0x7f9880065180 unknown :-1 s=CLOSED pgs=2988 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:27.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.139+0000 7f9896e72700 1 --2- 172.21.15.47:0/2827751117 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9890102210 0x7f9890107fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:27.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.139+0000 7f9896e72700 1 --2- 172.21.15.47:0/2827751117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98900f8cb0 0x7f989010de10 unknown :-1 s=CLOSED pgs=2480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:27.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.139+0000 7f9896e72700 1 --2- 172.21.15.47:0/2827751117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98900f9610 0x7f989010e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:27.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.139+0000 7f9896e72700 1 -- 172.21.15.47:0/2827751117 >> 172.21.15.47:0/2827751117 conn(0x7f98900f2820 msgr2=0x7f98900f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:45:27.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.139+0000 7f9896e72700 1 -- 172.21.15.47:0/2827751117 shutdown_connections 2024-02-23T00:45:27.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.141+0000 7f9896e72700 1 -- 172.21.15.47:0/2827751117 wait complete. 2024-02-23T00:45:27.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.144+0000 7f9896e72700 1 librados: shutdown 2024-02-23T00:45:27.152 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is set 2024-02-23T00:45:27.206 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:45:27.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.212+0000 7fd64edfb700 1 -- 172.21.15.47:0/3979784674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6480fe510 msgr2=0x7fd6480f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:27.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.212+0000 7fd64edfb700 1 --2- 172.21.15.47:0/3979784674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6480fe510 0x7fd6480f7d40 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 rx=0x7fd634004870 tx=0x7fd634009cb0).stop 2024-02-23T00:45:27.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.212+0000 7fd64edfb700 1 -- 172.21.15.47:0/3979784674 shutdown_connections 2024-02-23T00:45:27.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.212+0000 7fd64edfb700 1 --2- 172.21.15.47:0/3979784674 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd6480f8420 0x7fd648104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:27.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.212+0000 7fd64edfb700 1 --2- 172.21.15.47:0/3979784674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6480fdbb0 0x7fd6480fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:45:27.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.212+0000 7fd64edfb700 1 --2- 172.21.15.47:0/3979784674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6480fe510 0x7fd6480f7d40 unknown :-1 s=CLOSED pgs=2481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:27.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.212+0000 7fd64edfb700 1 -- 172.21.15.47:0/3979784674 >> 172.21.15.47:0/3979784674 conn(0x7fd6480f3800 msgr2=0x7fd6480f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:45:27.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.212+0000 7fd64edfb700 1 -- 172.21.15.47:0/3979784674 shutdown_connections 2024-02-23T00:45:27.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.212+0000 7fd64edfb700 1 -- 172.21.15.47:0/3979784674 wait complete. 2024-02-23T00:45:27.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.213+0000 7fd64edfb700 1 librados: starting msgr at 2024-02-23T00:45:27.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.213+0000 7fd64edfb700 1 librados: starting objecter 2024-02-23T00:45:27.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.213+0000 7fd64edfb700 1 Processor -- start 2024-02-23T00:45:27.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.213+0000 7fd64edfb700 1 -- start start 2024-02-23T00:45:27.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.213+0000 7fd64edfb700 1 librados: setting wanted keys 2024-02-23T00:45:27.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.213+0000 7fd64edfb700 1 librados: calling monclient init 2024-02-23T00:45:27.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.213+0000 7fd64edfb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6480f8420 0x7fd648103910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.213+0000 7fd64edfb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6480fdbb0 0x7fd648103e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.213+0000 7fd64edfb700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd6480fe510 0x7fd648122dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.213+0000 7fd64edfb700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd648104570 con 0x7fd6480f8420 2024-02-23T00:45:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.213+0000 7fd64edfb700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6481232b0 con 0x7fd6480fdbb0 2024-02-23T00:45:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.213+0000 7fd64edfb700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6481233f0 con 0x7fd6480fe510 2024-02-23T00:45:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.213+0000 7fd64cb97700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6480f8420 0x7fd648103910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.213+0000 7fd64cb97700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6480f8420 0x7fd648103910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52046/0 (socket says 172.21.15.47:52046) 2024-02-23T00:45:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.213+0000 7fd64cb97700 1 -- 172.21.15.47:0/758952144 learned_addr learned my addr 172.21.15.47:0/758952144 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:45:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.213+0000 7fd64d398700 1 --2- 172.21.15.47:0/758952144 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd6480fe510 0x7fd648122dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.214+0000 7fd647fff700 1 --2- 172.21.15.47:0/758952144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6480fdbb0 0x7fd648103e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.214+0000 7fd64cb97700 1 -- 172.21.15.47:0/758952144 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd6480fe510 msgr2=0x7fd648122dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.214+0000 7fd64cb97700 1 --2- 172.21.15.47:0/758952144 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd6480fe510 0x7fd648122dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.214+0000 7fd64cb97700 1 -- 172.21.15.47:0/758952144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6480fdbb0 msgr2=0x7fd648103e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.214+0000 7fd64cb97700 1 --2- 172.21.15.47:0/758952144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6480fdbb0 0x7fd648103e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:27.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.214+0000 7fd64cb97700 1 -- 172.21.15.47:0/758952144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd634005040 con 0x7fd6480f8420 2024-02-23T00:45:27.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.214+0000 7fd64d398700 1 --2- 172.21.15.47:0/758952144 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd6480fe510 0x7fd648122dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:45:27.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.214+0000 7fd647fff700 1 --2- 172.21.15.47:0/758952144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6480fdbb0 0x7fd648103e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-23T00:45:27.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.214+0000 7fd64cb97700 1 --2- 172.21.15.47:0/758952144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6480f8420 0x7fd648103910 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 rx=0x7fd63c00b530 tx=0x7fd63c0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:27.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.214+0000 7fd645ffb700 10 client.?.objecter ms_handle_connect 0x7fd6480f8420 2024-02-23T00:45:27.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.214+0000 7fd645ffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:45:27.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.214+0000 7fd645ffb700 1 -- 172.21.15.47:0/758952144 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c002320 con 0x7fd6480f8420 2024-02-23T00:45:27.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.214+0000 7fd64edfb700 1 -- 172.21.15.47:0/758952144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6481236d0 con 0x7fd6480f8420 2024-02-23T00:45:27.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.214+0000 7fd64edfb700 10 client.30567.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:45:27.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.214+0000 7fd64edfb700 1 -- 172.21.15.47:0/758952144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd648123c10 con 0x7fd6480f8420 2024-02-23T00:45:27.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.214+0000 7fd64edfb700 1 librados: init done 2024-02-23T00:45:27.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.215+0000 7fd645ffb700 1 -- 172.21.15.47:0/758952144 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fd63c002910 con 0x7fd6480f8420 2024-02-23T00:45:27.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.215+0000 7fd645ffb700 1 -- 172.21.15.47:0/758952144 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c0128c0 con 0x7fd6480f8420 2024-02-23T00:45:27.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.216+0000 7fd64edfb700 1 -- 172.21.15.47:0/758952144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd62c004f50 con 0x7fd6480f8420 2024-02-23T00:45:27.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.217+0000 7fd645ffb700 1 -- 172.21.15.47:0/758952144 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fd63c002a80 con 0x7fd6480f8420 2024-02-23T00:45:27.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.217+0000 7fd645ffb700 1 --2- 172.21.15.47:0/758952144 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd638062cc0 0x7fd638065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:27.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.217+0000 7fd645ffb700 1 -- 172.21.15.47:0/758952144 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(255..255 src has 48..255) v4 ==== 10200+0+0 (secure 0 0 0) 0x7fd63c07e000 con 0x7fd6480f8420 2024-02-23T00:45:27.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.217+0000 7fd645ffb700 10 client.30567.objecter ms_dispatch 0x7fd648001550 osd_map(255..255 src has 48..255) v4 2024-02-23T00:45:27.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.217+0000 7fd645ffb700 3 client.30567.objecter handle_osd_map got epochs [255,255] > 0 2024-02-23T00:45:27.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.217+0000 7fd645ffb700 3 client.30567.objecter handle_osd_map decoding full epoch 255 2024-02-23T00:45:27.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.218+0000 7fd645ffb700 20 client.30567.objecter dump_active .. 0 homeless 2024-02-23T00:45:27.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.218+0000 7fd647fff700 1 --2- 172.21.15.47:0/758952144 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd638062cc0 0x7fd638065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:27.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.218+0000 7fd645ffb700 10 client.30567.objecter ms_handle_connect 0x7fd638062cc0 2024-02-23T00:45:27.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.218+0000 7fd647fff700 1 --2- 172.21.15.47:0/758952144 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd638062cc0 0x7fd638065180 secure :-1 s=READY pgs=2989 cs=0 l=1 rev1=1 rx=0x7fd634004ee0 tx=0x7fd634009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:27.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.221+0000 7fd645ffb700 1 -- 172.21.15.47:0/758952144 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fd63c04d260 con 0x7fd6480f8420 2024-02-23T00:45:27.240 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:27.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.462+0000 7fd64edfb700 1 -- 172.21.15.47:0/758952144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd62c0054d0 con 0x7fd6480f8420 2024-02-23T00:45:27.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.463+0000 7fd645ffb700 1 -- 172.21.15.47:0/758952144 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+2564 (secure 0 0 0) 0x7fd63c050880 con 0x7fd6480f8420 2024-02-23T00:45:27.466 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:45:27.466 INFO:teuthology.orchestra.run.smithi047.stdout:{"fsid":"fce2617e-d1e2-11ee-95c0-87774f69a715","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noscrub flag(s) set","count":7},"muted":false},"POOL_APP_NOT_ENABLED":{"severity":"HEALTH_WARN","summary":{"message":"1 pool(s) do not have an application enabled","count":1},"muted":false}},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":612,"monmap":{"epoch":3,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":255,"num_osds":8,"num_up_osds":8,"osd_up_since":1708648745,"num_in_osds":8,"osd_in_since":1708648727,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":217},{"state_name":"unknown","count":32},{"state_name":"active+clean+snaptrim","count":1}],"num_pgs":250,"num_pools":10,"num_objects":358,"data_bytes":9047,"bytes_used":2539474944,"bytes_avail":765420810240,"bytes_total":767960285184,"unknown_pgs_ratio":0.12800000607967377,"read_bytes_sec":1279,"read_op_per_sec":1,"write_op_per_sec":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"services":{"dashboard":"https://172.21.15.47:8443/","prometheus":"http://172.21.15.47:9283/"}},"servicemap":{"epoch":23,"modified":"2024-02-23T00:43:27.194339+0000","services":{"rgw":{"daemons":{"summary":"","14460":{"start_epoch":12,"start_stamp":"2024-02-23T00:39:21.905359+0000","gid":14460,"addr":"172.21.15.47:0/1573276039","metadata":{"arch":"x86_64","ceph_release":"pacific","ceph_version":"ceph version 16.2.14-550-geb66ed92 (eb66ed921e744301e6863be6353618d63967ea59) pacific (stable)","ceph_version_short":"16.2.14-550-geb66ed92","container_hostname":"smithi047","container_image":"quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:44277403522cf37edeee63b04643fd6c6e9416f13c3731a771e3cc8e53b7d010","cpu":"Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz","distro":"centos","distro_description":"CentOS Stream 8","distro_version":"8","frontend_config#0":"beast port=80","frontend_type#0":"beast","hostname":"smithi047","id":"foo.a","kernel_description":"#1 SMP Fri Feb 9 14:58:00 UTC 2024","kernel_version":"4.18.0-540.el8.x86_64","mem_swap_kb":"0","mem_total_kb":"32543848","num_handles":"1","os":"Linux","pid":"7","realm_id":"","realm_name":"","zone_id":"52970d38-d172-4ec0-ad23-3afdfeba31e4","zone_name":"default","zonegroup_id":"343d5407-1492-440c-b380-09e4750e9af7","zonegroup_name":"default"},"task_status":{}}}}}},"progress_events":{}} 2024-02-23T00:45:27.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.465+0000 7fd64edfb700 10 librados: watch_flush enter 2024-02-23T00:45:27.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.465+0000 7fd64edfb700 10 librados: watch_flush exit 2024-02-23T00:45:27.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.465+0000 7fd64edfb700 20 client.30567.objecter shutdown clearing up homeless session... 2024-02-23T00:45:27.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.465+0000 7fd64edfb700 10 client.30567.objecter successfully canceled tick 2024-02-23T00:45:27.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.465+0000 7fd64edfb700 1 -- 172.21.15.47:0/758952144 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd638062cc0 msgr2=0x7fd638065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:27.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.465+0000 7fd64edfb700 1 --2- 172.21.15.47:0/758952144 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd638062cc0 0x7fd638065180 secure :-1 s=READY pgs=2989 cs=0 l=1 rev1=1 rx=0x7fd634004ee0 tx=0x7fd634009cb0).stop 2024-02-23T00:45:27.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.465+0000 7fd64edfb700 1 -- 172.21.15.47:0/758952144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6480f8420 msgr2=0x7fd648103910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:27.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.465+0000 7fd64edfb700 1 --2- 172.21.15.47:0/758952144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6480f8420 0x7fd648103910 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 rx=0x7fd63c00b530 tx=0x7fd63c0090f0).stop 2024-02-23T00:45:27.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.465+0000 7fd64edfb700 1 -- 172.21.15.47:0/758952144 shutdown_connections 2024-02-23T00:45:27.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.465+0000 7fd64edfb700 1 --2- 172.21.15.47:0/758952144 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd638062cc0 0x7fd638065180 unknown :-1 s=CLOSED pgs=2989 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:27.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.465+0000 7fd64edfb700 1 --2- 172.21.15.47:0/758952144 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd6480fe510 0x7fd648122dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:27.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.465+0000 7fd64edfb700 1 --2- 172.21.15.47:0/758952144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6480f8420 0x7fd648103910 unknown :-1 s=CLOSED pgs=2482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:27.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.465+0000 7fd64edfb700 1 --2- 172.21.15.47:0/758952144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6480fdbb0 0x7fd648103e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:27.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.465+0000 7fd64edfb700 1 -- 172.21.15.47:0/758952144 >> 172.21.15.47:0/758952144 conn(0x7fd6480f3800 msgr2=0x7fd6480f97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:45:27.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.465+0000 7fd64edfb700 1 -- 172.21.15.47:0/758952144 shutdown_connections 2024-02-23T00:45:27.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.465+0000 7fd64edfb700 1 -- 172.21.15.47:0/758952144 wait complete. 2024-02-23T00:45:27.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:27.465+0000 7fd64edfb700 1 librados: shutdown 2024-02-23T00:45:27.589 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:45:27.592 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:27.593 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:45:27.693 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:45:27.722 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:27.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:26.716104+0000 osd.1 (osd.1) 137 : cluster [DBG] 15.4 scrub starts 2024-02-23T00:45:27.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:26.719007+0000 osd.1 (osd.1) 138 : cluster [DBG] 15.4 scrub ok 2024-02-23T00:45:27.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:26.828339+0000 mon.b (mon.1) 314 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:27.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:26.883834+0000 mon.a (mon.0) 1733 : audit [INF] from='client.? 172.21.15.47:0/2827751117' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:45:27.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:26.933039+0000 osd.3 (osd.3) 145 : cluster [DBG] 15.1f deep-scrub starts 2024-02-23T00:45:27.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:26.934943+0000 osd.3 (osd.3) 146 : cluster [DBG] 15.1f deep-scrub ok 2024-02-23T00:45:27.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:27.127363+0000 mon.a (mon.0) 1734 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:45:27.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:27.132182+0000 mon.a (mon.0) 1735 : audit [INF] from='client.? 172.21.15.47:0/3327007590' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi047-82386-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:27.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:27.132305+0000 mon.a (mon.0) 1736 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi047-82378-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:27.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:27.132380+0000 mon.a (mon.0) 1737 : audit [INF] from='client.? 172.21.15.47:0/2827751117' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:45:27.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:27.132425+0000 mon.a (mon.0) 1738 : cluster [DBG] osdmap e255: 8 total, 8 up, 8 in 2024-02-23T00:45:27.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:27.138311+0000 mon.c (mon.2) 170 : audit [INF] from='client.? 172.21.15.47:0/538221556' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi047-82378-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi047-82378-35"}]: dispatch 2024-02-23T00:45:27.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:27.138815+0000 mon.a (mon.0) 1739 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi047-82378-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi047-82378-35"}]: dispatch 2024-02-23T00:45:27.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:27.163693+0000 mon.a (mon.0) 1740 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:45:27.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:27.464283+0000 mon.a (mon.0) 1741 : audit [DBG] from='client.? 172.21.15.47:0/758952144' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-23T00:45:27.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:26.716104+0000 osd.1 (osd.1) 137 : cluster [DBG] 15.4 scrub starts 2024-02-23T00:45:27.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:26.719007+0000 osd.1 (osd.1) 138 : cluster [DBG] 15.4 scrub ok 2024-02-23T00:45:27.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:26.828339+0000 mon.b (mon.1) 314 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:27.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:26.883834+0000 mon.a (mon.0) 1733 : audit [INF] from='client.? 172.21.15.47:0/2827751117' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:45:27.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:26.933039+0000 osd.3 (osd.3) 145 : cluster [DBG] 15.1f deep-scrub starts 2024-02-23T00:45:27.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:26.934943+0000 osd.3 (osd.3) 146 : cluster [DBG] 15.1f deep-scrub ok 2024-02-23T00:45:27.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:27.127363+0000 mon.a (mon.0) 1734 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:45:27.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:27.132182+0000 mon.a (mon.0) 1735 : audit [INF] from='client.? 172.21.15.47:0/3327007590' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi047-82386-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:27.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:27.132305+0000 mon.a (mon.0) 1736 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi047-82378-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:27.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:27.132380+0000 mon.a (mon.0) 1737 : audit [INF] from='client.? 172.21.15.47:0/2827751117' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:45:27.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:27.132425+0000 mon.a (mon.0) 1738 : cluster [DBG] osdmap e255: 8 total, 8 up, 8 in 2024-02-23T00:45:27.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:27.138311+0000 mon.c (mon.2) 170 : audit [INF] from='client.? 172.21.15.47:0/538221556' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi047-82378-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi047-82378-35"}]: dispatch 2024-02-23T00:45:27.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:27.138815+0000 mon.a (mon.0) 1739 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi047-82378-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi047-82378-35"}]: dispatch 2024-02-23T00:45:27.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:27.163693+0000 mon.a (mon.0) 1740 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:45:27.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:27.464283+0000 mon.a (mon.0) 1741 : audit [DBG] from='client.? 172.21.15.47:0/758952144' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-23T00:45:27.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:26.716104+0000 osd.1 (osd.1) 137 : cluster [DBG] 15.4 scrub starts 2024-02-23T00:45:27.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:26.719007+0000 osd.1 (osd.1) 138 : cluster [DBG] 15.4 scrub ok 2024-02-23T00:45:27.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:26.828339+0000 mon.b (mon.1) 314 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:27.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:26.883834+0000 mon.a (mon.0) 1733 : audit [INF] from='client.? 172.21.15.47:0/2827751117' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:45:27.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:26.933039+0000 osd.3 (osd.3) 145 : cluster [DBG] 15.1f deep-scrub starts 2024-02-23T00:45:27.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:26.934943+0000 osd.3 (osd.3) 146 : cluster [DBG] 15.1f deep-scrub ok 2024-02-23T00:45:27.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:27.127363+0000 mon.a (mon.0) 1734 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:45:27.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:27.132182+0000 mon.a (mon.0) 1735 : audit [INF] from='client.? 172.21.15.47:0/3327007590' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi047-82386-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:27.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:27.132305+0000 mon.a (mon.0) 1736 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi047-82378-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:27.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:27.132380+0000 mon.a (mon.0) 1737 : audit [INF] from='client.? 172.21.15.47:0/2827751117' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:45:27.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:27.132425+0000 mon.a (mon.0) 1738 : cluster [DBG] osdmap e255: 8 total, 8 up, 8 in 2024-02-23T00:45:27.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:27.138311+0000 mon.c (mon.2) 170 : audit [INF] from='client.? 172.21.15.47:0/538221556' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi047-82378-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi047-82378-35"}]: dispatch 2024-02-23T00:45:27.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:27.138815+0000 mon.a (mon.0) 1739 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi047-82378-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi047-82378-35"}]: dispatch 2024-02-23T00:45:27.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:27.163693+0000 mon.a (mon.0) 1740 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:45:27.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:27.464283+0000 mon.a (mon.0) 1741 : audit [DBG] from='client.? 172.21.15.47:0/758952144' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-23T00:45:28.172 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:45:28.175 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:28.175 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:45:28.276 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:45:28.309 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:28.331 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:45:28.334 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:28.334 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:45:28.435 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:45:28.464 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:28.736 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:26.757398+0000 osd.7 (osd.7) 133 : cluster [DBG] 15.d scrub starts 2024-02-23T00:45:28.736 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:26.760857+0000 osd.7 (osd.7) 134 : cluster [DBG] 15.d scrub ok 2024-02-23T00:45:28.737 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:27.240944+0000 mgr.y (mgr.14517) 272 : cluster [DBG] pgmap v358: 282 pgs: 1 active+clean+snaptrim, 281 active+clean; 9.0 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-23T00:45:28.737 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:27.829491+0000 mon.b (mon.1) 315 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:28.737 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:28.569092+0000 mon.a (mon.0) 1742 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:45:28.737 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:28.569158+0000 mon.a (mon.0) 1743 : cluster [DBG] osdmap e256: 8 total, 8 up, 8 in 2024-02-23T00:45:28.737 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:28.570115+0000 mon.a (mon.0) 1744 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-27"}]: dispatch 2024-02-23T00:45:28.737 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:28.587127+0000 mon.c (mon.2) 171 : audit [INF] from='client.? 172.21.15.47:0/2348961114' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi047-82386-45"}]: dispatch 2024-02-23T00:45:28.737 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:28.587481+0000 mon.a (mon.0) 1745 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi047-82386-45"}]: dispatch 2024-02-23T00:45:28.737 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:28.588171+0000 mon.c (mon.2) 172 : audit [INF] from='client.? 172.21.15.47:0/2348961114' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi047-82386-45"}]: dispatch 2024-02-23T00:45:28.737 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:28.588427+0000 mon.a (mon.0) 1746 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi047-82386-45"}]: dispatch 2024-02-23T00:45:28.737 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:28.588834+0000 mon.c (mon.2) 173 : audit [INF] from='client.? 172.21.15.47:0/2348961114' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi047-82386-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:28.737 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:28.589091+0000 mon.a (mon.0) 1747 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi047-82386-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:28.738 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:26.757398+0000 osd.7 (osd.7) 133 : cluster [DBG] 15.d scrub starts 2024-02-23T00:45:28.738 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:26.760857+0000 osd.7 (osd.7) 134 : cluster [DBG] 15.d scrub ok 2024-02-23T00:45:28.738 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:27.240944+0000 mgr.y (mgr.14517) 272 : cluster [DBG] pgmap v358: 282 pgs: 1 active+clean+snaptrim, 281 active+clean; 9.0 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-23T00:45:28.738 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:27.829491+0000 mon.b (mon.1) 315 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:28.738 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:28.569092+0000 mon.a (mon.0) 1742 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:45:28.738 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:28.569158+0000 mon.a (mon.0) 1743 : cluster [DBG] osdmap e256: 8 total, 8 up, 8 in 2024-02-23T00:45:28.738 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:28.570115+0000 mon.a (mon.0) 1744 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-27"}]: dispatch 2024-02-23T00:45:28.738 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:28.587127+0000 mon.c (mon.2) 171 : audit [INF] from='client.? 172.21.15.47:0/2348961114' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi047-82386-45"}]: dispatch 2024-02-23T00:45:28.738 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:28.587481+0000 mon.a (mon.0) 1745 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi047-82386-45"}]: dispatch 2024-02-23T00:45:28.738 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:28.588171+0000 mon.c (mon.2) 172 : audit [INF] from='client.? 172.21.15.47:0/2348961114' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi047-82386-45"}]: dispatch 2024-02-23T00:45:28.738 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:28.588427+0000 mon.a (mon.0) 1746 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi047-82386-45"}]: dispatch 2024-02-23T00:45:28.738 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:28.588834+0000 mon.c (mon.2) 173 : audit [INF] from='client.? 172.21.15.47:0/2348961114' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi047-82386-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:28.738 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:28.589091+0000 mon.a (mon.0) 1747 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi047-82386-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:28.754 INFO:tasks.cephadm.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-02-23T00:45:28.755 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph pg dump --format=json 2024-02-23T00:45:28.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:26.757398+0000 osd.7 (osd.7) 133 : cluster [DBG] 15.d scrub starts 2024-02-23T00:45:28.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:26.760857+0000 osd.7 (osd.7) 134 : cluster [DBG] 15.d scrub ok 2024-02-23T00:45:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:27.240944+0000 mgr.y (mgr.14517) 272 : cluster [DBG] pgmap v358: 282 pgs: 1 active+clean+snaptrim, 281 active+clean; 9.0 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-23T00:45:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:27.829491+0000 mon.b (mon.1) 315 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:28.569092+0000 mon.a (mon.0) 1742 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:45:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:28.569158+0000 mon.a (mon.0) 1743 : cluster [DBG] osdmap e256: 8 total, 8 up, 8 in 2024-02-23T00:45:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:28.570115+0000 mon.a (mon.0) 1744 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-27"}]: dispatch 2024-02-23T00:45:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:28.587127+0000 mon.c (mon.2) 171 : audit [INF] from='client.? 172.21.15.47:0/2348961114' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi047-82386-45"}]: dispatch 2024-02-23T00:45:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:28.587481+0000 mon.a (mon.0) 1745 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi047-82386-45"}]: dispatch 2024-02-23T00:45:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:28.588171+0000 mon.c (mon.2) 172 : audit [INF] from='client.? 172.21.15.47:0/2348961114' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi047-82386-45"}]: dispatch 2024-02-23T00:45:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:28.588427+0000 mon.a (mon.0) 1746 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi047-82386-45"}]: dispatch 2024-02-23T00:45:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:28.588834+0000 mon.c (mon.2) 173 : audit [INF] from='client.? 172.21.15.47:0/2348961114' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi047-82386-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:28.589091+0000 mon.a (mon.0) 1747 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi047-82386-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:29.216 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:45:29.222 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:29.222 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:45:29.323 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:45:29.350 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:29.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:28.830644+0000 mon.b (mon.1) 316 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:29.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:29.135030+0000 mon.a (mon.0) 1748 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi047-82378-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi047-82378-35"}]': finished 2024-02-23T00:45:29.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:29.135129+0000 mon.a (mon.0) 1749 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-27"}]': finished 2024-02-23T00:45:29.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:29.135175+0000 mon.a (mon.0) 1750 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi047-82386-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:29.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:29.135249+0000 mon.a (mon.0) 1751 : cluster [DBG] osdmap e257: 8 total, 8 up, 8 in 2024-02-23T00:45:29.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:29.136783+0000 mon.a (mon.0) 1752 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-27", "mode": "writeback"}]: dispatch 2024-02-23T00:45:29.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:29.137377+0000 mon.c (mon.2) 174 : audit [INF] from='client.? 172.21.15.47:0/2348961114' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi047-82386-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi047-82386-45"}]: dispatch 2024-02-23T00:45:29.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:29.137811+0000 mon.a (mon.0) 1753 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi047-82386-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi047-82386-45"}]: dispatch 2024-02-23T00:45:29.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:28.830644+0000 mon.b (mon.1) 316 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:29.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:29.135030+0000 mon.a (mon.0) 1748 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi047-82378-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi047-82378-35"}]': finished 2024-02-23T00:45:29.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:29.135129+0000 mon.a (mon.0) 1749 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-27"}]': finished 2024-02-23T00:45:29.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:29.135175+0000 mon.a (mon.0) 1750 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi047-82386-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:29.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:29.135249+0000 mon.a (mon.0) 1751 : cluster [DBG] osdmap e257: 8 total, 8 up, 8 in 2024-02-23T00:45:29.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:29.136783+0000 mon.a (mon.0) 1752 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-27", "mode": "writeback"}]: dispatch 2024-02-23T00:45:29.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:29.137377+0000 mon.c (mon.2) 174 : audit [INF] from='client.? 172.21.15.47:0/2348961114' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi047-82386-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi047-82386-45"}]: dispatch 2024-02-23T00:45:29.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:29.137811+0000 mon.a (mon.0) 1753 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi047-82386-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi047-82386-45"}]: dispatch 2024-02-23T00:45:29.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:28.830644+0000 mon.b (mon.1) 316 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:29.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:29.135030+0000 mon.a (mon.0) 1748 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi047-82378-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi047-82378-35"}]': finished 2024-02-23T00:45:29.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:29.135129+0000 mon.a (mon.0) 1749 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-27"}]': finished 2024-02-23T00:45:29.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:29.135175+0000 mon.a (mon.0) 1750 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi047-82386-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:29.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:29.135249+0000 mon.a (mon.0) 1751 : cluster [DBG] osdmap e257: 8 total, 8 up, 8 in 2024-02-23T00:45:29.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:29.136783+0000 mon.a (mon.0) 1752 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-27", "mode": "writeback"}]: dispatch 2024-02-23T00:45:29.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:29.137377+0000 mon.c (mon.2) 174 : audit [INF] from='client.? 172.21.15.47:0/2348961114' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi047-82386-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi047-82386-45"}]: dispatch 2024-02-23T00:45:29.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:29.137811+0000 mon.a (mon.0) 1753 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi047-82386-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi047-82386-45"}]: dispatch 2024-02-23T00:45:30.000 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:45:30.003 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:30.003 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:45:30.104 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:45:30.138 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:30.160 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:45:30.163 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:30.163 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:45:30.164 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:45:30.264 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:45:30.294 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:30.314 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:45:30.316 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:30.317 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:45:30.391 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:45:30.417 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:45:30.429 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:45:30.429 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:45:30.447 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:30.469 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:45:30.471 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:30.471 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:45:30.572 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:45:30.605 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:30.706 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:45:30.711 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:29.241361+0000 mgr.y (mgr.14517) 273 : cluster [DBG] pgmap v361: 258 pgs: 8 unknown, 1 active+clean+snaptrim, 249 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-23T00:45:30.711 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:29.340578+0000 osd.7 (osd.7) 135 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:45:30.711 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:29.340764+0000 osd.7 (osd.7) 136 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:45:30.711 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:29.831782+0000 mon.b (mon.1) 317 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:30.711 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:30.125581+0000 mon.a (mon.0) 1754 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:45:30.711 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:30.130676+0000 mon.a (mon.0) 1755 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-27", "mode": "writeback"}]': finished 2024-02-23T00:45:30.711 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:30.574674+0000 mon.a (mon.0) 1756 : cluster [DBG] osdmap e258: 8 total, 8 up, 8 in 2024-02-23T00:45:30.712 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:45:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:45:30] "GET /metrics HTTP/1.1" 200 192377 "" "Prometheus/2.33.4" 2024-02-23T00:45:30.712 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:30.713 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:45:30.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:29.241361+0000 mgr.y (mgr.14517) 273 : cluster [DBG] pgmap v361: 258 pgs: 8 unknown, 1 active+clean+snaptrim, 249 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-23T00:45:30.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:29.340578+0000 osd.7 (osd.7) 135 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:45:30.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:29.340764+0000 osd.7 (osd.7) 136 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:45:30.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:29.831782+0000 mon.b (mon.1) 317 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:30.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:30.125581+0000 mon.a (mon.0) 1754 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:45:30.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:30.130676+0000 mon.a (mon.0) 1755 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-27", "mode": "writeback"}]': finished 2024-02-23T00:45:30.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:30.574674+0000 mon.a (mon.0) 1756 : cluster [DBG] osdmap e258: 8 total, 8 up, 8 in 2024-02-23T00:45:30.813 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:45:30.843 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:30.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:29.241361+0000 mgr.y (mgr.14517) 273 : cluster [DBG] pgmap v361: 258 pgs: 8 unknown, 1 active+clean+snaptrim, 249 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-23T00:45:30.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:29.340578+0000 osd.7 (osd.7) 135 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:45:30.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:29.340764+0000 osd.7 (osd.7) 136 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:45:30.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:29.831782+0000 mon.b (mon.1) 317 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:30.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:30.125581+0000 mon.a (mon.0) 1754 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:45:30.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:30.130676+0000 mon.a (mon.0) 1755 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-27", "mode": "writeback"}]': finished 2024-02-23T00:45:30.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:30.574674+0000 mon.a (mon.0) 1756 : cluster [DBG] osdmap e258: 8 total, 8 up, 8 in 2024-02-23T00:45:31.392 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:45:31.395 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:31.395 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:45:31.496 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:45:31.525 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:31.545 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:45:31.558 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:31.559 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:45:31.660 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:45:31.691 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:31.712 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:45:31.715 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:31.716 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:45:31.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:30.832610+0000 mon.b (mon.1) 318 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:31.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:31.132426+0000 mon.a (mon.0) 1757 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi047-82386-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi047-82386-45"}]': finished 2024-02-23T00:45:31.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:31.132547+0000 mon.a (mon.0) 1758 : cluster [DBG] osdmap e259: 8 total, 8 up, 8 in 2024-02-23T00:45:31.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:31.136533+0000 mon.c (mon.2) 175 : audit [INF] from='client.? 172.21.15.47:0/538221556' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi047-82378-35"}]: dispatch 2024-02-23T00:45:31.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:31.140259+0000 mon.a (mon.0) 1759 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi047-82378-35"}]: dispatch 2024-02-23T00:45:31.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:31.161911+0000 mon.a (mon.0) 1760 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:45:31.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:30.832610+0000 mon.b (mon.1) 318 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:31.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:31.132426+0000 mon.a (mon.0) 1757 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi047-82386-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi047-82386-45"}]': finished 2024-02-23T00:45:31.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:31.132547+0000 mon.a (mon.0) 1758 : cluster [DBG] osdmap e259: 8 total, 8 up, 8 in 2024-02-23T00:45:31.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:31.136533+0000 mon.c (mon.2) 175 : audit [INF] from='client.? 172.21.15.47:0/538221556' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi047-82378-35"}]: dispatch 2024-02-23T00:45:31.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:31.140259+0000 mon.a (mon.0) 1759 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi047-82378-35"}]: dispatch 2024-02-23T00:45:31.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:31.161911+0000 mon.a (mon.0) 1760 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:45:31.817 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:45:31.846 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:31.865 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-23T00:45:31.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:30.832610+0000 mon.b (mon.1) 318 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:31.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:31.132426+0000 mon.a (mon.0) 1757 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi047-82386-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi047-82386-45"}]': finished 2024-02-23T00:45:31.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:31.132547+0000 mon.a (mon.0) 1758 : cluster [DBG] osdmap e259: 8 total, 8 up, 8 in 2024-02-23T00:45:31.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:31.136533+0000 mon.c (mon.2) 175 : audit [INF] from='client.? 172.21.15.47:0/538221556' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi047-82378-35"}]: dispatch 2024-02-23T00:45:31.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:31.140259+0000 mon.a (mon.0) 1759 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi047-82378-35"}]: dispatch 2024-02-23T00:45:31.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:31.161911+0000 mon.a (mon.0) 1760 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:45:31.996 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:45:32.003 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:32.003 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:45:32.104 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:45:32.180 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:32.200 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:45:32.203 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:32.203 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:45:32.304 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:45:32.332 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:32.573 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.2 dump_ops_in_flight 2024-02-23T00:45:32.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:31.241750+0000 mgr.y (mgr.14517) 274 : cluster [DBG] pgmap v364: 258 pgs: 8 unknown, 1 active+clean+snaptrim, 249 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:45:32.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:31.833569+0000 mon.b (mon.1) 319 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:32.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:32.136532+0000 mon.a (mon.0) 1761 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi047-82378-35"}]': finished 2024-02-23T00:45:32.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:32.136656+0000 mon.a (mon.0) 1762 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:45:32.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:32.136703+0000 mon.a (mon.0) 1763 : cluster [DBG] osdmap e260: 8 total, 8 up, 8 in 2024-02-23T00:45:32.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:32.138277+0000 mon.a (mon.0) 1764 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-27"}]: dispatch 2024-02-23T00:45:32.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:32.138309+0000 mon.c (mon.2) 176 : audit [INF] from='client.? 172.21.15.47:0/538221556' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi047-82378-35"}]: dispatch 2024-02-23T00:45:32.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:32.138643+0000 mon.a (mon.0) 1765 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi047-82378-35"}]: dispatch 2024-02-23T00:45:32.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:31.241750+0000 mgr.y (mgr.14517) 274 : cluster [DBG] pgmap v364: 258 pgs: 8 unknown, 1 active+clean+snaptrim, 249 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:45:32.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:31.833569+0000 mon.b (mon.1) 319 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:32.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:32.136532+0000 mon.a (mon.0) 1761 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi047-82378-35"}]': finished 2024-02-23T00:45:32.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:32.136656+0000 mon.a (mon.0) 1762 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:45:32.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:32.136703+0000 mon.a (mon.0) 1763 : cluster [DBG] osdmap e260: 8 total, 8 up, 8 in 2024-02-23T00:45:32.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:32.138277+0000 mon.a (mon.0) 1764 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-27"}]: dispatch 2024-02-23T00:45:32.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:32.138309+0000 mon.c (mon.2) 176 : audit [INF] from='client.? 172.21.15.47:0/538221556' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi047-82378-35"}]: dispatch 2024-02-23T00:45:32.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:32.138643+0000 mon.a (mon.0) 1765 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi047-82378-35"}]: dispatch 2024-02-23T00:45:32.943 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:45:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:45:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:45:32.944 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:45:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:45:32.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:31.241750+0000 mgr.y (mgr.14517) 274 : cluster [DBG] pgmap v364: 258 pgs: 8 unknown, 1 active+clean+snaptrim, 249 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:45:32.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:31.833569+0000 mon.b (mon.1) 319 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:32.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:32.136532+0000 mon.a (mon.0) 1761 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi047-82378-35"}]': finished 2024-02-23T00:45:32.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:32.136656+0000 mon.a (mon.0) 1762 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:45:32.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:32.136703+0000 mon.a (mon.0) 1763 : cluster [DBG] osdmap e260: 8 total, 8 up, 8 in 2024-02-23T00:45:32.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:32.138277+0000 mon.a (mon.0) 1764 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-27"}]: dispatch 2024-02-23T00:45:32.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:32.138309+0000 mon.c (mon.2) 176 : audit [INF] from='client.? 172.21.15.47:0/538221556' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi047-82378-35"}]: dispatch 2024-02-23T00:45:32.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:32.138643+0000 mon.a (mon.0) 1765 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi047-82378-35"}]: dispatch 2024-02-23T00:45:33.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.503+0000 7fdcd80e9700 1 -- 172.21.15.47:0/1630714017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcd00f9670 msgr2=0x7fdcd0101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:33.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.503+0000 7fdcd80e9700 1 --2- 172.21.15.47:0/1630714017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcd00f9670 0x7fdcd0101d80 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 rx=0x7fdcc0004870 tx=0x7fdcc0009cb0).stop 2024-02-23T00:45:33.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.504+0000 7fdcd80e9700 1 -- 172.21.15.47:0/1630714017 shutdown_connections 2024-02-23T00:45:33.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.504+0000 7fdcd80e9700 1 --2- 172.21.15.47:0/1630714017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcd0102270 0x7fdcd0104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:33.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.504+0000 7fdcd80e9700 1 --2- 172.21.15.47:0/1630714017 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdcd00f8d10 0x7fdcd00f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:33.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.504+0000 7fdcd80e9700 1 --2- 172.21.15.47:0/1630714017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcd00f9670 0x7fdcd0101d80 unknown :-1 s=CLOSED pgs=2484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:33.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.504+0000 7fdcd80e9700 1 -- 172.21.15.47:0/1630714017 >> 172.21.15.47:0/1630714017 conn(0x7fdcd00f2820 msgr2=0x7fdcd00f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:45:33.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.504+0000 7fdcd80e9700 1 -- 172.21.15.47:0/1630714017 shutdown_connections 2024-02-23T00:45:33.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.504+0000 7fdcd80e9700 1 -- 172.21.15.47:0/1630714017 wait complete. 2024-02-23T00:45:33.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.504+0000 7fdcd80e9700 1 librados: starting msgr at 2024-02-23T00:45:33.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.504+0000 7fdcd80e9700 1 librados: starting objecter 2024-02-23T00:45:33.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.504+0000 7fdcd80e9700 1 Processor -- start 2024-02-23T00:45:33.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.504+0000 7fdcd80e9700 1 -- start start 2024-02-23T00:45:33.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.504+0000 7fdcd80e9700 1 librados: setting wanted keys 2024-02-23T00:45:33.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.504+0000 7fdcd80e9700 1 librados: calling monclient init 2024-02-23T00:45:33.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.505+0000 7fdcd80e9700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcd00f8d10 0x7fdcd00f6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:33.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.505+0000 7fdcd80e9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcd00f9670 0x7fdcd00f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:33.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.505+0000 7fdcd80e9700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdcd0102270 0x7fdcd00f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:33.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.505+0000 7fdcd80e9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd0116530 con 0x7fdcd00f9670 2024-02-23T00:45:33.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.505+0000 7fdcd80e9700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd01166a0 con 0x7fdcd00f8d10 2024-02-23T00:45:33.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.505+0000 7fdcd80e9700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd0116860 con 0x7fdcd0102270 2024-02-23T00:45:33.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.505+0000 7fdcd5684700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcd00f9670 0x7fdcd00f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:33.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.505+0000 7fdcd6686700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdcd0102270 0x7fdcd00f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:33.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.505+0000 7fdcd5684700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcd00f9670 0x7fdcd00f71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52110/0 (socket says 172.21.15.47:52110) 2024-02-23T00:45:33.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.505+0000 7fdcd6686700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdcd0102270 0x7fdcd00f76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:48424/0 (socket says 172.21.15.47:48424) 2024-02-23T00:45:33.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.505+0000 7fdcd5684700 1 -- 172.21.15.47:0/1432776762 learned_addr learned my addr 172.21.15.47:0/1432776762 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:45:33.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.505+0000 7fdcd5e85700 1 --2- 172.21.15.47:0/1432776762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcd00f8d10 0x7fdcd00f6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:33.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.505+0000 7fdcd5684700 1 -- 172.21.15.47:0/1432776762 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdcd0102270 msgr2=0x7fdcd00f76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:33.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.505+0000 7fdcd5684700 1 --2- 172.21.15.47:0/1432776762 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdcd0102270 0x7fdcd00f76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:33.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.505+0000 7fdcd5684700 1 -- 172.21.15.47:0/1432776762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcd00f8d10 msgr2=0x7fdcd00f6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:33.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.505+0000 7fdcd5684700 1 --2- 172.21.15.47:0/1432776762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcd00f8d10 0x7fdcd00f6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:33.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.505+0000 7fdcd5684700 1 -- 172.21.15.47:0/1432776762 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc0005040 con 0x7fdcd00f9670 2024-02-23T00:45:33.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.505+0000 7fdcd6686700 1 --2- 172.21.15.47:0/1432776762 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdcd0102270 0x7fdcd00f76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:45:33.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.505+0000 7fdcd5684700 1 --2- 172.21.15.47:0/1432776762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcd00f9670 0x7fdcd00f71b0 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 rx=0x7fdcc0009da0 tx=0x7fdcc0009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:33.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.505+0000 7fdcc6ffd700 10 client.?.objecter ms_handle_connect 0x7fdcd00f9670 2024-02-23T00:45:33.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.505+0000 7fdcc6ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:45:33.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.505+0000 7fdcc6ffd700 1 -- 172.21.15.47:0/1432776762 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc000e030 con 0x7fdcd00f9670 2024-02-23T00:45:33.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.505+0000 7fdcd80e9700 1 -- 172.21.15.47:0/1432776762 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd0116ae0 con 0x7fdcd00f9670 2024-02-23T00:45:33.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.505+0000 7fdcc6ffd700 1 -- 172.21.15.47:0/1432776762 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fdcc0013d70 con 0x7fdcd00f9670 2024-02-23T00:45:33.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.505+0000 7fdcc6ffd700 1 -- 172.21.15.47:0/1432776762 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0018070 con 0x7fdcd00f9670 2024-02-23T00:45:33.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.505+0000 7fdcd80e9700 10 client.30585.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:45:33.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.505+0000 7fdcd80e9700 1 -- 172.21.15.47:0/1432776762 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd0116e90 con 0x7fdcd00f9670 2024-02-23T00:45:33.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.505+0000 7fdcd80e9700 1 librados: init done 2024-02-23T00:45:33.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.506+0000 7fdcd80e9700 1 -- 172.21.15.47:0/1432776762 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcd005fa60 con 0x7fdcd00f9670 2024-02-23T00:45:33.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.507+0000 7fdcc6ffd700 1 -- 172.21.15.47:0/1432776762 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fdcc0009240 con 0x7fdcd00f9670 2024-02-23T00:45:33.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.507+0000 7fdcc6ffd700 1 --2- 172.21.15.47:0/1432776762 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdcbc062cc0 0x7fdcbc065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:33.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.507+0000 7fdcc6ffd700 1 -- 172.21.15.47:0/1432776762 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(261..261 src has 48..261) v4 ==== 9957+0+0 (secure 0 0 0) 0x7fdcc007ea90 con 0x7fdcd00f9670 2024-02-23T00:45:33.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.507+0000 7fdcc6ffd700 10 client.30585.objecter ms_dispatch 0x7fdcd0001550 osd_map(261..261 src has 48..261) v4 2024-02-23T00:45:33.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.507+0000 7fdcc6ffd700 3 client.30585.objecter handle_osd_map got epochs [261,261] > 0 2024-02-23T00:45:33.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.507+0000 7fdcc6ffd700 3 client.30585.objecter handle_osd_map decoding full epoch 261 2024-02-23T00:45:33.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.507+0000 7fdcd5e85700 1 --2- 172.21.15.47:0/1432776762 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdcbc062cc0 0x7fdcbc065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:33.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.507+0000 7fdcc6ffd700 20 client.30585.objecter dump_active .. 0 homeless 2024-02-23T00:45:33.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.508+0000 7fdcc6ffd700 10 client.30585.objecter ms_handle_connect 0x7fdcbc062cc0 2024-02-23T00:45:33.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.508+0000 7fdcd5e85700 1 --2- 172.21.15.47:0/1432776762 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdcbc062cc0 0x7fdcbc065180 secure :-1 s=READY pgs=2992 cs=0 l=1 rev1=1 rx=0x7fdccc004790 tx=0x7fdccc00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:33.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.512+0000 7fdcc6ffd700 1 -- 172.21.15.47:0/1432776762 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fdcc004ddc0 con 0x7fdcd00f9670 2024-02-23T00:45:33.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.685+0000 7fdcd80e9700 1 -- 172.21.15.47:0/1432776762 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdcd00f81a0 con 0x7fdcbc062cc0 2024-02-23T00:45:33.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.702+0000 7fdcc6ffd700 1 -- 172.21.15.47:0/1432776762 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+635251 (secure 0 0 0) 0x7fdcd00f81a0 con 0x7fdcbc062cc0 2024-02-23T00:45:33.712 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:45:33.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.706+0000 7fdcd80e9700 10 librados: watch_flush enter 2024-02-23T00:45:33.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.707+0000 7fdcd80e9700 10 librados: watch_flush exit 2024-02-23T00:45:33.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.707+0000 7fdcd80e9700 20 client.30585.objecter shutdown clearing up homeless session... 2024-02-23T00:45:33.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.707+0000 7fdcd80e9700 10 client.30585.objecter successfully canceled tick 2024-02-23T00:45:33.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.707+0000 7fdcd80e9700 1 -- 172.21.15.47:0/1432776762 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdcbc062cc0 msgr2=0x7fdcbc065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:33.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.707+0000 7fdcd80e9700 1 --2- 172.21.15.47:0/1432776762 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdcbc062cc0 0x7fdcbc065180 secure :-1 s=READY pgs=2992 cs=0 l=1 rev1=1 rx=0x7fdccc004790 tx=0x7fdccc00b040).stop 2024-02-23T00:45:33.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.707+0000 7fdcd80e9700 1 -- 172.21.15.47:0/1432776762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcd00f9670 msgr2=0x7fdcd00f71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:33.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.707+0000 7fdcd80e9700 1 --2- 172.21.15.47:0/1432776762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcd00f9670 0x7fdcd00f71b0 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 rx=0x7fdcc0009da0 tx=0x7fdcc0009cb0).stop 2024-02-23T00:45:33.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.707+0000 7fdcd80e9700 1 -- 172.21.15.47:0/1432776762 shutdown_connections 2024-02-23T00:45:33.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.707+0000 7fdcd80e9700 1 --2- 172.21.15.47:0/1432776762 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdcbc062cc0 0x7fdcbc065180 unknown :-1 s=CLOSED pgs=2992 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:33.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.707+0000 7fdcd80e9700 1 --2- 172.21.15.47:0/1432776762 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdcd0102270 0x7fdcd00f76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:33.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.707+0000 7fdcd80e9700 1 --2- 172.21.15.47:0/1432776762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcd00f8d10 0x7fdcd00f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:33.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.707+0000 7fdcd80e9700 1 --2- 172.21.15.47:0/1432776762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcd00f9670 0x7fdcd00f71b0 unknown :-1 s=CLOSED pgs=2485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:33.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.707+0000 7fdcd80e9700 1 -- 172.21.15.47:0/1432776762 >> 172.21.15.47:0/1432776762 conn(0x7fdcd00f2820 msgr2=0x7fdcd00f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:45:33.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.707+0000 7fdcd80e9700 1 -- 172.21.15.47:0/1432776762 shutdown_connections 2024-02-23T00:45:33.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.707+0000 7fdcd80e9700 1 -- 172.21.15.47:0/1432776762 wait complete. 2024-02-23T00:45:33.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:33.707+0000 7fdcd80e9700 1 librados: shutdown 2024-02-23T00:45:33.720 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-02-23T00:45:33.814 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:45:33.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:32.450794+0000 mgr.y (mgr.14517) 275 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:45:33.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:32.643080+0000 osd.6 (osd.6) 107 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:45:33.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:32.643330+0000 osd.6 (osd.6) 108 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:45:33.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:32.834882+0000 mon.b (mon.1) 320 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:33.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:32.880747+0000 osd.4 (osd.4) 121 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:45:33.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:32.880910+0000 osd.4 (osd.4) 122 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:45:33.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:33.134520+0000 mon.a (mon.0) 1766 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:45:33.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:33.139035+0000 mon.a (mon.0) 1767 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-27"}]': finished 2024-02-23T00:45:33.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:33.139125+0000 mon.a (mon.0) 1768 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi047-82378-35"}]': finished 2024-02-23T00:45:33.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:33.139213+0000 mon.a (mon.0) 1769 : cluster [DBG] osdmap e261: 8 total, 8 up, 8 in 2024-02-23T00:45:33.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:33.145307+0000 mon.c (mon.2) 177 : audit [INF] from='client.? 172.21.15.47:0/2348961114' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi047-82386-45"}]: dispatch 2024-02-23T00:45:33.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:33.145781+0000 mon.a (mon.0) 1770 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi047-82386-45"}]: dispatch 2024-02-23T00:45:33.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:33.167615+0000 mon.a (mon.0) 1771 : audit [INF] from='client.? 172.21.15.47:0/1933959576' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi047-82378-36"}]: dispatch 2024-02-23T00:45:33.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:33.168077+0000 mon.a (mon.0) 1772 : audit [INF] from='client.? 172.21.15.47:0/1933959576' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi047-82378-36"}]: dispatch 2024-02-23T00:45:33.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:33.168449+0000 mon.a (mon.0) 1773 : audit [INF] from='client.? 172.21.15.47:0/1933959576' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi047-82378-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:34.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:32.450794+0000 mgr.y (mgr.14517) 275 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:45:34.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:32.643080+0000 osd.6 (osd.6) 107 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:45:34.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:32.643330+0000 osd.6 (osd.6) 108 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:45:34.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:32.834882+0000 mon.b (mon.1) 320 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:34.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:32.880747+0000 osd.4 (osd.4) 121 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:45:34.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:32.880910+0000 osd.4 (osd.4) 122 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:45:34.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:33.134520+0000 mon.a (mon.0) 1766 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:45:34.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:33.139035+0000 mon.a (mon.0) 1767 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-27"}]': finished 2024-02-23T00:45:34.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:33.139125+0000 mon.a (mon.0) 1768 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi047-82378-35"}]': finished 2024-02-23T00:45:34.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:33.139213+0000 mon.a (mon.0) 1769 : cluster [DBG] osdmap e261: 8 total, 8 up, 8 in 2024-02-23T00:45:34.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:33.145307+0000 mon.c (mon.2) 177 : audit [INF] from='client.? 172.21.15.47:0/2348961114' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi047-82386-45"}]: dispatch 2024-02-23T00:45:34.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:33.145781+0000 mon.a (mon.0) 1770 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi047-82386-45"}]: dispatch 2024-02-23T00:45:34.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:33.167615+0000 mon.a (mon.0) 1771 : audit [INF] from='client.? 172.21.15.47:0/1933959576' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi047-82378-36"}]: dispatch 2024-02-23T00:45:34.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:33.168077+0000 mon.a (mon.0) 1772 : audit [INF] from='client.? 172.21.15.47:0/1933959576' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi047-82378-36"}]: dispatch 2024-02-23T00:45:34.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:33.168449+0000 mon.a (mon.0) 1773 : audit [INF] from='client.? 172.21.15.47:0/1933959576' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi047-82378-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:34.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:32.450794+0000 mgr.y (mgr.14517) 275 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:45:34.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:32.643080+0000 osd.6 (osd.6) 107 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:45:34.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:32.643330+0000 osd.6 (osd.6) 108 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:45:34.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:32.834882+0000 mon.b (mon.1) 320 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:34.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:32.880747+0000 osd.4 (osd.4) 121 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:45:34.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:32.880910+0000 osd.4 (osd.4) 122 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:45:34.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:33.134520+0000 mon.a (mon.0) 1766 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:45:34.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:33.139035+0000 mon.a (mon.0) 1767 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-27"}]': finished 2024-02-23T00:45:34.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:33.139125+0000 mon.a (mon.0) 1768 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi047-82378-35"}]': finished 2024-02-23T00:45:34.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:33.139213+0000 mon.a (mon.0) 1769 : cluster [DBG] osdmap e261: 8 total, 8 up, 8 in 2024-02-23T00:45:34.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:33.145307+0000 mon.c (mon.2) 177 : audit [INF] from='client.? 172.21.15.47:0/2348961114' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi047-82386-45"}]: dispatch 2024-02-23T00:45:34.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:33.145781+0000 mon.a (mon.0) 1770 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi047-82386-45"}]: dispatch 2024-02-23T00:45:34.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:33.167615+0000 mon.a (mon.0) 1771 : audit [INF] from='client.? 172.21.15.47:0/1933959576' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi047-82378-36"}]: dispatch 2024-02-23T00:45:34.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:33.168077+0000 mon.a (mon.0) 1772 : audit [INF] from='client.? 172.21.15.47:0/1933959576' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi047-82378-36"}]: dispatch 2024-02-23T00:45:34.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:33.168449+0000 mon.a (mon.0) 1773 : audit [INF] from='client.? 172.21.15.47:0/1933959576' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi047-82378-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:34.812 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:45:34.819 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:34.819 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:45:34.920 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:45:34.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:33.242196+0000 mgr.y (mgr.14517) 276 : cluster [DBG] pgmap v367: 250 pgs: 13 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 233 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-23T00:45:34.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:33.836091+0000 mon.b (mon.1) 321 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:34.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:34.143731+0000 mon.a (mon.0) 1774 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi047-82386-45"}]': finished 2024-02-23T00:45:34.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:34.143827+0000 mon.a (mon.0) 1775 : audit [INF] from='client.? 172.21.15.47:0/1933959576' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi047-82378-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:34.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:34.143919+0000 mon.a (mon.0) 1776 : cluster [DBG] osdmap e262: 8 total, 8 up, 8 in 2024-02-23T00:45:34.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:34.148035+0000 mon.c (mon.2) 178 : audit [INF] from='client.? 172.21.15.47:0/2348961114' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi047-82386-45"}]: dispatch 2024-02-23T00:45:34.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:34.584212+0000 mon.a (mon.0) 1777 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi047-82386-45"}]: dispatch 2024-02-23T00:45:34.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:34.584522+0000 mon.a (mon.0) 1778 : audit [INF] from='client.? 172.21.15.47:0/1933959576' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi047-82378-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi047-82378-36"}]: dispatch 2024-02-23T00:45:34.949 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:33.242196+0000 mgr.y (mgr.14517) 276 : cluster [DBG] pgmap v367: 250 pgs: 13 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 233 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-23T00:45:34.949 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:33.836091+0000 mon.b (mon.1) 321 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:34.949 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:34.143731+0000 mon.a (mon.0) 1774 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi047-82386-45"}]': finished 2024-02-23T00:45:34.949 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:34.143827+0000 mon.a (mon.0) 1775 : audit [INF] from='client.? 172.21.15.47:0/1933959576' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi047-82378-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:34.949 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:34.143919+0000 mon.a (mon.0) 1776 : cluster [DBG] osdmap e262: 8 total, 8 up, 8 in 2024-02-23T00:45:34.949 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:34.148035+0000 mon.c (mon.2) 178 : audit [INF] from='client.? 172.21.15.47:0/2348961114' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi047-82386-45"}]: dispatch 2024-02-23T00:45:34.949 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:34.584212+0000 mon.a (mon.0) 1777 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi047-82386-45"}]: dispatch 2024-02-23T00:45:34.949 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:34.584522+0000 mon.a (mon.0) 1778 : audit [INF] from='client.? 172.21.15.47:0/1933959576' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi047-82378-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi047-82378-36"}]: dispatch 2024-02-23T00:45:34.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:33.242196+0000 mgr.y (mgr.14517) 276 : cluster [DBG] pgmap v367: 250 pgs: 13 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 233 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-23T00:45:34.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:33.836091+0000 mon.b (mon.1) 321 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:34.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:34.143731+0000 mon.a (mon.0) 1774 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi047-82386-45"}]': finished 2024-02-23T00:45:34.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:34.143827+0000 mon.a (mon.0) 1775 : audit [INF] from='client.? 172.21.15.47:0/1933959576' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi047-82378-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:34.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:34.143919+0000 mon.a (mon.0) 1776 : cluster [DBG] osdmap e262: 8 total, 8 up, 8 in 2024-02-23T00:45:34.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:34.148035+0000 mon.c (mon.2) 178 : audit [INF] from='client.? 172.21.15.47:0/2348961114' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi047-82386-45"}]: dispatch 2024-02-23T00:45:34.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:34.584212+0000 mon.a (mon.0) 1777 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi047-82386-45"}]: dispatch 2024-02-23T00:45:34.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:34.584522+0000 mon.a (mon.0) 1778 : audit [INF] from='client.? 172.21.15.47:0/1933959576' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi047-82378-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi047-82378-36"}]: dispatch 2024-02-23T00:45:34.951 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:35.703 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:45:35.706 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:35.707 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:45:35.807 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:45:35.838 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:35.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:33.686596+0000 mgr.y (mgr.14517) 277 : audit [DBG] from='client.30585 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:45:35.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:34.837137+0000 mon.b (mon.1) 322 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:35.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:35.132798+0000 mon.a (mon.0) 1779 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi047-82386-45"}]': finished 2024-02-23T00:45:35.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:35.156366+0000 mon.c (mon.2) 179 : audit [INF] from='client.? 172.21.15.47:0/3947607643' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi047-82386-46"}]: dispatch 2024-02-23T00:45:35.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:35.571704+0000 mon.a (mon.0) 1780 : cluster [DBG] osdmap e263: 8 total, 8 up, 8 in 2024-02-23T00:45:35.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:35.577629+0000 mon.a (mon.0) 1781 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:35.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:35.577974+0000 mon.a (mon.0) 1782 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi047-82386-46"}]: dispatch 2024-02-23T00:45:35.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:35.578584+0000 mon.c (mon.2) 180 : audit [INF] from='client.? 172.21.15.47:0/3947607643' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi047-82386-46"}]: dispatch 2024-02-23T00:45:35.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:35.578919+0000 mon.a (mon.0) 1783 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi047-82386-46"}]: dispatch 2024-02-23T00:45:35.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:35.579408+0000 mon.c (mon.2) 181 : audit [INF] from='client.? 172.21.15.47:0/3947607643' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi047-82386-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:35.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:35.579707+0000 mon.a (mon.0) 1784 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi047-82386-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:36.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:33.686596+0000 mgr.y (mgr.14517) 277 : audit [DBG] from='client.30585 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:45:36.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:34.837137+0000 mon.b (mon.1) 322 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:36.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:35.132798+0000 mon.a (mon.0) 1779 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi047-82386-45"}]': finished 2024-02-23T00:45:36.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:35.156366+0000 mon.c (mon.2) 179 : audit [INF] from='client.? 172.21.15.47:0/3947607643' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi047-82386-46"}]: dispatch 2024-02-23T00:45:36.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:35.571704+0000 mon.a (mon.0) 1780 : cluster [DBG] osdmap e263: 8 total, 8 up, 8 in 2024-02-23T00:45:36.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:35.577629+0000 mon.a (mon.0) 1781 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:36.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:35.577974+0000 mon.a (mon.0) 1782 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi047-82386-46"}]: dispatch 2024-02-23T00:45:36.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:35.578584+0000 mon.c (mon.2) 180 : audit [INF] from='client.? 172.21.15.47:0/3947607643' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi047-82386-46"}]: dispatch 2024-02-23T00:45:36.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:35.578919+0000 mon.a (mon.0) 1783 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi047-82386-46"}]: dispatch 2024-02-23T00:45:36.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:35.579408+0000 mon.c (mon.2) 181 : audit [INF] from='client.? 172.21.15.47:0/3947607643' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi047-82386-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:36.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:35.579707+0000 mon.a (mon.0) 1784 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi047-82386-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:36.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:33.686596+0000 mgr.y (mgr.14517) 277 : audit [DBG] from='client.30585 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:45:36.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:34.837137+0000 mon.b (mon.1) 322 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:36.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:35.132798+0000 mon.a (mon.0) 1779 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi047-82386-45"}]': finished 2024-02-23T00:45:36.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:35.156366+0000 mon.c (mon.2) 179 : audit [INF] from='client.? 172.21.15.47:0/3947607643' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi047-82386-46"}]: dispatch 2024-02-23T00:45:36.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:35.571704+0000 mon.a (mon.0) 1780 : cluster [DBG] osdmap e263: 8 total, 8 up, 8 in 2024-02-23T00:45:36.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:35.577629+0000 mon.a (mon.0) 1781 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:36.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:35.577974+0000 mon.a (mon.0) 1782 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi047-82386-46"}]: dispatch 2024-02-23T00:45:36.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:35.578584+0000 mon.c (mon.2) 180 : audit [INF] from='client.? 172.21.15.47:0/3947607643' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi047-82386-46"}]: dispatch 2024-02-23T00:45:36.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:35.578919+0000 mon.a (mon.0) 1783 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi047-82386-46"}]: dispatch 2024-02-23T00:45:36.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:35.579408+0000 mon.c (mon.2) 181 : audit [INF] from='client.? 172.21.15.47:0/3947607643' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi047-82386-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:36.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:35.579707+0000 mon.a (mon.0) 1784 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi047-82386-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:36.339 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:45:36.360 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-02-23T00:45:33.241957+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":9043,"num_objects":359,"num_object_clones":1,"num_object_copies":1077,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":359,"num_whiteouts":0,"num_read":1574,"num_read_kb":1200,"num_write":1265,"num_write_kb":638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":947,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2620,"ondisk_log_size":2620,"up":750,"acting":750,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":771,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2497852,"kb_used_data":33980,"kb_used_omap":0,"kb_used_meta":2463744,"kb_avail":747463364,"statfs":{"total":767960285184,"available":765402484736,"internally_reserved":0,"allocated":34795520,"data_stored":20509959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2522873856},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-32,"num_objects":-5,"num_object_clones":-4,"num_object_copies":-15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5,"num_whiteouts":-1,"num_read":9,"num_read_kb":7,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000837"},"pg_stats":[{"pgid":"164.1b","version":"0'0","reported_seq":19,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.140888+0000","last_change":"2024-02-23T00:45:31.140888+0000","last_active":"2024-02-23T00:45:31.140888+0000","last_peered":"2024-02-23T00:45:31.140888+0000","last_clean":"2024-02-23T00:45:31.140888+0000","last_became_active":"2024-02-23T00:45:26.139003+0000","last_became_peered":"2024-02-23T00:45:26.139003+0000","last_unstale":"2024-02-23T00:45:31.140888+0000","last_undegraded":"2024-02-23T00:45:31.140888+0000","last_fullsized":"2024-02-23T00:45:31.140888+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.122997+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,3],"acting":[1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"11","length":"1"}]},{"pgid":"164.1a","version":"0'0","reported_seq":18,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.145091+0000","last_change":"2024-02-23T00:45:31.145091+0000","last_active":"2024-02-23T00:45:31.145091+0000","last_peered":"2024-02-23T00:45:31.145091+0000","last_clean":"2024-02-23T00:45:31.145091+0000","last_became_active":"2024-02-23T00:45:26.154401+0000","last_became_peered":"2024-02-23T00:45:26.154401+0000","last_unstale":"2024-02-23T00:45:31.145091+0000","last_undegraded":"2024-02-23T00:45:31.145091+0000","last_fullsized":"2024-02-23T00:45:31.145091+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.122997+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"11","length":"1"}]},{"pgid":"164.19","version":"0'0","reported_seq":19,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.136385+0000","last_change":"2024-02-23T00:45:31.136385+0000","last_active":"2024-02-23T00:45:31.136385+0000","last_peered":"2024-02-23T00:45:31.136385+0000","last_clean":"2024-02-23T00:45:31.136385+0000","last_became_active":"2024-02-23T00:45:26.139018+0000","last_became_peered":"2024-02-23T00:45:26.139018+0000","last_unstale":"2024-02-23T00:45:31.136385+0000","last_undegraded":"2024-02-23T00:45:31.136385+0000","last_fullsized":"2024-02-23T00:45:31.136385+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.122997+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"11","length":"1"}]},{"pgid":"164.18","version":"0'0","reported_seq":19,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.141369+0000","last_change":"2024-02-23T00:45:31.141369+0000","last_active":"2024-02-23T00:45:31.141369+0000","last_peered":"2024-02-23T00:45:31.141369+0000","last_clean":"2024-02-23T00:45:31.141369+0000","last_became_active":"2024-02-23T00:45:26.148664+0000","last_became_peered":"2024-02-23T00:45:26.148664+0000","last_unstale":"2024-02-23T00:45:31.141369+0000","last_undegraded":"2024-02-23T00:45:31.141369+0000","last_fullsized":"2024-02-23T00:45:31.141369+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.122997+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"11","length":"1"}]},{"pgid":"164.1f","version":"0'0","reported_seq":16,"reported_epoch":259,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-23T00:45:31.148245+0000","last_change":"2024-02-23T00:45:31.148230+0000","last_active":"2024-02-23T00:45:31.148245+0000","last_peered":"2024-02-23T00:45:31.148245+0000","last_clean":"2024-02-23T00:45:31.148245+0000","last_became_active":"2024-02-23T00:45:26.143962+0000","last_became_peered":"2024-02-23T00:45:26.143962+0000","last_unstale":"2024-02-23T00:45:31.148245+0000","last_undegraded":"2024-02-23T00:45:31.148245+0000","last_fullsized":"2024-02-23T00:45:31.148245+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.122997+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"164.1e","version":"0'0","reported_seq":18,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.139075+0000","last_change":"2024-02-23T00:45:31.139551+0000","last_active":"2024-02-23T00:45:31.139075+0000","last_peered":"2024-02-23T00:45:31.139075+0000","last_clean":"2024-02-23T00:45:31.139075+0000","last_became_active":"2024-02-23T00:45:26.152063+0000","last_became_peered":"2024-02-23T00:45:26.152063+0000","last_unstale":"2024-02-23T00:45:31.139075+0000","last_undegraded":"2024-02-23T00:45:31.139075+0000","last_fullsized":"2024-02-23T00:45:31.139075+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.122997+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"11","length":"1"}]},{"pgid":"164.1d","version":"0'0","reported_seq":18,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.144827+0000","last_change":"2024-02-23T00:45:31.144827+0000","last_active":"2024-02-23T00:45:31.144827+0000","last_peered":"2024-02-23T00:45:31.144827+0000","last_clean":"2024-02-23T00:45:31.144827+0000","last_became_active":"2024-02-23T00:45:26.149899+0000","last_became_peered":"2024-02-23T00:45:26.149899+0000","last_unstale":"2024-02-23T00:45:31.144827+0000","last_undegraded":"2024-02-23T00:45:31.144827+0000","last_fullsized":"2024-02-23T00:45:31.144827+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.122997+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"11","length":"1"}]},{"pgid":"164.1c","version":"0'0","reported_seq":19,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.142663+0000","last_change":"2024-02-23T00:45:31.142663+0000","last_active":"2024-02-23T00:45:31.142663+0000","last_peered":"2024-02-23T00:45:31.142663+0000","last_clean":"2024-02-23T00:45:31.142663+0000","last_became_active":"2024-02-23T00:45:26.142347+0000","last_became_peered":"2024-02-23T00:45:26.142347+0000","last_unstale":"2024-02-23T00:45:31.142663+0000","last_undegraded":"2024-02-23T00:45:31.142663+0000","last_fullsized":"2024-02-23T00:45:31.142663+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.122997+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"11","length":"1"}]},{"pgid":"164.13","version":"0'0","reported_seq":18,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.142828+0000","last_change":"2024-02-23T00:45:31.142828+0000","last_active":"2024-02-23T00:45:31.142828+0000","last_peered":"2024-02-23T00:45:31.142828+0000","last_clean":"2024-02-23T00:45:31.142828+0000","last_became_active":"2024-02-23T00:45:26.137794+0000","last_became_peered":"2024-02-23T00:45:26.137794+0000","last_unstale":"2024-02-23T00:45:31.142828+0000","last_undegraded":"2024-02-23T00:45:31.142828+0000","last_fullsized":"2024-02-23T00:45:31.142828+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.122997+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"11","length":"1"}]},{"pgid":"164.12","version":"0'0","reported_seq":18,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.145743+0000","last_change":"2024-02-23T00:45:31.145743+0000","last_active":"2024-02-23T00:45:31.145743+0000","last_peered":"2024-02-23T00:45:31.145743+0000","last_clean":"2024-02-23T00:45:31.145743+0000","last_became_active":"2024-02-23T00:45:26.153776+0000","last_became_peered":"2024-02-23T00:45:26.153776+0000","last_unstale":"2024-02-23T00:45:31.145743+0000","last_undegraded":"2024-02-23T00:45:31.145743+0000","last_fullsized":"2024-02-23T00:45:31.145743+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.122997+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"11","length":"1"}]},{"pgid":"164.11","version":"0'0","reported_seq":19,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.142811+0000","last_change":"2024-02-23T00:45:31.142811+0000","last_active":"2024-02-23T00:45:31.142811+0000","last_peered":"2024-02-23T00:45:31.142811+0000","last_clean":"2024-02-23T00:45:31.142811+0000","last_became_active":"2024-02-23T00:45:26.149146+0000","last_became_peered":"2024-02-23T00:45:26.149146+0000","last_unstale":"2024-02-23T00:45:31.142811+0000","last_undegraded":"2024-02-23T00:45:31.142811+0000","last_fullsized":"2024-02-23T00:45:31.142811+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.122997+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"11","length":"1"}]},{"pgid":"164.10","version":"0'0","reported_seq":19,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.140502+0000","last_change":"2024-02-23T00:45:31.140502+0000","last_active":"2024-02-23T00:45:31.140502+0000","last_peered":"2024-02-23T00:45:31.140502+0000","last_clean":"2024-02-23T00:45:31.140502+0000","last_became_active":"2024-02-23T00:45:26.148744+0000","last_became_peered":"2024-02-23T00:45:26.148744+0000","last_unstale":"2024-02-23T00:45:31.140502+0000","last_undegraded":"2024-02-23T00:45:31.140502+0000","last_fullsized":"2024-02-23T00:45:31.140502+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.122997+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"11","length":"1"}]},{"pgid":"164.17","version":"0'0","reported_seq":18,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.147445+0000","last_change":"2024-02-23T00:45:31.147445+0000","last_active":"2024-02-23T00:45:31.147445+0000","last_peered":"2024-02-23T00:45:31.147445+0000","last_clean":"2024-02-23T00:45:31.147445+0000","last_became_active":"2024-02-23T00:45:26.143437+0000","last_became_peered":"2024-02-23T00:45:26.143437+0000","last_unstale":"2024-02-23T00:45:31.147445+0000","last_undegraded":"2024-02-23T00:45:31.147445+0000","last_fullsized":"2024-02-23T00:45:31.147445+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.122997+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"11","length":"1"}]},{"pgid":"164.16","version":"0'0","reported_seq":19,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.150408+0000","last_change":"2024-02-23T00:45:31.150408+0000","last_active":"2024-02-23T00:45:31.150408+0000","last_peered":"2024-02-23T00:45:31.150408+0000","last_clean":"2024-02-23T00:45:31.150408+0000","last_became_active":"2024-02-23T00:45:26.153531+0000","last_became_peered":"2024-02-23T00:45:26.153531+0000","last_unstale":"2024-02-23T00:45:31.150408+0000","last_undegraded":"2024-02-23T00:45:31.150408+0000","last_fullsized":"2024-02-23T00:45:31.150408+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.122997+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"11","length":"1"}]},{"pgid":"164.15","version":"0'0","reported_seq":19,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.142330+0000","last_change":"2024-02-23T00:45:31.142330+0000","last_active":"2024-02-23T00:45:31.142330+0000","last_peered":"2024-02-23T00:45:31.142330+0000","last_clean":"2024-02-23T00:45:31.142330+0000","last_became_active":"2024-02-23T00:45:26.153546+0000","last_became_peered":"2024-02-23T00:45:26.153546+0000","last_unstale":"2024-02-23T00:45:31.142330+0000","last_undegraded":"2024-02-23T00:45:31.142330+0000","last_fullsized":"2024-02-23T00:45:31.142330+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.122997+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"11","length":"1"}]},{"pgid":"164.14","version":"0'0","reported_seq":14,"reported_epoch":259,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-23T00:45:31.137141+0000","last_change":"2024-02-23T00:45:31.137141+0000","last_active":"2024-02-23T00:45:31.137141+0000","last_peered":"2024-02-23T00:45:31.137141+0000","last_clean":"2024-02-23T00:45:31.137141+0000","last_became_active":"2024-02-23T00:45:26.150289+0000","last_became_peered":"2024-02-23T00:45:26.150289+0000","last_unstale":"2024-02-23T00:45:31.137141+0000","last_undegraded":"2024-02-23T00:45:31.137141+0000","last_fullsized":"2024-02-23T00:45:31.137141+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.122997+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"164.b","version":"0'0","reported_seq":19,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.141886+0000","last_change":"2024-02-23T00:45:31.141886+0000","last_active":"2024-02-23T00:45:31.141886+0000","last_peered":"2024-02-23T00:45:31.141886+0000","last_clean":"2024-02-23T00:45:31.141886+0000","last_became_active":"2024-02-23T00:45:26.151255+0000","last_became_peered":"2024-02-23T00:45:26.151255+0000","last_unstale":"2024-02-23T00:45:31.141886+0000","last_undegraded":"2024-02-23T00:45:31.141886+0000","last_fullsized":"2024-02-23T00:45:31.141886+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.122997+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"11","length":"1"}]},{"pgid":"164.a","version":"0'0","reported_seq":14,"reported_epoch":259,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-23T00:45:31.135870+0000","last_change":"2024-02-23T00:45:31.135870+0000","last_active":"2024-02-23T00:45:31.135870+0000","last_peered":"2024-02-23T00:45:31.135870+0000","last_clean":"2024-02-23T00:45:31.135870+0000","last_became_active":"2024-02-23T00:45:26.151281+0000","last_became_peered":"2024-02-23T00:45:26.151281+0000","last_unstale":"2024-02-23T00:45:31.135870+0000","last_undegraded":"2024-02-23T00:45:31.135870+0000","last_fullsized":"2024-02-23T00:45:31.135870+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.122997+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"164.9","version":"0'0","reported_seq":18,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.138288+0000","last_change":"2024-02-23T00:45:31.138611+0000","last_active":"2024-02-23T00:45:31.138288+0000","last_peered":"2024-02-23T00:45:31.138288+0000","last_clean":"2024-02-23T00:45:31.138288+0000","last_became_active":"2024-02-23T00:45:26.151415+0000","last_became_peered":"2024-02-23T00:45:26.151415+0000","last_unstale":"2024-02-23T00:45:31.138288+0000","last_undegraded":"2024-02-23T00:45:31.138288+0000","last_fullsized":"2024-02-23T00:45:31.138288+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.122997+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"11","length":"1"}]},{"pgid":"164.8","version":"0'0","reported_seq":18,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.137413+0000","last_change":"2024-02-23T00:45:31.138615+0000","last_active":"2024-02-23T00:45:31.137413+0000","last_peered":"2024-02-23T00:45:31.137413+0000","last_clean":"2024-02-23T00:45:31.137413+0000","last_became_active":"2024-02-23T00:45:26.150523+0000","last_became_peered":"2024-02-23T00:45:26.150523+0000","last_unstale":"2024-02-23T00:45:31.137413+0000","last_undegraded":"2024-02-23T00:45:31.137413+0000","last_fullsized":"2024-02-23T00:45:31.137413+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.122997+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"11","length":"1"}]},{"pgid":"164.f","version":"0'0","reported_seq":18,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.139024+0000","last_change":"2024-02-23T00:45:31.140381+0000","last_active":"2024-02-23T00:45:31.139024+0000","last_peered":"2024-02-23T00:45:31.139024+0000","last_clean":"2024-02-23T00:45:31.139024+0000","last_became_active":"2024-02-23T00:45:26.154759+0000","last_became_peered":"2024-02-23T00:45:26.154759+0000","last_unstale":"2024-02-23T00:45:31.139024+0000","last_undegraded":"2024-02-23T00:45:31.139024+0000","last_fullsized":"2024-02-23T00:45:31.139024+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.122997+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"11","length":"1"}]},{"pgid":"164.e","version":"0'0","reported_seq":19,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.150712+0000","last_change":"2024-02-23T00:45:31.150712+0000","last_active":"2024-02-23T00:45:31.150712+0000","last_peered":"2024-02-23T00:45:31.150712+0000","last_clean":"2024-02-23T00:45:31.150712+0000","last_became_active":"2024-02-23T00:45:26.157461+0000","last_became_peered":"2024-02-23T00:45:26.157461+0000","last_unstale":"2024-02-23T00:45:31.150712+0000","last_undegraded":"2024-02-23T00:45:31.150712+0000","last_fullsized":"2024-02-23T00:45:31.150712+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.122997+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"11","length":"1"}]},{"pgid":"164.d","version":"0'0","reported_seq":18,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.140827+0000","last_change":"2024-02-23T00:45:31.141394+0000","last_active":"2024-02-23T00:45:31.140827+0000","last_peered":"2024-02-23T00:45:31.140827+0000","last_clean":"2024-02-23T00:45:31.140827+0000","last_became_active":"2024-02-23T00:45:26.153519+0000","last_became_peered":"2024-02-23T00:45:26.153519+0000","last_unstale":"2024-02-23T00:45:31.140827+0000","last_undegraded":"2024-02-23T00:45:31.140827+0000","last_fullsized":"2024-02-23T00:45:31.140827+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.122997+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"11","length":"1"}]},{"pgid":"164.c","version":"0'0","reported_seq":19,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.150719+0000","last_change":"2024-02-23T00:45:31.150719+0000","last_active":"2024-02-23T00:45:31.150719+0000","last_peered":"2024-02-23T00:45:31.150719+0000","last_clean":"2024-02-23T00:45:31.150719+0000","last_became_active":"2024-02-23T00:45:26.151939+0000","last_became_peered":"2024-02-23T00:45:26.151939+0000","last_unstale":"2024-02-23T00:45:31.150719+0000","last_undegraded":"2024-02-23T00:45:31.150719+0000","last_fullsized":"2024-02-23T00:45:31.150719+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.122997+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"11","length":"1"}]},{"pgid":"164.3","version":"0'0","reported_seq":16,"reported_epoch":259,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-23T00:45:31.146834+0000","last_change":"2024-02-23T00:45:31.146820+0000","last_active":"2024-02-23T00:45:31.146834+0000","last_peered":"2024-02-23T00:45:31.146834+0000","last_clean":"2024-02-23T00:45:31.146834+0000","last_became_active":"2024-02-23T00:45:26.154394+0000","last_became_peered":"2024-02-23T00:45:26.154394+0000","last_unstale":"2024-02-23T00:45:31.146834+0000","last_undegraded":"2024-02-23T00:45:31.146834+0000","last_fullsized":"2024-02-23T00:45:31.146834+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.122997+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"164.2","version":"0'0","reported_seq":18,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.146369+0000","last_change":"2024-02-23T00:45:31.146369+0000","last_active":"2024-02-23T00:45:31.146369+0000","last_peered":"2024-02-23T00:45:31.146369+0000","last_clean":"2024-02-23T00:45:31.146369+0000","last_became_active":"2024-02-23T00:45:26.143830+0000","last_became_peered":"2024-02-23T00:45:26.143830+0000","last_unstale":"2024-02-23T00:45:31.146369+0000","last_undegraded":"2024-02-23T00:45:31.146369+0000","last_fullsized":"2024-02-23T00:45:31.146369+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.122997+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"11","length":"1"}]},{"pgid":"164.1","version":"0'0","reported_seq":19,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.136918+0000","last_change":"2024-02-23T00:45:31.136918+0000","last_active":"2024-02-23T00:45:31.136918+0000","last_peered":"2024-02-23T00:45:31.136918+0000","last_clean":"2024-02-23T00:45:31.136918+0000","last_became_active":"2024-02-23T00:45:26.149574+0000","last_became_peered":"2024-02-23T00:45:26.149574+0000","last_unstale":"2024-02-23T00:45:31.136918+0000","last_undegraded":"2024-02-23T00:45:31.136918+0000","last_fullsized":"2024-02-23T00:45:31.136918+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.122997+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"11","length":"1"}]},{"pgid":"164.0","version":"259'4","reported_seq":26,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.161531+0000","last_change":"2024-02-23T00:45:31.145481+0000","last_active":"2024-02-23T00:45:31.161531+0000","last_peered":"2024-02-23T00:45:31.161531+0000","last_clean":"2024-02-23T00:45:31.161531+0000","last_became_active":"2024-02-23T00:45:26.140748+0000","last_became_peered":"2024-02-23T00:45:26.140748+0000","last_unstale":"2024-02-23T00:45:31.161531+0000","last_undegraded":"2024-02-23T00:45:31.161531+0000","last_fullsized":"2024-02-23T00:45:31.161531+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.122997+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,3],"acting":[2,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"11","length":"1"}]},{"pgid":"164.7","version":"0'0","reported_seq":18,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.140435+0000","last_change":"2024-02-23T00:45:31.140757+0000","last_active":"2024-02-23T00:45:31.140435+0000","last_peered":"2024-02-23T00:45:31.140435+0000","last_clean":"2024-02-23T00:45:31.140435+0000","last_became_active":"2024-02-23T00:45:26.155192+0000","last_became_peered":"2024-02-23T00:45:26.155192+0000","last_unstale":"2024-02-23T00:45:31.140435+0000","last_undegraded":"2024-02-23T00:45:31.140435+0000","last_fullsized":"2024-02-23T00:45:31.140435+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.122997+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"11","length":"1"}]},{"pgid":"164.6","version":"0'0","reported_seq":19,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.143592+0000","last_change":"2024-02-23T00:45:31.143592+0000","last_active":"2024-02-23T00:45:31.143592+0000","last_peered":"2024-02-23T00:45:31.143592+0000","last_clean":"2024-02-23T00:45:31.143592+0000","last_became_active":"2024-02-23T00:45:26.153753+0000","last_became_peered":"2024-02-23T00:45:26.153753+0000","last_unstale":"2024-02-23T00:45:31.143592+0000","last_undegraded":"2024-02-23T00:45:31.143592+0000","last_fullsized":"2024-02-23T00:45:31.143592+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.122997+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"11","length":"1"}]},{"pgid":"164.5","version":"0'0","reported_seq":18,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.147075+0000","last_change":"2024-02-23T00:45:31.147075+0000","last_active":"2024-02-23T00:45:31.147075+0000","last_peered":"2024-02-23T00:45:31.147075+0000","last_clean":"2024-02-23T00:45:31.147075+0000","last_became_active":"2024-02-23T00:45:26.149457+0000","last_became_peered":"2024-02-23T00:45:26.149457+0000","last_unstale":"2024-02-23T00:45:31.147075+0000","last_undegraded":"2024-02-23T00:45:31.147075+0000","last_fullsized":"2024-02-23T00:45:31.147075+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.122997+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"11","length":"1"}]},{"pgid":"164.4","version":"0'0","reported_seq":18,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.146463+0000","last_change":"2024-02-23T00:45:31.146463+0000","last_active":"2024-02-23T00:45:31.146463+0000","last_peered":"2024-02-23T00:45:31.146463+0000","last_clean":"2024-02-23T00:45:31.146463+0000","last_became_active":"2024-02-23T00:45:26.150899+0000","last_became_peered":"2024-02-23T00:45:26.150899+0000","last_unstale":"2024-02-23T00:45:31.146463+0000","last_undegraded":"2024-02-23T00:45:31.146463+0000","last_fullsized":"2024-02-23T00:45:31.146463+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:25.122997+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.122997+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"11","length":"1"}]},{"pgid":"62.0","version":"259'40","reported_seq":214,"reported_epoch":259,"state":"active+clean+snaptrim","last_fresh":"2024-02-23T00:45:31.135474+0000","last_change":"2024-02-23T00:45:31.135474+0000","last_active":"2024-02-23T00:45:31.135474+0000","last_peered":"2024-02-23T00:45:31.135474+0000","last_clean":"2024-02-23T00:45:31.135474+0000","last_became_active":"2024-02-23T00:43:17.855316+0000","last_became_peered":"2024-02-23T00:43:17.855316+0000","last_unstale":"2024-02-23T00:45:31.135474+0000","last_undegraded":"2024-02-23T00:45:31.135474+0000","last_fullsized":"2024-02-23T00:45:31.135474+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":31,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":1,"num_write":13,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.1","version":"230'87","reported_seq":256,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.135186+0000","last_change":"2024-02-23T00:45:31.135186+0000","last_active":"2024-02-23T00:45:31.135186+0000","last_peered":"2024-02-23T00:45:31.135186+0000","last_clean":"2024-02-23T00:45:31.135186+0000","last_became_active":"2024-02-23T00:43:17.851866+0000","last_became_peered":"2024-02-23T00:43:17.851866+0000","last_unstale":"2024-02-23T00:45:31.135186+0000","last_undegraded":"2024-02-23T00:45:31.135186+0000","last_fullsized":"2024-02-23T00:45:31.135186+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":87,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":63,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.2","version":"229'68","reported_seq":236,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.147100+0000","last_change":"2024-02-23T00:45:31.147100+0000","last_active":"2024-02-23T00:45:31.147100+0000","last_peered":"2024-02-23T00:45:31.147100+0000","last_clean":"2024-02-23T00:45:31.147100+0000","last_became_active":"2024-02-23T00:43:17.804792+0000","last_became_peered":"2024-02-23T00:43:17.804792+0000","last_unstale":"2024-02-23T00:45:31.147100+0000","last_undegraded":"2024-02-23T00:45:31.147100+0000","last_fullsized":"2024-02-23T00:45:31.147100+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.3","version":"246'20","reported_seq":199,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.144447+0000","last_change":"2024-02-23T00:45:31.144447+0000","last_active":"2024-02-23T00:45:31.144447+0000","last_peered":"2024-02-23T00:45:31.144447+0000","last_clean":"2024-02-23T00:45:31.144447+0000","last_became_active":"2024-02-23T00:43:17.834686+0000","last_became_peered":"2024-02-23T00:43:17.834686+0000","last_unstale":"2024-02-23T00:45:31.144447+0000","last_undegraded":"2024-02-23T00:45:31.144447+0000","last_fullsized":"2024-02-23T00:45:31.144447+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.4","version":"207'17","reported_seq":192,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.150414+0000","last_change":"2024-02-23T00:45:31.150414+0000","last_active":"2024-02-23T00:45:31.150414+0000","last_peered":"2024-02-23T00:45:31.150414+0000","last_clean":"2024-02-23T00:45:31.150414+0000","last_became_active":"2024-02-23T00:43:17.810343+0000","last_became_peered":"2024-02-23T00:43:17.810343+0000","last_unstale":"2024-02-23T00:45:31.150414+0000","last_undegraded":"2024-02-23T00:45:31.150414+0000","last_fullsized":"2024-02-23T00:45:31.150414+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.5","version":"237'51","reported_seq":223,"reported_epoch":259,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-23T00:45:31.145594+0000","last_change":"2024-02-23T00:45:31.145576+0000","last_active":"2024-02-23T00:45:31.145594+0000","last_peered":"2024-02-23T00:45:31.145594+0000","last_clean":"2024-02-23T00:45:31.145594+0000","last_became_active":"2024-02-23T00:43:17.855339+0000","last_became_peered":"2024-02-23T00:43:17.855339+0000","last_unstale":"2024-02-23T00:45:31.145594+0000","last_undegraded":"2024-02-23T00:45:31.145594+0000","last_fullsized":"2024-02-23T00:45:31.145594+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.6","version":"247'51","reported_seq":223,"reported_epoch":259,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-23T00:45:31.140520+0000","last_change":"2024-02-23T00:45:31.140520+0000","last_active":"2024-02-23T00:45:31.140520+0000","last_peered":"2024-02-23T00:45:31.140520+0000","last_clean":"2024-02-23T00:45:31.140520+0000","last_became_active":"2024-02-23T00:43:17.838838+0000","last_became_peered":"2024-02-23T00:43:17.838838+0000","last_unstale":"2024-02-23T00:45:31.140520+0000","last_undegraded":"2024-02-23T00:45:31.140520+0000","last_fullsized":"2024-02-23T00:45:31.140520+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.7","version":"215'68","reported_seq":234,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.146762+0000","last_change":"2024-02-23T00:45:31.146762+0000","last_active":"2024-02-23T00:45:31.146762+0000","last_peered":"2024-02-23T00:45:31.146762+0000","last_clean":"2024-02-23T00:45:31.146762+0000","last_became_active":"2024-02-23T00:43:17.846548+0000","last_became_peered":"2024-02-23T00:43:17.846548+0000","last_unstale":"2024-02-23T00:45:31.146762+0000","last_undegraded":"2024-02-23T00:45:31.146762+0000","last_fullsized":"2024-02-23T00:45:31.146762+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'68","last_scrub_stamp":"2024-02-23T00:45:25.351414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.351414+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.8","version":"259'90","reported_seq":250,"reported_epoch":259,"state":"active+clean+snaptrim","last_fresh":"2024-02-23T00:45:31.135405+0000","last_change":"2024-02-23T00:45:31.135405+0000","last_active":"2024-02-23T00:45:31.135405+0000","last_peered":"2024-02-23T00:45:31.135405+0000","last_clean":"2024-02-23T00:45:31.135405+0000","last_became_active":"2024-02-23T00:43:17.842613+0000","last_became_peered":"2024-02-23T00:43:17.842613+0000","last_unstale":"2024-02-23T00:45:31.135405+0000","last_undegraded":"2024-02-23T00:45:31.135405+0000","last_fullsized":"2024-02-23T00:45:31.135405+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":90,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.9","version":"259'39","reported_seq":212,"reported_epoch":259,"state":"active+clean+snaptrim","last_fresh":"2024-02-23T00:45:31.151113+0000","last_change":"2024-02-23T00:45:31.146625+0000","last_active":"2024-02-23T00:45:31.151113+0000","last_peered":"2024-02-23T00:45:31.151113+0000","last_clean":"2024-02-23T00:45:31.151113+0000","last_became_active":"2024-02-23T00:43:17.834297+0000","last_became_peered":"2024-02-23T00:43:17.834297+0000","last_unstale":"2024-02-23T00:45:31.151113+0000","last_undegraded":"2024-02-23T00:45:31.151113+0000","last_fullsized":"2024-02-23T00:45:31.151113+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":27,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":12,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.a","version":"253'90","reported_seq":250,"reported_epoch":259,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-23T00:45:31.136536+0000","last_change":"2024-02-23T00:45:31.136536+0000","last_active":"2024-02-23T00:45:31.136536+0000","last_peered":"2024-02-23T00:45:31.136536+0000","last_clean":"2024-02-23T00:45:31.136536+0000","last_became_active":"2024-02-23T00:43:17.839142+0000","last_became_peered":"2024-02-23T00:43:17.839142+0000","last_unstale":"2024-02-23T00:45:31.136536+0000","last_undegraded":"2024-02-23T00:45:31.136536+0000","last_fullsized":"2024-02-23T00:45:31.136536+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":90,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.b","version":"243'68","reported_seq":228,"reported_epoch":259,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-23T00:45:31.135876+0000","last_change":"2024-02-23T00:45:31.135876+0000","last_active":"2024-02-23T00:45:31.135876+0000","last_peered":"2024-02-23T00:45:31.135876+0000","last_clean":"2024-02-23T00:45:31.135876+0000","last_became_active":"2024-02-23T00:43:17.855321+0000","last_became_peered":"2024-02-23T00:43:17.855321+0000","last_unstale":"2024-02-23T00:45:31.135876+0000","last_undegraded":"2024-02-23T00:45:31.135876+0000","last_fullsized":"2024-02-23T00:45:31.135876+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.c","version":"203'34","reported_seq":208,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.142986+0000","last_change":"2024-02-23T00:45:31.142986+0000","last_active":"2024-02-23T00:45:31.142986+0000","last_peered":"2024-02-23T00:45:31.142986+0000","last_clean":"2024-02-23T00:45:31.142986+0000","last_became_active":"2024-02-23T00:43:17.829044+0000","last_became_peered":"2024-02-23T00:43:17.829044+0000","last_unstale":"2024-02-23T00:45:31.142986+0000","last_undegraded":"2024-02-23T00:45:31.142986+0000","last_fullsized":"2024-02-23T00:45:31.142986+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.d","version":"259'56","reported_seq":221,"reported_epoch":259,"state":"active+clean+snaptrim","last_fresh":"2024-02-23T00:45:31.149879+0000","last_change":"2024-02-23T00:45:31.146849+0000","last_active":"2024-02-23T00:45:31.149879+0000","last_peered":"2024-02-23T00:45:31.149879+0000","last_clean":"2024-02-23T00:45:31.149879+0000","last_became_active":"2024-02-23T00:43:17.841614+0000","last_became_peered":"2024-02-23T00:43:17.841614+0000","last_unstale":"2024-02-23T00:45:31.149879+0000","last_undegraded":"2024-02-23T00:45:31.149879+0000","last_fullsized":"2024-02-23T00:45:31.149879+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":56,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":17,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.e","version":"211'51","reported_seq":219,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.149105+0000","last_change":"2024-02-23T00:45:31.149105+0000","last_active":"2024-02-23T00:45:31.149105+0000","last_peered":"2024-02-23T00:45:31.149105+0000","last_clean":"2024-02-23T00:45:31.149105+0000","last_became_active":"2024-02-23T00:43:17.845956+0000","last_became_peered":"2024-02-23T00:43:17.845956+0000","last_unstale":"2024-02-23T00:45:31.149105+0000","last_undegraded":"2024-02-23T00:45:31.149105+0000","last_fullsized":"2024-02-23T00:45:31.149105+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.f","version":"241'136","reported_seq":287,"reported_epoch":259,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-23T00:45:31.145887+0000","last_change":"2024-02-23T00:45:31.145866+0000","last_active":"2024-02-23T00:45:31.145887+0000","last_peered":"2024-02-23T00:45:31.145887+0000","last_clean":"2024-02-23T00:45:31.145887+0000","last_became_active":"2024-02-23T00:43:17.855331+0000","last_became_peered":"2024-02-23T00:43:17.855331+0000","last_unstale":"2024-02-23T00:45:31.145887+0000","last_undegraded":"2024-02-23T00:45:31.145887+0000","last_fullsized":"2024-02-23T00:45:31.145887+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":136,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":88,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":40,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.10","version":"219'68","reported_seq":237,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.135638+0000","last_change":"2024-02-23T00:45:31.135638+0000","last_active":"2024-02-23T00:45:31.135638+0000","last_peered":"2024-02-23T00:45:31.135638+0000","last_clean":"2024-02-23T00:45:31.135638+0000","last_became_active":"2024-02-23T00:43:17.819865+0000","last_became_peered":"2024-02-23T00:43:17.819865+0000","last_unstale":"2024-02-23T00:45:31.135638+0000","last_undegraded":"2024-02-23T00:45:31.135638+0000","last_fullsized":"2024-02-23T00:45:31.135638+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.11","version":"215'51","reported_seq":217,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.134829+0000","last_change":"2024-02-23T00:45:31.134829+0000","last_active":"2024-02-23T00:45:31.134829+0000","last_peered":"2024-02-23T00:45:31.134829+0000","last_clean":"2024-02-23T00:45:31.134829+0000","last_became_active":"2024-02-23T00:43:17.852697+0000","last_became_peered":"2024-02-23T00:43:17.852697+0000","last_unstale":"2024-02-23T00:45:31.134829+0000","last_undegraded":"2024-02-23T00:45:31.134829+0000","last_fullsized":"2024-02-23T00:45:31.134829+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.12","version":"240'73","reported_seq":243,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.146190+0000","last_change":"2024-02-23T00:45:31.146190+0000","last_active":"2024-02-23T00:45:31.146190+0000","last_peered":"2024-02-23T00:45:31.146190+0000","last_clean":"2024-02-23T00:45:31.146190+0000","last_became_active":"2024-02-23T00:43:17.856118+0000","last_became_peered":"2024-02-23T00:43:17.856118+0000","last_unstale":"2024-02-23T00:45:31.146190+0000","last_undegraded":"2024-02-23T00:45:31.146190+0000","last_fullsized":"2024-02-23T00:45:31.146190+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":73,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":49,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.13","version":"205'34","reported_seq":209,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.148241+0000","last_change":"2024-02-23T00:45:31.148241+0000","last_active":"2024-02-23T00:45:31.148241+0000","last_peered":"2024-02-23T00:45:31.148241+0000","last_clean":"2024-02-23T00:45:31.148241+0000","last_became_active":"2024-02-23T00:43:17.857823+0000","last_became_peered":"2024-02-23T00:43:17.857823+0000","last_unstale":"2024-02-23T00:45:31.148241+0000","last_undegraded":"2024-02-23T00:45:31.148241+0000","last_fullsized":"2024-02-23T00:45:31.148241+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.14","version":"239'39","reported_seq":206,"reported_epoch":259,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-23T00:45:31.136935+0000","last_change":"2024-02-23T00:45:31.136935+0000","last_active":"2024-02-23T00:45:31.136935+0000","last_peered":"2024-02-23T00:45:31.136935+0000","last_clean":"2024-02-23T00:45:31.136935+0000","last_became_active":"2024-02-23T00:43:17.816138+0000","last_became_peered":"2024-02-23T00:43:17.816138+0000","last_unstale":"2024-02-23T00:45:31.136935+0000","last_undegraded":"2024-02-23T00:45:31.136935+0000","last_fullsized":"2024-02-23T00:45:31.136935+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":27,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":12,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.15","version":"201'17","reported_seq":195,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.143544+0000","last_change":"2024-02-23T00:45:31.143544+0000","last_active":"2024-02-23T00:45:31.143544+0000","last_peered":"2024-02-23T00:45:31.143544+0000","last_clean":"2024-02-23T00:45:31.143544+0000","last_became_active":"2024-02-23T00:43:17.843834+0000","last_became_peered":"2024-02-23T00:43:17.843834+0000","last_unstale":"2024-02-23T00:45:31.143544+0000","last_undegraded":"2024-02-23T00:45:31.143544+0000","last_fullsized":"2024-02-23T00:45:31.143544+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.16","version":"240'23","reported_seq":198,"reported_epoch":259,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-23T00:45:31.145700+0000","last_change":"2024-02-23T00:45:31.145677+0000","last_active":"2024-02-23T00:45:31.145700+0000","last_peered":"2024-02-23T00:45:31.145700+0000","last_clean":"2024-02-23T00:45:31.145700+0000","last_became_active":"2024-02-23T00:43:17.834291+0000","last_became_peered":"2024-02-23T00:43:17.834291+0000","last_unstale":"2024-02-23T00:45:31.145700+0000","last_undegraded":"2024-02-23T00:45:31.145700+0000","last_fullsized":"2024-02-23T00:45:31.145700+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":14,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.17","version":"229'57","reported_seq":234,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.142517+0000","last_change":"2024-02-23T00:45:31.142517+0000","last_active":"2024-02-23T00:45:31.142517+0000","last_peered":"2024-02-23T00:45:31.142517+0000","last_clean":"2024-02-23T00:45:31.142517+0000","last_became_active":"2024-02-23T00:43:17.843525+0000","last_became_peered":"2024-02-23T00:43:17.843525+0000","last_unstale":"2024-02-23T00:45:31.142517+0000","last_undegraded":"2024-02-23T00:45:31.142517+0000","last_fullsized":"2024-02-23T00:45:31.142517+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":57,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":18,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.19","version":"240'95","reported_seq":267,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.140310+0000","last_change":"2024-02-23T00:45:31.140310+0000","last_active":"2024-02-23T00:45:31.140310+0000","last_peered":"2024-02-23T00:45:31.140310+0000","last_clean":"2024-02-23T00:45:31.140310+0000","last_became_active":"2024-02-23T00:43:17.815830+0000","last_became_peered":"2024-02-23T00:43:17.815830+0000","last_unstale":"2024-02-23T00:45:31.140310+0000","last_undegraded":"2024-02-23T00:45:31.140310+0000","last_fullsized":"2024-02-23T00:45:31.140310+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":95,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":65,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":7,"num_read_kb":0,"num_write":29,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.18","version":"150'5","reported_seq":186,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.141554+0000","last_change":"2024-02-23T00:45:31.141554+0000","last_active":"2024-02-23T00:45:31.141554+0000","last_peered":"2024-02-23T00:45:31.141554+0000","last_clean":"2024-02-23T00:45:31.141554+0000","last_became_active":"2024-02-23T00:43:17.840351+0000","last_became_peered":"2024-02-23T00:43:17.840351+0000","last_unstale":"2024-02-23T00:45:31.141554+0000","last_undegraded":"2024-02-23T00:45:31.141554+0000","last_fullsized":"2024-02-23T00:45:31.141554+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.1b","version":"229'68","reported_seq":233,"reported_epoch":259,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-23T00:45:31.146529+0000","last_change":"2024-02-23T00:45:31.146510+0000","last_active":"2024-02-23T00:45:31.146529+0000","last_peered":"2024-02-23T00:45:31.146529+0000","last_clean":"2024-02-23T00:45:31.146529+0000","last_became_active":"2024-02-23T00:43:17.803043+0000","last_became_peered":"2024-02-23T00:43:17.803043+0000","last_unstale":"2024-02-23T00:45:31.146529+0000","last_undegraded":"2024-02-23T00:45:31.146529+0000","last_fullsized":"2024-02-23T00:45:31.146529+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.1a","version":"225'34","reported_seq":205,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.146687+0000","last_change":"2024-02-23T00:45:31.146687+0000","last_active":"2024-02-23T00:45:31.146687+0000","last_peered":"2024-02-23T00:45:31.146687+0000","last_clean":"2024-02-23T00:45:31.146687+0000","last_became_active":"2024-02-23T00:43:17.798285+0000","last_became_peered":"2024-02-23T00:43:17.798285+0000","last_unstale":"2024-02-23T00:45:31.146687+0000","last_undegraded":"2024-02-23T00:45:31.146687+0000","last_fullsized":"2024-02-23T00:45:31.146687+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.1d","version":"215'104","reported_seq":269,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.140512+0000","last_change":"2024-02-23T00:45:31.141111+0000","last_active":"2024-02-23T00:45:31.140512+0000","last_peered":"2024-02-23T00:45:31.140512+0000","last_clean":"2024-02-23T00:45:31.140512+0000","last_became_active":"2024-02-23T00:43:17.828194+0000","last_became_peered":"2024-02-23T00:43:17.828194+0000","last_unstale":"2024-02-23T00:45:31.140512+0000","last_undegraded":"2024-02-23T00:45:31.140512+0000","last_fullsized":"2024-02-23T00:45:31.140512+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":104,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":74,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":32,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.1c","version":"223'51","reported_seq":219,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.147476+0000","last_change":"2024-02-23T00:45:31.147476+0000","last_active":"2024-02-23T00:45:31.147476+0000","last_peered":"2024-02-23T00:45:31.147476+0000","last_clean":"2024-02-23T00:45:31.147476+0000","last_became_active":"2024-02-23T00:43:17.804718+0000","last_became_peered":"2024-02-23T00:43:17.804718+0000","last_unstale":"2024-02-23T00:45:31.147476+0000","last_undegraded":"2024-02-23T00:45:31.147476+0000","last_fullsized":"2024-02-23T00:45:31.147476+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.1f","version":"240'40","reported_seq":216,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.146635+0000","last_change":"2024-02-23T00:45:31.146635+0000","last_active":"2024-02-23T00:45:31.146635+0000","last_peered":"2024-02-23T00:45:31.146635+0000","last_clean":"2024-02-23T00:45:31.146635+0000","last_became_active":"2024-02-23T00:43:17.839914+0000","last_became_peered":"2024-02-23T00:43:17.839914+0000","last_unstale":"2024-02-23T00:45:31.146635+0000","last_undegraded":"2024-02-23T00:45:31.146635+0000","last_fullsized":"2024-02-23T00:45:31.146635+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":13,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.1e","version":"245'51","reported_seq":216,"reported_epoch":259,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-23T00:45:31.140368+0000","last_change":"2024-02-23T00:45:31.140368+0000","last_active":"2024-02-23T00:45:31.140368+0000","last_peered":"2024-02-23T00:45:31.140368+0000","last_clean":"2024-02-23T00:45:31.140368+0000","last_became_active":"2024-02-23T00:43:17.838682+0000","last_became_peered":"2024-02-23T00:43:17.838682+0000","last_unstale":"2024-02-23T00:45:31.140368+0000","last_undegraded":"2024-02-23T00:45:31.140368+0000","last_fullsized":"2024-02-23T00:45:31.140368+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"5.9","version":"67'8","reported_seq":297,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.447145+0000","last_change":"2024-02-23T00:42:47.985647+0000","last_active":"2024-02-23T00:45:30.447145+0000","last_peered":"2024-02-23T00:45:30.447145+0000","last_clean":"2024-02-23T00:45:30.447145+0000","last_became_active":"2024-02-23T00:39:18.793414+0000","last_became_peered":"2024-02-23T00:39:18.793414+0000","last_unstale":"2024-02-23T00:45:30.447145+0000","last_undegraded":"2024-02-23T00:45:30.447145+0000","last_fullsized":"2024-02-23T00:45:30.447145+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:42:47.985573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:47.985573+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"68'27","reported_seq":276,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.160034+0000","last_change":"2024-02-23T00:45:04.138883+0000","last_active":"2024-02-23T00:45:30.160034+0000","last_peered":"2024-02-23T00:45:30.160034+0000","last_clean":"2024-02-23T00:45:30.160034+0000","last_became_active":"2024-02-23T00:39:16.859401+0000","last_became_peered":"2024-02-23T00:39:16.859401+0000","last_unstale":"2024-02-23T00:45:30.160034+0000","last_undegraded":"2024-02-23T00:45:30.160034+0000","last_fullsized":"2024-02-23T00:45:30.160034+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'27","last_scrub_stamp":"2024-02-23T00:45:04.138751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.138751+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":222,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.139221+0000","last_change":"2024-02-23T00:43:42.320439+0000","last_active":"2024-02-23T00:45:31.139221+0000","last_peered":"2024-02-23T00:45:31.139221+0000","last_clean":"2024-02-23T00:45:31.139221+0000","last_became_active":"2024-02-23T00:39:14.784436+0000","last_became_peered":"2024-02-23T00:39:14.784436+0000","last_unstale":"2024-02-23T00:45:31.139221+0000","last_undegraded":"2024-02-23T00:45:31.139221+0000","last_fullsized":"2024-02-23T00:45:31.139221+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:42.320346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:04.330426+0000","last_clean_scrub_stamp":"2024-02-23T00:43:42.320346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":235,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.159962+0000","last_change":"2024-02-23T00:44:21.972446+0000","last_active":"2024-02-23T00:45:30.159962+0000","last_peered":"2024-02-23T00:45:30.159962+0000","last_clean":"2024-02-23T00:45:30.159962+0000","last_became_active":"2024-02-23T00:43:32.282545+0000","last_became_peered":"2024-02-23T00:43:32.282545+0000","last_unstale":"2024-02-23T00:45:30.159962+0000","last_undegraded":"2024-02-23T00:45:30.159962+0000","last_fullsized":"2024-02-23T00:45:30.159962+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:21.972383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:21.972383+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.3","version":"110'5","reported_seq":170,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.146711+0000","last_change":"2024-02-23T00:45:17.356489+0000","last_active":"2024-02-23T00:45:31.146711+0000","last_peered":"2024-02-23T00:45:31.146711+0000","last_clean":"2024-02-23T00:45:31.146711+0000","last_became_active":"2024-02-23T00:42:58.835612+0000","last_became_peered":"2024-02-23T00:42:58.835612+0000","last_unstale":"2024-02-23T00:45:31.146711+0000","last_undegraded":"2024-02-23T00:45:31.146711+0000","last_fullsized":"2024-02-23T00:45:31.146711+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'5","last_scrub_stamp":"2024-02-23T00:45:17.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:17.356406+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":164,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.160557+0000","last_change":"2024-02-23T00:45:17.181400+0000","last_active":"2024-02-23T00:45:30.160557+0000","last_peered":"2024-02-23T00:45:30.160557+0000","last_clean":"2024-02-23T00:45:30.160557+0000","last_became_active":"2024-02-23T00:42:58.828846+0000","last_became_peered":"2024-02-23T00:42:58.828846+0000","last_unstale":"2024-02-23T00:45:30.160557+0000","last_undegraded":"2024-02-23T00:45:30.160557+0000","last_fullsized":"2024-02-23T00:45:30.160557+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:17.181335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:17.181335+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"68'15","reported_seq":253,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.149918+0000","last_change":"2024-02-23T00:43:54.355050+0000","last_active":"2024-02-23T00:45:31.149918+0000","last_peered":"2024-02-23T00:45:31.149918+0000","last_clean":"2024-02-23T00:45:31.149918+0000","last_became_active":"2024-02-23T00:39:16.783701+0000","last_became_peered":"2024-02-23T00:39:16.783701+0000","last_unstale":"2024-02-23T00:45:31.149918+0000","last_undegraded":"2024-02-23T00:45:31.149918+0000","last_fullsized":"2024-02-23T00:45:31.149918+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:43:54.354974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:54.354974+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":218,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.137455+0000","last_change":"2024-02-23T00:42:48.591600+0000","last_active":"2024-02-23T00:45:31.137455+0000","last_peered":"2024-02-23T00:45:31.137455+0000","last_clean":"2024-02-23T00:45:31.137455+0000","last_became_active":"2024-02-23T00:39:14.772493+0000","last_became_peered":"2024-02-23T00:39:14.772493+0000","last_unstale":"2024-02-23T00:45:31.137455+0000","last_undegraded":"2024-02-23T00:45:31.137455+0000","last_fullsized":"2024-02-23T00:45:31.137455+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:48.591486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:48.591486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":218,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.139785+0000","last_change":"2024-02-23T00:44:22.380954+0000","last_active":"2024-02-23T00:45:31.139785+0000","last_peered":"2024-02-23T00:45:31.139785+0000","last_clean":"2024-02-23T00:45:31.139785+0000","last_became_active":"2024-02-23T00:39:18.794476+0000","last_became_peered":"2024-02-23T00:39:18.794476+0000","last_unstale":"2024-02-23T00:45:31.139785+0000","last_undegraded":"2024-02-23T00:45:31.139785+0000","last_fullsized":"2024-02-23T00:45:31.139785+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:22.380841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:08.335330+0000","last_clean_scrub_stamp":"2024-02-23T00:44:22.380841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":231,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.148623+0000","last_change":"2024-02-23T00:43:35.079353+0000","last_active":"2024-02-23T00:45:31.148623+0000","last_peered":"2024-02-23T00:45:31.148623+0000","last_clean":"2024-02-23T00:45:31.148623+0000","last_became_active":"2024-02-23T00:43:35.079004+0000","last_became_peered":"2024-02-23T00:43:35.079004+0000","last_unstale":"2024-02-23T00:45:31.148623+0000","last_undegraded":"2024-02-23T00:45:31.148623+0000","last_fullsized":"2024-02-23T00:45:31.148623+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:46.359646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:46.359646+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"110'5","reported_seq":169,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.138330+0000","last_change":"2024-02-23T00:45:06.483322+0000","last_active":"2024-02-23T00:45:31.138330+0000","last_peered":"2024-02-23T00:45:31.138330+0000","last_clean":"2024-02-23T00:45:31.138330+0000","last_became_active":"2024-02-23T00:42:58.741558+0000","last_became_peered":"2024-02-23T00:42:58.741558+0000","last_unstale":"2024-02-23T00:45:31.138330+0000","last_undegraded":"2024-02-23T00:45:31.138330+0000","last_fullsized":"2024-02-23T00:45:31.138330+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'5","last_scrub_stamp":"2024-02-23T00:45:06.483252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.483252+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":165,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.149211+0000","last_change":"2024-02-23T00:45:22.597544+0000","last_active":"2024-02-23T00:45:31.149211+0000","last_peered":"2024-02-23T00:45:31.149211+0000","last_clean":"2024-02-23T00:45:31.149211+0000","last_became_active":"2024-02-23T00:42:58.867305+0000","last_became_peered":"2024-02-23T00:42:58.867305+0000","last_unstale":"2024-02-23T00:45:31.149211+0000","last_undegraded":"2024-02-23T00:45:31.149211+0000","last_fullsized":"2024-02-23T00:45:31.149211+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:22.597434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:22.597434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"88'35","reported_seq":276,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.146405+0000","last_change":"2024-02-23T00:43:54.438282+0000","last_active":"2024-02-23T00:45:31.146405+0000","last_peered":"2024-02-23T00:45:31.146405+0000","last_clean":"2024-02-23T00:45:31.146405+0000","last_became_active":"2024-02-23T00:39:16.857413+0000","last_became_peered":"2024-02-23T00:39:16.857413+0000","last_unstale":"2024-02-23T00:45:31.146405+0000","last_undegraded":"2024-02-23T00:45:31.146405+0000","last_fullsized":"2024-02-23T00:45:31.146405+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'35","last_scrub_stamp":"2024-02-23T00:43:54.438168+0000","last_deep_scrub":"88'35","last_deep_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_clean_scrub_stamp":"2024-02-23T00:43:54.438168+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":222,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.140577+0000","last_change":"2024-02-23T00:43:53.317283+0000","last_active":"2024-02-23T00:45:31.140577+0000","last_peered":"2024-02-23T00:45:31.140577+0000","last_clean":"2024-02-23T00:45:31.140577+0000","last_became_active":"2024-02-23T00:39:14.776705+0000","last_became_peered":"2024-02-23T00:39:14.776705+0000","last_unstale":"2024-02-23T00:45:31.140577+0000","last_undegraded":"2024-02-23T00:45:31.140577+0000","last_fullsized":"2024-02-23T00:45:31.140577+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:53.317202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_clean_scrub_stamp":"2024-02-23T00:43:53.317202+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":218,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.137429+0000","last_change":"2024-02-23T00:43:44.313672+0000","last_active":"2024-02-23T00:45:31.137429+0000","last_peered":"2024-02-23T00:45:31.137429+0000","last_clean":"2024-02-23T00:45:31.137429+0000","last_became_active":"2024-02-23T00:39:18.792718+0000","last_became_peered":"2024-02-23T00:39:18.792718+0000","last_unstale":"2024-02-23T00:45:31.137429+0000","last_undegraded":"2024-02-23T00:45:31.137429+0000","last_fullsized":"2024-02-23T00:45:31.137429+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:44.313501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:44.313501+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":232,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.446305+0000","last_change":"2024-02-23T00:44:36.758043+0000","last_active":"2024-02-23T00:45:30.446305+0000","last_peered":"2024-02-23T00:45:30.446305+0000","last_clean":"2024-02-23T00:45:30.446305+0000","last_became_active":"2024-02-23T00:43:39.470999+0000","last_became_peered":"2024-02-23T00:43:39.470999+0000","last_unstale":"2024-02-23T00:45:30.446305+0000","last_undegraded":"2024-02-23T00:45:30.446305+0000","last_fullsized":"2024-02-23T00:45:30.446305+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:36.757960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:36.757960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"110'5","reported_seq":169,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.139679+0000","last_change":"2024-02-23T00:45:23.832372+0000","last_active":"2024-02-23T00:45:31.139679+0000","last_peered":"2024-02-23T00:45:31.139679+0000","last_clean":"2024-02-23T00:45:31.139679+0000","last_became_active":"2024-02-23T00:42:58.684001+0000","last_became_peered":"2024-02-23T00:42:58.684001+0000","last_unstale":"2024-02-23T00:45:31.139679+0000","last_undegraded":"2024-02-23T00:45:31.139679+0000","last_fullsized":"2024-02-23T00:45:31.139679+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'5","last_scrub_stamp":"2024-02-23T00:45:23.832291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.832291+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":160,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.132879+0000","last_change":"2024-02-23T00:42:57.641727+0000","last_active":"2024-02-23T00:45:31.132879+0000","last_peered":"2024-02-23T00:45:31.132879+0000","last_clean":"2024-02-23T00:45:31.132879+0000","last_became_active":"2024-02-23T00:42:57.639013+0000","last_became_peered":"2024-02-23T00:42:57.639013+0000","last_unstale":"2024-02-23T00:45:31.132879+0000","last_undegraded":"2024-02-23T00:45:31.132879+0000","last_fullsized":"2024-02-23T00:45:31.132879+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"67'17","reported_seq":271,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.146226+0000","last_change":"2024-02-23T00:43:45.194207+0000","last_active":"2024-02-23T00:45:31.146226+0000","last_peered":"2024-02-23T00:45:31.146226+0000","last_clean":"2024-02-23T00:45:31.146226+0000","last_became_active":"2024-02-23T00:40:42.933806+0000","last_became_peered":"2024-02-23T00:40:42.933806+0000","last_unstale":"2024-02-23T00:45:31.146226+0000","last_undegraded":"2024-02-23T00:45:31.146226+0000","last_fullsized":"2024-02-23T00:45:31.146226+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'17","last_scrub_stamp":"2024-02-23T00:43:45.194139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.194139+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":193,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.137811+0000","last_change":"2024-02-23T00:43:31.362599+0000","last_active":"2024-02-23T00:45:31.137811+0000","last_peered":"2024-02-23T00:45:31.137811+0000","last_clean":"2024-02-23T00:45:31.137811+0000","last_became_active":"2024-02-23T00:40:42.931041+0000","last_became_peered":"2024-02-23T00:40:42.931041+0000","last_unstale":"2024-02-23T00:45:31.137811+0000","last_undegraded":"2024-02-23T00:45:31.137811+0000","last_fullsized":"2024-02-23T00:45:31.137811+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:31.362510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:31.362510+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":219,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.133181+0000","last_change":"2024-02-23T00:44:22.804352+0000","last_active":"2024-02-23T00:45:31.133181+0000","last_peered":"2024-02-23T00:45:31.133181+0000","last_clean":"2024-02-23T00:45:31.133181+0000","last_became_active":"2024-02-23T00:39:18.799875+0000","last_became_peered":"2024-02-23T00:39:18.799875+0000","last_unstale":"2024-02-23T00:45:31.133181+0000","last_undegraded":"2024-02-23T00:45:31.133181+0000","last_fullsized":"2024-02-23T00:45:31.133181+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:22.804279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:22.804279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":234,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.141696+0000","last_change":"2024-02-23T00:44:36.481327+0000","last_active":"2024-02-23T00:45:31.141696+0000","last_peered":"2024-02-23T00:45:31.141696+0000","last_clean":"2024-02-23T00:45:31.141696+0000","last_became_active":"2024-02-23T00:43:08.885257+0000","last_became_peered":"2024-02-23T00:43:08.885257+0000","last_unstale":"2024-02-23T00:45:31.141696+0000","last_undegraded":"2024-02-23T00:45:31.141696+0000","last_fullsized":"2024-02-23T00:45:31.141696+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:36.481256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:36.481256+0000","last_clean_scrub_stamp":"2024-02-23T00:44:36.481256+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.6","version":"110'9","reported_seq":169,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.149975+0000","last_change":"2024-02-23T00:42:58.871934+0000","last_active":"2024-02-23T00:45:31.149975+0000","last_peered":"2024-02-23T00:45:31.149975+0000","last_clean":"2024-02-23T00:45:31.149975+0000","last_became_active":"2024-02-23T00:42:58.870147+0000","last_became_peered":"2024-02-23T00:42:58.870147+0000","last_unstale":"2024-02-23T00:45:31.149975+0000","last_undegraded":"2024-02-23T00:45:31.149975+0000","last_fullsized":"2024-02-23T00:45:31.149975+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":165,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.134554+0000","last_change":"2024-02-23T00:45:23.744990+0000","last_active":"2024-02-23T00:45:31.134554+0000","last_peered":"2024-02-23T00:45:31.134554+0000","last_clean":"2024-02-23T00:45:31.134554+0000","last_became_active":"2024-02-23T00:42:57.675306+0000","last_became_peered":"2024-02-23T00:42:57.675306+0000","last_unstale":"2024-02-23T00:45:31.134554+0000","last_undegraded":"2024-02-23T00:45:31.134554+0000","last_fullsized":"2024-02-23T00:45:31.134554+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:23.744846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.744846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"67'10","reported_seq":236,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.146087+0000","last_change":"2024-02-23T00:45:25.175258+0000","last_active":"2024-02-23T00:45:31.146087+0000","last_peered":"2024-02-23T00:45:31.146087+0000","last_clean":"2024-02-23T00:45:31.146087+0000","last_became_active":"2024-02-23T00:39:16.777530+0000","last_became_peered":"2024-02-23T00:39:16.777530+0000","last_unstale":"2024-02-23T00:45:31.146087+0000","last_undegraded":"2024-02-23T00:45:31.146087+0000","last_fullsized":"2024-02-23T00:45:31.146087+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'10","last_scrub_stamp":"2024-02-23T00:45:25.175173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.175173+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":223,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.135291+0000","last_change":"2024-02-23T00:45:10.693332+0000","last_active":"2024-02-23T00:45:31.135291+0000","last_peered":"2024-02-23T00:45:31.135291+0000","last_clean":"2024-02-23T00:45:31.135291+0000","last_became_active":"2024-02-23T00:39:14.780379+0000","last_became_peered":"2024-02-23T00:39:14.780379+0000","last_unstale":"2024-02-23T00:45:31.135291+0000","last_undegraded":"2024-02-23T00:45:31.135291+0000","last_fullsized":"2024-02-23T00:45:31.135291+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.693273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.575242+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.693273+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"67'8","reported_seq":301,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.142176+0000","last_change":"2024-02-23T00:44:25.470245+0000","last_active":"2024-02-23T00:45:31.142176+0000","last_peered":"2024-02-23T00:45:31.142176+0000","last_clean":"2024-02-23T00:45:31.142176+0000","last_became_active":"2024-02-23T00:39:18.799544+0000","last_became_peered":"2024-02-23T00:39:18.799544+0000","last_unstale":"2024-02-23T00:45:31.142176+0000","last_undegraded":"2024-02-23T00:45:31.142176+0000","last_fullsized":"2024-02-23T00:45:31.142176+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:44:25.470174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:25.470174+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":232,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.146161+0000","last_change":"2024-02-23T00:43:36.205958+0000","last_active":"2024-02-23T00:45:31.146161+0000","last_peered":"2024-02-23T00:45:31.146161+0000","last_clean":"2024-02-23T00:45:31.146161+0000","last_became_active":"2024-02-23T00:43:13.338697+0000","last_became_peered":"2024-02-23T00:43:13.338697+0000","last_unstale":"2024-02-23T00:45:31.146161+0000","last_undegraded":"2024-02-23T00:45:31.146161+0000","last_fullsized":"2024-02-23T00:45:31.146161+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:36.205886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:15.081153+0000","last_clean_scrub_stamp":"2024-02-23T00:43:36.205886+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":160,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.138052+0000","last_change":"2024-02-23T00:42:57.686234+0000","last_active":"2024-02-23T00:45:31.138052+0000","last_peered":"2024-02-23T00:45:31.138052+0000","last_clean":"2024-02-23T00:45:31.138052+0000","last_became_active":"2024-02-23T00:42:57.685579+0000","last_became_peered":"2024-02-23T00:42:57.685579+0000","last_unstale":"2024-02-23T00:45:31.138052+0000","last_undegraded":"2024-02-23T00:45:31.138052+0000","last_fullsized":"2024-02-23T00:45:31.138052+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":165,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.133019+0000","last_change":"2024-02-23T00:45:22.874761+0000","last_active":"2024-02-23T00:45:31.133019+0000","last_peered":"2024-02-23T00:45:31.133019+0000","last_clean":"2024-02-23T00:45:31.133019+0000","last_became_active":"2024-02-23T00:42:58.765106+0000","last_became_peered":"2024-02-23T00:42:58.765106+0000","last_unstale":"2024-02-23T00:45:31.133019+0000","last_undegraded":"2024-02-23T00:45:31.133019+0000","last_fullsized":"2024-02-23T00:45:31.133019+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:22.874696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:22.874696+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"67'15","reported_seq":246,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.138133+0000","last_change":"2024-02-23T00:44:19.830383+0000","last_active":"2024-02-23T00:45:31.138133+0000","last_peered":"2024-02-23T00:45:31.138133+0000","last_clean":"2024-02-23T00:45:31.138133+0000","last_became_active":"2024-02-23T00:39:16.789757+0000","last_became_peered":"2024-02-23T00:39:16.789757+0000","last_unstale":"2024-02-23T00:45:31.138133+0000","last_undegraded":"2024-02-23T00:45:31.138133+0000","last_fullsized":"2024-02-23T00:45:31.138133+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:44:19.830309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:44:19.830309+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":223,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.133472+0000","last_change":"2024-02-23T00:44:29.773000+0000","last_active":"2024-02-23T00:45:31.133472+0000","last_peered":"2024-02-23T00:45:31.133472+0000","last_clean":"2024-02-23T00:45:31.133472+0000","last_became_active":"2024-02-23T00:39:14.780526+0000","last_became_peered":"2024-02-23T00:39:14.780526+0000","last_unstale":"2024-02-23T00:45:31.133472+0000","last_undegraded":"2024-02-23T00:45:31.133472+0000","last_fullsized":"2024-02-23T00:45:31.133472+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:29.772933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:29.772933+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":302,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.146070+0000","last_change":"2024-02-23T00:44:23.109508+0000","last_active":"2024-02-23T00:45:31.146070+0000","last_peered":"2024-02-23T00:45:31.146070+0000","last_clean":"2024-02-23T00:45:31.146070+0000","last_became_active":"2024-02-23T00:39:18.800440+0000","last_became_peered":"2024-02-23T00:39:18.800440+0000","last_unstale":"2024-02-23T00:45:31.146070+0000","last_undegraded":"2024-02-23T00:45:31.146070+0000","last_fullsized":"2024-02-23T00:45:31.146070+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:44:23.109439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.109439+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":235,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.161345+0000","last_change":"2024-02-23T00:44:22.976713+0000","last_active":"2024-02-23T00:45:30.161345+0000","last_peered":"2024-02-23T00:45:30.161345+0000","last_clean":"2024-02-23T00:45:30.161345+0000","last_became_active":"2024-02-23T00:43:16.650527+0000","last_became_peered":"2024-02-23T00:43:16.650527+0000","last_unstale":"2024-02-23T00:45:30.161345+0000","last_undegraded":"2024-02-23T00:45:30.161345+0000","last_fullsized":"2024-02-23T00:45:30.161345+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:22.976611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:10.666299+0000","last_clean_scrub_stamp":"2024-02-23T00:44:22.976611+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":165,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.132925+0000","last_change":"2024-02-23T00:45:26.719081+0000","last_active":"2024-02-23T00:45:31.132925+0000","last_peered":"2024-02-23T00:45:31.132925+0000","last_clean":"2024-02-23T00:45:31.132925+0000","last_became_active":"2024-02-23T00:42:57.671472+0000","last_became_peered":"2024-02-23T00:42:57.671472+0000","last_unstale":"2024-02-23T00:45:31.132925+0000","last_undegraded":"2024-02-23T00:45:31.132925+0000","last_fullsized":"2024-02-23T00:45:31.132925+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:26.719011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:26.719011+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,3],"acting":[1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":160,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.144279+0000","last_change":"2024-02-23T00:42:58.834770+0000","last_active":"2024-02-23T00:45:31.144279+0000","last_peered":"2024-02-23T00:45:31.144279+0000","last_clean":"2024-02-23T00:45:31.144279+0000","last_became_active":"2024-02-23T00:42:58.833924+0000","last_became_peered":"2024-02-23T00:42:58.833924+0000","last_unstale":"2024-02-23T00:45:31.144279+0000","last_undegraded":"2024-02-23T00:45:31.144279+0000","last_fullsized":"2024-02-23T00:45:31.144279+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"68'17","reported_seq":273,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.145901+0000","last_change":"2024-02-23T00:43:33.151471+0000","last_active":"2024-02-23T00:45:31.145901+0000","last_peered":"2024-02-23T00:45:31.145901+0000","last_clean":"2024-02-23T00:45:31.145901+0000","last_became_active":"2024-02-23T00:40:42.933759+0000","last_became_peered":"2024-02-23T00:40:42.933759+0000","last_unstale":"2024-02-23T00:45:31.145901+0000","last_undegraded":"2024-02-23T00:45:31.145901+0000","last_fullsized":"2024-02-23T00:45:31.145901+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:43:33.151401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:33.151401+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":220,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.446658+0000","last_change":"2024-02-23T00:44:18.886545+0000","last_active":"2024-02-23T00:45:30.446658+0000","last_peered":"2024-02-23T00:45:30.446658+0000","last_clean":"2024-02-23T00:45:30.446658+0000","last_became_active":"2024-02-23T00:39:14.784121+0000","last_became_peered":"2024-02-23T00:39:14.784121+0000","last_unstale":"2024-02-23T00:45:30.446658+0000","last_undegraded":"2024-02-23T00:45:30.446658+0000","last_fullsized":"2024-02-23T00:45:30.446658+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:18.886477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:18.886477+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":219,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.161493+0000","last_change":"2024-02-23T00:45:08.118391+0000","last_active":"2024-02-23T00:45:30.161493+0000","last_peered":"2024-02-23T00:45:30.161493+0000","last_clean":"2024-02-23T00:45:30.161493+0000","last_became_active":"2024-02-23T00:39:18.800777+0000","last_became_peered":"2024-02-23T00:39:18.800777+0000","last_unstale":"2024-02-23T00:45:30.161493+0000","last_undegraded":"2024-02-23T00:45:30.161493+0000","last_fullsized":"2024-02-23T00:45:30.161493+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.118302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.118302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"67'1","reported_seq":236,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.134555+0000","last_change":"2024-02-23T00:43:56.091057+0000","last_active":"2024-02-23T00:45:31.134555+0000","last_peered":"2024-02-23T00:45:31.134555+0000","last_clean":"2024-02-23T00:45:31.134555+0000","last_became_active":"2024-02-23T00:43:22.866277+0000","last_became_peered":"2024-02-23T00:43:22.866277+0000","last_unstale":"2024-02-23T00:45:31.134555+0000","last_undegraded":"2024-02-23T00:45:31.134555+0000","last_fullsized":"2024-02-23T00:45:31.134555+0000","mapping_epoch":65,"log_start":"67'1","ondisk_log_start":"67'1","created":65,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:43:56.090982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:09.079918+0000","last_clean_scrub_stamp":"2024-02-23T00:43:56.090982+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":164,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.142275+0000","last_change":"2024-02-23T00:45:06.651852+0000","last_active":"2024-02-23T00:45:31.142275+0000","last_peered":"2024-02-23T00:45:31.142275+0000","last_clean":"2024-02-23T00:45:31.142275+0000","last_became_active":"2024-02-23T00:42:58.718615+0000","last_became_peered":"2024-02-23T00:42:58.718615+0000","last_unstale":"2024-02-23T00:45:31.142275+0000","last_undegraded":"2024-02-23T00:45:31.142275+0000","last_fullsized":"2024-02-23T00:45:31.142275+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.651711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.651711+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":160,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.134601+0000","last_change":"2024-02-23T00:42:57.655252+0000","last_active":"2024-02-23T00:45:31.134601+0000","last_peered":"2024-02-23T00:45:31.134601+0000","last_clean":"2024-02-23T00:45:31.134601+0000","last_became_active":"2024-02-23T00:42:57.653279+0000","last_became_peered":"2024-02-23T00:42:57.653279+0000","last_unstale":"2024-02-23T00:45:31.134601+0000","last_undegraded":"2024-02-23T00:45:31.134601+0000","last_fullsized":"2024-02-23T00:45:31.134601+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":298,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.132486+0000","last_change":"2024-02-23T00:42:53.978054+0000","last_active":"2024-02-23T00:45:31.132486+0000","last_peered":"2024-02-23T00:45:31.132486+0000","last_clean":"2024-02-23T00:45:31.132486+0000","last_became_active":"2024-02-23T00:39:18.798536+0000","last_became_peered":"2024-02-23T00:39:18.798536+0000","last_unstale":"2024-02-23T00:45:31.132486+0000","last_undegraded":"2024-02-23T00:45:31.132486+0000","last_fullsized":"2024-02-23T00:45:31.132486+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:42:53.977950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.977950+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"68'20","reported_seq":257,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.144624+0000","last_change":"2024-02-23T00:45:05.291909+0000","last_active":"2024-02-23T00:45:31.144624+0000","last_peered":"2024-02-23T00:45:31.144624+0000","last_clean":"2024-02-23T00:45:31.144624+0000","last_became_active":"2024-02-23T00:39:16.780130+0000","last_became_peered":"2024-02-23T00:39:16.780130+0000","last_unstale":"2024-02-23T00:45:31.144624+0000","last_undegraded":"2024-02-23T00:45:31.144624+0000","last_fullsized":"2024-02-23T00:45:31.144624+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'20","last_scrub_stamp":"2024-02-23T00:45:05.291836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:05.291836+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":224,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.141517+0000","last_change":"2024-02-23T00:43:51.391999+0000","last_active":"2024-02-23T00:45:31.141517+0000","last_peered":"2024-02-23T00:45:31.141517+0000","last_clean":"2024-02-23T00:45:31.141517+0000","last_became_active":"2024-02-23T00:39:14.781999+0000","last_became_peered":"2024-02-23T00:39:14.781999+0000","last_unstale":"2024-02-23T00:45:31.141517+0000","last_undegraded":"2024-02-23T00:45:31.141517+0000","last_fullsized":"2024-02-23T00:45:31.141517+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:51.391939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:51.391939+0000","last_clean_scrub_stamp":"2024-02-23T00:43:51.391939+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":214,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.447490+0000","last_change":"2024-02-23T00:44:24.759694+0000","last_active":"2024-02-23T00:45:30.447490+0000","last_peered":"2024-02-23T00:45:30.447490+0000","last_clean":"2024-02-23T00:45:30.447490+0000","last_became_active":"2024-02-23T00:39:21.161928+0000","last_became_peered":"2024-02-23T00:39:21.161928+0000","last_unstale":"2024-02-23T00:45:30.447490+0000","last_undegraded":"2024-02-23T00:45:30.447490+0000","last_fullsized":"2024-02-23T00:45:30.447490+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:24.759627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.945642+0000","last_clean_scrub_stamp":"2024-02-23T00:44:24.759627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"110'10","reported_seq":174,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.139979+0000","last_change":"2024-02-23T00:45:24.377454+0000","last_active":"2024-02-23T00:45:31.139979+0000","last_peered":"2024-02-23T00:45:31.139979+0000","last_clean":"2024-02-23T00:45:31.139979+0000","last_became_active":"2024-02-23T00:42:58.696292+0000","last_became_peered":"2024-02-23T00:42:58.696292+0000","last_unstale":"2024-02-23T00:45:31.139979+0000","last_undegraded":"2024-02-23T00:45:31.139979+0000","last_fullsized":"2024-02-23T00:45:31.139979+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'10","last_scrub_stamp":"2024-02-23T00:45:24.377329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:24.377329+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":159,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.161651+0000","last_change":"2024-02-23T00:42:58.717788+0000","last_active":"2024-02-23T00:45:30.161651+0000","last_peered":"2024-02-23T00:45:30.161651+0000","last_clean":"2024-02-23T00:45:30.161651+0000","last_became_active":"2024-02-23T00:42:58.717371+0000","last_became_peered":"2024-02-23T00:42:58.717371+0000","last_unstale":"2024-02-23T00:45:30.161651+0000","last_undegraded":"2024-02-23T00:45:30.161651+0000","last_fullsized":"2024-02-23T00:45:30.161651+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":219,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.144999+0000","last_change":"2024-02-23T00:44:34.059194+0000","last_active":"2024-02-23T00:45:31.144999+0000","last_peered":"2024-02-23T00:45:31.144999+0000","last_clean":"2024-02-23T00:45:31.144999+0000","last_became_active":"2024-02-23T00:39:18.798829+0000","last_became_peered":"2024-02-23T00:39:18.798829+0000","last_unstale":"2024-02-23T00:45:31.144999+0000","last_undegraded":"2024-02-23T00:45:31.144999+0000","last_fullsized":"2024-02-23T00:45:31.144999+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:34.059129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:34.059129+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"251'40","reported_seq":294,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.132425+0000","last_change":"2024-02-23T00:45:23.910609+0000","last_active":"2024-02-23T00:45:31.132425+0000","last_peered":"2024-02-23T00:45:31.132425+0000","last_clean":"2024-02-23T00:45:31.132425+0000","last_became_active":"2024-02-23T00:39:16.859002+0000","last_became_peered":"2024-02-23T00:39:16.859002+0000","last_unstale":"2024-02-23T00:45:31.132425+0000","last_undegraded":"2024-02-23T00:45:31.132425+0000","last_fullsized":"2024-02-23T00:45:31.132425+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'40","last_scrub_stamp":"2024-02-23T00:45:23.910546+0000","last_deep_scrub":"251'40","last_deep_scrub_stamp":"2024-02-23T00:45:23.910546+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.910546+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":223,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.142040+0000","last_change":"2024-02-23T00:45:16.664785+0000","last_active":"2024-02-23T00:45:31.142040+0000","last_peered":"2024-02-23T00:45:31.142040+0000","last_clean":"2024-02-23T00:45:31.142040+0000","last_became_active":"2024-02-23T00:39:14.779193+0000","last_became_peered":"2024-02-23T00:39:14.779193+0000","last_unstale":"2024-02-23T00:45:31.142040+0000","last_undegraded":"2024-02-23T00:45:31.142040+0000","last_fullsized":"2024-02-23T00:45:31.142040+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.664682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.664682+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":217,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.145445+0000","last_change":"2024-02-23T00:45:03.228087+0000","last_active":"2024-02-23T00:45:31.145445+0000","last_peered":"2024-02-23T00:45:31.145445+0000","last_clean":"2024-02-23T00:45:31.145445+0000","last_became_active":"2024-02-23T00:39:20.795271+0000","last_became_peered":"2024-02-23T00:39:20.795271+0000","last_unstale":"2024-02-23T00:45:31.145445+0000","last_undegraded":"2024-02-23T00:45:31.145445+0000","last_fullsized":"2024-02-23T00:45:31.145445+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:03.227995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:45:03.227995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"110'2","reported_seq":166,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.137338+0000","last_change":"2024-02-23T00:45:10.465187+0000","last_active":"2024-02-23T00:45:31.137338+0000","last_peered":"2024-02-23T00:45:31.137338+0000","last_clean":"2024-02-23T00:45:31.137338+0000","last_became_active":"2024-02-23T00:42:58.814093+0000","last_became_peered":"2024-02-23T00:42:58.814093+0000","last_unstale":"2024-02-23T00:45:31.137338+0000","last_undegraded":"2024-02-23T00:45:31.137338+0000","last_fullsized":"2024-02-23T00:45:31.137338+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'2","last_scrub_stamp":"2024-02-23T00:45:10.465108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.465108+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":165,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.137973+0000","last_change":"2024-02-23T00:45:13.874437+0000","last_active":"2024-02-23T00:45:31.137973+0000","last_peered":"2024-02-23T00:45:31.137973+0000","last_clean":"2024-02-23T00:45:31.137973+0000","last_became_active":"2024-02-23T00:42:58.809283+0000","last_became_peered":"2024-02-23T00:42:58.809283+0000","last_unstale":"2024-02-23T00:45:31.137973+0000","last_undegraded":"2024-02-23T00:45:31.137973+0000","last_fullsized":"2024-02-23T00:45:31.137973+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:13.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:13.874373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":218,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.146320+0000","last_change":"2024-02-23T00:44:19.312509+0000","last_active":"2024-02-23T00:45:31.146320+0000","last_peered":"2024-02-23T00:45:31.146320+0000","last_clean":"2024-02-23T00:45:31.146320+0000","last_became_active":"2024-02-23T00:39:18.796664+0000","last_became_peered":"2024-02-23T00:39:18.796664+0000","last_unstale":"2024-02-23T00:45:31.146320+0000","last_undegraded":"2024-02-23T00:45:31.146320+0000","last_fullsized":"2024-02-23T00:45:31.146320+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:19.312418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:19.312418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"203'34","reported_seq":282,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.149794+0000","last_change":"2024-02-23T00:44:35.644033+0000","last_active":"2024-02-23T00:45:31.149794+0000","last_peered":"2024-02-23T00:45:31.149794+0000","last_clean":"2024-02-23T00:45:31.149794+0000","last_became_active":"2024-02-23T00:39:16.857278+0000","last_became_peered":"2024-02-23T00:39:16.857278+0000","last_unstale":"2024-02-23T00:45:31.149794+0000","last_undegraded":"2024-02-23T00:45:31.149794+0000","last_fullsized":"2024-02-23T00:45:31.149794+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'34","last_scrub_stamp":"2024-02-23T00:44:35.643931+0000","last_deep_scrub":"203'34","last_deep_scrub_stamp":"2024-02-23T00:44:35.643931+0000","last_clean_scrub_stamp":"2024-02-23T00:44:35.643931+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":220,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.449596+0000","last_change":"2024-02-23T00:45:12.730663+0000","last_active":"2024-02-23T00:45:30.449596+0000","last_peered":"2024-02-23T00:45:30.449596+0000","last_clean":"2024-02-23T00:45:30.449596+0000","last_became_active":"2024-02-23T00:39:14.783662+0000","last_became_peered":"2024-02-23T00:39:14.783662+0000","last_unstale":"2024-02-23T00:45:30.449596+0000","last_undegraded":"2024-02-23T00:45:30.449596+0000","last_fullsized":"2024-02-23T00:45:30.449596+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.730539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.730539+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":217,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.135723+0000","last_change":"2024-02-23T00:45:04.585266+0000","last_active":"2024-02-23T00:45:31.135723+0000","last_peered":"2024-02-23T00:45:31.135723+0000","last_clean":"2024-02-23T00:45:31.135723+0000","last_became_active":"2024-02-23T00:39:21.160060+0000","last_became_peered":"2024-02-23T00:39:21.160060+0000","last_unstale":"2024-02-23T00:45:31.135723+0000","last_undegraded":"2024-02-23T00:45:31.135723+0000","last_fullsized":"2024-02-23T00:45:31.135723+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:04.585179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.585179+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.8","version":"110'2","reported_seq":166,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.137668+0000","last_change":"2024-02-23T00:44:33.389680+0000","last_active":"2024-02-23T00:45:31.137668+0000","last_peered":"2024-02-23T00:45:31.137668+0000","last_clean":"2024-02-23T00:45:31.137668+0000","last_became_active":"2024-02-23T00:42:58.809584+0000","last_became_peered":"2024-02-23T00:42:58.809584+0000","last_unstale":"2024-02-23T00:45:31.137668+0000","last_undegraded":"2024-02-23T00:45:31.137668+0000","last_fullsized":"2024-02-23T00:45:31.137668+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'2","last_scrub_stamp":"2024-02-23T00:44:33.389605+0000","last_deep_scrub":"110'2","last_deep_scrub_stamp":"2024-02-23T00:44:33.389605+0000","last_clean_scrub_stamp":"2024-02-23T00:44:33.389605+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":164,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.160994+0000","last_change":"2024-02-23T00:45:18.185097+0000","last_active":"2024-02-23T00:45:30.160994+0000","last_peered":"2024-02-23T00:45:30.160994+0000","last_clean":"2024-02-23T00:45:30.160994+0000","last_became_active":"2024-02-23T00:42:58.726652+0000","last_became_peered":"2024-02-23T00:42:58.726652+0000","last_unstale":"2024-02-23T00:45:30.160994+0000","last_undegraded":"2024-02-23T00:45:30.160994+0000","last_fullsized":"2024-02-23T00:45:30.160994+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:18.184992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:18.184992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":219,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.161544+0000","last_change":"2024-02-23T00:43:45.054194+0000","last_active":"2024-02-23T00:45:30.161544+0000","last_peered":"2024-02-23T00:45:30.161544+0000","last_clean":"2024-02-23T00:45:30.161544+0000","last_became_active":"2024-02-23T00:39:18.800378+0000","last_became_peered":"2024-02-23T00:39:18.800378+0000","last_unstale":"2024-02-23T00:45:30.161544+0000","last_undegraded":"2024-02-23T00:45:30.161544+0000","last_fullsized":"2024-02-23T00:45:30.161544+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:45.054115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:59.694033+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.054115+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"95'18","reported_seq":246,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.149589+0000","last_change":"2024-02-23T00:44:18.630435+0000","last_active":"2024-02-23T00:45:31.149589+0000","last_peered":"2024-02-23T00:45:31.149589+0000","last_clean":"2024-02-23T00:45:31.149589+0000","last_became_active":"2024-02-23T00:39:16.787143+0000","last_became_peered":"2024-02-23T00:39:16.787143+0000","last_unstale":"2024-02-23T00:45:31.149589+0000","last_undegraded":"2024-02-23T00:45:31.149589+0000","last_fullsized":"2024-02-23T00:45:31.149589+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'18","last_scrub_stamp":"2024-02-23T00:44:18.630363+0000","last_deep_scrub":"95'18","last_deep_scrub_stamp":"2024-02-23T00:42:01.195545+0000","last_clean_scrub_stamp":"2024-02-23T00:44:18.630363+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":220,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.446443+0000","last_change":"2024-02-23T00:42:50.947420+0000","last_active":"2024-02-23T00:45:30.446443+0000","last_peered":"2024-02-23T00:45:30.446443+0000","last_clean":"2024-02-23T00:45:30.446443+0000","last_became_active":"2024-02-23T00:39:14.784578+0000","last_became_peered":"2024-02-23T00:39:14.784578+0000","last_unstale":"2024-02-23T00:45:30.446443+0000","last_undegraded":"2024-02-23T00:45:30.446443+0000","last_fullsized":"2024-02-23T00:45:30.446443+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:50.947349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:50.947349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":224,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.148334+0000","last_change":"2024-02-23T00:43:48.258338+0000","last_active":"2024-02-23T00:45:31.148334+0000","last_peered":"2024-02-23T00:45:31.148334+0000","last_clean":"2024-02-23T00:45:31.148334+0000","last_became_active":"2024-02-23T00:39:14.775446+0000","last_became_peered":"2024-02-23T00:39:14.775446+0000","last_unstale":"2024-02-23T00:45:31.148334+0000","last_undegraded":"2024-02-23T00:45:31.148334+0000","last_fullsized":"2024-02-23T00:45:31.148334+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:48.258269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.258269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":212,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.140753+0000","last_change":"2024-02-23T00:43:47.905617+0000","last_active":"2024-02-23T00:45:31.140753+0000","last_peered":"2024-02-23T00:45:31.140753+0000","last_clean":"2024-02-23T00:45:31.140753+0000","last_became_active":"2024-02-23T00:39:20.797242+0000","last_became_peered":"2024-02-23T00:39:20.797242+0000","last_unstale":"2024-02-23T00:45:31.140753+0000","last_undegraded":"2024-02-23T00:45:31.140753+0000","last_fullsized":"2024-02-23T00:45:31.140753+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:47.905501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:47.905501+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"110'5","reported_seq":168,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.449345+0000","last_change":"2024-02-23T00:45:26.760937+0000","last_active":"2024-02-23T00:45:30.449345+0000","last_peered":"2024-02-23T00:45:30.449345+0000","last_clean":"2024-02-23T00:45:30.449345+0000","last_became_active":"2024-02-23T00:42:58.742998+0000","last_became_peered":"2024-02-23T00:42:58.742998+0000","last_unstale":"2024-02-23T00:45:30.449345+0000","last_undegraded":"2024-02-23T00:45:30.449345+0000","last_fullsized":"2024-02-23T00:45:30.449345+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'5","last_scrub_stamp":"2024-02-23T00:45:26.760862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:26.760862+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":164,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.138730+0000","last_change":"2024-02-23T00:44:35.360128+0000","last_active":"2024-02-23T00:45:31.138730+0000","last_peered":"2024-02-23T00:45:31.138730+0000","last_clean":"2024-02-23T00:45:31.138730+0000","last_became_active":"2024-02-23T00:42:58.690858+0000","last_became_peered":"2024-02-23T00:42:58.690858+0000","last_unstale":"2024-02-23T00:45:31.138730+0000","last_undegraded":"2024-02-23T00:45:31.138730+0000","last_fullsized":"2024-02-23T00:45:31.138730+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:35.360054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:35.360054+0000","last_clean_scrub_stamp":"2024-02-23T00:44:35.360054+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":216,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.448718+0000","last_change":"2024-02-23T00:45:16.746109+0000","last_active":"2024-02-23T00:45:30.448718+0000","last_peered":"2024-02-23T00:45:30.448718+0000","last_clean":"2024-02-23T00:45:30.448718+0000","last_became_active":"2024-02-23T00:39:18.793801+0000","last_became_peered":"2024-02-23T00:39:18.793801+0000","last_unstale":"2024-02-23T00:45:30.448718+0000","last_undegraded":"2024-02-23T00:45:30.448718+0000","last_fullsized":"2024-02-23T00:45:30.448718+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.746013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.746013+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"67'14","reported_seq":242,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.141869+0000","last_change":"2024-02-23T00:43:44.369862+0000","last_active":"2024-02-23T00:45:31.141869+0000","last_peered":"2024-02-23T00:45:31.141869+0000","last_clean":"2024-02-23T00:45:31.141869+0000","last_became_active":"2024-02-23T00:39:16.858880+0000","last_became_peered":"2024-02-23T00:39:16.858880+0000","last_unstale":"2024-02-23T00:45:31.141869+0000","last_undegraded":"2024-02-23T00:45:31.141869+0000","last_fullsized":"2024-02-23T00:45:31.141869+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'14","last_scrub_stamp":"2024-02-23T00:43:44.369792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:44.369792+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":230,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.446556+0000","last_change":"2024-02-23T00:45:08.796153+0000","last_active":"2024-02-23T00:45:30.446556+0000","last_peered":"2024-02-23T00:45:30.446556+0000","last_clean":"2024-02-23T00:45:30.446556+0000","last_became_active":"2024-02-23T00:39:08.248249+0000","last_became_peered":"2024-02-23T00:39:08.248249+0000","last_unstale":"2024-02-23T00:45:30.446556+0000","last_undegraded":"2024-02-23T00:45:30.446556+0000","last_fullsized":"2024-02-23T00:45:30.446556+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.796076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:49.977248+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.796076+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":223,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.160596+0000","last_change":"2024-02-23T00:45:22.272078+0000","last_active":"2024-02-23T00:45:30.160596+0000","last_peered":"2024-02-23T00:45:30.160596+0000","last_clean":"2024-02-23T00:45:30.160596+0000","last_became_active":"2024-02-23T00:39:14.788888+0000","last_became_peered":"2024-02-23T00:39:14.788888+0000","last_unstale":"2024-02-23T00:45:30.160596+0000","last_undegraded":"2024-02-23T00:45:30.160596+0000","last_fullsized":"2024-02-23T00:45:30.160596+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:22.272003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:22.272003+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":224,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.132883+0000","last_change":"2024-02-23T00:44:33.143663+0000","last_active":"2024-02-23T00:45:31.132883+0000","last_peered":"2024-02-23T00:45:31.132883+0000","last_clean":"2024-02-23T00:45:31.132883+0000","last_became_active":"2024-02-23T00:39:14.783489+0000","last_became_peered":"2024-02-23T00:39:14.783489+0000","last_unstale":"2024-02-23T00:45:31.132883+0000","last_undegraded":"2024-02-23T00:45:31.132883+0000","last_fullsized":"2024-02-23T00:45:31.132883+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:33.143592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:33.143592+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":235,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.161573+0000","last_change":"2024-02-23T00:44:20.958888+0000","last_active":"2024-02-23T00:45:30.161573+0000","last_peered":"2024-02-23T00:45:30.161573+0000","last_clean":"2024-02-23T00:45:30.161573+0000","last_became_active":"2024-02-23T00:43:42.168435+0000","last_became_peered":"2024-02-23T00:43:42.168435+0000","last_unstale":"2024-02-23T00:45:30.161573+0000","last_undegraded":"2024-02-23T00:45:30.161573+0000","last_fullsized":"2024-02-23T00:45:30.161573+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:20.958797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:20.958797+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":166,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.138959+0000","last_change":"2024-02-23T00:45:21.383725+0000","last_active":"2024-02-23T00:45:31.138959+0000","last_peered":"2024-02-23T00:45:31.138959+0000","last_clean":"2024-02-23T00:45:31.138959+0000","last_became_active":"2024-02-23T00:42:58.721439+0000","last_became_peered":"2024-02-23T00:42:58.721439+0000","last_unstale":"2024-02-23T00:45:31.138959+0000","last_undegraded":"2024-02-23T00:45:31.138959+0000","last_fullsized":"2024-02-23T00:45:31.138959+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:21.383626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:21.383626+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":163,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.447164+0000","last_change":"2024-02-23T00:45:17.763618+0000","last_active":"2024-02-23T00:45:30.447164+0000","last_peered":"2024-02-23T00:45:30.447164+0000","last_clean":"2024-02-23T00:45:30.447164+0000","last_became_active":"2024-02-23T00:42:58.701502+0000","last_became_peered":"2024-02-23T00:42:58.701502+0000","last_unstale":"2024-02-23T00:45:30.447164+0000","last_undegraded":"2024-02-23T00:45:30.447164+0000","last_fullsized":"2024-02-23T00:45:30.447164+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:17.763482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:17.763482+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":218,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.139415+0000","last_change":"2024-02-23T00:43:46.293962+0000","last_active":"2024-02-23T00:45:31.139415+0000","last_peered":"2024-02-23T00:45:31.139415+0000","last_clean":"2024-02-23T00:45:31.139415+0000","last_became_active":"2024-02-23T00:39:18.789330+0000","last_became_peered":"2024-02-23T00:39:18.789330+0000","last_unstale":"2024-02-23T00:45:31.139415+0000","last_undegraded":"2024-02-23T00:45:31.139415+0000","last_fullsized":"2024-02-23T00:45:31.139415+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:46.293864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.264240+0000","last_clean_scrub_stamp":"2024-02-23T00:43:46.293864+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"68'19","reported_seq":263,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.138274+0000","last_change":"2024-02-23T00:45:07.697475+0000","last_active":"2024-02-23T00:45:31.138274+0000","last_peered":"2024-02-23T00:45:31.138274+0000","last_clean":"2024-02-23T00:45:31.138274+0000","last_became_active":"2024-02-23T00:39:16.787484+0000","last_became_peered":"2024-02-23T00:39:16.787484+0000","last_unstale":"2024-02-23T00:45:31.138274+0000","last_undegraded":"2024-02-23T00:45:31.138274+0000","last_fullsized":"2024-02-23T00:45:31.138274+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'19","last_scrub_stamp":"2024-02-23T00:45:07.697394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:07.697394+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":222,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.137931+0000","last_change":"2024-02-23T00:43:54.305009+0000","last_active":"2024-02-23T00:45:31.137931+0000","last_peered":"2024-02-23T00:45:31.137931+0000","last_clean":"2024-02-23T00:45:31.137931+0000","last_became_active":"2024-02-23T00:39:14.772411+0000","last_became_peered":"2024-02-23T00:39:14.772411+0000","last_unstale":"2024-02-23T00:45:31.137931+0000","last_undegraded":"2024-02-23T00:45:31.137931+0000","last_fullsized":"2024-02-23T00:45:31.137931+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:54.304948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:54.304948+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"68'5","reported_seq":590,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.659515+0000","last_change":"2024-02-23T00:43:48.876428+0000","last_active":"2024-02-23T00:45:31.659515+0000","last_peered":"2024-02-23T00:45:31.659515+0000","last_clean":"2024-02-23T00:45:31.659515+0000","last_became_active":"2024-02-23T00:39:14.779109+0000","last_became_peered":"2024-02-23T00:39:14.779109+0000","last_unstale":"2024-02-23T00:45:31.659515+0000","last_undegraded":"2024-02-23T00:45:31.659515+0000","last_fullsized":"2024-02-23T00:45:31.659515+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'5","last_scrub_stamp":"2024-02-23T00:43:48.876356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.876356+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":369,"num_read_kb":364,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":214,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.449557+0000","last_change":"2024-02-23T00:43:47.794616+0000","last_active":"2024-02-23T00:45:30.449557+0000","last_peered":"2024-02-23T00:45:30.449557+0000","last_clean":"2024-02-23T00:45:30.449557+0000","last_became_active":"2024-02-23T00:39:21.160457+0000","last_became_peered":"2024-02-23T00:39:21.160457+0000","last_unstale":"2024-02-23T00:45:30.449557+0000","last_undegraded":"2024-02-23T00:45:30.449557+0000","last_fullsized":"2024-02-23T00:45:30.449557+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:47.794534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:47.794534+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":159,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.160510+0000","last_change":"2024-02-23T00:42:58.831449+0000","last_active":"2024-02-23T00:45:30.160510+0000","last_peered":"2024-02-23T00:45:30.160510+0000","last_clean":"2024-02-23T00:45:30.160510+0000","last_became_active":"2024-02-23T00:42:58.830815+0000","last_became_peered":"2024-02-23T00:42:58.830815+0000","last_unstale":"2024-02-23T00:45:30.160510+0000","last_undegraded":"2024-02-23T00:45:30.160510+0000","last_fullsized":"2024-02-23T00:45:30.160510+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":160,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.133121+0000","last_change":"2024-02-23T00:42:57.653669+0000","last_active":"2024-02-23T00:45:31.133121+0000","last_peered":"2024-02-23T00:45:31.133121+0000","last_clean":"2024-02-23T00:45:31.133121+0000","last_became_active":"2024-02-23T00:42:57.653355+0000","last_became_peered":"2024-02-23T00:42:57.653355+0000","last_unstale":"2024-02-23T00:45:31.133121+0000","last_undegraded":"2024-02-23T00:45:31.133121+0000","last_fullsized":"2024-02-23T00:45:31.133121+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":220,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.149193+0000","last_change":"2024-02-23T00:44:20.626116+0000","last_active":"2024-02-23T00:45:31.149193+0000","last_peered":"2024-02-23T00:45:31.149193+0000","last_clean":"2024-02-23T00:45:31.149193+0000","last_became_active":"2024-02-23T00:39:18.802134+0000","last_became_peered":"2024-02-23T00:39:18.802134+0000","last_unstale":"2024-02-23T00:45:31.149193+0000","last_undegraded":"2024-02-23T00:45:31.149193+0000","last_fullsized":"2024-02-23T00:45:31.149193+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:20.626052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:20.626052+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"86'35","reported_seq":275,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.160564+0000","last_change":"2024-02-23T00:42:47.988747+0000","last_active":"2024-02-23T00:45:30.160564+0000","last_peered":"2024-02-23T00:45:30.160564+0000","last_clean":"2024-02-23T00:45:30.160564+0000","last_became_active":"2024-02-23T00:40:42.933322+0000","last_became_peered":"2024-02-23T00:40:42.933322+0000","last_unstale":"2024-02-23T00:45:30.160564+0000","last_undegraded":"2024-02-23T00:45:30.160564+0000","last_fullsized":"2024-02-23T00:45:30.160564+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'35","last_scrub_stamp":"2024-02-23T00:42:47.988676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:47.988676+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":223,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.160298+0000","last_change":"2024-02-23T00:44:23.965228+0000","last_active":"2024-02-23T00:45:30.160298+0000","last_peered":"2024-02-23T00:45:30.160298+0000","last_clean":"2024-02-23T00:45:30.160298+0000","last_became_active":"2024-02-23T00:39:14.784015+0000","last_became_peered":"2024-02-23T00:39:14.784015+0000","last_unstale":"2024-02-23T00:45:30.160298+0000","last_undegraded":"2024-02-23T00:45:30.160298+0000","last_fullsized":"2024-02-23T00:45:30.160298+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:23.965125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.965125+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":231,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.137831+0000","last_change":"2024-02-23T00:43:45.144499+0000","last_active":"2024-02-23T00:45:31.137831+0000","last_peered":"2024-02-23T00:45:31.137831+0000","last_clean":"2024-02-23T00:45:31.137831+0000","last_became_active":"2024-02-23T00:43:45.144043+0000","last_became_peered":"2024-02-23T00:43:45.144043+0000","last_unstale":"2024-02-23T00:45:31.137831+0000","last_undegraded":"2024-02-23T00:45:31.137831+0000","last_fullsized":"2024-02-23T00:45:31.137831+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:40.062493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:40.062493+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":164,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.141110+0000","last_change":"2024-02-23T00:44:31.410733+0000","last_active":"2024-02-23T00:45:31.141110+0000","last_peered":"2024-02-23T00:45:31.141110+0000","last_clean":"2024-02-23T00:45:31.141110+0000","last_became_active":"2024-02-23T00:42:58.754813+0000","last_became_peered":"2024-02-23T00:42:58.754813+0000","last_unstale":"2024-02-23T00:45:31.141110+0000","last_undegraded":"2024-02-23T00:45:31.141110+0000","last_fullsized":"2024-02-23T00:45:31.141110+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:31.410670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:31.410670+0000","last_clean_scrub_stamp":"2024-02-23T00:44:31.410670+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":164,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.139632+0000","last_change":"2024-02-23T00:45:05.490595+0000","last_active":"2024-02-23T00:45:31.139632+0000","last_peered":"2024-02-23T00:45:31.139632+0000","last_clean":"2024-02-23T00:45:31.139632+0000","last_became_active":"2024-02-23T00:42:58.766579+0000","last_became_peered":"2024-02-23T00:42:58.766579+0000","last_unstale":"2024-02-23T00:45:31.139632+0000","last_undegraded":"2024-02-23T00:45:31.139632+0000","last_fullsized":"2024-02-23T00:45:31.139632+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:05.490531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:05.490531+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"91'9","reported_seq":304,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.149998+0000","last_change":"2024-02-23T00:44:28.619036+0000","last_active":"2024-02-23T00:45:31.149998+0000","last_peered":"2024-02-23T00:45:31.149998+0000","last_clean":"2024-02-23T00:45:31.149998+0000","last_became_active":"2024-02-23T00:39:18.801297+0000","last_became_peered":"2024-02-23T00:39:18.801297+0000","last_unstale":"2024-02-23T00:45:31.149998+0000","last_undegraded":"2024-02-23T00:45:31.149998+0000","last_fullsized":"2024-02-23T00:45:31.149998+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'9","last_scrub_stamp":"2024-02-23T00:44:28.618923+0000","last_deep_scrub":"91'9","last_deep_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_clean_scrub_stamp":"2024-02-23T00:44:28.618923+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"68'16","reported_seq":251,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.135622+0000","last_change":"2024-02-23T00:45:11.713800+0000","last_active":"2024-02-23T00:45:31.135622+0000","last_peered":"2024-02-23T00:45:31.135622+0000","last_clean":"2024-02-23T00:45:31.135622+0000","last_became_active":"2024-02-23T00:39:16.787468+0000","last_became_peered":"2024-02-23T00:39:16.787468+0000","last_unstale":"2024-02-23T00:45:31.135622+0000","last_undegraded":"2024-02-23T00:45:31.135622+0000","last_fullsized":"2024-02-23T00:45:31.135622+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:45:11.713731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:11.713731+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":223,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.133001+0000","last_change":"2024-02-23T00:45:19.635062+0000","last_active":"2024-02-23T00:45:31.133001+0000","last_peered":"2024-02-23T00:45:31.133001+0000","last_clean":"2024-02-23T00:45:31.133001+0000","last_became_active":"2024-02-23T00:39:14.779952+0000","last_became_peered":"2024-02-23T00:39:14.779952+0000","last_unstale":"2024-02-23T00:45:31.133001+0000","last_undegraded":"2024-02-23T00:45:31.133001+0000","last_fullsized":"2024-02-23T00:45:31.133001+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:19.635002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:19.635002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"107'2","reported_seq":220,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.146970+0000","last_change":"2024-02-23T00:45:08.662754+0000","last_active":"2024-02-23T00:45:31.146970+0000","last_peered":"2024-02-23T00:45:31.146970+0000","last_clean":"2024-02-23T00:45:31.146970+0000","last_became_active":"2024-02-23T00:39:20.800858+0000","last_became_peered":"2024-02-23T00:39:20.800858+0000","last_unstale":"2024-02-23T00:45:31.146970+0000","last_undegraded":"2024-02-23T00:45:31.146970+0000","last_fullsized":"2024-02-23T00:45:31.146970+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'2","last_scrub_stamp":"2024-02-23T00:45:08.662667+0000","last_deep_scrub":"107'2","last_deep_scrub_stamp":"2024-02-23T00:42:48.362739+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.662667+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":164,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.139808+0000","last_change":"2024-02-23T00:45:19.711196+0000","last_active":"2024-02-23T00:45:31.139808+0000","last_peered":"2024-02-23T00:45:31.139808+0000","last_clean":"2024-02-23T00:45:31.139808+0000","last_became_active":"2024-02-23T00:42:58.726861+0000","last_became_peered":"2024-02-23T00:42:58.726861+0000","last_unstale":"2024-02-23T00:45:31.139808+0000","last_undegraded":"2024-02-23T00:45:31.139808+0000","last_fullsized":"2024-02-23T00:45:31.139808+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:19.711117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:19.711117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":164,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.146070+0000","last_change":"2024-02-23T00:45:04.621794+0000","last_active":"2024-02-23T00:45:31.146070+0000","last_peered":"2024-02-23T00:45:31.146070+0000","last_clean":"2024-02-23T00:45:31.146070+0000","last_became_active":"2024-02-23T00:42:58.664282+0000","last_became_peered":"2024-02-23T00:42:58.664282+0000","last_unstale":"2024-02-23T00:45:31.146070+0000","last_undegraded":"2024-02-23T00:45:31.146070+0000","last_fullsized":"2024-02-23T00:45:31.146070+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:04.621714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.621714+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":218,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.137904+0000","last_change":"2024-02-23T00:43:51.261426+0000","last_active":"2024-02-23T00:45:31.137904+0000","last_peered":"2024-02-23T00:45:31.137904+0000","last_clean":"2024-02-23T00:45:31.137904+0000","last_became_active":"2024-02-23T00:39:18.796323+0000","last_became_peered":"2024-02-23T00:39:18.796323+0000","last_unstale":"2024-02-23T00:45:31.137904+0000","last_undegraded":"2024-02-23T00:45:31.137904+0000","last_fullsized":"2024-02-23T00:45:31.137904+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:51.261367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_clean_scrub_stamp":"2024-02-23T00:43:51.261367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"68'25","reported_seq":270,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.144907+0000","last_change":"2024-02-23T00:43:46.158231+0000","last_active":"2024-02-23T00:45:31.144907+0000","last_peered":"2024-02-23T00:45:31.144907+0000","last_clean":"2024-02-23T00:45:31.144907+0000","last_became_active":"2024-02-23T00:39:16.783926+0000","last_became_peered":"2024-02-23T00:39:16.783926+0000","last_unstale":"2024-02-23T00:45:31.144907+0000","last_undegraded":"2024-02-23T00:45:31.144907+0000","last_fullsized":"2024-02-23T00:45:31.144907+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'25","last_scrub_stamp":"2024-02-23T00:43:46.158149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:46.158149+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"95'4","reported_seq":238,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.148391+0000","last_change":"2024-02-23T00:44:31.106105+0000","last_active":"2024-02-23T00:45:31.148391+0000","last_peered":"2024-02-23T00:45:31.148391+0000","last_clean":"2024-02-23T00:45:31.148391+0000","last_became_active":"2024-02-23T00:39:14.779205+0000","last_became_peered":"2024-02-23T00:39:14.779205+0000","last_unstale":"2024-02-23T00:45:31.148391+0000","last_undegraded":"2024-02-23T00:45:31.148391+0000","last_fullsized":"2024-02-23T00:45:31.148391+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'4","last_scrub_stamp":"2024-02-23T00:44:31.106012+0000","last_deep_scrub":"95'4","last_deep_scrub_stamp":"2024-02-23T00:41:43.874397+0000","last_clean_scrub_stamp":"2024-02-23T00:44:31.106012+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":236,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.133802+0000","last_change":"2024-02-23T00:43:52.036315+0000","last_active":"2024-02-23T00:45:31.133802+0000","last_peered":"2024-02-23T00:45:31.133802+0000","last_clean":"2024-02-23T00:45:31.133802+0000","last_became_active":"2024-02-23T00:43:27.611660+0000","last_became_peered":"2024-02-23T00:43:27.611660+0000","last_unstale":"2024-02-23T00:45:31.133802+0000","last_undegraded":"2024-02-23T00:45:31.133802+0000","last_fullsized":"2024-02-23T00:45:31.133802+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:52.036249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:52.036249+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":164,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.139709+0000","last_change":"2024-02-23T00:45:16.389060+0000","last_active":"2024-02-23T00:45:31.139709+0000","last_peered":"2024-02-23T00:45:31.139709+0000","last_clean":"2024-02-23T00:45:31.139709+0000","last_became_active":"2024-02-23T00:42:58.809926+0000","last_became_peered":"2024-02-23T00:42:58.809926+0000","last_unstale":"2024-02-23T00:45:31.139709+0000","last_undegraded":"2024-02-23T00:45:31.139709+0000","last_fullsized":"2024-02-23T00:45:31.139709+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.388988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.388988+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":163,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.449052+0000","last_change":"2024-02-23T00:45:20.829124+0000","last_active":"2024-02-23T00:45:30.449052+0000","last_peered":"2024-02-23T00:45:30.449052+0000","last_clean":"2024-02-23T00:45:30.449052+0000","last_became_active":"2024-02-23T00:42:58.744983+0000","last_became_peered":"2024-02-23T00:42:58.744983+0000","last_unstale":"2024-02-23T00:45:30.449052+0000","last_undegraded":"2024-02-23T00:45:30.449052+0000","last_fullsized":"2024-02-23T00:45:30.449052+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:20.829052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:20.829052+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"68'21","reported_seq":263,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.149681+0000","last_change":"2024-02-23T00:43:52.309599+0000","last_active":"2024-02-23T00:45:31.149681+0000","last_peered":"2024-02-23T00:45:31.149681+0000","last_clean":"2024-02-23T00:45:31.149681+0000","last_became_active":"2024-02-23T00:39:16.858760+0000","last_became_peered":"2024-02-23T00:39:16.858760+0000","last_unstale":"2024-02-23T00:45:31.149681+0000","last_undegraded":"2024-02-23T00:45:31.149681+0000","last_fullsized":"2024-02-23T00:45:31.149681+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'21","last_scrub_stamp":"2024-02-23T00:43:52.309532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:52.309532+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":227,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.137816+0000","last_change":"2024-02-23T00:45:22.376128+0000","last_active":"2024-02-23T00:45:31.137816+0000","last_peered":"2024-02-23T00:45:31.137816+0000","last_clean":"2024-02-23T00:45:31.137816+0000","last_became_active":"2024-02-23T00:39:14.774244+0000","last_became_peered":"2024-02-23T00:39:14.774244+0000","last_unstale":"2024-02-23T00:45:31.137816+0000","last_undegraded":"2024-02-23T00:45:31.137816+0000","last_fullsized":"2024-02-23T00:45:31.137816+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:22.376063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:22.376063+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":220,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.139606+0000","last_change":"2024-02-23T00:43:44.951800+0000","last_active":"2024-02-23T00:45:31.139606+0000","last_peered":"2024-02-23T00:45:31.139606+0000","last_clean":"2024-02-23T00:45:31.139606+0000","last_became_active":"2024-02-23T00:39:18.798624+0000","last_became_peered":"2024-02-23T00:39:18.798624+0000","last_unstale":"2024-02-23T00:45:31.139606+0000","last_undegraded":"2024-02-23T00:45:31.139606+0000","last_fullsized":"2024-02-23T00:45:31.139606+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:44.951696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:46.998875+0000","last_clean_scrub_stamp":"2024-02-23T00:43:44.951696+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":216,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.140439+0000","last_change":"2024-02-23T00:44:26.432766+0000","last_active":"2024-02-23T00:45:31.140439+0000","last_peered":"2024-02-23T00:45:31.140439+0000","last_clean":"2024-02-23T00:45:31.140439+0000","last_became_active":"2024-02-23T00:39:20.796041+0000","last_became_peered":"2024-02-23T00:39:20.796041+0000","last_unstale":"2024-02-23T00:45:31.140439+0000","last_undegraded":"2024-02-23T00:45:31.140439+0000","last_fullsized":"2024-02-23T00:45:31.140439+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:26.432703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:26.432703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1d","version":"110'7","reported_seq":172,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.146066+0000","last_change":"2024-02-23T00:45:12.307598+0000","last_active":"2024-02-23T00:45:31.146066+0000","last_peered":"2024-02-23T00:45:31.146066+0000","last_clean":"2024-02-23T00:45:31.146066+0000","last_became_active":"2024-02-23T00:42:58.754763+0000","last_became_peered":"2024-02-23T00:42:58.754763+0000","last_unstale":"2024-02-23T00:45:31.146066+0000","last_undegraded":"2024-02-23T00:45:31.146066+0000","last_fullsized":"2024-02-23T00:45:31.146066+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'7","last_scrub_stamp":"2024-02-23T00:45:12.307531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.307531+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":165,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.147207+0000","last_change":"2024-02-23T00:44:26.595443+0000","last_active":"2024-02-23T00:45:31.147207+0000","last_peered":"2024-02-23T00:45:31.147207+0000","last_clean":"2024-02-23T00:45:31.147207+0000","last_became_active":"2024-02-23T00:42:58.825015+0000","last_became_peered":"2024-02-23T00:42:58.825015+0000","last_unstale":"2024-02-23T00:45:31.147207+0000","last_undegraded":"2024-02-23T00:45:31.147207+0000","last_fullsized":"2024-02-23T00:45:31.147207+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:26.595346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:44:26.595346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"107'16","reported_seq":250,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.139523+0000","last_change":"2024-02-23T00:45:07.012086+0000","last_active":"2024-02-23T00:45:31.139523+0000","last_peered":"2024-02-23T00:45:31.139523+0000","last_clean":"2024-02-23T00:45:31.139523+0000","last_became_active":"2024-02-23T00:39:16.858449+0000","last_became_peered":"2024-02-23T00:39:16.858449+0000","last_unstale":"2024-02-23T00:45:31.139523+0000","last_undegraded":"2024-02-23T00:45:31.139523+0000","last_fullsized":"2024-02-23T00:45:31.139523+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'16","last_scrub_stamp":"2024-02-23T00:45:07.011956+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-02-23T00:42:52.964510+0000","last_clean_scrub_stamp":"2024-02-23T00:45:07.011956+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":222,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.139405+0000","last_change":"2024-02-23T00:45:13.613736+0000","last_active":"2024-02-23T00:45:31.139405+0000","last_peered":"2024-02-23T00:45:31.139405+0000","last_clean":"2024-02-23T00:45:31.139405+0000","last_became_active":"2024-02-23T00:39:14.783164+0000","last_became_peered":"2024-02-23T00:39:14.783164+0000","last_unstale":"2024-02-23T00:45:31.139405+0000","last_undegraded":"2024-02-23T00:45:31.139405+0000","last_fullsized":"2024-02-23T00:45:31.139405+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:13.613654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:13.613654+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":219,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.160892+0000","last_change":"2024-02-23T00:43:40.971316+0000","last_active":"2024-02-23T00:45:30.160892+0000","last_peered":"2024-02-23T00:45:30.160892+0000","last_clean":"2024-02-23T00:45:30.160892+0000","last_became_active":"2024-02-23T00:39:18.800297+0000","last_became_peered":"2024-02-23T00:39:18.800297+0000","last_unstale":"2024-02-23T00:45:30.160892+0000","last_undegraded":"2024-02-23T00:45:30.160892+0000","last_fullsized":"2024-02-23T00:45:30.160892+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:40.971243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:40.971243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":214,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.449430+0000","last_change":"2024-02-23T00:43:46.760431+0000","last_active":"2024-02-23T00:45:30.449430+0000","last_peered":"2024-02-23T00:45:30.449430+0000","last_clean":"2024-02-23T00:45:30.449430+0000","last_became_active":"2024-02-23T00:39:21.162036+0000","last_became_peered":"2024-02-23T00:39:21.162036+0000","last_unstale":"2024-02-23T00:45:30.449430+0000","last_undegraded":"2024-02-23T00:45:30.449430+0000","last_fullsized":"2024-02-23T00:45:30.449430+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:46.760348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:46.760348+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":165,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.134364+0000","last_change":"2024-02-23T00:45:18.644663+0000","last_active":"2024-02-23T00:45:31.134364+0000","last_peered":"2024-02-23T00:45:31.134364+0000","last_clean":"2024-02-23T00:45:31.134364+0000","last_became_active":"2024-02-23T00:42:57.648267+0000","last_became_peered":"2024-02-23T00:42:57.648267+0000","last_unstale":"2024-02-23T00:45:31.134364+0000","last_undegraded":"2024-02-23T00:45:31.134364+0000","last_fullsized":"2024-02-23T00:45:31.134364+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:18.644538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:18.644538+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1","version":"110'7","reported_seq":178,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.133924+0000","last_change":"2024-02-23T00:45:18.947859+0000","last_active":"2024-02-23T00:45:31.133924+0000","last_peered":"2024-02-23T00:45:31.133924+0000","last_clean":"2024-02-23T00:45:31.133924+0000","last_became_active":"2024-02-23T00:42:58.777251+0000","last_became_peered":"2024-02-23T00:42:58.777251+0000","last_unstale":"2024-02-23T00:45:31.133924+0000","last_undegraded":"2024-02-23T00:45:31.133924+0000","last_fullsized":"2024-02-23T00:45:31.133924+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'7","last_scrub_stamp":"2024-02-23T00:45:18.947746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:18.947746+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"68'22","reported_seq":190,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.140252+0000","last_change":"2024-02-23T00:44:31.391929+0000","last_active":"2024-02-23T00:45:31.140252+0000","last_peered":"2024-02-23T00:45:31.140252+0000","last_clean":"2024-02-23T00:45:31.140252+0000","last_became_active":"2024-02-23T00:42:43.521189+0000","last_became_peered":"2024-02-23T00:42:43.521189+0000","last_unstale":"2024-02-23T00:45:31.140252+0000","last_undegraded":"2024-02-23T00:45:31.140252+0000","last_fullsized":"2024-02-23T00:45:31.140252+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'22","last_scrub_stamp":"2024-02-23T00:44:31.391819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:44:31.391819+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":37,"num_read_kb":30,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,7],"acting":[2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":223,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.160105+0000","last_change":"2024-02-23T00:44:18.946624+0000","last_active":"2024-02-23T00:45:30.160105+0000","last_peered":"2024-02-23T00:45:30.160105+0000","last_clean":"2024-02-23T00:45:30.160105+0000","last_became_active":"2024-02-23T00:39:14.785590+0000","last_became_peered":"2024-02-23T00:39:14.785590+0000","last_unstale":"2024-02-23T00:45:30.160105+0000","last_undegraded":"2024-02-23T00:45:30.160105+0000","last_fullsized":"2024-02-23T00:45:30.160105+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:18.946560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_clean_scrub_stamp":"2024-02-23T00:44:18.946560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"67'8","reported_seq":302,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.161235+0000","last_change":"2024-02-23T00:44:29.038226+0000","last_active":"2024-02-23T00:45:30.161235+0000","last_peered":"2024-02-23T00:45:30.161235+0000","last_clean":"2024-02-23T00:45:30.161235+0000","last_became_active":"2024-02-23T00:39:18.803566+0000","last_became_peered":"2024-02-23T00:39:18.803566+0000","last_unstale":"2024-02-23T00:45:30.161235+0000","last_undegraded":"2024-02-23T00:45:30.161235+0000","last_fullsized":"2024-02-23T00:45:30.161235+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:44:29.038134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:29.038134+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"254'6","reported_seq":171,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.139841+0000","last_change":"2024-02-23T00:45:26.935006+0000","last_active":"2024-02-23T00:45:31.139841+0000","last_peered":"2024-02-23T00:45:31.139841+0000","last_clean":"2024-02-23T00:45:31.139841+0000","last_became_active":"2024-02-23T00:42:58.758349+0000","last_became_peered":"2024-02-23T00:42:58.758349+0000","last_unstale":"2024-02-23T00:45:31.139841+0000","last_undegraded":"2024-02-23T00:45:31.139841+0000","last_fullsized":"2024-02-23T00:45:31.139841+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"254'6","last_scrub_stamp":"2024-02-23T00:45:26.934947+0000","last_deep_scrub":"254'6","last_deep_scrub_stamp":"2024-02-23T00:45:26.934947+0000","last_clean_scrub_stamp":"2024-02-23T00:45:26.934947+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":158,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.447201+0000","last_change":"2024-02-23T00:42:58.766295+0000","last_active":"2024-02-23T00:45:30.447201+0000","last_peered":"2024-02-23T00:45:30.447201+0000","last_clean":"2024-02-23T00:45:30.447201+0000","last_became_active":"2024-02-23T00:42:58.757424+0000","last_became_peered":"2024-02-23T00:42:58.757424+0000","last_unstale":"2024-02-23T00:45:30.447201+0000","last_undegraded":"2024-02-23T00:45:30.447201+0000","last_fullsized":"2024-02-23T00:45:30.447201+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"68'15","reported_seq":252,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.161157+0000","last_change":"2024-02-23T00:44:25.998879+0000","last_active":"2024-02-23T00:45:30.161157+0000","last_peered":"2024-02-23T00:45:30.161157+0000","last_clean":"2024-02-23T00:45:30.161157+0000","last_became_active":"2024-02-23T00:39:16.858953+0000","last_became_peered":"2024-02-23T00:39:16.858953+0000","last_unstale":"2024-02-23T00:45:30.161157+0000","last_undegraded":"2024-02-23T00:45:30.161157+0000","last_fullsized":"2024-02-23T00:45:30.161157+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:44:25.998804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:44:25.998804+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":224,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.149874+0000","last_change":"2024-02-23T00:43:45.309194+0000","last_active":"2024-02-23T00:45:31.149874+0000","last_peered":"2024-02-23T00:45:31.149874+0000","last_clean":"2024-02-23T00:45:31.149874+0000","last_became_active":"2024-02-23T00:39:14.773772+0000","last_became_peered":"2024-02-23T00:39:14.773772+0000","last_unstale":"2024-02-23T00:45:31.149874+0000","last_undegraded":"2024-02-23T00:45:31.149874+0000","last_fullsized":"2024-02-23T00:45:31.149874+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:45.309083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.309083+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"67'8","reported_seq":298,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.134949+0000","last_change":"2024-02-23T00:43:39.086199+0000","last_active":"2024-02-23T00:45:31.134949+0000","last_peered":"2024-02-23T00:45:31.134949+0000","last_clean":"2024-02-23T00:45:31.134949+0000","last_became_active":"2024-02-23T00:39:18.787313+0000","last_became_peered":"2024-02-23T00:39:18.787313+0000","last_unstale":"2024-02-23T00:45:31.134949+0000","last_undegraded":"2024-02-23T00:45:31.134949+0000","last_fullsized":"2024-02-23T00:45:31.134949+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:43:39.086107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:39.086107+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":164,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.138878+0000","last_change":"2024-02-23T00:44:34.437892+0000","last_active":"2024-02-23T00:45:31.138878+0000","last_peered":"2024-02-23T00:45:31.138878+0000","last_clean":"2024-02-23T00:45:31.138878+0000","last_became_active":"2024-02-23T00:42:58.745471+0000","last_became_peered":"2024-02-23T00:42:58.745471+0000","last_unstale":"2024-02-23T00:45:31.138878+0000","last_undegraded":"2024-02-23T00:45:31.138878+0000","last_fullsized":"2024-02-23T00:45:31.138878+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:34.437785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:34.437785+0000","last_clean_scrub_stamp":"2024-02-23T00:44:34.437785+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":160,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.134632+0000","last_change":"2024-02-23T00:42:57.652853+0000","last_active":"2024-02-23T00:45:31.134632+0000","last_peered":"2024-02-23T00:45:31.134632+0000","last_clean":"2024-02-23T00:45:31.134632+0000","last_became_active":"2024-02-23T00:42:57.652488+0000","last_became_peered":"2024-02-23T00:42:57.652488+0000","last_unstale":"2024-02-23T00:45:31.134632+0000","last_undegraded":"2024-02-23T00:45:31.134632+0000","last_fullsized":"2024-02-23T00:45:31.134632+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"67'9","reported_seq":232,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.135385+0000","last_change":"2024-02-23T00:42:57.705378+0000","last_active":"2024-02-23T00:45:31.135385+0000","last_peered":"2024-02-23T00:45:31.135385+0000","last_clean":"2024-02-23T00:45:31.135385+0000","last_became_active":"2024-02-23T00:39:16.786936+0000","last_became_peered":"2024-02-23T00:39:16.786936+0000","last_unstale":"2024-02-23T00:45:31.135385+0000","last_undegraded":"2024-02-23T00:45:31.135385+0000","last_fullsized":"2024-02-23T00:45:31.135385+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'9","last_scrub_stamp":"2024-02-23T00:42:57.705261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.705261+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":227,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.143231+0000","last_change":"2024-02-23T00:44:24.350861+0000","last_active":"2024-02-23T00:45:31.143231+0000","last_peered":"2024-02-23T00:45:31.143231+0000","last_clean":"2024-02-23T00:45:31.143231+0000","last_became_active":"2024-02-23T00:39:14.778455+0000","last_became_peered":"2024-02-23T00:39:14.778455+0000","last_unstale":"2024-02-23T00:45:31.143231+0000","last_undegraded":"2024-02-23T00:45:31.143231+0000","last_fullsized":"2024-02-23T00:45:31.143231+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:24.350797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:24.350797+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":220,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.133717+0000","last_change":"2024-02-23T00:44:37.105079+0000","last_active":"2024-02-23T00:45:31.133717+0000","last_peered":"2024-02-23T00:45:31.133717+0000","last_clean":"2024-02-23T00:45:31.133717+0000","last_became_active":"2024-02-23T00:39:18.799052+0000","last_became_peered":"2024-02-23T00:39:18.799052+0000","last_unstale":"2024-02-23T00:45:31.133717+0000","last_undegraded":"2024-02-23T00:45:31.133717+0000","last_fullsized":"2024-02-23T00:45:31.133717+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:37.105013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_clean_scrub_stamp":"2024-02-23T00:44:37.105013+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":213,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.146893+0000","last_change":"2024-02-23T00:42:50.425748+0000","last_active":"2024-02-23T00:45:31.146893+0000","last_peered":"2024-02-23T00:45:31.146893+0000","last_clean":"2024-02-23T00:45:31.146893+0000","last_became_active":"2024-02-23T00:39:20.802648+0000","last_became_peered":"2024-02-23T00:39:20.802648+0000","last_unstale":"2024-02-23T00:45:31.146893+0000","last_undegraded":"2024-02-23T00:45:31.146893+0000","last_fullsized":"2024-02-23T00:45:31.146893+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:50.425641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:50.425641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":165,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.144537+0000","last_change":"2024-02-23T00:45:15.352093+0000","last_active":"2024-02-23T00:45:31.144537+0000","last_peered":"2024-02-23T00:45:31.144537+0000","last_clean":"2024-02-23T00:45:31.144537+0000","last_became_active":"2024-02-23T00:42:58.833531+0000","last_became_peered":"2024-02-23T00:42:58.833531+0000","last_unstale":"2024-02-23T00:45:31.144537+0000","last_undegraded":"2024-02-23T00:45:31.144537+0000","last_fullsized":"2024-02-23T00:45:31.144537+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:15.352027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:15.352027+0000","last_clean_scrub_stamp":"2024-02-23T00:45:15.352027+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":165,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.149011+0000","last_change":"2024-02-23T00:44:23.543049+0000","last_active":"2024-02-23T00:45:31.149011+0000","last_peered":"2024-02-23T00:45:31.149011+0000","last_clean":"2024-02-23T00:45:31.149011+0000","last_became_active":"2024-02-23T00:42:58.835862+0000","last_became_peered":"2024-02-23T00:42:58.835862+0000","last_unstale":"2024-02-23T00:45:31.149011+0000","last_undegraded":"2024-02-23T00:45:31.149011+0000","last_fullsized":"2024-02-23T00:45:31.149011+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:23.542975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.542975+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"67'11","reported_seq":240,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.147029+0000","last_change":"2024-02-23T00:45:13.315510+0000","last_active":"2024-02-23T00:45:31.147029+0000","last_peered":"2024-02-23T00:45:31.147029+0000","last_clean":"2024-02-23T00:45:31.147029+0000","last_became_active":"2024-02-23T00:39:16.783896+0000","last_became_peered":"2024-02-23T00:39:16.783896+0000","last_unstale":"2024-02-23T00:45:31.147029+0000","last_undegraded":"2024-02-23T00:45:31.147029+0000","last_fullsized":"2024-02-23T00:45:31.147029+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'11","last_scrub_stamp":"2024-02-23T00:45:13.315434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:13.315434+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"202'2","reported_seq":231,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.135822+0000","last_change":"2024-02-23T00:44:34.720705+0000","last_active":"2024-02-23T00:45:31.135822+0000","last_peered":"2024-02-23T00:45:31.135822+0000","last_clean":"2024-02-23T00:45:31.135822+0000","last_became_active":"2024-02-23T00:39:14.775804+0000","last_became_peered":"2024-02-23T00:39:14.775804+0000","last_unstale":"2024-02-23T00:45:31.135822+0000","last_undegraded":"2024-02-23T00:45:31.135822+0000","last_fullsized":"2024-02-23T00:45:31.135822+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"202'2","last_scrub_stamp":"2024-02-23T00:44:34.720640+0000","last_deep_scrub":"202'2","last_deep_scrub_stamp":"2024-02-23T00:44:34.720640+0000","last_clean_scrub_stamp":"2024-02-23T00:44:34.720640+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":214,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.135461+0000","last_change":"2024-02-23T00:43:31.903157+0000","last_active":"2024-02-23T00:45:31.135461+0000","last_peered":"2024-02-23T00:45:31.135461+0000","last_clean":"2024-02-23T00:45:31.135461+0000","last_became_active":"2024-02-23T00:39:18.800382+0000","last_became_peered":"2024-02-23T00:39:18.800382+0000","last_unstale":"2024-02-23T00:45:31.135461+0000","last_undegraded":"2024-02-23T00:45:31.135461+0000","last_fullsized":"2024-02-23T00:45:31.135461+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:31.903031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:31.903031+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":218,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.132074+0000","last_change":"2024-02-23T00:44:23.152677+0000","last_active":"2024-02-23T00:45:31.132074+0000","last_peered":"2024-02-23T00:45:31.132074+0000","last_clean":"2024-02-23T00:45:31.132074+0000","last_became_active":"2024-02-23T00:39:20.799053+0000","last_became_peered":"2024-02-23T00:39:20.799053+0000","last_unstale":"2024-02-23T00:45:31.132074+0000","last_undegraded":"2024-02-23T00:45:31.132074+0000","last_fullsized":"2024-02-23T00:45:31.132074+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:23.152602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.152602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"110'2","reported_seq":167,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.132178+0000","last_change":"2024-02-23T00:45:09.917568+0000","last_active":"2024-02-23T00:45:31.132178+0000","last_peered":"2024-02-23T00:45:31.132178+0000","last_clean":"2024-02-23T00:45:31.132178+0000","last_became_active":"2024-02-23T00:42:58.698725+0000","last_became_peered":"2024-02-23T00:42:58.698725+0000","last_unstale":"2024-02-23T00:45:31.132178+0000","last_undegraded":"2024-02-23T00:45:31.132178+0000","last_fullsized":"2024-02-23T00:45:31.132178+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'2","last_scrub_stamp":"2024-02-23T00:45:09.917420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:09.917420+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":160,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.135716+0000","last_change":"2024-02-23T00:42:57.652742+0000","last_active":"2024-02-23T00:45:31.135716+0000","last_peered":"2024-02-23T00:45:31.135716+0000","last_clean":"2024-02-23T00:45:31.135716+0000","last_became_active":"2024-02-23T00:42:57.652438+0000","last_became_peered":"2024-02-23T00:42:57.652438+0000","last_unstale":"2024-02-23T00:45:31.135716+0000","last_undegraded":"2024-02-23T00:45:31.135716+0000","last_fullsized":"2024-02-23T00:45:31.135716+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"67'4","reported_seq":228,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.132077+0000","last_change":"2024-02-23T00:45:20.967211+0000","last_active":"2024-02-23T00:45:31.132077+0000","last_peered":"2024-02-23T00:45:31.132077+0000","last_clean":"2024-02-23T00:45:31.132077+0000","last_became_active":"2024-02-23T00:39:16.785830+0000","last_became_peered":"2024-02-23T00:39:16.785830+0000","last_unstale":"2024-02-23T00:45:31.132077+0000","last_undegraded":"2024-02-23T00:45:31.132077+0000","last_fullsized":"2024-02-23T00:45:31.132077+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'4","last_scrub_stamp":"2024-02-23T00:45:20.967095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:20.967095+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":223,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.160808+0000","last_change":"2024-02-23T00:43:38.947741+0000","last_active":"2024-02-23T00:45:30.160808+0000","last_peered":"2024-02-23T00:45:30.160808+0000","last_clean":"2024-02-23T00:45:30.160808+0000","last_became_active":"2024-02-23T00:39:14.783325+0000","last_became_peered":"2024-02-23T00:39:14.783325+0000","last_unstale":"2024-02-23T00:45:30.160808+0000","last_undegraded":"2024-02-23T00:45:30.160808+0000","last_fullsized":"2024-02-23T00:45:30.160808+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:38.947675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:38.947675+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":223,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.139325+0000","last_change":"2024-02-23T00:45:20.761547+0000","last_active":"2024-02-23T00:45:31.139325+0000","last_peered":"2024-02-23T00:45:31.139325+0000","last_clean":"2024-02-23T00:45:31.139325+0000","last_became_active":"2024-02-23T00:39:18.797287+0000","last_became_peered":"2024-02-23T00:39:18.797287+0000","last_unstale":"2024-02-23T00:45:31.139325+0000","last_undegraded":"2024-02-23T00:45:31.139325+0000","last_fullsized":"2024-02-23T00:45:31.139325+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:20.761440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:20.761440+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":214,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.446973+0000","last_change":"2024-02-23T00:45:10.741762+0000","last_active":"2024-02-23T00:45:30.446973+0000","last_peered":"2024-02-23T00:45:30.446973+0000","last_clean":"2024-02-23T00:45:30.446973+0000","last_became_active":"2024-02-23T00:39:20.795351+0000","last_became_peered":"2024-02-23T00:39:20.795351+0000","last_unstale":"2024-02-23T00:45:30.446973+0000","last_undegraded":"2024-02-23T00:45:30.446973+0000","last_fullsized":"2024-02-23T00:45:30.446973+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.741694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.741694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1b","version":"110'10","reported_seq":173,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.446861+0000","last_change":"2024-02-23T00:45:14.710146+0000","last_active":"2024-02-23T00:45:30.446861+0000","last_peered":"2024-02-23T00:45:30.446861+0000","last_clean":"2024-02-23T00:45:30.446861+0000","last_became_active":"2024-02-23T00:42:58.749126+0000","last_became_peered":"2024-02-23T00:42:58.749126+0000","last_unstale":"2024-02-23T00:45:30.446861+0000","last_undegraded":"2024-02-23T00:45:30.446861+0000","last_fullsized":"2024-02-23T00:45:30.446861+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'10","last_scrub_stamp":"2024-02-23T00:45:14.710075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.710075+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":158,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.449226+0000","last_change":"2024-02-23T00:42:58.741674+0000","last_active":"2024-02-23T00:45:30.449226+0000","last_peered":"2024-02-23T00:45:30.449226+0000","last_clean":"2024-02-23T00:45:30.449226+0000","last_became_active":"2024-02-23T00:42:58.741365+0000","last_became_peered":"2024-02-23T00:42:58.741365+0000","last_unstale":"2024-02-23T00:45:30.449226+0000","last_undegraded":"2024-02-23T00:45:30.449226+0000","last_fullsized":"2024-02-23T00:45:30.449226+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"68'17","reported_seq":256,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.134137+0000","last_change":"2024-02-23T00:43:43.997579+0000","last_active":"2024-02-23T00:45:31.134137+0000","last_peered":"2024-02-23T00:45:31.134137+0000","last_clean":"2024-02-23T00:45:31.134137+0000","last_became_active":"2024-02-23T00:39:16.857232+0000","last_became_peered":"2024-02-23T00:39:16.857232+0000","last_unstale":"2024-02-23T00:45:31.134137+0000","last_undegraded":"2024-02-23T00:45:31.134137+0000","last_fullsized":"2024-02-23T00:45:31.134137+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:43:43.997452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:43.997452+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":222,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.142114+0000","last_change":"2024-02-23T00:43:52.419206+0000","last_active":"2024-02-23T00:45:31.142114+0000","last_peered":"2024-02-23T00:45:31.142114+0000","last_clean":"2024-02-23T00:45:31.142114+0000","last_became_active":"2024-02-23T00:39:14.784092+0000","last_became_peered":"2024-02-23T00:39:14.784092+0000","last_unstale":"2024-02-23T00:45:31.142114+0000","last_undegraded":"2024-02-23T00:45:31.142114+0000","last_fullsized":"2024-02-23T00:45:31.142114+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:52.419100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:52.419100+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":216,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.446950+0000","last_change":"2024-02-23T00:44:25.797288+0000","last_active":"2024-02-23T00:45:30.446950+0000","last_peered":"2024-02-23T00:45:30.446950+0000","last_clean":"2024-02-23T00:45:30.446950+0000","last_became_active":"2024-02-23T00:39:18.791457+0000","last_became_peered":"2024-02-23T00:39:18.791457+0000","last_unstale":"2024-02-23T00:45:30.446950+0000","last_undegraded":"2024-02-23T00:45:30.446950+0000","last_fullsized":"2024-02-23T00:45:30.446950+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:25.797184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:25.797184+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":213,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.133891+0000","last_change":"2024-02-23T00:43:45.965636+0000","last_active":"2024-02-23T00:45:31.133891+0000","last_peered":"2024-02-23T00:45:31.133891+0000","last_clean":"2024-02-23T00:45:31.133891+0000","last_became_active":"2024-02-23T00:39:21.160376+0000","last_became_peered":"2024-02-23T00:39:21.160376+0000","last_unstale":"2024-02-23T00:45:31.133891+0000","last_undegraded":"2024-02-23T00:45:31.133891+0000","last_fullsized":"2024-02-23T00:45:31.133891+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:45.965570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.965570+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1a","version":"110'5","reported_seq":164,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.159946+0000","last_change":"2024-02-23T00:42:58.738588+0000","last_active":"2024-02-23T00:45:30.159946+0000","last_peered":"2024-02-23T00:45:30.159946+0000","last_clean":"2024-02-23T00:45:30.159946+0000","last_became_active":"2024-02-23T00:42:58.736706+0000","last_became_peered":"2024-02-23T00:42:58.736706+0000","last_unstale":"2024-02-23T00:45:30.159946+0000","last_undegraded":"2024-02-23T00:45:30.159946+0000","last_fullsized":"2024-02-23T00:45:30.159946+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":164,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.141461+0000","last_change":"2024-02-23T00:45:23.378334+0000","last_active":"2024-02-23T00:45:31.141461+0000","last_peered":"2024-02-23T00:45:31.141461+0000","last_clean":"2024-02-23T00:45:31.141461+0000","last_became_active":"2024-02-23T00:42:58.702580+0000","last_became_peered":"2024-02-23T00:42:58.702580+0000","last_unstale":"2024-02-23T00:45:31.141461+0000","last_undegraded":"2024-02-23T00:45:31.141461+0000","last_fullsized":"2024-02-23T00:45:31.141461+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:23.378270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.378270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"68'16","reported_seq":252,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.150063+0000","last_change":"2024-02-23T00:45:04.672612+0000","last_active":"2024-02-23T00:45:31.150063+0000","last_peered":"2024-02-23T00:45:31.150063+0000","last_clean":"2024-02-23T00:45:31.150063+0000","last_became_active":"2024-02-23T00:39:16.858945+0000","last_became_peered":"2024-02-23T00:39:16.858945+0000","last_unstale":"2024-02-23T00:45:31.150063+0000","last_undegraded":"2024-02-23T00:45:31.150063+0000","last_fullsized":"2024-02-23T00:45:31.150063+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:45:04.672537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.672537+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":230,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.159860+0000","last_change":"2024-02-23T00:43:44.031862+0000","last_active":"2024-02-23T00:45:30.159860+0000","last_peered":"2024-02-23T00:45:30.159860+0000","last_clean":"2024-02-23T00:45:30.159860+0000","last_became_active":"2024-02-23T00:39:14.785363+0000","last_became_peered":"2024-02-23T00:39:14.785363+0000","last_unstale":"2024-02-23T00:45:30.159860+0000","last_undegraded":"2024-02-23T00:45:30.159860+0000","last_fullsized":"2024-02-23T00:45:30.159860+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"80'2","last_scrub_stamp":"2024-02-23T00:43:44.031793+0000","last_deep_scrub":"80'2","last_deep_scrub_stamp":"2024-02-23T00:40:15.942725+0000","last_clean_scrub_stamp":"2024-02-23T00:43:44.031793+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"93'9","reported_seq":305,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.141612+0000","last_change":"2024-02-23T00:43:50.393789+0000","last_active":"2024-02-23T00:45:31.141612+0000","last_peered":"2024-02-23T00:45:31.141612+0000","last_clean":"2024-02-23T00:45:31.141612+0000","last_became_active":"2024-02-23T00:39:18.796680+0000","last_became_peered":"2024-02-23T00:39:18.796680+0000","last_unstale":"2024-02-23T00:45:31.141612+0000","last_undegraded":"2024-02-23T00:45:31.141612+0000","last_fullsized":"2024-02-23T00:45:31.141612+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-02-23T00:43:50.393713+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-02-23T00:43:50.393713+0000","last_clean_scrub_stamp":"2024-02-23T00:43:50.393713+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":158,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.449698+0000","last_change":"2024-02-23T00:42:58.805923+0000","last_active":"2024-02-23T00:45:30.449698+0000","last_peered":"2024-02-23T00:45:30.449698+0000","last_clean":"2024-02-23T00:45:30.449698+0000","last_became_active":"2024-02-23T00:42:58.803900+0000","last_became_peered":"2024-02-23T00:42:58.803900+0000","last_unstale":"2024-02-23T00:45:30.449698+0000","last_undegraded":"2024-02-23T00:45:30.449698+0000","last_fullsized":"2024-02-23T00:45:30.449698+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":160,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.147459+0000","last_change":"2024-02-23T00:42:58.850530+0000","last_active":"2024-02-23T00:45:31.147459+0000","last_peered":"2024-02-23T00:45:31.147459+0000","last_clean":"2024-02-23T00:45:31.147459+0000","last_became_active":"2024-02-23T00:42:58.850072+0000","last_became_peered":"2024-02-23T00:42:58.850072+0000","last_unstale":"2024-02-23T00:45:31.147459+0000","last_undegraded":"2024-02-23T00:45:31.147459+0000","last_fullsized":"2024-02-23T00:45:31.147459+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"93'23","reported_seq":265,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.141724+0000","last_change":"2024-02-23T00:44:28.387225+0000","last_active":"2024-02-23T00:45:31.141724+0000","last_peered":"2024-02-23T00:45:31.141724+0000","last_clean":"2024-02-23T00:45:31.141724+0000","last_became_active":"2024-02-23T00:39:16.782660+0000","last_became_peered":"2024-02-23T00:39:16.782660+0000","last_unstale":"2024-02-23T00:45:31.141724+0000","last_undegraded":"2024-02-23T00:45:31.141724+0000","last_fullsized":"2024-02-23T00:45:31.141724+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'23","last_scrub_stamp":"2024-02-23T00:44:28.387155+0000","last_deep_scrub":"93'23","last_deep_scrub_stamp":"2024-02-23T00:41:35.954516+0000","last_clean_scrub_stamp":"2024-02-23T00:44:28.387155+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":224,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.132230+0000","last_change":"2024-02-23T00:44:21.157372+0000","last_active":"2024-02-23T00:45:31.132230+0000","last_peered":"2024-02-23T00:45:31.132230+0000","last_clean":"2024-02-23T00:45:31.132230+0000","last_became_active":"2024-02-23T00:39:14.781560+0000","last_became_peered":"2024-02-23T00:39:14.781560+0000","last_unstale":"2024-02-23T00:45:31.132230+0000","last_undegraded":"2024-02-23T00:45:31.132230+0000","last_fullsized":"2024-02-23T00:45:31.132230+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:21.157316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:21.157316+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":220,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.150140+0000","last_change":"2024-02-23T00:43:49.238770+0000","last_active":"2024-02-23T00:45:31.150140+0000","last_peered":"2024-02-23T00:45:31.150140+0000","last_clean":"2024-02-23T00:45:31.150140+0000","last_became_active":"2024-02-23T00:39:18.790602+0000","last_became_peered":"2024-02-23T00:39:18.790602+0000","last_unstale":"2024-02-23T00:45:31.150140+0000","last_undegraded":"2024-02-23T00:45:31.150140+0000","last_fullsized":"2024-02-23T00:45:31.150140+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:49.238704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:53.189881+0000","last_clean_scrub_stamp":"2024-02-23T00:43:49.238704+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.14","version":"110'2","reported_seq":161,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.145985+0000","last_change":"2024-02-23T00:42:58.804637+0000","last_active":"2024-02-23T00:45:31.145985+0000","last_peered":"2024-02-23T00:45:31.145985+0000","last_clean":"2024-02-23T00:45:31.145985+0000","last_became_active":"2024-02-23T00:42:58.804344+0000","last_became_peered":"2024-02-23T00:42:58.804344+0000","last_unstale":"2024-02-23T00:45:31.145985+0000","last_undegraded":"2024-02-23T00:45:31.145985+0000","last_fullsized":"2024-02-23T00:45:31.145985+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":165,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.132309+0000","last_change":"2024-02-23T00:45:15.933508+0000","last_active":"2024-02-23T00:45:31.132309+0000","last_peered":"2024-02-23T00:45:31.132309+0000","last_clean":"2024-02-23T00:45:31.132309+0000","last_became_active":"2024-02-23T00:42:58.774720+0000","last_became_peered":"2024-02-23T00:42:58.774720+0000","last_unstale":"2024-02-23T00:45:31.132309+0000","last_undegraded":"2024-02-23T00:45:31.132309+0000","last_fullsized":"2024-02-23T00:45:31.132309+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:15.933438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:15.933438+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"67'15","reported_seq":245,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.140660+0000","last_change":"2024-02-23T00:43:36.449809+0000","last_active":"2024-02-23T00:45:31.140660+0000","last_peered":"2024-02-23T00:45:31.140660+0000","last_clean":"2024-02-23T00:45:31.140660+0000","last_became_active":"2024-02-23T00:39:16.787138+0000","last_became_peered":"2024-02-23T00:39:16.787138+0000","last_unstale":"2024-02-23T00:45:31.140660+0000","last_undegraded":"2024-02-23T00:45:31.140660+0000","last_fullsized":"2024-02-23T00:45:31.140660+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:43:36.449687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:36.449687+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":222,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.143073+0000","last_change":"2024-02-23T00:43:45.362071+0000","last_active":"2024-02-23T00:45:31.143073+0000","last_peered":"2024-02-23T00:45:31.143073+0000","last_clean":"2024-02-23T00:45:31.143073+0000","last_became_active":"2024-02-23T00:39:14.782494+0000","last_became_peered":"2024-02-23T00:39:14.782494+0000","last_unstale":"2024-02-23T00:45:31.143073+0000","last_undegraded":"2024-02-23T00:45:31.143073+0000","last_fullsized":"2024-02-23T00:45:31.143073+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:45.361978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.361978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":219,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.137600+0000","last_change":"2024-02-23T00:45:03.615758+0000","last_active":"2024-02-23T00:45:31.137600+0000","last_peered":"2024-02-23T00:45:31.137600+0000","last_clean":"2024-02-23T00:45:31.137600+0000","last_became_active":"2024-02-23T00:39:18.801039+0000","last_became_peered":"2024-02-23T00:39:18.801039+0000","last_unstale":"2024-02-23T00:45:31.137600+0000","last_undegraded":"2024-02-23T00:45:31.137600+0000","last_fullsized":"2024-02-23T00:45:31.137600+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:03.615692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.641689+0000","last_clean_scrub_stamp":"2024-02-23T00:45:03.615692+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"110'5","reported_seq":170,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.145979+0000","last_change":"2024-02-23T00:45:14.317657+0000","last_active":"2024-02-23T00:45:31.145979+0000","last_peered":"2024-02-23T00:45:31.145979+0000","last_clean":"2024-02-23T00:45:31.145979+0000","last_became_active":"2024-02-23T00:42:58.783343+0000","last_became_peered":"2024-02-23T00:42:58.783343+0000","last_unstale":"2024-02-23T00:45:31.145979+0000","last_undegraded":"2024-02-23T00:45:31.145979+0000","last_fullsized":"2024-02-23T00:45:31.145979+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'5","last_scrub_stamp":"2024-02-23T00:45:14.317573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.317573+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":165,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.133481+0000","last_change":"2024-02-23T00:45:21.917072+0000","last_active":"2024-02-23T00:45:31.133481+0000","last_peered":"2024-02-23T00:45:31.133481+0000","last_clean":"2024-02-23T00:45:31.133481+0000","last_became_active":"2024-02-23T00:42:58.770759+0000","last_became_peered":"2024-02-23T00:42:58.770759+0000","last_unstale":"2024-02-23T00:45:31.133481+0000","last_undegraded":"2024-02-23T00:45:31.133481+0000","last_fullsized":"2024-02-23T00:45:31.133481+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:21.916996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:21.916996+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"68'18","reported_seq":260,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.134822+0000","last_change":"2024-02-23T00:43:51.079824+0000","last_active":"2024-02-23T00:45:31.134822+0000","last_peered":"2024-02-23T00:45:31.134822+0000","last_clean":"2024-02-23T00:45:31.134822+0000","last_became_active":"2024-02-23T00:39:16.782803+0000","last_became_peered":"2024-02-23T00:39:16.782803+0000","last_unstale":"2024-02-23T00:45:31.134822+0000","last_undegraded":"2024-02-23T00:45:31.134822+0000","last_fullsized":"2024-02-23T00:45:31.134822+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'18","last_scrub_stamp":"2024-02-23T00:43:51.079756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:51.079756+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":224,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.133635+0000","last_change":"2024-02-23T00:43:48.030906+0000","last_active":"2024-02-23T00:45:31.133635+0000","last_peered":"2024-02-23T00:45:31.133635+0000","last_clean":"2024-02-23T00:45:31.133635+0000","last_became_active":"2024-02-23T00:39:14.782163+0000","last_became_peered":"2024-02-23T00:39:14.782163+0000","last_unstale":"2024-02-23T00:45:31.133635+0000","last_undegraded":"2024-02-23T00:45:31.133635+0000","last_fullsized":"2024-02-23T00:45:31.133635+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:48.030846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.030846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":219,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.146169+0000","last_change":"2024-02-23T00:44:37.068899+0000","last_active":"2024-02-23T00:45:31.146169+0000","last_peered":"2024-02-23T00:45:31.146169+0000","last_clean":"2024-02-23T00:45:31.146169+0000","last_became_active":"2024-02-23T00:39:18.794903+0000","last_became_peered":"2024-02-23T00:39:18.794903+0000","last_unstale":"2024-02-23T00:45:31.146169+0000","last_undegraded":"2024-02-23T00:45:31.146169+0000","last_fullsized":"2024-02-23T00:45:31.146169+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:37.068834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_clean_scrub_stamp":"2024-02-23T00:44:37.068834+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"110'5","reported_seq":165,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.150237+0000","last_change":"2024-02-23T00:42:58.839927+0000","last_active":"2024-02-23T00:45:31.150237+0000","last_peered":"2024-02-23T00:45:31.150237+0000","last_clean":"2024-02-23T00:45:31.150237+0000","last_became_active":"2024-02-23T00:42:58.838833+0000","last_became_peered":"2024-02-23T00:42:58.838833+0000","last_unstale":"2024-02-23T00:45:31.150237+0000","last_undegraded":"2024-02-23T00:45:31.150237+0000","last_fullsized":"2024-02-23T00:45:31.150237+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":159,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.160756+0000","last_change":"2024-02-23T00:42:58.698567+0000","last_active":"2024-02-23T00:45:30.160756+0000","last_peered":"2024-02-23T00:45:30.160756+0000","last_clean":"2024-02-23T00:45:30.160756+0000","last_became_active":"2024-02-23T00:42:58.698258+0000","last_became_peered":"2024-02-23T00:42:58.698258+0000","last_unstale":"2024-02-23T00:45:30.160756+0000","last_undegraded":"2024-02-23T00:45:30.160756+0000","last_fullsized":"2024-02-23T00:45:30.160756+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"95'27","reported_seq":273,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.144798+0000","last_change":"2024-02-23T00:43:52.124336+0000","last_active":"2024-02-23T00:45:31.144798+0000","last_peered":"2024-02-23T00:45:31.144798+0000","last_clean":"2024-02-23T00:45:31.144798+0000","last_became_active":"2024-02-23T00:39:16.786791+0000","last_became_peered":"2024-02-23T00:39:16.786791+0000","last_unstale":"2024-02-23T00:45:31.144798+0000","last_undegraded":"2024-02-23T00:45:31.144798+0000","last_fullsized":"2024-02-23T00:45:31.144798+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'27","last_scrub_stamp":"2024-02-23T00:43:52.124247+0000","last_deep_scrub":"95'27","last_deep_scrub_stamp":"2024-02-23T00:41:55.919895+0000","last_clean_scrub_stamp":"2024-02-23T00:43:52.124247+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":220,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.447777+0000","last_change":"2024-02-23T00:44:27.759571+0000","last_active":"2024-02-23T00:45:30.447777+0000","last_peered":"2024-02-23T00:45:30.447777+0000","last_clean":"2024-02-23T00:45:30.447777+0000","last_became_active":"2024-02-23T00:39:14.787148+0000","last_became_peered":"2024-02-23T00:39:14.787148+0000","last_unstale":"2024-02-23T00:45:30.447777+0000","last_undegraded":"2024-02-23T00:45:30.447777+0000","last_fullsized":"2024-02-23T00:45:30.447777+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:27.759499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:27.759499+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":219,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.160704+0000","last_change":"2024-02-23T00:45:10.090353+0000","last_active":"2024-02-23T00:45:30.160704+0000","last_peered":"2024-02-23T00:45:30.160704+0000","last_clean":"2024-02-23T00:45:30.160704+0000","last_became_active":"2024-02-23T00:39:18.796431+0000","last_became_peered":"2024-02-23T00:39:18.796431+0000","last_unstale":"2024-02-23T00:45:30.160704+0000","last_undegraded":"2024-02-23T00:45:30.160704+0000","last_fullsized":"2024-02-23T00:45:30.160704+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.090270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.090270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.11","version":"110'7","reported_seq":171,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.137711+0000","last_change":"2024-02-23T00:44:30.366468+0000","last_active":"2024-02-23T00:45:31.137711+0000","last_peered":"2024-02-23T00:45:31.137711+0000","last_clean":"2024-02-23T00:45:31.137711+0000","last_became_active":"2024-02-23T00:42:58.810765+0000","last_became_peered":"2024-02-23T00:42:58.810765+0000","last_unstale":"2024-02-23T00:45:31.137711+0000","last_undegraded":"2024-02-23T00:45:31.137711+0000","last_fullsized":"2024-02-23T00:45:31.137711+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'7","last_scrub_stamp":"2024-02-23T00:44:30.366391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:44:30.366391+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":159,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.160750+0000","last_change":"2024-02-23T00:42:58.833221+0000","last_active":"2024-02-23T00:45:30.160750+0000","last_peered":"2024-02-23T00:45:30.160750+0000","last_clean":"2024-02-23T00:45:30.160750+0000","last_became_active":"2024-02-23T00:42:58.832885+0000","last_became_peered":"2024-02-23T00:42:58.832885+0000","last_unstale":"2024-02-23T00:45:30.160750+0000","last_undegraded":"2024-02-23T00:45:30.160750+0000","last_fullsized":"2024-02-23T00:45:30.160750+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"82'15","reported_seq":250,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.147170+0000","last_change":"2024-02-23T00:44:35.095478+0000","last_active":"2024-02-23T00:45:31.147170+0000","last_peered":"2024-02-23T00:45:31.147170+0000","last_clean":"2024-02-23T00:45:31.147170+0000","last_became_active":"2024-02-23T00:39:16.782762+0000","last_became_peered":"2024-02-23T00:39:16.782762+0000","last_unstale":"2024-02-23T00:45:31.147170+0000","last_undegraded":"2024-02-23T00:45:31.147170+0000","last_fullsized":"2024-02-23T00:45:31.147170+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'15","last_scrub_stamp":"2024-02-23T00:44:35.095409+0000","last_deep_scrub":"82'15","last_deep_scrub_stamp":"2024-02-23T00:41:50.975683+0000","last_clean_scrub_stamp":"2024-02-23T00:44:35.095409+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":220,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.449482+0000","last_change":"2024-02-23T00:43:44.794995+0000","last_active":"2024-02-23T00:45:30.449482+0000","last_peered":"2024-02-23T00:45:30.449482+0000","last_clean":"2024-02-23T00:45:30.449482+0000","last_became_active":"2024-02-23T00:39:14.783469+0000","last_became_peered":"2024-02-23T00:39:14.783469+0000","last_unstale":"2024-02-23T00:45:30.449482+0000","last_undegraded":"2024-02-23T00:45:30.449482+0000","last_fullsized":"2024-02-23T00:45:30.449482+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:44.794928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:44.794928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":218,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.447037+0000","last_change":"2024-02-23T00:43:41.817455+0000","last_active":"2024-02-23T00:45:30.447037+0000","last_peered":"2024-02-23T00:45:30.447037+0000","last_clean":"2024-02-23T00:45:30.447037+0000","last_became_active":"2024-02-23T00:39:18.796952+0000","last_became_peered":"2024-02-23T00:39:18.796952+0000","last_unstale":"2024-02-23T00:45:30.447037+0000","last_undegraded":"2024-02-23T00:45:30.447037+0000","last_fullsized":"2024-02-23T00:45:30.447037+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:41.817387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:41.817387+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":165,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.144166+0000","last_change":"2024-02-23T00:45:08.215139+0000","last_active":"2024-02-23T00:45:31.144166+0000","last_peered":"2024-02-23T00:45:31.144166+0000","last_clean":"2024-02-23T00:45:31.144166+0000","last_became_active":"2024-02-23T00:42:58.726021+0000","last_became_peered":"2024-02-23T00:42:58.726021+0000","last_unstale":"2024-02-23T00:45:31.144166+0000","last_undegraded":"2024-02-23T00:45:31.144166+0000","last_fullsized":"2024-02-23T00:45:31.144166+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.215063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.215063+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":167,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.145692+0000","last_change":"2024-02-23T00:45:23.224771+0000","last_active":"2024-02-23T00:45:31.145692+0000","last_peered":"2024-02-23T00:45:31.145692+0000","last_clean":"2024-02-23T00:45:31.145692+0000","last_became_active":"2024-02-23T00:42:58.758147+0000","last_became_peered":"2024-02-23T00:42:58.758147+0000","last_unstale":"2024-02-23T00:45:31.145692+0000","last_undegraded":"2024-02-23T00:45:31.145692+0000","last_fullsized":"2024-02-23T00:45:31.145692+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:23.224663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:23.224663+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.224663+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"68'15","reported_seq":252,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.144376+0000","last_change":"2024-02-23T00:43:49.139235+0000","last_active":"2024-02-23T00:45:31.144376+0000","last_peered":"2024-02-23T00:45:31.144376+0000","last_clean":"2024-02-23T00:45:31.144376+0000","last_became_active":"2024-02-23T00:39:16.784606+0000","last_became_peered":"2024-02-23T00:39:16.784606+0000","last_unstale":"2024-02-23T00:45:31.144376+0000","last_undegraded":"2024-02-23T00:45:31.144376+0000","last_fullsized":"2024-02-23T00:45:31.144376+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:43:49.139156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:49.139156+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":224,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.137747+0000","last_change":"2024-02-23T00:44:32.170717+0000","last_active":"2024-02-23T00:45:31.137747+0000","last_peered":"2024-02-23T00:45:31.137747+0000","last_clean":"2024-02-23T00:45:31.137747+0000","last_became_active":"2024-02-23T00:39:14.785362+0000","last_became_peered":"2024-02-23T00:39:14.785362+0000","last_unstale":"2024-02-23T00:45:31.137747+0000","last_undegraded":"2024-02-23T00:45:31.137747+0000","last_fullsized":"2024-02-23T00:45:31.137747+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:32.170554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:32.170554+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":214,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.133364+0000","last_change":"2024-02-23T00:42:44.553722+0000","last_active":"2024-02-23T00:45:31.133364+0000","last_peered":"2024-02-23T00:45:31.133364+0000","last_clean":"2024-02-23T00:45:31.133364+0000","last_became_active":"2024-02-23T00:39:18.796312+0000","last_became_peered":"2024-02-23T00:39:18.796312+0000","last_unstale":"2024-02-23T00:45:31.133364+0000","last_undegraded":"2024-02-23T00:45:31.133364+0000","last_fullsized":"2024-02-23T00:45:31.133364+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:44.553621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:44.553621+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.13","version":"110'9","reported_seq":174,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.134724+0000","last_change":"2024-02-23T00:45:21.719969+0000","last_active":"2024-02-23T00:45:31.134724+0000","last_peered":"2024-02-23T00:45:31.134724+0000","last_clean":"2024-02-23T00:45:31.134724+0000","last_became_active":"2024-02-23T00:42:57.655617+0000","last_became_peered":"2024-02-23T00:42:57.655617+0000","last_unstale":"2024-02-23T00:45:31.134724+0000","last_undegraded":"2024-02-23T00:45:31.134724+0000","last_fullsized":"2024-02-23T00:45:31.134724+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'9","last_scrub_stamp":"2024-02-23T00:45:21.719900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:21.719900+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":165,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.149827+0000","last_change":"2024-02-23T00:45:09.641006+0000","last_active":"2024-02-23T00:45:31.149827+0000","last_peered":"2024-02-23T00:45:31.149827+0000","last_clean":"2024-02-23T00:45:31.149827+0000","last_became_active":"2024-02-23T00:42:58.838257+0000","last_became_peered":"2024-02-23T00:42:58.838257+0000","last_unstale":"2024-02-23T00:45:31.149827+0000","last_undegraded":"2024-02-23T00:45:31.149827+0000","last_fullsized":"2024-02-23T00:45:31.149827+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:09.640936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:09.640936+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"88'50","reported_seq":318,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.134244+0000","last_change":"2024-02-23T00:45:06.038488+0000","last_active":"2024-02-23T00:45:31.134244+0000","last_peered":"2024-02-23T00:45:31.134244+0000","last_clean":"2024-02-23T00:45:31.134244+0000","last_became_active":"2024-02-23T00:39:16.780144+0000","last_became_peered":"2024-02-23T00:39:16.780144+0000","last_unstale":"2024-02-23T00:45:31.134244+0000","last_undegraded":"2024-02-23T00:45:31.134244+0000","last_fullsized":"2024-02-23T00:45:31.134244+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'50","last_scrub_stamp":"2024-02-23T00:45:06.038374+0000","last_deep_scrub":"88'50","last_deep_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.038374+0000","log_size":50,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":224,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.149555+0000","last_change":"2024-02-23T00:45:20.573233+0000","last_active":"2024-02-23T00:45:31.149555+0000","last_peered":"2024-02-23T00:45:31.149555+0000","last_clean":"2024-02-23T00:45:31.149555+0000","last_became_active":"2024-02-23T00:39:14.782000+0000","last_became_peered":"2024-02-23T00:39:14.782000+0000","last_unstale":"2024-02-23T00:45:31.149555+0000","last_undegraded":"2024-02-23T00:45:31.149555+0000","last_fullsized":"2024-02-23T00:45:31.149555+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:20.573168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_clean_scrub_stamp":"2024-02-23T00:45:20.573168+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":214,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.144492+0000","last_change":"2024-02-23T00:43:48.126631+0000","last_active":"2024-02-23T00:45:31.144492+0000","last_peered":"2024-02-23T00:45:31.144492+0000","last_clean":"2024-02-23T00:45:31.144492+0000","last_became_active":"2024-02-23T00:39:18.799723+0000","last_became_peered":"2024-02-23T00:39:18.799723+0000","last_unstale":"2024-02-23T00:45:31.144492+0000","last_undegraded":"2024-02-23T00:45:31.144492+0000","last_fullsized":"2024-02-23T00:45:31.144492+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:48.126552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.126552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":165,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-02-23T00:45:31.133404+0000","last_change":"2024-02-23T00:45:24.942600+0000","last_active":"2024-02-23T00:45:31.133404+0000","last_peered":"2024-02-23T00:45:31.133404+0000","last_clean":"2024-02-23T00:45:31.133404+0000","last_became_active":"2024-02-23T00:42:58.765643+0000","last_became_peered":"2024-02-23T00:42:58.765643+0000","last_unstale":"2024-02-23T00:45:31.133404+0000","last_undegraded":"2024-02-23T00:45:31.133404+0000","last_fullsized":"2024-02-23T00:45:31.133404+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:24.942535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:24.942535+0000","last_clean_scrub_stamp":"2024-02-23T00:45:24.942535+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":164,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-02-23T00:45:30.161301+0000","last_change":"2024-02-23T00:45:14.209403+0000","last_active":"2024-02-23T00:45:30.161301+0000","last_peered":"2024-02-23T00:45:30.161301+0000","last_clean":"2024-02-23T00:45:30.161301+0000","last_became_active":"2024-02-23T00:42:58.824573+0000","last_became_peered":"2024-02-23T00:42:58.824573+0000","last_unstale":"2024-02-23T00:45:30.161301+0000","last_undegraded":"2024-02-23T00:45:30.161301+0000","last_fullsized":"2024-02-23T00:45:30.161301+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.209308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.209308+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0],"acting":[5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":164,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":62,"num_pg":32,"stat_sum":{"num_bytes":1194,"num_objects":118,"num_object_clones":1,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":118,"num_whiteouts":0,"num_read":94,"num_read_kb":1,"num_write":536,"num_write_kb":532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1437696,"data_stored":3582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1776,"ondisk_log_size":1776,"up":96,"acting":96,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":22,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":66,"acting":66,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1080,"num_read_kb":805,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":651,"ondisk_log_size":651,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":369,"num_read_kb":364,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2048,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":196608,"data_stored":6144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101,"ondisk_log_size":101,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234063,"num_pgs":103,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":317896,"kb_used_data":4280,"kb_used_omap":0,"kb_used_meta":313600,"kb_avail":93427256,"statfs":{"total":95995035648,"available":95669510144,"internally_reserved":0,"allocated":4382720,"data_stored":2563061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321126400},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.28000000000000003,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.747,"5min":4.9420000000000002,"15min":4.9420000000000002},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.53200000000000003,"15min":0.51600000000000001},"min":{"1min":0.32600000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0589999999999999,"5min":4.9729999999999999,"15min":4.9729999999999999},"last":0.42999999999999999}]},{"osd":1,"last update":"Fri Feb 23 00:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.77400000000000002,"15min":0.626},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":5.4539999999999997,"5min":10.786,"15min":10.786},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.78700000000000003,"15min":0.629},"min":{"1min":0.29399999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":11.885999999999999,"5min":11.885999999999999,"15min":11.885999999999999},"last":0.35399999999999998}]},{"osd":2,"last update":"Fri Feb 23 00:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59299999999999997,"15min":0.55900000000000005},"min":{"1min":0.35899999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":4.0449999999999999,"5min":5.0789999999999997,"15min":5.0789999999999997},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.57399999999999995,"15min":0.54800000000000004},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":1.04,"5min":5.1280000000000001,"15min":5.1280000000000001},"last":0.51300000000000001}]},{"osd":3,"last update":"Fri Feb 23 00:45:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":0.80800000000000005,"15min":0.62},"min":{"1min":0.35499999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":15.712999999999999,"5min":15.712999999999999,"15min":15.712999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.71699999999999997,"15min":0.61799999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.004,"5min":9.4090000000000007,"15min":9.4090000000000007},"last":0.60599999999999998}]},{"osd":4,"last update":"Fri Feb 23 00:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.44500000000000001,"15min":0.436},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":2.0350000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.55100000000000005,"15min":0.48599999999999999},"min":{"1min":0.28599999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":11.236000000000001,"5min":11.236000000000001,"15min":11.236000000000001},"last":0.66400000000000003}]},{"osd":5,"last update":"Fri Feb 23 00:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.51700000000000002,"15min":0.49099999999999999},"min":{"1min":0.29699999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":5.4000000000000004,"5min":5.4000000000000004,"15min":5.4000000000000004},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.50800000000000001,"15min":0.48599999999999999},"min":{"1min":0.37,"5min":0.29299999999999998,"15min":0.26700000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.65100000000000002}]},{"osd":6,"last update":"Fri Feb 23 00:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51000000000000001,"15min":0.50700000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.496},{"interface":"front","average":{"1min":1.093,"5min":0.61799999999999999,"15min":0.54400000000000004},"min":{"1min":0.35599999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":13.395,"5min":13.395,"15min":13.395},"last":1.2370000000000001}]}]},{"osd":6,"up_from":47,"seq":201863462998,"num_pgs":92,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309664,"kb_used_data":4240,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93435488,"statfs":{"total":95995035648,"available":95677939712,"internally_reserved":0,"allocated":4341760,"data_stored":2563316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:44:49 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.98799999999999999,"15min":0.71199999999999997},"min":{"1min":0.29699999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61399999999999999,"5min":30.641999999999999,"15min":30.641999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.98199999999999998,"15min":0.71799999999999997},"min":{"1min":0.311,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1,"5min":29.292000000000002,"15min":29.292000000000002},"last":0.51500000000000001}]},{"osd":1,"last update":"Fri Feb 23 00:44:49 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.98599999999999999,"15min":0.69799999999999995},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.59799999999999998,"5min":28.186,"15min":28.186},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.48899999999999999,"5min":1.4910000000000001,"15min":0.88900000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.97599999999999998,"5min":33.104999999999997,"15min":33.104999999999997},"last":0.443}]},{"osd":2,"last update":"Fri Feb 23 00:44:49 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":1.0449999999999999,"15min":0.73899999999999999},"min":{"1min":0.32800000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.1200000000000001,"5min":19.960000000000001,"15min":19.960000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.84799999999999998,"15min":0.65600000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0549999999999999,"5min":8.5860000000000003,"15min":8.5860000000000003},"last":0.499}]},{"osd":3,"last update":"Fri Feb 23 00:44:49 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.76000000000000001,"15min":0.64200000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.0069999999999999,"5min":19.562000000000001,"15min":19.562000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.5,"5min":0.85599999999999998,"15min":0.68200000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.94299999999999995,"5min":14.199,"15min":14.199},"last":0.54400000000000004}]},{"osd":4,"last update":"Fri Feb 23 00:44:49 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.57899999999999996,"15min":0.46100000000000002},"min":{"1min":0.25900000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.55800000000000005,"5min":11.109999999999999,"15min":11.109999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.751,"15min":0.53300000000000003},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.65600000000000003,"5min":16.734000000000002,"15min":16.734000000000002},"last":0.50800000000000001}]},{"osd":5,"last update":"Fri Feb 23 00:44:49 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.629,"15min":0.48199999999999998},"min":{"1min":0.29299999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.59699999999999998,"5min":14.154999999999999,"15min":14.154999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.63500000000000001,"15min":0.5},"min":{"1min":0.312,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.67800000000000005,"5min":6.6100000000000003,"15min":6.6100000000000003},"last":0.57199999999999995}]},{"osd":7,"last update":"Fri Feb 23 00:45:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.58399999999999996,"15min":0.52000000000000002},"min":{"1min":0.29299999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.745,"5min":8.548,"15min":8.548},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.496,"5min":0.752,"15min":0.59399999999999997},"min":{"1min":0.33900000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.68899999999999995,"5min":15.618,"15min":15.618},"last":0.52500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574960,"num_pgs":98,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309628,"kb_used_data":4200,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93435524,"statfs":{"total":95995035648,"available":95677976576,"internally_reserved":0,"allocated":4300800,"data_stored":2564135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:44:43 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":1.298,"15min":0.66500000000000004},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":1.637,"5min":78.361000000000004,"15min":78.361000000000004},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.57299999999999995,"15min":0.437},"min":{"1min":0.218,"5min":0.191,"15min":0.191},"max":{"1min":1.284,"5min":5.8849999999999998,"15min":5.8849999999999998},"last":0.28399999999999997}]},{"osd":2,"last update":"Fri Feb 23 00:45:03 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":1.1850000000000001,"15min":0.65900000000000003},"min":{"1min":0.247,"5min":0.219,"15min":0.188},"max":{"1min":0.63,"5min":48.606000000000002,"15min":48.606000000000002},"last":0.25},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.88300000000000001,"15min":0.58599999999999997},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.249},"max":{"1min":0.82599999999999996,"5min":23.911000000000001,"15min":23.911000000000001},"last":0.38500000000000001}]},{"osd":3,"last update":"Fri Feb 23 00:44:38 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.89800000000000002,"15min":0.63},"min":{"1min":0.29899999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":2.1949999999999998,"5min":35.244999999999997,"15min":35.244999999999997},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":1.135,"15min":0.68799999999999994},"min":{"1min":0.27100000000000002,"5min":0.253,"15min":0.248},"max":{"1min":6.0510000000000002,"5min":56.853999999999999,"15min":56.853999999999999},"last":0.31}]},{"osd":4,"last update":"Fri Feb 23 00:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":1.2350000000000001,"15min":0.81999999999999995},"min":{"1min":0.33700000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72899999999999998,"5min":37.646999999999998,"15min":37.646999999999998},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.875,"15min":0.69999999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83499999999999996,"5min":9.6859999999999999,"15min":9.6859999999999999},"last":0.377}]},{"osd":5,"last update":"Fri Feb 23 00:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":1.224,"15min":0.84599999999999997},"min":{"1min":0.41099999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83399999999999996,"5min":48.792000000000002,"15min":48.792000000000002},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.83699999999999997,"15min":0.755},"min":{"1min":0.45100000000000001,"5min":0.38100000000000001,"15min":0.34200000000000003},"max":{"1min":0.76500000000000001,"5min":11.872999999999999,"15min":11.872999999999999},"last":0.48799999999999999}]},{"osd":6,"last update":"Fri Feb 23 00:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":1.2,"15min":0.82999999999999996},"min":{"1min":0.39200000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.90200000000000002,"5min":40.741,"15min":40.741},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.77800000000000002,"15min":0.69599999999999995},"min":{"1min":0.46100000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77200000000000002,"5min":6.0069999999999997,"15min":6.0069999999999997},"last":0.52200000000000002}]},{"osd":7,"last update":"Fri Feb 23 00:45:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.91000000000000003,"15min":0.71899999999999997},"min":{"1min":0.42499999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82799999999999996,"5min":20.225000000000001,"15min":20.225000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.94199999999999995,"15min":0.72799999999999998},"min":{"1min":0.42799999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85999999999999999,"5min":20.254999999999999,"15min":20.254999999999999},"last":0.51600000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738486,"num_pgs":88,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":313724,"kb_used_data":4204,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93431428,"statfs":{"total":95995035648,"available":95673782272,"internally_reserved":0,"allocated":4304896,"data_stored":2563338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 23 00:44:42 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":1.0980000000000001,"15min":0.59099999999999997},"min":{"1min":0.25900000000000001,"5min":0.223,"15min":0.19500000000000001},"max":{"1min":0.82799999999999996,"5min":28.856999999999999,"15min":28.856999999999999},"last":0.309},{"interface":"front","average":{"1min":1.3480000000000001,"5min":0.97799999999999998,"15min":0.56100000000000005},"min":{"1min":0.22700000000000001,"5min":0.218,"15min":0.20799999999999999},"max":{"1min":12.923,"5min":20.577000000000002,"15min":20.577000000000002},"last":0.42299999999999999}]},{"osd":2,"last update":"Fri Feb 23 00:45:06 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.026,"15min":0.60399999999999998},"min":{"1min":0.308,"5min":0.26000000000000001,"15min":0.20499999999999999},"max":{"1min":13.67,"5min":13.863,"15min":13.863},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.88400000000000001,"15min":0.54900000000000004},"min":{"1min":0.25700000000000001,"5min":0.24299999999999999,"15min":0.19800000000000001},"max":{"1min":0.60099999999999998,"5min":10.455,"15min":10.455},"last":0.34799999999999998}]},{"osd":3,"last update":"Fri Feb 23 00:44:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.389,"15min":0.76400000000000001},"min":{"1min":0.28699999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":13.266,"5min":35.566000000000003,"15min":35.566000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":1.6859999999999999,"5min":1.24,"15min":0.751},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29399999999999998},"max":{"1min":16.931999999999999,"5min":20.702999999999999,"15min":20.702999999999999},"last":0.48499999999999999}]},{"osd":4,"last update":"Fri Feb 23 00:44:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.038,"15min":0.81299999999999994},"min":{"1min":0.36199999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":10.584,"5min":10.584,"15min":10.584},"last":0.753},{"interface":"front","average":{"1min":1.306,"5min":0.89600000000000002,"15min":0.79700000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":13.029999999999999,"5min":13.029999999999999,"15min":13.029999999999999},"last":0.78400000000000003}]},{"osd":5,"last update":"Fri Feb 23 00:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":1.014,"15min":0.74399999999999999},"min":{"1min":0.37,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88200000000000001,"5min":10.397,"15min":10.397},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":1.079,"15min":0.80900000000000005},"min":{"1min":0.44700000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86299999999999999,"5min":17.297000000000001,"15min":17.297000000000001},"last":0.84999999999999998}]},{"osd":6,"last update":"Fri Feb 23 00:44:57 2024","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.1160000000000001,"15min":0.80200000000000005},"min":{"1min":0.434,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":13.237,"5min":17.373999999999999,"15min":17.373999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.3029999999999999,"5min":0.90200000000000002,"15min":0.72399999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":13.086,"5min":13.086,"15min":13.086},"last":0.82599999999999996}]},{"osd":7,"last update":"Fri Feb 23 00:45:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":1.081,"15min":0.81299999999999994},"min":{"1min":0.48199999999999998,"5min":0.41699999999999998,"15min":0.39400000000000002},"max":{"1min":0.82599999999999996,"5min":17.399999999999999,"15min":17.399999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.494,"5min":1.0640000000000001,"15min":0.78900000000000003},"min":{"1min":0.45400000000000001,"5min":0.433,"15min":0.43099999999999999},"max":{"1min":16.427,"5min":16.427,"15min":16.427},"last":0.72899999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411435,"num_pgs":89,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309732,"kb_used_data":4312,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93435420,"statfs":{"total":95995035648,"available":95677870080,"internally_reserved":0,"allocated":4415488,"data_stored":2563171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:45:02 2024","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":0.70199999999999996,"15min":0.48699999999999999},"min":{"1min":0.20499999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":13.138999999999999,"5min":13.657,"15min":13.657},"last":0.32800000000000001},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2649999999999999,"15min":0.68000000000000005},"min":{"1min":0.25700000000000001,"5min":0.246,"15min":0.23599999999999999},"max":{"1min":12.705,"5min":42.343000000000004,"15min":42.343000000000004},"last":0.40400000000000003}]},{"osd":1,"last update":"Fri Feb 23 00:45:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":0.96999999999999997,"15min":0.57499999999999996},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":8.8569999999999993,"5min":18.952000000000002,"15min":18.952000000000002},"last":0.30599999999999999},{"interface":"front","average":{"1min":1.6419999999999999,"5min":0.76800000000000002,"15min":0.52300000000000002},"min":{"1min":0.26100000000000001,"5min":0.23999999999999999,"15min":0.23599999999999999},"max":{"1min":15.391,"5min":15.391,"15min":15.391},"last":0.46600000000000003}]},{"osd":3,"last update":"Fri Feb 23 00:44:34 2024","interfaces":[{"interface":"back","average":{"1min":2.3879999999999999,"5min":1.1879999999999999,"15min":0.72199999999999998},"min":{"1min":0.251,"5min":0.25,"15min":0.25},"max":{"1min":25.635999999999999,"5min":25.635999999999999,"15min":25.635999999999999},"last":0.439},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.008,"15min":0.65000000000000002},"min":{"1min":0.27600000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":14.472,"5min":19.960000000000001,"15min":19.960000000000001},"last":0.38800000000000001}]},{"osd":4,"last update":"Fri Feb 23 00:45:02 2024","interfaces":[{"interface":"back","average":{"1min":2.1920000000000002,"5min":0.998,"15min":0.73099999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":13.673,"5min":13.673,"15min":13.673},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.044,"15min":0.73499999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.33900000000000002},"max":{"1min":8.7870000000000008,"5min":10.103999999999999,"15min":10.103999999999999},"last":0.52100000000000002}]},{"osd":5,"last update":"Fri Feb 23 00:45:23 2024","interfaces":[{"interface":"back","average":{"1min":1.784,"5min":0.90600000000000003,"15min":0.69099999999999995},"min":{"1min":0.40699999999999997,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":15.612,"5min":15.612,"15min":15.612},"last":0.57799999999999996},{"interface":"front","average":{"1min":1.335,"5min":0.88,"15min":0.70299999999999996},"min":{"1min":0.376,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":13.632999999999999,"5min":13.632999999999999,"15min":13.632999999999999},"last":0.50700000000000001}]},{"osd":6,"last update":"Fri Feb 23 00:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":0.872,"15min":0.70599999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":11.544,"5min":11.544,"15min":11.544},"last":0.53800000000000003},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0029999999999999,"15min":0.73199999999999998},"min":{"1min":0.38800000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":8.8290000000000006,"5min":10.132999999999999,"15min":10.132999999999999},"last":0.55300000000000005}]},{"osd":7,"last update":"Fri Feb 23 00:45:15 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":0.879,"15min":0.74299999999999999},"min":{"1min":0.40600000000000003,"5min":0.377,"15min":0.377},"max":{"1min":11.599,"5min":11.599,"15min":11.599},"last":0.56599999999999995},{"interface":"front","average":{"1min":1.494,"5min":1.097,"15min":0.80000000000000004},"min":{"1min":0.42399999999999999,"5min":0.374,"15min":0.374},"max":{"1min":8.7469999999999999,"5min":10.148999999999999,"15min":10.148999999999999},"last":0.57299999999999995}]}]},{"osd":3,"up_from":26,"seq":111669149796,"num_pgs":108,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":313780,"kb_used_data":4268,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93431372,"statfs":{"total":95995035648,"available":95673724928,"internally_reserved":0,"allocated":4370432,"data_stored":2564442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:44:32 2024","interfaces":[{"interface":"back","average":{"1min":1.5489999999999999,"5min":0.754,"15min":0.54600000000000004},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":12.212999999999999,"5min":12.212999999999999,"15min":12.212999999999999},"last":0.315},{"interface":"front","average":{"1min":1.802,"5min":0.90400000000000003,"15min":0.60799999999999998},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":11.714,"5min":11.714,"15min":11.714},"last":0.29699999999999999}]},{"osd":1,"last update":"Fri Feb 23 00:44:32 2024","interfaces":[{"interface":"back","average":{"1min":1.627,"5min":0.88500000000000001,"15min":0.61499999999999999},"min":{"1min":0.23599999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":11.742000000000001,"5min":11.742000000000001,"15min":11.742000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":1.278,"5min":0.77300000000000002,"15min":0.56100000000000005},"min":{"1min":0.24199999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":6.383,"5min":6.383,"15min":6.383},"last":0.27500000000000002}]},{"osd":2,"last update":"Fri Feb 23 00:44:32 2024","interfaces":[{"interface":"back","average":{"1min":1.8939999999999999,"5min":0.95399999999999996,"15min":0.65100000000000002},"min":{"1min":0.28899999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":13.359999999999999,"5min":13.359999999999999,"15min":13.359999999999999},"last":0.33100000000000002},{"interface":"front","average":{"1min":2.403,"5min":1.01,"15min":0.66300000000000003},"min":{"1min":0.26200000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":21.122,"5min":21.122,"15min":21.122},"last":0.35999999999999999}]},{"osd":4,"last update":"Fri Feb 23 00:44:57 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.97399999999999998,"15min":0.72499999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":8.125,"5min":11.644,"15min":11.644},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.85399999999999998,"15min":0.68000000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":6.2549999999999999,"5min":6.2549999999999999,"15min":6.2549999999999999},"last":0.443}]},{"osd":5,"last update":"Fri Feb 23 00:45:23 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.76300000000000001,"15min":0.65900000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72999999999999998,"5min":5.6360000000000001,"15min":5.6360000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":1.016,"15min":0.75600000000000001},"min":{"1min":0.40699999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79300000000000004,"5min":11.597,"15min":11.597},"last":0.626}]},{"osd":6,"last update":"Fri Feb 23 00:44:46 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88100000000000001,"15min":0.70199999999999996},"min":{"1min":0.40899999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":6.343,"5min":6.343,"15min":6.343},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.80200000000000005,"15min":0.68600000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":2.8260000000000001,"5min":5.6680000000000001,"15min":5.6680000000000001},"last":0.53700000000000003}]},{"osd":7,"last update":"Fri Feb 23 00:45:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":1.0589999999999999,"15min":0.77800000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.39000000000000001},"max":{"1min":2.6040000000000001,"5min":11.67,"15min":11.67},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.95499999999999996,"15min":0.75700000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":2.6179999999999999,"5min":6.3010000000000002,"15min":6.3010000000000002},"last":0.56499999999999995}]}]},{"osd":4,"up_from":32,"seq":137438953568,"num_pgs":98,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":313804,"kb_used_data":4284,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93431348,"statfs":{"total":95995035648,"available":95673700352,"internally_reserved":0,"allocated":4386816,"data_stored":2564097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:44:59 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.52100000000000002,"15min":0.56100000000000005},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.59499999999999997,"5min":3.5219999999999998,"15min":3.5219999999999998},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.66900000000000004,"15min":0.61399999999999999},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.64100000000000001,"5min":8.6140000000000008,"15min":8.6140000000000008},"last":0.45900000000000002}]},{"osd":1,"last update":"Fri Feb 23 00:44:59 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.66700000000000004,"15min":0.61299999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.27800000000000002},"max":{"1min":0.69799999999999995,"5min":9.6110000000000007,"15min":9.6110000000000007},"last":0.439},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.66300000000000003,"15min":0.60499999999999998},"min":{"1min":0.33500000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.67400000000000004,"5min":6.6950000000000003,"15min":6.6950000000000003},"last":0.47399999999999998}]},{"osd":2,"last update":"Fri Feb 23 00:44:59 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.624,"15min":0.59399999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64800000000000002,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":0.62},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68700000000000006,"5min":5.8440000000000003,"15min":5.8440000000000003},"last":0.65000000000000002}]},{"osd":3,"last update":"Fri Feb 23 00:44:59 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.65800000000000003,"15min":0.60599999999999998},"min":{"1min":0.34399999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65900000000000003,"5min":6.5540000000000003,"15min":6.5540000000000003},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.66200000000000003,"15min":0.60899999999999999},"min":{"1min":0.32700000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73699999999999999,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":0.78000000000000003}]},{"osd":5,"last update":"Fri Feb 23 00:45:23 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.41899999999999998,"15min":0.38300000000000001},"min":{"1min":0.25600000000000001,"5min":0.251,"15min":0.246},"max":{"1min":0.64700000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.48699999999999999,"15min":0.435},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.28899999999999998},"max":{"1min":0.71599999999999997,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.65700000000000003}]},{"osd":6,"last update":"Fri Feb 23 00:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.45000000000000001,"15min":0.41299999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.68100000000000005,"5min":0.75,"15min":0.75},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.45400000000000001,"15min":0.41299999999999998},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.66100000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.67100000000000004}]},{"osd":7,"last update":"Fri Feb 23 00:45:17 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30599999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.67100000000000004,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.497,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.70599999999999996,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.627}]}]},{"osd":5,"up_from":39,"seq":167503724635,"num_pgs":95,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309624,"kb_used_data":4192,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93435528,"statfs":{"total":95995035648,"available":95677980672,"internally_reserved":0,"allocated":4292608,"data_stored":2564399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.78800000000000003,"15min":0.60199999999999998},"min":{"1min":0.28299999999999997,"5min":0.246,"15min":0.246},"max":{"1min":0.67900000000000005,"5min":23.881,"15min":23.881},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.68200000000000005,"15min":0.57899999999999996},"min":{"1min":0.28999999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.57799999999999996,"5min":5.4649999999999999,"15min":5.4649999999999999},"last":0.51800000000000002}]},{"osd":1,"last update":"Fri Feb 23 00:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.88600000000000001,"15min":0.64800000000000002},"min":{"1min":0.30199999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.74099999999999999,"5min":18.175999999999998,"15min":18.175999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.78300000000000003,"15min":0.624},"min":{"1min":0.35699999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.60299999999999998,"5min":9.0950000000000006,"15min":9.0950000000000006},"last":0.56499999999999995}]},{"osd":2,"last update":"Fri Feb 23 00:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":1.0169999999999999,"15min":0.70599999999999996},"min":{"1min":0.32600000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.59799999999999998,"5min":22.282,"15min":22.282},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.49199999999999999,"5min":1.0489999999999999,"15min":0.70899999999999996},"min":{"1min":0.33200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.63700000000000001,"5min":23.925000000000001,"15min":23.925000000000001},"last":0.40500000000000003}]},{"osd":3,"last update":"Fri Feb 23 00:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.77700000000000002,"15min":0.61399999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.63700000000000001,"5min":6.5730000000000004,"15min":6.5730000000000004},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.79900000000000004,"15min":0.64000000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33500000000000002},"max":{"1min":0.64800000000000002,"5min":8.1760000000000002,"15min":8.1760000000000002},"last":0.65300000000000002}]},{"osd":4,"last update":"Fri Feb 23 00:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.42899999999999999,"15min":0.39600000000000002},"min":{"1min":0.26400000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.57899999999999996,"5min":0.68999999999999995,"15min":0.70399999999999996},"last":0.626},{"interface":"front","average":{"1min":0.437,"5min":0.51800000000000002,"15min":0.42899999999999999},"min":{"1min":0.30199999999999999,"5min":0.27500000000000002,"15min":0.27300000000000002},"max":{"1min":0.52500000000000002,"5min":3.879,"15min":3.879},"last":0.61199999999999999}]},{"osd":6,"last update":"Fri Feb 23 00:44:44 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.48399999999999999,"15min":0.44400000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.98099999999999998,"5min":1.736,"15min":1.736},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.49299999999999999,"15min":0.45600000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.54700000000000004}]},{"osd":7,"last update":"Fri Feb 23 00:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.57799999999999996,"15min":0.53200000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.71299999999999997,"5min":7.0300000000000002,"15min":7.0300000000000002},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.57799999999999996,"15min":0.53100000000000003},"min":{"1min":0.33400000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.58499999999999996,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":0.66300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":192512,"data_stored":485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":188416,"data_stored":476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":229376,"data_stored":570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":164,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":164,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":164,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":164,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":164,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":164,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":164,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":164,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-23T00:45:36.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.045+0000 7f4be7bba700 1 -- 172.21.15.47:0/1926090016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4be00fdbb0 msgr2=0x7f4be00fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:36.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.045+0000 7f4be7bba700 1 --2- 172.21.15.47:0/1926090016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4be00fdbb0 0x7f4be00fdf90 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 rx=0x7f4bdc0047b0 tx=0x7f4bdc009cb0).stop 2024-02-23T00:45:36.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.045+0000 7f4be7bba700 1 -- 172.21.15.47:0/1926090016 shutdown_connections 2024-02-23T00:45:36.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.045+0000 7f4be7bba700 1 --2- 172.21.15.47:0/1926090016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4be00f8420 0x7f4be0104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.045+0000 7f4be7bba700 1 --2- 172.21.15.47:0/1926090016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4be00fdbb0 0x7f4be00fdf90 unknown :-1 s=CLOSED pgs=2486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.045+0000 7f4be7bba700 1 --2- 172.21.15.47:0/1926090016 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4be00fe510 0x7f4be00f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.045+0000 7f4be7bba700 1 -- 172.21.15.47:0/1926090016 >> 172.21.15.47:0/1926090016 conn(0x7f4be00f3800 msgr2=0x7f4be00f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:45:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.045+0000 7f4be7bba700 1 -- 172.21.15.47:0/1926090016 shutdown_connections 2024-02-23T00:45:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.045+0000 7f4be7bba700 1 -- 172.21.15.47:0/1926090016 wait complete. 2024-02-23T00:45:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.045+0000 7f4be7bba700 1 librados: starting msgr at 2024-02-23T00:45:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.045+0000 7f4be7bba700 1 librados: starting objecter 2024-02-23T00:45:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.045+0000 7f4be7bba700 1 Processor -- start 2024-02-23T00:45:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.045+0000 7f4be7bba700 1 -- start start 2024-02-23T00:45:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.045+0000 7f4be7bba700 1 librados: setting wanted keys 2024-02-23T00:45:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.045+0000 7f4be7bba700 1 librados: calling monclient init 2024-02-23T00:45:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.046+0000 7f4be7bba700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4be00f8420 0x7f4be010df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.046+0000 7f4be7bba700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4be00fdbb0 0x7f4be010e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.046+0000 7f4be7bba700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4be00fe510 0x7f4be01080b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.046+0000 7f4be7bba700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be01085a0 con 0x7f4be00f8420 2024-02-23T00:45:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.046+0000 7f4be7bba700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be0108710 con 0x7f4be00fdbb0 2024-02-23T00:45:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.046+0000 7f4be7bba700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be01088d0 con 0x7f4be00fe510 2024-02-23T00:45:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.046+0000 7f4be6157700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4be00fe510 0x7f4be01080b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.046+0000 7f4be6157700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4be00fe510 0x7f4be01080b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:48450/0 (socket says 172.21.15.47:48450) 2024-02-23T00:45:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.046+0000 7f4be6157700 1 -- 172.21.15.47:0/3971772591 learned_addr learned my addr 172.21.15.47:0/3971772591 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:45:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.046+0000 7f4be5956700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4be00f8420 0x7f4be010df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.046+0000 7f4be5155700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4be00fdbb0 0x7f4be010e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.046+0000 7f4be6157700 1 -- 172.21.15.47:0/3971772591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4be00fdbb0 msgr2=0x7f4be010e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.046+0000 7f4be6157700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4be00fdbb0 0x7f4be010e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.046+0000 7f4be6157700 1 -- 172.21.15.47:0/3971772591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4be00f8420 msgr2=0x7f4be010df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.046+0000 7f4be6157700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4be00f8420 0x7f4be010df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.046+0000 7f4be6157700 1 -- 172.21.15.47:0/3971772591 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd0005040 con 0x7f4be00fe510 2024-02-23T00:45:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.046+0000 7f4be6157700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4be00fe510 0x7f4be01080b0 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 rx=0x7f4bd800c960 tx=0x7f4bd8004d60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:36.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.046+0000 7f4bd6ffd700 10 client.?.objecter ms_handle_connect 0x7f4be00fe510 2024-02-23T00:45:36.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.046+0000 7f4bd6ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:45:36.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.046+0000 7f4bd6ffd700 1 -- 172.21.15.47:0/3971772591 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd800a5c0 con 0x7f4be00fe510 2024-02-23T00:45:36.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.046+0000 7f4bd6ffd700 1 -- 172.21.15.47:0/3971772591 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f4bd800abb0 con 0x7f4be00fe510 2024-02-23T00:45:36.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.046+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bdc005040 con 0x7f4be00fe510 2024-02-23T00:45:36.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.046+0000 7f4bd6ffd700 1 -- 172.21.15.47:0/3971772591 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8012ae0 con 0x7f4be00fe510 2024-02-23T00:45:36.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.046+0000 7f4be7bba700 10 client.40934.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:45:36.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.046+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be0108f10 con 0x7f4be00fe510 2024-02-23T00:45:36.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.046+0000 7f4be7bba700 1 librados: init done 2024-02-23T00:45:36.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.047+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f4be0106e90 con 0x7f4be00fe510 2024-02-23T00:45:36.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.048+0000 7f4bd6ffd700 1 -- 172.21.15.47:0/3971772591 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f4bd800a720 con 0x7f4be00fe510 2024-02-23T00:45:36.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.048+0000 7f4bd6ffd700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4bcc062cc0 0x7f4bcc065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:36.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.048+0000 7f4bd6ffd700 1 -- 172.21.15.47:0/3971772591 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(263..263 src has 48..263) v4 ==== 9966+0+0 (secure 0 0 0) 0x7f4bd807d570 con 0x7f4be00fe510 2024-02-23T00:45:36.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.048+0000 7f4bd6ffd700 10 client.40934.objecter ms_dispatch 0x7f4be0001550 osd_map(263..263 src has 48..263) v4 2024-02-23T00:45:36.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.048+0000 7f4bd6ffd700 3 client.40934.objecter handle_osd_map got epochs [263,263] > 0 2024-02-23T00:45:36.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.048+0000 7f4bd6ffd700 3 client.40934.objecter handle_osd_map decoding full epoch 263 2024-02-23T00:45:36.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.048+0000 7f4be5956700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4bcc062cc0 0x7f4bcc065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:36.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.048+0000 7f4bd6ffd700 20 client.40934.objecter dump_active .. 0 homeless 2024-02-23T00:45:36.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.048+0000 7f4bd6ffd700 1 -- 172.21.15.47:0/3971772591 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v263) v1 ==== 54+0+16 (secure 0 0 0) 0x7f4bd8084070 con 0x7f4be00fe510 2024-02-23T00:45:36.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.048+0000 7f4be5956700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4bcc062cc0 0x7f4bcc065180 secure :-1 s=READY pgs=2994 cs=0 l=1 rev1=1 rx=0x7f4bdc004e20 tx=0x7f4bdc009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:36.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.048+0000 7f4bd6ffd700 10 client.40934.objecter ms_handle_connect 0x7f4bcc062cc0 2024-02-23T00:45:36.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.049+0000 7f4be7bba700 10 client.40934.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:36.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.049+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_assign -1 1 2024-02-23T00:45:36.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.049+0000 7f4be7bba700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f4be0028540 0x7f4be00fe950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:36.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.049+0000 7f4be7bba700 20 client.40934.objecter _get_session s=0x7f4be0109590 osd=0 3 2024-02-23T00:45:36.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.049+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be0109590 osd=0 3 2024-02-23T00:45:36.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.049+0000 7f4be7bba700 20 client.40934.objecter _get_session s=0x7f4be0109590 osd=0 3 2024-02-23T00:45:36.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.049+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_remove -1 1 2024-02-23T00:45:36.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.049+0000 7f4be7bba700 20 client.40934.objecter get_session s=0x7f4be0109590 osd=0 3 2024-02-23T00:45:36.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.049+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_assign 0 1 2024-02-23T00:45:36.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.049+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be0109590 osd=0 4 2024-02-23T00:45:36.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.049+0000 7f4be7bba700 10 client.40934.objecter _send_command 1 2024-02-23T00:45:36.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.049+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4be005a040 con 0x7f4be0028540 2024-02-23T00:45:36.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.049+0000 7f4be5155700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f4be0028540 0x7f4be00fe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:36.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.049+0000 7f4be5155700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f4be0028540 0x7f4be00fe950 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:36.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.049+0000 7f4bd6ffd700 10 client.40934.objecter ms_handle_connect 0x7f4be0028540 2024-02-23T00:45:36.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.050+0000 7f4bd6ffd700 1 -- 172.21.15.47:0/3971772591 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4be005a040 con 0x7f4be0028540 2024-02-23T00:45:36.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.050+0000 7f4bd6ffd700 10 client.40934.objecter ms_dispatch 0x7f4be0001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:45:36.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.050+0000 7f4bd6ffd700 10 client.40934.objecter _finish_command 1 = system:0 2024-02-23T00:45:36.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.050+0000 7f4bd6ffd700 20 client.40934.objecter put_session s=0x7f4be0109590 osd=0 4 2024-02-23T00:45:36.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.050+0000 7f4bd6ffd700 15 client.40934.objecter _session_command_op_remove 0 1 2024-02-23T00:45:36.765 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:45:36.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.065+0000 7f4be7bba700 10 client.40934.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:45:36.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.065+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_assign -1 2 2024-02-23T00:45:36.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.065+0000 7f4be7bba700 20 client.40934.objecter _get_session s=0x7f4be0109590 osd=0 3 2024-02-23T00:45:36.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.065+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be0109590 osd=0 3 2024-02-23T00:45:36.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.065+0000 7f4be7bba700 20 client.40934.objecter _get_session s=0x7f4be0109590 osd=0 3 2024-02-23T00:45:36.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.065+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_remove -1 2 2024-02-23T00:45:36.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.065+0000 7f4be7bba700 20 client.40934.objecter get_session s=0x7f4be0109590 osd=0 3 2024-02-23T00:45:36.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.065+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_assign 0 2 2024-02-23T00:45:36.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.065+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be0109590 osd=0 4 2024-02-23T00:45:36.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.065+0000 7f4be7bba700 10 client.40934.objecter _send_command 2 2024-02-23T00:45:36.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.065+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4be00fee40 con 0x7f4be0028540 2024-02-23T00:45:36.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.066+0000 7f4bd6ffd700 1 -- 172.21.15.47:0/3971772591 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4be00fee40 con 0x7f4be0028540 2024-02-23T00:45:36.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.066+0000 7f4bd6ffd700 10 client.40934.objecter ms_dispatch 0x7f4be0001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:45:36.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.066+0000 7f4bd6ffd700 10 client.40934.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:45:36.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.066+0000 7f4bd6ffd700 20 client.40934.objecter put_session s=0x7f4be0109590 osd=0 4 2024-02-23T00:45:36.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.066+0000 7f4bd6ffd700 15 client.40934.objecter _session_command_op_remove 0 2 2024-02-23T00:45:36.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.067+0000 7f4be7bba700 10 client.40934.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:36.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.067+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_assign -1 3 2024-02-23T00:45:36.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.067+0000 7f4be7bba700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f4be0059710 0x7f4be011b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:36.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.067+0000 7f4be7bba700 20 client.40934.objecter _get_session s=0x7f4be00595a0 osd=1 3 2024-02-23T00:45:36.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.067+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be00595a0 osd=1 3 2024-02-23T00:45:36.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.067+0000 7f4be7bba700 20 client.40934.objecter _get_session s=0x7f4be00595a0 osd=1 3 2024-02-23T00:45:36.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.067+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_remove -1 3 2024-02-23T00:45:36.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.067+0000 7f4be7bba700 20 client.40934.objecter get_session s=0x7f4be00595a0 osd=1 3 2024-02-23T00:45:36.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.067+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_assign 1 3 2024-02-23T00:45:36.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.067+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be00595a0 osd=1 4 2024-02-23T00:45:36.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.067+0000 7f4be7bba700 10 client.40934.objecter _send_command 3 2024-02-23T00:45:36.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.067+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4be006cf20 con 0x7f4be0059710 2024-02-23T00:45:36.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.067+0000 7f4be5956700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f4be0059710 0x7f4be011b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:36.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.067+0000 7f4bd6ffd700 10 client.40934.objecter ms_handle_connect 0x7f4be0059710 2024-02-23T00:45:36.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.067+0000 7f4be5956700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f4be0059710 0x7f4be011b250 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:36.768 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:45:36.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.068+0000 7f4bd6ffd700 1 -- 172.21.15.47:0/3971772591 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4be006cf20 con 0x7f4be0059710 2024-02-23T00:45:36.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.068+0000 7f4bd6ffd700 10 client.40934.objecter ms_dispatch 0x7f4be0001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:45:36.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.068+0000 7f4bd6ffd700 10 client.40934.objecter _finish_command 3 = system:0 2024-02-23T00:45:36.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.068+0000 7f4bd6ffd700 20 client.40934.objecter put_session s=0x7f4be00595a0 osd=1 4 2024-02-23T00:45:36.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.068+0000 7f4bd6ffd700 15 client.40934.objecter _session_command_op_remove 1 3 2024-02-23T00:45:36.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.082+0000 7f4be7bba700 10 client.40934.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:45:36.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.082+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_assign -1 4 2024-02-23T00:45:36.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.082+0000 7f4be7bba700 20 client.40934.objecter _get_session s=0x7f4be00595a0 osd=1 3 2024-02-23T00:45:36.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.082+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be00595a0 osd=1 3 2024-02-23T00:45:36.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.082+0000 7f4be7bba700 20 client.40934.objecter _get_session s=0x7f4be00595a0 osd=1 3 2024-02-23T00:45:36.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.082+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_remove -1 4 2024-02-23T00:45:36.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.082+0000 7f4be7bba700 20 client.40934.objecter get_session s=0x7f4be00595a0 osd=1 3 2024-02-23T00:45:36.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.082+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_assign 1 4 2024-02-23T00:45:36.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.082+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be00595a0 osd=1 4 2024-02-23T00:45:36.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.082+0000 7f4be7bba700 10 client.40934.objecter _send_command 4 2024-02-23T00:45:36.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.082+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4be00282e0 con 0x7f4be0059710 2024-02-23T00:45:36.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.084+0000 7f4bd6ffd700 1 -- 172.21.15.47:0/3971772591 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4be00282e0 con 0x7f4be0059710 2024-02-23T00:45:36.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.084+0000 7f4bd6ffd700 10 client.40934.objecter ms_dispatch 0x7f4be0001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:45:36.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.084+0000 7f4bd6ffd700 10 client.40934.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:45:36.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.084+0000 7f4bd6ffd700 20 client.40934.objecter put_session s=0x7f4be00595a0 osd=1 4 2024-02-23T00:45:36.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.084+0000 7f4bd6ffd700 15 client.40934.objecter _session_command_op_remove 1 4 2024-02-23T00:45:36.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.084+0000 7f4be7bba700 10 client.40934.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:36.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.084+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_assign -1 5 2024-02-23T00:45:36.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.084+0000 7f4be7bba700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f4be0027770 0x7f4be00ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:36.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.084+0000 7f4be7bba700 20 client.40934.objecter _get_session s=0x7f4be0027600 osd=2 3 2024-02-23T00:45:36.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.084+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be0027600 osd=2 3 2024-02-23T00:45:36.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.084+0000 7f4be7bba700 20 client.40934.objecter _get_session s=0x7f4be0027600 osd=2 3 2024-02-23T00:45:36.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.084+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_remove -1 5 2024-02-23T00:45:36.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.084+0000 7f4be7bba700 20 client.40934.objecter get_session s=0x7f4be0027600 osd=2 3 2024-02-23T00:45:36.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.084+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_assign 2 5 2024-02-23T00:45:36.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.084+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be0027600 osd=2 4 2024-02-23T00:45:36.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.084+0000 7f4be7bba700 10 client.40934.objecter _send_command 5 2024-02-23T00:45:36.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.084+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4be0121030 con 0x7f4be0027770 2024-02-23T00:45:36.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.084+0000 7f4be5155700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f4be0027770 0x7f4be00ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:36.771 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:45:36.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.084+0000 7f4be5155700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f4be0027770 0x7f4be00ffcf0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:36.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.084+0000 7f4bd6ffd700 10 client.40934.objecter ms_handle_connect 0x7f4be0027770 2024-02-23T00:45:36.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.085+0000 7f4bd6ffd700 1 -- 172.21.15.47:0/3971772591 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4be0121030 con 0x7f4be0027770 2024-02-23T00:45:36.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.085+0000 7f4bd6ffd700 10 client.40934.objecter ms_dispatch 0x7f4be0001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:45:36.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.085+0000 7f4bd6ffd700 10 client.40934.objecter _finish_command 5 = system:0 2024-02-23T00:45:36.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.085+0000 7f4bd6ffd700 20 client.40934.objecter put_session s=0x7f4be0027600 osd=2 4 2024-02-23T00:45:36.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.085+0000 7f4bd6ffd700 15 client.40934.objecter _session_command_op_remove 2 5 2024-02-23T00:45:36.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.099+0000 7f4be7bba700 10 client.40934.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:45:36.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.099+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_assign -1 6 2024-02-23T00:45:36.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.099+0000 7f4be7bba700 20 client.40934.objecter _get_session s=0x7f4be0027600 osd=2 3 2024-02-23T00:45:36.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.099+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be0027600 osd=2 3 2024-02-23T00:45:36.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.099+0000 7f4be7bba700 20 client.40934.objecter _get_session s=0x7f4be0027600 osd=2 3 2024-02-23T00:45:36.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.099+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_remove -1 6 2024-02-23T00:45:36.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.099+0000 7f4be7bba700 20 client.40934.objecter get_session s=0x7f4be0027600 osd=2 3 2024-02-23T00:45:36.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.099+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_assign 2 6 2024-02-23T00:45:36.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.099+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be0027600 osd=2 4 2024-02-23T00:45:36.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.099+0000 7f4be7bba700 10 client.40934.objecter _send_command 6 2024-02-23T00:45:36.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.099+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4be00ff250 con 0x7f4be0027770 2024-02-23T00:45:36.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.101+0000 7f4bd6ffd700 1 -- 172.21.15.47:0/3971772591 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4be00ff250 con 0x7f4be0027770 2024-02-23T00:45:36.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.101+0000 7f4bd6ffd700 10 client.40934.objecter ms_dispatch 0x7f4be0001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:45:36.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.101+0000 7f4bd6ffd700 10 client.40934.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:45:36.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.101+0000 7f4bd6ffd700 20 client.40934.objecter put_session s=0x7f4be0027600 osd=2 4 2024-02-23T00:45:36.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.101+0000 7f4bd6ffd700 15 client.40934.objecter _session_command_op_remove 2 6 2024-02-23T00:45:36.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.101+0000 7f4be7bba700 10 client.40934.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:36.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.101+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_assign -1 7 2024-02-23T00:45:36.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.101+0000 7f4be7bba700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f4be011c420 0x7f4be011fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:36.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.101+0000 7f4be7bba700 20 client.40934.objecter _get_session s=0x7f4be00ff630 osd=3 3 2024-02-23T00:45:36.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.101+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be00ff630 osd=3 3 2024-02-23T00:45:36.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.101+0000 7f4be7bba700 20 client.40934.objecter _get_session s=0x7f4be00ff630 osd=3 3 2024-02-23T00:45:36.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.101+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_remove -1 7 2024-02-23T00:45:36.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.101+0000 7f4be7bba700 20 client.40934.objecter get_session s=0x7f4be00ff630 osd=3 3 2024-02-23T00:45:36.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.101+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_assign 3 7 2024-02-23T00:45:36.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.101+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be00ff630 osd=3 4 2024-02-23T00:45:36.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.101+0000 7f4be7bba700 10 client.40934.objecter _send_command 7 2024-02-23T00:45:36.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.101+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4be0120050 con 0x7f4be011c420 2024-02-23T00:45:36.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.101+0000 7f4be6157700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f4be011c420 0x7f4be011fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:36.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.101+0000 7f4bd6ffd700 10 client.40934.objecter ms_handle_connect 0x7f4be011c420 2024-02-23T00:45:36.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.101+0000 7f4be6157700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f4be011c420 0x7f4be011fa50 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:36.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.102+0000 7f4bd6ffd700 1 -- 172.21.15.47:0/3971772591 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4be0120050 con 0x7f4be011c420 2024-02-23T00:45:36.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.102+0000 7f4bd6ffd700 10 client.40934.objecter ms_dispatch 0x7f4be0001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:45:36.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.102+0000 7f4bd6ffd700 10 client.40934.objecter _finish_command 7 = system:0 2024-02-23T00:45:36.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.102+0000 7f4bd6ffd700 20 client.40934.objecter put_session s=0x7f4be00ff630 osd=3 4 2024-02-23T00:45:36.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.102+0000 7f4bd6ffd700 15 client.40934.objecter _session_command_op_remove 3 7 2024-02-23T00:45:36.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.117+0000 7f4be7bba700 10 client.40934.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:45:36.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.117+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_assign -1 8 2024-02-23T00:45:36.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.117+0000 7f4be7bba700 20 client.40934.objecter _get_session s=0x7f4be00ff630 osd=3 3 2024-02-23T00:45:36.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.117+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be00ff630 osd=3 3 2024-02-23T00:45:36.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.117+0000 7f4be7bba700 20 client.40934.objecter _get_session s=0x7f4be00ff630 osd=3 3 2024-02-23T00:45:36.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.117+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_remove -1 8 2024-02-23T00:45:36.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.117+0000 7f4be7bba700 20 client.40934.objecter get_session s=0x7f4be00ff630 osd=3 3 2024-02-23T00:45:36.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.117+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_assign 3 8 2024-02-23T00:45:36.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.117+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be00ff630 osd=3 4 2024-02-23T00:45:36.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.117+0000 7f4be7bba700 10 client.40934.objecter _send_command 8 2024-02-23T00:45:36.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.117+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4be011b980 con 0x7f4be011c420 2024-02-23T00:45:36.775 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:45:36.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.118+0000 7f4bd6ffd700 1 -- 172.21.15.47:0/3971772591 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4be011b980 con 0x7f4be011c420 2024-02-23T00:45:36.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.118+0000 7f4bd6ffd700 10 client.40934.objecter ms_dispatch 0x7f4be0001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:45:36.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.118+0000 7f4bd6ffd700 10 client.40934.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:45:36.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.118+0000 7f4bd6ffd700 20 client.40934.objecter put_session s=0x7f4be00ff630 osd=3 4 2024-02-23T00:45:36.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.118+0000 7f4bd6ffd700 15 client.40934.objecter _session_command_op_remove 3 8 2024-02-23T00:45:36.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.118+0000 7f4be7bba700 10 client.40934.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:36.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.118+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_assign -1 9 2024-02-23T00:45:36.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.118+0000 7f4be7bba700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f4be01231b0 0x7f4be01255a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:36.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.118+0000 7f4be7bba700 20 client.40934.objecter _get_session s=0x7f4be011b980 osd=4 3 2024-02-23T00:45:36.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.118+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be011b980 osd=4 3 2024-02-23T00:45:36.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.118+0000 7f4be7bba700 20 client.40934.objecter _get_session s=0x7f4be011b980 osd=4 3 2024-02-23T00:45:36.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.118+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_remove -1 9 2024-02-23T00:45:36.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.118+0000 7f4be7bba700 20 client.40934.objecter get_session s=0x7f4be011b980 osd=4 3 2024-02-23T00:45:36.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.118+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_assign 4 9 2024-02-23T00:45:36.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.118+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be011b980 osd=4 4 2024-02-23T00:45:36.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.118+0000 7f4be7bba700 10 client.40934.objecter _send_command 9 2024-02-23T00:45:36.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.118+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4be011bb50 con 0x7f4be01231b0 2024-02-23T00:45:36.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.119+0000 7f4be5956700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f4be01231b0 0x7f4be01255a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:36.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.119+0000 7f4bd6ffd700 10 client.40934.objecter ms_handle_connect 0x7f4be01231b0 2024-02-23T00:45:36.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.119+0000 7f4be5956700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f4be01231b0 0x7f4be01255a0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:36.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.120+0000 7f4bd6ffd700 1 -- 172.21.15.47:0/3971772591 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4be011bb50 con 0x7f4be01231b0 2024-02-23T00:45:36.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.120+0000 7f4bd6ffd700 10 client.40934.objecter ms_dispatch 0x7f4be0001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:45:36.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.120+0000 7f4bd6ffd700 10 client.40934.objecter _finish_command 9 = system:0 2024-02-23T00:45:36.777 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:45:36.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.120+0000 7f4bd6ffd700 20 client.40934.objecter put_session s=0x7f4be011b980 osd=4 4 2024-02-23T00:45:36.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.120+0000 7f4bd6ffd700 15 client.40934.objecter _session_command_op_remove 4 9 2024-02-23T00:45:36.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.139+0000 7f4be7bba700 10 client.40934.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:45:36.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.139+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_assign -1 10 2024-02-23T00:45:36.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.139+0000 7f4be7bba700 20 client.40934.objecter _get_session s=0x7f4be011b980 osd=4 3 2024-02-23T00:45:36.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.139+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be011b980 osd=4 3 2024-02-23T00:45:36.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.139+0000 7f4be7bba700 20 client.40934.objecter _get_session s=0x7f4be011b980 osd=4 3 2024-02-23T00:45:36.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.139+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_remove -1 10 2024-02-23T00:45:36.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.139+0000 7f4be7bba700 20 client.40934.objecter get_session s=0x7f4be011b980 osd=4 3 2024-02-23T00:45:36.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.139+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_assign 4 10 2024-02-23T00:45:36.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.139+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be011b980 osd=4 4 2024-02-23T00:45:36.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.139+0000 7f4be7bba700 10 client.40934.objecter _send_command 10 2024-02-23T00:45:36.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.139+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4be011baf0 con 0x7f4be01231b0 2024-02-23T00:45:36.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.153+0000 7f4bd6ffd700 1 -- 172.21.15.47:0/3971772591 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4be011baf0 con 0x7f4be01231b0 2024-02-23T00:45:36.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.153+0000 7f4bd6ffd700 10 client.40934.objecter ms_dispatch 0x7f4be0001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:45:36.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.153+0000 7f4bd6ffd700 10 client.40934.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:45:36.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.153+0000 7f4bd6ffd700 20 client.40934.objecter put_session s=0x7f4be011b980 osd=4 4 2024-02-23T00:45:36.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.153+0000 7f4bd6ffd700 15 client.40934.objecter _session_command_op_remove 4 10 2024-02-23T00:45:36.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.153+0000 7f4be7bba700 10 client.40934.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:36.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.153+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_assign -1 11 2024-02-23T00:45:36.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.153+0000 7f4be7bba700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f4be0125fa0 0x7f4be0128390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:36.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.153+0000 7f4be7bba700 20 client.40934.objecter _get_session s=0x7f4be011baf0 osd=5 3 2024-02-23T00:45:36.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.153+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be011baf0 osd=5 3 2024-02-23T00:45:36.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.153+0000 7f4be7bba700 20 client.40934.objecter _get_session s=0x7f4be011baf0 osd=5 3 2024-02-23T00:45:36.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.153+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_remove -1 11 2024-02-23T00:45:36.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.153+0000 7f4be7bba700 20 client.40934.objecter get_session s=0x7f4be011baf0 osd=5 3 2024-02-23T00:45:36.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.153+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_assign 5 11 2024-02-23T00:45:36.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.153+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be011baf0 osd=5 4 2024-02-23T00:45:36.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.153+0000 7f4be7bba700 10 client.40934.objecter _send_command 11 2024-02-23T00:45:36.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.153+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4be012b0e0 con 0x7f4be0125fa0 2024-02-23T00:45:36.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.158+0000 7f4be5155700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f4be0125fa0 0x7f4be0128390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:36.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.158+0000 7f4bd6ffd700 10 client.40934.objecter ms_handle_connect 0x7f4be0125fa0 2024-02-23T00:45:36.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.158+0000 7f4be5155700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f4be0125fa0 0x7f4be0128390 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:36.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.159+0000 7f4bd6ffd700 1 -- 172.21.15.47:0/3971772591 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4be012b0e0 con 0x7f4be0125fa0 2024-02-23T00:45:36.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.159+0000 7f4bd6ffd700 10 client.40934.objecter ms_dispatch 0x7f4be0001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:45:36.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.159+0000 7f4bd6ffd700 10 client.40934.objecter _finish_command 11 = system:0 2024-02-23T00:45:36.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.159+0000 7f4bd6ffd700 20 client.40934.objecter put_session s=0x7f4be011baf0 osd=5 4 2024-02-23T00:45:36.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.159+0000 7f4bd6ffd700 15 client.40934.objecter _session_command_op_remove 5 11 2024-02-23T00:45:36.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.174+0000 7f4be7bba700 10 client.40934.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:45:36.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.174+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_assign -1 12 2024-02-23T00:45:36.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.174+0000 7f4be7bba700 20 client.40934.objecter _get_session s=0x7f4be011baf0 osd=5 3 2024-02-23T00:45:36.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.174+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be011baf0 osd=5 3 2024-02-23T00:45:36.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.174+0000 7f4be7bba700 20 client.40934.objecter _get_session s=0x7f4be011baf0 osd=5 3 2024-02-23T00:45:36.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.174+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_remove -1 12 2024-02-23T00:45:36.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.174+0000 7f4be7bba700 20 client.40934.objecter get_session s=0x7f4be011baf0 osd=5 3 2024-02-23T00:45:36.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.174+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_assign 5 12 2024-02-23T00:45:36.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.174+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be011baf0 osd=5 4 2024-02-23T00:45:36.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.174+0000 7f4be7bba700 10 client.40934.objecter _send_command 12 2024-02-23T00:45:36.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.174+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4be012b0e0 con 0x7f4be0125fa0 2024-02-23T00:45:36.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.176+0000 7f4bd6ffd700 1 -- 172.21.15.47:0/3971772591 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4be012b0e0 con 0x7f4be0125fa0 2024-02-23T00:45:36.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.176+0000 7f4bd6ffd700 10 client.40934.objecter ms_dispatch 0x7f4be0001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:45:36.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.176+0000 7f4bd6ffd700 10 client.40934.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:45:36.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.176+0000 7f4bd6ffd700 20 client.40934.objecter put_session s=0x7f4be011baf0 osd=5 4 2024-02-23T00:45:36.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.176+0000 7f4bd6ffd700 15 client.40934.objecter _session_command_op_remove 5 12 2024-02-23T00:45:36.782 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:45:36.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.177+0000 7f4be7bba700 10 client.40934.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:36.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.177+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_assign -1 13 2024-02-23T00:45:36.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.177+0000 7f4be7bba700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f4be0128e30 0x7f4be01292b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:36.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.177+0000 7f4be7bba700 20 client.40934.objecter _get_session s=0x7f4be012b0e0 osd=6 3 2024-02-23T00:45:36.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.177+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be012b0e0 osd=6 3 2024-02-23T00:45:36.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.177+0000 7f4be7bba700 20 client.40934.objecter _get_session s=0x7f4be012b0e0 osd=6 3 2024-02-23T00:45:36.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.177+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_remove -1 13 2024-02-23T00:45:36.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.177+0000 7f4be7bba700 20 client.40934.objecter get_session s=0x7f4be012b0e0 osd=6 3 2024-02-23T00:45:36.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.177+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_assign 6 13 2024-02-23T00:45:36.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.177+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be012b0e0 osd=6 4 2024-02-23T00:45:36.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.177+0000 7f4be7bba700 10 client.40934.objecter _send_command 13 2024-02-23T00:45:36.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.177+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4be012e410 con 0x7f4be0128e30 2024-02-23T00:45:36.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.177+0000 7f4be6157700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f4be0128e30 0x7f4be01292b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:36.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.177+0000 7f4bd6ffd700 10 client.40934.objecter ms_handle_connect 0x7f4be0128e30 2024-02-23T00:45:36.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.177+0000 7f4be6157700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f4be0128e30 0x7f4be01292b0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:36.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.178+0000 7f4bd6ffd700 1 -- 172.21.15.47:0/3971772591 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4be012e410 con 0x7f4be0128e30 2024-02-23T00:45:36.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.178+0000 7f4bd6ffd700 10 client.40934.objecter ms_dispatch 0x7f4be0001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:45:36.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.178+0000 7f4bd6ffd700 10 client.40934.objecter _finish_command 13 = system:0 2024-02-23T00:45:36.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.178+0000 7f4bd6ffd700 20 client.40934.objecter put_session s=0x7f4be012b0e0 osd=6 4 2024-02-23T00:45:36.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.178+0000 7f4bd6ffd700 15 client.40934.objecter _session_command_op_remove 6 13 2024-02-23T00:45:36.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.192+0000 7f4be7bba700 10 client.40934.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:45:36.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.192+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_assign -1 14 2024-02-23T00:45:36.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.192+0000 7f4be7bba700 20 client.40934.objecter _get_session s=0x7f4be012b0e0 osd=6 3 2024-02-23T00:45:36.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.192+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be012b0e0 osd=6 3 2024-02-23T00:45:36.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.192+0000 7f4be7bba700 20 client.40934.objecter _get_session s=0x7f4be012b0e0 osd=6 3 2024-02-23T00:45:36.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.192+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_remove -1 14 2024-02-23T00:45:36.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.192+0000 7f4be7bba700 20 client.40934.objecter get_session s=0x7f4be012b0e0 osd=6 3 2024-02-23T00:45:36.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.192+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_assign 6 14 2024-02-23T00:45:36.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.192+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be012b0e0 osd=6 4 2024-02-23T00:45:36.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.192+0000 7f4be7bba700 10 client.40934.objecter _send_command 14 2024-02-23T00:45:36.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.192+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4be01298b0 con 0x7f4be0128e30 2024-02-23T00:45:36.785 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:45:36.785 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:45:36.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.194+0000 7f4bd6ffd700 1 -- 172.21.15.47:0/3971772591 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4be01298b0 con 0x7f4be0128e30 2024-02-23T00:45:36.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.194+0000 7f4bd6ffd700 10 client.40934.objecter ms_dispatch 0x7f4be0001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:45:36.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.194+0000 7f4bd6ffd700 10 client.40934.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:45:36.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.194+0000 7f4bd6ffd700 20 client.40934.objecter put_session s=0x7f4be012b0e0 osd=6 4 2024-02-23T00:45:36.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.194+0000 7f4bd6ffd700 15 client.40934.objecter _session_command_op_remove 6 14 2024-02-23T00:45:36.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.195+0000 7f4be7bba700 10 client.40934.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:36.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.195+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_assign -1 15 2024-02-23T00:45:36.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.195+0000 7f4be7bba700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f4be0129f30 0x7f4be012a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:36.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.195+0000 7f4be7bba700 20 client.40934.objecter _get_session s=0x7f4be01298b0 osd=7 3 2024-02-23T00:45:36.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.195+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be01298b0 osd=7 3 2024-02-23T00:45:36.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.195+0000 7f4be7bba700 20 client.40934.objecter _get_session s=0x7f4be01298b0 osd=7 3 2024-02-23T00:45:36.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.195+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_remove -1 15 2024-02-23T00:45:36.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.195+0000 7f4be7bba700 20 client.40934.objecter get_session s=0x7f4be01298b0 osd=7 3 2024-02-23T00:45:36.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.195+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_assign 7 15 2024-02-23T00:45:36.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.195+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be01298b0 osd=7 4 2024-02-23T00:45:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.195+0000 7f4be7bba700 10 client.40934.objecter _send_command 15 2024-02-23T00:45:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.195+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4be012aa20 con 0x7f4be0129f30 2024-02-23T00:45:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.195+0000 7f4be5956700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f4be0129f30 0x7f4be012a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.195+0000 7f4bd6ffd700 10 client.40934.objecter ms_handle_connect 0x7f4be0129f30 2024-02-23T00:45:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.195+0000 7f4be5956700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f4be0129f30 0x7f4be012a390 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.196+0000 7f4bd6ffd700 1 -- 172.21.15.47:0/3971772591 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4be012aa20 con 0x7f4be0129f30 2024-02-23T00:45:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.196+0000 7f4bd6ffd700 10 client.40934.objecter ms_dispatch 0x7f4be0001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:45:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.196+0000 7f4bd6ffd700 10 client.40934.objecter _finish_command 15 = system:0 2024-02-23T00:45:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.196+0000 7f4bd6ffd700 20 client.40934.objecter put_session s=0x7f4be01298b0 osd=7 4 2024-02-23T00:45:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.196+0000 7f4bd6ffd700 15 client.40934.objecter _session_command_op_remove 7 15 2024-02-23T00:45:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.210+0000 7f4be7bba700 10 client.40934.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:45:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.210+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_assign -1 16 2024-02-23T00:45:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.210+0000 7f4be7bba700 20 client.40934.objecter _get_session s=0x7f4be01298b0 osd=7 3 2024-02-23T00:45:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.210+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be01298b0 osd=7 3 2024-02-23T00:45:36.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.210+0000 7f4be7bba700 20 client.40934.objecter _get_session s=0x7f4be01298b0 osd=7 3 2024-02-23T00:45:36.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.210+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_remove -1 16 2024-02-23T00:45:36.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.210+0000 7f4be7bba700 20 client.40934.objecter get_session s=0x7f4be01298b0 osd=7 3 2024-02-23T00:45:36.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.210+0000 7f4be7bba700 15 client.40934.objecter _session_command_op_assign 7 16 2024-02-23T00:45:36.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.210+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be01298b0 osd=7 4 2024-02-23T00:45:36.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.210+0000 7f4be7bba700 10 client.40934.objecter _send_command 16 2024-02-23T00:45:36.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.210+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4be012aa20 con 0x7f4be0129f30 2024-02-23T00:45:36.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.212+0000 7f4bd6ffd700 1 -- 172.21.15.47:0/3971772591 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4be012aa20 con 0x7f4be0129f30 2024-02-23T00:45:36.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.212+0000 7f4bd6ffd700 10 client.40934.objecter ms_dispatch 0x7f4be0001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:45:36.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.212+0000 7f4bd6ffd700 10 client.40934.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:45:36.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.212+0000 7f4bd6ffd700 20 client.40934.objecter put_session s=0x7f4be01298b0 osd=7 4 2024-02-23T00:45:36.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.212+0000 7f4bd6ffd700 15 client.40934.objecter _session_command_op_remove 7 16 2024-02-23T00:45:36.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.212+0000 7f4be7bba700 10 librados: watch_flush enter 2024-02-23T00:45:36.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 10 librados: watch_flush exit 2024-02-23T00:45:36.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 10 client.40934.objecter close_session for osd.0 2024-02-23T00:45:36.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f4be0028540 msgr2=0x7f4be00fe950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:36.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f4be0028540 0x7f4be00fe950 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:36.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be0109590 osd=0 1 2024-02-23T00:45:36.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 10 client.40934.objecter close_session for osd.1 2024-02-23T00:45:36.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f4be0059710 msgr2=0x7f4be011b250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:36.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f4be0059710 0x7f4be011b250 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:36.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be00595a0 osd=1 1 2024-02-23T00:45:36.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 10 client.40934.objecter close_session for osd.2 2024-02-23T00:45:36.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f4be0027770 msgr2=0x7f4be00ffcf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:36.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f4be0027770 0x7f4be00ffcf0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:36.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be0027600 osd=2 1 2024-02-23T00:45:36.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 10 client.40934.objecter close_session for osd.3 2024-02-23T00:45:36.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f4be011c420 msgr2=0x7f4be011fa50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:36.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f4be011c420 0x7f4be011fa50 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:36.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be6157700 1 -- 172.21.15.47:0/3971772591 reap_dead start 2024-02-23T00:45:36.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be00ff630 osd=3 1 2024-02-23T00:45:36.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 10 client.40934.objecter close_session for osd.4 2024-02-23T00:45:36.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f4be01231b0 msgr2=0x7f4be01255a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:36.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f4be01231b0 0x7f4be01255a0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:36.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be6157700 1 -- 172.21.15.47:0/3971772591 reap_dead start 2024-02-23T00:45:36.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be011b980 osd=4 1 2024-02-23T00:45:36.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 10 client.40934.objecter close_session for osd.5 2024-02-23T00:45:36.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f4be0125fa0 msgr2=0x7f4be0128390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:36.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f4be0125fa0 0x7f4be0128390 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:36.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be011baf0 osd=5 1 2024-02-23T00:45:36.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 10 client.40934.objecter close_session for osd.6 2024-02-23T00:45:36.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f4be0128e30 msgr2=0x7f4be01292b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:36.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f4be0128e30 0x7f4be01292b0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:36.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be012b0e0 osd=6 1 2024-02-23T00:45:36.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 10 client.40934.objecter close_session for osd.7 2024-02-23T00:45:36.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f4be0129f30 msgr2=0x7f4be012a390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:36.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f4be0129f30 0x7f4be012a390 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:36.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 20 client.40934.objecter put_session s=0x7f4be01298b0 osd=7 1 2024-02-23T00:45:36.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 20 client.40934.objecter shutdown clearing up homeless session... 2024-02-23T00:45:36.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 10 client.40934.objecter successfully canceled tick 2024-02-23T00:45:36.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4bcc062cc0 msgr2=0x7f4bcc065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:36.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4bcc062cc0 0x7f4bcc065180 secure :-1 s=READY pgs=2994 cs=0 l=1 rev1=1 rx=0x7f4bdc004e20 tx=0x7f4bdc009cb0).stop 2024-02-23T00:45:36.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4be00fe510 msgr2=0x7f4be01080b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:36.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 1 --2- 172.21.15.47:0/3971772591 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4be00fe510 0x7f4be01080b0 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 rx=0x7f4bd800c960 tx=0x7f4bd8004d60).stop 2024-02-23T00:45:36.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be6157700 1 -- 172.21.15.47:0/3971772591 reap_dead start 2024-02-23T00:45:36.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 shutdown_connections 2024-02-23T00:45:36.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 >> 172.21.15.47:0/3971772591 conn(0x7f4be00f3800 msgr2=0x7f4be00fb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:45:36.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 shutdown_connections 2024-02-23T00:45:36.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 1 -- 172.21.15.47:0/3971772591 wait complete. 2024-02-23T00:45:36.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:36.213+0000 7f4be7bba700 1 librados: shutdown 2024-02-23T00:45:36.792 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:45:36.792 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:45:36.792 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:45:36.792 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:45:36.792 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:45:36.792 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:45:36.792 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:45:36.792 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:45:36.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:35.242612+0000 mgr.y (mgr.14517) 278 : cluster [DBG] pgmap v370: 250 pgs: 32 unknown, 9 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 205 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:45:36.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:35.838480+0000 mon.b (mon.1) 323 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:36.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:36.049461+0000 mon.c (mon.2) 182 : audit [DBG] from='client.? 172.21.15.47:0/3971772591' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:45:36.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:36.134930+0000 mon.a (mon.0) 1785 : audit [INF] from='client.? 172.21.15.47:0/1933959576' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi047-82378-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi047-82378-36"}]': finished 2024-02-23T00:45:36.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:36.135108+0000 mon.a (mon.0) 1786 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:36.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:36.135227+0000 mon.a (mon.0) 1787 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi047-82386-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:36.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:36.135291+0000 mon.a (mon.0) 1788 : cluster [DBG] osdmap e264: 8 total, 8 up, 8 in 2024-02-23T00:45:36.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:36.148049+0000 mon.a (mon.0) 1789 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:45:36.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:36.150045+0000 mon.c (mon.2) 183 : audit [INF] from='client.? 172.21.15.47:0/3947607643' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi047-82386-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi047-82386-46"}]: dispatch 2024-02-23T00:45:36.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:36.151297+0000 mon.a (mon.0) 1790 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi047-82386-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi047-82386-46"}]: dispatch 2024-02-23T00:45:37.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:35.242612+0000 mgr.y (mgr.14517) 278 : cluster [DBG] pgmap v370: 250 pgs: 32 unknown, 9 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 205 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:45:37.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:35.838480+0000 mon.b (mon.1) 323 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:37.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:36.049461+0000 mon.c (mon.2) 182 : audit [DBG] from='client.? 172.21.15.47:0/3971772591' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:45:37.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:36.134930+0000 mon.a (mon.0) 1785 : audit [INF] from='client.? 172.21.15.47:0/1933959576' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi047-82378-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi047-82378-36"}]': finished 2024-02-23T00:45:37.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:36.135108+0000 mon.a (mon.0) 1786 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:37.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:36.135227+0000 mon.a (mon.0) 1787 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi047-82386-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:37.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:36.135291+0000 mon.a (mon.0) 1788 : cluster [DBG] osdmap e264: 8 total, 8 up, 8 in 2024-02-23T00:45:37.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:36.148049+0000 mon.a (mon.0) 1789 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:45:37.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:36.150045+0000 mon.c (mon.2) 183 : audit [INF] from='client.? 172.21.15.47:0/3947607643' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi047-82386-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi047-82386-46"}]: dispatch 2024-02-23T00:45:37.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:36.151297+0000 mon.a (mon.0) 1790 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi047-82386-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi047-82386-46"}]: dispatch 2024-02-23T00:45:37.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:35.242612+0000 mgr.y (mgr.14517) 278 : cluster [DBG] pgmap v370: 250 pgs: 32 unknown, 9 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 205 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:45:37.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:35.838480+0000 mon.b (mon.1) 323 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:37.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:36.049461+0000 mon.c (mon.2) 182 : audit [DBG] from='client.? 172.21.15.47:0/3971772591' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:45:37.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:36.134930+0000 mon.a (mon.0) 1785 : audit [INF] from='client.? 172.21.15.47:0/1933959576' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi047-82378-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi047-82378-36"}]': finished 2024-02-23T00:45:37.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:36.135108+0000 mon.a (mon.0) 1786 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:37.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:36.135227+0000 mon.a (mon.0) 1787 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi047-82386-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:37.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:36.135291+0000 mon.a (mon.0) 1788 : cluster [DBG] osdmap e264: 8 total, 8 up, 8 in 2024-02-23T00:45:37.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:36.148049+0000 mon.a (mon.0) 1789 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:45:37.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:36.150045+0000 mon.c (mon.2) 183 : audit [INF] from='client.? 172.21.15.47:0/3947607643' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi047-82386-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi047-82386-46"}]: dispatch 2024-02-23T00:45:37.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:36.151297+0000 mon.a (mon.0) 1790 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi047-82386-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi047-82386-46"}]: dispatch 2024-02-23T00:45:37.885 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:36.839509+0000 mon.b (mon.1) 324 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:37.885 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:37.137000+0000 mon.a (mon.0) 1791 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:45:37.885 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:37.576857+0000 mon.a (mon.0) 1792 : cluster [DBG] osdmap e265: 8 total, 8 up, 8 in 2024-02-23T00:45:37.885 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:37.577425+0000 mon.a (mon.0) 1793 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-29"}]: dispatch 2024-02-23T00:45:37.886 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:36.839509+0000 mon.b (mon.1) 324 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:37.886 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:37.137000+0000 mon.a (mon.0) 1791 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:45:37.886 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:37.576857+0000 mon.a (mon.0) 1792 : cluster [DBG] osdmap e265: 8 total, 8 up, 8 in 2024-02-23T00:45:37.886 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:37.577425+0000 mon.a (mon.0) 1793 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-29"}]: dispatch 2024-02-23T00:45:37.918 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:45:37.924 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:37.924 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:45:37.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:36.839509+0000 mon.b (mon.1) 324 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:37.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:37.137000+0000 mon.a (mon.0) 1791 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:45:37.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:37.576857+0000 mon.a (mon.0) 1792 : cluster [DBG] osdmap e265: 8 total, 8 up, 8 in 2024-02-23T00:45:37.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:37.577425+0000 mon.a (mon.0) 1793 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-29"}]: dispatch 2024-02-23T00:45:38.025 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:45:38.055 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:38.179 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:45:38.182 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:38.182 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:45:38.283 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:45:38.314 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:38.336 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:45:38.338 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:38.339 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:45:38.440 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:45:38.469 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:38.489 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:45:38.491 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:38.492 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:45:38.593 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:45:38.622 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:38.642 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:45:38.644 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:38.645 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:45:38.745 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:45:38.771 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:37.243480+0000 mgr.y (mgr.14517) 279 : cluster [DBG] pgmap v373: 258 pgs: 3 creating+activating, 2 creating+peering, 253 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-23T00:45:38.771 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:37.840933+0000 mon.b (mon.1) 325 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:38.771 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:38.140679+0000 mon.a (mon.0) 1794 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi047-82386-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi047-82386-46"}]': finished 2024-02-23T00:45:38.771 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:38.140776+0000 mon.a (mon.0) 1795 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-29"}]': finished 2024-02-23T00:45:38.771 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:38.140834+0000 mon.a (mon.0) 1796 : cluster [DBG] osdmap e266: 8 total, 8 up, 8 in 2024-02-23T00:45:38.772 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:38.149417+0000 mon.a (mon.0) 1797 : audit [INF] from='client.? 172.21.15.47:0/1933959576' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi047-82378-36"}]: dispatch 2024-02-23T00:45:38.772 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:38.149755+0000 mon.a (mon.0) 1798 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-29", "mode": "writeback"}]: dispatch 2024-02-23T00:45:38.772 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:37.243480+0000 mgr.y (mgr.14517) 279 : cluster [DBG] pgmap v373: 258 pgs: 3 creating+activating, 2 creating+peering, 253 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-23T00:45:38.772 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:37.840933+0000 mon.b (mon.1) 325 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:38.772 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:38.140679+0000 mon.a (mon.0) 1794 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi047-82386-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi047-82386-46"}]': finished 2024-02-23T00:45:38.772 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:38.140776+0000 mon.a (mon.0) 1795 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-29"}]': finished 2024-02-23T00:45:38.772 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:38.140834+0000 mon.a (mon.0) 1796 : cluster [DBG] osdmap e266: 8 total, 8 up, 8 in 2024-02-23T00:45:38.772 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:38.149417+0000 mon.a (mon.0) 1797 : audit [INF] from='client.? 172.21.15.47:0/1933959576' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi047-82378-36"}]: dispatch 2024-02-23T00:45:38.772 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:38.149755+0000 mon.a (mon.0) 1798 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-29", "mode": "writeback"}]: dispatch 2024-02-23T00:45:38.774 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:38.849 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:45:38.851 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:38.852 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:45:38.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:37.243480+0000 mgr.y (mgr.14517) 279 : cluster [DBG] pgmap v373: 258 pgs: 3 creating+activating, 2 creating+peering, 253 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-23T00:45:38.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:37.840933+0000 mon.b (mon.1) 325 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:38.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:38.140679+0000 mon.a (mon.0) 1794 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi047-82386-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi047-82386-46"}]': finished 2024-02-23T00:45:38.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:38.140776+0000 mon.a (mon.0) 1795 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-29"}]': finished 2024-02-23T00:45:38.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:38.140834+0000 mon.a (mon.0) 1796 : cluster [DBG] osdmap e266: 8 total, 8 up, 8 in 2024-02-23T00:45:38.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:38.149417+0000 mon.a (mon.0) 1797 : audit [INF] from='client.? 172.21.15.47:0/1933959576' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi047-82378-36"}]: dispatch 2024-02-23T00:45:38.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:38.149755+0000 mon.a (mon.0) 1798 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-29", "mode": "writeback"}]: dispatch 2024-02-23T00:45:38.953 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:45:38.980 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:39.371 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph pg dump --format=json 2024-02-23T00:45:39.529 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:45:39.532 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:39.532 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:45:39.588 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:45:39.634 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:45:39.660 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:38.701556+0000 mon.c (mon.2) 184 : audit [INF] from='client.? 172.21.15.47:0/3224310251' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:45:39.661 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:38.701855+0000 mon.a (mon.0) 1799 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:45:39.661 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:38.842108+0000 mon.b (mon.1) 326 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:39.661 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:39.138646+0000 mon.a (mon.0) 1800 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:45:39.661 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:39.138671+0000 mon.a (mon.0) 1801 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:45:39.661 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:39.143125+0000 mon.a (mon.0) 1802 : audit [INF] from='client.? 172.21.15.47:0/1933959576' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi047-82378-36"}]': finished 2024-02-23T00:45:39.661 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:39.143261+0000 mon.a (mon.0) 1803 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-29", "mode": "writeback"}]': finished 2024-02-23T00:45:39.661 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:39.143342+0000 mon.a (mon.0) 1804 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:45:39.661 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:39.143403+0000 mon.a (mon.0) 1805 : cluster [DBG] osdmap e267: 8 total, 8 up, 8 in 2024-02-23T00:45:39.661 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:39.149893+0000 mon.a (mon.0) 1806 : audit [INF] from='client.? 172.21.15.47:0/1933959576' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi047-82378-36"}]: dispatch 2024-02-23T00:45:39.661 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:39.225044+0000 mon.a (mon.0) 1807 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:45:39.661 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:38.701556+0000 mon.c (mon.2) 184 : audit [INF] from='client.? 172.21.15.47:0/3224310251' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:45:39.662 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:38.701855+0000 mon.a (mon.0) 1799 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:45:39.662 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:38.842108+0000 mon.b (mon.1) 326 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:39.662 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:39.138646+0000 mon.a (mon.0) 1800 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:45:39.662 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:39.138671+0000 mon.a (mon.0) 1801 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:45:39.662 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:39.143125+0000 mon.a (mon.0) 1802 : audit [INF] from='client.? 172.21.15.47:0/1933959576' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi047-82378-36"}]': finished 2024-02-23T00:45:39.662 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:39.143261+0000 mon.a (mon.0) 1803 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-29", "mode": "writeback"}]': finished 2024-02-23T00:45:39.662 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:39.143342+0000 mon.a (mon.0) 1804 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:45:39.662 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:39.143403+0000 mon.a (mon.0) 1805 : cluster [DBG] osdmap e267: 8 total, 8 up, 8 in 2024-02-23T00:45:39.662 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:39.149893+0000 mon.a (mon.0) 1806 : audit [INF] from='client.? 172.21.15.47:0/1933959576' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi047-82378-36"}]: dispatch 2024-02-23T00:45:39.662 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:39.225044+0000 mon.a (mon.0) 1807 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:45:39.663 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:39.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.503+0000 7f8db59a3700 1 -- 172.21.15.47:0/1130726041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8db00fe510 msgr2=0x7f8db00f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:39.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.503+0000 7f8db59a3700 1 --2- 172.21.15.47:0/1130726041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8db00fe510 0x7f8db00f7d40 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 rx=0x7f8d98004870 tx=0x7f8d98009cb0).stop 2024-02-23T00:45:39.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.503+0000 7f8db59a3700 1 -- 172.21.15.47:0/1130726041 shutdown_connections 2024-02-23T00:45:39.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.503+0000 7f8db59a3700 1 --2- 172.21.15.47:0/1130726041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8db00f8420 0x7f8db0104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:39.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.503+0000 7f8db59a3700 1 --2- 172.21.15.47:0/1130726041 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8db00fdbb0 0x7f8db00fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.503+0000 7f8db59a3700 1 --2- 172.21.15.47:0/1130726041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8db00fe510 0x7f8db00f7d40 unknown :-1 s=CLOSED pgs=2487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.503+0000 7f8db59a3700 1 -- 172.21.15.47:0/1130726041 >> 172.21.15.47:0/1130726041 conn(0x7f8db00f3800 msgr2=0x7f8db00f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:45:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.503+0000 7f8db59a3700 1 -- 172.21.15.47:0/1130726041 shutdown_connections 2024-02-23T00:45:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.503+0000 7f8db59a3700 1 -- 172.21.15.47:0/1130726041 wait complete. 2024-02-23T00:45:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.504+0000 7f8db59a3700 1 librados: starting msgr at 2024-02-23T00:45:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.504+0000 7f8db59a3700 1 librados: starting objecter 2024-02-23T00:45:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.504+0000 7f8db59a3700 1 Processor -- start 2024-02-23T00:45:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.504+0000 7f8db59a3700 1 -- start start 2024-02-23T00:45:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.504+0000 7f8db59a3700 1 librados: setting wanted keys 2024-02-23T00:45:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.504+0000 7f8db59a3700 1 librados: calling monclient init 2024-02-23T00:45:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.504+0000 7f8db59a3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8db00f8420 0x7f8db01100a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.504+0000 7f8db59a3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8db00fdbb0 0x7f8db0110590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.504+0000 7f8db59a3700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8db00fe510 0x7f8db010a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.504+0000 7f8db59a3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db010a730 con 0x7f8db00fdbb0 2024-02-23T00:45:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.504+0000 7f8db59a3700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db010a8a0 con 0x7f8db00f8420 2024-02-23T00:45:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.504+0000 7f8db59a3700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db010aa60 con 0x7f8db00fe510 2024-02-23T00:45:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.504+0000 7f8dae7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8db00fdbb0 0x7f8db0110590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.504+0000 7f8daf7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8db00fe510 0x7f8db010a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.504+0000 7f8dae7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8db00fdbb0 0x7f8db0110590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59216/0 (socket says 172.21.15.47:59216) 2024-02-23T00:45:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.504+0000 7f8daf7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8db00fe510 0x7f8db010a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:44506/0 (socket says 172.21.15.47:44506) 2024-02-23T00:45:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.504+0000 7f8dae7fc700 1 -- 172.21.15.47:0/3224310251 learned_addr learned my addr 172.21.15.47:0/3224310251 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:45:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.505+0000 7f8daeffd700 1 --2- 172.21.15.47:0/3224310251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8db00f8420 0x7f8db01100a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.505+0000 7f8daf7fe700 1 -- 172.21.15.47:0/3224310251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8db00f8420 msgr2=0x7f8db01100a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.505+0000 7f8daf7fe700 1 --2- 172.21.15.47:0/3224310251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8db00f8420 0x7f8db01100a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.505+0000 7f8daf7fe700 1 -- 172.21.15.47:0/3224310251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8db00fdbb0 msgr2=0x7f8db0110590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.505+0000 7f8daf7fe700 1 --2- 172.21.15.47:0/3224310251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8db00fdbb0 0x7f8db0110590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.505+0000 7f8daf7fe700 1 -- 172.21.15.47:0/3224310251 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d98005040 con 0x7f8db00fe510 2024-02-23T00:45:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.505+0000 7f8dae7fc700 1 --2- 172.21.15.47:0/3224310251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8db00fdbb0 0x7f8db0110590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:45:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.505+0000 7f8daeffd700 1 --2- 172.21.15.47:0/3224310251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8db00f8420 0x7f8db01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:45:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.505+0000 7f8daf7fe700 1 --2- 172.21.15.47:0/3224310251 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8db00fe510 0x7f8db010a240 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 rx=0x7f8da800d8d0 tx=0x7f8da8004db0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.505+0000 7f8da7fff700 10 client.?.objecter ms_handle_connect 0x7f8db00fe510 2024-02-23T00:45:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.505+0000 7f8da7fff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:45:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.505+0000 7f8da7fff700 1 -- 172.21.15.47:0/3224310251 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da8004020 con 0x7f8db00fe510 2024-02-23T00:45:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.505+0000 7f8db59a3700 1 -- 172.21.15.47:0/3224310251 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db010ad40 con 0x7f8db00fe510 2024-02-23T00:45:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.505+0000 7f8da7fff700 1 -- 172.21.15.47:0/3224310251 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f8da800f040 con 0x7f8db00fe510 2024-02-23T00:45:39.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.505+0000 7f8da7fff700 1 -- 172.21.15.47:0/3224310251 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da8003430 con 0x7f8db00fe510 2024-02-23T00:45:39.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.505+0000 7f8db59a3700 10 client.40940.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:45:39.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.505+0000 7f8db59a3700 1 -- 172.21.15.47:0/3224310251 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db010b0c0 con 0x7f8db00fe510 2024-02-23T00:45:39.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.505+0000 7f8db59a3700 1 librados: init done 2024-02-23T00:45:39.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.506+0000 7f8db59a3700 1 -- 172.21.15.47:0/3224310251 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db005ee80 con 0x7f8db00fe510 2024-02-23T00:45:39.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.507+0000 7f8da7fff700 1 -- 172.21.15.47:0/3224310251 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f8da800b510 con 0x7f8db00fe510 2024-02-23T00:45:39.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.507+0000 7f8da7fff700 1 --2- 172.21.15.47:0/3224310251 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8d9c062cb0 0x7f8d9c065170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:39.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.507+0000 7f8da7fff700 1 -- 172.21.15.47:0/3224310251 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(266..266 src has 48..266) v4 ==== 10755+0+0 (secure 0 0 0) 0x7f8da807d760 con 0x7f8db00fe510 2024-02-23T00:45:39.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.507+0000 7f8da7fff700 10 client.40940.objecter ms_dispatch 0x7f8db0001550 osd_map(266..266 src has 48..266) v4 2024-02-23T00:45:39.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.507+0000 7f8da7fff700 3 client.40940.objecter handle_osd_map got epochs [266,266] > 0 2024-02-23T00:45:39.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.507+0000 7f8da7fff700 3 client.40940.objecter handle_osd_map decoding full epoch 266 2024-02-23T00:45:39.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.507+0000 7f8daeffd700 1 --2- 172.21.15.47:0/3224310251 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8d9c062cb0 0x7f8d9c065170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:39.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.507+0000 7f8da7fff700 20 client.40940.objecter dump_active .. 0 homeless 2024-02-23T00:45:39.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.507+0000 7f8da7fff700 10 client.40940.objecter ms_handle_connect 0x7f8d9c062cb0 2024-02-23T00:45:39.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.507+0000 7f8daeffd700 1 --2- 172.21.15.47:0/3224310251 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8d9c062cb0 0x7f8d9c065170 secure :-1 s=READY pgs=2995 cs=0 l=1 rev1=1 rx=0x7f8da0007fd0 tx=0x7f8da000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:39.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.512+0000 7f8da7fff700 1 -- 172.21.15.47:0/3224310251 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f8da804c770 con 0x7f8db00fe510 2024-02-23T00:45:39.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:38.700+0000 7f8db59a3700 1 -- 172.21.15.47:0/3224310251 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7f8db005fa60 con 0x7f8db00fe510 2024-02-23T00:45:39.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:39.147+0000 7f8da7fff700 1 -- 172.21.15.47:0/3224310251 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v267) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8da804fd90 con 0x7f8db00fe510 2024-02-23T00:45:39.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:39.152+0000 7f8db59a3700 10 librados: watch_flush enter 2024-02-23T00:45:39.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:39.153+0000 7f8db59a3700 10 librados: watch_flush exit 2024-02-23T00:45:39.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:39.153+0000 7f8db59a3700 20 client.40940.objecter shutdown clearing up homeless session... 2024-02-23T00:45:39.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:39.153+0000 7f8db59a3700 10 client.40940.objecter successfully canceled tick 2024-02-23T00:45:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:39.153+0000 7f8db59a3700 1 -- 172.21.15.47:0/3224310251 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8d9c062cb0 msgr2=0x7f8d9c065170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:39.153+0000 7f8db59a3700 1 --2- 172.21.15.47:0/3224310251 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8d9c062cb0 0x7f8d9c065170 secure :-1 s=READY pgs=2995 cs=0 l=1 rev1=1 rx=0x7f8da0007fd0 tx=0x7f8da000b040).stop 2024-02-23T00:45:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:39.153+0000 7f8db59a3700 1 -- 172.21.15.47:0/3224310251 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8db00fe510 msgr2=0x7f8db010a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:39.153+0000 7f8db59a3700 1 --2- 172.21.15.47:0/3224310251 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8db00fe510 0x7f8db010a240 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 rx=0x7f8da800d8d0 tx=0x7f8da8004db0).stop 2024-02-23T00:45:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:39.153+0000 7f8db59a3700 1 -- 172.21.15.47:0/3224310251 shutdown_connections 2024-02-23T00:45:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:39.153+0000 7f8db59a3700 1 --2- 172.21.15.47:0/3224310251 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8d9c062cb0 0x7f8d9c065170 unknown :-1 s=CLOSED pgs=2995 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:39.153+0000 7f8db59a3700 1 --2- 172.21.15.47:0/3224310251 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8db00fe510 0x7f8db010a240 unknown :-1 s=CLOSED pgs=2579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:39.153+0000 7f8db59a3700 1 --2- 172.21.15.47:0/3224310251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8db00f8420 0x7f8db01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:39.153+0000 7f8db59a3700 1 --2- 172.21.15.47:0/3224310251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8db00fdbb0 0x7f8db0110590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:39.153+0000 7f8db59a3700 1 -- 172.21.15.47:0/3224310251 >> 172.21.15.47:0/3224310251 conn(0x7f8db00f3800 msgr2=0x7f8db006af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:45:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:39.153+0000 7f8db59a3700 1 -- 172.21.15.47:0/3224310251 shutdown_connections 2024-02-23T00:45:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:39.153+0000 7f8db59a3700 1 -- 172.21.15.47:0/3224310251 wait complete. 2024-02-23T00:45:39.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:39.153+0000 7f8db59a3700 1 librados: shutdown 2024-02-23T00:45:39.707 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is set 2024-02-23T00:45:39.911 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:45:39.914 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:39.914 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:45:39.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:38.701556+0000 mon.c (mon.2) 184 : audit [INF] from='client.? 172.21.15.47:0/3224310251' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:45:39.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:38.701855+0000 mon.a (mon.0) 1799 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:45:39.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:38.842108+0000 mon.b (mon.1) 326 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:39.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:39.138646+0000 mon.a (mon.0) 1800 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:45:39.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:39.138671+0000 mon.a (mon.0) 1801 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:45:39.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:39.143125+0000 mon.a (mon.0) 1802 : audit [INF] from='client.? 172.21.15.47:0/1933959576' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi047-82378-36"}]': finished 2024-02-23T00:45:39.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:39.143261+0000 mon.a (mon.0) 1803 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-29", "mode": "writeback"}]': finished 2024-02-23T00:45:39.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:39.143342+0000 mon.a (mon.0) 1804 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:45:39.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:39.143403+0000 mon.a (mon.0) 1805 : cluster [DBG] osdmap e267: 8 total, 8 up, 8 in 2024-02-23T00:45:39.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:39.149893+0000 mon.a (mon.0) 1806 : audit [INF] from='client.? 172.21.15.47:0/1933959576' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi047-82378-36"}]: dispatch 2024-02-23T00:45:39.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:39.225044+0000 mon.a (mon.0) 1807 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:45:40.015 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:45:40.044 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:40.072 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:45:40.084 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:40.084 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:45:40.185 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:45:40.217 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:40.238 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:45:40.241 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:40.241 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:45:40.342 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:45:40.370 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:40.408 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:45:40.413 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:40.413 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:45:40.413 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:45:40.415 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:45:40.514 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:45:40.546 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:40.567 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:45:40.569 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:40.570 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:45:40.644 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:45:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:45:40] "GET /metrics HTTP/1.1" 200 192377 "" "Prometheus/2.33.4" 2024-02-23T00:45:40.671 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:45:40.700 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:40.720 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:45:40.722 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:40.722 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:45:40.823 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:45:40.849 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:39.243981+0000 mgr.y (mgr.14517) 280 : cluster [DBG] pgmap v376: 258 pgs: 8 unknown, 250 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:45:40.849 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:39.843182+0000 mon.b (mon.1) 327 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:40.849 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:40.134475+0000 mon.a (mon.0) 1808 : audit [INF] from='client.? 172.21.15.47:0/1933959576' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi047-82378-36"}]': finished 2024-02-23T00:45:40.849 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:40.134573+0000 mon.a (mon.0) 1809 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:45:40.849 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:40.134624+0000 mon.a (mon.0) 1810 : cluster [DBG] osdmap e268: 8 total, 8 up, 8 in 2024-02-23T00:45:40.850 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:40.139065+0000 mon.a (mon.0) 1811 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-29"}]: dispatch 2024-02-23T00:45:40.850 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:40.139405+0000 mon.c (mon.2) 185 : audit [INF] from='client.? 172.21.15.47:0/3947607643' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi047-82386-46"}]: dispatch 2024-02-23T00:45:40.850 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:40.139981+0000 mon.a (mon.0) 1812 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi047-82386-46"}]: dispatch 2024-02-23T00:45:40.850 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:40.156220+0000 mon.a (mon.0) 1813 : audit [INF] from='client.? 172.21.15.47:0/1534291680' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi047-82378-37"}]: dispatch 2024-02-23T00:45:40.850 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:40.157060+0000 mon.a (mon.0) 1814 : audit [INF] from='client.? 172.21.15.47:0/1534291680' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi047-82378-37"}]: dispatch 2024-02-23T00:45:40.850 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:40.157355+0000 mon.a (mon.0) 1815 : audit [INF] from='client.? 172.21.15.47:0/1534291680' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi047-82378-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:40.851 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:40.870 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:45:40.872 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:40.873 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:45:40.974 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:45:41.001 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:39.243981+0000 mgr.y (mgr.14517) 280 : cluster [DBG] pgmap v376: 258 pgs: 8 unknown, 250 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:45:41.001 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:39.843182+0000 mon.b (mon.1) 327 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:41.001 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:40.134475+0000 mon.a (mon.0) 1808 : audit [INF] from='client.? 172.21.15.47:0/1933959576' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi047-82378-36"}]': finished 2024-02-23T00:45:41.001 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:40.134573+0000 mon.a (mon.0) 1809 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:45:41.001 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:40.134624+0000 mon.a (mon.0) 1810 : cluster [DBG] osdmap e268: 8 total, 8 up, 8 in 2024-02-23T00:45:41.001 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:40.139065+0000 mon.a (mon.0) 1811 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-29"}]: dispatch 2024-02-23T00:45:41.001 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:40.139405+0000 mon.c (mon.2) 185 : audit [INF] from='client.? 172.21.15.47:0/3947607643' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi047-82386-46"}]: dispatch 2024-02-23T00:45:41.001 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:40.139981+0000 mon.a (mon.0) 1812 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi047-82386-46"}]: dispatch 2024-02-23T00:45:41.001 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:40.156220+0000 mon.a (mon.0) 1813 : audit [INF] from='client.? 172.21.15.47:0/1534291680' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi047-82378-37"}]: dispatch 2024-02-23T00:45:41.001 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:40.157060+0000 mon.a (mon.0) 1814 : audit [INF] from='client.? 172.21.15.47:0/1534291680' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi047-82378-37"}]: dispatch 2024-02-23T00:45:41.002 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:40.157355+0000 mon.a (mon.0) 1815 : audit [INF] from='client.? 172.21.15.47:0/1534291680' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi047-82378-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:41.002 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:39.243981+0000 mgr.y (mgr.14517) 280 : cluster [DBG] pgmap v376: 258 pgs: 8 unknown, 250 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:45:41.002 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:39.843182+0000 mon.b (mon.1) 327 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:41.002 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:40.134475+0000 mon.a (mon.0) 1808 : audit [INF] from='client.? 172.21.15.47:0/1933959576' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi047-82378-36"}]': finished 2024-02-23T00:45:41.002 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:40.134573+0000 mon.a (mon.0) 1809 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:45:41.002 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:40.134624+0000 mon.a (mon.0) 1810 : cluster [DBG] osdmap e268: 8 total, 8 up, 8 in 2024-02-23T00:45:41.002 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:40.139065+0000 mon.a (mon.0) 1811 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-29"}]: dispatch 2024-02-23T00:45:41.002 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:40.139405+0000 mon.c (mon.2) 185 : audit [INF] from='client.? 172.21.15.47:0/3947607643' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi047-82386-46"}]: dispatch 2024-02-23T00:45:41.003 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:40.139981+0000 mon.a (mon.0) 1812 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi047-82386-46"}]: dispatch 2024-02-23T00:45:41.003 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:40.156220+0000 mon.a (mon.0) 1813 : audit [INF] from='client.? 172.21.15.47:0/1534291680' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi047-82378-37"}]: dispatch 2024-02-23T00:45:41.003 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:40.157060+0000 mon.a (mon.0) 1814 : audit [INF] from='client.? 172.21.15.47:0/1534291680' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi047-82378-37"}]: dispatch 2024-02-23T00:45:41.003 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:40.157355+0000 mon.a (mon.0) 1815 : audit [INF] from='client.? 172.21.15.47:0/1534291680' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi047-82378-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:41.003 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:41.056 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:45:41.059 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:41.059 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:45:41.160 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:45:41.189 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:41.208 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:45:41.211 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:41.212 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:45:41.312 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:45:41.339 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:41.357 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:45:41.360 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:41.360 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:45:41.461 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:45:41.487 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:41.506 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:45:41.508 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:41.509 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:45:41.609 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:45:41.638 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:41.693 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:45:41.695 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:41.696 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:45:41.797 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:45:41.822 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:40.844178+0000 mon.b (mon.1) 328 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:41.822 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:41.131797+0000 mon.a (mon.0) 1816 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:45:41.822 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:41.136143+0000 mon.a (mon.0) 1817 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-29"}]': finished 2024-02-23T00:45:41.822 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:41.136220+0000 mon.a (mon.0) 1818 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi047-82386-46"}]': finished 2024-02-23T00:45:41.822 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:41.136290+0000 mon.a (mon.0) 1819 : audit [INF] from='client.? 172.21.15.47:0/1534291680' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi047-82378-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:41.822 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:41.136355+0000 mon.a (mon.0) 1820 : cluster [DBG] osdmap e269: 8 total, 8 up, 8 in 2024-02-23T00:45:41.823 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:41.138980+0000 mon.c (mon.2) 186 : audit [INF] from='client.? 172.21.15.47:0/3947607643' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi047-82386-46"}]: dispatch 2024-02-23T00:45:41.823 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:41.139124+0000 mon.a (mon.0) 1821 : audit [INF] from='client.? 172.21.15.47:0/1534291680' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi047-82378-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi047-82378-37"}]: dispatch 2024-02-23T00:45:41.823 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:41.139582+0000 mon.a (mon.0) 1822 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi047-82386-46"}]: dispatch 2024-02-23T00:45:41.823 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:41.300579+0000 mon.a (mon.0) 1823 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:45:41.823 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:41.842 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:45:41.845 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:41.845 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:45:41.946 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:45:41.959 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:40.844178+0000 mon.b (mon.1) 328 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:41.959 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:41.131797+0000 mon.a (mon.0) 1816 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:45:41.959 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:41.136143+0000 mon.a (mon.0) 1817 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-29"}]': finished 2024-02-23T00:45:41.959 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:41.136220+0000 mon.a (mon.0) 1818 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi047-82386-46"}]': finished 2024-02-23T00:45:41.959 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:41.136290+0000 mon.a (mon.0) 1819 : audit [INF] from='client.? 172.21.15.47:0/1534291680' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi047-82378-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:41.960 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:41.136355+0000 mon.a (mon.0) 1820 : cluster [DBG] osdmap e269: 8 total, 8 up, 8 in 2024-02-23T00:45:41.960 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:41.138980+0000 mon.c (mon.2) 186 : audit [INF] from='client.? 172.21.15.47:0/3947607643' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi047-82386-46"}]: dispatch 2024-02-23T00:45:41.960 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:41.139124+0000 mon.a (mon.0) 1821 : audit [INF] from='client.? 172.21.15.47:0/1534291680' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi047-82378-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi047-82378-37"}]: dispatch 2024-02-23T00:45:41.960 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:41.139582+0000 mon.a (mon.0) 1822 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi047-82386-46"}]: dispatch 2024-02-23T00:45:41.960 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:41.300579+0000 mon.a (mon.0) 1823 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:45:41.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:40.844178+0000 mon.b (mon.1) 328 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:41.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:41.131797+0000 mon.a (mon.0) 1816 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:45:41.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:41.136143+0000 mon.a (mon.0) 1817 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-29"}]': finished 2024-02-23T00:45:41.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:41.136220+0000 mon.a (mon.0) 1818 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi047-82386-46"}]': finished 2024-02-23T00:45:41.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:41.136290+0000 mon.a (mon.0) 1819 : audit [INF] from='client.? 172.21.15.47:0/1534291680' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi047-82378-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:41.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:41.136355+0000 mon.a (mon.0) 1820 : cluster [DBG] osdmap e269: 8 total, 8 up, 8 in 2024-02-23T00:45:41.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:41.138980+0000 mon.c (mon.2) 186 : audit [INF] from='client.? 172.21.15.47:0/3947607643' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi047-82386-46"}]: dispatch 2024-02-23T00:45:41.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:41.139124+0000 mon.a (mon.0) 1821 : audit [INF] from='client.? 172.21.15.47:0/1534291680' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi047-82378-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi047-82378-37"}]: dispatch 2024-02-23T00:45:41.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:41.139582+0000 mon.a (mon.0) 1822 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi047-82386-46"}]: dispatch 2024-02-23T00:45:41.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:41.300579+0000 mon.a (mon.0) 1823 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:45:41.976 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:42.001 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:45:42.005 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:42.005 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:45:42.106 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:45:42.136 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:42.240 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:45:42.240 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-02-23T00:45:42.241 INFO:teuthology.orchestra.run.smithi047.stdout: "num_ops": 0 2024-02-23T00:45:42.241 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:45:42.246 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:45:42.249 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:42.249 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:45:42.350 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:45:42.379 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:42.396 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:45:42.398 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:42.399 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:45:42.442 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset noscrub 2024-02-23T00:45:42.499 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:45:42.528 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:42.693 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:45:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:45:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:45:42.693 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:45:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:45:43.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:41.244400+0000 mgr.y (mgr.14517) 281 : cluster [DBG] pgmap v379: 250 pgs: 250 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:45:43.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:41.845025+0000 mon.b (mon.1) 329 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:43.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:42.581556+0000 mon.a (mon.0) 1824 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi047-82386-46"}]': finished 2024-02-23T00:45:43.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:42.581647+0000 mon.a (mon.0) 1825 : cluster [DBG] osdmap e270: 8 total, 8 up, 8 in 2024-02-23T00:45:43.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:42.582348+0000 mon.a (mon.0) 1826 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.1a", "id": [7, 5]}]: dispatch 2024-02-23T00:45:43.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:41.244400+0000 mgr.y (mgr.14517) 281 : cluster [DBG] pgmap v379: 250 pgs: 250 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:45:43.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:41.845025+0000 mon.b (mon.1) 329 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:43.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:42.581556+0000 mon.a (mon.0) 1824 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi047-82386-46"}]': finished 2024-02-23T00:45:43.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:42.581647+0000 mon.a (mon.0) 1825 : cluster [DBG] osdmap e270: 8 total, 8 up, 8 in 2024-02-23T00:45:43.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:42.582348+0000 mon.a (mon.0) 1826 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.1a", "id": [7, 5]}]: dispatch 2024-02-23T00:45:43.373 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.2 dump_blocked_ops 2024-02-23T00:45:43.378 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:45:43.380 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:43.381 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:45:43.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:41.244400+0000 mgr.y (mgr.14517) 281 : cluster [DBG] pgmap v379: 250 pgs: 250 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:45:43.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:41.845025+0000 mon.b (mon.1) 329 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:43.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:42.581556+0000 mon.a (mon.0) 1824 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi047-82386-46"}]': finished 2024-02-23T00:45:43.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:42.581647+0000 mon.a (mon.0) 1825 : cluster [DBG] osdmap e270: 8 total, 8 up, 8 in 2024-02-23T00:45:43.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:42.582348+0000 mon.a (mon.0) 1826 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.1a", "id": [7, 5]}]: dispatch 2024-02-23T00:45:43.482 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:45:43.514 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:43.534 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:45:43.537 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:43.537 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:45:43.638 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:45:43.665 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:43.914 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:45:43.917 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:43.919 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:45:44.020 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:45:44.048 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:44.223 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:42.461699+0000 mgr.y (mgr.14517) 282 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:45:44.224 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:42.599978+0000 mon.c (mon.2) 187 : audit [INF] from='client.? 172.21.15.47:0/2181654827' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi047-82386-47"}]: dispatch 2024-02-23T00:45:44.224 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:42.845839+0000 mon.b (mon.1) 330 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:44.224 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:42.846394+0000 mon.b (mon.1) 331 : audit [INF] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pg_num","val":"11"}]: dispatch 2024-02-23T00:45:44.224 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:43.018393+0000 mon.a (mon.0) 1827 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi047-82386-47"}]: dispatch 2024-02-23T00:45:44.224 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:43.018694+0000 mon.a (mon.0) 1828 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pg_num","val":"11"}]: dispatch 2024-02-23T00:45:44.224 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:43.018996+0000 mon.c (mon.2) 188 : audit [INF] from='client.? 172.21.15.47:0/2181654827' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi047-82386-47"}]: dispatch 2024-02-23T00:45:44.224 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:43.023300+0000 mon.a (mon.0) 1829 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi047-82386-47"}]: dispatch 2024-02-23T00:45:44.224 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:43.023797+0000 mon.c (mon.2) 189 : audit [INF] from='client.? 172.21.15.47:0/2181654827' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi047-82386-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:44.224 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:43.024061+0000 mon.a (mon.0) 1830 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi047-82386-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:44.225 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:43.143265+0000 mon.a (mon.0) 1831 : audit [INF] from='client.? 172.21.15.47:0/1534291680' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi047-82378-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi047-82378-37"}]': finished 2024-02-23T00:45:44.225 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:43.143376+0000 mon.a (mon.0) 1832 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.1a", "id": [7, 5]}]': finished 2024-02-23T00:45:44.225 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:43.143456+0000 mon.a (mon.0) 1833 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pg_num","val":"11"}]': finished 2024-02-23T00:45:44.225 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:43.143533+0000 mon.a (mon.0) 1834 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi047-82386-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:44.225 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:43.143612+0000 mon.a (mon.0) 1835 : cluster [DBG] osdmap e271: 8 total, 8 up, 8 in 2024-02-23T00:45:44.225 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:43.151428+0000 mon.b (mon.1) 332 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:44.225 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:43.152441+0000 mon.b (mon.1) 333 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:44.225 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:43.153165+0000 mon.a (mon.0) 1836 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:44.225 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:43.153385+0000 mon.c (mon.2) 190 : audit [INF] from='client.? 172.21.15.47:0/2181654827' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi047-82386-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi047-82386-47"}]: dispatch 2024-02-23T00:45:44.225 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:43.153624+0000 mon.b (mon.1) 334 : audit [INF] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-23T00:45:44.225 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:43.153801+0000 mon.a (mon.0) 1837 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi047-82386-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi047-82386-47"}]: dispatch 2024-02-23T00:45:44.225 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:43.154637+0000 mon.a (mon.0) 1838 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-23T00:45:44.225 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:43.244785+0000 mgr.y (mgr.14517) 283 : cluster [DBG] pgmap v382: 258 pgs: 40 unknown, 218 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:45:44.226 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:43.245766+0000 mon.a (mon.0) 1839 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pgp_num_actual", "val": "11"}]: dispatch 2024-02-23T00:45:44.226 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:42.461699+0000 mgr.y (mgr.14517) 282 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:45:44.226 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:42.599978+0000 mon.c (mon.2) 187 : audit [INF] from='client.? 172.21.15.47:0/2181654827' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi047-82386-47"}]: dispatch 2024-02-23T00:45:44.226 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:42.845839+0000 mon.b (mon.1) 330 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:44.226 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:42.846394+0000 mon.b (mon.1) 331 : audit [INF] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pg_num","val":"11"}]: dispatch 2024-02-23T00:45:44.226 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:43.018393+0000 mon.a (mon.0) 1827 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi047-82386-47"}]: dispatch 2024-02-23T00:45:44.226 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:43.018694+0000 mon.a (mon.0) 1828 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pg_num","val":"11"}]: dispatch 2024-02-23T00:45:44.226 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:43.018996+0000 mon.c (mon.2) 188 : audit [INF] from='client.? 172.21.15.47:0/2181654827' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi047-82386-47"}]: dispatch 2024-02-23T00:45:44.227 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:43.023300+0000 mon.a (mon.0) 1829 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi047-82386-47"}]: dispatch 2024-02-23T00:45:44.227 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:43.023797+0000 mon.c (mon.2) 189 : audit [INF] from='client.? 172.21.15.47:0/2181654827' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi047-82386-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:44.227 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:43.024061+0000 mon.a (mon.0) 1830 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi047-82386-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:44.227 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:43.143265+0000 mon.a (mon.0) 1831 : audit [INF] from='client.? 172.21.15.47:0/1534291680' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi047-82378-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi047-82378-37"}]': finished 2024-02-23T00:45:44.227 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:43.143376+0000 mon.a (mon.0) 1832 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.1a", "id": [7, 5]}]': finished 2024-02-23T00:45:44.227 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:43.143456+0000 mon.a (mon.0) 1833 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pg_num","val":"11"}]': finished 2024-02-23T00:45:44.227 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:43.143533+0000 mon.a (mon.0) 1834 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi047-82386-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:44.227 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:43.143612+0000 mon.a (mon.0) 1835 : cluster [DBG] osdmap e271: 8 total, 8 up, 8 in 2024-02-23T00:45:44.227 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:43.151428+0000 mon.b (mon.1) 332 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:44.227 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:43.152441+0000 mon.b (mon.1) 333 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:44.227 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:43.153165+0000 mon.a (mon.0) 1836 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:44.228 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:43.153385+0000 mon.c (mon.2) 190 : audit [INF] from='client.? 172.21.15.47:0/2181654827' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi047-82386-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi047-82386-47"}]: dispatch 2024-02-23T00:45:44.228 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:43.153624+0000 mon.b (mon.1) 334 : audit [INF] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-23T00:45:44.228 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:43.153801+0000 mon.a (mon.0) 1837 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi047-82386-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi047-82386-47"}]: dispatch 2024-02-23T00:45:44.228 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:43.154637+0000 mon.a (mon.0) 1838 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-23T00:45:44.228 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:43.244785+0000 mgr.y (mgr.14517) 283 : cluster [DBG] pgmap v382: 258 pgs: 40 unknown, 218 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:45:44.228 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:43.245766+0000 mon.a (mon.0) 1839 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pgp_num_actual", "val": "11"}]: dispatch 2024-02-23T00:45:44.250 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:45:44.252 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:44.252 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:45:44.353 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:45:44.382 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:44.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:42.461699+0000 mgr.y (mgr.14517) 282 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:45:44.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:42.599978+0000 mon.c (mon.2) 187 : audit [INF] from='client.? 172.21.15.47:0/2181654827' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi047-82386-47"}]: dispatch 2024-02-23T00:45:44.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:42.845839+0000 mon.b (mon.1) 330 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:44.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:42.846394+0000 mon.b (mon.1) 331 : audit [INF] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pg_num","val":"11"}]: dispatch 2024-02-23T00:45:44.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:43.018393+0000 mon.a (mon.0) 1827 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi047-82386-47"}]: dispatch 2024-02-23T00:45:44.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:43.018694+0000 mon.a (mon.0) 1828 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pg_num","val":"11"}]: dispatch 2024-02-23T00:45:44.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:43.018996+0000 mon.c (mon.2) 188 : audit [INF] from='client.? 172.21.15.47:0/2181654827' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi047-82386-47"}]: dispatch 2024-02-23T00:45:44.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:43.023300+0000 mon.a (mon.0) 1829 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi047-82386-47"}]: dispatch 2024-02-23T00:45:44.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:43.023797+0000 mon.c (mon.2) 189 : audit [INF] from='client.? 172.21.15.47:0/2181654827' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi047-82386-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:44.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:43.024061+0000 mon.a (mon.0) 1830 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi047-82386-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:44.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:43.143265+0000 mon.a (mon.0) 1831 : audit [INF] from='client.? 172.21.15.47:0/1534291680' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi047-82378-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi047-82378-37"}]': finished 2024-02-23T00:45:44.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:43.143376+0000 mon.a (mon.0) 1832 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.1a", "id": [7, 5]}]': finished 2024-02-23T00:45:44.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:43.143456+0000 mon.a (mon.0) 1833 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pg_num","val":"11"}]': finished 2024-02-23T00:45:44.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:43.143533+0000 mon.a (mon.0) 1834 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi047-82386-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:44.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:43.143612+0000 mon.a (mon.0) 1835 : cluster [DBG] osdmap e271: 8 total, 8 up, 8 in 2024-02-23T00:45:44.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:43.151428+0000 mon.b (mon.1) 332 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:44.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:43.152441+0000 mon.b (mon.1) 333 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:44.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:43.153165+0000 mon.a (mon.0) 1836 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:44.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:43.153385+0000 mon.c (mon.2) 190 : audit [INF] from='client.? 172.21.15.47:0/2181654827' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi047-82386-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi047-82386-47"}]: dispatch 2024-02-23T00:45:44.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:43.153624+0000 mon.b (mon.1) 334 : audit [INF] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-23T00:45:44.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:43.153801+0000 mon.a (mon.0) 1837 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi047-82386-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi047-82386-47"}]: dispatch 2024-02-23T00:45:44.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:43.154637+0000 mon.a (mon.0) 1838 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-23T00:45:44.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:43.244785+0000 mgr.y (mgr.14517) 283 : cluster [DBG] pgmap v382: 258 pgs: 40 unknown, 218 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:45:44.446 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:43.245766+0000 mon.a (mon.0) 1839 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pgp_num_actual", "val": "11"}]: dispatch 2024-02-23T00:45:45.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.599+0000 7ff5fe85c700 1 -- 172.21.15.47:0/4181804348 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff5f80fdbb0 msgr2=0x7ff5f80fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:45.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.599+0000 7ff5fe85c700 1 --2- 172.21.15.47:0/4181804348 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff5f80fdbb0 0x7ff5f80fdf90 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 rx=0x7ff5e80047b0 tx=0x7ff5e8009cb0).stop 2024-02-23T00:45:45.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.599+0000 7ff5fe85c700 1 -- 172.21.15.47:0/4181804348 shutdown_connections 2024-02-23T00:45:45.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.599+0000 7ff5fe85c700 1 --2- 172.21.15.47:0/4181804348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5f80f8420 0x7ff5f8104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:45.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.599+0000 7ff5fe85c700 1 --2- 172.21.15.47:0/4181804348 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff5f80fdbb0 0x7ff5f80fdf90 unknown :-1 s=CLOSED pgs=2582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:45.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.599+0000 7ff5fe85c700 1 --2- 172.21.15.47:0/4181804348 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5f80fe510 0x7ff5f80f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:45.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.599+0000 7ff5fe85c700 1 -- 172.21.15.47:0/4181804348 >> 172.21.15.47:0/4181804348 conn(0x7ff5f80f3800 msgr2=0x7ff5f80f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:45:45.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.599+0000 7ff5fe85c700 1 -- 172.21.15.47:0/4181804348 shutdown_connections 2024-02-23T00:45:45.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.599+0000 7ff5fe85c700 1 -- 172.21.15.47:0/4181804348 wait complete. 2024-02-23T00:45:45.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.600+0000 7ff5fe85c700 1 librados: starting msgr at 2024-02-23T00:45:45.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.600+0000 7ff5fe85c700 1 librados: starting objecter 2024-02-23T00:45:45.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.600+0000 7ff5fe85c700 1 Processor -- start 2024-02-23T00:45:45.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.600+0000 7ff5fe85c700 1 -- start start 2024-02-23T00:45:45.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.600+0000 7ff5fe85c700 1 librados: setting wanted keys 2024-02-23T00:45:45.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.600+0000 7ff5fe85c700 1 librados: calling monclient init 2024-02-23T00:45:45.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.600+0000 7ff5fe85c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5f80f8420 0x7ff5f810df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:45.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.600+0000 7ff5fe85c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff5f80fdbb0 0x7ff5f810e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:45.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.600+0000 7ff5fe85c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5f80fe510 0x7ff5f81080b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:45.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.600+0000 7ff5fe85c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f81085a0 con 0x7ff5f80fe510 2024-02-23T00:45:45.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.600+0000 7ff5fe85c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f8108710 con 0x7ff5f80f8420 2024-02-23T00:45:45.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.600+0000 7ff5fe85c700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f81088d0 con 0x7ff5f80fdbb0 2024-02-23T00:45:45.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.601+0000 7ff5fcdf9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5f80fe510 0x7ff5f81080b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:45.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.601+0000 7ff5f77fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff5f80fdbb0 0x7ff5f810e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:45.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.601+0000 7ff5f77fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff5f80fdbb0 0x7ff5f810e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:44564/0 (socket says 172.21.15.47:44564) 2024-02-23T00:45:45.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.601+0000 7ff5f77fe700 1 -- 172.21.15.47:0/940369356 learned_addr learned my addr 172.21.15.47:0/940369356 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:45:45.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.601+0000 7ff5f7fff700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5f80f8420 0x7ff5f810df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:45.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.601+0000 7ff5f77fe700 1 -- 172.21.15.47:0/940369356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5f80f8420 msgr2=0x7ff5f810df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:45.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.601+0000 7ff5f77fe700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5f80f8420 0x7ff5f810df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:45.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.601+0000 7ff5f77fe700 1 -- 172.21.15.47:0/940369356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5f80fe510 msgr2=0x7ff5f81080b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:45.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.601+0000 7ff5f77fe700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5f80fe510 0x7ff5f81080b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:45.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.601+0000 7ff5f77fe700 1 -- 172.21.15.47:0/940369356 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ec005040 con 0x7ff5f80fdbb0 2024-02-23T00:45:45.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.601+0000 7ff5f77fe700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff5f80fdbb0 0x7ff5f810e400 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 rx=0x7ff5ec009400 tx=0x7ff5ec009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:45.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.601+0000 7ff5f57fa700 10 client.?.objecter ms_handle_connect 0x7ff5f80fdbb0 2024-02-23T00:45:45.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.601+0000 7ff5f57fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:45:45.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.601+0000 7ff5f57fa700 1 -- 172.21.15.47:0/940369356 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec007a40 con 0x7ff5f80fdbb0 2024-02-23T00:45:45.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.601+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e8005040 con 0x7ff5f80fdbb0 2024-02-23T00:45:45.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.601+0000 7ff5f57fa700 1 -- 172.21.15.47:0/940369356 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7ff5ec00f040 con 0x7ff5f80fdbb0 2024-02-23T00:45:45.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.601+0000 7ff5f57fa700 1 -- 172.21.15.47:0/940369356 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec014900 con 0x7ff5f80fdbb0 2024-02-23T00:45:45.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.601+0000 7ff5fe85c700 10 client.40958.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:45:45.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.601+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5f8114d40 con 0x7ff5f80fdbb0 2024-02-23T00:45:45.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.601+0000 7ff5fe85c700 1 librados: init done 2024-02-23T00:45:45.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.602+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7ff5f8060280 con 0x7ff5f80fdbb0 2024-02-23T00:45:45.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.603+0000 7ff5f57fa700 1 -- 172.21.15.47:0/940369356 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7ff5ec014d30 con 0x7ff5f80fdbb0 2024-02-23T00:45:45.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.603+0000 7ff5f57fa700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff5e0062cc0 0x7ff5e0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:45.027 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:45:45.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.603+0000 7ff5f57fa700 1 -- 172.21.15.47:0/940369356 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(272..272 src has 48..272) v4 ==== 10726+0+0 (secure 0 0 0) 0x7ff5ec07d800 con 0x7ff5f80fdbb0 2024-02-23T00:45:45.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.603+0000 7ff5f57fa700 10 client.40958.objecter ms_dispatch 0x7ff5f8001550 osd_map(272..272 src has 48..272) v4 2024-02-23T00:45:45.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.603+0000 7ff5f57fa700 3 client.40958.objecter handle_osd_map got epochs [272,272] > 0 2024-02-23T00:45:45.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.603+0000 7ff5f57fa700 3 client.40958.objecter handle_osd_map decoding full epoch 272 2024-02-23T00:45:45.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.603+0000 7ff5f7fff700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff5e0062cc0 0x7ff5e0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:45.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.603+0000 7ff5f57fa700 20 client.40958.objecter dump_active .. 0 homeless 2024-02-23T00:45:45.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.603+0000 7ff5f57fa700 1 -- 172.21.15.47:0/940369356 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v272) v1 ==== 54+0+16 (secure 0 0 0) 0x7ff5ec084070 con 0x7ff5f80fdbb0 2024-02-23T00:45:45.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.603+0000 7ff5f57fa700 10 client.40958.objecter ms_handle_connect 0x7ff5e0062cc0 2024-02-23T00:45:45.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.603+0000 7ff5f7fff700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff5e0062cc0 0x7ff5e0065180 secure :-1 s=READY pgs=2998 cs=0 l=1 rev1=1 rx=0x7ff5e8004e20 tx=0x7ff5e8009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:45.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.603+0000 7ff5fe85c700 10 client.40958.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:45.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.603+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_assign -1 1 2024-02-23T00:45:45.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.603+0000 7ff5fe85c700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7ff5f80fee60 0x7ff5f80ff240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:45.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.603+0000 7ff5fe85c700 20 client.40958.objecter _get_session s=0x7ff5f805ee80 osd=0 3 2024-02-23T00:45:45.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.603+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f805ee80 osd=0 3 2024-02-23T00:45:45.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.603+0000 7ff5fe85c700 20 client.40958.objecter _get_session s=0x7ff5f805ee80 osd=0 3 2024-02-23T00:45:45.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.603+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_remove -1 1 2024-02-23T00:45:45.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.603+0000 7ff5fe85c700 20 client.40958.objecter get_session s=0x7ff5f805ee80 osd=0 3 2024-02-23T00:45:45.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.603+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_assign 0 1 2024-02-23T00:45:45.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.603+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f805ee80 osd=0 4 2024-02-23T00:45:45.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.603+0000 7ff5fe85c700 10 client.40958.objecter _send_command 1 2024-02-23T00:45:45.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.603+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff5f8106e90 con 0x7ff5f80fee60 2024-02-23T00:45:45.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.604+0000 7ff5fcdf9700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7ff5f80fee60 0x7ff5f80ff240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:45.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.604+0000 7ff5f57fa700 10 client.40958.objecter ms_handle_connect 0x7ff5f80fee60 2024-02-23T00:45:45.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.604+0000 7ff5fcdf9700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7ff5f80fee60 0x7ff5f80ff240 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:45.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.604+0000 7ff5f57fa700 1 -- 172.21.15.47:0/940369356 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff5f8106e90 con 0x7ff5f80fee60 2024-02-23T00:45:45.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.604+0000 7ff5f57fa700 10 client.40958.objecter ms_dispatch 0x7ff5f8001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:45:45.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.604+0000 7ff5f57fa700 10 client.40958.objecter _finish_command 1 = system:0 2024-02-23T00:45:45.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.604+0000 7ff5f57fa700 20 client.40958.objecter put_session s=0x7ff5f805ee80 osd=0 4 2024-02-23T00:45:45.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.604+0000 7ff5f57fa700 15 client.40958.objecter _session_command_op_remove 0 1 2024-02-23T00:45:45.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.619+0000 7ff5fe85c700 10 client.40958.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:45:45.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.619+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_assign -1 2 2024-02-23T00:45:45.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.619+0000 7ff5fe85c700 20 client.40958.objecter _get_session s=0x7ff5f805ee80 osd=0 3 2024-02-23T00:45:45.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.619+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f805ee80 osd=0 3 2024-02-23T00:45:45.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.619+0000 7ff5fe85c700 20 client.40958.objecter _get_session s=0x7ff5f805ee80 osd=0 3 2024-02-23T00:45:45.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.619+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_remove -1 2 2024-02-23T00:45:45.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.619+0000 7ff5fe85c700 20 client.40958.objecter get_session s=0x7ff5f805ee80 osd=0 3 2024-02-23T00:45:45.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.619+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_assign 0 2 2024-02-23T00:45:45.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.619+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f805ee80 osd=0 4 2024-02-23T00:45:45.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.619+0000 7ff5fe85c700 10 client.40958.objecter _send_command 2 2024-02-23T00:45:45.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.619+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff5f80273c0 con 0x7ff5f80fee60 2024-02-23T00:45:45.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.621+0000 7ff5f57fa700 1 -- 172.21.15.47:0/940369356 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff5f80273c0 con 0x7ff5f80fee60 2024-02-23T00:45:45.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.621+0000 7ff5f57fa700 10 client.40958.objecter ms_dispatch 0x7ff5f8001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:45:45.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.621+0000 7ff5f57fa700 10 client.40958.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:45:45.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.621+0000 7ff5f57fa700 20 client.40958.objecter put_session s=0x7ff5f805ee80 osd=0 4 2024-02-23T00:45:45.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.621+0000 7ff5f57fa700 15 client.40958.objecter _session_command_op_remove 0 2 2024-02-23T00:45:45.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.621+0000 7ff5fe85c700 10 client.40958.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:45.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.621+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_assign -1 3 2024-02-23T00:45:45.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.621+0000 7ff5fe85c700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7ff5f80598f0 0x7ff5f8059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:45.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.621+0000 7ff5fe85c700 20 client.40958.objecter _get_session s=0x7ff5f8028710 osd=1 3 2024-02-23T00:45:45.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.621+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f8028710 osd=1 3 2024-02-23T00:45:45.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.621+0000 7ff5fe85c700 20 client.40958.objecter _get_session s=0x7ff5f8028710 osd=1 3 2024-02-23T00:45:45.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.621+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_remove -1 3 2024-02-23T00:45:45.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.621+0000 7ff5fe85c700 20 client.40958.objecter get_session s=0x7ff5f8028710 osd=1 3 2024-02-23T00:45:45.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.621+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_assign 1 3 2024-02-23T00:45:45.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.621+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f8028710 osd=1 4 2024-02-23T00:45:45.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.621+0000 7ff5fe85c700 10 client.40958.objecter _send_command 3 2024-02-23T00:45:45.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.621+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff5f811b650 con 0x7ff5f80598f0 2024-02-23T00:45:45.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.621+0000 7ff5f7fff700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7ff5f80598f0 0x7ff5f8059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:45.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.622+0000 7ff5f57fa700 10 client.40958.objecter ms_handle_connect 0x7ff5f80598f0 2024-02-23T00:45:45.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.622+0000 7ff5f7fff700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7ff5f80598f0 0x7ff5f8059cd0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:45.032 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:45:45.032 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:45:45.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.622+0000 7ff5f57fa700 1 -- 172.21.15.47:0/940369356 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff5f811b650 con 0x7ff5f80598f0 2024-02-23T00:45:45.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.622+0000 7ff5f57fa700 10 client.40958.objecter ms_dispatch 0x7ff5f8001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:45:45.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.622+0000 7ff5f57fa700 10 client.40958.objecter _finish_command 3 = system:0 2024-02-23T00:45:45.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.622+0000 7ff5f57fa700 20 client.40958.objecter put_session s=0x7ff5f8028710 osd=1 4 2024-02-23T00:45:45.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.622+0000 7ff5f57fa700 15 client.40958.objecter _session_command_op_remove 1 3 2024-02-23T00:45:45.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.637+0000 7ff5fe85c700 10 client.40958.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:45:45.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.637+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_assign -1 4 2024-02-23T00:45:45.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.637+0000 7ff5fe85c700 20 client.40958.objecter _get_session s=0x7ff5f8028710 osd=1 3 2024-02-23T00:45:45.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.637+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f8028710 osd=1 3 2024-02-23T00:45:45.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.637+0000 7ff5fe85c700 20 client.40958.objecter _get_session s=0x7ff5f8028710 osd=1 3 2024-02-23T00:45:45.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.637+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_remove -1 4 2024-02-23T00:45:45.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.637+0000 7ff5fe85c700 20 client.40958.objecter get_session s=0x7ff5f8028710 osd=1 3 2024-02-23T00:45:45.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.637+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_assign 1 4 2024-02-23T00:45:45.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.637+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f8028710 osd=1 4 2024-02-23T00:45:45.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.637+0000 7ff5fe85c700 10 client.40958.objecter _send_command 4 2024-02-23T00:45:45.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.637+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff5f8059360 con 0x7ff5f80598f0 2024-02-23T00:45:45.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.639+0000 7ff5f57fa700 1 -- 172.21.15.47:0/940369356 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff5f8059360 con 0x7ff5f80598f0 2024-02-23T00:45:45.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.639+0000 7ff5f57fa700 10 client.40958.objecter ms_dispatch 0x7ff5f8001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:45:45.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.639+0000 7ff5f57fa700 10 client.40958.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:45:45.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.639+0000 7ff5f57fa700 20 client.40958.objecter put_session s=0x7ff5f8028710 osd=1 4 2024-02-23T00:45:45.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.639+0000 7ff5f57fa700 15 client.40958.objecter _session_command_op_remove 1 4 2024-02-23T00:45:45.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.639+0000 7ff5fe85c700 10 client.40958.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:45.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.639+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_assign -1 5 2024-02-23T00:45:45.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.639+0000 7ff5fe85c700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7ff5f811a070 0x7ff5f811a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:45.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.639+0000 7ff5fe85c700 20 client.40958.objecter _get_session s=0x7ff5f8027d60 osd=2 3 2024-02-23T00:45:45.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.639+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f8027d60 osd=2 3 2024-02-23T00:45:45.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.639+0000 7ff5fe85c700 20 client.40958.objecter _get_session s=0x7ff5f8027d60 osd=2 3 2024-02-23T00:45:45.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.639+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_remove -1 5 2024-02-23T00:45:45.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.639+0000 7ff5fe85c700 20 client.40958.objecter get_session s=0x7ff5f8027d60 osd=2 3 2024-02-23T00:45:45.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.639+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_assign 2 5 2024-02-23T00:45:45.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.639+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f8027d60 osd=2 4 2024-02-23T00:45:45.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.639+0000 7ff5fe85c700 10 client.40958.objecter _send_command 5 2024-02-23T00:45:45.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.639+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff5f80283e0 con 0x7ff5f811a070 2024-02-23T00:45:45.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.639+0000 7ff5f77fe700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7ff5f811a070 0x7ff5f811a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:45.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.639+0000 7ff5f57fa700 10 client.40958.objecter ms_handle_connect 0x7ff5f811a070 2024-02-23T00:45:45.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.639+0000 7ff5f77fe700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7ff5f811a070 0x7ff5f811a4b0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:45.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.640+0000 7ff5f57fa700 1 -- 172.21.15.47:0/940369356 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff5f80283e0 con 0x7ff5f811a070 2024-02-23T00:45:45.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.640+0000 7ff5f57fa700 10 client.40958.objecter ms_dispatch 0x7ff5f8001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:45:45.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.640+0000 7ff5f57fa700 10 client.40958.objecter _finish_command 5 = system:0 2024-02-23T00:45:45.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.640+0000 7ff5f57fa700 20 client.40958.objecter put_session s=0x7ff5f8027d60 osd=2 4 2024-02-23T00:45:45.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.640+0000 7ff5f57fa700 15 client.40958.objecter _session_command_op_remove 2 5 2024-02-23T00:45:45.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.654+0000 7ff5fe85c700 10 client.40958.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:45:45.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.654+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_assign -1 6 2024-02-23T00:45:45.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.654+0000 7ff5fe85c700 20 client.40958.objecter _get_session s=0x7ff5f8027d60 osd=2 3 2024-02-23T00:45:45.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.654+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f8027d60 osd=2 3 2024-02-23T00:45:45.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.654+0000 7ff5fe85c700 20 client.40958.objecter _get_session s=0x7ff5f8027d60 osd=2 3 2024-02-23T00:45:45.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.654+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_remove -1 6 2024-02-23T00:45:45.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.654+0000 7ff5fe85c700 20 client.40958.objecter get_session s=0x7ff5f8027d60 osd=2 3 2024-02-23T00:45:45.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.654+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_assign 2 6 2024-02-23T00:45:45.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.654+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f8027d60 osd=2 4 2024-02-23T00:45:45.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.654+0000 7ff5fe85c700 10 client.40958.objecter _send_command 6 2024-02-23T00:45:45.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.654+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff5f8028550 con 0x7ff5f811a070 2024-02-23T00:45:45.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.656+0000 7ff5f57fa700 1 -- 172.21.15.47:0/940369356 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff5f8028550 con 0x7ff5f811a070 2024-02-23T00:45:45.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.656+0000 7ff5f57fa700 10 client.40958.objecter ms_dispatch 0x7ff5f8001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:45:45.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.656+0000 7ff5f57fa700 10 client.40958.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:45:45.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.656+0000 7ff5f57fa700 20 client.40958.objecter put_session s=0x7ff5f8027d60 osd=2 4 2024-02-23T00:45:45.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.656+0000 7ff5f57fa700 15 client.40958.objecter _session_command_op_remove 2 6 2024-02-23T00:45:45.039 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:45:45.039 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:45:45.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.656+0000 7ff5fe85c700 10 client.40958.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:45.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.656+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_assign -1 7 2024-02-23T00:45:45.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.656+0000 7ff5fe85c700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7ff5f811b170 0x7ff5f811fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:45.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.656+0000 7ff5fe85c700 20 client.40958.objecter _get_session s=0x7ff5f8027a10 osd=3 3 2024-02-23T00:45:45.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.656+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f8027a10 osd=3 3 2024-02-23T00:45:45.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.656+0000 7ff5fe85c700 20 client.40958.objecter _get_session s=0x7ff5f8027a10 osd=3 3 2024-02-23T00:45:45.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.656+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_remove -1 7 2024-02-23T00:45:45.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.656+0000 7ff5fe85c700 20 client.40958.objecter get_session s=0x7ff5f8027a10 osd=3 3 2024-02-23T00:45:45.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.656+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_assign 3 7 2024-02-23T00:45:45.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.656+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f8027a10 osd=3 4 2024-02-23T00:45:45.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.656+0000 7ff5fe85c700 10 client.40958.objecter _send_command 7 2024-02-23T00:45:45.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.656+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff5f8027b80 con 0x7ff5f811b170 2024-02-23T00:45:45.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.656+0000 7ff5fcdf9700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7ff5f811b170 0x7ff5f811fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:45.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.656+0000 7ff5f57fa700 10 client.40958.objecter ms_handle_connect 0x7ff5f811b170 2024-02-23T00:45:45.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.656+0000 7ff5fcdf9700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7ff5f811b170 0x7ff5f811fcf0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:45.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.657+0000 7ff5f57fa700 1 -- 172.21.15.47:0/940369356 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff5f8027b80 con 0x7ff5f811b170 2024-02-23T00:45:45.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.657+0000 7ff5f57fa700 10 client.40958.objecter ms_dispatch 0x7ff5f8001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:45:45.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.657+0000 7ff5f57fa700 10 client.40958.objecter _finish_command 7 = system:0 2024-02-23T00:45:45.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.657+0000 7ff5f57fa700 20 client.40958.objecter put_session s=0x7ff5f8027a10 osd=3 4 2024-02-23T00:45:45.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.657+0000 7ff5f57fa700 15 client.40958.objecter _session_command_op_remove 3 7 2024-02-23T00:45:45.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.671+0000 7ff5fe85c700 10 client.40958.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:45:45.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.671+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_assign -1 8 2024-02-23T00:45:45.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.671+0000 7ff5fe85c700 20 client.40958.objecter _get_session s=0x7ff5f8027a10 osd=3 3 2024-02-23T00:45:45.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.671+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f8027a10 osd=3 3 2024-02-23T00:45:45.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.671+0000 7ff5fe85c700 20 client.40958.objecter _get_session s=0x7ff5f8027a10 osd=3 3 2024-02-23T00:45:45.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.671+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_remove -1 8 2024-02-23T00:45:45.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.671+0000 7ff5fe85c700 20 client.40958.objecter get_session s=0x7ff5f8027a10 osd=3 3 2024-02-23T00:45:45.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.671+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_assign 3 8 2024-02-23T00:45:45.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.671+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f8027a10 osd=3 4 2024-02-23T00:45:45.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.671+0000 7ff5fe85c700 10 client.40958.objecter _send_command 8 2024-02-23T00:45:45.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.671+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff5f811abe0 con 0x7ff5f811b170 2024-02-23T00:45:45.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.673+0000 7ff5f57fa700 1 -- 172.21.15.47:0/940369356 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff5f811abe0 con 0x7ff5f811b170 2024-02-23T00:45:45.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.673+0000 7ff5f57fa700 10 client.40958.objecter ms_dispatch 0x7ff5f8001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:45:45.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.673+0000 7ff5f57fa700 10 client.40958.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:45:45.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.673+0000 7ff5f57fa700 20 client.40958.objecter put_session s=0x7ff5f8027a10 osd=3 4 2024-02-23T00:45:45.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.673+0000 7ff5f57fa700 15 client.40958.objecter _session_command_op_remove 3 8 2024-02-23T00:45:45.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.673+0000 7ff5fe85c700 10 client.40958.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:45.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.673+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_assign -1 9 2024-02-23T00:45:45.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.673+0000 7ff5fe85c700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7ff5f8119bd0 0x7ff5f8123370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:45.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.673+0000 7ff5fe85c700 20 client.40958.objecter _get_session s=0x7ff5f811a9a0 osd=4 3 2024-02-23T00:45:45.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.673+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f811a9a0 osd=4 3 2024-02-23T00:45:45.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.673+0000 7ff5fe85c700 20 client.40958.objecter _get_session s=0x7ff5f811a9a0 osd=4 3 2024-02-23T00:45:45.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.673+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_remove -1 9 2024-02-23T00:45:45.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.673+0000 7ff5fe85c700 20 client.40958.objecter get_session s=0x7ff5f811a9a0 osd=4 3 2024-02-23T00:45:45.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.673+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_assign 4 9 2024-02-23T00:45:45.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.673+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f811a9a0 osd=4 4 2024-02-23T00:45:45.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.673+0000 7ff5fe85c700 10 client.40958.objecter _send_command 9 2024-02-23T00:45:45.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.673+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff5f811ab10 con 0x7ff5f8119bd0 2024-02-23T00:45:45.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.673+0000 7ff5f7fff700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7ff5f8119bd0 0x7ff5f8123370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:45.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.674+0000 7ff5f57fa700 10 client.40958.objecter ms_handle_connect 0x7ff5f8119bd0 2024-02-23T00:45:45.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.674+0000 7ff5f7fff700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7ff5f8119bd0 0x7ff5f8123370 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:45.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.675+0000 7ff5f57fa700 1 -- 172.21.15.47:0/940369356 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff5f811ab10 con 0x7ff5f8119bd0 2024-02-23T00:45:45.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.675+0000 7ff5f57fa700 10 client.40958.objecter ms_dispatch 0x7ff5f8001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:45:45.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.675+0000 7ff5f57fa700 10 client.40958.objecter _finish_command 9 = system:0 2024-02-23T00:45:45.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.675+0000 7ff5f57fa700 20 client.40958.objecter put_session s=0x7ff5f811a9a0 osd=4 4 2024-02-23T00:45:45.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.675+0000 7ff5f57fa700 15 client.40958.objecter _session_command_op_remove 4 9 2024-02-23T00:45:45.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.689+0000 7ff5fe85c700 10 client.40958.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:45:45.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.689+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_assign -1 10 2024-02-23T00:45:45.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.689+0000 7ff5fe85c700 20 client.40958.objecter _get_session s=0x7ff5f811a9a0 osd=4 3 2024-02-23T00:45:45.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.689+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f811a9a0 osd=4 3 2024-02-23T00:45:45.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.689+0000 7ff5fe85c700 20 client.40958.objecter _get_session s=0x7ff5f811a9a0 osd=4 3 2024-02-23T00:45:45.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.689+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_remove -1 10 2024-02-23T00:45:45.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.689+0000 7ff5fe85c700 20 client.40958.objecter get_session s=0x7ff5f811a9a0 osd=4 3 2024-02-23T00:45:45.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.689+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_assign 4 10 2024-02-23T00:45:45.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.689+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f811a9a0 osd=4 4 2024-02-23T00:45:45.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.689+0000 7ff5fe85c700 10 client.40958.objecter _send_command 10 2024-02-23T00:45:45.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.689+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff5f811ab10 con 0x7ff5f8119bd0 2024-02-23T00:45:45.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.691+0000 7ff5f57fa700 1 -- 172.21.15.47:0/940369356 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff5f811ab10 con 0x7ff5f8119bd0 2024-02-23T00:45:45.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.691+0000 7ff5f57fa700 10 client.40958.objecter ms_dispatch 0x7ff5f8001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:45:45.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.691+0000 7ff5f57fa700 10 client.40958.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:45:45.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.691+0000 7ff5f57fa700 20 client.40958.objecter put_session s=0x7ff5f811a9a0 osd=4 4 2024-02-23T00:45:45.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.691+0000 7ff5f57fa700 15 client.40958.objecter _session_command_op_remove 4 10 2024-02-23T00:45:45.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.692+0000 7ff5fe85c700 10 client.40958.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:45.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.692+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_assign -1 11 2024-02-23T00:45:45.045 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:45:45.045 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:45:45.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.692+0000 7ff5fe85c700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7ff5f8123d70 0x7ff5f8126180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:45.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.692+0000 7ff5fe85c700 20 client.40958.objecter _get_session s=0x7ff5f811ab10 osd=5 3 2024-02-23T00:45:45.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.692+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f811ab10 osd=5 3 2024-02-23T00:45:45.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.692+0000 7ff5fe85c700 20 client.40958.objecter _get_session s=0x7ff5f811ab10 osd=5 3 2024-02-23T00:45:45.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.692+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_remove -1 11 2024-02-23T00:45:45.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.692+0000 7ff5fe85c700 20 client.40958.objecter get_session s=0x7ff5f811ab10 osd=5 3 2024-02-23T00:45:45.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.692+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_assign 5 11 2024-02-23T00:45:45.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.692+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f811ab10 osd=5 4 2024-02-23T00:45:45.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.692+0000 7ff5fe85c700 10 client.40958.objecter _send_command 11 2024-02-23T00:45:45.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.692+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff5f8128f80 con 0x7ff5f8123d70 2024-02-23T00:45:45.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.692+0000 7ff5f77fe700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7ff5f8123d70 0x7ff5f8126180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:45.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.692+0000 7ff5f57fa700 10 client.40958.objecter ms_handle_connect 0x7ff5f8123d70 2024-02-23T00:45:45.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.692+0000 7ff5f77fe700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7ff5f8123d70 0x7ff5f8126180 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:45.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.693+0000 7ff5f57fa700 1 -- 172.21.15.47:0/940369356 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff5f8128f80 con 0x7ff5f8123d70 2024-02-23T00:45:45.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.693+0000 7ff5f57fa700 10 client.40958.objecter ms_dispatch 0x7ff5f8001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:45:45.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.693+0000 7ff5f57fa700 10 client.40958.objecter _finish_command 11 = system:0 2024-02-23T00:45:45.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.693+0000 7ff5f57fa700 20 client.40958.objecter put_session s=0x7ff5f811ab10 osd=5 4 2024-02-23T00:45:45.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.693+0000 7ff5f57fa700 15 client.40958.objecter _session_command_op_remove 5 11 2024-02-23T00:45:45.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.708+0000 7ff5fe85c700 10 client.40958.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:45:45.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.708+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_assign -1 12 2024-02-23T00:45:45.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.708+0000 7ff5fe85c700 20 client.40958.objecter _get_session s=0x7ff5f811ab10 osd=5 3 2024-02-23T00:45:45.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.708+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f811ab10 osd=5 3 2024-02-23T00:45:45.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.708+0000 7ff5fe85c700 20 client.40958.objecter _get_session s=0x7ff5f811ab10 osd=5 3 2024-02-23T00:45:45.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.708+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_remove -1 12 2024-02-23T00:45:45.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.708+0000 7ff5fe85c700 20 client.40958.objecter get_session s=0x7ff5f811ab10 osd=5 3 2024-02-23T00:45:45.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.708+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_assign 5 12 2024-02-23T00:45:45.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.708+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f811ab10 osd=5 4 2024-02-23T00:45:45.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.708+0000 7ff5fe85c700 10 client.40958.objecter _send_command 12 2024-02-23T00:45:45.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.708+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff5f8128f80 con 0x7ff5f8123d70 2024-02-23T00:45:45.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.710+0000 7ff5f57fa700 1 -- 172.21.15.47:0/940369356 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff5f8128f80 con 0x7ff5f8123d70 2024-02-23T00:45:45.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.710+0000 7ff5f57fa700 10 client.40958.objecter ms_dispatch 0x7ff5f8001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:45:45.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.710+0000 7ff5f57fa700 10 client.40958.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:45:45.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.710+0000 7ff5f57fa700 20 client.40958.objecter put_session s=0x7ff5f811ab10 osd=5 4 2024-02-23T00:45:45.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.710+0000 7ff5f57fa700 15 client.40958.objecter _session_command_op_remove 5 12 2024-02-23T00:45:45.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.710+0000 7ff5fe85c700 10 client.40958.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:45.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.710+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_assign -1 13 2024-02-23T00:45:45.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.710+0000 7ff5fe85c700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7ff5f8128330 0x7ff5f8128790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:45.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.710+0000 7ff5fe85c700 20 client.40958.objecter _get_session s=0x7ff5f8128f80 osd=6 3 2024-02-23T00:45:45.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.710+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f8128f80 osd=6 3 2024-02-23T00:45:45.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.710+0000 7ff5fe85c700 20 client.40958.objecter _get_session s=0x7ff5f8128f80 osd=6 3 2024-02-23T00:45:45.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.710+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_remove -1 13 2024-02-23T00:45:45.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.710+0000 7ff5fe85c700 20 client.40958.objecter get_session s=0x7ff5f8128f80 osd=6 3 2024-02-23T00:45:45.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.710+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_assign 6 13 2024-02-23T00:45:45.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.710+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f8128f80 osd=6 4 2024-02-23T00:45:45.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.710+0000 7ff5fe85c700 10 client.40958.objecter _send_command 13 2024-02-23T00:45:45.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.710+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff5f812c310 con 0x7ff5f8128330 2024-02-23T00:45:45.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.710+0000 7ff5fcdf9700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7ff5f8128330 0x7ff5f8128790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:45.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.711+0000 7ff5fcdf9700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7ff5f8128330 0x7ff5f8128790 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:45.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.711+0000 7ff5f57fa700 10 client.40958.objecter ms_handle_connect 0x7ff5f8128330 2024-02-23T00:45:45.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.712+0000 7ff5f57fa700 1 -- 172.21.15.47:0/940369356 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff5f812c310 con 0x7ff5f8128330 2024-02-23T00:45:45.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.712+0000 7ff5f57fa700 10 client.40958.objecter ms_dispatch 0x7ff5f8001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:45:45.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.712+0000 7ff5f57fa700 10 client.40958.objecter _finish_command 13 = system:0 2024-02-23T00:45:45.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.712+0000 7ff5f57fa700 20 client.40958.objecter put_session s=0x7ff5f8128f80 osd=6 4 2024-02-23T00:45:45.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.712+0000 7ff5f57fa700 15 client.40958.objecter _session_command_op_remove 6 13 2024-02-23T00:45:45.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.726+0000 7ff5fe85c700 10 client.40958.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:45:45.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.726+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_assign -1 14 2024-02-23T00:45:45.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.726+0000 7ff5fe85c700 20 client.40958.objecter _get_session s=0x7ff5f8128f80 osd=6 3 2024-02-23T00:45:45.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.726+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f8128f80 osd=6 3 2024-02-23T00:45:45.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.726+0000 7ff5fe85c700 20 client.40958.objecter _get_session s=0x7ff5f8128f80 osd=6 3 2024-02-23T00:45:45.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.726+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_remove -1 14 2024-02-23T00:45:45.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.726+0000 7ff5fe85c700 20 client.40958.objecter get_session s=0x7ff5f8128f80 osd=6 3 2024-02-23T00:45:45.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.726+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_assign 6 14 2024-02-23T00:45:45.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.726+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f8128f80 osd=6 4 2024-02-23T00:45:45.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.726+0000 7ff5fe85c700 10 client.40958.objecter _send_command 14 2024-02-23T00:45:45.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.726+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff5f8126780 con 0x7ff5f8128330 2024-02-23T00:45:45.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.728+0000 7ff5f57fa700 1 -- 172.21.15.47:0/940369356 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff5f8126780 con 0x7ff5f8128330 2024-02-23T00:45:45.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.728+0000 7ff5f57fa700 10 client.40958.objecter ms_dispatch 0x7ff5f8001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:45:45.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.728+0000 7ff5f57fa700 10 client.40958.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:45:45.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.728+0000 7ff5f57fa700 20 client.40958.objecter put_session s=0x7ff5f8128f80 osd=6 4 2024-02-23T00:45:45.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.728+0000 7ff5f57fa700 15 client.40958.objecter _session_command_op_remove 6 14 2024-02-23T00:45:45.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.729+0000 7ff5fe85c700 10 client.40958.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:45.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.729+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_assign -1 15 2024-02-23T00:45:45.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.729+0000 7ff5fe85c700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7ff5f8126e00 0x7ff5f8127240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:45.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.729+0000 7ff5fe85c700 20 client.40958.objecter _get_session s=0x7ff5f8126780 osd=7 3 2024-02-23T00:45:45.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.729+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f8126780 osd=7 3 2024-02-23T00:45:45.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.729+0000 7ff5fe85c700 20 client.40958.objecter _get_session s=0x7ff5f8126780 osd=7 3 2024-02-23T00:45:45.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.729+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_remove -1 15 2024-02-23T00:45:45.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.729+0000 7ff5fe85c700 20 client.40958.objecter get_session s=0x7ff5f8126780 osd=7 3 2024-02-23T00:45:45.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.729+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_assign 7 15 2024-02-23T00:45:45.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.729+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f8126780 osd=7 4 2024-02-23T00:45:45.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.729+0000 7ff5fe85c700 10 client.40958.objecter _send_command 15 2024-02-23T00:45:45.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.729+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff5f8127920 con 0x7ff5f8126e00 2024-02-23T00:45:45.052 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:45:45.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.729+0000 7ff5f7fff700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7ff5f8126e00 0x7ff5f8127240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:45.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.729+0000 7ff5f57fa700 10 client.40958.objecter ms_handle_connect 0x7ff5f8126e00 2024-02-23T00:45:45.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.729+0000 7ff5f7fff700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7ff5f8126e00 0x7ff5f8127240 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:45.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.730+0000 7ff5f57fa700 1 -- 172.21.15.47:0/940369356 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff5f8127920 con 0x7ff5f8126e00 2024-02-23T00:45:45.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.730+0000 7ff5f57fa700 10 client.40958.objecter ms_dispatch 0x7ff5f8001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:45:45.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.730+0000 7ff5f57fa700 10 client.40958.objecter _finish_command 15 = system:0 2024-02-23T00:45:45.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.730+0000 7ff5f57fa700 20 client.40958.objecter put_session s=0x7ff5f8126780 osd=7 4 2024-02-23T00:45:45.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.730+0000 7ff5f57fa700 15 client.40958.objecter _session_command_op_remove 7 15 2024-02-23T00:45:45.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.744+0000 7ff5fe85c700 10 client.40958.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:45:45.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.744+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_assign -1 16 2024-02-23T00:45:45.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.744+0000 7ff5fe85c700 20 client.40958.objecter _get_session s=0x7ff5f8126780 osd=7 3 2024-02-23T00:45:45.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.744+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f8126780 osd=7 3 2024-02-23T00:45:45.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.744+0000 7ff5fe85c700 20 client.40958.objecter _get_session s=0x7ff5f8126780 osd=7 3 2024-02-23T00:45:45.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.744+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_remove -1 16 2024-02-23T00:45:45.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.744+0000 7ff5fe85c700 20 client.40958.objecter get_session s=0x7ff5f8126780 osd=7 3 2024-02-23T00:45:45.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.744+0000 7ff5fe85c700 15 client.40958.objecter _session_command_op_assign 7 16 2024-02-23T00:45:45.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.744+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f8126780 osd=7 4 2024-02-23T00:45:45.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.744+0000 7ff5fe85c700 10 client.40958.objecter _send_command 16 2024-02-23T00:45:45.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.744+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff5f8127920 con 0x7ff5f8126e00 2024-02-23T00:45:45.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5f57fa700 1 -- 172.21.15.47:0/940369356 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff5f8127920 con 0x7ff5f8126e00 2024-02-23T00:45:45.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5f57fa700 10 client.40958.objecter ms_dispatch 0x7ff5f8001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:45:45.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5f57fa700 10 client.40958.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:45:45.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5f57fa700 20 client.40958.objecter put_session s=0x7ff5f8126780 osd=7 4 2024-02-23T00:45:45.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5f57fa700 15 client.40958.objecter _session_command_op_remove 7 16 2024-02-23T00:45:45.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 10 librados: watch_flush enter 2024-02-23T00:45:45.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 10 librados: watch_flush exit 2024-02-23T00:45:45.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 10 client.40958.objecter close_session for osd.0 2024-02-23T00:45:45.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7ff5f80fee60 msgr2=0x7ff5f80ff240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:45.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7ff5f80fee60 0x7ff5f80ff240 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:45.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f805ee80 osd=0 1 2024-02-23T00:45:45.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 10 client.40958.objecter close_session for osd.1 2024-02-23T00:45:45.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7ff5f80598f0 msgr2=0x7ff5f8059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:45.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7ff5f80598f0 0x7ff5f8059cd0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:45.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f8028710 osd=1 1 2024-02-23T00:45:45.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 10 client.40958.objecter close_session for osd.2 2024-02-23T00:45:45.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7ff5f811a070 msgr2=0x7ff5f811a4b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:45.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7ff5f811a070 0x7ff5f811a4b0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:45.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f8027d60 osd=2 1 2024-02-23T00:45:45.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 10 client.40958.objecter close_session for osd.3 2024-02-23T00:45:45.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7ff5f811b170 msgr2=0x7ff5f811fcf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:45.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7ff5f811b170 0x7ff5f811fcf0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:45.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fcdf9700 1 -- 172.21.15.47:0/940369356 reap_dead start 2024-02-23T00:45:45.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f8027a10 osd=3 1 2024-02-23T00:45:45.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 10 client.40958.objecter close_session for osd.4 2024-02-23T00:45:45.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7ff5f8119bd0 msgr2=0x7ff5f8123370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:45.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7ff5f8119bd0 0x7ff5f8123370 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:45.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f811a9a0 osd=4 1 2024-02-23T00:45:45.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 10 client.40958.objecter close_session for osd.5 2024-02-23T00:45:45.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fcdf9700 1 -- 172.21.15.47:0/940369356 reap_dead start 2024-02-23T00:45:45.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7ff5f8123d70 msgr2=0x7ff5f8126180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:45.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7ff5f8123d70 0x7ff5f8126180 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:45.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f811ab10 osd=5 1 2024-02-23T00:45:45.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 10 client.40958.objecter close_session for osd.6 2024-02-23T00:45:45.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7ff5f8128330 msgr2=0x7ff5f8128790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:45.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7ff5f8128330 0x7ff5f8128790 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:45.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f8128f80 osd=6 1 2024-02-23T00:45:45.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 10 client.40958.objecter close_session for osd.7 2024-02-23T00:45:45.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7ff5f8126e00 msgr2=0x7ff5f8127240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:45.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7ff5f8126e00 0x7ff5f8127240 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:45.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 20 client.40958.objecter put_session s=0x7ff5f8126780 osd=7 1 2024-02-23T00:45:45.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 20 client.40958.objecter shutdown clearing up homeless session... 2024-02-23T00:45:45.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 10 client.40958.objecter successfully canceled tick 2024-02-23T00:45:45.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff5e0062cc0 msgr2=0x7ff5e0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:45.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff5e0062cc0 0x7ff5e0065180 secure :-1 s=READY pgs=2998 cs=0 l=1 rev1=1 rx=0x7ff5e8004e20 tx=0x7ff5e8009cb0).stop 2024-02-23T00:45:45.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff5f80fdbb0 msgr2=0x7ff5f810e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:45.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 1 --2- 172.21.15.47:0/940369356 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff5f80fdbb0 0x7ff5f810e400 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 rx=0x7ff5ec009400 tx=0x7ff5ec009cb0).stop 2024-02-23T00:45:45.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fcdf9700 1 -- 172.21.15.47:0/940369356 reap_dead start 2024-02-23T00:45:45.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 shutdown_connections 2024-02-23T00:45:45.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 >> 172.21.15.47:0/940369356 conn(0x7ff5f80f3800 msgr2=0x7ff5f80f5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:45:45.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 shutdown_connections 2024-02-23T00:45:45.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 1 -- 172.21.15.47:0/940369356 wait complete. 2024-02-23T00:45:45.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:44.747+0000 7ff5fe85c700 1 librados: shutdown 2024-02-23T00:45:45.057 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:45:45.057 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:45:45.057 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:45:45.057 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:45:45.057 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:45:45.058 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:45:45.058 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:45:45.058 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:45:45.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:44.148042+0000 mon.a (mon.0) 1840 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:45.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:44.589225+0000 mon.a (mon.0) 1841 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pgp_num","val":"11"}]': finished 2024-02-23T00:45:45.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:44.589294+0000 mon.a (mon.0) 1842 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pgp_num_actual", "val": "11"}]': finished 2024-02-23T00:45:45.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:44.589352+0000 mon.a (mon.0) 1843 : cluster [DBG] osdmap e272: 8 total, 8 up, 8 in 2024-02-23T00:45:45.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:44.590894+0000 mon.b (mon.1) 335 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:45.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:44.591185+0000 mon.a (mon.0) 1844 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:45:45.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:44.604405+0000 mon.c (mon.2) 191 : audit [DBG] from='client.? 172.21.15.47:0/940369356' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:45:45.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:45.136685+0000 mon.a (mon.0) 1845 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi047-82386-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi047-82386-47"}]': finished 2024-02-23T00:45:45.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:45.136934+0000 mon.a (mon.0) 1846 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:45:45.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:45.136998+0000 mon.a (mon.0) 1847 : cluster [DBG] osdmap e273: 8 total, 8 up, 8 in 2024-02-23T00:45:45.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:45.139483+0000 mon.a (mon.0) 1848 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-31"}]: dispatch 2024-02-23T00:45:45.454 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:44.148042+0000 mon.a (mon.0) 1840 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:45.454 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:44.589225+0000 mon.a (mon.0) 1841 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pgp_num","val":"11"}]': finished 2024-02-23T00:45:45.454 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:44.589294+0000 mon.a (mon.0) 1842 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pgp_num_actual", "val": "11"}]': finished 2024-02-23T00:45:45.454 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:44.589352+0000 mon.a (mon.0) 1843 : cluster [DBG] osdmap e272: 8 total, 8 up, 8 in 2024-02-23T00:45:45.454 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:44.590894+0000 mon.b (mon.1) 335 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:45.454 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:44.591185+0000 mon.a (mon.0) 1844 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:45:45.455 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:44.604405+0000 mon.c (mon.2) 191 : audit [DBG] from='client.? 172.21.15.47:0/940369356' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:45:45.455 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:45.136685+0000 mon.a (mon.0) 1845 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi047-82386-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi047-82386-47"}]': finished 2024-02-23T00:45:45.455 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:45.136934+0000 mon.a (mon.0) 1846 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:45:45.455 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:45.136998+0000 mon.a (mon.0) 1847 : cluster [DBG] osdmap e273: 8 total, 8 up, 8 in 2024-02-23T00:45:45.455 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:45.139483+0000 mon.a (mon.0) 1848 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-31"}]: dispatch 2024-02-23T00:45:45.455 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:44.148042+0000 mon.a (mon.0) 1840 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:45.455 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:44.589225+0000 mon.a (mon.0) 1841 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pgp_num","val":"11"}]': finished 2024-02-23T00:45:45.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:44.589294+0000 mon.a (mon.0) 1842 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pgp_num_actual", "val": "11"}]': finished 2024-02-23T00:45:45.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:44.589352+0000 mon.a (mon.0) 1843 : cluster [DBG] osdmap e272: 8 total, 8 up, 8 in 2024-02-23T00:45:45.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:44.590894+0000 mon.b (mon.1) 335 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:45.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:44.591185+0000 mon.a (mon.0) 1844 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:45:45.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:44.604405+0000 mon.c (mon.2) 191 : audit [DBG] from='client.? 172.21.15.47:0/940369356' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:45:45.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:45.136685+0000 mon.a (mon.0) 1845 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi047-82386-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi047-82386-47"}]': finished 2024-02-23T00:45:45.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:45.136934+0000 mon.a (mon.0) 1846 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:45:45.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:45.136998+0000 mon.a (mon.0) 1847 : cluster [DBG] osdmap e273: 8 total, 8 up, 8 in 2024-02-23T00:45:45.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:45.139483+0000 mon.a (mon.0) 1848 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-31"}]: dispatch 2024-02-23T00:45:45.465 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:45:45.469 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:45.469 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:45:45.570 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:45:45.601 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:45.623 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:45:45.625 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:45.626 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:45:45.726 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:45:45.756 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:45.777 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:45:45.779 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:45.780 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:45:45.881 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:45:45.909 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:45.929 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:45:45.931 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:45.932 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:45:46.032 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:45:46.066 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:46.087 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:45:46.089 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:46.090 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:45:46.160 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:45.152613+0000 mon.a (mon.0) 1849 : audit [INF] from='client.? 172.21.15.47:0/1534291680' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi047-82378-37"}]: dispatch 2024-02-23T00:45:46.160 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:45.245220+0000 mgr.y (mgr.14517) 284 : cluster [DBG] pgmap v385: 258 pgs: 40 unknown, 218 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:45:46.160 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:45.591810+0000 mon.b (mon.1) 336 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:46.160 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:46.137893+0000 mon.a (mon.0) 1850 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-31"}]': finished 2024-02-23T00:45:46.160 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:46.138036+0000 mon.a (mon.0) 1851 : audit [INF] from='client.? 172.21.15.47:0/1534291680' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi047-82378-37"}]': finished 2024-02-23T00:45:46.160 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:46.138103+0000 mon.a (mon.0) 1852 : cluster [DBG] osdmap e274: 8 total, 8 up, 8 in 2024-02-23T00:45:46.160 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:46.140827+0000 mon.a (mon.0) 1853 : audit [INF] from='client.? 172.21.15.47:0/1534291680' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi047-82378-37"}]: dispatch 2024-02-23T00:45:46.160 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:46.141141+0000 mon.a (mon.0) 1854 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-31", "mode": "writeback"}]: dispatch 2024-02-23T00:45:46.191 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:45:46.219 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:46.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:45.152613+0000 mon.a (mon.0) 1849 : audit [INF] from='client.? 172.21.15.47:0/1534291680' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi047-82378-37"}]: dispatch 2024-02-23T00:45:46.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:45.245220+0000 mgr.y (mgr.14517) 284 : cluster [DBG] pgmap v385: 258 pgs: 40 unknown, 218 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:45:46.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:45.591810+0000 mon.b (mon.1) 336 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:46.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:46.137893+0000 mon.a (mon.0) 1850 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-31"}]': finished 2024-02-23T00:45:46.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:46.138036+0000 mon.a (mon.0) 1851 : audit [INF] from='client.? 172.21.15.47:0/1534291680' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi047-82378-37"}]': finished 2024-02-23T00:45:46.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:46.138103+0000 mon.a (mon.0) 1852 : cluster [DBG] osdmap e274: 8 total, 8 up, 8 in 2024-02-23T00:45:46.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:46.140827+0000 mon.a (mon.0) 1853 : audit [INF] from='client.? 172.21.15.47:0/1534291680' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi047-82378-37"}]: dispatch 2024-02-23T00:45:46.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:46.141141+0000 mon.a (mon.0) 1854 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-31", "mode": "writeback"}]: dispatch 2024-02-23T00:45:46.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:45.152613+0000 mon.a (mon.0) 1849 : audit [INF] from='client.? 172.21.15.47:0/1534291680' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi047-82378-37"}]: dispatch 2024-02-23T00:45:46.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:45.245220+0000 mgr.y (mgr.14517) 284 : cluster [DBG] pgmap v385: 258 pgs: 40 unknown, 218 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:45:46.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:45.591810+0000 mon.b (mon.1) 336 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:46.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:46.137893+0000 mon.a (mon.0) 1850 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-31"}]': finished 2024-02-23T00:45:46.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:46.138036+0000 mon.a (mon.0) 1851 : audit [INF] from='client.? 172.21.15.47:0/1534291680' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi047-82378-37"}]': finished 2024-02-23T00:45:46.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:46.138103+0000 mon.a (mon.0) 1852 : cluster [DBG] osdmap e274: 8 total, 8 up, 8 in 2024-02-23T00:45:46.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:46.140827+0000 mon.a (mon.0) 1853 : audit [INF] from='client.? 172.21.15.47:0/1534291680' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi047-82378-37"}]: dispatch 2024-02-23T00:45:46.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:46.141141+0000 mon.a (mon.0) 1854 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-31", "mode": "writeback"}]: dispatch 2024-02-23T00:45:46.468 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:45:46.470 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:46.471 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:45:46.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.486+0000 7f76fe622700 1 -- 172.21.15.47:0/568065442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76f80fdbb0 msgr2=0x7f76f80fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:46.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.486+0000 7f76fe622700 1 --2- 172.21.15.47:0/568065442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76f80fdbb0 0x7f76f80fdf90 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 rx=0x7f76e8004870 tx=0x7f76e8009cb0).stop 2024-02-23T00:45:46.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.486+0000 7f76fe622700 1 -- 172.21.15.47:0/568065442 shutdown_connections 2024-02-23T00:45:46.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.486+0000 7f76fe622700 1 --2- 172.21.15.47:0/568065442 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f76f80f8420 0x7f76f8104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:46.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.486+0000 7f76fe622700 1 --2- 172.21.15.47:0/568065442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76f80fdbb0 0x7f76f80fdf90 unknown :-1 s=CLOSED pgs=2490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:46.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.486+0000 7f76fe622700 1 --2- 172.21.15.47:0/568065442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76f80fe510 0x7f76f80f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:46.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.486+0000 7f76fe622700 1 -- 172.21.15.47:0/568065442 >> 172.21.15.47:0/568065442 conn(0x7f76f80f3800 msgr2=0x7f76f80f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:45:46.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.486+0000 7f76fe622700 1 -- 172.21.15.47:0/568065442 shutdown_connections 2024-02-23T00:45:46.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.486+0000 7f76fe622700 1 -- 172.21.15.47:0/568065442 wait complete. 2024-02-23T00:45:46.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.487+0000 7f76fe622700 1 librados: starting msgr at 2024-02-23T00:45:46.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.487+0000 7f76fe622700 1 librados: starting objecter 2024-02-23T00:45:46.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.487+0000 7f76fe622700 1 Processor -- start 2024-02-23T00:45:46.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.487+0000 7f76fe622700 1 -- start start 2024-02-23T00:45:46.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.487+0000 7f76fe622700 1 librados: setting wanted keys 2024-02-23T00:45:46.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.487+0000 7f76fe622700 1 librados: calling monclient init 2024-02-23T00:45:46.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.488+0000 7f76fe622700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76f80f8420 0x7f76f810de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:46.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.488+0000 7f76fe622700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76f80fdbb0 0x7f76f810e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:46.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.488+0000 7f76fe622700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f76f80fe510 0x7f76f8107fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:46.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.488+0000 7f76fe622700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f8108490 con 0x7f76f80fdbb0 2024-02-23T00:45:46.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.488+0000 7f76fe622700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f8108600 con 0x7f76f80f8420 2024-02-23T00:45:46.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.488+0000 7f76fe622700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f81087c0 con 0x7f76f80fe510 2024-02-23T00:45:46.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.488+0000 7f76f77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76f80fdbb0 0x7f76f810e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:46.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.488+0000 7f76fcbbf700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f76f80fe510 0x7f76f8107fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:46.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.488+0000 7f76f77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76f80fdbb0 0x7f76f810e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60202/0 (socket says 172.21.15.47:60202) 2024-02-23T00:45:46.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.488+0000 7f76f77fe700 1 -- 172.21.15.47:0/1710775740 learned_addr learned my addr 172.21.15.47:0/1710775740 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:45:46.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.488+0000 7f76f7fff700 1 --2- 172.21.15.47:0/1710775740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76f80f8420 0x7f76f810de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:46.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.488+0000 7f76fcbbf700 1 -- 172.21.15.47:0/1710775740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76f80f8420 msgr2=0x7f76f810de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:46.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.488+0000 7f76fcbbf700 1 --2- 172.21.15.47:0/1710775740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76f80f8420 0x7f76f810de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:46.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.488+0000 7f76fcbbf700 1 -- 172.21.15.47:0/1710775740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76f80fdbb0 msgr2=0x7f76f810e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:46.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.488+0000 7f76fcbbf700 1 --2- 172.21.15.47:0/1710775740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76f80fdbb0 0x7f76f810e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:46.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.488+0000 7f76fcbbf700 1 -- 172.21.15.47:0/1710775740 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e8005040 con 0x7f76f80fe510 2024-02-23T00:45:46.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.488+0000 7f76f77fe700 1 --2- 172.21.15.47:0/1710775740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76f80fdbb0 0x7f76f810e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:45:46.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.488+0000 7f76fcbbf700 1 --2- 172.21.15.47:0/1710775740 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f76f80fe510 0x7f76f8107fa0 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 rx=0x7f76f000eb70 tx=0x7f76f000b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:46.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.488+0000 7f76f57fa700 10 client.?.objecter ms_handle_connect 0x7f76f80fe510 2024-02-23T00:45:46.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.488+0000 7f76f57fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:45:46.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.488+0000 7f76f57fa700 1 -- 172.21.15.47:0/1710775740 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f0009870 con 0x7f76f80fe510 2024-02-23T00:45:46.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.488+0000 7f76fe622700 1 -- 172.21.15.47:0/1710775740 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f8108a40 con 0x7f76f80fe510 2024-02-23T00:45:46.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.488+0000 7f76f57fa700 1 -- 172.21.15.47:0/1710775740 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f76f000f040 con 0x7f76f80fe510 2024-02-23T00:45:46.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.488+0000 7f76f57fa700 1 -- 172.21.15.47:0/1710775740 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f0007c70 con 0x7f76f80fe510 2024-02-23T00:45:46.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.488+0000 7f76fe622700 10 client.40964.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:45:46.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.488+0000 7f76fe622700 1 -- 172.21.15.47:0/1710775740 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f8114c30 con 0x7f76f80fe510 2024-02-23T00:45:46.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.488+0000 7f76fe622700 1 librados: init done 2024-02-23T00:45:46.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.489+0000 7f76fe622700 1 -- 172.21.15.47:0/1710775740 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76f805ee80 con 0x7f76f80fe510 2024-02-23T00:45:46.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.490+0000 7f76f57fa700 1 -- 172.21.15.47:0/1710775740 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f76f00040b0 con 0x7f76f80fe510 2024-02-23T00:45:46.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.490+0000 7f76f57fa700 1 --2- 172.21.15.47:0/1710775740 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f76e0062cc0 0x7f76e0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:46.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.490+0000 7f76f57fa700 1 -- 172.21.15.47:0/1710775740 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(274..274 src has 48..274) v4 ==== 10454+0+0 (secure 0 0 0) 0x7f76f007d860 con 0x7f76f80fe510 2024-02-23T00:45:46.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.490+0000 7f76f57fa700 10 client.40964.objecter ms_dispatch 0x7f76f8001550 osd_map(274..274 src has 48..274) v4 2024-02-23T00:45:46.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.490+0000 7f76f57fa700 3 client.40964.objecter handle_osd_map got epochs [274,274] > 0 2024-02-23T00:45:46.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.490+0000 7f76f57fa700 3 client.40964.objecter handle_osd_map decoding full epoch 274 2024-02-23T00:45:46.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.490+0000 7f76f7fff700 1 --2- 172.21.15.47:0/1710775740 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f76e0062cc0 0x7f76e0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:46.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.490+0000 7f76f57fa700 20 client.40964.objecter dump_active .. 0 homeless 2024-02-23T00:45:46.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.491+0000 7f76f57fa700 10 client.40964.objecter ms_handle_connect 0x7f76e0062cc0 2024-02-23T00:45:46.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.491+0000 7f76f7fff700 1 --2- 172.21.15.47:0/1710775740 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f76e0062cc0 0x7f76e0065180 secure :-1 s=READY pgs=2999 cs=0 l=1 rev1=1 rx=0x7f76e8004ee0 tx=0x7f76e800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:46.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.495+0000 7f76f57fa700 1 -- 172.21.15.47:0/1710775740 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f76f0049f60 con 0x7f76f80fe510 2024-02-23T00:45:46.568 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:45:46.572 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:45:46.601 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:46.607 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:45:46.620 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:45:46.623 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:46.623 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:45:46.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.674+0000 7f76fe622700 1 -- 172.21.15.47:0/1710775740 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f76f805fa60 con 0x7f76e0062cc0 2024-02-23T00:45:46.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.691+0000 7f76f57fa700 1 -- 172.21.15.47:0/1710775740 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+646776 (secure 0 0 0) 0x7f76f805fa60 con 0x7f76e0062cc0 2024-02-23T00:45:46.702 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:45:46.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.695+0000 7f76fe622700 10 librados: watch_flush enter 2024-02-23T00:45:46.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.695+0000 7f76fe622700 10 librados: watch_flush exit 2024-02-23T00:45:46.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.695+0000 7f76fe622700 20 client.40964.objecter shutdown clearing up homeless session... 2024-02-23T00:45:46.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.695+0000 7f76fe622700 10 client.40964.objecter successfully canceled tick 2024-02-23T00:45:46.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.695+0000 7f76fe622700 1 -- 172.21.15.47:0/1710775740 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f76e0062cc0 msgr2=0x7f76e0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:46.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.695+0000 7f76fe622700 1 --2- 172.21.15.47:0/1710775740 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f76e0062cc0 0x7f76e0065180 secure :-1 s=READY pgs=2999 cs=0 l=1 rev1=1 rx=0x7f76e8004ee0 tx=0x7f76e800b040).stop 2024-02-23T00:45:46.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.695+0000 7f76fe622700 1 -- 172.21.15.47:0/1710775740 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f76f80fe510 msgr2=0x7f76f8107fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:46.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.695+0000 7f76fe622700 1 --2- 172.21.15.47:0/1710775740 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f76f80fe510 0x7f76f8107fa0 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 rx=0x7f76f000eb70 tx=0x7f76f000b040).stop 2024-02-23T00:45:46.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.696+0000 7f76fe622700 1 -- 172.21.15.47:0/1710775740 shutdown_connections 2024-02-23T00:45:46.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.696+0000 7f76fe622700 1 --2- 172.21.15.47:0/1710775740 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f76e0062cc0 0x7f76e0065180 unknown :-1 s=CLOSED pgs=2999 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:46.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.696+0000 7f76fe622700 1 --2- 172.21.15.47:0/1710775740 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f76f80fe510 0x7f76f8107fa0 unknown :-1 s=CLOSED pgs=2584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:46.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.696+0000 7f76fe622700 1 --2- 172.21.15.47:0/1710775740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76f80f8420 0x7f76f810de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:46.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.696+0000 7f76fe622700 1 --2- 172.21.15.47:0/1710775740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76f80fdbb0 0x7f76f810e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:46.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.696+0000 7f76fe622700 1 -- 172.21.15.47:0/1710775740 >> 172.21.15.47:0/1710775740 conn(0x7f76f80f3800 msgr2=0x7f76f80f5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:45:46.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.696+0000 7f76fe622700 1 -- 172.21.15.47:0/1710775740 shutdown_connections 2024-02-23T00:45:46.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.696+0000 7f76fe622700 1 -- 172.21.15.47:0/1710775740 wait complete. 2024-02-23T00:45:46.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:46.696+0000 7f76fe622700 1 librados: shutdown 2024-02-23T00:45:46.709 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-02-23T00:45:46.724 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:45:46.771 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:46.846 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:45:46.849 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:46.849 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:45:46.950 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:45:46.980 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:47.102 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:45:47.105 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:47.105 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:45:47.206 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:45:47.237 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:47.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:46.592778+0000 mon.b (mon.1) 337 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:47.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:47.135673+0000 mon.a (mon.0) 1855 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:45:47.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:47.139935+0000 mon.a (mon.0) 1856 : audit [INF] from='client.? 172.21.15.47:0/1534291680' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi047-82378-37"}]': finished 2024-02-23T00:45:47.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:47.140014+0000 mon.a (mon.0) 1857 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-31", "mode": "writeback"}]': finished 2024-02-23T00:45:47.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:47.140083+0000 mon.a (mon.0) 1858 : cluster [DBG] osdmap e275: 8 total, 8 up, 8 in 2024-02-23T00:45:47.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:47.143797+0000 mon.c (mon.2) 192 : audit [INF] from='client.? 172.21.15.47:0/2181654827' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi047-82386-47"}]: dispatch 2024-02-23T00:45:47.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:47.144117+0000 mon.a (mon.0) 1859 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi047-82386-47"}]: dispatch 2024-02-23T00:45:47.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:46.592778+0000 mon.b (mon.1) 337 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:47.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:47.135673+0000 mon.a (mon.0) 1855 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:45:47.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:47.139935+0000 mon.a (mon.0) 1856 : audit [INF] from='client.? 172.21.15.47:0/1534291680' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi047-82378-37"}]': finished 2024-02-23T00:45:47.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:47.140014+0000 mon.a (mon.0) 1857 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-31", "mode": "writeback"}]': finished 2024-02-23T00:45:47.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:47.140083+0000 mon.a (mon.0) 1858 : cluster [DBG] osdmap e275: 8 total, 8 up, 8 in 2024-02-23T00:45:47.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:47.143797+0000 mon.c (mon.2) 192 : audit [INF] from='client.? 172.21.15.47:0/2181654827' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi047-82386-47"}]: dispatch 2024-02-23T00:45:47.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:47.144117+0000 mon.a (mon.0) 1859 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi047-82386-47"}]: dispatch 2024-02-23T00:45:47.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:46.592778+0000 mon.b (mon.1) 337 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:47.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:47.135673+0000 mon.a (mon.0) 1855 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:45:47.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:47.139935+0000 mon.a (mon.0) 1856 : audit [INF] from='client.? 172.21.15.47:0/1534291680' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi047-82378-37"}]': finished 2024-02-23T00:45:47.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:47.140014+0000 mon.a (mon.0) 1857 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-31", "mode": "writeback"}]': finished 2024-02-23T00:45:47.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:47.140083+0000 mon.a (mon.0) 1858 : cluster [DBG] osdmap e275: 8 total, 8 up, 8 in 2024-02-23T00:45:47.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:47.143797+0000 mon.c (mon.2) 192 : audit [INF] from='client.? 172.21.15.47:0/2181654827' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi047-82386-47"}]: dispatch 2024-02-23T00:45:47.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:47.144117+0000 mon.a (mon.0) 1859 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi047-82386-47"}]: dispatch 2024-02-23T00:45:47.788 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:45:47.791 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:47.791 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:45:47.809 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-02-23T00:45:46.135212+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":9051,"num_objects":360,"num_object_clones":1,"num_object_copies":1080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":360,"num_whiteouts":0,"num_read":1581,"num_read_kb":1208,"num_write":1264,"num_write_kb":639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":947,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2617,"ondisk_log_size":2617,"up":654,"acting":654,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":766,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2482896,"kb_used_data":35400,"kb_used_omap":0,"kb_used_meta":2447360,"kb_avail":747478320,"statfs":{"total":767960285184,"available":765417799680,"internally_reserved":0,"allocated":36249600,"data_stored":21459119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2506096640},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.996502"},"pg_stats":[{"pgid":"172.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:43.138389+0000","last_change":"2024-02-23T00:45:43.138389+0000","last_active":"2024-02-23T00:45:43.138389+0000","last_peered":"2024-02-23T00:45:43.138389+0000","last_clean":"2024-02-23T00:45:43.138389+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:43.138389+0000","last_undegraded":"2024-02-23T00:45:43.138389+0000","last_fullsized":"2024-02-23T00:45:43.138389+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_clean_scrub_stamp":"2024-02-23T00:45:43.138389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"172.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:43.138389+0000","last_change":"2024-02-23T00:45:43.138389+0000","last_active":"2024-02-23T00:45:43.138389+0000","last_peered":"2024-02-23T00:45:43.138389+0000","last_clean":"2024-02-23T00:45:43.138389+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:43.138389+0000","last_undegraded":"2024-02-23T00:45:43.138389+0000","last_fullsized":"2024-02-23T00:45:43.138389+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_clean_scrub_stamp":"2024-02-23T00:45:43.138389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"172.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:43.138389+0000","last_change":"2024-02-23T00:45:43.138389+0000","last_active":"2024-02-23T00:45:43.138389+0000","last_peered":"2024-02-23T00:45:43.138389+0000","last_clean":"2024-02-23T00:45:43.138389+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:43.138389+0000","last_undegraded":"2024-02-23T00:45:43.138389+0000","last_fullsized":"2024-02-23T00:45:43.138389+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_clean_scrub_stamp":"2024-02-23T00:45:43.138389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"172.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:43.138389+0000","last_change":"2024-02-23T00:45:43.138389+0000","last_active":"2024-02-23T00:45:43.138389+0000","last_peered":"2024-02-23T00:45:43.138389+0000","last_clean":"2024-02-23T00:45:43.138389+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:43.138389+0000","last_undegraded":"2024-02-23T00:45:43.138389+0000","last_fullsized":"2024-02-23T00:45:43.138389+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_clean_scrub_stamp":"2024-02-23T00:45:43.138389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"172.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:43.138389+0000","last_change":"2024-02-23T00:45:43.138389+0000","last_active":"2024-02-23T00:45:43.138389+0000","last_peered":"2024-02-23T00:45:43.138389+0000","last_clean":"2024-02-23T00:45:43.138389+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:43.138389+0000","last_undegraded":"2024-02-23T00:45:43.138389+0000","last_fullsized":"2024-02-23T00:45:43.138389+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_clean_scrub_stamp":"2024-02-23T00:45:43.138389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"172.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:43.138389+0000","last_change":"2024-02-23T00:45:43.138389+0000","last_active":"2024-02-23T00:45:43.138389+0000","last_peered":"2024-02-23T00:45:43.138389+0000","last_clean":"2024-02-23T00:45:43.138389+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:43.138389+0000","last_undegraded":"2024-02-23T00:45:43.138389+0000","last_fullsized":"2024-02-23T00:45:43.138389+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_clean_scrub_stamp":"2024-02-23T00:45:43.138389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"172.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:43.138389+0000","last_change":"2024-02-23T00:45:43.138389+0000","last_active":"2024-02-23T00:45:43.138389+0000","last_peered":"2024-02-23T00:45:43.138389+0000","last_clean":"2024-02-23T00:45:43.138389+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:43.138389+0000","last_undegraded":"2024-02-23T00:45:43.138389+0000","last_fullsized":"2024-02-23T00:45:43.138389+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_clean_scrub_stamp":"2024-02-23T00:45:43.138389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"172.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:43.138389+0000","last_change":"2024-02-23T00:45:43.138389+0000","last_active":"2024-02-23T00:45:43.138389+0000","last_peered":"2024-02-23T00:45:43.138389+0000","last_clean":"2024-02-23T00:45:43.138389+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:43.138389+0000","last_undegraded":"2024-02-23T00:45:43.138389+0000","last_fullsized":"2024-02-23T00:45:43.138389+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_clean_scrub_stamp":"2024-02-23T00:45:43.138389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"172.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:43.138389+0000","last_change":"2024-02-23T00:45:43.138389+0000","last_active":"2024-02-23T00:45:43.138389+0000","last_peered":"2024-02-23T00:45:43.138389+0000","last_clean":"2024-02-23T00:45:43.138389+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:43.138389+0000","last_undegraded":"2024-02-23T00:45:43.138389+0000","last_fullsized":"2024-02-23T00:45:43.138389+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_clean_scrub_stamp":"2024-02-23T00:45:43.138389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"172.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:43.138389+0000","last_change":"2024-02-23T00:45:43.138389+0000","last_active":"2024-02-23T00:45:43.138389+0000","last_peered":"2024-02-23T00:45:43.138389+0000","last_clean":"2024-02-23T00:45:43.138389+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:43.138389+0000","last_undegraded":"2024-02-23T00:45:43.138389+0000","last_fullsized":"2024-02-23T00:45:43.138389+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_clean_scrub_stamp":"2024-02-23T00:45:43.138389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"172.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:43.138389+0000","last_change":"2024-02-23T00:45:43.138389+0000","last_active":"2024-02-23T00:45:43.138389+0000","last_peered":"2024-02-23T00:45:43.138389+0000","last_clean":"2024-02-23T00:45:43.138389+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:43.138389+0000","last_undegraded":"2024-02-23T00:45:43.138389+0000","last_fullsized":"2024-02-23T00:45:43.138389+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_clean_scrub_stamp":"2024-02-23T00:45:43.138389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"172.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:43.138389+0000","last_change":"2024-02-23T00:45:43.138389+0000","last_active":"2024-02-23T00:45:43.138389+0000","last_peered":"2024-02-23T00:45:43.138389+0000","last_clean":"2024-02-23T00:45:43.138389+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:43.138389+0000","last_undegraded":"2024-02-23T00:45:43.138389+0000","last_fullsized":"2024-02-23T00:45:43.138389+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_clean_scrub_stamp":"2024-02-23T00:45:43.138389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"172.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:43.138389+0000","last_change":"2024-02-23T00:45:43.138389+0000","last_active":"2024-02-23T00:45:43.138389+0000","last_peered":"2024-02-23T00:45:43.138389+0000","last_clean":"2024-02-23T00:45:43.138389+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:43.138389+0000","last_undegraded":"2024-02-23T00:45:43.138389+0000","last_fullsized":"2024-02-23T00:45:43.138389+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_clean_scrub_stamp":"2024-02-23T00:45:43.138389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"172.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:43.138389+0000","last_change":"2024-02-23T00:45:43.138389+0000","last_active":"2024-02-23T00:45:43.138389+0000","last_peered":"2024-02-23T00:45:43.138389+0000","last_clean":"2024-02-23T00:45:43.138389+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:43.138389+0000","last_undegraded":"2024-02-23T00:45:43.138389+0000","last_fullsized":"2024-02-23T00:45:43.138389+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_clean_scrub_stamp":"2024-02-23T00:45:43.138389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"172.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:43.138389+0000","last_change":"2024-02-23T00:45:43.138389+0000","last_active":"2024-02-23T00:45:43.138389+0000","last_peered":"2024-02-23T00:45:43.138389+0000","last_clean":"2024-02-23T00:45:43.138389+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:43.138389+0000","last_undegraded":"2024-02-23T00:45:43.138389+0000","last_fullsized":"2024-02-23T00:45:43.138389+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_clean_scrub_stamp":"2024-02-23T00:45:43.138389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"172.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:43.138389+0000","last_change":"2024-02-23T00:45:43.138389+0000","last_active":"2024-02-23T00:45:43.138389+0000","last_peered":"2024-02-23T00:45:43.138389+0000","last_clean":"2024-02-23T00:45:43.138389+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:43.138389+0000","last_undegraded":"2024-02-23T00:45:43.138389+0000","last_fullsized":"2024-02-23T00:45:43.138389+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_clean_scrub_stamp":"2024-02-23T00:45:43.138389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"172.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:43.138389+0000","last_change":"2024-02-23T00:45:43.138389+0000","last_active":"2024-02-23T00:45:43.138389+0000","last_peered":"2024-02-23T00:45:43.138389+0000","last_clean":"2024-02-23T00:45:43.138389+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:43.138389+0000","last_undegraded":"2024-02-23T00:45:43.138389+0000","last_fullsized":"2024-02-23T00:45:43.138389+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_clean_scrub_stamp":"2024-02-23T00:45:43.138389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"172.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:43.138389+0000","last_change":"2024-02-23T00:45:43.138389+0000","last_active":"2024-02-23T00:45:43.138389+0000","last_peered":"2024-02-23T00:45:43.138389+0000","last_clean":"2024-02-23T00:45:43.138389+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:43.138389+0000","last_undegraded":"2024-02-23T00:45:43.138389+0000","last_fullsized":"2024-02-23T00:45:43.138389+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_clean_scrub_stamp":"2024-02-23T00:45:43.138389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"172.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:43.138389+0000","last_change":"2024-02-23T00:45:43.138389+0000","last_active":"2024-02-23T00:45:43.138389+0000","last_peered":"2024-02-23T00:45:43.138389+0000","last_clean":"2024-02-23T00:45:43.138389+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:43.138389+0000","last_undegraded":"2024-02-23T00:45:43.138389+0000","last_fullsized":"2024-02-23T00:45:43.138389+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_clean_scrub_stamp":"2024-02-23T00:45:43.138389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"172.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:43.138389+0000","last_change":"2024-02-23T00:45:43.138389+0000","last_active":"2024-02-23T00:45:43.138389+0000","last_peered":"2024-02-23T00:45:43.138389+0000","last_clean":"2024-02-23T00:45:43.138389+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:43.138389+0000","last_undegraded":"2024-02-23T00:45:43.138389+0000","last_fullsized":"2024-02-23T00:45:43.138389+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_clean_scrub_stamp":"2024-02-23T00:45:43.138389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"172.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:43.138389+0000","last_change":"2024-02-23T00:45:43.138389+0000","last_active":"2024-02-23T00:45:43.138389+0000","last_peered":"2024-02-23T00:45:43.138389+0000","last_clean":"2024-02-23T00:45:43.138389+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:43.138389+0000","last_undegraded":"2024-02-23T00:45:43.138389+0000","last_fullsized":"2024-02-23T00:45:43.138389+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_clean_scrub_stamp":"2024-02-23T00:45:43.138389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"172.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:43.138389+0000","last_change":"2024-02-23T00:45:43.138389+0000","last_active":"2024-02-23T00:45:43.138389+0000","last_peered":"2024-02-23T00:45:43.138389+0000","last_clean":"2024-02-23T00:45:43.138389+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:43.138389+0000","last_undegraded":"2024-02-23T00:45:43.138389+0000","last_fullsized":"2024-02-23T00:45:43.138389+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_clean_scrub_stamp":"2024-02-23T00:45:43.138389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"172.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:43.138389+0000","last_change":"2024-02-23T00:45:43.138389+0000","last_active":"2024-02-23T00:45:43.138389+0000","last_peered":"2024-02-23T00:45:43.138389+0000","last_clean":"2024-02-23T00:45:43.138389+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:43.138389+0000","last_undegraded":"2024-02-23T00:45:43.138389+0000","last_fullsized":"2024-02-23T00:45:43.138389+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_clean_scrub_stamp":"2024-02-23T00:45:43.138389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"172.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:43.138389+0000","last_change":"2024-02-23T00:45:43.138389+0000","last_active":"2024-02-23T00:45:43.138389+0000","last_peered":"2024-02-23T00:45:43.138389+0000","last_clean":"2024-02-23T00:45:43.138389+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:43.138389+0000","last_undegraded":"2024-02-23T00:45:43.138389+0000","last_fullsized":"2024-02-23T00:45:43.138389+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_clean_scrub_stamp":"2024-02-23T00:45:43.138389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"173.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:45.131586+0000","last_change":"2024-02-23T00:45:45.131586+0000","last_active":"2024-02-23T00:45:45.131586+0000","last_peered":"2024-02-23T00:45:45.131586+0000","last_clean":"2024-02-23T00:45:45.131586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:45.131586+0000","last_undegraded":"2024-02-23T00:45:45.131586+0000","last_fullsized":"2024-02-23T00:45:45.131586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:45.131586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:45.131586+0000","last_clean_scrub_stamp":"2024-02-23T00:45:45.131586+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"172.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:43.138389+0000","last_change":"2024-02-23T00:45:43.138389+0000","last_active":"2024-02-23T00:45:43.138389+0000","last_peered":"2024-02-23T00:45:43.138389+0000","last_clean":"2024-02-23T00:45:43.138389+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:43.138389+0000","last_undegraded":"2024-02-23T00:45:43.138389+0000","last_fullsized":"2024-02-23T00:45:43.138389+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_clean_scrub_stamp":"2024-02-23T00:45:43.138389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"173.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:45.131586+0000","last_change":"2024-02-23T00:45:45.131586+0000","last_active":"2024-02-23T00:45:45.131586+0000","last_peered":"2024-02-23T00:45:45.131586+0000","last_clean":"2024-02-23T00:45:45.131586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:45.131586+0000","last_undegraded":"2024-02-23T00:45:45.131586+0000","last_fullsized":"2024-02-23T00:45:45.131586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:45.131586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:45.131586+0000","last_clean_scrub_stamp":"2024-02-23T00:45:45.131586+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"172.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:43.138389+0000","last_change":"2024-02-23T00:45:43.138389+0000","last_active":"2024-02-23T00:45:43.138389+0000","last_peered":"2024-02-23T00:45:43.138389+0000","last_clean":"2024-02-23T00:45:43.138389+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:43.138389+0000","last_undegraded":"2024-02-23T00:45:43.138389+0000","last_fullsized":"2024-02-23T00:45:43.138389+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_clean_scrub_stamp":"2024-02-23T00:45:43.138389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"173.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:45.131586+0000","last_change":"2024-02-23T00:45:45.131586+0000","last_active":"2024-02-23T00:45:45.131586+0000","last_peered":"2024-02-23T00:45:45.131586+0000","last_clean":"2024-02-23T00:45:45.131586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:45.131586+0000","last_undegraded":"2024-02-23T00:45:45.131586+0000","last_fullsized":"2024-02-23T00:45:45.131586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:45.131586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:45.131586+0000","last_clean_scrub_stamp":"2024-02-23T00:45:45.131586+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"172.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:43.138389+0000","last_change":"2024-02-23T00:45:43.138389+0000","last_active":"2024-02-23T00:45:43.138389+0000","last_peered":"2024-02-23T00:45:43.138389+0000","last_clean":"2024-02-23T00:45:43.138389+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:43.138389+0000","last_undegraded":"2024-02-23T00:45:43.138389+0000","last_fullsized":"2024-02-23T00:45:43.138389+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_clean_scrub_stamp":"2024-02-23T00:45:43.138389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"173.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:45.131586+0000","last_change":"2024-02-23T00:45:45.131586+0000","last_active":"2024-02-23T00:45:45.131586+0000","last_peered":"2024-02-23T00:45:45.131586+0000","last_clean":"2024-02-23T00:45:45.131586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:45.131586+0000","last_undegraded":"2024-02-23T00:45:45.131586+0000","last_fullsized":"2024-02-23T00:45:45.131586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:45.131586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:45.131586+0000","last_clean_scrub_stamp":"2024-02-23T00:45:45.131586+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"172.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:43.138389+0000","last_change":"2024-02-23T00:45:43.138389+0000","last_active":"2024-02-23T00:45:43.138389+0000","last_peered":"2024-02-23T00:45:43.138389+0000","last_clean":"2024-02-23T00:45:43.138389+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:43.138389+0000","last_undegraded":"2024-02-23T00:45:43.138389+0000","last_fullsized":"2024-02-23T00:45:43.138389+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_clean_scrub_stamp":"2024-02-23T00:45:43.138389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"173.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:45.131586+0000","last_change":"2024-02-23T00:45:45.131586+0000","last_active":"2024-02-23T00:45:45.131586+0000","last_peered":"2024-02-23T00:45:45.131586+0000","last_clean":"2024-02-23T00:45:45.131586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:45.131586+0000","last_undegraded":"2024-02-23T00:45:45.131586+0000","last_fullsized":"2024-02-23T00:45:45.131586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:45.131586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:45.131586+0000","last_clean_scrub_stamp":"2024-02-23T00:45:45.131586+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"172.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:43.138389+0000","last_change":"2024-02-23T00:45:43.138389+0000","last_active":"2024-02-23T00:45:43.138389+0000","last_peered":"2024-02-23T00:45:43.138389+0000","last_clean":"2024-02-23T00:45:43.138389+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:43.138389+0000","last_undegraded":"2024-02-23T00:45:43.138389+0000","last_fullsized":"2024-02-23T00:45:43.138389+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_clean_scrub_stamp":"2024-02-23T00:45:43.138389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"173.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:45.131586+0000","last_change":"2024-02-23T00:45:45.131586+0000","last_active":"2024-02-23T00:45:45.131586+0000","last_peered":"2024-02-23T00:45:45.131586+0000","last_clean":"2024-02-23T00:45:45.131586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:45.131586+0000","last_undegraded":"2024-02-23T00:45:45.131586+0000","last_fullsized":"2024-02-23T00:45:45.131586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:45.131586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:45.131586+0000","last_clean_scrub_stamp":"2024-02-23T00:45:45.131586+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"172.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:43.138389+0000","last_change":"2024-02-23T00:45:43.138389+0000","last_active":"2024-02-23T00:45:43.138389+0000","last_peered":"2024-02-23T00:45:43.138389+0000","last_clean":"2024-02-23T00:45:43.138389+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:43.138389+0000","last_undegraded":"2024-02-23T00:45:43.138389+0000","last_fullsized":"2024-02-23T00:45:43.138389+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_clean_scrub_stamp":"2024-02-23T00:45:43.138389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"173.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:45.131586+0000","last_change":"2024-02-23T00:45:45.131586+0000","last_active":"2024-02-23T00:45:45.131586+0000","last_peered":"2024-02-23T00:45:45.131586+0000","last_clean":"2024-02-23T00:45:45.131586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:45.131586+0000","last_undegraded":"2024-02-23T00:45:45.131586+0000","last_fullsized":"2024-02-23T00:45:45.131586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:45.131586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:45.131586+0000","last_clean_scrub_stamp":"2024-02-23T00:45:45.131586+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"172.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:43.138389+0000","last_change":"2024-02-23T00:45:43.138389+0000","last_active":"2024-02-23T00:45:43.138389+0000","last_peered":"2024-02-23T00:45:43.138389+0000","last_clean":"2024-02-23T00:45:43.138389+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:43.138389+0000","last_undegraded":"2024-02-23T00:45:43.138389+0000","last_fullsized":"2024-02-23T00:45:43.138389+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_clean_scrub_stamp":"2024-02-23T00:45:43.138389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"173.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:45.131586+0000","last_change":"2024-02-23T00:45:45.131586+0000","last_active":"2024-02-23T00:45:45.131586+0000","last_peered":"2024-02-23T00:45:45.131586+0000","last_clean":"2024-02-23T00:45:45.131586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:45.131586+0000","last_undegraded":"2024-02-23T00:45:45.131586+0000","last_fullsized":"2024-02-23T00:45:45.131586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:45.131586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:45.131586+0000","last_clean_scrub_stamp":"2024-02-23T00:45:45.131586+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"172.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:45:43.138389+0000","last_change":"2024-02-23T00:45:43.138389+0000","last_active":"2024-02-23T00:45:43.138389+0000","last_peered":"2024-02-23T00:45:43.138389+0000","last_clean":"2024-02-23T00:45:43.138389+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:45:43.138389+0000","last_undegraded":"2024-02-23T00:45:43.138389+0000","last_fullsized":"2024-02-23T00:45:43.138389+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:43.138389+0000","last_clean_scrub_stamp":"2024-02-23T00:45:43.138389+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.0","version":"259'40","reported_seq":228,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.653764+0000","last_change":"2024-02-23T00:45:33.145590+0000","last_active":"2024-02-23T00:45:40.653764+0000","last_peered":"2024-02-23T00:45:40.653764+0000","last_clean":"2024-02-23T00:45:40.653764+0000","last_became_active":"2024-02-23T00:43:17.855316+0000","last_became_peered":"2024-02-23T00:43:17.855316+0000","last_unstale":"2024-02-23T00:45:40.653764+0000","last_undegraded":"2024-02-23T00:45:40.653764+0000","last_fullsized":"2024-02-23T00:45:40.653764+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":31,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":1,"num_write":13,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.1","version":"230'87","reported_seq":268,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.151524+0000","last_change":"2024-02-23T00:45:31.135186+0000","last_active":"2024-02-23T00:45:41.151524+0000","last_peered":"2024-02-23T00:45:41.151524+0000","last_clean":"2024-02-23T00:45:41.151524+0000","last_became_active":"2024-02-23T00:43:17.851866+0000","last_became_peered":"2024-02-23T00:43:17.851866+0000","last_unstale":"2024-02-23T00:45:41.151524+0000","last_undegraded":"2024-02-23T00:45:41.151524+0000","last_fullsized":"2024-02-23T00:45:41.151524+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":87,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":63,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.2","version":"229'68","reported_seq":247,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.604204+0000","last_change":"2024-02-23T00:45:31.147100+0000","last_active":"2024-02-23T00:45:40.604204+0000","last_peered":"2024-02-23T00:45:40.604204+0000","last_clean":"2024-02-23T00:45:40.604204+0000","last_became_active":"2024-02-23T00:43:17.804792+0000","last_became_peered":"2024-02-23T00:43:17.804792+0000","last_unstale":"2024-02-23T00:45:40.604204+0000","last_undegraded":"2024-02-23T00:45:40.604204+0000","last_fullsized":"2024-02-23T00:45:40.604204+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.3","version":"246'20","reported_seq":210,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.134396+0000","last_change":"2024-02-23T00:45:31.144447+0000","last_active":"2024-02-23T00:45:40.134396+0000","last_peered":"2024-02-23T00:45:40.134396+0000","last_clean":"2024-02-23T00:45:40.134396+0000","last_became_active":"2024-02-23T00:43:17.834686+0000","last_became_peered":"2024-02-23T00:43:17.834686+0000","last_unstale":"2024-02-23T00:45:40.134396+0000","last_undegraded":"2024-02-23T00:45:40.134396+0000","last_fullsized":"2024-02-23T00:45:40.134396+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.4","version":"207'17","reported_seq":203,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.558012+0000","last_change":"2024-02-23T00:45:31.150414+0000","last_active":"2024-02-23T00:45:40.558012+0000","last_peered":"2024-02-23T00:45:40.558012+0000","last_clean":"2024-02-23T00:45:40.558012+0000","last_became_active":"2024-02-23T00:43:17.810343+0000","last_became_peered":"2024-02-23T00:43:17.810343+0000","last_unstale":"2024-02-23T00:45:40.558012+0000","last_undegraded":"2024-02-23T00:45:40.558012+0000","last_fullsized":"2024-02-23T00:45:40.558012+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.5","version":"237'51","reported_seq":238,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.149262+0000","last_change":"2024-02-23T00:45:33.150945+0000","last_active":"2024-02-23T00:45:41.149262+0000","last_peered":"2024-02-23T00:45:41.149262+0000","last_clean":"2024-02-23T00:45:41.149262+0000","last_became_active":"2024-02-23T00:43:17.855339+0000","last_became_peered":"2024-02-23T00:43:17.855339+0000","last_unstale":"2024-02-23T00:45:41.149262+0000","last_undegraded":"2024-02-23T00:45:41.149262+0000","last_fullsized":"2024-02-23T00:45:41.149262+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.6","version":"247'51","reported_seq":238,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.655291+0000","last_change":"2024-02-23T00:45:33.147240+0000","last_active":"2024-02-23T00:45:40.655291+0000","last_peered":"2024-02-23T00:45:40.655291+0000","last_clean":"2024-02-23T00:45:40.655291+0000","last_became_active":"2024-02-23T00:43:17.838838+0000","last_became_peered":"2024-02-23T00:43:17.838838+0000","last_unstale":"2024-02-23T00:45:40.655291+0000","last_undegraded":"2024-02-23T00:45:40.655291+0000","last_fullsized":"2024-02-23T00:45:40.655291+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.7","version":"215'68","reported_seq":245,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.136145+0000","last_change":"2024-02-23T00:45:31.146762+0000","last_active":"2024-02-23T00:45:40.136145+0000","last_peered":"2024-02-23T00:45:40.136145+0000","last_clean":"2024-02-23T00:45:40.136145+0000","last_became_active":"2024-02-23T00:43:17.846548+0000","last_became_peered":"2024-02-23T00:43:17.846548+0000","last_unstale":"2024-02-23T00:45:40.136145+0000","last_undegraded":"2024-02-23T00:45:40.136145+0000","last_fullsized":"2024-02-23T00:45:40.136145+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'68","last_scrub_stamp":"2024-02-23T00:45:25.351414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.351414+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.8","version":"259'90","reported_seq":264,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.653542+0000","last_change":"2024-02-23T00:45:33.145585+0000","last_active":"2024-02-23T00:45:40.653542+0000","last_peered":"2024-02-23T00:45:40.653542+0000","last_clean":"2024-02-23T00:45:40.653542+0000","last_became_active":"2024-02-23T00:43:17.842613+0000","last_became_peered":"2024-02-23T00:43:17.842613+0000","last_unstale":"2024-02-23T00:45:40.653542+0000","last_undegraded":"2024-02-23T00:45:40.653542+0000","last_fullsized":"2024-02-23T00:45:40.653542+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":90,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.9","version":"259'39","reported_seq":226,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.148541+0000","last_change":"2024-02-23T00:45:33.151487+0000","last_active":"2024-02-23T00:45:41.148541+0000","last_peered":"2024-02-23T00:45:41.148541+0000","last_clean":"2024-02-23T00:45:41.148541+0000","last_became_active":"2024-02-23T00:43:17.834297+0000","last_became_peered":"2024-02-23T00:43:17.834297+0000","last_unstale":"2024-02-23T00:45:41.148541+0000","last_undegraded":"2024-02-23T00:45:41.148541+0000","last_fullsized":"2024-02-23T00:45:41.148541+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":12,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.a","version":"253'90","reported_seq":265,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.653902+0000","last_change":"2024-02-23T00:45:33.146399+0000","last_active":"2024-02-23T00:45:40.653902+0000","last_peered":"2024-02-23T00:45:40.653902+0000","last_clean":"2024-02-23T00:45:40.653902+0000","last_became_active":"2024-02-23T00:43:17.839142+0000","last_became_peered":"2024-02-23T00:43:17.839142+0000","last_unstale":"2024-02-23T00:45:40.653902+0000","last_undegraded":"2024-02-23T00:45:40.653902+0000","last_fullsized":"2024-02-23T00:45:40.653902+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":90,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.b","version":"243'68","reported_seq":243,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.654325+0000","last_change":"2024-02-23T00:45:33.146031+0000","last_active":"2024-02-23T00:45:40.654325+0000","last_peered":"2024-02-23T00:45:40.654325+0000","last_clean":"2024-02-23T00:45:40.654325+0000","last_became_active":"2024-02-23T00:43:17.855321+0000","last_became_peered":"2024-02-23T00:43:17.855321+0000","last_unstale":"2024-02-23T00:45:40.654325+0000","last_undegraded":"2024-02-23T00:45:40.654325+0000","last_fullsized":"2024-02-23T00:45:40.654325+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.c","version":"203'34","reported_seq":220,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.135202+0000","last_change":"2024-02-23T00:45:31.142986+0000","last_active":"2024-02-23T00:45:41.135202+0000","last_peered":"2024-02-23T00:45:41.135202+0000","last_clean":"2024-02-23T00:45:41.135202+0000","last_became_active":"2024-02-23T00:43:17.829044+0000","last_became_peered":"2024-02-23T00:43:17.829044+0000","last_unstale":"2024-02-23T00:45:41.135202+0000","last_undegraded":"2024-02-23T00:45:41.135202+0000","last_fullsized":"2024-02-23T00:45:41.135202+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.d","version":"259'56","reported_seq":235,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.149620+0000","last_change":"2024-02-23T00:45:33.150610+0000","last_active":"2024-02-23T00:45:41.149620+0000","last_peered":"2024-02-23T00:45:41.149620+0000","last_clean":"2024-02-23T00:45:41.149620+0000","last_became_active":"2024-02-23T00:43:17.841614+0000","last_became_peered":"2024-02-23T00:43:17.841614+0000","last_unstale":"2024-02-23T00:45:41.149620+0000","last_undegraded":"2024-02-23T00:45:41.149620+0000","last_fullsized":"2024-02-23T00:45:41.149620+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":56,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":17,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.e","version":"211'51","reported_seq":230,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.559014+0000","last_change":"2024-02-23T00:45:31.149105+0000","last_active":"2024-02-23T00:45:40.559014+0000","last_peered":"2024-02-23T00:45:40.559014+0000","last_clean":"2024-02-23T00:45:40.559014+0000","last_became_active":"2024-02-23T00:43:17.845956+0000","last_became_peered":"2024-02-23T00:43:17.845956+0000","last_unstale":"2024-02-23T00:45:40.559014+0000","last_undegraded":"2024-02-23T00:45:40.559014+0000","last_fullsized":"2024-02-23T00:45:40.559014+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.f","version":"241'136","reported_seq":302,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.149560+0000","last_change":"2024-02-23T00:45:33.151502+0000","last_active":"2024-02-23T00:45:41.149560+0000","last_peered":"2024-02-23T00:45:41.149560+0000","last_clean":"2024-02-23T00:45:41.149560+0000","last_became_active":"2024-02-23T00:43:17.855331+0000","last_became_peered":"2024-02-23T00:43:17.855331+0000","last_unstale":"2024-02-23T00:45:41.149560+0000","last_undegraded":"2024-02-23T00:45:41.149560+0000","last_fullsized":"2024-02-23T00:45:41.149560+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":136,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":88,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":40,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.10","version":"219'68","reported_seq":249,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.151394+0000","last_change":"2024-02-23T00:45:31.135638+0000","last_active":"2024-02-23T00:45:41.151394+0000","last_peered":"2024-02-23T00:45:41.151394+0000","last_clean":"2024-02-23T00:45:41.151394+0000","last_became_active":"2024-02-23T00:43:17.819865+0000","last_became_peered":"2024-02-23T00:43:17.819865+0000","last_unstale":"2024-02-23T00:45:41.151394+0000","last_undegraded":"2024-02-23T00:45:41.151394+0000","last_fullsized":"2024-02-23T00:45:41.151394+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.11","version":"215'51","reported_seq":229,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.151491+0000","last_change":"2024-02-23T00:45:31.134829+0000","last_active":"2024-02-23T00:45:41.151491+0000","last_peered":"2024-02-23T00:45:41.151491+0000","last_clean":"2024-02-23T00:45:41.151491+0000","last_became_active":"2024-02-23T00:43:17.852697+0000","last_became_peered":"2024-02-23T00:43:17.852697+0000","last_unstale":"2024-02-23T00:45:41.151491+0000","last_undegraded":"2024-02-23T00:45:41.151491+0000","last_fullsized":"2024-02-23T00:45:41.151491+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.12","version":"240'73","reported_seq":254,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.137494+0000","last_change":"2024-02-23T00:45:31.146190+0000","last_active":"2024-02-23T00:45:40.137494+0000","last_peered":"2024-02-23T00:45:40.137494+0000","last_clean":"2024-02-23T00:45:40.137494+0000","last_became_active":"2024-02-23T00:43:17.856118+0000","last_became_peered":"2024-02-23T00:43:17.856118+0000","last_unstale":"2024-02-23T00:45:40.137494+0000","last_undegraded":"2024-02-23T00:45:40.137494+0000","last_fullsized":"2024-02-23T00:45:40.137494+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":73,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":49,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.13","version":"205'34","reported_seq":220,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.557962+0000","last_change":"2024-02-23T00:45:31.148241+0000","last_active":"2024-02-23T00:45:40.557962+0000","last_peered":"2024-02-23T00:45:40.557962+0000","last_clean":"2024-02-23T00:45:40.557962+0000","last_became_active":"2024-02-23T00:43:17.857823+0000","last_became_peered":"2024-02-23T00:43:17.857823+0000","last_unstale":"2024-02-23T00:45:40.557962+0000","last_undegraded":"2024-02-23T00:45:40.557962+0000","last_fullsized":"2024-02-23T00:45:40.557962+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.14","version":"239'39","reported_seq":221,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.654062+0000","last_change":"2024-02-23T00:45:33.146454+0000","last_active":"2024-02-23T00:45:40.654062+0000","last_peered":"2024-02-23T00:45:40.654062+0000","last_clean":"2024-02-23T00:45:40.654062+0000","last_became_active":"2024-02-23T00:43:17.816138+0000","last_became_peered":"2024-02-23T00:43:17.816138+0000","last_unstale":"2024-02-23T00:45:40.654062+0000","last_undegraded":"2024-02-23T00:45:40.654062+0000","last_fullsized":"2024-02-23T00:45:40.654062+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":12,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.15","version":"201'17","reported_seq":207,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.136254+0000","last_change":"2024-02-23T00:45:31.143544+0000","last_active":"2024-02-23T00:45:41.136254+0000","last_peered":"2024-02-23T00:45:41.136254+0000","last_clean":"2024-02-23T00:45:41.136254+0000","last_became_active":"2024-02-23T00:43:17.843834+0000","last_became_peered":"2024-02-23T00:43:17.843834+0000","last_unstale":"2024-02-23T00:45:41.136254+0000","last_undegraded":"2024-02-23T00:45:41.136254+0000","last_fullsized":"2024-02-23T00:45:41.136254+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.16","version":"240'23","reported_seq":213,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.149396+0000","last_change":"2024-02-23T00:45:33.151166+0000","last_active":"2024-02-23T00:45:41.149396+0000","last_peered":"2024-02-23T00:45:41.149396+0000","last_clean":"2024-02-23T00:45:41.149396+0000","last_became_active":"2024-02-23T00:43:17.834291+0000","last_became_peered":"2024-02-23T00:43:17.834291+0000","last_unstale":"2024-02-23T00:45:41.149396+0000","last_undegraded":"2024-02-23T00:45:41.149396+0000","last_fullsized":"2024-02-23T00:45:41.149396+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":14,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.17","version":"229'57","reported_seq":245,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.605133+0000","last_change":"2024-02-23T00:45:31.142517+0000","last_active":"2024-02-23T00:45:40.605133+0000","last_peered":"2024-02-23T00:45:40.605133+0000","last_clean":"2024-02-23T00:45:40.605133+0000","last_became_active":"2024-02-23T00:43:17.843525+0000","last_became_peered":"2024-02-23T00:43:17.843525+0000","last_unstale":"2024-02-23T00:45:40.605133+0000","last_undegraded":"2024-02-23T00:45:40.605133+0000","last_fullsized":"2024-02-23T00:45:40.605133+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":57,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":18,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.19","version":"267'96","reported_seq":279,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.603587+0000","last_change":"2024-02-23T00:45:31.140310+0000","last_active":"2024-02-23T00:45:40.603587+0000","last_peered":"2024-02-23T00:45:40.603587+0000","last_clean":"2024-02-23T00:45:40.603587+0000","last_became_active":"2024-02-23T00:43:17.815830+0000","last_became_peered":"2024-02-23T00:43:17.815830+0000","last_unstale":"2024-02-23T00:45:40.603587+0000","last_undegraded":"2024-02-23T00:45:40.603587+0000","last_fullsized":"2024-02-23T00:45:40.603587+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":96,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":73,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":7,"num_read_kb":0,"num_write":30,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.18","version":"150'5","reported_seq":198,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.134756+0000","last_change":"2024-02-23T00:45:31.141554+0000","last_active":"2024-02-23T00:45:41.134756+0000","last_peered":"2024-02-23T00:45:41.134756+0000","last_clean":"2024-02-23T00:45:41.134756+0000","last_became_active":"2024-02-23T00:43:17.840351+0000","last_became_peered":"2024-02-23T00:43:17.840351+0000","last_unstale":"2024-02-23T00:45:41.134756+0000","last_undegraded":"2024-02-23T00:45:41.134756+0000","last_fullsized":"2024-02-23T00:45:41.134756+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1b","version":"229'68","reported_seq":248,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.148826+0000","last_change":"2024-02-23T00:45:33.151699+0000","last_active":"2024-02-23T00:45:41.148826+0000","last_peered":"2024-02-23T00:45:41.148826+0000","last_clean":"2024-02-23T00:45:41.148826+0000","last_became_active":"2024-02-23T00:43:17.803043+0000","last_became_peered":"2024-02-23T00:43:17.803043+0000","last_unstale":"2024-02-23T00:45:41.148826+0000","last_undegraded":"2024-02-23T00:45:41.148826+0000","last_fullsized":"2024-02-23T00:45:41.148826+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.1a","version":"225'34","reported_seq":216,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.604008+0000","last_change":"2024-02-23T00:45:31.146687+0000","last_active":"2024-02-23T00:45:40.604008+0000","last_peered":"2024-02-23T00:45:40.604008+0000","last_clean":"2024-02-23T00:45:40.604008+0000","last_became_active":"2024-02-23T00:43:17.798285+0000","last_became_peered":"2024-02-23T00:43:17.798285+0000","last_unstale":"2024-02-23T00:45:40.604008+0000","last_undegraded":"2024-02-23T00:45:40.604008+0000","last_fullsized":"2024-02-23T00:45:40.604008+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1d","version":"215'104","reported_seq":281,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.817129+0000","last_change":"2024-02-23T00:45:31.141111+0000","last_active":"2024-02-23T00:45:40.817129+0000","last_peered":"2024-02-23T00:45:40.817129+0000","last_clean":"2024-02-23T00:45:40.817129+0000","last_became_active":"2024-02-23T00:43:17.828194+0000","last_became_peered":"2024-02-23T00:43:17.828194+0000","last_unstale":"2024-02-23T00:45:40.817129+0000","last_undegraded":"2024-02-23T00:45:40.817129+0000","last_fullsized":"2024-02-23T00:45:40.817129+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":104,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":74,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":32,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"62.1c","version":"223'51","reported_seq":230,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.603984+0000","last_change":"2024-02-23T00:45:31.147476+0000","last_active":"2024-02-23T00:45:40.603984+0000","last_peered":"2024-02-23T00:45:40.603984+0000","last_clean":"2024-02-23T00:45:40.603984+0000","last_became_active":"2024-02-23T00:43:17.804718+0000","last_became_peered":"2024-02-23T00:43:17.804718+0000","last_unstale":"2024-02-23T00:45:40.603984+0000","last_undegraded":"2024-02-23T00:45:40.603984+0000","last_fullsized":"2024-02-23T00:45:40.603984+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1f","version":"240'40","reported_seq":227,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.135945+0000","last_change":"2024-02-23T00:45:31.146635+0000","last_active":"2024-02-23T00:45:40.135945+0000","last_peered":"2024-02-23T00:45:40.135945+0000","last_clean":"2024-02-23T00:45:40.135945+0000","last_became_active":"2024-02-23T00:43:17.839914+0000","last_became_peered":"2024-02-23T00:43:17.839914+0000","last_unstale":"2024-02-23T00:45:40.135945+0000","last_undegraded":"2024-02-23T00:45:40.135945+0000","last_fullsized":"2024-02-23T00:45:40.135945+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":13,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.1e","version":"245'51","reported_seq":231,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.655272+0000","last_change":"2024-02-23T00:45:33.146886+0000","last_active":"2024-02-23T00:45:40.655272+0000","last_peered":"2024-02-23T00:45:40.655272+0000","last_clean":"2024-02-23T00:45:40.655272+0000","last_became_active":"2024-02-23T00:43:17.838682+0000","last_became_peered":"2024-02-23T00:43:17.838682+0000","last_unstale":"2024-02-23T00:45:40.655272+0000","last_undegraded":"2024-02-23T00:45:40.655272+0000","last_fullsized":"2024-02-23T00:45:40.655272+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"67'8","reported_seq":309,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.653814+0000","last_change":"2024-02-23T00:42:47.985647+0000","last_active":"2024-02-23T00:45:40.653814+0000","last_peered":"2024-02-23T00:45:40.653814+0000","last_clean":"2024-02-23T00:45:40.653814+0000","last_became_active":"2024-02-23T00:39:18.793414+0000","last_became_peered":"2024-02-23T00:39:18.793414+0000","last_unstale":"2024-02-23T00:45:40.653814+0000","last_undegraded":"2024-02-23T00:45:40.653814+0000","last_fullsized":"2024-02-23T00:45:40.653814+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:42:47.985573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:47.985573+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"68'27","reported_seq":286,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.816385+0000","last_change":"2024-02-23T00:45:04.138883+0000","last_active":"2024-02-23T00:45:40.816385+0000","last_peered":"2024-02-23T00:45:40.816385+0000","last_clean":"2024-02-23T00:45:40.816385+0000","last_became_active":"2024-02-23T00:39:16.859401+0000","last_became_peered":"2024-02-23T00:39:16.859401+0000","last_unstale":"2024-02-23T00:45:40.816385+0000","last_undegraded":"2024-02-23T00:45:40.816385+0000","last_fullsized":"2024-02-23T00:45:40.816385+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'27","last_scrub_stamp":"2024-02-23T00:45:04.138751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.138751+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":231,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.136270+0000","last_change":"2024-02-23T00:43:42.320439+0000","last_active":"2024-02-23T00:45:40.136270+0000","last_peered":"2024-02-23T00:45:40.136270+0000","last_clean":"2024-02-23T00:45:40.136270+0000","last_became_active":"2024-02-23T00:39:14.784436+0000","last_became_peered":"2024-02-23T00:39:14.784436+0000","last_unstale":"2024-02-23T00:45:40.136270+0000","last_undegraded":"2024-02-23T00:45:40.136270+0000","last_fullsized":"2024-02-23T00:45:40.136270+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:42.320346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:04.330426+0000","last_clean_scrub_stamp":"2024-02-23T00:43:42.320346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":245,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.816568+0000","last_change":"2024-02-23T00:44:21.972446+0000","last_active":"2024-02-23T00:45:40.816568+0000","last_peered":"2024-02-23T00:45:40.816568+0000","last_clean":"2024-02-23T00:45:40.816568+0000","last_became_active":"2024-02-23T00:43:32.282545+0000","last_became_peered":"2024-02-23T00:43:32.282545+0000","last_unstale":"2024-02-23T00:45:40.816568+0000","last_undegraded":"2024-02-23T00:45:40.816568+0000","last_fullsized":"2024-02-23T00:45:40.816568+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:21.972383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:21.972383+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.3","version":"110'5","reported_seq":180,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.148943+0000","last_change":"2024-02-23T00:45:17.356489+0000","last_active":"2024-02-23T00:45:41.148943+0000","last_peered":"2024-02-23T00:45:41.148943+0000","last_clean":"2024-02-23T00:45:41.148943+0000","last_became_active":"2024-02-23T00:42:58.835612+0000","last_became_peered":"2024-02-23T00:42:58.835612+0000","last_unstale":"2024-02-23T00:45:41.148943+0000","last_undegraded":"2024-02-23T00:45:41.148943+0000","last_fullsized":"2024-02-23T00:45:41.148943+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'5","last_scrub_stamp":"2024-02-23T00:45:17.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:17.356406+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":174,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.816869+0000","last_change":"2024-02-23T00:45:17.181400+0000","last_active":"2024-02-23T00:45:40.816869+0000","last_peered":"2024-02-23T00:45:40.816869+0000","last_clean":"2024-02-23T00:45:40.816869+0000","last_became_active":"2024-02-23T00:42:58.828846+0000","last_became_peered":"2024-02-23T00:42:58.828846+0000","last_unstale":"2024-02-23T00:45:40.816869+0000","last_undegraded":"2024-02-23T00:45:40.816869+0000","last_fullsized":"2024-02-23T00:45:40.816869+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:17.181335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:17.181335+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"68'15","reported_seq":262,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.558203+0000","last_change":"2024-02-23T00:43:54.355050+0000","last_active":"2024-02-23T00:45:40.558203+0000","last_peered":"2024-02-23T00:45:40.558203+0000","last_clean":"2024-02-23T00:45:40.558203+0000","last_became_active":"2024-02-23T00:39:16.783701+0000","last_became_peered":"2024-02-23T00:39:16.783701+0000","last_unstale":"2024-02-23T00:45:40.558203+0000","last_undegraded":"2024-02-23T00:45:40.558203+0000","last_fullsized":"2024-02-23T00:45:40.558203+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:43:54.354974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:54.354974+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":228,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.153004+0000","last_change":"2024-02-23T00:42:48.591600+0000","last_active":"2024-02-23T00:45:41.153004+0000","last_peered":"2024-02-23T00:45:41.153004+0000","last_clean":"2024-02-23T00:45:41.153004+0000","last_became_active":"2024-02-23T00:39:14.772493+0000","last_became_peered":"2024-02-23T00:39:14.772493+0000","last_unstale":"2024-02-23T00:45:41.153004+0000","last_undegraded":"2024-02-23T00:45:41.153004+0000","last_fullsized":"2024-02-23T00:45:41.153004+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:48.591486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:48.591486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":227,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.137751+0000","last_change":"2024-02-23T00:44:22.380954+0000","last_active":"2024-02-23T00:45:40.137751+0000","last_peered":"2024-02-23T00:45:40.137751+0000","last_clean":"2024-02-23T00:45:40.137751+0000","last_became_active":"2024-02-23T00:39:18.794476+0000","last_became_peered":"2024-02-23T00:39:18.794476+0000","last_unstale":"2024-02-23T00:45:40.137751+0000","last_undegraded":"2024-02-23T00:45:40.137751+0000","last_fullsized":"2024-02-23T00:45:40.137751+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:22.380841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:08.335330+0000","last_clean_scrub_stamp":"2024-02-23T00:44:22.380841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":240,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.557891+0000","last_change":"2024-02-23T00:43:35.079353+0000","last_active":"2024-02-23T00:45:40.557891+0000","last_peered":"2024-02-23T00:45:40.557891+0000","last_clean":"2024-02-23T00:45:40.557891+0000","last_became_active":"2024-02-23T00:43:35.079004+0000","last_became_peered":"2024-02-23T00:43:35.079004+0000","last_unstale":"2024-02-23T00:45:40.557891+0000","last_undegraded":"2024-02-23T00:45:40.557891+0000","last_fullsized":"2024-02-23T00:45:40.557891+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:46.359646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:46.359646+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"110'5","reported_seq":178,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.137736+0000","last_change":"2024-02-23T00:45:06.483322+0000","last_active":"2024-02-23T00:45:40.137736+0000","last_peered":"2024-02-23T00:45:40.137736+0000","last_clean":"2024-02-23T00:45:40.137736+0000","last_became_active":"2024-02-23T00:42:58.741558+0000","last_became_peered":"2024-02-23T00:42:58.741558+0000","last_unstale":"2024-02-23T00:45:40.137736+0000","last_undegraded":"2024-02-23T00:45:40.137736+0000","last_fullsized":"2024-02-23T00:45:40.137736+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'5","last_scrub_stamp":"2024-02-23T00:45:06.483252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.483252+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":174,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.559392+0000","last_change":"2024-02-23T00:45:22.597544+0000","last_active":"2024-02-23T00:45:40.559392+0000","last_peered":"2024-02-23T00:45:40.559392+0000","last_clean":"2024-02-23T00:45:40.559392+0000","last_became_active":"2024-02-23T00:42:58.867305+0000","last_became_peered":"2024-02-23T00:42:58.867305+0000","last_unstale":"2024-02-23T00:45:40.559392+0000","last_undegraded":"2024-02-23T00:45:40.559392+0000","last_fullsized":"2024-02-23T00:45:40.559392+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:22.597434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:22.597434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"88'35","reported_seq":285,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.604809+0000","last_change":"2024-02-23T00:43:54.438282+0000","last_active":"2024-02-23T00:45:40.604809+0000","last_peered":"2024-02-23T00:45:40.604809+0000","last_clean":"2024-02-23T00:45:40.604809+0000","last_became_active":"2024-02-23T00:39:16.857413+0000","last_became_peered":"2024-02-23T00:39:16.857413+0000","last_unstale":"2024-02-23T00:45:40.604809+0000","last_undegraded":"2024-02-23T00:45:40.604809+0000","last_fullsized":"2024-02-23T00:45:40.604809+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'35","last_scrub_stamp":"2024-02-23T00:43:54.438168+0000","last_deep_scrub":"88'35","last_deep_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_clean_scrub_stamp":"2024-02-23T00:43:54.438168+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":231,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.136057+0000","last_change":"2024-02-23T00:43:53.317283+0000","last_active":"2024-02-23T00:45:40.136057+0000","last_peered":"2024-02-23T00:45:40.136057+0000","last_clean":"2024-02-23T00:45:40.136057+0000","last_became_active":"2024-02-23T00:39:14.776705+0000","last_became_peered":"2024-02-23T00:39:14.776705+0000","last_unstale":"2024-02-23T00:45:40.136057+0000","last_undegraded":"2024-02-23T00:45:40.136057+0000","last_fullsized":"2024-02-23T00:45:40.136057+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:53.317202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_clean_scrub_stamp":"2024-02-23T00:43:53.317202+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":227,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.134601+0000","last_change":"2024-02-23T00:43:44.313672+0000","last_active":"2024-02-23T00:45:40.134601+0000","last_peered":"2024-02-23T00:45:40.134601+0000","last_clean":"2024-02-23T00:45:40.134601+0000","last_became_active":"2024-02-23T00:39:18.792718+0000","last_became_peered":"2024-02-23T00:39:18.792718+0000","last_unstale":"2024-02-23T00:45:40.134601+0000","last_undegraded":"2024-02-23T00:45:40.134601+0000","last_fullsized":"2024-02-23T00:45:40.134601+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:44.313501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:44.313501+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":242,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.654569+0000","last_change":"2024-02-23T00:44:36.758043+0000","last_active":"2024-02-23T00:45:40.654569+0000","last_peered":"2024-02-23T00:45:40.654569+0000","last_clean":"2024-02-23T00:45:40.654569+0000","last_became_active":"2024-02-23T00:43:39.470999+0000","last_became_peered":"2024-02-23T00:43:39.470999+0000","last_unstale":"2024-02-23T00:45:40.654569+0000","last_undegraded":"2024-02-23T00:45:40.654569+0000","last_fullsized":"2024-02-23T00:45:40.654569+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:36.757960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:36.757960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"110'5","reported_seq":178,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.604245+0000","last_change":"2024-02-23T00:45:23.832372+0000","last_active":"2024-02-23T00:45:40.604245+0000","last_peered":"2024-02-23T00:45:40.604245+0000","last_clean":"2024-02-23T00:45:40.604245+0000","last_became_active":"2024-02-23T00:42:58.684001+0000","last_became_peered":"2024-02-23T00:42:58.684001+0000","last_unstale":"2024-02-23T00:45:40.604245+0000","last_undegraded":"2024-02-23T00:45:40.604245+0000","last_fullsized":"2024-02-23T00:45:40.604245+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'5","last_scrub_stamp":"2024-02-23T00:45:23.832291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.832291+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":170,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.152068+0000","last_change":"2024-02-23T00:42:57.641727+0000","last_active":"2024-02-23T00:45:41.152068+0000","last_peered":"2024-02-23T00:45:41.152068+0000","last_clean":"2024-02-23T00:45:41.152068+0000","last_became_active":"2024-02-23T00:42:57.639013+0000","last_became_peered":"2024-02-23T00:42:57.639013+0000","last_unstale":"2024-02-23T00:45:41.152068+0000","last_undegraded":"2024-02-23T00:45:41.152068+0000","last_fullsized":"2024-02-23T00:45:41.152068+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"67'17","reported_seq":281,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.149968+0000","last_change":"2024-02-23T00:43:45.194207+0000","last_active":"2024-02-23T00:45:41.149968+0000","last_peered":"2024-02-23T00:45:41.149968+0000","last_clean":"2024-02-23T00:45:41.149968+0000","last_became_active":"2024-02-23T00:40:42.933806+0000","last_became_peered":"2024-02-23T00:40:42.933806+0000","last_unstale":"2024-02-23T00:45:41.149968+0000","last_undegraded":"2024-02-23T00:45:41.149968+0000","last_fullsized":"2024-02-23T00:45:41.149968+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'17","last_scrub_stamp":"2024-02-23T00:43:45.194139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.194139+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":202,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.134874+0000","last_change":"2024-02-23T00:43:31.362599+0000","last_active":"2024-02-23T00:45:40.134874+0000","last_peered":"2024-02-23T00:45:40.134874+0000","last_clean":"2024-02-23T00:45:40.134874+0000","last_became_active":"2024-02-23T00:40:42.931041+0000","last_became_peered":"2024-02-23T00:40:42.931041+0000","last_unstale":"2024-02-23T00:45:40.134874+0000","last_undegraded":"2024-02-23T00:45:40.134874+0000","last_fullsized":"2024-02-23T00:45:40.134874+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:31.362510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:31.362510+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":229,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.152475+0000","last_change":"2024-02-23T00:44:22.804352+0000","last_active":"2024-02-23T00:45:41.152475+0000","last_peered":"2024-02-23T00:45:41.152475+0000","last_clean":"2024-02-23T00:45:41.152475+0000","last_became_active":"2024-02-23T00:39:18.799875+0000","last_became_peered":"2024-02-23T00:39:18.799875+0000","last_unstale":"2024-02-23T00:45:41.152475+0000","last_undegraded":"2024-02-23T00:45:41.152475+0000","last_fullsized":"2024-02-23T00:45:41.152475+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:22.804279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:22.804279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":243,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.136577+0000","last_change":"2024-02-23T00:44:36.481327+0000","last_active":"2024-02-23T00:45:40.136577+0000","last_peered":"2024-02-23T00:45:40.136577+0000","last_clean":"2024-02-23T00:45:40.136577+0000","last_became_active":"2024-02-23T00:43:08.885257+0000","last_became_peered":"2024-02-23T00:43:08.885257+0000","last_unstale":"2024-02-23T00:45:40.136577+0000","last_undegraded":"2024-02-23T00:45:40.136577+0000","last_fullsized":"2024-02-23T00:45:40.136577+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:36.481256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:36.481256+0000","last_clean_scrub_stamp":"2024-02-23T00:44:36.481256+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.6","version":"110'9","reported_seq":178,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.559395+0000","last_change":"2024-02-23T00:42:58.871934+0000","last_active":"2024-02-23T00:45:40.559395+0000","last_peered":"2024-02-23T00:45:40.559395+0000","last_clean":"2024-02-23T00:45:40.559395+0000","last_became_active":"2024-02-23T00:42:58.870147+0000","last_became_peered":"2024-02-23T00:42:58.870147+0000","last_unstale":"2024-02-23T00:45:40.559395+0000","last_undegraded":"2024-02-23T00:45:40.559395+0000","last_fullsized":"2024-02-23T00:45:40.559395+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":175,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.151997+0000","last_change":"2024-02-23T00:45:23.744990+0000","last_active":"2024-02-23T00:45:41.151997+0000","last_peered":"2024-02-23T00:45:41.151997+0000","last_clean":"2024-02-23T00:45:41.151997+0000","last_became_active":"2024-02-23T00:42:57.675306+0000","last_became_peered":"2024-02-23T00:42:57.675306+0000","last_unstale":"2024-02-23T00:45:41.151997+0000","last_undegraded":"2024-02-23T00:45:41.151997+0000","last_fullsized":"2024-02-23T00:45:41.151997+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:23.744846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.744846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"67'10","reported_seq":246,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.149193+0000","last_change":"2024-02-23T00:45:25.175258+0000","last_active":"2024-02-23T00:45:41.149193+0000","last_peered":"2024-02-23T00:45:41.149193+0000","last_clean":"2024-02-23T00:45:41.149193+0000","last_became_active":"2024-02-23T00:39:16.777530+0000","last_became_peered":"2024-02-23T00:39:16.777530+0000","last_unstale":"2024-02-23T00:45:41.149193+0000","last_undegraded":"2024-02-23T00:45:41.149193+0000","last_fullsized":"2024-02-23T00:45:41.149193+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'10","last_scrub_stamp":"2024-02-23T00:45:25.175173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.175173+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":233,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.151935+0000","last_change":"2024-02-23T00:45:10.693332+0000","last_active":"2024-02-23T00:45:41.151935+0000","last_peered":"2024-02-23T00:45:41.151935+0000","last_clean":"2024-02-23T00:45:41.151935+0000","last_became_active":"2024-02-23T00:39:14.780379+0000","last_became_peered":"2024-02-23T00:39:14.780379+0000","last_unstale":"2024-02-23T00:45:41.151935+0000","last_undegraded":"2024-02-23T00:45:41.151935+0000","last_fullsized":"2024-02-23T00:45:41.151935+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.693273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.575242+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.693273+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"67'8","reported_seq":312,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.135098+0000","last_change":"2024-02-23T00:44:25.470245+0000","last_active":"2024-02-23T00:45:40.135098+0000","last_peered":"2024-02-23T00:45:40.135098+0000","last_clean":"2024-02-23T00:45:40.135098+0000","last_became_active":"2024-02-23T00:39:18.799544+0000","last_became_peered":"2024-02-23T00:39:18.799544+0000","last_unstale":"2024-02-23T00:45:40.135098+0000","last_undegraded":"2024-02-23T00:45:40.135098+0000","last_fullsized":"2024-02-23T00:45:40.135098+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:44:25.470174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:25.470174+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":242,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.149819+0000","last_change":"2024-02-23T00:43:36.205958+0000","last_active":"2024-02-23T00:45:41.149819+0000","last_peered":"2024-02-23T00:45:41.149819+0000","last_clean":"2024-02-23T00:45:41.149819+0000","last_became_active":"2024-02-23T00:43:13.338697+0000","last_became_peered":"2024-02-23T00:43:13.338697+0000","last_unstale":"2024-02-23T00:45:41.149819+0000","last_undegraded":"2024-02-23T00:45:41.149819+0000","last_fullsized":"2024-02-23T00:45:41.149819+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:36.205886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:15.081153+0000","last_clean_scrub_stamp":"2024-02-23T00:43:36.205886+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":170,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.152675+0000","last_change":"2024-02-23T00:42:57.686234+0000","last_active":"2024-02-23T00:45:41.152675+0000","last_peered":"2024-02-23T00:45:41.152675+0000","last_clean":"2024-02-23T00:45:41.152675+0000","last_became_active":"2024-02-23T00:42:57.685579+0000","last_became_peered":"2024-02-23T00:42:57.685579+0000","last_unstale":"2024-02-23T00:45:41.152675+0000","last_undegraded":"2024-02-23T00:45:41.152675+0000","last_fullsized":"2024-02-23T00:45:41.152675+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":175,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.138107+0000","last_change":"2024-02-23T00:45:22.874761+0000","last_active":"2024-02-23T00:45:41.138107+0000","last_peered":"2024-02-23T00:45:41.138107+0000","last_clean":"2024-02-23T00:45:41.138107+0000","last_became_active":"2024-02-23T00:42:58.765106+0000","last_became_peered":"2024-02-23T00:42:58.765106+0000","last_unstale":"2024-02-23T00:45:41.138107+0000","last_undegraded":"2024-02-23T00:45:41.138107+0000","last_fullsized":"2024-02-23T00:45:41.138107+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:22.874696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:22.874696+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"67'15","reported_seq":256,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.152704+0000","last_change":"2024-02-23T00:44:19.830383+0000","last_active":"2024-02-23T00:45:41.152704+0000","last_peered":"2024-02-23T00:45:41.152704+0000","last_clean":"2024-02-23T00:45:41.152704+0000","last_became_active":"2024-02-23T00:39:16.789757+0000","last_became_peered":"2024-02-23T00:39:16.789757+0000","last_unstale":"2024-02-23T00:45:41.152704+0000","last_undegraded":"2024-02-23T00:45:41.152704+0000","last_fullsized":"2024-02-23T00:45:41.152704+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:44:19.830309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:44:19.830309+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":233,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.151790+0000","last_change":"2024-02-23T00:44:29.773000+0000","last_active":"2024-02-23T00:45:41.151790+0000","last_peered":"2024-02-23T00:45:41.151790+0000","last_clean":"2024-02-23T00:45:41.151790+0000","last_became_active":"2024-02-23T00:39:14.780526+0000","last_became_peered":"2024-02-23T00:39:14.780526+0000","last_unstale":"2024-02-23T00:45:41.151790+0000","last_undegraded":"2024-02-23T00:45:41.151790+0000","last_fullsized":"2024-02-23T00:45:41.151790+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:29.772933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:29.772933+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":314,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.149728+0000","last_change":"2024-02-23T00:44:23.109508+0000","last_active":"2024-02-23T00:45:41.149728+0000","last_peered":"2024-02-23T00:45:41.149728+0000","last_clean":"2024-02-23T00:45:41.149728+0000","last_became_active":"2024-02-23T00:39:18.800440+0000","last_became_peered":"2024-02-23T00:39:18.800440+0000","last_unstale":"2024-02-23T00:45:41.149728+0000","last_undegraded":"2024-02-23T00:45:41.149728+0000","last_fullsized":"2024-02-23T00:45:41.149728+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:44:23.109439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.109439+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":245,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.816088+0000","last_change":"2024-02-23T00:44:22.976713+0000","last_active":"2024-02-23T00:45:40.816088+0000","last_peered":"2024-02-23T00:45:40.816088+0000","last_clean":"2024-02-23T00:45:40.816088+0000","last_became_active":"2024-02-23T00:43:16.650527+0000","last_became_peered":"2024-02-23T00:43:16.650527+0000","last_unstale":"2024-02-23T00:45:40.816088+0000","last_undegraded":"2024-02-23T00:45:40.816088+0000","last_fullsized":"2024-02-23T00:45:40.816088+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:22.976611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:10.666299+0000","last_clean_scrub_stamp":"2024-02-23T00:44:22.976611+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":175,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.152261+0000","last_change":"2024-02-23T00:45:26.719081+0000","last_active":"2024-02-23T00:45:41.152261+0000","last_peered":"2024-02-23T00:45:41.152261+0000","last_clean":"2024-02-23T00:45:41.152261+0000","last_became_active":"2024-02-23T00:42:57.671472+0000","last_became_peered":"2024-02-23T00:42:57.671472+0000","last_unstale":"2024-02-23T00:45:41.152261+0000","last_undegraded":"2024-02-23T00:45:41.152261+0000","last_fullsized":"2024-02-23T00:45:41.152261+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:26.719011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:26.719011+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,3],"acting":[1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":170,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.148641+0000","last_change":"2024-02-23T00:42:58.834770+0000","last_active":"2024-02-23T00:45:41.148641+0000","last_peered":"2024-02-23T00:45:41.148641+0000","last_clean":"2024-02-23T00:45:41.148641+0000","last_became_active":"2024-02-23T00:42:58.833924+0000","last_became_peered":"2024-02-23T00:42:58.833924+0000","last_unstale":"2024-02-23T00:45:41.148641+0000","last_undegraded":"2024-02-23T00:45:41.148641+0000","last_fullsized":"2024-02-23T00:45:41.148641+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"68'17","reported_seq":283,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.149676+0000","last_change":"2024-02-23T00:43:33.151471+0000","last_active":"2024-02-23T00:45:41.149676+0000","last_peered":"2024-02-23T00:45:41.149676+0000","last_clean":"2024-02-23T00:45:41.149676+0000","last_became_active":"2024-02-23T00:40:42.933759+0000","last_became_peered":"2024-02-23T00:40:42.933759+0000","last_unstale":"2024-02-23T00:45:41.149676+0000","last_undegraded":"2024-02-23T00:45:41.149676+0000","last_fullsized":"2024-02-23T00:45:41.149676+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:43:33.151401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:33.151401+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":230,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.654511+0000","last_change":"2024-02-23T00:44:18.886545+0000","last_active":"2024-02-23T00:45:40.654511+0000","last_peered":"2024-02-23T00:45:40.654511+0000","last_clean":"2024-02-23T00:45:40.654511+0000","last_became_active":"2024-02-23T00:39:14.784121+0000","last_became_peered":"2024-02-23T00:39:14.784121+0000","last_unstale":"2024-02-23T00:45:40.654511+0000","last_undegraded":"2024-02-23T00:45:40.654511+0000","last_fullsized":"2024-02-23T00:45:40.654511+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:18.886477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:18.886477+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":229,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.817592+0000","last_change":"2024-02-23T00:45:08.118391+0000","last_active":"2024-02-23T00:45:40.817592+0000","last_peered":"2024-02-23T00:45:40.817592+0000","last_clean":"2024-02-23T00:45:40.817592+0000","last_became_active":"2024-02-23T00:39:18.800777+0000","last_became_peered":"2024-02-23T00:39:18.800777+0000","last_unstale":"2024-02-23T00:45:40.817592+0000","last_undegraded":"2024-02-23T00:45:40.817592+0000","last_fullsized":"2024-02-23T00:45:40.817592+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.118302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.118302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"67'1","reported_seq":246,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.135365+0000","last_change":"2024-02-23T00:43:56.091057+0000","last_active":"2024-02-23T00:45:41.135365+0000","last_peered":"2024-02-23T00:45:41.135365+0000","last_clean":"2024-02-23T00:45:41.135365+0000","last_became_active":"2024-02-23T00:43:22.866277+0000","last_became_peered":"2024-02-23T00:43:22.866277+0000","last_unstale":"2024-02-23T00:45:41.135365+0000","last_undegraded":"2024-02-23T00:45:41.135365+0000","last_fullsized":"2024-02-23T00:45:41.135365+0000","mapping_epoch":65,"log_start":"67'1","ondisk_log_start":"67'1","created":65,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:43:56.090982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:09.079918+0000","last_clean_scrub_stamp":"2024-02-23T00:43:56.090982+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":173,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.604694+0000","last_change":"2024-02-23T00:45:06.651852+0000","last_active":"2024-02-23T00:45:40.604694+0000","last_peered":"2024-02-23T00:45:40.604694+0000","last_clean":"2024-02-23T00:45:40.604694+0000","last_became_active":"2024-02-23T00:42:58.718615+0000","last_became_peered":"2024-02-23T00:42:58.718615+0000","last_unstale":"2024-02-23T00:45:40.604694+0000","last_undegraded":"2024-02-23T00:45:40.604694+0000","last_fullsized":"2024-02-23T00:45:40.604694+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.651711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.651711+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":170,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.151614+0000","last_change":"2024-02-23T00:42:57.655252+0000","last_active":"2024-02-23T00:45:41.151614+0000","last_peered":"2024-02-23T00:45:41.151614+0000","last_clean":"2024-02-23T00:45:41.151614+0000","last_became_active":"2024-02-23T00:42:57.653279+0000","last_became_peered":"2024-02-23T00:42:57.653279+0000","last_unstale":"2024-02-23T00:45:41.151614+0000","last_undegraded":"2024-02-23T00:45:41.151614+0000","last_fullsized":"2024-02-23T00:45:41.151614+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":310,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.134976+0000","last_change":"2024-02-23T00:42:53.978054+0000","last_active":"2024-02-23T00:45:41.134976+0000","last_peered":"2024-02-23T00:45:41.134976+0000","last_clean":"2024-02-23T00:45:41.134976+0000","last_became_active":"2024-02-23T00:39:18.798536+0000","last_became_peered":"2024-02-23T00:39:18.798536+0000","last_unstale":"2024-02-23T00:45:41.134976+0000","last_undegraded":"2024-02-23T00:45:41.134976+0000","last_fullsized":"2024-02-23T00:45:41.134976+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:42:53.977950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.977950+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"68'20","reported_seq":267,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.149027+0000","last_change":"2024-02-23T00:45:05.291909+0000","last_active":"2024-02-23T00:45:41.149027+0000","last_peered":"2024-02-23T00:45:41.149027+0000","last_clean":"2024-02-23T00:45:41.149027+0000","last_became_active":"2024-02-23T00:39:16.780130+0000","last_became_peered":"2024-02-23T00:39:16.780130+0000","last_unstale":"2024-02-23T00:45:41.149027+0000","last_undegraded":"2024-02-23T00:45:41.149027+0000","last_fullsized":"2024-02-23T00:45:41.149027+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'20","last_scrub_stamp":"2024-02-23T00:45:05.291836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:05.291836+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":233,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.605406+0000","last_change":"2024-02-23T00:43:51.391999+0000","last_active":"2024-02-23T00:45:40.605406+0000","last_peered":"2024-02-23T00:45:40.605406+0000","last_clean":"2024-02-23T00:45:40.605406+0000","last_became_active":"2024-02-23T00:39:14.781999+0000","last_became_peered":"2024-02-23T00:39:14.781999+0000","last_unstale":"2024-02-23T00:45:40.605406+0000","last_undegraded":"2024-02-23T00:45:40.605406+0000","last_fullsized":"2024-02-23T00:45:40.605406+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:51.391939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:51.391939+0000","last_clean_scrub_stamp":"2024-02-23T00:43:51.391939+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":224,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.655074+0000","last_change":"2024-02-23T00:44:24.759694+0000","last_active":"2024-02-23T00:45:40.655074+0000","last_peered":"2024-02-23T00:45:40.655074+0000","last_clean":"2024-02-23T00:45:40.655074+0000","last_became_active":"2024-02-23T00:39:21.161928+0000","last_became_peered":"2024-02-23T00:39:21.161928+0000","last_unstale":"2024-02-23T00:45:40.655074+0000","last_undegraded":"2024-02-23T00:45:40.655074+0000","last_fullsized":"2024-02-23T00:45:40.655074+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:24.759627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.945642+0000","last_clean_scrub_stamp":"2024-02-23T00:44:24.759627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"110'10","reported_seq":183,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.138047+0000","last_change":"2024-02-23T00:45:24.377454+0000","last_active":"2024-02-23T00:45:40.138047+0000","last_peered":"2024-02-23T00:45:40.138047+0000","last_clean":"2024-02-23T00:45:40.138047+0000","last_became_active":"2024-02-23T00:42:58.696292+0000","last_became_peered":"2024-02-23T00:42:58.696292+0000","last_unstale":"2024-02-23T00:45:40.138047+0000","last_undegraded":"2024-02-23T00:45:40.138047+0000","last_fullsized":"2024-02-23T00:45:40.138047+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'10","last_scrub_stamp":"2024-02-23T00:45:24.377329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:24.377329+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":169,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.817715+0000","last_change":"2024-02-23T00:42:58.717788+0000","last_active":"2024-02-23T00:45:40.817715+0000","last_peered":"2024-02-23T00:45:40.817715+0000","last_clean":"2024-02-23T00:45:40.817715+0000","last_became_active":"2024-02-23T00:42:58.717371+0000","last_became_peered":"2024-02-23T00:42:58.717371+0000","last_unstale":"2024-02-23T00:45:40.817715+0000","last_undegraded":"2024-02-23T00:45:40.817715+0000","last_fullsized":"2024-02-23T00:45:40.817715+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":229,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.149587+0000","last_change":"2024-02-23T00:44:34.059194+0000","last_active":"2024-02-23T00:45:41.149587+0000","last_peered":"2024-02-23T00:45:41.149587+0000","last_clean":"2024-02-23T00:45:41.149587+0000","last_became_active":"2024-02-23T00:39:18.798829+0000","last_became_peered":"2024-02-23T00:39:18.798829+0000","last_unstale":"2024-02-23T00:45:41.149587+0000","last_undegraded":"2024-02-23T00:45:41.149587+0000","last_fullsized":"2024-02-23T00:45:41.149587+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:34.059129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:34.059129+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"251'40","reported_seq":304,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.134930+0000","last_change":"2024-02-23T00:45:23.910609+0000","last_active":"2024-02-23T00:45:41.134930+0000","last_peered":"2024-02-23T00:45:41.134930+0000","last_clean":"2024-02-23T00:45:41.134930+0000","last_became_active":"2024-02-23T00:39:16.859002+0000","last_became_peered":"2024-02-23T00:39:16.859002+0000","last_unstale":"2024-02-23T00:45:41.134930+0000","last_undegraded":"2024-02-23T00:45:41.134930+0000","last_fullsized":"2024-02-23T00:45:41.134930+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'40","last_scrub_stamp":"2024-02-23T00:45:23.910546+0000","last_deep_scrub":"251'40","last_deep_scrub_stamp":"2024-02-23T00:45:23.910546+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.910546+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":233,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.152712+0000","last_change":"2024-02-23T00:45:16.664785+0000","last_active":"2024-02-23T00:45:41.152712+0000","last_peered":"2024-02-23T00:45:41.152712+0000","last_clean":"2024-02-23T00:45:41.152712+0000","last_became_active":"2024-02-23T00:39:14.779193+0000","last_became_peered":"2024-02-23T00:39:14.779193+0000","last_unstale":"2024-02-23T00:45:41.152712+0000","last_undegraded":"2024-02-23T00:45:41.152712+0000","last_fullsized":"2024-02-23T00:45:41.152712+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.664682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.664682+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":227,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.148972+0000","last_change":"2024-02-23T00:45:03.228087+0000","last_active":"2024-02-23T00:45:41.148972+0000","last_peered":"2024-02-23T00:45:41.148972+0000","last_clean":"2024-02-23T00:45:41.148972+0000","last_became_active":"2024-02-23T00:39:20.795271+0000","last_became_peered":"2024-02-23T00:39:20.795271+0000","last_unstale":"2024-02-23T00:45:41.148972+0000","last_undegraded":"2024-02-23T00:45:41.148972+0000","last_fullsized":"2024-02-23T00:45:41.148972+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:03.227995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:45:03.227995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"110'2","reported_seq":175,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.134519+0000","last_change":"2024-02-23T00:45:10.465187+0000","last_active":"2024-02-23T00:45:40.134519+0000","last_peered":"2024-02-23T00:45:40.134519+0000","last_clean":"2024-02-23T00:45:40.134519+0000","last_became_active":"2024-02-23T00:42:58.814093+0000","last_became_peered":"2024-02-23T00:42:58.814093+0000","last_unstale":"2024-02-23T00:45:40.134519+0000","last_undegraded":"2024-02-23T00:45:40.134519+0000","last_fullsized":"2024-02-23T00:45:40.134519+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'2","last_scrub_stamp":"2024-02-23T00:45:10.465108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.465108+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":175,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.137542+0000","last_change":"2024-02-23T00:45:13.874437+0000","last_active":"2024-02-23T00:45:41.137542+0000","last_peered":"2024-02-23T00:45:41.137542+0000","last_clean":"2024-02-23T00:45:41.137542+0000","last_became_active":"2024-02-23T00:42:58.809283+0000","last_became_peered":"2024-02-23T00:42:58.809283+0000","last_unstale":"2024-02-23T00:45:41.137542+0000","last_undegraded":"2024-02-23T00:45:41.137542+0000","last_fullsized":"2024-02-23T00:45:41.137542+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:13.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:13.874373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":227,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.604830+0000","last_change":"2024-02-23T00:44:19.312509+0000","last_active":"2024-02-23T00:45:40.604830+0000","last_peered":"2024-02-23T00:45:40.604830+0000","last_clean":"2024-02-23T00:45:40.604830+0000","last_became_active":"2024-02-23T00:39:18.796664+0000","last_became_peered":"2024-02-23T00:39:18.796664+0000","last_unstale":"2024-02-23T00:45:40.604830+0000","last_undegraded":"2024-02-23T00:45:40.604830+0000","last_fullsized":"2024-02-23T00:45:40.604830+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:19.312418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:19.312418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"203'34","reported_seq":291,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.558103+0000","last_change":"2024-02-23T00:44:35.644033+0000","last_active":"2024-02-23T00:45:40.558103+0000","last_peered":"2024-02-23T00:45:40.558103+0000","last_clean":"2024-02-23T00:45:40.558103+0000","last_became_active":"2024-02-23T00:39:16.857278+0000","last_became_peered":"2024-02-23T00:39:16.857278+0000","last_unstale":"2024-02-23T00:45:40.558103+0000","last_undegraded":"2024-02-23T00:45:40.558103+0000","last_fullsized":"2024-02-23T00:45:40.558103+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'34","last_scrub_stamp":"2024-02-23T00:44:35.643931+0000","last_deep_scrub":"203'34","last_deep_scrub_stamp":"2024-02-23T00:44:35.643931+0000","last_clean_scrub_stamp":"2024-02-23T00:44:35.643931+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":230,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.654640+0000","last_change":"2024-02-23T00:45:12.730663+0000","last_active":"2024-02-23T00:45:40.654640+0000","last_peered":"2024-02-23T00:45:40.654640+0000","last_clean":"2024-02-23T00:45:40.654640+0000","last_became_active":"2024-02-23T00:39:14.783662+0000","last_became_peered":"2024-02-23T00:39:14.783662+0000","last_unstale":"2024-02-23T00:45:40.654640+0000","last_undegraded":"2024-02-23T00:45:40.654640+0000","last_fullsized":"2024-02-23T00:45:40.654640+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.730539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.730539+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":227,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.151891+0000","last_change":"2024-02-23T00:45:04.585266+0000","last_active":"2024-02-23T00:45:41.151891+0000","last_peered":"2024-02-23T00:45:41.151891+0000","last_clean":"2024-02-23T00:45:41.151891+0000","last_became_active":"2024-02-23T00:39:21.160060+0000","last_became_peered":"2024-02-23T00:39:21.160060+0000","last_unstale":"2024-02-23T00:45:41.151891+0000","last_undegraded":"2024-02-23T00:45:41.151891+0000","last_fullsized":"2024-02-23T00:45:41.151891+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:04.585179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.585179+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.8","version":"110'2","reported_seq":175,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.137214+0000","last_change":"2024-02-23T00:44:33.389680+0000","last_active":"2024-02-23T00:45:40.137214+0000","last_peered":"2024-02-23T00:45:40.137214+0000","last_clean":"2024-02-23T00:45:40.137214+0000","last_became_active":"2024-02-23T00:42:58.809584+0000","last_became_peered":"2024-02-23T00:42:58.809584+0000","last_unstale":"2024-02-23T00:45:40.137214+0000","last_undegraded":"2024-02-23T00:45:40.137214+0000","last_fullsized":"2024-02-23T00:45:40.137214+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'2","last_scrub_stamp":"2024-02-23T00:44:33.389605+0000","last_deep_scrub":"110'2","last_deep_scrub_stamp":"2024-02-23T00:44:33.389605+0000","last_clean_scrub_stamp":"2024-02-23T00:44:33.389605+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":174,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.817449+0000","last_change":"2024-02-23T00:45:18.185097+0000","last_active":"2024-02-23T00:45:40.817449+0000","last_peered":"2024-02-23T00:45:40.817449+0000","last_clean":"2024-02-23T00:45:40.817449+0000","last_became_active":"2024-02-23T00:42:58.726652+0000","last_became_peered":"2024-02-23T00:42:58.726652+0000","last_unstale":"2024-02-23T00:45:40.817449+0000","last_undegraded":"2024-02-23T00:45:40.817449+0000","last_fullsized":"2024-02-23T00:45:40.817449+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:18.184992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:18.184992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":229,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.817603+0000","last_change":"2024-02-23T00:43:45.054194+0000","last_active":"2024-02-23T00:45:40.817603+0000","last_peered":"2024-02-23T00:45:40.817603+0000","last_clean":"2024-02-23T00:45:40.817603+0000","last_became_active":"2024-02-23T00:39:18.800378+0000","last_became_peered":"2024-02-23T00:39:18.800378+0000","last_unstale":"2024-02-23T00:45:40.817603+0000","last_undegraded":"2024-02-23T00:45:40.817603+0000","last_fullsized":"2024-02-23T00:45:40.817603+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:45.054115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:59.694033+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.054115+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"95'18","reported_seq":255,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.558994+0000","last_change":"2024-02-23T00:44:18.630435+0000","last_active":"2024-02-23T00:45:40.558994+0000","last_peered":"2024-02-23T00:45:40.558994+0000","last_clean":"2024-02-23T00:45:40.558994+0000","last_became_active":"2024-02-23T00:39:16.787143+0000","last_became_peered":"2024-02-23T00:39:16.787143+0000","last_unstale":"2024-02-23T00:45:40.558994+0000","last_undegraded":"2024-02-23T00:45:40.558994+0000","last_fullsized":"2024-02-23T00:45:40.558994+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'18","last_scrub_stamp":"2024-02-23T00:44:18.630363+0000","last_deep_scrub":"95'18","last_deep_scrub_stamp":"2024-02-23T00:42:01.195545+0000","last_clean_scrub_stamp":"2024-02-23T00:44:18.630363+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":230,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.654298+0000","last_change":"2024-02-23T00:42:50.947420+0000","last_active":"2024-02-23T00:45:40.654298+0000","last_peered":"2024-02-23T00:45:40.654298+0000","last_clean":"2024-02-23T00:45:40.654298+0000","last_became_active":"2024-02-23T00:39:14.784578+0000","last_became_peered":"2024-02-23T00:39:14.784578+0000","last_unstale":"2024-02-23T00:45:40.654298+0000","last_undegraded":"2024-02-23T00:45:40.654298+0000","last_fullsized":"2024-02-23T00:45:40.654298+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:50.947349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:50.947349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":233,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.557629+0000","last_change":"2024-02-23T00:43:48.258338+0000","last_active":"2024-02-23T00:45:40.557629+0000","last_peered":"2024-02-23T00:45:40.557629+0000","last_clean":"2024-02-23T00:45:40.557629+0000","last_became_active":"2024-02-23T00:39:14.775446+0000","last_became_peered":"2024-02-23T00:39:14.775446+0000","last_unstale":"2024-02-23T00:45:40.557629+0000","last_undegraded":"2024-02-23T00:45:40.557629+0000","last_fullsized":"2024-02-23T00:45:40.557629+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:48.258269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.258269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":222,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.152672+0000","last_change":"2024-02-23T00:43:47.905617+0000","last_active":"2024-02-23T00:45:41.152672+0000","last_peered":"2024-02-23T00:45:41.152672+0000","last_clean":"2024-02-23T00:45:41.152672+0000","last_became_active":"2024-02-23T00:39:20.797242+0000","last_became_peered":"2024-02-23T00:39:20.797242+0000","last_unstale":"2024-02-23T00:45:41.152672+0000","last_undegraded":"2024-02-23T00:45:41.152672+0000","last_fullsized":"2024-02-23T00:45:41.152672+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:47.905501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:47.905501+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"110'5","reported_seq":178,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.654066+0000","last_change":"2024-02-23T00:45:26.760937+0000","last_active":"2024-02-23T00:45:40.654066+0000","last_peered":"2024-02-23T00:45:40.654066+0000","last_clean":"2024-02-23T00:45:40.654066+0000","last_became_active":"2024-02-23T00:42:58.742998+0000","last_became_peered":"2024-02-23T00:42:58.742998+0000","last_unstale":"2024-02-23T00:45:40.654066+0000","last_undegraded":"2024-02-23T00:45:40.654066+0000","last_fullsized":"2024-02-23T00:45:40.654066+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'5","last_scrub_stamp":"2024-02-23T00:45:26.760862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:26.760862+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":173,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.603212+0000","last_change":"2024-02-23T00:44:35.360128+0000","last_active":"2024-02-23T00:45:40.603212+0000","last_peered":"2024-02-23T00:45:40.603212+0000","last_clean":"2024-02-23T00:45:40.603212+0000","last_became_active":"2024-02-23T00:42:58.690858+0000","last_became_peered":"2024-02-23T00:42:58.690858+0000","last_unstale":"2024-02-23T00:45:40.603212+0000","last_undegraded":"2024-02-23T00:45:40.603212+0000","last_fullsized":"2024-02-23T00:45:40.603212+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:35.360054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:35.360054+0000","last_clean_scrub_stamp":"2024-02-23T00:44:35.360054+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":226,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.654362+0000","last_change":"2024-02-23T00:45:16.746109+0000","last_active":"2024-02-23T00:45:40.654362+0000","last_peered":"2024-02-23T00:45:40.654362+0000","last_clean":"2024-02-23T00:45:40.654362+0000","last_became_active":"2024-02-23T00:39:18.793801+0000","last_became_peered":"2024-02-23T00:39:18.793801+0000","last_unstale":"2024-02-23T00:45:40.654362+0000","last_undegraded":"2024-02-23T00:45:40.654362+0000","last_fullsized":"2024-02-23T00:45:40.654362+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.746013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.746013+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"67'14","reported_seq":251,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.604600+0000","last_change":"2024-02-23T00:43:44.369862+0000","last_active":"2024-02-23T00:45:40.604600+0000","last_peered":"2024-02-23T00:45:40.604600+0000","last_clean":"2024-02-23T00:45:40.604600+0000","last_became_active":"2024-02-23T00:39:16.858880+0000","last_became_peered":"2024-02-23T00:39:16.858880+0000","last_unstale":"2024-02-23T00:45:40.604600+0000","last_undegraded":"2024-02-23T00:45:40.604600+0000","last_fullsized":"2024-02-23T00:45:40.604600+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'14","last_scrub_stamp":"2024-02-23T00:43:44.369792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:44.369792+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":240,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.654410+0000","last_change":"2024-02-23T00:45:08.796153+0000","last_active":"2024-02-23T00:45:40.654410+0000","last_peered":"2024-02-23T00:45:40.654410+0000","last_clean":"2024-02-23T00:45:40.654410+0000","last_became_active":"2024-02-23T00:39:08.248249+0000","last_became_peered":"2024-02-23T00:39:08.248249+0000","last_unstale":"2024-02-23T00:45:40.654410+0000","last_undegraded":"2024-02-23T00:45:40.654410+0000","last_fullsized":"2024-02-23T00:45:40.654410+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.796076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:49.977248+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.796076+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":233,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.817194+0000","last_change":"2024-02-23T00:45:22.272078+0000","last_active":"2024-02-23T00:45:40.817194+0000","last_peered":"2024-02-23T00:45:40.817194+0000","last_clean":"2024-02-23T00:45:40.817194+0000","last_became_active":"2024-02-23T00:39:14.788888+0000","last_became_peered":"2024-02-23T00:39:14.788888+0000","last_unstale":"2024-02-23T00:45:40.817194+0000","last_undegraded":"2024-02-23T00:45:40.817194+0000","last_fullsized":"2024-02-23T00:45:40.817194+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:22.272003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:22.272003+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":234,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.138041+0000","last_change":"2024-02-23T00:44:33.143663+0000","last_active":"2024-02-23T00:45:41.138041+0000","last_peered":"2024-02-23T00:45:41.138041+0000","last_clean":"2024-02-23T00:45:41.138041+0000","last_became_active":"2024-02-23T00:39:14.783489+0000","last_became_peered":"2024-02-23T00:39:14.783489+0000","last_unstale":"2024-02-23T00:45:41.138041+0000","last_undegraded":"2024-02-23T00:45:41.138041+0000","last_fullsized":"2024-02-23T00:45:41.138041+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:33.143592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:33.143592+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":245,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.817720+0000","last_change":"2024-02-23T00:44:20.958888+0000","last_active":"2024-02-23T00:45:40.817720+0000","last_peered":"2024-02-23T00:45:40.817720+0000","last_clean":"2024-02-23T00:45:40.817720+0000","last_became_active":"2024-02-23T00:43:42.168435+0000","last_became_peered":"2024-02-23T00:43:42.168435+0000","last_unstale":"2024-02-23T00:45:40.817720+0000","last_undegraded":"2024-02-23T00:45:40.817720+0000","last_fullsized":"2024-02-23T00:45:40.817720+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:20.958797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:20.958797+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":175,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.136071+0000","last_change":"2024-02-23T00:45:21.383725+0000","last_active":"2024-02-23T00:45:40.136071+0000","last_peered":"2024-02-23T00:45:40.136071+0000","last_clean":"2024-02-23T00:45:40.136071+0000","last_became_active":"2024-02-23T00:42:58.721439+0000","last_became_peered":"2024-02-23T00:42:58.721439+0000","last_unstale":"2024-02-23T00:45:40.136071+0000","last_undegraded":"2024-02-23T00:45:40.136071+0000","last_fullsized":"2024-02-23T00:45:40.136071+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:21.383626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:21.383626+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":173,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.654772+0000","last_change":"2024-02-23T00:45:17.763618+0000","last_active":"2024-02-23T00:45:40.654772+0000","last_peered":"2024-02-23T00:45:40.654772+0000","last_clean":"2024-02-23T00:45:40.654772+0000","last_became_active":"2024-02-23T00:42:58.701502+0000","last_became_peered":"2024-02-23T00:42:58.701502+0000","last_unstale":"2024-02-23T00:45:40.654772+0000","last_undegraded":"2024-02-23T00:45:40.654772+0000","last_fullsized":"2024-02-23T00:45:40.654772+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:17.763482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:17.763482+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":227,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.136483+0000","last_change":"2024-02-23T00:43:46.293962+0000","last_active":"2024-02-23T00:45:40.136483+0000","last_peered":"2024-02-23T00:45:40.136483+0000","last_clean":"2024-02-23T00:45:40.136483+0000","last_became_active":"2024-02-23T00:39:18.789330+0000","last_became_peered":"2024-02-23T00:39:18.789330+0000","last_unstale":"2024-02-23T00:45:40.136483+0000","last_undegraded":"2024-02-23T00:45:40.136483+0000","last_fullsized":"2024-02-23T00:45:40.136483+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:46.293864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.264240+0000","last_clean_scrub_stamp":"2024-02-23T00:43:46.293864+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"68'19","reported_seq":273,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.152831+0000","last_change":"2024-02-23T00:45:07.697475+0000","last_active":"2024-02-23T00:45:41.152831+0000","last_peered":"2024-02-23T00:45:41.152831+0000","last_clean":"2024-02-23T00:45:41.152831+0000","last_became_active":"2024-02-23T00:39:16.787484+0000","last_became_peered":"2024-02-23T00:39:16.787484+0000","last_unstale":"2024-02-23T00:45:41.152831+0000","last_undegraded":"2024-02-23T00:45:41.152831+0000","last_fullsized":"2024-02-23T00:45:41.152831+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'19","last_scrub_stamp":"2024-02-23T00:45:07.697394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:07.697394+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":231,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.134877+0000","last_change":"2024-02-23T00:43:54.305009+0000","last_active":"2024-02-23T00:45:40.134877+0000","last_peered":"2024-02-23T00:45:40.134877+0000","last_clean":"2024-02-23T00:45:40.134877+0000","last_became_active":"2024-02-23T00:39:14.772411+0000","last_became_peered":"2024-02-23T00:39:14.772411+0000","last_unstale":"2024-02-23T00:45:40.134877+0000","last_undegraded":"2024-02-23T00:45:40.134877+0000","last_fullsized":"2024-02-23T00:45:40.134877+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:54.304948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:54.304948+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"68'5","reported_seq":610,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.675538+0000","last_change":"2024-02-23T00:43:48.876428+0000","last_active":"2024-02-23T00:45:41.675538+0000","last_peered":"2024-02-23T00:45:41.675538+0000","last_clean":"2024-02-23T00:45:41.675538+0000","last_became_active":"2024-02-23T00:39:14.779109+0000","last_became_peered":"2024-02-23T00:39:14.779109+0000","last_unstale":"2024-02-23T00:45:41.675538+0000","last_undegraded":"2024-02-23T00:45:41.675538+0000","last_fullsized":"2024-02-23T00:45:41.675538+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'5","last_scrub_stamp":"2024-02-23T00:43:48.876356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.876356+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":379,"num_read_kb":374,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":224,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.654531+0000","last_change":"2024-02-23T00:43:47.794616+0000","last_active":"2024-02-23T00:45:40.654531+0000","last_peered":"2024-02-23T00:45:40.654531+0000","last_clean":"2024-02-23T00:45:40.654531+0000","last_became_active":"2024-02-23T00:39:21.160457+0000","last_became_peered":"2024-02-23T00:39:21.160457+0000","last_unstale":"2024-02-23T00:45:40.654531+0000","last_undegraded":"2024-02-23T00:45:40.654531+0000","last_fullsized":"2024-02-23T00:45:40.654531+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:47.794534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:47.794534+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":169,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.816817+0000","last_change":"2024-02-23T00:42:58.831449+0000","last_active":"2024-02-23T00:45:40.816817+0000","last_peered":"2024-02-23T00:45:40.816817+0000","last_clean":"2024-02-23T00:45:40.816817+0000","last_became_active":"2024-02-23T00:42:58.830815+0000","last_became_peered":"2024-02-23T00:42:58.830815+0000","last_unstale":"2024-02-23T00:45:40.816817+0000","last_undegraded":"2024-02-23T00:45:40.816817+0000","last_fullsized":"2024-02-23T00:45:40.816817+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":170,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.151593+0000","last_change":"2024-02-23T00:42:57.653669+0000","last_active":"2024-02-23T00:45:41.151593+0000","last_peered":"2024-02-23T00:45:41.151593+0000","last_clean":"2024-02-23T00:45:41.151593+0000","last_became_active":"2024-02-23T00:42:57.653355+0000","last_became_peered":"2024-02-23T00:42:57.653355+0000","last_unstale":"2024-02-23T00:45:41.151593+0000","last_undegraded":"2024-02-23T00:45:41.151593+0000","last_fullsized":"2024-02-23T00:45:41.151593+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":229,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.559311+0000","last_change":"2024-02-23T00:44:20.626116+0000","last_active":"2024-02-23T00:45:40.559311+0000","last_peered":"2024-02-23T00:45:40.559311+0000","last_clean":"2024-02-23T00:45:40.559311+0000","last_became_active":"2024-02-23T00:39:18.802134+0000","last_became_peered":"2024-02-23T00:39:18.802134+0000","last_unstale":"2024-02-23T00:45:40.559311+0000","last_undegraded":"2024-02-23T00:45:40.559311+0000","last_fullsized":"2024-02-23T00:45:40.559311+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:20.626052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:20.626052+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"86'35","reported_seq":287,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.816906+0000","last_change":"2024-02-23T00:42:47.988747+0000","last_active":"2024-02-23T00:45:40.816906+0000","last_peered":"2024-02-23T00:45:40.816906+0000","last_clean":"2024-02-23T00:45:40.816906+0000","last_became_active":"2024-02-23T00:40:42.933322+0000","last_became_peered":"2024-02-23T00:40:42.933322+0000","last_unstale":"2024-02-23T00:45:40.816906+0000","last_undegraded":"2024-02-23T00:45:40.816906+0000","last_fullsized":"2024-02-23T00:45:40.816906+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'35","last_scrub_stamp":"2024-02-23T00:42:47.988676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:47.988676+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":233,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.816800+0000","last_change":"2024-02-23T00:44:23.965228+0000","last_active":"2024-02-23T00:45:40.816800+0000","last_peered":"2024-02-23T00:45:40.816800+0000","last_clean":"2024-02-23T00:45:40.816800+0000","last_became_active":"2024-02-23T00:39:14.784015+0000","last_became_peered":"2024-02-23T00:39:14.784015+0000","last_unstale":"2024-02-23T00:45:40.816800+0000","last_undegraded":"2024-02-23T00:45:40.816800+0000","last_fullsized":"2024-02-23T00:45:40.816800+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:23.965125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.965125+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":241,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.137399+0000","last_change":"2024-02-23T00:43:45.144499+0000","last_active":"2024-02-23T00:45:41.137399+0000","last_peered":"2024-02-23T00:45:41.137399+0000","last_clean":"2024-02-23T00:45:41.137399+0000","last_became_active":"2024-02-23T00:43:45.144043+0000","last_became_peered":"2024-02-23T00:43:45.144043+0000","last_unstale":"2024-02-23T00:45:41.137399+0000","last_undegraded":"2024-02-23T00:45:41.137399+0000","last_fullsized":"2024-02-23T00:45:41.137399+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:40.062493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:40.062493+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":173,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.605321+0000","last_change":"2024-02-23T00:44:31.410733+0000","last_active":"2024-02-23T00:45:40.605321+0000","last_peered":"2024-02-23T00:45:40.605321+0000","last_clean":"2024-02-23T00:45:40.605321+0000","last_became_active":"2024-02-23T00:42:58.754813+0000","last_became_peered":"2024-02-23T00:42:58.754813+0000","last_unstale":"2024-02-23T00:45:40.605321+0000","last_undegraded":"2024-02-23T00:45:40.605321+0000","last_fullsized":"2024-02-23T00:45:40.605321+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:31.410670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:31.410670+0000","last_clean_scrub_stamp":"2024-02-23T00:44:31.410670+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":173,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.137604+0000","last_change":"2024-02-23T00:45:05.490595+0000","last_active":"2024-02-23T00:45:40.137604+0000","last_peered":"2024-02-23T00:45:40.137604+0000","last_clean":"2024-02-23T00:45:40.137604+0000","last_became_active":"2024-02-23T00:42:58.766579+0000","last_became_peered":"2024-02-23T00:42:58.766579+0000","last_unstale":"2024-02-23T00:45:40.137604+0000","last_undegraded":"2024-02-23T00:45:40.137604+0000","last_fullsized":"2024-02-23T00:45:40.137604+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:05.490531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:05.490531+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"91'9","reported_seq":315,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.558312+0000","last_change":"2024-02-23T00:44:28.619036+0000","last_active":"2024-02-23T00:45:40.558312+0000","last_peered":"2024-02-23T00:45:40.558312+0000","last_clean":"2024-02-23T00:45:40.558312+0000","last_became_active":"2024-02-23T00:39:18.801297+0000","last_became_peered":"2024-02-23T00:39:18.801297+0000","last_unstale":"2024-02-23T00:45:40.558312+0000","last_undegraded":"2024-02-23T00:45:40.558312+0000","last_fullsized":"2024-02-23T00:45:40.558312+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'9","last_scrub_stamp":"2024-02-23T00:44:28.618923+0000","last_deep_scrub":"91'9","last_deep_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_clean_scrub_stamp":"2024-02-23T00:44:28.618923+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"68'16","reported_seq":261,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.152121+0000","last_change":"2024-02-23T00:45:11.713800+0000","last_active":"2024-02-23T00:45:41.152121+0000","last_peered":"2024-02-23T00:45:41.152121+0000","last_clean":"2024-02-23T00:45:41.152121+0000","last_became_active":"2024-02-23T00:39:16.787468+0000","last_became_peered":"2024-02-23T00:39:16.787468+0000","last_unstale":"2024-02-23T00:45:41.152121+0000","last_undegraded":"2024-02-23T00:45:41.152121+0000","last_fullsized":"2024-02-23T00:45:41.152121+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:45:11.713731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:11.713731+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":233,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.152380+0000","last_change":"2024-02-23T00:45:19.635062+0000","last_active":"2024-02-23T00:45:41.152380+0000","last_peered":"2024-02-23T00:45:41.152380+0000","last_clean":"2024-02-23T00:45:41.152380+0000","last_became_active":"2024-02-23T00:39:14.779952+0000","last_became_peered":"2024-02-23T00:39:14.779952+0000","last_unstale":"2024-02-23T00:45:41.152380+0000","last_undegraded":"2024-02-23T00:45:41.152380+0000","last_fullsized":"2024-02-23T00:45:41.152380+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:19.635002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:19.635002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"107'2","reported_seq":229,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.557528+0000","last_change":"2024-02-23T00:45:08.662754+0000","last_active":"2024-02-23T00:45:40.557528+0000","last_peered":"2024-02-23T00:45:40.557528+0000","last_clean":"2024-02-23T00:45:40.557528+0000","last_became_active":"2024-02-23T00:39:20.800858+0000","last_became_peered":"2024-02-23T00:39:20.800858+0000","last_unstale":"2024-02-23T00:45:40.557528+0000","last_undegraded":"2024-02-23T00:45:40.557528+0000","last_fullsized":"2024-02-23T00:45:40.557528+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'2","last_scrub_stamp":"2024-02-23T00:45:08.662667+0000","last_deep_scrub":"107'2","last_deep_scrub_stamp":"2024-02-23T00:42:48.362739+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.662667+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":173,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.604348+0000","last_change":"2024-02-23T00:45:19.711196+0000","last_active":"2024-02-23T00:45:40.604348+0000","last_peered":"2024-02-23T00:45:40.604348+0000","last_clean":"2024-02-23T00:45:40.604348+0000","last_became_active":"2024-02-23T00:42:58.726861+0000","last_became_peered":"2024-02-23T00:42:58.726861+0000","last_unstale":"2024-02-23T00:45:40.604348+0000","last_undegraded":"2024-02-23T00:45:40.604348+0000","last_fullsized":"2024-02-23T00:45:40.604348+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:19.711117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:19.711117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":173,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.604465+0000","last_change":"2024-02-23T00:45:04.621794+0000","last_active":"2024-02-23T00:45:40.604465+0000","last_peered":"2024-02-23T00:45:40.604465+0000","last_clean":"2024-02-23T00:45:40.604465+0000","last_became_active":"2024-02-23T00:42:58.664282+0000","last_became_peered":"2024-02-23T00:42:58.664282+0000","last_unstale":"2024-02-23T00:45:40.604465+0000","last_undegraded":"2024-02-23T00:45:40.604465+0000","last_fullsized":"2024-02-23T00:45:40.604465+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:04.621714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.621714+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":227,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.137533+0000","last_change":"2024-02-23T00:43:51.261426+0000","last_active":"2024-02-23T00:45:40.137533+0000","last_peered":"2024-02-23T00:45:40.137533+0000","last_clean":"2024-02-23T00:45:40.137533+0000","last_became_active":"2024-02-23T00:39:18.796323+0000","last_became_peered":"2024-02-23T00:39:18.796323+0000","last_unstale":"2024-02-23T00:45:40.137533+0000","last_undegraded":"2024-02-23T00:45:40.137533+0000","last_fullsized":"2024-02-23T00:45:40.137533+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:51.261367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_clean_scrub_stamp":"2024-02-23T00:43:51.261367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"68'25","reported_seq":280,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.149404+0000","last_change":"2024-02-23T00:43:46.158231+0000","last_active":"2024-02-23T00:45:41.149404+0000","last_peered":"2024-02-23T00:45:41.149404+0000","last_clean":"2024-02-23T00:45:41.149404+0000","last_became_active":"2024-02-23T00:39:16.783926+0000","last_became_peered":"2024-02-23T00:39:16.783926+0000","last_unstale":"2024-02-23T00:45:41.149404+0000","last_undegraded":"2024-02-23T00:45:41.149404+0000","last_fullsized":"2024-02-23T00:45:41.149404+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'25","last_scrub_stamp":"2024-02-23T00:43:46.158149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:46.158149+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"95'4","reported_seq":248,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.149692+0000","last_change":"2024-02-23T00:44:31.106105+0000","last_active":"2024-02-23T00:45:41.149692+0000","last_peered":"2024-02-23T00:45:41.149692+0000","last_clean":"2024-02-23T00:45:41.149692+0000","last_became_active":"2024-02-23T00:39:14.779205+0000","last_became_peered":"2024-02-23T00:39:14.779205+0000","last_unstale":"2024-02-23T00:45:41.149692+0000","last_undegraded":"2024-02-23T00:45:41.149692+0000","last_fullsized":"2024-02-23T00:45:41.149692+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'4","last_scrub_stamp":"2024-02-23T00:44:31.106012+0000","last_deep_scrub":"95'4","last_deep_scrub_stamp":"2024-02-23T00:41:43.874397+0000","last_clean_scrub_stamp":"2024-02-23T00:44:31.106012+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":246,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.136666+0000","last_change":"2024-02-23T00:43:52.036315+0000","last_active":"2024-02-23T00:45:41.136666+0000","last_peered":"2024-02-23T00:45:41.136666+0000","last_clean":"2024-02-23T00:45:41.136666+0000","last_became_active":"2024-02-23T00:43:27.611660+0000","last_became_peered":"2024-02-23T00:43:27.611660+0000","last_unstale":"2024-02-23T00:45:41.136666+0000","last_undegraded":"2024-02-23T00:45:41.136666+0000","last_fullsized":"2024-02-23T00:45:41.136666+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:52.036249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:52.036249+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":173,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.137679+0000","last_change":"2024-02-23T00:45:16.389060+0000","last_active":"2024-02-23T00:45:40.137679+0000","last_peered":"2024-02-23T00:45:40.137679+0000","last_clean":"2024-02-23T00:45:40.137679+0000","last_became_active":"2024-02-23T00:42:58.809926+0000","last_became_peered":"2024-02-23T00:42:58.809926+0000","last_unstale":"2024-02-23T00:45:40.137679+0000","last_undegraded":"2024-02-23T00:45:40.137679+0000","last_fullsized":"2024-02-23T00:45:40.137679+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.388988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.388988+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":173,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.654607+0000","last_change":"2024-02-23T00:45:20.829124+0000","last_active":"2024-02-23T00:45:40.654607+0000","last_peered":"2024-02-23T00:45:40.654607+0000","last_clean":"2024-02-23T00:45:40.654607+0000","last_became_active":"2024-02-23T00:42:58.744983+0000","last_became_peered":"2024-02-23T00:42:58.744983+0000","last_unstale":"2024-02-23T00:45:40.654607+0000","last_undegraded":"2024-02-23T00:45:40.654607+0000","last_fullsized":"2024-02-23T00:45:40.654607+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:20.829052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:20.829052+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"68'21","reported_seq":272,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.559662+0000","last_change":"2024-02-23T00:43:52.309599+0000","last_active":"2024-02-23T00:45:40.559662+0000","last_peered":"2024-02-23T00:45:40.559662+0000","last_clean":"2024-02-23T00:45:40.559662+0000","last_became_active":"2024-02-23T00:39:16.858760+0000","last_became_peered":"2024-02-23T00:39:16.858760+0000","last_unstale":"2024-02-23T00:45:40.559662+0000","last_undegraded":"2024-02-23T00:45:40.559662+0000","last_fullsized":"2024-02-23T00:45:40.559662+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'21","last_scrub_stamp":"2024-02-23T00:43:52.309532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:52.309532+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":236,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.137459+0000","last_change":"2024-02-23T00:45:22.376128+0000","last_active":"2024-02-23T00:45:40.137459+0000","last_peered":"2024-02-23T00:45:40.137459+0000","last_clean":"2024-02-23T00:45:40.137459+0000","last_became_active":"2024-02-23T00:39:14.774244+0000","last_became_peered":"2024-02-23T00:39:14.774244+0000","last_unstale":"2024-02-23T00:45:40.137459+0000","last_undegraded":"2024-02-23T00:45:40.137459+0000","last_fullsized":"2024-02-23T00:45:40.137459+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:22.376063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:22.376063+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":230,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.136484+0000","last_change":"2024-02-23T00:43:44.951800+0000","last_active":"2024-02-23T00:45:41.136484+0000","last_peered":"2024-02-23T00:45:41.136484+0000","last_clean":"2024-02-23T00:45:41.136484+0000","last_became_active":"2024-02-23T00:39:18.798624+0000","last_became_peered":"2024-02-23T00:39:18.798624+0000","last_unstale":"2024-02-23T00:45:41.136484+0000","last_undegraded":"2024-02-23T00:45:41.136484+0000","last_fullsized":"2024-02-23T00:45:41.136484+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:44.951696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:46.998875+0000","last_clean_scrub_stamp":"2024-02-23T00:43:44.951696+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":225,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.135812+0000","last_change":"2024-02-23T00:44:26.432766+0000","last_active":"2024-02-23T00:45:40.135812+0000","last_peered":"2024-02-23T00:45:40.135812+0000","last_clean":"2024-02-23T00:45:40.135812+0000","last_became_active":"2024-02-23T00:39:20.796041+0000","last_became_peered":"2024-02-23T00:39:20.796041+0000","last_unstale":"2024-02-23T00:45:40.135812+0000","last_undegraded":"2024-02-23T00:45:40.135812+0000","last_fullsized":"2024-02-23T00:45:40.135812+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:26.432703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:26.432703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1d","version":"110'7","reported_seq":182,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.149875+0000","last_change":"2024-02-23T00:45:12.307598+0000","last_active":"2024-02-23T00:45:41.149875+0000","last_peered":"2024-02-23T00:45:41.149875+0000","last_clean":"2024-02-23T00:45:41.149875+0000","last_became_active":"2024-02-23T00:42:58.754763+0000","last_became_peered":"2024-02-23T00:42:58.754763+0000","last_unstale":"2024-02-23T00:45:41.149875+0000","last_undegraded":"2024-02-23T00:45:41.149875+0000","last_fullsized":"2024-02-23T00:45:41.149875+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'7","last_scrub_stamp":"2024-02-23T00:45:12.307531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.307531+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":174,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.557696+0000","last_change":"2024-02-23T00:44:26.595443+0000","last_active":"2024-02-23T00:45:40.557696+0000","last_peered":"2024-02-23T00:45:40.557696+0000","last_clean":"2024-02-23T00:45:40.557696+0000","last_became_active":"2024-02-23T00:42:58.825015+0000","last_became_peered":"2024-02-23T00:42:58.825015+0000","last_unstale":"2024-02-23T00:45:40.557696+0000","last_undegraded":"2024-02-23T00:45:40.557696+0000","last_fullsized":"2024-02-23T00:45:40.557696+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:26.595346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:44:26.595346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"107'16","reported_seq":260,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.136398+0000","last_change":"2024-02-23T00:45:07.012086+0000","last_active":"2024-02-23T00:45:41.136398+0000","last_peered":"2024-02-23T00:45:41.136398+0000","last_clean":"2024-02-23T00:45:41.136398+0000","last_became_active":"2024-02-23T00:39:16.858449+0000","last_became_peered":"2024-02-23T00:39:16.858449+0000","last_unstale":"2024-02-23T00:45:41.136398+0000","last_undegraded":"2024-02-23T00:45:41.136398+0000","last_fullsized":"2024-02-23T00:45:41.136398+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'16","last_scrub_stamp":"2024-02-23T00:45:07.011956+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-02-23T00:42:52.964510+0000","last_clean_scrub_stamp":"2024-02-23T00:45:07.011956+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":231,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.604410+0000","last_change":"2024-02-23T00:45:13.613736+0000","last_active":"2024-02-23T00:45:40.604410+0000","last_peered":"2024-02-23T00:45:40.604410+0000","last_clean":"2024-02-23T00:45:40.604410+0000","last_became_active":"2024-02-23T00:39:14.783164+0000","last_became_peered":"2024-02-23T00:39:14.783164+0000","last_unstale":"2024-02-23T00:45:40.604410+0000","last_undegraded":"2024-02-23T00:45:40.604410+0000","last_fullsized":"2024-02-23T00:45:40.604410+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:13.613654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:13.613654+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":229,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.817174+0000","last_change":"2024-02-23T00:43:40.971316+0000","last_active":"2024-02-23T00:45:40.817174+0000","last_peered":"2024-02-23T00:45:40.817174+0000","last_clean":"2024-02-23T00:45:40.817174+0000","last_became_active":"2024-02-23T00:39:18.800297+0000","last_became_peered":"2024-02-23T00:39:18.800297+0000","last_unstale":"2024-02-23T00:45:40.817174+0000","last_undegraded":"2024-02-23T00:45:40.817174+0000","last_fullsized":"2024-02-23T00:45:40.817174+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:40.971243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:40.971243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":224,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.654140+0000","last_change":"2024-02-23T00:43:46.760431+0000","last_active":"2024-02-23T00:45:40.654140+0000","last_peered":"2024-02-23T00:45:40.654140+0000","last_clean":"2024-02-23T00:45:40.654140+0000","last_became_active":"2024-02-23T00:39:21.162036+0000","last_became_peered":"2024-02-23T00:39:21.162036+0000","last_unstale":"2024-02-23T00:45:40.654140+0000","last_undegraded":"2024-02-23T00:45:40.654140+0000","last_fullsized":"2024-02-23T00:45:40.654140+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:46.760348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:46.760348+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":175,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.152526+0000","last_change":"2024-02-23T00:45:18.644663+0000","last_active":"2024-02-23T00:45:41.152526+0000","last_peered":"2024-02-23T00:45:41.152526+0000","last_clean":"2024-02-23T00:45:41.152526+0000","last_became_active":"2024-02-23T00:42:57.648267+0000","last_became_peered":"2024-02-23T00:42:57.648267+0000","last_unstale":"2024-02-23T00:45:41.152526+0000","last_undegraded":"2024-02-23T00:45:41.152526+0000","last_fullsized":"2024-02-23T00:45:41.152526+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:18.644538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:18.644538+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1","version":"110'7","reported_seq":188,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.137525+0000","last_change":"2024-02-23T00:45:18.947859+0000","last_active":"2024-02-23T00:45:41.137525+0000","last_peered":"2024-02-23T00:45:41.137525+0000","last_clean":"2024-02-23T00:45:41.137525+0000","last_became_active":"2024-02-23T00:42:58.777251+0000","last_became_peered":"2024-02-23T00:42:58.777251+0000","last_unstale":"2024-02-23T00:45:41.137525+0000","last_undegraded":"2024-02-23T00:45:41.137525+0000","last_fullsized":"2024-02-23T00:45:41.137525+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'7","last_scrub_stamp":"2024-02-23T00:45:18.947746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:18.947746+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"68'22","reported_seq":199,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.135609+0000","last_change":"2024-02-23T00:44:31.391929+0000","last_active":"2024-02-23T00:45:40.135609+0000","last_peered":"2024-02-23T00:45:40.135609+0000","last_clean":"2024-02-23T00:45:40.135609+0000","last_became_active":"2024-02-23T00:42:43.521189+0000","last_became_peered":"2024-02-23T00:42:43.521189+0000","last_unstale":"2024-02-23T00:45:40.135609+0000","last_undegraded":"2024-02-23T00:45:40.135609+0000","last_fullsized":"2024-02-23T00:45:40.135609+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'22","last_scrub_stamp":"2024-02-23T00:44:31.391819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:44:31.391819+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":37,"num_read_kb":30,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,7],"acting":[2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":233,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.816682+0000","last_change":"2024-02-23T00:44:18.946624+0000","last_active":"2024-02-23T00:45:40.816682+0000","last_peered":"2024-02-23T00:45:40.816682+0000","last_clean":"2024-02-23T00:45:40.816682+0000","last_became_active":"2024-02-23T00:39:14.785590+0000","last_became_peered":"2024-02-23T00:39:14.785590+0000","last_unstale":"2024-02-23T00:45:40.816682+0000","last_undegraded":"2024-02-23T00:45:40.816682+0000","last_fullsized":"2024-02-23T00:45:40.816682+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:18.946560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_clean_scrub_stamp":"2024-02-23T00:44:18.946560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"67'8","reported_seq":314,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.817771+0000","last_change":"2024-02-23T00:44:29.038226+0000","last_active":"2024-02-23T00:45:40.817771+0000","last_peered":"2024-02-23T00:45:40.817771+0000","last_clean":"2024-02-23T00:45:40.817771+0000","last_became_active":"2024-02-23T00:39:18.803566+0000","last_became_peered":"2024-02-23T00:39:18.803566+0000","last_unstale":"2024-02-23T00:45:40.817771+0000","last_undegraded":"2024-02-23T00:45:40.817771+0000","last_fullsized":"2024-02-23T00:45:40.817771+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:44:29.038134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:29.038134+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"254'6","reported_seq":181,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.136757+0000","last_change":"2024-02-23T00:45:26.935006+0000","last_active":"2024-02-23T00:45:41.136757+0000","last_peered":"2024-02-23T00:45:41.136757+0000","last_clean":"2024-02-23T00:45:41.136757+0000","last_became_active":"2024-02-23T00:42:58.758349+0000","last_became_peered":"2024-02-23T00:42:58.758349+0000","last_unstale":"2024-02-23T00:45:41.136757+0000","last_undegraded":"2024-02-23T00:45:41.136757+0000","last_fullsized":"2024-02-23T00:45:41.136757+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"254'6","last_scrub_stamp":"2024-02-23T00:45:26.934947+0000","last_deep_scrub":"254'6","last_deep_scrub_stamp":"2024-02-23T00:45:26.934947+0000","last_clean_scrub_stamp":"2024-02-23T00:45:26.934947+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":168,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.655049+0000","last_change":"2024-02-23T00:42:58.766295+0000","last_active":"2024-02-23T00:45:40.655049+0000","last_peered":"2024-02-23T00:45:40.655049+0000","last_clean":"2024-02-23T00:45:40.655049+0000","last_became_active":"2024-02-23T00:42:58.757424+0000","last_became_peered":"2024-02-23T00:42:58.757424+0000","last_unstale":"2024-02-23T00:45:40.655049+0000","last_undegraded":"2024-02-23T00:45:40.655049+0000","last_fullsized":"2024-02-23T00:45:40.655049+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"68'15","reported_seq":262,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.817628+0000","last_change":"2024-02-23T00:44:25.998879+0000","last_active":"2024-02-23T00:45:40.817628+0000","last_peered":"2024-02-23T00:45:40.817628+0000","last_clean":"2024-02-23T00:45:40.817628+0000","last_became_active":"2024-02-23T00:39:16.858953+0000","last_became_peered":"2024-02-23T00:39:16.858953+0000","last_unstale":"2024-02-23T00:45:40.817628+0000","last_undegraded":"2024-02-23T00:45:40.817628+0000","last_fullsized":"2024-02-23T00:45:40.817628+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:44:25.998804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:44:25.998804+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":233,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.558136+0000","last_change":"2024-02-23T00:43:45.309194+0000","last_active":"2024-02-23T00:45:40.558136+0000","last_peered":"2024-02-23T00:45:40.558136+0000","last_clean":"2024-02-23T00:45:40.558136+0000","last_became_active":"2024-02-23T00:39:14.773772+0000","last_became_peered":"2024-02-23T00:39:14.773772+0000","last_unstale":"2024-02-23T00:45:40.558136+0000","last_undegraded":"2024-02-23T00:45:40.558136+0000","last_fullsized":"2024-02-23T00:45:40.558136+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:45.309083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.309083+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"67'8","reported_seq":310,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.135793+0000","last_change":"2024-02-23T00:43:39.086199+0000","last_active":"2024-02-23T00:45:41.135793+0000","last_peered":"2024-02-23T00:45:41.135793+0000","last_clean":"2024-02-23T00:45:41.135793+0000","last_became_active":"2024-02-23T00:39:18.787313+0000","last_became_peered":"2024-02-23T00:39:18.787313+0000","last_unstale":"2024-02-23T00:45:41.135793+0000","last_undegraded":"2024-02-23T00:45:41.135793+0000","last_fullsized":"2024-02-23T00:45:41.135793+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:43:39.086107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:39.086107+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":173,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.135990+0000","last_change":"2024-02-23T00:44:34.437892+0000","last_active":"2024-02-23T00:45:40.135990+0000","last_peered":"2024-02-23T00:45:40.135990+0000","last_clean":"2024-02-23T00:45:40.135990+0000","last_became_active":"2024-02-23T00:42:58.745471+0000","last_became_peered":"2024-02-23T00:42:58.745471+0000","last_unstale":"2024-02-23T00:45:40.135990+0000","last_undegraded":"2024-02-23T00:45:40.135990+0000","last_fullsized":"2024-02-23T00:45:40.135990+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:34.437785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:34.437785+0000","last_clean_scrub_stamp":"2024-02-23T00:44:34.437785+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":170,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.152102+0000","last_change":"2024-02-23T00:42:57.652853+0000","last_active":"2024-02-23T00:45:41.152102+0000","last_peered":"2024-02-23T00:45:41.152102+0000","last_clean":"2024-02-23T00:45:41.152102+0000","last_became_active":"2024-02-23T00:42:57.652488+0000","last_became_peered":"2024-02-23T00:42:57.652488+0000","last_unstale":"2024-02-23T00:45:41.152102+0000","last_undegraded":"2024-02-23T00:45:41.152102+0000","last_fullsized":"2024-02-23T00:45:41.152102+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"67'9","reported_seq":242,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.152034+0000","last_change":"2024-02-23T00:42:57.705378+0000","last_active":"2024-02-23T00:45:41.152034+0000","last_peered":"2024-02-23T00:45:41.152034+0000","last_clean":"2024-02-23T00:45:41.152034+0000","last_became_active":"2024-02-23T00:39:16.786936+0000","last_became_peered":"2024-02-23T00:39:16.786936+0000","last_unstale":"2024-02-23T00:45:41.152034+0000","last_undegraded":"2024-02-23T00:45:41.152034+0000","last_fullsized":"2024-02-23T00:45:41.152034+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'9","last_scrub_stamp":"2024-02-23T00:42:57.705261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.705261+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":236,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.604205+0000","last_change":"2024-02-23T00:44:24.350861+0000","last_active":"2024-02-23T00:45:40.604205+0000","last_peered":"2024-02-23T00:45:40.604205+0000","last_clean":"2024-02-23T00:45:40.604205+0000","last_became_active":"2024-02-23T00:39:14.778455+0000","last_became_peered":"2024-02-23T00:39:14.778455+0000","last_unstale":"2024-02-23T00:45:40.604205+0000","last_undegraded":"2024-02-23T00:45:40.604205+0000","last_fullsized":"2024-02-23T00:45:40.604205+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:24.350797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:24.350797+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":230,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.136544+0000","last_change":"2024-02-23T00:44:37.105079+0000","last_active":"2024-02-23T00:45:41.136544+0000","last_peered":"2024-02-23T00:45:41.136544+0000","last_clean":"2024-02-23T00:45:41.136544+0000","last_became_active":"2024-02-23T00:39:18.799052+0000","last_became_peered":"2024-02-23T00:39:18.799052+0000","last_unstale":"2024-02-23T00:45:41.136544+0000","last_undegraded":"2024-02-23T00:45:41.136544+0000","last_fullsized":"2024-02-23T00:45:41.136544+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:37.105013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_clean_scrub_stamp":"2024-02-23T00:44:37.105013+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":222,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.557456+0000","last_change":"2024-02-23T00:42:50.425748+0000","last_active":"2024-02-23T00:45:40.557456+0000","last_peered":"2024-02-23T00:45:40.557456+0000","last_clean":"2024-02-23T00:45:40.557456+0000","last_became_active":"2024-02-23T00:39:20.802648+0000","last_became_peered":"2024-02-23T00:39:20.802648+0000","last_unstale":"2024-02-23T00:45:40.557456+0000","last_undegraded":"2024-02-23T00:45:40.557456+0000","last_fullsized":"2024-02-23T00:45:40.557456+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:50.425641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:42:50.425641+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":175,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.148812+0000","last_change":"2024-02-23T00:45:15.352093+0000","last_active":"2024-02-23T00:45:41.148812+0000","last_peered":"2024-02-23T00:45:41.148812+0000","last_clean":"2024-02-23T00:45:41.148812+0000","last_became_active":"2024-02-23T00:42:58.833531+0000","last_became_peered":"2024-02-23T00:42:58.833531+0000","last_unstale":"2024-02-23T00:45:41.148812+0000","last_undegraded":"2024-02-23T00:45:41.148812+0000","last_fullsized":"2024-02-23T00:45:41.148812+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:15.352027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:15.352027+0000","last_clean_scrub_stamp":"2024-02-23T00:45:15.352027+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":174,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.558322+0000","last_change":"2024-02-23T00:44:23.543049+0000","last_active":"2024-02-23T00:45:40.558322+0000","last_peered":"2024-02-23T00:45:40.558322+0000","last_clean":"2024-02-23T00:45:40.558322+0000","last_became_active":"2024-02-23T00:42:58.835862+0000","last_became_peered":"2024-02-23T00:42:58.835862+0000","last_unstale":"2024-02-23T00:45:40.558322+0000","last_undegraded":"2024-02-23T00:45:40.558322+0000","last_fullsized":"2024-02-23T00:45:40.558322+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:23.542975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.542975+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"67'11","reported_seq":250,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.149090+0000","last_change":"2024-02-23T00:45:13.315510+0000","last_active":"2024-02-23T00:45:41.149090+0000","last_peered":"2024-02-23T00:45:41.149090+0000","last_clean":"2024-02-23T00:45:41.149090+0000","last_became_active":"2024-02-23T00:39:16.783896+0000","last_became_peered":"2024-02-23T00:39:16.783896+0000","last_unstale":"2024-02-23T00:45:41.149090+0000","last_undegraded":"2024-02-23T00:45:41.149090+0000","last_fullsized":"2024-02-23T00:45:41.149090+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'11","last_scrub_stamp":"2024-02-23T00:45:13.315434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:13.315434+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"202'2","reported_seq":241,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.152767+0000","last_change":"2024-02-23T00:44:34.720705+0000","last_active":"2024-02-23T00:45:41.152767+0000","last_peered":"2024-02-23T00:45:41.152767+0000","last_clean":"2024-02-23T00:45:41.152767+0000","last_became_active":"2024-02-23T00:39:14.775804+0000","last_became_peered":"2024-02-23T00:39:14.775804+0000","last_unstale":"2024-02-23T00:45:41.152767+0000","last_undegraded":"2024-02-23T00:45:41.152767+0000","last_fullsized":"2024-02-23T00:45:41.152767+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"202'2","last_scrub_stamp":"2024-02-23T00:44:34.720640+0000","last_deep_scrub":"202'2","last_deep_scrub_stamp":"2024-02-23T00:44:34.720640+0000","last_clean_scrub_stamp":"2024-02-23T00:44:34.720640+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":224,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.152086+0000","last_change":"2024-02-23T00:43:31.903157+0000","last_active":"2024-02-23T00:45:41.152086+0000","last_peered":"2024-02-23T00:45:41.152086+0000","last_clean":"2024-02-23T00:45:41.152086+0000","last_became_active":"2024-02-23T00:39:18.800382+0000","last_became_peered":"2024-02-23T00:39:18.800382+0000","last_unstale":"2024-02-23T00:45:41.152086+0000","last_undegraded":"2024-02-23T00:45:41.152086+0000","last_fullsized":"2024-02-23T00:45:41.152086+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:31.903031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:31.903031+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":228,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.136870+0000","last_change":"2024-02-23T00:44:23.152677+0000","last_active":"2024-02-23T00:45:41.136870+0000","last_peered":"2024-02-23T00:45:41.136870+0000","last_clean":"2024-02-23T00:45:41.136870+0000","last_became_active":"2024-02-23T00:39:20.799053+0000","last_became_peered":"2024-02-23T00:39:20.799053+0000","last_unstale":"2024-02-23T00:45:41.136870+0000","last_undegraded":"2024-02-23T00:45:41.136870+0000","last_fullsized":"2024-02-23T00:45:41.136870+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:23.152602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.152602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"110'2","reported_seq":177,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.134844+0000","last_change":"2024-02-23T00:45:09.917568+0000","last_active":"2024-02-23T00:45:41.134844+0000","last_peered":"2024-02-23T00:45:41.134844+0000","last_clean":"2024-02-23T00:45:41.134844+0000","last_became_active":"2024-02-23T00:42:58.698725+0000","last_became_peered":"2024-02-23T00:42:58.698725+0000","last_unstale":"2024-02-23T00:45:41.134844+0000","last_undegraded":"2024-02-23T00:45:41.134844+0000","last_fullsized":"2024-02-23T00:45:41.134844+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'2","last_scrub_stamp":"2024-02-23T00:45:09.917420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:09.917420+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":170,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.152694+0000","last_change":"2024-02-23T00:42:57.652742+0000","last_active":"2024-02-23T00:45:41.152694+0000","last_peered":"2024-02-23T00:45:41.152694+0000","last_clean":"2024-02-23T00:45:41.152694+0000","last_became_active":"2024-02-23T00:42:57.652438+0000","last_became_peered":"2024-02-23T00:42:57.652438+0000","last_unstale":"2024-02-23T00:45:41.152694+0000","last_undegraded":"2024-02-23T00:45:41.152694+0000","last_fullsized":"2024-02-23T00:45:41.152694+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"67'4","reported_seq":238,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.134804+0000","last_change":"2024-02-23T00:45:20.967211+0000","last_active":"2024-02-23T00:45:41.134804+0000","last_peered":"2024-02-23T00:45:41.134804+0000","last_clean":"2024-02-23T00:45:41.134804+0000","last_became_active":"2024-02-23T00:39:16.785830+0000","last_became_peered":"2024-02-23T00:39:16.785830+0000","last_unstale":"2024-02-23T00:45:41.134804+0000","last_undegraded":"2024-02-23T00:45:41.134804+0000","last_fullsized":"2024-02-23T00:45:41.134804+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'4","last_scrub_stamp":"2024-02-23T00:45:20.967095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:20.967095+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":233,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.816870+0000","last_change":"2024-02-23T00:43:38.947741+0000","last_active":"2024-02-23T00:45:40.816870+0000","last_peered":"2024-02-23T00:45:40.816870+0000","last_clean":"2024-02-23T00:45:40.816870+0000","last_became_active":"2024-02-23T00:39:14.783325+0000","last_became_peered":"2024-02-23T00:39:14.783325+0000","last_unstale":"2024-02-23T00:45:40.816870+0000","last_undegraded":"2024-02-23T00:45:40.816870+0000","last_fullsized":"2024-02-23T00:45:40.816870+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:38.947675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:38.947675+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":232,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.603885+0000","last_change":"2024-02-23T00:45:20.761547+0000","last_active":"2024-02-23T00:45:40.603885+0000","last_peered":"2024-02-23T00:45:40.603885+0000","last_clean":"2024-02-23T00:45:40.603885+0000","last_became_active":"2024-02-23T00:39:18.797287+0000","last_became_peered":"2024-02-23T00:39:18.797287+0000","last_unstale":"2024-02-23T00:45:40.603885+0000","last_undegraded":"2024-02-23T00:45:40.603885+0000","last_fullsized":"2024-02-23T00:45:40.603885+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:20.761440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:20.761440+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":224,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.654138+0000","last_change":"2024-02-23T00:45:10.741762+0000","last_active":"2024-02-23T00:45:40.654138+0000","last_peered":"2024-02-23T00:45:40.654138+0000","last_clean":"2024-02-23T00:45:40.654138+0000","last_became_active":"2024-02-23T00:39:20.795351+0000","last_became_peered":"2024-02-23T00:39:20.795351+0000","last_unstale":"2024-02-23T00:45:40.654138+0000","last_undegraded":"2024-02-23T00:45:40.654138+0000","last_fullsized":"2024-02-23T00:45:40.654138+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.741694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.741694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1b","version":"110'10","reported_seq":183,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.654298+0000","last_change":"2024-02-23T00:45:14.710146+0000","last_active":"2024-02-23T00:45:40.654298+0000","last_peered":"2024-02-23T00:45:40.654298+0000","last_clean":"2024-02-23T00:45:40.654298+0000","last_became_active":"2024-02-23T00:42:58.749126+0000","last_became_peered":"2024-02-23T00:42:58.749126+0000","last_unstale":"2024-02-23T00:45:40.654298+0000","last_undegraded":"2024-02-23T00:45:40.654298+0000","last_fullsized":"2024-02-23T00:45:40.654298+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'10","last_scrub_stamp":"2024-02-23T00:45:14.710075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.710075+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":168,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.655484+0000","last_change":"2024-02-23T00:42:58.741674+0000","last_active":"2024-02-23T00:45:40.655484+0000","last_peered":"2024-02-23T00:45:40.655484+0000","last_clean":"2024-02-23T00:45:40.655484+0000","last_became_active":"2024-02-23T00:42:58.741365+0000","last_became_peered":"2024-02-23T00:42:58.741365+0000","last_unstale":"2024-02-23T00:45:40.655484+0000","last_undegraded":"2024-02-23T00:45:40.655484+0000","last_fullsized":"2024-02-23T00:45:40.655484+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"68'17","reported_seq":266,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.137624+0000","last_change":"2024-02-23T00:43:43.997579+0000","last_active":"2024-02-23T00:45:41.137624+0000","last_peered":"2024-02-23T00:45:41.137624+0000","last_clean":"2024-02-23T00:45:41.137624+0000","last_became_active":"2024-02-23T00:39:16.857232+0000","last_became_peered":"2024-02-23T00:39:16.857232+0000","last_unstale":"2024-02-23T00:45:41.137624+0000","last_undegraded":"2024-02-23T00:45:41.137624+0000","last_fullsized":"2024-02-23T00:45:41.137624+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:43:43.997452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:43.997452+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":231,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.605432+0000","last_change":"2024-02-23T00:43:52.419206+0000","last_active":"2024-02-23T00:45:40.605432+0000","last_peered":"2024-02-23T00:45:40.605432+0000","last_clean":"2024-02-23T00:45:40.605432+0000","last_became_active":"2024-02-23T00:39:14.784092+0000","last_became_peered":"2024-02-23T00:39:14.784092+0000","last_unstale":"2024-02-23T00:45:40.605432+0000","last_undegraded":"2024-02-23T00:45:40.605432+0000","last_fullsized":"2024-02-23T00:45:40.605432+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:52.419100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:52.419100+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":226,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.654738+0000","last_change":"2024-02-23T00:44:25.797288+0000","last_active":"2024-02-23T00:45:40.654738+0000","last_peered":"2024-02-23T00:45:40.654738+0000","last_clean":"2024-02-23T00:45:40.654738+0000","last_became_active":"2024-02-23T00:39:18.791457+0000","last_became_peered":"2024-02-23T00:39:18.791457+0000","last_unstale":"2024-02-23T00:45:40.654738+0000","last_undegraded":"2024-02-23T00:45:40.654738+0000","last_fullsized":"2024-02-23T00:45:40.654738+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:25.797184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:25.797184+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":223,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.136710+0000","last_change":"2024-02-23T00:43:45.965636+0000","last_active":"2024-02-23T00:45:41.136710+0000","last_peered":"2024-02-23T00:45:41.136710+0000","last_clean":"2024-02-23T00:45:41.136710+0000","last_became_active":"2024-02-23T00:39:21.160376+0000","last_became_peered":"2024-02-23T00:39:21.160376+0000","last_unstale":"2024-02-23T00:45:41.136710+0000","last_undegraded":"2024-02-23T00:45:41.136710+0000","last_fullsized":"2024-02-23T00:45:41.136710+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:45.965570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.965570+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1a","version":"110'5","reported_seq":174,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.816544+0000","last_change":"2024-02-23T00:42:58.738588+0000","last_active":"2024-02-23T00:45:40.816544+0000","last_peered":"2024-02-23T00:45:40.816544+0000","last_clean":"2024-02-23T00:45:40.816544+0000","last_became_active":"2024-02-23T00:42:58.736706+0000","last_became_peered":"2024-02-23T00:42:58.736706+0000","last_unstale":"2024-02-23T00:45:40.816544+0000","last_undegraded":"2024-02-23T00:45:40.816544+0000","last_fullsized":"2024-02-23T00:45:40.816544+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":173,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.136329+0000","last_change":"2024-02-23T00:45:23.378334+0000","last_active":"2024-02-23T00:45:40.136329+0000","last_peered":"2024-02-23T00:45:40.136329+0000","last_clean":"2024-02-23T00:45:40.136329+0000","last_became_active":"2024-02-23T00:42:58.702580+0000","last_became_peered":"2024-02-23T00:42:58.702580+0000","last_unstale":"2024-02-23T00:45:40.136329+0000","last_undegraded":"2024-02-23T00:45:40.136329+0000","last_fullsized":"2024-02-23T00:45:40.136329+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:23.378270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.378270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"68'16","reported_seq":261,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.559487+0000","last_change":"2024-02-23T00:45:04.672612+0000","last_active":"2024-02-23T00:45:40.559487+0000","last_peered":"2024-02-23T00:45:40.559487+0000","last_clean":"2024-02-23T00:45:40.559487+0000","last_became_active":"2024-02-23T00:39:16.858945+0000","last_became_peered":"2024-02-23T00:39:16.858945+0000","last_unstale":"2024-02-23T00:45:40.559487+0000","last_undegraded":"2024-02-23T00:45:40.559487+0000","last_fullsized":"2024-02-23T00:45:40.559487+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:45:04.672537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.672537+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":240,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.816415+0000","last_change":"2024-02-23T00:43:44.031862+0000","last_active":"2024-02-23T00:45:40.816415+0000","last_peered":"2024-02-23T00:45:40.816415+0000","last_clean":"2024-02-23T00:45:40.816415+0000","last_became_active":"2024-02-23T00:39:14.785363+0000","last_became_peered":"2024-02-23T00:39:14.785363+0000","last_unstale":"2024-02-23T00:45:40.816415+0000","last_undegraded":"2024-02-23T00:45:40.816415+0000","last_fullsized":"2024-02-23T00:45:40.816415+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"80'2","last_scrub_stamp":"2024-02-23T00:43:44.031793+0000","last_deep_scrub":"80'2","last_deep_scrub_stamp":"2024-02-23T00:40:15.942725+0000","last_clean_scrub_stamp":"2024-02-23T00:43:44.031793+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"93'9","reported_seq":316,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.605490+0000","last_change":"2024-02-23T00:43:50.393789+0000","last_active":"2024-02-23T00:45:40.605490+0000","last_peered":"2024-02-23T00:45:40.605490+0000","last_clean":"2024-02-23T00:45:40.605490+0000","last_became_active":"2024-02-23T00:39:18.796680+0000","last_became_peered":"2024-02-23T00:39:18.796680+0000","last_unstale":"2024-02-23T00:45:40.605490+0000","last_undegraded":"2024-02-23T00:45:40.605490+0000","last_fullsized":"2024-02-23T00:45:40.605490+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-02-23T00:43:50.393713+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-02-23T00:43:50.393713+0000","last_clean_scrub_stamp":"2024-02-23T00:43:50.393713+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":168,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.654879+0000","last_change":"2024-02-23T00:42:58.805923+0000","last_active":"2024-02-23T00:45:40.654879+0000","last_peered":"2024-02-23T00:45:40.654879+0000","last_clean":"2024-02-23T00:45:40.654879+0000","last_became_active":"2024-02-23T00:42:58.803900+0000","last_became_peered":"2024-02-23T00:42:58.803900+0000","last_unstale":"2024-02-23T00:45:40.654879+0000","last_undegraded":"2024-02-23T00:45:40.654879+0000","last_fullsized":"2024-02-23T00:45:40.654879+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":169,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.557900+0000","last_change":"2024-02-23T00:42:58.850530+0000","last_active":"2024-02-23T00:45:40.557900+0000","last_peered":"2024-02-23T00:45:40.557900+0000","last_clean":"2024-02-23T00:45:40.557900+0000","last_became_active":"2024-02-23T00:42:58.850072+0000","last_became_peered":"2024-02-23T00:42:58.850072+0000","last_unstale":"2024-02-23T00:45:40.557900+0000","last_undegraded":"2024-02-23T00:45:40.557900+0000","last_fullsized":"2024-02-23T00:45:40.557900+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"93'23","reported_seq":274,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.605560+0000","last_change":"2024-02-23T00:44:28.387225+0000","last_active":"2024-02-23T00:45:40.605560+0000","last_peered":"2024-02-23T00:45:40.605560+0000","last_clean":"2024-02-23T00:45:40.605560+0000","last_became_active":"2024-02-23T00:39:16.782660+0000","last_became_peered":"2024-02-23T00:39:16.782660+0000","last_unstale":"2024-02-23T00:45:40.605560+0000","last_undegraded":"2024-02-23T00:45:40.605560+0000","last_fullsized":"2024-02-23T00:45:40.605560+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'23","last_scrub_stamp":"2024-02-23T00:44:28.387155+0000","last_deep_scrub":"93'23","last_deep_scrub_stamp":"2024-02-23T00:41:35.954516+0000","last_clean_scrub_stamp":"2024-02-23T00:44:28.387155+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":234,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.137070+0000","last_change":"2024-02-23T00:44:21.157372+0000","last_active":"2024-02-23T00:45:41.137070+0000","last_peered":"2024-02-23T00:45:41.137070+0000","last_clean":"2024-02-23T00:45:41.137070+0000","last_became_active":"2024-02-23T00:39:14.781560+0000","last_became_peered":"2024-02-23T00:39:14.781560+0000","last_unstale":"2024-02-23T00:45:41.137070+0000","last_undegraded":"2024-02-23T00:45:41.137070+0000","last_fullsized":"2024-02-23T00:45:41.137070+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:21.157316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:21.157316+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":229,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.559541+0000","last_change":"2024-02-23T00:43:49.238770+0000","last_active":"2024-02-23T00:45:40.559541+0000","last_peered":"2024-02-23T00:45:40.559541+0000","last_clean":"2024-02-23T00:45:40.559541+0000","last_became_active":"2024-02-23T00:39:18.790602+0000","last_became_peered":"2024-02-23T00:39:18.790602+0000","last_unstale":"2024-02-23T00:45:40.559541+0000","last_undegraded":"2024-02-23T00:45:40.559541+0000","last_fullsized":"2024-02-23T00:45:40.559541+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:49.238704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:53.189881+0000","last_clean_scrub_stamp":"2024-02-23T00:43:49.238704+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.14","version":"110'2","reported_seq":170,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.604452+0000","last_change":"2024-02-23T00:42:58.804637+0000","last_active":"2024-02-23T00:45:40.604452+0000","last_peered":"2024-02-23T00:45:40.604452+0000","last_clean":"2024-02-23T00:45:40.604452+0000","last_became_active":"2024-02-23T00:42:58.804344+0000","last_became_peered":"2024-02-23T00:42:58.804344+0000","last_unstale":"2024-02-23T00:45:40.604452+0000","last_undegraded":"2024-02-23T00:45:40.604452+0000","last_fullsized":"2024-02-23T00:45:40.604452+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":175,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.137151+0000","last_change":"2024-02-23T00:45:15.933508+0000","last_active":"2024-02-23T00:45:41.137151+0000","last_peered":"2024-02-23T00:45:41.137151+0000","last_clean":"2024-02-23T00:45:41.137151+0000","last_became_active":"2024-02-23T00:42:58.774720+0000","last_became_peered":"2024-02-23T00:42:58.774720+0000","last_unstale":"2024-02-23T00:45:41.137151+0000","last_undegraded":"2024-02-23T00:45:41.137151+0000","last_fullsized":"2024-02-23T00:45:41.137151+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:15.933438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:15.933438+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"67'15","reported_seq":254,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.136146+0000","last_change":"2024-02-23T00:43:36.449809+0000","last_active":"2024-02-23T00:45:40.136146+0000","last_peered":"2024-02-23T00:45:40.136146+0000","last_clean":"2024-02-23T00:45:40.136146+0000","last_became_active":"2024-02-23T00:39:16.787138+0000","last_became_peered":"2024-02-23T00:39:16.787138+0000","last_unstale":"2024-02-23T00:45:40.136146+0000","last_undegraded":"2024-02-23T00:45:40.136146+0000","last_fullsized":"2024-02-23T00:45:40.136146+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:43:36.449687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:36.449687+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":231,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.604695+0000","last_change":"2024-02-23T00:43:45.362071+0000","last_active":"2024-02-23T00:45:40.604695+0000","last_peered":"2024-02-23T00:45:40.604695+0000","last_clean":"2024-02-23T00:45:40.604695+0000","last_became_active":"2024-02-23T00:39:14.782494+0000","last_became_peered":"2024-02-23T00:39:14.782494+0000","last_unstale":"2024-02-23T00:45:40.604695+0000","last_undegraded":"2024-02-23T00:45:40.604695+0000","last_fullsized":"2024-02-23T00:45:40.604695+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:45.361978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.361978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":229,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.152913+0000","last_change":"2024-02-23T00:45:03.615758+0000","last_active":"2024-02-23T00:45:41.152913+0000","last_peered":"2024-02-23T00:45:41.152913+0000","last_clean":"2024-02-23T00:45:41.152913+0000","last_became_active":"2024-02-23T00:39:18.801039+0000","last_became_peered":"2024-02-23T00:39:18.801039+0000","last_unstale":"2024-02-23T00:45:41.152913+0000","last_undegraded":"2024-02-23T00:45:41.152913+0000","last_fullsized":"2024-02-23T00:45:41.152913+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:03.615692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.641689+0000","last_clean_scrub_stamp":"2024-02-23T00:45:03.615692+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"110'5","reported_seq":180,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.149511+0000","last_change":"2024-02-23T00:45:14.317657+0000","last_active":"2024-02-23T00:45:41.149511+0000","last_peered":"2024-02-23T00:45:41.149511+0000","last_clean":"2024-02-23T00:45:41.149511+0000","last_became_active":"2024-02-23T00:42:58.783343+0000","last_became_peered":"2024-02-23T00:42:58.783343+0000","last_unstale":"2024-02-23T00:45:41.149511+0000","last_undegraded":"2024-02-23T00:45:41.149511+0000","last_fullsized":"2024-02-23T00:45:41.149511+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'5","last_scrub_stamp":"2024-02-23T00:45:14.317573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.317573+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":175,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.138324+0000","last_change":"2024-02-23T00:45:21.917072+0000","last_active":"2024-02-23T00:45:41.138324+0000","last_peered":"2024-02-23T00:45:41.138324+0000","last_clean":"2024-02-23T00:45:41.138324+0000","last_became_active":"2024-02-23T00:42:58.770759+0000","last_became_peered":"2024-02-23T00:42:58.770759+0000","last_unstale":"2024-02-23T00:45:41.138324+0000","last_undegraded":"2024-02-23T00:45:41.138324+0000","last_fullsized":"2024-02-23T00:45:41.138324+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:21.916996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:21.916996+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"68'18","reported_seq":270,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.136507+0000","last_change":"2024-02-23T00:43:51.079824+0000","last_active":"2024-02-23T00:45:41.136507+0000","last_peered":"2024-02-23T00:45:41.136507+0000","last_clean":"2024-02-23T00:45:41.136507+0000","last_became_active":"2024-02-23T00:39:16.782803+0000","last_became_peered":"2024-02-23T00:39:16.782803+0000","last_unstale":"2024-02-23T00:45:41.136507+0000","last_undegraded":"2024-02-23T00:45:41.136507+0000","last_fullsized":"2024-02-23T00:45:41.136507+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'18","last_scrub_stamp":"2024-02-23T00:43:51.079756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:51.079756+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":234,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.136466+0000","last_change":"2024-02-23T00:43:48.030906+0000","last_active":"2024-02-23T00:45:41.136466+0000","last_peered":"2024-02-23T00:45:41.136466+0000","last_clean":"2024-02-23T00:45:41.136466+0000","last_became_active":"2024-02-23T00:39:14.782163+0000","last_became_peered":"2024-02-23T00:39:14.782163+0000","last_unstale":"2024-02-23T00:45:41.136466+0000","last_undegraded":"2024-02-23T00:45:41.136466+0000","last_fullsized":"2024-02-23T00:45:41.136466+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:48.030846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.030846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":229,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.149455+0000","last_change":"2024-02-23T00:44:37.068899+0000","last_active":"2024-02-23T00:45:41.149455+0000","last_peered":"2024-02-23T00:45:41.149455+0000","last_clean":"2024-02-23T00:45:41.149455+0000","last_became_active":"2024-02-23T00:39:18.794903+0000","last_became_peered":"2024-02-23T00:39:18.794903+0000","last_unstale":"2024-02-23T00:45:41.149455+0000","last_undegraded":"2024-02-23T00:45:41.149455+0000","last_fullsized":"2024-02-23T00:45:41.149455+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:37.068834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_clean_scrub_stamp":"2024-02-23T00:44:37.068834+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"110'5","reported_seq":174,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.559590+0000","last_change":"2024-02-23T00:42:58.839927+0000","last_active":"2024-02-23T00:45:40.559590+0000","last_peered":"2024-02-23T00:45:40.559590+0000","last_clean":"2024-02-23T00:45:40.559590+0000","last_became_active":"2024-02-23T00:42:58.838833+0000","last_became_peered":"2024-02-23T00:42:58.838833+0000","last_unstale":"2024-02-23T00:45:40.559590+0000","last_undegraded":"2024-02-23T00:45:40.559590+0000","last_fullsized":"2024-02-23T00:45:40.559590+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":169,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.817319+0000","last_change":"2024-02-23T00:42:58.698567+0000","last_active":"2024-02-23T00:45:40.817319+0000","last_peered":"2024-02-23T00:45:40.817319+0000","last_clean":"2024-02-23T00:45:40.817319+0000","last_became_active":"2024-02-23T00:42:58.698258+0000","last_became_peered":"2024-02-23T00:42:58.698258+0000","last_unstale":"2024-02-23T00:45:40.817319+0000","last_undegraded":"2024-02-23T00:45:40.817319+0000","last_fullsized":"2024-02-23T00:45:40.817319+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"95'27","reported_seq":283,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.148692+0000","last_change":"2024-02-23T00:43:52.124336+0000","last_active":"2024-02-23T00:45:41.148692+0000","last_peered":"2024-02-23T00:45:41.148692+0000","last_clean":"2024-02-23T00:45:41.148692+0000","last_became_active":"2024-02-23T00:39:16.786791+0000","last_became_peered":"2024-02-23T00:39:16.786791+0000","last_unstale":"2024-02-23T00:45:41.148692+0000","last_undegraded":"2024-02-23T00:45:41.148692+0000","last_fullsized":"2024-02-23T00:45:41.148692+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'27","last_scrub_stamp":"2024-02-23T00:43:52.124247+0000","last_deep_scrub":"95'27","last_deep_scrub_stamp":"2024-02-23T00:41:55.919895+0000","last_clean_scrub_stamp":"2024-02-23T00:43:52.124247+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":230,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.654185+0000","last_change":"2024-02-23T00:44:27.759571+0000","last_active":"2024-02-23T00:45:40.654185+0000","last_peered":"2024-02-23T00:45:40.654185+0000","last_clean":"2024-02-23T00:45:40.654185+0000","last_became_active":"2024-02-23T00:39:14.787148+0000","last_became_peered":"2024-02-23T00:39:14.787148+0000","last_unstale":"2024-02-23T00:45:40.654185+0000","last_undegraded":"2024-02-23T00:45:40.654185+0000","last_fullsized":"2024-02-23T00:45:40.654185+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:27.759499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:27.759499+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":229,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.817223+0000","last_change":"2024-02-23T00:45:10.090353+0000","last_active":"2024-02-23T00:45:40.817223+0000","last_peered":"2024-02-23T00:45:40.817223+0000","last_clean":"2024-02-23T00:45:40.817223+0000","last_became_active":"2024-02-23T00:39:18.796431+0000","last_became_peered":"2024-02-23T00:39:18.796431+0000","last_unstale":"2024-02-23T00:45:40.817223+0000","last_undegraded":"2024-02-23T00:45:40.817223+0000","last_fullsized":"2024-02-23T00:45:40.817223+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.090270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.090270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.11","version":"110'7","reported_seq":180,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.134639+0000","last_change":"2024-02-23T00:44:30.366468+0000","last_active":"2024-02-23T00:45:40.134639+0000","last_peered":"2024-02-23T00:45:40.134639+0000","last_clean":"2024-02-23T00:45:40.134639+0000","last_became_active":"2024-02-23T00:42:58.810765+0000","last_became_peered":"2024-02-23T00:42:58.810765+0000","last_unstale":"2024-02-23T00:45:40.134639+0000","last_undegraded":"2024-02-23T00:45:40.134639+0000","last_fullsized":"2024-02-23T00:45:40.134639+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'7","last_scrub_stamp":"2024-02-23T00:44:30.366391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:44:30.366391+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":169,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.817264+0000","last_change":"2024-02-23T00:42:58.833221+0000","last_active":"2024-02-23T00:45:40.817264+0000","last_peered":"2024-02-23T00:45:40.817264+0000","last_clean":"2024-02-23T00:45:40.817264+0000","last_became_active":"2024-02-23T00:42:58.832885+0000","last_became_peered":"2024-02-23T00:42:58.832885+0000","last_unstale":"2024-02-23T00:45:40.817264+0000","last_undegraded":"2024-02-23T00:45:40.817264+0000","last_fullsized":"2024-02-23T00:45:40.817264+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"82'15","reported_seq":260,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.149467+0000","last_change":"2024-02-23T00:44:35.095478+0000","last_active":"2024-02-23T00:45:41.149467+0000","last_peered":"2024-02-23T00:45:41.149467+0000","last_clean":"2024-02-23T00:45:41.149467+0000","last_became_active":"2024-02-23T00:39:16.782762+0000","last_became_peered":"2024-02-23T00:39:16.782762+0000","last_unstale":"2024-02-23T00:45:41.149467+0000","last_undegraded":"2024-02-23T00:45:41.149467+0000","last_fullsized":"2024-02-23T00:45:41.149467+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'15","last_scrub_stamp":"2024-02-23T00:44:35.095409+0000","last_deep_scrub":"82'15","last_deep_scrub_stamp":"2024-02-23T00:41:50.975683+0000","last_clean_scrub_stamp":"2024-02-23T00:44:35.095409+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":230,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.654337+0000","last_change":"2024-02-23T00:43:44.794995+0000","last_active":"2024-02-23T00:45:40.654337+0000","last_peered":"2024-02-23T00:45:40.654337+0000","last_clean":"2024-02-23T00:45:40.654337+0000","last_became_active":"2024-02-23T00:39:14.783469+0000","last_became_peered":"2024-02-23T00:39:14.783469+0000","last_unstale":"2024-02-23T00:45:40.654337+0000","last_undegraded":"2024-02-23T00:45:40.654337+0000","last_fullsized":"2024-02-23T00:45:40.654337+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:44.794928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:44.794928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":228,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.654428+0000","last_change":"2024-02-23T00:43:41.817455+0000","last_active":"2024-02-23T00:45:40.654428+0000","last_peered":"2024-02-23T00:45:40.654428+0000","last_clean":"2024-02-23T00:45:40.654428+0000","last_became_active":"2024-02-23T00:39:18.796952+0000","last_became_peered":"2024-02-23T00:39:18.796952+0000","last_unstale":"2024-02-23T00:45:40.654428+0000","last_undegraded":"2024-02-23T00:45:40.654428+0000","last_fullsized":"2024-02-23T00:45:40.654428+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:41.817387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:41.817387+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":175,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.148459+0000","last_change":"2024-02-23T00:45:08.215139+0000","last_active":"2024-02-23T00:45:41.148459+0000","last_peered":"2024-02-23T00:45:41.148459+0000","last_clean":"2024-02-23T00:45:41.148459+0000","last_became_active":"2024-02-23T00:42:58.726021+0000","last_became_peered":"2024-02-23T00:42:58.726021+0000","last_unstale":"2024-02-23T00:45:41.148459+0000","last_undegraded":"2024-02-23T00:45:41.148459+0000","last_fullsized":"2024-02-23T00:45:41.148459+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.215063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.215063+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":177,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.149470+0000","last_change":"2024-02-23T00:45:23.224771+0000","last_active":"2024-02-23T00:45:41.149470+0000","last_peered":"2024-02-23T00:45:41.149470+0000","last_clean":"2024-02-23T00:45:41.149470+0000","last_became_active":"2024-02-23T00:42:58.758147+0000","last_became_peered":"2024-02-23T00:42:58.758147+0000","last_unstale":"2024-02-23T00:45:41.149470+0000","last_undegraded":"2024-02-23T00:45:41.149470+0000","last_fullsized":"2024-02-23T00:45:41.149470+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:23.224663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:23.224663+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.224663+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"68'15","reported_seq":262,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.148447+0000","last_change":"2024-02-23T00:43:49.139235+0000","last_active":"2024-02-23T00:45:41.148447+0000","last_peered":"2024-02-23T00:45:41.148447+0000","last_clean":"2024-02-23T00:45:41.148447+0000","last_became_active":"2024-02-23T00:39:16.784606+0000","last_became_peered":"2024-02-23T00:39:16.784606+0000","last_unstale":"2024-02-23T00:45:41.148447+0000","last_undegraded":"2024-02-23T00:45:41.148447+0000","last_fullsized":"2024-02-23T00:45:41.148447+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:43:49.139156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:49.139156+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":234,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.137318+0000","last_change":"2024-02-23T00:44:32.170717+0000","last_active":"2024-02-23T00:45:41.137318+0000","last_peered":"2024-02-23T00:45:41.137318+0000","last_clean":"2024-02-23T00:45:41.137318+0000","last_became_active":"2024-02-23T00:39:14.785362+0000","last_became_peered":"2024-02-23T00:39:14.785362+0000","last_unstale":"2024-02-23T00:45:41.137318+0000","last_undegraded":"2024-02-23T00:45:41.137318+0000","last_fullsized":"2024-02-23T00:45:41.137318+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:32.170554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:32.170554+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":224,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.151749+0000","last_change":"2024-02-23T00:42:44.553722+0000","last_active":"2024-02-23T00:45:41.151749+0000","last_peered":"2024-02-23T00:45:41.151749+0000","last_clean":"2024-02-23T00:45:41.151749+0000","last_became_active":"2024-02-23T00:39:18.796312+0000","last_became_peered":"2024-02-23T00:39:18.796312+0000","last_unstale":"2024-02-23T00:45:41.151749+0000","last_undegraded":"2024-02-23T00:45:41.151749+0000","last_fullsized":"2024-02-23T00:45:41.151749+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:44.553621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:44.553621+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.13","version":"110'9","reported_seq":184,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.152224+0000","last_change":"2024-02-23T00:45:21.719969+0000","last_active":"2024-02-23T00:45:41.152224+0000","last_peered":"2024-02-23T00:45:41.152224+0000","last_clean":"2024-02-23T00:45:41.152224+0000","last_became_active":"2024-02-23T00:42:57.655617+0000","last_became_peered":"2024-02-23T00:42:57.655617+0000","last_unstale":"2024-02-23T00:45:41.152224+0000","last_undegraded":"2024-02-23T00:45:41.152224+0000","last_fullsized":"2024-02-23T00:45:41.152224+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'9","last_scrub_stamp":"2024-02-23T00:45:21.719900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:21.719900+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":174,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.559215+0000","last_change":"2024-02-23T00:45:09.641006+0000","last_active":"2024-02-23T00:45:40.559215+0000","last_peered":"2024-02-23T00:45:40.559215+0000","last_clean":"2024-02-23T00:45:40.559215+0000","last_became_active":"2024-02-23T00:42:58.838257+0000","last_became_peered":"2024-02-23T00:42:58.838257+0000","last_unstale":"2024-02-23T00:45:40.559215+0000","last_undegraded":"2024-02-23T00:45:40.559215+0000","last_fullsized":"2024-02-23T00:45:40.559215+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:09.640936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:09.640936+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"88'50","reported_seq":328,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.137699+0000","last_change":"2024-02-23T00:45:06.038488+0000","last_active":"2024-02-23T00:45:41.137699+0000","last_peered":"2024-02-23T00:45:41.137699+0000","last_clean":"2024-02-23T00:45:41.137699+0000","last_became_active":"2024-02-23T00:39:16.780144+0000","last_became_peered":"2024-02-23T00:39:16.780144+0000","last_unstale":"2024-02-23T00:45:41.137699+0000","last_undegraded":"2024-02-23T00:45:41.137699+0000","last_fullsized":"2024-02-23T00:45:41.137699+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'50","last_scrub_stamp":"2024-02-23T00:45:06.038374+0000","last_deep_scrub":"88'50","last_deep_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.038374+0000","log_size":50,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":233,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.559466+0000","last_change":"2024-02-23T00:45:20.573233+0000","last_active":"2024-02-23T00:45:40.559466+0000","last_peered":"2024-02-23T00:45:40.559466+0000","last_clean":"2024-02-23T00:45:40.559466+0000","last_became_active":"2024-02-23T00:39:14.782000+0000","last_became_peered":"2024-02-23T00:39:14.782000+0000","last_unstale":"2024-02-23T00:45:40.559466+0000","last_undegraded":"2024-02-23T00:45:40.559466+0000","last_fullsized":"2024-02-23T00:45:40.559466+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:20.573168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_clean_scrub_stamp":"2024-02-23T00:45:20.573168+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":224,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.148660+0000","last_change":"2024-02-23T00:43:48.126631+0000","last_active":"2024-02-23T00:45:41.148660+0000","last_peered":"2024-02-23T00:45:41.148660+0000","last_clean":"2024-02-23T00:45:41.148660+0000","last_became_active":"2024-02-23T00:39:18.799723+0000","last_became_peered":"2024-02-23T00:39:18.799723+0000","last_unstale":"2024-02-23T00:45:41.148660+0000","last_undegraded":"2024-02-23T00:45:41.148660+0000","last_fullsized":"2024-02-23T00:45:41.148660+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:48.126552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.126552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":175,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-02-23T00:45:41.138285+0000","last_change":"2024-02-23T00:45:24.942600+0000","last_active":"2024-02-23T00:45:41.138285+0000","last_peered":"2024-02-23T00:45:41.138285+0000","last_clean":"2024-02-23T00:45:41.138285+0000","last_became_active":"2024-02-23T00:42:58.765643+0000","last_became_peered":"2024-02-23T00:42:58.765643+0000","last_unstale":"2024-02-23T00:45:41.138285+0000","last_undegraded":"2024-02-23T00:45:41.138285+0000","last_fullsized":"2024-02-23T00:45:41.138285+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:24.942535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:24.942535+0000","last_clean_scrub_stamp":"2024-02-23T00:45:24.942535+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":174,"reported_epoch":268,"state":"active+clean","last_fresh":"2024-02-23T00:45:40.817456+0000","last_change":"2024-02-23T00:45:14.209403+0000","last_active":"2024-02-23T00:45:40.817456+0000","last_peered":"2024-02-23T00:45:40.817456+0000","last_clean":"2024-02-23T00:45:40.817456+0000","last_became_active":"2024-02-23T00:42:58.824573+0000","last_became_peered":"2024-02-23T00:42:58.824573+0000","last_unstale":"2024-02-23T00:45:40.817456+0000","last_undegraded":"2024-02-23T00:45:40.817456+0000","last_fullsized":"2024-02-23T00:45:40.817456+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.209308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.209308+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0],"acting":[5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":172,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":173,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":62,"num_pg":32,"stat_sum":{"num_bytes":1202,"num_objects":119,"num_object_clones":1,"num_object_copies":357,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":119,"num_whiteouts":0,"num_read":94,"num_read_kb":1,"num_write":537,"num_write_kb":533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1449984,"data_stored":3606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1777,"ondisk_log_size":1777,"up":96,"acting":96,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":22,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":66,"acting":66,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1080,"num_read_kb":805,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":651,"ondisk_log_size":651,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":379,"num_read_kb":374,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2048,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":196608,"data_stored":6144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101,"ondisk_log_size":101,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234065,"num_pgs":101,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":301692,"kb_used_data":4456,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93443460,"statfs":{"total":95995035648,"available":95686103040,"internally_reserved":0,"allocated":4562944,"data_stored":2681703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.28000000000000003,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.747,"5min":4.9420000000000002,"15min":4.9420000000000002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.53200000000000003,"15min":0.51600000000000001},"min":{"1min":0.32600000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0589999999999999,"5min":4.9729999999999999,"15min":4.9729999999999999},"last":0.47399999999999998}]},{"osd":1,"last update":"Fri Feb 23 00:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.77400000000000002,"15min":0.626},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":5.4539999999999997,"5min":10.786,"15min":10.786},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.78700000000000003,"15min":0.629},"min":{"1min":0.29399999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":11.885999999999999,"5min":11.885999999999999,"15min":11.885999999999999},"last":0.35999999999999999}]},{"osd":2,"last update":"Fri Feb 23 00:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59299999999999997,"15min":0.55900000000000005},"min":{"1min":0.35899999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":4.0449999999999999,"5min":5.0789999999999997,"15min":5.0789999999999997},"last":0.436},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.57399999999999995,"15min":0.54800000000000004},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":1.04,"5min":5.1280000000000001,"15min":5.1280000000000001},"last":0.42799999999999999}]},{"osd":3,"last update":"Fri Feb 23 00:45:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":0.80800000000000005,"15min":0.62},"min":{"1min":0.35499999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":15.712999999999999,"5min":15.712999999999999,"15min":15.712999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.71699999999999997,"15min":0.61799999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.004,"5min":9.4090000000000007,"15min":9.4090000000000007},"last":0.39500000000000002}]},{"osd":4,"last update":"Fri Feb 23 00:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.44500000000000001,"15min":0.436},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":2.0350000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.55100000000000005,"15min":0.48599999999999999},"min":{"1min":0.28599999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":11.236000000000001,"5min":11.236000000000001,"15min":11.236000000000001},"last":0.40999999999999998}]},{"osd":5,"last update":"Fri Feb 23 00:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.51700000000000002,"15min":0.49099999999999999},"min":{"1min":0.29699999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":5.4000000000000004,"5min":5.4000000000000004,"15min":5.4000000000000004},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.50800000000000001,"15min":0.48599999999999999},"min":{"1min":0.37,"5min":0.29299999999999998,"15min":0.26700000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.45200000000000001}]},{"osd":6,"last update":"Fri Feb 23 00:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51000000000000001,"15min":0.50700000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.49299999999999999},{"interface":"front","average":{"1min":1.093,"5min":0.61799999999999999,"15min":0.54400000000000004},"min":{"1min":0.35599999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":13.395,"5min":13.395,"15min":13.395},"last":0.38400000000000001}]}]},{"osd":6,"up_from":47,"seq":201863463000,"num_pgs":86,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309844,"kb_used_data":4420,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93435308,"statfs":{"total":95995035648,"available":95677755392,"internally_reserved":0,"allocated":4526080,"data_stored":2681966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:44:49 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.98799999999999999,"15min":0.71199999999999997},"min":{"1min":0.29699999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61399999999999999,"5min":30.641999999999999,"15min":30.641999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.98199999999999998,"15min":0.71799999999999997},"min":{"1min":0.311,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1,"5min":29.292000000000002,"15min":29.292000000000002},"last":0.48099999999999998}]},{"osd":1,"last update":"Fri Feb 23 00:44:49 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.98599999999999999,"15min":0.69799999999999995},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.59799999999999998,"5min":28.186,"15min":28.186},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.48899999999999999,"5min":1.4910000000000001,"15min":0.88900000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.97599999999999998,"5min":33.104999999999997,"15min":33.104999999999997},"last":0.41299999999999998}]},{"osd":2,"last update":"Fri Feb 23 00:44:49 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":1.0449999999999999,"15min":0.73899999999999999},"min":{"1min":0.32800000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.1200000000000001,"5min":19.960000000000001,"15min":19.960000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.84799999999999998,"15min":0.65600000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0549999999999999,"5min":8.5860000000000003,"15min":8.5860000000000003},"last":0.67800000000000005}]},{"osd":3,"last update":"Fri Feb 23 00:44:49 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.76000000000000001,"15min":0.64200000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.0069999999999999,"5min":19.562000000000001,"15min":19.562000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.85599999999999998,"15min":0.68200000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.94299999999999995,"5min":14.199,"15min":14.199},"last":0.53500000000000003}]},{"osd":4,"last update":"Fri Feb 23 00:44:49 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.57899999999999996,"15min":0.46100000000000002},"min":{"1min":0.25900000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.55800000000000005,"5min":11.109999999999999,"15min":11.109999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.751,"15min":0.53300000000000003},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.65600000000000003,"5min":16.734000000000002,"15min":16.734000000000002},"last":0.55400000000000005}]},{"osd":5,"last update":"Fri Feb 23 00:44:49 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.629,"15min":0.48199999999999998},"min":{"1min":0.29299999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.59699999999999998,"5min":14.154999999999999,"15min":14.154999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.63500000000000001,"15min":0.5},"min":{"1min":0.312,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.67800000000000005,"5min":6.6100000000000003,"15min":6.6100000000000003},"last":0.39100000000000001}]},{"osd":7,"last update":"Fri Feb 23 00:45:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.58399999999999996,"15min":0.52000000000000002},"min":{"1min":0.29299999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.745,"5min":8.548,"15min":8.548},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.752,"15min":0.59399999999999997},"min":{"1min":0.33900000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.68899999999999995,"5min":15.618,"15min":15.618},"last":0.498}]}]},{"osd":1,"up_from":13,"seq":55834574962,"num_pgs":101,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309800,"kb_used_data":4376,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93435352,"statfs":{"total":95995035648,"available":95677800448,"internally_reserved":0,"allocated":4481024,"data_stored":2682777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:44:43 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":1.298,"15min":0.66500000000000004},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":1.637,"5min":78.361000000000004,"15min":78.361000000000004},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.57299999999999995,"15min":0.437},"min":{"1min":0.218,"5min":0.191,"15min":0.191},"max":{"1min":1.284,"5min":5.8849999999999998,"15min":5.8849999999999998},"last":0.39200000000000002}]},{"osd":2,"last update":"Fri Feb 23 00:45:03 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":1.1850000000000001,"15min":0.65900000000000003},"min":{"1min":0.247,"5min":0.219,"15min":0.188},"max":{"1min":0.63,"5min":48.606000000000002,"15min":48.606000000000002},"last":0.376},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.88300000000000001,"15min":0.58599999999999997},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.249},"max":{"1min":0.82599999999999996,"5min":23.911000000000001,"15min":23.911000000000001},"last":0.34000000000000002}]},{"osd":3,"last update":"Fri Feb 23 00:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.89400000000000002,"15min":0.624},"min":{"1min":0.28000000000000003,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.56100000000000005,"5min":35.244999999999997,"15min":35.244999999999997},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.38,"5min":1.1299999999999999,"15min":0.68200000000000005},"min":{"1min":0.28599999999999998,"5min":0.253,"15min":0.248},"max":{"1min":0.60299999999999998,"5min":56.853999999999999,"15min":56.853999999999999},"last":0.32700000000000001}]},{"osd":4,"last update":"Fri Feb 23 00:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":1.2350000000000001,"15min":0.81999999999999995},"min":{"1min":0.33700000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72899999999999998,"5min":37.646999999999998,"15min":37.646999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.875,"15min":0.69999999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83499999999999996,"5min":9.6859999999999999,"15min":9.6859999999999999},"last":0.49299999999999999}]},{"osd":5,"last update":"Fri Feb 23 00:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":1.224,"15min":0.84599999999999997},"min":{"1min":0.41099999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83399999999999996,"5min":48.792000000000002,"15min":48.792000000000002},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.83699999999999997,"15min":0.755},"min":{"1min":0.45100000000000001,"5min":0.38100000000000001,"15min":0.34200000000000003},"max":{"1min":0.76500000000000001,"5min":11.872999999999999,"15min":11.872999999999999},"last":0.501}]},{"osd":6,"last update":"Fri Feb 23 00:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":1.2,"15min":0.82999999999999996},"min":{"1min":0.39200000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.90200000000000002,"5min":40.741,"15min":40.741},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.77800000000000002,"15min":0.69599999999999995},"min":{"1min":0.46100000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77200000000000002,"5min":6.0069999999999997,"15min":6.0069999999999997},"last":0.59499999999999997}]},{"osd":7,"last update":"Fri Feb 23 00:45:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.91000000000000003,"15min":0.71899999999999997},"min":{"1min":0.42499999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82799999999999996,"5min":20.225000000000001,"15min":20.225000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.94199999999999995,"15min":0.72799999999999998},"min":{"1min":0.42799999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85999999999999999,"5min":20.254999999999999,"15min":20.254999999999999},"last":0.57499999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738488,"num_pgs":94,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":313904,"kb_used_data":4380,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93431248,"statfs":{"total":95995035648,"available":95673597952,"internally_reserved":0,"allocated":4485120,"data_stored":2681980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 23 00:44:42 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":1.0980000000000001,"15min":0.59099999999999997},"min":{"1min":0.25900000000000001,"5min":0.223,"15min":0.19500000000000001},"max":{"1min":0.82799999999999996,"5min":28.856999999999999,"15min":28.856999999999999},"last":0.27900000000000003},{"interface":"front","average":{"1min":1.3480000000000001,"5min":0.97799999999999998,"15min":0.56100000000000005},"min":{"1min":0.22700000000000001,"5min":0.218,"15min":0.20799999999999999},"max":{"1min":12.923,"5min":20.577000000000002,"15min":20.577000000000002},"last":0.47399999999999998}]},{"osd":2,"last update":"Fri Feb 23 00:45:06 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.026,"15min":0.60399999999999998},"min":{"1min":0.308,"5min":0.26000000000000001,"15min":0.20499999999999999},"max":{"1min":13.67,"5min":13.863,"15min":13.863},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.88400000000000001,"15min":0.54900000000000004},"min":{"1min":0.25700000000000001,"5min":0.24299999999999999,"15min":0.19800000000000001},"max":{"1min":0.60099999999999998,"5min":10.455,"15min":10.455},"last":0.33900000000000002}]},{"osd":3,"last update":"Fri Feb 23 00:45:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.5109999999999999,"15min":0.80300000000000005},"min":{"1min":0.27700000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":12.566000000000001,"5min":35.566000000000003,"15min":35.566000000000003},"last":0.40200000000000002},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.3660000000000001,"15min":0.79100000000000004},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":12.727,"5min":20.702999999999999,"15min":20.702999999999999},"last":0.48999999999999999}]},{"osd":4,"last update":"Fri Feb 23 00:44:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.038,"15min":0.81299999999999994},"min":{"1min":0.36199999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":10.584,"5min":10.584,"15min":10.584},"last":0.65300000000000002},{"interface":"front","average":{"1min":1.306,"5min":0.89600000000000002,"15min":0.79700000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":13.029999999999999,"5min":13.029999999999999,"15min":13.029999999999999},"last":0.69699999999999995}]},{"osd":5,"last update":"Fri Feb 23 00:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":1.014,"15min":0.74399999999999999},"min":{"1min":0.37,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88200000000000001,"5min":10.397,"15min":10.397},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":1.079,"15min":0.80900000000000005},"min":{"1min":0.44700000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86299999999999999,"5min":17.297000000000001,"15min":17.297000000000001},"last":0.77500000000000002}]},{"osd":6,"last update":"Fri Feb 23 00:44:57 2024","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.1160000000000001,"15min":0.80200000000000005},"min":{"1min":0.434,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":13.237,"5min":17.373999999999999,"15min":17.373999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":1.3029999999999999,"5min":0.90200000000000002,"15min":0.72399999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":13.086,"5min":13.086,"15min":13.086},"last":0.78800000000000003}]},{"osd":7,"last update":"Fri Feb 23 00:45:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":1.081,"15min":0.81299999999999994},"min":{"1min":0.48199999999999998,"5min":0.41699999999999998,"15min":0.39400000000000002},"max":{"1min":0.82599999999999996,"5min":17.399999999999999,"15min":17.399999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":1.494,"5min":1.0640000000000001,"15min":0.78900000000000003},"min":{"1min":0.45400000000000001,"5min":0.433,"15min":0.43099999999999999},"max":{"1min":16.427,"5min":16.427,"15min":16.427},"last":0.81200000000000006}]}]},{"osd":2,"up_from":18,"seq":77309411437,"num_pgs":92,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309916,"kb_used_data":4492,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93435236,"statfs":{"total":95995035648,"available":95677681664,"internally_reserved":0,"allocated":4599808,"data_stored":2681821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:45:02 2024","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":0.70199999999999996,"15min":0.48699999999999999},"min":{"1min":0.20499999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":13.138999999999999,"5min":13.657,"15min":13.657},"last":0.312},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2649999999999999,"15min":0.68000000000000005},"min":{"1min":0.25700000000000001,"5min":0.246,"15min":0.23599999999999999},"max":{"1min":12.705,"5min":42.343000000000004,"15min":42.343000000000004},"last":0.33500000000000002}]},{"osd":1,"last update":"Fri Feb 23 00:45:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":0.96999999999999997,"15min":0.57499999999999996},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":8.8569999999999993,"5min":18.952000000000002,"15min":18.952000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":1.6419999999999999,"5min":0.76800000000000002,"15min":0.52300000000000002},"min":{"1min":0.26100000000000001,"5min":0.23999999999999999,"15min":0.23599999999999999},"max":{"1min":15.391,"5min":15.391,"15min":15.391},"last":0.36699999999999999}]},{"osd":3,"last update":"Fri Feb 23 00:45:35 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":1.1779999999999999,"15min":0.71299999999999997},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.47499999999999998,"5min":25.635999999999999,"15min":25.635999999999999},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.41799999999999998,"5min":1.002,"15min":0.64600000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.0840000000000001,"5min":19.960000000000001,"15min":19.960000000000001},"last":0.50900000000000001}]},{"osd":4,"last update":"Fri Feb 23 00:45:02 2024","interfaces":[{"interface":"back","average":{"1min":2.1920000000000002,"5min":0.998,"15min":0.73099999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":13.673,"5min":13.673,"15min":13.673},"last":0.439},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.044,"15min":0.73499999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.33900000000000002},"max":{"1min":8.7870000000000008,"5min":10.103999999999999,"15min":10.103999999999999},"last":0.56799999999999995}]},{"osd":5,"last update":"Fri Feb 23 00:45:23 2024","interfaces":[{"interface":"back","average":{"1min":1.784,"5min":0.90600000000000003,"15min":0.69099999999999995},"min":{"1min":0.40699999999999997,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":15.612,"5min":15.612,"15min":15.612},"last":0.48399999999999999},{"interface":"front","average":{"1min":1.335,"5min":0.88,"15min":0.70299999999999996},"min":{"1min":0.376,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":13.632999999999999,"5min":13.632999999999999,"15min":13.632999999999999},"last":0.40100000000000002}]},{"osd":6,"last update":"Fri Feb 23 00:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":0.872,"15min":0.70599999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":11.544,"5min":11.544,"15min":11.544},"last":0.45400000000000001},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0029999999999999,"15min":0.73199999999999998},"min":{"1min":0.38800000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":8.8290000000000006,"5min":10.132999999999999,"15min":10.132999999999999},"last":0.58399999999999996}]},{"osd":7,"last update":"Fri Feb 23 00:45:15 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":0.879,"15min":0.74299999999999999},"min":{"1min":0.40600000000000003,"5min":0.377,"15min":0.377},"max":{"1min":11.599,"5min":11.599,"15min":11.599},"last":0.496},{"interface":"front","average":{"1min":1.494,"5min":1.097,"15min":0.80000000000000004},"min":{"1min":0.42399999999999999,"5min":0.374,"15min":0.374},"max":{"1min":8.7469999999999999,"5min":10.148999999999999,"15min":10.148999999999999},"last":0.55300000000000005}]}]},{"osd":3,"up_from":26,"seq":111669149798,"num_pgs":104,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":313960,"kb_used_data":4444,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93431192,"statfs":{"total":95995035648,"available":95673540608,"internally_reserved":0,"allocated":4550656,"data_stored":2683084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:45:35 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.746,"15min":0.53900000000000003},"min":{"1min":0.22800000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.56699999999999995,"5min":12.212999999999999,"15min":12.212999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.88900000000000001,"15min":0.59899999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.56899999999999995,"5min":11.714,"15min":11.714},"last":0.26700000000000002}]},{"osd":1,"last update":"Fri Feb 23 00:45:35 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.871,"15min":0.60799999999999998},"min":{"1min":0.29299999999999998,"5min":0.23400000000000001,"15min":0.22900000000000001},"max":{"1min":0.61299999999999999,"5min":11.742000000000001,"15min":11.742000000000001},"last":0.308},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.78200000000000003,"15min":0.56100000000000005},"min":{"1min":0.223,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":1.8300000000000001,"5min":6.383,"15min":6.383},"last":0.32400000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:45:35 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.95699999999999996,"15min":0.64900000000000002},"min":{"1min":0.26000000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.869,"5min":13.359999999999999,"15min":13.359999999999999},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.374,"5min":1.002,"15min":0.65500000000000003},"min":{"1min":0.27500000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54400000000000004,"5min":21.122,"15min":21.122},"last":0.29499999999999998}]},{"osd":4,"last update":"Fri Feb 23 00:44:57 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.97399999999999998,"15min":0.72499999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":8.125,"5min":11.644,"15min":11.644},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.85399999999999998,"15min":0.68000000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":6.2549999999999999,"5min":6.2549999999999999,"15min":6.2549999999999999},"last":0.58799999999999997}]},{"osd":5,"last update":"Fri Feb 23 00:45:23 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.76300000000000001,"15min":0.65900000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72999999999999998,"5min":5.6360000000000001,"15min":5.6360000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":1.016,"15min":0.75600000000000001},"min":{"1min":0.40699999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79300000000000004,"5min":11.597,"15min":11.597},"last":0.55300000000000005}]},{"osd":6,"last update":"Fri Feb 23 00:44:46 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88100000000000001,"15min":0.70199999999999996},"min":{"1min":0.40899999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":6.343,"5min":6.343,"15min":6.343},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.80200000000000005,"15min":0.68600000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":2.8260000000000001,"5min":5.6680000000000001,"15min":5.6680000000000001},"last":0.67500000000000004}]},{"osd":7,"last update":"Fri Feb 23 00:45:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":1.0589999999999999,"15min":0.77800000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.39000000000000001},"max":{"1min":2.6040000000000001,"5min":11.67,"15min":11.67},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.95499999999999996,"15min":0.75700000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":2.6179999999999999,"5min":6.3010000000000002,"15min":6.3010000000000002},"last":0.70499999999999996}]}]},{"osd":4,"up_from":32,"seq":137438953570,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":313984,"kb_used_data":4460,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93431168,"statfs":{"total":95995035648,"available":95673516032,"internally_reserved":0,"allocated":4567040,"data_stored":2682739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:44:59 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.52100000000000002,"15min":0.56100000000000005},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.59499999999999997,"5min":3.5219999999999998,"15min":3.5219999999999998},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.66900000000000004,"15min":0.61399999999999999},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.64100000000000001,"5min":8.6140000000000008,"15min":8.6140000000000008},"last":0.55400000000000005}]},{"osd":1,"last update":"Fri Feb 23 00:44:59 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.66700000000000004,"15min":0.61299999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.27800000000000002},"max":{"1min":0.69799999999999995,"5min":9.6110000000000007,"15min":9.6110000000000007},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.66300000000000003,"15min":0.60499999999999998},"min":{"1min":0.33500000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.67400000000000004,"5min":6.6950000000000003,"15min":6.6950000000000003},"last":0.54200000000000004}]},{"osd":2,"last update":"Fri Feb 23 00:44:59 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.624,"15min":0.59399999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64800000000000002,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68700000000000006,"5min":5.8440000000000003,"15min":5.8440000000000003},"last":0.52700000000000002}]},{"osd":3,"last update":"Fri Feb 23 00:44:59 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.65800000000000003,"15min":0.60599999999999998},"min":{"1min":0.34399999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65900000000000003,"5min":6.5540000000000003,"15min":6.5540000000000003},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.66200000000000003,"15min":0.60899999999999999},"min":{"1min":0.32700000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73699999999999999,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":0.498}]},{"osd":5,"last update":"Fri Feb 23 00:45:23 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.41899999999999998,"15min":0.38300000000000001},"min":{"1min":0.25600000000000001,"5min":0.251,"15min":0.246},"max":{"1min":0.64700000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.48699999999999999,"15min":0.435},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.28899999999999998},"max":{"1min":0.71599999999999997,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.371}]},{"osd":6,"last update":"Fri Feb 23 00:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.45000000000000001,"15min":0.41299999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.68100000000000005,"5min":0.75,"15min":0.75},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.45400000000000001,"15min":0.41299999999999998},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.66100000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.34699999999999998}]},{"osd":7,"last update":"Fri Feb 23 00:45:17 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30599999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.67100000000000004,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.497,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.70599999999999996,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.41899999999999998}]}]},{"osd":5,"up_from":39,"seq":167503724637,"num_pgs":83,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309796,"kb_used_data":4372,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93435356,"statfs":{"total":95995035648,"available":95677804544,"internally_reserved":0,"allocated":4476928,"data_stored":2683049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.78800000000000003,"15min":0.60199999999999998},"min":{"1min":0.28299999999999997,"5min":0.246,"15min":0.246},"max":{"1min":0.67900000000000005,"5min":23.881,"15min":23.881},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.68200000000000005,"15min":0.57899999999999996},"min":{"1min":0.28999999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.57799999999999996,"5min":5.4649999999999999,"15min":5.4649999999999999},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Feb 23 00:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.88600000000000001,"15min":0.64800000000000002},"min":{"1min":0.30199999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.74099999999999999,"5min":18.175999999999998,"15min":18.175999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.78300000000000003,"15min":0.624},"min":{"1min":0.35699999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.60299999999999998,"5min":9.0950000000000006,"15min":9.0950000000000006},"last":0.55800000000000005}]},{"osd":2,"last update":"Fri Feb 23 00:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":1.0169999999999999,"15min":0.70599999999999996},"min":{"1min":0.32600000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.59799999999999998,"5min":22.282,"15min":22.282},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.49199999999999999,"5min":1.0489999999999999,"15min":0.70899999999999996},"min":{"1min":0.33200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.63700000000000001,"5min":23.925000000000001,"15min":23.925000000000001},"last":0.434}]},{"osd":3,"last update":"Fri Feb 23 00:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.77700000000000002,"15min":0.61399999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.63700000000000001,"5min":6.5730000000000004,"15min":6.5730000000000004},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.505,"5min":0.79900000000000004,"15min":0.64000000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33500000000000002},"max":{"1min":0.64800000000000002,"5min":8.1760000000000002,"15min":8.1760000000000002},"last":0.503}]},{"osd":4,"last update":"Fri Feb 23 00:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.42899999999999999,"15min":0.39600000000000002},"min":{"1min":0.26400000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.57899999999999996,"5min":0.68999999999999995,"15min":0.70399999999999996},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.437,"5min":0.51800000000000002,"15min":0.42899999999999999},"min":{"1min":0.30199999999999999,"5min":0.27500000000000002,"15min":0.27300000000000002},"max":{"1min":0.52500000000000002,"5min":3.879,"15min":3.879},"last":0.54000000000000004}]},{"osd":6,"last update":"Fri Feb 23 00:44:44 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.48399999999999999,"15min":0.44400000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.98099999999999998,"5min":1.736,"15min":1.736},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.49299999999999999,"15min":0.45600000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.55000000000000004}]},{"osd":7,"last update":"Fri Feb 23 00:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.57799999999999996,"15min":0.53200000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.71299999999999997,"5min":7.0300000000000002,"15min":7.0300000000000002},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.57799999999999996,"15min":0.53100000000000003},"min":{"1min":0.33400000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.58499999999999996,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":0.51700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":192512,"data_stored":485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":192512,"data_stored":484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":229376,"data_stored":570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-23T00:45:47.820 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph status --format=json 2024-02-23T00:45:47.893 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:45:47.922 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:48.070 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:45:48.447 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:46.675659+0000 mgr.y (mgr.14517) 285 : audit [DBG] from='client.40964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:45:48.447 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:47.164613+0000 mon.a (mon.0) 1860 : audit [INF] from='client.? 172.21.15.47:0/1402665257' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi047-82378-38"}]: dispatch 2024-02-23T00:45:48.448 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:47.165057+0000 mon.a (mon.0) 1861 : audit [INF] from='client.? 172.21.15.47:0/1402665257' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi047-82378-38"}]: dispatch 2024-02-23T00:45:48.448 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:47.165406+0000 mon.a (mon.0) 1862 : audit [INF] from='client.? 172.21.15.47:0/1402665257' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi047-82378-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:48.448 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:47.246356+0000 mgr.y (mgr.14517) 286 : cluster [DBG] pgmap v388: 250 pgs: 250 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s; 325 B/s, 6 objects/s recovering 2024-02-23T00:45:48.448 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:47.247644+0000 mon.a (mon.0) 1863 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-23T00:45:48.448 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:47.254355+0000 mon.a (mon.0) 1864 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:45:48.448 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:47.594327+0000 mon.b (mon.1) 338 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:48.448 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:48.141291+0000 mon.a (mon.0) 1865 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi047-82386-47"}]': finished 2024-02-23T00:45:48.448 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:48.141379+0000 mon.a (mon.0) 1866 : audit [INF] from='client.? 172.21.15.47:0/1402665257' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi047-82378-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:48.448 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:48.141441+0000 mon.a (mon.0) 1867 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-23T00:45:48.448 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:48.141496+0000 mon.a (mon.0) 1868 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:45:48.448 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:48.141541+0000 mon.a (mon.0) 1869 : cluster [DBG] osdmap e276: 8 total, 8 up, 8 in 2024-02-23T00:45:48.448 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:48.144905+0000 mon.c (mon.2) 193 : audit [INF] from='client.? 172.21.15.47:0/2181654827' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi047-82386-47"}]: dispatch 2024-02-23T00:45:48.449 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:48.145418+0000 mon.a (mon.0) 1870 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-31"}]: dispatch 2024-02-23T00:45:48.449 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:48.145703+0000 mon.a (mon.0) 1871 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi047-82386-47"}]: dispatch 2024-02-23T00:45:48.449 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:48.145992+0000 mon.a (mon.0) 1872 : audit [INF] from='client.? 172.21.15.47:0/1402665257' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi047-82378-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi047-82378-38"}]: dispatch 2024-02-23T00:45:48.513 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:46.675659+0000 mgr.y (mgr.14517) 285 : audit [DBG] from='client.40964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:45:48.513 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:47.164613+0000 mon.a (mon.0) 1860 : audit [INF] from='client.? 172.21.15.47:0/1402665257' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi047-82378-38"}]: dispatch 2024-02-23T00:45:48.513 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:47.165057+0000 mon.a (mon.0) 1861 : audit [INF] from='client.? 172.21.15.47:0/1402665257' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi047-82378-38"}]: dispatch 2024-02-23T00:45:48.513 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:47.165406+0000 mon.a (mon.0) 1862 : audit [INF] from='client.? 172.21.15.47:0/1402665257' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi047-82378-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:48.513 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:47.246356+0000 mgr.y (mgr.14517) 286 : cluster [DBG] pgmap v388: 250 pgs: 250 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s; 325 B/s, 6 objects/s recovering 2024-02-23T00:45:48.513 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:47.247644+0000 mon.a (mon.0) 1863 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-23T00:45:48.513 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:47.254355+0000 mon.a (mon.0) 1864 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:45:48.513 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:47.594327+0000 mon.b (mon.1) 338 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:48.513 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:48.141291+0000 mon.a (mon.0) 1865 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi047-82386-47"}]': finished 2024-02-23T00:45:48.513 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:48.141379+0000 mon.a (mon.0) 1866 : audit [INF] from='client.? 172.21.15.47:0/1402665257' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi047-82378-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:48.514 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:48.141441+0000 mon.a (mon.0) 1867 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-23T00:45:48.514 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:48.141496+0000 mon.a (mon.0) 1868 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:45:48.514 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:48.141541+0000 mon.a (mon.0) 1869 : cluster [DBG] osdmap e276: 8 total, 8 up, 8 in 2024-02-23T00:45:48.514 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:46.675659+0000 mgr.y (mgr.14517) 285 : audit [DBG] from='client.40964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:45:48.514 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:47.164613+0000 mon.a (mon.0) 1860 : audit [INF] from='client.? 172.21.15.47:0/1402665257' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi047-82378-38"}]: dispatch 2024-02-23T00:45:48.514 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:47.165057+0000 mon.a (mon.0) 1861 : audit [INF] from='client.? 172.21.15.47:0/1402665257' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi047-82378-38"}]: dispatch 2024-02-23T00:45:48.514 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:47.165406+0000 mon.a (mon.0) 1862 : audit [INF] from='client.? 172.21.15.47:0/1402665257' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi047-82378-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:48.514 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:47.246356+0000 mgr.y (mgr.14517) 286 : cluster [DBG] pgmap v388: 250 pgs: 250 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s; 325 B/s, 6 objects/s recovering 2024-02-23T00:45:48.514 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:47.247644+0000 mon.a (mon.0) 1863 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-23T00:45:48.515 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:47.254355+0000 mon.a (mon.0) 1864 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:45:48.515 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:47.594327+0000 mon.b (mon.1) 338 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:48.515 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:48.141291+0000 mon.a (mon.0) 1865 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi047-82386-47"}]': finished 2024-02-23T00:45:48.515 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:48.141379+0000 mon.a (mon.0) 1866 : audit [INF] from='client.? 172.21.15.47:0/1402665257' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi047-82378-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:48.515 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:48.141441+0000 mon.a (mon.0) 1867 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-23T00:45:48.515 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:48.141496+0000 mon.a (mon.0) 1868 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:45:48.515 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:48.141541+0000 mon.a (mon.0) 1869 : cluster [DBG] osdmap e276: 8 total, 8 up, 8 in 2024-02-23T00:45:48.515 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:48.144905+0000 mon.c (mon.2) 193 : audit [INF] from='client.? 172.21.15.47:0/2181654827' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi047-82386-47"}]: dispatch 2024-02-23T00:45:48.515 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:48.145418+0000 mon.a (mon.0) 1870 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-31"}]: dispatch 2024-02-23T00:45:48.515 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:48.145703+0000 mon.a (mon.0) 1871 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi047-82386-47"}]: dispatch 2024-02-23T00:45:48.515 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:48.145992+0000 mon.a (mon.0) 1872 : audit [INF] from='client.? 172.21.15.47:0/1402665257' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi047-82378-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi047-82378-38"}]: dispatch 2024-02-23T00:45:48.515 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:48.144905+0000 mon.c (mon.2) 193 : audit [INF] from='client.? 172.21.15.47:0/2181654827' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi047-82386-47"}]: dispatch 2024-02-23T00:45:48.516 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:48.145418+0000 mon.a (mon.0) 1870 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-31"}]: dispatch 2024-02-23T00:45:48.516 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:48.145703+0000 mon.a (mon.0) 1871 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi047-82386-47"}]: dispatch 2024-02-23T00:45:48.516 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:48.145992+0000 mon.a (mon.0) 1872 : audit [INF] from='client.? 172.21.15.47:0/1402665257' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi047-82378-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi047-82378-38"}]: dispatch 2024-02-23T00:45:49.373 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:45:49.376 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:49.377 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:45:49.477 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:45:49.507 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:49.544 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:45:49.548 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:49.548 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:45:49.649 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:45:49.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.659+0000 7ff0ac32b700 1 -- 172.21.15.47:0/3299914276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0a40fe510 msgr2=0x7ff0a40f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:49.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.659+0000 7ff0ac32b700 1 --2- 172.21.15.47:0/3299914276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0a40fe510 0x7ff0a40f7d40 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 rx=0x7ff094004870 tx=0x7ff094009cb0).stop 2024-02-23T00:45:49.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.659+0000 7ff0ac32b700 1 -- 172.21.15.47:0/3299914276 shutdown_connections 2024-02-23T00:45:49.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.659+0000 7ff0ac32b700 1 --2- 172.21.15.47:0/3299914276 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff0a40f8420 0x7ff0a4104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:49.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.659+0000 7ff0ac32b700 1 --2- 172.21.15.47:0/3299914276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0a40fdbb0 0x7ff0a40fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:49.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.659+0000 7ff0ac32b700 1 --2- 172.21.15.47:0/3299914276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0a40fe510 0x7ff0a40f7d40 unknown :-1 s=CLOSED pgs=2492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:49.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.659+0000 7ff0ac32b700 1 -- 172.21.15.47:0/3299914276 >> 172.21.15.47:0/3299914276 conn(0x7ff0a40f3800 msgr2=0x7ff0a40f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:45:49.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.659+0000 7ff0ac32b700 1 -- 172.21.15.47:0/3299914276 shutdown_connections 2024-02-23T00:45:49.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.660+0000 7ff0ac32b700 1 -- 172.21.15.47:0/3299914276 wait complete. 2024-02-23T00:45:49.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.660+0000 7ff0ac32b700 1 librados: starting msgr at 2024-02-23T00:45:49.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.660+0000 7ff0ac32b700 1 librados: starting objecter 2024-02-23T00:45:49.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.660+0000 7ff0ac32b700 1 Processor -- start 2024-02-23T00:45:49.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.660+0000 7ff0ac32b700 1 -- start start 2024-02-23T00:45:49.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.660+0000 7ff0ac32b700 1 librados: setting wanted keys 2024-02-23T00:45:49.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.660+0000 7ff0ac32b700 1 librados: calling monclient init 2024-02-23T00:45:49.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.660+0000 7ff0ac32b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0a40f8420 0x7ff0a41100d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:49.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.660+0000 7ff0ac32b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0a40fdbb0 0x7ff0a41105c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:49.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.660+0000 7ff0ac32b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff0a40fe510 0x7ff0a410a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:49.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.660+0000 7ff0ac32b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a410a760 con 0x7ff0a40f8420 2024-02-23T00:45:49.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.660+0000 7ff0ac32b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a410a8d0 con 0x7ff0a40fdbb0 2024-02-23T00:45:49.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.660+0000 7ff0ac32b700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a410aa90 con 0x7ff0a40fe510 2024-02-23T00:45:49.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.661+0000 7ff0aa0c7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0a40f8420 0x7ff0a41100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:49.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.661+0000 7ff0aa0c7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0a40f8420 0x7ff0a41100d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60254/0 (socket says 172.21.15.47:60254) 2024-02-23T00:45:49.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.661+0000 7ff0aa0c7700 1 -- 172.21.15.47:0/3004093349 learned_addr learned my addr 172.21.15.47:0/3004093349 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:45:49.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.661+0000 7ff0a98c6700 1 --2- 172.21.15.47:0/3004093349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0a40fdbb0 0x7ff0a41105c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:49.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.661+0000 7ff0aa8c8700 1 --2- 172.21.15.47:0/3004093349 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff0a40fe510 0x7ff0a410a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:49.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.661+0000 7ff0aa0c7700 1 -- 172.21.15.47:0/3004093349 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff0a40fe510 msgr2=0x7ff0a410a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:49.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.661+0000 7ff0aa0c7700 1 --2- 172.21.15.47:0/3004093349 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff0a40fe510 0x7ff0a410a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:49.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.661+0000 7ff0aa0c7700 1 -- 172.21.15.47:0/3004093349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0a40fdbb0 msgr2=0x7ff0a41105c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:49.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.661+0000 7ff0aa0c7700 1 --2- 172.21.15.47:0/3004093349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0a40fdbb0 0x7ff0a41105c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:49.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.661+0000 7ff0aa0c7700 1 -- 172.21.15.47:0/3004093349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09c0097c0 con 0x7ff0a40f8420 2024-02-23T00:45:49.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.661+0000 7ff0a98c6700 1 --2- 172.21.15.47:0/3004093349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0a40fdbb0 0x7ff0a41105c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:45:49.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.661+0000 7ff0aa0c7700 1 --2- 172.21.15.47:0/3004093349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0a40f8420 0x7ff0a41100d0 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 rx=0x7ff0a000e970 tx=0x7ff0a000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:49.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.661+0000 7ff09b7fe700 10 client.?.objecter ms_handle_connect 0x7ff0a40f8420 2024-02-23T00:45:49.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.661+0000 7ff09b7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:45:49.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.661+0000 7ff09b7fe700 1 -- 172.21.15.47:0/3004093349 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0004660 con 0x7ff0a40f8420 2024-02-23T00:45:49.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.661+0000 7ff0ac32b700 1 -- 172.21.15.47:0/3004093349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff094005040 con 0x7ff0a40f8420 2024-02-23T00:45:49.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.661+0000 7ff09b7fe700 1 -- 172.21.15.47:0/3004093349 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7ff0a000f040 con 0x7ff0a40f8420 2024-02-23T00:45:49.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.661+0000 7ff09b7fe700 1 -- 172.21.15.47:0/3004093349 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a00146f0 con 0x7ff0a40f8420 2024-02-23T00:45:49.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.661+0000 7ff0ac32b700 10 client.30645.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:45:49.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.661+0000 7ff0ac32b700 1 -- 172.21.15.47:0/3004093349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a410b0f0 con 0x7ff0a40f8420 2024-02-23T00:45:49.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.662+0000 7ff0ac32b700 1 librados: init done 2024-02-23T00:45:49.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.663+0000 7ff0ac32b700 1 -- 172.21.15.47:0/3004093349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0a405ee80 con 0x7ff0a40f8420 2024-02-23T00:45:49.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.663+0000 7ff09b7fe700 1 -- 172.21.15.47:0/3004093349 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7ff0a0007b40 con 0x7ff0a40f8420 2024-02-23T00:45:49.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.663+0000 7ff09b7fe700 1 --2- 172.21.15.47:0/3004093349 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff090062cc0 0x7ff090065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:49.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.664+0000 7ff09b7fe700 1 -- 172.21.15.47:0/3004093349 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(277..277 src has 48..277) v4 ==== 9924+0+0 (secure 0 0 0) 0x7ff0a007d3f0 con 0x7ff0a40f8420 2024-02-23T00:45:49.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.664+0000 7ff09b7fe700 10 client.30645.objecter ms_dispatch 0x7ff0a4001550 osd_map(277..277 src has 48..277) v4 2024-02-23T00:45:49.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.664+0000 7ff09b7fe700 3 client.30645.objecter handle_osd_map got epochs [277,277] > 0 2024-02-23T00:45:49.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.664+0000 7ff09b7fe700 3 client.30645.objecter handle_osd_map decoding full epoch 277 2024-02-23T00:45:49.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.664+0000 7ff0a98c6700 1 --2- 172.21.15.47:0/3004093349 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff090062cc0 0x7ff090065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:49.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.664+0000 7ff09b7fe700 20 client.30645.objecter dump_active .. 0 homeless 2024-02-23T00:45:49.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.664+0000 7ff09b7fe700 10 client.30645.objecter ms_handle_connect 0x7ff090062cc0 2024-02-23T00:45:49.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.664+0000 7ff0a98c6700 1 --2- 172.21.15.47:0/3004093349 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff090062cc0 0x7ff090065180 secure :-1 s=READY pgs=3002 cs=0 l=1 rev1=1 rx=0x7ff094004ee0 tx=0x7ff09400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:49.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.668+0000 7ff09b7fe700 1 -- 172.21.15.47:0/3004093349 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ff0a004c7f0 con 0x7ff0a40f8420 2024-02-23T00:45:49.680 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:49.692 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:45:49.700 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:45:49.703 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:49.703 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:45:49.804 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:45:49.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:48.595476+0000 mon.b (mon.1) 339 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:49.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:49.138849+0000 mon.a (mon.0) 1873 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:45:49.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:49.143310+0000 mon.a (mon.0) 1874 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-31"}]': finished 2024-02-23T00:45:49.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:48.595476+0000 mon.b (mon.1) 339 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:49.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:49.138849+0000 mon.a (mon.0) 1873 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:45:49.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:49.143310+0000 mon.a (mon.0) 1874 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-31"}]': finished 2024-02-23T00:45:49.833 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:49.853 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:45:49.855 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:48.595476+0000 mon.b (mon.1) 339 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:45:49.855 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:49.138849+0000 mon.a (mon.0) 1873 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:45:49.855 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:49.143310+0000 mon.a (mon.0) 1874 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-31"}]': finished 2024-02-23T00:45:49.855 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:49.856 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:45:49.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:49.869+0000 7ff0ac32b700 1 -- 172.21.15.47:0/3004093349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7ff0a405fa60 con 0x7ff0a40f8420 2024-02-23T00:45:49.957 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:45:49.987 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:50.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:50.149+0000 7ff09b7fe700 1 -- 172.21.15.47:0/3004093349 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v278) v1 ==== 91+0+0 (secure 0 0 0) 0x7ff0a0019070 con 0x7ff0a40f8420 2024-02-23T00:45:50.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:50.153+0000 7ff0ac32b700 10 librados: watch_flush enter 2024-02-23T00:45:50.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:50.157+0000 7ff0ac32b700 10 librados: watch_flush exit 2024-02-23T00:45:50.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:50.157+0000 7ff0ac32b700 20 client.30645.objecter shutdown clearing up homeless session... 2024-02-23T00:45:50.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:50.157+0000 7ff0ac32b700 10 client.30645.objecter successfully canceled tick 2024-02-23T00:45:50.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:50.158+0000 7ff0ac32b700 1 -- 172.21.15.47:0/3004093349 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff090062cc0 msgr2=0x7ff090065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:50.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:50.158+0000 7ff0ac32b700 1 --2- 172.21.15.47:0/3004093349 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff090062cc0 0x7ff090065180 secure :-1 s=READY pgs=3002 cs=0 l=1 rev1=1 rx=0x7ff094004ee0 tx=0x7ff09400b040).stop 2024-02-23T00:45:50.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:50.158+0000 7ff0ac32b700 1 -- 172.21.15.47:0/3004093349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0a40f8420 msgr2=0x7ff0a41100d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:50.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:50.158+0000 7ff0ac32b700 1 --2- 172.21.15.47:0/3004093349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0a40f8420 0x7ff0a41100d0 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 rx=0x7ff0a000e970 tx=0x7ff0a000b040).stop 2024-02-23T00:45:50.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:50.162+0000 7ff0ac32b700 1 -- 172.21.15.47:0/3004093349 shutdown_connections 2024-02-23T00:45:50.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:50.162+0000 7ff0ac32b700 1 --2- 172.21.15.47:0/3004093349 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff090062cc0 0x7ff090065180 unknown :-1 s=CLOSED pgs=3002 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:50.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:50.162+0000 7ff0ac32b700 1 --2- 172.21.15.47:0/3004093349 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff0a40fe510 0x7ff0a410a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:50.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:50.162+0000 7ff0ac32b700 1 --2- 172.21.15.47:0/3004093349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0a40f8420 0x7ff0a41100d0 unknown :-1 s=CLOSED pgs=2493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:50.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:50.162+0000 7ff0ac32b700 1 --2- 172.21.15.47:0/3004093349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0a40fdbb0 0x7ff0a41105c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:50.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:50.162+0000 7ff0ac32b700 1 -- 172.21.15.47:0/3004093349 >> 172.21.15.47:0/3004093349 conn(0x7ff0a40f3800 msgr2=0x7ff0a406af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:45:50.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:50.170+0000 7ff0ac32b700 1 -- 172.21.15.47:0/3004093349 shutdown_connections 2024-02-23T00:45:50.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:50.170+0000 7ff0ac32b700 1 -- 172.21.15.47:0/3004093349 wait complete. 2024-02-23T00:45:50.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:50.175+0000 7ff0ac32b700 1 librados: shutdown 2024-02-23T00:45:50.178 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is unset 2024-02-23T00:45:50.240 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:45:50.243 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:50.244 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:45:50.345 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:45:50.372 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:50.807 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:45:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:45:50] "GET /metrics HTTP/1.1" 200 190059 "" "Prometheus/2.33.4" 2024-02-23T00:45:50.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:49.143372+0000 mon.a (mon.0) 1875 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi047-82386-47"}]': finished 2024-02-23T00:45:50.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:49.165531+0000 mon.c (mon.2) 194 : audit [INF] from='client.? 172.21.15.47:0/3615725416' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi047-82386-48"}]: dispatch 2024-02-23T00:45:50.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:49.246791+0000 mgr.y (mgr.14517) 287 : cluster [DBG] pgmap v391: 250 pgs: 250 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s; 325 B/s, 6 objects/s recovering 2024-02-23T00:45:50.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:49.579752+0000 mon.a (mon.0) 1876 : cluster [DBG] osdmap e277: 8 total, 8 up, 8 in 2024-02-23T00:45:50.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:49.581895+0000 mon.a (mon.0) 1877 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi047-82386-48"}]: dispatch 2024-02-23T00:45:50.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:49.585709+0000 mon.c (mon.2) 195 : audit [INF] from='client.? 172.21.15.47:0/3615725416' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi047-82386-48"}]: dispatch 2024-02-23T00:45:50.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:49.586887+0000 mon.a (mon.0) 1878 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi047-82386-48"}]: dispatch 2024-02-23T00:45:50.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:49.587491+0000 mon.c (mon.2) 196 : audit [INF] from='client.? 172.21.15.47:0/3615725416' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi047-82386-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:50.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:49.587869+0000 mon.a (mon.0) 1879 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi047-82386-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:50.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:49.870840+0000 mon.a (mon.0) 1880 : audit [INF] from='client.? 172.21.15.47:0/3004093349' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:45:50.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:50.140897+0000 mon.a (mon.0) 1881 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:45:50.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:50.150077+0000 mon.a (mon.0) 1882 : audit [INF] from='client.? 172.21.15.47:0/1402665257' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi047-82378-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi047-82378-38"}]': finished 2024-02-23T00:45:50.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:50.150209+0000 mon.a (mon.0) 1883 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi047-82386-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:50.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:50.150319+0000 mon.a (mon.0) 1884 : audit [INF] from='client.? 172.21.15.47:0/3004093349' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:45:50.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:50.150378+0000 mon.a (mon.0) 1885 : cluster [DBG] osdmap e278: 8 total, 8 up, 8 in 2024-02-23T00:45:50.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:49.143372+0000 mon.a (mon.0) 1875 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi047-82386-47"}]': finished 2024-02-23T00:45:50.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:49.165531+0000 mon.c (mon.2) 194 : audit [INF] from='client.? 172.21.15.47:0/3615725416' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi047-82386-48"}]: dispatch 2024-02-23T00:45:50.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:49.246791+0000 mgr.y (mgr.14517) 287 : cluster [DBG] pgmap v391: 250 pgs: 250 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s; 325 B/s, 6 objects/s recovering 2024-02-23T00:45:50.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:49.579752+0000 mon.a (mon.0) 1876 : cluster [DBG] osdmap e277: 8 total, 8 up, 8 in 2024-02-23T00:45:50.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:49.581895+0000 mon.a (mon.0) 1877 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi047-82386-48"}]: dispatch 2024-02-23T00:45:50.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:49.585709+0000 mon.c (mon.2) 195 : audit [INF] from='client.? 172.21.15.47:0/3615725416' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi047-82386-48"}]: dispatch 2024-02-23T00:45:50.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:49.586887+0000 mon.a (mon.0) 1878 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi047-82386-48"}]: dispatch 2024-02-23T00:45:50.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:49.587491+0000 mon.c (mon.2) 196 : audit [INF] from='client.? 172.21.15.47:0/3615725416' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi047-82386-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:50.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:49.587869+0000 mon.a (mon.0) 1879 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi047-82386-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:50.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:49.870840+0000 mon.a (mon.0) 1880 : audit [INF] from='client.? 172.21.15.47:0/3004093349' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:45:50.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:50.140897+0000 mon.a (mon.0) 1881 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:45:50.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:50.150077+0000 mon.a (mon.0) 1882 : audit [INF] from='client.? 172.21.15.47:0/1402665257' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi047-82378-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi047-82378-38"}]': finished 2024-02-23T00:45:50.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:50.150209+0000 mon.a (mon.0) 1883 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi047-82386-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:50.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:50.150319+0000 mon.a (mon.0) 1884 : audit [INF] from='client.? 172.21.15.47:0/3004093349' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:45:50.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:50.150378+0000 mon.a (mon.0) 1885 : cluster [DBG] osdmap e278: 8 total, 8 up, 8 in 2024-02-23T00:45:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:45:50.924 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:50.924 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:45:50.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:49.143372+0000 mon.a (mon.0) 1875 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi047-82386-47"}]': finished 2024-02-23T00:45:50.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:49.165531+0000 mon.c (mon.2) 194 : audit [INF] from='client.? 172.21.15.47:0/3615725416' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi047-82386-48"}]: dispatch 2024-02-23T00:45:50.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:49.246791+0000 mgr.y (mgr.14517) 287 : cluster [DBG] pgmap v391: 250 pgs: 250 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s; 325 B/s, 6 objects/s recovering 2024-02-23T00:45:50.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:49.579752+0000 mon.a (mon.0) 1876 : cluster [DBG] osdmap e277: 8 total, 8 up, 8 in 2024-02-23T00:45:50.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:49.581895+0000 mon.a (mon.0) 1877 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi047-82386-48"}]: dispatch 2024-02-23T00:45:50.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:49.585709+0000 mon.c (mon.2) 195 : audit [INF] from='client.? 172.21.15.47:0/3615725416' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi047-82386-48"}]: dispatch 2024-02-23T00:45:50.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:49.586887+0000 mon.a (mon.0) 1878 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi047-82386-48"}]: dispatch 2024-02-23T00:45:50.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:49.587491+0000 mon.c (mon.2) 196 : audit [INF] from='client.? 172.21.15.47:0/3615725416' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi047-82386-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:50.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:49.587869+0000 mon.a (mon.0) 1879 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi047-82386-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:50.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:49.870840+0000 mon.a (mon.0) 1880 : audit [INF] from='client.? 172.21.15.47:0/3004093349' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:45:50.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:50.140897+0000 mon.a (mon.0) 1881 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:45:50.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:50.150077+0000 mon.a (mon.0) 1882 : audit [INF] from='client.? 172.21.15.47:0/1402665257' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi047-82378-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi047-82378-38"}]': finished 2024-02-23T00:45:50.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:50.150209+0000 mon.a (mon.0) 1883 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi047-82386-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:50.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:50.150319+0000 mon.a (mon.0) 1884 : audit [INF] from='client.? 172.21.15.47:0/3004093349' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:45:50.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:50.150378+0000 mon.a (mon.0) 1885 : cluster [DBG] osdmap e278: 8 total, 8 up, 8 in 2024-02-23T00:45:51.025 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:45:51.051 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:51.069 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:45:51.072 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:51.072 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:45:51.173 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:45:51.202 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:51.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:50.170185+0000 mon.c (mon.2) 197 : audit [INF] from='client.? 172.21.15.47:0/3615725416' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi047-82386-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi047-82386-48"}]: dispatch 2024-02-23T00:45:51.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:50.602031+0000 mon.a (mon.0) 1886 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:45:51.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:50.602739+0000 mon.a (mon.0) 1887 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi047-82386-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi047-82386-48"}]: dispatch 2024-02-23T00:45:51.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:51.024772+0000 mon.a (mon.0) 1888 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:45:51.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:51.025472+0000 mon.a (mon.0) 1889 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:45:51.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:51.030516+0000 mon.a (mon.0) 1890 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:45:51.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:51.102576+0000 osd.2 (osd.2) 121 : cluster [DBG] 15.1a scrub starts 2024-02-23T00:45:51.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:51.103682+0000 osd.2 (osd.2) 122 : cluster [DBG] 15.1a scrub ok 2024-02-23T00:45:51.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:51.586515+0000 mon.a (mon.0) 1891 : cluster [DBG] osdmap e279: 8 total, 8 up, 8 in 2024-02-23T00:45:51.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:51.589049+0000 mon.a (mon.0) 1892 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-02-23T00:45:51.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:51.589404+0000 mon.a (mon.0) 1893 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:52.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:50.170185+0000 mon.c (mon.2) 197 : audit [INF] from='client.? 172.21.15.47:0/3615725416' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi047-82386-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi047-82386-48"}]: dispatch 2024-02-23T00:45:52.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:50.602031+0000 mon.a (mon.0) 1886 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:45:52.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:50.602739+0000 mon.a (mon.0) 1887 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi047-82386-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi047-82386-48"}]: dispatch 2024-02-23T00:45:52.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:51.024772+0000 mon.a (mon.0) 1888 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:45:52.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:51.025472+0000 mon.a (mon.0) 1889 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:45:52.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:51.030516+0000 mon.a (mon.0) 1890 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:45:52.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:51.102576+0000 osd.2 (osd.2) 121 : cluster [DBG] 15.1a scrub starts 2024-02-23T00:45:52.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:51.103682+0000 osd.2 (osd.2) 122 : cluster [DBG] 15.1a scrub ok 2024-02-23T00:45:52.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:51.586515+0000 mon.a (mon.0) 1891 : cluster [DBG] osdmap e279: 8 total, 8 up, 8 in 2024-02-23T00:45:52.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:51.589049+0000 mon.a (mon.0) 1892 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-02-23T00:45:52.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:51.589404+0000 mon.a (mon.0) 1893 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:52.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:50.170185+0000 mon.c (mon.2) 197 : audit [INF] from='client.? 172.21.15.47:0/3615725416' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi047-82386-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi047-82386-48"}]: dispatch 2024-02-23T00:45:52.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:50.602031+0000 mon.a (mon.0) 1886 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:45:52.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:50.602739+0000 mon.a (mon.0) 1887 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi047-82386-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi047-82386-48"}]: dispatch 2024-02-23T00:45:52.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:51.024772+0000 mon.a (mon.0) 1888 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:45:52.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:51.025472+0000 mon.a (mon.0) 1889 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:45:52.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:51.030516+0000 mon.a (mon.0) 1890 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:45:52.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:51.102576+0000 osd.2 (osd.2) 121 : cluster [DBG] 15.1a scrub starts 2024-02-23T00:45:52.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:51.103682+0000 osd.2 (osd.2) 122 : cluster [DBG] 15.1a scrub ok 2024-02-23T00:45:52.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:51.586515+0000 mon.a (mon.0) 1891 : cluster [DBG] osdmap e279: 8 total, 8 up, 8 in 2024-02-23T00:45:52.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:51.589049+0000 mon.a (mon.0) 1892 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-02-23T00:45:52.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:51.589404+0000 mon.a (mon.0) 1893 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:45:52.151 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:45:52.154 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:52.155 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:45:52.256 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:45:52.288 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:52.310 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:45:52.312 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:52.313 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:45:52.413 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:45:52.442 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:52.608 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:45:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:45:52.627 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:45:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:45:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:45:52.784 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:50.684950+0000 osd.0 (osd.0) 117 : cluster [DBG] 6.9 scrub starts 2024-02-23T00:45:52.784 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:50.685820+0000 osd.0 (osd.0) 118 : cluster [DBG] 6.9 scrub ok 2024-02-23T00:45:52.784 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:51.247224+0000 mgr.y (mgr.14517) 288 : cluster [DBG] pgmap v394: 257 pgs: 40 unknown, 217 active+clean; 8.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:45:52.784 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:51.257204+0000 osd.4 (osd.4) 123 : cluster [DBG] 62.d scrub starts 2024-02-23T00:45:52.784 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:51.258737+0000 osd.4 (osd.4) 124 : cluster [DBG] 62.d scrub ok 2024-02-23T00:45:52.784 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:51.868150+0000 osd.3 (osd.3) 147 : cluster [DBG] 5.0 scrub starts 2024-02-23T00:45:52.784 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:51.871614+0000 osd.3 (osd.3) 148 : cluster [DBG] 5.0 scrub ok 2024-02-23T00:45:52.784 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:51.982089+0000 osd.5 (osd.5) 121 : cluster [DBG] 18.c scrub starts 2024-02-23T00:45:52.784 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:51.985019+0000 osd.5 (osd.5) 122 : cluster [DBG] 18.c scrub ok 2024-02-23T00:45:52.784 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:52.064969+0000 osd.2 (osd.2) 123 : cluster [DBG] 62.12 scrub starts 2024-02-23T00:45:52.785 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:52.066502+0000 osd.2 (osd.2) 124 : cluster [DBG] 62.12 scrub ok 2024-02-23T00:45:52.785 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:52.155974+0000 mon.a (mon.0) 1894 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi047-82386-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi047-82386-48"}]': finished 2024-02-23T00:45:52.785 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:52.156257+0000 mon.a (mon.0) 1895 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "30"}]': finished 2024-02-23T00:45:52.785 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:52.156344+0000 mon.a (mon.0) 1896 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:52.785 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:52.156409+0000 mon.a (mon.0) 1897 : cluster [DBG] osdmap e280: 8 total, 8 up, 8 in 2024-02-23T00:45:52.785 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:52.166600+0000 mon.a (mon.0) 1898 : audit [INF] from='client.? 172.21.15.47:0/1402665257' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi047-82378-38"}]: dispatch 2024-02-23T00:45:52.785 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:52.170212+0000 mon.a (mon.0) 1899 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:45:52.785 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:50.684950+0000 osd.0 (osd.0) 117 : cluster [DBG] 6.9 scrub starts 2024-02-23T00:45:52.785 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:50.685820+0000 osd.0 (osd.0) 118 : cluster [DBG] 6.9 scrub ok 2024-02-23T00:45:52.785 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:51.247224+0000 mgr.y (mgr.14517) 288 : cluster [DBG] pgmap v394: 257 pgs: 40 unknown, 217 active+clean; 8.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:45:52.785 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:51.257204+0000 osd.4 (osd.4) 123 : cluster [DBG] 62.d scrub starts 2024-02-23T00:45:52.786 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:51.258737+0000 osd.4 (osd.4) 124 : cluster [DBG] 62.d scrub ok 2024-02-23T00:45:52.786 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:51.868150+0000 osd.3 (osd.3) 147 : cluster [DBG] 5.0 scrub starts 2024-02-23T00:45:52.786 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:51.871614+0000 osd.3 (osd.3) 148 : cluster [DBG] 5.0 scrub ok 2024-02-23T00:45:52.786 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:51.982089+0000 osd.5 (osd.5) 121 : cluster [DBG] 18.c scrub starts 2024-02-23T00:45:52.786 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:51.985019+0000 osd.5 (osd.5) 122 : cluster [DBG] 18.c scrub ok 2024-02-23T00:45:52.786 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:52.064969+0000 osd.2 (osd.2) 123 : cluster [DBG] 62.12 scrub starts 2024-02-23T00:45:52.786 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:52.066502+0000 osd.2 (osd.2) 124 : cluster [DBG] 62.12 scrub ok 2024-02-23T00:45:52.786 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:52.155974+0000 mon.a (mon.0) 1894 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi047-82386-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi047-82386-48"}]': finished 2024-02-23T00:45:52.786 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:52.156257+0000 mon.a (mon.0) 1895 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "30"}]': finished 2024-02-23T00:45:52.786 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:52.156344+0000 mon.a (mon.0) 1896 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:52.786 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:52.156409+0000 mon.a (mon.0) 1897 : cluster [DBG] osdmap e280: 8 total, 8 up, 8 in 2024-02-23T00:45:52.786 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:52.166600+0000 mon.a (mon.0) 1898 : audit [INF] from='client.? 172.21.15.47:0/1402665257' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi047-82378-38"}]: dispatch 2024-02-23T00:45:52.786 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:52.170212+0000 mon.a (mon.0) 1899 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:45:52.792 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:45:52.795 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:52.796 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:45:52.896 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:45:52.928 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:50.684950+0000 osd.0 (osd.0) 117 : cluster [DBG] 6.9 scrub starts 2024-02-23T00:45:52.928 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:50.685820+0000 osd.0 (osd.0) 118 : cluster [DBG] 6.9 scrub ok 2024-02-23T00:45:52.928 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:51.247224+0000 mgr.y (mgr.14517) 288 : cluster [DBG] pgmap v394: 257 pgs: 40 unknown, 217 active+clean; 8.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:45:52.928 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:51.257204+0000 osd.4 (osd.4) 123 : cluster [DBG] 62.d scrub starts 2024-02-23T00:45:52.928 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:51.258737+0000 osd.4 (osd.4) 124 : cluster [DBG] 62.d scrub ok 2024-02-23T00:45:52.928 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:51.868150+0000 osd.3 (osd.3) 147 : cluster [DBG] 5.0 scrub starts 2024-02-23T00:45:52.929 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:51.871614+0000 osd.3 (osd.3) 148 : cluster [DBG] 5.0 scrub ok 2024-02-23T00:45:52.929 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:51.982089+0000 osd.5 (osd.5) 121 : cluster [DBG] 18.c scrub starts 2024-02-23T00:45:52.929 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:51.985019+0000 osd.5 (osd.5) 122 : cluster [DBG] 18.c scrub ok 2024-02-23T00:45:52.929 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:52.064969+0000 osd.2 (osd.2) 123 : cluster [DBG] 62.12 scrub starts 2024-02-23T00:45:52.929 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:52.066502+0000 osd.2 (osd.2) 124 : cluster [DBG] 62.12 scrub ok 2024-02-23T00:45:52.929 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:52.155974+0000 mon.a (mon.0) 1894 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi047-82386-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi047-82386-48"}]': finished 2024-02-23T00:45:52.929 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:52.156257+0000 mon.a (mon.0) 1895 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "30"}]': finished 2024-02-23T00:45:52.929 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:52.156344+0000 mon.a (mon.0) 1896 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:45:52.929 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:52.156409+0000 mon.a (mon.0) 1897 : cluster [DBG] osdmap e280: 8 total, 8 up, 8 in 2024-02-23T00:45:52.929 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:52.166600+0000 mon.a (mon.0) 1898 : audit [INF] from='client.? 172.21.15.47:0/1402665257' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi047-82378-38"}]: dispatch 2024-02-23T00:45:52.929 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:52.170212+0000 mon.a (mon.0) 1899 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:45:52.930 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:52.951 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:45:52.954 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:52.955 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:45:53.055 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:45:53.090 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:53.113 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:45:53.115 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:53.116 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:45:53.217 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:45:53.247 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:53.268 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:45:53.270 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:53.271 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:45:53.371 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:45:53.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.313+0000 7f6eb04dc700 1 -- 172.21.15.47:0/2539883444 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6ea8102210 msgr2=0x7f6ea8104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:53.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.313+0000 7f6eb04dc700 1 --2- 172.21.15.47:0/2539883444 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6ea8102210 0x7f6ea8104600 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 rx=0x7f6ea0009c30 tx=0x7f6ea0004c50).stop 2024-02-23T00:45:53.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.313+0000 7f6eb04dc700 1 -- 172.21.15.47:0/2539883444 shutdown_connections 2024-02-23T00:45:53.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.313+0000 7f6eb04dc700 1 --2- 172.21.15.47:0/2539883444 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6ea8102210 0x7f6ea8104600 unknown :-1 s=CLOSED pgs=2588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:53.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.313+0000 7f6eb04dc700 1 --2- 172.21.15.47:0/2539883444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ea80f8cb0 0x7f6ea80f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:53.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.313+0000 7f6eb04dc700 1 --2- 172.21.15.47:0/2539883444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ea80f9610 0x7f6ea8101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:53.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.313+0000 7f6eb04dc700 1 -- 172.21.15.47:0/2539883444 >> 172.21.15.47:0/2539883444 conn(0x7f6ea80f2820 msgr2=0x7f6ea80f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:45:53.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.313+0000 7f6eb04dc700 1 -- 172.21.15.47:0/2539883444 shutdown_connections 2024-02-23T00:45:53.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.313+0000 7f6eb04dc700 1 -- 172.21.15.47:0/2539883444 wait complete. 2024-02-23T00:45:53.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.313+0000 7f6eb04dc700 1 librados: starting msgr at 2024-02-23T00:45:53.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.313+0000 7f6eb04dc700 1 librados: starting objecter 2024-02-23T00:45:53.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.314+0000 7f6eb04dc700 1 Processor -- start 2024-02-23T00:45:53.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.314+0000 7f6eb04dc700 1 -- start start 2024-02-23T00:45:53.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.314+0000 7f6eb04dc700 1 librados: setting wanted keys 2024-02-23T00:45:53.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.314+0000 7f6eb04dc700 1 librados: calling monclient init 2024-02-23T00:45:53.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.314+0000 7f6eb04dc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6ea80f8cb0 0x7f6ea810de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:53.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.314+0000 7f6eb04dc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ea80f9610 0x7f6ea810e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:53.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.314+0000 7f6eb04dc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ea8102210 0x7f6ea8108040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:53.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.314+0000 7f6eb04dc700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea8108530 con 0x7f6ea8102210 2024-02-23T00:45:53.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.314+0000 7f6eb04dc700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea81086a0 con 0x7f6ea80f9610 2024-02-23T00:45:53.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.314+0000 7f6eb04dc700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea8108810 con 0x7f6ea80f8cb0 2024-02-23T00:45:53.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.314+0000 7f6eae278700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6ea80f8cb0 0x7f6ea810de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:53.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.314+0000 7f6eaea79700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ea8102210 0x7f6ea8108040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:53.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.314+0000 7f6eae278700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6ea80f8cb0 0x7f6ea810de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:39856/0 (socket says 172.21.15.47:39856) 2024-02-23T00:45:53.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.314+0000 7f6eae278700 1 -- 172.21.15.47:0/1663515292 learned_addr learned my addr 172.21.15.47:0/1663515292 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:45:53.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.314+0000 7f6eada77700 1 --2- 172.21.15.47:0/1663515292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ea80f9610 0x7f6ea810e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:53.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.314+0000 7f6eae278700 1 -- 172.21.15.47:0/1663515292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ea80f9610 msgr2=0x7f6ea810e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:53.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.314+0000 7f6eae278700 1 --2- 172.21.15.47:0/1663515292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ea80f9610 0x7f6ea810e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:53.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.314+0000 7f6eae278700 1 -- 172.21.15.47:0/1663515292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ea8102210 msgr2=0x7f6ea8108040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:53.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.314+0000 7f6eae278700 1 --2- 172.21.15.47:0/1663515292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ea8102210 0x7f6ea8108040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:53.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.314+0000 7f6eae278700 1 -- 172.21.15.47:0/1663515292 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea4005040 con 0x7f6ea80f8cb0 2024-02-23T00:45:53.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.314+0000 7f6eae278700 1 --2- 172.21.15.47:0/1663515292 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6ea80f8cb0 0x7f6ea810de60 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 rx=0x7f6e980041d0 tx=0x7f6e98009f60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:53.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.314+0000 7f6e9f7fe700 10 client.?.objecter ms_handle_connect 0x7f6ea80f8cb0 2024-02-23T00:45:53.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.314+0000 7f6e9f7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:45:53.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.315+0000 7f6e9f7fe700 1 -- 172.21.15.47:0/1663515292 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e98007b40 con 0x7f6ea80f8cb0 2024-02-23T00:45:53.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.315+0000 7f6e9f7fe700 1 -- 172.21.15.47:0/1663515292 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f6e9800f040 con 0x7f6ea80f8cb0 2024-02-23T00:45:53.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.315+0000 7f6e9f7fe700 1 -- 172.21.15.47:0/1663515292 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e98014730 con 0x7f6ea80f8cb0 2024-02-23T00:45:53.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.315+0000 7f6eb04dc700 1 -- 172.21.15.47:0/1663515292 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea00097c0 con 0x7f6ea80f8cb0 2024-02-23T00:45:53.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.315+0000 7f6eb04dc700 10 client.40985.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:45:53.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.315+0000 7f6eb04dc700 1 -- 172.21.15.47:0/1663515292 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea8108e00 con 0x7f6ea80f8cb0 2024-02-23T00:45:53.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.315+0000 7f6eb04dc700 1 librados: init done 2024-02-23T00:45:53.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.316+0000 7f6eb04dc700 1 -- 172.21.15.47:0/1663515292 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ea805fa60 con 0x7f6ea80f8cb0 2024-02-23T00:45:53.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.316+0000 7f6e9f7fe700 1 -- 172.21.15.47:0/1663515292 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f6e98009110 con 0x7f6ea80f8cb0 2024-02-23T00:45:53.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.316+0000 7f6e9f7fe700 1 --2- 172.21.15.47:0/1663515292 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6e94062cc0 0x7f6e94065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:53.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.316+0000 7f6e9f7fe700 1 -- 172.21.15.47:0/1663515292 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(281..281 src has 48..281) v4 ==== 10448+0+0 (secure 0 0 0) 0x7f6e9807db40 con 0x7f6ea80f8cb0 2024-02-23T00:45:53.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.316+0000 7f6e9f7fe700 10 client.40985.objecter ms_dispatch 0x7f6ea8001550 osd_map(281..281 src has 48..281) v4 2024-02-23T00:45:53.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.316+0000 7f6e9f7fe700 3 client.40985.objecter handle_osd_map got epochs [281,281] > 0 2024-02-23T00:45:53.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.316+0000 7f6e9f7fe700 3 client.40985.objecter handle_osd_map decoding full epoch 281 2024-02-23T00:45:53.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.317+0000 7f6eada77700 1 --2- 172.21.15.47:0/1663515292 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6e94062cc0 0x7f6e94065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:53.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.317+0000 7f6e9f7fe700 20 client.40985.objecter dump_active .. 0 homeless 2024-02-23T00:45:53.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.317+0000 7f6e9f7fe700 10 client.40985.objecter ms_handle_connect 0x7f6e94062cc0 2024-02-23T00:45:53.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.317+0000 7f6eada77700 1 --2- 172.21.15.47:0/1663515292 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6e94062cc0 0x7f6e94065180 secure :-1 s=READY pgs=3003 cs=0 l=1 rev1=1 rx=0x7f6ea81097e0 tx=0x7f6ea400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:53.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.321+0000 7f6e9f7fe700 1 -- 172.21.15.47:0/1663515292 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f6e9804cc90 con 0x7f6ea80f8cb0 2024-02-23T00:45:53.406 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:53.425 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:45:53.428 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:53.428 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:45:53.438 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:45:53.529 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:45:53.558 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:53.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.573+0000 7f6eb04dc700 1 -- 172.21.15.47:0/1663515292 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f6ea8109520 con 0x7f6ea80f8cb0 2024-02-23T00:45:53.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.574+0000 7f6e9f7fe700 1 -- 172.21.15.47:0/1663515292 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+2458 (secure 0 0 0) 0x7f6e980502b0 con 0x7f6ea80f8cb0 2024-02-23T00:45:53.575 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:45:53.576 INFO:teuthology.orchestra.run.smithi047.stdout:{"fsid":"fce2617e-d1e2-11ee-95c0-87774f69a715","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"nodeep-scrub flag(s) set","count":12},"muted":false},"POOL_APP_NOT_ENABLED":{"severity":"HEALTH_WARN","summary":{"message":"1 pool(s) do not have an application enabled","count":1},"muted":false}},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":638,"monmap":{"epoch":3,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":281,"num_osds":8,"num_up_osds":8,"osd_up_since":1708648745,"num_in_osds":8,"osd_in_since":1708648727,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":217},{"state_name":"unknown","count":40}],"num_pgs":257,"num_pools":11,"num_objects":360,"data_bytes":8931,"bytes_used":2547396608,"bytes_avail":765412888576,"bytes_total":767960285184,"unknown_pgs_ratio":0.15564201772212982},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"services":{"dashboard":"https://172.21.15.47:8443/","prometheus":"http://172.21.15.47:9283/"}},"servicemap":{"epoch":23,"modified":"2024-02-23T00:43:27.194339+0000","services":{"rgw":{"daemons":{"summary":"","14460":{"start_epoch":12,"start_stamp":"2024-02-23T00:39:21.905359+0000","gid":14460,"addr":"172.21.15.47:0/1573276039","metadata":{"arch":"x86_64","ceph_release":"pacific","ceph_version":"ceph version 16.2.14-550-geb66ed92 (eb66ed921e744301e6863be6353618d63967ea59) pacific (stable)","ceph_version_short":"16.2.14-550-geb66ed92","container_hostname":"smithi047","container_image":"quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:44277403522cf37edeee63b04643fd6c6e9416f13c3731a771e3cc8e53b7d010","cpu":"Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz","distro":"centos","distro_description":"CentOS Stream 8","distro_version":"8","frontend_config#0":"beast port=80","frontend_type#0":"beast","hostname":"smithi047","id":"foo.a","kernel_description":"#1 SMP Fri Feb 9 14:58:00 UTC 2024","kernel_version":"4.18.0-540.el8.x86_64","mem_swap_kb":"0","mem_total_kb":"32543848","num_handles":"1","os":"Linux","pid":"7","realm_id":"","realm_name":"","zone_id":"52970d38-d172-4ec0-ad23-3afdfeba31e4","zone_name":"default","zonegroup_id":"343d5407-1492-440c-b380-09e4750e9af7","zonegroup_name":"default"},"task_status":{}}}}}},"progress_events":{}} 2024-02-23T00:45:53.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.576+0000 7f6eb04dc700 10 librados: watch_flush enter 2024-02-23T00:45:53.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.576+0000 7f6eb04dc700 10 librados: watch_flush exit 2024-02-23T00:45:53.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.576+0000 7f6eb04dc700 20 client.40985.objecter shutdown clearing up homeless session... 2024-02-23T00:45:53.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.576+0000 7f6eb04dc700 10 client.40985.objecter successfully canceled tick 2024-02-23T00:45:53.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.576+0000 7f6eb04dc700 1 -- 172.21.15.47:0/1663515292 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6e94062cc0 msgr2=0x7f6e94065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:53.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.576+0000 7f6eb04dc700 1 --2- 172.21.15.47:0/1663515292 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6e94062cc0 0x7f6e94065180 secure :-1 s=READY pgs=3003 cs=0 l=1 rev1=1 rx=0x7f6ea81097e0 tx=0x7f6ea400b040).stop 2024-02-23T00:45:53.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.576+0000 7f6eb04dc700 1 -- 172.21.15.47:0/1663515292 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6ea80f8cb0 msgr2=0x7f6ea810de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:53.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.576+0000 7f6eb04dc700 1 --2- 172.21.15.47:0/1663515292 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6ea80f8cb0 0x7f6ea810de60 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 rx=0x7f6e980041d0 tx=0x7f6e98009f60).stop 2024-02-23T00:45:53.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.576+0000 7f6eb04dc700 1 -- 172.21.15.47:0/1663515292 shutdown_connections 2024-02-23T00:45:53.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.576+0000 7f6eb04dc700 1 --2- 172.21.15.47:0/1663515292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ea8102210 0x7f6ea8108040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:53.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.576+0000 7f6eb04dc700 1 --2- 172.21.15.47:0/1663515292 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6e94062cc0 0x7f6e94065180 unknown :-1 s=CLOSED pgs=3003 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:53.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.576+0000 7f6eb04dc700 1 --2- 172.21.15.47:0/1663515292 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6ea80f8cb0 0x7f6ea810de60 unknown :-1 s=CLOSED pgs=2589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:53.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.576+0000 7f6eb04dc700 1 --2- 172.21.15.47:0/1663515292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ea80f9610 0x7f6ea810e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:53.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.576+0000 7f6eb04dc700 1 -- 172.21.15.47:0/1663515292 >> 172.21.15.47:0/1663515292 conn(0x7f6ea80f2820 msgr2=0x7f6ea80f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:45:53.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.576+0000 7f6eb04dc700 1 -- 172.21.15.47:0/1663515292 shutdown_connections 2024-02-23T00:45:53.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.576+0000 7f6eb04dc700 1 -- 172.21.15.47:0/1663515292 wait complete. 2024-02-23T00:45:53.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:53.576+0000 7f6eb04dc700 1 librados: shutdown 2024-02-23T00:45:53.756 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-23T00:45:53.782 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:52.466729+0000 mgr.y (mgr.14517) 289 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:45:53.782 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:52.935983+0000 osd.5 (osd.5) 123 : cluster [DBG] 4.4 scrub starts 2024-02-23T00:45:53.782 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:52.939447+0000 osd.5 (osd.5) 124 : cluster [DBG] 4.4 scrub ok 2024-02-23T00:45:53.783 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:53.065390+0000 osd.2 (osd.2) 125 : cluster [DBG] 62.3 scrub starts 2024-02-23T00:45:53.783 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:53.067726+0000 osd.2 (osd.2) 126 : cluster [DBG] 62.3 scrub ok 2024-02-23T00:45:53.783 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:53.141552+0000 osd.6 (osd.6) 109 : cluster [DBG] 15.15 scrub starts 2024-02-23T00:45:53.783 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:53.142467+0000 osd.6 (osd.6) 110 : cluster [DBG] 15.15 scrub ok 2024-02-23T00:45:53.783 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:53.157464+0000 mon.a (mon.0) 1900 : audit [INF] from='client.? 172.21.15.47:0/1402665257' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi047-82378-38"}]': finished 2024-02-23T00:45:53.783 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:53.157531+0000 mon.a (mon.0) 1901 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:45:53.783 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:53.157624+0000 mon.a (mon.0) 1902 : cluster [DBG] osdmap e281: 8 total, 8 up, 8 in 2024-02-23T00:45:53.783 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:53.160545+0000 mon.a (mon.0) 1903 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-33"}]: dispatch 2024-02-23T00:45:53.783 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:53.160699+0000 mon.a (mon.0) 1904 : audit [INF] from='client.? 172.21.15.47:0/1402665257' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi047-82378-38"}]: dispatch 2024-02-23T00:45:53.783 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:53.574243+0000 mon.c (mon.2) 198 : audit [DBG] from='client.? 172.21.15.47:0/1663515292' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-23T00:45:53.783 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:52.466729+0000 mgr.y (mgr.14517) 289 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:45:53.784 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:52.935983+0000 osd.5 (osd.5) 123 : cluster [DBG] 4.4 scrub starts 2024-02-23T00:45:53.784 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:52.939447+0000 osd.5 (osd.5) 124 : cluster [DBG] 4.4 scrub ok 2024-02-23T00:45:53.784 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:53.065390+0000 osd.2 (osd.2) 125 : cluster [DBG] 62.3 scrub starts 2024-02-23T00:45:53.784 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:53.067726+0000 osd.2 (osd.2) 126 : cluster [DBG] 62.3 scrub ok 2024-02-23T00:45:53.784 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:53.141552+0000 osd.6 (osd.6) 109 : cluster [DBG] 15.15 scrub starts 2024-02-23T00:45:53.784 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:53.142467+0000 osd.6 (osd.6) 110 : cluster [DBG] 15.15 scrub ok 2024-02-23T00:45:53.784 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:53.157464+0000 mon.a (mon.0) 1900 : audit [INF] from='client.? 172.21.15.47:0/1402665257' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi047-82378-38"}]': finished 2024-02-23T00:45:53.784 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:53.157531+0000 mon.a (mon.0) 1901 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:45:53.784 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:53.157624+0000 mon.a (mon.0) 1902 : cluster [DBG] osdmap e281: 8 total, 8 up, 8 in 2024-02-23T00:45:53.784 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:53.160545+0000 mon.a (mon.0) 1903 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-33"}]: dispatch 2024-02-23T00:45:53.784 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:53.160699+0000 mon.a (mon.0) 1904 : audit [INF] from='client.? 172.21.15.47:0/1402665257' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi047-82378-38"}]: dispatch 2024-02-23T00:45:53.784 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:53.574243+0000 mon.c (mon.2) 198 : audit [DBG] from='client.? 172.21.15.47:0/1663515292' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-23T00:45:53.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:52.466729+0000 mgr.y (mgr.14517) 289 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:45:53.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:52.935983+0000 osd.5 (osd.5) 123 : cluster [DBG] 4.4 scrub starts 2024-02-23T00:45:53.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:52.939447+0000 osd.5 (osd.5) 124 : cluster [DBG] 4.4 scrub ok 2024-02-23T00:45:53.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:53.065390+0000 osd.2 (osd.2) 125 : cluster [DBG] 62.3 scrub starts 2024-02-23T00:45:53.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:53.067726+0000 osd.2 (osd.2) 126 : cluster [DBG] 62.3 scrub ok 2024-02-23T00:45:53.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:53.141552+0000 osd.6 (osd.6) 109 : cluster [DBG] 15.15 scrub starts 2024-02-23T00:45:53.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:53.142467+0000 osd.6 (osd.6) 110 : cluster [DBG] 15.15 scrub ok 2024-02-23T00:45:53.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:53.157464+0000 mon.a (mon.0) 1900 : audit [INF] from='client.? 172.21.15.47:0/1402665257' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi047-82378-38"}]': finished 2024-02-23T00:45:53.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:53.157531+0000 mon.a (mon.0) 1901 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:45:53.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:53.157624+0000 mon.a (mon.0) 1902 : cluster [DBG] osdmap e281: 8 total, 8 up, 8 in 2024-02-23T00:45:53.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:53.160545+0000 mon.a (mon.0) 1903 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-33"}]: dispatch 2024-02-23T00:45:53.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:53.160699+0000 mon.a (mon.0) 1904 : audit [INF] from='client.? 172.21.15.47:0/1402665257' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi047-82378-38"}]: dispatch 2024-02-23T00:45:53.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:53.574243+0000 mon.c (mon.2) 198 : audit [DBG] from='client.? 172.21.15.47:0/1663515292' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-23T00:45:54.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:52.792427+0000 osd.7 (osd.7) 137 : cluster [DBG] 2.0 scrub starts 2024-02-23T00:45:54.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:52.795520+0000 osd.7 (osd.7) 138 : cluster [DBG] 2.0 scrub ok 2024-02-23T00:45:54.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:53.247663+0000 mgr.y (mgr.14517) 290 : cluster [DBG] pgmap v397: 257 pgs: 14 creating+activating, 2 peering, 8 creating+peering, 233 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:45:54.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:54.161199+0000 mon.a (mon.0) 1905 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-33"}]': finished 2024-02-23T00:45:54.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:54.161267+0000 mon.a (mon.0) 1906 : audit [INF] from='client.? 172.21.15.47:0/1402665257' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi047-82378-38"}]': finished 2024-02-23T00:45:54.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:54.161351+0000 mon.a (mon.0) 1907 : cluster [DBG] osdmap e282: 8 total, 8 up, 8 in 2024-02-23T00:45:54.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:54.166174+0000 mon.a (mon.0) 1908 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-33", "mode": "writeback"}]: dispatch 2024-02-23T00:45:54.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:54.166494+0000 mon.c (mon.2) 199 : audit [INF] from='client.? 172.21.15.47:0/3615725416' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi047-82386-48"}]: dispatch 2024-02-23T00:45:54.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:54.167223+0000 mon.a (mon.0) 1909 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi047-82386-48"}]: dispatch 2024-02-23T00:45:54.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:54.183311+0000 mon.c (mon.2) 200 : audit [INF] from='client.? 172.21.15.47:0/422202595' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi047-82378-39"}]: dispatch 2024-02-23T00:45:54.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:54.183577+0000 mon.a (mon.0) 1910 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi047-82378-39"}]: dispatch 2024-02-23T00:45:54.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:54.184187+0000 mon.c (mon.2) 201 : audit [INF] from='client.? 172.21.15.47:0/422202595' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi047-82378-39"}]: dispatch 2024-02-23T00:45:54.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:54.184417+0000 mon.a (mon.0) 1911 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi047-82378-39"}]: dispatch 2024-02-23T00:45:54.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:54.184802+0000 mon.c (mon.2) 202 : audit [INF] from='client.? 172.21.15.47:0/422202595' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi047-82378-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:54.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:54.185051+0000 mon.a (mon.0) 1912 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi047-82378-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:55.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:52.792427+0000 osd.7 (osd.7) 137 : cluster [DBG] 2.0 scrub starts 2024-02-23T00:45:55.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:52.795520+0000 osd.7 (osd.7) 138 : cluster [DBG] 2.0 scrub ok 2024-02-23T00:45:55.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:53.247663+0000 mgr.y (mgr.14517) 290 : cluster [DBG] pgmap v397: 257 pgs: 14 creating+activating, 2 peering, 8 creating+peering, 233 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:45:55.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:54.161199+0000 mon.a (mon.0) 1905 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-33"}]': finished 2024-02-23T00:45:55.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:54.161267+0000 mon.a (mon.0) 1906 : audit [INF] from='client.? 172.21.15.47:0/1402665257' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi047-82378-38"}]': finished 2024-02-23T00:45:55.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:54.161351+0000 mon.a (mon.0) 1907 : cluster [DBG] osdmap e282: 8 total, 8 up, 8 in 2024-02-23T00:45:55.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:54.166174+0000 mon.a (mon.0) 1908 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-33", "mode": "writeback"}]: dispatch 2024-02-23T00:45:55.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:54.166494+0000 mon.c (mon.2) 199 : audit [INF] from='client.? 172.21.15.47:0/3615725416' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi047-82386-48"}]: dispatch 2024-02-23T00:45:55.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:54.167223+0000 mon.a (mon.0) 1909 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi047-82386-48"}]: dispatch 2024-02-23T00:45:55.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:54.183311+0000 mon.c (mon.2) 200 : audit [INF] from='client.? 172.21.15.47:0/422202595' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi047-82378-39"}]: dispatch 2024-02-23T00:45:55.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:54.183577+0000 mon.a (mon.0) 1910 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi047-82378-39"}]: dispatch 2024-02-23T00:45:55.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:54.184187+0000 mon.c (mon.2) 201 : audit [INF] from='client.? 172.21.15.47:0/422202595' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi047-82378-39"}]: dispatch 2024-02-23T00:45:55.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:54.184417+0000 mon.a (mon.0) 1911 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi047-82378-39"}]: dispatch 2024-02-23T00:45:55.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:54.184802+0000 mon.c (mon.2) 202 : audit [INF] from='client.? 172.21.15.47:0/422202595' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi047-82378-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:55.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:54.185051+0000 mon.a (mon.0) 1912 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi047-82378-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:55.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:52.792427+0000 osd.7 (osd.7) 137 : cluster [DBG] 2.0 scrub starts 2024-02-23T00:45:55.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:52.795520+0000 osd.7 (osd.7) 138 : cluster [DBG] 2.0 scrub ok 2024-02-23T00:45:55.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:53.247663+0000 mgr.y (mgr.14517) 290 : cluster [DBG] pgmap v397: 257 pgs: 14 creating+activating, 2 peering, 8 creating+peering, 233 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:45:55.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:54.161199+0000 mon.a (mon.0) 1905 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-33"}]': finished 2024-02-23T00:45:55.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:54.161267+0000 mon.a (mon.0) 1906 : audit [INF] from='client.? 172.21.15.47:0/1402665257' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi047-82378-38"}]': finished 2024-02-23T00:45:55.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:54.161351+0000 mon.a (mon.0) 1907 : cluster [DBG] osdmap e282: 8 total, 8 up, 8 in 2024-02-23T00:45:55.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:54.166174+0000 mon.a (mon.0) 1908 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-33", "mode": "writeback"}]: dispatch 2024-02-23T00:45:55.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:54.166494+0000 mon.c (mon.2) 199 : audit [INF] from='client.? 172.21.15.47:0/3615725416' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi047-82386-48"}]: dispatch 2024-02-23T00:45:55.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:54.167223+0000 mon.a (mon.0) 1909 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi047-82386-48"}]: dispatch 2024-02-23T00:45:55.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:54.183311+0000 mon.c (mon.2) 200 : audit [INF] from='client.? 172.21.15.47:0/422202595' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi047-82378-39"}]: dispatch 2024-02-23T00:45:55.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:54.183577+0000 mon.a (mon.0) 1910 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi047-82378-39"}]: dispatch 2024-02-23T00:45:55.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:54.184187+0000 mon.c (mon.2) 201 : audit [INF] from='client.? 172.21.15.47:0/422202595' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi047-82378-39"}]: dispatch 2024-02-23T00:45:55.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:54.184417+0000 mon.a (mon.0) 1911 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi047-82378-39"}]: dispatch 2024-02-23T00:45:55.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:54.184802+0000 mon.c (mon.2) 202 : audit [INF] from='client.? 172.21.15.47:0/422202595' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi047-82378-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:55.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:54.185051+0000 mon.a (mon.0) 1912 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi047-82378-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:45:55.413 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:55.414 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:45:55.515 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:45:55.546 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:55.567 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:45:55.570 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:55.570 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:45:55.671 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:45:55.701 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:55.806 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:45:55.808 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:55.809 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:45:55.910 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:45:55.939 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:54.309210+0000 osd.4 (osd.4) 125 : cluster [DBG] 18.19 scrub starts 2024-02-23T00:45:55.939 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:54.311431+0000 osd.4 (osd.4) 126 : cluster [DBG] 18.19 scrub ok 2024-02-23T00:45:55.939 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:54.750257+0000 osd.7 (osd.7) 139 : cluster [DBG] 18.2 scrub starts 2024-02-23T00:45:55.939 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:55.107828+0000 osd.6 (osd.6) 111 : cluster [DBG] 62.19 scrub starts 2024-02-23T00:45:55.940 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:55.109545+0000 osd.6 (osd.6) 112 : cluster [DBG] 62.19 scrub ok 2024-02-23T00:45:55.940 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:55.135527+0000 mon.a (mon.0) 1913 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:45:55.940 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:55.140997+0000 mon.a (mon.0) 1914 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-33", "mode": "writeback"}]': finished 2024-02-23T00:45:55.940 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:55.141076+0000 mon.a (mon.0) 1915 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi047-82386-48"}]': finished 2024-02-23T00:45:55.940 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:55.141194+0000 mon.a (mon.0) 1916 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi047-82378-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:55.940 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:55.141262+0000 mon.a (mon.0) 1917 : cluster [DBG] osdmap e283: 8 total, 8 up, 8 in 2024-02-23T00:45:55.940 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:55.142946+0000 mon.c (mon.2) 203 : audit [INF] from='client.? 172.21.15.47:0/3615725416' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi047-82386-48"}]: dispatch 2024-02-23T00:45:55.940 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:55.143289+0000 mon.c (mon.2) 204 : audit [INF] from='client.? 172.21.15.47:0/422202595' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi047-82378-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi047-82378-39"}]: dispatch 2024-02-23T00:45:55.940 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:55.144023+0000 mon.a (mon.0) 1918 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi047-82386-48"}]: dispatch 2024-02-23T00:45:55.940 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:55.144672+0000 mon.a (mon.0) 1919 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi047-82378-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi047-82378-39"}]: dispatch 2024-02-23T00:45:55.940 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:55.249130+0000 mon.a (mon.0) 1920 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "29"}]: dispatch 2024-02-23T00:45:55.941 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:55.962 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:45:55.965 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:55.966 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:45:56.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:54.309210+0000 osd.4 (osd.4) 125 : cluster [DBG] 18.19 scrub starts 2024-02-23T00:45:56.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:54.311431+0000 osd.4 (osd.4) 126 : cluster [DBG] 18.19 scrub ok 2024-02-23T00:45:56.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:54.750257+0000 osd.7 (osd.7) 139 : cluster [DBG] 18.2 scrub starts 2024-02-23T00:45:56.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:55.107828+0000 osd.6 (osd.6) 111 : cluster [DBG] 62.19 scrub starts 2024-02-23T00:45:56.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:55.109545+0000 osd.6 (osd.6) 112 : cluster [DBG] 62.19 scrub ok 2024-02-23T00:45:56.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:55.135527+0000 mon.a (mon.0) 1913 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:45:56.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:55.140997+0000 mon.a (mon.0) 1914 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-33", "mode": "writeback"}]': finished 2024-02-23T00:45:56.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:55.141076+0000 mon.a (mon.0) 1915 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi047-82386-48"}]': finished 2024-02-23T00:45:56.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:55.141194+0000 mon.a (mon.0) 1916 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi047-82378-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:56.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:55.141262+0000 mon.a (mon.0) 1917 : cluster [DBG] osdmap e283: 8 total, 8 up, 8 in 2024-02-23T00:45:56.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:55.142946+0000 mon.c (mon.2) 203 : audit [INF] from='client.? 172.21.15.47:0/3615725416' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi047-82386-48"}]: dispatch 2024-02-23T00:45:56.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:55.143289+0000 mon.c (mon.2) 204 : audit [INF] from='client.? 172.21.15.47:0/422202595' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi047-82378-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi047-82378-39"}]: dispatch 2024-02-23T00:45:56.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:55.144023+0000 mon.a (mon.0) 1918 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi047-82386-48"}]: dispatch 2024-02-23T00:45:56.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:55.144672+0000 mon.a (mon.0) 1919 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi047-82378-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi047-82378-39"}]: dispatch 2024-02-23T00:45:56.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:55.249130+0000 mon.a (mon.0) 1920 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "29"}]: dispatch 2024-02-23T00:45:56.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:54.309210+0000 osd.4 (osd.4) 125 : cluster [DBG] 18.19 scrub starts 2024-02-23T00:45:56.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:54.311431+0000 osd.4 (osd.4) 126 : cluster [DBG] 18.19 scrub ok 2024-02-23T00:45:56.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:54.750257+0000 osd.7 (osd.7) 139 : cluster [DBG] 18.2 scrub starts 2024-02-23T00:45:56.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:55.107828+0000 osd.6 (osd.6) 111 : cluster [DBG] 62.19 scrub starts 2024-02-23T00:45:56.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:55.109545+0000 osd.6 (osd.6) 112 : cluster [DBG] 62.19 scrub ok 2024-02-23T00:45:56.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:55.135527+0000 mon.a (mon.0) 1913 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:45:56.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:55.140997+0000 mon.a (mon.0) 1914 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-33", "mode": "writeback"}]': finished 2024-02-23T00:45:56.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:55.141076+0000 mon.a (mon.0) 1915 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi047-82386-48"}]': finished 2024-02-23T00:45:56.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:55.141194+0000 mon.a (mon.0) 1916 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi047-82378-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:56.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:55.141262+0000 mon.a (mon.0) 1917 : cluster [DBG] osdmap e283: 8 total, 8 up, 8 in 2024-02-23T00:45:56.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:55.142946+0000 mon.c (mon.2) 203 : audit [INF] from='client.? 172.21.15.47:0/3615725416' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi047-82386-48"}]: dispatch 2024-02-23T00:45:56.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:55.143289+0000 mon.c (mon.2) 204 : audit [INF] from='client.? 172.21.15.47:0/422202595' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi047-82378-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi047-82378-39"}]: dispatch 2024-02-23T00:45:56.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:55.144023+0000 mon.a (mon.0) 1918 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi047-82386-48"}]: dispatch 2024-02-23T00:45:56.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:55.144672+0000 mon.a (mon.0) 1919 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi047-82378-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi047-82378-39"}]: dispatch 2024-02-23T00:45:56.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:55.249130+0000 mon.a (mon.0) 1920 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "29"}]: dispatch 2024-02-23T00:45:56.067 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:45:56.096 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:56.171 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:45:56.174 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:56.175 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:45:56.276 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:45:56.293 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:45:56.293 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:45:56.307 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:56.328 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:45:56.331 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:56.332 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:45:56.432 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:45:56.460 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:56.764 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:54.753258+0000 osd.7 (osd.7) 140 : cluster [DBG] 18.2 scrub ok 2024-02-23T00:45:56.764 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:55.248219+0000 mgr.y (mgr.14517) 291 : cluster [DBG] pgmap v400: 248 pgs: 14 creating+activating, 1 peering, 233 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s; 31 B/s, 0 objects/s recovering 2024-02-23T00:45:56.764 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:55.705103+0000 osd.7 (osd.7) 141 : cluster [DBG] 5.9 scrub starts 2024-02-23T00:45:56.765 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:55.708571+0000 osd.7 (osd.7) 142 : cluster [DBG] 5.9 scrub ok 2024-02-23T00:45:56.765 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:56.050586+0000 osd.2 (osd.2) 127 : cluster [DBG] 2.b scrub starts 2024-02-23T00:45:56.765 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:56.051571+0000 osd.2 (osd.2) 128 : cluster [DBG] 2.b scrub ok 2024-02-23T00:45:56.765 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:56.142218+0000 mon.a (mon.0) 1921 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi047-82386-48"}]': finished 2024-02-23T00:45:56.765 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:56.163493+0000 mon.c (mon.2) 205 : audit [INF] from='client.? 172.21.15.47:0/3371139177' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi047-82386-49"}]: dispatch 2024-02-23T00:45:56.765 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:56.577557+0000 mon.a (mon.0) 1922 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "29"}]': finished 2024-02-23T00:45:56.765 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:56.577612+0000 mon.a (mon.0) 1923 : cluster [DBG] osdmap e284: 8 total, 8 up, 8 in 2024-02-23T00:45:56.765 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:56.578758+0000 mon.a (mon.0) 1924 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi047-82386-49"}]: dispatch 2024-02-23T00:45:56.765 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:56.579323+0000 mon.c (mon.2) 206 : audit [INF] from='client.? 172.21.15.47:0/3371139177' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi047-82386-49"}]: dispatch 2024-02-23T00:45:56.765 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:56.582513+0000 mon.a (mon.0) 1925 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi047-82386-49"}]: dispatch 2024-02-23T00:45:56.765 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:56.584589+0000 mon.a (mon.0) 1926 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:45:56.765 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:56.584905+0000 mon.c (mon.2) 207 : audit [INF] from='client.? 172.21.15.47:0/3371139177' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi047-82386-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:56.765 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:56.585283+0000 mon.a (mon.0) 1927 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi047-82386-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:56.766 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:54.753258+0000 osd.7 (osd.7) 140 : cluster [DBG] 18.2 scrub ok 2024-02-23T00:45:56.766 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:55.248219+0000 mgr.y (mgr.14517) 291 : cluster [DBG] pgmap v400: 248 pgs: 14 creating+activating, 1 peering, 233 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s; 31 B/s, 0 objects/s recovering 2024-02-23T00:45:56.766 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:55.705103+0000 osd.7 (osd.7) 141 : cluster [DBG] 5.9 scrub starts 2024-02-23T00:45:56.766 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:55.708571+0000 osd.7 (osd.7) 142 : cluster [DBG] 5.9 scrub ok 2024-02-23T00:45:56.766 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:56.050586+0000 osd.2 (osd.2) 127 : cluster [DBG] 2.b scrub starts 2024-02-23T00:45:56.766 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:56.051571+0000 osd.2 (osd.2) 128 : cluster [DBG] 2.b scrub ok 2024-02-23T00:45:56.766 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:56.142218+0000 mon.a (mon.0) 1921 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi047-82386-48"}]': finished 2024-02-23T00:45:56.766 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:56.163493+0000 mon.c (mon.2) 205 : audit [INF] from='client.? 172.21.15.47:0/3371139177' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi047-82386-49"}]: dispatch 2024-02-23T00:45:56.766 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:56.577557+0000 mon.a (mon.0) 1922 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "29"}]': finished 2024-02-23T00:45:56.766 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:56.577612+0000 mon.a (mon.0) 1923 : cluster [DBG] osdmap e284: 8 total, 8 up, 8 in 2024-02-23T00:45:56.766 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:56.578758+0000 mon.a (mon.0) 1924 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi047-82386-49"}]: dispatch 2024-02-23T00:45:56.766 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:56.579323+0000 mon.c (mon.2) 206 : audit [INF] from='client.? 172.21.15.47:0/3371139177' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi047-82386-49"}]: dispatch 2024-02-23T00:45:56.767 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:56.582513+0000 mon.a (mon.0) 1925 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi047-82386-49"}]: dispatch 2024-02-23T00:45:56.767 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:56.584589+0000 mon.a (mon.0) 1926 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:45:56.767 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:56.584905+0000 mon.c (mon.2) 207 : audit [INF] from='client.? 172.21.15.47:0/3371139177' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi047-82386-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:56.767 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:56.585283+0000 mon.a (mon.0) 1927 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi047-82386-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:56.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:54.753258+0000 osd.7 (osd.7) 140 : cluster [DBG] 18.2 scrub ok 2024-02-23T00:45:56.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:55.248219+0000 mgr.y (mgr.14517) 291 : cluster [DBG] pgmap v400: 248 pgs: 14 creating+activating, 1 peering, 233 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s; 31 B/s, 0 objects/s recovering 2024-02-23T00:45:56.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:55.705103+0000 osd.7 (osd.7) 141 : cluster [DBG] 5.9 scrub starts 2024-02-23T00:45:56.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:55.708571+0000 osd.7 (osd.7) 142 : cluster [DBG] 5.9 scrub ok 2024-02-23T00:45:56.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:56.050586+0000 osd.2 (osd.2) 127 : cluster [DBG] 2.b scrub starts 2024-02-23T00:45:56.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:56.051571+0000 osd.2 (osd.2) 128 : cluster [DBG] 2.b scrub ok 2024-02-23T00:45:56.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:56.142218+0000 mon.a (mon.0) 1921 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi047-82386-48"}]': finished 2024-02-23T00:45:56.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:56.163493+0000 mon.c (mon.2) 205 : audit [INF] from='client.? 172.21.15.47:0/3371139177' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi047-82386-49"}]: dispatch 2024-02-23T00:45:56.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:56.577557+0000 mon.a (mon.0) 1922 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "29"}]': finished 2024-02-23T00:45:56.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:56.577612+0000 mon.a (mon.0) 1923 : cluster [DBG] osdmap e284: 8 total, 8 up, 8 in 2024-02-23T00:45:56.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:56.578758+0000 mon.a (mon.0) 1924 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi047-82386-49"}]: dispatch 2024-02-23T00:45:56.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:56.579323+0000 mon.c (mon.2) 206 : audit [INF] from='client.? 172.21.15.47:0/3371139177' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi047-82386-49"}]: dispatch 2024-02-23T00:45:56.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:56.582513+0000 mon.a (mon.0) 1925 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi047-82386-49"}]: dispatch 2024-02-23T00:45:56.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:56.584589+0000 mon.a (mon.0) 1926 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:45:56.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:56.584905+0000 mon.c (mon.2) 207 : audit [INF] from='client.? 172.21.15.47:0/3371139177' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi047-82386-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:56.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:56.585283+0000 mon.a (mon.0) 1927 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi047-82386-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:45:57.209 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:45:57.211 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:57.212 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:45:57.313 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:45:57.342 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:57.441 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:45:57.443 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:57.444 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:45:57.544 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:45:57.576 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:57.597 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:45:57.600 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:57.600 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:45:57.701 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:45:57.732 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:57.753 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:45:57.756 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:57.757 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:45:57.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:56.353643+0000 osd.4 (osd.4) 127 : cluster [DBG] 62.16 scrub starts 2024-02-23T00:45:57.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:56.354876+0000 osd.4 (osd.4) 128 : cluster [DBG] 62.16 scrub ok 2024-02-23T00:45:57.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:56.705537+0000 osd.7 (osd.7) 143 : cluster [DBG] 18.6 scrub starts 2024-02-23T00:45:57.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:56.706784+0000 osd.7 (osd.7) 144 : cluster [DBG] 18.6 scrub ok 2024-02-23T00:45:57.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:57.133959+0000 osd.6 (osd.6) 113 : cluster [DBG] 62.2 scrub starts 2024-02-23T00:45:57.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:57.135719+0000 osd.6 (osd.6) 114 : cluster [DBG] 62.2 scrub ok 2024-02-23T00:45:57.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:57.144457+0000 mon.a (mon.0) 1928 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi047-82378-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi047-82378-39"}]': finished 2024-02-23T00:45:57.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:57.144625+0000 mon.a (mon.0) 1929 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi047-82386-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:57.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:57.144712+0000 mon.a (mon.0) 1930 : cluster [DBG] osdmap e285: 8 total, 8 up, 8 in 2024-02-23T00:45:57.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:57.146688+0000 mon.c (mon.2) 208 : audit [INF] from='client.? 172.21.15.47:0/3371139177' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi047-82386-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi047-82386-49"}]: dispatch 2024-02-23T00:45:57.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:57.147978+0000 mon.a (mon.0) 1931 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi047-82386-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi047-82386-49"}]: dispatch 2024-02-23T00:45:57.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:57.193272+0000 mon.a (mon.0) 1932 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:45:57.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:56.353643+0000 osd.4 (osd.4) 127 : cluster [DBG] 62.16 scrub starts 2024-02-23T00:45:57.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:56.354876+0000 osd.4 (osd.4) 128 : cluster [DBG] 62.16 scrub ok 2024-02-23T00:45:57.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:56.705537+0000 osd.7 (osd.7) 143 : cluster [DBG] 18.6 scrub starts 2024-02-23T00:45:57.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:56.706784+0000 osd.7 (osd.7) 144 : cluster [DBG] 18.6 scrub ok 2024-02-23T00:45:57.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:57.133959+0000 osd.6 (osd.6) 113 : cluster [DBG] 62.2 scrub starts 2024-02-23T00:45:57.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:57.135719+0000 osd.6 (osd.6) 114 : cluster [DBG] 62.2 scrub ok 2024-02-23T00:45:57.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:57.144457+0000 mon.a (mon.0) 1928 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi047-82378-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi047-82378-39"}]': finished 2024-02-23T00:45:57.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:57.144625+0000 mon.a (mon.0) 1929 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi047-82386-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:57.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:57.144712+0000 mon.a (mon.0) 1930 : cluster [DBG] osdmap e285: 8 total, 8 up, 8 in 2024-02-23T00:45:57.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:57.146688+0000 mon.c (mon.2) 208 : audit [INF] from='client.? 172.21.15.47:0/3371139177' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi047-82386-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi047-82386-49"}]: dispatch 2024-02-23T00:45:57.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:57.147978+0000 mon.a (mon.0) 1931 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi047-82386-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi047-82386-49"}]: dispatch 2024-02-23T00:45:57.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:57.193272+0000 mon.a (mon.0) 1932 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:45:57.857 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:45:57.885 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:56.353643+0000 osd.4 (osd.4) 127 : cluster [DBG] 62.16 scrub starts 2024-02-23T00:45:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:56.354876+0000 osd.4 (osd.4) 128 : cluster [DBG] 62.16 scrub ok 2024-02-23T00:45:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:56.705537+0000 osd.7 (osd.7) 143 : cluster [DBG] 18.6 scrub starts 2024-02-23T00:45:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:56.706784+0000 osd.7 (osd.7) 144 : cluster [DBG] 18.6 scrub ok 2024-02-23T00:45:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:57.133959+0000 osd.6 (osd.6) 113 : cluster [DBG] 62.2 scrub starts 2024-02-23T00:45:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:57.135719+0000 osd.6 (osd.6) 114 : cluster [DBG] 62.2 scrub ok 2024-02-23T00:45:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:57.144457+0000 mon.a (mon.0) 1928 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi047-82378-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi047-82378-39"}]': finished 2024-02-23T00:45:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:57.144625+0000 mon.a (mon.0) 1929 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi047-82386-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:45:57.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:57.144712+0000 mon.a (mon.0) 1930 : cluster [DBG] osdmap e285: 8 total, 8 up, 8 in 2024-02-23T00:45:57.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:57.146688+0000 mon.c (mon.2) 208 : audit [INF] from='client.? 172.21.15.47:0/3371139177' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi047-82386-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi047-82386-49"}]: dispatch 2024-02-23T00:45:57.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:57.147978+0000 mon.a (mon.0) 1931 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi047-82386-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi047-82386-49"}]: dispatch 2024-02-23T00:45:57.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:57.193272+0000 mon.a (mon.0) 1932 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:45:57.991 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:45:57.994 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:57.994 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:45:58.005 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.2 dump_historic_ops 2024-02-23T00:45:58.096 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:45:58.127 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:58.148 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:45:58.151 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:58.152 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:45:58.253 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:45:58.281 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:58.300 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:45:58.303 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:58.304 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:45:58.404 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:45:58.434 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:58.443 INFO:tasks.cephadm.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-02-23T00:45:58.443 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph pg dump --format=json 2024-02-23T00:45:58.461 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:45:58.468 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:45:58.468 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:45:58.476 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:45:58.570 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:45:58.600 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:45:58.737 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:57.249147+0000 mgr.y (mgr.14517) 292 : cluster [DBG] pgmap v403: 256 pgs: 2 clean+premerge+peered, 8 creating+peering, 246 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-23T00:45:58.737 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:57.873642+0000 osd.3 (osd.3) 149 : cluster [DBG] 62.c scrub starts 2024-02-23T00:45:58.738 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:57.874879+0000 osd.3 (osd.3) 150 : cluster [DBG] 62.c scrub ok 2024-02-23T00:45:58.738 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:58.072669+0000 osd.2 (osd.2) 129 : cluster [DBG] 62.1f scrub starts 2024-02-23T00:45:58.738 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:58.074856+0000 osd.2 (osd.2) 130 : cluster [DBG] 62.1f scrub ok 2024-02-23T00:45:58.738 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:58.125656+0000 osd.6 (osd.6) 115 : cluster [DBG] 62.17 scrub starts 2024-02-23T00:45:58.738 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:58.127297+0000 osd.6 (osd.6) 116 : cluster [DBG] 62.17 scrub ok 2024-02-23T00:45:58.738 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:58.592280+0000 mon.a (mon.0) 1933 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:45:58.738 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:58.592340+0000 mon.a (mon.0) 1934 : cluster [DBG] osdmap e286: 8 total, 8 up, 8 in 2024-02-23T00:45:58.738 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:58.595969+0000 mon.a (mon.0) 1935 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-33"}]: dispatch 2024-02-23T00:45:58.738 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:57.249147+0000 mgr.y (mgr.14517) 292 : cluster [DBG] pgmap v403: 256 pgs: 2 clean+premerge+peered, 8 creating+peering, 246 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-23T00:45:58.739 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:57.873642+0000 osd.3 (osd.3) 149 : cluster [DBG] 62.c scrub starts 2024-02-23T00:45:58.739 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:57.874879+0000 osd.3 (osd.3) 150 : cluster [DBG] 62.c scrub ok 2024-02-23T00:45:58.739 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:58.072669+0000 osd.2 (osd.2) 129 : cluster [DBG] 62.1f scrub starts 2024-02-23T00:45:58.739 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:58.074856+0000 osd.2 (osd.2) 130 : cluster [DBG] 62.1f scrub ok 2024-02-23T00:45:58.739 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:58.125656+0000 osd.6 (osd.6) 115 : cluster [DBG] 62.17 scrub starts 2024-02-23T00:45:58.739 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:58.127297+0000 osd.6 (osd.6) 116 : cluster [DBG] 62.17 scrub ok 2024-02-23T00:45:58.739 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:58.592280+0000 mon.a (mon.0) 1933 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:45:58.739 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:58.592340+0000 mon.a (mon.0) 1934 : cluster [DBG] osdmap e286: 8 total, 8 up, 8 in 2024-02-23T00:45:58.739 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:58.595969+0000 mon.a (mon.0) 1935 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-33"}]: dispatch 2024-02-23T00:45:58.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:57.249147+0000 mgr.y (mgr.14517) 292 : cluster [DBG] pgmap v403: 256 pgs: 2 clean+premerge+peered, 8 creating+peering, 246 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-23T00:45:58.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:57.873642+0000 osd.3 (osd.3) 149 : cluster [DBG] 62.c scrub starts 2024-02-23T00:45:58.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:57.874879+0000 osd.3 (osd.3) 150 : cluster [DBG] 62.c scrub ok 2024-02-23T00:45:58.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:58.072669+0000 osd.2 (osd.2) 129 : cluster [DBG] 62.1f scrub starts 2024-02-23T00:45:58.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:58.074856+0000 osd.2 (osd.2) 130 : cluster [DBG] 62.1f scrub ok 2024-02-23T00:45:58.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:58.125656+0000 osd.6 (osd.6) 115 : cluster [DBG] 62.17 scrub starts 2024-02-23T00:45:58.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:58.127297+0000 osd.6 (osd.6) 116 : cluster [DBG] 62.17 scrub ok 2024-02-23T00:45:58.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:58.592280+0000 mon.a (mon.0) 1933 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:45:58.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:58.592340+0000 mon.a (mon.0) 1934 : cluster [DBG] osdmap e286: 8 total, 8 up, 8 in 2024-02-23T00:45:58.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:58.595969+0000 mon.a (mon.0) 1935 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-33"}]: dispatch 2024-02-23T00:45:59.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.210+0000 7f363f181700 1 -- 172.21.15.47:0/3028571316 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f36380fcbb0 msgr2=0x7f36380fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:59.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.210+0000 7f363f181700 1 --2- 172.21.15.47:0/3028571316 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f36380fcbb0 0x7f36380fcf90 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 rx=0x7f362c004870 tx=0x7f362c009cb0).stop 2024-02-23T00:45:59.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.210+0000 7f363f181700 1 -- 172.21.15.47:0/3028571316 shutdown_connections 2024-02-23T00:45:59.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.210+0000 7f363f181700 1 --2- 172.21.15.47:0/3028571316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36380f7400 0x7f3638103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:59.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.210+0000 7f363f181700 1 --2- 172.21.15.47:0/3028571316 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f36380fcbb0 0x7f36380fcf90 unknown :-1 s=CLOSED pgs=2594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:59.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.210+0000 7f363f181700 1 --2- 172.21.15.47:0/3028571316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36380fd510 0x7f36380f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:45:59.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.210+0000 7f363f181700 1 -- 172.21.15.47:0/3028571316 >> 172.21.15.47:0/3028571316 conn(0x7f36380f2820 msgr2=0x7f36380f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:45:59.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.211+0000 7f363f181700 1 -- 172.21.15.47:0/3028571316 shutdown_connections 2024-02-23T00:45:59.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.211+0000 7f363f181700 1 -- 172.21.15.47:0/3028571316 wait complete. 2024-02-23T00:45:59.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.211+0000 7f363f181700 1 librados: starting msgr at 2024-02-23T00:45:59.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.211+0000 7f363f181700 1 librados: starting objecter 2024-02-23T00:45:59.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.211+0000 7f363f181700 1 Processor -- start 2024-02-23T00:45:59.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.211+0000 7f363f181700 1 -- start start 2024-02-23T00:45:59.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.211+0000 7f363f181700 1 librados: setting wanted keys 2024-02-23T00:45:59.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.211+0000 7f363f181700 1 librados: calling monclient init 2024-02-23T00:45:59.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.211+0000 7f363f181700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f36380f7400 0x7f36380f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:59.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.211+0000 7f363f181700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36380fcbb0 0x7f36380f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:59.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.211+0000 7f363f181700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36380fd510 0x7f36380f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:59.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.211+0000 7f363f181700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3638116580 con 0x7f36380fcbb0 2024-02-23T00:45:59.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.211+0000 7f363f181700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36381166f0 con 0x7f36380fd510 2024-02-23T00:45:59.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.211+0000 7f363f181700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36381168b0 con 0x7f36380f7400 2024-02-23T00:45:59.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.212+0000 7f363cf1d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f36380f7400 0x7f36380f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:59.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.212+0000 7f363cf1d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f36380f7400 0x7f36380f89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:56298/0 (socket says 172.21.15.47:56298) 2024-02-23T00:45:59.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.212+0000 7f363cf1d700 1 -- 172.21.15.47:0/3088814386 learned_addr learned my addr 172.21.15.47:0/3088814386 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:45:59.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.212+0000 7f3637fff700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36380fcbb0 0x7f36380f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:59.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.212+0000 7f363d71e700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36380fd510 0x7f36380f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:59.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.212+0000 7f363cf1d700 1 -- 172.21.15.47:0/3088814386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36380fd510 msgr2=0x7f36380f9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:59.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.212+0000 7f363cf1d700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36380fd510 0x7f36380f9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:59.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.212+0000 7f363cf1d700 1 -- 172.21.15.47:0/3088814386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36380fcbb0 msgr2=0x7f36380f8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:59.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.212+0000 7f363cf1d700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36380fcbb0 0x7f36380f8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:59.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.212+0000 7f363cf1d700 1 -- 172.21.15.47:0/3088814386 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36300097c0 con 0x7f36380f7400 2024-02-23T00:45:59.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.212+0000 7f363d71e700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36380fd510 0x7f36380f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-23T00:45:59.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.212+0000 7f363cf1d700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f36380f7400 0x7f36380f89b0 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 rx=0x7f362c009da0 tx=0x7f362c009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:59.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.212+0000 7f3635ffb700 10 client.?.objecter ms_handle_connect 0x7f36380f7400 2024-02-23T00:45:59.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.212+0000 7f3635ffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:45:59.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.212+0000 7f3635ffb700 1 -- 172.21.15.47:0/3088814386 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c00e030 con 0x7f36380f7400 2024-02-23T00:45:59.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.212+0000 7f3635ffb700 1 -- 172.21.15.47:0/3088814386 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f362c013e70 con 0x7f36380f7400 2024-02-23T00:45:59.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.212+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362c005040 con 0x7f36380f7400 2024-02-23T00:45:59.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.212+0000 7f3635ffb700 1 -- 172.21.15.47:0/3088814386 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c018070 con 0x7f36380f7400 2024-02-23T00:45:59.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.212+0000 7f363f181700 10 client.41003.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:45:59.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.212+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3638116d50 con 0x7f36380f7400 2024-02-23T00:45:59.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.212+0000 7f363f181700 1 librados: init done 2024-02-23T00:45:59.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.213+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f363805fa60 con 0x7f36380f7400 2024-02-23T00:45:59.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.214+0000 7f3635ffb700 1 -- 172.21.15.47:0/3088814386 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f362c009240 con 0x7f36380f7400 2024-02-23T00:45:59.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.214+0000 7f3635ffb700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3628062cc0 0x7f3628065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:59.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.214+0000 7f3635ffb700 1 -- 172.21.15.47:0/3088814386 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(287..287 src has 48..287) v4 ==== 10736+0+0 (secure 0 0 0) 0x7f362c07f680 con 0x7f36380f7400 2024-02-23T00:45:59.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.214+0000 7f3635ffb700 10 client.41003.objecter ms_dispatch 0x7f3638001550 osd_map(287..287 src has 48..287) v4 2024-02-23T00:45:59.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.214+0000 7f3635ffb700 3 client.41003.objecter handle_osd_map got epochs [287,287] > 0 2024-02-23T00:45:59.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.214+0000 7f3635ffb700 3 client.41003.objecter handle_osd_map decoding full epoch 287 2024-02-23T00:45:59.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.214+0000 7f3637fff700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3628062cc0 0x7f3628065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:59.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.214+0000 7f3635ffb700 20 client.41003.objecter dump_active .. 0 homeless 2024-02-23T00:45:59.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.214+0000 7f3635ffb700 1 -- 172.21.15.47:0/3088814386 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v287) v1 ==== 54+0+16 (secure 0 0 0) 0x7f362c07fb00 con 0x7f36380f7400 2024-02-23T00:45:59.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.215+0000 7f3635ffb700 10 client.41003.objecter ms_handle_connect 0x7f3628062cc0 2024-02-23T00:45:59.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.215+0000 7f3637fff700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3628062cc0 0x7f3628065180 secure :-1 s=READY pgs=3006 cs=0 l=1 rev1=1 rx=0x7f36380fa150 tx=0x7f3624009060).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:59.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.215+0000 7f363f181700 10 client.41003.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:59.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.215+0000 7f363f181700 15 client.41003.objecter _session_command_op_assign -1 1 2024-02-23T00:45:59.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.215+0000 7f363f181700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f36380ffed0 0x7f36381002b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:59.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.215+0000 7f363f181700 20 client.41003.objecter _get_session s=0x7f36381169f0 osd=0 3 2024-02-23T00:45:59.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.215+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f36381169f0 osd=0 3 2024-02-23T00:45:59.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.215+0000 7f363f181700 20 client.41003.objecter _get_session s=0x7f36381169f0 osd=0 3 2024-02-23T00:45:59.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.215+0000 7f363f181700 15 client.41003.objecter _session_command_op_remove -1 1 2024-02-23T00:45:59.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.215+0000 7f363f181700 20 client.41003.objecter get_session s=0x7f36381169f0 osd=0 3 2024-02-23T00:45:59.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.215+0000 7f363f181700 15 client.41003.objecter _session_command_op_assign 0 1 2024-02-23T00:45:59.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.215+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f36381169f0 osd=0 4 2024-02-23T00:45:59.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.215+0000 7f363f181700 10 client.41003.objecter _send_command 1 2024-02-23T00:45:59.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.215+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3638105ee0 con 0x7f36380ffed0 2024-02-23T00:45:59.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.215+0000 7f363d71e700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f36380ffed0 0x7f36381002b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:59.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.215+0000 7f3635ffb700 10 client.41003.objecter ms_handle_connect 0x7f36380ffed0 2024-02-23T00:45:59.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.215+0000 7f363d71e700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f36380ffed0 0x7f36381002b0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:59.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.216+0000 7f3635ffb700 1 -- 172.21.15.47:0/3088814386 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f3638105ee0 con 0x7f36380ffed0 2024-02-23T00:45:59.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.216+0000 7f3635ffb700 10 client.41003.objecter ms_dispatch 0x7f3638001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:45:59.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.216+0000 7f3635ffb700 10 client.41003.objecter _finish_command 1 = system:0 2024-02-23T00:45:59.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.216+0000 7f3635ffb700 20 client.41003.objecter put_session s=0x7f36381169f0 osd=0 4 2024-02-23T00:45:59.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.216+0000 7f3635ffb700 15 client.41003.objecter _session_command_op_remove 0 1 2024-02-23T00:45:59.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.231+0000 7f363f181700 10 client.41003.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:45:59.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.231+0000 7f363f181700 15 client.41003.objecter _session_command_op_assign -1 2 2024-02-23T00:45:59.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.231+0000 7f363f181700 20 client.41003.objecter _get_session s=0x7f36381169f0 osd=0 3 2024-02-23T00:45:59.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.231+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f36381169f0 osd=0 3 2024-02-23T00:45:59.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.231+0000 7f363f181700 20 client.41003.objecter _get_session s=0x7f36381169f0 osd=0 3 2024-02-23T00:45:59.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.231+0000 7f363f181700 15 client.41003.objecter _session_command_op_remove -1 2 2024-02-23T00:45:59.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.231+0000 7f363f181700 20 client.41003.objecter get_session s=0x7f36381169f0 osd=0 3 2024-02-23T00:45:59.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.231+0000 7f363f181700 15 client.41003.objecter _session_command_op_assign 0 2 2024-02-23T00:45:59.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.231+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f36381169f0 osd=0 4 2024-02-23T00:45:59.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.231+0000 7f363f181700 10 client.41003.objecter _send_command 2 2024-02-23T00:45:59.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.231+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f36380273c0 con 0x7f36380ffed0 2024-02-23T00:45:59.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.232+0000 7f3635ffb700 1 -- 172.21.15.47:0/3088814386 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f36380273c0 con 0x7f36380ffed0 2024-02-23T00:45:59.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.232+0000 7f3635ffb700 10 client.41003.objecter ms_dispatch 0x7f3638001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:45:59.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.232+0000 7f3635ffb700 10 client.41003.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:45:59.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.232+0000 7f3635ffb700 20 client.41003.objecter put_session s=0x7f36381169f0 osd=0 4 2024-02-23T00:45:59.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.232+0000 7f3635ffb700 15 client.41003.objecter _session_command_op_remove 0 2 2024-02-23T00:45:59.235 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:45:59.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.233+0000 7f363f181700 10 client.41003.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:59.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.233+0000 7f363f181700 15 client.41003.objecter _session_command_op_assign -1 3 2024-02-23T00:45:59.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.233+0000 7f363f181700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f36380598f0 0x7f3638059d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:59.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.233+0000 7f363f181700 20 client.41003.objecter _get_session s=0x7f36380286b0 osd=1 3 2024-02-23T00:45:59.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.233+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f36380286b0 osd=1 3 2024-02-23T00:45:59.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.233+0000 7f363f181700 20 client.41003.objecter _get_session s=0x7f36380286b0 osd=1 3 2024-02-23T00:45:59.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.233+0000 7f363f181700 15 client.41003.objecter _session_command_op_remove -1 3 2024-02-23T00:45:59.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.233+0000 7f363f181700 20 client.41003.objecter get_session s=0x7f36380286b0 osd=1 3 2024-02-23T00:45:59.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.233+0000 7f363f181700 15 client.41003.objecter _session_command_op_assign 1 3 2024-02-23T00:45:59.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.233+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f36380286b0 osd=1 4 2024-02-23T00:45:59.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.233+0000 7f363f181700 10 client.41003.objecter _send_command 3 2024-02-23T00:45:59.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.233+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3638028820 con 0x7f36380598f0 2024-02-23T00:45:59.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.233+0000 7f363cf1d700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f36380598f0 0x7f3638059d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:59.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.233+0000 7f3635ffb700 10 client.41003.objecter ms_handle_connect 0x7f36380598f0 2024-02-23T00:45:59.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.233+0000 7f363cf1d700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f36380598f0 0x7f3638059d30 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:59.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.234+0000 7f3635ffb700 1 -- 172.21.15.47:0/3088814386 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f362c01d420 con 0x7f36380598f0 2024-02-23T00:45:59.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.234+0000 7f3635ffb700 10 client.41003.objecter ms_dispatch 0x7f3638001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:45:59.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.234+0000 7f3635ffb700 10 client.41003.objecter _finish_command 3 = system:0 2024-02-23T00:45:59.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.234+0000 7f3635ffb700 20 client.41003.objecter put_session s=0x7f36380286b0 osd=1 4 2024-02-23T00:45:59.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.234+0000 7f3635ffb700 15 client.41003.objecter _session_command_op_remove 1 3 2024-02-23T00:45:59.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.249+0000 7f363f181700 10 client.41003.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:45:59.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.249+0000 7f363f181700 15 client.41003.objecter _session_command_op_assign -1 4 2024-02-23T00:45:59.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.249+0000 7f363f181700 20 client.41003.objecter _get_session s=0x7f36380286b0 osd=1 3 2024-02-23T00:45:59.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.249+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f36380286b0 osd=1 3 2024-02-23T00:45:59.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.249+0000 7f363f181700 20 client.41003.objecter _get_session s=0x7f36380286b0 osd=1 3 2024-02-23T00:45:59.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.249+0000 7f363f181700 15 client.41003.objecter _session_command_op_remove -1 4 2024-02-23T00:45:59.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.249+0000 7f363f181700 20 client.41003.objecter get_session s=0x7f36380286b0 osd=1 3 2024-02-23T00:45:59.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.249+0000 7f363f181700 15 client.41003.objecter _session_command_op_assign 1 4 2024-02-23T00:45:59.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.249+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f36380286b0 osd=1 4 2024-02-23T00:45:59.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.249+0000 7f363f181700 10 client.41003.objecter _send_command 4 2024-02-23T00:45:59.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.249+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f3638059360 con 0x7f36380598f0 2024-02-23T00:45:59.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.250+0000 7f3635ffb700 1 -- 172.21.15.47:0/3088814386 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f3638059360 con 0x7f36380598f0 2024-02-23T00:45:59.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.250+0000 7f3635ffb700 10 client.41003.objecter ms_dispatch 0x7f3638001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:45:59.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.250+0000 7f3635ffb700 10 client.41003.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:45:59.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.250+0000 7f3635ffb700 20 client.41003.objecter put_session s=0x7f36380286b0 osd=1 4 2024-02-23T00:45:59.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.250+0000 7f3635ffb700 15 client.41003.objecter _session_command_op_remove 1 4 2024-02-23T00:45:59.253 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:45:59.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.251+0000 7f363f181700 10 client.41003.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:59.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.251+0000 7f363f181700 15 client.41003.objecter _session_command_op_assign -1 5 2024-02-23T00:45:59.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.251+0000 7f363f181700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f3638028210 0x7f363811e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:59.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.251+0000 7f363f181700 20 client.41003.objecter _get_session s=0x7f3638059770 osd=2 3 2024-02-23T00:45:59.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.251+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f3638059770 osd=2 3 2024-02-23T00:45:59.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.251+0000 7f363f181700 20 client.41003.objecter _get_session s=0x7f3638059770 osd=2 3 2024-02-23T00:45:59.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.251+0000 7f363f181700 15 client.41003.objecter _session_command_op_remove -1 5 2024-02-23T00:45:59.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.251+0000 7f363f181700 20 client.41003.objecter get_session s=0x7f3638059770 osd=2 3 2024-02-23T00:45:59.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.251+0000 7f363f181700 15 client.41003.objecter _session_command_op_assign 2 5 2024-02-23T00:45:59.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.251+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f3638059770 osd=2 4 2024-02-23T00:45:59.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.251+0000 7f363f181700 10 client.41003.objecter _send_command 5 2024-02-23T00:45:59.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.251+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3638120dd0 con 0x7f3638028210 2024-02-23T00:45:59.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.251+0000 7f3637fff700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f3638028210 0x7f363811e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:59.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.251+0000 7f3635ffb700 10 client.41003.objecter ms_handle_connect 0x7f3638028210 2024-02-23T00:45:59.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.251+0000 7f3637fff700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f3638028210 0x7f363811e180 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:59.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.252+0000 7f3635ffb700 1 -- 172.21.15.47:0/3088814386 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f3638120dd0 con 0x7f3638028210 2024-02-23T00:45:59.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.252+0000 7f3635ffb700 10 client.41003.objecter ms_dispatch 0x7f3638001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:45:59.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.252+0000 7f3635ffb700 10 client.41003.objecter _finish_command 5 = system:0 2024-02-23T00:45:59.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.252+0000 7f3635ffb700 20 client.41003.objecter put_session s=0x7f3638059770 osd=2 4 2024-02-23T00:45:59.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.252+0000 7f3635ffb700 15 client.41003.objecter _session_command_op_remove 2 5 2024-02-23T00:45:59.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.266+0000 7f363f181700 10 client.41003.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:45:59.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.266+0000 7f363f181700 15 client.41003.objecter _session_command_op_assign -1 6 2024-02-23T00:45:59.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.266+0000 7f363f181700 20 client.41003.objecter _get_session s=0x7f3638059770 osd=2 3 2024-02-23T00:45:59.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.266+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f3638059770 osd=2 3 2024-02-23T00:45:59.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.266+0000 7f363f181700 20 client.41003.objecter _get_session s=0x7f3638059770 osd=2 3 2024-02-23T00:45:59.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.266+0000 7f363f181700 15 client.41003.objecter _session_command_op_remove -1 6 2024-02-23T00:45:59.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.266+0000 7f363f181700 20 client.41003.objecter get_session s=0x7f3638059770 osd=2 3 2024-02-23T00:45:59.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.266+0000 7f363f181700 15 client.41003.objecter _session_command_op_assign 2 6 2024-02-23T00:45:59.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.266+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f3638059770 osd=2 4 2024-02-23T00:45:59.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.266+0000 7f363f181700 10 client.41003.objecter _send_command 6 2024-02-23T00:45:59.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.266+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f3638027770 con 0x7f3638028210 2024-02-23T00:45:59.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.268+0000 7f3635ffb700 1 -- 172.21.15.47:0/3088814386 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f3638027770 con 0x7f3638028210 2024-02-23T00:45:59.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.268+0000 7f3635ffb700 10 client.41003.objecter ms_dispatch 0x7f3638001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:45:59.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.268+0000 7f3635ffb700 10 client.41003.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:45:59.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.268+0000 7f3635ffb700 20 client.41003.objecter put_session s=0x7f3638059770 osd=2 4 2024-02-23T00:45:59.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.268+0000 7f3635ffb700 15 client.41003.objecter _session_command_op_remove 2 6 2024-02-23T00:45:59.271 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:45:59.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.268+0000 7f363f181700 10 client.41003.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:59.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.268+0000 7f363f181700 15 client.41003.objecter _session_command_op_assign -1 7 2024-02-23T00:45:59.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.268+0000 7f363f181700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f363811f7f0 0x7f363811fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:59.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.268+0000 7f363f181700 20 client.41003.objecter _get_session s=0x7f363811b7c0 osd=3 3 2024-02-23T00:45:59.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.268+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f363811b7c0 osd=3 3 2024-02-23T00:45:59.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.268+0000 7f363f181700 20 client.41003.objecter _get_session s=0x7f363811b7c0 osd=3 3 2024-02-23T00:45:59.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.268+0000 7f363f181700 15 client.41003.objecter _session_command_op_remove -1 7 2024-02-23T00:45:59.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.268+0000 7f363f181700 20 client.41003.objecter get_session s=0x7f363811b7c0 osd=3 3 2024-02-23T00:45:59.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.268+0000 7f363f181700 15 client.41003.objecter _session_command_op_assign 3 7 2024-02-23T00:45:59.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.268+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f363811b7c0 osd=3 4 2024-02-23T00:45:59.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.268+0000 7f363f181700 10 client.41003.objecter _send_command 7 2024-02-23T00:45:59.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.268+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f363811bf40 con 0x7f363811f7f0 2024-02-23T00:45:59.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.268+0000 7f363d71e700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f363811f7f0 0x7f363811fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:59.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.268+0000 7f3635ffb700 10 client.41003.objecter ms_handle_connect 0x7f363811f7f0 2024-02-23T00:45:59.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.268+0000 7f363d71e700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f363811f7f0 0x7f363811fc10 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:59.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.269+0000 7f3635ffb700 1 -- 172.21.15.47:0/3088814386 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f363811bf40 con 0x7f363811f7f0 2024-02-23T00:45:59.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.269+0000 7f3635ffb700 10 client.41003.objecter ms_dispatch 0x7f3638001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:45:59.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.269+0000 7f3635ffb700 10 client.41003.objecter _finish_command 7 = system:0 2024-02-23T00:45:59.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.269+0000 7f3635ffb700 20 client.41003.objecter put_session s=0x7f363811b7c0 osd=3 4 2024-02-23T00:45:59.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.269+0000 7f3635ffb700 15 client.41003.objecter _session_command_op_remove 3 7 2024-02-23T00:45:59.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.284+0000 7f363f181700 10 client.41003.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:45:59.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.284+0000 7f363f181700 15 client.41003.objecter _session_command_op_assign -1 8 2024-02-23T00:45:59.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.284+0000 7f363f181700 20 client.41003.objecter _get_session s=0x7f363811b7c0 osd=3 3 2024-02-23T00:45:59.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.284+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f363811b7c0 osd=3 3 2024-02-23T00:45:59.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.284+0000 7f363f181700 20 client.41003.objecter _get_session s=0x7f363811b7c0 osd=3 3 2024-02-23T00:45:59.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.284+0000 7f363f181700 15 client.41003.objecter _session_command_op_remove -1 8 2024-02-23T00:45:59.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.284+0000 7f363f181700 20 client.41003.objecter get_session s=0x7f363811b7c0 osd=3 3 2024-02-23T00:45:59.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.284+0000 7f363f181700 15 client.41003.objecter _session_command_op_assign 3 8 2024-02-23T00:45:59.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.284+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f363811b7c0 osd=3 4 2024-02-23T00:45:59.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.284+0000 7f363f181700 10 client.41003.objecter _send_command 8 2024-02-23T00:45:59.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.284+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f36381208d0 con 0x7f363811f7f0 2024-02-23T00:45:59.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.285+0000 7f3635ffb700 1 -- 172.21.15.47:0/3088814386 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f36381208d0 con 0x7f363811f7f0 2024-02-23T00:45:59.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.285+0000 7f3635ffb700 10 client.41003.objecter ms_dispatch 0x7f3638001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:45:59.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.285+0000 7f3635ffb700 10 client.41003.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:45:59.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.285+0000 7f3635ffb700 20 client.41003.objecter put_session s=0x7f363811b7c0 osd=3 4 2024-02-23T00:45:59.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.285+0000 7f3635ffb700 15 client.41003.objecter _session_command_op_remove 3 8 2024-02-23T00:45:59.288 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:45:59.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.285+0000 7f363f181700 10 client.41003.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:59.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.285+0000 7f363f181700 15 client.41003.objecter _session_command_op_assign -1 9 2024-02-23T00:45:59.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.285+0000 7f363f181700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f3638122f80 0x7f3638125370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:59.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.285+0000 7f363f181700 20 client.41003.objecter _get_session s=0x7f3638120100 osd=4 3 2024-02-23T00:45:59.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.285+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f3638120100 osd=4 3 2024-02-23T00:45:59.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.285+0000 7f363f181700 20 client.41003.objecter _get_session s=0x7f3638120100 osd=4 3 2024-02-23T00:45:59.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.285+0000 7f363f181700 15 client.41003.objecter _session_command_op_remove -1 9 2024-02-23T00:45:59.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.285+0000 7f363f181700 20 client.41003.objecter get_session s=0x7f3638120100 osd=4 3 2024-02-23T00:45:59.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.285+0000 7f363f181700 15 client.41003.objecter _session_command_op_assign 4 9 2024-02-23T00:45:59.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.285+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f3638120100 osd=4 4 2024-02-23T00:45:59.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.285+0000 7f363f181700 10 client.41003.objecter _send_command 9 2024-02-23T00:45:59.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.285+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f36381208d0 con 0x7f3638122f80 2024-02-23T00:45:59.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.286+0000 7f363cf1d700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f3638122f80 0x7f3638125370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:59.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.286+0000 7f3635ffb700 10 client.41003.objecter ms_handle_connect 0x7f3638122f80 2024-02-23T00:45:59.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.286+0000 7f363cf1d700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f3638122f80 0x7f3638125370 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:59.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.287+0000 7f3635ffb700 1 -- 172.21.15.47:0/3088814386 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f36381208d0 con 0x7f3638122f80 2024-02-23T00:45:59.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.287+0000 7f3635ffb700 10 client.41003.objecter ms_dispatch 0x7f3638001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:45:59.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.287+0000 7f3635ffb700 10 client.41003.objecter _finish_command 9 = system:0 2024-02-23T00:45:59.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.287+0000 7f3635ffb700 20 client.41003.objecter put_session s=0x7f3638120100 osd=4 4 2024-02-23T00:45:59.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.287+0000 7f3635ffb700 15 client.41003.objecter _session_command_op_remove 4 9 2024-02-23T00:45:59.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.302+0000 7f363f181700 10 client.41003.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:45:59.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.302+0000 7f363f181700 15 client.41003.objecter _session_command_op_assign -1 10 2024-02-23T00:45:59.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.302+0000 7f363f181700 20 client.41003.objecter _get_session s=0x7f3638120100 osd=4 3 2024-02-23T00:45:59.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.302+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f3638120100 osd=4 3 2024-02-23T00:45:59.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.302+0000 7f363f181700 20 client.41003.objecter _get_session s=0x7f3638120100 osd=4 3 2024-02-23T00:45:59.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.302+0000 7f363f181700 15 client.41003.objecter _session_command_op_remove -1 10 2024-02-23T00:45:59.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.302+0000 7f363f181700 20 client.41003.objecter get_session s=0x7f3638120100 osd=4 3 2024-02-23T00:45:59.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.302+0000 7f363f181700 15 client.41003.objecter _session_command_op_assign 4 10 2024-02-23T00:45:59.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.302+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f3638120100 osd=4 4 2024-02-23T00:45:59.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.302+0000 7f363f181700 10 client.41003.objecter _send_command 10 2024-02-23T00:45:59.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.302+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f36381208d0 con 0x7f3638122f80 2024-02-23T00:45:59.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.304+0000 7f3635ffb700 1 -- 172.21.15.47:0/3088814386 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f36381208d0 con 0x7f3638122f80 2024-02-23T00:45:59.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.304+0000 7f3635ffb700 10 client.41003.objecter ms_dispatch 0x7f3638001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:45:59.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.304+0000 7f3635ffb700 10 client.41003.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:45:59.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.304+0000 7f3635ffb700 20 client.41003.objecter put_session s=0x7f3638120100 osd=4 4 2024-02-23T00:45:59.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.304+0000 7f3635ffb700 15 client.41003.objecter _session_command_op_remove 4 10 2024-02-23T00:45:59.307 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:45:59.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.304+0000 7f363f181700 10 client.41003.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:59.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.304+0000 7f363f181700 15 client.41003.objecter _session_command_op_assign -1 11 2024-02-23T00:45:59.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.304+0000 7f363f181700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f3638125e00 0x7f3638126260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:59.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.304+0000 7f363f181700 20 client.41003.objecter _get_session s=0x7f36381208d0 osd=5 3 2024-02-23T00:45:59.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.304+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f36381208d0 osd=5 3 2024-02-23T00:45:59.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.304+0000 7f363f181700 20 client.41003.objecter _get_session s=0x7f36381208d0 osd=5 3 2024-02-23T00:45:59.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.304+0000 7f363f181700 15 client.41003.objecter _session_command_op_remove -1 11 2024-02-23T00:45:59.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.304+0000 7f363f181700 20 client.41003.objecter get_session s=0x7f36381208d0 osd=5 3 2024-02-23T00:45:59.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.304+0000 7f363f181700 15 client.41003.objecter _session_command_op_assign 5 11 2024-02-23T00:45:59.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.304+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f36381208d0 osd=5 4 2024-02-23T00:45:59.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.304+0000 7f363f181700 10 client.41003.objecter _send_command 11 2024-02-23T00:45:59.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.304+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f363812b1b0 con 0x7f3638125e00 2024-02-23T00:45:59.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.305+0000 7f3637fff700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f3638125e00 0x7f3638126260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:59.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.305+0000 7f3635ffb700 10 client.41003.objecter ms_handle_connect 0x7f3638125e00 2024-02-23T00:45:59.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.305+0000 7f3637fff700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f3638125e00 0x7f3638126260 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:59.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.306+0000 7f3635ffb700 1 -- 172.21.15.47:0/3088814386 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f363812b1b0 con 0x7f3638125e00 2024-02-23T00:45:59.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.306+0000 7f3635ffb700 10 client.41003.objecter ms_dispatch 0x7f3638001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:45:59.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.306+0000 7f3635ffb700 10 client.41003.objecter _finish_command 11 = system:0 2024-02-23T00:45:59.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.306+0000 7f3635ffb700 20 client.41003.objecter put_session s=0x7f36381208d0 osd=5 4 2024-02-23T00:45:59.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.306+0000 7f3635ffb700 15 client.41003.objecter _session_command_op_remove 5 11 2024-02-23T00:45:59.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.321+0000 7f363f181700 10 client.41003.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:45:59.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.321+0000 7f363f181700 15 client.41003.objecter _session_command_op_assign -1 12 2024-02-23T00:45:59.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.321+0000 7f363f181700 20 client.41003.objecter _get_session s=0x7f36381208d0 osd=5 3 2024-02-23T00:45:59.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.321+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f36381208d0 osd=5 3 2024-02-23T00:45:59.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.321+0000 7f363f181700 20 client.41003.objecter _get_session s=0x7f36381208d0 osd=5 3 2024-02-23T00:45:59.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.321+0000 7f363f181700 15 client.41003.objecter _session_command_op_remove -1 12 2024-02-23T00:45:59.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.321+0000 7f363f181700 20 client.41003.objecter get_session s=0x7f36381208d0 osd=5 3 2024-02-23T00:45:59.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.321+0000 7f363f181700 15 client.41003.objecter _session_command_op_assign 5 12 2024-02-23T00:45:59.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.321+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f36381208d0 osd=5 4 2024-02-23T00:45:59.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.321+0000 7f363f181700 10 client.41003.objecter _send_command 12 2024-02-23T00:45:59.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.321+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f3638126860 con 0x7f3638125e00 2024-02-23T00:45:59.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.323+0000 7f3635ffb700 1 -- 172.21.15.47:0/3088814386 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f3638126860 con 0x7f3638125e00 2024-02-23T00:45:59.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.323+0000 7f3635ffb700 10 client.41003.objecter ms_dispatch 0x7f3638001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:45:59.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.323+0000 7f3635ffb700 10 client.41003.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:45:59.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.323+0000 7f3635ffb700 20 client.41003.objecter put_session s=0x7f36381208d0 osd=5 4 2024-02-23T00:45:59.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.323+0000 7f3635ffb700 15 client.41003.objecter _session_command_op_remove 5 12 2024-02-23T00:45:59.326 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:45:59.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.323+0000 7f363f181700 10 client.41003.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:59.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.323+0000 7f363f181700 15 client.41003.objecter _session_command_op_assign -1 13 2024-02-23T00:45:59.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.323+0000 7f363f181700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f3638126f70 0x7f36381273b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:59.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.323+0000 7f363f181700 20 client.41003.objecter _get_session s=0x7f3638126860 osd=6 3 2024-02-23T00:45:59.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.323+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f3638126860 osd=6 3 2024-02-23T00:45:59.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.323+0000 7f363f181700 20 client.41003.objecter _get_session s=0x7f3638126860 osd=6 3 2024-02-23T00:45:59.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.323+0000 7f363f181700 15 client.41003.objecter _session_command_op_remove -1 13 2024-02-23T00:45:59.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.323+0000 7f363f181700 20 client.41003.objecter get_session s=0x7f3638126860 osd=6 3 2024-02-23T00:45:59.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.323+0000 7f363f181700 15 client.41003.objecter _session_command_op_assign 6 13 2024-02-23T00:45:59.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.323+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f3638126860 osd=6 4 2024-02-23T00:45:59.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.323+0000 7f363f181700 10 client.41003.objecter _send_command 13 2024-02-23T00:45:59.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.323+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3638127a60 con 0x7f3638126f70 2024-02-23T00:45:59.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.323+0000 7f363d71e700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f3638126f70 0x7f36381273b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:59.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.324+0000 7f3635ffb700 10 client.41003.objecter ms_handle_connect 0x7f3638126f70 2024-02-23T00:45:59.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.324+0000 7f363d71e700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f3638126f70 0x7f36381273b0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:59.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.324+0000 7f3635ffb700 1 -- 172.21.15.47:0/3088814386 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f3638127a60 con 0x7f3638126f70 2024-02-23T00:45:59.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.324+0000 7f3635ffb700 10 client.41003.objecter ms_dispatch 0x7f3638001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:45:59.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.324+0000 7f3635ffb700 10 client.41003.objecter _finish_command 13 = system:0 2024-02-23T00:45:59.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.324+0000 7f3635ffb700 20 client.41003.objecter put_session s=0x7f3638126860 osd=6 4 2024-02-23T00:45:59.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.324+0000 7f3635ffb700 15 client.41003.objecter _session_command_op_remove 6 13 2024-02-23T00:45:59.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.340+0000 7f363f181700 10 client.41003.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:45:59.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.340+0000 7f363f181700 15 client.41003.objecter _session_command_op_assign -1 14 2024-02-23T00:45:59.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.340+0000 7f363f181700 20 client.41003.objecter _get_session s=0x7f3638126860 osd=6 3 2024-02-23T00:45:59.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.340+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f3638126860 osd=6 3 2024-02-23T00:45:59.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.340+0000 7f363f181700 20 client.41003.objecter _get_session s=0x7f3638126860 osd=6 3 2024-02-23T00:45:59.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.340+0000 7f363f181700 15 client.41003.objecter _session_command_op_remove -1 14 2024-02-23T00:45:59.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.340+0000 7f363f181700 20 client.41003.objecter get_session s=0x7f3638126860 osd=6 3 2024-02-23T00:45:59.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.340+0000 7f363f181700 15 client.41003.objecter _session_command_op_assign 6 14 2024-02-23T00:45:59.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.340+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f3638126860 osd=6 4 2024-02-23T00:45:59.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.340+0000 7f363f181700 10 client.41003.objecter _send_command 14 2024-02-23T00:45:59.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.340+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f3638127a60 con 0x7f3638126f70 2024-02-23T00:45:59.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.342+0000 7f3635ffb700 1 -- 172.21.15.47:0/3088814386 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f3638127a60 con 0x7f3638126f70 2024-02-23T00:45:59.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.342+0000 7f3635ffb700 10 client.41003.objecter ms_dispatch 0x7f3638001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:45:59.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.342+0000 7f3635ffb700 10 client.41003.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:45:59.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.342+0000 7f3635ffb700 20 client.41003.objecter put_session s=0x7f3638126860 osd=6 4 2024-02-23T00:45:59.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.342+0000 7f3635ffb700 15 client.41003.objecter _session_command_op_remove 6 14 2024-02-23T00:45:59.345 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:45:59.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.342+0000 7f363f181700 10 client.41003.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:45:59.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.342+0000 7f363f181700 15 client.41003.objecter _session_command_op_assign -1 15 2024-02-23T00:45:59.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.342+0000 7f363f181700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f363812c5a0 0x7f363812ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:45:59.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.342+0000 7f363f181700 20 client.41003.objecter _get_session s=0x7f3638127a60 osd=7 3 2024-02-23T00:45:59.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.342+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f3638127a60 osd=7 3 2024-02-23T00:45:59.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.342+0000 7f363f181700 20 client.41003.objecter _get_session s=0x7f3638127a60 osd=7 3 2024-02-23T00:45:59.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.342+0000 7f363f181700 15 client.41003.objecter _session_command_op_remove -1 15 2024-02-23T00:45:59.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.342+0000 7f363f181700 20 client.41003.objecter get_session s=0x7f3638127a60 osd=7 3 2024-02-23T00:45:59.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.342+0000 7f363f181700 15 client.41003.objecter _session_command_op_assign 7 15 2024-02-23T00:45:59.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.342+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f3638127a60 osd=7 4 2024-02-23T00:45:59.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.342+0000 7f363f181700 10 client.41003.objecter _send_command 15 2024-02-23T00:45:59.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.342+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3638131850 con 0x7f363812c5a0 2024-02-23T00:45:59.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.342+0000 7f363cf1d700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f363812c5a0 0x7f363812ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:45:59.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.343+0000 7f3635ffb700 10 client.41003.objecter ms_handle_connect 0x7f363812c5a0 2024-02-23T00:45:59.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.343+0000 7f363cf1d700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f363812c5a0 0x7f363812ea10 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:45:59.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.344+0000 7f3635ffb700 1 -- 172.21.15.47:0/3088814386 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f3638131850 con 0x7f363812c5a0 2024-02-23T00:45:59.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.344+0000 7f3635ffb700 10 client.41003.objecter ms_dispatch 0x7f3638001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:45:59.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.344+0000 7f3635ffb700 10 client.41003.objecter _finish_command 15 = system:0 2024-02-23T00:45:59.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.344+0000 7f3635ffb700 20 client.41003.objecter put_session s=0x7f3638127a60 osd=7 4 2024-02-23T00:45:59.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.344+0000 7f3635ffb700 15 client.41003.objecter _session_command_op_remove 7 15 2024-02-23T00:45:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.359+0000 7f363f181700 10 client.41003.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:45:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.359+0000 7f363f181700 15 client.41003.objecter _session_command_op_assign -1 16 2024-02-23T00:45:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.359+0000 7f363f181700 20 client.41003.objecter _get_session s=0x7f3638127a60 osd=7 3 2024-02-23T00:45:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.359+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f3638127a60 osd=7 3 2024-02-23T00:45:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.359+0000 7f363f181700 20 client.41003.objecter _get_session s=0x7f3638127a60 osd=7 3 2024-02-23T00:45:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.359+0000 7f363f181700 15 client.41003.objecter _session_command_op_remove -1 16 2024-02-23T00:45:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.359+0000 7f363f181700 20 client.41003.objecter get_session s=0x7f3638127a60 osd=7 3 2024-02-23T00:45:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.359+0000 7f363f181700 15 client.41003.objecter _session_command_op_assign 7 16 2024-02-23T00:45:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.359+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f3638127a60 osd=7 4 2024-02-23T00:45:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.359+0000 7f363f181700 10 client.41003.objecter _send_command 16 2024-02-23T00:45:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.359+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f3638131850 con 0x7f363812c5a0 2024-02-23T00:45:59.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f3635ffb700 1 -- 172.21.15.47:0/3088814386 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f3638131850 con 0x7f363812c5a0 2024-02-23T00:45:59.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f3635ffb700 10 client.41003.objecter ms_dispatch 0x7f3638001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:45:59.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f3635ffb700 10 client.41003.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:45:59.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f3635ffb700 20 client.41003.objecter put_session s=0x7f3638127a60 osd=7 4 2024-02-23T00:45:59.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f3635ffb700 15 client.41003.objecter _session_command_op_remove 7 16 2024-02-23T00:45:59.364 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:45:59.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 10 librados: watch_flush enter 2024-02-23T00:45:59.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 10 librados: watch_flush exit 2024-02-23T00:45:59.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 10 client.41003.objecter close_session for osd.0 2024-02-23T00:45:59.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f36380ffed0 msgr2=0x7f36381002b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:59.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f36380ffed0 0x7f36381002b0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f36381169f0 osd=0 1 2024-02-23T00:45:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 10 client.41003.objecter close_session for osd.1 2024-02-23T00:45:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f36380598f0 msgr2=0x7f3638059d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f36380598f0 0x7f3638059d30 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f36380286b0 osd=1 1 2024-02-23T00:45:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 10 client.41003.objecter close_session for osd.2 2024-02-23T00:45:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f3638028210 msgr2=0x7f363811e180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f3638028210 0x7f363811e180 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f3638059770 osd=2 1 2024-02-23T00:45:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 10 client.41003.objecter close_session for osd.3 2024-02-23T00:45:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f363811f7f0 msgr2=0x7f363811fc10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f363811f7f0 0x7f363811fc10 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363d71e700 1 -- 172.21.15.47:0/3088814386 reap_dead start 2024-02-23T00:45:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f363811b7c0 osd=3 1 2024-02-23T00:45:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 10 client.41003.objecter close_session for osd.4 2024-02-23T00:45:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f3638122f80 msgr2=0x7f3638125370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f3638122f80 0x7f3638125370 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f3638120100 osd=4 1 2024-02-23T00:45:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 10 client.41003.objecter close_session for osd.5 2024-02-23T00:45:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363d71e700 1 -- 172.21.15.47:0/3088814386 reap_dead start 2024-02-23T00:45:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f3638125e00 msgr2=0x7f3638126260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f3638125e00 0x7f3638126260 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f36381208d0 osd=5 1 2024-02-23T00:45:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 10 client.41003.objecter close_session for osd.6 2024-02-23T00:45:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f3638126f70 msgr2=0x7f36381273b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f3638126f70 0x7f36381273b0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f3638126860 osd=6 1 2024-02-23T00:45:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 10 client.41003.objecter close_session for osd.7 2024-02-23T00:45:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f363812c5a0 msgr2=0x7f363812ea10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f363812c5a0 0x7f363812ea10 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:45:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 20 client.41003.objecter put_session s=0x7f3638127a60 osd=7 1 2024-02-23T00:45:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 20 client.41003.objecter shutdown clearing up homeless session... 2024-02-23T00:45:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 10 client.41003.objecter successfully canceled tick 2024-02-23T00:45:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3628062cc0 msgr2=0x7f3628065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3628062cc0 0x7f3628065180 secure :-1 s=READY pgs=3006 cs=0 l=1 rev1=1 rx=0x7f36380fa150 tx=0x7f3624009060).stop 2024-02-23T00:45:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f36380f7400 msgr2=0x7f36380f89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:45:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363f181700 1 --2- 172.21.15.47:0/3088814386 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f36380f7400 0x7f36380f89b0 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 rx=0x7f362c009da0 tx=0x7f362c009cb0).stop 2024-02-23T00:45:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.361+0000 7f363d71e700 1 -- 172.21.15.47:0/3088814386 reap_dead start 2024-02-23T00:45:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.362+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 shutdown_connections 2024-02-23T00:45:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.362+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 >> 172.21.15.47:0/3088814386 conn(0x7f36380f2820 msgr2=0x7f3638060f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:45:59.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.362+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 shutdown_connections 2024-02-23T00:45:59.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.362+0000 7f363f181700 1 -- 172.21.15.47:0/3088814386 wait complete. 2024-02-23T00:45:59.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:45:59.362+0000 7f363f181700 1 librados: shutdown 2024-02-23T00:45:59.369 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:45:59.369 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:45:59.369 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:45:59.369 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:45:59.369 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:45:59.369 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:45:59.369 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:45:59.369 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:45:59.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:58.612614+0000 osd.0 (osd.0) 119 : cluster [DBG] 18.8 scrub starts 2024-02-23T00:45:59.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:58.613493+0000 osd.0 (osd.0) 120 : cluster [DBG] 18.8 scrub ok 2024-02-23T00:45:59.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:58.722603+0000 osd.7 (osd.7) 145 : cluster [DBG] 62.1e scrub starts 2024-02-23T00:45:59.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:58.723982+0000 osd.7 (osd.7) 146 : cluster [DBG] 62.1e scrub ok 2024-02-23T00:45:59.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:59.150117+0000 mon.a (mon.0) 1936 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi047-82386-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi047-82386-49"}]': finished 2024-02-23T00:45:59.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:59.150271+0000 mon.a (mon.0) 1937 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-33"}]': finished 2024-02-23T00:45:59.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:59.150320+0000 mon.a (mon.0) 1938 : cluster [DBG] osdmap e287: 8 total, 8 up, 8 in 2024-02-23T00:45:59.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:59.154470+0000 mon.c (mon.2) 209 : audit [INF] from='client.? 172.21.15.47:0/422202595' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi047-82378-39"}]: dispatch 2024-02-23T00:45:59.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:59.155098+0000 mon.a (mon.0) 1939 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi047-82378-39"}]: dispatch 2024-02-23T00:45:59.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:59.215028+0000 mon.c (mon.2) 210 : audit [DBG] from='client.? 172.21.15.47:0/3088814386' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:45:59.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:45:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:45:59.250608+0000 mon.a (mon.0) 1940 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "28"}]: dispatch 2024-02-23T00:46:00.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:58.612614+0000 osd.0 (osd.0) 119 : cluster [DBG] 18.8 scrub starts 2024-02-23T00:46:00.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:58.613493+0000 osd.0 (osd.0) 120 : cluster [DBG] 18.8 scrub ok 2024-02-23T00:46:00.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:58.722603+0000 osd.7 (osd.7) 145 : cluster [DBG] 62.1e scrub starts 2024-02-23T00:46:00.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:58.723982+0000 osd.7 (osd.7) 146 : cluster [DBG] 62.1e scrub ok 2024-02-23T00:46:00.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:59.150117+0000 mon.a (mon.0) 1936 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi047-82386-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi047-82386-49"}]': finished 2024-02-23T00:46:00.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:59.150271+0000 mon.a (mon.0) 1937 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-33"}]': finished 2024-02-23T00:46:00.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:59.150320+0000 mon.a (mon.0) 1938 : cluster [DBG] osdmap e287: 8 total, 8 up, 8 in 2024-02-23T00:46:00.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:59.154470+0000 mon.c (mon.2) 209 : audit [INF] from='client.? 172.21.15.47:0/422202595' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi047-82378-39"}]: dispatch 2024-02-23T00:46:00.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:59.155098+0000 mon.a (mon.0) 1939 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi047-82378-39"}]: dispatch 2024-02-23T00:46:00.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:59.215028+0000 mon.c (mon.2) 210 : audit [DBG] from='client.? 172.21.15.47:0/3088814386' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:46:00.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:45:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:45:59.250608+0000 mon.a (mon.0) 1940 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "28"}]: dispatch 2024-02-23T00:46:00.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:58.612614+0000 osd.0 (osd.0) 119 : cluster [DBG] 18.8 scrub starts 2024-02-23T00:46:00.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:58.613493+0000 osd.0 (osd.0) 120 : cluster [DBG] 18.8 scrub ok 2024-02-23T00:46:00.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:58.722603+0000 osd.7 (osd.7) 145 : cluster [DBG] 62.1e scrub starts 2024-02-23T00:46:00.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:58.723982+0000 osd.7 (osd.7) 146 : cluster [DBG] 62.1e scrub ok 2024-02-23T00:46:00.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:59.150117+0000 mon.a (mon.0) 1936 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi047-82386-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi047-82386-49"}]': finished 2024-02-23T00:46:00.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:59.150271+0000 mon.a (mon.0) 1937 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-33"}]': finished 2024-02-23T00:46:00.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:59.150320+0000 mon.a (mon.0) 1938 : cluster [DBG] osdmap e287: 8 total, 8 up, 8 in 2024-02-23T00:46:00.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:59.154470+0000 mon.c (mon.2) 209 : audit [INF] from='client.? 172.21.15.47:0/422202595' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi047-82378-39"}]: dispatch 2024-02-23T00:46:00.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:59.155098+0000 mon.a (mon.0) 1939 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi047-82378-39"}]: dispatch 2024-02-23T00:46:00.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:59.215028+0000 mon.c (mon.2) 210 : audit [DBG] from='client.? 172.21.15.47:0/3088814386' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:46:00.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:45:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:45:59.250608+0000 mon.a (mon.0) 1940 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "28"}]: dispatch 2024-02-23T00:46:00.571 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:46:00.716 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:46:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:46:00] "GET /metrics HTTP/1.1" 200 190205 "" "Prometheus/2.33.4" 2024-02-23T00:46:00.717 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:59.249622+0000 mgr.y (mgr.14517) 293 : cluster [DBG] pgmap v406: 255 pgs: 1 clean+premerge+peered, 8 unknown, 246 active+clean; 8.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:00.717 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:59.678212+0000 osd.7 (osd.7) 147 : cluster [DBG] 62.a scrub starts 2024-02-23T00:46:00.717 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:59.681665+0000 osd.7 (osd.7) 148 : cluster [DBG] 62.a scrub ok 2024-02-23T00:46:00.717 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:59.941379+0000 osd.5 (osd.5) 125 : cluster [DBG] 18.17 scrub starts 2024-02-23T00:46:00.717 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:45:59.944579+0000 osd.5 (osd.5) 126 : cluster [DBG] 18.17 scrub ok 2024-02-23T00:46:00.718 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:00.156833+0000 mon.a (mon.0) 1941 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi047-82378-39"}]': finished 2024-02-23T00:46:00.718 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:00.156942+0000 mon.a (mon.0) 1942 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "28"}]': finished 2024-02-23T00:46:00.718 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:00.156996+0000 mon.a (mon.0) 1943 : cluster [DBG] osdmap e288: 8 total, 8 up, 8 in 2024-02-23T00:46:00.718 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:00.160768+0000 mon.c (mon.2) 211 : audit [INF] from='client.? 172.21.15.47:0/422202595' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi047-82378-39"}]: dispatch 2024-02-23T00:46:00.718 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:00.162914+0000 mon.a (mon.0) 1944 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi047-82378-39"}]: dispatch 2024-02-23T00:46:00.718 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:00.195524+0000 mon.a (mon.0) 1945 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:46:00.718 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:59.249622+0000 mgr.y (mgr.14517) 293 : cluster [DBG] pgmap v406: 255 pgs: 1 clean+premerge+peered, 8 unknown, 246 active+clean; 8.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:00.718 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:59.678212+0000 osd.7 (osd.7) 147 : cluster [DBG] 62.a scrub starts 2024-02-23T00:46:00.718 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:59.681665+0000 osd.7 (osd.7) 148 : cluster [DBG] 62.a scrub ok 2024-02-23T00:46:00.718 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:59.941379+0000 osd.5 (osd.5) 125 : cluster [DBG] 18.17 scrub starts 2024-02-23T00:46:00.718 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:45:59.944579+0000 osd.5 (osd.5) 126 : cluster [DBG] 18.17 scrub ok 2024-02-23T00:46:00.719 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:00.156833+0000 mon.a (mon.0) 1941 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi047-82378-39"}]': finished 2024-02-23T00:46:00.719 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:00.156942+0000 mon.a (mon.0) 1942 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "28"}]': finished 2024-02-23T00:46:00.719 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:00.156996+0000 mon.a (mon.0) 1943 : cluster [DBG] osdmap e288: 8 total, 8 up, 8 in 2024-02-23T00:46:00.719 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:00.160768+0000 mon.c (mon.2) 211 : audit [INF] from='client.? 172.21.15.47:0/422202595' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi047-82378-39"}]: dispatch 2024-02-23T00:46:00.719 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:00.162914+0000 mon.a (mon.0) 1944 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi047-82378-39"}]: dispatch 2024-02-23T00:46:00.719 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:00.195524+0000 mon.a (mon.0) 1945 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:46:00.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:59.249622+0000 mgr.y (mgr.14517) 293 : cluster [DBG] pgmap v406: 255 pgs: 1 clean+premerge+peered, 8 unknown, 246 active+clean; 8.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:00.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:59.678212+0000 osd.7 (osd.7) 147 : cluster [DBG] 62.a scrub starts 2024-02-23T00:46:00.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:59.681665+0000 osd.7 (osd.7) 148 : cluster [DBG] 62.a scrub ok 2024-02-23T00:46:00.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:59.941379+0000 osd.5 (osd.5) 125 : cluster [DBG] 18.17 scrub starts 2024-02-23T00:46:00.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:45:59.944579+0000 osd.5 (osd.5) 126 : cluster [DBG] 18.17 scrub ok 2024-02-23T00:46:00.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:00.156833+0000 mon.a (mon.0) 1941 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi047-82378-39"}]': finished 2024-02-23T00:46:00.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:00.156942+0000 mon.a (mon.0) 1942 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "28"}]': finished 2024-02-23T00:46:00.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:00.156996+0000 mon.a (mon.0) 1943 : cluster [DBG] osdmap e288: 8 total, 8 up, 8 in 2024-02-23T00:46:00.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:00.160768+0000 mon.c (mon.2) 211 : audit [INF] from='client.? 172.21.15.47:0/422202595' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi047-82378-39"}]: dispatch 2024-02-23T00:46:00.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:00.162914+0000 mon.a (mon.0) 1944 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi047-82378-39"}]: dispatch 2024-02-23T00:46:00.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:00.195524+0000 mon.a (mon.0) 1945 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:46:00.980 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:46:00.983 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:00.984 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:46:01.085 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:46:01.117 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:01.138 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:46:01.140 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:01.141 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:46:01.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.193+0000 7f2a28b9d700 1 -- 172.21.15.47:0/290930655 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2a240f9ab0 msgr2=0x7f2a240f9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:01.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.193+0000 7f2a28b9d700 1 --2- 172.21.15.47:0/290930655 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2a240f9ab0 0x7f2a240f9e90 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 rx=0x7f2a14009be0 tx=0x7f2a14004d10).stop 2024-02-23T00:46:01.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.194+0000 7f2a28b9d700 1 -- 172.21.15.47:0/290930655 shutdown_connections 2024-02-23T00:46:01.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.194+0000 7f2a28b9d700 1 --2- 172.21.15.47:0/290930655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a240f8440 0x7f2a24104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:01.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.194+0000 7f2a28b9d700 1 --2- 172.21.15.47:0/290930655 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2a240f9ab0 0x7f2a240f9e90 unknown :-1 s=CLOSED pgs=2597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:01.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.194+0000 7f2a28b9d700 1 --2- 172.21.15.47:0/290930655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a240fa380 0x7f2a240f7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:01.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.194+0000 7f2a28b9d700 1 -- 172.21.15.47:0/290930655 >> 172.21.15.47:0/290930655 conn(0x7f2a240f3800 msgr2=0x7f2a240f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:46:01.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.194+0000 7f2a28b9d700 1 -- 172.21.15.47:0/290930655 shutdown_connections 2024-02-23T00:46:01.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.194+0000 7f2a28b9d700 1 -- 172.21.15.47:0/290930655 wait complete. 2024-02-23T00:46:01.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.194+0000 7f2a28b9d700 1 librados: starting msgr at 2024-02-23T00:46:01.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.194+0000 7f2a28b9d700 1 librados: starting objecter 2024-02-23T00:46:01.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.194+0000 7f2a28b9d700 1 Processor -- start 2024-02-23T00:46:01.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.194+0000 7f2a28b9d700 1 -- start start 2024-02-23T00:46:01.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.194+0000 7f2a28b9d700 1 librados: setting wanted keys 2024-02-23T00:46:01.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.194+0000 7f2a28b9d700 1 librados: calling monclient init 2024-02-23T00:46:01.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.195+0000 7f2a28b9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a240f8440 0x7f2a2410e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:01.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.195+0000 7f2a28b9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a240f9ab0 0x7f2a2410e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:01.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.195+0000 7f2a28b9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2a240fa380 0x7f2a24108240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:01.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.195+0000 7f2a28b9d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a24108730 con 0x7f2a240f9ab0 2024-02-23T00:46:01.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.195+0000 7f2a28b9d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a241088a0 con 0x7f2a240f8440 2024-02-23T00:46:01.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.195+0000 7f2a28b9d700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a24108a60 con 0x7f2a240fa380 2024-02-23T00:46:01.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.195+0000 7f2a22d9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2a240fa380 0x7f2a24108240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:01.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.195+0000 7f2a21d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a240f9ab0 0x7f2a2410e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:01.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.195+0000 7f2a22d9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2a240fa380 0x7f2a24108240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:56334/0 (socket says 172.21.15.47:56334) 2024-02-23T00:46:01.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.195+0000 7f2a21d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a240f9ab0 0x7f2a2410e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34254/0 (socket says 172.21.15.47:34254) 2024-02-23T00:46:01.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.195+0000 7f2a22d9d700 1 -- 172.21.15.47:0/1710420069 learned_addr learned my addr 172.21.15.47:0/1710420069 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:46:01.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.195+0000 7f2a2259c700 1 --2- 172.21.15.47:0/1710420069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a240f8440 0x7f2a2410e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:01.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.195+0000 7f2a21d9b700 1 -- 172.21.15.47:0/1710420069 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2a240fa380 msgr2=0x7f2a24108240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:01.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.195+0000 7f2a21d9b700 1 --2- 172.21.15.47:0/1710420069 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2a240fa380 0x7f2a24108240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:01.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.195+0000 7f2a21d9b700 1 -- 172.21.15.47:0/1710420069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a240f8440 msgr2=0x7f2a2410e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:01.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.195+0000 7f2a21d9b700 1 --2- 172.21.15.47:0/1710420069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a240f8440 0x7f2a2410e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:01.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.195+0000 7f2a21d9b700 1 -- 172.21.15.47:0/1710420069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a18005040 con 0x7f2a240f9ab0 2024-02-23T00:46:01.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.195+0000 7f2a21d9b700 1 --2- 172.21.15.47:0/1710420069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a240f9ab0 0x7f2a2410e590 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 rx=0x7f2a18004920 tx=0x7f2a18009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:01.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.195+0000 7f2a137fe700 10 client.?.objecter ms_handle_connect 0x7f2a240f9ab0 2024-02-23T00:46:01.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.195+0000 7f2a137fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:46:01.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.195+0000 7f2a137fe700 1 -- 172.21.15.47:0/1710420069 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a18007a40 con 0x7f2a240f9ab0 2024-02-23T00:46:01.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.195+0000 7f2a137fe700 1 -- 172.21.15.47:0/1710420069 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f2a1800f040 con 0x7f2a240f9ab0 2024-02-23T00:46:01.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.195+0000 7f2a137fe700 1 -- 172.21.15.47:0/1710420069 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a18014900 con 0x7f2a240f9ab0 2024-02-23T00:46:01.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.196+0000 7f2a28b9d700 1 -- 172.21.15.47:0/1710420069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a140097c0 con 0x7f2a240f9ab0 2024-02-23T00:46:01.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.196+0000 7f2a28b9d700 10 client.30681.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:46:01.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.196+0000 7f2a28b9d700 1 -- 172.21.15.47:0/1710420069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a24109050 con 0x7f2a240f9ab0 2024-02-23T00:46:01.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.196+0000 7f2a28b9d700 1 librados: init done 2024-02-23T00:46:01.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.196+0000 7f2a28b9d700 1 -- 172.21.15.47:0/1710420069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a2405ee80 con 0x7f2a240f9ab0 2024-02-23T00:46:01.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.197+0000 7f2a137fe700 1 -- 172.21.15.47:0/1710420069 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f2a18014d30 con 0x7f2a240f9ab0 2024-02-23T00:46:01.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.197+0000 7f2a137fe700 1 --2- 172.21.15.47:0/1710420069 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2a0c062cc0 0x7f2a0c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:01.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.197+0000 7f2a137fe700 1 -- 172.21.15.47:0/1710420069 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(289..289 src has 48..289) v4 ==== 9988+0+0 (secure 0 0 0) 0x7f2a1807d430 con 0x7f2a240f9ab0 2024-02-23T00:46:01.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.197+0000 7f2a137fe700 10 client.30681.objecter ms_dispatch 0x7f2a24001550 osd_map(289..289 src has 48..289) v4 2024-02-23T00:46:01.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.197+0000 7f2a137fe700 3 client.30681.objecter handle_osd_map got epochs [289,289] > 0 2024-02-23T00:46:01.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.197+0000 7f2a137fe700 3 client.30681.objecter handle_osd_map decoding full epoch 289 2024-02-23T00:46:01.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.197+0000 7f2a2259c700 1 --2- 172.21.15.47:0/1710420069 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2a0c062cc0 0x7f2a0c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:01.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.198+0000 7f2a137fe700 20 client.30681.objecter dump_active .. 0 homeless 2024-02-23T00:46:01.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.198+0000 7f2a137fe700 10 client.30681.objecter ms_handle_connect 0x7f2a0c062cc0 2024-02-23T00:46:01.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.198+0000 7f2a2259c700 1 --2- 172.21.15.47:0/1710420069 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2a0c062cc0 0x7f2a0c065180 secure :-1 s=READY pgs=3008 cs=0 l=1 rev1=1 rx=0x7f2a14003380 tx=0x7f2a14004d10).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:01.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.202+0000 7f2a137fe700 1 -- 172.21.15.47:0/1710420069 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f2a18049f60 con 0x7f2a240f9ab0 2024-02-23T00:46:01.242 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:46:01.270 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:01.289 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:46:01.292 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:01.292 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:46:01.393 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:46:01.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:01.391+0000 7f2a28b9d700 1 -- 172.21.15.47:0/1710420069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7f2a24107020 con 0x7f2a240f9ab0 2024-02-23T00:46:01.428 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:01.676 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:46:01.678 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:01.679 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:46:01.780 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:46:01.809 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:01.829 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:46:01.832 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:01.835 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:46:01.936 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:46:01.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:00.557638+0000 osd.0 (osd.0) 121 : cluster [DBG] 6.10 scrub starts 2024-02-23T00:46:01.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:00.558652+0000 osd.0 (osd.0) 122 : cluster [DBG] 6.10 scrub ok 2024-02-23T00:46:01.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:00.724643+0000 osd.7 (osd.7) 149 : cluster [DBG] 62.6 scrub starts 2024-02-23T00:46:01.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:00.726007+0000 osd.7 (osd.7) 150 : cluster [DBG] 62.6 scrub ok 2024-02-23T00:46:01.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:00.942252+0000 osd.5 (osd.5) 127 : cluster [DBG] 18.b scrub starts 2024-02-23T00:46:01.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:00.943539+0000 osd.5 (osd.5) 128 : cluster [DBG] 18.b scrub ok 2024-02-23T00:46:01.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:01.157645+0000 mon.a (mon.0) 1946 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi047-82378-39"}]': finished 2024-02-23T00:46:01.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:01.157768+0000 mon.a (mon.0) 1947 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:46:01.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:01.157814+0000 mon.a (mon.0) 1948 : cluster [DBG] osdmap e289: 8 total, 8 up, 8 in 2024-02-23T00:46:01.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:01.165074+0000 mon.a (mon.0) 1949 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-33"}]: dispatch 2024-02-23T00:46:01.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:01.165802+0000 mon.c (mon.2) 212 : audit [INF] from='client.? 172.21.15.47:0/3371139177' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi047-82386-49"}]: dispatch 2024-02-23T00:46:01.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:01.166123+0000 mon.a (mon.0) 1950 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi047-82386-49"}]: dispatch 2024-02-23T00:46:01.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:01.182275+0000 mon.c (mon.2) 213 : audit [INF] from='client.? 172.21.15.47:0/2639020958' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi047-82378-40"}]: dispatch 2024-02-23T00:46:01.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:01.182622+0000 mon.a (mon.0) 1951 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi047-82378-40"}]: dispatch 2024-02-23T00:46:01.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:01.183086+0000 mon.c (mon.2) 214 : audit [INF] from='client.? 172.21.15.47:0/2639020958' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi047-82378-40"}]: dispatch 2024-02-23T00:46:01.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:01.183334+0000 mon.a (mon.0) 1952 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi047-82378-40"}]: dispatch 2024-02-23T00:46:01.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:01.183740+0000 mon.c (mon.2) 215 : audit [INF] from='client.? 172.21.15.47:0/2639020958' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi047-82378-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:01.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:01.183968+0000 mon.a (mon.0) 1953 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi047-82378-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:01.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:01.392909+0000 mon.a (mon.0) 1954 : audit [INF] from='client.? 172.21.15.47:0/1710420069' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:46:01.964 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:00.557638+0000 osd.0 (osd.0) 121 : cluster [DBG] 6.10 scrub starts 2024-02-23T00:46:01.964 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:00.558652+0000 osd.0 (osd.0) 122 : cluster [DBG] 6.10 scrub ok 2024-02-23T00:46:01.964 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:00.724643+0000 osd.7 (osd.7) 149 : cluster [DBG] 62.6 scrub starts 2024-02-23T00:46:01.964 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:00.726007+0000 osd.7 (osd.7) 150 : cluster [DBG] 62.6 scrub ok 2024-02-23T00:46:01.964 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:00.942252+0000 osd.5 (osd.5) 127 : cluster [DBG] 18.b scrub starts 2024-02-23T00:46:01.964 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:00.943539+0000 osd.5 (osd.5) 128 : cluster [DBG] 18.b scrub ok 2024-02-23T00:46:01.964 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:01.157645+0000 mon.a (mon.0) 1946 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi047-82378-39"}]': finished 2024-02-23T00:46:01.965 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:01.157768+0000 mon.a (mon.0) 1947 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:46:01.965 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:01.157814+0000 mon.a (mon.0) 1948 : cluster [DBG] osdmap e289: 8 total, 8 up, 8 in 2024-02-23T00:46:01.965 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:01.165074+0000 mon.a (mon.0) 1949 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-33"}]: dispatch 2024-02-23T00:46:01.965 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:01.165802+0000 mon.c (mon.2) 212 : audit [INF] from='client.? 172.21.15.47:0/3371139177' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi047-82386-49"}]: dispatch 2024-02-23T00:46:01.965 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:01.166123+0000 mon.a (mon.0) 1950 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi047-82386-49"}]: dispatch 2024-02-23T00:46:01.965 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:01.182275+0000 mon.c (mon.2) 213 : audit [INF] from='client.? 172.21.15.47:0/2639020958' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi047-82378-40"}]: dispatch 2024-02-23T00:46:01.965 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:01.182622+0000 mon.a (mon.0) 1951 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi047-82378-40"}]: dispatch 2024-02-23T00:46:01.965 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:01.183086+0000 mon.c (mon.2) 214 : audit [INF] from='client.? 172.21.15.47:0/2639020958' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi047-82378-40"}]: dispatch 2024-02-23T00:46:01.965 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:01.183334+0000 mon.a (mon.0) 1952 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi047-82378-40"}]: dispatch 2024-02-23T00:46:01.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:00.557638+0000 osd.0 (osd.0) 121 : cluster [DBG] 6.10 scrub starts 2024-02-23T00:46:01.966 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:00.558652+0000 osd.0 (osd.0) 122 : cluster [DBG] 6.10 scrub ok 2024-02-23T00:46:01.966 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:00.724643+0000 osd.7 (osd.7) 149 : cluster [DBG] 62.6 scrub starts 2024-02-23T00:46:01.966 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:00.726007+0000 osd.7 (osd.7) 150 : cluster [DBG] 62.6 scrub ok 2024-02-23T00:46:01.966 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:00.942252+0000 osd.5 (osd.5) 127 : cluster [DBG] 18.b scrub starts 2024-02-23T00:46:01.966 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:00.943539+0000 osd.5 (osd.5) 128 : cluster [DBG] 18.b scrub ok 2024-02-23T00:46:01.966 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:01.157645+0000 mon.a (mon.0) 1946 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi047-82378-39"}]': finished 2024-02-23T00:46:01.966 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:01.157768+0000 mon.a (mon.0) 1947 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:46:01.966 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:01.157814+0000 mon.a (mon.0) 1948 : cluster [DBG] osdmap e289: 8 total, 8 up, 8 in 2024-02-23T00:46:01.966 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:01.165074+0000 mon.a (mon.0) 1949 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-33"}]: dispatch 2024-02-23T00:46:01.966 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:01.165802+0000 mon.c (mon.2) 212 : audit [INF] from='client.? 172.21.15.47:0/3371139177' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi047-82386-49"}]: dispatch 2024-02-23T00:46:01.967 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:01.166123+0000 mon.a (mon.0) 1950 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi047-82386-49"}]: dispatch 2024-02-23T00:46:01.967 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:01.182275+0000 mon.c (mon.2) 213 : audit [INF] from='client.? 172.21.15.47:0/2639020958' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi047-82378-40"}]: dispatch 2024-02-23T00:46:01.967 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:01.182622+0000 mon.a (mon.0) 1951 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi047-82378-40"}]: dispatch 2024-02-23T00:46:01.967 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:01.183086+0000 mon.c (mon.2) 214 : audit [INF] from='client.? 172.21.15.47:0/2639020958' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi047-82378-40"}]: dispatch 2024-02-23T00:46:01.967 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:01.183334+0000 mon.a (mon.0) 1952 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi047-82378-40"}]: dispatch 2024-02-23T00:46:01.967 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:01.183740+0000 mon.c (mon.2) 215 : audit [INF] from='client.? 172.21.15.47:0/2639020958' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi047-82378-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:01.967 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:01.183968+0000 mon.a (mon.0) 1953 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi047-82378-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:01.967 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:01.392909+0000 mon.a (mon.0) 1954 : audit [INF] from='client.? 172.21.15.47:0/1710420069' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:46:01.968 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:01.183740+0000 mon.c (mon.2) 215 : audit [INF] from='client.? 172.21.15.47:0/2639020958' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi047-82378-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:01.968 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:01.183968+0000 mon.a (mon.0) 1953 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi047-82378-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:01.968 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:01.392909+0000 mon.a (mon.0) 1954 : audit [INF] from='client.? 172.21.15.47:0/1710420069' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:46:01.968 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:02.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:02.161+0000 7f2a137fe700 1 -- 172.21.15.47:0/1710420069 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v290) v1 ==== 101+0+0 (secure 0 0 0) 0x7f2a1804c760 con 0x7f2a240f9ab0 2024-02-23T00:46:02.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:02.164+0000 7f2a28b9d700 10 librados: watch_flush enter 2024-02-23T00:46:02.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:02.164+0000 7f2a28b9d700 10 librados: watch_flush exit 2024-02-23T00:46:02.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:02.164+0000 7f2a28b9d700 20 client.30681.objecter shutdown clearing up homeless session... 2024-02-23T00:46:02.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:02.164+0000 7f2a28b9d700 10 client.30681.objecter successfully canceled tick 2024-02-23T00:46:02.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:02.164+0000 7f2a28b9d700 1 -- 172.21.15.47:0/1710420069 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2a0c062cc0 msgr2=0x7f2a0c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:02.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:02.164+0000 7f2a28b9d700 1 --2- 172.21.15.47:0/1710420069 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2a0c062cc0 0x7f2a0c065180 secure :-1 s=READY pgs=3008 cs=0 l=1 rev1=1 rx=0x7f2a14003380 tx=0x7f2a14004d10).stop 2024-02-23T00:46:02.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:02.164+0000 7f2a28b9d700 1 -- 172.21.15.47:0/1710420069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a240f9ab0 msgr2=0x7f2a2410e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:02.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:02.164+0000 7f2a28b9d700 1 --2- 172.21.15.47:0/1710420069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a240f9ab0 0x7f2a2410e590 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 rx=0x7f2a18004920 tx=0x7f2a18009cb0).stop 2024-02-23T00:46:02.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:02.165+0000 7f2a28b9d700 1 -- 172.21.15.47:0/1710420069 shutdown_connections 2024-02-23T00:46:02.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:02.165+0000 7f2a28b9d700 1 --2- 172.21.15.47:0/1710420069 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2a0c062cc0 0x7f2a0c065180 unknown :-1 s=CLOSED pgs=3008 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:02.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:02.165+0000 7f2a28b9d700 1 --2- 172.21.15.47:0/1710420069 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2a240fa380 0x7f2a24108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:02.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:02.165+0000 7f2a28b9d700 1 --2- 172.21.15.47:0/1710420069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a240f8440 0x7f2a2410e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:02.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:02.165+0000 7f2a28b9d700 1 --2- 172.21.15.47:0/1710420069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a240f9ab0 0x7f2a2410e590 unknown :-1 s=CLOSED pgs=2495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:02.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:02.165+0000 7f2a28b9d700 1 -- 172.21.15.47:0/1710420069 >> 172.21.15.47:0/1710420069 conn(0x7f2a240f3800 msgr2=0x7f2a240fcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:46:02.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:02.165+0000 7f2a28b9d700 1 -- 172.21.15.47:0/1710420069 shutdown_connections 2024-02-23T00:46:02.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:02.165+0000 7f2a28b9d700 1 -- 172.21.15.47:0/1710420069 wait complete. 2024-02-23T00:46:02.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:02.165+0000 7f2a28b9d700 1 librados: shutdown 2024-02-23T00:46:02.170 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is unset 2024-02-23T00:46:02.924 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:01.250055+0000 mgr.y (mgr.14517) 294 : cluster [DBG] pgmap v409: 247 pgs: 1 clean+premerge+peered, 246 active+clean; 8.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:46:02.924 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:02.069706+0000 osd.6 (osd.6) 117 : cluster [DBG] 62.1a scrub starts 2024-02-23T00:46:02.924 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:02.071760+0000 osd.6 (osd.6) 118 : cluster [DBG] 62.1a scrub ok 2024-02-23T00:46:02.924 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:02.155248+0000 mon.a (mon.0) 1955 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:46:02.924 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:02.155279+0000 mon.a (mon.0) 1956 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:46:02.925 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:02.162329+0000 mon.a (mon.0) 1957 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-33"}]': finished 2024-02-23T00:46:02.925 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:02.162395+0000 mon.a (mon.0) 1958 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi047-82386-49"}]': finished 2024-02-23T00:46:02.925 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:02.162470+0000 mon.a (mon.0) 1959 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi047-82378-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:02.925 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:02.162558+0000 mon.a (mon.0) 1960 : audit [INF] from='client.? 172.21.15.47:0/1710420069' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:46:02.925 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:02.162612+0000 mon.a (mon.0) 1961 : cluster [DBG] osdmap e290: 8 total, 8 up, 8 in 2024-02-23T00:46:02.925 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:02.168103+0000 mon.c (mon.2) 216 : audit [INF] from='client.? 172.21.15.47:0/3371139177' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi047-82386-49"}]: dispatch 2024-02-23T00:46:02.925 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:02.168442+0000 mon.c (mon.2) 217 : audit [INF] from='client.? 172.21.15.47:0/2639020958' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi047-82378-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi047-82378-40"}]: dispatch 2024-02-23T00:46:02.925 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:02.169171+0000 mon.a (mon.0) 1962 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi047-82386-49"}]: dispatch 2024-02-23T00:46:02.925 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:02.169789+0000 mon.a (mon.0) 1963 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi047-82378-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi047-82378-40"}]: dispatch 2024-02-23T00:46:02.925 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:01.250055+0000 mgr.y (mgr.14517) 294 : cluster [DBG] pgmap v409: 247 pgs: 1 clean+premerge+peered, 246 active+clean; 8.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:46:02.926 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:02.069706+0000 osd.6 (osd.6) 117 : cluster [DBG] 62.1a scrub starts 2024-02-23T00:46:02.926 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:02.071760+0000 osd.6 (osd.6) 118 : cluster [DBG] 62.1a scrub ok 2024-02-23T00:46:02.926 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:02.155248+0000 mon.a (mon.0) 1955 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:46:02.926 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:02.155279+0000 mon.a (mon.0) 1956 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:46:02.926 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:02.162329+0000 mon.a (mon.0) 1957 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-33"}]': finished 2024-02-23T00:46:02.926 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:02.162395+0000 mon.a (mon.0) 1958 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi047-82386-49"}]': finished 2024-02-23T00:46:02.926 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:02.162470+0000 mon.a (mon.0) 1959 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi047-82378-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:02.926 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:02.162558+0000 mon.a (mon.0) 1960 : audit [INF] from='client.? 172.21.15.47:0/1710420069' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:46:02.926 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:02.162612+0000 mon.a (mon.0) 1961 : cluster [DBG] osdmap e290: 8 total, 8 up, 8 in 2024-02-23T00:46:02.926 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:02.168103+0000 mon.c (mon.2) 216 : audit [INF] from='client.? 172.21.15.47:0/3371139177' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi047-82386-49"}]: dispatch 2024-02-23T00:46:02.926 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:02.168442+0000 mon.c (mon.2) 217 : audit [INF] from='client.? 172.21.15.47:0/2639020958' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi047-82378-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi047-82378-40"}]: dispatch 2024-02-23T00:46:02.926 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:02.169171+0000 mon.a (mon.0) 1962 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi047-82386-49"}]: dispatch 2024-02-23T00:46:02.926 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:02.169789+0000 mon.a (mon.0) 1963 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi047-82378-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi047-82378-40"}]: dispatch 2024-02-23T00:46:02.943 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:46:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:46:02.943 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:46:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:46:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:46:02.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:01.250055+0000 mgr.y (mgr.14517) 294 : cluster [DBG] pgmap v409: 247 pgs: 1 clean+premerge+peered, 246 active+clean; 8.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:46:02.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:02.069706+0000 osd.6 (osd.6) 117 : cluster [DBG] 62.1a scrub starts 2024-02-23T00:46:02.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:02.071760+0000 osd.6 (osd.6) 118 : cluster [DBG] 62.1a scrub ok 2024-02-23T00:46:02.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:02.155248+0000 mon.a (mon.0) 1955 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:46:02.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:02.155279+0000 mon.a (mon.0) 1956 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:46:02.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:02.162329+0000 mon.a (mon.0) 1957 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-33"}]': finished 2024-02-23T00:46:02.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:02.162395+0000 mon.a (mon.0) 1958 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi047-82386-49"}]': finished 2024-02-23T00:46:02.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:02.162470+0000 mon.a (mon.0) 1959 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi047-82378-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:02.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:02.162558+0000 mon.a (mon.0) 1960 : audit [INF] from='client.? 172.21.15.47:0/1710420069' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:46:02.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:02.162612+0000 mon.a (mon.0) 1961 : cluster [DBG] osdmap e290: 8 total, 8 up, 8 in 2024-02-23T00:46:02.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:02.168103+0000 mon.c (mon.2) 216 : audit [INF] from='client.? 172.21.15.47:0/3371139177' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi047-82386-49"}]: dispatch 2024-02-23T00:46:02.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:02.168442+0000 mon.c (mon.2) 217 : audit [INF] from='client.? 172.21.15.47:0/2639020958' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi047-82378-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi047-82378-40"}]: dispatch 2024-02-23T00:46:02.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:02.169171+0000 mon.a (mon.0) 1962 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi047-82386-49"}]: dispatch 2024-02-23T00:46:02.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:02.169789+0000 mon.a (mon.0) 1963 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi047-82378-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi047-82378-40"}]: dispatch 2024-02-23T00:46:03.514 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:46:03.519 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:46:03.522 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:03.523 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:46:03.623 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:46:03.652 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:03.852 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:46:03.855 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:03.855 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:46:03.956 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:46:03.988 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:04.009 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:46:04.012 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:04.013 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:46:04.093 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:02.477190+0000 mgr.y (mgr.14517) 295 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:46:04.093 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:03.164491+0000 mon.a (mon.0) 1964 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi047-82386-49"}]': finished 2024-02-23T00:46:04.093 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:03.603357+0000 mon.a (mon.0) 1965 : cluster [DBG] osdmap e291: 8 total, 8 up, 8 in 2024-02-23T00:46:04.093 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:02.477190+0000 mgr.y (mgr.14517) 295 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:46:04.094 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:03.164491+0000 mon.a (mon.0) 1964 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi047-82386-49"}]': finished 2024-02-23T00:46:04.094 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:03.603357+0000 mon.a (mon.0) 1965 : cluster [DBG] osdmap e291: 8 total, 8 up, 8 in 2024-02-23T00:46:04.113 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:46:04.142 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:04.162 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:46:04.164 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:04.165 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:46:04.266 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:46:04.281 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:46:04.291 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:02.477190+0000 mgr.y (mgr.14517) 295 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:46:04.292 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:03.164491+0000 mon.a (mon.0) 1964 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi047-82386-49"}]': finished 2024-02-23T00:46:04.292 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:03.603357+0000 mon.a (mon.0) 1965 : cluster [DBG] osdmap e291: 8 total, 8 up, 8 in 2024-02-23T00:46:04.293 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:04.313 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:46:04.315 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:04.316 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:46:04.417 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:46:04.444 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:04.463 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:46:04.465 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:04.466 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:46:04.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.355+0000 7fec9ee7a700 1 -- 172.21.15.47:0/1275676837 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fec980fe510 msgr2=0x7fec980f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.355+0000 7fec9ee7a700 1 --2- 172.21.15.47:0/1275676837 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fec980fe510 0x7fec980f7d40 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 rx=0x7fec84004870 tx=0x7fec84009cb0).stop 2024-02-23T00:46:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.356+0000 7fec9ee7a700 1 -- 172.21.15.47:0/1275676837 shutdown_connections 2024-02-23T00:46:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.356+0000 7fec9ee7a700 1 --2- 172.21.15.47:0/1275676837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec980f8420 0x7fec98104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.356+0000 7fec9ee7a700 1 --2- 172.21.15.47:0/1275676837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec980fdbb0 0x7fec980fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.356+0000 7fec9ee7a700 1 --2- 172.21.15.47:0/1275676837 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fec980fe510 0x7fec980f7d40 unknown :-1 s=CLOSED pgs=2600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.356+0000 7fec9ee7a700 1 -- 172.21.15.47:0/1275676837 >> 172.21.15.47:0/1275676837 conn(0x7fec980f3800 msgr2=0x7fec980f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:46:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.356+0000 7fec9ee7a700 1 -- 172.21.15.47:0/1275676837 shutdown_connections 2024-02-23T00:46:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.356+0000 7fec9ee7a700 1 -- 172.21.15.47:0/1275676837 wait complete. 2024-02-23T00:46:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.356+0000 7fec9ee7a700 1 librados: starting msgr at 2024-02-23T00:46:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.356+0000 7fec9ee7a700 1 librados: starting objecter 2024-02-23T00:46:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.356+0000 7fec9ee7a700 1 Processor -- start 2024-02-23T00:46:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.356+0000 7fec9ee7a700 1 -- start start 2024-02-23T00:46:04.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.356+0000 7fec9ee7a700 1 librados: setting wanted keys 2024-02-23T00:46:04.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.356+0000 7fec9ee7a700 1 librados: calling monclient init 2024-02-23T00:46:04.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.357+0000 7fec9ee7a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec980f8420 0x7fec9806b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:04.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.357+0000 7fec9ee7a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec980fdbb0 0x7fec9806b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:04.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.357+0000 7fec9ee7a700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fec980fe510 0x7fec9806bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:04.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.357+0000 7fec9ee7a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec98116370 con 0x7fec980f8420 2024-02-23T00:46:04.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.357+0000 7fec9ee7a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec981164e0 con 0x7fec980fdbb0 2024-02-23T00:46:04.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.357+0000 7fec9ee7a700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec981166a0 con 0x7fec980fe510 2024-02-23T00:46:04.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.357+0000 7fec9d417700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fec980fe510 0x7fec9806bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:04.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.357+0000 7fec9d417700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fec980fe510 0x7fec9806bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:56374/0 (socket says 172.21.15.47:56374) 2024-02-23T00:46:04.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.357+0000 7fec97fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec980fdbb0 0x7fec9806b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:04.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.357+0000 7fec9d417700 1 -- 172.21.15.47:0/1942385975 learned_addr learned my addr 172.21.15.47:0/1942385975 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:46:04.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.357+0000 7fec9cc16700 1 --2- 172.21.15.47:0/1942385975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec980f8420 0x7fec9806b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:04.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.357+0000 7fec97fff700 1 -- 172.21.15.47:0/1942385975 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fec980fe510 msgr2=0x7fec9806bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:04.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.357+0000 7fec97fff700 1 --2- 172.21.15.47:0/1942385975 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fec980fe510 0x7fec9806bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:04.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.357+0000 7fec97fff700 1 -- 172.21.15.47:0/1942385975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec980f8420 msgr2=0x7fec9806b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:04.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.357+0000 7fec97fff700 1 --2- 172.21.15.47:0/1942385975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec980f8420 0x7fec9806b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:04.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.357+0000 7fec97fff700 1 -- 172.21.15.47:0/1942385975 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec84005040 con 0x7fec980fdbb0 2024-02-23T00:46:04.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.357+0000 7fec9d417700 1 --2- 172.21.15.47:0/1942385975 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fec980fe510 0x7fec9806bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:46:04.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.357+0000 7fec97fff700 1 --2- 172.21.15.47:0/1942385975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec980fdbb0 0x7fec9806b5c0 secure :-1 s=READY pgs=3968 cs=0 l=1 rev1=1 rx=0x7fec84004710 tx=0x7fec84009cb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:04.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.357+0000 7fec95ffb700 10 client.?.objecter ms_handle_connect 0x7fec980fdbb0 2024-02-23T00:46:04.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.357+0000 7fec95ffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:46:04.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.358+0000 7fec95ffb700 1 -- 172.21.15.47:0/1942385975 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8400e030 con 0x7fec980fdbb0 2024-02-23T00:46:04.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.358+0000 7fec95ffb700 1 -- 172.21.15.47:0/1942385975 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fec84013db0 con 0x7fec980fdbb0 2024-02-23T00:46:04.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.358+0000 7fec9ee7a700 1 -- 172.21.15.47:0/1942385975 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec98116920 con 0x7fec980fdbb0 2024-02-23T00:46:04.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.358+0000 7fec95ffb700 1 -- 172.21.15.47:0/1942385975 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec84018070 con 0x7fec980fdbb0 2024-02-23T00:46:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.358+0000 7fec9ee7a700 10 client.43084.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:46:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.358+0000 7fec9ee7a700 1 -- 172.21.15.47:0/1942385975 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec98116d50 con 0x7fec980fdbb0 2024-02-23T00:46:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.358+0000 7fec9ee7a700 1 librados: init done 2024-02-23T00:46:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.359+0000 7fec9ee7a700 1 -- 172.21.15.47:0/1942385975 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec98106f20 con 0x7fec980fdbb0 2024-02-23T00:46:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.360+0000 7fec95ffb700 1 -- 172.21.15.47:0/1942385975 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fec84009240 con 0x7fec980fdbb0 2024-02-23T00:46:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.361+0000 7fec95ffb700 1 --2- 172.21.15.47:0/1942385975 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fec88062cc0 0x7fec88065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.361+0000 7fec95ffb700 1 -- 172.21.15.47:0/1942385975 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(291..291 src has 48..291) v4 ==== 9595+0+0 (secure 0 0 0) 0x7fec8407ead0 con 0x7fec980fdbb0 2024-02-23T00:46:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.361+0000 7fec95ffb700 10 client.43084.objecter ms_dispatch 0x7fec98001550 osd_map(291..291 src has 48..291) v4 2024-02-23T00:46:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.361+0000 7fec95ffb700 3 client.43084.objecter handle_osd_map got epochs [291,291] > 0 2024-02-23T00:46:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.361+0000 7fec95ffb700 3 client.43084.objecter handle_osd_map decoding full epoch 291 2024-02-23T00:46:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.361+0000 7fec9cc16700 1 --2- 172.21.15.47:0/1942385975 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fec88062cc0 0x7fec88065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.361+0000 7fec95ffb700 20 client.43084.objecter dump_active .. 0 homeless 2024-02-23T00:46:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.361+0000 7fec95ffb700 10 client.43084.objecter ms_handle_connect 0x7fec88062cc0 2024-02-23T00:46:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.361+0000 7fec9cc16700 1 --2- 172.21.15.47:0/1942385975 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fec88062cc0 0x7fec88065180 secure :-1 s=READY pgs=3010 cs=0 l=1 rev1=1 rx=0x7fec8c004dc0 tx=0x7fec8c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:04.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.365+0000 7fec95ffb700 1 -- 172.21.15.47:0/1942385975 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fec8404def0 con 0x7fec980fdbb0 2024-02-23T00:46:04.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.557+0000 7fec9ee7a700 1 -- 172.21.15.47:0/1942385975 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fec9805fa60 con 0x7fec88062cc0 2024-02-23T00:46:04.567 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:46:04.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.576+0000 7fec95ffb700 1 -- 172.21.15.47:0/1942385975 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+709231 (secure 0 0 0) 0x7fec9805fa60 con 0x7fec88062cc0 2024-02-23T00:46:04.587 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:46:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.580+0000 7fec9ee7a700 10 librados: watch_flush enter 2024-02-23T00:46:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.580+0000 7fec9ee7a700 10 librados: watch_flush exit 2024-02-23T00:46:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.580+0000 7fec9ee7a700 20 client.43084.objecter shutdown clearing up homeless session... 2024-02-23T00:46:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.580+0000 7fec9ee7a700 10 client.43084.objecter successfully canceled tick 2024-02-23T00:46:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.580+0000 7fec9ee7a700 1 -- 172.21.15.47:0/1942385975 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fec88062cc0 msgr2=0x7fec88065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.580+0000 7fec9ee7a700 1 --2- 172.21.15.47:0/1942385975 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fec88062cc0 0x7fec88065180 secure :-1 s=READY pgs=3010 cs=0 l=1 rev1=1 rx=0x7fec8c004dc0 tx=0x7fec8c00b040).stop 2024-02-23T00:46:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.580+0000 7fec9ee7a700 1 -- 172.21.15.47:0/1942385975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec980fdbb0 msgr2=0x7fec9806b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.580+0000 7fec9ee7a700 1 --2- 172.21.15.47:0/1942385975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec980fdbb0 0x7fec9806b5c0 secure :-1 s=READY pgs=3968 cs=0 l=1 rev1=1 rx=0x7fec84004710 tx=0x7fec84009cb0).stop 2024-02-23T00:46:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.580+0000 7fec9ee7a700 1 -- 172.21.15.47:0/1942385975 shutdown_connections 2024-02-23T00:46:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.580+0000 7fec9ee7a700 1 --2- 172.21.15.47:0/1942385975 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fec88062cc0 0x7fec88065180 unknown :-1 s=CLOSED pgs=3010 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.580+0000 7fec9ee7a700 1 --2- 172.21.15.47:0/1942385975 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fec980fe510 0x7fec9806bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.580+0000 7fec9ee7a700 1 --2- 172.21.15.47:0/1942385975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec980f8420 0x7fec9806b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:04.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.580+0000 7fec9ee7a700 1 --2- 172.21.15.47:0/1942385975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec980fdbb0 0x7fec9806b5c0 unknown :-1 s=CLOSED pgs=3968 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:04.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.580+0000 7fec9ee7a700 1 -- 172.21.15.47:0/1942385975 >> 172.21.15.47:0/1942385975 conn(0x7fec980f3800 msgr2=0x7fec980f5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:46:04.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.580+0000 7fec9ee7a700 1 -- 172.21.15.47:0/1942385975 shutdown_connections 2024-02-23T00:46:04.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.580+0000 7fec9ee7a700 1 -- 172.21.15.47:0/1942385975 wait complete. 2024-02-23T00:46:04.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:04.581+0000 7fec9ee7a700 1 librados: shutdown 2024-02-23T00:46:04.596 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-02-23T00:46:04.596 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:04.611 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:46:04.613 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:04.614 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:46:04.715 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:46:04.741 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:04.760 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:46:04.762 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:04.763 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:46:04.864 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:46:04.892 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:04.943 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set noscrub 2024-02-23T00:46:05.091 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:46:05.094 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:05.094 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:46:05.195 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:46:05.220 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:05.233 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:03.250427+0000 mgr.y (mgr.14517) 296 : cluster [DBG] pgmap v412: 214 pgs: 1 active+clean+snaptrim, 1 peering, 212 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 3 op/s; 31 B/s, 0 objects/s recovering 2024-02-23T00:46:05.233 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:03.792126+0000 osd.7 (osd.7) 151 : cluster [DBG] 62.14 scrub starts 2024-02-23T00:46:05.233 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:03.795601+0000 osd.7 (osd.7) 152 : cluster [DBG] 62.14 scrub ok 2024-02-23T00:46:05.233 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:03.924074+0000 osd.3 (osd.3) 151 : cluster [DBG] 62.18 scrub starts 2024-02-23T00:46:05.233 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:03.927773+0000 osd.3 (osd.3) 152 : cluster [DBG] 62.18 scrub ok 2024-02-23T00:46:05.233 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:03.949787+0000 osd.2 (osd.2) 131 : cluster [DBG] 4.1c scrub starts 2024-02-23T00:46:05.233 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:03.951520+0000 osd.2 (osd.2) 132 : cluster [DBG] 4.1c scrub ok 2024-02-23T00:46:05.233 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:04.043020+0000 mon.a (mon.0) 1966 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "27"}]: dispatch 2024-02-23T00:46:05.233 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:04.502851+0000 mon.a (mon.0) 1967 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi047-82378-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi047-82378-40"}]': finished 2024-02-23T00:46:05.234 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:04.502983+0000 mon.a (mon.0) 1968 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "27"}]': finished 2024-02-23T00:46:05.234 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:04.503081+0000 mon.a (mon.0) 1969 : cluster [DBG] osdmap e292: 8 total, 8 up, 8 in 2024-02-23T00:46:05.234 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:04.506072+0000 mon.a (mon.0) 1970 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:46:05.234 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:04.512802+0000 mon.c (mon.2) 218 : audit [INF] from='client.? 172.21.15.47:0/3569018718' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi047-82386-50","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:46:05.234 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:04.513285+0000 mon.a (mon.0) 1971 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi047-82386-50","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:46:05.234 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:04.558646+0000 mgr.y (mgr.14517) 297 : audit [DBG] from='client.43084 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:46:05.234 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:03.250427+0000 mgr.y (mgr.14517) 296 : cluster [DBG] pgmap v412: 214 pgs: 1 active+clean+snaptrim, 1 peering, 212 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 3 op/s; 31 B/s, 0 objects/s recovering 2024-02-23T00:46:05.234 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:03.792126+0000 osd.7 (osd.7) 151 : cluster [DBG] 62.14 scrub starts 2024-02-23T00:46:05.234 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:03.795601+0000 osd.7 (osd.7) 152 : cluster [DBG] 62.14 scrub ok 2024-02-23T00:46:05.234 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:03.924074+0000 osd.3 (osd.3) 151 : cluster [DBG] 62.18 scrub starts 2024-02-23T00:46:05.234 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:03.927773+0000 osd.3 (osd.3) 152 : cluster [DBG] 62.18 scrub ok 2024-02-23T00:46:05.235 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:03.949787+0000 osd.2 (osd.2) 131 : cluster [DBG] 4.1c scrub starts 2024-02-23T00:46:05.235 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:03.951520+0000 osd.2 (osd.2) 132 : cluster [DBG] 4.1c scrub ok 2024-02-23T00:46:05.235 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:04.043020+0000 mon.a (mon.0) 1966 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "27"}]: dispatch 2024-02-23T00:46:05.235 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:04.502851+0000 mon.a (mon.0) 1967 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi047-82378-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi047-82378-40"}]': finished 2024-02-23T00:46:05.235 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:04.502983+0000 mon.a (mon.0) 1968 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "27"}]': finished 2024-02-23T00:46:05.235 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:04.503081+0000 mon.a (mon.0) 1969 : cluster [DBG] osdmap e292: 8 total, 8 up, 8 in 2024-02-23T00:46:05.235 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:04.506072+0000 mon.a (mon.0) 1970 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:46:05.235 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:04.512802+0000 mon.c (mon.2) 218 : audit [INF] from='client.? 172.21.15.47:0/3569018718' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi047-82386-50","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:46:05.235 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:04.513285+0000 mon.a (mon.0) 1971 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi047-82386-50","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:46:05.235 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:04.558646+0000 mgr.y (mgr.14517) 297 : audit [DBG] from='client.43084 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:46:05.239 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:46:05.241 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:05.242 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:46:05.342 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:46:05.370 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:05.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:03.250427+0000 mgr.y (mgr.14517) 296 : cluster [DBG] pgmap v412: 214 pgs: 1 active+clean+snaptrim, 1 peering, 212 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 3 op/s; 31 B/s, 0 objects/s recovering 2024-02-23T00:46:05.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:03.792126+0000 osd.7 (osd.7) 151 : cluster [DBG] 62.14 scrub starts 2024-02-23T00:46:05.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:03.795601+0000 osd.7 (osd.7) 152 : cluster [DBG] 62.14 scrub ok 2024-02-23T00:46:05.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:03.924074+0000 osd.3 (osd.3) 151 : cluster [DBG] 62.18 scrub starts 2024-02-23T00:46:05.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:03.927773+0000 osd.3 (osd.3) 152 : cluster [DBG] 62.18 scrub ok 2024-02-23T00:46:05.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:03.949787+0000 osd.2 (osd.2) 131 : cluster [DBG] 4.1c scrub starts 2024-02-23T00:46:05.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:03.951520+0000 osd.2 (osd.2) 132 : cluster [DBG] 4.1c scrub ok 2024-02-23T00:46:05.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:04.043020+0000 mon.a (mon.0) 1966 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "27"}]: dispatch 2024-02-23T00:46:05.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:04.502851+0000 mon.a (mon.0) 1967 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi047-82378-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi047-82378-40"}]': finished 2024-02-23T00:46:05.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:04.502983+0000 mon.a (mon.0) 1968 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "27"}]': finished 2024-02-23T00:46:05.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:04.503081+0000 mon.a (mon.0) 1969 : cluster [DBG] osdmap e292: 8 total, 8 up, 8 in 2024-02-23T00:46:05.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:04.506072+0000 mon.a (mon.0) 1970 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:46:05.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:04.512802+0000 mon.c (mon.2) 218 : audit [INF] from='client.? 172.21.15.47:0/3569018718' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi047-82386-50","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:46:05.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:04.513285+0000 mon.a (mon.0) 1971 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi047-82386-50","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:46:05.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:04.558646+0000 mgr.y (mgr.14517) 297 : audit [DBG] from='client.43084 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:46:05.692 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2024-02-23T00:46:04.500005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":9053,"num_objects":362,"num_object_clones":2,"num_object_copies":1086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":362,"num_whiteouts":0,"num_read":1605,"num_read_kb":1231,"num_write":1269,"num_write_kb":643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":2249,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2609,"ondisk_log_size":2609,"up":642,"acting":642,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":760,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2493928,"kb_used_data":38264,"kb_used_omap":0,"kb_used_meta":2455552,"kb_avail":747467288,"statfs":{"total":767960285184,"available":765406502912,"internally_reserved":0,"allocated":39182336,"data_stored":23407651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2514485248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":127,"num_objects":0,"num_object_clones":-1,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":-1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.346251"},"pg_stats":[{"pgid":"181.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"180.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"181.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"180.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"181.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"180.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"181.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"180.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"181.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"180.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"181.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"180.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"181.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"180.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"181.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"180.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"181.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"180.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"181.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"180.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"181.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"180.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"181.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"180.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"181.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"180.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"181.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"180.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"181.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"180.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"181.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"180.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"181.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"180.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"181.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"180.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"181.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"180.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"181.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"180.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"181.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"180.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"181.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"180.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"181.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"180.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"181.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"180.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"181.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"180.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"179.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"181.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"180.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"179.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"181.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"180.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"179.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"181.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"180.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"179.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"181.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"180.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"179.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"181.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"180.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"179.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"181.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"180.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"179.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"181.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"180.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"179.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:04.493464+0000","last_change":"2024-02-23T00:46:04.493464+0000","last_active":"2024-02-23T00:46:04.493464+0000","last_peered":"2024-02-23T00:46:04.493464+0000","last_clean":"2024-02-23T00:46:04.493464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:04.493464+0000","last_undegraded":"2024-02-23T00:46:04.493464+0000","last_fullsized":"2024-02-23T00:46:04.493464+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:04.493464+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.493464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.0","version":"259'40","reported_seq":252,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.172234+0000","last_change":"2024-02-23T00:46:00.172234+0000","last_active":"2024-02-23T00:46:00.172234+0000","last_peered":"2024-02-23T00:46:00.172234+0000","last_clean":"2024-02-23T00:46:00.172234+0000","last_became_active":"2024-02-23T00:43:17.855316+0000","last_became_peered":"2024-02-23T00:43:17.855316+0000","last_unstale":"2024-02-23T00:46:00.172234+0000","last_undegraded":"2024-02-23T00:46:00.172234+0000","last_fullsized":"2024-02-23T00:46:00.172234+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":31,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":1,"num_write":13,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.1","version":"230'87","reported_seq":292,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.159399+0000","last_change":"2024-02-23T00:46:00.167004+0000","last_active":"2024-02-23T00:46:01.159399+0000","last_peered":"2024-02-23T00:46:01.159399+0000","last_clean":"2024-02-23T00:46:01.159399+0000","last_became_active":"2024-02-23T00:43:17.851866+0000","last_became_peered":"2024-02-23T00:43:17.851866+0000","last_unstale":"2024-02-23T00:46:01.159399+0000","last_undegraded":"2024-02-23T00:46:01.159399+0000","last_fullsized":"2024-02-23T00:46:01.159399+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":87,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":63,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.2","version":"229'68","reported_seq":277,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.206335+0000","last_change":"2024-02-23T00:46:00.168915+0000","last_active":"2024-02-23T00:46:01.206335+0000","last_peered":"2024-02-23T00:46:01.206335+0000","last_clean":"2024-02-23T00:46:01.206335+0000","last_became_active":"2024-02-23T00:43:17.804792+0000","last_became_peered":"2024-02-23T00:43:17.804792+0000","last_unstale":"2024-02-23T00:46:01.206335+0000","last_undegraded":"2024-02-23T00:46:01.206335+0000","last_fullsized":"2024-02-23T00:46:01.206335+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'68","last_scrub_stamp":"2024-02-23T00:45:57.135726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:57.135726+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.3","version":"246'20","reported_seq":240,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.298967+0000","last_change":"2024-02-23T00:46:00.172490+0000","last_active":"2024-02-23T00:46:01.298967+0000","last_peered":"2024-02-23T00:46:01.298967+0000","last_clean":"2024-02-23T00:46:01.298967+0000","last_became_active":"2024-02-23T00:43:17.834686+0000","last_became_peered":"2024-02-23T00:43:17.834686+0000","last_unstale":"2024-02-23T00:46:01.298967+0000","last_undegraded":"2024-02-23T00:46:01.298967+0000","last_fullsized":"2024-02-23T00:46:01.298967+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"246'20","last_scrub_stamp":"2024-02-23T00:45:53.067737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:53.067737+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.4","version":"207'17","reported_seq":228,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.301838+0000","last_change":"2024-02-23T00:46:00.183807+0000","last_active":"2024-02-23T00:46:01.301838+0000","last_peered":"2024-02-23T00:46:01.301838+0000","last_clean":"2024-02-23T00:46:01.301838+0000","last_became_active":"2024-02-23T00:43:17.810343+0000","last_became_peered":"2024-02-23T00:43:17.810343+0000","last_unstale":"2024-02-23T00:46:01.301838+0000","last_undegraded":"2024-02-23T00:46:01.301838+0000","last_fullsized":"2024-02-23T00:46:01.301838+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.5","version":"237'51","reported_seq":262,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.158792+0000","last_change":"2024-02-23T00:46:00.160547+0000","last_active":"2024-02-23T00:46:01.158792+0000","last_peered":"2024-02-23T00:46:01.158792+0000","last_clean":"2024-02-23T00:46:01.158792+0000","last_became_active":"2024-02-23T00:43:17.855339+0000","last_became_peered":"2024-02-23T00:43:17.855339+0000","last_unstale":"2024-02-23T00:46:01.158792+0000","last_undegraded":"2024-02-23T00:46:01.158792+0000","last_fullsized":"2024-02-23T00:46:01.158792+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.6","version":"247'51","reported_seq":267,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.726138+0000","last_change":"2024-02-23T00:46:00.726103+0000","last_active":"2024-02-23T00:46:00.726138+0000","last_peered":"2024-02-23T00:46:00.726138+0000","last_clean":"2024-02-23T00:46:00.726138+0000","last_became_active":"2024-02-23T00:43:17.838838+0000","last_became_peered":"2024-02-23T00:43:17.838838+0000","last_unstale":"2024-02-23T00:46:00.726138+0000","last_undegraded":"2024-02-23T00:46:00.726138+0000","last_fullsized":"2024-02-23T00:46:00.726138+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'51","last_scrub_stamp":"2024-02-23T00:46:00.726012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:00.726012+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.7","version":"215'68","reported_seq":270,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.301279+0000","last_change":"2024-02-23T00:46:00.175978+0000","last_active":"2024-02-23T00:46:01.301279+0000","last_peered":"2024-02-23T00:46:01.301279+0000","last_clean":"2024-02-23T00:46:01.301279+0000","last_became_active":"2024-02-23T00:43:17.846548+0000","last_became_peered":"2024-02-23T00:43:17.846548+0000","last_unstale":"2024-02-23T00:46:01.301279+0000","last_undegraded":"2024-02-23T00:46:01.301279+0000","last_fullsized":"2024-02-23T00:46:01.301279+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'68","last_scrub_stamp":"2024-02-23T00:45:25.351414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.351414+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.8","version":"259'90","reported_seq":288,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.165597+0000","last_change":"2024-02-23T00:46:00.165597+0000","last_active":"2024-02-23T00:46:00.165597+0000","last_peered":"2024-02-23T00:46:00.165597+0000","last_clean":"2024-02-23T00:46:00.165597+0000","last_became_active":"2024-02-23T00:43:17.842613+0000","last_became_peered":"2024-02-23T00:43:17.842613+0000","last_unstale":"2024-02-23T00:46:00.165597+0000","last_undegraded":"2024-02-23T00:46:00.165597+0000","last_fullsized":"2024-02-23T00:46:00.165597+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":90,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.9","version":"259'39","reported_seq":252,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.302651+0000","last_change":"2024-02-23T00:46:01.302651+0000","last_active":"2024-02-23T00:46:01.302651+0000","last_peered":"2024-02-23T00:46:01.302651+0000","last_clean":"2024-02-23T00:46:01.302651+0000","last_became_active":"2024-02-23T00:43:17.834297+0000","last_became_peered":"2024-02-23T00:43:17.834297+0000","last_unstale":"2024-02-23T00:46:01.302651+0000","last_undegraded":"2024-02-23T00:46:01.302651+0000","last_fullsized":"2024-02-23T00:46:01.302651+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":12,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.a","version":"253'90","reported_seq":294,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.166527+0000","last_change":"2024-02-23T00:46:00.166527+0000","last_active":"2024-02-23T00:46:00.166527+0000","last_peered":"2024-02-23T00:46:00.166527+0000","last_clean":"2024-02-23T00:46:00.166527+0000","last_became_active":"2024-02-23T00:43:17.839142+0000","last_became_peered":"2024-02-23T00:43:17.839142+0000","last_unstale":"2024-02-23T00:46:00.166527+0000","last_undegraded":"2024-02-23T00:46:00.166527+0000","last_fullsized":"2024-02-23T00:46:00.166527+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"253'90","last_scrub_stamp":"2024-02-23T00:45:59.681669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:59.681669+0000","log_size":90,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.b","version":"288'76","reported_seq":274,"reported_epoch":288,"state":"active+clean+snaptrim","last_fresh":"2024-02-23T00:46:00.182335+0000","last_change":"2024-02-23T00:46:00.166963+0000","last_active":"2024-02-23T00:46:00.182335+0000","last_peered":"2024-02-23T00:46:00.182335+0000","last_clean":"2024-02-23T00:46:00.182335+0000","last_became_active":"2024-02-23T00:43:17.855321+0000","last_became_peered":"2024-02-23T00:43:17.855321+0000","last_unstale":"2024-02-23T00:46:00.182335+0000","last_undegraded":"2024-02-23T00:46:00.182335+0000","last_fullsized":"2024-02-23T00:46:00.182335+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":76,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":46,"num_objects":6,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":23,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.c","version":"203'34","reported_seq":249,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.161977+0000","last_change":"2024-02-23T00:46:00.163652+0000","last_active":"2024-02-23T00:46:01.161977+0000","last_peered":"2024-02-23T00:46:01.161977+0000","last_clean":"2024-02-23T00:46:01.161977+0000","last_became_active":"2024-02-23T00:43:17.829044+0000","last_became_peered":"2024-02-23T00:43:17.829044+0000","last_unstale":"2024-02-23T00:46:01.161977+0000","last_undegraded":"2024-02-23T00:46:01.161977+0000","last_fullsized":"2024-02-23T00:46:01.161977+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'34","last_scrub_stamp":"2024-02-23T00:45:57.874883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:57.874883+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.d","version":"259'56","reported_seq":264,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.158593+0000","last_change":"2024-02-23T00:46:00.159535+0000","last_active":"2024-02-23T00:46:01.158593+0000","last_peered":"2024-02-23T00:46:01.158593+0000","last_clean":"2024-02-23T00:46:01.158593+0000","last_became_active":"2024-02-23T00:43:17.841614+0000","last_became_peered":"2024-02-23T00:43:17.841614+0000","last_unstale":"2024-02-23T00:46:01.158593+0000","last_undegraded":"2024-02-23T00:46:01.158593+0000","last_fullsized":"2024-02-23T00:46:01.158593+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'56","last_scrub_stamp":"2024-02-23T00:45:51.258745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:51.258745+0000","log_size":56,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":17,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.e","version":"211'51","reported_seq":255,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.301306+0000","last_change":"2024-02-23T00:46:00.186524+0000","last_active":"2024-02-23T00:46:01.301306+0000","last_peered":"2024-02-23T00:46:01.301306+0000","last_clean":"2024-02-23T00:46:01.301306+0000","last_became_active":"2024-02-23T00:43:17.845956+0000","last_became_peered":"2024-02-23T00:43:17.845956+0000","last_unstale":"2024-02-23T00:46:01.301306+0000","last_undegraded":"2024-02-23T00:46:01.301306+0000","last_fullsized":"2024-02-23T00:46:01.301306+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.f","version":"241'136","reported_seq":326,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.158914+0000","last_change":"2024-02-23T00:46:00.160973+0000","last_active":"2024-02-23T00:46:01.158914+0000","last_peered":"2024-02-23T00:46:01.158914+0000","last_clean":"2024-02-23T00:46:01.158914+0000","last_became_active":"2024-02-23T00:43:17.855331+0000","last_became_peered":"2024-02-23T00:43:17.855331+0000","last_unstale":"2024-02-23T00:46:01.158914+0000","last_undegraded":"2024-02-23T00:46:01.158914+0000","last_fullsized":"2024-02-23T00:46:01.158914+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":136,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":88,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":40,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.10","version":"219'68","reported_seq":273,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.157651+0000","last_change":"2024-02-23T00:46:00.167847+0000","last_active":"2024-02-23T00:46:01.157651+0000","last_peered":"2024-02-23T00:46:01.157651+0000","last_clean":"2024-02-23T00:46:01.157651+0000","last_became_active":"2024-02-23T00:43:17.819865+0000","last_became_peered":"2024-02-23T00:43:17.819865+0000","last_unstale":"2024-02-23T00:46:01.157651+0000","last_undegraded":"2024-02-23T00:46:01.157651+0000","last_fullsized":"2024-02-23T00:46:01.157651+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.11","version":"215'51","reported_seq":253,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.159389+0000","last_change":"2024-02-23T00:46:00.164366+0000","last_active":"2024-02-23T00:46:01.159389+0000","last_peered":"2024-02-23T00:46:01.159389+0000","last_clean":"2024-02-23T00:46:01.159389+0000","last_became_active":"2024-02-23T00:43:17.852697+0000","last_became_peered":"2024-02-23T00:43:17.852697+0000","last_unstale":"2024-02-23T00:46:01.159389+0000","last_undegraded":"2024-02-23T00:46:01.159389+0000","last_fullsized":"2024-02-23T00:46:01.159389+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.12","version":"240'73","reported_seq":284,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.300320+0000","last_change":"2024-02-23T00:46:00.176356+0000","last_active":"2024-02-23T00:46:01.300320+0000","last_peered":"2024-02-23T00:46:01.300320+0000","last_clean":"2024-02-23T00:46:01.300320+0000","last_became_active":"2024-02-23T00:43:17.856118+0000","last_became_peered":"2024-02-23T00:43:17.856118+0000","last_unstale":"2024-02-23T00:46:01.300320+0000","last_undegraded":"2024-02-23T00:46:01.300320+0000","last_fullsized":"2024-02-23T00:46:01.300320+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'73","last_scrub_stamp":"2024-02-23T00:45:52.066508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:52.066508+0000","log_size":73,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":49,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.13","version":"205'34","reported_seq":245,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.299342+0000","last_change":"2024-02-23T00:46:00.182497+0000","last_active":"2024-02-23T00:46:01.299342+0000","last_peered":"2024-02-23T00:46:01.299342+0000","last_clean":"2024-02-23T00:46:01.299342+0000","last_became_active":"2024-02-23T00:43:17.857823+0000","last_became_peered":"2024-02-23T00:43:17.857823+0000","last_unstale":"2024-02-23T00:46:01.299342+0000","last_undegraded":"2024-02-23T00:46:01.299342+0000","last_fullsized":"2024-02-23T00:46:01.299342+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.14","version":"239'39","reported_seq":245,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.167602+0000","last_change":"2024-02-23T00:46:00.167602+0000","last_active":"2024-02-23T00:46:00.167602+0000","last_peered":"2024-02-23T00:46:00.167602+0000","last_clean":"2024-02-23T00:46:00.167602+0000","last_became_active":"2024-02-23T00:43:17.816138+0000","last_became_peered":"2024-02-23T00:43:17.816138+0000","last_unstale":"2024-02-23T00:46:00.167602+0000","last_undegraded":"2024-02-23T00:46:00.167602+0000","last_fullsized":"2024-02-23T00:46:00.167602+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":12,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.15","version":"201'17","reported_seq":231,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.158531+0000","last_change":"2024-02-23T00:46:00.163852+0000","last_active":"2024-02-23T00:46:01.158531+0000","last_peered":"2024-02-23T00:46:01.158531+0000","last_clean":"2024-02-23T00:46:01.158531+0000","last_became_active":"2024-02-23T00:43:17.843834+0000","last_became_peered":"2024-02-23T00:43:17.843834+0000","last_unstale":"2024-02-23T00:46:01.158531+0000","last_undegraded":"2024-02-23T00:46:01.158531+0000","last_fullsized":"2024-02-23T00:46:01.158531+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.16","version":"240'23","reported_seq":242,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.159558+0000","last_change":"2024-02-23T00:46:00.160705+0000","last_active":"2024-02-23T00:46:01.159558+0000","last_peered":"2024-02-23T00:46:01.159558+0000","last_clean":"2024-02-23T00:46:01.159558+0000","last_became_active":"2024-02-23T00:43:17.834291+0000","last_became_peered":"2024-02-23T00:43:17.834291+0000","last_unstale":"2024-02-23T00:46:01.159558+0000","last_undegraded":"2024-02-23T00:46:01.159558+0000","last_fullsized":"2024-02-23T00:46:01.159558+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'23","last_scrub_stamp":"2024-02-23T00:45:56.354881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:56.354881+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":14,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.17","version":"229'57","reported_seq":275,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.207759+0000","last_change":"2024-02-23T00:46:00.166482+0000","last_active":"2024-02-23T00:46:01.207759+0000","last_peered":"2024-02-23T00:46:01.207759+0000","last_clean":"2024-02-23T00:46:01.207759+0000","last_became_active":"2024-02-23T00:43:17.843525+0000","last_became_peered":"2024-02-23T00:43:17.843525+0000","last_unstale":"2024-02-23T00:46:01.207759+0000","last_undegraded":"2024-02-23T00:46:01.207759+0000","last_fullsized":"2024-02-23T00:46:01.207759+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'57","last_scrub_stamp":"2024-02-23T00:45:58.127302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:58.127302+0000","log_size":57,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":18,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.19","version":"267'96","reported_seq":309,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.205774+0000","last_change":"2024-02-23T00:46:00.163245+0000","last_active":"2024-02-23T00:46:01.205774+0000","last_peered":"2024-02-23T00:46:01.205774+0000","last_clean":"2024-02-23T00:46:01.205774+0000","last_became_active":"2024-02-23T00:43:17.815830+0000","last_became_peered":"2024-02-23T00:43:17.815830+0000","last_unstale":"2024-02-23T00:46:01.205774+0000","last_undegraded":"2024-02-23T00:46:01.205774+0000","last_fullsized":"2024-02-23T00:46:01.205774+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'96","last_scrub_stamp":"2024-02-23T00:45:55.109551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:55.109551+0000","log_size":96,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":73,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":7,"num_read_kb":0,"num_write":30,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.18","version":"275'7","reported_seq":226,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.159915+0000","last_change":"2024-02-23T00:46:00.160019+0000","last_active":"2024-02-23T00:46:01.159915+0000","last_peered":"2024-02-23T00:46:01.159915+0000","last_clean":"2024-02-23T00:46:01.159915+0000","last_became_active":"2024-02-23T00:43:17.840351+0000","last_became_peered":"2024-02-23T00:43:17.840351+0000","last_unstale":"2024-02-23T00:46:01.159915+0000","last_undegraded":"2024-02-23T00:46:01.159915+0000","last_fullsized":"2024-02-23T00:46:01.159915+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.1b","version":"229'68","reported_seq":272,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.158606+0000","last_change":"2024-02-23T00:46:00.158327+0000","last_active":"2024-02-23T00:46:01.158606+0000","last_peered":"2024-02-23T00:46:01.158606+0000","last_clean":"2024-02-23T00:46:01.158606+0000","last_became_active":"2024-02-23T00:43:17.803043+0000","last_became_peered":"2024-02-23T00:43:17.803043+0000","last_unstale":"2024-02-23T00:46:01.158606+0000","last_undegraded":"2024-02-23T00:46:01.158606+0000","last_fullsized":"2024-02-23T00:46:01.158606+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.1a","version":"225'34","reported_seq":277,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:02.071976+0000","last_change":"2024-02-23T00:46:02.071916+0000","last_active":"2024-02-23T00:46:02.071976+0000","last_peered":"2024-02-23T00:46:02.071976+0000","last_clean":"2024-02-23T00:46:02.071976+0000","last_became_active":"2024-02-23T00:45:44.178195+0000","last_became_peered":"2024-02-23T00:45:44.178195+0000","last_unstale":"2024-02-23T00:46:02.071976+0000","last_undegraded":"2024-02-23T00:46:02.071976+0000","last_fullsized":"2024-02-23T00:46:02.071976+0000","mapping_epoch":271,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":272,"parent":"0.0","parent_split_bits":0,"last_scrub":"225'34","last_scrub_stamp":"2024-02-23T00:46:02.071770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:02.071770+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.1d","version":"215'104","reported_seq":305,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.174618+0000","last_change":"2024-02-23T00:46:00.174618+0000","last_active":"2024-02-23T00:46:00.174618+0000","last_peered":"2024-02-23T00:46:00.174618+0000","last_clean":"2024-02-23T00:46:00.174618+0000","last_became_active":"2024-02-23T00:43:17.828194+0000","last_became_peered":"2024-02-23T00:43:17.828194+0000","last_unstale":"2024-02-23T00:46:00.174618+0000","last_undegraded":"2024-02-23T00:46:00.174618+0000","last_fullsized":"2024-02-23T00:46:00.174618+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":104,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":74,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":32,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.1c","version":"223'51","reported_seq":255,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.206116+0000","last_change":"2024-02-23T00:46:00.168460+0000","last_active":"2024-02-23T00:46:01.206116+0000","last_peered":"2024-02-23T00:46:01.206116+0000","last_clean":"2024-02-23T00:46:01.206116+0000","last_became_active":"2024-02-23T00:43:17.804718+0000","last_became_peered":"2024-02-23T00:43:17.804718+0000","last_unstale":"2024-02-23T00:46:01.206116+0000","last_undegraded":"2024-02-23T00:46:01.206116+0000","last_fullsized":"2024-02-23T00:46:01.206116+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.1f","version":"240'40","reported_seq":257,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.301224+0000","last_change":"2024-02-23T00:46:00.174886+0000","last_active":"2024-02-23T00:46:01.301224+0000","last_peered":"2024-02-23T00:46:01.301224+0000","last_clean":"2024-02-23T00:46:01.301224+0000","last_became_active":"2024-02-23T00:43:17.839914+0000","last_became_peered":"2024-02-23T00:43:17.839914+0000","last_unstale":"2024-02-23T00:46:01.301224+0000","last_undegraded":"2024-02-23T00:46:01.301224+0000","last_fullsized":"2024-02-23T00:46:01.301224+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'40","last_scrub_stamp":"2024-02-23T00:45:58.074860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:58.074860+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":13,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.1e","version":"245'51","reported_seq":260,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.172469+0000","last_change":"2024-02-23T00:46:00.172469+0000","last_active":"2024-02-23T00:46:00.172469+0000","last_peered":"2024-02-23T00:46:00.172469+0000","last_clean":"2024-02-23T00:46:00.172469+0000","last_became_active":"2024-02-23T00:43:17.838682+0000","last_became_peered":"2024-02-23T00:43:17.838682+0000","last_unstale":"2024-02-23T00:46:00.172469+0000","last_undegraded":"2024-02-23T00:46:00.172469+0000","last_fullsized":"2024-02-23T00:46:00.172469+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"245'51","last_scrub_stamp":"2024-02-23T00:45:58.723990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:58.723990+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"5.9","version":"67'8","reported_seq":338,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.168987+0000","last_change":"2024-02-23T00:45:55.708656+0000","last_active":"2024-02-23T00:46:00.168987+0000","last_peered":"2024-02-23T00:46:00.168987+0000","last_clean":"2024-02-23T00:46:00.168987+0000","last_became_active":"2024-02-23T00:39:18.793414+0000","last_became_peered":"2024-02-23T00:39:18.793414+0000","last_unstale":"2024-02-23T00:46:00.168987+0000","last_undegraded":"2024-02-23T00:46:00.168987+0000","last_fullsized":"2024-02-23T00:46:00.168987+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:45:55.708576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:55.708576+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"68'27","reported_seq":306,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.167955+0000","last_change":"2024-02-23T00:45:04.138883+0000","last_active":"2024-02-23T00:46:00.167955+0000","last_peered":"2024-02-23T00:46:00.167955+0000","last_clean":"2024-02-23T00:46:00.167955+0000","last_became_active":"2024-02-23T00:39:16.859401+0000","last_became_peered":"2024-02-23T00:39:16.859401+0000","last_unstale":"2024-02-23T00:46:00.167955+0000","last_undegraded":"2024-02-23T00:46:00.167955+0000","last_fullsized":"2024-02-23T00:46:00.167955+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'27","last_scrub_stamp":"2024-02-23T00:45:04.138751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.138751+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":252,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.302711+0000","last_change":"2024-02-23T00:43:42.320439+0000","last_active":"2024-02-23T00:46:01.302711+0000","last_peered":"2024-02-23T00:46:01.302711+0000","last_clean":"2024-02-23T00:46:01.302711+0000","last_became_active":"2024-02-23T00:39:14.784436+0000","last_became_peered":"2024-02-23T00:39:14.784436+0000","last_unstale":"2024-02-23T00:46:01.302711+0000","last_undegraded":"2024-02-23T00:46:01.302711+0000","last_fullsized":"2024-02-23T00:46:01.302711+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:42.320346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:04.330426+0000","last_clean_scrub_stamp":"2024-02-23T00:43:42.320346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":265,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.172443+0000","last_change":"2024-02-23T00:44:21.972446+0000","last_active":"2024-02-23T00:46:00.172443+0000","last_peered":"2024-02-23T00:46:00.172443+0000","last_clean":"2024-02-23T00:46:00.172443+0000","last_became_active":"2024-02-23T00:43:32.282545+0000","last_became_peered":"2024-02-23T00:43:32.282545+0000","last_unstale":"2024-02-23T00:46:00.172443+0000","last_undegraded":"2024-02-23T00:46:00.172443+0000","last_fullsized":"2024-02-23T00:46:00.172443+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:21.972383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:21.972383+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.3","version":"110'5","reported_seq":200,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.158884+0000","last_change":"2024-02-23T00:45:17.356489+0000","last_active":"2024-02-23T00:46:01.158884+0000","last_peered":"2024-02-23T00:46:01.158884+0000","last_clean":"2024-02-23T00:46:01.158884+0000","last_became_active":"2024-02-23T00:42:58.835612+0000","last_became_peered":"2024-02-23T00:42:58.835612+0000","last_unstale":"2024-02-23T00:46:01.158884+0000","last_undegraded":"2024-02-23T00:46:01.158884+0000","last_fullsized":"2024-02-23T00:46:01.158884+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'5","last_scrub_stamp":"2024-02-23T00:45:17.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:17.356406+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":194,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.171444+0000","last_change":"2024-02-23T00:45:17.181400+0000","last_active":"2024-02-23T00:46:00.171444+0000","last_peered":"2024-02-23T00:46:00.171444+0000","last_clean":"2024-02-23T00:46:00.171444+0000","last_became_active":"2024-02-23T00:42:58.828846+0000","last_became_peered":"2024-02-23T00:42:58.828846+0000","last_unstale":"2024-02-23T00:46:00.171444+0000","last_undegraded":"2024-02-23T00:46:00.171444+0000","last_fullsized":"2024-02-23T00:46:00.171444+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:17.181335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:17.181335+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"68'15","reported_seq":283,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.299704+0000","last_change":"2024-02-23T00:43:54.355050+0000","last_active":"2024-02-23T00:46:01.299704+0000","last_peered":"2024-02-23T00:46:01.299704+0000","last_clean":"2024-02-23T00:46:01.299704+0000","last_became_active":"2024-02-23T00:39:16.783701+0000","last_became_peered":"2024-02-23T00:39:16.783701+0000","last_unstale":"2024-02-23T00:46:01.299704+0000","last_undegraded":"2024-02-23T00:46:01.299704+0000","last_fullsized":"2024-02-23T00:46:01.299704+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:43:54.354974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:54.354974+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":248,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.161218+0000","last_change":"2024-02-23T00:42:48.591600+0000","last_active":"2024-02-23T00:46:01.161218+0000","last_peered":"2024-02-23T00:46:01.161218+0000","last_clean":"2024-02-23T00:46:01.161218+0000","last_became_active":"2024-02-23T00:39:14.772493+0000","last_became_peered":"2024-02-23T00:39:14.772493+0000","last_unstale":"2024-02-23T00:46:01.161218+0000","last_undegraded":"2024-02-23T00:46:01.161218+0000","last_fullsized":"2024-02-23T00:46:01.161218+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:48.591486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:42:48.591486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":248,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.300555+0000","last_change":"2024-02-23T00:44:22.380954+0000","last_active":"2024-02-23T00:46:01.300555+0000","last_peered":"2024-02-23T00:46:01.300555+0000","last_clean":"2024-02-23T00:46:01.300555+0000","last_became_active":"2024-02-23T00:39:18.794476+0000","last_became_peered":"2024-02-23T00:39:18.794476+0000","last_unstale":"2024-02-23T00:46:01.300555+0000","last_undegraded":"2024-02-23T00:46:01.300555+0000","last_fullsized":"2024-02-23T00:46:01.300555+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:22.380841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:08.335330+0000","last_clean_scrub_stamp":"2024-02-23T00:44:22.380841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":266,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.299587+0000","last_change":"2024-02-23T00:45:50.685890+0000","last_active":"2024-02-23T00:46:01.299587+0000","last_peered":"2024-02-23T00:46:01.299587+0000","last_clean":"2024-02-23T00:46:01.299587+0000","last_became_active":"2024-02-23T00:43:35.079004+0000","last_became_peered":"2024-02-23T00:43:35.079004+0000","last_unstale":"2024-02-23T00:46:01.299587+0000","last_undegraded":"2024-02-23T00:46:01.299587+0000","last_fullsized":"2024-02-23T00:46:01.299587+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:50.685824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:45:50.685824+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"110'5","reported_seq":199,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.299786+0000","last_change":"2024-02-23T00:45:06.483322+0000","last_active":"2024-02-23T00:46:01.299786+0000","last_peered":"2024-02-23T00:46:01.299786+0000","last_clean":"2024-02-23T00:46:01.299786+0000","last_became_active":"2024-02-23T00:42:58.741558+0000","last_became_peered":"2024-02-23T00:42:58.741558+0000","last_unstale":"2024-02-23T00:46:01.299786+0000","last_undegraded":"2024-02-23T00:46:01.299786+0000","last_fullsized":"2024-02-23T00:46:01.299786+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'5","last_scrub_stamp":"2024-02-23T00:45:06.483252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.483252+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":195,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.300967+0000","last_change":"2024-02-23T00:45:22.597544+0000","last_active":"2024-02-23T00:46:01.300967+0000","last_peered":"2024-02-23T00:46:01.300967+0000","last_clean":"2024-02-23T00:46:01.300967+0000","last_became_active":"2024-02-23T00:42:58.867305+0000","last_became_peered":"2024-02-23T00:42:58.867305+0000","last_unstale":"2024-02-23T00:46:01.300967+0000","last_undegraded":"2024-02-23T00:46:01.300967+0000","last_fullsized":"2024-02-23T00:46:01.300967+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:22.597434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:22.597434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"88'35","reported_seq":306,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.208044+0000","last_change":"2024-02-23T00:43:54.438282+0000","last_active":"2024-02-23T00:46:01.208044+0000","last_peered":"2024-02-23T00:46:01.208044+0000","last_clean":"2024-02-23T00:46:01.208044+0000","last_became_active":"2024-02-23T00:39:16.857413+0000","last_became_peered":"2024-02-23T00:39:16.857413+0000","last_unstale":"2024-02-23T00:46:01.208044+0000","last_undegraded":"2024-02-23T00:46:01.208044+0000","last_fullsized":"2024-02-23T00:46:01.208044+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'35","last_scrub_stamp":"2024-02-23T00:43:54.438168+0000","last_deep_scrub":"88'35","last_deep_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_clean_scrub_stamp":"2024-02-23T00:43:54.438168+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":252,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.302175+0000","last_change":"2024-02-23T00:43:53.317283+0000","last_active":"2024-02-23T00:46:01.302175+0000","last_peered":"2024-02-23T00:46:01.302175+0000","last_clean":"2024-02-23T00:46:01.302175+0000","last_became_active":"2024-02-23T00:39:14.776705+0000","last_became_peered":"2024-02-23T00:39:14.776705+0000","last_unstale":"2024-02-23T00:46:01.302175+0000","last_undegraded":"2024-02-23T00:46:01.302175+0000","last_fullsized":"2024-02-23T00:46:01.302175+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:53.317202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_clean_scrub_stamp":"2024-02-23T00:43:53.317202+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":248,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.300859+0000","last_change":"2024-02-23T00:43:44.313672+0000","last_active":"2024-02-23T00:46:01.300859+0000","last_peered":"2024-02-23T00:46:01.300859+0000","last_clean":"2024-02-23T00:46:01.300859+0000","last_became_active":"2024-02-23T00:39:18.792718+0000","last_became_peered":"2024-02-23T00:39:18.792718+0000","last_unstale":"2024-02-23T00:46:01.300859+0000","last_undegraded":"2024-02-23T00:46:01.300859+0000","last_fullsized":"2024-02-23T00:46:01.300859+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:44.313501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:44.313501+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":262,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.163669+0000","last_change":"2024-02-23T00:44:36.758043+0000","last_active":"2024-02-23T00:46:00.163669+0000","last_peered":"2024-02-23T00:46:00.163669+0000","last_clean":"2024-02-23T00:46:00.163669+0000","last_became_active":"2024-02-23T00:43:39.470999+0000","last_became_peered":"2024-02-23T00:43:39.470999+0000","last_unstale":"2024-02-23T00:46:00.163669+0000","last_undegraded":"2024-02-23T00:46:00.163669+0000","last_fullsized":"2024-02-23T00:46:00.163669+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:36.757960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:36.757960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"110'5","reported_seq":199,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.206614+0000","last_change":"2024-02-23T00:45:23.832372+0000","last_active":"2024-02-23T00:46:01.206614+0000","last_peered":"2024-02-23T00:46:01.206614+0000","last_clean":"2024-02-23T00:46:01.206614+0000","last_became_active":"2024-02-23T00:42:58.684001+0000","last_became_peered":"2024-02-23T00:42:58.684001+0000","last_unstale":"2024-02-23T00:46:01.206614+0000","last_undegraded":"2024-02-23T00:46:01.206614+0000","last_fullsized":"2024-02-23T00:46:01.206614+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'5","last_scrub_stamp":"2024-02-23T00:45:23.832291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.832291+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":190,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.160128+0000","last_change":"2024-02-23T00:42:57.641727+0000","last_active":"2024-02-23T00:46:01.160128+0000","last_peered":"2024-02-23T00:46:01.160128+0000","last_clean":"2024-02-23T00:46:01.160128+0000","last_became_active":"2024-02-23T00:42:57.639013+0000","last_became_peered":"2024-02-23T00:42:57.639013+0000","last_unstale":"2024-02-23T00:46:01.160128+0000","last_undegraded":"2024-02-23T00:46:01.160128+0000","last_fullsized":"2024-02-23T00:46:01.160128+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"67'17","reported_seq":301,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.159087+0000","last_change":"2024-02-23T00:43:45.194207+0000","last_active":"2024-02-23T00:46:01.159087+0000","last_peered":"2024-02-23T00:46:01.159087+0000","last_clean":"2024-02-23T00:46:01.159087+0000","last_became_active":"2024-02-23T00:40:42.933806+0000","last_became_peered":"2024-02-23T00:40:42.933806+0000","last_unstale":"2024-02-23T00:46:01.159087+0000","last_undegraded":"2024-02-23T00:46:01.159087+0000","last_fullsized":"2024-02-23T00:46:01.159087+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'17","last_scrub_stamp":"2024-02-23T00:43:45.194139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.194139+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":228,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.300992+0000","last_change":"2024-02-23T00:45:56.051644+0000","last_active":"2024-02-23T00:46:01.300992+0000","last_peered":"2024-02-23T00:46:01.300992+0000","last_clean":"2024-02-23T00:46:01.300992+0000","last_became_active":"2024-02-23T00:40:42.931041+0000","last_became_peered":"2024-02-23T00:40:42.931041+0000","last_unstale":"2024-02-23T00:46:01.300992+0000","last_undegraded":"2024-02-23T00:46:01.300992+0000","last_fullsized":"2024-02-23T00:46:01.300992+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:56.051575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:56.051575+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":249,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.160543+0000","last_change":"2024-02-23T00:44:22.804352+0000","last_active":"2024-02-23T00:46:01.160543+0000","last_peered":"2024-02-23T00:46:01.160543+0000","last_clean":"2024-02-23T00:46:01.160543+0000","last_became_active":"2024-02-23T00:39:18.799875+0000","last_became_peered":"2024-02-23T00:39:18.799875+0000","last_unstale":"2024-02-23T00:46:01.160543+0000","last_undegraded":"2024-02-23T00:46:01.160543+0000","last_fullsized":"2024-02-23T00:46:01.160543+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:22.804279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:22.804279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":264,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.301515+0000","last_change":"2024-02-23T00:44:36.481327+0000","last_active":"2024-02-23T00:46:01.301515+0000","last_peered":"2024-02-23T00:46:01.301515+0000","last_clean":"2024-02-23T00:46:01.301515+0000","last_became_active":"2024-02-23T00:43:08.885257+0000","last_became_peered":"2024-02-23T00:43:08.885257+0000","last_unstale":"2024-02-23T00:46:01.301515+0000","last_undegraded":"2024-02-23T00:46:01.301515+0000","last_fullsized":"2024-02-23T00:46:01.301515+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:36.481256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:36.481256+0000","last_clean_scrub_stamp":"2024-02-23T00:44:36.481256+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.6","version":"110'9","reported_seq":199,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.301934+0000","last_change":"2024-02-23T00:42:58.871934+0000","last_active":"2024-02-23T00:46:01.301934+0000","last_peered":"2024-02-23T00:46:01.301934+0000","last_clean":"2024-02-23T00:46:01.301934+0000","last_became_active":"2024-02-23T00:42:58.870147+0000","last_became_peered":"2024-02-23T00:42:58.870147+0000","last_unstale":"2024-02-23T00:46:01.301934+0000","last_undegraded":"2024-02-23T00:46:01.301934+0000","last_fullsized":"2024-02-23T00:46:01.301934+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":195,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.162031+0000","last_change":"2024-02-23T00:45:23.744990+0000","last_active":"2024-02-23T00:46:01.162031+0000","last_peered":"2024-02-23T00:46:01.162031+0000","last_clean":"2024-02-23T00:46:01.162031+0000","last_became_active":"2024-02-23T00:42:57.675306+0000","last_became_peered":"2024-02-23T00:42:57.675306+0000","last_unstale":"2024-02-23T00:46:01.162031+0000","last_undegraded":"2024-02-23T00:46:01.162031+0000","last_fullsized":"2024-02-23T00:46:01.162031+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:23.744846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.744846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"67'10","reported_seq":266,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.158848+0000","last_change":"2024-02-23T00:45:25.175258+0000","last_active":"2024-02-23T00:46:01.158848+0000","last_peered":"2024-02-23T00:46:01.158848+0000","last_clean":"2024-02-23T00:46:01.158848+0000","last_became_active":"2024-02-23T00:39:16.777530+0000","last_became_peered":"2024-02-23T00:39:16.777530+0000","last_unstale":"2024-02-23T00:46:01.158848+0000","last_undegraded":"2024-02-23T00:46:01.158848+0000","last_fullsized":"2024-02-23T00:46:01.158848+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'10","last_scrub_stamp":"2024-02-23T00:45:25.175173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.175173+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":253,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.158841+0000","last_change":"2024-02-23T00:45:10.693332+0000","last_active":"2024-02-23T00:46:01.158841+0000","last_peered":"2024-02-23T00:46:01.158841+0000","last_clean":"2024-02-23T00:46:01.158841+0000","last_became_active":"2024-02-23T00:39:14.780379+0000","last_became_peered":"2024-02-23T00:39:14.780379+0000","last_unstale":"2024-02-23T00:46:01.158841+0000","last_undegraded":"2024-02-23T00:46:01.158841+0000","last_fullsized":"2024-02-23T00:46:01.158841+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.693273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.575242+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.693273+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"67'8","reported_seq":337,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.300724+0000","last_change":"2024-02-23T00:44:25.470245+0000","last_active":"2024-02-23T00:46:01.300724+0000","last_peered":"2024-02-23T00:46:01.300724+0000","last_clean":"2024-02-23T00:46:01.300724+0000","last_became_active":"2024-02-23T00:39:18.799544+0000","last_became_peered":"2024-02-23T00:39:18.799544+0000","last_unstale":"2024-02-23T00:46:01.300724+0000","last_undegraded":"2024-02-23T00:46:01.300724+0000","last_fullsized":"2024-02-23T00:46:01.300724+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:44:25.470174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:25.470174+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":262,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.159838+0000","last_change":"2024-02-23T00:43:36.205958+0000","last_active":"2024-02-23T00:46:01.159838+0000","last_peered":"2024-02-23T00:46:01.159838+0000","last_clean":"2024-02-23T00:46:01.159838+0000","last_became_active":"2024-02-23T00:43:13.338697+0000","last_became_peered":"2024-02-23T00:43:13.338697+0000","last_unstale":"2024-02-23T00:46:01.159838+0000","last_undegraded":"2024-02-23T00:46:01.159838+0000","last_fullsized":"2024-02-23T00:46:01.159838+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:36.205886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:15.081153+0000","last_clean_scrub_stamp":"2024-02-23T00:43:36.205886+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":190,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.159913+0000","last_change":"2024-02-23T00:42:57.686234+0000","last_active":"2024-02-23T00:46:01.159913+0000","last_peered":"2024-02-23T00:46:01.159913+0000","last_clean":"2024-02-23T00:46:01.159913+0000","last_became_active":"2024-02-23T00:42:57.685579+0000","last_became_peered":"2024-02-23T00:42:57.685579+0000","last_unstale":"2024-02-23T00:46:01.159913+0000","last_undegraded":"2024-02-23T00:46:01.159913+0000","last_fullsized":"2024-02-23T00:46:01.159913+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":195,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.157497+0000","last_change":"2024-02-23T00:45:22.874761+0000","last_active":"2024-02-23T00:46:01.157497+0000","last_peered":"2024-02-23T00:46:01.157497+0000","last_clean":"2024-02-23T00:46:01.157497+0000","last_became_active":"2024-02-23T00:42:58.765106+0000","last_became_peered":"2024-02-23T00:42:58.765106+0000","last_unstale":"2024-02-23T00:46:01.157497+0000","last_undegraded":"2024-02-23T00:46:01.157497+0000","last_fullsized":"2024-02-23T00:46:01.157497+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:22.874696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:22.874696+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"67'15","reported_seq":276,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.160000+0000","last_change":"2024-02-23T00:44:19.830383+0000","last_active":"2024-02-23T00:46:01.160000+0000","last_peered":"2024-02-23T00:46:01.160000+0000","last_clean":"2024-02-23T00:46:01.160000+0000","last_became_active":"2024-02-23T00:39:16.789757+0000","last_became_peered":"2024-02-23T00:39:16.789757+0000","last_unstale":"2024-02-23T00:46:01.160000+0000","last_undegraded":"2024-02-23T00:46:01.160000+0000","last_fullsized":"2024-02-23T00:46:01.160000+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:44:19.830309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:44:19.830309+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":253,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.159687+0000","last_change":"2024-02-23T00:44:29.773000+0000","last_active":"2024-02-23T00:46:01.159687+0000","last_peered":"2024-02-23T00:46:01.159687+0000","last_clean":"2024-02-23T00:46:01.159687+0000","last_became_active":"2024-02-23T00:39:14.780526+0000","last_became_peered":"2024-02-23T00:39:14.780526+0000","last_unstale":"2024-02-23T00:46:01.159687+0000","last_undegraded":"2024-02-23T00:46:01.159687+0000","last_fullsized":"2024-02-23T00:46:01.159687+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:29.772933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:29.772933+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":338,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.159789+0000","last_change":"2024-02-23T00:44:23.109508+0000","last_active":"2024-02-23T00:46:01.159789+0000","last_peered":"2024-02-23T00:46:01.159789+0000","last_clean":"2024-02-23T00:46:01.159789+0000","last_became_active":"2024-02-23T00:39:18.800440+0000","last_became_peered":"2024-02-23T00:39:18.800440+0000","last_unstale":"2024-02-23T00:46:01.159789+0000","last_undegraded":"2024-02-23T00:46:01.159789+0000","last_fullsized":"2024-02-23T00:46:01.159789+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:44:23.109439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.109439+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":265,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.170478+0000","last_change":"2024-02-23T00:44:22.976713+0000","last_active":"2024-02-23T00:46:00.170478+0000","last_peered":"2024-02-23T00:46:00.170478+0000","last_clean":"2024-02-23T00:46:00.170478+0000","last_became_active":"2024-02-23T00:43:16.650527+0000","last_became_peered":"2024-02-23T00:43:16.650527+0000","last_unstale":"2024-02-23T00:46:00.170478+0000","last_undegraded":"2024-02-23T00:46:00.170478+0000","last_fullsized":"2024-02-23T00:46:00.170478+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:22.976611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:10.666299+0000","last_clean_scrub_stamp":"2024-02-23T00:44:22.976611+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":195,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.160214+0000","last_change":"2024-02-23T00:45:26.719081+0000","last_active":"2024-02-23T00:46:01.160214+0000","last_peered":"2024-02-23T00:46:01.160214+0000","last_clean":"2024-02-23T00:46:01.160214+0000","last_became_active":"2024-02-23T00:42:57.671472+0000","last_became_peered":"2024-02-23T00:42:57.671472+0000","last_unstale":"2024-02-23T00:46:01.160214+0000","last_undegraded":"2024-02-23T00:46:01.160214+0000","last_fullsized":"2024-02-23T00:46:01.160214+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:26.719011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:26.719011+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,3],"acting":[1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":195,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.158282+0000","last_change":"2024-02-23T00:45:54.311565+0000","last_active":"2024-02-23T00:46:01.158282+0000","last_peered":"2024-02-23T00:46:01.158282+0000","last_clean":"2024-02-23T00:46:01.158282+0000","last_became_active":"2024-02-23T00:42:58.833924+0000","last_became_peered":"2024-02-23T00:42:58.833924+0000","last_unstale":"2024-02-23T00:46:01.158282+0000","last_undegraded":"2024-02-23T00:46:01.158282+0000","last_fullsized":"2024-02-23T00:46:01.158282+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:54.311442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:54.311442+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"68'17","reported_seq":303,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.159632+0000","last_change":"2024-02-23T00:43:33.151471+0000","last_active":"2024-02-23T00:46:01.159632+0000","last_peered":"2024-02-23T00:46:01.159632+0000","last_clean":"2024-02-23T00:46:01.159632+0000","last_became_active":"2024-02-23T00:40:42.933759+0000","last_became_peered":"2024-02-23T00:40:42.933759+0000","last_unstale":"2024-02-23T00:46:01.159632+0000","last_undegraded":"2024-02-23T00:46:01.159632+0000","last_fullsized":"2024-02-23T00:46:01.159632+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:43:33.151401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:33.151401+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":250,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.164025+0000","last_change":"2024-02-23T00:44:18.886545+0000","last_active":"2024-02-23T00:46:00.164025+0000","last_peered":"2024-02-23T00:46:00.164025+0000","last_clean":"2024-02-23T00:46:00.164025+0000","last_became_active":"2024-02-23T00:39:14.784121+0000","last_became_peered":"2024-02-23T00:39:14.784121+0000","last_unstale":"2024-02-23T00:46:00.164025+0000","last_undegraded":"2024-02-23T00:46:00.164025+0000","last_fullsized":"2024-02-23T00:46:00.164025+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:18.886477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:18.886477+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":249,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.172593+0000","last_change":"2024-02-23T00:45:08.118391+0000","last_active":"2024-02-23T00:46:00.172593+0000","last_peered":"2024-02-23T00:46:00.172593+0000","last_clean":"2024-02-23T00:46:00.172593+0000","last_became_active":"2024-02-23T00:39:18.800777+0000","last_became_peered":"2024-02-23T00:39:18.800777+0000","last_unstale":"2024-02-23T00:46:00.172593+0000","last_undegraded":"2024-02-23T00:46:00.172593+0000","last_fullsized":"2024-02-23T00:46:00.172593+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.118302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.118302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"67'1","reported_seq":266,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.162059+0000","last_change":"2024-02-23T00:43:56.091057+0000","last_active":"2024-02-23T00:46:01.162059+0000","last_peered":"2024-02-23T00:46:01.162059+0000","last_clean":"2024-02-23T00:46:01.162059+0000","last_became_active":"2024-02-23T00:43:22.866277+0000","last_became_peered":"2024-02-23T00:43:22.866277+0000","last_unstale":"2024-02-23T00:46:01.162059+0000","last_undegraded":"2024-02-23T00:46:01.162059+0000","last_fullsized":"2024-02-23T00:46:01.162059+0000","mapping_epoch":65,"log_start":"67'1","ondisk_log_start":"67'1","created":65,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:43:56.090982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:09.079918+0000","last_clean_scrub_stamp":"2024-02-23T00:43:56.090982+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":194,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.206847+0000","last_change":"2024-02-23T00:45:06.651852+0000","last_active":"2024-02-23T00:46:01.206847+0000","last_peered":"2024-02-23T00:46:01.206847+0000","last_clean":"2024-02-23T00:46:01.206847+0000","last_became_active":"2024-02-23T00:42:58.718615+0000","last_became_peered":"2024-02-23T00:42:58.718615+0000","last_unstale":"2024-02-23T00:46:01.206847+0000","last_undegraded":"2024-02-23T00:46:01.206847+0000","last_fullsized":"2024-02-23T00:46:01.206847+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.651711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.651711+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":190,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.158020+0000","last_change":"2024-02-23T00:42:57.655252+0000","last_active":"2024-02-23T00:46:01.158020+0000","last_peered":"2024-02-23T00:46:01.158020+0000","last_clean":"2024-02-23T00:46:01.158020+0000","last_became_active":"2024-02-23T00:42:57.653279+0000","last_became_peered":"2024-02-23T00:42:57.653279+0000","last_unstale":"2024-02-23T00:46:01.158020+0000","last_undegraded":"2024-02-23T00:46:01.158020+0000","last_fullsized":"2024-02-23T00:46:01.158020+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":339,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.160340+0000","last_change":"2024-02-23T00:45:51.871688+0000","last_active":"2024-02-23T00:46:01.160340+0000","last_peered":"2024-02-23T00:46:01.160340+0000","last_clean":"2024-02-23T00:46:01.160340+0000","last_became_active":"2024-02-23T00:39:18.798536+0000","last_became_peered":"2024-02-23T00:39:18.798536+0000","last_unstale":"2024-02-23T00:46:01.160340+0000","last_undegraded":"2024-02-23T00:46:01.160340+0000","last_fullsized":"2024-02-23T00:46:01.160340+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:45:51.871618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:51.871618+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"68'20","reported_seq":287,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.158171+0000","last_change":"2024-02-23T00:45:05.291909+0000","last_active":"2024-02-23T00:46:01.158171+0000","last_peered":"2024-02-23T00:46:01.158171+0000","last_clean":"2024-02-23T00:46:01.158171+0000","last_became_active":"2024-02-23T00:39:16.780130+0000","last_became_peered":"2024-02-23T00:39:16.780130+0000","last_unstale":"2024-02-23T00:46:01.158171+0000","last_undegraded":"2024-02-23T00:46:01.158171+0000","last_fullsized":"2024-02-23T00:46:01.158171+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'20","last_scrub_stamp":"2024-02-23T00:45:05.291836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:05.291836+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":254,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.208816+0000","last_change":"2024-02-23T00:43:51.391999+0000","last_active":"2024-02-23T00:46:01.208816+0000","last_peered":"2024-02-23T00:46:01.208816+0000","last_clean":"2024-02-23T00:46:01.208816+0000","last_became_active":"2024-02-23T00:39:14.781999+0000","last_became_peered":"2024-02-23T00:39:14.781999+0000","last_unstale":"2024-02-23T00:46:01.208816+0000","last_undegraded":"2024-02-23T00:46:01.208816+0000","last_fullsized":"2024-02-23T00:46:01.208816+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:51.391939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:51.391939+0000","last_clean_scrub_stamp":"2024-02-23T00:43:51.391939+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":244,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.164426+0000","last_change":"2024-02-23T00:44:24.759694+0000","last_active":"2024-02-23T00:46:00.164426+0000","last_peered":"2024-02-23T00:46:00.164426+0000","last_clean":"2024-02-23T00:46:00.164426+0000","last_became_active":"2024-02-23T00:39:21.161928+0000","last_became_peered":"2024-02-23T00:39:21.161928+0000","last_unstale":"2024-02-23T00:46:00.164426+0000","last_undegraded":"2024-02-23T00:46:00.164426+0000","last_fullsized":"2024-02-23T00:46:00.164426+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:24.759627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.945642+0000","last_clean_scrub_stamp":"2024-02-23T00:44:24.759627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"110'10","reported_seq":204,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.300749+0000","last_change":"2024-02-23T00:45:24.377454+0000","last_active":"2024-02-23T00:46:01.300749+0000","last_peered":"2024-02-23T00:46:01.300749+0000","last_clean":"2024-02-23T00:46:01.300749+0000","last_became_active":"2024-02-23T00:42:58.696292+0000","last_became_peered":"2024-02-23T00:42:58.696292+0000","last_unstale":"2024-02-23T00:46:01.300749+0000","last_undegraded":"2024-02-23T00:46:01.300749+0000","last_fullsized":"2024-02-23T00:46:01.300749+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'10","last_scrub_stamp":"2024-02-23T00:45:24.377329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:24.377329+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":194,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.172688+0000","last_change":"2024-02-23T00:45:59.944702+0000","last_active":"2024-02-23T00:46:00.172688+0000","last_peered":"2024-02-23T00:46:00.172688+0000","last_clean":"2024-02-23T00:46:00.172688+0000","last_became_active":"2024-02-23T00:42:58.717371+0000","last_became_peered":"2024-02-23T00:42:58.717371+0000","last_unstale":"2024-02-23T00:46:00.172688+0000","last_undegraded":"2024-02-23T00:46:00.172688+0000","last_fullsized":"2024-02-23T00:46:00.172688+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:59.944587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:59.944587+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":249,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.158455+0000","last_change":"2024-02-23T00:44:34.059194+0000","last_active":"2024-02-23T00:46:01.158455+0000","last_peered":"2024-02-23T00:46:01.158455+0000","last_clean":"2024-02-23T00:46:01.158455+0000","last_became_active":"2024-02-23T00:39:18.798829+0000","last_became_peered":"2024-02-23T00:39:18.798829+0000","last_unstale":"2024-02-23T00:46:01.158455+0000","last_undegraded":"2024-02-23T00:46:01.158455+0000","last_fullsized":"2024-02-23T00:46:01.158455+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:34.059129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:34.059129+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"251'40","reported_seq":324,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.160256+0000","last_change":"2024-02-23T00:45:23.910609+0000","last_active":"2024-02-23T00:46:01.160256+0000","last_peered":"2024-02-23T00:46:01.160256+0000","last_clean":"2024-02-23T00:46:01.160256+0000","last_became_active":"2024-02-23T00:39:16.859002+0000","last_became_peered":"2024-02-23T00:39:16.859002+0000","last_unstale":"2024-02-23T00:46:01.160256+0000","last_undegraded":"2024-02-23T00:46:01.160256+0000","last_fullsized":"2024-02-23T00:46:01.160256+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'40","last_scrub_stamp":"2024-02-23T00:45:23.910546+0000","last_deep_scrub":"251'40","last_deep_scrub_stamp":"2024-02-23T00:45:23.910546+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.910546+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":253,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.157593+0000","last_change":"2024-02-23T00:45:16.664785+0000","last_active":"2024-02-23T00:46:01.157593+0000","last_peered":"2024-02-23T00:46:01.157593+0000","last_clean":"2024-02-23T00:46:01.157593+0000","last_became_active":"2024-02-23T00:39:14.779193+0000","last_became_peered":"2024-02-23T00:39:14.779193+0000","last_unstale":"2024-02-23T00:46:01.157593+0000","last_undegraded":"2024-02-23T00:46:01.157593+0000","last_fullsized":"2024-02-23T00:46:01.157593+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.664682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.664682+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":247,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.158548+0000","last_change":"2024-02-23T00:45:03.228087+0000","last_active":"2024-02-23T00:46:01.158548+0000","last_peered":"2024-02-23T00:46:01.158548+0000","last_clean":"2024-02-23T00:46:01.158548+0000","last_became_active":"2024-02-23T00:39:20.795271+0000","last_became_peered":"2024-02-23T00:39:20.795271+0000","last_unstale":"2024-02-23T00:46:01.158548+0000","last_undegraded":"2024-02-23T00:46:01.158548+0000","last_fullsized":"2024-02-23T00:46:01.158548+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:03.227995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:45:03.227995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"110'2","reported_seq":44,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.159455+0000","last_change":"2024-02-23T00:45:45.168073+0000","last_active":"2024-02-23T00:46:01.159455+0000","last_peered":"2024-02-23T00:46:01.159455+0000","last_clean":"2024-02-23T00:46:01.159455+0000","last_became_active":"2024-02-23T00:45:45.162258+0000","last_became_peered":"2024-02-23T00:45:45.162258+0000","last_unstale":"2024-02-23T00:46:01.159455+0000","last_undegraded":"2024-02-23T00:46:01.159455+0000","last_fullsized":"2024-02-23T00:46:01.159455+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'2","last_scrub_stamp":"2024-02-23T00:45:10.465108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.465108+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":195,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.161397+0000","last_change":"2024-02-23T00:45:13.874437+0000","last_active":"2024-02-23T00:46:01.161397+0000","last_peered":"2024-02-23T00:46:01.161397+0000","last_clean":"2024-02-23T00:46:01.161397+0000","last_became_active":"2024-02-23T00:42:58.809283+0000","last_became_peered":"2024-02-23T00:42:58.809283+0000","last_unstale":"2024-02-23T00:46:01.161397+0000","last_undegraded":"2024-02-23T00:46:01.161397+0000","last_fullsized":"2024-02-23T00:46:01.161397+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:13.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:13.874373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":248,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.207060+0000","last_change":"2024-02-23T00:44:19.312509+0000","last_active":"2024-02-23T00:46:01.207060+0000","last_peered":"2024-02-23T00:46:01.207060+0000","last_clean":"2024-02-23T00:46:01.207060+0000","last_became_active":"2024-02-23T00:39:18.796664+0000","last_became_peered":"2024-02-23T00:39:18.796664+0000","last_unstale":"2024-02-23T00:46:01.207060+0000","last_undegraded":"2024-02-23T00:46:01.207060+0000","last_fullsized":"2024-02-23T00:46:01.207060+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:19.312418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:19.312418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"203'34","reported_seq":312,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.299504+0000","last_change":"2024-02-23T00:44:35.644033+0000","last_active":"2024-02-23T00:46:01.299504+0000","last_peered":"2024-02-23T00:46:01.299504+0000","last_clean":"2024-02-23T00:46:01.299504+0000","last_became_active":"2024-02-23T00:39:16.857278+0000","last_became_peered":"2024-02-23T00:39:16.857278+0000","last_unstale":"2024-02-23T00:46:01.299504+0000","last_undegraded":"2024-02-23T00:46:01.299504+0000","last_fullsized":"2024-02-23T00:46:01.299504+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'34","last_scrub_stamp":"2024-02-23T00:44:35.643931+0000","last_deep_scrub":"203'34","last_deep_scrub_stamp":"2024-02-23T00:44:35.643931+0000","last_clean_scrub_stamp":"2024-02-23T00:44:35.643931+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":250,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.166968+0000","last_change":"2024-02-23T00:45:12.730663+0000","last_active":"2024-02-23T00:46:00.166968+0000","last_peered":"2024-02-23T00:46:00.166968+0000","last_clean":"2024-02-23T00:46:00.166968+0000","last_became_active":"2024-02-23T00:39:14.783662+0000","last_became_peered":"2024-02-23T00:39:14.783662+0000","last_unstale":"2024-02-23T00:46:00.166968+0000","last_undegraded":"2024-02-23T00:46:00.166968+0000","last_fullsized":"2024-02-23T00:46:00.166968+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.730539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.730539+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":247,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.159760+0000","last_change":"2024-02-23T00:45:04.585266+0000","last_active":"2024-02-23T00:46:01.159760+0000","last_peered":"2024-02-23T00:46:01.159760+0000","last_clean":"2024-02-23T00:46:01.159760+0000","last_became_active":"2024-02-23T00:39:21.160060+0000","last_became_peered":"2024-02-23T00:39:21.160060+0000","last_unstale":"2024-02-23T00:46:01.159760+0000","last_undegraded":"2024-02-23T00:46:01.159760+0000","last_fullsized":"2024-02-23T00:46:01.159760+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:04.585179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.585179+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.8","version":"110'2","reported_seq":196,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.299371+0000","last_change":"2024-02-23T00:44:33.389680+0000","last_active":"2024-02-23T00:46:01.299371+0000","last_peered":"2024-02-23T00:46:01.299371+0000","last_clean":"2024-02-23T00:46:01.299371+0000","last_became_active":"2024-02-23T00:42:58.809584+0000","last_became_peered":"2024-02-23T00:42:58.809584+0000","last_unstale":"2024-02-23T00:46:01.299371+0000","last_undegraded":"2024-02-23T00:46:01.299371+0000","last_fullsized":"2024-02-23T00:46:01.299371+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'2","last_scrub_stamp":"2024-02-23T00:44:33.389605+0000","last_deep_scrub":"110'2","last_deep_scrub_stamp":"2024-02-23T00:44:33.389605+0000","last_clean_scrub_stamp":"2024-02-23T00:44:33.389605+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":194,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.169971+0000","last_change":"2024-02-23T00:45:18.185097+0000","last_active":"2024-02-23T00:46:00.169971+0000","last_peered":"2024-02-23T00:46:00.169971+0000","last_clean":"2024-02-23T00:46:00.169971+0000","last_became_active":"2024-02-23T00:42:58.726652+0000","last_became_peered":"2024-02-23T00:42:58.726652+0000","last_unstale":"2024-02-23T00:46:00.169971+0000","last_undegraded":"2024-02-23T00:46:00.169971+0000","last_fullsized":"2024-02-23T00:46:00.169971+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:18.184992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:18.184992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":249,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.172610+0000","last_change":"2024-02-23T00:43:45.054194+0000","last_active":"2024-02-23T00:46:00.172610+0000","last_peered":"2024-02-23T00:46:00.172610+0000","last_clean":"2024-02-23T00:46:00.172610+0000","last_became_active":"2024-02-23T00:39:18.800378+0000","last_became_peered":"2024-02-23T00:39:18.800378+0000","last_unstale":"2024-02-23T00:46:00.172610+0000","last_undegraded":"2024-02-23T00:46:00.172610+0000","last_fullsized":"2024-02-23T00:46:00.172610+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:45.054115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:59.694033+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.054115+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"95'18","reported_seq":276,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.301451+0000","last_change":"2024-02-23T00:44:18.630435+0000","last_active":"2024-02-23T00:46:01.301451+0000","last_peered":"2024-02-23T00:46:01.301451+0000","last_clean":"2024-02-23T00:46:01.301451+0000","last_became_active":"2024-02-23T00:39:16.787143+0000","last_became_peered":"2024-02-23T00:39:16.787143+0000","last_unstale":"2024-02-23T00:46:01.301451+0000","last_undegraded":"2024-02-23T00:46:01.301451+0000","last_fullsized":"2024-02-23T00:46:01.301451+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'18","last_scrub_stamp":"2024-02-23T00:44:18.630363+0000","last_deep_scrub":"95'18","last_deep_scrub_stamp":"2024-02-23T00:42:01.195545+0000","last_clean_scrub_stamp":"2024-02-23T00:44:18.630363+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":255,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.163829+0000","last_change":"2024-02-23T00:45:52.795624+0000","last_active":"2024-02-23T00:46:00.163829+0000","last_peered":"2024-02-23T00:46:00.163829+0000","last_clean":"2024-02-23T00:46:00.163829+0000","last_became_active":"2024-02-23T00:39:14.784578+0000","last_became_peered":"2024-02-23T00:39:14.784578+0000","last_unstale":"2024-02-23T00:46:00.163829+0000","last_undegraded":"2024-02-23T00:46:00.163829+0000","last_fullsized":"2024-02-23T00:46:00.163829+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:52.795527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:52.795527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":254,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.299324+0000","last_change":"2024-02-23T00:43:48.258338+0000","last_active":"2024-02-23T00:46:01.299324+0000","last_peered":"2024-02-23T00:46:01.299324+0000","last_clean":"2024-02-23T00:46:01.299324+0000","last_became_active":"2024-02-23T00:39:14.775446+0000","last_became_peered":"2024-02-23T00:39:14.775446+0000","last_unstale":"2024-02-23T00:46:01.299324+0000","last_undegraded":"2024-02-23T00:46:01.299324+0000","last_fullsized":"2024-02-23T00:46:01.299324+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:48.258269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.258269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":242,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.161019+0000","last_change":"2024-02-23T00:43:47.905617+0000","last_active":"2024-02-23T00:46:01.161019+0000","last_peered":"2024-02-23T00:46:01.161019+0000","last_clean":"2024-02-23T00:46:01.161019+0000","last_became_active":"2024-02-23T00:39:20.797242+0000","last_became_peered":"2024-02-23T00:39:20.797242+0000","last_unstale":"2024-02-23T00:46:01.161019+0000","last_undegraded":"2024-02-23T00:46:01.161019+0000","last_fullsized":"2024-02-23T00:46:01.161019+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:47.905501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:47.905501+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"110'7","reported_seq":50,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.206438+0000","last_change":"2024-02-23T00:45:59.164904+0000","last_active":"2024-02-23T00:46:01.206438+0000","last_peered":"2024-02-23T00:46:01.206438+0000","last_clean":"2024-02-23T00:46:01.206438+0000","last_became_active":"2024-02-23T00:45:59.164483+0000","last_became_peered":"2024-02-23T00:45:59.164483+0000","last_unstale":"2024-02-23T00:46:01.206438+0000","last_undegraded":"2024-02-23T00:46:01.206438+0000","last_fullsized":"2024-02-23T00:46:01.206438+0000","mapping_epoch":272,"log_start":"110'7","ondisk_log_start":"110'7","created":108,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'5","last_scrub_stamp":"2024-02-23T00:45:26.760862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:26.760862+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":194,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.205065+0000","last_change":"2024-02-23T00:44:35.360128+0000","last_active":"2024-02-23T00:46:01.205065+0000","last_peered":"2024-02-23T00:46:01.205065+0000","last_clean":"2024-02-23T00:46:01.205065+0000","last_became_active":"2024-02-23T00:42:58.690858+0000","last_became_peered":"2024-02-23T00:42:58.690858+0000","last_unstale":"2024-02-23T00:46:01.205065+0000","last_undegraded":"2024-02-23T00:46:01.205065+0000","last_fullsized":"2024-02-23T00:46:01.205065+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:35.360054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:35.360054+0000","last_clean_scrub_stamp":"2024-02-23T00:44:35.360054+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":246,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.165368+0000","last_change":"2024-02-23T00:45:16.746109+0000","last_active":"2024-02-23T00:46:00.165368+0000","last_peered":"2024-02-23T00:46:00.165368+0000","last_clean":"2024-02-23T00:46:00.165368+0000","last_became_active":"2024-02-23T00:39:18.793801+0000","last_became_peered":"2024-02-23T00:39:18.793801+0000","last_unstale":"2024-02-23T00:46:00.165368+0000","last_undegraded":"2024-02-23T00:46:00.165368+0000","last_fullsized":"2024-02-23T00:46:00.165368+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.746013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.746013+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"67'14","reported_seq":272,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.206658+0000","last_change":"2024-02-23T00:43:44.369862+0000","last_active":"2024-02-23T00:46:01.206658+0000","last_peered":"2024-02-23T00:46:01.206658+0000","last_clean":"2024-02-23T00:46:01.206658+0000","last_became_active":"2024-02-23T00:39:16.858880+0000","last_became_peered":"2024-02-23T00:39:16.858880+0000","last_unstale":"2024-02-23T00:46:01.206658+0000","last_undegraded":"2024-02-23T00:46:01.206658+0000","last_fullsized":"2024-02-23T00:46:01.206658+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'14","last_scrub_stamp":"2024-02-23T00:43:44.369792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:44.369792+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":260,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.163935+0000","last_change":"2024-02-23T00:45:08.796153+0000","last_active":"2024-02-23T00:46:00.163935+0000","last_peered":"2024-02-23T00:46:00.163935+0000","last_clean":"2024-02-23T00:46:00.163935+0000","last_became_active":"2024-02-23T00:39:08.248249+0000","last_became_peered":"2024-02-23T00:39:08.248249+0000","last_unstale":"2024-02-23T00:46:00.163935+0000","last_undegraded":"2024-02-23T00:46:00.163935+0000","last_fullsized":"2024-02-23T00:46:00.163935+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.796076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:49.977248+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.796076+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":253,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.169304+0000","last_change":"2024-02-23T00:45:22.272078+0000","last_active":"2024-02-23T00:46:00.169304+0000","last_peered":"2024-02-23T00:46:00.169304+0000","last_clean":"2024-02-23T00:46:00.169304+0000","last_became_active":"2024-02-23T00:39:14.788888+0000","last_became_peered":"2024-02-23T00:39:14.788888+0000","last_unstale":"2024-02-23T00:46:00.169304+0000","last_undegraded":"2024-02-23T00:46:00.169304+0000","last_fullsized":"2024-02-23T00:46:00.169304+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:22.272003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:22.272003+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":254,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.157416+0000","last_change":"2024-02-23T00:44:33.143663+0000","last_active":"2024-02-23T00:46:01.157416+0000","last_peered":"2024-02-23T00:46:01.157416+0000","last_clean":"2024-02-23T00:46:01.157416+0000","last_became_active":"2024-02-23T00:39:14.783489+0000","last_became_peered":"2024-02-23T00:39:14.783489+0000","last_unstale":"2024-02-23T00:46:01.157416+0000","last_undegraded":"2024-02-23T00:46:01.157416+0000","last_fullsized":"2024-02-23T00:46:01.157416+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:33.143592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:33.143592+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":265,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.172707+0000","last_change":"2024-02-23T00:44:20.958888+0000","last_active":"2024-02-23T00:46:00.172707+0000","last_peered":"2024-02-23T00:46:00.172707+0000","last_clean":"2024-02-23T00:46:00.172707+0000","last_became_active":"2024-02-23T00:43:42.168435+0000","last_became_peered":"2024-02-23T00:43:42.168435+0000","last_unstale":"2024-02-23T00:46:00.172707+0000","last_undegraded":"2024-02-23T00:46:00.172707+0000","last_fullsized":"2024-02-23T00:46:00.172707+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:20.958797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:20.958797+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":45,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.301196+0000","last_change":"2024-02-23T00:45:55.156529+0000","last_active":"2024-02-23T00:46:01.301196+0000","last_peered":"2024-02-23T00:46:01.301196+0000","last_clean":"2024-02-23T00:46:01.301196+0000","last_became_active":"2024-02-23T00:45:55.156026+0000","last_became_peered":"2024-02-23T00:45:55.156026+0000","last_unstale":"2024-02-23T00:46:01.301196+0000","last_undegraded":"2024-02-23T00:46:01.301196+0000","last_fullsized":"2024-02-23T00:46:01.301196+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:21.383626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:21.383626+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":193,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.164164+0000","last_change":"2024-02-23T00:45:17.763618+0000","last_active":"2024-02-23T00:46:00.164164+0000","last_peered":"2024-02-23T00:46:00.164164+0000","last_clean":"2024-02-23T00:46:00.164164+0000","last_became_active":"2024-02-23T00:42:58.701502+0000","last_became_peered":"2024-02-23T00:42:58.701502+0000","last_unstale":"2024-02-23T00:46:00.164164+0000","last_undegraded":"2024-02-23T00:46:00.164164+0000","last_fullsized":"2024-02-23T00:46:00.164164+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:17.763482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:17.763482+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":248,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.302826+0000","last_change":"2024-02-23T00:43:46.293962+0000","last_active":"2024-02-23T00:46:01.302826+0000","last_peered":"2024-02-23T00:46:01.302826+0000","last_clean":"2024-02-23T00:46:01.302826+0000","last_became_active":"2024-02-23T00:39:18.789330+0000","last_became_peered":"2024-02-23T00:39:18.789330+0000","last_unstale":"2024-02-23T00:46:01.302826+0000","last_undegraded":"2024-02-23T00:46:01.302826+0000","last_fullsized":"2024-02-23T00:46:01.302826+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:46.293864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.264240+0000","last_clean_scrub_stamp":"2024-02-23T00:43:46.293864+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"68'19","reported_seq":293,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.160140+0000","last_change":"2024-02-23T00:45:07.697475+0000","last_active":"2024-02-23T00:46:01.160140+0000","last_peered":"2024-02-23T00:46:01.160140+0000","last_clean":"2024-02-23T00:46:01.160140+0000","last_became_active":"2024-02-23T00:39:16.787484+0000","last_became_peered":"2024-02-23T00:39:16.787484+0000","last_unstale":"2024-02-23T00:46:01.160140+0000","last_undegraded":"2024-02-23T00:46:01.160140+0000","last_fullsized":"2024-02-23T00:46:01.160140+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'19","last_scrub_stamp":"2024-02-23T00:45:07.697394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:07.697394+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":252,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.299727+0000","last_change":"2024-02-23T00:43:54.305009+0000","last_active":"2024-02-23T00:46:01.299727+0000","last_peered":"2024-02-23T00:46:01.299727+0000","last_clean":"2024-02-23T00:46:01.299727+0000","last_became_active":"2024-02-23T00:39:14.772411+0000","last_became_peered":"2024-02-23T00:39:14.772411+0000","last_unstale":"2024-02-23T00:46:01.299727+0000","last_undegraded":"2024-02-23T00:46:01.299727+0000","last_fullsized":"2024-02-23T00:46:01.299727+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:54.304948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:54.304948+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"68'5","reported_seq":650,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.710190+0000","last_change":"2024-02-23T00:43:48.876428+0000","last_active":"2024-02-23T00:46:01.710190+0000","last_peered":"2024-02-23T00:46:01.710190+0000","last_clean":"2024-02-23T00:46:01.710190+0000","last_became_active":"2024-02-23T00:39:14.779109+0000","last_became_peered":"2024-02-23T00:39:14.779109+0000","last_unstale":"2024-02-23T00:46:01.710190+0000","last_undegraded":"2024-02-23T00:46:01.710190+0000","last_fullsized":"2024-02-23T00:46:01.710190+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'5","last_scrub_stamp":"2024-02-23T00:43:48.876356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.876356+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":399,"num_read_kb":394,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":244,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.166884+0000","last_change":"2024-02-23T00:43:47.794616+0000","last_active":"2024-02-23T00:46:00.166884+0000","last_peered":"2024-02-23T00:46:00.166884+0000","last_clean":"2024-02-23T00:46:00.166884+0000","last_became_active":"2024-02-23T00:39:21.160457+0000","last_became_peered":"2024-02-23T00:39:21.160457+0000","last_unstale":"2024-02-23T00:46:00.166884+0000","last_undegraded":"2024-02-23T00:46:00.166884+0000","last_fullsized":"2024-02-23T00:46:00.166884+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:47.794534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:47.794534+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":37,"reported_epoch":290,"state":"peering","last_fresh":"2024-02-23T00:46:02.167370+0000","last_change":"2024-02-23T00:46:02.167586+0000","last_active":"2024-02-23T00:46:00.165525+0000","last_peered":"2024-02-23T00:46:01.207908+0000","last_clean":"2024-02-23T00:46:01.207908+0000","last_became_active":"2024-02-23T00:45:45.162018+0000","last_became_peered":"2024-02-23T00:46:01.207141+0000","last_unstale":"2024-02-23T00:46:02.167370+0000","last_undegraded":"2024-02-23T00:46:02.167370+0000","last_fullsized":"2024-02-23T00:46:02.167370+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,3],"acting":[1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":190,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.159495+0000","last_change":"2024-02-23T00:42:57.653669+0000","last_active":"2024-02-23T00:46:01.159495+0000","last_peered":"2024-02-23T00:46:01.159495+0000","last_clean":"2024-02-23T00:46:01.159495+0000","last_became_active":"2024-02-23T00:42:57.653355+0000","last_became_peered":"2024-02-23T00:42:57.653355+0000","last_unstale":"2024-02-23T00:46:01.159495+0000","last_undegraded":"2024-02-23T00:46:01.159495+0000","last_fullsized":"2024-02-23T00:46:01.159495+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":250,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.300890+0000","last_change":"2024-02-23T00:44:20.626116+0000","last_active":"2024-02-23T00:46:01.300890+0000","last_peered":"2024-02-23T00:46:01.300890+0000","last_clean":"2024-02-23T00:46:01.300890+0000","last_became_active":"2024-02-23T00:39:18.802134+0000","last_became_peered":"2024-02-23T00:39:18.802134+0000","last_unstale":"2024-02-23T00:46:01.300890+0000","last_undegraded":"2024-02-23T00:46:01.300890+0000","last_fullsized":"2024-02-23T00:46:01.300890+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:20.626052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:20.626052+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"86'35","reported_seq":316,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.171647+0000","last_change":"2024-02-23T00:45:52.939553+0000","last_active":"2024-02-23T00:46:00.171647+0000","last_peered":"2024-02-23T00:46:00.171647+0000","last_clean":"2024-02-23T00:46:00.171647+0000","last_became_active":"2024-02-23T00:40:42.933322+0000","last_became_peered":"2024-02-23T00:40:42.933322+0000","last_unstale":"2024-02-23T00:46:00.171647+0000","last_undegraded":"2024-02-23T00:46:00.171647+0000","last_fullsized":"2024-02-23T00:46:00.171647+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'35","last_scrub_stamp":"2024-02-23T00:45:52.939452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:52.939452+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":253,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.173154+0000","last_change":"2024-02-23T00:44:23.965228+0000","last_active":"2024-02-23T00:46:00.173154+0000","last_peered":"2024-02-23T00:46:00.173154+0000","last_clean":"2024-02-23T00:46:00.173154+0000","last_became_active":"2024-02-23T00:39:14.784015+0000","last_became_peered":"2024-02-23T00:39:14.784015+0000","last_unstale":"2024-02-23T00:46:00.173154+0000","last_undegraded":"2024-02-23T00:46:00.173154+0000","last_fullsized":"2024-02-23T00:46:00.173154+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:23.965125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.965125+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":261,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.161255+0000","last_change":"2024-02-23T00:43:45.144499+0000","last_active":"2024-02-23T00:46:01.161255+0000","last_peered":"2024-02-23T00:46:01.161255+0000","last_clean":"2024-02-23T00:46:01.161255+0000","last_became_active":"2024-02-23T00:43:45.144043+0000","last_became_peered":"2024-02-23T00:43:45.144043+0000","last_unstale":"2024-02-23T00:46:01.161255+0000","last_undegraded":"2024-02-23T00:46:01.161255+0000","last_fullsized":"2024-02-23T00:46:01.161255+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:40.062493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:40.062493+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.f","version":"254'6","reported_seq":45,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.159735+0000","last_change":"2024-02-23T00:45:51.167662+0000","last_active":"2024-02-23T00:46:01.159735+0000","last_peered":"2024-02-23T00:46:01.159735+0000","last_clean":"2024-02-23T00:46:01.159735+0000","last_became_active":"2024-02-23T00:45:51.167136+0000","last_became_peered":"2024-02-23T00:45:51.167136+0000","last_unstale":"2024-02-23T00:46:01.159735+0000","last_undegraded":"2024-02-23T00:46:01.159735+0000","last_fullsized":"2024-02-23T00:46:01.159735+0000","mapping_epoch":272,"log_start":"254'6","ondisk_log_start":"254'6","created":108,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:31.410670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:31.410670+0000","last_clean_scrub_stamp":"2024-02-23T00:44:31.410670+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":194,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.300404+0000","last_change":"2024-02-23T00:45:05.490595+0000","last_active":"2024-02-23T00:46:01.300404+0000","last_peered":"2024-02-23T00:46:01.300404+0000","last_clean":"2024-02-23T00:46:01.300404+0000","last_became_active":"2024-02-23T00:42:58.766579+0000","last_became_peered":"2024-02-23T00:42:58.766579+0000","last_unstale":"2024-02-23T00:46:01.300404+0000","last_undegraded":"2024-02-23T00:46:01.300404+0000","last_fullsized":"2024-02-23T00:46:01.300404+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:05.490531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:05.490531+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"91'9","reported_seq":340,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.299844+0000","last_change":"2024-02-23T00:44:28.619036+0000","last_active":"2024-02-23T00:46:01.299844+0000","last_peered":"2024-02-23T00:46:01.299844+0000","last_clean":"2024-02-23T00:46:01.299844+0000","last_became_active":"2024-02-23T00:39:18.801297+0000","last_became_peered":"2024-02-23T00:39:18.801297+0000","last_unstale":"2024-02-23T00:46:01.299844+0000","last_undegraded":"2024-02-23T00:46:01.299844+0000","last_fullsized":"2024-02-23T00:46:01.299844+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'9","last_scrub_stamp":"2024-02-23T00:44:28.618923+0000","last_deep_scrub":"91'9","last_deep_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_clean_scrub_stamp":"2024-02-23T00:44:28.618923+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"68'16","reported_seq":281,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.159197+0000","last_change":"2024-02-23T00:45:11.713800+0000","last_active":"2024-02-23T00:46:01.159197+0000","last_peered":"2024-02-23T00:46:01.159197+0000","last_clean":"2024-02-23T00:46:01.159197+0000","last_became_active":"2024-02-23T00:39:16.787468+0000","last_became_peered":"2024-02-23T00:39:16.787468+0000","last_unstale":"2024-02-23T00:46:01.159197+0000","last_undegraded":"2024-02-23T00:46:01.159197+0000","last_fullsized":"2024-02-23T00:46:01.159197+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:45:11.713731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:11.713731+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":253,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.160345+0000","last_change":"2024-02-23T00:45:19.635062+0000","last_active":"2024-02-23T00:46:01.160345+0000","last_peered":"2024-02-23T00:46:01.160345+0000","last_clean":"2024-02-23T00:46:01.160345+0000","last_became_active":"2024-02-23T00:39:14.779952+0000","last_became_peered":"2024-02-23T00:39:14.779952+0000","last_unstale":"2024-02-23T00:46:01.160345+0000","last_undegraded":"2024-02-23T00:46:01.160345+0000","last_fullsized":"2024-02-23T00:46:01.160345+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:19.635002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:19.635002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"107'2","reported_seq":250,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.299975+0000","last_change":"2024-02-23T00:45:08.662754+0000","last_active":"2024-02-23T00:46:01.299975+0000","last_peered":"2024-02-23T00:46:01.299975+0000","last_clean":"2024-02-23T00:46:01.299975+0000","last_became_active":"2024-02-23T00:39:20.800858+0000","last_became_peered":"2024-02-23T00:39:20.800858+0000","last_unstale":"2024-02-23T00:46:01.299975+0000","last_undegraded":"2024-02-23T00:46:01.299975+0000","last_fullsized":"2024-02-23T00:46:01.299975+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'2","last_scrub_stamp":"2024-02-23T00:45:08.662667+0000","last_deep_scrub":"107'2","last_deep_scrub_stamp":"2024-02-23T00:42:48.362739+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.662667+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":194,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.206756+0000","last_change":"2024-02-23T00:45:19.711196+0000","last_active":"2024-02-23T00:46:01.206756+0000","last_peered":"2024-02-23T00:46:01.206756+0000","last_clean":"2024-02-23T00:46:01.206756+0000","last_became_active":"2024-02-23T00:42:58.726861+0000","last_became_peered":"2024-02-23T00:42:58.726861+0000","last_unstale":"2024-02-23T00:46:01.206756+0000","last_undegraded":"2024-02-23T00:46:01.206756+0000","last_fullsized":"2024-02-23T00:46:01.206756+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:19.711117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:19.711117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":194,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.206676+0000","last_change":"2024-02-23T00:45:04.621794+0000","last_active":"2024-02-23T00:46:01.206676+0000","last_peered":"2024-02-23T00:46:01.206676+0000","last_clean":"2024-02-23T00:46:01.206676+0000","last_became_active":"2024-02-23T00:42:58.664282+0000","last_became_peered":"2024-02-23T00:42:58.664282+0000","last_unstale":"2024-02-23T00:46:01.206676+0000","last_undegraded":"2024-02-23T00:46:01.206676+0000","last_fullsized":"2024-02-23T00:46:01.206676+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:04.621714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.621714+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":248,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.299587+0000","last_change":"2024-02-23T00:43:51.261426+0000","last_active":"2024-02-23T00:46:01.299587+0000","last_peered":"2024-02-23T00:46:01.299587+0000","last_clean":"2024-02-23T00:46:01.299587+0000","last_became_active":"2024-02-23T00:39:18.796323+0000","last_became_peered":"2024-02-23T00:39:18.796323+0000","last_unstale":"2024-02-23T00:46:01.299587+0000","last_undegraded":"2024-02-23T00:46:01.299587+0000","last_fullsized":"2024-02-23T00:46:01.299587+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:51.261367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_clean_scrub_stamp":"2024-02-23T00:43:51.261367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"68'25","reported_seq":300,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.158438+0000","last_change":"2024-02-23T00:43:46.158231+0000","last_active":"2024-02-23T00:46:01.158438+0000","last_peered":"2024-02-23T00:46:01.158438+0000","last_clean":"2024-02-23T00:46:01.158438+0000","last_became_active":"2024-02-23T00:39:16.783926+0000","last_became_peered":"2024-02-23T00:39:16.783926+0000","last_unstale":"2024-02-23T00:46:01.158438+0000","last_undegraded":"2024-02-23T00:46:01.158438+0000","last_fullsized":"2024-02-23T00:46:01.158438+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'25","last_scrub_stamp":"2024-02-23T00:43:46.158149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:46.158149+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"95'4","reported_seq":268,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.159101+0000","last_change":"2024-02-23T00:44:31.106105+0000","last_active":"2024-02-23T00:46:01.159101+0000","last_peered":"2024-02-23T00:46:01.159101+0000","last_clean":"2024-02-23T00:46:01.159101+0000","last_became_active":"2024-02-23T00:39:14.779205+0000","last_became_peered":"2024-02-23T00:39:14.779205+0000","last_unstale":"2024-02-23T00:46:01.159101+0000","last_undegraded":"2024-02-23T00:46:01.159101+0000","last_fullsized":"2024-02-23T00:46:01.159101+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'4","last_scrub_stamp":"2024-02-23T00:44:31.106012+0000","last_deep_scrub":"95'4","last_deep_scrub_stamp":"2024-02-23T00:41:43.874397+0000","last_clean_scrub_stamp":"2024-02-23T00:44:31.106012+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":266,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.157912+0000","last_change":"2024-02-23T00:43:52.036315+0000","last_active":"2024-02-23T00:46:01.157912+0000","last_peered":"2024-02-23T00:46:01.157912+0000","last_clean":"2024-02-23T00:46:01.157912+0000","last_became_active":"2024-02-23T00:43:27.611660+0000","last_became_peered":"2024-02-23T00:43:27.611660+0000","last_unstale":"2024-02-23T00:46:01.157912+0000","last_undegraded":"2024-02-23T00:46:01.157912+0000","last_fullsized":"2024-02-23T00:46:01.157912+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:52.036249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:52.036249+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":194,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.300480+0000","last_change":"2024-02-23T00:45:16.389060+0000","last_active":"2024-02-23T00:46:01.300480+0000","last_peered":"2024-02-23T00:46:01.300480+0000","last_clean":"2024-02-23T00:46:01.300480+0000","last_became_active":"2024-02-23T00:42:58.809926+0000","last_became_peered":"2024-02-23T00:42:58.809926+0000","last_unstale":"2024-02-23T00:46:01.300480+0000","last_undegraded":"2024-02-23T00:46:01.300480+0000","last_fullsized":"2024-02-23T00:46:01.300480+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.388988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.388988+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":193,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.170844+0000","last_change":"2024-02-23T00:45:20.829124+0000","last_active":"2024-02-23T00:46:00.170844+0000","last_peered":"2024-02-23T00:46:00.170844+0000","last_clean":"2024-02-23T00:46:00.170844+0000","last_became_active":"2024-02-23T00:42:58.744983+0000","last_became_peered":"2024-02-23T00:42:58.744983+0000","last_unstale":"2024-02-23T00:46:00.170844+0000","last_undegraded":"2024-02-23T00:46:00.170844+0000","last_fullsized":"2024-02-23T00:46:00.170844+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:20.829052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:20.829052+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"68'21","reported_seq":293,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.301536+0000","last_change":"2024-02-23T00:43:52.309599+0000","last_active":"2024-02-23T00:46:01.301536+0000","last_peered":"2024-02-23T00:46:01.301536+0000","last_clean":"2024-02-23T00:46:01.301536+0000","last_became_active":"2024-02-23T00:39:16.858760+0000","last_became_peered":"2024-02-23T00:39:16.858760+0000","last_unstale":"2024-02-23T00:46:01.301536+0000","last_undegraded":"2024-02-23T00:46:01.301536+0000","last_fullsized":"2024-02-23T00:46:01.301536+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'21","last_scrub_stamp":"2024-02-23T00:43:52.309532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:52.309532+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":257,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.299513+0000","last_change":"2024-02-23T00:45:22.376128+0000","last_active":"2024-02-23T00:46:01.299513+0000","last_peered":"2024-02-23T00:46:01.299513+0000","last_clean":"2024-02-23T00:46:01.299513+0000","last_became_active":"2024-02-23T00:39:14.774244+0000","last_became_peered":"2024-02-23T00:39:14.774244+0000","last_unstale":"2024-02-23T00:46:01.299513+0000","last_undegraded":"2024-02-23T00:46:01.299513+0000","last_fullsized":"2024-02-23T00:46:01.299513+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:22.376063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:22.376063+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":250,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.160301+0000","last_change":"2024-02-23T00:43:44.951800+0000","last_active":"2024-02-23T00:46:01.160301+0000","last_peered":"2024-02-23T00:46:01.160301+0000","last_clean":"2024-02-23T00:46:01.160301+0000","last_became_active":"2024-02-23T00:39:18.798624+0000","last_became_peered":"2024-02-23T00:39:18.798624+0000","last_unstale":"2024-02-23T00:46:01.160301+0000","last_undegraded":"2024-02-23T00:46:01.160301+0000","last_fullsized":"2024-02-23T00:46:01.160301+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:44.951696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:46.998875+0000","last_clean_scrub_stamp":"2024-02-23T00:43:44.951696+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":246,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.302064+0000","last_change":"2024-02-23T00:44:26.432766+0000","last_active":"2024-02-23T00:46:01.302064+0000","last_peered":"2024-02-23T00:46:01.302064+0000","last_clean":"2024-02-23T00:46:01.302064+0000","last_became_active":"2024-02-23T00:39:20.796041+0000","last_became_peered":"2024-02-23T00:39:20.796041+0000","last_unstale":"2024-02-23T00:46:01.302064+0000","last_undegraded":"2024-02-23T00:46:01.302064+0000","last_fullsized":"2024-02-23T00:46:01.302064+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:26.432703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:26.432703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":195,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.300181+0000","last_change":"2024-02-23T00:44:26.595443+0000","last_active":"2024-02-23T00:46:01.300181+0000","last_peered":"2024-02-23T00:46:01.300181+0000","last_clean":"2024-02-23T00:46:01.300181+0000","last_became_active":"2024-02-23T00:42:58.825015+0000","last_became_peered":"2024-02-23T00:42:58.825015+0000","last_unstale":"2024-02-23T00:46:01.300181+0000","last_undegraded":"2024-02-23T00:46:01.300181+0000","last_fullsized":"2024-02-23T00:46:01.300181+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:26.595346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:44:26.595346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"107'16","reported_seq":280,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.160100+0000","last_change":"2024-02-23T00:45:07.012086+0000","last_active":"2024-02-23T00:46:01.160100+0000","last_peered":"2024-02-23T00:46:01.160100+0000","last_clean":"2024-02-23T00:46:01.160100+0000","last_became_active":"2024-02-23T00:39:16.858449+0000","last_became_peered":"2024-02-23T00:39:16.858449+0000","last_unstale":"2024-02-23T00:46:01.160100+0000","last_undegraded":"2024-02-23T00:46:01.160100+0000","last_fullsized":"2024-02-23T00:46:01.160100+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'16","last_scrub_stamp":"2024-02-23T00:45:07.011956+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-02-23T00:42:52.964510+0000","last_clean_scrub_stamp":"2024-02-23T00:45:07.011956+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":252,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.205976+0000","last_change":"2024-02-23T00:45:13.613736+0000","last_active":"2024-02-23T00:46:01.205976+0000","last_peered":"2024-02-23T00:46:01.205976+0000","last_clean":"2024-02-23T00:46:01.205976+0000","last_became_active":"2024-02-23T00:39:14.783164+0000","last_became_peered":"2024-02-23T00:39:14.783164+0000","last_unstale":"2024-02-23T00:46:01.205976+0000","last_undegraded":"2024-02-23T00:46:01.205976+0000","last_fullsized":"2024-02-23T00:46:01.205976+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:13.613654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:13.613654+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":249,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.171694+0000","last_change":"2024-02-23T00:43:40.971316+0000","last_active":"2024-02-23T00:46:00.171694+0000","last_peered":"2024-02-23T00:46:00.171694+0000","last_clean":"2024-02-23T00:46:00.171694+0000","last_became_active":"2024-02-23T00:39:18.800297+0000","last_became_peered":"2024-02-23T00:39:18.800297+0000","last_unstale":"2024-02-23T00:46:00.171694+0000","last_undegraded":"2024-02-23T00:46:00.171694+0000","last_fullsized":"2024-02-23T00:46:00.171694+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:40.971243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:40.971243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":244,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.166640+0000","last_change":"2024-02-23T00:43:46.760431+0000","last_active":"2024-02-23T00:46:00.166640+0000","last_peered":"2024-02-23T00:46:00.166640+0000","last_clean":"2024-02-23T00:46:00.166640+0000","last_became_active":"2024-02-23T00:39:21.162036+0000","last_became_peered":"2024-02-23T00:39:21.162036+0000","last_unstale":"2024-02-23T00:46:00.166640+0000","last_undegraded":"2024-02-23T00:46:00.166640+0000","last_fullsized":"2024-02-23T00:46:00.166640+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:46.760348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:46.760348+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"110'7","reported_seq":208,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.158581+0000","last_change":"2024-02-23T00:45:18.947859+0000","last_active":"2024-02-23T00:46:01.158581+0000","last_peered":"2024-02-23T00:46:01.158581+0000","last_clean":"2024-02-23T00:46:01.158581+0000","last_became_active":"2024-02-23T00:42:58.777251+0000","last_became_peered":"2024-02-23T00:42:58.777251+0000","last_unstale":"2024-02-23T00:46:01.158581+0000","last_undegraded":"2024-02-23T00:46:01.158581+0000","last_fullsized":"2024-02-23T00:46:01.158581+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'7","last_scrub_stamp":"2024-02-23T00:45:18.947746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:18.947746+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"68'22","reported_seq":220,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.301817+0000","last_change":"2024-02-23T00:44:31.391929+0000","last_active":"2024-02-23T00:46:01.301817+0000","last_peered":"2024-02-23T00:46:01.301817+0000","last_clean":"2024-02-23T00:46:01.301817+0000","last_became_active":"2024-02-23T00:42:43.521189+0000","last_became_peered":"2024-02-23T00:42:43.521189+0000","last_unstale":"2024-02-23T00:46:01.301817+0000","last_undegraded":"2024-02-23T00:46:01.301817+0000","last_fullsized":"2024-02-23T00:46:01.301817+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'22","last_scrub_stamp":"2024-02-23T00:44:31.391819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:44:31.391819+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":37,"num_read_kb":30,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,7],"acting":[2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":253,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.173097+0000","last_change":"2024-02-23T00:44:18.946624+0000","last_active":"2024-02-23T00:46:00.173097+0000","last_peered":"2024-02-23T00:46:00.173097+0000","last_clean":"2024-02-23T00:46:00.173097+0000","last_became_active":"2024-02-23T00:39:14.785590+0000","last_became_peered":"2024-02-23T00:39:14.785590+0000","last_unstale":"2024-02-23T00:46:00.173097+0000","last_undegraded":"2024-02-23T00:46:00.173097+0000","last_fullsized":"2024-02-23T00:46:00.173097+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:18.946560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_clean_scrub_stamp":"2024-02-23T00:44:18.946560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"67'8","reported_seq":338,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.170302+0000","last_change":"2024-02-23T00:44:29.038226+0000","last_active":"2024-02-23T00:46:00.170302+0000","last_peered":"2024-02-23T00:46:00.170302+0000","last_clean":"2024-02-23T00:46:00.170302+0000","last_became_active":"2024-02-23T00:39:18.803566+0000","last_became_peered":"2024-02-23T00:39:18.803566+0000","last_unstale":"2024-02-23T00:46:00.170302+0000","last_undegraded":"2024-02-23T00:46:00.170302+0000","last_fullsized":"2024-02-23T00:46:00.170302+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:44:29.038134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:29.038134+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":193,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.165771+0000","last_change":"2024-02-23T00:45:54.753371+0000","last_active":"2024-02-23T00:46:00.165771+0000","last_peered":"2024-02-23T00:46:00.165771+0000","last_clean":"2024-02-23T00:46:00.165771+0000","last_became_active":"2024-02-23T00:42:58.757424+0000","last_became_peered":"2024-02-23T00:42:58.757424+0000","last_unstale":"2024-02-23T00:46:00.165771+0000","last_undegraded":"2024-02-23T00:46:00.165771+0000","last_fullsized":"2024-02-23T00:46:00.165771+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:54.753267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:54.753267+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"68'15","reported_seq":282,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.170138+0000","last_change":"2024-02-23T00:44:25.998879+0000","last_active":"2024-02-23T00:46:00.170138+0000","last_peered":"2024-02-23T00:46:00.170138+0000","last_clean":"2024-02-23T00:46:00.170138+0000","last_became_active":"2024-02-23T00:39:16.858953+0000","last_became_peered":"2024-02-23T00:39:16.858953+0000","last_unstale":"2024-02-23T00:46:00.170138+0000","last_undegraded":"2024-02-23T00:46:00.170138+0000","last_fullsized":"2024-02-23T00:46:00.170138+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:44:25.998804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:44:25.998804+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":254,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.299610+0000","last_change":"2024-02-23T00:43:45.309194+0000","last_active":"2024-02-23T00:46:01.299610+0000","last_peered":"2024-02-23T00:46:01.299610+0000","last_clean":"2024-02-23T00:46:01.299610+0000","last_became_active":"2024-02-23T00:39:14.773772+0000","last_became_peered":"2024-02-23T00:39:14.773772+0000","last_unstale":"2024-02-23T00:46:01.299610+0000","last_undegraded":"2024-02-23T00:46:01.299610+0000","last_fullsized":"2024-02-23T00:46:01.299610+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:45.309083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.309083+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"67'8","reported_seq":334,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.162318+0000","last_change":"2024-02-23T00:43:39.086199+0000","last_active":"2024-02-23T00:46:01.162318+0000","last_peered":"2024-02-23T00:46:01.162318+0000","last_clean":"2024-02-23T00:46:01.162318+0000","last_became_active":"2024-02-23T00:39:18.787313+0000","last_became_peered":"2024-02-23T00:39:18.787313+0000","last_unstale":"2024-02-23T00:46:01.162318+0000","last_undegraded":"2024-02-23T00:46:01.162318+0000","last_fullsized":"2024-02-23T00:46:01.162318+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:43:39.086107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:39.086107+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":190,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.162069+0000","last_change":"2024-02-23T00:42:57.652853+0000","last_active":"2024-02-23T00:46:01.162069+0000","last_peered":"2024-02-23T00:46:01.162069+0000","last_clean":"2024-02-23T00:46:01.162069+0000","last_became_active":"2024-02-23T00:42:57.652488+0000","last_became_peered":"2024-02-23T00:42:57.652488+0000","last_unstale":"2024-02-23T00:46:01.162069+0000","last_undegraded":"2024-02-23T00:46:01.162069+0000","last_fullsized":"2024-02-23T00:46:01.162069+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"67'9","reported_seq":262,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.159000+0000","last_change":"2024-02-23T00:42:57.705378+0000","last_active":"2024-02-23T00:46:01.159000+0000","last_peered":"2024-02-23T00:46:01.159000+0000","last_clean":"2024-02-23T00:46:01.159000+0000","last_became_active":"2024-02-23T00:39:16.786936+0000","last_became_peered":"2024-02-23T00:39:16.786936+0000","last_unstale":"2024-02-23T00:46:01.159000+0000","last_undegraded":"2024-02-23T00:46:01.159000+0000","last_fullsized":"2024-02-23T00:46:01.159000+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'9","last_scrub_stamp":"2024-02-23T00:42:57.705261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.705261+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":257,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.206324+0000","last_change":"2024-02-23T00:44:24.350861+0000","last_active":"2024-02-23T00:46:01.206324+0000","last_peered":"2024-02-23T00:46:01.206324+0000","last_clean":"2024-02-23T00:46:01.206324+0000","last_became_active":"2024-02-23T00:39:14.778455+0000","last_became_peered":"2024-02-23T00:39:14.778455+0000","last_unstale":"2024-02-23T00:46:01.206324+0000","last_undegraded":"2024-02-23T00:46:01.206324+0000","last_fullsized":"2024-02-23T00:46:01.206324+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:24.350797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:24.350797+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":250,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.157827+0000","last_change":"2024-02-23T00:44:37.105079+0000","last_active":"2024-02-23T00:46:01.157827+0000","last_peered":"2024-02-23T00:46:01.157827+0000","last_clean":"2024-02-23T00:46:01.157827+0000","last_became_active":"2024-02-23T00:39:18.799052+0000","last_became_peered":"2024-02-23T00:39:18.799052+0000","last_unstale":"2024-02-23T00:46:01.157827+0000","last_undegraded":"2024-02-23T00:46:01.157827+0000","last_fullsized":"2024-02-23T00:46:01.157827+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:37.105013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_clean_scrub_stamp":"2024-02-23T00:44:37.105013+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":248,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.299882+0000","last_change":"2024-02-23T00:46:00.558774+0000","last_active":"2024-02-23T00:46:01.299882+0000","last_peered":"2024-02-23T00:46:01.299882+0000","last_clean":"2024-02-23T00:46:01.299882+0000","last_became_active":"2024-02-23T00:39:20.802648+0000","last_became_peered":"2024-02-23T00:39:20.802648+0000","last_unstale":"2024-02-23T00:46:01.299882+0000","last_undegraded":"2024-02-23T00:46:01.299882+0000","last_fullsized":"2024-02-23T00:46:01.299882+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:00.558661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:46:00.558661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":27,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.204806+0000","last_change":"2024-02-23T00:45:45.158617+0000","last_active":"2024-02-23T00:46:01.204806+0000","last_peered":"2024-02-23T00:46:01.204806+0000","last_clean":"2024-02-23T00:46:01.204806+0000","last_became_active":"2024-02-23T00:45:45.156262+0000","last_became_peered":"2024-02-23T00:45:45.156262+0000","last_unstale":"2024-02-23T00:46:01.204806+0000","last_undegraded":"2024-02-23T00:46:01.204806+0000","last_fullsized":"2024-02-23T00:46:01.204806+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:15.352027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:15.352027+0000","last_clean_scrub_stamp":"2024-02-23T00:45:15.352027+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":195,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.302116+0000","last_change":"2024-02-23T00:44:23.543049+0000","last_active":"2024-02-23T00:46:01.302116+0000","last_peered":"2024-02-23T00:46:01.302116+0000","last_clean":"2024-02-23T00:46:01.302116+0000","last_became_active":"2024-02-23T00:42:58.835862+0000","last_became_peered":"2024-02-23T00:42:58.835862+0000","last_unstale":"2024-02-23T00:46:01.302116+0000","last_undegraded":"2024-02-23T00:46:01.302116+0000","last_fullsized":"2024-02-23T00:46:01.302116+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:23.542975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.542975+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"67'11","reported_seq":270,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.159140+0000","last_change":"2024-02-23T00:45:13.315510+0000","last_active":"2024-02-23T00:46:01.159140+0000","last_peered":"2024-02-23T00:46:01.159140+0000","last_clean":"2024-02-23T00:46:01.159140+0000","last_became_active":"2024-02-23T00:39:16.783896+0000","last_became_peered":"2024-02-23T00:39:16.783896+0000","last_unstale":"2024-02-23T00:46:01.159140+0000","last_undegraded":"2024-02-23T00:46:01.159140+0000","last_fullsized":"2024-02-23T00:46:01.159140+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'11","last_scrub_stamp":"2024-02-23T00:45:13.315434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:13.315434+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"202'2","reported_seq":261,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.161135+0000","last_change":"2024-02-23T00:44:34.720705+0000","last_active":"2024-02-23T00:46:01.161135+0000","last_peered":"2024-02-23T00:46:01.161135+0000","last_clean":"2024-02-23T00:46:01.161135+0000","last_became_active":"2024-02-23T00:39:14.775804+0000","last_became_peered":"2024-02-23T00:39:14.775804+0000","last_unstale":"2024-02-23T00:46:01.161135+0000","last_undegraded":"2024-02-23T00:46:01.161135+0000","last_fullsized":"2024-02-23T00:46:01.161135+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"202'2","last_scrub_stamp":"2024-02-23T00:44:34.720640+0000","last_deep_scrub":"202'2","last_deep_scrub_stamp":"2024-02-23T00:44:34.720640+0000","last_clean_scrub_stamp":"2024-02-23T00:44:34.720640+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":244,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.159095+0000","last_change":"2024-02-23T00:43:31.903157+0000","last_active":"2024-02-23T00:46:01.159095+0000","last_peered":"2024-02-23T00:46:01.159095+0000","last_clean":"2024-02-23T00:46:01.159095+0000","last_became_active":"2024-02-23T00:39:18.800382+0000","last_became_peered":"2024-02-23T00:39:18.800382+0000","last_unstale":"2024-02-23T00:46:01.159095+0000","last_undegraded":"2024-02-23T00:46:01.159095+0000","last_fullsized":"2024-02-23T00:46:01.159095+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:31.903031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:31.903031+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":248,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.158086+0000","last_change":"2024-02-23T00:44:23.152677+0000","last_active":"2024-02-23T00:46:01.158086+0000","last_peered":"2024-02-23T00:46:01.158086+0000","last_clean":"2024-02-23T00:46:01.158086+0000","last_became_active":"2024-02-23T00:39:20.799053+0000","last_became_peered":"2024-02-23T00:39:20.799053+0000","last_unstale":"2024-02-23T00:46:01.158086+0000","last_undegraded":"2024-02-23T00:46:01.158086+0000","last_fullsized":"2024-02-23T00:46:01.158086+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:23.152602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.152602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"110'2","reported_seq":32,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.298976+0000","last_change":"2024-02-23T00:45:45.162543+0000","last_active":"2024-02-23T00:46:01.298976+0000","last_peered":"2024-02-23T00:46:01.298976+0000","last_clean":"2024-02-23T00:46:01.298976+0000","last_became_active":"2024-02-23T00:45:45.144393+0000","last_became_peered":"2024-02-23T00:45:45.144393+0000","last_unstale":"2024-02-23T00:46:01.298976+0000","last_undegraded":"2024-02-23T00:46:01.298976+0000","last_fullsized":"2024-02-23T00:46:01.298976+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'2","last_scrub_stamp":"2024-02-23T00:45:09.917420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:09.917420+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":190,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.161048+0000","last_change":"2024-02-23T00:42:57.652742+0000","last_active":"2024-02-23T00:46:01.161048+0000","last_peered":"2024-02-23T00:46:01.161048+0000","last_clean":"2024-02-23T00:46:01.161048+0000","last_became_active":"2024-02-23T00:42:57.652438+0000","last_became_peered":"2024-02-23T00:42:57.652438+0000","last_unstale":"2024-02-23T00:46:01.161048+0000","last_undegraded":"2024-02-23T00:46:01.161048+0000","last_fullsized":"2024-02-23T00:46:01.161048+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"67'4","reported_seq":258,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.160001+0000","last_change":"2024-02-23T00:45:20.967211+0000","last_active":"2024-02-23T00:46:01.160001+0000","last_peered":"2024-02-23T00:46:01.160001+0000","last_clean":"2024-02-23T00:46:01.160001+0000","last_became_active":"2024-02-23T00:39:16.785830+0000","last_became_peered":"2024-02-23T00:39:16.785830+0000","last_unstale":"2024-02-23T00:46:01.160001+0000","last_undegraded":"2024-02-23T00:46:01.160001+0000","last_fullsized":"2024-02-23T00:46:01.160001+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'4","last_scrub_stamp":"2024-02-23T00:45:20.967095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:20.967095+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":253,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.171543+0000","last_change":"2024-02-23T00:43:38.947741+0000","last_active":"2024-02-23T00:46:00.171543+0000","last_peered":"2024-02-23T00:46:00.171543+0000","last_clean":"2024-02-23T00:46:00.171543+0000","last_became_active":"2024-02-23T00:39:14.783325+0000","last_became_peered":"2024-02-23T00:39:14.783325+0000","last_unstale":"2024-02-23T00:46:00.171543+0000","last_undegraded":"2024-02-23T00:46:00.171543+0000","last_fullsized":"2024-02-23T00:46:00.171543+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:38.947675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:38.947675+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":253,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.205877+0000","last_change":"2024-02-23T00:45:20.761547+0000","last_active":"2024-02-23T00:46:01.205877+0000","last_peered":"2024-02-23T00:46:01.205877+0000","last_clean":"2024-02-23T00:46:01.205877+0000","last_became_active":"2024-02-23T00:39:18.797287+0000","last_became_peered":"2024-02-23T00:39:18.797287+0000","last_unstale":"2024-02-23T00:46:01.205877+0000","last_undegraded":"2024-02-23T00:46:01.205877+0000","last_fullsized":"2024-02-23T00:46:01.205877+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:20.761440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:20.761440+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":244,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.163835+0000","last_change":"2024-02-23T00:45:10.741762+0000","last_active":"2024-02-23T00:46:00.163835+0000","last_peered":"2024-02-23T00:46:00.163835+0000","last_clean":"2024-02-23T00:46:00.163835+0000","last_became_active":"2024-02-23T00:39:20.795351+0000","last_became_peered":"2024-02-23T00:39:20.795351+0000","last_unstale":"2024-02-23T00:46:00.163835+0000","last_undegraded":"2024-02-23T00:46:00.163835+0000","last_fullsized":"2024-02-23T00:46:00.163835+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.741694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.741694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1b","version":"110'10","reported_seq":53,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.158769+0000","last_change":"2024-02-23T00:45:45.173671+0000","last_active":"2024-02-23T00:46:01.158769+0000","last_peered":"2024-02-23T00:46:01.158769+0000","last_clean":"2024-02-23T00:46:01.158769+0000","last_became_active":"2024-02-23T00:45:45.162718+0000","last_became_peered":"2024-02-23T00:45:45.162718+0000","last_unstale":"2024-02-23T00:46:01.158769+0000","last_undegraded":"2024-02-23T00:46:01.158769+0000","last_fullsized":"2024-02-23T00:46:01.158769+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'10","last_scrub_stamp":"2024-02-23T00:45:14.710075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.710075+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":193,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.171804+0000","last_change":"2024-02-23T00:45:56.706901+0000","last_active":"2024-02-23T00:46:00.171804+0000","last_peered":"2024-02-23T00:46:00.171804+0000","last_clean":"2024-02-23T00:46:00.171804+0000","last_became_active":"2024-02-23T00:42:58.741365+0000","last_became_peered":"2024-02-23T00:42:58.741365+0000","last_unstale":"2024-02-23T00:46:00.171804+0000","last_undegraded":"2024-02-23T00:46:00.171804+0000","last_fullsized":"2024-02-23T00:46:00.171804+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:56.706792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:56.706792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"68'17","reported_seq":286,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.158799+0000","last_change":"2024-02-23T00:43:43.997579+0000","last_active":"2024-02-23T00:46:01.158799+0000","last_peered":"2024-02-23T00:46:01.158799+0000","last_clean":"2024-02-23T00:46:01.158799+0000","last_became_active":"2024-02-23T00:39:16.857232+0000","last_became_peered":"2024-02-23T00:39:16.857232+0000","last_unstale":"2024-02-23T00:46:01.158799+0000","last_undegraded":"2024-02-23T00:46:01.158799+0000","last_fullsized":"2024-02-23T00:46:01.158799+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:43:43.997452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:43.997452+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":252,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.207072+0000","last_change":"2024-02-23T00:43:52.419206+0000","last_active":"2024-02-23T00:46:01.207072+0000","last_peered":"2024-02-23T00:46:01.207072+0000","last_clean":"2024-02-23T00:46:01.207072+0000","last_became_active":"2024-02-23T00:39:14.784092+0000","last_became_peered":"2024-02-23T00:39:14.784092+0000","last_unstale":"2024-02-23T00:46:01.207072+0000","last_undegraded":"2024-02-23T00:46:01.207072+0000","last_fullsized":"2024-02-23T00:46:01.207072+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:52.419100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:52.419100+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":246,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.164329+0000","last_change":"2024-02-23T00:44:25.797288+0000","last_active":"2024-02-23T00:46:00.164329+0000","last_peered":"2024-02-23T00:46:00.164329+0000","last_clean":"2024-02-23T00:46:00.164329+0000","last_became_active":"2024-02-23T00:39:18.791457+0000","last_became_peered":"2024-02-23T00:39:18.791457+0000","last_unstale":"2024-02-23T00:46:00.164329+0000","last_undegraded":"2024-02-23T00:46:00.164329+0000","last_fullsized":"2024-02-23T00:46:00.164329+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:25.797184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:25.797184+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":243,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.157994+0000","last_change":"2024-02-23T00:43:45.965636+0000","last_active":"2024-02-23T00:46:01.157994+0000","last_peered":"2024-02-23T00:46:01.157994+0000","last_clean":"2024-02-23T00:46:01.157994+0000","last_became_active":"2024-02-23T00:39:21.160376+0000","last_became_peered":"2024-02-23T00:39:21.160376+0000","last_unstale":"2024-02-23T00:46:01.157994+0000","last_undegraded":"2024-02-23T00:46:01.157994+0000","last_fullsized":"2024-02-23T00:46:01.157994+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:45.965570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.965570+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1a","version":"110'5","reported_seq":36,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.300100+0000","last_change":"2024-02-23T00:45:51.103803+0000","last_active":"2024-02-23T00:46:01.300100+0000","last_peered":"2024-02-23T00:46:01.300100+0000","last_clean":"2024-02-23T00:46:01.300100+0000","last_became_active":"2024-02-23T00:45:45.151529+0000","last_became_peered":"2024-02-23T00:45:45.151529+0000","last_unstale":"2024-02-23T00:46:01.300100+0000","last_undegraded":"2024-02-23T00:46:01.300100+0000","last_fullsized":"2024-02-23T00:46:01.300100+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'5","last_scrub_stamp":"2024-02-23T00:45:51.103691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:51.103691+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":194,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.301380+0000","last_change":"2024-02-23T00:45:23.378334+0000","last_active":"2024-02-23T00:46:01.301380+0000","last_peered":"2024-02-23T00:46:01.301380+0000","last_clean":"2024-02-23T00:46:01.301380+0000","last_became_active":"2024-02-23T00:42:58.702580+0000","last_became_peered":"2024-02-23T00:42:58.702580+0000","last_unstale":"2024-02-23T00:46:01.301380+0000","last_undegraded":"2024-02-23T00:46:01.301380+0000","last_fullsized":"2024-02-23T00:46:01.301380+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:23.378270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.378270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"68'16","reported_seq":282,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.302024+0000","last_change":"2024-02-23T00:45:04.672612+0000","last_active":"2024-02-23T00:46:01.302024+0000","last_peered":"2024-02-23T00:46:01.302024+0000","last_clean":"2024-02-23T00:46:01.302024+0000","last_became_active":"2024-02-23T00:39:16.858945+0000","last_became_peered":"2024-02-23T00:39:16.858945+0000","last_unstale":"2024-02-23T00:46:01.302024+0000","last_undegraded":"2024-02-23T00:46:01.302024+0000","last_fullsized":"2024-02-23T00:46:01.302024+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:45:04.672537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.672537+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":260,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.167881+0000","last_change":"2024-02-23T00:43:44.031862+0000","last_active":"2024-02-23T00:46:00.167881+0000","last_peered":"2024-02-23T00:46:00.167881+0000","last_clean":"2024-02-23T00:46:00.167881+0000","last_became_active":"2024-02-23T00:39:14.785363+0000","last_became_peered":"2024-02-23T00:39:14.785363+0000","last_unstale":"2024-02-23T00:46:00.167881+0000","last_undegraded":"2024-02-23T00:46:00.167881+0000","last_fullsized":"2024-02-23T00:46:00.167881+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"80'2","last_scrub_stamp":"2024-02-23T00:43:44.031793+0000","last_deep_scrub":"80'2","last_deep_scrub_stamp":"2024-02-23T00:40:15.942725+0000","last_clean_scrub_stamp":"2024-02-23T00:43:44.031793+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"93'9","reported_seq":341,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.208392+0000","last_change":"2024-02-23T00:43:50.393789+0000","last_active":"2024-02-23T00:46:01.208392+0000","last_peered":"2024-02-23T00:46:01.208392+0000","last_clean":"2024-02-23T00:46:01.208392+0000","last_became_active":"2024-02-23T00:39:18.796680+0000","last_became_peered":"2024-02-23T00:39:18.796680+0000","last_unstale":"2024-02-23T00:46:01.208392+0000","last_undegraded":"2024-02-23T00:46:01.208392+0000","last_fullsized":"2024-02-23T00:46:01.208392+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-02-23T00:43:50.393713+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-02-23T00:43:50.393713+0000","last_clean_scrub_stamp":"2024-02-23T00:43:50.393713+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":32,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.204889+0000","last_change":"2024-02-23T00:45:53.142552+0000","last_active":"2024-02-23T00:46:01.204889+0000","last_peered":"2024-02-23T00:46:01.204889+0000","last_clean":"2024-02-23T00:46:01.204889+0000","last_became_active":"2024-02-23T00:45:45.156323+0000","last_became_peered":"2024-02-23T00:45:45.156323+0000","last_unstale":"2024-02-23T00:46:01.204889+0000","last_undegraded":"2024-02-23T00:46:01.204889+0000","last_fullsized":"2024-02-23T00:46:01.204889+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:53.142473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:53.142473+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":195,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.300472+0000","last_change":"2024-02-23T00:45:58.613561+0000","last_active":"2024-02-23T00:46:01.300472+0000","last_peered":"2024-02-23T00:46:01.300472+0000","last_clean":"2024-02-23T00:46:01.300472+0000","last_became_active":"2024-02-23T00:42:58.850072+0000","last_became_peered":"2024-02-23T00:42:58.850072+0000","last_unstale":"2024-02-23T00:46:01.300472+0000","last_undegraded":"2024-02-23T00:46:01.300472+0000","last_fullsized":"2024-02-23T00:46:01.300472+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:58.613497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:58.613497+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"93'23","reported_seq":295,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.208482+0000","last_change":"2024-02-23T00:44:28.387225+0000","last_active":"2024-02-23T00:46:01.208482+0000","last_peered":"2024-02-23T00:46:01.208482+0000","last_clean":"2024-02-23T00:46:01.208482+0000","last_became_active":"2024-02-23T00:39:16.782660+0000","last_became_peered":"2024-02-23T00:39:16.782660+0000","last_unstale":"2024-02-23T00:46:01.208482+0000","last_undegraded":"2024-02-23T00:46:01.208482+0000","last_fullsized":"2024-02-23T00:46:01.208482+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'23","last_scrub_stamp":"2024-02-23T00:44:28.387155+0000","last_deep_scrub":"93'23","last_deep_scrub_stamp":"2024-02-23T00:41:35.954516+0000","last_clean_scrub_stamp":"2024-02-23T00:44:28.387155+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":254,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.158245+0000","last_change":"2024-02-23T00:44:21.157372+0000","last_active":"2024-02-23T00:46:01.158245+0000","last_peered":"2024-02-23T00:46:01.158245+0000","last_clean":"2024-02-23T00:46:01.158245+0000","last_became_active":"2024-02-23T00:39:14.781560+0000","last_became_peered":"2024-02-23T00:39:14.781560+0000","last_unstale":"2024-02-23T00:46:01.158245+0000","last_undegraded":"2024-02-23T00:46:01.158245+0000","last_fullsized":"2024-02-23T00:46:01.158245+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:21.157316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:21.157316+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":250,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.302700+0000","last_change":"2024-02-23T00:43:49.238770+0000","last_active":"2024-02-23T00:46:01.302700+0000","last_peered":"2024-02-23T00:46:01.302700+0000","last_clean":"2024-02-23T00:46:01.302700+0000","last_became_active":"2024-02-23T00:39:18.790602+0000","last_became_peered":"2024-02-23T00:39:18.790602+0000","last_unstale":"2024-02-23T00:46:01.302700+0000","last_undegraded":"2024-02-23T00:46:01.302700+0000","last_fullsized":"2024-02-23T00:46:01.302700+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:49.238704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:53.189881+0000","last_clean_scrub_stamp":"2024-02-23T00:43:49.238704+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.14","version":"110'2","reported_seq":31,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.160042+0000","last_change":"2024-02-23T00:45:45.250750+0000","last_active":"2024-02-23T00:46:01.160042+0000","last_peered":"2024-02-23T00:46:01.160042+0000","last_clean":"2024-02-23T00:46:01.160042+0000","last_became_active":"2024-02-23T00:45:45.161233+0000","last_became_peered":"2024-02-23T00:45:45.161233+0000","last_unstale":"2024-02-23T00:46:01.160042+0000","last_undegraded":"2024-02-23T00:46:01.160042+0000","last_fullsized":"2024-02-23T00:46:01.160042+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,3],"acting":[1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":195,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.158327+0000","last_change":"2024-02-23T00:45:15.933508+0000","last_active":"2024-02-23T00:46:01.158327+0000","last_peered":"2024-02-23T00:46:01.158327+0000","last_clean":"2024-02-23T00:46:01.158327+0000","last_became_active":"2024-02-23T00:42:58.774720+0000","last_became_peered":"2024-02-23T00:42:58.774720+0000","last_unstale":"2024-02-23T00:46:01.158327+0000","last_undegraded":"2024-02-23T00:46:01.158327+0000","last_fullsized":"2024-02-23T00:46:01.158327+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:15.933438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:15.933438+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"67'15","reported_seq":275,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.302202+0000","last_change":"2024-02-23T00:43:36.449809+0000","last_active":"2024-02-23T00:46:01.302202+0000","last_peered":"2024-02-23T00:46:01.302202+0000","last_clean":"2024-02-23T00:46:01.302202+0000","last_became_active":"2024-02-23T00:39:16.787138+0000","last_became_peered":"2024-02-23T00:39:16.787138+0000","last_unstale":"2024-02-23T00:46:01.302202+0000","last_undegraded":"2024-02-23T00:46:01.302202+0000","last_fullsized":"2024-02-23T00:46:01.302202+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:43:36.449687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:36.449687+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":252,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.206554+0000","last_change":"2024-02-23T00:43:45.362071+0000","last_active":"2024-02-23T00:46:01.206554+0000","last_peered":"2024-02-23T00:46:01.206554+0000","last_clean":"2024-02-23T00:46:01.206554+0000","last_became_active":"2024-02-23T00:39:14.782494+0000","last_became_peered":"2024-02-23T00:39:14.782494+0000","last_unstale":"2024-02-23T00:46:01.206554+0000","last_undegraded":"2024-02-23T00:46:01.206554+0000","last_fullsized":"2024-02-23T00:46:01.206554+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:45.361978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.361978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":249,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.161357+0000","last_change":"2024-02-23T00:45:03.615758+0000","last_active":"2024-02-23T00:46:01.161357+0000","last_peered":"2024-02-23T00:46:01.161357+0000","last_clean":"2024-02-23T00:46:01.161357+0000","last_became_active":"2024-02-23T00:39:18.801039+0000","last_became_peered":"2024-02-23T00:39:18.801039+0000","last_unstale":"2024-02-23T00:46:01.161357+0000","last_undegraded":"2024-02-23T00:46:01.161357+0000","last_fullsized":"2024-02-23T00:46:01.161357+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:03.615692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.641689+0000","last_clean_scrub_stamp":"2024-02-23T00:45:03.615692+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"110'5","reported_seq":34,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.159644+0000","last_change":"2024-02-23T00:45:45.166028+0000","last_active":"2024-02-23T00:46:01.159644+0000","last_peered":"2024-02-23T00:46:01.159644+0000","last_clean":"2024-02-23T00:46:01.159644+0000","last_became_active":"2024-02-23T00:45:45.154195+0000","last_became_peered":"2024-02-23T00:45:45.154195+0000","last_unstale":"2024-02-23T00:46:01.159644+0000","last_undegraded":"2024-02-23T00:46:01.159644+0000","last_fullsized":"2024-02-23T00:46:01.159644+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'5","last_scrub_stamp":"2024-02-23T00:45:14.317573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.317573+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":195,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.157708+0000","last_change":"2024-02-23T00:45:21.917072+0000","last_active":"2024-02-23T00:46:01.157708+0000","last_peered":"2024-02-23T00:46:01.157708+0000","last_clean":"2024-02-23T00:46:01.157708+0000","last_became_active":"2024-02-23T00:42:58.770759+0000","last_became_peered":"2024-02-23T00:42:58.770759+0000","last_unstale":"2024-02-23T00:46:01.157708+0000","last_undegraded":"2024-02-23T00:46:01.157708+0000","last_fullsized":"2024-02-23T00:46:01.157708+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:21.916996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:21.916996+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"68'18","reported_seq":290,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.158778+0000","last_change":"2024-02-23T00:43:51.079824+0000","last_active":"2024-02-23T00:46:01.158778+0000","last_peered":"2024-02-23T00:46:01.158778+0000","last_clean":"2024-02-23T00:46:01.158778+0000","last_became_active":"2024-02-23T00:39:16.782803+0000","last_became_peered":"2024-02-23T00:39:16.782803+0000","last_unstale":"2024-02-23T00:46:01.158778+0000","last_undegraded":"2024-02-23T00:46:01.158778+0000","last_fullsized":"2024-02-23T00:46:01.158778+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'18","last_scrub_stamp":"2024-02-23T00:43:51.079756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:51.079756+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":254,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.157678+0000","last_change":"2024-02-23T00:43:48.030906+0000","last_active":"2024-02-23T00:46:01.157678+0000","last_peered":"2024-02-23T00:46:01.157678+0000","last_clean":"2024-02-23T00:46:01.157678+0000","last_became_active":"2024-02-23T00:39:14.782163+0000","last_became_peered":"2024-02-23T00:39:14.782163+0000","last_unstale":"2024-02-23T00:46:01.157678+0000","last_undegraded":"2024-02-23T00:46:01.157678+0000","last_fullsized":"2024-02-23T00:46:01.157678+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:48.030846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.030846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":249,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.158908+0000","last_change":"2024-02-23T00:44:37.068899+0000","last_active":"2024-02-23T00:46:01.158908+0000","last_peered":"2024-02-23T00:46:01.158908+0000","last_clean":"2024-02-23T00:46:01.158908+0000","last_became_active":"2024-02-23T00:39:18.794903+0000","last_became_peered":"2024-02-23T00:39:18.794903+0000","last_unstale":"2024-02-23T00:46:01.158908+0000","last_undegraded":"2024-02-23T00:46:01.158908+0000","last_fullsized":"2024-02-23T00:46:01.158908+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:37.068834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_clean_scrub_stamp":"2024-02-23T00:44:37.068834+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"110'5","reported_seq":209,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.301085+0000","last_change":"2024-02-23T00:45:45.164183+0000","last_active":"2024-02-23T00:46:01.301085+0000","last_peered":"2024-02-23T00:46:01.301085+0000","last_clean":"2024-02-23T00:46:01.301085+0000","last_became_active":"2024-02-23T00:45:45.157467+0000","last_became_peered":"2024-02-23T00:45:45.157467+0000","last_unstale":"2024-02-23T00:46:01.301085+0000","last_undegraded":"2024-02-23T00:46:01.301085+0000","last_fullsized":"2024-02-23T00:46:01.301085+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":194,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.943661+0000","last_change":"2024-02-23T00:46:00.943625+0000","last_active":"2024-02-23T00:46:00.943661+0000","last_peered":"2024-02-23T00:46:00.943661+0000","last_clean":"2024-02-23T00:46:00.943661+0000","last_became_active":"2024-02-23T00:42:58.698258+0000","last_became_peered":"2024-02-23T00:42:58.698258+0000","last_unstale":"2024-02-23T00:46:00.943661+0000","last_undegraded":"2024-02-23T00:46:00.943661+0000","last_fullsized":"2024-02-23T00:46:00.943661+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:00.943545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:46:00.943545+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"95'27","reported_seq":303,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.158141+0000","last_change":"2024-02-23T00:43:52.124336+0000","last_active":"2024-02-23T00:46:01.158141+0000","last_peered":"2024-02-23T00:46:01.158141+0000","last_clean":"2024-02-23T00:46:01.158141+0000","last_became_active":"2024-02-23T00:39:16.786791+0000","last_became_peered":"2024-02-23T00:39:16.786791+0000","last_unstale":"2024-02-23T00:46:01.158141+0000","last_undegraded":"2024-02-23T00:46:01.158141+0000","last_fullsized":"2024-02-23T00:46:01.158141+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'27","last_scrub_stamp":"2024-02-23T00:43:52.124247+0000","last_deep_scrub":"95'27","last_deep_scrub_stamp":"2024-02-23T00:41:55.919895+0000","last_clean_scrub_stamp":"2024-02-23T00:43:52.124247+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":250,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.165118+0000","last_change":"2024-02-23T00:44:27.759571+0000","last_active":"2024-02-23T00:46:00.165118+0000","last_peered":"2024-02-23T00:46:00.165118+0000","last_clean":"2024-02-23T00:46:00.165118+0000","last_became_active":"2024-02-23T00:39:14.787148+0000","last_became_peered":"2024-02-23T00:39:14.787148+0000","last_unstale":"2024-02-23T00:46:00.165118+0000","last_undegraded":"2024-02-23T00:46:00.165118+0000","last_fullsized":"2024-02-23T00:46:00.165118+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:27.759499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:27.759499+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":249,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.169383+0000","last_change":"2024-02-23T00:45:10.090353+0000","last_active":"2024-02-23T00:46:00.169383+0000","last_peered":"2024-02-23T00:46:00.169383+0000","last_clean":"2024-02-23T00:46:00.169383+0000","last_became_active":"2024-02-23T00:39:18.796431+0000","last_became_peered":"2024-02-23T00:39:18.796431+0000","last_unstale":"2024-02-23T00:46:00.169383+0000","last_undegraded":"2024-02-23T00:46:00.169383+0000","last_fullsized":"2024-02-23T00:46:00.169383+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.090270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.090270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.11","version":"110'7","reported_seq":38,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.205742+0000","last_change":"2024-02-23T00:45:45.170820+0000","last_active":"2024-02-23T00:46:01.205742+0000","last_peered":"2024-02-23T00:46:01.205742+0000","last_clean":"2024-02-23T00:46:01.205742+0000","last_became_active":"2024-02-23T00:45:45.155700+0000","last_became_peered":"2024-02-23T00:45:45.155700+0000","last_unstale":"2024-02-23T00:46:01.205742+0000","last_undegraded":"2024-02-23T00:46:01.205742+0000","last_fullsized":"2024-02-23T00:46:01.205742+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'7","last_scrub_stamp":"2024-02-23T00:44:30.366391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:44:30.366391+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":194,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.171968+0000","last_change":"2024-02-23T00:45:51.985111+0000","last_active":"2024-02-23T00:46:00.171968+0000","last_peered":"2024-02-23T00:46:00.171968+0000","last_clean":"2024-02-23T00:46:00.171968+0000","last_became_active":"2024-02-23T00:42:58.832885+0000","last_became_peered":"2024-02-23T00:42:58.832885+0000","last_unstale":"2024-02-23T00:46:00.171968+0000","last_undegraded":"2024-02-23T00:46:00.171968+0000","last_fullsized":"2024-02-23T00:46:00.171968+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:51.985023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:51.985023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"82'15","reported_seq":280,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.159306+0000","last_change":"2024-02-23T00:44:35.095478+0000","last_active":"2024-02-23T00:46:01.159306+0000","last_peered":"2024-02-23T00:46:01.159306+0000","last_clean":"2024-02-23T00:46:01.159306+0000","last_became_active":"2024-02-23T00:39:16.782762+0000","last_became_peered":"2024-02-23T00:39:16.782762+0000","last_unstale":"2024-02-23T00:46:01.159306+0000","last_undegraded":"2024-02-23T00:46:01.159306+0000","last_fullsized":"2024-02-23T00:46:01.159306+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'15","last_scrub_stamp":"2024-02-23T00:44:35.095409+0000","last_deep_scrub":"82'15","last_deep_scrub_stamp":"2024-02-23T00:41:50.975683+0000","last_clean_scrub_stamp":"2024-02-23T00:44:35.095409+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":250,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.166727+0000","last_change":"2024-02-23T00:43:44.794995+0000","last_active":"2024-02-23T00:46:00.166727+0000","last_peered":"2024-02-23T00:46:00.166727+0000","last_clean":"2024-02-23T00:46:00.166727+0000","last_became_active":"2024-02-23T00:39:14.783469+0000","last_became_peered":"2024-02-23T00:39:14.783469+0000","last_unstale":"2024-02-23T00:46:00.166727+0000","last_undegraded":"2024-02-23T00:46:00.166727+0000","last_fullsized":"2024-02-23T00:46:00.166727+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:44.794928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:44.794928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":248,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.165668+0000","last_change":"2024-02-23T00:43:41.817455+0000","last_active":"2024-02-23T00:46:00.165668+0000","last_peered":"2024-02-23T00:46:00.165668+0000","last_clean":"2024-02-23T00:46:00.165668+0000","last_became_active":"2024-02-23T00:39:18.796952+0000","last_became_peered":"2024-02-23T00:39:18.796952+0000","last_unstale":"2024-02-23T00:46:00.165668+0000","last_undegraded":"2024-02-23T00:46:00.165668+0000","last_fullsized":"2024-02-23T00:46:00.165668+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:41.817387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:41.817387+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":27,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.298604+0000","last_change":"2024-02-23T00:45:45.160209+0000","last_active":"2024-02-23T00:46:01.298604+0000","last_peered":"2024-02-23T00:46:01.298604+0000","last_clean":"2024-02-23T00:46:01.298604+0000","last_became_active":"2024-02-23T00:45:45.159835+0000","last_became_peered":"2024-02-23T00:45:45.159835+0000","last_unstale":"2024-02-23T00:46:01.298604+0000","last_undegraded":"2024-02-23T00:46:01.298604+0000","last_fullsized":"2024-02-23T00:46:01.298604+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.215063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.215063+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":197,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.158754+0000","last_change":"2024-02-23T00:45:23.224771+0000","last_active":"2024-02-23T00:46:01.158754+0000","last_peered":"2024-02-23T00:46:01.158754+0000","last_clean":"2024-02-23T00:46:01.158754+0000","last_became_active":"2024-02-23T00:42:58.758147+0000","last_became_peered":"2024-02-23T00:42:58.758147+0000","last_unstale":"2024-02-23T00:46:01.158754+0000","last_undegraded":"2024-02-23T00:46:01.158754+0000","last_fullsized":"2024-02-23T00:46:01.158754+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:23.224663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:23.224663+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.224663+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"68'15","reported_seq":282,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.157876+0000","last_change":"2024-02-23T00:43:49.139235+0000","last_active":"2024-02-23T00:46:01.157876+0000","last_peered":"2024-02-23T00:46:01.157876+0000","last_clean":"2024-02-23T00:46:01.157876+0000","last_became_active":"2024-02-23T00:39:16.784606+0000","last_became_peered":"2024-02-23T00:39:16.784606+0000","last_unstale":"2024-02-23T00:46:01.157876+0000","last_undegraded":"2024-02-23T00:46:01.157876+0000","last_fullsized":"2024-02-23T00:46:01.157876+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:43:49.139156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:49.139156+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":254,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.161171+0000","last_change":"2024-02-23T00:44:32.170717+0000","last_active":"2024-02-23T00:46:01.161171+0000","last_peered":"2024-02-23T00:46:01.161171+0000","last_clean":"2024-02-23T00:46:01.161171+0000","last_became_active":"2024-02-23T00:39:14.785362+0000","last_became_peered":"2024-02-23T00:39:14.785362+0000","last_unstale":"2024-02-23T00:46:01.161171+0000","last_undegraded":"2024-02-23T00:46:01.161171+0000","last_fullsized":"2024-02-23T00:46:01.161171+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:32.170554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:32.170554+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":244,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.159632+0000","last_change":"2024-02-23T00:42:44.553722+0000","last_active":"2024-02-23T00:46:01.159632+0000","last_peered":"2024-02-23T00:46:01.159632+0000","last_clean":"2024-02-23T00:46:01.159632+0000","last_became_active":"2024-02-23T00:39:18.796312+0000","last_became_peered":"2024-02-23T00:39:18.796312+0000","last_unstale":"2024-02-23T00:46:01.159632+0000","last_undegraded":"2024-02-23T00:46:01.159632+0000","last_fullsized":"2024-02-23T00:46:01.159632+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:44.553621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:44.553621+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.13","version":"110'9","reported_seq":38,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.158360+0000","last_change":"2024-02-23T00:45:45.176912+0000","last_active":"2024-02-23T00:46:01.158360+0000","last_peered":"2024-02-23T00:46:01.158360+0000","last_clean":"2024-02-23T00:46:01.158360+0000","last_became_active":"2024-02-23T00:45:45.162350+0000","last_became_peered":"2024-02-23T00:45:45.162350+0000","last_unstale":"2024-02-23T00:46:01.158360+0000","last_undegraded":"2024-02-23T00:46:01.158360+0000","last_fullsized":"2024-02-23T00:46:01.158360+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'9","last_scrub_stamp":"2024-02-23T00:45:21.719900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:21.719900+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":195,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.301672+0000","last_change":"2024-02-23T00:45:09.641006+0000","last_active":"2024-02-23T00:46:01.301672+0000","last_peered":"2024-02-23T00:46:01.301672+0000","last_clean":"2024-02-23T00:46:01.301672+0000","last_became_active":"2024-02-23T00:42:58.838257+0000","last_became_peered":"2024-02-23T00:42:58.838257+0000","last_unstale":"2024-02-23T00:46:01.301672+0000","last_undegraded":"2024-02-23T00:46:01.301672+0000","last_fullsized":"2024-02-23T00:46:01.301672+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:09.640936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:09.640936+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"88'50","reported_seq":348,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.158892+0000","last_change":"2024-02-23T00:45:06.038488+0000","last_active":"2024-02-23T00:46:01.158892+0000","last_peered":"2024-02-23T00:46:01.158892+0000","last_clean":"2024-02-23T00:46:01.158892+0000","last_became_active":"2024-02-23T00:39:16.780144+0000","last_became_peered":"2024-02-23T00:39:16.780144+0000","last_unstale":"2024-02-23T00:46:01.158892+0000","last_undegraded":"2024-02-23T00:46:01.158892+0000","last_fullsized":"2024-02-23T00:46:01.158892+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'50","last_scrub_stamp":"2024-02-23T00:45:06.038374+0000","last_deep_scrub":"88'50","last_deep_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.038374+0000","log_size":50,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":254,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.299888+0000","last_change":"2024-02-23T00:45:20.573233+0000","last_active":"2024-02-23T00:46:01.299888+0000","last_peered":"2024-02-23T00:46:01.299888+0000","last_clean":"2024-02-23T00:46:01.299888+0000","last_became_active":"2024-02-23T00:39:14.782000+0000","last_became_peered":"2024-02-23T00:39:14.782000+0000","last_unstale":"2024-02-23T00:46:01.299888+0000","last_undegraded":"2024-02-23T00:46:01.299888+0000","last_fullsized":"2024-02-23T00:46:01.299888+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:20.573168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_clean_scrub_stamp":"2024-02-23T00:45:20.573168+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":244,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.158019+0000","last_change":"2024-02-23T00:43:48.126631+0000","last_active":"2024-02-23T00:46:01.158019+0000","last_peered":"2024-02-23T00:46:01.158019+0000","last_clean":"2024-02-23T00:46:01.158019+0000","last_became_active":"2024-02-23T00:39:18.799723+0000","last_became_peered":"2024-02-23T00:39:18.799723+0000","last_unstale":"2024-02-23T00:46:01.158019+0000","last_undegraded":"2024-02-23T00:46:01.158019+0000","last_fullsized":"2024-02-23T00:46:01.158019+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:48.126552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.126552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":27,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-02-23T00:46:01.299995+0000","last_change":"2024-02-23T00:45:45.157332+0000","last_active":"2024-02-23T00:46:01.299995+0000","last_peered":"2024-02-23T00:46:01.299995+0000","last_clean":"2024-02-23T00:46:01.299995+0000","last_became_active":"2024-02-23T00:45:45.156946+0000","last_became_peered":"2024-02-23T00:45:45.156946+0000","last_unstale":"2024-02-23T00:46:01.299995+0000","last_undegraded":"2024-02-23T00:46:01.299995+0000","last_fullsized":"2024-02-23T00:46:01.299995+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:24.942535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:24.942535+0000","last_clean_scrub_stamp":"2024-02-23T00:45:24.942535+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":194,"reported_epoch":288,"state":"active+clean","last_fresh":"2024-02-23T00:46:00.172256+0000","last_change":"2024-02-23T00:45:14.209403+0000","last_active":"2024-02-23T00:46:00.172256+0000","last_peered":"2024-02-23T00:46:00.172256+0000","last_clean":"2024-02-23T00:46:00.172256+0000","last_became_active":"2024-02-23T00:42:58.824573+0000","last_became_peered":"2024-02-23T00:42:58.824573+0000","last_unstale":"2024-02-23T00:46:00.172256+0000","last_undegraded":"2024-02-23T00:46:00.172256+0000","last_fullsized":"2024-02-23T00:46:00.172256+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.209308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.209308+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0],"acting":[5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":181,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":180,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":179,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":62,"num_pg":32,"stat_sum":{"num_bytes":1204,"num_objects":121,"num_object_clones":2,"num_object_copies":363,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":121,"num_whiteouts":0,"num_read":98,"num_read_kb":4,"num_write":542,"num_write_kb":537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1474560,"data_stored":3612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1787,"ondisk_log_size":1787,"up":96,"acting":96,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":22,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":66,"acting":66,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1080,"num_read_kb":805,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":651,"ondisk_log_size":651,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":399,"num_read_kb":394,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":8},{"poolid":15,"num_pg":28,"stat_sum":{"num_bytes":2048,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":196608,"data_stored":6144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":84,"acting":84,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234069,"num_pgs":98,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":302032,"kb_used_data":4800,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93443120,"statfs":{"total":95995035648,"available":95685754880,"internally_reserved":0,"allocated":4915200,"data_stored":2924992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.28000000000000003,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.747,"5min":4.9420000000000002,"15min":4.9420000000000002},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.53200000000000003,"15min":0.51600000000000001},"min":{"1min":0.32600000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0589999999999999,"5min":4.9729999999999999,"15min":4.9729999999999999},"last":0.51500000000000001}]},{"osd":1,"last update":"Fri Feb 23 00:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.77400000000000002,"15min":0.626},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":5.4539999999999997,"5min":10.786,"15min":10.786},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.78700000000000003,"15min":0.629},"min":{"1min":0.29399999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":11.885999999999999,"5min":11.885999999999999,"15min":11.885999999999999},"last":0.46400000000000002}]},{"osd":2,"last update":"Fri Feb 23 00:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59299999999999997,"15min":0.55900000000000005},"min":{"1min":0.35899999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":4.0449999999999999,"5min":5.0789999999999997,"15min":5.0789999999999997},"last":0.499},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.57399999999999995,"15min":0.54800000000000004},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":1.04,"5min":5.1280000000000001,"15min":5.1280000000000001},"last":0.57999999999999996}]},{"osd":3,"last update":"Fri Feb 23 00:45:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":0.80800000000000005,"15min":0.62},"min":{"1min":0.35499999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":15.712999999999999,"5min":15.712999999999999,"15min":15.712999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.71699999999999997,"15min":0.61799999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.004,"5min":9.4090000000000007,"15min":9.4090000000000007},"last":0.65000000000000002}]},{"osd":4,"last update":"Fri Feb 23 00:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.44500000000000001,"15min":0.436},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":2.0350000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.55100000000000005,"15min":0.48599999999999999},"min":{"1min":0.28599999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":11.236000000000001,"5min":11.236000000000001,"15min":11.236000000000001},"last":0.59299999999999997}]},{"osd":5,"last update":"Fri Feb 23 00:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.51700000000000002,"15min":0.49099999999999999},"min":{"1min":0.29699999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":5.4000000000000004,"5min":5.4000000000000004,"15min":5.4000000000000004},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.50800000000000001,"15min":0.48599999999999999},"min":{"1min":0.37,"5min":0.29299999999999998,"15min":0.26700000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.67800000000000005}]},{"osd":6,"last update":"Fri Feb 23 00:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51000000000000001,"15min":0.50700000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.629},{"interface":"front","average":{"1min":1.093,"5min":0.61799999999999999,"15min":0.54400000000000004},"min":{"1min":0.35599999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":13.395,"5min":13.395,"15min":13.395},"last":0.66200000000000003}]}]},{"osd":6,"up_from":47,"seq":201863463004,"num_pgs":95,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":314296,"kb_used_data":4780,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93430856,"statfs":{"total":95995035648,"available":95673196544,"internally_reserved":0,"allocated":4894720,"data_stored":2925787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.97299999999999998,"15min":0.70499999999999996},"min":{"1min":0.33500000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.65000000000000002,"5min":30.641999999999999,"15min":30.641999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.97499999999999998,"15min":0.71399999999999997},"min":{"1min":0.42999999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.71199999999999997,"5min":29.292000000000002,"15min":29.292000000000002},"last":0.36199999999999999}]},{"osd":1,"last update":"Fri Feb 23 00:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.98499999999999999,"15min":0.69599999999999995},"min":{"1min":0.40600000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.76700000000000002,"5min":28.186,"15min":28.186},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":1.48,"15min":0.88200000000000001},"min":{"1min":0.39700000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.67800000000000005,"5min":33.104999999999997,"15min":33.104999999999997},"last":0.45300000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":1.034,"15min":0.73399999999999999},"min":{"1min":0.38500000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.75600000000000001,"5min":19.960000000000001,"15min":19.960000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.84299999999999997,"15min":0.65500000000000003},"min":{"1min":0.38600000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.79300000000000004,"5min":8.5860000000000003,"15min":8.5860000000000003},"last":0.5}]},{"osd":3,"last update":"Fri Feb 23 00:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.75600000000000001,"15min":0.64000000000000001},"min":{"1min":0.439,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80000000000000004,"5min":19.562000000000001,"15min":19.562000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.84699999999999998,"15min":0.67900000000000005},"min":{"1min":0.35399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69999999999999996,"5min":14.199,"15min":14.199},"last":0.51600000000000001}]},{"osd":4,"last update":"Fri Feb 23 00:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.58299999999999996,"15min":0.46500000000000002},"min":{"1min":0.27700000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.70299999999999996,"5min":11.109999999999999,"15min":11.109999999999999},"last":0.44},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.752,"15min":0.53600000000000003},"min":{"1min":0.314,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.71099999999999997,"5min":16.734000000000002,"15min":16.734000000000002},"last":0.58699999999999997}]},{"osd":5,"last update":"Fri Feb 23 00:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.63700000000000001,"15min":0.48799999999999999},"min":{"1min":0.32400000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.745,"5min":14.154999999999999,"15min":14.154999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.63500000000000001,"15min":0.504},"min":{"1min":0.27900000000000003,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.67600000000000005,"5min":6.6100000000000003,"15min":6.6100000000000003},"last":0.48899999999999999}]},{"osd":7,"last update":"Fri Feb 23 00:45:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.58399999999999996,"15min":0.52000000000000002},"min":{"1min":0.29299999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.745,"5min":8.548,"15min":8.548},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.496,"5min":0.752,"15min":0.59399999999999997},"min":{"1min":0.33900000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.68899999999999995,"5min":15.618,"15min":15.618},"last":0.64100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574966,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":310168,"kb_used_data":4740,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93434984,"statfs":{"total":95995035648,"available":95677423616,"internally_reserved":0,"allocated":4853760,"data_stored":2926472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:45:43 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":1.2909999999999999,"15min":0.66400000000000003},"min":{"1min":0.221,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.48799999999999999,"5min":78.361000000000004,"15min":78.361000000000004},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.57199999999999995,"15min":0.437},"min":{"1min":0.245,"5min":0.218,"15min":0.191},"max":{"1min":0.41199999999999998,"5min":5.8849999999999998,"15min":5.8849999999999998},"last":0.45300000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:45:03 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":1.1850000000000001,"15min":0.65900000000000003},"min":{"1min":0.247,"5min":0.219,"15min":0.188},"max":{"1min":0.63,"5min":48.606000000000002,"15min":48.606000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.88300000000000001,"15min":0.58599999999999997},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.249},"max":{"1min":0.82599999999999996,"5min":23.911000000000001,"15min":23.911000000000001},"last":0.37}]},{"osd":3,"last update":"Fri Feb 23 00:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.89400000000000002,"15min":0.624},"min":{"1min":0.28000000000000003,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.56100000000000005,"5min":35.244999999999997,"15min":35.244999999999997},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.38,"5min":1.1299999999999999,"15min":0.68200000000000005},"min":{"1min":0.28599999999999998,"5min":0.253,"15min":0.248},"max":{"1min":0.60299999999999998,"5min":56.853999999999999,"15min":56.853999999999999},"last":0.39100000000000001}]},{"osd":4,"last update":"Fri Feb 23 00:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":1.2310000000000001,"15min":0.81699999999999995},"min":{"1min":0.34599999999999997,"5min":0.33700000000000002,"15min":0.29299999999999998},"max":{"1min":0.82399999999999995,"5min":37.646999999999998,"15min":37.646999999999998},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.86599999999999999,"15min":0.69699999999999995},"min":{"1min":0.375,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80700000000000005,"5min":9.6859999999999999,"15min":9.6859999999999999},"last":0.61699999999999999}]},{"osd":5,"last update":"Fri Feb 23 00:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":1.224,"15min":0.84599999999999997},"min":{"1min":0.41099999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83399999999999996,"5min":48.792000000000002,"15min":48.792000000000002},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.83699999999999997,"15min":0.755},"min":{"1min":0.45100000000000001,"5min":0.38100000000000001,"15min":0.34200000000000003},"max":{"1min":0.76500000000000001,"5min":11.872999999999999,"15min":11.872999999999999},"last":0.59999999999999998}]},{"osd":6,"last update":"Fri Feb 23 00:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":1.1890000000000001,"15min":0.82299999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.79400000000000004,"5min":40.741,"15min":40.741},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.77200000000000002,"15min":0.68999999999999995},"min":{"1min":0.41999999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.872,"5min":6.0069999999999997,"15min":6.0069999999999997},"last":0.65400000000000003}]},{"osd":7,"last update":"Fri Feb 23 00:45:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.91000000000000003,"15min":0.71899999999999997},"min":{"1min":0.42499999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82799999999999996,"5min":20.225000000000001,"15min":20.225000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.94199999999999995,"15min":0.72799999999999998},"min":{"1min":0.42799999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85999999999999999,"5min":20.254999999999999,"15min":20.254999999999999},"last":0.57299999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738492,"num_pgs":90,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":314264,"kb_used_data":4744,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93430888,"statfs":{"total":95995035648,"available":95673229312,"internally_reserved":0,"allocated":4857856,"data_stored":2925675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 23 00:45:44 2024","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":1.089,"15min":0.59199999999999997},"min":{"1min":0.22600000000000001,"5min":0.223,"15min":0.19500000000000001},"max":{"1min":0.60899999999999999,"5min":28.856999999999999,"15min":28.856999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":1.026,"5min":1.1000000000000001,"15min":0.60699999999999998},"min":{"1min":0.25600000000000001,"5min":0.218,"15min":0.20799999999999999},"max":{"1min":13.162000000000001,"5min":20.577000000000002,"15min":20.577000000000002},"last":0.32000000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:45:06 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.026,"15min":0.60399999999999998},"min":{"1min":0.308,"5min":0.26000000000000001,"15min":0.20499999999999999},"max":{"1min":13.67,"5min":13.863,"15min":13.863},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.88400000000000001,"15min":0.54900000000000004},"min":{"1min":0.25700000000000001,"5min":0.24299999999999999,"15min":0.19800000000000001},"max":{"1min":0.60099999999999998,"5min":10.455,"15min":10.455},"last":0.29699999999999999}]},{"osd":3,"last update":"Fri Feb 23 00:45:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.5109999999999999,"15min":0.80300000000000005},"min":{"1min":0.27700000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":12.566000000000001,"5min":35.566000000000003,"15min":35.566000000000003},"last":0.28299999999999997},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.3660000000000001,"15min":0.79100000000000004},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":12.727,"5min":20.702999999999999,"15min":20.702999999999999},"last":0.433}]},{"osd":4,"last update":"Fri Feb 23 00:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":1.0409999999999999,"15min":0.80300000000000005},"min":{"1min":0.36399999999999999,"5min":0.36199999999999999,"15min":0.34699999999999998},"max":{"1min":0.753,"5min":10.584,"15min":10.584},"last":0.375},{"interface":"front","average":{"1min":1.258,"5min":1.032,"15min":0.82999999999999996},"min":{"1min":0.38900000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":13.198,"5min":13.198,"15min":13.198},"last":0.48299999999999998}]},{"osd":5,"last update":"Fri Feb 23 00:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":1.014,"15min":0.74399999999999999},"min":{"1min":0.37,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88200000000000001,"5min":10.397,"15min":10.397},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":1.079,"15min":0.80900000000000005},"min":{"1min":0.44700000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86299999999999999,"5min":17.297000000000001,"15min":17.297000000000001},"last":0.52400000000000002}]},{"osd":6,"last update":"Fri Feb 23 00:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":1.1080000000000001,"15min":0.79900000000000004},"min":{"1min":0.44400000000000001,"5min":0.42299999999999999,"15min":0.38400000000000001},"max":{"1min":0.85799999999999998,"5min":17.373999999999999,"15min":17.373999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.0409999999999999,"15min":0.77000000000000002},"min":{"1min":0.42199999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":13.973000000000001,"5min":13.973000000000001,"15min":13.973000000000001},"last":0.58899999999999997}]},{"osd":7,"last update":"Fri Feb 23 00:45:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":1.081,"15min":0.81299999999999994},"min":{"1min":0.48199999999999998,"5min":0.41699999999999998,"15min":0.39400000000000002},"max":{"1min":0.82599999999999996,"5min":17.399999999999999,"15min":17.399999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":1.494,"5min":1.0640000000000001,"15min":0.78900000000000003},"min":{"1min":0.45400000000000001,"5min":0.433,"15min":0.43099999999999999},"max":{"1min":16.427,"5min":16.427,"15min":16.427},"last":0.56599999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411441,"num_pgs":85,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":310268,"kb_used_data":4848,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93434884,"statfs":{"total":95995035648,"available":95677321216,"internally_reserved":0,"allocated":4964352,"data_stored":2925514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:45:02 2024","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":0.70199999999999996,"15min":0.48699999999999999},"min":{"1min":0.20499999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":13.138999999999999,"5min":13.657,"15min":13.657},"last":0.33700000000000002},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2649999999999999,"15min":0.68000000000000005},"min":{"1min":0.25700000000000001,"5min":0.246,"15min":0.23599999999999999},"max":{"1min":12.705,"5min":42.343000000000004,"15min":42.343000000000004},"last":0.36599999999999999}]},{"osd":1,"last update":"Fri Feb 23 00:45:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":0.96999999999999997,"15min":0.57499999999999996},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":8.8569999999999993,"5min":18.952000000000002,"15min":18.952000000000002},"last":0.28399999999999997},{"interface":"front","average":{"1min":1.6419999999999999,"5min":0.76800000000000002,"15min":0.52300000000000002},"min":{"1min":0.26100000000000001,"5min":0.23999999999999999,"15min":0.23599999999999999},"max":{"1min":15.391,"5min":15.391,"15min":15.391},"last":0.27400000000000002}]},{"osd":3,"last update":"Fri Feb 23 00:45:35 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":1.1779999999999999,"15min":0.71299999999999997},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.47499999999999998,"5min":25.635999999999999,"15min":25.635999999999999},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.41799999999999998,"5min":1.002,"15min":0.64600000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.0840000000000001,"5min":19.960000000000001,"15min":19.960000000000001},"last":0.34499999999999997}]},{"osd":4,"last update":"Fri Feb 23 00:45:02 2024","interfaces":[{"interface":"back","average":{"1min":2.1920000000000002,"5min":0.998,"15min":0.73099999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":13.673,"5min":13.673,"15min":13.673},"last":0.44900000000000001},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.044,"15min":0.73499999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.33900000000000002},"max":{"1min":8.7870000000000008,"5min":10.103999999999999,"15min":10.103999999999999},"last":0.379}]},{"osd":5,"last update":"Fri Feb 23 00:45:23 2024","interfaces":[{"interface":"back","average":{"1min":1.784,"5min":0.90600000000000003,"15min":0.69099999999999995},"min":{"1min":0.40699999999999997,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":15.612,"5min":15.612,"15min":15.612},"last":0.498},{"interface":"front","average":{"1min":1.335,"5min":0.88,"15min":0.70299999999999996},"min":{"1min":0.376,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":13.632999999999999,"5min":13.632999999999999,"15min":13.632999999999999},"last":0.54700000000000004}]},{"osd":6,"last update":"Fri Feb 23 00:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.88500000000000001,"15min":0.71099999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.5249999999999999,"5min":11.544,"15min":11.544},"last":0.505},{"interface":"front","average":{"1min":0.84599999999999997,"5min":1.05,"15min":0.749},"min":{"1min":0.45400000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":4.1909999999999998,"5min":10.132999999999999,"15min":10.132999999999999},"last":0.41899999999999998}]},{"osd":7,"last update":"Fri Feb 23 00:45:15 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":0.879,"15min":0.74299999999999999},"min":{"1min":0.40600000000000003,"5min":0.377,"15min":0.377},"max":{"1min":11.599,"5min":11.599,"15min":11.599},"last":0.56599999999999995},{"interface":"front","average":{"1min":1.494,"5min":1.097,"15min":0.80000000000000004},"min":{"1min":0.42399999999999999,"5min":0.374,"15min":0.374},"max":{"1min":8.7469999999999999,"5min":10.148999999999999,"15min":10.148999999999999},"last":0.51800000000000002}]}]},{"osd":3,"up_from":26,"seq":111669149802,"num_pgs":115,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":314332,"kb_used_data":4820,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93430820,"statfs":{"total":95995035648,"available":95673159680,"internally_reserved":0,"allocated":4935680,"data_stored":2927163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:45:35 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.746,"15min":0.53900000000000003},"min":{"1min":0.22800000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.56699999999999995,"5min":12.212999999999999,"15min":12.212999999999999},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.88900000000000001,"15min":0.59899999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.56899999999999995,"5min":11.714,"15min":11.714},"last":1.081}]},{"osd":1,"last update":"Fri Feb 23 00:45:35 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.871,"15min":0.60799999999999998},"min":{"1min":0.29299999999999998,"5min":0.23400000000000001,"15min":0.22900000000000001},"max":{"1min":0.61299999999999999,"5min":11.742000000000001,"15min":11.742000000000001},"last":6.851},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.78200000000000003,"15min":0.56100000000000005},"min":{"1min":0.223,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":1.8300000000000001,"5min":6.383,"15min":6.383},"last":8.2270000000000003}]},{"osd":2,"last update":"Fri Feb 23 00:45:35 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.95699999999999996,"15min":0.64900000000000002},"min":{"1min":0.26000000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.869,"5min":13.359999999999999,"15min":13.359999999999999},"last":8.0269999999999992},{"interface":"front","average":{"1min":0.374,"5min":1.002,"15min":0.65500000000000003},"min":{"1min":0.27500000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54400000000000004,"5min":21.122,"15min":21.122},"last":1.425}]},{"osd":4,"last update":"Fri Feb 23 00:45:58 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.083,"15min":0.75900000000000001},"min":{"1min":0.36399999999999999,"5min":0.317,"15min":0.317},"max":{"1min":11.558,"5min":11.644,"15min":11.644},"last":1.2070000000000001},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.93600000000000005,"15min":0.70399999999999996},"min":{"1min":0.40400000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":6.1589999999999998,"5min":6.2549999999999999,"15min":6.2549999999999999},"last":12.430999999999999}]},{"osd":5,"last update":"Fri Feb 23 00:45:23 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.76300000000000001,"15min":0.65900000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72999999999999998,"5min":5.6360000000000001,"15min":5.6360000000000001},"last":2.125},{"interface":"front","average":{"1min":0.57399999999999995,"5min":1.016,"15min":0.75600000000000001},"min":{"1min":0.40699999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79300000000000004,"5min":11.597,"15min":11.597},"last":3.4340000000000002}]},{"osd":6,"last update":"Fri Feb 23 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.90500000000000003,"15min":0.70599999999999996},"min":{"1min":0.39000000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.0539999999999998,"5min":6.343,"15min":6.343},"last":6.9420000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.80000000000000004,"15min":0.68200000000000005},"min":{"1min":0.442,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.71699999999999997,"5min":5.6680000000000001,"15min":5.6680000000000001},"last":1.647}]},{"osd":7,"last update":"Fri Feb 23 00:45:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":1.0589999999999999,"15min":0.77800000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.39000000000000001},"max":{"1min":2.6040000000000001,"5min":11.67,"15min":11.67},"last":3.4990000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.95499999999999996,"15min":0.75700000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":2.6179999999999999,"5min":6.3010000000000002,"15min":6.3010000000000002},"last":7.9800000000000004}]}]},{"osd":4,"up_from":32,"seq":137438953574,"num_pgs":95,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":314316,"kb_used_data":4796,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93430836,"statfs":{"total":95995035648,"available":95673176064,"internally_reserved":0,"allocated":4911104,"data_stored":2925538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:44:59 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.52100000000000002,"15min":0.56100000000000005},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.59499999999999997,"5min":3.5219999999999998,"15min":3.5219999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.66900000000000004,"15min":0.61399999999999999},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.64100000000000001,"5min":8.6140000000000008,"15min":8.6140000000000008},"last":0.52900000000000003}]},{"osd":1,"last update":"Fri Feb 23 00:44:59 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.66700000000000004,"15min":0.61299999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.27800000000000002},"max":{"1min":0.69799999999999995,"5min":9.6110000000000007,"15min":9.6110000000000007},"last":0.499},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.66300000000000003,"15min":0.60499999999999998},"min":{"1min":0.33500000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.67400000000000004,"5min":6.6950000000000003,"15min":6.6950000000000003},"last":0.60599999999999998}]},{"osd":2,"last update":"Fri Feb 23 00:44:59 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.624,"15min":0.59399999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64800000000000002,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68700000000000006,"5min":5.8440000000000003,"15min":5.8440000000000003},"last":0.50600000000000001}]},{"osd":3,"last update":"Fri Feb 23 00:44:59 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.65800000000000003,"15min":0.60599999999999998},"min":{"1min":0.34399999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65900000000000003,"5min":6.5540000000000003,"15min":6.5540000000000003},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.66200000000000003,"15min":0.60899999999999999},"min":{"1min":0.32700000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73699999999999999,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":0.54300000000000004}]},{"osd":5,"last update":"Fri Feb 23 00:45:23 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.41899999999999998,"15min":0.38300000000000001},"min":{"1min":0.25600000000000001,"5min":0.251,"15min":0.246},"max":{"1min":0.64700000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.48699999999999999,"15min":0.435},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.28899999999999998},"max":{"1min":0.71599999999999997,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.56299999999999994}]},{"osd":6,"last update":"Fri Feb 23 00:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.45300000000000001,"15min":0.41699999999999998},"min":{"1min":0.28699999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.68500000000000005,"5min":0.75,"15min":0.75},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45500000000000002,"15min":0.41699999999999998},"min":{"1min":0.27000000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.67100000000000004,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.42799999999999999}]},{"osd":7,"last update":"Fri Feb 23 00:45:17 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30599999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.67100000000000004,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.70599999999999996,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.55500000000000005}]}]},{"osd":5,"up_from":39,"seq":167503724641,"num_pgs":83,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":314252,"kb_used_data":4736,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93430900,"statfs":{"total":95995035648,"available":95673241600,"internally_reserved":0,"allocated":4849664,"data_stored":2926510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.78800000000000003,"15min":0.60199999999999998},"min":{"1min":0.28299999999999997,"5min":0.246,"15min":0.246},"max":{"1min":0.67900000000000005,"5min":23.881,"15min":23.881},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.68200000000000005,"15min":0.57899999999999996},"min":{"1min":0.28999999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.57799999999999996,"5min":5.4649999999999999,"15min":5.4649999999999999},"last":0.42599999999999999}]},{"osd":1,"last update":"Fri Feb 23 00:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.88600000000000001,"15min":0.64800000000000002},"min":{"1min":0.30199999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.74099999999999999,"5min":18.175999999999998,"15min":18.175999999999998},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.78300000000000003,"15min":0.624},"min":{"1min":0.35699999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.60299999999999998,"5min":9.0950000000000006,"15min":9.0950000000000006},"last":0.56999999999999995}]},{"osd":2,"last update":"Fri Feb 23 00:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":1.0169999999999999,"15min":0.70599999999999996},"min":{"1min":0.32600000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.59799999999999998,"5min":22.282,"15min":22.282},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.49199999999999999,"5min":1.0489999999999999,"15min":0.70899999999999996},"min":{"1min":0.33200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.63700000000000001,"5min":23.925000000000001,"15min":23.925000000000001},"last":0.53600000000000003}]},{"osd":3,"last update":"Fri Feb 23 00:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.77700000000000002,"15min":0.61399999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.63700000000000001,"5min":6.5730000000000004,"15min":6.5730000000000004},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.79900000000000004,"15min":0.64000000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33500000000000002},"max":{"1min":0.64800000000000002,"5min":8.1760000000000002,"15min":8.1760000000000002},"last":0.66100000000000003}]},{"osd":4,"last update":"Fri Feb 23 00:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.42899999999999999,"15min":0.39600000000000002},"min":{"1min":0.26400000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.57899999999999996,"5min":0.68999999999999995,"15min":0.70399999999999996},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.51800000000000002,"15min":0.42899999999999999},"min":{"1min":0.30199999999999999,"5min":0.27500000000000002,"15min":0.27300000000000002},"max":{"1min":0.52500000000000002,"5min":3.879,"15min":3.879},"last":0.63100000000000001}]},{"osd":6,"last update":"Fri Feb 23 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.48899999999999999,"15min":0.44600000000000001},"min":{"1min":0.30599999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70999999999999996,"5min":1.736,"15min":1.736},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.49199999999999999,"15min":0.45900000000000002},"min":{"1min":0.311,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73099999999999998,"5min":0.871,"15min":0.871},"last":0.69699999999999995}]},{"osd":7,"last update":"Fri Feb 23 00:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.57799999999999996,"15min":0.53200000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.71299999999999997,"5min":7.0300000000000002,"15min":7.0300000000000002},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.57799999999999996,"15min":0.53100000000000003},"min":{"1min":0.33400000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.58499999999999996,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":0.68999999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":172032,"data_stored":417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":192512,"data_stored":485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":192512,"data_stored":484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":229376,"data_stored":550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-23T00:46:05.739 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:05.740 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:46:05.740 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:46:05.841 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:46:05.869 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:06.269 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:04.474911+0000 osd.1 (osd.1) 139 : cluster [DBG] 2.d scrub starts 2024-02-23T00:46:06.269 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:04.476148+0000 osd.1 (osd.1) 140 : cluster [DBG] 2.d scrub ok 2024-02-23T00:46:06.269 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:04.941000+0000 osd.2 (osd.2) 133 : cluster [DBG] 2.e scrub starts 2024-02-23T00:46:06.269 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:04.942086+0000 osd.2 (osd.2) 134 : cluster [DBG] 2.e scrub ok 2024-02-23T00:46:06.269 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:04.997095+0000 osd.5 (osd.5) 129 : cluster [DBG] 62.1d scrub starts 2024-02-23T00:46:06.269 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:05.000607+0000 osd.5 (osd.5) 130 : cluster [DBG] 62.1d scrub ok 2024-02-23T00:46:06.269 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:05.143279+0000 mon.a (mon.0) 1972 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:46:06.269 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:05.143380+0000 mon.a (mon.0) 1973 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi047-82386-50","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:46:06.269 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:05.143557+0000 mon.a (mon.0) 1974 : cluster [DBG] osdmap e293: 8 total, 8 up, 8 in 2024-02-23T00:46:06.269 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:05.173077+0000 mon.a (mon.0) 1975 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:46:06.270 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:05.250871+0000 mgr.y (mgr.14517) 298 : cluster [DBG] pgmap v415: 286 pgs: 72 unknown, 1 active+clean+snaptrim, 1 peering, 212 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 3 op/s; 31 B/s, 0 objects/s recovering 2024-02-23T00:46:06.270 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:04.474911+0000 osd.1 (osd.1) 139 : cluster [DBG] 2.d scrub starts 2024-02-23T00:46:06.270 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:04.476148+0000 osd.1 (osd.1) 140 : cluster [DBG] 2.d scrub ok 2024-02-23T00:46:06.270 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:04.941000+0000 osd.2 (osd.2) 133 : cluster [DBG] 2.e scrub starts 2024-02-23T00:46:06.270 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:04.942086+0000 osd.2 (osd.2) 134 : cluster [DBG] 2.e scrub ok 2024-02-23T00:46:06.270 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:04.997095+0000 osd.5 (osd.5) 129 : cluster [DBG] 62.1d scrub starts 2024-02-23T00:46:06.270 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:05.000607+0000 osd.5 (osd.5) 130 : cluster [DBG] 62.1d scrub ok 2024-02-23T00:46:06.270 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:05.143279+0000 mon.a (mon.0) 1972 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:46:06.270 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:05.143380+0000 mon.a (mon.0) 1973 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi047-82386-50","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:46:06.270 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:05.143557+0000 mon.a (mon.0) 1974 : cluster [DBG] osdmap e293: 8 total, 8 up, 8 in 2024-02-23T00:46:06.270 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:05.173077+0000 mon.a (mon.0) 1975 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:46:06.270 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:05.250871+0000 mgr.y (mgr.14517) 298 : cluster [DBG] pgmap v415: 286 pgs: 72 unknown, 1 active+clean+snaptrim, 1 peering, 212 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 3 op/s; 31 B/s, 0 objects/s recovering 2024-02-23T00:46:06.305 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:46:06.323 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:46:06.328 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:06.329 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:46:06.429 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:46:06.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:04.474911+0000 osd.1 (osd.1) 139 : cluster [DBG] 2.d scrub starts 2024-02-23T00:46:06.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:04.476148+0000 osd.1 (osd.1) 140 : cluster [DBG] 2.d scrub ok 2024-02-23T00:46:06.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:04.941000+0000 osd.2 (osd.2) 133 : cluster [DBG] 2.e scrub starts 2024-02-23T00:46:06.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:04.942086+0000 osd.2 (osd.2) 134 : cluster [DBG] 2.e scrub ok 2024-02-23T00:46:06.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:04.997095+0000 osd.5 (osd.5) 129 : cluster [DBG] 62.1d scrub starts 2024-02-23T00:46:06.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:05.000607+0000 osd.5 (osd.5) 130 : cluster [DBG] 62.1d scrub ok 2024-02-23T00:46:06.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:05.143279+0000 mon.a (mon.0) 1972 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:46:06.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:05.143380+0000 mon.a (mon.0) 1973 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi047-82386-50","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:46:06.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:05.143557+0000 mon.a (mon.0) 1974 : cluster [DBG] osdmap e293: 8 total, 8 up, 8 in 2024-02-23T00:46:06.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:05.173077+0000 mon.a (mon.0) 1975 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:46:06.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:05.250871+0000 mgr.y (mgr.14517) 298 : cluster [DBG] pgmap v415: 286 pgs: 72 unknown, 1 active+clean+snaptrim, 1 peering, 212 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 3 op/s; 31 B/s, 0 objects/s recovering 2024-02-23T00:46:06.461 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:06.481 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:46:06.484 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:06.484 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:46:06.585 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:46:06.614 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:06.633 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:46:06.636 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:06.637 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:46:06.737 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:46:06.765 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:06.784 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:46:06.786 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:06.787 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:46:06.888 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:46:06.914 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:06.933 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:46:06.935 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:06.936 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:46:07.036 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:46:07.069 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:07.156 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:05.817133+0000 osd.7 (osd.7) 153 : cluster [DBG] 62.8 scrub starts 2024-02-23T00:46:07.157 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:05.820642+0000 osd.7 (osd.7) 154 : cluster [DBG] 62.8 scrub ok 2024-02-23T00:46:07.157 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:05.909299+0000 osd.2 (osd.2) 135 : cluster [DBG] 2.c scrub starts 2024-02-23T00:46:07.157 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:05.910321+0000 osd.2 (osd.2) 136 : cluster [DBG] 2.c scrub ok 2024-02-23T00:46:07.157 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:06.149306+0000 mon.a (mon.0) 1976 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:46:07.157 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:06.149412+0000 mon.a (mon.0) 1977 : cluster [DBG] osdmap e294: 8 total, 8 up, 8 in 2024-02-23T00:46:07.157 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:06.150632+0000 mon.c (mon.2) 219 : audit [INF] from='client.? 172.21.15.47:0/2639020958' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi047-82378-40"}]: dispatch 2024-02-23T00:46:07.157 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:06.152780+0000 mon.a (mon.0) 1978 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-35"}]: dispatch 2024-02-23T00:46:07.157 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:06.153085+0000 mon.a (mon.0) 1979 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi047-82378-40"}]: dispatch 2024-02-23T00:46:07.157 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:06.173740+0000 mon.c (mon.2) 220 : audit [INF] from='client.? 172.21.15.47:0/2123205582' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi047-82386-51"}]: dispatch 2024-02-23T00:46:07.157 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:06.174072+0000 mon.a (mon.0) 1980 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi047-82386-51"}]: dispatch 2024-02-23T00:46:07.157 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:06.174729+0000 mon.c (mon.2) 221 : audit [INF] from='client.? 172.21.15.47:0/2123205582' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi047-82386-51"}]: dispatch 2024-02-23T00:46:07.157 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:06.174976+0000 mon.a (mon.0) 1981 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi047-82386-51"}]: dispatch 2024-02-23T00:46:07.158 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:06.175379+0000 mon.c (mon.2) 222 : audit [INF] from='client.? 172.21.15.47:0/2123205582' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi047-82386-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:07.158 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:06.175635+0000 mon.a (mon.0) 1982 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi047-82386-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:07.187 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:46:07.187 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 20, 2024-02-23T00:46:07.187 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 600, 2024-02-23T00:46:07.187 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [ 2024-02-23T00:46:07.187 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.188 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(client.15087.0:1 29.1a e111/109 29:5c6776db:::0.obj:head v 111'13, mlcod=110'11)", 2024-02-23T00:46:07.188 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:42:59.769352+0000", 2024-02-23T00:46:07.188 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 187.282163209, 2024-02-23T00:46:07.188 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.026973831, 2024-02-23T00:46:07.188 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:46:07.188 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:07.188 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:46:07.188 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.188 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:46:07.188 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.769352+0000", 2024-02-23T00:46:07.188 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:46:07.188 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.189 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.189 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:46:07.189 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.769352+0000", 2024-02-23T00:46:07.189 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:46:07.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.189 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.189 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:46:07.189 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.769352+0000", 2024-02-23T00:46:07.189 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5441999999999999e-05 2024-02-23T00:46:07.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.189 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.190 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:46:07.190 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.769367+0000", 2024-02-23T00:46:07.190 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.3549999999999999e-06 2024-02-23T00:46:07.190 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.190 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.190 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:46:07.190 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.769374+0000", 2024-02-23T00:46:07.190 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.3180000000000003e-06 2024-02-23T00:46:07.190 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.190 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.190 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:46:07.190 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.769381+0000", 2024-02-23T00:46:07.191 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0045879120000000004 2024-02-23T00:46:07.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.191 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.191 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:46:07.191 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.773969+0000", 2024-02-23T00:46:07.191 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.3527e-05 2024-02-23T00:46:07.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.191 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.191 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:46:07.191 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.773992+0000", 2024-02-23T00:46:07.191 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.022291986999999999 2024-02-23T00:46:07.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.191 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.191 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:46:07.192 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.796284+0000", 2024-02-23T00:46:07.192 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.1721000000000003e-05 2024-02-23T00:46:07.192 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.192 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.192 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:46:07.192 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.796326+0000", 2024-02-23T00:46:07.192 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.026973831 2024-02-23T00:46:07.192 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.192 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:46:07.192 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.192 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.192 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.192 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24832.0:15 21.15 21:ac8d7e44:test-rados-api-smithi047-82517-4::foo:head [watch unwatch cookie 94456142960576] snapc 0=[] ondisk+write+known_if_redirected e111)", 2024-02-23T00:46:07.193 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:00.909423+0000", 2024-02-23T00:46:07.193 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 186.14209182600001, 2024-02-23T00:46:07.193 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.048904532000000001, 2024-02-23T00:46:07.193 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:46:07.193 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:46:07.193 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:46:07.193 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24832", 2024-02-23T00:46:07.193 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/2893985577", 2024-02-23T00:46:07.193 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 15 2024-02-23T00:46:07.193 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.193 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:46:07.193 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.193 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:46:07.193 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.909423+0000", 2024-02-23T00:46:07.193 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:46:07.194 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.194 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.194 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:46:07.194 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.909423+0000", 2024-02-23T00:46:07.194 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999976 2024-02-23T00:46:07.194 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.194 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.194 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:46:07.194 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.909421+0000", 2024-02-23T00:46:07.194 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.5889999999999992e-06 2024-02-23T00:46:07.194 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.195 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.195 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:46:07.195 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.909430+0000", 2024-02-23T00:46:07.195 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.025760463000000001 2024-02-23T00:46:07.195 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.195 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.195 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:46:07.195 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.935191+0000", 2024-02-23T00:46:07.195 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002311156 2024-02-23T00:46:07.195 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.195 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.195 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:46:07.195 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.937502+0000", 2024-02-23T00:46:07.195 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00039955999999999998 2024-02-23T00:46:07.196 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.196 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.196 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:46:07.196 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.937902+0000", 2024-02-23T00:46:07.196 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.8650999999999998e-05 2024-02-23T00:46:07.196 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.196 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.196 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:46:07.196 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.937960+0000", 2024-02-23T00:46:07.196 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.020367645 2024-02-23T00:46:07.196 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.196 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.196 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:46:07.197 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.958328+0000", 2024-02-23T00:46:07.197 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.048904532000000001 2024-02-23T00:46:07.197 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.197 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:46:07.197 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.197 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.197 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.197 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24775.0:2059 20.10 20:0c23f028:test-rados-api-smithi047-82457-16::big:head [writefull 0~25165836] snapc 0=[] ondisk+write+known_if_redirected e112)", 2024-02-23T00:46:07.197 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:01.316014+0000", 2024-02-23T00:46:07.197 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 185.735501615, 2024-02-23T00:46:07.197 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.116455373, 2024-02-23T00:46:07.198 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:46:07.198 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:07.198 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:46:07.198 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24775", 2024-02-23T00:46:07.198 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1191937413", 2024-02-23T00:46:07.198 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2059 2024-02-23T00:46:07.198 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.198 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:46:07.198 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.198 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:46:07.198 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.316014+0000", 2024-02-23T00:46:07.198 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:46:07.198 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.199 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.199 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:46:07.199 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.316014+0000", 2024-02-23T00:46:07.199 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999948 2024-02-23T00:46:07.199 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.199 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.199 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:46:07.199 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.316008+0000", 2024-02-23T00:46:07.199 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.019319273000000001 2024-02-23T00:46:07.199 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.199 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.199 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:46:07.199 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.335328+0000", 2024-02-23T00:46:07.199 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.7900000000000001e-05 2024-02-23T00:46:07.200 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.200 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.200 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:46:07.200 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.335356+0000", 2024-02-23T00:46:07.200 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.3740000000000004e-06 2024-02-23T00:46:07.200 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.200 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.200 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:46:07.200 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.335365+0000", 2024-02-23T00:46:07.200 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.4500999999999998e-05 2024-02-23T00:46:07.200 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.200 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.200 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:46:07.201 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.335429+0000", 2024-02-23T00:46:07.201 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000157138 2024-02-23T00:46:07.201 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.201 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.201 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:46:07.201 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.335587+0000", 2024-02-23T00:46:07.201 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000189459 2024-02-23T00:46:07.201 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.201 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.201 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 5,6", 2024-02-23T00:46:07.201 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.335776+0000", 2024-02-23T00:46:07.201 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.028177522 2024-02-23T00:46:07.201 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.201 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.201 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:46:07.202 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.363954+0000", 2024-02-23T00:46:07.202 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.042488566999999998 2024-02-23T00:46:07.202 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.202 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.202 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:46:07.202 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.406442+0000", 2024-02-23T00:46:07.202 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.025953894000000002 2024-02-23T00:46:07.202 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.202 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.202 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:46:07.202 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.432396+0000", 2024-02-23T00:46:07.202 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.9501e-05 2024-02-23T00:46:07.202 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.202 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.203 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:46:07.203 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.432435+0000", 2024-02-23T00:46:07.203 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.3473999999999998e-05 2024-02-23T00:46:07.203 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.203 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.203 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:46:07.203 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.432469+0000", 2024-02-23T00:46:07.203 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.116455373 2024-02-23T00:46:07.203 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.203 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:46:07.203 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.203 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.203 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.204 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.14883.0:8 25.9 25:916e78a7:test-rados-api-smithi047-82496-4::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e115)", 2024-02-23T00:46:07.204 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:04.314816+0000", 2024-02-23T00:46:07.204 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 182.73669875300001, 2024-02-23T00:46:07.204 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029852838999999999, 2024-02-23T00:46:07.204 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:46:07.204 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:07.204 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:46:07.204 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.14883", 2024-02-23T00:46:07.204 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1494681915", 2024-02-23T00:46:07.204 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 8 2024-02-23T00:46:07.204 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.204 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:46:07.204 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.205 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:46:07.205 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.314816+0000", 2024-02-23T00:46:07.205 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:46:07.205 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.205 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.205 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:46:07.205 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.314816+0000", 2024-02-23T00:46:07.205 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:46:07.205 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.205 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.205 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:46:07.205 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.314815+0000", 2024-02-23T00:46:07.205 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00058973700000000001 2024-02-23T00:46:07.205 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.206 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.206 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:46:07.206 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.315404+0000", 2024-02-23T00:46:07.206 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0613e-05 2024-02-23T00:46:07.206 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.206 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.206 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:46:07.206 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.315415+0000", 2024-02-23T00:46:07.206 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.267e-06 2024-02-23T00:46:07.206 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.206 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.206 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:46:07.206 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.315422+0000", 2024-02-23T00:46:07.206 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014541700000000001 2024-02-23T00:46:07.207 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.207 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.207 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:46:07.207 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.315568+0000", 2024-02-23T00:46:07.207 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012594799999999999 2024-02-23T00:46:07.207 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.207 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.207 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:46:07.207 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.315694+0000", 2024-02-23T00:46:07.207 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00016006899999999999 2024-02-23T00:46:07.207 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.207 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.207 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 0,1", 2024-02-23T00:46:07.208 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.315854+0000", 2024-02-23T00:46:07.208 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0030685589999999998 2024-02-23T00:46:07.208 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.208 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.208 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:46:07.208 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.318922+0000", 2024-02-23T00:46:07.208 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0044993280000000004 2024-02-23T00:46:07.208 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.208 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.208 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:46:07.208 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.323422+0000", 2024-02-23T00:46:07.208 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021145840999999999 2024-02-23T00:46:07.208 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.208 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.209 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:46:07.209 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.344568+0000", 2024-02-23T00:46:07.209 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.4187999999999997e-05 2024-02-23T00:46:07.209 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.209 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.209 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:46:07.209 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.344622+0000", 2024-02-23T00:46:07.209 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.7571000000000002e-05 2024-02-23T00:46:07.209 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.209 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.209 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:46:07.209 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.344669+0000", 2024-02-23T00:46:07.209 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029852838999999999 2024-02-23T00:46:07.209 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.209 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:46:07.209 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.210 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.210 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.210 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(client.25120.0:2 41.1a e116/115 41:5c6776db:::0.obj:head v 116'1, mlcod=0'0)", 2024-02-23T00:46:07.210 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:05.279600+0000", 2024-02-23T00:46:07.210 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 181.771914799, 2024-02-23T00:46:07.210 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.019639007999999999, 2024-02-23T00:46:07.210 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:46:07.210 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:07.210 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:46:07.210 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.210 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:46:07.210 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:05.279600+0000", 2024-02-23T00:46:07.210 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:46:07.210 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.210 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.210 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:46:07.211 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:05.279600+0000", 2024-02-23T00:46:07.211 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:46:07.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.211 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.211 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:46:07.211 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:05.279600+0000", 2024-02-23T00:46:07.211 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.7436999999999999e-05 2024-02-23T00:46:07.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.211 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.211 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:46:07.211 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:05.279617+0000", 2024-02-23T00:46:07.211 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8709999999999997e-06 2024-02-23T00:46:07.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.211 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.211 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:46:07.211 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:05.279625+0000", 2024-02-23T00:46:07.212 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.5129999999999999e-06 2024-02-23T00:46:07.212 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.212 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.212 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:46:07.212 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:05.279633+0000", 2024-02-23T00:46:07.212 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.015584344999999999 2024-02-23T00:46:07.212 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.212 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.212 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:46:07.212 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:05.295217+0000", 2024-02-23T00:46:07.212 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.9414000000000001e-05 2024-02-23T00:46:07.212 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.212 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.213 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:46:07.213 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:05.295237+0000", 2024-02-23T00:46:07.213 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0039728480000000002 2024-02-23T00:46:07.213 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.213 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.213 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:46:07.213 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:05.299209+0000", 2024-02-23T00:46:07.213 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0018e-05 2024-02-23T00:46:07.213 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.213 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.213 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:46:07.213 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:05.299239+0000", 2024-02-23T00:46:07.213 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.019639007999999999 2024-02-23T00:46:07.213 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.213 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:46:07.214 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.214 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.214 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.214 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(osd.1.0:297 20.5 e140/109 -22:a70ec860:::temp_20.5_0_24170_1:head v 0'0, mlcod=112'1)", 2024-02-23T00:46:07.214 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:32.554993+0000", 2024-02-23T00:46:07.214 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 154.496521868, 2024-02-23T00:46:07.214 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.014865562000000001, 2024-02-23T00:46:07.214 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:46:07.214 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:07.214 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:46:07.214 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.214 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:46:07.214 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.554993+0000", 2024-02-23T00:46:07.214 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:46:07.214 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.215 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.215 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:46:07.215 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.554993+0000", 2024-02-23T00:46:07.215 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:46:07.215 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.215 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.215 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:46:07.215 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.554992+0000", 2024-02-23T00:46:07.215 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0043631269999999996 2024-02-23T00:46:07.215 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.215 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.215 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:46:07.215 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.559355+0000", 2024-02-23T00:46:07.215 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.7300999999999999e-05 2024-02-23T00:46:07.215 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.216 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.216 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:46:07.216 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.559372+0000", 2024-02-23T00:46:07.216 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.927e-06 2024-02-23T00:46:07.216 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.216 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.216 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:46:07.216 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.559382+0000", 2024-02-23T00:46:07.216 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00085867500000000004 2024-02-23T00:46:07.216 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.216 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.216 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:46:07.217 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.560241+0000", 2024-02-23T00:46:07.217 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.7087999999999999e-05 2024-02-23T00:46:07.217 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.217 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.217 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:46:07.217 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.560268+0000", 2024-02-23T00:46:07.217 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0095370769999999997 2024-02-23T00:46:07.217 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.217 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.217 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:46:07.217 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.569805+0000", 2024-02-23T00:46:07.217 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.4162000000000001e-05 2024-02-23T00:46:07.217 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.218 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.218 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:46:07.218 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.569859+0000", 2024-02-23T00:46:07.218 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.014865562000000001 2024-02-23T00:46:07.218 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.218 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:46:07.218 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.218 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.218 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.218 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(osd.1.0:298 20.5 e140/109 -22:a70ec860:::temp_20.5_0_24170_1:head v 0'0, mlcod=112'1)", 2024-02-23T00:46:07.218 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:32.581619+0000", 2024-02-23T00:46:07.218 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 154.46989618200001, 2024-02-23T00:46:07.219 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.027222695000000002, 2024-02-23T00:46:07.219 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:46:07.219 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:07.219 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:46:07.219 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.219 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:46:07.219 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.581619+0000", 2024-02-23T00:46:07.219 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:46:07.219 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.219 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.219 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:46:07.219 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.581619+0000", 2024-02-23T00:46:07.220 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999976 2024-02-23T00:46:07.220 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.220 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.220 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:46:07.220 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.581616+0000", 2024-02-23T00:46:07.220 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0060049730000000003 2024-02-23T00:46:07.220 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.220 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.220 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:46:07.220 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.587621+0000", 2024-02-23T00:46:07.220 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.9785e-05 2024-02-23T00:46:07.220 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.221 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.221 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:46:07.221 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.587651+0000", 2024-02-23T00:46:07.221 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2072999999999999e-05 2024-02-23T00:46:07.221 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.221 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.221 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:46:07.221 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.587663+0000", 2024-02-23T00:46:07.221 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000194685 2024-02-23T00:46:07.221 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.221 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.221 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:46:07.221 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.587858+0000", 2024-02-23T00:46:07.222 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8759e-05 2024-02-23T00:46:07.222 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.222 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.222 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:46:07.222 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.587887+0000", 2024-02-23T00:46:07.222 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.020889690999999998 2024-02-23T00:46:07.222 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.222 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.222 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:46:07.222 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.608776+0000", 2024-02-23T00:46:07.222 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.5248999999999998e-05 2024-02-23T00:46:07.222 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.222 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.223 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:46:07.223 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.608842+0000", 2024-02-23T00:46:07.223 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.027222695000000002 2024-02-23T00:46:07.223 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.223 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:46:07.223 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.223 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.223 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.223 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(osd.1.0:299 20.5 e140/109 -22:a70ec860:::temp_20.5_0_24170_1:head v 0'0, mlcod=112'1)", 2024-02-23T00:46:07.223 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:32.615170+0000", 2024-02-23T00:46:07.223 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 154.43634484500001, 2024-02-23T00:46:07.223 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021461912, 2024-02-23T00:46:07.223 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:46:07.224 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:07.224 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:46:07.224 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.224 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:46:07.224 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.615170+0000", 2024-02-23T00:46:07.224 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:46:07.224 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.224 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.224 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:46:07.224 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.615170+0000", 2024-02-23T00:46:07.224 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999976 2024-02-23T00:46:07.224 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.224 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.225 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:46:07.225 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.615168+0000", 2024-02-23T00:46:07.225 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0063487930000000001 2024-02-23T00:46:07.225 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.225 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.225 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:46:07.225 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.621517+0000", 2024-02-23T00:46:07.225 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.3031000000000003e-05 2024-02-23T00:46:07.225 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.225 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.225 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:46:07.226 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.621550+0000", 2024-02-23T00:46:07.226 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2002e-05 2024-02-23T00:46:07.226 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.226 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.226 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:46:07.226 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.621562+0000", 2024-02-23T00:46:07.226 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.0396000000000007e-05 2024-02-23T00:46:07.226 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.226 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.226 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:46:07.226 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.621652+0000", 2024-02-23T00:46:07.227 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.4893e-05 2024-02-23T00:46:07.227 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.227 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.227 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:46:07.227 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.621677+0000", 2024-02-23T00:46:07.227 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.014898613999999999 2024-02-23T00:46:07.227 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.227 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.227 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:46:07.227 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.636576+0000", 2024-02-23T00:46:07.227 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.6641999999999997e-05 2024-02-23T00:46:07.228 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.228 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.228 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:46:07.228 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.636632+0000", 2024-02-23T00:46:07.228 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021461912 2024-02-23T00:46:07.228 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.228 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:46:07.228 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.228 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.228 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.229 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(client.25529.0:42 63.e e142/127 63:7037fd60:test-rados-api-smithi047-82496-10::foo:head v 142'13, mlcod=140'10)", 2024-02-23T00:46:07.229 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:34.508189+0000", 2024-02-23T00:46:07.229 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 152.54332607399999, 2024-02-23T00:46:07.229 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.046943154000000001, 2024-02-23T00:46:07.229 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:46:07.229 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:07.229 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:46:07.229 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.229 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:46:07.229 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.508189+0000", 2024-02-23T00:46:07.229 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:46:07.230 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.230 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.230 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:46:07.230 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.508189+0000", 2024-02-23T00:46:07.230 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.999999 2024-02-23T00:46:07.230 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.230 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.230 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:46:07.230 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.508188+0000", 2024-02-23T00:46:07.230 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0925e-05 2024-02-23T00:46:07.230 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.230 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.231 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:46:07.231 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.508199+0000", 2024-02-23T00:46:07.231 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.5159999999999994e-06 2024-02-23T00:46:07.231 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.231 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.231 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:46:07.231 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.508208+0000", 2024-02-23T00:46:07.231 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.1750000000000002e-06 2024-02-23T00:46:07.231 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.231 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.231 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:46:07.231 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.508214+0000", 2024-02-23T00:46:07.232 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00015588599999999999 2024-02-23T00:46:07.232 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.232 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.232 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:46:07.232 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.508370+0000", 2024-02-23T00:46:07.232 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1435000000000001e-05 2024-02-23T00:46:07.232 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.232 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.232 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:46:07.232 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.508391+0000", 2024-02-23T00:46:07.232 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.046710464 2024-02-23T00:46:07.232 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.233 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.233 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:46:07.233 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.555102+0000", 2024-02-23T00:46:07.233 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0552000000000002e-05 2024-02-23T00:46:07.233 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.233 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.233 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:46:07.233 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.555132+0000", 2024-02-23T00:46:07.233 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.046943154000000001 2024-02-23T00:46:07.233 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.233 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:46:07.234 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.234 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.234 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.234 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDScrubReserve(6.e RELEASE e144)", 2024-02-23T00:46:07.234 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:36.207155+0000", 2024-02-23T00:46:07.234 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 150.84435992300001, 2024-02-23T00:46:07.234 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.024392191000000001, 2024-02-23T00:46:07.234 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:46:07.234 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:46:07.234 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:46:07.234 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.235 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:46:07.235 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.207155+0000", 2024-02-23T00:46:07.235 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:46:07.235 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.235 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.235 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:46:07.235 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.207155+0000", 2024-02-23T00:46:07.235 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999986 2024-02-23T00:46:07.235 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.235 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.235 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:46:07.235 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.207154+0000", 2024-02-23T00:46:07.235 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.7485999999999996e-05 2024-02-23T00:46:07.235 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.236 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.236 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:46:07.236 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.207241+0000", 2024-02-23T00:46:07.236 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.394e-05 2024-02-23T00:46:07.236 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.236 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.236 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:46:07.236 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.207255+0000", 2024-02-23T00:46:07.236 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.6729999999999992e-06 2024-02-23T00:46:07.236 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.236 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.236 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:46:07.236 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.207263+0000", 2024-02-23T00:46:07.236 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.016180798 2024-02-23T00:46:07.237 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.237 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.237 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:46:07.237 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.223444+0000", 2024-02-23T00:46:07.237 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.7590000000000003e-06 2024-02-23T00:46:07.237 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.237 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.237 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-02-23T00:46:07.237 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.223452+0000", 2024-02-23T00:46:07.237 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.008038847 2024-02-23T00:46:07.237 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.237 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.237 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:46:07.237 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.231491+0000", 2024-02-23T00:46:07.238 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1171000000000001e-05 2024-02-23T00:46:07.238 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.238 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.238 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:46:07.238 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.231512+0000", 2024-02-23T00:46:07.238 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.4987000000000002e-05 2024-02-23T00:46:07.238 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.238 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.238 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:46:07.238 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.231547+0000", 2024-02-23T00:46:07.238 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.024392191000000001 2024-02-23T00:46:07.238 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.238 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:46:07.238 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.238 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.239 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.239 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWrite(79.6s2 145/139 ECSubWrite(tid=229, reqid=client.31603.0:7, at_version=145'1, trim_to=0'0, roll_forward_to=0'0))", 2024-02-23T00:46:07.239 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:37.220998+0000", 2024-02-23T00:46:07.239 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 149.830517617, 2024-02-23T00:46:07.239 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029900639999999999, 2024-02-23T00:46:07.239 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:46:07.239 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-02-23T00:46:07.239 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:46:07.239 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.239 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:46:07.239 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.220998+0000", 2024-02-23T00:46:07.239 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:46:07.239 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.239 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.240 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:46:07.240 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.220998+0000", 2024-02-23T00:46:07.240 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:46:07.240 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.240 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.240 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:46:07.240 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.220997+0000", 2024-02-23T00:46:07.240 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.4640000000000001e-05 2024-02-23T00:46:07.240 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.240 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.240 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:46:07.240 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.221022+0000", 2024-02-23T00:46:07.240 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.4220000000000006e-06 2024-02-23T00:46:07.240 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.241 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.241 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:46:07.241 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.221031+0000", 2024-02-23T00:46:07.241 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.4699999999999996e-06 2024-02-23T00:46:07.241 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.241 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.241 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:46:07.241 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.221039+0000", 2024-02-23T00:46:07.241 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0063503040000000002 2024-02-23T00:46:07.241 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.241 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.241 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:46:07.241 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.227389+0000", 2024-02-23T00:46:07.241 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.9499999999999996e-06 2024-02-23T00:46:07.241 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.242 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.242 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-02-23T00:46:07.242 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.227399+0000", 2024-02-23T00:46:07.242 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023482154000000002 2024-02-23T00:46:07.242 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.242 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.242 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-02-23T00:46:07.242 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.250881+0000", 2024-02-23T00:46:07.242 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.7016000000000001e-05 2024-02-23T00:46:07.242 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.242 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.242 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:46:07.242 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.250898+0000", 2024-02-23T00:46:07.242 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029900639999999999 2024-02-23T00:46:07.243 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.243 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:46:07.243 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.243 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.243 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.243 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(client.15270.0:248 62.14 e145/126 62:28b98b91:test-rados-api-smithi047-82465-12::bar:head v 145'1, mlcod=0'0)", 2024-02-23T00:46:07.243 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:37.222926+0000", 2024-02-23T00:46:07.243 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 149.828589677, 2024-02-23T00:46:07.243 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029711336000000001, 2024-02-23T00:46:07.243 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:46:07.243 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:07.243 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:46:07.243 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.243 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:46:07.243 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.222926+0000", 2024-02-23T00:46:07.244 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:46:07.244 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.244 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.244 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:46:07.244 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.222926+0000", 2024-02-23T00:46:07.244 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:46:07.244 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.244 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.244 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:46:07.244 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.222925+0000", 2024-02-23T00:46:07.244 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.8849999999999994e-06 2024-02-23T00:46:07.244 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.245 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.245 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:46:07.245 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.222935+0000", 2024-02-23T00:46:07.245 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.3560000000000004e-06 2024-02-23T00:46:07.245 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.245 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.245 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:46:07.245 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.222942+0000", 2024-02-23T00:46:07.245 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.0180000000000004e-06 2024-02-23T00:46:07.245 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.245 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.245 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:46:07.245 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.222949+0000", 2024-02-23T00:46:07.246 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001121143 2024-02-23T00:46:07.246 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.246 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.246 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:46:07.246 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.224070+0000", 2024-02-23T00:46:07.246 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5017e-05 2024-02-23T00:46:07.246 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.246 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.246 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:46:07.246 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.224085+0000", 2024-02-23T00:46:07.246 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.028523943999999999 2024-02-23T00:46:07.246 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.247 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.247 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:46:07.247 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.252609+0000", 2024-02-23T00:46:07.247 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.7416e-05 2024-02-23T00:46:07.247 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.247 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.247 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:46:07.247 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.252637+0000", 2024-02-23T00:46:07.247 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029711336000000001 2024-02-23T00:46:07.247 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.247 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:46:07.247 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.247 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.247 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.247 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(client.15270.0:252 62.18 e146/126 62:1be66eec:test-rados-api-smithi047-82465-12::foo:head v 146'3, mlcod=0'0)", 2024-02-23T00:46:07.248 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:38.406443+0000", 2024-02-23T00:46:07.248 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 148.645072572, 2024-02-23T00:46:07.248 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021146257000000002, 2024-02-23T00:46:07.248 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:46:07.248 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:07.248 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:46:07.248 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.248 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:46:07.248 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.406443+0000", 2024-02-23T00:46:07.248 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:46:07.248 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.248 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.248 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:46:07.248 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.406443+0000", 2024-02-23T00:46:07.249 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:46:07.249 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.249 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.249 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:46:07.249 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.406442+0000", 2024-02-23T00:46:07.249 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0117e-05 2024-02-23T00:46:07.249 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.249 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.249 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:46:07.249 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.406452+0000", 2024-02-23T00:46:07.249 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.5140000000000004e-06 2024-02-23T00:46:07.249 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.249 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.249 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:46:07.249 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.406460+0000", 2024-02-23T00:46:07.249 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8930000000000005e-06 2024-02-23T00:46:07.250 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.250 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.250 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:46:07.250 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.406468+0000", 2024-02-23T00:46:07.250 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010643227 2024-02-23T00:46:07.250 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.250 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.250 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:46:07.250 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.417111+0000", 2024-02-23T00:46:07.250 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1742999999999998e-05 2024-02-23T00:46:07.250 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.250 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.250 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:46:07.250 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.417133+0000", 2024-02-23T00:46:07.250 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010419154999999999 2024-02-23T00:46:07.250 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.251 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.251 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:46:07.251 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.427552+0000", 2024-02-23T00:46:07.251 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.6937000000000002e-05 2024-02-23T00:46:07.251 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.251 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.251 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:46:07.251 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.427589+0000", 2024-02-23T00:46:07.251 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021146257000000002 2024-02-23T00:46:07.251 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.251 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:46:07.251 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.251 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.251 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.252 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.40541.0:2 116.6 116:602f83fe:::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e181)", 2024-02-23T00:46:07.252 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:13.469852+0000", 2024-02-23T00:46:07.252 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 113.58166356, 2024-02-23T00:46:07.252 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.042655076, 2024-02-23T00:46:07.252 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:46:07.252 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:07.252 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:46:07.252 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.40541", 2024-02-23T00:46:07.252 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/865084253", 2024-02-23T00:46:07.252 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2 2024-02-23T00:46:07.252 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.252 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:46:07.252 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.253 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:46:07.253 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.469852+0000", 2024-02-23T00:46:07.253 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:46:07.253 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.253 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.253 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:46:07.253 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.469852+0000", 2024-02-23T00:46:07.253 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:46:07.253 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.253 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.253 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:46:07.253 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.469850+0000", 2024-02-23T00:46:07.254 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.166e-06 2024-02-23T00:46:07.254 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.254 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.254 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:46:07.254 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.469859+0000", 2024-02-23T00:46:07.254 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2982999999999999e-05 2024-02-23T00:46:07.254 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.254 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.254 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:46:07.254 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.469872+0000", 2024-02-23T00:46:07.254 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.6010000000000001e-06 2024-02-23T00:46:07.254 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.254 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.255 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:46:07.255 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.469881+0000", 2024-02-23T00:46:07.255 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.8393999999999996e-05 2024-02-23T00:46:07.255 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.255 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.255 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:46:07.255 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.469950+0000", 2024-02-23T00:46:07.255 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.5129999999999999e-06 2024-02-23T00:46:07.255 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.255 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.255 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-02-23T00:46:07.255 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.469957+0000", 2024-02-23T00:46:07.255 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.013799713 2024-02-23T00:46:07.255 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.256 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.256 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:46:07.256 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.483757+0000", 2024-02-23T00:46:07.256 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8869999999999992e-06 2024-02-23T00:46:07.256 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.256 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.256 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:46:07.256 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.483765+0000", 2024-02-23T00:46:07.256 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.015659230999999999 2024-02-23T00:46:07.256 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.256 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.256 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:46:07.256 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.499424+0000", 2024-02-23T00:46:07.257 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00013537900000000001 2024-02-23T00:46:07.257 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.257 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.257 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:46:07.257 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.499559+0000", 2024-02-23T00:46:07.257 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000169433 2024-02-23T00:46:07.257 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.257 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.257 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 5,6", 2024-02-23T00:46:07.257 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.499729+0000", 2024-02-23T00:46:07.257 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012107788 2024-02-23T00:46:07.257 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.257 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.257 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:46:07.258 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.511837+0000", 2024-02-23T00:46:07.258 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00053866299999999999 2024-02-23T00:46:07.258 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.258 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.258 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:46:07.258 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.512375+0000", 2024-02-23T00:46:07.258 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.4773000000000001e-05 2024-02-23T00:46:07.258 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.258 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.258 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:46:07.258 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.512410+0000", 2024-02-23T00:46:07.258 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.5303000000000002e-05 2024-02-23T00:46:07.258 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.258 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.259 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:46:07.259 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.512465+0000", 2024-02-23T00:46:07.259 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.1287e-05 2024-02-23T00:46:07.259 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.259 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.259 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:46:07.259 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.512507+0000", 2024-02-23T00:46:07.259 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.042655076 2024-02-23T00:46:07.259 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.259 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:46:07.259 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.259 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.259 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.259 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.40517.0:2 115.1b 115:d8841f39:test-rados-api-smithi047-82457-26::foo:head [writefull 0~8388608] snapc 0=[] ondisk+write+known_if_redirected e182)", 2024-02-23T00:46:07.259 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:14.489687+0000", 2024-02-23T00:46:07.260 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 112.561828164, 2024-02-23T00:46:07.260 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.02886706, 2024-02-23T00:46:07.260 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:46:07.260 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:07.260 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:46:07.260 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.40517", 2024-02-23T00:46:07.260 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/3108721855", 2024-02-23T00:46:07.260 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2 2024-02-23T00:46:07.260 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.260 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:46:07.261 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.261 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:46:07.261 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.489687+0000", 2024-02-23T00:46:07.261 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:46:07.261 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.261 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.261 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:46:07.261 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.489687+0000", 2024-02-23T00:46:07.261 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999976 2024-02-23T00:46:07.261 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.261 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.261 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:46:07.261 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.489684+0000", 2024-02-23T00:46:07.261 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0052556039999999997 2024-02-23T00:46:07.262 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.262 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.262 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:46:07.262 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.494940+0000", 2024-02-23T00:46:07.262 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.5494e-05 2024-02-23T00:46:07.262 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.262 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.262 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:46:07.262 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.494966+0000", 2024-02-23T00:46:07.262 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.4069999999999999e-06 2024-02-23T00:46:07.262 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.262 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.262 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:46:07.262 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.494975+0000", 2024-02-23T00:46:07.263 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.6199999999999996e-05 2024-02-23T00:46:07.263 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.263 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.263 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:46:07.263 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.495041+0000", 2024-02-23T00:46:07.263 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000167143 2024-02-23T00:46:07.263 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.263 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.263 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:46:07.263 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.495208+0000", 2024-02-23T00:46:07.263 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000190027 2024-02-23T00:46:07.263 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.263 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.263 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 0,1", 2024-02-23T00:46:07.264 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.495398+0000", 2024-02-23T00:46:07.264 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.013352564000000001 2024-02-23T00:46:07.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.264 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.264 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:46:07.264 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.508751+0000", 2024-02-23T00:46:07.264 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0094962550000000003 2024-02-23T00:46:07.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.264 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.264 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:46:07.264 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.518247+0000", 2024-02-23T00:46:07.264 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000230545 2024-02-23T00:46:07.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.265 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.265 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:46:07.265 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.518478+0000", 2024-02-23T00:46:07.265 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.8698000000000001e-05 2024-02-23T00:46:07.265 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.265 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.265 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:46:07.265 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.518516+0000", 2024-02-23T00:46:07.265 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7685999999999998e-05 2024-02-23T00:46:07.265 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.265 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.265 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:46:07.265 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.518554+0000", 2024-02-23T00:46:07.266 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.02886706 2024-02-23T00:46:07.266 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.266 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:46:07.266 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.266 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.266 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.266 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(osd.4.0:475 62.d e199/126 62:b51f391b:test-rados-api-smithi047-82465-14::foo92:4 v 199'33, mlcod=197'29)", 2024-02-23T00:46:07.266 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:31.146324+0000", 2024-02-23T00:46:07.266 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 95.905191133000002, 2024-02-23T00:46:07.266 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.015434286, 2024-02-23T00:46:07.266 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:46:07.266 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:07.266 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:46:07.266 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.267 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:46:07.267 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.146324+0000", 2024-02-23T00:46:07.267 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:46:07.267 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.267 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.267 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:46:07.267 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.146324+0000", 2024-02-23T00:46:07.267 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:46:07.267 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.267 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.267 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:46:07.267 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.146324+0000", 2024-02-23T00:46:07.267 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.3489999999999997e-06 2024-02-23T00:46:07.267 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.268 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.268 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:46:07.268 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.146330+0000", 2024-02-23T00:46:07.268 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1277000000000001e-05 2024-02-23T00:46:07.268 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.268 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.268 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:46:07.268 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.146341+0000", 2024-02-23T00:46:07.268 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.4909999999999999e-06 2024-02-23T00:46:07.268 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.268 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.268 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:46:07.269 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.146349+0000", 2024-02-23T00:46:07.269 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.0483999999999998e-05 2024-02-23T00:46:07.269 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.269 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.269 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:46:07.269 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.146419+0000", 2024-02-23T00:46:07.269 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1820000000000001e-05 2024-02-23T00:46:07.269 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.269 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.269 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:46:07.269 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.146441+0000", 2024-02-23T00:46:07.269 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.015284246 2024-02-23T00:46:07.269 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.269 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.270 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:46:07.270 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.161725+0000", 2024-02-23T00:46:07.270 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.2932000000000003e-05 2024-02-23T00:46:07.270 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.270 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.270 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:46:07.270 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.161758+0000", 2024-02-23T00:46:07.270 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.015434286 2024-02-23T00:46:07.270 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.270 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:46:07.270 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.270 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.270 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.270 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(osd.4.0:477 62.f e199/126 62:f64c5448:test-rados-api-smithi047-82465-14::foo96:4 v 199'80, mlcod=197'74)", 2024-02-23T00:46:07.271 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:31.164866+0000", 2024-02-23T00:46:07.271 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 95.886649573, 2024-02-23T00:46:07.271 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018575001000000001, 2024-02-23T00:46:07.271 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:46:07.271 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:07.271 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:46:07.271 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.271 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:46:07.271 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164866+0000", 2024-02-23T00:46:07.271 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:46:07.271 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.271 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.271 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:46:07.271 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164866+0000", 2024-02-23T00:46:07.272 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:46:07.272 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.272 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.272 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:46:07.272 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164865+0000", 2024-02-23T00:46:07.272 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1764e-05 2024-02-23T00:46:07.272 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.272 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.272 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:46:07.272 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164877+0000", 2024-02-23T00:46:07.272 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.5299999999999996e-06 2024-02-23T00:46:07.272 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.272 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.272 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:46:07.273 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164885+0000", 2024-02-23T00:46:07.273 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.5619999999999997e-06 2024-02-23T00:46:07.273 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.273 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.273 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:46:07.273 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164890+0000", 2024-02-23T00:46:07.273 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0022547829999999998 2024-02-23T00:46:07.273 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.273 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.273 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:46:07.273 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.167145+0000", 2024-02-23T00:46:07.273 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.8792000000000003e-05 2024-02-23T00:46:07.273 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.273 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.273 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:46:07.274 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.167193+0000", 2024-02-23T00:46:07.274 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.016211952000000002 2024-02-23T00:46:07.274 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.274 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.274 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:46:07.274 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.183405+0000", 2024-02-23T00:46:07.274 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.5181000000000001e-05 2024-02-23T00:46:07.274 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.274 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.274 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:46:07.274 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.183441+0000", 2024-02-23T00:46:07.274 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018575001000000001 2024-02-23T00:46:07.274 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.274 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:46:07.275 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.275 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.275 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.275 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.40754.0:2 145.6 145:602f83fe:::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e222)", 2024-02-23T00:46:07.275 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:54.146645+0000", 2024-02-23T00:46:07.275 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 72.904870446999993, 2024-02-23T00:46:07.275 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021068969, 2024-02-23T00:46:07.275 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:46:07.275 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:07.275 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:46:07.275 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.40754", 2024-02-23T00:46:07.275 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/2235208660", 2024-02-23T00:46:07.275 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2 2024-02-23T00:46:07.275 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.275 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:46:07.275 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.276 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:46:07.276 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.146645+0000", 2024-02-23T00:46:07.276 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:46:07.276 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.276 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.276 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:46:07.276 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.146645+0000", 2024-02-23T00:46:07.276 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999976 2024-02-23T00:46:07.276 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.276 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.276 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:46:07.276 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.146643+0000", 2024-02-23T00:46:07.277 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.918e-06 2024-02-23T00:46:07.277 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.277 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.277 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:46:07.277 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.146651+0000", 2024-02-23T00:46:07.277 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5065000000000001e-05 2024-02-23T00:46:07.277 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.277 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.277 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:46:07.277 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.146667+0000", 2024-02-23T00:46:07.277 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1534e-05 2024-02-23T00:46:07.277 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.277 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.277 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:46:07.277 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.146678+0000", 2024-02-23T00:46:07.277 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.5680000000000002e-05 2024-02-23T00:46:07.278 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.278 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.278 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:46:07.278 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.146734+0000", 2024-02-23T00:46:07.278 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.5449999999999998e-06 2024-02-23T00:46:07.278 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.278 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.278 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-02-23T00:46:07.278 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.146741+0000", 2024-02-23T00:46:07.278 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0075490319999999998 2024-02-23T00:46:07.278 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.278 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.278 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:46:07.278 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.154290+0000", 2024-02-23T00:46:07.278 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.8569999999999997e-06 2024-02-23T00:46:07.278 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.279 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.279 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:46:07.279 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.154297+0000", 2024-02-23T00:46:07.279 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0090633199999999997 2024-02-23T00:46:07.279 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.279 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.279 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:46:07.279 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.163361+0000", 2024-02-23T00:46:07.279 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000107849 2024-02-23T00:46:07.279 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.279 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.279 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:46:07.279 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.163468+0000", 2024-02-23T00:46:07.279 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00015853699999999999 2024-02-23T00:46:07.279 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.279 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.280 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 1,4", 2024-02-23T00:46:07.280 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.163627+0000", 2024-02-23T00:46:07.280 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00066408199999999998 2024-02-23T00:46:07.280 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.280 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.280 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:46:07.280 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.164291+0000", 2024-02-23T00:46:07.280 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000633827 2024-02-23T00:46:07.280 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.280 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.280 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:46:07.280 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.164925+0000", 2024-02-23T00:46:07.281 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002695514 2024-02-23T00:46:07.281 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.281 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.281 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:46:07.281 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.167620+0000", 2024-02-23T00:46:07.281 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.2404999999999997e-05 2024-02-23T00:46:07.281 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.281 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.281 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:46:07.281 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.167673+0000", 2024-02-23T00:46:07.281 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.0967000000000003e-05 2024-02-23T00:46:07.281 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.281 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.281 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:46:07.282 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.167714+0000", 2024-02-23T00:46:07.282 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021068969 2024-02-23T00:46:07.282 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.282 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:46:07.282 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.282 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.282 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.282 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.15270.0:2078 62.3 62:c3aa4723:test-rados-api-smithi047-82465-26::foo:head [writefull 0~8] snapc 0=[] ondisk+write+known_if_redirected e245)", 2024-02-23T00:46:07.282 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:45:17.141076+0000", 2024-02-23T00:46:07.282 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 49.910439021999998, 2024-02-23T00:46:07.282 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029775917999999998, 2024-02-23T00:46:07.282 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:46:07.282 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:07.282 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:46:07.283 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.15270", 2024-02-23T00:46:07.283 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/4133111990", 2024-02-23T00:46:07.283 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2078 2024-02-23T00:46:07.283 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.283 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:46:07.283 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.283 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:46:07.283 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.141076+0000", 2024-02-23T00:46:07.283 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:46:07.283 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.283 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.283 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:46:07.283 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.141076+0000", 2024-02-23T00:46:07.284 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999976 2024-02-23T00:46:07.284 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.284 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.284 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:46:07.284 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.141074+0000", 2024-02-23T00:46:07.284 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.3340000000000008e-06 2024-02-23T00:46:07.284 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.284 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.284 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:46:07.284 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.141083+0000", 2024-02-23T00:46:07.284 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3730999999999999e-05 2024-02-23T00:46:07.284 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.284 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.284 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:46:07.285 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.141097+0000", 2024-02-23T00:46:07.285 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.3159999999999997e-06 2024-02-23T00:46:07.285 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.285 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.285 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:46:07.285 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.141105+0000", 2024-02-23T00:46:07.285 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.1898999999999999e-05 2024-02-23T00:46:07.285 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.285 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.285 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:46:07.285 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.141157+0000", 2024-02-23T00:46:07.285 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00016548999999999999 2024-02-23T00:46:07.285 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.285 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.285 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:46:07.286 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.141323+0000", 2024-02-23T00:46:07.286 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00020201299999999999 2024-02-23T00:46:07.286 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.286 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.286 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 1,4", 2024-02-23T00:46:07.286 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.141525+0000", 2024-02-23T00:46:07.286 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0095168300000000004 2024-02-23T00:46:07.286 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.286 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.286 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:46:07.286 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.151042+0000", 2024-02-23T00:46:07.286 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018502253 2024-02-23T00:46:07.286 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.286 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.287 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:46:07.287 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.169544+0000", 2024-02-23T00:46:07.287 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001218156 2024-02-23T00:46:07.287 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.287 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.287 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:46:07.287 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.170762+0000", 2024-02-23T00:46:07.287 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.3652000000000002e-05 2024-02-23T00:46:07.287 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.287 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.287 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:46:07.287 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.170806+0000", 2024-02-23T00:46:07.287 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.6415000000000002e-05 2024-02-23T00:46:07.287 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.287 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.288 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:46:07.288 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.170852+0000", 2024-02-23T00:46:07.288 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029775917999999998 2024-02-23T00:46:07.288 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.288 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:46:07.288 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.288 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.288 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.288 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.30483.0:9 155.c 155:3197012c:::foo_rmxattr:head [rmxattr attr2 in=5b] snapc 0=[] ondisk+write+known_if_redirected e243)", 2024-02-23T00:46:07.288 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:45:17.146980+0000", 2024-02-23T00:46:07.288 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 49.904535000000003, 2024-02-23T00:46:07.288 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.022584933000000001, 2024-02-23T00:46:07.288 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:46:07.288 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:46:07.289 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:46:07.289 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.30483", 2024-02-23T00:46:07.289 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/3044143263", 2024-02-23T00:46:07.289 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 9 2024-02-23T00:46:07.289 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.289 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:46:07.289 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.289 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:46:07.289 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.146980+0000", 2024-02-23T00:46:07.289 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:46:07.289 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.289 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.289 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:46:07.289 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.146980+0000", 2024-02-23T00:46:07.289 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:46:07.290 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.290 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.290 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:46:07.290 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.146978+0000", 2024-02-23T00:46:07.290 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0020000000000006e-06 2024-02-23T00:46:07.290 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.290 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.290 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:46:07.290 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.146986+0000", 2024-02-23T00:46:07.290 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4829e-05 2024-02-23T00:46:07.290 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.290 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.290 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:46:07.290 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.147001+0000", 2024-02-23T00:46:07.291 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.8660000000000001e-06 2024-02-23T00:46:07.291 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.291 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.291 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:46:07.291 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.147011+0000", 2024-02-23T00:46:07.291 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0099999999999995e-05 2024-02-23T00:46:07.291 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.291 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.291 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:46:07.291 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.147091+0000", 2024-02-23T00:46:07.291 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00010793000000000001 2024-02-23T00:46:07.291 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.291 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.291 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:46:07.292 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.147199+0000", 2024-02-23T00:46:07.292 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.022366104000000001 2024-02-23T00:46:07.292 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:46:07.292 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:46:07.292 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:46:07.292 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.169565+0000", 2024-02-23T00:46:07.292 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.022584933000000001 2024-02-23T00:46:07.292 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.292 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:46:07.292 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.292 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:46:07.292 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:46:07.295 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:46:07.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:05.817133+0000 osd.7 (osd.7) 153 : cluster [DBG] 62.8 scrub starts 2024-02-23T00:46:07.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:05.820642+0000 osd.7 (osd.7) 154 : cluster [DBG] 62.8 scrub ok 2024-02-23T00:46:07.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:05.909299+0000 osd.2 (osd.2) 135 : cluster [DBG] 2.c scrub starts 2024-02-23T00:46:07.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:05.910321+0000 osd.2 (osd.2) 136 : cluster [DBG] 2.c scrub ok 2024-02-23T00:46:07.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:06.149306+0000 mon.a (mon.0) 1976 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:46:07.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:06.149412+0000 mon.a (mon.0) 1977 : cluster [DBG] osdmap e294: 8 total, 8 up, 8 in 2024-02-23T00:46:07.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:06.150632+0000 mon.c (mon.2) 219 : audit [INF] from='client.? 172.21.15.47:0/2639020958' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi047-82378-40"}]: dispatch 2024-02-23T00:46:07.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:06.152780+0000 mon.a (mon.0) 1978 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-35"}]: dispatch 2024-02-23T00:46:07.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:06.153085+0000 mon.a (mon.0) 1979 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi047-82378-40"}]: dispatch 2024-02-23T00:46:07.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:06.173740+0000 mon.c (mon.2) 220 : audit [INF] from='client.? 172.21.15.47:0/2123205582' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi047-82386-51"}]: dispatch 2024-02-23T00:46:07.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:06.174072+0000 mon.a (mon.0) 1980 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi047-82386-51"}]: dispatch 2024-02-23T00:46:07.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:06.174729+0000 mon.c (mon.2) 221 : audit [INF] from='client.? 172.21.15.47:0/2123205582' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi047-82386-51"}]: dispatch 2024-02-23T00:46:07.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:06.174976+0000 mon.a (mon.0) 1981 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi047-82386-51"}]: dispatch 2024-02-23T00:46:07.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:06.175379+0000 mon.c (mon.2) 222 : audit [INF] from='client.? 172.21.15.47:0/2123205582' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi047-82386-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:07.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:06.175635+0000 mon.a (mon.0) 1982 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi047-82386-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:07.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:05.817133+0000 osd.7 (osd.7) 153 : cluster [DBG] 62.8 scrub starts 2024-02-23T00:46:07.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:05.820642+0000 osd.7 (osd.7) 154 : cluster [DBG] 62.8 scrub ok 2024-02-23T00:46:07.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:05.909299+0000 osd.2 (osd.2) 135 : cluster [DBG] 2.c scrub starts 2024-02-23T00:46:07.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:05.910321+0000 osd.2 (osd.2) 136 : cluster [DBG] 2.c scrub ok 2024-02-23T00:46:07.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:06.149306+0000 mon.a (mon.0) 1976 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:46:07.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:06.149412+0000 mon.a (mon.0) 1977 : cluster [DBG] osdmap e294: 8 total, 8 up, 8 in 2024-02-23T00:46:07.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:06.150632+0000 mon.c (mon.2) 219 : audit [INF] from='client.? 172.21.15.47:0/2639020958' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi047-82378-40"}]: dispatch 2024-02-23T00:46:07.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:06.152780+0000 mon.a (mon.0) 1978 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-35"}]: dispatch 2024-02-23T00:46:07.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:06.153085+0000 mon.a (mon.0) 1979 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi047-82378-40"}]: dispatch 2024-02-23T00:46:07.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:06.173740+0000 mon.c (mon.2) 220 : audit [INF] from='client.? 172.21.15.47:0/2123205582' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi047-82386-51"}]: dispatch 2024-02-23T00:46:07.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:06.174072+0000 mon.a (mon.0) 1980 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi047-82386-51"}]: dispatch 2024-02-23T00:46:07.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:06.174729+0000 mon.c (mon.2) 221 : audit [INF] from='client.? 172.21.15.47:0/2123205582' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi047-82386-51"}]: dispatch 2024-02-23T00:46:07.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:06.174976+0000 mon.a (mon.0) 1981 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi047-82386-51"}]: dispatch 2024-02-23T00:46:07.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:06.175379+0000 mon.c (mon.2) 222 : audit [INF] from='client.? 172.21.15.47:0/2123205582' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi047-82386-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:07.310 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:06.175635+0000 mon.a (mon.0) 1982 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi047-82386-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:07.418 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:46:07.421 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:07.422 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:46:07.523 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:46:07.551 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:07.571 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:46:07.573 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:07.574 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:46:07.674 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:46:07.701 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:07.719 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:46:07.721 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:07.722 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:46:07.823 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:46:07.859 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:07.986 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.3 dump_ops_in_flight 2024-02-23T00:46:08.270 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:46:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:46:08.293 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:08.294 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:46:08.395 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:46:08.421 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:08.620 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:46:08.624 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:08.625 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:46:08.726 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:46:08.740 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph pg dump --format=json 2024-02-23T00:46:08.753 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:08.793 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:46:08.802 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:08.802 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:46:08.903 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:46:08.929 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:06.254311+0000 osd.3 (osd.3) 153 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:46:08.929 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:06.254466+0000 osd.3 (osd.3) 154 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:46:08.929 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:06.774270+0000 osd.7 (osd.7) 155 : cluster [DBG] 62.0 scrub starts 2024-02-23T00:46:08.929 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:06.777987+0000 osd.7 (osd.7) 156 : cluster [DBG] 62.0 scrub ok 2024-02-23T00:46:08.929 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:07.149090+0000 mon.a (mon.0) 1983 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-35"}]': finished 2024-02-23T00:46:08.929 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:07.149194+0000 mon.a (mon.0) 1984 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi047-82378-40"}]': finished 2024-02-23T00:46:08.929 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:07.149273+0000 mon.a (mon.0) 1985 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi047-82386-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:08.929 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:07.149337+0000 mon.a (mon.0) 1986 : cluster [DBG] osdmap e295: 8 total, 8 up, 8 in 2024-02-23T00:46:08.929 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:07.151818+0000 mon.c (mon.2) 223 : audit [INF] from='client.? 172.21.15.47:0/2639020958' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi047-82378-40"}]: dispatch 2024-02-23T00:46:08.929 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:07.152791+0000 mon.a (mon.0) 1987 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-35", "mode": "writeback"}]: dispatch 2024-02-23T00:46:08.929 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:07.153756+0000 mon.c (mon.2) 224 : audit [INF] from='client.? 172.21.15.47:0/2123205582' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi047-82386-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi047-82386-51"}]: dispatch 2024-02-23T00:46:08.929 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:07.157834+0000 mon.a (mon.0) 1988 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi047-82378-40"}]: dispatch 2024-02-23T00:46:08.930 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:07.158055+0000 mon.a (mon.0) 1989 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi047-82386-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi047-82386-51"}]: dispatch 2024-02-23T00:46:08.930 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:07.251797+0000 mgr.y (mgr.14517) 299 : cluster [DBG] pgmap v418: 245 pgs: 245 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:08.930 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:07.252711+0000 mon.a (mon.0) 1990 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-02-23T00:46:08.930 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:07.538989+0000 osd.0 (osd.0) 123 : cluster [DBG] 15.16 deep-scrub starts 2024-02-23T00:46:08.930 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:07.542172+0000 osd.0 (osd.0) 124 : cluster [DBG] 15.16 deep-scrub ok 2024-02-23T00:46:08.930 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:08.146635+0000 mon.a (mon.0) 1991 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:46:08.930 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:06.254311+0000 osd.3 (osd.3) 153 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:46:08.930 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:06.254466+0000 osd.3 (osd.3) 154 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:46:08.931 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:06.774270+0000 osd.7 (osd.7) 155 : cluster [DBG] 62.0 scrub starts 2024-02-23T00:46:08.931 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:06.777987+0000 osd.7 (osd.7) 156 : cluster [DBG] 62.0 scrub ok 2024-02-23T00:46:08.931 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:07.149090+0000 mon.a (mon.0) 1983 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-35"}]': finished 2024-02-23T00:46:08.931 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:07.149194+0000 mon.a (mon.0) 1984 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi047-82378-40"}]': finished 2024-02-23T00:46:08.931 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:07.149273+0000 mon.a (mon.0) 1985 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi047-82386-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:08.931 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:07.149337+0000 mon.a (mon.0) 1986 : cluster [DBG] osdmap e295: 8 total, 8 up, 8 in 2024-02-23T00:46:08.931 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:07.151818+0000 mon.c (mon.2) 223 : audit [INF] from='client.? 172.21.15.47:0/2639020958' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi047-82378-40"}]: dispatch 2024-02-23T00:46:08.931 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:07.152791+0000 mon.a (mon.0) 1987 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-35", "mode": "writeback"}]: dispatch 2024-02-23T00:46:08.931 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:07.153756+0000 mon.c (mon.2) 224 : audit [INF] from='client.? 172.21.15.47:0/2123205582' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi047-82386-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi047-82386-51"}]: dispatch 2024-02-23T00:46:08.931 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:07.157834+0000 mon.a (mon.0) 1988 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi047-82378-40"}]: dispatch 2024-02-23T00:46:08.931 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:07.158055+0000 mon.a (mon.0) 1989 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi047-82386-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi047-82386-51"}]: dispatch 2024-02-23T00:46:08.931 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:07.251797+0000 mgr.y (mgr.14517) 299 : cluster [DBG] pgmap v418: 245 pgs: 245 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:08.931 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:07.252711+0000 mon.a (mon.0) 1990 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-02-23T00:46:08.931 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:07.538989+0000 osd.0 (osd.0) 123 : cluster [DBG] 15.16 deep-scrub starts 2024-02-23T00:46:08.932 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:07.542172+0000 osd.0 (osd.0) 124 : cluster [DBG] 15.16 deep-scrub ok 2024-02-23T00:46:08.932 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:08.146635+0000 mon.a (mon.0) 1991 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:46:08.932 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:08.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:06.254311+0000 osd.3 (osd.3) 153 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:46:08.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:06.254466+0000 osd.3 (osd.3) 154 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:46:08.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:06.774270+0000 osd.7 (osd.7) 155 : cluster [DBG] 62.0 scrub starts 2024-02-23T00:46:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:06.777987+0000 osd.7 (osd.7) 156 : cluster [DBG] 62.0 scrub ok 2024-02-23T00:46:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:07.149090+0000 mon.a (mon.0) 1983 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-35"}]': finished 2024-02-23T00:46:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:07.149194+0000 mon.a (mon.0) 1984 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi047-82378-40"}]': finished 2024-02-23T00:46:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:07.149273+0000 mon.a (mon.0) 1985 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi047-82386-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:07.149337+0000 mon.a (mon.0) 1986 : cluster [DBG] osdmap e295: 8 total, 8 up, 8 in 2024-02-23T00:46:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:07.151818+0000 mon.c (mon.2) 223 : audit [INF] from='client.? 172.21.15.47:0/2639020958' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi047-82378-40"}]: dispatch 2024-02-23T00:46:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:07.152791+0000 mon.a (mon.0) 1987 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-35", "mode": "writeback"}]: dispatch 2024-02-23T00:46:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:07.153756+0000 mon.c (mon.2) 224 : audit [INF] from='client.? 172.21.15.47:0/2123205582' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi047-82386-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi047-82386-51"}]: dispatch 2024-02-23T00:46:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:07.157834+0000 mon.a (mon.0) 1988 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi047-82378-40"}]: dispatch 2024-02-23T00:46:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:07.158055+0000 mon.a (mon.0) 1989 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi047-82386-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi047-82386-51"}]: dispatch 2024-02-23T00:46:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:07.251797+0000 mgr.y (mgr.14517) 299 : cluster [DBG] pgmap v418: 245 pgs: 245 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:07.252711+0000 mon.a (mon.0) 1990 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-02-23T00:46:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:07.538989+0000 osd.0 (osd.0) 123 : cluster [DBG] 15.16 deep-scrub starts 2024-02-23T00:46:08.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:07.542172+0000 osd.0 (osd.0) 124 : cluster [DBG] 15.16 deep-scrub ok 2024-02-23T00:46:08.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:08.146635+0000 mon.a (mon.0) 1991 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:46:09.783 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:07.814409+0000 osd.7 (osd.7) 157 : cluster [DBG] 62.b scrub starts 2024-02-23T00:46:09.783 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:07.817878+0000 osd.7 (osd.7) 158 : cluster [DBG] 62.b scrub ok 2024-02-23T00:46:09.783 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:08.026069+0000 osd.6 (osd.6) 119 : cluster [DBG] 62.1c scrub starts 2024-02-23T00:46:09.783 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:08.027644+0000 osd.6 (osd.6) 120 : cluster [DBG] 62.1c scrub ok 2024-02-23T00:46:09.783 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:08.151150+0000 mon.a (mon.0) 1992 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-35", "mode": "writeback"}]': finished 2024-02-23T00:46:09.784 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:08.151230+0000 mon.a (mon.0) 1993 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi047-82378-40"}]': finished 2024-02-23T00:46:09.784 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:08.173736+0000 mon.c (mon.2) 225 : audit [INF] from='client.? 172.21.15.47:0/297870923' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi047-82378-41"}]: dispatch 2024-02-23T00:46:09.784 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:08.528864+0000 osd.0 (osd.0) 125 : cluster [DBG] 15.6 scrub starts 2024-02-23T00:46:09.784 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:08.530430+0000 osd.0 (osd.0) 126 : cluster [DBG] 15.6 scrub ok 2024-02-23T00:46:09.784 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:08.593230+0000 mon.a (mon.0) 1994 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "26"}]': finished 2024-02-23T00:46:09.784 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:08.593300+0000 mon.a (mon.0) 1995 : cluster [DBG] osdmap e296: 8 total, 8 up, 8 in 2024-02-23T00:46:09.784 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:08.597900+0000 mon.a (mon.0) 1996 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi047-82378-41"}]: dispatch 2024-02-23T00:46:09.784 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:08.602471+0000 mon.c (mon.2) 226 : audit [INF] from='client.? 172.21.15.47:0/297870923' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi047-82378-41"}]: dispatch 2024-02-23T00:46:09.784 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:08.602704+0000 mon.a (mon.0) 1997 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi047-82378-41"}]: dispatch 2024-02-23T00:46:09.785 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:08.603199+0000 mon.c (mon.2) 227 : audit [INF] from='client.? 172.21.15.47:0/297870923' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi047-82378-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:09.785 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:07.814409+0000 osd.7 (osd.7) 157 : cluster [DBG] 62.b scrub starts 2024-02-23T00:46:09.785 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:07.817878+0000 osd.7 (osd.7) 158 : cluster [DBG] 62.b scrub ok 2024-02-23T00:46:09.785 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:08.026069+0000 osd.6 (osd.6) 119 : cluster [DBG] 62.1c scrub starts 2024-02-23T00:46:09.785 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:08.027644+0000 osd.6 (osd.6) 120 : cluster [DBG] 62.1c scrub ok 2024-02-23T00:46:09.785 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:08.151150+0000 mon.a (mon.0) 1992 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-35", "mode": "writeback"}]': finished 2024-02-23T00:46:09.785 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:08.151230+0000 mon.a (mon.0) 1993 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi047-82378-40"}]': finished 2024-02-23T00:46:09.786 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:08.173736+0000 mon.c (mon.2) 225 : audit [INF] from='client.? 172.21.15.47:0/297870923' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi047-82378-41"}]: dispatch 2024-02-23T00:46:09.786 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:08.528864+0000 osd.0 (osd.0) 125 : cluster [DBG] 15.6 scrub starts 2024-02-23T00:46:09.786 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:08.530430+0000 osd.0 (osd.0) 126 : cluster [DBG] 15.6 scrub ok 2024-02-23T00:46:09.786 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:08.593230+0000 mon.a (mon.0) 1994 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "26"}]': finished 2024-02-23T00:46:09.786 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:08.593300+0000 mon.a (mon.0) 1995 : cluster [DBG] osdmap e296: 8 total, 8 up, 8 in 2024-02-23T00:46:09.786 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:08.597900+0000 mon.a (mon.0) 1996 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi047-82378-41"}]: dispatch 2024-02-23T00:46:09.786 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:08.602471+0000 mon.c (mon.2) 226 : audit [INF] from='client.? 172.21.15.47:0/297870923' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi047-82378-41"}]: dispatch 2024-02-23T00:46:09.786 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:08.602704+0000 mon.a (mon.0) 1997 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi047-82378-41"}]: dispatch 2024-02-23T00:46:09.786 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:08.603199+0000 mon.c (mon.2) 227 : audit [INF] from='client.? 172.21.15.47:0/297870923' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi047-82378-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:09.787 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:08.603470+0000 mon.a (mon.0) 1998 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi047-82378-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:09.787 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:08.629051+0000 mon.a (mon.0) 1999 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:46:09.787 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:09.086636+0000 osd.5 (osd.5) 131 : cluster [DBG] 2.18 scrub starts 2024-02-23T00:46:09.787 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:09.088243+0000 osd.5 (osd.5) 132 : cluster [DBG] 2.18 scrub ok 2024-02-23T00:46:09.787 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:09.152539+0000 mon.a (mon.0) 2000 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi047-82386-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi047-82386-51"}]': finished 2024-02-23T00:46:09.787 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:09.152669+0000 mon.a (mon.0) 2001 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi047-82378-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:09.787 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:09.152744+0000 mon.a (mon.0) 2002 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:46:09.787 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:09.152796+0000 mon.a (mon.0) 2003 : cluster [DBG] osdmap e297: 8 total, 8 up, 8 in 2024-02-23T00:46:09.787 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:09.155792+0000 mon.a (mon.0) 2004 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-35"}]: dispatch 2024-02-23T00:46:09.787 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:09.156623+0000 mon.c (mon.2) 228 : audit [INF] from='client.? 172.21.15.47:0/297870923' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi047-82378-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi047-82378-41"}]: dispatch 2024-02-23T00:46:09.787 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:09.161593+0000 mon.a (mon.0) 2005 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi047-82378-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi047-82378-41"}]: dispatch 2024-02-23T00:46:09.788 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:08.603470+0000 mon.a (mon.0) 1998 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi047-82378-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:09.788 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:08.629051+0000 mon.a (mon.0) 1999 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:46:09.788 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:09.086636+0000 osd.5 (osd.5) 131 : cluster [DBG] 2.18 scrub starts 2024-02-23T00:46:09.788 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:09.088243+0000 osd.5 (osd.5) 132 : cluster [DBG] 2.18 scrub ok 2024-02-23T00:46:09.788 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:09.152539+0000 mon.a (mon.0) 2000 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi047-82386-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi047-82386-51"}]': finished 2024-02-23T00:46:09.788 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:09.152669+0000 mon.a (mon.0) 2001 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi047-82378-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:09.788 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:09.152744+0000 mon.a (mon.0) 2002 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:46:09.788 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:09.152796+0000 mon.a (mon.0) 2003 : cluster [DBG] osdmap e297: 8 total, 8 up, 8 in 2024-02-23T00:46:09.788 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:09.155792+0000 mon.a (mon.0) 2004 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-35"}]: dispatch 2024-02-23T00:46:09.788 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:09.156623+0000 mon.c (mon.2) 228 : audit [INF] from='client.? 172.21.15.47:0/297870923' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi047-82378-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi047-82378-41"}]: dispatch 2024-02-23T00:46:09.788 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:09.161593+0000 mon.a (mon.0) 2005 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi047-82378-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi047-82378-41"}]: dispatch 2024-02-23T00:46:09.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:07.814409+0000 osd.7 (osd.7) 157 : cluster [DBG] 62.b scrub starts 2024-02-23T00:46:09.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:07.817878+0000 osd.7 (osd.7) 158 : cluster [DBG] 62.b scrub ok 2024-02-23T00:46:09.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:08.026069+0000 osd.6 (osd.6) 119 : cluster [DBG] 62.1c scrub starts 2024-02-23T00:46:09.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:08.027644+0000 osd.6 (osd.6) 120 : cluster [DBG] 62.1c scrub ok 2024-02-23T00:46:09.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:08.151150+0000 mon.a (mon.0) 1992 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-35", "mode": "writeback"}]': finished 2024-02-23T00:46:09.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:08.151230+0000 mon.a (mon.0) 1993 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi047-82378-40"}]': finished 2024-02-23T00:46:09.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:08.173736+0000 mon.c (mon.2) 225 : audit [INF] from='client.? 172.21.15.47:0/297870923' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi047-82378-41"}]: dispatch 2024-02-23T00:46:09.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:08.528864+0000 osd.0 (osd.0) 125 : cluster [DBG] 15.6 scrub starts 2024-02-23T00:46:09.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:08.530430+0000 osd.0 (osd.0) 126 : cluster [DBG] 15.6 scrub ok 2024-02-23T00:46:09.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:08.593230+0000 mon.a (mon.0) 1994 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "26"}]': finished 2024-02-23T00:46:09.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:08.593300+0000 mon.a (mon.0) 1995 : cluster [DBG] osdmap e296: 8 total, 8 up, 8 in 2024-02-23T00:46:09.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:08.597900+0000 mon.a (mon.0) 1996 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi047-82378-41"}]: dispatch 2024-02-23T00:46:09.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:08.602471+0000 mon.c (mon.2) 226 : audit [INF] from='client.? 172.21.15.47:0/297870923' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi047-82378-41"}]: dispatch 2024-02-23T00:46:09.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:08.602704+0000 mon.a (mon.0) 1997 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi047-82378-41"}]: dispatch 2024-02-23T00:46:09.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:08.603199+0000 mon.c (mon.2) 227 : audit [INF] from='client.? 172.21.15.47:0/297870923' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi047-82378-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:09.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:08.603470+0000 mon.a (mon.0) 1998 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi047-82378-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:09.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:08.629051+0000 mon.a (mon.0) 1999 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:46:09.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:09.086636+0000 osd.5 (osd.5) 131 : cluster [DBG] 2.18 scrub starts 2024-02-23T00:46:09.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:09.088243+0000 osd.5 (osd.5) 132 : cluster [DBG] 2.18 scrub ok 2024-02-23T00:46:09.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:09.152539+0000 mon.a (mon.0) 2000 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi047-82386-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi047-82386-51"}]': finished 2024-02-23T00:46:09.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:09.152669+0000 mon.a (mon.0) 2001 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi047-82378-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:09.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:09.152744+0000 mon.a (mon.0) 2002 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:46:09.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:09.152796+0000 mon.a (mon.0) 2003 : cluster [DBG] osdmap e297: 8 total, 8 up, 8 in 2024-02-23T00:46:09.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:09.155792+0000 mon.a (mon.0) 2004 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-35"}]: dispatch 2024-02-23T00:46:09.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:09.156623+0000 mon.c (mon.2) 228 : audit [INF] from='client.? 172.21.15.47:0/297870923' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi047-82378-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi047-82378-41"}]: dispatch 2024-02-23T00:46:09.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:09.161593+0000 mon.a (mon.0) 2005 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi047-82378-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi047-82378-41"}]: dispatch 2024-02-23T00:46:10.081 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:46:10.084 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:10.084 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:46:10.185 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:46:10.214 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:10.234 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:46:10.236 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:10.237 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:46:10.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.264+0000 7f3711d45700 1 -- 172.21.15.47:0/3334054274 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f370c0f89a0 msgr2=0x7f370c0f8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:10.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.264+0000 7f3711d45700 1 --2- 172.21.15.47:0/3334054274 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f370c0f89a0 0x7f370c0f8d80 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 rx=0x7f36fc004870 tx=0x7f36fc009cb0).stop 2024-02-23T00:46:10.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.264+0000 7f3711d45700 1 -- 172.21.15.47:0/3334054274 shutdown_connections 2024-02-23T00:46:10.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.264+0000 7f3711d45700 1 --2- 172.21.15.47:0/3334054274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f370c0f7330 0x7f370c103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:10.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.264+0000 7f3711d45700 1 --2- 172.21.15.47:0/3334054274 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f370c0f89a0 0x7f370c0f8d80 unknown :-1 s=CLOSED pgs=2603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:10.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.264+0000 7f3711d45700 1 --2- 172.21.15.47:0/3334054274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f370c0f9270 0x7f370c0f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:10.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.264+0000 7f3711d45700 1 -- 172.21.15.47:0/3334054274 >> 172.21.15.47:0/3334054274 conn(0x7f370c0f26f0 msgr2=0x7f370c0f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:46:10.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.264+0000 7f3711d45700 1 -- 172.21.15.47:0/3334054274 shutdown_connections 2024-02-23T00:46:10.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.264+0000 7f3711d45700 1 -- 172.21.15.47:0/3334054274 wait complete. 2024-02-23T00:46:10.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.264+0000 7f3711d45700 1 librados: starting msgr at 2024-02-23T00:46:10.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.264+0000 7f3711d45700 1 librados: starting objecter 2024-02-23T00:46:10.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.264+0000 7f3711d45700 1 Processor -- start 2024-02-23T00:46:10.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.265+0000 7f3711d45700 1 -- start start 2024-02-23T00:46:10.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.265+0000 7f3711d45700 1 librados: setting wanted keys 2024-02-23T00:46:10.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.265+0000 7f3711d45700 1 librados: calling monclient init 2024-02-23T00:46:10.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.265+0000 7f3711d45700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f370c0f7330 0x7f370c1090c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:10.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.265+0000 7f3711d45700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f370c0f89a0 0x7f370c1095b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:10.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.265+0000 7f3711d45700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f370c0f9270 0x7f370c109b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:10.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.265+0000 7f3711d45700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c1142a0 con 0x7f370c0f9270 2024-02-23T00:46:10.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.265+0000 7f3711d45700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c114410 con 0x7f370c0f7330 2024-02-23T00:46:10.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.265+0000 7f3711d45700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f370c1145d0 con 0x7f370c0f89a0 2024-02-23T00:46:10.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.265+0000 7f370affd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f370c0f89a0 0x7f370c1095b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:10.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.265+0000 7f370affd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f370c0f89a0 0x7f370c1095b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:36860/0 (socket says 172.21.15.47:36860) 2024-02-23T00:46:10.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.265+0000 7f370affd700 1 -- 172.21.15.47:0/3408953682 learned_addr learned my addr 172.21.15.47:0/3408953682 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:46:10.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.265+0000 7f370bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f370c0f9270 0x7f370c109b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:10.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.265+0000 7f370b7fe700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f370c0f7330 0x7f370c1090c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:10.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.265+0000 7f370affd700 1 -- 172.21.15.47:0/3408953682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f370c0f7330 msgr2=0x7f370c1090c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:10.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.265+0000 7f370affd700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f370c0f7330 0x7f370c1090c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:10.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.265+0000 7f370affd700 1 -- 172.21.15.47:0/3408953682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f370c0f9270 msgr2=0x7f370c109b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:10.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.265+0000 7f370affd700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f370c0f9270 0x7f370c109b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:10.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.265+0000 7f370affd700 1 -- 172.21.15.47:0/3408953682 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36fc005040 con 0x7f370c0f89a0 2024-02-23T00:46:10.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.266+0000 7f370affd700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f370c0f89a0 0x7f370c1095b0 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 rx=0x7f36f400e970 tx=0x7f36f400b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:10.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.266+0000 7f3708ff9700 10 client.?.objecter ms_handle_connect 0x7f370c0f89a0 2024-02-23T00:46:10.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.266+0000 7f3708ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:46:10.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.266+0000 7f3708ff9700 1 -- 172.21.15.47:0/3408953682 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f400ee60 con 0x7f370c0f89a0 2024-02-23T00:46:10.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.266+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f370c1148b0 con 0x7f370c0f89a0 2024-02-23T00:46:10.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.266+0000 7f3708ff9700 1 -- 172.21.15.47:0/3408953682 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f36f400f040 con 0x7f370c0f89a0 2024-02-23T00:46:10.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.266+0000 7f3708ff9700 1 -- 172.21.15.47:0/3408953682 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f400c5b0 con 0x7f370c0f89a0 2024-02-23T00:46:10.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.266+0000 7f3711d45700 10 client.41045.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:46:10.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.266+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f370c114e70 con 0x7f370c0f89a0 2024-02-23T00:46:10.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.266+0000 7f3711d45700 1 librados: init done 2024-02-23T00:46:10.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.267+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f370c05ee80 con 0x7f370c0f89a0 2024-02-23T00:46:10.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.268+0000 7f3708ff9700 1 -- 172.21.15.47:0/3408953682 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f36f4009090 con 0x7f370c0f89a0 2024-02-23T00:46:10.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.268+0000 7f3708ff9700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f36f8062cc0 0x7f36f8065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:10.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.268+0000 7f3708ff9700 1 -- 172.21.15.47:0/3408953682 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(298..298 src has 48..298) v4 ==== 10820+0+0 (secure 0 0 0) 0x7f36f407dea0 con 0x7f370c0f89a0 2024-02-23T00:46:10.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.268+0000 7f3708ff9700 10 client.41045.objecter ms_dispatch 0x7f370c001550 osd_map(298..298 src has 48..298) v4 2024-02-23T00:46:10.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.268+0000 7f3708ff9700 3 client.41045.objecter handle_osd_map got epochs [298,298] > 0 2024-02-23T00:46:10.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.268+0000 7f3708ff9700 3 client.41045.objecter handle_osd_map decoding full epoch 298 2024-02-23T00:46:10.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.268+0000 7f370b7fe700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f36f8062cc0 0x7f36f8065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:10.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.268+0000 7f3708ff9700 20 client.41045.objecter dump_active .. 0 homeless 2024-02-23T00:46:10.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.268+0000 7f3708ff9700 1 -- 172.21.15.47:0/3408953682 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v298) v1 ==== 54+0+16 (secure 0 0 0) 0x7f36f407e340 con 0x7f370c0f89a0 2024-02-23T00:46:10.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.268+0000 7f3708ff9700 10 client.41045.objecter ms_handle_connect 0x7f36f8062cc0 2024-02-23T00:46:10.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.268+0000 7f370b7fe700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f36f8062cc0 0x7f36f8065180 secure :-1 s=READY pgs=3013 cs=0 l=1 rev1=1 rx=0x7f36fc009da0 tx=0x7f36fc00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:10.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.268+0000 7f3711d45700 10 client.41045.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:10.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.268+0000 7f3711d45700 15 client.41045.objecter _session_command_op_assign -1 1 2024-02-23T00:46:10.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.268+0000 7f3711d45700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f370c028540 0x7f370c0fd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:10.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.268+0000 7f3711d45700 20 client.41045.objecter _get_session s=0x7f370c105e40 osd=0 3 2024-02-23T00:46:10.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.268+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c105e40 osd=0 3 2024-02-23T00:46:10.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.268+0000 7f3711d45700 20 client.41045.objecter _get_session s=0x7f370c105e40 osd=0 3 2024-02-23T00:46:10.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.268+0000 7f3711d45700 15 client.41045.objecter _session_command_op_remove -1 1 2024-02-23T00:46:10.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.268+0000 7f3711d45700 20 client.41045.objecter get_session s=0x7f370c105e40 osd=0 3 2024-02-23T00:46:10.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.268+0000 7f3711d45700 15 client.41045.objecter _session_command_op_assign 0 1 2024-02-23T00:46:10.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.268+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c105e40 osd=0 4 2024-02-23T00:46:10.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.268+0000 7f3711d45700 10 client.41045.objecter _send_command 1 2024-02-23T00:46:10.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.268+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f370c05a040 con 0x7f370c028540 2024-02-23T00:46:10.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.269+0000 7f370bfff700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f370c028540 0x7f370c0fd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:10.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.269+0000 7f3708ff9700 10 client.41045.objecter ms_handle_connect 0x7f370c028540 2024-02-23T00:46:10.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.269+0000 7f370bfff700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f370c028540 0x7f370c0fd890 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:10.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.269+0000 7f3708ff9700 1 -- 172.21.15.47:0/3408953682 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f370c05a040 con 0x7f370c028540 2024-02-23T00:46:10.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.269+0000 7f3708ff9700 10 client.41045.objecter ms_dispatch 0x7f370c001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:46:10.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.269+0000 7f3708ff9700 10 client.41045.objecter _finish_command 1 = system:0 2024-02-23T00:46:10.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.269+0000 7f3708ff9700 20 client.41045.objecter put_session s=0x7f370c105e40 osd=0 4 2024-02-23T00:46:10.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.269+0000 7f3708ff9700 15 client.41045.objecter _session_command_op_remove 0 1 2024-02-23T00:46:10.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.285+0000 7f3711d45700 10 client.41045.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:46:10.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.285+0000 7f3711d45700 15 client.41045.objecter _session_command_op_assign -1 2 2024-02-23T00:46:10.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.285+0000 7f3711d45700 20 client.41045.objecter _get_session s=0x7f370c105e40 osd=0 3 2024-02-23T00:46:10.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.285+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c105e40 osd=0 3 2024-02-23T00:46:10.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.285+0000 7f3711d45700 20 client.41045.objecter _get_session s=0x7f370c105e40 osd=0 3 2024-02-23T00:46:10.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.285+0000 7f3711d45700 15 client.41045.objecter _session_command_op_remove -1 2 2024-02-23T00:46:10.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.285+0000 7f3711d45700 20 client.41045.objecter get_session s=0x7f370c105e40 osd=0 3 2024-02-23T00:46:10.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.285+0000 7f3711d45700 15 client.41045.objecter _session_command_op_assign 0 2 2024-02-23T00:46:10.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.285+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c105e40 osd=0 4 2024-02-23T00:46:10.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.285+0000 7f3711d45700 10 client.41045.objecter _send_command 2 2024-02-23T00:46:10.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.285+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f370c0fdd80 con 0x7f370c028540 2024-02-23T00:46:10.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.286+0000 7f3708ff9700 1 -- 172.21.15.47:0/3408953682 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f370c0fdd80 con 0x7f370c028540 2024-02-23T00:46:10.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.286+0000 7f3708ff9700 10 client.41045.objecter ms_dispatch 0x7f370c001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:46:10.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.286+0000 7f3708ff9700 10 client.41045.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:46:10.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.286+0000 7f3708ff9700 20 client.41045.objecter put_session s=0x7f370c105e40 osd=0 4 2024-02-23T00:46:10.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.286+0000 7f3708ff9700 15 client.41045.objecter _session_command_op_remove 0 2 2024-02-23T00:46:10.289 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:46:10.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.286+0000 7f3711d45700 10 client.41045.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:10.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.286+0000 7f3711d45700 15 client.41045.objecter _session_command_op_assign -1 3 2024-02-23T00:46:10.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.286+0000 7f3711d45700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f370c059710 0x7f370c119020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:10.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.286+0000 7f3711d45700 20 client.41045.objecter _get_session s=0x7f370c0595a0 osd=1 3 2024-02-23T00:46:10.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.286+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c0595a0 osd=1 3 2024-02-23T00:46:10.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.286+0000 7f3711d45700 20 client.41045.objecter _get_session s=0x7f370c0595a0 osd=1 3 2024-02-23T00:46:10.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.286+0000 7f3711d45700 15 client.41045.objecter _session_command_op_remove -1 3 2024-02-23T00:46:10.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.286+0000 7f3711d45700 20 client.41045.objecter get_session s=0x7f370c0595a0 osd=1 3 2024-02-23T00:46:10.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.286+0000 7f3711d45700 15 client.41045.objecter _session_command_op_assign 1 3 2024-02-23T00:46:10.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.286+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c0595a0 osd=1 4 2024-02-23T00:46:10.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.286+0000 7f3711d45700 10 client.41045.objecter _send_command 3 2024-02-23T00:46:10.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.286+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f370c0280a0 con 0x7f370c059710 2024-02-23T00:46:10.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.287+0000 7f370b7fe700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f370c059710 0x7f370c119020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:10.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.287+0000 7f3708ff9700 10 client.41045.objecter ms_handle_connect 0x7f370c059710 2024-02-23T00:46:10.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.287+0000 7f370b7fe700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f370c059710 0x7f370c119020 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:10.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.287+0000 7f3708ff9700 1 -- 172.21.15.47:0/3408953682 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f370c0280a0 con 0x7f370c059710 2024-02-23T00:46:10.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.287+0000 7f3708ff9700 10 client.41045.objecter ms_dispatch 0x7f370c001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:46:10.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.287+0000 7f3708ff9700 10 client.41045.objecter _finish_command 3 = system:0 2024-02-23T00:46:10.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.287+0000 7f3708ff9700 20 client.41045.objecter put_session s=0x7f370c0595a0 osd=1 4 2024-02-23T00:46:10.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.287+0000 7f3708ff9700 15 client.41045.objecter _session_command_op_remove 1 3 2024-02-23T00:46:10.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.302+0000 7f3711d45700 10 client.41045.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:46:10.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.302+0000 7f3711d45700 15 client.41045.objecter _session_command_op_assign -1 4 2024-02-23T00:46:10.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.302+0000 7f3711d45700 20 client.41045.objecter _get_session s=0x7f370c0595a0 osd=1 3 2024-02-23T00:46:10.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.302+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c0595a0 osd=1 3 2024-02-23T00:46:10.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.302+0000 7f3711d45700 20 client.41045.objecter _get_session s=0x7f370c0595a0 osd=1 3 2024-02-23T00:46:10.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.302+0000 7f3711d45700 15 client.41045.objecter _session_command_op_remove -1 4 2024-02-23T00:46:10.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.302+0000 7f3711d45700 20 client.41045.objecter get_session s=0x7f370c0595a0 osd=1 3 2024-02-23T00:46:10.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.302+0000 7f3711d45700 15 client.41045.objecter _session_command_op_assign 1 4 2024-02-23T00:46:10.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.302+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c0595a0 osd=1 4 2024-02-23T00:46:10.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.302+0000 7f3711d45700 10 client.41045.objecter _send_command 4 2024-02-23T00:46:10.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.302+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f370c119510 con 0x7f370c059710 2024-02-23T00:46:10.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.304+0000 7f3708ff9700 1 -- 172.21.15.47:0/3408953682 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f370c119510 con 0x7f370c059710 2024-02-23T00:46:10.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.304+0000 7f3708ff9700 10 client.41045.objecter ms_dispatch 0x7f370c001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:46:10.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.304+0000 7f3708ff9700 10 client.41045.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:46:10.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.304+0000 7f3708ff9700 20 client.41045.objecter put_session s=0x7f370c0595a0 osd=1 4 2024-02-23T00:46:10.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.304+0000 7f3708ff9700 15 client.41045.objecter _session_command_op_remove 1 4 2024-02-23T00:46:10.307 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:46:10.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.304+0000 7f3711d45700 10 client.41045.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:10.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.304+0000 7f3711d45700 15 client.41045.objecter _session_command_op_assign -1 5 2024-02-23T00:46:10.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.304+0000 7f3711d45700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f370c027770 0x7f370c0fec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:10.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.304+0000 7f3711d45700 20 client.41045.objecter _get_session s=0x7f370c027600 osd=2 3 2024-02-23T00:46:10.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.304+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c027600 osd=2 3 2024-02-23T00:46:10.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.304+0000 7f3711d45700 20 client.41045.objecter _get_session s=0x7f370c027600 osd=2 3 2024-02-23T00:46:10.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.304+0000 7f3711d45700 15 client.41045.objecter _session_command_op_remove -1 5 2024-02-23T00:46:10.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.304+0000 7f3711d45700 20 client.41045.objecter get_session s=0x7f370c027600 osd=2 3 2024-02-23T00:46:10.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.304+0000 7f3711d45700 15 client.41045.objecter _session_command_op_assign 2 5 2024-02-23T00:46:10.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.304+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c027600 osd=2 4 2024-02-23T00:46:10.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.304+0000 7f3711d45700 10 client.41045.objecter _send_command 5 2024-02-23T00:46:10.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.304+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f370c11efd0 con 0x7f370c027770 2024-02-23T00:46:10.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.304+0000 7f370affd700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f370c027770 0x7f370c0fec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:10.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.304+0000 7f3708ff9700 10 client.41045.objecter ms_handle_connect 0x7f370c027770 2024-02-23T00:46:10.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.304+0000 7f370affd700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f370c027770 0x7f370c0fec30 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:10.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.305+0000 7f3708ff9700 1 -- 172.21.15.47:0/3408953682 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f370c11efd0 con 0x7f370c027770 2024-02-23T00:46:10.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.305+0000 7f3708ff9700 10 client.41045.objecter ms_dispatch 0x7f370c001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:46:10.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.305+0000 7f3708ff9700 10 client.41045.objecter _finish_command 5 = system:0 2024-02-23T00:46:10.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.305+0000 7f3708ff9700 20 client.41045.objecter put_session s=0x7f370c027600 osd=2 4 2024-02-23T00:46:10.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.305+0000 7f3708ff9700 15 client.41045.objecter _session_command_op_remove 2 5 2024-02-23T00:46:10.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.319+0000 7f3711d45700 10 client.41045.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:46:10.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.319+0000 7f3711d45700 15 client.41045.objecter _session_command_op_assign -1 6 2024-02-23T00:46:10.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.319+0000 7f3711d45700 20 client.41045.objecter _get_session s=0x7f370c027600 osd=2 3 2024-02-23T00:46:10.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.319+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c027600 osd=2 3 2024-02-23T00:46:10.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.319+0000 7f3711d45700 20 client.41045.objecter _get_session s=0x7f370c027600 osd=2 3 2024-02-23T00:46:10.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.319+0000 7f3711d45700 15 client.41045.objecter _session_command_op_remove -1 6 2024-02-23T00:46:10.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.319+0000 7f3711d45700 20 client.41045.objecter get_session s=0x7f370c027600 osd=2 3 2024-02-23T00:46:10.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.319+0000 7f3711d45700 15 client.41045.objecter _session_command_op_assign 2 6 2024-02-23T00:46:10.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.319+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c027600 osd=2 4 2024-02-23T00:46:10.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.319+0000 7f3711d45700 10 client.41045.objecter _send_command 6 2024-02-23T00:46:10.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.319+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f370c0fe190 con 0x7f370c027770 2024-02-23T00:46:10.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.321+0000 7f3708ff9700 1 -- 172.21.15.47:0/3408953682 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f370c0fe190 con 0x7f370c027770 2024-02-23T00:46:10.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.321+0000 7f3708ff9700 10 client.41045.objecter ms_dispatch 0x7f370c001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:46:10.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.321+0000 7f3708ff9700 10 client.41045.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:46:10.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.321+0000 7f3708ff9700 20 client.41045.objecter put_session s=0x7f370c027600 osd=2 4 2024-02-23T00:46:10.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.321+0000 7f3708ff9700 15 client.41045.objecter _session_command_op_remove 2 6 2024-02-23T00:46:10.324 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:46:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.321+0000 7f3711d45700 10 client.41045.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.321+0000 7f3711d45700 15 client.41045.objecter _session_command_op_assign -1 7 2024-02-23T00:46:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.321+0000 7f3711d45700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f370c11a360 0x7f370c11d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.321+0000 7f3711d45700 20 client.41045.objecter _get_session s=0x7f370c0fe5a0 osd=3 3 2024-02-23T00:46:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.321+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c0fe5a0 osd=3 3 2024-02-23T00:46:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.321+0000 7f3711d45700 20 client.41045.objecter _get_session s=0x7f370c0fe5a0 osd=3 3 2024-02-23T00:46:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.321+0000 7f3711d45700 15 client.41045.objecter _session_command_op_remove -1 7 2024-02-23T00:46:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.321+0000 7f3711d45700 20 client.41045.objecter get_session s=0x7f370c0fe5a0 osd=3 3 2024-02-23T00:46:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.321+0000 7f3711d45700 15 client.41045.objecter _session_command_op_assign 3 7 2024-02-23T00:46:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.321+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c0fe5a0 osd=3 4 2024-02-23T00:46:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.321+0000 7f3711d45700 10 client.41045.objecter _send_command 7 2024-02-23T00:46:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.321+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f370c11df70 con 0x7f370c11a360 2024-02-23T00:46:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.321+0000 7f370bfff700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f370c11a360 0x7f370c11d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.321+0000 7f370bfff700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f370c11a360 0x7f370c11d9f0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.321+0000 7f3708ff9700 10 client.41045.objecter ms_handle_connect 0x7f370c11a360 2024-02-23T00:46:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.322+0000 7f3708ff9700 1 -- 172.21.15.47:0/3408953682 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f370c11df70 con 0x7f370c11a360 2024-02-23T00:46:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.322+0000 7f3708ff9700 10 client.41045.objecter ms_dispatch 0x7f370c001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:46:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.322+0000 7f3708ff9700 10 client.41045.objecter _finish_command 7 = system:0 2024-02-23T00:46:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.322+0000 7f3708ff9700 20 client.41045.objecter put_session s=0x7f370c0fe5a0 osd=3 4 2024-02-23T00:46:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.322+0000 7f3708ff9700 15 client.41045.objecter _session_command_op_remove 3 7 2024-02-23T00:46:10.337 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:46:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.337+0000 7f3711d45700 10 client.41045.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:46:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.337+0000 7f3711d45700 15 client.41045.objecter _session_command_op_assign -1 8 2024-02-23T00:46:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.337+0000 7f3711d45700 20 client.41045.objecter _get_session s=0x7f370c0fe5a0 osd=3 3 2024-02-23T00:46:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.337+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c0fe5a0 osd=3 3 2024-02-23T00:46:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.337+0000 7f3711d45700 20 client.41045.objecter _get_session s=0x7f370c0fe5a0 osd=3 3 2024-02-23T00:46:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.337+0000 7f3711d45700 15 client.41045.objecter _session_command_op_remove -1 8 2024-02-23T00:46:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.337+0000 7f3711d45700 20 client.41045.objecter get_session s=0x7f370c0fe5a0 osd=3 3 2024-02-23T00:46:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.337+0000 7f3711d45700 15 client.41045.objecter _session_command_op_assign 3 8 2024-02-23T00:46:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.337+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c0fe5a0 osd=3 4 2024-02-23T00:46:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.337+0000 7f3711d45700 10 client.41045.objecter _send_command 8 2024-02-23T00:46:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.337+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f370c1198c0 con 0x7f370c11a360 2024-02-23T00:46:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.339+0000 7f3708ff9700 1 -- 172.21.15.47:0/3408953682 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f370c1198c0 con 0x7f370c11a360 2024-02-23T00:46:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.339+0000 7f3708ff9700 10 client.41045.objecter ms_dispatch 0x7f370c001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:46:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.339+0000 7f3708ff9700 10 client.41045.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:46:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.339+0000 7f3708ff9700 20 client.41045.objecter put_session s=0x7f370c0fe5a0 osd=3 4 2024-02-23T00:46:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.339+0000 7f3708ff9700 15 client.41045.objecter _session_command_op_remove 3 8 2024-02-23T00:46:10.342 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:46:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.339+0000 7f3711d45700 10 client.41045.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.339+0000 7f3711d45700 15 client.41045.objecter _session_command_op_assign -1 9 2024-02-23T00:46:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.339+0000 7f3711d45700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f370c121150 0x7f370c123540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.339+0000 7f3711d45700 20 client.41045.objecter _get_session s=0x7f370c119680 osd=4 3 2024-02-23T00:46:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.339+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c119680 osd=4 3 2024-02-23T00:46:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.339+0000 7f3711d45700 20 client.41045.objecter _get_session s=0x7f370c119680 osd=4 3 2024-02-23T00:46:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.339+0000 7f3711d45700 15 client.41045.objecter _session_command_op_remove -1 9 2024-02-23T00:46:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.339+0000 7f3711d45700 20 client.41045.objecter get_session s=0x7f370c119680 osd=4 3 2024-02-23T00:46:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.339+0000 7f3711d45700 15 client.41045.objecter _session_command_op_assign 4 9 2024-02-23T00:46:10.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.339+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c119680 osd=4 4 2024-02-23T00:46:10.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.339+0000 7f3711d45700 10 client.41045.objecter _send_command 9 2024-02-23T00:46:10.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.339+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f370c1197f0 con 0x7f370c121150 2024-02-23T00:46:10.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.339+0000 7f370b7fe700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f370c121150 0x7f370c123540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:10.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.340+0000 7f3708ff9700 10 client.41045.objecter ms_handle_connect 0x7f370c121150 2024-02-23T00:46:10.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.340+0000 7f370b7fe700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f370c121150 0x7f370c123540 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:10.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.341+0000 7f3708ff9700 1 -- 172.21.15.47:0/3408953682 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f370c1197f0 con 0x7f370c121150 2024-02-23T00:46:10.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.341+0000 7f3708ff9700 10 client.41045.objecter ms_dispatch 0x7f370c001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:46:10.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.341+0000 7f3708ff9700 10 client.41045.objecter _finish_command 9 = system:0 2024-02-23T00:46:10.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.341+0000 7f3708ff9700 20 client.41045.objecter put_session s=0x7f370c119680 osd=4 4 2024-02-23T00:46:10.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.341+0000 7f3708ff9700 15 client.41045.objecter _session_command_op_remove 4 9 2024-02-23T00:46:10.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.355+0000 7f3711d45700 10 client.41045.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:46:10.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.355+0000 7f3711d45700 15 client.41045.objecter _session_command_op_assign -1 10 2024-02-23T00:46:10.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.355+0000 7f3711d45700 20 client.41045.objecter _get_session s=0x7f370c119680 osd=4 3 2024-02-23T00:46:10.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.355+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c119680 osd=4 3 2024-02-23T00:46:10.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.355+0000 7f3711d45700 20 client.41045.objecter _get_session s=0x7f370c119680 osd=4 3 2024-02-23T00:46:10.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.355+0000 7f3711d45700 15 client.41045.objecter _session_command_op_remove -1 10 2024-02-23T00:46:10.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.355+0000 7f3711d45700 20 client.41045.objecter get_session s=0x7f370c119680 osd=4 3 2024-02-23T00:46:10.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.355+0000 7f3711d45700 15 client.41045.objecter _session_command_op_assign 4 10 2024-02-23T00:46:10.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.355+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c119680 osd=4 4 2024-02-23T00:46:10.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.355+0000 7f3711d45700 10 client.41045.objecter _send_command 10 2024-02-23T00:46:10.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.355+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f370c1197f0 con 0x7f370c121150 2024-02-23T00:46:10.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.357+0000 7f3708ff9700 1 -- 172.21.15.47:0/3408953682 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f370c1197f0 con 0x7f370c121150 2024-02-23T00:46:10.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.357+0000 7f3708ff9700 10 client.41045.objecter ms_dispatch 0x7f370c001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:46:10.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.357+0000 7f3708ff9700 10 client.41045.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:46:10.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.357+0000 7f3708ff9700 20 client.41045.objecter put_session s=0x7f370c119680 osd=4 4 2024-02-23T00:46:10.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.357+0000 7f3708ff9700 15 client.41045.objecter _session_command_op_remove 4 10 2024-02-23T00:46:10.360 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:46:10.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.357+0000 7f3711d45700 10 client.41045.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:10.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.357+0000 7f3711d45700 15 client.41045.objecter _session_command_op_assign -1 11 2024-02-23T00:46:10.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.357+0000 7f3711d45700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f370c123f40 0x7f370c126330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:10.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.357+0000 7f3711d45700 20 client.41045.objecter _get_session s=0x7f370c1197f0 osd=5 3 2024-02-23T00:46:10.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.357+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c1197f0 osd=5 3 2024-02-23T00:46:10.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.357+0000 7f3711d45700 20 client.41045.objecter _get_session s=0x7f370c1197f0 osd=5 3 2024-02-23T00:46:10.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.357+0000 7f3711d45700 15 client.41045.objecter _session_command_op_remove -1 11 2024-02-23T00:46:10.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.357+0000 7f3711d45700 20 client.41045.objecter get_session s=0x7f370c1197f0 osd=5 3 2024-02-23T00:46:10.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.357+0000 7f3711d45700 15 client.41045.objecter _session_command_op_assign 5 11 2024-02-23T00:46:10.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.357+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c1197f0 osd=5 4 2024-02-23T00:46:10.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.357+0000 7f3711d45700 10 client.41045.objecter _send_command 11 2024-02-23T00:46:10.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.357+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f370c106eb0 con 0x7f370c123f40 2024-02-23T00:46:10.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.358+0000 7f370affd700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f370c123f40 0x7f370c126330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:10.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.358+0000 7f3708ff9700 10 client.41045.objecter ms_handle_connect 0x7f370c123f40 2024-02-23T00:46:10.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.358+0000 7f370affd700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f370c123f40 0x7f370c126330 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:10.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.359+0000 7f3708ff9700 1 -- 172.21.15.47:0/3408953682 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f370c106eb0 con 0x7f370c123f40 2024-02-23T00:46:10.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.359+0000 7f3708ff9700 10 client.41045.objecter ms_dispatch 0x7f370c001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:46:10.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.359+0000 7f3708ff9700 10 client.41045.objecter _finish_command 11 = system:0 2024-02-23T00:46:10.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.359+0000 7f3708ff9700 20 client.41045.objecter put_session s=0x7f370c1197f0 osd=5 4 2024-02-23T00:46:10.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.359+0000 7f3708ff9700 15 client.41045.objecter _session_command_op_remove 5 11 2024-02-23T00:46:10.366 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:10.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.373+0000 7f3711d45700 10 client.41045.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:46:10.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.373+0000 7f3711d45700 15 client.41045.objecter _session_command_op_assign -1 12 2024-02-23T00:46:10.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.373+0000 7f3711d45700 20 client.41045.objecter _get_session s=0x7f370c1197f0 osd=5 3 2024-02-23T00:46:10.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.373+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c1197f0 osd=5 3 2024-02-23T00:46:10.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.373+0000 7f3711d45700 20 client.41045.objecter _get_session s=0x7f370c1197f0 osd=5 3 2024-02-23T00:46:10.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.373+0000 7f3711d45700 15 client.41045.objecter _session_command_op_remove -1 12 2024-02-23T00:46:10.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.373+0000 7f3711d45700 20 client.41045.objecter get_session s=0x7f370c1197f0 osd=5 3 2024-02-23T00:46:10.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.373+0000 7f3711d45700 15 client.41045.objecter _session_command_op_assign 5 12 2024-02-23T00:46:10.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.373+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c1197f0 osd=5 4 2024-02-23T00:46:10.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.373+0000 7f3711d45700 10 client.41045.objecter _send_command 12 2024-02-23T00:46:10.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.373+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f370c106eb0 con 0x7f370c123f40 2024-02-23T00:46:10.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.375+0000 7f3708ff9700 1 -- 172.21.15.47:0/3408953682 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f370c106eb0 con 0x7f370c123f40 2024-02-23T00:46:10.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.375+0000 7f3708ff9700 10 client.41045.objecter ms_dispatch 0x7f370c001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:46:10.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.375+0000 7f3708ff9700 10 client.41045.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:46:10.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.375+0000 7f3708ff9700 20 client.41045.objecter put_session s=0x7f370c1197f0 osd=5 4 2024-02-23T00:46:10.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.375+0000 7f3708ff9700 15 client.41045.objecter _session_command_op_remove 5 12 2024-02-23T00:46:10.378 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:46:10.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.375+0000 7f3711d45700 10 client.41045.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:10.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.376+0000 7f3711d45700 15 client.41045.objecter _session_command_op_assign -1 13 2024-02-23T00:46:10.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.376+0000 7f3711d45700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f370c1283e0 0x7f370c128860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:10.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.376+0000 7f3711d45700 20 client.41045.objecter _get_session s=0x7f370c106eb0 osd=6 3 2024-02-23T00:46:10.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.376+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c106eb0 osd=6 3 2024-02-23T00:46:10.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.376+0000 7f3711d45700 20 client.41045.objecter _get_session s=0x7f370c106eb0 osd=6 3 2024-02-23T00:46:10.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.376+0000 7f3711d45700 15 client.41045.objecter _session_command_op_remove -1 13 2024-02-23T00:46:10.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.376+0000 7f3711d45700 20 client.41045.objecter get_session s=0x7f370c106eb0 osd=6 3 2024-02-23T00:46:10.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.376+0000 7f3711d45700 15 client.41045.objecter _session_command_op_assign 6 13 2024-02-23T00:46:10.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.376+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c106eb0 osd=6 4 2024-02-23T00:46:10.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.376+0000 7f3711d45700 10 client.41045.objecter _send_command 13 2024-02-23T00:46:10.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.376+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f370c12e640 con 0x7f370c1283e0 2024-02-23T00:46:10.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.376+0000 7f370bfff700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f370c1283e0 0x7f370c128860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:10.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.376+0000 7f370bfff700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f370c1283e0 0x7f370c128860 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:10.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.376+0000 7f3708ff9700 10 client.41045.objecter ms_handle_connect 0x7f370c1283e0 2024-02-23T00:46:10.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.377+0000 7f3708ff9700 1 -- 172.21.15.47:0/3408953682 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f370c12e640 con 0x7f370c1283e0 2024-02-23T00:46:10.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.377+0000 7f3708ff9700 10 client.41045.objecter ms_dispatch 0x7f370c001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:46:10.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.377+0000 7f3708ff9700 10 client.41045.objecter _finish_command 13 = system:0 2024-02-23T00:46:10.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.377+0000 7f3708ff9700 20 client.41045.objecter put_session s=0x7f370c106eb0 osd=6 4 2024-02-23T00:46:10.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.377+0000 7f3708ff9700 15 client.41045.objecter _session_command_op_remove 6 13 2024-02-23T00:46:10.385 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:46:10.388 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:10.388 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:46:10.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.391+0000 7f3711d45700 10 client.41045.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:46:10.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.391+0000 7f3711d45700 15 client.41045.objecter _session_command_op_assign -1 14 2024-02-23T00:46:10.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.391+0000 7f3711d45700 20 client.41045.objecter _get_session s=0x7f370c106eb0 osd=6 3 2024-02-23T00:46:10.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.391+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c106eb0 osd=6 3 2024-02-23T00:46:10.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.391+0000 7f3711d45700 20 client.41045.objecter _get_session s=0x7f370c106eb0 osd=6 3 2024-02-23T00:46:10.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.391+0000 7f3711d45700 15 client.41045.objecter _session_command_op_remove -1 14 2024-02-23T00:46:10.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.391+0000 7f3711d45700 20 client.41045.objecter get_session s=0x7f370c106eb0 osd=6 3 2024-02-23T00:46:10.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.391+0000 7f3711d45700 15 client.41045.objecter _session_command_op_assign 6 14 2024-02-23T00:46:10.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.391+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c106eb0 osd=6 4 2024-02-23T00:46:10.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.391+0000 7f3711d45700 10 client.41045.objecter _send_command 14 2024-02-23T00:46:10.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.391+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f370c126870 con 0x7f370c1283e0 2024-02-23T00:46:10.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.393+0000 7f3708ff9700 1 -- 172.21.15.47:0/3408953682 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f370c126870 con 0x7f370c1283e0 2024-02-23T00:46:10.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.393+0000 7f3708ff9700 10 client.41045.objecter ms_dispatch 0x7f370c001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:46:10.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.393+0000 7f3708ff9700 10 client.41045.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:46:10.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.393+0000 7f3708ff9700 20 client.41045.objecter put_session s=0x7f370c106eb0 osd=6 4 2024-02-23T00:46:10.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.393+0000 7f3708ff9700 15 client.41045.objecter _session_command_op_remove 6 14 2024-02-23T00:46:10.396 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:46:10.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.393+0000 7f3711d45700 10 client.41045.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:10.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.393+0000 7f3711d45700 15 client.41045.objecter _session_command_op_assign -1 15 2024-02-23T00:46:10.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.393+0000 7f3711d45700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f370c126ef0 0x7f370c127370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:10.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.393+0000 7f3711d45700 20 client.41045.objecter _get_session s=0x7f370c126870 osd=7 3 2024-02-23T00:46:10.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.393+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c126870 osd=7 3 2024-02-23T00:46:10.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.393+0000 7f3711d45700 20 client.41045.objecter _get_session s=0x7f370c126870 osd=7 3 2024-02-23T00:46:10.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.393+0000 7f3711d45700 15 client.41045.objecter _session_command_op_remove -1 15 2024-02-23T00:46:10.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.393+0000 7f3711d45700 20 client.41045.objecter get_session s=0x7f370c126870 osd=7 3 2024-02-23T00:46:10.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.393+0000 7f3711d45700 15 client.41045.objecter _session_command_op_assign 7 15 2024-02-23T00:46:10.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.393+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c126870 osd=7 4 2024-02-23T00:46:10.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.393+0000 7f3711d45700 10 client.41045.objecter _send_command 15 2024-02-23T00:46:10.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.393+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f370c1279b0 con 0x7f370c126ef0 2024-02-23T00:46:10.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.394+0000 7f370b7fe700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f370c126ef0 0x7f370c127370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:10.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.394+0000 7f3708ff9700 10 client.41045.objecter ms_handle_connect 0x7f370c126ef0 2024-02-23T00:46:10.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.394+0000 7f370b7fe700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f370c126ef0 0x7f370c127370 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:10.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.395+0000 7f3708ff9700 1 -- 172.21.15.47:0/3408953682 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f370c1279b0 con 0x7f370c126ef0 2024-02-23T00:46:10.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.395+0000 7f3708ff9700 10 client.41045.objecter ms_dispatch 0x7f370c001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:46:10.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.395+0000 7f3708ff9700 10 client.41045.objecter _finish_command 15 = system:0 2024-02-23T00:46:10.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.395+0000 7f3708ff9700 20 client.41045.objecter put_session s=0x7f370c126870 osd=7 4 2024-02-23T00:46:10.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.395+0000 7f3708ff9700 15 client.41045.objecter _session_command_op_remove 7 15 2024-02-23T00:46:10.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.409+0000 7f3711d45700 10 client.41045.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:46:10.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.409+0000 7f3711d45700 15 client.41045.objecter _session_command_op_assign -1 16 2024-02-23T00:46:10.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.409+0000 7f3711d45700 20 client.41045.objecter _get_session s=0x7f370c126870 osd=7 3 2024-02-23T00:46:10.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.409+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c126870 osd=7 3 2024-02-23T00:46:10.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.409+0000 7f3711d45700 20 client.41045.objecter _get_session s=0x7f370c126870 osd=7 3 2024-02-23T00:46:10.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.409+0000 7f3711d45700 15 client.41045.objecter _session_command_op_remove -1 16 2024-02-23T00:46:10.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.409+0000 7f3711d45700 20 client.41045.objecter get_session s=0x7f370c126870 osd=7 3 2024-02-23T00:46:10.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.409+0000 7f3711d45700 15 client.41045.objecter _session_command_op_assign 7 16 2024-02-23T00:46:10.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.409+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c126870 osd=7 4 2024-02-23T00:46:10.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.409+0000 7f3711d45700 10 client.41045.objecter _send_command 16 2024-02-23T00:46:10.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.409+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f370c1279b0 con 0x7f370c126ef0 2024-02-23T00:46:10.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3708ff9700 1 -- 172.21.15.47:0/3408953682 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f370c1279b0 con 0x7f370c126ef0 2024-02-23T00:46:10.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3708ff9700 10 client.41045.objecter ms_dispatch 0x7f370c001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:46:10.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3708ff9700 10 client.41045.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:46:10.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3708ff9700 20 client.41045.objecter put_session s=0x7f370c126870 osd=7 4 2024-02-23T00:46:10.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3708ff9700 15 client.41045.objecter _session_command_op_remove 7 16 2024-02-23T00:46:10.414 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:46:10.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 10 librados: watch_flush enter 2024-02-23T00:46:10.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 10 librados: watch_flush exit 2024-02-23T00:46:10.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 10 client.41045.objecter close_session for osd.0 2024-02-23T00:46:10.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f370c028540 msgr2=0x7f370c0fd890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:10.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f370c028540 0x7f370c0fd890 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:10.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c105e40 osd=0 1 2024-02-23T00:46:10.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 10 client.41045.objecter close_session for osd.1 2024-02-23T00:46:10.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f370c059710 msgr2=0x7f370c119020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:10.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f370c059710 0x7f370c119020 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:10.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c0595a0 osd=1 1 2024-02-23T00:46:10.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 10 client.41045.objecter close_session for osd.2 2024-02-23T00:46:10.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f370c027770 msgr2=0x7f370c0fec30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:10.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f370c027770 0x7f370c0fec30 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:10.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c027600 osd=2 1 2024-02-23T00:46:10.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 10 client.41045.objecter close_session for osd.3 2024-02-23T00:46:10.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f370c11a360 msgr2=0x7f370c11d9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:10.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f370c11a360 0x7f370c11d9f0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:10.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f370bfff700 1 -- 172.21.15.47:0/3408953682 reap_dead start 2024-02-23T00:46:10.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c0fe5a0 osd=3 1 2024-02-23T00:46:10.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 10 client.41045.objecter close_session for osd.4 2024-02-23T00:46:10.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f370c121150 msgr2=0x7f370c123540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:10.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f370c121150 0x7f370c123540 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:10.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f370bfff700 1 -- 172.21.15.47:0/3408953682 reap_dead start 2024-02-23T00:46:10.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c119680 osd=4 1 2024-02-23T00:46:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 10 client.41045.objecter close_session for osd.5 2024-02-23T00:46:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f370c123f40 msgr2=0x7f370c126330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f370c123f40 0x7f370c126330 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c1197f0 osd=5 1 2024-02-23T00:46:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 10 client.41045.objecter close_session for osd.6 2024-02-23T00:46:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f370c1283e0 msgr2=0x7f370c128860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f370c1283e0 0x7f370c128860 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c106eb0 osd=6 1 2024-02-23T00:46:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 10 client.41045.objecter close_session for osd.7 2024-02-23T00:46:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f370c126ef0 msgr2=0x7f370c127370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f370c126ef0 0x7f370c127370 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 20 client.41045.objecter put_session s=0x7f370c126870 osd=7 1 2024-02-23T00:46:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 20 client.41045.objecter shutdown clearing up homeless session... 2024-02-23T00:46:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.411+0000 7f3711d45700 10 client.41045.objecter successfully canceled tick 2024-02-23T00:46:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.412+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f36f8062cc0 msgr2=0x7f36f8065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.412+0000 7f3711d45700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f36f8062cc0 0x7f36f8065180 secure :-1 s=READY pgs=3013 cs=0 l=1 rev1=1 rx=0x7f36fc009da0 tx=0x7f36fc00b040).stop 2024-02-23T00:46:10.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.412+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f370c0f89a0 msgr2=0x7f370c1095b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:10.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.412+0000 7f3711d45700 1 --2- 172.21.15.47:0/3408953682 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f370c0f89a0 0x7f370c1095b0 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 rx=0x7f36f400e970 tx=0x7f36f400b040).stop 2024-02-23T00:46:10.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.412+0000 7f370bfff700 1 -- 172.21.15.47:0/3408953682 reap_dead start 2024-02-23T00:46:10.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.412+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 shutdown_connections 2024-02-23T00:46:10.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.412+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 >> 172.21.15.47:0/3408953682 conn(0x7f370c0f26f0 msgr2=0x7f370c0fc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:46:10.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.412+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 shutdown_connections 2024-02-23T00:46:10.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.412+0000 7f3711d45700 1 -- 172.21.15.47:0/3408953682 wait complete. 2024-02-23T00:46:10.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:10.412+0000 7f3711d45700 1 librados: shutdown 2024-02-23T00:46:10.419 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:46:10.419 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:46:10.419 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:46:10.419 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:46:10.419 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:46:10.419 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:46:10.419 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:46:10.419 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:46:10.489 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:46:10.520 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:46:10.554 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:10.554 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:46:10.655 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:46:10.683 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:10.807 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:46:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:46:10] "GET /metrics HTTP/1.1" 200 190205 "" "Prometheus/2.33.4" 2024-02-23T00:46:10.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:08.791473+0000 osd.7 (osd.7) 159 : cluster [DBG] 2.1d scrub starts 2024-02-23T00:46:10.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:08.794347+0000 osd.7 (osd.7) 160 : cluster [DBG] 2.1d scrub ok 2024-02-23T00:46:10.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:09.252388+0000 mgr.y (mgr.14517) 300 : cluster [DBG] pgmap v421: 253 pgs: 8 unknown, 245 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:10.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:09.498026+0000 osd.0 (osd.0) 127 : cluster [DBG] 62.13 scrub starts 2024-02-23T00:46:10.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:09.499803+0000 osd.0 (osd.0) 128 : cluster [DBG] 62.13 scrub ok 2024-02-23T00:46:10.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:09.923194+0000 osd.2 (osd.2) 137 : cluster [DBG] 5.6 scrub starts 2024-02-23T00:46:10.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:09.924380+0000 osd.2 (osd.2) 138 : cluster [DBG] 5.6 scrub ok 2024-02-23T00:46:10.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:10.010061+0000 osd.6 (osd.6) 121 : cluster [DBG] 2.1a scrub starts 2024-02-23T00:46:10.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:10.010991+0000 osd.6 (osd.6) 122 : cluster [DBG] 2.1a scrub ok 2024-02-23T00:46:10.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:10.140637+0000 mon.a (mon.0) 2006 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:46:10.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:10.145361+0000 mon.a (mon.0) 2007 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-35"}]': finished 2024-02-23T00:46:10.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:10.268806+0000 mon.c (mon.2) 229 : audit [DBG] from='client.? 172.21.15.47:0/3408953682' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:46:10.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:10.586908+0000 mon.a (mon.0) 2008 : cluster [DBG] osdmap e298: 8 total, 8 up, 8 in 2024-02-23T00:46:10.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:08.791473+0000 osd.7 (osd.7) 159 : cluster [DBG] 2.1d scrub starts 2024-02-23T00:46:10.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:08.794347+0000 osd.7 (osd.7) 160 : cluster [DBG] 2.1d scrub ok 2024-02-23T00:46:10.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:09.252388+0000 mgr.y (mgr.14517) 300 : cluster [DBG] pgmap v421: 253 pgs: 8 unknown, 245 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:10.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:09.498026+0000 osd.0 (osd.0) 127 : cluster [DBG] 62.13 scrub starts 2024-02-23T00:46:10.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:09.499803+0000 osd.0 (osd.0) 128 : cluster [DBG] 62.13 scrub ok 2024-02-23T00:46:10.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:09.923194+0000 osd.2 (osd.2) 137 : cluster [DBG] 5.6 scrub starts 2024-02-23T00:46:10.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:09.924380+0000 osd.2 (osd.2) 138 : cluster [DBG] 5.6 scrub ok 2024-02-23T00:46:10.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:10.010061+0000 osd.6 (osd.6) 121 : cluster [DBG] 2.1a scrub starts 2024-02-23T00:46:10.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:10.010991+0000 osd.6 (osd.6) 122 : cluster [DBG] 2.1a scrub ok 2024-02-23T00:46:10.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:10.140637+0000 mon.a (mon.0) 2006 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:46:10.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:10.145361+0000 mon.a (mon.0) 2007 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-35"}]': finished 2024-02-23T00:46:10.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:10.268806+0000 mon.c (mon.2) 229 : audit [DBG] from='client.? 172.21.15.47:0/3408953682' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:46:10.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:10.586908+0000 mon.a (mon.0) 2008 : cluster [DBG] osdmap e298: 8 total, 8 up, 8 in 2024-02-23T00:46:10.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:08.791473+0000 osd.7 (osd.7) 159 : cluster [DBG] 2.1d scrub starts 2024-02-23T00:46:10.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:08.794347+0000 osd.7 (osd.7) 160 : cluster [DBG] 2.1d scrub ok 2024-02-23T00:46:10.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:09.252388+0000 mgr.y (mgr.14517) 300 : cluster [DBG] pgmap v421: 253 pgs: 8 unknown, 245 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:10.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:09.498026+0000 osd.0 (osd.0) 127 : cluster [DBG] 62.13 scrub starts 2024-02-23T00:46:10.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:09.499803+0000 osd.0 (osd.0) 128 : cluster [DBG] 62.13 scrub ok 2024-02-23T00:46:10.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:09.923194+0000 osd.2 (osd.2) 137 : cluster [DBG] 5.6 scrub starts 2024-02-23T00:46:10.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:09.924380+0000 osd.2 (osd.2) 138 : cluster [DBG] 5.6 scrub ok 2024-02-23T00:46:10.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:10.010061+0000 osd.6 (osd.6) 121 : cluster [DBG] 2.1a scrub starts 2024-02-23T00:46:10.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:10.010991+0000 osd.6 (osd.6) 122 : cluster [DBG] 2.1a scrub ok 2024-02-23T00:46:10.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:10.140637+0000 mon.a (mon.0) 2006 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:46:10.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:10.145361+0000 mon.a (mon.0) 2007 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-35"}]': finished 2024-02-23T00:46:10.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:10.268806+0000 mon.c (mon.2) 229 : audit [DBG] from='client.? 172.21.15.47:0/3408953682' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:46:10.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:10.586908+0000 mon.a (mon.0) 2008 : cluster [DBG] osdmap e298: 8 total, 8 up, 8 in 2024-02-23T00:46:11.291 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:46:11.295 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:11.296 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:46:11.397 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:46:11.424 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:11.630 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:46:11.634 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:11.634 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:46:11.735 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:46:11.767 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:11.788 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:46:11.790 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:11.791 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:46:11.892 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:46:11.910 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:09.829336+0000 osd.7 (osd.7) 161 : cluster [DBG] 5.1a scrub starts 2024-02-23T00:46:11.910 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:09.832248+0000 osd.7 (osd.7) 162 : cluster [DBG] 5.1a scrub ok 2024-02-23T00:46:11.910 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:10.438140+0000 osd.4 (osd.4) 129 : cluster [DBG] 62.9 scrub starts 2024-02-23T00:46:11.910 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:10.439368+0000 osd.4 (osd.4) 130 : cluster [DBG] 62.9 scrub ok 2024-02-23T00:46:11.910 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:11.146489+0000 mon.a (mon.0) 2009 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi047-82378-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi047-82378-41"}]': finished 2024-02-23T00:46:11.910 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:11.146638+0000 mon.a (mon.0) 2010 : cluster [DBG] osdmap e299: 8 total, 8 up, 8 in 2024-02-23T00:46:11.910 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:11.161292+0000 mon.c (mon.2) 230 : audit [INF] from='client.? 172.21.15.47:0/2123205582' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi047-82386-51"}]: dispatch 2024-02-23T00:46:11.910 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:11.161678+0000 mon.a (mon.0) 2011 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi047-82386-51"}]: dispatch 2024-02-23T00:46:11.910 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:09.829336+0000 osd.7 (osd.7) 161 : cluster [DBG] 5.1a scrub starts 2024-02-23T00:46:11.911 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:09.832248+0000 osd.7 (osd.7) 162 : cluster [DBG] 5.1a scrub ok 2024-02-23T00:46:11.911 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:10.438140+0000 osd.4 (osd.4) 129 : cluster [DBG] 62.9 scrub starts 2024-02-23T00:46:11.911 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:10.439368+0000 osd.4 (osd.4) 130 : cluster [DBG] 62.9 scrub ok 2024-02-23T00:46:11.911 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:11.146489+0000 mon.a (mon.0) 2009 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi047-82378-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi047-82378-41"}]': finished 2024-02-23T00:46:11.911 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:11.146638+0000 mon.a (mon.0) 2010 : cluster [DBG] osdmap e299: 8 total, 8 up, 8 in 2024-02-23T00:46:11.911 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:11.161292+0000 mon.c (mon.2) 230 : audit [INF] from='client.? 172.21.15.47:0/2123205582' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi047-82386-51"}]: dispatch 2024-02-23T00:46:11.911 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:11.161678+0000 mon.a (mon.0) 2011 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi047-82386-51"}]: dispatch 2024-02-23T00:46:11.919 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:09.829336+0000 osd.7 (osd.7) 161 : cluster [DBG] 5.1a scrub starts 2024-02-23T00:46:11.919 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:09.832248+0000 osd.7 (osd.7) 162 : cluster [DBG] 5.1a scrub ok 2024-02-23T00:46:11.919 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:10.438140+0000 osd.4 (osd.4) 129 : cluster [DBG] 62.9 scrub starts 2024-02-23T00:46:11.919 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:10.439368+0000 osd.4 (osd.4) 130 : cluster [DBG] 62.9 scrub ok 2024-02-23T00:46:11.919 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:11.146489+0000 mon.a (mon.0) 2009 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi047-82378-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi047-82378-41"}]': finished 2024-02-23T00:46:11.920 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:11.146638+0000 mon.a (mon.0) 2010 : cluster [DBG] osdmap e299: 8 total, 8 up, 8 in 2024-02-23T00:46:11.920 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:11.161292+0000 mon.c (mon.2) 230 : audit [INF] from='client.? 172.21.15.47:0/2123205582' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi047-82386-51"}]: dispatch 2024-02-23T00:46:11.920 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:11.161678+0000 mon.a (mon.0) 2011 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi047-82386-51"}]: dispatch 2024-02-23T00:46:11.921 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:11.941 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:46:11.944 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:11.944 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:46:12.045 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:46:12.074 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:12.273 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:46:12.275 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:12.276 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:46:12.300 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:46:12.377 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:46:12.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.863+0000 7fef584d6700 1 -- 172.21.15.47:0/1628261446 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fef500fdbb0 msgr2=0x7fef500fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:12.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.863+0000 7fef584d6700 1 --2- 172.21.15.47:0/1628261446 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fef500fdbb0 0x7fef500fdf90 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 rx=0x7fef4c004870 tx=0x7fef4c009cb0).stop 2024-02-23T00:46:12.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.863+0000 7fef584d6700 1 -- 172.21.15.47:0/1628261446 shutdown_connections 2024-02-23T00:46:12.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.863+0000 7fef584d6700 1 --2- 172.21.15.47:0/1628261446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef500f8420 0x7fef50104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:12.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.863+0000 7fef584d6700 1 --2- 172.21.15.47:0/1628261446 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fef500fdbb0 0x7fef500fdf90 unknown :-1 s=CLOSED pgs=2605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:12.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.863+0000 7fef584d6700 1 --2- 172.21.15.47:0/1628261446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef500fe510 0x7fef500f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:12.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.863+0000 7fef584d6700 1 -- 172.21.15.47:0/1628261446 >> 172.21.15.47:0/1628261446 conn(0x7fef500f3800 msgr2=0x7fef500f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:46:12.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.863+0000 7fef584d6700 1 -- 172.21.15.47:0/1628261446 shutdown_connections 2024-02-23T00:46:12.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.864+0000 7fef584d6700 1 -- 172.21.15.47:0/1628261446 wait complete. 2024-02-23T00:46:12.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.864+0000 7fef584d6700 1 librados: starting msgr at 2024-02-23T00:46:12.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.864+0000 7fef584d6700 1 librados: starting objecter 2024-02-23T00:46:12.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.864+0000 7fef584d6700 1 Processor -- start 2024-02-23T00:46:12.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.864+0000 7fef584d6700 1 -- start start 2024-02-23T00:46:12.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.864+0000 7fef584d6700 1 librados: setting wanted keys 2024-02-23T00:46:12.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.864+0000 7fef584d6700 1 librados: calling monclient init 2024-02-23T00:46:12.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.864+0000 7fef584d6700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef500f8420 0x7fef5010df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:12.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.864+0000 7fef584d6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef500fdbb0 0x7fef5010e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:12.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.864+0000 7fef584d6700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fef500fe510 0x7fef50108110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:12.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.864+0000 7fef584d6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef50108600 con 0x7fef500fdbb0 2024-02-23T00:46:12.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.864+0000 7fef584d6700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef50108770 con 0x7fef500f8420 2024-02-23T00:46:12.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.864+0000 7fef584d6700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef50108930 con 0x7fef500fe510 2024-02-23T00:46:12.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.864+0000 7fef55a71700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef500fdbb0 0x7fef5010e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:12.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.864+0000 7fef56a73700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fef500fe510 0x7fef50108110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:12.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.865+0000 7fef55a71700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef500fdbb0 0x7fef5010e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58690/0 (socket says 172.21.15.47:58690) 2024-02-23T00:46:12.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.865+0000 7fef56a73700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fef500fe510 0x7fef50108110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:36876/0 (socket says 172.21.15.47:36876) 2024-02-23T00:46:12.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.865+0000 7fef55a71700 1 -- 172.21.15.47:0/452951888 learned_addr learned my addr 172.21.15.47:0/452951888 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:46:12.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.865+0000 7fef56272700 1 --2- 172.21.15.47:0/452951888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef500f8420 0x7fef5010df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:12.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.865+0000 7fef55a71700 1 -- 172.21.15.47:0/452951888 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fef500fe510 msgr2=0x7fef50108110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:12.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.865+0000 7fef55a71700 1 --2- 172.21.15.47:0/452951888 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fef500fe510 0x7fef50108110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:12.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.865+0000 7fef55a71700 1 -- 172.21.15.47:0/452951888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef500f8420 msgr2=0x7fef5010df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:12.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.865+0000 7fef55a71700 1 --2- 172.21.15.47:0/452951888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef500f8420 0x7fef5010df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:12.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.865+0000 7fef55a71700 1 -- 172.21.15.47:0/452951888 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef4c005040 con 0x7fef500fdbb0 2024-02-23T00:46:12.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.865+0000 7fef56272700 1 --2- 172.21.15.47:0/452951888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef500f8420 0x7fef5010df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:46:12.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.865+0000 7fef55a71700 1 --2- 172.21.15.47:0/452951888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef500fdbb0 0x7fef5010e460 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 rx=0x7fef4000e970 tx=0x7fef4000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:12.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.865+0000 7fef477fe700 10 client.?.objecter ms_handle_connect 0x7fef500fdbb0 2024-02-23T00:46:12.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.865+0000 7fef477fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:46:12.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.865+0000 7fef477fe700 1 -- 172.21.15.47:0/452951888 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4000ee60 con 0x7fef500fdbb0 2024-02-23T00:46:12.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.865+0000 7fef584d6700 1 -- 172.21.15.47:0/452951888 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef50108c10 con 0x7fef500fdbb0 2024-02-23T00:46:12.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.865+0000 7fef584d6700 10 client.30717.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:46:12.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.865+0000 7fef584d6700 1 -- 172.21.15.47:0/452951888 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef50108f90 con 0x7fef500fdbb0 2024-02-23T00:46:12.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.865+0000 7fef584d6700 1 librados: init done 2024-02-23T00:46:12.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.865+0000 7fef477fe700 1 -- 172.21.15.47:0/452951888 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fef4000f040 con 0x7fef500fdbb0 2024-02-23T00:46:12.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.865+0000 7fef477fe700 1 -- 172.21.15.47:0/452951888 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4000c5b0 con 0x7fef500fdbb0 2024-02-23T00:46:12.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.866+0000 7fef584d6700 1 -- 172.21.15.47:0/452951888 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef34004f50 con 0x7fef500fdbb0 2024-02-23T00:46:12.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.867+0000 7fef477fe700 1 -- 172.21.15.47:0/452951888 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fef4000c9e0 con 0x7fef500fdbb0 2024-02-23T00:46:12.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.867+0000 7fef477fe700 1 --2- 172.21.15.47:0/452951888 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fef3c062cc0 0x7fef3c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:12.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.867+0000 7fef477fe700 1 -- 172.21.15.47:0/452951888 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(299..299 src has 48..299) v4 ==== 10385+0+0 (secure 0 0 0) 0x7fef4007d750 con 0x7fef500fdbb0 2024-02-23T00:46:12.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.867+0000 7fef477fe700 10 client.30717.objecter ms_dispatch 0x7fef50001550 osd_map(299..299 src has 48..299) v4 2024-02-23T00:46:12.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.867+0000 7fef477fe700 3 client.30717.objecter handle_osd_map got epochs [299,299] > 0 2024-02-23T00:46:12.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.867+0000 7fef477fe700 3 client.30717.objecter handle_osd_map decoding full epoch 299 2024-02-23T00:46:12.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.867+0000 7fef56272700 1 --2- 172.21.15.47:0/452951888 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fef3c062cc0 0x7fef3c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:12.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.867+0000 7fef477fe700 20 client.30717.objecter dump_active .. 0 homeless 2024-02-23T00:46:12.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.868+0000 7fef477fe700 10 client.30717.objecter ms_handle_connect 0x7fef3c062cc0 2024-02-23T00:46:12.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.868+0000 7fef56272700 1 --2- 172.21.15.47:0/452951888 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fef3c062cc0 0x7fef3c065180 secure :-1 s=READY pgs=3014 cs=0 l=1 rev1=1 rx=0x7fef4c004ee0 tx=0x7fef4c009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:12.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:11.872+0000 7fef477fe700 1 -- 172.21.15.47:0/452951888 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fef4004c980 con 0x7fef500fdbb0 2024-02-23T00:46:12.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:12.055+0000 7fef584d6700 1 -- 172.21.15.47:0/452951888 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7fef34005780 con 0x7fef500fdbb0 2024-02-23T00:46:12.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:12.150+0000 7fef477fe700 1 -- 172.21.15.47:0/452951888 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v300) v1 ==== 87+0+0 (secure 0 0 0) 0x7fef40019070 con 0x7fef500fdbb0 2024-02-23T00:46:12.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:12.156+0000 7fef584d6700 10 librados: watch_flush enter 2024-02-23T00:46:12.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:12.157+0000 7fef584d6700 10 librados: watch_flush exit 2024-02-23T00:46:12.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:12.157+0000 7fef584d6700 20 client.30717.objecter shutdown clearing up homeless session... 2024-02-23T00:46:12.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:12.157+0000 7fef584d6700 10 client.30717.objecter successfully canceled tick 2024-02-23T00:46:12.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:12.157+0000 7fef584d6700 1 -- 172.21.15.47:0/452951888 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fef3c062cc0 msgr2=0x7fef3c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:12.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:12.157+0000 7fef584d6700 1 --2- 172.21.15.47:0/452951888 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fef3c062cc0 0x7fef3c065180 secure :-1 s=READY pgs=3014 cs=0 l=1 rev1=1 rx=0x7fef4c004ee0 tx=0x7fef4c009cb0).stop 2024-02-23T00:46:12.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:12.157+0000 7fef584d6700 1 -- 172.21.15.47:0/452951888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef500fdbb0 msgr2=0x7fef5010e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:12.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:12.157+0000 7fef584d6700 1 --2- 172.21.15.47:0/452951888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef500fdbb0 0x7fef5010e460 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 rx=0x7fef4000e970 tx=0x7fef4000b040).stop 2024-02-23T00:46:12.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:12.158+0000 7fef584d6700 1 -- 172.21.15.47:0/452951888 shutdown_connections 2024-02-23T00:46:12.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:12.158+0000 7fef584d6700 1 --2- 172.21.15.47:0/452951888 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fef3c062cc0 0x7fef3c065180 secure :-1 s=CLOSED pgs=3014 cs=0 l=1 rev1=1 rx=0x7fef4c004ee0 tx=0x7fef4c009cb0).stop 2024-02-23T00:46:12.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:12.158+0000 7fef584d6700 1 --2- 172.21.15.47:0/452951888 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fef500fe510 0x7fef50108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:12.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:12.158+0000 7fef584d6700 1 --2- 172.21.15.47:0/452951888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef500f8420 0x7fef5010df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:12.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:12.158+0000 7fef584d6700 1 --2- 172.21.15.47:0/452951888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef500fdbb0 0x7fef5010e460 unknown :-1 s=CLOSED pgs=2497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:12.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:12.158+0000 7fef584d6700 1 -- 172.21.15.47:0/452951888 >> 172.21.15.47:0/452951888 conn(0x7fef500f3800 msgr2=0x7fef500fb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:46:12.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:12.162+0000 7fef584d6700 1 -- 172.21.15.47:0/452951888 shutdown_connections 2024-02-23T00:46:12.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:12.162+0000 7fef584d6700 1 -- 172.21.15.47:0/452951888 wait complete. 2024-02-23T00:46:12.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:12.162+0000 7fef584d6700 1 librados: shutdown 2024-02-23T00:46:12.384 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is set 2024-02-23T00:46:12.416 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:12.437 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:46:12.439 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:12.440 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:46:12.540 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:46:12.569 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:12.589 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:46:12.591 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:12.592 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:46:12.693 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:46:12.694 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:46:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:46:12.694 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:46:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:46:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:46:12.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:10.877415+0000 osd.2 (osd.2) 139 : cluster [DBG] 2.1 scrub starts 2024-02-23T00:46:12.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:10.878201+0000 osd.2 (osd.2) 140 : cluster [DBG] 2.1 scrub ok 2024-02-23T00:46:12.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:11.253016+0000 mgr.y (mgr.14517) 301 : cluster [DBG] pgmap v424: 220 pgs: 8 unknown, 212 active+clean; 8.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:46:12.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:11.490469+0000 osd.0 (osd.0) 129 : cluster [DBG] 62.e scrub starts 2024-02-23T00:46:12.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:11.491633+0000 osd.0 (osd.0) 130 : cluster [DBG] 62.e scrub ok 2024-02-23T00:46:12.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:11.604245+0000 mon.a (mon.0) 2012 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "25"}]: dispatch 2024-02-23T00:46:12.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:11.612516+0000 mon.a (mon.0) 2013 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:46:12.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:11.827747+0000 osd.3 (osd.3) 155 : cluster [DBG] 62.15 deep-scrub starts 2024-02-23T00:46:12.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:11.830575+0000 osd.3 (osd.3) 156 : cluster [DBG] 62.15 deep-scrub ok 2024-02-23T00:46:12.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:11.978204+0000 osd.6 (osd.6) 123 : cluster [DBG] 4.5 scrub starts 2024-02-23T00:46:12.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:11.980082+0000 osd.6 (osd.6) 124 : cluster [DBG] 4.5 scrub ok 2024-02-23T00:46:12.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:12.057533+0000 mon.a (mon.0) 2014 : audit [INF] from='client.? 172.21.15.47:0/452951888' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:46:12.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:12.144475+0000 mon.a (mon.0) 2015 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:46:12.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:12.151074+0000 mon.a (mon.0) 2016 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi047-82386-51"}]': finished 2024-02-23T00:46:12.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:12.151234+0000 mon.a (mon.0) 2017 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "25"}]': finished 2024-02-23T00:46:12.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:12.151293+0000 mon.a (mon.0) 2018 : audit [INF] from='client.? 172.21.15.47:0/452951888' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:46:12.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:12.151347+0000 mon.a (mon.0) 2019 : cluster [DBG] osdmap e300: 8 total, 8 up, 8 in 2024-02-23T00:46:12.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:12.153926+0000 mon.c (mon.2) 231 : audit [INF] from='client.? 172.21.15.47:0/2123205582' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi047-82386-51"}]: dispatch 2024-02-23T00:46:12.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:12.156897+0000 mon.a (mon.0) 2020 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi047-82386-51"}]: dispatch 2024-02-23T00:46:12.697 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:12.158896+0000 osd.5 (osd.5) 133 : cluster [DBG] 2.16 scrub starts 2024-02-23T00:46:12.697 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:12.162177+0000 mon.a (mon.0) 2021 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:46:12.728 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:13.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:10.877415+0000 osd.2 (osd.2) 139 : cluster [DBG] 2.1 scrub starts 2024-02-23T00:46:13.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:10.878201+0000 osd.2 (osd.2) 140 : cluster [DBG] 2.1 scrub ok 2024-02-23T00:46:13.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:11.253016+0000 mgr.y (mgr.14517) 301 : cluster [DBG] pgmap v424: 220 pgs: 8 unknown, 212 active+clean; 8.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:46:13.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:11.490469+0000 osd.0 (osd.0) 129 : cluster [DBG] 62.e scrub starts 2024-02-23T00:46:13.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:11.491633+0000 osd.0 (osd.0) 130 : cluster [DBG] 62.e scrub ok 2024-02-23T00:46:13.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:11.604245+0000 mon.a (mon.0) 2012 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "25"}]: dispatch 2024-02-23T00:46:13.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:11.612516+0000 mon.a (mon.0) 2013 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:46:13.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:11.827747+0000 osd.3 (osd.3) 155 : cluster [DBG] 62.15 deep-scrub starts 2024-02-23T00:46:13.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:11.830575+0000 osd.3 (osd.3) 156 : cluster [DBG] 62.15 deep-scrub ok 2024-02-23T00:46:13.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:11.978204+0000 osd.6 (osd.6) 123 : cluster [DBG] 4.5 scrub starts 2024-02-23T00:46:13.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:11.980082+0000 osd.6 (osd.6) 124 : cluster [DBG] 4.5 scrub ok 2024-02-23T00:46:13.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:12.057533+0000 mon.a (mon.0) 2014 : audit [INF] from='client.? 172.21.15.47:0/452951888' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:46:13.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:12.144475+0000 mon.a (mon.0) 2015 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:46:13.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:12.151074+0000 mon.a (mon.0) 2016 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi047-82386-51"}]': finished 2024-02-23T00:46:13.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:12.151234+0000 mon.a (mon.0) 2017 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "25"}]': finished 2024-02-23T00:46:13.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:12.151293+0000 mon.a (mon.0) 2018 : audit [INF] from='client.? 172.21.15.47:0/452951888' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:46:13.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:12.151347+0000 mon.a (mon.0) 2019 : cluster [DBG] osdmap e300: 8 total, 8 up, 8 in 2024-02-23T00:46:13.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:10.877415+0000 osd.2 (osd.2) 139 : cluster [DBG] 2.1 scrub starts 2024-02-23T00:46:13.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:10.878201+0000 osd.2 (osd.2) 140 : cluster [DBG] 2.1 scrub ok 2024-02-23T00:46:13.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:11.253016+0000 mgr.y (mgr.14517) 301 : cluster [DBG] pgmap v424: 220 pgs: 8 unknown, 212 active+clean; 8.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:46:13.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:11.490469+0000 osd.0 (osd.0) 129 : cluster [DBG] 62.e scrub starts 2024-02-23T00:46:13.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:11.491633+0000 osd.0 (osd.0) 130 : cluster [DBG] 62.e scrub ok 2024-02-23T00:46:13.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:11.604245+0000 mon.a (mon.0) 2012 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "25"}]: dispatch 2024-02-23T00:46:13.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:11.612516+0000 mon.a (mon.0) 2013 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:46:13.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:11.827747+0000 osd.3 (osd.3) 155 : cluster [DBG] 62.15 deep-scrub starts 2024-02-23T00:46:13.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:11.830575+0000 osd.3 (osd.3) 156 : cluster [DBG] 62.15 deep-scrub ok 2024-02-23T00:46:13.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:11.978204+0000 osd.6 (osd.6) 123 : cluster [DBG] 4.5 scrub starts 2024-02-23T00:46:13.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:11.980082+0000 osd.6 (osd.6) 124 : cluster [DBG] 4.5 scrub ok 2024-02-23T00:46:13.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:12.057533+0000 mon.a (mon.0) 2014 : audit [INF] from='client.? 172.21.15.47:0/452951888' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:46:13.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:12.144475+0000 mon.a (mon.0) 2015 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:46:13.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:12.151074+0000 mon.a (mon.0) 2016 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi047-82386-51"}]': finished 2024-02-23T00:46:13.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:12.151234+0000 mon.a (mon.0) 2017 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "25"}]': finished 2024-02-23T00:46:13.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:12.151293+0000 mon.a (mon.0) 2018 : audit [INF] from='client.? 172.21.15.47:0/452951888' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:46:13.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:12.151347+0000 mon.a (mon.0) 2019 : cluster [DBG] osdmap e300: 8 total, 8 up, 8 in 2024-02-23T00:46:13.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:12.153926+0000 mon.c (mon.2) 231 : audit [INF] from='client.? 172.21.15.47:0/2123205582' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi047-82386-51"}]: dispatch 2024-02-23T00:46:13.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:12.156897+0000 mon.a (mon.0) 2020 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi047-82386-51"}]: dispatch 2024-02-23T00:46:13.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:12.158896+0000 osd.5 (osd.5) 133 : cluster [DBG] 2.16 scrub starts 2024-02-23T00:46:13.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:12.162177+0000 mon.a (mon.0) 2021 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:46:13.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:12.153926+0000 mon.c (mon.2) 231 : audit [INF] from='client.? 172.21.15.47:0/2123205582' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi047-82386-51"}]: dispatch 2024-02-23T00:46:13.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:12.156897+0000 mon.a (mon.0) 2020 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi047-82386-51"}]: dispatch 2024-02-23T00:46:13.061 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:12.158896+0000 osd.5 (osd.5) 133 : cluster [DBG] 2.16 scrub starts 2024-02-23T00:46:13.061 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:12.162177+0000 mon.a (mon.0) 2021 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:46:13.114 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:46:13.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:12.488452+0000 mgr.y (mgr.14517) 302 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:46:13.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:13.153348+0000 mon.a (mon.0) 2022 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi047-82386-51"}]': finished 2024-02-23T00:46:13.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:13.153491+0000 mon.a (mon.0) 2023 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:46:13.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:13.153594+0000 mon.a (mon.0) 2024 : cluster [DBG] osdmap e301: 8 total, 8 up, 8 in 2024-02-23T00:46:13.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:13.162295+0000 mon.a (mon.0) 2025 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:46:13.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:13.170535+0000 mon.c (mon.2) 232 : audit [INF] from='client.? 172.21.15.47:0/297870923' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi047-82378-41"}]: dispatch 2024-02-23T00:46:13.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:13.170964+0000 mon.a (mon.0) 2026 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi047-82378-41"}]: dispatch 2024-02-23T00:46:13.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:13.191888+0000 mon.c (mon.2) 233 : audit [INF] from='client.? 172.21.15.47:0/234645247' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi047-82386-52"}]: dispatch 2024-02-23T00:46:13.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:13.192219+0000 mon.a (mon.0) 2027 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi047-82386-52"}]: dispatch 2024-02-23T00:46:13.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:13.192727+0000 mon.c (mon.2) 234 : audit [INF] from='client.? 172.21.15.47:0/234645247' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi047-82386-52"}]: dispatch 2024-02-23T00:46:13.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:13.193014+0000 mon.a (mon.0) 2028 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi047-82386-52"}]: dispatch 2024-02-23T00:46:13.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:13.193435+0000 mon.c (mon.2) 235 : audit [INF] from='client.? 172.21.15.47:0/234645247' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi047-82386-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:13.611 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:13.193747+0000 mon.a (mon.0) 2029 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi047-82386-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:13.612 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:12.488452+0000 mgr.y (mgr.14517) 302 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:46:13.612 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:13.153348+0000 mon.a (mon.0) 2022 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi047-82386-51"}]': finished 2024-02-23T00:46:13.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:12.488452+0000 mgr.y (mgr.14517) 302 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:46:13.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:13.153348+0000 mon.a (mon.0) 2022 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi047-82386-51"}]': finished 2024-02-23T00:46:13.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:13.153491+0000 mon.a (mon.0) 2023 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:46:13.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:13.153594+0000 mon.a (mon.0) 2024 : cluster [DBG] osdmap e301: 8 total, 8 up, 8 in 2024-02-23T00:46:13.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:13.162295+0000 mon.a (mon.0) 2025 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:46:13.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:13.170535+0000 mon.c (mon.2) 232 : audit [INF] from='client.? 172.21.15.47:0/297870923' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi047-82378-41"}]: dispatch 2024-02-23T00:46:13.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:13.170964+0000 mon.a (mon.0) 2026 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi047-82378-41"}]: dispatch 2024-02-23T00:46:13.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:13.191888+0000 mon.c (mon.2) 233 : audit [INF] from='client.? 172.21.15.47:0/234645247' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi047-82386-52"}]: dispatch 2024-02-23T00:46:13.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:13.192219+0000 mon.a (mon.0) 2027 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi047-82386-52"}]: dispatch 2024-02-23T00:46:13.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:13.192727+0000 mon.c (mon.2) 234 : audit [INF] from='client.? 172.21.15.47:0/234645247' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi047-82386-52"}]: dispatch 2024-02-23T00:46:13.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:13.193014+0000 mon.a (mon.0) 2028 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi047-82386-52"}]: dispatch 2024-02-23T00:46:13.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:13.193435+0000 mon.c (mon.2) 235 : audit [INF] from='client.? 172.21.15.47:0/234645247' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi047-82386-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:13.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:13.193747+0000 mon.a (mon.0) 2029 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi047-82386-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:14.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:13.153491+0000 mon.a (mon.0) 2023 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:46:14.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:13.153594+0000 mon.a (mon.0) 2024 : cluster [DBG] osdmap e301: 8 total, 8 up, 8 in 2024-02-23T00:46:14.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:13.162295+0000 mon.a (mon.0) 2025 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:46:14.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:13.170535+0000 mon.c (mon.2) 232 : audit [INF] from='client.? 172.21.15.47:0/297870923' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi047-82378-41"}]: dispatch 2024-02-23T00:46:14.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:13.170964+0000 mon.a (mon.0) 2026 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi047-82378-41"}]: dispatch 2024-02-23T00:46:14.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:13.191888+0000 mon.c (mon.2) 233 : audit [INF] from='client.? 172.21.15.47:0/234645247' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi047-82386-52"}]: dispatch 2024-02-23T00:46:14.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:13.192219+0000 mon.a (mon.0) 2027 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi047-82386-52"}]: dispatch 2024-02-23T00:46:14.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:13.192727+0000 mon.c (mon.2) 234 : audit [INF] from='client.? 172.21.15.47:0/234645247' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi047-82386-52"}]: dispatch 2024-02-23T00:46:14.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:13.193014+0000 mon.a (mon.0) 2028 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi047-82386-52"}]: dispatch 2024-02-23T00:46:14.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:13.193435+0000 mon.c (mon.2) 235 : audit [INF] from='client.? 172.21.15.47:0/234645247' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi047-82386-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:14.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:13.193747+0000 mon.a (mon.0) 2029 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi047-82386-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:14.285 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:46:14.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:13.253561+0000 mgr.y (mgr.14517) 303 : cluster [DBG] pgmap v427: 244 pgs: 2 peering, 32 creating+peering, 210 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-23T00:46:14.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:14.159687+0000 mon.a (mon.0) 2030 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:46:14.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:14.159748+0000 mon.a (mon.0) 2031 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi047-82378-41"}]': finished 2024-02-23T00:46:14.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:14.159802+0000 mon.a (mon.0) 2032 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi047-82386-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:14.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:14.159855+0000 mon.a (mon.0) 2033 : cluster [DBG] osdmap e302: 8 total, 8 up, 8 in 2024-02-23T00:46:14.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:14.160711+0000 mon.c (mon.2) 236 : audit [INF] from='client.? 172.21.15.47:0/297870923' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi047-82378-41"}]: dispatch 2024-02-23T00:46:14.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:14.162038+0000 mon.a (mon.0) 2034 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-37"}]: dispatch 2024-02-23T00:46:14.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:14.162340+0000 mon.a (mon.0) 2035 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi047-82378-41"}]: dispatch 2024-02-23T00:46:14.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:14.163422+0000 mon.c (mon.2) 237 : audit [INF] from='client.? 172.21.15.47:0/234645247' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi047-82386-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi047-82386-52"}]: dispatch 2024-02-23T00:46:14.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:14.163735+0000 mon.a (mon.0) 2036 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi047-82386-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi047-82386-52"}]: dispatch 2024-02-23T00:46:15.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:13.253561+0000 mgr.y (mgr.14517) 303 : cluster [DBG] pgmap v427: 244 pgs: 2 peering, 32 creating+peering, 210 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-23T00:46:15.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:14.159687+0000 mon.a (mon.0) 2030 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:46:15.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:14.159748+0000 mon.a (mon.0) 2031 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi047-82378-41"}]': finished 2024-02-23T00:46:15.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:14.159802+0000 mon.a (mon.0) 2032 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi047-82386-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:15.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:14.159855+0000 mon.a (mon.0) 2033 : cluster [DBG] osdmap e302: 8 total, 8 up, 8 in 2024-02-23T00:46:15.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:14.160711+0000 mon.c (mon.2) 236 : audit [INF] from='client.? 172.21.15.47:0/297870923' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi047-82378-41"}]: dispatch 2024-02-23T00:46:15.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:14.162038+0000 mon.a (mon.0) 2034 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-37"}]: dispatch 2024-02-23T00:46:15.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:14.162340+0000 mon.a (mon.0) 2035 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi047-82378-41"}]: dispatch 2024-02-23T00:46:15.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:14.163422+0000 mon.c (mon.2) 237 : audit [INF] from='client.? 172.21.15.47:0/234645247' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi047-82386-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi047-82386-52"}]: dispatch 2024-02-23T00:46:15.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:14.163735+0000 mon.a (mon.0) 2036 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi047-82386-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi047-82386-52"}]: dispatch 2024-02-23T00:46:15.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:13.253561+0000 mgr.y (mgr.14517) 303 : cluster [DBG] pgmap v427: 244 pgs: 2 peering, 32 creating+peering, 210 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-23T00:46:15.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:14.159687+0000 mon.a (mon.0) 2030 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:46:15.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:14.159748+0000 mon.a (mon.0) 2031 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi047-82378-41"}]': finished 2024-02-23T00:46:15.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:14.159802+0000 mon.a (mon.0) 2032 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi047-82386-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:15.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:14.159855+0000 mon.a (mon.0) 2033 : cluster [DBG] osdmap e302: 8 total, 8 up, 8 in 2024-02-23T00:46:15.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:14.160711+0000 mon.c (mon.2) 236 : audit [INF] from='client.? 172.21.15.47:0/297870923' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi047-82378-41"}]: dispatch 2024-02-23T00:46:15.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:14.162038+0000 mon.a (mon.0) 2034 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-37"}]: dispatch 2024-02-23T00:46:15.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:14.162340+0000 mon.a (mon.0) 2035 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi047-82378-41"}]: dispatch 2024-02-23T00:46:15.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:14.163422+0000 mon.c (mon.2) 237 : audit [INF] from='client.? 172.21.15.47:0/234645247' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi047-82386-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi047-82386-52"}]: dispatch 2024-02-23T00:46:15.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:14.163735+0000 mon.a (mon.0) 2036 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi047-82386-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi047-82386-52"}]: dispatch 2024-02-23T00:46:15.122 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-23T00:46:15.159 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAiete 2024-02-23T00:46:15.159 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (6982 ms) 2024-02-23T00:46:15.159 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2024-02-23T00:46:15.159 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (7033 ms) 2024-02-23T00:46:15.159 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2024-02-23T00:46:15.159 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (7007 ms) 2024-02-23T00:46:15.159 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2024-02-23T00:46:15.159 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (6975 ms) 2024-02-23T00:46:15.159 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2024-02-23T00:46:15.159 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (6999 ms) 2024-02-23T00:46:15.159 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2024-02-23T00:46:15.159 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (7022 ms) 2024-02-23T00:46:15.159 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2024-02-23T00:46:15.159 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (6999 ms) 2024-02-23T00:46:15.160 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2024-02-23T00:46:15.160 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (6990 ms) 2024-02-23T00:46:15.160 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2024-02-23T00:46:15.160 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (7001 ms) 2024-02-23T00:46:15.160 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-02-23T00:46:15.160 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (6997 ms) 2024-02-23T00:46:15.160 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2024-02-23T00:46:15.160 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (7004 ms) 2024-02-23T00:46:15.160 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2024-02-23T00:46:15.160 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (7023 ms) 2024-02-23T00:46:15.160 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2024-02-23T00:46:15.160 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (6998 ms) 2024-02-23T00:46:15.160 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2024-02-23T00:46:15.160 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (6991 ms) 2024-02-23T00:46:15.161 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2024-02-23T00:46:15.161 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (6998 ms) 2024-02-23T00:46:15.161 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (112020 ms total) 2024-02-23T00:46:15.161 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: 2024-02-23T00:46:15.161 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [----------] Global test environment tear-down 2024-02-23T00:46:15.161 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [==========] 41 tests from 2 test suites ran. (207929 ms total) 2024-02-23T00:46:15.161 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio: [ PASSED ] 41 tests. 2024-02-23T00:46:15.737 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:46:16.081 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:46:16.083 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:16.084 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:46:16.185 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:46:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:16.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:15.146777+0000 mon.a (mon.0) 2037 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-37"}]': finished 2024-02-23T00:46:16.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:15.146852+0000 mon.a (mon.0) 2038 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi047-82378-41"}]': finished 2024-02-23T00:46:16.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:15.254240+0000 mgr.y (mgr.14517) 304 : cluster [DBG] pgmap v430: 243 pgs: 1 peering, 32 creating+peering, 210 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s; 31 B/s, 0 objects/s recovering 2024-02-23T00:46:16.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:15.579548+0000 mon.a (mon.0) 2039 : cluster [DBG] osdmap e303: 8 total, 8 up, 8 in 2024-02-23T00:46:16.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:15.580348+0000 mon.a (mon.0) 2040 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-37", "mode": "writeback"}]: dispatch 2024-02-23T00:46:16.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:15.585224+0000 mon.a (mon.0) 2041 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "24"}]: dispatch 2024-02-23T00:46:16.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:15.146777+0000 mon.a (mon.0) 2037 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-37"}]': finished 2024-02-23T00:46:16.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:15.146852+0000 mon.a (mon.0) 2038 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi047-82378-41"}]': finished 2024-02-23T00:46:16.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:15.254240+0000 mgr.y (mgr.14517) 304 : cluster [DBG] pgmap v430: 243 pgs: 1 peering, 32 creating+peering, 210 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s; 31 B/s, 0 objects/s recovering 2024-02-23T00:46:16.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:15.579548+0000 mon.a (mon.0) 2039 : cluster [DBG] osdmap e303: 8 total, 8 up, 8 in 2024-02-23T00:46:16.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:15.580348+0000 mon.a (mon.0) 2040 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-37", "mode": "writeback"}]: dispatch 2024-02-23T00:46:16.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:15.585224+0000 mon.a (mon.0) 2041 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "24"}]: dispatch 2024-02-23T00:46:16.366 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:46:16.369 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:16.370 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:46:16.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:15.146777+0000 mon.a (mon.0) 2037 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-37"}]': finished 2024-02-23T00:46:16.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:15.146852+0000 mon.a (mon.0) 2038 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi047-82378-41"}]': finished 2024-02-23T00:46:16.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:15.254240+0000 mgr.y (mgr.14517) 304 : cluster [DBG] pgmap v430: 243 pgs: 1 peering, 32 creating+peering, 210 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s; 31 B/s, 0 objects/s recovering 2024-02-23T00:46:16.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:15.579548+0000 mon.a (mon.0) 2039 : cluster [DBG] osdmap e303: 8 total, 8 up, 8 in 2024-02-23T00:46:16.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:15.580348+0000 mon.a (mon.0) 2040 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-37", "mode": "writeback"}]: dispatch 2024-02-23T00:46:16.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:15.585224+0000 mon.a (mon.0) 2041 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "24"}]: dispatch 2024-02-23T00:46:16.470 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:46:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:16.650 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:46:16.652 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:16.653 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:46:16.753 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:46:16.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.757+0000 7f9bd1a9b700 1 -- 172.21.15.47:0/4051207425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bcc0fdbb0 msgr2=0x7f9bcc0fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:16.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.757+0000 7f9bd1a9b700 1 --2- 172.21.15.47:0/4051207425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bcc0fdbb0 0x7f9bcc0fdf90 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 rx=0x7f9bbc0047b0 tx=0x7f9bbc009cb0).stop 2024-02-23T00:46:16.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.757+0000 7f9bd1a9b700 1 -- 172.21.15.47:0/4051207425 shutdown_connections 2024-02-23T00:46:16.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.757+0000 7f9bd1a9b700 1 --2- 172.21.15.47:0/4051207425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9bcc0f8420 0x7f9bcc104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:16.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.757+0000 7f9bd1a9b700 1 --2- 172.21.15.47:0/4051207425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bcc0fdbb0 0x7f9bcc0fdf90 unknown :-1 s=CLOSED pgs=2499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:16.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.757+0000 7f9bd1a9b700 1 --2- 172.21.15.47:0/4051207425 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9bcc0fe510 0x7f9bcc0f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:16.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.757+0000 7f9bd1a9b700 1 -- 172.21.15.47:0/4051207425 >> 172.21.15.47:0/4051207425 conn(0x7f9bcc0f3800 msgr2=0x7f9bcc0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:46:16.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.757+0000 7f9bd1a9b700 1 -- 172.21.15.47:0/4051207425 shutdown_connections 2024-02-23T00:46:16.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.758+0000 7f9bd1a9b700 1 -- 172.21.15.47:0/4051207425 wait complete. 2024-02-23T00:46:16.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.758+0000 7f9bd1a9b700 1 librados: starting msgr at 2024-02-23T00:46:16.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.758+0000 7f9bd1a9b700 1 librados: starting objecter 2024-02-23T00:46:16.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.758+0000 7f9bd1a9b700 1 Processor -- start 2024-02-23T00:46:16.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.758+0000 7f9bd1a9b700 1 -- start start 2024-02-23T00:46:16.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.758+0000 7f9bd1a9b700 1 librados: setting wanted keys 2024-02-23T00:46:16.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.758+0000 7f9bd1a9b700 1 librados: calling monclient init 2024-02-23T00:46:16.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.758+0000 7f9bd1a9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9bcc0f8420 0x7f9bcc06b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:16.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.758+0000 7f9bd1a9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bcc0fdbb0 0x7f9bcc06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:16.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.758+0000 7f9bd1a9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9bcc0fe510 0x7f9bcc06bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:16.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.759+0000 7f9bd1a9b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc116400 con 0x7f9bcc0fdbb0 2024-02-23T00:46:16.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.759+0000 7f9bd1a9b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc116570 con 0x7f9bcc0f8420 2024-02-23T00:46:16.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.759+0000 7f9bd1a9b700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc116730 con 0x7f9bcc0fe510 2024-02-23T00:46:16.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.759+0000 7f9bcbfff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9bcc0fe510 0x7f9bcc06bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:16.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.759+0000 7f9bc2ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bcc0fdbb0 0x7f9bcc06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:16.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.759+0000 7f9bcbfff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9bcc0fe510 0x7f9bcc06bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:58986/0 (socket says 172.21.15.47:58986) 2024-02-23T00:46:16.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.759+0000 7f9bc2ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bcc0fdbb0 0x7f9bcc06b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46502/0 (socket says 172.21.15.47:46502) 2024-02-23T00:46:16.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.759+0000 7f9bcbfff700 1 -- 172.21.15.47:0/801586363 learned_addr learned my addr 172.21.15.47:0/801586363 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:46:16.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.759+0000 7f9bcb7fe700 1 --2- 172.21.15.47:0/801586363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9bcc0f8420 0x7f9bcc06b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.759+0000 7f9bcbfff700 1 -- 172.21.15.47:0/801586363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9bcc0f8420 msgr2=0x7f9bcc06b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.759+0000 7f9bcbfff700 1 --2- 172.21.15.47:0/801586363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9bcc0f8420 0x7f9bcc06b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.759+0000 7f9bcbfff700 1 -- 172.21.15.47:0/801586363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bcc0fdbb0 msgr2=0x7f9bcc06b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.759+0000 7f9bcbfff700 1 --2- 172.21.15.47:0/801586363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bcc0fdbb0 0x7f9bcc06b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.759+0000 7f9bcbfff700 1 -- 172.21.15.47:0/801586363 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb4005040 con 0x7f9bcc0fe510 2024-02-23T00:46:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.759+0000 7f9bcb7fe700 1 --2- 172.21.15.47:0/801586363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9bcc0f8420 0x7f9bcc06b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:46:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.759+0000 7f9bc2ffd700 1 --2- 172.21.15.47:0/801586363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bcc0fdbb0 0x7f9bcc06b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:46:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.759+0000 7f9bcbfff700 1 --2- 172.21.15.47:0/801586363 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9bcc0fe510 0x7f9bcc06bab0 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 rx=0x7f9bc400d970 tx=0x7f9bc4004e60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.759+0000 7f9bc9775700 10 client.?.objecter ms_handle_connect 0x7f9bcc0fe510 2024-02-23T00:46:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.759+0000 7f9bc9775700 10 client.?.objecter resend_mon_ops 2024-02-23T00:46:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.759+0000 7f9bc9775700 1 -- 172.21.15.47:0/801586363 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc400de60 con 0x7f9bcc0fe510 2024-02-23T00:46:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.759+0000 7f9bc9775700 1 -- 172.21.15.47:0/801586363 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f9bc400f040 con 0x7f9bcc0fe510 2024-02-23T00:46:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.759+0000 7f9bd1a9b700 1 -- 172.21.15.47:0/801586363 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bbc005040 con 0x7f9bcc0fe510 2024-02-23T00:46:16.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.759+0000 7f9bc9775700 1 -- 172.21.15.47:0/801586363 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc40036c0 con 0x7f9bcc0fe510 2024-02-23T00:46:16.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.760+0000 7f9bd1a9b700 10 client.41063.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:46:16.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.760+0000 7f9bd1a9b700 1 -- 172.21.15.47:0/801586363 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bcc116e40 con 0x7f9bcc0fe510 2024-02-23T00:46:16.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.760+0000 7f9bd1a9b700 1 librados: init done 2024-02-23T00:46:16.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.761+0000 7f9bd1a9b700 1 -- 172.21.15.47:0/801586363 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bcc106e90 con 0x7f9bcc0fe510 2024-02-23T00:46:16.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.761+0000 7f9bc9775700 1 -- 172.21.15.47:0/801586363 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f9bc4004020 con 0x7f9bcc0fe510 2024-02-23T00:46:16.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.761+0000 7f9bc9775700 1 --2- 172.21.15.47:0/801586363 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9bb8062cc0 0x7f9bb8065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:16.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.761+0000 7f9bc9775700 1 -- 172.21.15.47:0/801586363 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(304..304 src has 48..304) v4 ==== 10393+0+0 (secure 0 0 0) 0x7f9bc407d580 con 0x7f9bcc0fe510 2024-02-23T00:46:16.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.761+0000 7f9bc9775700 10 client.41063.objecter ms_dispatch 0x7f9bcc001550 osd_map(304..304 src has 48..304) v4 2024-02-23T00:46:16.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.761+0000 7f9bc9775700 3 client.41063.objecter handle_osd_map got epochs [304,304] > 0 2024-02-23T00:46:16.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.761+0000 7f9bc9775700 3 client.41063.objecter handle_osd_map decoding full epoch 304 2024-02-23T00:46:16.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.762+0000 7f9bcb7fe700 1 --2- 172.21.15.47:0/801586363 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9bb8062cc0 0x7f9bb8065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:16.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.762+0000 7f9bc9775700 20 client.41063.objecter dump_active .. 0 homeless 2024-02-23T00:46:16.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.762+0000 7f9bc9775700 10 client.41063.objecter ms_handle_connect 0x7f9bb8062cc0 2024-02-23T00:46:16.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.762+0000 7f9bcb7fe700 1 --2- 172.21.15.47:0/801586363 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9bb8062cc0 0x7f9bb8065180 secure :-1 s=READY pgs=3016 cs=0 l=1 rev1=1 rx=0x7f9bbc004e20 tx=0x7f9bbc009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:16.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.766+0000 7f9bc9775700 1 -- 172.21.15.47:0/801586363 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f9bc404c780 con 0x7f9bcc0fe510 2024-02-23T00:46:16.783 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:16.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.939+0000 7f9bd1a9b700 1 -- 172.21.15.47:0/801586363 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9bcc05fbd0 con 0x7f9bb8062cc0 2024-02-23T00:46:16.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.956+0000 7f9bc9775700 1 -- 172.21.15.47:0/801586363 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+633838 (secure 0 0 0) 0x7f9bcc05fbd0 con 0x7f9bb8062cc0 2024-02-23T00:46:16.966 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:46:16.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.961+0000 7f9bd1a9b700 10 librados: watch_flush enter 2024-02-23T00:46:16.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.961+0000 7f9bd1a9b700 10 librados: watch_flush exit 2024-02-23T00:46:16.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.961+0000 7f9bd1a9b700 20 client.41063.objecter shutdown clearing up homeless session... 2024-02-23T00:46:16.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.961+0000 7f9bd1a9b700 10 client.41063.objecter successfully canceled tick 2024-02-23T00:46:16.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.961+0000 7f9bd1a9b700 1 -- 172.21.15.47:0/801586363 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9bb8062cc0 msgr2=0x7f9bb8065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:16.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.961+0000 7f9bd1a9b700 1 --2- 172.21.15.47:0/801586363 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9bb8062cc0 0x7f9bb8065180 secure :-1 s=READY pgs=3016 cs=0 l=1 rev1=1 rx=0x7f9bbc004e20 tx=0x7f9bbc009cb0).stop 2024-02-23T00:46:16.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.961+0000 7f9bd1a9b700 1 -- 172.21.15.47:0/801586363 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9bcc0fe510 msgr2=0x7f9bcc06bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:16.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.961+0000 7f9bd1a9b700 1 --2- 172.21.15.47:0/801586363 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9bcc0fe510 0x7f9bcc06bab0 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 rx=0x7f9bc400d970 tx=0x7f9bc4004e60).stop 2024-02-23T00:46:16.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.961+0000 7f9bd1a9b700 1 -- 172.21.15.47:0/801586363 shutdown_connections 2024-02-23T00:46:16.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.961+0000 7f9bd1a9b700 1 --2- 172.21.15.47:0/801586363 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9bb8062cc0 0x7f9bb8065180 unknown :-1 s=CLOSED pgs=3016 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:16.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.961+0000 7f9bd1a9b700 1 --2- 172.21.15.47:0/801586363 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9bcc0fe510 0x7f9bcc06bab0 unknown :-1 s=CLOSED pgs=2607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:16.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.961+0000 7f9bd1a9b700 1 --2- 172.21.15.47:0/801586363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9bcc0f8420 0x7f9bcc06b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:16.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.961+0000 7f9bd1a9b700 1 --2- 172.21.15.47:0/801586363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bcc0fdbb0 0x7f9bcc06b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:16.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.961+0000 7f9bd1a9b700 1 -- 172.21.15.47:0/801586363 >> 172.21.15.47:0/801586363 conn(0x7f9bcc0f3800 msgr2=0x7f9bcc0fb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:46:16.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.961+0000 7f9bd1a9b700 1 -- 172.21.15.47:0/801586363 shutdown_connections 2024-02-23T00:46:16.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.961+0000 7f9bd1a9b700 1 -- 172.21.15.47:0/801586363 wait complete. 2024-02-23T00:46:16.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:16.961+0000 7f9bd1a9b700 1 librados: shutdown 2024-02-23T00:46:16.976 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-02-23T00:46:17.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:16.144748+0000 mon.a (mon.0) 2042 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:46:17.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:16.153480+0000 mon.a (mon.0) 2043 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi047-82386-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi047-82386-52"}]': finished 2024-02-23T00:46:17.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:16.153558+0000 mon.a (mon.0) 2044 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-37", "mode": "writeback"}]': finished 2024-02-23T00:46:17.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:16.153620+0000 mon.a (mon.0) 2045 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "24"}]': finished 2024-02-23T00:46:17.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:16.153667+0000 mon.a (mon.0) 2046 : cluster [DBG] osdmap e304: 8 total, 8 up, 8 in 2024-02-23T00:46:17.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:16.262262+0000 mon.a (mon.0) 2047 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:46:17.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:16.144748+0000 mon.a (mon.0) 2042 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:46:17.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:16.153480+0000 mon.a (mon.0) 2043 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi047-82386-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi047-82386-52"}]': finished 2024-02-23T00:46:17.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:16.153558+0000 mon.a (mon.0) 2044 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-37", "mode": "writeback"}]': finished 2024-02-23T00:46:17.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:16.153620+0000 mon.a (mon.0) 2045 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "24"}]': finished 2024-02-23T00:46:17.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:16.153667+0000 mon.a (mon.0) 2046 : cluster [DBG] osdmap e304: 8 total, 8 up, 8 in 2024-02-23T00:46:17.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:16.262262+0000 mon.a (mon.0) 2047 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:46:17.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:16.144748+0000 mon.a (mon.0) 2042 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:46:17.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:16.153480+0000 mon.a (mon.0) 2043 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi047-82386-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi047-82386-52"}]': finished 2024-02-23T00:46:17.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:16.153558+0000 mon.a (mon.0) 2044 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-37", "mode": "writeback"}]': finished 2024-02-23T00:46:17.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:16.153620+0000 mon.a (mon.0) 2045 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "24"}]': finished 2024-02-23T00:46:17.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:16.153667+0000 mon.a (mon.0) 2046 : cluster [DBG] osdmap e304: 8 total, 8 up, 8 in 2024-02-23T00:46:17.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:16.262262+0000 mon.a (mon.0) 2047 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:46:17.633 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:46:17.637 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:17.637 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:46:17.738 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:46:17.775 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:17.796 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:46:17.798 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:17.799 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:46:17.899 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:46:17.933 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:18.163 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:16.940793+0000 mgr.y (mgr.14517) 305 : audit [DBG] from='client.41063 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:46:18.163 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:17.158043+0000 mon.a (mon.0) 2048 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:46:18.163 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:17.158102+0000 mon.a (mon.0) 2049 : cluster [DBG] osdmap e305: 8 total, 8 up, 8 in 2024-02-23T00:46:18.163 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:17.163990+0000 mon.a (mon.0) 2050 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-37"}]: dispatch 2024-02-23T00:46:18.163 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:17.255442+0000 mgr.y (mgr.14517) 306 : cluster [DBG] pgmap v433: 251 pgs: 5 creating+activating, 2 clean+premerge+peered, 244 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 87 op/s 2024-02-23T00:46:18.363 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:46:18.366 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:18.367 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:46:18.378 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:46:18.378 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:46:18.468 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:46:18.499 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:18.521 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:46:18.524 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:18.524 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:46:18.525 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:16.940793+0000 mgr.y (mgr.14517) 305 : audit [DBG] from='client.41063 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:46:18.525 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:17.158043+0000 mon.a (mon.0) 2048 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:46:18.525 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:17.158102+0000 mon.a (mon.0) 2049 : cluster [DBG] osdmap e305: 8 total, 8 up, 8 in 2024-02-23T00:46:18.525 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:17.163990+0000 mon.a (mon.0) 2050 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-37"}]: dispatch 2024-02-23T00:46:18.525 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:17.255442+0000 mgr.y (mgr.14517) 306 : cluster [DBG] pgmap v433: 251 pgs: 5 creating+activating, 2 clean+premerge+peered, 244 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 87 op/s 2024-02-23T00:46:18.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:16.940793+0000 mgr.y (mgr.14517) 305 : audit [DBG] from='client.41063 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:46:18.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:17.158043+0000 mon.a (mon.0) 2048 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:46:18.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:17.158102+0000 mon.a (mon.0) 2049 : cluster [DBG] osdmap e305: 8 total, 8 up, 8 in 2024-02-23T00:46:18.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:17.163990+0000 mon.a (mon.0) 2050 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-37"}]: dispatch 2024-02-23T00:46:18.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:17.255442+0000 mgr.y (mgr.14517) 306 : cluster [DBG] pgmap v433: 251 pgs: 5 creating+activating, 2 clean+premerge+peered, 244 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 87 op/s 2024-02-23T00:46:18.625 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:46:18.654 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:18.675 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:46:18.678 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:18.678 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:46:18.779 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:46:18.808 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:18.826 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:46:18.828 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:18.829 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:46:18.929 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:46:18.963 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:19.454 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:46:19.476 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":431,"stamp":"2024-02-23T00:46:16.149568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":9061,"num_objects":363,"num_object_clones":2,"num_object_copies":1089,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":363,"num_whiteouts":0,"num_read":1615,"num_read_kb":1241,"num_write":1274,"num_write_kb":648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":2249,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2589,"ondisk_log_size":2589,"up":729,"acting":729,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":737,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2495364,"kb_used_data":39692,"kb_used_omap":0,"kb_used_meta":2455552,"kb_avail":747465852,"statfs":{"total":767960285184,"available":765405032448,"internally_reserved":0,"allocated":40644608,"data_stored":24369613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2514485248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":136,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001197"},"pg_stats":[{"pgid":"184.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:12.143847+0000","last_change":"2024-02-23T00:46:12.150025+0000","last_active":"2024-02-23T00:46:12.143847+0000","last_peered":"2024-02-23T00:46:12.143847+0000","last_clean":"2024-02-23T00:46:12.143847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:12.143847+0000","last_undegraded":"2024-02-23T00:46:12.143847+0000","last_fullsized":"2024-02-23T00:46:12.143847+0000","mapping_epoch":300,"log_start":"0'0","ondisk_log_start":"0'0","created":300,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_clean_scrub_stamp":"2024-02-23T00:46:12.143847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"184.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:12.143847+0000","last_change":"2024-02-23T00:46:12.149865+0000","last_active":"2024-02-23T00:46:12.143847+0000","last_peered":"2024-02-23T00:46:12.143847+0000","last_clean":"2024-02-23T00:46:12.143847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:12.143847+0000","last_undegraded":"2024-02-23T00:46:12.143847+0000","last_fullsized":"2024-02-23T00:46:12.143847+0000","mapping_epoch":300,"log_start":"0'0","ondisk_log_start":"0'0","created":300,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_clean_scrub_stamp":"2024-02-23T00:46:12.143847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"184.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:12.143847+0000","last_change":"2024-02-23T00:46:12.153854+0000","last_active":"2024-02-23T00:46:12.143847+0000","last_peered":"2024-02-23T00:46:12.143847+0000","last_clean":"2024-02-23T00:46:12.143847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:12.143847+0000","last_undegraded":"2024-02-23T00:46:12.143847+0000","last_fullsized":"2024-02-23T00:46:12.143847+0000","mapping_epoch":300,"log_start":"0'0","ondisk_log_start":"0'0","created":300,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_clean_scrub_stamp":"2024-02-23T00:46:12.143847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"184.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:12.143847+0000","last_change":"2024-02-23T00:46:12.162586+0000","last_active":"2024-02-23T00:46:12.143847+0000","last_peered":"2024-02-23T00:46:12.143847+0000","last_clean":"2024-02-23T00:46:12.143847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:12.143847+0000","last_undegraded":"2024-02-23T00:46:12.143847+0000","last_fullsized":"2024-02-23T00:46:12.143847+0000","mapping_epoch":300,"log_start":"0'0","ondisk_log_start":"0'0","created":300,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_clean_scrub_stamp":"2024-02-23T00:46:12.143847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"184.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:12.143847+0000","last_change":"2024-02-23T00:46:12.162221+0000","last_active":"2024-02-23T00:46:12.143847+0000","last_peered":"2024-02-23T00:46:12.143847+0000","last_clean":"2024-02-23T00:46:12.143847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:12.143847+0000","last_undegraded":"2024-02-23T00:46:12.143847+0000","last_fullsized":"2024-02-23T00:46:12.143847+0000","mapping_epoch":300,"log_start":"0'0","ondisk_log_start":"0'0","created":300,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_clean_scrub_stamp":"2024-02-23T00:46:12.143847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"184.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:12.143847+0000","last_change":"2024-02-23T00:46:12.161837+0000","last_active":"2024-02-23T00:46:12.143847+0000","last_peered":"2024-02-23T00:46:12.143847+0000","last_clean":"2024-02-23T00:46:12.143847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:12.143847+0000","last_undegraded":"2024-02-23T00:46:12.143847+0000","last_fullsized":"2024-02-23T00:46:12.143847+0000","mapping_epoch":300,"log_start":"0'0","ondisk_log_start":"0'0","created":300,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_clean_scrub_stamp":"2024-02-23T00:46:12.143847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6,7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"184.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:12.143847+0000","last_change":"2024-02-23T00:46:12.161881+0000","last_active":"2024-02-23T00:46:12.143847+0000","last_peered":"2024-02-23T00:46:12.143847+0000","last_clean":"2024-02-23T00:46:12.143847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:12.143847+0000","last_undegraded":"2024-02-23T00:46:12.143847+0000","last_fullsized":"2024-02-23T00:46:12.143847+0000","mapping_epoch":300,"log_start":"0'0","ondisk_log_start":"0'0","created":300,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_clean_scrub_stamp":"2024-02-23T00:46:12.143847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"184.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:12.143847+0000","last_change":"2024-02-23T00:46:12.153856+0000","last_active":"2024-02-23T00:46:12.143847+0000","last_peered":"2024-02-23T00:46:12.143847+0000","last_clean":"2024-02-23T00:46:12.143847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:12.143847+0000","last_undegraded":"2024-02-23T00:46:12.143847+0000","last_fullsized":"2024-02-23T00:46:12.143847+0000","mapping_epoch":300,"log_start":"0'0","ondisk_log_start":"0'0","created":300,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_clean_scrub_stamp":"2024-02-23T00:46:12.143847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"184.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:12.143847+0000","last_change":"2024-02-23T00:46:12.159863+0000","last_active":"2024-02-23T00:46:12.143847+0000","last_peered":"2024-02-23T00:46:12.143847+0000","last_clean":"2024-02-23T00:46:12.143847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:12.143847+0000","last_undegraded":"2024-02-23T00:46:12.143847+0000","last_fullsized":"2024-02-23T00:46:12.143847+0000","mapping_epoch":300,"log_start":"0'0","ondisk_log_start":"0'0","created":300,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_clean_scrub_stamp":"2024-02-23T00:46:12.143847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"184.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:12.143847+0000","last_change":"2024-02-23T00:46:12.156843+0000","last_active":"2024-02-23T00:46:12.143847+0000","last_peered":"2024-02-23T00:46:12.143847+0000","last_clean":"2024-02-23T00:46:12.143847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:12.143847+0000","last_undegraded":"2024-02-23T00:46:12.143847+0000","last_fullsized":"2024-02-23T00:46:12.143847+0000","mapping_epoch":300,"log_start":"0'0","ondisk_log_start":"0'0","created":300,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_clean_scrub_stamp":"2024-02-23T00:46:12.143847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"184.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:12.143847+0000","last_change":"2024-02-23T00:46:12.162179+0000","last_active":"2024-02-23T00:46:12.143847+0000","last_peered":"2024-02-23T00:46:12.143847+0000","last_clean":"2024-02-23T00:46:12.143847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:12.143847+0000","last_undegraded":"2024-02-23T00:46:12.143847+0000","last_fullsized":"2024-02-23T00:46:12.143847+0000","mapping_epoch":300,"log_start":"0'0","ondisk_log_start":"0'0","created":300,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_clean_scrub_stamp":"2024-02-23T00:46:12.143847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"184.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:12.143847+0000","last_change":"2024-02-23T00:46:12.162020+0000","last_active":"2024-02-23T00:46:12.143847+0000","last_peered":"2024-02-23T00:46:12.143847+0000","last_clean":"2024-02-23T00:46:12.143847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:12.143847+0000","last_undegraded":"2024-02-23T00:46:12.143847+0000","last_fullsized":"2024-02-23T00:46:12.143847+0000","mapping_epoch":300,"log_start":"0'0","ondisk_log_start":"0'0","created":300,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_clean_scrub_stamp":"2024-02-23T00:46:12.143847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"184.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:12.143847+0000","last_change":"2024-02-23T00:46:12.155452+0000","last_active":"2024-02-23T00:46:12.143847+0000","last_peered":"2024-02-23T00:46:12.143847+0000","last_clean":"2024-02-23T00:46:12.143847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:12.143847+0000","last_undegraded":"2024-02-23T00:46:12.143847+0000","last_fullsized":"2024-02-23T00:46:12.143847+0000","mapping_epoch":300,"log_start":"0'0","ondisk_log_start":"0'0","created":300,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_clean_scrub_stamp":"2024-02-23T00:46:12.143847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"184.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:12.143847+0000","last_change":"2024-02-23T00:46:12.161348+0000","last_active":"2024-02-23T00:46:12.143847+0000","last_peered":"2024-02-23T00:46:12.143847+0000","last_clean":"2024-02-23T00:46:12.143847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:12.143847+0000","last_undegraded":"2024-02-23T00:46:12.143847+0000","last_fullsized":"2024-02-23T00:46:12.143847+0000","mapping_epoch":300,"log_start":"0'0","ondisk_log_start":"0'0","created":300,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_clean_scrub_stamp":"2024-02-23T00:46:12.143847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"184.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:12.143847+0000","last_change":"2024-02-23T00:46:12.157607+0000","last_active":"2024-02-23T00:46:12.143847+0000","last_peered":"2024-02-23T00:46:12.143847+0000","last_clean":"2024-02-23T00:46:12.143847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:12.143847+0000","last_undegraded":"2024-02-23T00:46:12.143847+0000","last_fullsized":"2024-02-23T00:46:12.143847+0000","mapping_epoch":300,"log_start":"0'0","ondisk_log_start":"0'0","created":300,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_clean_scrub_stamp":"2024-02-23T00:46:12.143847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"184.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:12.143847+0000","last_change":"2024-02-23T00:46:12.154624+0000","last_active":"2024-02-23T00:46:12.143847+0000","last_peered":"2024-02-23T00:46:12.143847+0000","last_clean":"2024-02-23T00:46:12.143847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:12.143847+0000","last_undegraded":"2024-02-23T00:46:12.143847+0000","last_fullsized":"2024-02-23T00:46:12.143847+0000","mapping_epoch":300,"log_start":"0'0","ondisk_log_start":"0'0","created":300,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_clean_scrub_stamp":"2024-02-23T00:46:12.143847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"185.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:16.144307+0000","last_change":"2024-02-23T00:46:16.144307+0000","last_active":"2024-02-23T00:46:16.144307+0000","last_peered":"2024-02-23T00:46:16.144307+0000","last_clean":"2024-02-23T00:46:16.144307+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:16.144307+0000","last_undegraded":"2024-02-23T00:46:16.144307+0000","last_fullsized":"2024-02-23T00:46:16.144307+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:16.144307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:16.144307+0000","last_clean_scrub_stamp":"2024-02-23T00:46:16.144307+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"184.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:12.143847+0000","last_change":"2024-02-23T00:46:12.163159+0000","last_active":"2024-02-23T00:46:12.143847+0000","last_peered":"2024-02-23T00:46:12.143847+0000","last_clean":"2024-02-23T00:46:12.143847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:12.143847+0000","last_undegraded":"2024-02-23T00:46:12.143847+0000","last_fullsized":"2024-02-23T00:46:12.143847+0000","mapping_epoch":300,"log_start":"0'0","ondisk_log_start":"0'0","created":300,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_clean_scrub_stamp":"2024-02-23T00:46:12.143847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"185.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:16.144307+0000","last_change":"2024-02-23T00:46:16.144307+0000","last_active":"2024-02-23T00:46:16.144307+0000","last_peered":"2024-02-23T00:46:16.144307+0000","last_clean":"2024-02-23T00:46:16.144307+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:16.144307+0000","last_undegraded":"2024-02-23T00:46:16.144307+0000","last_fullsized":"2024-02-23T00:46:16.144307+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:16.144307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:16.144307+0000","last_clean_scrub_stamp":"2024-02-23T00:46:16.144307+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"184.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:12.143847+0000","last_change":"2024-02-23T00:46:12.161987+0000","last_active":"2024-02-23T00:46:12.143847+0000","last_peered":"2024-02-23T00:46:12.143847+0000","last_clean":"2024-02-23T00:46:12.143847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:12.143847+0000","last_undegraded":"2024-02-23T00:46:12.143847+0000","last_fullsized":"2024-02-23T00:46:12.143847+0000","mapping_epoch":300,"log_start":"0'0","ondisk_log_start":"0'0","created":300,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_clean_scrub_stamp":"2024-02-23T00:46:12.143847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"185.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:16.144307+0000","last_change":"2024-02-23T00:46:16.144307+0000","last_active":"2024-02-23T00:46:16.144307+0000","last_peered":"2024-02-23T00:46:16.144307+0000","last_clean":"2024-02-23T00:46:16.144307+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:16.144307+0000","last_undegraded":"2024-02-23T00:46:16.144307+0000","last_fullsized":"2024-02-23T00:46:16.144307+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:16.144307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:16.144307+0000","last_clean_scrub_stamp":"2024-02-23T00:46:16.144307+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"184.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:12.143847+0000","last_change":"2024-02-23T00:46:12.150391+0000","last_active":"2024-02-23T00:46:12.143847+0000","last_peered":"2024-02-23T00:46:12.143847+0000","last_clean":"2024-02-23T00:46:12.143847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:12.143847+0000","last_undegraded":"2024-02-23T00:46:12.143847+0000","last_fullsized":"2024-02-23T00:46:12.143847+0000","mapping_epoch":300,"log_start":"0'0","ondisk_log_start":"0'0","created":300,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_clean_scrub_stamp":"2024-02-23T00:46:12.143847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"185.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:16.144307+0000","last_change":"2024-02-23T00:46:16.144307+0000","last_active":"2024-02-23T00:46:16.144307+0000","last_peered":"2024-02-23T00:46:16.144307+0000","last_clean":"2024-02-23T00:46:16.144307+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:16.144307+0000","last_undegraded":"2024-02-23T00:46:16.144307+0000","last_fullsized":"2024-02-23T00:46:16.144307+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:16.144307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:16.144307+0000","last_clean_scrub_stamp":"2024-02-23T00:46:16.144307+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"184.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:12.143847+0000","last_change":"2024-02-23T00:46:12.152973+0000","last_active":"2024-02-23T00:46:12.143847+0000","last_peered":"2024-02-23T00:46:12.143847+0000","last_clean":"2024-02-23T00:46:12.143847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:12.143847+0000","last_undegraded":"2024-02-23T00:46:12.143847+0000","last_fullsized":"2024-02-23T00:46:12.143847+0000","mapping_epoch":300,"log_start":"0'0","ondisk_log_start":"0'0","created":300,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_clean_scrub_stamp":"2024-02-23T00:46:12.143847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"185.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:16.144307+0000","last_change":"2024-02-23T00:46:16.144307+0000","last_active":"2024-02-23T00:46:16.144307+0000","last_peered":"2024-02-23T00:46:16.144307+0000","last_clean":"2024-02-23T00:46:16.144307+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:16.144307+0000","last_undegraded":"2024-02-23T00:46:16.144307+0000","last_fullsized":"2024-02-23T00:46:16.144307+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:16.144307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:16.144307+0000","last_clean_scrub_stamp":"2024-02-23T00:46:16.144307+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"184.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:12.143847+0000","last_change":"2024-02-23T00:46:12.152971+0000","last_active":"2024-02-23T00:46:12.143847+0000","last_peered":"2024-02-23T00:46:12.143847+0000","last_clean":"2024-02-23T00:46:12.143847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:12.143847+0000","last_undegraded":"2024-02-23T00:46:12.143847+0000","last_fullsized":"2024-02-23T00:46:12.143847+0000","mapping_epoch":300,"log_start":"0'0","ondisk_log_start":"0'0","created":300,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_clean_scrub_stamp":"2024-02-23T00:46:12.143847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"185.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:16.144307+0000","last_change":"2024-02-23T00:46:16.144307+0000","last_active":"2024-02-23T00:46:16.144307+0000","last_peered":"2024-02-23T00:46:16.144307+0000","last_clean":"2024-02-23T00:46:16.144307+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:16.144307+0000","last_undegraded":"2024-02-23T00:46:16.144307+0000","last_fullsized":"2024-02-23T00:46:16.144307+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:16.144307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:16.144307+0000","last_clean_scrub_stamp":"2024-02-23T00:46:16.144307+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"184.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:12.143847+0000","last_change":"2024-02-23T00:46:12.154970+0000","last_active":"2024-02-23T00:46:12.143847+0000","last_peered":"2024-02-23T00:46:12.143847+0000","last_clean":"2024-02-23T00:46:12.143847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:12.143847+0000","last_undegraded":"2024-02-23T00:46:12.143847+0000","last_fullsized":"2024-02-23T00:46:12.143847+0000","mapping_epoch":300,"log_start":"0'0","ondisk_log_start":"0'0","created":300,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_clean_scrub_stamp":"2024-02-23T00:46:12.143847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"185.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:16.144307+0000","last_change":"2024-02-23T00:46:16.144307+0000","last_active":"2024-02-23T00:46:16.144307+0000","last_peered":"2024-02-23T00:46:16.144307+0000","last_clean":"2024-02-23T00:46:16.144307+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:16.144307+0000","last_undegraded":"2024-02-23T00:46:16.144307+0000","last_fullsized":"2024-02-23T00:46:16.144307+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:16.144307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:16.144307+0000","last_clean_scrub_stamp":"2024-02-23T00:46:16.144307+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"184.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:12.143847+0000","last_change":"2024-02-23T00:46:12.150042+0000","last_active":"2024-02-23T00:46:12.143847+0000","last_peered":"2024-02-23T00:46:12.143847+0000","last_clean":"2024-02-23T00:46:12.143847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:12.143847+0000","last_undegraded":"2024-02-23T00:46:12.143847+0000","last_fullsized":"2024-02-23T00:46:12.143847+0000","mapping_epoch":300,"log_start":"0'0","ondisk_log_start":"0'0","created":300,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_clean_scrub_stamp":"2024-02-23T00:46:12.143847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"185.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:46:16.144307+0000","last_change":"2024-02-23T00:46:16.144307+0000","last_active":"2024-02-23T00:46:16.144307+0000","last_peered":"2024-02-23T00:46:16.144307+0000","last_clean":"2024-02-23T00:46:16.144307+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:16.144307+0000","last_undegraded":"2024-02-23T00:46:16.144307+0000","last_fullsized":"2024-02-23T00:46:16.144307+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:16.144307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:16.144307+0000","last_clean_scrub_stamp":"2024-02-23T00:46:16.144307+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"184.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:12.143847+0000","last_change":"2024-02-23T00:46:12.153373+0000","last_active":"2024-02-23T00:46:12.143847+0000","last_peered":"2024-02-23T00:46:12.143847+0000","last_clean":"2024-02-23T00:46:12.143847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:12.143847+0000","last_undegraded":"2024-02-23T00:46:12.143847+0000","last_fullsized":"2024-02-23T00:46:12.143847+0000","mapping_epoch":300,"log_start":"0'0","ondisk_log_start":"0'0","created":300,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_clean_scrub_stamp":"2024-02-23T00:46:12.143847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,5],"acting":[4,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"184.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:12.143847+0000","last_change":"2024-02-23T00:46:12.152643+0000","last_active":"2024-02-23T00:46:12.143847+0000","last_peered":"2024-02-23T00:46:12.143847+0000","last_clean":"2024-02-23T00:46:12.143847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:12.143847+0000","last_undegraded":"2024-02-23T00:46:12.143847+0000","last_fullsized":"2024-02-23T00:46:12.143847+0000","mapping_epoch":300,"log_start":"0'0","ondisk_log_start":"0'0","created":300,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_clean_scrub_stamp":"2024-02-23T00:46:12.143847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,3],"acting":[2,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"184.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:12.143847+0000","last_change":"2024-02-23T00:46:12.154972+0000","last_active":"2024-02-23T00:46:12.143847+0000","last_peered":"2024-02-23T00:46:12.143847+0000","last_clean":"2024-02-23T00:46:12.143847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:12.143847+0000","last_undegraded":"2024-02-23T00:46:12.143847+0000","last_fullsized":"2024-02-23T00:46:12.143847+0000","mapping_epoch":300,"log_start":"0'0","ondisk_log_start":"0'0","created":300,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_clean_scrub_stamp":"2024-02-23T00:46:12.143847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"184.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:12.143847+0000","last_change":"2024-02-23T00:46:12.152674+0000","last_active":"2024-02-23T00:46:12.143847+0000","last_peered":"2024-02-23T00:46:12.143847+0000","last_clean":"2024-02-23T00:46:12.143847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:12.143847+0000","last_undegraded":"2024-02-23T00:46:12.143847+0000","last_fullsized":"2024-02-23T00:46:12.143847+0000","mapping_epoch":300,"log_start":"0'0","ondisk_log_start":"0'0","created":300,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_clean_scrub_stamp":"2024-02-23T00:46:12.143847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"184.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:12.143847+0000","last_change":"2024-02-23T00:46:12.152811+0000","last_active":"2024-02-23T00:46:12.143847+0000","last_peered":"2024-02-23T00:46:12.143847+0000","last_clean":"2024-02-23T00:46:12.143847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:12.143847+0000","last_undegraded":"2024-02-23T00:46:12.143847+0000","last_fullsized":"2024-02-23T00:46:12.143847+0000","mapping_epoch":300,"log_start":"0'0","ondisk_log_start":"0'0","created":300,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_clean_scrub_stamp":"2024-02-23T00:46:12.143847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"184.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:12.143847+0000","last_change":"2024-02-23T00:46:12.153260+0000","last_active":"2024-02-23T00:46:12.143847+0000","last_peered":"2024-02-23T00:46:12.143847+0000","last_clean":"2024-02-23T00:46:12.143847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:12.143847+0000","last_undegraded":"2024-02-23T00:46:12.143847+0000","last_fullsized":"2024-02-23T00:46:12.143847+0000","mapping_epoch":300,"log_start":"0'0","ondisk_log_start":"0'0","created":300,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_clean_scrub_stamp":"2024-02-23T00:46:12.143847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"184.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:12.143847+0000","last_change":"2024-02-23T00:46:12.152628+0000","last_active":"2024-02-23T00:46:12.143847+0000","last_peered":"2024-02-23T00:46:12.143847+0000","last_clean":"2024-02-23T00:46:12.143847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:12.143847+0000","last_undegraded":"2024-02-23T00:46:12.143847+0000","last_fullsized":"2024-02-23T00:46:12.143847+0000","mapping_epoch":300,"log_start":"0'0","ondisk_log_start":"0'0","created":300,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_clean_scrub_stamp":"2024-02-23T00:46:12.143847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"184.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:12.143847+0000","last_change":"2024-02-23T00:46:12.149825+0000","last_active":"2024-02-23T00:46:12.143847+0000","last_peered":"2024-02-23T00:46:12.143847+0000","last_clean":"2024-02-23T00:46:12.143847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:12.143847+0000","last_undegraded":"2024-02-23T00:46:12.143847+0000","last_fullsized":"2024-02-23T00:46:12.143847+0000","mapping_epoch":300,"log_start":"0'0","ondisk_log_start":"0'0","created":300,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_clean_scrub_stamp":"2024-02-23T00:46:12.143847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"184.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:12.143847+0000","last_change":"2024-02-23T00:46:12.155794+0000","last_active":"2024-02-23T00:46:12.143847+0000","last_peered":"2024-02-23T00:46:12.143847+0000","last_clean":"2024-02-23T00:46:12.143847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:12.143847+0000","last_undegraded":"2024-02-23T00:46:12.143847+0000","last_fullsized":"2024-02-23T00:46:12.143847+0000","mapping_epoch":300,"log_start":"0'0","ondisk_log_start":"0'0","created":300,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:12.143847+0000","last_clean_scrub_stamp":"2024-02-23T00:46:12.143847+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.0","version":"259'40","reported_seq":269,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.159129+0000","last_change":"2024-02-23T00:46:06.778112+0000","last_active":"2024-02-23T00:46:11.159129+0000","last_peered":"2024-02-23T00:46:11.159129+0000","last_clean":"2024-02-23T00:46:11.159129+0000","last_became_active":"2024-02-23T00:43:17.855316+0000","last_became_peered":"2024-02-23T00:43:17.855316+0000","last_unstale":"2024-02-23T00:46:11.159129+0000","last_undegraded":"2024-02-23T00:46:11.159129+0000","last_fullsized":"2024-02-23T00:46:11.159129+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'40","last_scrub_stamp":"2024-02-23T00:46:06.777995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:06.777995+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":31,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":1,"num_write":13,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.1","version":"230'87","reported_seq":303,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.149105+0000","last_change":"2024-02-23T00:46:00.167004+0000","last_active":"2024-02-23T00:46:11.149105+0000","last_peered":"2024-02-23T00:46:11.149105+0000","last_clean":"2024-02-23T00:46:11.149105+0000","last_became_active":"2024-02-23T00:43:17.851866+0000","last_became_peered":"2024-02-23T00:43:17.851866+0000","last_unstale":"2024-02-23T00:46:11.149105+0000","last_undegraded":"2024-02-23T00:46:11.149105+0000","last_fullsized":"2024-02-23T00:46:11.149105+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":87,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":63,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.2","version":"229'68","reported_seq":288,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.168412+0000","last_change":"2024-02-23T00:46:00.168915+0000","last_active":"2024-02-23T00:46:11.168412+0000","last_peered":"2024-02-23T00:46:11.168412+0000","last_clean":"2024-02-23T00:46:11.168412+0000","last_became_active":"2024-02-23T00:43:17.804792+0000","last_became_peered":"2024-02-23T00:43:17.804792+0000","last_unstale":"2024-02-23T00:46:11.168412+0000","last_undegraded":"2024-02-23T00:46:11.168412+0000","last_fullsized":"2024-02-23T00:46:11.168412+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'68","last_scrub_stamp":"2024-02-23T00:45:57.135726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:57.135726+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.3","version":"246'20","reported_seq":251,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.153948+0000","last_change":"2024-02-23T00:46:00.172490+0000","last_active":"2024-02-23T00:46:11.153948+0000","last_peered":"2024-02-23T00:46:11.153948+0000","last_clean":"2024-02-23T00:46:11.153948+0000","last_became_active":"2024-02-23T00:43:17.834686+0000","last_became_peered":"2024-02-23T00:43:17.834686+0000","last_unstale":"2024-02-23T00:46:11.153948+0000","last_undegraded":"2024-02-23T00:46:11.153948+0000","last_fullsized":"2024-02-23T00:46:11.153948+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"246'20","last_scrub_stamp":"2024-02-23T00:45:53.067737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:53.067737+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.4","version":"207'17","reported_seq":239,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.157047+0000","last_change":"2024-02-23T00:46:00.183807+0000","last_active":"2024-02-23T00:46:11.157047+0000","last_peered":"2024-02-23T00:46:11.157047+0000","last_clean":"2024-02-23T00:46:11.157047+0000","last_became_active":"2024-02-23T00:43:17.810343+0000","last_became_peered":"2024-02-23T00:43:17.810343+0000","last_unstale":"2024-02-23T00:46:11.157047+0000","last_undegraded":"2024-02-23T00:46:11.157047+0000","last_fullsized":"2024-02-23T00:46:11.157047+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.5","version":"237'51","reported_seq":273,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.147435+0000","last_change":"2024-02-23T00:46:00.160547+0000","last_active":"2024-02-23T00:46:11.147435+0000","last_peered":"2024-02-23T00:46:11.147435+0000","last_clean":"2024-02-23T00:46:11.147435+0000","last_became_active":"2024-02-23T00:43:17.855339+0000","last_became_peered":"2024-02-23T00:43:17.855339+0000","last_unstale":"2024-02-23T00:46:11.147435+0000","last_undegraded":"2024-02-23T00:46:11.147435+0000","last_fullsized":"2024-02-23T00:46:11.147435+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.6","version":"247'51","reported_seq":279,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.165551+0000","last_change":"2024-02-23T00:46:00.726103+0000","last_active":"2024-02-23T00:46:11.165551+0000","last_peered":"2024-02-23T00:46:11.165551+0000","last_clean":"2024-02-23T00:46:11.165551+0000","last_became_active":"2024-02-23T00:43:17.838838+0000","last_became_peered":"2024-02-23T00:43:17.838838+0000","last_unstale":"2024-02-23T00:46:11.165551+0000","last_undegraded":"2024-02-23T00:46:11.165551+0000","last_fullsized":"2024-02-23T00:46:11.165551+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'51","last_scrub_stamp":"2024-02-23T00:46:00.726012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:00.726012+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.7","version":"215'68","reported_seq":281,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.156199+0000","last_change":"2024-02-23T00:46:00.175978+0000","last_active":"2024-02-23T00:46:11.156199+0000","last_peered":"2024-02-23T00:46:11.156199+0000","last_clean":"2024-02-23T00:46:11.156199+0000","last_became_active":"2024-02-23T00:43:17.846548+0000","last_became_peered":"2024-02-23T00:43:17.846548+0000","last_unstale":"2024-02-23T00:46:11.156199+0000","last_undegraded":"2024-02-23T00:46:11.156199+0000","last_fullsized":"2024-02-23T00:46:11.156199+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'68","last_scrub_stamp":"2024-02-23T00:45:25.351414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.351414+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.8","version":"259'90","reported_seq":305,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.159043+0000","last_change":"2024-02-23T00:46:05.820778+0000","last_active":"2024-02-23T00:46:11.159043+0000","last_peered":"2024-02-23T00:46:11.159043+0000","last_clean":"2024-02-23T00:46:11.159043+0000","last_became_active":"2024-02-23T00:43:17.842613+0000","last_became_peered":"2024-02-23T00:43:17.842613+0000","last_unstale":"2024-02-23T00:46:11.159043+0000","last_undegraded":"2024-02-23T00:46:11.159043+0000","last_fullsized":"2024-02-23T00:46:11.159043+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'90","last_scrub_stamp":"2024-02-23T00:46:05.820651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:05.820651+0000","log_size":90,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.9","version":"259'39","reported_seq":270,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.146428+0000","last_change":"2024-02-23T00:46:10.439444+0000","last_active":"2024-02-23T00:46:11.146428+0000","last_peered":"2024-02-23T00:46:11.146428+0000","last_clean":"2024-02-23T00:46:11.146428+0000","last_became_active":"2024-02-23T00:43:17.834297+0000","last_became_peered":"2024-02-23T00:43:17.834297+0000","last_unstale":"2024-02-23T00:46:11.146428+0000","last_undegraded":"2024-02-23T00:46:11.146428+0000","last_fullsized":"2024-02-23T00:46:11.146428+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'39","last_scrub_stamp":"2024-02-23T00:46:10.439373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:10.439373+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":12,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.a","version":"253'90","reported_seq":306,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.160143+0000","last_change":"2024-02-23T00:46:00.166527+0000","last_active":"2024-02-23T00:46:11.160143+0000","last_peered":"2024-02-23T00:46:11.160143+0000","last_clean":"2024-02-23T00:46:11.160143+0000","last_became_active":"2024-02-23T00:43:17.839142+0000","last_became_peered":"2024-02-23T00:43:17.839142+0000","last_unstale":"2024-02-23T00:46:11.160143+0000","last_undegraded":"2024-02-23T00:46:11.160143+0000","last_fullsized":"2024-02-23T00:46:11.160143+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"253'90","last_scrub_stamp":"2024-02-23T00:45:59.681669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:59.681669+0000","log_size":90,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.b","version":"288'76","reported_seq":293,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.165168+0000","last_change":"2024-02-23T00:46:07.817973+0000","last_active":"2024-02-23T00:46:11.165168+0000","last_peered":"2024-02-23T00:46:11.165168+0000","last_clean":"2024-02-23T00:46:11.165168+0000","last_became_active":"2024-02-23T00:43:17.855321+0000","last_became_peered":"2024-02-23T00:43:17.855321+0000","last_unstale":"2024-02-23T00:46:11.165168+0000","last_undegraded":"2024-02-23T00:46:11.165168+0000","last_fullsized":"2024-02-23T00:46:11.165168+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"288'76","last_scrub_stamp":"2024-02-23T00:46:07.817883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:07.817883+0000","log_size":76,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":46,"num_objects":6,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":23,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.c","version":"203'34","reported_seq":260,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.151300+0000","last_change":"2024-02-23T00:46:00.163652+0000","last_active":"2024-02-23T00:46:11.151300+0000","last_peered":"2024-02-23T00:46:11.151300+0000","last_clean":"2024-02-23T00:46:11.151300+0000","last_became_active":"2024-02-23T00:43:17.829044+0000","last_became_peered":"2024-02-23T00:43:17.829044+0000","last_unstale":"2024-02-23T00:46:11.151300+0000","last_undegraded":"2024-02-23T00:46:11.151300+0000","last_fullsized":"2024-02-23T00:46:11.151300+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'34","last_scrub_stamp":"2024-02-23T00:45:57.874883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:57.874883+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.d","version":"259'56","reported_seq":275,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.147331+0000","last_change":"2024-02-23T00:46:00.159535+0000","last_active":"2024-02-23T00:46:11.147331+0000","last_peered":"2024-02-23T00:46:11.147331+0000","last_clean":"2024-02-23T00:46:11.147331+0000","last_became_active":"2024-02-23T00:43:17.841614+0000","last_became_peered":"2024-02-23T00:43:17.841614+0000","last_unstale":"2024-02-23T00:46:11.147331+0000","last_undegraded":"2024-02-23T00:46:11.147331+0000","last_fullsized":"2024-02-23T00:46:11.147331+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'56","last_scrub_stamp":"2024-02-23T00:45:51.258745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:51.258745+0000","log_size":56,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":17,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.e","version":"211'51","reported_seq":271,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.491740+0000","last_change":"2024-02-23T00:46:11.491708+0000","last_active":"2024-02-23T00:46:11.491740+0000","last_peered":"2024-02-23T00:46:11.491740+0000","last_clean":"2024-02-23T00:46:11.491740+0000","last_became_active":"2024-02-23T00:43:17.845956+0000","last_became_peered":"2024-02-23T00:43:17.845956+0000","last_unstale":"2024-02-23T00:46:11.491740+0000","last_undegraded":"2024-02-23T00:46:11.491740+0000","last_fullsized":"2024-02-23T00:46:11.491740+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'51","last_scrub_stamp":"2024-02-23T00:46:11.491637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:11.491637+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.f","version":"241'136","reported_seq":337,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.149936+0000","last_change":"2024-02-23T00:46:00.160973+0000","last_active":"2024-02-23T00:46:11.149936+0000","last_peered":"2024-02-23T00:46:11.149936+0000","last_clean":"2024-02-23T00:46:11.149936+0000","last_became_active":"2024-02-23T00:43:17.855331+0000","last_became_peered":"2024-02-23T00:43:17.855331+0000","last_unstale":"2024-02-23T00:46:11.149936+0000","last_undegraded":"2024-02-23T00:46:11.149936+0000","last_fullsized":"2024-02-23T00:46:11.149936+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":136,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":88,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":40,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.10","version":"219'68","reported_seq":284,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.150171+0000","last_change":"2024-02-23T00:46:00.167847+0000","last_active":"2024-02-23T00:46:11.150171+0000","last_peered":"2024-02-23T00:46:11.150171+0000","last_clean":"2024-02-23T00:46:11.150171+0000","last_became_active":"2024-02-23T00:43:17.819865+0000","last_became_peered":"2024-02-23T00:43:17.819865+0000","last_unstale":"2024-02-23T00:46:11.150171+0000","last_undegraded":"2024-02-23T00:46:11.150171+0000","last_fullsized":"2024-02-23T00:46:11.150171+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.11","version":"215'51","reported_seq":264,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.149008+0000","last_change":"2024-02-23T00:46:00.164366+0000","last_active":"2024-02-23T00:46:11.149008+0000","last_peered":"2024-02-23T00:46:11.149008+0000","last_clean":"2024-02-23T00:46:11.149008+0000","last_became_active":"2024-02-23T00:43:17.852697+0000","last_became_peered":"2024-02-23T00:43:17.852697+0000","last_unstale":"2024-02-23T00:46:11.149008+0000","last_undegraded":"2024-02-23T00:46:11.149008+0000","last_fullsized":"2024-02-23T00:46:11.149008+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.12","version":"240'73","reported_seq":295,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.155211+0000","last_change":"2024-02-23T00:46:00.176356+0000","last_active":"2024-02-23T00:46:11.155211+0000","last_peered":"2024-02-23T00:46:11.155211+0000","last_clean":"2024-02-23T00:46:11.155211+0000","last_became_active":"2024-02-23T00:43:17.856118+0000","last_became_peered":"2024-02-23T00:43:17.856118+0000","last_unstale":"2024-02-23T00:46:11.155211+0000","last_undegraded":"2024-02-23T00:46:11.155211+0000","last_fullsized":"2024-02-23T00:46:11.155211+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'73","last_scrub_stamp":"2024-02-23T00:45:52.066508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:52.066508+0000","log_size":73,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":49,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.13","version":"205'34","reported_seq":261,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.160823+0000","last_change":"2024-02-23T00:46:09.499938+0000","last_active":"2024-02-23T00:46:11.160823+0000","last_peered":"2024-02-23T00:46:11.160823+0000","last_clean":"2024-02-23T00:46:11.160823+0000","last_became_active":"2024-02-23T00:43:17.857823+0000","last_became_peered":"2024-02-23T00:43:17.857823+0000","last_unstale":"2024-02-23T00:46:11.160823+0000","last_undegraded":"2024-02-23T00:46:11.160823+0000","last_fullsized":"2024-02-23T00:46:11.160823+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'34","last_scrub_stamp":"2024-02-23T00:46:09.499812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:09.499812+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.14","version":"296'44","reported_seq":267,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.164799+0000","last_change":"2024-02-23T00:46:03.795687+0000","last_active":"2024-02-23T00:46:11.164799+0000","last_peered":"2024-02-23T00:46:11.164799+0000","last_clean":"2024-02-23T00:46:11.164799+0000","last_became_active":"2024-02-23T00:43:17.816138+0000","last_became_peered":"2024-02-23T00:43:17.816138+0000","last_unstale":"2024-02-23T00:46:11.164799+0000","last_undegraded":"2024-02-23T00:46:11.164799+0000","last_fullsized":"2024-02-23T00:46:11.164799+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'39","last_scrub_stamp":"2024-02-23T00:46:03.795605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:03.795605+0000","log_size":44,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":35,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":17,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.15","version":"299'18","reported_seq":248,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.830666+0000","last_change":"2024-02-23T00:46:11.830641+0000","last_active":"2024-02-23T00:46:11.830666+0000","last_peered":"2024-02-23T00:46:11.830666+0000","last_clean":"2024-02-23T00:46:11.830666+0000","last_became_active":"2024-02-23T00:43:17.843834+0000","last_became_peered":"2024-02-23T00:43:17.843834+0000","last_unstale":"2024-02-23T00:46:11.830666+0000","last_undegraded":"2024-02-23T00:46:11.830666+0000","last_fullsized":"2024-02-23T00:46:11.830666+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"299'18","last_scrub_stamp":"2024-02-23T00:46:11.830580+0000","last_deep_scrub":"299'18","last_deep_scrub_stamp":"2024-02-23T00:46:11.830580+0000","last_clean_scrub_stamp":"2024-02-23T00:46:11.830580+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.16","version":"240'23","reported_seq":253,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.148683+0000","last_change":"2024-02-23T00:46:00.160705+0000","last_active":"2024-02-23T00:46:11.148683+0000","last_peered":"2024-02-23T00:46:11.148683+0000","last_clean":"2024-02-23T00:46:11.148683+0000","last_became_active":"2024-02-23T00:43:17.834291+0000","last_became_peered":"2024-02-23T00:43:17.834291+0000","last_unstale":"2024-02-23T00:46:11.148683+0000","last_undegraded":"2024-02-23T00:46:11.148683+0000","last_fullsized":"2024-02-23T00:46:11.148683+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'23","last_scrub_stamp":"2024-02-23T00:45:56.354881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:56.354881+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":14,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.17","version":"229'57","reported_seq":286,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.163998+0000","last_change":"2024-02-23T00:46:00.166482+0000","last_active":"2024-02-23T00:46:11.163998+0000","last_peered":"2024-02-23T00:46:11.163998+0000","last_clean":"2024-02-23T00:46:11.163998+0000","last_became_active":"2024-02-23T00:43:17.843525+0000","last_became_peered":"2024-02-23T00:43:17.843525+0000","last_unstale":"2024-02-23T00:46:11.163998+0000","last_undegraded":"2024-02-23T00:46:11.163998+0000","last_fullsized":"2024-02-23T00:46:11.163998+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'57","last_scrub_stamp":"2024-02-23T00:45:58.127302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:58.127302+0000","log_size":57,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":18,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.19","version":"267'96","reported_seq":320,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.166978+0000","last_change":"2024-02-23T00:46:00.163245+0000","last_active":"2024-02-23T00:46:11.166978+0000","last_peered":"2024-02-23T00:46:11.166978+0000","last_clean":"2024-02-23T00:46:11.166978+0000","last_became_active":"2024-02-23T00:43:17.815830+0000","last_became_peered":"2024-02-23T00:43:17.815830+0000","last_unstale":"2024-02-23T00:46:11.166978+0000","last_undegraded":"2024-02-23T00:46:11.166978+0000","last_fullsized":"2024-02-23T00:46:11.166978+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'96","last_scrub_stamp":"2024-02-23T00:45:55.109551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:55.109551+0000","log_size":96,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":73,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":7,"num_read_kb":0,"num_write":30,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.18","version":"275'7","reported_seq":242,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.145434+0000","last_change":"2024-02-23T00:46:03.927901+0000","last_active":"2024-02-23T00:46:11.145434+0000","last_peered":"2024-02-23T00:46:11.145434+0000","last_clean":"2024-02-23T00:46:11.145434+0000","last_became_active":"2024-02-23T00:43:17.840351+0000","last_became_peered":"2024-02-23T00:43:17.840351+0000","last_unstale":"2024-02-23T00:46:11.145434+0000","last_undegraded":"2024-02-23T00:46:11.145434+0000","last_fullsized":"2024-02-23T00:46:11.145434+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"275'7","last_scrub_stamp":"2024-02-23T00:46:03.927783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:03.927783+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1b","version":"229'68","reported_seq":283,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.146641+0000","last_change":"2024-02-23T00:46:00.158327+0000","last_active":"2024-02-23T00:46:11.146641+0000","last_peered":"2024-02-23T00:46:11.146641+0000","last_clean":"2024-02-23T00:46:11.146641+0000","last_became_active":"2024-02-23T00:43:17.803043+0000","last_became_peered":"2024-02-23T00:43:17.803043+0000","last_unstale":"2024-02-23T00:46:11.146641+0000","last_undegraded":"2024-02-23T00:46:11.146641+0000","last_fullsized":"2024-02-23T00:46:11.146641+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.1a","version":"225'34","reported_seq":288,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.168317+0000","last_change":"2024-02-23T00:46:02.071916+0000","last_active":"2024-02-23T00:46:11.168317+0000","last_peered":"2024-02-23T00:46:11.168317+0000","last_clean":"2024-02-23T00:46:11.168317+0000","last_became_active":"2024-02-23T00:45:44.178195+0000","last_became_peered":"2024-02-23T00:45:44.178195+0000","last_unstale":"2024-02-23T00:46:11.168317+0000","last_undegraded":"2024-02-23T00:46:11.168317+0000","last_fullsized":"2024-02-23T00:46:11.168317+0000","mapping_epoch":271,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":272,"parent":"0.0","parent_split_bits":0,"last_scrub":"225'34","last_scrub_stamp":"2024-02-23T00:46:02.071770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:02.071770+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1d","version":"215'104","reported_seq":322,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.157803+0000","last_change":"2024-02-23T00:46:05.000709+0000","last_active":"2024-02-23T00:46:11.157803+0000","last_peered":"2024-02-23T00:46:11.157803+0000","last_clean":"2024-02-23T00:46:11.157803+0000","last_became_active":"2024-02-23T00:43:17.828194+0000","last_became_peered":"2024-02-23T00:43:17.828194+0000","last_unstale":"2024-02-23T00:46:11.157803+0000","last_undegraded":"2024-02-23T00:46:11.157803+0000","last_fullsized":"2024-02-23T00:46:11.157803+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'104","last_scrub_stamp":"2024-02-23T00:46:05.000615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:05.000615+0000","log_size":104,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":74,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":32,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"62.1c","version":"223'51","reported_seq":271,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.168542+0000","last_change":"2024-02-23T00:46:08.027717+0000","last_active":"2024-02-23T00:46:11.168542+0000","last_peered":"2024-02-23T00:46:11.168542+0000","last_clean":"2024-02-23T00:46:11.168542+0000","last_became_active":"2024-02-23T00:43:17.804718+0000","last_became_peered":"2024-02-23T00:43:17.804718+0000","last_unstale":"2024-02-23T00:46:11.168542+0000","last_undegraded":"2024-02-23T00:46:11.168542+0000","last_fullsized":"2024-02-23T00:46:11.168542+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'51","last_scrub_stamp":"2024-02-23T00:46:08.027648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:08.027648+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1f","version":"240'40","reported_seq":268,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.156104+0000","last_change":"2024-02-23T00:46:00.174886+0000","last_active":"2024-02-23T00:46:11.156104+0000","last_peered":"2024-02-23T00:46:11.156104+0000","last_clean":"2024-02-23T00:46:11.156104+0000","last_became_active":"2024-02-23T00:43:17.839914+0000","last_became_peered":"2024-02-23T00:43:17.839914+0000","last_unstale":"2024-02-23T00:46:11.156104+0000","last_undegraded":"2024-02-23T00:46:11.156104+0000","last_fullsized":"2024-02-23T00:46:11.156104+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'40","last_scrub_stamp":"2024-02-23T00:45:58.074860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:58.074860+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":13,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.1e","version":"245'51","reported_seq":272,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.165447+0000","last_change":"2024-02-23T00:46:00.172469+0000","last_active":"2024-02-23T00:46:11.165447+0000","last_peered":"2024-02-23T00:46:11.165447+0000","last_clean":"2024-02-23T00:46:11.165447+0000","last_became_active":"2024-02-23T00:43:17.838682+0000","last_became_peered":"2024-02-23T00:43:17.838682+0000","last_unstale":"2024-02-23T00:46:11.165447+0000","last_undegraded":"2024-02-23T00:46:11.165447+0000","last_fullsized":"2024-02-23T00:46:11.165447+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"245'51","last_scrub_stamp":"2024-02-23T00:45:58.723990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:58.723990+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"67'8","reported_seq":351,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.161633+0000","last_change":"2024-02-23T00:45:55.708656+0000","last_active":"2024-02-23T00:46:11.161633+0000","last_peered":"2024-02-23T00:46:11.161633+0000","last_clean":"2024-02-23T00:46:11.161633+0000","last_became_active":"2024-02-23T00:39:18.793414+0000","last_became_peered":"2024-02-23T00:39:18.793414+0000","last_unstale":"2024-02-23T00:46:11.161633+0000","last_undegraded":"2024-02-23T00:46:11.161633+0000","last_fullsized":"2024-02-23T00:46:11.161633+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:45:55.708576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:55.708576+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"68'27","reported_seq":317,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.155437+0000","last_change":"2024-02-23T00:45:04.138883+0000","last_active":"2024-02-23T00:46:11.155437+0000","last_peered":"2024-02-23T00:46:11.155437+0000","last_clean":"2024-02-23T00:46:11.155437+0000","last_became_active":"2024-02-23T00:39:16.859401+0000","last_became_peered":"2024-02-23T00:39:16.859401+0000","last_unstale":"2024-02-23T00:46:11.155437+0000","last_undegraded":"2024-02-23T00:46:11.155437+0000","last_fullsized":"2024-02-23T00:46:11.155437+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'27","last_scrub_stamp":"2024-02-23T00:45:04.138751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.138751+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":267,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.153977+0000","last_change":"2024-02-23T00:46:04.942156+0000","last_active":"2024-02-23T00:46:11.153977+0000","last_peered":"2024-02-23T00:46:11.153977+0000","last_clean":"2024-02-23T00:46:11.153977+0000","last_became_active":"2024-02-23T00:39:14.784436+0000","last_became_peered":"2024-02-23T00:39:14.784436+0000","last_unstale":"2024-02-23T00:46:11.153977+0000","last_undegraded":"2024-02-23T00:46:11.153977+0000","last_fullsized":"2024-02-23T00:46:11.153977+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.942091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:04.330426+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.942091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":276,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.156422+0000","last_change":"2024-02-23T00:44:21.972446+0000","last_active":"2024-02-23T00:46:11.156422+0000","last_peered":"2024-02-23T00:46:11.156422+0000","last_clean":"2024-02-23T00:46:11.156422+0000","last_became_active":"2024-02-23T00:43:32.282545+0000","last_became_peered":"2024-02-23T00:43:32.282545+0000","last_unstale":"2024-02-23T00:46:11.156422+0000","last_undegraded":"2024-02-23T00:46:11.156422+0000","last_fullsized":"2024-02-23T00:46:11.156422+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:21.972383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:21.972383+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.3","version":"110'5","reported_seq":210,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.146699+0000","last_change":"2024-02-23T00:45:17.356489+0000","last_active":"2024-02-23T00:46:11.146699+0000","last_peered":"2024-02-23T00:46:11.146699+0000","last_clean":"2024-02-23T00:46:11.146699+0000","last_became_active":"2024-02-23T00:42:58.835612+0000","last_became_peered":"2024-02-23T00:42:58.835612+0000","last_unstale":"2024-02-23T00:46:11.146699+0000","last_undegraded":"2024-02-23T00:46:11.146699+0000","last_fullsized":"2024-02-23T00:46:11.146699+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'5","last_scrub_stamp":"2024-02-23T00:45:17.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:17.356406+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":205,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.156948+0000","last_change":"2024-02-23T00:45:17.181400+0000","last_active":"2024-02-23T00:46:11.156948+0000","last_peered":"2024-02-23T00:46:11.156948+0000","last_clean":"2024-02-23T00:46:11.156948+0000","last_became_active":"2024-02-23T00:42:58.828846+0000","last_became_peered":"2024-02-23T00:42:58.828846+0000","last_unstale":"2024-02-23T00:46:11.156948+0000","last_undegraded":"2024-02-23T00:46:11.156948+0000","last_fullsized":"2024-02-23T00:46:11.156948+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:17.181335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:17.181335+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"68'15","reported_seq":293,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.161217+0000","last_change":"2024-02-23T00:43:54.355050+0000","last_active":"2024-02-23T00:46:11.161217+0000","last_peered":"2024-02-23T00:46:11.161217+0000","last_clean":"2024-02-23T00:46:11.161217+0000","last_became_active":"2024-02-23T00:39:16.783701+0000","last_became_peered":"2024-02-23T00:39:16.783701+0000","last_unstale":"2024-02-23T00:46:11.161217+0000","last_undegraded":"2024-02-23T00:46:11.161217+0000","last_fullsized":"2024-02-23T00:46:11.161217+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:43:54.354974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:54.354974+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":263,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.150229+0000","last_change":"2024-02-23T00:46:04.476277+0000","last_active":"2024-02-23T00:46:11.150229+0000","last_peered":"2024-02-23T00:46:11.150229+0000","last_clean":"2024-02-23T00:46:11.150229+0000","last_became_active":"2024-02-23T00:39:14.772493+0000","last_became_peered":"2024-02-23T00:39:14.772493+0000","last_unstale":"2024-02-23T00:46:11.150229+0000","last_undegraded":"2024-02-23T00:46:11.150229+0000","last_fullsized":"2024-02-23T00:46:11.150229+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.476156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.476156+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":258,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.155368+0000","last_change":"2024-02-23T00:44:22.380954+0000","last_active":"2024-02-23T00:46:11.155368+0000","last_peered":"2024-02-23T00:46:11.155368+0000","last_clean":"2024-02-23T00:46:11.155368+0000","last_became_active":"2024-02-23T00:39:18.794476+0000","last_became_peered":"2024-02-23T00:39:18.794476+0000","last_unstale":"2024-02-23T00:46:11.155368+0000","last_undegraded":"2024-02-23T00:46:11.155368+0000","last_fullsized":"2024-02-23T00:46:11.155368+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:22.380841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:08.335330+0000","last_clean_scrub_stamp":"2024-02-23T00:44:22.380841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":276,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.159152+0000","last_change":"2024-02-23T00:45:50.685890+0000","last_active":"2024-02-23T00:46:11.159152+0000","last_peered":"2024-02-23T00:46:11.159152+0000","last_clean":"2024-02-23T00:46:11.159152+0000","last_became_active":"2024-02-23T00:43:35.079004+0000","last_became_peered":"2024-02-23T00:43:35.079004+0000","last_unstale":"2024-02-23T00:46:11.159152+0000","last_undegraded":"2024-02-23T00:46:11.159152+0000","last_fullsized":"2024-02-23T00:46:11.159152+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:50.685824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:45:50.685824+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"110'5","reported_seq":209,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.155336+0000","last_change":"2024-02-23T00:45:06.483322+0000","last_active":"2024-02-23T00:46:11.155336+0000","last_peered":"2024-02-23T00:46:11.155336+0000","last_clean":"2024-02-23T00:46:11.155336+0000","last_became_active":"2024-02-23T00:42:58.741558+0000","last_became_peered":"2024-02-23T00:42:58.741558+0000","last_unstale":"2024-02-23T00:46:11.155336+0000","last_undegraded":"2024-02-23T00:46:11.155336+0000","last_fullsized":"2024-02-23T00:46:11.155336+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'5","last_scrub_stamp":"2024-02-23T00:45:06.483252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.483252+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":205,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.163371+0000","last_change":"2024-02-23T00:45:22.597544+0000","last_active":"2024-02-23T00:46:11.163371+0000","last_peered":"2024-02-23T00:46:11.163371+0000","last_clean":"2024-02-23T00:46:11.163371+0000","last_became_active":"2024-02-23T00:42:58.867305+0000","last_became_peered":"2024-02-23T00:42:58.867305+0000","last_unstale":"2024-02-23T00:46:11.163371+0000","last_undegraded":"2024-02-23T00:46:11.163371+0000","last_fullsized":"2024-02-23T00:46:11.163371+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:22.597434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:22.597434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"88'35","reported_seq":316,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.168860+0000","last_change":"2024-02-23T00:43:54.438282+0000","last_active":"2024-02-23T00:46:11.168860+0000","last_peered":"2024-02-23T00:46:11.168860+0000","last_clean":"2024-02-23T00:46:11.168860+0000","last_became_active":"2024-02-23T00:39:16.857413+0000","last_became_peered":"2024-02-23T00:39:16.857413+0000","last_unstale":"2024-02-23T00:46:11.168860+0000","last_undegraded":"2024-02-23T00:46:11.168860+0000","last_fullsized":"2024-02-23T00:46:11.168860+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'35","last_scrub_stamp":"2024-02-23T00:43:54.438168+0000","last_deep_scrub":"88'35","last_deep_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_clean_scrub_stamp":"2024-02-23T00:43:54.438168+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":267,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.153760+0000","last_change":"2024-02-23T00:46:05.910392+0000","last_active":"2024-02-23T00:46:11.153760+0000","last_peered":"2024-02-23T00:46:11.153760+0000","last_clean":"2024-02-23T00:46:11.153760+0000","last_became_active":"2024-02-23T00:39:14.776705+0000","last_became_peered":"2024-02-23T00:39:14.776705+0000","last_unstale":"2024-02-23T00:46:11.153760+0000","last_undegraded":"2024-02-23T00:46:11.153760+0000","last_fullsized":"2024-02-23T00:46:11.153760+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:05.910325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_clean_scrub_stamp":"2024-02-23T00:46:05.910325+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":258,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.154054+0000","last_change":"2024-02-23T00:43:44.313672+0000","last_active":"2024-02-23T00:46:11.154054+0000","last_peered":"2024-02-23T00:46:11.154054+0000","last_clean":"2024-02-23T00:46:11.154054+0000","last_became_active":"2024-02-23T00:39:18.792718+0000","last_became_peered":"2024-02-23T00:39:18.792718+0000","last_unstale":"2024-02-23T00:46:11.154054+0000","last_undegraded":"2024-02-23T00:46:11.154054+0000","last_fullsized":"2024-02-23T00:46:11.154054+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:44.313501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:44.313501+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":273,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.159212+0000","last_change":"2024-02-23T00:44:36.758043+0000","last_active":"2024-02-23T00:46:11.159212+0000","last_peered":"2024-02-23T00:46:11.159212+0000","last_clean":"2024-02-23T00:46:11.159212+0000","last_became_active":"2024-02-23T00:43:39.470999+0000","last_became_peered":"2024-02-23T00:43:39.470999+0000","last_unstale":"2024-02-23T00:46:11.159212+0000","last_undegraded":"2024-02-23T00:46:11.159212+0000","last_fullsized":"2024-02-23T00:46:11.159212+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:36.757960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:36.757960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"110'5","reported_seq":209,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.167711+0000","last_change":"2024-02-23T00:45:23.832372+0000","last_active":"2024-02-23T00:46:11.167711+0000","last_peered":"2024-02-23T00:46:11.167711+0000","last_clean":"2024-02-23T00:46:11.167711+0000","last_became_active":"2024-02-23T00:42:58.684001+0000","last_became_peered":"2024-02-23T00:42:58.684001+0000","last_unstale":"2024-02-23T00:46:11.167711+0000","last_undegraded":"2024-02-23T00:46:11.167711+0000","last_fullsized":"2024-02-23T00:46:11.167711+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'5","last_scrub_stamp":"2024-02-23T00:45:23.832291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.832291+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":200,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.149108+0000","last_change":"2024-02-23T00:42:57.641727+0000","last_active":"2024-02-23T00:46:11.149108+0000","last_peered":"2024-02-23T00:46:11.149108+0000","last_clean":"2024-02-23T00:46:11.149108+0000","last_became_active":"2024-02-23T00:42:57.639013+0000","last_became_peered":"2024-02-23T00:42:57.639013+0000","last_unstale":"2024-02-23T00:46:11.149108+0000","last_undegraded":"2024-02-23T00:46:11.149108+0000","last_fullsized":"2024-02-23T00:46:11.149108+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"67'17","reported_seq":311,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.148029+0000","last_change":"2024-02-23T00:43:45.194207+0000","last_active":"2024-02-23T00:46:11.148029+0000","last_peered":"2024-02-23T00:46:11.148029+0000","last_clean":"2024-02-23T00:46:11.148029+0000","last_became_active":"2024-02-23T00:40:42.933806+0000","last_became_peered":"2024-02-23T00:40:42.933806+0000","last_unstale":"2024-02-23T00:46:11.148029+0000","last_undegraded":"2024-02-23T00:46:11.148029+0000","last_fullsized":"2024-02-23T00:46:11.148029+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'17","last_scrub_stamp":"2024-02-23T00:43:45.194139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.194139+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":238,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.154339+0000","last_change":"2024-02-23T00:45:56.051644+0000","last_active":"2024-02-23T00:46:11.154339+0000","last_peered":"2024-02-23T00:46:11.154339+0000","last_clean":"2024-02-23T00:46:11.154339+0000","last_became_active":"2024-02-23T00:40:42.931041+0000","last_became_peered":"2024-02-23T00:40:42.931041+0000","last_unstale":"2024-02-23T00:46:11.154339+0000","last_undegraded":"2024-02-23T00:46:11.154339+0000","last_fullsized":"2024-02-23T00:46:11.154339+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:56.051575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:56.051575+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":259,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.150395+0000","last_change":"2024-02-23T00:44:22.804352+0000","last_active":"2024-02-23T00:46:11.150395+0000","last_peered":"2024-02-23T00:46:11.150395+0000","last_clean":"2024-02-23T00:46:11.150395+0000","last_became_active":"2024-02-23T00:39:18.799875+0000","last_became_peered":"2024-02-23T00:39:18.799875+0000","last_unstale":"2024-02-23T00:46:11.150395+0000","last_undegraded":"2024-02-23T00:46:11.150395+0000","last_fullsized":"2024-02-23T00:46:11.150395+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:22.804279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:22.804279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":274,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.156568+0000","last_change":"2024-02-23T00:44:36.481327+0000","last_active":"2024-02-23T00:46:11.156568+0000","last_peered":"2024-02-23T00:46:11.156568+0000","last_clean":"2024-02-23T00:46:11.156568+0000","last_became_active":"2024-02-23T00:43:08.885257+0000","last_became_peered":"2024-02-23T00:43:08.885257+0000","last_unstale":"2024-02-23T00:46:11.156568+0000","last_undegraded":"2024-02-23T00:46:11.156568+0000","last_fullsized":"2024-02-23T00:46:11.156568+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:36.481256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:36.481256+0000","last_clean_scrub_stamp":"2024-02-23T00:44:36.481256+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.6","version":"110'9","reported_seq":214,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.162713+0000","last_change":"2024-02-23T00:46:08.530503+0000","last_active":"2024-02-23T00:46:11.162713+0000","last_peered":"2024-02-23T00:46:11.162713+0000","last_clean":"2024-02-23T00:46:11.162713+0000","last_became_active":"2024-02-23T00:42:58.870147+0000","last_became_peered":"2024-02-23T00:42:58.870147+0000","last_unstale":"2024-02-23T00:46:11.162713+0000","last_undegraded":"2024-02-23T00:46:11.162713+0000","last_fullsized":"2024-02-23T00:46:11.162713+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'9","last_scrub_stamp":"2024-02-23T00:46:08.530434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:46:08.530434+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":205,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.146946+0000","last_change":"2024-02-23T00:45:23.744990+0000","last_active":"2024-02-23T00:46:11.146946+0000","last_peered":"2024-02-23T00:46:11.146946+0000","last_clean":"2024-02-23T00:46:11.146946+0000","last_became_active":"2024-02-23T00:42:57.675306+0000","last_became_peered":"2024-02-23T00:42:57.675306+0000","last_unstale":"2024-02-23T00:46:11.146946+0000","last_undegraded":"2024-02-23T00:46:11.146946+0000","last_fullsized":"2024-02-23T00:46:11.146946+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:23.744846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.744846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"67'10","reported_seq":276,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.147981+0000","last_change":"2024-02-23T00:45:25.175258+0000","last_active":"2024-02-23T00:46:11.147981+0000","last_peered":"2024-02-23T00:46:11.147981+0000","last_clean":"2024-02-23T00:46:11.147981+0000","last_became_active":"2024-02-23T00:39:16.777530+0000","last_became_peered":"2024-02-23T00:39:16.777530+0000","last_unstale":"2024-02-23T00:46:11.147981+0000","last_undegraded":"2024-02-23T00:46:11.147981+0000","last_fullsized":"2024-02-23T00:46:11.147981+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'10","last_scrub_stamp":"2024-02-23T00:45:25.175173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.175173+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":263,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.147376+0000","last_change":"2024-02-23T00:45:10.693332+0000","last_active":"2024-02-23T00:46:11.147376+0000","last_peered":"2024-02-23T00:46:11.147376+0000","last_clean":"2024-02-23T00:46:11.147376+0000","last_became_active":"2024-02-23T00:39:14.780379+0000","last_became_peered":"2024-02-23T00:39:14.780379+0000","last_unstale":"2024-02-23T00:46:11.147376+0000","last_undegraded":"2024-02-23T00:46:11.147376+0000","last_fullsized":"2024-02-23T00:46:11.147376+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.693273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.575242+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.693273+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"67'8","reported_seq":349,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.154267+0000","last_change":"2024-02-23T00:44:25.470245+0000","last_active":"2024-02-23T00:46:11.154267+0000","last_peered":"2024-02-23T00:46:11.154267+0000","last_clean":"2024-02-23T00:46:11.154267+0000","last_became_active":"2024-02-23T00:39:18.799544+0000","last_became_peered":"2024-02-23T00:39:18.799544+0000","last_unstale":"2024-02-23T00:46:11.154267+0000","last_undegraded":"2024-02-23T00:46:11.154267+0000","last_fullsized":"2024-02-23T00:46:11.154267+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:44:25.470174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:25.470174+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":272,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.148911+0000","last_change":"2024-02-23T00:43:36.205958+0000","last_active":"2024-02-23T00:46:11.148911+0000","last_peered":"2024-02-23T00:46:11.148911+0000","last_clean":"2024-02-23T00:46:11.148911+0000","last_became_active":"2024-02-23T00:43:13.338697+0000","last_became_peered":"2024-02-23T00:43:13.338697+0000","last_unstale":"2024-02-23T00:46:11.148911+0000","last_undegraded":"2024-02-23T00:46:11.148911+0000","last_fullsized":"2024-02-23T00:46:11.148911+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:36.205886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:15.081153+0000","last_clean_scrub_stamp":"2024-02-23T00:43:36.205886+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":200,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.150543+0000","last_change":"2024-02-23T00:42:57.686234+0000","last_active":"2024-02-23T00:46:11.150543+0000","last_peered":"2024-02-23T00:46:11.150543+0000","last_clean":"2024-02-23T00:46:11.150543+0000","last_became_active":"2024-02-23T00:42:57.685579+0000","last_became_peered":"2024-02-23T00:42:57.685579+0000","last_unstale":"2024-02-23T00:46:11.150543+0000","last_undegraded":"2024-02-23T00:46:11.150543+0000","last_fullsized":"2024-02-23T00:46:11.150543+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":205,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.150331+0000","last_change":"2024-02-23T00:45:22.874761+0000","last_active":"2024-02-23T00:46:11.150331+0000","last_peered":"2024-02-23T00:46:11.150331+0000","last_clean":"2024-02-23T00:46:11.150331+0000","last_became_active":"2024-02-23T00:42:58.765106+0000","last_became_peered":"2024-02-23T00:42:58.765106+0000","last_unstale":"2024-02-23T00:46:11.150331+0000","last_undegraded":"2024-02-23T00:46:11.150331+0000","last_fullsized":"2024-02-23T00:46:11.150331+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:22.874696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:22.874696+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"67'15","reported_seq":286,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.150628+0000","last_change":"2024-02-23T00:44:19.830383+0000","last_active":"2024-02-23T00:46:11.150628+0000","last_peered":"2024-02-23T00:46:11.150628+0000","last_clean":"2024-02-23T00:46:11.150628+0000","last_became_active":"2024-02-23T00:39:16.789757+0000","last_became_peered":"2024-02-23T00:39:16.789757+0000","last_unstale":"2024-02-23T00:46:11.150628+0000","last_undegraded":"2024-02-23T00:46:11.150628+0000","last_fullsized":"2024-02-23T00:46:11.150628+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:44:19.830309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:44:19.830309+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":263,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.149412+0000","last_change":"2024-02-23T00:44:29.773000+0000","last_active":"2024-02-23T00:46:11.149412+0000","last_peered":"2024-02-23T00:46:11.149412+0000","last_clean":"2024-02-23T00:46:11.149412+0000","last_became_active":"2024-02-23T00:39:14.780526+0000","last_became_peered":"2024-02-23T00:39:14.780526+0000","last_unstale":"2024-02-23T00:46:11.149412+0000","last_undegraded":"2024-02-23T00:46:11.149412+0000","last_fullsized":"2024-02-23T00:46:11.149412+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:29.772933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:29.772933+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":350,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.149086+0000","last_change":"2024-02-23T00:44:23.109508+0000","last_active":"2024-02-23T00:46:11.149086+0000","last_peered":"2024-02-23T00:46:11.149086+0000","last_clean":"2024-02-23T00:46:11.149086+0000","last_became_active":"2024-02-23T00:39:18.800440+0000","last_became_peered":"2024-02-23T00:39:18.800440+0000","last_unstale":"2024-02-23T00:46:11.149086+0000","last_undegraded":"2024-02-23T00:46:11.149086+0000","last_fullsized":"2024-02-23T00:46:11.149086+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:44:23.109439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.109439+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":276,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.158360+0000","last_change":"2024-02-23T00:44:22.976713+0000","last_active":"2024-02-23T00:46:11.158360+0000","last_peered":"2024-02-23T00:46:11.158360+0000","last_clean":"2024-02-23T00:46:11.158360+0000","last_became_active":"2024-02-23T00:43:16.650527+0000","last_became_peered":"2024-02-23T00:43:16.650527+0000","last_unstale":"2024-02-23T00:46:11.158360+0000","last_undegraded":"2024-02-23T00:46:11.158360+0000","last_fullsized":"2024-02-23T00:46:11.158360+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:22.976611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:10.666299+0000","last_clean_scrub_stamp":"2024-02-23T00:44:22.976611+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":205,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.149646+0000","last_change":"2024-02-23T00:45:26.719081+0000","last_active":"2024-02-23T00:46:11.149646+0000","last_peered":"2024-02-23T00:46:11.149646+0000","last_clean":"2024-02-23T00:46:11.149646+0000","last_became_active":"2024-02-23T00:42:57.671472+0000","last_became_peered":"2024-02-23T00:42:57.671472+0000","last_unstale":"2024-02-23T00:46:11.149646+0000","last_undegraded":"2024-02-23T00:46:11.149646+0000","last_fullsized":"2024-02-23T00:46:11.149646+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:26.719011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:26.719011+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,3],"acting":[1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":205,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.146801+0000","last_change":"2024-02-23T00:45:54.311565+0000","last_active":"2024-02-23T00:46:11.146801+0000","last_peered":"2024-02-23T00:46:11.146801+0000","last_clean":"2024-02-23T00:46:11.146801+0000","last_became_active":"2024-02-23T00:42:58.833924+0000","last_became_peered":"2024-02-23T00:42:58.833924+0000","last_unstale":"2024-02-23T00:46:11.146801+0000","last_undegraded":"2024-02-23T00:46:11.146801+0000","last_fullsized":"2024-02-23T00:46:11.146801+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:54.311442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:54.311442+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"68'17","reported_seq":313,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.148800+0000","last_change":"2024-02-23T00:43:33.151471+0000","last_active":"2024-02-23T00:46:11.148800+0000","last_peered":"2024-02-23T00:46:11.148800+0000","last_clean":"2024-02-23T00:46:11.148800+0000","last_became_active":"2024-02-23T00:40:42.933759+0000","last_became_peered":"2024-02-23T00:40:42.933759+0000","last_unstale":"2024-02-23T00:46:11.148800+0000","last_undegraded":"2024-02-23T00:46:11.148800+0000","last_fullsized":"2024-02-23T00:46:11.148800+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:43:33.151401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:33.151401+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":261,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.159522+0000","last_change":"2024-02-23T00:44:18.886545+0000","last_active":"2024-02-23T00:46:11.159522+0000","last_peered":"2024-02-23T00:46:11.159522+0000","last_clean":"2024-02-23T00:46:11.159522+0000","last_became_active":"2024-02-23T00:39:14.784121+0000","last_became_peered":"2024-02-23T00:39:14.784121+0000","last_unstale":"2024-02-23T00:46:11.159522+0000","last_undegraded":"2024-02-23T00:46:11.159522+0000","last_fullsized":"2024-02-23T00:46:11.159522+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:18.886477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:18.886477+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":260,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.157299+0000","last_change":"2024-02-23T00:45:08.118391+0000","last_active":"2024-02-23T00:46:11.157299+0000","last_peered":"2024-02-23T00:46:11.157299+0000","last_clean":"2024-02-23T00:46:11.157299+0000","last_became_active":"2024-02-23T00:39:18.800777+0000","last_became_peered":"2024-02-23T00:39:18.800777+0000","last_unstale":"2024-02-23T00:46:11.157299+0000","last_undegraded":"2024-02-23T00:46:11.157299+0000","last_fullsized":"2024-02-23T00:46:11.157299+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.118302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.118302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"67'1","reported_seq":276,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.151406+0000","last_change":"2024-02-23T00:43:56.091057+0000","last_active":"2024-02-23T00:46:11.151406+0000","last_peered":"2024-02-23T00:46:11.151406+0000","last_clean":"2024-02-23T00:46:11.151406+0000","last_became_active":"2024-02-23T00:43:22.866277+0000","last_became_peered":"2024-02-23T00:43:22.866277+0000","last_unstale":"2024-02-23T00:46:11.151406+0000","last_undegraded":"2024-02-23T00:46:11.151406+0000","last_fullsized":"2024-02-23T00:46:11.151406+0000","mapping_epoch":65,"log_start":"67'1","ondisk_log_start":"67'1","created":65,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:43:56.090982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:09.079918+0000","last_clean_scrub_stamp":"2024-02-23T00:43:56.090982+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":204,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.166232+0000","last_change":"2024-02-23T00:45:06.651852+0000","last_active":"2024-02-23T00:46:11.166232+0000","last_peered":"2024-02-23T00:46:11.166232+0000","last_clean":"2024-02-23T00:46:11.166232+0000","last_became_active":"2024-02-23T00:42:58.718615+0000","last_became_peered":"2024-02-23T00:42:58.718615+0000","last_unstale":"2024-02-23T00:46:11.166232+0000","last_undegraded":"2024-02-23T00:46:11.166232+0000","last_fullsized":"2024-02-23T00:46:11.166232+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.651711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.651711+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":200,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.150567+0000","last_change":"2024-02-23T00:42:57.655252+0000","last_active":"2024-02-23T00:46:11.150567+0000","last_peered":"2024-02-23T00:46:11.150567+0000","last_clean":"2024-02-23T00:46:11.150567+0000","last_became_active":"2024-02-23T00:42:57.653279+0000","last_became_peered":"2024-02-23T00:42:57.653279+0000","last_unstale":"2024-02-23T00:46:11.150567+0000","last_undegraded":"2024-02-23T00:46:11.150567+0000","last_fullsized":"2024-02-23T00:46:11.150567+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":351,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.145860+0000","last_change":"2024-02-23T00:45:51.871688+0000","last_active":"2024-02-23T00:46:11.145860+0000","last_peered":"2024-02-23T00:46:11.145860+0000","last_clean":"2024-02-23T00:46:11.145860+0000","last_became_active":"2024-02-23T00:39:18.798536+0000","last_became_peered":"2024-02-23T00:39:18.798536+0000","last_unstale":"2024-02-23T00:46:11.145860+0000","last_undegraded":"2024-02-23T00:46:11.145860+0000","last_fullsized":"2024-02-23T00:46:11.145860+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:45:51.871618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:51.871618+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"68'20","reported_seq":297,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.147109+0000","last_change":"2024-02-23T00:45:05.291909+0000","last_active":"2024-02-23T00:46:11.147109+0000","last_peered":"2024-02-23T00:46:11.147109+0000","last_clean":"2024-02-23T00:46:11.147109+0000","last_became_active":"2024-02-23T00:39:16.780130+0000","last_became_peered":"2024-02-23T00:39:16.780130+0000","last_unstale":"2024-02-23T00:46:11.147109+0000","last_undegraded":"2024-02-23T00:46:11.147109+0000","last_fullsized":"2024-02-23T00:46:11.147109+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'20","last_scrub_stamp":"2024-02-23T00:45:05.291836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:05.291836+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":264,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.164160+0000","last_change":"2024-02-23T00:43:51.391999+0000","last_active":"2024-02-23T00:46:11.164160+0000","last_peered":"2024-02-23T00:46:11.164160+0000","last_clean":"2024-02-23T00:46:11.164160+0000","last_became_active":"2024-02-23T00:39:14.781999+0000","last_became_peered":"2024-02-23T00:39:14.781999+0000","last_unstale":"2024-02-23T00:46:11.164160+0000","last_undegraded":"2024-02-23T00:46:11.164160+0000","last_fullsized":"2024-02-23T00:46:11.164160+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:51.391939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:51.391939+0000","last_clean_scrub_stamp":"2024-02-23T00:43:51.391939+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":255,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.165713+0000","last_change":"2024-02-23T00:44:24.759694+0000","last_active":"2024-02-23T00:46:11.165713+0000","last_peered":"2024-02-23T00:46:11.165713+0000","last_clean":"2024-02-23T00:46:11.165713+0000","last_became_active":"2024-02-23T00:39:21.161928+0000","last_became_peered":"2024-02-23T00:39:21.161928+0000","last_unstale":"2024-02-23T00:46:11.165713+0000","last_undegraded":"2024-02-23T00:46:11.165713+0000","last_fullsized":"2024-02-23T00:46:11.165713+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:24.759627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.945642+0000","last_clean_scrub_stamp":"2024-02-23T00:44:24.759627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"110'10","reported_seq":232,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.159484+0000","last_change":"2024-02-23T00:46:11.156483+0000","last_active":"2024-02-23T00:46:11.159484+0000","last_peered":"2024-02-23T00:46:11.159484+0000","last_clean":"2024-02-23T00:46:11.159484+0000","last_became_active":"2024-02-23T00:46:11.156132+0000","last_became_peered":"2024-02-23T00:46:11.156132+0000","last_unstale":"2024-02-23T00:46:11.159484+0000","last_undegraded":"2024-02-23T00:46:11.159484+0000","last_fullsized":"2024-02-23T00:46:11.159484+0000","mapping_epoch":108,"log_start":"110'10","ondisk_log_start":"110'10","created":108,"last_epoch_clean":299,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'10","last_scrub_stamp":"2024-02-23T00:45:24.377329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:24.377329+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":384,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":205,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.157552+0000","last_change":"2024-02-23T00:45:59.944702+0000","last_active":"2024-02-23T00:46:11.157552+0000","last_peered":"2024-02-23T00:46:11.157552+0000","last_clean":"2024-02-23T00:46:11.157552+0000","last_became_active":"2024-02-23T00:42:58.717371+0000","last_became_peered":"2024-02-23T00:42:58.717371+0000","last_unstale":"2024-02-23T00:46:11.157552+0000","last_undegraded":"2024-02-23T00:46:11.157552+0000","last_fullsized":"2024-02-23T00:46:11.157552+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:59.944587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:59.944587+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":259,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.147694+0000","last_change":"2024-02-23T00:44:34.059194+0000","last_active":"2024-02-23T00:46:11.147694+0000","last_peered":"2024-02-23T00:46:11.147694+0000","last_clean":"2024-02-23T00:46:11.147694+0000","last_became_active":"2024-02-23T00:39:18.798829+0000","last_became_peered":"2024-02-23T00:39:18.798829+0000","last_unstale":"2024-02-23T00:46:11.147694+0000","last_undegraded":"2024-02-23T00:46:11.147694+0000","last_fullsized":"2024-02-23T00:46:11.147694+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:34.059129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:34.059129+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"251'40","reported_seq":334,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.145769+0000","last_change":"2024-02-23T00:45:23.910609+0000","last_active":"2024-02-23T00:46:11.145769+0000","last_peered":"2024-02-23T00:46:11.145769+0000","last_clean":"2024-02-23T00:46:11.145769+0000","last_became_active":"2024-02-23T00:39:16.859002+0000","last_became_peered":"2024-02-23T00:39:16.859002+0000","last_unstale":"2024-02-23T00:46:11.145769+0000","last_undegraded":"2024-02-23T00:46:11.145769+0000","last_fullsized":"2024-02-23T00:46:11.145769+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'40","last_scrub_stamp":"2024-02-23T00:45:23.910546+0000","last_deep_scrub":"251'40","last_deep_scrub_stamp":"2024-02-23T00:45:23.910546+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.910546+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":263,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.146799+0000","last_change":"2024-02-23T00:45:16.664785+0000","last_active":"2024-02-23T00:46:11.146799+0000","last_peered":"2024-02-23T00:46:11.146799+0000","last_clean":"2024-02-23T00:46:11.146799+0000","last_became_active":"2024-02-23T00:39:14.779193+0000","last_became_peered":"2024-02-23T00:39:14.779193+0000","last_unstale":"2024-02-23T00:46:11.146799+0000","last_undegraded":"2024-02-23T00:46:11.146799+0000","last_fullsized":"2024-02-23T00:46:11.146799+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.664682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.664682+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":257,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.148105+0000","last_change":"2024-02-23T00:45:03.228087+0000","last_active":"2024-02-23T00:46:11.148105+0000","last_peered":"2024-02-23T00:46:11.148105+0000","last_clean":"2024-02-23T00:46:11.148105+0000","last_became_active":"2024-02-23T00:39:20.795271+0000","last_became_peered":"2024-02-23T00:39:20.795271+0000","last_unstale":"2024-02-23T00:46:11.148105+0000","last_undegraded":"2024-02-23T00:46:11.148105+0000","last_fullsized":"2024-02-23T00:46:11.148105+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:03.227995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:45:03.227995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"110'10","reported_seq":72,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.147406+0000","last_change":"2024-02-23T00:46:07.156105+0000","last_active":"2024-02-23T00:46:11.147406+0000","last_peered":"2024-02-23T00:46:11.147406+0000","last_clean":"2024-02-23T00:46:11.147406+0000","last_became_active":"2024-02-23T00:46:07.155723+0000","last_became_peered":"2024-02-23T00:46:07.155723+0000","last_unstale":"2024-02-23T00:46:11.147406+0000","last_undegraded":"2024-02-23T00:46:11.147406+0000","last_fullsized":"2024-02-23T00:46:11.147406+0000","mapping_epoch":272,"log_start":"110'10","ondisk_log_start":"110'10","created":108,"last_epoch_clean":295,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'2","last_scrub_stamp":"2024-02-23T00:45:10.465108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.465108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":205,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.155756+0000","last_change":"2024-02-23T00:45:13.874437+0000","last_active":"2024-02-23T00:46:11.155756+0000","last_peered":"2024-02-23T00:46:11.155756+0000","last_clean":"2024-02-23T00:46:11.155756+0000","last_became_active":"2024-02-23T00:42:58.809283+0000","last_became_peered":"2024-02-23T00:42:58.809283+0000","last_unstale":"2024-02-23T00:46:11.155756+0000","last_undegraded":"2024-02-23T00:46:11.155756+0000","last_fullsized":"2024-02-23T00:46:11.155756+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:13.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:13.874373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":258,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.168788+0000","last_change":"2024-02-23T00:44:19.312509+0000","last_active":"2024-02-23T00:46:11.168788+0000","last_peered":"2024-02-23T00:46:11.168788+0000","last_clean":"2024-02-23T00:46:11.168788+0000","last_became_active":"2024-02-23T00:39:18.796664+0000","last_became_peered":"2024-02-23T00:39:18.796664+0000","last_unstale":"2024-02-23T00:46:11.168788+0000","last_undegraded":"2024-02-23T00:46:11.168788+0000","last_fullsized":"2024-02-23T00:46:11.168788+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:19.312418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:19.312418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"203'34","reported_seq":322,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.161007+0000","last_change":"2024-02-23T00:44:35.644033+0000","last_active":"2024-02-23T00:46:11.161007+0000","last_peered":"2024-02-23T00:46:11.161007+0000","last_clean":"2024-02-23T00:46:11.161007+0000","last_became_active":"2024-02-23T00:39:16.857278+0000","last_became_peered":"2024-02-23T00:39:16.857278+0000","last_unstale":"2024-02-23T00:46:11.161007+0000","last_undegraded":"2024-02-23T00:46:11.161007+0000","last_fullsized":"2024-02-23T00:46:11.161007+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'34","last_scrub_stamp":"2024-02-23T00:44:35.643931+0000","last_deep_scrub":"203'34","last_deep_scrub_stamp":"2024-02-23T00:44:35.643931+0000","last_clean_scrub_stamp":"2024-02-23T00:44:35.643931+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":261,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.165594+0000","last_change":"2024-02-23T00:45:12.730663+0000","last_active":"2024-02-23T00:46:11.165594+0000","last_peered":"2024-02-23T00:46:11.165594+0000","last_clean":"2024-02-23T00:46:11.165594+0000","last_became_active":"2024-02-23T00:39:14.783662+0000","last_became_peered":"2024-02-23T00:39:14.783662+0000","last_unstale":"2024-02-23T00:46:11.165594+0000","last_undegraded":"2024-02-23T00:46:11.165594+0000","last_fullsized":"2024-02-23T00:46:11.165594+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.730539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.730539+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":257,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.149496+0000","last_change":"2024-02-23T00:45:04.585266+0000","last_active":"2024-02-23T00:46:11.149496+0000","last_peered":"2024-02-23T00:46:11.149496+0000","last_clean":"2024-02-23T00:46:11.149496+0000","last_became_active":"2024-02-23T00:39:21.160060+0000","last_became_peered":"2024-02-23T00:39:21.160060+0000","last_unstale":"2024-02-23T00:46:11.149496+0000","last_undegraded":"2024-02-23T00:46:11.149496+0000","last_fullsized":"2024-02-23T00:46:11.149496+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:04.585179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.585179+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.8","version":"110'2","reported_seq":206,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.154661+0000","last_change":"2024-02-23T00:44:33.389680+0000","last_active":"2024-02-23T00:46:11.154661+0000","last_peered":"2024-02-23T00:46:11.154661+0000","last_clean":"2024-02-23T00:46:11.154661+0000","last_became_active":"2024-02-23T00:42:58.809584+0000","last_became_peered":"2024-02-23T00:42:58.809584+0000","last_unstale":"2024-02-23T00:46:11.154661+0000","last_undegraded":"2024-02-23T00:46:11.154661+0000","last_fullsized":"2024-02-23T00:46:11.154661+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'2","last_scrub_stamp":"2024-02-23T00:44:33.389605+0000","last_deep_scrub":"110'2","last_deep_scrub_stamp":"2024-02-23T00:44:33.389605+0000","last_clean_scrub_stamp":"2024-02-23T00:44:33.389605+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":205,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.157889+0000","last_change":"2024-02-23T00:45:18.185097+0000","last_active":"2024-02-23T00:46:11.157889+0000","last_peered":"2024-02-23T00:46:11.157889+0000","last_clean":"2024-02-23T00:46:11.157889+0000","last_became_active":"2024-02-23T00:42:58.726652+0000","last_became_peered":"2024-02-23T00:42:58.726652+0000","last_unstale":"2024-02-23T00:46:11.157889+0000","last_undegraded":"2024-02-23T00:46:11.157889+0000","last_fullsized":"2024-02-23T00:46:11.157889+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:18.184992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:18.184992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":260,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.157394+0000","last_change":"2024-02-23T00:43:45.054194+0000","last_active":"2024-02-23T00:46:11.157394+0000","last_peered":"2024-02-23T00:46:11.157394+0000","last_clean":"2024-02-23T00:46:11.157394+0000","last_became_active":"2024-02-23T00:39:18.800378+0000","last_became_peered":"2024-02-23T00:39:18.800378+0000","last_unstale":"2024-02-23T00:46:11.157394+0000","last_undegraded":"2024-02-23T00:46:11.157394+0000","last_fullsized":"2024-02-23T00:46:11.157394+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:45.054115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:59.694033+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.054115+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"95'18","reported_seq":286,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.162426+0000","last_change":"2024-02-23T00:44:18.630435+0000","last_active":"2024-02-23T00:46:11.162426+0000","last_peered":"2024-02-23T00:46:11.162426+0000","last_clean":"2024-02-23T00:46:11.162426+0000","last_became_active":"2024-02-23T00:39:16.787143+0000","last_became_peered":"2024-02-23T00:39:16.787143+0000","last_unstale":"2024-02-23T00:46:11.162426+0000","last_undegraded":"2024-02-23T00:46:11.162426+0000","last_fullsized":"2024-02-23T00:46:11.162426+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'18","last_scrub_stamp":"2024-02-23T00:44:18.630363+0000","last_deep_scrub":"95'18","last_deep_scrub_stamp":"2024-02-23T00:42:01.195545+0000","last_clean_scrub_stamp":"2024-02-23T00:44:18.630363+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":266,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.159358+0000","last_change":"2024-02-23T00:45:52.795624+0000","last_active":"2024-02-23T00:46:11.159358+0000","last_peered":"2024-02-23T00:46:11.159358+0000","last_clean":"2024-02-23T00:46:11.159358+0000","last_became_active":"2024-02-23T00:39:14.784578+0000","last_became_peered":"2024-02-23T00:39:14.784578+0000","last_unstale":"2024-02-23T00:46:11.159358+0000","last_undegraded":"2024-02-23T00:46:11.159358+0000","last_fullsized":"2024-02-23T00:46:11.159358+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:52.795527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:52.795527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":264,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.158872+0000","last_change":"2024-02-23T00:43:48.258338+0000","last_active":"2024-02-23T00:46:11.158872+0000","last_peered":"2024-02-23T00:46:11.158872+0000","last_clean":"2024-02-23T00:46:11.158872+0000","last_became_active":"2024-02-23T00:39:14.775446+0000","last_became_peered":"2024-02-23T00:39:14.775446+0000","last_unstale":"2024-02-23T00:46:11.158872+0000","last_undegraded":"2024-02-23T00:46:11.158872+0000","last_fullsized":"2024-02-23T00:46:11.158872+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:48.258269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.258269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":252,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.150479+0000","last_change":"2024-02-23T00:43:47.905617+0000","last_active":"2024-02-23T00:46:11.150479+0000","last_peered":"2024-02-23T00:46:11.150479+0000","last_clean":"2024-02-23T00:46:11.150479+0000","last_became_active":"2024-02-23T00:39:20.797242+0000","last_became_peered":"2024-02-23T00:39:20.797242+0000","last_unstale":"2024-02-23T00:46:11.150479+0000","last_undegraded":"2024-02-23T00:46:11.150479+0000","last_fullsized":"2024-02-23T00:46:11.150479+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:47.905501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:47.905501+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"110'7","reported_seq":60,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.165946+0000","last_change":"2024-02-23T00:45:59.164904+0000","last_active":"2024-02-23T00:46:11.165946+0000","last_peered":"2024-02-23T00:46:11.165946+0000","last_clean":"2024-02-23T00:46:11.165946+0000","last_became_active":"2024-02-23T00:45:59.164483+0000","last_became_peered":"2024-02-23T00:45:59.164483+0000","last_unstale":"2024-02-23T00:46:11.165946+0000","last_undegraded":"2024-02-23T00:46:11.165946+0000","last_fullsized":"2024-02-23T00:46:11.165946+0000","mapping_epoch":272,"log_start":"110'7","ondisk_log_start":"110'7","created":108,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'5","last_scrub_stamp":"2024-02-23T00:45:26.760862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:26.760862+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":204,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.163225+0000","last_change":"2024-02-23T00:44:35.360128+0000","last_active":"2024-02-23T00:46:11.163225+0000","last_peered":"2024-02-23T00:46:11.163225+0000","last_clean":"2024-02-23T00:46:11.163225+0000","last_became_active":"2024-02-23T00:42:58.690858+0000","last_became_peered":"2024-02-23T00:42:58.690858+0000","last_unstale":"2024-02-23T00:46:11.163225+0000","last_undegraded":"2024-02-23T00:46:11.163225+0000","last_fullsized":"2024-02-23T00:46:11.163225+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:35.360054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:35.360054+0000","last_clean_scrub_stamp":"2024-02-23T00:44:35.360054+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":257,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.165117+0000","last_change":"2024-02-23T00:45:16.746109+0000","last_active":"2024-02-23T00:46:11.165117+0000","last_peered":"2024-02-23T00:46:11.165117+0000","last_clean":"2024-02-23T00:46:11.165117+0000","last_became_active":"2024-02-23T00:39:18.793801+0000","last_became_peered":"2024-02-23T00:39:18.793801+0000","last_unstale":"2024-02-23T00:46:11.165117+0000","last_undegraded":"2024-02-23T00:46:11.165117+0000","last_fullsized":"2024-02-23T00:46:11.165117+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.746013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.746013+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"67'14","reported_seq":287,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.980194+0000","last_change":"2024-02-23T00:46:11.980163+0000","last_active":"2024-02-23T00:46:11.980194+0000","last_peered":"2024-02-23T00:46:11.980194+0000","last_clean":"2024-02-23T00:46:11.980194+0000","last_became_active":"2024-02-23T00:39:16.858880+0000","last_became_peered":"2024-02-23T00:39:16.858880+0000","last_unstale":"2024-02-23T00:46:11.980194+0000","last_undegraded":"2024-02-23T00:46:11.980194+0000","last_fullsized":"2024-02-23T00:46:11.980194+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'14","last_scrub_stamp":"2024-02-23T00:46:11.980087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:11.980087+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":271,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.159443+0000","last_change":"2024-02-23T00:45:08.796153+0000","last_active":"2024-02-23T00:46:11.159443+0000","last_peered":"2024-02-23T00:46:11.159443+0000","last_clean":"2024-02-23T00:46:11.159443+0000","last_became_active":"2024-02-23T00:39:08.248249+0000","last_became_peered":"2024-02-23T00:39:08.248249+0000","last_unstale":"2024-02-23T00:46:11.159443+0000","last_undegraded":"2024-02-23T00:46:11.159443+0000","last_fullsized":"2024-02-23T00:46:11.159443+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.796076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:49.977248+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.796076+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":264,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.160026+0000","last_change":"2024-02-23T00:45:22.272078+0000","last_active":"2024-02-23T00:46:11.160026+0000","last_peered":"2024-02-23T00:46:11.160026+0000","last_clean":"2024-02-23T00:46:11.160026+0000","last_became_active":"2024-02-23T00:39:14.788888+0000","last_became_peered":"2024-02-23T00:39:14.788888+0000","last_unstale":"2024-02-23T00:46:11.160026+0000","last_undegraded":"2024-02-23T00:46:11.160026+0000","last_fullsized":"2024-02-23T00:46:11.160026+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:22.272003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:22.272003+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":264,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.150199+0000","last_change":"2024-02-23T00:44:33.143663+0000","last_active":"2024-02-23T00:46:11.150199+0000","last_peered":"2024-02-23T00:46:11.150199+0000","last_clean":"2024-02-23T00:46:11.150199+0000","last_became_active":"2024-02-23T00:39:14.783489+0000","last_became_peered":"2024-02-23T00:39:14.783489+0000","last_unstale":"2024-02-23T00:46:11.150199+0000","last_undegraded":"2024-02-23T00:46:11.150199+0000","last_fullsized":"2024-02-23T00:46:11.150199+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:33.143592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:33.143592+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":276,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.157468+0000","last_change":"2024-02-23T00:44:20.958888+0000","last_active":"2024-02-23T00:46:11.157468+0000","last_peered":"2024-02-23T00:46:11.157468+0000","last_clean":"2024-02-23T00:46:11.157468+0000","last_became_active":"2024-02-23T00:43:42.168435+0000","last_became_peered":"2024-02-23T00:43:42.168435+0000","last_unstale":"2024-02-23T00:46:11.157468+0000","last_undegraded":"2024-02-23T00:46:11.157468+0000","last_fullsized":"2024-02-23T00:46:11.157468+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:20.958797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:20.958797+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":55,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.156811+0000","last_change":"2024-02-23T00:45:55.156529+0000","last_active":"2024-02-23T00:46:11.156811+0000","last_peered":"2024-02-23T00:46:11.156811+0000","last_clean":"2024-02-23T00:46:11.156811+0000","last_became_active":"2024-02-23T00:45:55.156026+0000","last_became_peered":"2024-02-23T00:45:55.156026+0000","last_unstale":"2024-02-23T00:46:11.156811+0000","last_undegraded":"2024-02-23T00:46:11.156811+0000","last_fullsized":"2024-02-23T00:46:11.156811+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:21.383626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:21.383626+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":204,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.165484+0000","last_change":"2024-02-23T00:45:17.763618+0000","last_active":"2024-02-23T00:46:11.165484+0000","last_peered":"2024-02-23T00:46:11.165484+0000","last_clean":"2024-02-23T00:46:11.165484+0000","last_became_active":"2024-02-23T00:42:58.701502+0000","last_became_peered":"2024-02-23T00:42:58.701502+0000","last_unstale":"2024-02-23T00:46:11.165484+0000","last_undegraded":"2024-02-23T00:46:11.165484+0000","last_fullsized":"2024-02-23T00:46:11.165484+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:17.763482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:17.763482+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":263,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.154103+0000","last_change":"2024-02-23T00:46:09.924457+0000","last_active":"2024-02-23T00:46:11.154103+0000","last_peered":"2024-02-23T00:46:11.154103+0000","last_clean":"2024-02-23T00:46:11.154103+0000","last_became_active":"2024-02-23T00:39:18.789330+0000","last_became_peered":"2024-02-23T00:39:18.789330+0000","last_unstale":"2024-02-23T00:46:11.154103+0000","last_undegraded":"2024-02-23T00:46:11.154103+0000","last_fullsized":"2024-02-23T00:46:11.154103+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:09.924384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.264240+0000","last_clean_scrub_stamp":"2024-02-23T00:46:09.924384+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"68'19","reported_seq":303,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.150770+0000","last_change":"2024-02-23T00:45:07.697475+0000","last_active":"2024-02-23T00:46:11.150770+0000","last_peered":"2024-02-23T00:46:11.150770+0000","last_clean":"2024-02-23T00:46:11.150770+0000","last_became_active":"2024-02-23T00:39:16.787484+0000","last_became_peered":"2024-02-23T00:39:16.787484+0000","last_unstale":"2024-02-23T00:46:11.150770+0000","last_undegraded":"2024-02-23T00:46:11.150770+0000","last_fullsized":"2024-02-23T00:46:11.150770+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'19","last_scrub_stamp":"2024-02-23T00:45:07.697394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:07.697394+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":267,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.153548+0000","last_change":"2024-02-23T00:46:10.878272+0000","last_active":"2024-02-23T00:46:11.153548+0000","last_peered":"2024-02-23T00:46:11.153548+0000","last_clean":"2024-02-23T00:46:11.153548+0000","last_became_active":"2024-02-23T00:39:14.772411+0000","last_became_peered":"2024-02-23T00:39:14.772411+0000","last_unstale":"2024-02-23T00:46:11.153548+0000","last_undegraded":"2024-02-23T00:46:11.153548+0000","last_fullsized":"2024-02-23T00:46:11.153548+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:10.878204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:10.878204+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"68'5","reported_seq":670,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.727496+0000","last_change":"2024-02-23T00:43:48.876428+0000","last_active":"2024-02-23T00:46:11.727496+0000","last_peered":"2024-02-23T00:46:11.727496+0000","last_clean":"2024-02-23T00:46:11.727496+0000","last_became_active":"2024-02-23T00:39:14.779109+0000","last_became_peered":"2024-02-23T00:39:14.779109+0000","last_unstale":"2024-02-23T00:46:11.727496+0000","last_undegraded":"2024-02-23T00:46:11.727496+0000","last_fullsized":"2024-02-23T00:46:11.727496+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'5","last_scrub_stamp":"2024-02-23T00:43:48.876356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.876356+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":409,"num_read_kb":404,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":255,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.165504+0000","last_change":"2024-02-23T00:43:47.794616+0000","last_active":"2024-02-23T00:46:11.165504+0000","last_peered":"2024-02-23T00:46:11.165504+0000","last_clean":"2024-02-23T00:46:11.165504+0000","last_became_active":"2024-02-23T00:39:21.160457+0000","last_became_peered":"2024-02-23T00:39:21.160457+0000","last_unstale":"2024-02-23T00:46:11.165504+0000","last_undegraded":"2024-02-23T00:46:11.165504+0000","last_fullsized":"2024-02-23T00:46:11.165504+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:47.794534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:47.794534+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":55,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.147729+0000","last_change":"2024-02-23T00:46:03.662711+0000","last_active":"2024-02-23T00:46:11.147729+0000","last_peered":"2024-02-23T00:46:11.147729+0000","last_clean":"2024-02-23T00:46:11.147729+0000","last_became_active":"2024-02-23T00:46:03.662395+0000","last_became_peered":"2024-02-23T00:46:03.662395+0000","last_unstale":"2024-02-23T00:46:11.147729+0000","last_undegraded":"2024-02-23T00:46:11.147729+0000","last_fullsized":"2024-02-23T00:46:11.147729+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,3],"acting":[1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":200,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.149176+0000","last_change":"2024-02-23T00:42:57.653669+0000","last_active":"2024-02-23T00:46:11.149176+0000","last_peered":"2024-02-23T00:46:11.149176+0000","last_clean":"2024-02-23T00:46:11.149176+0000","last_became_active":"2024-02-23T00:42:57.653355+0000","last_became_peered":"2024-02-23T00:42:57.653355+0000","last_unstale":"2024-02-23T00:46:11.149176+0000","last_undegraded":"2024-02-23T00:46:11.149176+0000","last_fullsized":"2024-02-23T00:46:11.149176+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":260,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.163290+0000","last_change":"2024-02-23T00:44:20.626116+0000","last_active":"2024-02-23T00:46:11.163290+0000","last_peered":"2024-02-23T00:46:11.163290+0000","last_clean":"2024-02-23T00:46:11.163290+0000","last_became_active":"2024-02-23T00:39:18.802134+0000","last_became_peered":"2024-02-23T00:39:18.802134+0000","last_unstale":"2024-02-23T00:46:11.163290+0000","last_undegraded":"2024-02-23T00:46:11.163290+0000","last_fullsized":"2024-02-23T00:46:11.163290+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:20.626052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:20.626052+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"86'35","reported_seq":329,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.156088+0000","last_change":"2024-02-23T00:45:52.939553+0000","last_active":"2024-02-23T00:46:11.156088+0000","last_peered":"2024-02-23T00:46:11.156088+0000","last_clean":"2024-02-23T00:46:11.156088+0000","last_became_active":"2024-02-23T00:40:42.933322+0000","last_became_peered":"2024-02-23T00:40:42.933322+0000","last_unstale":"2024-02-23T00:46:11.156088+0000","last_undegraded":"2024-02-23T00:46:11.156088+0000","last_fullsized":"2024-02-23T00:46:11.156088+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'35","last_scrub_stamp":"2024-02-23T00:45:52.939452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:52.939452+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":264,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.156627+0000","last_change":"2024-02-23T00:44:23.965228+0000","last_active":"2024-02-23T00:46:11.156627+0000","last_peered":"2024-02-23T00:46:11.156627+0000","last_clean":"2024-02-23T00:46:11.156627+0000","last_became_active":"2024-02-23T00:39:14.784015+0000","last_became_peered":"2024-02-23T00:39:14.784015+0000","last_unstale":"2024-02-23T00:46:11.156627+0000","last_undegraded":"2024-02-23T00:46:11.156627+0000","last_fullsized":"2024-02-23T00:46:11.156627+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:23.965125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.965125+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":271,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.155624+0000","last_change":"2024-02-23T00:43:45.144499+0000","last_active":"2024-02-23T00:46:11.155624+0000","last_peered":"2024-02-23T00:46:11.155624+0000","last_clean":"2024-02-23T00:46:11.155624+0000","last_became_active":"2024-02-23T00:43:45.144043+0000","last_became_peered":"2024-02-23T00:43:45.144043+0000","last_unstale":"2024-02-23T00:46:11.155624+0000","last_undegraded":"2024-02-23T00:46:11.155624+0000","last_fullsized":"2024-02-23T00:46:11.155624+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:40.062493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:40.062493+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.f","version":"254'6","reported_seq":55,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.150405+0000","last_change":"2024-02-23T00:45:51.167662+0000","last_active":"2024-02-23T00:46:11.150405+0000","last_peered":"2024-02-23T00:46:11.150405+0000","last_clean":"2024-02-23T00:46:11.150405+0000","last_became_active":"2024-02-23T00:45:51.167136+0000","last_became_peered":"2024-02-23T00:45:51.167136+0000","last_unstale":"2024-02-23T00:46:11.150405+0000","last_undegraded":"2024-02-23T00:46:11.150405+0000","last_fullsized":"2024-02-23T00:46:11.150405+0000","mapping_epoch":272,"log_start":"254'6","ondisk_log_start":"254'6","created":108,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:31.410670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:31.410670+0000","last_clean_scrub_stamp":"2024-02-23T00:44:31.410670+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":204,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.155281+0000","last_change":"2024-02-23T00:45:05.490595+0000","last_active":"2024-02-23T00:46:11.155281+0000","last_peered":"2024-02-23T00:46:11.155281+0000","last_clean":"2024-02-23T00:46:11.155281+0000","last_became_active":"2024-02-23T00:42:58.766579+0000","last_became_peered":"2024-02-23T00:42:58.766579+0000","last_unstale":"2024-02-23T00:46:11.155281+0000","last_undegraded":"2024-02-23T00:46:11.155281+0000","last_fullsized":"2024-02-23T00:46:11.155281+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:05.490531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:05.490531+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"91'9","reported_seq":352,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.161354+0000","last_change":"2024-02-23T00:44:28.619036+0000","last_active":"2024-02-23T00:46:11.161354+0000","last_peered":"2024-02-23T00:46:11.161354+0000","last_clean":"2024-02-23T00:46:11.161354+0000","last_became_active":"2024-02-23T00:39:18.801297+0000","last_became_peered":"2024-02-23T00:39:18.801297+0000","last_unstale":"2024-02-23T00:46:11.161354+0000","last_undegraded":"2024-02-23T00:46:11.161354+0000","last_fullsized":"2024-02-23T00:46:11.161354+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'9","last_scrub_stamp":"2024-02-23T00:44:28.618923+0000","last_deep_scrub":"91'9","last_deep_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_clean_scrub_stamp":"2024-02-23T00:44:28.618923+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"68'16","reported_seq":291,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.147684+0000","last_change":"2024-02-23T00:45:11.713800+0000","last_active":"2024-02-23T00:46:11.147684+0000","last_peered":"2024-02-23T00:46:11.147684+0000","last_clean":"2024-02-23T00:46:11.147684+0000","last_became_active":"2024-02-23T00:39:16.787468+0000","last_became_peered":"2024-02-23T00:39:16.787468+0000","last_unstale":"2024-02-23T00:46:11.147684+0000","last_undegraded":"2024-02-23T00:46:11.147684+0000","last_fullsized":"2024-02-23T00:46:11.147684+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:45:11.713731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:11.713731+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":263,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.150177+0000","last_change":"2024-02-23T00:45:19.635062+0000","last_active":"2024-02-23T00:46:11.150177+0000","last_peered":"2024-02-23T00:46:11.150177+0000","last_clean":"2024-02-23T00:46:11.150177+0000","last_became_active":"2024-02-23T00:39:14.779952+0000","last_became_peered":"2024-02-23T00:39:14.779952+0000","last_unstale":"2024-02-23T00:46:11.150177+0000","last_undegraded":"2024-02-23T00:46:11.150177+0000","last_fullsized":"2024-02-23T00:46:11.150177+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:19.635002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:19.635002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"107'2","reported_seq":260,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.157426+0000","last_change":"2024-02-23T00:45:08.662754+0000","last_active":"2024-02-23T00:46:11.157426+0000","last_peered":"2024-02-23T00:46:11.157426+0000","last_clean":"2024-02-23T00:46:11.157426+0000","last_became_active":"2024-02-23T00:39:20.800858+0000","last_became_peered":"2024-02-23T00:39:20.800858+0000","last_unstale":"2024-02-23T00:46:11.157426+0000","last_undegraded":"2024-02-23T00:46:11.157426+0000","last_fullsized":"2024-02-23T00:46:11.157426+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'2","last_scrub_stamp":"2024-02-23T00:45:08.662667+0000","last_deep_scrub":"107'2","last_deep_scrub_stamp":"2024-02-23T00:42:48.362739+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.662667+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":205,"reported_epoch":300,"state":"peering","last_fresh":"2024-02-23T00:46:12.159895+0000","last_change":"2024-02-23T00:46:12.160118+0000","last_active":"2024-02-23T00:46:12.159895+0000","last_peered":"2024-02-23T00:46:11.167797+0000","last_clean":"2024-02-23T00:46:11.167797+0000","last_became_active":"2024-02-23T00:42:58.726861+0000","last_became_peered":"2024-02-23T00:42:58.726861+0000","last_unstale":"2024-02-23T00:46:12.159895+0000","last_undegraded":"2024-02-23T00:46:12.159895+0000","last_fullsized":"2024-02-23T00:46:12.159895+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:19.711117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:19.711117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":204,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.169482+0000","last_change":"2024-02-23T00:45:04.621794+0000","last_active":"2024-02-23T00:46:11.169482+0000","last_peered":"2024-02-23T00:46:11.169482+0000","last_clean":"2024-02-23T00:46:11.169482+0000","last_became_active":"2024-02-23T00:42:58.664282+0000","last_became_peered":"2024-02-23T00:42:58.664282+0000","last_unstale":"2024-02-23T00:46:11.169482+0000","last_undegraded":"2024-02-23T00:46:11.169482+0000","last_fullsized":"2024-02-23T00:46:11.169482+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:04.621714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.621714+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":258,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.154824+0000","last_change":"2024-02-23T00:43:51.261426+0000","last_active":"2024-02-23T00:46:11.154824+0000","last_peered":"2024-02-23T00:46:11.154824+0000","last_clean":"2024-02-23T00:46:11.154824+0000","last_became_active":"2024-02-23T00:39:18.796323+0000","last_became_peered":"2024-02-23T00:39:18.796323+0000","last_unstale":"2024-02-23T00:46:11.154824+0000","last_undegraded":"2024-02-23T00:46:11.154824+0000","last_fullsized":"2024-02-23T00:46:11.154824+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:51.261367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_clean_scrub_stamp":"2024-02-23T00:43:51.261367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"68'25","reported_seq":310,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.147592+0000","last_change":"2024-02-23T00:43:46.158231+0000","last_active":"2024-02-23T00:46:11.147592+0000","last_peered":"2024-02-23T00:46:11.147592+0000","last_clean":"2024-02-23T00:46:11.147592+0000","last_became_active":"2024-02-23T00:39:16.783926+0000","last_became_peered":"2024-02-23T00:39:16.783926+0000","last_unstale":"2024-02-23T00:46:11.147592+0000","last_undegraded":"2024-02-23T00:46:11.147592+0000","last_fullsized":"2024-02-23T00:46:11.147592+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'25","last_scrub_stamp":"2024-02-23T00:43:46.158149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:46.158149+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"95'4","reported_seq":278,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.150150+0000","last_change":"2024-02-23T00:44:31.106105+0000","last_active":"2024-02-23T00:46:11.150150+0000","last_peered":"2024-02-23T00:46:11.150150+0000","last_clean":"2024-02-23T00:46:11.150150+0000","last_became_active":"2024-02-23T00:39:14.779205+0000","last_became_peered":"2024-02-23T00:39:14.779205+0000","last_unstale":"2024-02-23T00:46:11.150150+0000","last_undegraded":"2024-02-23T00:46:11.150150+0000","last_fullsized":"2024-02-23T00:46:11.150150+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'4","last_scrub_stamp":"2024-02-23T00:44:31.106012+0000","last_deep_scrub":"95'4","last_deep_scrub_stamp":"2024-02-23T00:41:43.874397+0000","last_clean_scrub_stamp":"2024-02-23T00:44:31.106012+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":276,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.149301+0000","last_change":"2024-02-23T00:43:52.036315+0000","last_active":"2024-02-23T00:46:11.149301+0000","last_peered":"2024-02-23T00:46:11.149301+0000","last_clean":"2024-02-23T00:46:11.149301+0000","last_became_active":"2024-02-23T00:43:27.611660+0000","last_became_peered":"2024-02-23T00:43:27.611660+0000","last_unstale":"2024-02-23T00:46:11.149301+0000","last_undegraded":"2024-02-23T00:46:11.149301+0000","last_fullsized":"2024-02-23T00:46:11.149301+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:52.036249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:52.036249+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":204,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.155327+0000","last_change":"2024-02-23T00:45:16.389060+0000","last_active":"2024-02-23T00:46:11.155327+0000","last_peered":"2024-02-23T00:46:11.155327+0000","last_clean":"2024-02-23T00:46:11.155327+0000","last_became_active":"2024-02-23T00:42:58.809926+0000","last_became_peered":"2024-02-23T00:42:58.809926+0000","last_unstale":"2024-02-23T00:46:11.155327+0000","last_undegraded":"2024-02-23T00:46:11.155327+0000","last_fullsized":"2024-02-23T00:46:11.155327+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.388988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.388988+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":204,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.168925+0000","last_change":"2024-02-23T00:45:20.829124+0000","last_active":"2024-02-23T00:46:11.168925+0000","last_peered":"2024-02-23T00:46:11.168925+0000","last_clean":"2024-02-23T00:46:11.168925+0000","last_became_active":"2024-02-23T00:42:58.744983+0000","last_became_peered":"2024-02-23T00:42:58.744983+0000","last_unstale":"2024-02-23T00:46:11.168925+0000","last_undegraded":"2024-02-23T00:46:11.168925+0000","last_fullsized":"2024-02-23T00:46:11.168925+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:20.829052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:20.829052+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"68'21","reported_seq":303,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.162502+0000","last_change":"2024-02-23T00:43:52.309599+0000","last_active":"2024-02-23T00:46:11.162502+0000","last_peered":"2024-02-23T00:46:11.162502+0000","last_clean":"2024-02-23T00:46:11.162502+0000","last_became_active":"2024-02-23T00:39:16.858760+0000","last_became_peered":"2024-02-23T00:39:16.858760+0000","last_unstale":"2024-02-23T00:46:11.162502+0000","last_undegraded":"2024-02-23T00:46:11.162502+0000","last_fullsized":"2024-02-23T00:46:11.162502+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'21","last_scrub_stamp":"2024-02-23T00:43:52.309532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:52.309532+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":267,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.154750+0000","last_change":"2024-02-23T00:45:22.376128+0000","last_active":"2024-02-23T00:46:11.154750+0000","last_peered":"2024-02-23T00:46:11.154750+0000","last_clean":"2024-02-23T00:46:11.154750+0000","last_became_active":"2024-02-23T00:39:14.774244+0000","last_became_peered":"2024-02-23T00:39:14.774244+0000","last_unstale":"2024-02-23T00:46:11.154750+0000","last_undegraded":"2024-02-23T00:46:11.154750+0000","last_fullsized":"2024-02-23T00:46:11.154750+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:22.376063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:22.376063+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":260,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.149280+0000","last_change":"2024-02-23T00:43:44.951800+0000","last_active":"2024-02-23T00:46:11.149280+0000","last_peered":"2024-02-23T00:46:11.149280+0000","last_clean":"2024-02-23T00:46:11.149280+0000","last_became_active":"2024-02-23T00:39:18.798624+0000","last_became_peered":"2024-02-23T00:39:18.798624+0000","last_unstale":"2024-02-23T00:46:11.149280+0000","last_undegraded":"2024-02-23T00:46:11.149280+0000","last_fullsized":"2024-02-23T00:46:11.149280+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:44.951696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:46.998875+0000","last_clean_scrub_stamp":"2024-02-23T00:43:44.951696+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":256,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.153618+0000","last_change":"2024-02-23T00:44:26.432766+0000","last_active":"2024-02-23T00:46:11.153618+0000","last_peered":"2024-02-23T00:46:11.153618+0000","last_clean":"2024-02-23T00:46:11.153618+0000","last_became_active":"2024-02-23T00:39:20.796041+0000","last_became_peered":"2024-02-23T00:39:20.796041+0000","last_unstale":"2024-02-23T00:46:11.153618+0000","last_undegraded":"2024-02-23T00:46:11.153618+0000","last_fullsized":"2024-02-23T00:46:11.153618+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:26.432703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:26.432703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":205,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.157626+0000","last_change":"2024-02-23T00:44:26.595443+0000","last_active":"2024-02-23T00:46:11.157626+0000","last_peered":"2024-02-23T00:46:11.157626+0000","last_clean":"2024-02-23T00:46:11.157626+0000","last_became_active":"2024-02-23T00:42:58.825015+0000","last_became_peered":"2024-02-23T00:42:58.825015+0000","last_unstale":"2024-02-23T00:46:11.157626+0000","last_undegraded":"2024-02-23T00:46:11.157626+0000","last_fullsized":"2024-02-23T00:46:11.157626+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:26.595346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:44:26.595346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"107'16","reported_seq":290,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.149201+0000","last_change":"2024-02-23T00:45:07.012086+0000","last_active":"2024-02-23T00:46:11.149201+0000","last_peered":"2024-02-23T00:46:11.149201+0000","last_clean":"2024-02-23T00:46:11.149201+0000","last_became_active":"2024-02-23T00:39:16.858449+0000","last_became_peered":"2024-02-23T00:39:16.858449+0000","last_unstale":"2024-02-23T00:46:11.149201+0000","last_undegraded":"2024-02-23T00:46:11.149201+0000","last_fullsized":"2024-02-23T00:46:11.149201+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'16","last_scrub_stamp":"2024-02-23T00:45:07.011956+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-02-23T00:42:52.964510+0000","last_clean_scrub_stamp":"2024-02-23T00:45:07.011956+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":262,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.167411+0000","last_change":"2024-02-23T00:45:13.613736+0000","last_active":"2024-02-23T00:46:11.167411+0000","last_peered":"2024-02-23T00:46:11.167411+0000","last_clean":"2024-02-23T00:46:11.167411+0000","last_became_active":"2024-02-23T00:39:14.783164+0000","last_became_peered":"2024-02-23T00:39:14.783164+0000","last_unstale":"2024-02-23T00:46:11.167411+0000","last_undegraded":"2024-02-23T00:46:11.167411+0000","last_fullsized":"2024-02-23T00:46:11.167411+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:13.613654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:13.613654+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":260,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.157120+0000","last_change":"2024-02-23T00:43:40.971316+0000","last_active":"2024-02-23T00:46:11.157120+0000","last_peered":"2024-02-23T00:46:11.157120+0000","last_clean":"2024-02-23T00:46:11.157120+0000","last_became_active":"2024-02-23T00:39:18.800297+0000","last_became_peered":"2024-02-23T00:39:18.800297+0000","last_unstale":"2024-02-23T00:46:11.157120+0000","last_undegraded":"2024-02-23T00:46:11.157120+0000","last_fullsized":"2024-02-23T00:46:11.157120+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:40.971243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:40.971243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":255,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.165203+0000","last_change":"2024-02-23T00:43:46.760431+0000","last_active":"2024-02-23T00:46:11.165203+0000","last_peered":"2024-02-23T00:46:11.165203+0000","last_clean":"2024-02-23T00:46:11.165203+0000","last_became_active":"2024-02-23T00:39:21.162036+0000","last_became_peered":"2024-02-23T00:39:21.162036+0000","last_unstale":"2024-02-23T00:46:11.165203+0000","last_undegraded":"2024-02-23T00:46:11.165203+0000","last_fullsized":"2024-02-23T00:46:11.165203+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:46.760348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:46.760348+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"110'7","reported_seq":218,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.147420+0000","last_change":"2024-02-23T00:45:18.947859+0000","last_active":"2024-02-23T00:46:11.147420+0000","last_peered":"2024-02-23T00:46:11.147420+0000","last_clean":"2024-02-23T00:46:11.147420+0000","last_became_active":"2024-02-23T00:42:58.777251+0000","last_became_peered":"2024-02-23T00:42:58.777251+0000","last_unstale":"2024-02-23T00:46:11.147420+0000","last_undegraded":"2024-02-23T00:46:11.147420+0000","last_fullsized":"2024-02-23T00:46:11.147420+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'7","last_scrub_stamp":"2024-02-23T00:45:18.947746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:18.947746+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"68'22","reported_seq":230,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.153488+0000","last_change":"2024-02-23T00:44:31.391929+0000","last_active":"2024-02-23T00:46:11.153488+0000","last_peered":"2024-02-23T00:46:11.153488+0000","last_clean":"2024-02-23T00:46:11.153488+0000","last_became_active":"2024-02-23T00:42:43.521189+0000","last_became_peered":"2024-02-23T00:42:43.521189+0000","last_unstale":"2024-02-23T00:46:11.153488+0000","last_undegraded":"2024-02-23T00:46:11.153488+0000","last_fullsized":"2024-02-23T00:46:11.153488+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'22","last_scrub_stamp":"2024-02-23T00:44:31.391819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:44:31.391819+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":37,"num_read_kb":30,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,7],"acting":[2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":264,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.156524+0000","last_change":"2024-02-23T00:44:18.946624+0000","last_active":"2024-02-23T00:46:11.156524+0000","last_peered":"2024-02-23T00:46:11.156524+0000","last_clean":"2024-02-23T00:46:11.156524+0000","last_became_active":"2024-02-23T00:39:14.785590+0000","last_became_peered":"2024-02-23T00:39:14.785590+0000","last_unstale":"2024-02-23T00:46:11.156524+0000","last_undegraded":"2024-02-23T00:46:11.156524+0000","last_fullsized":"2024-02-23T00:46:11.156524+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:18.946560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_clean_scrub_stamp":"2024-02-23T00:44:18.946560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"67'8","reported_seq":351,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.158197+0000","last_change":"2024-02-23T00:44:29.038226+0000","last_active":"2024-02-23T00:46:11.158197+0000","last_peered":"2024-02-23T00:46:11.158197+0000","last_clean":"2024-02-23T00:46:11.158197+0000","last_became_active":"2024-02-23T00:39:18.803566+0000","last_became_peered":"2024-02-23T00:39:18.803566+0000","last_unstale":"2024-02-23T00:46:11.158197+0000","last_undegraded":"2024-02-23T00:46:11.158197+0000","last_fullsized":"2024-02-23T00:46:11.158197+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:44:29.038134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:29.038134+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":204,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.163033+0000","last_change":"2024-02-23T00:45:54.753371+0000","last_active":"2024-02-23T00:46:11.163033+0000","last_peered":"2024-02-23T00:46:11.163033+0000","last_clean":"2024-02-23T00:46:11.163033+0000","last_became_active":"2024-02-23T00:42:58.757424+0000","last_became_peered":"2024-02-23T00:42:58.757424+0000","last_unstale":"2024-02-23T00:46:11.163033+0000","last_undegraded":"2024-02-23T00:46:11.163033+0000","last_fullsized":"2024-02-23T00:46:11.163033+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:54.753267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:54.753267+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"68'15","reported_seq":293,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.158044+0000","last_change":"2024-02-23T00:44:25.998879+0000","last_active":"2024-02-23T00:46:11.158044+0000","last_peered":"2024-02-23T00:46:11.158044+0000","last_clean":"2024-02-23T00:46:11.158044+0000","last_became_active":"2024-02-23T00:39:16.858953+0000","last_became_peered":"2024-02-23T00:39:16.858953+0000","last_unstale":"2024-02-23T00:46:11.158044+0000","last_undegraded":"2024-02-23T00:46:11.158044+0000","last_fullsized":"2024-02-23T00:46:11.158044+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:44:25.998804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:44:25.998804+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":264,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.161133+0000","last_change":"2024-02-23T00:43:45.309194+0000","last_active":"2024-02-23T00:46:11.161133+0000","last_peered":"2024-02-23T00:46:11.161133+0000","last_clean":"2024-02-23T00:46:11.161133+0000","last_became_active":"2024-02-23T00:39:14.773772+0000","last_became_peered":"2024-02-23T00:39:14.773772+0000","last_unstale":"2024-02-23T00:46:11.161133+0000","last_undegraded":"2024-02-23T00:46:11.161133+0000","last_fullsized":"2024-02-23T00:46:11.161133+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:45.309083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.309083+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"67'8","reported_seq":346,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.151721+0000","last_change":"2024-02-23T00:43:39.086199+0000","last_active":"2024-02-23T00:46:11.151721+0000","last_peered":"2024-02-23T00:46:11.151721+0000","last_clean":"2024-02-23T00:46:11.151721+0000","last_became_active":"2024-02-23T00:39:18.787313+0000","last_became_peered":"2024-02-23T00:39:18.787313+0000","last_unstale":"2024-02-23T00:46:11.151721+0000","last_undegraded":"2024-02-23T00:46:11.151721+0000","last_fullsized":"2024-02-23T00:46:11.151721+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:43:39.086107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:39.086107+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":200,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.147026+0000","last_change":"2024-02-23T00:42:57.652853+0000","last_active":"2024-02-23T00:46:11.147026+0000","last_peered":"2024-02-23T00:46:11.147026+0000","last_clean":"2024-02-23T00:46:11.147026+0000","last_became_active":"2024-02-23T00:42:57.652488+0000","last_became_peered":"2024-02-23T00:42:57.652488+0000","last_unstale":"2024-02-23T00:46:11.147026+0000","last_undegraded":"2024-02-23T00:46:11.147026+0000","last_fullsized":"2024-02-23T00:46:11.147026+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"67'9","reported_seq":272,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.147513+0000","last_change":"2024-02-23T00:42:57.705378+0000","last_active":"2024-02-23T00:46:11.147513+0000","last_peered":"2024-02-23T00:46:11.147513+0000","last_clean":"2024-02-23T00:46:11.147513+0000","last_became_active":"2024-02-23T00:39:16.786936+0000","last_became_peered":"2024-02-23T00:39:16.786936+0000","last_unstale":"2024-02-23T00:46:11.147513+0000","last_undegraded":"2024-02-23T00:46:11.147513+0000","last_fullsized":"2024-02-23T00:46:11.147513+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'9","last_scrub_stamp":"2024-02-23T00:42:57.705261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:42:57.705261+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":267,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.168698+0000","last_change":"2024-02-23T00:44:24.350861+0000","last_active":"2024-02-23T00:46:11.168698+0000","last_peered":"2024-02-23T00:46:11.168698+0000","last_clean":"2024-02-23T00:46:11.168698+0000","last_became_active":"2024-02-23T00:39:14.778455+0000","last_became_peered":"2024-02-23T00:39:14.778455+0000","last_unstale":"2024-02-23T00:46:11.168698+0000","last_undegraded":"2024-02-23T00:46:11.168698+0000","last_fullsized":"2024-02-23T00:46:11.168698+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:24.350797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:24.350797+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":260,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.149216+0000","last_change":"2024-02-23T00:44:37.105079+0000","last_active":"2024-02-23T00:46:11.149216+0000","last_peered":"2024-02-23T00:46:11.149216+0000","last_clean":"2024-02-23T00:46:11.149216+0000","last_became_active":"2024-02-23T00:39:18.799052+0000","last_became_peered":"2024-02-23T00:39:18.799052+0000","last_unstale":"2024-02-23T00:46:11.149216+0000","last_undegraded":"2024-02-23T00:46:11.149216+0000","last_fullsized":"2024-02-23T00:46:11.149216+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:37.105013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_clean_scrub_stamp":"2024-02-23T00:44:37.105013+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":258,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.157332+0000","last_change":"2024-02-23T00:46:00.558774+0000","last_active":"2024-02-23T00:46:11.157332+0000","last_peered":"2024-02-23T00:46:11.157332+0000","last_clean":"2024-02-23T00:46:11.157332+0000","last_became_active":"2024-02-23T00:39:20.802648+0000","last_became_peered":"2024-02-23T00:39:20.802648+0000","last_unstale":"2024-02-23T00:46:11.157332+0000","last_undegraded":"2024-02-23T00:46:11.157332+0000","last_fullsized":"2024-02-23T00:46:11.157332+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:00.558661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:46:00.558661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":205,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.157214+0000","last_change":"2024-02-23T00:44:23.543049+0000","last_active":"2024-02-23T00:46:11.157214+0000","last_peered":"2024-02-23T00:46:11.157214+0000","last_clean":"2024-02-23T00:46:11.157214+0000","last_became_active":"2024-02-23T00:42:58.835862+0000","last_became_peered":"2024-02-23T00:42:58.835862+0000","last_unstale":"2024-02-23T00:46:11.157214+0000","last_undegraded":"2024-02-23T00:46:11.157214+0000","last_fullsized":"2024-02-23T00:46:11.157214+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:23.542975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.542975+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"67'11","reported_seq":280,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.146882+0000","last_change":"2024-02-23T00:45:13.315510+0000","last_active":"2024-02-23T00:46:11.146882+0000","last_peered":"2024-02-23T00:46:11.146882+0000","last_clean":"2024-02-23T00:46:11.146882+0000","last_became_active":"2024-02-23T00:39:16.783896+0000","last_became_peered":"2024-02-23T00:39:16.783896+0000","last_unstale":"2024-02-23T00:46:11.146882+0000","last_undegraded":"2024-02-23T00:46:11.146882+0000","last_fullsized":"2024-02-23T00:46:11.146882+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'11","last_scrub_stamp":"2024-02-23T00:45:13.315434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:13.315434+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"202'2","reported_seq":271,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.150189+0000","last_change":"2024-02-23T00:44:34.720705+0000","last_active":"2024-02-23T00:46:11.150189+0000","last_peered":"2024-02-23T00:46:11.150189+0000","last_clean":"2024-02-23T00:46:11.150189+0000","last_became_active":"2024-02-23T00:39:14.775804+0000","last_became_peered":"2024-02-23T00:39:14.775804+0000","last_unstale":"2024-02-23T00:46:11.150189+0000","last_undegraded":"2024-02-23T00:46:11.150189+0000","last_fullsized":"2024-02-23T00:46:11.150189+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"202'2","last_scrub_stamp":"2024-02-23T00:44:34.720640+0000","last_deep_scrub":"202'2","last_deep_scrub_stamp":"2024-02-23T00:44:34.720640+0000","last_clean_scrub_stamp":"2024-02-23T00:44:34.720640+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":254,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.147601+0000","last_change":"2024-02-23T00:43:31.903157+0000","last_active":"2024-02-23T00:46:11.147601+0000","last_peered":"2024-02-23T00:46:11.147601+0000","last_clean":"2024-02-23T00:46:11.147601+0000","last_became_active":"2024-02-23T00:39:18.800382+0000","last_became_peered":"2024-02-23T00:39:18.800382+0000","last_unstale":"2024-02-23T00:46:11.147601+0000","last_undegraded":"2024-02-23T00:46:11.147601+0000","last_fullsized":"2024-02-23T00:46:11.147601+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:31.903031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:31.903031+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":258,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.146979+0000","last_change":"2024-02-23T00:44:23.152677+0000","last_active":"2024-02-23T00:46:11.146979+0000","last_peered":"2024-02-23T00:46:11.146979+0000","last_clean":"2024-02-23T00:46:11.146979+0000","last_became_active":"2024-02-23T00:39:20.799053+0000","last_became_peered":"2024-02-23T00:39:20.799053+0000","last_unstale":"2024-02-23T00:46:11.146979+0000","last_undegraded":"2024-02-23T00:46:11.146979+0000","last_fullsized":"2024-02-23T00:46:11.146979+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:23.152602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.152602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"110'2","reported_seq":42,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.154047+0000","last_change":"2024-02-23T00:45:45.162543+0000","last_active":"2024-02-23T00:46:11.154047+0000","last_peered":"2024-02-23T00:46:11.154047+0000","last_clean":"2024-02-23T00:46:11.154047+0000","last_became_active":"2024-02-23T00:45:45.144393+0000","last_became_peered":"2024-02-23T00:45:45.144393+0000","last_unstale":"2024-02-23T00:46:11.154047+0000","last_undegraded":"2024-02-23T00:46:11.154047+0000","last_fullsized":"2024-02-23T00:46:11.154047+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'2","last_scrub_stamp":"2024-02-23T00:45:09.917420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:09.917420+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":200,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.150124+0000","last_change":"2024-02-23T00:42:57.652742+0000","last_active":"2024-02-23T00:46:11.150124+0000","last_peered":"2024-02-23T00:46:11.150124+0000","last_clean":"2024-02-23T00:46:11.150124+0000","last_became_active":"2024-02-23T00:42:57.652438+0000","last_became_peered":"2024-02-23T00:42:57.652438+0000","last_unstale":"2024-02-23T00:46:11.150124+0000","last_undegraded":"2024-02-23T00:46:11.150124+0000","last_fullsized":"2024-02-23T00:46:11.150124+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"67'4","reported_seq":268,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.145534+0000","last_change":"2024-02-23T00:45:20.967211+0000","last_active":"2024-02-23T00:46:11.145534+0000","last_peered":"2024-02-23T00:46:11.145534+0000","last_clean":"2024-02-23T00:46:11.145534+0000","last_became_active":"2024-02-23T00:39:16.785830+0000","last_became_peered":"2024-02-23T00:39:16.785830+0000","last_unstale":"2024-02-23T00:46:11.145534+0000","last_undegraded":"2024-02-23T00:46:11.145534+0000","last_fullsized":"2024-02-23T00:46:11.145534+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'4","last_scrub_stamp":"2024-02-23T00:45:20.967095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:20.967095+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":267,"reported_epoch":300,"state":"active+clean","last_fresh":"2024-02-23T00:46:12.164261+0000","last_change":"2024-02-23T00:46:12.168122+0000","last_active":"2024-02-23T00:46:12.164261+0000","last_peered":"2024-02-23T00:46:12.164261+0000","last_clean":"2024-02-23T00:46:12.164261+0000","last_became_active":"2024-02-23T00:39:14.783325+0000","last_became_peered":"2024-02-23T00:39:14.783325+0000","last_unstale":"2024-02-23T00:46:12.164261+0000","last_undegraded":"2024-02-23T00:46:12.164261+0000","last_fullsized":"2024-02-23T00:46:12.164261+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:38.947675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:38.947675+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":263,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.167237+0000","last_change":"2024-02-23T00:45:20.761547+0000","last_active":"2024-02-23T00:46:11.167237+0000","last_peered":"2024-02-23T00:46:11.167237+0000","last_clean":"2024-02-23T00:46:11.167237+0000","last_became_active":"2024-02-23T00:39:18.797287+0000","last_became_peered":"2024-02-23T00:39:18.797287+0000","last_unstale":"2024-02-23T00:46:11.167237+0000","last_undegraded":"2024-02-23T00:46:11.167237+0000","last_fullsized":"2024-02-23T00:46:11.167237+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:20.761440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:20.761440+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":255,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.162544+0000","last_change":"2024-02-23T00:45:10.741762+0000","last_active":"2024-02-23T00:46:11.162544+0000","last_peered":"2024-02-23T00:46:11.162544+0000","last_clean":"2024-02-23T00:46:11.162544+0000","last_became_active":"2024-02-23T00:39:20.795351+0000","last_became_peered":"2024-02-23T00:39:20.795351+0000","last_unstale":"2024-02-23T00:46:11.162544+0000","last_undegraded":"2024-02-23T00:46:11.162544+0000","last_fullsized":"2024-02-23T00:46:11.162544+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.741694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.741694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":204,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.165826+0000","last_change":"2024-02-23T00:45:56.706901+0000","last_active":"2024-02-23T00:46:11.165826+0000","last_peered":"2024-02-23T00:46:11.165826+0000","last_clean":"2024-02-23T00:46:11.165826+0000","last_became_active":"2024-02-23T00:42:58.741365+0000","last_became_peered":"2024-02-23T00:42:58.741365+0000","last_unstale":"2024-02-23T00:46:11.165826+0000","last_undegraded":"2024-02-23T00:46:11.165826+0000","last_fullsized":"2024-02-23T00:46:11.165826+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:56.706792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:56.706792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"68'17","reported_seq":296,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.147518+0000","last_change":"2024-02-23T00:43:43.997579+0000","last_active":"2024-02-23T00:46:11.147518+0000","last_peered":"2024-02-23T00:46:11.147518+0000","last_clean":"2024-02-23T00:46:11.147518+0000","last_became_active":"2024-02-23T00:39:16.857232+0000","last_became_peered":"2024-02-23T00:39:16.857232+0000","last_unstale":"2024-02-23T00:46:11.147518+0000","last_undegraded":"2024-02-23T00:46:11.147518+0000","last_fullsized":"2024-02-23T00:46:11.147518+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:43:43.997452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:43.997452+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":262,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.164083+0000","last_change":"2024-02-23T00:43:52.419206+0000","last_active":"2024-02-23T00:46:11.164083+0000","last_peered":"2024-02-23T00:46:11.164083+0000","last_clean":"2024-02-23T00:46:11.164083+0000","last_became_active":"2024-02-23T00:39:14.784092+0000","last_became_peered":"2024-02-23T00:39:14.784092+0000","last_unstale":"2024-02-23T00:46:11.164083+0000","last_undegraded":"2024-02-23T00:46:11.164083+0000","last_fullsized":"2024-02-23T00:46:11.164083+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:52.419100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:52.419100+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":257,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.159773+0000","last_change":"2024-02-23T00:44:25.797288+0000","last_active":"2024-02-23T00:46:11.159773+0000","last_peered":"2024-02-23T00:46:11.159773+0000","last_clean":"2024-02-23T00:46:11.159773+0000","last_became_active":"2024-02-23T00:39:18.791457+0000","last_became_peered":"2024-02-23T00:39:18.791457+0000","last_unstale":"2024-02-23T00:46:11.159773+0000","last_undegraded":"2024-02-23T00:46:11.159773+0000","last_fullsized":"2024-02-23T00:46:11.159773+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:25.797184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:25.797184+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":253,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.149377+0000","last_change":"2024-02-23T00:43:45.965636+0000","last_active":"2024-02-23T00:46:11.149377+0000","last_peered":"2024-02-23T00:46:11.149377+0000","last_clean":"2024-02-23T00:46:11.149377+0000","last_became_active":"2024-02-23T00:39:21.160376+0000","last_became_peered":"2024-02-23T00:39:21.160376+0000","last_unstale":"2024-02-23T00:46:11.149377+0000","last_undegraded":"2024-02-23T00:46:11.149377+0000","last_fullsized":"2024-02-23T00:46:11.149377+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:45.965570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.965570+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":204,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.156332+0000","last_change":"2024-02-23T00:45:23.378334+0000","last_active":"2024-02-23T00:46:11.156332+0000","last_peered":"2024-02-23T00:46:11.156332+0000","last_clean":"2024-02-23T00:46:11.156332+0000","last_became_active":"2024-02-23T00:42:58.702580+0000","last_became_peered":"2024-02-23T00:42:58.702580+0000","last_unstale":"2024-02-23T00:46:11.156332+0000","last_undegraded":"2024-02-23T00:46:11.156332+0000","last_fullsized":"2024-02-23T00:46:11.156332+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:23.378270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.378270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"68'16","reported_seq":292,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.162747+0000","last_change":"2024-02-23T00:45:04.672612+0000","last_active":"2024-02-23T00:46:11.162747+0000","last_peered":"2024-02-23T00:46:11.162747+0000","last_clean":"2024-02-23T00:46:11.162747+0000","last_became_active":"2024-02-23T00:39:16.858945+0000","last_became_peered":"2024-02-23T00:39:16.858945+0000","last_unstale":"2024-02-23T00:46:11.162747+0000","last_undegraded":"2024-02-23T00:46:11.162747+0000","last_fullsized":"2024-02-23T00:46:11.162747+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:45:04.672537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.672537+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":276,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.155514+0000","last_change":"2024-02-23T00:46:09.088334+0000","last_active":"2024-02-23T00:46:11.155514+0000","last_peered":"2024-02-23T00:46:11.155514+0000","last_clean":"2024-02-23T00:46:11.155514+0000","last_became_active":"2024-02-23T00:39:14.785363+0000","last_became_peered":"2024-02-23T00:39:14.785363+0000","last_unstale":"2024-02-23T00:46:11.155514+0000","last_undegraded":"2024-02-23T00:46:11.155514+0000","last_fullsized":"2024-02-23T00:46:11.155514+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"80'2","last_scrub_stamp":"2024-02-23T00:46:09.088249+0000","last_deep_scrub":"80'2","last_deep_scrub_stamp":"2024-02-23T00:40:15.942725+0000","last_clean_scrub_stamp":"2024-02-23T00:46:09.088249+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"93'9","reported_seq":353,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.164259+0000","last_change":"2024-02-23T00:43:50.393789+0000","last_active":"2024-02-23T00:46:11.164259+0000","last_peered":"2024-02-23T00:46:11.164259+0000","last_clean":"2024-02-23T00:46:11.164259+0000","last_became_active":"2024-02-23T00:39:18.796680+0000","last_became_peered":"2024-02-23T00:39:18.796680+0000","last_unstale":"2024-02-23T00:46:11.164259+0000","last_undegraded":"2024-02-23T00:46:11.164259+0000","last_fullsized":"2024-02-23T00:46:11.164259+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-02-23T00:43:50.393713+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-02-23T00:43:50.393713+0000","last_clean_scrub_stamp":"2024-02-23T00:43:50.393713+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":42,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.165870+0000","last_change":"2024-02-23T00:45:53.142552+0000","last_active":"2024-02-23T00:46:11.165870+0000","last_peered":"2024-02-23T00:46:11.165870+0000","last_clean":"2024-02-23T00:46:11.165870+0000","last_became_active":"2024-02-23T00:45:45.156323+0000","last_became_peered":"2024-02-23T00:45:45.156323+0000","last_unstale":"2024-02-23T00:46:11.165870+0000","last_undegraded":"2024-02-23T00:46:11.165870+0000","last_fullsized":"2024-02-23T00:46:11.165870+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:53.142473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:53.142473+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":205,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.157940+0000","last_change":"2024-02-23T00:45:58.613561+0000","last_active":"2024-02-23T00:46:11.157940+0000","last_peered":"2024-02-23T00:46:11.157940+0000","last_clean":"2024-02-23T00:46:11.157940+0000","last_became_active":"2024-02-23T00:42:58.850072+0000","last_became_peered":"2024-02-23T00:42:58.850072+0000","last_unstale":"2024-02-23T00:46:11.157940+0000","last_undegraded":"2024-02-23T00:46:11.157940+0000","last_fullsized":"2024-02-23T00:46:11.157940+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:58.613497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:58.613497+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"93'23","reported_seq":305,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.164347+0000","last_change":"2024-02-23T00:44:28.387225+0000","last_active":"2024-02-23T00:46:11.164347+0000","last_peered":"2024-02-23T00:46:11.164347+0000","last_clean":"2024-02-23T00:46:11.164347+0000","last_became_active":"2024-02-23T00:39:16.782660+0000","last_became_peered":"2024-02-23T00:39:16.782660+0000","last_unstale":"2024-02-23T00:46:11.164347+0000","last_undegraded":"2024-02-23T00:46:11.164347+0000","last_fullsized":"2024-02-23T00:46:11.164347+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'23","last_scrub_stamp":"2024-02-23T00:44:28.387155+0000","last_deep_scrub":"93'23","last_deep_scrub_stamp":"2024-02-23T00:41:35.954516+0000","last_clean_scrub_stamp":"2024-02-23T00:44:28.387155+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":264,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.147123+0000","last_change":"2024-02-23T00:44:21.157372+0000","last_active":"2024-02-23T00:46:11.147123+0000","last_peered":"2024-02-23T00:46:11.147123+0000","last_clean":"2024-02-23T00:46:11.147123+0000","last_became_active":"2024-02-23T00:39:14.781560+0000","last_became_peered":"2024-02-23T00:39:14.781560+0000","last_unstale":"2024-02-23T00:46:11.147123+0000","last_undegraded":"2024-02-23T00:46:11.147123+0000","last_fullsized":"2024-02-23T00:46:11.147123+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:21.157316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:21.157316+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":260,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.162827+0000","last_change":"2024-02-23T00:43:49.238770+0000","last_active":"2024-02-23T00:46:11.162827+0000","last_peered":"2024-02-23T00:46:11.162827+0000","last_clean":"2024-02-23T00:46:11.162827+0000","last_became_active":"2024-02-23T00:39:18.790602+0000","last_became_peered":"2024-02-23T00:39:18.790602+0000","last_unstale":"2024-02-23T00:46:11.162827+0000","last_undegraded":"2024-02-23T00:46:11.162827+0000","last_fullsized":"2024-02-23T00:46:11.162827+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:49.238704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:53.189881+0000","last_clean_scrub_stamp":"2024-02-23T00:43:49.238704+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.14","version":"110'2","reported_seq":41,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.149026+0000","last_change":"2024-02-23T00:45:45.250750+0000","last_active":"2024-02-23T00:46:11.149026+0000","last_peered":"2024-02-23T00:46:11.149026+0000","last_clean":"2024-02-23T00:46:11.149026+0000","last_became_active":"2024-02-23T00:45:45.161233+0000","last_became_peered":"2024-02-23T00:45:45.161233+0000","last_unstale":"2024-02-23T00:46:11.149026+0000","last_undegraded":"2024-02-23T00:46:11.149026+0000","last_fullsized":"2024-02-23T00:46:11.149026+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,3],"acting":[1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":205,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.147207+0000","last_change":"2024-02-23T00:45:15.933508+0000","last_active":"2024-02-23T00:46:11.147207+0000","last_peered":"2024-02-23T00:46:11.147207+0000","last_clean":"2024-02-23T00:46:11.147207+0000","last_became_active":"2024-02-23T00:42:58.774720+0000","last_became_peered":"2024-02-23T00:42:58.774720+0000","last_unstale":"2024-02-23T00:46:11.147207+0000","last_undegraded":"2024-02-23T00:46:11.147207+0000","last_fullsized":"2024-02-23T00:46:11.147207+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:15.933438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:15.933438+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"67'15","reported_seq":290,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.153861+0000","last_change":"2024-02-23T00:46:03.951592+0000","last_active":"2024-02-23T00:46:11.153861+0000","last_peered":"2024-02-23T00:46:11.153861+0000","last_clean":"2024-02-23T00:46:11.153861+0000","last_became_active":"2024-02-23T00:39:16.787138+0000","last_became_peered":"2024-02-23T00:39:16.787138+0000","last_unstale":"2024-02-23T00:46:11.153861+0000","last_undegraded":"2024-02-23T00:46:11.153861+0000","last_fullsized":"2024-02-23T00:46:11.153861+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:46:03.951524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:03.951524+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":267,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.168705+0000","last_change":"2024-02-23T00:46:10.011072+0000","last_active":"2024-02-23T00:46:11.168705+0000","last_peered":"2024-02-23T00:46:11.168705+0000","last_clean":"2024-02-23T00:46:11.168705+0000","last_became_active":"2024-02-23T00:39:14.782494+0000","last_became_peered":"2024-02-23T00:39:14.782494+0000","last_unstale":"2024-02-23T00:46:11.168705+0000","last_undegraded":"2024-02-23T00:46:11.168705+0000","last_fullsized":"2024-02-23T00:46:11.168705+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:10.010996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:10.010996+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":259,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.150317+0000","last_change":"2024-02-23T00:45:03.615758+0000","last_active":"2024-02-23T00:46:11.150317+0000","last_peered":"2024-02-23T00:46:11.150317+0000","last_clean":"2024-02-23T00:46:11.150317+0000","last_became_active":"2024-02-23T00:39:18.801039+0000","last_became_peered":"2024-02-23T00:39:18.801039+0000","last_unstale":"2024-02-23T00:46:11.150317+0000","last_undegraded":"2024-02-23T00:46:11.150317+0000","last_fullsized":"2024-02-23T00:46:11.150317+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:03.615692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.641689+0000","last_clean_scrub_stamp":"2024-02-23T00:45:03.615692+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"110'5","reported_seq":44,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.150306+0000","last_change":"2024-02-23T00:45:45.166028+0000","last_active":"2024-02-23T00:46:11.150306+0000","last_peered":"2024-02-23T00:46:11.150306+0000","last_clean":"2024-02-23T00:46:11.150306+0000","last_became_active":"2024-02-23T00:45:45.154195+0000","last_became_peered":"2024-02-23T00:45:45.154195+0000","last_unstale":"2024-02-23T00:46:11.150306+0000","last_undegraded":"2024-02-23T00:46:11.150306+0000","last_fullsized":"2024-02-23T00:46:11.150306+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'5","last_scrub_stamp":"2024-02-23T00:45:14.317573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.317573+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":205,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.150616+0000","last_change":"2024-02-23T00:45:21.917072+0000","last_active":"2024-02-23T00:46:11.150616+0000","last_peered":"2024-02-23T00:46:11.150616+0000","last_clean":"2024-02-23T00:46:11.150616+0000","last_became_active":"2024-02-23T00:42:58.770759+0000","last_became_peered":"2024-02-23T00:42:58.770759+0000","last_unstale":"2024-02-23T00:46:11.150616+0000","last_undegraded":"2024-02-23T00:46:11.150616+0000","last_fullsized":"2024-02-23T00:46:11.150616+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:21.916996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:21.916996+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"68'18","reported_seq":300,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.146903+0000","last_change":"2024-02-23T00:43:51.079824+0000","last_active":"2024-02-23T00:46:11.146903+0000","last_peered":"2024-02-23T00:46:11.146903+0000","last_clean":"2024-02-23T00:46:11.146903+0000","last_became_active":"2024-02-23T00:39:16.782803+0000","last_became_peered":"2024-02-23T00:39:16.782803+0000","last_unstale":"2024-02-23T00:46:11.146903+0000","last_undegraded":"2024-02-23T00:46:11.146903+0000","last_fullsized":"2024-02-23T00:46:11.146903+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'18","last_scrub_stamp":"2024-02-23T00:43:51.079756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:51.079756+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":264,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.149128+0000","last_change":"2024-02-23T00:43:48.030906+0000","last_active":"2024-02-23T00:46:11.149128+0000","last_peered":"2024-02-23T00:46:11.149128+0000","last_clean":"2024-02-23T00:46:11.149128+0000","last_became_active":"2024-02-23T00:39:14.782163+0000","last_became_peered":"2024-02-23T00:39:14.782163+0000","last_unstale":"2024-02-23T00:46:11.149128+0000","last_undegraded":"2024-02-23T00:46:11.149128+0000","last_fullsized":"2024-02-23T00:46:11.149128+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:48.030846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.030846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":259,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.148114+0000","last_change":"2024-02-23T00:44:37.068899+0000","last_active":"2024-02-23T00:46:11.148114+0000","last_peered":"2024-02-23T00:46:11.148114+0000","last_clean":"2024-02-23T00:46:11.148114+0000","last_became_active":"2024-02-23T00:39:18.794903+0000","last_became_peered":"2024-02-23T00:39:18.794903+0000","last_unstale":"2024-02-23T00:46:11.148114+0000","last_undegraded":"2024-02-23T00:46:11.148114+0000","last_fullsized":"2024-02-23T00:46:11.148114+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:37.068834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_clean_scrub_stamp":"2024-02-23T00:44:37.068834+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"294'6","reported_seq":225,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.162842+0000","last_change":"2024-02-23T00:46:07.542250+0000","last_active":"2024-02-23T00:46:11.162842+0000","last_peered":"2024-02-23T00:46:11.162842+0000","last_clean":"2024-02-23T00:46:11.162842+0000","last_became_active":"2024-02-23T00:45:45.157467+0000","last_became_peered":"2024-02-23T00:45:45.157467+0000","last_unstale":"2024-02-23T00:46:11.162842+0000","last_undegraded":"2024-02-23T00:46:11.162842+0000","last_fullsized":"2024-02-23T00:46:11.162842+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"294'6","last_scrub_stamp":"2024-02-23T00:46:07.542181+0000","last_deep_scrub":"294'6","last_deep_scrub_stamp":"2024-02-23T00:46:07.542181+0000","last_clean_scrub_stamp":"2024-02-23T00:46:07.542181+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":205,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.160237+0000","last_change":"2024-02-23T00:46:00.943625+0000","last_active":"2024-02-23T00:46:11.160237+0000","last_peered":"2024-02-23T00:46:11.160237+0000","last_clean":"2024-02-23T00:46:11.160237+0000","last_became_active":"2024-02-23T00:42:58.698258+0000","last_became_peered":"2024-02-23T00:42:58.698258+0000","last_unstale":"2024-02-23T00:46:11.160237+0000","last_undegraded":"2024-02-23T00:46:11.160237+0000","last_fullsized":"2024-02-23T00:46:11.160237+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:00.943545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:46:00.943545+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"95'27","reported_seq":313,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.147771+0000","last_change":"2024-02-23T00:43:52.124336+0000","last_active":"2024-02-23T00:46:11.147771+0000","last_peered":"2024-02-23T00:46:11.147771+0000","last_clean":"2024-02-23T00:46:11.147771+0000","last_became_active":"2024-02-23T00:39:16.786791+0000","last_became_peered":"2024-02-23T00:39:16.786791+0000","last_unstale":"2024-02-23T00:46:11.147771+0000","last_undegraded":"2024-02-23T00:46:11.147771+0000","last_fullsized":"2024-02-23T00:46:11.147771+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'27","last_scrub_stamp":"2024-02-23T00:43:52.124247+0000","last_deep_scrub":"95'27","last_deep_scrub_stamp":"2024-02-23T00:41:55.919895+0000","last_clean_scrub_stamp":"2024-02-23T00:43:52.124247+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":261,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.164922+0000","last_change":"2024-02-23T00:44:27.759571+0000","last_active":"2024-02-23T00:46:11.164922+0000","last_peered":"2024-02-23T00:46:11.164922+0000","last_clean":"2024-02-23T00:46:11.164922+0000","last_became_active":"2024-02-23T00:39:14.787148+0000","last_became_peered":"2024-02-23T00:39:14.787148+0000","last_unstale":"2024-02-23T00:46:11.164922+0000","last_undegraded":"2024-02-23T00:46:11.164922+0000","last_fullsized":"2024-02-23T00:46:11.164922+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:27.759499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:27.759499+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":260,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.160102+0000","last_change":"2024-02-23T00:45:10.090353+0000","last_active":"2024-02-23T00:46:11.160102+0000","last_peered":"2024-02-23T00:46:11.160102+0000","last_clean":"2024-02-23T00:46:11.160102+0000","last_became_active":"2024-02-23T00:39:18.796431+0000","last_became_peered":"2024-02-23T00:39:18.796431+0000","last_unstale":"2024-02-23T00:46:11.160102+0000","last_undegraded":"2024-02-23T00:46:11.160102+0000","last_fullsized":"2024-02-23T00:46:11.160102+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.090270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.090270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.11","version":"110'7","reported_seq":48,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.166856+0000","last_change":"2024-02-23T00:45:45.170820+0000","last_active":"2024-02-23T00:46:11.166856+0000","last_peered":"2024-02-23T00:46:11.166856+0000","last_clean":"2024-02-23T00:46:11.166856+0000","last_became_active":"2024-02-23T00:45:45.155700+0000","last_became_peered":"2024-02-23T00:45:45.155700+0000","last_unstale":"2024-02-23T00:46:11.166856+0000","last_undegraded":"2024-02-23T00:46:11.166856+0000","last_fullsized":"2024-02-23T00:46:11.166856+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'7","last_scrub_stamp":"2024-02-23T00:44:30.366391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:44:30.366391+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":205,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.156346+0000","last_change":"2024-02-23T00:45:51.985111+0000","last_active":"2024-02-23T00:46:11.156346+0000","last_peered":"2024-02-23T00:46:11.156346+0000","last_clean":"2024-02-23T00:46:11.156346+0000","last_became_active":"2024-02-23T00:42:58.832885+0000","last_became_peered":"2024-02-23T00:42:58.832885+0000","last_unstale":"2024-02-23T00:46:11.156346+0000","last_undegraded":"2024-02-23T00:46:11.156346+0000","last_fullsized":"2024-02-23T00:46:11.156346+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:51.985023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:51.985023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"82'15","reported_seq":290,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.147139+0000","last_change":"2024-02-23T00:44:35.095478+0000","last_active":"2024-02-23T00:46:11.147139+0000","last_peered":"2024-02-23T00:46:11.147139+0000","last_clean":"2024-02-23T00:46:11.147139+0000","last_became_active":"2024-02-23T00:39:16.782762+0000","last_became_peered":"2024-02-23T00:39:16.782762+0000","last_unstale":"2024-02-23T00:46:11.147139+0000","last_undegraded":"2024-02-23T00:46:11.147139+0000","last_fullsized":"2024-02-23T00:46:11.147139+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'15","last_scrub_stamp":"2024-02-23T00:44:35.095409+0000","last_deep_scrub":"82'15","last_deep_scrub_stamp":"2024-02-23T00:41:50.975683+0000","last_clean_scrub_stamp":"2024-02-23T00:44:35.095409+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":266,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.165338+0000","last_change":"2024-02-23T00:46:08.794420+0000","last_active":"2024-02-23T00:46:11.165338+0000","last_peered":"2024-02-23T00:46:11.165338+0000","last_clean":"2024-02-23T00:46:11.165338+0000","last_became_active":"2024-02-23T00:39:14.783469+0000","last_became_peered":"2024-02-23T00:39:14.783469+0000","last_unstale":"2024-02-23T00:46:11.165338+0000","last_undegraded":"2024-02-23T00:46:11.165338+0000","last_fullsized":"2024-02-23T00:46:11.165338+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:08.794352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:08.794352+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":264,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.162825+0000","last_change":"2024-02-23T00:46:09.832330+0000","last_active":"2024-02-23T00:46:11.162825+0000","last_peered":"2024-02-23T00:46:11.162825+0000","last_clean":"2024-02-23T00:46:11.162825+0000","last_became_active":"2024-02-23T00:39:18.796952+0000","last_became_peered":"2024-02-23T00:39:18.796952+0000","last_unstale":"2024-02-23T00:46:11.162825+0000","last_undegraded":"2024-02-23T00:46:11.162825+0000","last_fullsized":"2024-02-23T00:46:11.162825+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:09.832253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:09.832253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":37,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.153727+0000","last_change":"2024-02-23T00:45:45.160209+0000","last_active":"2024-02-23T00:46:11.153727+0000","last_peered":"2024-02-23T00:46:11.153727+0000","last_clean":"2024-02-23T00:46:11.153727+0000","last_became_active":"2024-02-23T00:45:45.159835+0000","last_became_peered":"2024-02-23T00:45:45.159835+0000","last_unstale":"2024-02-23T00:46:11.153727+0000","last_undegraded":"2024-02-23T00:46:11.153727+0000","last_fullsized":"2024-02-23T00:46:11.153727+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.215063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.215063+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":207,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.147590+0000","last_change":"2024-02-23T00:45:23.224771+0000","last_active":"2024-02-23T00:46:11.147590+0000","last_peered":"2024-02-23T00:46:11.147590+0000","last_clean":"2024-02-23T00:46:11.147590+0000","last_became_active":"2024-02-23T00:42:58.758147+0000","last_became_peered":"2024-02-23T00:42:58.758147+0000","last_unstale":"2024-02-23T00:46:11.147590+0000","last_undegraded":"2024-02-23T00:46:11.147590+0000","last_fullsized":"2024-02-23T00:46:11.147590+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:23.224663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:23.224663+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.224663+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"68'15","reported_seq":292,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.146243+0000","last_change":"2024-02-23T00:43:49.139235+0000","last_active":"2024-02-23T00:46:11.146243+0000","last_peered":"2024-02-23T00:46:11.146243+0000","last_clean":"2024-02-23T00:46:11.146243+0000","last_became_active":"2024-02-23T00:39:16.784606+0000","last_became_peered":"2024-02-23T00:39:16.784606+0000","last_unstale":"2024-02-23T00:46:11.146243+0000","last_undegraded":"2024-02-23T00:46:11.146243+0000","last_fullsized":"2024-02-23T00:46:11.146243+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:43:49.139156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:49.139156+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":264,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.155541+0000","last_change":"2024-02-23T00:44:32.170717+0000","last_active":"2024-02-23T00:46:11.155541+0000","last_peered":"2024-02-23T00:46:11.155541+0000","last_clean":"2024-02-23T00:46:11.155541+0000","last_became_active":"2024-02-23T00:39:14.785362+0000","last_became_peered":"2024-02-23T00:39:14.785362+0000","last_unstale":"2024-02-23T00:46:11.155541+0000","last_undegraded":"2024-02-23T00:46:11.155541+0000","last_fullsized":"2024-02-23T00:46:11.155541+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:32.170554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:32.170554+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":254,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.149328+0000","last_change":"2024-02-23T00:42:44.553722+0000","last_active":"2024-02-23T00:46:11.149328+0000","last_peered":"2024-02-23T00:46:11.149328+0000","last_clean":"2024-02-23T00:46:11.149328+0000","last_became_active":"2024-02-23T00:39:18.796312+0000","last_became_peered":"2024-02-23T00:39:18.796312+0000","last_unstale":"2024-02-23T00:46:11.149328+0000","last_undegraded":"2024-02-23T00:46:11.149328+0000","last_fullsized":"2024-02-23T00:46:11.149328+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:44.553621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:44.553621+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.13","version":"110'9","reported_seq":48,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.146540+0000","last_change":"2024-02-23T00:45:45.176912+0000","last_active":"2024-02-23T00:46:11.146540+0000","last_peered":"2024-02-23T00:46:11.146540+0000","last_clean":"2024-02-23T00:46:11.146540+0000","last_became_active":"2024-02-23T00:45:45.162350+0000","last_became_peered":"2024-02-23T00:45:45.162350+0000","last_unstale":"2024-02-23T00:46:11.146540+0000","last_undegraded":"2024-02-23T00:46:11.146540+0000","last_fullsized":"2024-02-23T00:46:11.146540+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'9","last_scrub_stamp":"2024-02-23T00:45:21.719900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:21.719900+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":205,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.162615+0000","last_change":"2024-02-23T00:45:09.641006+0000","last_active":"2024-02-23T00:46:11.162615+0000","last_peered":"2024-02-23T00:46:11.162615+0000","last_clean":"2024-02-23T00:46:11.162615+0000","last_became_active":"2024-02-23T00:42:58.838257+0000","last_became_peered":"2024-02-23T00:42:58.838257+0000","last_unstale":"2024-02-23T00:46:11.162615+0000","last_undegraded":"2024-02-23T00:46:11.162615+0000","last_fullsized":"2024-02-23T00:46:11.162615+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:09.640936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:09.640936+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"88'50","reported_seq":358,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.147563+0000","last_change":"2024-02-23T00:45:06.038488+0000","last_active":"2024-02-23T00:46:11.147563+0000","last_peered":"2024-02-23T00:46:11.147563+0000","last_clean":"2024-02-23T00:46:11.147563+0000","last_became_active":"2024-02-23T00:39:16.780144+0000","last_became_peered":"2024-02-23T00:39:16.780144+0000","last_unstale":"2024-02-23T00:46:11.147563+0000","last_undegraded":"2024-02-23T00:46:11.147563+0000","last_fullsized":"2024-02-23T00:46:11.147563+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'50","last_scrub_stamp":"2024-02-23T00:45:06.038374+0000","last_deep_scrub":"88'50","last_deep_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.038374+0000","log_size":50,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":264,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.163000+0000","last_change":"2024-02-23T00:45:20.573233+0000","last_active":"2024-02-23T00:46:11.163000+0000","last_peered":"2024-02-23T00:46:11.163000+0000","last_clean":"2024-02-23T00:46:11.163000+0000","last_became_active":"2024-02-23T00:39:14.782000+0000","last_became_peered":"2024-02-23T00:39:14.782000+0000","last_unstale":"2024-02-23T00:46:11.163000+0000","last_undegraded":"2024-02-23T00:46:11.163000+0000","last_fullsized":"2024-02-23T00:46:11.163000+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:20.573168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_clean_scrub_stamp":"2024-02-23T00:45:20.573168+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":254,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.146493+0000","last_change":"2024-02-23T00:43:48.126631+0000","last_active":"2024-02-23T00:46:11.146493+0000","last_peered":"2024-02-23T00:46:11.146493+0000","last_clean":"2024-02-23T00:46:11.146493+0000","last_became_active":"2024-02-23T00:39:18.799723+0000","last_became_peered":"2024-02-23T00:39:18.799723+0000","last_unstale":"2024-02-23T00:46:11.146493+0000","last_undegraded":"2024-02-23T00:46:11.146493+0000","last_fullsized":"2024-02-23T00:46:11.146493+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:48.126552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.126552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":37,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.155073+0000","last_change":"2024-02-23T00:45:45.157332+0000","last_active":"2024-02-23T00:46:11.155073+0000","last_peered":"2024-02-23T00:46:11.155073+0000","last_clean":"2024-02-23T00:46:11.155073+0000","last_became_active":"2024-02-23T00:45:45.156946+0000","last_became_peered":"2024-02-23T00:45:45.156946+0000","last_unstale":"2024-02-23T00:46:11.155073+0000","last_undegraded":"2024-02-23T00:46:11.155073+0000","last_fullsized":"2024-02-23T00:46:11.155073+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:24.942535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:24.942535+0000","last_clean_scrub_stamp":"2024-02-23T00:45:24.942535+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":205,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-02-23T00:46:11.156915+0000","last_change":"2024-02-23T00:45:14.209403+0000","last_active":"2024-02-23T00:46:11.156915+0000","last_peered":"2024-02-23T00:46:11.156915+0000","last_clean":"2024-02-23T00:46:11.156915+0000","last_became_active":"2024-02-23T00:42:58.824573+0000","last_became_peered":"2024-02-23T00:42:58.824573+0000","last_unstale":"2024-02-23T00:46:11.156915+0000","last_undegraded":"2024-02-23T00:46:11.156915+0000","last_fullsized":"2024-02-23T00:46:11.156915+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.209308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.209308+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0],"acting":[5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":185,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":184,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":8},{"poolid":62,"num_pg":32,"stat_sum":{"num_bytes":1212,"num_objects":122,"num_object_clones":2,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":0,"num_read":98,"num_read_kb":4,"num_write":547,"num_write_kb":542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1486848,"data_stored":3636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1793,"ondisk_log_size":1793,"up":96,"acting":96,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":22,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":66,"acting":66,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1080,"num_read_kb":805,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":651,"ondisk_log_size":651,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":409,"num_read_kb":404,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":8},{"poolid":15,"num_pg":25,"stat_sum":{"num_bytes":2048,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":196608,"data_stored":6144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57,"ondisk_log_size":57,"up":75,"acting":75,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234071,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":302208,"kb_used_data":4984,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93442944,"statfs":{"total":95995035648,"available":95685574656,"internally_reserved":0,"allocated":5103616,"data_stored":3045243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.28000000000000003,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.747,"5min":4.9420000000000002,"15min":4.9420000000000002},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.53200000000000003,"15min":0.51600000000000001},"min":{"1min":0.32600000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0589999999999999,"5min":4.9729999999999999,"15min":4.9729999999999999},"last":0.71699999999999997}]},{"osd":1,"last update":"Fri Feb 23 00:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.77400000000000002,"15min":0.626},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":5.4539999999999997,"5min":10.786,"15min":10.786},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.78700000000000003,"15min":0.629},"min":{"1min":0.29399999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":11.885999999999999,"5min":11.885999999999999,"15min":11.885999999999999},"last":0.67000000000000004}]},{"osd":2,"last update":"Fri Feb 23 00:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59299999999999997,"15min":0.55900000000000005},"min":{"1min":0.35899999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":4.0449999999999999,"5min":5.0789999999999997,"15min":5.0789999999999997},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.57399999999999995,"15min":0.54800000000000004},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":1.04,"5min":5.1280000000000001,"15min":5.1280000000000001},"last":0.63200000000000001}]},{"osd":3,"last update":"Fri Feb 23 00:45:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":0.80800000000000005,"15min":0.62},"min":{"1min":0.35499999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":15.712999999999999,"5min":15.712999999999999,"15min":15.712999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.71699999999999997,"15min":0.61799999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.004,"5min":9.4090000000000007,"15min":9.4090000000000007},"last":0.73099999999999998}]},{"osd":4,"last update":"Fri Feb 23 00:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.44500000000000001,"15min":0.436},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":2.0350000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.55100000000000005,"15min":0.48599999999999999},"min":{"1min":0.28599999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":11.236000000000001,"5min":11.236000000000001,"15min":11.236000000000001},"last":0.38600000000000001}]},{"osd":5,"last update":"Fri Feb 23 00:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.51700000000000002,"15min":0.49099999999999999},"min":{"1min":0.29699999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":5.4000000000000004,"5min":5.4000000000000004,"15min":5.4000000000000004},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.50800000000000001,"15min":0.48599999999999999},"min":{"1min":0.37,"5min":0.29299999999999998,"15min":0.26700000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.52700000000000002}]},{"osd":6,"last update":"Fri Feb 23 00:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51000000000000001,"15min":0.50700000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.48099999999999998},{"interface":"front","average":{"1min":1.093,"5min":0.61799999999999999,"15min":0.54400000000000004},"min":{"1min":0.35599999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":13.395,"5min":13.395,"15min":13.395},"last":0.434}]}]},{"osd":6,"up_from":47,"seq":201863463006,"num_pgs":91,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":314480,"kb_used_data":4960,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93430672,"statfs":{"total":95995035648,"available":95673008128,"internally_reserved":0,"allocated":5079040,"data_stored":3046030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.97299999999999998,"15min":0.70499999999999996},"min":{"1min":0.33500000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.65000000000000002,"5min":30.641999999999999,"15min":30.641999999999999},"last":0.377},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.97499999999999998,"15min":0.71399999999999997},"min":{"1min":0.42999999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.71199999999999997,"5min":29.292000000000002,"15min":29.292000000000002},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri Feb 23 00:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.98499999999999999,"15min":0.69599999999999995},"min":{"1min":0.40600000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.76700000000000002,"5min":28.186,"15min":28.186},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":1.48,"15min":0.88200000000000001},"min":{"1min":0.39700000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.67800000000000005,"5min":33.104999999999997,"15min":33.104999999999997},"last":0.35699999999999998}]},{"osd":2,"last update":"Fri Feb 23 00:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":1.034,"15min":0.73399999999999999},"min":{"1min":0.38500000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.75600000000000001,"5min":19.960000000000001,"15min":19.960000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.84299999999999997,"15min":0.65500000000000003},"min":{"1min":0.38600000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.79300000000000004,"5min":8.5860000000000003,"15min":8.5860000000000003},"last":0.42099999999999999}]},{"osd":3,"last update":"Fri Feb 23 00:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.75600000000000001,"15min":0.64000000000000001},"min":{"1min":0.439,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80000000000000004,"5min":19.562000000000001,"15min":19.562000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.84699999999999998,"15min":0.67900000000000005},"min":{"1min":0.35399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69999999999999996,"5min":14.199,"15min":14.199},"last":0.495}]},{"osd":4,"last update":"Fri Feb 23 00:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.58299999999999996,"15min":0.46500000000000002},"min":{"1min":0.27700000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.70299999999999996,"5min":11.109999999999999,"15min":11.109999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.752,"15min":0.53600000000000003},"min":{"1min":0.314,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.71099999999999997,"5min":16.734000000000002,"15min":16.734000000000002},"last":0.51400000000000001}]},{"osd":5,"last update":"Fri Feb 23 00:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.63700000000000001,"15min":0.48799999999999999},"min":{"1min":0.32400000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.745,"5min":14.154999999999999,"15min":14.154999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.63500000000000001,"15min":0.504},"min":{"1min":0.27900000000000003,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.67600000000000005,"5min":6.6100000000000003,"15min":6.6100000000000003},"last":0.56299999999999994}]},{"osd":7,"last update":"Fri Feb 23 00:45:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.58399999999999996,"15min":0.52000000000000002},"min":{"1min":0.29299999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.745,"5min":8.548,"15min":8.548},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.496,"5min":0.752,"15min":0.59399999999999997},"min":{"1min":0.33900000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.68899999999999995,"5min":15.618,"15min":15.618},"last":0.57999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":100,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":310336,"kb_used_data":4912,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93434816,"statfs":{"total":95995035648,"available":95677251584,"internally_reserved":0,"allocated":5029888,"data_stored":3046713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:45:43 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":1.2909999999999999,"15min":0.66400000000000003},"min":{"1min":0.221,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.48799999999999999,"5min":78.361000000000004,"15min":78.361000000000004},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.57199999999999995,"15min":0.437},"min":{"1min":0.245,"5min":0.218,"15min":0.191},"max":{"1min":0.41199999999999998,"5min":5.8849999999999998,"15min":5.8849999999999998},"last":0.33600000000000002}]},{"osd":2,"last update":"Fri Feb 23 00:46:04 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":1.181,"15min":0.65600000000000003},"min":{"1min":0.25,"5min":0.219,"15min":0.188},"max":{"1min":0.63800000000000001,"5min":48.606000000000002,"15min":48.606000000000002},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.879,"15min":0.58399999999999996},"min":{"1min":0.26700000000000002,"5min":0.25800000000000001,"15min":0.249},"max":{"1min":1.3460000000000001,"5min":23.911000000000001,"15min":23.911000000000001},"last":0.44600000000000001}]},{"osd":3,"last update":"Fri Feb 23 00:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.89400000000000002,"15min":0.624},"min":{"1min":0.28000000000000003,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.56100000000000005,"5min":35.244999999999997,"15min":35.244999999999997},"last":0.374},{"interface":"front","average":{"1min":0.38,"5min":1.1299999999999999,"15min":0.68200000000000005},"min":{"1min":0.28599999999999998,"5min":0.253,"15min":0.248},"max":{"1min":0.60299999999999998,"5min":56.853999999999999,"15min":56.853999999999999},"last":0.436}]},{"osd":4,"last update":"Fri Feb 23 00:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":1.2310000000000001,"15min":0.81699999999999995},"min":{"1min":0.34599999999999997,"5min":0.33700000000000002,"15min":0.29299999999999998},"max":{"1min":0.82399999999999995,"5min":37.646999999999998,"15min":37.646999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.86599999999999999,"15min":0.69699999999999995},"min":{"1min":0.375,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80700000000000005,"5min":9.6859999999999999,"15min":9.6859999999999999},"last":0.53600000000000003}]},{"osd":5,"last update":"Fri Feb 23 00:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":1.224,"15min":0.84599999999999997},"min":{"1min":0.41099999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83399999999999996,"5min":48.792000000000002,"15min":48.792000000000002},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.83699999999999997,"15min":0.755},"min":{"1min":0.45100000000000001,"5min":0.38100000000000001,"15min":0.34200000000000003},"max":{"1min":0.76500000000000001,"5min":11.872999999999999,"15min":11.872999999999999},"last":0.59999999999999998}]},{"osd":6,"last update":"Fri Feb 23 00:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":1.1890000000000001,"15min":0.82299999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.79400000000000004,"5min":40.741,"15min":40.741},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.77200000000000002,"15min":0.68999999999999995},"min":{"1min":0.41999999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.872,"5min":6.0069999999999997,"15min":6.0069999999999997},"last":0.67100000000000004}]},{"osd":7,"last update":"Fri Feb 23 00:45:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.91000000000000003,"15min":0.71899999999999997},"min":{"1min":0.42499999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82799999999999996,"5min":20.225000000000001,"15min":20.225000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.94199999999999995,"15min":0.72799999999999998},"min":{"1min":0.42799999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85999999999999999,"5min":20.254999999999999,"15min":20.254999999999999},"last":0.58299999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738494,"num_pgs":90,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":314448,"kb_used_data":4924,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93430704,"statfs":{"total":95995035648,"available":95673040896,"internally_reserved":0,"allocated":5042176,"data_stored":3045918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 23 00:45:44 2024","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":1.089,"15min":0.59199999999999997},"min":{"1min":0.22600000000000001,"5min":0.223,"15min":0.19500000000000001},"max":{"1min":0.60899999999999999,"5min":28.856999999999999,"15min":28.856999999999999},"last":0.26500000000000001},{"interface":"front","average":{"1min":1.026,"5min":1.1000000000000001,"15min":0.60699999999999998},"min":{"1min":0.25600000000000001,"5min":0.218,"15min":0.20799999999999999},"max":{"1min":13.162000000000001,"5min":20.577000000000002,"15min":20.577000000000002},"last":0.28699999999999998}]},{"osd":2,"last update":"Fri Feb 23 00:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":1.014,"15min":0.60299999999999998},"min":{"1min":0.27900000000000003,"5min":0.26000000000000001,"15min":0.20499999999999999},"max":{"1min":0.55200000000000005,"5min":13.863,"15min":13.863},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.318,"5min":0.86899999999999999,"15min":0.54500000000000004},"min":{"1min":0.219,"5min":0.219,"15min":0.19800000000000001},"max":{"1min":0.373,"5min":10.455,"15min":10.455},"last":0.40300000000000002}]},{"osd":3,"last update":"Fri Feb 23 00:45:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.5109999999999999,"15min":0.80300000000000005},"min":{"1min":0.27700000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":12.566000000000001,"5min":35.566000000000003,"15min":35.566000000000003},"last":0.32500000000000001},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.3660000000000001,"15min":0.79100000000000004},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":12.727,"5min":20.702999999999999,"15min":20.702999999999999},"last":0.38900000000000001}]},{"osd":4,"last update":"Fri Feb 23 00:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":1.0409999999999999,"15min":0.80300000000000005},"min":{"1min":0.36399999999999999,"5min":0.36199999999999999,"15min":0.34699999999999998},"max":{"1min":0.753,"5min":10.584,"15min":10.584},"last":0.61199999999999999},{"interface":"front","average":{"1min":1.258,"5min":1.032,"15min":0.82999999999999996},"min":{"1min":0.38900000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":13.198,"5min":13.198,"15min":13.198},"last":0.59299999999999997}]},{"osd":5,"last update":"Fri Feb 23 00:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":1.014,"15min":0.74399999999999999},"min":{"1min":0.37,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88200000000000001,"5min":10.397,"15min":10.397},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":1.079,"15min":0.80900000000000005},"min":{"1min":0.44700000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86299999999999999,"5min":17.297000000000001,"15min":17.297000000000001},"last":0.60499999999999998}]},{"osd":6,"last update":"Fri Feb 23 00:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":1.1080000000000001,"15min":0.79900000000000004},"min":{"1min":0.44400000000000001,"5min":0.42299999999999999,"15min":0.38400000000000001},"max":{"1min":0.85799999999999998,"5min":17.373999999999999,"15min":17.373999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":1.321,"5min":1.0409999999999999,"15min":0.77000000000000002},"min":{"1min":0.42199999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":13.973000000000001,"5min":13.973000000000001,"15min":13.973000000000001},"last":0.53500000000000003}]},{"osd":7,"last update":"Fri Feb 23 00:45:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":1.081,"15min":0.81299999999999994},"min":{"1min":0.48199999999999998,"5min":0.41699999999999998,"15min":0.39400000000000002},"max":{"1min":0.82599999999999996,"5min":17.399999999999999,"15min":17.399999999999999},"last":0.623},{"interface":"front","average":{"1min":1.494,"5min":1.0640000000000001,"15min":0.78900000000000003},"min":{"1min":0.45400000000000001,"5min":0.433,"15min":0.43099999999999999},"max":{"1min":16.427,"5min":16.427,"15min":16.427},"last":0.56699999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411443,"num_pgs":88,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":310456,"kb_used_data":5032,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93434696,"statfs":{"total":95995035648,"available":95677128704,"internally_reserved":0,"allocated":5152768,"data_stored":3045765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:46:04 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.69399999999999995,"15min":0.48399999999999999},"min":{"1min":0.22900000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.61199999999999999,"5min":13.657,"15min":13.657},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.374,"5min":1.262,"15min":0.68000000000000005},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.624,"5min":42.343000000000004,"15min":42.343000000000004},"last":0.27400000000000002}]},{"osd":1,"last update":"Fri Feb 23 00:46:04 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.96199999999999997,"15min":0.57399999999999995},"min":{"1min":0.254,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.53500000000000003,"5min":18.952000000000002,"15min":18.952000000000002},"last":0.311},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.75800000000000001,"15min":0.52200000000000002},"min":{"1min":0.26300000000000001,"5min":0.23999999999999999,"15min":0.23599999999999999},"max":{"1min":0.56499999999999995,"5min":15.391,"15min":15.391},"last":0.29899999999999999}]},{"osd":3,"last update":"Fri Feb 23 00:45:35 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":1.1779999999999999,"15min":0.71299999999999997},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.47499999999999998,"5min":25.635999999999999,"15min":25.635999999999999},"last":0.379},{"interface":"front","average":{"1min":0.41799999999999998,"5min":1.002,"15min":0.64600000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.0840000000000001,"5min":19.960000000000001,"15min":19.960000000000001},"last":0.40899999999999997}]},{"osd":4,"last update":"Fri Feb 23 00:46:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.99199999999999999,"15min":0.72799999999999998},"min":{"1min":0.41299999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.74099999999999999,"5min":13.673,"15min":13.673},"last":0.496},{"interface":"front","average":{"1min":0.57599999999999996,"5min":1.0389999999999999,"15min":0.73499999999999999},"min":{"1min":0.379,"5min":0.36899999999999999,"15min":0.33900000000000002},"max":{"1min":0.90000000000000002,"5min":10.103999999999999,"15min":10.103999999999999},"last":0.51200000000000001}]},{"osd":5,"last update":"Fri Feb 23 00:45:23 2024","interfaces":[{"interface":"back","average":{"1min":1.784,"5min":0.90600000000000003,"15min":0.69099999999999995},"min":{"1min":0.40699999999999997,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":15.612,"5min":15.612,"15min":15.612},"last":0.44700000000000001},{"interface":"front","average":{"1min":1.335,"5min":0.88,"15min":0.70299999999999996},"min":{"1min":0.376,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":13.632999999999999,"5min":13.632999999999999,"15min":13.632999999999999},"last":0.52900000000000003}]},{"osd":6,"last update":"Fri Feb 23 00:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.88500000000000001,"15min":0.71099999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.5249999999999999,"5min":11.544,"15min":11.544},"last":0.503},{"interface":"front","average":{"1min":0.84599999999999997,"5min":1.05,"15min":0.749},"min":{"1min":0.45400000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":4.1909999999999998,"5min":10.132999999999999,"15min":10.132999999999999},"last":0.439}]},{"osd":7,"last update":"Fri Feb 23 00:45:15 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":0.879,"15min":0.74299999999999999},"min":{"1min":0.40600000000000003,"5min":0.377,"15min":0.377},"max":{"1min":11.599,"5min":11.599,"15min":11.599},"last":0.54800000000000004},{"interface":"front","average":{"1min":1.494,"5min":1.097,"15min":0.80000000000000004},"min":{"1min":0.42399999999999999,"5min":0.374,"15min":0.374},"max":{"1min":8.7469999999999999,"5min":10.148999999999999,"15min":10.148999999999999},"last":0.46100000000000002}]}]},{"osd":3,"up_from":26,"seq":111669149804,"num_pgs":101,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":314516,"kb_used_data":5000,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93430636,"statfs":{"total":95995035648,"available":95672971264,"internally_reserved":0,"allocated":5120000,"data_stored":3047406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:45:35 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.746,"15min":0.53900000000000003},"min":{"1min":0.22800000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.56699999999999995,"5min":12.212999999999999,"15min":12.212999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.88900000000000001,"15min":0.59899999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.56899999999999995,"5min":11.714,"15min":11.714},"last":0.40400000000000003}]},{"osd":1,"last update":"Fri Feb 23 00:45:35 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.871,"15min":0.60799999999999998},"min":{"1min":0.29299999999999998,"5min":0.23400000000000001,"15min":0.22900000000000001},"max":{"1min":0.61299999999999999,"5min":11.742000000000001,"15min":11.742000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.78200000000000003,"15min":0.56100000000000005},"min":{"1min":0.223,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":1.8300000000000001,"5min":6.383,"15min":6.383},"last":0.63500000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:45:35 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.95699999999999996,"15min":0.64900000000000002},"min":{"1min":0.26000000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.869,"5min":13.359999999999999,"15min":13.359999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.374,"5min":1.002,"15min":0.65500000000000003},"min":{"1min":0.27500000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54400000000000004,"5min":21.122,"15min":21.122},"last":0.74399999999999999}]},{"osd":4,"last update":"Fri Feb 23 00:45:58 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.083,"15min":0.75900000000000001},"min":{"1min":0.36399999999999999,"5min":0.317,"15min":0.317},"max":{"1min":11.558,"5min":11.644,"15min":11.644},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.93600000000000005,"15min":0.70399999999999996},"min":{"1min":0.40400000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":6.1589999999999998,"5min":6.2549999999999999,"15min":6.2549999999999999},"last":0.71699999999999997}]},{"osd":5,"last update":"Fri Feb 23 00:45:23 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.76300000000000001,"15min":0.65900000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72999999999999998,"5min":5.6360000000000001,"15min":5.6360000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":1.016,"15min":0.75600000000000001},"min":{"1min":0.40699999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79300000000000004,"5min":11.597,"15min":11.597},"last":0.67400000000000004}]},{"osd":6,"last update":"Fri Feb 23 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.90500000000000003,"15min":0.70599999999999996},"min":{"1min":0.39000000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.0539999999999998,"5min":6.343,"15min":6.343},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.80000000000000004,"15min":0.68200000000000005},"min":{"1min":0.442,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.71699999999999997,"5min":5.6680000000000001,"15min":5.6680000000000001},"last":0.69099999999999995}]},{"osd":7,"last update":"Fri Feb 23 00:45:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":1.0589999999999999,"15min":0.77800000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.39000000000000001},"max":{"1min":2.6040000000000001,"5min":11.67,"15min":11.67},"last":0.628},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.95499999999999996,"15min":0.75700000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":2.6179999999999999,"5min":6.3010000000000002,"15min":6.3010000000000002},"last":0.80900000000000005}]}]},{"osd":4,"up_from":32,"seq":137438953576,"num_pgs":91,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":314496,"kb_used_data":4972,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93430656,"statfs":{"total":95995035648,"available":95672991744,"internally_reserved":0,"allocated":5091328,"data_stored":3045787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:46:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51900000000000002,"15min":0.55600000000000005},"min":{"1min":0.30199999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.90900000000000003,"5min":3.5219999999999998,"15min":3.5219999999999998},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.66400000000000003,"15min":0.60799999999999998},"min":{"1min":0.38400000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.71499999999999997,"5min":8.6140000000000008,"15min":8.6140000000000008},"last":0.47099999999999997}]},{"osd":1,"last update":"Fri Feb 23 00:46:02 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.66500000000000004,"15min":0.60599999999999998},"min":{"1min":0.33600000000000002,"5min":0.28299999999999997,"15min":0.27800000000000002},"max":{"1min":0.66300000000000003,"5min":9.6110000000000007,"15min":9.6110000000000007},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.65400000000000003,"15min":0.59999999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.72699999999999998,"5min":6.6950000000000003,"15min":6.6950000000000003},"last":0.53800000000000003}]},{"osd":2,"last update":"Fri Feb 23 00:46:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.622,"15min":0.59099999999999997},"min":{"1min":0.34000000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.83099999999999996,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.59799999999999998,"15min":0.58899999999999997},"min":{"1min":0.34799999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74399999999999999,"5min":5.8440000000000003,"15min":5.8440000000000003},"last":0.55700000000000005}]},{"osd":3,"last update":"Fri Feb 23 00:46:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.65500000000000003,"15min":0.60399999999999998},"min":{"1min":0.35899999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80700000000000005,"5min":6.5540000000000003,"15min":6.5540000000000003},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.65900000000000003,"15min":0.60599999999999998},"min":{"1min":0.33100000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78000000000000003,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":0.45100000000000001}]},{"osd":5,"last update":"Fri Feb 23 00:45:23 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.41899999999999998,"15min":0.38300000000000001},"min":{"1min":0.25600000000000001,"5min":0.251,"15min":0.246},"max":{"1min":0.64700000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.48699999999999999,"15min":0.435},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.28899999999999998},"max":{"1min":0.71599999999999997,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.59599999999999997}]},{"osd":6,"last update":"Fri Feb 23 00:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.45300000000000001,"15min":0.41699999999999998},"min":{"1min":0.28699999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.68500000000000005,"5min":0.75,"15min":0.75},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45500000000000002,"15min":0.41699999999999998},"min":{"1min":0.27000000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.67100000000000004,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.54500000000000004}]},{"osd":7,"last update":"Fri Feb 23 00:45:17 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30599999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.67100000000000004,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.497,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.70599999999999996,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.58699999999999997}]}]},{"osd":5,"up_from":39,"seq":167503724643,"num_pgs":77,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":314424,"kb_used_data":4908,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93430728,"statfs":{"total":95995035648,"available":95673065472,"internally_reserved":0,"allocated":5025792,"data_stored":3046751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.78800000000000003,"15min":0.60199999999999998},"min":{"1min":0.28299999999999997,"5min":0.246,"15min":0.246},"max":{"1min":0.67900000000000005,"5min":23.881,"15min":23.881},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.68200000000000005,"15min":0.57899999999999996},"min":{"1min":0.28999999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.57799999999999996,"5min":5.4649999999999999,"15min":5.4649999999999999},"last":0.58299999999999996}]},{"osd":1,"last update":"Fri Feb 23 00:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.88600000000000001,"15min":0.64800000000000002},"min":{"1min":0.30199999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.74099999999999999,"5min":18.175999999999998,"15min":18.175999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.78300000000000003,"15min":0.624},"min":{"1min":0.35699999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.60299999999999998,"5min":9.0950000000000006,"15min":9.0950000000000006},"last":0.47699999999999998}]},{"osd":2,"last update":"Fri Feb 23 00:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":1.0169999999999999,"15min":0.70599999999999996},"min":{"1min":0.32600000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.59799999999999998,"5min":22.282,"15min":22.282},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":1.0489999999999999,"15min":0.70899999999999996},"min":{"1min":0.33200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.63700000000000001,"5min":23.925000000000001,"15min":23.925000000000001},"last":0.67600000000000005}]},{"osd":3,"last update":"Fri Feb 23 00:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.77700000000000002,"15min":0.61399999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.63700000000000001,"5min":6.5730000000000004,"15min":6.5730000000000004},"last":0.5},{"interface":"front","average":{"1min":0.505,"5min":0.79900000000000004,"15min":0.64000000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33500000000000002},"max":{"1min":0.64800000000000002,"5min":8.1760000000000002,"15min":8.1760000000000002},"last":0.65200000000000002}]},{"osd":4,"last update":"Fri Feb 23 00:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.42899999999999999,"15min":0.39600000000000002},"min":{"1min":0.26400000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.57899999999999996,"5min":0.68999999999999995,"15min":0.70399999999999996},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.51800000000000002,"15min":0.42899999999999999},"min":{"1min":0.30199999999999999,"5min":0.27500000000000002,"15min":0.27300000000000002},"max":{"1min":0.52500000000000002,"5min":3.879,"15min":3.879},"last":0.44900000000000001}]},{"osd":6,"last update":"Fri Feb 23 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.48899999999999999,"15min":0.44600000000000001},"min":{"1min":0.30599999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70999999999999996,"5min":1.736,"15min":1.736},"last":0.379},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.49199999999999999,"15min":0.45900000000000002},"min":{"1min":0.311,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73099999999999998,"5min":0.871,"15min":0.871},"last":0.41499999999999998}]},{"osd":7,"last update":"Fri Feb 23 00:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.57799999999999996,"15min":0.53200000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.71299999999999997,"5min":7.0300000000000002,"15min":7.0300000000000002},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.57799999999999996,"15min":0.53100000000000003},"min":{"1min":0.33400000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.58499999999999996,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":0.432}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":274432,"data_stored":647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":172032,"data_stored":417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":196608,"data_stored":493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":192512,"data_stored":484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":233472,"data_stored":558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":184,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":184,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":184,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":184,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":184,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":184,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":184,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":184,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-23T00:46:19.486 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph status --format=json 2024-02-23T00:46:19.757 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:17.784322+0000 osd.1 (osd.1) 141 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:46:19.757 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:17.784508+0000 osd.1 (osd.1) 142 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:46:19.757 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:18.155592+0000 mon.a (mon.0) 2051 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:46:19.757 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:18.164801+0000 mon.a (mon.0) 2052 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-37"}]': finished 2024-02-23T00:46:19.758 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:18.164912+0000 mon.a (mon.0) 2053 : cluster [DBG] osdmap e306: 8 total, 8 up, 8 in 2024-02-23T00:46:19.758 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:18.168446+0000 mon.c (mon.2) 238 : audit [INF] from='client.? 172.21.15.47:0/234645247' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi047-82386-52"}]: dispatch 2024-02-23T00:46:19.758 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:18.169086+0000 mon.a (mon.0) 2054 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi047-82386-52"}]: dispatch 2024-02-23T00:46:19.758 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:18.552531+0000 osd.0 (osd.0) 131 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:46:19.758 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:18.552733+0000 osd.0 (osd.0) 132 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:46:19.758 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:17.784322+0000 osd.1 (osd.1) 141 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:46:19.758 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:17.784508+0000 osd.1 (osd.1) 142 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:46:19.758 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:18.155592+0000 mon.a (mon.0) 2051 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:46:19.758 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:18.164801+0000 mon.a (mon.0) 2052 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-37"}]': finished 2024-02-23T00:46:19.758 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:18.164912+0000 mon.a (mon.0) 2053 : cluster [DBG] osdmap e306: 8 total, 8 up, 8 in 2024-02-23T00:46:19.758 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:18.168446+0000 mon.c (mon.2) 238 : audit [INF] from='client.? 172.21.15.47:0/234645247' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi047-82386-52"}]: dispatch 2024-02-23T00:46:19.759 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:18.169086+0000 mon.a (mon.0) 2054 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi047-82386-52"}]: dispatch 2024-02-23T00:46:19.759 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:18.552531+0000 osd.0 (osd.0) 131 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:46:19.759 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:18.552733+0000 osd.0 (osd.0) 132 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:46:19.813 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:46:19.816 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:19.816 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:46:19.917 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:46:19.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:17.784322+0000 osd.1 (osd.1) 141 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:46:19.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:17.784508+0000 osd.1 (osd.1) 142 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:46:19.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:18.155592+0000 mon.a (mon.0) 2051 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:46:19.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:18.164801+0000 mon.a (mon.0) 2052 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-37"}]': finished 2024-02-23T00:46:19.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:18.164912+0000 mon.a (mon.0) 2053 : cluster [DBG] osdmap e306: 8 total, 8 up, 8 in 2024-02-23T00:46:19.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:18.168446+0000 mon.c (mon.2) 238 : audit [INF] from='client.? 172.21.15.47:0/234645247' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi047-82386-52"}]: dispatch 2024-02-23T00:46:19.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:18.169086+0000 mon.a (mon.0) 2054 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi047-82386-52"}]: dispatch 2024-02-23T00:46:19.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:18.552531+0000 osd.0 (osd.0) 131 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:46:19.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:18.552733+0000 osd.0 (osd.0) 132 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:46:19.947 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:20.807 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:46:20] "GET /metrics HTTP/1.1" 200 190316 "" "Prometheus/2.33.4" 2024-02-23T00:46:20.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:19.164420+0000 mon.a (mon.0) 2055 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi047-82386-52"}]': finished 2024-02-23T00:46:20.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:19.164532+0000 mon.a (mon.0) 2056 : cluster [DBG] osdmap e307: 8 total, 8 up, 8 in 2024-02-23T00:46:20.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:19.173323+0000 mon.c (mon.2) 239 : audit [INF] from='client.? 172.21.15.47:0/234645247' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi047-82386-52"}]: dispatch 2024-02-23T00:46:20.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:19.239538+0000 mon.b (mon.1) 340 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:20.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:19.255839+0000 mgr.y (mgr.14517) 307 : cluster [DBG] pgmap v436: 210 pgs: 1 clean+premerge+peered, 209 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:20.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:19.619692+0000 mon.a (mon.0) 2057 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "23"}]: dispatch 2024-02-23T00:46:20.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:19.619879+0000 mon.a (mon.0) 2058 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi047-82386-52"}]: dispatch 2024-02-23T00:46:20.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:20.015312+0000 mon.c (mon.2) 240 : audit [DBG] from='client.? 172.21.15.47:0/2630437159' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:46:20.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:20.170285+0000 mon.a (mon.0) 2059 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "23"}]': finished 2024-02-23T00:46:20.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:20.170365+0000 mon.a (mon.0) 2060 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi047-82386-52"}]': finished 2024-02-23T00:46:20.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:20.170436+0000 mon.a (mon.0) 2061 : cluster [DBG] osdmap e308: 8 total, 8 up, 8 in 2024-02-23T00:46:20.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:20.173772+0000 mon.a (mon.0) 2062 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:46:20.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:20.197411+0000 mon.a (mon.0) 2063 : audit [INF] from='client.? 172.21.15.47:0/1977736697' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi047-82386-53"}]: dispatch 2024-02-23T00:46:20.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:20.198398+0000 mon.a (mon.0) 2064 : audit [INF] from='client.? 172.21.15.47:0/1977736697' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi047-82386-53"}]: dispatch 2024-02-23T00:46:20.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:20.198751+0000 mon.a (mon.0) 2065 : audit [INF] from='client.? 172.21.15.47:0/1977736697' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi047-82386-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:20.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:20.240459+0000 mon.b (mon.1) 341 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:20.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:19.164420+0000 mon.a (mon.0) 2055 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi047-82386-52"}]': finished 2024-02-23T00:46:20.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:19.164532+0000 mon.a (mon.0) 2056 : cluster [DBG] osdmap e307: 8 total, 8 up, 8 in 2024-02-23T00:46:20.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:19.173323+0000 mon.c (mon.2) 239 : audit [INF] from='client.? 172.21.15.47:0/234645247' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi047-82386-52"}]: dispatch 2024-02-23T00:46:20.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:19.239538+0000 mon.b (mon.1) 340 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:20.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:19.255839+0000 mgr.y (mgr.14517) 307 : cluster [DBG] pgmap v436: 210 pgs: 1 clean+premerge+peered, 209 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:20.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:19.619692+0000 mon.a (mon.0) 2057 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "23"}]: dispatch 2024-02-23T00:46:20.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:19.619879+0000 mon.a (mon.0) 2058 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi047-82386-52"}]: dispatch 2024-02-23T00:46:20.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:20.015312+0000 mon.c (mon.2) 240 : audit [DBG] from='client.? 172.21.15.47:0/2630437159' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:46:20.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:20.170285+0000 mon.a (mon.0) 2059 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "23"}]': finished 2024-02-23T00:46:20.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:20.170365+0000 mon.a (mon.0) 2060 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi047-82386-52"}]': finished 2024-02-23T00:46:20.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:20.170436+0000 mon.a (mon.0) 2061 : cluster [DBG] osdmap e308: 8 total, 8 up, 8 in 2024-02-23T00:46:20.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:20.173772+0000 mon.a (mon.0) 2062 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:46:20.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:20.197411+0000 mon.a (mon.0) 2063 : audit [INF] from='client.? 172.21.15.47:0/1977736697' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi047-82386-53"}]: dispatch 2024-02-23T00:46:20.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:20.198398+0000 mon.a (mon.0) 2064 : audit [INF] from='client.? 172.21.15.47:0/1977736697' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi047-82386-53"}]: dispatch 2024-02-23T00:46:20.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:20.198751+0000 mon.a (mon.0) 2065 : audit [INF] from='client.? 172.21.15.47:0/1977736697' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi047-82386-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:20.810 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:20.240459+0000 mon.b (mon.1) 341 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:20.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.010+0000 7f2ff978b700 1 -- 172.21.15.47:0/1507686982 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2ff40fd510 msgr2=0x7f2ff40f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:20.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.010+0000 7f2ff978b700 1 --2- 172.21.15.47:0/1507686982 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2ff40fd510 0x7f2ff40f6d40 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 rx=0x7f2fdc004870 tx=0x7f2fdc009cb0).stop 2024-02-23T00:46:20.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.010+0000 7f2ff978b700 1 -- 172.21.15.47:0/1507686982 shutdown_connections 2024-02-23T00:46:20.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.010+0000 7f2ff978b700 1 --2- 172.21.15.47:0/1507686982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ff40f7400 0x7f2ff4103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:20.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.010+0000 7f2ff978b700 1 --2- 172.21.15.47:0/1507686982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ff40fcbb0 0x7f2ff40fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:20.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.010+0000 7f2ff978b700 1 --2- 172.21.15.47:0/1507686982 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2ff40fd510 0x7f2ff40f6d40 unknown :-1 s=CLOSED pgs=2608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:20.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.010+0000 7f2ff978b700 1 -- 172.21.15.47:0/1507686982 >> 172.21.15.47:0/1507686982 conn(0x7f2ff40f2820 msgr2=0x7f2ff40f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:46:20.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.010+0000 7f2ff978b700 1 -- 172.21.15.47:0/1507686982 shutdown_connections 2024-02-23T00:46:20.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.010+0000 7f2ff978b700 1 -- 172.21.15.47:0/1507686982 wait complete. 2024-02-23T00:46:20.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.011+0000 7f2ff978b700 1 librados: starting msgr at 2024-02-23T00:46:20.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.011+0000 7f2ff978b700 1 librados: starting objecter 2024-02-23T00:46:20.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.011+0000 7f2ff978b700 1 Processor -- start 2024-02-23T00:46:20.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.011+0000 7f2ff978b700 1 -- start start 2024-02-23T00:46:20.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.011+0000 7f2ff978b700 1 librados: setting wanted keys 2024-02-23T00:46:20.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.011+0000 7f2ff978b700 1 librados: calling monclient init 2024-02-23T00:46:20.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.011+0000 7f2ff978b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2ff40f7400 0x7f2ff40f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:20.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.011+0000 7f2ff978b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ff40fcbb0 0x7f2ff40f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:20.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.011+0000 7f2ff978b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ff40fd510 0x7f2ff40f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:20.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.011+0000 7f2ff978b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff4116570 con 0x7f2ff40fd510 2024-02-23T00:46:20.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.011+0000 7f2ff978b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff41166e0 con 0x7f2ff40fcbb0 2024-02-23T00:46:20.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.011+0000 7f2ff978b700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff41168a0 con 0x7f2ff40f7400 2024-02-23T00:46:20.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.012+0000 7f2ff2ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2ff40f7400 0x7f2ff40f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:20.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.012+0000 7f2ff37fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ff40fd510 0x7f2ff40f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:20.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.012+0000 7f2ff37fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ff40fd510 0x7f2ff40f9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46520/0 (socket says 172.21.15.47:46520) 2024-02-23T00:46:20.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.012+0000 7f2ff37fe700 1 -- 172.21.15.47:0/2630437159 learned_addr learned my addr 172.21.15.47:0/2630437159 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:46:20.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.012+0000 7f2ff27fc700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ff40fcbb0 0x7f2ff40f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:20.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.012+0000 7f2ff2ffd700 1 -- 172.21.15.47:0/2630437159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ff40fcbb0 msgr2=0x7f2ff40f8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:20.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.012+0000 7f2ff2ffd700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ff40fcbb0 0x7f2ff40f8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:20.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.012+0000 7f2ff2ffd700 1 -- 172.21.15.47:0/2630437159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ff40fd510 msgr2=0x7f2ff40f9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:20.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.012+0000 7f2ff2ffd700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ff40fd510 0x7f2ff40f9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:20.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.012+0000 7f2ff2ffd700 1 -- 172.21.15.47:0/2630437159 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe4005040 con 0x7f2ff40f7400 2024-02-23T00:46:20.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.012+0000 7f2ff37fe700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ff40fd510 0x7f2ff40f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:46:20.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.012+0000 7f2ff2ffd700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2ff40f7400 0x7f2ff40f89b0 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 rx=0x7f2fe4004920 tx=0x7f2fe4009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:20.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.012+0000 7f2febfff700 10 client.?.objecter ms_handle_connect 0x7f2ff40f7400 2024-02-23T00:46:20.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.012+0000 7f2febfff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:46:20.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.012+0000 7f2febfff700 1 -- 172.21.15.47:0/2630437159 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe400e030 con 0x7f2ff40f7400 2024-02-23T00:46:20.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.012+0000 7f2febfff700 1 -- 172.21.15.47:0/2630437159 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f2fe400f040 con 0x7f2ff40f7400 2024-02-23T00:46:20.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.012+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fdc005040 con 0x7f2ff40f7400 2024-02-23T00:46:20.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.012+0000 7f2febfff700 1 -- 172.21.15.47:0/2630437159 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe4014720 con 0x7f2ff40f7400 2024-02-23T00:46:20.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.012+0000 7f2ff978b700 10 client.41069.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:46:20.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.012+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff4116f30 con 0x7f2ff40f7400 2024-02-23T00:46:20.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.012+0000 7f2ff978b700 1 librados: init done 2024-02-23T00:46:20.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.013+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f2ff405fa60 con 0x7f2ff40f7400 2024-02-23T00:46:20.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.013+0000 7f2febfff700 1 -- 172.21.15.47:0/2630437159 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f2fe4010500 con 0x7f2ff40f7400 2024-02-23T00:46:20.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.014+0000 7f2febfff700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2fe0062cc0 0x7f2fe0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:20.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.014+0000 7f2febfff700 1 -- 172.21.15.47:0/2630437159 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(307..307 src has 48..307) v4 ==== 9284+0+0 (secure 0 0 0) 0x7f2fe407d6a0 con 0x7f2ff40f7400 2024-02-23T00:46:20.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.014+0000 7f2febfff700 10 client.41069.objecter ms_dispatch 0x7f2ff4001550 osd_map(307..307 src has 48..307) v4 2024-02-23T00:46:20.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.014+0000 7f2febfff700 3 client.41069.objecter handle_osd_map got epochs [307,307] > 0 2024-02-23T00:46:20.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.014+0000 7f2febfff700 3 client.41069.objecter handle_osd_map decoding full epoch 307 2024-02-23T00:46:20.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.014+0000 7f2ff27fc700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2fe0062cc0 0x7f2fe0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:20.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.014+0000 7f2febfff700 20 client.41069.objecter dump_active .. 0 homeless 2024-02-23T00:46:20.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.014+0000 7f2febfff700 1 -- 172.21.15.47:0/2630437159 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v307) v1 ==== 54+0+16 (secure 0 0 0) 0x7f2fe4084070 con 0x7f2ff40f7400 2024-02-23T00:46:20.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.014+0000 7f2febfff700 10 client.41069.objecter ms_handle_connect 0x7f2fe0062cc0 2024-02-23T00:46:20.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.014+0000 7f2ff978b700 10 client.41069.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:20.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.014+0000 7f2ff27fc700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2fe0062cc0 0x7f2fe0065180 secure :-1 s=READY pgs=3017 cs=0 l=1 rev1=1 rx=0x7f2ff40fa150 tx=0x7f2fdc009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:20.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.014+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_assign -1 1 2024-02-23T00:46:20.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.014+0000 7f2ff978b700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f2ff40ffed0 0x7f2ff41002f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:20.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.014+0000 7f2ff978b700 20 client.41069.objecter _get_session s=0x7f2ff405ee80 osd=0 3 2024-02-23T00:46:20.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.014+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff405ee80 osd=0 3 2024-02-23T00:46:20.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.014+0000 7f2ff978b700 20 client.41069.objecter _get_session s=0x7f2ff405ee80 osd=0 3 2024-02-23T00:46:20.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.014+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_remove -1 1 2024-02-23T00:46:20.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.014+0000 7f2ff978b700 20 client.41069.objecter get_session s=0x7f2ff405ee80 osd=0 3 2024-02-23T00:46:20.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.014+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_assign 0 1 2024-02-23T00:46:20.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.014+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff405ee80 osd=0 4 2024-02-23T00:46:20.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.014+0000 7f2ff978b700 10 client.41069.objecter _send_command 1 2024-02-23T00:46:20.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.014+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ff4105ef0 con 0x7f2ff40ffed0 2024-02-23T00:46:20.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.015+0000 7f2ff37fe700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f2ff40ffed0 0x7f2ff41002f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:20.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.015+0000 7f2febfff700 10 client.41069.objecter ms_handle_connect 0x7f2ff40ffed0 2024-02-23T00:46:20.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.015+0000 7f2ff37fe700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f2ff40ffed0 0x7f2ff41002f0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:20.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.015+0000 7f2febfff700 1 -- 172.21.15.47:0/2630437159 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2ff4105ef0 con 0x7f2ff40ffed0 2024-02-23T00:46:20.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.015+0000 7f2febfff700 10 client.41069.objecter ms_dispatch 0x7f2ff4001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:46:20.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.015+0000 7f2febfff700 10 client.41069.objecter _finish_command 1 = system:0 2024-02-23T00:46:20.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.015+0000 7f2febfff700 20 client.41069.objecter put_session s=0x7f2ff405ee80 osd=0 4 2024-02-23T00:46:20.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.015+0000 7f2febfff700 15 client.41069.objecter _session_command_op_remove 0 1 2024-02-23T00:46:20.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.030+0000 7f2ff978b700 10 client.41069.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:46:20.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.030+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_assign -1 2 2024-02-23T00:46:20.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.030+0000 7f2ff978b700 20 client.41069.objecter _get_session s=0x7f2ff405ee80 osd=0 3 2024-02-23T00:46:20.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.030+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff405ee80 osd=0 3 2024-02-23T00:46:20.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.030+0000 7f2ff978b700 20 client.41069.objecter _get_session s=0x7f2ff405ee80 osd=0 3 2024-02-23T00:46:20.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.030+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_remove -1 2 2024-02-23T00:46:20.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.030+0000 7f2ff978b700 20 client.41069.objecter get_session s=0x7f2ff405ee80 osd=0 3 2024-02-23T00:46:20.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.030+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_assign 0 2 2024-02-23T00:46:20.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.030+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff405ee80 osd=0 4 2024-02-23T00:46:20.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.030+0000 7f2ff978b700 10 client.41069.objecter _send_command 2 2024-02-23T00:46:20.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.030+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2ff40273c0 con 0x7f2ff40ffed0 2024-02-23T00:46:20.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.032+0000 7f2febfff700 1 -- 172.21.15.47:0/2630437159 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2ff40273c0 con 0x7f2ff40ffed0 2024-02-23T00:46:20.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.032+0000 7f2febfff700 10 client.41069.objecter ms_dispatch 0x7f2ff4001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:46:20.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.032+0000 7f2febfff700 10 client.41069.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:46:20.834 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:46:20.835 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:46:20.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.032+0000 7f2febfff700 20 client.41069.objecter put_session s=0x7f2ff405ee80 osd=0 4 2024-02-23T00:46:20.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.032+0000 7f2febfff700 15 client.41069.objecter _session_command_op_remove 0 2 2024-02-23T00:46:20.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.032+0000 7f2ff978b700 10 client.41069.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:20.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.032+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_assign -1 3 2024-02-23T00:46:20.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.032+0000 7f2ff978b700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f2ff40598f0 0x7f2ff4059cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:20.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.032+0000 7f2ff978b700 20 client.41069.objecter _get_session s=0x7f2ff4028740 osd=1 3 2024-02-23T00:46:20.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.032+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff4028740 osd=1 3 2024-02-23T00:46:20.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.032+0000 7f2ff978b700 20 client.41069.objecter _get_session s=0x7f2ff4028740 osd=1 3 2024-02-23T00:46:20.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.032+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_remove -1 3 2024-02-23T00:46:20.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.032+0000 7f2ff978b700 20 client.41069.objecter get_session s=0x7f2ff4028740 osd=1 3 2024-02-23T00:46:20.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.032+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_assign 1 3 2024-02-23T00:46:20.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.032+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff4028740 osd=1 4 2024-02-23T00:46:20.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.032+0000 7f2ff978b700 10 client.41069.objecter _send_command 3 2024-02-23T00:46:20.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.032+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ff411d840 con 0x7f2ff40598f0 2024-02-23T00:46:20.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.032+0000 7f2ff2ffd700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f2ff40598f0 0x7f2ff4059cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:20.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.033+0000 7f2febfff700 10 client.41069.objecter ms_handle_connect 0x7f2ff40598f0 2024-02-23T00:46:20.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.033+0000 7f2ff2ffd700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f2ff40598f0 0x7f2ff4059cf0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:20.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.033+0000 7f2febfff700 1 -- 172.21.15.47:0/2630437159 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2ff411d840 con 0x7f2ff40598f0 2024-02-23T00:46:20.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.033+0000 7f2febfff700 10 client.41069.objecter ms_dispatch 0x7f2ff4001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:46:20.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.033+0000 7f2febfff700 10 client.41069.objecter _finish_command 3 = system:0 2024-02-23T00:46:20.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.033+0000 7f2febfff700 20 client.41069.objecter put_session s=0x7f2ff4028740 osd=1 4 2024-02-23T00:46:20.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.033+0000 7f2febfff700 15 client.41069.objecter _session_command_op_remove 1 3 2024-02-23T00:46:20.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.048+0000 7f2ff978b700 10 client.41069.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:46:20.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.048+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_assign -1 4 2024-02-23T00:46:20.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.048+0000 7f2ff978b700 20 client.41069.objecter _get_session s=0x7f2ff4028740 osd=1 3 2024-02-23T00:46:20.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.048+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff4028740 osd=1 3 2024-02-23T00:46:20.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.048+0000 7f2ff978b700 20 client.41069.objecter _get_session s=0x7f2ff4028740 osd=1 3 2024-02-23T00:46:20.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.048+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_remove -1 4 2024-02-23T00:46:20.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.048+0000 7f2ff978b700 20 client.41069.objecter get_session s=0x7f2ff4028740 osd=1 3 2024-02-23T00:46:20.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.048+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_assign 1 4 2024-02-23T00:46:20.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.048+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff4028740 osd=1 4 2024-02-23T00:46:20.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.048+0000 7f2ff978b700 10 client.41069.objecter _send_command 4 2024-02-23T00:46:20.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.048+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2ff4059360 con 0x7f2ff40598f0 2024-02-23T00:46:20.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.049+0000 7f2febfff700 1 -- 172.21.15.47:0/2630437159 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2ff4059360 con 0x7f2ff40598f0 2024-02-23T00:46:20.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.050+0000 7f2febfff700 10 client.41069.objecter ms_dispatch 0x7f2ff4001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:46:20.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.050+0000 7f2febfff700 10 client.41069.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:46:20.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.050+0000 7f2febfff700 20 client.41069.objecter put_session s=0x7f2ff4028740 osd=1 4 2024-02-23T00:46:20.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.050+0000 7f2febfff700 15 client.41069.objecter _session_command_op_remove 1 4 2024-02-23T00:46:20.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.050+0000 7f2ff978b700 10 client.41069.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:20.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.050+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_assign -1 5 2024-02-23T00:46:20.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.050+0000 7f2ff978b700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f2ff40282a0 0x7f2ff411c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:20.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.050+0000 7f2ff978b700 20 client.41069.objecter _get_session s=0x7f2ff4059770 osd=2 3 2024-02-23T00:46:20.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.050+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff4059770 osd=2 3 2024-02-23T00:46:20.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.050+0000 7f2ff978b700 20 client.41069.objecter _get_session s=0x7f2ff4059770 osd=2 3 2024-02-23T00:46:20.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.050+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_remove -1 5 2024-02-23T00:46:20.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.050+0000 7f2ff978b700 20 client.41069.objecter get_session s=0x7f2ff4059770 osd=2 3 2024-02-23T00:46:20.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.050+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_assign 2 5 2024-02-23T00:46:20.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.050+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff4059770 osd=2 4 2024-02-23T00:46:20.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.050+0000 7f2ff978b700 10 client.41069.objecter _send_command 5 2024-02-23T00:46:20.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.050+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ff411c7e0 con 0x7f2ff40282a0 2024-02-23T00:46:20.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.050+0000 7f2ff27fc700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f2ff40282a0 0x7f2ff411c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:20.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.050+0000 7f2febfff700 10 client.41069.objecter ms_handle_connect 0x7f2ff40282a0 2024-02-23T00:46:20.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.050+0000 7f2ff27fc700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f2ff40282a0 0x7f2ff411c260 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:20.839 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:46:20.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.051+0000 7f2febfff700 1 -- 172.21.15.47:0/2630437159 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2ff411c7e0 con 0x7f2ff40282a0 2024-02-23T00:46:20.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.051+0000 7f2febfff700 10 client.41069.objecter ms_dispatch 0x7f2ff4001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:46:20.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.051+0000 7f2febfff700 10 client.41069.objecter _finish_command 5 = system:0 2024-02-23T00:46:20.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.051+0000 7f2febfff700 20 client.41069.objecter put_session s=0x7f2ff4059770 osd=2 4 2024-02-23T00:46:20.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.051+0000 7f2febfff700 15 client.41069.objecter _session_command_op_remove 2 5 2024-02-23T00:46:20.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.065+0000 7f2ff978b700 10 client.41069.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:46:20.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.065+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_assign -1 6 2024-02-23T00:46:20.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.065+0000 7f2ff978b700 20 client.41069.objecter _get_session s=0x7f2ff4059770 osd=2 3 2024-02-23T00:46:20.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.065+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff4059770 osd=2 3 2024-02-23T00:46:20.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.065+0000 7f2ff978b700 20 client.41069.objecter _get_session s=0x7f2ff4059770 osd=2 3 2024-02-23T00:46:20.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.065+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_remove -1 6 2024-02-23T00:46:20.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.065+0000 7f2ff978b700 20 client.41069.objecter get_session s=0x7f2ff4059770 osd=2 3 2024-02-23T00:46:20.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.065+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_assign 2 6 2024-02-23T00:46:20.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.065+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff4059770 osd=2 4 2024-02-23T00:46:20.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.065+0000 7f2ff978b700 10 client.41069.objecter _send_command 6 2024-02-23T00:46:20.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.065+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2ff4027800 con 0x7f2ff40282a0 2024-02-23T00:46:20.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.067+0000 7f2febfff700 1 -- 172.21.15.47:0/2630437159 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2ff4027800 con 0x7f2ff40282a0 2024-02-23T00:46:20.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.067+0000 7f2febfff700 10 client.41069.objecter ms_dispatch 0x7f2ff4001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:46:20.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.067+0000 7f2febfff700 10 client.41069.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:46:20.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.067+0000 7f2febfff700 20 client.41069.objecter put_session s=0x7f2ff4059770 osd=2 4 2024-02-23T00:46:20.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.067+0000 7f2febfff700 15 client.41069.objecter _session_command_op_remove 2 6 2024-02-23T00:46:20.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.067+0000 7f2ff978b700 10 client.41069.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:20.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.067+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_assign -1 7 2024-02-23T00:46:20.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.067+0000 7f2ff978b700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f2ff411f9c0 0x7f2ff4121db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:20.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.067+0000 7f2ff978b700 20 client.41069.objecter _get_session s=0x7f2ff4027bb0 osd=3 3 2024-02-23T00:46:20.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.067+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff4027bb0 osd=3 3 2024-02-23T00:46:20.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.067+0000 7f2ff978b700 20 client.41069.objecter _get_session s=0x7f2ff4027bb0 osd=3 3 2024-02-23T00:46:20.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.067+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_remove -1 7 2024-02-23T00:46:20.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.067+0000 7f2ff978b700 20 client.41069.objecter get_session s=0x7f2ff4027bb0 osd=3 3 2024-02-23T00:46:20.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.067+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_assign 3 7 2024-02-23T00:46:20.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.067+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff4027bb0 osd=3 4 2024-02-23T00:46:20.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.067+0000 7f2ff978b700 10 client.41069.objecter _send_command 7 2024-02-23T00:46:20.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.067+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ff4124a00 con 0x7f2ff411f9c0 2024-02-23T00:46:20.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.067+0000 7f2ff37fe700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f2ff411f9c0 0x7f2ff4121db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:20.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.068+0000 7f2febfff700 10 client.41069.objecter ms_handle_connect 0x7f2ff411f9c0 2024-02-23T00:46:20.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.068+0000 7f2ff37fe700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f2ff411f9c0 0x7f2ff4121db0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:20.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.068+0000 7f2febfff700 1 -- 172.21.15.47:0/2630437159 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2ff4124a00 con 0x7f2ff411f9c0 2024-02-23T00:46:20.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.068+0000 7f2febfff700 10 client.41069.objecter ms_dispatch 0x7f2ff4001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:46:20.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.068+0000 7f2febfff700 10 client.41069.objecter _finish_command 7 = system:0 2024-02-23T00:46:20.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.068+0000 7f2febfff700 20 client.41069.objecter put_session s=0x7f2ff4027bb0 osd=3 4 2024-02-23T00:46:20.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.068+0000 7f2febfff700 15 client.41069.objecter _session_command_op_remove 3 7 2024-02-23T00:46:20.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.086+0000 7f2ff978b700 10 client.41069.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:46:20.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.086+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_assign -1 8 2024-02-23T00:46:20.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.086+0000 7f2ff978b700 20 client.41069.objecter _get_session s=0x7f2ff4027bb0 osd=3 3 2024-02-23T00:46:20.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.086+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff4027bb0 osd=3 3 2024-02-23T00:46:20.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.086+0000 7f2ff978b700 20 client.41069.objecter _get_session s=0x7f2ff4027bb0 osd=3 3 2024-02-23T00:46:20.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.086+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_remove -1 8 2024-02-23T00:46:20.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.086+0000 7f2ff978b700 20 client.41069.objecter get_session s=0x7f2ff4027bb0 osd=3 3 2024-02-23T00:46:20.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.086+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_assign 3 8 2024-02-23T00:46:20.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.086+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff4027bb0 osd=3 4 2024-02-23T00:46:20.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.086+0000 7f2ff978b700 10 client.41069.objecter _send_command 8 2024-02-23T00:46:20.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.086+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2ff411cb90 con 0x7f2ff411f9c0 2024-02-23T00:46:20.844 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:46:20.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.087+0000 7f2febfff700 1 -- 172.21.15.47:0/2630437159 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2ff411cb90 con 0x7f2ff411f9c0 2024-02-23T00:46:20.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.087+0000 7f2febfff700 10 client.41069.objecter ms_dispatch 0x7f2ff4001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:46:20.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.087+0000 7f2febfff700 10 client.41069.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:46:20.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.087+0000 7f2febfff700 20 client.41069.objecter put_session s=0x7f2ff4027bb0 osd=3 4 2024-02-23T00:46:20.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.087+0000 7f2febfff700 15 client.41069.objecter _session_command_op_remove 3 8 2024-02-23T00:46:20.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.087+0000 7f2ff978b700 10 client.41069.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:20.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.087+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_assign -1 9 2024-02-23T00:46:20.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.087+0000 7f2ff978b700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2ff411bdc0 0x7f2ff4123420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:20.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.088+0000 7f2ff978b700 20 client.41069.objecter _get_session s=0x7f2ff411cb90 osd=4 3 2024-02-23T00:46:20.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.088+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff411cb90 osd=4 3 2024-02-23T00:46:20.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.088+0000 7f2ff978b700 20 client.41069.objecter _get_session s=0x7f2ff411cb90 osd=4 3 2024-02-23T00:46:20.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.088+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_remove -1 9 2024-02-23T00:46:20.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.088+0000 7f2ff978b700 20 client.41069.objecter get_session s=0x7f2ff411cb90 osd=4 3 2024-02-23T00:46:20.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.088+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_assign 4 9 2024-02-23T00:46:20.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.088+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff411cb90 osd=4 4 2024-02-23T00:46:20.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.088+0000 7f2ff978b700 10 client.41069.objecter _send_command 9 2024-02-23T00:46:20.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.088+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ff411cf70 con 0x7f2ff411bdc0 2024-02-23T00:46:20.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.088+0000 7f2ff2ffd700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2ff411bdc0 0x7f2ff4123420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:20.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.088+0000 7f2febfff700 10 client.41069.objecter ms_handle_connect 0x7f2ff411bdc0 2024-02-23T00:46:20.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.088+0000 7f2ff2ffd700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2ff411bdc0 0x7f2ff4123420 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:20.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.089+0000 7f2febfff700 1 -- 172.21.15.47:0/2630437159 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2ff411cf70 con 0x7f2ff411bdc0 2024-02-23T00:46:20.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.089+0000 7f2febfff700 10 client.41069.objecter ms_dispatch 0x7f2ff4001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:46:20.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.089+0000 7f2febfff700 10 client.41069.objecter _finish_command 9 = system:0 2024-02-23T00:46:20.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.089+0000 7f2febfff700 20 client.41069.objecter put_session s=0x7f2ff411cb90 osd=4 4 2024-02-23T00:46:20.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.089+0000 7f2febfff700 15 client.41069.objecter _session_command_op_remove 4 9 2024-02-23T00:46:20.846 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:46:20.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.104+0000 7f2ff978b700 10 client.41069.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:46:20.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.104+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_assign -1 10 2024-02-23T00:46:20.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.104+0000 7f2ff978b700 20 client.41069.objecter _get_session s=0x7f2ff411cb90 osd=4 3 2024-02-23T00:46:20.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.104+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff411cb90 osd=4 3 2024-02-23T00:46:20.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.104+0000 7f2ff978b700 20 client.41069.objecter _get_session s=0x7f2ff411cb90 osd=4 3 2024-02-23T00:46:20.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.104+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_remove -1 10 2024-02-23T00:46:20.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.104+0000 7f2ff978b700 20 client.41069.objecter get_session s=0x7f2ff411cb90 osd=4 3 2024-02-23T00:46:20.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.104+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_assign 4 10 2024-02-23T00:46:20.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.104+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff411cb90 osd=4 4 2024-02-23T00:46:20.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.104+0000 7f2ff978b700 10 client.41069.objecter _send_command 10 2024-02-23T00:46:20.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.104+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2ff411cf30 con 0x7f2ff411bdc0 2024-02-23T00:46:20.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.106+0000 7f2febfff700 1 -- 172.21.15.47:0/2630437159 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2ff411cf30 con 0x7f2ff411bdc0 2024-02-23T00:46:20.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.106+0000 7f2febfff700 10 client.41069.objecter ms_dispatch 0x7f2ff4001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:46:20.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.106+0000 7f2febfff700 10 client.41069.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:46:20.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.106+0000 7f2febfff700 20 client.41069.objecter put_session s=0x7f2ff411cb90 osd=4 4 2024-02-23T00:46:20.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.106+0000 7f2febfff700 15 client.41069.objecter _session_command_op_remove 4 10 2024-02-23T00:46:20.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.107+0000 7f2ff978b700 10 client.41069.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:20.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.107+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_assign -1 11 2024-02-23T00:46:20.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.107+0000 7f2ff978b700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f2ff4123e50 0x7f2ff41242d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:20.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.107+0000 7f2ff978b700 20 client.41069.objecter _get_session s=0x7f2ff411cf30 osd=5 3 2024-02-23T00:46:20.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.107+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff411cf30 osd=5 3 2024-02-23T00:46:20.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.107+0000 7f2ff978b700 20 client.41069.objecter _get_session s=0x7f2ff411cf30 osd=5 3 2024-02-23T00:46:20.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.107+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_remove -1 11 2024-02-23T00:46:20.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.107+0000 7f2ff978b700 20 client.41069.objecter get_session s=0x7f2ff411cf30 osd=5 3 2024-02-23T00:46:20.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.107+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_assign 5 11 2024-02-23T00:46:20.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.107+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff411cf30 osd=5 4 2024-02-23T00:46:20.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.107+0000 7f2ff978b700 10 client.41069.objecter _send_command 11 2024-02-23T00:46:20.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.107+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ff412b2f0 con 0x7f2ff4123e50 2024-02-23T00:46:20.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.107+0000 7f2ff27fc700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f2ff4123e50 0x7f2ff41242d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:20.849 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:46:20.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.107+0000 7f2febfff700 10 client.41069.objecter ms_handle_connect 0x7f2ff4123e50 2024-02-23T00:46:20.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.107+0000 7f2ff27fc700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f2ff4123e50 0x7f2ff41242d0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:20.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.108+0000 7f2febfff700 1 -- 172.21.15.47:0/2630437159 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2ff412b2f0 con 0x7f2ff4123e50 2024-02-23T00:46:20.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.108+0000 7f2febfff700 10 client.41069.objecter ms_dispatch 0x7f2ff4001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:46:20.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.108+0000 7f2febfff700 10 client.41069.objecter _finish_command 11 = system:0 2024-02-23T00:46:20.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.108+0000 7f2febfff700 20 client.41069.objecter put_session s=0x7f2ff411cf30 osd=5 4 2024-02-23T00:46:20.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.108+0000 7f2febfff700 15 client.41069.objecter _session_command_op_remove 5 11 2024-02-23T00:46:20.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.123+0000 7f2ff978b700 10 client.41069.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:46:20.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.123+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_assign -1 12 2024-02-23T00:46:20.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.123+0000 7f2ff978b700 20 client.41069.objecter _get_session s=0x7f2ff411cf30 osd=5 3 2024-02-23T00:46:20.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.123+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff411cf30 osd=5 3 2024-02-23T00:46:20.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.123+0000 7f2ff978b700 20 client.41069.objecter _get_session s=0x7f2ff411cf30 osd=5 3 2024-02-23T00:46:20.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.123+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_remove -1 12 2024-02-23T00:46:20.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.123+0000 7f2ff978b700 20 client.41069.objecter get_session s=0x7f2ff411cf30 osd=5 3 2024-02-23T00:46:20.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.123+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_assign 5 12 2024-02-23T00:46:20.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.123+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff411cf30 osd=5 4 2024-02-23T00:46:20.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.123+0000 7f2ff978b700 10 client.41069.objecter _send_command 12 2024-02-23T00:46:20.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.123+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2ff4128b90 con 0x7f2ff4123e50 2024-02-23T00:46:20.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.125+0000 7f2febfff700 1 -- 172.21.15.47:0/2630437159 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2ff4128b90 con 0x7f2ff4123e50 2024-02-23T00:46:20.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.125+0000 7f2febfff700 10 client.41069.objecter ms_dispatch 0x7f2ff4001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:46:20.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.125+0000 7f2febfff700 10 client.41069.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:46:20.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.125+0000 7f2febfff700 20 client.41069.objecter put_session s=0x7f2ff411cf30 osd=5 4 2024-02-23T00:46:20.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.125+0000 7f2febfff700 15 client.41069.objecter _session_command_op_remove 5 12 2024-02-23T00:46:20.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.125+0000 7f2ff978b700 10 client.41069.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:20.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.125+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_assign -1 13 2024-02-23T00:46:20.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.125+0000 7f2ff978b700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f2ff4129240 0x7f2ff412b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:20.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.125+0000 7f2ff978b700 20 client.41069.objecter _get_session s=0x7f2ff4128b90 osd=6 3 2024-02-23T00:46:20.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.125+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff4128b90 osd=6 3 2024-02-23T00:46:20.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.125+0000 7f2ff978b700 20 client.41069.objecter _get_session s=0x7f2ff4128b90 osd=6 3 2024-02-23T00:46:20.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.125+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_remove -1 13 2024-02-23T00:46:20.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.125+0000 7f2ff978b700 20 client.41069.objecter get_session s=0x7f2ff4128b90 osd=6 3 2024-02-23T00:46:20.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.125+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_assign 6 13 2024-02-23T00:46:20.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.125+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff4128b90 osd=6 4 2024-02-23T00:46:20.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.125+0000 7f2ff978b700 10 client.41069.objecter _send_command 13 2024-02-23T00:46:20.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.125+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ff412e4b0 con 0x7f2ff4129240 2024-02-23T00:46:20.852 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:46:20.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.125+0000 7f2ff37fe700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f2ff4129240 0x7f2ff412b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:20.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.125+0000 7f2febfff700 10 client.41069.objecter ms_handle_connect 0x7f2ff4129240 2024-02-23T00:46:20.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.125+0000 7f2ff37fe700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f2ff4129240 0x7f2ff412b630 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:20.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.126+0000 7f2febfff700 1 -- 172.21.15.47:0/2630437159 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2ff412e4b0 con 0x7f2ff4129240 2024-02-23T00:46:20.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.126+0000 7f2febfff700 10 client.41069.objecter ms_dispatch 0x7f2ff4001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:46:20.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.126+0000 7f2febfff700 10 client.41069.objecter _finish_command 13 = system:0 2024-02-23T00:46:20.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.126+0000 7f2febfff700 20 client.41069.objecter put_session s=0x7f2ff4128b90 osd=6 4 2024-02-23T00:46:20.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.126+0000 7f2febfff700 15 client.41069.objecter _session_command_op_remove 6 13 2024-02-23T00:46:20.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.141+0000 7f2ff978b700 10 client.41069.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:46:20.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.141+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_assign -1 14 2024-02-23T00:46:20.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.141+0000 7f2ff978b700 20 client.41069.objecter _get_session s=0x7f2ff4128b90 osd=6 3 2024-02-23T00:46:20.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.141+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff4128b90 osd=6 3 2024-02-23T00:46:20.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.141+0000 7f2ff978b700 20 client.41069.objecter _get_session s=0x7f2ff4128b90 osd=6 3 2024-02-23T00:46:20.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.141+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_remove -1 14 2024-02-23T00:46:20.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.141+0000 7f2ff978b700 20 client.41069.objecter get_session s=0x7f2ff4128b90 osd=6 3 2024-02-23T00:46:20.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.141+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_assign 6 14 2024-02-23T00:46:20.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.141+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff4128b90 osd=6 4 2024-02-23T00:46:20.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.141+0000 7f2ff978b700 10 client.41069.objecter _send_command 14 2024-02-23T00:46:20.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.141+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2ff412e470 con 0x7f2ff4129240 2024-02-23T00:46:20.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.142+0000 7f2febfff700 1 -- 172.21.15.47:0/2630437159 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2ff412e470 con 0x7f2ff4129240 2024-02-23T00:46:20.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.142+0000 7f2febfff700 10 client.41069.objecter ms_dispatch 0x7f2ff4001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:46:20.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.142+0000 7f2febfff700 10 client.41069.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:46:20.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.142+0000 7f2febfff700 20 client.41069.objecter put_session s=0x7f2ff4128b90 osd=6 4 2024-02-23T00:46:20.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.142+0000 7f2febfff700 15 client.41069.objecter _session_command_op_remove 6 14 2024-02-23T00:46:20.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.143+0000 7f2ff978b700 10 client.41069.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:20.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.143+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_assign -1 15 2024-02-23T00:46:20.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.143+0000 7f2ff978b700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f2ff412d750 0x7f2ff412dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:20.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.143+0000 7f2ff978b700 20 client.41069.objecter _get_session s=0x7f2ff412e470 osd=7 3 2024-02-23T00:46:20.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.143+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff412e470 osd=7 3 2024-02-23T00:46:20.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.143+0000 7f2ff978b700 20 client.41069.objecter _get_session s=0x7f2ff412e470 osd=7 3 2024-02-23T00:46:20.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.143+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_remove -1 15 2024-02-23T00:46:20.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.143+0000 7f2ff978b700 20 client.41069.objecter get_session s=0x7f2ff412e470 osd=7 3 2024-02-23T00:46:20.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.143+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_assign 7 15 2024-02-23T00:46:20.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.143+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff412e470 osd=7 4 2024-02-23T00:46:20.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.143+0000 7f2ff978b700 10 client.41069.objecter _send_command 15 2024-02-23T00:46:20.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.143+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ff41317e0 con 0x7f2ff412d750 2024-02-23T00:46:20.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.143+0000 7f2ff2ffd700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f2ff412d750 0x7f2ff412dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:20.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.143+0000 7f2febfff700 10 client.41069.objecter ms_handle_connect 0x7f2ff412d750 2024-02-23T00:46:20.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.143+0000 7f2ff2ffd700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f2ff412d750 0x7f2ff412dbd0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:20.855 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:46:20.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.144+0000 7f2febfff700 1 -- 172.21.15.47:0/2630437159 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2ff41317e0 con 0x7f2ff412d750 2024-02-23T00:46:20.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.144+0000 7f2febfff700 10 client.41069.objecter ms_dispatch 0x7f2ff4001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:46:20.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.144+0000 7f2febfff700 10 client.41069.objecter _finish_command 15 = system:0 2024-02-23T00:46:20.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.144+0000 7f2febfff700 20 client.41069.objecter put_session s=0x7f2ff412e470 osd=7 4 2024-02-23T00:46:20.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.144+0000 7f2febfff700 15 client.41069.objecter _session_command_op_remove 7 15 2024-02-23T00:46:20.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.159+0000 7f2ff978b700 10 client.41069.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:46:20.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.159+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_assign -1 16 2024-02-23T00:46:20.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.159+0000 7f2ff978b700 20 client.41069.objecter _get_session s=0x7f2ff412e470 osd=7 3 2024-02-23T00:46:20.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.159+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff412e470 osd=7 3 2024-02-23T00:46:20.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.159+0000 7f2ff978b700 20 client.41069.objecter _get_session s=0x7f2ff412e470 osd=7 3 2024-02-23T00:46:20.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.159+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_remove -1 16 2024-02-23T00:46:20.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.159+0000 7f2ff978b700 20 client.41069.objecter get_session s=0x7f2ff412e470 osd=7 3 2024-02-23T00:46:20.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.159+0000 7f2ff978b700 15 client.41069.objecter _session_command_op_assign 7 16 2024-02-23T00:46:20.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.159+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff412e470 osd=7 4 2024-02-23T00:46:20.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.159+0000 7f2ff978b700 10 client.41069.objecter _send_command 16 2024-02-23T00:46:20.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.159+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2ff412bc30 con 0x7f2ff412d750 2024-02-23T00:46:20.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.161+0000 7f2febfff700 1 -- 172.21.15.47:0/2630437159 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2ff412bc30 con 0x7f2ff412d750 2024-02-23T00:46:20.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.161+0000 7f2febfff700 10 client.41069.objecter ms_dispatch 0x7f2ff4001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:46:20.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.161+0000 7f2febfff700 10 client.41069.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:46:20.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.161+0000 7f2febfff700 20 client.41069.objecter put_session s=0x7f2ff412e470 osd=7 4 2024-02-23T00:46:20.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.161+0000 7f2febfff700 15 client.41069.objecter _session_command_op_remove 7 16 2024-02-23T00:46:20.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.161+0000 7f2ff978b700 10 librados: watch_flush enter 2024-02-23T00:46:20.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.161+0000 7f2ff978b700 10 librados: watch_flush exit 2024-02-23T00:46:20.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.161+0000 7f2ff978b700 10 client.41069.objecter close_session for osd.0 2024-02-23T00:46:20.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.161+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f2ff40ffed0 msgr2=0x7f2ff41002f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:20.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.161+0000 7f2ff978b700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f2ff40ffed0 0x7f2ff41002f0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:20.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.161+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff405ee80 osd=0 1 2024-02-23T00:46:20.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.161+0000 7f2ff978b700 10 client.41069.objecter close_session for osd.1 2024-02-23T00:46:20.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.161+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f2ff40598f0 msgr2=0x7f2ff4059cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:20.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.161+0000 7f2ff978b700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f2ff40598f0 0x7f2ff4059cf0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:20.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.161+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff4028740 osd=1 1 2024-02-23T00:46:20.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.161+0000 7f2ff978b700 10 client.41069.objecter close_session for osd.2 2024-02-23T00:46:20.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.161+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f2ff40282a0 msgr2=0x7f2ff411c260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:20.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.161+0000 7f2ff978b700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f2ff40282a0 0x7f2ff411c260 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:20.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.161+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff4059770 osd=2 1 2024-02-23T00:46:20.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.161+0000 7f2ff978b700 10 client.41069.objecter close_session for osd.3 2024-02-23T00:46:20.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.161+0000 7f2ff37fe700 1 -- 172.21.15.47:0/2630437159 reap_dead start 2024-02-23T00:46:20.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.161+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f2ff411f9c0 msgr2=0x7f2ff4121db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:20.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.161+0000 7f2ff978b700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f2ff411f9c0 0x7f2ff4121db0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:20.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.162+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff4027bb0 osd=3 1 2024-02-23T00:46:20.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.162+0000 7f2ff978b700 10 client.41069.objecter close_session for osd.4 2024-02-23T00:46:20.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.162+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2ff411bdc0 msgr2=0x7f2ff4123420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:20.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.162+0000 7f2ff978b700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2ff411bdc0 0x7f2ff4123420 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:20.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.162+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff411cb90 osd=4 1 2024-02-23T00:46:20.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.162+0000 7f2ff978b700 10 client.41069.objecter close_session for osd.5 2024-02-23T00:46:20.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.162+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f2ff4123e50 msgr2=0x7f2ff41242d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:20.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.162+0000 7f2ff978b700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f2ff4123e50 0x7f2ff41242d0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:20.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.162+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff411cf30 osd=5 1 2024-02-23T00:46:20.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.162+0000 7f2ff978b700 10 client.41069.objecter close_session for osd.6 2024-02-23T00:46:20.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.162+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f2ff4129240 msgr2=0x7f2ff412b630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:20.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.162+0000 7f2ff978b700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f2ff4129240 0x7f2ff412b630 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:20.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.162+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff4128b90 osd=6 1 2024-02-23T00:46:20.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.162+0000 7f2ff978b700 10 client.41069.objecter close_session for osd.7 2024-02-23T00:46:20.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.162+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f2ff412d750 msgr2=0x7f2ff412dbd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:20.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.162+0000 7f2ff978b700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f2ff412d750 0x7f2ff412dbd0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:20.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.162+0000 7f2ff978b700 20 client.41069.objecter put_session s=0x7f2ff412e470 osd=7 1 2024-02-23T00:46:20.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.162+0000 7f2ff978b700 20 client.41069.objecter shutdown clearing up homeless session... 2024-02-23T00:46:20.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.162+0000 7f2ff978b700 10 client.41069.objecter successfully canceled tick 2024-02-23T00:46:20.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.162+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2fe0062cc0 msgr2=0x7f2fe0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:20.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.162+0000 7f2ff978b700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2fe0062cc0 0x7f2fe0065180 secure :-1 s=READY pgs=3017 cs=0 l=1 rev1=1 rx=0x7f2ff40fa150 tx=0x7f2fdc009cb0).stop 2024-02-23T00:46:20.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.162+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2ff40f7400 msgr2=0x7f2ff40f89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:20.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.162+0000 7f2ff978b700 1 --2- 172.21.15.47:0/2630437159 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2ff40f7400 0x7f2ff40f89b0 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 rx=0x7f2fe4004920 tx=0x7f2fe4009cb0).stop 2024-02-23T00:46:20.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.162+0000 7f2ff37fe700 1 -- 172.21.15.47:0/2630437159 reap_dead start 2024-02-23T00:46:20.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.162+0000 7f2ff37fe700 1 -- 172.21.15.47:0/2630437159 reap_dead start 2024-02-23T00:46:20.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.162+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 shutdown_connections 2024-02-23T00:46:20.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.162+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 >> 172.21.15.47:0/2630437159 conn(0x7f2ff40f2820 msgr2=0x7f2ff4060eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:46:20.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.162+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 shutdown_connections 2024-02-23T00:46:20.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.162+0000 7f2ff978b700 1 -- 172.21.15.47:0/2630437159 wait complete. 2024-02-23T00:46:20.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:20.162+0000 7f2ff978b700 1 librados: shutdown 2024-02-23T00:46:20.860 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:46:20.860 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:46:20.860 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:46:20.860 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:46:20.860 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:46:20.861 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:46:20.861 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:46:20.861 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:46:20.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:19.164420+0000 mon.a (mon.0) 2055 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi047-82386-52"}]': finished 2024-02-23T00:46:20.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:19.164532+0000 mon.a (mon.0) 2056 : cluster [DBG] osdmap e307: 8 total, 8 up, 8 in 2024-02-23T00:46:20.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:19.173323+0000 mon.c (mon.2) 239 : audit [INF] from='client.? 172.21.15.47:0/234645247' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi047-82386-52"}]: dispatch 2024-02-23T00:46:20.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:19.239538+0000 mon.b (mon.1) 340 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:20.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:19.255839+0000 mgr.y (mgr.14517) 307 : cluster [DBG] pgmap v436: 210 pgs: 1 clean+premerge+peered, 209 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:20.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:19.619692+0000 mon.a (mon.0) 2057 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "23"}]: dispatch 2024-02-23T00:46:20.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:19.619879+0000 mon.a (mon.0) 2058 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi047-82386-52"}]: dispatch 2024-02-23T00:46:20.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:20.015312+0000 mon.c (mon.2) 240 : audit [DBG] from='client.? 172.21.15.47:0/2630437159' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:46:20.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:20.170285+0000 mon.a (mon.0) 2059 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "23"}]': finished 2024-02-23T00:46:20.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:20.170365+0000 mon.a (mon.0) 2060 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi047-82386-52"}]': finished 2024-02-23T00:46:20.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:20.170436+0000 mon.a (mon.0) 2061 : cluster [DBG] osdmap e308: 8 total, 8 up, 8 in 2024-02-23T00:46:20.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:20.173772+0000 mon.a (mon.0) 2062 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:46:20.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:20.197411+0000 mon.a (mon.0) 2063 : audit [INF] from='client.? 172.21.15.47:0/1977736697' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi047-82386-53"}]: dispatch 2024-02-23T00:46:20.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:20.198398+0000 mon.a (mon.0) 2064 : audit [INF] from='client.? 172.21.15.47:0/1977736697' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi047-82386-53"}]: dispatch 2024-02-23T00:46:20.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:20.198751+0000 mon.a (mon.0) 2065 : audit [INF] from='client.? 172.21.15.47:0/1977736697' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi047-82386-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:20.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:20.240459+0000 mon.b (mon.1) 341 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:21.097 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:46:21.100 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:21.101 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:46:21.201 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:46:21.232 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:22.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.264+0000 7f2fcd91b700 1 -- 172.21.15.47:0/3941828218 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2fc80f7330 msgr2=0x7f2fc8103930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:22.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.264+0000 7f2fcd91b700 1 --2- 172.21.15.47:0/3941828218 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2fc80f7330 0x7f2fc8103930 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 rx=0x7f2fc0009be0 tx=0x7f2fc0004d10).stop 2024-02-23T00:46:22.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.264+0000 7f2fcd91b700 1 -- 172.21.15.47:0/3941828218 shutdown_connections 2024-02-23T00:46:22.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.264+0000 7f2fcd91b700 1 --2- 172.21.15.47:0/3941828218 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2fc80f7330 0x7f2fc8103930 unknown :-1 s=CLOSED pgs=2611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:22.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.264+0000 7f2fcd91b700 1 --2- 172.21.15.47:0/3941828218 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2fc80fcac0 0x7f2fc80fcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:22.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.264+0000 7f2fcd91b700 1 --2- 172.21.15.47:0/3941828218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fc80fd420 0x7f2fc80f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:22.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.264+0000 7f2fcd91b700 1 -- 172.21.15.47:0/3941828218 >> 172.21.15.47:0/3941828218 conn(0x7f2fc80f26f0 msgr2=0x7f2fc80f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:46:22.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.265+0000 7f2fcd91b700 1 -- 172.21.15.47:0/3941828218 shutdown_connections 2024-02-23T00:46:22.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.265+0000 7f2fcd91b700 1 -- 172.21.15.47:0/3941828218 wait complete. 2024-02-23T00:46:22.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.265+0000 7f2fcd91b700 1 librados: starting msgr at 2024-02-23T00:46:22.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.265+0000 7f2fcd91b700 1 librados: starting objecter 2024-02-23T00:46:22.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.265+0000 7f2fcd91b700 1 Processor -- start 2024-02-23T00:46:22.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.266+0000 7f2fcd91b700 1 -- start start 2024-02-23T00:46:22.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.266+0000 7f2fcd91b700 1 librados: setting wanted keys 2024-02-23T00:46:22.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.266+0000 7f2fcd91b700 1 librados: calling monclient init 2024-02-23T00:46:22.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.266+0000 7f2fcd91b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2fc80f7330 0x7f2fc8108e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:22.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.266+0000 7f2fcd91b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2fc80fcac0 0x7f2fc8109340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:22.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.266+0000 7f2fcd91b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fc80fd420 0x7f2fc8109900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:22.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.266+0000 7f2fcd91b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc8114040 con 0x7f2fc80fd420 2024-02-23T00:46:22.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.266+0000 7f2fcd91b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc81141b0 con 0x7f2fc80f7330 2024-02-23T00:46:22.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.266+0000 7f2fcd91b700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc8114370 con 0x7f2fc80fcac0 2024-02-23T00:46:22.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.266+0000 7f2fc67fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2fc80fcac0 0x7f2fc8109340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:22.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.266+0000 7f2fc67fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2fc80fcac0 0x7f2fc8109340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:59030/0 (socket says 172.21.15.47:59030) 2024-02-23T00:46:22.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.266+0000 7f2fc67fc700 1 -- 172.21.15.47:0/3456758300 learned_addr learned my addr 172.21.15.47:0/3456758300 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:46:22.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.266+0000 7f2fc77fe700 1 --2- 172.21.15.47:0/3456758300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fc80fd420 0x7f2fc8109900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:22.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.266+0000 7f2fc6ffd700 1 --2- 172.21.15.47:0/3456758300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2fc80f7330 0x7f2fc8108e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:22.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.266+0000 7f2fc67fc700 1 -- 172.21.15.47:0/3456758300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2fc80f7330 msgr2=0x7f2fc8108e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:22.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.266+0000 7f2fc67fc700 1 --2- 172.21.15.47:0/3456758300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2fc80f7330 0x7f2fc8108e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:22.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.266+0000 7f2fc67fc700 1 -- 172.21.15.47:0/3456758300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fc80fd420 msgr2=0x7f2fc8109900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:22.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.266+0000 7f2fc67fc700 1 --2- 172.21.15.47:0/3456758300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fc80fd420 0x7f2fc8109900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:22.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.266+0000 7f2fc67fc700 1 -- 172.21.15.47:0/3456758300 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc00097c0 con 0x7f2fc80fcac0 2024-02-23T00:46:22.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.266+0000 7f2fc6ffd700 1 --2- 172.21.15.47:0/3456758300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2fc80f7330 0x7f2fc8108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:46:22.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.267+0000 7f2fc67fc700 1 --2- 172.21.15.47:0/3456758300 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2fc80fcac0 0x7f2fc8109340 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 rx=0x7f2fbc00e970 tx=0x7f2fbc00b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:22.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.267+0000 7f2faffff700 10 client.?.objecter ms_handle_connect 0x7f2fc80fcac0 2024-02-23T00:46:22.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.267+0000 7f2faffff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:46:22.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.267+0000 7f2faffff700 1 -- 172.21.15.47:0/3456758300 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc00ee60 con 0x7f2fc80fcac0 2024-02-23T00:46:22.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.267+0000 7f2faffff700 1 -- 172.21.15.47:0/3456758300 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f2fbc00f040 con 0x7f2fc80fcac0 2024-02-23T00:46:22.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.267+0000 7f2fcd91b700 1 -- 172.21.15.47:0/3456758300 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc8114650 con 0x7f2fc80fcac0 2024-02-23T00:46:22.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.267+0000 7f2faffff700 1 -- 172.21.15.47:0/3456758300 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc00c5b0 con 0x7f2fc80fcac0 2024-02-23T00:46:22.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.267+0000 7f2fcd91b700 10 client.41081.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:46:22.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.267+0000 7f2fcd91b700 1 -- 172.21.15.47:0/3456758300 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc8114c10 con 0x7f2fc80fcac0 2024-02-23T00:46:22.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.267+0000 7f2fcd91b700 1 librados: init done 2024-02-23T00:46:22.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.268+0000 7f2fcd91b700 1 -- 172.21.15.47:0/3456758300 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fc805f990 con 0x7f2fc80fcac0 2024-02-23T00:46:22.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.269+0000 7f2faffff700 1 -- 172.21.15.47:0/3456758300 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f2fbc009090 con 0x7f2fc80fcac0 2024-02-23T00:46:22.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.269+0000 7f2faffff700 1 --2- 172.21.15.47:0/3456758300 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2fb0062cc0 0x7f2fb0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:22.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.270+0000 7f2faffff700 1 -- 172.21.15.47:0/3456758300 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(310..310 src has 48..310) v4 ==== 9960+0+0 (secure 0 0 0) 0x7f2fbc07d630 con 0x7f2fc80fcac0 2024-02-23T00:46:22.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.270+0000 7f2faffff700 10 client.41081.objecter ms_dispatch 0x7f2fc8001550 osd_map(310..310 src has 48..310) v4 2024-02-23T00:46:22.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.270+0000 7f2faffff700 3 client.41081.objecter handle_osd_map got epochs [310,310] > 0 2024-02-23T00:46:22.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.270+0000 7f2faffff700 3 client.41081.objecter handle_osd_map decoding full epoch 310 2024-02-23T00:46:22.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.270+0000 7f2fc6ffd700 1 --2- 172.21.15.47:0/3456758300 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2fb0062cc0 0x7f2fb0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:22.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.270+0000 7f2faffff700 20 client.41081.objecter dump_active .. 0 homeless 2024-02-23T00:46:22.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.270+0000 7f2faffff700 10 client.41081.objecter ms_handle_connect 0x7f2fb0062cc0 2024-02-23T00:46:22.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.270+0000 7f2fc6ffd700 1 --2- 172.21.15.47:0/3456758300 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2fb0062cc0 0x7f2fb0065180 secure :-1 s=READY pgs=3019 cs=0 l=1 rev1=1 rx=0x7f2fb8007fd0 tx=0x7f2fb800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:22.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.273+0000 7f2faffff700 1 -- 172.21.15.47:0/3456758300 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f2fbc04af60 con 0x7f2fc80fcac0 2024-02-23T00:46:22.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:21.172616+0000 mon.a (mon.0) 2066 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:46:22.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:21.172772+0000 mon.a (mon.0) 2067 : audit [INF] from='client.? 172.21.15.47:0/1977736697' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi047-82386-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:22.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:21.172841+0000 mon.a (mon.0) 2068 : cluster [DBG] osdmap e309: 8 total, 8 up, 8 in 2024-02-23T00:46:22.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:21.177730+0000 mon.a (mon.0) 2069 : audit [INF] from='client.? 172.21.15.47:0/1977736697' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi047-82386-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi047-82386-53"}]: dispatch 2024-02-23T00:46:22.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:21.181143+0000 mon.a (mon.0) 2070 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:46:22.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:21.241494+0000 mon.b (mon.1) 342 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:22.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:21.256436+0000 mgr.y (mgr.14517) 308 : cluster [DBG] pgmap v439: 242 pgs: 32 unknown, 1 clean+premerge+peered, 209 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:46:22.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:21.172616+0000 mon.a (mon.0) 2066 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:46:22.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:21.172772+0000 mon.a (mon.0) 2067 : audit [INF] from='client.? 172.21.15.47:0/1977736697' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi047-82386-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:22.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:21.172841+0000 mon.a (mon.0) 2068 : cluster [DBG] osdmap e309: 8 total, 8 up, 8 in 2024-02-23T00:46:22.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:21.177730+0000 mon.a (mon.0) 2069 : audit [INF] from='client.? 172.21.15.47:0/1977736697' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi047-82386-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi047-82386-53"}]: dispatch 2024-02-23T00:46:22.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:21.181143+0000 mon.a (mon.0) 2070 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:46:22.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:21.241494+0000 mon.b (mon.1) 342 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:22.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:21.256436+0000 mgr.y (mgr.14517) 308 : cluster [DBG] pgmap v439: 242 pgs: 32 unknown, 1 clean+premerge+peered, 209 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:46:22.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:21.172616+0000 mon.a (mon.0) 2066 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:46:22.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:21.172772+0000 mon.a (mon.0) 2067 : audit [INF] from='client.? 172.21.15.47:0/1977736697' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi047-82386-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:22.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:21.172841+0000 mon.a (mon.0) 2068 : cluster [DBG] osdmap e309: 8 total, 8 up, 8 in 2024-02-23T00:46:22.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:21.177730+0000 mon.a (mon.0) 2069 : audit [INF] from='client.? 172.21.15.47:0/1977736697' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi047-82386-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi047-82386-53"}]: dispatch 2024-02-23T00:46:22.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:21.181143+0000 mon.a (mon.0) 2070 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:46:22.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:21.241494+0000 mon.b (mon.1) 342 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:22.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:21.256436+0000 mgr.y (mgr.14517) 308 : cluster [DBG] pgmap v439: 242 pgs: 32 unknown, 1 clean+premerge+peered, 209 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:46:22.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:22.460+0000 7f2fcd91b700 1 -- 172.21.15.47:0/3456758300 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7f2fc805ee80 con 0x7f2fc80fcac0 2024-02-23T00:46:22.943 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:46:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:46:22.943 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:46:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:46:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:46:23.151 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:46:23.175 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.3 dump_blocked_ops 2024-02-23T00:46:23.184 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:46:23.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:23.183+0000 7f2faffff700 1 -- 172.21.15.47:0/3456758300 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v311) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2fbc019070 con 0x7f2fc80fcac0 2024-02-23T00:46:23.191 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:23.192 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:46:23.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:23.188+0000 7f2fcd91b700 10 librados: watch_flush enter 2024-02-23T00:46:23.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:23.189+0000 7f2fcd91b700 10 librados: watch_flush exit 2024-02-23T00:46:23.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:23.189+0000 7f2fcd91b700 20 client.41081.objecter shutdown clearing up homeless session... 2024-02-23T00:46:23.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:23.189+0000 7f2fcd91b700 10 client.41081.objecter successfully canceled tick 2024-02-23T00:46:23.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:23.189+0000 7f2fcd91b700 1 -- 172.21.15.47:0/3456758300 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2fb0062cc0 msgr2=0x7f2fb0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:23.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:23.189+0000 7f2fcd91b700 1 --2- 172.21.15.47:0/3456758300 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2fb0062cc0 0x7f2fb0065180 secure :-1 s=READY pgs=3019 cs=0 l=1 rev1=1 rx=0x7f2fb8007fd0 tx=0x7f2fb800b040).stop 2024-02-23T00:46:23.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:23.189+0000 7f2fcd91b700 1 -- 172.21.15.47:0/3456758300 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2fc80fcac0 msgr2=0x7f2fc8109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:23.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:23.189+0000 7f2fcd91b700 1 --2- 172.21.15.47:0/3456758300 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2fc80fcac0 0x7f2fc8109340 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 rx=0x7f2fbc00e970 tx=0x7f2fbc00b040).stop 2024-02-23T00:46:23.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:23.193+0000 7f2fcd91b700 1 -- 172.21.15.47:0/3456758300 shutdown_connections 2024-02-23T00:46:23.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:23.193+0000 7f2fcd91b700 1 --2- 172.21.15.47:0/3456758300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fc80fd420 0x7f2fc8109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:23.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:23.193+0000 7f2fcd91b700 1 --2- 172.21.15.47:0/3456758300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2fc80f7330 0x7f2fc8108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:23.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:23.193+0000 7f2fcd91b700 1 --2- 172.21.15.47:0/3456758300 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2fb0062cc0 0x7f2fb0065180 unknown :-1 s=CLOSED pgs=3019 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:23.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:23.193+0000 7f2fcd91b700 1 --2- 172.21.15.47:0/3456758300 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2fc80fcac0 0x7f2fc8109340 unknown :-1 s=CLOSED pgs=2612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:23.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:23.193+0000 7f2fcd91b700 1 -- 172.21.15.47:0/3456758300 >> 172.21.15.47:0/3456758300 conn(0x7f2fc80f26f0 msgr2=0x7f2fc80f4080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:46:23.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:23.195+0000 7f2fcd91b700 1 -- 172.21.15.47:0/3456758300 shutdown_connections 2024-02-23T00:46:23.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:23.195+0000 7f2fcd91b700 1 -- 172.21.15.47:0/3456758300 wait complete. 2024-02-23T00:46:23.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:23.195+0000 7f2fcd91b700 1 librados: shutdown 2024-02-23T00:46:23.200 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is set 2024-02-23T00:46:23.292 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:46:23.323 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:23.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:21.900532+0000 osd.2 (osd.2) 141 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:46:23.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:21.900749+0000 osd.2 (osd.2) 142 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:46:23.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:22.242701+0000 mon.b (mon.1) 343 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:23.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:22.461789+0000 mon.c (mon.2) 241 : audit [INF] from='client.? 172.21.15.47:0/3456758300' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:46:23.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:22.499443+0000 mgr.y (mgr.14517) 309 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:46:23.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:22.615875+0000 mon.a (mon.0) 2071 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:46:23.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:22.615956+0000 mon.a (mon.0) 2072 : cluster [DBG] osdmap e310: 8 total, 8 up, 8 in 2024-02-23T00:46:23.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:22.616962+0000 mon.a (mon.0) 2073 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-39","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:46:23.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:22.617239+0000 mon.a (mon.0) 2074 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:46:23.466 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:21.900532+0000 osd.2 (osd.2) 141 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:46:23.466 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:21.900749+0000 osd.2 (osd.2) 142 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:46:23.466 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:22.242701+0000 mon.b (mon.1) 343 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:23.466 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:22.461789+0000 mon.c (mon.2) 241 : audit [INF] from='client.? 172.21.15.47:0/3456758300' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:46:23.466 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:22.499443+0000 mgr.y (mgr.14517) 309 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:46:23.466 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:22.615875+0000 mon.a (mon.0) 2071 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:46:23.466 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:22.615956+0000 mon.a (mon.0) 2072 : cluster [DBG] osdmap e310: 8 total, 8 up, 8 in 2024-02-23T00:46:23.466 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:22.616962+0000 mon.a (mon.0) 2073 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-39","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:46:23.467 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:22.617239+0000 mon.a (mon.0) 2074 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:46:23.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:21.900532+0000 osd.2 (osd.2) 141 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:46:23.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:21.900749+0000 osd.2 (osd.2) 142 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:46:23.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:22.242701+0000 mon.b (mon.1) 343 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:23.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:22.461789+0000 mon.c (mon.2) 241 : audit [INF] from='client.? 172.21.15.47:0/3456758300' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:46:23.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:22.499443+0000 mgr.y (mgr.14517) 309 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:46:23.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:22.615875+0000 mon.a (mon.0) 2071 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:46:23.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:22.615956+0000 mon.a (mon.0) 2072 : cluster [DBG] osdmap e310: 8 total, 8 up, 8 in 2024-02-23T00:46:23.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:22.616962+0000 mon.a (mon.0) 2073 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-39","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:46:23.468 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:22.617239+0000 mon.a (mon.0) 2074 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:46:23.475 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:46:23.478 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:23.478 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:46:23.579 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:46:23.611 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:23.633 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:46:23.635 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:23.636 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:46:23.736 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:46:23.766 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:23.786 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:46:23.789 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:23.789 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:46:23.890 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:46:23.920 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:24.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:23.174768+0000 mon.a (mon.0) 2075 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:46:24.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:23.184185+0000 mon.a (mon.0) 2076 : audit [INF] from='client.? 172.21.15.47:0/1977736697' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi047-82386-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi047-82386-53"}]': finished 2024-02-23T00:46:24.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:23.184297+0000 mon.a (mon.0) 2077 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-39","var": "hit_set_count","val": "2"}]': finished 2024-02-23T00:46:24.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:23.184353+0000 mon.a (mon.0) 2078 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:46:24.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:23.184411+0000 mon.a (mon.0) 2079 : cluster [DBG] osdmap e311: 8 total, 8 up, 8 in 2024-02-23T00:46:24.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:23.194514+0000 mon.a (mon.0) 2080 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-39","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:46:24.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:23.243623+0000 mon.b (mon.1) 344 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:24.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:23.256806+0000 mgr.y (mgr.14517) 310 : cluster [DBG] pgmap v442: 249 pgs: 8 unknown, 1 peering, 240 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-02-23T00:46:24.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:23.257792+0000 mon.a (mon.0) 2081 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "22"}]: dispatch 2024-02-23T00:46:24.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:23.174768+0000 mon.a (mon.0) 2075 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:46:24.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:23.184185+0000 mon.a (mon.0) 2076 : audit [INF] from='client.? 172.21.15.47:0/1977736697' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi047-82386-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi047-82386-53"}]': finished 2024-02-23T00:46:24.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:23.184297+0000 mon.a (mon.0) 2077 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-39","var": "hit_set_count","val": "2"}]': finished 2024-02-23T00:46:24.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:23.184353+0000 mon.a (mon.0) 2078 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:46:24.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:23.184411+0000 mon.a (mon.0) 2079 : cluster [DBG] osdmap e311: 8 total, 8 up, 8 in 2024-02-23T00:46:24.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:23.194514+0000 mon.a (mon.0) 2080 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-39","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:46:24.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:23.243623+0000 mon.b (mon.1) 344 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:24.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:23.256806+0000 mgr.y (mgr.14517) 310 : cluster [DBG] pgmap v442: 249 pgs: 8 unknown, 1 peering, 240 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-02-23T00:46:24.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:23.257792+0000 mon.a (mon.0) 2081 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "22"}]: dispatch 2024-02-23T00:46:24.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:23.174768+0000 mon.a (mon.0) 2075 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:46:24.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:23.184185+0000 mon.a (mon.0) 2076 : audit [INF] from='client.? 172.21.15.47:0/1977736697' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi047-82386-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi047-82386-53"}]': finished 2024-02-23T00:46:24.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:23.184297+0000 mon.a (mon.0) 2077 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-39","var": "hit_set_count","val": "2"}]': finished 2024-02-23T00:46:24.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:23.184353+0000 mon.a (mon.0) 2078 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:46:24.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:23.184411+0000 mon.a (mon.0) 2079 : cluster [DBG] osdmap e311: 8 total, 8 up, 8 in 2024-02-23T00:46:24.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:23.194514+0000 mon.a (mon.0) 2080 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-39","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:46:24.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:23.243623+0000 mon.b (mon.1) 344 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:24.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:23.256806+0000 mgr.y (mgr.14517) 310 : cluster [DBG] pgmap v442: 249 pgs: 8 unknown, 1 peering, 240 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-02-23T00:46:24.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:23.257792+0000 mon.a (mon.0) 2081 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "22"}]: dispatch 2024-02-23T00:46:24.671 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:46:24.674 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:24.675 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:46:24.703 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:46:24.775 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:46:24.807 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:24.828 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:46:24.831 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:24.832 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:46:24.859 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:46:24.932 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:46:24.962 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:24.982 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:46:24.985 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:24.985 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:46:25.086 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:46:25.114 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:25.229 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:24.186786+0000 mon.a (mon.0) 2082 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-39","var": "hit_set_period","val": "600"}]': finished 2024-02-23T00:46:25.229 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:24.186893+0000 mon.a (mon.0) 2083 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "22"}]': finished 2024-02-23T00:46:25.229 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:24.186948+0000 mon.a (mon.0) 2084 : cluster [DBG] osdmap e312: 8 total, 8 up, 8 in 2024-02-23T00:46:25.229 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:24.191619+0000 mon.a (mon.0) 2085 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-39","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-23T00:46:25.229 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:24.244539+0000 mon.b (mon.1) 345 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:25.229 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:25.148907+0000 mon.a (mon.0) 2086 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-39","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-23T00:46:25.230 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:25.149006+0000 mon.a (mon.0) 2087 : cluster [DBG] osdmap e313: 8 total, 8 up, 8 in 2024-02-23T00:46:25.230 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:25.155137+0000 mon.a (mon.0) 2088 : audit [INF] from='client.? 172.21.15.47:0/1977736697' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi047-82386-53"}]: dispatch 2024-02-23T00:46:25.230 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:24.186786+0000 mon.a (mon.0) 2082 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-39","var": "hit_set_period","val": "600"}]': finished 2024-02-23T00:46:25.230 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:24.186893+0000 mon.a (mon.0) 2083 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "22"}]': finished 2024-02-23T00:46:25.230 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:24.186948+0000 mon.a (mon.0) 2084 : cluster [DBG] osdmap e312: 8 total, 8 up, 8 in 2024-02-23T00:46:25.230 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:24.191619+0000 mon.a (mon.0) 2085 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-39","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-23T00:46:25.230 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:24.244539+0000 mon.b (mon.1) 345 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:25.230 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:25.148907+0000 mon.a (mon.0) 2086 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-39","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-23T00:46:25.230 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:25.149006+0000 mon.a (mon.0) 2087 : cluster [DBG] osdmap e313: 8 total, 8 up, 8 in 2024-02-23T00:46:25.230 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:25.155137+0000 mon.a (mon.0) 2088 : audit [INF] from='client.? 172.21.15.47:0/1977736697' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi047-82386-53"}]: dispatch 2024-02-23T00:46:25.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:24.186786+0000 mon.a (mon.0) 2082 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-39","var": "hit_set_period","val": "600"}]': finished 2024-02-23T00:46:25.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:24.186893+0000 mon.a (mon.0) 2083 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "22"}]': finished 2024-02-23T00:46:25.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:24.186948+0000 mon.a (mon.0) 2084 : cluster [DBG] osdmap e312: 8 total, 8 up, 8 in 2024-02-23T00:46:25.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:24.191619+0000 mon.a (mon.0) 2085 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-39","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-23T00:46:25.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:24.244539+0000 mon.b (mon.1) 345 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:25.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:25.148907+0000 mon.a (mon.0) 2086 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-39","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-23T00:46:25.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:25.149006+0000 mon.a (mon.0) 2087 : cluster [DBG] osdmap e313: 8 total, 8 up, 8 in 2024-02-23T00:46:25.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:25.155137+0000 mon.a (mon.0) 2088 : audit [INF] from='client.? 172.21.15.47:0/1977736697' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi047-82386-53"}]: dispatch 2024-02-23T00:46:26.348 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset noscrub 2024-02-23T00:46:26.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:25.245747+0000 mon.b (mon.1) 346 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:26.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:25.257291+0000 mgr.y (mgr.14517) 311 : cluster [DBG] pgmap v445: 241 pgs: 1 peering, 240 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-02-23T00:46:26.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:26.151844+0000 mon.a (mon.0) 2089 : audit [INF] from='client.? 172.21.15.47:0/1977736697' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi047-82386-53"}]': finished 2024-02-23T00:46:26.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:26.151899+0000 mon.a (mon.0) 2090 : cluster [DBG] osdmap e314: 8 total, 8 up, 8 in 2024-02-23T00:46:26.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:26.152441+0000 mon.a (mon.0) 2091 : audit [INF] from='client.? 172.21.15.47:0/1977736697' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi047-82386-53"}]: dispatch 2024-02-23T00:46:26.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:25.245747+0000 mon.b (mon.1) 346 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:26.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:25.257291+0000 mgr.y (mgr.14517) 311 : cluster [DBG] pgmap v445: 241 pgs: 1 peering, 240 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-02-23T00:46:26.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:26.151844+0000 mon.a (mon.0) 2089 : audit [INF] from='client.? 172.21.15.47:0/1977736697' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi047-82386-53"}]': finished 2024-02-23T00:46:26.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:26.151899+0000 mon.a (mon.0) 2090 : cluster [DBG] osdmap e314: 8 total, 8 up, 8 in 2024-02-23T00:46:26.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:26.152441+0000 mon.a (mon.0) 2091 : audit [INF] from='client.? 172.21.15.47:0/1977736697' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi047-82386-53"}]: dispatch 2024-02-23T00:46:26.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:25.245747+0000 mon.b (mon.1) 346 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:26.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:25.257291+0000 mgr.y (mgr.14517) 311 : cluster [DBG] pgmap v445: 241 pgs: 1 peering, 240 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-02-23T00:46:26.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:26.151844+0000 mon.a (mon.0) 2089 : audit [INF] from='client.? 172.21.15.47:0/1977736697' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi047-82386-53"}]': finished 2024-02-23T00:46:26.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:26.151899+0000 mon.a (mon.0) 2090 : cluster [DBG] osdmap e314: 8 total, 8 up, 8 in 2024-02-23T00:46:26.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:26.152441+0000 mon.a (mon.0) 2091 : audit [INF] from='client.? 172.21.15.47:0/1977736697' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi047-82386-53"}]: dispatch 2024-02-23T00:46:27.366 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:46:27.368 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:27.369 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:46:27.470 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:46:27.497 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:26.194640+0000 mon.a (mon.0) 2092 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:46:27.497 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:26.195218+0000 mon.a (mon.0) 2093 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-39"}]: dispatch 2024-02-23T00:46:27.497 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:26.247164+0000 mon.b (mon.1) 347 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:27.498 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:26.302034+0000 mon.a (mon.0) 2094 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:46:27.498 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:27.153157+0000 mon.a (mon.0) 2095 : audit [INF] from='client.? 172.21.15.47:0/1977736697' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi047-82386-53"}]': finished 2024-02-23T00:46:27.498 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:27.153298+0000 mon.a (mon.0) 2096 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-39"}]': finished 2024-02-23T00:46:27.498 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:27.153351+0000 mon.a (mon.0) 2097 : cluster [DBG] osdmap e315: 8 total, 8 up, 8 in 2024-02-23T00:46:27.498 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:27.176764+0000 mon.c (mon.2) 242 : audit [INF] from='client.? 172.21.15.47:0/2888114840' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi047-82386-54"}]: dispatch 2024-02-23T00:46:27.498 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:27.177054+0000 mon.a (mon.0) 2098 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi047-82386-54"}]: dispatch 2024-02-23T00:46:27.498 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:27.177804+0000 mon.c (mon.2) 243 : audit [INF] from='client.? 172.21.15.47:0/2888114840' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi047-82386-54"}]: dispatch 2024-02-23T00:46:27.498 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:27.178060+0000 mon.a (mon.0) 2099 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi047-82386-54"}]: dispatch 2024-02-23T00:46:27.498 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:27.178601+0000 mon.c (mon.2) 244 : audit [INF] from='client.? 172.21.15.47:0/2888114840' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi047-82386-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:27.498 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:27.178847+0000 mon.a (mon.0) 2100 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi047-82386-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:27.498 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:26.194640+0000 mon.a (mon.0) 2092 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:46:27.499 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:26.195218+0000 mon.a (mon.0) 2093 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-39"}]: dispatch 2024-02-23T00:46:27.499 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:26.247164+0000 mon.b (mon.1) 347 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:27.499 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:26.302034+0000 mon.a (mon.0) 2094 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:46:27.499 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:27.153157+0000 mon.a (mon.0) 2095 : audit [INF] from='client.? 172.21.15.47:0/1977736697' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi047-82386-53"}]': finished 2024-02-23T00:46:27.499 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:27.153298+0000 mon.a (mon.0) 2096 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-39"}]': finished 2024-02-23T00:46:27.499 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:27.153351+0000 mon.a (mon.0) 2097 : cluster [DBG] osdmap e315: 8 total, 8 up, 8 in 2024-02-23T00:46:27.499 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:27.176764+0000 mon.c (mon.2) 242 : audit [INF] from='client.? 172.21.15.47:0/2888114840' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi047-82386-54"}]: dispatch 2024-02-23T00:46:27.499 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:27.177054+0000 mon.a (mon.0) 2098 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi047-82386-54"}]: dispatch 2024-02-23T00:46:27.499 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:27.177804+0000 mon.c (mon.2) 243 : audit [INF] from='client.? 172.21.15.47:0/2888114840' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi047-82386-54"}]: dispatch 2024-02-23T00:46:27.499 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:27.178060+0000 mon.a (mon.0) 2099 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi047-82386-54"}]: dispatch 2024-02-23T00:46:27.499 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:27.178601+0000 mon.c (mon.2) 244 : audit [INF] from='client.? 172.21.15.47:0/2888114840' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi047-82386-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:27.499 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:27.178847+0000 mon.a (mon.0) 2100 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi047-82386-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:27.500 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:27.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.585+0000 7f168c086700 1 -- 172.21.15.47:0/3359856022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16840fd510 msgr2=0x7f16840f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:27.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.585+0000 7f168c086700 1 --2- 172.21.15.47:0/3359856022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16840fd510 0x7f16840f6d40 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 rx=0x7f1674004870 tx=0x7f1674009cb0).stop 2024-02-23T00:46:27.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.585+0000 7f168c086700 1 -- 172.21.15.47:0/3359856022 shutdown_connections 2024-02-23T00:46:27.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.585+0000 7f168c086700 1 --2- 172.21.15.47:0/3359856022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16840f7400 0x7f1684103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:27.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.585+0000 7f168c086700 1 --2- 172.21.15.47:0/3359856022 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f16840fcbb0 0x7f16840fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:27.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.585+0000 7f168c086700 1 --2- 172.21.15.47:0/3359856022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16840fd510 0x7f16840f6d40 unknown :-1 s=CLOSED pgs=2501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:27.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.585+0000 7f168c086700 1 -- 172.21.15.47:0/3359856022 >> 172.21.15.47:0/3359856022 conn(0x7f16840f2820 msgr2=0x7f16840f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:46:27.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.585+0000 7f168c086700 1 -- 172.21.15.47:0/3359856022 shutdown_connections 2024-02-23T00:46:27.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.585+0000 7f168c086700 1 -- 172.21.15.47:0/3359856022 wait complete. 2024-02-23T00:46:27.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.585+0000 7f168c086700 1 librados: starting msgr at 2024-02-23T00:46:27.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.585+0000 7f168c086700 1 librados: starting objecter 2024-02-23T00:46:27.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.585+0000 7f168c086700 1 Processor -- start 2024-02-23T00:46:27.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.586+0000 7f168c086700 1 -- start start 2024-02-23T00:46:27.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.586+0000 7f168c086700 1 librados: setting wanted keys 2024-02-23T00:46:27.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.586+0000 7f168c086700 1 librados: calling monclient init 2024-02-23T00:46:27.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.586+0000 7f168c086700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16840f7400 0x7f168410e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:27.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.586+0000 7f168c086700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16840fcbb0 0x7f168410e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:27.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.586+0000 7f168c086700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f16840fd510 0x7f16841081c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:27.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.586+0000 7f168c086700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16841086b0 con 0x7f16840fcbb0 2024-02-23T00:46:27.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.586+0000 7f168c086700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1684108820 con 0x7f16840f7400 2024-02-23T00:46:27.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.586+0000 7f168c086700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16841089e0 con 0x7f16840fd510 2024-02-23T00:46:27.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.586+0000 7f1689621700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16840fcbb0 0x7f168410e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:27.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.586+0000 7f1689621700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16840fcbb0 0x7f168410e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43772/0 (socket says 172.21.15.47:43772) 2024-02-23T00:46:27.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.586+0000 7f168a623700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f16840fd510 0x7f16841081c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:27.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.586+0000 7f1689621700 1 -- 172.21.15.47:0/3218220533 learned_addr learned my addr 172.21.15.47:0/3218220533 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:46:27.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.586+0000 7f1689e22700 1 --2- 172.21.15.47:0/3218220533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16840f7400 0x7f168410e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:27.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.586+0000 7f1689621700 1 -- 172.21.15.47:0/3218220533 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f16840fd510 msgr2=0x7f16841081c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:27.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.586+0000 7f1689621700 1 --2- 172.21.15.47:0/3218220533 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f16840fd510 0x7f16841081c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:27.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.586+0000 7f1689621700 1 -- 172.21.15.47:0/3218220533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16840f7400 msgr2=0x7f168410e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:27.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.586+0000 7f1689621700 1 --2- 172.21.15.47:0/3218220533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16840f7400 0x7f168410e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:27.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.586+0000 7f1689621700 1 -- 172.21.15.47:0/3218220533 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1680005040 con 0x7f16840fcbb0 2024-02-23T00:46:27.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.586+0000 7f168a623700 1 --2- 172.21.15.47:0/3218220533 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f16840fd510 0x7f16841081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:46:27.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.586+0000 7f1689621700 1 --2- 172.21.15.47:0/3218220533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16840fcbb0 0x7f168410e510 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 rx=0x7f1674009da0 tx=0x7f1674009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:27.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.586+0000 7f167affd700 10 client.?.objecter ms_handle_connect 0x7f16840fcbb0 2024-02-23T00:46:27.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.586+0000 7f167affd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:46:27.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.586+0000 7f167affd700 1 -- 172.21.15.47:0/3218220533 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167400e030 con 0x7f16840fcbb0 2024-02-23T00:46:27.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.586+0000 7f167affd700 1 -- 172.21.15.47:0/3218220533 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f1674013d70 con 0x7f16840fcbb0 2024-02-23T00:46:27.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.586+0000 7f168c086700 1 -- 172.21.15.47:0/3218220533 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1674005040 con 0x7f16840fcbb0 2024-02-23T00:46:27.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.587+0000 7f167affd700 1 -- 172.21.15.47:0/3218220533 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1674018070 con 0x7f16840fcbb0 2024-02-23T00:46:27.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.587+0000 7f168c086700 10 client.30759.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:46:27.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.587+0000 7f168c086700 1 -- 172.21.15.47:0/3218220533 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1684108e80 con 0x7f16840fcbb0 2024-02-23T00:46:27.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.587+0000 7f168c086700 1 librados: init done 2024-02-23T00:46:27.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.587+0000 7f168c086700 1 -- 172.21.15.47:0/3218220533 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1684109070 con 0x7f16840fcbb0 2024-02-23T00:46:27.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.588+0000 7f167affd700 1 -- 172.21.15.47:0/3218220533 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f1674009240 con 0x7f16840fcbb0 2024-02-23T00:46:27.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.588+0000 7f167affd700 1 --2- 172.21.15.47:0/3218220533 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1670062cc0 0x7f1670065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:27.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.588+0000 7f167affd700 1 -- 172.21.15.47:0/3218220533 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(315..315 src has 48..315) v4 ==== 9557+0+0 (secure 0 0 0) 0x7f167407e960 con 0x7f16840fcbb0 2024-02-23T00:46:27.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.588+0000 7f167affd700 10 client.30759.objecter ms_dispatch 0x7f1684001550 osd_map(315..315 src has 48..315) v4 2024-02-23T00:46:27.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.588+0000 7f167affd700 3 client.30759.objecter handle_osd_map got epochs [315,315] > 0 2024-02-23T00:46:27.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.588+0000 7f167affd700 3 client.30759.objecter handle_osd_map decoding full epoch 315 2024-02-23T00:46:27.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.588+0000 7f1689e22700 1 --2- 172.21.15.47:0/3218220533 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1670062cc0 0x7f1670065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:27.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.588+0000 7f167affd700 20 client.30759.objecter dump_active .. 0 homeless 2024-02-23T00:46:27.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.589+0000 7f167affd700 10 client.30759.objecter ms_handle_connect 0x7f1670062cc0 2024-02-23T00:46:27.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.589+0000 7f1689e22700 1 --2- 172.21.15.47:0/3218220533 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1670062cc0 0x7f1670065180 secure :-1 s=READY pgs=3021 cs=0 l=1 rev1=1 rx=0x7f16800092d0 tx=0x7f168000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:27.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.593+0000 7f167affd700 1 -- 172.21.15.47:0/3218220533 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f167404de20 con 0x7f16840fcbb0 2024-02-23T00:46:27.623 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:46:27.626 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:27.626 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:46:27.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:26.194640+0000 mon.a (mon.0) 2092 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:46:27.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:26.195218+0000 mon.a (mon.0) 2093 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-39"}]: dispatch 2024-02-23T00:46:27.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:26.247164+0000 mon.b (mon.1) 347 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:27.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:26.302034+0000 mon.a (mon.0) 2094 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:46:27.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:27.153157+0000 mon.a (mon.0) 2095 : audit [INF] from='client.? 172.21.15.47:0/1977736697' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi047-82386-53"}]': finished 2024-02-23T00:46:27.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:27.153298+0000 mon.a (mon.0) 2096 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-39"}]': finished 2024-02-23T00:46:27.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:27.153351+0000 mon.a (mon.0) 2097 : cluster [DBG] osdmap e315: 8 total, 8 up, 8 in 2024-02-23T00:46:27.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:27.176764+0000 mon.c (mon.2) 242 : audit [INF] from='client.? 172.21.15.47:0/2888114840' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi047-82386-54"}]: dispatch 2024-02-23T00:46:27.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:27.177054+0000 mon.a (mon.0) 2098 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi047-82386-54"}]: dispatch 2024-02-23T00:46:27.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:27.177804+0000 mon.c (mon.2) 243 : audit [INF] from='client.? 172.21.15.47:0/2888114840' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi047-82386-54"}]: dispatch 2024-02-23T00:46:27.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:27.178060+0000 mon.a (mon.0) 2099 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi047-82386-54"}]: dispatch 2024-02-23T00:46:27.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:27.178601+0000 mon.c (mon.2) 244 : audit [INF] from='client.? 172.21.15.47:0/2888114840' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi047-82386-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:27.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:27.178847+0000 mon.a (mon.0) 2100 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi047-82386-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:27.727 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:46:27.754 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:27.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.837+0000 7f168c086700 1 -- 172.21.15.47:0/3218220533 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f168405fa60 con 0x7f16840fcbb0 2024-02-23T00:46:27.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.838+0000 7f167affd700 1 -- 172.21.15.47:0/3218220533 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+2718 (secure 0 0 0) 0x7f1674051440 con 0x7f16840fcbb0 2024-02-23T00:46:27.841 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:46:27.841 INFO:teuthology.orchestra.run.smithi047.stdout:{"fsid":"fce2617e-d1e2-11ee-95c0-87774f69a715","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noscrub,nodeep-scrub flag(s) set","count":20},"muted":false},"POOL_APP_NOT_ENABLED":{"severity":"HEALTH_WARN","summary":{"message":"1 pool(s) do not have an application enabled","count":1},"muted":false}},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":673,"monmap":{"epoch":3,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":315,"num_osds":8,"num_up_osds":8,"osd_up_since":1708648745,"num_in_osds":8,"osd_in_since":1708648727,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":240},{"state_name":"peering","count":1}],"num_pgs":241,"num_pools":10,"num_objects":364,"data_bytes":4009069,"bytes_used":2577059840,"bytes_avail":765383225344,"bytes_total":767960285184,"inactive_pgs_ratio":0.0041493778117001057,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"read_bytes_sec":1279,"write_bytes_sec":4351,"read_op_per_sec":1,"write_op_per_sec":8},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"services":{"dashboard":"https://172.21.15.47:8443/","prometheus":"http://172.21.15.47:9283/"}},"servicemap":{"epoch":23,"modified":"2024-02-23T00:43:27.194339+0000","services":{"rgw":{"daemons":{"summary":"","14460":{"start_epoch":12,"start_stamp":"2024-02-23T00:39:21.905359+0000","gid":14460,"addr":"172.21.15.47:0/1573276039","metadata":{"arch":"x86_64","ceph_release":"pacific","ceph_version":"ceph version 16.2.14-550-geb66ed92 (eb66ed921e744301e6863be6353618d63967ea59) pacific (stable)","ceph_version_short":"16.2.14-550-geb66ed92","container_hostname":"smithi047","container_image":"quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:44277403522cf37edeee63b04643fd6c6e9416f13c3731a771e3cc8e53b7d010","cpu":"Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz","distro":"centos","distro_description":"CentOS Stream 8","distro_version":"8","frontend_config#0":"beast port=80","frontend_type#0":"beast","hostname":"smithi047","id":"foo.a","kernel_description":"#1 SMP Fri Feb 9 14:58:00 UTC 2024","kernel_version":"4.18.0-540.el8.x86_64","mem_swap_kb":"0","mem_total_kb":"32543848","num_handles":"1","os":"Linux","pid":"7","realm_id":"","realm_name":"","zone_id":"52970d38-d172-4ec0-ad23-3afdfeba31e4","zone_name":"default","zonegroup_id":"343d5407-1492-440c-b380-09e4750e9af7","zonegroup_name":"default"},"task_status":{}}}}}},"progress_events":{}} 2024-02-23T00:46:27.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.840+0000 7f168c086700 10 librados: watch_flush enter 2024-02-23T00:46:27.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.840+0000 7f168c086700 10 librados: watch_flush exit 2024-02-23T00:46:27.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.840+0000 7f168c086700 20 client.30759.objecter shutdown clearing up homeless session... 2024-02-23T00:46:27.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.840+0000 7f168c086700 10 client.30759.objecter successfully canceled tick 2024-02-23T00:46:27.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.840+0000 7f168c086700 1 -- 172.21.15.47:0/3218220533 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1670062cc0 msgr2=0x7f1670065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:27.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.840+0000 7f168c086700 1 --2- 172.21.15.47:0/3218220533 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1670062cc0 0x7f1670065180 secure :-1 s=READY pgs=3021 cs=0 l=1 rev1=1 rx=0x7f16800092d0 tx=0x7f168000b040).stop 2024-02-23T00:46:27.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.840+0000 7f168c086700 1 -- 172.21.15.47:0/3218220533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16840fcbb0 msgr2=0x7f168410e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:27.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.840+0000 7f168c086700 1 --2- 172.21.15.47:0/3218220533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16840fcbb0 0x7f168410e510 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 rx=0x7f1674009da0 tx=0x7f1674009cb0).stop 2024-02-23T00:46:27.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.840+0000 7f168c086700 1 -- 172.21.15.47:0/3218220533 shutdown_connections 2024-02-23T00:46:27.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.840+0000 7f168c086700 1 --2- 172.21.15.47:0/3218220533 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1670062cc0 0x7f1670065180 unknown :-1 s=CLOSED pgs=3021 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:27.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.840+0000 7f168c086700 1 --2- 172.21.15.47:0/3218220533 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f16840fd510 0x7f16841081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:27.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.840+0000 7f168c086700 1 --2- 172.21.15.47:0/3218220533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16840f7400 0x7f168410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:27.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.840+0000 7f168c086700 1 --2- 172.21.15.47:0/3218220533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16840fcbb0 0x7f168410e510 unknown :-1 s=CLOSED pgs=2502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:27.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.840+0000 7f168c086700 1 -- 172.21.15.47:0/3218220533 >> 172.21.15.47:0/3218220533 conn(0x7f16840f2820 msgr2=0x7f1684060e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:46:27.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.840+0000 7f168c086700 1 -- 172.21.15.47:0/3218220533 shutdown_connections 2024-02-23T00:46:27.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.840+0000 7f168c086700 1 -- 172.21.15.47:0/3218220533 wait complete. 2024-02-23T00:46:27.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:27.840+0000 7f168c086700 1 librados: shutdown 2024-02-23T00:46:27.852 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:46:27.855 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:27.855 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:46:27.956 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:46:27.988 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:28.009 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:46:28.012 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:28.012 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:46:28.113 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:46:28.142 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:28.157 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [==========] Running 72 tests from 4 test suites. 2024-02-23T00:46:28.158 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [----------] Global test environment set-up. 2024-02-23T00:46:28.158 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2024-02-23T00:46:28.158 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: seed 82465 2024-02-23T00:46:28.158 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2024-02-23T00:46:28.158 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (274 ms) 2024-02-23T00:46:28.158 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2024-02-23T00:46:28.158 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (15286 ms) 2024-02-23T00:46:28.158 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2024-02-23T00:46:28.158 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (23 ms) 2024-02-23T00:46:28.158 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (15583 ms total) 2024-02-23T00:46:28.158 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: 2024-02-23T00:46:28.158 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [----------] 43 tests from LibRadosTwoPoolsPP 2024-02-23T00:46:28.158 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2024-02-23T00:46:28.158 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (8453 ms) 2024-02-23T00:46:28.159 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2024-02-23T00:46:28.159 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (8732 ms) 2024-02-23T00:46:28.159 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-02-23T00:46:28.159 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (10177 ms) 2024-02-23T00:46:28.159 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-02-23T00:46:28.159 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: my_snaps [4] 2024-02-23T00:46:28.159 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: my_snaps [5,4] 2024-02-23T00:46:28.159 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: my_snaps [6,5,4] 2024-02-23T00:46:28.159 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: my_snaps [7,6,5,4] 2024-02-23T00:46:28.159 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: promoting some heads 2024-02-23T00:46:28.159 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: promoting from clones for snap 7 2024-02-23T00:46:28.159 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: promoting from clones for snap 6 2024-02-23T00:46:28.159 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: promoting from clones for snap 5 2024-02-23T00:46:28.159 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: promoting from clones for snap 4 2024-02-23T00:46:28.159 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: waiting for scrubs... 2024-02-23T00:46:28.159 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: done waiting 2024-02-23T00:46:28.160 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (46010 ms) 2024-02-23T00:46:28.160 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-02-23T00:46:28.160 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (9995 ms) 2024-02-23T00:46:28.160 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-02-23T00:46:28.160 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (7993 ms) 2024-02-23T00:46:28.160 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-02-23T00:46:28.160 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (8011 ms) 2024-02-23T00:46:28.160 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2024-02-23T00:46:28.160 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (7991 ms) 2024-02-23T00:46:28.160 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-02-23T00:46:28.160 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (10012 ms) 2024-02-23T00:46:28.160 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-02-23T00:46:28.160 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (9000 ms) 2024-02-23T00:46:28.160 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-02-23T00:46:28.161 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (10004 ms) 2024-02-23T00:46:28.161 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-02-23T00:46:28.161 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (8436 ms) 2024-02-23T00:46:28.161 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2024-02-23T00:46:28.161 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (7575 ms) 2024-02-23T00:46:28.161 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-02-23T00:46:28.161 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (13886 ms) 2024-02-23T00:46:28.161 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-02-23T00:46:28.161 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (7118 ms) 2024-02-23T00:46:28.161 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-02-23T00:46:28.161 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (8011 ms) 2024-02-23T00:46:28.161 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetRead 2024-02-23T00:46:28.161 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: hmm, no HitSet yet 2024-02-23T00:46:28.161 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: ok, hit_set contains 186:602f83fe:::foo:head 2024-02-23T00:46:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:46:28.354 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:28.355 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:46:28.456 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:46:28.488 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:28.510 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:46:28.514 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:28.515 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:46:28.609 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:27.248292+0000 mon.b (mon.1) 348 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:28.609 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:27.258149+0000 mgr.y (mgr.14517) 312 : cluster [DBG] pgmap v448: 240 pgs: 240 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:28.610 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:27.259020+0000 mon.a (mon.0) 2101 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "21"}]: dispatch 2024-02-23T00:46:28.610 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:27.839277+0000 mon.a (mon.0) 2102 : audit [DBG] from='client.? 172.21.15.47:0/3218220533' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-23T00:46:28.610 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:28.154420+0000 mon.a (mon.0) 2103 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi047-82386-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:28.610 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:28.154537+0000 mon.a (mon.0) 2104 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "21"}]': finished 2024-02-23T00:46:28.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:27.248292+0000 mon.b (mon.1) 348 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:28.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:27.258149+0000 mgr.y (mgr.14517) 312 : cluster [DBG] pgmap v448: 240 pgs: 240 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:28.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:27.259020+0000 mon.a (mon.0) 2101 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "21"}]: dispatch 2024-02-23T00:46:28.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:27.839277+0000 mon.a (mon.0) 2102 : audit [DBG] from='client.? 172.21.15.47:0/3218220533' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-23T00:46:28.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:28.154420+0000 mon.a (mon.0) 2103 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi047-82386-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:28.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:28.154537+0000 mon.a (mon.0) 2104 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "21"}]': finished 2024-02-23T00:46:28.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:28.154588+0000 mon.a (mon.0) 2105 : cluster [DBG] osdmap e316: 8 total, 8 up, 8 in 2024-02-23T00:46:28.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:28.156359+0000 mon.c (mon.2) 245 : audit [INF] from='client.? 172.21.15.47:0/2888114840' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi047-82386-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi047-82386-54"}]: dispatch 2024-02-23T00:46:28.616 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:46:28.644 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:28.664 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:46:28.666 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:28.667 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:46:28.682 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:46:28.687 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:46:28.767 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:46:28.792 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:27.248292+0000 mon.b (mon.1) 348 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:28.792 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:27.258149+0000 mgr.y (mgr.14517) 312 : cluster [DBG] pgmap v448: 240 pgs: 240 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:28.792 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:27.259020+0000 mon.a (mon.0) 2101 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "21"}]: dispatch 2024-02-23T00:46:28.792 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:27.839277+0000 mon.a (mon.0) 2102 : audit [DBG] from='client.? 172.21.15.47:0/3218220533' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-23T00:46:28.792 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:28.154420+0000 mon.a (mon.0) 2103 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi047-82386-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:28.792 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:28.154537+0000 mon.a (mon.0) 2104 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "21"}]': finished 2024-02-23T00:46:28.793 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:28.154588+0000 mon.a (mon.0) 2105 : cluster [DBG] osdmap e316: 8 total, 8 up, 8 in 2024-02-23T00:46:28.793 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:28.156359+0000 mon.c (mon.2) 245 : audit [INF] from='client.? 172.21.15.47:0/2888114840' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi047-82386-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi047-82386-54"}]: dispatch 2024-02-23T00:46:28.794 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:28.814 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:46:28.816 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:28.816 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:46:28.917 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:46:28.944 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:28.963 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:46:28.965 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:28.965 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:46:29.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:28.154588+0000 mon.a (mon.0) 2105 : cluster [DBG] osdmap e316: 8 total, 8 up, 8 in 2024-02-23T00:46:29.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:28.156359+0000 mon.c (mon.2) 245 : audit [INF] from='client.? 172.21.15.47:0/2888114840' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi047-82386-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi047-82386-54"}]: dispatch 2024-02-23T00:46:29.067 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:46:29.096 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:29.317 INFO:tasks.cephadm.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-02-23T00:46:29.317 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph pg dump --format=json 2024-02-23T00:46:29.524 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:46:29.528 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:29.528 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:46:29.629 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:46:29.659 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:29.678 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:46:29.681 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:29.682 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:46:29.782 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:46:29.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:28.249604+0000 mon.b (mon.1) 349 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:29.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:28.600777+0000 mon.a (mon.0) 2106 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi047-82386-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi047-82386-54"}]: dispatch 2024-02-23T00:46:29.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:29.250572+0000 mon.b (mon.1) 350 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:29.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:29.251243+0000 mon.b (mon.1) 351 : audit [INF] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pg_num","val":"11"}]: dispatch 2024-02-23T00:46:29.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:29.603478+0000 mon.a (mon.0) 2107 : cluster [DBG] osdmap e317: 8 total, 8 up, 8 in 2024-02-23T00:46:29.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:28.249604+0000 mon.b (mon.1) 349 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:29.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:28.600777+0000 mon.a (mon.0) 2106 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi047-82386-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi047-82386-54"}]: dispatch 2024-02-23T00:46:29.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:29.250572+0000 mon.b (mon.1) 350 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:29.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:29.251243+0000 mon.b (mon.1) 351 : audit [INF] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pg_num","val":"11"}]: dispatch 2024-02-23T00:46:29.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:29.603478+0000 mon.a (mon.0) 2107 : cluster [DBG] osdmap e317: 8 total, 8 up, 8 in 2024-02-23T00:46:29.812 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:29.832 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:46:29.835 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:29.835 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:46:29.936 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:46:29.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:28.249604+0000 mon.b (mon.1) 349 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:29.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:28.600777+0000 mon.a (mon.0) 2106 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi047-82386-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi047-82386-54"}]: dispatch 2024-02-23T00:46:29.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:29.250572+0000 mon.b (mon.1) 350 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:29.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:29.251243+0000 mon.b (mon.1) 351 : audit [INF] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pg_num","val":"11"}]: dispatch 2024-02-23T00:46:29.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:29.603478+0000 mon.a (mon.0) 2107 : cluster [DBG] osdmap e317: 8 total, 8 up, 8 in 2024-02-23T00:46:29.968 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:29.990 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:46:29.993 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:29.993 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:46:30.094 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:46:30.123 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:30.143 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:46:30.145 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:30.146 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:46:30.246 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:46:30.276 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:30.330 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:46:30.330 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-02-23T00:46:30.330 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-02-23T00:46:30.330 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-02-23T00:46:30.330 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:46:30.332 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:46:30.334 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:30.335 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:46:30.436 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:46:30.464 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:30.557 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:46:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:46:30] "GET /metrics HTTP/1.1" 200 190463 "" "Prometheus/2.33.4" 2024-02-23T00:46:30.712 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:46:30.716 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:30.716 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:46:30.817 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:46:30.844 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:30.926 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:29.258570+0000 mgr.y (mgr.14517) 313 : cluster [DBG] pgmap v451: 240 pgs: 32 unknown, 208 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:30.926 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:29.317269+0000 osd.5 (osd.5) 134 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:46:30.926 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:29.317427+0000 osd.5 (osd.5) 135 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:46:30.926 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:29.606300+0000 mon.a (mon.0) 2108 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pg_num","val":"11"}]: dispatch 2024-02-23T00:46:30.926 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:29.606495+0000 mon.a (mon.0) 2109 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:46:30.927 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:30.161989+0000 mon.a (mon.0) 2110 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi047-82386-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi047-82386-54"}]': finished 2024-02-23T00:46:30.927 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:30.162222+0000 mon.a (mon.0) 2111 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pg_num","val":"11"}]': finished 2024-02-23T00:46:30.927 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:30.162294+0000 mon.a (mon.0) 2112 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:46:30.927 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:30.162349+0000 mon.a (mon.0) 2113 : cluster [DBG] osdmap e318: 8 total, 8 up, 8 in 2024-02-23T00:46:30.927 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:30.167701+0000 mon.b (mon.1) 352 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:30.927 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:30.182948+0000 mon.a (mon.0) 2114 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi047-82465-6","var": "pg_num","format": "json"}]: dispatch 2024-02-23T00:46:30.927 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:30.183857+0000 mon.a (mon.0) 2115 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:46:30.927 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:29.258570+0000 mgr.y (mgr.14517) 313 : cluster [DBG] pgmap v451: 240 pgs: 32 unknown, 208 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:30.927 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:29.317269+0000 osd.5 (osd.5) 134 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:46:30.927 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:29.317427+0000 osd.5 (osd.5) 135 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:46:30.928 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:29.606300+0000 mon.a (mon.0) 2108 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pg_num","val":"11"}]: dispatch 2024-02-23T00:46:30.928 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:29.606495+0000 mon.a (mon.0) 2109 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:46:30.928 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:30.161989+0000 mon.a (mon.0) 2110 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi047-82386-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi047-82386-54"}]': finished 2024-02-23T00:46:30.928 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:30.162222+0000 mon.a (mon.0) 2111 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pg_num","val":"11"}]': finished 2024-02-23T00:46:30.928 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:30.162294+0000 mon.a (mon.0) 2112 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:46:30.928 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:30.162349+0000 mon.a (mon.0) 2113 : cluster [DBG] osdmap e318: 8 total, 8 up, 8 in 2024-02-23T00:46:30.928 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:30.167701+0000 mon.b (mon.1) 352 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:30.928 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:30.182948+0000 mon.a (mon.0) 2114 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi047-82465-6","var": "pg_num","format": "json"}]: dispatch 2024-02-23T00:46:30.928 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:30.183857+0000 mon.a (mon.0) 2115 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:46:30.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:29.258570+0000 mgr.y (mgr.14517) 313 : cluster [DBG] pgmap v451: 240 pgs: 32 unknown, 208 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:30.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:29.317269+0000 osd.5 (osd.5) 134 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:46:30.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:29.317427+0000 osd.5 (osd.5) 135 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:46:30.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:29.606300+0000 mon.a (mon.0) 2108 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pg_num","val":"11"}]: dispatch 2024-02-23T00:46:30.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:29.606495+0000 mon.a (mon.0) 2109 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:46:30.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:30.161989+0000 mon.a (mon.0) 2110 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi047-82386-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi047-82386-54"}]': finished 2024-02-23T00:46:30.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:30.162222+0000 mon.a (mon.0) 2111 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pg_num","val":"11"}]': finished 2024-02-23T00:46:30.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:30.162294+0000 mon.a (mon.0) 2112 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:46:30.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:30.162349+0000 mon.a (mon.0) 2113 : cluster [DBG] osdmap e318: 8 total, 8 up, 8 in 2024-02-23T00:46:30.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:30.167701+0000 mon.b (mon.1) 352 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:30.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:30.182948+0000 mon.a (mon.0) 2114 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi047-82465-6","var": "pg_num","format": "json"}]: dispatch 2024-02-23T00:46:30.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:30.183857+0000 mon.a (mon.0) 2115 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:46:32.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:31.163210+0000 mon.a (mon.0) 2116 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:46:32.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:31.163302+0000 mon.a (mon.0) 2117 : cluster [DBG] osdmap e319: 8 total, 8 up, 8 in 2024-02-23T00:46:32.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:31.166327+0000 mon.a (mon.0) 2118 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-41","var": "hit_set_count","val": "8"}]: dispatch 2024-02-23T00:46:32.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:31.168713+0000 mon.b (mon.1) 353 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:32.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:31.258934+0000 mgr.y (mgr.14517) 314 : cluster [DBG] pgmap v454: 247 pgs: 40 unknown, 207 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:46:32.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:31.259805+0000 mon.a (mon.0) 2119 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "20"}]: dispatch 2024-02-23T00:46:32.523 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:31.163210+0000 mon.a (mon.0) 2116 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:46:32.523 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:31.163302+0000 mon.a (mon.0) 2117 : cluster [DBG] osdmap e319: 8 total, 8 up, 8 in 2024-02-23T00:46:32.523 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:31.166327+0000 mon.a (mon.0) 2118 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-41","var": "hit_set_count","val": "8"}]: dispatch 2024-02-23T00:46:32.523 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:31.168713+0000 mon.b (mon.1) 353 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:32.523 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:31.258934+0000 mgr.y (mgr.14517) 314 : cluster [DBG] pgmap v454: 247 pgs: 40 unknown, 207 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:46:32.523 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:31.259805+0000 mon.a (mon.0) 2119 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "20"}]: dispatch 2024-02-23T00:46:32.524 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:31.163210+0000 mon.a (mon.0) 2116 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:46:32.524 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:31.163302+0000 mon.a (mon.0) 2117 : cluster [DBG] osdmap e319: 8 total, 8 up, 8 in 2024-02-23T00:46:32.524 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:31.166327+0000 mon.a (mon.0) 2118 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-41","var": "hit_set_count","val": "8"}]: dispatch 2024-02-23T00:46:32.524 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:31.168713+0000 mon.b (mon.1) 353 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:32.524 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:31.258934+0000 mgr.y (mgr.14517) 314 : cluster [DBG] pgmap v454: 247 pgs: 40 unknown, 207 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:46:32.524 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:31.259805+0000 mon.a (mon.0) 2119 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "20"}]: dispatch 2024-02-23T00:46:32.541 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.3 dump_historic_ops 2024-02-23T00:46:32.568 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:46:32.607 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:46:32.611 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:32.612 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:46:32.712 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:46:32.744 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:32.766 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:46:32.768 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:46:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:46:32.768 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:46:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:46:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:46:32.769 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:32.769 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:46:32.870 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:46:32.899 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:46:33.012 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:33.012 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:46:33.113 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:46:33.145 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:33.167 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:46:33.170 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:33.171 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:46:33.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:32.167696+0000 mon.a (mon.0) 2120 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-41","var": "hit_set_count","val": "8"}]': finished 2024-02-23T00:46:33.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:32.167773+0000 mon.a (mon.0) 2121 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "20"}]': finished 2024-02-23T00:46:33.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:32.167848+0000 mon.a (mon.0) 2122 : cluster [DBG] osdmap e320: 8 total, 8 up, 8 in 2024-02-23T00:46:33.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:32.169726+0000 mon.b (mon.1) 354 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:33.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:32.170447+0000 mon.a (mon.0) 2123 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:46:33.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:32.170722+0000 mon.c (mon.2) 246 : audit [INF] from='client.? 172.21.15.47:0/2888114840' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi047-82386-54"}]: dispatch 2024-02-23T00:46:33.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:32.171044+0000 mon.a (mon.0) 2124 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi047-82386-54"}]: dispatch 2024-02-23T00:46:33.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:32.504537+0000 mgr.y (mgr.14517) 315 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:46:33.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.216+0000 7fc7e4220700 1 -- 172.21.15.47:0/1466805498 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7dc0fdbd0 msgr2=0x7fc7dc0fdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:33.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.216+0000 7fc7e4220700 1 --2- 172.21.15.47:0/1466805498 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7dc0fdbd0 0x7fc7dc0fdfb0 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 rx=0x7fc7d8004870 tx=0x7fc7d8009cb0).stop 2024-02-23T00:46:33.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.216+0000 7fc7e4220700 1 -- 172.21.15.47:0/1466805498 shutdown_connections 2024-02-23T00:46:33.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.216+0000 7fc7e4220700 1 --2- 172.21.15.47:0/1466805498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7dc0f6a90 0x7fc7dc104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:33.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.216+0000 7fc7e4220700 1 --2- 172.21.15.47:0/1466805498 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7dc0fdbd0 0x7fc7dc0fdfb0 unknown :-1 s=CLOSED pgs=2615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:33.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.216+0000 7fc7e4220700 1 --2- 172.21.15.47:0/1466805498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7dc0fe530 0x7fc7dc0f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:33.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.216+0000 7fc7e4220700 1 -- 172.21.15.47:0/1466805498 >> 172.21.15.47:0/1466805498 conn(0x7fc7dc06c370 msgr2=0x7fc7dc06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:46:33.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.216+0000 7fc7e4220700 1 -- 172.21.15.47:0/1466805498 shutdown_connections 2024-02-23T00:46:33.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.216+0000 7fc7e4220700 1 -- 172.21.15.47:0/1466805498 wait complete. 2024-02-23T00:46:33.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.217+0000 7fc7e4220700 1 librados: starting msgr at 2024-02-23T00:46:33.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.217+0000 7fc7e4220700 1 librados: starting objecter 2024-02-23T00:46:33.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.217+0000 7fc7e4220700 1 Processor -- start 2024-02-23T00:46:33.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.217+0000 7fc7e4220700 1 -- start start 2024-02-23T00:46:33.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.217+0000 7fc7e4220700 1 librados: setting wanted keys 2024-02-23T00:46:33.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.217+0000 7fc7e4220700 1 librados: calling monclient init 2024-02-23T00:46:33.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.217+0000 7fc7e4220700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7dc0f6a90 0x7fc7dc10df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:33.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.217+0000 7fc7e4220700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7dc0fdbd0 0x7fc7dc10e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:33.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.217+0000 7fc7e4220700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7dc0fe530 0x7fc7dc1080f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:33.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.217+0000 7fc7e4220700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc1085e0 con 0x7fc7dc0f6a90 2024-02-23T00:46:33.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.217+0000 7fc7e4220700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc108750 con 0x7fc7dc0fe530 2024-02-23T00:46:33.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.217+0000 7fc7e4220700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc108910 con 0x7fc7dc0fdbd0 2024-02-23T00:46:33.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.217+0000 7fc7e17bb700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7dc0fdbd0 0x7fc7dc10e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:33.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.217+0000 7fc7e17bb700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7dc0fdbd0 0x7fc7dc10e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:58210/0 (socket says 172.21.15.47:58210) 2024-02-23T00:46:33.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.217+0000 7fc7e17bb700 1 -- 172.21.15.47:0/842112939 learned_addr learned my addr 172.21.15.47:0/842112939 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:46:33.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.217+0000 7fc7e27bd700 1 --2- 172.21.15.47:0/842112939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7dc0fe530 0x7fc7dc1080f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:33.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.218+0000 7fc7e17bb700 1 -- 172.21.15.47:0/842112939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7dc0fe530 msgr2=0x7fc7dc1080f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:33.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.218+0000 7fc7e17bb700 1 --2- 172.21.15.47:0/842112939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7dc0fe530 0x7fc7dc1080f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:33.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.218+0000 7fc7e17bb700 1 -- 172.21.15.47:0/842112939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7dc0f6a90 msgr2=0x7fc7dc10df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:33.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.218+0000 7fc7e17bb700 1 --2- 172.21.15.47:0/842112939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7dc0f6a90 0x7fc7dc10df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:46:33.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.218+0000 7fc7e17bb700 1 -- 172.21.15.47:0/842112939 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d40097c0 con 0x7fc7dc0fdbd0 2024-02-23T00:46:33.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.218+0000 7fc7e17bb700 1 --2- 172.21.15.47:0/842112939 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7dc0fdbd0 0x7fc7dc10e440 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 rx=0x7fc7cc0027f0 tx=0x7fc7cc002b00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:33.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.218+0000 7fc7d2ffd700 10 client.?.objecter ms_handle_connect 0x7fc7dc0fdbd0 2024-02-23T00:46:33.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.218+0000 7fc7d2ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:46:33.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.218+0000 7fc7d2ffd700 1 -- 172.21.15.47:0/842112939 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc007b40 con 0x7fc7dc0fdbd0 2024-02-23T00:46:33.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.218+0000 7fc7d2ffd700 1 -- 172.21.15.47:0/842112939 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fc7cc00f040 con 0x7fc7dc0fdbd0 2024-02-23T00:46:33.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.218+0000 7fc7d2ffd700 1 -- 172.21.15.47:0/842112939 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc00d660 con 0x7fc7dc0fdbd0 2024-02-23T00:46:33.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.218+0000 7fc7e4220700 1 -- 172.21.15.47:0/842112939 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d8005040 con 0x7fc7dc0fdbd0 2024-02-23T00:46:33.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.218+0000 7fc7e4220700 10 client.41099.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:46:33.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.218+0000 7fc7e4220700 1 -- 172.21.15.47:0/842112939 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7dc108f50 con 0x7fc7dc0fdbd0 2024-02-23T00:46:33.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.218+0000 7fc7e4220700 1 librados: init done 2024-02-23T00:46:33.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.219+0000 7fc7e4220700 1 -- 172.21.15.47:0/842112939 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7dc05fa40 con 0x7fc7dc0fdbd0 2024-02-23T00:46:33.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.220+0000 7fc7d2ffd700 1 -- 172.21.15.47:0/842112939 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fc7cc009110 con 0x7fc7dc0fdbd0 2024-02-23T00:46:33.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.220+0000 7fc7d2ffd700 1 --2- 172.21.15.47:0/842112939 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc7c8062cc0 0x7fc7c8065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:33.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.220+0000 7fc7d2ffd700 1 -- 172.21.15.47:0/842112939 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(321..321 src has 48..321) v4 ==== 9668+0+0 (secure 0 0 0) 0x7fc7cc07d5a0 con 0x7fc7dc0fdbd0 2024-02-23T00:46:33.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.220+0000 7fc7d2ffd700 10 client.41099.objecter ms_dispatch 0x7fc7dc001550 osd_map(321..321 src has 48..321) v4 2024-02-23T00:46:33.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.220+0000 7fc7d2ffd700 3 client.41099.objecter handle_osd_map got epochs [321,321] > 0 2024-02-23T00:46:33.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.220+0000 7fc7d2ffd700 3 client.41099.objecter handle_osd_map decoding full epoch 321 2024-02-23T00:46:33.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.220+0000 7fc7e1fbc700 1 --2- 172.21.15.47:0/842112939 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc7c8062cc0 0x7fc7c8065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:33.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.220+0000 7fc7d2ffd700 20 client.41099.objecter dump_active .. 0 homeless 2024-02-23T00:46:33.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.221+0000 7fc7d2ffd700 10 client.41099.objecter ms_handle_connect 0x7fc7c8062cc0 2024-02-23T00:46:33.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.221+0000 7fc7e1fbc700 1 --2- 172.21.15.47:0/842112939 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc7c8062cc0 0x7fc7c8065180 secure :-1 s=READY pgs=3022 cs=0 l=1 rev1=1 rx=0x7fc7d80043d0 tx=0x7fc7d8009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:33.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.224+0000 7fc7d2ffd700 1 -- 172.21.15.47:0/842112939 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fc7cc04cb20 con 0x7fc7dc0fdbd0 2024-02-23T00:46:33.271 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:46:33.300 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:33.321 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:46:33.323 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:33.324 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:46:33.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:33.407+0000 7fc7e4220700 1 -- 172.21.15.47:0/842112939 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7fc7dc106e90 con 0x7fc7dc0fdbd0 2024-02-23T00:46:33.424 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:46:33.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:32.167696+0000 mon.a (mon.0) 2120 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-41","var": "hit_set_count","val": "8"}]': finished 2024-02-23T00:46:33.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:32.167773+0000 mon.a (mon.0) 2121 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "20"}]': finished 2024-02-23T00:46:33.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:32.167848+0000 mon.a (mon.0) 2122 : cluster [DBG] osdmap e320: 8 total, 8 up, 8 in 2024-02-23T00:46:33.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:32.169726+0000 mon.b (mon.1) 354 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:33.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:32.170447+0000 mon.a (mon.0) 2123 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:46:33.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:32.170722+0000 mon.c (mon.2) 246 : audit [INF] from='client.? 172.21.15.47:0/2888114840' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi047-82386-54"}]: dispatch 2024-02-23T00:46:33.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:32.171044+0000 mon.a (mon.0) 2124 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi047-82386-54"}]: dispatch 2024-02-23T00:46:33.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:32.504537+0000 mgr.y (mgr.14517) 315 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:46:33.452 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:33.472 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:46:33.474 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:33.474 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:46:33.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:32.167696+0000 mon.a (mon.0) 2120 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-41","var": "hit_set_count","val": "8"}]': finished 2024-02-23T00:46:33.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:32.167773+0000 mon.a (mon.0) 2121 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "20"}]': finished 2024-02-23T00:46:33.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:32.167848+0000 mon.a (mon.0) 2122 : cluster [DBG] osdmap e320: 8 total, 8 up, 8 in 2024-02-23T00:46:33.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:32.169726+0000 mon.b (mon.1) 354 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:33.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:32.170447+0000 mon.a (mon.0) 2123 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:46:33.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:32.170722+0000 mon.c (mon.2) 246 : audit [INF] from='client.? 172.21.15.47:0/2888114840' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi047-82386-54"}]: dispatch 2024-02-23T00:46:33.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:32.171044+0000 mon.a (mon.0) 2124 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi047-82386-54"}]: dispatch 2024-02-23T00:46:33.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:32.504537+0000 mgr.y (mgr.14517) 315 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:46:33.575 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:46:33.602 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:33.641 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:46:33.643 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:33.644 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:46:33.676 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:46:33.744 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:46:33.772 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:33.923 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:46:33.926 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:33.927 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:46:34.027 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:46:34.057 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:34.077 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:46:34.079 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:34.080 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:46:34.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:34.174+0000 7fc7d2ffd700 1 -- 172.21.15.47:0/842112939 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v322) v1 ==== 91+0+0 (secure 0 0 0) 0x7fc7cc0093c0 con 0x7fc7dc0fdbd0 2024-02-23T00:46:34.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:34.177+0000 7fc7e4220700 10 librados: watch_flush enter 2024-02-23T00:46:34.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:34.177+0000 7fc7e4220700 10 librados: watch_flush exit 2024-02-23T00:46:34.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:34.177+0000 7fc7e4220700 20 client.41099.objecter shutdown clearing up homeless session... 2024-02-23T00:46:34.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:34.177+0000 7fc7e4220700 10 client.41099.objecter successfully canceled tick 2024-02-23T00:46:34.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:34.177+0000 7fc7e4220700 1 -- 172.21.15.47:0/842112939 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc7c8062cc0 msgr2=0x7fc7c8065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:34.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:34.177+0000 7fc7e4220700 1 --2- 172.21.15.47:0/842112939 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc7c8062cc0 0x7fc7c8065180 secure :-1 s=READY pgs=3022 cs=0 l=1 rev1=1 rx=0x7fc7d80043d0 tx=0x7fc7d8009cb0).stop 2024-02-23T00:46:34.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:34.178+0000 7fc7e4220700 1 -- 172.21.15.47:0/842112939 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7dc0fdbd0 msgr2=0x7fc7dc10e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:34.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:34.178+0000 7fc7e4220700 1 --2- 172.21.15.47:0/842112939 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7dc0fdbd0 0x7fc7dc10e440 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 rx=0x7fc7cc0027f0 tx=0x7fc7cc002b00).stop 2024-02-23T00:46:34.181 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:46:34.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:34.179+0000 7fc7e4220700 1 -- 172.21.15.47:0/842112939 shutdown_connections 2024-02-23T00:46:34.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:34.179+0000 7fc7e4220700 1 --2- 172.21.15.47:0/842112939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7dc0fe530 0x7fc7dc1080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:34.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:34.179+0000 7fc7e4220700 1 --2- 172.21.15.47:0/842112939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7dc0f6a90 0x7fc7dc10df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:46:34.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:34.179+0000 7fc7e4220700 1 --2- 172.21.15.47:0/842112939 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc7c8062cc0 0x7fc7c8065180 secure :-1 s=CLOSED pgs=3022 cs=0 l=1 rev1=1 rx=0x7fc7d80043d0 tx=0x7fc7d8009cb0).stop 2024-02-23T00:46:34.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:34.179+0000 7fc7e4220700 1 --2- 172.21.15.47:0/842112939 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7dc0fdbd0 0x7fc7dc10e440 secure :-1 s=CLOSED pgs=2616 cs=0 l=1 rev1=1 rx=0x7fc7cc0027f0 tx=0x7fc7cc002b00).stop 2024-02-23T00:46:34.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:34.179+0000 7fc7e4220700 1 -- 172.21.15.47:0/842112939 >> 172.21.15.47:0/842112939 conn(0x7fc7dc06c370 msgr2=0x7fc7dc0fb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:46:34.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:34.189+0000 7fc7e4220700 1 -- 172.21.15.47:0/842112939 shutdown_connections 2024-02-23T00:46:34.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:34.189+0000 7fc7e4220700 1 -- 172.21.15.47:0/842112939 wait complete. 2024-02-23T00:46:34.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:34.189+0000 7fc7e4220700 1 librados: shutdown 2024-02-23T00:46:34.192 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is unset 2024-02-23T00:46:34.211 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:34.233 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:46:34.236 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:34.236 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:46:34.337 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:46:34.363 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:33.168787+0000 mon.a (mon.0) 2125 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-41","var": "hit_set_period","val": "600"}]': finished 2024-02-23T00:46:34.363 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:33.168864+0000 mon.a (mon.0) 2126 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi047-82386-54"}]': finished 2024-02-23T00:46:34.363 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:33.168938+0000 mon.a (mon.0) 2127 : cluster [DBG] osdmap e321: 8 total, 8 up, 8 in 2024-02-23T00:46:34.363 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:33.170646+0000 mon.a (mon.0) 2128 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-41","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-23T00:46:34.363 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:33.170940+0000 mon.c (mon.2) 247 : audit [INF] from='client.? 172.21.15.47:0/2888114840' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi047-82386-54"}]: dispatch 2024-02-23T00:46:34.363 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:33.174922+0000 mon.b (mon.1) 355 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:34.363 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:33.176924+0000 mon.a (mon.0) 2129 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi047-82386-54"}]: dispatch 2024-02-23T00:46:34.363 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:33.259356+0000 mgr.y (mgr.14517) 316 : cluster [DBG] pgmap v457: 239 pgs: 2 peering, 237 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:34.363 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:33.408704+0000 mon.c (mon.2) 248 : audit [INF] from='client.? 172.21.15.47:0/842112939' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:46:34.363 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:33.409004+0000 mon.a (mon.0) 2130 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:46:34.363 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:34.166245+0000 mon.a (mon.0) 2131 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:46:34.364 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:34.384 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:46:34.387 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:34.387 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:46:34.425 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (8984 ms) 2024-02-23T00:46:34.425 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-02-23T00:46:34.426 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg_num = 32 2024-02-23T00:46:34.426 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg 0 ls 1708649195,0 2024-02-23T00:46:34.426 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg 1 ls 1708649195,0 2024-02-23T00:46:34.426 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg 2 ls 1708649195,0 2024-02-23T00:46:34.426 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg 3 ls 1708649195,0 2024-02-23T00:46:34.426 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg 4 ls 1708649195,0 2024-02-23T00:46:34.426 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg 5 ls 1708649195,0 2024-02-23T00:46:34.426 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg 6 ls 1708649195,0 2024-02-23T00:46:34.426 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg 7 ls 1708649195,0 2024-02-23T00:46:34.426 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg 8 ls 1708649195,0 2024-02-23T00:46:34.426 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg 9 ls 1708649195,0 2024-02-23T00:46:34.426 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg 10 ls 1708649195,0 2024-02-23T00:46:34.427 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg 11 ls 1708649195,0 2024-02-23T00:46:34.427 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg 12 ls 1708649195,0 2024-02-23T00:46:34.427 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg 13 ls 1708649195,0 2024-02-23T00:46:34.427 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg 14 ls 1708649195,0 2024-02-23T00:46:34.427 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg 15 ls 1708649195,0 2024-02-23T00:46:34.427 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg 16 ls 1708649195,0 2024-02-23T00:46:34.427 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg 17 ls 1708649195,0 2024-02-23T00:46:34.427 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg 18 ls 1708649195,0 2024-02-23T00:46:34.427 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg 19 ls 1708649195,0 2024-02-23T00:46:34.427 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg 20 ls 1708649195,0 2024-02-23T00:46:34.427 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg 21 ls 1708649195,0 2024-02-23T00:46:34.427 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg 22 ls 1708649195,0 2024-02-23T00:46:34.427 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg 23 ls 1708649195,0 2024-02-23T00:46:34.427 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg 24 ls 1708649195,0 2024-02-23T00:46:34.428 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg 25 ls 1708649195,0 2024-02-23T00:46:34.428 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg 26 ls 1708649195,0 2024-02-23T00:46:34.428 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg 27 ls 1708649195,0 2024-02-23T00:46:34.428 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg 28 ls 1708649195,0 2024-02-23T00:46:34.428 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg 29 ls 1708649195,0 2024-02-23T00:46:34.428 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg 30 ls 1708649195,0 2024-02-23T00:46:34.428 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg 31 ls 1708649195,0 2024-02-23T00:46:34.428 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: pg_num = 32 2024-02-23T00:46:34.428 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:6cac518f:::0:head 2024-02-23T00:46:34.428 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:02547ec2:::1:head 2024-02-23T00:46:34.428 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:f905c69b:::2:head 2024-02-23T00:46:34.428 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:cfc208b3:::3:head 2024-02-23T00:46:34.428 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:d83876eb:::4:head 2024-02-23T00:46:34.429 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:b29083e3:::5:head 2024-02-23T00:46:34.429 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:c4fdafeb:::6:head 2024-02-23T00:46:34.429 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:5c6b0b28:::7:head 2024-02-23T00:46:34.429 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:bd63b0f1:::8:head 2024-02-23T00:46:34.429 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:e960b815:::9:head 2024-02-23T00:46:34.429 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:52ea6a34:::10:head 2024-02-23T00:46:34.429 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:89d3ae78:::11:head 2024-02-23T00:46:34.429 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:de5d7c5f:::12:head 2024-02-23T00:46:34.429 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:566253c9:::13:head 2024-02-23T00:46:34.429 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:62a1935d:::14:head 2024-02-23T00:46:34.429 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:863748b0:::15:head 2024-02-23T00:46:34.429 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:3958e169:::16:head 2024-02-23T00:46:34.429 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:4d4dabf9:::17:head 2024-02-23T00:46:34.429 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:8391935d:::18:head 2024-02-23T00:46:34.430 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:28883081:::19:head 2024-02-23T00:46:34.430 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:69259c59:::20:head 2024-02-23T00:46:34.430 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:4bdb80b7:::21:head 2024-02-23T00:46:34.430 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:a11c5d71:::22:head 2024-02-23T00:46:34.430 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:271af37b:::23:head 2024-02-23T00:46:34.430 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:95b121be:::24:head 2024-02-23T00:46:34.430 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:58d1031b:::25:head 2024-02-23T00:46:34.430 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:0a050783:::26:head 2024-02-23T00:46:34.430 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:c709704c:::27:head 2024-02-23T00:46:34.430 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:cbe56eaf:::28:head 2024-02-23T00:46:34.430 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:86b4b162:::29:head 2024-02-23T00:46:34.430 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:70d89383:::30:head 2024-02-23T00:46:34.430 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:dd450c7c:::31:head 2024-02-23T00:46:34.431 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:6d5729b1:::32:head 2024-02-23T00:46:34.431 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:c388f3fb:::33:head 2024-02-23T00:46:34.431 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:56cfea31:::34:head 2024-02-23T00:46:34.431 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:9dbc1bf7:::35:head 2024-02-23T00:46:34.431 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:40b74ccd:::36:head 2024-02-23T00:46:34.431 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:4d5aaf42:::37:head 2024-02-23T00:46:34.431 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:920f362c:::38:head 2024-02-23T00:46:34.431 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:6cc53222:::39:head 2024-02-23T00:46:34.431 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:9cad833f:::40:head 2024-02-23T00:46:34.431 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:1ea84d41:::41:head 2024-02-23T00:46:34.431 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:c4480ef6:::42:head 2024-02-23T00:46:34.431 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:a694361e:::43:head 2024-02-23T00:46:34.431 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:d1bd33e9:::44:head 2024-02-23T00:46:34.432 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:ddc2cd5d:::45:head 2024-02-23T00:46:34.432 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:2b782207:::46:head 2024-02-23T00:46:34.432 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:7b187fca:::47:head 2024-02-23T00:46:34.432 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:90ecdf6f:::48:head 2024-02-23T00:46:34.432 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:a5ed95fe:::49:head 2024-02-23T00:46:34.432 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:ea0eaa55:::50:head 2024-02-23T00:46:34.432 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:f33ef17b:::51:head 2024-02-23T00:46:34.432 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:a0d1b2f6:::52:head 2024-02-23T00:46:34.432 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:60c5229e:::53:head 2024-02-23T00:46:34.432 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:edcbc575:::54:head 2024-02-23T00:46:34.432 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:102cf253:::55:head 2024-02-23T00:46:34.432 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:efb7fb0b:::56:head 2024-02-23T00:46:34.432 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:50d0a326:::57:head 2024-02-23T00:46:34.433 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:d4dc5daf:::58:head 2024-02-23T00:46:34.433 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:3a130462:::59:head 2024-02-23T00:46:34.433 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:ec87ed71:::60:head 2024-02-23T00:46:34.433 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:d5bc9454:::61:head 2024-02-23T00:46:34.433 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:3ddfe313:::62:head 2024-02-23T00:46:34.433 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:7c2816b9:::63:head 2024-02-23T00:46:34.433 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:47e00e4d:::64:head 2024-02-23T00:46:34.433 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:c6410c18:::65:head 2024-02-23T00:46:34.433 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:b48ed237:::66:head 2024-02-23T00:46:34.433 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:cd63ad31:::67:head 2024-02-23T00:46:34.433 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:b179e92b:::68:head 2024-02-23T00:46:34.433 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:0d9f741a:::69:head 2024-02-23T00:46:34.433 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:6d3352ae:::70:head 2024-02-23T00:46:34.434 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:c6d5c19e:::71:head 2024-02-23T00:46:34.434 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:bc4729c3:::72:head 2024-02-23T00:46:34.434 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:77e930b9:::73:head 2024-02-23T00:46:34.434 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:0abeecfd:::74:head 2024-02-23T00:46:34.434 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:b7c37e15:::75:head 2024-02-23T00:46:34.434 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:b6378398:::76:head 2024-02-23T00:46:34.434 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:02bd68de:::77:head 2024-02-23T00:46:34.434 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:cc795d2d:::78:head 2024-02-23T00:46:34.434 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:630d4fea:::79:head 2024-02-23T00:46:34.434 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:e0d29ef5:::80:head 2024-02-23T00:46:34.434 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:fd6f13d2:::81:head 2024-02-23T00:46:34.434 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:606461d5:::82:head 2024-02-23T00:46:34.434 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:eadbdc43:::83:head 2024-02-23T00:46:34.434 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:8761d0bb:::84:head 2024-02-23T00:46:34.434 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:9ef0186f:::85:head 2024-02-23T00:46:34.435 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:e0d41294:::86:head 2024-02-23T00:46:34.435 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:961de695:::87:head 2024-02-23T00:46:34.435 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:1423148f:::88:head 2024-02-23T00:46:34.435 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:633a8fa2:::89:head 2024-02-23T00:46:34.435 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:a8653809:::90:head 2024-02-23T00:46:34.435 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:3dac8b33:::91:head 2024-02-23T00:46:34.435 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:35aad435:::92:head 2024-02-23T00:46:34.435 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:f6dcc343:::93:head 2024-02-23T00:46:34.435 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:dbbdad87:::94:head 2024-02-23T00:46:34.435 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:1cb48ce0:::95:head 2024-02-23T00:46:34.435 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:03cd461c:::96:head 2024-02-23T00:46:34.435 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:17a4ea99:::97:head 2024-02-23T00:46:34.435 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:9993c9a7:::98:head 2024-02-23T00:46:34.435 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:6394211c:::99:head 2024-02-23T00:46:34.436 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:94c7ae57:::100:head 2024-02-23T00:46:34.436 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:6fdee5bb:::101:head 2024-02-23T00:46:34.436 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:9a477fd1:::102:head 2024-02-23T00:46:34.436 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:eb850916:::103:head 2024-02-23T00:46:34.436 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:affc56b9:::104:head 2024-02-23T00:46:34.436 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:b42dc814:::105:head 2024-02-23T00:46:34.436 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:f319f8f0:::106:head 2024-02-23T00:46:34.436 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:9a40b9de:::107:head 2024-02-23T00:46:34.436 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:8b524f28:::108:head 2024-02-23T00:46:34.436 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:e3de589f:::109:head 2024-02-23T00:46:34.436 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:90f90a5b:::110:head 2024-02-23T00:46:34.436 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:a7b4f1d7:::111:head 2024-02-23T00:46:34.436 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:af51766e:::112:head 2024-02-23T00:46:34.436 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:b6f90bd1:::113:head 2024-02-23T00:46:34.436 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:e0261208:::114:head 2024-02-23T00:46:34.437 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:c9569ef7:::115:head 2024-02-23T00:46:34.437 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:61bebe50:::116:head 2024-02-23T00:46:34.437 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:fe93412b:::117:head 2024-02-23T00:46:34.437 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:d3d38bee:::118:head 2024-02-23T00:46:34.437 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:3100ba0c:::119:head 2024-02-23T00:46:34.437 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:d0560ada:::120:head 2024-02-23T00:46:34.437 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:f0ea8b35:::121:head 2024-02-23T00:46:34.437 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:766f231a:::122:head 2024-02-23T00:46:34.437 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:a07a2582:::123:head 2024-02-23T00:46:34.437 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:bd7c6b3a:::124:head 2024-02-23T00:46:34.437 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:fb2ddaff:::125:head 2024-02-23T00:46:34.437 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:4408e1fe:::126:head 2024-02-23T00:46:34.437 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:ee1df7a7:::127:head 2024-02-23T00:46:34.437 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:c3002909:::128:head 2024-02-23T00:46:34.438 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:4f48ffa9:::129:head 2024-02-23T00:46:34.438 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:edf38733:::130:head 2024-02-23T00:46:34.438 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:c08425c0:::131:head 2024-02-23T00:46:34.438 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:5f902d98:::132:head 2024-02-23T00:46:34.438 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:41ea2c93:::133:head 2024-02-23T00:46:34.438 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:813cee13:::134:head 2024-02-23T00:46:34.438 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:0131818d:::135:head 2024-02-23T00:46:34.438 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:26ba5a85:::136:head 2024-02-23T00:46:34.438 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:381b8a5a:::137:head 2024-02-23T00:46:34.438 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:28797e47:::138:head 2024-02-23T00:46:34.438 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:bfca7f22:::139:head 2024-02-23T00:46:34.438 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:36807075:::140:head 2024-02-23T00:46:34.438 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:80b03975:::141:head 2024-02-23T00:46:34.438 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:5c15709b:::142:head 2024-02-23T00:46:34.439 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:f39ea15e:::143:head 2024-02-23T00:46:34.439 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:ea992956:::144:head 2024-02-23T00:46:34.439 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:48887b1c:::145:head 2024-02-23T00:46:34.439 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:9f24a9dd:::146:head 2024-02-23T00:46:34.439 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:987f100b:::147:head 2024-02-23T00:46:34.439 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:d2dd3581:::148:head 2024-02-23T00:46:34.439 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:7fed1808:::149:head 2024-02-23T00:46:34.439 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:c80b70e9:::150:head 2024-02-23T00:46:34.439 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:85ed90f9:::151:head 2024-02-23T00:46:34.439 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:36428b24:::152:head 2024-02-23T00:46:34.439 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:d044c34a:::153:head 2024-02-23T00:46:34.439 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:7c18bf58:::154:head 2024-02-23T00:46:34.439 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:d1c21232:::155:head 2024-02-23T00:46:34.439 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:a7a3c575:::156:head 2024-02-23T00:46:34.440 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:87da0633:::157:head 2024-02-23T00:46:34.440 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:d5ac3822:::158:head 2024-02-23T00:46:34.440 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:3f20522d:::159:head 2024-02-23T00:46:34.440 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:6ca26563:::160:head 2024-02-23T00:46:34.440 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:532ce135:::161:head 2024-02-23T00:46:34.440 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:c78863e6:::162:head 2024-02-23T00:46:34.440 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:7462ddf6:::163:head 2024-02-23T00:46:34.440 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:5d165639:::164:head 2024-02-23T00:46:34.440 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:f43765fc:::165:head 2024-02-23T00:46:34.487 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:46:34.517 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:33.168787+0000 mon.a (mon.0) 2125 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-41","var": "hit_set_period","val": "600"}]': finished 2024-02-23T00:46:34.517 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:33.168864+0000 mon.a (mon.0) 2126 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi047-82386-54"}]': finished 2024-02-23T00:46:34.517 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:33.168938+0000 mon.a (mon.0) 2127 : cluster [DBG] osdmap e321: 8 total, 8 up, 8 in 2024-02-23T00:46:34.517 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:33.170646+0000 mon.a (mon.0) 2128 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-41","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-23T00:46:34.518 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:33.170940+0000 mon.c (mon.2) 247 : audit [INF] from='client.? 172.21.15.47:0/2888114840' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi047-82386-54"}]: dispatch 2024-02-23T00:46:34.518 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:33.174922+0000 mon.b (mon.1) 355 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:34.518 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:33.176924+0000 mon.a (mon.0) 2129 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi047-82386-54"}]: dispatch 2024-02-23T00:46:34.518 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:33.259356+0000 mgr.y (mgr.14517) 316 : cluster [DBG] pgmap v457: 239 pgs: 2 peering, 237 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:34.518 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:33.408704+0000 mon.c (mon.2) 248 : audit [INF] from='client.? 172.21.15.47:0/842112939' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:46:34.518 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:33.409004+0000 mon.a (mon.0) 2130 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:46:34.518 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:34.166245+0000 mon.a (mon.0) 2131 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:46:34.518 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:33.168787+0000 mon.a (mon.0) 2125 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-41","var": "hit_set_period","val": "600"}]': finished 2024-02-23T00:46:34.518 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:33.168864+0000 mon.a (mon.0) 2126 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi047-82386-54"}]': finished 2024-02-23T00:46:34.519 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:33.168938+0000 mon.a (mon.0) 2127 : cluster [DBG] osdmap e321: 8 total, 8 up, 8 in 2024-02-23T00:46:34.519 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:33.170646+0000 mon.a (mon.0) 2128 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-41","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-23T00:46:34.519 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:33.170940+0000 mon.c (mon.2) 247 : audit [INF] from='client.? 172.21.15.47:0/2888114840' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi047-82386-54"}]: dispatch 2024-02-23T00:46:34.519 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:33.174922+0000 mon.b (mon.1) 355 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:34.519 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:33.176924+0000 mon.a (mon.0) 2129 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi047-82386-54"}]: dispatch 2024-02-23T00:46:34.519 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:33.259356+0000 mgr.y (mgr.14517) 316 : cluster [DBG] pgmap v457: 239 pgs: 2 peering, 237 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:34.519 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:33.408704+0000 mon.c (mon.2) 248 : audit [INF] from='client.? 172.21.15.47:0/842112939' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:46:34.519 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:33.409004+0000 mon.a (mon.0) 2130 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:46:34.519 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:34.166245+0000 mon.a (mon.0) 2131 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:46:34.520 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:34.594 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:46:34.597 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:34.598 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:46:34.699 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:46:34.725 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:34.744 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:46:34.746 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:34.747 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:46:34.847 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:46:34.876 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:35.076 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:46:35.079 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:35.079 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:46:35.180 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:46:35.206 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:35.228 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:46:35.231 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:35.231 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:46:35.332 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:46:35.363 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:34.170712+0000 mon.a (mon.0) 2132 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-41","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-23T00:46:35.363 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:34.170789+0000 mon.a (mon.0) 2133 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi047-82386-54"}]': finished 2024-02-23T00:46:35.363 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:34.170873+0000 mon.a (mon.0) 2134 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:46:35.363 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:34.170926+0000 mon.a (mon.0) 2135 : cluster [DBG] osdmap e322: 8 total, 8 up, 8 in 2024-02-23T00:46:35.364 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:34.175866+0000 mon.b (mon.1) 356 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:35.364 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:34.207128+0000 mon.a (mon.0) 2136 : audit [INF] from='client.? 172.21.15.47:0/427115143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi047-82386-55"}]: dispatch 2024-02-23T00:46:35.364 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:34.207921+0000 mon.a (mon.0) 2137 : audit [INF] from='client.? 172.21.15.47:0/427115143' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi047-82386-55"}]: dispatch 2024-02-23T00:46:35.364 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:34.208504+0000 mon.a (mon.0) 2138 : audit [INF] from='client.? 172.21.15.47:0/427115143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi047-82386-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:35.364 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:34.424224+0000 mon.a (mon.0) 2139 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi047-82465-41","var": "pg_num","format": "json"}]: dispatch 2024-02-23T00:46:35.364 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:34.538421+0000 mon.a (mon.0) 2140 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:46:35.364 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:34.538761+0000 mon.a (mon.0) 2141 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-41"}]: dispatch 2024-02-23T00:46:35.364 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:34.718666+0000 osd.7 (osd.7) 163 : cluster [DBG] 6.15 scrub starts 2024-02-23T00:46:35.364 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:34.721560+0000 osd.7 (osd.7) 164 : cluster [DBG] 6.15 scrub ok 2024-02-23T00:46:35.364 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:35.151924+0000 mon.a (mon.0) 2142 : audit [INF] from='client.? 172.21.15.47:0/427115143' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi047-82386-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:35.364 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:35.151980+0000 mon.a (mon.0) 2143 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-41"}]': finished 2024-02-23T00:46:35.364 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:35.152023+0000 mon.a (mon.0) 2144 : cluster [DBG] osdmap e323: 8 total, 8 up, 8 in 2024-02-23T00:46:35.365 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:35.158334+0000 mon.a (mon.0) 2145 : audit [INF] from='client.? 172.21.15.47:0/427115143' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi047-82386-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi047-82386-55"}]: dispatch 2024-02-23T00:46:35.365 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:35.177266+0000 mon.b (mon.1) 357 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:35.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:34.170712+0000 mon.a (mon.0) 2132 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-41","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-23T00:46:35.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:34.170789+0000 mon.a (mon.0) 2133 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi047-82386-54"}]': finished 2024-02-23T00:46:35.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:34.170873+0000 mon.a (mon.0) 2134 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:46:35.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:34.170926+0000 mon.a (mon.0) 2135 : cluster [DBG] osdmap e322: 8 total, 8 up, 8 in 2024-02-23T00:46:35.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:34.175866+0000 mon.b (mon.1) 356 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:35.366 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:34.207128+0000 mon.a (mon.0) 2136 : audit [INF] from='client.? 172.21.15.47:0/427115143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi047-82386-55"}]: dispatch 2024-02-23T00:46:35.366 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:34.207921+0000 mon.a (mon.0) 2137 : audit [INF] from='client.? 172.21.15.47:0/427115143' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi047-82386-55"}]: dispatch 2024-02-23T00:46:35.366 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:34.208504+0000 mon.a (mon.0) 2138 : audit [INF] from='client.? 172.21.15.47:0/427115143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi047-82386-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:35.366 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:34.424224+0000 mon.a (mon.0) 2139 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi047-82465-41","var": "pg_num","format": "json"}]: dispatch 2024-02-23T00:46:35.366 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:34.538421+0000 mon.a (mon.0) 2140 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:46:35.366 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:34.538761+0000 mon.a (mon.0) 2141 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-41"}]: dispatch 2024-02-23T00:46:35.366 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:34.718666+0000 osd.7 (osd.7) 163 : cluster [DBG] 6.15 scrub starts 2024-02-23T00:46:35.366 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:34.721560+0000 osd.7 (osd.7) 164 : cluster [DBG] 6.15 scrub ok 2024-02-23T00:46:35.366 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:35.151924+0000 mon.a (mon.0) 2142 : audit [INF] from='client.? 172.21.15.47:0/427115143' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi047-82386-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:35.366 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:35.151980+0000 mon.a (mon.0) 2143 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-41"}]': finished 2024-02-23T00:46:35.366 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:35.152023+0000 mon.a (mon.0) 2144 : cluster [DBG] osdmap e323: 8 total, 8 up, 8 in 2024-02-23T00:46:35.366 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:35.158334+0000 mon.a (mon.0) 2145 : audit [INF] from='client.? 172.21.15.47:0/427115143' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi047-82386-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi047-82386-55"}]: dispatch 2024-02-23T00:46:35.366 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:35.177266+0000 mon.b (mon.1) 357 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:35.367 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:35.389 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:46:35.392 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:35.392 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:46:35.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.401+0000 7ff03faff700 1 -- 172.21.15.47:0/3824935204 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff0380fcbb0 msgr2=0x7ff0380fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:35.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.401+0000 7ff03faff700 1 --2- 172.21.15.47:0/3824935204 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff0380fcbb0 0x7ff0380fcf90 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 rx=0x7ff034004870 tx=0x7ff034009cb0).stop 2024-02-23T00:46:35.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.401+0000 7ff03faff700 1 -- 172.21.15.47:0/3824935204 shutdown_connections 2024-02-23T00:46:35.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.401+0000 7ff03faff700 1 --2- 172.21.15.47:0/3824935204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0380f7400 0x7ff038103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:35.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.401+0000 7ff03faff700 1 --2- 172.21.15.47:0/3824935204 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff0380fcbb0 0x7ff0380fcf90 unknown :-1 s=CLOSED pgs=2617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:35.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.401+0000 7ff03faff700 1 --2- 172.21.15.47:0/3824935204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0380fd510 0x7ff0380f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:35.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.401+0000 7ff03faff700 1 -- 172.21.15.47:0/3824935204 >> 172.21.15.47:0/3824935204 conn(0x7ff0380f2820 msgr2=0x7ff0380f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:46:35.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.401+0000 7ff03faff700 1 -- 172.21.15.47:0/3824935204 shutdown_connections 2024-02-23T00:46:35.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.401+0000 7ff03faff700 1 -- 172.21.15.47:0/3824935204 wait complete. 2024-02-23T00:46:35.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.402+0000 7ff03faff700 1 librados: starting msgr at 2024-02-23T00:46:35.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.402+0000 7ff03faff700 1 librados: starting objecter 2024-02-23T00:46:35.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.402+0000 7ff03faff700 1 Processor -- start 2024-02-23T00:46:35.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.402+0000 7ff03faff700 1 -- start start 2024-02-23T00:46:35.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.402+0000 7ff03faff700 1 librados: setting wanted keys 2024-02-23T00:46:35.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.402+0000 7ff03faff700 1 librados: calling monclient init 2024-02-23T00:46:35.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.402+0000 7ff03faff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0380f7400 0x7ff03810cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:35.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.402+0000 7ff03faff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0380fcbb0 0x7ff03810d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:35.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.402+0000 7ff03faff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff0380fd510 0x7ff038107120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:35.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.402+0000 7ff03faff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff038107610 con 0x7ff0380fcbb0 2024-02-23T00:46:35.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.402+0000 7ff03faff700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff038107780 con 0x7ff0380f7400 2024-02-23T00:46:35.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.402+0000 7ff03faff700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff038107940 con 0x7ff0380fd510 2024-02-23T00:46:35.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.403+0000 7ff03d09a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0380fcbb0 0x7ff03810d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:35.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.403+0000 7ff03d09a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0380fcbb0 0x7ff03810d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43836/0 (socket says 172.21.15.47:43836) 2024-02-23T00:46:35.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.403+0000 7ff03d09a700 1 -- 172.21.15.47:0/3832319785 learned_addr learned my addr 172.21.15.47:0/3832319785 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:46:35.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.403+0000 7ff03d89b700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0380f7400 0x7ff03810cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:35.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.403+0000 7ff03e09c700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff0380fd510 0x7ff038107120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:35.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.403+0000 7ff03d09a700 1 -- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff0380fd510 msgr2=0x7ff038107120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:35.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.403+0000 7ff03d09a700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff0380fd510 0x7ff038107120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:35.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.403+0000 7ff03d09a700 1 -- 172.21.15.47:0/3832319785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0380f7400 msgr2=0x7ff03810cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:35.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.403+0000 7ff03d09a700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0380f7400 0x7ff03810cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:35.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.403+0000 7ff03d09a700 1 -- 172.21.15.47:0/3832319785 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0300097c0 con 0x7ff0380fcbb0 2024-02-23T00:46:35.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.403+0000 7ff03e09c700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff0380fd510 0x7ff038107120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:46:35.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.403+0000 7ff03d89b700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0380f7400 0x7ff03810cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:46:35.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.403+0000 7ff03d09a700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0380fcbb0 0x7ff03810d470 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 rx=0x7ff028010ba0 tx=0x7ff02800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:35.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.403+0000 7ff02effd700 10 client.?.objecter ms_handle_connect 0x7ff0380fcbb0 2024-02-23T00:46:35.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.403+0000 7ff02effd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:46:35.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.403+0000 7ff02effd700 1 -- 172.21.15.47:0/3832319785 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff028004680 con 0x7ff0380fcbb0 2024-02-23T00:46:35.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.403+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff034005040 con 0x7ff0380fcbb0 2024-02-23T00:46:35.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.403+0000 7ff02effd700 1 -- 172.21.15.47:0/3832319785 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7ff02800e890 con 0x7ff0380fcbb0 2024-02-23T00:46:35.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.403+0000 7ff02effd700 1 -- 172.21.15.47:0/3832319785 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff028009500 con 0x7ff0380fcbb0 2024-02-23T00:46:35.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.403+0000 7ff03faff700 10 client.30774.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:46:35.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.403+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff038107f20 con 0x7ff0380fcbb0 2024-02-23T00:46:35.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.403+0000 7ff03faff700 1 librados: init done 2024-02-23T00:46:35.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.404+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7ff03805fa60 con 0x7ff0380fcbb0 2024-02-23T00:46:35.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.405+0000 7ff02effd700 1 -- 172.21.15.47:0/3832319785 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7ff028007b40 con 0x7ff0380fcbb0 2024-02-23T00:46:35.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.405+0000 7ff02effd700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff024062cc0 0x7ff024065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:35.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.405+0000 7ff02effd700 1 -- 172.21.15.47:0/3832319785 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(323..323 src has 48..323) v4 ==== 9846+0+0 (secure 0 0 0) 0x7ff02807f350 con 0x7ff0380fcbb0 2024-02-23T00:46:35.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.405+0000 7ff02effd700 10 client.30774.objecter ms_dispatch 0x7ff038001550 osd_map(323..323 src has 48..323) v4 2024-02-23T00:46:35.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.405+0000 7ff02effd700 3 client.30774.objecter handle_osd_map got epochs [323,323] > 0 2024-02-23T00:46:35.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.405+0000 7ff02effd700 3 client.30774.objecter handle_osd_map decoding full epoch 323 2024-02-23T00:46:35.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.406+0000 7ff02effd700 20 client.30774.objecter dump_active .. 0 homeless 2024-02-23T00:46:35.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.406+0000 7ff02effd700 1 -- 172.21.15.47:0/3832319785 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v323) v1 ==== 54+0+16 (secure 0 0 0) 0x7ff028086070 con 0x7ff0380fcbb0 2024-02-23T00:46:35.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.406+0000 7ff03d89b700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff024062cc0 0x7ff024065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:35.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.406+0000 7ff03faff700 10 client.30774.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:35.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.406+0000 7ff03faff700 15 client.30774.objecter _session_command_op_assign -1 1 2024-02-23T00:46:35.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.406+0000 7ff03faff700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7ff0380273c0 0x7ff0380277a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:35.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.406+0000 7ff03faff700 20 client.30774.objecter _get_session s=0x7ff038105ee0 osd=0 3 2024-02-23T00:46:35.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.406+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff038105ee0 osd=0 3 2024-02-23T00:46:35.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.406+0000 7ff03faff700 20 client.30774.objecter _get_session s=0x7ff038105ee0 osd=0 3 2024-02-23T00:46:35.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.406+0000 7ff03faff700 15 client.30774.objecter _session_command_op_remove -1 1 2024-02-23T00:46:35.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.406+0000 7ff03faff700 20 client.30774.objecter get_session s=0x7ff038105ee0 osd=0 3 2024-02-23T00:46:35.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.406+0000 7ff03faff700 15 client.30774.objecter _session_command_op_assign 0 1 2024-02-23T00:46:35.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.406+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff038105ee0 osd=0 4 2024-02-23T00:46:35.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.406+0000 7ff03faff700 10 client.30774.objecter _send_command 1 2024-02-23T00:46:35.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.406+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff0380fef70 con 0x7ff0380273c0 2024-02-23T00:46:35.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.406+0000 7ff03e09c700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7ff0380273c0 0x7ff0380277a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:35.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.406+0000 7ff02effd700 10 client.30774.objecter ms_handle_connect 0x7ff024062cc0 2024-02-23T00:46:35.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.406+0000 7ff03d89b700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff024062cc0 0x7ff024065180 secure :-1 s=READY pgs=3024 cs=0 l=1 rev1=1 rx=0x7ff034009da0 tx=0x7ff03400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:35.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.406+0000 7ff03e09c700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7ff0380273c0 0x7ff0380277a0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:35.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.406+0000 7ff02effd700 10 client.30774.objecter ms_handle_connect 0x7ff0380273c0 2024-02-23T00:46:35.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.407+0000 7ff02effd700 1 -- 172.21.15.47:0/3832319785 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff0380fef70 con 0x7ff0380273c0 2024-02-23T00:46:35.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.407+0000 7ff02effd700 10 client.30774.objecter ms_dispatch 0x7ff038001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:46:35.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.407+0000 7ff02effd700 10 client.30774.objecter _finish_command 1 = system:0 2024-02-23T00:46:35.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.407+0000 7ff02effd700 20 client.30774.objecter put_session s=0x7ff038105ee0 osd=0 4 2024-02-23T00:46:35.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.407+0000 7ff02effd700 15 client.30774.objecter _session_command_op_remove 0 1 2024-02-23T00:46:35.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.424+0000 7ff03faff700 10 client.30774.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:46:35.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.424+0000 7ff03faff700 15 client.30774.objecter _session_command_op_assign -1 2 2024-02-23T00:46:35.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.424+0000 7ff03faff700 20 client.30774.objecter _get_session s=0x7ff038105ee0 osd=0 3 2024-02-23T00:46:35.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.424+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff038105ee0 osd=0 3 2024-02-23T00:46:35.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.424+0000 7ff03faff700 20 client.30774.objecter _get_session s=0x7ff038105ee0 osd=0 3 2024-02-23T00:46:35.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.424+0000 7ff03faff700 15 client.30774.objecter _session_command_op_remove -1 2 2024-02-23T00:46:35.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.424+0000 7ff03faff700 20 client.30774.objecter get_session s=0x7ff038105ee0 osd=0 3 2024-02-23T00:46:35.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.424+0000 7ff03faff700 15 client.30774.objecter _session_command_op_assign 0 2 2024-02-23T00:46:35.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.424+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff038105ee0 osd=0 4 2024-02-23T00:46:35.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.424+0000 7ff03faff700 10 client.30774.objecter _send_command 2 2024-02-23T00:46:35.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.424+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff038061d30 con 0x7ff0380273c0 2024-02-23T00:46:35.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.424+0000 7ff02effd700 1 -- 172.21.15.47:0/3832319785 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff038061d30 con 0x7ff0380273c0 2024-02-23T00:46:35.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.424+0000 7ff02effd700 10 client.30774.objecter ms_dispatch 0x7ff038001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:46:35.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.424+0000 7ff02effd700 10 client.30774.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:46:35.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.424+0000 7ff02effd700 20 client.30774.objecter put_session s=0x7ff038105ee0 osd=0 4 2024-02-23T00:46:35.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.424+0000 7ff02effd700 15 client.30774.objecter _session_command_op_remove 0 2 2024-02-23T00:46:35.427 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:46:35.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.424+0000 7ff03faff700 10 client.30774.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:35.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.424+0000 7ff03faff700 15 client.30774.objecter _session_command_op_assign -1 3 2024-02-23T00:46:35.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.424+0000 7ff03faff700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7ff038061050 0x7ff0380fd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:35.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.424+0000 7ff03faff700 20 client.30774.objecter _get_session s=0x7ff0380ff0e0 osd=1 3 2024-02-23T00:46:35.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.424+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff0380ff0e0 osd=1 3 2024-02-23T00:46:35.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.424+0000 7ff03faff700 20 client.30774.objecter _get_session s=0x7ff0380ff0e0 osd=1 3 2024-02-23T00:46:35.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.424+0000 7ff03faff700 15 client.30774.objecter _session_command_op_remove -1 3 2024-02-23T00:46:35.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.424+0000 7ff03faff700 20 client.30774.objecter get_session s=0x7ff0380ff0e0 osd=1 3 2024-02-23T00:46:35.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.424+0000 7ff03faff700 15 client.30774.objecter _session_command_op_assign 1 3 2024-02-23T00:46:35.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.424+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff0380ff0e0 osd=1 4 2024-02-23T00:46:35.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.424+0000 7ff03faff700 10 client.30774.objecter _send_command 3 2024-02-23T00:46:35.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.424+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff038061430 con 0x7ff038061050 2024-02-23T00:46:35.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.425+0000 7ff03d89b700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7ff038061050 0x7ff0380fd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:35.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.425+0000 7ff02effd700 10 client.30774.objecter ms_handle_connect 0x7ff038061050 2024-02-23T00:46:35.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.425+0000 7ff03d89b700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7ff038061050 0x7ff0380fd990 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:35.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.425+0000 7ff02effd700 1 -- 172.21.15.47:0/3832319785 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff038061430 con 0x7ff038061050 2024-02-23T00:46:35.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.425+0000 7ff02effd700 10 client.30774.objecter ms_dispatch 0x7ff038001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:46:35.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.425+0000 7ff02effd700 10 client.30774.objecter _finish_command 3 = system:0 2024-02-23T00:46:35.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.425+0000 7ff02effd700 20 client.30774.objecter put_session s=0x7ff0380ff0e0 osd=1 4 2024-02-23T00:46:35.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.425+0000 7ff02effd700 15 client.30774.objecter _session_command_op_remove 1 3 2024-02-23T00:46:35.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.440+0000 7ff03faff700 10 client.30774.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:46:35.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.440+0000 7ff03faff700 15 client.30774.objecter _session_command_op_assign -1 4 2024-02-23T00:46:35.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.440+0000 7ff03faff700 20 client.30774.objecter _get_session s=0x7ff0380ff0e0 osd=1 3 2024-02-23T00:46:35.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.440+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff0380ff0e0 osd=1 3 2024-02-23T00:46:35.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.440+0000 7ff03faff700 20 client.30774.objecter _get_session s=0x7ff0380ff0e0 osd=1 3 2024-02-23T00:46:35.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.440+0000 7ff03faff700 15 client.30774.objecter _session_command_op_remove -1 4 2024-02-23T00:46:35.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.440+0000 7ff03faff700 20 client.30774.objecter get_session s=0x7ff0380ff0e0 osd=1 3 2024-02-23T00:46:35.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.440+0000 7ff03faff700 15 client.30774.objecter _session_command_op_assign 1 4 2024-02-23T00:46:35.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.440+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff0380ff0e0 osd=1 4 2024-02-23T00:46:35.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.440+0000 7ff03faff700 10 client.30774.objecter _send_command 4 2024-02-23T00:46:35.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.440+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff038027ed0 con 0x7ff038061050 2024-02-23T00:46:35.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.442+0000 7ff02effd700 1 -- 172.21.15.47:0/3832319785 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff038027ed0 con 0x7ff038061050 2024-02-23T00:46:35.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.442+0000 7ff02effd700 10 client.30774.objecter ms_dispatch 0x7ff038001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:46:35.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.442+0000 7ff02effd700 10 client.30774.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:46:35.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.442+0000 7ff02effd700 20 client.30774.objecter put_session s=0x7ff0380ff0e0 osd=1 4 2024-02-23T00:46:35.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.442+0000 7ff02effd700 15 client.30774.objecter _session_command_op_remove 1 4 2024-02-23T00:46:35.445 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:46:35.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.442+0000 7ff03faff700 10 client.30774.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:35.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.442+0000 7ff03faff700 15 client.30774.objecter _session_command_op_assign -1 5 2024-02-23T00:46:35.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.442+0000 7ff03faff700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7ff0380feb60 0x7ff038119de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:35.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.442+0000 7ff03faff700 20 client.30774.objecter _get_session s=0x7ff0380282e0 osd=2 3 2024-02-23T00:46:35.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.442+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff0380282e0 osd=2 3 2024-02-23T00:46:35.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.442+0000 7ff03faff700 20 client.30774.objecter _get_session s=0x7ff0380282e0 osd=2 3 2024-02-23T00:46:35.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.442+0000 7ff03faff700 15 client.30774.objecter _session_command_op_remove -1 5 2024-02-23T00:46:35.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.442+0000 7ff03faff700 20 client.30774.objecter get_session s=0x7ff0380282e0 osd=2 3 2024-02-23T00:46:35.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.442+0000 7ff03faff700 15 client.30774.objecter _session_command_op_assign 2 5 2024-02-23T00:46:35.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.442+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff0380282e0 osd=2 4 2024-02-23T00:46:35.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.442+0000 7ff03faff700 10 client.30774.objecter _send_command 5 2024-02-23T00:46:35.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.442+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff03811cac0 con 0x7ff0380feb60 2024-02-23T00:46:35.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.442+0000 7ff03d09a700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7ff0380feb60 0x7ff038119de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:35.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.442+0000 7ff03d09a700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7ff0380feb60 0x7ff038119de0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:35.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.442+0000 7ff02effd700 10 client.30774.objecter ms_handle_connect 0x7ff0380feb60 2024-02-23T00:46:35.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.443+0000 7ff02effd700 1 -- 172.21.15.47:0/3832319785 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff03811cac0 con 0x7ff0380feb60 2024-02-23T00:46:35.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.443+0000 7ff02effd700 10 client.30774.objecter ms_dispatch 0x7ff038001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:46:35.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.443+0000 7ff02effd700 10 client.30774.objecter _finish_command 5 = system:0 2024-02-23T00:46:35.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.443+0000 7ff02effd700 20 client.30774.objecter put_session s=0x7ff0380282e0 osd=2 4 2024-02-23T00:46:35.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.443+0000 7ff02effd700 15 client.30774.objecter _session_command_op_remove 2 5 2024-02-23T00:46:35.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.457+0000 7ff03faff700 10 client.30774.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:46:35.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.457+0000 7ff03faff700 15 client.30774.objecter _session_command_op_assign -1 6 2024-02-23T00:46:35.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.457+0000 7ff03faff700 20 client.30774.objecter _get_session s=0x7ff0380282e0 osd=2 3 2024-02-23T00:46:35.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.457+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff0380282e0 osd=2 3 2024-02-23T00:46:35.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.457+0000 7ff03faff700 20 client.30774.objecter _get_session s=0x7ff0380282e0 osd=2 3 2024-02-23T00:46:35.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.457+0000 7ff03faff700 15 client.30774.objecter _session_command_op_remove -1 6 2024-02-23T00:46:35.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.457+0000 7ff03faff700 20 client.30774.objecter get_session s=0x7ff0380282e0 osd=2 3 2024-02-23T00:46:35.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.457+0000 7ff03faff700 15 client.30774.objecter _session_command_op_assign 2 6 2024-02-23T00:46:35.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.457+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff0380282e0 osd=2 4 2024-02-23T00:46:35.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.457+0000 7ff03faff700 10 client.30774.objecter _send_command 6 2024-02-23T00:46:35.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.457+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff0380fe0c0 con 0x7ff0380feb60 2024-02-23T00:46:35.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.459+0000 7ff02effd700 1 -- 172.21.15.47:0/3832319785 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff0380fe0c0 con 0x7ff0380feb60 2024-02-23T00:46:35.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.459+0000 7ff02effd700 10 client.30774.objecter ms_dispatch 0x7ff038001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:46:35.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.459+0000 7ff02effd700 10 client.30774.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:46:35.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.459+0000 7ff02effd700 20 client.30774.objecter put_session s=0x7ff0380282e0 osd=2 4 2024-02-23T00:46:35.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.459+0000 7ff02effd700 15 client.30774.objecter _session_command_op_remove 2 6 2024-02-23T00:46:35.462 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:46:35.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.459+0000 7ff03faff700 10 client.30774.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:35.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.459+0000 7ff03faff700 15 client.30774.objecter _session_command_op_assign -1 7 2024-02-23T00:46:35.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.459+0000 7ff03faff700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7ff038059e50 0x7ff03811b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:35.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.459+0000 7ff03faff700 20 client.30774.objecter _get_session s=0x7ff0380fe4c0 osd=3 3 2024-02-23T00:46:35.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.459+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff0380fe4c0 osd=3 3 2024-02-23T00:46:35.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.459+0000 7ff03faff700 20 client.30774.objecter _get_session s=0x7ff0380fe4c0 osd=3 3 2024-02-23T00:46:35.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.459+0000 7ff03faff700 15 client.30774.objecter _session_command_op_remove -1 7 2024-02-23T00:46:35.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.459+0000 7ff03faff700 20 client.30774.objecter get_session s=0x7ff0380fe4c0 osd=3 3 2024-02-23T00:46:35.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.459+0000 7ff03faff700 15 client.30774.objecter _session_command_op_assign 3 7 2024-02-23T00:46:35.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.459+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff0380fe4c0 osd=3 4 2024-02-23T00:46:35.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.459+0000 7ff03faff700 10 client.30774.objecter _send_command 7 2024-02-23T00:46:35.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.459+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff03811bae0 con 0x7ff038059e50 2024-02-23T00:46:35.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.459+0000 7ff03e09c700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7ff038059e50 0x7ff03811b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:35.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.459+0000 7ff02effd700 10 client.30774.objecter ms_handle_connect 0x7ff038059e50 2024-02-23T00:46:35.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.459+0000 7ff03e09c700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7ff038059e50 0x7ff03811b450 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:35.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.460+0000 7ff02effd700 1 -- 172.21.15.47:0/3832319785 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff03811bae0 con 0x7ff038059e50 2024-02-23T00:46:35.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.460+0000 7ff02effd700 10 client.30774.objecter ms_dispatch 0x7ff038001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:46:35.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.460+0000 7ff02effd700 10 client.30774.objecter _finish_command 7 = system:0 2024-02-23T00:46:35.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.460+0000 7ff02effd700 20 client.30774.objecter put_session s=0x7ff0380fe4c0 osd=3 4 2024-02-23T00:46:35.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.460+0000 7ff02effd700 15 client.30774.objecter _session_command_op_remove 3 7 2024-02-23T00:46:35.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.474+0000 7ff03faff700 10 client.30774.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:46:35.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.474+0000 7ff03faff700 15 client.30774.objecter _session_command_op_assign -1 8 2024-02-23T00:46:35.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.474+0000 7ff03faff700 20 client.30774.objecter _get_session s=0x7ff0380fe4c0 osd=3 3 2024-02-23T00:46:35.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.474+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff0380fe4c0 osd=3 3 2024-02-23T00:46:35.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.474+0000 7ff03faff700 20 client.30774.objecter _get_session s=0x7ff0380fe4c0 osd=3 3 2024-02-23T00:46:35.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.474+0000 7ff03faff700 15 client.30774.objecter _session_command_op_remove -1 8 2024-02-23T00:46:35.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.474+0000 7ff03faff700 20 client.30774.objecter get_session s=0x7ff0380fe4c0 osd=3 3 2024-02-23T00:46:35.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.474+0000 7ff03faff700 15 client.30774.objecter _session_command_op_assign 3 8 2024-02-23T00:46:35.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.474+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff0380fe4c0 osd=3 4 2024-02-23T00:46:35.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.474+0000 7ff03faff700 10 client.30774.objecter _send_command 8 2024-02-23T00:46:35.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.474+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff038059360 con 0x7ff038059e50 2024-02-23T00:46:35.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.476+0000 7ff02effd700 1 -- 172.21.15.47:0/3832319785 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff038059360 con 0x7ff038059e50 2024-02-23T00:46:35.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.476+0000 7ff02effd700 10 client.30774.objecter ms_dispatch 0x7ff038001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:46:35.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.476+0000 7ff02effd700 10 client.30774.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:46:35.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.476+0000 7ff02effd700 20 client.30774.objecter put_session s=0x7ff0380fe4c0 osd=3 4 2024-02-23T00:46:35.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.476+0000 7ff02effd700 15 client.30774.objecter _session_command_op_remove 3 8 2024-02-23T00:46:35.479 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:46:35.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.476+0000 7ff03faff700 10 client.30774.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:35.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.476+0000 7ff03faff700 15 client.30774.objecter _session_command_op_assign -1 9 2024-02-23T00:46:35.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.476+0000 7ff03faff700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7ff03811ec40 0x7ff038121030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:35.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.476+0000 7ff03faff700 20 client.30774.objecter _get_session s=0x7ff038059500 osd=4 3 2024-02-23T00:46:35.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.476+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff038059500 osd=4 3 2024-02-23T00:46:35.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.476+0000 7ff03faff700 20 client.30774.objecter _get_session s=0x7ff038059500 osd=4 3 2024-02-23T00:46:35.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.476+0000 7ff03faff700 15 client.30774.objecter _session_command_op_remove -1 9 2024-02-23T00:46:35.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.476+0000 7ff03faff700 20 client.30774.objecter get_session s=0x7ff038059500 osd=4 3 2024-02-23T00:46:35.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.476+0000 7ff03faff700 15 client.30774.objecter _session_command_op_assign 4 9 2024-02-23T00:46:35.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.476+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff038059500 osd=4 4 2024-02-23T00:46:35.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.476+0000 7ff03faff700 10 client.30774.objecter _send_command 9 2024-02-23T00:46:35.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.476+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff038059700 con 0x7ff03811ec40 2024-02-23T00:46:35.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.477+0000 7ff03d89b700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7ff03811ec40 0x7ff038121030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:35.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.477+0000 7ff02effd700 10 client.30774.objecter ms_handle_connect 0x7ff03811ec40 2024-02-23T00:46:35.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.477+0000 7ff03d89b700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7ff03811ec40 0x7ff038121030 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:35.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.477+0000 7ff02effd700 1 -- 172.21.15.47:0/3832319785 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff038059700 con 0x7ff03811ec40 2024-02-23T00:46:35.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.477+0000 7ff02effd700 10 client.30774.objecter ms_dispatch 0x7ff038001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:46:35.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.477+0000 7ff02effd700 10 client.30774.objecter _finish_command 9 = system:0 2024-02-23T00:46:35.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.477+0000 7ff02effd700 20 client.30774.objecter put_session s=0x7ff038059500 osd=4 4 2024-02-23T00:46:35.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.477+0000 7ff02effd700 15 client.30774.objecter _session_command_op_remove 4 9 2024-02-23T00:46:35.494 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:46:35.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.492+0000 7ff03faff700 10 client.30774.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:46:35.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.492+0000 7ff03faff700 15 client.30774.objecter _session_command_op_assign -1 10 2024-02-23T00:46:35.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.492+0000 7ff03faff700 20 client.30774.objecter _get_session s=0x7ff038059500 osd=4 3 2024-02-23T00:46:35.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.492+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff038059500 osd=4 3 2024-02-23T00:46:35.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.492+0000 7ff03faff700 20 client.30774.objecter _get_session s=0x7ff038059500 osd=4 3 2024-02-23T00:46:35.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.492+0000 7ff03faff700 15 client.30774.objecter _session_command_op_remove -1 10 2024-02-23T00:46:35.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.492+0000 7ff03faff700 20 client.30774.objecter get_session s=0x7ff038059500 osd=4 3 2024-02-23T00:46:35.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.492+0000 7ff03faff700 15 client.30774.objecter _session_command_op_assign 4 10 2024-02-23T00:46:35.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.492+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff038059500 osd=4 4 2024-02-23T00:46:35.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.492+0000 7ff03faff700 10 client.30774.objecter _send_command 10 2024-02-23T00:46:35.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.492+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff038059700 con 0x7ff03811ec40 2024-02-23T00:46:35.497 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:46:35.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.495+0000 7ff02effd700 1 -- 172.21.15.47:0/3832319785 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff038059700 con 0x7ff03811ec40 2024-02-23T00:46:35.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.495+0000 7ff02effd700 10 client.30774.objecter ms_dispatch 0x7ff038001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:46:35.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.495+0000 7ff02effd700 10 client.30774.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:46:35.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.495+0000 7ff02effd700 20 client.30774.objecter put_session s=0x7ff038059500 osd=4 4 2024-02-23T00:46:35.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.495+0000 7ff02effd700 15 client.30774.objecter _session_command_op_remove 4 10 2024-02-23T00:46:35.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.495+0000 7ff03faff700 10 client.30774.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:35.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.495+0000 7ff03faff700 15 client.30774.objecter _session_command_op_assign -1 11 2024-02-23T00:46:35.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.495+0000 7ff03faff700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7ff038121a30 0x7ff038123ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:35.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.495+0000 7ff03faff700 20 client.30774.objecter _get_session s=0x7ff038059700 osd=5 3 2024-02-23T00:46:35.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.495+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff038059700 osd=5 3 2024-02-23T00:46:35.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.495+0000 7ff03faff700 20 client.30774.objecter _get_session s=0x7ff038059700 osd=5 3 2024-02-23T00:46:35.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.495+0000 7ff03faff700 15 client.30774.objecter _session_command_op_remove -1 11 2024-02-23T00:46:35.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.495+0000 7ff03faff700 20 client.30774.objecter get_session s=0x7ff038059700 osd=5 3 2024-02-23T00:46:35.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.495+0000 7ff03faff700 15 client.30774.objecter _session_command_op_assign 5 11 2024-02-23T00:46:35.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.495+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff038059700 osd=5 4 2024-02-23T00:46:35.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.495+0000 7ff03faff700 10 client.30774.objecter _send_command 11 2024-02-23T00:46:35.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.495+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff038126ce0 con 0x7ff038121a30 2024-02-23T00:46:35.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.495+0000 7ff03d09a700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7ff038121a30 0x7ff038123ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:35.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.495+0000 7ff02effd700 10 client.30774.objecter ms_handle_connect 0x7ff038121a30 2024-02-23T00:46:35.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.495+0000 7ff03d09a700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7ff038121a30 0x7ff038123ea0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:35.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.495+0000 7ff02effd700 1 -- 172.21.15.47:0/3832319785 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff038126ce0 con 0x7ff038121a30 2024-02-23T00:46:35.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.495+0000 7ff02effd700 10 client.30774.objecter ms_dispatch 0x7ff038001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:46:35.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.495+0000 7ff02effd700 10 client.30774.objecter _finish_command 11 = system:0 2024-02-23T00:46:35.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.496+0000 7ff02effd700 20 client.30774.objecter put_session s=0x7ff038059700 osd=5 4 2024-02-23T00:46:35.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.496+0000 7ff02effd700 15 client.30774.objecter _session_command_op_remove 5 11 2024-02-23T00:46:35.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.511+0000 7ff03faff700 10 client.30774.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:46:35.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.511+0000 7ff03faff700 15 client.30774.objecter _session_command_op_assign -1 12 2024-02-23T00:46:35.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.511+0000 7ff03faff700 20 client.30774.objecter _get_session s=0x7ff038059700 osd=5 3 2024-02-23T00:46:35.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.511+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff038059700 osd=5 3 2024-02-23T00:46:35.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.511+0000 7ff03faff700 20 client.30774.objecter _get_session s=0x7ff038059700 osd=5 3 2024-02-23T00:46:35.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.511+0000 7ff03faff700 15 client.30774.objecter _session_command_op_remove -1 12 2024-02-23T00:46:35.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.511+0000 7ff03faff700 20 client.30774.objecter get_session s=0x7ff038059700 osd=5 3 2024-02-23T00:46:35.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.511+0000 7ff03faff700 15 client.30774.objecter _session_command_op_assign 5 12 2024-02-23T00:46:35.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.511+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff038059700 osd=5 4 2024-02-23T00:46:35.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.511+0000 7ff03faff700 10 client.30774.objecter _send_command 12 2024-02-23T00:46:35.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.511+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff038125ab0 con 0x7ff038121a30 2024-02-23T00:46:35.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.513+0000 7ff02effd700 1 -- 172.21.15.47:0/3832319785 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff038125ab0 con 0x7ff038121a30 2024-02-23T00:46:35.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.513+0000 7ff02effd700 10 client.30774.objecter ms_dispatch 0x7ff038001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:46:35.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.513+0000 7ff02effd700 10 client.30774.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:46:35.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.513+0000 7ff02effd700 20 client.30774.objecter put_session s=0x7ff038059700 osd=5 4 2024-02-23T00:46:35.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.513+0000 7ff02effd700 15 client.30774.objecter _session_command_op_remove 5 12 2024-02-23T00:46:35.515 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:46:35.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.513+0000 7ff03faff700 10 client.30774.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:35.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.513+0000 7ff03faff700 15 client.30774.objecter _session_command_op_assign -1 13 2024-02-23T00:46:35.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.513+0000 7ff03faff700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7ff038126210 0x7ff038126670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:35.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.513+0000 7ff03faff700 20 client.30774.objecter _get_session s=0x7ff038125ab0 osd=6 3 2024-02-23T00:46:35.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.513+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff038125ab0 osd=6 3 2024-02-23T00:46:35.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.513+0000 7ff03faff700 20 client.30774.objecter _get_session s=0x7ff038125ab0 osd=6 3 2024-02-23T00:46:35.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.513+0000 7ff03faff700 15 client.30774.objecter _session_command_op_remove -1 13 2024-02-23T00:46:35.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.513+0000 7ff03faff700 20 client.30774.objecter get_session s=0x7ff038125ab0 osd=6 3 2024-02-23T00:46:35.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.513+0000 7ff03faff700 15 client.30774.objecter _session_command_op_assign 6 13 2024-02-23T00:46:35.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.513+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff038125ab0 osd=6 4 2024-02-23T00:46:35.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.513+0000 7ff03faff700 10 client.30774.objecter _send_command 13 2024-02-23T00:46:35.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.513+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff038125720 con 0x7ff038126210 2024-02-23T00:46:35.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.513+0000 7ff03e09c700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7ff038126210 0x7ff038126670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:35.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.513+0000 7ff02effd700 10 client.30774.objecter ms_handle_connect 0x7ff038126210 2024-02-23T00:46:35.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.513+0000 7ff03e09c700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7ff038126210 0x7ff038126670 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:35.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.513+0000 7ff02effd700 1 -- 172.21.15.47:0/3832319785 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff038125720 con 0x7ff038126210 2024-02-23T00:46:35.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.513+0000 7ff02effd700 10 client.30774.objecter ms_dispatch 0x7ff038001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:46:35.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.513+0000 7ff02effd700 10 client.30774.objecter _finish_command 13 = system:0 2024-02-23T00:46:35.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.513+0000 7ff02effd700 20 client.30774.objecter put_session s=0x7ff038125ab0 osd=6 4 2024-02-23T00:46:35.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.513+0000 7ff02effd700 15 client.30774.objecter _session_command_op_remove 6 13 2024-02-23T00:46:35.521 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:35.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.529+0000 7ff03faff700 10 client.30774.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:46:35.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.529+0000 7ff03faff700 15 client.30774.objecter _session_command_op_assign -1 14 2024-02-23T00:46:35.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.529+0000 7ff03faff700 20 client.30774.objecter _get_session s=0x7ff038125ab0 osd=6 3 2024-02-23T00:46:35.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.529+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff038125ab0 osd=6 3 2024-02-23T00:46:35.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.529+0000 7ff03faff700 20 client.30774.objecter _get_session s=0x7ff038125ab0 osd=6 3 2024-02-23T00:46:35.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.529+0000 7ff03faff700 15 client.30774.objecter _session_command_op_remove -1 14 2024-02-23T00:46:35.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.529+0000 7ff03faff700 20 client.30774.objecter get_session s=0x7ff038125ab0 osd=6 3 2024-02-23T00:46:35.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.529+0000 7ff03faff700 15 client.30774.objecter _session_command_op_assign 6 14 2024-02-23T00:46:35.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.529+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff038125ab0 osd=6 4 2024-02-23T00:46:35.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.529+0000 7ff03faff700 10 client.30774.objecter _send_command 14 2024-02-23T00:46:35.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.529+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff038125720 con 0x7ff038126210 2024-02-23T00:46:35.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.531+0000 7ff02effd700 1 -- 172.21.15.47:0/3832319785 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff038125720 con 0x7ff038126210 2024-02-23T00:46:35.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.531+0000 7ff02effd700 10 client.30774.objecter ms_dispatch 0x7ff038001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:46:35.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.531+0000 7ff02effd700 10 client.30774.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:46:35.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.531+0000 7ff02effd700 20 client.30774.objecter put_session s=0x7ff038125ab0 osd=6 4 2024-02-23T00:46:35.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.531+0000 7ff02effd700 15 client.30774.objecter _session_command_op_remove 6 14 2024-02-23T00:46:35.533 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:46:35.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.531+0000 7ff03faff700 10 client.30774.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:35.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.531+0000 7ff03faff700 15 client.30774.objecter _session_command_op_assign -1 15 2024-02-23T00:46:35.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.531+0000 7ff03faff700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7ff038124a00 0x7ff038124e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:35.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.531+0000 7ff03faff700 20 client.30774.objecter _get_session s=0x7ff038125720 osd=7 3 2024-02-23T00:46:35.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.531+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff038125720 osd=7 3 2024-02-23T00:46:35.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.531+0000 7ff03faff700 20 client.30774.objecter _get_session s=0x7ff038125720 osd=7 3 2024-02-23T00:46:35.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.531+0000 7ff03faff700 15 client.30774.objecter _session_command_op_remove -1 15 2024-02-23T00:46:35.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.531+0000 7ff03faff700 20 client.30774.objecter get_session s=0x7ff038125720 osd=7 3 2024-02-23T00:46:35.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.531+0000 7ff03faff700 15 client.30774.objecter _session_command_op_assign 7 15 2024-02-23T00:46:35.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.531+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff038125720 osd=7 4 2024-02-23T00:46:35.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.531+0000 7ff03faff700 10 client.30774.objecter _send_command 15 2024-02-23T00:46:35.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.531+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff038125470 con 0x7ff038124a00 2024-02-23T00:46:35.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.531+0000 7ff03d89b700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7ff038124a00 0x7ff038124e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:35.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.532+0000 7ff02effd700 10 client.30774.objecter ms_handle_connect 0x7ff038124a00 2024-02-23T00:46:35.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.532+0000 7ff03d89b700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7ff038124a00 0x7ff038124e60 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:35.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.532+0000 7ff02effd700 1 -- 172.21.15.47:0/3832319785 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff038125470 con 0x7ff038124a00 2024-02-23T00:46:35.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.532+0000 7ff02effd700 10 client.30774.objecter ms_dispatch 0x7ff038001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:46:35.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.532+0000 7ff02effd700 10 client.30774.objecter _finish_command 15 = system:0 2024-02-23T00:46:35.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.532+0000 7ff02effd700 20 client.30774.objecter put_session s=0x7ff038125720 osd=7 4 2024-02-23T00:46:35.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.532+0000 7ff02effd700 15 client.30774.objecter _session_command_op_remove 7 15 2024-02-23T00:46:35.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.547+0000 7ff03faff700 10 client.30774.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:46:35.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.547+0000 7ff03faff700 15 client.30774.objecter _session_command_op_assign -1 16 2024-02-23T00:46:35.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.547+0000 7ff03faff700 20 client.30774.objecter _get_session s=0x7ff038125720 osd=7 3 2024-02-23T00:46:35.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.547+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff038125720 osd=7 3 2024-02-23T00:46:35.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.547+0000 7ff03faff700 20 client.30774.objecter _get_session s=0x7ff038125720 osd=7 3 2024-02-23T00:46:35.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.547+0000 7ff03faff700 15 client.30774.objecter _session_command_op_remove -1 16 2024-02-23T00:46:35.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.547+0000 7ff03faff700 20 client.30774.objecter get_session s=0x7ff038125720 osd=7 3 2024-02-23T00:46:35.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.547+0000 7ff03faff700 15 client.30774.objecter _session_command_op_assign 7 16 2024-02-23T00:46:35.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.547+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff038125720 osd=7 4 2024-02-23T00:46:35.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.547+0000 7ff03faff700 10 client.30774.objecter _send_command 16 2024-02-23T00:46:35.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.547+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff038125430 con 0x7ff038124a00 2024-02-23T00:46:35.552 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:46:35.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.549+0000 7ff02effd700 1 -- 172.21.15.47:0/3832319785 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff038125430 con 0x7ff038124a00 2024-02-23T00:46:35.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.549+0000 7ff02effd700 10 client.30774.objecter ms_dispatch 0x7ff038001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:46:35.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.549+0000 7ff02effd700 10 client.30774.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:46:35.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.549+0000 7ff02effd700 20 client.30774.objecter put_session s=0x7ff038125720 osd=7 4 2024-02-23T00:46:35.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.549+0000 7ff02effd700 15 client.30774.objecter _session_command_op_remove 7 16 2024-02-23T00:46:35.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.549+0000 7ff03faff700 10 librados: watch_flush enter 2024-02-23T00:46:35.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.549+0000 7ff03faff700 10 librados: watch_flush exit 2024-02-23T00:46:35.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.549+0000 7ff03faff700 10 client.30774.objecter close_session for osd.0 2024-02-23T00:46:35.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.549+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7ff0380273c0 msgr2=0x7ff0380277a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:35.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.549+0000 7ff03faff700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7ff0380273c0 0x7ff0380277a0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:35.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.549+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff038105ee0 osd=0 1 2024-02-23T00:46:35.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.549+0000 7ff03faff700 10 client.30774.objecter close_session for osd.1 2024-02-23T00:46:35.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.549+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7ff038061050 msgr2=0x7ff0380fd990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:35.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.549+0000 7ff03faff700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7ff038061050 0x7ff0380fd990 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:35.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.549+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff0380ff0e0 osd=1 1 2024-02-23T00:46:35.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.549+0000 7ff03faff700 10 client.30774.objecter close_session for osd.2 2024-02-23T00:46:35.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.549+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7ff0380feb60 msgr2=0x7ff038119de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:35.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.549+0000 7ff03faff700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7ff0380feb60 0x7ff038119de0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:35.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.549+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff0380282e0 osd=2 1 2024-02-23T00:46:35.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.549+0000 7ff03faff700 10 client.30774.objecter close_session for osd.3 2024-02-23T00:46:35.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.549+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7ff038059e50 msgr2=0x7ff03811b450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:35.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.549+0000 7ff03faff700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7ff038059e50 0x7ff03811b450 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:35.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.549+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff0380fe4c0 osd=3 1 2024-02-23T00:46:35.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.549+0000 7ff03faff700 10 client.30774.objecter close_session for osd.4 2024-02-23T00:46:35.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.549+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7ff03811ec40 msgr2=0x7ff038121030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:35.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.550+0000 7ff03faff700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7ff03811ec40 0x7ff038121030 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:35.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.550+0000 7ff03e09c700 1 -- 172.21.15.47:0/3832319785 reap_dead start 2024-02-23T00:46:35.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.550+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff038059500 osd=4 1 2024-02-23T00:46:35.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.550+0000 7ff03faff700 10 client.30774.objecter close_session for osd.5 2024-02-23T00:46:35.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.550+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7ff038121a30 msgr2=0x7ff038123ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:35.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.550+0000 7ff03faff700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7ff038121a30 0x7ff038123ea0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:35.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.550+0000 7ff03e09c700 1 -- 172.21.15.47:0/3832319785 reap_dead start 2024-02-23T00:46:35.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.550+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff038059700 osd=5 1 2024-02-23T00:46:35.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.550+0000 7ff03faff700 10 client.30774.objecter close_session for osd.6 2024-02-23T00:46:35.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.550+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7ff038126210 msgr2=0x7ff038126670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:35.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.550+0000 7ff03faff700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7ff038126210 0x7ff038126670 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:35.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.550+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff038125ab0 osd=6 1 2024-02-23T00:46:35.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.550+0000 7ff03faff700 10 client.30774.objecter close_session for osd.7 2024-02-23T00:46:35.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.550+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7ff038124a00 msgr2=0x7ff038124e60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:35.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.550+0000 7ff03faff700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7ff038124a00 0x7ff038124e60 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:35.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.550+0000 7ff03e09c700 1 -- 172.21.15.47:0/3832319785 reap_dead start 2024-02-23T00:46:35.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.550+0000 7ff03faff700 20 client.30774.objecter put_session s=0x7ff038125720 osd=7 1 2024-02-23T00:46:35.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.550+0000 7ff03faff700 20 client.30774.objecter shutdown clearing up homeless session... 2024-02-23T00:46:35.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.550+0000 7ff03faff700 10 client.30774.objecter successfully canceled tick 2024-02-23T00:46:35.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.550+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff024062cc0 msgr2=0x7ff024065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:35.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.550+0000 7ff03faff700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff024062cc0 0x7ff024065180 secure :-1 s=READY pgs=3024 cs=0 l=1 rev1=1 rx=0x7ff034009da0 tx=0x7ff03400b040).stop 2024-02-23T00:46:35.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.550+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0380fcbb0 msgr2=0x7ff03810d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:35.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.550+0000 7ff03faff700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0380fcbb0 0x7ff03810d470 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 rx=0x7ff028010ba0 tx=0x7ff02800b040).stop 2024-02-23T00:46:35.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.550+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 shutdown_connections 2024-02-23T00:46:35.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.550+0000 7ff03faff700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0380fcbb0 0x7ff03810d470 unknown :-1 s=CLOSED pgs=2504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:35.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.550+0000 7ff03faff700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff024062cc0 0x7ff024065180 unknown :-1 s=CLOSED pgs=3024 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:35.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.550+0000 7ff03faff700 1 --2- 172.21.15.47:0/3832319785 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7ff038124a00 0x7ff038124e60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:35.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.550+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 >> 172.21.15.47:0/3832319785 conn(0x7ff0380f2820 msgr2=0x7ff0380fb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:46:35.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.550+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 shutdown_connections 2024-02-23T00:46:35.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.550+0000 7ff03faff700 1 -- 172.21.15.47:0/3832319785 wait complete. 2024-02-23T00:46:35.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:35.550+0000 7ff03faff700 1 librados: shutdown 2024-02-23T00:46:35.558 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:46:35.558 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:46:35.558 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:46:35.558 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:46:35.558 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:46:35.558 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:46:35.558 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:46:35.558 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:46:35.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:34.170712+0000 mon.a (mon.0) 2132 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-41","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-23T00:46:35.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:34.170789+0000 mon.a (mon.0) 2133 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi047-82386-54"}]': finished 2024-02-23T00:46:35.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:34.170873+0000 mon.a (mon.0) 2134 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:46:35.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:34.170926+0000 mon.a (mon.0) 2135 : cluster [DBG] osdmap e322: 8 total, 8 up, 8 in 2024-02-23T00:46:35.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:34.175866+0000 mon.b (mon.1) 356 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:35.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:34.207128+0000 mon.a (mon.0) 2136 : audit [INF] from='client.? 172.21.15.47:0/427115143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi047-82386-55"}]: dispatch 2024-02-23T00:46:35.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:34.207921+0000 mon.a (mon.0) 2137 : audit [INF] from='client.? 172.21.15.47:0/427115143' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi047-82386-55"}]: dispatch 2024-02-23T00:46:35.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:34.208504+0000 mon.a (mon.0) 2138 : audit [INF] from='client.? 172.21.15.47:0/427115143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi047-82386-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:35.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:34.424224+0000 mon.a (mon.0) 2139 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi047-82465-41","var": "pg_num","format": "json"}]: dispatch 2024-02-23T00:46:35.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:34.538421+0000 mon.a (mon.0) 2140 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:46:35.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:34.538761+0000 mon.a (mon.0) 2141 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-41"}]: dispatch 2024-02-23T00:46:35.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:34.718666+0000 osd.7 (osd.7) 163 : cluster [DBG] 6.15 scrub starts 2024-02-23T00:46:35.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:34.721560+0000 osd.7 (osd.7) 164 : cluster [DBG] 6.15 scrub ok 2024-02-23T00:46:35.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:35.151924+0000 mon.a (mon.0) 2142 : audit [INF] from='client.? 172.21.15.47:0/427115143' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi047-82386-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:35.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:35.151980+0000 mon.a (mon.0) 2143 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-41"}]': finished 2024-02-23T00:46:35.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:35.152023+0000 mon.a (mon.0) 2144 : cluster [DBG] osdmap e323: 8 total, 8 up, 8 in 2024-02-23T00:46:35.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:35.158334+0000 mon.a (mon.0) 2145 : audit [INF] from='client.? 172.21.15.47:0/427115143' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi047-82386-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi047-82386-55"}]: dispatch 2024-02-23T00:46:35.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:35.177266+0000 mon.b (mon.1) 357 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:36.571 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:46:36.575 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:36.576 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:46:36.676 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:46:36.708 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:36.730 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:46:36.733 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:36.733 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:46:36.834 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:46:36.867 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:37.094 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-23T00:46:37.122 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:34.725238+0000 osd.2 (osd.2) 143 : cluster [DBG] 5.b scrub starts 2024-02-23T00:46:37.122 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:34.726023+0000 osd.2 (osd.2) 144 : cluster [DBG] 5.b scrub ok 2024-02-23T00:46:37.122 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:35.219068+0000 osd.6 (osd.6) 125 : cluster [DBG] 2.7 scrub starts 2024-02-23T00:46:37.122 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:35.220208+0000 osd.6 (osd.6) 126 : cluster [DBG] 2.7 scrub ok 2024-02-23T00:46:37.122 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:35.259784+0000 mgr.y (mgr.14517) 317 : cluster [DBG] pgmap v460: 238 pgs: 1 peering, 237 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:37.123 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:35.260792+0000 mon.a (mon.0) 2146 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "19"}]: dispatch 2024-02-23T00:46:37.123 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:35.316443+0000 osd.4 (osd.4) 131 : cluster [DBG] 62.f scrub starts 2024-02-23T00:46:37.123 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:35.318566+0000 osd.4 (osd.4) 132 : cluster [DBG] 62.f scrub ok 2024-02-23T00:46:37.123 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:35.406872+0000 mon.a (mon.0) 2147 : audit [DBG] from='client.? 172.21.15.47:0/3832319785' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:46:37.123 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:35.625980+0000 osd.0 (osd.0) 133 : cluster [DBG] 62.4 scrub starts 2024-02-23T00:46:37.123 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:35.627475+0000 osd.0 (osd.0) 134 : cluster [DBG] 62.4 scrub ok 2024-02-23T00:46:37.123 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:34.725238+0000 osd.2 (osd.2) 143 : cluster [DBG] 5.b scrub starts 2024-02-23T00:46:37.123 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:34.726023+0000 osd.2 (osd.2) 144 : cluster [DBG] 5.b scrub ok 2024-02-23T00:46:37.123 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:35.219068+0000 osd.6 (osd.6) 125 : cluster [DBG] 2.7 scrub starts 2024-02-23T00:46:37.124 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:35.220208+0000 osd.6 (osd.6) 126 : cluster [DBG] 2.7 scrub ok 2024-02-23T00:46:37.124 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:35.259784+0000 mgr.y (mgr.14517) 317 : cluster [DBG] pgmap v460: 238 pgs: 1 peering, 237 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:37.124 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:35.260792+0000 mon.a (mon.0) 2146 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "19"}]: dispatch 2024-02-23T00:46:37.124 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:35.316443+0000 osd.4 (osd.4) 131 : cluster [DBG] 62.f scrub starts 2024-02-23T00:46:37.124 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:35.318566+0000 osd.4 (osd.4) 132 : cluster [DBG] 62.f scrub ok 2024-02-23T00:46:37.124 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:35.406872+0000 mon.a (mon.0) 2147 : audit [DBG] from='client.? 172.21.15.47:0/3832319785' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:46:37.124 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:35.625980+0000 osd.0 (osd.0) 133 : cluster [DBG] 62.4 scrub starts 2024-02-23T00:46:37.124 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:35.627475+0000 osd.0 (osd.0) 134 : cluster [DBG] 62.4 scrub ok 2024-02-23T00:46:37.420 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:46:37.423 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:37.423 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:46:37.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:34.725238+0000 osd.2 (osd.2) 143 : cluster [DBG] 5.b scrub starts 2024-02-23T00:46:37.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:34.726023+0000 osd.2 (osd.2) 144 : cluster [DBG] 5.b scrub ok 2024-02-23T00:46:37.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:35.219068+0000 osd.6 (osd.6) 125 : cluster [DBG] 2.7 scrub starts 2024-02-23T00:46:37.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:35.220208+0000 osd.6 (osd.6) 126 : cluster [DBG] 2.7 scrub ok 2024-02-23T00:46:37.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:35.259784+0000 mgr.y (mgr.14517) 317 : cluster [DBG] pgmap v460: 238 pgs: 1 peering, 237 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:37.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:35.260792+0000 mon.a (mon.0) 2146 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "19"}]: dispatch 2024-02-23T00:46:37.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:35.316443+0000 osd.4 (osd.4) 131 : cluster [DBG] 62.f scrub starts 2024-02-23T00:46:37.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:35.318566+0000 osd.4 (osd.4) 132 : cluster [DBG] 62.f scrub ok 2024-02-23T00:46:37.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:35.406872+0000 mon.a (mon.0) 2147 : audit [DBG] from='client.? 172.21.15.47:0/3832319785' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:46:37.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:35.625980+0000 osd.0 (osd.0) 133 : cluster [DBG] 62.4 scrub starts 2024-02-23T00:46:37.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:35.627475+0000 osd.0 (osd.0) 134 : cluster [DBG] 62.4 scrub ok 2024-02-23T00:46:37.524 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:46:37.552 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:37.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.619+0000 7fb3cff30700 1 -- 172.21.15.47:0/2697252193 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3c80fdbb0 msgr2=0x7fb3c80fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:37.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.619+0000 7fb3cff30700 1 --2- 172.21.15.47:0/2697252193 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3c80fdbb0 0x7fb3c80fdf90 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 rx=0x7fb3b8009be0 tx=0x7fb3b8004d10).stop 2024-02-23T00:46:37.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.620+0000 7fb3cff30700 1 -- 172.21.15.47:0/2697252193 shutdown_connections 2024-02-23T00:46:37.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.620+0000 7fb3cff30700 1 --2- 172.21.15.47:0/2697252193 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb3c80f8420 0x7fb3c8104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.620+0000 7fb3cff30700 1 --2- 172.21.15.47:0/2697252193 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3c80fdbb0 0x7fb3c80fdf90 unknown :-1 s=CLOSED pgs=2505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.620+0000 7fb3cff30700 1 --2- 172.21.15.47:0/2697252193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3c80fe510 0x7fb3c80f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.620+0000 7fb3cff30700 1 -- 172.21.15.47:0/2697252193 >> 172.21.15.47:0/2697252193 conn(0x7fb3c80f3800 msgr2=0x7fb3c80f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:46:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.620+0000 7fb3cff30700 1 -- 172.21.15.47:0/2697252193 shutdown_connections 2024-02-23T00:46:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.620+0000 7fb3cff30700 1 -- 172.21.15.47:0/2697252193 wait complete. 2024-02-23T00:46:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.620+0000 7fb3cff30700 1 librados: starting msgr at 2024-02-23T00:46:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.620+0000 7fb3cff30700 1 librados: starting objecter 2024-02-23T00:46:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.620+0000 7fb3cff30700 1 Processor -- start 2024-02-23T00:46:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.620+0000 7fb3cff30700 1 -- start start 2024-02-23T00:46:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.620+0000 7fb3cff30700 1 librados: setting wanted keys 2024-02-23T00:46:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.620+0000 7fb3cff30700 1 librados: calling monclient init 2024-02-23T00:46:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.620+0000 7fb3cff30700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb3c80f8420 0x7fb3c806b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.620+0000 7fb3cff30700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3c80fdbb0 0x7fb3c806b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.620+0000 7fb3cff30700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3c80fe510 0x7fb3c806bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.620+0000 7fb3cff30700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c81163e0 con 0x7fb3c80fdbb0 2024-02-23T00:46:37.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.620+0000 7fb3cff30700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c8116550 con 0x7fb3c80fe510 2024-02-23T00:46:37.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.620+0000 7fb3cff30700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c8116710 con 0x7fb3c80f8420 2024-02-23T00:46:37.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.621+0000 7fb3cdccc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb3c80f8420 0x7fb3c806b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:37.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.621+0000 7fb3cd4cb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3c80fdbb0 0x7fb3c806b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:37.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.621+0000 7fb3cdccc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb3c80f8420 0x7fb3c806b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:40422/0 (socket says 172.21.15.47:40422) 2024-02-23T00:46:37.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.621+0000 7fb3cdccc700 1 -- 172.21.15.47:0/2193367657 learned_addr learned my addr 172.21.15.47:0/2193367657 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:46:37.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.621+0000 7fb3ce4cd700 1 --2- 172.21.15.47:0/2193367657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3c80fe510 0x7fb3c806bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:37.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.621+0000 7fb3cdccc700 1 -- 172.21.15.47:0/2193367657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3c80fe510 msgr2=0x7fb3c806bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:37.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.621+0000 7fb3cdccc700 1 --2- 172.21.15.47:0/2193367657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3c80fe510 0x7fb3c806bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:37.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.621+0000 7fb3cdccc700 1 -- 172.21.15.47:0/2193367657 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3c80fdbb0 msgr2=0x7fb3c806b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:37.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.621+0000 7fb3cdccc700 1 --2- 172.21.15.47:0/2193367657 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3c80fdbb0 0x7fb3c806b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:37.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.621+0000 7fb3cdccc700 1 -- 172.21.15.47:0/2193367657 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c0005040 con 0x7fb3c80f8420 2024-02-23T00:46:37.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.621+0000 7fb3ce4cd700 1 --2- 172.21.15.47:0/2193367657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3c80fe510 0x7fb3c806bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:46:37.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.621+0000 7fb3cdccc700 1 --2- 172.21.15.47:0/2193367657 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb3c80f8420 0x7fb3c806b0d0 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 rx=0x7fb3b8004750 tx=0x7fb3b8004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:37.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.621+0000 7fb3c6ffd700 10 client.?.objecter ms_handle_connect 0x7fb3c80f8420 2024-02-23T00:46:37.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.621+0000 7fb3c6ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:46:37.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.621+0000 7fb3c6ffd700 1 -- 172.21.15.47:0/2193367657 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b800e040 con 0x7fb3c80f8420 2024-02-23T00:46:37.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.621+0000 7fb3cff30700 1 -- 172.21.15.47:0/2193367657 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b80097c0 con 0x7fb3c80f8420 2024-02-23T00:46:37.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.621+0000 7fb3c6ffd700 1 -- 172.21.15.47:0/2193367657 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fb3b8018040 con 0x7fb3c80f8420 2024-02-23T00:46:37.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.621+0000 7fb3c6ffd700 1 -- 172.21.15.47:0/2193367657 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8013d60 con 0x7fb3c80f8420 2024-02-23T00:46:37.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.621+0000 7fb3cff30700 10 client.41117.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:46:37.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.621+0000 7fb3cff30700 1 -- 172.21.15.47:0/2193367657 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c8116c00 con 0x7fb3c80f8420 2024-02-23T00:46:37.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.621+0000 7fb3cff30700 1 librados: init done 2024-02-23T00:46:37.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.622+0000 7fb3cff30700 1 -- 172.21.15.47:0/2193367657 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3c805ee80 con 0x7fb3c80f8420 2024-02-23T00:46:37.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.623+0000 7fb3c6ffd700 1 -- 172.21.15.47:0/2193367657 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fb3b8007b20 con 0x7fb3c80f8420 2024-02-23T00:46:37.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.623+0000 7fb3c6ffd700 1 --2- 172.21.15.47:0/2193367657 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb3b0062cc0 0x7fb3b0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:37.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.623+0000 7fb3c6ffd700 1 -- 172.21.15.47:0/2193367657 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(325..325 src has 48..325) v4 ==== 10356+0+0 (secure 0 0 0) 0x7fb3b807e790 con 0x7fb3c80f8420 2024-02-23T00:46:37.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.623+0000 7fb3c6ffd700 10 client.41117.objecter ms_dispatch 0x7fb3c8001550 osd_map(325..325 src has 48..325) v4 2024-02-23T00:46:37.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.623+0000 7fb3c6ffd700 3 client.41117.objecter handle_osd_map got epochs [325,325] > 0 2024-02-23T00:46:37.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.623+0000 7fb3c6ffd700 3 client.41117.objecter handle_osd_map decoding full epoch 325 2024-02-23T00:46:37.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.623+0000 7fb3cd4cb700 1 --2- 172.21.15.47:0/2193367657 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb3b0062cc0 0x7fb3b0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:37.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.623+0000 7fb3c6ffd700 20 client.41117.objecter dump_active .. 0 homeless 2024-02-23T00:46:37.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.623+0000 7fb3c6ffd700 10 client.41117.objecter ms_handle_connect 0x7fb3b0062cc0 2024-02-23T00:46:37.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.623+0000 7fb3cd4cb700 1 --2- 172.21.15.47:0/2193367657 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb3b0062cc0 0x7fb3b0065180 secure :-1 s=READY pgs=3025 cs=0 l=1 rev1=1 rx=0x7fb3c806c870 tx=0x7fb3bc00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:37.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.627+0000 7fb3c6ffd700 1 -- 172.21.15.47:0/2193367657 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb3b804d9e0 con 0x7fb3c80f8420 2024-02-23T00:46:37.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.799+0000 7fb3cff30700 1 -- 172.21.15.47:0/2193367657 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb3c805fa60 con 0x7fb3b0062cc0 2024-02-23T00:46:37.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.816+0000 7fb3c6ffd700 1 -- 172.21.15.47:0/2193367657 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+623663 (secure 0 0 0) 0x7fb3c805fa60 con 0x7fb3b0062cc0 2024-02-23T00:46:37.826 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:46:37.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.820+0000 7fb3cff30700 10 librados: watch_flush enter 2024-02-23T00:46:37.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.820+0000 7fb3cff30700 10 librados: watch_flush exit 2024-02-23T00:46:37.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.820+0000 7fb3cff30700 20 client.41117.objecter shutdown clearing up homeless session... 2024-02-23T00:46:37.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.820+0000 7fb3cff30700 10 client.41117.objecter successfully canceled tick 2024-02-23T00:46:37.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.820+0000 7fb3cff30700 1 -- 172.21.15.47:0/2193367657 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb3b0062cc0 msgr2=0x7fb3b0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:37.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.820+0000 7fb3cff30700 1 --2- 172.21.15.47:0/2193367657 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb3b0062cc0 0x7fb3b0065180 secure :-1 s=READY pgs=3025 cs=0 l=1 rev1=1 rx=0x7fb3c806c870 tx=0x7fb3bc00b040).stop 2024-02-23T00:46:37.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.820+0000 7fb3cff30700 1 -- 172.21.15.47:0/2193367657 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb3c80f8420 msgr2=0x7fb3c806b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:37.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.820+0000 7fb3cff30700 1 --2- 172.21.15.47:0/2193367657 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb3c80f8420 0x7fb3c806b0d0 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 rx=0x7fb3b8004750 tx=0x7fb3b8004d10).stop 2024-02-23T00:46:37.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.821+0000 7fb3cff30700 1 -- 172.21.15.47:0/2193367657 shutdown_connections 2024-02-23T00:46:37.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.821+0000 7fb3cff30700 1 --2- 172.21.15.47:0/2193367657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3c80fe510 0x7fb3c806bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:37.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.821+0000 7fb3cff30700 1 --2- 172.21.15.47:0/2193367657 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb3b0062cc0 0x7fb3b0065180 unknown :-1 s=CLOSED pgs=3025 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:37.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.821+0000 7fb3cff30700 1 --2- 172.21.15.47:0/2193367657 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb3c80f8420 0x7fb3c806b0d0 unknown :-1 s=CLOSED pgs=2618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:37.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.821+0000 7fb3cff30700 1 --2- 172.21.15.47:0/2193367657 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3c80fdbb0 0x7fb3c806b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:37.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.821+0000 7fb3cff30700 1 -- 172.21.15.47:0/2193367657 >> 172.21.15.47:0/2193367657 conn(0x7fb3c80f3800 msgr2=0x7fb3c80f5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:46:37.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.821+0000 7fb3cff30700 1 -- 172.21.15.47:0/2193367657 shutdown_connections 2024-02-23T00:46:37.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.821+0000 7fb3cff30700 1 -- 172.21.15.47:0/2193367657 wait complete. 2024-02-23T00:46:37.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:37.821+0000 7fb3cff30700 1 librados: shutdown 2024-02-23T00:46:37.832 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-02-23T00:46:38.201 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:46:38.203 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:38.204 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:46:38.305 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:46:38.331 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:35.265587+0000 osd.5 (osd.5) 136 : cluster [DBG] 2.16 scrub starts 2024-02-23T00:46:38.331 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:35.268488+0000 osd.5 (osd.5) 137 : cluster [DBG] 2.16 scrub ok 2024-02-23T00:46:38.331 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:35.406317+0000 osd.1 (osd.1) 143 : cluster [DBG] 18.18 scrub starts 2024-02-23T00:46:38.331 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:35.407460+0000 osd.1 (osd.1) 144 : cluster [DBG] 18.18 scrub ok 2024-02-23T00:46:38.331 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:36.178309+0000 mon.b (mon.1) 358 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:38.331 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:36.398675+0000 osd.1 (osd.1) 145 : cluster [DBG] 4.12 scrub starts 2024-02-23T00:46:38.331 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:36.406763+0000 osd.1 (osd.1) 146 : cluster [DBG] 4.12 scrub ok 2024-02-23T00:46:38.331 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:36.589770+0000 osd.0 (osd.0) 135 : cluster [DBG] 5.1e scrub starts 2024-02-23T00:46:38.331 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:36.592880+0000 osd.0 (osd.0) 136 : cluster [DBG] 5.1e scrub ok 2024-02-23T00:46:38.331 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:36.608424+0000 mon.a (mon.0) 2148 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "19"}]': finished 2024-02-23T00:46:38.332 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:36.608555+0000 mon.a (mon.0) 2149 : cluster [DBG] osdmap e324: 8 total, 8 up, 8 in 2024-02-23T00:46:38.332 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:36.722747+0000 osd.7 (osd.7) 165 : cluster [DBG] 6.5 scrub starts 2024-02-23T00:46:38.332 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:36.725675+0000 osd.7 (osd.7) 166 : cluster [DBG] 6.5 scrub ok 2024-02-23T00:46:38.332 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:36.782961+0000 osd.2 (osd.2) 145 : cluster [DBG] 5.8 scrub starts 2024-02-23T00:46:38.332 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:36.783818+0000 osd.2 (osd.2) 146 : cluster [DBG] 5.8 scrub ok 2024-02-23T00:46:38.332 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:36.898559+0000 osd.3 (osd.3) 157 : cluster [DBG] 5.14 scrub starts 2024-02-23T00:46:38.332 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:36.900978+0000 osd.3 (osd.3) 158 : cluster [DBG] 5.14 scrub ok 2024-02-23T00:46:38.332 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:37.161846+0000 mon.a (mon.0) 2150 : audit [INF] from='client.? 172.21.15.47:0/427115143' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi047-82386-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi047-82386-55"}]': finished 2024-02-23T00:46:38.332 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:37.161972+0000 mon.a (mon.0) 2151 : cluster [DBG] osdmap e325: 8 total, 8 up, 8 in 2024-02-23T00:46:38.332 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:35.265587+0000 osd.5 (osd.5) 136 : cluster [DBG] 2.16 scrub starts 2024-02-23T00:46:38.333 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:35.268488+0000 osd.5 (osd.5) 137 : cluster [DBG] 2.16 scrub ok 2024-02-23T00:46:38.333 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:35.406317+0000 osd.1 (osd.1) 143 : cluster [DBG] 18.18 scrub starts 2024-02-23T00:46:38.333 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:35.407460+0000 osd.1 (osd.1) 144 : cluster [DBG] 18.18 scrub ok 2024-02-23T00:46:38.333 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:36.178309+0000 mon.b (mon.1) 358 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:38.333 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:36.398675+0000 osd.1 (osd.1) 145 : cluster [DBG] 4.12 scrub starts 2024-02-23T00:46:38.333 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:36.406763+0000 osd.1 (osd.1) 146 : cluster [DBG] 4.12 scrub ok 2024-02-23T00:46:38.333 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:36.589770+0000 osd.0 (osd.0) 135 : cluster [DBG] 5.1e scrub starts 2024-02-23T00:46:38.333 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:36.592880+0000 osd.0 (osd.0) 136 : cluster [DBG] 5.1e scrub ok 2024-02-23T00:46:38.333 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:36.608424+0000 mon.a (mon.0) 2148 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "19"}]': finished 2024-02-23T00:46:38.333 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:36.608555+0000 mon.a (mon.0) 2149 : cluster [DBG] osdmap e324: 8 total, 8 up, 8 in 2024-02-23T00:46:38.333 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:36.722747+0000 osd.7 (osd.7) 165 : cluster [DBG] 6.5 scrub starts 2024-02-23T00:46:38.333 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:36.725675+0000 osd.7 (osd.7) 166 : cluster [DBG] 6.5 scrub ok 2024-02-23T00:46:38.333 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:36.782961+0000 osd.2 (osd.2) 145 : cluster [DBG] 5.8 scrub starts 2024-02-23T00:46:38.333 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:36.783818+0000 osd.2 (osd.2) 146 : cluster [DBG] 5.8 scrub ok 2024-02-23T00:46:38.334 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:36.898559+0000 osd.3 (osd.3) 157 : cluster [DBG] 5.14 scrub starts 2024-02-23T00:46:38.334 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:36.900978+0000 osd.3 (osd.3) 158 : cluster [DBG] 5.14 scrub ok 2024-02-23T00:46:38.334 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:37.161846+0000 mon.a (mon.0) 2150 : audit [INF] from='client.? 172.21.15.47:0/427115143' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi047-82386-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi047-82386-55"}]': finished 2024-02-23T00:46:38.334 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:37.161972+0000 mon.a (mon.0) 2151 : cluster [DBG] osdmap e325: 8 total, 8 up, 8 in 2024-02-23T00:46:38.334 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:37.167818+0000 mon.a (mon.0) 2152 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:46:38.334 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:37.179160+0000 mon.b (mon.1) 359 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:38.334 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:37.228830+0000 osd.6 (osd.6) 127 : cluster [DBG] 5.1f scrub starts 2024-02-23T00:46:38.334 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:37.230024+0000 osd.6 (osd.6) 128 : cluster [DBG] 5.1f scrub ok 2024-02-23T00:46:38.334 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:37.260725+0000 mgr.y (mgr.14517) 318 : cluster [DBG] pgmap v463: 246 pgs: 2 activating, 40 creating+peering, 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:38.334 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:37.395553+0000 osd.4 (osd.4) 133 : cluster [DBG] 62.1b scrub starts 2024-02-23T00:46:38.334 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:37.397461+0000 osd.4 (osd.4) 134 : cluster [DBG] 62.1b scrub ok 2024-02-23T00:46:38.335 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:37.167818+0000 mon.a (mon.0) 2152 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:46:38.335 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:37.179160+0000 mon.b (mon.1) 359 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:38.335 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:37.228830+0000 osd.6 (osd.6) 127 : cluster [DBG] 5.1f scrub starts 2024-02-23T00:46:38.335 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:37.230024+0000 osd.6 (osd.6) 128 : cluster [DBG] 5.1f scrub ok 2024-02-23T00:46:38.335 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:37.260725+0000 mgr.y (mgr.14517) 318 : cluster [DBG] pgmap v463: 246 pgs: 2 activating, 40 creating+peering, 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:38.335 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:37.395553+0000 osd.4 (osd.4) 133 : cluster [DBG] 62.1b scrub starts 2024-02-23T00:46:38.335 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:37.397461+0000 osd.4 (osd.4) 134 : cluster [DBG] 62.1b scrub ok 2024-02-23T00:46:38.335 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:38.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:35.265587+0000 osd.5 (osd.5) 136 : cluster [DBG] 2.16 scrub starts 2024-02-23T00:46:38.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:35.268488+0000 osd.5 (osd.5) 137 : cluster [DBG] 2.16 scrub ok 2024-02-23T00:46:38.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:35.406317+0000 osd.1 (osd.1) 143 : cluster [DBG] 18.18 scrub starts 2024-02-23T00:46:38.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:35.407460+0000 osd.1 (osd.1) 144 : cluster [DBG] 18.18 scrub ok 2024-02-23T00:46:38.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:36.178309+0000 mon.b (mon.1) 358 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:38.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:36.398675+0000 osd.1 (osd.1) 145 : cluster [DBG] 4.12 scrub starts 2024-02-23T00:46:38.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:36.406763+0000 osd.1 (osd.1) 146 : cluster [DBG] 4.12 scrub ok 2024-02-23T00:46:38.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:36.589770+0000 osd.0 (osd.0) 135 : cluster [DBG] 5.1e scrub starts 2024-02-23T00:46:38.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:36.592880+0000 osd.0 (osd.0) 136 : cluster [DBG] 5.1e scrub ok 2024-02-23T00:46:38.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:36.608424+0000 mon.a (mon.0) 2148 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "19"}]': finished 2024-02-23T00:46:38.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:36.608555+0000 mon.a (mon.0) 2149 : cluster [DBG] osdmap e324: 8 total, 8 up, 8 in 2024-02-23T00:46:38.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:36.722747+0000 osd.7 (osd.7) 165 : cluster [DBG] 6.5 scrub starts 2024-02-23T00:46:38.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:36.725675+0000 osd.7 (osd.7) 166 : cluster [DBG] 6.5 scrub ok 2024-02-23T00:46:38.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:36.782961+0000 osd.2 (osd.2) 145 : cluster [DBG] 5.8 scrub starts 2024-02-23T00:46:38.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:36.783818+0000 osd.2 (osd.2) 146 : cluster [DBG] 5.8 scrub ok 2024-02-23T00:46:38.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:36.898559+0000 osd.3 (osd.3) 157 : cluster [DBG] 5.14 scrub starts 2024-02-23T00:46:38.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:36.900978+0000 osd.3 (osd.3) 158 : cluster [DBG] 5.14 scrub ok 2024-02-23T00:46:38.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:37.161846+0000 mon.a (mon.0) 2150 : audit [INF] from='client.? 172.21.15.47:0/427115143' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi047-82386-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi047-82386-55"}]': finished 2024-02-23T00:46:38.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:37.161972+0000 mon.a (mon.0) 2151 : cluster [DBG] osdmap e325: 8 total, 8 up, 8 in 2024-02-23T00:46:38.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:37.167818+0000 mon.a (mon.0) 2152 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:46:38.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:37.179160+0000 mon.b (mon.1) 359 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:38.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:37.228830+0000 osd.6 (osd.6) 127 : cluster [DBG] 5.1f scrub starts 2024-02-23T00:46:38.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:37.230024+0000 osd.6 (osd.6) 128 : cluster [DBG] 5.1f scrub ok 2024-02-23T00:46:38.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:37.260725+0000 mgr.y (mgr.14517) 318 : cluster [DBG] pgmap v463: 246 pgs: 2 activating, 40 creating+peering, 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:38.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:37.395553+0000 osd.4 (osd.4) 133 : cluster [DBG] 62.1b scrub starts 2024-02-23T00:46:38.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:37.397461+0000 osd.4 (osd.4) 134 : cluster [DBG] 62.1b scrub ok 2024-02-23T00:46:38.480 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:46:38.483 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:38.483 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:46:38.584 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:46:38.612 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:46:38.865 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:38.865 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:46:38.966 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:46:38.998 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:39.020 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:46:39.023 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:39.023 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:46:39.124 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:46:39.153 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:39.360 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:46:39.363 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:39.363 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:46:39.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:37.329928+0000 osd.5 (osd.5) 138 : cluster [DBG] 5.7 scrub starts 2024-02-23T00:46:39.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:37.332969+0000 osd.5 (osd.5) 139 : cluster [DBG] 5.7 scrub ok 2024-02-23T00:46:39.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:37.799860+0000 mgr.y (mgr.14517) 319 : audit [DBG] from='client.41117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:46:39.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:38.167793+0000 mon.a (mon.0) 2153 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:46:39.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:38.167950+0000 mon.a (mon.0) 2154 : cluster [DBG] osdmap e326: 8 total, 8 up, 8 in 2024-02-23T00:46:39.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:38.172836+0000 mon.a (mon.0) 2155 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:46:39.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:38.180071+0000 mon.b (mon.1) 360 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:39.464 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:46:39.496 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:39.515 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:37.329928+0000 osd.5 (osd.5) 138 : cluster [DBG] 5.7 scrub starts 2024-02-23T00:46:39.515 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:37.332969+0000 osd.5 (osd.5) 139 : cluster [DBG] 5.7 scrub ok 2024-02-23T00:46:39.515 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:37.799860+0000 mgr.y (mgr.14517) 319 : audit [DBG] from='client.41117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:46:39.515 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:38.167793+0000 mon.a (mon.0) 2153 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:46:39.516 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:38.167950+0000 mon.a (mon.0) 2154 : cluster [DBG] osdmap e326: 8 total, 8 up, 8 in 2024-02-23T00:46:39.516 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:38.172836+0000 mon.a (mon.0) 2155 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:46:39.516 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:38.180071+0000 mon.b (mon.1) 360 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:39.516 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:37.329928+0000 osd.5 (osd.5) 138 : cluster [DBG] 5.7 scrub starts 2024-02-23T00:46:39.516 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:37.332969+0000 osd.5 (osd.5) 139 : cluster [DBG] 5.7 scrub ok 2024-02-23T00:46:39.516 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:37.799860+0000 mgr.y (mgr.14517) 319 : audit [DBG] from='client.41117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:46:39.516 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:38.167793+0000 mon.a (mon.0) 2153 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:46:39.516 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:38.167950+0000 mon.a (mon.0) 2154 : cluster [DBG] osdmap e326: 8 total, 8 up, 8 in 2024-02-23T00:46:39.516 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:38.172836+0000 mon.a (mon.0) 2155 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:46:39.516 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:38.180071+0000 mon.b (mon.1) 360 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:39.518 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:46:39.520 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:39.521 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:46:39.622 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:46:39.651 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:39.775 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:46:39.782 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:39.782 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:46:39.883 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:46:39.916 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:39.931 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:46:39.931 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:46:39.937 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:46:39.939 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:39.940 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:46:40.041 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:46:40.071 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:40.091 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:46:40.094 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:40.094 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:46:40.195 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:46:40.225 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:40.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:39.169922+0000 mon.a (mon.0) 2156 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:46:40.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:39.169989+0000 mon.a (mon.0) 2157 : cluster [DBG] osdmap e327: 8 total, 8 up, 8 in 2024-02-23T00:46:40.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:39.172336+0000 mon.a (mon.0) 2158 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-43","var": "hit_set_count","val": "3"}]: dispatch 2024-02-23T00:46:40.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:39.177595+0000 mon.a (mon.0) 2159 : audit [INF] from='client.? 172.21.15.47:0/427115143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi047-82386-55"}]: dispatch 2024-02-23T00:46:40.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:39.183381+0000 mon.b (mon.1) 361 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:40.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:39.261226+0000 mgr.y (mgr.14517) 320 : cluster [DBG] pgmap v466: 237 pgs: 1 activating, 32 creating+peering, 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:40.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:39.262128+0000 mon.a (mon.0) 2160 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "18"}]: dispatch 2024-02-23T00:46:40.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:39.712966+0000 osd.7 (osd.7) 167 : cluster [DBG] 5.10 scrub starts 2024-02-23T00:46:40.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:39.715953+0000 osd.7 (osd.7) 168 : cluster [DBG] 5.10 scrub ok 2024-02-23T00:46:40.455 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:39.169922+0000 mon.a (mon.0) 2156 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:46:40.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:39.169989+0000 mon.a (mon.0) 2157 : cluster [DBG] osdmap e327: 8 total, 8 up, 8 in 2024-02-23T00:46:40.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:39.172336+0000 mon.a (mon.0) 2158 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-43","var": "hit_set_count","val": "3"}]: dispatch 2024-02-23T00:46:40.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:39.177595+0000 mon.a (mon.0) 2159 : audit [INF] from='client.? 172.21.15.47:0/427115143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi047-82386-55"}]: dispatch 2024-02-23T00:46:40.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:39.183381+0000 mon.b (mon.1) 361 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:40.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:39.261226+0000 mgr.y (mgr.14517) 320 : cluster [DBG] pgmap v466: 237 pgs: 1 activating, 32 creating+peering, 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:40.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:39.262128+0000 mon.a (mon.0) 2160 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "18"}]: dispatch 2024-02-23T00:46:40.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:39.712966+0000 osd.7 (osd.7) 167 : cluster [DBG] 5.10 scrub starts 2024-02-23T00:46:40.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:39.715953+0000 osd.7 (osd.7) 168 : cluster [DBG] 5.10 scrub ok 2024-02-23T00:46:40.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:39.169922+0000 mon.a (mon.0) 2156 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:46:40.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:39.169989+0000 mon.a (mon.0) 2157 : cluster [DBG] osdmap e327: 8 total, 8 up, 8 in 2024-02-23T00:46:40.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:39.172336+0000 mon.a (mon.0) 2158 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-43","var": "hit_set_count","val": "3"}]: dispatch 2024-02-23T00:46:40.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:39.177595+0000 mon.a (mon.0) 2159 : audit [INF] from='client.? 172.21.15.47:0/427115143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi047-82386-55"}]: dispatch 2024-02-23T00:46:40.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:39.183381+0000 mon.b (mon.1) 361 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:40.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:39.261226+0000 mgr.y (mgr.14517) 320 : cluster [DBG] pgmap v466: 237 pgs: 1 activating, 32 creating+peering, 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:40.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:39.262128+0000 mon.a (mon.0) 2160 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "18"}]: dispatch 2024-02-23T00:46:40.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:39.712966+0000 osd.7 (osd.7) 167 : cluster [DBG] 5.10 scrub starts 2024-02-23T00:46:40.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:39.715953+0000 osd.7 (osd.7) 168 : cluster [DBG] 5.10 scrub ok 2024-02-23T00:46:40.733 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:46:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:46:40] "GET /metrics HTTP/1.1" 200 190463 "" "Prometheus/2.33.4" 2024-02-23T00:46:40.754 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.4 dump_ops_in_flight 2024-02-23T00:46:40.976 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:46:40.980 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:40.980 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:46:41.005 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":463,"stamp":"2024-02-23T00:46:37.260010+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4009069,"num_objects":364,"num_object_clones":2,"num_object_copies":1092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":364,"num_whiteouts":0,"num_read":1640,"num_read_kb":1266,"num_write":1307,"num_write_kb":4571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":2249,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2590,"ondisk_log_size":2590,"up":738,"acting":738,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":715,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2518652,"kb_used_data":54780,"kb_used_omap":0,"kb_used_meta":2463744,"kb_avail":747442564,"statfs":{"total":767960285184,"available":765381185536,"internally_reserved":0,"allocated":56094720,"data_stored":38599541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2522873856},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000882"},"pg_stats":[{"pgid":"191.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.165135+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"191.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.167397+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,3],"acting":[1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"191.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.161202+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"191.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.161891+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7],"acting":[6,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"191.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.170439+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"191.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.168105+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"191.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.161424+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"191.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.164307+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"191.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.166616+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"191.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.167059+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"191.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.161943+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2],"acting":[6,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"191.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.166562+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"191.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.170100+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"191.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.168422+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"191.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.162900+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"191.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.161794+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"191.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.162352+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,3],"acting":[6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"191.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.164359+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"191.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.167264+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"191.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.164956+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"191.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.168009+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"191.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.165002+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"191.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.164782+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"191.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.165977+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"191.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.163689+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"190.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.161921+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"191.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.161422+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"190.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.166378+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"191.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.166195+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"190.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.163727+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"191.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.161452+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"190.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.161386+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"191.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.165749+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"190.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.166559+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,4],"acting":[1,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"191.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.167597+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6,7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"190.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.167532+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"191.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.165643+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"190.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.167127+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"191.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.164657+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6,7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"190.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:46:37.156905+0000","last_change":"2024-02-23T00:46:37.163601+0000","last_active":"2024-02-23T00:46:37.156905+0000","last_peered":"2024-02-23T00:46:37.156905+0000","last_clean":"2024-02-23T00:46:37.156905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:46:37.156905+0000","last_undegraded":"2024-02-23T00:46:37.156905+0000","last_fullsized":"2024-02-23T00:46:37.156905+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.0","version":"259'40","reported_seq":294,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.161895+0000","last_change":"2024-02-23T00:46:06.778112+0000","last_active":"2024-02-23T00:46:36.161895+0000","last_peered":"2024-02-23T00:46:36.161895+0000","last_clean":"2024-02-23T00:46:36.161895+0000","last_became_active":"2024-02-23T00:43:17.855316+0000","last_became_peered":"2024-02-23T00:43:17.855316+0000","last_unstale":"2024-02-23T00:46:36.161895+0000","last_undegraded":"2024-02-23T00:46:36.161895+0000","last_fullsized":"2024-02-23T00:46:36.161895+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'40","last_scrub_stamp":"2024-02-23T00:46:06.777995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:06.777995+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":31,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":1,"num_write":13,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.1","version":"230'87","reported_seq":328,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.400573+0000","last_change":"2024-02-23T00:46:00.167004+0000","last_active":"2024-02-23T00:46:36.400573+0000","last_peered":"2024-02-23T00:46:36.400573+0000","last_clean":"2024-02-23T00:46:36.400573+0000","last_became_active":"2024-02-23T00:43:17.851866+0000","last_became_peered":"2024-02-23T00:43:17.851866+0000","last_unstale":"2024-02-23T00:46:36.400573+0000","last_undegraded":"2024-02-23T00:46:36.400573+0000","last_fullsized":"2024-02-23T00:46:36.400573+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":87,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":63,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.2","version":"229'68","reported_seq":313,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.172764+0000","last_change":"2024-02-23T00:46:00.168915+0000","last_active":"2024-02-23T00:46:36.172764+0000","last_peered":"2024-02-23T00:46:36.172764+0000","last_clean":"2024-02-23T00:46:36.172764+0000","last_became_active":"2024-02-23T00:43:17.804792+0000","last_became_peered":"2024-02-23T00:43:17.804792+0000","last_unstale":"2024-02-23T00:46:36.172764+0000","last_undegraded":"2024-02-23T00:46:36.172764+0000","last_fullsized":"2024-02-23T00:46:36.172764+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'68","last_scrub_stamp":"2024-02-23T00:45:57.135726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:57.135726+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.3","version":"246'20","reported_seq":276,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.160775+0000","last_change":"2024-02-23T00:46:00.172490+0000","last_active":"2024-02-23T00:46:36.160775+0000","last_peered":"2024-02-23T00:46:36.160775+0000","last_clean":"2024-02-23T00:46:36.160775+0000","last_became_active":"2024-02-23T00:43:17.834686+0000","last_became_peered":"2024-02-23T00:43:17.834686+0000","last_unstale":"2024-02-23T00:46:36.160775+0000","last_undegraded":"2024-02-23T00:46:36.160775+0000","last_fullsized":"2024-02-23T00:46:36.160775+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"246'20","last_scrub_stamp":"2024-02-23T00:45:53.067737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:53.067737+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.4","version":"207'17","reported_seq":269,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.592273+0000","last_change":"2024-02-23T00:46:35.627548+0000","last_active":"2024-02-23T00:46:36.592273+0000","last_peered":"2024-02-23T00:46:36.592273+0000","last_clean":"2024-02-23T00:46:36.592273+0000","last_became_active":"2024-02-23T00:43:17.810343+0000","last_became_peered":"2024-02-23T00:43:17.810343+0000","last_unstale":"2024-02-23T00:46:36.592273+0000","last_undegraded":"2024-02-23T00:46:36.592273+0000","last_fullsized":"2024-02-23T00:46:36.592273+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"207'17","last_scrub_stamp":"2024-02-23T00:46:35.627480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:35.627480+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.5","version":"237'51","reported_seq":298,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.169645+0000","last_change":"2024-02-23T00:46:00.160547+0000","last_active":"2024-02-23T00:46:36.169645+0000","last_peered":"2024-02-23T00:46:36.169645+0000","last_clean":"2024-02-23T00:46:36.169645+0000","last_became_active":"2024-02-23T00:43:17.855339+0000","last_became_peered":"2024-02-23T00:43:17.855339+0000","last_unstale":"2024-02-23T00:46:36.169645+0000","last_undegraded":"2024-02-23T00:46:36.169645+0000","last_fullsized":"2024-02-23T00:46:36.169645+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.6","version":"247'51","reported_seq":304,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.163024+0000","last_change":"2024-02-23T00:46:00.726103+0000","last_active":"2024-02-23T00:46:36.163024+0000","last_peered":"2024-02-23T00:46:36.163024+0000","last_clean":"2024-02-23T00:46:36.163024+0000","last_became_active":"2024-02-23T00:43:17.838838+0000","last_became_peered":"2024-02-23T00:43:17.838838+0000","last_unstale":"2024-02-23T00:46:36.163024+0000","last_undegraded":"2024-02-23T00:46:36.163024+0000","last_fullsized":"2024-02-23T00:46:36.163024+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'51","last_scrub_stamp":"2024-02-23T00:46:00.726012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:00.726012+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.7","version":"215'68","reported_seq":306,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.160973+0000","last_change":"2024-02-23T00:46:00.175978+0000","last_active":"2024-02-23T00:46:36.160973+0000","last_peered":"2024-02-23T00:46:36.160973+0000","last_clean":"2024-02-23T00:46:36.160973+0000","last_became_active":"2024-02-23T00:43:17.846548+0000","last_became_peered":"2024-02-23T00:43:17.846548+0000","last_unstale":"2024-02-23T00:46:36.160973+0000","last_undegraded":"2024-02-23T00:46:36.160973+0000","last_fullsized":"2024-02-23T00:46:36.160973+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'68","last_scrub_stamp":"2024-02-23T00:45:25.351414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.351414+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.8","version":"259'90","reported_seq":330,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.161702+0000","last_change":"2024-02-23T00:46:05.820778+0000","last_active":"2024-02-23T00:46:36.161702+0000","last_peered":"2024-02-23T00:46:36.161702+0000","last_clean":"2024-02-23T00:46:36.161702+0000","last_became_active":"2024-02-23T00:43:17.842613+0000","last_became_peered":"2024-02-23T00:43:17.842613+0000","last_unstale":"2024-02-23T00:46:36.161702+0000","last_undegraded":"2024-02-23T00:46:36.161702+0000","last_fullsized":"2024-02-23T00:46:36.161702+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'90","last_scrub_stamp":"2024-02-23T00:46:05.820651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:05.820651+0000","log_size":90,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.9","version":"259'39","reported_seq":295,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.166634+0000","last_change":"2024-02-23T00:46:10.439444+0000","last_active":"2024-02-23T00:46:36.166634+0000","last_peered":"2024-02-23T00:46:36.166634+0000","last_clean":"2024-02-23T00:46:36.166634+0000","last_became_active":"2024-02-23T00:43:17.834297+0000","last_became_peered":"2024-02-23T00:43:17.834297+0000","last_unstale":"2024-02-23T00:46:36.166634+0000","last_undegraded":"2024-02-23T00:46:36.166634+0000","last_fullsized":"2024-02-23T00:46:36.166634+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'39","last_scrub_stamp":"2024-02-23T00:46:10.439373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:10.439373+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":12,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.a","version":"253'90","reported_seq":331,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.162450+0000","last_change":"2024-02-23T00:46:00.166527+0000","last_active":"2024-02-23T00:46:36.162450+0000","last_peered":"2024-02-23T00:46:36.162450+0000","last_clean":"2024-02-23T00:46:36.162450+0000","last_became_active":"2024-02-23T00:43:17.839142+0000","last_became_peered":"2024-02-23T00:43:17.839142+0000","last_unstale":"2024-02-23T00:46:36.162450+0000","last_undegraded":"2024-02-23T00:46:36.162450+0000","last_fullsized":"2024-02-23T00:46:36.162450+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"253'90","last_scrub_stamp":"2024-02-23T00:45:59.681669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:59.681669+0000","log_size":90,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.b","version":"288'76","reported_seq":318,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.162129+0000","last_change":"2024-02-23T00:46:07.817973+0000","last_active":"2024-02-23T00:46:36.162129+0000","last_peered":"2024-02-23T00:46:36.162129+0000","last_clean":"2024-02-23T00:46:36.162129+0000","last_became_active":"2024-02-23T00:43:17.855321+0000","last_became_peered":"2024-02-23T00:43:17.855321+0000","last_unstale":"2024-02-23T00:46:36.162129+0000","last_undegraded":"2024-02-23T00:46:36.162129+0000","last_fullsized":"2024-02-23T00:46:36.162129+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"288'76","last_scrub_stamp":"2024-02-23T00:46:07.817883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:07.817883+0000","log_size":76,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":46,"num_objects":6,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":23,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.c","version":"203'34","reported_seq":285,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.160784+0000","last_change":"2024-02-23T00:46:00.163652+0000","last_active":"2024-02-23T00:46:36.160784+0000","last_peered":"2024-02-23T00:46:36.160784+0000","last_clean":"2024-02-23T00:46:36.160784+0000","last_became_active":"2024-02-23T00:43:17.829044+0000","last_became_peered":"2024-02-23T00:43:17.829044+0000","last_unstale":"2024-02-23T00:46:36.160784+0000","last_undegraded":"2024-02-23T00:46:36.160784+0000","last_fullsized":"2024-02-23T00:46:36.160784+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'34","last_scrub_stamp":"2024-02-23T00:45:57.874883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:57.874883+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.d","version":"259'56","reported_seq":300,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.169508+0000","last_change":"2024-02-23T00:46:00.159535+0000","last_active":"2024-02-23T00:46:36.169508+0000","last_peered":"2024-02-23T00:46:36.169508+0000","last_clean":"2024-02-23T00:46:36.169508+0000","last_became_active":"2024-02-23T00:43:17.841614+0000","last_became_peered":"2024-02-23T00:43:17.841614+0000","last_unstale":"2024-02-23T00:46:36.169508+0000","last_undegraded":"2024-02-23T00:46:36.169508+0000","last_fullsized":"2024-02-23T00:46:36.169508+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'56","last_scrub_stamp":"2024-02-23T00:45:51.258745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:51.258745+0000","log_size":56,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":17,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.e","version":"211'51","reported_seq":296,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.593501+0000","last_change":"2024-02-23T00:46:11.491708+0000","last_active":"2024-02-23T00:46:36.593501+0000","last_peered":"2024-02-23T00:46:36.593501+0000","last_clean":"2024-02-23T00:46:36.593501+0000","last_became_active":"2024-02-23T00:43:17.845956+0000","last_became_peered":"2024-02-23T00:43:17.845956+0000","last_unstale":"2024-02-23T00:46:36.593501+0000","last_undegraded":"2024-02-23T00:46:36.593501+0000","last_fullsized":"2024-02-23T00:46:36.593501+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'51","last_scrub_stamp":"2024-02-23T00:46:11.491637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:11.491637+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.f","version":"241'136","reported_seq":366,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-02-23T00:46:35.318666+0000","last_change":"2024-02-23T00:46:35.318635+0000","last_active":"2024-02-23T00:46:35.318666+0000","last_peered":"2024-02-23T00:46:35.318666+0000","last_clean":"2024-02-23T00:46:35.318666+0000","last_became_active":"2024-02-23T00:43:17.855331+0000","last_became_peered":"2024-02-23T00:43:17.855331+0000","last_unstale":"2024-02-23T00:46:35.318666+0000","last_undegraded":"2024-02-23T00:46:35.318666+0000","last_fullsized":"2024-02-23T00:46:35.318666+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"241'136","last_scrub_stamp":"2024-02-23T00:46:35.318571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:35.318571+0000","log_size":136,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":88,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":40,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.10","version":"219'68","reported_seq":309,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.400793+0000","last_change":"2024-02-23T00:46:00.167847+0000","last_active":"2024-02-23T00:46:36.400793+0000","last_peered":"2024-02-23T00:46:36.400793+0000","last_clean":"2024-02-23T00:46:36.400793+0000","last_became_active":"2024-02-23T00:43:17.819865+0000","last_became_peered":"2024-02-23T00:43:17.819865+0000","last_unstale":"2024-02-23T00:46:36.400793+0000","last_undegraded":"2024-02-23T00:46:36.400793+0000","last_fullsized":"2024-02-23T00:46:36.400793+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.11","version":"215'51","reported_seq":289,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.400430+0000","last_change":"2024-02-23T00:46:00.164366+0000","last_active":"2024-02-23T00:46:36.400430+0000","last_peered":"2024-02-23T00:46:36.400430+0000","last_clean":"2024-02-23T00:46:36.400430+0000","last_became_active":"2024-02-23T00:43:17.852697+0000","last_became_peered":"2024-02-23T00:43:17.852697+0000","last_unstale":"2024-02-23T00:46:36.400430+0000","last_undegraded":"2024-02-23T00:46:36.400430+0000","last_fullsized":"2024-02-23T00:46:36.400430+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.12","version":"240'73","reported_seq":320,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.162442+0000","last_change":"2024-02-23T00:46:00.176356+0000","last_active":"2024-02-23T00:46:36.162442+0000","last_peered":"2024-02-23T00:46:36.162442+0000","last_clean":"2024-02-23T00:46:36.162442+0000","last_became_active":"2024-02-23T00:43:17.856118+0000","last_became_peered":"2024-02-23T00:43:17.856118+0000","last_unstale":"2024-02-23T00:46:36.162442+0000","last_undegraded":"2024-02-23T00:46:36.162442+0000","last_fullsized":"2024-02-23T00:46:36.162442+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'73","last_scrub_stamp":"2024-02-23T00:45:52.066508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:52.066508+0000","log_size":73,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":49,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.13","version":"205'34","reported_seq":286,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.591442+0000","last_change":"2024-02-23T00:46:09.499938+0000","last_active":"2024-02-23T00:46:36.591442+0000","last_peered":"2024-02-23T00:46:36.591442+0000","last_clean":"2024-02-23T00:46:36.591442+0000","last_became_active":"2024-02-23T00:43:17.857823+0000","last_became_peered":"2024-02-23T00:43:17.857823+0000","last_unstale":"2024-02-23T00:46:36.591442+0000","last_undegraded":"2024-02-23T00:46:36.591442+0000","last_fullsized":"2024-02-23T00:46:36.591442+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'34","last_scrub_stamp":"2024-02-23T00:46:09.499812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:09.499812+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.14","version":"296'44","reported_seq":292,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.163115+0000","last_change":"2024-02-23T00:46:03.795687+0000","last_active":"2024-02-23T00:46:36.163115+0000","last_peered":"2024-02-23T00:46:36.163115+0000","last_clean":"2024-02-23T00:46:36.163115+0000","last_became_active":"2024-02-23T00:43:17.816138+0000","last_became_peered":"2024-02-23T00:43:17.816138+0000","last_unstale":"2024-02-23T00:46:36.163115+0000","last_undegraded":"2024-02-23T00:46:36.163115+0000","last_fullsized":"2024-02-23T00:46:36.163115+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'39","last_scrub_stamp":"2024-02-23T00:46:03.795605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:03.795605+0000","log_size":44,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":35,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":17,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.15","version":"299'18","reported_seq":273,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.160063+0000","last_change":"2024-02-23T00:46:11.830641+0000","last_active":"2024-02-23T00:46:36.160063+0000","last_peered":"2024-02-23T00:46:36.160063+0000","last_clean":"2024-02-23T00:46:36.160063+0000","last_became_active":"2024-02-23T00:43:17.843834+0000","last_became_peered":"2024-02-23T00:43:17.843834+0000","last_unstale":"2024-02-23T00:46:36.160063+0000","last_undegraded":"2024-02-23T00:46:36.160063+0000","last_fullsized":"2024-02-23T00:46:36.160063+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"299'18","last_scrub_stamp":"2024-02-23T00:46:11.830580+0000","last_deep_scrub":"299'18","last_deep_scrub_stamp":"2024-02-23T00:46:11.830580+0000","last_clean_scrub_stamp":"2024-02-23T00:46:11.830580+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.16","version":"240'23","reported_seq":278,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.168939+0000","last_change":"2024-02-23T00:46:00.160705+0000","last_active":"2024-02-23T00:46:36.168939+0000","last_peered":"2024-02-23T00:46:36.168939+0000","last_clean":"2024-02-23T00:46:36.168939+0000","last_became_active":"2024-02-23T00:43:17.834291+0000","last_became_peered":"2024-02-23T00:43:17.834291+0000","last_unstale":"2024-02-23T00:46:36.168939+0000","last_undegraded":"2024-02-23T00:46:36.168939+0000","last_fullsized":"2024-02-23T00:46:36.168939+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'23","last_scrub_stamp":"2024-02-23T00:45:56.354881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:56.354881+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":14,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.17","version":"229'57","reported_seq":311,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.171892+0000","last_change":"2024-02-23T00:46:00.166482+0000","last_active":"2024-02-23T00:46:36.171892+0000","last_peered":"2024-02-23T00:46:36.171892+0000","last_clean":"2024-02-23T00:46:36.171892+0000","last_became_active":"2024-02-23T00:43:17.843525+0000","last_became_peered":"2024-02-23T00:43:17.843525+0000","last_unstale":"2024-02-23T00:46:36.171892+0000","last_undegraded":"2024-02-23T00:46:36.171892+0000","last_fullsized":"2024-02-23T00:46:36.171892+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'57","last_scrub_stamp":"2024-02-23T00:45:58.127302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:58.127302+0000","log_size":57,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":18,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.19","version":"267'96","reported_seq":345,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.170795+0000","last_change":"2024-02-23T00:46:00.163245+0000","last_active":"2024-02-23T00:46:36.170795+0000","last_peered":"2024-02-23T00:46:36.170795+0000","last_clean":"2024-02-23T00:46:36.170795+0000","last_became_active":"2024-02-23T00:43:17.815830+0000","last_became_peered":"2024-02-23T00:43:17.815830+0000","last_unstale":"2024-02-23T00:46:36.170795+0000","last_undegraded":"2024-02-23T00:46:36.170795+0000","last_fullsized":"2024-02-23T00:46:36.170795+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'96","last_scrub_stamp":"2024-02-23T00:45:55.109551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:55.109551+0000","log_size":96,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":73,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":7,"num_read_kb":0,"num_write":30,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.18","version":"275'7","reported_seq":267,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.158615+0000","last_change":"2024-02-23T00:46:03.927901+0000","last_active":"2024-02-23T00:46:36.158615+0000","last_peered":"2024-02-23T00:46:36.158615+0000","last_clean":"2024-02-23T00:46:36.158615+0000","last_became_active":"2024-02-23T00:43:17.840351+0000","last_became_peered":"2024-02-23T00:43:17.840351+0000","last_unstale":"2024-02-23T00:46:36.158615+0000","last_undegraded":"2024-02-23T00:46:36.158615+0000","last_fullsized":"2024-02-23T00:46:36.158615+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"275'7","last_scrub_stamp":"2024-02-23T00:46:03.927783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:03.927783+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1b","version":"304'69","reported_seq":308,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-02-23T00:46:35.177493+0000","last_change":"2024-02-23T00:46:00.158327+0000","last_active":"2024-02-23T00:46:35.177493+0000","last_peered":"2024-02-23T00:46:35.177493+0000","last_clean":"2024-02-23T00:46:35.177493+0000","last_became_active":"2024-02-23T00:43:17.803043+0000","last_became_peered":"2024-02-23T00:43:17.803043+0000","last_unstale":"2024-02-23T00:46:35.177493+0000","last_undegraded":"2024-02-23T00:46:35.177493+0000","last_fullsized":"2024-02-23T00:46:35.177493+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":69,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4000052,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":21,"num_write_kb":3927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.1a","version":"225'34","reported_seq":313,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.172643+0000","last_change":"2024-02-23T00:46:02.071916+0000","last_active":"2024-02-23T00:46:36.172643+0000","last_peered":"2024-02-23T00:46:36.172643+0000","last_clean":"2024-02-23T00:46:36.172643+0000","last_became_active":"2024-02-23T00:45:44.178195+0000","last_became_peered":"2024-02-23T00:45:44.178195+0000","last_unstale":"2024-02-23T00:46:36.172643+0000","last_undegraded":"2024-02-23T00:46:36.172643+0000","last_fullsized":"2024-02-23T00:46:36.172643+0000","mapping_epoch":271,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":272,"parent":"0.0","parent_split_bits":0,"last_scrub":"225'34","last_scrub_stamp":"2024-02-23T00:46:02.071770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:02.071770+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1d","version":"215'104","reported_seq":347,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.298951+0000","last_change":"2024-02-23T00:46:05.000709+0000","last_active":"2024-02-23T00:46:36.298951+0000","last_peered":"2024-02-23T00:46:36.298951+0000","last_clean":"2024-02-23T00:46:36.298951+0000","last_became_active":"2024-02-23T00:43:17.828194+0000","last_became_peered":"2024-02-23T00:43:17.828194+0000","last_unstale":"2024-02-23T00:46:36.298951+0000","last_undegraded":"2024-02-23T00:46:36.298951+0000","last_fullsized":"2024-02-23T00:46:36.298951+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'104","last_scrub_stamp":"2024-02-23T00:46:05.000615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:05.000615+0000","log_size":104,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":74,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":32,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"62.1c","version":"223'51","reported_seq":296,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.171681+0000","last_change":"2024-02-23T00:46:08.027717+0000","last_active":"2024-02-23T00:46:36.171681+0000","last_peered":"2024-02-23T00:46:36.171681+0000","last_clean":"2024-02-23T00:46:36.171681+0000","last_became_active":"2024-02-23T00:43:17.804718+0000","last_became_peered":"2024-02-23T00:43:17.804718+0000","last_unstale":"2024-02-23T00:46:36.171681+0000","last_undegraded":"2024-02-23T00:46:36.171681+0000","last_fullsized":"2024-02-23T00:46:36.171681+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'51","last_scrub_stamp":"2024-02-23T00:46:08.027648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:08.027648+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1f","version":"240'40","reported_seq":293,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.160875+0000","last_change":"2024-02-23T00:46:00.174886+0000","last_active":"2024-02-23T00:46:36.160875+0000","last_peered":"2024-02-23T00:46:36.160875+0000","last_clean":"2024-02-23T00:46:36.160875+0000","last_became_active":"2024-02-23T00:43:17.839914+0000","last_became_peered":"2024-02-23T00:43:17.839914+0000","last_unstale":"2024-02-23T00:46:36.160875+0000","last_undegraded":"2024-02-23T00:46:36.160875+0000","last_fullsized":"2024-02-23T00:46:36.160875+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'40","last_scrub_stamp":"2024-02-23T00:45:58.074860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:58.074860+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":13,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.1e","version":"245'51","reported_seq":297,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.163015+0000","last_change":"2024-02-23T00:46:00.172469+0000","last_active":"2024-02-23T00:46:36.163015+0000","last_peered":"2024-02-23T00:46:36.163015+0000","last_clean":"2024-02-23T00:46:36.163015+0000","last_became_active":"2024-02-23T00:43:17.838682+0000","last_became_peered":"2024-02-23T00:43:17.838682+0000","last_unstale":"2024-02-23T00:46:36.163015+0000","last_undegraded":"2024-02-23T00:46:36.163015+0000","last_fullsized":"2024-02-23T00:46:36.163015+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"245'51","last_scrub_stamp":"2024-02-23T00:45:58.723990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:58.723990+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"67'8","reported_seq":381,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.162557+0000","last_change":"2024-02-23T00:45:55.708656+0000","last_active":"2024-02-23T00:46:36.162557+0000","last_peered":"2024-02-23T00:46:36.162557+0000","last_clean":"2024-02-23T00:46:36.162557+0000","last_became_active":"2024-02-23T00:39:18.793414+0000","last_became_peered":"2024-02-23T00:39:18.793414+0000","last_unstale":"2024-02-23T00:46:36.162557+0000","last_undegraded":"2024-02-23T00:46:36.162557+0000","last_fullsized":"2024-02-23T00:46:36.162557+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:45:55.708576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:55.708576+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"68'27","reported_seq":342,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.297913+0000","last_change":"2024-02-23T00:45:04.138883+0000","last_active":"2024-02-23T00:46:36.297913+0000","last_peered":"2024-02-23T00:46:36.297913+0000","last_clean":"2024-02-23T00:46:36.297913+0000","last_became_active":"2024-02-23T00:39:16.859401+0000","last_became_peered":"2024-02-23T00:39:16.859401+0000","last_unstale":"2024-02-23T00:46:36.297913+0000","last_undegraded":"2024-02-23T00:46:36.297913+0000","last_fullsized":"2024-02-23T00:46:36.297913+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'27","last_scrub_stamp":"2024-02-23T00:45:04.138751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.138751+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":292,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.162201+0000","last_change":"2024-02-23T00:46:04.942156+0000","last_active":"2024-02-23T00:46:36.162201+0000","last_peered":"2024-02-23T00:46:36.162201+0000","last_clean":"2024-02-23T00:46:36.162201+0000","last_became_active":"2024-02-23T00:39:14.784436+0000","last_became_peered":"2024-02-23T00:39:14.784436+0000","last_unstale":"2024-02-23T00:46:36.162201+0000","last_undegraded":"2024-02-23T00:46:36.162201+0000","last_fullsized":"2024-02-23T00:46:36.162201+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.942091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:04.330426+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.942091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":301,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.297827+0000","last_change":"2024-02-23T00:44:21.972446+0000","last_active":"2024-02-23T00:46:36.297827+0000","last_peered":"2024-02-23T00:46:36.297827+0000","last_clean":"2024-02-23T00:46:36.297827+0000","last_became_active":"2024-02-23T00:43:32.282545+0000","last_became_peered":"2024-02-23T00:43:32.282545+0000","last_unstale":"2024-02-23T00:46:36.297827+0000","last_undegraded":"2024-02-23T00:46:36.297827+0000","last_fullsized":"2024-02-23T00:46:36.297827+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:21.972383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:21.972383+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.3","version":"307'7","reported_seq":239,"reported_epoch":325,"state":"activating","last_fresh":"2024-02-23T00:46:37.171276+0000","last_change":"2024-02-23T00:46:37.171470+0000","last_active":"2024-02-23T00:46:36.166246+0000","last_peered":"2024-02-23T00:46:35.177582+0000","last_clean":"2024-02-23T00:46:35.177582+0000","last_became_active":"2024-02-23T00:42:58.835612+0000","last_became_peered":"2024-02-23T00:42:58.835612+0000","last_unstale":"2024-02-23T00:46:37.171276+0000","last_undegraded":"2024-02-23T00:46:37.171276+0000","last_fullsized":"2024-02-23T00:46:37.171276+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'5","last_scrub_stamp":"2024-02-23T00:45:17.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:17.356406+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":["4","3","6"],"object_location_counts":[{"shards":"3,4,6","objects":1}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":230,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.299709+0000","last_change":"2024-02-23T00:45:17.181400+0000","last_active":"2024-02-23T00:46:36.299709+0000","last_peered":"2024-02-23T00:46:36.299709+0000","last_clean":"2024-02-23T00:46:36.299709+0000","last_became_active":"2024-02-23T00:42:58.828846+0000","last_became_peered":"2024-02-23T00:42:58.828846+0000","last_unstale":"2024-02-23T00:46:36.299709+0000","last_undegraded":"2024-02-23T00:46:36.299709+0000","last_fullsized":"2024-02-23T00:46:36.299709+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:17.181335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:17.181335+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"68'15","reported_seq":318,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.591673+0000","last_change":"2024-02-23T00:43:54.355050+0000","last_active":"2024-02-23T00:46:36.591673+0000","last_peered":"2024-02-23T00:46:36.591673+0000","last_clean":"2024-02-23T00:46:36.591673+0000","last_became_active":"2024-02-23T00:39:16.783701+0000","last_became_peered":"2024-02-23T00:39:16.783701+0000","last_unstale":"2024-02-23T00:46:36.591673+0000","last_undegraded":"2024-02-23T00:46:36.591673+0000","last_fullsized":"2024-02-23T00:46:36.591673+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:43:54.354974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:54.354974+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":288,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.403858+0000","last_change":"2024-02-23T00:46:04.476277+0000","last_active":"2024-02-23T00:46:36.403858+0000","last_peered":"2024-02-23T00:46:36.403858+0000","last_clean":"2024-02-23T00:46:36.403858+0000","last_became_active":"2024-02-23T00:39:14.772493+0000","last_became_peered":"2024-02-23T00:39:14.772493+0000","last_unstale":"2024-02-23T00:46:36.403858+0000","last_undegraded":"2024-02-23T00:46:36.403858+0000","last_fullsized":"2024-02-23T00:46:36.403858+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.476156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.476156+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":283,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.162679+0000","last_change":"2024-02-23T00:44:22.380954+0000","last_active":"2024-02-23T00:46:36.162679+0000","last_peered":"2024-02-23T00:46:36.162679+0000","last_clean":"2024-02-23T00:46:36.162679+0000","last_became_active":"2024-02-23T00:39:18.794476+0000","last_became_peered":"2024-02-23T00:39:18.794476+0000","last_unstale":"2024-02-23T00:46:36.162679+0000","last_undegraded":"2024-02-23T00:46:36.162679+0000","last_fullsized":"2024-02-23T00:46:36.162679+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:22.380841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:08.335330+0000","last_clean_scrub_stamp":"2024-02-23T00:44:22.380841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":301,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.592228+0000","last_change":"2024-02-23T00:45:50.685890+0000","last_active":"2024-02-23T00:46:36.592228+0000","last_peered":"2024-02-23T00:46:36.592228+0000","last_clean":"2024-02-23T00:46:36.592228+0000","last_became_active":"2024-02-23T00:43:35.079004+0000","last_became_peered":"2024-02-23T00:43:35.079004+0000","last_unstale":"2024-02-23T00:46:36.592228+0000","last_undegraded":"2024-02-23T00:46:36.592228+0000","last_fullsized":"2024-02-23T00:46:36.592228+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:50.685824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:45:50.685824+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"307'7","reported_seq":236,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.163500+0000","last_change":"2024-02-23T00:45:06.483322+0000","last_active":"2024-02-23T00:46:36.163500+0000","last_peered":"2024-02-23T00:46:36.163500+0000","last_clean":"2024-02-23T00:46:36.163500+0000","last_became_active":"2024-02-23T00:42:58.741558+0000","last_became_peered":"2024-02-23T00:42:58.741558+0000","last_unstale":"2024-02-23T00:46:36.163500+0000","last_undegraded":"2024-02-23T00:46:36.163500+0000","last_fullsized":"2024-02-23T00:46:36.163500+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'5","last_scrub_stamp":"2024-02-23T00:45:06.483252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.483252+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":230,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.591603+0000","last_change":"2024-02-23T00:45:22.597544+0000","last_active":"2024-02-23T00:46:36.591603+0000","last_peered":"2024-02-23T00:46:36.591603+0000","last_clean":"2024-02-23T00:46:36.591603+0000","last_became_active":"2024-02-23T00:42:58.867305+0000","last_became_peered":"2024-02-23T00:42:58.867305+0000","last_unstale":"2024-02-23T00:46:36.591603+0000","last_undegraded":"2024-02-23T00:46:36.591603+0000","last_fullsized":"2024-02-23T00:46:36.591603+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:22.597434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:22.597434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"88'35","reported_seq":341,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.173646+0000","last_change":"2024-02-23T00:43:54.438282+0000","last_active":"2024-02-23T00:46:36.173646+0000","last_peered":"2024-02-23T00:46:36.173646+0000","last_clean":"2024-02-23T00:46:36.173646+0000","last_became_active":"2024-02-23T00:39:16.857413+0000","last_became_peered":"2024-02-23T00:39:16.857413+0000","last_unstale":"2024-02-23T00:46:36.173646+0000","last_undegraded":"2024-02-23T00:46:36.173646+0000","last_fullsized":"2024-02-23T00:46:36.173646+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'35","last_scrub_stamp":"2024-02-23T00:43:54.438168+0000","last_deep_scrub":"88'35","last_deep_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_clean_scrub_stamp":"2024-02-23T00:43:54.438168+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":292,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.165018+0000","last_change":"2024-02-23T00:46:05.910392+0000","last_active":"2024-02-23T00:46:36.165018+0000","last_peered":"2024-02-23T00:46:36.165018+0000","last_clean":"2024-02-23T00:46:36.165018+0000","last_became_active":"2024-02-23T00:39:14.776705+0000","last_became_peered":"2024-02-23T00:39:14.776705+0000","last_unstale":"2024-02-23T00:46:36.165018+0000","last_undegraded":"2024-02-23T00:46:36.165018+0000","last_fullsized":"2024-02-23T00:46:36.165018+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:05.910325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_clean_scrub_stamp":"2024-02-23T00:46:05.910325+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":288,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.160865+0000","last_change":"2024-02-23T00:46:34.726091+0000","last_active":"2024-02-23T00:46:36.160865+0000","last_peered":"2024-02-23T00:46:36.160865+0000","last_clean":"2024-02-23T00:46:36.160865+0000","last_became_active":"2024-02-23T00:39:18.792718+0000","last_became_peered":"2024-02-23T00:39:18.792718+0000","last_unstale":"2024-02-23T00:46:36.160865+0000","last_undegraded":"2024-02-23T00:46:36.160865+0000","last_fullsized":"2024-02-23T00:46:36.160865+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:34.726027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:34.726027+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":298,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.162011+0000","last_change":"2024-02-23T00:44:36.758043+0000","last_active":"2024-02-23T00:46:36.162011+0000","last_peered":"2024-02-23T00:46:36.162011+0000","last_clean":"2024-02-23T00:46:36.162011+0000","last_became_active":"2024-02-23T00:43:39.470999+0000","last_became_peered":"2024-02-23T00:43:39.470999+0000","last_unstale":"2024-02-23T00:46:36.162011+0000","last_undegraded":"2024-02-23T00:46:36.162011+0000","last_fullsized":"2024-02-23T00:46:36.162011+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:36.757960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:36.757960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"307'7","reported_seq":236,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.171560+0000","last_change":"2024-02-23T00:45:23.832372+0000","last_active":"2024-02-23T00:46:36.171560+0000","last_peered":"2024-02-23T00:46:36.171560+0000","last_clean":"2024-02-23T00:46:36.171560+0000","last_became_active":"2024-02-23T00:42:58.684001+0000","last_became_peered":"2024-02-23T00:42:58.684001+0000","last_unstale":"2024-02-23T00:46:36.171560+0000","last_undegraded":"2024-02-23T00:46:36.171560+0000","last_fullsized":"2024-02-23T00:46:36.171560+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'5","last_scrub_stamp":"2024-02-23T00:45:23.832291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.832291+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":225,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.400530+0000","last_change":"2024-02-23T00:42:57.641727+0000","last_active":"2024-02-23T00:46:36.400530+0000","last_peered":"2024-02-23T00:46:36.400530+0000","last_clean":"2024-02-23T00:46:36.400530+0000","last_became_active":"2024-02-23T00:42:57.639013+0000","last_became_peered":"2024-02-23T00:42:57.639013+0000","last_unstale":"2024-02-23T00:46:36.400530+0000","last_undegraded":"2024-02-23T00:46:36.400530+0000","last_fullsized":"2024-02-23T00:46:36.400530+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"67'17","reported_seq":335,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-02-23T00:46:35.177647+0000","last_change":"2024-02-23T00:43:45.194207+0000","last_active":"2024-02-23T00:46:35.177647+0000","last_peered":"2024-02-23T00:46:35.177647+0000","last_clean":"2024-02-23T00:46:35.177647+0000","last_became_active":"2024-02-23T00:40:42.933806+0000","last_became_peered":"2024-02-23T00:40:42.933806+0000","last_unstale":"2024-02-23T00:46:35.177647+0000","last_undegraded":"2024-02-23T00:46:35.177647+0000","last_fullsized":"2024-02-23T00:46:35.177647+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'17","last_scrub_stamp":"2024-02-23T00:43:45.194139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.194139+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":263,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.161611+0000","last_change":"2024-02-23T00:45:56.051644+0000","last_active":"2024-02-23T00:46:36.161611+0000","last_peered":"2024-02-23T00:46:36.161611+0000","last_clean":"2024-02-23T00:46:36.161611+0000","last_became_active":"2024-02-23T00:40:42.931041+0000","last_became_peered":"2024-02-23T00:40:42.931041+0000","last_unstale":"2024-02-23T00:46:36.161611+0000","last_undegraded":"2024-02-23T00:46:36.161611+0000","last_fullsized":"2024-02-23T00:46:36.161611+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:56.051575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:56.051575+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":284,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.401024+0000","last_change":"2024-02-23T00:44:22.804352+0000","last_active":"2024-02-23T00:46:36.401024+0000","last_peered":"2024-02-23T00:46:36.401024+0000","last_clean":"2024-02-23T00:46:36.401024+0000","last_became_active":"2024-02-23T00:39:18.799875+0000","last_became_peered":"2024-02-23T00:39:18.799875+0000","last_unstale":"2024-02-23T00:46:36.401024+0000","last_undegraded":"2024-02-23T00:46:36.401024+0000","last_fullsized":"2024-02-23T00:46:36.401024+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:22.804279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:22.804279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":299,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.161963+0000","last_change":"2024-02-23T00:44:36.481327+0000","last_active":"2024-02-23T00:46:36.161963+0000","last_peered":"2024-02-23T00:46:36.161963+0000","last_clean":"2024-02-23T00:46:36.161963+0000","last_became_active":"2024-02-23T00:43:08.885257+0000","last_became_peered":"2024-02-23T00:43:08.885257+0000","last_unstale":"2024-02-23T00:46:36.161963+0000","last_undegraded":"2024-02-23T00:46:36.161963+0000","last_fullsized":"2024-02-23T00:46:36.161963+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:36.481256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:36.481256+0000","last_clean_scrub_stamp":"2024-02-23T00:44:36.481256+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.6","version":"307'11","reported_seq":259,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.593840+0000","last_change":"2024-02-23T00:46:27.159701+0000","last_active":"2024-02-23T00:46:36.593840+0000","last_peered":"2024-02-23T00:46:36.593840+0000","last_clean":"2024-02-23T00:46:36.593840+0000","last_became_active":"2024-02-23T00:46:27.159344+0000","last_became_peered":"2024-02-23T00:46:27.159344+0000","last_unstale":"2024-02-23T00:46:36.593840+0000","last_undegraded":"2024-02-23T00:46:36.593840+0000","last_fullsized":"2024-02-23T00:46:36.593840+0000","mapping_epoch":108,"log_start":"307'11","ondisk_log_start":"307'11","created":108,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'9","last_scrub_stamp":"2024-02-23T00:46:08.530434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:46:08.530434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":230,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.400753+0000","last_change":"2024-02-23T00:45:23.744990+0000","last_active":"2024-02-23T00:46:36.400753+0000","last_peered":"2024-02-23T00:46:36.400753+0000","last_clean":"2024-02-23T00:46:36.400753+0000","last_became_active":"2024-02-23T00:42:57.675306+0000","last_became_peered":"2024-02-23T00:42:57.675306+0000","last_unstale":"2024-02-23T00:46:36.400753+0000","last_undegraded":"2024-02-23T00:46:36.400753+0000","last_fullsized":"2024-02-23T00:46:36.400753+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:23.744846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.744846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"67'10","reported_seq":301,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.168303+0000","last_change":"2024-02-23T00:45:25.175258+0000","last_active":"2024-02-23T00:46:36.168303+0000","last_peered":"2024-02-23T00:46:36.168303+0000","last_clean":"2024-02-23T00:46:36.168303+0000","last_became_active":"2024-02-23T00:39:16.777530+0000","last_became_peered":"2024-02-23T00:39:16.777530+0000","last_unstale":"2024-02-23T00:46:36.168303+0000","last_undegraded":"2024-02-23T00:46:36.168303+0000","last_fullsized":"2024-02-23T00:46:36.168303+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'10","last_scrub_stamp":"2024-02-23T00:45:25.175173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.175173+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":288,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.400138+0000","last_change":"2024-02-23T00:45:10.693332+0000","last_active":"2024-02-23T00:46:36.400138+0000","last_peered":"2024-02-23T00:46:36.400138+0000","last_clean":"2024-02-23T00:46:36.400138+0000","last_became_active":"2024-02-23T00:39:14.780379+0000","last_became_peered":"2024-02-23T00:39:14.780379+0000","last_unstale":"2024-02-23T00:46:36.400138+0000","last_undegraded":"2024-02-23T00:46:36.400138+0000","last_fullsized":"2024-02-23T00:46:36.400138+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.693273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.575242+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.693273+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"67'8","reported_seq":379,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.164824+0000","last_change":"2024-02-23T00:44:25.470245+0000","last_active":"2024-02-23T00:46:36.164824+0000","last_peered":"2024-02-23T00:46:36.164824+0000","last_clean":"2024-02-23T00:46:36.164824+0000","last_became_active":"2024-02-23T00:39:18.799544+0000","last_became_peered":"2024-02-23T00:39:18.799544+0000","last_unstale":"2024-02-23T00:46:36.164824+0000","last_undegraded":"2024-02-23T00:46:36.164824+0000","last_fullsized":"2024-02-23T00:46:36.164824+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:44:25.470174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:25.470174+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":296,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-02-23T00:46:35.177688+0000","last_change":"2024-02-23T00:43:36.205958+0000","last_active":"2024-02-23T00:46:35.177688+0000","last_peered":"2024-02-23T00:46:35.177688+0000","last_clean":"2024-02-23T00:46:35.177688+0000","last_became_active":"2024-02-23T00:43:13.338697+0000","last_became_peered":"2024-02-23T00:43:13.338697+0000","last_unstale":"2024-02-23T00:46:35.177688+0000","last_undegraded":"2024-02-23T00:46:35.177688+0000","last_fullsized":"2024-02-23T00:46:35.177688+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:36.205886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:15.081153+0000","last_clean_scrub_stamp":"2024-02-23T00:43:36.205886+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.7","version":"307'7","reported_seq":243,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.402010+0000","last_change":"2024-02-23T00:46:23.199817+0000","last_active":"2024-02-23T00:46:36.402010+0000","last_peered":"2024-02-23T00:46:36.402010+0000","last_clean":"2024-02-23T00:46:36.402010+0000","last_became_active":"2024-02-23T00:46:23.199475+0000","last_became_peered":"2024-02-23T00:46:23.199475+0000","last_unstale":"2024-02-23T00:46:36.402010+0000","last_undegraded":"2024-02-23T00:46:36.402010+0000","last_fullsized":"2024-02-23T00:46:36.402010+0000","mapping_epoch":108,"log_start":"307'7","ondisk_log_start":"307'7","created":108,"last_epoch_clean":311,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":230,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.160560+0000","last_change":"2024-02-23T00:45:22.874761+0000","last_active":"2024-02-23T00:46:36.160560+0000","last_peered":"2024-02-23T00:46:36.160560+0000","last_clean":"2024-02-23T00:46:36.160560+0000","last_became_active":"2024-02-23T00:42:58.765106+0000","last_became_peered":"2024-02-23T00:42:58.765106+0000","last_unstale":"2024-02-23T00:46:36.160560+0000","last_undegraded":"2024-02-23T00:46:36.160560+0000","last_fullsized":"2024-02-23T00:46:36.160560+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:22.874696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:22.874696+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"67'15","reported_seq":311,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.402096+0000","last_change":"2024-02-23T00:44:19.830383+0000","last_active":"2024-02-23T00:46:36.402096+0000","last_peered":"2024-02-23T00:46:36.402096+0000","last_clean":"2024-02-23T00:46:36.402096+0000","last_became_active":"2024-02-23T00:39:16.789757+0000","last_became_peered":"2024-02-23T00:39:16.789757+0000","last_unstale":"2024-02-23T00:46:36.402096+0000","last_undegraded":"2024-02-23T00:46:36.402096+0000","last_fullsized":"2024-02-23T00:46:36.402096+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:44:19.830309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:44:19.830309+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":288,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.401131+0000","last_change":"2024-02-23T00:44:29.773000+0000","last_active":"2024-02-23T00:46:36.401131+0000","last_peered":"2024-02-23T00:46:36.401131+0000","last_clean":"2024-02-23T00:46:36.401131+0000","last_became_active":"2024-02-23T00:39:14.780526+0000","last_became_peered":"2024-02-23T00:39:14.780526+0000","last_unstale":"2024-02-23T00:46:36.401131+0000","last_undegraded":"2024-02-23T00:46:36.401131+0000","last_fullsized":"2024-02-23T00:46:36.401131+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:29.772933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:29.772933+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":380,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.168927+0000","last_change":"2024-02-23T00:44:23.109508+0000","last_active":"2024-02-23T00:46:36.168927+0000","last_peered":"2024-02-23T00:46:36.168927+0000","last_clean":"2024-02-23T00:46:36.168927+0000","last_became_active":"2024-02-23T00:39:18.800440+0000","last_became_peered":"2024-02-23T00:39:18.800440+0000","last_unstale":"2024-02-23T00:46:36.168927+0000","last_undegraded":"2024-02-23T00:46:36.168927+0000","last_fullsized":"2024-02-23T00:46:36.168927+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:44:23.109439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.109439+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":301,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.299686+0000","last_change":"2024-02-23T00:44:22.976713+0000","last_active":"2024-02-23T00:46:36.299686+0000","last_peered":"2024-02-23T00:46:36.299686+0000","last_clean":"2024-02-23T00:46:36.299686+0000","last_became_active":"2024-02-23T00:43:16.650527+0000","last_became_peered":"2024-02-23T00:43:16.650527+0000","last_unstale":"2024-02-23T00:46:36.299686+0000","last_undegraded":"2024-02-23T00:46:36.299686+0000","last_fullsized":"2024-02-23T00:46:36.299686+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:22.976611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:10.666299+0000","last_clean_scrub_stamp":"2024-02-23T00:44:22.976611+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"110'2","reported_seq":248,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.400631+0000","last_change":"2024-02-23T00:46:35.159640+0000","last_active":"2024-02-23T00:46:36.400631+0000","last_peered":"2024-02-23T00:46:36.400631+0000","last_clean":"2024-02-23T00:46:36.400631+0000","last_became_active":"2024-02-23T00:46:35.159266+0000","last_became_peered":"2024-02-23T00:46:35.159266+0000","last_unstale":"2024-02-23T00:46:36.400631+0000","last_undegraded":"2024-02-23T00:46:36.400631+0000","last_fullsized":"2024-02-23T00:46:36.400631+0000","mapping_epoch":108,"log_start":"110'2","ondisk_log_start":"110'2","created":108,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:26.719011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:26.719011+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,3],"acting":[1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":230,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.166730+0000","last_change":"2024-02-23T00:45:54.311565+0000","last_active":"2024-02-23T00:46:36.166730+0000","last_peered":"2024-02-23T00:46:36.166730+0000","last_clean":"2024-02-23T00:46:36.166730+0000","last_became_active":"2024-02-23T00:42:58.833924+0000","last_became_peered":"2024-02-23T00:42:58.833924+0000","last_unstale":"2024-02-23T00:46:36.166730+0000","last_undegraded":"2024-02-23T00:46:36.166730+0000","last_fullsized":"2024-02-23T00:46:36.166730+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:54.311442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:54.311442+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"68'17","reported_seq":338,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.168425+0000","last_change":"2024-02-23T00:43:33.151471+0000","last_active":"2024-02-23T00:46:36.168425+0000","last_peered":"2024-02-23T00:46:36.168425+0000","last_clean":"2024-02-23T00:46:36.168425+0000","last_became_active":"2024-02-23T00:40:42.933759+0000","last_became_peered":"2024-02-23T00:40:42.933759+0000","last_unstale":"2024-02-23T00:46:36.168425+0000","last_undegraded":"2024-02-23T00:46:36.168425+0000","last_fullsized":"2024-02-23T00:46:36.168425+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:43:33.151401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:33.151401+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":286,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.162578+0000","last_change":"2024-02-23T00:44:18.886545+0000","last_active":"2024-02-23T00:46:36.162578+0000","last_peered":"2024-02-23T00:46:36.162578+0000","last_clean":"2024-02-23T00:46:36.162578+0000","last_became_active":"2024-02-23T00:39:14.784121+0000","last_became_peered":"2024-02-23T00:39:14.784121+0000","last_unstale":"2024-02-23T00:46:36.162578+0000","last_undegraded":"2024-02-23T00:46:36.162578+0000","last_fullsized":"2024-02-23T00:46:36.162578+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:18.886477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:18.886477+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":285,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.300597+0000","last_change":"2024-02-23T00:45:08.118391+0000","last_active":"2024-02-23T00:46:36.300597+0000","last_peered":"2024-02-23T00:46:36.300597+0000","last_clean":"2024-02-23T00:46:36.300597+0000","last_became_active":"2024-02-23T00:39:18.800777+0000","last_became_peered":"2024-02-23T00:39:18.800777+0000","last_unstale":"2024-02-23T00:46:36.300597+0000","last_undegraded":"2024-02-23T00:46:36.300597+0000","last_fullsized":"2024-02-23T00:46:36.300597+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.118302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.118302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"67'1","reported_seq":301,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.160943+0000","last_change":"2024-02-23T00:43:56.091057+0000","last_active":"2024-02-23T00:46:36.160943+0000","last_peered":"2024-02-23T00:46:36.160943+0000","last_clean":"2024-02-23T00:46:36.160943+0000","last_became_active":"2024-02-23T00:43:22.866277+0000","last_became_peered":"2024-02-23T00:43:22.866277+0000","last_unstale":"2024-02-23T00:46:36.160943+0000","last_undegraded":"2024-02-23T00:46:36.160943+0000","last_fullsized":"2024-02-23T00:46:36.160943+0000","mapping_epoch":65,"log_start":"67'1","ondisk_log_start":"67'1","created":65,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:43:56.090982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:09.079918+0000","last_clean_scrub_stamp":"2024-02-23T00:43:56.090982+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":247,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.173772+0000","last_change":"2024-02-23T00:46:31.172051+0000","last_active":"2024-02-23T00:46:36.173772+0000","last_peered":"2024-02-23T00:46:36.173772+0000","last_clean":"2024-02-23T00:46:36.173772+0000","last_became_active":"2024-02-23T00:46:31.171712+0000","last_became_peered":"2024-02-23T00:46:31.171712+0000","last_unstale":"2024-02-23T00:46:36.173772+0000","last_undegraded":"2024-02-23T00:46:36.173772+0000","last_fullsized":"2024-02-23T00:46:36.173772+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.651711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.651711+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":230,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.401354+0000","last_change":"2024-02-23T00:46:35.407560+0000","last_active":"2024-02-23T00:46:36.401354+0000","last_peered":"2024-02-23T00:46:36.401354+0000","last_clean":"2024-02-23T00:46:36.401354+0000","last_became_active":"2024-02-23T00:42:57.653279+0000","last_became_peered":"2024-02-23T00:42:57.653279+0000","last_unstale":"2024-02-23T00:46:36.401354+0000","last_undegraded":"2024-02-23T00:46:36.401354+0000","last_fullsized":"2024-02-23T00:46:36.401354+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:35.407468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:46:35.407468+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":381,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.158810+0000","last_change":"2024-02-23T00:45:51.871688+0000","last_active":"2024-02-23T00:46:36.158810+0000","last_peered":"2024-02-23T00:46:36.158810+0000","last_clean":"2024-02-23T00:46:36.158810+0000","last_became_active":"2024-02-23T00:39:18.798536+0000","last_became_peered":"2024-02-23T00:39:18.798536+0000","last_unstale":"2024-02-23T00:46:36.158810+0000","last_undegraded":"2024-02-23T00:46:36.158810+0000","last_fullsized":"2024-02-23T00:46:36.158810+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:45:51.871618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:51.871618+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"68'20","reported_seq":322,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.167014+0000","last_change":"2024-02-23T00:45:05.291909+0000","last_active":"2024-02-23T00:46:36.167014+0000","last_peered":"2024-02-23T00:46:36.167014+0000","last_clean":"2024-02-23T00:46:36.167014+0000","last_became_active":"2024-02-23T00:39:16.780130+0000","last_became_peered":"2024-02-23T00:39:16.780130+0000","last_unstale":"2024-02-23T00:46:36.167014+0000","last_undegraded":"2024-02-23T00:46:36.167014+0000","last_fullsized":"2024-02-23T00:46:36.167014+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'20","last_scrub_stamp":"2024-02-23T00:45:05.291836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:05.291836+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":294,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.172155+0000","last_change":"2024-02-23T00:46:35.220317+0000","last_active":"2024-02-23T00:46:36.172155+0000","last_peered":"2024-02-23T00:46:36.172155+0000","last_clean":"2024-02-23T00:46:36.172155+0000","last_became_active":"2024-02-23T00:39:14.781999+0000","last_became_peered":"2024-02-23T00:39:14.781999+0000","last_unstale":"2024-02-23T00:46:36.172155+0000","last_undegraded":"2024-02-23T00:46:36.172155+0000","last_fullsized":"2024-02-23T00:46:36.172155+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:35.220225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:51.391939+0000","last_clean_scrub_stamp":"2024-02-23T00:46:35.220225+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":280,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.162921+0000","last_change":"2024-02-23T00:44:24.759694+0000","last_active":"2024-02-23T00:46:36.162921+0000","last_peered":"2024-02-23T00:46:36.162921+0000","last_clean":"2024-02-23T00:46:36.162921+0000","last_became_active":"2024-02-23T00:39:21.161928+0000","last_became_peered":"2024-02-23T00:39:21.161928+0000","last_unstale":"2024-02-23T00:46:36.162921+0000","last_undegraded":"2024-02-23T00:46:36.162921+0000","last_fullsized":"2024-02-23T00:46:36.162921+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:24.759627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.945642+0000","last_clean_scrub_stamp":"2024-02-23T00:44:24.759627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"307'16","reported_seq":263,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.162908+0000","last_change":"2024-02-23T00:46:11.156483+0000","last_active":"2024-02-23T00:46:36.162908+0000","last_peered":"2024-02-23T00:46:36.162908+0000","last_clean":"2024-02-23T00:46:36.162908+0000","last_became_active":"2024-02-23T00:46:11.156132+0000","last_became_peered":"2024-02-23T00:46:11.156132+0000","last_unstale":"2024-02-23T00:46:36.162908+0000","last_undegraded":"2024-02-23T00:46:36.162908+0000","last_fullsized":"2024-02-23T00:46:36.162908+0000","mapping_epoch":108,"log_start":"110'10","ondisk_log_start":"110'10","created":108,"last_epoch_clean":299,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'10","last_scrub_stamp":"2024-02-23T00:45:24.377329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:24.377329+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":384,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":230,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.300909+0000","last_change":"2024-02-23T00:45:59.944702+0000","last_active":"2024-02-23T00:46:36.300909+0000","last_peered":"2024-02-23T00:46:36.300909+0000","last_clean":"2024-02-23T00:46:36.300909+0000","last_became_active":"2024-02-23T00:42:58.717371+0000","last_became_peered":"2024-02-23T00:42:58.717371+0000","last_unstale":"2024-02-23T00:46:36.300909+0000","last_undegraded":"2024-02-23T00:46:36.300909+0000","last_fullsized":"2024-02-23T00:46:36.300909+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:59.944587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:59.944587+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":284,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.167551+0000","last_change":"2024-02-23T00:44:34.059194+0000","last_active":"2024-02-23T00:46:36.167551+0000","last_peered":"2024-02-23T00:46:36.167551+0000","last_clean":"2024-02-23T00:46:36.167551+0000","last_became_active":"2024-02-23T00:39:18.798829+0000","last_became_peered":"2024-02-23T00:39:18.798829+0000","last_unstale":"2024-02-23T00:46:36.167551+0000","last_undegraded":"2024-02-23T00:46:36.167551+0000","last_fullsized":"2024-02-23T00:46:36.167551+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:34.059129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:34.059129+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"251'40","reported_seq":359,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.158764+0000","last_change":"2024-02-23T00:45:23.910609+0000","last_active":"2024-02-23T00:46:36.158764+0000","last_peered":"2024-02-23T00:46:36.158764+0000","last_clean":"2024-02-23T00:46:36.158764+0000","last_became_active":"2024-02-23T00:39:16.859002+0000","last_became_peered":"2024-02-23T00:39:16.859002+0000","last_unstale":"2024-02-23T00:46:36.158764+0000","last_undegraded":"2024-02-23T00:46:36.158764+0000","last_fullsized":"2024-02-23T00:46:36.158764+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'40","last_scrub_stamp":"2024-02-23T00:45:23.910546+0000","last_deep_scrub":"251'40","last_deep_scrub_stamp":"2024-02-23T00:45:23.910546+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.910546+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":288,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.401693+0000","last_change":"2024-02-23T00:45:16.664785+0000","last_active":"2024-02-23T00:46:36.401693+0000","last_peered":"2024-02-23T00:46:36.401693+0000","last_clean":"2024-02-23T00:46:36.401693+0000","last_became_active":"2024-02-23T00:39:14.779193+0000","last_became_peered":"2024-02-23T00:39:14.779193+0000","last_unstale":"2024-02-23T00:46:36.401693+0000","last_undegraded":"2024-02-23T00:46:36.401693+0000","last_fullsized":"2024-02-23T00:46:36.401693+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.664682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.664682+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":282,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.167894+0000","last_change":"2024-02-23T00:45:03.228087+0000","last_active":"2024-02-23T00:46:36.167894+0000","last_peered":"2024-02-23T00:46:36.167894+0000","last_clean":"2024-02-23T00:46:36.167894+0000","last_became_active":"2024-02-23T00:39:20.795271+0000","last_became_peered":"2024-02-23T00:39:20.795271+0000","last_unstale":"2024-02-23T00:46:36.167894+0000","last_undegraded":"2024-02-23T00:46:36.167894+0000","last_fullsized":"2024-02-23T00:46:36.167894+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:03.227995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:45:03.227995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"307'14","reported_seq":100,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-02-23T00:46:35.178182+0000","last_change":"2024-02-23T00:46:07.156105+0000","last_active":"2024-02-23T00:46:35.178182+0000","last_peered":"2024-02-23T00:46:35.178182+0000","last_clean":"2024-02-23T00:46:35.178182+0000","last_became_active":"2024-02-23T00:46:07.155723+0000","last_became_peered":"2024-02-23T00:46:07.155723+0000","last_unstale":"2024-02-23T00:46:35.178182+0000","last_undegraded":"2024-02-23T00:46:35.178182+0000","last_fullsized":"2024-02-23T00:46:35.178182+0000","mapping_epoch":272,"log_start":"110'10","ondisk_log_start":"110'10","created":108,"last_epoch_clean":295,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'2","last_scrub_stamp":"2024-02-23T00:45:10.465108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.465108+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":230,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.163239+0000","last_change":"2024-02-23T00:45:13.874437+0000","last_active":"2024-02-23T00:46:36.163239+0000","last_peered":"2024-02-23T00:46:36.163239+0000","last_clean":"2024-02-23T00:46:36.163239+0000","last_became_active":"2024-02-23T00:42:58.809283+0000","last_became_peered":"2024-02-23T00:42:58.809283+0000","last_unstale":"2024-02-23T00:46:36.163239+0000","last_undegraded":"2024-02-23T00:46:36.163239+0000","last_fullsized":"2024-02-23T00:46:36.163239+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:13.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:13.874373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":283,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.173616+0000","last_change":"2024-02-23T00:44:19.312509+0000","last_active":"2024-02-23T00:46:36.173616+0000","last_peered":"2024-02-23T00:46:36.173616+0000","last_clean":"2024-02-23T00:46:36.173616+0000","last_became_active":"2024-02-23T00:39:18.796664+0000","last_became_peered":"2024-02-23T00:39:18.796664+0000","last_unstale":"2024-02-23T00:46:36.173616+0000","last_undegraded":"2024-02-23T00:46:36.173616+0000","last_fullsized":"2024-02-23T00:46:36.173616+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:19.312418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:19.312418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"203'34","reported_seq":347,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.591558+0000","last_change":"2024-02-23T00:44:35.644033+0000","last_active":"2024-02-23T00:46:36.591558+0000","last_peered":"2024-02-23T00:46:36.591558+0000","last_clean":"2024-02-23T00:46:36.591558+0000","last_became_active":"2024-02-23T00:39:16.857278+0000","last_became_peered":"2024-02-23T00:39:16.857278+0000","last_unstale":"2024-02-23T00:46:36.591558+0000","last_undegraded":"2024-02-23T00:46:36.591558+0000","last_fullsized":"2024-02-23T00:46:36.591558+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'34","last_scrub_stamp":"2024-02-23T00:44:35.643931+0000","last_deep_scrub":"203'34","last_deep_scrub_stamp":"2024-02-23T00:44:35.643931+0000","last_clean_scrub_stamp":"2024-02-23T00:44:35.643931+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":286,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.163964+0000","last_change":"2024-02-23T00:45:12.730663+0000","last_active":"2024-02-23T00:46:36.163964+0000","last_peered":"2024-02-23T00:46:36.163964+0000","last_clean":"2024-02-23T00:46:36.163964+0000","last_became_active":"2024-02-23T00:39:14.783662+0000","last_became_peered":"2024-02-23T00:39:14.783662+0000","last_unstale":"2024-02-23T00:46:36.163964+0000","last_undegraded":"2024-02-23T00:46:36.163964+0000","last_fullsized":"2024-02-23T00:46:36.163964+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.730539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.730539+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":282,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.402151+0000","last_change":"2024-02-23T00:45:04.585266+0000","last_active":"2024-02-23T00:46:36.402151+0000","last_peered":"2024-02-23T00:46:36.402151+0000","last_clean":"2024-02-23T00:46:36.402151+0000","last_became_active":"2024-02-23T00:39:21.160060+0000","last_became_peered":"2024-02-23T00:39:21.160060+0000","last_unstale":"2024-02-23T00:46:36.402151+0000","last_undegraded":"2024-02-23T00:46:36.402151+0000","last_fullsized":"2024-02-23T00:46:36.402151+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:04.585179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.585179+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.8","version":"110'2","reported_seq":249,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.162395+0000","last_change":"2024-02-23T00:46:19.180741+0000","last_active":"2024-02-23T00:46:36.162395+0000","last_peered":"2024-02-23T00:46:36.162395+0000","last_clean":"2024-02-23T00:46:36.162395+0000","last_became_active":"2024-02-23T00:46:19.180391+0000","last_became_peered":"2024-02-23T00:46:19.180391+0000","last_unstale":"2024-02-23T00:46:36.162395+0000","last_undegraded":"2024-02-23T00:46:36.162395+0000","last_fullsized":"2024-02-23T00:46:36.162395+0000","mapping_epoch":108,"log_start":"110'2","ondisk_log_start":"110'2","created":108,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'2","last_scrub_stamp":"2024-02-23T00:44:33.389605+0000","last_deep_scrub":"110'2","last_deep_scrub_stamp":"2024-02-23T00:44:33.389605+0000","last_clean_scrub_stamp":"2024-02-23T00:44:33.389605+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":230,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.298987+0000","last_change":"2024-02-23T00:45:18.185097+0000","last_active":"2024-02-23T00:46:36.298987+0000","last_peered":"2024-02-23T00:46:36.298987+0000","last_clean":"2024-02-23T00:46:36.298987+0000","last_became_active":"2024-02-23T00:42:58.726652+0000","last_became_peered":"2024-02-23T00:42:58.726652+0000","last_unstale":"2024-02-23T00:46:36.298987+0000","last_undegraded":"2024-02-23T00:46:36.298987+0000","last_fullsized":"2024-02-23T00:46:36.298987+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:18.184992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:18.184992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":285,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.300709+0000","last_change":"2024-02-23T00:43:45.054194+0000","last_active":"2024-02-23T00:46:36.300709+0000","last_peered":"2024-02-23T00:46:36.300709+0000","last_clean":"2024-02-23T00:46:36.300709+0000","last_became_active":"2024-02-23T00:39:18.800378+0000","last_became_peered":"2024-02-23T00:39:18.800378+0000","last_unstale":"2024-02-23T00:46:36.300709+0000","last_undegraded":"2024-02-23T00:46:36.300709+0000","last_fullsized":"2024-02-23T00:46:36.300709+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:45.054115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:59.694033+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.054115+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"95'18","reported_seq":311,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.593588+0000","last_change":"2024-02-23T00:44:18.630435+0000","last_active":"2024-02-23T00:46:36.593588+0000","last_peered":"2024-02-23T00:46:36.593588+0000","last_clean":"2024-02-23T00:46:36.593588+0000","last_became_active":"2024-02-23T00:39:16.787143+0000","last_became_peered":"2024-02-23T00:39:16.787143+0000","last_unstale":"2024-02-23T00:46:36.593588+0000","last_undegraded":"2024-02-23T00:46:36.593588+0000","last_fullsized":"2024-02-23T00:46:36.593588+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'18","last_scrub_stamp":"2024-02-23T00:44:18.630363+0000","last_deep_scrub":"95'18","last_deep_scrub_stamp":"2024-02-23T00:42:01.195545+0000","last_clean_scrub_stamp":"2024-02-23T00:44:18.630363+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":291,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.162184+0000","last_change":"2024-02-23T00:45:52.795624+0000","last_active":"2024-02-23T00:46:36.162184+0000","last_peered":"2024-02-23T00:46:36.162184+0000","last_clean":"2024-02-23T00:46:36.162184+0000","last_became_active":"2024-02-23T00:39:14.784578+0000","last_became_peered":"2024-02-23T00:39:14.784578+0000","last_unstale":"2024-02-23T00:46:36.162184+0000","last_undegraded":"2024-02-23T00:46:36.162184+0000","last_fullsized":"2024-02-23T00:46:36.162184+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:52.795527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:52.795527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":289,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.591110+0000","last_change":"2024-02-23T00:43:48.258338+0000","last_active":"2024-02-23T00:46:36.591110+0000","last_peered":"2024-02-23T00:46:36.591110+0000","last_clean":"2024-02-23T00:46:36.591110+0000","last_became_active":"2024-02-23T00:39:14.775446+0000","last_became_peered":"2024-02-23T00:39:14.775446+0000","last_unstale":"2024-02-23T00:46:36.591110+0000","last_undegraded":"2024-02-23T00:46:36.591110+0000","last_fullsized":"2024-02-23T00:46:36.591110+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:48.258269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.258269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":277,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.401137+0000","last_change":"2024-02-23T00:43:47.905617+0000","last_active":"2024-02-23T00:46:36.401137+0000","last_peered":"2024-02-23T00:46:36.401137+0000","last_clean":"2024-02-23T00:46:36.401137+0000","last_became_active":"2024-02-23T00:39:20.797242+0000","last_became_peered":"2024-02-23T00:39:20.797242+0000","last_unstale":"2024-02-23T00:46:36.401137+0000","last_undegraded":"2024-02-23T00:46:36.401137+0000","last_fullsized":"2024-02-23T00:46:36.401137+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:47.905501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:47.905501+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"307'11","reported_seq":89,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.174332+0000","last_change":"2024-02-23T00:45:59.164904+0000","last_active":"2024-02-23T00:46:36.174332+0000","last_peered":"2024-02-23T00:46:36.174332+0000","last_clean":"2024-02-23T00:46:36.174332+0000","last_became_active":"2024-02-23T00:45:59.164483+0000","last_became_peered":"2024-02-23T00:45:59.164483+0000","last_unstale":"2024-02-23T00:46:36.174332+0000","last_undegraded":"2024-02-23T00:46:36.174332+0000","last_fullsized":"2024-02-23T00:46:36.174332+0000","mapping_epoch":272,"log_start":"110'7","ondisk_log_start":"110'7","created":108,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'5","last_scrub_stamp":"2024-02-23T00:45:26.760862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:26.760862+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":229,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.172393+0000","last_change":"2024-02-23T00:44:35.360128+0000","last_active":"2024-02-23T00:46:36.172393+0000","last_peered":"2024-02-23T00:46:36.172393+0000","last_clean":"2024-02-23T00:46:36.172393+0000","last_became_active":"2024-02-23T00:42:58.690858+0000","last_became_peered":"2024-02-23T00:42:58.690858+0000","last_unstale":"2024-02-23T00:46:36.172393+0000","last_undegraded":"2024-02-23T00:46:36.172393+0000","last_fullsized":"2024-02-23T00:46:36.172393+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:35.360054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:35.360054+0000","last_clean_scrub_stamp":"2024-02-23T00:44:35.360054+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":282,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.163724+0000","last_change":"2024-02-23T00:45:16.746109+0000","last_active":"2024-02-23T00:46:36.163724+0000","last_peered":"2024-02-23T00:46:36.163724+0000","last_clean":"2024-02-23T00:46:36.163724+0000","last_became_active":"2024-02-23T00:39:18.793801+0000","last_became_peered":"2024-02-23T00:39:18.793801+0000","last_unstale":"2024-02-23T00:46:36.163724+0000","last_undegraded":"2024-02-23T00:46:36.163724+0000","last_fullsized":"2024-02-23T00:46:36.163724+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.746013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.746013+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"67'14","reported_seq":312,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.172152+0000","last_change":"2024-02-23T00:46:11.980163+0000","last_active":"2024-02-23T00:46:36.172152+0000","last_peered":"2024-02-23T00:46:36.172152+0000","last_clean":"2024-02-23T00:46:36.172152+0000","last_became_active":"2024-02-23T00:39:16.858880+0000","last_became_peered":"2024-02-23T00:39:16.858880+0000","last_unstale":"2024-02-23T00:46:36.172152+0000","last_undegraded":"2024-02-23T00:46:36.172152+0000","last_fullsized":"2024-02-23T00:46:36.172152+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'14","last_scrub_stamp":"2024-02-23T00:46:11.980087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:11.980087+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":296,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.162472+0000","last_change":"2024-02-23T00:45:08.796153+0000","last_active":"2024-02-23T00:46:36.162472+0000","last_peered":"2024-02-23T00:46:36.162472+0000","last_clean":"2024-02-23T00:46:36.162472+0000","last_became_active":"2024-02-23T00:39:08.248249+0000","last_became_peered":"2024-02-23T00:39:08.248249+0000","last_unstale":"2024-02-23T00:46:36.162472+0000","last_undegraded":"2024-02-23T00:46:36.162472+0000","last_fullsized":"2024-02-23T00:46:36.162472+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.796076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:49.977248+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.796076+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":289,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.299884+0000","last_change":"2024-02-23T00:45:22.272078+0000","last_active":"2024-02-23T00:46:36.299884+0000","last_peered":"2024-02-23T00:46:36.299884+0000","last_clean":"2024-02-23T00:46:36.299884+0000","last_became_active":"2024-02-23T00:39:14.788888+0000","last_became_peered":"2024-02-23T00:39:14.788888+0000","last_unstale":"2024-02-23T00:46:36.299884+0000","last_undegraded":"2024-02-23T00:46:36.299884+0000","last_fullsized":"2024-02-23T00:46:36.299884+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:22.272003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:22.272003+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":289,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.159785+0000","last_change":"2024-02-23T00:44:33.143663+0000","last_active":"2024-02-23T00:46:36.159785+0000","last_peered":"2024-02-23T00:46:36.159785+0000","last_clean":"2024-02-23T00:46:36.159785+0000","last_became_active":"2024-02-23T00:39:14.783489+0000","last_became_peered":"2024-02-23T00:39:14.783489+0000","last_unstale":"2024-02-23T00:46:36.159785+0000","last_undegraded":"2024-02-23T00:46:36.159785+0000","last_fullsized":"2024-02-23T00:46:36.159785+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:33.143592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:33.143592+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":301,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.300809+0000","last_change":"2024-02-23T00:44:20.958888+0000","last_active":"2024-02-23T00:46:36.300809+0000","last_peered":"2024-02-23T00:46:36.300809+0000","last_clean":"2024-02-23T00:46:36.300809+0000","last_became_active":"2024-02-23T00:43:42.168435+0000","last_became_peered":"2024-02-23T00:43:42.168435+0000","last_unstale":"2024-02-23T00:46:36.300809+0000","last_undegraded":"2024-02-23T00:46:36.300809+0000","last_fullsized":"2024-02-23T00:46:36.300809+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:20.958797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:20.958797+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":80,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.593374+0000","last_change":"2024-02-23T00:45:55.156529+0000","last_active":"2024-02-23T00:46:36.593374+0000","last_peered":"2024-02-23T00:46:36.593374+0000","last_clean":"2024-02-23T00:46:36.593374+0000","last_became_active":"2024-02-23T00:45:55.156026+0000","last_became_peered":"2024-02-23T00:45:55.156026+0000","last_unstale":"2024-02-23T00:46:36.593374+0000","last_undegraded":"2024-02-23T00:46:36.593374+0000","last_fullsized":"2024-02-23T00:46:36.593374+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:21.383626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:21.383626+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":229,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.162477+0000","last_change":"2024-02-23T00:45:17.763618+0000","last_active":"2024-02-23T00:46:36.162477+0000","last_peered":"2024-02-23T00:46:36.162477+0000","last_clean":"2024-02-23T00:46:36.162477+0000","last_became_active":"2024-02-23T00:42:58.701502+0000","last_became_peered":"2024-02-23T00:42:58.701502+0000","last_unstale":"2024-02-23T00:46:36.162477+0000","last_undegraded":"2024-02-23T00:46:36.162477+0000","last_fullsized":"2024-02-23T00:46:36.162477+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:17.763482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:17.763482+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":288,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.162941+0000","last_change":"2024-02-23T00:46:09.924457+0000","last_active":"2024-02-23T00:46:36.162941+0000","last_peered":"2024-02-23T00:46:36.162941+0000","last_clean":"2024-02-23T00:46:36.162941+0000","last_became_active":"2024-02-23T00:39:18.789330+0000","last_became_peered":"2024-02-23T00:39:18.789330+0000","last_unstale":"2024-02-23T00:46:36.162941+0000","last_undegraded":"2024-02-23T00:46:36.162941+0000","last_fullsized":"2024-02-23T00:46:36.162941+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:09.924384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.264240+0000","last_clean_scrub_stamp":"2024-02-23T00:46:09.924384+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"68'19","reported_seq":328,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.402261+0000","last_change":"2024-02-23T00:45:07.697475+0000","last_active":"2024-02-23T00:46:36.402261+0000","last_peered":"2024-02-23T00:46:36.402261+0000","last_clean":"2024-02-23T00:46:36.402261+0000","last_became_active":"2024-02-23T00:39:16.787484+0000","last_became_peered":"2024-02-23T00:39:16.787484+0000","last_unstale":"2024-02-23T00:46:36.402261+0000","last_undegraded":"2024-02-23T00:46:36.402261+0000","last_fullsized":"2024-02-23T00:46:36.402261+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'19","last_scrub_stamp":"2024-02-23T00:45:07.697394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:07.697394+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":292,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.160565+0000","last_change":"2024-02-23T00:46:10.878272+0000","last_active":"2024-02-23T00:46:36.160565+0000","last_peered":"2024-02-23T00:46:36.160565+0000","last_clean":"2024-02-23T00:46:36.160565+0000","last_became_active":"2024-02-23T00:39:14.772411+0000","last_became_peered":"2024-02-23T00:39:14.772411+0000","last_unstale":"2024-02-23T00:46:36.160565+0000","last_undegraded":"2024-02-23T00:46:36.160565+0000","last_fullsized":"2024-02-23T00:46:36.160565+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:10.878204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:10.878204+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"68'5","reported_seq":720,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.769209+0000","last_change":"2024-02-23T00:43:48.876428+0000","last_active":"2024-02-23T00:46:36.769209+0000","last_peered":"2024-02-23T00:46:36.769209+0000","last_clean":"2024-02-23T00:46:36.769209+0000","last_became_active":"2024-02-23T00:39:14.779109+0000","last_became_peered":"2024-02-23T00:39:14.779109+0000","last_unstale":"2024-02-23T00:46:36.769209+0000","last_undegraded":"2024-02-23T00:46:36.769209+0000","last_fullsized":"2024-02-23T00:46:36.769209+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'5","last_scrub_stamp":"2024-02-23T00:43:48.876356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.876356+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":434,"num_read_kb":429,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":285,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.725831+0000","last_change":"2024-02-23T00:46:36.725799+0000","last_active":"2024-02-23T00:46:36.725831+0000","last_peered":"2024-02-23T00:46:36.725831+0000","last_clean":"2024-02-23T00:46:36.725831+0000","last_became_active":"2024-02-23T00:39:21.160457+0000","last_became_peered":"2024-02-23T00:39:21.160457+0000","last_unstale":"2024-02-23T00:46:36.725831+0000","last_undegraded":"2024-02-23T00:46:36.725831+0000","last_fullsized":"2024-02-23T00:46:36.725831+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:36.725686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:46:36.725686+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":80,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.400447+0000","last_change":"2024-02-23T00:46:03.662711+0000","last_active":"2024-02-23T00:46:36.400447+0000","last_peered":"2024-02-23T00:46:36.400447+0000","last_clean":"2024-02-23T00:46:36.400447+0000","last_became_active":"2024-02-23T00:46:03.662395+0000","last_became_peered":"2024-02-23T00:46:03.662395+0000","last_unstale":"2024-02-23T00:46:36.400447+0000","last_undegraded":"2024-02-23T00:46:36.400447+0000","last_fullsized":"2024-02-23T00:46:36.400447+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,3],"acting":[1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":225,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.400716+0000","last_change":"2024-02-23T00:42:57.653669+0000","last_active":"2024-02-23T00:46:36.400716+0000","last_peered":"2024-02-23T00:46:36.400716+0000","last_clean":"2024-02-23T00:46:36.400716+0000","last_became_active":"2024-02-23T00:42:57.653355+0000","last_became_peered":"2024-02-23T00:42:57.653355+0000","last_unstale":"2024-02-23T00:46:36.400716+0000","last_undegraded":"2024-02-23T00:46:36.400716+0000","last_fullsized":"2024-02-23T00:46:36.400716+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":285,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.591518+0000","last_change":"2024-02-23T00:44:20.626116+0000","last_active":"2024-02-23T00:46:36.591518+0000","last_peered":"2024-02-23T00:46:36.591518+0000","last_clean":"2024-02-23T00:46:36.591518+0000","last_became_active":"2024-02-23T00:39:18.802134+0000","last_became_peered":"2024-02-23T00:39:18.802134+0000","last_unstale":"2024-02-23T00:46:36.591518+0000","last_undegraded":"2024-02-23T00:46:36.591518+0000","last_fullsized":"2024-02-23T00:46:36.591518+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:20.626052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:20.626052+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"86'35","reported_seq":359,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.298575+0000","last_change":"2024-02-23T00:45:52.939553+0000","last_active":"2024-02-23T00:46:36.298575+0000","last_peered":"2024-02-23T00:46:36.298575+0000","last_clean":"2024-02-23T00:46:36.298575+0000","last_became_active":"2024-02-23T00:40:42.933322+0000","last_became_peered":"2024-02-23T00:40:42.933322+0000","last_unstale":"2024-02-23T00:46:36.298575+0000","last_undegraded":"2024-02-23T00:46:36.298575+0000","last_fullsized":"2024-02-23T00:46:36.298575+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'35","last_scrub_stamp":"2024-02-23T00:45:52.939452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:52.939452+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":289,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.299077+0000","last_change":"2024-02-23T00:44:23.965228+0000","last_active":"2024-02-23T00:46:36.299077+0000","last_peered":"2024-02-23T00:46:36.299077+0000","last_clean":"2024-02-23T00:46:36.299077+0000","last_became_active":"2024-02-23T00:39:14.784015+0000","last_became_peered":"2024-02-23T00:39:14.784015+0000","last_unstale":"2024-02-23T00:46:36.299077+0000","last_undegraded":"2024-02-23T00:46:36.299077+0000","last_fullsized":"2024-02-23T00:46:36.299077+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:23.965125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.965125+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":296,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.162523+0000","last_change":"2024-02-23T00:43:45.144499+0000","last_active":"2024-02-23T00:46:36.162523+0000","last_peered":"2024-02-23T00:46:36.162523+0000","last_clean":"2024-02-23T00:46:36.162523+0000","last_became_active":"2024-02-23T00:43:45.144043+0000","last_became_peered":"2024-02-23T00:43:45.144043+0000","last_unstale":"2024-02-23T00:46:36.162523+0000","last_undegraded":"2024-02-23T00:46:36.162523+0000","last_fullsized":"2024-02-23T00:46:36.162523+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:40.062493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:40.062493+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.f","version":"307'8","reported_seq":82,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.402332+0000","last_change":"2024-02-23T00:45:51.167662+0000","last_active":"2024-02-23T00:46:36.402332+0000","last_peered":"2024-02-23T00:46:36.402332+0000","last_clean":"2024-02-23T00:46:36.402332+0000","last_became_active":"2024-02-23T00:45:51.167136+0000","last_became_peered":"2024-02-23T00:45:51.167136+0000","last_unstale":"2024-02-23T00:46:36.402332+0000","last_undegraded":"2024-02-23T00:46:36.402332+0000","last_fullsized":"2024-02-23T00:46:36.402332+0000","mapping_epoch":272,"log_start":"254'6","ondisk_log_start":"254'6","created":108,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:31.410670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:31.410670+0000","last_clean_scrub_stamp":"2024-02-23T00:44:31.410670+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":229,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.162522+0000","last_change":"2024-02-23T00:45:05.490595+0000","last_active":"2024-02-23T00:46:36.162522+0000","last_peered":"2024-02-23T00:46:36.162522+0000","last_clean":"2024-02-23T00:46:36.162522+0000","last_became_active":"2024-02-23T00:42:58.766579+0000","last_became_peered":"2024-02-23T00:42:58.766579+0000","last_unstale":"2024-02-23T00:46:36.162522+0000","last_undegraded":"2024-02-23T00:46:36.162522+0000","last_fullsized":"2024-02-23T00:46:36.162522+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:05.490531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:05.490531+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"91'9","reported_seq":382,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.591756+0000","last_change":"2024-02-23T00:44:28.619036+0000","last_active":"2024-02-23T00:46:36.591756+0000","last_peered":"2024-02-23T00:46:36.591756+0000","last_clean":"2024-02-23T00:46:36.591756+0000","last_became_active":"2024-02-23T00:39:18.801297+0000","last_became_peered":"2024-02-23T00:39:18.801297+0000","last_unstale":"2024-02-23T00:46:36.591756+0000","last_undegraded":"2024-02-23T00:46:36.591756+0000","last_fullsized":"2024-02-23T00:46:36.591756+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'9","last_scrub_stamp":"2024-02-23T00:44:28.618923+0000","last_deep_scrub":"91'9","last_deep_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_clean_scrub_stamp":"2024-02-23T00:44:28.618923+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"68'16","reported_seq":316,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.400469+0000","last_change":"2024-02-23T00:45:11.713800+0000","last_active":"2024-02-23T00:46:36.400469+0000","last_peered":"2024-02-23T00:46:36.400469+0000","last_clean":"2024-02-23T00:46:36.400469+0000","last_became_active":"2024-02-23T00:39:16.787468+0000","last_became_peered":"2024-02-23T00:39:16.787468+0000","last_unstale":"2024-02-23T00:46:36.400469+0000","last_undegraded":"2024-02-23T00:46:36.400469+0000","last_fullsized":"2024-02-23T00:46:36.400469+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:45:11.713731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:11.713731+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":288,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.400794+0000","last_change":"2024-02-23T00:45:19.635062+0000","last_active":"2024-02-23T00:46:36.400794+0000","last_peered":"2024-02-23T00:46:36.400794+0000","last_clean":"2024-02-23T00:46:36.400794+0000","last_became_active":"2024-02-23T00:39:14.779952+0000","last_became_peered":"2024-02-23T00:39:14.779952+0000","last_unstale":"2024-02-23T00:46:36.400794+0000","last_undegraded":"2024-02-23T00:46:36.400794+0000","last_fullsized":"2024-02-23T00:46:36.400794+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:19.635002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:19.635002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"107'2","reported_seq":285,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.590818+0000","last_change":"2024-02-23T00:45:08.662754+0000","last_active":"2024-02-23T00:46:36.590818+0000","last_peered":"2024-02-23T00:46:36.590818+0000","last_clean":"2024-02-23T00:46:36.590818+0000","last_became_active":"2024-02-23T00:39:20.800858+0000","last_became_peered":"2024-02-23T00:39:20.800858+0000","last_unstale":"2024-02-23T00:46:36.590818+0000","last_undegraded":"2024-02-23T00:46:36.590818+0000","last_fullsized":"2024-02-23T00:46:36.590818+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'2","last_scrub_stamp":"2024-02-23T00:45:08.662667+0000","last_deep_scrub":"107'2","last_deep_scrub_stamp":"2024-02-23T00:42:48.362739+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.662667+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":247,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.171876+0000","last_change":"2024-02-23T00:46:15.806517+0000","last_active":"2024-02-23T00:46:36.171876+0000","last_peered":"2024-02-23T00:46:36.171876+0000","last_clean":"2024-02-23T00:46:36.171876+0000","last_became_active":"2024-02-23T00:46:15.806146+0000","last_became_peered":"2024-02-23T00:46:15.806146+0000","last_unstale":"2024-02-23T00:46:36.171876+0000","last_undegraded":"2024-02-23T00:46:36.171876+0000","last_fullsized":"2024-02-23T00:46:36.171876+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:19.711117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:19.711117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":229,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.172469+0000","last_change":"2024-02-23T00:45:04.621794+0000","last_active":"2024-02-23T00:46:36.172469+0000","last_peered":"2024-02-23T00:46:36.172469+0000","last_clean":"2024-02-23T00:46:36.172469+0000","last_became_active":"2024-02-23T00:42:58.664282+0000","last_became_peered":"2024-02-23T00:42:58.664282+0000","last_unstale":"2024-02-23T00:46:36.172469+0000","last_undegraded":"2024-02-23T00:46:36.172469+0000","last_fullsized":"2024-02-23T00:46:36.172469+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:04.621714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.621714+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":288,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.783914+0000","last_change":"2024-02-23T00:46:36.783887+0000","last_active":"2024-02-23T00:46:36.783914+0000","last_peered":"2024-02-23T00:46:36.783914+0000","last_clean":"2024-02-23T00:46:36.783914+0000","last_became_active":"2024-02-23T00:39:18.796323+0000","last_became_peered":"2024-02-23T00:39:18.796323+0000","last_unstale":"2024-02-23T00:46:36.783914+0000","last_undegraded":"2024-02-23T00:46:36.783914+0000","last_fullsized":"2024-02-23T00:46:36.783914+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:36.783823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_clean_scrub_stamp":"2024-02-23T00:46:36.783823+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"68'25","reported_seq":335,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.167447+0000","last_change":"2024-02-23T00:43:46.158231+0000","last_active":"2024-02-23T00:46:36.167447+0000","last_peered":"2024-02-23T00:46:36.167447+0000","last_clean":"2024-02-23T00:46:36.167447+0000","last_became_active":"2024-02-23T00:39:16.783926+0000","last_became_peered":"2024-02-23T00:39:16.783926+0000","last_unstale":"2024-02-23T00:46:36.167447+0000","last_undegraded":"2024-02-23T00:46:36.167447+0000","last_fullsized":"2024-02-23T00:46:36.167447+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'25","last_scrub_stamp":"2024-02-23T00:43:46.158149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:46.158149+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"95'4","reported_seq":302,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-02-23T00:46:35.177998+0000","last_change":"2024-02-23T00:44:31.106105+0000","last_active":"2024-02-23T00:46:35.177998+0000","last_peered":"2024-02-23T00:46:35.177998+0000","last_clean":"2024-02-23T00:46:35.177998+0000","last_became_active":"2024-02-23T00:39:14.779205+0000","last_became_peered":"2024-02-23T00:39:14.779205+0000","last_unstale":"2024-02-23T00:46:35.177998+0000","last_undegraded":"2024-02-23T00:46:35.177998+0000","last_fullsized":"2024-02-23T00:46:35.177998+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'4","last_scrub_stamp":"2024-02-23T00:44:31.106012+0000","last_deep_scrub":"95'4","last_deep_scrub_stamp":"2024-02-23T00:41:43.874397+0000","last_clean_scrub_stamp":"2024-02-23T00:44:31.106012+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":301,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.163609+0000","last_change":"2024-02-23T00:43:52.036315+0000","last_active":"2024-02-23T00:46:36.163609+0000","last_peered":"2024-02-23T00:46:36.163609+0000","last_clean":"2024-02-23T00:46:36.163609+0000","last_became_active":"2024-02-23T00:43:27.611660+0000","last_became_peered":"2024-02-23T00:43:27.611660+0000","last_unstale":"2024-02-23T00:46:36.163609+0000","last_undegraded":"2024-02-23T00:46:36.163609+0000","last_fullsized":"2024-02-23T00:46:36.163609+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:52.036249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:52.036249+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":229,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.162607+0000","last_change":"2024-02-23T00:45:16.389060+0000","last_active":"2024-02-23T00:46:36.162607+0000","last_peered":"2024-02-23T00:46:36.162607+0000","last_clean":"2024-02-23T00:46:36.162607+0000","last_became_active":"2024-02-23T00:42:58.809926+0000","last_became_peered":"2024-02-23T00:42:58.809926+0000","last_unstale":"2024-02-23T00:46:36.162607+0000","last_undegraded":"2024-02-23T00:46:36.162607+0000","last_fullsized":"2024-02-23T00:46:36.162607+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.388988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.388988+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":229,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.163419+0000","last_change":"2024-02-23T00:45:20.829124+0000","last_active":"2024-02-23T00:46:36.163419+0000","last_peered":"2024-02-23T00:46:36.163419+0000","last_clean":"2024-02-23T00:46:36.163419+0000","last_became_active":"2024-02-23T00:42:58.744983+0000","last_became_peered":"2024-02-23T00:42:58.744983+0000","last_unstale":"2024-02-23T00:46:36.163419+0000","last_undegraded":"2024-02-23T00:46:36.163419+0000","last_fullsized":"2024-02-23T00:46:36.163419+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:20.829052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:20.829052+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"68'21","reported_seq":328,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.593642+0000","last_change":"2024-02-23T00:43:52.309599+0000","last_active":"2024-02-23T00:46:36.593642+0000","last_peered":"2024-02-23T00:46:36.593642+0000","last_clean":"2024-02-23T00:46:36.593642+0000","last_became_active":"2024-02-23T00:39:16.858760+0000","last_became_peered":"2024-02-23T00:39:16.858760+0000","last_unstale":"2024-02-23T00:46:36.593642+0000","last_undegraded":"2024-02-23T00:46:36.593642+0000","last_fullsized":"2024-02-23T00:46:36.593642+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'21","last_scrub_stamp":"2024-02-23T00:43:52.309532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:52.309532+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":292,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.162559+0000","last_change":"2024-02-23T00:45:22.376128+0000","last_active":"2024-02-23T00:46:36.162559+0000","last_peered":"2024-02-23T00:46:36.162559+0000","last_clean":"2024-02-23T00:46:36.162559+0000","last_became_active":"2024-02-23T00:39:14.774244+0000","last_became_peered":"2024-02-23T00:39:14.774244+0000","last_unstale":"2024-02-23T00:46:36.162559+0000","last_undegraded":"2024-02-23T00:46:36.162559+0000","last_fullsized":"2024-02-23T00:46:36.162559+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:22.376063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:22.376063+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":285,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.162997+0000","last_change":"2024-02-23T00:43:44.951800+0000","last_active":"2024-02-23T00:46:36.162997+0000","last_peered":"2024-02-23T00:46:36.162997+0000","last_clean":"2024-02-23T00:46:36.162997+0000","last_became_active":"2024-02-23T00:39:18.798624+0000","last_became_peered":"2024-02-23T00:39:18.798624+0000","last_unstale":"2024-02-23T00:46:36.162997+0000","last_undegraded":"2024-02-23T00:46:36.162997+0000","last_fullsized":"2024-02-23T00:46:36.162997+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:44.951696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:46.998875+0000","last_clean_scrub_stamp":"2024-02-23T00:43:44.951696+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":281,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.164896+0000","last_change":"2024-02-23T00:44:26.432766+0000","last_active":"2024-02-23T00:46:36.164896+0000","last_peered":"2024-02-23T00:46:36.164896+0000","last_clean":"2024-02-23T00:46:36.164896+0000","last_became_active":"2024-02-23T00:39:20.796041+0000","last_became_peered":"2024-02-23T00:39:20.796041+0000","last_unstale":"2024-02-23T00:46:36.164896+0000","last_undegraded":"2024-02-23T00:46:36.164896+0000","last_fullsized":"2024-02-23T00:46:36.164896+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:26.432703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:26.432703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":230,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.590990+0000","last_change":"2024-02-23T00:44:26.595443+0000","last_active":"2024-02-23T00:46:36.590990+0000","last_peered":"2024-02-23T00:46:36.590990+0000","last_clean":"2024-02-23T00:46:36.590990+0000","last_became_active":"2024-02-23T00:42:58.825015+0000","last_became_peered":"2024-02-23T00:42:58.825015+0000","last_unstale":"2024-02-23T00:46:36.590990+0000","last_undegraded":"2024-02-23T00:46:36.590990+0000","last_fullsized":"2024-02-23T00:46:36.590990+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:26.595346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:44:26.595346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"107'16","reported_seq":315,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.162911+0000","last_change":"2024-02-23T00:45:07.012086+0000","last_active":"2024-02-23T00:46:36.162911+0000","last_peered":"2024-02-23T00:46:36.162911+0000","last_clean":"2024-02-23T00:46:36.162911+0000","last_became_active":"2024-02-23T00:39:16.858449+0000","last_became_peered":"2024-02-23T00:39:16.858449+0000","last_unstale":"2024-02-23T00:46:36.162911+0000","last_undegraded":"2024-02-23T00:46:36.162911+0000","last_fullsized":"2024-02-23T00:46:36.162911+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'16","last_scrub_stamp":"2024-02-23T00:45:07.011956+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-02-23T00:42:52.964510+0000","last_clean_scrub_stamp":"2024-02-23T00:45:07.011956+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":287,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.171165+0000","last_change":"2024-02-23T00:45:13.613736+0000","last_active":"2024-02-23T00:46:36.171165+0000","last_peered":"2024-02-23T00:46:36.171165+0000","last_clean":"2024-02-23T00:46:36.171165+0000","last_became_active":"2024-02-23T00:39:14.783164+0000","last_became_peered":"2024-02-23T00:39:14.783164+0000","last_unstale":"2024-02-23T00:46:36.171165+0000","last_undegraded":"2024-02-23T00:46:36.171165+0000","last_fullsized":"2024-02-23T00:46:36.171165+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:13.613654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:13.613654+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":285,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.300248+0000","last_change":"2024-02-23T00:43:40.971316+0000","last_active":"2024-02-23T00:46:36.300248+0000","last_peered":"2024-02-23T00:46:36.300248+0000","last_clean":"2024-02-23T00:46:36.300248+0000","last_became_active":"2024-02-23T00:39:18.800297+0000","last_became_peered":"2024-02-23T00:39:18.800297+0000","last_unstale":"2024-02-23T00:46:36.300248+0000","last_undegraded":"2024-02-23T00:46:36.300248+0000","last_fullsized":"2024-02-23T00:46:36.300248+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:40.971243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:40.971243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":285,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.163809+0000","last_change":"2024-02-23T00:46:34.721645+0000","last_active":"2024-02-23T00:46:36.163809+0000","last_peered":"2024-02-23T00:46:36.163809+0000","last_clean":"2024-02-23T00:46:36.163809+0000","last_became_active":"2024-02-23T00:39:21.162036+0000","last_became_peered":"2024-02-23T00:39:21.162036+0000","last_unstale":"2024-02-23T00:46:36.163809+0000","last_undegraded":"2024-02-23T00:46:36.163809+0000","last_fullsized":"2024-02-23T00:46:36.163809+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:34.721564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:46:34.721564+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"110'7","reported_seq":243,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.159346+0000","last_change":"2024-02-23T00:45:18.947859+0000","last_active":"2024-02-23T00:46:36.159346+0000","last_peered":"2024-02-23T00:46:36.159346+0000","last_clean":"2024-02-23T00:46:36.159346+0000","last_became_active":"2024-02-23T00:42:58.777251+0000","last_became_peered":"2024-02-23T00:42:58.777251+0000","last_unstale":"2024-02-23T00:46:36.159346+0000","last_undegraded":"2024-02-23T00:46:36.159346+0000","last_fullsized":"2024-02-23T00:46:36.159346+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'7","last_scrub_stamp":"2024-02-23T00:45:18.947746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:18.947746+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"68'22","reported_seq":255,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.164158+0000","last_change":"2024-02-23T00:44:31.391929+0000","last_active":"2024-02-23T00:46:36.164158+0000","last_peered":"2024-02-23T00:46:36.164158+0000","last_clean":"2024-02-23T00:46:36.164158+0000","last_became_active":"2024-02-23T00:42:43.521189+0000","last_became_peered":"2024-02-23T00:42:43.521189+0000","last_unstale":"2024-02-23T00:46:36.164158+0000","last_undegraded":"2024-02-23T00:46:36.164158+0000","last_fullsized":"2024-02-23T00:46:36.164158+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'22","last_scrub_stamp":"2024-02-23T00:44:31.391819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:44:31.391819+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":37,"num_read_kb":30,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,7],"acting":[2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":289,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.298980+0000","last_change":"2024-02-23T00:44:18.946624+0000","last_active":"2024-02-23T00:46:36.298980+0000","last_peered":"2024-02-23T00:46:36.298980+0000","last_clean":"2024-02-23T00:46:36.298980+0000","last_became_active":"2024-02-23T00:39:14.785590+0000","last_became_peered":"2024-02-23T00:39:14.785590+0000","last_unstale":"2024-02-23T00:46:36.298980+0000","last_undegraded":"2024-02-23T00:46:36.298980+0000","last_fullsized":"2024-02-23T00:46:36.298980+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:18.946560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_clean_scrub_stamp":"2024-02-23T00:44:18.946560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"67'8","reported_seq":381,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.299575+0000","last_change":"2024-02-23T00:44:29.038226+0000","last_active":"2024-02-23T00:46:36.299575+0000","last_peered":"2024-02-23T00:46:36.299575+0000","last_clean":"2024-02-23T00:46:36.299575+0000","last_became_active":"2024-02-23T00:39:18.803566+0000","last_became_peered":"2024-02-23T00:39:18.803566+0000","last_unstale":"2024-02-23T00:46:36.299575+0000","last_undegraded":"2024-02-23T00:46:36.299575+0000","last_fullsized":"2024-02-23T00:46:36.299575+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:44:29.038134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:29.038134+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":229,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.163372+0000","last_change":"2024-02-23T00:45:54.753371+0000","last_active":"2024-02-23T00:46:36.163372+0000","last_peered":"2024-02-23T00:46:36.163372+0000","last_clean":"2024-02-23T00:46:36.163372+0000","last_became_active":"2024-02-23T00:42:58.757424+0000","last_became_peered":"2024-02-23T00:42:58.757424+0000","last_unstale":"2024-02-23T00:46:36.163372+0000","last_undegraded":"2024-02-23T00:46:36.163372+0000","last_fullsized":"2024-02-23T00:46:36.163372+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:54.753267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:54.753267+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"68'15","reported_seq":318,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.299403+0000","last_change":"2024-02-23T00:44:25.998879+0000","last_active":"2024-02-23T00:46:36.299403+0000","last_peered":"2024-02-23T00:46:36.299403+0000","last_clean":"2024-02-23T00:46:36.299403+0000","last_became_active":"2024-02-23T00:39:16.858953+0000","last_became_peered":"2024-02-23T00:39:16.858953+0000","last_unstale":"2024-02-23T00:46:36.299403+0000","last_undegraded":"2024-02-23T00:46:36.299403+0000","last_fullsized":"2024-02-23T00:46:36.299403+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:44:25.998804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:44:25.998804+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":289,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.591630+0000","last_change":"2024-02-23T00:43:45.309194+0000","last_active":"2024-02-23T00:46:36.591630+0000","last_peered":"2024-02-23T00:46:36.591630+0000","last_clean":"2024-02-23T00:46:36.591630+0000","last_became_active":"2024-02-23T00:39:14.773772+0000","last_became_peered":"2024-02-23T00:39:14.773772+0000","last_unstale":"2024-02-23T00:46:36.591630+0000","last_undegraded":"2024-02-23T00:46:36.591630+0000","last_fullsized":"2024-02-23T00:46:36.591630+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:45.309083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.309083+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"67'8","reported_seq":381,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.901207+0000","last_change":"2024-02-23T00:46:36.901142+0000","last_active":"2024-02-23T00:46:36.901207+0000","last_peered":"2024-02-23T00:46:36.901207+0000","last_clean":"2024-02-23T00:46:36.901207+0000","last_became_active":"2024-02-23T00:39:18.787313+0000","last_became_peered":"2024-02-23T00:39:18.787313+0000","last_unstale":"2024-02-23T00:46:36.901207+0000","last_undegraded":"2024-02-23T00:46:36.901207+0000","last_fullsized":"2024-02-23T00:46:36.901207+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:46:36.900988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:36.900988+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":225,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.400842+0000","last_change":"2024-02-23T00:42:57.652853+0000","last_active":"2024-02-23T00:46:36.400842+0000","last_peered":"2024-02-23T00:46:36.400842+0000","last_clean":"2024-02-23T00:46:36.400842+0000","last_became_active":"2024-02-23T00:42:57.652488+0000","last_became_peered":"2024-02-23T00:42:57.652488+0000","last_unstale":"2024-02-23T00:46:36.400842+0000","last_undegraded":"2024-02-23T00:46:36.400842+0000","last_fullsized":"2024-02-23T00:46:36.400842+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"67'9","reported_seq":302,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.406888+0000","last_change":"2024-02-23T00:46:36.406841+0000","last_active":"2024-02-23T00:46:36.406888+0000","last_peered":"2024-02-23T00:46:36.406888+0000","last_clean":"2024-02-23T00:46:36.406888+0000","last_became_active":"2024-02-23T00:39:16.786936+0000","last_became_peered":"2024-02-23T00:39:16.786936+0000","last_unstale":"2024-02-23T00:46:36.406888+0000","last_undegraded":"2024-02-23T00:46:36.406888+0000","last_fullsized":"2024-02-23T00:46:36.406888+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'9","last_scrub_stamp":"2024-02-23T00:46:36.406771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:36.406771+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":292,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.171873+0000","last_change":"2024-02-23T00:44:24.350861+0000","last_active":"2024-02-23T00:46:36.171873+0000","last_peered":"2024-02-23T00:46:36.171873+0000","last_clean":"2024-02-23T00:46:36.171873+0000","last_became_active":"2024-02-23T00:39:14.778455+0000","last_became_peered":"2024-02-23T00:39:14.778455+0000","last_unstale":"2024-02-23T00:46:36.171873+0000","last_undegraded":"2024-02-23T00:46:36.171873+0000","last_fullsized":"2024-02-23T00:46:36.171873+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:24.350797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:24.350797+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":285,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.163241+0000","last_change":"2024-02-23T00:44:37.105079+0000","last_active":"2024-02-23T00:46:36.163241+0000","last_peered":"2024-02-23T00:46:36.163241+0000","last_clean":"2024-02-23T00:46:36.163241+0000","last_became_active":"2024-02-23T00:39:18.799052+0000","last_became_peered":"2024-02-23T00:39:18.799052+0000","last_unstale":"2024-02-23T00:46:36.163241+0000","last_undegraded":"2024-02-23T00:46:36.163241+0000","last_fullsized":"2024-02-23T00:46:36.163241+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:37.105013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_clean_scrub_stamp":"2024-02-23T00:44:37.105013+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":283,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.590682+0000","last_change":"2024-02-23T00:46:00.558774+0000","last_active":"2024-02-23T00:46:36.590682+0000","last_peered":"2024-02-23T00:46:36.590682+0000","last_clean":"2024-02-23T00:46:36.590682+0000","last_became_active":"2024-02-23T00:39:20.802648+0000","last_became_peered":"2024-02-23T00:39:20.802648+0000","last_unstale":"2024-02-23T00:46:36.590682+0000","last_undegraded":"2024-02-23T00:46:36.590682+0000","last_fullsized":"2024-02-23T00:46:36.590682+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:00.558661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:46:00.558661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":230,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.592481+0000","last_change":"2024-02-23T00:44:23.543049+0000","last_active":"2024-02-23T00:46:36.592481+0000","last_peered":"2024-02-23T00:46:36.592481+0000","last_clean":"2024-02-23T00:46:36.592481+0000","last_became_active":"2024-02-23T00:42:58.835862+0000","last_became_peered":"2024-02-23T00:42:58.835862+0000","last_unstale":"2024-02-23T00:46:36.592481+0000","last_undegraded":"2024-02-23T00:46:36.592481+0000","last_fullsized":"2024-02-23T00:46:36.592481+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:23.542975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.542975+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"67'11","reported_seq":304,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-02-23T00:46:35.177794+0000","last_change":"2024-02-23T00:45:13.315510+0000","last_active":"2024-02-23T00:46:35.177794+0000","last_peered":"2024-02-23T00:46:35.177794+0000","last_clean":"2024-02-23T00:46:35.177794+0000","last_became_active":"2024-02-23T00:39:16.783896+0000","last_became_peered":"2024-02-23T00:39:16.783896+0000","last_unstale":"2024-02-23T00:46:35.177794+0000","last_undegraded":"2024-02-23T00:46:35.177794+0000","last_fullsized":"2024-02-23T00:46:35.177794+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'11","last_scrub_stamp":"2024-02-23T00:45:13.315434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:13.315434+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"202'2","reported_seq":296,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.403817+0000","last_change":"2024-02-23T00:44:34.720705+0000","last_active":"2024-02-23T00:46:36.403817+0000","last_peered":"2024-02-23T00:46:36.403817+0000","last_clean":"2024-02-23T00:46:36.403817+0000","last_became_active":"2024-02-23T00:39:14.775804+0000","last_became_peered":"2024-02-23T00:39:14.775804+0000","last_unstale":"2024-02-23T00:46:36.403817+0000","last_undegraded":"2024-02-23T00:46:36.403817+0000","last_fullsized":"2024-02-23T00:46:36.403817+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"202'2","last_scrub_stamp":"2024-02-23T00:44:34.720640+0000","last_deep_scrub":"202'2","last_deep_scrub_stamp":"2024-02-23T00:44:34.720640+0000","last_clean_scrub_stamp":"2024-02-23T00:44:34.720640+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":279,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.400379+0000","last_change":"2024-02-23T00:43:31.903157+0000","last_active":"2024-02-23T00:46:36.400379+0000","last_peered":"2024-02-23T00:46:36.400379+0000","last_clean":"2024-02-23T00:46:36.400379+0000","last_became_active":"2024-02-23T00:39:18.800382+0000","last_became_peered":"2024-02-23T00:39:18.800382+0000","last_unstale":"2024-02-23T00:46:36.400379+0000","last_undegraded":"2024-02-23T00:46:36.400379+0000","last_fullsized":"2024-02-23T00:46:36.400379+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:31.903031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:31.903031+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":283,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.158854+0000","last_change":"2024-02-23T00:44:23.152677+0000","last_active":"2024-02-23T00:46:36.158854+0000","last_peered":"2024-02-23T00:46:36.158854+0000","last_clean":"2024-02-23T00:46:36.158854+0000","last_became_active":"2024-02-23T00:39:20.799053+0000","last_became_peered":"2024-02-23T00:39:20.799053+0000","last_unstale":"2024-02-23T00:46:36.158854+0000","last_undegraded":"2024-02-23T00:46:36.158854+0000","last_fullsized":"2024-02-23T00:46:36.158854+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:23.152602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.152602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":225,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.403773+0000","last_change":"2024-02-23T00:42:57.652742+0000","last_active":"2024-02-23T00:46:36.403773+0000","last_peered":"2024-02-23T00:46:36.403773+0000","last_clean":"2024-02-23T00:46:36.403773+0000","last_became_active":"2024-02-23T00:42:57.652438+0000","last_became_peered":"2024-02-23T00:42:57.652438+0000","last_unstale":"2024-02-23T00:46:36.403773+0000","last_undegraded":"2024-02-23T00:46:36.403773+0000","last_fullsized":"2024-02-23T00:46:36.403773+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"67'4","reported_seq":293,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.158677+0000","last_change":"2024-02-23T00:45:20.967211+0000","last_active":"2024-02-23T00:46:36.158677+0000","last_peered":"2024-02-23T00:46:36.158677+0000","last_clean":"2024-02-23T00:46:36.158677+0000","last_became_active":"2024-02-23T00:39:16.785830+0000","last_became_peered":"2024-02-23T00:39:16.785830+0000","last_unstale":"2024-02-23T00:46:36.158677+0000","last_undegraded":"2024-02-23T00:46:36.158677+0000","last_fullsized":"2024-02-23T00:46:36.158677+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'4","last_scrub_stamp":"2024-02-23T00:45:20.967095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:20.967095+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":297,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.299770+0000","last_change":"2024-02-23T00:46:35.268544+0000","last_active":"2024-02-23T00:46:36.299770+0000","last_peered":"2024-02-23T00:46:36.299770+0000","last_clean":"2024-02-23T00:46:36.299770+0000","last_became_active":"2024-02-23T00:39:14.783325+0000","last_became_peered":"2024-02-23T00:39:14.783325+0000","last_unstale":"2024-02-23T00:46:36.299770+0000","last_undegraded":"2024-02-23T00:46:36.299770+0000","last_fullsized":"2024-02-23T00:46:36.299770+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:35.268491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:35.268491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":288,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.171035+0000","last_change":"2024-02-23T00:45:20.761547+0000","last_active":"2024-02-23T00:46:36.171035+0000","last_peered":"2024-02-23T00:46:36.171035+0000","last_clean":"2024-02-23T00:46:36.171035+0000","last_became_active":"2024-02-23T00:39:18.797287+0000","last_became_peered":"2024-02-23T00:39:18.797287+0000","last_unstale":"2024-02-23T00:46:36.171035+0000","last_undegraded":"2024-02-23T00:46:36.171035+0000","last_fullsized":"2024-02-23T00:46:36.171035+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:20.761440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:20.761440+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":280,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.162580+0000","last_change":"2024-02-23T00:45:10.741762+0000","last_active":"2024-02-23T00:46:36.162580+0000","last_peered":"2024-02-23T00:46:36.162580+0000","last_clean":"2024-02-23T00:46:36.162580+0000","last_became_active":"2024-02-23T00:39:20.795351+0000","last_became_peered":"2024-02-23T00:39:20.795351+0000","last_unstale":"2024-02-23T00:46:36.162580+0000","last_undegraded":"2024-02-23T00:46:36.162580+0000","last_fullsized":"2024-02-23T00:46:36.162580+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.741694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.741694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":229,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.164079+0000","last_change":"2024-02-23T00:45:56.706901+0000","last_active":"2024-02-23T00:46:36.164079+0000","last_peered":"2024-02-23T00:46:36.164079+0000","last_clean":"2024-02-23T00:46:36.164079+0000","last_became_active":"2024-02-23T00:42:58.741365+0000","last_became_peered":"2024-02-23T00:42:58.741365+0000","last_unstale":"2024-02-23T00:46:36.164079+0000","last_undegraded":"2024-02-23T00:46:36.164079+0000","last_fullsized":"2024-02-23T00:46:36.164079+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:56.706792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:56.706792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"68'17","reported_seq":321,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.159608+0000","last_change":"2024-02-23T00:43:43.997579+0000","last_active":"2024-02-23T00:46:36.159608+0000","last_peered":"2024-02-23T00:46:36.159608+0000","last_clean":"2024-02-23T00:46:36.159608+0000","last_became_active":"2024-02-23T00:39:16.857232+0000","last_became_peered":"2024-02-23T00:39:16.857232+0000","last_unstale":"2024-02-23T00:46:36.159608+0000","last_undegraded":"2024-02-23T00:46:36.159608+0000","last_fullsized":"2024-02-23T00:46:36.159608+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:43:43.997452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:43.997452+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":287,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.172001+0000","last_change":"2024-02-23T00:43:52.419206+0000","last_active":"2024-02-23T00:46:36.172001+0000","last_peered":"2024-02-23T00:46:36.172001+0000","last_clean":"2024-02-23T00:46:36.172001+0000","last_became_active":"2024-02-23T00:39:14.784092+0000","last_became_peered":"2024-02-23T00:39:14.784092+0000","last_unstale":"2024-02-23T00:46:36.172001+0000","last_undegraded":"2024-02-23T00:46:36.172001+0000","last_fullsized":"2024-02-23T00:46:36.172001+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:52.419100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:52.419100+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":282,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.162901+0000","last_change":"2024-02-23T00:44:25.797288+0000","last_active":"2024-02-23T00:46:36.162901+0000","last_peered":"2024-02-23T00:46:36.162901+0000","last_clean":"2024-02-23T00:46:36.162901+0000","last_became_active":"2024-02-23T00:39:18.791457+0000","last_became_peered":"2024-02-23T00:39:18.791457+0000","last_unstale":"2024-02-23T00:46:36.162901+0000","last_undegraded":"2024-02-23T00:46:36.162901+0000","last_fullsized":"2024-02-23T00:46:36.162901+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:25.797184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:25.797184+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":278,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.163346+0000","last_change":"2024-02-23T00:43:45.965636+0000","last_active":"2024-02-23T00:46:36.163346+0000","last_peered":"2024-02-23T00:46:36.163346+0000","last_clean":"2024-02-23T00:46:36.163346+0000","last_became_active":"2024-02-23T00:39:21.160376+0000","last_became_peered":"2024-02-23T00:39:21.160376+0000","last_unstale":"2024-02-23T00:46:36.163346+0000","last_undegraded":"2024-02-23T00:46:36.163346+0000","last_fullsized":"2024-02-23T00:46:36.163346+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:45.965570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.965570+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":229,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.161119+0000","last_change":"2024-02-23T00:45:23.378334+0000","last_active":"2024-02-23T00:46:36.161119+0000","last_peered":"2024-02-23T00:46:36.161119+0000","last_clean":"2024-02-23T00:46:36.161119+0000","last_became_active":"2024-02-23T00:42:58.702580+0000","last_became_peered":"2024-02-23T00:42:58.702580+0000","last_unstale":"2024-02-23T00:46:36.161119+0000","last_undegraded":"2024-02-23T00:46:36.161119+0000","last_fullsized":"2024-02-23T00:46:36.161119+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:23.378270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.378270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"68'16","reported_seq":317,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.593884+0000","last_change":"2024-02-23T00:45:04.672612+0000","last_active":"2024-02-23T00:46:36.593884+0000","last_peered":"2024-02-23T00:46:36.593884+0000","last_clean":"2024-02-23T00:46:36.593884+0000","last_became_active":"2024-02-23T00:39:16.858945+0000","last_became_peered":"2024-02-23T00:39:16.858945+0000","last_unstale":"2024-02-23T00:46:36.593884+0000","last_undegraded":"2024-02-23T00:46:36.593884+0000","last_fullsized":"2024-02-23T00:46:36.593884+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:45:04.672537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.672537+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":301,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.297955+0000","last_change":"2024-02-23T00:46:09.088334+0000","last_active":"2024-02-23T00:46:36.297955+0000","last_peered":"2024-02-23T00:46:36.297955+0000","last_clean":"2024-02-23T00:46:36.297955+0000","last_became_active":"2024-02-23T00:39:14.785363+0000","last_became_peered":"2024-02-23T00:39:14.785363+0000","last_unstale":"2024-02-23T00:46:36.297955+0000","last_undegraded":"2024-02-23T00:46:36.297955+0000","last_fullsized":"2024-02-23T00:46:36.297955+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"80'2","last_scrub_stamp":"2024-02-23T00:46:09.088249+0000","last_deep_scrub":"80'2","last_deep_scrub_stamp":"2024-02-23T00:40:15.942725+0000","last_clean_scrub_stamp":"2024-02-23T00:46:09.088249+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"93'9","reported_seq":383,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.172533+0000","last_change":"2024-02-23T00:43:50.393789+0000","last_active":"2024-02-23T00:46:36.172533+0000","last_peered":"2024-02-23T00:46:36.172533+0000","last_clean":"2024-02-23T00:46:36.172533+0000","last_became_active":"2024-02-23T00:39:18.796680+0000","last_became_peered":"2024-02-23T00:39:18.796680+0000","last_unstale":"2024-02-23T00:46:36.172533+0000","last_undegraded":"2024-02-23T00:46:36.172533+0000","last_fullsized":"2024-02-23T00:46:36.172533+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-02-23T00:43:50.393713+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-02-23T00:43:50.393713+0000","last_clean_scrub_stamp":"2024-02-23T00:43:50.393713+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":230,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.593743+0000","last_change":"2024-02-23T00:45:58.613561+0000","last_active":"2024-02-23T00:46:36.593743+0000","last_peered":"2024-02-23T00:46:36.593743+0000","last_clean":"2024-02-23T00:46:36.593743+0000","last_became_active":"2024-02-23T00:42:58.850072+0000","last_became_peered":"2024-02-23T00:42:58.850072+0000","last_unstale":"2024-02-23T00:46:36.593743+0000","last_undegraded":"2024-02-23T00:46:36.593743+0000","last_fullsized":"2024-02-23T00:46:36.593743+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:58.613497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:58.613497+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"93'23","reported_seq":330,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.172367+0000","last_change":"2024-02-23T00:44:28.387225+0000","last_active":"2024-02-23T00:46:36.172367+0000","last_peered":"2024-02-23T00:46:36.172367+0000","last_clean":"2024-02-23T00:46:36.172367+0000","last_became_active":"2024-02-23T00:39:16.782660+0000","last_became_peered":"2024-02-23T00:39:16.782660+0000","last_unstale":"2024-02-23T00:46:36.172367+0000","last_undegraded":"2024-02-23T00:46:36.172367+0000","last_fullsized":"2024-02-23T00:46:36.172367+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'23","last_scrub_stamp":"2024-02-23T00:44:28.387155+0000","last_deep_scrub":"93'23","last_deep_scrub_stamp":"2024-02-23T00:41:35.954516+0000","last_clean_scrub_stamp":"2024-02-23T00:44:28.387155+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":289,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.159008+0000","last_change":"2024-02-23T00:44:21.157372+0000","last_active":"2024-02-23T00:46:36.159008+0000","last_peered":"2024-02-23T00:46:36.159008+0000","last_clean":"2024-02-23T00:46:36.159008+0000","last_became_active":"2024-02-23T00:39:14.781560+0000","last_became_peered":"2024-02-23T00:39:14.781560+0000","last_unstale":"2024-02-23T00:46:36.159008+0000","last_undegraded":"2024-02-23T00:46:36.159008+0000","last_fullsized":"2024-02-23T00:46:36.159008+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:21.157316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:21.157316+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":290,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.593925+0000","last_change":"2024-02-23T00:46:36.592971+0000","last_active":"2024-02-23T00:46:36.593925+0000","last_peered":"2024-02-23T00:46:36.593925+0000","last_clean":"2024-02-23T00:46:36.593925+0000","last_became_active":"2024-02-23T00:39:18.790602+0000","last_became_peered":"2024-02-23T00:39:18.790602+0000","last_unstale":"2024-02-23T00:46:36.593925+0000","last_undegraded":"2024-02-23T00:46:36.593925+0000","last_fullsized":"2024-02-23T00:46:36.593925+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:36.592887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:53.189881+0000","last_clean_scrub_stamp":"2024-02-23T00:46:36.592887+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":230,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.159091+0000","last_change":"2024-02-23T00:45:15.933508+0000","last_active":"2024-02-23T00:46:36.159091+0000","last_peered":"2024-02-23T00:46:36.159091+0000","last_clean":"2024-02-23T00:46:36.159091+0000","last_became_active":"2024-02-23T00:42:58.774720+0000","last_became_peered":"2024-02-23T00:42:58.774720+0000","last_unstale":"2024-02-23T00:46:36.159091+0000","last_undegraded":"2024-02-23T00:46:36.159091+0000","last_fullsized":"2024-02-23T00:46:36.159091+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:15.933438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:15.933438+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"67'15","reported_seq":315,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.165057+0000","last_change":"2024-02-23T00:46:03.951592+0000","last_active":"2024-02-23T00:46:36.165057+0000","last_peered":"2024-02-23T00:46:36.165057+0000","last_clean":"2024-02-23T00:46:36.165057+0000","last_became_active":"2024-02-23T00:39:16.787138+0000","last_became_peered":"2024-02-23T00:39:16.787138+0000","last_unstale":"2024-02-23T00:46:36.165057+0000","last_undegraded":"2024-02-23T00:46:36.165057+0000","last_fullsized":"2024-02-23T00:46:36.165057+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:46:03.951524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:03.951524+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":292,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.173487+0000","last_change":"2024-02-23T00:46:10.011072+0000","last_active":"2024-02-23T00:46:36.173487+0000","last_peered":"2024-02-23T00:46:36.173487+0000","last_clean":"2024-02-23T00:46:36.173487+0000","last_became_active":"2024-02-23T00:39:14.782494+0000","last_became_peered":"2024-02-23T00:39:14.782494+0000","last_unstale":"2024-02-23T00:46:36.173487+0000","last_undegraded":"2024-02-23T00:46:36.173487+0000","last_fullsized":"2024-02-23T00:46:36.173487+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:10.010996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:10.010996+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":284,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.403897+0000","last_change":"2024-02-23T00:45:03.615758+0000","last_active":"2024-02-23T00:46:36.403897+0000","last_peered":"2024-02-23T00:46:36.403897+0000","last_clean":"2024-02-23T00:46:36.403897+0000","last_became_active":"2024-02-23T00:39:18.801039+0000","last_became_peered":"2024-02-23T00:39:18.801039+0000","last_unstale":"2024-02-23T00:46:36.403897+0000","last_undegraded":"2024-02-23T00:46:36.403897+0000","last_fullsized":"2024-02-23T00:46:36.403897+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:03.615692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.641689+0000","last_clean_scrub_stamp":"2024-02-23T00:45:03.615692+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":230,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.160905+0000","last_change":"2024-02-23T00:45:21.917072+0000","last_active":"2024-02-23T00:46:36.160905+0000","last_peered":"2024-02-23T00:46:36.160905+0000","last_clean":"2024-02-23T00:46:36.160905+0000","last_became_active":"2024-02-23T00:42:58.770759+0000","last_became_peered":"2024-02-23T00:42:58.770759+0000","last_unstale":"2024-02-23T00:46:36.160905+0000","last_undegraded":"2024-02-23T00:46:36.160905+0000","last_fullsized":"2024-02-23T00:46:36.160905+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:21.916996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:21.916996+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"68'18","reported_seq":325,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.160345+0000","last_change":"2024-02-23T00:43:51.079824+0000","last_active":"2024-02-23T00:46:36.160345+0000","last_peered":"2024-02-23T00:46:36.160345+0000","last_clean":"2024-02-23T00:46:36.160345+0000","last_became_active":"2024-02-23T00:39:16.782803+0000","last_became_peered":"2024-02-23T00:39:16.782803+0000","last_unstale":"2024-02-23T00:46:36.160345+0000","last_undegraded":"2024-02-23T00:46:36.160345+0000","last_fullsized":"2024-02-23T00:46:36.160345+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'18","last_scrub_stamp":"2024-02-23T00:43:51.079756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:51.079756+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":289,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.163097+0000","last_change":"2024-02-23T00:43:48.030906+0000","last_active":"2024-02-23T00:46:36.163097+0000","last_peered":"2024-02-23T00:46:36.163097+0000","last_clean":"2024-02-23T00:46:36.163097+0000","last_became_active":"2024-02-23T00:39:14.782163+0000","last_became_peered":"2024-02-23T00:39:14.782163+0000","last_unstale":"2024-02-23T00:46:36.163097+0000","last_undegraded":"2024-02-23T00:46:36.163097+0000","last_fullsized":"2024-02-23T00:46:36.163097+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:48.030846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.030846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":284,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.168173+0000","last_change":"2024-02-23T00:44:37.068899+0000","last_active":"2024-02-23T00:46:36.168173+0000","last_peered":"2024-02-23T00:46:36.168173+0000","last_clean":"2024-02-23T00:46:36.168173+0000","last_became_active":"2024-02-23T00:39:18.794903+0000","last_became_peered":"2024-02-23T00:39:18.794903+0000","last_unstale":"2024-02-23T00:46:36.168173+0000","last_undegraded":"2024-02-23T00:46:36.168173+0000","last_fullsized":"2024-02-23T00:46:36.168173+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:37.068834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_clean_scrub_stamp":"2024-02-23T00:44:37.068834+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":230,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.300486+0000","last_change":"2024-02-23T00:46:00.943625+0000","last_active":"2024-02-23T00:46:36.300486+0000","last_peered":"2024-02-23T00:46:36.300486+0000","last_clean":"2024-02-23T00:46:36.300486+0000","last_became_active":"2024-02-23T00:42:58.698258+0000","last_became_peered":"2024-02-23T00:42:58.698258+0000","last_unstale":"2024-02-23T00:46:36.300486+0000","last_undegraded":"2024-02-23T00:46:36.300486+0000","last_fullsized":"2024-02-23T00:46:36.300486+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:00.943545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:46:00.943545+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"95'27","reported_seq":338,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.167401+0000","last_change":"2024-02-23T00:43:52.124336+0000","last_active":"2024-02-23T00:46:36.167401+0000","last_peered":"2024-02-23T00:46:36.167401+0000","last_clean":"2024-02-23T00:46:36.167401+0000","last_became_active":"2024-02-23T00:39:16.786791+0000","last_became_peered":"2024-02-23T00:39:16.786791+0000","last_unstale":"2024-02-23T00:46:36.167401+0000","last_undegraded":"2024-02-23T00:46:36.167401+0000","last_fullsized":"2024-02-23T00:46:36.167401+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'27","last_scrub_stamp":"2024-02-23T00:43:52.124247+0000","last_deep_scrub":"95'27","last_deep_scrub_stamp":"2024-02-23T00:41:55.919895+0000","last_clean_scrub_stamp":"2024-02-23T00:43:52.124247+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":286,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.163441+0000","last_change":"2024-02-23T00:44:27.759571+0000","last_active":"2024-02-23T00:46:36.163441+0000","last_peered":"2024-02-23T00:46:36.163441+0000","last_clean":"2024-02-23T00:46:36.163441+0000","last_became_active":"2024-02-23T00:39:14.787148+0000","last_became_peered":"2024-02-23T00:39:14.787148+0000","last_unstale":"2024-02-23T00:46:36.163441+0000","last_undegraded":"2024-02-23T00:46:36.163441+0000","last_fullsized":"2024-02-23T00:46:36.163441+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:27.759499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:27.759499+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":285,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.299957+0000","last_change":"2024-02-23T00:45:10.090353+0000","last_active":"2024-02-23T00:46:36.299957+0000","last_peered":"2024-02-23T00:46:36.299957+0000","last_clean":"2024-02-23T00:46:36.299957+0000","last_became_active":"2024-02-23T00:39:18.796431+0000","last_became_peered":"2024-02-23T00:39:18.796431+0000","last_unstale":"2024-02-23T00:46:36.299957+0000","last_undegraded":"2024-02-23T00:46:36.299957+0000","last_fullsized":"2024-02-23T00:46:36.299957+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.090270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.090270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.11","version":"307'9","reported_seq":75,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.170693+0000","last_change":"2024-02-23T00:45:45.170820+0000","last_active":"2024-02-23T00:46:36.170693+0000","last_peered":"2024-02-23T00:46:36.170693+0000","last_clean":"2024-02-23T00:46:36.170693+0000","last_became_active":"2024-02-23T00:45:45.155700+0000","last_became_peered":"2024-02-23T00:45:45.155700+0000","last_unstale":"2024-02-23T00:46:36.170693+0000","last_undegraded":"2024-02-23T00:46:36.170693+0000","last_fullsized":"2024-02-23T00:46:36.170693+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'7","last_scrub_stamp":"2024-02-23T00:44:30.366391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:44:30.366391+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":230,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.299857+0000","last_change":"2024-02-23T00:45:51.985111+0000","last_active":"2024-02-23T00:46:36.299857+0000","last_peered":"2024-02-23T00:46:36.299857+0000","last_clean":"2024-02-23T00:46:36.299857+0000","last_became_active":"2024-02-23T00:42:58.832885+0000","last_became_peered":"2024-02-23T00:42:58.832885+0000","last_unstale":"2024-02-23T00:46:36.299857+0000","last_undegraded":"2024-02-23T00:46:36.299857+0000","last_fullsized":"2024-02-23T00:46:36.299857+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:51.985023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:51.985023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"82'15","reported_seq":314,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-02-23T00:46:35.178088+0000","last_change":"2024-02-23T00:44:35.095478+0000","last_active":"2024-02-23T00:46:35.178088+0000","last_peered":"2024-02-23T00:46:35.178088+0000","last_clean":"2024-02-23T00:46:35.178088+0000","last_became_active":"2024-02-23T00:39:16.782762+0000","last_became_peered":"2024-02-23T00:39:16.782762+0000","last_unstale":"2024-02-23T00:46:35.178088+0000","last_undegraded":"2024-02-23T00:46:35.178088+0000","last_fullsized":"2024-02-23T00:46:35.178088+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'15","last_scrub_stamp":"2024-02-23T00:44:35.095409+0000","last_deep_scrub":"82'15","last_deep_scrub_stamp":"2024-02-23T00:41:50.975683+0000","last_clean_scrub_stamp":"2024-02-23T00:44:35.095409+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":291,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.163846+0000","last_change":"2024-02-23T00:46:08.794420+0000","last_active":"2024-02-23T00:46:36.163846+0000","last_peered":"2024-02-23T00:46:36.163846+0000","last_clean":"2024-02-23T00:46:36.163846+0000","last_became_active":"2024-02-23T00:39:14.783469+0000","last_became_peered":"2024-02-23T00:39:14.783469+0000","last_unstale":"2024-02-23T00:46:36.163846+0000","last_undegraded":"2024-02-23T00:46:36.163846+0000","last_fullsized":"2024-02-23T00:46:36.163846+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:08.794352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:08.794352+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":289,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.163027+0000","last_change":"2024-02-23T00:46:09.832330+0000","last_active":"2024-02-23T00:46:36.163027+0000","last_peered":"2024-02-23T00:46:36.163027+0000","last_clean":"2024-02-23T00:46:36.163027+0000","last_became_active":"2024-02-23T00:39:18.796952+0000","last_became_peered":"2024-02-23T00:39:18.796952+0000","last_unstale":"2024-02-23T00:46:36.163027+0000","last_undegraded":"2024-02-23T00:46:36.163027+0000","last_fullsized":"2024-02-23T00:46:36.163027+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:09.832253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:09.832253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":62,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.160089+0000","last_change":"2024-02-23T00:45:45.160209+0000","last_active":"2024-02-23T00:46:36.160089+0000","last_peered":"2024-02-23T00:46:36.160089+0000","last_clean":"2024-02-23T00:46:36.160089+0000","last_became_active":"2024-02-23T00:45:45.159835+0000","last_became_peered":"2024-02-23T00:45:45.159835+0000","last_unstale":"2024-02-23T00:46:36.160089+0000","last_undegraded":"2024-02-23T00:46:36.160089+0000","last_fullsized":"2024-02-23T00:46:36.160089+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.215063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.215063+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":231,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-02-23T00:46:35.177288+0000","last_change":"2024-02-23T00:45:23.224771+0000","last_active":"2024-02-23T00:46:35.177288+0000","last_peered":"2024-02-23T00:46:35.177288+0000","last_clean":"2024-02-23T00:46:35.177288+0000","last_became_active":"2024-02-23T00:42:58.758147+0000","last_became_peered":"2024-02-23T00:42:58.758147+0000","last_unstale":"2024-02-23T00:46:35.177288+0000","last_undegraded":"2024-02-23T00:46:35.177288+0000","last_fullsized":"2024-02-23T00:46:35.177288+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:23.224663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:23.224663+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.224663+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"68'15","reported_seq":317,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.167072+0000","last_change":"2024-02-23T00:43:49.139235+0000","last_active":"2024-02-23T00:46:36.167072+0000","last_peered":"2024-02-23T00:46:36.167072+0000","last_clean":"2024-02-23T00:46:36.167072+0000","last_became_active":"2024-02-23T00:39:16.784606+0000","last_became_peered":"2024-02-23T00:39:16.784606+0000","last_unstale":"2024-02-23T00:46:36.167072+0000","last_undegraded":"2024-02-23T00:46:36.167072+0000","last_fullsized":"2024-02-23T00:46:36.167072+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:43:49.139156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:49.139156+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":289,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.162428+0000","last_change":"2024-02-23T00:44:32.170717+0000","last_active":"2024-02-23T00:46:36.162428+0000","last_peered":"2024-02-23T00:46:36.162428+0000","last_clean":"2024-02-23T00:46:36.162428+0000","last_became_active":"2024-02-23T00:39:14.785362+0000","last_became_peered":"2024-02-23T00:39:14.785362+0000","last_unstale":"2024-02-23T00:46:36.162428+0000","last_undegraded":"2024-02-23T00:46:36.162428+0000","last_fullsized":"2024-02-23T00:46:36.162428+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:32.170554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:32.170554+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":279,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.401019+0000","last_change":"2024-02-23T00:42:44.553722+0000","last_active":"2024-02-23T00:46:36.401019+0000","last_peered":"2024-02-23T00:46:36.401019+0000","last_clean":"2024-02-23T00:46:36.401019+0000","last_became_active":"2024-02-23T00:39:18.796312+0000","last_became_peered":"2024-02-23T00:39:18.796312+0000","last_unstale":"2024-02-23T00:46:36.401019+0000","last_undegraded":"2024-02-23T00:46:36.401019+0000","last_fullsized":"2024-02-23T00:46:36.401019+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:44.553621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:44.553621+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.13","version":"307'11","reported_seq":77,"reported_epoch":325,"state":"activating","last_fresh":"2024-02-23T00:46:37.170774+0000","last_change":"2024-02-23T00:46:37.170944+0000","last_active":"2024-02-23T00:46:36.166123+0000","last_peered":"2024-02-23T00:46:35.178043+0000","last_clean":"2024-02-23T00:46:35.178043+0000","last_became_active":"2024-02-23T00:45:45.162350+0000","last_became_peered":"2024-02-23T00:45:45.162350+0000","last_unstale":"2024-02-23T00:46:37.170774+0000","last_undegraded":"2024-02-23T00:46:37.170774+0000","last_fullsized":"2024-02-23T00:46:37.170774+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'9","last_scrub_stamp":"2024-02-23T00:45:21.719900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:21.719900+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":["4","3","6"],"object_location_counts":[{"shards":"3,4,6","objects":1}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":230,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.593754+0000","last_change":"2024-02-23T00:45:09.641006+0000","last_active":"2024-02-23T00:46:36.593754+0000","last_peered":"2024-02-23T00:46:36.593754+0000","last_clean":"2024-02-23T00:46:36.593754+0000","last_became_active":"2024-02-23T00:42:58.838257+0000","last_became_peered":"2024-02-23T00:42:58.838257+0000","last_unstale":"2024-02-23T00:46:36.593754+0000","last_undegraded":"2024-02-23T00:46:36.593754+0000","last_fullsized":"2024-02-23T00:46:36.593754+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:09.640936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:09.640936+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"88'50","reported_seq":383,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.159705+0000","last_change":"2024-02-23T00:45:06.038488+0000","last_active":"2024-02-23T00:46:36.159705+0000","last_peered":"2024-02-23T00:46:36.159705+0000","last_clean":"2024-02-23T00:46:36.159705+0000","last_became_active":"2024-02-23T00:39:16.780144+0000","last_became_peered":"2024-02-23T00:39:16.780144+0000","last_unstale":"2024-02-23T00:46:36.159705+0000","last_undegraded":"2024-02-23T00:46:36.159705+0000","last_fullsized":"2024-02-23T00:46:36.159705+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'50","last_scrub_stamp":"2024-02-23T00:45:06.038374+0000","last_deep_scrub":"88'50","last_deep_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.038374+0000","log_size":50,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":289,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.594046+0000","last_change":"2024-02-23T00:45:20.573233+0000","last_active":"2024-02-23T00:46:36.594046+0000","last_peered":"2024-02-23T00:46:36.594046+0000","last_clean":"2024-02-23T00:46:36.594046+0000","last_became_active":"2024-02-23T00:39:14.782000+0000","last_became_peered":"2024-02-23T00:39:14.782000+0000","last_unstale":"2024-02-23T00:46:36.594046+0000","last_undegraded":"2024-02-23T00:46:36.594046+0000","last_fullsized":"2024-02-23T00:46:36.594046+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:20.573168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_clean_scrub_stamp":"2024-02-23T00:45:20.573168+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":279,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.167475+0000","last_change":"2024-02-23T00:43:48.126631+0000","last_active":"2024-02-23T00:46:36.167475+0000","last_peered":"2024-02-23T00:46:36.167475+0000","last_clean":"2024-02-23T00:46:36.167475+0000","last_became_active":"2024-02-23T00:39:18.799723+0000","last_became_peered":"2024-02-23T00:39:18.799723+0000","last_unstale":"2024-02-23T00:46:36.167475+0000","last_undegraded":"2024-02-23T00:46:36.167475+0000","last_fullsized":"2024-02-23T00:46:36.167475+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:48.126552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.126552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":62,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.162292+0000","last_change":"2024-02-23T00:45:45.157332+0000","last_active":"2024-02-23T00:46:36.162292+0000","last_peered":"2024-02-23T00:46:36.162292+0000","last_clean":"2024-02-23T00:46:36.162292+0000","last_became_active":"2024-02-23T00:45:45.156946+0000","last_became_peered":"2024-02-23T00:45:45.156946+0000","last_unstale":"2024-02-23T00:46:36.162292+0000","last_undegraded":"2024-02-23T00:46:36.162292+0000","last_fullsized":"2024-02-23T00:46:36.162292+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:24.942535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:24.942535+0000","last_clean_scrub_stamp":"2024-02-23T00:45:24.942535+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":230,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-02-23T00:46:36.299918+0000","last_change":"2024-02-23T00:45:14.209403+0000","last_active":"2024-02-23T00:46:36.299918+0000","last_peered":"2024-02-23T00:46:36.299918+0000","last_clean":"2024-02-23T00:46:36.299918+0000","last_became_active":"2024-02-23T00:42:58.824573+0000","last_became_peered":"2024-02-23T00:42:58.824573+0000","last_unstale":"2024-02-23T00:46:36.299918+0000","last_undegraded":"2024-02-23T00:46:36.299918+0000","last_fullsized":"2024-02-23T00:46:36.299918+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.209308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.209308+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0],"acting":[5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":62,"num_pg":32,"stat_sum":{"num_bytes":4001220,"num_objects":123,"num_object_clones":2,"num_object_copies":369,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":0,"num_read":98,"num_read_kb":4,"num_write":548,"num_write_kb":4449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":13492224,"data_stored":12003660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1794,"ondisk_log_size":1794,"up":96,"acting":96,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":191,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":7},{"poolid":6,"num_pg":22,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":66,"acting":66,"num_store_stats":8},{"poolid":190,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1080,"num_read_kb":805,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":651,"ondisk_log_size":651,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":434,"num_read_kb":429,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":8},{"poolid":15,"num_pg":20,"stat_sum":{"num_bytes":2048,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":196608,"data_stored":6144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57,"ondisk_log_size":57,"up":60,"acting":60,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234076,"num_pgs":95,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":310644,"kb_used_data":9320,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93434508,"statfs":{"total":95995035648,"available":95676936192,"internally_reserved":0,"allocated":9543680,"data_stored":7328327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:46:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54900000000000004,"15min":0.54600000000000004},"min":{"1min":0.315,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.76400000000000001,"5min":4.9420000000000002,"15min":4.9420000000000002},"last":0.503},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54600000000000004,"15min":0.51800000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.77800000000000002,"5min":4.9729999999999999,"15min":4.9729999999999999},"last":0.64700000000000002}]},{"osd":1,"last update":"Fri Feb 23 00:46:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.78600000000000003,"15min":0.627},"min":{"1min":0.40600000000000003,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70499999999999996,"5min":10.786,"15min":10.786},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.79300000000000004,"15min":0.628},"min":{"1min":0.317,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.73399999999999999,"5min":11.885999999999999,"15min":11.885999999999999},"last":0.54500000000000004}]},{"osd":2,"last update":"Fri Feb 23 00:46:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.60399999999999998,"15min":0.55900000000000005},"min":{"1min":0.40000000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71199999999999997,"5min":5.0789999999999997,"15min":5.0789999999999997},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.58299999999999996,"15min":0.54800000000000004},"min":{"1min":0.38300000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.69399999999999995,"5min":5.1280000000000001,"15min":5.1280000000000001},"last":0.46100000000000002}]},{"osd":3,"last update":"Fri Feb 23 00:46:26 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.81000000000000005,"15min":0.61899999999999999},"min":{"1min":0.34499999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67600000000000005,"5min":15.712999999999999,"15min":15.712999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.73799999999999999,"15min":0.621},"min":{"1min":0.376,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.1659999999999999,"5min":9.4090000000000007,"15min":9.4090000000000007},"last":0.57999999999999996}]},{"osd":4,"last update":"Fri Feb 23 00:46:26 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.45800000000000002,"15min":0.437},"min":{"1min":0.28599999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.65300000000000002,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.56999999999999995,"15min":0.48899999999999999},"min":{"1min":0.38600000000000001,"5min":0.28599999999999998,"15min":0.28199999999999997},"max":{"1min":0.66700000000000004,"5min":11.236000000000001,"15min":11.236000000000001},"last":0.30199999999999999}]},{"osd":5,"last update":"Fri Feb 23 00:46:26 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.53600000000000003,"15min":0.49299999999999999},"min":{"1min":0.38100000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.68400000000000005,"5min":5.4000000000000004,"15min":5.4000000000000004},"last":0.314},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52300000000000002,"15min":0.48999999999999999},"min":{"1min":0.376,"5min":0.29299999999999998,"15min":0.26700000000000002},"max":{"1min":0.67800000000000005,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.36499999999999999}]},{"osd":6,"last update":"Fri Feb 23 00:46:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52400000000000002,"15min":0.50900000000000001},"min":{"1min":0.42499999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.66000000000000003,"5min":1.034,"15min":1.034},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.64200000000000002,"15min":0.54800000000000004},"min":{"1min":0.38400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.2370000000000001,"5min":13.395,"15min":13.395},"last":0.38100000000000001}]}]},{"osd":6,"up_from":47,"seq":201863463011,"num_pgs":85,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":314912,"kb_used_data":5388,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93430240,"statfs":{"total":95995035648,"available":95672565760,"internally_reserved":0,"allocated":5517312,"data_stored":3329106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.97299999999999998,"15min":0.70499999999999996},"min":{"1min":0.33500000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.65000000000000002,"5min":30.641999999999999,"15min":30.641999999999999},"last":0.315},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.97499999999999998,"15min":0.71399999999999997},"min":{"1min":0.42999999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.71199999999999997,"5min":29.292000000000002,"15min":29.292000000000002},"last":0.26100000000000001}]},{"osd":1,"last update":"Fri Feb 23 00:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.98499999999999999,"15min":0.69599999999999995},"min":{"1min":0.40600000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.76700000000000002,"5min":28.186,"15min":28.186},"last":0.247},{"interface":"front","average":{"1min":0.48799999999999999,"5min":1.48,"15min":0.88200000000000001},"min":{"1min":0.39700000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.67800000000000005,"5min":33.104999999999997,"15min":33.104999999999997},"last":0.32600000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":1.034,"15min":0.73399999999999999},"min":{"1min":0.38500000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.75600000000000001,"5min":19.960000000000001,"15min":19.960000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.84299999999999997,"15min":0.65500000000000003},"min":{"1min":0.38600000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.79300000000000004,"5min":8.5860000000000003,"15min":8.5860000000000003},"last":0.38300000000000001}]},{"osd":3,"last update":"Fri Feb 23 00:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.75600000000000001,"15min":0.64000000000000001},"min":{"1min":0.439,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80000000000000004,"5min":19.562000000000001,"15min":19.562000000000001},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.84699999999999998,"15min":0.67900000000000005},"min":{"1min":0.35399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69999999999999996,"5min":14.199,"15min":14.199},"last":0.46000000000000002}]},{"osd":4,"last update":"Fri Feb 23 00:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.58299999999999996,"15min":0.46500000000000002},"min":{"1min":0.27700000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.70299999999999996,"5min":11.109999999999999,"15min":11.109999999999999},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.752,"15min":0.53600000000000003},"min":{"1min":0.314,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.71099999999999997,"5min":16.734000000000002,"15min":16.734000000000002},"last":0.26900000000000002}]},{"osd":5,"last update":"Fri Feb 23 00:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.63700000000000001,"15min":0.48799999999999999},"min":{"1min":0.32400000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.745,"5min":14.154999999999999,"15min":14.154999999999999},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.63500000000000001,"15min":0.504},"min":{"1min":0.27900000000000003,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.67600000000000005,"5min":6.6100000000000003,"15min":6.6100000000000003},"last":0.36399999999999999}]},{"osd":7,"last update":"Fri Feb 23 00:46:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.59299999999999997,"15min":0.52400000000000002},"min":{"1min":0.371,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.78100000000000003,"5min":8.548,"15min":8.548},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.76700000000000002,"15min":0.59899999999999998},"min":{"1min":0.47199999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.76300000000000001,"5min":15.618,"15min":15.618},"last":0.34200000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574973,"num_pgs":100,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":318744,"kb_used_data":9228,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93426408,"statfs":{"total":95995035648,"available":95668641792,"internally_reserved":0,"allocated":9449472,"data_stored":7318229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:45:43 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":1.2909999999999999,"15min":0.66400000000000003},"min":{"1min":0.221,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.48799999999999999,"5min":78.361000000000004,"15min":78.361000000000004},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.57199999999999995,"15min":0.437},"min":{"1min":0.245,"5min":0.218,"15min":0.191},"max":{"1min":0.41199999999999998,"5min":5.8849999999999998,"15min":5.8849999999999998},"last":0.28299999999999997}]},{"osd":2,"last update":"Fri Feb 23 00:46:04 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":1.181,"15min":0.65600000000000003},"min":{"1min":0.25,"5min":0.219,"15min":0.188},"max":{"1min":0.63800000000000001,"5min":48.606000000000002,"15min":48.606000000000002},"last":0.245},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.879,"15min":0.58399999999999996},"min":{"1min":0.26700000000000002,"5min":0.25800000000000001,"15min":0.249},"max":{"1min":1.3460000000000001,"5min":23.911000000000001,"15min":23.911000000000001},"last":0.38200000000000001}]},{"osd":3,"last update":"Fri Feb 23 00:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.89400000000000002,"15min":0.624},"min":{"1min":0.28000000000000003,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.56100000000000005,"5min":35.244999999999997,"15min":35.244999999999997},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.38,"5min":1.1299999999999999,"15min":0.68200000000000005},"min":{"1min":0.28599999999999998,"5min":0.253,"15min":0.248},"max":{"1min":0.60299999999999998,"5min":56.853999999999999,"15min":56.853999999999999},"last":0.31}]},{"osd":4,"last update":"Fri Feb 23 00:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":1.2310000000000001,"15min":0.81699999999999995},"min":{"1min":0.34599999999999997,"5min":0.33700000000000002,"15min":0.29299999999999998},"max":{"1min":0.82399999999999995,"5min":37.646999999999998,"15min":37.646999999999998},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.86599999999999999,"15min":0.69699999999999995},"min":{"1min":0.375,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80700000000000005,"5min":9.6859999999999999,"15min":9.6859999999999999},"last":0.35899999999999999}]},{"osd":5,"last update":"Fri Feb 23 00:46:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":1.2150000000000001,"15min":0.83899999999999997},"min":{"1min":0.39800000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79100000000000004,"5min":48.792000000000002,"15min":48.792000000000002},"last":0.372},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.81399999999999995,"15min":0.745},"min":{"1min":0.45000000000000001,"5min":0.38100000000000001,"15min":0.34200000000000003},"max":{"1min":0.78800000000000003,"5min":11.872999999999999,"15min":11.872999999999999},"last":0.34100000000000003}]},{"osd":6,"last update":"Fri Feb 23 00:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":1.1890000000000001,"15min":0.82299999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.79400000000000004,"5min":40.741,"15min":40.741},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.77200000000000002,"15min":0.68999999999999995},"min":{"1min":0.41999999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.872,"5min":6.0069999999999997,"15min":6.0069999999999997},"last":0.442}]},{"osd":7,"last update":"Fri Feb 23 00:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.89200000000000002,"15min":0.71599999999999997},"min":{"1min":0.39300000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80800000000000005,"5min":20.225000000000001,"15min":20.225000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.93000000000000005,"15min":0.72399999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.73199999999999998,"5min":20.254999999999999,"15min":20.254999999999999},"last":0.45000000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738499,"num_pgs":85,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":318952,"kb_used_data":5332,"kb_used_omap":0,"kb_used_meta":313600,"kb_avail":93426200,"statfs":{"total":95995035648,"available":95668428800,"internally_reserved":0,"allocated":5459968,"data_stored":3317426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321126400},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 23 00:45:44 2024","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":1.089,"15min":0.59199999999999997},"min":{"1min":0.22600000000000001,"5min":0.223,"15min":0.19500000000000001},"max":{"1min":0.60899999999999999,"5min":28.856999999999999,"15min":28.856999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":1.026,"5min":1.1000000000000001,"15min":0.60699999999999998},"min":{"1min":0.25600000000000001,"5min":0.218,"15min":0.20799999999999999},"max":{"1min":13.162000000000001,"5min":20.577000000000002,"15min":20.577000000000002},"last":0.35299999999999998}]},{"osd":2,"last update":"Fri Feb 23 00:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":1.014,"15min":0.60299999999999998},"min":{"1min":0.27900000000000003,"5min":0.26000000000000001,"15min":0.20499999999999999},"max":{"1min":0.55200000000000005,"5min":13.863,"15min":13.863},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.318,"5min":0.86899999999999999,"15min":0.54500000000000004},"min":{"1min":0.219,"5min":0.219,"15min":0.19800000000000001},"max":{"1min":0.373,"5min":10.455,"15min":10.455},"last":0.28399999999999997}]},{"osd":3,"last update":"Fri Feb 23 00:45:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.5109999999999999,"15min":0.80300000000000005},"min":{"1min":0.27700000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":12.566000000000001,"5min":35.566000000000003,"15min":35.566000000000003},"last":0.39000000000000001},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.3660000000000001,"15min":0.79100000000000004},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":12.727,"5min":20.702999999999999,"15min":20.702999999999999},"last":0.40300000000000002}]},{"osd":4,"last update":"Fri Feb 23 00:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":1.0409999999999999,"15min":0.80300000000000005},"min":{"1min":0.36399999999999999,"5min":0.36199999999999999,"15min":0.34699999999999998},"max":{"1min":0.753,"5min":10.584,"15min":10.584},"last":0.64200000000000002},{"interface":"front","average":{"1min":1.258,"5min":1.032,"15min":0.82999999999999996},"min":{"1min":0.38900000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":13.198,"5min":13.198,"15min":13.198},"last":0.63300000000000001}]},{"osd":5,"last update":"Fri Feb 23 00:46:28 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":1.004,"15min":0.73799999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72499999999999998,"5min":10.397,"15min":10.397},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":1.077,"15min":0.80300000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84999999999999998,"5min":17.297000000000001,"15min":17.297000000000001},"last":0.622}]},{"osd":6,"last update":"Fri Feb 23 00:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":1.1080000000000001,"15min":0.79900000000000004},"min":{"1min":0.44400000000000001,"5min":0.42299999999999999,"15min":0.38400000000000001},"max":{"1min":0.85799999999999998,"5min":17.373999999999999,"15min":17.373999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":1.321,"5min":1.0409999999999999,"15min":0.77000000000000002},"min":{"1min":0.42199999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":13.973000000000001,"5min":13.973000000000001,"15min":13.973000000000001},"last":0.65400000000000003}]},{"osd":7,"last update":"Fri Feb 23 00:46:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":1.083,"15min":0.80700000000000005},"min":{"1min":0.45200000000000001,"5min":0.41799999999999998,"15min":0.39400000000000002},"max":{"1min":0.81399999999999995,"5min":17.399999999999999,"15min":17.399999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.629,"5min":1.0669999999999999,"15min":0.78800000000000003},"min":{"1min":0.49299999999999999,"5min":0.433,"15min":0.43099999999999999},"max":{"1min":0.81200000000000006,"5min":16.427,"15min":16.427},"last":0.61099999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411448,"num_pgs":88,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":314988,"kb_used_data":5460,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93430164,"statfs":{"total":95995035648,"available":95672487936,"internally_reserved":0,"allocated":5591040,"data_stored":3328841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:46:04 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.69399999999999995,"15min":0.48399999999999999},"min":{"1min":0.22900000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.61199999999999999,"5min":13.657,"15min":13.657},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.374,"5min":1.262,"15min":0.68000000000000005},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.624,"5min":42.343000000000004,"15min":42.343000000000004},"last":0.498}]},{"osd":1,"last update":"Fri Feb 23 00:46:04 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.96199999999999997,"15min":0.57399999999999995},"min":{"1min":0.254,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.53500000000000003,"5min":18.952000000000002,"15min":18.952000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.75800000000000001,"15min":0.52200000000000002},"min":{"1min":0.26300000000000001,"5min":0.23999999999999999,"15min":0.23599999999999999},"max":{"1min":0.56499999999999995,"5min":15.391,"15min":15.391},"last":0.40300000000000002}]},{"osd":3,"last update":"Fri Feb 23 00:45:35 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":1.1779999999999999,"15min":0.71299999999999997},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.47499999999999998,"5min":25.635999999999999,"15min":25.635999999999999},"last":0.436},{"interface":"front","average":{"1min":0.41799999999999998,"5min":1.002,"15min":0.64600000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.0840000000000001,"5min":19.960000000000001,"15min":19.960000000000001},"last":0.51100000000000001}]},{"osd":4,"last update":"Fri Feb 23 00:46:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.99199999999999999,"15min":0.72799999999999998},"min":{"1min":0.41299999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.74099999999999999,"5min":13.673,"15min":13.673},"last":0.621},{"interface":"front","average":{"1min":0.57599999999999996,"5min":1.0389999999999999,"15min":0.73499999999999999},"min":{"1min":0.379,"5min":0.36899999999999999,"15min":0.33900000000000002},"max":{"1min":0.90000000000000002,"5min":10.103999999999999,"15min":10.103999999999999},"last":0.63500000000000001}]},{"osd":5,"last update":"Fri Feb 23 00:46:27 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.90100000000000002,"15min":0.68799999999999994},"min":{"1min":0.40200000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.89000000000000001,"5min":15.612,"15min":15.612},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.86199999999999999,"15min":0.69799999999999995},"min":{"1min":0.40100000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.752,"5min":13.632999999999999,"15min":13.632999999999999},"last":0.64500000000000002}]},{"osd":6,"last update":"Fri Feb 23 00:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.88500000000000001,"15min":0.71099999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.5249999999999999,"5min":11.544,"15min":11.544},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":1.05,"15min":0.749},"min":{"1min":0.45400000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":4.1909999999999998,"5min":10.132999999999999,"15min":10.132999999999999},"last":0.61299999999999999}]},{"osd":7,"last update":"Fri Feb 23 00:46:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.86599999999999999,"15min":0.73599999999999999},"min":{"1min":0.40999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.81999999999999995,"5min":11.599,"15min":11.599},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.625,"5min":1.1020000000000001,"15min":0.79800000000000004},"min":{"1min":0.46100000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.92200000000000004,"5min":10.148999999999999,"15min":10.148999999999999},"last":0.65400000000000003}]}]},{"osd":3,"up_from":26,"seq":111669149809,"num_pgs":96,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":302660,"kb_used_data":5428,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93442492,"statfs":{"total":95995035648,"available":95685111808,"internally_reserved":0,"allocated":5558272,"data_stored":3330482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:45:35 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.746,"15min":0.53900000000000003},"min":{"1min":0.22800000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.56699999999999995,"5min":12.212999999999999,"15min":12.212999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.88900000000000001,"15min":0.59899999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.56899999999999995,"5min":11.714,"15min":11.714},"last":0.41899999999999998}]},{"osd":1,"last update":"Fri Feb 23 00:45:35 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.871,"15min":0.60799999999999998},"min":{"1min":0.29299999999999998,"5min":0.23400000000000001,"15min":0.22900000000000001},"max":{"1min":0.61299999999999999,"5min":11.742000000000001,"15min":11.742000000000001},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.78200000000000003,"15min":0.56100000000000005},"min":{"1min":0.223,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":1.8300000000000001,"5min":6.383,"15min":6.383},"last":0.373}]},{"osd":2,"last update":"Fri Feb 23 00:45:35 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.95699999999999996,"15min":0.64900000000000002},"min":{"1min":0.26000000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.869,"5min":13.359999999999999,"15min":13.359999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.374,"5min":1.002,"15min":0.65500000000000003},"min":{"1min":0.27500000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54400000000000004,"5min":21.122,"15min":21.122},"last":0.45700000000000002}]},{"osd":4,"last update":"Fri Feb 23 00:45:58 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.083,"15min":0.75900000000000001},"min":{"1min":0.36399999999999999,"5min":0.317,"15min":0.317},"max":{"1min":11.558,"5min":11.644,"15min":11.644},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.93600000000000005,"15min":0.70399999999999996},"min":{"1min":0.40400000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":6.1589999999999998,"5min":6.2549999999999999,"15min":6.2549999999999999},"last":0.84699999999999998}]},{"osd":5,"last update":"Fri Feb 23 00:46:26 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.79800000000000004,"15min":0.66600000000000004},"min":{"1min":0.441,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":2.125,"5min":5.6360000000000001,"15min":5.6360000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.1639999999999999,"15min":0.80200000000000005},"min":{"1min":0.45000000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":11.648,"5min":11.648,"15min":11.648},"last":0.749}]},{"osd":6,"last update":"Fri Feb 23 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.90500000000000003,"15min":0.70599999999999996},"min":{"1min":0.39000000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.0539999999999998,"5min":6.343,"15min":6.343},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.80000000000000004,"15min":0.68200000000000005},"min":{"1min":0.442,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.71699999999999997,"5min":5.6680000000000001,"15min":5.6680000000000001},"last":0.68600000000000005}]},{"osd":7,"last update":"Fri Feb 23 00:46:22 2024","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.206,"15min":0.82599999999999996},"min":{"1min":0.48399999999999999,"5min":0.41099999999999998,"15min":0.39000000000000001},"max":{"1min":11.676,"5min":11.676,"15min":11.676},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.411,"5min":1.1160000000000001,"15min":0.80800000000000005},"min":{"1min":0.51600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":7.9800000000000004,"5min":7.9800000000000004,"15min":7.9800000000000004},"last":0.873}]}]},{"osd":4,"up_from":32,"seq":137438953581,"num_pgs":90,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":322920,"kb_used_data":9308,"kb_used_omap":0,"kb_used_meta":313600,"kb_avail":93422232,"statfs":{"total":95995035648,"available":95664365568,"internally_reserved":0,"allocated":9531392,"data_stored":7328871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321126400},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:46:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51900000000000002,"15min":0.55600000000000005},"min":{"1min":0.30199999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.90900000000000003,"5min":3.5219999999999998,"15min":3.5219999999999998},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.66400000000000003,"15min":0.60799999999999998},"min":{"1min":0.38400000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.71499999999999997,"5min":8.6140000000000008,"15min":8.6140000000000008},"last":0.318}]},{"osd":1,"last update":"Fri Feb 23 00:46:02 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.66500000000000004,"15min":0.60599999999999998},"min":{"1min":0.33600000000000002,"5min":0.28299999999999997,"15min":0.27800000000000002},"max":{"1min":0.66300000000000003,"5min":9.6110000000000007,"15min":9.6110000000000007},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.65400000000000003,"15min":0.59999999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.72699999999999998,"5min":6.6950000000000003,"15min":6.6950000000000003},"last":0.32800000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:46:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.622,"15min":0.59099999999999997},"min":{"1min":0.34000000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.83099999999999996,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.59799999999999998,"15min":0.58899999999999997},"min":{"1min":0.34799999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74399999999999999,"5min":5.8440000000000003,"15min":5.8440000000000003},"last":0.34499999999999997}]},{"osd":3,"last update":"Fri Feb 23 00:46:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.65500000000000003,"15min":0.60399999999999998},"min":{"1min":0.35899999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80700000000000005,"5min":6.5540000000000003,"15min":6.5540000000000003},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.65900000000000003,"15min":0.60599999999999998},"min":{"1min":0.33100000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78000000000000003,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":0.39800000000000002}]},{"osd":5,"last update":"Fri Feb 23 00:46:25 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.41499999999999998,"15min":0.39000000000000001},"min":{"1min":0.28000000000000003,"5min":0.252,"15min":0.246},"max":{"1min":0.67200000000000004,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.48099999999999998,"15min":0.442},"min":{"1min":0.29999999999999999,"5min":0.29599999999999999,"15min":0.28899999999999998},"max":{"1min":0.71999999999999997,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.40699999999999997}]},{"osd":6,"last update":"Fri Feb 23 00:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.45300000000000001,"15min":0.41699999999999998},"min":{"1min":0.28699999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.68500000000000005,"5min":0.75,"15min":0.75},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45500000000000002,"15min":0.41699999999999998},"min":{"1min":0.27000000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.67100000000000004,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.36899999999999999}]},{"osd":7,"last update":"Fri Feb 23 00:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.47699999999999998,"15min":0.48299999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.254},"max":{"1min":0.77800000000000002,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.52900000000000003,"15min":0.53500000000000003},"min":{"1min":0.34699999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78800000000000003,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.44600000000000001}]}]},{"osd":5,"up_from":39,"seq":167503724648,"num_pgs":76,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":314832,"kb_used_data":5316,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93430320,"statfs":{"total":95995035648,"available":95672647680,"internally_reserved":0,"allocated":5443584,"data_stored":3318259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.78200000000000003,"15min":0.59999999999999998},"min":{"1min":0.33200000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.68500000000000005,"5min":23.881,"15min":23.881},"last":0.371},{"interface":"front","average":{"1min":0.5,"5min":0.68799999999999994,"15min":0.57699999999999996},"min":{"1min":0.40100000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73899999999999999,"5min":5.4649999999999999,"15min":5.4649999999999999},"last":0.35499999999999998}]},{"osd":1,"last update":"Fri Feb 23 00:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.89400000000000002,"15min":0.64900000000000002},"min":{"1min":0.318,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.89600000000000002,"5min":18.175999999999998,"15min":18.175999999999998},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.78300000000000003,"15min":0.62},"min":{"1min":0.32300000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.70699999999999996,"5min":9.0950000000000006,"15min":9.0950000000000006},"last":0.29099999999999998}]},{"osd":2,"last update":"Fri Feb 23 00:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":1.018,"15min":0.70299999999999996},"min":{"1min":0.34100000000000003,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70599999999999996,"5min":22.282,"15min":22.282},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":1.0509999999999999,"15min":0.70899999999999996},"min":{"1min":0.378,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77200000000000002,"5min":23.925000000000001,"15min":23.925000000000001},"last":0.46999999999999997}]},{"osd":3,"last update":"Fri Feb 23 00:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.78100000000000003,"15min":0.61299999999999999},"min":{"1min":0.38500000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.77900000000000003,"5min":6.5730000000000004,"15min":6.5730000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.80500000000000005,"15min":0.64100000000000001},"min":{"1min":0.40699999999999997,"5min":0.33600000000000002,"15min":0.33500000000000002},"max":{"1min":0.753,"5min":8.1760000000000002,"15min":8.1760000000000002},"last":0.40799999999999997}]},{"osd":4,"last update":"Fri Feb 23 00:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.42999999999999999,"15min":0.40100000000000002},"min":{"1min":0.29499999999999998,"5min":0.247,"15min":0.247},"max":{"1min":0.64600000000000002,"5min":0.68999999999999995,"15min":0.70399999999999996},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.51600000000000001,"15min":0.436},"min":{"1min":0.27700000000000002,"5min":0.27500000000000002,"15min":0.27300000000000002},"max":{"1min":0.67000000000000004,"5min":3.879,"15min":3.879},"last":0.33400000000000002}]},{"osd":6,"last update":"Fri Feb 23 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.48899999999999999,"15min":0.44600000000000001},"min":{"1min":0.30599999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70999999999999996,"5min":1.736,"15min":1.736},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.49199999999999999,"15min":0.45900000000000002},"min":{"1min":0.311,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73099999999999998,"5min":0.871,"15min":0.871},"last":0.436}]},{"osd":7,"last update":"Fri Feb 23 00:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.57699999999999996,"15min":0.53400000000000003},"min":{"1min":0.34799999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.753,"5min":7.0300000000000002,"15min":7.0300000000000002},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.58299999999999996,"15min":0.53300000000000003},"min":{"1min":0.33700000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.68999999999999995,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":0.314}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4132864,"data_stored":4000334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":274432,"data_stored":647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":172032,"data_stored":417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4198400,"data_stored":4000501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":192512,"data_stored":484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4235264,"data_stored":4000566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":190,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":190,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":190,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":190,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":191,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":191,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":191,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":191,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":191,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":191,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":191,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-23T00:46:41.016 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:46:41.027 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:46:41.082 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:46:41.108 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:41.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:39.375401+0000 osd.6 (osd.6) 129 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:46:41.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:39.375644+0000 osd.6 (osd.6) 130 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:46:41.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:40.170939+0000 mon.a (mon.0) 2161 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-43","var": "hit_set_count","val": "3"}]': finished 2024-02-23T00:46:41.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:40.171012+0000 mon.a (mon.0) 2162 : audit [INF] from='client.? 172.21.15.47:0/427115143' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi047-82386-55"}]': finished 2024-02-23T00:46:41.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:40.171066+0000 mon.a (mon.0) 2163 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "18"}]': finished 2024-02-23T00:46:41.311 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:40.171111+0000 mon.a (mon.0) 2164 : cluster [DBG] osdmap e328: 8 total, 8 up, 8 in 2024-02-23T00:46:41.311 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:40.174950+0000 mon.a (mon.0) 2165 : audit [INF] from='client.? 172.21.15.47:0/427115143' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi047-82386-55"}]: dispatch 2024-02-23T00:46:41.311 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:40.178259+0000 mon.a (mon.0) 2166 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-43","var": "hit_set_period","val": "3"}]: dispatch 2024-02-23T00:46:41.311 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:40.184341+0000 mon.b (mon.1) 362 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:41.311 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:40.377849+0000 osd.4 (osd.4) 135 : cluster [DBG] 62.5 scrub starts 2024-02-23T00:46:41.311 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:40.379154+0000 osd.4 (osd.4) 136 : cluster [DBG] 62.5 scrub ok 2024-02-23T00:46:41.311 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:41.172949+0000 mon.a (mon.0) 2167 : audit [INF] from='client.? 172.21.15.47:0/427115143' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi047-82386-55"}]': finished 2024-02-23T00:46:41.312 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:41.173028+0000 mon.a (mon.0) 2168 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-43","var": "hit_set_period","val": "3"}]': finished 2024-02-23T00:46:41.312 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:41.173093+0000 mon.a (mon.0) 2169 : cluster [DBG] osdmap e329: 8 total, 8 up, 8 in 2024-02-23T00:46:41.312 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:41.174174+0000 mon.a (mon.0) 2170 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-43","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-23T00:46:41.312 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:39.375401+0000 osd.6 (osd.6) 129 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:46:41.312 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:39.375644+0000 osd.6 (osd.6) 130 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:46:41.312 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:40.170939+0000 mon.a (mon.0) 2161 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-43","var": "hit_set_count","val": "3"}]': finished 2024-02-23T00:46:41.312 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:40.171012+0000 mon.a (mon.0) 2162 : audit [INF] from='client.? 172.21.15.47:0/427115143' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi047-82386-55"}]': finished 2024-02-23T00:46:41.312 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:40.171066+0000 mon.a (mon.0) 2163 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "18"}]': finished 2024-02-23T00:46:41.312 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:40.171111+0000 mon.a (mon.0) 2164 : cluster [DBG] osdmap e328: 8 total, 8 up, 8 in 2024-02-23T00:46:41.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:40.174950+0000 mon.a (mon.0) 2165 : audit [INF] from='client.? 172.21.15.47:0/427115143' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi047-82386-55"}]: dispatch 2024-02-23T00:46:41.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:40.178259+0000 mon.a (mon.0) 2166 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-43","var": "hit_set_period","val": "3"}]: dispatch 2024-02-23T00:46:41.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:40.184341+0000 mon.b (mon.1) 362 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:41.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:40.377849+0000 osd.4 (osd.4) 135 : cluster [DBG] 62.5 scrub starts 2024-02-23T00:46:41.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:40.379154+0000 osd.4 (osd.4) 136 : cluster [DBG] 62.5 scrub ok 2024-02-23T00:46:41.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:41.172949+0000 mon.a (mon.0) 2167 : audit [INF] from='client.? 172.21.15.47:0/427115143' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi047-82386-55"}]': finished 2024-02-23T00:46:41.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:41.173028+0000 mon.a (mon.0) 2168 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-43","var": "hit_set_period","val": "3"}]': finished 2024-02-23T00:46:41.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:41.173093+0000 mon.a (mon.0) 2169 : cluster [DBG] osdmap e329: 8 total, 8 up, 8 in 2024-02-23T00:46:41.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:41.174174+0000 mon.a (mon.0) 2170 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-43","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-23T00:46:41.356 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:46:41.359 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:39.375401+0000 osd.6 (osd.6) 129 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:46:41.359 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:39.375644+0000 osd.6 (osd.6) 130 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:46:41.359 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:40.170939+0000 mon.a (mon.0) 2161 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-43","var": "hit_set_count","val": "3"}]': finished 2024-02-23T00:46:41.359 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:40.171012+0000 mon.a (mon.0) 2162 : audit [INF] from='client.? 172.21.15.47:0/427115143' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi047-82386-55"}]': finished 2024-02-23T00:46:41.359 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:40.171066+0000 mon.a (mon.0) 2163 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "18"}]': finished 2024-02-23T00:46:41.359 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:40.171111+0000 mon.a (mon.0) 2164 : cluster [DBG] osdmap e328: 8 total, 8 up, 8 in 2024-02-23T00:46:41.359 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:40.174950+0000 mon.a (mon.0) 2165 : audit [INF] from='client.? 172.21.15.47:0/427115143' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi047-82386-55"}]: dispatch 2024-02-23T00:46:41.359 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:40.178259+0000 mon.a (mon.0) 2166 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-43","var": "hit_set_period","val": "3"}]: dispatch 2024-02-23T00:46:41.359 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:40.184341+0000 mon.b (mon.1) 362 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:41.359 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:40.377849+0000 osd.4 (osd.4) 135 : cluster [DBG] 62.5 scrub starts 2024-02-23T00:46:41.359 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:40.379154+0000 osd.4 (osd.4) 136 : cluster [DBG] 62.5 scrub ok 2024-02-23T00:46:41.359 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:41.172949+0000 mon.a (mon.0) 2167 : audit [INF] from='client.? 172.21.15.47:0/427115143' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi047-82386-55"}]': finished 2024-02-23T00:46:41.360 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:41.173028+0000 mon.a (mon.0) 2168 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-43","var": "hit_set_period","val": "3"}]': finished 2024-02-23T00:46:41.360 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:41.173093+0000 mon.a (mon.0) 2169 : cluster [DBG] osdmap e329: 8 total, 8 up, 8 in 2024-02-23T00:46:41.360 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:41.174174+0000 mon.a (mon.0) 2170 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-43","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-23T00:46:41.360 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:41.361 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:46:41.362 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:46:41.462 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:46:41.490 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:41.944 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:46:41.947 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:41.947 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:46:41.989 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:46:42.048 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:46:42.077 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:42.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:40.350565+0000 osd.1 (osd.1) 147 : cluster [DBG] 18.1c scrub starts 2024-02-23T00:46:42.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:40.353994+0000 osd.1 (osd.1) 148 : cluster [DBG] 18.1c scrub ok 2024-02-23T00:46:42.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:41.185688+0000 mon.b (mon.1) 363 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:42.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:41.195222+0000 mon.a (mon.0) 2171 : audit [INF] from='client.? 172.21.15.47:0/2392849431' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi047-82386-56"}]: dispatch 2024-02-23T00:46:42.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:41.196123+0000 mon.a (mon.0) 2172 : audit [INF] from='client.? 172.21.15.47:0/2392849431' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi047-82386-56"}]: dispatch 2024-02-23T00:46:42.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:41.196439+0000 mon.a (mon.0) 2173 : audit [INF] from='client.? 172.21.15.47:0/2392849431' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi047-82386-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:42.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:41.261724+0000 mgr.y (mgr.14517) 321 : cluster [DBG] pgmap v469: 237 pgs: 1 activating, 32 creating+peering, 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:46:42.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:41.288107+0000 osd.6 (osd.6) 131 : cluster [DBG] 4.a scrub starts 2024-02-23T00:46:42.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:42.174144+0000 mon.a (mon.0) 2174 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-43","var": "hit_set_type","val": "bloom"}]': finished 2024-02-23T00:46:42.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:42.174228+0000 mon.a (mon.0) 2175 : audit [INF] from='client.? 172.21.15.47:0/2392849431' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi047-82386-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:42.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:42.174300+0000 mon.a (mon.0) 2176 : cluster [DBG] osdmap e330: 8 total, 8 up, 8 in 2024-02-23T00:46:42.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:42.177522+0000 mon.a (mon.0) 2177 : audit [INF] from='client.? 172.21.15.47:0/2392849431' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi047-82386-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi047-82386-56"}]: dispatch 2024-02-23T00:46:42.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:42.179542+0000 mon.a (mon.0) 2178 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-43","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-23T00:46:42.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:40.350565+0000 osd.1 (osd.1) 147 : cluster [DBG] 18.1c scrub starts 2024-02-23T00:46:42.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:40.353994+0000 osd.1 (osd.1) 148 : cluster [DBG] 18.1c scrub ok 2024-02-23T00:46:42.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:41.185688+0000 mon.b (mon.1) 363 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:42.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:41.195222+0000 mon.a (mon.0) 2171 : audit [INF] from='client.? 172.21.15.47:0/2392849431' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi047-82386-56"}]: dispatch 2024-02-23T00:46:42.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:41.196123+0000 mon.a (mon.0) 2172 : audit [INF] from='client.? 172.21.15.47:0/2392849431' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi047-82386-56"}]: dispatch 2024-02-23T00:46:42.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:41.196439+0000 mon.a (mon.0) 2173 : audit [INF] from='client.? 172.21.15.47:0/2392849431' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi047-82386-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:42.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:41.261724+0000 mgr.y (mgr.14517) 321 : cluster [DBG] pgmap v469: 237 pgs: 1 activating, 32 creating+peering, 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:46:42.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:41.288107+0000 osd.6 (osd.6) 131 : cluster [DBG] 4.a scrub starts 2024-02-23T00:46:42.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:42.174144+0000 mon.a (mon.0) 2174 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-43","var": "hit_set_type","val": "bloom"}]': finished 2024-02-23T00:46:42.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:42.174228+0000 mon.a (mon.0) 2175 : audit [INF] from='client.? 172.21.15.47:0/2392849431' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi047-82386-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:42.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:42.174300+0000 mon.a (mon.0) 2176 : cluster [DBG] osdmap e330: 8 total, 8 up, 8 in 2024-02-23T00:46:42.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:42.177522+0000 mon.a (mon.0) 2177 : audit [INF] from='client.? 172.21.15.47:0/2392849431' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi047-82386-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi047-82386-56"}]: dispatch 2024-02-23T00:46:42.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:42.179542+0000 mon.a (mon.0) 2178 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-43","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-23T00:46:42.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:40.350565+0000 osd.1 (osd.1) 147 : cluster [DBG] 18.1c scrub starts 2024-02-23T00:46:42.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:40.353994+0000 osd.1 (osd.1) 148 : cluster [DBG] 18.1c scrub ok 2024-02-23T00:46:42.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:41.185688+0000 mon.b (mon.1) 363 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:42.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:41.195222+0000 mon.a (mon.0) 2171 : audit [INF] from='client.? 172.21.15.47:0/2392849431' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi047-82386-56"}]: dispatch 2024-02-23T00:46:42.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:41.196123+0000 mon.a (mon.0) 2172 : audit [INF] from='client.? 172.21.15.47:0/2392849431' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi047-82386-56"}]: dispatch 2024-02-23T00:46:42.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:41.196439+0000 mon.a (mon.0) 2173 : audit [INF] from='client.? 172.21.15.47:0/2392849431' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi047-82386-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:42.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:41.261724+0000 mgr.y (mgr.14517) 321 : cluster [DBG] pgmap v469: 237 pgs: 1 activating, 32 creating+peering, 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:46:42.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:41.288107+0000 osd.6 (osd.6) 131 : cluster [DBG] 4.a scrub starts 2024-02-23T00:46:42.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:42.174144+0000 mon.a (mon.0) 2174 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-43","var": "hit_set_type","val": "bloom"}]': finished 2024-02-23T00:46:42.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:42.174228+0000 mon.a (mon.0) 2175 : audit [INF] from='client.? 172.21.15.47:0/2392849431' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi047-82386-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:42.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:42.174300+0000 mon.a (mon.0) 2176 : cluster [DBG] osdmap e330: 8 total, 8 up, 8 in 2024-02-23T00:46:42.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:42.177522+0000 mon.a (mon.0) 2177 : audit [INF] from='client.? 172.21.15.47:0/2392849431' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi047-82386-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi047-82386-56"}]: dispatch 2024-02-23T00:46:42.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:42.179542+0000 mon.a (mon.0) 2178 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-43","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-23T00:46:42.944 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:46:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:46:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:46:42.944 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:46:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:46:43.228 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:46:43.230 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:46:43.230 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:46:43.231 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:43.232 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:46:43.333 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:46:43.362 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:43.510 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:46:43.512 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:43.513 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:46:43.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.466+0000 7f78f6953700 1 -- 172.21.15.47:0/235387604 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f78f00fdbb0 msgr2=0x7f78f00fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:43.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.466+0000 7f78f6953700 1 --2- 172.21.15.47:0/235387604 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f78f00fdbb0 0x7f78f00fdf90 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 rx=0x7f78e0004870 tx=0x7f78e0009cb0).stop 2024-02-23T00:46:43.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.466+0000 7f78f6953700 1 -- 172.21.15.47:0/235387604 shutdown_connections 2024-02-23T00:46:43.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.466+0000 7f78f6953700 1 --2- 172.21.15.47:0/235387604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78f00f8420 0x7f78f0104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:43.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.466+0000 7f78f6953700 1 --2- 172.21.15.47:0/235387604 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f78f00fdbb0 0x7f78f00fdf90 unknown :-1 s=CLOSED pgs=2619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:43.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.466+0000 7f78f6953700 1 --2- 172.21.15.47:0/235387604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78f00fe510 0x7f78f00f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:43.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.466+0000 7f78f6953700 1 -- 172.21.15.47:0/235387604 >> 172.21.15.47:0/235387604 conn(0x7f78f00f3800 msgr2=0x7f78f00f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:46:43.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.466+0000 7f78f6953700 1 -- 172.21.15.47:0/235387604 shutdown_connections 2024-02-23T00:46:43.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.466+0000 7f78f6953700 1 -- 172.21.15.47:0/235387604 wait complete. 2024-02-23T00:46:43.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.467+0000 7f78f6953700 1 librados: starting msgr at 2024-02-23T00:46:43.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.467+0000 7f78f6953700 1 librados: starting objecter 2024-02-23T00:46:43.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.467+0000 7f78f6953700 1 Processor -- start 2024-02-23T00:46:43.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.467+0000 7f78f6953700 1 -- start start 2024-02-23T00:46:43.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.467+0000 7f78f6953700 1 librados: setting wanted keys 2024-02-23T00:46:43.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.467+0000 7f78f6953700 1 librados: calling monclient init 2024-02-23T00:46:43.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.467+0000 7f78f6953700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f78f00f8420 0x7f78f01100d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:43.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.467+0000 7f78f6953700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78f00fdbb0 0x7f78f01105c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:43.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.467+0000 7f78f6953700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78f00fe510 0x7f78f010a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:43.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.467+0000 7f78f6953700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f010a760 con 0x7f78f00fdbb0 2024-02-23T00:46:43.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.467+0000 7f78f6953700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f010a8d0 con 0x7f78f00fe510 2024-02-23T00:46:43.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.467+0000 7f78f6953700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f010aa90 con 0x7f78f00f8420 2024-02-23T00:46:43.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.467+0000 7f78effff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f78f00f8420 0x7f78f01100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:43.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.467+0000 7f78ef7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78f00fdbb0 0x7f78f01105c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:43.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.467+0000 7f78effff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f78f00f8420 0x7f78f01100d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:40460/0 (socket says 172.21.15.47:40460) 2024-02-23T00:46:43.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.467+0000 7f78effff700 1 -- 172.21.15.47:0/3447852807 learned_addr learned my addr 172.21.15.47:0/3447852807 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:46:43.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.467+0000 7f78ef7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78f00fdbb0 0x7f78f01105c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57030/0 (socket says 172.21.15.47:57030) 2024-02-23T00:46:43.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.467+0000 7f78f4ef0700 1 --2- 172.21.15.47:0/3447852807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78f00fe510 0x7f78f010a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:43.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.468+0000 7f78effff700 1 -- 172.21.15.47:0/3447852807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78f00fe510 msgr2=0x7f78f010a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:43.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.468+0000 7f78effff700 1 --2- 172.21.15.47:0/3447852807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78f00fe510 0x7f78f010a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:43.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.468+0000 7f78effff700 1 -- 172.21.15.47:0/3447852807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78f00fdbb0 msgr2=0x7f78f01105c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:43.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.468+0000 7f78effff700 1 --2- 172.21.15.47:0/3447852807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78f00fdbb0 0x7f78f01105c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:43.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.468+0000 7f78effff700 1 -- 172.21.15.47:0/3447852807 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e0005040 con 0x7f78f00f8420 2024-02-23T00:46:43.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.468+0000 7f78effff700 1 --2- 172.21.15.47:0/3447852807 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f78f00f8420 0x7f78f01100d0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 rx=0x7f78e0013680 tx=0x7f78e0009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:43.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.468+0000 7f78ed7fa700 10 client.?.objecter ms_handle_connect 0x7f78f00f8420 2024-02-23T00:46:43.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.468+0000 7f78ed7fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:46:43.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.468+0000 7f78ed7fa700 1 -- 172.21.15.47:0/3447852807 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e000e030 con 0x7f78f00f8420 2024-02-23T00:46:43.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.468+0000 7f78ed7fa700 1 -- 172.21.15.47:0/3447852807 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f78e0013e30 con 0x7f78f00f8420 2024-02-23T00:46:43.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.468+0000 7f78f6953700 1 -- 172.21.15.47:0/3447852807 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f010ad10 con 0x7f78f00f8420 2024-02-23T00:46:43.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.468+0000 7f78ed7fa700 1 -- 172.21.15.47:0/3447852807 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e0018070 con 0x7f78f00f8420 2024-02-23T00:46:43.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.468+0000 7f78f6953700 10 client.41129.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:46:43.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.468+0000 7f78f6953700 1 -- 172.21.15.47:0/3447852807 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f0116f00 con 0x7f78f00f8420 2024-02-23T00:46:43.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.468+0000 7f78f6953700 1 librados: init done 2024-02-23T00:46:43.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.469+0000 7f78f6953700 1 -- 172.21.15.47:0/3447852807 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f005ee80 con 0x7f78f00f8420 2024-02-23T00:46:43.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.470+0000 7f78ed7fa700 1 -- 172.21.15.47:0/3447852807 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f78e0009240 con 0x7f78f00f8420 2024-02-23T00:46:43.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.470+0000 7f78ed7fa700 1 --2- 172.21.15.47:0/3447852807 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f78d8062cc0 0x7f78d8065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:43.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.470+0000 7f78ed7fa700 1 -- 172.21.15.47:0/3447852807 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(331..331 src has 48..331) v4 ==== 9976+0+0 (secure 0 0 0) 0x7f78e007f1c0 con 0x7f78f00f8420 2024-02-23T00:46:43.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.470+0000 7f78ed7fa700 10 client.41129.objecter ms_dispatch 0x7f78f0001550 osd_map(331..331 src has 48..331) v4 2024-02-23T00:46:43.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.470+0000 7f78ed7fa700 3 client.41129.objecter handle_osd_map got epochs [331,331] > 0 2024-02-23T00:46:43.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.470+0000 7f78ed7fa700 3 client.41129.objecter handle_osd_map decoding full epoch 331 2024-02-23T00:46:43.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.470+0000 7f78ef7fe700 1 --2- 172.21.15.47:0/3447852807 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f78d8062cc0 0x7f78d8065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:43.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.470+0000 7f78ed7fa700 20 client.41129.objecter dump_active .. 0 homeless 2024-02-23T00:46:43.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.471+0000 7f78ed7fa700 10 client.41129.objecter ms_handle_connect 0x7f78d8062cc0 2024-02-23T00:46:43.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.471+0000 7f78ef7fe700 1 --2- 172.21.15.47:0/3447852807 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f78d8062cc0 0x7f78d8065180 secure :-1 s=READY pgs=3027 cs=0 l=1 rev1=1 rx=0x7f78f010ba10 tx=0x7f78e400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:43.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.474+0000 7f78ed7fa700 1 -- 172.21.15.47:0/3447852807 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f78e004e560 con 0x7f78f00f8420 2024-02-23T00:46:43.614 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:46:43.649 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:43.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:43.684+0000 7f78f6953700 1 -- 172.21.15.47:0/3447852807 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7f78f0106f20 con 0x7f78f00f8420 2024-02-23T00:46:43.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:41.290473+0000 osd.6 (osd.6) 132 : cluster [DBG] 4.a scrub ok 2024-02-23T00:46:43.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:42.187528+0000 mon.b (mon.1) 364 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:43.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:42.325673+0000 osd.4 (osd.4) 137 : cluster [DBG] 6.e scrub starts 2024-02-23T00:46:43.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:42.326561+0000 osd.4 (osd.4) 138 : cluster [DBG] 6.e scrub ok 2024-02-23T00:46:43.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:42.515672+0000 mgr.y (mgr.14517) 322 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:46:43.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:41.290473+0000 osd.6 (osd.6) 132 : cluster [DBG] 4.a scrub ok 2024-02-23T00:46:43.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:42.187528+0000 mon.b (mon.1) 364 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:43.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:42.325673+0000 osd.4 (osd.4) 137 : cluster [DBG] 6.e scrub starts 2024-02-23T00:46:43.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:42.326561+0000 osd.4 (osd.4) 138 : cluster [DBG] 6.e scrub ok 2024-02-23T00:46:43.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:42.515672+0000 mgr.y (mgr.14517) 322 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:46:43.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:41.290473+0000 osd.6 (osd.6) 132 : cluster [DBG] 4.a scrub ok 2024-02-23T00:46:43.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:42.187528+0000 mon.b (mon.1) 364 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:43.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:42.325673+0000 osd.4 (osd.4) 137 : cluster [DBG] 6.e scrub starts 2024-02-23T00:46:43.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:42.326561+0000 osd.4 (osd.4) 138 : cluster [DBG] 6.e scrub ok 2024-02-23T00:46:43.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:42.515672+0000 mgr.y (mgr.14517) 322 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:46:43.993 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.4 dump_blocked_ops 2024-02-23T00:46:44.016 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph pg dump --format=json 2024-02-23T00:46:44.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:44.180+0000 7f78ed7fa700 1 -- 172.21.15.47:0/3447852807 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v332) v1 ==== 101+0+0 (secure 0 0 0) 0x7f78e0051b80 con 0x7f78f00f8420 2024-02-23T00:46:44.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:44.182+0000 7f78f6953700 10 librados: watch_flush enter 2024-02-23T00:46:44.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:44.183+0000 7f78f6953700 10 librados: watch_flush exit 2024-02-23T00:46:44.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:44.183+0000 7f78f6953700 20 client.41129.objecter shutdown clearing up homeless session... 2024-02-23T00:46:44.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:44.183+0000 7f78f6953700 10 client.41129.objecter successfully canceled tick 2024-02-23T00:46:44.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:44.183+0000 7f78f6953700 1 -- 172.21.15.47:0/3447852807 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f78d8062cc0 msgr2=0x7f78d8065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:44.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:44.183+0000 7f78f6953700 1 --2- 172.21.15.47:0/3447852807 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f78d8062cc0 0x7f78d8065180 secure :-1 s=READY pgs=3027 cs=0 l=1 rev1=1 rx=0x7f78f010ba10 tx=0x7f78e400b040).stop 2024-02-23T00:46:44.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:44.184+0000 7f78f6953700 1 -- 172.21.15.47:0/3447852807 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f78f00f8420 msgr2=0x7f78f01100d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:44.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:44.184+0000 7f78f6953700 1 --2- 172.21.15.47:0/3447852807 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f78f00f8420 0x7f78f01100d0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 rx=0x7f78e0013680 tx=0x7f78e0009cb0).stop 2024-02-23T00:46:44.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:44.184+0000 7f78f6953700 1 -- 172.21.15.47:0/3447852807 shutdown_connections 2024-02-23T00:46:44.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:44.184+0000 7f78f6953700 1 --2- 172.21.15.47:0/3447852807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78f00fe510 0x7f78f010a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:44.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:44.184+0000 7f78f6953700 1 --2- 172.21.15.47:0/3447852807 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f78d8062cc0 0x7f78d8065180 unknown :-1 s=CLOSED pgs=3027 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:44.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:44.184+0000 7f78f6953700 1 --2- 172.21.15.47:0/3447852807 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f78f00f8420 0x7f78f01100d0 unknown :-1 s=CLOSED pgs=2620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:44.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:44.184+0000 7f78f6953700 1 --2- 172.21.15.47:0/3447852807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78f00fdbb0 0x7f78f01105c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:44.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:44.184+0000 7f78f6953700 1 -- 172.21.15.47:0/3447852807 >> 172.21.15.47:0/3447852807 conn(0x7f78f00f3800 msgr2=0x7f78f01032c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:46:44.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:44.185+0000 7f78f6953700 1 -- 172.21.15.47:0/3447852807 shutdown_connections 2024-02-23T00:46:44.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:44.186+0000 7f78f6953700 1 -- 172.21.15.47:0/3447852807 wait complete. 2024-02-23T00:46:44.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:44.187+0000 7f78f6953700 1 librados: shutdown 2024-02-23T00:46:44.190 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is unset 2024-02-23T00:46:44.263 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:46:44.731 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:46:44.734 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:44.735 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:46:44.835 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:46:44.853 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:42.255069+0000 osd.5 (osd.5) 140 : cluster [DBG] 5.16 scrub starts 2024-02-23T00:46:44.853 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:42.257955+0000 osd.5 (osd.5) 141 : cluster [DBG] 5.16 scrub ok 2024-02-23T00:46:44.853 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:42.328331+0000 osd.1 (osd.1) 149 : cluster [DBG] 18.11 scrub starts 2024-02-23T00:46:44.854 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:42.331543+0000 osd.1 (osd.1) 150 : cluster [DBG] 18.11 scrub ok 2024-02-23T00:46:44.854 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:42.691428+0000 osd.0 (osd.0) 137 : cluster [DBG] 2.13 scrub starts 2024-02-23T00:46:44.854 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:42.692425+0000 osd.0 (osd.0) 138 : cluster [DBG] 2.13 scrub ok 2024-02-23T00:46:44.854 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:42.937332+0000 osd.3 (osd.3) 159 : cluster [DBG] 6.6 scrub starts 2024-02-23T00:46:44.854 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:42.938200+0000 osd.3 (osd.3) 160 : cluster [DBG] 6.6 scrub ok 2024-02-23T00:46:44.854 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:43.189413+0000 mon.b (mon.1) 365 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:44.854 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:43.262105+0000 mgr.y (mgr.14517) 323 : cluster [DBG] pgmap v472: 236 pgs: 1 clean+premerge+peered, 235 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s; 31 B/s, 0 objects/s recovering 2024-02-23T00:46:44.854 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:43.620175+0000 mon.a (mon.0) 2179 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-43","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-23T00:46:44.854 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:43.620266+0000 mon.a (mon.0) 2180 : cluster [DBG] osdmap e331: 8 total, 8 up, 8 in 2024-02-23T00:46:44.854 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:43.627388+0000 mon.a (mon.0) 2181 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "17"}]: dispatch 2024-02-23T00:46:44.854 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:43.685508+0000 mon.c (mon.2) 249 : audit [INF] from='client.? 172.21.15.47:0/3447852807' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:46:44.854 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:43.685838+0000 mon.a (mon.0) 2182 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:46:44.854 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:43.718987+0000 osd.7 (osd.7) 169 : cluster [DBG] 2.8 scrub starts 2024-02-23T00:46:44.854 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:43.722476+0000 osd.7 (osd.7) 170 : cluster [DBG] 2.8 scrub ok 2024-02-23T00:46:44.855 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:43.944186+0000 osd.3 (osd.3) 161 : cluster [DBG] 6.13 scrub starts 2024-02-23T00:46:44.855 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:43.945015+0000 osd.3 (osd.3) 162 : cluster [DBG] 6.13 scrub ok 2024-02-23T00:46:44.855 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:44.173247+0000 mon.a (mon.0) 2183 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:46:44.855 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:44.179684+0000 mon.a (mon.0) 2184 : audit [INF] from='client.? 172.21.15.47:0/2392849431' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi047-82386-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi047-82386-56"}]': finished 2024-02-23T00:46:44.855 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:44.179781+0000 mon.a (mon.0) 2185 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "17"}]': finished 2024-02-23T00:46:44.855 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:44.179830+0000 mon.a (mon.0) 2186 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:46:44.855 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:44.179877+0000 mon.a (mon.0) 2187 : cluster [DBG] osdmap e332: 8 total, 8 up, 8 in 2024-02-23T00:46:44.855 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:44.190708+0000 mon.b (mon.1) 366 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:44.855 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:44.283852+0000 osd.6 (osd.6) 133 : cluster [DBG] 2.17 deep-scrub starts 2024-02-23T00:46:44.855 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:44.285074+0000 osd.6 (osd.6) 134 : cluster [DBG] 2.17 deep-scrub ok 2024-02-23T00:46:44.864 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:44.880 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:42.255069+0000 osd.5 (osd.5) 140 : cluster [DBG] 5.16 scrub starts 2024-02-23T00:46:44.880 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:42.257955+0000 osd.5 (osd.5) 141 : cluster [DBG] 5.16 scrub ok 2024-02-23T00:46:44.880 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:42.328331+0000 osd.1 (osd.1) 149 : cluster [DBG] 18.11 scrub starts 2024-02-23T00:46:44.880 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:42.331543+0000 osd.1 (osd.1) 150 : cluster [DBG] 18.11 scrub ok 2024-02-23T00:46:44.880 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:42.691428+0000 osd.0 (osd.0) 137 : cluster [DBG] 2.13 scrub starts 2024-02-23T00:46:44.880 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:42.692425+0000 osd.0 (osd.0) 138 : cluster [DBG] 2.13 scrub ok 2024-02-23T00:46:44.880 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:42.937332+0000 osd.3 (osd.3) 159 : cluster [DBG] 6.6 scrub starts 2024-02-23T00:46:44.880 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:42.938200+0000 osd.3 (osd.3) 160 : cluster [DBG] 6.6 scrub ok 2024-02-23T00:46:44.880 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:43.189413+0000 mon.b (mon.1) 365 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:44.880 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:43.262105+0000 mgr.y (mgr.14517) 323 : cluster [DBG] pgmap v472: 236 pgs: 1 clean+premerge+peered, 235 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s; 31 B/s, 0 objects/s recovering 2024-02-23T00:46:44.881 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:43.620175+0000 mon.a (mon.0) 2179 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-43","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-23T00:46:44.881 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:43.620266+0000 mon.a (mon.0) 2180 : cluster [DBG] osdmap e331: 8 total, 8 up, 8 in 2024-02-23T00:46:44.881 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:43.627388+0000 mon.a (mon.0) 2181 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "17"}]: dispatch 2024-02-23T00:46:44.881 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:43.685508+0000 mon.c (mon.2) 249 : audit [INF] from='client.? 172.21.15.47:0/3447852807' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:46:44.881 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:43.685838+0000 mon.a (mon.0) 2182 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:46:44.881 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:43.718987+0000 osd.7 (osd.7) 169 : cluster [DBG] 2.8 scrub starts 2024-02-23T00:46:44.881 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:43.722476+0000 osd.7 (osd.7) 170 : cluster [DBG] 2.8 scrub ok 2024-02-23T00:46:44.881 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:43.944186+0000 osd.3 (osd.3) 161 : cluster [DBG] 6.13 scrub starts 2024-02-23T00:46:44.881 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:43.945015+0000 osd.3 (osd.3) 162 : cluster [DBG] 6.13 scrub ok 2024-02-23T00:46:44.881 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:44.173247+0000 mon.a (mon.0) 2183 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:46:44.881 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:44.179684+0000 mon.a (mon.0) 2184 : audit [INF] from='client.? 172.21.15.47:0/2392849431' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi047-82386-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi047-82386-56"}]': finished 2024-02-23T00:46:44.881 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:44.179781+0000 mon.a (mon.0) 2185 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "17"}]': finished 2024-02-23T00:46:44.881 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:44.179830+0000 mon.a (mon.0) 2186 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:46:44.881 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:44.179877+0000 mon.a (mon.0) 2187 : cluster [DBG] osdmap e332: 8 total, 8 up, 8 in 2024-02-23T00:46:44.882 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:44.190708+0000 mon.b (mon.1) 366 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:44.882 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:44.283852+0000 osd.6 (osd.6) 133 : cluster [DBG] 2.17 deep-scrub starts 2024-02-23T00:46:44.882 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:44.285074+0000 osd.6 (osd.6) 134 : cluster [DBG] 2.17 deep-scrub ok 2024-02-23T00:46:44.882 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:42.255069+0000 osd.5 (osd.5) 140 : cluster [DBG] 5.16 scrub starts 2024-02-23T00:46:44.882 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:42.257955+0000 osd.5 (osd.5) 141 : cluster [DBG] 5.16 scrub ok 2024-02-23T00:46:44.882 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:42.328331+0000 osd.1 (osd.1) 149 : cluster [DBG] 18.11 scrub starts 2024-02-23T00:46:44.882 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:42.331543+0000 osd.1 (osd.1) 150 : cluster [DBG] 18.11 scrub ok 2024-02-23T00:46:44.882 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:42.691428+0000 osd.0 (osd.0) 137 : cluster [DBG] 2.13 scrub starts 2024-02-23T00:46:44.882 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:42.692425+0000 osd.0 (osd.0) 138 : cluster [DBG] 2.13 scrub ok 2024-02-23T00:46:44.882 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:42.937332+0000 osd.3 (osd.3) 159 : cluster [DBG] 6.6 scrub starts 2024-02-23T00:46:44.883 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:42.938200+0000 osd.3 (osd.3) 160 : cluster [DBG] 6.6 scrub ok 2024-02-23T00:46:44.883 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:43.189413+0000 mon.b (mon.1) 365 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:44.883 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:43.262105+0000 mgr.y (mgr.14517) 323 : cluster [DBG] pgmap v472: 236 pgs: 1 clean+premerge+peered, 235 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s; 31 B/s, 0 objects/s recovering 2024-02-23T00:46:44.883 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:43.620175+0000 mon.a (mon.0) 2179 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-43","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-23T00:46:44.883 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:43.620266+0000 mon.a (mon.0) 2180 : cluster [DBG] osdmap e331: 8 total, 8 up, 8 in 2024-02-23T00:46:44.883 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:43.627388+0000 mon.a (mon.0) 2181 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "17"}]: dispatch 2024-02-23T00:46:44.883 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:43.685508+0000 mon.c (mon.2) 249 : audit [INF] from='client.? 172.21.15.47:0/3447852807' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:46:44.883 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:43.685838+0000 mon.a (mon.0) 2182 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:46:44.883 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:43.718987+0000 osd.7 (osd.7) 169 : cluster [DBG] 2.8 scrub starts 2024-02-23T00:46:44.883 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:43.722476+0000 osd.7 (osd.7) 170 : cluster [DBG] 2.8 scrub ok 2024-02-23T00:46:44.883 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:43.944186+0000 osd.3 (osd.3) 161 : cluster [DBG] 6.13 scrub starts 2024-02-23T00:46:44.883 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:43.945015+0000 osd.3 (osd.3) 162 : cluster [DBG] 6.13 scrub ok 2024-02-23T00:46:44.883 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:44.173247+0000 mon.a (mon.0) 2183 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:46:44.883 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:44.179684+0000 mon.a (mon.0) 2184 : audit [INF] from='client.? 172.21.15.47:0/2392849431' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi047-82386-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi047-82386-56"}]': finished 2024-02-23T00:46:44.884 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:44.179781+0000 mon.a (mon.0) 2185 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "17"}]': finished 2024-02-23T00:46:44.884 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:44.179830+0000 mon.a (mon.0) 2186 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:46:44.884 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:44.179877+0000 mon.a (mon.0) 2187 : cluster [DBG] osdmap e332: 8 total, 8 up, 8 in 2024-02-23T00:46:44.884 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:44.190708+0000 mon.b (mon.1) 366 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:44.884 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:44.283852+0000 osd.6 (osd.6) 133 : cluster [DBG] 2.17 deep-scrub starts 2024-02-23T00:46:44.884 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:44.285074+0000 osd.6 (osd.6) 134 : cluster [DBG] 2.17 deep-scrub ok 2024-02-23T00:46:44.885 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:46:44.891 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:44.892 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:46:44.992 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:46:45.020 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:45.218 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:46:45.220 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:45.221 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:46:45.322 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:46:45.349 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:45.548 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:46:45.550 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:45.551 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:46:45.651 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:46:45.681 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:45.701 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:46:45.704 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:45.704 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:46:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.794+0000 7f4ff26fb700 1 -- 172.21.15.47:0/3615920133 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4fec0f8420 msgr2=0x7f4fec104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.794+0000 7f4ff26fb700 1 --2- 172.21.15.47:0/3615920133 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4fec0f8420 0x7f4fec104a20 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 rx=0x7f4fe4009c30 tx=0x7f4fe4004c50).stop 2024-02-23T00:46:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.794+0000 7f4ff26fb700 1 -- 172.21.15.47:0/3615920133 shutdown_connections 2024-02-23T00:46:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.794+0000 7f4ff26fb700 1 --2- 172.21.15.47:0/3615920133 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4fec0f8420 0x7f4fec104a20 unknown :-1 s=CLOSED pgs=2621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.794+0000 7f4ff26fb700 1 --2- 172.21.15.47:0/3615920133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fec0fdbb0 0x7f4fec0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.794+0000 7f4ff26fb700 1 --2- 172.21.15.47:0/3615920133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fec0fe510 0x7f4fec0f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.794+0000 7f4ff26fb700 1 -- 172.21.15.47:0/3615920133 >> 172.21.15.47:0/3615920133 conn(0x7f4fec0f3800 msgr2=0x7f4fec0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:46:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.794+0000 7f4ff26fb700 1 -- 172.21.15.47:0/3615920133 shutdown_connections 2024-02-23T00:46:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.794+0000 7f4ff26fb700 1 -- 172.21.15.47:0/3615920133 wait complete. 2024-02-23T00:46:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.794+0000 7f4ff26fb700 1 librados: starting msgr at 2024-02-23T00:46:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.794+0000 7f4ff26fb700 1 librados: starting objecter 2024-02-23T00:46:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.795+0000 7f4ff26fb700 1 Processor -- start 2024-02-23T00:46:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.795+0000 7f4ff26fb700 1 -- start start 2024-02-23T00:46:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.795+0000 7f4ff26fb700 1 librados: setting wanted keys 2024-02-23T00:46:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.795+0000 7f4ff26fb700 1 librados: calling monclient init 2024-02-23T00:46:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.795+0000 7f4ff26fb700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4fec0f8420 0x7f4fec10ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:45.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.795+0000 7f4ff26fb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fec0fdbb0 0x7f4fec110440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:45.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.795+0000 7f4ff26fb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fec0fe510 0x7f4fec10a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:45.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.795+0000 7f4ff26fb700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec10a630 con 0x7f4fec0fe510 2024-02-23T00:46:45.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.795+0000 7f4ff26fb700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec10a7a0 con 0x7f4fec0fdbb0 2024-02-23T00:46:45.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.795+0000 7f4ff26fb700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec10a960 con 0x7f4fec0f8420 2024-02-23T00:46:45.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.795+0000 7f4febfff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4fec0f8420 0x7f4fec10ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:45.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.795+0000 7f4febfff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4fec0f8420 0x7f4fec10ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:40480/0 (socket says 172.21.15.47:40480) 2024-02-23T00:46:45.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.795+0000 7f4febfff700 1 -- 172.21.15.47:0/2372005460 learned_addr learned my addr 172.21.15.47:0/2372005460 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:46:45.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.795+0000 7f4feb7fe700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fec0fdbb0 0x7f4fec110440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:45.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.795+0000 7f4ff0c98700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fec0fe510 0x7f4fec10a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:45.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.795+0000 7f4febfff700 1 -- 172.21.15.47:0/2372005460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fec0fdbb0 msgr2=0x7f4fec110440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:45.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.795+0000 7f4febfff700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fec0fdbb0 0x7f4fec110440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:45.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.796+0000 7f4febfff700 1 -- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fec0fe510 msgr2=0x7f4fec10a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:45.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.796+0000 7f4febfff700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fec0fe510 0x7f4fec10a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.796+0000 7f4febfff700 1 -- 172.21.15.47:0/2372005460 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fdc005040 con 0x7f4fec0f8420 2024-02-23T00:46:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.796+0000 7f4febfff700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4fec0f8420 0x7f4fec10ff50 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 rx=0x7f4fdc009400 tx=0x7f4fdc009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.796+0000 7f4fe97fa700 10 client.?.objecter ms_handle_connect 0x7f4fec0f8420 2024-02-23T00:46:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.796+0000 7f4fe97fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:46:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.796+0000 7f4fe97fa700 1 -- 172.21.15.47:0/2372005460 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc00e030 con 0x7f4fec0f8420 2024-02-23T00:46:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.796+0000 7f4fe97fa700 1 -- 172.21.15.47:0/2372005460 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f4fdc00f040 con 0x7f4fec0f8420 2024-02-23T00:46:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.796+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe40097c0 con 0x7f4fec0f8420 2024-02-23T00:46:45.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.796+0000 7f4fe97fa700 1 -- 172.21.15.47:0/2372005460 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc014730 con 0x7f4fec0f8420 2024-02-23T00:46:45.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.796+0000 7f4ff26fb700 10 client.41135.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:46:45.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.796+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fec116d80 con 0x7f4fec0f8420 2024-02-23T00:46:45.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.796+0000 7f4ff26fb700 1 librados: init done 2024-02-23T00:46:45.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.797+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f4fec109320 con 0x7f4fec0f8420 2024-02-23T00:46:45.805 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:46:45.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.797+0000 7f4fe97fa700 1 -- 172.21.15.47:0/2372005460 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f4fdc010530 con 0x7f4fec0f8420 2024-02-23T00:46:45.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.798+0000 7f4fe97fa700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4fd4062cc0 0x7f4fd4065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.798+0000 7f4fe97fa700 1 -- 172.21.15.47:0/2372005460 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(333..333 src has 48..333) v4 ==== 10379+0+0 (secure 0 0 0) 0x7f4fdc07db20 con 0x7f4fec0f8420 2024-02-23T00:46:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.798+0000 7f4fe97fa700 10 client.41135.objecter ms_dispatch 0x7f4fec001550 osd_map(333..333 src has 48..333) v4 2024-02-23T00:46:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.798+0000 7f4fe97fa700 3 client.41135.objecter handle_osd_map got epochs [333,333] > 0 2024-02-23T00:46:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.798+0000 7f4fe97fa700 3 client.41135.objecter handle_osd_map decoding full epoch 333 2024-02-23T00:46:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.798+0000 7f4feb7fe700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4fd4062cc0 0x7f4fd4065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.798+0000 7f4fe97fa700 20 client.41135.objecter dump_active .. 0 homeless 2024-02-23T00:46:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.798+0000 7f4fe97fa700 1 -- 172.21.15.47:0/2372005460 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v333) v1 ==== 54+0+16 (secure 0 0 0) 0x7f4fdc082220 con 0x7f4fec0f8420 2024-02-23T00:46:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.798+0000 7f4ff26fb700 10 client.41135.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.798+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_assign -1 1 2024-02-23T00:46:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.798+0000 7f4ff26fb700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f4fec0fee80 0x7f4fec0ff260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.798+0000 7f4ff26fb700 20 client.41135.objecter _get_session s=0x7f4fec10b5d0 osd=0 3 2024-02-23T00:46:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.798+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec10b5d0 osd=0 3 2024-02-23T00:46:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.798+0000 7f4ff26fb700 20 client.41135.objecter _get_session s=0x7f4fec10b5d0 osd=0 3 2024-02-23T00:46:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.798+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_remove -1 1 2024-02-23T00:46:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.798+0000 7f4ff26fb700 20 client.41135.objecter get_session s=0x7f4fec10b5d0 osd=0 3 2024-02-23T00:46:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.798+0000 7f4fe97fa700 10 client.41135.objecter ms_handle_connect 0x7f4fd4062cc0 2024-02-23T00:46:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.798+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_assign 0 1 2024-02-23T00:46:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.798+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec10b5d0 osd=0 4 2024-02-23T00:46:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.798+0000 7f4ff26fb700 10 client.41135.objecter _send_command 1 2024-02-23T00:46:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.798+0000 7f4feb7fe700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4fd4062cc0 0x7f4fd4065180 secure :-1 s=READY pgs=3028 cs=0 l=1 rev1=1 rx=0x7f4fec10b890 tx=0x7f4fe000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:45.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.798+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4fec0ff750 con 0x7f4fec0fee80 2024-02-23T00:46:45.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.799+0000 7f4ff0c98700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f4fec0fee80 0x7f4fec0ff260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:45.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.799+0000 7f4fe97fa700 10 client.41135.objecter ms_handle_connect 0x7f4fec0fee80 2024-02-23T00:46:45.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.799+0000 7f4ff0c98700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f4fec0fee80 0x7f4fec0ff260 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:45.810 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:46:45.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.799+0000 7f4fe97fa700 1 -- 172.21.15.47:0/2372005460 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4fec0ff750 con 0x7f4fec0fee80 2024-02-23T00:46:45.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.799+0000 7f4fe97fa700 10 client.41135.objecter ms_dispatch 0x7f4fec001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:46:45.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.799+0000 7f4fe97fa700 10 client.41135.objecter _finish_command 1 = system:0 2024-02-23T00:46:45.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.800+0000 7f4fe97fa700 20 client.41135.objecter put_session s=0x7f4fec10b5d0 osd=0 4 2024-02-23T00:46:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.800+0000 7f4fe97fa700 15 client.41135.objecter _session_command_op_remove 0 1 2024-02-23T00:46:45.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:44.612172+0000 osd.0 (osd.0) 139 : cluster [DBG] 3.1 scrub starts 2024-02-23T00:46:45.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:44.613008+0000 osd.0 (osd.0) 140 : cluster [DBG] 3.1 scrub ok 2024-02-23T00:46:45.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:44.909913+0000 osd.3 (osd.3) 163 : cluster [DBG] 4.1d scrub starts 2024-02-23T00:46:45.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:44.911274+0000 osd.3 (osd.3) 164 : cluster [DBG] 4.1d scrub ok 2024-02-23T00:46:45.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:45.155409+0000 mon.a (mon.0) 2188 : cluster [DBG] osdmap e333: 8 total, 8 up, 8 in 2024-02-23T00:46:45.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:45.192427+0000 mon.b (mon.1) 367 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:45.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:44.612172+0000 osd.0 (osd.0) 139 : cluster [DBG] 3.1 scrub starts 2024-02-23T00:46:45.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:44.613008+0000 osd.0 (osd.0) 140 : cluster [DBG] 3.1 scrub ok 2024-02-23T00:46:45.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:44.909913+0000 osd.3 (osd.3) 163 : cluster [DBG] 4.1d scrub starts 2024-02-23T00:46:45.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:44.911274+0000 osd.3 (osd.3) 164 : cluster [DBG] 4.1d scrub ok 2024-02-23T00:46:45.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:45.155409+0000 mon.a (mon.0) 2188 : cluster [DBG] osdmap e333: 8 total, 8 up, 8 in 2024-02-23T00:46:45.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:45.192427+0000 mon.b (mon.1) 367 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.823+0000 7f4ff26fb700 10 client.41135.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:46:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.823+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_assign -1 2 2024-02-23T00:46:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.823+0000 7f4ff26fb700 20 client.41135.objecter _get_session s=0x7f4fec10b5d0 osd=0 3 2024-02-23T00:46:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.823+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec10b5d0 osd=0 3 2024-02-23T00:46:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.823+0000 7f4ff26fb700 20 client.41135.objecter _get_session s=0x7f4fec10b5d0 osd=0 3 2024-02-23T00:46:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.823+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_remove -1 2 2024-02-23T00:46:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.823+0000 7f4ff26fb700 20 client.41135.objecter get_session s=0x7f4fec10b5d0 osd=0 3 2024-02-23T00:46:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.823+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_assign 0 2 2024-02-23T00:46:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.823+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec10b5d0 osd=0 4 2024-02-23T00:46:45.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.823+0000 7f4ff26fb700 10 client.41135.objecter _send_command 2 2024-02-23T00:46:45.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.823+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4fec0ff8c0 con 0x7f4fec0fee80 2024-02-23T00:46:45.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.824+0000 7f4fe97fa700 1 -- 172.21.15.47:0/2372005460 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4fec0ff8c0 con 0x7f4fec0fee80 2024-02-23T00:46:45.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.824+0000 7f4fe97fa700 10 client.41135.objecter ms_dispatch 0x7f4fec001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:46:45.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.824+0000 7f4fe97fa700 10 client.41135.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:46:45.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.824+0000 7f4fe97fa700 20 client.41135.objecter put_session s=0x7f4fec10b5d0 osd=0 4 2024-02-23T00:46:45.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.824+0000 7f4fe97fa700 15 client.41135.objecter _session_command_op_remove 0 2 2024-02-23T00:46:45.827 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:46:45.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.825+0000 7f4ff26fb700 10 client.41135.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:45.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.825+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_assign -1 3 2024-02-23T00:46:45.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.825+0000 7f4ff26fb700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f4fec028540 0x7f4fec11af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:45.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.825+0000 7f4ff26fb700 20 client.41135.objecter _get_session s=0x7f4fec0fc750 osd=1 3 2024-02-23T00:46:45.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.825+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec0fc750 osd=1 3 2024-02-23T00:46:45.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.825+0000 7f4ff26fb700 20 client.41135.objecter _get_session s=0x7f4fec0fc750 osd=1 3 2024-02-23T00:46:45.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.825+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_remove -1 3 2024-02-23T00:46:45.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.825+0000 7f4ff26fb700 20 client.41135.objecter get_session s=0x7f4fec0fc750 osd=1 3 2024-02-23T00:46:45.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.825+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_assign 1 3 2024-02-23T00:46:45.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.825+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec0fc750 osd=1 4 2024-02-23T00:46:45.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.825+0000 7f4ff26fb700 10 client.41135.objecter _send_command 3 2024-02-23T00:46:45.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.825+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4fec0fc8c0 con 0x7f4fec028540 2024-02-23T00:46:45.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.825+0000 7f4febfff700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f4fec028540 0x7f4fec11af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:45.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.825+0000 7f4fe97fa700 10 client.41135.objecter ms_handle_connect 0x7f4fec028540 2024-02-23T00:46:45.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.825+0000 7f4febfff700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f4fec028540 0x7f4fec11af30 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:45.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.826+0000 7f4fe97fa700 1 -- 172.21.15.47:0/2372005460 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4fec0fc8c0 con 0x7f4fec028540 2024-02-23T00:46:45.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.826+0000 7f4fe97fa700 10 client.41135.objecter ms_dispatch 0x7f4fec001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:46:45.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.826+0000 7f4fe97fa700 10 client.41135.objecter _finish_command 3 = system:0 2024-02-23T00:46:45.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.826+0000 7f4fe97fa700 20 client.41135.objecter put_session s=0x7f4fec0fc750 osd=1 4 2024-02-23T00:46:45.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.826+0000 7f4fe97fa700 15 client.41135.objecter _session_command_op_remove 1 3 2024-02-23T00:46:45.839 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.849+0000 7f4ff26fb700 10 client.41135.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:46:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.849+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_assign -1 4 2024-02-23T00:46:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.849+0000 7f4ff26fb700 20 client.41135.objecter _get_session s=0x7f4fec0fc750 osd=1 3 2024-02-23T00:46:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.849+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec0fc750 osd=1 3 2024-02-23T00:46:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.849+0000 7f4ff26fb700 20 client.41135.objecter _get_session s=0x7f4fec0fc750 osd=1 3 2024-02-23T00:46:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.849+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_remove -1 4 2024-02-23T00:46:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.849+0000 7f4ff26fb700 20 client.41135.objecter get_session s=0x7f4fec0fc750 osd=1 3 2024-02-23T00:46:45.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.849+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_assign 1 4 2024-02-23T00:46:45.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.849+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec0fc750 osd=1 4 2024-02-23T00:46:45.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.849+0000 7f4ff26fb700 10 client.41135.objecter _send_command 4 2024-02-23T00:46:45.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.849+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4fec05a040 con 0x7f4fec028540 2024-02-23T00:46:45.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.851+0000 7f4fe97fa700 1 -- 172.21.15.47:0/2372005460 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4fec05a040 con 0x7f4fec028540 2024-02-23T00:46:45.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.851+0000 7f4fe97fa700 10 client.41135.objecter ms_dispatch 0x7f4fec001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:46:45.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.851+0000 7f4fe97fa700 10 client.41135.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:46:45.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.851+0000 7f4fe97fa700 20 client.41135.objecter put_session s=0x7f4fec0fc750 osd=1 4 2024-02-23T00:46:45.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.851+0000 7f4fe97fa700 15 client.41135.objecter _session_command_op_remove 1 4 2024-02-23T00:46:45.854 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:46:45.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.851+0000 7f4ff26fb700 10 client.41135.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:45.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.851+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_assign -1 5 2024-02-23T00:46:45.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.851+0000 7f4ff26fb700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f4fec0594d0 0x7f4fec11e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:45.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.851+0000 7f4ff26fb700 20 client.41135.objecter _get_session s=0x7f4fec059360 osd=2 3 2024-02-23T00:46:45.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.851+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec059360 osd=2 3 2024-02-23T00:46:45.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.851+0000 7f4ff26fb700 20 client.41135.objecter _get_session s=0x7f4fec059360 osd=2 3 2024-02-23T00:46:45.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.851+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_remove -1 5 2024-02-23T00:46:45.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.851+0000 7f4ff26fb700 20 client.41135.objecter get_session s=0x7f4fec059360 osd=2 3 2024-02-23T00:46:45.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.851+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_assign 2 5 2024-02-23T00:46:45.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.851+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec059360 osd=2 4 2024-02-23T00:46:45.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.851+0000 7f4ff26fb700 10 client.41135.objecter _send_command 5 2024-02-23T00:46:45.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.851+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4fec028130 con 0x7f4fec0594d0 2024-02-23T00:46:45.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.851+0000 7f4feb7fe700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f4fec0594d0 0x7f4fec11e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:45.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.852+0000 7f4fe97fa700 10 client.41135.objecter ms_handle_connect 0x7f4fec0594d0 2024-02-23T00:46:45.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.852+0000 7f4feb7fe700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f4fec0594d0 0x7f4fec11e5b0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:45.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.852+0000 7f4fe97fa700 1 -- 172.21.15.47:0/2372005460 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4fec028130 con 0x7f4fec0594d0 2024-02-23T00:46:45.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.852+0000 7f4fe97fa700 10 client.41135.objecter ms_dispatch 0x7f4fec001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:46:45.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.852+0000 7f4fe97fa700 10 client.41135.objecter _finish_command 5 = system:0 2024-02-23T00:46:45.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.852+0000 7f4fe97fa700 20 client.41135.objecter put_session s=0x7f4fec059360 osd=2 4 2024-02-23T00:46:45.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.852+0000 7f4fe97fa700 15 client.41135.objecter _session_command_op_remove 2 5 2024-02-23T00:46:45.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.867+0000 7f4ff26fb700 10 client.41135.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:46:45.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.867+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_assign -1 6 2024-02-23T00:46:45.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.867+0000 7f4ff26fb700 20 client.41135.objecter _get_session s=0x7f4fec059360 osd=2 3 2024-02-23T00:46:45.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.867+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec059360 osd=2 3 2024-02-23T00:46:45.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.867+0000 7f4ff26fb700 20 client.41135.objecter _get_session s=0x7f4fec059360 osd=2 3 2024-02-23T00:46:45.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.867+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_remove -1 6 2024-02-23T00:46:45.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.867+0000 7f4ff26fb700 20 client.41135.objecter get_session s=0x7f4fec059360 osd=2 3 2024-02-23T00:46:45.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.867+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_assign 2 6 2024-02-23T00:46:45.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.867+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec059360 osd=2 4 2024-02-23T00:46:45.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.867+0000 7f4ff26fb700 10 client.41135.objecter _send_command 6 2024-02-23T00:46:45.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.867+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4fec11eaa0 con 0x7f4fec0594d0 2024-02-23T00:46:45.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.869+0000 7f4fe97fa700 1 -- 172.21.15.47:0/2372005460 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4fec11eaa0 con 0x7f4fec0594d0 2024-02-23T00:46:45.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.869+0000 7f4fe97fa700 10 client.41135.objecter ms_dispatch 0x7f4fec001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:46:45.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.869+0000 7f4fe97fa700 10 client.41135.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:46:45.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.869+0000 7f4fe97fa700 20 client.41135.objecter put_session s=0x7f4fec059360 osd=2 4 2024-02-23T00:46:45.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.869+0000 7f4fe97fa700 15 client.41135.objecter _session_command_op_remove 2 6 2024-02-23T00:46:45.872 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:46:45.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.869+0000 7f4ff26fb700 10 client.41135.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:45.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.869+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_assign -1 7 2024-02-23T00:46:45.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.869+0000 7f4ff26fb700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f4fec11c100 0x7f4fec121da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:45.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.869+0000 7f4ff26fb700 20 client.41135.objecter _get_session s=0x7f4fec027600 osd=3 3 2024-02-23T00:46:45.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.869+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec027600 osd=3 3 2024-02-23T00:46:45.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.869+0000 7f4ff26fb700 20 client.41135.objecter _get_session s=0x7f4fec027600 osd=3 3 2024-02-23T00:46:45.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.869+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_remove -1 7 2024-02-23T00:46:45.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.869+0000 7f4ff26fb700 20 client.41135.objecter get_session s=0x7f4fec027600 osd=3 3 2024-02-23T00:46:45.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.869+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_assign 3 7 2024-02-23T00:46:45.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.869+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec027600 osd=3 4 2024-02-23T00:46:45.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.869+0000 7f4ff26fb700 10 client.41135.objecter _send_command 7 2024-02-23T00:46:45.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.869+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4fec027960 con 0x7f4fec11c100 2024-02-23T00:46:45.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.869+0000 7f4ff0c98700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f4fec11c100 0x7f4fec121da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:45.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.870+0000 7f4fe97fa700 10 client.41135.objecter ms_handle_connect 0x7f4fec11c100 2024-02-23T00:46:45.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.870+0000 7f4ff0c98700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f4fec11c100 0x7f4fec121da0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:45.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.870+0000 7f4fe97fa700 1 -- 172.21.15.47:0/2372005460 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4fec027960 con 0x7f4fec11c100 2024-02-23T00:46:45.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.870+0000 7f4fe97fa700 10 client.41135.objecter ms_dispatch 0x7f4fec001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:46:45.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.870+0000 7f4fe97fa700 10 client.41135.objecter _finish_command 7 = system:0 2024-02-23T00:46:45.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.870+0000 7f4fe97fa700 20 client.41135.objecter put_session s=0x7f4fec027600 osd=3 4 2024-02-23T00:46:45.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.870+0000 7f4fe97fa700 15 client.41135.objecter _session_command_op_remove 3 7 2024-02-23T00:46:45.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.885+0000 7f4ff26fb700 10 client.41135.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:46:45.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.885+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_assign -1 8 2024-02-23T00:46:45.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.885+0000 7f4ff26fb700 20 client.41135.objecter _get_session s=0x7f4fec027600 osd=3 3 2024-02-23T00:46:45.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.885+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec027600 osd=3 3 2024-02-23T00:46:45.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.885+0000 7f4ff26fb700 20 client.41135.objecter _get_session s=0x7f4fec027600 osd=3 3 2024-02-23T00:46:45.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.885+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_remove -1 8 2024-02-23T00:46:45.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.885+0000 7f4ff26fb700 20 client.41135.objecter get_session s=0x7f4fec027600 osd=3 3 2024-02-23T00:46:45.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.885+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_assign 3 8 2024-02-23T00:46:45.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.885+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec027600 osd=3 4 2024-02-23T00:46:45.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.885+0000 7f4ff26fb700 10 client.41135.objecter _send_command 8 2024-02-23T00:46:45.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.885+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4fec11b660 con 0x7f4fec11c100 2024-02-23T00:46:45.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.886+0000 7f4fe97fa700 1 -- 172.21.15.47:0/2372005460 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4fec11b660 con 0x7f4fec11c100 2024-02-23T00:46:45.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.886+0000 7f4fe97fa700 10 client.41135.objecter ms_dispatch 0x7f4fec001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:46:45.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.886+0000 7f4fe97fa700 10 client.41135.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:46:45.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.886+0000 7f4fe97fa700 20 client.41135.objecter put_session s=0x7f4fec027600 osd=3 4 2024-02-23T00:46:45.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.886+0000 7f4fe97fa700 15 client.41135.objecter _session_command_op_remove 3 8 2024-02-23T00:46:45.889 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:46:45.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.886+0000 7f4ff26fb700 10 client.41135.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:45.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.886+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_assign -1 9 2024-02-23T00:46:45.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.886+0000 7f4ff26fb700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f4fec11f8f0 0x7f4fec125420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:45.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.886+0000 7f4ff26fb700 20 client.41135.objecter _get_session s=0x7f4fec11b820 osd=4 3 2024-02-23T00:46:45.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.886+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec11b820 osd=4 3 2024-02-23T00:46:45.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.886+0000 7f4ff26fb700 20 client.41135.objecter _get_session s=0x7f4fec11b820 osd=4 3 2024-02-23T00:46:45.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.886+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_remove -1 9 2024-02-23T00:46:45.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.886+0000 7f4ff26fb700 20 client.41135.objecter get_session s=0x7f4fec11b820 osd=4 3 2024-02-23T00:46:45.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.886+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_assign 4 9 2024-02-23T00:46:45.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.886+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec11b820 osd=4 4 2024-02-23T00:46:45.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.886+0000 7f4ff26fb700 10 client.41135.objecter _send_command 9 2024-02-23T00:46:45.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.886+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4fec11ba40 con 0x7f4fec11f8f0 2024-02-23T00:46:45.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.887+0000 7f4febfff700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f4fec11f8f0 0x7f4fec125420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:45.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.887+0000 7f4fe97fa700 10 client.41135.objecter ms_handle_connect 0x7f4fec11f8f0 2024-02-23T00:46:45.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.887+0000 7f4febfff700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f4fec11f8f0 0x7f4fec125420 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:45.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.888+0000 7f4fe97fa700 1 -- 172.21.15.47:0/2372005460 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4fec11ba40 con 0x7f4fec11f8f0 2024-02-23T00:46:45.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.888+0000 7f4fe97fa700 10 client.41135.objecter ms_dispatch 0x7f4fec001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:46:45.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.888+0000 7f4fe97fa700 10 client.41135.objecter _finish_command 9 = system:0 2024-02-23T00:46:45.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.888+0000 7f4fe97fa700 20 client.41135.objecter put_session s=0x7f4fec11b820 osd=4 4 2024-02-23T00:46:45.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.888+0000 7f4fe97fa700 15 client.41135.objecter _session_command_op_remove 4 9 2024-02-23T00:46:45.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.902+0000 7f4ff26fb700 10 client.41135.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:46:45.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.902+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_assign -1 10 2024-02-23T00:46:45.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.902+0000 7f4ff26fb700 20 client.41135.objecter _get_session s=0x7f4fec11b820 osd=4 3 2024-02-23T00:46:45.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.902+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec11b820 osd=4 3 2024-02-23T00:46:45.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.902+0000 7f4ff26fb700 20 client.41135.objecter _get_session s=0x7f4fec11b820 osd=4 3 2024-02-23T00:46:45.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.902+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_remove -1 10 2024-02-23T00:46:45.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.902+0000 7f4ff26fb700 20 client.41135.objecter get_session s=0x7f4fec11b820 osd=4 3 2024-02-23T00:46:45.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.902+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_assign 4 10 2024-02-23T00:46:45.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.902+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec11b820 osd=4 4 2024-02-23T00:46:45.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.902+0000 7f4ff26fb700 10 client.41135.objecter _send_command 10 2024-02-23T00:46:45.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.902+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4fec11ba40 con 0x7f4fec11f8f0 2024-02-23T00:46:45.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.904+0000 7f4fe97fa700 1 -- 172.21.15.47:0/2372005460 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4fec11ba40 con 0x7f4fec11f8f0 2024-02-23T00:46:45.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.904+0000 7f4fe97fa700 10 client.41135.objecter ms_dispatch 0x7f4fec001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:46:45.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.904+0000 7f4fe97fa700 10 client.41135.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:46:45.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.904+0000 7f4fe97fa700 20 client.41135.objecter put_session s=0x7f4fec11b820 osd=4 4 2024-02-23T00:46:45.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.904+0000 7f4fe97fa700 15 client.41135.objecter _session_command_op_remove 4 10 2024-02-23T00:46:45.907 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:46:45.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.904+0000 7f4ff26fb700 10 client.41135.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:45.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.904+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_assign -1 11 2024-02-23T00:46:45.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.904+0000 7f4ff26fb700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f4fec125e20 0x7f4fec128290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:45.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.904+0000 7f4ff26fb700 20 client.41135.objecter _get_session s=0x7f4fec11ba40 osd=5 3 2024-02-23T00:46:45.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.904+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec11ba40 osd=5 3 2024-02-23T00:46:45.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.904+0000 7f4ff26fb700 20 client.41135.objecter _get_session s=0x7f4fec11ba40 osd=5 3 2024-02-23T00:46:45.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.904+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_remove -1 11 2024-02-23T00:46:45.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.904+0000 7f4ff26fb700 20 client.41135.objecter get_session s=0x7f4fec11ba40 osd=5 3 2024-02-23T00:46:45.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.904+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_assign 5 11 2024-02-23T00:46:45.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.904+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec11ba40 osd=5 4 2024-02-23T00:46:45.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.904+0000 7f4ff26fb700 10 client.41135.objecter _send_command 11 2024-02-23T00:46:45.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.904+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4fec12b0d0 con 0x7f4fec125e20 2024-02-23T00:46:45.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.905+0000 7f4feb7fe700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f4fec125e20 0x7f4fec128290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:45.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.905+0000 7f4fe97fa700 10 client.41135.objecter ms_handle_connect 0x7f4fec125e20 2024-02-23T00:46:45.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.905+0000 7f4feb7fe700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f4fec125e20 0x7f4fec128290 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:45.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.906+0000 7f4fe97fa700 1 -- 172.21.15.47:0/2372005460 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4fec12b0d0 con 0x7f4fec125e20 2024-02-23T00:46:45.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.906+0000 7f4fe97fa700 10 client.41135.objecter ms_dispatch 0x7f4fec001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:46:45.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.906+0000 7f4fe97fa700 10 client.41135.objecter _finish_command 11 = system:0 2024-02-23T00:46:45.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.906+0000 7f4fe97fa700 20 client.41135.objecter put_session s=0x7f4fec11ba40 osd=5 4 2024-02-23T00:46:45.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.906+0000 7f4fe97fa700 15 client.41135.objecter _session_command_op_remove 5 11 2024-02-23T00:46:45.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.921+0000 7f4ff26fb700 10 client.41135.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:46:45.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.921+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_assign -1 12 2024-02-23T00:46:45.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.921+0000 7f4ff26fb700 20 client.41135.objecter _get_session s=0x7f4fec11ba40 osd=5 3 2024-02-23T00:46:45.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.921+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec11ba40 osd=5 3 2024-02-23T00:46:45.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.921+0000 7f4ff26fb700 20 client.41135.objecter _get_session s=0x7f4fec11ba40 osd=5 3 2024-02-23T00:46:45.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.921+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_remove -1 12 2024-02-23T00:46:45.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.921+0000 7f4ff26fb700 20 client.41135.objecter get_session s=0x7f4fec11ba40 osd=5 3 2024-02-23T00:46:45.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.921+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_assign 5 12 2024-02-23T00:46:45.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.921+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec11ba40 osd=5 4 2024-02-23T00:46:45.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.921+0000 7f4ff26fb700 10 client.41135.objecter _send_command 12 2024-02-23T00:46:45.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.921+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4fec129ea0 con 0x7f4fec125e20 2024-02-23T00:46:45.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.922+0000 7f4fe97fa700 1 -- 172.21.15.47:0/2372005460 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4fec129ea0 con 0x7f4fec125e20 2024-02-23T00:46:45.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.922+0000 7f4fe97fa700 10 client.41135.objecter ms_dispatch 0x7f4fec001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:46:45.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.922+0000 7f4fe97fa700 10 client.41135.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:46:45.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.922+0000 7f4fe97fa700 20 client.41135.objecter put_session s=0x7f4fec11ba40 osd=5 4 2024-02-23T00:46:45.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.922+0000 7f4fe97fa700 15 client.41135.objecter _session_command_op_remove 5 12 2024-02-23T00:46:45.925 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:46:45.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.923+0000 7f4ff26fb700 10 client.41135.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:45.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.923+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_assign -1 13 2024-02-23T00:46:45.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.923+0000 7f4ff26fb700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f4fec12a5b0 0x7f4fec12aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:45.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.923+0000 7f4ff26fb700 20 client.41135.objecter _get_session s=0x7f4fec129ea0 osd=6 3 2024-02-23T00:46:45.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.923+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec129ea0 osd=6 3 2024-02-23T00:46:45.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.923+0000 7f4ff26fb700 20 client.41135.objecter _get_session s=0x7f4fec129ea0 osd=6 3 2024-02-23T00:46:45.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.923+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_remove -1 13 2024-02-23T00:46:45.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.923+0000 7f4ff26fb700 20 client.41135.objecter get_session s=0x7f4fec129ea0 osd=6 3 2024-02-23T00:46:45.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.923+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_assign 6 13 2024-02-23T00:46:45.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.923+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec129ea0 osd=6 4 2024-02-23T00:46:45.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.923+0000 7f4ff26fb700 10 client.41135.objecter _send_command 13 2024-02-23T00:46:45.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.923+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4fec12e310 con 0x7f4fec12a5b0 2024-02-23T00:46:45.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.923+0000 7f4ff0c98700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f4fec12a5b0 0x7f4fec12aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:45.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.923+0000 7f4ff0c98700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f4fec12a5b0 0x7f4fec12aa10 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:45.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.923+0000 7f4fe97fa700 10 client.41135.objecter ms_handle_connect 0x7f4fec12a5b0 2024-02-23T00:46:45.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.924+0000 7f4fe97fa700 1 -- 172.21.15.47:0/2372005460 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4fec12e310 con 0x7f4fec12a5b0 2024-02-23T00:46:45.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.924+0000 7f4fe97fa700 10 client.41135.objecter ms_dispatch 0x7f4fec001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:46:45.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.924+0000 7f4fe97fa700 10 client.41135.objecter _finish_command 13 = system:0 2024-02-23T00:46:45.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.924+0000 7f4fe97fa700 20 client.41135.objecter put_session s=0x7f4fec129ea0 osd=6 4 2024-02-23T00:46:45.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.924+0000 7f4fe97fa700 15 client.41135.objecter _session_command_op_remove 6 13 2024-02-23T00:46:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.938+0000 7f4ff26fb700 10 client.41135.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:46:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.938+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_assign -1 14 2024-02-23T00:46:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.938+0000 7f4ff26fb700 20 client.41135.objecter _get_session s=0x7f4fec129ea0 osd=6 3 2024-02-23T00:46:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.938+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec129ea0 osd=6 3 2024-02-23T00:46:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.938+0000 7f4ff26fb700 20 client.41135.objecter _get_session s=0x7f4fec129ea0 osd=6 3 2024-02-23T00:46:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.938+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_remove -1 14 2024-02-23T00:46:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.938+0000 7f4ff26fb700 20 client.41135.objecter get_session s=0x7f4fec129ea0 osd=6 3 2024-02-23T00:46:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.938+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_assign 6 14 2024-02-23T00:46:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.938+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec129ea0 osd=6 4 2024-02-23T00:46:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.938+0000 7f4ff26fb700 10 client.41135.objecter _send_command 14 2024-02-23T00:46:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.938+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4fec12e310 con 0x7f4fec12a5b0 2024-02-23T00:46:45.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.940+0000 7f4fe97fa700 1 -- 172.21.15.47:0/2372005460 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4fec12e310 con 0x7f4fec12a5b0 2024-02-23T00:46:45.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.940+0000 7f4fe97fa700 10 client.41135.objecter ms_dispatch 0x7f4fec001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:46:45.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.940+0000 7f4fe97fa700 10 client.41135.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:46:45.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.940+0000 7f4fe97fa700 20 client.41135.objecter put_session s=0x7f4fec129ea0 osd=6 4 2024-02-23T00:46:45.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.940+0000 7f4fe97fa700 15 client.41135.objecter _session_command_op_remove 6 14 2024-02-23T00:46:45.943 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:46:45.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.940+0000 7f4ff26fb700 10 client.41135.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:45.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.940+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_assign -1 15 2024-02-23T00:46:45.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.940+0000 7f4ff26fb700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f4fec12d680 0x7f4fec12dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:45.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.940+0000 7f4ff26fb700 20 client.41135.objecter _get_session s=0x7f4fec12e310 osd=7 3 2024-02-23T00:46:45.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.940+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec12e310 osd=7 3 2024-02-23T00:46:45.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.940+0000 7f4ff26fb700 20 client.41135.objecter _get_session s=0x7f4fec12e310 osd=7 3 2024-02-23T00:46:45.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.940+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_remove -1 15 2024-02-23T00:46:45.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.940+0000 7f4ff26fb700 20 client.41135.objecter get_session s=0x7f4fec12e310 osd=7 3 2024-02-23T00:46:45.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.940+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_assign 7 15 2024-02-23T00:46:45.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.940+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec12e310 osd=7 4 2024-02-23T00:46:45.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.940+0000 7f4ff26fb700 10 client.41135.objecter _send_command 15 2024-02-23T00:46:45.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.940+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4fec1316a0 con 0x7f4fec12d680 2024-02-23T00:46:45.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.941+0000 7f4febfff700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f4fec12d680 0x7f4fec12dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:45.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.941+0000 7f4fe97fa700 10 client.41135.objecter ms_handle_connect 0x7f4fec12d680 2024-02-23T00:46:45.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.941+0000 7f4febfff700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f4fec12d680 0x7f4fec12dae0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:45.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.942+0000 7f4fe97fa700 1 -- 172.21.15.47:0/2372005460 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4fec1316a0 con 0x7f4fec12d680 2024-02-23T00:46:45.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.942+0000 7f4fe97fa700 10 client.41135.objecter ms_dispatch 0x7f4fec001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:46:45.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.942+0000 7f4fe97fa700 10 client.41135.objecter _finish_command 15 = system:0 2024-02-23T00:46:45.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.942+0000 7f4fe97fa700 20 client.41135.objecter put_session s=0x7f4fec12e310 osd=7 4 2024-02-23T00:46:45.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.942+0000 7f4fe97fa700 15 client.41135.objecter _session_command_op_remove 7 15 2024-02-23T00:46:45.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:44.612172+0000 osd.0 (osd.0) 139 : cluster [DBG] 3.1 scrub starts 2024-02-23T00:46:45.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:44.613008+0000 osd.0 (osd.0) 140 : cluster [DBG] 3.1 scrub ok 2024-02-23T00:46:45.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:44.909913+0000 osd.3 (osd.3) 163 : cluster [DBG] 4.1d scrub starts 2024-02-23T00:46:45.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:44.911274+0000 osd.3 (osd.3) 164 : cluster [DBG] 4.1d scrub ok 2024-02-23T00:46:45.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:45.155409+0000 mon.a (mon.0) 2188 : cluster [DBG] osdmap e333: 8 total, 8 up, 8 in 2024-02-23T00:46:45.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:45.192427+0000 mon.b (mon.1) 367 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:45.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.956+0000 7f4ff26fb700 10 client.41135.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:46:45.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.956+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_assign -1 16 2024-02-23T00:46:45.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.956+0000 7f4ff26fb700 20 client.41135.objecter _get_session s=0x7f4fec12e310 osd=7 3 2024-02-23T00:46:45.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.956+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec12e310 osd=7 3 2024-02-23T00:46:45.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.956+0000 7f4ff26fb700 20 client.41135.objecter _get_session s=0x7f4fec12e310 osd=7 3 2024-02-23T00:46:45.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.956+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_remove -1 16 2024-02-23T00:46:45.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.956+0000 7f4ff26fb700 20 client.41135.objecter get_session s=0x7f4fec12e310 osd=7 3 2024-02-23T00:46:45.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.956+0000 7f4ff26fb700 15 client.41135.objecter _session_command_op_assign 7 16 2024-02-23T00:46:45.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.956+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec12e310 osd=7 4 2024-02-23T00:46:45.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.956+0000 7f4ff26fb700 10 client.41135.objecter _send_command 16 2024-02-23T00:46:45.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.956+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4fec128890 con 0x7f4fec12d680 2024-02-23T00:46:45.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4fe97fa700 1 -- 172.21.15.47:0/2372005460 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4fec128890 con 0x7f4fec12d680 2024-02-23T00:46:45.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4fe97fa700 10 client.41135.objecter ms_dispatch 0x7f4fec001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:46:45.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4fe97fa700 10 client.41135.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:46:45.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4fe97fa700 20 client.41135.objecter put_session s=0x7f4fec12e310 osd=7 4 2024-02-23T00:46:45.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4fe97fa700 15 client.41135.objecter _session_command_op_remove 7 16 2024-02-23T00:46:45.961 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:46:45.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 10 librados: watch_flush enter 2024-02-23T00:46:45.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 10 librados: watch_flush exit 2024-02-23T00:46:45.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 10 client.41135.objecter close_session for osd.0 2024-02-23T00:46:45.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f4fec0fee80 msgr2=0x7f4fec0ff260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:45.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f4fec0fee80 0x7f4fec0ff260 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:45.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec10b5d0 osd=0 1 2024-02-23T00:46:45.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 10 client.41135.objecter close_session for osd.1 2024-02-23T00:46:45.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f4fec028540 msgr2=0x7f4fec11af30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:45.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f4fec028540 0x7f4fec11af30 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:45.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec0fc750 osd=1 1 2024-02-23T00:46:45.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 10 client.41135.objecter close_session for osd.2 2024-02-23T00:46:45.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f4fec0594d0 msgr2=0x7f4fec11e5b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:45.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f4fec0594d0 0x7f4fec11e5b0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:45.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec059360 osd=2 1 2024-02-23T00:46:45.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 10 client.41135.objecter close_session for osd.3 2024-02-23T00:46:45.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f4fec11c100 msgr2=0x7f4fec121da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:45.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f4fec11c100 0x7f4fec121da0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:45.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec027600 osd=3 1 2024-02-23T00:46:45.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 10 client.41135.objecter close_session for osd.4 2024-02-23T00:46:45.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f4fec11f8f0 msgr2=0x7f4fec125420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:45.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f4fec11f8f0 0x7f4fec125420 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:45.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec11b820 osd=4 1 2024-02-23T00:46:45.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 10 client.41135.objecter close_session for osd.5 2024-02-23T00:46:45.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f4fec125e20 msgr2=0x7f4fec128290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:45.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f4fec125e20 0x7f4fec128290 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:45.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec11ba40 osd=5 1 2024-02-23T00:46:45.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 10 client.41135.objecter close_session for osd.6 2024-02-23T00:46:45.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f4fec12a5b0 msgr2=0x7f4fec12aa10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:45.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f4fec12a5b0 0x7f4fec12aa10 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:45.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff0c98700 1 -- 172.21.15.47:0/2372005460 reap_dead start 2024-02-23T00:46:45.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec129ea0 osd=6 1 2024-02-23T00:46:45.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 10 client.41135.objecter close_session for osd.7 2024-02-23T00:46:45.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f4fec12d680 msgr2=0x7f4fec12dae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:45.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f4fec12d680 0x7f4fec12dae0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:45.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff0c98700 1 -- 172.21.15.47:0/2372005460 reap_dead start 2024-02-23T00:46:45.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 20 client.41135.objecter put_session s=0x7f4fec12e310 osd=7 1 2024-02-23T00:46:45.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff0c98700 1 -- 172.21.15.47:0/2372005460 reap_dead start 2024-02-23T00:46:45.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 20 client.41135.objecter shutdown clearing up homeless session... 2024-02-23T00:46:45.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 10 client.41135.objecter successfully canceled tick 2024-02-23T00:46:45.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff0c98700 1 -- 172.21.15.47:0/2372005460 reap_dead start 2024-02-23T00:46:45.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4fd4062cc0 msgr2=0x7f4fd4065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:45.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4fd4062cc0 0x7f4fd4065180 secure :-1 s=READY pgs=3028 cs=0 l=1 rev1=1 rx=0x7f4fec10b890 tx=0x7f4fe000b040).stop 2024-02-23T00:46:45.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4fec0f8420 msgr2=0x7f4fec10ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:45.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4fec0f8420 0x7f4fec10ff50 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 rx=0x7f4fdc009400 tx=0x7f4fdc009cb0).stop 2024-02-23T00:46:45.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 shutdown_connections 2024-02-23T00:46:45.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4fec0f8420 0x7f4fec10ff50 unknown :-1 s=CLOSED pgs=2622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:45.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 1 --2- 172.21.15.47:0/2372005460 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4fd4062cc0 0x7f4fd4065180 unknown :-1 s=CLOSED pgs=3028 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.958+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 >> 172.21.15.47:0/2372005460 conn(0x7f4fec0f3800 msgr2=0x7f4fec0fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:46:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.959+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 shutdown_connections 2024-02-23T00:46:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.959+0000 7f4ff26fb700 1 -- 172.21.15.47:0/2372005460 wait complete. 2024-02-23T00:46:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:45.959+0000 7f4ff26fb700 1 librados: shutdown 2024-02-23T00:46:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:46:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:46:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:46:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:46:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:46:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:46:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:46:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:46:46.088 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:46:46.091 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:46.092 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:46:46.193 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:46:46.224 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:46.334 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:46:46.335 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-02-23T00:46:46.335 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-02-23T00:46:46.335 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-02-23T00:46:46.335 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:46:46.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:45.204377+0000 osd.5 (osd.5) 142 : cluster [DBG] 6.7 scrub starts 2024-02-23T00:46:46.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:45.207271+0000 osd.5 (osd.5) 143 : cluster [DBG] 6.7 scrub ok 2024-02-23T00:46:46.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:45.262662+0000 mgr.y (mgr.14517) 324 : cluster [DBG] pgmap v475: 244 pgs: 8 unknown, 1 clean+premerge+peered, 235 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s; 31 B/s, 0 objects/s recovering 2024-02-23T00:46:46.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:45.293901+0000 osd.6 (osd.6) 135 : cluster [DBG] 5.2 scrub starts 2024-02-23T00:46:46.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:45.294701+0000 osd.6 (osd.6) 136 : cluster [DBG] 5.2 scrub ok 2024-02-23T00:46:46.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:45.623409+0000 osd.0 (osd.0) 141 : cluster [DBG] 4.b scrub starts 2024-02-23T00:46:46.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:45.625122+0000 osd.0 (osd.0) 142 : cluster [DBG] 4.b scrub ok 2024-02-23T00:46:46.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:45.673460+0000 osd.2 (osd.2) 147 : cluster [DBG] 5.a scrub starts 2024-02-23T00:46:46.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:45.674387+0000 osd.2 (osd.2) 148 : cluster [DBG] 5.a scrub ok 2024-02-23T00:46:46.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:45.799233+0000 mon.c (mon.2) 250 : audit [DBG] from='client.? 172.21.15.47:0/2372005460' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:46:46.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:45.912490+0000 osd.3 (osd.3) 165 : cluster [DBG] 5.17 deep-scrub starts 2024-02-23T00:46:46.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:45.913517+0000 osd.3 (osd.3) 166 : cluster [DBG] 5.17 deep-scrub ok 2024-02-23T00:46:46.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:46.164267+0000 mon.a (mon.0) 2189 : cluster [DBG] osdmap e334: 8 total, 8 up, 8 in 2024-02-23T00:46:46.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:46.166074+0000 mon.a (mon.0) 2190 : audit [INF] from='client.? 172.21.15.47:0/2392849431' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi047-82386-56"}]: dispatch 2024-02-23T00:46:46.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:46.193283+0000 mon.b (mon.1) 368 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:46.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:45.204377+0000 osd.5 (osd.5) 142 : cluster [DBG] 6.7 scrub starts 2024-02-23T00:46:46.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:45.207271+0000 osd.5 (osd.5) 143 : cluster [DBG] 6.7 scrub ok 2024-02-23T00:46:46.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:45.262662+0000 mgr.y (mgr.14517) 324 : cluster [DBG] pgmap v475: 244 pgs: 8 unknown, 1 clean+premerge+peered, 235 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s; 31 B/s, 0 objects/s recovering 2024-02-23T00:46:46.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:45.293901+0000 osd.6 (osd.6) 135 : cluster [DBG] 5.2 scrub starts 2024-02-23T00:46:46.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:45.294701+0000 osd.6 (osd.6) 136 : cluster [DBG] 5.2 scrub ok 2024-02-23T00:46:46.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:45.623409+0000 osd.0 (osd.0) 141 : cluster [DBG] 4.b scrub starts 2024-02-23T00:46:46.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:45.625122+0000 osd.0 (osd.0) 142 : cluster [DBG] 4.b scrub ok 2024-02-23T00:46:46.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:45.673460+0000 osd.2 (osd.2) 147 : cluster [DBG] 5.a scrub starts 2024-02-23T00:46:46.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:45.674387+0000 osd.2 (osd.2) 148 : cluster [DBG] 5.a scrub ok 2024-02-23T00:46:46.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:45.799233+0000 mon.c (mon.2) 250 : audit [DBG] from='client.? 172.21.15.47:0/2372005460' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:46:46.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:45.912490+0000 osd.3 (osd.3) 165 : cluster [DBG] 5.17 deep-scrub starts 2024-02-23T00:46:46.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:45.913517+0000 osd.3 (osd.3) 166 : cluster [DBG] 5.17 deep-scrub ok 2024-02-23T00:46:46.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:46.164267+0000 mon.a (mon.0) 2189 : cluster [DBG] osdmap e334: 8 total, 8 up, 8 in 2024-02-23T00:46:46.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:46.166074+0000 mon.a (mon.0) 2190 : audit [INF] from='client.? 172.21.15.47:0/2392849431' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi047-82386-56"}]: dispatch 2024-02-23T00:46:46.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:46.193283+0000 mon.b (mon.1) 368 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:46.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:45.204377+0000 osd.5 (osd.5) 142 : cluster [DBG] 6.7 scrub starts 2024-02-23T00:46:46.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:45.207271+0000 osd.5 (osd.5) 143 : cluster [DBG] 6.7 scrub ok 2024-02-23T00:46:46.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:45.262662+0000 mgr.y (mgr.14517) 324 : cluster [DBG] pgmap v475: 244 pgs: 8 unknown, 1 clean+premerge+peered, 235 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s; 31 B/s, 0 objects/s recovering 2024-02-23T00:46:46.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:45.293901+0000 osd.6 (osd.6) 135 : cluster [DBG] 5.2 scrub starts 2024-02-23T00:46:46.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:45.294701+0000 osd.6 (osd.6) 136 : cluster [DBG] 5.2 scrub ok 2024-02-23T00:46:46.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:45.623409+0000 osd.0 (osd.0) 141 : cluster [DBG] 4.b scrub starts 2024-02-23T00:46:46.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:45.625122+0000 osd.0 (osd.0) 142 : cluster [DBG] 4.b scrub ok 2024-02-23T00:46:46.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:45.673460+0000 osd.2 (osd.2) 147 : cluster [DBG] 5.a scrub starts 2024-02-23T00:46:46.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:45.674387+0000 osd.2 (osd.2) 148 : cluster [DBG] 5.a scrub ok 2024-02-23T00:46:46.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:45.799233+0000 mon.c (mon.2) 250 : audit [DBG] from='client.? 172.21.15.47:0/2372005460' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:46:46.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:45.912490+0000 osd.3 (osd.3) 165 : cluster [DBG] 5.17 deep-scrub starts 2024-02-23T00:46:46.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:45.913517+0000 osd.3 (osd.3) 166 : cluster [DBG] 5.17 deep-scrub ok 2024-02-23T00:46:46.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:46.164267+0000 mon.a (mon.0) 2189 : cluster [DBG] osdmap e334: 8 total, 8 up, 8 in 2024-02-23T00:46:46.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:46.166074+0000 mon.a (mon.0) 2190 : audit [INF] from='client.? 172.21.15.47:0/2392849431' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi047-82386-56"}]: dispatch 2024-02-23T00:46:46.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:46.193283+0000 mon.b (mon.1) 368 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:47.091 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.4 dump_historic_ops 2024-02-23T00:46:47.304 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set noscrub 2024-02-23T00:46:47.364 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:46:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:46:47.376 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:47.377 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:46:47.477 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:46:47.506 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:47.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:46.200062+0000 osd.5 (osd.5) 144 : cluster [DBG] 2.12 scrub starts 2024-02-23T00:46:47.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:46.201055+0000 osd.5 (osd.5) 145 : cluster [DBG] 2.12 scrub ok 2024-02-23T00:46:47.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:46.643817+0000 osd.0 (osd.0) 143 : cluster [DBG] 4.16 scrub starts 2024-02-23T00:46:47.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:46.645149+0000 osd.0 (osd.0) 144 : cluster [DBG] 4.16 scrub ok 2024-02-23T00:46:47.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:46.683808+0000 osd.7 (osd.7) 171 : cluster [DBG] 6.3 deep-scrub starts 2024-02-23T00:46:47.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:46.686760+0000 osd.7 (osd.7) 172 : cluster [DBG] 6.3 deep-scrub ok 2024-02-23T00:46:47.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:47.166783+0000 mon.a (mon.0) 2191 : audit [INF] from='client.? 172.21.15.47:0/2392849431' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi047-82386-56"}]': finished 2024-02-23T00:46:47.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:47.166870+0000 mon.a (mon.0) 2192 : cluster [DBG] osdmap e335: 8 total, 8 up, 8 in 2024-02-23T00:46:47.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:47.168414+0000 mon.a (mon.0) 2193 : audit [INF] from='client.? 172.21.15.47:0/2392849431' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi047-82386-56"}]: dispatch 2024-02-23T00:46:47.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:47.194127+0000 mon.b (mon.1) 369 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:47.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:47.264497+0000 mon.a (mon.0) 2194 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "16"}]: dispatch 2024-02-23T00:46:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:46.200062+0000 osd.5 (osd.5) 144 : cluster [DBG] 2.12 scrub starts 2024-02-23T00:46:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:46.201055+0000 osd.5 (osd.5) 145 : cluster [DBG] 2.12 scrub ok 2024-02-23T00:46:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:46.643817+0000 osd.0 (osd.0) 143 : cluster [DBG] 4.16 scrub starts 2024-02-23T00:46:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:46.645149+0000 osd.0 (osd.0) 144 : cluster [DBG] 4.16 scrub ok 2024-02-23T00:46:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:46.683808+0000 osd.7 (osd.7) 171 : cluster [DBG] 6.3 deep-scrub starts 2024-02-23T00:46:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:46.686760+0000 osd.7 (osd.7) 172 : cluster [DBG] 6.3 deep-scrub ok 2024-02-23T00:46:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:47.166783+0000 mon.a (mon.0) 2191 : audit [INF] from='client.? 172.21.15.47:0/2392849431' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi047-82386-56"}]': finished 2024-02-23T00:46:48.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:47.166870+0000 mon.a (mon.0) 2192 : cluster [DBG] osdmap e335: 8 total, 8 up, 8 in 2024-02-23T00:46:48.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:47.168414+0000 mon.a (mon.0) 2193 : audit [INF] from='client.? 172.21.15.47:0/2392849431' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi047-82386-56"}]: dispatch 2024-02-23T00:46:48.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:47.194127+0000 mon.b (mon.1) 369 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:48.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:47.264497+0000 mon.a (mon.0) 2194 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "16"}]: dispatch 2024-02-23T00:46:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:46.200062+0000 osd.5 (osd.5) 144 : cluster [DBG] 2.12 scrub starts 2024-02-23T00:46:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:46.201055+0000 osd.5 (osd.5) 145 : cluster [DBG] 2.12 scrub ok 2024-02-23T00:46:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:46.643817+0000 osd.0 (osd.0) 143 : cluster [DBG] 4.16 scrub starts 2024-02-23T00:46:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:46.645149+0000 osd.0 (osd.0) 144 : cluster [DBG] 4.16 scrub ok 2024-02-23T00:46:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:46.683808+0000 osd.7 (osd.7) 171 : cluster [DBG] 6.3 deep-scrub starts 2024-02-23T00:46:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:46.686760+0000 osd.7 (osd.7) 172 : cluster [DBG] 6.3 deep-scrub ok 2024-02-23T00:46:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:47.166783+0000 mon.a (mon.0) 2191 : audit [INF] from='client.? 172.21.15.47:0/2392849431' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi047-82386-56"}]': finished 2024-02-23T00:46:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:47.166870+0000 mon.a (mon.0) 2192 : cluster [DBG] osdmap e335: 8 total, 8 up, 8 in 2024-02-23T00:46:48.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:47.168414+0000 mon.a (mon.0) 2193 : audit [INF] from='client.? 172.21.15.47:0/2392849431' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi047-82386-56"}]: dispatch 2024-02-23T00:46:48.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:47.194127+0000 mon.b (mon.1) 369 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:48.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:47.264497+0000 mon.a (mon.0) 2194 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "16"}]: dispatch 2024-02-23T00:46:48.234 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:46:48.237 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:48.237 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:46:48.338 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:46:48.368 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:48.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.930+0000 7f2e07f95700 1 -- 172.21.15.47:0/1297355387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e000fcbb0 msgr2=0x7f2e000fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:48.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.930+0000 7f2e07f95700 1 --2- 172.21.15.47:0/1297355387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e000fcbb0 0x7f2e000fcf90 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 rx=0x7f2dfc004870 tx=0x7f2dfc009cb0).stop 2024-02-23T00:46:48.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.930+0000 7f2e07f95700 1 -- 172.21.15.47:0/1297355387 shutdown_connections 2024-02-23T00:46:48.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.930+0000 7f2e07f95700 1 --2- 172.21.15.47:0/1297355387 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2e000f7400 0x7f2e00103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:48.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.930+0000 7f2e07f95700 1 --2- 172.21.15.47:0/1297355387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e000fcbb0 0x7f2e000fcf90 unknown :-1 s=CLOSED pgs=2508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:48.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.930+0000 7f2e07f95700 1 --2- 172.21.15.47:0/1297355387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e000fd510 0x7f2e000f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:48.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.930+0000 7f2e07f95700 1 -- 172.21.15.47:0/1297355387 >> 172.21.15.47:0/1297355387 conn(0x7f2e000f2820 msgr2=0x7f2e000f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:46:48.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.930+0000 7f2e07f95700 1 -- 172.21.15.47:0/1297355387 shutdown_connections 2024-02-23T00:46:48.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.930+0000 7f2e07f95700 1 -- 172.21.15.47:0/1297355387 wait complete. 2024-02-23T00:46:48.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.930+0000 7f2e07f95700 1 librados: starting msgr at 2024-02-23T00:46:48.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.930+0000 7f2e07f95700 1 librados: starting objecter 2024-02-23T00:46:48.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.931+0000 7f2e07f95700 1 Processor -- start 2024-02-23T00:46:48.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.931+0000 7f2e07f95700 1 -- start start 2024-02-23T00:46:48.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.931+0000 7f2e07f95700 1 librados: setting wanted keys 2024-02-23T00:46:48.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.931+0000 7f2e07f95700 1 librados: calling monclient init 2024-02-23T00:46:48.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.931+0000 7f2e07f95700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e000f7400 0x7f2e0010dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:48.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.931+0000 7f2e07f95700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e000fcbb0 0x7f2e0010e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:48.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.931+0000 7f2e07f95700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2e000fd510 0x7f2e00108180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:48.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.931+0000 7f2e07f95700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e00108670 con 0x7f2e000fcbb0 2024-02-23T00:46:48.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.931+0000 7f2e07f95700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e001087e0 con 0x7f2e000f7400 2024-02-23T00:46:48.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.931+0000 7f2e07f95700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e001089a0 con 0x7f2e000fd510 2024-02-23T00:46:48.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.931+0000 7f2e05530700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e000fcbb0 0x7f2e0010e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:48.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.931+0000 7f2e06532700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2e000fd510 0x7f2e00108180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:48.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.931+0000 7f2e05530700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e000fcbb0 0x7f2e0010e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52492/0 (socket says 172.21.15.47:52492) 2024-02-23T00:46:48.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.931+0000 7f2e05530700 1 -- 172.21.15.47:0/4093876427 learned_addr learned my addr 172.21.15.47:0/4093876427 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:46:48.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.931+0000 7f2e05d31700 1 --2- 172.21.15.47:0/4093876427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e000f7400 0x7f2e0010dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:48.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.931+0000 7f2e06532700 1 -- 172.21.15.47:0/4093876427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e000f7400 msgr2=0x7f2e0010dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:48.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.931+0000 7f2e06532700 1 --2- 172.21.15.47:0/4093876427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e000f7400 0x7f2e0010dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:48.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.931+0000 7f2e06532700 1 -- 172.21.15.47:0/4093876427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e000fcbb0 msgr2=0x7f2e0010e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:48.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.931+0000 7f2e06532700 1 --2- 172.21.15.47:0/4093876427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e000fcbb0 0x7f2e0010e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:48.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.931+0000 7f2e06532700 1 -- 172.21.15.47:0/4093876427 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df80097c0 con 0x7f2e000fd510 2024-02-23T00:46:48.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.931+0000 7f2e05530700 1 --2- 172.21.15.47:0/4093876427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e000fcbb0 0x7f2e0010e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:46:48.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.931+0000 7f2e06532700 1 --2- 172.21.15.47:0/4093876427 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2e000fd510 0x7f2e00108180 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 rx=0x7f2df8001cb0 tx=0x7f2df8004c50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:48.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.931+0000 7f2df6ffd700 10 client.?.objecter ms_handle_connect 0x7f2e000fd510 2024-02-23T00:46:48.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.931+0000 7f2df6ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:46:48.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.932+0000 7f2df6ffd700 1 -- 172.21.15.47:0/4093876427 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df80079b0 con 0x7f2e000fd510 2024-02-23T00:46:48.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.932+0000 7f2df6ffd700 1 -- 172.21.15.47:0/4093876427 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f2df80090d0 con 0x7f2e000fd510 2024-02-23T00:46:48.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.932+0000 7f2e07f95700 1 -- 172.21.15.47:0/4093876427 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dfc005040 con 0x7f2e000fd510 2024-02-23T00:46:48.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.932+0000 7f2df6ffd700 1 -- 172.21.15.47:0/4093876427 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df800fba0 con 0x7f2e000fd510 2024-02-23T00:46:48.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.932+0000 7f2e07f95700 10 client.41141.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:46:48.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.932+0000 7f2e07f95700 1 -- 172.21.15.47:0/4093876427 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e00108f90 con 0x7f2e000fd510 2024-02-23T00:46:48.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.932+0000 7f2e07f95700 1 librados: init done 2024-02-23T00:46:48.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.933+0000 7f2e07f95700 1 -- 172.21.15.47:0/4093876427 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e0005fa60 con 0x7f2e000fd510 2024-02-23T00:46:48.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.934+0000 7f2df6ffd700 1 -- 172.21.15.47:0/4093876427 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f2df8007b10 con 0x7f2e000fd510 2024-02-23T00:46:48.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.934+0000 7f2df6ffd700 1 --2- 172.21.15.47:0/4093876427 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2dec062cc0 0x7f2dec065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:48.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.934+0000 7f2df6ffd700 1 -- 172.21.15.47:0/4093876427 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(335..335 src has 48..335) v4 ==== 9693+0+0 (secure 0 0 0) 0x7f2df807e010 con 0x7f2e000fd510 2024-02-23T00:46:48.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.934+0000 7f2df6ffd700 10 client.41141.objecter ms_dispatch 0x7f2e00001550 osd_map(335..335 src has 48..335) v4 2024-02-23T00:46:48.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.934+0000 7f2df6ffd700 3 client.41141.objecter handle_osd_map got epochs [335,335] > 0 2024-02-23T00:46:48.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.934+0000 7f2df6ffd700 3 client.41141.objecter handle_osd_map decoding full epoch 335 2024-02-23T00:46:48.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.934+0000 7f2e05d31700 1 --2- 172.21.15.47:0/4093876427 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2dec062cc0 0x7f2dec065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:48.413 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:46:48.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.934+0000 7f2df6ffd700 20 client.41141.objecter dump_active .. 0 homeless 2024-02-23T00:46:48.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.935+0000 7f2df6ffd700 10 client.41141.objecter ms_handle_connect 0x7f2dec062cc0 2024-02-23T00:46:48.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.935+0000 7f2e05d31700 1 --2- 172.21.15.47:0/4093876427 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2dec062cc0 0x7f2dec065180 secure :-1 s=READY pgs=3029 cs=0 l=1 rev1=1 rx=0x7f2dfc004ee0 tx=0x7f2dfc00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:48.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:47.938+0000 7f2df6ffd700 1 -- 172.21.15.47:0/4093876427 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f2df804af60 con 0x7f2e000fd510 2024-02-23T00:46:48.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:48.110+0000 7f2e07f95700 1 -- 172.21.15.47:0/4093876427 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2e00109660 con 0x7f2dec062cc0 2024-02-23T00:46:48.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:48.126+0000 7f2df6ffd700 1 -- 172.21.15.47:0/4093876427 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+599027 (secure 0 0 0) 0x7f2e00109660 con 0x7f2dec062cc0 2024-02-23T00:46:48.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:48.410+0000 7f2e07f95700 10 librados: watch_flush enter 2024-02-23T00:46:48.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:48.410+0000 7f2e07f95700 10 librados: watch_flush exit 2024-02-23T00:46:48.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:48.410+0000 7f2e07f95700 20 client.41141.objecter shutdown clearing up homeless session... 2024-02-23T00:46:48.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:48.410+0000 7f2e07f95700 10 client.41141.objecter successfully canceled tick 2024-02-23T00:46:48.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:48.410+0000 7f2e07f95700 1 -- 172.21.15.47:0/4093876427 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2dec062cc0 msgr2=0x7f2dec065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:48.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:48.410+0000 7f2e07f95700 1 --2- 172.21.15.47:0/4093876427 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2dec062cc0 0x7f2dec065180 secure :-1 s=READY pgs=3029 cs=0 l=1 rev1=1 rx=0x7f2dfc004ee0 tx=0x7f2dfc00b040).stop 2024-02-23T00:46:48.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:48.410+0000 7f2e07f95700 1 -- 172.21.15.47:0/4093876427 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2e000fd510 msgr2=0x7f2e00108180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:48.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:48.410+0000 7f2e07f95700 1 --2- 172.21.15.47:0/4093876427 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2e000fd510 0x7f2e00108180 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 rx=0x7f2df8001cb0 tx=0x7f2df8004c50).stop 2024-02-23T00:46:48.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:48.410+0000 7f2e07f95700 1 -- 172.21.15.47:0/4093876427 shutdown_connections 2024-02-23T00:46:48.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:48.410+0000 7f2e07f95700 1 --2- 172.21.15.47:0/4093876427 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2dec062cc0 0x7f2dec065180 unknown :-1 s=CLOSED pgs=3029 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:48.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:48.410+0000 7f2e07f95700 1 --2- 172.21.15.47:0/4093876427 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2e000fd510 0x7f2e00108180 unknown :-1 s=CLOSED pgs=2623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:48.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:48.410+0000 7f2e07f95700 1 --2- 172.21.15.47:0/4093876427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e000f7400 0x7f2e0010dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:48.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:48.410+0000 7f2e07f95700 1 --2- 172.21.15.47:0/4093876427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e000fcbb0 0x7f2e0010e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:48.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:48.410+0000 7f2e07f95700 1 -- 172.21.15.47:0/4093876427 >> 172.21.15.47:0/4093876427 conn(0x7f2e000f2820 msgr2=0x7f2e000f4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:46:48.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:48.410+0000 7f2e07f95700 1 -- 172.21.15.47:0/4093876427 shutdown_connections 2024-02-23T00:46:48.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:48.410+0000 7f2e07f95700 1 -- 172.21.15.47:0/4093876427 wait complete. 2024-02-23T00:46:48.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:48.410+0000 7f2e07f95700 1 librados: shutdown 2024-02-23T00:46:48.426 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-02-23T00:46:48.426 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:46:48.427 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:46:48.528 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:46:48.556 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:48.680 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:46:48.680 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:46:48.682 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:48.682 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:46:48.783 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:46:48.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:47.222809+0000 osd.1 (osd.1) 151 : cluster [DBG] 5.19 deep-scrub starts 2024-02-23T00:46:48.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:47.226325+0000 osd.1 (osd.1) 152 : cluster [DBG] 5.19 deep-scrub ok 2024-02-23T00:46:48.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:47.263585+0000 mgr.y (mgr.14517) 325 : cluster [DBG] pgmap v478: 235 pgs: 235 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-23T00:46:48.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:47.686255+0000 osd.7 (osd.7) 173 : cluster [DBG] 2.1c scrub starts 2024-02-23T00:46:48.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:47.689128+0000 osd.7 (osd.7) 174 : cluster [DBG] 2.1c scrub ok 2024-02-23T00:46:48.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:47.918378+0000 osd.3 (osd.3) 167 : cluster [DBG] 6.b scrub starts 2024-02-23T00:46:48.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:47.919350+0000 osd.3 (osd.3) 168 : cluster [DBG] 6.b scrub ok 2024-02-23T00:46:48.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:48.127039+0000 osd.5 (osd.5) 146 : cluster [DBG] 6.d deep-scrub starts 2024-02-23T00:46:48.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:48.130100+0000 osd.5 (osd.5) 147 : cluster [DBG] 6.d deep-scrub ok 2024-02-23T00:46:48.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:48.170439+0000 mon.a (mon.0) 2195 : audit [INF] from='client.? 172.21.15.47:0/2392849431' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi047-82386-56"}]': finished 2024-02-23T00:46:48.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:48.170521+0000 mon.a (mon.0) 2196 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "16"}]': finished 2024-02-23T00:46:48.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:48.170574+0000 mon.a (mon.0) 2197 : cluster [DBG] osdmap e336: 8 total, 8 up, 8 in 2024-02-23T00:46:48.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:48.193822+0000 mon.c (mon.2) 251 : audit [INF] from='client.? 172.21.15.47:0/1633196565' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi047-82386-57"}]: dispatch 2024-02-23T00:46:48.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:48.194173+0000 mon.a (mon.0) 2198 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi047-82386-57"}]: dispatch 2024-02-23T00:46:48.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:48.194665+0000 mon.c (mon.2) 252 : audit [INF] from='client.? 172.21.15.47:0/1633196565' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi047-82386-57"}]: dispatch 2024-02-23T00:46:48.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:48.194931+0000 mon.a (mon.0) 2199 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi047-82386-57"}]: dispatch 2024-02-23T00:46:48.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:48.195034+0000 mon.b (mon.1) 370 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:48.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:48.195465+0000 mon.c (mon.2) 253 : audit [INF] from='client.? 172.21.15.47:0/1633196565' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi047-82386-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:48.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:48.195548+0000 mon.b (mon.1) 371 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:48.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:48.195877+0000 mon.a (mon.0) 2200 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi047-82386-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:48.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:48.195967+0000 mon.b (mon.1) 372 : audit [INF] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-23T00:46:48.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:48.196181+0000 mon.a (mon.0) 2201 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-23T00:46:48.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:47.222809+0000 osd.1 (osd.1) 151 : cluster [DBG] 5.19 deep-scrub starts 2024-02-23T00:46:48.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:47.226325+0000 osd.1 (osd.1) 152 : cluster [DBG] 5.19 deep-scrub ok 2024-02-23T00:46:48.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:47.263585+0000 mgr.y (mgr.14517) 325 : cluster [DBG] pgmap v478: 235 pgs: 235 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-23T00:46:48.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:47.686255+0000 osd.7 (osd.7) 173 : cluster [DBG] 2.1c scrub starts 2024-02-23T00:46:48.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:47.689128+0000 osd.7 (osd.7) 174 : cluster [DBG] 2.1c scrub ok 2024-02-23T00:46:48.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:47.918378+0000 osd.3 (osd.3) 167 : cluster [DBG] 6.b scrub starts 2024-02-23T00:46:48.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:47.919350+0000 osd.3 (osd.3) 168 : cluster [DBG] 6.b scrub ok 2024-02-23T00:46:48.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:48.127039+0000 osd.5 (osd.5) 146 : cluster [DBG] 6.d deep-scrub starts 2024-02-23T00:46:48.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:48.130100+0000 osd.5 (osd.5) 147 : cluster [DBG] 6.d deep-scrub ok 2024-02-23T00:46:48.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:48.170439+0000 mon.a (mon.0) 2195 : audit [INF] from='client.? 172.21.15.47:0/2392849431' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi047-82386-56"}]': finished 2024-02-23T00:46:48.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:48.170521+0000 mon.a (mon.0) 2196 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "16"}]': finished 2024-02-23T00:46:48.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:48.170574+0000 mon.a (mon.0) 2197 : cluster [DBG] osdmap e336: 8 total, 8 up, 8 in 2024-02-23T00:46:48.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:48.193822+0000 mon.c (mon.2) 251 : audit [INF] from='client.? 172.21.15.47:0/1633196565' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi047-82386-57"}]: dispatch 2024-02-23T00:46:48.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:48.194173+0000 mon.a (mon.0) 2198 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi047-82386-57"}]: dispatch 2024-02-23T00:46:48.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:48.194665+0000 mon.c (mon.2) 252 : audit [INF] from='client.? 172.21.15.47:0/1633196565' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi047-82386-57"}]: dispatch 2024-02-23T00:46:48.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:48.194931+0000 mon.a (mon.0) 2199 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi047-82386-57"}]: dispatch 2024-02-23T00:46:48.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:48.195034+0000 mon.b (mon.1) 370 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:48.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:48.195465+0000 mon.c (mon.2) 253 : audit [INF] from='client.? 172.21.15.47:0/1633196565' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi047-82386-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:48.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:48.195548+0000 mon.b (mon.1) 371 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:48.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:48.195877+0000 mon.a (mon.0) 2200 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi047-82386-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:48.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:48.195967+0000 mon.b (mon.1) 372 : audit [INF] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-23T00:46:48.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:48.196181+0000 mon.a (mon.0) 2201 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-23T00:46:48.811 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:48.910 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:46:48.990 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:47.222809+0000 osd.1 (osd.1) 151 : cluster [DBG] 5.19 deep-scrub starts 2024-02-23T00:46:48.990 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:47.226325+0000 osd.1 (osd.1) 152 : cluster [DBG] 5.19 deep-scrub ok 2024-02-23T00:46:48.991 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:47.263585+0000 mgr.y (mgr.14517) 325 : cluster [DBG] pgmap v478: 235 pgs: 235 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-23T00:46:48.991 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:47.686255+0000 osd.7 (osd.7) 173 : cluster [DBG] 2.1c scrub starts 2024-02-23T00:46:48.991 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:47.689128+0000 osd.7 (osd.7) 174 : cluster [DBG] 2.1c scrub ok 2024-02-23T00:46:48.991 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:47.918378+0000 osd.3 (osd.3) 167 : cluster [DBG] 6.b scrub starts 2024-02-23T00:46:48.991 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:47.919350+0000 osd.3 (osd.3) 168 : cluster [DBG] 6.b scrub ok 2024-02-23T00:46:48.991 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:48.127039+0000 osd.5 (osd.5) 146 : cluster [DBG] 6.d deep-scrub starts 2024-02-23T00:46:48.991 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:48.130100+0000 osd.5 (osd.5) 147 : cluster [DBG] 6.d deep-scrub ok 2024-02-23T00:46:48.991 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:48.170439+0000 mon.a (mon.0) 2195 : audit [INF] from='client.? 172.21.15.47:0/2392849431' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi047-82386-56"}]': finished 2024-02-23T00:46:48.991 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:48.170521+0000 mon.a (mon.0) 2196 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "16"}]': finished 2024-02-23T00:46:48.991 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:48.170574+0000 mon.a (mon.0) 2197 : cluster [DBG] osdmap e336: 8 total, 8 up, 8 in 2024-02-23T00:46:48.991 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:48.193822+0000 mon.c (mon.2) 251 : audit [INF] from='client.? 172.21.15.47:0/1633196565' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi047-82386-57"}]: dispatch 2024-02-23T00:46:48.991 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:48.194173+0000 mon.a (mon.0) 2198 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi047-82386-57"}]: dispatch 2024-02-23T00:46:48.991 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:48.194665+0000 mon.c (mon.2) 252 : audit [INF] from='client.? 172.21.15.47:0/1633196565' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi047-82386-57"}]: dispatch 2024-02-23T00:46:48.991 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:48.194931+0000 mon.a (mon.0) 2199 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi047-82386-57"}]: dispatch 2024-02-23T00:46:48.992 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:48.195034+0000 mon.b (mon.1) 370 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:48.992 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:48.195465+0000 mon.c (mon.2) 253 : audit [INF] from='client.? 172.21.15.47:0/1633196565' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi047-82386-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:48.992 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:48.195548+0000 mon.b (mon.1) 371 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:48.992 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:48.195877+0000 mon.a (mon.0) 2200 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi047-82386-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:48.992 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:48.195967+0000 mon.b (mon.1) 372 : audit [INF] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-23T00:46:48.992 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:48.196181+0000 mon.a (mon.0) 2201 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-23T00:46:48.992 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:48.993 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:46:49.093 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:46:49.122 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:49.607 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:46:49.607 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-02-23T00:46:49.607 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-02-23T00:46:49.607 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-02-23T00:46:49.607 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.607 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.24769.0:15 8.14 8:2afeb74e:test-rados-api-smithi047-82428-7::foo:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected e112)", 2024-02-23T00:46:49.607 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:00.985676+0000", 2024-02-23T00:46:49.608 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 228.61749062999999, 2024-02-23T00:46:49.608 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013198675, 2024-02-23T00:46:49.608 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:46:49.608 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:49.608 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:46:49.608 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.24769", 2024-02-23T00:46:49.608 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/820656190", 2024-02-23T00:46:49.608 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 15 2024-02-23T00:46:49.608 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.608 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:46:49.608 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.609 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:46:49.609 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.985676+0000", 2024-02-23T00:46:49.609 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:46:49.609 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.609 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.609 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:46:49.609 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.985676+0000", 2024-02-23T00:46:49.609 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:46:49.609 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.609 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.609 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:46:49.609 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.985675+0000", 2024-02-23T00:46:49.609 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6940000000000003e-06 2024-02-23T00:46:49.609 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.610 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.610 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:46:49.610 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.985680+0000", 2024-02-23T00:46:49.610 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.9999999999999999e-06 2024-02-23T00:46:49.610 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.610 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.610 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:46:49.610 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.985687+0000", 2024-02-23T00:46:49.610 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3930000000000001e-06 2024-02-23T00:46:49.610 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.610 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.610 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:46:49.610 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.985691+0000", 2024-02-23T00:46:49.610 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.5889999999999997e-05 2024-02-23T00:46:49.611 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.611 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.611 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:46:49.611 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.985727+0000", 2024-02-23T00:46:49.611 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9975000000000006e-05 2024-02-23T00:46:49.611 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.611 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.611 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:46:49.611 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.985807+0000", 2024-02-23T00:46:49.612 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000125964 2024-02-23T00:46:49.612 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.612 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.612 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0,2", 2024-02-23T00:46:49.612 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.985933+0000", 2024-02-23T00:46:49.612 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000320188 2024-02-23T00:46:49.612 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.612 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.613 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:46:49.613 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.986253+0000", 2024-02-23T00:46:49.613 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0063198509999999996 2024-02-23T00:46:49.613 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.613 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.613 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:46:49.613 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.992573+0000", 2024-02-23T00:46:49.613 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061739760000000003 2024-02-23T00:46:49.614 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.614 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.614 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:46:49.614 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.998747+0000", 2024-02-23T00:46:49.614 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0014999999999999e-05 2024-02-23T00:46:49.614 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.614 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.614 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:46:49.615 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.998827+0000", 2024-02-23T00:46:49.615 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7908999999999998e-05 2024-02-23T00:46:49.615 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.615 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.615 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:46:49.615 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.998875+0000", 2024-02-23T00:46:49.615 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013198675 2024-02-23T00:46:49.615 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.616 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:46:49.616 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.616 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.616 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.616 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.15165.0:8 29.10 e112/109 29:0cdc28ab:::47.obj:head v 112'27, mlcod=112'25)", 2024-02-23T00:46:49.616 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:01.975830+0000", 2024-02-23T00:46:49.616 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 227.62733703000001, 2024-02-23T00:46:49.616 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022123151000000001, 2024-02-23T00:46:49.616 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:46:49.617 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:49.617 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:46:49.617 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.617 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:46:49.617 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.975830+0000", 2024-02-23T00:46:49.617 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:46:49.617 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.618 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.618 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:46:49.618 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.975830+0000", 2024-02-23T00:46:49.618 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:46:49.618 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.618 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.618 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:46:49.618 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.975828+0000", 2024-02-23T00:46:49.618 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5201000000000001e-05 2024-02-23T00:46:49.619 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.619 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.619 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:46:49.619 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.975843+0000", 2024-02-23T00:46:49.619 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9936999999999999e-05 2024-02-23T00:46:49.619 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.619 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.619 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:46:49.620 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.975863+0000", 2024-02-23T00:46:49.620 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.5790000000000004e-06 2024-02-23T00:46:49.620 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.620 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.620 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:46:49.620 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.975872+0000", 2024-02-23T00:46:49.620 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.2378000000000005e-05 2024-02-23T00:46:49.621 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.621 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.621 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:46:49.621 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.975934+0000", 2024-02-23T00:46:49.621 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7061000000000001e-05 2024-02-23T00:46:49.621 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.621 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.621 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:46:49.621 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.975961+0000", 2024-02-23T00:46:49.622 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021960121999999999 2024-02-23T00:46:49.622 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.622 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.622 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:46:49.622 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.997922+0000", 2024-02-23T00:46:49.622 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.1309999999999997e-05 2024-02-23T00:46:49.622 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.622 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.623 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:46:49.623 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.997953+0000", 2024-02-23T00:46:49.623 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022123151000000001 2024-02-23T00:46:49.623 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.623 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:46:49.623 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.623 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.623 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.624 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.15165.0:9 29.17 e112/109 29:eb69ee62:::27.obj:head v 112'18, mlcod=112'16)", 2024-02-23T00:46:49.624 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:01.998811+0000", 2024-02-23T00:46:49.624 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 227.60435583200001, 2024-02-23T00:46:49.624 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.018678802000000001, 2024-02-23T00:46:49.624 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:46:49.624 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:49.624 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:46:49.624 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.625 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:46:49.625 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.998811+0000", 2024-02-23T00:46:49.625 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:46:49.625 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.625 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.625 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:46:49.625 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.998811+0000", 2024-02-23T00:46:49.625 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:46:49.625 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.625 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.625 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:46:49.625 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.998810+0000", 2024-02-23T00:46:49.626 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1049999999999995e-06 2024-02-23T00:46:49.626 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.626 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.626 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:46:49.626 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.998818+0000", 2024-02-23T00:46:49.626 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2082999999999999e-05 2024-02-23T00:46:49.626 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.626 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.626 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:46:49.626 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.998830+0000", 2024-02-23T00:46:49.626 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8040000000000004e-06 2024-02-23T00:46:49.626 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.626 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.626 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:46:49.626 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.998837+0000", 2024-02-23T00:46:49.627 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0862000000000003e-05 2024-02-23T00:46:49.627 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.627 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.627 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:46:49.627 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.998878+0000", 2024-02-23T00:46:49.627 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8145999999999999e-05 2024-02-23T00:46:49.627 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.627 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.627 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:46:49.627 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.998896+0000", 2024-02-23T00:46:49.627 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.018492277000000001 2024-02-23T00:46:49.627 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.627 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.627 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:46:49.628 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.017388+0000", 2024-02-23T00:46:49.628 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000101394 2024-02-23T00:46:49.628 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.628 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.628 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:46:49.628 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.017490+0000", 2024-02-23T00:46:49.628 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.018678802000000001 2024-02-23T00:46:49.628 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.628 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:46:49.628 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.628 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.628 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.628 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.15165.0:35 29.2 29:452ae4ca:::6.obj:head [write 0~225] snapc 0=[] ondisk+write+known_if_redirected e112)", 2024-02-23T00:46:49.629 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:02.058811+0000", 2024-02-23T00:46:49.629 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 227.544355596, 2024-02-23T00:46:49.629 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0050541459999999998, 2024-02-23T00:46:49.629 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:46:49.629 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:49.629 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:46:49.629 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.15165", 2024-02-23T00:46:49.629 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1835198847", 2024-02-23T00:46:49.629 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 35 2024-02-23T00:46:49.629 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.629 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:46:49.630 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.630 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:46:49.630 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.058811+0000", 2024-02-23T00:46:49.630 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:46:49.630 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.630 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.630 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:46:49.630 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.058811+0000", 2024-02-23T00:46:49.630 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:46:49.630 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.630 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.630 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:46:49.630 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.058809+0000", 2024-02-23T00:46:49.630 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0716e-05 2024-02-23T00:46:49.631 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.631 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.631 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:46:49.631 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.058820+0000", 2024-02-23T00:46:49.631 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6276000000000001e-05 2024-02-23T00:46:49.631 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.631 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.631 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:46:49.631 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.058836+0000", 2024-02-23T00:46:49.631 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3060000000000003e-06 2024-02-23T00:46:49.631 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.631 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.631 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:46:49.632 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.058842+0000", 2024-02-23T00:46:49.632 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.2403e-05 2024-02-23T00:46:49.632 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.632 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.632 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:46:49.632 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.058885+0000", 2024-02-23T00:46:49.632 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011220100000000001 2024-02-23T00:46:49.632 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.632 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.632 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:46:49.632 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.058997+0000", 2024-02-23T00:46:49.632 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011846899999999999 2024-02-23T00:46:49.632 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.632 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.632 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0,2", 2024-02-23T00:46:49.633 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.059115+0000", 2024-02-23T00:46:49.633 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000424592 2024-02-23T00:46:49.633 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.633 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.633 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:46:49.633 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.059540+0000", 2024-02-23T00:46:49.633 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000268719 2024-02-23T00:46:49.633 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.633 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.633 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:46:49.633 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.059809+0000", 2024-02-23T00:46:49.633 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.003963622 2024-02-23T00:46:49.633 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.633 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.634 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:46:49.634 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.063772+0000", 2024-02-23T00:46:49.634 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.2289e-05 2024-02-23T00:46:49.634 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.634 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.634 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:46:49.634 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.063825+0000", 2024-02-23T00:46:49.634 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0559999999999998e-05 2024-02-23T00:46:49.634 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.634 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.634 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:46:49.634 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.063865+0000", 2024-02-23T00:46:49.634 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0050541459999999998 2024-02-23T00:46:49.634 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.634 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:46:49.635 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.635 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.635 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.635 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.15099.0:3 34.9 34:9294e480:nspace::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e114)", 2024-02-23T00:46:49.635 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:03.239954+0000", 2024-02-23T00:46:49.635 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 226.36321229199999, 2024-02-23T00:46:49.635 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0064735499999999998, 2024-02-23T00:46:49.635 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:46:49.635 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:49.635 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:46:49.635 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.15099", 2024-02-23T00:46:49.635 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/3592004948", 2024-02-23T00:46:49.635 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 3 2024-02-23T00:46:49.635 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.636 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:46:49.636 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.636 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:46:49.636 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.239954+0000", 2024-02-23T00:46:49.636 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:46:49.636 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.636 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.636 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:46:49.636 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.239954+0000", 2024-02-23T00:46:49.636 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:46:49.636 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.636 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.636 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:46:49.636 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.239953+0000", 2024-02-23T00:46:49.637 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.327e-06 2024-02-23T00:46:49.637 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.637 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.637 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:46:49.637 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.239958+0000", 2024-02-23T00:46:49.637 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.7379999999999994e-06 2024-02-23T00:46:49.637 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.637 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.637 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:46:49.637 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.239966+0000", 2024-02-23T00:46:49.637 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4240000000000001e-06 2024-02-23T00:46:49.637 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.637 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.637 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:46:49.637 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.239972+0000", 2024-02-23T00:46:49.638 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.2553999999999999e-05 2024-02-23T00:46:49.638 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.638 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.638 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:46:49.638 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.240014+0000", 2024-02-23T00:46:49.638 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1692000000000002e-05 2024-02-23T00:46:49.638 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.638 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.638 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:46:49.638 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.240106+0000", 2024-02-23T00:46:49.638 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103417 2024-02-23T00:46:49.638 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.638 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.638 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0,3", 2024-02-23T00:46:49.639 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.240209+0000", 2024-02-23T00:46:49.639 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00036850300000000001 2024-02-23T00:46:49.639 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.639 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.639 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:46:49.639 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.240578+0000", 2024-02-23T00:46:49.639 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0052171199999999996 2024-02-23T00:46:49.639 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.639 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.639 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:46:49.639 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.245795+0000", 2024-02-23T00:46:49.639 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00057471099999999999 2024-02-23T00:46:49.639 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.639 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.639 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:46:49.639 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.246370+0000", 2024-02-23T00:46:49.640 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.2317999999999998e-05 2024-02-23T00:46:49.640 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.640 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.640 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:46:49.640 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.246402+0000", 2024-02-23T00:46:49.640 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.6115e-05 2024-02-23T00:46:49.640 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.640 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.640 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:46:49.640 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.246428+0000", 2024-02-23T00:46:49.640 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0064735499999999998 2024-02-23T00:46:49.640 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.640 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:46:49.641 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.641 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.641 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.641 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.25120.0:2 41.1a 41:5c6776db:::0.obj:head [write 0~103] snapc 0=[] ondisk+write+known_if_redirected e115)", 2024-02-23T00:46:49.641 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:04.257379+0000", 2024-02-23T00:46:49.641 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 225.345788032, 2024-02-23T00:46:49.641 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.042556126, 2024-02-23T00:46:49.641 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:46:49.641 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:49.641 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:46:49.641 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.25120", 2024-02-23T00:46:49.641 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/484148516", 2024-02-23T00:46:49.641 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 2 2024-02-23T00:46:49.642 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.642 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:46:49.642 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.642 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:46:49.642 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.257379+0000", 2024-02-23T00:46:49.642 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:46:49.642 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.642 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.642 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:46:49.642 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.257379+0000", 2024-02-23T00:46:49.642 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999976 2024-02-23T00:46:49.643 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.643 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.643 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:46:49.643 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.257376+0000", 2024-02-23T00:46:49.643 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1412e-05 2024-02-23T00:46:49.643 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.643 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.643 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:46:49.643 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.257388+0000", 2024-02-23T00:46:49.643 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5347999999999999e-05 2024-02-23T00:46:49.643 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.643 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.643 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:46:49.644 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.257403+0000", 2024-02-23T00:46:49.644 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0669999999999992e-06 2024-02-23T00:46:49.644 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.644 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.644 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:46:49.644 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.257411+0000", 2024-02-23T00:46:49.644 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012477369 2024-02-23T00:46:49.644 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.644 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.644 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:46:49.644 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.269889+0000", 2024-02-23T00:46:49.644 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5549999999999997e-06 2024-02-23T00:46:49.644 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.644 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.645 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:46:49.645 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.269896+0000", 2024-02-23T00:46:49.645 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.98403268899999996 2024-02-23T00:46:49.645 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.645 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.645 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:46:49.645 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:05.253929+0000", 2024-02-23T00:46:49.645 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8703999999999996e-05 2024-02-23T00:46:49.645 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.645 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.645 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:46:49.645 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:05.254028+0000", 2024-02-23T00:46:49.645 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011678199999999999 2024-02-23T00:46:49.645 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.645 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.646 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1,2", 2024-02-23T00:46:49.646 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:05.254144+0000", 2024-02-23T00:46:49.646 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0017152720000000001 2024-02-23T00:46:49.646 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.646 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.646 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:46:49.646 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:05.255860+0000", 2024-02-23T00:46:49.646 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035976222000000002 2024-02-23T00:46:49.646 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.646 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.646 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:46:49.646 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:05.291836+0000", 2024-02-23T00:46:49.646 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0080063189999999996 2024-02-23T00:46:49.646 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.647 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.647 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:46:49.647 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:05.299842+0000", 2024-02-23T00:46:49.647 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1570000000000003e-05 2024-02-23T00:46:49.647 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.647 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.647 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:46:49.647 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:05.299894+0000", 2024-02-23T00:46:49.647 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1026000000000003e-05 2024-02-23T00:46:49.647 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.647 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.647 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:46:49.647 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:05.299935+0000", 2024-02-23T00:46:49.647 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.042556126 2024-02-23T00:46:49.648 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.648 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:46:49.648 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.648 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.648 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.648 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.25132.0:1 45.1s0 45:88c56992:test-rados-api-smithi047-82394-16::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e117)", 2024-02-23T00:46:49.648 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:06.598180+0000", 2024-02-23T00:46:49.648 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 223.00498695499999, 2024-02-23T00:46:49.648 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.414473189, 2024-02-23T00:46:49.648 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:46:49.648 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:49.648 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:46:49.648 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.25132", 2024-02-23T00:46:49.648 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1728200087", 2024-02-23T00:46:49.649 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1 2024-02-23T00:46:49.649 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.649 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:46:49.649 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.649 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:46:49.649 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:06.598180+0000", 2024-02-23T00:46:49.649 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:46:49.649 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.649 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.649 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:46:49.649 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:06.598180+0000", 2024-02-23T00:46:49.649 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:46:49.649 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.649 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.650 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:46:49.650 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:06.598178+0000", 2024-02-23T00:46:49.650 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.221e-06 2024-02-23T00:46:49.650 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.650 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.650 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:46:49.650 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:06.598185+0000", 2024-02-23T00:46:49.650 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1851000000000001e-05 2024-02-23T00:46:49.650 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.650 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.650 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:46:49.650 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:06.598197+0000", 2024-02-23T00:46:49.650 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9759999999999995e-06 2024-02-23T00:46:49.650 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.651 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.651 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:46:49.651 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:06.598205+0000", 2024-02-23T00:46:49.651 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1051999999999998e-05 2024-02-23T00:46:49.651 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.651 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.651 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:46:49.651 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:06.598266+0000", 2024-02-23T00:46:49.651 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5469999999999999e-06 2024-02-23T00:46:49.651 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.651 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.651 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:46:49.651 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:06.598269+0000", 2024-02-23T00:46:49.652 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.404963942 2024-02-23T00:46:49.652 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.652 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.652 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:46:49.652 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:08.003233+0000", 2024-02-23T00:46:49.652 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000114925 2024-02-23T00:46:49.652 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.652 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.652 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:46:49.652 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:08.003348+0000", 2024-02-23T00:46:49.652 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038947999999999999 2024-02-23T00:46:49.652 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.652 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.653 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:46:49.653 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:08.003737+0000", 2024-02-23T00:46:49.653 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00066348700000000004 2024-02-23T00:46:49.653 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.653 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.653 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:46:49.653 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:08.004401+0000", 2024-02-23T00:46:49.653 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0082121090000000004 2024-02-23T00:46:49.653 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.653 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.653 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:46:49.653 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:08.012613+0000", 2024-02-23T00:46:49.653 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0266999999999999e-05 2024-02-23T00:46:49.654 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.654 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.654 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:46:49.654 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:08.012653+0000", 2024-02-23T00:46:49.654 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.414473189 2024-02-23T00:46:49.654 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.654 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:46:49.654 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.654 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.654 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.654 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.15147.0:1 44.0s0 44:1ecacb5d:test-rados-api-smithi047-82504-7::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e117)", 2024-02-23T00:46:49.654 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:07.476730+0000", 2024-02-23T00:46:49.654 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 222.12643630700001, 2024-02-23T00:46:49.654 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.52719580399999999, 2024-02-23T00:46:49.655 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:46:49.655 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:49.655 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:46:49.655 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.15147", 2024-02-23T00:46:49.655 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/640613617", 2024-02-23T00:46:49.655 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1 2024-02-23T00:46:49.655 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.655 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:46:49.655 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.655 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:46:49.655 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.476730+0000", 2024-02-23T00:46:49.655 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:46:49.655 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.655 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.656 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:46:49.656 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.476730+0000", 2024-02-23T00:46:49.656 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999976 2024-02-23T00:46:49.656 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.656 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.656 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:46:49.656 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.476728+0000", 2024-02-23T00:46:49.656 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4539999999999996e-06 2024-02-23T00:46:49.656 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.656 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.656 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:46:49.656 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.476738+0000", 2024-02-23T00:46:49.656 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1059e-05 2024-02-23T00:46:49.657 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.657 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.657 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:46:49.657 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.476749+0000", 2024-02-23T00:46:49.657 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.4679999999999992e-06 2024-02-23T00:46:49.657 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.657 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.657 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:46:49.657 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.476757+0000", 2024-02-23T00:46:49.657 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5839000000000004e-05 2024-02-23T00:46:49.657 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.657 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.657 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:46:49.657 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.476823+0000", 2024-02-23T00:46:49.657 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0300000000000001e-06 2024-02-23T00:46:49.658 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.658 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.658 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:46:49.658 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.476828+0000", 2024-02-23T00:46:49.658 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.48895222900000002 2024-02-23T00:46:49.658 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.658 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.658 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:46:49.658 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.965780+0000", 2024-02-23T00:46:49.658 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012051400000000001 2024-02-23T00:46:49.658 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.658 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.658 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:46:49.658 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.965901+0000", 2024-02-23T00:46:49.659 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040527500000000003 2024-02-23T00:46:49.659 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.659 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.659 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:46:49.659 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.966306+0000", 2024-02-23T00:46:49.659 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0058061459999999999 2024-02-23T00:46:49.659 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.659 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.659 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:46:49.659 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.972112+0000", 2024-02-23T00:46:49.659 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.031746051999999997 2024-02-23T00:46:49.659 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.659 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.659 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:46:49.659 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:08.003858+0000", 2024-02-23T00:46:49.660 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7968999999999999e-05 2024-02-23T00:46:49.660 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.660 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.660 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:46:49.660 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:08.003926+0000", 2024-02-23T00:46:49.660 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.52719580399999999 2024-02-23T00:46:49.660 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.660 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:46:49.660 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.660 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.660 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.660 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.31603.0:1 79.2s0 79:55492aac:test-rados-api-smithi047-82487-10::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e139)", 2024-02-23T00:46:49.661 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:31.194267+0000", 2024-02-23T00:46:49.661 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 198.40889940400001, 2024-02-23T00:46:49.661 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1197622819999999, 2024-02-23T00:46:49.661 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:46:49.661 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:49.661 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:46:49.661 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.31603", 2024-02-23T00:46:49.661 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1361361966", 2024-02-23T00:46:49.661 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1 2024-02-23T00:46:49.661 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.661 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:46:49.661 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.661 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:46:49.661 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:31.194267+0000", 2024-02-23T00:46:49.662 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:46:49.662 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.662 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.662 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:46:49.662 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:31.194267+0000", 2024-02-23T00:46:49.662 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:46:49.662 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.662 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.662 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:46:49.662 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:31.194266+0000", 2024-02-23T00:46:49.662 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.2689999999999999e-06 2024-02-23T00:46:49.662 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.662 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.663 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:46:49.663 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:31.194271+0000", 2024-02-23T00:46:49.663 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.6400000000000003e-06 2024-02-23T00:46:49.663 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.663 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.663 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:46:49.663 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:31.194280+0000", 2024-02-23T00:46:49.663 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4630000000000004e-06 2024-02-23T00:46:49.663 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.663 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.663 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:46:49.663 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:31.194288+0000", 2024-02-23T00:46:49.663 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7209999999999998e-05 2024-02-23T00:46:49.663 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.664 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.664 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:46:49.664 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:31.194325+0000", 2024-02-23T00:46:49.664 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7709999999999998e-06 2024-02-23T00:46:49.664 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.664 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.664 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:46:49.664 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:31.194328+0000", 2024-02-23T00:46:49.664 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.077833724 2024-02-23T00:46:49.664 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.664 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.664 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:46:49.664 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.272161+0000", 2024-02-23T00:46:49.665 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013754699999999999 2024-02-23T00:46:49.665 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.665 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.665 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:46:49.665 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.272299+0000", 2024-02-23T00:46:49.665 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00036407399999999998 2024-02-23T00:46:49.665 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.665 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.665 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:46:49.665 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.272663+0000", 2024-02-23T00:46:49.665 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031999099999999999 2024-02-23T00:46:49.665 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.665 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.665 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:46:49.665 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.272983+0000", 2024-02-23T00:46:49.665 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.040999727999999999 2024-02-23T00:46:49.666 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.666 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.666 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:46:49.666 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.313983+0000", 2024-02-23T00:46:49.666 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7034999999999999e-05 2024-02-23T00:46:49.666 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.666 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.666 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:46:49.666 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.314030+0000", 2024-02-23T00:46:49.666 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1197622819999999 2024-02-23T00:46:49.666 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.666 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:46:49.666 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.666 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.666 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.667 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.24775.0:2077 20.1d 20:bda0dbd6:test-rados-api-smithi047-82457-17::small.copy:head [copy-from ver 0] snapc 0=[] ondisk+write+known_if_redirected e112)", 2024-02-23T00:46:49.667 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:32.070997+0000", 2024-02-23T00:46:49.667 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 197.53217001100001, 2024-02-23T00:46:49.667 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083743199999999993, 2024-02-23T00:46:49.667 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:46:49.667 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:49.667 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:46:49.667 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.24775", 2024-02-23T00:46:49.667 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1191937413", 2024-02-23T00:46:49.667 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 2077 2024-02-23T00:46:49.667 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.667 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:46:49.667 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.667 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:46:49.668 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.070997+0000", 2024-02-23T00:46:49.668 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:46:49.668 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.668 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.668 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:46:49.668 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.070997+0000", 2024-02-23T00:46:49.668 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:46:49.668 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.668 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.668 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:46:49.668 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.070995+0000", 2024-02-23T00:46:49.668 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2429000000000001e-05 2024-02-23T00:46:49.668 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.668 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.669 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:46:49.669 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.071007+0000", 2024-02-23T00:46:49.669 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5676999999999999e-05 2024-02-23T00:46:49.669 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.669 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.669 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:46:49.669 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.071023+0000", 2024-02-23T00:46:49.669 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.9829999999999999e-06 2024-02-23T00:46:49.669 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.669 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.669 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:46:49.669 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.071030+0000", 2024-02-23T00:46:49.669 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.3088e-05 2024-02-23T00:46:49.669 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.670 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.670 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:46:49.670 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.071123+0000", 2024-02-23T00:46:49.670 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015489200000000001 2024-02-23T00:46:49.670 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.670 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.670 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:46:49.670 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.071278+0000", 2024-02-23T00:46:49.670 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097897200000000005 2024-02-23T00:46:49.670 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.670 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.670 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0,7", 2024-02-23T00:46:49.670 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.072257+0000", 2024-02-23T00:46:49.670 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00039726399999999998 2024-02-23T00:46:49.671 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.671 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.671 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:46:49.671 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.072654+0000", 2024-02-23T00:46:49.671 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000233985 2024-02-23T00:46:49.671 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.671 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.671 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:46:49.671 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.072888+0000", 2024-02-23T00:46:49.671 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0064254150000000003 2024-02-23T00:46:49.671 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.671 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.671 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:46:49.672 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.079313+0000", 2024-02-23T00:46:49.672 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.1680000000000002e-05 2024-02-23T00:46:49.672 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.672 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.672 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:46:49.672 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.079345+0000", 2024-02-23T00:46:49.672 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5998000000000001e-05 2024-02-23T00:46:49.672 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.672 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.672 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:46:49.672 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.079371+0000", 2024-02-23T00:46:49.672 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083743199999999993 2024-02-23T00:46:49.672 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.673 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:46:49.673 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.673 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.673 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.673 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.15270.0:1006 93.b 93:d5ab6721:test-rados-api-smithi047-82465-14::foo54:5 [read 0~1] snapc 0=[] ondisk+read+known_if_redirected e162)", 2024-02-23T00:46:49.673 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:54.437968+0000", 2024-02-23T00:46:49.673 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 175.165198678, 2024-02-23T00:46:49.673 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0049791150000000001, 2024-02-23T00:46:49.673 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:46:49.673 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:46:49.673 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:46:49.673 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.15270", 2024-02-23T00:46:49.673 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/4133111990", 2024-02-23T00:46:49.673 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1006 2024-02-23T00:46:49.673 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.674 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:46:49.674 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.674 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:46:49.674 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:54.437968+0000", 2024-02-23T00:46:49.674 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:46:49.674 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.674 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.674 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:46:49.674 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:54.437968+0000", 2024-02-23T00:46:49.674 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:46:49.674 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.674 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.674 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:46:49.675 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:54.437967+0000", 2024-02-23T00:46:49.675 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3979999999999998e-06 2024-02-23T00:46:49.675 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.675 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.675 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:46:49.675 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:54.437973+0000", 2024-02-23T00:46:49.675 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0387999999999999e-05 2024-02-23T00:46:49.675 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.675 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.675 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:46:49.675 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:54.437983+0000", 2024-02-23T00:46:49.675 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6410000000000001e-06 2024-02-23T00:46:49.675 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.675 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.676 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:46:49.676 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:54.437988+0000", 2024-02-23T00:46:49.676 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.2209000000000003e-05 2024-02-23T00:46:49.676 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.676 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.676 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:46:49.676 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:54.438020+0000", 2024-02-23T00:46:49.676 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00018767600000000001 2024-02-23T00:46:49.676 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.676 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.676 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for blocked object", 2024-02-23T00:46:49.676 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:54.438207+0000", 2024-02-23T00:46:49.676 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00082996299999999997 2024-02-23T00:46:49.676 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.677 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.677 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:46:49.677 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:54.439037+0000", 2024-02-23T00:46:49.677 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000105398 2024-02-23T00:46:49.677 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.677 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.677 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for rw locks", 2024-02-23T00:46:49.677 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:54.439143+0000", 2024-02-23T00:46:49.677 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00075957100000000001 2024-02-23T00:46:49.677 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.677 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.677 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:46:49.677 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:54.439902+0000", 2024-02-23T00:46:49.677 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000105351 2024-02-23T00:46:49.677 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.678 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.678 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:46:49.678 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:54.440008+0000", 2024-02-23T00:46:49.678 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0029394059999999999 2024-02-23T00:46:49.678 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.678 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.678 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:46:49.678 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:54.442947+0000", 2024-02-23T00:46:49.678 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0049791150000000001 2024-02-23T00:46:49.678 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.678 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:46:49.678 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.678 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.678 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.678 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(osd.1.0:425 62.11 e195/126 62:8f236abd:test-rados-api-smithi047-82465-14::foo31:7 v 195'33, mlcod=193'29)", 2024-02-23T00:46:49.679 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:44:27.133318+0000", 2024-02-23T00:46:49.679 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 142.46984847900001, 2024-02-23T00:46:49.679 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010179812, 2024-02-23T00:46:49.679 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:46:49.679 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:49.679 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:46:49.679 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.679 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:46:49.679 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:27.133318+0000", 2024-02-23T00:46:49.679 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:46:49.679 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.679 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.679 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:46:49.679 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:27.133318+0000", 2024-02-23T00:46:49.679 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:46:49.680 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.680 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.680 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:46:49.680 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:27.133317+0000", 2024-02-23T00:46:49.680 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1359999999999999e-06 2024-02-23T00:46:49.680 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.680 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.680 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:46:49.680 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:27.133325+0000", 2024-02-23T00:46:49.680 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3862e-05 2024-02-23T00:46:49.680 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.680 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.681 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:46:49.681 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:27.133338+0000", 2024-02-23T00:46:49.681 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1419999999999998e-06 2024-02-23T00:46:49.681 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.681 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.681 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:46:49.681 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:27.133345+0000", 2024-02-23T00:46:49.681 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3120000000000006e-05 2024-02-23T00:46:49.681 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.681 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.681 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:46:49.681 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:27.133408+0000", 2024-02-23T00:46:49.681 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1288e-05 2024-02-23T00:46:49.681 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.682 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.682 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:46:49.682 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:27.133429+0000", 2024-02-23T00:46:49.682 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010047270000000001 2024-02-23T00:46:49.682 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.682 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.682 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:46:49.682 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:27.143476+0000", 2024-02-23T00:46:49.682 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1761000000000001e-05 2024-02-23T00:46:49.682 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.682 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.682 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:46:49.682 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:27.143498+0000", 2024-02-23T00:46:49.683 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010179812 2024-02-23T00:46:49.683 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.683 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:46:49.683 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.683 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.683 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.683 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(osd.3.0:1452 62.c e199/126 62:303239da:test-rados-api-smithi047-82465-14::foo78:4 v 199'28, mlcod=195'24)", 2024-02-23T00:46:49.683 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:44:31.163303+0000", 2024-02-23T00:46:49.683 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 138.43986383199999, 2024-02-23T00:46:49.683 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0058507990000000003, 2024-02-23T00:46:49.683 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:46:49.683 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:49.683 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:46:49.684 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.684 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:46:49.684 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.163303+0000", 2024-02-23T00:46:49.684 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:46:49.684 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.684 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.684 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:46:49.684 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.163303+0000", 2024-02-23T00:46:49.684 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999995 2024-02-23T00:46:49.684 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.684 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.684 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:46:49.684 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.163302+0000", 2024-02-23T00:46:49.684 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5060000000000001e-06 2024-02-23T00:46:49.685 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.685 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.685 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:46:49.685 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.163309+0000", 2024-02-23T00:46:49.685 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9638000000000001e-05 2024-02-23T00:46:49.685 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.685 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.685 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:46:49.685 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.163329+0000", 2024-02-23T00:46:49.685 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4399999999999999e-06 2024-02-23T00:46:49.685 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.685 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.685 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:46:49.685 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.163336+0000", 2024-02-23T00:46:49.686 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1777000000000002e-05 2024-02-23T00:46:49.686 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.686 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.686 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:46:49.686 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.163388+0000", 2024-02-23T00:46:49.686 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.3034e-05 2024-02-23T00:46:49.686 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.686 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.686 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:46:49.686 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.163411+0000", 2024-02-23T00:46:49.686 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.005707714 2024-02-23T00:46:49.686 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.686 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.687 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:46:49.687 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.169119+0000", 2024-02-23T00:46:49.687 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.5179999999999999e-05 2024-02-23T00:46:49.687 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.687 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.687 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:46:49.687 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.169154+0000", 2024-02-23T00:46:49.687 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0058507990000000003 2024-02-23T00:46:49.687 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.687 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:46:49.687 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.687 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.688 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.688 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(osd.3.0:1453 62.c e199/126 62:37762822:test-rados-api-smithi047-82465-14::foo50:4 v 199'30, mlcod=195'24)", 2024-02-23T00:46:49.688 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:44:31.163348+0000", 2024-02-23T00:46:49.688 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 138.43981857, 2024-02-23T00:46:49.688 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0060680919999999998, 2024-02-23T00:46:49.688 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:46:49.688 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:49.688 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:46:49.688 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.688 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:46:49.688 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.163348+0000", 2024-02-23T00:46:49.688 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:46:49.688 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.689 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.689 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:46:49.689 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.163348+0000", 2024-02-23T00:46:49.689 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999995 2024-02-23T00:46:49.689 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.689 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.689 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:46:49.689 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.163348+0000", 2024-02-23T00:46:49.689 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.3500000000000003e-06 2024-02-23T00:46:49.689 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.689 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.689 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:46:49.689 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.163357+0000", 2024-02-23T00:46:49.690 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6599999999999998e-06 2024-02-23T00:46:49.690 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.690 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.690 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:46:49.690 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.163364+0000", 2024-02-23T00:46:49.690 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.2200000000000003e-06 2024-02-23T00:46:49.690 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.690 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.690 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:46:49.690 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.163368+0000", 2024-02-23T00:46:49.690 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00048882099999999996 2024-02-23T00:46:49.690 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.690 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.691 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:46:49.691 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.163857+0000", 2024-02-23T00:46:49.691 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2456e-05 2024-02-23T00:46:49.691 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.691 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.691 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:46:49.691 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.163879+0000", 2024-02-23T00:46:49.691 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0055061169999999996 2024-02-23T00:46:49.691 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.691 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.692 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:46:49.692 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.169385+0000", 2024-02-23T00:46:49.692 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.0917000000000002e-05 2024-02-23T00:46:49.692 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.692 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.692 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:46:49.692 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.169416+0000", 2024-02-23T00:46:49.692 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0060680919999999998 2024-02-23T00:46:49.692 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.692 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:46:49.692 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.692 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.692 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.692 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.15270.0:1536 139.9 139:93182763:test-rados-api-smithi047-82465-18::bar:head [stat,delete] snapc 0=[] ondisk+write+known_if_redirected e214)", 2024-02-23T00:46:49.693 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:44:46.557748+0000", 2024-02-23T00:46:49.693 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 123.045419119, 2024-02-23T00:46:49.693 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010162584000000001, 2024-02-23T00:46:49.693 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:46:49.693 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:46:49.693 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:46:49.693 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.15270", 2024-02-23T00:46:49.693 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/4133111990", 2024-02-23T00:46:49.693 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1536 2024-02-23T00:46:49.693 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.693 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:46:49.693 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.693 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:46:49.694 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:46.557748+0000", 2024-02-23T00:46:49.694 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:46:49.694 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.694 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.694 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:46:49.694 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:46.557748+0000", 2024-02-23T00:46:49.694 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999976 2024-02-23T00:46:49.694 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.694 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.694 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:46:49.694 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:46.557745+0000", 2024-02-23T00:46:49.694 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2969999999999999e-05 2024-02-23T00:46:49.694 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.694 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.694 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:46:49.695 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:46.557758+0000", 2024-02-23T00:46:49.695 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7716e-05 2024-02-23T00:46:49.695 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.695 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.695 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:46:49.695 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:46.557786+0000", 2024-02-23T00:46:49.695 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.065e-05 2024-02-23T00:46:49.695 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.695 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.695 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:46:49.695 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:46.557797+0000", 2024-02-23T00:46:49.695 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8850000000000007e-05 2024-02-23T00:46:49.695 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.695 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.696 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:46:49.696 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:46.557866+0000", 2024-02-23T00:46:49.696 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6620000000000005e-06 2024-02-23T00:46:49.696 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.696 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.696 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op must wait for map", 2024-02-23T00:46:49.696 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:46.557873+0000", 2024-02-23T00:46:49.696 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0052936110000000001 2024-02-23T00:46:49.696 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.696 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.696 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:46:49.696 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:46.563167+0000", 2024-02-23T00:46:49.696 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029901100000000001 2024-02-23T00:46:49.696 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.697 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.697 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for blocked object", 2024-02-23T00:46:49.697 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:46.563466+0000", 2024-02-23T00:46:49.697 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014262630000000001 2024-02-23T00:46:49.697 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.697 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.697 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:46:49.697 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:46.564892+0000", 2024-02-23T00:46:49.697 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000108467 2024-02-23T00:46:49.697 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.697 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.697 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:46:49.697 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:46.565001+0000", 2024-02-23T00:46:49.697 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0029096719999999999 2024-02-23T00:46:49.697 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.698 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.698 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:46:49.698 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:46.567910+0000", 2024-02-23T00:46:49.698 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010162584000000001 2024-02-23T00:46:49.698 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.698 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:46:49.698 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.698 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.698 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.698 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.40808.0:1 154.6s0 154:602f83fe:::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e236)", 2024-02-23T00:46:49.698 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:45:08.147420+0000", 2024-02-23T00:46:49.698 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 101.455746745, 2024-02-23T00:46:49.698 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0216719649999999, 2024-02-23T00:46:49.698 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:46:49.698 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:49.699 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:46:49.699 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.40808", 2024-02-23T00:46:49.699 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/2440014932", 2024-02-23T00:46:49.699 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1 2024-02-23T00:46:49.699 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.699 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:46:49.699 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.699 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:46:49.699 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:08.147420+0000", 2024-02-23T00:46:49.699 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:46:49.699 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.699 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.699 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:46:49.699 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:08.147420+0000", 2024-02-23T00:46:49.700 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999976 2024-02-23T00:46:49.700 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.700 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.700 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:46:49.700 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:08.147418+0000", 2024-02-23T00:46:49.700 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0395e-05 2024-02-23T00:46:49.700 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.700 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.700 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:46:49.700 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:08.147428+0000", 2024-02-23T00:46:49.700 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6918e-05 2024-02-23T00:46:49.700 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.700 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.701 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:46:49.701 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:08.147445+0000", 2024-02-23T00:46:49.701 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0013999999999998e-05 2024-02-23T00:46:49.701 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.701 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.701 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:46:49.701 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:08.147465+0000", 2024-02-23T00:46:49.701 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4432000000000003e-05 2024-02-23T00:46:49.701 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.701 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.701 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:46:49.701 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:08.147520+0000", 2024-02-23T00:46:49.701 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.473e-06 2024-02-23T00:46:49.701 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.701 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.702 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:46:49.702 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:08.147523+0000", 2024-02-23T00:46:49.702 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0146129070000001 2024-02-23T00:46:49.702 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.702 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.702 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:46:49.702 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.162136+0000", 2024-02-23T00:46:49.702 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000107526 2024-02-23T00:46:49.702 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.702 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.702 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:46:49.702 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.162243+0000", 2024-02-23T00:46:49.702 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00048819800000000002 2024-02-23T00:46:49.702 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.702 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.703 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:46:49.703 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.162732+0000", 2024-02-23T00:46:49.703 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00091639300000000005 2024-02-23T00:46:49.703 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.703 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.703 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:46:49.703 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.163648+0000", 2024-02-23T00:46:49.703 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0053861350000000002 2024-02-23T00:46:49.703 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.703 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.703 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:46:49.703 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.169034+0000", 2024-02-23T00:46:49.703 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.7728e-05 2024-02-23T00:46:49.703 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.704 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.704 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:46:49.704 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.169092+0000", 2024-02-23T00:46:49.704 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0216719649999999 2024-02-23T00:46:49.704 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.704 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:46:49.704 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.704 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.704 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.704 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.30483.0:2 155.7 155:e19d0d35:::oid_RmXattrPP_:head [append 0~128] snapc 0=[] ondisk+write+known_if_redirected e237)", 2024-02-23T00:46:49.704 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:45:09.145192+0000", 2024-02-23T00:46:49.704 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 100.457974604, 2024-02-23T00:46:49.704 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.023838841, 2024-02-23T00:46:49.704 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:46:49.705 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:49.705 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:46:49.705 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.30483", 2024-02-23T00:46:49.705 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/3044143263", 2024-02-23T00:46:49.705 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 2 2024-02-23T00:46:49.705 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.705 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:46:49.705 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.705 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:46:49.705 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.145192+0000", 2024-02-23T00:46:49.705 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:46:49.705 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.705 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.705 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:46:49.705 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.145192+0000", 2024-02-23T00:46:49.706 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999976 2024-02-23T00:46:49.706 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.706 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.706 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:46:49.706 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.145190+0000", 2024-02-23T00:46:49.706 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1411e-05 2024-02-23T00:46:49.706 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.706 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.706 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:46:49.706 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.145201+0000", 2024-02-23T00:46:49.706 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.2265999999999999e-05 2024-02-23T00:46:49.706 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.706 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.706 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:46:49.707 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.145233+0000", 2024-02-23T00:46:49.707 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0781000000000001e-05 2024-02-23T00:46:49.707 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.707 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.707 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:46:49.707 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.145244+0000", 2024-02-23T00:46:49.707 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9168e-05 2024-02-23T00:46:49.707 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.707 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.707 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:46:49.707 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.145303+0000", 2024-02-23T00:46:49.707 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5449999999999992e-06 2024-02-23T00:46:49.707 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.707 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.708 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:46:49.708 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.145313+0000", 2024-02-23T00:46:49.708 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.016702787 2024-02-23T00:46:49.708 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.708 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.708 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:46:49.708 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.162016+0000", 2024-02-23T00:46:49.708 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013284700000000001 2024-02-23T00:46:49.708 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.708 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.708 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:46:49.708 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.162148+0000", 2024-02-23T00:46:49.708 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017232599999999999 2024-02-23T00:46:49.709 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.709 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.709 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0,2", 2024-02-23T00:46:49.709 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.162321+0000", 2024-02-23T00:46:49.709 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00092248799999999995 2024-02-23T00:46:49.709 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.709 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.709 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:46:49.709 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.163243+0000", 2024-02-23T00:46:49.709 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00090715999999999995 2024-02-23T00:46:49.709 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.709 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.709 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:46:49.710 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.164150+0000", 2024-02-23T00:46:49.710 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00481891 2024-02-23T00:46:49.710 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.710 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.710 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:46:49.710 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.168969+0000", 2024-02-23T00:46:49.710 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4189000000000001e-05 2024-02-23T00:46:49.710 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.710 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.710 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:46:49.710 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.169003+0000", 2024-02-23T00:46:49.710 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7521999999999998e-05 2024-02-23T00:46:49.710 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.710 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.710 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:46:49.711 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.169031+0000", 2024-02-23T00:46:49.711 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.023838841 2024-02-23T00:46:49.711 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.711 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:46:49.711 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.711 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.711 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.711 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.30483.0:4 155.7 155:e19d0d35:::oid_RmXattrPP_:head [rmxattr attr1 in=5b] snapc 0=[] ondisk+write+known_if_redirected e237)", 2024-02-23T00:46:49.711 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:45:11.156695+0000", 2024-02-23T00:46:49.711 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 98.446471704999993, 2024-02-23T00:46:49.711 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010290539, 2024-02-23T00:46:49.711 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:46:49.712 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:49.712 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:46:49.712 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.30483", 2024-02-23T00:46:49.712 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/3044143263", 2024-02-23T00:46:49.712 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 4 2024-02-23T00:46:49.712 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.712 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:46:49.712 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.712 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:46:49.712 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:11.156695+0000", 2024-02-23T00:46:49.712 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:46:49.712 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.712 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.713 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:46:49.713 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:11.156695+0000", 2024-02-23T00:46:49.713 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:46:49.713 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.713 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.713 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:46:49.713 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:11.156694+0000", 2024-02-23T00:46:49.713 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.9450000000000004e-06 2024-02-23T00:46:49.713 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.713 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.713 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:46:49.713 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:11.156701+0000", 2024-02-23T00:46:49.713 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5896000000000002e-05 2024-02-23T00:46:49.713 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.713 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.714 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:46:49.714 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:11.156717+0000", 2024-02-23T00:46:49.714 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1049999999999995e-06 2024-02-23T00:46:49.714 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.714 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.714 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:46:49.714 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:11.156725+0000", 2024-02-23T00:46:49.714 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8067999999999999e-05 2024-02-23T00:46:49.714 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.714 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.714 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:46:49.714 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:11.156793+0000", 2024-02-23T00:46:49.714 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3289999999999997e-05 2024-02-23T00:46:49.715 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.715 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.715 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:46:49.715 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:11.156876+0000", 2024-02-23T00:46:49.715 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000113872 2024-02-23T00:46:49.715 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.715 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.715 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0,2", 2024-02-23T00:46:49.715 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:11.156990+0000", 2024-02-23T00:46:49.715 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0073364820000000001 2024-02-23T00:46:49.715 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.715 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.715 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:46:49.715 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:11.164326+0000", 2024-02-23T00:46:49.715 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.0790000000000002e-05 2024-02-23T00:46:49.715 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.716 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.716 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:46:49.716 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:11.164357+0000", 2024-02-23T00:46:49.716 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0025387470000000001 2024-02-23T00:46:49.716 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.716 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.716 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:46:49.716 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:11.166896+0000", 2024-02-23T00:46:49.716 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9968e-05 2024-02-23T00:46:49.716 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.716 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.716 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:46:49.716 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:11.166946+0000", 2024-02-23T00:46:49.716 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9716999999999999e-05 2024-02-23T00:46:49.717 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.717 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.717 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:46:49.717 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:11.166986+0000", 2024-02-23T00:46:49.717 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010290539 2024-02-23T00:46:49.717 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.717 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:46:49.717 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.717 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.717 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.717 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.40928.0:1 170.7s0 170:fd8eaea7:nspace::oid_SimpleWritePP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e266)", 2024-02-23T00:46:49.717 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:45:38.149323+0000", 2024-02-23T00:46:49.717 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 71.453843907999996, 2024-02-23T00:46:49.718 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0017361360000001, 2024-02-23T00:46:49.718 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:46:49.718 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:49.718 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:46:49.718 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.40928", 2024-02-23T00:46:49.718 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/3947607643", 2024-02-23T00:46:49.718 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1 2024-02-23T00:46:49.718 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.718 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:46:49.718 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.718 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:46:49.718 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:38.149323+0000", 2024-02-23T00:46:49.718 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:46:49.718 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.718 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.718 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:46:49.719 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:38.149323+0000", 2024-02-23T00:46:49.719 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999971 2024-02-23T00:46:49.719 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.719 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.719 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:46:49.719 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:38.149320+0000", 2024-02-23T00:46:49.719 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0796e-05 2024-02-23T00:46:49.719 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.719 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.719 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:46:49.719 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:38.149331+0000", 2024-02-23T00:46:49.719 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6829000000000001e-05 2024-02-23T00:46:49.719 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.719 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.720 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:46:49.720 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:38.149348+0000", 2024-02-23T00:46:49.720 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0499000000000001e-05 2024-02-23T00:46:49.720 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.720 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.720 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:46:49.720 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:38.149358+0000", 2024-02-23T00:46:49.720 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8974000000000002e-05 2024-02-23T00:46:49.720 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.720 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.720 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:46:49.720 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:38.149407+0000", 2024-02-23T00:46:49.720 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3039999999999998e-06 2024-02-23T00:46:49.720 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.721 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.721 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:46:49.721 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:38.149412+0000", 2024-02-23T00:46:49.721 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.99911529799999999 2024-02-23T00:46:49.721 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.721 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.721 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:46:49.721 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:39.148527+0000", 2024-02-23T00:46:49.721 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6186000000000004e-05 2024-02-23T00:46:49.721 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.721 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.721 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:46:49.721 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:39.148623+0000", 2024-02-23T00:46:49.722 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040401199999999998 2024-02-23T00:46:49.722 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.722 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.722 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:46:49.722 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:39.149027+0000", 2024-02-23T00:46:49.722 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00044522000000000002 2024-02-23T00:46:49.722 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.722 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.722 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:46:49.722 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:39.149472+0000", 2024-02-23T00:46:49.722 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001539931 2024-02-23T00:46:49.722 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.722 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.722 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:46:49.722 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:39.151012+0000", 2024-02-23T00:46:49.722 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6737000000000003e-05 2024-02-23T00:46:49.723 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.723 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.723 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:46:49.723 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:39.151059+0000", 2024-02-23T00:46:49.723 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0017361360000001 2024-02-23T00:46:49.723 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.723 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:46:49.723 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.723 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.723 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.723 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(osd.5.0:371 175.b e288/279 175:d3d11105:test-rados-api-smithi047-82465-34::foo:14 v 288'11, mlcod=285'8)", 2024-02-23T00:46:49.723 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:46:00.171325+0000", 2024-02-23T00:46:49.723 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 49.431841886999997, 2024-02-23T00:46:49.723 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0065955129999999999, 2024-02-23T00:46:49.723 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:46:49.724 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:46:49.724 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:46:49.724 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.724 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:46:49.724 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:00.171325+0000", 2024-02-23T00:46:49.724 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:46:49.724 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.724 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.724 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:46:49.724 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:00.171325+0000", 2024-02-23T00:46:49.724 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999995 2024-02-23T00:46:49.724 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.724 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.725 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:46:49.725 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:00.171324+0000", 2024-02-23T00:46:49.725 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.8379999999999998e-06 2024-02-23T00:46:49.725 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.725 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.725 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:46:49.725 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:00.171330+0000", 2024-02-23T00:46:49.725 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0871e-05 2024-02-23T00:46:49.725 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.725 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.725 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:46:49.725 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:00.171341+0000", 2024-02-23T00:46:49.725 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4309999999999997e-06 2024-02-23T00:46:49.725 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.725 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.726 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:46:49.726 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:00.171349+0000", 2024-02-23T00:46:49.726 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012786699999999999 2024-02-23T00:46:49.726 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.726 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.726 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:46:49.726 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:00.171476+0000", 2024-02-23T00:46:49.726 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4640000000000001e-05 2024-02-23T00:46:49.726 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.726 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.726 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:46:49.726 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:00.171501+0000", 2024-02-23T00:46:49.726 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0063964579999999998 2024-02-23T00:46:49.726 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.726 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.726 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:46:49.727 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:00.177897+0000", 2024-02-23T00:46:49.727 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2878e-05 2024-02-23T00:46:49.727 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:46:49.727 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:46:49.727 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:46:49.727 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:00.177920+0000", 2024-02-23T00:46:49.727 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0065955129999999999 2024-02-23T00:46:49.727 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.727 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:46:49.727 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.727 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:46:49.727 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:46:49.727 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:46:49.729 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":478,"stamp":"2024-02-23T00:46:47.262858+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4009154,"num_objects":366,"num_object_clones":2,"num_object_copies":1098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":365,"num_whiteouts":0,"num_read":1650,"num_read_kb":1276,"num_write":1312,"num_write_kb":4576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":2249,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":84,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2561,"ondisk_log_size":2561,"up":705,"acting":705,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":725,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2520072,"kb_used_data":56220,"kb_used_omap":0,"kb_used_meta":2463744,"kb_avail":747441144,"statfs":{"total":767960285184,"available":765379731456,"internally_reserved":0,"allocated":57569280,"data_stored":39538095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2522873856},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":85,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":84,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000973"},"pg_stats":[{"pgid":"191.1f","version":"0'0","reported_seq":17,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.155362+0000","last_change":"2024-02-23T00:46:38.180001+0000","last_active":"2024-02-23T00:46:45.155362+0000","last_peered":"2024-02-23T00:46:45.155362+0000","last_clean":"2024-02-23T00:46:45.155362+0000","last_became_active":"2024-02-23T00:46:38.179704+0000","last_became_peered":"2024-02-23T00:46:38.179704+0000","last_unstale":"2024-02-23T00:46:45.155362+0000","last_undegraded":"2024-02-23T00:46:45.155362+0000","last_fullsized":"2024-02-23T00:46:45.155362+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"191.1e","version":"0'0","reported_seq":17,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.162665+0000","last_change":"2024-02-23T00:46:38.192782+0000","last_active":"2024-02-23T00:46:45.162665+0000","last_peered":"2024-02-23T00:46:45.162665+0000","last_clean":"2024-02-23T00:46:45.162665+0000","last_became_active":"2024-02-23T00:46:38.192456+0000","last_became_peered":"2024-02-23T00:46:38.192456+0000","last_unstale":"2024-02-23T00:46:45.162665+0000","last_undegraded":"2024-02-23T00:46:45.162665+0000","last_fullsized":"2024-02-23T00:46:45.162665+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,3],"acting":[1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"191.1d","version":"0'0","reported_seq":18,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.313363+0000","last_change":"2024-02-23T00:46:38.183407+0000","last_active":"2024-02-23T00:46:46.313363+0000","last_peered":"2024-02-23T00:46:46.313363+0000","last_clean":"2024-02-23T00:46:46.313363+0000","last_became_active":"2024-02-23T00:46:38.183094+0000","last_became_peered":"2024-02-23T00:46:38.183094+0000","last_unstale":"2024-02-23T00:46:46.313363+0000","last_undegraded":"2024-02-23T00:46:46.313363+0000","last_fullsized":"2024-02-23T00:46:46.313363+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"191.1c","version":"0'0","reported_seq":18,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.166999+0000","last_change":"2024-02-23T00:46:38.184229+0000","last_active":"2024-02-23T00:46:46.166999+0000","last_peered":"2024-02-23T00:46:46.166999+0000","last_clean":"2024-02-23T00:46:46.166999+0000","last_became_active":"2024-02-23T00:46:38.183918+0000","last_became_peered":"2024-02-23T00:46:38.183918+0000","last_unstale":"2024-02-23T00:46:46.166999+0000","last_undegraded":"2024-02-23T00:46:46.166999+0000","last_fullsized":"2024-02-23T00:46:46.166999+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7],"acting":[6,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"191.1b","version":"0'0","reported_seq":17,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.155399+0000","last_change":"2024-02-23T00:46:38.188172+0000","last_active":"2024-02-23T00:46:45.155399+0000","last_peered":"2024-02-23T00:46:45.155399+0000","last_clean":"2024-02-23T00:46:45.155399+0000","last_became_active":"2024-02-23T00:46:38.187907+0000","last_became_peered":"2024-02-23T00:46:38.187907+0000","last_unstale":"2024-02-23T00:46:45.155399+0000","last_undegraded":"2024-02-23T00:46:45.155399+0000","last_fullsized":"2024-02-23T00:46:45.155399+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"191.1a","version":"0'0","reported_seq":17,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.202136+0000","last_change":"2024-02-23T00:46:38.192688+0000","last_active":"2024-02-23T00:46:46.202136+0000","last_peered":"2024-02-23T00:46:46.202136+0000","last_clean":"2024-02-23T00:46:46.202136+0000","last_became_active":"2024-02-23T00:46:38.192348+0000","last_became_peered":"2024-02-23T00:46:38.192348+0000","last_unstale":"2024-02-23T00:46:46.202136+0000","last_undegraded":"2024-02-23T00:46:46.202136+0000","last_fullsized":"2024-02-23T00:46:46.202136+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"191.19","version":"0'0","reported_seq":18,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.166763+0000","last_change":"2024-02-23T00:46:38.187508+0000","last_active":"2024-02-23T00:46:46.166763+0000","last_peered":"2024-02-23T00:46:46.166763+0000","last_clean":"2024-02-23T00:46:46.166763+0000","last_became_active":"2024-02-23T00:46:38.187152+0000","last_became_peered":"2024-02-23T00:46:38.187152+0000","last_unstale":"2024-02-23T00:46:46.166763+0000","last_undegraded":"2024-02-23T00:46:46.166763+0000","last_fullsized":"2024-02-23T00:46:46.166763+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"191.18","version":"0'0","reported_seq":17,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.154145+0000","last_change":"2024-02-23T00:46:38.185432+0000","last_active":"2024-02-23T00:46:45.154145+0000","last_peered":"2024-02-23T00:46:45.154145+0000","last_clean":"2024-02-23T00:46:45.154145+0000","last_became_active":"2024-02-23T00:46:38.185150+0000","last_became_peered":"2024-02-23T00:46:38.185150+0000","last_unstale":"2024-02-23T00:46:45.154145+0000","last_undegraded":"2024-02-23T00:46:45.154145+0000","last_fullsized":"2024-02-23T00:46:45.154145+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"191.17","version":"0'0","reported_seq":17,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.162750+0000","last_change":"2024-02-23T00:46:38.194944+0000","last_active":"2024-02-23T00:46:45.162750+0000","last_peered":"2024-02-23T00:46:45.162750+0000","last_clean":"2024-02-23T00:46:45.162750+0000","last_became_active":"2024-02-23T00:46:38.194656+0000","last_became_peered":"2024-02-23T00:46:38.194656+0000","last_unstale":"2024-02-23T00:46:45.162750+0000","last_undegraded":"2024-02-23T00:46:45.162750+0000","last_fullsized":"2024-02-23T00:46:45.162750+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"191.16","version":"0'0","reported_seq":17,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.162765+0000","last_change":"2024-02-23T00:46:38.190201+0000","last_active":"2024-02-23T00:46:45.162765+0000","last_peered":"2024-02-23T00:46:45.162765+0000","last_clean":"2024-02-23T00:46:45.162765+0000","last_became_active":"2024-02-23T00:46:38.189903+0000","last_became_peered":"2024-02-23T00:46:38.189903+0000","last_unstale":"2024-02-23T00:46:45.162765+0000","last_undegraded":"2024-02-23T00:46:45.162765+0000","last_fullsized":"2024-02-23T00:46:45.162765+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"191.15","version":"0'0","reported_seq":18,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.168882+0000","last_change":"2024-02-23T00:46:38.180504+0000","last_active":"2024-02-23T00:46:46.168882+0000","last_peered":"2024-02-23T00:46:46.168882+0000","last_clean":"2024-02-23T00:46:46.168882+0000","last_became_active":"2024-02-23T00:46:38.180175+0000","last_became_peered":"2024-02-23T00:46:38.180175+0000","last_unstale":"2024-02-23T00:46:46.168882+0000","last_undegraded":"2024-02-23T00:46:46.168882+0000","last_fullsized":"2024-02-23T00:46:46.168882+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2],"acting":[6,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"191.14","version":"0'0","reported_seq":17,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.201754+0000","last_change":"2024-02-23T00:46:38.195183+0000","last_active":"2024-02-23T00:46:46.201754+0000","last_peered":"2024-02-23T00:46:46.201754+0000","last_clean":"2024-02-23T00:46:46.201754+0000","last_became_active":"2024-02-23T00:46:38.194831+0000","last_became_peered":"2024-02-23T00:46:38.194831+0000","last_unstale":"2024-02-23T00:46:46.201754+0000","last_undegraded":"2024-02-23T00:46:46.201754+0000","last_fullsized":"2024-02-23T00:46:46.201754+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"191.13","version":"0'0","reported_seq":17,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.155582+0000","last_change":"2024-02-23T00:46:38.187412+0000","last_active":"2024-02-23T00:46:45.155582+0000","last_peered":"2024-02-23T00:46:45.155582+0000","last_clean":"2024-02-23T00:46:45.155582+0000","last_became_active":"2024-02-23T00:46:38.187106+0000","last_became_peered":"2024-02-23T00:46:38.187106+0000","last_unstale":"2024-02-23T00:46:45.155582+0000","last_undegraded":"2024-02-23T00:46:45.155582+0000","last_fullsized":"2024-02-23T00:46:45.155582+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"191.12","version":"0'0","reported_seq":17,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.200869+0000","last_change":"2024-02-23T00:46:38.195016+0000","last_active":"2024-02-23T00:46:46.200869+0000","last_peered":"2024-02-23T00:46:46.200869+0000","last_clean":"2024-02-23T00:46:46.200869+0000","last_became_active":"2024-02-23T00:46:38.194770+0000","last_became_peered":"2024-02-23T00:46:38.194770+0000","last_unstale":"2024-02-23T00:46:46.200869+0000","last_undegraded":"2024-02-23T00:46:46.200869+0000","last_fullsized":"2024-02-23T00:46:46.200869+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"191.11","version":"0'0","reported_seq":18,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.312932+0000","last_change":"2024-02-23T00:46:38.185674+0000","last_active":"2024-02-23T00:46:46.312932+0000","last_peered":"2024-02-23T00:46:46.312932+0000","last_clean":"2024-02-23T00:46:46.312932+0000","last_became_active":"2024-02-23T00:46:38.185321+0000","last_became_peered":"2024-02-23T00:46:38.185321+0000","last_unstale":"2024-02-23T00:46:46.312932+0000","last_undegraded":"2024-02-23T00:46:46.312932+0000","last_fullsized":"2024-02-23T00:46:46.312932+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"191.10","version":"0'0","reported_seq":18,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.313197+0000","last_change":"2024-02-23T00:46:38.187413+0000","last_active":"2024-02-23T00:46:46.313197+0000","last_peered":"2024-02-23T00:46:46.313197+0000","last_clean":"2024-02-23T00:46:46.313197+0000","last_became_active":"2024-02-23T00:46:38.187105+0000","last_became_peered":"2024-02-23T00:46:38.187105+0000","last_unstale":"2024-02-23T00:46:46.313197+0000","last_undegraded":"2024-02-23T00:46:46.313197+0000","last_fullsized":"2024-02-23T00:46:46.313197+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"191.f","version":"0'0","reported_seq":18,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.168328+0000","last_change":"2024-02-23T00:46:38.186109+0000","last_active":"2024-02-23T00:46:46.168328+0000","last_peered":"2024-02-23T00:46:46.168328+0000","last_clean":"2024-02-23T00:46:46.168328+0000","last_became_active":"2024-02-23T00:46:38.185810+0000","last_became_peered":"2024-02-23T00:46:38.185810+0000","last_unstale":"2024-02-23T00:46:46.168328+0000","last_undegraded":"2024-02-23T00:46:46.168328+0000","last_fullsized":"2024-02-23T00:46:46.168328+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,3],"acting":[6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"191.e","version":"0'0","reported_seq":17,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.155298+0000","last_change":"2024-02-23T00:46:38.175042+0000","last_active":"2024-02-23T00:46:45.155298+0000","last_peered":"2024-02-23T00:46:45.155298+0000","last_clean":"2024-02-23T00:46:45.155298+0000","last_became_active":"2024-02-23T00:46:38.174723+0000","last_became_peered":"2024-02-23T00:46:38.174723+0000","last_unstale":"2024-02-23T00:46:45.155298+0000","last_undegraded":"2024-02-23T00:46:45.155298+0000","last_fullsized":"2024-02-23T00:46:45.155298+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"191.d","version":"0'0","reported_seq":17,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.164462+0000","last_change":"2024-02-23T00:46:38.191087+0000","last_active":"2024-02-23T00:46:45.164462+0000","last_peered":"2024-02-23T00:46:45.164462+0000","last_clean":"2024-02-23T00:46:45.164462+0000","last_became_active":"2024-02-23T00:46:38.190757+0000","last_became_peered":"2024-02-23T00:46:38.190757+0000","last_unstale":"2024-02-23T00:46:45.164462+0000","last_undegraded":"2024-02-23T00:46:45.164462+0000","last_fullsized":"2024-02-23T00:46:45.164462+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"191.c","version":"0'0","reported_seq":18,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.164546+0000","last_change":"2024-02-23T00:46:38.178506+0000","last_active":"2024-02-23T00:46:46.164546+0000","last_peered":"2024-02-23T00:46:46.164546+0000","last_clean":"2024-02-23T00:46:46.164546+0000","last_became_active":"2024-02-23T00:46:38.178186+0000","last_became_peered":"2024-02-23T00:46:38.178186+0000","last_unstale":"2024-02-23T00:46:46.164546+0000","last_undegraded":"2024-02-23T00:46:46.164546+0000","last_fullsized":"2024-02-23T00:46:46.164546+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"191.b","version":"0'0","reported_seq":17,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.201156+0000","last_change":"2024-02-23T00:46:38.192963+0000","last_active":"2024-02-23T00:46:46.201156+0000","last_peered":"2024-02-23T00:46:46.201156+0000","last_clean":"2024-02-23T00:46:46.201156+0000","last_became_active":"2024-02-23T00:46:38.192489+0000","last_became_peered":"2024-02-23T00:46:38.192489+0000","last_unstale":"2024-02-23T00:46:46.201156+0000","last_undegraded":"2024-02-23T00:46:46.201156+0000","last_fullsized":"2024-02-23T00:46:46.201156+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"191.a","version":"0'0","reported_seq":18,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.164477+0000","last_change":"2024-02-23T00:46:38.190227+0000","last_active":"2024-02-23T00:46:46.164477+0000","last_peered":"2024-02-23T00:46:46.164477+0000","last_clean":"2024-02-23T00:46:46.164477+0000","last_became_active":"2024-02-23T00:46:38.189900+0000","last_became_peered":"2024-02-23T00:46:38.189900+0000","last_unstale":"2024-02-23T00:46:46.164477+0000","last_undegraded":"2024-02-23T00:46:46.164477+0000","last_fullsized":"2024-02-23T00:46:46.164477+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"191.9","version":"0'0","reported_seq":17,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.200810+0000","last_change":"2024-02-23T00:46:38.190750+0000","last_active":"2024-02-23T00:46:46.200810+0000","last_peered":"2024-02-23T00:46:46.200810+0000","last_clean":"2024-02-23T00:46:46.200810+0000","last_became_active":"2024-02-23T00:46:38.190446+0000","last_became_peered":"2024-02-23T00:46:38.190446+0000","last_unstale":"2024-02-23T00:46:46.200810+0000","last_undegraded":"2024-02-23T00:46:46.200810+0000","last_fullsized":"2024-02-23T00:46:46.200810+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"191.8","version":"0'0","reported_seq":18,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.170749+0000","last_change":"2024-02-23T00:46:38.191150+0000","last_active":"2024-02-23T00:46:46.170749+0000","last_peered":"2024-02-23T00:46:46.170749+0000","last_clean":"2024-02-23T00:46:46.170749+0000","last_became_active":"2024-02-23T00:46:38.190558+0000","last_became_peered":"2024-02-23T00:46:38.190558+0000","last_unstale":"2024-02-23T00:46:46.170749+0000","last_undegraded":"2024-02-23T00:46:46.170749+0000","last_fullsized":"2024-02-23T00:46:46.170749+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"191.6","version":"334'5","reported_seq":23,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.635544+0000","last_change":"2024-02-23T00:46:38.178595+0000","last_active":"2024-02-23T00:46:46.635544+0000","last_peered":"2024-02-23T00:46:46.635544+0000","last_clean":"2024-02-23T00:46:46.635544+0000","last_became_active":"2024-02-23T00:46:38.178310+0000","last_became_peered":"2024-02-23T00:46:38.178310+0000","last_unstale":"2024-02-23T00:46:46.635544+0000","last_undegraded":"2024-02-23T00:46:46.635544+0000","last_fullsized":"2024-02-23T00:46:46.635544+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":85,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":84,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"191.7","version":"0'0","reported_seq":18,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.168670+0000","last_change":"2024-02-23T00:46:38.187570+0000","last_active":"2024-02-23T00:46:46.168670+0000","last_peered":"2024-02-23T00:46:46.168670+0000","last_clean":"2024-02-23T00:46:46.168670+0000","last_became_active":"2024-02-23T00:46:38.187231+0000","last_became_peered":"2024-02-23T00:46:38.187231+0000","last_unstale":"2024-02-23T00:46:46.168670+0000","last_undegraded":"2024-02-23T00:46:46.168670+0000","last_fullsized":"2024-02-23T00:46:46.168670+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"191.4","version":"0'0","reported_seq":17,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.200913+0000","last_change":"2024-02-23T00:46:38.192231+0000","last_active":"2024-02-23T00:46:46.200913+0000","last_peered":"2024-02-23T00:46:46.200913+0000","last_clean":"2024-02-23T00:46:46.200913+0000","last_became_active":"2024-02-23T00:46:38.191895+0000","last_became_peered":"2024-02-23T00:46:38.191895+0000","last_unstale":"2024-02-23T00:46:46.200913+0000","last_undegraded":"2024-02-23T00:46:46.200913+0000","last_fullsized":"2024-02-23T00:46:46.200913+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"191.5","version":"0'0","reported_seq":18,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.168926+0000","last_change":"2024-02-23T00:46:38.193664+0000","last_active":"2024-02-23T00:46:46.168926+0000","last_peered":"2024-02-23T00:46:46.168926+0000","last_clean":"2024-02-23T00:46:46.168926+0000","last_became_active":"2024-02-23T00:46:38.193138+0000","last_became_peered":"2024-02-23T00:46:38.193138+0000","last_unstale":"2024-02-23T00:46:46.168926+0000","last_undegraded":"2024-02-23T00:46:46.168926+0000","last_fullsized":"2024-02-23T00:46:46.168926+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"191.2","version":"0'0","reported_seq":17,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.154533+0000","last_change":"2024-02-23T00:46:38.183100+0000","last_active":"2024-02-23T00:46:45.154533+0000","last_peered":"2024-02-23T00:46:45.154533+0000","last_clean":"2024-02-23T00:46:45.154533+0000","last_became_active":"2024-02-23T00:46:38.182783+0000","last_became_peered":"2024-02-23T00:46:38.182783+0000","last_unstale":"2024-02-23T00:46:45.154533+0000","last_undegraded":"2024-02-23T00:46:45.154533+0000","last_fullsized":"2024-02-23T00:46:45.154533+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"191.3","version":"0'0","reported_seq":17,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.201015+0000","last_change":"2024-02-23T00:46:38.194969+0000","last_active":"2024-02-23T00:46:46.201015+0000","last_peered":"2024-02-23T00:46:46.201015+0000","last_clean":"2024-02-23T00:46:46.201015+0000","last_became_active":"2024-02-23T00:46:38.194609+0000","last_became_peered":"2024-02-23T00:46:38.194609+0000","last_unstale":"2024-02-23T00:46:46.201015+0000","last_undegraded":"2024-02-23T00:46:46.201015+0000","last_fullsized":"2024-02-23T00:46:46.201015+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"191.0","version":"0'0","reported_seq":18,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.170813+0000","last_change":"2024-02-23T00:46:38.190940+0000","last_active":"2024-02-23T00:46:46.170813+0000","last_peered":"2024-02-23T00:46:46.170813+0000","last_clean":"2024-02-23T00:46:46.170813+0000","last_became_active":"2024-02-23T00:46:38.190429+0000","last_became_peered":"2024-02-23T00:46:38.190429+0000","last_unstale":"2024-02-23T00:46:46.170813+0000","last_undegraded":"2024-02-23T00:46:46.170813+0000","last_fullsized":"2024-02-23T00:46:46.170813+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"191.1","version":"0'0","reported_seq":17,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.154209+0000","last_change":"2024-02-23T00:46:38.178413+0000","last_active":"2024-02-23T00:46:45.154209+0000","last_peered":"2024-02-23T00:46:45.154209+0000","last_clean":"2024-02-23T00:46:45.154209+0000","last_became_active":"2024-02-23T00:46:38.178068+0000","last_became_peered":"2024-02-23T00:46:38.178068+0000","last_unstale":"2024-02-23T00:46:45.154209+0000","last_undegraded":"2024-02-23T00:46:45.154209+0000","last_fullsized":"2024-02-23T00:46:45.154209+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:37.156905+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.156905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.0","version":"259'40","reported_seq":304,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.171105+0000","last_change":"2024-02-23T00:46:06.778112+0000","last_active":"2024-02-23T00:46:46.171105+0000","last_peered":"2024-02-23T00:46:46.171105+0000","last_clean":"2024-02-23T00:46:46.171105+0000","last_became_active":"2024-02-23T00:43:17.855316+0000","last_became_peered":"2024-02-23T00:43:17.855316+0000","last_unstale":"2024-02-23T00:46:46.171105+0000","last_undegraded":"2024-02-23T00:46:46.171105+0000","last_fullsized":"2024-02-23T00:46:46.171105+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'40","last_scrub_stamp":"2024-02-23T00:46:06.777995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:06.777995+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":31,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":1,"num_write":13,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.1","version":"230'87","reported_seq":337,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.160002+0000","last_change":"2024-02-23T00:46:00.167004+0000","last_active":"2024-02-23T00:46:45.160002+0000","last_peered":"2024-02-23T00:46:45.160002+0000","last_clean":"2024-02-23T00:46:45.160002+0000","last_became_active":"2024-02-23T00:43:17.851866+0000","last_became_peered":"2024-02-23T00:43:17.851866+0000","last_unstale":"2024-02-23T00:46:45.160002+0000","last_undegraded":"2024-02-23T00:46:45.160002+0000","last_fullsized":"2024-02-23T00:46:45.160002+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":87,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":63,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.2","version":"229'68","reported_seq":323,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.167109+0000","last_change":"2024-02-23T00:46:00.168915+0000","last_active":"2024-02-23T00:46:46.167109+0000","last_peered":"2024-02-23T00:46:46.167109+0000","last_clean":"2024-02-23T00:46:46.167109+0000","last_became_active":"2024-02-23T00:43:17.804792+0000","last_became_peered":"2024-02-23T00:43:17.804792+0000","last_unstale":"2024-02-23T00:46:46.167109+0000","last_undegraded":"2024-02-23T00:46:46.167109+0000","last_fullsized":"2024-02-23T00:46:46.167109+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'68","last_scrub_stamp":"2024-02-23T00:45:57.135726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:57.135726+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.3","version":"246'20","reported_seq":285,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.155550+0000","last_change":"2024-02-23T00:46:00.172490+0000","last_active":"2024-02-23T00:46:45.155550+0000","last_peered":"2024-02-23T00:46:45.155550+0000","last_clean":"2024-02-23T00:46:45.155550+0000","last_became_active":"2024-02-23T00:43:17.834686+0000","last_became_peered":"2024-02-23T00:43:17.834686+0000","last_unstale":"2024-02-23T00:46:45.155550+0000","last_undegraded":"2024-02-23T00:46:45.155550+0000","last_fullsized":"2024-02-23T00:46:45.155550+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"246'20","last_scrub_stamp":"2024-02-23T00:45:53.067737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:53.067737+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.4","version":"207'17","reported_seq":279,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.314320+0000","last_change":"2024-02-23T00:46:35.627548+0000","last_active":"2024-02-23T00:46:46.314320+0000","last_peered":"2024-02-23T00:46:46.314320+0000","last_clean":"2024-02-23T00:46:46.314320+0000","last_became_active":"2024-02-23T00:43:17.810343+0000","last_became_peered":"2024-02-23T00:43:17.810343+0000","last_unstale":"2024-02-23T00:46:46.314320+0000","last_undegraded":"2024-02-23T00:46:46.314320+0000","last_fullsized":"2024-02-23T00:46:46.314320+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"207'17","last_scrub_stamp":"2024-02-23T00:46:35.627480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:35.627480+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.5","version":"237'51","reported_seq":312,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.154930+0000","last_change":"2024-02-23T00:46:40.379258+0000","last_active":"2024-02-23T00:46:45.154930+0000","last_peered":"2024-02-23T00:46:45.154930+0000","last_clean":"2024-02-23T00:46:45.154930+0000","last_became_active":"2024-02-23T00:43:17.855339+0000","last_became_peered":"2024-02-23T00:43:17.855339+0000","last_unstale":"2024-02-23T00:46:45.154930+0000","last_undegraded":"2024-02-23T00:46:45.154930+0000","last_fullsized":"2024-02-23T00:46:45.154930+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'51","last_scrub_stamp":"2024-02-23T00:46:40.379159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:40.379159+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.6","version":"247'51","reported_seq":314,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.171841+0000","last_change":"2024-02-23T00:46:00.726103+0000","last_active":"2024-02-23T00:46:46.171841+0000","last_peered":"2024-02-23T00:46:46.171841+0000","last_clean":"2024-02-23T00:46:46.171841+0000","last_became_active":"2024-02-23T00:43:17.838838+0000","last_became_peered":"2024-02-23T00:43:17.838838+0000","last_unstale":"2024-02-23T00:46:46.171841+0000","last_undegraded":"2024-02-23T00:46:46.171841+0000","last_fullsized":"2024-02-23T00:46:46.171841+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'51","last_scrub_stamp":"2024-02-23T00:46:00.726012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:00.726012+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.7","version":"215'68","reported_seq":315,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.156295+0000","last_change":"2024-02-23T00:46:00.175978+0000","last_active":"2024-02-23T00:46:45.156295+0000","last_peered":"2024-02-23T00:46:45.156295+0000","last_clean":"2024-02-23T00:46:45.156295+0000","last_became_active":"2024-02-23T00:43:17.846548+0000","last_became_peered":"2024-02-23T00:43:17.846548+0000","last_unstale":"2024-02-23T00:46:45.156295+0000","last_undegraded":"2024-02-23T00:46:45.156295+0000","last_fullsized":"2024-02-23T00:46:45.156295+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'68","last_scrub_stamp":"2024-02-23T00:45:25.351414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.351414+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.8","version":"259'90","reported_seq":340,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.171015+0000","last_change":"2024-02-23T00:46:05.820778+0000","last_active":"2024-02-23T00:46:46.171015+0000","last_peered":"2024-02-23T00:46:46.171015+0000","last_clean":"2024-02-23T00:46:46.171015+0000","last_became_active":"2024-02-23T00:43:17.842613+0000","last_became_peered":"2024-02-23T00:43:17.842613+0000","last_unstale":"2024-02-23T00:46:46.171015+0000","last_undegraded":"2024-02-23T00:46:46.171015+0000","last_fullsized":"2024-02-23T00:46:46.171015+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'90","last_scrub_stamp":"2024-02-23T00:46:05.820651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:05.820651+0000","log_size":90,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.9","version":"259'39","reported_seq":304,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.154607+0000","last_change":"2024-02-23T00:46:10.439444+0000","last_active":"2024-02-23T00:46:45.154607+0000","last_peered":"2024-02-23T00:46:45.154607+0000","last_clean":"2024-02-23T00:46:45.154607+0000","last_became_active":"2024-02-23T00:43:17.834297+0000","last_became_peered":"2024-02-23T00:43:17.834297+0000","last_unstale":"2024-02-23T00:46:45.154607+0000","last_undegraded":"2024-02-23T00:46:45.154607+0000","last_fullsized":"2024-02-23T00:46:45.154607+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'39","last_scrub_stamp":"2024-02-23T00:46:10.439373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:10.439373+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":12,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.a","version":"253'90","reported_seq":341,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.171030+0000","last_change":"2024-02-23T00:46:00.166527+0000","last_active":"2024-02-23T00:46:46.171030+0000","last_peered":"2024-02-23T00:46:46.171030+0000","last_clean":"2024-02-23T00:46:46.171030+0000","last_became_active":"2024-02-23T00:43:17.839142+0000","last_became_peered":"2024-02-23T00:43:17.839142+0000","last_unstale":"2024-02-23T00:46:46.171030+0000","last_undegraded":"2024-02-23T00:46:46.171030+0000","last_fullsized":"2024-02-23T00:46:46.171030+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"253'90","last_scrub_stamp":"2024-02-23T00:45:59.681669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:59.681669+0000","log_size":90,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.b","version":"288'76","reported_seq":328,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.171098+0000","last_change":"2024-02-23T00:46:07.817973+0000","last_active":"2024-02-23T00:46:46.171098+0000","last_peered":"2024-02-23T00:46:46.171098+0000","last_clean":"2024-02-23T00:46:46.171098+0000","last_became_active":"2024-02-23T00:43:17.855321+0000","last_became_peered":"2024-02-23T00:43:17.855321+0000","last_unstale":"2024-02-23T00:46:46.171098+0000","last_undegraded":"2024-02-23T00:46:46.171098+0000","last_fullsized":"2024-02-23T00:46:46.171098+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"288'76","last_scrub_stamp":"2024-02-23T00:46:07.817883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:07.817883+0000","log_size":76,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":46,"num_objects":6,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":23,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.c","version":"203'34","reported_seq":295,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.164795+0000","last_change":"2024-02-23T00:46:00.163652+0000","last_active":"2024-02-23T00:46:46.164795+0000","last_peered":"2024-02-23T00:46:46.164795+0000","last_clean":"2024-02-23T00:46:46.164795+0000","last_became_active":"2024-02-23T00:43:17.829044+0000","last_became_peered":"2024-02-23T00:43:17.829044+0000","last_unstale":"2024-02-23T00:46:46.164795+0000","last_undegraded":"2024-02-23T00:46:46.164795+0000","last_fullsized":"2024-02-23T00:46:46.164795+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'34","last_scrub_stamp":"2024-02-23T00:45:57.874883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:57.874883+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.d","version":"259'56","reported_seq":309,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.154825+0000","last_change":"2024-02-23T00:46:00.159535+0000","last_active":"2024-02-23T00:46:45.154825+0000","last_peered":"2024-02-23T00:46:45.154825+0000","last_clean":"2024-02-23T00:46:45.154825+0000","last_became_active":"2024-02-23T00:43:17.841614+0000","last_became_peered":"2024-02-23T00:43:17.841614+0000","last_unstale":"2024-02-23T00:46:45.154825+0000","last_undegraded":"2024-02-23T00:46:45.154825+0000","last_fullsized":"2024-02-23T00:46:45.154825+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'56","last_scrub_stamp":"2024-02-23T00:45:51.258745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:51.258745+0000","log_size":56,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":17,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.e","version":"211'51","reported_seq":306,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.315475+0000","last_change":"2024-02-23T00:46:11.491708+0000","last_active":"2024-02-23T00:46:46.315475+0000","last_peered":"2024-02-23T00:46:46.315475+0000","last_clean":"2024-02-23T00:46:46.315475+0000","last_became_active":"2024-02-23T00:43:17.845956+0000","last_became_peered":"2024-02-23T00:43:17.845956+0000","last_unstale":"2024-02-23T00:46:46.315475+0000","last_undegraded":"2024-02-23T00:46:46.315475+0000","last_fullsized":"2024-02-23T00:46:46.315475+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'51","last_scrub_stamp":"2024-02-23T00:46:11.491637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:11.491637+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.f","version":"241'136","reported_seq":376,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.155707+0000","last_change":"2024-02-23T00:46:35.318635+0000","last_active":"2024-02-23T00:46:45.155707+0000","last_peered":"2024-02-23T00:46:45.155707+0000","last_clean":"2024-02-23T00:46:45.155707+0000","last_became_active":"2024-02-23T00:43:17.855331+0000","last_became_peered":"2024-02-23T00:43:17.855331+0000","last_unstale":"2024-02-23T00:46:45.155707+0000","last_undegraded":"2024-02-23T00:46:45.155707+0000","last_fullsized":"2024-02-23T00:46:45.155707+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"241'136","last_scrub_stamp":"2024-02-23T00:46:35.318571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:35.318571+0000","log_size":136,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":88,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":40,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.10","version":"219'68","reported_seq":318,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.158841+0000","last_change":"2024-02-23T00:46:00.167847+0000","last_active":"2024-02-23T00:46:45.158841+0000","last_peered":"2024-02-23T00:46:45.158841+0000","last_clean":"2024-02-23T00:46:45.158841+0000","last_became_active":"2024-02-23T00:43:17.819865+0000","last_became_peered":"2024-02-23T00:43:17.819865+0000","last_unstale":"2024-02-23T00:46:45.158841+0000","last_undegraded":"2024-02-23T00:46:45.158841+0000","last_fullsized":"2024-02-23T00:46:45.158841+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.11","version":"215'51","reported_seq":298,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.159909+0000","last_change":"2024-02-23T00:46:00.164366+0000","last_active":"2024-02-23T00:46:45.159909+0000","last_peered":"2024-02-23T00:46:45.159909+0000","last_clean":"2024-02-23T00:46:45.159909+0000","last_became_active":"2024-02-23T00:43:17.852697+0000","last_became_peered":"2024-02-23T00:43:17.852697+0000","last_unstale":"2024-02-23T00:46:45.159909+0000","last_undegraded":"2024-02-23T00:46:45.159909+0000","last_fullsized":"2024-02-23T00:46:45.159909+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.12","version":"240'73","reported_seq":329,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.159355+0000","last_change":"2024-02-23T00:46:00.176356+0000","last_active":"2024-02-23T00:46:45.159355+0000","last_peered":"2024-02-23T00:46:45.159355+0000","last_clean":"2024-02-23T00:46:45.159355+0000","last_became_active":"2024-02-23T00:43:17.856118+0000","last_became_peered":"2024-02-23T00:43:17.856118+0000","last_unstale":"2024-02-23T00:46:45.159355+0000","last_undegraded":"2024-02-23T00:46:45.159355+0000","last_fullsized":"2024-02-23T00:46:45.159355+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'73","last_scrub_stamp":"2024-02-23T00:45:52.066508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:52.066508+0000","log_size":73,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":49,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.13","version":"205'34","reported_seq":296,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.314058+0000","last_change":"2024-02-23T00:46:09.499938+0000","last_active":"2024-02-23T00:46:46.314058+0000","last_peered":"2024-02-23T00:46:46.314058+0000","last_clean":"2024-02-23T00:46:46.314058+0000","last_became_active":"2024-02-23T00:43:17.857823+0000","last_became_peered":"2024-02-23T00:43:17.857823+0000","last_unstale":"2024-02-23T00:46:46.314058+0000","last_undegraded":"2024-02-23T00:46:46.314058+0000","last_fullsized":"2024-02-23T00:46:46.314058+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'34","last_scrub_stamp":"2024-02-23T00:46:09.499812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:09.499812+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.14","version":"296'44","reported_seq":302,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.171644+0000","last_change":"2024-02-23T00:46:03.795687+0000","last_active":"2024-02-23T00:46:46.171644+0000","last_peered":"2024-02-23T00:46:46.171644+0000","last_clean":"2024-02-23T00:46:46.171644+0000","last_became_active":"2024-02-23T00:43:17.816138+0000","last_became_peered":"2024-02-23T00:43:17.816138+0000","last_unstale":"2024-02-23T00:46:46.171644+0000","last_undegraded":"2024-02-23T00:46:46.171644+0000","last_fullsized":"2024-02-23T00:46:46.171644+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'39","last_scrub_stamp":"2024-02-23T00:46:03.795605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:03.795605+0000","log_size":44,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":35,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":17,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.15","version":"299'18","reported_seq":283,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.164902+0000","last_change":"2024-02-23T00:46:11.830641+0000","last_active":"2024-02-23T00:46:46.164902+0000","last_peered":"2024-02-23T00:46:46.164902+0000","last_clean":"2024-02-23T00:46:46.164902+0000","last_became_active":"2024-02-23T00:43:17.843834+0000","last_became_peered":"2024-02-23T00:43:17.843834+0000","last_unstale":"2024-02-23T00:46:46.164902+0000","last_undegraded":"2024-02-23T00:46:46.164902+0000","last_fullsized":"2024-02-23T00:46:46.164902+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"299'18","last_scrub_stamp":"2024-02-23T00:46:11.830580+0000","last_deep_scrub":"299'18","last_deep_scrub_stamp":"2024-02-23T00:46:11.830580+0000","last_clean_scrub_stamp":"2024-02-23T00:46:11.830580+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.16","version":"240'23","reported_seq":287,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.155029+0000","last_change":"2024-02-23T00:46:00.160705+0000","last_active":"2024-02-23T00:46:45.155029+0000","last_peered":"2024-02-23T00:46:45.155029+0000","last_clean":"2024-02-23T00:46:45.155029+0000","last_became_active":"2024-02-23T00:43:17.834291+0000","last_became_peered":"2024-02-23T00:43:17.834291+0000","last_unstale":"2024-02-23T00:46:45.155029+0000","last_undegraded":"2024-02-23T00:46:45.155029+0000","last_fullsized":"2024-02-23T00:46:45.155029+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'23","last_scrub_stamp":"2024-02-23T00:45:56.354881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:56.354881+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":14,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.17","version":"229'57","reported_seq":321,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.168444+0000","last_change":"2024-02-23T00:46:00.166482+0000","last_active":"2024-02-23T00:46:46.168444+0000","last_peered":"2024-02-23T00:46:46.168444+0000","last_clean":"2024-02-23T00:46:46.168444+0000","last_became_active":"2024-02-23T00:43:17.843525+0000","last_became_peered":"2024-02-23T00:43:17.843525+0000","last_unstale":"2024-02-23T00:46:46.168444+0000","last_undegraded":"2024-02-23T00:46:46.168444+0000","last_fullsized":"2024-02-23T00:46:46.168444+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'57","last_scrub_stamp":"2024-02-23T00:45:58.127302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:58.127302+0000","log_size":57,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":18,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.19","version":"267'96","reported_seq":355,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.166847+0000","last_change":"2024-02-23T00:46:00.163245+0000","last_active":"2024-02-23T00:46:46.166847+0000","last_peered":"2024-02-23T00:46:46.166847+0000","last_clean":"2024-02-23T00:46:46.166847+0000","last_became_active":"2024-02-23T00:43:17.815830+0000","last_became_peered":"2024-02-23T00:43:17.815830+0000","last_unstale":"2024-02-23T00:46:46.166847+0000","last_undegraded":"2024-02-23T00:46:46.166847+0000","last_fullsized":"2024-02-23T00:46:46.166847+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'96","last_scrub_stamp":"2024-02-23T00:45:55.109551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:55.109551+0000","log_size":96,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":73,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":7,"num_read_kb":0,"num_write":30,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.18","version":"275'7","reported_seq":277,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.163660+0000","last_change":"2024-02-23T00:46:03.927901+0000","last_active":"2024-02-23T00:46:46.163660+0000","last_peered":"2024-02-23T00:46:46.163660+0000","last_clean":"2024-02-23T00:46:46.163660+0000","last_became_active":"2024-02-23T00:43:17.840351+0000","last_became_peered":"2024-02-23T00:43:17.840351+0000","last_unstale":"2024-02-23T00:46:46.163660+0000","last_undegraded":"2024-02-23T00:46:46.163660+0000","last_fullsized":"2024-02-23T00:46:46.163660+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"275'7","last_scrub_stamp":"2024-02-23T00:46:03.927783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:03.927783+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1b","version":"304'69","reported_seq":323,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.155553+0000","last_change":"2024-02-23T00:46:37.397546+0000","last_active":"2024-02-23T00:46:45.155553+0000","last_peered":"2024-02-23T00:46:45.155553+0000","last_clean":"2024-02-23T00:46:45.155553+0000","last_became_active":"2024-02-23T00:43:17.803043+0000","last_became_peered":"2024-02-23T00:43:17.803043+0000","last_unstale":"2024-02-23T00:46:45.155553+0000","last_undegraded":"2024-02-23T00:46:45.155553+0000","last_fullsized":"2024-02-23T00:46:45.155553+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"304'69","last_scrub_stamp":"2024-02-23T00:46:37.397467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.397467+0000","log_size":69,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4000052,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":21,"num_write_kb":3927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.1a","version":"225'34","reported_seq":323,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.166839+0000","last_change":"2024-02-23T00:46:02.071916+0000","last_active":"2024-02-23T00:46:46.166839+0000","last_peered":"2024-02-23T00:46:46.166839+0000","last_clean":"2024-02-23T00:46:46.166839+0000","last_became_active":"2024-02-23T00:45:44.178195+0000","last_became_peered":"2024-02-23T00:45:44.178195+0000","last_unstale":"2024-02-23T00:46:46.166839+0000","last_undegraded":"2024-02-23T00:46:46.166839+0000","last_fullsized":"2024-02-23T00:46:46.166839+0000","mapping_epoch":271,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":272,"parent":"0.0","parent_split_bits":0,"last_scrub":"225'34","last_scrub_stamp":"2024-02-23T00:46:02.071770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:02.071770+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1d","version":"215'104","reported_seq":356,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.202875+0000","last_change":"2024-02-23T00:46:05.000709+0000","last_active":"2024-02-23T00:46:46.202875+0000","last_peered":"2024-02-23T00:46:46.202875+0000","last_clean":"2024-02-23T00:46:46.202875+0000","last_became_active":"2024-02-23T00:43:17.828194+0000","last_became_peered":"2024-02-23T00:43:17.828194+0000","last_unstale":"2024-02-23T00:46:46.202875+0000","last_undegraded":"2024-02-23T00:46:46.202875+0000","last_fullsized":"2024-02-23T00:46:46.202875+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'104","last_scrub_stamp":"2024-02-23T00:46:05.000615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:05.000615+0000","log_size":104,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":74,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":32,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"62.1c","version":"223'51","reported_seq":306,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.167983+0000","last_change":"2024-02-23T00:46:08.027717+0000","last_active":"2024-02-23T00:46:46.167983+0000","last_peered":"2024-02-23T00:46:46.167983+0000","last_clean":"2024-02-23T00:46:46.167983+0000","last_became_active":"2024-02-23T00:43:17.804718+0000","last_became_peered":"2024-02-23T00:43:17.804718+0000","last_unstale":"2024-02-23T00:46:46.167983+0000","last_undegraded":"2024-02-23T00:46:46.167983+0000","last_fullsized":"2024-02-23T00:46:46.167983+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'51","last_scrub_stamp":"2024-02-23T00:46:08.027648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:08.027648+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1f","version":"240'40","reported_seq":302,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.156205+0000","last_change":"2024-02-23T00:46:00.174886+0000","last_active":"2024-02-23T00:46:45.156205+0000","last_peered":"2024-02-23T00:46:45.156205+0000","last_clean":"2024-02-23T00:46:45.156205+0000","last_became_active":"2024-02-23T00:43:17.839914+0000","last_became_peered":"2024-02-23T00:43:17.839914+0000","last_unstale":"2024-02-23T00:46:45.156205+0000","last_undegraded":"2024-02-23T00:46:45.156205+0000","last_fullsized":"2024-02-23T00:46:45.156205+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'40","last_scrub_stamp":"2024-02-23T00:45:58.074860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:58.074860+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":13,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.1e","version":"245'51","reported_seq":307,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.171749+0000","last_change":"2024-02-23T00:46:00.172469+0000","last_active":"2024-02-23T00:46:46.171749+0000","last_peered":"2024-02-23T00:46:46.171749+0000","last_clean":"2024-02-23T00:46:46.171749+0000","last_became_active":"2024-02-23T00:43:17.838682+0000","last_became_peered":"2024-02-23T00:43:17.838682+0000","last_unstale":"2024-02-23T00:46:46.171749+0000","last_undegraded":"2024-02-23T00:46:46.171749+0000","last_fullsized":"2024-02-23T00:46:46.171749+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"245'51","last_scrub_stamp":"2024-02-23T00:45:58.723990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:58.723990+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"67'8","reported_seq":393,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.171274+0000","last_change":"2024-02-23T00:45:55.708656+0000","last_active":"2024-02-23T00:46:46.171274+0000","last_peered":"2024-02-23T00:46:46.171274+0000","last_clean":"2024-02-23T00:46:46.171274+0000","last_became_active":"2024-02-23T00:39:18.793414+0000","last_became_peered":"2024-02-23T00:39:18.793414+0000","last_unstale":"2024-02-23T00:46:46.171274+0000","last_undegraded":"2024-02-23T00:46:46.171274+0000","last_fullsized":"2024-02-23T00:46:46.171274+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:45:55.708576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:55.708576+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"68'27","reported_seq":351,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.201417+0000","last_change":"2024-02-23T00:45:04.138883+0000","last_active":"2024-02-23T00:46:46.201417+0000","last_peered":"2024-02-23T00:46:46.201417+0000","last_clean":"2024-02-23T00:46:46.201417+0000","last_became_active":"2024-02-23T00:39:16.859401+0000","last_became_peered":"2024-02-23T00:39:16.859401+0000","last_unstale":"2024-02-23T00:46:46.201417+0000","last_undegraded":"2024-02-23T00:46:46.201417+0000","last_fullsized":"2024-02-23T00:46:46.201417+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'27","last_scrub_stamp":"2024-02-23T00:45:04.138751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.138751+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":301,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.156693+0000","last_change":"2024-02-23T00:46:04.942156+0000","last_active":"2024-02-23T00:46:45.156693+0000","last_peered":"2024-02-23T00:46:45.156693+0000","last_clean":"2024-02-23T00:46:45.156693+0000","last_became_active":"2024-02-23T00:39:14.784436+0000","last_became_peered":"2024-02-23T00:39:14.784436+0000","last_unstale":"2024-02-23T00:46:45.156693+0000","last_undegraded":"2024-02-23T00:46:45.156693+0000","last_fullsized":"2024-02-23T00:46:45.156693+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.942091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:04.330426+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.942091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":310,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.202029+0000","last_change":"2024-02-23T00:44:21.972446+0000","last_active":"2024-02-23T00:46:46.202029+0000","last_peered":"2024-02-23T00:46:46.202029+0000","last_clean":"2024-02-23T00:46:46.202029+0000","last_became_active":"2024-02-23T00:43:32.282545+0000","last_became_peered":"2024-02-23T00:43:32.282545+0000","last_unstale":"2024-02-23T00:46:46.202029+0000","last_undegraded":"2024-02-23T00:46:46.202029+0000","last_fullsized":"2024-02-23T00:46:46.202029+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:21.972383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:21.972383+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.3","version":"307'11","reported_seq":264,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.155653+0000","last_change":"2024-02-23T00:46:39.176892+0000","last_active":"2024-02-23T00:46:45.155653+0000","last_peered":"2024-02-23T00:46:45.155653+0000","last_clean":"2024-02-23T00:46:45.155653+0000","last_became_active":"2024-02-23T00:46:39.176574+0000","last_became_peered":"2024-02-23T00:46:39.176574+0000","last_unstale":"2024-02-23T00:46:45.155653+0000","last_undegraded":"2024-02-23T00:46:45.155653+0000","last_fullsized":"2024-02-23T00:46:45.155653+0000","mapping_epoch":108,"log_start":"307'11","ondisk_log_start":"307'11","created":108,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'5","last_scrub_stamp":"2024-02-23T00:45:17.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:17.356406+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":239,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.202939+0000","last_change":"2024-02-23T00:45:17.181400+0000","last_active":"2024-02-23T00:46:46.202939+0000","last_peered":"2024-02-23T00:46:46.202939+0000","last_clean":"2024-02-23T00:46:46.202939+0000","last_became_active":"2024-02-23T00:42:58.828846+0000","last_became_peered":"2024-02-23T00:42:58.828846+0000","last_unstale":"2024-02-23T00:46:46.202939+0000","last_undegraded":"2024-02-23T00:46:46.202939+0000","last_fullsized":"2024-02-23T00:46:46.202939+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:17.181335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:17.181335+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"68'15","reported_seq":333,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.314515+0000","last_change":"2024-02-23T00:46:45.625235+0000","last_active":"2024-02-23T00:46:46.314515+0000","last_peered":"2024-02-23T00:46:46.314515+0000","last_clean":"2024-02-23T00:46:46.314515+0000","last_became_active":"2024-02-23T00:39:16.783701+0000","last_became_peered":"2024-02-23T00:39:16.783701+0000","last_unstale":"2024-02-23T00:46:46.314515+0000","last_undegraded":"2024-02-23T00:46:46.314515+0000","last_fullsized":"2024-02-23T00:46:46.314515+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:46:45.625126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:45.625126+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":297,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.164843+0000","last_change":"2024-02-23T00:46:04.476277+0000","last_active":"2024-02-23T00:46:45.164843+0000","last_peered":"2024-02-23T00:46:45.164843+0000","last_clean":"2024-02-23T00:46:45.164843+0000","last_became_active":"2024-02-23T00:39:14.772493+0000","last_became_peered":"2024-02-23T00:39:14.772493+0000","last_unstale":"2024-02-23T00:46:45.164843+0000","last_undegraded":"2024-02-23T00:46:45.164843+0000","last_fullsized":"2024-02-23T00:46:45.164843+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.476156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.476156+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":297,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.674498+0000","last_change":"2024-02-23T00:46:45.674467+0000","last_active":"2024-02-23T00:46:45.674498+0000","last_peered":"2024-02-23T00:46:45.674498+0000","last_clean":"2024-02-23T00:46:45.674498+0000","last_became_active":"2024-02-23T00:39:18.794476+0000","last_became_peered":"2024-02-23T00:39:18.794476+0000","last_unstale":"2024-02-23T00:46:45.674498+0000","last_undegraded":"2024-02-23T00:46:45.674498+0000","last_fullsized":"2024-02-23T00:46:45.674498+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:45.674391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:08.335330+0000","last_clean_scrub_stamp":"2024-02-23T00:46:45.674391+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":311,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.314964+0000","last_change":"2024-02-23T00:45:50.685890+0000","last_active":"2024-02-23T00:46:46.314964+0000","last_peered":"2024-02-23T00:46:46.314964+0000","last_clean":"2024-02-23T00:46:46.314964+0000","last_became_active":"2024-02-23T00:43:35.079004+0000","last_became_peered":"2024-02-23T00:43:35.079004+0000","last_unstale":"2024-02-23T00:46:46.314964+0000","last_undegraded":"2024-02-23T00:46:46.314964+0000","last_fullsized":"2024-02-23T00:46:46.314964+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:50.685824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:45:50.685824+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"307'7","reported_seq":245,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.155796+0000","last_change":"2024-02-23T00:45:06.483322+0000","last_active":"2024-02-23T00:46:45.155796+0000","last_peered":"2024-02-23T00:46:45.155796+0000","last_clean":"2024-02-23T00:46:45.155796+0000","last_became_active":"2024-02-23T00:42:58.741558+0000","last_became_peered":"2024-02-23T00:42:58.741558+0000","last_unstale":"2024-02-23T00:46:45.155796+0000","last_undegraded":"2024-02-23T00:46:45.155796+0000","last_fullsized":"2024-02-23T00:46:45.155796+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'5","last_scrub_stamp":"2024-02-23T00:45:06.483252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.483252+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":240,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.315592+0000","last_change":"2024-02-23T00:45:22.597544+0000","last_active":"2024-02-23T00:46:46.315592+0000","last_peered":"2024-02-23T00:46:46.315592+0000","last_clean":"2024-02-23T00:46:46.315592+0000","last_became_active":"2024-02-23T00:42:58.867305+0000","last_became_peered":"2024-02-23T00:42:58.867305+0000","last_unstale":"2024-02-23T00:46:46.315592+0000","last_undegraded":"2024-02-23T00:46:46.315592+0000","last_fullsized":"2024-02-23T00:46:46.315592+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:22.597434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:22.597434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"88'35","reported_seq":356,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.167648+0000","last_change":"2024-02-23T00:46:41.290546+0000","last_active":"2024-02-23T00:46:46.167648+0000","last_peered":"2024-02-23T00:46:46.167648+0000","last_clean":"2024-02-23T00:46:46.167648+0000","last_became_active":"2024-02-23T00:39:16.857413+0000","last_became_peered":"2024-02-23T00:39:16.857413+0000","last_unstale":"2024-02-23T00:46:46.167648+0000","last_undegraded":"2024-02-23T00:46:46.167648+0000","last_fullsized":"2024-02-23T00:46:46.167648+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'35","last_scrub_stamp":"2024-02-23T00:46:41.290478+0000","last_deep_scrub":"88'35","last_deep_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_clean_scrub_stamp":"2024-02-23T00:46:41.290478+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":301,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.157529+0000","last_change":"2024-02-23T00:46:05.910392+0000","last_active":"2024-02-23T00:46:45.157529+0000","last_peered":"2024-02-23T00:46:45.157529+0000","last_clean":"2024-02-23T00:46:45.157529+0000","last_became_active":"2024-02-23T00:39:14.776705+0000","last_became_peered":"2024-02-23T00:39:14.776705+0000","last_unstale":"2024-02-23T00:46:45.157529+0000","last_undegraded":"2024-02-23T00:46:45.157529+0000","last_fullsized":"2024-02-23T00:46:45.157529+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:05.910325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_clean_scrub_stamp":"2024-02-23T00:46:05.910325+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":297,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.155632+0000","last_change":"2024-02-23T00:46:34.726091+0000","last_active":"2024-02-23T00:46:45.155632+0000","last_peered":"2024-02-23T00:46:45.155632+0000","last_clean":"2024-02-23T00:46:45.155632+0000","last_became_active":"2024-02-23T00:39:18.792718+0000","last_became_peered":"2024-02-23T00:39:18.792718+0000","last_unstale":"2024-02-23T00:46:45.155632+0000","last_undegraded":"2024-02-23T00:46:45.155632+0000","last_fullsized":"2024-02-23T00:46:45.155632+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:34.726027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:34.726027+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":308,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.171482+0000","last_change":"2024-02-23T00:44:36.758043+0000","last_active":"2024-02-23T00:46:46.171482+0000","last_peered":"2024-02-23T00:46:46.171482+0000","last_clean":"2024-02-23T00:46:46.171482+0000","last_became_active":"2024-02-23T00:43:39.470999+0000","last_became_peered":"2024-02-23T00:43:39.470999+0000","last_unstale":"2024-02-23T00:46:46.171482+0000","last_undegraded":"2024-02-23T00:46:46.171482+0000","last_fullsized":"2024-02-23T00:46:46.171482+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:36.757960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:36.757960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"307'9","reported_seq":264,"reported_epoch":335,"state":"active+clean","last_fresh":"2024-02-23T00:46:47.174810+0000","last_change":"2024-02-23T00:46:47.174810+0000","last_active":"2024-02-23T00:46:47.174810+0000","last_peered":"2024-02-23T00:46:47.174810+0000","last_clean":"2024-02-23T00:46:47.174810+0000","last_became_active":"2024-02-23T00:46:47.174465+0000","last_became_peered":"2024-02-23T00:46:47.174465+0000","last_unstale":"2024-02-23T00:46:47.174810+0000","last_undegraded":"2024-02-23T00:46:47.174810+0000","last_fullsized":"2024-02-23T00:46:47.174810+0000","mapping_epoch":108,"log_start":"307'9","ondisk_log_start":"307'9","created":108,"last_epoch_clean":335,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'5","last_scrub_stamp":"2024-02-23T00:45:23.832291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.832291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":239,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.163342+0000","last_change":"2024-02-23T00:46:40.354058+0000","last_active":"2024-02-23T00:46:45.163342+0000","last_peered":"2024-02-23T00:46:45.163342+0000","last_clean":"2024-02-23T00:46:45.163342+0000","last_became_active":"2024-02-23T00:42:57.639013+0000","last_became_peered":"2024-02-23T00:42:57.639013+0000","last_unstale":"2024-02-23T00:46:45.163342+0000","last_undegraded":"2024-02-23T00:46:45.163342+0000","last_fullsized":"2024-02-23T00:46:45.163342+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:40.353999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:46:40.353999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"67'17","reported_seq":345,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.155481+0000","last_change":"2024-02-23T00:43:45.194207+0000","last_active":"2024-02-23T00:46:45.155481+0000","last_peered":"2024-02-23T00:46:45.155481+0000","last_clean":"2024-02-23T00:46:45.155481+0000","last_became_active":"2024-02-23T00:40:42.933806+0000","last_became_peered":"2024-02-23T00:40:42.933806+0000","last_unstale":"2024-02-23T00:46:45.155481+0000","last_undegraded":"2024-02-23T00:46:45.155481+0000","last_fullsized":"2024-02-23T00:46:45.155481+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'17","last_scrub_stamp":"2024-02-23T00:43:45.194139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.194139+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":272,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.155888+0000","last_change":"2024-02-23T00:45:56.051644+0000","last_active":"2024-02-23T00:46:45.155888+0000","last_peered":"2024-02-23T00:46:45.155888+0000","last_clean":"2024-02-23T00:46:45.155888+0000","last_became_active":"2024-02-23T00:40:42.931041+0000","last_became_peered":"2024-02-23T00:40:42.931041+0000","last_unstale":"2024-02-23T00:46:45.155888+0000","last_undegraded":"2024-02-23T00:46:45.155888+0000","last_fullsized":"2024-02-23T00:46:45.155888+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:56.051575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:56.051575+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":293,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.163867+0000","last_change":"2024-02-23T00:44:22.804352+0000","last_active":"2024-02-23T00:46:45.163867+0000","last_peered":"2024-02-23T00:46:45.163867+0000","last_clean":"2024-02-23T00:46:45.163867+0000","last_became_active":"2024-02-23T00:39:18.799875+0000","last_became_peered":"2024-02-23T00:39:18.799875+0000","last_unstale":"2024-02-23T00:46:45.163867+0000","last_undegraded":"2024-02-23T00:46:45.163867+0000","last_fullsized":"2024-02-23T00:46:45.163867+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:22.804279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:22.804279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":308,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.156761+0000","last_change":"2024-02-23T00:44:36.481327+0000","last_active":"2024-02-23T00:46:45.156761+0000","last_peered":"2024-02-23T00:46:45.156761+0000","last_clean":"2024-02-23T00:46:45.156761+0000","last_became_active":"2024-02-23T00:43:08.885257+0000","last_became_peered":"2024-02-23T00:43:08.885257+0000","last_unstale":"2024-02-23T00:46:45.156761+0000","last_undegraded":"2024-02-23T00:46:45.156761+0000","last_fullsized":"2024-02-23T00:46:45.156761+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:36.481256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:36.481256+0000","last_clean_scrub_stamp":"2024-02-23T00:44:36.481256+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.6","version":"307'11","reported_seq":269,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.315933+0000","last_change":"2024-02-23T00:46:27.159701+0000","last_active":"2024-02-23T00:46:46.315933+0000","last_peered":"2024-02-23T00:46:46.315933+0000","last_clean":"2024-02-23T00:46:46.315933+0000","last_became_active":"2024-02-23T00:46:27.159344+0000","last_became_peered":"2024-02-23T00:46:27.159344+0000","last_unstale":"2024-02-23T00:46:46.315933+0000","last_undegraded":"2024-02-23T00:46:46.315933+0000","last_fullsized":"2024-02-23T00:46:46.315933+0000","mapping_epoch":108,"log_start":"307'11","ondisk_log_start":"307'11","created":108,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'9","last_scrub_stamp":"2024-02-23T00:46:08.530434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:46:08.530434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":239,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.163907+0000","last_change":"2024-02-23T00:45:23.744990+0000","last_active":"2024-02-23T00:46:45.163907+0000","last_peered":"2024-02-23T00:46:45.163907+0000","last_clean":"2024-02-23T00:46:45.163907+0000","last_became_active":"2024-02-23T00:42:57.675306+0000","last_became_peered":"2024-02-23T00:42:57.675306+0000","last_unstale":"2024-02-23T00:46:45.163907+0000","last_undegraded":"2024-02-23T00:46:45.163907+0000","last_fullsized":"2024-02-23T00:46:45.163907+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:23.744846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.744846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"67'10","reported_seq":310,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.155099+0000","last_change":"2024-02-23T00:45:25.175258+0000","last_active":"2024-02-23T00:46:45.155099+0000","last_peered":"2024-02-23T00:46:45.155099+0000","last_clean":"2024-02-23T00:46:45.155099+0000","last_became_active":"2024-02-23T00:39:16.777530+0000","last_became_peered":"2024-02-23T00:39:16.777530+0000","last_unstale":"2024-02-23T00:46:45.155099+0000","last_undegraded":"2024-02-23T00:46:45.155099+0000","last_fullsized":"2024-02-23T00:46:45.155099+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'10","last_scrub_stamp":"2024-02-23T00:45:25.175173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.175173+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":297,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.163554+0000","last_change":"2024-02-23T00:45:10.693332+0000","last_active":"2024-02-23T00:46:45.163554+0000","last_peered":"2024-02-23T00:46:45.163554+0000","last_clean":"2024-02-23T00:46:45.163554+0000","last_became_active":"2024-02-23T00:39:14.780379+0000","last_became_peered":"2024-02-23T00:39:14.780379+0000","last_unstale":"2024-02-23T00:46:45.163554+0000","last_undegraded":"2024-02-23T00:46:45.163554+0000","last_fullsized":"2024-02-23T00:46:45.163554+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.693273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.575242+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.693273+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"67'8","reported_seq":390,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.155261+0000","last_change":"2024-02-23T00:44:25.470245+0000","last_active":"2024-02-23T00:46:45.155261+0000","last_peered":"2024-02-23T00:46:45.155261+0000","last_clean":"2024-02-23T00:46:45.155261+0000","last_became_active":"2024-02-23T00:39:18.799544+0000","last_became_peered":"2024-02-23T00:39:18.799544+0000","last_unstale":"2024-02-23T00:46:45.155261+0000","last_undegraded":"2024-02-23T00:46:45.155261+0000","last_fullsized":"2024-02-23T00:46:45.155261+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:44:25.470174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:25.470174+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":311,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.155529+0000","last_change":"2024-02-23T00:46:42.326628+0000","last_active":"2024-02-23T00:46:45.155529+0000","last_peered":"2024-02-23T00:46:45.155529+0000","last_clean":"2024-02-23T00:46:45.155529+0000","last_became_active":"2024-02-23T00:43:13.338697+0000","last_became_peered":"2024-02-23T00:43:13.338697+0000","last_unstale":"2024-02-23T00:46:45.155529+0000","last_undegraded":"2024-02-23T00:46:45.155529+0000","last_fullsized":"2024-02-23T00:46:45.155529+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:42.326566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:15.081153+0000","last_clean_scrub_stamp":"2024-02-23T00:46:42.326566+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.7","version":"307'7","reported_seq":252,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.163304+0000","last_change":"2024-02-23T00:46:23.199817+0000","last_active":"2024-02-23T00:46:45.163304+0000","last_peered":"2024-02-23T00:46:45.163304+0000","last_clean":"2024-02-23T00:46:45.163304+0000","last_became_active":"2024-02-23T00:46:23.199475+0000","last_became_peered":"2024-02-23T00:46:23.199475+0000","last_unstale":"2024-02-23T00:46:45.163304+0000","last_undegraded":"2024-02-23T00:46:45.163304+0000","last_fullsized":"2024-02-23T00:46:45.163304+0000","mapping_epoch":108,"log_start":"307'7","ondisk_log_start":"307'7","created":108,"last_epoch_clean":311,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":240,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.163943+0000","last_change":"2024-02-23T00:45:22.874761+0000","last_active":"2024-02-23T00:46:46.163943+0000","last_peered":"2024-02-23T00:46:46.163943+0000","last_clean":"2024-02-23T00:46:46.163943+0000","last_became_active":"2024-02-23T00:42:58.765106+0000","last_became_peered":"2024-02-23T00:42:58.765106+0000","last_unstale":"2024-02-23T00:46:46.163943+0000","last_undegraded":"2024-02-23T00:46:46.163943+0000","last_fullsized":"2024-02-23T00:46:46.163943+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:22.874696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:22.874696+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"67'15","reported_seq":320,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.163309+0000","last_change":"2024-02-23T00:44:19.830383+0000","last_active":"2024-02-23T00:46:45.163309+0000","last_peered":"2024-02-23T00:46:45.163309+0000","last_clean":"2024-02-23T00:46:45.163309+0000","last_became_active":"2024-02-23T00:39:16.789757+0000","last_became_peered":"2024-02-23T00:39:16.789757+0000","last_unstale":"2024-02-23T00:46:45.163309+0000","last_undegraded":"2024-02-23T00:46:45.163309+0000","last_fullsized":"2024-02-23T00:46:45.163309+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:44:19.830309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:44:19.830309+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":297,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.160406+0000","last_change":"2024-02-23T00:44:29.773000+0000","last_active":"2024-02-23T00:46:45.160406+0000","last_peered":"2024-02-23T00:46:45.160406+0000","last_clean":"2024-02-23T00:46:45.160406+0000","last_became_active":"2024-02-23T00:39:14.780526+0000","last_became_peered":"2024-02-23T00:39:14.780526+0000","last_unstale":"2024-02-23T00:46:45.160406+0000","last_undegraded":"2024-02-23T00:46:45.160406+0000","last_fullsized":"2024-02-23T00:46:45.160406+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:29.772933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:29.772933+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":391,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.155432+0000","last_change":"2024-02-23T00:44:23.109508+0000","last_active":"2024-02-23T00:46:45.155432+0000","last_peered":"2024-02-23T00:46:45.155432+0000","last_clean":"2024-02-23T00:46:45.155432+0000","last_became_active":"2024-02-23T00:39:18.800440+0000","last_became_peered":"2024-02-23T00:39:18.800440+0000","last_unstale":"2024-02-23T00:46:45.155432+0000","last_undegraded":"2024-02-23T00:46:45.155432+0000","last_fullsized":"2024-02-23T00:46:45.155432+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:44:23.109439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.109439+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":310,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.203409+0000","last_change":"2024-02-23T00:44:22.976713+0000","last_active":"2024-02-23T00:46:46.203409+0000","last_peered":"2024-02-23T00:46:46.203409+0000","last_clean":"2024-02-23T00:46:46.203409+0000","last_became_active":"2024-02-23T00:43:16.650527+0000","last_became_peered":"2024-02-23T00:43:16.650527+0000","last_unstale":"2024-02-23T00:46:46.203409+0000","last_undegraded":"2024-02-23T00:46:46.203409+0000","last_fullsized":"2024-02-23T00:46:46.203409+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:22.976611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:10.666299+0000","last_clean_scrub_stamp":"2024-02-23T00:44:22.976611+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"110'2","reported_seq":257,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.163433+0000","last_change":"2024-02-23T00:46:35.159640+0000","last_active":"2024-02-23T00:46:45.163433+0000","last_peered":"2024-02-23T00:46:45.163433+0000","last_clean":"2024-02-23T00:46:45.163433+0000","last_became_active":"2024-02-23T00:46:35.159266+0000","last_became_peered":"2024-02-23T00:46:35.159266+0000","last_unstale":"2024-02-23T00:46:45.163433+0000","last_undegraded":"2024-02-23T00:46:45.163433+0000","last_fullsized":"2024-02-23T00:46:45.163433+0000","mapping_epoch":108,"log_start":"110'2","ondisk_log_start":"110'2","created":108,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:26.719011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:26.719011+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,3],"acting":[1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":239,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.154527+0000","last_change":"2024-02-23T00:45:54.311565+0000","last_active":"2024-02-23T00:46:45.154527+0000","last_peered":"2024-02-23T00:46:45.154527+0000","last_clean":"2024-02-23T00:46:45.154527+0000","last_became_active":"2024-02-23T00:42:58.833924+0000","last_became_peered":"2024-02-23T00:42:58.833924+0000","last_unstale":"2024-02-23T00:46:45.154527+0000","last_undegraded":"2024-02-23T00:46:45.154527+0000","last_fullsized":"2024-02-23T00:46:45.154527+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:54.311442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:54.311442+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"68'17","reported_seq":347,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.155121+0000","last_change":"2024-02-23T00:43:33.151471+0000","last_active":"2024-02-23T00:46:45.155121+0000","last_peered":"2024-02-23T00:46:45.155121+0000","last_clean":"2024-02-23T00:46:45.155121+0000","last_became_active":"2024-02-23T00:40:42.933759+0000","last_became_peered":"2024-02-23T00:40:42.933759+0000","last_unstale":"2024-02-23T00:46:45.155121+0000","last_undegraded":"2024-02-23T00:46:45.155121+0000","last_fullsized":"2024-02-23T00:46:45.155121+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:43:33.151401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:33.151401+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":301,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.171560+0000","last_change":"2024-02-23T00:46:43.722544+0000","last_active":"2024-02-23T00:46:46.171560+0000","last_peered":"2024-02-23T00:46:46.171560+0000","last_clean":"2024-02-23T00:46:46.171560+0000","last_became_active":"2024-02-23T00:39:14.784121+0000","last_became_peered":"2024-02-23T00:39:14.784121+0000","last_unstale":"2024-02-23T00:46:46.171560+0000","last_undegraded":"2024-02-23T00:46:46.171560+0000","last_fullsized":"2024-02-23T00:46:46.171560+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:43.722480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:43.722480+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":294,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.203320+0000","last_change":"2024-02-23T00:45:08.118391+0000","last_active":"2024-02-23T00:46:46.203320+0000","last_peered":"2024-02-23T00:46:46.203320+0000","last_clean":"2024-02-23T00:46:46.203320+0000","last_became_active":"2024-02-23T00:39:18.800777+0000","last_became_peered":"2024-02-23T00:39:18.800777+0000","last_unstale":"2024-02-23T00:46:46.203320+0000","last_undegraded":"2024-02-23T00:46:46.203320+0000","last_fullsized":"2024-02-23T00:46:46.203320+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.118302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.118302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"67'1","reported_seq":311,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.164918+0000","last_change":"2024-02-23T00:43:56.091057+0000","last_active":"2024-02-23T00:46:46.164918+0000","last_peered":"2024-02-23T00:46:46.164918+0000","last_clean":"2024-02-23T00:46:46.164918+0000","last_became_active":"2024-02-23T00:43:22.866277+0000","last_became_peered":"2024-02-23T00:43:22.866277+0000","last_unstale":"2024-02-23T00:46:46.164918+0000","last_undegraded":"2024-02-23T00:46:46.164918+0000","last_fullsized":"2024-02-23T00:46:46.164918+0000","mapping_epoch":65,"log_start":"67'1","ondisk_log_start":"67'1","created":65,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:43:56.090982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:09.079918+0000","last_clean_scrub_stamp":"2024-02-23T00:43:56.090982+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":257,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.169154+0000","last_change":"2024-02-23T00:46:31.172051+0000","last_active":"2024-02-23T00:46:46.169154+0000","last_peered":"2024-02-23T00:46:46.169154+0000","last_clean":"2024-02-23T00:46:46.169154+0000","last_became_active":"2024-02-23T00:46:31.171712+0000","last_became_peered":"2024-02-23T00:46:31.171712+0000","last_unstale":"2024-02-23T00:46:46.169154+0000","last_undegraded":"2024-02-23T00:46:46.169154+0000","last_fullsized":"2024-02-23T00:46:46.169154+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:06.651711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.651711+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":239,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.159258+0000","last_change":"2024-02-23T00:46:35.407560+0000","last_active":"2024-02-23T00:46:45.159258+0000","last_peered":"2024-02-23T00:46:45.159258+0000","last_clean":"2024-02-23T00:46:45.159258+0000","last_became_active":"2024-02-23T00:42:57.653279+0000","last_became_peered":"2024-02-23T00:42:57.653279+0000","last_unstale":"2024-02-23T00:46:45.159258+0000","last_undegraded":"2024-02-23T00:46:45.159258+0000","last_fullsized":"2024-02-23T00:46:45.159258+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:35.407468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:46:35.407468+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":393,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.163835+0000","last_change":"2024-02-23T00:45:51.871688+0000","last_active":"2024-02-23T00:46:46.163835+0000","last_peered":"2024-02-23T00:46:46.163835+0000","last_clean":"2024-02-23T00:46:46.163835+0000","last_became_active":"2024-02-23T00:39:18.798536+0000","last_became_peered":"2024-02-23T00:39:18.798536+0000","last_unstale":"2024-02-23T00:46:46.163835+0000","last_undegraded":"2024-02-23T00:46:46.163835+0000","last_fullsized":"2024-02-23T00:46:46.163835+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:45:51.871618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:51.871618+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"68'20","reported_seq":331,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.154770+0000","last_change":"2024-02-23T00:45:05.291909+0000","last_active":"2024-02-23T00:46:45.154770+0000","last_peered":"2024-02-23T00:46:45.154770+0000","last_clean":"2024-02-23T00:46:45.154770+0000","last_became_active":"2024-02-23T00:39:16.780130+0000","last_became_peered":"2024-02-23T00:39:16.780130+0000","last_unstale":"2024-02-23T00:46:45.154770+0000","last_undegraded":"2024-02-23T00:46:45.154770+0000","last_fullsized":"2024-02-23T00:46:45.154770+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'20","last_scrub_stamp":"2024-02-23T00:45:05.291836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:05.291836+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":304,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.168599+0000","last_change":"2024-02-23T00:46:35.220317+0000","last_active":"2024-02-23T00:46:46.168599+0000","last_peered":"2024-02-23T00:46:46.168599+0000","last_clean":"2024-02-23T00:46:46.168599+0000","last_became_active":"2024-02-23T00:39:14.781999+0000","last_became_peered":"2024-02-23T00:39:14.781999+0000","last_unstale":"2024-02-23T00:46:46.168599+0000","last_undegraded":"2024-02-23T00:46:46.168599+0000","last_fullsized":"2024-02-23T00:46:46.168599+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:35.220225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:51.391939+0000","last_clean_scrub_stamp":"2024-02-23T00:46:35.220225+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":295,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.686840+0000","last_change":"2024-02-23T00:46:46.686819+0000","last_active":"2024-02-23T00:46:46.686840+0000","last_peered":"2024-02-23T00:46:46.686840+0000","last_clean":"2024-02-23T00:46:46.686840+0000","last_became_active":"2024-02-23T00:39:21.161928+0000","last_became_peered":"2024-02-23T00:39:21.161928+0000","last_unstale":"2024-02-23T00:46:46.686840+0000","last_undegraded":"2024-02-23T00:46:46.686840+0000","last_fullsized":"2024-02-23T00:46:46.686840+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:46.686764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:46.686764+0000","last_clean_scrub_stamp":"2024-02-23T00:46:46.686764+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"307'16","reported_seq":272,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.159583+0000","last_change":"2024-02-23T00:46:11.156483+0000","last_active":"2024-02-23T00:46:45.159583+0000","last_peered":"2024-02-23T00:46:45.159583+0000","last_clean":"2024-02-23T00:46:45.159583+0000","last_became_active":"2024-02-23T00:46:11.156132+0000","last_became_peered":"2024-02-23T00:46:11.156132+0000","last_unstale":"2024-02-23T00:46:45.159583+0000","last_undegraded":"2024-02-23T00:46:45.159583+0000","last_fullsized":"2024-02-23T00:46:45.159583+0000","mapping_epoch":108,"log_start":"110'10","ondisk_log_start":"110'10","created":108,"last_epoch_clean":299,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'10","last_scrub_stamp":"2024-02-23T00:45:24.377329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:24.377329+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":384,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":239,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.203390+0000","last_change":"2024-02-23T00:45:59.944702+0000","last_active":"2024-02-23T00:46:46.203390+0000","last_peered":"2024-02-23T00:46:46.203390+0000","last_clean":"2024-02-23T00:46:46.203390+0000","last_became_active":"2024-02-23T00:42:58.717371+0000","last_became_peered":"2024-02-23T00:42:58.717371+0000","last_unstale":"2024-02-23T00:46:46.203390+0000","last_undegraded":"2024-02-23T00:46:46.203390+0000","last_fullsized":"2024-02-23T00:46:46.203390+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:59.944587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:59.944587+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":293,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.154992+0000","last_change":"2024-02-23T00:44:34.059194+0000","last_active":"2024-02-23T00:46:45.154992+0000","last_peered":"2024-02-23T00:46:45.154992+0000","last_clean":"2024-02-23T00:46:45.154992+0000","last_became_active":"2024-02-23T00:39:18.798829+0000","last_became_peered":"2024-02-23T00:39:18.798829+0000","last_unstale":"2024-02-23T00:46:45.154992+0000","last_undegraded":"2024-02-23T00:46:45.154992+0000","last_fullsized":"2024-02-23T00:46:45.154992+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:34.059129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:34.059129+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"251'40","reported_seq":369,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.163797+0000","last_change":"2024-02-23T00:45:23.910609+0000","last_active":"2024-02-23T00:46:46.163797+0000","last_peered":"2024-02-23T00:46:46.163797+0000","last_clean":"2024-02-23T00:46:46.163797+0000","last_became_active":"2024-02-23T00:39:16.859002+0000","last_became_peered":"2024-02-23T00:39:16.859002+0000","last_unstale":"2024-02-23T00:46:46.163797+0000","last_undegraded":"2024-02-23T00:46:46.163797+0000","last_fullsized":"2024-02-23T00:46:46.163797+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'40","last_scrub_stamp":"2024-02-23T00:45:23.910546+0000","last_deep_scrub":"251'40","last_deep_scrub_stamp":"2024-02-23T00:45:23.910546+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.910546+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":297,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.162989+0000","last_change":"2024-02-23T00:45:16.664785+0000","last_active":"2024-02-23T00:46:45.162989+0000","last_peered":"2024-02-23T00:46:45.162989+0000","last_clean":"2024-02-23T00:46:45.162989+0000","last_became_active":"2024-02-23T00:39:14.779193+0000","last_became_peered":"2024-02-23T00:39:14.779193+0000","last_unstale":"2024-02-23T00:46:45.162989+0000","last_undegraded":"2024-02-23T00:46:45.162989+0000","last_fullsized":"2024-02-23T00:46:45.162989+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.664682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.664682+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":291,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.155386+0000","last_change":"2024-02-23T00:45:03.228087+0000","last_active":"2024-02-23T00:46:45.155386+0000","last_peered":"2024-02-23T00:46:45.155386+0000","last_clean":"2024-02-23T00:46:45.155386+0000","last_became_active":"2024-02-23T00:39:20.795271+0000","last_became_peered":"2024-02-23T00:39:20.795271+0000","last_unstale":"2024-02-23T00:46:45.155386+0000","last_undegraded":"2024-02-23T00:46:45.155386+0000","last_fullsized":"2024-02-23T00:46:45.155386+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:03.227995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:45:03.227995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"307'14","reported_seq":110,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.155902+0000","last_change":"2024-02-23T00:46:07.156105+0000","last_active":"2024-02-23T00:46:45.155902+0000","last_peered":"2024-02-23T00:46:45.155902+0000","last_clean":"2024-02-23T00:46:45.155902+0000","last_became_active":"2024-02-23T00:46:07.155723+0000","last_became_peered":"2024-02-23T00:46:07.155723+0000","last_unstale":"2024-02-23T00:46:45.155902+0000","last_undegraded":"2024-02-23T00:46:45.155902+0000","last_fullsized":"2024-02-23T00:46:45.155902+0000","mapping_epoch":272,"log_start":"110'10","ondisk_log_start":"110'10","created":108,"last_epoch_clean":295,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'2","last_scrub_stamp":"2024-02-23T00:45:10.465108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.465108+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":240,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.165857+0000","last_change":"2024-02-23T00:45:13.874437+0000","last_active":"2024-02-23T00:46:46.165857+0000","last_peered":"2024-02-23T00:46:46.165857+0000","last_clean":"2024-02-23T00:46:46.165857+0000","last_became_active":"2024-02-23T00:42:58.809283+0000","last_became_peered":"2024-02-23T00:42:58.809283+0000","last_unstale":"2024-02-23T00:46:46.165857+0000","last_undegraded":"2024-02-23T00:46:46.165857+0000","last_fullsized":"2024-02-23T00:46:46.165857+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:13.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:13.874373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":298,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.167578+0000","last_change":"2024-02-23T00:46:45.294769+0000","last_active":"2024-02-23T00:46:46.167578+0000","last_peered":"2024-02-23T00:46:46.167578+0000","last_clean":"2024-02-23T00:46:46.167578+0000","last_became_active":"2024-02-23T00:39:18.796664+0000","last_became_peered":"2024-02-23T00:39:18.796664+0000","last_unstale":"2024-02-23T00:46:46.167578+0000","last_undegraded":"2024-02-23T00:46:46.167578+0000","last_fullsized":"2024-02-23T00:46:46.167578+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:45.294704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:45.294704+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"203'34","reported_seq":357,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.314274+0000","last_change":"2024-02-23T00:44:35.644033+0000","last_active":"2024-02-23T00:46:46.314274+0000","last_peered":"2024-02-23T00:46:46.314274+0000","last_clean":"2024-02-23T00:46:46.314274+0000","last_became_active":"2024-02-23T00:39:16.857278+0000","last_became_peered":"2024-02-23T00:39:16.857278+0000","last_unstale":"2024-02-23T00:46:46.314274+0000","last_undegraded":"2024-02-23T00:46:46.314274+0000","last_fullsized":"2024-02-23T00:46:46.314274+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'34","last_scrub_stamp":"2024-02-23T00:44:35.643931+0000","last_deep_scrub":"203'34","last_deep_scrub_stamp":"2024-02-23T00:44:35.643931+0000","last_clean_scrub_stamp":"2024-02-23T00:44:35.643931+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":296,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.172538+0000","last_change":"2024-02-23T00:45:12.730663+0000","last_active":"2024-02-23T00:46:46.172538+0000","last_peered":"2024-02-23T00:46:46.172538+0000","last_clean":"2024-02-23T00:46:46.172538+0000","last_became_active":"2024-02-23T00:39:14.783662+0000","last_became_peered":"2024-02-23T00:39:14.783662+0000","last_unstale":"2024-02-23T00:46:46.172538+0000","last_undegraded":"2024-02-23T00:46:46.172538+0000","last_fullsized":"2024-02-23T00:46:46.172538+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.730539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.730539+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":291,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.160500+0000","last_change":"2024-02-23T00:45:04.585266+0000","last_active":"2024-02-23T00:46:45.160500+0000","last_peered":"2024-02-23T00:46:45.160500+0000","last_clean":"2024-02-23T00:46:45.160500+0000","last_became_active":"2024-02-23T00:39:21.160060+0000","last_became_peered":"2024-02-23T00:39:21.160060+0000","last_unstale":"2024-02-23T00:46:45.160500+0000","last_undegraded":"2024-02-23T00:46:45.160500+0000","last_fullsized":"2024-02-23T00:46:45.160500+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:04.585179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.585179+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.8","version":"110'2","reported_seq":258,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.155343+0000","last_change":"2024-02-23T00:46:19.180741+0000","last_active":"2024-02-23T00:46:45.155343+0000","last_peered":"2024-02-23T00:46:45.155343+0000","last_clean":"2024-02-23T00:46:45.155343+0000","last_became_active":"2024-02-23T00:46:19.180391+0000","last_became_peered":"2024-02-23T00:46:19.180391+0000","last_unstale":"2024-02-23T00:46:45.155343+0000","last_undegraded":"2024-02-23T00:46:45.155343+0000","last_fullsized":"2024-02-23T00:46:45.155343+0000","mapping_epoch":108,"log_start":"110'2","ondisk_log_start":"110'2","created":108,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'2","last_scrub_stamp":"2024-02-23T00:44:33.389605+0000","last_deep_scrub":"110'2","last_deep_scrub_stamp":"2024-02-23T00:44:33.389605+0000","last_clean_scrub_stamp":"2024-02-23T00:44:33.389605+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":239,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.202969+0000","last_change":"2024-02-23T00:45:18.185097+0000","last_active":"2024-02-23T00:46:46.202969+0000","last_peered":"2024-02-23T00:46:46.202969+0000","last_clean":"2024-02-23T00:46:46.202969+0000","last_became_active":"2024-02-23T00:42:58.726652+0000","last_became_peered":"2024-02-23T00:42:58.726652+0000","last_unstale":"2024-02-23T00:46:46.202969+0000","last_undegraded":"2024-02-23T00:46:46.202969+0000","last_fullsized":"2024-02-23T00:46:46.202969+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:18.184992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:18.184992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":299,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.203375+0000","last_change":"2024-02-23T00:46:37.333068+0000","last_active":"2024-02-23T00:46:46.203375+0000","last_peered":"2024-02-23T00:46:46.203375+0000","last_clean":"2024-02-23T00:46:46.203375+0000","last_became_active":"2024-02-23T00:39:18.800378+0000","last_became_peered":"2024-02-23T00:39:18.800378+0000","last_unstale":"2024-02-23T00:46:46.203375+0000","last_undegraded":"2024-02-23T00:46:46.203375+0000","last_fullsized":"2024-02-23T00:46:46.203375+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.332973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:59.694033+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.332973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"95'18","reported_seq":321,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.315627+0000","last_change":"2024-02-23T00:44:18.630435+0000","last_active":"2024-02-23T00:46:46.315627+0000","last_peered":"2024-02-23T00:46:46.315627+0000","last_clean":"2024-02-23T00:46:46.315627+0000","last_became_active":"2024-02-23T00:39:16.787143+0000","last_became_peered":"2024-02-23T00:39:16.787143+0000","last_unstale":"2024-02-23T00:46:46.315627+0000","last_undegraded":"2024-02-23T00:46:46.315627+0000","last_fullsized":"2024-02-23T00:46:46.315627+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'18","last_scrub_stamp":"2024-02-23T00:44:18.630363+0000","last_deep_scrub":"95'18","last_deep_scrub_stamp":"2024-02-23T00:42:01.195545+0000","last_clean_scrub_stamp":"2024-02-23T00:44:18.630363+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":301,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.171396+0000","last_change":"2024-02-23T00:45:52.795624+0000","last_active":"2024-02-23T00:46:46.171396+0000","last_peered":"2024-02-23T00:46:46.171396+0000","last_clean":"2024-02-23T00:46:46.171396+0000","last_became_active":"2024-02-23T00:39:14.784578+0000","last_became_peered":"2024-02-23T00:39:14.784578+0000","last_unstale":"2024-02-23T00:46:46.171396+0000","last_undegraded":"2024-02-23T00:46:46.171396+0000","last_fullsized":"2024-02-23T00:46:46.171396+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:52.795527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:52.795527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":304,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.314710+0000","last_change":"2024-02-23T00:46:44.613082+0000","last_active":"2024-02-23T00:46:46.314710+0000","last_peered":"2024-02-23T00:46:46.314710+0000","last_clean":"2024-02-23T00:46:46.314710+0000","last_became_active":"2024-02-23T00:39:14.775446+0000","last_became_peered":"2024-02-23T00:39:14.775446+0000","last_unstale":"2024-02-23T00:46:46.314710+0000","last_undegraded":"2024-02-23T00:46:46.314710+0000","last_fullsized":"2024-02-23T00:46:46.314710+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:44.613012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:44.613012+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":286,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.163950+0000","last_change":"2024-02-23T00:43:47.905617+0000","last_active":"2024-02-23T00:46:45.163950+0000","last_peered":"2024-02-23T00:46:45.163950+0000","last_clean":"2024-02-23T00:46:45.163950+0000","last_became_active":"2024-02-23T00:39:20.797242+0000","last_became_peered":"2024-02-23T00:39:20.797242+0000","last_unstale":"2024-02-23T00:46:45.163950+0000","last_undegraded":"2024-02-23T00:46:45.163950+0000","last_fullsized":"2024-02-23T00:46:45.163950+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:47.905501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:47.905501+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"307'11","reported_seq":99,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.168979+0000","last_change":"2024-02-23T00:45:59.164904+0000","last_active":"2024-02-23T00:46:46.168979+0000","last_peered":"2024-02-23T00:46:46.168979+0000","last_clean":"2024-02-23T00:46:46.168979+0000","last_became_active":"2024-02-23T00:45:59.164483+0000","last_became_peered":"2024-02-23T00:45:59.164483+0000","last_unstale":"2024-02-23T00:46:46.168979+0000","last_undegraded":"2024-02-23T00:46:46.168979+0000","last_fullsized":"2024-02-23T00:46:46.168979+0000","mapping_epoch":272,"log_start":"110'7","ondisk_log_start":"110'7","created":108,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'5","last_scrub_stamp":"2024-02-23T00:45:26.760862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:26.760862+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":239,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.166550+0000","last_change":"2024-02-23T00:44:35.360128+0000","last_active":"2024-02-23T00:46:46.166550+0000","last_peered":"2024-02-23T00:46:46.166550+0000","last_clean":"2024-02-23T00:46:46.166550+0000","last_became_active":"2024-02-23T00:42:58.690858+0000","last_became_peered":"2024-02-23T00:42:58.690858+0000","last_unstale":"2024-02-23T00:46:46.166550+0000","last_undegraded":"2024-02-23T00:46:46.166550+0000","last_fullsized":"2024-02-23T00:46:46.166550+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:35.360054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:35.360054+0000","last_clean_scrub_stamp":"2024-02-23T00:44:35.360054+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":292,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.172071+0000","last_change":"2024-02-23T00:45:16.746109+0000","last_active":"2024-02-23T00:46:46.172071+0000","last_peered":"2024-02-23T00:46:46.172071+0000","last_clean":"2024-02-23T00:46:46.172071+0000","last_became_active":"2024-02-23T00:39:18.793801+0000","last_became_peered":"2024-02-23T00:39:18.793801+0000","last_unstale":"2024-02-23T00:46:46.172071+0000","last_undegraded":"2024-02-23T00:46:46.172071+0000","last_fullsized":"2024-02-23T00:46:46.172071+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.746013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.746013+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"67'14","reported_seq":322,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.169066+0000","last_change":"2024-02-23T00:46:11.980163+0000","last_active":"2024-02-23T00:46:46.169066+0000","last_peered":"2024-02-23T00:46:46.169066+0000","last_clean":"2024-02-23T00:46:46.169066+0000","last_became_active":"2024-02-23T00:39:16.858880+0000","last_became_peered":"2024-02-23T00:39:16.858880+0000","last_unstale":"2024-02-23T00:46:46.169066+0000","last_undegraded":"2024-02-23T00:46:46.169066+0000","last_fullsized":"2024-02-23T00:46:46.169066+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'14","last_scrub_stamp":"2024-02-23T00:46:11.980087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:11.980087+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":306,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.171549+0000","last_change":"2024-02-23T00:45:08.796153+0000","last_active":"2024-02-23T00:46:46.171549+0000","last_peered":"2024-02-23T00:46:46.171549+0000","last_clean":"2024-02-23T00:46:46.171549+0000","last_became_active":"2024-02-23T00:39:08.248249+0000","last_became_peered":"2024-02-23T00:39:08.248249+0000","last_unstale":"2024-02-23T00:46:46.171549+0000","last_undegraded":"2024-02-23T00:46:46.171549+0000","last_fullsized":"2024-02-23T00:46:46.171549+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.796076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:49.977248+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.796076+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":298,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.202853+0000","last_change":"2024-02-23T00:45:22.272078+0000","last_active":"2024-02-23T00:46:46.202853+0000","last_peered":"2024-02-23T00:46:46.202853+0000","last_clean":"2024-02-23T00:46:46.202853+0000","last_became_active":"2024-02-23T00:39:14.788888+0000","last_became_peered":"2024-02-23T00:39:14.788888+0000","last_unstale":"2024-02-23T00:46:46.202853+0000","last_undegraded":"2024-02-23T00:46:46.202853+0000","last_fullsized":"2024-02-23T00:46:46.202853+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:22.272003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:22.272003+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":299,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.163852+0000","last_change":"2024-02-23T00:44:33.143663+0000","last_active":"2024-02-23T00:46:46.163852+0000","last_peered":"2024-02-23T00:46:46.163852+0000","last_clean":"2024-02-23T00:46:46.163852+0000","last_became_active":"2024-02-23T00:39:14.783489+0000","last_became_peered":"2024-02-23T00:39:14.783489+0000","last_unstale":"2024-02-23T00:46:46.163852+0000","last_undegraded":"2024-02-23T00:46:46.163852+0000","last_fullsized":"2024-02-23T00:46:46.163852+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:33.143592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:33.143592+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":315,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.200957+0000","last_change":"2024-02-23T00:46:45.207341+0000","last_active":"2024-02-23T00:46:46.200957+0000","last_peered":"2024-02-23T00:46:46.200957+0000","last_clean":"2024-02-23T00:46:46.200957+0000","last_became_active":"2024-02-23T00:43:42.168435+0000","last_became_peered":"2024-02-23T00:43:42.168435+0000","last_unstale":"2024-02-23T00:46:46.200957+0000","last_undegraded":"2024-02-23T00:46:46.200957+0000","last_fullsized":"2024-02-23T00:46:46.200957+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:45.207275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:46:45.207275+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":90,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.315369+0000","last_change":"2024-02-23T00:45:55.156529+0000","last_active":"2024-02-23T00:46:46.315369+0000","last_peered":"2024-02-23T00:46:46.315369+0000","last_clean":"2024-02-23T00:46:46.315369+0000","last_became_active":"2024-02-23T00:45:55.156026+0000","last_became_peered":"2024-02-23T00:45:55.156026+0000","last_unstale":"2024-02-23T00:46:46.315369+0000","last_undegraded":"2024-02-23T00:46:46.315369+0000","last_fullsized":"2024-02-23T00:46:46.315369+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:21.383626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:21.383626+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":239,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.171382+0000","last_change":"2024-02-23T00:45:17.763618+0000","last_active":"2024-02-23T00:46:46.171382+0000","last_peered":"2024-02-23T00:46:46.171382+0000","last_clean":"2024-02-23T00:46:46.171382+0000","last_became_active":"2024-02-23T00:42:58.701502+0000","last_became_peered":"2024-02-23T00:42:58.701502+0000","last_unstale":"2024-02-23T00:46:46.171382+0000","last_undegraded":"2024-02-23T00:46:46.171382+0000","last_fullsized":"2024-02-23T00:46:46.171382+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:17.763482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:17.763482+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":297,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.157602+0000","last_change":"2024-02-23T00:46:09.924457+0000","last_active":"2024-02-23T00:46:45.157602+0000","last_peered":"2024-02-23T00:46:45.157602+0000","last_clean":"2024-02-23T00:46:45.157602+0000","last_became_active":"2024-02-23T00:39:18.789330+0000","last_became_peered":"2024-02-23T00:39:18.789330+0000","last_unstale":"2024-02-23T00:46:45.157602+0000","last_undegraded":"2024-02-23T00:46:45.157602+0000","last_fullsized":"2024-02-23T00:46:45.157602+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:09.924384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.264240+0000","last_clean_scrub_stamp":"2024-02-23T00:46:09.924384+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"68'19","reported_seq":337,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.163517+0000","last_change":"2024-02-23T00:45:07.697475+0000","last_active":"2024-02-23T00:46:45.163517+0000","last_peered":"2024-02-23T00:46:45.163517+0000","last_clean":"2024-02-23T00:46:45.163517+0000","last_became_active":"2024-02-23T00:39:16.787484+0000","last_became_peered":"2024-02-23T00:39:16.787484+0000","last_unstale":"2024-02-23T00:46:45.163517+0000","last_undegraded":"2024-02-23T00:46:45.163517+0000","last_fullsized":"2024-02-23T00:46:45.163517+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'19","last_scrub_stamp":"2024-02-23T00:45:07.697394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:07.697394+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":301,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.154857+0000","last_change":"2024-02-23T00:46:10.878272+0000","last_active":"2024-02-23T00:46:45.154857+0000","last_peered":"2024-02-23T00:46:45.154857+0000","last_clean":"2024-02-23T00:46:45.154857+0000","last_became_active":"2024-02-23T00:39:14.772411+0000","last_became_peered":"2024-02-23T00:39:14.772411+0000","last_unstale":"2024-02-23T00:46:45.154857+0000","last_undegraded":"2024-02-23T00:46:45.154857+0000","last_fullsized":"2024-02-23T00:46:45.154857+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:10.878204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:10.878204+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"68'5","reported_seq":739,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.169351+0000","last_change":"2024-02-23T00:43:48.876428+0000","last_active":"2024-02-23T00:46:46.169351+0000","last_peered":"2024-02-23T00:46:46.169351+0000","last_clean":"2024-02-23T00:46:46.169351+0000","last_became_active":"2024-02-23T00:39:14.779109+0000","last_became_peered":"2024-02-23T00:39:14.779109+0000","last_unstale":"2024-02-23T00:46:46.169351+0000","last_undegraded":"2024-02-23T00:46:46.169351+0000","last_fullsized":"2024-02-23T00:46:46.169351+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'5","last_scrub_stamp":"2024-02-23T00:43:48.876356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.876356+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":444,"num_read_kb":439,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":295,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.172477+0000","last_change":"2024-02-23T00:46:36.725799+0000","last_active":"2024-02-23T00:46:46.172477+0000","last_peered":"2024-02-23T00:46:46.172477+0000","last_clean":"2024-02-23T00:46:46.172477+0000","last_became_active":"2024-02-23T00:39:21.160457+0000","last_became_peered":"2024-02-23T00:39:21.160457+0000","last_unstale":"2024-02-23T00:46:46.172477+0000","last_undegraded":"2024-02-23T00:46:46.172477+0000","last_fullsized":"2024-02-23T00:46:46.172477+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:36.725686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:46:36.725686+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":89,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.162813+0000","last_change":"2024-02-23T00:46:03.662711+0000","last_active":"2024-02-23T00:46:45.162813+0000","last_peered":"2024-02-23T00:46:45.162813+0000","last_clean":"2024-02-23T00:46:45.162813+0000","last_became_active":"2024-02-23T00:46:03.662395+0000","last_became_peered":"2024-02-23T00:46:03.662395+0000","last_unstale":"2024-02-23T00:46:45.162813+0000","last_undegraded":"2024-02-23T00:46:45.162813+0000","last_fullsized":"2024-02-23T00:46:45.162813+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,3],"acting":[1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":239,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.160082+0000","last_change":"2024-02-23T00:46:42.331610+0000","last_active":"2024-02-23T00:46:45.160082+0000","last_peered":"2024-02-23T00:46:45.160082+0000","last_clean":"2024-02-23T00:46:45.160082+0000","last_became_active":"2024-02-23T00:42:57.653355+0000","last_became_peered":"2024-02-23T00:42:57.653355+0000","last_unstale":"2024-02-23T00:46:45.160082+0000","last_undegraded":"2024-02-23T00:46:45.160082+0000","last_fullsized":"2024-02-23T00:46:45.160082+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:42.331547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:46:42.331547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":295,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.315554+0000","last_change":"2024-02-23T00:44:20.626116+0000","last_active":"2024-02-23T00:46:46.315554+0000","last_peered":"2024-02-23T00:46:46.315554+0000","last_clean":"2024-02-23T00:46:46.315554+0000","last_became_active":"2024-02-23T00:39:18.802134+0000","last_became_peered":"2024-02-23T00:39:18.802134+0000","last_unstale":"2024-02-23T00:46:46.315554+0000","last_undegraded":"2024-02-23T00:46:46.315554+0000","last_fullsized":"2024-02-23T00:46:46.315554+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:20.626052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:20.626052+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"86'35","reported_seq":370,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.202796+0000","last_change":"2024-02-23T00:45:52.939553+0000","last_active":"2024-02-23T00:46:46.202796+0000","last_peered":"2024-02-23T00:46:46.202796+0000","last_clean":"2024-02-23T00:46:46.202796+0000","last_became_active":"2024-02-23T00:40:42.933322+0000","last_became_peered":"2024-02-23T00:40:42.933322+0000","last_unstale":"2024-02-23T00:46:46.202796+0000","last_undegraded":"2024-02-23T00:46:46.202796+0000","last_fullsized":"2024-02-23T00:46:46.202796+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'35","last_scrub_stamp":"2024-02-23T00:45:52.939452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:52.939452+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":298,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.202076+0000","last_change":"2024-02-23T00:44:23.965228+0000","last_active":"2024-02-23T00:46:46.202076+0000","last_peered":"2024-02-23T00:46:46.202076+0000","last_clean":"2024-02-23T00:46:46.202076+0000","last_became_active":"2024-02-23T00:39:14.784015+0000","last_became_peered":"2024-02-23T00:39:14.784015+0000","last_unstale":"2024-02-23T00:46:46.202076+0000","last_undegraded":"2024-02-23T00:46:46.202076+0000","last_fullsized":"2024-02-23T00:46:46.202076+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:23.965125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.965125+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":311,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.165749+0000","last_change":"2024-02-23T00:46:42.938278+0000","last_active":"2024-02-23T00:46:46.165749+0000","last_peered":"2024-02-23T00:46:46.165749+0000","last_clean":"2024-02-23T00:46:46.165749+0000","last_became_active":"2024-02-23T00:43:45.144043+0000","last_became_peered":"2024-02-23T00:43:45.144043+0000","last_unstale":"2024-02-23T00:46:46.165749+0000","last_undegraded":"2024-02-23T00:46:46.165749+0000","last_fullsized":"2024-02-23T00:46:46.165749+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:42.938209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:46:42.938209+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.f","version":"307'8","reported_seq":91,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.163185+0000","last_change":"2024-02-23T00:45:51.167662+0000","last_active":"2024-02-23T00:46:45.163185+0000","last_peered":"2024-02-23T00:46:45.163185+0000","last_clean":"2024-02-23T00:46:45.163185+0000","last_became_active":"2024-02-23T00:45:51.167136+0000","last_became_peered":"2024-02-23T00:45:51.167136+0000","last_unstale":"2024-02-23T00:46:45.163185+0000","last_undegraded":"2024-02-23T00:46:45.163185+0000","last_fullsized":"2024-02-23T00:46:45.163185+0000","mapping_epoch":272,"log_start":"254'6","ondisk_log_start":"254'6","created":108,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:31.410670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:31.410670+0000","last_clean_scrub_stamp":"2024-02-23T00:44:31.410670+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":238,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.159393+0000","last_change":"2024-02-23T00:45:05.490595+0000","last_active":"2024-02-23T00:46:45.159393+0000","last_peered":"2024-02-23T00:46:45.159393+0000","last_clean":"2024-02-23T00:46:45.159393+0000","last_became_active":"2024-02-23T00:42:58.766579+0000","last_became_peered":"2024-02-23T00:42:58.766579+0000","last_unstale":"2024-02-23T00:46:45.159393+0000","last_undegraded":"2024-02-23T00:46:45.159393+0000","last_fullsized":"2024-02-23T00:46:45.159393+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:05.490531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:05.490531+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"91'9","reported_seq":394,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.314657+0000","last_change":"2024-02-23T00:44:28.619036+0000","last_active":"2024-02-23T00:46:46.314657+0000","last_peered":"2024-02-23T00:46:46.314657+0000","last_clean":"2024-02-23T00:46:46.314657+0000","last_became_active":"2024-02-23T00:39:18.801297+0000","last_became_peered":"2024-02-23T00:39:18.801297+0000","last_unstale":"2024-02-23T00:46:46.314657+0000","last_undegraded":"2024-02-23T00:46:46.314657+0000","last_fullsized":"2024-02-23T00:46:46.314657+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'9","last_scrub_stamp":"2024-02-23T00:44:28.618923+0000","last_deep_scrub":"91'9","last_deep_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_clean_scrub_stamp":"2024-02-23T00:44:28.618923+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"68'16","reported_seq":325,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.163685+0000","last_change":"2024-02-23T00:45:11.713800+0000","last_active":"2024-02-23T00:46:45.163685+0000","last_peered":"2024-02-23T00:46:45.163685+0000","last_clean":"2024-02-23T00:46:45.163685+0000","last_became_active":"2024-02-23T00:39:16.787468+0000","last_became_peered":"2024-02-23T00:39:16.787468+0000","last_unstale":"2024-02-23T00:46:45.163685+0000","last_undegraded":"2024-02-23T00:46:45.163685+0000","last_fullsized":"2024-02-23T00:46:45.163685+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:45:11.713731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:11.713731+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":297,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.163701+0000","last_change":"2024-02-23T00:45:19.635062+0000","last_active":"2024-02-23T00:46:45.163701+0000","last_peered":"2024-02-23T00:46:45.163701+0000","last_clean":"2024-02-23T00:46:45.163701+0000","last_became_active":"2024-02-23T00:39:14.779952+0000","last_became_peered":"2024-02-23T00:39:14.779952+0000","last_unstale":"2024-02-23T00:46:45.163701+0000","last_undegraded":"2024-02-23T00:46:45.163701+0000","last_fullsized":"2024-02-23T00:46:45.163701+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:19.635002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:19.635002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"107'2","reported_seq":295,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.313304+0000","last_change":"2024-02-23T00:45:08.662754+0000","last_active":"2024-02-23T00:46:46.313304+0000","last_peered":"2024-02-23T00:46:46.313304+0000","last_clean":"2024-02-23T00:46:46.313304+0000","last_became_active":"2024-02-23T00:39:20.800858+0000","last_became_peered":"2024-02-23T00:39:20.800858+0000","last_unstale":"2024-02-23T00:46:46.313304+0000","last_undegraded":"2024-02-23T00:46:46.313304+0000","last_fullsized":"2024-02-23T00:46:46.313304+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'2","last_scrub_stamp":"2024-02-23T00:45:08.662667+0000","last_deep_scrub":"107'2","last_deep_scrub_stamp":"2024-02-23T00:42:48.362739+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.662667+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":257,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.168511+0000","last_change":"2024-02-23T00:46:15.806517+0000","last_active":"2024-02-23T00:46:46.168511+0000","last_peered":"2024-02-23T00:46:46.168511+0000","last_clean":"2024-02-23T00:46:46.168511+0000","last_became_active":"2024-02-23T00:46:15.806146+0000","last_became_peered":"2024-02-23T00:46:15.806146+0000","last_unstale":"2024-02-23T00:46:46.168511+0000","last_undegraded":"2024-02-23T00:46:46.168511+0000","last_fullsized":"2024-02-23T00:46:46.168511+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:19.711117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:19.711117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":239,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.168179+0000","last_change":"2024-02-23T00:45:04.621794+0000","last_active":"2024-02-23T00:46:46.168179+0000","last_peered":"2024-02-23T00:46:46.168179+0000","last_clean":"2024-02-23T00:46:46.168179+0000","last_became_active":"2024-02-23T00:42:58.664282+0000","last_became_peered":"2024-02-23T00:42:58.664282+0000","last_unstale":"2024-02-23T00:46:46.168179+0000","last_undegraded":"2024-02-23T00:46:46.168179+0000","last_fullsized":"2024-02-23T00:46:46.168179+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:04.621714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.621714+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":297,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.155573+0000","last_change":"2024-02-23T00:46:36.783887+0000","last_active":"2024-02-23T00:46:45.155573+0000","last_peered":"2024-02-23T00:46:45.155573+0000","last_clean":"2024-02-23T00:46:45.155573+0000","last_became_active":"2024-02-23T00:39:18.796323+0000","last_became_peered":"2024-02-23T00:39:18.796323+0000","last_unstale":"2024-02-23T00:46:45.155573+0000","last_undegraded":"2024-02-23T00:46:45.155573+0000","last_fullsized":"2024-02-23T00:46:45.155573+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:36.783823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_clean_scrub_stamp":"2024-02-23T00:46:36.783823+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"68'25","reported_seq":344,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.154946+0000","last_change":"2024-02-23T00:43:46.158231+0000","last_active":"2024-02-23T00:46:45.154946+0000","last_peered":"2024-02-23T00:46:45.154946+0000","last_clean":"2024-02-23T00:46:45.154946+0000","last_became_active":"2024-02-23T00:39:16.783926+0000","last_became_peered":"2024-02-23T00:39:16.783926+0000","last_unstale":"2024-02-23T00:46:45.154946+0000","last_undegraded":"2024-02-23T00:46:45.154946+0000","last_fullsized":"2024-02-23T00:46:45.154946+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'25","last_scrub_stamp":"2024-02-23T00:43:46.158149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:46.158149+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"95'4","reported_seq":312,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.155595+0000","last_change":"2024-02-23T00:44:31.106105+0000","last_active":"2024-02-23T00:46:45.155595+0000","last_peered":"2024-02-23T00:46:45.155595+0000","last_clean":"2024-02-23T00:46:45.155595+0000","last_became_active":"2024-02-23T00:39:14.779205+0000","last_became_peered":"2024-02-23T00:39:14.779205+0000","last_unstale":"2024-02-23T00:46:45.155595+0000","last_undegraded":"2024-02-23T00:46:45.155595+0000","last_fullsized":"2024-02-23T00:46:45.155595+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'4","last_scrub_stamp":"2024-02-23T00:44:31.106012+0000","last_deep_scrub":"95'4","last_deep_scrub_stamp":"2024-02-23T00:41:43.874397+0000","last_clean_scrub_stamp":"2024-02-23T00:44:31.106012+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":311,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.166918+0000","last_change":"2024-02-23T00:43:52.036315+0000","last_active":"2024-02-23T00:46:46.166918+0000","last_peered":"2024-02-23T00:46:46.166918+0000","last_clean":"2024-02-23T00:46:46.166918+0000","last_became_active":"2024-02-23T00:43:27.611660+0000","last_became_peered":"2024-02-23T00:43:27.611660+0000","last_unstale":"2024-02-23T00:46:46.166918+0000","last_undegraded":"2024-02-23T00:46:46.166918+0000","last_fullsized":"2024-02-23T00:46:46.166918+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:52.036249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:52.036249+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":256,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.159443+0000","last_change":"2024-02-23T00:46:43.632400+0000","last_active":"2024-02-23T00:46:45.159443+0000","last_peered":"2024-02-23T00:46:45.159443+0000","last_clean":"2024-02-23T00:46:45.159443+0000","last_became_active":"2024-02-23T00:46:43.632080+0000","last_became_peered":"2024-02-23T00:46:43.632080+0000","last_unstale":"2024-02-23T00:46:45.159443+0000","last_undegraded":"2024-02-23T00:46:45.159443+0000","last_fullsized":"2024-02-23T00:46:45.159443+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":331,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.388988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.388988+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":239,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.172364+0000","last_change":"2024-02-23T00:45:20.829124+0000","last_active":"2024-02-23T00:46:46.172364+0000","last_peered":"2024-02-23T00:46:46.172364+0000","last_clean":"2024-02-23T00:46:46.172364+0000","last_became_active":"2024-02-23T00:42:58.744983+0000","last_became_peered":"2024-02-23T00:42:58.744983+0000","last_unstale":"2024-02-23T00:46:46.172364+0000","last_undegraded":"2024-02-23T00:46:46.172364+0000","last_fullsized":"2024-02-23T00:46:46.172364+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:20.829052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:20.829052+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"68'21","reported_seq":343,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.645261+0000","last_change":"2024-02-23T00:46:46.645234+0000","last_active":"2024-02-23T00:46:46.645261+0000","last_peered":"2024-02-23T00:46:46.645261+0000","last_clean":"2024-02-23T00:46:46.645261+0000","last_became_active":"2024-02-23T00:39:16.858760+0000","last_became_peered":"2024-02-23T00:39:16.858760+0000","last_unstale":"2024-02-23T00:46:46.645261+0000","last_undegraded":"2024-02-23T00:46:46.645261+0000","last_fullsized":"2024-02-23T00:46:46.645261+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'21","last_scrub_stamp":"2024-02-23T00:46:46.645153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:46.645153+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":301,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.155492+0000","last_change":"2024-02-23T00:45:22.376128+0000","last_active":"2024-02-23T00:46:45.155492+0000","last_peered":"2024-02-23T00:46:45.155492+0000","last_clean":"2024-02-23T00:46:45.155492+0000","last_became_active":"2024-02-23T00:39:14.774244+0000","last_became_peered":"2024-02-23T00:39:14.774244+0000","last_unstale":"2024-02-23T00:46:45.155492+0000","last_undegraded":"2024-02-23T00:46:45.155492+0000","last_fullsized":"2024-02-23T00:46:45.155492+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:22.376063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:22.376063+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":300,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.165653+0000","last_change":"2024-02-23T00:46:45.913635+0000","last_active":"2024-02-23T00:46:46.165653+0000","last_peered":"2024-02-23T00:46:46.165653+0000","last_clean":"2024-02-23T00:46:46.165653+0000","last_became_active":"2024-02-23T00:39:18.798624+0000","last_became_peered":"2024-02-23T00:39:18.798624+0000","last_unstale":"2024-02-23T00:46:46.165653+0000","last_undegraded":"2024-02-23T00:46:46.165653+0000","last_fullsized":"2024-02-23T00:46:46.165653+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:45.913526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:45.913526+0000","last_clean_scrub_stamp":"2024-02-23T00:46:45.913526+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":290,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.157443+0000","last_change":"2024-02-23T00:44:26.432766+0000","last_active":"2024-02-23T00:46:45.157443+0000","last_peered":"2024-02-23T00:46:45.157443+0000","last_clean":"2024-02-23T00:46:45.157443+0000","last_became_active":"2024-02-23T00:39:20.796041+0000","last_became_peered":"2024-02-23T00:39:20.796041+0000","last_unstale":"2024-02-23T00:46:45.157443+0000","last_undegraded":"2024-02-23T00:46:45.157443+0000","last_fullsized":"2024-02-23T00:46:45.157443+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:26.432703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:26.432703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":240,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.313483+0000","last_change":"2024-02-23T00:44:26.595443+0000","last_active":"2024-02-23T00:46:46.313483+0000","last_peered":"2024-02-23T00:46:46.313483+0000","last_clean":"2024-02-23T00:46:46.313483+0000","last_became_active":"2024-02-23T00:42:58.825015+0000","last_became_peered":"2024-02-23T00:42:58.825015+0000","last_unstale":"2024-02-23T00:46:46.313483+0000","last_undegraded":"2024-02-23T00:46:46.313483+0000","last_fullsized":"2024-02-23T00:46:46.313483+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:26.595346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:44:26.595346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"107'16","reported_seq":325,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.165640+0000","last_change":"2024-02-23T00:45:07.012086+0000","last_active":"2024-02-23T00:46:46.165640+0000","last_peered":"2024-02-23T00:46:46.165640+0000","last_clean":"2024-02-23T00:46:46.165640+0000","last_became_active":"2024-02-23T00:39:16.858449+0000","last_became_peered":"2024-02-23T00:39:16.858449+0000","last_unstale":"2024-02-23T00:46:46.165640+0000","last_undegraded":"2024-02-23T00:46:46.165640+0000","last_fullsized":"2024-02-23T00:46:46.165640+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'16","last_scrub_stamp":"2024-02-23T00:45:07.011956+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-02-23T00:42:52.964510+0000","last_clean_scrub_stamp":"2024-02-23T00:45:07.011956+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":297,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.167190+0000","last_change":"2024-02-23T00:45:13.613736+0000","last_active":"2024-02-23T00:46:46.167190+0000","last_peered":"2024-02-23T00:46:46.167190+0000","last_clean":"2024-02-23T00:46:46.167190+0000","last_became_active":"2024-02-23T00:39:14.783164+0000","last_became_peered":"2024-02-23T00:39:14.783164+0000","last_unstale":"2024-02-23T00:46:46.167190+0000","last_undegraded":"2024-02-23T00:46:46.167190+0000","last_fullsized":"2024-02-23T00:46:46.167190+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:13.613654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:13.613654+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":299,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.203159+0000","last_change":"2024-02-23T00:46:42.258031+0000","last_active":"2024-02-23T00:46:46.203159+0000","last_peered":"2024-02-23T00:46:46.203159+0000","last_clean":"2024-02-23T00:46:46.203159+0000","last_became_active":"2024-02-23T00:39:18.800297+0000","last_became_peered":"2024-02-23T00:39:18.800297+0000","last_unstale":"2024-02-23T00:46:46.203159+0000","last_undegraded":"2024-02-23T00:46:46.203159+0000","last_fullsized":"2024-02-23T00:46:46.203159+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:42.257958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:42.257958+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":295,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.172376+0000","last_change":"2024-02-23T00:46:34.721645+0000","last_active":"2024-02-23T00:46:46.172376+0000","last_peered":"2024-02-23T00:46:46.172376+0000","last_clean":"2024-02-23T00:46:46.172376+0000","last_became_active":"2024-02-23T00:39:21.162036+0000","last_became_peered":"2024-02-23T00:39:21.162036+0000","last_unstale":"2024-02-23T00:46:46.172376+0000","last_undegraded":"2024-02-23T00:46:46.172376+0000","last_fullsized":"2024-02-23T00:46:46.172376+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:34.721564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:46:34.721564+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"110'7","reported_seq":253,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.164453+0000","last_change":"2024-02-23T00:45:18.947859+0000","last_active":"2024-02-23T00:46:46.164453+0000","last_peered":"2024-02-23T00:46:46.164453+0000","last_clean":"2024-02-23T00:46:46.164453+0000","last_became_active":"2024-02-23T00:42:58.777251+0000","last_became_peered":"2024-02-23T00:42:58.777251+0000","last_unstale":"2024-02-23T00:46:46.164453+0000","last_undegraded":"2024-02-23T00:46:46.164453+0000","last_fullsized":"2024-02-23T00:46:46.164453+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'7","last_scrub_stamp":"2024-02-23T00:45:18.947746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:18.947746+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"68'22","reported_seq":264,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.157239+0000","last_change":"2024-02-23T00:44:31.391929+0000","last_active":"2024-02-23T00:46:45.157239+0000","last_peered":"2024-02-23T00:46:45.157239+0000","last_clean":"2024-02-23T00:46:45.157239+0000","last_became_active":"2024-02-23T00:42:43.521189+0000","last_became_peered":"2024-02-23T00:42:43.521189+0000","last_unstale":"2024-02-23T00:46:45.157239+0000","last_undegraded":"2024-02-23T00:46:45.157239+0000","last_fullsized":"2024-02-23T00:46:45.157239+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'22","last_scrub_stamp":"2024-02-23T00:44:31.391819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:44:31.391819+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":37,"num_read_kb":30,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,7],"acting":[2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":303,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.202089+0000","last_change":"2024-02-23T00:46:46.201157+0000","last_active":"2024-02-23T00:46:46.202089+0000","last_peered":"2024-02-23T00:46:46.202089+0000","last_clean":"2024-02-23T00:46:46.202089+0000","last_became_active":"2024-02-23T00:39:14.785590+0000","last_became_peered":"2024-02-23T00:39:14.785590+0000","last_unstale":"2024-02-23T00:46:46.202089+0000","last_undegraded":"2024-02-23T00:46:46.202089+0000","last_fullsized":"2024-02-23T00:46:46.202089+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:46.201063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_clean_scrub_stamp":"2024-02-23T00:46:46.201063+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"67'8","reported_seq":392,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.203269+0000","last_change":"2024-02-23T00:44:29.038226+0000","last_active":"2024-02-23T00:46:46.203269+0000","last_peered":"2024-02-23T00:46:46.203269+0000","last_clean":"2024-02-23T00:46:46.203269+0000","last_became_active":"2024-02-23T00:39:18.803566+0000","last_became_peered":"2024-02-23T00:39:18.803566+0000","last_unstale":"2024-02-23T00:46:46.203269+0000","last_undegraded":"2024-02-23T00:46:46.203269+0000","last_fullsized":"2024-02-23T00:46:46.203269+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:44:29.038134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:29.038134+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":239,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.171231+0000","last_change":"2024-02-23T00:45:54.753371+0000","last_active":"2024-02-23T00:46:46.171231+0000","last_peered":"2024-02-23T00:46:46.171231+0000","last_clean":"2024-02-23T00:46:46.171231+0000","last_became_active":"2024-02-23T00:42:58.757424+0000","last_became_peered":"2024-02-23T00:42:58.757424+0000","last_unstale":"2024-02-23T00:46:46.171231+0000","last_undegraded":"2024-02-23T00:46:46.171231+0000","last_fullsized":"2024-02-23T00:46:46.171231+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:54.753267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:54.753267+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"68'15","reported_seq":327,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.203111+0000","last_change":"2024-02-23T00:44:25.998879+0000","last_active":"2024-02-23T00:46:46.203111+0000","last_peered":"2024-02-23T00:46:46.203111+0000","last_clean":"2024-02-23T00:46:46.203111+0000","last_became_active":"2024-02-23T00:39:16.858953+0000","last_became_peered":"2024-02-23T00:39:16.858953+0000","last_unstale":"2024-02-23T00:46:46.203111+0000","last_undegraded":"2024-02-23T00:46:46.203111+0000","last_fullsized":"2024-02-23T00:46:46.203111+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:44:25.998804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:44:25.998804+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":304,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.314430+0000","last_change":"2024-02-23T00:46:42.692530+0000","last_active":"2024-02-23T00:46:46.314430+0000","last_peered":"2024-02-23T00:46:46.314430+0000","last_clean":"2024-02-23T00:46:46.314430+0000","last_became_active":"2024-02-23T00:39:14.773772+0000","last_became_peered":"2024-02-23T00:39:14.773772+0000","last_unstale":"2024-02-23T00:46:46.314430+0000","last_undegraded":"2024-02-23T00:46:46.314430+0000","last_fullsized":"2024-02-23T00:46:46.314430+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:42.692434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:42.692434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"67'8","reported_seq":393,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.165288+0000","last_change":"2024-02-23T00:46:36.901142+0000","last_active":"2024-02-23T00:46:46.165288+0000","last_peered":"2024-02-23T00:46:46.165288+0000","last_clean":"2024-02-23T00:46:46.165288+0000","last_became_active":"2024-02-23T00:39:18.787313+0000","last_became_peered":"2024-02-23T00:39:18.787313+0000","last_unstale":"2024-02-23T00:46:46.165288+0000","last_undegraded":"2024-02-23T00:46:46.165288+0000","last_fullsized":"2024-02-23T00:46:46.165288+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:46:36.900988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:36.900988+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":234,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.164029+0000","last_change":"2024-02-23T00:42:57.652853+0000","last_active":"2024-02-23T00:46:45.164029+0000","last_peered":"2024-02-23T00:46:45.164029+0000","last_clean":"2024-02-23T00:46:45.164029+0000","last_became_active":"2024-02-23T00:42:57.652488+0000","last_became_peered":"2024-02-23T00:42:57.652488+0000","last_unstale":"2024-02-23T00:46:45.164029+0000","last_undegraded":"2024-02-23T00:46:45.164029+0000","last_fullsized":"2024-02-23T00:46:45.164029+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"67'9","reported_seq":311,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.163593+0000","last_change":"2024-02-23T00:46:36.406841+0000","last_active":"2024-02-23T00:46:45.163593+0000","last_peered":"2024-02-23T00:46:45.163593+0000","last_clean":"2024-02-23T00:46:45.163593+0000","last_became_active":"2024-02-23T00:39:16.786936+0000","last_became_peered":"2024-02-23T00:39:16.786936+0000","last_unstale":"2024-02-23T00:46:45.163593+0000","last_undegraded":"2024-02-23T00:46:45.163593+0000","last_fullsized":"2024-02-23T00:46:45.163593+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'9","last_scrub_stamp":"2024-02-23T00:46:36.406771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:36.406771+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":302,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.168072+0000","last_change":"2024-02-23T00:44:24.350861+0000","last_active":"2024-02-23T00:46:46.168072+0000","last_peered":"2024-02-23T00:46:46.168072+0000","last_clean":"2024-02-23T00:46:46.168072+0000","last_became_active":"2024-02-23T00:39:14.778455+0000","last_became_peered":"2024-02-23T00:39:14.778455+0000","last_unstale":"2024-02-23T00:46:46.168072+0000","last_undegraded":"2024-02-23T00:46:46.168072+0000","last_fullsized":"2024-02-23T00:46:46.168072+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:24.350797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:24.350797+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":295,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.166941+0000","last_change":"2024-02-23T00:44:37.105079+0000","last_active":"2024-02-23T00:46:46.166941+0000","last_peered":"2024-02-23T00:46:46.166941+0000","last_clean":"2024-02-23T00:46:46.166941+0000","last_became_active":"2024-02-23T00:39:18.799052+0000","last_became_peered":"2024-02-23T00:39:18.799052+0000","last_unstale":"2024-02-23T00:46:46.166941+0000","last_undegraded":"2024-02-23T00:46:46.166941+0000","last_fullsized":"2024-02-23T00:46:46.166941+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:37.105013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_clean_scrub_stamp":"2024-02-23T00:44:37.105013+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":293,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.313305+0000","last_change":"2024-02-23T00:46:00.558774+0000","last_active":"2024-02-23T00:46:46.313305+0000","last_peered":"2024-02-23T00:46:46.313305+0000","last_clean":"2024-02-23T00:46:46.313305+0000","last_became_active":"2024-02-23T00:39:20.802648+0000","last_became_peered":"2024-02-23T00:39:20.802648+0000","last_unstale":"2024-02-23T00:46:46.313305+0000","last_undegraded":"2024-02-23T00:46:46.313305+0000","last_fullsized":"2024-02-23T00:46:46.313305+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:00.558661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:46:00.558661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":240,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.314794+0000","last_change":"2024-02-23T00:44:23.543049+0000","last_active":"2024-02-23T00:46:46.314794+0000","last_peered":"2024-02-23T00:46:46.314794+0000","last_clean":"2024-02-23T00:46:46.314794+0000","last_became_active":"2024-02-23T00:42:58.835862+0000","last_became_peered":"2024-02-23T00:42:58.835862+0000","last_unstale":"2024-02-23T00:46:46.314794+0000","last_undegraded":"2024-02-23T00:46:46.314794+0000","last_fullsized":"2024-02-23T00:46:46.314794+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:23.542975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.542975+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"67'11","reported_seq":314,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.155717+0000","last_change":"2024-02-23T00:45:13.315510+0000","last_active":"2024-02-23T00:46:45.155717+0000","last_peered":"2024-02-23T00:46:45.155717+0000","last_clean":"2024-02-23T00:46:45.155717+0000","last_became_active":"2024-02-23T00:39:16.783896+0000","last_became_peered":"2024-02-23T00:39:16.783896+0000","last_unstale":"2024-02-23T00:46:45.155717+0000","last_undegraded":"2024-02-23T00:46:45.155717+0000","last_fullsized":"2024-02-23T00:46:45.155717+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'11","last_scrub_stamp":"2024-02-23T00:45:13.315434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:13.315434+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"202'2","reported_seq":305,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.164802+0000","last_change":"2024-02-23T00:44:34.720705+0000","last_active":"2024-02-23T00:46:45.164802+0000","last_peered":"2024-02-23T00:46:45.164802+0000","last_clean":"2024-02-23T00:46:45.164802+0000","last_became_active":"2024-02-23T00:39:14.775804+0000","last_became_peered":"2024-02-23T00:39:14.775804+0000","last_unstale":"2024-02-23T00:46:45.164802+0000","last_undegraded":"2024-02-23T00:46:45.164802+0000","last_fullsized":"2024-02-23T00:46:45.164802+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"202'2","last_scrub_stamp":"2024-02-23T00:44:34.720640+0000","last_deep_scrub":"202'2","last_deep_scrub_stamp":"2024-02-23T00:44:34.720640+0000","last_clean_scrub_stamp":"2024-02-23T00:44:34.720640+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":288,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.163644+0000","last_change":"2024-02-23T00:43:31.903157+0000","last_active":"2024-02-23T00:46:45.163644+0000","last_peered":"2024-02-23T00:46:45.163644+0000","last_clean":"2024-02-23T00:46:45.163644+0000","last_became_active":"2024-02-23T00:39:18.800382+0000","last_became_peered":"2024-02-23T00:39:18.800382+0000","last_unstale":"2024-02-23T00:46:45.163644+0000","last_undegraded":"2024-02-23T00:46:45.163644+0000","last_fullsized":"2024-02-23T00:46:45.163644+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:31.903031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:31.903031+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":293,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.163961+0000","last_change":"2024-02-23T00:44:23.152677+0000","last_active":"2024-02-23T00:46:46.163961+0000","last_peered":"2024-02-23T00:46:46.163961+0000","last_clean":"2024-02-23T00:46:46.163961+0000","last_became_active":"2024-02-23T00:39:20.799053+0000","last_became_peered":"2024-02-23T00:39:20.799053+0000","last_unstale":"2024-02-23T00:46:46.163961+0000","last_undegraded":"2024-02-23T00:46:46.163961+0000","last_fullsized":"2024-02-23T00:46:46.163961+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:23.152602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.152602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":234,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.164746+0000","last_change":"2024-02-23T00:42:57.652742+0000","last_active":"2024-02-23T00:46:45.164746+0000","last_peered":"2024-02-23T00:46:45.164746+0000","last_clean":"2024-02-23T00:46:45.164746+0000","last_became_active":"2024-02-23T00:42:57.652438+0000","last_became_peered":"2024-02-23T00:42:57.652438+0000","last_unstale":"2024-02-23T00:46:45.164746+0000","last_undegraded":"2024-02-23T00:46:45.164746+0000","last_fullsized":"2024-02-23T00:46:45.164746+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:42:53.113727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"67'4","reported_seq":303,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.163701+0000","last_change":"2024-02-23T00:45:20.967211+0000","last_active":"2024-02-23T00:46:46.163701+0000","last_peered":"2024-02-23T00:46:46.163701+0000","last_clean":"2024-02-23T00:46:46.163701+0000","last_became_active":"2024-02-23T00:39:16.785830+0000","last_became_peered":"2024-02-23T00:39:16.785830+0000","last_unstale":"2024-02-23T00:46:46.163701+0000","last_undegraded":"2024-02-23T00:46:46.163701+0000","last_fullsized":"2024-02-23T00:46:46.163701+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'4","last_scrub_stamp":"2024-02-23T00:45:20.967095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:20.967095+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":306,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.203290+0000","last_change":"2024-02-23T00:46:35.268544+0000","last_active":"2024-02-23T00:46:46.203290+0000","last_peered":"2024-02-23T00:46:46.203290+0000","last_clean":"2024-02-23T00:46:46.203290+0000","last_became_active":"2024-02-23T00:39:14.783325+0000","last_became_peered":"2024-02-23T00:39:14.783325+0000","last_unstale":"2024-02-23T00:46:46.203290+0000","last_undegraded":"2024-02-23T00:46:46.203290+0000","last_fullsized":"2024-02-23T00:46:46.203290+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:35.268491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:35.268491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":298,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.167050+0000","last_change":"2024-02-23T00:45:20.761547+0000","last_active":"2024-02-23T00:46:46.167050+0000","last_peered":"2024-02-23T00:46:46.167050+0000","last_clean":"2024-02-23T00:46:46.167050+0000","last_became_active":"2024-02-23T00:39:18.797287+0000","last_became_peered":"2024-02-23T00:39:18.797287+0000","last_unstale":"2024-02-23T00:46:46.167050+0000","last_undegraded":"2024-02-23T00:46:46.167050+0000","last_fullsized":"2024-02-23T00:46:46.167050+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:20.761440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:20.761440+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":290,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.171068+0000","last_change":"2024-02-23T00:45:10.741762+0000","last_active":"2024-02-23T00:46:46.171068+0000","last_peered":"2024-02-23T00:46:46.171068+0000","last_clean":"2024-02-23T00:46:46.171068+0000","last_became_active":"2024-02-23T00:39:20.795351+0000","last_became_peered":"2024-02-23T00:39:20.795351+0000","last_unstale":"2024-02-23T00:46:46.171068+0000","last_undegraded":"2024-02-23T00:46:46.171068+0000","last_fullsized":"2024-02-23T00:46:46.171068+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.741694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.741694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":239,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.172122+0000","last_change":"2024-02-23T00:45:56.706901+0000","last_active":"2024-02-23T00:46:46.172122+0000","last_peered":"2024-02-23T00:46:46.172122+0000","last_clean":"2024-02-23T00:46:46.172122+0000","last_became_active":"2024-02-23T00:42:58.741365+0000","last_became_peered":"2024-02-23T00:42:58.741365+0000","last_unstale":"2024-02-23T00:46:46.172122+0000","last_undegraded":"2024-02-23T00:46:46.172122+0000","last_fullsized":"2024-02-23T00:46:46.172122+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:56.706792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:56.706792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"68'17","reported_seq":331,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.164645+0000","last_change":"2024-02-23T00:43:43.997579+0000","last_active":"2024-02-23T00:46:46.164645+0000","last_peered":"2024-02-23T00:46:46.164645+0000","last_clean":"2024-02-23T00:46:46.164645+0000","last_became_active":"2024-02-23T00:39:16.857232+0000","last_became_peered":"2024-02-23T00:39:16.857232+0000","last_unstale":"2024-02-23T00:46:46.164645+0000","last_undegraded":"2024-02-23T00:46:46.164645+0000","last_fullsized":"2024-02-23T00:46:46.164645+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:43:43.997452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:43.997452+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":302,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.168529+0000","last_change":"2024-02-23T00:46:44.285195+0000","last_active":"2024-02-23T00:46:46.168529+0000","last_peered":"2024-02-23T00:46:46.168529+0000","last_clean":"2024-02-23T00:46:46.168529+0000","last_became_active":"2024-02-23T00:39:14.784092+0000","last_became_peered":"2024-02-23T00:39:14.784092+0000","last_unstale":"2024-02-23T00:46:46.168529+0000","last_undegraded":"2024-02-23T00:46:46.168529+0000","last_fullsized":"2024-02-23T00:46:46.168529+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:44.285083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:44.285083+0000","last_clean_scrub_stamp":"2024-02-23T00:46:44.285083+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":297,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.171734+0000","last_change":"2024-02-23T00:46:39.716023+0000","last_active":"2024-02-23T00:46:46.171734+0000","last_peered":"2024-02-23T00:46:46.171734+0000","last_clean":"2024-02-23T00:46:46.171734+0000","last_became_active":"2024-02-23T00:39:18.791457+0000","last_became_peered":"2024-02-23T00:39:18.791457+0000","last_unstale":"2024-02-23T00:46:46.171734+0000","last_undegraded":"2024-02-23T00:46:46.171734+0000","last_fullsized":"2024-02-23T00:46:46.171734+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:39.715957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:39.715957+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":293,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.166969+0000","last_change":"2024-02-23T00:46:43.945089+0000","last_active":"2024-02-23T00:46:46.166969+0000","last_peered":"2024-02-23T00:46:46.166969+0000","last_clean":"2024-02-23T00:46:46.166969+0000","last_became_active":"2024-02-23T00:39:21.160376+0000","last_became_peered":"2024-02-23T00:39:21.160376+0000","last_unstale":"2024-02-23T00:46:46.166969+0000","last_undegraded":"2024-02-23T00:46:46.166969+0000","last_fullsized":"2024-02-23T00:46:46.166969+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:43.945020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:46:43.945020+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":238,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.156441+0000","last_change":"2024-02-23T00:45:23.378334+0000","last_active":"2024-02-23T00:46:45.156441+0000","last_peered":"2024-02-23T00:46:45.156441+0000","last_clean":"2024-02-23T00:46:45.156441+0000","last_became_active":"2024-02-23T00:42:58.702580+0000","last_became_peered":"2024-02-23T00:42:58.702580+0000","last_unstale":"2024-02-23T00:46:45.156441+0000","last_undegraded":"2024-02-23T00:46:45.156441+0000","last_fullsized":"2024-02-23T00:46:45.156441+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:23.378270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.378270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"68'16","reported_seq":327,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.315936+0000","last_change":"2024-02-23T00:45:04.672612+0000","last_active":"2024-02-23T00:46:46.315936+0000","last_peered":"2024-02-23T00:46:46.315936+0000","last_clean":"2024-02-23T00:46:46.315936+0000","last_became_active":"2024-02-23T00:39:16.858945+0000","last_became_peered":"2024-02-23T00:39:16.858945+0000","last_unstale":"2024-02-23T00:46:46.315936+0000","last_undegraded":"2024-02-23T00:46:46.315936+0000","last_fullsized":"2024-02-23T00:46:46.315936+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:45:04.672537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.672537+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":310,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.201488+0000","last_change":"2024-02-23T00:46:09.088334+0000","last_active":"2024-02-23T00:46:46.201488+0000","last_peered":"2024-02-23T00:46:46.201488+0000","last_clean":"2024-02-23T00:46:46.201488+0000","last_became_active":"2024-02-23T00:39:14.785363+0000","last_became_peered":"2024-02-23T00:39:14.785363+0000","last_unstale":"2024-02-23T00:46:46.201488+0000","last_undegraded":"2024-02-23T00:46:46.201488+0000","last_fullsized":"2024-02-23T00:46:46.201488+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"80'2","last_scrub_stamp":"2024-02-23T00:46:09.088249+0000","last_deep_scrub":"80'2","last_deep_scrub_stamp":"2024-02-23T00:40:15.942725+0000","last_clean_scrub_stamp":"2024-02-23T00:46:09.088249+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"93'9","reported_seq":400,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.168752+0000","last_change":"2024-02-23T00:46:37.230102+0000","last_active":"2024-02-23T00:46:46.168752+0000","last_peered":"2024-02-23T00:46:46.168752+0000","last_clean":"2024-02-23T00:46:46.168752+0000","last_became_active":"2024-02-23T00:39:18.796680+0000","last_became_peered":"2024-02-23T00:39:18.796680+0000","last_unstale":"2024-02-23T00:46:46.168752+0000","last_undegraded":"2024-02-23T00:46:46.168752+0000","last_fullsized":"2024-02-23T00:46:46.168752+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-02-23T00:46:37.230030+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-02-23T00:43:50.393713+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.230030+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":240,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.313811+0000","last_change":"2024-02-23T00:45:58.613561+0000","last_active":"2024-02-23T00:46:46.313811+0000","last_peered":"2024-02-23T00:46:46.313811+0000","last_clean":"2024-02-23T00:46:46.313811+0000","last_became_active":"2024-02-23T00:42:58.850072+0000","last_became_peered":"2024-02-23T00:42:58.850072+0000","last_unstale":"2024-02-23T00:46:46.313811+0000","last_undegraded":"2024-02-23T00:46:46.313811+0000","last_fullsized":"2024-02-23T00:46:46.313811+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:58.613497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:58.613497+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"93'23","reported_seq":340,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.168832+0000","last_change":"2024-02-23T00:44:28.387225+0000","last_active":"2024-02-23T00:46:46.168832+0000","last_peered":"2024-02-23T00:46:46.168832+0000","last_clean":"2024-02-23T00:46:46.168832+0000","last_became_active":"2024-02-23T00:39:16.782660+0000","last_became_peered":"2024-02-23T00:39:16.782660+0000","last_unstale":"2024-02-23T00:46:46.168832+0000","last_undegraded":"2024-02-23T00:46:46.168832+0000","last_fullsized":"2024-02-23T00:46:46.168832+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'23","last_scrub_stamp":"2024-02-23T00:44:28.387155+0000","last_deep_scrub":"93'23","last_deep_scrub_stamp":"2024-02-23T00:41:35.954516+0000","last_clean_scrub_stamp":"2024-02-23T00:44:28.387155+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":299,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.164111+0000","last_change":"2024-02-23T00:44:21.157372+0000","last_active":"2024-02-23T00:46:46.164111+0000","last_peered":"2024-02-23T00:46:46.164111+0000","last_clean":"2024-02-23T00:46:46.164111+0000","last_became_active":"2024-02-23T00:39:14.781560+0000","last_became_peered":"2024-02-23T00:39:14.781560+0000","last_unstale":"2024-02-23T00:46:46.164111+0000","last_undegraded":"2024-02-23T00:46:46.164111+0000","last_fullsized":"2024-02-23T00:46:46.164111+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:21.157316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:21.157316+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":300,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.316036+0000","last_change":"2024-02-23T00:46:36.592971+0000","last_active":"2024-02-23T00:46:46.316036+0000","last_peered":"2024-02-23T00:46:46.316036+0000","last_clean":"2024-02-23T00:46:46.316036+0000","last_became_active":"2024-02-23T00:39:18.790602+0000","last_became_peered":"2024-02-23T00:39:18.790602+0000","last_unstale":"2024-02-23T00:46:46.316036+0000","last_undegraded":"2024-02-23T00:46:46.316036+0000","last_fullsized":"2024-02-23T00:46:46.316036+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:36.592887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:53.189881+0000","last_clean_scrub_stamp":"2024-02-23T00:46:36.592887+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":240,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.164202+0000","last_change":"2024-02-23T00:45:15.933508+0000","last_active":"2024-02-23T00:46:46.164202+0000","last_peered":"2024-02-23T00:46:46.164202+0000","last_clean":"2024-02-23T00:46:46.164202+0000","last_became_active":"2024-02-23T00:42:58.774720+0000","last_became_peered":"2024-02-23T00:42:58.774720+0000","last_unstale":"2024-02-23T00:46:46.164202+0000","last_undegraded":"2024-02-23T00:46:46.164202+0000","last_fullsized":"2024-02-23T00:46:46.164202+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:15.933438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:15.933438+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"67'15","reported_seq":324,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.157586+0000","last_change":"2024-02-23T00:46:03.951592+0000","last_active":"2024-02-23T00:46:45.157586+0000","last_peered":"2024-02-23T00:46:45.157586+0000","last_clean":"2024-02-23T00:46:45.157586+0000","last_became_active":"2024-02-23T00:39:16.787138+0000","last_became_peered":"2024-02-23T00:39:16.787138+0000","last_unstale":"2024-02-23T00:46:45.157586+0000","last_undegraded":"2024-02-23T00:46:45.157586+0000","last_fullsized":"2024-02-23T00:46:45.157586+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:46:03.951524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:03.951524+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":302,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.167438+0000","last_change":"2024-02-23T00:46:10.011072+0000","last_active":"2024-02-23T00:46:46.167438+0000","last_peered":"2024-02-23T00:46:46.167438+0000","last_clean":"2024-02-23T00:46:46.167438+0000","last_became_active":"2024-02-23T00:39:14.782494+0000","last_became_peered":"2024-02-23T00:39:14.782494+0000","last_unstale":"2024-02-23T00:46:46.167438+0000","last_undegraded":"2024-02-23T00:46:46.167438+0000","last_fullsized":"2024-02-23T00:46:46.167438+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:10.010996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:10.010996+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":293,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.164944+0000","last_change":"2024-02-23T00:45:03.615758+0000","last_active":"2024-02-23T00:46:45.164944+0000","last_peered":"2024-02-23T00:46:45.164944+0000","last_clean":"2024-02-23T00:46:45.164944+0000","last_became_active":"2024-02-23T00:39:18.801039+0000","last_became_peered":"2024-02-23T00:39:18.801039+0000","last_unstale":"2024-02-23T00:46:45.164944+0000","last_undegraded":"2024-02-23T00:46:45.164944+0000","last_fullsized":"2024-02-23T00:46:45.164944+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:03.615692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.641689+0000","last_clean_scrub_stamp":"2024-02-23T00:45:03.615692+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":240,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.164583+0000","last_change":"2024-02-23T00:45:21.917072+0000","last_active":"2024-02-23T00:46:46.164583+0000","last_peered":"2024-02-23T00:46:46.164583+0000","last_clean":"2024-02-23T00:46:46.164583+0000","last_became_active":"2024-02-23T00:42:58.770759+0000","last_became_peered":"2024-02-23T00:42:58.770759+0000","last_unstale":"2024-02-23T00:46:46.164583+0000","last_undegraded":"2024-02-23T00:46:46.164583+0000","last_fullsized":"2024-02-23T00:46:46.164583+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:21.916996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:21.916996+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"68'18","reported_seq":340,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.165076+0000","last_change":"2024-02-23T00:46:44.911346+0000","last_active":"2024-02-23T00:46:46.165076+0000","last_peered":"2024-02-23T00:46:46.165076+0000","last_clean":"2024-02-23T00:46:46.165076+0000","last_became_active":"2024-02-23T00:39:16.782803+0000","last_became_peered":"2024-02-23T00:39:16.782803+0000","last_unstale":"2024-02-23T00:46:46.165076+0000","last_undegraded":"2024-02-23T00:46:46.165076+0000","last_fullsized":"2024-02-23T00:46:46.165076+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'18","last_scrub_stamp":"2024-02-23T00:46:44.911278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:44.911278+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":299,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.166848+0000","last_change":"2024-02-23T00:43:48.030906+0000","last_active":"2024-02-23T00:46:46.166848+0000","last_peered":"2024-02-23T00:46:46.166848+0000","last_clean":"2024-02-23T00:46:46.166848+0000","last_became_active":"2024-02-23T00:39:14.782163+0000","last_became_peered":"2024-02-23T00:39:14.782163+0000","last_unstale":"2024-02-23T00:46:46.166848+0000","last_undegraded":"2024-02-23T00:46:46.166848+0000","last_fullsized":"2024-02-23T00:46:46.166848+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:48.030846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.030846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":293,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.155184+0000","last_change":"2024-02-23T00:44:37.068899+0000","last_active":"2024-02-23T00:46:45.155184+0000","last_peered":"2024-02-23T00:46:45.155184+0000","last_clean":"2024-02-23T00:46:45.155184+0000","last_became_active":"2024-02-23T00:39:18.794903+0000","last_became_peered":"2024-02-23T00:39:18.794903+0000","last_unstale":"2024-02-23T00:46:45.155184+0000","last_undegraded":"2024-02-23T00:46:45.155184+0000","last_fullsized":"2024-02-23T00:46:45.155184+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:37.068834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_clean_scrub_stamp":"2024-02-23T00:44:37.068834+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":239,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.202962+0000","last_change":"2024-02-23T00:46:00.943625+0000","last_active":"2024-02-23T00:46:46.202962+0000","last_peered":"2024-02-23T00:46:46.202962+0000","last_clean":"2024-02-23T00:46:46.202962+0000","last_became_active":"2024-02-23T00:42:58.698258+0000","last_became_peered":"2024-02-23T00:42:58.698258+0000","last_unstale":"2024-02-23T00:46:46.202962+0000","last_undegraded":"2024-02-23T00:46:46.202962+0000","last_fullsized":"2024-02-23T00:46:46.202962+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:00.943545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:46:00.943545+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"95'27","reported_seq":347,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.154329+0000","last_change":"2024-02-23T00:43:52.124336+0000","last_active":"2024-02-23T00:46:45.154329+0000","last_peered":"2024-02-23T00:46:45.154329+0000","last_clean":"2024-02-23T00:46:45.154329+0000","last_became_active":"2024-02-23T00:39:16.786791+0000","last_became_peered":"2024-02-23T00:39:16.786791+0000","last_unstale":"2024-02-23T00:46:45.154329+0000","last_undegraded":"2024-02-23T00:46:45.154329+0000","last_fullsized":"2024-02-23T00:46:45.154329+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'27","last_scrub_stamp":"2024-02-23T00:43:52.124247+0000","last_deep_scrub":"95'27","last_deep_scrub_stamp":"2024-02-23T00:41:55.919895+0000","last_clean_scrub_stamp":"2024-02-23T00:43:52.124247+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":296,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.171855+0000","last_change":"2024-02-23T00:44:27.759571+0000","last_active":"2024-02-23T00:46:46.171855+0000","last_peered":"2024-02-23T00:46:46.171855+0000","last_clean":"2024-02-23T00:46:46.171855+0000","last_became_active":"2024-02-23T00:39:14.787148+0000","last_became_peered":"2024-02-23T00:39:14.787148+0000","last_unstale":"2024-02-23T00:46:46.171855+0000","last_undegraded":"2024-02-23T00:46:46.171855+0000","last_fullsized":"2024-02-23T00:46:46.171855+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:27.759499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:27.759499+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":294,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.202859+0000","last_change":"2024-02-23T00:45:10.090353+0000","last_active":"2024-02-23T00:46:46.202859+0000","last_peered":"2024-02-23T00:46:46.202859+0000","last_clean":"2024-02-23T00:46:46.202859+0000","last_became_active":"2024-02-23T00:39:18.796431+0000","last_became_peered":"2024-02-23T00:39:18.796431+0000","last_unstale":"2024-02-23T00:46:46.202859+0000","last_undegraded":"2024-02-23T00:46:46.202859+0000","last_fullsized":"2024-02-23T00:46:46.202859+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.090270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.090270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":239,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.203004+0000","last_change":"2024-02-23T00:45:51.985111+0000","last_active":"2024-02-23T00:46:46.203004+0000","last_peered":"2024-02-23T00:46:46.203004+0000","last_clean":"2024-02-23T00:46:46.203004+0000","last_became_active":"2024-02-23T00:42:58.832885+0000","last_became_peered":"2024-02-23T00:42:58.832885+0000","last_unstale":"2024-02-23T00:46:46.203004+0000","last_undegraded":"2024-02-23T00:46:46.203004+0000","last_fullsized":"2024-02-23T00:46:46.203004+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:51.985023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:51.985023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"82'15","reported_seq":324,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.155842+0000","last_change":"2024-02-23T00:44:35.095478+0000","last_active":"2024-02-23T00:46:45.155842+0000","last_peered":"2024-02-23T00:46:45.155842+0000","last_clean":"2024-02-23T00:46:45.155842+0000","last_became_active":"2024-02-23T00:39:16.782762+0000","last_became_peered":"2024-02-23T00:39:16.782762+0000","last_unstale":"2024-02-23T00:46:45.155842+0000","last_undegraded":"2024-02-23T00:46:45.155842+0000","last_fullsized":"2024-02-23T00:46:45.155842+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'15","last_scrub_stamp":"2024-02-23T00:44:35.095409+0000","last_deep_scrub":"82'15","last_deep_scrub_stamp":"2024-02-23T00:41:50.975683+0000","last_clean_scrub_stamp":"2024-02-23T00:44:35.095409+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":301,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.172381+0000","last_change":"2024-02-23T00:46:08.794420+0000","last_active":"2024-02-23T00:46:46.172381+0000","last_peered":"2024-02-23T00:46:46.172381+0000","last_clean":"2024-02-23T00:46:46.172381+0000","last_became_active":"2024-02-23T00:39:14.783469+0000","last_became_peered":"2024-02-23T00:39:14.783469+0000","last_unstale":"2024-02-23T00:46:46.172381+0000","last_undegraded":"2024-02-23T00:46:46.172381+0000","last_fullsized":"2024-02-23T00:46:46.172381+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:08.794352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:08.794352+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":299,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.171110+0000","last_change":"2024-02-23T00:46:09.832330+0000","last_active":"2024-02-23T00:46:46.171110+0000","last_peered":"2024-02-23T00:46:46.171110+0000","last_clean":"2024-02-23T00:46:46.171110+0000","last_became_active":"2024-02-23T00:39:18.796952+0000","last_became_peered":"2024-02-23T00:39:18.796952+0000","last_unstale":"2024-02-23T00:46:46.171110+0000","last_undegraded":"2024-02-23T00:46:46.171110+0000","last_fullsized":"2024-02-23T00:46:46.171110+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:09.832253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:09.832253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":71,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.154007+0000","last_change":"2024-02-23T00:45:45.160209+0000","last_active":"2024-02-23T00:46:45.154007+0000","last_peered":"2024-02-23T00:46:45.154007+0000","last_clean":"2024-02-23T00:46:45.154007+0000","last_became_active":"2024-02-23T00:45:45.159835+0000","last_became_peered":"2024-02-23T00:45:45.159835+0000","last_unstale":"2024-02-23T00:46:45.154007+0000","last_undegraded":"2024-02-23T00:46:45.154007+0000","last_fullsized":"2024-02-23T00:46:45.154007+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.215063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.215063+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":241,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.155006+0000","last_change":"2024-02-23T00:45:23.224771+0000","last_active":"2024-02-23T00:46:45.155006+0000","last_peered":"2024-02-23T00:46:45.155006+0000","last_clean":"2024-02-23T00:46:45.155006+0000","last_became_active":"2024-02-23T00:42:58.758147+0000","last_became_peered":"2024-02-23T00:42:58.758147+0000","last_unstale":"2024-02-23T00:46:45.155006+0000","last_undegraded":"2024-02-23T00:46:45.155006+0000","last_fullsized":"2024-02-23T00:46:45.155006+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:23.224663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:23.224663+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.224663+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"68'15","reported_seq":326,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.154407+0000","last_change":"2024-02-23T00:43:49.139235+0000","last_active":"2024-02-23T00:46:45.154407+0000","last_peered":"2024-02-23T00:46:45.154407+0000","last_clean":"2024-02-23T00:46:45.154407+0000","last_became_active":"2024-02-23T00:39:16.784606+0000","last_became_peered":"2024-02-23T00:39:16.784606+0000","last_unstale":"2024-02-23T00:46:45.154407+0000","last_undegraded":"2024-02-23T00:46:45.154407+0000","last_fullsized":"2024-02-23T00:46:45.154407+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:43:49.139156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:49.139156+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":299,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.165669+0000","last_change":"2024-02-23T00:44:32.170717+0000","last_active":"2024-02-23T00:46:46.165669+0000","last_peered":"2024-02-23T00:46:46.165669+0000","last_clean":"2024-02-23T00:46:46.165669+0000","last_became_active":"2024-02-23T00:39:14.785362+0000","last_became_peered":"2024-02-23T00:39:14.785362+0000","last_unstale":"2024-02-23T00:46:46.165669+0000","last_undegraded":"2024-02-23T00:46:46.165669+0000","last_fullsized":"2024-02-23T00:46:46.165669+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:32.170554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:32.170554+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":288,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.160322+0000","last_change":"2024-02-23T00:42:44.553722+0000","last_active":"2024-02-23T00:46:45.160322+0000","last_peered":"2024-02-23T00:46:45.160322+0000","last_clean":"2024-02-23T00:46:45.160322+0000","last_became_active":"2024-02-23T00:39:18.796312+0000","last_became_peered":"2024-02-23T00:39:18.796312+0000","last_unstale":"2024-02-23T00:46:45.160322+0000","last_undegraded":"2024-02-23T00:46:45.160322+0000","last_fullsized":"2024-02-23T00:46:45.160322+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:42:44.553621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:42:44.553621+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":240,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.315832+0000","last_change":"2024-02-23T00:45:09.641006+0000","last_active":"2024-02-23T00:46:46.315832+0000","last_peered":"2024-02-23T00:46:46.315832+0000","last_clean":"2024-02-23T00:46:46.315832+0000","last_became_active":"2024-02-23T00:42:58.838257+0000","last_became_peered":"2024-02-23T00:42:58.838257+0000","last_unstale":"2024-02-23T00:46:46.315832+0000","last_undegraded":"2024-02-23T00:46:46.315832+0000","last_fullsized":"2024-02-23T00:46:46.315832+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:09.640936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:09.640936+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"88'50","reported_seq":393,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.164731+0000","last_change":"2024-02-23T00:45:06.038488+0000","last_active":"2024-02-23T00:46:46.164731+0000","last_peered":"2024-02-23T00:46:46.164731+0000","last_clean":"2024-02-23T00:46:46.164731+0000","last_became_active":"2024-02-23T00:39:16.780144+0000","last_became_peered":"2024-02-23T00:39:16.780144+0000","last_unstale":"2024-02-23T00:46:46.164731+0000","last_undegraded":"2024-02-23T00:46:46.164731+0000","last_fullsized":"2024-02-23T00:46:46.164731+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'50","last_scrub_stamp":"2024-02-23T00:45:06.038374+0000","last_deep_scrub":"88'50","last_deep_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.038374+0000","log_size":50,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":299,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.315869+0000","last_change":"2024-02-23T00:45:20.573233+0000","last_active":"2024-02-23T00:46:46.315869+0000","last_peered":"2024-02-23T00:46:46.315869+0000","last_clean":"2024-02-23T00:46:46.315869+0000","last_became_active":"2024-02-23T00:39:14.782000+0000","last_became_peered":"2024-02-23T00:39:14.782000+0000","last_unstale":"2024-02-23T00:46:46.315869+0000","last_undegraded":"2024-02-23T00:46:46.315869+0000","last_fullsized":"2024-02-23T00:46:46.315869+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:20.573168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_clean_scrub_stamp":"2024-02-23T00:45:20.573168+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":288,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-02-23T00:46:45.154578+0000","last_change":"2024-02-23T00:43:48.126631+0000","last_active":"2024-02-23T00:46:45.154578+0000","last_peered":"2024-02-23T00:46:45.154578+0000","last_clean":"2024-02-23T00:46:45.154578+0000","last_became_active":"2024-02-23T00:39:18.799723+0000","last_became_peered":"2024-02-23T00:39:18.799723+0000","last_unstale":"2024-02-23T00:46:45.154578+0000","last_undegraded":"2024-02-23T00:46:45.154578+0000","last_fullsized":"2024-02-23T00:46:45.154578+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:48.126552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.126552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":239,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-02-23T00:46:46.203185+0000","last_change":"2024-02-23T00:45:14.209403+0000","last_active":"2024-02-23T00:46:46.203185+0000","last_peered":"2024-02-23T00:46:46.203185+0000","last_clean":"2024-02-23T00:46:46.203185+0000","last_became_active":"2024-02-23T00:42:58.824573+0000","last_became_peered":"2024-02-23T00:42:58.824573+0000","last_unstale":"2024-02-23T00:46:46.203185+0000","last_undegraded":"2024-02-23T00:46:46.203185+0000","last_fullsized":"2024-02-23T00:46:46.203185+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.209308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.209308+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0],"acting":[5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":62,"num_pg":32,"stat_sum":{"num_bytes":4001220,"num_objects":123,"num_object_clones":2,"num_object_copies":369,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":0,"num_read":98,"num_read_kb":4,"num_write":548,"num_write_kb":4449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":13492224,"data_stored":12003660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1794,"ondisk_log_size":1794,"up":96,"acting":96,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":191,"num_pg":32,"stat_sum":{"num_bytes":85,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":84,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":22,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":66,"acting":66,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1080,"num_read_kb":805,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":651,"ondisk_log_size":651,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":444,"num_read_kb":439,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":8},{"poolid":15,"num_pg":17,"stat_sum":{"num_bytes":2048,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":196608,"data_stored":6144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23,"ondisk_log_size":23,"up":51,"acting":51,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234078,"num_pgs":96,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":310816,"kb_used_data":9488,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93434336,"statfs":{"total":95995035648,"available":95676760064,"internally_reserved":0,"allocated":9715712,"data_stored":7440045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:46:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54900000000000004,"15min":0.54600000000000004},"min":{"1min":0.315,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.76400000000000001,"5min":4.9420000000000002,"15min":4.9420000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54600000000000004,"15min":0.51800000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.77800000000000002,"5min":4.9729999999999999,"15min":4.9729999999999999},"last":0.51300000000000001}]},{"osd":1,"last update":"Fri Feb 23 00:46:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.78600000000000003,"15min":0.627},"min":{"1min":0.40600000000000003,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70499999999999996,"5min":10.786,"15min":10.786},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.79300000000000004,"15min":0.628},"min":{"1min":0.317,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.73399999999999999,"5min":11.885999999999999,"15min":11.885999999999999},"last":0.54700000000000004}]},{"osd":2,"last update":"Fri Feb 23 00:46:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.60399999999999998,"15min":0.55900000000000005},"min":{"1min":0.40000000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71199999999999997,"5min":5.0789999999999997,"15min":5.0789999999999997},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.58299999999999996,"15min":0.54800000000000004},"min":{"1min":0.38300000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.69399999999999995,"5min":5.1280000000000001,"15min":5.1280000000000001},"last":0.58999999999999997}]},{"osd":3,"last update":"Fri Feb 23 00:46:26 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.81000000000000005,"15min":0.61899999999999999},"min":{"1min":0.34499999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67600000000000005,"5min":15.712999999999999,"15min":15.712999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.73799999999999999,"15min":0.621},"min":{"1min":0.376,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.1659999999999999,"5min":9.4090000000000007,"15min":9.4090000000000007},"last":0.505}]},{"osd":4,"last update":"Fri Feb 23 00:46:26 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.45800000000000002,"15min":0.437},"min":{"1min":0.28599999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.65300000000000002,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":0.317},{"interface":"front","average":{"1min":0.499,"5min":0.56999999999999995,"15min":0.48899999999999999},"min":{"1min":0.38600000000000001,"5min":0.28599999999999998,"15min":0.28199999999999997},"max":{"1min":0.66700000000000004,"5min":11.236000000000001,"15min":11.236000000000001},"last":0.32500000000000001}]},{"osd":5,"last update":"Fri Feb 23 00:46:26 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.53600000000000003,"15min":0.49299999999999999},"min":{"1min":0.38100000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.68400000000000005,"5min":5.4000000000000004,"15min":5.4000000000000004},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52300000000000002,"15min":0.48999999999999999},"min":{"1min":0.376,"5min":0.29299999999999998,"15min":0.26700000000000002},"max":{"1min":0.67800000000000005,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.374}]},{"osd":6,"last update":"Fri Feb 23 00:46:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52400000000000002,"15min":0.50900000000000001},"min":{"1min":0.42499999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.66000000000000003,"5min":1.034,"15min":1.034},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.64200000000000002,"15min":0.54800000000000004},"min":{"1min":0.38400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.2370000000000001,"5min":13.395,"15min":13.395},"last":0.35599999999999998}]}]},{"osd":6,"up_from":47,"seq":201863463013,"num_pgs":89,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":315076,"kb_used_data":5560,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93430076,"statfs":{"total":95995035648,"available":95672397824,"internally_reserved":0,"allocated":5693440,"data_stored":3440825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.97299999999999998,"15min":0.70499999999999996},"min":{"1min":0.33500000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.65000000000000002,"5min":30.641999999999999,"15min":30.641999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.97499999999999998,"15min":0.71399999999999997},"min":{"1min":0.42999999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.71199999999999997,"5min":29.292000000000002,"15min":29.292000000000002},"last":0.44900000000000001}]},{"osd":1,"last update":"Fri Feb 23 00:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.98499999999999999,"15min":0.69599999999999995},"min":{"1min":0.40600000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.76700000000000002,"5min":28.186,"15min":28.186},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":1.48,"15min":0.88200000000000001},"min":{"1min":0.39700000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.67800000000000005,"5min":33.104999999999997,"15min":33.104999999999997},"last":0.48899999999999999}]},{"osd":2,"last update":"Fri Feb 23 00:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":1.034,"15min":0.73399999999999999},"min":{"1min":0.38500000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.75600000000000001,"5min":19.960000000000001,"15min":19.960000000000001},"last":0.377},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.84299999999999997,"15min":0.65500000000000003},"min":{"1min":0.38600000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.79300000000000004,"5min":8.5860000000000003,"15min":8.5860000000000003},"last":0.39100000000000001}]},{"osd":3,"last update":"Fri Feb 23 00:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.75600000000000001,"15min":0.64000000000000001},"min":{"1min":0.439,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80000000000000004,"5min":19.562000000000001,"15min":19.562000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.84699999999999998,"15min":0.67900000000000005},"min":{"1min":0.35399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69999999999999996,"5min":14.199,"15min":14.199},"last":0.41599999999999998}]},{"osd":4,"last update":"Fri Feb 23 00:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.58299999999999996,"15min":0.46500000000000002},"min":{"1min":0.27700000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.70299999999999996,"5min":11.109999999999999,"15min":11.109999999999999},"last":0.316},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.752,"15min":0.53600000000000003},"min":{"1min":0.314,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.71099999999999997,"5min":16.734000000000002,"15min":16.734000000000002},"last":0.46300000000000002}]},{"osd":5,"last update":"Fri Feb 23 00:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.63700000000000001,"15min":0.48799999999999999},"min":{"1min":0.32400000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.745,"5min":14.154999999999999,"15min":14.154999999999999},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.63500000000000001,"15min":0.504},"min":{"1min":0.27900000000000003,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.67600000000000005,"5min":6.6100000000000003,"15min":6.6100000000000003},"last":0.438}]},{"osd":7,"last update":"Fri Feb 23 00:46:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.59299999999999997,"15min":0.52400000000000002},"min":{"1min":0.371,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.78100000000000003,"5min":8.548,"15min":8.548},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.76700000000000002,"15min":0.59899999999999998},"min":{"1min":0.47199999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.76300000000000001,"5min":15.618,"15min":15.618},"last":0.498}]}]},{"osd":1,"up_from":13,"seq":55834574975,"num_pgs":94,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":318940,"kb_used_data":9416,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93426212,"statfs":{"total":95995035648,"available":95668441088,"internally_reserved":0,"allocated":9641984,"data_stored":7441515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:46:45 2024","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":1.2789999999999999,"15min":0.66300000000000003},"min":{"1min":0.23400000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.41799999999999998,"5min":78.361000000000004,"15min":78.361000000000004},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.56299999999999994,"15min":0.437},"min":{"1min":0.27800000000000002,"5min":0.218,"15min":0.191},"max":{"1min":0.51700000000000002,"5min":5.8849999999999998,"15min":5.8849999999999998},"last":0.51700000000000002}]},{"osd":2,"last update":"Fri Feb 23 00:46:04 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":1.181,"15min":0.65600000000000003},"min":{"1min":0.25,"5min":0.219,"15min":0.188},"max":{"1min":0.63800000000000001,"5min":48.606000000000002,"15min":48.606000000000002},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.879,"15min":0.58399999999999996},"min":{"1min":0.26700000000000002,"5min":0.25800000000000001,"15min":0.249},"max":{"1min":1.3460000000000001,"5min":23.911000000000001,"15min":23.911000000000001},"last":0.55700000000000005}]},{"osd":3,"last update":"Fri Feb 23 00:46:45 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.88800000000000001,"15min":0.61699999999999999},"min":{"1min":0.26700000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.61499999999999999,"5min":35.244999999999997,"15min":35.244999999999997},"last":0.443},{"interface":"front","average":{"1min":0.39900000000000002,"5min":1.1299999999999999,"15min":0.67800000000000005},"min":{"1min":0.27400000000000002,"5min":0.253,"15min":0.248},"max":{"1min":0.63900000000000001,"5min":56.853999999999999,"15min":56.853999999999999},"last":0.5}]},{"osd":4,"last update":"Fri Feb 23 00:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":1.2310000000000001,"15min":0.81699999999999995},"min":{"1min":0.34599999999999997,"5min":0.33700000000000002,"15min":0.29299999999999998},"max":{"1min":0.82399999999999995,"5min":37.646999999999998,"15min":37.646999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.86599999999999999,"15min":0.69699999999999995},"min":{"1min":0.375,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80700000000000005,"5min":9.6859999999999999,"15min":9.6859999999999999},"last":0.61299999999999999}]},{"osd":5,"last update":"Fri Feb 23 00:46:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":1.2150000000000001,"15min":0.83899999999999997},"min":{"1min":0.39800000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79100000000000004,"5min":48.792000000000002,"15min":48.792000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.81399999999999995,"15min":0.745},"min":{"1min":0.45000000000000001,"5min":0.38100000000000001,"15min":0.34200000000000003},"max":{"1min":0.78800000000000003,"5min":11.872999999999999,"15min":11.872999999999999},"last":0.60199999999999998}]},{"osd":6,"last update":"Fri Feb 23 00:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":1.1890000000000001,"15min":0.82299999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.79400000000000004,"5min":40.741,"15min":40.741},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.77200000000000002,"15min":0.68999999999999995},"min":{"1min":0.41999999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.872,"5min":6.0069999999999997,"15min":6.0069999999999997},"last":0.64400000000000002}]},{"osd":7,"last update":"Fri Feb 23 00:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.89200000000000002,"15min":0.71599999999999997},"min":{"1min":0.39300000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80800000000000005,"5min":20.225000000000001,"15min":20.225000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.93000000000000005,"15min":0.72399999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.73199999999999998,"5min":20.254999999999999,"15min":20.254999999999999},"last":0.65200000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738501,"num_pgs":86,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":319144,"kb_used_data":5528,"kb_used_omap":0,"kb_used_meta":313600,"kb_avail":93426008,"statfs":{"total":95995035648,"available":95668232192,"internally_reserved":0,"allocated":5660672,"data_stored":3440797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321126400},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 23 00:46:46 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":1.0880000000000001,"15min":0.59199999999999997},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.19500000000000001},"max":{"1min":0.47899999999999998,"5min":28.856999999999999,"15min":28.856999999999999},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.33300000000000002,"5min":1.091,"15min":0.60699999999999998},"min":{"1min":0.25,"5min":0.218,"15min":0.20799999999999999},"max":{"1min":0.55700000000000005,"5min":20.577000000000002,"15min":20.577000000000002},"last":0.27400000000000002}]},{"osd":2,"last update":"Fri Feb 23 00:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":1.014,"15min":0.60299999999999998},"min":{"1min":0.27900000000000003,"5min":0.26000000000000001,"15min":0.20499999999999999},"max":{"1min":0.55200000000000005,"5min":13.863,"15min":13.863},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.318,"5min":0.86899999999999999,"15min":0.54500000000000004},"min":{"1min":0.219,"5min":0.219,"15min":0.19800000000000001},"max":{"1min":0.373,"5min":10.455,"15min":10.455},"last":0.245}]},{"osd":3,"last update":"Fri Feb 23 00:46:38 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":1.5069999999999999,"15min":0.79600000000000004},"min":{"1min":0.255,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.504,"5min":35.566000000000003,"15min":35.566000000000003},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.39100000000000001,"5min":1.361,"15min":0.78400000000000003},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.52600000000000002,"5min":20.702999999999999,"15min":20.702999999999999},"last":0.34799999999999998}]},{"osd":4,"last update":"Fri Feb 23 00:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":1.0409999999999999,"15min":0.80300000000000005},"min":{"1min":0.36399999999999999,"5min":0.36199999999999999,"15min":0.34699999999999998},"max":{"1min":0.753,"5min":10.584,"15min":10.584},"last":0.32500000000000001},{"interface":"front","average":{"1min":1.258,"5min":1.032,"15min":0.82999999999999996},"min":{"1min":0.38900000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":13.198,"5min":13.198,"15min":13.198},"last":0.38100000000000001}]},{"osd":5,"last update":"Fri Feb 23 00:46:28 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":1.004,"15min":0.73799999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72499999999999998,"5min":10.397,"15min":10.397},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":1.077,"15min":0.80300000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84999999999999998,"5min":17.297000000000001,"15min":17.297000000000001},"last":0.52900000000000003}]},{"osd":6,"last update":"Fri Feb 23 00:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":1.1080000000000001,"15min":0.79900000000000004},"min":{"1min":0.44400000000000001,"5min":0.42299999999999999,"15min":0.38400000000000001},"max":{"1min":0.85799999999999998,"5min":17.373999999999999,"15min":17.373999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":1.321,"5min":1.0409999999999999,"15min":0.77000000000000002},"min":{"1min":0.42199999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":13.973000000000001,"5min":13.973000000000001,"15min":13.973000000000001},"last":0.497}]},{"osd":7,"last update":"Fri Feb 23 00:46:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":1.083,"15min":0.80700000000000005},"min":{"1min":0.45200000000000001,"5min":0.41799999999999998,"15min":0.39400000000000002},"max":{"1min":0.81399999999999995,"5min":17.399999999999999,"15min":17.399999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.629,"5min":1.0669999999999999,"15min":0.78800000000000003},"min":{"1min":0.49299999999999999,"5min":0.433,"15min":0.43099999999999999},"max":{"1min":0.81200000000000006,"5min":16.427,"15min":16.427},"last":0.54000000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411450,"num_pgs":81,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":315144,"kb_used_data":5628,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93430008,"statfs":{"total":95995035648,"available":95672328192,"internally_reserved":0,"allocated":5763072,"data_stored":3440559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:46:04 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.69399999999999995,"15min":0.48399999999999999},"min":{"1min":0.22900000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.61199999999999999,"5min":13.657,"15min":13.657},"last":0.376},{"interface":"front","average":{"1min":0.374,"5min":1.262,"15min":0.68000000000000005},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.624,"5min":42.343000000000004,"15min":42.343000000000004},"last":0.438}]},{"osd":1,"last update":"Fri Feb 23 00:46:04 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.96199999999999997,"15min":0.57399999999999995},"min":{"1min":0.254,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.53500000000000003,"5min":18.952000000000002,"15min":18.952000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.75800000000000001,"15min":0.52200000000000002},"min":{"1min":0.26300000000000001,"5min":0.23999999999999999,"15min":0.23599999999999999},"max":{"1min":0.56499999999999995,"5min":15.391,"15min":15.391},"last":0.41299999999999998}]},{"osd":3,"last update":"Fri Feb 23 00:46:39 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":1.175,"15min":0.70699999999999996},"min":{"1min":0.26600000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.89100000000000001,"5min":25.635999999999999,"15min":25.635999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":1.006,"15min":0.64300000000000002},"min":{"1min":0.29099999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.80700000000000005,"5min":19.960000000000001,"15min":19.960000000000001},"last":0.48799999999999999}]},{"osd":4,"last update":"Fri Feb 23 00:46:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.99199999999999999,"15min":0.72799999999999998},"min":{"1min":0.41299999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.74099999999999999,"5min":13.673,"15min":13.673},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.57599999999999996,"5min":1.0389999999999999,"15min":0.73499999999999999},"min":{"1min":0.379,"5min":0.36899999999999999,"15min":0.33900000000000002},"max":{"1min":0.90000000000000002,"5min":10.103999999999999,"15min":10.103999999999999},"last":0.56100000000000005}]},{"osd":5,"last update":"Fri Feb 23 00:46:27 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.90100000000000002,"15min":0.68799999999999994},"min":{"1min":0.40200000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.89000000000000001,"5min":15.612,"15min":15.612},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.86199999999999999,"15min":0.69799999999999995},"min":{"1min":0.40100000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.752,"5min":13.632999999999999,"15min":13.632999999999999},"last":0.57799999999999996}]},{"osd":6,"last update":"Fri Feb 23 00:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.88500000000000001,"15min":0.71099999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.5249999999999999,"5min":11.544,"15min":11.544},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.84599999999999997,"5min":1.05,"15min":0.749},"min":{"1min":0.45400000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":4.1909999999999998,"5min":10.132999999999999,"15min":10.132999999999999},"last":0.67900000000000005}]},{"osd":7,"last update":"Fri Feb 23 00:46:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.86599999999999999,"15min":0.73599999999999999},"min":{"1min":0.40999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.81999999999999995,"5min":11.599,"15min":11.599},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.625,"5min":1.1020000000000001,"15min":0.79800000000000004},"min":{"1min":0.46100000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.92200000000000004,"5min":10.148999999999999,"15min":10.148999999999999},"last":0.65600000000000003}]}]},{"osd":3,"up_from":26,"seq":111669149811,"num_pgs":103,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":302828,"kb_used_data":5604,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93442324,"statfs":{"total":95995035648,"available":95684939776,"internally_reserved":0,"allocated":5738496,"data_stored":3442285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:46:37 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.75,"15min":0.53700000000000003},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":1.274,"5min":12.212999999999999,"15min":12.212999999999999},"last":0.311},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.95799999999999996,"15min":0.61699999999999999},"min":{"1min":0.23899999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":5.9800000000000004,"5min":11.714,"15min":11.714},"last":0.28299999999999997}]},{"osd":1,"last update":"Fri Feb 23 00:46:37 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.012,"15min":0.64900000000000002},"min":{"1min":0.26200000000000001,"5min":0.23599999999999999,"15min":0.22900000000000001},"max":{"1min":6.851,"5min":11.742000000000001,"15min":11.742000000000001},"last":0.22900000000000001},{"interface":"front","average":{"1min":1.1479999999999999,"5min":0.93500000000000005,"15min":0.60699999999999998},"min":{"1min":0.30299999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":8.2270000000000003,"5min":8.2270000000000003,"15min":8.2270000000000003},"last":0.26700000000000002}]},{"osd":2,"last update":"Fri Feb 23 00:46:37 2024","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.097,"15min":0.69099999999999995},"min":{"1min":0.307,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":8.0269999999999992,"5min":13.359999999999999,"15min":13.359999999999999},"last":0.318},{"interface":"front","average":{"1min":0.58699999999999997,"5min":1.0329999999999999,"15min":0.66100000000000003},"min":{"1min":0.28499999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":2.5459999999999998,"5min":21.122,"15min":21.122},"last":0.45500000000000002}]},{"osd":4,"last update":"Fri Feb 23 00:45:58 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.083,"15min":0.75900000000000001},"min":{"1min":0.36399999999999999,"5min":0.317,"15min":0.317},"max":{"1min":11.558,"5min":11.644,"15min":11.644},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.93600000000000005,"15min":0.70399999999999996},"min":{"1min":0.40400000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":6.1589999999999998,"5min":6.2549999999999999,"15min":6.2549999999999999},"last":0.39300000000000002}]},{"osd":5,"last update":"Fri Feb 23 00:46:26 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.79800000000000004,"15min":0.66600000000000004},"min":{"1min":0.441,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":2.125,"5min":5.6360000000000001,"15min":5.6360000000000001},"last":0.502},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.1639999999999999,"15min":0.80200000000000005},"min":{"1min":0.45000000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":11.648,"5min":11.648,"15min":11.648},"last":0.46999999999999997}]},{"osd":6,"last update":"Fri Feb 23 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.90500000000000003,"15min":0.70599999999999996},"min":{"1min":0.39000000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.0539999999999998,"5min":6.343,"15min":6.343},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.80000000000000004,"15min":0.68200000000000005},"min":{"1min":0.442,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.71699999999999997,"5min":5.6680000000000001,"15min":5.6680000000000001},"last":0.57899999999999996}]},{"osd":7,"last update":"Fri Feb 23 00:46:22 2024","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.206,"15min":0.82599999999999996},"min":{"1min":0.48399999999999999,"5min":0.41099999999999998,"15min":0.39000000000000001},"max":{"1min":11.676,"5min":11.676,"15min":11.676},"last":0.51000000000000001},{"interface":"front","average":{"1min":1.411,"5min":1.1160000000000001,"15min":0.80800000000000005},"min":{"1min":0.51600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":7.9800000000000004,"5min":7.9800000000000004,"15min":7.9800000000000004},"last":0.434}]}]},{"osd":4,"up_from":32,"seq":137438953583,"num_pgs":96,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":323088,"kb_used_data":9476,"kb_used_omap":0,"kb_used_meta":313600,"kb_avail":93422064,"statfs":{"total":95995035648,"available":95664193536,"internally_reserved":0,"allocated":9703424,"data_stored":7440589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321126400},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:46:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51900000000000002,"15min":0.55600000000000005},"min":{"1min":0.30199999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.90900000000000003,"5min":3.5219999999999998,"15min":3.5219999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.66400000000000003,"15min":0.60799999999999998},"min":{"1min":0.38400000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.71499999999999997,"5min":8.6140000000000008,"15min":8.6140000000000008},"last":0.44}]},{"osd":1,"last update":"Fri Feb 23 00:46:02 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.66500000000000004,"15min":0.60599999999999998},"min":{"1min":0.33600000000000002,"5min":0.28299999999999997,"15min":0.27800000000000002},"max":{"1min":0.66300000000000003,"5min":9.6110000000000007,"15min":9.6110000000000007},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.65400000000000003,"15min":0.59999999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.72699999999999998,"5min":6.6950000000000003,"15min":6.6950000000000003},"last":0.40600000000000003}]},{"osd":2,"last update":"Fri Feb 23 00:46:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.622,"15min":0.59099999999999997},"min":{"1min":0.34000000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.83099999999999996,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.59799999999999998,"15min":0.58899999999999997},"min":{"1min":0.34799999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74399999999999999,"5min":5.8440000000000003,"15min":5.8440000000000003},"last":0.51100000000000001}]},{"osd":3,"last update":"Fri Feb 23 00:46:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.65500000000000003,"15min":0.60399999999999998},"min":{"1min":0.35899999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80700000000000005,"5min":6.5540000000000003,"15min":6.5540000000000003},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.65900000000000003,"15min":0.60599999999999998},"min":{"1min":0.33100000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78000000000000003,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":0.52000000000000002}]},{"osd":5,"last update":"Fri Feb 23 00:46:25 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.41499999999999998,"15min":0.39000000000000001},"min":{"1min":0.28000000000000003,"5min":0.252,"15min":0.246},"max":{"1min":0.67200000000000004,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.48099999999999998,"15min":0.442},"min":{"1min":0.29999999999999999,"5min":0.29599999999999999,"15min":0.28899999999999998},"max":{"1min":0.71999999999999997,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.502}]},{"osd":6,"last update":"Fri Feb 23 00:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.45300000000000001,"15min":0.41699999999999998},"min":{"1min":0.28699999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.68500000000000005,"5min":0.75,"15min":0.75},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45500000000000002,"15min":0.41699999999999998},"min":{"1min":0.27000000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.67100000000000004,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.495}]},{"osd":7,"last update":"Fri Feb 23 00:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.47699999999999998,"15min":0.48299999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.254},"max":{"1min":0.77800000000000002,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.52900000000000003,"15min":0.53500000000000003},"min":{"1min":0.34699999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78800000000000003,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.55400000000000005}]}]},{"osd":5,"up_from":39,"seq":167503724650,"num_pgs":80,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":315036,"kb_used_data":5520,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93430116,"statfs":{"total":95995035648,"available":95672438784,"internally_reserved":0,"allocated":5652480,"data_stored":3451480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.78200000000000003,"15min":0.59999999999999998},"min":{"1min":0.33200000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.68500000000000005,"5min":23.881,"15min":23.881},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.68799999999999994,"15min":0.57699999999999996},"min":{"1min":0.40100000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73899999999999999,"5min":5.4649999999999999,"15min":5.4649999999999999},"last":0.44800000000000001}]},{"osd":1,"last update":"Fri Feb 23 00:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.89400000000000002,"15min":0.64900000000000002},"min":{"1min":0.318,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.89600000000000002,"5min":18.175999999999998,"15min":18.175999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.78300000000000003,"15min":0.62},"min":{"1min":0.32300000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.70699999999999996,"5min":9.0950000000000006,"15min":9.0950000000000006},"last":0.54000000000000004}]},{"osd":2,"last update":"Fri Feb 23 00:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":1.018,"15min":0.70299999999999996},"min":{"1min":0.34100000000000003,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70599999999999996,"5min":22.282,"15min":22.282},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":1.0509999999999999,"15min":0.70899999999999996},"min":{"1min":0.378,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77200000000000002,"5min":23.925000000000001,"15min":23.925000000000001},"last":0.59899999999999998}]},{"osd":3,"last update":"Fri Feb 23 00:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.78100000000000003,"15min":0.61299999999999999},"min":{"1min":0.38500000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.77900000000000003,"5min":6.5730000000000004,"15min":6.5730000000000004},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.80500000000000005,"15min":0.64100000000000001},"min":{"1min":0.40699999999999997,"5min":0.33600000000000002,"15min":0.33500000000000002},"max":{"1min":0.753,"5min":8.1760000000000002,"15min":8.1760000000000002},"last":0.503}]},{"osd":4,"last update":"Fri Feb 23 00:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.42999999999999999,"15min":0.40100000000000002},"min":{"1min":0.29499999999999998,"5min":0.247,"15min":0.247},"max":{"1min":0.64600000000000002,"5min":0.68999999999999995,"15min":0.70399999999999996},"last":0.375},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.51600000000000001,"15min":0.436},"min":{"1min":0.27700000000000002,"5min":0.27500000000000002,"15min":0.27300000000000002},"max":{"1min":0.67000000000000004,"5min":3.879,"15min":3.879},"last":0.55000000000000004}]},{"osd":6,"last update":"Fri Feb 23 00:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.48899999999999999,"15min":0.44600000000000001},"min":{"1min":0.30599999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70999999999999996,"5min":1.736,"15min":1.736},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.49199999999999999,"15min":0.45900000000000002},"min":{"1min":0.311,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73099999999999998,"5min":0.871,"15min":0.871},"last":0.56200000000000006}]},{"osd":7,"last update":"Fri Feb 23 00:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.57699999999999996,"15min":0.53400000000000003},"min":{"1min":0.34799999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.753,"5min":7.0300000000000002,"15min":7.0300000000000002},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.58299999999999996,"15min":0.53300000000000003},"min":{"1min":0.33700000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.68999999999999995,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":0.51100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4132864,"data_stored":4000334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":274432,"data_stored":647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":172032,"data_stored":417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4198400,"data_stored":4000501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":192512,"data_stored":484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4235264,"data_stored":4000566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":191,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":85,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":191,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":191,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":191,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":85,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":191,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":191,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":191,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":85,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":191,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-23T00:46:49.739 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-02-23T00:46:49.739 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:49.739 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-23T00:46:49.739 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-02-23T00:46:49.739 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-02-23T00:46:49.739 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-02-23T00:46:49.739 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.3 config set filestore_inject_stall 3 2024-02-23T00:46:49.740 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:46:49.740 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:46:49.842 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:46:49.873 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:49.893 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:46:49.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:48.111401+0000 mgr.y (mgr.14517) 326 : audit [DBG] from='client.41141 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:46:49.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:48.316192+0000 osd.4 (osd.4) 139 : cluster [DBG] 4.e deep-scrub starts 2024-02-23T00:46:49.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:48.320080+0000 osd.4 (osd.4) 140 : cluster [DBG] 4.e deep-scrub ok 2024-02-23T00:46:49.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:48.689768+0000 osd.7 (osd.7) 175 : cluster [DBG] 6.8 deep-scrub starts 2024-02-23T00:46:49.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:48.692675+0000 osd.7 (osd.7) 176 : cluster [DBG] 6.8 deep-scrub ok 2024-02-23T00:46:49.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:49.156938+0000 osd.5 (osd.5) 148 : cluster [DBG] 5.15 deep-scrub starts 2024-02-23T00:46:49.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:49.160986+0000 osd.5 (osd.5) 149 : cluster [DBG] 5.15 deep-scrub ok 2024-02-23T00:46:49.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:49.172215+0000 mon.a (mon.0) 2202 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi047-82386-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:49.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:49.172307+0000 mon.a (mon.0) 2203 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pgp_num","val":"11"}]': finished 2024-02-23T00:46:49.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:49.172367+0000 mon.a (mon.0) 2204 : cluster [DBG] osdmap e337: 8 total, 8 up, 8 in 2024-02-23T00:46:49.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:49.173720+0000 mon.b (mon.1) 373 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:49.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:49.175649+0000 mon.c (mon.2) 254 : audit [INF] from='client.? 172.21.15.47:0/1633196565' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi047-82386-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi047-82386-57"}]: dispatch 2024-02-23T00:46:49.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:49.176082+0000 mon.a (mon.0) 2205 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi047-82386-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi047-82386-57"}]: dispatch 2024-02-23T00:46:49.896 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:49.897 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:46:49.998 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:46:50.023 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:48.111401+0000 mgr.y (mgr.14517) 326 : audit [DBG] from='client.41141 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:46:50.024 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:48.316192+0000 osd.4 (osd.4) 139 : cluster [DBG] 4.e deep-scrub starts 2024-02-23T00:46:50.024 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:48.320080+0000 osd.4 (osd.4) 140 : cluster [DBG] 4.e deep-scrub ok 2024-02-23T00:46:50.024 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:48.689768+0000 osd.7 (osd.7) 175 : cluster [DBG] 6.8 deep-scrub starts 2024-02-23T00:46:50.024 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:48.692675+0000 osd.7 (osd.7) 176 : cluster [DBG] 6.8 deep-scrub ok 2024-02-23T00:46:50.024 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:49.156938+0000 osd.5 (osd.5) 148 : cluster [DBG] 5.15 deep-scrub starts 2024-02-23T00:46:50.024 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:49.160986+0000 osd.5 (osd.5) 149 : cluster [DBG] 5.15 deep-scrub ok 2024-02-23T00:46:50.024 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:49.172215+0000 mon.a (mon.0) 2202 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi047-82386-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:50.024 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:49.172307+0000 mon.a (mon.0) 2203 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pgp_num","val":"11"}]': finished 2024-02-23T00:46:50.024 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:49.172367+0000 mon.a (mon.0) 2204 : cluster [DBG] osdmap e337: 8 total, 8 up, 8 in 2024-02-23T00:46:50.024 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:49.173720+0000 mon.b (mon.1) 373 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:50.024 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:49.175649+0000 mon.c (mon.2) 254 : audit [INF] from='client.? 172.21.15.47:0/1633196565' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi047-82386-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi047-82386-57"}]: dispatch 2024-02-23T00:46:50.025 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:49.176082+0000 mon.a (mon.0) 2205 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi047-82386-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi047-82386-57"}]: dispatch 2024-02-23T00:46:50.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:48.111401+0000 mgr.y (mgr.14517) 326 : audit [DBG] from='client.41141 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:46:50.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:48.316192+0000 osd.4 (osd.4) 139 : cluster [DBG] 4.e deep-scrub starts 2024-02-23T00:46:50.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:48.320080+0000 osd.4 (osd.4) 140 : cluster [DBG] 4.e deep-scrub ok 2024-02-23T00:46:50.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:48.689768+0000 osd.7 (osd.7) 175 : cluster [DBG] 6.8 deep-scrub starts 2024-02-23T00:46:50.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:48.692675+0000 osd.7 (osd.7) 176 : cluster [DBG] 6.8 deep-scrub ok 2024-02-23T00:46:50.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:49.156938+0000 osd.5 (osd.5) 148 : cluster [DBG] 5.15 deep-scrub starts 2024-02-23T00:46:50.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:49.160986+0000 osd.5 (osd.5) 149 : cluster [DBG] 5.15 deep-scrub ok 2024-02-23T00:46:50.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:49.172215+0000 mon.a (mon.0) 2202 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi047-82386-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:50.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:49.172307+0000 mon.a (mon.0) 2203 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi047-82419-1","var":"pgp_num","val":"11"}]': finished 2024-02-23T00:46:50.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:49.172367+0000 mon.a (mon.0) 2204 : cluster [DBG] osdmap e337: 8 total, 8 up, 8 in 2024-02-23T00:46:50.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:49.173720+0000 mon.b (mon.1) 373 : audit [DBG] from='client.? 172.21.15.47:0/2979574659' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-23T00:46:50.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:49.175649+0000 mon.c (mon.2) 254 : audit [INF] from='client.? 172.21.15.47:0/1633196565' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi047-82386-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi047-82386-57"}]: dispatch 2024-02-23T00:46:50.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:49.176082+0000 mon.a (mon.0) 2205 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi047-82386-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi047-82386-57"}]: dispatch 2024-02-23T00:46:50.026 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:50.046 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:46:50.048 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:50.049 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:46:50.150 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:46:50.178 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:50.205 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:46:50.427 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:46:50.429 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:50.430 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:46:50.444 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.5 dump_ops_in_flight 2024-02-23T00:46:50.531 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:46:50.557 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:50.577 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:46:50.579 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:50.580 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:46:50.680 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:46:50.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:49.243012+0000 osd.1 (osd.1) 153 : cluster [DBG] 18.5 deep-scrub starts 2024-02-23T00:46:50.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:49.243950+0000 osd.1 (osd.1) 154 : cluster [DBG] 18.5 deep-scrub ok 2024-02-23T00:46:50.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:49.264042+0000 mgr.y (mgr.14517) 327 : cluster [DBG] pgmap v481: 235 pgs: 235 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-23T00:46:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:49.270916+0000 osd.4 (osd.4) 141 : cluster [DBG] 5.18 scrub starts 2024-02-23T00:46:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:49.271701+0000 osd.4 (osd.4) 142 : cluster [DBG] 5.18 scrub ok 2024-02-23T00:46:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:49.392222+0000 osd.6 (osd.6) 137 : cluster [DBG] 2.14 scrub starts 2024-02-23T00:46:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:49.392988+0000 osd.6 (osd.6) 138 : cluster [DBG] 2.14 scrub ok 2024-02-23T00:46:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:49.663738+0000 osd.2 (osd.2) 149 : cluster [DBG] 5.d scrub starts 2024-02-23T00:46:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:49.665266+0000 osd.2 (osd.2) 150 : cluster [DBG] 5.d scrub ok 2024-02-23T00:46:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:49.702098+0000 osd.7 (osd.7) 177 : cluster [DBG] 5.4 scrub starts 2024-02-23T00:46:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:49.704981+0000 osd.7 (osd.7) 178 : cluster [DBG] 5.4 scrub ok 2024-02-23T00:46:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:49.919870+0000 osd.3 (osd.3) 169 : cluster [DBG] 4.11 scrub starts 2024-02-23T00:46:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:49.921538+0000 osd.3 (osd.3) 170 : cluster [DBG] 4.11 scrub ok 2024-02-23T00:46:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:50.605013+0000 mon.a (mon.0) 2206 : cluster [DBG] osdmap e338: 8 total, 8 up, 8 in 2024-02-23T00:46:50.708 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:46:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:46:50] "GET /metrics HTTP/1.1" 200 190571 "" "Prometheus/2.33.4" 2024-02-23T00:46:50.708 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:49.243012+0000 osd.1 (osd.1) 153 : cluster [DBG] 18.5 deep-scrub starts 2024-02-23T00:46:50.709 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:49.243950+0000 osd.1 (osd.1) 154 : cluster [DBG] 18.5 deep-scrub ok 2024-02-23T00:46:50.709 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:49.264042+0000 mgr.y (mgr.14517) 327 : cluster [DBG] pgmap v481: 235 pgs: 235 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-23T00:46:50.709 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:49.270916+0000 osd.4 (osd.4) 141 : cluster [DBG] 5.18 scrub starts 2024-02-23T00:46:50.709 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:49.271701+0000 osd.4 (osd.4) 142 : cluster [DBG] 5.18 scrub ok 2024-02-23T00:46:50.709 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:49.392222+0000 osd.6 (osd.6) 137 : cluster [DBG] 2.14 scrub starts 2024-02-23T00:46:50.709 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:49.392988+0000 osd.6 (osd.6) 138 : cluster [DBG] 2.14 scrub ok 2024-02-23T00:46:50.709 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:49.663738+0000 osd.2 (osd.2) 149 : cluster [DBG] 5.d scrub starts 2024-02-23T00:46:50.709 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:49.665266+0000 osd.2 (osd.2) 150 : cluster [DBG] 5.d scrub ok 2024-02-23T00:46:50.709 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:49.702098+0000 osd.7 (osd.7) 177 : cluster [DBG] 5.4 scrub starts 2024-02-23T00:46:50.709 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:49.704981+0000 osd.7 (osd.7) 178 : cluster [DBG] 5.4 scrub ok 2024-02-23T00:46:50.709 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:49.919870+0000 osd.3 (osd.3) 169 : cluster [DBG] 4.11 scrub starts 2024-02-23T00:46:50.709 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:49.921538+0000 osd.3 (osd.3) 170 : cluster [DBG] 4.11 scrub ok 2024-02-23T00:46:50.709 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:50.605013+0000 mon.a (mon.0) 2206 : cluster [DBG] osdmap e338: 8 total, 8 up, 8 in 2024-02-23T00:46:50.710 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:49.243012+0000 osd.1 (osd.1) 153 : cluster [DBG] 18.5 deep-scrub starts 2024-02-23T00:46:50.710 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:49.243950+0000 osd.1 (osd.1) 154 : cluster [DBG] 18.5 deep-scrub ok 2024-02-23T00:46:50.710 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:49.264042+0000 mgr.y (mgr.14517) 327 : cluster [DBG] pgmap v481: 235 pgs: 235 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-23T00:46:50.710 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:49.270916+0000 osd.4 (osd.4) 141 : cluster [DBG] 5.18 scrub starts 2024-02-23T00:46:50.710 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:49.271701+0000 osd.4 (osd.4) 142 : cluster [DBG] 5.18 scrub ok 2024-02-23T00:46:50.710 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:49.392222+0000 osd.6 (osd.6) 137 : cluster [DBG] 2.14 scrub starts 2024-02-23T00:46:50.710 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:49.392988+0000 osd.6 (osd.6) 138 : cluster [DBG] 2.14 scrub ok 2024-02-23T00:46:50.710 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:49.663738+0000 osd.2 (osd.2) 149 : cluster [DBG] 5.d scrub starts 2024-02-23T00:46:50.710 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:49.665266+0000 osd.2 (osd.2) 150 : cluster [DBG] 5.d scrub ok 2024-02-23T00:46:50.710 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:49.702098+0000 osd.7 (osd.7) 177 : cluster [DBG] 5.4 scrub starts 2024-02-23T00:46:50.710 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:49.704981+0000 osd.7 (osd.7) 178 : cluster [DBG] 5.4 scrub ok 2024-02-23T00:46:50.711 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:49.919870+0000 osd.3 (osd.3) 169 : cluster [DBG] 4.11 scrub starts 2024-02-23T00:46:50.711 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:49.921538+0000 osd.3 (osd.3) 170 : cluster [DBG] 4.11 scrub ok 2024-02-23T00:46:50.711 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:50.605013+0000 mon.a (mon.0) 2206 : cluster [DBG] osdmap e338: 8 total, 8 up, 8 in 2024-02-23T00:46:50.712 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:50.760 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:46:50.960 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:46:50.962 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:50.963 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:46:51.063 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:46:51.089 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:51.110 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:46:51.112 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:51.112 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:46:51.213 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:46:51.248 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:51.286 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:46:51.290 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:51.291 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:46:51.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.306+0000 7fec005f7700 1 -- 172.21.15.47:0/1928917343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febf80fdbb0 msgr2=0x7febf80fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:51.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.306+0000 7fec005f7700 1 --2- 172.21.15.47:0/1928917343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febf80fdbb0 0x7febf80fdf90 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 rx=0x7febf4004870 tx=0x7febf4009cb0).stop 2024-02-23T00:46:51.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.306+0000 7fec005f7700 1 -- 172.21.15.47:0/1928917343 shutdown_connections 2024-02-23T00:46:51.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.306+0000 7fec005f7700 1 --2- 172.21.15.47:0/1928917343 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7febf80f8420 0x7febf8104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:46:51.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.306+0000 7fec005f7700 1 --2- 172.21.15.47:0/1928917343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febf80fdbb0 0x7febf80fdf90 unknown :-1 s=CLOSED pgs=2510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:51.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.306+0000 7fec005f7700 1 --2- 172.21.15.47:0/1928917343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febf80fe510 0x7febf80f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:51.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.306+0000 7fec005f7700 1 -- 172.21.15.47:0/1928917343 >> 172.21.15.47:0/1928917343 conn(0x7febf80f3800 msgr2=0x7febf80f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:46:51.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.306+0000 7fec005f7700 1 -- 172.21.15.47:0/1928917343 shutdown_connections 2024-02-23T00:46:51.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.307+0000 7fec005f7700 1 -- 172.21.15.47:0/1928917343 wait complete. 2024-02-23T00:46:51.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.307+0000 7fec005f7700 1 librados: starting msgr at 2024-02-23T00:46:51.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.307+0000 7fec005f7700 1 librados: starting objecter 2024-02-23T00:46:51.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.307+0000 7fec005f7700 1 Processor -- start 2024-02-23T00:46:51.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.307+0000 7fec005f7700 1 -- start start 2024-02-23T00:46:51.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.307+0000 7fec005f7700 1 librados: setting wanted keys 2024-02-23T00:46:51.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.307+0000 7fec005f7700 1 librados: calling monclient init 2024-02-23T00:46:51.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.308+0000 7fec005f7700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7febf80f8420 0x7febf810dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:51.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.308+0000 7fec005f7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febf80fdbb0 0x7febf810e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:51.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.308+0000 7fec005f7700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febf80fe510 0x7febf8108150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:51.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.308+0000 7fec005f7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf8108640 con 0x7febf80fdbb0 2024-02-23T00:46:51.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.308+0000 7fec005f7700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf81087b0 con 0x7febf80fe510 2024-02-23T00:46:51.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.308+0000 7fec005f7700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf8108970 con 0x7febf80f8420 2024-02-23T00:46:51.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.308+0000 7febfdb92700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febf80fdbb0 0x7febf810e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:51.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.308+0000 7febfe393700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7febf80f8420 0x7febf810dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:51.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.308+0000 7febfdb92700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febf80fdbb0 0x7febf810e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52516/0 (socket says 172.21.15.47:52516) 2024-02-23T00:46:51.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.308+0000 7febfdb92700 1 -- 172.21.15.47:0/4056702266 learned_addr learned my addr 172.21.15.47:0/4056702266 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:46:51.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.308+0000 7febfeb94700 1 --2- 172.21.15.47:0/4056702266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febf80fe510 0x7febf8108150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:51.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.308+0000 7febfe393700 1 -- 172.21.15.47:0/4056702266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febf80fe510 msgr2=0x7febf8108150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:51.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.308+0000 7febfe393700 1 --2- 172.21.15.47:0/4056702266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febf80fe510 0x7febf8108150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:51.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.308+0000 7febfe393700 1 -- 172.21.15.47:0/4056702266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febf80fdbb0 msgr2=0x7febf810e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:51.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.308+0000 7febfe393700 1 --2- 172.21.15.47:0/4056702266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febf80fdbb0 0x7febf810e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:51.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.308+0000 7febfe393700 1 -- 172.21.15.47:0/4056702266 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf4005040 con 0x7febf80f8420 2024-02-23T00:46:51.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.308+0000 7febfeb94700 1 --2- 172.21.15.47:0/4056702266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febf80fe510 0x7febf8108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:46:51.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.308+0000 7febfdb92700 1 --2- 172.21.15.47:0/4056702266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febf80fdbb0 0x7febf810e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:46:51.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.308+0000 7febfe393700 1 --2- 172.21.15.47:0/4056702266 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7febf80f8420 0x7febf810dfb0 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 rx=0x7febf4009da0 tx=0x7febf4009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:51.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.308+0000 7febef7fe700 10 client.?.objecter ms_handle_connect 0x7febf80f8420 2024-02-23T00:46:51.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.308+0000 7febef7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:46:51.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.309+0000 7febef7fe700 1 -- 172.21.15.47:0/4056702266 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf400e030 con 0x7febf80f8420 2024-02-23T00:46:51.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.309+0000 7febef7fe700 1 -- 172.21.15.47:0/4056702266 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7febf4009520 con 0x7febf80f8420 2024-02-23T00:46:51.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.309+0000 7fec005f7700 1 -- 172.21.15.47:0/4056702266 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf8108bf0 con 0x7febf80f8420 2024-02-23T00:46:51.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.309+0000 7febef7fe700 1 -- 172.21.15.47:0/4056702266 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf401c070 con 0x7febf80f8420 2024-02-23T00:46:51.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.309+0000 7fec005f7700 10 client.41150.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:46:51.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.309+0000 7fec005f7700 1 -- 172.21.15.47:0/4056702266 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf8062700 con 0x7febf80f8420 2024-02-23T00:46:51.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.309+0000 7fec005f7700 1 librados: init done 2024-02-23T00:46:51.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.310+0000 7fec005f7700 1 -- 172.21.15.47:0/4056702266 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf805f950 con 0x7febf80f8420 2024-02-23T00:46:51.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.310+0000 7febef7fe700 1 -- 172.21.15.47:0/4056702266 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7febf40125d0 con 0x7febf80f8420 2024-02-23T00:46:51.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.311+0000 7febef7fe700 1 --2- 172.21.15.47:0/4056702266 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7febe4062cc0 0x7febe4065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:51.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.311+0000 7febef7fe700 1 -- 172.21.15.47:0/4056702266 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(339..339 src has 48..339) v4 ==== 10399+0+0 (secure 0 0 0) 0x7febf4083100 con 0x7febf80f8420 2024-02-23T00:46:51.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.311+0000 7febef7fe700 10 client.41150.objecter ms_dispatch 0x7febf8001550 osd_map(339..339 src has 48..339) v4 2024-02-23T00:46:51.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.311+0000 7febef7fe700 3 client.41150.objecter handle_osd_map got epochs [339,339] > 0 2024-02-23T00:46:51.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.311+0000 7febef7fe700 3 client.41150.objecter handle_osd_map decoding full epoch 339 2024-02-23T00:46:51.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.311+0000 7febef7fe700 20 client.41150.objecter dump_active .. 0 homeless 2024-02-23T00:46:51.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.312+0000 7febfdb92700 1 --2- 172.21.15.47:0/4056702266 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7febe4062cc0 0x7febe4065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:51.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.312+0000 7febfdb92700 1 --2- 172.21.15.47:0/4056702266 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7febe4062cc0 0x7febe4065180 secure :-1 s=READY pgs=3031 cs=0 l=1 rev1=1 rx=0x7febe8004fd0 tx=0x7febe800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:51.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.312+0000 7febef7fe700 10 client.41150.objecter ms_handle_connect 0x7febe4062cc0 2024-02-23T00:46:51.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.315+0000 7febef7fe700 1 -- 172.21.15.47:0/4056702266 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7febf4052280 con 0x7febf80f8420 2024-02-23T00:46:51.323 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:46:51.357 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:46:51.391 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:46:51.420 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:51.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:51.528+0000 7fec005f7700 1 -- 172.21.15.47:0/4056702266 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7febf8106e60 con 0x7febf80f8420 2024-02-23T00:46:51.626 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:46:51.629 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:51.629 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:46:51.730 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:46:51.758 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:51.777 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:46:51.779 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:51.780 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:46:51.880 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:46:51.892 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:50.384259+0000 osd.6 (osd.6) 139 : cluster [DBG] 18.10 scrub starts 2024-02-23T00:46:51.892 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:50.385107+0000 osd.6 (osd.6) 140 : cluster [DBG] 18.10 scrub ok 2024-02-23T00:46:51.893 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:51.034869+0000 mon.a (mon.0) 2207 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:46:51.893 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:51.177378+0000 mon.a (mon.0) 2208 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi047-82386-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi047-82386-57"}]': finished 2024-02-23T00:46:51.893 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:51.177489+0000 mon.a (mon.0) 2209 : cluster [DBG] osdmap e339: 8 total, 8 up, 8 in 2024-02-23T00:46:51.893 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:51.265679+0000 mon.a (mon.0) 2210 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "15"}]: dispatch 2024-02-23T00:46:51.893 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:51.461244+0000 mon.a (mon.0) 2211 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:46:51.893 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:51.461934+0000 mon.a (mon.0) 2212 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:46:51.893 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:51.466189+0000 mon.a (mon.0) 2213 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:46:51.893 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:51.529523+0000 mon.c (mon.2) 255 : audit [INF] from='client.? 172.21.15.47:0/4056702266' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:46:51.893 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:51.529849+0000 mon.a (mon.0) 2214 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:46:51.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:50.384259+0000 osd.6 (osd.6) 139 : cluster [DBG] 18.10 scrub starts 2024-02-23T00:46:51.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:50.385107+0000 osd.6 (osd.6) 140 : cluster [DBG] 18.10 scrub ok 2024-02-23T00:46:51.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:51.034869+0000 mon.a (mon.0) 2207 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:46:51.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:51.177378+0000 mon.a (mon.0) 2208 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi047-82386-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi047-82386-57"}]': finished 2024-02-23T00:46:51.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:51.177489+0000 mon.a (mon.0) 2209 : cluster [DBG] osdmap e339: 8 total, 8 up, 8 in 2024-02-23T00:46:51.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:51.265679+0000 mon.a (mon.0) 2210 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "15"}]: dispatch 2024-02-23T00:46:51.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:51.461244+0000 mon.a (mon.0) 2211 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:46:51.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:51.461934+0000 mon.a (mon.0) 2212 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:46:51.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:51.466189+0000 mon.a (mon.0) 2213 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:46:51.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:51.529523+0000 mon.c (mon.2) 255 : audit [INF] from='client.? 172.21.15.47:0/4056702266' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:46:51.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:51.529849+0000 mon.a (mon.0) 2214 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:46:51.901 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:50.384259+0000 osd.6 (osd.6) 139 : cluster [DBG] 18.10 scrub starts 2024-02-23T00:46:51.902 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:50.385107+0000 osd.6 (osd.6) 140 : cluster [DBG] 18.10 scrub ok 2024-02-23T00:46:51.902 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:51.034869+0000 mon.a (mon.0) 2207 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:46:51.902 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:51.177378+0000 mon.a (mon.0) 2208 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi047-82386-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi047-82386-57"}]': finished 2024-02-23T00:46:51.902 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:51.177489+0000 mon.a (mon.0) 2209 : cluster [DBG] osdmap e339: 8 total, 8 up, 8 in 2024-02-23T00:46:51.902 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:51.265679+0000 mon.a (mon.0) 2210 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "15"}]: dispatch 2024-02-23T00:46:51.902 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:51.461244+0000 mon.a (mon.0) 2211 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:46:51.902 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:51.461934+0000 mon.a (mon.0) 2212 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:46:51.902 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:51.466189+0000 mon.a (mon.0) 2213 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:46:51.902 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:51.529523+0000 mon.c (mon.2) 255 : audit [INF] from='client.? 172.21.15.47:0/4056702266' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:46:51.902 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:51.529849+0000 mon.a (mon.0) 2214 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:46:51.909 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:52.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:52.190+0000 7febef7fe700 1 -- 172.21.15.47:0/4056702266 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v340) v1 ==== 87+0+0 (secure 0 0 0) 0x7febf401d270 con 0x7febf80f8420 2024-02-23T00:46:52.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:52.194+0000 7fec005f7700 10 librados: watch_flush enter 2024-02-23T00:46:52.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:52.195+0000 7fec005f7700 10 librados: watch_flush exit 2024-02-23T00:46:52.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:52.195+0000 7fec005f7700 20 client.41150.objecter shutdown clearing up homeless session... 2024-02-23T00:46:52.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:52.195+0000 7fec005f7700 10 client.41150.objecter successfully canceled tick 2024-02-23T00:46:52.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:52.195+0000 7fec005f7700 1 -- 172.21.15.47:0/4056702266 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7febe4062cc0 msgr2=0x7febe4065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:52.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:52.195+0000 7fec005f7700 1 --2- 172.21.15.47:0/4056702266 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7febe4062cc0 0x7febe4065180 secure :-1 s=READY pgs=3031 cs=0 l=1 rev1=1 rx=0x7febe8004fd0 tx=0x7febe800b040).stop 2024-02-23T00:46:52.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:52.195+0000 7fec005f7700 1 -- 172.21.15.47:0/4056702266 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7febf80f8420 msgr2=0x7febf810dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:52.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:52.195+0000 7fec005f7700 1 --2- 172.21.15.47:0/4056702266 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7febf80f8420 0x7febf810dfb0 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 rx=0x7febf4009da0 tx=0x7febf4009cb0).stop 2024-02-23T00:46:52.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:52.195+0000 7fec005f7700 1 -- 172.21.15.47:0/4056702266 shutdown_connections 2024-02-23T00:46:52.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:52.195+0000 7fec005f7700 1 --2- 172.21.15.47:0/4056702266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febf80fe510 0x7febf8108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:52.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:52.195+0000 7fec005f7700 1 --2- 172.21.15.47:0/4056702266 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7febe4062cc0 0x7febe4065180 unknown :-1 s=CLOSED pgs=3031 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:52.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:52.195+0000 7fec005f7700 1 --2- 172.21.15.47:0/4056702266 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7febf80f8420 0x7febf810dfb0 unknown :-1 s=CLOSED pgs=2625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:52.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:52.195+0000 7fec005f7700 1 --2- 172.21.15.47:0/4056702266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febf80fdbb0 0x7febf810e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:52.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:52.195+0000 7fec005f7700 1 -- 172.21.15.47:0/4056702266 >> 172.21.15.47:0/4056702266 conn(0x7febf80f3800 msgr2=0x7febf80fc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:46:52.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:52.195+0000 7fec005f7700 1 -- 172.21.15.47:0/4056702266 shutdown_connections 2024-02-23T00:46:52.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:52.195+0000 7fec005f7700 1 -- 172.21.15.47:0/4056702266 wait complete. 2024-02-23T00:46:52.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:52.196+0000 7fec005f7700 1 librados: shutdown 2024-02-23T00:46:52.199 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is set 2024-02-23T00:46:52.574 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:46:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:46:52.574 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:46:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:46:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:46:52.758 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:46:52.761 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:52.762 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:46:52.862 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:46:52.891 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:51.251812+0000 osd.1 (osd.1) 155 : cluster [DBG] 18.3 deep-scrub starts 2024-02-23T00:46:52.891 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:51.254898+0000 osd.1 (osd.1) 156 : cluster [DBG] 18.3 deep-scrub ok 2024-02-23T00:46:52.891 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:51.264427+0000 mgr.y (mgr.14517) 328 : cluster [DBG] pgmap v484: 242 pgs: 8 unknown, 234 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:46:52.891 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:51.331738+0000 osd.4 (osd.4) 143 : cluster [DBG] 4.9 scrub starts 2024-02-23T00:46:52.891 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:51.333892+0000 osd.4 (osd.4) 144 : cluster [DBG] 4.9 scrub ok 2024-02-23T00:46:52.892 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:51.430113+0000 osd.6 (osd.6) 141 : cluster [DBG] 4.1f scrub starts 2024-02-23T00:46:52.892 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:51.431833+0000 osd.6 (osd.6) 142 : cluster [DBG] 4.1f scrub ok 2024-02-23T00:46:52.892 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:51.603307+0000 osd.2 (osd.2) 151 : cluster [DBG] 6.14 scrub starts 2024-02-23T00:46:52.892 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:51.604141+0000 osd.2 (osd.2) 152 : cluster [DBG] 6.14 scrub ok 2024-02-23T00:46:52.892 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:52.175224+0000 mon.a (mon.0) 2215 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:46:52.892 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:52.181582+0000 mon.a (mon.0) 2216 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "15"}]': finished 2024-02-23T00:46:52.892 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:52.181708+0000 mon.a (mon.0) 2217 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:46:52.892 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:52.181831+0000 mon.a (mon.0) 2218 : cluster [DBG] osdmap e340: 8 total, 8 up, 8 in 2024-02-23T00:46:52.892 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:52.210875+0000 mon.c (mon.2) 256 : audit [INF] from='client.? 172.21.15.47:0/3949512893' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi047-82419-2"}]: dispatch 2024-02-23T00:46:52.892 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:52.211224+0000 mon.a (mon.0) 2219 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi047-82419-2"}]: dispatch 2024-02-23T00:46:52.892 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:52.211923+0000 mon.c (mon.2) 257 : audit [INF] from='client.? 172.21.15.47:0/3949512893' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi047-82419-2"}]: dispatch 2024-02-23T00:46:52.892 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:52.212189+0000 mon.a (mon.0) 2220 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi047-82419-2"}]: dispatch 2024-02-23T00:46:52.892 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:52.212804+0000 mon.c (mon.2) 258 : audit [INF] from='client.? 172.21.15.47:0/3949512893' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi047-82419-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:52.893 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:52.213101+0000 mon.a (mon.0) 2221 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi047-82419-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:52.893 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:51.251812+0000 osd.1 (osd.1) 155 : cluster [DBG] 18.3 deep-scrub starts 2024-02-23T00:46:52.893 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:51.254898+0000 osd.1 (osd.1) 156 : cluster [DBG] 18.3 deep-scrub ok 2024-02-23T00:46:52.893 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:51.264427+0000 mgr.y (mgr.14517) 328 : cluster [DBG] pgmap v484: 242 pgs: 8 unknown, 234 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:46:52.893 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:51.331738+0000 osd.4 (osd.4) 143 : cluster [DBG] 4.9 scrub starts 2024-02-23T00:46:52.893 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:51.333892+0000 osd.4 (osd.4) 144 : cluster [DBG] 4.9 scrub ok 2024-02-23T00:46:52.893 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:51.430113+0000 osd.6 (osd.6) 141 : cluster [DBG] 4.1f scrub starts 2024-02-23T00:46:52.893 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:51.431833+0000 osd.6 (osd.6) 142 : cluster [DBG] 4.1f scrub ok 2024-02-23T00:46:52.893 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:51.603307+0000 osd.2 (osd.2) 151 : cluster [DBG] 6.14 scrub starts 2024-02-23T00:46:52.893 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:51.604141+0000 osd.2 (osd.2) 152 : cluster [DBG] 6.14 scrub ok 2024-02-23T00:46:52.893 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:52.175224+0000 mon.a (mon.0) 2215 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:46:52.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:52.181582+0000 mon.a (mon.0) 2216 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "15"}]': finished 2024-02-23T00:46:52.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:52.181708+0000 mon.a (mon.0) 2217 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:46:52.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:52.181831+0000 mon.a (mon.0) 2218 : cluster [DBG] osdmap e340: 8 total, 8 up, 8 in 2024-02-23T00:46:52.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:52.210875+0000 mon.c (mon.2) 256 : audit [INF] from='client.? 172.21.15.47:0/3949512893' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi047-82419-2"}]: dispatch 2024-02-23T00:46:52.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:52.211224+0000 mon.a (mon.0) 2219 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi047-82419-2"}]: dispatch 2024-02-23T00:46:52.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:52.211923+0000 mon.c (mon.2) 257 : audit [INF] from='client.? 172.21.15.47:0/3949512893' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi047-82419-2"}]: dispatch 2024-02-23T00:46:52.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:52.212189+0000 mon.a (mon.0) 2220 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi047-82419-2"}]: dispatch 2024-02-23T00:46:52.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:52.212804+0000 mon.c (mon.2) 258 : audit [INF] from='client.? 172.21.15.47:0/3949512893' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi047-82419-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:52.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:52.213101+0000 mon.a (mon.0) 2221 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi047-82419-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:52.895 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:52.900 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:51.251812+0000 osd.1 (osd.1) 155 : cluster [DBG] 18.3 deep-scrub starts 2024-02-23T00:46:52.900 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:51.254898+0000 osd.1 (osd.1) 156 : cluster [DBG] 18.3 deep-scrub ok 2024-02-23T00:46:52.900 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:51.264427+0000 mgr.y (mgr.14517) 328 : cluster [DBG] pgmap v484: 242 pgs: 8 unknown, 234 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:46:52.900 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:51.331738+0000 osd.4 (osd.4) 143 : cluster [DBG] 4.9 scrub starts 2024-02-23T00:46:52.900 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:51.333892+0000 osd.4 (osd.4) 144 : cluster [DBG] 4.9 scrub ok 2024-02-23T00:46:52.901 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:51.430113+0000 osd.6 (osd.6) 141 : cluster [DBG] 4.1f scrub starts 2024-02-23T00:46:52.901 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:51.431833+0000 osd.6 (osd.6) 142 : cluster [DBG] 4.1f scrub ok 2024-02-23T00:46:52.901 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:51.603307+0000 osd.2 (osd.2) 151 : cluster [DBG] 6.14 scrub starts 2024-02-23T00:46:52.901 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:51.604141+0000 osd.2 (osd.2) 152 : cluster [DBG] 6.14 scrub ok 2024-02-23T00:46:52.901 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:52.175224+0000 mon.a (mon.0) 2215 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:46:52.901 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:52.181582+0000 mon.a (mon.0) 2216 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi047-82419-1", "var": "pg_num_actual", "val": "15"}]': finished 2024-02-23T00:46:52.901 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:52.181708+0000 mon.a (mon.0) 2217 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:46:52.901 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:52.181831+0000 mon.a (mon.0) 2218 : cluster [DBG] osdmap e340: 8 total, 8 up, 8 in 2024-02-23T00:46:52.901 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:52.210875+0000 mon.c (mon.2) 256 : audit [INF] from='client.? 172.21.15.47:0/3949512893' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi047-82419-2"}]: dispatch 2024-02-23T00:46:52.901 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:52.211224+0000 mon.a (mon.0) 2219 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi047-82419-2"}]: dispatch 2024-02-23T00:46:52.901 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:52.211923+0000 mon.c (mon.2) 257 : audit [INF] from='client.? 172.21.15.47:0/3949512893' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi047-82419-2"}]: dispatch 2024-02-23T00:46:52.901 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:52.212189+0000 mon.a (mon.0) 2220 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi047-82419-2"}]: dispatch 2024-02-23T00:46:52.901 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:52.212804+0000 mon.c (mon.2) 258 : audit [INF] from='client.? 172.21.15.47:0/3949512893' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi047-82419-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:52.901 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:52.213101+0000 mon.a (mon.0) 2221 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi047-82419-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:53.181 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:46:53.181 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-02-23T00:46:53.181 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 0 2024-02-23T00:46:53.181 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:46:53.546 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:46:53.549 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:53.550 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:46:53.651 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:46:53.683 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:53.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:52.224554+0000 osd.7 (osd.7) 179 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:46:53.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:52.224722+0000 osd.7 (osd.7) 180 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:46:53.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:52.525117+0000 mgr.y (mgr.14517) 329 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:46:53.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:53.182829+0000 mon.a (mon.0) 2222 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi047-82419-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:53.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:53.182919+0000 mon.a (mon.0) 2223 : cluster [DBG] osdmap e341: 8 total, 8 up, 8 in 2024-02-23T00:46:53.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:53.184825+0000 mon.c (mon.2) 259 : audit [INF] from='client.? 172.21.15.47:0/3949512893' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi047-82419-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi047-82419-2"}]: dispatch 2024-02-23T00:46:53.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:53.185444+0000 mon.a (mon.0) 2224 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi047-82419-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi047-82419-2"}]: dispatch 2024-02-23T00:46:53.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:53.186107+0000 mon.c (mon.2) 260 : audit [INF] from='client.? 172.21.15.47:0/1633196565' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi047-82386-57"}]: dispatch 2024-02-23T00:46:53.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:53.186596+0000 mon.a (mon.0) 2225 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi047-82386-57"}]: dispatch 2024-02-23T00:46:53.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:52.224554+0000 osd.7 (osd.7) 179 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:46:53.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:52.224722+0000 osd.7 (osd.7) 180 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:46:53.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:52.525117+0000 mgr.y (mgr.14517) 329 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:46:53.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:53.182829+0000 mon.a (mon.0) 2222 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi047-82419-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:53.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:53.182919+0000 mon.a (mon.0) 2223 : cluster [DBG] osdmap e341: 8 total, 8 up, 8 in 2024-02-23T00:46:53.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:53.184825+0000 mon.c (mon.2) 259 : audit [INF] from='client.? 172.21.15.47:0/3949512893' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi047-82419-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi047-82419-2"}]: dispatch 2024-02-23T00:46:53.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:53.185444+0000 mon.a (mon.0) 2224 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi047-82419-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi047-82419-2"}]: dispatch 2024-02-23T00:46:53.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:53.186107+0000 mon.c (mon.2) 260 : audit [INF] from='client.? 172.21.15.47:0/1633196565' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi047-82386-57"}]: dispatch 2024-02-23T00:46:53.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:53.186596+0000 mon.a (mon.0) 2225 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi047-82386-57"}]: dispatch 2024-02-23T00:46:53.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:52.224554+0000 osd.7 (osd.7) 179 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:46:53.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:52.224722+0000 osd.7 (osd.7) 180 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:46:53.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:52.525117+0000 mgr.y (mgr.14517) 329 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:46:53.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:53.182829+0000 mon.a (mon.0) 2222 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi047-82419-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:53.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:53.182919+0000 mon.a (mon.0) 2223 : cluster [DBG] osdmap e341: 8 total, 8 up, 8 in 2024-02-23T00:46:53.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:53.184825+0000 mon.c (mon.2) 259 : audit [INF] from='client.? 172.21.15.47:0/3949512893' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi047-82419-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi047-82419-2"}]: dispatch 2024-02-23T00:46:53.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:53.185444+0000 mon.a (mon.0) 2224 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi047-82419-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi047-82419-2"}]: dispatch 2024-02-23T00:46:53.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:53.186107+0000 mon.c (mon.2) 260 : audit [INF] from='client.? 172.21.15.47:0/1633196565' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi047-82386-57"}]: dispatch 2024-02-23T00:46:53.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:53.186596+0000 mon.a (mon.0) 2225 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi047-82386-57"}]: dispatch 2024-02-23T00:46:54.446 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.5 dump_blocked_ops 2024-02-23T00:46:54.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:53.265032+0000 mgr.y (mgr.14517) 330 : cluster [DBG] pgmap v487: 218 pgs: 218 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:54.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:54.629321+0000 mon.a (mon.0) 2226 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi047-82386-57"}]': finished 2024-02-23T00:46:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:54.629400+0000 mon.a (mon.0) 2227 : cluster [DBG] osdmap e342: 8 total, 8 up, 8 in 2024-02-23T00:46:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:54.629987+0000 mon.c (mon.2) 261 : audit [INF] from='client.? 172.21.15.47:0/1633196565' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi047-82386-57"}]: dispatch 2024-02-23T00:46:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:54.630280+0000 mon.a (mon.0) 2228 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi047-82386-57"}]: dispatch 2024-02-23T00:46:54.718 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:46:54.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:53.265032+0000 mgr.y (mgr.14517) 330 : cluster [DBG] pgmap v487: 218 pgs: 218 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:54.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:54.629321+0000 mon.a (mon.0) 2226 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi047-82386-57"}]': finished 2024-02-23T00:46:54.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:54.629400+0000 mon.a (mon.0) 2227 : cluster [DBG] osdmap e342: 8 total, 8 up, 8 in 2024-02-23T00:46:54.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:54.629987+0000 mon.c (mon.2) 261 : audit [INF] from='client.? 172.21.15.47:0/1633196565' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi047-82386-57"}]: dispatch 2024-02-23T00:46:54.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:54.630280+0000 mon.a (mon.0) 2228 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi047-82386-57"}]: dispatch 2024-02-23T00:46:54.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:53.265032+0000 mgr.y (mgr.14517) 330 : cluster [DBG] pgmap v487: 218 pgs: 218 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:54.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:54.629321+0000 mon.a (mon.0) 2226 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi047-82386-57"}]': finished 2024-02-23T00:46:54.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:54.629400+0000 mon.a (mon.0) 2227 : cluster [DBG] osdmap e342: 8 total, 8 up, 8 in 2024-02-23T00:46:54.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:54.629987+0000 mon.c (mon.2) 261 : audit [INF] from='client.? 172.21.15.47:0/1633196565' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi047-82386-57"}]: dispatch 2024-02-23T00:46:54.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:54.630280+0000 mon.a (mon.0) 2228 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi047-82386-57"}]: dispatch 2024-02-23T00:46:54.934 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:46:54.937 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:54.938 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:46:55.038 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:46:55.067 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:55.086 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:46:55.088 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:55.089 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:46:55.189 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:46:55.215 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:55.288 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:46:55.290 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:55.291 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:46:55.392 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:46:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:55.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.608+0000 7f39b8deb700 1 -- 172.21.15.47:0/1782014920 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39b40f7400 msgr2=0x7f39b4103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:55.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.608+0000 7f39b8deb700 1 --2- 172.21.15.47:0/1782014920 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39b40f7400 0x7f39b4103a20 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 rx=0x7f39ac009be0 tx=0x7f39ac004d10).stop 2024-02-23T00:46:55.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.608+0000 7f39b8deb700 1 -- 172.21.15.47:0/1782014920 shutdown_connections 2024-02-23T00:46:55.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.608+0000 7f39b8deb700 1 --2- 172.21.15.47:0/1782014920 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39b40f7400 0x7f39b4103a20 unknown :-1 s=CLOSED pgs=2630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:55.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.608+0000 7f39b8deb700 1 --2- 172.21.15.47:0/1782014920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39b40fcbb0 0x7f39b40fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:55.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.608+0000 7f39b8deb700 1 --2- 172.21.15.47:0/1782014920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39b40fd510 0x7f39b40f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:55.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.608+0000 7f39b8deb700 1 -- 172.21.15.47:0/1782014920 >> 172.21.15.47:0/1782014920 conn(0x7f39b40f2820 msgr2=0x7f39b40f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:46:55.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.608+0000 7f39b8deb700 1 -- 172.21.15.47:0/1782014920 shutdown_connections 2024-02-23T00:46:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.608+0000 7f39b8deb700 1 -- 172.21.15.47:0/1782014920 wait complete. 2024-02-23T00:46:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.609+0000 7f39b8deb700 1 librados: starting msgr at 2024-02-23T00:46:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.609+0000 7f39b8deb700 1 librados: starting objecter 2024-02-23T00:46:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.609+0000 7f39b8deb700 1 Processor -- start 2024-02-23T00:46:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.609+0000 7f39b8deb700 1 -- start start 2024-02-23T00:46:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.609+0000 7f39b8deb700 1 librados: setting wanted keys 2024-02-23T00:46:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.609+0000 7f39b8deb700 1 librados: calling monclient init 2024-02-23T00:46:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.609+0000 7f39b8deb700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39b40f7400 0x7f39b410cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.610+0000 7f39b8deb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39b40fcbb0 0x7f39b410d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.610+0000 7f39b8deb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39b40fd510 0x7f39b4106d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.610+0000 7f39b8deb700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b4107240 con 0x7f39b40fcbb0 2024-02-23T00:46:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.610+0000 7f39b8deb700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b41073b0 con 0x7f39b40fd510 2024-02-23T00:46:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.610+0000 7f39b8deb700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b4107570 con 0x7f39b40f7400 2024-02-23T00:46:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.610+0000 7f39b259c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39b40f7400 0x7f39b410cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.610+0000 7f39b259c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39b40f7400 0x7f39b410cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:40468/0 (socket says 172.21.15.47:40468) 2024-02-23T00:46:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.610+0000 7f39b259c700 1 -- 172.21.15.47:0/1108403066 learned_addr learned my addr 172.21.15.47:0/1108403066 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:46:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.610+0000 7f39b2d9d700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39b40fd510 0x7f39b4106d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.610+0000 7f39b1d9b700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39b40fcbb0 0x7f39b410d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.610+0000 7f39b259c700 1 -- 172.21.15.47:0/1108403066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39b40fd510 msgr2=0x7f39b4106d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.610+0000 7f39b259c700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39b40fd510 0x7f39b4106d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.610+0000 7f39b259c700 1 -- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39b40fcbb0 msgr2=0x7f39b410d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.610+0000 7f39b259c700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39b40fcbb0 0x7f39b410d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.610+0000 7f39b259c700 1 -- 172.21.15.47:0/1108403066 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39ac0097c0 con 0x7f39b40f7400 2024-02-23T00:46:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.610+0000 7f39b259c700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39b40f7400 0x7f39b410cbb0 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 rx=0x7f39a400c7c0 tx=0x7f39a4007ef0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.610+0000 7f39ab7fe700 10 client.?.objecter ms_handle_connect 0x7f39b40f7400 2024-02-23T00:46:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.610+0000 7f39ab7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:46:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.610+0000 7f39ab7fe700 1 -- 172.21.15.47:0/1108403066 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a4004430 con 0x7f39b40f7400 2024-02-23T00:46:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.610+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39b4107850 con 0x7f39b40f7400 2024-02-23T00:46:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.610+0000 7f39ab7fe700 1 -- 172.21.15.47:0/1108403066 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f39a400e930 con 0x7f39b40f7400 2024-02-23T00:46:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.610+0000 7f39ab7fe700 1 -- 172.21.15.47:0/1108403066 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a40139a0 con 0x7f39b40f7400 2024-02-23T00:46:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.610+0000 7f39b8deb700 10 client.41168.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:46:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.610+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b4107bd0 con 0x7f39b40f7400 2024-02-23T00:46:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.610+0000 7f39b8deb700 1 librados: init done 2024-02-23T00:46:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.611+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f39b405fa60 con 0x7f39b40f7400 2024-02-23T00:46:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.612+0000 7f39ab7fe700 1 -- 172.21.15.47:0/1108403066 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f39a400eaa0 con 0x7f39b40f7400 2024-02-23T00:46:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.612+0000 7f39ab7fe700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f39a0062cc0 0x7f39a0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.612+0000 7f39ab7fe700 1 -- 172.21.15.47:0/1108403066 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(343..343 src has 48..343) v4 ==== 10021+0+0 (secure 0 0 0) 0x7f39a407d840 con 0x7f39b40f7400 2024-02-23T00:46:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.612+0000 7f39ab7fe700 10 client.41168.objecter ms_dispatch 0x7f39b4001550 osd_map(343..343 src has 48..343) v4 2024-02-23T00:46:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.612+0000 7f39ab7fe700 3 client.41168.objecter handle_osd_map got epochs [343,343] > 0 2024-02-23T00:46:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.612+0000 7f39ab7fe700 3 client.41168.objecter handle_osd_map decoding full epoch 343 2024-02-23T00:46:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.612+0000 7f39b1d9b700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f39a0062cc0 0x7f39a0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.613+0000 7f39ab7fe700 20 client.41168.objecter dump_active .. 0 homeless 2024-02-23T00:46:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.613+0000 7f39ab7fe700 1 -- 172.21.15.47:0/1108403066 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v343) v1 ==== 54+0+16 (secure 0 0 0) 0x7f39a407e0a0 con 0x7f39b40f7400 2024-02-23T00:46:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.613+0000 7f39ab7fe700 10 client.41168.objecter ms_handle_connect 0x7f39a0062cc0 2024-02-23T00:46:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.613+0000 7f39b1d9b700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f39a0062cc0 0x7f39a0065180 secure :-1 s=READY pgs=3034 cs=0 l=1 rev1=1 rx=0x7f39b41084f0 tx=0x7f399c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.613+0000 7f39b8deb700 10 client.41168.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.613+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_assign -1 1 2024-02-23T00:46:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.613+0000 7f39b8deb700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f39b40fdea0 0x7f39b40fe280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.613+0000 7f39b8deb700 20 client.41168.objecter _get_session s=0x7f39b405ee80 osd=0 3 2024-02-23T00:46:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.613+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b405ee80 osd=0 3 2024-02-23T00:46:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.613+0000 7f39b8deb700 20 client.41168.objecter _get_session s=0x7f39b405ee80 osd=0 3 2024-02-23T00:46:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.613+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_remove -1 1 2024-02-23T00:46:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.613+0000 7f39b8deb700 20 client.41168.objecter get_session s=0x7f39b405ee80 osd=0 3 2024-02-23T00:46:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.613+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_assign 0 1 2024-02-23T00:46:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.613+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b405ee80 osd=0 4 2024-02-23T00:46:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.613+0000 7f39b8deb700 10 client.41168.objecter _send_command 1 2024-02-23T00:46:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.613+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39b4105ea0 con 0x7f39b40fdea0 2024-02-23T00:46:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.613+0000 7f39b2d9d700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f39b40fdea0 0x7f39b40fe280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.613+0000 7f39ab7fe700 10 client.41168.objecter ms_handle_connect 0x7f39b40fdea0 2024-02-23T00:46:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.613+0000 7f39b2d9d700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f39b40fdea0 0x7f39b40fe280 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.614+0000 7f39ab7fe700 1 -- 172.21.15.47:0/1108403066 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f39b4105ea0 con 0x7f39b40fdea0 2024-02-23T00:46:55.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.614+0000 7f39ab7fe700 10 client.41168.objecter ms_dispatch 0x7f39b4001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:46:55.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.614+0000 7f39ab7fe700 10 client.41168.objecter _finish_command 1 = system:0 2024-02-23T00:46:55.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.614+0000 7f39ab7fe700 20 client.41168.objecter put_session s=0x7f39b405ee80 osd=0 4 2024-02-23T00:46:55.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.614+0000 7f39ab7fe700 15 client.41168.objecter _session_command_op_remove 0 1 2024-02-23T00:46:55.622 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:46:55.625 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:55.625 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:46:55.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.630+0000 7f39b8deb700 10 client.41168.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:46:55.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.630+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_assign -1 2 2024-02-23T00:46:55.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.630+0000 7f39b8deb700 20 client.41168.objecter _get_session s=0x7f39b405ee80 osd=0 3 2024-02-23T00:46:55.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.630+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b405ee80 osd=0 3 2024-02-23T00:46:55.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.630+0000 7f39b8deb700 20 client.41168.objecter _get_session s=0x7f39b405ee80 osd=0 3 2024-02-23T00:46:55.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.630+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_remove -1 2 2024-02-23T00:46:55.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.630+0000 7f39b8deb700 20 client.41168.objecter get_session s=0x7f39b405ee80 osd=0 3 2024-02-23T00:46:55.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.630+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_assign 0 2 2024-02-23T00:46:55.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.630+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b405ee80 osd=0 4 2024-02-23T00:46:55.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.630+0000 7f39b8deb700 10 client.41168.objecter _send_command 2 2024-02-23T00:46:55.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.630+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f39b40fe9b0 con 0x7f39b40fdea0 2024-02-23T00:46:55.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.632+0000 7f39ab7fe700 1 -- 172.21.15.47:0/1108403066 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f39ac018040 con 0x7f39b40fdea0 2024-02-23T00:46:55.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.632+0000 7f39ab7fe700 10 client.41168.objecter ms_dispatch 0x7f39b4001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:46:55.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.632+0000 7f39ab7fe700 10 client.41168.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:46:55.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.632+0000 7f39ab7fe700 20 client.41168.objecter put_session s=0x7f39b405ee80 osd=0 4 2024-02-23T00:46:55.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.632+0000 7f39ab7fe700 15 client.41168.objecter _session_command_op_remove 0 2 2024-02-23T00:46:55.635 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:46:55.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.632+0000 7f39b8deb700 10 client.41168.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:55.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.632+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_assign -1 3 2024-02-23T00:46:55.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.632+0000 7f39b8deb700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f39b40f4be0 0x7f39b40273c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:55.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.632+0000 7f39b8deb700 20 client.41168.objecter _get_session s=0x7f39b40f4a70 osd=1 3 2024-02-23T00:46:55.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.632+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b40f4a70 osd=1 3 2024-02-23T00:46:55.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.632+0000 7f39b8deb700 20 client.41168.objecter _get_session s=0x7f39b40f4a70 osd=1 3 2024-02-23T00:46:55.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.632+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_remove -1 3 2024-02-23T00:46:55.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.632+0000 7f39b8deb700 20 client.41168.objecter get_session s=0x7f39b40f4a70 osd=1 3 2024-02-23T00:46:55.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.632+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_assign 1 3 2024-02-23T00:46:55.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.632+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b40f4a70 osd=1 4 2024-02-23T00:46:55.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.632+0000 7f39b8deb700 10 client.41168.objecter _send_command 3 2024-02-23T00:46:55.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.632+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39b4061a90 con 0x7f39b40f4be0 2024-02-23T00:46:55.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.632+0000 7f39b259c700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f39b40f4be0 0x7f39b40273c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:55.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.632+0000 7f39ab7fe700 10 client.41168.objecter ms_handle_connect 0x7f39b40f4be0 2024-02-23T00:46:55.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.632+0000 7f39b259c700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f39b40f4be0 0x7f39b40273c0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:55.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.633+0000 7f39ab7fe700 1 -- 172.21.15.47:0/1108403066 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f39b4061a90 con 0x7f39b40f4be0 2024-02-23T00:46:55.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.633+0000 7f39ab7fe700 10 client.41168.objecter ms_dispatch 0x7f39b4001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:46:55.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.633+0000 7f39ab7fe700 10 client.41168.objecter _finish_command 3 = system:0 2024-02-23T00:46:55.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.633+0000 7f39ab7fe700 20 client.41168.objecter put_session s=0x7f39b40f4a70 osd=1 4 2024-02-23T00:46:55.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.633+0000 7f39ab7fe700 15 client.41168.objecter _session_command_op_remove 1 3 2024-02-23T00:46:55.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.648+0000 7f39b8deb700 10 client.41168.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:46:55.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.648+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_assign -1 4 2024-02-23T00:46:55.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.648+0000 7f39b8deb700 20 client.41168.objecter _get_session s=0x7f39b40f4a70 osd=1 3 2024-02-23T00:46:55.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.648+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b40f4a70 osd=1 3 2024-02-23T00:46:55.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.648+0000 7f39b8deb700 20 client.41168.objecter _get_session s=0x7f39b40f4a70 osd=1 3 2024-02-23T00:46:55.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.648+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_remove -1 4 2024-02-23T00:46:55.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.648+0000 7f39b8deb700 20 client.41168.objecter get_session s=0x7f39b40f4a70 osd=1 3 2024-02-23T00:46:55.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.648+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_assign 1 4 2024-02-23T00:46:55.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.648+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b40f4a70 osd=1 4 2024-02-23T00:46:55.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.648+0000 7f39b8deb700 10 client.41168.objecter _send_command 4 2024-02-23T00:46:55.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.648+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f39b40278b0 con 0x7f39b40f4be0 2024-02-23T00:46:55.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.649+0000 7f39ab7fe700 1 -- 172.21.15.47:0/1108403066 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f39b40278b0 con 0x7f39b40f4be0 2024-02-23T00:46:55.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.649+0000 7f39ab7fe700 10 client.41168.objecter ms_dispatch 0x7f39b4001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:46:55.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.649+0000 7f39ab7fe700 10 client.41168.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:46:55.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.649+0000 7f39ab7fe700 20 client.41168.objecter put_session s=0x7f39b40f4a70 osd=1 4 2024-02-23T00:46:55.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.649+0000 7f39ab7fe700 15 client.41168.objecter _session_command_op_remove 1 4 2024-02-23T00:46:55.653 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:46:55.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.650+0000 7f39b8deb700 10 client.41168.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:55.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.650+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_assign -1 5 2024-02-23T00:46:55.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.650+0000 7f39b8deb700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f39b4061160 0x7f39b4119c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:55.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.650+0000 7f39b8deb700 20 client.41168.objecter _get_session s=0x7f39b4060ff0 osd=2 3 2024-02-23T00:46:55.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.650+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b4060ff0 osd=2 3 2024-02-23T00:46:55.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.650+0000 7f39b8deb700 20 client.41168.objecter _get_session s=0x7f39b4060ff0 osd=2 3 2024-02-23T00:46:55.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.650+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_remove -1 5 2024-02-23T00:46:55.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.650+0000 7f39b8deb700 20 client.41168.objecter get_session s=0x7f39b4060ff0 osd=2 3 2024-02-23T00:46:55.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.650+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_assign 2 5 2024-02-23T00:46:55.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.650+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b4060ff0 osd=2 4 2024-02-23T00:46:55.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.650+0000 7f39b8deb700 10 client.41168.objecter _send_command 5 2024-02-23T00:46:55.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.650+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39b411c910 con 0x7f39b4061160 2024-02-23T00:46:55.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.650+0000 7f39b1d9b700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f39b4061160 0x7f39b4119c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:55.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.650+0000 7f39ab7fe700 10 client.41168.objecter ms_handle_connect 0x7f39b4061160 2024-02-23T00:46:55.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.650+0000 7f39b1d9b700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f39b4061160 0x7f39b4119c30 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:55.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.651+0000 7f39ab7fe700 1 -- 172.21.15.47:0/1108403066 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f39b411c910 con 0x7f39b4061160 2024-02-23T00:46:55.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.651+0000 7f39ab7fe700 10 client.41168.objecter ms_dispatch 0x7f39b4001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:46:55.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.651+0000 7f39ab7fe700 10 client.41168.objecter _finish_command 5 = system:0 2024-02-23T00:46:55.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.651+0000 7f39ab7fe700 20 client.41168.objecter put_session s=0x7f39b4060ff0 osd=2 4 2024-02-23T00:46:55.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.651+0000 7f39ab7fe700 15 client.41168.objecter _session_command_op_remove 2 5 2024-02-23T00:46:55.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.667+0000 7f39b8deb700 10 client.41168.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:46:55.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.667+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_assign -1 6 2024-02-23T00:46:55.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.667+0000 7f39b8deb700 20 client.41168.objecter _get_session s=0x7f39b4060ff0 osd=2 3 2024-02-23T00:46:55.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.667+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b4060ff0 osd=2 3 2024-02-23T00:46:55.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.667+0000 7f39b8deb700 20 client.41168.objecter _get_session s=0x7f39b4060ff0 osd=2 3 2024-02-23T00:46:55.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.667+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_remove -1 6 2024-02-23T00:46:55.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.667+0000 7f39b8deb700 20 client.41168.objecter get_session s=0x7f39b4060ff0 osd=2 3 2024-02-23T00:46:55.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.667+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_assign 2 6 2024-02-23T00:46:55.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.667+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b4060ff0 osd=2 4 2024-02-23T00:46:55.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.667+0000 7f39b8deb700 10 client.41168.objecter _send_command 6 2024-02-23T00:46:55.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.667+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f39b4027cc0 con 0x7f39b4061160 2024-02-23T00:46:55.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.668+0000 7f39ab7fe700 1 -- 172.21.15.47:0/1108403066 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f39b4027cc0 con 0x7f39b4061160 2024-02-23T00:46:55.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.668+0000 7f39ab7fe700 10 client.41168.objecter ms_dispatch 0x7f39b4001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:46:55.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.668+0000 7f39ab7fe700 10 client.41168.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:46:55.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.668+0000 7f39ab7fe700 20 client.41168.objecter put_session s=0x7f39b4060ff0 osd=2 4 2024-02-23T00:46:55.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.668+0000 7f39ab7fe700 15 client.41168.objecter _session_command_op_remove 2 6 2024-02-23T00:46:55.671 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:46:55.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.668+0000 7f39b8deb700 10 client.41168.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:55.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.668+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_assign -1 7 2024-02-23T00:46:55.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.668+0000 7f39b8deb700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f39b4059e90 0x7f39b411b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:55.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.668+0000 7f39b8deb700 20 client.41168.objecter _get_session s=0x7f39b40280d0 osd=3 3 2024-02-23T00:46:55.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.668+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b40280d0 osd=3 3 2024-02-23T00:46:55.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.668+0000 7f39b8deb700 20 client.41168.objecter _get_session s=0x7f39b40280d0 osd=3 3 2024-02-23T00:46:55.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.668+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_remove -1 7 2024-02-23T00:46:55.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.668+0000 7f39b8deb700 20 client.41168.objecter get_session s=0x7f39b40280d0 osd=3 3 2024-02-23T00:46:55.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.668+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_assign 3 7 2024-02-23T00:46:55.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.668+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b40280d0 osd=3 4 2024-02-23T00:46:55.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.668+0000 7f39b8deb700 10 client.41168.objecter _send_command 7 2024-02-23T00:46:55.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.668+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39b411b9c0 con 0x7f39b4059e90 2024-02-23T00:46:55.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.669+0000 7f39b2d9d700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f39b4059e90 0x7f39b411b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:55.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.669+0000 7f39ab7fe700 10 client.41168.objecter ms_handle_connect 0x7f39b4059e90 2024-02-23T00:46:55.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.669+0000 7f39b2d9d700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f39b4059e90 0x7f39b411b2e0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:55.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.669+0000 7f39ab7fe700 1 -- 172.21.15.47:0/1108403066 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f39b411b9c0 con 0x7f39b4059e90 2024-02-23T00:46:55.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.669+0000 7f39ab7fe700 10 client.41168.objecter ms_dispatch 0x7f39b4001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:46:55.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.669+0000 7f39ab7fe700 10 client.41168.objecter _finish_command 7 = system:0 2024-02-23T00:46:55.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.669+0000 7f39ab7fe700 20 client.41168.objecter put_session s=0x7f39b40280d0 osd=3 4 2024-02-23T00:46:55.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.669+0000 7f39ab7fe700 15 client.41168.objecter _session_command_op_remove 3 7 2024-02-23T00:46:55.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.684+0000 7f39b8deb700 10 client.41168.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:46:55.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.684+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_assign -1 8 2024-02-23T00:46:55.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.684+0000 7f39b8deb700 20 client.41168.objecter _get_session s=0x7f39b40280d0 osd=3 3 2024-02-23T00:46:55.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.684+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b40280d0 osd=3 3 2024-02-23T00:46:55.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.684+0000 7f39b8deb700 20 client.41168.objecter _get_session s=0x7f39b40280d0 osd=3 3 2024-02-23T00:46:55.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.684+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_remove -1 8 2024-02-23T00:46:55.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.684+0000 7f39b8deb700 20 client.41168.objecter get_session s=0x7f39b40280d0 osd=3 3 2024-02-23T00:46:55.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.684+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_assign 3 8 2024-02-23T00:46:55.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.684+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b40280d0 osd=3 4 2024-02-23T00:46:55.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.684+0000 7f39b8deb700 10 client.41168.objecter _send_command 8 2024-02-23T00:46:55.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.684+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f39b411c390 con 0x7f39b4059e90 2024-02-23T00:46:55.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.685+0000 7f39ab7fe700 1 -- 172.21.15.47:0/1108403066 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f39b411c390 con 0x7f39b4059e90 2024-02-23T00:46:55.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.685+0000 7f39ab7fe700 10 client.41168.objecter ms_dispatch 0x7f39b4001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:46:55.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.685+0000 7f39ab7fe700 10 client.41168.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:46:55.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.686+0000 7f39ab7fe700 20 client.41168.objecter put_session s=0x7f39b40280d0 osd=3 4 2024-02-23T00:46:55.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.686+0000 7f39ab7fe700 15 client.41168.objecter _session_command_op_remove 3 8 2024-02-23T00:46:55.688 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:46:55.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.686+0000 7f39b8deb700 10 client.41168.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:55.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.686+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_assign -1 9 2024-02-23T00:46:55.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.686+0000 7f39b8deb700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f39b411ea90 0x7f39b4120f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:55.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.686+0000 7f39b8deb700 20 client.41168.objecter _get_session s=0x7f39b411bbc0 osd=4 3 2024-02-23T00:46:55.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.686+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b411bbc0 osd=4 3 2024-02-23T00:46:55.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.686+0000 7f39b8deb700 20 client.41168.objecter _get_session s=0x7f39b411bbc0 osd=4 3 2024-02-23T00:46:55.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.686+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_remove -1 9 2024-02-23T00:46:55.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.686+0000 7f39b8deb700 20 client.41168.objecter get_session s=0x7f39b411bbc0 osd=4 3 2024-02-23T00:46:55.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.686+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_assign 4 9 2024-02-23T00:46:55.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.686+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b411bbc0 osd=4 4 2024-02-23T00:46:55.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.686+0000 7f39b8deb700 10 client.41168.objecter _send_command 9 2024-02-23T00:46:55.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.686+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39b411c2d0 con 0x7f39b411ea90 2024-02-23T00:46:55.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.686+0000 7f39b259c700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f39b411ea90 0x7f39b4120f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:55.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.686+0000 7f39ab7fe700 10 client.41168.objecter ms_handle_connect 0x7f39b411ea90 2024-02-23T00:46:55.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.686+0000 7f39b259c700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f39b411ea90 0x7f39b4120f00 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:55.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.687+0000 7f39ab7fe700 1 -- 172.21.15.47:0/1108403066 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f39b411c2d0 con 0x7f39b411ea90 2024-02-23T00:46:55.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.687+0000 7f39ab7fe700 10 client.41168.objecter ms_dispatch 0x7f39b4001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:46:55.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.687+0000 7f39ab7fe700 10 client.41168.objecter _finish_command 9 = system:0 2024-02-23T00:46:55.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.687+0000 7f39ab7fe700 20 client.41168.objecter put_session s=0x7f39b411bbc0 osd=4 4 2024-02-23T00:46:55.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.687+0000 7f39ab7fe700 15 client.41168.objecter _session_command_op_remove 4 9 2024-02-23T00:46:55.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.702+0000 7f39b8deb700 10 client.41168.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:46:55.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.702+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_assign -1 10 2024-02-23T00:46:55.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.702+0000 7f39b8deb700 20 client.41168.objecter _get_session s=0x7f39b411bbc0 osd=4 3 2024-02-23T00:46:55.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.702+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b411bbc0 osd=4 3 2024-02-23T00:46:55.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.702+0000 7f39b8deb700 20 client.41168.objecter _get_session s=0x7f39b411bbc0 osd=4 3 2024-02-23T00:46:55.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.702+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_remove -1 10 2024-02-23T00:46:55.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.702+0000 7f39b8deb700 20 client.41168.objecter get_session s=0x7f39b411bbc0 osd=4 3 2024-02-23T00:46:55.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.702+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_assign 4 10 2024-02-23T00:46:55.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.702+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b411bbc0 osd=4 4 2024-02-23T00:46:55.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.702+0000 7f39b8deb700 10 client.41168.objecter _send_command 10 2024-02-23T00:46:55.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.702+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f39b411c2d0 con 0x7f39b411ea90 2024-02-23T00:46:55.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.703+0000 7f39ab7fe700 1 -- 172.21.15.47:0/1108403066 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f39b411c2d0 con 0x7f39b411ea90 2024-02-23T00:46:55.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.703+0000 7f39ab7fe700 10 client.41168.objecter ms_dispatch 0x7f39b4001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:46:55.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.703+0000 7f39ab7fe700 10 client.41168.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:46:55.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.703+0000 7f39ab7fe700 20 client.41168.objecter put_session s=0x7f39b411bbc0 osd=4 4 2024-02-23T00:46:55.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.703+0000 7f39ab7fe700 15 client.41168.objecter _session_command_op_remove 4 10 2024-02-23T00:46:55.706 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:46:55.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.704+0000 7f39b8deb700 10 client.41168.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:55.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.704+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_assign -1 11 2024-02-23T00:46:55.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.704+0000 7f39b8deb700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f39b4121900 0x7f39b4121d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:55.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.704+0000 7f39b8deb700 20 client.41168.objecter _get_session s=0x7f39b411c2d0 osd=5 3 2024-02-23T00:46:55.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.704+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b411c2d0 osd=5 3 2024-02-23T00:46:55.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.704+0000 7f39b8deb700 20 client.41168.objecter _get_session s=0x7f39b411c2d0 osd=5 3 2024-02-23T00:46:55.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.704+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_remove -1 11 2024-02-23T00:46:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.704+0000 7f39b8deb700 20 client.41168.objecter get_session s=0x7f39b411c2d0 osd=5 3 2024-02-23T00:46:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.704+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_assign 5 11 2024-02-23T00:46:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.704+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b411c2d0 osd=5 4 2024-02-23T00:46:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.704+0000 7f39b8deb700 10 client.41168.objecter _send_command 11 2024-02-23T00:46:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.704+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39b412afb0 con 0x7f39b4121900 2024-02-23T00:46:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.704+0000 7f39b1d9b700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f39b4121900 0x7f39b4121d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.704+0000 7f39ab7fe700 10 client.41168.objecter ms_handle_connect 0x7f39b4121900 2024-02-23T00:46:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.704+0000 7f39b1d9b700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f39b4121900 0x7f39b4121d60 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.705+0000 7f39ab7fe700 1 -- 172.21.15.47:0/1108403066 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f39b412afb0 con 0x7f39b4121900 2024-02-23T00:46:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.705+0000 7f39ab7fe700 10 client.41168.objecter ms_dispatch 0x7f39b4001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:46:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.705+0000 7f39ab7fe700 10 client.41168.objecter _finish_command 11 = system:0 2024-02-23T00:46:55.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.705+0000 7f39ab7fe700 20 client.41168.objecter put_session s=0x7f39b411c2d0 osd=5 4 2024-02-23T00:46:55.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.705+0000 7f39ab7fe700 15 client.41168.objecter _session_command_op_remove 5 11 2024-02-23T00:46:55.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.719+0000 7f39b8deb700 10 client.41168.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:46:55.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.719+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_assign -1 12 2024-02-23T00:46:55.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.719+0000 7f39b8deb700 20 client.41168.objecter _get_session s=0x7f39b411c2d0 osd=5 3 2024-02-23T00:46:55.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.719+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b411c2d0 osd=5 3 2024-02-23T00:46:55.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.719+0000 7f39b8deb700 20 client.41168.objecter _get_session s=0x7f39b411c2d0 osd=5 3 2024-02-23T00:46:55.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.719+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_remove -1 12 2024-02-23T00:46:55.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.719+0000 7f39b8deb700 20 client.41168.objecter get_session s=0x7f39b411c2d0 osd=5 3 2024-02-23T00:46:55.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.719+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_assign 5 12 2024-02-23T00:46:55.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.719+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b411c2d0 osd=5 4 2024-02-23T00:46:55.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.719+0000 7f39b8deb700 10 client.41168.objecter _send_command 12 2024-02-23T00:46:55.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.719+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f39b41287a0 con 0x7f39b4121900 2024-02-23T00:46:55.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.721+0000 7f39ab7fe700 1 -- 172.21.15.47:0/1108403066 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f39b41287a0 con 0x7f39b4121900 2024-02-23T00:46:55.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.721+0000 7f39ab7fe700 10 client.41168.objecter ms_dispatch 0x7f39b4001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:46:55.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.721+0000 7f39ab7fe700 10 client.41168.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:46:55.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.721+0000 7f39ab7fe700 20 client.41168.objecter put_session s=0x7f39b411c2d0 osd=5 4 2024-02-23T00:46:55.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.721+0000 7f39ab7fe700 15 client.41168.objecter _session_command_op_remove 5 12 2024-02-23T00:46:55.724 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:46:55.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.721+0000 7f39b8deb700 10 client.41168.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:55.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.721+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_assign -1 13 2024-02-23T00:46:55.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.721+0000 7f39b8deb700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f39b412a5f0 0x7f39b41287a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:55.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.721+0000 7f39b8deb700 20 client.41168.objecter _get_session s=0x7f39b4129f70 osd=6 3 2024-02-23T00:46:55.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.721+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b4129f70 osd=6 3 2024-02-23T00:46:55.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.721+0000 7f39b8deb700 20 client.41168.objecter _get_session s=0x7f39b4129f70 osd=6 3 2024-02-23T00:46:55.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.721+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_remove -1 13 2024-02-23T00:46:55.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.721+0000 7f39b8deb700 20 client.41168.objecter get_session s=0x7f39b4129f70 osd=6 3 2024-02-23T00:46:55.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.721+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_assign 6 13 2024-02-23T00:46:55.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.721+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b4129f70 osd=6 4 2024-02-23T00:46:55.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.721+0000 7f39b8deb700 10 client.41168.objecter _send_command 13 2024-02-23T00:46:55.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.721+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39b412ab90 con 0x7f39b412a5f0 2024-02-23T00:46:55.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.721+0000 7f39b2d9d700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f39b412a5f0 0x7f39b41287a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:55.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.722+0000 7f39ab7fe700 10 client.41168.objecter ms_handle_connect 0x7f39b412a5f0 2024-02-23T00:46:55.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.722+0000 7f39b2d9d700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f39b412a5f0 0x7f39b41287a0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:55.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.722+0000 7f39ab7fe700 1 -- 172.21.15.47:0/1108403066 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f39b412ab90 con 0x7f39b412a5f0 2024-02-23T00:46:55.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.722+0000 7f39ab7fe700 10 client.41168.objecter ms_dispatch 0x7f39b4001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:46:55.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.722+0000 7f39ab7fe700 10 client.41168.objecter _finish_command 13 = system:0 2024-02-23T00:46:55.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.722+0000 7f39ab7fe700 20 client.41168.objecter put_session s=0x7f39b4129f70 osd=6 4 2024-02-23T00:46:55.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.722+0000 7f39ab7fe700 15 client.41168.objecter _session_command_op_remove 6 13 2024-02-23T00:46:55.727 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:46:55.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.737+0000 7f39b8deb700 10 client.41168.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:46:55.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.737+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_assign -1 14 2024-02-23T00:46:55.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.737+0000 7f39b8deb700 20 client.41168.objecter _get_session s=0x7f39b4129f70 osd=6 3 2024-02-23T00:46:55.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.737+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b4129f70 osd=6 3 2024-02-23T00:46:55.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.737+0000 7f39b8deb700 20 client.41168.objecter _get_session s=0x7f39b4129f70 osd=6 3 2024-02-23T00:46:55.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.737+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_remove -1 14 2024-02-23T00:46:55.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.737+0000 7f39b8deb700 20 client.41168.objecter get_session s=0x7f39b4129f70 osd=6 3 2024-02-23T00:46:55.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.737+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_assign 6 14 2024-02-23T00:46:55.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.737+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b4129f70 osd=6 4 2024-02-23T00:46:55.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.737+0000 7f39b8deb700 10 client.41168.objecter _send_command 14 2024-02-23T00:46:55.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.737+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f39b412ab90 con 0x7f39b412a5f0 2024-02-23T00:46:55.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.738+0000 7f39ab7fe700 1 -- 172.21.15.47:0/1108403066 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f39b412ab90 con 0x7f39b412a5f0 2024-02-23T00:46:55.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.738+0000 7f39ab7fe700 10 client.41168.objecter ms_dispatch 0x7f39b4001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:46:55.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.738+0000 7f39ab7fe700 10 client.41168.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:46:55.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.738+0000 7f39ab7fe700 20 client.41168.objecter put_session s=0x7f39b4129f70 osd=6 4 2024-02-23T00:46:55.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.738+0000 7f39ab7fe700 15 client.41168.objecter _session_command_op_remove 6 14 2024-02-23T00:46:55.741 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:46:55.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.739+0000 7f39b8deb700 10 client.41168.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:46:55.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.739+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_assign -1 15 2024-02-23T00:46:55.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.739+0000 7f39b8deb700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f39b41291a0 0x7f39b4129620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:46:55.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.739+0000 7f39b8deb700 20 client.41168.objecter _get_session s=0x7f39b412ab90 osd=7 3 2024-02-23T00:46:55.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.739+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b412ab90 osd=7 3 2024-02-23T00:46:55.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.739+0000 7f39b8deb700 20 client.41168.objecter _get_session s=0x7f39b412ab90 osd=7 3 2024-02-23T00:46:55.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.739+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_remove -1 15 2024-02-23T00:46:55.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.739+0000 7f39b8deb700 20 client.41168.objecter get_session s=0x7f39b412ab90 osd=7 3 2024-02-23T00:46:55.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.739+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_assign 7 15 2024-02-23T00:46:55.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.739+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b412ab90 osd=7 4 2024-02-23T00:46:55.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.739+0000 7f39b8deb700 10 client.41168.objecter _send_command 15 2024-02-23T00:46:55.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.739+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39b4129bc0 con 0x7f39b41291a0 2024-02-23T00:46:55.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.739+0000 7f39b259c700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f39b41291a0 0x7f39b4129620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:46:55.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.739+0000 7f39ab7fe700 10 client.41168.objecter ms_handle_connect 0x7f39b41291a0 2024-02-23T00:46:55.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.739+0000 7f39b259c700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f39b41291a0 0x7f39b4129620 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:46:55.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.740+0000 7f39ab7fe700 1 -- 172.21.15.47:0/1108403066 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f39b4129bc0 con 0x7f39b41291a0 2024-02-23T00:46:55.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.740+0000 7f39ab7fe700 10 client.41168.objecter ms_dispatch 0x7f39b4001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:46:55.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.740+0000 7f39ab7fe700 10 client.41168.objecter _finish_command 15 = system:0 2024-02-23T00:46:55.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.740+0000 7f39ab7fe700 20 client.41168.objecter put_session s=0x7f39b412ab90 osd=7 4 2024-02-23T00:46:55.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.740+0000 7f39ab7fe700 15 client.41168.objecter _session_command_op_remove 7 15 2024-02-23T00:46:55.754 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.755+0000 7f39b8deb700 10 client.41168.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:46:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.755+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_assign -1 16 2024-02-23T00:46:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.755+0000 7f39b8deb700 20 client.41168.objecter _get_session s=0x7f39b412ab90 osd=7 3 2024-02-23T00:46:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.755+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b412ab90 osd=7 3 2024-02-23T00:46:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.755+0000 7f39b8deb700 20 client.41168.objecter _get_session s=0x7f39b412ab90 osd=7 3 2024-02-23T00:46:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.755+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_remove -1 16 2024-02-23T00:46:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.755+0000 7f39b8deb700 20 client.41168.objecter get_session s=0x7f39b412ab90 osd=7 3 2024-02-23T00:46:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.755+0000 7f39b8deb700 15 client.41168.objecter _session_command_op_assign 7 16 2024-02-23T00:46:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.755+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b412ab90 osd=7 4 2024-02-23T00:46:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.755+0000 7f39b8deb700 10 client.41168.objecter _send_command 16 2024-02-23T00:46:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.755+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f39b4129bc0 con 0x7f39b41291a0 2024-02-23T00:46:55.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.756+0000 7f39ab7fe700 1 -- 172.21.15.47:0/1108403066 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f39b4129bc0 con 0x7f39b41291a0 2024-02-23T00:46:55.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39ab7fe700 10 client.41168.objecter ms_dispatch 0x7f39b4001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:46:55.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39ab7fe700 10 client.41168.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:46:55.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39ab7fe700 20 client.41168.objecter put_session s=0x7f39b412ab90 osd=7 4 2024-02-23T00:46:55.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39ab7fe700 15 client.41168.objecter _session_command_op_remove 7 16 2024-02-23T00:46:55.759 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:46:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 10 librados: watch_flush enter 2024-02-23T00:46:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 10 librados: watch_flush exit 2024-02-23T00:46:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 10 client.41168.objecter close_session for osd.0 2024-02-23T00:46:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f39b40fdea0 msgr2=0x7f39b40fe280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f39b40fdea0 0x7f39b40fe280 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b405ee80 osd=0 1 2024-02-23T00:46:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 10 client.41168.objecter close_session for osd.1 2024-02-23T00:46:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f39b40f4be0 msgr2=0x7f39b40273c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f39b40f4be0 0x7f39b40273c0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b40f4a70 osd=1 1 2024-02-23T00:46:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 10 client.41168.objecter close_session for osd.2 2024-02-23T00:46:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f39b4061160 msgr2=0x7f39b4119c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f39b4061160 0x7f39b4119c30 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b4060ff0 osd=2 1 2024-02-23T00:46:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 10 client.41168.objecter close_session for osd.3 2024-02-23T00:46:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f39b4059e90 msgr2=0x7f39b411b2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f39b4059e90 0x7f39b411b2e0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b40280d0 osd=3 1 2024-02-23T00:46:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 10 client.41168.objecter close_session for osd.4 2024-02-23T00:46:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f39b411ea90 msgr2=0x7f39b4120f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f39b411ea90 0x7f39b4120f00 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b411bbc0 osd=4 1 2024-02-23T00:46:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 10 client.41168.objecter close_session for osd.5 2024-02-23T00:46:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f39b4121900 msgr2=0x7f39b4121d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f39b4121900 0x7f39b4121d60 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b411c2d0 osd=5 1 2024-02-23T00:46:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 10 client.41168.objecter close_session for osd.6 2024-02-23T00:46:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f39b412a5f0 msgr2=0x7f39b41287a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f39b412a5f0 0x7f39b41287a0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b4129f70 osd=6 1 2024-02-23T00:46:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 10 client.41168.objecter close_session for osd.7 2024-02-23T00:46:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f39b41291a0 msgr2=0x7f39b4129620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:55.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f39b41291a0 0x7f39b4129620 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:55.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 20 client.41168.objecter put_session s=0x7f39b412ab90 osd=7 1 2024-02-23T00:46:55.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 20 client.41168.objecter shutdown clearing up homeless session... 2024-02-23T00:46:55.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 10 client.41168.objecter successfully canceled tick 2024-02-23T00:46:55.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b2d9d700 1 -- 172.21.15.47:0/1108403066 reap_dead start 2024-02-23T00:46:55.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b2d9d700 1 -- 172.21.15.47:0/1108403066 reap_dead start 2024-02-23T00:46:55.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b2d9d700 1 -- 172.21.15.47:0/1108403066 reap_dead start 2024-02-23T00:46:55.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b2d9d700 1 -- 172.21.15.47:0/1108403066 reap_dead start 2024-02-23T00:46:55.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b2d9d700 1 -- 172.21.15.47:0/1108403066 reap_dead start 2024-02-23T00:46:55.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f39a0062cc0 msgr2=0x7f39a0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:55.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f39a0062cc0 0x7f39a0065180 secure :-1 s=READY pgs=3034 cs=0 l=1 rev1=1 rx=0x7f39b41084f0 tx=0x7f399c00b040).stop 2024-02-23T00:46:55.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39b40f7400 msgr2=0x7f39b410cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:46:55.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39b40f7400 0x7f39b410cbb0 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 rx=0x7f39a400c7c0 tx=0x7f39a4007ef0).stop 2024-02-23T00:46:55.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 shutdown_connections 2024-02-23T00:46:55.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39b40f7400 0x7f39b410cbb0 unknown :-1 s=CLOSED pgs=2631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:55.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 1 --2- 172.21.15.47:0/1108403066 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f39a0062cc0 0x7f39a0065180 unknown :-1 s=CLOSED pgs=3034 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:46:55.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.757+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 >> 172.21.15.47:0/1108403066 conn(0x7f39b40f2820 msgr2=0x7f39b40f4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:46:55.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.758+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 shutdown_connections 2024-02-23T00:46:55.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.758+0000 7f39b8deb700 1 -- 172.21.15.47:0/1108403066 wait complete. 2024-02-23T00:46:55.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:46:55.758+0000 7f39b8deb700 1 librados: shutdown 2024-02-23T00:46:55.765 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:46:55.765 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:46:55.765 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:46:55.765 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:46:55.766 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:46:55.766 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:46:55.766 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:46:55.766 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:46:55.883 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:46:55.884 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "filestore_inject_stall = '3' (not observed, change may require restart) " 2024-02-23T00:46:55.884 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:46:56.173 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:55.158277+0000 mon.a (mon.0) 2229 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi047-82419-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi047-82419-2"}]': finished 2024-02-23T00:46:56.173 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:55.158394+0000 mon.a (mon.0) 2230 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi047-82386-57"}]': finished 2024-02-23T00:46:56.173 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:55.158466+0000 mon.a (mon.0) 2231 : cluster [DBG] osdmap e343: 8 total, 8 up, 8 in 2024-02-23T00:46:56.173 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:55.188929+0000 mon.c (mon.2) 262 : audit [INF] from='client.? 172.21.15.47:0/3507985757' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi047-82386-58"}]: dispatch 2024-02-23T00:46:56.173 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:55.189395+0000 mon.a (mon.0) 2232 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi047-82386-58"}]: dispatch 2024-02-23T00:46:56.173 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:55.190063+0000 mon.c (mon.2) 263 : audit [INF] from='client.? 172.21.15.47:0/3507985757' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi047-82386-58"}]: dispatch 2024-02-23T00:46:56.173 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:55.190439+0000 mon.a (mon.0) 2233 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi047-82386-58"}]: dispatch 2024-02-23T00:46:56.173 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:55.191009+0000 mon.c (mon.2) 264 : audit [INF] from='client.? 172.21.15.47:0/3507985757' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi047-82386-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:56.174 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:55.191390+0000 mon.a (mon.0) 2234 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi047-82386-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:56.174 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:55.265690+0000 mgr.y (mgr.14517) 331 : cluster [DBG] pgmap v490: 226 pgs: 8 unknown, 218 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:56.174 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:55.613119+0000 mon.c (mon.2) 265 : audit [DBG] from='client.? 172.21.15.47:0/1108403066' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:46:56.174 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:55.665897+0000 mon.a (mon.0) 2235 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:46:56.174 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:55.666306+0000 mon.a (mon.0) 2236 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-43"}]: dispatch 2024-02-23T00:46:56.174 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:55.850138+0000 osd.4 (osd.4) 145 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:46:56.174 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:55.850278+0000 osd.4 (osd.4) 146 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:46:56.204 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:46:56.206 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:56.207 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:46:56.221 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:ntry=6 expected=6 2024-02-23T00:46:56.222 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : seek to 15:bd63b0f1:::8:head 2024-02-23T00:46:56.222 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : cursor()=15:bd63b0f1:::8:head expected=15:bd63b0f1:::8:head 2024-02-23T00:46:56.222 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > 15:bd63b0f1:::8:head -> 8 2024-02-23T00:46:56.222 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : entry=8 expected=8 2024-02-23T00:46:56.222 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : seek to 15:b29083e3:::5:head 2024-02-23T00:46:56.222 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : cursor()=15:b29083e3:::5:head expected=15:b29083e3:::5:head 2024-02-23T00:46:56.222 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > 15:b29083e3:::5:head -> 5 2024-02-23T00:46:56.222 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : entry=5 expected=5 2024-02-23T00:46:56.222 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : seek to 15:89d3ae78:::11:head 2024-02-23T00:46:56.222 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : cursor()=15:89d3ae78:::11:head expected=15:89d3ae78:::11:head 2024-02-23T00:46:56.222 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > 15:89d3ae78:::11:head -> 11 2024-02-23T00:46:56.223 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : entry=11 expected=11 2024-02-23T00:46:56.223 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : seek to 15:863748b0:::15:head 2024-02-23T00:46:56.223 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : cursor()=15:863748b0:::15:head expected=15:863748b0:::15:head 2024-02-23T00:46:56.223 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > 15:863748b0:::15:head -> 15 2024-02-23T00:46:56.223 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : entry=15 expected=15 2024-02-23T00:46:56.223 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : seek to 15:6cac518f:::0:head 2024-02-23T00:46:56.223 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : cursor()=15:6cac518f:::0:head expected=15:6cac518f:::0:head 2024-02-23T00:46:56.223 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > 15:6cac518f:::0:head -> 0 2024-02-23T00:46:56.223 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : entry=0 expected=0 2024-02-23T00:46:56.223 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : seek to 15:52ea6a34:::10:head 2024-02-23T00:46:56.223 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : cursor()=15:52ea6a34:::10:head expected=15:52ea6a34:::10:head 2024-02-23T00:46:56.223 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > 15:52ea6a34:::10:head -> 10 2024-02-23T00:46:56.223 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : entry=10 expected=10 2024-02-23T00:46:56.224 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : seek to 15:5c6b0b28:::7:head 2024-02-23T00:46:56.224 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : cursor()=15:5c6b0b28:::7:head expected=15:5c6b0b28:::7:head 2024-02-23T00:46:56.224 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > 15:5c6b0b28:::7:head -> 7 2024-02-23T00:46:56.224 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : entry=7 expected=7 2024-02-23T00:46:56.224 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : seek to 15:02547ec2:::1:head 2024-02-23T00:46:56.224 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : cursor()=15:02547ec2:::1:head expected=15:02547ec2:::1:head 2024-02-23T00:46:56.224 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: > 15:02547ec2:::1:head -> 1 2024-02-23T00:46:56.224 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: : entry=1 expected=1 2024-02-23T00:46:56.224 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (312 ms) 2024-02-23T00:46:56.224 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2024-02-23T00:46:56.224 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (199790 ms) 2024-02-23T00:46:56.224 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2024-02-23T00:46:56.224 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: split 0/5 -> MIN 15:33333333::::head 2024-02-23T00:46:56.225 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: split 1/5 -> 15:33333333::::head 15:66666666::::head 2024-02-23T00:46:56.225 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: split 2/5 -> 15:66666666::::head 15:99999999::::head 2024-02-23T00:46:56.225 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: split 3/5 -> 15:99999999::::head 15:cccccccc::::head 2024-02-23T00:46:56.225 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: split 4/5 -> 15:cccccccc::::head MAX 2024-02-23T00:46:56.225 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (31994 ms) 2024-02-23T00:46:56.225 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [----------] 7 tests from LibRadosList (232605 ms total) 2024-02-23T00:46:56.225 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 2024-02-23T00:46:56.225 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [----------] 3 tests from LibRadosListEC 2024-02-23T00:46:56.225 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2024-02-23T00:46:56.225 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (1022 ms) 2024-02-23T00:46:56.225 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2024-02-23T00:46:56.225 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: myset foo1,foo2,foo3 2024-02-23T00:46:56.225 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: foo1 2024-02-23T00:46:56.225 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: foo2 2024-02-23T00:46:56.226 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: foo3 2024-02-23T00:46:56.226 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: myset foo1,foo4,foo5 2024-02-23T00:46:56.226 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: foo4 2024-02-23T00:46:56.226 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: foo5 2024-02-23T00:46:56.226 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: foo1 2024-02-23T00:46:56.226 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: myset foo6,foo7 2024-02-23T00:46:56.226 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: foo7 2024-02-23T00:46:56.226 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: foo6 2024-02-23T00:46:56.226 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-02-23T00:46:56.226 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: ns1:foo4 2024-02-23T00:46:56.226 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: ns1:foo5 2024-02-23T00:46:56.226 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: ns2:foo7 2024-02-23T00:46:56.226 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: ns2:foo6 2024-02-23T00:46:56.227 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: ns1:foo1 2024-02-23T00:46:56.227 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: :foo1 2024-02-23T00:46:56.227 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: :foo2 2024-02-23T00:46:56.227 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: :foo3 2024-02-23T00:46:56.227 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (36 ms) 2024-02-23T00:46:56.307 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:46:56.336 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:56.533 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:46:56.536 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:56.536 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:46:56.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:55.158277+0000 mon.a (mon.0) 2229 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi047-82419-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi047-82419-2"}]': finished 2024-02-23T00:46:56.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:55.158394+0000 mon.a (mon.0) 2230 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi047-82386-57"}]': finished 2024-02-23T00:46:56.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:55.158466+0000 mon.a (mon.0) 2231 : cluster [DBG] osdmap e343: 8 total, 8 up, 8 in 2024-02-23T00:46:56.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:55.188929+0000 mon.c (mon.2) 262 : audit [INF] from='client.? 172.21.15.47:0/3507985757' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi047-82386-58"}]: dispatch 2024-02-23T00:46:56.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:55.189395+0000 mon.a (mon.0) 2232 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi047-82386-58"}]: dispatch 2024-02-23T00:46:56.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:55.190063+0000 mon.c (mon.2) 263 : audit [INF] from='client.? 172.21.15.47:0/3507985757' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi047-82386-58"}]: dispatch 2024-02-23T00:46:56.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:55.190439+0000 mon.a (mon.0) 2233 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi047-82386-58"}]: dispatch 2024-02-23T00:46:56.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:55.191009+0000 mon.c (mon.2) 264 : audit [INF] from='client.? 172.21.15.47:0/3507985757' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi047-82386-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:56.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:55.191390+0000 mon.a (mon.0) 2234 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi047-82386-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:56.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:55.265690+0000 mgr.y (mgr.14517) 331 : cluster [DBG] pgmap v490: 226 pgs: 8 unknown, 218 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:56.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:55.613119+0000 mon.c (mon.2) 265 : audit [DBG] from='client.? 172.21.15.47:0/1108403066' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:46:56.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:55.665897+0000 mon.a (mon.0) 2235 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:46:56.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:55.158277+0000 mon.a (mon.0) 2229 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi047-82419-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi047-82419-2"}]': finished 2024-02-23T00:46:56.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:55.158394+0000 mon.a (mon.0) 2230 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi047-82386-57"}]': finished 2024-02-23T00:46:56.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:55.158466+0000 mon.a (mon.0) 2231 : cluster [DBG] osdmap e343: 8 total, 8 up, 8 in 2024-02-23T00:46:56.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:55.188929+0000 mon.c (mon.2) 262 : audit [INF] from='client.? 172.21.15.47:0/3507985757' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi047-82386-58"}]: dispatch 2024-02-23T00:46:56.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:55.189395+0000 mon.a (mon.0) 2232 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi047-82386-58"}]: dispatch 2024-02-23T00:46:56.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:55.190063+0000 mon.c (mon.2) 263 : audit [INF] from='client.? 172.21.15.47:0/3507985757' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi047-82386-58"}]: dispatch 2024-02-23T00:46:56.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:55.190439+0000 mon.a (mon.0) 2233 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi047-82386-58"}]: dispatch 2024-02-23T00:46:56.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:55.191009+0000 mon.c (mon.2) 264 : audit [INF] from='client.? 172.21.15.47:0/3507985757' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi047-82386-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:56.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:55.191390+0000 mon.a (mon.0) 2234 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi047-82386-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:46:56.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:55.265690+0000 mgr.y (mgr.14517) 331 : cluster [DBG] pgmap v490: 226 pgs: 8 unknown, 218 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:56.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:55.613119+0000 mon.c (mon.2) 265 : audit [DBG] from='client.? 172.21.15.47:0/1108403066' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:46:56.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:55.665897+0000 mon.a (mon.0) 2235 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:46:56.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:55.666306+0000 mon.a (mon.0) 2236 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-43"}]: dispatch 2024-02-23T00:46:56.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:55.850138+0000 osd.4 (osd.4) 145 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:46:56.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:55.850278+0000 osd.4 (osd.4) 146 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:46:56.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:55.666306+0000 mon.a (mon.0) 2236 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-43"}]: dispatch 2024-02-23T00:46:56.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:55.850138+0000 osd.4 (osd.4) 145 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:46:56.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:55.850278+0000 osd.4 (osd.4) 146 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:46:56.637 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:46:56.667 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:56.768 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:46:56.768 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:46:56.771 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:46:56.774 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:56.775 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:46:56.876 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:46:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:56.933 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:46:56.936 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:56.936 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:46:57.037 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:46:57.068 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:57.547 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.5 dump_historic_ops 2024-02-23T00:46:57.622 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:46:57.632 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:57.632 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:46:57.733 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:46:57.758 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:56.160292+0000 mon.a (mon.0) 2237 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi047-82386-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:57.759 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:56.160392+0000 mon.a (mon.0) 2238 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-43"}]': finished 2024-02-23T00:46:57.759 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:56.160449+0000 mon.a (mon.0) 2239 : cluster [DBG] osdmap e344: 8 total, 8 up, 8 in 2024-02-23T00:46:57.759 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:56.163315+0000 mon.c (mon.2) 266 : audit [INF] from='client.? 172.21.15.47:0/3507985757' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi047-82386-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi047-82386-58"}]: dispatch 2024-02-23T00:46:57.759 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:56.164138+0000 mon.a (mon.0) 2240 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi047-82386-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi047-82386-58"}]: dispatch 2024-02-23T00:46:57.759 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:56.160292+0000 mon.a (mon.0) 2237 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi047-82386-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:57.759 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:56.160392+0000 mon.a (mon.0) 2238 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-43"}]': finished 2024-02-23T00:46:57.759 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:56.160449+0000 mon.a (mon.0) 2239 : cluster [DBG] osdmap e344: 8 total, 8 up, 8 in 2024-02-23T00:46:57.759 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:56.163315+0000 mon.c (mon.2) 266 : audit [INF] from='client.? 172.21.15.47:0/3507985757' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi047-82386-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi047-82386-58"}]: dispatch 2024-02-23T00:46:57.759 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:56.164138+0000 mon.a (mon.0) 2240 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi047-82386-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi047-82386-58"}]: dispatch 2024-02-23T00:46:57.760 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:57.816 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:46:57.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:56.160292+0000 mon.a (mon.0) 2237 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi047-82386-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:46:57.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:56.160392+0000 mon.a (mon.0) 2238 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-43"}]': finished 2024-02-23T00:46:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:56.160449+0000 mon.a (mon.0) 2239 : cluster [DBG] osdmap e344: 8 total, 8 up, 8 in 2024-02-23T00:46:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:56.163315+0000 mon.c (mon.2) 266 : audit [INF] from='client.? 172.21.15.47:0/3507985757' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi047-82386-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi047-82386-58"}]: dispatch 2024-02-23T00:46:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:56.164138+0000 mon.a (mon.0) 2240 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi047-82386-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi047-82386-58"}]: dispatch 2024-02-23T00:46:58.010 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:46:58.013 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:58.013 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:46:58.114 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:46:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:58.161 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:46:58.164 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:58.165 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:46:58.265 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:46:58.293 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:58.312 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:46:58.314 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:58.315 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:46:58.416 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:46:58.446 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:58.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:57.266405+0000 mgr.y (mgr.14517) 332 : cluster [DBG] pgmap v493: 186 pgs: 186 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:58.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:57.595414+0000 mon.a (mon.0) 2241 : cluster [DBG] osdmap e345: 8 total, 8 up, 8 in 2024-02-23T00:46:58.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:57.596255+0000 mon.c (mon.2) 267 : audit [INF] from='client.? 172.21.15.47:0/3949512893' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi047-82419-2"}]: dispatch 2024-02-23T00:46:58.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:57.597473+0000 mon.a (mon.0) 2242 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi047-82419-2"}]: dispatch 2024-02-23T00:46:58.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:58.166921+0000 mon.a (mon.0) 2243 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi047-82386-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi047-82386-58"}]': finished 2024-02-23T00:46:58.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:58.167037+0000 mon.a (mon.0) 2244 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi047-82419-2"}]': finished 2024-02-23T00:46:58.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:58.167129+0000 mon.a (mon.0) 2245 : cluster [DBG] osdmap e346: 8 total, 8 up, 8 in 2024-02-23T00:46:58.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:58.168302+0000 mon.c (mon.2) 268 : audit [INF] from='client.? 172.21.15.47:0/3949512893' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi047-82419-2"}]: dispatch 2024-02-23T00:46:58.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:58.169134+0000 mon.a (mon.0) 2246 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi047-82419-2"}]: dispatch 2024-02-23T00:46:58.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:46:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:58.172628+0000 mon.a (mon.0) 2247 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:46:58.795 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:46:58.798 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:58.798 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:46:58.899 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:46:58.903 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-23T00:46:58.927 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:58.933 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:57.266405+0000 mgr.y (mgr.14517) 332 : cluster [DBG] pgmap v493: 186 pgs: 186 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:58.933 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:57.595414+0000 mon.a (mon.0) 2241 : cluster [DBG] osdmap e345: 8 total, 8 up, 8 in 2024-02-23T00:46:58.933 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:57.596255+0000 mon.c (mon.2) 267 : audit [INF] from='client.? 172.21.15.47:0/3949512893' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi047-82419-2"}]: dispatch 2024-02-23T00:46:58.933 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:57.597473+0000 mon.a (mon.0) 2242 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi047-82419-2"}]: dispatch 2024-02-23T00:46:58.934 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:58.166921+0000 mon.a (mon.0) 2243 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi047-82386-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi047-82386-58"}]': finished 2024-02-23T00:46:58.934 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:58.167037+0000 mon.a (mon.0) 2244 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi047-82419-2"}]': finished 2024-02-23T00:46:58.934 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:58.167129+0000 mon.a (mon.0) 2245 : cluster [DBG] osdmap e346: 8 total, 8 up, 8 in 2024-02-23T00:46:58.934 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:58.168302+0000 mon.c (mon.2) 268 : audit [INF] from='client.? 172.21.15.47:0/3949512893' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi047-82419-2"}]: dispatch 2024-02-23T00:46:58.934 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:58.169134+0000 mon.a (mon.0) 2246 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi047-82419-2"}]: dispatch 2024-02-23T00:46:58.934 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:46:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:58.172628+0000 mon.a (mon.0) 2247 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:46:58.934 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:57.266405+0000 mgr.y (mgr.14517) 332 : cluster [DBG] pgmap v493: 186 pgs: 186 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:46:58.934 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:57.595414+0000 mon.a (mon.0) 2241 : cluster [DBG] osdmap e345: 8 total, 8 up, 8 in 2024-02-23T00:46:58.934 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:57.596255+0000 mon.c (mon.2) 267 : audit [INF] from='client.? 172.21.15.47:0/3949512893' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi047-82419-2"}]: dispatch 2024-02-23T00:46:58.934 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:57.597473+0000 mon.a (mon.0) 2242 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi047-82419-2"}]: dispatch 2024-02-23T00:46:58.935 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:58.166921+0000 mon.a (mon.0) 2243 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi047-82386-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi047-82386-58"}]': finished 2024-02-23T00:46:58.935 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:58.167037+0000 mon.a (mon.0) 2244 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi047-82419-2"}]': finished 2024-02-23T00:46:58.935 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:58.167129+0000 mon.a (mon.0) 2245 : cluster [DBG] osdmap e346: 8 total, 8 up, 8 in 2024-02-23T00:46:58.935 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:58.168302+0000 mon.c (mon.2) 268 : audit [INF] from='client.? 172.21.15.47:0/3949512893' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi047-82419-2"}]: dispatch 2024-02-23T00:46:58.935 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:58.169134+0000 mon.a (mon.0) 2246 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi047-82419-2"}]: dispatch 2024-02-23T00:46:58.935 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:46:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:58.172628+0000 mon.a (mon.0) 2247 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:46:59.201 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:46:59.276 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:46:59.278 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:59.278 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:46:59.379 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:46:59.413 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:59.512 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:46:59.517 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:59.518 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:46:59.618 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:46:59.646 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:46:59.794 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:46:59.797 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:46:59.798 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:46:59.898 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:46:59.905 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:46:59.906 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:46:59.928 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:00.006 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:47:00.126 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:47:00.129 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:00.129 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:47:00.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:59.168475+0000 mon.a (mon.0) 2248 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi047-82419-2"}]': finished 2024-02-23T00:47:00.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:59.168570+0000 mon.a (mon.0) 2249 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:47:00.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:59.168679+0000 mon.a (mon.0) 2250 : cluster [DBG] osdmap e347: 8 total, 8 up, 8 in 2024-02-23T00:47:00.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:46:59.266767+0000 mgr.y (mgr.14517) 333 : cluster [DBG] pgmap v496: 226 pgs: 40 unknown, 186 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:47:00.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:46:59.655776+0000 mon.a (mon.0) 2251 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:47:00.230 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:47:00.258 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:00.356 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:47:00.372 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:00.372 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:47:00.455 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:59.168475+0000 mon.a (mon.0) 2248 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi047-82419-2"}]': finished 2024-02-23T00:47:00.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:59.168570+0000 mon.a (mon.0) 2249 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:47:00.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:59.168679+0000 mon.a (mon.0) 2250 : cluster [DBG] osdmap e347: 8 total, 8 up, 8 in 2024-02-23T00:47:00.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:46:59.266767+0000 mgr.y (mgr.14517) 333 : cluster [DBG] pgmap v496: 226 pgs: 40 unknown, 186 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:47:00.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:46:59.655776+0000 mon.a (mon.0) 2251 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:47:00.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:59.168475+0000 mon.a (mon.0) 2248 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi047-82419-2"}]': finished 2024-02-23T00:47:00.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:59.168570+0000 mon.a (mon.0) 2249 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:47:00.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:59.168679+0000 mon.a (mon.0) 2250 : cluster [DBG] osdmap e347: 8 total, 8 up, 8 in 2024-02-23T00:47:00.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:46:59.266767+0000 mgr.y (mgr.14517) 333 : cluster [DBG] pgmap v496: 226 pgs: 40 unknown, 186 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:47:00.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:46:59.655776+0000 mon.a (mon.0) 2251 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:47:00.473 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:47:00.499 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:00.679 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.6 dump_ops_in_flight 2024-02-23T00:47:00.698 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:47:00.700 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:00.700 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:47:00.755 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:47:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:47:00] "GET /metrics HTTP/1.1" 200 190742 "" "Prometheus/2.33.4" 2024-02-23T00:47:00.801 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:47:00.827 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:00.848 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:47:00.851 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:00.852 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:47:00.952 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:47:00.969 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:47:00.984 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:01.085 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:47:01.091 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:01.091 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:47:01.192 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:47:01.220 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:01.243 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:47:01.246 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:01.246 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:47:01.347 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:47:01.376 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:00.175942+0000 mon.a (mon.0) 2252 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:47:01.376 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:00.176024+0000 mon.a (mon.0) 2253 : cluster [DBG] osdmap e348: 8 total, 8 up, 8 in 2024-02-23T00:47:01.376 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:00.176921+0000 mon.a (mon.0) 2254 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-45"}]: dispatch 2024-02-23T00:47:01.376 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:00.183943+0000 mon.c (mon.2) 269 : audit [INF] from='client.? 172.21.15.47:0/3507985757' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi047-82386-58"}]: dispatch 2024-02-23T00:47:01.376 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:00.184394+0000 mon.a (mon.0) 2255 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi047-82386-58"}]: dispatch 2024-02-23T00:47:01.376 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:00.186582+0000 mon.b (mon.1) 374 : audit [INF] from='client.? 172.21.15.47:0/2489350951' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82419-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:47:01.376 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:00.187284+0000 mon.a (mon.0) 2256 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82419-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:47:01.377 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:00.175942+0000 mon.a (mon.0) 2252 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:47:01.377 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:00.176024+0000 mon.a (mon.0) 2253 : cluster [DBG] osdmap e348: 8 total, 8 up, 8 in 2024-02-23T00:47:01.377 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:00.176921+0000 mon.a (mon.0) 2254 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-45"}]: dispatch 2024-02-23T00:47:01.377 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:00.183943+0000 mon.c (mon.2) 269 : audit [INF] from='client.? 172.21.15.47:0/3507985757' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi047-82386-58"}]: dispatch 2024-02-23T00:47:01.377 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:00.184394+0000 mon.a (mon.0) 2255 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi047-82386-58"}]: dispatch 2024-02-23T00:47:01.377 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:00.186582+0000 mon.b (mon.1) 374 : audit [INF] from='client.? 172.21.15.47:0/2489350951' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82419-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:47:01.377 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:00.187284+0000 mon.a (mon.0) 2256 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82419-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:47:01.378 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:01.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.390+0000 7fdc51657700 1 -- 172.21.15.47:0/4050394831 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc4c0f7400 msgr2=0x7fdc4c103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:01.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.390+0000 7fdc51657700 1 --2- 172.21.15.47:0/4050394831 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc4c0f7400 0x7fdc4c103a20 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 rx=0x7fdc44009be0 tx=0x7fdc44004d10).stop 2024-02-23T00:47:01.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.390+0000 7fdc51657700 1 -- 172.21.15.47:0/4050394831 shutdown_connections 2024-02-23T00:47:01.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.390+0000 7fdc51657700 1 --2- 172.21.15.47:0/4050394831 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc4c0f7400 0x7fdc4c103a20 unknown :-1 s=CLOSED pgs=2632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:01.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.390+0000 7fdc51657700 1 --2- 172.21.15.47:0/4050394831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc4c0fcbb0 0x7fdc4c0fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:01.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.390+0000 7fdc51657700 1 --2- 172.21.15.47:0/4050394831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc4c0fd510 0x7fdc4c0f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:01.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.390+0000 7fdc51657700 1 -- 172.21.15.47:0/4050394831 >> 172.21.15.47:0/4050394831 conn(0x7fdc4c0f2820 msgr2=0x7fdc4c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:01.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.390+0000 7fdc51657700 1 -- 172.21.15.47:0/4050394831 shutdown_connections 2024-02-23T00:47:01.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.390+0000 7fdc51657700 1 -- 172.21.15.47:0/4050394831 wait complete. 2024-02-23T00:47:01.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.391+0000 7fdc51657700 1 librados: starting msgr at 2024-02-23T00:47:01.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.391+0000 7fdc51657700 1 librados: starting objecter 2024-02-23T00:47:01.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.391+0000 7fdc51657700 1 Processor -- start 2024-02-23T00:47:01.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.391+0000 7fdc51657700 1 -- start start 2024-02-23T00:47:01.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.391+0000 7fdc51657700 1 librados: setting wanted keys 2024-02-23T00:47:01.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.391+0000 7fdc51657700 1 librados: calling monclient init 2024-02-23T00:47:01.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.391+0000 7fdc51657700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc4c0f7400 0x7fdc4c10de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:01.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.391+0000 7fdc51657700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc4c0fcbb0 0x7fdc4c10e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:01.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.391+0000 7fdc51657700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc4c0fd510 0x7fdc4c108070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:01.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.391+0000 7fdc51657700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c108560 con 0x7fdc4c0fd510 2024-02-23T00:47:01.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.391+0000 7fdc51657700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c1086d0 con 0x7fdc4c0f7400 2024-02-23T00:47:01.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.391+0000 7fdc51657700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c108840 con 0x7fdc4c0fcbb0 2024-02-23T00:47:01.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.391+0000 7fdc4a7fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc4c0fcbb0 0x7fdc4c10e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:01.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.392+0000 7fdc4affd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc4c0f7400 0x7fdc4c10de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:01.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.392+0000 7fdc4affd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc4c0f7400 0x7fdc4c10de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.47:45026/0 (socket says 172.21.15.47:45026) 2024-02-23T00:47:01.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.392+0000 7fdc4a7fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc4c0fcbb0 0x7fdc4c10e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:38150/0 (socket says 172.21.15.47:38150) 2024-02-23T00:47:01.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.392+0000 7fdc4affd700 1 -- 172.21.15.47:0/3830340996 learned_addr learned my addr 172.21.15.47:0/3830340996 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:47:01.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.392+0000 7fdc4b7fe700 1 --2- 172.21.15.47:0/3830340996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc4c0fd510 0x7fdc4c108070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:01.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.392+0000 7fdc4affd700 1 -- 172.21.15.47:0/3830340996 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc4c0fcbb0 msgr2=0x7fdc4c10e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:01.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.392+0000 7fdc4affd700 1 --2- 172.21.15.47:0/3830340996 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc4c0fcbb0 0x7fdc4c10e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:01.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.392+0000 7fdc4affd700 1 -- 172.21.15.47:0/3830340996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc4c0fd510 msgr2=0x7fdc4c108070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:01.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.392+0000 7fdc4affd700 1 --2- 172.21.15.47:0/3830340996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc4c0fd510 0x7fdc4c108070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:01.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.392+0000 7fdc4affd700 1 -- 172.21.15.47:0/3830340996 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc440097c0 con 0x7fdc4c0f7400 2024-02-23T00:47:01.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.392+0000 7fdc4b7fe700 1 --2- 172.21.15.47:0/3830340996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc4c0fd510 0x7fdc4c108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:47:01.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.392+0000 7fdc4affd700 1 --2- 172.21.15.47:0/3830340996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc4c0f7400 0x7fdc4c10de90 secure :-1 s=READY pgs=3973 cs=0 l=1 rev1=1 rx=0x7fdc3c00e990 tx=0x7fdc3c00b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:01.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.392+0000 7fdc33fff700 10 client.?.objecter ms_handle_connect 0x7fdc4c0f7400 2024-02-23T00:47:01.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.392+0000 7fdc33fff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:47:01.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.392+0000 7fdc33fff700 1 -- 172.21.15.47:0/3830340996 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c00ee80 con 0x7fdc4c0f7400 2024-02-23T00:47:01.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.392+0000 7fdc51657700 1 -- 172.21.15.47:0/3830340996 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc4c108ac0 con 0x7fdc4c0f7400 2024-02-23T00:47:01.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.392+0000 7fdc33fff700 1 -- 172.21.15.47:0/3830340996 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fdc3c00f040 con 0x7fdc4c0f7400 2024-02-23T00:47:01.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.392+0000 7fdc33fff700 1 -- 172.21.15.47:0/3830340996 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c00c840 con 0x7fdc4c0f7400 2024-02-23T00:47:01.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.392+0000 7fdc51657700 10 client.43231.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:47:01.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.392+0000 7fdc51657700 1 -- 172.21.15.47:0/3830340996 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc4c114d00 con 0x7fdc4c0f7400 2024-02-23T00:47:01.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.392+0000 7fdc51657700 1 librados: init done 2024-02-23T00:47:01.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.393+0000 7fdc51657700 1 -- 172.21.15.47:0/3830340996 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc4c05fa60 con 0x7fdc4c0f7400 2024-02-23T00:47:01.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.394+0000 7fdc33fff700 1 -- 172.21.15.47:0/3830340996 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fdc3c00cc70 con 0x7fdc4c0f7400 2024-02-23T00:47:01.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.394+0000 7fdc33fff700 1 --2- 172.21.15.47:0/3830340996 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdc34062cc0 0x7fdc34065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:01.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.394+0000 7fdc33fff700 1 -- 172.21.15.47:0/3830340996 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(349..349 src has 48..349) v4 ==== 9682+0+0 (secure 0 0 0) 0x7fdc3c07d460 con 0x7fdc4c0f7400 2024-02-23T00:47:01.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.394+0000 7fdc33fff700 10 client.43231.objecter ms_dispatch 0x7fdc4c001550 osd_map(349..349 src has 48..349) v4 2024-02-23T00:47:01.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.394+0000 7fdc33fff700 3 client.43231.objecter handle_osd_map got epochs [349,349] > 0 2024-02-23T00:47:01.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.394+0000 7fdc33fff700 3 client.43231.objecter handle_osd_map decoding full epoch 349 2024-02-23T00:47:01.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.394+0000 7fdc4a7fc700 1 --2- 172.21.15.47:0/3830340996 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdc34062cc0 0x7fdc34065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:01.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.395+0000 7fdc33fff700 20 client.43231.objecter dump_active .. 0 homeless 2024-02-23T00:47:01.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.395+0000 7fdc33fff700 10 client.43231.objecter ms_handle_connect 0x7fdc34062cc0 2024-02-23T00:47:01.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.395+0000 7fdc4a7fc700 1 --2- 172.21.15.47:0/3830340996 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdc34062cc0 0x7fdc34065180 secure :-1 s=READY pgs=3036 cs=0 l=1 rev1=1 rx=0x7fdc4c109810 tx=0x7fdc4000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:01.399 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:47:01.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.399+0000 7fdc33fff700 1 -- 172.21.15.47:0/3830340996 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fdc3c049f60 con 0x7fdc4c0f7400 2024-02-23T00:47:01.402 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:01.402 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:47:01.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:00.175942+0000 mon.a (mon.0) 2252 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:47:01.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:00.176024+0000 mon.a (mon.0) 2253 : cluster [DBG] osdmap e348: 8 total, 8 up, 8 in 2024-02-23T00:47:01.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:00.176921+0000 mon.a (mon.0) 2254 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-45"}]: dispatch 2024-02-23T00:47:01.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:00.183943+0000 mon.c (mon.2) 269 : audit [INF] from='client.? 172.21.15.47:0/3507985757' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi047-82386-58"}]: dispatch 2024-02-23T00:47:01.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:00.184394+0000 mon.a (mon.0) 2255 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi047-82386-58"}]: dispatch 2024-02-23T00:47:01.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:00.186582+0000 mon.b (mon.1) 374 : audit [INF] from='client.? 172.21.15.47:0/2489350951' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82419-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:47:01.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:00.187284+0000 mon.a (mon.0) 2256 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82419-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:47:01.467 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:47:01.503 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:47:01.532 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:01.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:01.584+0000 7fdc51657700 1 -- 172.21.15.47:0/3830340996 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7fdc4c109550 con 0x7fdc4c0f7400 2024-02-23T00:47:01.779 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:47:01.782 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:01.782 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:47:01.883 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:47:01.911 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:01.963 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:47:01.970 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:01.971 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:47:02.072 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:47:02.102 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:02.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:02.179+0000 7fdc33fff700 1 -- 172.21.15.47:0/3830340996 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v350) v1 ==== 97+0+0 (secure 0 0 0) 0x7fdc3c04c8a0 con 0x7fdc4c0f7400 2024-02-23T00:47:02.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:02.182+0000 7fdc51657700 10 librados: watch_flush enter 2024-02-23T00:47:02.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:02.183+0000 7fdc51657700 10 librados: watch_flush exit 2024-02-23T00:47:02.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:02.183+0000 7fdc51657700 20 client.43231.objecter shutdown clearing up homeless session... 2024-02-23T00:47:02.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:02.183+0000 7fdc51657700 10 client.43231.objecter successfully canceled tick 2024-02-23T00:47:02.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:02.183+0000 7fdc51657700 1 -- 172.21.15.47:0/3830340996 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdc34062cc0 msgr2=0x7fdc34065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:02.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:02.183+0000 7fdc51657700 1 --2- 172.21.15.47:0/3830340996 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdc34062cc0 0x7fdc34065180 secure :-1 s=READY pgs=3036 cs=0 l=1 rev1=1 rx=0x7fdc4c109810 tx=0x7fdc4000b040).stop 2024-02-23T00:47:02.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:02.183+0000 7fdc51657700 1 -- 172.21.15.47:0/3830340996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc4c0f7400 msgr2=0x7fdc4c10de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:02.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:02.183+0000 7fdc51657700 1 --2- 172.21.15.47:0/3830340996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc4c0f7400 0x7fdc4c10de90 secure :-1 s=READY pgs=3973 cs=0 l=1 rev1=1 rx=0x7fdc3c00e990 tx=0x7fdc3c00b040).stop 2024-02-23T00:47:02.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:02.184+0000 7fdc51657700 1 -- 172.21.15.47:0/3830340996 shutdown_connections 2024-02-23T00:47:02.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:02.184+0000 7fdc51657700 1 --2- 172.21.15.47:0/3830340996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc4c0fd510 0x7fdc4c108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:02.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:02.184+0000 7fdc51657700 1 --2- 172.21.15.47:0/3830340996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc4c0f7400 0x7fdc4c10de90 unknown :-1 s=CLOSED pgs=3973 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:02.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:02.184+0000 7fdc51657700 1 --2- 172.21.15.47:0/3830340996 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdc34062cc0 0x7fdc34065180 unknown :-1 s=CLOSED pgs=3036 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:02.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:02.184+0000 7fdc51657700 1 --2- 172.21.15.47:0/3830340996 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc4c0fcbb0 0x7fdc4c10e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:02.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:02.184+0000 7fdc51657700 1 -- 172.21.15.47:0/3830340996 >> 172.21.15.47:0/3830340996 conn(0x7fdc4c0f2820 msgr2=0x7fdc4c060dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:02.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:02.184+0000 7fdc51657700 1 -- 172.21.15.47:0/3830340996 shutdown_connections 2024-02-23T00:47:02.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:02.184+0000 7fdc51657700 1 -- 172.21.15.47:0/3830340996 wait complete. 2024-02-23T00:47:02.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:02.184+0000 7fdc51657700 1 librados: shutdown 2024-02-23T00:47:02.188 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is set 2024-02-23T00:47:02.190 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsStart 2024-02-23T00:47:02.190 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 1 0 2024-02-23T00:47:02.190 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 10 0 2024-02-23T00:47:02.190 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 13 0 2024-02-23T00:47:02.190 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 7 0 2024-02-23T00:47:02.190 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 14 0 2024-02-23T00:47:02.190 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 0 0 2024-02-23T00:47:02.190 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 15 0 2024-02-23T00:47:02.190 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 11 0 2024-02-23T00:47:02.190 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 5 0 2024-02-23T00:47:02.191 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 8 0 2024-02-23T00:47:02.191 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 6 0 2024-02-23T00:47:02.191 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 3 0 2024-02-23T00:47:02.191 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 4 0 2024-02-23T00:47:02.191 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 12 0 2024-02-23T00:47:02.191 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 9 0 2024-02-23T00:47:02.191 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 2 0 2024-02-23T00:47:02.191 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-02-23T00:47:02.191 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (46 ms) 2024-02-23T00:47:02.191 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [----------] 3 tests from LibRadosListEC (1104 ms total) 2024-02-23T00:47:02.191 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 2024-02-23T00:47:02.191 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [----------] 1 test from LibRadosListNP 2024-02-23T00:47:02.191 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2024-02-23T00:47:02.192 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (2997 ms) 2024-02-23T00:47:02.192 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [----------] 1 test from LibRadosListNP (2997 ms total) 2024-02-23T00:47:02.192 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: 2024-02-23T00:47:02.192 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [----------] Global test environment tear-down 2024-02-23T00:47:02.192 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [==========] 11 tests from 3 test suites ran. (254936 ms total) 2024-02-23T00:47:02.192 INFO:tasks.workunit.client.0.smithi047.stdout: api_list: [ PASSED ] 11 tests. 2024-02-23T00:47:02.193 INFO:tasks.workunit.client.0.smithi047.stderr:+ for t in "${!pids[@]}" 2024-02-23T00:47:02.193 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82515 2024-02-23T00:47:02.193 INFO:tasks.workunit.client.0.smithi047.stderr:+ wait 82515 2024-02-23T00:47:02.193 INFO:tasks.workunit.client.0.smithi047.stderr:+ for t in "${!pids[@]}" 2024-02-23T00:47:02.193 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82507 2024-02-23T00:47:02.193 INFO:tasks.workunit.client.0.smithi047.stderr:+ wait 82507 2024-02-23T00:47:02.194 INFO:tasks.workunit.client.0.smithi047.stderr:+ for t in "${!pids[@]}" 2024-02-23T00:47:02.194 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82571 2024-02-23T00:47:02.194 INFO:tasks.workunit.client.0.smithi047.stderr:+ wait 82571 2024-02-23T00:47:02.194 INFO:tasks.workunit.client.0.smithi047.stderr:+ for t in "${!pids[@]}" 2024-02-23T00:47:02.194 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82381 2024-02-23T00:47:02.194 INFO:tasks.workunit.client.0.smithi047.stderr:+ wait 82381 2024-02-23T00:47:02.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:01.176523+0000 mon.a (mon.0) 2257 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-45"}]': finished 2024-02-23T00:47:02.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:01.176647+0000 mon.a (mon.0) 2258 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi047-82386-58"}]': finished 2024-02-23T00:47:02.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:01.176744+0000 mon.a (mon.0) 2259 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82419-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:47:02.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:01.176846+0000 mon.a (mon.0) 2260 : cluster [DBG] osdmap e349: 8 total, 8 up, 8 in 2024-02-23T00:47:02.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:01.179698+0000 mon.c (mon.2) 270 : audit [INF] from='client.? 172.21.15.47:0/3507985757' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi047-82386-58"}]: dispatch 2024-02-23T00:47:02.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:01.180841+0000 mon.a (mon.0) 2261 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-45", "mode": "writeback"}]: dispatch 2024-02-23T00:47:02.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:01.193775+0000 mon.a (mon.0) 2262 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi047-82386-58"}]: dispatch 2024-02-23T00:47:02.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:01.234071+0000 mon.b (mon.1) 375 : audit [INF] from='client.? 172.21.15.47:0/2489350951' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi047-82419-3","pool2":"test-rados-api-smithi047-82419-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-23T00:47:02.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:01.234423+0000 mon.a (mon.0) 2263 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi047-82419-3","pool2":"test-rados-api-smithi047-82419-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-23T00:47:02.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:01.267236+0000 mgr.y (mgr.14517) 334 : cluster [DBG] pgmap v499: 250 pgs: 64 unknown, 186 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:47:02.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:01.586252+0000 mon.b (mon.1) 376 : audit [INF] from='client.? 172.21.15.47:0/3830340996' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:47:02.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:01.586700+0000 mon.a (mon.0) 2264 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:47:02.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:02.173820+0000 mon.a (mon.0) 2265 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:47:02.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:02.173848+0000 mon.a (mon.0) 2266 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:47:02.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:02.178082+0000 mon.a (mon.0) 2267 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-45", "mode": "writeback"}]': finished 2024-02-23T00:47:02.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:02.178146+0000 mon.a (mon.0) 2268 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi047-82386-58"}]': finished 2024-02-23T00:47:02.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:02.178208+0000 mon.a (mon.0) 2269 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi047-82419-3","pool2":"test-rados-api-smithi047-82419-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-23T00:47:02.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:02.178286+0000 mon.a (mon.0) 2270 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:47:02.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:02.178339+0000 mon.a (mon.0) 2271 : cluster [DBG] osdmap e350: 8 total, 8 up, 8 in 2024-02-23T00:47:02.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:02.183539+0000 mon.a (mon.0) 2272 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:47:02.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:01.176523+0000 mon.a (mon.0) 2257 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-45"}]': finished 2024-02-23T00:47:02.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:01.176647+0000 mon.a (mon.0) 2258 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi047-82386-58"}]': finished 2024-02-23T00:47:02.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:01.176744+0000 mon.a (mon.0) 2259 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82419-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:47:02.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:01.176846+0000 mon.a (mon.0) 2260 : cluster [DBG] osdmap e349: 8 total, 8 up, 8 in 2024-02-23T00:47:02.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:01.179698+0000 mon.c (mon.2) 270 : audit [INF] from='client.? 172.21.15.47:0/3507985757' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi047-82386-58"}]: dispatch 2024-02-23T00:47:02.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:01.180841+0000 mon.a (mon.0) 2261 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-45", "mode": "writeback"}]: dispatch 2024-02-23T00:47:02.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:01.193775+0000 mon.a (mon.0) 2262 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi047-82386-58"}]: dispatch 2024-02-23T00:47:02.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:01.234071+0000 mon.b (mon.1) 375 : audit [INF] from='client.? 172.21.15.47:0/2489350951' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi047-82419-3","pool2":"test-rados-api-smithi047-82419-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-23T00:47:02.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:01.234423+0000 mon.a (mon.0) 2263 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi047-82419-3","pool2":"test-rados-api-smithi047-82419-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-23T00:47:02.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:01.267236+0000 mgr.y (mgr.14517) 334 : cluster [DBG] pgmap v499: 250 pgs: 64 unknown, 186 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:47:02.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:01.586252+0000 mon.b (mon.1) 376 : audit [INF] from='client.? 172.21.15.47:0/3830340996' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:47:02.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:01.586700+0000 mon.a (mon.0) 2264 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:47:02.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:02.173820+0000 mon.a (mon.0) 2265 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:47:02.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:02.173848+0000 mon.a (mon.0) 2266 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:47:02.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:02.178082+0000 mon.a (mon.0) 2267 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-45", "mode": "writeback"}]': finished 2024-02-23T00:47:02.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:02.178146+0000 mon.a (mon.0) 2268 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi047-82386-58"}]': finished 2024-02-23T00:47:02.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:02.178208+0000 mon.a (mon.0) 2269 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi047-82419-3","pool2":"test-rados-api-smithi047-82419-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-23T00:47:02.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:02.178286+0000 mon.a (mon.0) 2270 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:47:02.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:02.178339+0000 mon.a (mon.0) 2271 : cluster [DBG] osdmap e350: 8 total, 8 up, 8 in 2024-02-23T00:47:02.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:02.183539+0000 mon.a (mon.0) 2272 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:47:02.462 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-23T00:47:02.462 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-23T00:47:02.463 INFO:tasks.thrashosds.thrasher:Shrinking pool default.rgw.meta 2024-02-23T00:47:02.463 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool default.rgw.meta by 10 min 8 2024-02-23T00:47:02.463 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph pg dump --format=json 2024-02-23T00:47:02.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:01.176523+0000 mon.a (mon.0) 2257 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-45"}]': finished 2024-02-23T00:47:02.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:01.176647+0000 mon.a (mon.0) 2258 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi047-82386-58"}]': finished 2024-02-23T00:47:02.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:01.176744+0000 mon.a (mon.0) 2259 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82419-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:47:02.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:01.176846+0000 mon.a (mon.0) 2260 : cluster [DBG] osdmap e349: 8 total, 8 up, 8 in 2024-02-23T00:47:02.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:01.179698+0000 mon.c (mon.2) 270 : audit [INF] from='client.? 172.21.15.47:0/3507985757' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi047-82386-58"}]: dispatch 2024-02-23T00:47:02.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:01.180841+0000 mon.a (mon.0) 2261 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-45", "mode": "writeback"}]: dispatch 2024-02-23T00:47:02.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:01.193775+0000 mon.a (mon.0) 2262 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi047-82386-58"}]: dispatch 2024-02-23T00:47:02.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:01.234071+0000 mon.b (mon.1) 375 : audit [INF] from='client.? 172.21.15.47:0/2489350951' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi047-82419-3","pool2":"test-rados-api-smithi047-82419-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-23T00:47:02.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:01.234423+0000 mon.a (mon.0) 2263 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi047-82419-3","pool2":"test-rados-api-smithi047-82419-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-23T00:47:02.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:01.267236+0000 mgr.y (mgr.14517) 334 : cluster [DBG] pgmap v499: 250 pgs: 64 unknown, 186 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:47:02.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:01.586252+0000 mon.b (mon.1) 376 : audit [INF] from='client.? 172.21.15.47:0/3830340996' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:47:02.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:01.586700+0000 mon.a (mon.0) 2264 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:47:02.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:02.173820+0000 mon.a (mon.0) 2265 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:47:02.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:02.173848+0000 mon.a (mon.0) 2266 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:47:02.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:02.178082+0000 mon.a (mon.0) 2267 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-45", "mode": "writeback"}]': finished 2024-02-23T00:47:02.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:02.178146+0000 mon.a (mon.0) 2268 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi047-82386-58"}]': finished 2024-02-23T00:47:02.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:02.178208+0000 mon.a (mon.0) 2269 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi047-82419-3","pool2":"test-rados-api-smithi047-82419-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-23T00:47:02.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:02.178286+0000 mon.a (mon.0) 2270 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:47:02.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:02.178339+0000 mon.a (mon.0) 2271 : cluster [DBG] osdmap e350: 8 total, 8 up, 8 in 2024-02-23T00:47:02.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:02.183539+0000 mon.a (mon.0) 2272 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:47:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:47:02.555 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:02.555 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:47:02.656 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:47:02.683 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:02.832 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:47:02.834 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:47:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:47:02.834 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:47:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:47:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:47:02.835 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:02.835 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:47:02.843 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:47:02.844 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:47:02.936 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:47:02.964 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:03.062 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:47:03.065 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:03.065 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:47:03.166 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:47:03.201 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:03.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:02.525905+0000 mgr.y (mgr.14517) 335 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:47:03.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:03.180485+0000 mon.a (mon.0) 2273 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "hit_set_count","val": "2"}]': finished 2024-02-23T00:47:03.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:03.180602+0000 mon.a (mon.0) 2274 : cluster [DBG] osdmap e351: 8 total, 8 up, 8 in 2024-02-23T00:47:03.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:03.182102+0000 mon.a (mon.0) 2275 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:47:03.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:03.189831+0000 mon.a (mon.0) 2276 : audit [INF] from='client.? 172.21.15.47:0/2013956265' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi047-82386-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:47:03.550 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:47:03.554 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:03.554 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:47:03.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:02.525905+0000 mgr.y (mgr.14517) 335 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:47:03.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:03.180485+0000 mon.a (mon.0) 2273 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "hit_set_count","val": "2"}]': finished 2024-02-23T00:47:03.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:03.180602+0000 mon.a (mon.0) 2274 : cluster [DBG] osdmap e351: 8 total, 8 up, 8 in 2024-02-23T00:47:03.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:03.182102+0000 mon.a (mon.0) 2275 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:47:03.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:03.189831+0000 mon.a (mon.0) 2276 : audit [INF] from='client.? 172.21.15.47:0/2013956265' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi047-82386-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:47:03.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:02.525905+0000 mgr.y (mgr.14517) 335 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:47:03.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:03.180485+0000 mon.a (mon.0) 2273 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "hit_set_count","val": "2"}]': finished 2024-02-23T00:47:03.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:03.180602+0000 mon.a (mon.0) 2274 : cluster [DBG] osdmap e351: 8 total, 8 up, 8 in 2024-02-23T00:47:03.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:03.182102+0000 mon.a (mon.0) 2275 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:47:03.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:03.189831+0000 mon.a (mon.0) 2276 : audit [INF] from='client.? 172.21.15.47:0/2013956265' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi047-82386-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:47:03.570 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.6 dump_blocked_ops 2024-02-23T00:47:03.655 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:47:03.683 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:03.702 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:47:03.704 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:03.705 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:47:03.806 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:47:03.834 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:03.854 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:47:03.857 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:03.858 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:47:03.888 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:47:03.959 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:47:03.991 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:04.014 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:47:04.017 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:04.018 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:47:04.118 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:47:04.149 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:04.222 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:47:04.225 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:04.226 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:47:04.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:03.267819+0000 mgr.y (mgr.14517) 336 : cluster [DBG] pgmap v502: 250 pgs: 32 unknown, 218 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-23T00:47:04.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:04.181098+0000 mon.a (mon.0) 2277 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "hit_set_period","val": "600"}]': finished 2024-02-23T00:47:04.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:04.181289+0000 mon.a (mon.0) 2278 : audit [INF] from='client.? 172.21.15.47:0/2013956265' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi047-82386-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:47:04.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:04.181368+0000 mon.a (mon.0) 2279 : cluster [DBG] osdmap e352: 8 total, 8 up, 8 in 2024-02-23T00:47:04.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:04.184655+0000 mon.a (mon.0) 2280 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-23T00:47:04.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:03.267819+0000 mgr.y (mgr.14517) 336 : cluster [DBG] pgmap v502: 250 pgs: 32 unknown, 218 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-23T00:47:04.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:04.181098+0000 mon.a (mon.0) 2277 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "hit_set_period","val": "600"}]': finished 2024-02-23T00:47:04.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:04.181289+0000 mon.a (mon.0) 2278 : audit [INF] from='client.? 172.21.15.47:0/2013956265' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi047-82386-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:47:04.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:04.181368+0000 mon.a (mon.0) 2279 : cluster [DBG] osdmap e352: 8 total, 8 up, 8 in 2024-02-23T00:47:04.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:04.184655+0000 mon.a (mon.0) 2280 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-23T00:47:04.326 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:47:04.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.315+0000 7f22db489700 1 -- 172.21.15.47:0/147730986 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f22d40f7330 msgr2=0x7f22d4103930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:04.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.315+0000 7f22db489700 1 --2- 172.21.15.47:0/147730986 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f22d40f7330 0x7f22d4103930 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 rx=0x7f22cc009be0 tx=0x7f22cc004d10).stop 2024-02-23T00:47:04.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.315+0000 7f22db489700 1 -- 172.21.15.47:0/147730986 shutdown_connections 2024-02-23T00:47:04.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.315+0000 7f22db489700 1 --2- 172.21.15.47:0/147730986 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f22d40f7330 0x7f22d4103930 unknown :-1 s=CLOSED pgs=2633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:04.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.315+0000 7f22db489700 1 --2- 172.21.15.47:0/147730986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22d40fcac0 0x7f22d40fcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:04.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.315+0000 7f22db489700 1 --2- 172.21.15.47:0/147730986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22d40fd420 0x7f22d40f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:04.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.315+0000 7f22db489700 1 -- 172.21.15.47:0/147730986 >> 172.21.15.47:0/147730986 conn(0x7f22d40f2710 msgr2=0x7f22d40f4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:04.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.315+0000 7f22db489700 1 -- 172.21.15.47:0/147730986 shutdown_connections 2024-02-23T00:47:04.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.315+0000 7f22db489700 1 -- 172.21.15.47:0/147730986 wait complete. 2024-02-23T00:47:04.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.315+0000 7f22db489700 1 librados: starting msgr at 2024-02-23T00:47:04.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.315+0000 7f22db489700 1 librados: starting objecter 2024-02-23T00:47:04.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.315+0000 7f22db489700 1 Processor -- start 2024-02-23T00:47:04.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.315+0000 7f22db489700 1 -- start start 2024-02-23T00:47:04.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.315+0000 7f22db489700 1 librados: setting wanted keys 2024-02-23T00:47:04.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.315+0000 7f22db489700 1 librados: calling monclient init 2024-02-23T00:47:04.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.316+0000 7f22db489700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22d40f7330 0x7f22d4108f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:04.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.316+0000 7f22db489700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22d40fcac0 0x7f22d4109420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:04.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.316+0000 7f22db489700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f22d40fd420 0x7f22d4109a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:04.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.316+0000 7f22db489700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d4114190 con 0x7f22d40fcac0 2024-02-23T00:47:04.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.316+0000 7f22db489700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d4114300 con 0x7f22d40f7330 2024-02-23T00:47:04.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.316+0000 7f22db489700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d41144c0 con 0x7f22d40fd420 2024-02-23T00:47:04.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.316+0000 7f22d8a24700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22d40fcac0 0x7f22d4109420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:04.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.316+0000 7f22d9a26700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f22d40fd420 0x7f22d4109a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:04.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.316+0000 7f22d8a24700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22d40fcac0 0x7f22d4109420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47244/0 (socket says 172.21.15.47:47244) 2024-02-23T00:47:04.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.316+0000 7f22d9a26700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f22d40fd420 0x7f22d4109a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:38204/0 (socket says 172.21.15.47:38204) 2024-02-23T00:47:04.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.316+0000 7f22d8a24700 1 -- 172.21.15.47:0/970342121 learned_addr learned my addr 172.21.15.47:0/970342121 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:47:04.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.316+0000 7f22d9225700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22d40f7330 0x7f22d4108f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:04.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.316+0000 7f22d9a26700 1 -- 172.21.15.47:0/970342121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22d40f7330 msgr2=0x7f22d4108f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:04.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.316+0000 7f22d9a26700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22d40f7330 0x7f22d4108f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:04.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.316+0000 7f22d9a26700 1 -- 172.21.15.47:0/970342121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22d40fcac0 msgr2=0x7f22d4109420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:04.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.316+0000 7f22d9a26700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22d40fcac0 0x7f22d4109420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:04.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.316+0000 7f22d9a26700 1 -- 172.21.15.47:0/970342121 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c4005040 con 0x7f22d40fd420 2024-02-23T00:47:04.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.316+0000 7f22d8a24700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22d40fcac0 0x7f22d4109420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:47:04.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.316+0000 7f22d9a26700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f22d40fd420 0x7f22d4109a50 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 rx=0x7f22cc009790 tx=0x7f22cc004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:04.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.316+0000 7f22ca7fc700 10 client.?.objecter ms_handle_connect 0x7f22d40fd420 2024-02-23T00:47:04.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.316+0000 7f22ca7fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:47:04.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.316+0000 7f22ca7fc700 1 -- 172.21.15.47:0/970342121 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc00e040 con 0x7f22d40fd420 2024-02-23T00:47:04.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.316+0000 7f22ca7fc700 1 -- 172.21.15.47:0/970342121 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f22cc018040 con 0x7f22d40fd420 2024-02-23T00:47:04.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.316+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22cc0097c0 con 0x7f22d40fd420 2024-02-23T00:47:04.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.316+0000 7f22ca7fc700 1 -- 172.21.15.47:0/970342121 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc013c40 con 0x7f22d40fd420 2024-02-23T00:47:04.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.316+0000 7f22db489700 10 client.41186.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:47:04.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.316+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d4114960 con 0x7f22d40fd420 2024-02-23T00:47:04.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.317+0000 7f22db489700 1 librados: init done 2024-02-23T00:47:04.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.317+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f22d40606d0 con 0x7f22d40fd420 2024-02-23T00:47:04.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.318+0000 7f22ca7fc700 1 -- 172.21.15.47:0/970342121 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f22cc013420 con 0x7f22d40fd420 2024-02-23T00:47:04.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.318+0000 7f22ca7fc700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f22c0062cc0 0x7f22c0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:04.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.318+0000 7f22ca7fc700 1 -- 172.21.15.47:0/970342121 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(352..352 src has 48..352) v4 ==== 9575+0+0 (secure 0 0 0) 0x7f22cc07f810 con 0x7f22d40fd420 2024-02-23T00:47:04.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.318+0000 7f22ca7fc700 10 client.41186.objecter ms_dispatch 0x7f22d4001550 osd_map(352..352 src has 48..352) v4 2024-02-23T00:47:04.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.318+0000 7f22ca7fc700 3 client.41186.objecter handle_osd_map got epochs [352,352] > 0 2024-02-23T00:47:04.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.318+0000 7f22ca7fc700 3 client.41186.objecter handle_osd_map decoding full epoch 352 2024-02-23T00:47:04.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.318+0000 7f22d9225700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f22c0062cc0 0x7f22c0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:04.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.319+0000 7f22ca7fc700 20 client.41186.objecter dump_active .. 0 homeless 2024-02-23T00:47:04.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.319+0000 7f22ca7fc700 1 -- 172.21.15.47:0/970342121 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v352) v1 ==== 54+0+16 (secure 0 0 0) 0x7f22cc0800a0 con 0x7f22d40fd420 2024-02-23T00:47:04.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.319+0000 7f22db489700 10 client.41186.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:04.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.319+0000 7f22db489700 15 client.41186.objecter _session_command_op_assign -1 1 2024-02-23T00:47:04.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.319+0000 7f22db489700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f22d40fdd70 0x7f22d40fe150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:04.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.319+0000 7f22db489700 20 client.41186.objecter _get_session s=0x7f22d405ee80 osd=0 3 2024-02-23T00:47:04.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.319+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d405ee80 osd=0 3 2024-02-23T00:47:04.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.319+0000 7f22db489700 20 client.41186.objecter _get_session s=0x7f22d405ee80 osd=0 3 2024-02-23T00:47:04.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.319+0000 7f22ca7fc700 10 client.41186.objecter ms_handle_connect 0x7f22c0062cc0 2024-02-23T00:47:04.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.319+0000 7f22db489700 15 client.41186.objecter _session_command_op_remove -1 1 2024-02-23T00:47:04.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.319+0000 7f22db489700 20 client.41186.objecter get_session s=0x7f22d405ee80 osd=0 3 2024-02-23T00:47:04.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.319+0000 7f22db489700 15 client.41186.objecter _session_command_op_assign 0 1 2024-02-23T00:47:04.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.319+0000 7f22d9225700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f22c0062cc0 0x7f22c0065180 secure :-1 s=READY pgs=3038 cs=0 l=1 rev1=1 rx=0x7f22d00048b0 tx=0x7f22d000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:04.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.319+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d405ee80 osd=0 4 2024-02-23T00:47:04.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.319+0000 7f22db489700 10 client.41186.objecter _send_command 1 2024-02-23T00:47:04.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.319+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f22d4105de0 con 0x7f22d40fdd70 2024-02-23T00:47:04.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.319+0000 7f22d8a24700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f22d40fdd70 0x7f22d40fe150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:04.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.319+0000 7f22ca7fc700 10 client.41186.objecter ms_handle_connect 0x7f22d40fdd70 2024-02-23T00:47:04.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.319+0000 7f22d8a24700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f22d40fdd70 0x7f22d40fe150 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:04.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.320+0000 7f22ca7fc700 1 -- 172.21.15.47:0/970342121 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f22d4105de0 con 0x7f22d40fdd70 2024-02-23T00:47:04.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.320+0000 7f22ca7fc700 10 client.41186.objecter ms_dispatch 0x7f22d4001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:47:04.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.320+0000 7f22ca7fc700 10 client.41186.objecter _finish_command 1 = system:0 2024-02-23T00:47:04.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.320+0000 7f22ca7fc700 20 client.41186.objecter put_session s=0x7f22d405ee80 osd=0 4 2024-02-23T00:47:04.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.320+0000 7f22ca7fc700 15 client.41186.objecter _session_command_op_remove 0 1 2024-02-23T00:47:04.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.335+0000 7f22db489700 10 client.41186.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:47:04.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.335+0000 7f22db489700 15 client.41186.objecter _session_command_op_assign -1 2 2024-02-23T00:47:04.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.335+0000 7f22db489700 20 client.41186.objecter _get_session s=0x7f22d405ee80 osd=0 3 2024-02-23T00:47:04.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.335+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d405ee80 osd=0 3 2024-02-23T00:47:04.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.335+0000 7f22db489700 20 client.41186.objecter _get_session s=0x7f22d405ee80 osd=0 3 2024-02-23T00:47:04.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.335+0000 7f22db489700 15 client.41186.objecter _session_command_op_remove -1 2 2024-02-23T00:47:04.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.335+0000 7f22db489700 20 client.41186.objecter get_session s=0x7f22d405ee80 osd=0 3 2024-02-23T00:47:04.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.335+0000 7f22db489700 15 client.41186.objecter _session_command_op_assign 0 2 2024-02-23T00:47:04.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.335+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d405ee80 osd=0 4 2024-02-23T00:47:04.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.335+0000 7f22db489700 10 client.41186.objecter _send_command 2 2024-02-23T00:47:04.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.335+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f22d40273c0 con 0x7f22d40fdd70 2024-02-23T00:47:04.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.336+0000 7f22ca7fc700 1 -- 172.21.15.47:0/970342121 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f22d40273c0 con 0x7f22d40fdd70 2024-02-23T00:47:04.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.336+0000 7f22ca7fc700 10 client.41186.objecter ms_dispatch 0x7f22d4001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:47:04.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.336+0000 7f22ca7fc700 10 client.41186.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:47:04.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.336+0000 7f22ca7fc700 20 client.41186.objecter put_session s=0x7f22d405ee80 osd=0 4 2024-02-23T00:47:04.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.336+0000 7f22ca7fc700 15 client.41186.objecter _session_command_op_remove 0 2 2024-02-23T00:47:04.343 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:47:04.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.337+0000 7f22db489700 10 client.41186.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:04.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.337+0000 7f22db489700 15 client.41186.objecter _session_command_op_assign -1 3 2024-02-23T00:47:04.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.337+0000 7f22db489700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f22d40598f0 0x7f22d4059cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:04.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.337+0000 7f22db489700 20 client.41186.objecter _get_session s=0x7f22d4028710 osd=1 3 2024-02-23T00:47:04.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.337+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d4028710 osd=1 3 2024-02-23T00:47:04.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.337+0000 7f22db489700 20 client.41186.objecter _get_session s=0x7f22d4028710 osd=1 3 2024-02-23T00:47:04.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.337+0000 7f22db489700 15 client.41186.objecter _session_command_op_remove -1 3 2024-02-23T00:47:04.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.337+0000 7f22db489700 20 client.41186.objecter get_session s=0x7f22d4028710 osd=1 3 2024-02-23T00:47:04.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.337+0000 7f22db489700 15 client.41186.objecter _session_command_op_assign 1 3 2024-02-23T00:47:04.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.337+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d4028710 osd=1 4 2024-02-23T00:47:04.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.337+0000 7f22db489700 10 client.41186.objecter _send_command 3 2024-02-23T00:47:04.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.337+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f22d4108d30 con 0x7f22d40598f0 2024-02-23T00:47:04.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.337+0000 7f22d9225700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f22d40598f0 0x7f22d4059cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:04.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.337+0000 7f22ca7fc700 10 client.41186.objecter ms_handle_connect 0x7f22d40598f0 2024-02-23T00:47:04.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.337+0000 7f22d9225700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f22d40598f0 0x7f22d4059cf0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:04.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.338+0000 7f22ca7fc700 1 -- 172.21.15.47:0/970342121 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f22d4108d30 con 0x7f22d40598f0 2024-02-23T00:47:04.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.338+0000 7f22ca7fc700 10 client.41186.objecter ms_dispatch 0x7f22d4001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:47:04.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.338+0000 7f22ca7fc700 10 client.41186.objecter _finish_command 3 = system:0 2024-02-23T00:47:04.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.338+0000 7f22ca7fc700 20 client.41186.objecter put_session s=0x7f22d4028710 osd=1 4 2024-02-23T00:47:04.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.338+0000 7f22ca7fc700 15 client.41186.objecter _session_command_op_remove 1 3 2024-02-23T00:47:04.355 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:04.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.353+0000 7f22db489700 10 client.41186.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:47:04.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.353+0000 7f22db489700 15 client.41186.objecter _session_command_op_assign -1 4 2024-02-23T00:47:04.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.353+0000 7f22db489700 20 client.41186.objecter _get_session s=0x7f22d4028710 osd=1 3 2024-02-23T00:47:04.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.353+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d4028710 osd=1 3 2024-02-23T00:47:04.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.353+0000 7f22db489700 20 client.41186.objecter _get_session s=0x7f22d4028710 osd=1 3 2024-02-23T00:47:04.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.353+0000 7f22db489700 15 client.41186.objecter _session_command_op_remove -1 4 2024-02-23T00:47:04.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.353+0000 7f22db489700 20 client.41186.objecter get_session s=0x7f22d4028710 osd=1 3 2024-02-23T00:47:04.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.353+0000 7f22db489700 15 client.41186.objecter _session_command_op_assign 1 4 2024-02-23T00:47:04.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.353+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d4028710 osd=1 4 2024-02-23T00:47:04.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.353+0000 7f22db489700 10 client.41186.objecter _send_command 4 2024-02-23T00:47:04.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.353+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f22d4059360 con 0x7f22d40598f0 2024-02-23T00:47:04.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.355+0000 7f22ca7fc700 1 -- 172.21.15.47:0/970342121 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f22d4059360 con 0x7f22d40598f0 2024-02-23T00:47:04.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.355+0000 7f22ca7fc700 10 client.41186.objecter ms_dispatch 0x7f22d4001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:47:04.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.355+0000 7f22ca7fc700 10 client.41186.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:47:04.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.355+0000 7f22ca7fc700 20 client.41186.objecter put_session s=0x7f22d4028710 osd=1 4 2024-02-23T00:47:04.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.355+0000 7f22ca7fc700 15 client.41186.objecter _session_command_op_remove 1 4 2024-02-23T00:47:04.358 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:47:04.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.356+0000 7f22db489700 10 client.41186.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:04.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.356+0000 7f22db489700 15 client.41186.objecter _session_command_op_assign -1 5 2024-02-23T00:47:04.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.356+0000 7f22db489700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f22d4028270 0x7f22d411e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:04.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.356+0000 7f22db489700 20 client.41186.objecter _get_session s=0x7f22d4059710 osd=2 3 2024-02-23T00:47:04.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.356+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d4059710 osd=2 3 2024-02-23T00:47:04.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.356+0000 7f22db489700 20 client.41186.objecter _get_session s=0x7f22d4059710 osd=2 3 2024-02-23T00:47:04.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.356+0000 7f22db489700 15 client.41186.objecter _session_command_op_remove -1 5 2024-02-23T00:47:04.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.356+0000 7f22db489700 20 client.41186.objecter get_session s=0x7f22d4059710 osd=2 3 2024-02-23T00:47:04.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.356+0000 7f22db489700 15 client.41186.objecter _session_command_op_assign 2 5 2024-02-23T00:47:04.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.356+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d4059710 osd=2 4 2024-02-23T00:47:04.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.356+0000 7f22db489700 10 client.41186.objecter _send_command 5 2024-02-23T00:47:04.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.356+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f22d4120d80 con 0x7f22d4028270 2024-02-23T00:47:04.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.356+0000 7f22d8a24700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f22d4028270 0x7f22d411e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:04.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.356+0000 7f22ca7fc700 10 client.41186.objecter ms_handle_connect 0x7f22d4028270 2024-02-23T00:47:04.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.356+0000 7f22d8a24700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f22d4028270 0x7f22d411e130 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:04.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.357+0000 7f22ca7fc700 1 -- 172.21.15.47:0/970342121 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f22d4120d80 con 0x7f22d4028270 2024-02-23T00:47:04.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.357+0000 7f22ca7fc700 10 client.41186.objecter ms_dispatch 0x7f22d4001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:47:04.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.357+0000 7f22ca7fc700 10 client.41186.objecter _finish_command 5 = system:0 2024-02-23T00:47:04.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.357+0000 7f22ca7fc700 20 client.41186.objecter put_session s=0x7f22d4059710 osd=2 4 2024-02-23T00:47:04.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.357+0000 7f22ca7fc700 15 client.41186.objecter _session_command_op_remove 2 5 2024-02-23T00:47:04.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.373+0000 7f22db489700 10 client.41186.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:47:04.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.373+0000 7f22db489700 15 client.41186.objecter _session_command_op_assign -1 6 2024-02-23T00:47:04.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.373+0000 7f22db489700 20 client.41186.objecter _get_session s=0x7f22d4059710 osd=2 3 2024-02-23T00:47:04.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.373+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d4059710 osd=2 3 2024-02-23T00:47:04.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.373+0000 7f22db489700 20 client.41186.objecter _get_session s=0x7f22d4059710 osd=2 3 2024-02-23T00:47:04.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.373+0000 7f22db489700 15 client.41186.objecter _session_command_op_remove -1 6 2024-02-23T00:47:04.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.373+0000 7f22db489700 20 client.41186.objecter get_session s=0x7f22d4059710 osd=2 3 2024-02-23T00:47:04.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.373+0000 7f22db489700 15 client.41186.objecter _session_command_op_assign 2 6 2024-02-23T00:47:04.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.373+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d4059710 osd=2 4 2024-02-23T00:47:04.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.373+0000 7f22db489700 10 client.41186.objecter _send_command 6 2024-02-23T00:47:04.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.373+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f22d40277d0 con 0x7f22d4028270 2024-02-23T00:47:04.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.374+0000 7f22ca7fc700 1 -- 172.21.15.47:0/970342121 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f22d40277d0 con 0x7f22d4028270 2024-02-23T00:47:04.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.374+0000 7f22ca7fc700 10 client.41186.objecter ms_dispatch 0x7f22d4001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:47:04.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.374+0000 7f22ca7fc700 10 client.41186.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:47:04.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.374+0000 7f22ca7fc700 20 client.41186.objecter put_session s=0x7f22d4059710 osd=2 4 2024-02-23T00:47:04.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.374+0000 7f22ca7fc700 15 client.41186.objecter _session_command_op_remove 2 6 2024-02-23T00:47:04.377 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:47:04.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.375+0000 7f22db489700 10 client.41186.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:04.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.375+0000 7f22db489700 15 client.41186.objecter _session_command_op_assign -1 7 2024-02-23T00:47:04.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.375+0000 7f22db489700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f22d411f7a0 0x7f22d411fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:04.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.375+0000 7f22db489700 20 client.41186.objecter _get_session s=0x7f22d411b770 osd=3 3 2024-02-23T00:47:04.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.375+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d411b770 osd=3 3 2024-02-23T00:47:04.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.375+0000 7f22db489700 20 client.41186.objecter _get_session s=0x7f22d411b770 osd=3 3 2024-02-23T00:47:04.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.375+0000 7f22db489700 15 client.41186.objecter _session_command_op_remove -1 7 2024-02-23T00:47:04.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.375+0000 7f22db489700 20 client.41186.objecter get_session s=0x7f22d411b770 osd=3 3 2024-02-23T00:47:04.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.375+0000 7f22db489700 15 client.41186.objecter _session_command_op_assign 3 7 2024-02-23T00:47:04.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.375+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d411b770 osd=3 4 2024-02-23T00:47:04.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.375+0000 7f22db489700 10 client.41186.objecter _send_command 7 2024-02-23T00:47:04.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.375+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f22d411be40 con 0x7f22d411f7a0 2024-02-23T00:47:04.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.375+0000 7f22d9a26700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f22d411f7a0 0x7f22d411fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:04.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.375+0000 7f22ca7fc700 10 client.41186.objecter ms_handle_connect 0x7f22d411f7a0 2024-02-23T00:47:04.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.375+0000 7f22d9a26700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f22d411f7a0 0x7f22d411fba0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:04.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.376+0000 7f22ca7fc700 1 -- 172.21.15.47:0/970342121 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f22d411be40 con 0x7f22d411f7a0 2024-02-23T00:47:04.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.376+0000 7f22ca7fc700 10 client.41186.objecter ms_dispatch 0x7f22d4001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:47:04.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.376+0000 7f22ca7fc700 10 client.41186.objecter _finish_command 7 = system:0 2024-02-23T00:47:04.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.376+0000 7f22ca7fc700 20 client.41186.objecter put_session s=0x7f22d411b770 osd=3 4 2024-02-23T00:47:04.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.376+0000 7f22ca7fc700 15 client.41186.objecter _session_command_op_remove 3 7 2024-02-23T00:47:04.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.391+0000 7f22db489700 10 client.41186.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:47:04.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.392+0000 7f22db489700 15 client.41186.objecter _session_command_op_assign -1 8 2024-02-23T00:47:04.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.392+0000 7f22db489700 20 client.41186.objecter _get_session s=0x7f22d411b770 osd=3 3 2024-02-23T00:47:04.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.392+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d411b770 osd=3 3 2024-02-23T00:47:04.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.392+0000 7f22db489700 20 client.41186.objecter _get_session s=0x7f22d411b770 osd=3 3 2024-02-23T00:47:04.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.392+0000 7f22db489700 15 client.41186.objecter _session_command_op_remove -1 8 2024-02-23T00:47:04.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.392+0000 7f22db489700 20 client.41186.objecter get_session s=0x7f22d411b770 osd=3 3 2024-02-23T00:47:04.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.392+0000 7f22db489700 15 client.41186.objecter _session_command_op_assign 3 8 2024-02-23T00:47:04.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.392+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d411b770 osd=3 4 2024-02-23T00:47:04.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.392+0000 7f22db489700 10 client.41186.objecter _send_command 8 2024-02-23T00:47:04.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.392+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f22d4120860 con 0x7f22d411f7a0 2024-02-23T00:47:04.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.393+0000 7f22ca7fc700 1 -- 172.21.15.47:0/970342121 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f22d4120860 con 0x7f22d411f7a0 2024-02-23T00:47:04.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.393+0000 7f22ca7fc700 10 client.41186.objecter ms_dispatch 0x7f22d4001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:47:04.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.393+0000 7f22ca7fc700 10 client.41186.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:47:04.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.393+0000 7f22ca7fc700 20 client.41186.objecter put_session s=0x7f22d411b770 osd=3 4 2024-02-23T00:47:04.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.393+0000 7f22ca7fc700 15 client.41186.objecter _session_command_op_remove 3 8 2024-02-23T00:47:04.396 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:47:04.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.394+0000 7f22db489700 10 client.41186.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:04.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.394+0000 7f22db489700 15 client.41186.objecter _session_command_op_assign -1 9 2024-02-23T00:47:04.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.394+0000 7f22db489700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f22d4122f30 0x7f22d4125320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:04.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.394+0000 7f22db489700 20 client.41186.objecter _get_session s=0x7f22d4120860 osd=4 3 2024-02-23T00:47:04.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.394+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d4120860 osd=4 3 2024-02-23T00:47:04.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.394+0000 7f22db489700 20 client.41186.objecter _get_session s=0x7f22d4120860 osd=4 3 2024-02-23T00:47:04.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.394+0000 7f22db489700 15 client.41186.objecter _session_command_op_remove -1 9 2024-02-23T00:47:04.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.394+0000 7f22db489700 20 client.41186.objecter get_session s=0x7f22d4120860 osd=4 3 2024-02-23T00:47:04.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.394+0000 7f22db489700 15 client.41186.objecter _session_command_op_assign 4 9 2024-02-23T00:47:04.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.394+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d4120860 osd=4 4 2024-02-23T00:47:04.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.394+0000 7f22db489700 10 client.41186.objecter _send_command 9 2024-02-23T00:47:04.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.394+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f22d4120b70 con 0x7f22d4122f30 2024-02-23T00:47:04.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.394+0000 7f22d9225700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f22d4122f30 0x7f22d4125320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:04.398 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:47:04.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.394+0000 7f22ca7fc700 10 client.41186.objecter ms_handle_connect 0x7f22d4122f30 2024-02-23T00:47:04.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.394+0000 7f22d9225700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f22d4122f30 0x7f22d4125320 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:04.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.395+0000 7f22ca7fc700 1 -- 172.21.15.47:0/970342121 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f22d4120b70 con 0x7f22d4122f30 2024-02-23T00:47:04.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.395+0000 7f22ca7fc700 10 client.41186.objecter ms_dispatch 0x7f22d4001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:47:04.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.395+0000 7f22ca7fc700 10 client.41186.objecter _finish_command 9 = system:0 2024-02-23T00:47:04.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.395+0000 7f22ca7fc700 20 client.41186.objecter put_session s=0x7f22d4120860 osd=4 4 2024-02-23T00:47:04.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.395+0000 7f22ca7fc700 15 client.41186.objecter _session_command_op_remove 4 9 2024-02-23T00:47:04.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.410+0000 7f22db489700 10 client.41186.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:47:04.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.410+0000 7f22db489700 15 client.41186.objecter _session_command_op_assign -1 10 2024-02-23T00:47:04.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.410+0000 7f22db489700 20 client.41186.objecter _get_session s=0x7f22d4120860 osd=4 3 2024-02-23T00:47:04.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.410+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d4120860 osd=4 3 2024-02-23T00:47:04.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.410+0000 7f22db489700 20 client.41186.objecter _get_session s=0x7f22d4120860 osd=4 3 2024-02-23T00:47:04.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.410+0000 7f22db489700 15 client.41186.objecter _session_command_op_remove -1 10 2024-02-23T00:47:04.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.410+0000 7f22db489700 20 client.41186.objecter get_session s=0x7f22d4120860 osd=4 3 2024-02-23T00:47:04.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.410+0000 7f22db489700 15 client.41186.objecter _session_command_op_assign 4 10 2024-02-23T00:47:04.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.410+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d4120860 osd=4 4 2024-02-23T00:47:04.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.410+0000 7f22db489700 10 client.41186.objecter _send_command 10 2024-02-23T00:47:04.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.410+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f22d4120b70 con 0x7f22d4122f30 2024-02-23T00:47:04.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.412+0000 7f22ca7fc700 1 -- 172.21.15.47:0/970342121 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f22d4120b70 con 0x7f22d4122f30 2024-02-23T00:47:04.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.412+0000 7f22ca7fc700 10 client.41186.objecter ms_dispatch 0x7f22d4001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:47:04.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.412+0000 7f22ca7fc700 10 client.41186.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:47:04.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.412+0000 7f22ca7fc700 20 client.41186.objecter put_session s=0x7f22d4120860 osd=4 4 2024-02-23T00:47:04.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.412+0000 7f22ca7fc700 15 client.41186.objecter _session_command_op_remove 4 10 2024-02-23T00:47:04.415 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:47:04.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.412+0000 7f22db489700 10 client.41186.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:04.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.412+0000 7f22db489700 15 client.41186.objecter _session_command_op_assign -1 11 2024-02-23T00:47:04.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.412+0000 7f22db489700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f22d4125d20 0x7f22d4128190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:04.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.412+0000 7f22db489700 20 client.41186.objecter _get_session s=0x7f22d4120b70 osd=5 3 2024-02-23T00:47:04.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.412+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d4120b70 osd=5 3 2024-02-23T00:47:04.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.412+0000 7f22db489700 20 client.41186.objecter _get_session s=0x7f22d4120b70 osd=5 3 2024-02-23T00:47:04.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.412+0000 7f22db489700 15 client.41186.objecter _session_command_op_remove -1 11 2024-02-23T00:47:04.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.412+0000 7f22db489700 20 client.41186.objecter get_session s=0x7f22d4120b70 osd=5 3 2024-02-23T00:47:04.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.412+0000 7f22db489700 15 client.41186.objecter _session_command_op_assign 5 11 2024-02-23T00:47:04.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.412+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d4120b70 osd=5 4 2024-02-23T00:47:04.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.412+0000 7f22db489700 10 client.41186.objecter _send_command 11 2024-02-23T00:47:04.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.412+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f22d412afd0 con 0x7f22d4125d20 2024-02-23T00:47:04.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.413+0000 7f22d8a24700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f22d4125d20 0x7f22d4128190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:04.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.414+0000 7f22ca7fc700 10 client.41186.objecter ms_handle_connect 0x7f22d4125d20 2024-02-23T00:47:04.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.414+0000 7f22d8a24700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f22d4125d20 0x7f22d4128190 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:04.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.414+0000 7f22ca7fc700 1 -- 172.21.15.47:0/970342121 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f22d412afd0 con 0x7f22d4125d20 2024-02-23T00:47:04.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.414+0000 7f22ca7fc700 10 client.41186.objecter ms_dispatch 0x7f22d4001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:47:04.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.414+0000 7f22ca7fc700 10 client.41186.objecter _finish_command 11 = system:0 2024-02-23T00:47:04.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.414+0000 7f22ca7fc700 20 client.41186.objecter put_session s=0x7f22d4120b70 osd=5 4 2024-02-23T00:47:04.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.414+0000 7f22ca7fc700 15 client.41186.objecter _session_command_op_remove 5 11 2024-02-23T00:47:04.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.430+0000 7f22db489700 10 client.41186.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:47:04.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.430+0000 7f22db489700 15 client.41186.objecter _session_command_op_assign -1 12 2024-02-23T00:47:04.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.430+0000 7f22db489700 20 client.41186.objecter _get_session s=0x7f22d4120b70 osd=5 3 2024-02-23T00:47:04.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.430+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d4120b70 osd=5 3 2024-02-23T00:47:04.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.430+0000 7f22db489700 20 client.41186.objecter _get_session s=0x7f22d4120b70 osd=5 3 2024-02-23T00:47:04.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.430+0000 7f22db489700 15 client.41186.objecter _session_command_op_remove -1 12 2024-02-23T00:47:04.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.430+0000 7f22db489700 20 client.41186.objecter get_session s=0x7f22d4120b70 osd=5 3 2024-02-23T00:47:04.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.430+0000 7f22db489700 15 client.41186.objecter _session_command_op_assign 5 12 2024-02-23T00:47:04.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.430+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d4120b70 osd=5 4 2024-02-23T00:47:04.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.430+0000 7f22db489700 10 client.41186.objecter _send_command 12 2024-02-23T00:47:04.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.430+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f22d412afd0 con 0x7f22d4125d20 2024-02-23T00:47:04.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.431+0000 7f22ca7fc700 1 -- 172.21.15.47:0/970342121 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f22d412afd0 con 0x7f22d4125d20 2024-02-23T00:47:04.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.431+0000 7f22ca7fc700 10 client.41186.objecter ms_dispatch 0x7f22d4001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:47:04.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.431+0000 7f22ca7fc700 10 client.41186.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:47:04.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.431+0000 7f22ca7fc700 20 client.41186.objecter put_session s=0x7f22d4120b70 osd=5 4 2024-02-23T00:47:04.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.431+0000 7f22ca7fc700 15 client.41186.objecter _session_command_op_remove 5 12 2024-02-23T00:47:04.434 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:47:04.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.432+0000 7f22db489700 10 client.41186.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:04.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.432+0000 7f22db489700 15 client.41186.objecter _session_command_op_assign -1 13 2024-02-23T00:47:04.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.432+0000 7f22db489700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f22d412a300 0x7f22d412a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:04.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.432+0000 7f22db489700 20 client.41186.objecter _get_session s=0x7f22d412afd0 osd=6 3 2024-02-23T00:47:04.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.432+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d412afd0 osd=6 3 2024-02-23T00:47:04.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.432+0000 7f22db489700 20 client.41186.objecter _get_session s=0x7f22d412afd0 osd=6 3 2024-02-23T00:47:04.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.432+0000 7f22db489700 15 client.41186.objecter _session_command_op_remove -1 13 2024-02-23T00:47:04.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.432+0000 7f22db489700 20 client.41186.objecter get_session s=0x7f22d412afd0 osd=6 3 2024-02-23T00:47:04.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.432+0000 7f22db489700 15 client.41186.objecter _session_command_op_assign 6 13 2024-02-23T00:47:04.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.432+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d412afd0 osd=6 4 2024-02-23T00:47:04.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.432+0000 7f22db489700 10 client.41186.objecter _send_command 13 2024-02-23T00:47:04.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.432+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f22d412e300 con 0x7f22d412a300 2024-02-23T00:47:04.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.432+0000 7f22d9a26700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f22d412a300 0x7f22d412a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:04.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.432+0000 7f22ca7fc700 10 client.41186.objecter ms_handle_connect 0x7f22d412a300 2024-02-23T00:47:04.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.432+0000 7f22d9a26700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f22d412a300 0x7f22d412a780 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:04.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.433+0000 7f22ca7fc700 1 -- 172.21.15.47:0/970342121 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f22d412e300 con 0x7f22d412a300 2024-02-23T00:47:04.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.433+0000 7f22ca7fc700 10 client.41186.objecter ms_dispatch 0x7f22d4001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:47:04.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.433+0000 7f22ca7fc700 10 client.41186.objecter _finish_command 13 = system:0 2024-02-23T00:47:04.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.433+0000 7f22ca7fc700 20 client.41186.objecter put_session s=0x7f22d412afd0 osd=6 4 2024-02-23T00:47:04.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.433+0000 7f22ca7fc700 15 client.41186.objecter _session_command_op_remove 6 13 2024-02-23T00:47:04.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:03.267819+0000 mgr.y (mgr.14517) 336 : cluster [DBG] pgmap v502: 250 pgs: 32 unknown, 218 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-23T00:47:04.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:04.181098+0000 mon.a (mon.0) 2277 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "hit_set_period","val": "600"}]': finished 2024-02-23T00:47:04.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:04.181289+0000 mon.a (mon.0) 2278 : audit [INF] from='client.? 172.21.15.47:0/2013956265' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi047-82386-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:47:04.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:04.181368+0000 mon.a (mon.0) 2279 : cluster [DBG] osdmap e352: 8 total, 8 up, 8 in 2024-02-23T00:47:04.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:04.184655+0000 mon.a (mon.0) 2280 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-23T00:47:04.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.448+0000 7f22db489700 10 client.41186.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:47:04.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.448+0000 7f22db489700 15 client.41186.objecter _session_command_op_assign -1 14 2024-02-23T00:47:04.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.448+0000 7f22db489700 20 client.41186.objecter _get_session s=0x7f22d412afd0 osd=6 3 2024-02-23T00:47:04.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.448+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d412afd0 osd=6 3 2024-02-23T00:47:04.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.448+0000 7f22db489700 20 client.41186.objecter _get_session s=0x7f22d412afd0 osd=6 3 2024-02-23T00:47:04.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.448+0000 7f22db489700 15 client.41186.objecter _session_command_op_remove -1 14 2024-02-23T00:47:04.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.448+0000 7f22db489700 20 client.41186.objecter get_session s=0x7f22d412afd0 osd=6 3 2024-02-23T00:47:04.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.448+0000 7f22db489700 15 client.41186.objecter _session_command_op_assign 6 14 2024-02-23T00:47:04.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.448+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d412afd0 osd=6 4 2024-02-23T00:47:04.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.448+0000 7f22db489700 10 client.41186.objecter _send_command 14 2024-02-23T00:47:04.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.448+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f22d4128790 con 0x7f22d412a300 2024-02-23T00:47:04.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.449+0000 7f22ca7fc700 1 -- 172.21.15.47:0/970342121 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f22d4128790 con 0x7f22d412a300 2024-02-23T00:47:04.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.449+0000 7f22ca7fc700 10 client.41186.objecter ms_dispatch 0x7f22d4001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:47:04.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.449+0000 7f22ca7fc700 10 client.41186.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:47:04.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.449+0000 7f22ca7fc700 20 client.41186.objecter put_session s=0x7f22d412afd0 osd=6 4 2024-02-23T00:47:04.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.449+0000 7f22ca7fc700 15 client.41186.objecter _session_command_op_remove 6 14 2024-02-23T00:47:04.452 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:47:04.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.450+0000 7f22db489700 10 client.41186.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:04.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.450+0000 7f22db489700 15 client.41186.objecter _session_command_op_assign -1 15 2024-02-23T00:47:04.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.450+0000 7f22db489700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f22d4128e10 0x7f22d4129270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:04.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.450+0000 7f22db489700 20 client.41186.objecter _get_session s=0x7f22d4128790 osd=7 3 2024-02-23T00:47:04.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.450+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d4128790 osd=7 3 2024-02-23T00:47:04.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.450+0000 7f22db489700 20 client.41186.objecter _get_session s=0x7f22d4128790 osd=7 3 2024-02-23T00:47:04.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.450+0000 7f22db489700 15 client.41186.objecter _session_command_op_remove -1 15 2024-02-23T00:47:04.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.450+0000 7f22db489700 20 client.41186.objecter get_session s=0x7f22d4128790 osd=7 3 2024-02-23T00:47:04.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.450+0000 7f22db489700 15 client.41186.objecter _session_command_op_assign 7 15 2024-02-23T00:47:04.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.450+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d4128790 osd=7 4 2024-02-23T00:47:04.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.450+0000 7f22db489700 10 client.41186.objecter _send_command 15 2024-02-23T00:47:04.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.450+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f22d4129900 con 0x7f22d4128e10 2024-02-23T00:47:04.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.450+0000 7f22d9225700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f22d4128e10 0x7f22d4129270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:04.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.450+0000 7f22d9225700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f22d4128e10 0x7f22d4129270 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:04.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.450+0000 7f22ca7fc700 10 client.41186.objecter ms_handle_connect 0x7f22d4128e10 2024-02-23T00:47:04.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.451+0000 7f22ca7fc700 1 -- 172.21.15.47:0/970342121 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f22d4129900 con 0x7f22d4128e10 2024-02-23T00:47:04.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.451+0000 7f22ca7fc700 10 client.41186.objecter ms_dispatch 0x7f22d4001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:47:04.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.451+0000 7f22ca7fc700 10 client.41186.objecter _finish_command 15 = system:0 2024-02-23T00:47:04.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.451+0000 7f22ca7fc700 20 client.41186.objecter put_session s=0x7f22d4128790 osd=7 4 2024-02-23T00:47:04.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.451+0000 7f22ca7fc700 15 client.41186.objecter _session_command_op_remove 7 15 2024-02-23T00:47:04.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.466+0000 7f22db489700 10 client.41186.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:47:04.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.466+0000 7f22db489700 15 client.41186.objecter _session_command_op_assign -1 16 2024-02-23T00:47:04.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.466+0000 7f22db489700 20 client.41186.objecter _get_session s=0x7f22d4128790 osd=7 3 2024-02-23T00:47:04.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.466+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d4128790 osd=7 3 2024-02-23T00:47:04.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.466+0000 7f22db489700 20 client.41186.objecter _get_session s=0x7f22d4128790 osd=7 3 2024-02-23T00:47:04.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.466+0000 7f22db489700 15 client.41186.objecter _session_command_op_remove -1 16 2024-02-23T00:47:04.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.466+0000 7f22db489700 20 client.41186.objecter get_session s=0x7f22d4128790 osd=7 3 2024-02-23T00:47:04.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.466+0000 7f22db489700 15 client.41186.objecter _session_command_op_assign 7 16 2024-02-23T00:47:04.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.466+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d4128790 osd=7 4 2024-02-23T00:47:04.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.466+0000 7f22db489700 10 client.41186.objecter _send_command 16 2024-02-23T00:47:04.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.466+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f22d4129900 con 0x7f22d4128e10 2024-02-23T00:47:04.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.467+0000 7f22ca7fc700 1 -- 172.21.15.47:0/970342121 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f22d4129900 con 0x7f22d4128e10 2024-02-23T00:47:04.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.467+0000 7f22ca7fc700 10 client.41186.objecter ms_dispatch 0x7f22d4001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:47:04.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.467+0000 7f22ca7fc700 10 client.41186.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:47:04.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.467+0000 7f22ca7fc700 20 client.41186.objecter put_session s=0x7f22d4128790 osd=7 4 2024-02-23T00:47:04.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.467+0000 7f22ca7fc700 15 client.41186.objecter _session_command_op_remove 7 16 2024-02-23T00:47:04.470 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:47:04.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 10 librados: watch_flush enter 2024-02-23T00:47:04.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 10 librados: watch_flush exit 2024-02-23T00:47:04.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 10 client.41186.objecter close_session for osd.0 2024-02-23T00:47:04.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f22d40fdd70 msgr2=0x7f22d40fe150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:04.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f22d40fdd70 0x7f22d40fe150 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:04.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d405ee80 osd=0 1 2024-02-23T00:47:04.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 10 client.41186.objecter close_session for osd.1 2024-02-23T00:47:04.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f22d40598f0 msgr2=0x7f22d4059cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:04.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f22d40598f0 0x7f22d4059cf0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:04.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d4028710 osd=1 1 2024-02-23T00:47:04.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 10 client.41186.objecter close_session for osd.2 2024-02-23T00:47:04.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f22d4028270 msgr2=0x7f22d411e130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:04.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f22d4028270 0x7f22d411e130 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:04.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d4059710 osd=2 1 2024-02-23T00:47:04.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 10 client.41186.objecter close_session for osd.3 2024-02-23T00:47:04.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f22d411f7a0 msgr2=0x7f22d411fba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:04.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f22d411f7a0 0x7f22d411fba0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:04.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22d9a26700 1 -- 172.21.15.47:0/970342121 reap_dead start 2024-02-23T00:47:04.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d411b770 osd=3 1 2024-02-23T00:47:04.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 10 client.41186.objecter close_session for osd.4 2024-02-23T00:47:04.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f22d4122f30 msgr2=0x7f22d4125320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:04.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f22d4122f30 0x7f22d4125320 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22d9a26700 1 -- 172.21.15.47:0/970342121 reap_dead start 2024-02-23T00:47:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d4120860 osd=4 1 2024-02-23T00:47:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 10 client.41186.objecter close_session for osd.5 2024-02-23T00:47:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f22d4125d20 msgr2=0x7f22d4128190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f22d4125d20 0x7f22d4128190 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d4120b70 osd=5 1 2024-02-23T00:47:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 10 client.41186.objecter close_session for osd.6 2024-02-23T00:47:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f22d412a300 msgr2=0x7f22d412a780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f22d412a300 0x7f22d412a780 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d412afd0 osd=6 1 2024-02-23T00:47:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 10 client.41186.objecter close_session for osd.7 2024-02-23T00:47:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f22d4128e10 msgr2=0x7f22d4129270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f22d4128e10 0x7f22d4129270 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 20 client.41186.objecter put_session s=0x7f22d4128790 osd=7 1 2024-02-23T00:47:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 20 client.41186.objecter shutdown clearing up homeless session... 2024-02-23T00:47:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 10 client.41186.objecter successfully canceled tick 2024-02-23T00:47:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f22c0062cc0 msgr2=0x7f22c0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f22c0062cc0 0x7f22c0065180 secure :-1 s=READY pgs=3038 cs=0 l=1 rev1=1 rx=0x7f22d00048b0 tx=0x7f22d000b040).stop 2024-02-23T00:47:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f22d40fd420 msgr2=0x7f22d4109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 1 --2- 172.21.15.47:0/970342121 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f22d40fd420 0x7f22d4109a50 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 rx=0x7f22cc009790 tx=0x7f22cc004d10).stop 2024-02-23T00:47:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22d9a26700 1 -- 172.21.15.47:0/970342121 reap_dead start 2024-02-23T00:47:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22d9a26700 1 -- 172.21.15.47:0/970342121 reap_dead start 2024-02-23T00:47:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 shutdown_connections 2024-02-23T00:47:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 >> 172.21.15.47:0/970342121 conn(0x7f22d40f2710 msgr2=0x7f22d40f4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 shutdown_connections 2024-02-23T00:47:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 1 -- 172.21.15.47:0/970342121 wait complete. 2024-02-23T00:47:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:04.468+0000 7f22db489700 1 librados: shutdown 2024-02-23T00:47:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:47:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:47:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:47:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:47:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:47:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:47:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:47:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:47:04.703 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:47:04.705 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:04.706 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:47:04.807 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:47:04.834 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:05.081 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:47:05.084 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:05.084 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:47:05.185 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:47:05.217 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:05.237 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:47:05.243 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:05.244 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:47:05.345 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:47:05.348 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:04.319056+0000 mon.c (mon.2) 271 : audit [DBG] from='client.? 172.21.15.47:0/970342121' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:47:05.348 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:05.156757+0000 mon.a (mon.0) 2281 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:47:05.349 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:05.161261+0000 mon.a (mon.0) 2282 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-23T00:47:05.349 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:05.161347+0000 mon.a (mon.0) 2283 : cluster [DBG] osdmap e353: 8 total, 8 up, 8 in 2024-02-23T00:47:05.349 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:05.166087+0000 mon.a (mon.0) 2284 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-23T00:47:05.349 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:05.184741+0000 mon.a (mon.0) 2285 : audit [INF] from='client.? 172.21.15.47:0/3942253766' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi047-82386-60"}]: dispatch 2024-02-23T00:47:05.349 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:05.185521+0000 mon.a (mon.0) 2286 : audit [INF] from='client.? 172.21.15.47:0/3942253766' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi047-82386-60"}]: dispatch 2024-02-23T00:47:05.349 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:05.185861+0000 mon.a (mon.0) 2287 : audit [INF] from='client.? 172.21.15.47:0/3942253766' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi047-82386-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:47:05.373 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:05.466 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:04.319056+0000 mon.c (mon.2) 271 : audit [DBG] from='client.? 172.21.15.47:0/970342121' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:47:05.467 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:05.156757+0000 mon.a (mon.0) 2281 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:47:05.467 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:05.161261+0000 mon.a (mon.0) 2282 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-23T00:47:05.467 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:05.161347+0000 mon.a (mon.0) 2283 : cluster [DBG] osdmap e353: 8 total, 8 up, 8 in 2024-02-23T00:47:05.467 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:05.166087+0000 mon.a (mon.0) 2284 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-23T00:47:05.467 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:05.184741+0000 mon.a (mon.0) 2285 : audit [INF] from='client.? 172.21.15.47:0/3942253766' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi047-82386-60"}]: dispatch 2024-02-23T00:47:05.467 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:05.185521+0000 mon.a (mon.0) 2286 : audit [INF] from='client.? 172.21.15.47:0/3942253766' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi047-82386-60"}]: dispatch 2024-02-23T00:47:05.467 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:05.185861+0000 mon.a (mon.0) 2287 : audit [INF] from='client.? 172.21.15.47:0/3942253766' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi047-82386-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:47:05.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:04.319056+0000 mon.c (mon.2) 271 : audit [DBG] from='client.? 172.21.15.47:0/970342121' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:47:05.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:05.156757+0000 mon.a (mon.0) 2281 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:47:05.468 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:05.161261+0000 mon.a (mon.0) 2282 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-23T00:47:05.468 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:05.161347+0000 mon.a (mon.0) 2283 : cluster [DBG] osdmap e353: 8 total, 8 up, 8 in 2024-02-23T00:47:05.468 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:05.166087+0000 mon.a (mon.0) 2284 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-23T00:47:05.468 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:05.184741+0000 mon.a (mon.0) 2285 : audit [INF] from='client.? 172.21.15.47:0/3942253766' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi047-82386-60"}]: dispatch 2024-02-23T00:47:05.468 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:05.185521+0000 mon.a (mon.0) 2286 : audit [INF] from='client.? 172.21.15.47:0/3942253766' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi047-82386-60"}]: dispatch 2024-02-23T00:47:05.468 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:05.185861+0000 mon.a (mon.0) 2287 : audit [INF] from='client.? 172.21.15.47:0/3942253766' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi047-82386-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:47:05.613 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:47:05.614 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-02-23T00:47:05.614 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-02-23T00:47:05.614 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-02-23T00:47:05.614 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:47:05.622 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:47:05.624 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:05.625 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:47:05.693 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset noscrub 2024-02-23T00:47:05.725 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:47:05.753 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:05.851 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:47:05.854 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:05.854 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:47:05.955 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:47:05.982 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:06.104 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:47:06.303 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.6 dump_historic_ops 2024-02-23T00:47:06.330 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:47:06.333 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:06.333 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:47:06.434 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:47:06.464 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:06.549 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:05.268281+0000 mgr.y (mgr.14517) 337 : cluster [DBG] pgmap v505: 218 pgs: 218 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-23T00:47:06.549 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:06.162766+0000 mon.a (mon.0) 2288 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-23T00:47:06.549 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:06.162843+0000 mon.a (mon.0) 2289 : audit [INF] from='client.? 172.21.15.47:0/3942253766' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi047-82386-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:47:06.549 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:06.162898+0000 mon.a (mon.0) 2290 : cluster [DBG] osdmap e354: 8 total, 8 up, 8 in 2024-02-23T00:47:06.549 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:06.164506+0000 mon.a (mon.0) 2291 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-23T00:47:06.549 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:06.164822+0000 mon.a (mon.0) 2292 : audit [INF] from='client.? 172.21.15.47:0/3942253766' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi047-82386-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi047-82386-60"}]: dispatch 2024-02-23T00:47:06.549 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:05.268281+0000 mgr.y (mgr.14517) 337 : cluster [DBG] pgmap v505: 218 pgs: 218 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-23T00:47:06.550 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:06.162766+0000 mon.a (mon.0) 2288 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-23T00:47:06.550 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:06.162843+0000 mon.a (mon.0) 2289 : audit [INF] from='client.? 172.21.15.47:0/3942253766' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi047-82386-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:47:06.550 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:06.162898+0000 mon.a (mon.0) 2290 : cluster [DBG] osdmap e354: 8 total, 8 up, 8 in 2024-02-23T00:47:06.550 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:06.164506+0000 mon.a (mon.0) 2291 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-23T00:47:06.550 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:06.164822+0000 mon.a (mon.0) 2292 : audit [INF] from='client.? 172.21.15.47:0/3942253766' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi047-82386-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi047-82386-60"}]: dispatch 2024-02-23T00:47:06.599 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:47:06.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:05.268281+0000 mgr.y (mgr.14517) 337 : cluster [DBG] pgmap v505: 218 pgs: 218 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-23T00:47:06.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:06.162766+0000 mon.a (mon.0) 2288 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-23T00:47:06.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:06.162843+0000 mon.a (mon.0) 2289 : audit [INF] from='client.? 172.21.15.47:0/3942253766' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi047-82386-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:47:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:06.162898+0000 mon.a (mon.0) 2290 : cluster [DBG] osdmap e354: 8 total, 8 up, 8 in 2024-02-23T00:47:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:06.164506+0000 mon.a (mon.0) 2291 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-23T00:47:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:06.164822+0000 mon.a (mon.0) 2292 : audit [INF] from='client.? 172.21.15.47:0/3942253766' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi047-82386-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi047-82386-60"}]: dispatch 2024-02-23T00:47:07.113 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:47:07.116 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:07.117 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:47:07.217 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:47:07.247 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:07.259 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:47:07.395 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:47:07.397 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:07.398 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:47:07.498 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:47:07.526 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:08.074 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:47:08.078 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:08.078 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:47:08.179 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:47:08.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.192+0000 7fbbbf43e700 1 -- 172.21.15.47:0/3817030886 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbbb80f7400 msgr2=0x7fbbb8103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:08.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.192+0000 7fbbbf43e700 1 --2- 172.21.15.47:0/3817030886 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbbb80f7400 0x7fbbb8103a20 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 rx=0x7fbbb0009c30 tx=0x7fbbb0004c50).stop 2024-02-23T00:47:08.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.192+0000 7fbbbf43e700 1 -- 172.21.15.47:0/3817030886 shutdown_connections 2024-02-23T00:47:08.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.192+0000 7fbbbf43e700 1 --2- 172.21.15.47:0/3817030886 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbbb80f7400 0x7fbbb8103a20 unknown :-1 s=CLOSED pgs=2636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:08.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.192+0000 7fbbbf43e700 1 --2- 172.21.15.47:0/3817030886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbb80fcbb0 0x7fbbb80fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:08.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.192+0000 7fbbbf43e700 1 --2- 172.21.15.47:0/3817030886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbbb80fd510 0x7fbbb80f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:08.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.192+0000 7fbbbf43e700 1 -- 172.21.15.47:0/3817030886 >> 172.21.15.47:0/3817030886 conn(0x7fbbb80f2820 msgr2=0x7fbbb80f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:08.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.193+0000 7fbbbf43e700 1 -- 172.21.15.47:0/3817030886 shutdown_connections 2024-02-23T00:47:08.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.193+0000 7fbbbf43e700 1 -- 172.21.15.47:0/3817030886 wait complete. 2024-02-23T00:47:08.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.193+0000 7fbbbf43e700 1 librados: starting msgr at 2024-02-23T00:47:08.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.193+0000 7fbbbf43e700 1 librados: starting objecter 2024-02-23T00:47:08.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.193+0000 7fbbbf43e700 1 Processor -- start 2024-02-23T00:47:08.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.193+0000 7fbbbf43e700 1 -- start start 2024-02-23T00:47:08.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.193+0000 7fbbbf43e700 1 librados: setting wanted keys 2024-02-23T00:47:08.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.193+0000 7fbbbf43e700 1 librados: calling monclient init 2024-02-23T00:47:08.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.194+0000 7fbbbf43e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbbb80f7400 0x7fbbb810de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:08.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.194+0000 7fbbbf43e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbb80fcbb0 0x7fbbb810e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:08.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.194+0000 7fbbbf43e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbbb80fd510 0x7fbbb8107ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:08.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.194+0000 7fbbbf43e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb81084e0 con 0x7fbbb80fcbb0 2024-02-23T00:47:08.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.194+0000 7fbbbf43e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb8108650 con 0x7fbbb80f7400 2024-02-23T00:47:08.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.194+0000 7fbbbf43e700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb8108810 con 0x7fbbb80fd510 2024-02-23T00:47:08.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.194+0000 7fbbbc9d9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbb80fcbb0 0x7fbbb810e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:08.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.194+0000 7fbbbd1da700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbbb80f7400 0x7fbbb810de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:08.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.194+0000 7fbbbc9d9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbb80fcbb0 0x7fbbb810e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48808/0 (socket says 172.21.15.47:48808) 2024-02-23T00:47:08.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.194+0000 7fbbbc9d9700 1 -- 172.21.15.47:0/3005576393 learned_addr learned my addr 172.21.15.47:0/3005576393 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:47:08.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.194+0000 7fbbbd9db700 1 --2- 172.21.15.47:0/3005576393 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbbb80fd510 0x7fbbb8107ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:08.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.194+0000 7fbbbd1da700 1 -- 172.21.15.47:0/3005576393 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbbb80fd510 msgr2=0x7fbbb8107ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:08.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.194+0000 7fbbbd1da700 1 --2- 172.21.15.47:0/3005576393 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbbb80fd510 0x7fbbb8107ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:08.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.194+0000 7fbbbd1da700 1 -- 172.21.15.47:0/3005576393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbb80fcbb0 msgr2=0x7fbbb810e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:08.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.194+0000 7fbbbd1da700 1 --2- 172.21.15.47:0/3005576393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbb80fcbb0 0x7fbbb810e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:08.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.194+0000 7fbbbd1da700 1 -- 172.21.15.47:0/3005576393 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb4005040 con 0x7fbbb80f7400 2024-02-23T00:47:08.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.194+0000 7fbbbc9d9700 1 --2- 172.21.15.47:0/3005576393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbb80fcbb0 0x7fbbb810e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:47:08.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.194+0000 7fbbbd9db700 1 --2- 172.21.15.47:0/3005576393 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbbb80fd510 0x7fbbb8107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-23T00:47:08.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.194+0000 7fbbbd1da700 1 --2- 172.21.15.47:0/3005576393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbbb80f7400 0x7fbbb810de50 secure :-1 s=READY pgs=3974 cs=0 l=1 rev1=1 rx=0x7fbbb40091f0 tx=0x7fbbb4009cb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:08.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.194+0000 7fbbae7fc700 10 client.?.objecter ms_handle_connect 0x7fbbb80f7400 2024-02-23T00:47:08.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.194+0000 7fbbae7fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:47:08.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.195+0000 7fbbae7fc700 1 -- 172.21.15.47:0/3005576393 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb40079e0 con 0x7fbbb80f7400 2024-02-23T00:47:08.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.195+0000 7fbbae7fc700 1 -- 172.21.15.47:0/3005576393 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fbbb400f040 con 0x7fbbb80f7400 2024-02-23T00:47:08.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.195+0000 7fbbbf43e700 1 -- 172.21.15.47:0/3005576393 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb00097c0 con 0x7fbbb80f7400 2024-02-23T00:47:08.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.195+0000 7fbbbf43e700 10 client.43255.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:47:08.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.195+0000 7fbbbf43e700 1 -- 172.21.15.47:0/3005576393 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb8108e50 con 0x7fbbb80f7400 2024-02-23T00:47:08.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.195+0000 7fbbbf43e700 1 librados: init done 2024-02-23T00:47:08.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.195+0000 7fbbae7fc700 1 -- 172.21.15.47:0/3005576393 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb40147e0 con 0x7fbbb80f7400 2024-02-23T00:47:08.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.196+0000 7fbbbf43e700 1 -- 172.21.15.47:0/3005576393 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb9c004f50 con 0x7fbbb80f7400 2024-02-23T00:47:08.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.196+0000 7fbbae7fc700 1 -- 172.21.15.47:0/3005576393 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fbbb4014c30 con 0x7fbbb80f7400 2024-02-23T00:47:08.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.196+0000 7fbbae7fc700 1 --2- 172.21.15.47:0/3005576393 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbba4062cc0 0x7fbba4065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:08.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.197+0000 7fbbae7fc700 1 -- 172.21.15.47:0/3005576393 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(356..356 src has 48..356) v4 ==== 10017+0+0 (secure 0 0 0) 0x7fbbb407d5f0 con 0x7fbbb80f7400 2024-02-23T00:47:08.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.197+0000 7fbbae7fc700 10 client.43255.objecter ms_dispatch 0x7fbbb8001550 osd_map(356..356 src has 48..356) v4 2024-02-23T00:47:08.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.197+0000 7fbbae7fc700 3 client.43255.objecter handle_osd_map got epochs [356,356] > 0 2024-02-23T00:47:08.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.197+0000 7fbbae7fc700 3 client.43255.objecter handle_osd_map decoding full epoch 356 2024-02-23T00:47:08.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.197+0000 7fbbbc9d9700 1 --2- 172.21.15.47:0/3005576393 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbba4062cc0 0x7fbba4065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:08.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.197+0000 7fbbae7fc700 20 client.43255.objecter dump_active .. 0 homeless 2024-02-23T00:47:08.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.197+0000 7fbbae7fc700 10 client.43255.objecter ms_handle_connect 0x7fbba4062cc0 2024-02-23T00:47:08.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.197+0000 7fbbbc9d9700 1 --2- 172.21.15.47:0/3005576393 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbba4062cc0 0x7fbba4065180 secure :-1 s=READY pgs=3040 cs=0 l=1 rev1=1 rx=0x7fbbb8109790 tx=0x7fbba800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:08.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.202+0000 7fbbae7fc700 1 -- 172.21.15.47:0/3005576393 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fbbb4049f60 con 0x7fbbb80f7400 2024-02-23T00:47:08.210 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:08.247 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:47:08.249 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:08.250 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:47:08.351 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:47:08.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.372+0000 7fbbbf43e700 1 -- 172.21.15.47:0/3005576393 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbb9c000bf0 con 0x7fbba4062cc0 2024-02-23T00:47:08.377 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:47:08.377 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-02-23T00:47:08.377 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-02-23T00:47:08.377 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-02-23T00:47:08.377 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.377 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.24311.0:91 5.1f 5:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e64)", 2024-02-23T00:47:08.377 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:39:18.792878+0000", 2024-02-23T00:47:08.377 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 469.579976988, 2024-02-23T00:47:08.377 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0095175999999999993, 2024-02-23T00:47:08.377 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:47:08.377 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:47:08.377 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:47:08.377 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.24311", 2024-02-23T00:47:08.378 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/3723202188", 2024-02-23T00:47:08.378 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 91 2024-02-23T00:47:08.378 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.378 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:47:08.378 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.378 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:47:08.378 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:18.792878+0000", 2024-02-23T00:47:08.378 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:47:08.378 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.378 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.378 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:47:08.378 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:18.792878+0000", 2024-02-23T00:47:08.378 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:47:08.378 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.378 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.379 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:47:08.379 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:18.792876+0000", 2024-02-23T00:47:08.379 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8930000000000004e-06 2024-02-23T00:47:08.379 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.379 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.379 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:47:08.379 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:18.792881+0000", 2024-02-23T00:47:08.379 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4120000000000007e-06 2024-02-23T00:47:08.379 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.379 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.379 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:47:08.379 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:18.792891+0000", 2024-02-23T00:47:08.379 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6730000000000003e-06 2024-02-23T00:47:08.379 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.379 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.380 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:47:08.380 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:18.792897+0000", 2024-02-23T00:47:08.380 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2461999999999998e-05 2024-02-23T00:47:08.380 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.380 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.380 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:47:08.380 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:18.792970+0000", 2024-02-23T00:47:08.380 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4780000000000002e-06 2024-02-23T00:47:08.380 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.380 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.380 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:47:08.380 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:18.792977+0000", 2024-02-23T00:47:08.380 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0045110549999999999 2024-02-23T00:47:08.380 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.381 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.381 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:47:08.381 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:18.797488+0000", 2024-02-23T00:47:08.381 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6799999999999997e-05 2024-02-23T00:47:08.381 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.381 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.381 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:47:08.381 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:18.797555+0000", 2024-02-23T00:47:08.381 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012065099999999999 2024-02-23T00:47:08.381 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.381 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.381 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 4,7", 2024-02-23T00:47:08.381 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:18.797676+0000", 2024-02-23T00:47:08.381 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00068942600000000004 2024-02-23T00:47:08.382 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.382 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.382 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:47:08.382 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:18.798365+0000", 2024-02-23T00:47:08.382 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0022167089999999999 2024-02-23T00:47:08.382 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.382 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.382 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:08.382 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:18.800582+0000", 2024-02-23T00:47:08.382 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0017482750000000001 2024-02-23T00:47:08.382 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.382 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.382 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:08.382 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:18.802330+0000", 2024-02-23T00:47:08.383 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.6409999999999999e-05 2024-02-23T00:47:08.383 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.383 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.383 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:47:08.383 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:18.802367+0000", 2024-02-23T00:47:08.383 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.885e-05 2024-02-23T00:47:08.383 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.383 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.383 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:47:08.383 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:39:18.802395+0000", 2024-02-23T00:47:08.383 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0095175999999999993 2024-02-23T00:47:08.383 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.383 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:47:08.383 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.383 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.384 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.384 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.24775.0:2059 20.10 e112/109 20:0c23f028:test-rados-api-smithi047-82457-16::big:head v 112'2, mlcod=0'0)", 2024-02-23T00:47:08.384 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:01.336050+0000", 2024-02-23T00:47:08.384 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 247.036804953, 2024-02-23T00:47:08.384 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.096228685999999994, 2024-02-23T00:47:08.384 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:47:08.384 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:47:08.384 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:47:08.384 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.384 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:47:08.384 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.336050+0000", 2024-02-23T00:47:08.384 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:47:08.384 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.385 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.385 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:47:08.385 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.336050+0000", 2024-02-23T00:47:08.385 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:47:08.385 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.385 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.385 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:47:08.385 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.336049+0000", 2024-02-23T00:47:08.385 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.058788094999999999 2024-02-23T00:47:08.385 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.385 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.385 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:47:08.385 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.394837+0000", 2024-02-23T00:47:08.385 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5380000000000001e-05 2024-02-23T00:47:08.385 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.386 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.386 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:47:08.386 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.394862+0000", 2024-02-23T00:47:08.386 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4617000000000002e-05 2024-02-23T00:47:08.386 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.386 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.386 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:47:08.386 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.394887+0000", 2024-02-23T00:47:08.386 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0985000000000001e-05 2024-02-23T00:47:08.386 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.386 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.386 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:47:08.386 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.394928+0000", 2024-02-23T00:47:08.387 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5004e-05 2024-02-23T00:47:08.387 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.387 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.387 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:47:08.387 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.394953+0000", 2024-02-23T00:47:08.387 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037289546 2024-02-23T00:47:08.387 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.387 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.387 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:47:08.387 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.432242+0000", 2024-02-23T00:47:08.387 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.6211000000000003e-05 2024-02-23T00:47:08.387 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.387 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.387 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:47:08.387 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.432279+0000", 2024-02-23T00:47:08.387 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.096228685999999994 2024-02-23T00:47:08.388 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.388 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:47:08.388 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.388 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.388 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.388 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.24775.0:2061 20.12 20:4d27b8a4:test-rados-api-smithi047-82457-16::big.copy:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected e112)", 2024-02-23T00:47:08.388 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:01.433768+0000", 2024-02-23T00:47:08.388 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 246.93908650899999, 2024-02-23T00:47:08.388 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.075757888999999995, 2024-02-23T00:47:08.388 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:47:08.388 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:47:08.388 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:47:08.388 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.24775", 2024-02-23T00:47:08.388 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1191937413", 2024-02-23T00:47:08.389 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 2061 2024-02-23T00:47:08.389 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.389 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:47:08.389 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.389 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:47:08.389 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.433768+0000", 2024-02-23T00:47:08.389 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:47:08.389 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.389 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.389 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:47:08.389 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.433768+0000", 2024-02-23T00:47:08.389 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:47:08.389 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.389 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.389 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:47:08.389 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.433767+0000", 2024-02-23T00:47:08.390 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0820000000000004e-06 2024-02-23T00:47:08.390 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.390 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.390 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:47:08.390 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.433773+0000", 2024-02-23T00:47:08.390 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1532000000000001e-05 2024-02-23T00:47:08.390 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.390 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.390 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:47:08.390 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.433785+0000", 2024-02-23T00:47:08.390 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8760000000000004e-06 2024-02-23T00:47:08.390 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.390 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.390 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:47:08.390 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.433789+0000", 2024-02-23T00:47:08.390 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4573999999999998e-05 2024-02-23T00:47:08.391 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.391 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.391 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:47:08.391 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.433824+0000", 2024-02-23T00:47:08.391 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.7882999999999999e-05 2024-02-23T00:47:08.391 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.391 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.391 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:47:08.391 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.433912+0000", 2024-02-23T00:47:08.391 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.063142300999999998 2024-02-23T00:47:08.391 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.391 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.391 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1,5", 2024-02-23T00:47:08.391 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.497054+0000", 2024-02-23T00:47:08.391 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010301874000000001 2024-02-23T00:47:08.391 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.392 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.392 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:08.392 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.507356+0000", 2024-02-23T00:47:08.392 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0017280170000000001 2024-02-23T00:47:08.392 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.392 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.392 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:47:08.392 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.509084+0000", 2024-02-23T00:47:08.392 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000380064 2024-02-23T00:47:08.392 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.392 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.392 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:08.392 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.509464+0000", 2024-02-23T00:47:08.392 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.5374999999999999e-05 2024-02-23T00:47:08.393 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.393 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.393 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:47:08.393 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.509499+0000", 2024-02-23T00:47:08.393 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.6704e-05 2024-02-23T00:47:08.393 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.393 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.393 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:47:08.393 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.509526+0000", 2024-02-23T00:47:08.393 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.075757888999999995 2024-02-23T00:47:08.393 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.393 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:47:08.393 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.393 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.394 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.394 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.24775.0:2062 20.12 20:4d27b8a4:test-rados-api-smithi047-82457-16::big.copy:head [read 0~25165836] snapc 0=[] ondisk+read+known_if_redirected e112)", 2024-02-23T00:47:08.394 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:01.509764+0000", 2024-02-23T00:47:08.394 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 246.86309085600001, 2024-02-23T00:47:08.394 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017898609999999999, 2024-02-23T00:47:08.394 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:47:08.394 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:47:08.394 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:47:08.394 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.24775", 2024-02-23T00:47:08.394 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1191937413", 2024-02-23T00:47:08.394 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 2062 2024-02-23T00:47:08.394 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.394 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:47:08.394 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.394 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:47:08.395 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.509764+0000", 2024-02-23T00:47:08.395 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:47:08.395 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.395 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.395 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:47:08.395 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.509764+0000", 2024-02-23T00:47:08.395 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:47:08.395 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.395 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.395 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:47:08.395 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.509763+0000", 2024-02-23T00:47:08.395 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6070000000000001e-06 2024-02-23T00:47:08.395 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.395 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.395 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:47:08.396 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.509767+0000", 2024-02-23T00:47:08.396 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6620000000000005e-06 2024-02-23T00:47:08.396 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.396 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.396 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:47:08.396 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.509775+0000", 2024-02-23T00:47:08.396 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.692e-06 2024-02-23T00:47:08.396 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.396 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.396 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:47:08.396 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.509779+0000", 2024-02-23T00:47:08.396 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.1749999999999999e-05 2024-02-23T00:47:08.396 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.396 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.396 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:47:08.396 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.509810+0000", 2024-02-23T00:47:08.397 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1403999999999994e-05 2024-02-23T00:47:08.397 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.397 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.397 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:47:08.397 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.509902+0000", 2024-02-23T00:47:08.397 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017760796999999998 2024-02-23T00:47:08.397 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.397 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.397 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:47:08.397 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.527663+0000", 2024-02-23T00:47:08.397 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017898609999999999 2024-02-23T00:47:08.397 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.397 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:47:08.397 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.398 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.398 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.398 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.24775.0:2069 20.13 20:cde74feb:test-rados-api-smithi047-82457-17::big:head [writefull 0~25165828] snapc 0=[] ondisk+write+known_if_redirected e112)", 2024-02-23T00:47:08.398 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:01.673523+0000", 2024-02-23T00:47:08.398 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 246.69933175099999, 2024-02-23T00:47:08.398 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.11814886500000001, 2024-02-23T00:47:08.398 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:47:08.398 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:47:08.398 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:47:08.398 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.24775", 2024-02-23T00:47:08.398 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1191937413", 2024-02-23T00:47:08.398 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 2069 2024-02-23T00:47:08.398 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.398 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:47:08.398 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.399 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:47:08.399 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.673523+0000", 2024-02-23T00:47:08.399 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:47:08.399 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.399 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.399 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:47:08.399 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.673523+0000", 2024-02-23T00:47:08.399 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:47:08.399 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.399 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.399 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:47:08.399 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.673522+0000", 2024-02-23T00:47:08.399 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038374284000000002 2024-02-23T00:47:08.399 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.399 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.399 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:47:08.399 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.711896+0000", 2024-02-23T00:47:08.400 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.6546999999999998e-05 2024-02-23T00:47:08.400 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.400 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.400 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:47:08.400 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.711922+0000", 2024-02-23T00:47:08.400 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5843e-05 2024-02-23T00:47:08.400 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.400 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.400 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:47:08.400 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.711948+0000", 2024-02-23T00:47:08.400 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9999e-05 2024-02-23T00:47:08.400 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.400 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.400 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:47:08.400 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.711998+0000", 2024-02-23T00:47:08.401 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5387e-05 2024-02-23T00:47:08.401 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.401 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.401 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:47:08.401 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.712094+0000", 2024-02-23T00:47:08.401 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000114737 2024-02-23T00:47:08.401 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.401 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.401 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0,7", 2024-02-23T00:47:08.401 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.712208+0000", 2024-02-23T00:47:08.401 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.057665738000000001 2024-02-23T00:47:08.401 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.401 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.401 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:47:08.401 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.769874+0000", 2024-02-23T00:47:08.401 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00055062100000000005 2024-02-23T00:47:08.402 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.402 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.402 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:08.402 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.770425+0000", 2024-02-23T00:47:08.402 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021182478000000001 2024-02-23T00:47:08.402 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.402 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.402 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:08.402 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.791607+0000", 2024-02-23T00:47:08.402 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.6732e-05 2024-02-23T00:47:08.402 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.402 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.402 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:47:08.402 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.791644+0000", 2024-02-23T00:47:08.402 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.8017e-05 2024-02-23T00:47:08.403 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.403 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.403 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:47:08.403 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.791672+0000", 2024-02-23T00:47:08.403 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.11814886500000001 2024-02-23T00:47:08.403 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.403 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:47:08.403 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.403 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.403 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.403 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.25117.0:5 29.d 29:b3c717c4:::4.obj:head [write 0~81] snapc 0=[] ondisk+write+known_if_redirected e112)", 2024-02-23T00:47:08.403 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:01.872132+0000", 2024-02-23T00:47:08.403 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 246.50072265700001, 2024-02-23T00:47:08.403 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.016411598999999999, 2024-02-23T00:47:08.403 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:47:08.404 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:47:08.404 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:47:08.404 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.25117", 2024-02-23T00:47:08.404 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/2400431714", 2024-02-23T00:47:08.404 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 5 2024-02-23T00:47:08.404 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.404 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:47:08.404 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.404 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:47:08.404 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.872132+0000", 2024-02-23T00:47:08.404 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:47:08.404 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.404 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.404 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:47:08.404 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.872132+0000", 2024-02-23T00:47:08.405 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:47:08.405 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.405 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.405 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:47:08.405 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.872130+0000", 2024-02-23T00:47:08.405 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1380000000000001e-06 2024-02-23T00:47:08.405 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.405 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.405 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:47:08.405 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.872137+0000", 2024-02-23T00:47:08.405 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6099999999999995e-06 2024-02-23T00:47:08.405 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.405 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.405 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:47:08.405 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.872147+0000", 2024-02-23T00:47:08.406 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.5269999999999998e-06 2024-02-23T00:47:08.406 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.406 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.406 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:47:08.406 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.872156+0000", 2024-02-23T00:47:08.406 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3844000000000002e-05 2024-02-23T00:47:08.406 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.406 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.406 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:47:08.406 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.872199+0000", 2024-02-23T00:47:08.406 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5797999999999995e-05 2024-02-23T00:47:08.406 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.406 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.406 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:47:08.406 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.872275+0000", 2024-02-23T00:47:08.406 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011332600000000001 2024-02-23T00:47:08.406 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.407 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.407 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1,5", 2024-02-23T00:47:08.407 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.872389+0000", 2024-02-23T00:47:08.407 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00063997499999999998 2024-02-23T00:47:08.407 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.407 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.407 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:08.407 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.873029+0000", 2024-02-23T00:47:08.407 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013714357 2024-02-23T00:47:08.407 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.407 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.407 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:47:08.407 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.886743+0000", 2024-02-23T00:47:08.407 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0017436089999999999 2024-02-23T00:47:08.407 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.408 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.408 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:08.408 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.888487+0000", 2024-02-23T00:47:08.408 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.2922000000000001e-05 2024-02-23T00:47:08.408 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.408 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.408 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:47:08.408 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.888519+0000", 2024-02-23T00:47:08.408 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4289000000000001e-05 2024-02-23T00:47:08.408 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.408 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.408 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:47:08.408 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.888544+0000", 2024-02-23T00:47:08.408 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.016411598999999999 2024-02-23T00:47:08.409 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.409 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:47:08.409 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.409 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.409 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.409 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.15165.0:8 29.10 29:0cdc28ab:::47.obj:head [write 0~70] snapc 0=[] ondisk+write+known_if_redirected e112)", 2024-02-23T00:47:08.409 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:01.975422+0000", 2024-02-23T00:47:08.409 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 246.39743248600001, 2024-02-23T00:47:08.409 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022703524999999999, 2024-02-23T00:47:08.409 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:47:08.409 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:47:08.409 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:47:08.409 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.15165", 2024-02-23T00:47:08.409 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1835198847", 2024-02-23T00:47:08.409 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 8 2024-02-23T00:47:08.409 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.410 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:47:08.410 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.410 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:47:08.410 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.975422+0000", 2024-02-23T00:47:08.410 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:47:08.410 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.410 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.410 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:47:08.410 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.975422+0000", 2024-02-23T00:47:08.410 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:47:08.410 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.410 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.410 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:47:08.410 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.975420+0000", 2024-02-23T00:47:08.410 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1963999999999999e-05 2024-02-23T00:47:08.411 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.411 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.411 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:47:08.411 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.975432+0000", 2024-02-23T00:47:08.411 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0162000000000001e-05 2024-02-23T00:47:08.411 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.411 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.411 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:47:08.411 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.975452+0000", 2024-02-23T00:47:08.411 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3839999999999997e-06 2024-02-23T00:47:08.411 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.411 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.411 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:47:08.411 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.975461+0000", 2024-02-23T00:47:08.411 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0436999999999998e-05 2024-02-23T00:47:08.411 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.412 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.412 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:47:08.412 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.975511+0000", 2024-02-23T00:47:08.412 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.8695999999999993e-05 2024-02-23T00:47:08.412 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.412 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.412 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:47:08.412 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.975590+0000", 2024-02-23T00:47:08.412 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012875899999999999 2024-02-23T00:47:08.412 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.412 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.412 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0,4", 2024-02-23T00:47:08.412 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.975719+0000", 2024-02-23T00:47:08.412 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033619700000000001 2024-02-23T00:47:08.413 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.413 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.413 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:08.413 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.979081+0000", 2024-02-23T00:47:08.413 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.018800372999999999 2024-02-23T00:47:08.413 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.413 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.413 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:47:08.413 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.997881+0000", 2024-02-23T00:47:08.413 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016630700000000001 2024-02-23T00:47:08.413 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.413 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.413 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:08.413 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.998047+0000", 2024-02-23T00:47:08.414 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.5732000000000003e-05 2024-02-23T00:47:08.414 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.414 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.414 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:47:08.414 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.998083+0000", 2024-02-23T00:47:08.414 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.2759000000000002e-05 2024-02-23T00:47:08.414 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.414 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.414 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:47:08.414 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.998126+0000", 2024-02-23T00:47:08.414 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022703524999999999 2024-02-23T00:47:08.414 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.414 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:47:08.414 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.414 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.415 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.415 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.15270.0:109 62.17 62:eabbf5ab:test-rados-api-smithi047-82465-10::foo:head [writefull 0~8] snapc 0=[] ondisk+write+known_if_redirected e137)", 2024-02-23T00:47:08.415 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:28.549238+0000", 2024-02-23T00:47:08.415 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 219.823616612, 2024-02-23T00:47:08.415 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.098006276000000003, 2024-02-23T00:47:08.415 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:47:08.415 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:47:08.415 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:47:08.415 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.15270", 2024-02-23T00:47:08.415 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/4133111990", 2024-02-23T00:47:08.415 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 109 2024-02-23T00:47:08.415 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.415 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:47:08.415 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.416 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:47:08.416 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:28.549238+0000", 2024-02-23T00:47:08.416 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:47:08.416 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.416 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.416 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:47:08.416 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:28.549238+0000", 2024-02-23T00:47:08.416 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:47:08.416 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.416 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.416 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:47:08.416 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:28.549236+0000", 2024-02-23T00:47:08.416 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8490000000000001e-06 2024-02-23T00:47:08.416 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.417 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.417 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:47:08.417 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:28.549246+0000", 2024-02-23T00:47:08.417 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2259000000000001e-05 2024-02-23T00:47:08.417 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.417 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.417 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:47:08.417 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:28.549258+0000", 2024-02-23T00:47:08.417 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.8800000000000008e-06 2024-02-23T00:47:08.417 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.417 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.417 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:47:08.417 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:28.549266+0000", 2024-02-23T00:47:08.417 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014179700000000001 2024-02-23T00:47:08.417 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.417 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.418 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:47:08.418 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:28.549408+0000", 2024-02-23T00:47:08.418 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.798e-06 2024-02-23T00:47:08.418 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.418 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.418 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op must wait for map", 2024-02-23T00:47:08.418 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:28.549415+0000", 2024-02-23T00:47:08.418 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013846720999999999 2024-02-23T00:47:08.418 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.418 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.418 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:47:08.418 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:28.563262+0000", 2024-02-23T00:47:08.418 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012987799999999999 2024-02-23T00:47:08.418 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.419 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.419 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:47:08.419 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:28.563391+0000", 2024-02-23T00:47:08.419 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014830900000000001 2024-02-23T00:47:08.419 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.419 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.419 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 3,7", 2024-02-23T00:47:08.419 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:28.563540+0000", 2024-02-23T00:47:08.419 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010630498 2024-02-23T00:47:08.419 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.419 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.419 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:47:08.419 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:28.574170+0000", 2024-02-23T00:47:08.419 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00020087300000000001 2024-02-23T00:47:08.419 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.419 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.420 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:08.420 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:28.574371+0000", 2024-02-23T00:47:08.420 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.072805961000000002 2024-02-23T00:47:08.420 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.420 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.420 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:08.420 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:28.647177+0000", 2024-02-23T00:47:08.420 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.2069000000000001e-05 2024-02-23T00:47:08.420 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.420 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.420 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:47:08.420 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:28.647209+0000", 2024-02-23T00:47:08.420 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.5355000000000003e-05 2024-02-23T00:47:08.420 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.420 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.421 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:47:08.421 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:28.647244+0000", 2024-02-23T00:47:08.421 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.098006276000000003 2024-02-23T00:47:08.421 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.421 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:47:08.421 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.421 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.421 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.421 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.21318.0:2 77.7 77:e732be06:::oid_WaitForSafePP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e139)", 2024-02-23T00:47:08.421 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:31.195374+0000", 2024-02-23T00:47:08.421 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 217.17748035299999, 2024-02-23T00:47:08.421 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034690925999999997, 2024-02-23T00:47:08.421 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:47:08.421 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:47:08.421 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:47:08.422 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.21318", 2024-02-23T00:47:08.422 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/3936817760", 2024-02-23T00:47:08.422 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 2 2024-02-23T00:47:08.422 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.422 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:47:08.422 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.422 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:47:08.422 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:31.195374+0000", 2024-02-23T00:47:08.422 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:47:08.422 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.422 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.422 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:47:08.422 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:31.195374+0000", 2024-02-23T00:47:08.422 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:47:08.422 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.422 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.423 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:47:08.423 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:31.195373+0000", 2024-02-23T00:47:08.423 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4160000000000001e-06 2024-02-23T00:47:08.423 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.423 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.423 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:47:08.423 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:31.195380+0000", 2024-02-23T00:47:08.423 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0341e-05 2024-02-23T00:47:08.423 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.423 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.423 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:47:08.423 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:31.195390+0000", 2024-02-23T00:47:08.423 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8189999999999999e-06 2024-02-23T00:47:08.423 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.423 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.424 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:47:08.424 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:31.195395+0000", 2024-02-23T00:47:08.424 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.1242999999999999e-05 2024-02-23T00:47:08.424 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.424 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.424 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:47:08.424 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:31.195426+0000", 2024-02-23T00:47:08.424 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7524999999999999e-05 2024-02-23T00:47:08.424 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.424 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.424 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:47:08.424 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:31.195523+0000", 2024-02-23T00:47:08.424 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011125599999999999 2024-02-23T00:47:08.424 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.425 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.425 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 3,7", 2024-02-23T00:47:08.425 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:31.195635+0000", 2024-02-23T00:47:08.425 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00082721900000000002 2024-02-23T00:47:08.425 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.425 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.425 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:47:08.425 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:31.196462+0000", 2024-02-23T00:47:08.425 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3867000000000002e-05 2024-02-23T00:47:08.425 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.425 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.425 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:08.425 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:31.196546+0000", 2024-02-23T00:47:08.425 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033430306999999999 2024-02-23T00:47:08.425 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.425 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.425 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:08.426 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:31.229976+0000", 2024-02-23T00:47:08.426 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1518999999999999e-05 2024-02-23T00:47:08.426 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.426 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.426 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:47:08.426 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:31.230028+0000", 2024-02-23T00:47:08.426 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7741000000000003e-05 2024-02-23T00:47:08.426 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.426 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.426 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:47:08.426 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:31.230065+0000", 2024-02-23T00:47:08.426 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034690925999999997 2024-02-23T00:47:08.426 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.426 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:47:08.426 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.426 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.427 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.427 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(osd.3.26:15 20.13 20:cde74feb:test-rados-api-smithi047-82457-17::big:head [assert-version v1,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected e140)", 2024-02-23T00:47:08.427 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:32.193380+0000", 2024-02-23T00:47:08.427 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 216.17947466000001, 2024-02-23T00:47:08.427 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.019716304, 2024-02-23T00:47:08.427 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:47:08.427 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:47:08.427 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:47:08.427 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.427 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:47:08.427 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.193380+0000", 2024-02-23T00:47:08.427 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:47:08.427 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.427 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.427 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:47:08.427 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.193380+0000", 2024-02-23T00:47:08.428 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999976 2024-02-23T00:47:08.428 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.428 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.428 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:47:08.428 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.193378+0000", 2024-02-23T00:47:08.428 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.5569999999999996e-06 2024-02-23T00:47:08.428 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.428 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.428 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:47:08.428 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.193386+0000", 2024-02-23T00:47:08.428 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0040551980000000003 2024-02-23T00:47:08.428 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.428 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.428 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:47:08.428 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.197442+0000", 2024-02-23T00:47:08.429 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.596e-05 2024-02-23T00:47:08.429 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.429 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.429 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:47:08.429 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.197458+0000", 2024-02-23T00:47:08.429 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078653649999999992 2024-02-23T00:47:08.429 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.429 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.429 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:47:08.429 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.205323+0000", 2024-02-23T00:47:08.429 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.7967000000000007e-05 2024-02-23T00:47:08.429 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.429 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.429 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:47:08.429 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.205411+0000", 2024-02-23T00:47:08.430 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076854879999999999 2024-02-23T00:47:08.430 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.430 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.430 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:47:08.430 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.213096+0000", 2024-02-23T00:47:08.430 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.019716304 2024-02-23T00:47:08.430 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.430 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:47:08.430 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.430 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.430 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.430 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.34321.0:2 83.6 83:602f83fe:::foo:head [append 0~128] snapc 0=[] ondisk+write+known_if_redirected e143)", 2024-02-23T00:47:08.430 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:34.914885+0000", 2024-02-23T00:47:08.430 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 213.45796976700001, 2024-02-23T00:47:08.430 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.24681693699999999, 2024-02-23T00:47:08.430 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:47:08.431 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:47:08.431 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:47:08.431 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.34321", 2024-02-23T00:47:08.431 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1819343708", 2024-02-23T00:47:08.431 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 2 2024-02-23T00:47:08.431 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.431 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:47:08.431 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.431 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:47:08.431 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:34.914885+0000", 2024-02-23T00:47:08.431 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:47:08.431 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.431 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.431 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:47:08.431 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:34.914885+0000", 2024-02-23T00:47:08.431 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:47:08.432 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.432 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.432 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:47:08.432 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:34.914883+0000", 2024-02-23T00:47:08.432 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.3700000000000001e-06 2024-02-23T00:47:08.432 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.432 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.432 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:47:08.432 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:34.914892+0000", 2024-02-23T00:47:08.432 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2955999999999999e-05 2024-02-23T00:47:08.432 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.432 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.432 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:47:08.432 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:34.914905+0000", 2024-02-23T00:47:08.433 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1000000000000004e-06 2024-02-23T00:47:08.433 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.433 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.433 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:47:08.433 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:34.914913+0000", 2024-02-23T00:47:08.433 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026787499999999999 2024-02-23T00:47:08.433 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.433 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.433 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:47:08.433 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:34.915181+0000", 2024-02-23T00:47:08.433 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1440000000000003e-06 2024-02-23T00:47:08.433 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.433 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.433 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:47:08.434 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:34.915189+0000", 2024-02-23T00:47:08.434 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.210715972 2024-02-23T00:47:08.434 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.434 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.434 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:47:08.434 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:35.125905+0000", 2024-02-23T00:47:08.434 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012273500000000001 2024-02-23T00:47:08.434 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.434 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.434 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:47:08.434 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:35.126028+0000", 2024-02-23T00:47:08.434 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000157777 2024-02-23T00:47:08.434 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.434 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.434 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1,4", 2024-02-23T00:47:08.435 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:35.126186+0000", 2024-02-23T00:47:08.435 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00092571699999999995 2024-02-23T00:47:08.435 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.435 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.435 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:47:08.435 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:35.127112+0000", 2024-02-23T00:47:08.435 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00059311599999999998 2024-02-23T00:47:08.435 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.435 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.435 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:08.435 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:35.127705+0000", 2024-02-23T00:47:08.435 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033930928999999999 2024-02-23T00:47:08.435 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.435 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.435 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:08.436 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:35.161636+0000", 2024-02-23T00:47:08.436 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.6928999999999998e-05 2024-02-23T00:47:08.436 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.436 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.436 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:47:08.436 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:35.161673+0000", 2024-02-23T00:47:08.436 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.9442999999999999e-05 2024-02-23T00:47:08.436 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.436 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.436 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:47:08.436 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:35.161702+0000", 2024-02-23T00:47:08.436 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.24681693699999999 2024-02-23T00:47:08.436 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.436 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:47:08.436 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.436 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.437 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.437 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.34321.0:3 83.6 83:602f83fe:::foo:head [setxattr attr1 (12) in=5b] snapc 0=[] ondisk+write+known_if_redirected e143)", 2024-02-23T00:47:08.437 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:35.163516+0000", 2024-02-23T00:47:08.437 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 213.20933844199999, 2024-02-23T00:47:08.437 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021754109000000001, 2024-02-23T00:47:08.437 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:47:08.437 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:47:08.437 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:47:08.437 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.34321", 2024-02-23T00:47:08.437 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1819343708", 2024-02-23T00:47:08.437 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 3 2024-02-23T00:47:08.437 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.437 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:47:08.437 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.437 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:47:08.437 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:35.163516+0000", 2024-02-23T00:47:08.438 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:47:08.438 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.438 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.438 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:47:08.438 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:35.163516+0000", 2024-02-23T00:47:08.438 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:47:08.438 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.438 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.438 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:47:08.438 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:35.163515+0000", 2024-02-23T00:47:08.438 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.5189999999999992e-06 2024-02-23T00:47:08.438 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.438 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.438 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:47:08.439 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:35.163523+0000", 2024-02-23T00:47:08.439 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4439e-05 2024-02-23T00:47:08.439 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.439 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.439 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:47:08.439 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:35.163538+0000", 2024-02-23T00:47:08.439 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.148e-06 2024-02-23T00:47:08.439 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.439 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.439 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:47:08.439 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:35.163545+0000", 2024-02-23T00:47:08.439 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.2592000000000001e-05 2024-02-23T00:47:08.439 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.439 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.439 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:47:08.439 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:35.163587+0000", 2024-02-23T00:47:08.440 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.5397999999999998e-05 2024-02-23T00:47:08.440 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.440 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.440 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:47:08.440 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:35.163673+0000", 2024-02-23T00:47:08.440 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000145427 2024-02-23T00:47:08.440 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.440 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.440 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1,4", 2024-02-23T00:47:08.440 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:35.163818+0000", 2024-02-23T00:47:08.440 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297884 2024-02-23T00:47:08.440 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.440 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.440 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:47:08.440 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:35.164116+0000", 2024-02-23T00:47:08.441 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00030529800000000002 2024-02-23T00:47:08.441 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.441 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.441 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:08.441 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:35.164421+0000", 2024-02-23T00:47:08.441 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.020780567999999999 2024-02-23T00:47:08.441 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.441 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.441 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:08.441 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:35.185202+0000", 2024-02-23T00:47:08.441 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0697e-05 2024-02-23T00:47:08.441 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.441 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.441 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:47:08.441 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:35.185243+0000", 2024-02-23T00:47:08.442 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7729000000000002e-05 2024-02-23T00:47:08.442 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.442 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.442 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:47:08.442 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:35.185270+0000", 2024-02-23T00:47:08.442 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021754109000000001 2024-02-23T00:47:08.442 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.442 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:47:08.442 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.442 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.442 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.442 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.15270.0:254 62.17 62:ecbfedba:test-rados-api-smithi047-82465-12::baz:head [delete] snapc 2=[2] ondisk+write+known_if_redirected e146)", 2024-02-23T00:47:08.442 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:38.449129+0000", 2024-02-23T00:47:08.442 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 209.92372600100001, 2024-02-23T00:47:08.442 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013572450999999999, 2024-02-23T00:47:08.442 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:47:08.443 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:47:08.443 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:47:08.443 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.15270", 2024-02-23T00:47:08.443 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/4133111990", 2024-02-23T00:47:08.443 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 254 2024-02-23T00:47:08.443 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.443 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:47:08.443 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.443 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:47:08.443 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.449129+0000", 2024-02-23T00:47:08.443 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:47:08.443 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.443 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.443 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:47:08.444 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.449129+0000", 2024-02-23T00:47:08.444 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:47:08.444 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.444 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.444 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:47:08.444 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.449127+0000", 2024-02-23T00:47:08.444 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.5390000000000007e-06 2024-02-23T00:47:08.444 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.444 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.444 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:47:08.444 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.449136+0000", 2024-02-23T00:47:08.444 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3937000000000001e-05 2024-02-23T00:47:08.444 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.444 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.445 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:47:08.445 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.449150+0000", 2024-02-23T00:47:08.445 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0859999999999996e-06 2024-02-23T00:47:08.445 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.445 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.445 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:47:08.445 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.449157+0000", 2024-02-23T00:47:08.445 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3842999999999999e-05 2024-02-23T00:47:08.445 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.445 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.445 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:47:08.445 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.449211+0000", 2024-02-23T00:47:08.445 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7236999999999993e-05 2024-02-23T00:47:08.445 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.445 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.446 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:47:08.446 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.449278+0000", 2024-02-23T00:47:08.446 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012528699999999999 2024-02-23T00:47:08.446 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.446 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.446 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 3,7", 2024-02-23T00:47:08.446 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.449403+0000", 2024-02-23T00:47:08.446 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038648799999999999 2024-02-23T00:47:08.446 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.446 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.446 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:47:08.446 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.449790+0000", 2024-02-23T00:47:08.446 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000232776 2024-02-23T00:47:08.446 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.446 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.447 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:08.447 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.450022+0000", 2024-02-23T00:47:08.447 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012559282999999999 2024-02-23T00:47:08.447 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.447 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.447 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:08.447 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.462582+0000", 2024-02-23T00:47:08.447 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0325000000000001e-05 2024-02-23T00:47:08.447 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.447 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.447 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:47:08.447 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.462662+0000", 2024-02-23T00:47:08.447 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9329999999999998e-05 2024-02-23T00:47:08.447 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.447 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.447 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:47:08.448 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.462701+0000", 2024-02-23T00:47:08.448 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013572450999999999 2024-02-23T00:47:08.448 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.448 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:47:08.448 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.448 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.448 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.448 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.15270.0:255 62.19 62:9df158ee:test-rados-api-smithi047-82465-12::bam:head [writefull 0~5] snapc 2=[2] ondisk+write+known_if_redirected e146)", 2024-02-23T00:47:08.448 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:38.470658+0000", 2024-02-23T00:47:08.448 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 209.90219657099999, 2024-02-23T00:47:08.448 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011753636, 2024-02-23T00:47:08.448 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:47:08.448 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:47:08.448 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:47:08.448 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.15270", 2024-02-23T00:47:08.449 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/4133111990", 2024-02-23T00:47:08.449 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 255 2024-02-23T00:47:08.449 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.449 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:47:08.449 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.449 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:47:08.449 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.470658+0000", 2024-02-23T00:47:08.449 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:47:08.449 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.449 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.449 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:47:08.449 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.470658+0000", 2024-02-23T00:47:08.449 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:47:08.449 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.450 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.450 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:47:08.450 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.470657+0000", 2024-02-23T00:47:08.450 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.677e-06 2024-02-23T00:47:08.450 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.450 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.450 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:47:08.450 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.470663+0000", 2024-02-23T00:47:08.450 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0288e-05 2024-02-23T00:47:08.450 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.450 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.450 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:47:08.450 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.470673+0000", 2024-02-23T00:47:08.450 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6009999999999997e-06 2024-02-23T00:47:08.450 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.451 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.451 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:47:08.451 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.470678+0000", 2024-02-23T00:47:08.451 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.3661999999999997e-05 2024-02-23T00:47:08.451 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.451 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.451 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:47:08.451 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.470711+0000", 2024-02-23T00:47:08.451 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4898000000000002e-05 2024-02-23T00:47:08.451 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.451 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.451 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:47:08.451 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.470766+0000", 2024-02-23T00:47:08.451 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000113916 2024-02-23T00:47:08.451 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.452 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.452 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 2,5", 2024-02-23T00:47:08.452 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.470880+0000", 2024-02-23T00:47:08.452 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037174000000000002 2024-02-23T00:47:08.452 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.452 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.452 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:47:08.452 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.471252+0000", 2024-02-23T00:47:08.452 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031977600000000002 2024-02-23T00:47:08.452 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.452 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.452 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:08.452 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.471572+0000", 2024-02-23T00:47:08.452 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010788345 2024-02-23T00:47:08.453 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.453 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.453 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:08.453 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.482360+0000", 2024-02-23T00:47:08.453 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.9320999999999999e-05 2024-02-23T00:47:08.453 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.453 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.453 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:47:08.453 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.482389+0000", 2024-02-23T00:47:08.453 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2395000000000002e-05 2024-02-23T00:47:08.453 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.453 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.453 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:47:08.453 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.482412+0000", 2024-02-23T00:47:08.454 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011753636 2024-02-23T00:47:08.454 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.454 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:47:08.454 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.454 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.454 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.454 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "MOSDECSubOpWrite(113.0s2 182/179 ECSubWrite(tid=298, reqid=osd.0.0:298, at_version=0'0, trim_to=0'0, roll_forward_to=0'0))", 2024-02-23T00:47:08.454 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:44:14.536400+0000", 2024-02-23T00:47:08.454 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 173.836454624, 2024-02-23T00:47:08.454 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.019394870000000002, 2024-02-23T00:47:08.454 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:47:08.454 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "reached pg", 2024-02-23T00:47:08.454 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:47:08.454 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.454 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:47:08.455 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:14.536400+0000", 2024-02-23T00:47:08.455 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:47:08.455 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.455 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.455 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:47:08.455 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:14.536400+0000", 2024-02-23T00:47:08.455 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:47:08.455 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.455 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.455 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:47:08.455 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:14.536399+0000", 2024-02-23T00:47:08.455 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01006662 2024-02-23T00:47:08.455 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.456 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.456 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:47:08.456 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:14.546466+0000", 2024-02-23T00:47:08.456 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5305e-05 2024-02-23T00:47:08.456 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.456 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.456 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:47:08.456 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:14.546481+0000", 2024-02-23T00:47:08.456 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7009999999999999e-06 2024-02-23T00:47:08.456 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.456 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.456 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:47:08.456 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:14.546488+0000", 2024-02-23T00:47:08.456 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.2846000000000002e-05 2024-02-23T00:47:08.456 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.457 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.457 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:47:08.457 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:14.546540+0000", 2024-02-23T00:47:08.457 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1051e-05 2024-02-23T00:47:08.457 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.457 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.457 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:47:08.457 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:14.546552+0000", 2024-02-23T00:47:08.457 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0092241489999999992 2024-02-23T00:47:08.457 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.457 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.457 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:47:08.457 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:14.555776+0000", 2024-02-23T00:47:08.457 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9364999999999998e-05 2024-02-23T00:47:08.457 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.457 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.458 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:47:08.458 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:14.555795+0000", 2024-02-23T00:47:08.458 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.019394870000000002 2024-02-23T00:47:08.458 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.458 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:47:08.458 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.458 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.458 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.458 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "MOSDECSubOpWrite(113.0s2 182/179 ECSubWrite(tid=299, reqid=osd.0.0:299, at_version=0'0, trim_to=0'0, roll_forward_to=0'0))", 2024-02-23T00:47:08.458 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:44:14.546500+0000", 2024-02-23T00:47:08.458 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 173.82635441599999, 2024-02-23T00:47:08.458 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0093119009999999992, 2024-02-23T00:47:08.458 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:47:08.458 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "reached pg", 2024-02-23T00:47:08.459 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:47:08.459 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.459 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:47:08.459 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:14.546500+0000", 2024-02-23T00:47:08.459 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:47:08.459 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.459 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.459 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:47:08.459 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:14.546500+0000", 2024-02-23T00:47:08.459 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:47:08.459 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.459 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.459 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:47:08.459 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:14.546500+0000", 2024-02-23T00:47:08.459 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1511e-05 2024-02-23T00:47:08.460 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.460 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.460 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:47:08.460 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:14.546511+0000", 2024-02-23T00:47:08.460 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.9859999999999997e-06 2024-02-23T00:47:08.460 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.460 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.460 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:47:08.460 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:14.546518+0000", 2024-02-23T00:47:08.460 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3839999999999996e-06 2024-02-23T00:47:08.460 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.460 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.460 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:47:08.460 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:14.546524+0000", 2024-02-23T00:47:08.461 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.003263653 2024-02-23T00:47:08.461 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.461 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.461 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:47:08.461 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:14.549787+0000", 2024-02-23T00:47:08.461 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.9199999999999999e-06 2024-02-23T00:47:08.461 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.461 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.461 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:47:08.461 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:14.549797+0000", 2024-02-23T00:47:08.461 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0060091379999999998 2024-02-23T00:47:08.461 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.461 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.461 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:47:08.461 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:14.555806+0000", 2024-02-23T00:47:08.461 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.071e-06 2024-02-23T00:47:08.462 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.462 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.462 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:47:08.462 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:14.555812+0000", 2024-02-23T00:47:08.462 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0093119009999999992 2024-02-23T00:47:08.462 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.462 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:47:08.462 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.462 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.462 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.462 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.30234.0:2 120.6 120:602f83fe:::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e187)", 2024-02-23T00:47:08.462 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:44:19.144296+0000", 2024-02-23T00:47:08.462 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 169.22855881300001, 2024-02-23T00:47:08.462 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094867179999999999, 2024-02-23T00:47:08.462 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:47:08.462 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:47:08.463 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:47:08.463 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.30234", 2024-02-23T00:47:08.463 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1351400035", 2024-02-23T00:47:08.463 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 2 2024-02-23T00:47:08.463 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.463 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:47:08.463 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.463 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:47:08.463 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:19.144296+0000", 2024-02-23T00:47:08.463 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:47:08.463 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.463 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.463 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:47:08.463 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:19.144296+0000", 2024-02-23T00:47:08.463 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:47:08.464 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.464 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.464 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:47:08.464 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:19.144294+0000", 2024-02-23T00:47:08.464 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.483e-05 2024-02-23T00:47:08.464 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.464 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.464 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:47:08.464 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:19.144309+0000", 2024-02-23T00:47:08.464 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0437e-05 2024-02-23T00:47:08.464 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.464 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.464 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:47:08.464 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:19.144319+0000", 2024-02-23T00:47:08.464 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.7829999999999999e-06 2024-02-23T00:47:08.465 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.465 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.465 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:47:08.465 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:19.144327+0000", 2024-02-23T00:47:08.465 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.4919999999999997e-05 2024-02-23T00:47:08.465 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.465 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.465 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:47:08.465 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:19.144372+0000", 2024-02-23T00:47:08.465 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.397e-06 2024-02-23T00:47:08.465 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.465 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.465 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:47:08.465 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:19.144381+0000", 2024-02-23T00:47:08.465 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0019337130000000001 2024-02-23T00:47:08.466 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.466 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.466 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:47:08.466 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:19.146315+0000", 2024-02-23T00:47:08.466 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012784299999999999 2024-02-23T00:47:08.466 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.466 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.466 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:47:08.466 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:19.146443+0000", 2024-02-23T00:47:08.466 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000132712 2024-02-23T00:47:08.466 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.466 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.466 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 2,3", 2024-02-23T00:47:08.466 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:19.146576+0000", 2024-02-23T00:47:08.466 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00039647800000000002 2024-02-23T00:47:08.467 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.467 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.467 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:47:08.467 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:19.146972+0000", 2024-02-23T00:47:08.467 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00065165999999999998 2024-02-23T00:47:08.467 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.467 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.467 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:08.467 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:19.147624+0000", 2024-02-23T00:47:08.467 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.006091043 2024-02-23T00:47:08.467 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.467 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.467 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:08.467 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:19.153715+0000", 2024-02-23T00:47:08.467 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.0809000000000003e-05 2024-02-23T00:47:08.468 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.468 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.468 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:47:08.468 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:19.153746+0000", 2024-02-23T00:47:08.468 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.6990000000000003e-05 2024-02-23T00:47:08.468 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.468 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.468 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:47:08.468 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:19.153783+0000", 2024-02-23T00:47:08.468 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094867179999999999 2024-02-23T00:47:08.468 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.468 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:47:08.468 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.468 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.469 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.469 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.15270.0:1107 93.2 93:40000000:%01:::head [pgnls start_epoch 196] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e196)", 2024-02-23T00:47:08.469 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:44:28.130248+0000", 2024-02-23T00:47:08.469 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 160.242606704, 2024-02-23T00:47:08.469 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01379084, 2024-02-23T00:47:08.469 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:47:08.469 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:47:08.469 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:47:08.469 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.15270", 2024-02-23T00:47:08.469 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/4133111990", 2024-02-23T00:47:08.469 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1107 2024-02-23T00:47:08.469 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.469 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:47:08.469 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.469 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:47:08.470 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:28.130248+0000", 2024-02-23T00:47:08.470 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:47:08.470 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.470 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.470 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:47:08.470 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:28.130248+0000", 2024-02-23T00:47:08.470 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:47:08.470 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.470 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.470 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:47:08.470 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:28.130246+0000", 2024-02-23T00:47:08.470 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0906e-05 2024-02-23T00:47:08.470 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.470 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.470 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:47:08.471 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:28.130257+0000", 2024-02-23T00:47:08.471 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1101999999999999e-05 2024-02-23T00:47:08.471 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.471 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.471 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:47:08.471 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:28.130278+0000", 2024-02-23T00:47:08.471 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9120000000000004e-06 2024-02-23T00:47:08.471 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.471 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.471 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:47:08.471 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:28.130287+0000", 2024-02-23T00:47:08.471 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3773000000000002e-05 2024-02-23T00:47:08.471 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.471 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.471 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:47:08.472 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:28.130341+0000", 2024-02-23T00:47:08.472 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6469999999999998e-06 2024-02-23T00:47:08.472 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.472 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.472 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op must wait for map", 2024-02-23T00:47:08.472 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:28.130348+0000", 2024-02-23T00:47:08.472 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010428855000000001 2024-02-23T00:47:08.472 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.472 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.472 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:47:08.472 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:28.140777+0000", 2024-02-23T00:47:08.472 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3723000000000001e-05 2024-02-23T00:47:08.472 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.472 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.473 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:47:08.473 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:28.140821+0000", 2024-02-23T00:47:08.473 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0032179499999999998 2024-02-23T00:47:08.473 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.473 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.473 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:47:08.473 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:28.144039+0000", 2024-02-23T00:47:08.473 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01379084 2024-02-23T00:47:08.473 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.473 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:47:08.473 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.473 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.473 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.473 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.40658.0:2 130.c 130:311cbe94:::oid_ExecuteClassPP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e198)", 2024-02-23T00:47:08.473 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:44:30.140683+0000", 2024-02-23T00:47:08.474 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 158.232171677, 2024-02-23T00:47:08.474 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015265476, 2024-02-23T00:47:08.474 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:47:08.474 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:47:08.474 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:47:08.474 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.40658", 2024-02-23T00:47:08.474 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/187838867", 2024-02-23T00:47:08.474 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 2 2024-02-23T00:47:08.474 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.474 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:47:08.474 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.474 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:47:08.474 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:30.140683+0000", 2024-02-23T00:47:08.474 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:47:08.475 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.475 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.475 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:47:08.475 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:30.140683+0000", 2024-02-23T00:47:08.475 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:47:08.475 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.475 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.475 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:47:08.475 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:30.140682+0000", 2024-02-23T00:47:08.475 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4200000000000001e-06 2024-02-23T00:47:08.475 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.475 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.475 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:47:08.475 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:30.140689+0000", 2024-02-23T00:47:08.475 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2286000000000001e-05 2024-02-23T00:47:08.476 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.476 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.476 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:47:08.476 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:30.140701+0000", 2024-02-23T00:47:08.476 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0879999999999997e-06 2024-02-23T00:47:08.476 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.476 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.476 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:47:08.476 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:30.140709+0000", 2024-02-23T00:47:08.476 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021544199999999999 2024-02-23T00:47:08.476 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.476 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.476 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:47:08.477 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:30.140924+0000", 2024-02-23T00:47:08.477 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5929999999999995e-06 2024-02-23T00:47:08.477 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.477 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.477 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:47:08.477 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:30.140934+0000", 2024-02-23T00:47:08.477 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010745156 2024-02-23T00:47:08.477 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.477 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.477 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:47:08.477 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:30.151679+0000", 2024-02-23T00:47:08.477 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012590999999999999 2024-02-23T00:47:08.477 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.477 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.477 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:47:08.478 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:30.151805+0000", 2024-02-23T00:47:08.478 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000122247 2024-02-23T00:47:08.478 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.478 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.478 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0,1", 2024-02-23T00:47:08.478 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:30.151927+0000", 2024-02-23T00:47:08.478 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000348227 2024-02-23T00:47:08.478 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.478 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.478 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:47:08.478 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:30.152275+0000", 2024-02-23T00:47:08.478 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001108075 2024-02-23T00:47:08.478 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.478 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.478 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:08.479 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:30.153383+0000", 2024-02-23T00:47:08.479 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0024612649999999998 2024-02-23T00:47:08.479 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.479 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.479 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:08.479 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:30.155844+0000", 2024-02-23T00:47:08.479 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4400999999999995e-05 2024-02-23T00:47:08.479 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.479 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.479 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:47:08.479 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:30.155909+0000", 2024-02-23T00:47:08.479 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9765999999999998e-05 2024-02-23T00:47:08.479 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.479 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.480 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:47:08.480 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:30.155949+0000", 2024-02-23T00:47:08.480 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015265476 2024-02-23T00:47:08.480 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.480 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:47:08.480 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.480 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.480 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.480 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.40718.0:1 141.6s0 141:602f83fe:::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e215)", 2024-02-23T00:47:08.480 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:44:47.129127+0000", 2024-02-23T00:47:08.480 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 141.24372807500001, 2024-02-23T00:47:08.480 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0249020049999999, 2024-02-23T00:47:08.480 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:47:08.480 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:47:08.480 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:47:08.481 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.40718", 2024-02-23T00:47:08.481 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/2198787178", 2024-02-23T00:47:08.481 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1 2024-02-23T00:47:08.481 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.481 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:47:08.481 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.481 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:47:08.481 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:47.129127+0000", 2024-02-23T00:47:08.481 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:47:08.481 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.481 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.481 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:47:08.481 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:47.129127+0000", 2024-02-23T00:47:08.481 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:47:08.481 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.482 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.482 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:47:08.482 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:47.129125+0000", 2024-02-23T00:47:08.482 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4760000000000001e-06 2024-02-23T00:47:08.482 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.482 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.482 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:47:08.482 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:47.129132+0000", 2024-02-23T00:47:08.482 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2574e-05 2024-02-23T00:47:08.482 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.482 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.482 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:47:08.482 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:47.129145+0000", 2024-02-23T00:47:08.482 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9560000000000003e-06 2024-02-23T00:47:08.482 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.483 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.483 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:47:08.483 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:47.129154+0000", 2024-02-23T00:47:08.483 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.6130000000000001e-05 2024-02-23T00:47:08.483 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.483 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.483 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:47:08.483 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:47.129190+0000", 2024-02-23T00:47:08.483 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.27e-06 2024-02-23T00:47:08.483 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.483 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.483 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:47:08.483 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:47.129193+0000", 2024-02-23T00:47:08.483 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.018406728 2024-02-23T00:47:08.483 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.484 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.484 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:47:08.484 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:48.147600+0000", 2024-02-23T00:47:08.484 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014156800000000001 2024-02-23T00:47:08.484 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.484 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.484 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:47:08.484 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:48.147742+0000", 2024-02-23T00:47:08.484 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00034513399999999998 2024-02-23T00:47:08.484 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.484 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.484 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:47:08.484 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:48.148087+0000", 2024-02-23T00:47:08.484 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.002126803 2024-02-23T00:47:08.484 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.485 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.485 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:47:08.485 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:48.150213+0000", 2024-02-23T00:47:08.485 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0037600369999999999 2024-02-23T00:47:08.485 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.485 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.485 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:47:08.485 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:48.153974+0000", 2024-02-23T00:47:08.485 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.5216999999999997e-05 2024-02-23T00:47:08.485 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:47:08.485 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:47:08.485 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:47:08.485 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:48.154029+0000", 2024-02-23T00:47:08.485 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0249020049999999 2024-02-23T00:47:08.485 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.486 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:47:08.486 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.486 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:47:08.486 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:47:08.486 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:47:08.486 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:08.487 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:07.164259+0000 mon.a (mon.0) 2293 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-23T00:47:08.487 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:07.269124+0000 mgr.y (mgr.14517) 338 : cluster [DBG] pgmap v508: 218 pgs: 218 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:47:08.487 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:07.611102+0000 mon.a (mon.0) 2294 : cluster [DBG] osdmap e355: 8 total, 8 up, 8 in 2024-02-23T00:47:08.487 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:07.611716+0000 mon.a (mon.0) 2295 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-23T00:47:08.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.388+0000 7fbbae7fc700 1 -- 172.21.15.47:0/3005576393 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+576665 (secure 0 0 0) 0x7fbb9c000bf0 con 0x7fbba4062cc0 2024-02-23T00:47:08.493 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:47:08.497 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:07.164259+0000 mon.a (mon.0) 2293 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-23T00:47:08.497 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:07.269124+0000 mgr.y (mgr.14517) 338 : cluster [DBG] pgmap v508: 218 pgs: 218 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:47:08.497 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:07.611102+0000 mon.a (mon.0) 2294 : cluster [DBG] osdmap e355: 8 total, 8 up, 8 in 2024-02-23T00:47:08.497 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:07.611716+0000 mon.a (mon.0) 2295 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-23T00:47:08.498 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:07.164259+0000 mon.a (mon.0) 2293 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-23T00:47:08.498 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:07.269124+0000 mgr.y (mgr.14517) 338 : cluster [DBG] pgmap v508: 218 pgs: 218 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:47:08.498 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:07.611102+0000 mon.a (mon.0) 2294 : cluster [DBG] osdmap e355: 8 total, 8 up, 8 in 2024-02-23T00:47:08.498 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:07.611716+0000 mon.a (mon.0) 2295 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-23T00:47:08.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.393+0000 7fbbbf43e700 10 librados: watch_flush enter 2024-02-23T00:47:08.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.393+0000 7fbbbf43e700 10 librados: watch_flush exit 2024-02-23T00:47:08.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.393+0000 7fbbbf43e700 20 client.43255.objecter shutdown clearing up homeless session... 2024-02-23T00:47:08.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.393+0000 7fbbbf43e700 10 client.43255.objecter successfully canceled tick 2024-02-23T00:47:08.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.393+0000 7fbbbf43e700 1 -- 172.21.15.47:0/3005576393 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbba4062cc0 msgr2=0x7fbba4065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:08.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.393+0000 7fbbbf43e700 1 --2- 172.21.15.47:0/3005576393 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbba4062cc0 0x7fbba4065180 secure :-1 s=READY pgs=3040 cs=0 l=1 rev1=1 rx=0x7fbbb8109790 tx=0x7fbba800b040).stop 2024-02-23T00:47:08.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.393+0000 7fbbbf43e700 1 -- 172.21.15.47:0/3005576393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbbb80f7400 msgr2=0x7fbbb810de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:08.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.393+0000 7fbbbf43e700 1 --2- 172.21.15.47:0/3005576393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbbb80f7400 0x7fbbb810de50 secure :-1 s=READY pgs=3974 cs=0 l=1 rev1=1 rx=0x7fbbb40091f0 tx=0x7fbbb4009cb0).stop 2024-02-23T00:47:08.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.393+0000 7fbbbf43e700 1 -- 172.21.15.47:0/3005576393 shutdown_connections 2024-02-23T00:47:08.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.393+0000 7fbbbf43e700 1 --2- 172.21.15.47:0/3005576393 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbba4062cc0 0x7fbba4065180 unknown :-1 s=CLOSED pgs=3040 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:08.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.393+0000 7fbbbf43e700 1 --2- 172.21.15.47:0/3005576393 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbbb80fd510 0x7fbbb8107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:08.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.393+0000 7fbbbf43e700 1 --2- 172.21.15.47:0/3005576393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbbb80f7400 0x7fbbb810de50 unknown :-1 s=CLOSED pgs=3974 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:08.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.393+0000 7fbbbf43e700 1 --2- 172.21.15.47:0/3005576393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbb80fcbb0 0x7fbbb810e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:08.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.393+0000 7fbbbf43e700 1 -- 172.21.15.47:0/3005576393 >> 172.21.15.47:0/3005576393 conn(0x7fbbb80f2820 msgr2=0x7fbbb80fae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:08.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.393+0000 7fbbbf43e700 1 -- 172.21.15.47:0/3005576393 shutdown_connections 2024-02-23T00:47:08.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.393+0000 7fbbbf43e700 1 -- 172.21.15.47:0/3005576393 wait complete. 2024-02-23T00:47:08.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:08.393+0000 7fbbbf43e700 1 librados: shutdown 2024-02-23T00:47:08.500 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-02-23T00:47:09.049 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.7 dump_ops_in_flight 2024-02-23T00:47:09.324 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:47:09.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:08.171364+0000 mon.a (mon.0) 2296 : audit [INF] from='client.? 172.21.15.47:0/3942253766' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi047-82386-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi047-82386-60"}]': finished 2024-02-23T00:47:09.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:08.171456+0000 mon.a (mon.0) 2297 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-23T00:47:09.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:08.171525+0000 mon.a (mon.0) 2298 : cluster [DBG] osdmap e356: 8 total, 8 up, 8 in 2024-02-23T00:47:09.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:08.204779+0000 mon.a (mon.0) 2299 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:47:09.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:08.373515+0000 mgr.y (mgr.14517) 339 : audit [DBG] from='client.43255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:47:09.535 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:47:09.536 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:08.171364+0000 mon.a (mon.0) 2296 : audit [INF] from='client.? 172.21.15.47:0/3942253766' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi047-82386-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi047-82386-60"}]': finished 2024-02-23T00:47:09.537 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:08.171456+0000 mon.a (mon.0) 2297 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-23T00:47:09.537 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:08.171525+0000 mon.a (mon.0) 2298 : cluster [DBG] osdmap e356: 8 total, 8 up, 8 in 2024-02-23T00:47:09.537 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:08.204779+0000 mon.a (mon.0) 2299 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:47:09.537 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:08.373515+0000 mgr.y (mgr.14517) 339 : audit [DBG] from='client.43255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:47:09.537 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:08.171364+0000 mon.a (mon.0) 2296 : audit [INF] from='client.? 172.21.15.47:0/3942253766' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi047-82386-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi047-82386-60"}]': finished 2024-02-23T00:47:09.537 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:08.171456+0000 mon.a (mon.0) 2297 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-45","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-23T00:47:09.537 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:08.171525+0000 mon.a (mon.0) 2298 : cluster [DBG] osdmap e356: 8 total, 8 up, 8 in 2024-02-23T00:47:09.537 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:08.204779+0000 mon.a (mon.0) 2299 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:47:09.537 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:08.373515+0000 mgr.y (mgr.14517) 339 : audit [DBG] from='client.43255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:47:09.538 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:09.538 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:47:09.639 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:47:09.667 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:09.686 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:47:09.688 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:09.689 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:47:09.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.644+0000 7f35bdce2700 1 -- 172.21.15.47:0/1133167064 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35b80f7400 msgr2=0x7f35b8103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:09.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.644+0000 7f35bdce2700 1 --2- 172.21.15.47:0/1133167064 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35b80f7400 0x7f35b8103a20 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 rx=0x7f35b0009c30 tx=0x7f35b0004c50).stop 2024-02-23T00:47:09.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.644+0000 7f35bdce2700 1 -- 172.21.15.47:0/1133167064 shutdown_connections 2024-02-23T00:47:09.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.644+0000 7f35bdce2700 1 --2- 172.21.15.47:0/1133167064 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35b80f7400 0x7f35b8103a20 unknown :-1 s=CLOSED pgs=2637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:09.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.644+0000 7f35bdce2700 1 --2- 172.21.15.47:0/1133167064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35b80fcbb0 0x7f35b80fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:09.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.644+0000 7f35bdce2700 1 --2- 172.21.15.47:0/1133167064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35b80fd510 0x7f35b80f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:09.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.644+0000 7f35bdce2700 1 -- 172.21.15.47:0/1133167064 >> 172.21.15.47:0/1133167064 conn(0x7f35b80f2820 msgr2=0x7f35b80f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:09.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.644+0000 7f35bdce2700 1 -- 172.21.15.47:0/1133167064 shutdown_connections 2024-02-23T00:47:09.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.644+0000 7f35bdce2700 1 -- 172.21.15.47:0/1133167064 wait complete. 2024-02-23T00:47:09.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.645+0000 7f35bdce2700 1 librados: starting msgr at 2024-02-23T00:47:09.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.645+0000 7f35bdce2700 1 librados: starting objecter 2024-02-23T00:47:09.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.645+0000 7f35bdce2700 1 Processor -- start 2024-02-23T00:47:09.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.645+0000 7f35bdce2700 1 -- start start 2024-02-23T00:47:09.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.645+0000 7f35bdce2700 1 librados: setting wanted keys 2024-02-23T00:47:09.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.645+0000 7f35bdce2700 1 librados: calling monclient init 2024-02-23T00:47:09.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.646+0000 7f35bdce2700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35b80f7400 0x7f35b8108fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:09.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.646+0000 7f35bdce2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35b80fcbb0 0x7f35b81094a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:09.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.646+0000 7f35bdce2700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35b80fd510 0x7f35b8109a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:09.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.646+0000 7f35bdce2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b81141a0 con 0x7f35b80fcbb0 2024-02-23T00:47:09.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.646+0000 7f35bdce2700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b8114310 con 0x7f35b80fd510 2024-02-23T00:47:09.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.646+0000 7f35bdce2700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b81144d0 con 0x7f35b80f7400 2024-02-23T00:47:09.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.646+0000 7f35b77fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35b80f7400 0x7f35b8108fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:09.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.646+0000 7f35b77fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35b80f7400 0x7f35b8108fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:36524/0 (socket says 172.21.15.47:36524) 2024-02-23T00:47:09.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.646+0000 7f35b77fe700 1 -- 172.21.15.47:0/488990342 learned_addr learned my addr 172.21.15.47:0/488990342 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:47:09.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.646+0000 7f35aeffd700 1 --2- 172.21.15.47:0/488990342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35b80fcbb0 0x7f35b81094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:09.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.646+0000 7f35b7fff700 1 --2- 172.21.15.47:0/488990342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35b80fd510 0x7f35b8109a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:09.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.646+0000 7f35b77fe700 1 -- 172.21.15.47:0/488990342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35b80fd510 msgr2=0x7f35b8109a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:09.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.646+0000 7f35b77fe700 1 --2- 172.21.15.47:0/488990342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35b80fd510 0x7f35b8109a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:09.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.646+0000 7f35b77fe700 1 -- 172.21.15.47:0/488990342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35b80fcbb0 msgr2=0x7f35b81094a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:09.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.646+0000 7f35b77fe700 1 --2- 172.21.15.47:0/488990342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35b80fcbb0 0x7f35b81094a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:09.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.646+0000 7f35b77fe700 1 -- 172.21.15.47:0/488990342 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a8005040 con 0x7f35b80f7400 2024-02-23T00:47:09.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.646+0000 7f35b77fe700 1 --2- 172.21.15.47:0/488990342 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35b80f7400 0x7f35b8108fb0 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 rx=0x7f35a8009400 tx=0x7f35a8009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:09.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.646+0000 7f35b57fa700 10 client.?.objecter ms_handle_connect 0x7f35b80f7400 2024-02-23T00:47:09.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.646+0000 7f35b57fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:47:09.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.646+0000 7f35b57fa700 1 -- 172.21.15.47:0/488990342 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8007a40 con 0x7f35b80f7400 2024-02-23T00:47:09.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.646+0000 7f35b57fa700 1 -- 172.21.15.47:0/488990342 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f35a800f040 con 0x7f35b80f7400 2024-02-23T00:47:09.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.646+0000 7f35bdce2700 1 -- 172.21.15.47:0/488990342 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b00097c0 con 0x7f35b80f7400 2024-02-23T00:47:09.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.646+0000 7f35b57fa700 1 -- 172.21.15.47:0/488990342 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8014900 con 0x7f35b80f7400 2024-02-23T00:47:09.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.646+0000 7f35bdce2700 10 client.41201.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:47:09.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.646+0000 7f35bdce2700 1 -- 172.21.15.47:0/488990342 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b8114b60 con 0x7f35b80f7400 2024-02-23T00:47:09.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.646+0000 7f35bdce2700 1 librados: init done 2024-02-23T00:47:09.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.647+0000 7f35bdce2700 1 -- 172.21.15.47:0/488990342 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b805fa60 con 0x7f35b80f7400 2024-02-23T00:47:09.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.648+0000 7f35b57fa700 1 -- 172.21.15.47:0/488990342 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f35a8014d30 con 0x7f35b80f7400 2024-02-23T00:47:09.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.648+0000 7f35b57fa700 1 --2- 172.21.15.47:0/488990342 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f35a4062cc0 0x7f35a4065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:09.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.648+0000 7f35b57fa700 1 -- 172.21.15.47:0/488990342 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(357..357 src has 48..357) v4 ==== 10017+0+0 (secure 0 0 0) 0x7f35a807d490 con 0x7f35b80f7400 2024-02-23T00:47:09.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.648+0000 7f35b57fa700 10 client.41201.objecter ms_dispatch 0x7f35b8001550 osd_map(357..357 src has 48..357) v4 2024-02-23T00:47:09.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.648+0000 7f35b57fa700 3 client.41201.objecter handle_osd_map got epochs [357,357] > 0 2024-02-23T00:47:09.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.648+0000 7f35b57fa700 3 client.41201.objecter handle_osd_map decoding full epoch 357 2024-02-23T00:47:09.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.648+0000 7f35aeffd700 1 --2- 172.21.15.47:0/488990342 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f35a4062cc0 0x7f35a4065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:09.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.648+0000 7f35b57fa700 20 client.41201.objecter dump_active .. 0 homeless 2024-02-23T00:47:09.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.648+0000 7f35b57fa700 10 client.41201.objecter ms_handle_connect 0x7f35a4062cc0 2024-02-23T00:47:09.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.648+0000 7f35aeffd700 1 --2- 172.21.15.47:0/488990342 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f35a4062cc0 0x7f35a4065180 secure :-1 s=READY pgs=3041 cs=0 l=1 rev1=1 rx=0x7f35b8061720 tx=0x7f35a000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:09.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.652+0000 7f35b57fa700 1 -- 172.21.15.47:0/488990342 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f35a8049f60 con 0x7f35b80f7400 2024-02-23T00:47:09.790 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:47:09.822 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:09.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:09.839+0000 7f35bdce2700 1 -- 172.21.15.47:0/488990342 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7f35b8061460 con 0x7f35b80f7400 2024-02-23T00:47:10.172 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:47:10.183 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:10.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:10.180+0000 7f35b57fa700 1 -- 172.21.15.47:0/488990342 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v358) v1 ==== 91+0+0 (secure 0 0 0) 0x7f35a804c800 con 0x7f35b80f7400 2024-02-23T00:47:10.184 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:47:10.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:10.183+0000 7f35bdce2700 10 librados: watch_flush enter 2024-02-23T00:47:10.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:10.183+0000 7f35bdce2700 10 librados: watch_flush exit 2024-02-23T00:47:10.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:10.183+0000 7f35bdce2700 20 client.41201.objecter shutdown clearing up homeless session... 2024-02-23T00:47:10.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:10.183+0000 7f35bdce2700 10 client.41201.objecter successfully canceled tick 2024-02-23T00:47:10.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:10.183+0000 7f35bdce2700 1 -- 172.21.15.47:0/488990342 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f35a4062cc0 msgr2=0x7f35a4065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:10.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:10.183+0000 7f35bdce2700 1 --2- 172.21.15.47:0/488990342 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f35a4062cc0 0x7f35a4065180 secure :-1 s=READY pgs=3041 cs=0 l=1 rev1=1 rx=0x7f35b8061720 tx=0x7f35a000b040).stop 2024-02-23T00:47:10.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:10.183+0000 7f35bdce2700 1 -- 172.21.15.47:0/488990342 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35b80f7400 msgr2=0x7f35b8108fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:10.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:10.183+0000 7f35bdce2700 1 --2- 172.21.15.47:0/488990342 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35b80f7400 0x7f35b8108fb0 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 rx=0x7f35a8009400 tx=0x7f35a8009cb0).stop 2024-02-23T00:47:10.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:10.186+0000 7f35bdce2700 1 -- 172.21.15.47:0/488990342 shutdown_connections 2024-02-23T00:47:10.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:10.186+0000 7f35bdce2700 1 --2- 172.21.15.47:0/488990342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35b80fd510 0x7f35b8109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:10.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:10.186+0000 7f35bdce2700 1 --2- 172.21.15.47:0/488990342 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f35a4062cc0 0x7f35a4065180 unknown :-1 s=CLOSED pgs=3041 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:10.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:10.186+0000 7f35bdce2700 1 --2- 172.21.15.47:0/488990342 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35b80f7400 0x7f35b8108fb0 unknown :-1 s=CLOSED pgs=2638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:10.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:10.186+0000 7f35bdce2700 1 --2- 172.21.15.47:0/488990342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35b80fcbb0 0x7f35b81094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:10.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:10.186+0000 7f35bdce2700 1 -- 172.21.15.47:0/488990342 >> 172.21.15.47:0/488990342 conn(0x7f35b80f2820 msgr2=0x7f35b80f4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:10.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:10.189+0000 7f35bdce2700 1 -- 172.21.15.47:0/488990342 shutdown_connections 2024-02-23T00:47:10.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:10.189+0000 7f35bdce2700 1 -- 172.21.15.47:0/488990342 wait complete. 2024-02-23T00:47:10.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:10.189+0000 7f35bdce2700 1 librados: shutdown 2024-02-23T00:47:10.194 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is unset 2024-02-23T00:47:10.210 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":509,"stamp":"2024-02-23T00:47:08.167799+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4007181,"num_objects":368,"num_object_clones":2,"num_object_copies":1104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":368,"num_whiteouts":0,"num_read":1670,"num_read_kb":1296,"num_write":1195,"num_write_kb":4517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2560,"ondisk_log_size":2560,"up":654,"acting":654,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":684,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2498256,"kb_used_data":58968,"kb_used_omap":0,"kb_used_meta":2439168,"kb_avail":747462960,"statfs":{"total":767960285184,"available":765402071040,"internally_reserved":0,"allocated":60383232,"data_stored":41261312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2497708032},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.989322"},"pg_stats":[{"pgid":"196.1f","version":"0'0","reported_seq":17,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.570947+0000","last_change":"2024-02-23T00:46:59.187439+0000","last_active":"2024-02-23T00:47:06.570947+0000","last_peered":"2024-02-23T00:47:06.570947+0000","last_clean":"2024-02-23T00:47:06.570947+0000","last_became_active":"2024-02-23T00:46:59.186974+0000","last_became_peered":"2024-02-23T00:46:59.186974+0000","last_unstale":"2024-02-23T00:47:06.570947+0000","last_undegraded":"2024-02-23T00:47:06.570947+0000","last_fullsized":"2024-02-23T00:47:06.570947+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":346,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_clean_scrub_stamp":"2024-02-23T00:46:58.158910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"196.1e","version":"0'0","reported_seq":17,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.162693+0000","last_change":"2024-02-23T00:46:59.197417+0000","last_active":"2024-02-23T00:47:06.162693+0000","last_peered":"2024-02-23T00:47:06.162693+0000","last_clean":"2024-02-23T00:47:06.162693+0000","last_became_active":"2024-02-23T00:46:59.197106+0000","last_became_peered":"2024-02-23T00:46:59.197106+0000","last_unstale":"2024-02-23T00:47:06.162693+0000","last_undegraded":"2024-02-23T00:47:06.162693+0000","last_fullsized":"2024-02-23T00:47:06.162693+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":346,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_clean_scrub_stamp":"2024-02-23T00:46:58.158910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"196.1d","version":"0'0","reported_seq":16,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.417934+0000","last_change":"2024-02-23T00:46:59.183510+0000","last_active":"2024-02-23T00:47:05.417934+0000","last_peered":"2024-02-23T00:47:05.417934+0000","last_clean":"2024-02-23T00:47:05.417934+0000","last_became_active":"2024-02-23T00:46:59.183136+0000","last_became_peered":"2024-02-23T00:46:59.183136+0000","last_unstale":"2024-02-23T00:47:05.417934+0000","last_undegraded":"2024-02-23T00:47:05.417934+0000","last_fullsized":"2024-02-23T00:47:05.417934+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":346,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_clean_scrub_stamp":"2024-02-23T00:46:58.158910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"196.1c","version":"0'0","reported_seq":17,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.161898+0000","last_change":"2024-02-23T00:46:59.193317+0000","last_active":"2024-02-23T00:47:06.161898+0000","last_peered":"2024-02-23T00:47:06.161898+0000","last_clean":"2024-02-23T00:47:06.161898+0000","last_became_active":"2024-02-23T00:46:59.192999+0000","last_became_peered":"2024-02-23T00:46:59.192999+0000","last_unstale":"2024-02-23T00:47:06.161898+0000","last_undegraded":"2024-02-23T00:47:06.161898+0000","last_fullsized":"2024-02-23T00:47:06.161898+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":346,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_clean_scrub_stamp":"2024-02-23T00:46:58.158910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"196.1b","version":"0'0","reported_seq":16,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.571175+0000","last_change":"2024-02-23T00:46:59.181003+0000","last_active":"2024-02-23T00:47:06.571175+0000","last_peered":"2024-02-23T00:47:06.571175+0000","last_clean":"2024-02-23T00:47:06.571175+0000","last_became_active":"2024-02-23T00:46:59.180793+0000","last_became_peered":"2024-02-23T00:46:59.180793+0000","last_unstale":"2024-02-23T00:47:06.571175+0000","last_undegraded":"2024-02-23T00:47:06.571175+0000","last_fullsized":"2024-02-23T00:47:06.571175+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":346,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_clean_scrub_stamp":"2024-02-23T00:46:58.158910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"196.1a","version":"0'0","reported_seq":15,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.419248+0000","last_change":"2024-02-23T00:46:59.183732+0000","last_active":"2024-02-23T00:47:05.419248+0000","last_peered":"2024-02-23T00:47:05.419248+0000","last_clean":"2024-02-23T00:47:05.419248+0000","last_became_active":"2024-02-23T00:46:59.183420+0000","last_became_peered":"2024-02-23T00:46:59.183420+0000","last_unstale":"2024-02-23T00:47:05.419248+0000","last_undegraded":"2024-02-23T00:47:05.419248+0000","last_fullsized":"2024-02-23T00:47:05.419248+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":346,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_clean_scrub_stamp":"2024-02-23T00:46:58.158910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"196.19","version":"0'0","reported_seq":16,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.415555+0000","last_change":"2024-02-23T00:46:59.177351+0000","last_active":"2024-02-23T00:47:05.415555+0000","last_peered":"2024-02-23T00:47:05.415555+0000","last_clean":"2024-02-23T00:47:05.415555+0000","last_became_active":"2024-02-23T00:46:59.177006+0000","last_became_peered":"2024-02-23T00:46:59.177006+0000","last_unstale":"2024-02-23T00:47:05.415555+0000","last_undegraded":"2024-02-23T00:47:05.415555+0000","last_fullsized":"2024-02-23T00:47:05.415555+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":346,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_clean_scrub_stamp":"2024-02-23T00:46:58.158910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"196.18","version":"0'0","reported_seq":16,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.483986+0000","last_change":"2024-02-23T00:46:59.189424+0000","last_active":"2024-02-23T00:47:05.483986+0000","last_peered":"2024-02-23T00:47:05.483986+0000","last_clean":"2024-02-23T00:47:05.483986+0000","last_became_active":"2024-02-23T00:46:59.189122+0000","last_became_peered":"2024-02-23T00:46:59.189122+0000","last_unstale":"2024-02-23T00:47:05.483986+0000","last_undegraded":"2024-02-23T00:47:05.483986+0000","last_fullsized":"2024-02-23T00:47:05.483986+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":346,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_clean_scrub_stamp":"2024-02-23T00:46:58.158910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"196.17","version":"0'0","reported_seq":17,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.162842+0000","last_change":"2024-02-23T00:46:59.192293+0000","last_active":"2024-02-23T00:47:06.162842+0000","last_peered":"2024-02-23T00:47:06.162842+0000","last_clean":"2024-02-23T00:47:06.162842+0000","last_became_active":"2024-02-23T00:46:59.191979+0000","last_became_peered":"2024-02-23T00:46:59.191979+0000","last_unstale":"2024-02-23T00:47:06.162842+0000","last_undegraded":"2024-02-23T00:47:06.162842+0000","last_fullsized":"2024-02-23T00:47:06.162842+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":346,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_clean_scrub_stamp":"2024-02-23T00:46:58.158910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"196.16","version":"0'0","reported_seq":16,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.484959+0000","last_change":"2024-02-23T00:46:59.192494+0000","last_active":"2024-02-23T00:47:05.484959+0000","last_peered":"2024-02-23T00:47:05.484959+0000","last_clean":"2024-02-23T00:47:05.484959+0000","last_became_active":"2024-02-23T00:46:59.192232+0000","last_became_peered":"2024-02-23T00:46:59.192232+0000","last_unstale":"2024-02-23T00:47:05.484959+0000","last_undegraded":"2024-02-23T00:47:05.484959+0000","last_fullsized":"2024-02-23T00:47:05.484959+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":346,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_clean_scrub_stamp":"2024-02-23T00:46:58.158910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"196.15","version":"0'0","reported_seq":16,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.485368+0000","last_change":"2024-02-23T00:46:59.185929+0000","last_active":"2024-02-23T00:47:05.485368+0000","last_peered":"2024-02-23T00:47:05.485368+0000","last_clean":"2024-02-23T00:47:05.485368+0000","last_became_active":"2024-02-23T00:46:59.185602+0000","last_became_peered":"2024-02-23T00:46:59.185602+0000","last_unstale":"2024-02-23T00:47:05.485368+0000","last_undegraded":"2024-02-23T00:47:05.485368+0000","last_fullsized":"2024-02-23T00:47:05.485368+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":346,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_clean_scrub_stamp":"2024-02-23T00:46:58.158910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"196.14","version":"0'0","reported_seq":17,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.570103+0000","last_change":"2024-02-23T00:46:59.186867+0000","last_active":"2024-02-23T00:47:06.570103+0000","last_peered":"2024-02-23T00:47:06.570103+0000","last_clean":"2024-02-23T00:47:06.570103+0000","last_became_active":"2024-02-23T00:46:59.186594+0000","last_became_peered":"2024-02-23T00:46:59.186594+0000","last_unstale":"2024-02-23T00:47:06.570103+0000","last_undegraded":"2024-02-23T00:47:06.570103+0000","last_fullsized":"2024-02-23T00:47:06.570103+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":346,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_clean_scrub_stamp":"2024-02-23T00:46:58.158910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"196.13","version":"0'0","reported_seq":15,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.417020+0000","last_change":"2024-02-23T00:46:59.188047+0000","last_active":"2024-02-23T00:47:05.417020+0000","last_peered":"2024-02-23T00:47:05.417020+0000","last_clean":"2024-02-23T00:47:05.417020+0000","last_became_active":"2024-02-23T00:46:59.187742+0000","last_became_peered":"2024-02-23T00:46:59.187742+0000","last_unstale":"2024-02-23T00:47:05.417020+0000","last_undegraded":"2024-02-23T00:47:05.417020+0000","last_fullsized":"2024-02-23T00:47:05.417020+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":346,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_clean_scrub_stamp":"2024-02-23T00:46:58.158910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"196.12","version":"0'0","reported_seq":16,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.570611+0000","last_change":"2024-02-23T00:46:59.192459+0000","last_active":"2024-02-23T00:47:06.570611+0000","last_peered":"2024-02-23T00:47:06.570611+0000","last_clean":"2024-02-23T00:47:06.570611+0000","last_became_active":"2024-02-23T00:46:59.192147+0000","last_became_peered":"2024-02-23T00:46:59.192147+0000","last_unstale":"2024-02-23T00:47:06.570611+0000","last_undegraded":"2024-02-23T00:47:06.570611+0000","last_fullsized":"2024-02-23T00:47:06.570611+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":346,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_clean_scrub_stamp":"2024-02-23T00:46:58.158910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"196.11","version":"0'0","reported_seq":16,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.484245+0000","last_change":"2024-02-23T00:46:59.192851+0000","last_active":"2024-02-23T00:47:05.484245+0000","last_peered":"2024-02-23T00:47:05.484245+0000","last_clean":"2024-02-23T00:47:05.484245+0000","last_became_active":"2024-02-23T00:46:59.192604+0000","last_became_peered":"2024-02-23T00:46:59.192604+0000","last_unstale":"2024-02-23T00:47:05.484245+0000","last_undegraded":"2024-02-23T00:47:05.484245+0000","last_fullsized":"2024-02-23T00:47:05.484245+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":346,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_clean_scrub_stamp":"2024-02-23T00:46:58.158910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"196.10","version":"0'0","reported_seq":15,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.161360+0000","last_change":"2024-02-23T00:46:59.188018+0000","last_active":"2024-02-23T00:47:05.161360+0000","last_peered":"2024-02-23T00:47:05.161360+0000","last_clean":"2024-02-23T00:47:05.161360+0000","last_became_active":"2024-02-23T00:46:59.186816+0000","last_became_peered":"2024-02-23T00:46:59.186816+0000","last_unstale":"2024-02-23T00:47:05.161360+0000","last_undegraded":"2024-02-23T00:47:05.161360+0000","last_fullsized":"2024-02-23T00:47:05.161360+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":346,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_clean_scrub_stamp":"2024-02-23T00:46:58.158910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"196.f","version":"0'0","reported_seq":15,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.162317+0000","last_change":"2024-02-23T00:46:59.188041+0000","last_active":"2024-02-23T00:47:05.162317+0000","last_peered":"2024-02-23T00:47:05.162317+0000","last_clean":"2024-02-23T00:47:05.162317+0000","last_became_active":"2024-02-23T00:46:59.187726+0000","last_became_peered":"2024-02-23T00:46:59.187726+0000","last_unstale":"2024-02-23T00:47:05.162317+0000","last_undegraded":"2024-02-23T00:47:05.162317+0000","last_fullsized":"2024-02-23T00:47:05.162317+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":346,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_clean_scrub_stamp":"2024-02-23T00:46:58.158910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"196.e","version":"0'0","reported_seq":17,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.162925+0000","last_change":"2024-02-23T00:46:59.190230+0000","last_active":"2024-02-23T00:47:06.162925+0000","last_peered":"2024-02-23T00:47:06.162925+0000","last_clean":"2024-02-23T00:47:06.162925+0000","last_became_active":"2024-02-23T00:46:59.189909+0000","last_became_peered":"2024-02-23T00:46:59.189909+0000","last_unstale":"2024-02-23T00:47:06.162925+0000","last_undegraded":"2024-02-23T00:47:06.162925+0000","last_fullsized":"2024-02-23T00:47:06.162925+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":346,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_clean_scrub_stamp":"2024-02-23T00:46:58.158910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"196.d","version":"0'0","reported_seq":16,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.485588+0000","last_change":"2024-02-23T00:46:59.194516+0000","last_active":"2024-02-23T00:47:05.485588+0000","last_peered":"2024-02-23T00:47:05.485588+0000","last_clean":"2024-02-23T00:47:05.485588+0000","last_became_active":"2024-02-23T00:46:59.194227+0000","last_became_peered":"2024-02-23T00:46:59.194227+0000","last_unstale":"2024-02-23T00:47:05.485588+0000","last_undegraded":"2024-02-23T00:47:05.485588+0000","last_fullsized":"2024-02-23T00:47:05.485588+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":346,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_clean_scrub_stamp":"2024-02-23T00:46:58.158910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"196.c","version":"0'0","reported_seq":15,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.416832+0000","last_change":"2024-02-23T00:46:59.182674+0000","last_active":"2024-02-23T00:47:05.416832+0000","last_peered":"2024-02-23T00:47:05.416832+0000","last_clean":"2024-02-23T00:47:05.416832+0000","last_became_active":"2024-02-23T00:46:59.182358+0000","last_became_peered":"2024-02-23T00:46:59.182358+0000","last_unstale":"2024-02-23T00:47:05.416832+0000","last_undegraded":"2024-02-23T00:47:05.416832+0000","last_fullsized":"2024-02-23T00:47:05.416832+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":346,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_clean_scrub_stamp":"2024-02-23T00:46:58.158910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5],"acting":[6,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"196.b","version":"0'0","reported_seq":16,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.571385+0000","last_change":"2024-02-23T00:46:59.191496+0000","last_active":"2024-02-23T00:47:06.571385+0000","last_peered":"2024-02-23T00:47:06.571385+0000","last_clean":"2024-02-23T00:47:06.571385+0000","last_became_active":"2024-02-23T00:46:59.190993+0000","last_became_peered":"2024-02-23T00:46:59.190993+0000","last_unstale":"2024-02-23T00:47:06.571385+0000","last_undegraded":"2024-02-23T00:47:06.571385+0000","last_fullsized":"2024-02-23T00:47:06.571385+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":346,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_clean_scrub_stamp":"2024-02-23T00:46:58.158910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"196.a","version":"0'0","reported_seq":16,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.484147+0000","last_change":"2024-02-23T00:46:59.186760+0000","last_active":"2024-02-23T00:47:05.484147+0000","last_peered":"2024-02-23T00:47:05.484147+0000","last_clean":"2024-02-23T00:47:05.484147+0000","last_became_active":"2024-02-23T00:46:59.186315+0000","last_became_peered":"2024-02-23T00:46:59.186315+0000","last_unstale":"2024-02-23T00:47:05.484147+0000","last_undegraded":"2024-02-23T00:47:05.484147+0000","last_fullsized":"2024-02-23T00:47:05.484147+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":346,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_clean_scrub_stamp":"2024-02-23T00:46:58.158910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"196.9","version":"0'0","reported_seq":17,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.572388+0000","last_change":"2024-02-23T00:46:59.185493+0000","last_active":"2024-02-23T00:47:06.572388+0000","last_peered":"2024-02-23T00:47:06.572388+0000","last_clean":"2024-02-23T00:47:06.572388+0000","last_became_active":"2024-02-23T00:46:59.185204+0000","last_became_peered":"2024-02-23T00:46:59.185204+0000","last_unstale":"2024-02-23T00:47:06.572388+0000","last_undegraded":"2024-02-23T00:47:06.572388+0000","last_fullsized":"2024-02-23T00:47:06.572388+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":346,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_clean_scrub_stamp":"2024-02-23T00:46:58.158910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,7],"acting":[4,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"196.8","version":"0'0","reported_seq":16,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.663781+0000","last_change":"2024-02-23T00:46:59.180173+0000","last_active":"2024-02-23T00:47:05.663781+0000","last_peered":"2024-02-23T00:47:05.663781+0000","last_clean":"2024-02-23T00:47:05.663781+0000","last_became_active":"2024-02-23T00:46:59.179867+0000","last_became_peered":"2024-02-23T00:46:59.179867+0000","last_unstale":"2024-02-23T00:47:05.663781+0000","last_undegraded":"2024-02-23T00:47:05.663781+0000","last_fullsized":"2024-02-23T00:47:05.663781+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":346,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_clean_scrub_stamp":"2024-02-23T00:46:58.158910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"199.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:47:08.161899+0000","last_change":"2024-02-23T00:47:08.161899+0000","last_active":"2024-02-23T00:47:08.161899+0000","last_peered":"2024-02-23T00:47:08.161899+0000","last_clean":"2024-02-23T00:47:08.161899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:47:08.161899+0000","last_undegraded":"2024-02-23T00:47:08.161899+0000","last_fullsized":"2024-02-23T00:47:08.161899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:08.161899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:47:08.161899+0000","last_clean_scrub_stamp":"2024-02-23T00:47:08.161899+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"196.0","version":"0'0","reported_seq":17,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.161213+0000","last_change":"2024-02-23T00:46:59.179870+0000","last_active":"2024-02-23T00:47:06.161213+0000","last_peered":"2024-02-23T00:47:06.161213+0000","last_clean":"2024-02-23T00:47:06.161213+0000","last_became_active":"2024-02-23T00:46:59.179552+0000","last_became_peered":"2024-02-23T00:46:59.179552+0000","last_unstale":"2024-02-23T00:47:06.161213+0000","last_undegraded":"2024-02-23T00:47:06.161213+0000","last_fullsized":"2024-02-23T00:47:06.161213+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":346,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_clean_scrub_stamp":"2024-02-23T00:46:58.158910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"199.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:47:08.161899+0000","last_change":"2024-02-23T00:47:08.161899+0000","last_active":"2024-02-23T00:47:08.161899+0000","last_peered":"2024-02-23T00:47:08.161899+0000","last_clean":"2024-02-23T00:47:08.161899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:47:08.161899+0000","last_undegraded":"2024-02-23T00:47:08.161899+0000","last_fullsized":"2024-02-23T00:47:08.161899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:08.161899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:47:08.161899+0000","last_clean_scrub_stamp":"2024-02-23T00:47:08.161899+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"196.1","version":"0'0","reported_seq":17,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.161282+0000","last_change":"2024-02-23T00:46:59.192919+0000","last_active":"2024-02-23T00:47:06.161282+0000","last_peered":"2024-02-23T00:47:06.161282+0000","last_clean":"2024-02-23T00:47:06.161282+0000","last_became_active":"2024-02-23T00:46:59.192632+0000","last_became_peered":"2024-02-23T00:46:59.192632+0000","last_unstale":"2024-02-23T00:47:06.161282+0000","last_undegraded":"2024-02-23T00:47:06.161282+0000","last_fullsized":"2024-02-23T00:47:06.161282+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":346,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_clean_scrub_stamp":"2024-02-23T00:46:58.158910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"199.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:47:08.161899+0000","last_change":"2024-02-23T00:47:08.161899+0000","last_active":"2024-02-23T00:47:08.161899+0000","last_peered":"2024-02-23T00:47:08.161899+0000","last_clean":"2024-02-23T00:47:08.161899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:47:08.161899+0000","last_undegraded":"2024-02-23T00:47:08.161899+0000","last_fullsized":"2024-02-23T00:47:08.161899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:08.161899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:47:08.161899+0000","last_clean_scrub_stamp":"2024-02-23T00:47:08.161899+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"196.2","version":"0'0","reported_seq":15,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.167518+0000","last_change":"2024-02-23T00:46:59.180333+0000","last_active":"2024-02-23T00:47:05.167518+0000","last_peered":"2024-02-23T00:47:05.167518+0000","last_clean":"2024-02-23T00:47:05.167518+0000","last_became_active":"2024-02-23T00:46:59.180006+0000","last_became_peered":"2024-02-23T00:46:59.180006+0000","last_unstale":"2024-02-23T00:47:05.167518+0000","last_undegraded":"2024-02-23T00:47:05.167518+0000","last_fullsized":"2024-02-23T00:47:05.167518+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":346,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_clean_scrub_stamp":"2024-02-23T00:46:58.158910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"199.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:47:08.161899+0000","last_change":"2024-02-23T00:47:08.161899+0000","last_active":"2024-02-23T00:47:08.161899+0000","last_peered":"2024-02-23T00:47:08.161899+0000","last_clean":"2024-02-23T00:47:08.161899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:47:08.161899+0000","last_undegraded":"2024-02-23T00:47:08.161899+0000","last_fullsized":"2024-02-23T00:47:08.161899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:08.161899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:47:08.161899+0000","last_clean_scrub_stamp":"2024-02-23T00:47:08.161899+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"196.3","version":"0'0","reported_seq":16,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.571460+0000","last_change":"2024-02-23T00:46:59.193687+0000","last_active":"2024-02-23T00:47:06.571460+0000","last_peered":"2024-02-23T00:47:06.571460+0000","last_clean":"2024-02-23T00:47:06.571460+0000","last_became_active":"2024-02-23T00:46:59.193374+0000","last_became_peered":"2024-02-23T00:46:59.193374+0000","last_unstale":"2024-02-23T00:47:06.571460+0000","last_undegraded":"2024-02-23T00:47:06.571460+0000","last_fullsized":"2024-02-23T00:47:06.571460+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":346,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_clean_scrub_stamp":"2024-02-23T00:46:58.158910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"199.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:47:08.161899+0000","last_change":"2024-02-23T00:47:08.161899+0000","last_active":"2024-02-23T00:47:08.161899+0000","last_peered":"2024-02-23T00:47:08.161899+0000","last_clean":"2024-02-23T00:47:08.161899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:47:08.161899+0000","last_undegraded":"2024-02-23T00:47:08.161899+0000","last_fullsized":"2024-02-23T00:47:08.161899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:08.161899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:47:08.161899+0000","last_clean_scrub_stamp":"2024-02-23T00:47:08.161899+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"196.4","version":"0'0","reported_seq":16,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.570538+0000","last_change":"2024-02-23T00:46:59.187994+0000","last_active":"2024-02-23T00:47:06.570538+0000","last_peered":"2024-02-23T00:47:06.570538+0000","last_clean":"2024-02-23T00:47:06.570538+0000","last_became_active":"2024-02-23T00:46:59.187709+0000","last_became_peered":"2024-02-23T00:46:59.187709+0000","last_unstale":"2024-02-23T00:47:06.570538+0000","last_undegraded":"2024-02-23T00:47:06.570538+0000","last_fullsized":"2024-02-23T00:47:06.570538+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":346,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_clean_scrub_stamp":"2024-02-23T00:46:58.158910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"199.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:47:08.161899+0000","last_change":"2024-02-23T00:47:08.161899+0000","last_active":"2024-02-23T00:47:08.161899+0000","last_peered":"2024-02-23T00:47:08.161899+0000","last_clean":"2024-02-23T00:47:08.161899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:47:08.161899+0000","last_undegraded":"2024-02-23T00:47:08.161899+0000","last_fullsized":"2024-02-23T00:47:08.161899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:08.161899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:47:08.161899+0000","last_clean_scrub_stamp":"2024-02-23T00:47:08.161899+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"196.5","version":"0'0","reported_seq":16,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.485156+0000","last_change":"2024-02-23T00:46:59.183365+0000","last_active":"2024-02-23T00:47:05.485156+0000","last_peered":"2024-02-23T00:47:05.485156+0000","last_clean":"2024-02-23T00:47:05.485156+0000","last_became_active":"2024-02-23T00:46:59.183039+0000","last_became_peered":"2024-02-23T00:46:59.183039+0000","last_unstale":"2024-02-23T00:47:05.485156+0000","last_undegraded":"2024-02-23T00:47:05.485156+0000","last_fullsized":"2024-02-23T00:47:05.485156+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":346,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_clean_scrub_stamp":"2024-02-23T00:46:58.158910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"199.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:47:08.161899+0000","last_change":"2024-02-23T00:47:08.161899+0000","last_active":"2024-02-23T00:47:08.161899+0000","last_peered":"2024-02-23T00:47:08.161899+0000","last_clean":"2024-02-23T00:47:08.161899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:47:08.161899+0000","last_undegraded":"2024-02-23T00:47:08.161899+0000","last_fullsized":"2024-02-23T00:47:08.161899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:08.161899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:47:08.161899+0000","last_clean_scrub_stamp":"2024-02-23T00:47:08.161899+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"196.6","version":"0'0","reported_seq":15,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.161920+0000","last_change":"2024-02-23T00:46:59.187449+0000","last_active":"2024-02-23T00:47:05.161920+0000","last_peered":"2024-02-23T00:47:05.161920+0000","last_clean":"2024-02-23T00:47:05.161920+0000","last_became_active":"2024-02-23T00:46:59.187169+0000","last_became_peered":"2024-02-23T00:46:59.187169+0000","last_unstale":"2024-02-23T00:47:05.161920+0000","last_undegraded":"2024-02-23T00:47:05.161920+0000","last_fullsized":"2024-02-23T00:47:05.161920+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":346,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_clean_scrub_stamp":"2024-02-23T00:46:58.158910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"199.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:47:08.161899+0000","last_change":"2024-02-23T00:47:08.161899+0000","last_active":"2024-02-23T00:47:08.161899+0000","last_peered":"2024-02-23T00:47:08.161899+0000","last_clean":"2024-02-23T00:47:08.161899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:47:08.161899+0000","last_undegraded":"2024-02-23T00:47:08.161899+0000","last_fullsized":"2024-02-23T00:47:08.161899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:08.161899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:47:08.161899+0000","last_clean_scrub_stamp":"2024-02-23T00:47:08.161899+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"196.7","version":"0'0","reported_seq":16,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.666201+0000","last_change":"2024-02-23T00:46:59.180955+0000","last_active":"2024-02-23T00:47:05.666201+0000","last_peered":"2024-02-23T00:47:05.666201+0000","last_clean":"2024-02-23T00:47:05.666201+0000","last_became_active":"2024-02-23T00:46:59.180655+0000","last_became_peered":"2024-02-23T00:46:59.180655+0000","last_unstale":"2024-02-23T00:47:05.666201+0000","last_undegraded":"2024-02-23T00:47:05.666201+0000","last_fullsized":"2024-02-23T00:47:05.666201+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":346,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:58.158910+0000","last_clean_scrub_stamp":"2024-02-23T00:46:58.158910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.0","version":"259'40","reported_seq":323,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.484076+0000","last_change":"2024-02-23T00:46:06.778112+0000","last_active":"2024-02-23T00:47:05.484076+0000","last_peered":"2024-02-23T00:47:05.484076+0000","last_clean":"2024-02-23T00:47:05.484076+0000","last_became_active":"2024-02-23T00:43:17.855316+0000","last_became_peered":"2024-02-23T00:43:17.855316+0000","last_unstale":"2024-02-23T00:47:05.484076+0000","last_undegraded":"2024-02-23T00:47:05.484076+0000","last_fullsized":"2024-02-23T00:47:05.484076+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'40","last_scrub_stamp":"2024-02-23T00:46:06.777995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:06.777995+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":31,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":1,"num_write":13,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.1","version":"230'87","reported_seq":356,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.570486+0000","last_change":"2024-02-23T00:46:00.167004+0000","last_active":"2024-02-23T00:47:06.570486+0000","last_peered":"2024-02-23T00:47:06.570486+0000","last_clean":"2024-02-23T00:47:06.570486+0000","last_became_active":"2024-02-23T00:43:17.851866+0000","last_became_peered":"2024-02-23T00:43:17.851866+0000","last_unstale":"2024-02-23T00:47:06.570486+0000","last_undegraded":"2024-02-23T00:47:06.570486+0000","last_fullsized":"2024-02-23T00:47:06.570486+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":87,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":63,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.2","version":"229'68","reported_seq":341,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.419348+0000","last_change":"2024-02-23T00:46:00.168915+0000","last_active":"2024-02-23T00:47:05.419348+0000","last_peered":"2024-02-23T00:47:05.419348+0000","last_clean":"2024-02-23T00:47:05.419348+0000","last_became_active":"2024-02-23T00:43:17.804792+0000","last_became_peered":"2024-02-23T00:43:17.804792+0000","last_unstale":"2024-02-23T00:47:05.419348+0000","last_undegraded":"2024-02-23T00:47:05.419348+0000","last_fullsized":"2024-02-23T00:47:05.419348+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'68","last_scrub_stamp":"2024-02-23T00:45:57.135726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:57.135726+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.3","version":"246'20","reported_seq":305,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.664432+0000","last_change":"2024-02-23T00:46:00.172490+0000","last_active":"2024-02-23T00:47:05.664432+0000","last_peered":"2024-02-23T00:47:05.664432+0000","last_clean":"2024-02-23T00:47:05.664432+0000","last_became_active":"2024-02-23T00:43:17.834686+0000","last_became_peered":"2024-02-23T00:43:17.834686+0000","last_unstale":"2024-02-23T00:47:05.664432+0000","last_undegraded":"2024-02-23T00:47:05.664432+0000","last_fullsized":"2024-02-23T00:47:05.664432+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"246'20","last_scrub_stamp":"2024-02-23T00:45:53.067737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:53.067737+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.4","version":"207'17","reported_seq":297,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.162260+0000","last_change":"2024-02-23T00:46:35.627548+0000","last_active":"2024-02-23T00:47:05.162260+0000","last_peered":"2024-02-23T00:47:05.162260+0000","last_clean":"2024-02-23T00:47:05.162260+0000","last_became_active":"2024-02-23T00:43:17.810343+0000","last_became_peered":"2024-02-23T00:43:17.810343+0000","last_unstale":"2024-02-23T00:47:05.162260+0000","last_undegraded":"2024-02-23T00:47:05.162260+0000","last_fullsized":"2024-02-23T00:47:05.162260+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"207'17","last_scrub_stamp":"2024-02-23T00:46:35.627480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:35.627480+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.5","version":"237'51","reported_seq":333,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.571763+0000","last_change":"2024-02-23T00:46:40.379258+0000","last_active":"2024-02-23T00:47:06.571763+0000","last_peered":"2024-02-23T00:47:06.571763+0000","last_clean":"2024-02-23T00:47:06.571763+0000","last_became_active":"2024-02-23T00:43:17.855339+0000","last_became_peered":"2024-02-23T00:43:17.855339+0000","last_unstale":"2024-02-23T00:47:06.571763+0000","last_undegraded":"2024-02-23T00:47:06.571763+0000","last_fullsized":"2024-02-23T00:47:06.571763+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'51","last_scrub_stamp":"2024-02-23T00:46:40.379159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:40.379159+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.6","version":"347'52","reported_seq":334,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.485242+0000","last_change":"2024-02-23T00:46:00.726103+0000","last_active":"2024-02-23T00:47:05.485242+0000","last_peered":"2024-02-23T00:47:05.485242+0000","last_clean":"2024-02-23T00:47:05.485242+0000","last_became_active":"2024-02-23T00:43:17.838838+0000","last_became_peered":"2024-02-23T00:43:17.838838+0000","last_unstale":"2024-02-23T00:47:05.485242+0000","last_undegraded":"2024-02-23T00:47:05.485242+0000","last_fullsized":"2024-02-23T00:47:05.485242+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"247'51","last_scrub_stamp":"2024-02-23T00:46:00.726012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:00.726012+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.7","version":"347'69","reported_seq":336,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.666451+0000","last_change":"2024-02-23T00:46:00.175978+0000","last_active":"2024-02-23T00:47:05.666451+0000","last_peered":"2024-02-23T00:47:05.666451+0000","last_clean":"2024-02-23T00:47:05.666451+0000","last_became_active":"2024-02-23T00:43:17.846548+0000","last_became_peered":"2024-02-23T00:43:17.846548+0000","last_unstale":"2024-02-23T00:47:05.666451+0000","last_undegraded":"2024-02-23T00:47:05.666451+0000","last_fullsized":"2024-02-23T00:47:05.666451+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'68","last_scrub_stamp":"2024-02-23T00:45:25.351414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.351414+0000","log_size":69,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":52,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":21,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.8","version":"347'91","reported_seq":360,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.483752+0000","last_change":"2024-02-23T00:46:05.820778+0000","last_active":"2024-02-23T00:47:05.483752+0000","last_peered":"2024-02-23T00:47:05.483752+0000","last_clean":"2024-02-23T00:47:05.483752+0000","last_became_active":"2024-02-23T00:43:17.842613+0000","last_became_peered":"2024-02-23T00:43:17.842613+0000","last_unstale":"2024-02-23T00:47:05.483752+0000","last_undegraded":"2024-02-23T00:47:05.483752+0000","last_fullsized":"2024-02-23T00:47:05.483752+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'90","last_scrub_stamp":"2024-02-23T00:46:05.820651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:05.820651+0000","log_size":91,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":68,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":28,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.9","version":"259'39","reported_seq":325,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.572673+0000","last_change":"2024-02-23T00:46:10.439444+0000","last_active":"2024-02-23T00:47:06.572673+0000","last_peered":"2024-02-23T00:47:06.572673+0000","last_clean":"2024-02-23T00:47:06.572673+0000","last_became_active":"2024-02-23T00:43:17.834297+0000","last_became_peered":"2024-02-23T00:43:17.834297+0000","last_unstale":"2024-02-23T00:47:06.572673+0000","last_undegraded":"2024-02-23T00:47:06.572673+0000","last_fullsized":"2024-02-23T00:47:06.572673+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'39","last_scrub_stamp":"2024-02-23T00:46:10.439373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:10.439373+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":12,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.a","version":"347'91","reported_seq":361,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.483904+0000","last_change":"2024-02-23T00:46:00.166527+0000","last_active":"2024-02-23T00:47:05.483904+0000","last_peered":"2024-02-23T00:47:05.483904+0000","last_clean":"2024-02-23T00:47:05.483904+0000","last_became_active":"2024-02-23T00:43:17.839142+0000","last_became_peered":"2024-02-23T00:43:17.839142+0000","last_unstale":"2024-02-23T00:47:05.483904+0000","last_undegraded":"2024-02-23T00:47:05.483904+0000","last_fullsized":"2024-02-23T00:47:05.483904+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"253'90","last_scrub_stamp":"2024-02-23T00:45:59.681669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:59.681669+0000","log_size":91,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":68,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":28,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.b","version":"347'77","reported_seq":348,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.484624+0000","last_change":"2024-02-23T00:46:07.817973+0000","last_active":"2024-02-23T00:47:05.484624+0000","last_peered":"2024-02-23T00:47:05.484624+0000","last_clean":"2024-02-23T00:47:05.484624+0000","last_became_active":"2024-02-23T00:43:17.855321+0000","last_became_peered":"2024-02-23T00:43:17.855321+0000","last_unstale":"2024-02-23T00:47:05.484624+0000","last_undegraded":"2024-02-23T00:47:05.484624+0000","last_fullsized":"2024-02-23T00:47:05.484624+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"288'76","last_scrub_stamp":"2024-02-23T00:46:07.817883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:07.817883+0000","log_size":77,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":54,"num_objects":7,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":24,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.c","version":"203'34","reported_seq":315,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.162287+0000","last_change":"2024-02-23T00:46:00.163652+0000","last_active":"2024-02-23T00:47:06.162287+0000","last_peered":"2024-02-23T00:47:06.162287+0000","last_clean":"2024-02-23T00:47:06.162287+0000","last_became_active":"2024-02-23T00:43:17.829044+0000","last_became_peered":"2024-02-23T00:43:17.829044+0000","last_unstale":"2024-02-23T00:47:06.162287+0000","last_undegraded":"2024-02-23T00:47:06.162287+0000","last_fullsized":"2024-02-23T00:47:06.162287+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'34","last_scrub_stamp":"2024-02-23T00:45:57.874883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:57.874883+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.d","version":"259'56","reported_seq":330,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.571714+0000","last_change":"2024-02-23T00:46:00.159535+0000","last_active":"2024-02-23T00:47:06.571714+0000","last_peered":"2024-02-23T00:47:06.571714+0000","last_clean":"2024-02-23T00:47:06.571714+0000","last_became_active":"2024-02-23T00:43:17.841614+0000","last_became_peered":"2024-02-23T00:43:17.841614+0000","last_unstale":"2024-02-23T00:47:06.571714+0000","last_undegraded":"2024-02-23T00:47:06.571714+0000","last_fullsized":"2024-02-23T00:47:06.571714+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'56","last_scrub_stamp":"2024-02-23T00:45:51.258745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:51.258745+0000","log_size":56,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":17,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.e","version":"211'51","reported_seq":324,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.162017+0000","last_change":"2024-02-23T00:46:11.491708+0000","last_active":"2024-02-23T00:47:05.162017+0000","last_peered":"2024-02-23T00:47:05.162017+0000","last_clean":"2024-02-23T00:47:05.162017+0000","last_became_active":"2024-02-23T00:43:17.845956+0000","last_became_peered":"2024-02-23T00:43:17.845956+0000","last_unstale":"2024-02-23T00:47:05.162017+0000","last_undegraded":"2024-02-23T00:47:05.162017+0000","last_fullsized":"2024-02-23T00:47:05.162017+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'51","last_scrub_stamp":"2024-02-23T00:46:11.491637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:11.491637+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.f","version":"241'136","reported_seq":397,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.571038+0000","last_change":"2024-02-23T00:46:35.318635+0000","last_active":"2024-02-23T00:47:06.571038+0000","last_peered":"2024-02-23T00:47:06.571038+0000","last_clean":"2024-02-23T00:47:06.571038+0000","last_became_active":"2024-02-23T00:43:17.855331+0000","last_became_peered":"2024-02-23T00:43:17.855331+0000","last_unstale":"2024-02-23T00:47:06.571038+0000","last_undegraded":"2024-02-23T00:47:06.571038+0000","last_fullsized":"2024-02-23T00:47:06.571038+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"241'136","last_scrub_stamp":"2024-02-23T00:46:35.318571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:35.318571+0000","log_size":136,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":88,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":40,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.10","version":"347'69","reported_seq":338,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.569571+0000","last_change":"2024-02-23T00:46:00.167847+0000","last_active":"2024-02-23T00:47:06.569571+0000","last_peered":"2024-02-23T00:47:06.569571+0000","last_clean":"2024-02-23T00:47:06.569571+0000","last_became_active":"2024-02-23T00:43:17.819865+0000","last_became_peered":"2024-02-23T00:43:17.819865+0000","last_unstale":"2024-02-23T00:47:06.569571+0000","last_undegraded":"2024-02-23T00:47:06.569571+0000","last_fullsized":"2024-02-23T00:47:06.569571+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":69,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":52,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":21,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.11","version":"347'53","reported_seq":319,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.570398+0000","last_change":"2024-02-23T00:46:00.164366+0000","last_active":"2024-02-23T00:47:06.570398+0000","last_peered":"2024-02-23T00:47:06.570398+0000","last_clean":"2024-02-23T00:47:06.570398+0000","last_became_active":"2024-02-23T00:43:17.852697+0000","last_became_peered":"2024-02-23T00:43:17.852697+0000","last_unstale":"2024-02-23T00:47:06.570398+0000","last_undegraded":"2024-02-23T00:47:06.570398+0000","last_fullsized":"2024-02-23T00:47:06.570398+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:43:16.533032+0000","log_size":53,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":49,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":17,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.12","version":"240'73","reported_seq":349,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.664057+0000","last_change":"2024-02-23T00:46:00.176356+0000","last_active":"2024-02-23T00:47:05.664057+0000","last_peered":"2024-02-23T00:47:05.664057+0000","last_clean":"2024-02-23T00:47:05.664057+0000","last_became_active":"2024-02-23T00:43:17.856118+0000","last_became_peered":"2024-02-23T00:43:17.856118+0000","last_unstale":"2024-02-23T00:47:05.664057+0000","last_undegraded":"2024-02-23T00:47:05.664057+0000","last_fullsized":"2024-02-23T00:47:05.664057+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'73","last_scrub_stamp":"2024-02-23T00:45:52.066508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:52.066508+0000","log_size":73,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":49,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.13","version":"205'34","reported_seq":314,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.164826+0000","last_change":"2024-02-23T00:46:09.499938+0000","last_active":"2024-02-23T00:47:05.164826+0000","last_peered":"2024-02-23T00:47:05.164826+0000","last_clean":"2024-02-23T00:47:05.164826+0000","last_became_active":"2024-02-23T00:43:17.857823+0000","last_became_peered":"2024-02-23T00:43:17.857823+0000","last_unstale":"2024-02-23T00:47:05.164826+0000","last_undegraded":"2024-02-23T00:47:05.164826+0000","last_fullsized":"2024-02-23T00:47:05.164826+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'34","last_scrub_stamp":"2024-02-23T00:46:09.499812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:09.499812+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.14","version":"347'46","reported_seq":323,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.484973+0000","last_change":"2024-02-23T00:46:03.795687+0000","last_active":"2024-02-23T00:47:05.484973+0000","last_peered":"2024-02-23T00:47:05.484973+0000","last_clean":"2024-02-23T00:47:05.484973+0000","last_became_active":"2024-02-23T00:43:17.816138+0000","last_became_peered":"2024-02-23T00:43:17.816138+0000","last_unstale":"2024-02-23T00:47:05.484973+0000","last_undegraded":"2024-02-23T00:47:05.484973+0000","last_fullsized":"2024-02-23T00:47:05.484973+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'39","last_scrub_stamp":"2024-02-23T00:46:03.795605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:03.795605+0000","log_size":46,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":51,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":19,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.15","version":"347'19","reported_seq":304,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.162001+0000","last_change":"2024-02-23T00:46:11.830641+0000","last_active":"2024-02-23T00:47:06.162001+0000","last_peered":"2024-02-23T00:47:06.162001+0000","last_clean":"2024-02-23T00:47:06.162001+0000","last_became_active":"2024-02-23T00:43:17.843834+0000","last_became_peered":"2024-02-23T00:43:17.843834+0000","last_unstale":"2024-02-23T00:47:06.162001+0000","last_undegraded":"2024-02-23T00:47:06.162001+0000","last_fullsized":"2024-02-23T00:47:06.162001+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"299'18","last_scrub_stamp":"2024-02-23T00:46:11.830580+0000","last_deep_scrub":"299'18","last_deep_scrub_stamp":"2024-02-23T00:46:11.830580+0000","last_clean_scrub_stamp":"2024-02-23T00:46:11.830580+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.16","version":"347'27","reported_seq":312,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.571277+0000","last_change":"2024-02-23T00:46:00.160705+0000","last_active":"2024-02-23T00:47:06.571277+0000","last_peered":"2024-02-23T00:47:06.571277+0000","last_clean":"2024-02-23T00:47:06.571277+0000","last_became_active":"2024-02-23T00:43:17.834291+0000","last_became_peered":"2024-02-23T00:43:17.834291+0000","last_unstale":"2024-02-23T00:47:06.571277+0000","last_undegraded":"2024-02-23T00:47:06.571277+0000","last_fullsized":"2024-02-23T00:47:06.571277+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'23","last_scrub_stamp":"2024-02-23T00:45:56.354881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:56.354881+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":46,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":12,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.17","version":"347'59","reported_seq":341,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.418399+0000","last_change":"2024-02-23T00:46:00.166482+0000","last_active":"2024-02-23T00:47:05.418399+0000","last_peered":"2024-02-23T00:47:05.418399+0000","last_clean":"2024-02-23T00:47:05.418399+0000","last_became_active":"2024-02-23T00:43:17.843525+0000","last_became_peered":"2024-02-23T00:43:17.843525+0000","last_unstale":"2024-02-23T00:47:05.418399+0000","last_undegraded":"2024-02-23T00:47:05.418399+0000","last_fullsized":"2024-02-23T00:47:05.418399+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'57","last_scrub_stamp":"2024-02-23T00:45:58.127302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:58.127302+0000","log_size":59,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":20,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.19","version":"267'96","reported_seq":373,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.419340+0000","last_change":"2024-02-23T00:46:00.163245+0000","last_active":"2024-02-23T00:47:05.419340+0000","last_peered":"2024-02-23T00:47:05.419340+0000","last_clean":"2024-02-23T00:47:05.419340+0000","last_became_active":"2024-02-23T00:43:17.815830+0000","last_became_peered":"2024-02-23T00:43:17.815830+0000","last_unstale":"2024-02-23T00:47:05.419340+0000","last_undegraded":"2024-02-23T00:47:05.419340+0000","last_fullsized":"2024-02-23T00:47:05.419340+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'96","last_scrub_stamp":"2024-02-23T00:45:55.109551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:55.109551+0000","log_size":96,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":73,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":7,"num_read_kb":0,"num_write":30,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.18","version":"275'7","reported_seq":297,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.161125+0000","last_change":"2024-02-23T00:46:03.927901+0000","last_active":"2024-02-23T00:47:06.161125+0000","last_peered":"2024-02-23T00:47:06.161125+0000","last_clean":"2024-02-23T00:47:06.161125+0000","last_became_active":"2024-02-23T00:43:17.840351+0000","last_became_peered":"2024-02-23T00:43:17.840351+0000","last_unstale":"2024-02-23T00:47:06.161125+0000","last_undegraded":"2024-02-23T00:47:06.161125+0000","last_fullsized":"2024-02-23T00:47:06.161125+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"275'7","last_scrub_stamp":"2024-02-23T00:46:03.927783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:03.927783+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1b","version":"347'70","reported_seq":345,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.573431+0000","last_change":"2024-02-23T00:46:37.397546+0000","last_active":"2024-02-23T00:47:06.573431+0000","last_peered":"2024-02-23T00:47:06.573431+0000","last_clean":"2024-02-23T00:47:06.573431+0000","last_became_active":"2024-02-23T00:43:17.803043+0000","last_became_peered":"2024-02-23T00:43:17.803043+0000","last_unstale":"2024-02-23T00:47:06.573431+0000","last_undegraded":"2024-02-23T00:47:06.573431+0000","last_fullsized":"2024-02-23T00:47:06.573431+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"304'69","last_scrub_stamp":"2024-02-23T00:46:37.397467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.397467+0000","log_size":70,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4000060,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":22,"num_write_kb":3928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.1a","version":"347'35","reported_seq":342,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.420086+0000","last_change":"2024-02-23T00:46:02.071916+0000","last_active":"2024-02-23T00:47:05.420086+0000","last_peered":"2024-02-23T00:47:05.420086+0000","last_clean":"2024-02-23T00:47:05.420086+0000","last_became_active":"2024-02-23T00:45:44.178195+0000","last_became_peered":"2024-02-23T00:45:44.178195+0000","last_unstale":"2024-02-23T00:47:05.420086+0000","last_undegraded":"2024-02-23T00:47:05.420086+0000","last_fullsized":"2024-02-23T00:47:05.420086+0000","mapping_epoch":271,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":272,"parent":"0.0","parent_split_bits":0,"last_scrub":"225'34","last_scrub_stamp":"2024-02-23T00:46:02.071770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:02.071770+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":30,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":11,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1d","version":"215'104","reported_seq":375,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.417991+0000","last_change":"2024-02-23T00:46:05.000709+0000","last_active":"2024-02-23T00:47:05.417991+0000","last_peered":"2024-02-23T00:47:05.417991+0000","last_clean":"2024-02-23T00:47:05.417991+0000","last_became_active":"2024-02-23T00:43:17.828194+0000","last_became_peered":"2024-02-23T00:43:17.828194+0000","last_unstale":"2024-02-23T00:47:05.417991+0000","last_undegraded":"2024-02-23T00:47:05.417991+0000","last_fullsized":"2024-02-23T00:47:05.417991+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'104","last_scrub_stamp":"2024-02-23T00:46:05.000615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:05.000615+0000","log_size":104,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":74,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":32,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"62.1c","version":"223'51","reported_seq":324,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.416974+0000","last_change":"2024-02-23T00:46:08.027717+0000","last_active":"2024-02-23T00:47:05.416974+0000","last_peered":"2024-02-23T00:47:05.416974+0000","last_clean":"2024-02-23T00:47:05.416974+0000","last_became_active":"2024-02-23T00:43:17.804718+0000","last_became_peered":"2024-02-23T00:43:17.804718+0000","last_unstale":"2024-02-23T00:47:05.416974+0000","last_undegraded":"2024-02-23T00:47:05.416974+0000","last_fullsized":"2024-02-23T00:47:05.416974+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'51","last_scrub_stamp":"2024-02-23T00:46:08.027648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:08.027648+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1f","version":"347'41","reported_seq":323,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.666358+0000","last_change":"2024-02-23T00:46:00.174886+0000","last_active":"2024-02-23T00:47:05.666358+0000","last_peered":"2024-02-23T00:47:05.666358+0000","last_clean":"2024-02-23T00:47:05.666358+0000","last_became_active":"2024-02-23T00:43:17.839914+0000","last_became_peered":"2024-02-23T00:43:17.839914+0000","last_unstale":"2024-02-23T00:47:05.666358+0000","last_undegraded":"2024-02-23T00:47:05.666358+0000","last_fullsized":"2024-02-23T00:47:05.666358+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'40","last_scrub_stamp":"2024-02-23T00:45:58.074860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:58.074860+0000","log_size":41,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":35,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":14,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.1e","version":"245'51","reported_seq":326,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.484958+0000","last_change":"2024-02-23T00:46:00.172469+0000","last_active":"2024-02-23T00:47:05.484958+0000","last_peered":"2024-02-23T00:47:05.484958+0000","last_clean":"2024-02-23T00:47:05.484958+0000","last_became_active":"2024-02-23T00:43:17.838682+0000","last_became_peered":"2024-02-23T00:43:17.838682+0000","last_unstale":"2024-02-23T00:47:05.484958+0000","last_undegraded":"2024-02-23T00:47:05.484958+0000","last_fullsized":"2024-02-23T00:47:05.484958+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"245'51","last_scrub_stamp":"2024-02-23T00:45:58.723990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:58.723990+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"67'8","reported_seq":416,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.484666+0000","last_change":"2024-02-23T00:45:55.708656+0000","last_active":"2024-02-23T00:47:05.484666+0000","last_peered":"2024-02-23T00:47:05.484666+0000","last_clean":"2024-02-23T00:47:05.484666+0000","last_became_active":"2024-02-23T00:39:18.793414+0000","last_became_peered":"2024-02-23T00:39:18.793414+0000","last_unstale":"2024-02-23T00:47:05.484666+0000","last_undegraded":"2024-02-23T00:47:05.484666+0000","last_fullsized":"2024-02-23T00:47:05.484666+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:45:55.708576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:55.708576+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"68'27","reported_seq":370,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.415990+0000","last_change":"2024-02-23T00:45:04.138883+0000","last_active":"2024-02-23T00:47:05.415990+0000","last_peered":"2024-02-23T00:47:05.415990+0000","last_clean":"2024-02-23T00:47:05.415990+0000","last_became_active":"2024-02-23T00:39:16.859401+0000","last_became_peered":"2024-02-23T00:39:16.859401+0000","last_unstale":"2024-02-23T00:47:05.415990+0000","last_undegraded":"2024-02-23T00:47:05.415990+0000","last_fullsized":"2024-02-23T00:47:05.415990+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'27","last_scrub_stamp":"2024-02-23T00:45:04.138751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.138751+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":321,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.666791+0000","last_change":"2024-02-23T00:46:04.942156+0000","last_active":"2024-02-23T00:47:05.666791+0000","last_peered":"2024-02-23T00:47:05.666791+0000","last_clean":"2024-02-23T00:47:05.666791+0000","last_became_active":"2024-02-23T00:39:14.784436+0000","last_became_peered":"2024-02-23T00:39:14.784436+0000","last_unstale":"2024-02-23T00:47:05.666791+0000","last_undegraded":"2024-02-23T00:47:05.666791+0000","last_fullsized":"2024-02-23T00:47:05.666791+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.942091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:04.330426+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.942091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":329,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.416168+0000","last_change":"2024-02-23T00:44:21.972446+0000","last_active":"2024-02-23T00:47:05.416168+0000","last_peered":"2024-02-23T00:47:05.416168+0000","last_clean":"2024-02-23T00:47:05.416168+0000","last_became_active":"2024-02-23T00:43:32.282545+0000","last_became_peered":"2024-02-23T00:43:32.282545+0000","last_unstale":"2024-02-23T00:47:05.416168+0000","last_undegraded":"2024-02-23T00:47:05.416168+0000","last_fullsized":"2024-02-23T00:47:05.416168+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:21.972383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:21.972383+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":258,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.417547+0000","last_change":"2024-02-23T00:45:17.181400+0000","last_active":"2024-02-23T00:47:05.417547+0000","last_peered":"2024-02-23T00:47:05.417547+0000","last_clean":"2024-02-23T00:47:05.417547+0000","last_became_active":"2024-02-23T00:42:58.828846+0000","last_became_peered":"2024-02-23T00:42:58.828846+0000","last_unstale":"2024-02-23T00:47:05.417547+0000","last_undegraded":"2024-02-23T00:47:05.417547+0000","last_fullsized":"2024-02-23T00:47:05.417547+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:17.181335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:17.181335+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"68'15","reported_seq":351,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.167097+0000","last_change":"2024-02-23T00:46:45.625235+0000","last_active":"2024-02-23T00:47:05.167097+0000","last_peered":"2024-02-23T00:47:05.167097+0000","last_clean":"2024-02-23T00:47:05.167097+0000","last_became_active":"2024-02-23T00:39:16.783701+0000","last_became_peered":"2024-02-23T00:39:16.783701+0000","last_unstale":"2024-02-23T00:47:05.167097+0000","last_undegraded":"2024-02-23T00:47:05.167097+0000","last_fullsized":"2024-02-23T00:47:05.167097+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:46:45.625126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:45.625126+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":316,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.570781+0000","last_change":"2024-02-23T00:46:04.476277+0000","last_active":"2024-02-23T00:47:06.570781+0000","last_peered":"2024-02-23T00:47:06.570781+0000","last_clean":"2024-02-23T00:47:06.570781+0000","last_became_active":"2024-02-23T00:39:14.772493+0000","last_became_peered":"2024-02-23T00:39:14.772493+0000","last_unstale":"2024-02-23T00:47:06.570781+0000","last_undegraded":"2024-02-23T00:47:06.570781+0000","last_fullsized":"2024-02-23T00:47:06.570781+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.476156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.476156+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":317,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.664899+0000","last_change":"2024-02-23T00:46:45.674467+0000","last_active":"2024-02-23T00:47:05.664899+0000","last_peered":"2024-02-23T00:47:05.664899+0000","last_clean":"2024-02-23T00:47:05.664899+0000","last_became_active":"2024-02-23T00:39:18.794476+0000","last_became_peered":"2024-02-23T00:39:18.794476+0000","last_unstale":"2024-02-23T00:47:05.664899+0000","last_undegraded":"2024-02-23T00:47:05.664899+0000","last_fullsized":"2024-02-23T00:47:05.664899+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:45.674391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:08.335330+0000","last_clean_scrub_stamp":"2024-02-23T00:46:45.674391+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":329,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.167026+0000","last_change":"2024-02-23T00:45:50.685890+0000","last_active":"2024-02-23T00:47:05.167026+0000","last_peered":"2024-02-23T00:47:05.167026+0000","last_clean":"2024-02-23T00:47:05.167026+0000","last_became_active":"2024-02-23T00:43:35.079004+0000","last_became_peered":"2024-02-23T00:43:35.079004+0000","last_unstale":"2024-02-23T00:47:05.167026+0000","last_undegraded":"2024-02-23T00:47:05.167026+0000","last_fullsized":"2024-02-23T00:47:05.167026+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:50.685824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:45:50.685824+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":258,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.163667+0000","last_change":"2024-02-23T00:45:22.597544+0000","last_active":"2024-02-23T00:47:05.163667+0000","last_peered":"2024-02-23T00:47:05.163667+0000","last_clean":"2024-02-23T00:47:05.163667+0000","last_became_active":"2024-02-23T00:42:58.867305+0000","last_became_peered":"2024-02-23T00:42:58.867305+0000","last_unstale":"2024-02-23T00:47:05.163667+0000","last_undegraded":"2024-02-23T00:47:05.163667+0000","last_fullsized":"2024-02-23T00:47:05.163667+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:22.597434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:22.597434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"88'35","reported_seq":374,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.419956+0000","last_change":"2024-02-23T00:46:41.290546+0000","last_active":"2024-02-23T00:47:05.419956+0000","last_peered":"2024-02-23T00:47:05.419956+0000","last_clean":"2024-02-23T00:47:05.419956+0000","last_became_active":"2024-02-23T00:39:16.857413+0000","last_became_peered":"2024-02-23T00:39:16.857413+0000","last_unstale":"2024-02-23T00:47:05.419956+0000","last_undegraded":"2024-02-23T00:47:05.419956+0000","last_fullsized":"2024-02-23T00:47:05.419956+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'35","last_scrub_stamp":"2024-02-23T00:46:41.290478+0000","last_deep_scrub":"88'35","last_deep_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_clean_scrub_stamp":"2024-02-23T00:46:41.290478+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":321,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.666031+0000","last_change":"2024-02-23T00:46:05.910392+0000","last_active":"2024-02-23T00:47:05.666031+0000","last_peered":"2024-02-23T00:47:05.666031+0000","last_clean":"2024-02-23T00:47:05.666031+0000","last_became_active":"2024-02-23T00:39:14.776705+0000","last_became_peered":"2024-02-23T00:39:14.776705+0000","last_unstale":"2024-02-23T00:47:05.666031+0000","last_undegraded":"2024-02-23T00:47:05.666031+0000","last_fullsized":"2024-02-23T00:47:05.666031+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:05.910325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_clean_scrub_stamp":"2024-02-23T00:46:05.910325+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":317,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.664523+0000","last_change":"2024-02-23T00:46:34.726091+0000","last_active":"2024-02-23T00:47:05.664523+0000","last_peered":"2024-02-23T00:47:05.664523+0000","last_clean":"2024-02-23T00:47:05.664523+0000","last_became_active":"2024-02-23T00:39:18.792718+0000","last_became_peered":"2024-02-23T00:39:18.792718+0000","last_unstale":"2024-02-23T00:47:05.664523+0000","last_undegraded":"2024-02-23T00:47:05.664523+0000","last_fullsized":"2024-02-23T00:47:05.664523+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:34.726027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:34.726027+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":332,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.484163+0000","last_change":"2024-02-23T00:46:48.692753+0000","last_active":"2024-02-23T00:47:05.484163+0000","last_peered":"2024-02-23T00:47:05.484163+0000","last_clean":"2024-02-23T00:47:05.484163+0000","last_became_active":"2024-02-23T00:43:39.470999+0000","last_became_peered":"2024-02-23T00:43:39.470999+0000","last_unstale":"2024-02-23T00:47:05.484163+0000","last_undegraded":"2024-02-23T00:47:05.484163+0000","last_fullsized":"2024-02-23T00:47:05.484163+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:48.692679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:48.692679+0000","last_clean_scrub_stamp":"2024-02-23T00:46:48.692679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":258,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.570622+0000","last_change":"2024-02-23T00:46:40.354058+0000","last_active":"2024-02-23T00:47:06.570622+0000","last_peered":"2024-02-23T00:47:06.570622+0000","last_clean":"2024-02-23T00:47:06.570622+0000","last_became_active":"2024-02-23T00:42:57.639013+0000","last_became_peered":"2024-02-23T00:42:57.639013+0000","last_unstale":"2024-02-23T00:47:06.570622+0000","last_undegraded":"2024-02-23T00:47:06.570622+0000","last_fullsized":"2024-02-23T00:47:06.570622+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:40.353999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:46:40.353999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"67'17","reported_seq":366,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.572068+0000","last_change":"2024-02-23T00:43:45.194207+0000","last_active":"2024-02-23T00:47:06.572068+0000","last_peered":"2024-02-23T00:47:06.572068+0000","last_clean":"2024-02-23T00:47:06.572068+0000","last_became_active":"2024-02-23T00:40:42.933806+0000","last_became_peered":"2024-02-23T00:40:42.933806+0000","last_unstale":"2024-02-23T00:47:06.572068+0000","last_undegraded":"2024-02-23T00:47:06.572068+0000","last_fullsized":"2024-02-23T00:47:06.572068+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'17","last_scrub_stamp":"2024-02-23T00:43:45.194139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:45.194139+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":292,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.664747+0000","last_change":"2024-02-23T00:45:56.051644+0000","last_active":"2024-02-23T00:47:05.664747+0000","last_peered":"2024-02-23T00:47:05.664747+0000","last_clean":"2024-02-23T00:47:05.664747+0000","last_became_active":"2024-02-23T00:40:42.931041+0000","last_became_peered":"2024-02-23T00:40:42.931041+0000","last_unstale":"2024-02-23T00:47:05.664747+0000","last_undegraded":"2024-02-23T00:47:05.664747+0000","last_fullsized":"2024-02-23T00:47:05.664747+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:56.051575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:56.051575+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":312,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.571031+0000","last_change":"2024-02-23T00:44:22.804352+0000","last_active":"2024-02-23T00:47:06.571031+0000","last_peered":"2024-02-23T00:47:06.571031+0000","last_clean":"2024-02-23T00:47:06.571031+0000","last_became_active":"2024-02-23T00:39:18.799875+0000","last_became_peered":"2024-02-23T00:39:18.799875+0000","last_unstale":"2024-02-23T00:47:06.571031+0000","last_undegraded":"2024-02-23T00:47:06.571031+0000","last_fullsized":"2024-02-23T00:47:06.571031+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:22.804279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:22.804279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":328,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.666712+0000","last_change":"2024-02-23T00:44:36.481327+0000","last_active":"2024-02-23T00:47:05.666712+0000","last_peered":"2024-02-23T00:47:05.666712+0000","last_clean":"2024-02-23T00:47:05.666712+0000","last_became_active":"2024-02-23T00:43:08.885257+0000","last_became_peered":"2024-02-23T00:43:08.885257+0000","last_unstale":"2024-02-23T00:47:05.666712+0000","last_undegraded":"2024-02-23T00:47:05.666712+0000","last_fullsized":"2024-02-23T00:47:05.666712+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:36.481256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:36.481256+0000","last_clean_scrub_stamp":"2024-02-23T00:44:36.481256+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":258,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.571551+0000","last_change":"2024-02-23T00:45:23.744990+0000","last_active":"2024-02-23T00:47:06.571551+0000","last_peered":"2024-02-23T00:47:06.571551+0000","last_clean":"2024-02-23T00:47:06.571551+0000","last_became_active":"2024-02-23T00:42:57.675306+0000","last_became_peered":"2024-02-23T00:42:57.675306+0000","last_unstale":"2024-02-23T00:47:06.571551+0000","last_undegraded":"2024-02-23T00:47:06.571551+0000","last_fullsized":"2024-02-23T00:47:06.571551+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:23.744846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.744846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"67'10","reported_seq":331,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.570310+0000","last_change":"2024-02-23T00:45:25.175258+0000","last_active":"2024-02-23T00:47:06.570310+0000","last_peered":"2024-02-23T00:47:06.570310+0000","last_clean":"2024-02-23T00:47:06.570310+0000","last_became_active":"2024-02-23T00:39:16.777530+0000","last_became_peered":"2024-02-23T00:39:16.777530+0000","last_unstale":"2024-02-23T00:47:06.570310+0000","last_undegraded":"2024-02-23T00:47:06.570310+0000","last_fullsized":"2024-02-23T00:47:06.570310+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'10","last_scrub_stamp":"2024-02-23T00:45:25.175173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.175173+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":316,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.570939+0000","last_change":"2024-02-23T00:45:10.693332+0000","last_active":"2024-02-23T00:47:06.570939+0000","last_peered":"2024-02-23T00:47:06.570939+0000","last_clean":"2024-02-23T00:47:06.570939+0000","last_became_active":"2024-02-23T00:39:14.780379+0000","last_became_peered":"2024-02-23T00:39:14.780379+0000","last_unstale":"2024-02-23T00:47:06.570939+0000","last_undegraded":"2024-02-23T00:47:06.570939+0000","last_fullsized":"2024-02-23T00:47:06.570939+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.693273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.575242+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.693273+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"67'8","reported_seq":419,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.664678+0000","last_change":"2024-02-23T00:46:49.665345+0000","last_active":"2024-02-23T00:47:05.664678+0000","last_peered":"2024-02-23T00:47:05.664678+0000","last_clean":"2024-02-23T00:47:05.664678+0000","last_became_active":"2024-02-23T00:39:18.799544+0000","last_became_peered":"2024-02-23T00:39:18.799544+0000","last_unstale":"2024-02-23T00:47:05.664678+0000","last_undegraded":"2024-02-23T00:47:05.664678+0000","last_fullsized":"2024-02-23T00:47:05.664678+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:46:49.665271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:49.665271+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":332,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.571577+0000","last_change":"2024-02-23T00:46:42.326628+0000","last_active":"2024-02-23T00:47:06.571577+0000","last_peered":"2024-02-23T00:47:06.571577+0000","last_clean":"2024-02-23T00:47:06.571577+0000","last_became_active":"2024-02-23T00:43:13.338697+0000","last_became_peered":"2024-02-23T00:43:13.338697+0000","last_unstale":"2024-02-23T00:47:06.571577+0000","last_undegraded":"2024-02-23T00:47:06.571577+0000","last_fullsized":"2024-02-23T00:47:06.571577+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:42.326566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:15.081153+0000","last_clean_scrub_stamp":"2024-02-23T00:46:42.326566+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":260,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.161798+0000","last_change":"2024-02-23T00:45:22.874761+0000","last_active":"2024-02-23T00:47:06.161798+0000","last_peered":"2024-02-23T00:47:06.161798+0000","last_clean":"2024-02-23T00:47:06.161798+0000","last_became_active":"2024-02-23T00:42:58.765106+0000","last_became_peered":"2024-02-23T00:42:58.765106+0000","last_unstale":"2024-02-23T00:47:06.161798+0000","last_undegraded":"2024-02-23T00:47:06.161798+0000","last_fullsized":"2024-02-23T00:47:06.161798+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:22.874696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:22.874696+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"67'15","reported_seq":339,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.571043+0000","last_change":"2024-02-23T00:44:19.830383+0000","last_active":"2024-02-23T00:47:06.571043+0000","last_peered":"2024-02-23T00:47:06.571043+0000","last_clean":"2024-02-23T00:47:06.571043+0000","last_became_active":"2024-02-23T00:39:16.789757+0000","last_became_peered":"2024-02-23T00:39:16.789757+0000","last_unstale":"2024-02-23T00:47:06.571043+0000","last_undegraded":"2024-02-23T00:47:06.571043+0000","last_fullsized":"2024-02-23T00:47:06.571043+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:44:19.830309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:44:19.830309+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":316,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.570842+0000","last_change":"2024-02-23T00:44:29.773000+0000","last_active":"2024-02-23T00:47:06.570842+0000","last_peered":"2024-02-23T00:47:06.570842+0000","last_clean":"2024-02-23T00:47:06.570842+0000","last_became_active":"2024-02-23T00:39:14.780526+0000","last_became_peered":"2024-02-23T00:39:14.780526+0000","last_unstale":"2024-02-23T00:47:06.570842+0000","last_undegraded":"2024-02-23T00:47:06.570842+0000","last_fullsized":"2024-02-23T00:47:06.570842+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:29.772933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:29.772933+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":416,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.571463+0000","last_change":"2024-02-23T00:44:23.109508+0000","last_active":"2024-02-23T00:47:06.571463+0000","last_peered":"2024-02-23T00:47:06.571463+0000","last_clean":"2024-02-23T00:47:06.571463+0000","last_became_active":"2024-02-23T00:39:18.800440+0000","last_became_peered":"2024-02-23T00:39:18.800440+0000","last_unstale":"2024-02-23T00:47:06.571463+0000","last_undegraded":"2024-02-23T00:47:06.571463+0000","last_fullsized":"2024-02-23T00:47:06.571463+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:44:23.109439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.109439+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":334,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.418633+0000","last_change":"2024-02-23T00:46:48.130175+0000","last_active":"2024-02-23T00:47:05.418633+0000","last_peered":"2024-02-23T00:47:05.418633+0000","last_clean":"2024-02-23T00:47:05.418633+0000","last_became_active":"2024-02-23T00:43:16.650527+0000","last_became_peered":"2024-02-23T00:43:16.650527+0000","last_unstale":"2024-02-23T00:47:05.418633+0000","last_undegraded":"2024-02-23T00:47:05.418633+0000","last_fullsized":"2024-02-23T00:47:05.418633+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:48.130104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:48.130104+0000","last_clean_scrub_stamp":"2024-02-23T00:46:48.130104+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":260,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.572706+0000","last_change":"2024-02-23T00:45:54.311565+0000","last_active":"2024-02-23T00:47:06.572706+0000","last_peered":"2024-02-23T00:47:06.572706+0000","last_clean":"2024-02-23T00:47:06.572706+0000","last_became_active":"2024-02-23T00:42:58.833924+0000","last_became_peered":"2024-02-23T00:42:58.833924+0000","last_unstale":"2024-02-23T00:47:06.572706+0000","last_undegraded":"2024-02-23T00:47:06.572706+0000","last_fullsized":"2024-02-23T00:47:06.572706+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:54.311442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:54.311442+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"336'23","reported_seq":379,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.571368+0000","last_change":"2024-02-23T00:46:48.320148+0000","last_active":"2024-02-23T00:47:06.571368+0000","last_peered":"2024-02-23T00:47:06.571368+0000","last_clean":"2024-02-23T00:47:06.571368+0000","last_became_active":"2024-02-23T00:40:42.933759+0000","last_became_peered":"2024-02-23T00:40:42.933759+0000","last_unstale":"2024-02-23T00:47:06.571368+0000","last_undegraded":"2024-02-23T00:47:06.571368+0000","last_fullsized":"2024-02-23T00:47:06.571368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"336'23","last_scrub_stamp":"2024-02-23T00:46:48.320085+0000","last_deep_scrub":"336'23","last_deep_scrub_stamp":"2024-02-23T00:46:48.320085+0000","last_clean_scrub_stamp":"2024-02-23T00:46:48.320085+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":320,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.484831+0000","last_change":"2024-02-23T00:46:43.722544+0000","last_active":"2024-02-23T00:47:05.484831+0000","last_peered":"2024-02-23T00:47:05.484831+0000","last_clean":"2024-02-23T00:47:05.484831+0000","last_became_active":"2024-02-23T00:39:14.784121+0000","last_became_peered":"2024-02-23T00:39:14.784121+0000","last_unstale":"2024-02-23T00:47:05.484831+0000","last_undegraded":"2024-02-23T00:47:05.484831+0000","last_fullsized":"2024-02-23T00:47:05.484831+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:43.722480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:43.722480+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":313,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.419407+0000","last_change":"2024-02-23T00:45:08.118391+0000","last_active":"2024-02-23T00:47:05.419407+0000","last_peered":"2024-02-23T00:47:05.419407+0000","last_clean":"2024-02-23T00:47:05.419407+0000","last_became_active":"2024-02-23T00:39:18.800777+0000","last_became_peered":"2024-02-23T00:39:18.800777+0000","last_unstale":"2024-02-23T00:47:05.419407+0000","last_undegraded":"2024-02-23T00:47:05.419407+0000","last_fullsized":"2024-02-23T00:47:05.419407+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.118302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.118302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"67'1","reported_seq":331,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.162203+0000","last_change":"2024-02-23T00:43:56.091057+0000","last_active":"2024-02-23T00:47:06.162203+0000","last_peered":"2024-02-23T00:47:06.162203+0000","last_clean":"2024-02-23T00:47:06.162203+0000","last_became_active":"2024-02-23T00:43:22.866277+0000","last_became_peered":"2024-02-23T00:43:22.866277+0000","last_unstale":"2024-02-23T00:47:06.162203+0000","last_undegraded":"2024-02-23T00:47:06.162203+0000","last_fullsized":"2024-02-23T00:47:06.162203+0000","mapping_epoch":65,"log_start":"67'1","ondisk_log_start":"67'1","created":65,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:43:56.090982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:09.079918+0000","last_clean_scrub_stamp":"2024-02-23T00:43:56.090982+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":258,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.569950+0000","last_change":"2024-02-23T00:46:35.407560+0000","last_active":"2024-02-23T00:47:06.569950+0000","last_peered":"2024-02-23T00:47:06.569950+0000","last_clean":"2024-02-23T00:47:06.569950+0000","last_became_active":"2024-02-23T00:42:57.653279+0000","last_became_peered":"2024-02-23T00:42:57.653279+0000","last_unstale":"2024-02-23T00:47:06.569950+0000","last_undegraded":"2024-02-23T00:47:06.569950+0000","last_fullsized":"2024-02-23T00:47:06.569950+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:35.407468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:46:35.407468+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":417,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.161333+0000","last_change":"2024-02-23T00:45:51.871688+0000","last_active":"2024-02-23T00:47:06.161333+0000","last_peered":"2024-02-23T00:47:06.161333+0000","last_clean":"2024-02-23T00:47:06.161333+0000","last_became_active":"2024-02-23T00:39:18.798536+0000","last_became_peered":"2024-02-23T00:39:18.798536+0000","last_unstale":"2024-02-23T00:47:06.161333+0000","last_undegraded":"2024-02-23T00:47:06.161333+0000","last_fullsized":"2024-02-23T00:47:06.161333+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:45:51.871618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:51.871618+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"68'20","reported_seq":352,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.572817+0000","last_change":"2024-02-23T00:45:05.291909+0000","last_active":"2024-02-23T00:47:06.572817+0000","last_peered":"2024-02-23T00:47:06.572817+0000","last_clean":"2024-02-23T00:47:06.572817+0000","last_became_active":"2024-02-23T00:39:16.780130+0000","last_became_peered":"2024-02-23T00:39:16.780130+0000","last_unstale":"2024-02-23T00:47:06.572817+0000","last_undegraded":"2024-02-23T00:47:06.572817+0000","last_fullsized":"2024-02-23T00:47:06.572817+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'20","last_scrub_stamp":"2024-02-23T00:45:05.291836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:05.291836+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":322,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.419331+0000","last_change":"2024-02-23T00:46:35.220317+0000","last_active":"2024-02-23T00:47:05.419331+0000","last_peered":"2024-02-23T00:47:05.419331+0000","last_clean":"2024-02-23T00:47:05.419331+0000","last_became_active":"2024-02-23T00:39:14.781999+0000","last_became_peered":"2024-02-23T00:39:14.781999+0000","last_unstale":"2024-02-23T00:47:05.419331+0000","last_undegraded":"2024-02-23T00:47:05.419331+0000","last_fullsized":"2024-02-23T00:47:05.419331+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:35.220225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:51.391939+0000","last_clean_scrub_stamp":"2024-02-23T00:46:35.220225+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":314,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.485410+0000","last_change":"2024-02-23T00:46:46.686819+0000","last_active":"2024-02-23T00:47:05.485410+0000","last_peered":"2024-02-23T00:47:05.485410+0000","last_clean":"2024-02-23T00:47:05.485410+0000","last_became_active":"2024-02-23T00:39:21.161928+0000","last_became_peered":"2024-02-23T00:39:21.161928+0000","last_unstale":"2024-02-23T00:47:05.485410+0000","last_undegraded":"2024-02-23T00:47:05.485410+0000","last_fullsized":"2024-02-23T00:47:05.485410+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:46.686764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:46.686764+0000","last_clean_scrub_stamp":"2024-02-23T00:46:46.686764+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":258,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.419547+0000","last_change":"2024-02-23T00:45:59.944702+0000","last_active":"2024-02-23T00:47:05.419547+0000","last_peered":"2024-02-23T00:47:05.419547+0000","last_clean":"2024-02-23T00:47:05.419547+0000","last_became_active":"2024-02-23T00:42:58.717371+0000","last_became_peered":"2024-02-23T00:42:58.717371+0000","last_unstale":"2024-02-23T00:47:05.419547+0000","last_undegraded":"2024-02-23T00:47:05.419547+0000","last_fullsized":"2024-02-23T00:47:05.419547+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:59.944587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:59.944587+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":314,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.573072+0000","last_change":"2024-02-23T00:44:34.059194+0000","last_active":"2024-02-23T00:47:06.573072+0000","last_peered":"2024-02-23T00:47:06.573072+0000","last_clean":"2024-02-23T00:47:06.573072+0000","last_became_active":"2024-02-23T00:39:18.798829+0000","last_became_peered":"2024-02-23T00:39:18.798829+0000","last_unstale":"2024-02-23T00:47:06.573072+0000","last_undegraded":"2024-02-23T00:47:06.573072+0000","last_fullsized":"2024-02-23T00:47:06.573072+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:34.059129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:34.059129+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"251'40","reported_seq":389,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.161349+0000","last_change":"2024-02-23T00:45:23.910609+0000","last_active":"2024-02-23T00:47:06.161349+0000","last_peered":"2024-02-23T00:47:06.161349+0000","last_clean":"2024-02-23T00:47:06.161349+0000","last_became_active":"2024-02-23T00:39:16.859002+0000","last_became_peered":"2024-02-23T00:39:16.859002+0000","last_unstale":"2024-02-23T00:47:06.161349+0000","last_undegraded":"2024-02-23T00:47:06.161349+0000","last_fullsized":"2024-02-23T00:47:06.161349+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'40","last_scrub_stamp":"2024-02-23T00:45:23.910546+0000","last_deep_scrub":"251'40","last_deep_scrub_stamp":"2024-02-23T00:45:23.910546+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.910546+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":316,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.570127+0000","last_change":"2024-02-23T00:45:16.664785+0000","last_active":"2024-02-23T00:47:06.570127+0000","last_peered":"2024-02-23T00:47:06.570127+0000","last_clean":"2024-02-23T00:47:06.570127+0000","last_became_active":"2024-02-23T00:39:14.779193+0000","last_became_peered":"2024-02-23T00:39:14.779193+0000","last_unstale":"2024-02-23T00:47:06.570127+0000","last_undegraded":"2024-02-23T00:47:06.570127+0000","last_fullsized":"2024-02-23T00:47:06.570127+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.664682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.664682+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":312,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.573033+0000","last_change":"2024-02-23T00:45:03.228087+0000","last_active":"2024-02-23T00:47:06.573033+0000","last_peered":"2024-02-23T00:47:06.573033+0000","last_clean":"2024-02-23T00:47:06.573033+0000","last_became_active":"2024-02-23T00:39:20.795271+0000","last_became_peered":"2024-02-23T00:39:20.795271+0000","last_unstale":"2024-02-23T00:47:06.573033+0000","last_undegraded":"2024-02-23T00:47:06.573033+0000","last_fullsized":"2024-02-23T00:47:06.573033+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:03.227995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:45:03.227995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":260,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.163440+0000","last_change":"2024-02-23T00:45:13.874437+0000","last_active":"2024-02-23T00:47:06.163440+0000","last_peered":"2024-02-23T00:47:06.163440+0000","last_clean":"2024-02-23T00:47:06.163440+0000","last_became_active":"2024-02-23T00:42:58.809283+0000","last_became_peered":"2024-02-23T00:42:58.809283+0000","last_unstale":"2024-02-23T00:47:06.163440+0000","last_undegraded":"2024-02-23T00:47:06.163440+0000","last_fullsized":"2024-02-23T00:47:06.163440+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:13.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:13.874373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":316,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.419871+0000","last_change":"2024-02-23T00:46:45.294769+0000","last_active":"2024-02-23T00:47:05.419871+0000","last_peered":"2024-02-23T00:47:05.419871+0000","last_clean":"2024-02-23T00:47:05.419871+0000","last_became_active":"2024-02-23T00:39:18.796664+0000","last_became_peered":"2024-02-23T00:39:18.796664+0000","last_unstale":"2024-02-23T00:47:05.419871+0000","last_undegraded":"2024-02-23T00:47:05.419871+0000","last_fullsized":"2024-02-23T00:47:05.419871+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:45.294704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:45.294704+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"203'34","reported_seq":375,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.165050+0000","last_change":"2024-02-23T00:44:35.644033+0000","last_active":"2024-02-23T00:47:05.165050+0000","last_peered":"2024-02-23T00:47:05.165050+0000","last_clean":"2024-02-23T00:47:05.165050+0000","last_became_active":"2024-02-23T00:39:16.857278+0000","last_became_peered":"2024-02-23T00:39:16.857278+0000","last_unstale":"2024-02-23T00:47:05.165050+0000","last_undegraded":"2024-02-23T00:47:05.165050+0000","last_fullsized":"2024-02-23T00:47:05.165050+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'34","last_scrub_stamp":"2024-02-23T00:44:35.643931+0000","last_deep_scrub":"203'34","last_deep_scrub_stamp":"2024-02-23T00:44:35.643931+0000","last_clean_scrub_stamp":"2024-02-23T00:44:35.643931+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":315,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.486305+0000","last_change":"2024-02-23T00:45:12.730663+0000","last_active":"2024-02-23T00:47:05.486305+0000","last_peered":"2024-02-23T00:47:05.486305+0000","last_clean":"2024-02-23T00:47:05.486305+0000","last_became_active":"2024-02-23T00:39:14.783662+0000","last_became_peered":"2024-02-23T00:39:14.783662+0000","last_unstale":"2024-02-23T00:47:05.486305+0000","last_undegraded":"2024-02-23T00:47:05.486305+0000","last_fullsized":"2024-02-23T00:47:05.486305+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:12.730539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:12.730539+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":310,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.570940+0000","last_change":"2024-02-23T00:45:04.585266+0000","last_active":"2024-02-23T00:47:06.570940+0000","last_peered":"2024-02-23T00:47:06.570940+0000","last_clean":"2024-02-23T00:47:06.570940+0000","last_became_active":"2024-02-23T00:39:21.160060+0000","last_became_peered":"2024-02-23T00:39:21.160060+0000","last_unstale":"2024-02-23T00:47:06.570940+0000","last_undegraded":"2024-02-23T00:47:06.570940+0000","last_fullsized":"2024-02-23T00:47:06.570940+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:04.585179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.585179+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":258,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.418038+0000","last_change":"2024-02-23T00:45:18.185097+0000","last_active":"2024-02-23T00:47:05.418038+0000","last_peered":"2024-02-23T00:47:05.418038+0000","last_clean":"2024-02-23T00:47:05.418038+0000","last_became_active":"2024-02-23T00:42:58.726652+0000","last_became_peered":"2024-02-23T00:42:58.726652+0000","last_unstale":"2024-02-23T00:47:05.418038+0000","last_undegraded":"2024-02-23T00:47:05.418038+0000","last_fullsized":"2024-02-23T00:47:05.418038+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:18.184992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:18.184992+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":318,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.419461+0000","last_change":"2024-02-23T00:46:37.333068+0000","last_active":"2024-02-23T00:47:05.419461+0000","last_peered":"2024-02-23T00:47:05.419461+0000","last_clean":"2024-02-23T00:47:05.419461+0000","last_became_active":"2024-02-23T00:39:18.800378+0000","last_became_peered":"2024-02-23T00:39:18.800378+0000","last_unstale":"2024-02-23T00:47:05.419461+0000","last_undegraded":"2024-02-23T00:47:05.419461+0000","last_fullsized":"2024-02-23T00:47:05.419461+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.332973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:59.694033+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.332973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"95'18","reported_seq":339,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.162174+0000","last_change":"2024-02-23T00:44:18.630435+0000","last_active":"2024-02-23T00:47:05.162174+0000","last_peered":"2024-02-23T00:47:05.162174+0000","last_clean":"2024-02-23T00:47:05.162174+0000","last_became_active":"2024-02-23T00:39:16.787143+0000","last_became_peered":"2024-02-23T00:39:16.787143+0000","last_unstale":"2024-02-23T00:47:05.162174+0000","last_undegraded":"2024-02-23T00:47:05.162174+0000","last_fullsized":"2024-02-23T00:47:05.162174+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'18","last_scrub_stamp":"2024-02-23T00:44:18.630363+0000","last_deep_scrub":"95'18","last_deep_scrub_stamp":"2024-02-23T00:42:01.195545+0000","last_clean_scrub_stamp":"2024-02-23T00:44:18.630363+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":320,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.485067+0000","last_change":"2024-02-23T00:45:52.795624+0000","last_active":"2024-02-23T00:47:05.485067+0000","last_peered":"2024-02-23T00:47:05.485067+0000","last_clean":"2024-02-23T00:47:05.485067+0000","last_became_active":"2024-02-23T00:39:14.784578+0000","last_became_peered":"2024-02-23T00:39:14.784578+0000","last_unstale":"2024-02-23T00:47:05.485067+0000","last_undegraded":"2024-02-23T00:47:05.485067+0000","last_fullsized":"2024-02-23T00:47:05.485067+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:52.795527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:52.795527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":322,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.166886+0000","last_change":"2024-02-23T00:46:44.613082+0000","last_active":"2024-02-23T00:47:05.166886+0000","last_peered":"2024-02-23T00:47:05.166886+0000","last_clean":"2024-02-23T00:47:05.166886+0000","last_became_active":"2024-02-23T00:39:14.775446+0000","last_became_peered":"2024-02-23T00:39:14.775446+0000","last_unstale":"2024-02-23T00:47:05.166886+0000","last_undegraded":"2024-02-23T00:47:05.166886+0000","last_fullsized":"2024-02-23T00:47:05.166886+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:44.613012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:44.613012+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":305,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.571147+0000","last_change":"2024-02-23T00:43:47.905617+0000","last_active":"2024-02-23T00:47:06.571147+0000","last_peered":"2024-02-23T00:47:06.571147+0000","last_clean":"2024-02-23T00:47:06.571147+0000","last_became_active":"2024-02-23T00:39:20.797242+0000","last_became_peered":"2024-02-23T00:39:20.797242+0000","last_unstale":"2024-02-23T00:47:06.571147+0000","last_undegraded":"2024-02-23T00:47:06.571147+0000","last_fullsized":"2024-02-23T00:47:06.571147+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:47.905501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:43:47.905501+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":262,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.416368+0000","last_change":"2024-02-23T00:46:50.385176+0000","last_active":"2024-02-23T00:47:05.416368+0000","last_peered":"2024-02-23T00:47:05.416368+0000","last_clean":"2024-02-23T00:47:05.416368+0000","last_became_active":"2024-02-23T00:42:58.690858+0000","last_became_peered":"2024-02-23T00:42:58.690858+0000","last_unstale":"2024-02-23T00:47:05.416368+0000","last_undegraded":"2024-02-23T00:47:05.416368+0000","last_fullsized":"2024-02-23T00:47:05.416368+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:50.385111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:35.360054+0000","last_clean_scrub_stamp":"2024-02-23T00:46:50.385111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":316,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.485318+0000","last_change":"2024-02-23T00:46:49.705056+0000","last_active":"2024-02-23T00:47:05.485318+0000","last_peered":"2024-02-23T00:47:05.485318+0000","last_clean":"2024-02-23T00:47:05.485318+0000","last_became_active":"2024-02-23T00:39:18.793801+0000","last_became_peered":"2024-02-23T00:39:18.793801+0000","last_unstale":"2024-02-23T00:47:05.485318+0000","last_undegraded":"2024-02-23T00:47:05.485318+0000","last_fullsized":"2024-02-23T00:47:05.485318+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:49.704984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:49.704984+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"67'14","reported_seq":340,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.418410+0000","last_change":"2024-02-23T00:46:11.980163+0000","last_active":"2024-02-23T00:47:05.418410+0000","last_peered":"2024-02-23T00:47:05.418410+0000","last_clean":"2024-02-23T00:47:05.418410+0000","last_became_active":"2024-02-23T00:39:16.858880+0000","last_became_peered":"2024-02-23T00:39:16.858880+0000","last_unstale":"2024-02-23T00:47:05.418410+0000","last_undegraded":"2024-02-23T00:47:05.418410+0000","last_fullsized":"2024-02-23T00:47:05.418410+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'14","last_scrub_stamp":"2024-02-23T00:46:11.980087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:11.980087+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":325,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.484731+0000","last_change":"2024-02-23T00:45:08.796153+0000","last_active":"2024-02-23T00:47:05.484731+0000","last_peered":"2024-02-23T00:47:05.484731+0000","last_clean":"2024-02-23T00:47:05.484731+0000","last_became_active":"2024-02-23T00:39:08.248249+0000","last_became_peered":"2024-02-23T00:39:08.248249+0000","last_unstale":"2024-02-23T00:47:05.484731+0000","last_undegraded":"2024-02-23T00:47:05.484731+0000","last_fullsized":"2024-02-23T00:47:05.484731+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:08.796076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:49.977248+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.796076+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":317,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.417213+0000","last_change":"2024-02-23T00:45:22.272078+0000","last_active":"2024-02-23T00:47:05.417213+0000","last_peered":"2024-02-23T00:47:05.417213+0000","last_clean":"2024-02-23T00:47:05.417213+0000","last_became_active":"2024-02-23T00:39:14.788888+0000","last_became_peered":"2024-02-23T00:39:14.788888+0000","last_unstale":"2024-02-23T00:47:05.417213+0000","last_undegraded":"2024-02-23T00:47:05.417213+0000","last_fullsized":"2024-02-23T00:47:05.417213+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:22.272003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:22.272003+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":319,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.161633+0000","last_change":"2024-02-23T00:44:33.143663+0000","last_active":"2024-02-23T00:47:06.161633+0000","last_peered":"2024-02-23T00:47:06.161633+0000","last_clean":"2024-02-23T00:47:06.161633+0000","last_became_active":"2024-02-23T00:39:14.783489+0000","last_became_peered":"2024-02-23T00:39:14.783489+0000","last_unstale":"2024-02-23T00:47:06.161633+0000","last_undegraded":"2024-02-23T00:47:06.161633+0000","last_fullsized":"2024-02-23T00:47:06.161633+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:33.143592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:33.143592+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":334,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.419506+0000","last_change":"2024-02-23T00:46:45.207341+0000","last_active":"2024-02-23T00:47:05.419506+0000","last_peered":"2024-02-23T00:47:05.419506+0000","last_clean":"2024-02-23T00:47:05.419506+0000","last_became_active":"2024-02-23T00:43:42.168435+0000","last_became_peered":"2024-02-23T00:43:42.168435+0000","last_unstale":"2024-02-23T00:47:05.419506+0000","last_undegraded":"2024-02-23T00:47:05.419506+0000","last_fullsized":"2024-02-23T00:47:05.419506+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:45.207275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:46:45.207275+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":258,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.485193+0000","last_change":"2024-02-23T00:45:17.763618+0000","last_active":"2024-02-23T00:47:05.485193+0000","last_peered":"2024-02-23T00:47:05.485193+0000","last_clean":"2024-02-23T00:47:05.485193+0000","last_became_active":"2024-02-23T00:42:58.701502+0000","last_became_peered":"2024-02-23T00:42:58.701502+0000","last_unstale":"2024-02-23T00:47:05.485193+0000","last_undegraded":"2024-02-23T00:47:05.485193+0000","last_fullsized":"2024-02-23T00:47:05.485193+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:17.763482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:17.763482+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":317,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.666862+0000","last_change":"2024-02-23T00:46:09.924457+0000","last_active":"2024-02-23T00:47:05.666862+0000","last_peered":"2024-02-23T00:47:05.666862+0000","last_clean":"2024-02-23T00:47:05.666862+0000","last_became_active":"2024-02-23T00:39:18.789330+0000","last_became_peered":"2024-02-23T00:39:18.789330+0000","last_unstale":"2024-02-23T00:47:05.666862+0000","last_undegraded":"2024-02-23T00:47:05.666862+0000","last_fullsized":"2024-02-23T00:47:05.666862+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:09.924384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.264240+0000","last_clean_scrub_stamp":"2024-02-23T00:46:09.924384+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"68'19","reported_seq":356,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.571225+0000","last_change":"2024-02-23T00:45:07.697475+0000","last_active":"2024-02-23T00:47:06.571225+0000","last_peered":"2024-02-23T00:47:06.571225+0000","last_clean":"2024-02-23T00:47:06.571225+0000","last_became_active":"2024-02-23T00:39:16.787484+0000","last_became_peered":"2024-02-23T00:39:16.787484+0000","last_unstale":"2024-02-23T00:47:06.571225+0000","last_undegraded":"2024-02-23T00:47:06.571225+0000","last_fullsized":"2024-02-23T00:47:06.571225+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'19","last_scrub_stamp":"2024-02-23T00:45:07.697394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:07.697394+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":321,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.664029+0000","last_change":"2024-02-23T00:46:10.878272+0000","last_active":"2024-02-23T00:47:05.664029+0000","last_peered":"2024-02-23T00:47:05.664029+0000","last_clean":"2024-02-23T00:47:05.664029+0000","last_became_active":"2024-02-23T00:39:14.772411+0000","last_became_peered":"2024-02-23T00:39:14.772411+0000","last_unstale":"2024-02-23T00:47:05.664029+0000","last_undegraded":"2024-02-23T00:47:05.664029+0000","last_fullsized":"2024-02-23T00:47:05.664029+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:10.878204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:10.878204+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"68'5","reported_seq":778,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.811974+0000","last_change":"2024-02-23T00:43:48.876428+0000","last_active":"2024-02-23T00:47:06.811974+0000","last_peered":"2024-02-23T00:47:06.811974+0000","last_clean":"2024-02-23T00:47:06.811974+0000","last_became_active":"2024-02-23T00:39:14.779109+0000","last_became_peered":"2024-02-23T00:39:14.779109+0000","last_unstale":"2024-02-23T00:47:06.811974+0000","last_undegraded":"2024-02-23T00:47:06.811974+0000","last_fullsized":"2024-02-23T00:47:06.811974+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'5","last_scrub_stamp":"2024-02-23T00:43:48.876356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.876356+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":464,"num_read_kb":459,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":314,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.483559+0000","last_change":"2024-02-23T00:46:36.725799+0000","last_active":"2024-02-23T00:47:05.483559+0000","last_peered":"2024-02-23T00:47:05.483559+0000","last_clean":"2024-02-23T00:47:05.483559+0000","last_became_active":"2024-02-23T00:39:21.160457+0000","last_became_peered":"2024-02-23T00:39:21.160457+0000","last_unstale":"2024-02-23T00:47:05.483559+0000","last_undegraded":"2024-02-23T00:47:05.483559+0000","last_fullsized":"2024-02-23T00:47:05.483559+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:36.725686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:46:36.725686+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":258,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.570563+0000","last_change":"2024-02-23T00:46:42.331610+0000","last_active":"2024-02-23T00:47:06.570563+0000","last_peered":"2024-02-23T00:47:06.570563+0000","last_clean":"2024-02-23T00:47:06.570563+0000","last_became_active":"2024-02-23T00:42:57.653355+0000","last_became_peered":"2024-02-23T00:42:57.653355+0000","last_unstale":"2024-02-23T00:47:06.570563+0000","last_undegraded":"2024-02-23T00:47:06.570563+0000","last_fullsized":"2024-02-23T00:47:06.570563+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:42.331547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:46:42.331547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":313,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.163590+0000","last_change":"2024-02-23T00:44:20.626116+0000","last_active":"2024-02-23T00:47:05.163590+0000","last_peered":"2024-02-23T00:47:05.163590+0000","last_clean":"2024-02-23T00:47:05.163590+0000","last_became_active":"2024-02-23T00:39:18.802134+0000","last_became_peered":"2024-02-23T00:39:18.802134+0000","last_unstale":"2024-02-23T00:47:05.163590+0000","last_undegraded":"2024-02-23T00:47:05.163590+0000","last_fullsized":"2024-02-23T00:47:05.163590+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:20.626052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:44:20.626052+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"86'35","reported_seq":393,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.417046+0000","last_change":"2024-02-23T00:45:52.939553+0000","last_active":"2024-02-23T00:47:05.417046+0000","last_peered":"2024-02-23T00:47:05.417046+0000","last_clean":"2024-02-23T00:47:05.417046+0000","last_became_active":"2024-02-23T00:40:42.933322+0000","last_became_peered":"2024-02-23T00:40:42.933322+0000","last_unstale":"2024-02-23T00:47:05.417046+0000","last_undegraded":"2024-02-23T00:47:05.417046+0000","last_fullsized":"2024-02-23T00:47:05.417046+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'35","last_scrub_stamp":"2024-02-23T00:45:52.939452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:52.939452+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":317,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.417300+0000","last_change":"2024-02-23T00:44:23.965228+0000","last_active":"2024-02-23T00:47:05.417300+0000","last_peered":"2024-02-23T00:47:05.417300+0000","last_clean":"2024-02-23T00:47:05.417300+0000","last_became_active":"2024-02-23T00:39:14.784015+0000","last_became_peered":"2024-02-23T00:39:14.784015+0000","last_unstale":"2024-02-23T00:47:05.417300+0000","last_undegraded":"2024-02-23T00:47:05.417300+0000","last_fullsized":"2024-02-23T00:47:05.417300+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:23.965125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.965125+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":331,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.163325+0000","last_change":"2024-02-23T00:46:42.938278+0000","last_active":"2024-02-23T00:47:06.163325+0000","last_peered":"2024-02-23T00:47:06.163325+0000","last_clean":"2024-02-23T00:47:06.163325+0000","last_became_active":"2024-02-23T00:43:45.144043+0000","last_became_peered":"2024-02-23T00:43:45.144043+0000","last_unstale":"2024-02-23T00:47:06.163325+0000","last_undegraded":"2024-02-23T00:47:06.163325+0000","last_fullsized":"2024-02-23T00:47:06.163325+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:42.938209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:46:42.938209+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":258,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.664166+0000","last_change":"2024-02-23T00:45:05.490595+0000","last_active":"2024-02-23T00:47:05.664166+0000","last_peered":"2024-02-23T00:47:05.664166+0000","last_clean":"2024-02-23T00:47:05.664166+0000","last_became_active":"2024-02-23T00:42:58.766579+0000","last_became_peered":"2024-02-23T00:42:58.766579+0000","last_unstale":"2024-02-23T00:47:05.664166+0000","last_undegraded":"2024-02-23T00:47:05.664166+0000","last_fullsized":"2024-02-23T00:47:05.664166+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:05.490531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:05.490531+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"91'9","reported_seq":416,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.167250+0000","last_change":"2024-02-23T00:44:28.619036+0000","last_active":"2024-02-23T00:47:05.167250+0000","last_peered":"2024-02-23T00:47:05.167250+0000","last_clean":"2024-02-23T00:47:05.167250+0000","last_became_active":"2024-02-23T00:39:18.801297+0000","last_became_peered":"2024-02-23T00:39:18.801297+0000","last_unstale":"2024-02-23T00:47:05.167250+0000","last_undegraded":"2024-02-23T00:47:05.167250+0000","last_fullsized":"2024-02-23T00:47:05.167250+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'9","last_scrub_stamp":"2024-02-23T00:44:28.618923+0000","last_deep_scrub":"91'9","last_deep_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_clean_scrub_stamp":"2024-02-23T00:44:28.618923+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"68'16","reported_seq":344,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.571319+0000","last_change":"2024-02-23T00:45:11.713800+0000","last_active":"2024-02-23T00:47:06.571319+0000","last_peered":"2024-02-23T00:47:06.571319+0000","last_clean":"2024-02-23T00:47:06.571319+0000","last_became_active":"2024-02-23T00:39:16.787468+0000","last_became_peered":"2024-02-23T00:39:16.787468+0000","last_unstale":"2024-02-23T00:47:06.571319+0000","last_undegraded":"2024-02-23T00:47:06.571319+0000","last_fullsized":"2024-02-23T00:47:06.571319+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:45:11.713731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:11.713731+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":316,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.570753+0000","last_change":"2024-02-23T00:45:19.635062+0000","last_active":"2024-02-23T00:47:06.570753+0000","last_peered":"2024-02-23T00:47:06.570753+0000","last_clean":"2024-02-23T00:47:06.570753+0000","last_became_active":"2024-02-23T00:39:14.779952+0000","last_became_peered":"2024-02-23T00:39:14.779952+0000","last_unstale":"2024-02-23T00:47:06.570753+0000","last_undegraded":"2024-02-23T00:47:06.570753+0000","last_fullsized":"2024-02-23T00:47:06.570753+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:19.635002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:19.635002+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"107'2","reported_seq":313,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.161431+0000","last_change":"2024-02-23T00:45:08.662754+0000","last_active":"2024-02-23T00:47:05.161431+0000","last_peered":"2024-02-23T00:47:05.161431+0000","last_clean":"2024-02-23T00:47:05.161431+0000","last_became_active":"2024-02-23T00:39:20.800858+0000","last_became_peered":"2024-02-23T00:39:20.800858+0000","last_unstale":"2024-02-23T00:47:05.161431+0000","last_undegraded":"2024-02-23T00:47:05.161431+0000","last_fullsized":"2024-02-23T00:47:05.161431+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'2","last_scrub_stamp":"2024-02-23T00:45:08.662667+0000","last_deep_scrub":"107'2","last_deep_scrub_stamp":"2024-02-23T00:42:48.362739+0000","last_clean_scrub_stamp":"2024-02-23T00:45:08.662667+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":257,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.418022+0000","last_change":"2024-02-23T00:45:04.621794+0000","last_active":"2024-02-23T00:47:05.418022+0000","last_peered":"2024-02-23T00:47:05.418022+0000","last_clean":"2024-02-23T00:47:05.418022+0000","last_became_active":"2024-02-23T00:42:58.664282+0000","last_became_peered":"2024-02-23T00:42:58.664282+0000","last_unstale":"2024-02-23T00:47:05.418022+0000","last_undegraded":"2024-02-23T00:47:05.418022+0000","last_fullsized":"2024-02-23T00:47:05.418022+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:04.621714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.621714+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":317,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.664231+0000","last_change":"2024-02-23T00:46:36.783887+0000","last_active":"2024-02-23T00:47:05.664231+0000","last_peered":"2024-02-23T00:47:05.664231+0000","last_clean":"2024-02-23T00:47:05.664231+0000","last_became_active":"2024-02-23T00:39:18.796323+0000","last_became_peered":"2024-02-23T00:39:18.796323+0000","last_unstale":"2024-02-23T00:47:05.664231+0000","last_undegraded":"2024-02-23T00:47:05.664231+0000","last_fullsized":"2024-02-23T00:47:05.664231+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:36.783823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_clean_scrub_stamp":"2024-02-23T00:46:36.783823+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"68'25","reported_seq":370,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.572994+0000","last_change":"2024-02-23T00:46:51.333974+0000","last_active":"2024-02-23T00:47:06.572994+0000","last_peered":"2024-02-23T00:47:06.572994+0000","last_clean":"2024-02-23T00:47:06.572994+0000","last_became_active":"2024-02-23T00:39:16.783926+0000","last_became_peered":"2024-02-23T00:39:16.783926+0000","last_unstale":"2024-02-23T00:47:06.572994+0000","last_undegraded":"2024-02-23T00:47:06.572994+0000","last_fullsized":"2024-02-23T00:47:06.572994+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'25","last_scrub_stamp":"2024-02-23T00:46:51.333896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:51.333896+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"95'4","reported_seq":333,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.571242+0000","last_change":"2024-02-23T00:44:31.106105+0000","last_active":"2024-02-23T00:47:06.571242+0000","last_peered":"2024-02-23T00:47:06.571242+0000","last_clean":"2024-02-23T00:47:06.571242+0000","last_became_active":"2024-02-23T00:39:14.779205+0000","last_became_peered":"2024-02-23T00:39:14.779205+0000","last_unstale":"2024-02-23T00:47:06.571242+0000","last_undegraded":"2024-02-23T00:47:06.571242+0000","last_fullsized":"2024-02-23T00:47:06.571242+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'4","last_scrub_stamp":"2024-02-23T00:44:31.106012+0000","last_deep_scrub":"95'4","last_deep_scrub_stamp":"2024-02-23T00:41:43.874397+0000","last_clean_scrub_stamp":"2024-02-23T00:44:31.106012+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":336,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.162193+0000","last_change":"2024-02-23T00:46:47.919416+0000","last_active":"2024-02-23T00:47:06.162193+0000","last_peered":"2024-02-23T00:47:06.162193+0000","last_clean":"2024-02-23T00:47:06.162193+0000","last_became_active":"2024-02-23T00:43:27.611660+0000","last_became_peered":"2024-02-23T00:43:27.611660+0000","last_unstale":"2024-02-23T00:47:06.162193+0000","last_undegraded":"2024-02-23T00:47:06.162193+0000","last_fullsized":"2024-02-23T00:47:06.162193+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:47.919353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:46:47.919353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":258,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.486659+0000","last_change":"2024-02-23T00:45:20.829124+0000","last_active":"2024-02-23T00:47:05.486659+0000","last_peered":"2024-02-23T00:47:05.486659+0000","last_clean":"2024-02-23T00:47:05.486659+0000","last_became_active":"2024-02-23T00:42:58.744983+0000","last_became_peered":"2024-02-23T00:42:58.744983+0000","last_unstale":"2024-02-23T00:47:05.486659+0000","last_undegraded":"2024-02-23T00:47:05.486659+0000","last_fullsized":"2024-02-23T00:47:05.486659+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:20.829052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:20.829052+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"68'21","reported_seq":361,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.162272+0000","last_change":"2024-02-23T00:46:46.645234+0000","last_active":"2024-02-23T00:47:05.162272+0000","last_peered":"2024-02-23T00:47:05.162272+0000","last_clean":"2024-02-23T00:47:05.162272+0000","last_became_active":"2024-02-23T00:39:16.858760+0000","last_became_peered":"2024-02-23T00:39:16.858760+0000","last_unstale":"2024-02-23T00:47:05.162272+0000","last_undegraded":"2024-02-23T00:47:05.162272+0000","last_fullsized":"2024-02-23T00:47:05.162272+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'21","last_scrub_stamp":"2024-02-23T00:46:46.645153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:46.645153+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":321,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.664232+0000","last_change":"2024-02-23T00:45:22.376128+0000","last_active":"2024-02-23T00:47:05.664232+0000","last_peered":"2024-02-23T00:47:05.664232+0000","last_clean":"2024-02-23T00:47:05.664232+0000","last_became_active":"2024-02-23T00:39:14.774244+0000","last_became_peered":"2024-02-23T00:39:14.774244+0000","last_unstale":"2024-02-23T00:47:05.664232+0000","last_undegraded":"2024-02-23T00:47:05.664232+0000","last_fullsized":"2024-02-23T00:47:05.664232+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:22.376063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:22.376063+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":320,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.163196+0000","last_change":"2024-02-23T00:46:45.913635+0000","last_active":"2024-02-23T00:47:06.163196+0000","last_peered":"2024-02-23T00:47:06.163196+0000","last_clean":"2024-02-23T00:47:06.163196+0000","last_became_active":"2024-02-23T00:39:18.798624+0000","last_became_peered":"2024-02-23T00:39:18.798624+0000","last_unstale":"2024-02-23T00:47:06.163196+0000","last_undegraded":"2024-02-23T00:47:06.163196+0000","last_fullsized":"2024-02-23T00:47:06.163196+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:45.913526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:45.913526+0000","last_clean_scrub_stamp":"2024-02-23T00:46:45.913526+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":315,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.665939+0000","last_change":"2024-02-23T00:46:51.604219+0000","last_active":"2024-02-23T00:47:05.665939+0000","last_peered":"2024-02-23T00:47:05.665939+0000","last_clean":"2024-02-23T00:47:05.665939+0000","last_became_active":"2024-02-23T00:39:20.796041+0000","last_became_peered":"2024-02-23T00:39:20.796041+0000","last_unstale":"2024-02-23T00:47:05.665939+0000","last_undegraded":"2024-02-23T00:47:05.665939+0000","last_fullsized":"2024-02-23T00:47:05.665939+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:51.604144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:46:51.604144+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":258,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.161553+0000","last_change":"2024-02-23T00:44:26.595443+0000","last_active":"2024-02-23T00:47:05.161553+0000","last_peered":"2024-02-23T00:47:05.161553+0000","last_clean":"2024-02-23T00:47:05.161553+0000","last_became_active":"2024-02-23T00:42:58.825015+0000","last_became_peered":"2024-02-23T00:42:58.825015+0000","last_unstale":"2024-02-23T00:47:05.161553+0000","last_undegraded":"2024-02-23T00:47:05.161553+0000","last_fullsized":"2024-02-23T00:47:05.161553+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:26.595346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:44:26.595346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"107'16","reported_seq":345,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.163107+0000","last_change":"2024-02-23T00:45:07.012086+0000","last_active":"2024-02-23T00:47:06.163107+0000","last_peered":"2024-02-23T00:47:06.163107+0000","last_clean":"2024-02-23T00:47:06.163107+0000","last_became_active":"2024-02-23T00:39:16.858449+0000","last_became_peered":"2024-02-23T00:39:16.858449+0000","last_unstale":"2024-02-23T00:47:06.163107+0000","last_undegraded":"2024-02-23T00:47:06.163107+0000","last_fullsized":"2024-02-23T00:47:06.163107+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'16","last_scrub_stamp":"2024-02-23T00:45:07.011956+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-02-23T00:42:52.964510+0000","last_clean_scrub_stamp":"2024-02-23T00:45:07.011956+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":315,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.419997+0000","last_change":"2024-02-23T00:45:13.613736+0000","last_active":"2024-02-23T00:47:05.419997+0000","last_peered":"2024-02-23T00:47:05.419997+0000","last_clean":"2024-02-23T00:47:05.419997+0000","last_became_active":"2024-02-23T00:39:14.783164+0000","last_became_peered":"2024-02-23T00:39:14.783164+0000","last_unstale":"2024-02-23T00:47:05.419997+0000","last_undegraded":"2024-02-23T00:47:05.419997+0000","last_fullsized":"2024-02-23T00:47:05.419997+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:13.613654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:13.613654+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":318,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.417763+0000","last_change":"2024-02-23T00:46:42.258031+0000","last_active":"2024-02-23T00:47:05.417763+0000","last_peered":"2024-02-23T00:47:05.417763+0000","last_clean":"2024-02-23T00:47:05.417763+0000","last_became_active":"2024-02-23T00:39:18.800297+0000","last_became_peered":"2024-02-23T00:39:18.800297+0000","last_unstale":"2024-02-23T00:47:05.417763+0000","last_undegraded":"2024-02-23T00:47:05.417763+0000","last_fullsized":"2024-02-23T00:47:05.417763+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:42.257958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:42.257958+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":314,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.483585+0000","last_change":"2024-02-23T00:46:34.721645+0000","last_active":"2024-02-23T00:47:05.483585+0000","last_peered":"2024-02-23T00:47:05.483585+0000","last_clean":"2024-02-23T00:47:05.483585+0000","last_became_active":"2024-02-23T00:39:21.162036+0000","last_became_peered":"2024-02-23T00:39:21.162036+0000","last_unstale":"2024-02-23T00:47:05.483585+0000","last_undegraded":"2024-02-23T00:47:05.483585+0000","last_fullsized":"2024-02-23T00:47:05.483585+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:34.721564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:46:34.721564+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"110'7","reported_seq":273,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.161823+0000","last_change":"2024-02-23T00:45:18.947859+0000","last_active":"2024-02-23T00:47:06.161823+0000","last_peered":"2024-02-23T00:47:06.161823+0000","last_clean":"2024-02-23T00:47:06.161823+0000","last_became_active":"2024-02-23T00:42:58.777251+0000","last_became_peered":"2024-02-23T00:42:58.777251+0000","last_unstale":"2024-02-23T00:47:06.161823+0000","last_undegraded":"2024-02-23T00:47:06.161823+0000","last_fullsized":"2024-02-23T00:47:06.161823+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'7","last_scrub_stamp":"2024-02-23T00:45:18.947746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:18.947746+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"68'22","reported_seq":284,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.665851+0000","last_change":"2024-02-23T00:44:31.391929+0000","last_active":"2024-02-23T00:47:05.665851+0000","last_peered":"2024-02-23T00:47:05.665851+0000","last_clean":"2024-02-23T00:47:05.665851+0000","last_became_active":"2024-02-23T00:42:43.521189+0000","last_became_peered":"2024-02-23T00:42:43.521189+0000","last_unstale":"2024-02-23T00:47:05.665851+0000","last_undegraded":"2024-02-23T00:47:05.665851+0000","last_fullsized":"2024-02-23T00:47:05.665851+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'22","last_scrub_stamp":"2024-02-23T00:44:31.391819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:44:31.391819+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":37,"num_read_kb":30,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,7],"acting":[2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":322,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.416286+0000","last_change":"2024-02-23T00:46:46.201157+0000","last_active":"2024-02-23T00:47:05.416286+0000","last_peered":"2024-02-23T00:47:05.416286+0000","last_clean":"2024-02-23T00:47:05.416286+0000","last_became_active":"2024-02-23T00:39:14.785590+0000","last_became_peered":"2024-02-23T00:39:14.785590+0000","last_unstale":"2024-02-23T00:47:05.416286+0000","last_undegraded":"2024-02-23T00:47:05.416286+0000","last_fullsized":"2024-02-23T00:47:05.416286+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:46.201063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_clean_scrub_stamp":"2024-02-23T00:46:46.201063+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"336'9","reported_seq":421,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.418486+0000","last_change":"2024-02-23T00:46:49.161082+0000","last_active":"2024-02-23T00:47:05.418486+0000","last_peered":"2024-02-23T00:47:05.418486+0000","last_clean":"2024-02-23T00:47:05.418486+0000","last_became_active":"2024-02-23T00:39:18.803566+0000","last_became_peered":"2024-02-23T00:39:18.803566+0000","last_unstale":"2024-02-23T00:47:05.418486+0000","last_undegraded":"2024-02-23T00:47:05.418486+0000","last_fullsized":"2024-02-23T00:47:05.418486+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"336'9","last_scrub_stamp":"2024-02-23T00:46:49.160991+0000","last_deep_scrub":"336'9","last_deep_scrub_stamp":"2024-02-23T00:46:49.160991+0000","last_clean_scrub_stamp":"2024-02-23T00:46:49.160991+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":258,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.484270+0000","last_change":"2024-02-23T00:45:54.753371+0000","last_active":"2024-02-23T00:47:05.484270+0000","last_peered":"2024-02-23T00:47:05.484270+0000","last_clean":"2024-02-23T00:47:05.484270+0000","last_became_active":"2024-02-23T00:42:58.757424+0000","last_became_peered":"2024-02-23T00:42:58.757424+0000","last_unstale":"2024-02-23T00:47:05.484270+0000","last_undegraded":"2024-02-23T00:47:05.484270+0000","last_fullsized":"2024-02-23T00:47:05.484270+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:54.753267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:54.753267+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"68'15","reported_seq":346,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.418072+0000","last_change":"2024-02-23T00:44:25.998879+0000","last_active":"2024-02-23T00:47:05.418072+0000","last_peered":"2024-02-23T00:47:05.418072+0000","last_clean":"2024-02-23T00:47:05.418072+0000","last_became_active":"2024-02-23T00:39:16.858953+0000","last_became_peered":"2024-02-23T00:39:16.858953+0000","last_unstale":"2024-02-23T00:47:05.418072+0000","last_undegraded":"2024-02-23T00:47:05.418072+0000","last_fullsized":"2024-02-23T00:47:05.418072+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:44:25.998804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:44:25.998804+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":322,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.167889+0000","last_change":"2024-02-23T00:46:42.692530+0000","last_active":"2024-02-23T00:47:05.167889+0000","last_peered":"2024-02-23T00:47:05.167889+0000","last_clean":"2024-02-23T00:47:05.167889+0000","last_became_active":"2024-02-23T00:39:14.773772+0000","last_became_peered":"2024-02-23T00:39:14.773772+0000","last_unstale":"2024-02-23T00:47:05.167889+0000","last_undegraded":"2024-02-23T00:47:05.167889+0000","last_fullsized":"2024-02-23T00:47:05.167889+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:42.692434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:42.692434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"67'8","reported_seq":417,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.162454+0000","last_change":"2024-02-23T00:46:36.901142+0000","last_active":"2024-02-23T00:47:06.162454+0000","last_peered":"2024-02-23T00:47:06.162454+0000","last_clean":"2024-02-23T00:47:06.162454+0000","last_became_active":"2024-02-23T00:39:18.787313+0000","last_became_peered":"2024-02-23T00:39:18.787313+0000","last_unstale":"2024-02-23T00:47:06.162454+0000","last_undegraded":"2024-02-23T00:47:06.162454+0000","last_fullsized":"2024-02-23T00:47:06.162454+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:46:36.900988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:36.900988+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":258,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.571609+0000","last_change":"2024-02-23T00:46:51.254983+0000","last_active":"2024-02-23T00:47:06.571609+0000","last_peered":"2024-02-23T00:47:06.571609+0000","last_clean":"2024-02-23T00:47:06.571609+0000","last_became_active":"2024-02-23T00:42:57.652488+0000","last_became_peered":"2024-02-23T00:42:57.652488+0000","last_unstale":"2024-02-23T00:47:06.571609+0000","last_undegraded":"2024-02-23T00:47:06.571609+0000","last_fullsized":"2024-02-23T00:47:06.571609+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:51.254903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:51.254903+0000","last_clean_scrub_stamp":"2024-02-23T00:46:51.254903+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"67'9","reported_seq":330,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.571125+0000","last_change":"2024-02-23T00:46:36.406841+0000","last_active":"2024-02-23T00:47:06.571125+0000","last_peered":"2024-02-23T00:47:06.571125+0000","last_clean":"2024-02-23T00:47:06.571125+0000","last_became_active":"2024-02-23T00:39:16.786936+0000","last_became_peered":"2024-02-23T00:39:16.786936+0000","last_unstale":"2024-02-23T00:47:06.571125+0000","last_undegraded":"2024-02-23T00:47:06.571125+0000","last_fullsized":"2024-02-23T00:47:06.571125+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'9","last_scrub_stamp":"2024-02-23T00:46:36.406771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:36.406771+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":325,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.417108+0000","last_change":"2024-02-23T00:46:49.393060+0000","last_active":"2024-02-23T00:47:05.417108+0000","last_peered":"2024-02-23T00:47:05.417108+0000","last_clean":"2024-02-23T00:47:05.417108+0000","last_became_active":"2024-02-23T00:39:14.778455+0000","last_became_peered":"2024-02-23T00:39:14.778455+0000","last_unstale":"2024-02-23T00:47:05.417108+0000","last_undegraded":"2024-02-23T00:47:05.417108+0000","last_fullsized":"2024-02-23T00:47:05.417108+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:49.392993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:49.392993+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":315,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.162088+0000","last_change":"2024-02-23T00:44:37.105079+0000","last_active":"2024-02-23T00:47:06.162088+0000","last_peered":"2024-02-23T00:47:06.162088+0000","last_clean":"2024-02-23T00:47:06.162088+0000","last_became_active":"2024-02-23T00:39:18.799052+0000","last_became_peered":"2024-02-23T00:39:18.799052+0000","last_unstale":"2024-02-23T00:47:06.162088+0000","last_undegraded":"2024-02-23T00:47:06.162088+0000","last_fullsized":"2024-02-23T00:47:06.162088+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:37.105013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_clean_scrub_stamp":"2024-02-23T00:44:37.105013+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":311,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.161321+0000","last_change":"2024-02-23T00:46:00.558774+0000","last_active":"2024-02-23T00:47:05.161321+0000","last_peered":"2024-02-23T00:47:05.161321+0000","last_clean":"2024-02-23T00:47:05.161321+0000","last_became_active":"2024-02-23T00:39:20.802648+0000","last_became_peered":"2024-02-23T00:39:20.802648+0000","last_unstale":"2024-02-23T00:47:05.161321+0000","last_undegraded":"2024-02-23T00:47:05.161321+0000","last_fullsized":"2024-02-23T00:47:05.161321+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:00.558661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:46:00.558661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":258,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.162455+0000","last_change":"2024-02-23T00:44:23.543049+0000","last_active":"2024-02-23T00:47:05.162455+0000","last_peered":"2024-02-23T00:47:05.162455+0000","last_clean":"2024-02-23T00:47:05.162455+0000","last_became_active":"2024-02-23T00:42:58.835862+0000","last_became_peered":"2024-02-23T00:42:58.835862+0000","last_unstale":"2024-02-23T00:47:05.162455+0000","last_undegraded":"2024-02-23T00:47:05.162455+0000","last_fullsized":"2024-02-23T00:47:05.162455+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:23.542975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.542975+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"67'11","reported_seq":335,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.573523+0000","last_change":"2024-02-23T00:45:13.315510+0000","last_active":"2024-02-23T00:47:06.573523+0000","last_peered":"2024-02-23T00:47:06.573523+0000","last_clean":"2024-02-23T00:47:06.573523+0000","last_became_active":"2024-02-23T00:39:16.783896+0000","last_became_peered":"2024-02-23T00:39:16.783896+0000","last_unstale":"2024-02-23T00:47:06.573523+0000","last_undegraded":"2024-02-23T00:47:06.573523+0000","last_fullsized":"2024-02-23T00:47:06.573523+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'11","last_scrub_stamp":"2024-02-23T00:45:13.315434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:13.315434+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"202'2","reported_seq":324,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.570703+0000","last_change":"2024-02-23T00:44:34.720705+0000","last_active":"2024-02-23T00:47:06.570703+0000","last_peered":"2024-02-23T00:47:06.570703+0000","last_clean":"2024-02-23T00:47:06.570703+0000","last_became_active":"2024-02-23T00:39:14.775804+0000","last_became_peered":"2024-02-23T00:39:14.775804+0000","last_unstale":"2024-02-23T00:47:06.570703+0000","last_undegraded":"2024-02-23T00:47:06.570703+0000","last_fullsized":"2024-02-23T00:47:06.570703+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"202'2","last_scrub_stamp":"2024-02-23T00:44:34.720640+0000","last_deep_scrub":"202'2","last_deep_scrub_stamp":"2024-02-23T00:44:34.720640+0000","last_clean_scrub_stamp":"2024-02-23T00:44:34.720640+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":307,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.571216+0000","last_change":"2024-02-23T00:43:31.903157+0000","last_active":"2024-02-23T00:47:06.571216+0000","last_peered":"2024-02-23T00:47:06.571216+0000","last_clean":"2024-02-23T00:47:06.571216+0000","last_became_active":"2024-02-23T00:39:18.800382+0000","last_became_peered":"2024-02-23T00:39:18.800382+0000","last_unstale":"2024-02-23T00:47:06.571216+0000","last_undegraded":"2024-02-23T00:47:06.571216+0000","last_fullsized":"2024-02-23T00:47:06.571216+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:31.903031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:43:31.903031+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":313,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.161446+0000","last_change":"2024-02-23T00:44:23.152677+0000","last_active":"2024-02-23T00:47:06.161446+0000","last_peered":"2024-02-23T00:47:06.161446+0000","last_clean":"2024-02-23T00:47:06.161446+0000","last_became_active":"2024-02-23T00:39:20.799053+0000","last_became_peered":"2024-02-23T00:39:20.799053+0000","last_unstale":"2024-02-23T00:47:06.161446+0000","last_undegraded":"2024-02-23T00:47:06.161446+0000","last_fullsized":"2024-02-23T00:47:06.161446+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:23.152602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:44:23.152602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":258,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.570614+0000","last_change":"2024-02-23T00:46:49.244018+0000","last_active":"2024-02-23T00:47:06.570614+0000","last_peered":"2024-02-23T00:47:06.570614+0000","last_clean":"2024-02-23T00:47:06.570614+0000","last_became_active":"2024-02-23T00:42:57.652438+0000","last_became_peered":"2024-02-23T00:42:57.652438+0000","last_unstale":"2024-02-23T00:47:06.570614+0000","last_undegraded":"2024-02-23T00:47:06.570614+0000","last_fullsized":"2024-02-23T00:47:06.570614+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:49.243955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:49.243955+0000","last_clean_scrub_stamp":"2024-02-23T00:46:49.243955+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"67'4","reported_seq":323,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.161233+0000","last_change":"2024-02-23T00:45:20.967211+0000","last_active":"2024-02-23T00:47:06.161233+0000","last_peered":"2024-02-23T00:47:06.161233+0000","last_clean":"2024-02-23T00:47:06.161233+0000","last_became_active":"2024-02-23T00:39:16.785830+0000","last_became_peered":"2024-02-23T00:39:16.785830+0000","last_unstale":"2024-02-23T00:47:06.161233+0000","last_undegraded":"2024-02-23T00:47:06.161233+0000","last_fullsized":"2024-02-23T00:47:06.161233+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'4","last_scrub_stamp":"2024-02-23T00:45:20.967095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:20.967095+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":325,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.417628+0000","last_change":"2024-02-23T00:46:35.268544+0000","last_active":"2024-02-23T00:47:05.417628+0000","last_peered":"2024-02-23T00:47:05.417628+0000","last_clean":"2024-02-23T00:47:05.417628+0000","last_became_active":"2024-02-23T00:39:14.783325+0000","last_became_peered":"2024-02-23T00:39:14.783325+0000","last_unstale":"2024-02-23T00:47:05.417628+0000","last_undegraded":"2024-02-23T00:47:05.417628+0000","last_fullsized":"2024-02-23T00:47:05.417628+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:35.268491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:35.268491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":316,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.419560+0000","last_change":"2024-02-23T00:45:20.761547+0000","last_active":"2024-02-23T00:47:05.419560+0000","last_peered":"2024-02-23T00:47:05.419560+0000","last_clean":"2024-02-23T00:47:05.419560+0000","last_became_active":"2024-02-23T00:39:18.797287+0000","last_became_peered":"2024-02-23T00:39:18.797287+0000","last_unstale":"2024-02-23T00:47:05.419560+0000","last_undegraded":"2024-02-23T00:47:05.419560+0000","last_fullsized":"2024-02-23T00:47:05.419560+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:20.761440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:20.761440+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":309,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.484005+0000","last_change":"2024-02-23T00:45:10.741762+0000","last_active":"2024-02-23T00:47:05.484005+0000","last_peered":"2024-02-23T00:47:05.484005+0000","last_clean":"2024-02-23T00:47:05.484005+0000","last_became_active":"2024-02-23T00:39:20.795351+0000","last_became_peered":"2024-02-23T00:39:20.795351+0000","last_unstale":"2024-02-23T00:47:05.484005+0000","last_undegraded":"2024-02-23T00:47:05.484005+0000","last_fullsized":"2024-02-23T00:47:05.484005+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.741694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.741694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":258,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.486193+0000","last_change":"2024-02-23T00:45:56.706901+0000","last_active":"2024-02-23T00:47:05.486193+0000","last_peered":"2024-02-23T00:47:05.486193+0000","last_clean":"2024-02-23T00:47:05.486193+0000","last_became_active":"2024-02-23T00:42:58.741365+0000","last_became_peered":"2024-02-23T00:42:58.741365+0000","last_unstale":"2024-02-23T00:47:05.486193+0000","last_undegraded":"2024-02-23T00:47:05.486193+0000","last_fullsized":"2024-02-23T00:47:05.486193+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:56.706792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:56.706792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"68'17","reported_seq":356,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.163755+0000","last_change":"2024-02-23T00:46:49.921614+0000","last_active":"2024-02-23T00:47:06.163755+0000","last_peered":"2024-02-23T00:47:06.163755+0000","last_clean":"2024-02-23T00:47:06.163755+0000","last_became_active":"2024-02-23T00:39:16.857232+0000","last_became_peered":"2024-02-23T00:39:16.857232+0000","last_unstale":"2024-02-23T00:47:06.163755+0000","last_undegraded":"2024-02-23T00:47:06.163755+0000","last_fullsized":"2024-02-23T00:47:06.163755+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:46:49.921543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:49.921543+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":320,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.418979+0000","last_change":"2024-02-23T00:46:44.285195+0000","last_active":"2024-02-23T00:47:05.418979+0000","last_peered":"2024-02-23T00:47:05.418979+0000","last_clean":"2024-02-23T00:47:05.418979+0000","last_became_active":"2024-02-23T00:39:14.784092+0000","last_became_peered":"2024-02-23T00:39:14.784092+0000","last_unstale":"2024-02-23T00:47:05.418979+0000","last_undegraded":"2024-02-23T00:47:05.418979+0000","last_fullsized":"2024-02-23T00:47:05.418979+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:44.285083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:44.285083+0000","last_clean_scrub_stamp":"2024-02-23T00:46:44.285083+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":316,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.485105+0000","last_change":"2024-02-23T00:46:39.716023+0000","last_active":"2024-02-23T00:47:05.485105+0000","last_peered":"2024-02-23T00:47:05.485105+0000","last_clean":"2024-02-23T00:47:05.485105+0000","last_became_active":"2024-02-23T00:39:18.791457+0000","last_became_peered":"2024-02-23T00:39:18.791457+0000","last_unstale":"2024-02-23T00:47:05.485105+0000","last_undegraded":"2024-02-23T00:47:05.485105+0000","last_fullsized":"2024-02-23T00:47:05.485105+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:39.715957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:39.715957+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":313,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.162285+0000","last_change":"2024-02-23T00:46:43.945089+0000","last_active":"2024-02-23T00:47:06.162285+0000","last_peered":"2024-02-23T00:47:06.162285+0000","last_clean":"2024-02-23T00:47:06.162285+0000","last_became_active":"2024-02-23T00:39:21.160376+0000","last_became_peered":"2024-02-23T00:39:21.160376+0000","last_unstale":"2024-02-23T00:47:06.162285+0000","last_undegraded":"2024-02-23T00:47:06.162285+0000","last_fullsized":"2024-02-23T00:47:06.162285+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:43.945020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:46:43.945020+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":258,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.666561+0000","last_change":"2024-02-23T00:45:23.378334+0000","last_active":"2024-02-23T00:47:05.666561+0000","last_peered":"2024-02-23T00:47:05.666561+0000","last_clean":"2024-02-23T00:47:05.666561+0000","last_became_active":"2024-02-23T00:42:58.702580+0000","last_became_peered":"2024-02-23T00:42:58.702580+0000","last_unstale":"2024-02-23T00:47:05.666561+0000","last_undegraded":"2024-02-23T00:47:05.666561+0000","last_fullsized":"2024-02-23T00:47:05.666561+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:23.378270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.378270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"68'16","reported_seq":345,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.162696+0000","last_change":"2024-02-23T00:45:04.672612+0000","last_active":"2024-02-23T00:47:05.162696+0000","last_peered":"2024-02-23T00:47:05.162696+0000","last_clean":"2024-02-23T00:47:05.162696+0000","last_became_active":"2024-02-23T00:39:16.858945+0000","last_became_peered":"2024-02-23T00:39:16.858945+0000","last_unstale":"2024-02-23T00:47:05.162696+0000","last_undegraded":"2024-02-23T00:47:05.162696+0000","last_fullsized":"2024-02-23T00:47:05.162696+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:45:04.672537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:04.672537+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":329,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.416017+0000","last_change":"2024-02-23T00:46:09.088334+0000","last_active":"2024-02-23T00:47:05.416017+0000","last_peered":"2024-02-23T00:47:05.416017+0000","last_clean":"2024-02-23T00:47:05.416017+0000","last_became_active":"2024-02-23T00:39:14.785363+0000","last_became_peered":"2024-02-23T00:39:14.785363+0000","last_unstale":"2024-02-23T00:47:05.416017+0000","last_undegraded":"2024-02-23T00:47:05.416017+0000","last_fullsized":"2024-02-23T00:47:05.416017+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"80'2","last_scrub_stamp":"2024-02-23T00:46:09.088249+0000","last_deep_scrub":"80'2","last_deep_scrub_stamp":"2024-02-23T00:40:15.942725+0000","last_clean_scrub_stamp":"2024-02-23T00:46:09.088249+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"93'9","reported_seq":422,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.419414+0000","last_change":"2024-02-23T00:46:37.230102+0000","last_active":"2024-02-23T00:47:05.419414+0000","last_peered":"2024-02-23T00:47:05.419414+0000","last_clean":"2024-02-23T00:47:05.419414+0000","last_became_active":"2024-02-23T00:39:18.796680+0000","last_became_peered":"2024-02-23T00:39:18.796680+0000","last_unstale":"2024-02-23T00:47:05.419414+0000","last_undegraded":"2024-02-23T00:47:05.419414+0000","last_fullsized":"2024-02-23T00:47:05.419414+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-02-23T00:46:37.230030+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-02-23T00:43:50.393713+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.230030+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":258,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.161805+0000","last_change":"2024-02-23T00:45:58.613561+0000","last_active":"2024-02-23T00:47:05.161805+0000","last_peered":"2024-02-23T00:47:05.161805+0000","last_clean":"2024-02-23T00:47:05.161805+0000","last_became_active":"2024-02-23T00:42:58.850072+0000","last_became_peered":"2024-02-23T00:42:58.850072+0000","last_unstale":"2024-02-23T00:47:05.161805+0000","last_undegraded":"2024-02-23T00:47:05.161805+0000","last_fullsized":"2024-02-23T00:47:05.161805+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:58.613497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:58.613497+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"93'23","reported_seq":363,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.419498+0000","last_change":"2024-02-23T00:46:51.431911+0000","last_active":"2024-02-23T00:47:05.419498+0000","last_peered":"2024-02-23T00:47:05.419498+0000","last_clean":"2024-02-23T00:47:05.419498+0000","last_became_active":"2024-02-23T00:39:16.782660+0000","last_became_peered":"2024-02-23T00:39:16.782660+0000","last_unstale":"2024-02-23T00:47:05.419498+0000","last_undegraded":"2024-02-23T00:47:05.419498+0000","last_fullsized":"2024-02-23T00:47:05.419498+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'23","last_scrub_stamp":"2024-02-23T00:46:51.431838+0000","last_deep_scrub":"93'23","last_deep_scrub_stamp":"2024-02-23T00:41:35.954516+0000","last_clean_scrub_stamp":"2024-02-23T00:46:51.431838+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":319,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.161540+0000","last_change":"2024-02-23T00:44:21.157372+0000","last_active":"2024-02-23T00:47:06.161540+0000","last_peered":"2024-02-23T00:47:06.161540+0000","last_clean":"2024-02-23T00:47:06.161540+0000","last_became_active":"2024-02-23T00:39:14.781560+0000","last_became_peered":"2024-02-23T00:39:14.781560+0000","last_unstale":"2024-02-23T00:47:06.161540+0000","last_undegraded":"2024-02-23T00:47:06.161540+0000","last_fullsized":"2024-02-23T00:47:06.161540+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:21.157316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:21.157316+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":318,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.162782+0000","last_change":"2024-02-23T00:46:36.592971+0000","last_active":"2024-02-23T00:47:05.162782+0000","last_peered":"2024-02-23T00:47:05.162782+0000","last_clean":"2024-02-23T00:47:05.162782+0000","last_became_active":"2024-02-23T00:39:18.790602+0000","last_became_peered":"2024-02-23T00:39:18.790602+0000","last_unstale":"2024-02-23T00:47:05.162782+0000","last_undegraded":"2024-02-23T00:47:05.162782+0000","last_fullsized":"2024-02-23T00:47:05.162782+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:36.592887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:53.189881+0000","last_clean_scrub_stamp":"2024-02-23T00:46:36.592887+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":260,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.161542+0000","last_change":"2024-02-23T00:45:15.933508+0000","last_active":"2024-02-23T00:47:06.161542+0000","last_peered":"2024-02-23T00:47:06.161542+0000","last_clean":"2024-02-23T00:47:06.161542+0000","last_became_active":"2024-02-23T00:42:58.774720+0000","last_became_peered":"2024-02-23T00:42:58.774720+0000","last_unstale":"2024-02-23T00:47:06.161542+0000","last_undegraded":"2024-02-23T00:47:06.161542+0000","last_fullsized":"2024-02-23T00:47:06.161542+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:15.933438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:15.933438+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"67'15","reported_seq":344,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.666085+0000","last_change":"2024-02-23T00:46:03.951592+0000","last_active":"2024-02-23T00:47:05.666085+0000","last_peered":"2024-02-23T00:47:05.666085+0000","last_clean":"2024-02-23T00:47:05.666085+0000","last_became_active":"2024-02-23T00:39:16.787138+0000","last_became_peered":"2024-02-23T00:39:16.787138+0000","last_unstale":"2024-02-23T00:47:05.666085+0000","last_undegraded":"2024-02-23T00:47:05.666085+0000","last_fullsized":"2024-02-23T00:47:05.666085+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:46:03.951524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:03.951524+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":320,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.419738+0000","last_change":"2024-02-23T00:46:10.011072+0000","last_active":"2024-02-23T00:47:05.419738+0000","last_peered":"2024-02-23T00:47:05.419738+0000","last_clean":"2024-02-23T00:47:05.419738+0000","last_became_active":"2024-02-23T00:39:14.782494+0000","last_became_peered":"2024-02-23T00:39:14.782494+0000","last_unstale":"2024-02-23T00:47:05.419738+0000","last_undegraded":"2024-02-23T00:47:05.419738+0000","last_fullsized":"2024-02-23T00:47:05.419738+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:10.010996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:10.010996+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":312,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.570925+0000","last_change":"2024-02-23T00:45:03.615758+0000","last_active":"2024-02-23T00:47:06.570925+0000","last_peered":"2024-02-23T00:47:06.570925+0000","last_clean":"2024-02-23T00:47:06.570925+0000","last_became_active":"2024-02-23T00:39:18.801039+0000","last_became_peered":"2024-02-23T00:39:18.801039+0000","last_unstale":"2024-02-23T00:47:06.570925+0000","last_undegraded":"2024-02-23T00:47:06.570925+0000","last_fullsized":"2024-02-23T00:47:06.570925+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:03.615692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.641689+0000","last_clean_scrub_stamp":"2024-02-23T00:45:03.615692+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":260,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.162230+0000","last_change":"2024-02-23T00:45:21.917072+0000","last_active":"2024-02-23T00:47:06.162230+0000","last_peered":"2024-02-23T00:47:06.162230+0000","last_clean":"2024-02-23T00:47:06.162230+0000","last_became_active":"2024-02-23T00:42:58.770759+0000","last_became_peered":"2024-02-23T00:42:58.770759+0000","last_unstale":"2024-02-23T00:47:06.162230+0000","last_undegraded":"2024-02-23T00:47:06.162230+0000","last_fullsized":"2024-02-23T00:47:06.162230+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:21.916996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:21.916996+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"68'18","reported_seq":360,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.162163+0000","last_change":"2024-02-23T00:46:44.911346+0000","last_active":"2024-02-23T00:47:06.162163+0000","last_peered":"2024-02-23T00:47:06.162163+0000","last_clean":"2024-02-23T00:47:06.162163+0000","last_became_active":"2024-02-23T00:39:16.782803+0000","last_became_peered":"2024-02-23T00:39:16.782803+0000","last_unstale":"2024-02-23T00:47:06.162163+0000","last_undegraded":"2024-02-23T00:47:06.162163+0000","last_fullsized":"2024-02-23T00:47:06.162163+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'18","last_scrub_stamp":"2024-02-23T00:46:44.911278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:44.911278+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":319,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.162005+0000","last_change":"2024-02-23T00:43:48.030906+0000","last_active":"2024-02-23T00:47:06.162005+0000","last_peered":"2024-02-23T00:47:06.162005+0000","last_clean":"2024-02-23T00:47:06.162005+0000","last_became_active":"2024-02-23T00:39:14.782163+0000","last_became_peered":"2024-02-23T00:39:14.782163+0000","last_unstale":"2024-02-23T00:47:06.162005+0000","last_undegraded":"2024-02-23T00:47:06.162005+0000","last_fullsized":"2024-02-23T00:47:06.162005+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:43:48.030846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:43:48.030846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":314,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.570389+0000","last_change":"2024-02-23T00:44:37.068899+0000","last_active":"2024-02-23T00:47:06.570389+0000","last_peered":"2024-02-23T00:47:06.570389+0000","last_clean":"2024-02-23T00:47:06.570389+0000","last_became_active":"2024-02-23T00:39:18.794903+0000","last_became_peered":"2024-02-23T00:39:18.794903+0000","last_unstale":"2024-02-23T00:47:06.570389+0000","last_undegraded":"2024-02-23T00:47:06.570389+0000","last_fullsized":"2024-02-23T00:47:06.570389+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:37.068834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_clean_scrub_stamp":"2024-02-23T00:44:37.068834+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":258,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.417508+0000","last_change":"2024-02-23T00:46:00.943625+0000","last_active":"2024-02-23T00:47:05.417508+0000","last_peered":"2024-02-23T00:47:05.417508+0000","last_clean":"2024-02-23T00:47:05.417508+0000","last_became_active":"2024-02-23T00:42:58.698258+0000","last_became_peered":"2024-02-23T00:42:58.698258+0000","last_unstale":"2024-02-23T00:47:05.417508+0000","last_undegraded":"2024-02-23T00:47:05.417508+0000","last_fullsized":"2024-02-23T00:47:05.417508+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:00.943545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:46:00.943545+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"95'27","reported_seq":368,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.572336+0000","last_change":"2024-02-23T00:43:52.124336+0000","last_active":"2024-02-23T00:47:06.572336+0000","last_peered":"2024-02-23T00:47:06.572336+0000","last_clean":"2024-02-23T00:47:06.572336+0000","last_became_active":"2024-02-23T00:39:16.786791+0000","last_became_peered":"2024-02-23T00:39:16.786791+0000","last_unstale":"2024-02-23T00:47:06.572336+0000","last_undegraded":"2024-02-23T00:47:06.572336+0000","last_fullsized":"2024-02-23T00:47:06.572336+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'27","last_scrub_stamp":"2024-02-23T00:43:52.124247+0000","last_deep_scrub":"95'27","last_deep_scrub_stamp":"2024-02-23T00:41:55.919895+0000","last_clean_scrub_stamp":"2024-02-23T00:43:52.124247+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":320,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.485103+0000","last_change":"2024-02-23T00:46:47.689209+0000","last_active":"2024-02-23T00:47:05.485103+0000","last_peered":"2024-02-23T00:47:05.485103+0000","last_clean":"2024-02-23T00:47:05.485103+0000","last_became_active":"2024-02-23T00:39:14.787148+0000","last_became_peered":"2024-02-23T00:39:14.787148+0000","last_unstale":"2024-02-23T00:47:05.485103+0000","last_undegraded":"2024-02-23T00:47:05.485103+0000","last_fullsized":"2024-02-23T00:47:05.485103+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:47.689132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:47.689132+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":313,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.417369+0000","last_change":"2024-02-23T00:45:10.090353+0000","last_active":"2024-02-23T00:47:05.417369+0000","last_peered":"2024-02-23T00:47:05.417369+0000","last_clean":"2024-02-23T00:47:05.417369+0000","last_became_active":"2024-02-23T00:39:18.796431+0000","last_became_peered":"2024-02-23T00:39:18.796431+0000","last_unstale":"2024-02-23T00:47:05.417369+0000","last_undegraded":"2024-02-23T00:47:05.417369+0000","last_fullsized":"2024-02-23T00:47:05.417369+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.090270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.090270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":258,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.417224+0000","last_change":"2024-02-23T00:45:51.985111+0000","last_active":"2024-02-23T00:47:05.417224+0000","last_peered":"2024-02-23T00:47:05.417224+0000","last_clean":"2024-02-23T00:47:05.417224+0000","last_became_active":"2024-02-23T00:42:58.832885+0000","last_became_peered":"2024-02-23T00:42:58.832885+0000","last_unstale":"2024-02-23T00:47:05.417224+0000","last_undegraded":"2024-02-23T00:47:05.417224+0000","last_fullsized":"2024-02-23T00:47:05.417224+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:51.985023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:51.985023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"82'15","reported_seq":345,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.573596+0000","last_change":"2024-02-23T00:44:35.095478+0000","last_active":"2024-02-23T00:47:06.573596+0000","last_peered":"2024-02-23T00:47:06.573596+0000","last_clean":"2024-02-23T00:47:06.573596+0000","last_became_active":"2024-02-23T00:39:16.782762+0000","last_became_peered":"2024-02-23T00:39:16.782762+0000","last_unstale":"2024-02-23T00:47:06.573596+0000","last_undegraded":"2024-02-23T00:47:06.573596+0000","last_fullsized":"2024-02-23T00:47:06.573596+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'15","last_scrub_stamp":"2024-02-23T00:44:35.095409+0000","last_deep_scrub":"82'15","last_deep_scrub_stamp":"2024-02-23T00:41:50.975683+0000","last_clean_scrub_stamp":"2024-02-23T00:44:35.095409+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":320,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.485639+0000","last_change":"2024-02-23T00:46:08.794420+0000","last_active":"2024-02-23T00:47:05.485639+0000","last_peered":"2024-02-23T00:47:05.485639+0000","last_clean":"2024-02-23T00:47:05.485639+0000","last_became_active":"2024-02-23T00:39:14.783469+0000","last_became_peered":"2024-02-23T00:39:14.783469+0000","last_unstale":"2024-02-23T00:47:05.485639+0000","last_undegraded":"2024-02-23T00:47:05.485639+0000","last_fullsized":"2024-02-23T00:47:05.485639+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:08.794352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:08.794352+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":318,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.484090+0000","last_change":"2024-02-23T00:46:09.832330+0000","last_active":"2024-02-23T00:47:05.484090+0000","last_peered":"2024-02-23T00:47:05.484090+0000","last_clean":"2024-02-23T00:47:05.484090+0000","last_became_active":"2024-02-23T00:39:18.796952+0000","last_became_peered":"2024-02-23T00:39:18.796952+0000","last_unstale":"2024-02-23T00:47:05.484090+0000","last_undegraded":"2024-02-23T00:47:05.484090+0000","last_fullsized":"2024-02-23T00:47:05.484090+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:09.832253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:09.832253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":262,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.571811+0000","last_change":"2024-02-23T00:45:23.224771+0000","last_active":"2024-02-23T00:47:06.571811+0000","last_peered":"2024-02-23T00:47:06.571811+0000","last_clean":"2024-02-23T00:47:06.571811+0000","last_became_active":"2024-02-23T00:42:58.758147+0000","last_became_peered":"2024-02-23T00:42:58.758147+0000","last_unstale":"2024-02-23T00:47:06.571811+0000","last_undegraded":"2024-02-23T00:47:06.571811+0000","last_fullsized":"2024-02-23T00:47:06.571811+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:23.224663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:23.224663+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.224663+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"68'15","reported_seq":347,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.570795+0000","last_change":"2024-02-23T00:43:49.139235+0000","last_active":"2024-02-23T00:47:06.570795+0000","last_peered":"2024-02-23T00:47:06.570795+0000","last_clean":"2024-02-23T00:47:06.570795+0000","last_became_active":"2024-02-23T00:39:16.784606+0000","last_became_peered":"2024-02-23T00:39:16.784606+0000","last_unstale":"2024-02-23T00:47:06.570795+0000","last_undegraded":"2024-02-23T00:47:06.570795+0000","last_fullsized":"2024-02-23T00:47:06.570795+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:43:49.139156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:43:49.139156+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":319,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.163244+0000","last_change":"2024-02-23T00:44:32.170717+0000","last_active":"2024-02-23T00:47:06.163244+0000","last_peered":"2024-02-23T00:47:06.163244+0000","last_clean":"2024-02-23T00:47:06.163244+0000","last_became_active":"2024-02-23T00:39:14.785362+0000","last_became_peered":"2024-02-23T00:39:14.785362+0000","last_unstale":"2024-02-23T00:47:06.163244+0000","last_undegraded":"2024-02-23T00:47:06.163244+0000","last_fullsized":"2024-02-23T00:47:06.163244+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:44:32.170554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:44:32.170554+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":312,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.570768+0000","last_change":"2024-02-23T00:46:47.226397+0000","last_active":"2024-02-23T00:47:06.570768+0000","last_peered":"2024-02-23T00:47:06.570768+0000","last_clean":"2024-02-23T00:47:06.570768+0000","last_became_active":"2024-02-23T00:39:18.796312+0000","last_became_peered":"2024-02-23T00:39:18.796312+0000","last_unstale":"2024-02-23T00:47:06.570768+0000","last_undegraded":"2024-02-23T00:47:06.570768+0000","last_fullsized":"2024-02-23T00:47:06.570768+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:47.226331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:47.226331+0000","last_clean_scrub_stamp":"2024-02-23T00:46:47.226331+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":258,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.162466+0000","last_change":"2024-02-23T00:45:09.641006+0000","last_active":"2024-02-23T00:47:05.162466+0000","last_peered":"2024-02-23T00:47:05.162466+0000","last_clean":"2024-02-23T00:47:05.162466+0000","last_became_active":"2024-02-23T00:42:58.838257+0000","last_became_peered":"2024-02-23T00:42:58.838257+0000","last_unstale":"2024-02-23T00:47:05.162466+0000","last_undegraded":"2024-02-23T00:47:05.162466+0000","last_fullsized":"2024-02-23T00:47:05.162466+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:09.640936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:09.640936+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"88'50","reported_seq":413,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.163663+0000","last_change":"2024-02-23T00:45:06.038488+0000","last_active":"2024-02-23T00:47:06.163663+0000","last_peered":"2024-02-23T00:47:06.163663+0000","last_clean":"2024-02-23T00:47:06.163663+0000","last_became_active":"2024-02-23T00:39:16.780144+0000","last_became_peered":"2024-02-23T00:39:16.780144+0000","last_unstale":"2024-02-23T00:47:06.163663+0000","last_undegraded":"2024-02-23T00:47:06.163663+0000","last_fullsized":"2024-02-23T00:47:06.163663+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'50","last_scrub_stamp":"2024-02-23T00:45:06.038374+0000","last_deep_scrub":"88'50","last_deep_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_clean_scrub_stamp":"2024-02-23T00:45:06.038374+0000","log_size":50,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":317,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.162437+0000","last_change":"2024-02-23T00:45:20.573233+0000","last_active":"2024-02-23T00:47:05.162437+0000","last_peered":"2024-02-23T00:47:05.162437+0000","last_clean":"2024-02-23T00:47:05.162437+0000","last_became_active":"2024-02-23T00:39:14.782000+0000","last_became_peered":"2024-02-23T00:39:14.782000+0000","last_unstale":"2024-02-23T00:47:05.162437+0000","last_undegraded":"2024-02-23T00:47:05.162437+0000","last_fullsized":"2024-02-23T00:47:05.162437+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:20.573168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_clean_scrub_stamp":"2024-02-23T00:45:20.573168+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":314,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-02-23T00:47:06.571045+0000","last_change":"2024-02-23T00:46:49.271764+0000","last_active":"2024-02-23T00:47:06.571045+0000","last_peered":"2024-02-23T00:47:06.571045+0000","last_clean":"2024-02-23T00:47:06.571045+0000","last_became_active":"2024-02-23T00:39:18.799723+0000","last_became_peered":"2024-02-23T00:39:18.799723+0000","last_unstale":"2024-02-23T00:47:06.571045+0000","last_undegraded":"2024-02-23T00:47:06.571045+0000","last_fullsized":"2024-02-23T00:47:06.571045+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:49.271705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:49.271705+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":258,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-02-23T00:47:05.419155+0000","last_change":"2024-02-23T00:45:14.209403+0000","last_active":"2024-02-23T00:47:05.419155+0000","last_peered":"2024-02-23T00:47:05.419155+0000","last_clean":"2024-02-23T00:47:05.419155+0000","last_became_active":"2024-02-23T00:42:58.824573+0000","last_became_peered":"2024-02-23T00:42:58.824573+0000","last_unstale":"2024-02-23T00:47:05.419155+0000","last_undegraded":"2024-02-23T00:47:05.419155+0000","last_fullsized":"2024-02-23T00:47:05.419155+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:14.209308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:14.209308+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0],"acting":[5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":199,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":196,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":8},{"poolid":62,"num_pg":32,"stat_sum":{"num_bytes":4001380,"num_objects":143,"num_object_clones":2,"num_object_copies":429,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":143,"num_whiteouts":0,"num_read":98,"num_read_kb":4,"num_write":568,"num_write_kb":4469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":13737984,"data_stored":12004140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1814,"ondisk_log_size":1814,"up":96,"acting":96,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":22,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":66,"acting":66,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67,"ondisk_log_size":67,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1080,"num_read_kb":805,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":657,"ondisk_log_size":657,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":464,"num_read_kb":459,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234082,"num_pgs":104,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":311200,"kb_used_data":9876,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93433952,"statfs":{"total":95995035648,"available":95676366848,"internally_reserved":0,"allocated":10113024,"data_stored":7661137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:46:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54900000000000004,"15min":0.54600000000000004},"min":{"1min":0.315,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.76400000000000001,"5min":4.9420000000000002,"15min":4.9420000000000002},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54600000000000004,"15min":0.51800000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.77800000000000002,"5min":4.9729999999999999,"15min":4.9729999999999999},"last":0.495}]},{"osd":1,"last update":"Fri Feb 23 00:46:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.78600000000000003,"15min":0.627},"min":{"1min":0.40600000000000003,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70499999999999996,"5min":10.786,"15min":10.786},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.79300000000000004,"15min":0.628},"min":{"1min":0.317,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.73399999999999999,"5min":11.885999999999999,"15min":11.885999999999999},"last":0.56499999999999995}]},{"osd":2,"last update":"Fri Feb 23 00:46:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.60399999999999998,"15min":0.55900000000000005},"min":{"1min":0.40000000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71199999999999997,"5min":5.0789999999999997,"15min":5.0789999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.58299999999999996,"15min":0.54800000000000004},"min":{"1min":0.38300000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.69399999999999995,"5min":5.1280000000000001,"15min":5.1280000000000001},"last":0.55300000000000005}]},{"osd":3,"last update":"Fri Feb 23 00:46:26 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.81000000000000005,"15min":0.61899999999999999},"min":{"1min":0.34499999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67600000000000005,"5min":15.712999999999999,"15min":15.712999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.73799999999999999,"15min":0.621},"min":{"1min":0.376,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.1659999999999999,"5min":9.4090000000000007,"15min":9.4090000000000007},"last":0.58499999999999996}]},{"osd":4,"last update":"Fri Feb 23 00:46:26 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.45800000000000002,"15min":0.437},"min":{"1min":0.28599999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.65300000000000002,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":0.443},{"interface":"front","average":{"1min":0.499,"5min":0.56999999999999995,"15min":0.48899999999999999},"min":{"1min":0.38600000000000001,"5min":0.28599999999999998,"15min":0.28199999999999997},"max":{"1min":0.66700000000000004,"5min":11.236000000000001,"15min":11.236000000000001},"last":0.39400000000000002}]},{"osd":5,"last update":"Fri Feb 23 00:46:26 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.53600000000000003,"15min":0.49299999999999999},"min":{"1min":0.38100000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.68400000000000005,"5min":5.4000000000000004,"15min":5.4000000000000004},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52300000000000002,"15min":0.48999999999999999},"min":{"1min":0.376,"5min":0.29299999999999998,"15min":0.26700000000000002},"max":{"1min":0.67800000000000005,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.41799999999999998}]},{"osd":6,"last update":"Fri Feb 23 00:46:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52400000000000002,"15min":0.50900000000000001},"min":{"1min":0.42499999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.66000000000000003,"5min":1.034,"15min":1.034},"last":0.378},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.64200000000000002,"15min":0.54800000000000004},"min":{"1min":0.38400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.2370000000000001,"5min":13.395,"15min":13.395},"last":0.34000000000000002}]}]},{"osd":6,"up_from":47,"seq":201863463017,"num_pgs":76,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":319492,"kb_used_data":5876,"kb_used_omap":0,"kb_used_meta":313600,"kb_avail":93425660,"statfs":{"total":95995035648,"available":95667875840,"internally_reserved":0,"allocated":6017024,"data_stored":3650749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321126400},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:46:56 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.96399999999999997,"15min":0.69899999999999995},"min":{"1min":0.27300000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.74399999999999999,"5min":30.641999999999999,"15min":30.641999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.96399999999999997,"15min":0.70599999999999996},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.60799999999999998,"5min":29.292000000000002,"15min":29.292000000000002},"last":0.54500000000000004}]},{"osd":1,"last update":"Fri Feb 23 00:46:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.98799999999999999,"15min":0.69399999999999995},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.68000000000000005,"5min":28.186,"15min":28.186},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":1.4710000000000001,"15min":0.875},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.73299999999999998,"5min":33.104999999999997,"15min":33.104999999999997},"last":0.51600000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:46:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":1.032,"15min":0.72999999999999998},"min":{"1min":0.374,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.73599999999999999,"5min":19.960000000000001,"15min":19.960000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.83599999999999997,"15min":0.65000000000000002},"min":{"1min":0.34499999999999997,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.66400000000000003,"5min":8.5860000000000003,"15min":8.5860000000000003},"last":0.52900000000000003}]},{"osd":3,"last update":"Fri Feb 23 00:46:56 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.753,"15min":0.63600000000000001},"min":{"1min":0.29199999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.69299999999999995,"5min":19.562000000000001,"15min":19.562000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.84099999999999997,"15min":0.67400000000000004},"min":{"1min":0.33500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70699999999999996,"5min":14.199,"15min":14.199},"last":0.45500000000000002}]},{"osd":4,"last update":"Fri Feb 23 00:46:56 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.57899999999999996,"15min":0.46700000000000003},"min":{"1min":0.245,"5min":0.218,"15min":0.218},"max":{"1min":0.63200000000000001,"5min":11.109999999999999,"15min":11.109999999999999},"last":0.441},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.752,"15min":0.54000000000000004},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71199999999999997,"5min":16.734000000000002,"15min":16.734000000000002},"last":0.49299999999999999}]},{"osd":5,"last update":"Fri Feb 23 00:46:56 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.63700000000000001,"15min":0.49199999999999999},"min":{"1min":0.309,"5min":0.249,"15min":0.249},"max":{"1min":0.64500000000000002,"5min":14.154999999999999,"15min":14.154999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.63700000000000001,"15min":0.50900000000000001},"min":{"1min":0.36399999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.71099999999999997,"5min":6.6100000000000003,"15min":6.6100000000000003},"last":0.55600000000000005}]},{"osd":7,"last update":"Fri Feb 23 00:46:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.59299999999999997,"15min":0.52400000000000002},"min":{"1min":0.371,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.78100000000000003,"5min":8.548,"15min":8.548},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.76700000000000002,"15min":0.59899999999999998},"min":{"1min":0.47199999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.76300000000000001,"5min":15.618,"15min":15.618},"last":0.57099999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574979,"num_pgs":85,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":319284,"kb_used_data":9768,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93425868,"statfs":{"total":95995035648,"available":95668088832,"internally_reserved":0,"allocated":10002432,"data_stored":7651864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:46:45 2024","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":1.2789999999999999,"15min":0.66300000000000003},"min":{"1min":0.23400000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.41799999999999998,"5min":78.361000000000004,"15min":78.361000000000004},"last":0.251},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.56299999999999994,"15min":0.437},"min":{"1min":0.27800000000000002,"5min":0.218,"15min":0.191},"max":{"1min":0.51700000000000002,"5min":5.8849999999999998,"15min":5.8849999999999998},"last":0.40500000000000003}]},{"osd":2,"last update":"Fri Feb 23 00:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":1.1659999999999999,"15min":0.65100000000000002},"min":{"1min":0.214,"5min":0.214,"15min":0.188},"max":{"1min":0.42599999999999999,"5min":48.606000000000002,"15min":48.606000000000002},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.877,"15min":0.58199999999999996},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1,"5min":23.911000000000001,"15min":23.911000000000001},"last":0.26000000000000001}]},{"osd":3,"last update":"Fri Feb 23 00:46:45 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.88800000000000001,"15min":0.61699999999999999},"min":{"1min":0.26700000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.61499999999999999,"5min":35.244999999999997,"15min":35.244999999999997},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.39900000000000002,"5min":1.1299999999999999,"15min":0.67800000000000005},"min":{"1min":0.27400000000000002,"5min":0.253,"15min":0.248},"max":{"1min":0.63900000000000001,"5min":56.853999999999999,"15min":56.853999999999999},"last":0.28999999999999998}]},{"osd":4,"last update":"Fri Feb 23 00:46:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":1.216,"15min":0.81399999999999995},"min":{"1min":0.34999999999999998,"5min":0.33700000000000002,"15min":0.29299999999999998},"max":{"1min":0.71899999999999997,"5min":37.646999999999998,"15min":37.646999999999998},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.85199999999999998,"15min":0.69499999999999995},"min":{"1min":0.35899999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.75700000000000001,"5min":9.6859999999999999,"15min":9.6859999999999999},"last":0.37}]},{"osd":5,"last update":"Fri Feb 23 00:46:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":1.2150000000000001,"15min":0.83899999999999997},"min":{"1min":0.39800000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79100000000000004,"5min":48.792000000000002,"15min":48.792000000000002},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.81399999999999995,"15min":0.745},"min":{"1min":0.45000000000000001,"5min":0.38100000000000001,"15min":0.34200000000000003},"max":{"1min":0.78800000000000003,"5min":11.872999999999999,"15min":11.872999999999999},"last":0.45700000000000002}]},{"osd":6,"last update":"Fri Feb 23 00:46:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":1.173,"15min":0.81599999999999995},"min":{"1min":0.39800000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.73899999999999999,"5min":40.741,"15min":40.741},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.75900000000000001,"15min":0.68500000000000005},"min":{"1min":0.442,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71799999999999997,"5min":6.0069999999999997,"15min":6.0069999999999997},"last":0.496}]},{"osd":7,"last update":"Fri Feb 23 00:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.89200000000000002,"15min":0.71599999999999997},"min":{"1min":0.39300000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80800000000000005,"5min":20.225000000000001,"15min":20.225000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.93000000000000005,"15min":0.72399999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.73199999999999998,"5min":20.254999999999999,"15min":20.254999999999999},"last":0.44600000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738505,"num_pgs":77,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":303084,"kb_used_data":5856,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93442068,"statfs":{"total":95995035648,"available":95684677632,"internally_reserved":0,"allocated":5996544,"data_stored":3661460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 23 00:46:46 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":1.0880000000000001,"15min":0.59199999999999997},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.19500000000000001},"max":{"1min":0.47899999999999998,"5min":28.856999999999999,"15min":28.856999999999999},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.33300000000000002,"5min":1.091,"15min":0.60699999999999998},"min":{"1min":0.25,"5min":0.218,"15min":0.20799999999999999},"max":{"1min":0.55700000000000005,"5min":20.577000000000002,"15min":20.577000000000002},"last":0.41899999999999998}]},{"osd":2,"last update":"Fri Feb 23 00:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":1.014,"15min":0.60299999999999998},"min":{"1min":0.27900000000000003,"5min":0.26000000000000001,"15min":0.20499999999999999},"max":{"1min":0.55200000000000005,"5min":13.863,"15min":13.863},"last":0.376},{"interface":"front","average":{"1min":0.318,"5min":0.86899999999999999,"15min":0.54500000000000004},"min":{"1min":0.219,"5min":0.219,"15min":0.19800000000000001},"max":{"1min":0.373,"5min":10.455,"15min":10.455},"last":0.39200000000000002}]},{"osd":3,"last update":"Fri Feb 23 00:46:38 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":1.5069999999999999,"15min":0.79600000000000004},"min":{"1min":0.255,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.504,"5min":35.566000000000003,"15min":35.566000000000003},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.39100000000000001,"5min":1.361,"15min":0.78400000000000003},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.52600000000000002,"5min":20.702999999999999,"15min":20.702999999999999},"last":0.48599999999999999}]},{"osd":4,"last update":"Fri Feb 23 00:47:03 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":1.0349999999999999,"15min":0.79200000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80500000000000005,"5min":10.584,"15min":10.584},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":1.026,"15min":0.81799999999999995},"min":{"1min":0.38100000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78200000000000003,"5min":13.198,"15min":13.198},"last":0.60199999999999998}]},{"osd":5,"last update":"Fri Feb 23 00:46:28 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":1.004,"15min":0.73799999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72499999999999998,"5min":10.397,"15min":10.397},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":1.077,"15min":0.80300000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84999999999999998,"5min":17.297000000000001,"15min":17.297000000000001},"last":0.67300000000000004}]},{"osd":6,"last update":"Fri Feb 23 00:47:03 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":1.099,"15min":0.79400000000000004},"min":{"1min":0.43099999999999999,"5min":0.42299999999999999,"15min":0.38400000000000001},"max":{"1min":0.72099999999999997,"5min":17.373999999999999,"15min":17.373999999999999},"last":0.623},{"interface":"front","average":{"1min":0.56299999999999994,"5min":1.0369999999999999,"15min":0.76500000000000001},"min":{"1min":0.38400000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.82099999999999995,"5min":13.973000000000001,"15min":13.973000000000001},"last":0.66000000000000003}]},{"osd":7,"last update":"Fri Feb 23 00:46:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":1.083,"15min":0.80700000000000005},"min":{"1min":0.45200000000000001,"5min":0.41799999999999998,"15min":0.39400000000000002},"max":{"1min":0.81399999999999995,"5min":17.399999999999999,"15min":17.399999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.629,"5min":1.0669999999999999,"15min":0.78800000000000003},"min":{"1min":0.49299999999999999,"5min":0.433,"15min":0.43099999999999999},"max":{"1min":0.81200000000000006,"5min":16.427,"15min":16.427},"last":0.64100000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411454,"num_pgs":73,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":315524,"kb_used_data":6000,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93429628,"statfs":{"total":95995035648,"available":95671939072,"internally_reserved":0,"allocated":6144000,"data_stored":3661139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:46:04 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.69399999999999995,"15min":0.48399999999999999},"min":{"1min":0.22900000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.61199999999999999,"5min":13.657,"15min":13.657},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.374,"5min":1.262,"15min":0.68000000000000005},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.624,"5min":42.343000000000004,"15min":42.343000000000004},"last":0.44800000000000001}]},{"osd":1,"last update":"Fri Feb 23 00:46:04 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.96199999999999997,"15min":0.57399999999999995},"min":{"1min":0.254,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.53500000000000003,"5min":18.952000000000002,"15min":18.952000000000002},"last":0.434},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.75800000000000001,"15min":0.52200000000000002},"min":{"1min":0.26300000000000001,"5min":0.23999999999999999,"15min":0.23599999999999999},"max":{"1min":0.56499999999999995,"5min":15.391,"15min":15.391},"last":0.52900000000000003}]},{"osd":3,"last update":"Fri Feb 23 00:46:39 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":1.175,"15min":0.70699999999999996},"min":{"1min":0.26600000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.89100000000000001,"5min":25.635999999999999,"15min":25.635999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.42899999999999999,"5min":1.006,"15min":0.64300000000000002},"min":{"1min":0.29099999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.80700000000000005,"5min":19.960000000000001,"15min":19.960000000000001},"last":0.54700000000000004}]},{"osd":4,"last update":"Fri Feb 23 00:46:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.99199999999999999,"15min":0.72799999999999998},"min":{"1min":0.41299999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.74099999999999999,"5min":13.673,"15min":13.673},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":1.0389999999999999,"15min":0.73499999999999999},"min":{"1min":0.379,"5min":0.36899999999999999,"15min":0.33900000000000002},"max":{"1min":0.90000000000000002,"5min":10.103999999999999,"15min":10.103999999999999},"last":0.58599999999999997}]},{"osd":5,"last update":"Fri Feb 23 00:46:27 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.90100000000000002,"15min":0.68799999999999994},"min":{"1min":0.40200000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.89000000000000001,"5min":15.612,"15min":15.612},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.86199999999999999,"15min":0.69799999999999995},"min":{"1min":0.40100000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.752,"5min":13.632999999999999,"15min":13.632999999999999},"last":0.71299999999999997}]},{"osd":6,"last update":"Fri Feb 23 00:46:58 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.87,"15min":0.70899999999999996},"min":{"1min":0.42899999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82799999999999996,"5min":11.544,"15min":11.544},"last":0.746},{"interface":"front","average":{"1min":0.60199999999999998,"5min":1.0369999999999999,"15min":0.749},"min":{"1min":0.39500000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0069999999999999,"5min":10.132999999999999,"15min":10.132999999999999},"last":0.68799999999999994}]},{"osd":7,"last update":"Fri Feb 23 00:46:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.86599999999999999,"15min":0.73599999999999999},"min":{"1min":0.40999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.81999999999999995,"5min":11.599,"15min":11.599},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.625,"5min":1.1020000000000001,"15min":0.79800000000000004},"min":{"1min":0.46100000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.92200000000000004,"5min":10.148999999999999,"15min":10.148999999999999},"last":0.66600000000000004}]}]},{"osd":3,"up_from":26,"seq":111669149815,"num_pgs":89,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":303148,"kb_used_data":5916,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93442004,"statfs":{"total":95995035648,"available":95684612096,"internally_reserved":0,"allocated":6057984,"data_stored":3661836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:46:37 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.75,"15min":0.53700000000000003},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":1.274,"5min":12.212999999999999,"15min":12.212999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.95799999999999996,"15min":0.61699999999999999},"min":{"1min":0.23899999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":5.9800000000000004,"5min":11.714,"15min":11.714},"last":0.377}]},{"osd":1,"last update":"Fri Feb 23 00:46:37 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.012,"15min":0.64900000000000002},"min":{"1min":0.26200000000000001,"5min":0.23599999999999999,"15min":0.22900000000000001},"max":{"1min":6.851,"5min":11.742000000000001,"15min":11.742000000000001},"last":0.27200000000000002},{"interface":"front","average":{"1min":1.1479999999999999,"5min":0.93500000000000005,"15min":0.60699999999999998},"min":{"1min":0.30299999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":8.2270000000000003,"5min":8.2270000000000003,"15min":8.2270000000000003},"last":0.53600000000000003}]},{"osd":2,"last update":"Fri Feb 23 00:46:37 2024","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.097,"15min":0.69099999999999995},"min":{"1min":0.307,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":8.0269999999999992,"5min":13.359999999999999,"15min":13.359999999999999},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":1.0329999999999999,"15min":0.66100000000000003},"min":{"1min":0.28499999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":2.5459999999999998,"5min":21.122,"15min":21.122},"last":0.42399999999999999}]},{"osd":4,"last update":"Fri Feb 23 00:47:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":1.0760000000000001,"15min":0.75800000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.2070000000000001,"5min":11.644,"15min":11.644},"last":0.435},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.079,"15min":0.752},"min":{"1min":0.39300000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":12.430999999999999,"5min":12.430999999999999,"15min":12.430999999999999},"last":0.61399999999999999}]},{"osd":5,"last update":"Fri Feb 23 00:46:26 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.79800000000000004,"15min":0.66600000000000004},"min":{"1min":0.441,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":2.125,"5min":5.6360000000000001,"15min":5.6360000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.1639999999999999,"15min":0.80200000000000005},"min":{"1min":0.45000000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":11.648,"5min":11.648,"15min":11.648},"last":0.51400000000000001}]},{"osd":6,"last update":"Fri Feb 23 00:46:48 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.0409999999999999,"15min":0.75},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":6.9420000000000002,"5min":6.9420000000000002,"15min":6.9420000000000002},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.80200000000000005,"15min":0.68600000000000005},"min":{"1min":0.41899999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.647,"5min":5.6680000000000001,"15min":5.6680000000000001},"last":0.59699999999999998}]},{"osd":7,"last update":"Fri Feb 23 00:46:22 2024","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.206,"15min":0.82599999999999996},"min":{"1min":0.48399999999999999,"5min":0.41099999999999998,"15min":0.39000000000000001},"max":{"1min":11.676,"5min":11.676,"15min":11.676},"last":0.48499999999999999},{"interface":"front","average":{"1min":1.411,"5min":1.1160000000000001,"15min":0.80800000000000005},"min":{"1min":0.51600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":7.9800000000000004,"5min":7.9800000000000004,"15min":7.9800000000000004},"last":0.66700000000000004}]}]},{"osd":4,"up_from":32,"seq":137438953587,"num_pgs":97,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":307064,"kb_used_data":9832,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93438088,"statfs":{"total":95995035648,"available":95680602112,"internally_reserved":0,"allocated":10067968,"data_stored":7650946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:47:04 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.505,"15min":0.54900000000000004},"min":{"1min":0.29999999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.67400000000000004,"5min":3.5219999999999998,"15min":3.5219999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.65100000000000002,"15min":0.59999999999999998},"min":{"1min":0.318,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.68300000000000005,"5min":8.6140000000000008,"15min":8.6140000000000008},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Feb 23 00:47:04 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.64900000000000002,"15min":0.59799999999999998},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.64700000000000002,"5min":9.6110000000000007,"15min":9.6110000000000007},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.64300000000000002,"15min":0.59399999999999997},"min":{"1min":0.32800000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.69799999999999995,"5min":6.6950000000000003,"15min":6.6950000000000003},"last":0.59399999999999997}]},{"osd":2,"last update":"Fri Feb 23 00:47:04 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.61099999999999999,"15min":0.58599999999999997},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.65600000000000003,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.57899999999999996,"15min":0.58199999999999996},"min":{"1min":0.34499999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.61399999999999999,"5min":5.8440000000000003,"15min":5.8440000000000003},"last":0.502}]},{"osd":3,"last update":"Fri Feb 23 00:47:04 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.64100000000000001,"15min":0.59899999999999998},"min":{"1min":0.36199999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67300000000000004,"5min":6.5540000000000003,"15min":6.5540000000000003},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.64500000000000002,"15min":0.60099999999999998},"min":{"1min":0.39000000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69299999999999995,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":0.50800000000000001}]},{"osd":5,"last update":"Fri Feb 23 00:46:25 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.41499999999999998,"15min":0.39000000000000001},"min":{"1min":0.28000000000000003,"5min":0.252,"15min":0.246},"max":{"1min":0.67200000000000004,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.317},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.48099999999999998,"15min":0.442},"min":{"1min":0.29999999999999999,"5min":0.29599999999999999,"15min":0.28899999999999998},"max":{"1min":0.71999999999999997,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.36799999999999999}]},{"osd":6,"last update":"Fri Feb 23 00:47:01 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42999999999999999,"15min":0.41899999999999998},"min":{"1min":0.28699999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.62,"5min":0.75,"15min":0.75},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.438,"15min":0.42199999999999999},"min":{"1min":0.32500000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72699999999999998},"last":0.32900000000000001}]},{"osd":7,"last update":"Fri Feb 23 00:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.47699999999999998,"15min":0.48299999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.254},"max":{"1min":0.77800000000000002,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.52900000000000003,"15min":0.53500000000000003},"min":{"1min":0.34699999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78800000000000003,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.38500000000000001}]}]},{"osd":5,"up_from":39,"seq":167503724654,"num_pgs":83,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":319460,"kb_used_data":5844,"kb_used_omap":0,"kb_used_meta":313600,"kb_avail":93425692,"statfs":{"total":95995035648,"available":95667908608,"internally_reserved":0,"allocated":5984256,"data_stored":3662181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321126400},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.78200000000000003,"15min":0.59999999999999998},"min":{"1min":0.33200000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.68500000000000005,"5min":23.881,"15min":23.881},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.5,"5min":0.68799999999999994,"15min":0.57699999999999996},"min":{"1min":0.40100000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73899999999999999,"5min":5.4649999999999999,"15min":5.4649999999999999},"last":0.64200000000000002}]},{"osd":1,"last update":"Fri Feb 23 00:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.89400000000000002,"15min":0.64900000000000002},"min":{"1min":0.318,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.89600000000000002,"5min":18.175999999999998,"15min":18.175999999999998},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.78300000000000003,"15min":0.62},"min":{"1min":0.32300000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.70699999999999996,"5min":9.0950000000000006,"15min":9.0950000000000006},"last":0.61399999999999999}]},{"osd":2,"last update":"Fri Feb 23 00:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":1.018,"15min":0.70299999999999996},"min":{"1min":0.34100000000000003,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70599999999999996,"5min":22.282,"15min":22.282},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.54500000000000004,"5min":1.0509999999999999,"15min":0.70899999999999996},"min":{"1min":0.378,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77200000000000002,"5min":23.925000000000001,"15min":23.925000000000001},"last":0.56999999999999995}]},{"osd":3,"last update":"Fri Feb 23 00:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.78100000000000003,"15min":0.61299999999999999},"min":{"1min":0.38500000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.77900000000000003,"5min":6.5730000000000004,"15min":6.5730000000000004},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.80500000000000005,"15min":0.64100000000000001},"min":{"1min":0.40699999999999997,"5min":0.33600000000000002,"15min":0.33500000000000002},"max":{"1min":0.753,"5min":8.1760000000000002,"15min":8.1760000000000002},"last":0.68799999999999994}]},{"osd":4,"last update":"Fri Feb 23 00:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.42999999999999999,"15min":0.40100000000000002},"min":{"1min":0.29499999999999998,"5min":0.247,"15min":0.247},"max":{"1min":0.64600000000000002,"5min":0.68999999999999995,"15min":0.70399999999999996},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.51600000000000001,"15min":0.436},"min":{"1min":0.27700000000000002,"5min":0.27500000000000002,"15min":0.27300000000000002},"max":{"1min":0.67000000000000004,"5min":3.879,"15min":3.879},"last":0.46000000000000002}]},{"osd":6,"last update":"Fri Feb 23 00:46:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48099999999999998,"15min":0.45000000000000001},"min":{"1min":0.312,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70699999999999996,"5min":1.736,"15min":1.736},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48299999999999998,"15min":0.46300000000000002},"min":{"1min":0.28000000000000003,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73799999999999999,"5min":0.871,"15min":0.871},"last":0.36599999999999999}]},{"osd":7,"last update":"Fri Feb 23 00:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.57699999999999996,"15min":0.53400000000000003},"min":{"1min":0.34799999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.753,"5min":7.0300000000000002,"15min":7.0300000000000002},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.58299999999999996,"15min":0.53300000000000003},"min":{"1min":0.33700000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.68999999999999995,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":0.45000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":159744,"data_stored":391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4173824,"data_stored":4000414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":323584,"data_stored":743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":196608,"data_stored":465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4243456,"data_stored":4000589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":212992,"data_stored":524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4284416,"data_stored":4000662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":196,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":196,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":196,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":196,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":196,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":196,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":196,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":196,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-23T00:47:10.218 INFO:tasks.cephadm.ceph_manager.ceph:decrease pool size by 10 2024-02-23T00:47:10.219 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd pool set default.rgw.meta pg_num 12 2024-02-23T00:47:10.225 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:47:10.285 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:47:10.311 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:10.455 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:09.172116+0000 mon.a (mon.0) 2300 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:47:10.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:09.172197+0000 mon.a (mon.0) 2301 : cluster [DBG] osdmap e357: 8 total, 8 up, 8 in 2024-02-23T00:47:10.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:09.173297+0000 mon.a (mon.0) 2302 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-45"}]: dispatch 2024-02-23T00:47:10.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:09.269469+0000 mgr.y (mgr.14517) 340 : cluster [DBG] pgmap v511: 226 pgs: 8 unknown, 218 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:47:10.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:09.840603+0000 mon.c (mon.2) 272 : audit [INF] from='client.? 172.21.15.47:0/488990342' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:47:10.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:09.840928+0000 mon.a (mon.0) 2303 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:47:10.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:10.169011+0000 mon.a (mon.0) 2304 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:47:10.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:09.172116+0000 mon.a (mon.0) 2300 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:47:10.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:09.172197+0000 mon.a (mon.0) 2301 : cluster [DBG] osdmap e357: 8 total, 8 up, 8 in 2024-02-23T00:47:10.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:09.173297+0000 mon.a (mon.0) 2302 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-45"}]: dispatch 2024-02-23T00:47:10.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:09.269469+0000 mgr.y (mgr.14517) 340 : cluster [DBG] pgmap v511: 226 pgs: 8 unknown, 218 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:47:10.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:09.840603+0000 mon.c (mon.2) 272 : audit [INF] from='client.? 172.21.15.47:0/488990342' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:47:10.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:09.840928+0000 mon.a (mon.0) 2303 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:47:10.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:10.169011+0000 mon.a (mon.0) 2304 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:47:10.457 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:47:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:47:10] "GET /metrics HTTP/1.1" 200 190742 "" "Prometheus/2.33.4" 2024-02-23T00:47:10.520 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:09.172116+0000 mon.a (mon.0) 2300 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:47:10.520 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:09.172197+0000 mon.a (mon.0) 2301 : cluster [DBG] osdmap e357: 8 total, 8 up, 8 in 2024-02-23T00:47:10.520 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:09.173297+0000 mon.a (mon.0) 2302 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-45"}]: dispatch 2024-02-23T00:47:10.520 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:09.269469+0000 mgr.y (mgr.14517) 340 : cluster [DBG] pgmap v511: 226 pgs: 8 unknown, 218 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:47:10.520 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:09.840603+0000 mon.c (mon.2) 272 : audit [INF] from='client.? 172.21.15.47:0/488990342' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:47:10.520 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:09.840928+0000 mon.a (mon.0) 2303 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:47:10.521 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:10.169011+0000 mon.a (mon.0) 2304 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:47:10.560 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:47:10.565 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:10.565 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:47:10.666 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:47:10.693 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:10.791 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:47:10.793 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:10.794 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:47:10.894 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:47:10.922 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:11.063 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:47:11.066 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:11.067 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:47:11.069 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:47:11.069 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-02-23T00:47:11.069 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 0 2024-02-23T00:47:11.069 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:47:11.168 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:47:11.210 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:11.233 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:47:11.240 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:11.240 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:47:11.341 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:47:11.371 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:11.766 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:47:11.779 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.7 dump_blocked_ops 2024-02-23T00:47:11.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:10.175667+0000 mon.a (mon.0) 2305 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-45"}]': finished 2024-02-23T00:47:11.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:10.175773+0000 mon.a (mon.0) 2306 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:47:11.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:10.175836+0000 mon.a (mon.0) 2307 : cluster [DBG] osdmap e358: 8 total, 8 up, 8 in 2024-02-23T00:47:11.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:10.196589+0000 mon.a (mon.0) 2308 : audit [INF] from='client.? 172.21.15.47:0/3942253766' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi047-82386-60"}]: dispatch 2024-02-23T00:47:11.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:10.254574+0000 mon.a (mon.0) 2309 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:47:11.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:10.254975+0000 mon.a (mon.0) 2310 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-45"}]: dispatch 2024-02-23T00:47:11.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:11.191187+0000 mon.a (mon.0) 2311 : audit [INF] from='client.? 172.21.15.47:0/3942253766' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi047-82386-60"}]': finished 2024-02-23T00:47:11.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:11.191268+0000 mon.a (mon.0) 2312 : cluster [DBG] osdmap e359: 8 total, 8 up, 8 in 2024-02-23T00:47:11.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:11.196827+0000 mon.a (mon.0) 2313 : audit [INF] from='client.? 172.21.15.47:0/3942253766' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi047-82386-60"}]: dispatch 2024-02-23T00:47:11.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:10.175667+0000 mon.a (mon.0) 2305 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-45"}]': finished 2024-02-23T00:47:11.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:10.175773+0000 mon.a (mon.0) 2306 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:47:11.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:10.175836+0000 mon.a (mon.0) 2307 : cluster [DBG] osdmap e358: 8 total, 8 up, 8 in 2024-02-23T00:47:11.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:10.196589+0000 mon.a (mon.0) 2308 : audit [INF] from='client.? 172.21.15.47:0/3942253766' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi047-82386-60"}]: dispatch 2024-02-23T00:47:11.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:10.254574+0000 mon.a (mon.0) 2309 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:47:11.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:10.254975+0000 mon.a (mon.0) 2310 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-45"}]: dispatch 2024-02-23T00:47:11.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:11.191187+0000 mon.a (mon.0) 2311 : audit [INF] from='client.? 172.21.15.47:0/3942253766' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi047-82386-60"}]': finished 2024-02-23T00:47:11.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:11.191268+0000 mon.a (mon.0) 2312 : cluster [DBG] osdmap e359: 8 total, 8 up, 8 in 2024-02-23T00:47:11.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:11.196827+0000 mon.a (mon.0) 2313 : audit [INF] from='client.? 172.21.15.47:0/3942253766' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi047-82386-60"}]: dispatch 2024-02-23T00:47:11.820 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:47:11.822 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:11.823 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:47:11.924 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:47:11.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:10.175667+0000 mon.a (mon.0) 2305 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-45"}]': finished 2024-02-23T00:47:11.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:10.175773+0000 mon.a (mon.0) 2306 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:47:11.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:10.175836+0000 mon.a (mon.0) 2307 : cluster [DBG] osdmap e358: 8 total, 8 up, 8 in 2024-02-23T00:47:11.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:10.196589+0000 mon.a (mon.0) 2308 : audit [INF] from='client.? 172.21.15.47:0/3942253766' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi047-82386-60"}]: dispatch 2024-02-23T00:47:11.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:10.254574+0000 mon.a (mon.0) 2309 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:47:11.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:10.254975+0000 mon.a (mon.0) 2310 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-45"}]: dispatch 2024-02-23T00:47:11.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:11.191187+0000 mon.a (mon.0) 2311 : audit [INF] from='client.? 172.21.15.47:0/3942253766' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi047-82386-60"}]': finished 2024-02-23T00:47:11.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:11.191268+0000 mon.a (mon.0) 2312 : cluster [DBG] osdmap e359: 8 total, 8 up, 8 in 2024-02-23T00:47:11.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:11.196827+0000 mon.a (mon.0) 2313 : audit [INF] from='client.? 172.21.15.47:0/3942253766' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi047-82386-60"}]: dispatch 2024-02-23T00:47:11.952 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:11.971 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:47:11.974 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:11.974 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:47:12.075 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:47:12.088 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:47:12.102 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:12.122 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:47:12.125 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:12.125 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:47:12.203 INFO:tasks.workunit.client.0.smithi047.stdout:o.RoundTripAppendPP (3006 ms) 2024-02-23T00:47:12.203 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2024-02-23T00:47:12.203 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (3392 ms) 2024-02-23T00:47:12.203 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-02-23T00:47:12.203 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (2583 ms) 2024-02-23T00:47:12.203 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-02-23T00:47:12.203 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (3013 ms) 2024-02-23T00:47:12.204 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2024-02-23T00:47:12.204 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (2981 ms) 2024-02-23T00:47:12.204 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [----------] 31 tests from LibRadosAio (127905 ms total) 2024-02-23T00:47:12.204 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: 2024-02-23T00:47:12.204 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2024-02-23T00:47:12.204 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-02-23T00:47:12.204 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (3014 ms) 2024-02-23T00:47:12.204 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-02-23T00:47:12.204 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (9430 ms) 2024-02-23T00:47:12.204 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2024-02-23T00:47:12.204 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (14567 ms) 2024-02-23T00:47:12.204 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2024-02-23T00:47:12.204 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (3008 ms) 2024-02-23T00:47:12.205 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (30020 ms total) 2024-02-23T00:47:12.205 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: 2024-02-23T00:47:12.205 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2024-02-23T00:47:12.205 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-02-23T00:47:12.205 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (3427 ms) 2024-02-23T00:47:12.205 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (3428 ms total) 2024-02-23T00:47:12.205 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: 2024-02-23T00:47:12.205 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2024-02-23T00:47:12.205 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2024-02-23T00:47:12.205 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (14012 ms) 2024-02-23T00:47:12.205 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2024-02-23T00:47:12.205 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (6562 ms) 2024-02-23T00:47:12.205 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2024-02-23T00:47:12.205 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (7001 ms) 2024-02-23T00:47:12.205 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2024-02-23T00:47:12.205 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (7027 ms) 2024-02-23T00:47:12.206 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2024-02-23T00:47:12.206 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (2985 ms) 2024-02-23T00:47:12.206 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-02-23T00:47:12.206 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (7017 ms) 2024-02-23T00:47:12.206 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-02-23T00:47:12.206 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (7003 ms) 2024-02-23T00:47:12.206 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2024-02-23T00:47:12.206 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (6979 ms) 2024-02-23T00:47:12.206 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2024-02-23T00:47:12.206 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (7026 ms) 2024-02-23T00:47:12.206 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2024-02-23T00:47:12.206 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (6995 ms) 2024-02-23T00:47:12.206 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2024-02-23T00:47:12.206 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (6996 ms) 2024-02-23T00:47:12.207 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2024-02-23T00:47:12.207 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (6995 ms) 2024-02-23T00:47:12.207 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-02-23T00:47:12.207 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (7017 ms) 2024-02-23T00:47:12.207 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP2 2024-02-23T00:47:12.207 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (2981 ms) 2024-02-23T00:47:12.207 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2024-02-23T00:47:12.207 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (7034 ms) 2024-02-23T00:47:12.226 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:47:12.256 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:12.636 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:47:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:47:12.636 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:47:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:47:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:47:12.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:10.414630+0000 osd.6 (osd.6) 143 : cluster [DBG] 18.14 scrub starts 2024-02-23T00:47:12.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:10.415453+0000 osd.6 (osd.6) 144 : cluster [DBG] 18.14 scrub ok 2024-02-23T00:47:12.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:11.223578+0000 osd.5 (osd.5) 150 : cluster [DBG] 6.a scrub starts 2024-02-23T00:47:12.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:11.224381+0000 osd.5 (osd.5) 151 : cluster [DBG] 6.a scrub ok 2024-02-23T00:47:12.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:11.269830+0000 mgr.y (mgr.14517) 341 : cluster [DBG] pgmap v514: 186 pgs: 186 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:47:12.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:11.377338+0000 osd.1 (osd.1) 157 : cluster [DBG] 62.11 scrub starts 2024-02-23T00:47:12.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:11.380975+0000 osd.1 (osd.1) 158 : cluster [DBG] 62.11 scrub ok 2024-02-23T00:47:12.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:11.418655+0000 osd.6 (osd.6) 145 : cluster [DBG] 2.11 scrub starts 2024-02-23T00:47:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:11.419447+0000 osd.6 (osd.6) 146 : cluster [DBG] 2.11 scrub ok 2024-02-23T00:47:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:11.624724+0000 osd.2 (osd.2) 153 : cluster [DBG] 6.f scrub starts 2024-02-23T00:47:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:11.625649+0000 osd.2 (osd.2) 154 : cluster [DBG] 6.f scrub ok 2024-02-23T00:47:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:12.195367+0000 mon.a (mon.0) 2314 : audit [INF] from='client.? 172.21.15.47:0/3942253766' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi047-82386-60"}]': finished 2024-02-23T00:47:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:12.195457+0000 mon.a (mon.0) 2315 : cluster [DBG] osdmap e360: 8 total, 8 up, 8 in 2024-02-23T00:47:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:12.203294+0000 mon.a (mon.0) 2316 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:47:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:12.219042+0000 mon.a (mon.0) 2317 : audit [INF] from='client.? 172.21.15.47:0/526847179' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi047-82386-61"}]: dispatch 2024-02-23T00:47:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:12.219735+0000 mon.a (mon.0) 2318 : audit [INF] from='client.? 172.21.15.47:0/526847179' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi047-82386-61"}]: dispatch 2024-02-23T00:47:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:12.220062+0000 mon.a (mon.0) 2319 : audit [INF] from='client.? 172.21.15.47:0/526847179' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi047-82386-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:47:12.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:10.414630+0000 osd.6 (osd.6) 143 : cluster [DBG] 18.14 scrub starts 2024-02-23T00:47:12.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:10.415453+0000 osd.6 (osd.6) 144 : cluster [DBG] 18.14 scrub ok 2024-02-23T00:47:12.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:11.223578+0000 osd.5 (osd.5) 150 : cluster [DBG] 6.a scrub starts 2024-02-23T00:47:12.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:11.224381+0000 osd.5 (osd.5) 151 : cluster [DBG] 6.a scrub ok 2024-02-23T00:47:12.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:11.269830+0000 mgr.y (mgr.14517) 341 : cluster [DBG] pgmap v514: 186 pgs: 186 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:47:12.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:11.377338+0000 osd.1 (osd.1) 157 : cluster [DBG] 62.11 scrub starts 2024-02-23T00:47:12.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:11.380975+0000 osd.1 (osd.1) 158 : cluster [DBG] 62.11 scrub ok 2024-02-23T00:47:12.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:11.418655+0000 osd.6 (osd.6) 145 : cluster [DBG] 2.11 scrub starts 2024-02-23T00:47:12.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:11.419447+0000 osd.6 (osd.6) 146 : cluster [DBG] 2.11 scrub ok 2024-02-23T00:47:12.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:11.624724+0000 osd.2 (osd.2) 153 : cluster [DBG] 6.f scrub starts 2024-02-23T00:47:12.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:11.625649+0000 osd.2 (osd.2) 154 : cluster [DBG] 6.f scrub ok 2024-02-23T00:47:12.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:12.195367+0000 mon.a (mon.0) 2314 : audit [INF] from='client.? 172.21.15.47:0/3942253766' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi047-82386-60"}]': finished 2024-02-23T00:47:12.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:12.195457+0000 mon.a (mon.0) 2315 : cluster [DBG] osdmap e360: 8 total, 8 up, 8 in 2024-02-23T00:47:12.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:12.203294+0000 mon.a (mon.0) 2316 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:47:12.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:12.219042+0000 mon.a (mon.0) 2317 : audit [INF] from='client.? 172.21.15.47:0/526847179' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi047-82386-61"}]: dispatch 2024-02-23T00:47:12.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:12.219735+0000 mon.a (mon.0) 2318 : audit [INF] from='client.? 172.21.15.47:0/526847179' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi047-82386-61"}]: dispatch 2024-02-23T00:47:12.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:12.220062+0000 mon.a (mon.0) 2319 : audit [INF] from='client.? 172.21.15.47:0/526847179' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi047-82386-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:47:12.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:10.414630+0000 osd.6 (osd.6) 143 : cluster [DBG] 18.14 scrub starts 2024-02-23T00:47:12.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:10.415453+0000 osd.6 (osd.6) 144 : cluster [DBG] 18.14 scrub ok 2024-02-23T00:47:12.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:11.223578+0000 osd.5 (osd.5) 150 : cluster [DBG] 6.a scrub starts 2024-02-23T00:47:12.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:11.224381+0000 osd.5 (osd.5) 151 : cluster [DBG] 6.a scrub ok 2024-02-23T00:47:12.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:11.269830+0000 mgr.y (mgr.14517) 341 : cluster [DBG] pgmap v514: 186 pgs: 186 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:47:12.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:11.377338+0000 osd.1 (osd.1) 157 : cluster [DBG] 62.11 scrub starts 2024-02-23T00:47:12.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:11.380975+0000 osd.1 (osd.1) 158 : cluster [DBG] 62.11 scrub ok 2024-02-23T00:47:12.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:11.418655+0000 osd.6 (osd.6) 145 : cluster [DBG] 2.11 scrub starts 2024-02-23T00:47:12.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:11.419447+0000 osd.6 (osd.6) 146 : cluster [DBG] 2.11 scrub ok 2024-02-23T00:47:12.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:11.624724+0000 osd.2 (osd.2) 153 : cluster [DBG] 6.f scrub starts 2024-02-23T00:47:12.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:11.625649+0000 osd.2 (osd.2) 154 : cluster [DBG] 6.f scrub ok 2024-02-23T00:47:12.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:12.195367+0000 mon.a (mon.0) 2314 : audit [INF] from='client.? 172.21.15.47:0/3942253766' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi047-82386-60"}]': finished 2024-02-23T00:47:12.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:12.195457+0000 mon.a (mon.0) 2315 : cluster [DBG] osdmap e360: 8 total, 8 up, 8 in 2024-02-23T00:47:12.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:12.203294+0000 mon.a (mon.0) 2316 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:47:12.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:12.219042+0000 mon.a (mon.0) 2317 : audit [INF] from='client.? 172.21.15.47:0/526847179' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi047-82386-61"}]: dispatch 2024-02-23T00:47:12.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:12.219735+0000 mon.a (mon.0) 2318 : audit [INF] from='client.? 172.21.15.47:0/526847179' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi047-82386-61"}]: dispatch 2024-02-23T00:47:12.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:12.220062+0000 mon.a (mon.0) 2319 : audit [INF] from='client.? 172.21.15.47:0/526847179' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi047-82386-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:47:12.974 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-23T00:47:13.806 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:47:13.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:12.410391+0000 osd.6 (osd.6) 147 : cluster [DBG] 5.11 scrub starts 2024-02-23T00:47:13.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:12.411243+0000 osd.6 (osd.6) 148 : cluster [DBG] 5.11 scrub ok 2024-02-23T00:47:13.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:12.536551+0000 mgr.y (mgr.14517) 342 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:47:13.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:13.017783+0000 osd.3 (osd.3) 171 : cluster [DBG] 6.c scrub starts 2024-02-23T00:47:13.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:13.019366+0000 osd.3 (osd.3) 172 : cluster [DBG] 6.c scrub ok 2024-02-23T00:47:13.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:13.196948+0000 mon.a (mon.0) 2320 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:47:13.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:13.197060+0000 mon.a (mon.0) 2321 : audit [INF] from='client.? 172.21.15.47:0/526847179' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi047-82386-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:47:13.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:13.197126+0000 mon.a (mon.0) 2322 : cluster [DBG] osdmap e361: 8 total, 8 up, 8 in 2024-02-23T00:47:13.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:13.203418+0000 mon.a (mon.0) 2323 : audit [INF] from='client.? 172.21.15.47:0/526847179' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi047-82386-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi047-82386-61"}]: dispatch 2024-02-23T00:47:13.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:13.218086+0000 mon.a (mon.0) 2324 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:47:13.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:12.410391+0000 osd.6 (osd.6) 147 : cluster [DBG] 5.11 scrub starts 2024-02-23T00:47:13.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:12.411243+0000 osd.6 (osd.6) 148 : cluster [DBG] 5.11 scrub ok 2024-02-23T00:47:13.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:12.536551+0000 mgr.y (mgr.14517) 342 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:47:13.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:13.017783+0000 osd.3 (osd.3) 171 : cluster [DBG] 6.c scrub starts 2024-02-23T00:47:13.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:13.019366+0000 osd.3 (osd.3) 172 : cluster [DBG] 6.c scrub ok 2024-02-23T00:47:13.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:13.196948+0000 mon.a (mon.0) 2320 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:47:13.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:13.197060+0000 mon.a (mon.0) 2321 : audit [INF] from='client.? 172.21.15.47:0/526847179' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi047-82386-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:47:13.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:13.197126+0000 mon.a (mon.0) 2322 : cluster [DBG] osdmap e361: 8 total, 8 up, 8 in 2024-02-23T00:47:13.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:13.203418+0000 mon.a (mon.0) 2323 : audit [INF] from='client.? 172.21.15.47:0/526847179' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi047-82386-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi047-82386-61"}]: dispatch 2024-02-23T00:47:13.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:13.218086+0000 mon.a (mon.0) 2324 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:47:13.812 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:13.812 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:47:13.909 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:47:13.910 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-02-23T00:47:13.910 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-02-23T00:47:13.910 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-02-23T00:47:13.910 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:47:13.914 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:47:13.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:12.410391+0000 osd.6 (osd.6) 147 : cluster [DBG] 5.11 scrub starts 2024-02-23T00:47:13.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:12.411243+0000 osd.6 (osd.6) 148 : cluster [DBG] 5.11 scrub ok 2024-02-23T00:47:13.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:12.536551+0000 mgr.y (mgr.14517) 342 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:47:13.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:13.017783+0000 osd.3 (osd.3) 171 : cluster [DBG] 6.c scrub starts 2024-02-23T00:47:13.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:13.019366+0000 osd.3 (osd.3) 172 : cluster [DBG] 6.c scrub ok 2024-02-23T00:47:13.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:13.196948+0000 mon.a (mon.0) 2320 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:47:13.933 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:13.197060+0000 mon.a (mon.0) 2321 : audit [INF] from='client.? 172.21.15.47:0/526847179' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi047-82386-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:47:13.933 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:13.197126+0000 mon.a (mon.0) 2322 : cluster [DBG] osdmap e361: 8 total, 8 up, 8 in 2024-02-23T00:47:13.933 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:13.203418+0000 mon.a (mon.0) 2323 : audit [INF] from='client.? 172.21.15.47:0/526847179' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi047-82386-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi047-82386-61"}]: dispatch 2024-02-23T00:47:13.933 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:13.218086+0000 mon.a (mon.0) 2324 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:47:13.943 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:14.420 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:47:14.423 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:14.424 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:47:14.525 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:47:14.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.417+0000 7f9a1c6d2700 1 -- 172.21.15.47:0/3412598555 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9a140f8420 msgr2=0x7f9a14104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:14.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.417+0000 7f9a1c6d2700 1 --2- 172.21.15.47:0/3412598555 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9a140f8420 0x7f9a14104a20 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 rx=0x7f9a0c009be0 tx=0x7f9a0c004d10).stop 2024-02-23T00:47:14.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.417+0000 7f9a1c6d2700 1 -- 172.21.15.47:0/3412598555 shutdown_connections 2024-02-23T00:47:14.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.417+0000 7f9a1c6d2700 1 --2- 172.21.15.47:0/3412598555 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9a140f8420 0x7f9a14104a20 unknown :-1 s=CLOSED pgs=2639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:14.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.417+0000 7f9a1c6d2700 1 --2- 172.21.15.47:0/3412598555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a140fdbb0 0x7f9a140fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:14.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.417+0000 7f9a1c6d2700 1 --2- 172.21.15.47:0/3412598555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a140fe510 0x7f9a140f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:14.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.417+0000 7f9a1c6d2700 1 -- 172.21.15.47:0/3412598555 >> 172.21.15.47:0/3412598555 conn(0x7f9a140f3800 msgr2=0x7f9a140f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:14.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.419+0000 7f9a1c6d2700 1 -- 172.21.15.47:0/3412598555 shutdown_connections 2024-02-23T00:47:14.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.419+0000 7f9a1c6d2700 1 -- 172.21.15.47:0/3412598555 wait complete. 2024-02-23T00:47:14.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.419+0000 7f9a1c6d2700 1 librados: starting msgr at 2024-02-23T00:47:14.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.419+0000 7f9a1c6d2700 1 librados: starting objecter 2024-02-23T00:47:14.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.419+0000 7f9a1c6d2700 1 Processor -- start 2024-02-23T00:47:14.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.419+0000 7f9a1c6d2700 1 -- start start 2024-02-23T00:47:14.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.419+0000 7f9a1c6d2700 1 librados: setting wanted keys 2024-02-23T00:47:14.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.419+0000 7f9a1c6d2700 1 librados: calling monclient init 2024-02-23T00:47:14.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.420+0000 7f9a1c6d2700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9a140f8420 0x7f9a1410dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:14.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.420+0000 7f9a1c6d2700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a140fdbb0 0x7f9a1410e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:14.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.420+0000 7f9a1c6d2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a140fe510 0x7f9a14107f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:14.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.420+0000 7f9a1c6d2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a14108410 con 0x7f9a140fe510 2024-02-23T00:47:14.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.420+0000 7f9a1c6d2700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a14108580 con 0x7f9a140fdbb0 2024-02-23T00:47:14.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.420+0000 7f9a1c6d2700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a14108740 con 0x7f9a140f8420 2024-02-23T00:47:14.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.420+0000 7f9a1a46e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9a140f8420 0x7f9a1410dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:14.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.420+0000 7f9a1a46e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9a140f8420 0x7f9a1410dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:36550/0 (socket says 172.21.15.47:36550) 2024-02-23T00:47:14.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.420+0000 7f9a1ac6f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a140fe510 0x7f9a14107f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:14.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.420+0000 7f9a1a46e700 1 -- 172.21.15.47:0/2923629940 learned_addr learned my addr 172.21.15.47:0/2923629940 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:47:14.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.420+0000 7f9a1a46e700 1 -- 172.21.15.47:0/2923629940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a140fdbb0 msgr2=0x7f9a1410e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:14.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.420+0000 7f9a1a46e700 1 --2- 172.21.15.47:0/2923629940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a140fdbb0 0x7f9a1410e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:47:14.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.420+0000 7f9a1a46e700 1 -- 172.21.15.47:0/2923629940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a140fe510 msgr2=0x7f9a14107f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:14.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.420+0000 7f9a1a46e700 1 --2- 172.21.15.47:0/2923629940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a140fe510 0x7f9a14107f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:14.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.420+0000 7f9a1a46e700 1 -- 172.21.15.47:0/2923629940 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c0097c0 con 0x7f9a140f8420 2024-02-23T00:47:14.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.421+0000 7f9a1a46e700 1 --2- 172.21.15.47:0/2923629940 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9a140f8420 0x7f9a1410dd80 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 rx=0x7f9a1000e970 tx=0x7f9a1000b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:14.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.421+0000 7f9a0b7fe700 10 client.?.objecter ms_handle_connect 0x7f9a140f8420 2024-02-23T00:47:14.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.421+0000 7f9a0b7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:47:14.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.421+0000 7f9a0b7fe700 1 -- 172.21.15.47:0/2923629940 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a10004660 con 0x7f9a140f8420 2024-02-23T00:47:14.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.421+0000 7f9a0b7fe700 1 -- 172.21.15.47:0/2923629940 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f9a1000f040 con 0x7f9a140f8420 2024-02-23T00:47:14.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.421+0000 7f9a1c6d2700 1 -- 172.21.15.47:0/2923629940 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a141089c0 con 0x7f9a140f8420 2024-02-23T00:47:14.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.422+0000 7f9a0b7fe700 1 -- 172.21.15.47:0/2923629940 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1000c840 con 0x7f9a140f8420 2024-02-23T00:47:14.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.422+0000 7f9a1c6d2700 10 client.41213.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:47:14.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.422+0000 7f9a1c6d2700 1 -- 172.21.15.47:0/2923629940 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a1406af90 con 0x7f9a140f8420 2024-02-23T00:47:14.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.422+0000 7f9a1c6d2700 1 librados: init done 2024-02-23T00:47:14.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.422+0000 7f9a1c6d2700 1 -- 172.21.15.47:0/2923629940 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a14106ed0 con 0x7f9a140f8420 2024-02-23T00:47:14.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.423+0000 7f9a0b7fe700 1 -- 172.21.15.47:0/2923629940 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f9a10009160 con 0x7f9a140f8420 2024-02-23T00:47:14.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.424+0000 7f9a0b7fe700 1 --2- 172.21.15.47:0/2923629940 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9a00062cc0 0x7f9a00065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:14.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.424+0000 7f9a0b7fe700 1 -- 172.21.15.47:0/2923629940 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(362..362 src has 48..362) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f9a1007d780 con 0x7f9a140f8420 2024-02-23T00:47:14.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.424+0000 7f9a0b7fe700 10 client.41213.objecter ms_dispatch 0x7f9a14001550 osd_map(362..362 src has 48..362) v4 2024-02-23T00:47:14.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.424+0000 7f9a0b7fe700 3 client.41213.objecter handle_osd_map got epochs [362,362] > 0 2024-02-23T00:47:14.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.424+0000 7f9a0b7fe700 3 client.41213.objecter handle_osd_map decoding full epoch 362 2024-02-23T00:47:14.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.424+0000 7f9a19c6d700 1 --2- 172.21.15.47:0/2923629940 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9a00062cc0 0x7f9a00065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:14.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.424+0000 7f9a0b7fe700 20 client.41213.objecter dump_active .. 0 homeless 2024-02-23T00:47:14.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.424+0000 7f9a0b7fe700 10 client.41213.objecter ms_handle_connect 0x7f9a00062cc0 2024-02-23T00:47:14.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.424+0000 7f9a19c6d700 1 --2- 172.21.15.47:0/2923629940 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9a00062cc0 0x7f9a00065180 secure :-1 s=READY pgs=3043 cs=0 l=1 rev1=1 rx=0x7f9a141096c0 tx=0x7f9a0400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:14.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.428+0000 7f9a0b7fe700 1 -- 172.21.15.47:0/2923629940 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f9a1004cd60 con 0x7f9a140f8420 2024-02-23T00:47:14.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.608+0000 7f9a1c6d2700 1 -- 172.21.15.47:0/2923629940 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "12"} v 0) v1 -- 0x7f9a14108b50 con 0x7f9a140f8420 2024-02-23T00:47:14.648 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.7 dump_historic_ops 2024-02-23T00:47:14.674 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:12.797294+0000 osd.0 (osd.0) 145 : cluster [DBG] 5.3 scrub starts 2024-02-23T00:47:14.675 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:12.798309+0000 osd.0 (osd.0) 146 : cluster [DBG] 5.3 scrub ok 2024-02-23T00:47:14.675 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:13.270265+0000 mgr.y (mgr.14517) 343 : cluster [DBG] pgmap v517: 218 pgs: 11 creating+peering, 21 unknown, 186 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-23T00:47:14.675 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:14.525946+0000 mon.b (mon.1) 377 : audit [DBG] from='client.? 172.21.15.47:0/3756679349' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:47:14.675 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:14.609553+0000 mon.c (mon.2) 273 : audit [INF] from='client.? 172.21.15.47:0/2923629940' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "12"}]: dispatch 2024-02-23T00:47:14.675 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:14.639469+0000 mon.a (mon.0) 2325 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:47:14.675 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:14.639565+0000 mon.a (mon.0) 2326 : cluster [DBG] osdmap e362: 8 total, 8 up, 8 in 2024-02-23T00:47:14.675 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:14.640403+0000 mon.a (mon.0) 2327 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "12"}]: dispatch 2024-02-23T00:47:14.675 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:14.640715+0000 mon.a (mon.0) 2328 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-47"}]: dispatch 2024-02-23T00:47:14.675 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:12.797294+0000 osd.0 (osd.0) 145 : cluster [DBG] 5.3 scrub starts 2024-02-23T00:47:14.675 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:12.798309+0000 osd.0 (osd.0) 146 : cluster [DBG] 5.3 scrub ok 2024-02-23T00:47:14.676 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:13.270265+0000 mgr.y (mgr.14517) 343 : cluster [DBG] pgmap v517: 218 pgs: 11 creating+peering, 21 unknown, 186 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-23T00:47:14.676 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:14.525946+0000 mon.b (mon.1) 377 : audit [DBG] from='client.? 172.21.15.47:0/3756679349' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:47:14.676 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:14.609553+0000 mon.c (mon.2) 273 : audit [INF] from='client.? 172.21.15.47:0/2923629940' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "12"}]: dispatch 2024-02-23T00:47:14.676 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:14.639469+0000 mon.a (mon.0) 2325 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:47:14.676 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:14.639565+0000 mon.a (mon.0) 2326 : cluster [DBG] osdmap e362: 8 total, 8 up, 8 in 2024-02-23T00:47:14.676 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:14.640403+0000 mon.a (mon.0) 2327 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "12"}]: dispatch 2024-02-23T00:47:14.676 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:14.640715+0000 mon.a (mon.0) 2328 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-47"}]: dispatch 2024-02-23T00:47:14.676 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:14.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.521+0000 7ff7b12ef700 1 -- 172.21.15.47:0/2887219146 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff7ac1022d0 msgr2=0x7ff7ac1046c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:14.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.521+0000 7ff7b12ef700 1 --2- 172.21.15.47:0/2887219146 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff7ac1022d0 0x7ff7ac1046c0 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 rx=0x7ff7a4009be0 tx=0x7ff7a4004d10).stop 2024-02-23T00:47:14.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.521+0000 7ff7b12ef700 1 -- 172.21.15.47:0/2887219146 shutdown_connections 2024-02-23T00:47:14.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.521+0000 7ff7b12ef700 1 --2- 172.21.15.47:0/2887219146 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff7ac1022d0 0x7ff7ac1046c0 unknown :-1 s=CLOSED pgs=2641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:14.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.521+0000 7ff7b12ef700 1 --2- 172.21.15.47:0/2887219146 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7ac0f8d70 0x7ff7ac0f9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:14.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.521+0000 7ff7b12ef700 1 --2- 172.21.15.47:0/2887219146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7ac0f9640 0x7ff7ac101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:14.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.521+0000 7ff7b12ef700 1 -- 172.21.15.47:0/2887219146 >> 172.21.15.47:0/2887219146 conn(0x7ff7ac0f2820 msgr2=0x7ff7ac0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:14.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.521+0000 7ff7b12ef700 1 -- 172.21.15.47:0/2887219146 shutdown_connections 2024-02-23T00:47:14.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.521+0000 7ff7b12ef700 1 -- 172.21.15.47:0/2887219146 wait complete. 2024-02-23T00:47:14.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.522+0000 7ff7b12ef700 1 librados: starting msgr at 2024-02-23T00:47:14.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.522+0000 7ff7b12ef700 1 librados: starting objecter 2024-02-23T00:47:14.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.522+0000 7ff7b12ef700 1 Processor -- start 2024-02-23T00:47:14.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.522+0000 7ff7b12ef700 1 -- start start 2024-02-23T00:47:14.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.522+0000 7ff7b12ef700 1 librados: setting wanted keys 2024-02-23T00:47:14.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.522+0000 7ff7b12ef700 1 librados: calling monclient init 2024-02-23T00:47:14.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.522+0000 7ff7b12ef700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7ac0f8d70 0x7ff7ac0f6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:14.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.522+0000 7ff7b12ef700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7ac0f9640 0x7ff7ac0f7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:14.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.522+0000 7ff7b12ef700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff7ac1022d0 0x7ff7ac0f7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:14.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.522+0000 7ff7b12ef700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac116400 con 0x7ff7ac0f8d70 2024-02-23T00:47:14.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.522+0000 7ff7b12ef700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac116570 con 0x7ff7ac0f9640 2024-02-23T00:47:14.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.522+0000 7ff7b12ef700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac116730 con 0x7ff7ac1022d0 2024-02-23T00:47:14.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.522+0000 7ff7aaffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7ac0f8d70 0x7ff7ac0f6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:14.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.522+0000 7ff7aaffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7ac0f8d70 0x7ff7ac0f6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48860/0 (socket says 172.21.15.47:48860) 2024-02-23T00:47:14.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.522+0000 7ff7aaffd700 1 -- 172.21.15.47:0/3756679349 learned_addr learned my addr 172.21.15.47:0/3756679349 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:47:14.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.522+0000 7ff7aa7fc700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7ac0f9640 0x7ff7ac0f7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:14.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.522+0000 7ff7ab7fe700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff7ac1022d0 0x7ff7ac0f7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:14.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.523+0000 7ff7aa7fc700 1 -- 172.21.15.47:0/3756679349 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff7ac1022d0 msgr2=0x7ff7ac0f7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:14.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.523+0000 7ff7aa7fc700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff7ac1022d0 0x7ff7ac0f7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.523+0000 7ff7aa7fc700 1 -- 172.21.15.47:0/3756679349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7ac0f8d70 msgr2=0x7ff7ac0f6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.523+0000 7ff7aa7fc700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7ac0f8d70 0x7ff7ac0f6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.523+0000 7ff7aa7fc700 1 -- 172.21.15.47:0/3756679349 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a0005040 con 0x7ff7ac0f9640 2024-02-23T00:47:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.523+0000 7ff7aa7fc700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7ac0f9640 0x7ff7ac0f7210 secure :-1 s=READY pgs=3975 cs=0 l=1 rev1=1 rx=0x7ff7a0004920 tx=0x7ff7a0009cb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.523+0000 7ff793fff700 10 client.?.objecter ms_handle_connect 0x7ff7ac0f9640 2024-02-23T00:47:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.523+0000 7ff793fff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:47:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.523+0000 7ff793fff700 1 -- 172.21.15.47:0/3756679349 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a000e030 con 0x7ff7ac0f9640 2024-02-23T00:47:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.523+0000 7ff793fff700 1 -- 172.21.15.47:0/3756679349 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7ff7a000f040 con 0x7ff7ac0f9640 2024-02-23T00:47:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.523+0000 7ff793fff700 1 -- 172.21.15.47:0/3756679349 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0014720 con 0x7ff7ac0f9640 2024-02-23T00:47:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.523+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a40097c0 con 0x7ff7ac0f9640 2024-02-23T00:47:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.523+0000 7ff7b12ef700 10 client.43276.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:47:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.523+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7ac116cf0 con 0x7ff7ac0f9640 2024-02-23T00:47:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.523+0000 7ff7b12ef700 1 librados: init done 2024-02-23T00:47:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.524+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7ff7ac05fa60 con 0x7ff7ac0f9640 2024-02-23T00:47:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.525+0000 7ff793fff700 1 -- 172.21.15.47:0/3756679349 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7ff7a0010500 con 0x7ff7ac0f9640 2024-02-23T00:47:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.525+0000 7ff793fff700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff794062cc0 0x7ff794065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.525+0000 7ff7aaffd700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff794062cc0 0x7ff794065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.525+0000 7ff793fff700 1 -- 172.21.15.47:0/3756679349 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(362..362 src has 48..362) v4 ==== 9582+0+0 (secure 0 0 0) 0x7ff7a007de50 con 0x7ff7ac0f9640 2024-02-23T00:47:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.525+0000 7ff793fff700 10 client.43276.objecter ms_dispatch 0x7ff7ac001550 osd_map(362..362 src has 48..362) v4 2024-02-23T00:47:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.525+0000 7ff793fff700 3 client.43276.objecter handle_osd_map got epochs [362,362] > 0 2024-02-23T00:47:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.525+0000 7ff793fff700 3 client.43276.objecter handle_osd_map decoding full epoch 362 2024-02-23T00:47:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.526+0000 7ff793fff700 20 client.43276.objecter dump_active .. 0 homeless 2024-02-23T00:47:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.526+0000 7ff793fff700 1 -- 172.21.15.47:0/3756679349 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v362) v1 ==== 54+0+16 (secure 0 0 0) 0x7ff7a007e270 con 0x7ff7ac0f9640 2024-02-23T00:47:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.526+0000 7ff7aaffd700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff794062cc0 0x7ff794065180 secure :-1 s=READY pgs=3044 cs=0 l=1 rev1=1 rx=0x7ff79c004e80 tx=0x7ff79c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.526+0000 7ff793fff700 10 client.43276.objecter ms_handle_connect 0x7ff794062cc0 2024-02-23T00:47:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.526+0000 7ff7b12ef700 10 client.43276.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:14.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.526+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_assign -1 1 2024-02-23T00:47:14.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.526+0000 7ff7b12ef700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7ff7ac0fc2a0 0x7ff7ac0fc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:14.727 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:47:14.727 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:47:14.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.526+0000 7ff7b12ef700 20 client.43276.objecter _get_session s=0x7ff7ac05ee80 osd=0 3 2024-02-23T00:47:14.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.526+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac05ee80 osd=0 3 2024-02-23T00:47:14.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.526+0000 7ff7b12ef700 20 client.43276.objecter _get_session s=0x7ff7ac05ee80 osd=0 3 2024-02-23T00:47:14.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.526+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_remove -1 1 2024-02-23T00:47:14.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.526+0000 7ff7b12ef700 20 client.43276.objecter get_session s=0x7ff7ac05ee80 osd=0 3 2024-02-23T00:47:14.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.526+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_assign 0 1 2024-02-23T00:47:14.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.526+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac05ee80 osd=0 4 2024-02-23T00:47:14.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.526+0000 7ff7b12ef700 10 client.43276.objecter _send_command 1 2024-02-23T00:47:14.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.526+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff7ac107050 con 0x7ff7ac0fc2a0 2024-02-23T00:47:14.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.526+0000 7ff7ab7fe700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7ff7ac0fc2a0 0x7ff7ac0fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:14.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.526+0000 7ff793fff700 10 client.43276.objecter ms_handle_connect 0x7ff7ac0fc2a0 2024-02-23T00:47:14.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.526+0000 7ff7ab7fe700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7ff7ac0fc2a0 0x7ff7ac0fc680 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:14.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.527+0000 7ff793fff700 1 -- 172.21.15.47:0/3756679349 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff7ac107050 con 0x7ff7ac0fc2a0 2024-02-23T00:47:14.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.527+0000 7ff793fff700 10 client.43276.objecter ms_dispatch 0x7ff7ac001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:47:14.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.527+0000 7ff793fff700 10 client.43276.objecter _finish_command 1 = system:0 2024-02-23T00:47:14.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.527+0000 7ff793fff700 20 client.43276.objecter put_session s=0x7ff7ac05ee80 osd=0 4 2024-02-23T00:47:14.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.527+0000 7ff793fff700 15 client.43276.objecter _session_command_op_remove 0 1 2024-02-23T00:47:14.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.542+0000 7ff7b12ef700 10 client.43276.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:47:14.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.542+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_assign -1 2 2024-02-23T00:47:14.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.542+0000 7ff7b12ef700 20 client.43276.objecter _get_session s=0x7ff7ac05ee80 osd=0 3 2024-02-23T00:47:14.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.542+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac05ee80 osd=0 3 2024-02-23T00:47:14.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.542+0000 7ff7b12ef700 20 client.43276.objecter _get_session s=0x7ff7ac05ee80 osd=0 3 2024-02-23T00:47:14.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.542+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_remove -1 2 2024-02-23T00:47:14.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.542+0000 7ff7b12ef700 20 client.43276.objecter get_session s=0x7ff7ac05ee80 osd=0 3 2024-02-23T00:47:14.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.542+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_assign 0 2 2024-02-23T00:47:14.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.542+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac05ee80 osd=0 4 2024-02-23T00:47:14.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.542+0000 7ff7b12ef700 10 client.43276.objecter _send_command 2 2024-02-23T00:47:14.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.542+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff7ac0273c0 con 0x7ff7ac0fc2a0 2024-02-23T00:47:14.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.544+0000 7ff793fff700 1 -- 172.21.15.47:0/3756679349 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff7ac0273c0 con 0x7ff7ac0fc2a0 2024-02-23T00:47:14.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.544+0000 7ff793fff700 10 client.43276.objecter ms_dispatch 0x7ff7ac001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:47:14.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.544+0000 7ff793fff700 10 client.43276.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:47:14.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.544+0000 7ff793fff700 20 client.43276.objecter put_session s=0x7ff7ac05ee80 osd=0 4 2024-02-23T00:47:14.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.544+0000 7ff793fff700 15 client.43276.objecter _session_command_op_remove 0 2 2024-02-23T00:47:14.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.544+0000 7ff7b12ef700 10 client.43276.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:14.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.544+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_assign -1 3 2024-02-23T00:47:14.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.544+0000 7ff7b12ef700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7ff7ac0598f0 0x7ff7ac059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:14.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.544+0000 7ff7b12ef700 20 client.43276.objecter _get_session s=0x7ff7ac028710 osd=1 3 2024-02-23T00:47:14.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.544+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac028710 osd=1 3 2024-02-23T00:47:14.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.544+0000 7ff7b12ef700 20 client.43276.objecter _get_session s=0x7ff7ac028710 osd=1 3 2024-02-23T00:47:14.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.544+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_remove -1 3 2024-02-23T00:47:14.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.544+0000 7ff7b12ef700 20 client.43276.objecter get_session s=0x7ff7ac028710 osd=1 3 2024-02-23T00:47:14.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.544+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_assign 1 3 2024-02-23T00:47:14.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.544+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac028710 osd=1 4 2024-02-23T00:47:14.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.544+0000 7ff7b12ef700 10 client.43276.objecter _send_command 3 2024-02-23T00:47:14.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.544+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff7ac11d690 con 0x7ff7ac0598f0 2024-02-23T00:47:14.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.544+0000 7ff7aaffd700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7ff7ac0598f0 0x7ff7ac059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:14.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.544+0000 7ff793fff700 10 client.43276.objecter ms_handle_connect 0x7ff7ac0598f0 2024-02-23T00:47:14.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.544+0000 7ff7aaffd700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7ff7ac0598f0 0x7ff7ac059cd0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:14.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.546+0000 7ff793fff700 1 -- 172.21.15.47:0/3756679349 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff7ac11d690 con 0x7ff7ac0598f0 2024-02-23T00:47:14.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.546+0000 7ff793fff700 10 client.43276.objecter ms_dispatch 0x7ff7ac001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:47:14.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.546+0000 7ff793fff700 10 client.43276.objecter _finish_command 3 = system:0 2024-02-23T00:47:14.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.546+0000 7ff793fff700 20 client.43276.objecter put_session s=0x7ff7ac028710 osd=1 4 2024-02-23T00:47:14.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.546+0000 7ff793fff700 15 client.43276.objecter _session_command_op_remove 1 3 2024-02-23T00:47:14.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.561+0000 7ff7b12ef700 10 client.43276.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:47:14.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.561+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_assign -1 4 2024-02-23T00:47:14.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.561+0000 7ff7b12ef700 20 client.43276.objecter _get_session s=0x7ff7ac028710 osd=1 3 2024-02-23T00:47:14.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.561+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac028710 osd=1 3 2024-02-23T00:47:14.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.561+0000 7ff7b12ef700 20 client.43276.objecter _get_session s=0x7ff7ac028710 osd=1 3 2024-02-23T00:47:14.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.561+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_remove -1 4 2024-02-23T00:47:14.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.561+0000 7ff7b12ef700 20 client.43276.objecter get_session s=0x7ff7ac028710 osd=1 3 2024-02-23T00:47:14.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.561+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_assign 1 4 2024-02-23T00:47:14.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.561+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac028710 osd=1 4 2024-02-23T00:47:14.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.561+0000 7ff7b12ef700 10 client.43276.objecter _send_command 4 2024-02-23T00:47:14.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.561+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff7ac059360 con 0x7ff7ac0598f0 2024-02-23T00:47:14.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.564+0000 7ff793fff700 1 -- 172.21.15.47:0/3756679349 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff7ac059360 con 0x7ff7ac0598f0 2024-02-23T00:47:14.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.564+0000 7ff793fff700 10 client.43276.objecter ms_dispatch 0x7ff7ac001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:47:14.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.564+0000 7ff793fff700 10 client.43276.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:47:14.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.564+0000 7ff793fff700 20 client.43276.objecter put_session s=0x7ff7ac028710 osd=1 4 2024-02-23T00:47:14.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.564+0000 7ff793fff700 15 client.43276.objecter _session_command_op_remove 1 4 2024-02-23T00:47:14.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.564+0000 7ff7b12ef700 10 client.43276.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:14.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.564+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_assign -1 5 2024-02-23T00:47:14.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.564+0000 7ff7b12ef700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7ff7ac028270 0x7ff7ac11c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:14.733 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:47:14.733 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:47:14.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.564+0000 7ff7b12ef700 20 client.43276.objecter _get_session s=0x7ff7ac059770 osd=2 3 2024-02-23T00:47:14.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.564+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac059770 osd=2 3 2024-02-23T00:47:14.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.564+0000 7ff7b12ef700 20 client.43276.objecter _get_session s=0x7ff7ac059770 osd=2 3 2024-02-23T00:47:14.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.564+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_remove -1 5 2024-02-23T00:47:14.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.564+0000 7ff7b12ef700 20 client.43276.objecter get_session s=0x7ff7ac059770 osd=2 3 2024-02-23T00:47:14.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.564+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_assign 2 5 2024-02-23T00:47:14.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.564+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac059770 osd=2 4 2024-02-23T00:47:14.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.564+0000 7ff7b12ef700 10 client.43276.objecter _send_command 5 2024-02-23T00:47:14.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.564+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff7ac11c5a0 con 0x7ff7ac028270 2024-02-23T00:47:14.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.564+0000 7ff7aa7fc700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7ff7ac028270 0x7ff7ac11c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:14.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.565+0000 7ff793fff700 10 client.43276.objecter ms_handle_connect 0x7ff7ac028270 2024-02-23T00:47:14.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.565+0000 7ff7aa7fc700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7ff7ac028270 0x7ff7ac11c0b0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:14.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.566+0000 7ff793fff700 1 -- 172.21.15.47:0/3756679349 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff7ac11c5a0 con 0x7ff7ac028270 2024-02-23T00:47:14.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.566+0000 7ff793fff700 10 client.43276.objecter ms_dispatch 0x7ff7ac001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:47:14.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.566+0000 7ff793fff700 10 client.43276.objecter _finish_command 5 = system:0 2024-02-23T00:47:14.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.566+0000 7ff793fff700 20 client.43276.objecter put_session s=0x7ff7ac059770 osd=2 4 2024-02-23T00:47:14.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.566+0000 7ff793fff700 15 client.43276.objecter _session_command_op_remove 2 5 2024-02-23T00:47:14.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.580+0000 7ff7b12ef700 10 client.43276.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:47:14.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.580+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_assign -1 6 2024-02-23T00:47:14.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.580+0000 7ff7b12ef700 20 client.43276.objecter _get_session s=0x7ff7ac059770 osd=2 3 2024-02-23T00:47:14.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.580+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac059770 osd=2 3 2024-02-23T00:47:14.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.580+0000 7ff7b12ef700 20 client.43276.objecter _get_session s=0x7ff7ac059770 osd=2 3 2024-02-23T00:47:14.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.580+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_remove -1 6 2024-02-23T00:47:14.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.580+0000 7ff7b12ef700 20 client.43276.objecter get_session s=0x7ff7ac059770 osd=2 3 2024-02-23T00:47:14.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.580+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_assign 2 6 2024-02-23T00:47:14.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.580+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac059770 osd=2 4 2024-02-23T00:47:14.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.580+0000 7ff7b12ef700 10 client.43276.objecter _send_command 6 2024-02-23T00:47:14.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.580+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff7ac0277d0 con 0x7ff7ac028270 2024-02-23T00:47:14.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.582+0000 7ff793fff700 1 -- 172.21.15.47:0/3756679349 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff7ac0277d0 con 0x7ff7ac028270 2024-02-23T00:47:14.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.582+0000 7ff793fff700 10 client.43276.objecter ms_dispatch 0x7ff7ac001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:47:14.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.582+0000 7ff793fff700 10 client.43276.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:47:14.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.582+0000 7ff793fff700 20 client.43276.objecter put_session s=0x7ff7ac059770 osd=2 4 2024-02-23T00:47:14.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.582+0000 7ff793fff700 15 client.43276.objecter _session_command_op_remove 2 6 2024-02-23T00:47:14.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.583+0000 7ff7b12ef700 10 client.43276.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:14.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.583+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_assign -1 7 2024-02-23T00:47:14.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.583+0000 7ff7b12ef700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7ff7ac11f810 0x7ff7ac121c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:14.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.583+0000 7ff7b12ef700 20 client.43276.objecter _get_session s=0x7ff7ac027be0 osd=3 3 2024-02-23T00:47:14.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.583+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac027be0 osd=3 3 2024-02-23T00:47:14.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.583+0000 7ff7b12ef700 20 client.43276.objecter _get_session s=0x7ff7ac027be0 osd=3 3 2024-02-23T00:47:14.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.583+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_remove -1 7 2024-02-23T00:47:14.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.583+0000 7ff7b12ef700 20 client.43276.objecter get_session s=0x7ff7ac027be0 osd=3 3 2024-02-23T00:47:14.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.583+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_assign 3 7 2024-02-23T00:47:14.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.583+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac027be0 osd=3 4 2024-02-23T00:47:14.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.583+0000 7ff7b12ef700 10 client.43276.objecter _send_command 7 2024-02-23T00:47:14.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.583+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff7ac124850 con 0x7ff7ac11f810 2024-02-23T00:47:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.583+0000 7ff7ab7fe700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7ff7ac11f810 0x7ff7ac121c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.583+0000 7ff793fff700 10 client.43276.objecter ms_handle_connect 0x7ff7ac11f810 2024-02-23T00:47:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.583+0000 7ff7ab7fe700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7ff7ac11f810 0x7ff7ac121c00 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.584+0000 7ff793fff700 1 -- 172.21.15.47:0/3756679349 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff7ac124850 con 0x7ff7ac11f810 2024-02-23T00:47:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.584+0000 7ff793fff700 10 client.43276.objecter ms_dispatch 0x7ff7ac001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:47:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.584+0000 7ff793fff700 10 client.43276.objecter _finish_command 7 = system:0 2024-02-23T00:47:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.584+0000 7ff793fff700 20 client.43276.objecter put_session s=0x7ff7ac027be0 osd=3 4 2024-02-23T00:47:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.584+0000 7ff793fff700 15 client.43276.objecter _session_command_op_remove 3 7 2024-02-23T00:47:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.599+0000 7ff7b12ef700 10 client.43276.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:47:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.599+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_assign -1 8 2024-02-23T00:47:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.599+0000 7ff7b12ef700 20 client.43276.objecter _get_session s=0x7ff7ac027be0 osd=3 3 2024-02-23T00:47:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.599+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac027be0 osd=3 3 2024-02-23T00:47:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.599+0000 7ff7b12ef700 20 client.43276.objecter _get_session s=0x7ff7ac027be0 osd=3 3 2024-02-23T00:47:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.599+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_remove -1 8 2024-02-23T00:47:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.599+0000 7ff7b12ef700 20 client.43276.objecter get_session s=0x7ff7ac027be0 osd=3 3 2024-02-23T00:47:14.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.599+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_assign 3 8 2024-02-23T00:47:14.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.599+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac027be0 osd=3 4 2024-02-23T00:47:14.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.599+0000 7ff7b12ef700 10 client.43276.objecter _send_command 8 2024-02-23T00:47:14.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.599+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff7ac11c950 con 0x7ff7ac11f810 2024-02-23T00:47:14.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.601+0000 7ff793fff700 1 -- 172.21.15.47:0/3756679349 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff7ac11c950 con 0x7ff7ac11f810 2024-02-23T00:47:14.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.601+0000 7ff793fff700 10 client.43276.objecter ms_dispatch 0x7ff7ac001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:47:14.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.601+0000 7ff793fff700 10 client.43276.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:47:14.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.601+0000 7ff793fff700 20 client.43276.objecter put_session s=0x7ff7ac027be0 osd=3 4 2024-02-23T00:47:14.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.601+0000 7ff793fff700 15 client.43276.objecter _session_command_op_remove 3 8 2024-02-23T00:47:14.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.601+0000 7ff7b12ef700 10 client.43276.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:14.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.601+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_assign -1 9 2024-02-23T00:47:14.739 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:47:14.739 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:47:14.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.601+0000 7ff7b12ef700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7ff7ac11cac0 0x7ff7ac123270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:14.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.601+0000 7ff7b12ef700 20 client.43276.objecter _get_session s=0x7ff7ac11c950 osd=4 3 2024-02-23T00:47:14.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.601+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac11c950 osd=4 3 2024-02-23T00:47:14.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.601+0000 7ff7b12ef700 20 client.43276.objecter _get_session s=0x7ff7ac11c950 osd=4 3 2024-02-23T00:47:14.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.601+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_remove -1 9 2024-02-23T00:47:14.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.601+0000 7ff7b12ef700 20 client.43276.objecter get_session s=0x7ff7ac11c950 osd=4 3 2024-02-23T00:47:14.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.601+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_assign 4 9 2024-02-23T00:47:14.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.601+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac11c950 osd=4 4 2024-02-23T00:47:14.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.601+0000 7ff7b12ef700 10 client.43276.objecter _send_command 9 2024-02-23T00:47:14.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.601+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff7ac11be00 con 0x7ff7ac11cac0 2024-02-23T00:47:14.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.602+0000 7ff7aaffd700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7ff7ac11cac0 0x7ff7ac123270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:14.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.602+0000 7ff793fff700 10 client.43276.objecter ms_handle_connect 0x7ff7ac11cac0 2024-02-23T00:47:14.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.602+0000 7ff7aaffd700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7ff7ac11cac0 0x7ff7ac123270 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:14.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.603+0000 7ff793fff700 1 -- 172.21.15.47:0/3756679349 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff7ac11be00 con 0x7ff7ac11cac0 2024-02-23T00:47:14.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.603+0000 7ff793fff700 10 client.43276.objecter ms_dispatch 0x7ff7ac001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:47:14.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.603+0000 7ff793fff700 10 client.43276.objecter _finish_command 9 = system:0 2024-02-23T00:47:14.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.603+0000 7ff793fff700 20 client.43276.objecter put_session s=0x7ff7ac11c950 osd=4 4 2024-02-23T00:47:14.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.603+0000 7ff793fff700 15 client.43276.objecter _session_command_op_remove 4 9 2024-02-23T00:47:14.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.618+0000 7ff7b12ef700 10 client.43276.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:47:14.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.618+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_assign -1 10 2024-02-23T00:47:14.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.618+0000 7ff7b12ef700 20 client.43276.objecter _get_session s=0x7ff7ac11c950 osd=4 3 2024-02-23T00:47:14.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.618+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac11c950 osd=4 3 2024-02-23T00:47:14.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.618+0000 7ff7b12ef700 20 client.43276.objecter _get_session s=0x7ff7ac11c950 osd=4 3 2024-02-23T00:47:14.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.618+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_remove -1 10 2024-02-23T00:47:14.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.618+0000 7ff7b12ef700 20 client.43276.objecter get_session s=0x7ff7ac11c950 osd=4 3 2024-02-23T00:47:14.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.618+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_assign 4 10 2024-02-23T00:47:14.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.618+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac11c950 osd=4 4 2024-02-23T00:47:14.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.618+0000 7ff7b12ef700 10 client.43276.objecter _send_command 10 2024-02-23T00:47:14.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.618+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff7ac11be00 con 0x7ff7ac11cac0 2024-02-23T00:47:14.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.619+0000 7ff793fff700 1 -- 172.21.15.47:0/3756679349 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff7ac11be00 con 0x7ff7ac11cac0 2024-02-23T00:47:14.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.619+0000 7ff793fff700 10 client.43276.objecter ms_dispatch 0x7ff7ac001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:47:14.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.619+0000 7ff793fff700 10 client.43276.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:47:14.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.619+0000 7ff793fff700 20 client.43276.objecter put_session s=0x7ff7ac11c950 osd=4 4 2024-02-23T00:47:14.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.619+0000 7ff793fff700 15 client.43276.objecter _session_command_op_remove 4 10 2024-02-23T00:47:14.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.620+0000 7ff7b12ef700 10 client.43276.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:14.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.620+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_assign -1 11 2024-02-23T00:47:14.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.620+0000 7ff7b12ef700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7ff7ac123c70 0x7ff7ac1240d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:14.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.620+0000 7ff7b12ef700 20 client.43276.objecter _get_session s=0x7ff7ac11be00 osd=5 3 2024-02-23T00:47:14.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.620+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac11be00 osd=5 3 2024-02-23T00:47:14.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.620+0000 7ff7b12ef700 20 client.43276.objecter _get_session s=0x7ff7ac11be00 osd=5 3 2024-02-23T00:47:14.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.620+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_remove -1 11 2024-02-23T00:47:14.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.620+0000 7ff7b12ef700 20 client.43276.objecter get_session s=0x7ff7ac11be00 osd=5 3 2024-02-23T00:47:14.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.620+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_assign 5 11 2024-02-23T00:47:14.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.620+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac11be00 osd=5 4 2024-02-23T00:47:14.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.620+0000 7ff7b12ef700 10 client.43276.objecter _send_command 11 2024-02-23T00:47:14.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.620+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff7ac12b140 con 0x7ff7ac123c70 2024-02-23T00:47:14.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.620+0000 7ff7aa7fc700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7ff7ac123c70 0x7ff7ac1240d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:14.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.620+0000 7ff793fff700 10 client.43276.objecter ms_handle_connect 0x7ff7ac123c70 2024-02-23T00:47:14.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.620+0000 7ff7aa7fc700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7ff7ac123c70 0x7ff7ac1240d0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:14.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.621+0000 7ff793fff700 1 -- 172.21.15.47:0/3756679349 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff7ac12b140 con 0x7ff7ac123c70 2024-02-23T00:47:14.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.621+0000 7ff793fff700 10 client.43276.objecter ms_dispatch 0x7ff7ac001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:47:14.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.621+0000 7ff793fff700 10 client.43276.objecter _finish_command 11 = system:0 2024-02-23T00:47:14.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.621+0000 7ff793fff700 20 client.43276.objecter put_session s=0x7ff7ac11be00 osd=5 4 2024-02-23T00:47:14.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.621+0000 7ff793fff700 15 client.43276.objecter _session_command_op_remove 5 11 2024-02-23T00:47:14.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.635+0000 7ff7b12ef700 10 client.43276.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:47:14.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.635+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_assign -1 12 2024-02-23T00:47:14.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.635+0000 7ff7b12ef700 20 client.43276.objecter _get_session s=0x7ff7ac11be00 osd=5 3 2024-02-23T00:47:14.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.635+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac11be00 osd=5 3 2024-02-23T00:47:14.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.635+0000 7ff7b12ef700 20 client.43276.objecter _get_session s=0x7ff7ac11be00 osd=5 3 2024-02-23T00:47:14.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.635+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_remove -1 12 2024-02-23T00:47:14.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.635+0000 7ff7b12ef700 20 client.43276.objecter get_session s=0x7ff7ac11be00 osd=5 3 2024-02-23T00:47:14.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.635+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_assign 5 12 2024-02-23T00:47:14.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.635+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac11be00 osd=5 4 2024-02-23T00:47:14.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.635+0000 7ff7b12ef700 10 client.43276.objecter _send_command 12 2024-02-23T00:47:14.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.635+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff7ac1289e0 con 0x7ff7ac123c70 2024-02-23T00:47:14.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.637+0000 7ff793fff700 1 -- 172.21.15.47:0/3756679349 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff7ac1289e0 con 0x7ff7ac123c70 2024-02-23T00:47:14.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.637+0000 7ff793fff700 10 client.43276.objecter ms_dispatch 0x7ff7ac001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:47:14.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.637+0000 7ff793fff700 10 client.43276.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:47:14.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.637+0000 7ff793fff700 20 client.43276.objecter put_session s=0x7ff7ac11be00 osd=5 4 2024-02-23T00:47:14.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.637+0000 7ff793fff700 15 client.43276.objecter _session_command_op_remove 5 12 2024-02-23T00:47:14.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.637+0000 7ff7b12ef700 10 client.43276.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:14.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.637+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_assign -1 13 2024-02-23T00:47:14.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.637+0000 7ff7b12ef700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7ff7ac129060 0x7ff7ac12b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:14.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.637+0000 7ff7b12ef700 20 client.43276.objecter _get_session s=0x7ff7ac1289e0 osd=6 3 2024-02-23T00:47:14.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.637+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac1289e0 osd=6 3 2024-02-23T00:47:14.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.637+0000 7ff7b12ef700 20 client.43276.objecter _get_session s=0x7ff7ac1289e0 osd=6 3 2024-02-23T00:47:14.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.637+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_remove -1 13 2024-02-23T00:47:14.745 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:47:14.745 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:47:14.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.637+0000 7ff7b12ef700 20 client.43276.objecter get_session s=0x7ff7ac1289e0 osd=6 3 2024-02-23T00:47:14.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.637+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_assign 6 13 2024-02-23T00:47:14.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.637+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac1289e0 osd=6 4 2024-02-23T00:47:14.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.637+0000 7ff7b12ef700 10 client.43276.objecter _send_command 13 2024-02-23T00:47:14.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.637+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff7ac12e280 con 0x7ff7ac129060 2024-02-23T00:47:14.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.638+0000 7ff7ab7fe700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7ff7ac129060 0x7ff7ac12b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:14.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.638+0000 7ff793fff700 10 client.43276.objecter ms_handle_connect 0x7ff7ac129060 2024-02-23T00:47:14.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.638+0000 7ff7ab7fe700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7ff7ac129060 0x7ff7ac12b450 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:14.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.639+0000 7ff793fff700 1 -- 172.21.15.47:0/3756679349 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff7ac12e280 con 0x7ff7ac129060 2024-02-23T00:47:14.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.639+0000 7ff793fff700 10 client.43276.objecter ms_dispatch 0x7ff7ac001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:47:14.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.639+0000 7ff793fff700 10 client.43276.objecter _finish_command 13 = system:0 2024-02-23T00:47:14.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.639+0000 7ff793fff700 20 client.43276.objecter put_session s=0x7ff7ac1289e0 osd=6 4 2024-02-23T00:47:14.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.639+0000 7ff793fff700 15 client.43276.objecter _session_command_op_remove 6 13 2024-02-23T00:47:14.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.655+0000 7ff7b12ef700 10 client.43276.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:47:14.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.655+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_assign -1 14 2024-02-23T00:47:14.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.655+0000 7ff7b12ef700 20 client.43276.objecter _get_session s=0x7ff7ac1289e0 osd=6 3 2024-02-23T00:47:14.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.655+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac1289e0 osd=6 3 2024-02-23T00:47:14.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.655+0000 7ff7b12ef700 20 client.43276.objecter _get_session s=0x7ff7ac1289e0 osd=6 3 2024-02-23T00:47:14.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.655+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_remove -1 14 2024-02-23T00:47:14.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.655+0000 7ff7b12ef700 20 client.43276.objecter get_session s=0x7ff7ac1289e0 osd=6 3 2024-02-23T00:47:14.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.655+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_assign 6 14 2024-02-23T00:47:14.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.655+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac1289e0 osd=6 4 2024-02-23T00:47:14.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.655+0000 7ff7b12ef700 10 client.43276.objecter _send_command 14 2024-02-23T00:47:14.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.655+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff7ac12e240 con 0x7ff7ac129060 2024-02-23T00:47:14.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.657+0000 7ff793fff700 1 -- 172.21.15.47:0/3756679349 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff7ac12e240 con 0x7ff7ac129060 2024-02-23T00:47:14.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.657+0000 7ff793fff700 10 client.43276.objecter ms_dispatch 0x7ff7ac001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:47:14.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.657+0000 7ff793fff700 10 client.43276.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:47:14.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.657+0000 7ff793fff700 20 client.43276.objecter put_session s=0x7ff7ac1289e0 osd=6 4 2024-02-23T00:47:14.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.657+0000 7ff793fff700 15 client.43276.objecter _session_command_op_remove 6 14 2024-02-23T00:47:14.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.657+0000 7ff7b12ef700 10 client.43276.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:14.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.657+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_assign -1 15 2024-02-23T00:47:14.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.657+0000 7ff7b12ef700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7ff7ac12bf40 0x7ff7ac12c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:14.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.657+0000 7ff7b12ef700 20 client.43276.objecter _get_session s=0x7ff7ac12e240 osd=7 3 2024-02-23T00:47:14.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.657+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac12e240 osd=7 3 2024-02-23T00:47:14.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.657+0000 7ff7b12ef700 20 client.43276.objecter _get_session s=0x7ff7ac12e240 osd=7 3 2024-02-23T00:47:14.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.657+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_remove -1 15 2024-02-23T00:47:14.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.657+0000 7ff7b12ef700 20 client.43276.objecter get_session s=0x7ff7ac12e240 osd=7 3 2024-02-23T00:47:14.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.657+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_assign 7 15 2024-02-23T00:47:14.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.657+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac12e240 osd=7 4 2024-02-23T00:47:14.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.657+0000 7ff7b12ef700 10 client.43276.objecter _send_command 15 2024-02-23T00:47:14.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.657+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff7ac12e030 con 0x7ff7ac12bf40 2024-02-23T00:47:14.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.657+0000 7ff7aaffd700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7ff7ac12bf40 0x7ff7ac12c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:14.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.658+0000 7ff793fff700 10 client.43276.objecter ms_handle_connect 0x7ff7ac12bf40 2024-02-23T00:47:14.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.658+0000 7ff7aaffd700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7ff7ac12bf40 0x7ff7ac12c3c0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:14.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.658+0000 7ff793fff700 1 -- 172.21.15.47:0/3756679349 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff79c02b040 con 0x7ff7ac12bf40 2024-02-23T00:47:14.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.658+0000 7ff793fff700 10 client.43276.objecter ms_dispatch 0x7ff7ac001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:47:14.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.658+0000 7ff793fff700 10 client.43276.objecter _finish_command 15 = system:0 2024-02-23T00:47:14.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.658+0000 7ff793fff700 20 client.43276.objecter put_session s=0x7ff7ac12e240 osd=7 4 2024-02-23T00:47:14.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.658+0000 7ff793fff700 15 client.43276.objecter _session_command_op_remove 7 15 2024-02-23T00:47:14.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.674+0000 7ff7b12ef700 10 client.43276.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:47:14.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.674+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_assign -1 16 2024-02-23T00:47:14.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.674+0000 7ff7b12ef700 20 client.43276.objecter _get_session s=0x7ff7ac12e240 osd=7 3 2024-02-23T00:47:14.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.674+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac12e240 osd=7 3 2024-02-23T00:47:14.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.674+0000 7ff7b12ef700 20 client.43276.objecter _get_session s=0x7ff7ac12e240 osd=7 3 2024-02-23T00:47:14.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.674+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_remove -1 16 2024-02-23T00:47:14.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.674+0000 7ff7b12ef700 20 client.43276.objecter get_session s=0x7ff7ac12e240 osd=7 3 2024-02-23T00:47:14.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.674+0000 7ff7b12ef700 15 client.43276.objecter _session_command_op_assign 7 16 2024-02-23T00:47:14.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.674+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac12e240 osd=7 4 2024-02-23T00:47:14.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.674+0000 7ff7b12ef700 10 client.43276.objecter _send_command 16 2024-02-23T00:47:14.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.674+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff7ac130430 con 0x7ff7ac12bf40 2024-02-23T00:47:14.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.677+0000 7ff793fff700 1 -- 172.21.15.47:0/3756679349 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff7ac130430 con 0x7ff7ac12bf40 2024-02-23T00:47:14.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.677+0000 7ff793fff700 10 client.43276.objecter ms_dispatch 0x7ff7ac001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:47:14.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.677+0000 7ff793fff700 10 client.43276.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:47:14.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.677+0000 7ff793fff700 20 client.43276.objecter put_session s=0x7ff7ac12e240 osd=7 4 2024-02-23T00:47:14.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.677+0000 7ff793fff700 15 client.43276.objecter _session_command_op_remove 7 16 2024-02-23T00:47:14.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 10 librados: watch_flush enter 2024-02-23T00:47:14.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 10 librados: watch_flush exit 2024-02-23T00:47:14.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 10 client.43276.objecter close_session for osd.0 2024-02-23T00:47:14.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7ff7ac0fc2a0 msgr2=0x7ff7ac0fc680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:14.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7ff7ac0fc2a0 0x7ff7ac0fc680 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:14.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac05ee80 osd=0 1 2024-02-23T00:47:14.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 10 client.43276.objecter close_session for osd.1 2024-02-23T00:47:14.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7ff7ac0598f0 msgr2=0x7ff7ac059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:14.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7ff7ac0598f0 0x7ff7ac059cd0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:14.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac028710 osd=1 1 2024-02-23T00:47:14.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 10 client.43276.objecter close_session for osd.2 2024-02-23T00:47:14.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7ff7ac028270 msgr2=0x7ff7ac11c0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:14.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7ff7ac028270 0x7ff7ac11c0b0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:14.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac059770 osd=2 1 2024-02-23T00:47:14.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 10 client.43276.objecter close_session for osd.3 2024-02-23T00:47:14.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7ff7ac11f810 msgr2=0x7ff7ac121c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:14.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7ff7ac11f810 0x7ff7ac121c00 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:14.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac027be0 osd=3 1 2024-02-23T00:47:14.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 10 client.43276.objecter close_session for osd.4 2024-02-23T00:47:14.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7ab7fe700 1 -- 172.21.15.47:0/3756679349 reap_dead start 2024-02-23T00:47:14.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7ff7ac11cac0 msgr2=0x7ff7ac123270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:14.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7ff7ac11cac0 0x7ff7ac123270 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:14.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7ab7fe700 1 -- 172.21.15.47:0/3756679349 reap_dead start 2024-02-23T00:47:14.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac11c950 osd=4 1 2024-02-23T00:47:14.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 10 client.43276.objecter close_session for osd.5 2024-02-23T00:47:14.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7ff7ac123c70 msgr2=0x7ff7ac1240d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:14.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7ff7ac123c70 0x7ff7ac1240d0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:14.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac11be00 osd=5 1 2024-02-23T00:47:14.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 10 client.43276.objecter close_session for osd.6 2024-02-23T00:47:14.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7ff7ac129060 msgr2=0x7ff7ac12b450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:14.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7ff7ac129060 0x7ff7ac12b450 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:14.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac1289e0 osd=6 1 2024-02-23T00:47:14.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 10 client.43276.objecter close_session for osd.7 2024-02-23T00:47:14.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7ff7ac12bf40 msgr2=0x7ff7ac12c3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:14.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7ff7ac12bf40 0x7ff7ac12c3c0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:14.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 20 client.43276.objecter put_session s=0x7ff7ac12e240 osd=7 1 2024-02-23T00:47:14.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 20 client.43276.objecter shutdown clearing up homeless session... 2024-02-23T00:47:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 10 client.43276.objecter successfully canceled tick 2024-02-23T00:47:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff794062cc0 msgr2=0x7ff794065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff794062cc0 0x7ff794065180 secure :-1 s=READY pgs=3044 cs=0 l=1 rev1=1 rx=0x7ff79c004e80 tx=0x7ff79c00b040).stop 2024-02-23T00:47:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7ac0f9640 msgr2=0x7ff7ac0f7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 1 --2- 172.21.15.47:0/3756679349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7ac0f9640 0x7ff7ac0f7210 secure :-1 s=READY pgs=3975 cs=0 l=1 rev1=1 rx=0x7ff7a0004920 tx=0x7ff7a0009cb0).stop 2024-02-23T00:47:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7ab7fe700 1 -- 172.21.15.47:0/3756679349 reap_dead start 2024-02-23T00:47:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 shutdown_connections 2024-02-23T00:47:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 >> 172.21.15.47:0/3756679349 conn(0x7ff7ac0f2820 msgr2=0x7ff7ac0f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 shutdown_connections 2024-02-23T00:47:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 1 -- 172.21.15.47:0/3756679349 wait complete. 2024-02-23T00:47:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:14.678+0000 7ff7b12ef700 1 librados: shutdown 2024-02-23T00:47:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:47:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:47:14.754 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:47:14.754 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:47:14.754 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:47:14.754 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:47:14.754 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:47:14.754 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:47:14.754 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:14.755 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:47:14.755 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:47:14.755 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:47:14.856 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:47:14.884 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:14.923 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:47:14.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:12.797294+0000 osd.0 (osd.0) 145 : cluster [DBG] 5.3 scrub starts 2024-02-23T00:47:14.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:12.798309+0000 osd.0 (osd.0) 146 : cluster [DBG] 5.3 scrub ok 2024-02-23T00:47:14.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:13.270265+0000 mgr.y (mgr.14517) 343 : cluster [DBG] pgmap v517: 218 pgs: 11 creating+peering, 21 unknown, 186 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-23T00:47:14.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:14.525946+0000 mon.b (mon.1) 377 : audit [DBG] from='client.? 172.21.15.47:0/3756679349' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:47:14.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:14.609553+0000 mon.c (mon.2) 273 : audit [INF] from='client.? 172.21.15.47:0/2923629940' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "12"}]: dispatch 2024-02-23T00:47:14.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:14.639469+0000 mon.a (mon.0) 2325 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:47:14.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:14.639565+0000 mon.a (mon.0) 2326 : cluster [DBG] osdmap e362: 8 total, 8 up, 8 in 2024-02-23T00:47:14.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:14.640403+0000 mon.a (mon.0) 2327 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "12"}]: dispatch 2024-02-23T00:47:14.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:14.640715+0000 mon.a (mon.0) 2328 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-47"}]: dispatch 2024-02-23T00:47:15.008 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:47:15.011 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:15.011 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:47:15.112 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:47:15.140 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:15.162 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:47:15.166 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:15.167 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:47:15.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:15.166+0000 7f9a0b7fe700 1 -- 172.21.15.47:0/2923629940 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "12"}]=0 set pool 6 pg_num to 12 v363) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9a14108b50 con 0x7f9a140f8420 2024-02-23T00:47:15.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:15.170+0000 7f9a1c6d2700 10 librados: watch_flush enter 2024-02-23T00:47:15.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:15.170+0000 7f9a1c6d2700 10 librados: watch_flush exit 2024-02-23T00:47:15.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:15.170+0000 7f9a1c6d2700 20 client.41213.objecter shutdown clearing up homeless session... 2024-02-23T00:47:15.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:15.170+0000 7f9a1c6d2700 10 client.41213.objecter successfully canceled tick 2024-02-23T00:47:15.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:15.170+0000 7f9a1c6d2700 1 -- 172.21.15.47:0/2923629940 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9a00062cc0 msgr2=0x7f9a00065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:15.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:15.170+0000 7f9a1c6d2700 1 --2- 172.21.15.47:0/2923629940 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9a00062cc0 0x7f9a00065180 secure :-1 s=READY pgs=3043 cs=0 l=1 rev1=1 rx=0x7f9a141096c0 tx=0x7f9a0400b040).stop 2024-02-23T00:47:15.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:15.170+0000 7f9a1c6d2700 1 -- 172.21.15.47:0/2923629940 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9a140f8420 msgr2=0x7f9a1410dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:15.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:15.170+0000 7f9a1c6d2700 1 --2- 172.21.15.47:0/2923629940 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9a140f8420 0x7f9a1410dd80 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 rx=0x7f9a1000e970 tx=0x7f9a1000b040).stop 2024-02-23T00:47:15.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:15.170+0000 7f9a1c6d2700 1 -- 172.21.15.47:0/2923629940 shutdown_connections 2024-02-23T00:47:15.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:15.170+0000 7f9a1c6d2700 1 --2- 172.21.15.47:0/2923629940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a140fe510 0x7f9a14107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:15.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:15.170+0000 7f9a1c6d2700 1 --2- 172.21.15.47:0/2923629940 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9a00062cc0 0x7f9a00065180 unknown :-1 s=CLOSED pgs=3043 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:15.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:15.170+0000 7f9a1c6d2700 1 --2- 172.21.15.47:0/2923629940 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9a140f8420 0x7f9a1410dd80 unknown :-1 s=CLOSED pgs=2640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:15.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:15.170+0000 7f9a1c6d2700 1 --2- 172.21.15.47:0/2923629940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a140fdbb0 0x7f9a1410e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:47:15.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:15.170+0000 7f9a1c6d2700 1 -- 172.21.15.47:0/2923629940 >> 172.21.15.47:0/2923629940 conn(0x7f9a140f3800 msgr2=0x7f9a140f5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:15.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:15.170+0000 7f9a1c6d2700 1 -- 172.21.15.47:0/2923629940 shutdown_connections 2024-02-23T00:47:15.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:15.171+0000 7f9a1c6d2700 1 -- 172.21.15.47:0/2923629940 wait complete. 2024-02-23T00:47:15.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:15.176+0000 7f9a1c6d2700 1 librados: shutdown 2024-02-23T00:47:15.179 INFO:teuthology.orchestra.run.smithi047.stderr:set pool 6 pg_num to 12 2024-02-23T00:47:15.267 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:47:15.297 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:15.645 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:47:15.647 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:15.648 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:47:15.749 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:47:15.779 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:15.902 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:47:15.904 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:13.772401+0000 osd.0 (osd.0) 147 : cluster [DBG] 5.5 scrub starts 2024-02-23T00:47:15.905 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:13.773258+0000 osd.0 (osd.0) 148 : cluster [DBG] 5.5 scrub ok 2024-02-23T00:47:15.905 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:14.326488+0000 osd.4 (osd.4) 147 : cluster [DBG] 4.1a scrub starts 2024-02-23T00:47:15.905 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:14.328234+0000 osd.4 (osd.4) 148 : cluster [DBG] 4.1a scrub ok 2024-02-23T00:47:15.905 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:14.525077+0000 osd.7 (osd.7) 181 : cluster [DBG] 1.0 scrub starts 2024-02-23T00:47:15.905 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:14.531125+0000 osd.7 (osd.7) 182 : cluster [DBG] 1.0 scrub ok 2024-02-23T00:47:15.905 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:15.164073+0000 mon.a (mon.0) 2329 : audit [INF] from='client.? 172.21.15.47:0/526847179' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi047-82386-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi047-82386-61"}]': finished 2024-02-23T00:47:15.905 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:15.164170+0000 mon.a (mon.0) 2330 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "12"}]': finished 2024-02-23T00:47:15.905 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:15.164253+0000 mon.a (mon.0) 2331 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-47"}]': finished 2024-02-23T00:47:15.905 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:15.164308+0000 mon.a (mon.0) 2332 : cluster [DBG] osdmap e363: 8 total, 8 up, 8 in 2024-02-23T00:47:15.905 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:15.166842+0000 mon.a (mon.0) 2333 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-47", "mode": "readproxy"}]: dispatch 2024-02-23T00:47:15.905 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:15.271846+0000 mon.a (mon.0) 2334 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num_actual", "val": "12"}]: dispatch 2024-02-23T00:47:15.906 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:13.772401+0000 osd.0 (osd.0) 147 : cluster [DBG] 5.5 scrub starts 2024-02-23T00:47:15.906 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:13.773258+0000 osd.0 (osd.0) 148 : cluster [DBG] 5.5 scrub ok 2024-02-23T00:47:15.906 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:14.326488+0000 osd.4 (osd.4) 147 : cluster [DBG] 4.1a scrub starts 2024-02-23T00:47:15.906 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:14.328234+0000 osd.4 (osd.4) 148 : cluster [DBG] 4.1a scrub ok 2024-02-23T00:47:15.906 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:14.525077+0000 osd.7 (osd.7) 181 : cluster [DBG] 1.0 scrub starts 2024-02-23T00:47:15.906 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:14.531125+0000 osd.7 (osd.7) 182 : cluster [DBG] 1.0 scrub ok 2024-02-23T00:47:15.906 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:15.164073+0000 mon.a (mon.0) 2329 : audit [INF] from='client.? 172.21.15.47:0/526847179' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi047-82386-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi047-82386-61"}]': finished 2024-02-23T00:47:15.906 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:15.164170+0000 mon.a (mon.0) 2330 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "12"}]': finished 2024-02-23T00:47:15.906 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:15.164253+0000 mon.a (mon.0) 2331 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-47"}]': finished 2024-02-23T00:47:15.906 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:15.164308+0000 mon.a (mon.0) 2332 : cluster [DBG] osdmap e363: 8 total, 8 up, 8 in 2024-02-23T00:47:15.906 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:15.166842+0000 mon.a (mon.0) 2333 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-47", "mode": "readproxy"}]: dispatch 2024-02-23T00:47:15.906 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:15.271846+0000 mon.a (mon.0) 2334 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num_actual", "val": "12"}]: dispatch 2024-02-23T00:47:15.907 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:15.907 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:47:15.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:13.772401+0000 osd.0 (osd.0) 147 : cluster [DBG] 5.5 scrub starts 2024-02-23T00:47:15.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:13.773258+0000 osd.0 (osd.0) 148 : cluster [DBG] 5.5 scrub ok 2024-02-23T00:47:15.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:14.326488+0000 osd.4 (osd.4) 147 : cluster [DBG] 4.1a scrub starts 2024-02-23T00:47:15.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:14.328234+0000 osd.4 (osd.4) 148 : cluster [DBG] 4.1a scrub ok 2024-02-23T00:47:15.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:14.525077+0000 osd.7 (osd.7) 181 : cluster [DBG] 1.0 scrub starts 2024-02-23T00:47:15.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:14.531125+0000 osd.7 (osd.7) 182 : cluster [DBG] 1.0 scrub ok 2024-02-23T00:47:15.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:15.164073+0000 mon.a (mon.0) 2329 : audit [INF] from='client.? 172.21.15.47:0/526847179' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi047-82386-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi047-82386-61"}]': finished 2024-02-23T00:47:15.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:15.164170+0000 mon.a (mon.0) 2330 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "12"}]': finished 2024-02-23T00:47:15.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:15.164253+0000 mon.a (mon.0) 2331 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-47"}]': finished 2024-02-23T00:47:15.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:15.164308+0000 mon.a (mon.0) 2332 : cluster [DBG] osdmap e363: 8 total, 8 up, 8 in 2024-02-23T00:47:15.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:15.166842+0000 mon.a (mon.0) 2333 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-47", "mode": "readproxy"}]: dispatch 2024-02-23T00:47:15.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:15.271846+0000 mon.a (mon.0) 2334 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num_actual", "val": "12"}]: dispatch 2024-02-23T00:47:16.008 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:47:16.037 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:16.076 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:47:16.078 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:16.078 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:47:16.179 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:47:16.209 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:16.230 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:47:16.232 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:16.233 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:47:16.333 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:47:16.362 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:16.615 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:47:16.615 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:47:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:15.235962+0000 osd.5 (osd.5) 152 : cluster [DBG] 4.15 scrub starts 2024-02-23T00:47:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:15.239424+0000 osd.5 (osd.5) 153 : cluster [DBG] 4.15 scrub ok 2024-02-23T00:47:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:15.270768+0000 mgr.y (mgr.14517) 344 : cluster [DBG] pgmap v520: 226 pgs: 11 creating+peering, 29 unknown, 186 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-23T00:47:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:15.356076+0000 osd.4 (osd.4) 149 : cluster [DBG] 4.d scrub starts 2024-02-23T00:47:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:15.357871+0000 osd.4 (osd.4) 150 : cluster [DBG] 4.d scrub ok 2024-02-23T00:47:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:15.412681+0000 osd.1 (osd.1) 159 : cluster [DBG] 62.10 scrub starts 2024-02-23T00:47:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:15.414413+0000 osd.1 (osd.1) 160 : cluster [DBG] 62.10 scrub ok 2024-02-23T00:47:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:16.016279+0000 osd.3 (osd.3) 173 : cluster [DBG] 2.1b scrub starts 2024-02-23T00:47:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:16.017204+0000 osd.3 (osd.3) 174 : cluster [DBG] 2.1b scrub ok 2024-02-23T00:47:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:16.161182+0000 mon.a (mon.0) 2335 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:47:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:16.165375+0000 mon.a (mon.0) 2336 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-47", "mode": "readproxy"}]': finished 2024-02-23T00:47:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:16.165577+0000 mon.a (mon.0) 2337 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num_actual", "val": "12"}]': finished 2024-02-23T00:47:16.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:16.165645+0000 mon.a (mon.0) 2338 : cluster [DBG] osdmap e364: 8 total, 8 up, 8 in 2024-02-23T00:47:17.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:15.235962+0000 osd.5 (osd.5) 152 : cluster [DBG] 4.15 scrub starts 2024-02-23T00:47:17.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:15.239424+0000 osd.5 (osd.5) 153 : cluster [DBG] 4.15 scrub ok 2024-02-23T00:47:17.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:15.270768+0000 mgr.y (mgr.14517) 344 : cluster [DBG] pgmap v520: 226 pgs: 11 creating+peering, 29 unknown, 186 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-23T00:47:17.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:15.356076+0000 osd.4 (osd.4) 149 : cluster [DBG] 4.d scrub starts 2024-02-23T00:47:17.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:15.357871+0000 osd.4 (osd.4) 150 : cluster [DBG] 4.d scrub ok 2024-02-23T00:47:17.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:15.412681+0000 osd.1 (osd.1) 159 : cluster [DBG] 62.10 scrub starts 2024-02-23T00:47:17.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:15.414413+0000 osd.1 (osd.1) 160 : cluster [DBG] 62.10 scrub ok 2024-02-23T00:47:17.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:16.016279+0000 osd.3 (osd.3) 173 : cluster [DBG] 2.1b scrub starts 2024-02-23T00:47:17.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:16.017204+0000 osd.3 (osd.3) 174 : cluster [DBG] 2.1b scrub ok 2024-02-23T00:47:17.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:16.161182+0000 mon.a (mon.0) 2335 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:47:17.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:16.165375+0000 mon.a (mon.0) 2336 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-47", "mode": "readproxy"}]': finished 2024-02-23T00:47:17.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:16.165577+0000 mon.a (mon.0) 2337 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num_actual", "val": "12"}]': finished 2024-02-23T00:47:17.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:16.165645+0000 mon.a (mon.0) 2338 : cluster [DBG] osdmap e364: 8 total, 8 up, 8 in 2024-02-23T00:47:17.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:15.235962+0000 osd.5 (osd.5) 152 : cluster [DBG] 4.15 scrub starts 2024-02-23T00:47:17.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:15.239424+0000 osd.5 (osd.5) 153 : cluster [DBG] 4.15 scrub ok 2024-02-23T00:47:17.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:15.270768+0000 mgr.y (mgr.14517) 344 : cluster [DBG] pgmap v520: 226 pgs: 11 creating+peering, 29 unknown, 186 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-23T00:47:17.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:15.356076+0000 osd.4 (osd.4) 149 : cluster [DBG] 4.d scrub starts 2024-02-23T00:47:17.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:15.357871+0000 osd.4 (osd.4) 150 : cluster [DBG] 4.d scrub ok 2024-02-23T00:47:17.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:15.412681+0000 osd.1 (osd.1) 159 : cluster [DBG] 62.10 scrub starts 2024-02-23T00:47:17.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:15.414413+0000 osd.1 (osd.1) 160 : cluster [DBG] 62.10 scrub ok 2024-02-23T00:47:17.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:16.016279+0000 osd.3 (osd.3) 173 : cluster [DBG] 2.1b scrub starts 2024-02-23T00:47:17.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:16.017204+0000 osd.3 (osd.3) 174 : cluster [DBG] 2.1b scrub ok 2024-02-23T00:47:17.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:16.161182+0000 mon.a (mon.0) 2335 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:47:17.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:16.165375+0000 mon.a (mon.0) 2336 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-47", "mode": "readproxy"}]': finished 2024-02-23T00:47:17.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:16.165577+0000 mon.a (mon.0) 2337 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num_actual", "val": "12"}]': finished 2024-02-23T00:47:17.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:16.165645+0000 mon.a (mon.0) 2338 : cluster [DBG] osdmap e364: 8 total, 8 up, 8 in 2024-02-23T00:47:17.212 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:47:17.215 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:17.215 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:47:17.316 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:47:17.318 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.0 dump_ops_in_flight 2024-02-23T00:47:17.386 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:17.405 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:47:17.407 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:17.408 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:47:17.509 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:47:17.537 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:17.642 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:47:17.744 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:47:17.747 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:17.748 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:47:17.849 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:47:17.877 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:17.897 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:47:17.900 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:17.900 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:47:18.001 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:47:18.028 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:18.048 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:47:18.050 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:18.051 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:47:18.151 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:47:18.186 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:18.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.243+0000 7fa6e3d7a700 1 -- 172.21.15.47:0/755211352 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa6dc0fa380 msgr2=0x7fa6dc0f7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:18.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.243+0000 7fa6e3d7a700 1 --2- 172.21.15.47:0/755211352 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa6dc0fa380 0x7fa6dc0f7d60 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 rx=0x7fa6d80047b0 tx=0x7fa6d8009cb0).stop 2024-02-23T00:47:18.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.243+0000 7fa6e3d7a700 1 -- 172.21.15.47:0/755211352 shutdown_connections 2024-02-23T00:47:18.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.243+0000 7fa6e3d7a700 1 --2- 172.21.15.47:0/755211352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6dc0f8440 0x7fa6dc104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:18.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.243+0000 7fa6e3d7a700 1 --2- 172.21.15.47:0/755211352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6dc0f9ab0 0x7fa6dc0f9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:18.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.243+0000 7fa6e3d7a700 1 --2- 172.21.15.47:0/755211352 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa6dc0fa380 0x7fa6dc0f7d60 unknown :-1 s=CLOSED pgs=2642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:18.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.243+0000 7fa6e3d7a700 1 -- 172.21.15.47:0/755211352 >> 172.21.15.47:0/755211352 conn(0x7fa6dc0f3800 msgr2=0x7fa6dc0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:18.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.243+0000 7fa6e3d7a700 1 -- 172.21.15.47:0/755211352 shutdown_connections 2024-02-23T00:47:18.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.243+0000 7fa6e3d7a700 1 -- 172.21.15.47:0/755211352 wait complete. 2024-02-23T00:47:18.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.243+0000 7fa6e3d7a700 1 librados: starting msgr at 2024-02-23T00:47:18.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.243+0000 7fa6e3d7a700 1 librados: starting objecter 2024-02-23T00:47:18.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.243+0000 7fa6e3d7a700 1 Processor -- start 2024-02-23T00:47:18.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.243+0000 7fa6e3d7a700 1 -- start start 2024-02-23T00:47:18.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.243+0000 7fa6e3d7a700 1 librados: setting wanted keys 2024-02-23T00:47:18.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.243+0000 7fa6e3d7a700 1 librados: calling monclient init 2024-02-23T00:47:18.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.244+0000 7fa6e3d7a700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa6dc0f8440 0x7fa6dc06b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:18.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.244+0000 7fa6e3d7a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6dc0f9ab0 0x7fa6dc06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:18.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.244+0000 7fa6e3d7a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6dc0fa380 0x7fa6dc06bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:18.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.244+0000 7fa6e3d7a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc116490 con 0x7fa6dc0f9ab0 2024-02-23T00:47:18.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.244+0000 7fa6e3d7a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc116600 con 0x7fa6dc0fa380 2024-02-23T00:47:18.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.244+0000 7fa6e3d7a700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc1167c0 con 0x7fa6dc0f8440 2024-02-23T00:47:18.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.244+0000 7fa6e1b16700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa6dc0f8440 0x7fa6dc06b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:18.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.244+0000 7fa6e1315700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6dc0f9ab0 0x7fa6dc06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:18.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.244+0000 7fa6e1b16700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa6dc0f8440 0x7fa6dc06b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:33072/0 (socket says 172.21.15.47:33072) 2024-02-23T00:47:18.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.244+0000 7fa6e1315700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6dc0f9ab0 0x7fa6dc06b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48018/0 (socket says 172.21.15.47:48018) 2024-02-23T00:47:18.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.244+0000 7fa6e1b16700 1 -- 172.21.15.47:0/3048728256 learned_addr learned my addr 172.21.15.47:0/3048728256 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:47:18.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.244+0000 7fa6e2317700 1 --2- 172.21.15.47:0/3048728256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6dc0fa380 0x7fa6dc06bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:18.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.244+0000 7fa6e1b16700 1 -- 172.21.15.47:0/3048728256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6dc0fa380 msgr2=0x7fa6dc06bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:18.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.244+0000 7fa6e1b16700 1 --2- 172.21.15.47:0/3048728256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6dc0fa380 0x7fa6dc06bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:18.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.244+0000 7fa6e1b16700 1 -- 172.21.15.47:0/3048728256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6dc0f9ab0 msgr2=0x7fa6dc06b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:18.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.244+0000 7fa6e1b16700 1 --2- 172.21.15.47:0/3048728256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6dc0f9ab0 0x7fa6dc06b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:18.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.244+0000 7fa6e1b16700 1 -- 172.21.15.47:0/3048728256 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d40097c0 con 0x7fa6dc0f8440 2024-02-23T00:47:18.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.244+0000 7fa6e1315700 1 --2- 172.21.15.47:0/3048728256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6dc0f9ab0 0x7fa6dc06b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:47:18.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.244+0000 7fa6e1b16700 1 --2- 172.21.15.47:0/3048728256 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa6dc0f8440 0x7fa6dc06b0d0 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 rx=0x7fa6cc0041d0 tx=0x7fa6cc009f60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:18.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.244+0000 7fa6d2ffd700 10 client.?.objecter ms_handle_connect 0x7fa6dc0f8440 2024-02-23T00:47:18.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.244+0000 7fa6d2ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:47:18.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.244+0000 7fa6d2ffd700 1 -- 172.21.15.47:0/3048728256 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc007b40 con 0x7fa6dc0f8440 2024-02-23T00:47:18.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.244+0000 7fa6d2ffd700 1 -- 172.21.15.47:0/3048728256 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fa6cc00f040 con 0x7fa6dc0f8440 2024-02-23T00:47:18.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.244+0000 7fa6e3d7a700 1 -- 172.21.15.47:0/3048728256 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d8005040 con 0x7fa6dc0f8440 2024-02-23T00:47:18.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.244+0000 7fa6d2ffd700 1 -- 172.21.15.47:0/3048728256 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc014730 con 0x7fa6dc0f8440 2024-02-23T00:47:18.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.245+0000 7fa6e3d7a700 10 client.41225.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:47:18.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.245+0000 7fa6e3d7a700 1 -- 172.21.15.47:0/3048728256 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6dc116ed0 con 0x7fa6dc0f8440 2024-02-23T00:47:18.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.245+0000 7fa6e3d7a700 1 librados: init done 2024-02-23T00:47:18.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.245+0000 7fa6e3d7a700 1 -- 172.21.15.47:0/3048728256 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6dc05ee80 con 0x7fa6dc0f8440 2024-02-23T00:47:18.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.246+0000 7fa6d2ffd700 1 -- 172.21.15.47:0/3048728256 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fa6cc009110 con 0x7fa6dc0f8440 2024-02-23T00:47:18.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.246+0000 7fa6d2ffd700 1 --2- 172.21.15.47:0/3048728256 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa6c8062cc0 0x7fa6c8065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:18.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.247+0000 7fa6e1315700 1 --2- 172.21.15.47:0/3048728256 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa6c8062cc0 0x7fa6c8065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:18.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.247+0000 7fa6d2ffd700 1 -- 172.21.15.47:0/3048728256 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(366..366 src has 48..366) v4 ==== 9325+0+0 (secure 0 0 0) 0x7fa6cc07d6c0 con 0x7fa6dc0f8440 2024-02-23T00:47:18.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.247+0000 7fa6d2ffd700 10 client.41225.objecter ms_dispatch 0x7fa6dc001550 osd_map(366..366 src has 48..366) v4 2024-02-23T00:47:18.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.247+0000 7fa6d2ffd700 3 client.41225.objecter handle_osd_map got epochs [366,366] > 0 2024-02-23T00:47:18.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.247+0000 7fa6d2ffd700 3 client.41225.objecter handle_osd_map decoding full epoch 366 2024-02-23T00:47:18.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.247+0000 7fa6d2ffd700 20 client.41225.objecter dump_active .. 0 homeless 2024-02-23T00:47:18.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.247+0000 7fa6d2ffd700 10 client.41225.objecter ms_handle_connect 0x7fa6c8062cc0 2024-02-23T00:47:18.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.247+0000 7fa6e1315700 1 --2- 172.21.15.47:0/3048728256 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa6c8062cc0 0x7fa6c8065180 secure :-1 s=READY pgs=3045 cs=0 l=1 rev1=1 rx=0x7fa6d80044a0 tx=0x7fa6d8009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:18.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.251+0000 7fa6d2ffd700 1 -- 172.21.15.47:0/3048728256 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fa6cc04ccd0 con 0x7fa6dc0f8440 2024-02-23T00:47:18.286 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:47:18.289 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:47:18.293 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:16.456855+0000 osd.1 (osd.1) 161 : cluster [DBG] 62.1 scrub starts 2024-02-23T00:47:18.293 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:16.460306+0000 osd.1 (osd.1) 162 : cluster [DBG] 62.1 scrub ok 2024-02-23T00:47:18.293 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:16.729472+0000 osd.2 (osd.2) 155 : cluster [DBG] 4.14 scrub starts 2024-02-23T00:47:18.293 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:16.731624+0000 osd.2 (osd.2) 156 : cluster [DBG] 4.14 scrub ok 2024-02-23T00:47:18.293 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:17.167169+0000 mon.a (mon.0) 2339 : cluster [DBG] osdmap e365: 8 total, 8 up, 8 in 2024-02-23T00:47:18.293 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:17.170103+0000 mon.a (mon.0) 2340 : audit [INF] from='client.? 172.21.15.47:0/526847179' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi047-82386-61"}]: dispatch 2024-02-23T00:47:18.293 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:17.271697+0000 mgr.y (mgr.14517) 345 : cluster [DBG] pgmap v523: 218 pgs: 1 activating, 1 remapped+peering, 216 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:47:18.293 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:17.272807+0000 mon.a (mon.0) 2341 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "21"}]: dispatch 2024-02-23T00:47:18.293 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:17.445476+0000 osd.7 (osd.7) 183 : cluster [DBG] 2.5 scrub starts 2024-02-23T00:47:18.294 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:17.448370+0000 osd.7 (osd.7) 184 : cluster [DBG] 2.5 scrub ok 2024-02-23T00:47:18.294 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:18.295 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:47:18.295 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:16.456855+0000 osd.1 (osd.1) 161 : cluster [DBG] 62.1 scrub starts 2024-02-23T00:47:18.295 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:16.460306+0000 osd.1 (osd.1) 162 : cluster [DBG] 62.1 scrub ok 2024-02-23T00:47:18.295 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:16.729472+0000 osd.2 (osd.2) 155 : cluster [DBG] 4.14 scrub starts 2024-02-23T00:47:18.295 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:16.731624+0000 osd.2 (osd.2) 156 : cluster [DBG] 4.14 scrub ok 2024-02-23T00:47:18.295 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:17.167169+0000 mon.a (mon.0) 2339 : cluster [DBG] osdmap e365: 8 total, 8 up, 8 in 2024-02-23T00:47:18.295 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:17.170103+0000 mon.a (mon.0) 2340 : audit [INF] from='client.? 172.21.15.47:0/526847179' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi047-82386-61"}]: dispatch 2024-02-23T00:47:18.295 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:17.271697+0000 mgr.y (mgr.14517) 345 : cluster [DBG] pgmap v523: 218 pgs: 1 activating, 1 remapped+peering, 216 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:47:18.295 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:17.272807+0000 mon.a (mon.0) 2341 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "21"}]: dispatch 2024-02-23T00:47:18.295 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:17.445476+0000 osd.7 (osd.7) 183 : cluster [DBG] 2.5 scrub starts 2024-02-23T00:47:18.295 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:17.448370+0000 osd.7 (osd.7) 184 : cluster [DBG] 2.5 scrub ok 2024-02-23T00:47:18.325 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:47:18.396 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:47:18.422 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:18.441 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:47:18.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:16.456855+0000 osd.1 (osd.1) 161 : cluster [DBG] 62.1 scrub starts 2024-02-23T00:47:18.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:16.460306+0000 osd.1 (osd.1) 162 : cluster [DBG] 62.1 scrub ok 2024-02-23T00:47:18.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:16.729472+0000 osd.2 (osd.2) 155 : cluster [DBG] 4.14 scrub starts 2024-02-23T00:47:18.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:16.731624+0000 osd.2 (osd.2) 156 : cluster [DBG] 4.14 scrub ok 2024-02-23T00:47:18.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:17.167169+0000 mon.a (mon.0) 2339 : cluster [DBG] osdmap e365: 8 total, 8 up, 8 in 2024-02-23T00:47:18.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:17.170103+0000 mon.a (mon.0) 2340 : audit [INF] from='client.? 172.21.15.47:0/526847179' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi047-82386-61"}]: dispatch 2024-02-23T00:47:18.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:17.271697+0000 mgr.y (mgr.14517) 345 : cluster [DBG] pgmap v523: 218 pgs: 1 activating, 1 remapped+peering, 216 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:47:18.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:17.272807+0000 mon.a (mon.0) 2341 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "21"}]: dispatch 2024-02-23T00:47:18.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:17.445476+0000 osd.7 (osd.7) 183 : cluster [DBG] 2.5 scrub starts 2024-02-23T00:47:18.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:17.448370+0000 osd.7 (osd.7) 184 : cluster [DBG] 2.5 scrub ok 2024-02-23T00:47:18.444 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:18.445 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:47:18.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:18.447+0000 7fa6e3d7a700 1 -- 172.21.15.47:0/3048728256 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7fa6dc06c5b0 con 0x7fa6dc0f8440 2024-02-23T00:47:18.545 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:47:18.571 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:18.590 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:47:18.592 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:18.593 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:47:18.694 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:47:18.720 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:18.739 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:47:18.742 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:18.742 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:47:18.843 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:47:18.869 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:18.887 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:47:18.889 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:18.890 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:47:18.991 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:47:19.018 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:19.036 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:47:19.039 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:19.039 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:47:19.140 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:47:19.172 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:19.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:19.176+0000 7fa6d2ffd700 1 -- 172.21.15.47:0/3048728256 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v367) v1 ==== 101+0+0 (secure 0 0 0) 0x7fa6cc0502f0 con 0x7fa6dc0f8440 2024-02-23T00:47:19.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:19.179+0000 7fa6e3d7a700 10 librados: watch_flush enter 2024-02-23T00:47:19.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:19.179+0000 7fa6e3d7a700 10 librados: watch_flush exit 2024-02-23T00:47:19.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:19.179+0000 7fa6e3d7a700 20 client.41225.objecter shutdown clearing up homeless session... 2024-02-23T00:47:19.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:19.179+0000 7fa6e3d7a700 10 client.41225.objecter successfully canceled tick 2024-02-23T00:47:19.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:19.182+0000 7fa6e3d7a700 1 -- 172.21.15.47:0/3048728256 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa6c8062cc0 msgr2=0x7fa6c8065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:19.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:19.182+0000 7fa6e3d7a700 1 --2- 172.21.15.47:0/3048728256 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa6c8062cc0 0x7fa6c8065180 secure :-1 s=READY pgs=3045 cs=0 l=1 rev1=1 rx=0x7fa6d80044a0 tx=0x7fa6d8009cb0).stop 2024-02-23T00:47:19.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:19.182+0000 7fa6e3d7a700 1 -- 172.21.15.47:0/3048728256 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa6dc0f8440 msgr2=0x7fa6dc06b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:19.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:19.182+0000 7fa6e3d7a700 1 --2- 172.21.15.47:0/3048728256 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa6dc0f8440 0x7fa6dc06b0d0 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 rx=0x7fa6cc0041d0 tx=0x7fa6cc009f60).stop 2024-02-23T00:47:19.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:19.185+0000 7fa6e3d7a700 1 -- 172.21.15.47:0/3048728256 shutdown_connections 2024-02-23T00:47:19.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:19.185+0000 7fa6e3d7a700 1 --2- 172.21.15.47:0/3048728256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6dc0fa380 0x7fa6dc06bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:19.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:19.185+0000 7fa6e3d7a700 1 --2- 172.21.15.47:0/3048728256 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa6c8062cc0 0x7fa6c8065180 unknown :-1 s=CLOSED pgs=3045 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:19.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:19.185+0000 7fa6e3d7a700 1 --2- 172.21.15.47:0/3048728256 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa6dc0f8440 0x7fa6dc06b0d0 unknown :-1 s=CLOSED pgs=2643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:19.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:19.185+0000 7fa6e3d7a700 1 --2- 172.21.15.47:0/3048728256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6dc0f9ab0 0x7fa6dc06b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:19.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:19.185+0000 7fa6e3d7a700 1 -- 172.21.15.47:0/3048728256 >> 172.21.15.47:0/3048728256 conn(0x7fa6dc0f3800 msgr2=0x7fa6dc0f4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:19.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:19.185+0000 7fa6e3d7a700 1 -- 172.21.15.47:0/3048728256 shutdown_connections 2024-02-23T00:47:19.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:19.185+0000 7fa6e3d7a700 1 -- 172.21.15.47:0/3048728256 wait complete. 2024-02-23T00:47:19.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:19.185+0000 7fa6e3d7a700 1 librados: shutdown 2024-02-23T00:47:19.189 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is unset 2024-02-23T00:47:19.198 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:47:19.201 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:19.202 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:47:19.302 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:47:19.330 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:19.352 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:47:19.354 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:19.355 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:47:19.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:17.211636+0000 osd.5 (osd.5) 154 : cluster [DBG] 2.2 scrub starts 2024-02-23T00:47:19.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:17.212379+0000 osd.5 (osd.5) 155 : cluster [DBG] 2.2 scrub ok 2024-02-23T00:47:19.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:17.366638+0000 osd.6 (osd.6) 149 : cluster [DBG] 62.1a scrub starts 2024-02-23T00:47:19.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:17.368388+0000 osd.6 (osd.6) 150 : cluster [DBG] 62.1a scrub ok 2024-02-23T00:47:19.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:18.163951+0000 mon.a (mon.0) 2342 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-02-23T00:47:19.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:18.171576+0000 osd.3 (osd.3) 175 : cluster [DBG] 6.c starting backfill to osd.1 from (0'0,0'0] MAX to 67'1 2024-02-23T00:47:19.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:18.173682+0000 mon.a (mon.0) 2343 : audit [INF] from='client.? 172.21.15.47:0/526847179' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi047-82386-61"}]': finished 2024-02-23T00:47:19.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:18.173797+0000 mon.a (mon.0) 2344 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "21"}]': finished 2024-02-23T00:47:19.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:18.173850+0000 mon.a (mon.0) 2345 : cluster [DBG] osdmap e366: 8 total, 8 up, 8 in 2024-02-23T00:47:19.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:18.176335+0000 mon.a (mon.0) 2346 : audit [INF] from='client.? 172.21.15.47:0/526847179' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi047-82386-61"}]: dispatch 2024-02-23T00:47:19.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:18.449310+0000 mon.c (mon.2) 274 : audit [INF] from='client.? 172.21.15.47:0/3048728256' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:47:19.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:18.449613+0000 mon.a (mon.0) 2347 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:47:19.455 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:47:19.482 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:19.501 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:47:19.504 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:19.504 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:47:19.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:17.211636+0000 osd.5 (osd.5) 154 : cluster [DBG] 2.2 scrub starts 2024-02-23T00:47:19.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:17.212379+0000 osd.5 (osd.5) 155 : cluster [DBG] 2.2 scrub ok 2024-02-23T00:47:19.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:17.366638+0000 osd.6 (osd.6) 149 : cluster [DBG] 62.1a scrub starts 2024-02-23T00:47:19.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:17.368388+0000 osd.6 (osd.6) 150 : cluster [DBG] 62.1a scrub ok 2024-02-23T00:47:19.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:18.163951+0000 mon.a (mon.0) 2342 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-02-23T00:47:19.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:18.171576+0000 osd.3 (osd.3) 175 : cluster [DBG] 6.c starting backfill to osd.1 from (0'0,0'0] MAX to 67'1 2024-02-23T00:47:19.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:18.173682+0000 mon.a (mon.0) 2343 : audit [INF] from='client.? 172.21.15.47:0/526847179' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi047-82386-61"}]': finished 2024-02-23T00:47:19.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:18.173797+0000 mon.a (mon.0) 2344 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "21"}]': finished 2024-02-23T00:47:19.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:18.173850+0000 mon.a (mon.0) 2345 : cluster [DBG] osdmap e366: 8 total, 8 up, 8 in 2024-02-23T00:47:19.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:18.176335+0000 mon.a (mon.0) 2346 : audit [INF] from='client.? 172.21.15.47:0/526847179' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi047-82386-61"}]: dispatch 2024-02-23T00:47:19.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:18.449310+0000 mon.c (mon.2) 274 : audit [INF] from='client.? 172.21.15.47:0/3048728256' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:47:19.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:18.449613+0000 mon.a (mon.0) 2347 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:47:19.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:17.211636+0000 osd.5 (osd.5) 154 : cluster [DBG] 2.2 scrub starts 2024-02-23T00:47:19.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:17.212379+0000 osd.5 (osd.5) 155 : cluster [DBG] 2.2 scrub ok 2024-02-23T00:47:19.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:17.366638+0000 osd.6 (osd.6) 149 : cluster [DBG] 62.1a scrub starts 2024-02-23T00:47:19.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:17.368388+0000 osd.6 (osd.6) 150 : cluster [DBG] 62.1a scrub ok 2024-02-23T00:47:19.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:18.163951+0000 mon.a (mon.0) 2342 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-02-23T00:47:19.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:18.171576+0000 osd.3 (osd.3) 175 : cluster [DBG] 6.c starting backfill to osd.1 from (0'0,0'0] MAX to 67'1 2024-02-23T00:47:19.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:18.173682+0000 mon.a (mon.0) 2343 : audit [INF] from='client.? 172.21.15.47:0/526847179' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi047-82386-61"}]': finished 2024-02-23T00:47:19.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:18.173797+0000 mon.a (mon.0) 2344 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "21"}]': finished 2024-02-23T00:47:19.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:18.173850+0000 mon.a (mon.0) 2345 : cluster [DBG] osdmap e366: 8 total, 8 up, 8 in 2024-02-23T00:47:19.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:18.176335+0000 mon.a (mon.0) 2346 : audit [INF] from='client.? 172.21.15.47:0/526847179' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi047-82386-61"}]: dispatch 2024-02-23T00:47:19.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:18.449310+0000 mon.c (mon.2) 274 : audit [INF] from='client.? 172.21.15.47:0/3048728256' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:47:19.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:18.449613+0000 mon.a (mon.0) 2347 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:47:19.605 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:47:19.634 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:20.184 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:47:20.189 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:20.189 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:47:20.290 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:47:20.319 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:20.339 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:47:20.342 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:20.342 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:47:20.443 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:47:20.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:18.251499+0000 osd.5 (osd.5) 156 : cluster [DBG] 5.f scrub starts 2024-02-23T00:47:20.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:18.254389+0000 osd.5 (osd.5) 157 : cluster [DBG] 5.f scrub ok 2024-02-23T00:47:20.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:18.763559+0000 osd.0 (osd.0) 149 : cluster [DBG] 4.3 scrub starts 2024-02-23T00:47:20.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:18.765577+0000 osd.0 (osd.0) 150 : cluster [DBG] 4.3 scrub ok 2024-02-23T00:47:20.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:19.169302+0000 mon.a (mon.0) 2348 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:47:20.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:19.173752+0000 mon.a (mon.0) 2349 : audit [INF] from='client.? 172.21.15.47:0/526847179' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi047-82386-61"}]': finished 2024-02-23T00:47:20.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:19.173861+0000 mon.a (mon.0) 2350 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:47:20.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:19.173935+0000 mon.a (mon.0) 2351 : cluster [DBG] osdmap e367: 8 total, 8 up, 8 in 2024-02-23T00:47:20.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:19.198287+0000 mon.c (mon.2) 275 : audit [INF] from='client.? 172.21.15.47:0/748831795' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi047-82386-62"}]: dispatch 2024-02-23T00:47:20.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:19.198732+0000 mon.a (mon.0) 2352 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi047-82386-62"}]: dispatch 2024-02-23T00:47:20.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:19.199309+0000 mon.c (mon.2) 276 : audit [INF] from='client.? 172.21.15.47:0/748831795' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi047-82386-62"}]: dispatch 2024-02-23T00:47:20.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:19.199557+0000 mon.a (mon.0) 2353 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi047-82386-62"}]: dispatch 2024-02-23T00:47:20.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:19.200098+0000 mon.c (mon.2) 277 : audit [INF] from='client.? 172.21.15.47:0/748831795' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi047-82386-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:47:20.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:19.200411+0000 mon.a (mon.0) 2354 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi047-82386-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:47:20.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:19.272130+0000 mgr.y (mgr.14517) 346 : cluster [DBG] pgmap v526: 218 pgs: 1 activating, 1 remapped+peering, 216 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:47:20.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:19.330366+0000 osd.4 (osd.4) 151 : cluster [DBG] 4.18 scrub starts 2024-02-23T00:47:20.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:19.331671+0000 osd.4 (osd.4) 152 : cluster [DBG] 4.18 scrub ok 2024-02-23T00:47:20.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:19.483469+0000 osd.7 (osd.7) 185 : cluster [DBG] 18.1f scrub starts 2024-02-23T00:47:20.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:19.484654+0000 osd.7 (osd.7) 186 : cluster [DBG] 18.1f scrub ok 2024-02-23T00:47:20.455 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:18.251499+0000 osd.5 (osd.5) 156 : cluster [DBG] 5.f scrub starts 2024-02-23T00:47:20.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:18.254389+0000 osd.5 (osd.5) 157 : cluster [DBG] 5.f scrub ok 2024-02-23T00:47:20.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:18.763559+0000 osd.0 (osd.0) 149 : cluster [DBG] 4.3 scrub starts 2024-02-23T00:47:20.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:18.765577+0000 osd.0 (osd.0) 150 : cluster [DBG] 4.3 scrub ok 2024-02-23T00:47:20.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:19.169302+0000 mon.a (mon.0) 2348 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:47:20.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:19.173752+0000 mon.a (mon.0) 2349 : audit [INF] from='client.? 172.21.15.47:0/526847179' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi047-82386-61"}]': finished 2024-02-23T00:47:20.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:19.173861+0000 mon.a (mon.0) 2350 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:47:20.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:19.173935+0000 mon.a (mon.0) 2351 : cluster [DBG] osdmap e367: 8 total, 8 up, 8 in 2024-02-23T00:47:20.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:19.198287+0000 mon.c (mon.2) 275 : audit [INF] from='client.? 172.21.15.47:0/748831795' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi047-82386-62"}]: dispatch 2024-02-23T00:47:20.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:19.198732+0000 mon.a (mon.0) 2352 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi047-82386-62"}]: dispatch 2024-02-23T00:47:20.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:19.199309+0000 mon.c (mon.2) 276 : audit [INF] from='client.? 172.21.15.47:0/748831795' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi047-82386-62"}]: dispatch 2024-02-23T00:47:20.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:19.199557+0000 mon.a (mon.0) 2353 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi047-82386-62"}]: dispatch 2024-02-23T00:47:20.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:19.200098+0000 mon.c (mon.2) 277 : audit [INF] from='client.? 172.21.15.47:0/748831795' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi047-82386-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:47:20.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:19.200411+0000 mon.a (mon.0) 2354 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi047-82386-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:47:20.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:19.272130+0000 mgr.y (mgr.14517) 346 : cluster [DBG] pgmap v526: 218 pgs: 1 activating, 1 remapped+peering, 216 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:47:20.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:19.330366+0000 osd.4 (osd.4) 151 : cluster [DBG] 4.18 scrub starts 2024-02-23T00:47:20.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:19.331671+0000 osd.4 (osd.4) 152 : cluster [DBG] 4.18 scrub ok 2024-02-23T00:47:20.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:19.483469+0000 osd.7 (osd.7) 185 : cluster [DBG] 18.1f scrub starts 2024-02-23T00:47:20.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:19.484654+0000 osd.7 (osd.7) 186 : cluster [DBG] 18.1f scrub ok 2024-02-23T00:47:20.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:18.251499+0000 osd.5 (osd.5) 156 : cluster [DBG] 5.f scrub starts 2024-02-23T00:47:20.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:18.254389+0000 osd.5 (osd.5) 157 : cluster [DBG] 5.f scrub ok 2024-02-23T00:47:20.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:18.763559+0000 osd.0 (osd.0) 149 : cluster [DBG] 4.3 scrub starts 2024-02-23T00:47:20.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:18.765577+0000 osd.0 (osd.0) 150 : cluster [DBG] 4.3 scrub ok 2024-02-23T00:47:20.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:19.169302+0000 mon.a (mon.0) 2348 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:47:20.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:19.173752+0000 mon.a (mon.0) 2349 : audit [INF] from='client.? 172.21.15.47:0/526847179' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi047-82386-61"}]': finished 2024-02-23T00:47:20.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:19.173861+0000 mon.a (mon.0) 2350 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:47:20.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:19.173935+0000 mon.a (mon.0) 2351 : cluster [DBG] osdmap e367: 8 total, 8 up, 8 in 2024-02-23T00:47:20.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:19.198287+0000 mon.c (mon.2) 275 : audit [INF] from='client.? 172.21.15.47:0/748831795' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi047-82386-62"}]: dispatch 2024-02-23T00:47:20.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:19.198732+0000 mon.a (mon.0) 2352 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi047-82386-62"}]: dispatch 2024-02-23T00:47:20.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:19.199309+0000 mon.c (mon.2) 276 : audit [INF] from='client.? 172.21.15.47:0/748831795' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi047-82386-62"}]: dispatch 2024-02-23T00:47:20.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:19.199557+0000 mon.a (mon.0) 2353 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi047-82386-62"}]: dispatch 2024-02-23T00:47:20.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:19.200098+0000 mon.c (mon.2) 277 : audit [INF] from='client.? 172.21.15.47:0/748831795' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi047-82386-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:47:20.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:19.200411+0000 mon.a (mon.0) 2354 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi047-82386-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:47:20.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:19.272130+0000 mgr.y (mgr.14517) 346 : cluster [DBG] pgmap v526: 218 pgs: 1 activating, 1 remapped+peering, 216 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:47:20.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:19.330366+0000 osd.4 (osd.4) 151 : cluster [DBG] 4.18 scrub starts 2024-02-23T00:47:20.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:19.331671+0000 osd.4 (osd.4) 152 : cluster [DBG] 4.18 scrub ok 2024-02-23T00:47:20.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:19.483469+0000 osd.7 (osd.7) 185 : cluster [DBG] 18.1f scrub starts 2024-02-23T00:47:20.459 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:19.484654+0000 osd.7 (osd.7) 186 : cluster [DBG] 18.1f scrub ok 2024-02-23T00:47:20.472 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:20.491 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:47:20.494 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:20.494 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:47:20.595 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:47:20.621 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:20.639 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:47:20.641 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:20.642 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:47:20.743 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:47:20.769 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:20.789 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:47:20.791 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:20.792 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:47:20.800 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:47:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:47:20] "GET /metrics HTTP/1.1" 200 185870 "" "Prometheus/2.33.4" 2024-02-23T00:47:20.892 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:47:20.923 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:21.274 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:47:21.283 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:21.283 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:47:21.384 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:47:21.411 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:21.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:19.227412+0000 osd.5 (osd.5) 158 : cluster [DBG] 18.f scrub starts 2024-02-23T00:47:21.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:19.231073+0000 osd.5 (osd.5) 159 : cluster [DBG] 18.f scrub ok 2024-02-23T00:47:21.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:19.759248+0000 osd.0 (osd.0) 151 : cluster [DBG] 18.4 scrub starts 2024-02-23T00:47:21.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:19.760229+0000 osd.0 (osd.0) 152 : cluster [DBG] 18.4 scrub ok 2024-02-23T00:47:21.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:20.032423+0000 osd.3 (osd.3) 176 : cluster [DBG] 2.19 scrub starts 2024-02-23T00:47:21.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:20.033373+0000 osd.3 (osd.3) 177 : cluster [DBG] 2.19 scrub ok 2024-02-23T00:47:21.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:20.188561+0000 mon.a (mon.0) 2355 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi047-82386-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:47:21.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:20.188651+0000 mon.a (mon.0) 2356 : cluster [DBG] osdmap e368: 8 total, 8 up, 8 in 2024-02-23T00:47:21.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:20.191844+0000 mon.c (mon.2) 278 : audit [INF] from='client.? 172.21.15.47:0/748831795' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi047-82386-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi047-82386-62"}]: dispatch 2024-02-23T00:47:21.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:20.192151+0000 mon.a (mon.0) 2357 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi047-82386-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi047-82386-62"}]: dispatch 2024-02-23T00:47:21.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:19.227412+0000 osd.5 (osd.5) 158 : cluster [DBG] 18.f scrub starts 2024-02-23T00:47:21.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:19.231073+0000 osd.5 (osd.5) 159 : cluster [DBG] 18.f scrub ok 2024-02-23T00:47:21.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:19.759248+0000 osd.0 (osd.0) 151 : cluster [DBG] 18.4 scrub starts 2024-02-23T00:47:21.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:19.760229+0000 osd.0 (osd.0) 152 : cluster [DBG] 18.4 scrub ok 2024-02-23T00:47:21.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:20.032423+0000 osd.3 (osd.3) 176 : cluster [DBG] 2.19 scrub starts 2024-02-23T00:47:21.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:20.033373+0000 osd.3 (osd.3) 177 : cluster [DBG] 2.19 scrub ok 2024-02-23T00:47:21.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:20.188561+0000 mon.a (mon.0) 2355 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi047-82386-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:47:21.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:20.188651+0000 mon.a (mon.0) 2356 : cluster [DBG] osdmap e368: 8 total, 8 up, 8 in 2024-02-23T00:47:21.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:20.191844+0000 mon.c (mon.2) 278 : audit [INF] from='client.? 172.21.15.47:0/748831795' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi047-82386-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi047-82386-62"}]: dispatch 2024-02-23T00:47:21.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:20.192151+0000 mon.a (mon.0) 2357 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi047-82386-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi047-82386-62"}]: dispatch 2024-02-23T00:47:21.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:19.227412+0000 osd.5 (osd.5) 158 : cluster [DBG] 18.f scrub starts 2024-02-23T00:47:21.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:19.231073+0000 osd.5 (osd.5) 159 : cluster [DBG] 18.f scrub ok 2024-02-23T00:47:21.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:19.759248+0000 osd.0 (osd.0) 151 : cluster [DBG] 18.4 scrub starts 2024-02-23T00:47:21.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:19.760229+0000 osd.0 (osd.0) 152 : cluster [DBG] 18.4 scrub ok 2024-02-23T00:47:21.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:20.032423+0000 osd.3 (osd.3) 176 : cluster [DBG] 2.19 scrub starts 2024-02-23T00:47:21.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:20.033373+0000 osd.3 (osd.3) 177 : cluster [DBG] 2.19 scrub ok 2024-02-23T00:47:21.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:20.188561+0000 mon.a (mon.0) 2355 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi047-82386-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:47:21.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:20.188651+0000 mon.a (mon.0) 2356 : cluster [DBG] osdmap e368: 8 total, 8 up, 8 in 2024-02-23T00:47:21.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:20.191844+0000 mon.c (mon.2) 278 : audit [INF] from='client.? 172.21.15.47:0/748831795' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi047-82386-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi047-82386-62"}]: dispatch 2024-02-23T00:47:21.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:20.192151+0000 mon.a (mon.0) 2357 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi047-82386-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi047-82386-62"}]: dispatch 2024-02-23T00:47:21.751 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-23T00:47:21.751 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-23T00:47:21.751 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-23T00:47:21.751 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd out 2 2024-02-23T00:47:22.162 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set noscrub 2024-02-23T00:47:22.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.304+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/562695653 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fca9c0f8420 msgr2=0x7fca9c104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:22.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.304+0000 7fcaa0d1c700 1 --2- 172.21.15.47:0/562695653 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fca9c0f8420 0x7fca9c104a20 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 rx=0x7fca94009be0 tx=0x7fca94004d10).stop 2024-02-23T00:47:22.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.304+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/562695653 shutdown_connections 2024-02-23T00:47:22.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.304+0000 7fcaa0d1c700 1 --2- 172.21.15.47:0/562695653 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fca9c0f8420 0x7fca9c104a20 unknown :-1 s=CLOSED pgs=2646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:22.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.304+0000 7fcaa0d1c700 1 --2- 172.21.15.47:0/562695653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca9c0fdbb0 0x7fca9c0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:22.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.304+0000 7fcaa0d1c700 1 --2- 172.21.15.47:0/562695653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca9c0fe510 0x7fca9c0f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:22.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.304+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/562695653 >> 172.21.15.47:0/562695653 conn(0x7fca9c0f3800 msgr2=0x7fca9c0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:22.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.304+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/562695653 shutdown_connections 2024-02-23T00:47:22.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.304+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/562695653 wait complete. 2024-02-23T00:47:22.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.305+0000 7fcaa0d1c700 1 librados: starting msgr at 2024-02-23T00:47:22.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.305+0000 7fcaa0d1c700 1 librados: starting objecter 2024-02-23T00:47:22.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.305+0000 7fcaa0d1c700 1 Processor -- start 2024-02-23T00:47:22.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.305+0000 7fcaa0d1c700 1 -- start start 2024-02-23T00:47:22.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.305+0000 7fcaa0d1c700 1 librados: setting wanted keys 2024-02-23T00:47:22.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.305+0000 7fcaa0d1c700 1 librados: calling monclient init 2024-02-23T00:47:22.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.306+0000 7fcaa0d1c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca9c0f8420 0x7fca9c06b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:22.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.306+0000 7fcaa0d1c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fca9c0fdbb0 0x7fca9c06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:22.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.306+0000 7fcaa0d1c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca9c0fe510 0x7fca9c06bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:22.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.306+0000 7fcaa0d1c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c1160a0 con 0x7fca9c0fe510 2024-02-23T00:47:22.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.306+0000 7fcaa0d1c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c116210 con 0x7fca9c0f8420 2024-02-23T00:47:22.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.306+0000 7fcaa0d1c700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c1163d0 con 0x7fca9c0fdbb0 2024-02-23T00:47:22.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.306+0000 7fca99d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fca9c0fdbb0 0x7fca9c06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:22.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.306+0000 7fca99d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fca9c0fdbb0 0x7fca9c06b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:33100/0 (socket says 172.21.15.47:33100) 2024-02-23T00:47:22.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.306+0000 7fca99d9b700 1 -- 172.21.15.47:0/1442247812 learned_addr learned my addr 172.21.15.47:0/1442247812 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:47:22.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.306+0000 7fca9ad9d700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca9c0fe510 0x7fca9c06bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:22.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.306+0000 7fca9a59c700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca9c0f8420 0x7fca9c06b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:22.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.306+0000 7fca99d9b700 1 -- 172.21.15.47:0/1442247812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca9c0f8420 msgr2=0x7fca9c06b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:22.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.306+0000 7fca99d9b700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca9c0f8420 0x7fca9c06b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:22.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.306+0000 7fca99d9b700 1 -- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca9c0fe510 msgr2=0x7fca9c06bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:22.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.306+0000 7fca99d9b700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca9c0fe510 0x7fca9c06bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:22.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.306+0000 7fca99d9b700 1 -- 172.21.15.47:0/1442247812 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca940097c0 con 0x7fca9c0fdbb0 2024-02-23T00:47:22.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.306+0000 7fca99d9b700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fca9c0fdbb0 0x7fca9c06b5c0 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 rx=0x7fca9000c7c0 tx=0x7fca90007ef0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:22.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.306+0000 7fca8b7fe700 10 client.?.objecter ms_handle_connect 0x7fca9c0fdbb0 2024-02-23T00:47:22.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.306+0000 7fca8b7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:47:22.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.306+0000 7fca8b7fe700 1 -- 172.21.15.47:0/1442247812 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca90004430 con 0x7fca9c0fdbb0 2024-02-23T00:47:22.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.306+0000 7fca8b7fe700 1 -- 172.21.15.47:0/1442247812 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fca9000e930 con 0x7fca9c0fdbb0 2024-02-23T00:47:22.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.306+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca9c1166b0 con 0x7fca9c0fdbb0 2024-02-23T00:47:22.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.306+0000 7fca8b7fe700 1 -- 172.21.15.47:0/1442247812 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca900139a0 con 0x7fca9c0fdbb0 2024-02-23T00:47:22.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.307+0000 7fcaa0d1c700 10 client.41237.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:47:22.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.307+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca9c116c70 con 0x7fca9c0fdbb0 2024-02-23T00:47:22.354 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:47:22.354 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:47:22.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.307+0000 7fcaa0d1c700 1 librados: init done 2024-02-23T00:47:22.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.307+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fca9c106ed0 con 0x7fca9c0fdbb0 2024-02-23T00:47:22.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.308+0000 7fca8b7fe700 1 -- 172.21.15.47:0/1442247812 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fca9000eaa0 con 0x7fca9c0fdbb0 2024-02-23T00:47:22.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.308+0000 7fca8b7fe700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fca84062cc0 0x7fca84065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:22.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.309+0000 7fca8b7fe700 1 -- 172.21.15.47:0/1442247812 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(370..370 src has 48..370) v4 ==== 10024+0+0 (secure 0 0 0) 0x7fca9007d840 con 0x7fca9c0fdbb0 2024-02-23T00:47:22.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.309+0000 7fca8b7fe700 10 client.41237.objecter ms_dispatch 0x7fca9c001550 osd_map(370..370 src has 48..370) v4 2024-02-23T00:47:22.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.309+0000 7fca8b7fe700 3 client.41237.objecter handle_osd_map got epochs [370,370] > 0 2024-02-23T00:47:22.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.309+0000 7fca8b7fe700 3 client.41237.objecter handle_osd_map decoding full epoch 370 2024-02-23T00:47:22.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.309+0000 7fca9a59c700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fca84062cc0 0x7fca84065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:22.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.309+0000 7fca8b7fe700 20 client.41237.objecter dump_active .. 0 homeless 2024-02-23T00:47:22.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.309+0000 7fca8b7fe700 1 -- 172.21.15.47:0/1442247812 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v370) v1 ==== 54+0+16 (secure 0 0 0) 0x7fca9007e0a0 con 0x7fca9c0fdbb0 2024-02-23T00:47:22.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.309+0000 7fcaa0d1c700 10 client.41237.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:22.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.309+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_assign -1 1 2024-02-23T00:47:22.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.309+0000 7fca8b7fe700 10 client.41237.objecter ms_handle_connect 0x7fca84062cc0 2024-02-23T00:47:22.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.309+0000 7fcaa0d1c700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fca9c0598f0 0x7fca9c059cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:22.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.309+0000 7fca9a59c700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fca84062cc0 0x7fca84065180 secure :-1 s=READY pgs=3047 cs=0 l=1 rev1=1 rx=0x7fca8c0048b0 tx=0x7fca8c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:22.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.309+0000 7fcaa0d1c700 20 client.41237.objecter _get_session s=0x7fca9c06c5b0 osd=0 3 2024-02-23T00:47:22.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.309+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c06c5b0 osd=0 3 2024-02-23T00:47:22.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.309+0000 7fcaa0d1c700 20 client.41237.objecter _get_session s=0x7fca9c06c5b0 osd=0 3 2024-02-23T00:47:22.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.309+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_remove -1 1 2024-02-23T00:47:22.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.309+0000 7fcaa0d1c700 20 client.41237.objecter get_session s=0x7fca9c06c5b0 osd=0 3 2024-02-23T00:47:22.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.309+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_assign 0 1 2024-02-23T00:47:22.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.309+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c06c5b0 osd=0 4 2024-02-23T00:47:22.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.309+0000 7fcaa0d1c700 10 client.41237.objecter _send_command 1 2024-02-23T00:47:22.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.309+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca9c05fa60 con 0x7fca9c0598f0 2024-02-23T00:47:22.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.309+0000 7fca9ad9d700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fca9c0598f0 0x7fca9c059cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:22.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.309+0000 7fca8b7fe700 10 client.41237.objecter ms_handle_connect 0x7fca9c0598f0 2024-02-23T00:47:22.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.309+0000 7fca9ad9d700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fca9c0598f0 0x7fca9c059cf0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:22.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.310+0000 7fca8b7fe700 1 -- 172.21.15.47:0/1442247812 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fca9c05fa60 con 0x7fca9c0598f0 2024-02-23T00:47:22.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.310+0000 7fca8b7fe700 10 client.41237.objecter ms_dispatch 0x7fca9c001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:47:22.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.310+0000 7fca8b7fe700 10 client.41237.objecter _finish_command 1 = system:0 2024-02-23T00:47:22.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.310+0000 7fca8b7fe700 20 client.41237.objecter put_session s=0x7fca9c06c5b0 osd=0 4 2024-02-23T00:47:22.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.310+0000 7fca8b7fe700 15 client.41237.objecter _session_command_op_remove 0 1 2024-02-23T00:47:22.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.325+0000 7fcaa0d1c700 10 client.41237.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:47:22.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.325+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_assign -1 2 2024-02-23T00:47:22.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.325+0000 7fcaa0d1c700 20 client.41237.objecter _get_session s=0x7fca9c06c5b0 osd=0 3 2024-02-23T00:47:22.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.325+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c06c5b0 osd=0 3 2024-02-23T00:47:22.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.325+0000 7fcaa0d1c700 20 client.41237.objecter _get_session s=0x7fca9c06c5b0 osd=0 3 2024-02-23T00:47:22.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.325+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_remove -1 2 2024-02-23T00:47:22.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.325+0000 7fcaa0d1c700 20 client.41237.objecter get_session s=0x7fca9c06c5b0 osd=0 3 2024-02-23T00:47:22.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.325+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_assign 0 2 2024-02-23T00:47:22.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.325+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c06c5b0 osd=0 4 2024-02-23T00:47:22.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.325+0000 7fcaa0d1c700 10 client.41237.objecter _send_command 2 2024-02-23T00:47:22.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.325+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fca9c05fbd0 con 0x7fca9c0598f0 2024-02-23T00:47:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.327+0000 7fca8b7fe700 1 -- 172.21.15.47:0/1442247812 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fca9c05fbd0 con 0x7fca9c0598f0 2024-02-23T00:47:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.327+0000 7fca8b7fe700 10 client.41237.objecter ms_dispatch 0x7fca9c001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:47:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.327+0000 7fca8b7fe700 10 client.41237.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:47:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.327+0000 7fca8b7fe700 20 client.41237.objecter put_session s=0x7fca9c06c5b0 osd=0 4 2024-02-23T00:47:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.327+0000 7fca8b7fe700 15 client.41237.objecter _session_command_op_remove 0 2 2024-02-23T00:47:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.327+0000 7fcaa0d1c700 10 client.41237.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.327+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_assign -1 3 2024-02-23T00:47:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.327+0000 7fcaa0d1c700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fca9c028540 0x7fca9c11aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.327+0000 7fcaa0d1c700 20 client.41237.objecter _get_session s=0x7fca9c0595a0 osd=1 3 2024-02-23T00:47:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.327+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c0595a0 osd=1 3 2024-02-23T00:47:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.327+0000 7fcaa0d1c700 20 client.41237.objecter _get_session s=0x7fca9c0595a0 osd=1 3 2024-02-23T00:47:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.327+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_remove -1 3 2024-02-23T00:47:22.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.327+0000 7fcaa0d1c700 20 client.41237.objecter get_session s=0x7fca9c0595a0 osd=1 3 2024-02-23T00:47:22.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.327+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_assign 1 3 2024-02-23T00:47:22.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.327+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c0595a0 osd=1 4 2024-02-23T00:47:22.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.327+0000 7fcaa0d1c700 10 client.41237.objecter _send_command 3 2024-02-23T00:47:22.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.327+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca9c059710 con 0x7fca9c028540 2024-02-23T00:47:22.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.327+0000 7fca9a59c700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fca9c028540 0x7fca9c11aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:22.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.328+0000 7fca8b7fe700 10 client.41237.objecter ms_handle_connect 0x7fca9c028540 2024-02-23T00:47:22.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.328+0000 7fca9a59c700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fca9c028540 0x7fca9c11aeb0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:22.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.328+0000 7fca8b7fe700 1 -- 172.21.15.47:0/1442247812 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fca9c059710 con 0x7fca9c028540 2024-02-23T00:47:22.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.328+0000 7fca8b7fe700 10 client.41237.objecter ms_dispatch 0x7fca9c001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:47:22.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.328+0000 7fca8b7fe700 10 client.41237.objecter _finish_command 3 = system:0 2024-02-23T00:47:22.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.328+0000 7fca8b7fe700 20 client.41237.objecter put_session s=0x7fca9c0595a0 osd=1 4 2024-02-23T00:47:22.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.328+0000 7fca8b7fe700 15 client.41237.objecter _session_command_op_remove 1 3 2024-02-23T00:47:22.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.343+0000 7fcaa0d1c700 10 client.41237.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:47:22.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.343+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_assign -1 4 2024-02-23T00:47:22.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.343+0000 7fcaa0d1c700 20 client.41237.objecter _get_session s=0x7fca9c0595a0 osd=1 3 2024-02-23T00:47:22.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.343+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c0595a0 osd=1 3 2024-02-23T00:47:22.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.343+0000 7fcaa0d1c700 20 client.41237.objecter _get_session s=0x7fca9c0595a0 osd=1 3 2024-02-23T00:47:22.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.343+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_remove -1 4 2024-02-23T00:47:22.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.343+0000 7fcaa0d1c700 20 client.41237.objecter get_session s=0x7fca9c0595a0 osd=1 3 2024-02-23T00:47:22.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.343+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_assign 1 4 2024-02-23T00:47:22.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.343+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c0595a0 osd=1 4 2024-02-23T00:47:22.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.343+0000 7fcaa0d1c700 10 client.41237.objecter _send_command 4 2024-02-23T00:47:22.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.343+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fca9c0ff870 con 0x7fca9c028540 2024-02-23T00:47:22.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.345+0000 7fca8b7fe700 1 -- 172.21.15.47:0/1442247812 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fca9c0ff870 con 0x7fca9c028540 2024-02-23T00:47:22.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.345+0000 7fca8b7fe700 10 client.41237.objecter ms_dispatch 0x7fca9c001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:47:22.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.345+0000 7fca8b7fe700 10 client.41237.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:47:22.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.345+0000 7fca8b7fe700 20 client.41237.objecter put_session s=0x7fca9c0595a0 osd=1 4 2024-02-23T00:47:22.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.345+0000 7fca8b7fe700 15 client.41237.objecter _session_command_op_remove 1 4 2024-02-23T00:47:22.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.345+0000 7fcaa0d1c700 10 client.41237.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:22.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.345+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_assign -1 5 2024-02-23T00:47:22.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.345+0000 7fcaa0d1c700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fca9c0fed00 0x7fca9c11e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:22.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.345+0000 7fcaa0d1c700 20 client.41237.objecter _get_session s=0x7fca9c0feb90 osd=2 3 2024-02-23T00:47:22.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.345+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c0feb90 osd=2 3 2024-02-23T00:47:22.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.345+0000 7fcaa0d1c700 20 client.41237.objecter _get_session s=0x7fca9c0feb90 osd=2 3 2024-02-23T00:47:22.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.345+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_remove -1 5 2024-02-23T00:47:22.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.345+0000 7fcaa0d1c700 20 client.41237.objecter get_session s=0x7fca9c0feb90 osd=2 3 2024-02-23T00:47:22.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.345+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_assign 2 5 2024-02-23T00:47:22.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.345+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c0feb90 osd=2 4 2024-02-23T00:47:22.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.345+0000 7fcaa0d1c700 10 client.41237.objecter _send_command 5 2024-02-23T00:47:22.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.345+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca9c0281c0 con 0x7fca9c0fed00 2024-02-23T00:47:22.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.345+0000 7fca99d9b700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fca9c0fed00 0x7fca9c11e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:22.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.346+0000 7fca8b7fe700 10 client.41237.objecter ms_handle_connect 0x7fca9c0fed00 2024-02-23T00:47:22.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.346+0000 7fca99d9b700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fca9c0fed00 0x7fca9c11e530 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:22.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.346+0000 7fca8b7fe700 1 -- 172.21.15.47:0/1442247812 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fca9c0281c0 con 0x7fca9c0fed00 2024-02-23T00:47:22.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.346+0000 7fca8b7fe700 10 client.41237.objecter ms_dispatch 0x7fca9c001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:47:22.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.346+0000 7fca8b7fe700 10 client.41237.objecter _finish_command 5 = system:0 2024-02-23T00:47:22.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.346+0000 7fca8b7fe700 20 client.41237.objecter put_session s=0x7fca9c0feb90 osd=2 4 2024-02-23T00:47:22.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.346+0000 7fca8b7fe700 15 client.41237.objecter _session_command_op_remove 2 5 2024-02-23T00:47:22.364 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:47:22.365 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:20.241597+0000 osd.5 (osd.5) 160 : cluster [DBG] 4.8 scrub starts 2024-02-23T00:47:22.365 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:20.244295+0000 osd.5 (osd.5) 161 : cluster [DBG] 4.8 scrub ok 2024-02-23T00:47:22.365 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:20.794180+0000 osd.0 (osd.0) 153 : cluster [DBG] 4.6 scrub starts 2024-02-23T00:47:22.365 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:20.795877+0000 osd.0 (osd.0) 154 : cluster [DBG] 4.6 scrub ok 2024-02-23T00:47:22.365 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:21.272582+0000 mgr.y (mgr.14517) 347 : cluster [DBG] pgmap v529: 217 pgs: 1 activating, 1 remapped+peering, 215 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:47:22.365 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:21.633551+0000 mon.a (mon.0) 2358 : cluster [DBG] osdmap e369: 8 total, 8 up, 8 in 2024-02-23T00:47:22.365 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:21.634830+0000 mon.a (mon.0) 2359 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "20"}]: dispatch 2024-02-23T00:47:22.366 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:22.366 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:47:22.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.363+0000 7fcaa0d1c700 10 client.41237.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:47:22.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.363+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_assign -1 6 2024-02-23T00:47:22.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.363+0000 7fcaa0d1c700 20 client.41237.objecter _get_session s=0x7fca9c0feb90 osd=2 3 2024-02-23T00:47:22.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.363+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c0feb90 osd=2 3 2024-02-23T00:47:22.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.363+0000 7fcaa0d1c700 20 client.41237.objecter _get_session s=0x7fca9c0feb90 osd=2 3 2024-02-23T00:47:22.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.363+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_remove -1 6 2024-02-23T00:47:22.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.363+0000 7fcaa0d1c700 20 client.41237.objecter get_session s=0x7fca9c0feb90 osd=2 3 2024-02-23T00:47:22.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.363+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_assign 2 6 2024-02-23T00:47:22.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.363+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c0feb90 osd=2 4 2024-02-23T00:47:22.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.363+0000 7fcaa0d1c700 10 client.41237.objecter _send_command 6 2024-02-23T00:47:22.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.363+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fca9c028330 con 0x7fca9c0fed00 2024-02-23T00:47:22.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:20.241597+0000 osd.5 (osd.5) 160 : cluster [DBG] 4.8 scrub starts 2024-02-23T00:47:22.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:20.244295+0000 osd.5 (osd.5) 161 : cluster [DBG] 4.8 scrub ok 2024-02-23T00:47:22.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:20.794180+0000 osd.0 (osd.0) 153 : cluster [DBG] 4.6 scrub starts 2024-02-23T00:47:22.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:20.795877+0000 osd.0 (osd.0) 154 : cluster [DBG] 4.6 scrub ok 2024-02-23T00:47:22.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:21.272582+0000 mgr.y (mgr.14517) 347 : cluster [DBG] pgmap v529: 217 pgs: 1 activating, 1 remapped+peering, 215 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:47:22.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:21.633551+0000 mon.a (mon.0) 2358 : cluster [DBG] osdmap e369: 8 total, 8 up, 8 in 2024-02-23T00:47:22.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:21.634830+0000 mon.a (mon.0) 2359 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "20"}]: dispatch 2024-02-23T00:47:22.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.366+0000 7fca8b7fe700 1 -- 172.21.15.47:0/1442247812 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fca9c028330 con 0x7fca9c0fed00 2024-02-23T00:47:22.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.366+0000 7fca8b7fe700 10 client.41237.objecter ms_dispatch 0x7fca9c001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:47:22.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.366+0000 7fca8b7fe700 10 client.41237.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:47:22.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.366+0000 7fca8b7fe700 20 client.41237.objecter put_session s=0x7fca9c0feb90 osd=2 4 2024-02-23T00:47:22.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.366+0000 7fca8b7fe700 15 client.41237.objecter _session_command_op_remove 2 6 2024-02-23T00:47:22.369 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:47:22.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.366+0000 7fcaa0d1c700 10 client.41237.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:22.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.366+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_assign -1 7 2024-02-23T00:47:22.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.366+0000 7fcaa0d1c700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fca9c11c080 0x7fca9c121bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:22.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.366+0000 7fcaa0d1c700 20 client.41237.objecter _get_session s=0x7fca9c027840 osd=3 3 2024-02-23T00:47:22.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.366+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c027840 osd=3 3 2024-02-23T00:47:22.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.366+0000 7fcaa0d1c700 20 client.41237.objecter _get_session s=0x7fca9c027840 osd=3 3 2024-02-23T00:47:22.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.366+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_remove -1 7 2024-02-23T00:47:22.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.366+0000 7fcaa0d1c700 20 client.41237.objecter get_session s=0x7fca9c027840 osd=3 3 2024-02-23T00:47:22.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.366+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_assign 3 7 2024-02-23T00:47:22.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.366+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c027840 osd=3 4 2024-02-23T00:47:22.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.366+0000 7fcaa0d1c700 10 client.41237.objecter _send_command 7 2024-02-23T00:47:22.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.366+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca9c124850 con 0x7fca9c11c080 2024-02-23T00:47:22.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.366+0000 7fca9ad9d700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fca9c11c080 0x7fca9c121bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:22.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.367+0000 7fca9ad9d700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fca9c11c080 0x7fca9c121bb0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:22.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.367+0000 7fca8b7fe700 10 client.41237.objecter ms_handle_connect 0x7fca9c11c080 2024-02-23T00:47:22.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.367+0000 7fca8b7fe700 1 -- 172.21.15.47:0/1442247812 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fca9c124850 con 0x7fca9c11c080 2024-02-23T00:47:22.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.367+0000 7fca8b7fe700 10 client.41237.objecter ms_dispatch 0x7fca9c001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:47:22.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.367+0000 7fca8b7fe700 10 client.41237.objecter _finish_command 7 = system:0 2024-02-23T00:47:22.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.367+0000 7fca8b7fe700 20 client.41237.objecter put_session s=0x7fca9c027840 osd=3 4 2024-02-23T00:47:22.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.367+0000 7fca8b7fe700 15 client.41237.objecter _session_command_op_remove 3 7 2024-02-23T00:47:22.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.385+0000 7fcaa0d1c700 10 client.41237.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:47:22.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.385+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_assign -1 8 2024-02-23T00:47:22.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.385+0000 7fcaa0d1c700 20 client.41237.objecter _get_session s=0x7fca9c027840 osd=3 3 2024-02-23T00:47:22.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.385+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c027840 osd=3 3 2024-02-23T00:47:22.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.385+0000 7fcaa0d1c700 20 client.41237.objecter _get_session s=0x7fca9c027840 osd=3 3 2024-02-23T00:47:22.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.385+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_remove -1 8 2024-02-23T00:47:22.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.385+0000 7fcaa0d1c700 20 client.41237.objecter get_session s=0x7fca9c027840 osd=3 3 2024-02-23T00:47:22.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.385+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_assign 3 8 2024-02-23T00:47:22.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.385+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c027840 osd=3 4 2024-02-23T00:47:22.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.385+0000 7fcaa0d1c700 10 client.41237.objecter _send_command 8 2024-02-23T00:47:22.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.385+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fca9c11b5e0 con 0x7fca9c11c080 2024-02-23T00:47:22.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.387+0000 7fca8b7fe700 1 -- 172.21.15.47:0/1442247812 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fca9c11b5e0 con 0x7fca9c11c080 2024-02-23T00:47:22.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.387+0000 7fca8b7fe700 10 client.41237.objecter ms_dispatch 0x7fca9c001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:47:22.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.387+0000 7fca8b7fe700 10 client.41237.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:47:22.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.387+0000 7fca8b7fe700 20 client.41237.objecter put_session s=0x7fca9c027840 osd=3 4 2024-02-23T00:47:22.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.387+0000 7fca8b7fe700 15 client.41237.objecter _session_command_op_remove 3 8 2024-02-23T00:47:22.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.387+0000 7fcaa0d1c700 10 client.41237.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:22.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.387+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_assign -1 9 2024-02-23T00:47:22.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.387+0000 7fcaa0d1c700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fca9c11f700 0x7fca9c123220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:22.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.387+0000 7fcaa0d1c700 20 client.41237.objecter _get_session s=0x7fca9c11b5e0 osd=4 3 2024-02-23T00:47:22.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.387+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c11b5e0 osd=4 3 2024-02-23T00:47:22.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.387+0000 7fcaa0d1c700 20 client.41237.objecter _get_session s=0x7fca9c11b5e0 osd=4 3 2024-02-23T00:47:22.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.387+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_remove -1 9 2024-02-23T00:47:22.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.387+0000 7fcaa0d1c700 20 client.41237.objecter get_session s=0x7fca9c11b5e0 osd=4 3 2024-02-23T00:47:22.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.387+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_assign 4 9 2024-02-23T00:47:22.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.387+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c11b5e0 osd=4 4 2024-02-23T00:47:22.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.387+0000 7fcaa0d1c700 10 client.41237.objecter _send_command 9 2024-02-23T00:47:22.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.387+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca9c11b8b0 con 0x7fca9c11f700 2024-02-23T00:47:22.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.387+0000 7fca9a59c700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fca9c11f700 0x7fca9c123220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:22.392 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:47:22.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.388+0000 7fca8b7fe700 10 client.41237.objecter ms_handle_connect 0x7fca9c11f700 2024-02-23T00:47:22.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.388+0000 7fca9a59c700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fca9c11f700 0x7fca9c123220 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:22.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.389+0000 7fca8b7fe700 1 -- 172.21.15.47:0/1442247812 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fca9c11b8b0 con 0x7fca9c11f700 2024-02-23T00:47:22.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.389+0000 7fca8b7fe700 10 client.41237.objecter ms_dispatch 0x7fca9c001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:47:22.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.389+0000 7fca8b7fe700 10 client.41237.objecter _finish_command 9 = system:0 2024-02-23T00:47:22.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.389+0000 7fca8b7fe700 20 client.41237.objecter put_session s=0x7fca9c11b5e0 osd=4 4 2024-02-23T00:47:22.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.389+0000 7fca8b7fe700 15 client.41237.objecter _session_command_op_remove 4 9 2024-02-23T00:47:22.400 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:47:22.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.406+0000 7fcaa0d1c700 10 client.41237.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:47:22.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.406+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_assign -1 10 2024-02-23T00:47:22.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.406+0000 7fcaa0d1c700 20 client.41237.objecter _get_session s=0x7fca9c11b5e0 osd=4 3 2024-02-23T00:47:22.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.406+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c11b5e0 osd=4 3 2024-02-23T00:47:22.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.406+0000 7fcaa0d1c700 20 client.41237.objecter _get_session s=0x7fca9c11b5e0 osd=4 3 2024-02-23T00:47:22.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.406+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_remove -1 10 2024-02-23T00:47:22.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.406+0000 7fcaa0d1c700 20 client.41237.objecter get_session s=0x7fca9c11b5e0 osd=4 3 2024-02-23T00:47:22.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.406+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_assign 4 10 2024-02-23T00:47:22.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.406+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c11b5e0 osd=4 4 2024-02-23T00:47:22.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.406+0000 7fcaa0d1c700 10 client.41237.objecter _send_command 10 2024-02-23T00:47:22.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.406+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fca9c11b8b0 con 0x7fca9c11f700 2024-02-23T00:47:22.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.409+0000 7fca8b7fe700 1 -- 172.21.15.47:0/1442247812 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fca9c11b8b0 con 0x7fca9c11f700 2024-02-23T00:47:22.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.409+0000 7fca8b7fe700 10 client.41237.objecter ms_dispatch 0x7fca9c001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:47:22.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.409+0000 7fca8b7fe700 10 client.41237.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:47:22.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.409+0000 7fca8b7fe700 20 client.41237.objecter put_session s=0x7fca9c11b5e0 osd=4 4 2024-02-23T00:47:22.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.409+0000 7fca8b7fe700 15 client.41237.objecter _session_command_op_remove 4 10 2024-02-23T00:47:22.411 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:47:22.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.409+0000 7fcaa0d1c700 10 client.41237.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:22.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.409+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_assign -1 11 2024-02-23T00:47:22.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.409+0000 7fcaa0d1c700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fca9c123c20 0x7fca9c124060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:22.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.409+0000 7fcaa0d1c700 20 client.41237.objecter _get_session s=0x7fca9c11b8b0 osd=5 3 2024-02-23T00:47:22.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.409+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c11b8b0 osd=5 3 2024-02-23T00:47:22.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.409+0000 7fcaa0d1c700 20 client.41237.objecter _get_session s=0x7fca9c11b8b0 osd=5 3 2024-02-23T00:47:22.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.409+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_remove -1 11 2024-02-23T00:47:22.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.409+0000 7fcaa0d1c700 20 client.41237.objecter get_session s=0x7fca9c11b8b0 osd=5 3 2024-02-23T00:47:22.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.409+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_assign 5 11 2024-02-23T00:47:22.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.409+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c11b8b0 osd=5 4 2024-02-23T00:47:22.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.409+0000 7fcaa0d1c700 10 client.41237.objecter _send_command 11 2024-02-23T00:47:22.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.409+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca9c12b170 con 0x7fca9c123c20 2024-02-23T00:47:22.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.409+0000 7fca99d9b700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fca9c123c20 0x7fca9c124060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:22.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.410+0000 7fca8b7fe700 10 client.41237.objecter ms_handle_connect 0x7fca9c123c20 2024-02-23T00:47:22.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.410+0000 7fca99d9b700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fca9c123c20 0x7fca9c124060 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:22.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.411+0000 7fca8b7fe700 1 -- 172.21.15.47:0/1442247812 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fca9c12b170 con 0x7fca9c123c20 2024-02-23T00:47:22.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.411+0000 7fca8b7fe700 10 client.41237.objecter ms_dispatch 0x7fca9c001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:47:22.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.411+0000 7fca8b7fe700 10 client.41237.objecter _finish_command 11 = system:0 2024-02-23T00:47:22.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.411+0000 7fca8b7fe700 20 client.41237.objecter put_session s=0x7fca9c11b8b0 osd=5 4 2024-02-23T00:47:22.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.411+0000 7fca8b7fe700 15 client.41237.objecter _session_command_op_remove 5 11 2024-02-23T00:47:22.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.433+0000 7fcaa0d1c700 10 client.41237.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:47:22.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.433+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_assign -1 12 2024-02-23T00:47:22.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.433+0000 7fcaa0d1c700 20 client.41237.objecter _get_session s=0x7fca9c11b8b0 osd=5 3 2024-02-23T00:47:22.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.433+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c11b8b0 osd=5 3 2024-02-23T00:47:22.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.433+0000 7fcaa0d1c700 20 client.41237.objecter _get_session s=0x7fca9c11b8b0 osd=5 3 2024-02-23T00:47:22.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.433+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_remove -1 12 2024-02-23T00:47:22.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.433+0000 7fcaa0d1c700 20 client.41237.objecter get_session s=0x7fca9c11b8b0 osd=5 3 2024-02-23T00:47:22.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.433+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_assign 5 12 2024-02-23T00:47:22.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.433+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c11b8b0 osd=5 4 2024-02-23T00:47:22.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.433+0000 7fcaa0d1c700 10 client.41237.objecter _send_command 12 2024-02-23T00:47:22.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.433+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fca9c128a10 con 0x7fca9c123c20 2024-02-23T00:47:22.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.435+0000 7fca8b7fe700 1 -- 172.21.15.47:0/1442247812 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fca9c128a10 con 0x7fca9c123c20 2024-02-23T00:47:22.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.435+0000 7fca8b7fe700 10 client.41237.objecter ms_dispatch 0x7fca9c001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:47:22.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.435+0000 7fca8b7fe700 10 client.41237.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:47:22.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.435+0000 7fca8b7fe700 20 client.41237.objecter put_session s=0x7fca9c11b8b0 osd=5 4 2024-02-23T00:47:22.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.435+0000 7fca8b7fe700 15 client.41237.objecter _session_command_op_remove 5 12 2024-02-23T00:47:22.437 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:47:22.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.435+0000 7fcaa0d1c700 10 client.41237.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:22.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.435+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_assign -1 13 2024-02-23T00:47:22.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.435+0000 7fcaa0d1c700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fca9c129090 0x7fca9c12b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:22.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.435+0000 7fcaa0d1c700 20 client.41237.objecter _get_session s=0x7fca9c128a10 osd=6 3 2024-02-23T00:47:22.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.435+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c128a10 osd=6 3 2024-02-23T00:47:22.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.435+0000 7fcaa0d1c700 20 client.41237.objecter _get_session s=0x7fca9c128a10 osd=6 3 2024-02-23T00:47:22.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.435+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_remove -1 13 2024-02-23T00:47:22.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.435+0000 7fcaa0d1c700 20 client.41237.objecter get_session s=0x7fca9c128a10 osd=6 3 2024-02-23T00:47:22.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.435+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_assign 6 13 2024-02-23T00:47:22.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.435+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c128a10 osd=6 4 2024-02-23T00:47:22.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.435+0000 7fcaa0d1c700 10 client.41237.objecter _send_command 13 2024-02-23T00:47:22.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.435+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca9c12e2e0 con 0x7fca9c129090 2024-02-23T00:47:22.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.436+0000 7fca9ad9d700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fca9c129090 0x7fca9c12b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:22.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.436+0000 7fca8b7fe700 10 client.41237.objecter ms_handle_connect 0x7fca9c129090 2024-02-23T00:47:22.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.436+0000 7fca9ad9d700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fca9c129090 0x7fca9c12b4a0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:22.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.437+0000 7fca8b7fe700 1 -- 172.21.15.47:0/1442247812 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fca9c12e2e0 con 0x7fca9c129090 2024-02-23T00:47:22.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.437+0000 7fca8b7fe700 10 client.41237.objecter ms_dispatch 0x7fca9c001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:47:22.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.437+0000 7fca8b7fe700 10 client.41237.objecter _finish_command 13 = system:0 2024-02-23T00:47:22.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.437+0000 7fca8b7fe700 20 client.41237.objecter put_session s=0x7fca9c128a10 osd=6 4 2024-02-23T00:47:22.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.437+0000 7fca8b7fe700 15 client.41237.objecter _session_command_op_remove 6 13 2024-02-23T00:47:22.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.454+0000 7fcaa0d1c700 10 client.41237.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:47:22.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.454+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_assign -1 14 2024-02-23T00:47:22.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.454+0000 7fcaa0d1c700 20 client.41237.objecter _get_session s=0x7fca9c128a10 osd=6 3 2024-02-23T00:47:22.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.454+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c128a10 osd=6 3 2024-02-23T00:47:22.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.454+0000 7fcaa0d1c700 20 client.41237.objecter _get_session s=0x7fca9c128a10 osd=6 3 2024-02-23T00:47:22.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.454+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_remove -1 14 2024-02-23T00:47:22.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.454+0000 7fcaa0d1c700 20 client.41237.objecter get_session s=0x7fca9c128a10 osd=6 3 2024-02-23T00:47:22.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.454+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_assign 6 14 2024-02-23T00:47:22.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.454+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c128a10 osd=6 4 2024-02-23T00:47:22.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.454+0000 7fcaa0d1c700 10 client.41237.objecter _send_command 14 2024-02-23T00:47:22.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.454+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fca9c12e2e0 con 0x7fca9c129090 2024-02-23T00:47:22.457 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:47:22.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.456+0000 7fca8b7fe700 1 -- 172.21.15.47:0/1442247812 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fca9c12e2e0 con 0x7fca9c129090 2024-02-23T00:47:22.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.456+0000 7fca8b7fe700 10 client.41237.objecter ms_dispatch 0x7fca9c001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:47:22.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.456+0000 7fca8b7fe700 10 client.41237.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:47:22.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.456+0000 7fca8b7fe700 20 client.41237.objecter put_session s=0x7fca9c128a10 osd=6 4 2024-02-23T00:47:22.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.456+0000 7fca8b7fe700 15 client.41237.objecter _session_command_op_remove 6 14 2024-02-23T00:47:22.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.456+0000 7fcaa0d1c700 10 client.41237.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.456+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_assign -1 15 2024-02-23T00:47:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.456+0000 7fcaa0d1c700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fca9c12d610 0x7fca9c12da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.456+0000 7fcaa0d1c700 20 client.41237.objecter _get_session s=0x7fca9c12e2e0 osd=7 3 2024-02-23T00:47:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.456+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c12e2e0 osd=7 3 2024-02-23T00:47:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.456+0000 7fcaa0d1c700 20 client.41237.objecter _get_session s=0x7fca9c12e2e0 osd=7 3 2024-02-23T00:47:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.456+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_remove -1 15 2024-02-23T00:47:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.456+0000 7fcaa0d1c700 20 client.41237.objecter get_session s=0x7fca9c12e2e0 osd=7 3 2024-02-23T00:47:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.456+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_assign 7 15 2024-02-23T00:47:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.456+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c12e2e0 osd=7 4 2024-02-23T00:47:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.456+0000 7fcaa0d1c700 10 client.41237.objecter _send_command 15 2024-02-23T00:47:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.456+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca9c131610 con 0x7fca9c12d610 2024-02-23T00:47:22.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.456+0000 7fca9a59c700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fca9c12d610 0x7fca9c12da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:22.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.457+0000 7fca8b7fe700 10 client.41237.objecter ms_handle_connect 0x7fca9c12d610 2024-02-23T00:47:22.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.457+0000 7fca9a59c700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fca9c12d610 0x7fca9c12da90 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:22.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.458+0000 7fca8b7fe700 1 -- 172.21.15.47:0/1442247812 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fca9c131610 con 0x7fca9c12d610 2024-02-23T00:47:22.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.458+0000 7fca8b7fe700 10 client.41237.objecter ms_dispatch 0x7fca9c001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:47:22.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.458+0000 7fca8b7fe700 10 client.41237.objecter _finish_command 15 = system:0 2024-02-23T00:47:22.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.458+0000 7fca8b7fe700 20 client.41237.objecter put_session s=0x7fca9c12e2e0 osd=7 4 2024-02-23T00:47:22.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.458+0000 7fca8b7fe700 15 client.41237.objecter _session_command_op_remove 7 15 2024-02-23T00:47:22.467 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:47:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.473+0000 7fcaa0d1c700 10 client.41237.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:47:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.473+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_assign -1 16 2024-02-23T00:47:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.473+0000 7fcaa0d1c700 20 client.41237.objecter _get_session s=0x7fca9c12e2e0 osd=7 3 2024-02-23T00:47:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.473+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c12e2e0 osd=7 3 2024-02-23T00:47:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.473+0000 7fcaa0d1c700 20 client.41237.objecter _get_session s=0x7fca9c12e2e0 osd=7 3 2024-02-23T00:47:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.473+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_remove -1 16 2024-02-23T00:47:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.473+0000 7fcaa0d1c700 20 client.41237.objecter get_session s=0x7fca9c12e2e0 osd=7 3 2024-02-23T00:47:22.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.473+0000 7fcaa0d1c700 15 client.41237.objecter _session_command_op_assign 7 16 2024-02-23T00:47:22.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.473+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c12e2e0 osd=7 4 2024-02-23T00:47:22.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.473+0000 7fcaa0d1c700 10 client.41237.objecter _send_command 16 2024-02-23T00:47:22.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.473+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fca9c12baa0 con 0x7fca9c12d610 2024-02-23T00:47:22.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.474+0000 7fca8b7fe700 1 -- 172.21.15.47:0/1442247812 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fca9c12baa0 con 0x7fca9c12d610 2024-02-23T00:47:22.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.474+0000 7fca8b7fe700 10 client.41237.objecter ms_dispatch 0x7fca9c001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:47:22.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.474+0000 7fca8b7fe700 10 client.41237.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:47:22.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.474+0000 7fca8b7fe700 20 client.41237.objecter put_session s=0x7fca9c12e2e0 osd=7 4 2024-02-23T00:47:22.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.474+0000 7fca8b7fe700 15 client.41237.objecter _session_command_op_remove 7 16 2024-02-23T00:47:22.477 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:47:22.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 10 librados: watch_flush enter 2024-02-23T00:47:22.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 10 librados: watch_flush exit 2024-02-23T00:47:22.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 10 client.41237.objecter close_session for osd.0 2024-02-23T00:47:22.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fca9c0598f0 msgr2=0x7fca9c059cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:22.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fca9c0598f0 0x7fca9c059cf0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:22.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c06c5b0 osd=0 1 2024-02-23T00:47:22.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 10 client.41237.objecter close_session for osd.1 2024-02-23T00:47:22.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fca9c028540 msgr2=0x7fca9c11aeb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:22.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fca9c028540 0x7fca9c11aeb0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:22.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c0595a0 osd=1 1 2024-02-23T00:47:22.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 10 client.41237.objecter close_session for osd.2 2024-02-23T00:47:22.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fca9c0fed00 msgr2=0x7fca9c11e530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:22.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fca9c0fed00 0x7fca9c11e530 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:22.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c0feb90 osd=2 1 2024-02-23T00:47:22.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 10 client.41237.objecter close_session for osd.3 2024-02-23T00:47:22.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fca9ad9d700 1 -- 172.21.15.47:0/1442247812 reap_dead start 2024-02-23T00:47:22.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fca9c11c080 msgr2=0x7fca9c121bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:22.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fca9c11c080 0x7fca9c121bb0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:22.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c027840 osd=3 1 2024-02-23T00:47:22.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 10 client.41237.objecter close_session for osd.4 2024-02-23T00:47:22.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fca9c11f700 msgr2=0x7fca9c123220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:22.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fca9c11f700 0x7fca9c123220 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:22.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c11b5e0 osd=4 1 2024-02-23T00:47:22.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 10 client.41237.objecter close_session for osd.5 2024-02-23T00:47:22.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fca9c123c20 msgr2=0x7fca9c124060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:22.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fca9c123c20 0x7fca9c124060 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:22.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c11b8b0 osd=5 1 2024-02-23T00:47:22.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 10 client.41237.objecter close_session for osd.6 2024-02-23T00:47:22.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fca9c129090 msgr2=0x7fca9c12b4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:22.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fca9c129090 0x7fca9c12b4a0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:22.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c128a10 osd=6 1 2024-02-23T00:47:22.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 10 client.41237.objecter close_session for osd.7 2024-02-23T00:47:22.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fca9c12d610 msgr2=0x7fca9c12da90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:22.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fca9c12d610 0x7fca9c12da90 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:22.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 20 client.41237.objecter put_session s=0x7fca9c12e2e0 osd=7 1 2024-02-23T00:47:22.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 20 client.41237.objecter shutdown clearing up homeless session... 2024-02-23T00:47:22.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 10 client.41237.objecter successfully canceled tick 2024-02-23T00:47:22.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fca9ad9d700 1 -- 172.21.15.47:0/1442247812 reap_dead start 2024-02-23T00:47:22.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fca84062cc0 msgr2=0x7fca84065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:22.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fca84062cc0 0x7fca84065180 secure :-1 s=READY pgs=3047 cs=0 l=1 rev1=1 rx=0x7fca8c0048b0 tx=0x7fca8c00b040).stop 2024-02-23T00:47:22.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fca9c0fdbb0 msgr2=0x7fca9c06b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:22.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fca9c0fdbb0 0x7fca9c06b5c0 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 rx=0x7fca9000c7c0 tx=0x7fca90007ef0).stop 2024-02-23T00:47:22.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 shutdown_connections 2024-02-23T00:47:22.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fca9c0fdbb0 0x7fca9c06b5c0 unknown :-1 s=CLOSED pgs=2647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:22.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 1 --2- 172.21.15.47:0/1442247812 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fca84062cc0 0x7fca84065180 unknown :-1 s=CLOSED pgs=3047 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:22.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 >> 172.21.15.47:0/1442247812 conn(0x7fca9c0f3800 msgr2=0x7fca9c0f50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:22.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 shutdown_connections 2024-02-23T00:47:22.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 1 -- 172.21.15.47:0/1442247812 wait complete. 2024-02-23T00:47:22.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:22.475+0000 7fcaa0d1c700 1 librados: shutdown 2024-02-23T00:47:22.482 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:47:22.482 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:47:22.482 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:47:22.482 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:47:22.483 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:47:22.483 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:47:22.483 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:47:22.483 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:47:22.496 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:20.241597+0000 osd.5 (osd.5) 160 : cluster [DBG] 4.8 scrub starts 2024-02-23T00:47:22.497 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:20.244295+0000 osd.5 (osd.5) 161 : cluster [DBG] 4.8 scrub ok 2024-02-23T00:47:22.497 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:20.794180+0000 osd.0 (osd.0) 153 : cluster [DBG] 4.6 scrub starts 2024-02-23T00:47:22.497 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:20.795877+0000 osd.0 (osd.0) 154 : cluster [DBG] 4.6 scrub ok 2024-02-23T00:47:22.497 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:21.272582+0000 mgr.y (mgr.14517) 347 : cluster [DBG] pgmap v529: 217 pgs: 1 activating, 1 remapped+peering, 215 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-23T00:47:22.497 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:21.633551+0000 mon.a (mon.0) 2358 : cluster [DBG] osdmap e369: 8 total, 8 up, 8 in 2024-02-23T00:47:22.497 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:21.634830+0000 mon.a (mon.0) 2359 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "20"}]: dispatch 2024-02-23T00:47:22.499 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:22.520 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:47:22.523 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:22.523 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:47:22.624 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:47:22.654 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:22.675 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:47:22.678 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:22.678 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:47:22.779 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:47:22.806 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:47:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:47:22.807 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:47:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:47:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:47:22.808 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:22.828 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:47:22.830 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:22.831 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:47:22.932 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:47:22.963 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:22.983 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:47:22.988 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:22.988 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:47:23.089 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:47:23.118 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:23.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:21.603586+0000 osd.2 (osd.2) 157 : cluster [DBG] 18.12 scrub starts 2024-02-23T00:47:23.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:21.604588+0000 osd.2 (osd.2) 158 : cluster [DBG] 18.12 scrub ok 2024-02-23T00:47:23.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:22.199610+0000 mon.a (mon.0) 2360 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi047-82386-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi047-82386-62"}]': finished 2024-02-23T00:47:23.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:22.199698+0000 mon.a (mon.0) 2361 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "20"}]': finished 2024-02-23T00:47:23.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:22.199743+0000 mon.a (mon.0) 2362 : cluster [DBG] osdmap e370: 8 total, 8 up, 8 in 2024-02-23T00:47:23.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:22.308228+0000 mon.c (mon.2) 279 : audit [DBG] from='client.? 172.21.15.47:0/1442247812' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:47:23.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:22.445913+0000 osd.7 (osd.7) 187 : cluster [DBG] 18.13 scrub starts 2024-02-23T00:47:23.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:22.448937+0000 osd.7 (osd.7) 188 : cluster [DBG] 18.13 scrub ok 2024-02-23T00:47:23.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:22.547315+0000 mgr.y (mgr.14517) 348 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:47:23.214 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:21.603586+0000 osd.2 (osd.2) 157 : cluster [DBG] 18.12 scrub starts 2024-02-23T00:47:23.214 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:21.604588+0000 osd.2 (osd.2) 158 : cluster [DBG] 18.12 scrub ok 2024-02-23T00:47:23.214 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:22.199610+0000 mon.a (mon.0) 2360 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi047-82386-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi047-82386-62"}]': finished 2024-02-23T00:47:23.215 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:22.199698+0000 mon.a (mon.0) 2361 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "20"}]': finished 2024-02-23T00:47:23.215 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:22.199743+0000 mon.a (mon.0) 2362 : cluster [DBG] osdmap e370: 8 total, 8 up, 8 in 2024-02-23T00:47:23.215 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:22.308228+0000 mon.c (mon.2) 279 : audit [DBG] from='client.? 172.21.15.47:0/1442247812' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:47:23.215 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:22.445913+0000 osd.7 (osd.7) 187 : cluster [DBG] 18.13 scrub starts 2024-02-23T00:47:23.215 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:22.448937+0000 osd.7 (osd.7) 188 : cluster [DBG] 18.13 scrub ok 2024-02-23T00:47:23.215 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:22.547315+0000 mgr.y (mgr.14517) 348 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:47:23.367 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:47:23.369 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:23.370 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:47:23.471 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:47:23.500 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:21.603586+0000 osd.2 (osd.2) 157 : cluster [DBG] 18.12 scrub starts 2024-02-23T00:47:23.500 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:21.604588+0000 osd.2 (osd.2) 158 : cluster [DBG] 18.12 scrub ok 2024-02-23T00:47:23.500 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:22.199610+0000 mon.a (mon.0) 2360 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi047-82386-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi047-82386-62"}]': finished 2024-02-23T00:47:23.501 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:22.199698+0000 mon.a (mon.0) 2361 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "20"}]': finished 2024-02-23T00:47:23.501 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:22.199743+0000 mon.a (mon.0) 2362 : cluster [DBG] osdmap e370: 8 total, 8 up, 8 in 2024-02-23T00:47:23.501 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:22.308228+0000 mon.c (mon.2) 279 : audit [DBG] from='client.? 172.21.15.47:0/1442247812' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:47:23.501 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:22.445913+0000 osd.7 (osd.7) 187 : cluster [DBG] 18.13 scrub starts 2024-02-23T00:47:23.501 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:22.448937+0000 osd.7 (osd.7) 188 : cluster [DBG] 18.13 scrub ok 2024-02-23T00:47:23.501 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:22.547315+0000 mgr.y (mgr.14517) 348 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:47:23.502 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:23.524 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:47:23.526 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:23.527 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:47:23.627 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:47:23.659 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:24.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:22.312113+0000 osd.5 (osd.5) 162 : cluster [DBG] 5.1b scrub starts 2024-02-23T00:47:24.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:22.312995+0000 osd.5 (osd.5) 163 : cluster [DBG] 5.1b scrub ok 2024-02-23T00:47:24.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:22.368229+0000 osd.1 (osd.1) 163 : cluster [DBG] 5.12 scrub starts 2024-02-23T00:47:24.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:22.371489+0000 osd.1 (osd.1) 164 : cluster [DBG] 5.12 scrub ok 2024-02-23T00:47:24.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:23.200148+0000 mon.a (mon.0) 2363 : cluster [DBG] osdmap e371: 8 total, 8 up, 8 in 2024-02-23T00:47:24.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:23.273018+0000 mgr.y (mgr.14517) 349 : cluster [DBG] pgmap v532: 225 pgs: 1 active+remapped+backfill_toofull, 2 peering, 4 creating+peering, 4 unknown, 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/1107 objects misplaced (0.090%) 2024-02-23T00:47:24.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:23.482469+0000 osd.7 (osd.7) 189 : cluster [DBG] 5.9 scrub starts 2024-02-23T00:47:24.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:23.485940+0000 osd.7 (osd.7) 190 : cluster [DBG] 5.9 scrub ok 2024-02-23T00:47:24.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:24.066492+0000 osd.3 (osd.3) 178 : cluster [DBG] 2.1e scrub starts 2024-02-23T00:47:24.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:24.067416+0000 osd.3 (osd.3) 179 : cluster [DBG] 2.1e scrub ok 2024-02-23T00:47:24.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:22.312113+0000 osd.5 (osd.5) 162 : cluster [DBG] 5.1b scrub starts 2024-02-23T00:47:24.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:22.312995+0000 osd.5 (osd.5) 163 : cluster [DBG] 5.1b scrub ok 2024-02-23T00:47:24.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:22.368229+0000 osd.1 (osd.1) 163 : cluster [DBG] 5.12 scrub starts 2024-02-23T00:47:24.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:22.371489+0000 osd.1 (osd.1) 164 : cluster [DBG] 5.12 scrub ok 2024-02-23T00:47:24.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:23.200148+0000 mon.a (mon.0) 2363 : cluster [DBG] osdmap e371: 8 total, 8 up, 8 in 2024-02-23T00:47:24.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:23.273018+0000 mgr.y (mgr.14517) 349 : cluster [DBG] pgmap v532: 225 pgs: 1 active+remapped+backfill_toofull, 2 peering, 4 creating+peering, 4 unknown, 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/1107 objects misplaced (0.090%) 2024-02-23T00:47:24.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:23.482469+0000 osd.7 (osd.7) 189 : cluster [DBG] 5.9 scrub starts 2024-02-23T00:47:24.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:23.485940+0000 osd.7 (osd.7) 190 : cluster [DBG] 5.9 scrub ok 2024-02-23T00:47:24.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:24.066492+0000 osd.3 (osd.3) 178 : cluster [DBG] 2.1e scrub starts 2024-02-23T00:47:24.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:24.067416+0000 osd.3 (osd.3) 179 : cluster [DBG] 2.1e scrub ok 2024-02-23T00:47:24.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:22.312113+0000 osd.5 (osd.5) 162 : cluster [DBG] 5.1b scrub starts 2024-02-23T00:47:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:22.312995+0000 osd.5 (osd.5) 163 : cluster [DBG] 5.1b scrub ok 2024-02-23T00:47:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:22.368229+0000 osd.1 (osd.1) 163 : cluster [DBG] 5.12 scrub starts 2024-02-23T00:47:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:22.371489+0000 osd.1 (osd.1) 164 : cluster [DBG] 5.12 scrub ok 2024-02-23T00:47:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:23.200148+0000 mon.a (mon.0) 2363 : cluster [DBG] osdmap e371: 8 total, 8 up, 8 in 2024-02-23T00:47:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:23.273018+0000 mgr.y (mgr.14517) 349 : cluster [DBG] pgmap v532: 225 pgs: 1 active+remapped+backfill_toofull, 2 peering, 4 creating+peering, 4 unknown, 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/1107 objects misplaced (0.090%) 2024-02-23T00:47:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:23.482469+0000 osd.7 (osd.7) 189 : cluster [DBG] 5.9 scrub starts 2024-02-23T00:47:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:23.485940+0000 osd.7 (osd.7) 190 : cluster [DBG] 5.9 scrub ok 2024-02-23T00:47:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:24.066492+0000 osd.3 (osd.3) 178 : cluster [DBG] 2.1e scrub starts 2024-02-23T00:47:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:24.067416+0000 osd.3 (osd.3) 179 : cluster [DBG] 2.1e scrub ok 2024-02-23T00:47:24.708 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:47:24.711 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:24.712 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:47:24.812 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:47:24.843 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:24.992 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:47:24.995 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:24.995 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:47:25.096 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:47:25.124 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:25.208 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:47:25.208 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-02-23T00:47:25.208 INFO:teuthology.orchestra.run.smithi047.stdout: "num_ops": 0 2024-02-23T00:47:25.208 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:47:25.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:24.208060+0000 mon.a (mon.0) 2364 : cluster [WRN] Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-02-23T00:47:25.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:24.208097+0000 mon.a (mon.0) 2365 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-02-23T00:47:25.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:24.211743+0000 mon.a (mon.0) 2366 : cluster [DBG] osdmap e372: 8 total, 8 up, 8 in 2024-02-23T00:47:25.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:24.218969+0000 mon.c (mon.2) 280 : audit [INF] from='client.? 172.21.15.47:0/748831795' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi047-82386-62"}]: dispatch 2024-02-23T00:47:25.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:24.223185+0000 mon.a (mon.0) 2367 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi047-82386-62"}]: dispatch 2024-02-23T00:47:25.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:24.352355+0000 osd.4 (osd.4) 153 : cluster [DBG] 5.e scrub starts 2024-02-23T00:47:25.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:24.353883+0000 osd.4 (osd.4) 154 : cluster [DBG] 5.e scrub ok 2024-02-23T00:47:25.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:24.441375+0000 osd.7 (osd.7) 191 : cluster [DBG] 2.0 scrub starts 2024-02-23T00:47:25.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:24.444386+0000 osd.7 (osd.7) 192 : cluster [DBG] 2.0 scrub ok 2024-02-23T00:47:25.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:25.166704+0000 mon.a (mon.0) 2368 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi047-82386-62"}]': finished 2024-02-23T00:47:25.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:25.166777+0000 mon.a (mon.0) 2369 : cluster [DBG] osdmap e373: 8 total, 8 up, 8 in 2024-02-23T00:47:25.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:25.168262+0000 mon.c (mon.2) 281 : audit [INF] from='client.? 172.21.15.47:0/748831795' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi047-82386-62"}]: dispatch 2024-02-23T00:47:25.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:25.174947+0000 mon.a (mon.0) 2370 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi047-82386-62"}]: dispatch 2024-02-23T00:47:25.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:24.208060+0000 mon.a (mon.0) 2364 : cluster [WRN] Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-02-23T00:47:25.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:24.208097+0000 mon.a (mon.0) 2365 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-02-23T00:47:25.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:24.211743+0000 mon.a (mon.0) 2366 : cluster [DBG] osdmap e372: 8 total, 8 up, 8 in 2024-02-23T00:47:25.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:24.218969+0000 mon.c (mon.2) 280 : audit [INF] from='client.? 172.21.15.47:0/748831795' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi047-82386-62"}]: dispatch 2024-02-23T00:47:25.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:24.223185+0000 mon.a (mon.0) 2367 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi047-82386-62"}]: dispatch 2024-02-23T00:47:25.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:24.352355+0000 osd.4 (osd.4) 153 : cluster [DBG] 5.e scrub starts 2024-02-23T00:47:25.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:24.353883+0000 osd.4 (osd.4) 154 : cluster [DBG] 5.e scrub ok 2024-02-23T00:47:25.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:24.441375+0000 osd.7 (osd.7) 191 : cluster [DBG] 2.0 scrub starts 2024-02-23T00:47:25.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:24.444386+0000 osd.7 (osd.7) 192 : cluster [DBG] 2.0 scrub ok 2024-02-23T00:47:25.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:25.166704+0000 mon.a (mon.0) 2368 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi047-82386-62"}]': finished 2024-02-23T00:47:25.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:25.166777+0000 mon.a (mon.0) 2369 : cluster [DBG] osdmap e373: 8 total, 8 up, 8 in 2024-02-23T00:47:25.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:25.168262+0000 mon.c (mon.2) 281 : audit [INF] from='client.? 172.21.15.47:0/748831795' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi047-82386-62"}]: dispatch 2024-02-23T00:47:25.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:25.174947+0000 mon.a (mon.0) 2370 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi047-82386-62"}]: dispatch 2024-02-23T00:47:25.450 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:47:25.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:24.208060+0000 mon.a (mon.0) 2364 : cluster [WRN] Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-02-23T00:47:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:24.208097+0000 mon.a (mon.0) 2365 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-02-23T00:47:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:24.211743+0000 mon.a (mon.0) 2366 : cluster [DBG] osdmap e372: 8 total, 8 up, 8 in 2024-02-23T00:47:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:24.218969+0000 mon.c (mon.2) 280 : audit [INF] from='client.? 172.21.15.47:0/748831795' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi047-82386-62"}]: dispatch 2024-02-23T00:47:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:24.223185+0000 mon.a (mon.0) 2367 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi047-82386-62"}]: dispatch 2024-02-23T00:47:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:24.352355+0000 osd.4 (osd.4) 153 : cluster [DBG] 5.e scrub starts 2024-02-23T00:47:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:24.353883+0000 osd.4 (osd.4) 154 : cluster [DBG] 5.e scrub ok 2024-02-23T00:47:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:24.441375+0000 osd.7 (osd.7) 191 : cluster [DBG] 2.0 scrub starts 2024-02-23T00:47:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:24.444386+0000 osd.7 (osd.7) 192 : cluster [DBG] 2.0 scrub ok 2024-02-23T00:47:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:25.166704+0000 mon.a (mon.0) 2368 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi047-82386-62"}]': finished 2024-02-23T00:47:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:25.166777+0000 mon.a (mon.0) 2369 : cluster [DBG] osdmap e373: 8 total, 8 up, 8 in 2024-02-23T00:47:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:25.168262+0000 mon.c (mon.2) 281 : audit [INF] from='client.? 172.21.15.47:0/748831795' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi047-82386-62"}]: dispatch 2024-02-23T00:47:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:25.174947+0000 mon.a (mon.0) 2370 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi047-82386-62"}]: dispatch 2024-02-23T00:47:25.988 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.0 dump_blocked_ops 2024-02-23T00:47:26.243 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:24.791462+0000 osd.0 (osd.0) 155 : cluster [DBG] 18.0 scrub starts 2024-02-23T00:47:26.244 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:24.792460+0000 osd.0 (osd.0) 156 : cluster [DBG] 18.0 scrub ok 2024-02-23T00:47:26.244 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:25.273615+0000 mgr.y (mgr.14517) 350 : cluster [DBG] pgmap v535: 216 pgs: 1 active+remapped+backfill_toofull, 1 peering, 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/1107 objects misplaced (0.090%) 2024-02-23T00:47:26.244 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:25.274424+0000 mon.a (mon.0) 2371 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "19"}]: dispatch 2024-02-23T00:47:26.244 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:25.378471+0000 osd.4 (osd.4) 155 : cluster [DBG] 4.1b scrub starts 2024-02-23T00:47:26.245 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:24.791462+0000 osd.0 (osd.0) 155 : cluster [DBG] 18.0 scrub starts 2024-02-23T00:47:26.245 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:24.792460+0000 osd.0 (osd.0) 156 : cluster [DBG] 18.0 scrub ok 2024-02-23T00:47:26.246 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:25.273615+0000 mgr.y (mgr.14517) 350 : cluster [DBG] pgmap v535: 216 pgs: 1 active+remapped+backfill_toofull, 1 peering, 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/1107 objects misplaced (0.090%) 2024-02-23T00:47:26.246 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:25.274424+0000 mon.a (mon.0) 2371 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "19"}]: dispatch 2024-02-23T00:47:26.246 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:25.378471+0000 osd.4 (osd.4) 155 : cluster [DBG] 4.1b scrub starts 2024-02-23T00:47:26.246 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:25.380055+0000 osd.4 (osd.4) 156 : cluster [DBG] 4.1b scrub ok 2024-02-23T00:47:26.246 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:25.460800+0000 osd.7 (osd.7) 193 : cluster [DBG] 62.a scrub starts 2024-02-23T00:47:26.246 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:25.464273+0000 osd.7 (osd.7) 194 : cluster [DBG] 62.a scrub ok 2024-02-23T00:47:26.246 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:26.176550+0000 mon.a (mon.0) 2372 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi047-82386-62"}]': finished 2024-02-23T00:47:26.246 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:26.176635+0000 mon.a (mon.0) 2373 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "19"}]': finished 2024-02-23T00:47:26.246 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:26.176684+0000 mon.a (mon.0) 2374 : cluster [DBG] osdmap e374: 8 total, 8 up, 8 in 2024-02-23T00:47:26.246 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:26.195790+0000 mon.c (mon.2) 282 : audit [INF] from='client.? 172.21.15.47:0/3348293548' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi047-82386-63"}]: dispatch 2024-02-23T00:47:26.246 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:26.196201+0000 mon.a (mon.0) 2375 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi047-82386-63"}]: dispatch 2024-02-23T00:47:26.246 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:26.196815+0000 mon.c (mon.2) 283 : audit [INF] from='client.? 172.21.15.47:0/3348293548' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi047-82386-63"}]: dispatch 2024-02-23T00:47:26.246 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:26.197108+0000 mon.a (mon.0) 2376 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi047-82386-63"}]: dispatch 2024-02-23T00:47:26.247 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:26.197529+0000 mon.c (mon.2) 284 : audit [INF] from='client.? 172.21.15.47:0/3348293548' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi047-82386-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:47:26.247 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:26.197784+0000 mon.a (mon.0) 2377 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi047-82386-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:47:26.505 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:25.380055+0000 osd.4 (osd.4) 156 : cluster [DBG] 4.1b scrub ok 2024-02-23T00:47:26.505 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:25.460800+0000 osd.7 (osd.7) 193 : cluster [DBG] 62.a scrub starts 2024-02-23T00:47:26.505 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:25.464273+0000 osd.7 (osd.7) 194 : cluster [DBG] 62.a scrub ok 2024-02-23T00:47:26.505 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:26.176550+0000 mon.a (mon.0) 2372 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi047-82386-62"}]': finished 2024-02-23T00:47:26.505 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:26.176635+0000 mon.a (mon.0) 2373 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "19"}]': finished 2024-02-23T00:47:26.505 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:26.176684+0000 mon.a (mon.0) 2374 : cluster [DBG] osdmap e374: 8 total, 8 up, 8 in 2024-02-23T00:47:26.506 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:26.195790+0000 mon.c (mon.2) 282 : audit [INF] from='client.? 172.21.15.47:0/3348293548' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi047-82386-63"}]: dispatch 2024-02-23T00:47:26.506 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:26.196201+0000 mon.a (mon.0) 2375 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi047-82386-63"}]: dispatch 2024-02-23T00:47:26.506 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:26.196815+0000 mon.c (mon.2) 283 : audit [INF] from='client.? 172.21.15.47:0/3348293548' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi047-82386-63"}]: dispatch 2024-02-23T00:47:26.506 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:26.197108+0000 mon.a (mon.0) 2376 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi047-82386-63"}]: dispatch 2024-02-23T00:47:26.506 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:26.197529+0000 mon.c (mon.2) 284 : audit [INF] from='client.? 172.21.15.47:0/3348293548' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi047-82386-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:47:26.506 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:26.197784+0000 mon.a (mon.0) 2377 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi047-82386-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:47:26.604 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:47:26.606 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:26.607 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:47:26.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:24.791462+0000 osd.0 (osd.0) 155 : cluster [DBG] 18.0 scrub starts 2024-02-23T00:47:26.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:24.792460+0000 osd.0 (osd.0) 156 : cluster [DBG] 18.0 scrub ok 2024-02-23T00:47:26.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:25.273615+0000 mgr.y (mgr.14517) 350 : cluster [DBG] pgmap v535: 216 pgs: 1 active+remapped+backfill_toofull, 1 peering, 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/1107 objects misplaced (0.090%) 2024-02-23T00:47:26.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:25.274424+0000 mon.a (mon.0) 2371 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "19"}]: dispatch 2024-02-23T00:47:26.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:25.378471+0000 osd.4 (osd.4) 155 : cluster [DBG] 4.1b scrub starts 2024-02-23T00:47:26.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:25.380055+0000 osd.4 (osd.4) 156 : cluster [DBG] 4.1b scrub ok 2024-02-23T00:47:26.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:25.460800+0000 osd.7 (osd.7) 193 : cluster [DBG] 62.a scrub starts 2024-02-23T00:47:26.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:25.464273+0000 osd.7 (osd.7) 194 : cluster [DBG] 62.a scrub ok 2024-02-23T00:47:26.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:26.176550+0000 mon.a (mon.0) 2372 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi047-82386-62"}]': finished 2024-02-23T00:47:26.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:26.176635+0000 mon.a (mon.0) 2373 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "19"}]': finished 2024-02-23T00:47:26.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:26.176684+0000 mon.a (mon.0) 2374 : cluster [DBG] osdmap e374: 8 total, 8 up, 8 in 2024-02-23T00:47:26.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:26.195790+0000 mon.c (mon.2) 282 : audit [INF] from='client.? 172.21.15.47:0/3348293548' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi047-82386-63"}]: dispatch 2024-02-23T00:47:26.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:26.196201+0000 mon.a (mon.0) 2375 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi047-82386-63"}]: dispatch 2024-02-23T00:47:26.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:26.196815+0000 mon.c (mon.2) 283 : audit [INF] from='client.? 172.21.15.47:0/3348293548' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi047-82386-63"}]: dispatch 2024-02-23T00:47:26.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:26.197108+0000 mon.a (mon.0) 2376 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi047-82386-63"}]: dispatch 2024-02-23T00:47:26.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:26.197529+0000 mon.c (mon.2) 284 : audit [INF] from='client.? 172.21.15.47:0/3348293548' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi047-82386-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:47:26.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:26.197784+0000 mon.a (mon.0) 2377 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi047-82386-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:47:26.707 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:47:26.737 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:26.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.714+0000 7fd406b8e700 1 -- 172.21.15.47:0/4209791535 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd40006b0f0 msgr2=0x7fd40006b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:26.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.714+0000 7fd406b8e700 1 --2- 172.21.15.47:0/4209791535 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd40006b0f0 0x7fd40006b4d0 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 rx=0x7fd3f0009be0 tx=0x7fd3f0004d10).stop 2024-02-23T00:47:26.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.715+0000 7fd406b8e700 1 -- 172.21.15.47:0/4209791535 shutdown_connections 2024-02-23T00:47:26.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.715+0000 7fd406b8e700 1 --2- 172.21.15.47:0/4209791535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4001044c0 0x7fd4001068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:26.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.715+0000 7fd406b8e700 1 --2- 172.21.15.47:0/4209791535 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd40006b0f0 0x7fd40006b4d0 unknown :-1 s=CLOSED pgs=2649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:26.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.715+0000 7fd406b8e700 1 --2- 172.21.15.47:0/4209791535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd40006b9c0 0x7fd400103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:26.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.715+0000 7fd406b8e700 1 -- 172.21.15.47:0/4209791535 >> 172.21.15.47:0/4209791535 conn(0x7fd4000f3800 msgr2=0x7fd4000f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:26.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.715+0000 7fd406b8e700 1 -- 172.21.15.47:0/4209791535 shutdown_connections 2024-02-23T00:47:26.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.715+0000 7fd406b8e700 1 -- 172.21.15.47:0/4209791535 wait complete. 2024-02-23T00:47:26.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.715+0000 7fd406b8e700 1 librados: starting msgr at 2024-02-23T00:47:26.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.715+0000 7fd406b8e700 1 librados: starting objecter 2024-02-23T00:47:26.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.715+0000 7fd406b8e700 1 Processor -- start 2024-02-23T00:47:26.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.715+0000 7fd406b8e700 1 -- start start 2024-02-23T00:47:26.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.715+0000 7fd406b8e700 1 librados: setting wanted keys 2024-02-23T00:47:26.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.715+0000 7fd406b8e700 1 librados: calling monclient init 2024-02-23T00:47:26.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.715+0000 7fd406b8e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd40006b0f0 0x7fd4001102c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:26.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.715+0000 7fd406b8e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd40006b9c0 0x7fd4001107b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:26.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.715+0000 7fd406b8e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4001044c0 0x7fd40010a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:26.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.715+0000 7fd406b8e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40010a950 con 0x7fd4001044c0 2024-02-23T00:47:26.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.715+0000 7fd406b8e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40010aac0 con 0x7fd40006b0f0 2024-02-23T00:47:26.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.715+0000 7fd406b8e700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40010ac80 con 0x7fd40006b9c0 2024-02-23T00:47:26.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.716+0000 7fd40512b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4001044c0 0x7fd40010a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:26.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.716+0000 7fd40512b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4001044c0 0x7fd40010a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39158/0 (socket says 172.21.15.47:39158) 2024-02-23T00:47:26.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.716+0000 7fd40512b700 1 -- 172.21.15.47:0/1859591895 learned_addr learned my addr 172.21.15.47:0/1859591895 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:47:26.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.716+0000 7fd3fffff700 1 --2- 172.21.15.47:0/1859591895 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd40006b9c0 0x7fd4001107b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:26.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.716+0000 7fd40492a700 1 --2- 172.21.15.47:0/1859591895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd40006b0f0 0x7fd4001102c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:26.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.716+0000 7fd40512b700 1 -- 172.21.15.47:0/1859591895 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd40006b9c0 msgr2=0x7fd4001107b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:26.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.716+0000 7fd40512b700 1 --2- 172.21.15.47:0/1859591895 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd40006b9c0 0x7fd4001107b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:26.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.716+0000 7fd40512b700 1 -- 172.21.15.47:0/1859591895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd40006b0f0 msgr2=0x7fd4001102c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:26.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.716+0000 7fd40512b700 1 --2- 172.21.15.47:0/1859591895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd40006b0f0 0x7fd4001102c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:26.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.716+0000 7fd40512b700 1 -- 172.21.15.47:0/1859591895 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f00097c0 con 0x7fd4001044c0 2024-02-23T00:47:26.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.716+0000 7fd40512b700 1 --2- 172.21.15.47:0/1859591895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4001044c0 0x7fd40010a460 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 rx=0x7fd3f800e940 tx=0x7fd3f800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:26.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.716+0000 7fd3fdffb700 10 client.?.objecter ms_handle_connect 0x7fd4001044c0 2024-02-23T00:47:26.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.716+0000 7fd3fdffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:47:26.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.716+0000 7fd3fdffb700 1 -- 172.21.15.47:0/1859591895 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f8004620 con 0x7fd4001044c0 2024-02-23T00:47:26.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.716+0000 7fd406b8e700 1 -- 172.21.15.47:0/1859591895 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd40010af00 con 0x7fd4001044c0 2024-02-23T00:47:26.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.716+0000 7fd3fdffb700 1 -- 172.21.15.47:0/1859591895 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fd3f800f040 con 0x7fd4001044c0 2024-02-23T00:47:26.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.716+0000 7fd3fdffb700 1 -- 172.21.15.47:0/1859591895 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f80146f0 con 0x7fd4001044c0 2024-02-23T00:47:26.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.717+0000 7fd406b8e700 10 client.30921.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:47:26.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.717+0000 7fd406b8e700 1 -- 172.21.15.47:0/1859591895 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4001170f0 con 0x7fd4001044c0 2024-02-23T00:47:26.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.717+0000 7fd406b8e700 1 librados: init done 2024-02-23T00:47:26.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.717+0000 7fd406b8e700 1 -- 172.21.15.47:0/1859591895 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd40005ee80 con 0x7fd4001044c0 2024-02-23T00:47:26.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.718+0000 7fd3fdffb700 1 -- 172.21.15.47:0/1859591895 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fd3f8007b40 con 0x7fd4001044c0 2024-02-23T00:47:26.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.718+0000 7fd3fdffb700 1 --2- 172.21.15.47:0/1859591895 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd3e8062cc0 0x7fd3e8065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:26.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.718+0000 7fd3fdffb700 1 -- 172.21.15.47:0/1859591895 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(374..374 src has 48..374) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fd3f807d1f0 con 0x7fd4001044c0 2024-02-23T00:47:26.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.718+0000 7fd3fdffb700 10 client.30921.objecter ms_dispatch 0x7fd400001550 osd_map(374..374 src has 48..374) v4 2024-02-23T00:47:26.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.719+0000 7fd3fdffb700 3 client.30921.objecter handle_osd_map got epochs [374,374] > 0 2024-02-23T00:47:26.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.719+0000 7fd3fdffb700 3 client.30921.objecter handle_osd_map decoding full epoch 374 2024-02-23T00:47:26.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.719+0000 7fd40492a700 1 --2- 172.21.15.47:0/1859591895 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd3e8062cc0 0x7fd3e8065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:26.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.719+0000 7fd3fdffb700 20 client.30921.objecter dump_active .. 0 homeless 2024-02-23T00:47:26.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.719+0000 7fd3fdffb700 10 client.30921.objecter ms_handle_connect 0x7fd3e8062cc0 2024-02-23T00:47:26.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.719+0000 7fd40492a700 1 --2- 172.21.15.47:0/1859591895 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd3e8062cc0 0x7fd3e8065180 secure :-1 s=READY pgs=3049 cs=0 l=1 rev1=1 rx=0x7fd3f0004e00 tx=0x7fd3f0004d10).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:26.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.723+0000 7fd3fdffb700 1 -- 172.21.15.47:0/1859591895 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fd3f8049f60 con 0x7fd4001044c0 2024-02-23T00:47:26.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:26.922+0000 7fd406b8e700 1 -- 172.21.15.47:0/1859591895 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd out", "ids": ["2"]} v 0) v1 -- 0x7fd40010ba50 con 0x7fd4001044c0 2024-02-23T00:47:26.936 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:47:26.939 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:26.940 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:47:27.041 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:47:27.069 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:27.107 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:47:27.110 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:47:26.922+0000 7f3c89cf1700 -1 mon.a@0(leader).osd e374 definitely_dead 0 2024-02-23T00:47:27.110 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:27.111 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:47:27.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:27.181+0000 7fd3fdffb700 1 -- 172.21.15.47:0/1859591895 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd out", "ids": ["2"]}]=0 marked out osd.2. v375) v1 ==== 87+0+0 (secure 0 0 0) 0x7fd3f804c810 con 0x7fd4001044c0 2024-02-23T00:47:27.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:27.188+0000 7fd406b8e700 10 librados: watch_flush enter 2024-02-23T00:47:27.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:27.205+0000 7fd406b8e700 10 librados: watch_flush exit 2024-02-23T00:47:27.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:27.205+0000 7fd406b8e700 20 client.30921.objecter shutdown clearing up homeless session... 2024-02-23T00:47:27.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:27.205+0000 7fd406b8e700 10 client.30921.objecter successfully canceled tick 2024-02-23T00:47:27.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:27.208+0000 7fd406b8e700 1 -- 172.21.15.47:0/1859591895 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd3e8062cc0 msgr2=0x7fd3e8065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:27.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:27.208+0000 7fd406b8e700 1 --2- 172.21.15.47:0/1859591895 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd3e8062cc0 0x7fd3e8065180 secure :-1 s=READY pgs=3049 cs=0 l=1 rev1=1 rx=0x7fd3f0004e00 tx=0x7fd3f0004d10).stop 2024-02-23T00:47:27.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:27.208+0000 7fd406b8e700 1 -- 172.21.15.47:0/1859591895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4001044c0 msgr2=0x7fd40010a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:27.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:27.208+0000 7fd406b8e700 1 --2- 172.21.15.47:0/1859591895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4001044c0 0x7fd40010a460 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 rx=0x7fd3f800e940 tx=0x7fd3f800b040).stop 2024-02-23T00:47:27.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:27.208+0000 7fd406b8e700 1 -- 172.21.15.47:0/1859591895 shutdown_connections 2024-02-23T00:47:27.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:27.208+0000 7fd406b8e700 1 --2- 172.21.15.47:0/1859591895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4001044c0 0x7fd40010a460 unknown :-1 s=CLOSED pgs=2517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:27.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:27.208+0000 7fd406b8e700 1 --2- 172.21.15.47:0/1859591895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd40006b0f0 0x7fd4001102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:27.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:27.208+0000 7fd406b8e700 1 --2- 172.21.15.47:0/1859591895 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd3e8062cc0 0x7fd3e8065180 unknown :-1 s=CLOSED pgs=3049 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:27.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:27.208+0000 7fd406b8e700 1 --2- 172.21.15.47:0/1859591895 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd40006b9c0 0x7fd4001107b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:27.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:27.208+0000 7fd406b8e700 1 -- 172.21.15.47:0/1859591895 >> 172.21.15.47:0/1859591895 conn(0x7fd4000f3800 msgr2=0x7fd4000f4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:27.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:27.209+0000 7fd406b8e700 1 -- 172.21.15.47:0/1859591895 shutdown_connections 2024-02-23T00:47:27.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:27.209+0000 7fd406b8e700 1 -- 172.21.15.47:0/1859591895 wait complete. 2024-02-23T00:47:27.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:27.209+0000 7fd406b8e700 1 librados: shutdown 2024-02-23T00:47:27.211 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:47:27.212 INFO:teuthology.orchestra.run.smithi047.stderr:marked out osd.2. 2024-02-23T00:47:27.242 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:27.441 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:47:27.443 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:26.257603+0000 osd.6 (osd.6) 151 : cluster [DBG] 62.1c deep-scrub starts 2024-02-23T00:47:27.443 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:26.267284+0000 osd.6 (osd.6) 152 : cluster [DBG] 62.1c deep-scrub ok 2024-02-23T00:47:27.443 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:26.301819+0000 mon.a (mon.0) 2378 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:47:27.443 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:26.351975+0000 mon.a (mon.0) 2379 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:47:27.444 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:26.426962+0000 osd.7 (osd.7) 195 : cluster [DBG] 62.6 scrub starts 2024-02-23T00:47:27.444 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:26.428475+0000 osd.7 (osd.7) 196 : cluster [DBG] 62.6 scrub ok 2024-02-23T00:47:27.444 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:26.922846+0000 mon.a (mon.0) 2380 : audit [INF] from='client.? 172.21.15.47:0/1859591895' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["2"]}]: dispatch 2024-02-23T00:47:27.444 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:26.922928+0000 mon.a (mon.0) 2381 : cluster [INF] Client client.admin marked osd.2 out, while it was still marked up 2024-02-23T00:47:27.444 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:27.177973+0000 mon.a (mon.0) 2382 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi047-82386-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:47:27.444 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:27.178087+0000 mon.a (mon.0) 2383 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:47:27.444 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:27.178151+0000 mon.a (mon.0) 2384 : audit [INF] from='client.? 172.21.15.47:0/1859591895' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["2"]}]': finished 2024-02-23T00:47:27.444 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:27.178215+0000 mon.a (mon.0) 2385 : cluster [DBG] osdmap e375: 8 total, 8 up, 7 in 2024-02-23T00:47:27.444 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:27.182903+0000 mon.c (mon.2) 285 : audit [INF] from='client.? 172.21.15.47:0/3348293548' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi047-82386-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi047-82386-63"}]: dispatch 2024-02-23T00:47:27.444 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:27.197651+0000 mon.a (mon.0) 2386 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi047-82386-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi047-82386-63"}]: dispatch 2024-02-23T00:47:27.444 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:27.198244+0000 mon.a (mon.0) 2387 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-47"}]: dispatch 2024-02-23T00:47:27.445 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:26.257603+0000 osd.6 (osd.6) 151 : cluster [DBG] 62.1c deep-scrub starts 2024-02-23T00:47:27.445 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:26.267284+0000 osd.6 (osd.6) 152 : cluster [DBG] 62.1c deep-scrub ok 2024-02-23T00:47:27.445 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:26.301819+0000 mon.a (mon.0) 2378 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:47:27.445 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:26.351975+0000 mon.a (mon.0) 2379 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:47:27.445 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:26.426962+0000 osd.7 (osd.7) 195 : cluster [DBG] 62.6 scrub starts 2024-02-23T00:47:27.445 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:26.428475+0000 osd.7 (osd.7) 196 : cluster [DBG] 62.6 scrub ok 2024-02-23T00:47:27.445 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:26.922846+0000 mon.a (mon.0) 2380 : audit [INF] from='client.? 172.21.15.47:0/1859591895' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["2"]}]: dispatch 2024-02-23T00:47:27.445 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:26.922928+0000 mon.a (mon.0) 2381 : cluster [INF] Client client.admin marked osd.2 out, while it was still marked up 2024-02-23T00:47:27.445 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:27.177973+0000 mon.a (mon.0) 2382 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi047-82386-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:47:27.445 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:27.178087+0000 mon.a (mon.0) 2383 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:47:27.445 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:27.178151+0000 mon.a (mon.0) 2384 : audit [INF] from='client.? 172.21.15.47:0/1859591895' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["2"]}]': finished 2024-02-23T00:47:27.445 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:27.178215+0000 mon.a (mon.0) 2385 : cluster [DBG] osdmap e375: 8 total, 8 up, 7 in 2024-02-23T00:47:27.446 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:27.182903+0000 mon.c (mon.2) 285 : audit [INF] from='client.? 172.21.15.47:0/3348293548' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi047-82386-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi047-82386-63"}]: dispatch 2024-02-23T00:47:27.446 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:27.197651+0000 mon.a (mon.0) 2386 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi047-82386-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi047-82386-63"}]: dispatch 2024-02-23T00:47:27.446 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:27.198244+0000 mon.a (mon.0) 2387 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-47"}]: dispatch 2024-02-23T00:47:27.446 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:27.447 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:47:27.547 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:47:27.577 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:26.257603+0000 osd.6 (osd.6) 151 : cluster [DBG] 62.1c deep-scrub starts 2024-02-23T00:47:27.578 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:26.267284+0000 osd.6 (osd.6) 152 : cluster [DBG] 62.1c deep-scrub ok 2024-02-23T00:47:27.578 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:26.301819+0000 mon.a (mon.0) 2378 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:47:27.578 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:26.351975+0000 mon.a (mon.0) 2379 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:47:27.578 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:26.426962+0000 osd.7 (osd.7) 195 : cluster [DBG] 62.6 scrub starts 2024-02-23T00:47:27.578 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:26.428475+0000 osd.7 (osd.7) 196 : cluster [DBG] 62.6 scrub ok 2024-02-23T00:47:27.578 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:26.922846+0000 mon.a (mon.0) 2380 : audit [INF] from='client.? 172.21.15.47:0/1859591895' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["2"]}]: dispatch 2024-02-23T00:47:27.578 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:26.922928+0000 mon.a (mon.0) 2381 : cluster [INF] Client client.admin marked osd.2 out, while it was still marked up 2024-02-23T00:47:27.578 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:27.177973+0000 mon.a (mon.0) 2382 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi047-82386-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:47:27.578 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:27.178087+0000 mon.a (mon.0) 2383 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:47:27.578 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:27.178151+0000 mon.a (mon.0) 2384 : audit [INF] from='client.? 172.21.15.47:0/1859591895' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["2"]}]': finished 2024-02-23T00:47:27.578 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:27.178215+0000 mon.a (mon.0) 2385 : cluster [DBG] osdmap e375: 8 total, 8 up, 7 in 2024-02-23T00:47:27.578 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:27.182903+0000 mon.c (mon.2) 285 : audit [INF] from='client.? 172.21.15.47:0/3348293548' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi047-82386-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi047-82386-63"}]: dispatch 2024-02-23T00:47:27.579 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:27.197651+0000 mon.a (mon.0) 2386 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi047-82386-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi047-82386-63"}]: dispatch 2024-02-23T00:47:27.579 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:27.198244+0000 mon.a (mon.0) 2387 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-47"}]: dispatch 2024-02-23T00:47:27.580 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:27.601 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:47:27.604 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:27.604 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:47:27.705 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:47:27.735 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:27.755 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:47:27.758 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:27.758 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:47:27.859 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:47:27.888 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:28.508 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:47:28.658 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:27.226347+0000 osd.6 (osd.6) 153 : cluster [DBG] 62.17 deep-scrub starts 2024-02-23T00:47:28.658 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:27.229914+0000 osd.6 (osd.6) 154 : cluster [DBG] 62.17 deep-scrub ok 2024-02-23T00:47:28.658 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:27.274598+0000 mgr.y (mgr.14517) 351 : cluster [DBG] pgmap v538: 216 pgs: 1 active+remapped+backfill_toofull, 42 peering, 173 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1/1086 objects misplaced (0.092%) 2024-02-23T00:47:28.659 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:27.345236+0000 osd.4 (osd.4) 157 : cluster [DBG] 2.f scrub starts 2024-02-23T00:47:28.659 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:27.346533+0000 osd.4 (osd.4) 158 : cluster [DBG] 2.f scrub ok 2024-02-23T00:47:28.659 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:28.175956+0000 mon.a (mon.0) 2388 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:47:28.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:27.226347+0000 osd.6 (osd.6) 153 : cluster [DBG] 62.17 deep-scrub starts 2024-02-23T00:47:28.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:27.229914+0000 osd.6 (osd.6) 154 : cluster [DBG] 62.17 deep-scrub ok 2024-02-23T00:47:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:27.274598+0000 mgr.y (mgr.14517) 351 : cluster [DBG] pgmap v538: 216 pgs: 1 active+remapped+backfill_toofull, 42 peering, 173 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1/1086 objects misplaced (0.092%) 2024-02-23T00:47:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:27.345236+0000 osd.4 (osd.4) 157 : cluster [DBG] 2.f scrub starts 2024-02-23T00:47:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:27.346533+0000 osd.4 (osd.4) 158 : cluster [DBG] 2.f scrub ok 2024-02-23T00:47:28.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:28.175956+0000 mon.a (mon.0) 2388 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:47:28.977 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:27.226347+0000 osd.6 (osd.6) 153 : cluster [DBG] 62.17 deep-scrub starts 2024-02-23T00:47:28.977 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:27.229914+0000 osd.6 (osd.6) 154 : cluster [DBG] 62.17 deep-scrub ok 2024-02-23T00:47:28.977 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:27.274598+0000 mgr.y (mgr.14517) 351 : cluster [DBG] pgmap v538: 216 pgs: 1 active+remapped+backfill_toofull, 42 peering, 173 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1/1086 objects misplaced (0.092%) 2024-02-23T00:47:28.977 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:27.345236+0000 osd.4 (osd.4) 157 : cluster [DBG] 2.f scrub starts 2024-02-23T00:47:28.977 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:27.346533+0000 osd.4 (osd.4) 158 : cluster [DBG] 2.f scrub ok 2024-02-23T00:47:28.978 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:28.175956+0000 mon.a (mon.0) 2388 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:47:29.015 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:47:29.039 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:47:29.044 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:29.045 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:47:29.146 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:47:29.172 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:29.207 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:47:29.212 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:29.213 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:47:29.313 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:47:29.346 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:29.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.565+0000 7f1eaff32700 1 -- 172.21.15.47:0/2971257338 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1ea80f6a10 msgr2=0x7f1ea80f6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:29.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.565+0000 7f1eaff32700 1 --2- 172.21.15.47:0/2971257338 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1ea80f6a10 0x7f1ea80f6df0 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 rx=0x7f1ea40048a0 tx=0x7f1ea4009cb0).stop 2024-02-23T00:47:29.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.565+0000 7f1eaff32700 1 -- 172.21.15.47:0/2971257338 shutdown_connections 2024-02-23T00:47:29.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.565+0000 7f1eaff32700 1 --2- 172.21.15.47:0/2971257338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ea8104570 0x7f1ea8106960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:29.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.565+0000 7f1eaff32700 1 --2- 172.21.15.47:0/2971257338 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1ea80f6a10 0x7f1ea80f6df0 unknown :-1 s=CLOSED pgs=2650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:29.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.565+0000 7f1eaff32700 1 --2- 172.21.15.47:0/2971257338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ea80f72e0 0x7f1ea8104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:29.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.565+0000 7f1eaff32700 1 -- 172.21.15.47:0/2971257338 >> 172.21.15.47:0/2971257338 conn(0x7f1ea80f26f0 msgr2=0x7f1ea80f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:29.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.566+0000 7f1eaff32700 1 -- 172.21.15.47:0/2971257338 shutdown_connections 2024-02-23T00:47:29.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.566+0000 7f1eaff32700 1 -- 172.21.15.47:0/2971257338 wait complete. 2024-02-23T00:47:29.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.566+0000 7f1eaff32700 1 librados: starting msgr at 2024-02-23T00:47:29.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.566+0000 7f1eaff32700 1 librados: starting objecter 2024-02-23T00:47:29.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.566+0000 7f1eaff32700 1 Processor -- start 2024-02-23T00:47:29.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.566+0000 7f1eaff32700 1 -- start start 2024-02-23T00:47:29.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.566+0000 7f1eaff32700 1 librados: setting wanted keys 2024-02-23T00:47:29.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.566+0000 7f1eaff32700 1 librados: calling monclient init 2024-02-23T00:47:29.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.566+0000 7f1eaff32700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ea80f6a10 0x7f1ea81101e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:29.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.566+0000 7f1eaff32700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1ea80f72e0 0x7f1ea81106d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:29.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.566+0000 7f1eaff32700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ea8104570 0x7f1ea810a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:29.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.566+0000 7f1eaff32700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea810a870 con 0x7f1ea80f6a10 2024-02-23T00:47:29.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.566+0000 7f1eaff32700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea810a9e0 con 0x7f1ea8104570 2024-02-23T00:47:29.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.566+0000 7f1eaff32700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea810aba0 con 0x7f1ea80f72e0 2024-02-23T00:47:29.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.567+0000 7f1ead4cd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1ea80f72e0 0x7f1ea81106d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:29.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.567+0000 7f1ead4cd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1ea80f72e0 0x7f1ea81106d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:55416/0 (socket says 172.21.15.47:55416) 2024-02-23T00:47:29.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.567+0000 7f1ead4cd700 1 -- 172.21.15.47:0/245160150 learned_addr learned my addr 172.21.15.47:0/245160150 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:47:29.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.567+0000 7f1eadcce700 1 --2- 172.21.15.47:0/245160150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ea80f6a10 0x7f1ea81101e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:29.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.567+0000 7f1eae4cf700 1 --2- 172.21.15.47:0/245160150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ea8104570 0x7f1ea810a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:29.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.567+0000 7f1ead4cd700 1 -- 172.21.15.47:0/245160150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ea8104570 msgr2=0x7f1ea810a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:29.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.567+0000 7f1ead4cd700 1 --2- 172.21.15.47:0/245160150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ea8104570 0x7f1ea810a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:29.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.567+0000 7f1ead4cd700 1 -- 172.21.15.47:0/245160150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ea80f6a10 msgr2=0x7f1ea81101e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:29.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.567+0000 7f1ead4cd700 1 --2- 172.21.15.47:0/245160150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ea80f6a10 0x7f1ea81101e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:29.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.567+0000 7f1ead4cd700 1 -- 172.21.15.47:0/245160150 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea4005040 con 0x7f1ea80f72e0 2024-02-23T00:47:29.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.567+0000 7f1ead4cd700 1 --2- 172.21.15.47:0/245160150 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1ea80f72e0 0x7f1ea81106d0 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 rx=0x7f1e9800e970 tx=0x7f1e9800b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:29.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.567+0000 7f1e9effd700 10 client.?.objecter ms_handle_connect 0x7f1ea80f72e0 2024-02-23T00:47:29.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.567+0000 7f1e9effd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:47:29.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.567+0000 7f1e9effd700 1 -- 172.21.15.47:0/245160150 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e98004660 con 0x7f1ea80f72e0 2024-02-23T00:47:29.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.567+0000 7f1e9effd700 1 -- 172.21.15.47:0/245160150 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f1e9800f040 con 0x7f1ea80f72e0 2024-02-23T00:47:29.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.567+0000 7f1e9effd700 1 -- 172.21.15.47:0/245160150 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e980146f0 con 0x7f1ea80f72e0 2024-02-23T00:47:29.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.567+0000 7f1eaff32700 1 -- 172.21.15.47:0/245160150 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea810ae80 con 0x7f1ea80f72e0 2024-02-23T00:47:29.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.567+0000 7f1eaff32700 10 client.41255.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:47:29.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.567+0000 7f1eaff32700 1 -- 172.21.15.47:0/245160150 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea8117010 con 0x7f1ea80f72e0 2024-02-23T00:47:29.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.567+0000 7f1eaff32700 1 librados: init done 2024-02-23T00:47:29.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.568+0000 7f1eaff32700 1 -- 172.21.15.47:0/245160150 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ea805f990 con 0x7f1ea80f72e0 2024-02-23T00:47:29.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.569+0000 7f1e9effd700 1 -- 172.21.15.47:0/245160150 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f1e98007b40 con 0x7f1ea80f72e0 2024-02-23T00:47:29.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.569+0000 7f1e9effd700 1 --2- 172.21.15.47:0/245160150 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1e94062cc0 0x7f1e94065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:29.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.569+0000 7f1e9effd700 1 -- 172.21.15.47:0/245160150 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(377..377 src has 48..377) v4 ==== 9581+0+0 (secure 0 0 0) 0x7f1e9807d200 con 0x7f1ea80f72e0 2024-02-23T00:47:29.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.569+0000 7f1e9effd700 10 client.41255.objecter ms_dispatch 0x7f1ea8001550 osd_map(377..377 src has 48..377) v4 2024-02-23T00:47:29.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.569+0000 7f1e9effd700 3 client.41255.objecter handle_osd_map got epochs [377,377] > 0 2024-02-23T00:47:29.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.569+0000 7f1e9effd700 3 client.41255.objecter handle_osd_map decoding full epoch 377 2024-02-23T00:47:29.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.569+0000 7f1eadcce700 1 --2- 172.21.15.47:0/245160150 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1e94062cc0 0x7f1e94065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:29.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.570+0000 7f1e9effd700 20 client.41255.objecter dump_active .. 0 homeless 2024-02-23T00:47:29.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.570+0000 7f1e9effd700 10 client.41255.objecter ms_handle_connect 0x7f1e94062cc0 2024-02-23T00:47:29.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.570+0000 7f1eadcce700 1 --2- 172.21.15.47:0/245160150 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1e94062cc0 0x7f1e94065180 secure :-1 s=READY pgs=3050 cs=0 l=1 rev1=1 rx=0x7f1ea4005010 tx=0x7f1ea4009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:29.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.574+0000 7f1e9effd700 1 -- 172.21.15.47:0/245160150 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f1e9804c7e0 con 0x7f1ea80f72e0 2024-02-23T00:47:29.602 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:47:29.606 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:29.607 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:47:29.647 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:47:29.708 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:47:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:29.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:29.787+0000 7f1eaff32700 1 -- 172.21.15.47:0/245160150 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7f1ea805ee80 con 0x7f1ea80f72e0 2024-02-23T00:47:29.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:28.648058+0000 mon.a (mon.0) 2389 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-47"}]': finished 2024-02-23T00:47:29.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:28.648254+0000 mon.a (mon.0) 2390 : cluster [DBG] osdmap e376: 8 total, 8 up, 7 in 2024-02-23T00:47:29.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:28.648860+0000 mon.a (mon.0) 2391 : cluster [WRN] Health check failed: Reduced data availability: 5 pgs inactive, 8 pgs peering (PG_AVAILABILITY) 2024-02-23T00:47:29.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:28.663017+0000 mon.a (mon.0) 2392 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:47:29.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:28.663417+0000 mon.a (mon.0) 2393 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-47"}]: dispatch 2024-02-23T00:47:29.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:29.181994+0000 mon.a (mon.0) 2394 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi047-82386-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi047-82386-63"}]': finished 2024-02-23T00:47:29.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:29.182146+0000 mon.a (mon.0) 2395 : cluster [DBG] osdmap e377: 8 total, 8 up, 7 in 2024-02-23T00:47:29.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:29.641443+0000 mon.a (mon.0) 2396 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "18"}]: dispatch 2024-02-23T00:47:30.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:28.648058+0000 mon.a (mon.0) 2389 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-47"}]': finished 2024-02-23T00:47:30.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:28.648254+0000 mon.a (mon.0) 2390 : cluster [DBG] osdmap e376: 8 total, 8 up, 7 in 2024-02-23T00:47:30.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:28.648860+0000 mon.a (mon.0) 2391 : cluster [WRN] Health check failed: Reduced data availability: 5 pgs inactive, 8 pgs peering (PG_AVAILABILITY) 2024-02-23T00:47:30.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:28.663017+0000 mon.a (mon.0) 2392 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:47:30.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:28.663417+0000 mon.a (mon.0) 2393 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-47"}]: dispatch 2024-02-23T00:47:30.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:29.181994+0000 mon.a (mon.0) 2394 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi047-82386-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi047-82386-63"}]': finished 2024-02-23T00:47:30.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:29.182146+0000 mon.a (mon.0) 2395 : cluster [DBG] osdmap e377: 8 total, 8 up, 7 in 2024-02-23T00:47:30.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:29.641443+0000 mon.a (mon.0) 2396 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "18"}]: dispatch 2024-02-23T00:47:30.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:28.648058+0000 mon.a (mon.0) 2389 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-47"}]': finished 2024-02-23T00:47:30.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:28.648254+0000 mon.a (mon.0) 2390 : cluster [DBG] osdmap e376: 8 total, 8 up, 7 in 2024-02-23T00:47:30.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:28.648860+0000 mon.a (mon.0) 2391 : cluster [WRN] Health check failed: Reduced data availability: 5 pgs inactive, 8 pgs peering (PG_AVAILABILITY) 2024-02-23T00:47:30.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:28.663017+0000 mon.a (mon.0) 2392 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:47:30.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:28.663417+0000 mon.a (mon.0) 2393 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-47"}]: dispatch 2024-02-23T00:47:30.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:29.181994+0000 mon.a (mon.0) 2394 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi047-82386-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi047-82386-63"}]': finished 2024-02-23T00:47:30.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:29.182146+0000 mon.a (mon.0) 2395 : cluster [DBG] osdmap e377: 8 total, 8 up, 7 in 2024-02-23T00:47:30.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:29.641443+0000 mon.a (mon.0) 2396 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "18"}]: dispatch 2024-02-23T00:47:30.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:30.169+0000 7f1e9effd700 1 -- 172.21.15.47:0/245160150 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v378) v1 ==== 87+0+0 (secure 0 0 0) 0x7f1e98019070 con 0x7f1ea80f72e0 2024-02-23T00:47:30.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:30.178+0000 7f1eaff32700 10 librados: watch_flush enter 2024-02-23T00:47:30.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:30.179+0000 7f1eaff32700 10 librados: watch_flush exit 2024-02-23T00:47:30.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:30.179+0000 7f1eaff32700 20 client.41255.objecter shutdown clearing up homeless session... 2024-02-23T00:47:30.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:30.179+0000 7f1eaff32700 10 client.41255.objecter successfully canceled tick 2024-02-23T00:47:30.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:30.179+0000 7f1eaff32700 1 -- 172.21.15.47:0/245160150 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1e94062cc0 msgr2=0x7f1e94065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:30.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:30.179+0000 7f1eaff32700 1 --2- 172.21.15.47:0/245160150 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1e94062cc0 0x7f1e94065180 secure :-1 s=READY pgs=3050 cs=0 l=1 rev1=1 rx=0x7f1ea4005010 tx=0x7f1ea4009cb0).stop 2024-02-23T00:47:30.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:30.179+0000 7f1eaff32700 1 -- 172.21.15.47:0/245160150 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1ea80f72e0 msgr2=0x7f1ea81106d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:30.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:30.179+0000 7f1eaff32700 1 --2- 172.21.15.47:0/245160150 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1ea80f72e0 0x7f1ea81106d0 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 rx=0x7f1e9800e970 tx=0x7f1e9800b040).stop 2024-02-23T00:47:30.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:30.181+0000 7f1eaff32700 1 -- 172.21.15.47:0/245160150 shutdown_connections 2024-02-23T00:47:30.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:30.181+0000 7f1eaff32700 1 --2- 172.21.15.47:0/245160150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ea8104570 0x7f1ea810a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:30.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:30.181+0000 7f1eaff32700 1 --2- 172.21.15.47:0/245160150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ea80f6a10 0x7f1ea81101e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:30.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:30.181+0000 7f1eaff32700 1 --2- 172.21.15.47:0/245160150 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1e94062cc0 0x7f1e94065180 unknown :-1 s=CLOSED pgs=3050 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:30.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:30.181+0000 7f1eaff32700 1 --2- 172.21.15.47:0/245160150 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1ea80f72e0 0x7f1ea81106d0 unknown :-1 s=CLOSED pgs=2651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:30.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:30.181+0000 7f1eaff32700 1 -- 172.21.15.47:0/245160150 >> 172.21.15.47:0/245160150 conn(0x7f1ea80f26f0 msgr2=0x7f1ea80f4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:30.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:30.181+0000 7f1eaff32700 1 -- 172.21.15.47:0/245160150 shutdown_connections 2024-02-23T00:47:30.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:30.181+0000 7f1eaff32700 1 -- 172.21.15.47:0/245160150 wait complete. 2024-02-23T00:47:30.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:30.181+0000 7f1eaff32700 1 librados: shutdown 2024-02-23T00:47:30.184 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is set 2024-02-23T00:47:30.557 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:47:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:47:30] "GET /metrics HTTP/1.1" 200 188481 "" "Prometheus/2.33.4" 2024-02-23T00:47:30.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:29.275017+0000 mgr.y (mgr.14517) 352 : cluster [DBG] pgmap v541: 191 pgs: 8 unknown, 1 active+remapped+backfill_toofull, 31 peering, 151 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1/1086 objects misplaced (0.092%) 2024-02-23T00:47:30.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:29.788743+0000 mon.c (mon.2) 286 : audit [INF] from='client.? 172.21.15.47:0/245160150' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:47:30.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:29.789070+0000 mon.a (mon.0) 2397 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:47:30.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:30.164521+0000 mon.a (mon.0) 2398 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:47:30.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:30.168783+0000 mon.a (mon.0) 2399 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "18"}]': finished 2024-02-23T00:47:30.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:30.168891+0000 mon.a (mon.0) 2400 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:47:30.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:30.168953+0000 mon.a (mon.0) 2401 : cluster [DBG] osdmap e378: 8 total, 8 up, 7 in 2024-02-23T00:47:30.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:30.175287+0000 mon.a (mon.0) 2402 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:47:31.024 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:29.275017+0000 mgr.y (mgr.14517) 352 : cluster [DBG] pgmap v541: 191 pgs: 8 unknown, 1 active+remapped+backfill_toofull, 31 peering, 151 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1/1086 objects misplaced (0.092%) 2024-02-23T00:47:31.024 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:29.788743+0000 mon.c (mon.2) 286 : audit [INF] from='client.? 172.21.15.47:0/245160150' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:47:31.024 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:29.789070+0000 mon.a (mon.0) 2397 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:47:31.024 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:30.164521+0000 mon.a (mon.0) 2398 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:47:31.024 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:30.168783+0000 mon.a (mon.0) 2399 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "18"}]': finished 2024-02-23T00:47:31.024 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:30.168891+0000 mon.a (mon.0) 2400 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:47:31.025 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:30.168953+0000 mon.a (mon.0) 2401 : cluster [DBG] osdmap e378: 8 total, 8 up, 7 in 2024-02-23T00:47:31.025 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:30.175287+0000 mon.a (mon.0) 2402 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:47:31.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:29.275017+0000 mgr.y (mgr.14517) 352 : cluster [DBG] pgmap v541: 191 pgs: 8 unknown, 1 active+remapped+backfill_toofull, 31 peering, 151 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1/1086 objects misplaced (0.092%) 2024-02-23T00:47:31.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:29.788743+0000 mon.c (mon.2) 286 : audit [INF] from='client.? 172.21.15.47:0/245160150' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:47:31.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:29.789070+0000 mon.a (mon.0) 2397 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:47:31.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:30.164521+0000 mon.a (mon.0) 2398 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:47:31.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:30.168783+0000 mon.a (mon.0) 2399 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "18"}]': finished 2024-02-23T00:47:31.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:30.168891+0000 mon.a (mon.0) 2400 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:47:31.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:30.168953+0000 mon.a (mon.0) 2401 : cluster [DBG] osdmap e378: 8 total, 8 up, 7 in 2024-02-23T00:47:31.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:30.175287+0000 mon.a (mon.0) 2402 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:47:32.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:31.170686+0000 mon.a (mon.0) 2403 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:47:32.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:31.170915+0000 mon.a (mon.0) 2404 : cluster [DBG] osdmap e379: 8 total, 8 up, 7 in 2024-02-23T00:47:32.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:31.174364+0000 mon.c (mon.2) 287 : audit [INF] from='client.? 172.21.15.47:0/3348293548' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi047-82386-63"}]: dispatch 2024-02-23T00:47:32.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:31.174814+0000 mon.a (mon.0) 2405 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi047-82386-63"}]: dispatch 2024-02-23T00:47:32.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:31.200828+0000 mon.a (mon.0) 2406 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:47:32.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:31.275671+0000 mgr.y (mgr.14517) 353 : cluster [DBG] pgmap v544: 215 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 31 peering, 151 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1/1086 objects misplaced (0.092%) 2024-02-23T00:47:32.545 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:31.170686+0000 mon.a (mon.0) 2403 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:47:32.546 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:31.170915+0000 mon.a (mon.0) 2404 : cluster [DBG] osdmap e379: 8 total, 8 up, 7 in 2024-02-23T00:47:32.546 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:31.174364+0000 mon.c (mon.2) 287 : audit [INF] from='client.? 172.21.15.47:0/3348293548' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi047-82386-63"}]: dispatch 2024-02-23T00:47:32.546 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:31.174814+0000 mon.a (mon.0) 2405 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi047-82386-63"}]: dispatch 2024-02-23T00:47:32.546 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:31.200828+0000 mon.a (mon.0) 2406 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:47:32.546 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:31.275671+0000 mgr.y (mgr.14517) 353 : cluster [DBG] pgmap v544: 215 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 31 peering, 151 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1/1086 objects misplaced (0.092%) 2024-02-23T00:47:32.546 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:31.170686+0000 mon.a (mon.0) 2403 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:47:32.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:31.170915+0000 mon.a (mon.0) 2404 : cluster [DBG] osdmap e379: 8 total, 8 up, 7 in 2024-02-23T00:47:32.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:31.174364+0000 mon.c (mon.2) 287 : audit [INF] from='client.? 172.21.15.47:0/3348293548' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi047-82386-63"}]: dispatch 2024-02-23T00:47:32.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:31.174814+0000 mon.a (mon.0) 2405 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi047-82386-63"}]: dispatch 2024-02-23T00:47:32.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:31.200828+0000 mon.a (mon.0) 2406 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:47:32.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:31.275671+0000 mgr.y (mgr.14517) 353 : cluster [DBG] pgmap v544: 215 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 31 peering, 151 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1/1086 objects misplaced (0.092%) 2024-02-23T00:47:32.943 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:47:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:47:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:47:32.944 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:47:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:47:33.382 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-23T00:47:33.383 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-23T00:47:33.383 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-02-23T00:47:33.383 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight-by-utilization 1000 3.0 2024-02-23T00:47:33.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:32.175635+0000 mon.a (mon.0) 2407 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi047-82386-63"}]': finished 2024-02-23T00:47:33.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:32.175763+0000 mon.a (mon.0) 2408 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:47:33.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:32.175836+0000 mon.a (mon.0) 2409 : cluster [DBG] osdmap e380: 8 total, 8 up, 7 in 2024-02-23T00:47:33.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:32.176570+0000 mon.c (mon.2) 288 : audit [INF] from='client.? 172.21.15.47:0/3348293548' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi047-82386-63"}]: dispatch 2024-02-23T00:47:33.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:32.177611+0000 mon.a (mon.0) 2410 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi047-82386-63"}]: dispatch 2024-02-23T00:47:33.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:32.178219+0000 mon.a (mon.0) 2411 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-49"}]: dispatch 2024-02-23T00:47:33.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:32.558140+0000 mgr.y (mgr.14517) 354 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:47:33.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:32.175635+0000 mon.a (mon.0) 2407 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi047-82386-63"}]': finished 2024-02-23T00:47:33.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:32.175763+0000 mon.a (mon.0) 2408 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:47:33.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:32.175836+0000 mon.a (mon.0) 2409 : cluster [DBG] osdmap e380: 8 total, 8 up, 7 in 2024-02-23T00:47:33.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:32.176570+0000 mon.c (mon.2) 288 : audit [INF] from='client.? 172.21.15.47:0/3348293548' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi047-82386-63"}]: dispatch 2024-02-23T00:47:33.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:32.177611+0000 mon.a (mon.0) 2410 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi047-82386-63"}]: dispatch 2024-02-23T00:47:33.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:32.178219+0000 mon.a (mon.0) 2411 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-49"}]: dispatch 2024-02-23T00:47:33.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:32.558140+0000 mgr.y (mgr.14517) 354 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:47:33.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:32.175635+0000 mon.a (mon.0) 2407 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi047-82386-63"}]': finished 2024-02-23T00:47:33.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:32.175763+0000 mon.a (mon.0) 2408 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:47:33.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:32.175836+0000 mon.a (mon.0) 2409 : cluster [DBG] osdmap e380: 8 total, 8 up, 7 in 2024-02-23T00:47:33.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:32.176570+0000 mon.c (mon.2) 288 : audit [INF] from='client.? 172.21.15.47:0/3348293548' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi047-82386-63"}]: dispatch 2024-02-23T00:47:33.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:32.177611+0000 mon.a (mon.0) 2410 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi047-82386-63"}]: dispatch 2024-02-23T00:47:33.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:32.178219+0000 mon.a (mon.0) 2411 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-49"}]: dispatch 2024-02-23T00:47:33.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:32.558140+0000 mgr.y (mgr.14517) 354 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:47:34.196 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:47:34.200 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:34.201 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:47:34.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.214+0000 7fae1248f700 1 -- 172.21.15.47:0/1359634724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae0c0fe510 msgr2=0x7fae0c0f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:34.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.214+0000 7fae1248f700 1 --2- 172.21.15.47:0/1359634724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae0c0fe510 0x7fae0c0f7d40 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 rx=0x7fae00004870 tx=0x7fae00009cb0).stop 2024-02-23T00:47:34.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.215+0000 7fae1248f700 1 -- 172.21.15.47:0/1359634724 shutdown_connections 2024-02-23T00:47:34.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.215+0000 7fae1248f700 1 --2- 172.21.15.47:0/1359634724 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fae0c0f8420 0x7fae0c104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:34.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.215+0000 7fae1248f700 1 --2- 172.21.15.47:0/1359634724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae0c0fdbb0 0x7fae0c0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:34.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.215+0000 7fae1248f700 1 --2- 172.21.15.47:0/1359634724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae0c0fe510 0x7fae0c0f7d40 unknown :-1 s=CLOSED pgs=2520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:34.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.215+0000 7fae1248f700 1 -- 172.21.15.47:0/1359634724 >> 172.21.15.47:0/1359634724 conn(0x7fae0c0f3800 msgr2=0x7fae0c0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:34.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.215+0000 7fae1248f700 1 -- 172.21.15.47:0/1359634724 shutdown_connections 2024-02-23T00:47:34.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.215+0000 7fae1248f700 1 -- 172.21.15.47:0/1359634724 wait complete. 2024-02-23T00:47:34.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.215+0000 7fae1248f700 1 librados: starting msgr at 2024-02-23T00:47:34.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.215+0000 7fae1248f700 1 librados: starting objecter 2024-02-23T00:47:34.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.215+0000 7fae1248f700 1 Processor -- start 2024-02-23T00:47:34.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.215+0000 7fae1248f700 1 -- start start 2024-02-23T00:47:34.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.215+0000 7fae1248f700 1 librados: setting wanted keys 2024-02-23T00:47:34.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.215+0000 7fae1248f700 1 librados: calling monclient init 2024-02-23T00:47:34.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.216+0000 7fae1248f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae0c0f8420 0x7fae0c06b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:34.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.216+0000 7fae1248f700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fae0c0fdbb0 0x7fae0c06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:34.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.216+0000 7fae1248f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae0c0fe510 0x7fae0c06bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:34.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.216+0000 7fae1248f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c1162e0 con 0x7fae0c0fe510 2024-02-23T00:47:34.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.216+0000 7fae1248f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c116450 con 0x7fae0c0f8420 2024-02-23T00:47:34.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.216+0000 7fae1248f700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c116610 con 0x7fae0c0fdbb0 2024-02-23T00:47:34.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.216+0000 7fae0b7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fae0c0fdbb0 0x7fae0c06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:34.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.216+0000 7fae10a2c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae0c0fe510 0x7fae0c06bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:34.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.216+0000 7fae0b7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fae0c0fdbb0 0x7fae0c06b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:55462/0 (socket says 172.21.15.47:55462) 2024-02-23T00:47:34.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.216+0000 7fae0b7fe700 1 -- 172.21.15.47:0/872603830 learned_addr learned my addr 172.21.15.47:0/872603830 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:47:34.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.216+0000 7fae0bfff700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae0c0f8420 0x7fae0c06b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:34.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.216+0000 7fae10a2c700 1 -- 172.21.15.47:0/872603830 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fae0c0fdbb0 msgr2=0x7fae0c06b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:34.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.216+0000 7fae10a2c700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fae0c0fdbb0 0x7fae0c06b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:34.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.216+0000 7fae10a2c700 1 -- 172.21.15.47:0/872603830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae0c0f8420 msgr2=0x7fae0c06b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:34.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.216+0000 7fae10a2c700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae0c0f8420 0x7fae0c06b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:34.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.216+0000 7fae10a2c700 1 -- 172.21.15.47:0/872603830 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae040097c0 con 0x7fae0c0fe510 2024-02-23T00:47:34.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.216+0000 7fae10a2c700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae0c0fe510 0x7fae0c06bab0 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 rx=0x7fae04001cb0 tx=0x7fae04004c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:34.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.217+0000 7fae097fa700 10 client.?.objecter ms_handle_connect 0x7fae0c0fe510 2024-02-23T00:47:34.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.217+0000 7fae097fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:47:34.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.217+0000 7fae097fa700 1 -- 172.21.15.47:0/872603830 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae040079b0 con 0x7fae0c0fe510 2024-02-23T00:47:34.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.217+0000 7fae097fa700 1 -- 172.21.15.47:0/872603830 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fae040090d0 con 0x7fae0c0fe510 2024-02-23T00:47:34.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.217+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae00005040 con 0x7fae0c0fe510 2024-02-23T00:47:34.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.217+0000 7fae097fa700 1 -- 172.21.15.47:0/872603830 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0400fba0 con 0x7fae0c0fe510 2024-02-23T00:47:34.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.217+0000 7fae1248f700 10 client.30939.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:47:34.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.217+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae0c116ca0 con 0x7fae0c0fe510 2024-02-23T00:47:34.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.217+0000 7fae1248f700 1 librados: init done 2024-02-23T00:47:34.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.218+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fae0c106e60 con 0x7fae0c0fe510 2024-02-23T00:47:34.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.219+0000 7fae097fa700 1 -- 172.21.15.47:0/872603830 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fae04007b10 con 0x7fae0c0fe510 2024-02-23T00:47:34.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.219+0000 7fae097fa700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fadf4062cc0 0x7fadf4065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:34.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.219+0000 7fae097fa700 1 -- 172.21.15.47:0/872603830 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(382..382 src has 48..382) v4 ==== 9411+0+0 (secure 0 0 0) 0x7fae0407ddc0 con 0x7fae0c0fe510 2024-02-23T00:47:34.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.219+0000 7fae097fa700 10 client.30939.objecter ms_dispatch 0x7fae0c001550 osd_map(382..382 src has 48..382) v4 2024-02-23T00:47:34.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.219+0000 7fae097fa700 3 client.30939.objecter handle_osd_map got epochs [382,382] > 0 2024-02-23T00:47:34.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.219+0000 7fae097fa700 3 client.30939.objecter handle_osd_map decoding full epoch 382 2024-02-23T00:47:34.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.219+0000 7fae0bfff700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fadf4062cc0 0x7fadf4065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:34.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.219+0000 7fae097fa700 20 client.30939.objecter dump_active .. 0 homeless 2024-02-23T00:47:34.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.219+0000 7fae097fa700 1 -- 172.21.15.47:0/872603830 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v382) v1 ==== 54+0+16 (secure 0 0 0) 0x7fae0407e2a0 con 0x7fae0c0fe510 2024-02-23T00:47:34.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.220+0000 7fae0bfff700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fadf4062cc0 0x7fadf4065180 secure :-1 s=READY pgs=3052 cs=0 l=1 rev1=1 rx=0x7fadfc004e60 tx=0x7fadfc00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:34.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.220+0000 7fae097fa700 10 client.30939.objecter ms_handle_connect 0x7fadf4062cc0 2024-02-23T00:47:34.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.220+0000 7fae1248f700 10 client.30939.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:34.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.220+0000 7fae1248f700 15 client.30939.objecter _session_command_op_assign -1 1 2024-02-23T00:47:34.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.220+0000 7fae1248f700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fae0c0fee00 0x7fae0c0ff1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:34.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.220+0000 7fae1248f700 20 client.30939.objecter _get_session s=0x7fae0c116890 osd=0 3 2024-02-23T00:47:34.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.220+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c116890 osd=0 3 2024-02-23T00:47:34.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.220+0000 7fae1248f700 20 client.30939.objecter _get_session s=0x7fae0c116890 osd=0 3 2024-02-23T00:47:34.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.220+0000 7fae1248f700 15 client.30939.objecter _session_command_op_remove -1 1 2024-02-23T00:47:34.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.220+0000 7fae1248f700 20 client.30939.objecter get_session s=0x7fae0c116890 osd=0 3 2024-02-23T00:47:34.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.220+0000 7fae1248f700 15 client.30939.objecter _session_command_op_assign 0 1 2024-02-23T00:47:34.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.220+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c116890 osd=0 4 2024-02-23T00:47:34.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.220+0000 7fae1248f700 10 client.30939.objecter _send_command 1 2024-02-23T00:47:34.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.220+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fae0c06c5b0 con 0x7fae0c0fee00 2024-02-23T00:47:34.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.220+0000 7fae0b7fe700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fae0c0fee00 0x7fae0c0ff1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:34.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.220+0000 7fae097fa700 10 client.30939.objecter ms_handle_connect 0x7fae0c0fee00 2024-02-23T00:47:34.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.220+0000 7fae0b7fe700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fae0c0fee00 0x7fae0c0ff1e0 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:34.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.221+0000 7fae097fa700 1 -- 172.21.15.47:0/872603830 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fae0c06c5b0 con 0x7fae0c0fee00 2024-02-23T00:47:34.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.221+0000 7fae097fa700 10 client.30939.objecter ms_dispatch 0x7fae0c001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:47:34.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.221+0000 7fae097fa700 10 client.30939.objecter _finish_command 1 = system:0 2024-02-23T00:47:34.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.221+0000 7fae097fa700 20 client.30939.objecter put_session s=0x7fae0c116890 osd=0 4 2024-02-23T00:47:34.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.221+0000 7fae097fa700 15 client.30939.objecter _session_command_op_remove 0 1 2024-02-23T00:47:34.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.236+0000 7fae1248f700 10 client.30939.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:47:34.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.236+0000 7fae1248f700 15 client.30939.objecter _session_command_op_assign -1 2 2024-02-23T00:47:34.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.236+0000 7fae1248f700 20 client.30939.objecter _get_session s=0x7fae0c116890 osd=0 3 2024-02-23T00:47:34.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.236+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c116890 osd=0 3 2024-02-23T00:47:34.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.236+0000 7fae1248f700 20 client.30939.objecter _get_session s=0x7fae0c116890 osd=0 3 2024-02-23T00:47:34.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.236+0000 7fae1248f700 15 client.30939.objecter _session_command_op_remove -1 2 2024-02-23T00:47:34.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.236+0000 7fae1248f700 20 client.30939.objecter get_session s=0x7fae0c116890 osd=0 3 2024-02-23T00:47:34.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.236+0000 7fae1248f700 15 client.30939.objecter _session_command_op_assign 0 2 2024-02-23T00:47:34.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.236+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c116890 osd=0 4 2024-02-23T00:47:34.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.236+0000 7fae1248f700 10 client.30939.objecter _send_command 2 2024-02-23T00:47:34.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.236+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fae0c0ff910 con 0x7fae0c0fee00 2024-02-23T00:47:34.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.237+0000 7fae097fa700 1 -- 172.21.15.47:0/872603830 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fae00019040 con 0x7fae0c0fee00 2024-02-23T00:47:34.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.237+0000 7fae097fa700 10 client.30939.objecter ms_dispatch 0x7fae0c001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:47:34.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.237+0000 7fae097fa700 10 client.30939.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:47:34.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.237+0000 7fae097fa700 20 client.30939.objecter put_session s=0x7fae0c116890 osd=0 4 2024-02-23T00:47:34.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.237+0000 7fae097fa700 15 client.30939.objecter _session_command_op_remove 0 2 2024-02-23T00:47:34.240 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:47:34.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.237+0000 7fae1248f700 10 client.30939.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:34.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.237+0000 7fae1248f700 15 client.30939.objecter _session_command_op_assign -1 3 2024-02-23T00:47:34.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.237+0000 7fae1248f700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fae0c0598f0 0x7fae0c059d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:34.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.237+0000 7fae1248f700 20 client.30939.objecter _get_session s=0x7fae0c028540 osd=1 3 2024-02-23T00:47:34.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.237+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c028540 osd=1 3 2024-02-23T00:47:34.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.237+0000 7fae1248f700 20 client.30939.objecter _get_session s=0x7fae0c028540 osd=1 3 2024-02-23T00:47:34.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.237+0000 7fae1248f700 15 client.30939.objecter _session_command_op_remove -1 3 2024-02-23T00:47:34.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.237+0000 7fae1248f700 20 client.30939.objecter get_session s=0x7fae0c028540 osd=1 3 2024-02-23T00:47:34.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.237+0000 7fae1248f700 15 client.30939.objecter _session_command_op_assign 1 3 2024-02-23T00:47:34.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.237+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c028540 osd=1 4 2024-02-23T00:47:34.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.237+0000 7fae1248f700 10 client.30939.objecter _send_command 3 2024-02-23T00:47:34.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.237+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fae0c0286b0 con 0x7fae0c0598f0 2024-02-23T00:47:34.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.237+0000 7fae0bfff700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fae0c0598f0 0x7fae0c059d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:34.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.238+0000 7fae0bfff700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fae0c0598f0 0x7fae0c059d10 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:34.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.238+0000 7fae097fa700 10 client.30939.objecter ms_handle_connect 0x7fae0c0598f0 2024-02-23T00:47:34.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.238+0000 7fae097fa700 1 -- 172.21.15.47:0/872603830 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fae0c0286b0 con 0x7fae0c0598f0 2024-02-23T00:47:34.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.238+0000 7fae097fa700 10 client.30939.objecter ms_dispatch 0x7fae0c001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:47:34.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.238+0000 7fae097fa700 10 client.30939.objecter _finish_command 3 = system:0 2024-02-23T00:47:34.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.238+0000 7fae097fa700 20 client.30939.objecter put_session s=0x7fae0c028540 osd=1 4 2024-02-23T00:47:34.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.238+0000 7fae097fa700 15 client.30939.objecter _session_command_op_remove 1 3 2024-02-23T00:47:34.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.253+0000 7fae1248f700 10 client.30939.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:47:34.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.253+0000 7fae1248f700 15 client.30939.objecter _session_command_op_assign -1 4 2024-02-23T00:47:34.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.253+0000 7fae1248f700 20 client.30939.objecter _get_session s=0x7fae0c028540 osd=1 3 2024-02-23T00:47:34.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.253+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c028540 osd=1 3 2024-02-23T00:47:34.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.253+0000 7fae1248f700 20 client.30939.objecter _get_session s=0x7fae0c028540 osd=1 3 2024-02-23T00:47:34.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.253+0000 7fae1248f700 15 client.30939.objecter _session_command_op_remove -1 4 2024-02-23T00:47:34.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.253+0000 7fae1248f700 20 client.30939.objecter get_session s=0x7fae0c028540 osd=1 3 2024-02-23T00:47:34.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.253+0000 7fae1248f700 15 client.30939.objecter _session_command_op_assign 1 4 2024-02-23T00:47:34.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.253+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c028540 osd=1 4 2024-02-23T00:47:34.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.253+0000 7fae1248f700 10 client.30939.objecter _send_command 4 2024-02-23T00:47:34.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.253+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fae0c028820 con 0x7fae0c0598f0 2024-02-23T00:47:34.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.255+0000 7fae097fa700 1 -- 172.21.15.47:0/872603830 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fadfc00f040 con 0x7fae0c0598f0 2024-02-23T00:47:34.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.255+0000 7fae097fa700 10 client.30939.objecter ms_dispatch 0x7fae0c001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:47:34.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.255+0000 7fae097fa700 10 client.30939.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:47:34.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.255+0000 7fae097fa700 20 client.30939.objecter put_session s=0x7fae0c028540 osd=1 4 2024-02-23T00:47:34.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.255+0000 7fae097fa700 15 client.30939.objecter _session_command_op_remove 1 4 2024-02-23T00:47:34.258 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:47:34.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.255+0000 7fae1248f700 10 client.30939.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:34.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.255+0000 7fae1248f700 15 client.30939.objecter _session_command_op_assign -1 5 2024-02-23T00:47:34.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.255+0000 7fae1248f700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fae0c0280a0 0x7fae0c11e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:34.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.255+0000 7fae1248f700 20 client.30939.objecter _get_session s=0x7fae0c0595a0 osd=2 3 2024-02-23T00:47:34.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.255+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c0595a0 osd=2 3 2024-02-23T00:47:34.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.255+0000 7fae1248f700 20 client.30939.objecter _get_session s=0x7fae0c0595a0 osd=2 3 2024-02-23T00:47:34.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.255+0000 7fae1248f700 15 client.30939.objecter _session_command_op_remove -1 5 2024-02-23T00:47:34.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.255+0000 7fae1248f700 20 client.30939.objecter get_session s=0x7fae0c0595a0 osd=2 3 2024-02-23T00:47:34.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.255+0000 7fae1248f700 15 client.30939.objecter _session_command_op_assign 2 5 2024-02-23T00:47:34.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.255+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c0595a0 osd=2 4 2024-02-23T00:47:34.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.255+0000 7fae1248f700 10 client.30939.objecter _send_command 5 2024-02-23T00:47:34.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.255+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fae0c120e10 con 0x7fae0c0280a0 2024-02-23T00:47:34.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.255+0000 7fae0b7fe700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fae0c0280a0 0x7fae0c11e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:34.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.255+0000 7fae097fa700 10 client.30939.objecter ms_handle_connect 0x7fae0c0280a0 2024-02-23T00:47:34.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.255+0000 7fae0b7fe700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fae0c0280a0 0x7fae0c11e130 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:34.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.256+0000 7fae097fa700 1 -- 172.21.15.47:0/872603830 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fae0c120e10 con 0x7fae0c0280a0 2024-02-23T00:47:34.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.256+0000 7fae097fa700 10 client.30939.objecter ms_dispatch 0x7fae0c001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:47:34.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.256+0000 7fae097fa700 10 client.30939.objecter _finish_command 5 = system:0 2024-02-23T00:47:34.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.256+0000 7fae097fa700 20 client.30939.objecter put_session s=0x7fae0c0595a0 osd=2 4 2024-02-23T00:47:34.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.256+0000 7fae097fa700 15 client.30939.objecter _session_command_op_remove 2 5 2024-02-23T00:47:34.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.270+0000 7fae1248f700 10 client.30939.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:47:34.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.270+0000 7fae1248f700 15 client.30939.objecter _session_command_op_assign -1 6 2024-02-23T00:47:34.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.270+0000 7fae1248f700 20 client.30939.objecter _get_session s=0x7fae0c0595a0 osd=2 3 2024-02-23T00:47:34.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.270+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c0595a0 osd=2 3 2024-02-23T00:47:34.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.270+0000 7fae1248f700 20 client.30939.objecter _get_session s=0x7fae0c0595a0 osd=2 3 2024-02-23T00:47:34.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.270+0000 7fae1248f700 15 client.30939.objecter _session_command_op_remove -1 6 2024-02-23T00:47:34.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.270+0000 7fae1248f700 20 client.30939.objecter get_session s=0x7fae0c0595a0 osd=2 3 2024-02-23T00:47:34.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.270+0000 7fae1248f700 15 client.30939.objecter _session_command_op_assign 2 6 2024-02-23T00:47:34.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.270+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c0595a0 osd=2 4 2024-02-23T00:47:34.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.270+0000 7fae1248f700 10 client.30939.objecter _send_command 6 2024-02-23T00:47:34.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.270+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fae0c027600 con 0x7fae0c0280a0 2024-02-23T00:47:34.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.272+0000 7fae097fa700 1 -- 172.21.15.47:0/872603830 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fae0c027600 con 0x7fae0c0280a0 2024-02-23T00:47:34.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.272+0000 7fae097fa700 10 client.30939.objecter ms_dispatch 0x7fae0c001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:47:34.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.272+0000 7fae097fa700 10 client.30939.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:47:34.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.272+0000 7fae097fa700 20 client.30939.objecter put_session s=0x7fae0c0595a0 osd=2 4 2024-02-23T00:47:34.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.272+0000 7fae097fa700 15 client.30939.objecter _session_command_op_remove 2 6 2024-02-23T00:47:34.275 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:47:34.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.272+0000 7fae1248f700 10 client.30939.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:34.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.272+0000 7fae1248f700 15 client.30939.objecter _session_command_op_assign -1 7 2024-02-23T00:47:34.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.272+0000 7fae1248f700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fae0c11f7a0 0x7fae0c11fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:34.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.272+0000 7fae1248f700 20 client.30939.objecter _get_session s=0x7fae0c11b770 osd=3 3 2024-02-23T00:47:34.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.272+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c11b770 osd=3 3 2024-02-23T00:47:34.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.272+0000 7fae1248f700 20 client.30939.objecter _get_session s=0x7fae0c11b770 osd=3 3 2024-02-23T00:47:34.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.272+0000 7fae1248f700 15 client.30939.objecter _session_command_op_remove -1 7 2024-02-23T00:47:34.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.272+0000 7fae1248f700 20 client.30939.objecter get_session s=0x7fae0c11b770 osd=3 3 2024-02-23T00:47:34.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.272+0000 7fae1248f700 15 client.30939.objecter _session_command_op_assign 3 7 2024-02-23T00:47:34.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.272+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c11b770 osd=3 4 2024-02-23T00:47:34.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.272+0000 7fae1248f700 10 client.30939.objecter _send_command 7 2024-02-23T00:47:34.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.272+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fae0c11be40 con 0x7fae0c11f7a0 2024-02-23T00:47:34.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.272+0000 7fae10a2c700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fae0c11f7a0 0x7fae0c11fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:34.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.272+0000 7fae097fa700 10 client.30939.objecter ms_handle_connect 0x7fae0c11f7a0 2024-02-23T00:47:34.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.272+0000 7fae10a2c700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fae0c11f7a0 0x7fae0c11fba0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:34.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.273+0000 7fae097fa700 1 -- 172.21.15.47:0/872603830 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fae0c11be40 con 0x7fae0c11f7a0 2024-02-23T00:47:34.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.273+0000 7fae097fa700 10 client.30939.objecter ms_dispatch 0x7fae0c001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:47:34.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.273+0000 7fae097fa700 10 client.30939.objecter _finish_command 7 = system:0 2024-02-23T00:47:34.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.273+0000 7fae097fa700 20 client.30939.objecter put_session s=0x7fae0c11b770 osd=3 4 2024-02-23T00:47:34.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.273+0000 7fae097fa700 15 client.30939.objecter _session_command_op_remove 3 7 2024-02-23T00:47:34.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.287+0000 7fae1248f700 10 client.30939.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:47:34.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.287+0000 7fae1248f700 15 client.30939.objecter _session_command_op_assign -1 8 2024-02-23T00:47:34.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.287+0000 7fae1248f700 20 client.30939.objecter _get_session s=0x7fae0c11b770 osd=3 3 2024-02-23T00:47:34.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.287+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c11b770 osd=3 3 2024-02-23T00:47:34.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.287+0000 7fae1248f700 20 client.30939.objecter _get_session s=0x7fae0c11b770 osd=3 3 2024-02-23T00:47:34.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.287+0000 7fae1248f700 15 client.30939.objecter _session_command_op_remove -1 8 2024-02-23T00:47:34.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.287+0000 7fae1248f700 20 client.30939.objecter get_session s=0x7fae0c11b770 osd=3 3 2024-02-23T00:47:34.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.287+0000 7fae1248f700 15 client.30939.objecter _session_command_op_assign 3 8 2024-02-23T00:47:34.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.287+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c11b770 osd=3 4 2024-02-23T00:47:34.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.287+0000 7fae1248f700 10 client.30939.objecter _send_command 8 2024-02-23T00:47:34.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.287+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fae0c120860 con 0x7fae0c11f7a0 2024-02-23T00:47:34.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.289+0000 7fae097fa700 1 -- 172.21.15.47:0/872603830 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fae0c120860 con 0x7fae0c11f7a0 2024-02-23T00:47:34.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.289+0000 7fae097fa700 10 client.30939.objecter ms_dispatch 0x7fae0c001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:47:34.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.289+0000 7fae097fa700 10 client.30939.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:47:34.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.289+0000 7fae097fa700 20 client.30939.objecter put_session s=0x7fae0c11b770 osd=3 4 2024-02-23T00:47:34.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.289+0000 7fae097fa700 15 client.30939.objecter _session_command_op_remove 3 8 2024-02-23T00:47:34.292 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:47:34.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.289+0000 7fae1248f700 10 client.30939.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:34.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.289+0000 7fae1248f700 15 client.30939.objecter _session_command_op_assign -1 9 2024-02-23T00:47:34.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.289+0000 7fae1248f700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fae0c122fc0 0x7fae0c1253b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:34.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.289+0000 7fae1248f700 20 client.30939.objecter _get_session s=0x7fae0c120090 osd=4 3 2024-02-23T00:47:34.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.289+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c120090 osd=4 3 2024-02-23T00:47:34.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.289+0000 7fae1248f700 20 client.30939.objecter _get_session s=0x7fae0c120090 osd=4 3 2024-02-23T00:47:34.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.289+0000 7fae1248f700 15 client.30939.objecter _session_command_op_remove -1 9 2024-02-23T00:47:34.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.289+0000 7fae1248f700 20 client.30939.objecter get_session s=0x7fae0c120090 osd=4 3 2024-02-23T00:47:34.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.289+0000 7fae1248f700 15 client.30939.objecter _session_command_op_assign 4 9 2024-02-23T00:47:34.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.289+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c120090 osd=4 4 2024-02-23T00:47:34.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.289+0000 7fae1248f700 10 client.30939.objecter _send_command 9 2024-02-23T00:47:34.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.289+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fae0c120760 con 0x7fae0c122fc0 2024-02-23T00:47:34.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.289+0000 7fae0bfff700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fae0c122fc0 0x7fae0c1253b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:34.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.289+0000 7fae097fa700 10 client.30939.objecter ms_handle_connect 0x7fae0c122fc0 2024-02-23T00:47:34.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.289+0000 7fae0bfff700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fae0c122fc0 0x7fae0c1253b0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:34.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.290+0000 7fae097fa700 1 -- 172.21.15.47:0/872603830 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fae0c120760 con 0x7fae0c122fc0 2024-02-23T00:47:34.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.290+0000 7fae097fa700 10 client.30939.objecter ms_dispatch 0x7fae0c001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:47:34.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.290+0000 7fae097fa700 10 client.30939.objecter _finish_command 9 = system:0 2024-02-23T00:47:34.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.290+0000 7fae097fa700 20 client.30939.objecter put_session s=0x7fae0c120090 osd=4 4 2024-02-23T00:47:34.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.290+0000 7fae097fa700 15 client.30939.objecter _session_command_op_remove 4 9 2024-02-23T00:47:34.302 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:47:34.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.306+0000 7fae1248f700 10 client.30939.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:47:34.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.306+0000 7fae1248f700 15 client.30939.objecter _session_command_op_assign -1 10 2024-02-23T00:47:34.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.306+0000 7fae1248f700 20 client.30939.objecter _get_session s=0x7fae0c120090 osd=4 3 2024-02-23T00:47:34.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.306+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c120090 osd=4 3 2024-02-23T00:47:34.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.306+0000 7fae1248f700 20 client.30939.objecter _get_session s=0x7fae0c120090 osd=4 3 2024-02-23T00:47:34.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.306+0000 7fae1248f700 15 client.30939.objecter _session_command_op_remove -1 10 2024-02-23T00:47:34.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.306+0000 7fae1248f700 20 client.30939.objecter get_session s=0x7fae0c120090 osd=4 3 2024-02-23T00:47:34.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.306+0000 7fae1248f700 15 client.30939.objecter _session_command_op_assign 4 10 2024-02-23T00:47:34.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.306+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c120090 osd=4 4 2024-02-23T00:47:34.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.306+0000 7fae1248f700 10 client.30939.objecter _send_command 10 2024-02-23T00:47:34.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.306+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fae0c120760 con 0x7fae0c122fc0 2024-02-23T00:47:34.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.308+0000 7fae097fa700 1 -- 172.21.15.47:0/872603830 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fae0c120760 con 0x7fae0c122fc0 2024-02-23T00:47:34.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.308+0000 7fae097fa700 10 client.30939.objecter ms_dispatch 0x7fae0c001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:47:34.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.308+0000 7fae097fa700 10 client.30939.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:47:34.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.308+0000 7fae097fa700 20 client.30939.objecter put_session s=0x7fae0c120090 osd=4 4 2024-02-23T00:47:34.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.308+0000 7fae097fa700 15 client.30939.objecter _session_command_op_remove 4 10 2024-02-23T00:47:34.311 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:47:34.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.308+0000 7fae1248f700 10 client.30939.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:34.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.308+0000 7fae1248f700 15 client.30939.objecter _session_command_op_assign -1 11 2024-02-23T00:47:34.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.308+0000 7fae1248f700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fae0c125e00 0x7fae0c128230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:34.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.308+0000 7fae1248f700 20 client.30939.objecter _get_session s=0x7fae0c120760 osd=5 3 2024-02-23T00:47:34.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.308+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c120760 osd=5 3 2024-02-23T00:47:34.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.308+0000 7fae1248f700 20 client.30939.objecter _get_session s=0x7fae0c120760 osd=5 3 2024-02-23T00:47:34.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.308+0000 7fae1248f700 15 client.30939.objecter _session_command_op_remove -1 11 2024-02-23T00:47:34.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.308+0000 7fae1248f700 20 client.30939.objecter get_session s=0x7fae0c120760 osd=5 3 2024-02-23T00:47:34.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.308+0000 7fae1248f700 15 client.30939.objecter _session_command_op_assign 5 11 2024-02-23T00:47:34.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.308+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c120760 osd=5 4 2024-02-23T00:47:34.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.308+0000 7fae1248f700 10 client.30939.objecter _send_command 11 2024-02-23T00:47:34.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.308+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fae0c12b070 con 0x7fae0c125e00 2024-02-23T00:47:34.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.308+0000 7fae0b7fe700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fae0c125e00 0x7fae0c128230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:34.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.308+0000 7fae097fa700 10 client.30939.objecter ms_handle_connect 0x7fae0c125e00 2024-02-23T00:47:34.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.308+0000 7fae0b7fe700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fae0c125e00 0x7fae0c128230 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:34.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.309+0000 7fae097fa700 1 -- 172.21.15.47:0/872603830 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fae0c12b070 con 0x7fae0c125e00 2024-02-23T00:47:34.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.309+0000 7fae097fa700 10 client.30939.objecter ms_dispatch 0x7fae0c001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:47:34.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.309+0000 7fae097fa700 10 client.30939.objecter _finish_command 11 = system:0 2024-02-23T00:47:34.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.309+0000 7fae097fa700 20 client.30939.objecter put_session s=0x7fae0c120760 osd=5 4 2024-02-23T00:47:34.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.309+0000 7fae097fa700 15 client.30939.objecter _session_command_op_remove 5 11 2024-02-23T00:47:34.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.324+0000 7fae1248f700 10 client.30939.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:47:34.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.324+0000 7fae1248f700 15 client.30939.objecter _session_command_op_assign -1 12 2024-02-23T00:47:34.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.324+0000 7fae1248f700 20 client.30939.objecter _get_session s=0x7fae0c120760 osd=5 3 2024-02-23T00:47:34.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.324+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c120760 osd=5 3 2024-02-23T00:47:34.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.324+0000 7fae1248f700 20 client.30939.objecter _get_session s=0x7fae0c120760 osd=5 3 2024-02-23T00:47:34.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.324+0000 7fae1248f700 15 client.30939.objecter _session_command_op_remove -1 12 2024-02-23T00:47:34.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.324+0000 7fae1248f700 20 client.30939.objecter get_session s=0x7fae0c120760 osd=5 3 2024-02-23T00:47:34.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.324+0000 7fae1248f700 15 client.30939.objecter _session_command_op_assign 5 12 2024-02-23T00:47:34.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.324+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c120760 osd=5 4 2024-02-23T00:47:34.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.324+0000 7fae1248f700 10 client.30939.objecter _send_command 12 2024-02-23T00:47:34.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.324+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fae0c12b070 con 0x7fae0c125e00 2024-02-23T00:47:34.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.326+0000 7fae097fa700 1 -- 172.21.15.47:0/872603830 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fae0c12b070 con 0x7fae0c125e00 2024-02-23T00:47:34.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.326+0000 7fae097fa700 10 client.30939.objecter ms_dispatch 0x7fae0c001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:47:34.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.326+0000 7fae097fa700 10 client.30939.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:47:34.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.326+0000 7fae097fa700 20 client.30939.objecter put_session s=0x7fae0c120760 osd=5 4 2024-02-23T00:47:34.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.326+0000 7fae097fa700 15 client.30939.objecter _session_command_op_remove 5 12 2024-02-23T00:47:34.329 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:47:34.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.326+0000 7fae1248f700 10 client.30939.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:34.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.326+0000 7fae1248f700 15 client.30939.objecter _session_command_op_assign -1 13 2024-02-23T00:47:34.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.326+0000 7fae1248f700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fae0c12a3a0 0x7fae0c12a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:34.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.326+0000 7fae1248f700 20 client.30939.objecter _get_session s=0x7fae0c12b070 osd=6 3 2024-02-23T00:47:34.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.326+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c12b070 osd=6 3 2024-02-23T00:47:34.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.326+0000 7fae1248f700 20 client.30939.objecter _get_session s=0x7fae0c12b070 osd=6 3 2024-02-23T00:47:34.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.326+0000 7fae1248f700 15 client.30939.objecter _session_command_op_remove -1 13 2024-02-23T00:47:34.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.326+0000 7fae1248f700 20 client.30939.objecter get_session s=0x7fae0c12b070 osd=6 3 2024-02-23T00:47:34.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.326+0000 7fae1248f700 15 client.30939.objecter _session_command_op_assign 6 13 2024-02-23T00:47:34.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.326+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c12b070 osd=6 4 2024-02-23T00:47:34.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.326+0000 7fae1248f700 10 client.30939.objecter _send_command 13 2024-02-23T00:47:34.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.326+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fae0c12e3a0 con 0x7fae0c12a3a0 2024-02-23T00:47:34.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.326+0000 7fae10a2c700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fae0c12a3a0 0x7fae0c12a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:34.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.327+0000 7fae097fa700 10 client.30939.objecter ms_handle_connect 0x7fae0c12a3a0 2024-02-23T00:47:34.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.327+0000 7fae10a2c700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fae0c12a3a0 0x7fae0c12a820 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:34.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.327+0000 7fae097fa700 1 -- 172.21.15.47:0/872603830 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fae0c12e3a0 con 0x7fae0c12a3a0 2024-02-23T00:47:34.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.327+0000 7fae097fa700 10 client.30939.objecter ms_dispatch 0x7fae0c001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:47:34.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.327+0000 7fae097fa700 10 client.30939.objecter _finish_command 13 = system:0 2024-02-23T00:47:34.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.327+0000 7fae097fa700 20 client.30939.objecter put_session s=0x7fae0c12b070 osd=6 4 2024-02-23T00:47:34.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.327+0000 7fae097fa700 15 client.30939.objecter _session_command_op_remove 6 13 2024-02-23T00:47:34.332 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:34.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.342+0000 7fae1248f700 10 client.30939.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:47:34.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.342+0000 7fae1248f700 15 client.30939.objecter _session_command_op_assign -1 14 2024-02-23T00:47:34.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.342+0000 7fae1248f700 20 client.30939.objecter _get_session s=0x7fae0c12b070 osd=6 3 2024-02-23T00:47:34.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.342+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c12b070 osd=6 3 2024-02-23T00:47:34.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.342+0000 7fae1248f700 20 client.30939.objecter _get_session s=0x7fae0c12b070 osd=6 3 2024-02-23T00:47:34.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.342+0000 7fae1248f700 15 client.30939.objecter _session_command_op_remove -1 14 2024-02-23T00:47:34.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.342+0000 7fae1248f700 20 client.30939.objecter get_session s=0x7fae0c12b070 osd=6 3 2024-02-23T00:47:34.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.342+0000 7fae1248f700 15 client.30939.objecter _session_command_op_assign 6 14 2024-02-23T00:47:34.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.342+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c12b070 osd=6 4 2024-02-23T00:47:34.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.342+0000 7fae1248f700 10 client.30939.objecter _send_command 14 2024-02-23T00:47:34.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.342+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fae0c128830 con 0x7fae0c12a3a0 2024-02-23T00:47:34.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.343+0000 7fae097fa700 1 -- 172.21.15.47:0/872603830 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fae0c128830 con 0x7fae0c12a3a0 2024-02-23T00:47:34.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.343+0000 7fae097fa700 10 client.30939.objecter ms_dispatch 0x7fae0c001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:47:34.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.343+0000 7fae097fa700 10 client.30939.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:47:34.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.343+0000 7fae097fa700 20 client.30939.objecter put_session s=0x7fae0c12b070 osd=6 4 2024-02-23T00:47:34.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.343+0000 7fae097fa700 15 client.30939.objecter _session_command_op_remove 6 14 2024-02-23T00:47:34.346 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:47:34.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.343+0000 7fae1248f700 10 client.30939.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:34.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.344+0000 7fae1248f700 15 client.30939.objecter _session_command_op_assign -1 15 2024-02-23T00:47:34.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.344+0000 7fae1248f700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fae0c128eb0 0x7fae0c129310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:34.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.344+0000 7fae1248f700 20 client.30939.objecter _get_session s=0x7fae0c128830 osd=7 3 2024-02-23T00:47:34.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.344+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c128830 osd=7 3 2024-02-23T00:47:34.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.344+0000 7fae1248f700 20 client.30939.objecter _get_session s=0x7fae0c128830 osd=7 3 2024-02-23T00:47:34.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.344+0000 7fae1248f700 15 client.30939.objecter _session_command_op_remove -1 15 2024-02-23T00:47:34.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.344+0000 7fae1248f700 20 client.30939.objecter get_session s=0x7fae0c128830 osd=7 3 2024-02-23T00:47:34.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.344+0000 7fae1248f700 15 client.30939.objecter _session_command_op_assign 7 15 2024-02-23T00:47:34.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.344+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c128830 osd=7 4 2024-02-23T00:47:34.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.344+0000 7fae1248f700 10 client.30939.objecter _send_command 15 2024-02-23T00:47:34.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.344+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fae0c1299a0 con 0x7fae0c128eb0 2024-02-23T00:47:34.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.344+0000 7fae0bfff700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fae0c128eb0 0x7fae0c129310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:34.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.344+0000 7fae0bfff700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fae0c128eb0 0x7fae0c129310 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:34.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.344+0000 7fae097fa700 10 client.30939.objecter ms_handle_connect 0x7fae0c128eb0 2024-02-23T00:47:34.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.345+0000 7fae097fa700 1 -- 172.21.15.47:0/872603830 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fae0c1299a0 con 0x7fae0c128eb0 2024-02-23T00:47:34.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.345+0000 7fae097fa700 10 client.30939.objecter ms_dispatch 0x7fae0c001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:47:34.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.345+0000 7fae097fa700 10 client.30939.objecter _finish_command 15 = system:0 2024-02-23T00:47:34.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.345+0000 7fae097fa700 20 client.30939.objecter put_session s=0x7fae0c128830 osd=7 4 2024-02-23T00:47:34.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.345+0000 7fae097fa700 15 client.30939.objecter _session_command_op_remove 7 15 2024-02-23T00:47:34.352 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:47:34.354 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:34.355 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:47:34.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.359+0000 7fae1248f700 10 client.30939.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:47:34.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.359+0000 7fae1248f700 15 client.30939.objecter _session_command_op_assign -1 16 2024-02-23T00:47:34.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.359+0000 7fae1248f700 20 client.30939.objecter _get_session s=0x7fae0c128830 osd=7 3 2024-02-23T00:47:34.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.359+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c128830 osd=7 3 2024-02-23T00:47:34.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.359+0000 7fae1248f700 20 client.30939.objecter _get_session s=0x7fae0c128830 osd=7 3 2024-02-23T00:47:34.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.359+0000 7fae1248f700 15 client.30939.objecter _session_command_op_remove -1 16 2024-02-23T00:47:34.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.359+0000 7fae1248f700 20 client.30939.objecter get_session s=0x7fae0c128830 osd=7 3 2024-02-23T00:47:34.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.359+0000 7fae1248f700 15 client.30939.objecter _session_command_op_assign 7 16 2024-02-23T00:47:34.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.359+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c128830 osd=7 4 2024-02-23T00:47:34.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.359+0000 7fae1248f700 10 client.30939.objecter _send_command 16 2024-02-23T00:47:34.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.359+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fae0c1299a0 con 0x7fae0c128eb0 2024-02-23T00:47:34.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.361+0000 7fae097fa700 1 -- 172.21.15.47:0/872603830 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fae0c1299a0 con 0x7fae0c128eb0 2024-02-23T00:47:34.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.361+0000 7fae097fa700 10 client.30939.objecter ms_dispatch 0x7fae0c001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:47:34.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.361+0000 7fae097fa700 10 client.30939.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:47:34.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.361+0000 7fae097fa700 20 client.30939.objecter put_session s=0x7fae0c128830 osd=7 4 2024-02-23T00:47:34.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.361+0000 7fae097fa700 15 client.30939.objecter _session_command_op_remove 7 16 2024-02-23T00:47:34.365 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:47:34.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 10 librados: watch_flush enter 2024-02-23T00:47:34.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 10 librados: watch_flush exit 2024-02-23T00:47:34.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 10 client.30939.objecter close_session for osd.0 2024-02-23T00:47:34.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fae0c0fee00 msgr2=0x7fae0c0ff1e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:34.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fae0c0fee00 0x7fae0c0ff1e0 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:34.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c116890 osd=0 1 2024-02-23T00:47:34.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 10 client.30939.objecter close_session for osd.1 2024-02-23T00:47:34.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fae0c0598f0 msgr2=0x7fae0c059d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:34.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fae0c0598f0 0x7fae0c059d10 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:34.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c028540 osd=1 1 2024-02-23T00:47:34.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 10 client.30939.objecter close_session for osd.2 2024-02-23T00:47:34.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fae0c0280a0 msgr2=0x7fae0c11e130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:34.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fae0c0280a0 0x7fae0c11e130 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:34.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c0595a0 osd=2 1 2024-02-23T00:47:34.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 10 client.30939.objecter close_session for osd.3 2024-02-23T00:47:34.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fae0c11f7a0 msgr2=0x7fae0c11fba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:34.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fae0c11f7a0 0x7fae0c11fba0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:34.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c11b770 osd=3 1 2024-02-23T00:47:34.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 10 client.30939.objecter close_session for osd.4 2024-02-23T00:47:34.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fae0c122fc0 msgr2=0x7fae0c1253b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:34.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fae0c122fc0 0x7fae0c1253b0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:34.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c120090 osd=4 1 2024-02-23T00:47:34.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 10 client.30939.objecter close_session for osd.5 2024-02-23T00:47:34.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fae0c125e00 msgr2=0x7fae0c128230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:34.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae10a2c700 1 -- 172.21.15.47:0/872603830 reap_dead start 2024-02-23T00:47:34.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fae0c125e00 0x7fae0c128230 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:34.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae10a2c700 1 -- 172.21.15.47:0/872603830 reap_dead start 2024-02-23T00:47:34.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae10a2c700 1 -- 172.21.15.47:0/872603830 reap_dead start 2024-02-23T00:47:34.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c120760 osd=5 1 2024-02-23T00:47:34.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 10 client.30939.objecter close_session for osd.6 2024-02-23T00:47:34.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fae0c12a3a0 msgr2=0x7fae0c12a820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:34.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fae0c12a3a0 0x7fae0c12a820 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:34.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c12b070 osd=6 1 2024-02-23T00:47:34.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 10 client.30939.objecter close_session for osd.7 2024-02-23T00:47:34.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fae0c128eb0 msgr2=0x7fae0c129310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:34.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fae0c128eb0 0x7fae0c129310 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:34.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 20 client.30939.objecter put_session s=0x7fae0c128830 osd=7 1 2024-02-23T00:47:34.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 20 client.30939.objecter shutdown clearing up homeless session... 2024-02-23T00:47:34.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 10 client.30939.objecter successfully canceled tick 2024-02-23T00:47:34.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fadf4062cc0 msgr2=0x7fadf4065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:34.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fadf4062cc0 0x7fadf4065180 secure :-1 s=READY pgs=3052 cs=0 l=1 rev1=1 rx=0x7fadfc004e60 tx=0x7fadfc00b040).stop 2024-02-23T00:47:34.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae0c0fe510 msgr2=0x7fae0c06bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:34.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 1 --2- 172.21.15.47:0/872603830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae0c0fe510 0x7fae0c06bab0 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 rx=0x7fae04001cb0 tx=0x7fae04004c50).stop 2024-02-23T00:47:34.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae10a2c700 1 -- 172.21.15.47:0/872603830 reap_dead start 2024-02-23T00:47:34.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 shutdown_connections 2024-02-23T00:47:34.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 >> 172.21.15.47:0/872603830 conn(0x7fae0c0f3800 msgr2=0x7fae0c0fb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:34.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 shutdown_connections 2024-02-23T00:47:34.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 1 -- 172.21.15.47:0/872603830 wait complete. 2024-02-23T00:47:34.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:34.362+0000 7fae1248f700 1 librados: shutdown 2024-02-23T00:47:34.370 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:47:34.370 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:47:34.370 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:47:34.370 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:47:34.370 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:47:34.370 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:47:34.370 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:47:34.370 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:47:34.456 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:47:34.483 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:33.177752+0000 mon.a (mon.0) 2412 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi047-82386-63"}]': finished 2024-02-23T00:47:34.483 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:33.177872+0000 mon.a (mon.0) 2413 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-49"}]': finished 2024-02-23T00:47:34.483 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:33.177931+0000 mon.a (mon.0) 2414 : cluster [DBG] osdmap e381: 8 total, 8 up, 7 in 2024-02-23T00:47:34.483 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:33.180647+0000 mon.a (mon.0) 2415 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-49", "mode": "writeback"}]: dispatch 2024-02-23T00:47:34.484 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:33.204646+0000 mon.a (mon.0) 2416 : audit [INF] from='client.? 172.21.15.47:0/584262349' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi047-82386-64"}]: dispatch 2024-02-23T00:47:34.484 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:33.205512+0000 mon.a (mon.0) 2417 : audit [INF] from='client.? 172.21.15.47:0/584262349' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi047-82386-64"}]: dispatch 2024-02-23T00:47:34.484 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:33.205855+0000 mon.a (mon.0) 2418 : audit [INF] from='client.? 172.21.15.47:0/584262349' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi047-82386-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:47:34.484 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:33.276056+0000 mgr.y (mgr.14517) 355 : cluster [DBG] pgmap v547: 214 pgs: 1 active+recovering, 1 active+remapped+backfill_toofull, 1 peering, 211 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 8.5 KiB/s rd, 1023 B/s wr, 16 op/s; 3/1119 objects misplaced (0.268%); 509 B/s, 89 objects/s recovering 2024-02-23T00:47:34.484 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:33.276916+0000 mon.a (mon.0) 2419 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "17"}]: dispatch 2024-02-23T00:47:34.484 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:34.174178+0000 mon.a (mon.0) 2420 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:47:34.484 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:34.178376+0000 mon.a (mon.0) 2421 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-49", "mode": "writeback"}]': finished 2024-02-23T00:47:34.485 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:34.178435+0000 mon.a (mon.0) 2422 : audit [INF] from='client.? 172.21.15.47:0/584262349' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi047-82386-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:47:34.485 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:34.178491+0000 mon.a (mon.0) 2423 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "17"}]': finished 2024-02-23T00:47:34.485 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:34.178524+0000 mon.a (mon.0) 2424 : cluster [DBG] osdmap e382: 8 total, 8 up, 7 in 2024-02-23T00:47:34.485 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:34.181043+0000 mon.a (mon.0) 2425 : audit [INF] from='client.? 172.21.15.47:0/584262349' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi047-82386-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi047-82386-64"}]: dispatch 2024-02-23T00:47:34.485 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:33.177752+0000 mon.a (mon.0) 2412 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi047-82386-63"}]': finished 2024-02-23T00:47:34.486 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:33.177872+0000 mon.a (mon.0) 2413 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-49"}]': finished 2024-02-23T00:47:34.486 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:33.177931+0000 mon.a (mon.0) 2414 : cluster [DBG] osdmap e381: 8 total, 8 up, 7 in 2024-02-23T00:47:34.487 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:33.180647+0000 mon.a (mon.0) 2415 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-49", "mode": "writeback"}]: dispatch 2024-02-23T00:47:34.487 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:33.204646+0000 mon.a (mon.0) 2416 : audit [INF] from='client.? 172.21.15.47:0/584262349' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi047-82386-64"}]: dispatch 2024-02-23T00:47:34.487 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:33.205512+0000 mon.a (mon.0) 2417 : audit [INF] from='client.? 172.21.15.47:0/584262349' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi047-82386-64"}]: dispatch 2024-02-23T00:47:34.487 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:33.205855+0000 mon.a (mon.0) 2418 : audit [INF] from='client.? 172.21.15.47:0/584262349' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi047-82386-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:47:34.488 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:33.276056+0000 mgr.y (mgr.14517) 355 : cluster [DBG] pgmap v547: 214 pgs: 1 active+recovering, 1 active+remapped+backfill_toofull, 1 peering, 211 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 8.5 KiB/s rd, 1023 B/s wr, 16 op/s; 3/1119 objects misplaced (0.268%); 509 B/s, 89 objects/s recovering 2024-02-23T00:47:34.488 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:33.276916+0000 mon.a (mon.0) 2419 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "17"}]: dispatch 2024-02-23T00:47:34.489 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:34.174178+0000 mon.a (mon.0) 2420 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:47:34.489 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:34.178376+0000 mon.a (mon.0) 2421 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-49", "mode": "writeback"}]': finished 2024-02-23T00:47:34.489 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:34.178435+0000 mon.a (mon.0) 2422 : audit [INF] from='client.? 172.21.15.47:0/584262349' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi047-82386-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:47:34.489 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:34.178491+0000 mon.a (mon.0) 2423 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "17"}]': finished 2024-02-23T00:47:34.490 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:34.178524+0000 mon.a (mon.0) 2424 : cluster [DBG] osdmap e382: 8 total, 8 up, 7 in 2024-02-23T00:47:34.490 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:34.181043+0000 mon.a (mon.0) 2425 : audit [INF] from='client.? 172.21.15.47:0/584262349' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi047-82386-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi047-82386-64"}]: dispatch 2024-02-23T00:47:34.492 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:34.567 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-23T00:47:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:33.177752+0000 mon.a (mon.0) 2412 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi047-82386-63"}]': finished 2024-02-23T00:47:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:33.177872+0000 mon.a (mon.0) 2413 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-49"}]': finished 2024-02-23T00:47:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:33.177931+0000 mon.a (mon.0) 2414 : cluster [DBG] osdmap e381: 8 total, 8 up, 7 in 2024-02-23T00:47:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:33.180647+0000 mon.a (mon.0) 2415 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-49", "mode": "writeback"}]: dispatch 2024-02-23T00:47:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:33.204646+0000 mon.a (mon.0) 2416 : audit [INF] from='client.? 172.21.15.47:0/584262349' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi047-82386-64"}]: dispatch 2024-02-23T00:47:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:33.205512+0000 mon.a (mon.0) 2417 : audit [INF] from='client.? 172.21.15.47:0/584262349' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi047-82386-64"}]: dispatch 2024-02-23T00:47:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:33.205855+0000 mon.a (mon.0) 2418 : audit [INF] from='client.? 172.21.15.47:0/584262349' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi047-82386-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:47:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:33.276056+0000 mgr.y (mgr.14517) 355 : cluster [DBG] pgmap v547: 214 pgs: 1 active+recovering, 1 active+remapped+backfill_toofull, 1 peering, 211 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 8.5 KiB/s rd, 1023 B/s wr, 16 op/s; 3/1119 objects misplaced (0.268%); 509 B/s, 89 objects/s recovering 2024-02-23T00:47:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:33.276916+0000 mon.a (mon.0) 2419 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "17"}]: dispatch 2024-02-23T00:47:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:34.174178+0000 mon.a (mon.0) 2420 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:47:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:34.178376+0000 mon.a (mon.0) 2421 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-49", "mode": "writeback"}]': finished 2024-02-23T00:47:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:34.178435+0000 mon.a (mon.0) 2422 : audit [INF] from='client.? 172.21.15.47:0/584262349' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi047-82386-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:47:34.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:34.178491+0000 mon.a (mon.0) 2423 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "17"}]': finished 2024-02-23T00:47:34.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:34.178524+0000 mon.a (mon.0) 2424 : cluster [DBG] osdmap e382: 8 total, 8 up, 7 in 2024-02-23T00:47:34.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:34.181043+0000 mon.a (mon.0) 2425 : audit [INF] from='client.? 172.21.15.47:0/584262349' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi047-82386-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi047-82386-64"}]: dispatch 2024-02-23T00:47:35.001 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:47:35.593 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:47:35.594 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:47:35.943 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:34.181555+0000 mon.a (mon.0) 2426 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs inactive, 8 pgs peering) 2024-02-23T00:47:35.944 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:34.219524+0000 mon.a (mon.0) 2427 : audit [DBG] from='client.? 172.21.15.47:0/872603830' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:47:35.944 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:34.221304+0000 mon.a (mon.0) 2428 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-49","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:47:35.944 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:34.181555+0000 mon.a (mon.0) 2426 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs inactive, 8 pgs peering) 2024-02-23T00:47:35.944 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:34.219524+0000 mon.a (mon.0) 2427 : audit [DBG] from='client.? 172.21.15.47:0/872603830' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:47:35.944 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:34.221304+0000 mon.a (mon.0) 2428 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-49","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:47:35.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:34.181555+0000 mon.a (mon.0) 2426 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs inactive, 8 pgs peering) 2024-02-23T00:47:35.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:34.219524+0000 mon.a (mon.0) 2427 : audit [DBG] from='client.? 172.21.15.47:0/872603830' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:47:35.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:34.221304+0000 mon.a (mon.0) 2428 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-49","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:47:36.207 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:47:36.436 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:47:36.439 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:36.439 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:47:36.540 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:47:36.571 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:36.669 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:47:36.671 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:35.153614+0000 osd.3 (osd.3) 180 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:47:36.671 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:35.153736+0000 osd.3 (osd.3) 181 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:47:36.671 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:35.276505+0000 mgr.y (mgr.14517) 356 : cluster [DBG] pgmap v550: 214 pgs: 1 active+recovering, 1 active+remapped+backfill_toofull, 1 peering, 211 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 8.5 KiB/s rd, 1023 B/s wr, 16 op/s; 3/1119 objects misplaced (0.268%); 509 B/s, 89 objects/s recovering 2024-02-23T00:47:36.671 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:35.607263+0000 mon.a (mon.0) 2429 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-49","var": "hit_set_count","val": "2"}]': finished 2024-02-23T00:47:36.671 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:35.607356+0000 mon.a (mon.0) 2430 : cluster [DBG] osdmap e383: 8 total, 8 up, 7 in 2024-02-23T00:47:36.672 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:35.608978+0000 mon.a (mon.0) 2431 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-49","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:47:36.672 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:36.175677+0000 mon.a (mon.0) 2432 : audit [INF] from='client.? 172.21.15.47:0/584262349' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi047-82386-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi047-82386-64"}]': finished 2024-02-23T00:47:36.672 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:36.175769+0000 mon.a (mon.0) 2433 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-49","var": "hit_set_period","val": "600"}]': finished 2024-02-23T00:47:36.672 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:36.175844+0000 mon.a (mon.0) 2434 : cluster [DBG] osdmap e384: 8 total, 8 up, 7 in 2024-02-23T00:47:36.672 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:36.179639+0000 mon.a (mon.0) 2435 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-49","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-23T00:47:36.672 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:35.153614+0000 osd.3 (osd.3) 180 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:47:36.672 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:35.153736+0000 osd.3 (osd.3) 181 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:47:36.672 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:35.276505+0000 mgr.y (mgr.14517) 356 : cluster [DBG] pgmap v550: 214 pgs: 1 active+recovering, 1 active+remapped+backfill_toofull, 1 peering, 211 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 8.5 KiB/s rd, 1023 B/s wr, 16 op/s; 3/1119 objects misplaced (0.268%); 509 B/s, 89 objects/s recovering 2024-02-23T00:47:36.672 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:35.607263+0000 mon.a (mon.0) 2429 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-49","var": "hit_set_count","val": "2"}]': finished 2024-02-23T00:47:36.673 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:35.607356+0000 mon.a (mon.0) 2430 : cluster [DBG] osdmap e383: 8 total, 8 up, 7 in 2024-02-23T00:47:36.673 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:35.608978+0000 mon.a (mon.0) 2431 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-49","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:47:36.673 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:36.175677+0000 mon.a (mon.0) 2432 : audit [INF] from='client.? 172.21.15.47:0/584262349' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi047-82386-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi047-82386-64"}]': finished 2024-02-23T00:47:36.673 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:36.175769+0000 mon.a (mon.0) 2433 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-49","var": "hit_set_period","val": "600"}]': finished 2024-02-23T00:47:36.673 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:36.175844+0000 mon.a (mon.0) 2434 : cluster [DBG] osdmap e384: 8 total, 8 up, 7 in 2024-02-23T00:47:36.673 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:36.179639+0000 mon.a (mon.0) 2435 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-49","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-23T00:47:36.673 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:36.674 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:47:36.775 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:47:36.804 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:36.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:35.153614+0000 osd.3 (osd.3) 180 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:47:36.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:35.153736+0000 osd.3 (osd.3) 181 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:47:36.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:35.276505+0000 mgr.y (mgr.14517) 356 : cluster [DBG] pgmap v550: 214 pgs: 1 active+recovering, 1 active+remapped+backfill_toofull, 1 peering, 211 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 8.5 KiB/s rd, 1023 B/s wr, 16 op/s; 3/1119 objects misplaced (0.268%); 509 B/s, 89 objects/s recovering 2024-02-23T00:47:36.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:35.607263+0000 mon.a (mon.0) 2429 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-49","var": "hit_set_count","val": "2"}]': finished 2024-02-23T00:47:36.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:35.607356+0000 mon.a (mon.0) 2430 : cluster [DBG] osdmap e383: 8 total, 8 up, 7 in 2024-02-23T00:47:36.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:35.608978+0000 mon.a (mon.0) 2431 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-49","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:47:36.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:36.175677+0000 mon.a (mon.0) 2432 : audit [INF] from='client.? 172.21.15.47:0/584262349' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi047-82386-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi047-82386-64"}]': finished 2024-02-23T00:47:36.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:36.175769+0000 mon.a (mon.0) 2433 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-49","var": "hit_set_period","val": "600"}]': finished 2024-02-23T00:47:36.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:36.175844+0000 mon.a (mon.0) 2434 : cluster [DBG] osdmap e384: 8 total, 8 up, 7 in 2024-02-23T00:47:36.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:36.179639+0000 mon.a (mon.0) 2435 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-49","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-23T00:47:37.002 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:47:37.004 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:37.005 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:47:37.105 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:47:37.137 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:37.158 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:47:37.161 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:37.162 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:47:37.262 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:47:37.292 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:37.312 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:47:37.314 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:37.314 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:47:37.415 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:47:37.436 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.0 dump_historic_ops 2024-02-23T00:47:37.442 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:37.461 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:47:37.463 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:37.464 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:47:37.565 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:47:37.592 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:37.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:37.172562+0000 mon.a (mon.0) 2436 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:47:37.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:37.177021+0000 mon.a (mon.0) 2437 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-49","var": "hit_set_type","val": "bloom"}]': finished 2024-02-23T00:47:37.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:37.177130+0000 mon.a (mon.0) 2438 : cluster [DBG] osdmap e385: 8 total, 8 up, 7 in 2024-02-23T00:47:37.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:37.181000+0000 mon.a (mon.0) 2439 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-49","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-23T00:47:37.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:37.278312+0000 mon.a (mon.0) 2440 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "16"}]: dispatch 2024-02-23T00:47:37.738 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:37.172562+0000 mon.a (mon.0) 2436 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:47:37.738 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:37.177021+0000 mon.a (mon.0) 2437 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-49","var": "hit_set_type","val": "bloom"}]': finished 2024-02-23T00:47:37.738 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:37.177130+0000 mon.a (mon.0) 2438 : cluster [DBG] osdmap e385: 8 total, 8 up, 7 in 2024-02-23T00:47:37.738 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:37.181000+0000 mon.a (mon.0) 2439 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-49","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-23T00:47:37.738 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:37.278312+0000 mon.a (mon.0) 2440 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "16"}]: dispatch 2024-02-23T00:47:37.738 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:37.172562+0000 mon.a (mon.0) 2436 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:47:37.739 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:37.177021+0000 mon.a (mon.0) 2437 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-49","var": "hit_set_type","val": "bloom"}]': finished 2024-02-23T00:47:37.739 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:37.177130+0000 mon.a (mon.0) 2438 : cluster [DBG] osdmap e385: 8 total, 8 up, 7 in 2024-02-23T00:47:37.739 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:37.181000+0000 mon.a (mon.0) 2439 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-49","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-23T00:47:37.739 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:37.278312+0000 mon.a (mon.0) 2440 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "16"}]: dispatch 2024-02-23T00:47:37.964 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:47:38.769 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:47:38.771 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:37.277395+0000 mgr.y (mgr.14517) 357 : cluster [DBG] pgmap v553: 221 pgs: 2 creating+activating, 1 active+remapped+backfill_toofull, 2 creating+peering, 216 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s; 1/1131 objects misplaced (0.088%); 3 B/s, 0 objects/s recovering 2024-02-23T00:47:38.771 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:37.545242+0000 osd.5 (osd.5) 164 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:47:38.771 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:37.545385+0000 osd.5 (osd.5) 165 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:47:38.771 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:38.181497+0000 mon.a (mon.0) 2441 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-49","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-23T00:47:38.771 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:38.181597+0000 mon.a (mon.0) 2442 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "16"}]': finished 2024-02-23T00:47:38.771 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:38.181646+0000 mon.a (mon.0) 2443 : cluster [DBG] osdmap e386: 8 total, 8 up, 7 in 2024-02-23T00:47:38.771 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:38.188079+0000 mon.a (mon.0) 2444 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-49","var": "target_max_objects","val": "1"}]: dispatch 2024-02-23T00:47:38.771 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:38.189906+0000 mon.a (mon.0) 2445 : audit [INF] from='client.? 172.21.15.47:0/584262349' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi047-82386-64"}]: dispatch 2024-02-23T00:47:38.771 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:37.277395+0000 mgr.y (mgr.14517) 357 : cluster [DBG] pgmap v553: 221 pgs: 2 creating+activating, 1 active+remapped+backfill_toofull, 2 creating+peering, 216 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s; 1/1131 objects misplaced (0.088%); 3 B/s, 0 objects/s recovering 2024-02-23T00:47:38.771 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:37.545242+0000 osd.5 (osd.5) 164 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:47:38.772 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:37.545385+0000 osd.5 (osd.5) 165 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:47:38.772 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:38.181497+0000 mon.a (mon.0) 2441 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-49","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-23T00:47:38.772 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:38.181597+0000 mon.a (mon.0) 2442 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "16"}]': finished 2024-02-23T00:47:38.772 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:38.181646+0000 mon.a (mon.0) 2443 : cluster [DBG] osdmap e386: 8 total, 8 up, 7 in 2024-02-23T00:47:38.772 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:38.188079+0000 mon.a (mon.0) 2444 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-49","var": "target_max_objects","val": "1"}]: dispatch 2024-02-23T00:47:38.772 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:38.189906+0000 mon.a (mon.0) 2445 : audit [INF] from='client.? 172.21.15.47:0/584262349' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi047-82386-64"}]: dispatch 2024-02-23T00:47:38.772 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:38.773 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:47:38.874 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:47:38.902 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:38.921 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:47:38.926 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:38.926 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:47:38.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:37.277395+0000 mgr.y (mgr.14517) 357 : cluster [DBG] pgmap v553: 221 pgs: 2 creating+activating, 1 active+remapped+backfill_toofull, 2 creating+peering, 216 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s; 1/1131 objects misplaced (0.088%); 3 B/s, 0 objects/s recovering 2024-02-23T00:47:38.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:37.545242+0000 osd.5 (osd.5) 164 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:47:38.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:37.545385+0000 osd.5 (osd.5) 165 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:47:38.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:38.181497+0000 mon.a (mon.0) 2441 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-49","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-23T00:47:38.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:38.181597+0000 mon.a (mon.0) 2442 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "16"}]': finished 2024-02-23T00:47:38.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:38.181646+0000 mon.a (mon.0) 2443 : cluster [DBG] osdmap e386: 8 total, 8 up, 7 in 2024-02-23T00:47:38.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:38.188079+0000 mon.a (mon.0) 2444 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-49","var": "target_max_objects","val": "1"}]: dispatch 2024-02-23T00:47:38.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:38.189906+0000 mon.a (mon.0) 2445 : audit [INF] from='client.? 172.21.15.47:0/584262349' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi047-82386-64"}]: dispatch 2024-02-23T00:47:39.027 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:47:39.056 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:39.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.368+0000 7fd6aaec1700 1 -- 172.21.15.47:0/3412646939 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd6a4102180 msgr2=0x7fd6a4104570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:39.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.368+0000 7fd6aaec1700 1 --2- 172.21.15.47:0/3412646939 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd6a4102180 0x7fd6a4104570 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 rx=0x7fd69c009c30 tx=0x7fd69c004c50).stop 2024-02-23T00:47:39.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.368+0000 7fd6aaec1700 1 -- 172.21.15.47:0/3412646939 shutdown_connections 2024-02-23T00:47:39.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.368+0000 7fd6aaec1700 1 --2- 172.21.15.47:0/3412646939 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd6a4102180 0x7fd6a4104570 unknown :-1 s=CLOSED pgs=2652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:39.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.368+0000 7fd6aaec1700 1 --2- 172.21.15.47:0/3412646939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6a40f8c20 0x7fd6a40f9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:39.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.368+0000 7fd6aaec1700 1 --2- 172.21.15.47:0/3412646939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6a40f9580 0x7fd6a4101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:39.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.368+0000 7fd6aaec1700 1 -- 172.21.15.47:0/3412646939 >> 172.21.15.47:0/3412646939 conn(0x7fd6a40f26f0 msgr2=0x7fd6a40f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:39.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.368+0000 7fd6aaec1700 1 -- 172.21.15.47:0/3412646939 shutdown_connections 2024-02-23T00:47:39.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.368+0000 7fd6aaec1700 1 -- 172.21.15.47:0/3412646939 wait complete. 2024-02-23T00:47:39.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.368+0000 7fd6aaec1700 1 librados: starting msgr at 2024-02-23T00:47:39.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.368+0000 7fd6aaec1700 1 librados: starting objecter 2024-02-23T00:47:39.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.368+0000 7fd6aaec1700 1 Processor -- start 2024-02-23T00:47:39.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.368+0000 7fd6aaec1700 1 -- start start 2024-02-23T00:47:39.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.368+0000 7fd6aaec1700 1 librados: setting wanted keys 2024-02-23T00:47:39.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.368+0000 7fd6aaec1700 1 librados: calling monclient init 2024-02-23T00:47:39.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.368+0000 7fd6aaec1700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd6a40f8c20 0x7fd6a40f6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:39.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.368+0000 7fd6aaec1700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6a40f9580 0x7fd6a40f70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:39.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.369+0000 7fd6aaec1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6a4102180 0x7fd6a40f75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:39.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.369+0000 7fd6aaec1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a4116330 con 0x7fd6a4102180 2024-02-23T00:47:39.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.369+0000 7fd6aaec1700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a41164a0 con 0x7fd6a40f9580 2024-02-23T00:47:39.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.369+0000 7fd6aaec1700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a4116660 con 0x7fd6a40f8c20 2024-02-23T00:47:39.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.369+0000 7fd6a8c5d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd6a40f8c20 0x7fd6a40f6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:39.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.369+0000 7fd6a945e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6a4102180 0x7fd6a40f75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:39.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.369+0000 7fd6a945e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6a4102180 0x7fd6a40f75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35928/0 (socket says 172.21.15.47:35928) 2024-02-23T00:47:39.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.369+0000 7fd6a945e700 1 -- 172.21.15.47:0/3943094220 learned_addr learned my addr 172.21.15.47:0/3943094220 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:47:39.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.369+0000 7fd6a8c5d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd6a40f8c20 0x7fd6a40f6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:42006/0 (socket says 172.21.15.47:42006) 2024-02-23T00:47:39.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.369+0000 7fd6a3fff700 1 --2- 172.21.15.47:0/3943094220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6a40f9580 0x7fd6a40f70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:39.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.369+0000 7fd6a945e700 1 -- 172.21.15.47:0/3943094220 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd6a40f8c20 msgr2=0x7fd6a40f6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:39.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.369+0000 7fd6a945e700 1 --2- 172.21.15.47:0/3943094220 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd6a40f8c20 0x7fd6a40f6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:39.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.369+0000 7fd6a945e700 1 -- 172.21.15.47:0/3943094220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6a40f9580 msgr2=0x7fd6a40f70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:39.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.369+0000 7fd6a945e700 1 --2- 172.21.15.47:0/3943094220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6a40f9580 0x7fd6a40f70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:39.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.369+0000 7fd6a945e700 1 -- 172.21.15.47:0/3943094220 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd698005040 con 0x7fd6a4102180 2024-02-23T00:47:39.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.369+0000 7fd6a945e700 1 --2- 172.21.15.47:0/3943094220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6a4102180 0x7fd6a40f75b0 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 rx=0x7fd69c004890 tx=0x7fd69c004c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:39.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.369+0000 7fd6a1ffb700 10 client.?.objecter ms_handle_connect 0x7fd6a4102180 2024-02-23T00:47:39.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.369+0000 7fd6a1ffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:47:39.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.369+0000 7fd6a1ffb700 1 -- 172.21.15.47:0/3943094220 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c00e040 con 0x7fd6a4102180 2024-02-23T00:47:39.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.369+0000 7fd6aaec1700 1 -- 172.21.15.47:0/3943094220 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd69c0097c0 con 0x7fd6a4102180 2024-02-23T00:47:39.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.369+0000 7fd6a1ffb700 1 -- 172.21.15.47:0/3943094220 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fd69c018040 con 0x7fd6a4102180 2024-02-23T00:47:39.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.369+0000 7fd6a1ffb700 1 -- 172.21.15.47:0/3943094220 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c013d80 con 0x7fd6a4102180 2024-02-23T00:47:39.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.370+0000 7fd6aaec1700 10 client.30945.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:47:39.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.370+0000 7fd6aaec1700 1 -- 172.21.15.47:0/3943094220 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6a4116b50 con 0x7fd6a4102180 2024-02-23T00:47:39.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.370+0000 7fd6aaec1700 1 librados: init done 2024-02-23T00:47:39.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.370+0000 7fd6aaec1700 1 -- 172.21.15.47:0/3943094220 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6a405f990 con 0x7fd6a4102180 2024-02-23T00:47:39.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.371+0000 7fd6a1ffb700 1 -- 172.21.15.47:0/3943094220 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fd69c007c80 con 0x7fd6a4102180 2024-02-23T00:47:39.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.371+0000 7fd6a1ffb700 1 --2- 172.21.15.47:0/3943094220 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd694062cc0 0x7fd694065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:39.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.371+0000 7fd6a1ffb700 1 -- 172.21.15.47:0/3943094220 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(387..387 src has 48..387) v4 ==== 9256+0+0 (secure 0 0 0) 0x7fd69c07e7a0 con 0x7fd6a4102180 2024-02-23T00:47:39.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.371+0000 7fd6a1ffb700 10 client.30945.objecter ms_dispatch 0x7fd6a4001550 osd_map(387..387 src has 48..387) v4 2024-02-23T00:47:39.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.371+0000 7fd6a1ffb700 3 client.30945.objecter handle_osd_map got epochs [387,387] > 0 2024-02-23T00:47:39.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.371+0000 7fd6a1ffb700 3 client.30945.objecter handle_osd_map decoding full epoch 387 2024-02-23T00:47:39.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.371+0000 7fd6a8c5d700 1 --2- 172.21.15.47:0/3943094220 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd694062cc0 0x7fd694065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:39.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.371+0000 7fd6a1ffb700 20 client.30945.objecter dump_active .. 0 homeless 2024-02-23T00:47:39.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.372+0000 7fd6a1ffb700 10 client.30945.objecter ms_handle_connect 0x7fd694062cc0 2024-02-23T00:47:39.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.372+0000 7fd6a8c5d700 1 --2- 172.21.15.47:0/3943094220 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd694062cc0 0x7fd694065180 secure :-1 s=READY pgs=3053 cs=0 l=1 rev1=1 rx=0x7fd698004870 tx=0x7fd69800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:39.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.376+0000 7fd6a1ffb700 1 -- 172.21.15.47:0/3943094220 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fd69c04ddc0 con 0x7fd6a4102180 2024-02-23T00:47:39.405 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:47:39.407 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:39.408 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:47:39.509 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:47:39.537 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:39.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.555+0000 7fd6aaec1700 1 -- 172.21.15.47:0/3943094220 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "osd reweight-by-utilization", "oload": 1000, "max_change": 3.0, "target": ["mon-mgr", ""]}) v1 -- 0x7fd6a4106ee0 con 0x7fd694062cc0 2024-02-23T00:47:39.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.556+0000 7fd6a1ffb700 1 -- 172.21.15.47:0/3943094220 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 no change) v1 ==== 17+0+299 (secure 0 0 0) 0x7fd6a4106ee0 con 0x7fd694062cc0 2024-02-23T00:47:39.559 INFO:teuthology.orchestra.run.smithi047.stdout:moved 0 / 639 (0%) 2024-02-23T00:47:39.559 INFO:teuthology.orchestra.run.smithi047.stdout:avg 91.2857 2024-02-23T00:47:39.559 INFO:teuthology.orchestra.run.smithi047.stdout:stddev 7.90505 -> 7.90505 (expected baseline 8.84561) 2024-02-23T00:47:39.559 INFO:teuthology.orchestra.run.smithi047.stdout:min osd.4 with 84 -> 84 pgs (0.920188 -> 0.920188 * mean) 2024-02-23T00:47:39.559 INFO:teuthology.orchestra.run.smithi047.stdout:max osd.7 with 104 -> 104 pgs (1.13928 -> 1.13928 * mean) 2024-02-23T00:47:39.559 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:47:39.559 INFO:teuthology.orchestra.run.smithi047.stdout:oload 1000 2024-02-23T00:47:39.559 INFO:teuthology.orchestra.run.smithi047.stdout:max_change 3 2024-02-23T00:47:39.560 INFO:teuthology.orchestra.run.smithi047.stdout:max_change_osds 4 2024-02-23T00:47:39.560 INFO:teuthology.orchestra.run.smithi047.stdout:average_utilization 0.0033 2024-02-23T00:47:39.560 INFO:teuthology.orchestra.run.smithi047.stdout:overload_utilization 0.0329 2024-02-23T00:47:39.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.558+0000 7fd6aaec1700 10 librados: watch_flush enter 2024-02-23T00:47:39.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.558+0000 7fd6aaec1700 10 librados: watch_flush exit 2024-02-23T00:47:39.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.558+0000 7fd6aaec1700 20 client.30945.objecter shutdown clearing up homeless session... 2024-02-23T00:47:39.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.558+0000 7fd6aaec1700 10 client.30945.objecter successfully canceled tick 2024-02-23T00:47:39.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.558+0000 7fd6aaec1700 1 -- 172.21.15.47:0/3943094220 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd694062cc0 msgr2=0x7fd694065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:39.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.558+0000 7fd6aaec1700 1 --2- 172.21.15.47:0/3943094220 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd694062cc0 0x7fd694065180 secure :-1 s=READY pgs=3053 cs=0 l=1 rev1=1 rx=0x7fd698004870 tx=0x7fd69800b040).stop 2024-02-23T00:47:39.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.558+0000 7fd6aaec1700 1 -- 172.21.15.47:0/3943094220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6a4102180 msgr2=0x7fd6a40f75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:39.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.558+0000 7fd6aaec1700 1 --2- 172.21.15.47:0/3943094220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6a4102180 0x7fd6a40f75b0 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 rx=0x7fd69c004890 tx=0x7fd69c004c50).stop 2024-02-23T00:47:39.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.559+0000 7fd6aaec1700 1 -- 172.21.15.47:0/3943094220 shutdown_connections 2024-02-23T00:47:39.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.559+0000 7fd6aaec1700 1 --2- 172.21.15.47:0/3943094220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6a4102180 0x7fd6a40f75b0 unknown :-1 s=CLOSED pgs=2522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:39.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.559+0000 7fd6aaec1700 1 --2- 172.21.15.47:0/3943094220 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd694062cc0 0x7fd694065180 unknown :-1 s=CLOSED pgs=3053 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:39.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.559+0000 7fd6aaec1700 1 --2- 172.21.15.47:0/3943094220 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd6a40f8c20 0x7fd6a40f6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:39.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.559+0000 7fd6aaec1700 1 --2- 172.21.15.47:0/3943094220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6a40f9580 0x7fd6a40f70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:39.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.559+0000 7fd6aaec1700 1 -- 172.21.15.47:0/3943094220 >> 172.21.15.47:0/3943094220 conn(0x7fd6a40f26f0 msgr2=0x7fd6a40f32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:39.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.559+0000 7fd6aaec1700 1 -- 172.21.15.47:0/3943094220 shutdown_connections 2024-02-23T00:47:39.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.559+0000 7fd6aaec1700 1 -- 172.21.15.47:0/3943094220 wait complete. 2024-02-23T00:47:39.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:39.559+0000 7fd6aaec1700 1 librados: shutdown 2024-02-23T00:47:39.563 INFO:teuthology.orchestra.run.smithi047.stderr:no change 2024-02-23T00:47:40.085 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:47:40.088 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:40.088 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:47:40.189 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:47:40.228 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:40.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:39.182897+0000 mon.a (mon.0) 2446 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-49","var": "target_max_objects","val": "1"}]': finished 2024-02-23T00:47:40.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:39.182976+0000 mon.a (mon.0) 2447 : audit [INF] from='client.? 172.21.15.47:0/584262349' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi047-82386-64"}]': finished 2024-02-23T00:47:40.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:39.183033+0000 mon.a (mon.0) 2448 : cluster [DBG] osdmap e387: 8 total, 8 up, 7 in 2024-02-23T00:47:40.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:39.184490+0000 mon.a (mon.0) 2449 : audit [INF] from='client.? 172.21.15.47:0/584262349' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi047-82386-64"}]: dispatch 2024-02-23T00:47:40.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:39.277823+0000 mgr.y (mgr.14517) 358 : cluster [DBG] pgmap v556: 213 pgs: 1 active+remapped+backfill_toofull, 212 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s; 1/1131 objects misplaced (0.088%); 3 B/s, 0 objects/s recovering 2024-02-23T00:47:40.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:39.557106+0000 mgr.y (mgr.14517) 359 : audit [DBG] from='client.30945 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 1000, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:47:40.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:39.182897+0000 mon.a (mon.0) 2446 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-49","var": "target_max_objects","val": "1"}]': finished 2024-02-23T00:47:40.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:39.182976+0000 mon.a (mon.0) 2447 : audit [INF] from='client.? 172.21.15.47:0/584262349' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi047-82386-64"}]': finished 2024-02-23T00:47:40.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:39.183033+0000 mon.a (mon.0) 2448 : cluster [DBG] osdmap e387: 8 total, 8 up, 7 in 2024-02-23T00:47:40.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:39.184490+0000 mon.a (mon.0) 2449 : audit [INF] from='client.? 172.21.15.47:0/584262349' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi047-82386-64"}]: dispatch 2024-02-23T00:47:40.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:39.277823+0000 mgr.y (mgr.14517) 358 : cluster [DBG] pgmap v556: 213 pgs: 1 active+remapped+backfill_toofull, 212 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s; 1/1131 objects misplaced (0.088%); 3 B/s, 0 objects/s recovering 2024-02-23T00:47:40.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:39.557106+0000 mgr.y (mgr.14517) 359 : audit [DBG] from='client.30945 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 1000, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:47:40.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:39.182897+0000 mon.a (mon.0) 2446 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-49","var": "target_max_objects","val": "1"}]': finished 2024-02-23T00:47:40.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:39.182976+0000 mon.a (mon.0) 2447 : audit [INF] from='client.? 172.21.15.47:0/584262349' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi047-82386-64"}]': finished 2024-02-23T00:47:40.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:39.183033+0000 mon.a (mon.0) 2448 : cluster [DBG] osdmap e387: 8 total, 8 up, 7 in 2024-02-23T00:47:40.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:39.184490+0000 mon.a (mon.0) 2449 : audit [INF] from='client.? 172.21.15.47:0/584262349' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi047-82386-64"}]: dispatch 2024-02-23T00:47:40.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:39.277823+0000 mgr.y (mgr.14517) 358 : cluster [DBG] pgmap v556: 213 pgs: 1 active+remapped+backfill_toofull, 212 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s; 1/1131 objects misplaced (0.088%); 3 B/s, 0 objects/s recovering 2024-02-23T00:47:40.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:39.557106+0000 mgr.y (mgr.14517) 359 : audit [DBG] from='client.30945 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 1000, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:47:40.540 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-02-23T00:47:40.540 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd test-reweight-by-utilization 110 3.0 2024-02-23T00:47:40.770 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:47:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:47:40] "GET /metrics HTTP/1.1" 200 188481 "" "Prometheus/2.33.4" 2024-02-23T00:47:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:47:40.782 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:40.782 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:47:40.883 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:47:40.912 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.068+0000 7fa36fa10700 1 -- 172.21.15.47:0/2578539844 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa3680fcbb0 msgr2=0x7fa3680fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.068+0000 7fa36fa10700 1 --2- 172.21.15.47:0/2578539844 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa3680fcbb0 0x7fa3680fcf90 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 rx=0x7fa3580047b0 tx=0x7fa358009cb0).stop 2024-02-23T00:47:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.068+0000 7fa36fa10700 1 -- 172.21.15.47:0/2578539844 shutdown_connections 2024-02-23T00:47:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.068+0000 7fa36fa10700 1 --2- 172.21.15.47:0/2578539844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3680f7400 0x7fa368103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.068+0000 7fa36fa10700 1 --2- 172.21.15.47:0/2578539844 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa3680fcbb0 0x7fa3680fcf90 unknown :-1 s=CLOSED pgs=2655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.068+0000 7fa36fa10700 1 --2- 172.21.15.47:0/2578539844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3680fd510 0x7fa3680f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.068+0000 7fa36fa10700 1 -- 172.21.15.47:0/2578539844 >> 172.21.15.47:0/2578539844 conn(0x7fa3680f2820 msgr2=0x7fa3680f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.068+0000 7fa36fa10700 1 -- 172.21.15.47:0/2578539844 shutdown_connections 2024-02-23T00:47:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.068+0000 7fa36fa10700 1 -- 172.21.15.47:0/2578539844 wait complete. 2024-02-23T00:47:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.068+0000 7fa36fa10700 1 librados: starting msgr at 2024-02-23T00:47:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.068+0000 7fa36fa10700 1 librados: starting objecter 2024-02-23T00:47:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.068+0000 7fa36fa10700 1 Processor -- start 2024-02-23T00:47:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.068+0000 7fa36fa10700 1 -- start start 2024-02-23T00:47:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.068+0000 7fa36fa10700 1 librados: setting wanted keys 2024-02-23T00:47:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.068+0000 7fa36fa10700 1 librados: calling monclient init 2024-02-23T00:47:41.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.069+0000 7fa36fa10700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3680f7400 0x7fa36810dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:41.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.069+0000 7fa36fa10700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa3680fcbb0 0x7fa36810e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:41.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.069+0000 7fa36fa10700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3680fd510 0x7fa368108150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:41.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.069+0000 7fa36fa10700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa368108640 con 0x7fa3680fd510 2024-02-23T00:47:41.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.069+0000 7fa36fa10700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3681087b0 con 0x7fa3680f7400 2024-02-23T00:47:41.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.069+0000 7fa36fa10700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa368108970 con 0x7fa3680fcbb0 2024-02-23T00:47:41.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.069+0000 7fa36cfab700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa3680fcbb0 0x7fa36810e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:41.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.069+0000 7fa36cfab700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa3680fcbb0 0x7fa36810e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:42034/0 (socket says 172.21.15.47:42034) 2024-02-23T00:47:41.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.069+0000 7fa36cfab700 1 -- 172.21.15.47:0/133782857 learned_addr learned my addr 172.21.15.47:0/133782857 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:47:41.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.069+0000 7fa36dfad700 1 --2- 172.21.15.47:0/133782857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3680fd510 0x7fa368108150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:41.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.069+0000 7fa36d7ac700 1 --2- 172.21.15.47:0/133782857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3680f7400 0x7fa36810dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:41.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.069+0000 7fa36cfab700 1 -- 172.21.15.47:0/133782857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3680f7400 msgr2=0x7fa36810dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:41.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.069+0000 7fa36cfab700 1 --2- 172.21.15.47:0/133782857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3680f7400 0x7fa36810dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:41.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.069+0000 7fa36cfab700 1 -- 172.21.15.47:0/133782857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3680fd510 msgr2=0x7fa368108150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:41.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.069+0000 7fa36cfab700 1 --2- 172.21.15.47:0/133782857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3680fd510 0x7fa368108150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:41.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.069+0000 7fa36cfab700 1 -- 172.21.15.47:0/133782857 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3600097c0 con 0x7fa3680fcbb0 2024-02-23T00:47:41.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.069+0000 7fa36dfad700 1 --2- 172.21.15.47:0/133782857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3680fd510 0x7fa368108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:47:41.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.069+0000 7fa36cfab700 1 --2- 172.21.15.47:0/133782857 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa3680fcbb0 0x7fa36810e4a0 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 rx=0x7fa3640041d0 tx=0x7fa364009f60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:41.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.069+0000 7fa35e7fc700 10 client.?.objecter ms_handle_connect 0x7fa3680fcbb0 2024-02-23T00:47:41.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.069+0000 7fa35e7fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:47:41.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.069+0000 7fa35e7fc700 1 -- 172.21.15.47:0/133782857 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa364007b40 con 0x7fa3680fcbb0 2024-02-23T00:47:41.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.069+0000 7fa35e7fc700 1 -- 172.21.15.47:0/133782857 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fa36400f040 con 0x7fa3680fcbb0 2024-02-23T00:47:41.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.069+0000 7fa36fa10700 1 -- 172.21.15.47:0/133782857 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa358005040 con 0x7fa3680fcbb0 2024-02-23T00:47:41.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.070+0000 7fa35e7fc700 1 -- 172.21.15.47:0/133782857 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa364014690 con 0x7fa3680fcbb0 2024-02-23T00:47:41.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.070+0000 7fa36fa10700 10 client.41285.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:47:41.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.070+0000 7fa36fa10700 1 -- 172.21.15.47:0/133782857 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa368108fb0 con 0x7fa3680fcbb0 2024-02-23T00:47:41.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.070+0000 7fa36fa10700 1 librados: init done 2024-02-23T00:47:41.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.070+0000 7fa36fa10700 1 -- 172.21.15.47:0/133782857 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa36805fa60 con 0x7fa3680fcbb0 2024-02-23T00:47:41.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.071+0000 7fa35e7fc700 1 -- 172.21.15.47:0/133782857 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fa364009110 con 0x7fa3680fcbb0 2024-02-23T00:47:41.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.071+0000 7fa35e7fc700 1 --2- 172.21.15.47:0/133782857 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa354062cc0 0x7fa354065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:41.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.071+0000 7fa35e7fc700 1 -- 172.21.15.47:0/133782857 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(388..388 src has 48..388) v4 ==== 9155+0+0 (secure 0 0 0) 0x7fa36407d730 con 0x7fa3680fcbb0 2024-02-23T00:47:41.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.071+0000 7fa35e7fc700 10 client.41285.objecter ms_dispatch 0x7fa368001550 osd_map(388..388 src has 48..388) v4 2024-02-23T00:47:41.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.071+0000 7fa35e7fc700 3 client.41285.objecter handle_osd_map got epochs [388,388] > 0 2024-02-23T00:47:41.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.071+0000 7fa35e7fc700 3 client.41285.objecter handle_osd_map decoding full epoch 388 2024-02-23T00:47:41.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.071+0000 7fa36d7ac700 1 --2- 172.21.15.47:0/133782857 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa354062cc0 0x7fa354065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:41.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.072+0000 7fa35e7fc700 20 client.41285.objecter dump_active .. 0 homeless 2024-02-23T00:47:41.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.072+0000 7fa36d7ac700 1 --2- 172.21.15.47:0/133782857 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa354062cc0 0x7fa354065180 secure :-1 s=READY pgs=3055 cs=0 l=1 rev1=1 rx=0x7fa358004e20 tx=0x7fa358009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:41.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.072+0000 7fa35e7fc700 10 client.41285.objecter ms_handle_connect 0x7fa354062cc0 2024-02-23T00:47:41.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.076+0000 7fa35e7fc700 1 -- 172.21.15.47:0/133782857 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fa36404cd80 con 0x7fa3680fcbb0 2024-02-23T00:47:41.111 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:47:41.113 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:41.113 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:47:41.154 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:47:41.213 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:47:41.215 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:47:41.225 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:47:41.251 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:41.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:41.297+0000 7fa36fa10700 1 -- 172.21.15.47:0/133782857 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7fa36805ee80 con 0x7fa3680fcbb0 2024-02-23T00:47:41.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:39.827285+0000 osd.2 (osd.2) 159 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:47:41.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:39.827527+0000 osd.2 (osd.2) 160 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:47:41.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:40.180059+0000 mon.a (mon.0) 2450 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-23T00:47:41.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:40.190771+0000 mon.a (mon.0) 2451 : audit [INF] from='client.? 172.21.15.47:0/584262349' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi047-82386-64"}]': finished 2024-02-23T00:47:41.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:40.190850+0000 mon.a (mon.0) 2452 : cluster [DBG] osdmap e388: 8 total, 8 up, 7 in 2024-02-23T00:47:41.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:40.214850+0000 mon.c (mon.2) 289 : audit [INF] from='client.? 172.21.15.47:0/217455010' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi047-82386-65"}]: dispatch 2024-02-23T00:47:41.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:40.215350+0000 mon.a (mon.0) 2453 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi047-82386-65"}]: dispatch 2024-02-23T00:47:41.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:40.215877+0000 mon.c (mon.2) 290 : audit [INF] from='client.? 172.21.15.47:0/217455010' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi047-82386-65"}]: dispatch 2024-02-23T00:47:41.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:40.216133+0000 mon.a (mon.0) 2454 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi047-82386-65"}]: dispatch 2024-02-23T00:47:41.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:40.216647+0000 mon.c (mon.2) 291 : audit [INF] from='client.? 172.21.15.47:0/217455010' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi047-82386-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:47:41.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:40.216967+0000 mon.a (mon.0) 2455 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi047-82386-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:47:41.462 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:39.827285+0000 osd.2 (osd.2) 159 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:47:41.462 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:39.827527+0000 osd.2 (osd.2) 160 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:47:41.462 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:40.180059+0000 mon.a (mon.0) 2450 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-23T00:47:41.462 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:40.190771+0000 mon.a (mon.0) 2451 : audit [INF] from='client.? 172.21.15.47:0/584262349' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi047-82386-64"}]': finished 2024-02-23T00:47:41.462 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:40.190850+0000 mon.a (mon.0) 2452 : cluster [DBG] osdmap e388: 8 total, 8 up, 7 in 2024-02-23T00:47:41.463 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:40.214850+0000 mon.c (mon.2) 289 : audit [INF] from='client.? 172.21.15.47:0/217455010' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi047-82386-65"}]: dispatch 2024-02-23T00:47:41.463 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:40.215350+0000 mon.a (mon.0) 2453 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi047-82386-65"}]: dispatch 2024-02-23T00:47:41.463 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:40.215877+0000 mon.c (mon.2) 290 : audit [INF] from='client.? 172.21.15.47:0/217455010' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi047-82386-65"}]: dispatch 2024-02-23T00:47:41.463 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:40.216133+0000 mon.a (mon.0) 2454 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi047-82386-65"}]: dispatch 2024-02-23T00:47:41.463 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:40.216647+0000 mon.c (mon.2) 291 : audit [INF] from='client.? 172.21.15.47:0/217455010' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi047-82386-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:47:41.463 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:40.216967+0000 mon.a (mon.0) 2455 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi047-82386-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:47:41.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:39.827285+0000 osd.2 (osd.2) 159 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:47:41.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:39.827527+0000 osd.2 (osd.2) 160 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:47:41.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:40.180059+0000 mon.a (mon.0) 2450 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-23T00:47:41.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:40.190771+0000 mon.a (mon.0) 2451 : audit [INF] from='client.? 172.21.15.47:0/584262349' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi047-82386-64"}]': finished 2024-02-23T00:47:41.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:40.190850+0000 mon.a (mon.0) 2452 : cluster [DBG] osdmap e388: 8 total, 8 up, 7 in 2024-02-23T00:47:41.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:40.214850+0000 mon.c (mon.2) 289 : audit [INF] from='client.? 172.21.15.47:0/217455010' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi047-82386-65"}]: dispatch 2024-02-23T00:47:41.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:40.215350+0000 mon.a (mon.0) 2453 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi047-82386-65"}]: dispatch 2024-02-23T00:47:41.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:40.215877+0000 mon.c (mon.2) 290 : audit [INF] from='client.? 172.21.15.47:0/217455010' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi047-82386-65"}]: dispatch 2024-02-23T00:47:41.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:40.216133+0000 mon.a (mon.0) 2454 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi047-82386-65"}]: dispatch 2024-02-23T00:47:41.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:40.216647+0000 mon.c (mon.2) 291 : audit [INF] from='client.? 172.21.15.47:0/217455010' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi047-82386-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:47:41.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:40.216967+0000 mon.a (mon.0) 2455 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi047-82386-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:47:42.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:41.200733+0000 mon.a (mon.0) 2456 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi047-82386-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:47:42.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:41.200895+0000 mon.a (mon.0) 2457 : cluster [DBG] osdmap e389: 8 total, 8 up, 7 in 2024-02-23T00:47:42.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:41.205922+0000 mon.c (mon.2) 292 : audit [INF] from='client.? 172.21.15.47:0/217455010' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi047-82386-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi047-82386-65"}]: dispatch 2024-02-23T00:47:42.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:41.206918+0000 mon.a (mon.0) 2458 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi047-82386-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi047-82386-65"}]: dispatch 2024-02-23T00:47:42.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:41.278233+0000 mgr.y (mgr.14517) 360 : cluster [DBG] pgmap v559: 212 pgs: 1 active+remapped+backfill_toofull, 211 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1/1131 objects misplaced (0.088%) 2024-02-23T00:47:42.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:41.279798+0000 mon.a (mon.0) 2459 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "15"}]: dispatch 2024-02-23T00:47:42.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:41.299063+0000 mon.c (mon.2) 293 : audit [INF] from='client.? 172.21.15.47:0/133782857' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:47:42.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:41.299572+0000 mon.a (mon.0) 2460 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:47:42.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:41.302126+0000 mon.a (mon.0) 2461 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:47:42.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:41.495548+0000 mon.a (mon.0) 2462 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:47:42.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:41.200733+0000 mon.a (mon.0) 2456 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi047-82386-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:47:42.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:41.200895+0000 mon.a (mon.0) 2457 : cluster [DBG] osdmap e389: 8 total, 8 up, 7 in 2024-02-23T00:47:42.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:41.205922+0000 mon.c (mon.2) 292 : audit [INF] from='client.? 172.21.15.47:0/217455010' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi047-82386-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi047-82386-65"}]: dispatch 2024-02-23T00:47:42.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:41.206918+0000 mon.a (mon.0) 2458 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi047-82386-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi047-82386-65"}]: dispatch 2024-02-23T00:47:42.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:41.278233+0000 mgr.y (mgr.14517) 360 : cluster [DBG] pgmap v559: 212 pgs: 1 active+remapped+backfill_toofull, 211 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1/1131 objects misplaced (0.088%) 2024-02-23T00:47:42.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:41.279798+0000 mon.a (mon.0) 2459 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "15"}]: dispatch 2024-02-23T00:47:42.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:41.299063+0000 mon.c (mon.2) 293 : audit [INF] from='client.? 172.21.15.47:0/133782857' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:47:42.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:41.299572+0000 mon.a (mon.0) 2460 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:47:42.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:41.302126+0000 mon.a (mon.0) 2461 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:47:42.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:41.495548+0000 mon.a (mon.0) 2462 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:47:42.400 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:47:42.403 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:42.404 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:47:42.505 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:47:42.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:41.200733+0000 mon.a (mon.0) 2456 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi047-82386-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:47:42.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:41.200895+0000 mon.a (mon.0) 2457 : cluster [DBG] osdmap e389: 8 total, 8 up, 7 in 2024-02-23T00:47:42.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:41.205922+0000 mon.c (mon.2) 292 : audit [INF] from='client.? 172.21.15.47:0/217455010' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi047-82386-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi047-82386-65"}]: dispatch 2024-02-23T00:47:42.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:41.206918+0000 mon.a (mon.0) 2458 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi047-82386-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi047-82386-65"}]: dispatch 2024-02-23T00:47:42.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:41.278233+0000 mgr.y (mgr.14517) 360 : cluster [DBG] pgmap v559: 212 pgs: 1 active+remapped+backfill_toofull, 211 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1/1131 objects misplaced (0.088%) 2024-02-23T00:47:42.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:41.279798+0000 mon.a (mon.0) 2459 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "15"}]: dispatch 2024-02-23T00:47:42.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:41.299063+0000 mon.c (mon.2) 293 : audit [INF] from='client.? 172.21.15.47:0/133782857' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:47:42.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:41.299572+0000 mon.a (mon.0) 2460 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:47:42.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:41.302126+0000 mon.a (mon.0) 2461 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:47:42.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:41.495548+0000 mon.a (mon.0) 2462 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:47:42.540 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:42.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:42.643+0000 7fa35e7fc700 1 -- 172.21.15.47:0/133782857 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v390) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa3640503a0 con 0x7fa3680fcbb0 2024-02-23T00:47:42.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:42.645+0000 7fa36fa10700 10 librados: watch_flush enter 2024-02-23T00:47:42.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:42.645+0000 7fa36fa10700 10 librados: watch_flush exit 2024-02-23T00:47:42.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:42.645+0000 7fa36fa10700 20 client.41285.objecter shutdown clearing up homeless session... 2024-02-23T00:47:42.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:42.645+0000 7fa36fa10700 10 client.41285.objecter successfully canceled tick 2024-02-23T00:47:42.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:42.645+0000 7fa36fa10700 1 -- 172.21.15.47:0/133782857 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa354062cc0 msgr2=0x7fa354065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:42.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:42.645+0000 7fa36fa10700 1 --2- 172.21.15.47:0/133782857 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa354062cc0 0x7fa354065180 secure :-1 s=READY pgs=3055 cs=0 l=1 rev1=1 rx=0x7fa358004e20 tx=0x7fa358009cb0).stop 2024-02-23T00:47:42.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:42.645+0000 7fa36fa10700 1 -- 172.21.15.47:0/133782857 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa3680fcbb0 msgr2=0x7fa36810e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:42.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:42.645+0000 7fa36fa10700 1 --2- 172.21.15.47:0/133782857 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa3680fcbb0 0x7fa36810e4a0 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 rx=0x7fa3640041d0 tx=0x7fa364009f60).stop 2024-02-23T00:47:42.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:42.645+0000 7fa36fa10700 1 -- 172.21.15.47:0/133782857 shutdown_connections 2024-02-23T00:47:42.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:42.645+0000 7fa36fa10700 1 --2- 172.21.15.47:0/133782857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3680fd510 0x7fa368108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:42.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:42.645+0000 7fa36fa10700 1 --2- 172.21.15.47:0/133782857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3680f7400 0x7fa36810dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:42.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:42.645+0000 7fa36fa10700 1 --2- 172.21.15.47:0/133782857 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa354062cc0 0x7fa354065180 unknown :-1 s=CLOSED pgs=3055 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:42.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:42.645+0000 7fa36fa10700 1 --2- 172.21.15.47:0/133782857 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa3680fcbb0 0x7fa36810e4a0 unknown :-1 s=CLOSED pgs=2656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:42.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:42.645+0000 7fa36fa10700 1 -- 172.21.15.47:0/133782857 >> 172.21.15.47:0/133782857 conn(0x7fa3680f2820 msgr2=0x7fa368060e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:42.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:42.645+0000 7fa36fa10700 1 -- 172.21.15.47:0/133782857 shutdown_connections 2024-02-23T00:47:42.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:42.645+0000 7fa36fa10700 1 -- 172.21.15.47:0/133782857 wait complete. 2024-02-23T00:47:42.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:42.645+0000 7fa36fa10700 1 librados: shutdown 2024-02-23T00:47:42.650 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is set 2024-02-23T00:47:42.943 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:47:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:47:42.944 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:47:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:47:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:47:43.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:41.362936+0000 osd.6 (osd.6) 155 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:47:43.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:41.363213+0000 osd.6 (osd.6) 156 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:47:43.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:42.198143+0000 mon.a (mon.0) 2463 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:47:43.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:42.559694+0000 mgr.y (mgr.14517) 361 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:47:43.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:42.643862+0000 mon.a (mon.0) 2464 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "15"}]': finished 2024-02-23T00:47:43.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:42.643941+0000 mon.a (mon.0) 2465 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:47:43.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:42.644027+0000 mon.a (mon.0) 2466 : cluster [DBG] osdmap e390: 8 total, 8 up, 7 in 2024-02-23T00:47:43.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:43.152976+0000 mon.a (mon.0) 2467 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.10", "id": [7, 3]}]: dispatch 2024-02-23T00:47:43.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:43.153251+0000 mon.a (mon.0) 2468 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.11", "id": [4, 3]}]: dispatch 2024-02-23T00:47:43.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:43.153427+0000 mon.a (mon.0) 2469 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.13", "id": [0, 6]}]: dispatch 2024-02-23T00:47:43.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:43.153595+0000 mon.a (mon.0) 2470 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.f", "id": [7, 5]}]: dispatch 2024-02-23T00:47:43.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:43.153768+0000 mon.a (mon.0) 2471 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "204.2", "id": [5, 0]}]: dispatch 2024-02-23T00:47:43.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:43.153938+0000 mon.a (mon.0) 2472 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "204.4", "id": [5, 4]}]: dispatch 2024-02-23T00:47:43.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:41.362936+0000 osd.6 (osd.6) 155 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:47:43.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:41.363213+0000 osd.6 (osd.6) 156 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:47:43.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:42.198143+0000 mon.a (mon.0) 2463 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:47:43.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:42.559694+0000 mgr.y (mgr.14517) 361 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:47:43.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:42.643862+0000 mon.a (mon.0) 2464 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "15"}]': finished 2024-02-23T00:47:43.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:42.643941+0000 mon.a (mon.0) 2465 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:47:43.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:42.644027+0000 mon.a (mon.0) 2466 : cluster [DBG] osdmap e390: 8 total, 8 up, 7 in 2024-02-23T00:47:43.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:43.152976+0000 mon.a (mon.0) 2467 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.10", "id": [7, 3]}]: dispatch 2024-02-23T00:47:43.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:43.153251+0000 mon.a (mon.0) 2468 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.11", "id": [4, 3]}]: dispatch 2024-02-23T00:47:43.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:43.153427+0000 mon.a (mon.0) 2469 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.13", "id": [0, 6]}]: dispatch 2024-02-23T00:47:43.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:43.153595+0000 mon.a (mon.0) 2470 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.f", "id": [7, 5]}]: dispatch 2024-02-23T00:47:43.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:43.153768+0000 mon.a (mon.0) 2471 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "204.2", "id": [5, 0]}]: dispatch 2024-02-23T00:47:43.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:43.153938+0000 mon.a (mon.0) 2472 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "204.4", "id": [5, 4]}]: dispatch 2024-02-23T00:47:43.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:41.362936+0000 osd.6 (osd.6) 155 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:47:43.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:41.363213+0000 osd.6 (osd.6) 156 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:47:43.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:42.198143+0000 mon.a (mon.0) 2463 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:47:43.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:42.559694+0000 mgr.y (mgr.14517) 361 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:47:43.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:42.643862+0000 mon.a (mon.0) 2464 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "15"}]': finished 2024-02-23T00:47:43.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:42.643941+0000 mon.a (mon.0) 2465 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:47:43.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:42.644027+0000 mon.a (mon.0) 2466 : cluster [DBG] osdmap e390: 8 total, 8 up, 7 in 2024-02-23T00:47:43.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:43.152976+0000 mon.a (mon.0) 2467 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.10", "id": [7, 3]}]: dispatch 2024-02-23T00:47:43.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:43.153251+0000 mon.a (mon.0) 2468 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.11", "id": [4, 3]}]: dispatch 2024-02-23T00:47:43.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:43.153427+0000 mon.a (mon.0) 2469 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.13", "id": [0, 6]}]: dispatch 2024-02-23T00:47:43.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:43.153595+0000 mon.a (mon.0) 2470 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.f", "id": [7, 5]}]: dispatch 2024-02-23T00:47:43.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:43.153768+0000 mon.a (mon.0) 2471 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "204.2", "id": [5, 0]}]: dispatch 2024-02-23T00:47:43.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:43.153938+0000 mon.a (mon.0) 2472 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "204.4", "id": [5, 4]}]: dispatch 2024-02-23T00:47:44.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:43.211383+0000 mon.a (mon.0) 2473 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi047-82386-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi047-82386-65"}]': finished 2024-02-23T00:47:44.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:43.211532+0000 mon.a (mon.0) 2474 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.10", "id": [7, 3]}]': finished 2024-02-23T00:47:44.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:43.211606+0000 mon.a (mon.0) 2475 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.11", "id": [4, 3]}]': finished 2024-02-23T00:47:44.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:43.211671+0000 mon.a (mon.0) 2476 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.13", "id": [0, 6]}]': finished 2024-02-23T00:47:44.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:43.211737+0000 mon.a (mon.0) 2477 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.f", "id": [7, 5]}]': finished 2024-02-23T00:47:44.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:43.211801+0000 mon.a (mon.0) 2478 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "204.2", "id": [5, 0]}]': finished 2024-02-23T00:47:44.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:43.211862+0000 mon.a (mon.0) 2479 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "204.4", "id": [5, 4]}]': finished 2024-02-23T00:47:44.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:43.211925+0000 mon.a (mon.0) 2480 : cluster [DBG] osdmap e391: 8 total, 8 up, 7 in 2024-02-23T00:47:44.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:43.278901+0000 mgr.y (mgr.14517) 362 : cluster [DBG] pgmap v562: 220 pgs: 8 unknown, 1 active+remapped+backfill_toofull, 211 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s; 1/1125 objects misplaced (0.089%) 2024-02-23T00:47:44.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:43.211383+0000 mon.a (mon.0) 2473 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi047-82386-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi047-82386-65"}]': finished 2024-02-23T00:47:44.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:43.211532+0000 mon.a (mon.0) 2474 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.10", "id": [7, 3]}]': finished 2024-02-23T00:47:44.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:43.211606+0000 mon.a (mon.0) 2475 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.11", "id": [4, 3]}]': finished 2024-02-23T00:47:44.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:43.211671+0000 mon.a (mon.0) 2476 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.13", "id": [0, 6]}]': finished 2024-02-23T00:47:44.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:43.211737+0000 mon.a (mon.0) 2477 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.f", "id": [7, 5]}]': finished 2024-02-23T00:47:44.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:43.211801+0000 mon.a (mon.0) 2478 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "204.2", "id": [5, 0]}]': finished 2024-02-23T00:47:44.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:43.211862+0000 mon.a (mon.0) 2479 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "204.4", "id": [5, 4]}]': finished 2024-02-23T00:47:44.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:43.211925+0000 mon.a (mon.0) 2480 : cluster [DBG] osdmap e391: 8 total, 8 up, 7 in 2024-02-23T00:47:44.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:43.278901+0000 mgr.y (mgr.14517) 362 : cluster [DBG] pgmap v562: 220 pgs: 8 unknown, 1 active+remapped+backfill_toofull, 211 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s; 1/1125 objects misplaced (0.089%) 2024-02-23T00:47:44.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:43.211383+0000 mon.a (mon.0) 2473 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi047-82386-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi047-82386-65"}]': finished 2024-02-23T00:47:44.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:43.211532+0000 mon.a (mon.0) 2474 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.10", "id": [7, 3]}]': finished 2024-02-23T00:47:44.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:43.211606+0000 mon.a (mon.0) 2475 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.11", "id": [4, 3]}]': finished 2024-02-23T00:47:44.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:43.211671+0000 mon.a (mon.0) 2476 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.13", "id": [0, 6]}]': finished 2024-02-23T00:47:44.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:43.211737+0000 mon.a (mon.0) 2477 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.f", "id": [7, 5]}]': finished 2024-02-23T00:47:44.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:43.211801+0000 mon.a (mon.0) 2478 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "204.2", "id": [5, 0]}]': finished 2024-02-23T00:47:44.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:43.211862+0000 mon.a (mon.0) 2479 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "204.4", "id": [5, 4]}]': finished 2024-02-23T00:47:44.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:43.211925+0000 mon.a (mon.0) 2480 : cluster [DBG] osdmap e391: 8 total, 8 up, 7 in 2024-02-23T00:47:44.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:43.278901+0000 mgr.y (mgr.14517) 362 : cluster [DBG] pgmap v562: 220 pgs: 8 unknown, 1 active+remapped+backfill_toofull, 211 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s; 1/1125 objects misplaced (0.089%) 2024-02-23T00:47:45.089 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:47:45.093 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:45.093 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:47:45.194 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:47:45.231 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:45.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:44.220037+0000 mon.a (mon.0) 2481 : cluster [DBG] osdmap e392: 8 total, 8 up, 7 in 2024-02-23T00:47:45.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:45.173662+0000 mon.a (mon.0) 2482 : cluster [DBG] osdmap e393: 8 total, 8 up, 7 in 2024-02-23T00:47:45.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:45.175585+0000 mon.c (mon.2) 294 : audit [INF] from='client.? 172.21.15.47:0/217455010' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi047-82386-65"}]: dispatch 2024-02-23T00:47:45.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:45.175937+0000 mon.a (mon.0) 2483 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi047-82386-65"}]: dispatch 2024-02-23T00:47:45.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:44.220037+0000 mon.a (mon.0) 2481 : cluster [DBG] osdmap e392: 8 total, 8 up, 7 in 2024-02-23T00:47:45.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:45.173662+0000 mon.a (mon.0) 2482 : cluster [DBG] osdmap e393: 8 total, 8 up, 7 in 2024-02-23T00:47:45.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:45.175585+0000 mon.c (mon.2) 294 : audit [INF] from='client.? 172.21.15.47:0/217455010' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi047-82386-65"}]: dispatch 2024-02-23T00:47:45.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:45.175937+0000 mon.a (mon.0) 2483 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi047-82386-65"}]: dispatch 2024-02-23T00:47:45.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:44.220037+0000 mon.a (mon.0) 2481 : cluster [DBG] osdmap e392: 8 total, 8 up, 7 in 2024-02-23T00:47:45.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:45.173662+0000 mon.a (mon.0) 2482 : cluster [DBG] osdmap e393: 8 total, 8 up, 7 in 2024-02-23T00:47:45.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:45.175585+0000 mon.c (mon.2) 294 : audit [INF] from='client.? 172.21.15.47:0/217455010' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi047-82386-65"}]: dispatch 2024-02-23T00:47:45.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:45.175937+0000 mon.a (mon.0) 2483 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi047-82386-65"}]: dispatch 2024-02-23T00:47:46.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.410+0000 7f9cda238700 1 -- 172.21.15.47:0/3192870647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cd40f8420 msgr2=0x7f9cd4104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:46.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.410+0000 7f9cda238700 1 --2- 172.21.15.47:0/3192870647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cd40f8420 0x7f9cd4104a20 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 rx=0x7f9ccc009be0 tx=0x7f9ccc004d10).stop 2024-02-23T00:47:46.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.410+0000 7f9cda238700 1 -- 172.21.15.47:0/3192870647 shutdown_connections 2024-02-23T00:47:46.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.410+0000 7f9cda238700 1 --2- 172.21.15.47:0/3192870647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cd40f8420 0x7f9cd4104a20 unknown :-1 s=CLOSED pgs=2523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:46.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.410+0000 7f9cda238700 1 --2- 172.21.15.47:0/3192870647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9cd40fdbb0 0x7f9cd40fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:46.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.410+0000 7f9cda238700 1 --2- 172.21.15.47:0/3192870647 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9cd40fe510 0x7f9cd40f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:46.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.410+0000 7f9cda238700 1 -- 172.21.15.47:0/3192870647 >> 172.21.15.47:0/3192870647 conn(0x7f9cd40f3800 msgr2=0x7f9cd40f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:46.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.410+0000 7f9cda238700 1 -- 172.21.15.47:0/3192870647 shutdown_connections 2024-02-23T00:47:46.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.410+0000 7f9cda238700 1 -- 172.21.15.47:0/3192870647 wait complete. 2024-02-23T00:47:46.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.410+0000 7f9cda238700 1 librados: starting msgr at 2024-02-23T00:47:46.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.410+0000 7f9cda238700 1 librados: starting objecter 2024-02-23T00:47:46.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.410+0000 7f9cda238700 1 Processor -- start 2024-02-23T00:47:46.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.410+0000 7f9cda238700 1 -- start start 2024-02-23T00:47:46.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.410+0000 7f9cda238700 1 librados: setting wanted keys 2024-02-23T00:47:46.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.410+0000 7f9cda238700 1 librados: calling monclient init 2024-02-23T00:47:46.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.411+0000 7f9cda238700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cd40f8420 0x7f9cd410dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:46.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.411+0000 7f9cda238700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9cd40fdbb0 0x7f9cd410e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:46.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.411+0000 7f9cda238700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9cd40fe510 0x7f9cd4107f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:46.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.411+0000 7f9cda238700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd41083f0 con 0x7f9cd40f8420 2024-02-23T00:47:46.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.411+0000 7f9cda238700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd4108560 con 0x7f9cd40fdbb0 2024-02-23T00:47:46.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.411+0000 7f9cda238700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd4108720 con 0x7f9cd40fe510 2024-02-23T00:47:46.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.411+0000 7f9cd37fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cd40f8420 0x7f9cd410dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:46.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.411+0000 7f9cd3fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9cd40fe510 0x7f9cd4107f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:46.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.411+0000 7f9cd37fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cd40f8420 0x7f9cd410dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35990/0 (socket says 172.21.15.47:35990) 2024-02-23T00:47:46.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.411+0000 7f9cd37fe700 1 -- 172.21.15.47:0/1330192963 learned_addr learned my addr 172.21.15.47:0/1330192963 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:47:46.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.411+0000 7f9cd2ffd700 1 --2- 172.21.15.47:0/1330192963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9cd40fdbb0 0x7f9cd410e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:46.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.411+0000 7f9cd37fe700 1 -- 172.21.15.47:0/1330192963 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9cd40fe510 msgr2=0x7f9cd4107f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:46.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.411+0000 7f9cd37fe700 1 --2- 172.21.15.47:0/1330192963 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9cd40fe510 0x7f9cd4107f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:46.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.411+0000 7f9cd37fe700 1 -- 172.21.15.47:0/1330192963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9cd40fdbb0 msgr2=0x7f9cd410e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:46.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.411+0000 7f9cd37fe700 1 --2- 172.21.15.47:0/1330192963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9cd40fdbb0 0x7f9cd410e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:46.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.411+0000 7f9cd37fe700 1 -- 172.21.15.47:0/1330192963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc8005040 con 0x7f9cd40f8420 2024-02-23T00:47:46.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.411+0000 7f9cd2ffd700 1 --2- 172.21.15.47:0/1330192963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9cd40fdbb0 0x7f9cd410e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:47:46.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.411+0000 7f9cd3fff700 1 --2- 172.21.15.47:0/1330192963 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9cd40fe510 0x7f9cd4107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:47:46.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.411+0000 7f9cd37fe700 1 --2- 172.21.15.47:0/1330192963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cd40f8420 0x7f9cd410dd60 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 rx=0x7f9cc400e970 tx=0x7f9cc400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:46.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.411+0000 7f9cd0ff9700 10 client.?.objecter ms_handle_connect 0x7f9cd40f8420 2024-02-23T00:47:46.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.411+0000 7f9cd0ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:47:46.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.411+0000 7f9cd0ff9700 1 -- 172.21.15.47:0/1330192963 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc400ee60 con 0x7f9cd40f8420 2024-02-23T00:47:46.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.411+0000 7f9cda238700 1 -- 172.21.15.47:0/1330192963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ccc0097c0 con 0x7f9cd40f8420 2024-02-23T00:47:46.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.411+0000 7f9cd0ff9700 1 -- 172.21.15.47:0/1330192963 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f9cc400f040 con 0x7f9cd40f8420 2024-02-23T00:47:46.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.411+0000 7f9cd0ff9700 1 -- 172.21.15.47:0/1330192963 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc400c5b0 con 0x7f9cd40f8420 2024-02-23T00:47:46.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.411+0000 7f9cda238700 10 client.30963.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:47:46.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.412+0000 7f9cda238700 1 -- 172.21.15.47:0/1330192963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd4114b90 con 0x7f9cd40f8420 2024-02-23T00:47:46.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.412+0000 7f9cda238700 1 librados: init done 2024-02-23T00:47:46.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.412+0000 7f9cda238700 1 -- 172.21.15.47:0/1330192963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cd405ee80 con 0x7f9cd40f8420 2024-02-23T00:47:46.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.413+0000 7f9cd0ff9700 1 -- 172.21.15.47:0/1330192963 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f9cc4007b40 con 0x7f9cd40f8420 2024-02-23T00:47:46.156 INFO:teuthology.orchestra.run.smithi047.stdout:moved 0 / 633 (0%) 2024-02-23T00:47:46.156 INFO:teuthology.orchestra.run.smithi047.stdout:avg 90.4286 2024-02-23T00:47:46.156 INFO:teuthology.orchestra.run.smithi047.stdout:stddev 8.01529 -> 8.01529 (expected baseline 8.80399) 2024-02-23T00:47:46.157 INFO:teuthology.orchestra.run.smithi047.stdout:min osd.5 with 81 -> 81 pgs (0.895735 -> 0.895735 * mean) 2024-02-23T00:47:46.157 INFO:teuthology.orchestra.run.smithi047.stdout:max osd.7 with 102 -> 102 pgs (1.12796 -> 1.12796 * mean) 2024-02-23T00:47:46.157 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:47:46.157 INFO:teuthology.orchestra.run.smithi047.stdout:oload 110 2024-02-23T00:47:46.157 INFO:teuthology.orchestra.run.smithi047.stdout:max_change 3 2024-02-23T00:47:46.157 INFO:teuthology.orchestra.run.smithi047.stdout:max_change_osds 4 2024-02-23T00:47:46.157 INFO:teuthology.orchestra.run.smithi047.stdout:average_utilization 0.0033 2024-02-23T00:47:46.157 INFO:teuthology.orchestra.run.smithi047.stdout:overload_utilization 0.0036 2024-02-23T00:47:46.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.413+0000 7f9cd0ff9700 1 --2- 172.21.15.47:0/1330192963 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9cbc062cc0 0x7f9cbc065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:46.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.413+0000 7f9cd0ff9700 1 -- 172.21.15.47:0/1330192963 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(393..393 src has 48..393) v4 ==== 9753+0+0 (secure 0 0 0) 0x7f9cc407d4a0 con 0x7f9cd40f8420 2024-02-23T00:47:46.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.413+0000 7f9cd0ff9700 10 client.30963.objecter ms_dispatch 0x7f9cd4001550 osd_map(393..393 src has 48..393) v4 2024-02-23T00:47:46.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.413+0000 7f9cd0ff9700 3 client.30963.objecter handle_osd_map got epochs [393,393] > 0 2024-02-23T00:47:46.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.413+0000 7f9cd0ff9700 3 client.30963.objecter handle_osd_map decoding full epoch 393 2024-02-23T00:47:46.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.413+0000 7f9cd2ffd700 1 --2- 172.21.15.47:0/1330192963 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9cbc062cc0 0x7f9cbc065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:46.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.414+0000 7f9cd0ff9700 20 client.30963.objecter dump_active .. 0 homeless 2024-02-23T00:47:46.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.414+0000 7f9cd0ff9700 10 client.30963.objecter ms_handle_connect 0x7f9cbc062cc0 2024-02-23T00:47:46.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.414+0000 7f9cd2ffd700 1 --2- 172.21.15.47:0/1330192963 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9cbc062cc0 0x7f9cbc065180 secure :-1 s=READY pgs=3056 cs=0 l=1 rev1=1 rx=0x7f9cd41096a0 tx=0x7f9cc8009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:46.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.418+0000 7f9cd0ff9700 1 -- 172.21.15.47:0/1330192963 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f9cc404af60 con 0x7f9cd40f8420 2024-02-23T00:47:46.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.593+0000 7f9cda238700 1 -- 172.21.15.47:0/1330192963 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 3.0, "target": ["mon-mgr", ""]}) v1 -- 0x7f9cd405fad0 con 0x7f9cbc062cc0 2024-02-23T00:47:46.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.594+0000 7f9cd0ff9700 1 -- 172.21.15.47:0/1330192963 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 no change) v1 ==== 17+0+298 (secure 0 0 0) 0x7f9cd405fad0 con 0x7f9cbc062cc0 2024-02-23T00:47:46.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.595+0000 7f9cda238700 10 librados: watch_flush enter 2024-02-23T00:47:46.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.596+0000 7f9cda238700 10 librados: watch_flush exit 2024-02-23T00:47:46.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.596+0000 7f9cda238700 20 client.30963.objecter shutdown clearing up homeless session... 2024-02-23T00:47:46.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.596+0000 7f9cda238700 10 client.30963.objecter successfully canceled tick 2024-02-23T00:47:46.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.596+0000 7f9cda238700 1 -- 172.21.15.47:0/1330192963 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9cbc062cc0 msgr2=0x7f9cbc065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:46.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.596+0000 7f9cda238700 1 --2- 172.21.15.47:0/1330192963 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9cbc062cc0 0x7f9cbc065180 secure :-1 s=READY pgs=3056 cs=0 l=1 rev1=1 rx=0x7f9cd41096a0 tx=0x7f9cc8009cb0).stop 2024-02-23T00:47:46.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.596+0000 7f9cda238700 1 -- 172.21.15.47:0/1330192963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cd40f8420 msgr2=0x7f9cd410dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:46.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.596+0000 7f9cda238700 1 --2- 172.21.15.47:0/1330192963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cd40f8420 0x7f9cd410dd60 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 rx=0x7f9cc400e970 tx=0x7f9cc400b040).stop 2024-02-23T00:47:46.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.596+0000 7f9cda238700 1 -- 172.21.15.47:0/1330192963 shutdown_connections 2024-02-23T00:47:46.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.596+0000 7f9cda238700 1 --2- 172.21.15.47:0/1330192963 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9cbc062cc0 0x7f9cbc065180 unknown :-1 s=CLOSED pgs=3056 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:46.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.596+0000 7f9cda238700 1 --2- 172.21.15.47:0/1330192963 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9cd40fe510 0x7f9cd4107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:46.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.596+0000 7f9cda238700 1 --2- 172.21.15.47:0/1330192963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cd40f8420 0x7f9cd410dd60 unknown :-1 s=CLOSED pgs=2524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:46.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.596+0000 7f9cda238700 1 --2- 172.21.15.47:0/1330192963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9cd40fdbb0 0x7f9cd410e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:46.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.596+0000 7f9cda238700 1 -- 172.21.15.47:0/1330192963 >> 172.21.15.47:0/1330192963 conn(0x7f9cd40f3800 msgr2=0x7f9cd40f5150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:46.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.596+0000 7f9cda238700 1 -- 172.21.15.47:0/1330192963 shutdown_connections 2024-02-23T00:47:46.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.596+0000 7f9cda238700 1 -- 172.21.15.47:0/1330192963 wait complete. 2024-02-23T00:47:46.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:45.596+0000 7f9cda238700 1 librados: shutdown 2024-02-23T00:47:46.159 INFO:teuthology.orchestra.run.smithi047.stderr:no change 2024-02-23T00:47:46.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:45.279554+0000 mgr.y (mgr.14517) 363 : cluster [DBG] pgmap v565: 211 pgs: 1 active+remapped+backfill_toofull, 210 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s; 1/1125 objects misplaced (0.089%) 2024-02-23T00:47:46.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:45.280455+0000 mon.a (mon.0) 2484 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "14"}]: dispatch 2024-02-23T00:47:46.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:45.594481+0000 mgr.y (mgr.14517) 364 : audit [DBG] from='client.30963 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:47:46.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:46.174928+0000 mon.a (mon.0) 2485 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi047-82386-65"}]': finished 2024-02-23T00:47:46.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:46.175032+0000 mon.a (mon.0) 2486 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "14"}]': finished 2024-02-23T00:47:46.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:46.175094+0000 mon.a (mon.0) 2487 : cluster [DBG] osdmap e394: 8 total, 8 up, 7 in 2024-02-23T00:47:46.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:46.178175+0000 mon.c (mon.2) 295 : audit [INF] from='client.? 172.21.15.47:0/217455010' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi047-82386-65"}]: dispatch 2024-02-23T00:47:46.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:46.178964+0000 mon.a (mon.0) 2488 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi047-82386-65"}]: dispatch 2024-02-23T00:47:46.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:45.279554+0000 mgr.y (mgr.14517) 363 : cluster [DBG] pgmap v565: 211 pgs: 1 active+remapped+backfill_toofull, 210 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s; 1/1125 objects misplaced (0.089%) 2024-02-23T00:47:46.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:45.280455+0000 mon.a (mon.0) 2484 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "14"}]: dispatch 2024-02-23T00:47:46.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:45.594481+0000 mgr.y (mgr.14517) 364 : audit [DBG] from='client.30963 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:47:46.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:46.174928+0000 mon.a (mon.0) 2485 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi047-82386-65"}]': finished 2024-02-23T00:47:46.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:46.175032+0000 mon.a (mon.0) 2486 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "14"}]': finished 2024-02-23T00:47:46.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:46.175094+0000 mon.a (mon.0) 2487 : cluster [DBG] osdmap e394: 8 total, 8 up, 7 in 2024-02-23T00:47:46.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:46.178175+0000 mon.c (mon.2) 295 : audit [INF] from='client.? 172.21.15.47:0/217455010' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi047-82386-65"}]: dispatch 2024-02-23T00:47:46.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:46.178964+0000 mon.a (mon.0) 2488 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi047-82386-65"}]: dispatch 2024-02-23T00:47:46.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:45.279554+0000 mgr.y (mgr.14517) 363 : cluster [DBG] pgmap v565: 211 pgs: 1 active+remapped+backfill_toofull, 210 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s; 1/1125 objects misplaced (0.089%) 2024-02-23T00:47:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:45.280455+0000 mon.a (mon.0) 2484 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "14"}]: dispatch 2024-02-23T00:47:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:45.594481+0000 mgr.y (mgr.14517) 364 : audit [DBG] from='client.30963 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:47:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:46.174928+0000 mon.a (mon.0) 2485 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi047-82386-65"}]': finished 2024-02-23T00:47:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:46.175032+0000 mon.a (mon.0) 2486 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "14"}]': finished 2024-02-23T00:47:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:46.175094+0000 mon.a (mon.0) 2487 : cluster [DBG] osdmap e394: 8 total, 8 up, 7 in 2024-02-23T00:47:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:46.178175+0000 mon.c (mon.2) 295 : audit [INF] from='client.? 172.21.15.47:0/217455010' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi047-82386-65"}]: dispatch 2024-02-23T00:47:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:46.178964+0000 mon.a (mon.0) 2488 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi047-82386-65"}]: dispatch 2024-02-23T00:47:46.937 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-02-23T00:47:46.938 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight-by-utilization 110 1.0 2024-02-23T00:47:47.186 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-02-23T00:47:47.186 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (6977 ms) 2024-02-23T00:47:47.187 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2024-02-23T00:47:47.187 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (7001 ms) 2024-02-23T00:47:47.187 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2024-02-23T00:47:47.187 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (7007 ms) 2024-02-23T00:47:47.187 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2024-02-23T00:47:47.187 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (7007 ms) 2024-02-23T00:47:47.187 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2024-02-23T00:47:47.187 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (6989 ms) 2024-02-23T00:47:47.187 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (138611 ms total) 2024-02-23T00:47:47.187 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: 2024-02-23T00:47:47.187 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [----------] Global test environment tear-down 2024-02-23T00:47:47.187 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [==========] 56 tests from 4 test suites ran. (299964 ms total) 2024-02-23T00:47:47.187 INFO:tasks.workunit.client.0.smithi047.stdout: api_aio_pp: [ PASSED ] 56 tests. 2024-02-23T00:47:47.190 INFO:tasks.workunit.client.0.smithi047.stderr:+ for t in "${!pids[@]}" 2024-02-23T00:47:47.190 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82397 2024-02-23T00:47:47.190 INFO:tasks.workunit.client.0.smithi047.stderr:+ wait 82397 2024-02-23T00:47:47.190 INFO:tasks.workunit.client.0.smithi047.stderr:+ for t in "${!pids[@]}" 2024-02-23T00:47:47.190 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82470 2024-02-23T00:47:47.190 INFO:tasks.workunit.client.0.smithi047.stderr:+ wait 82470 2024-02-23T00:47:47.190 INFO:tasks.workunit.client.0.smithi047.stderr:+ for t in "${!pids[@]}" 2024-02-23T00:47:47.190 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82451 2024-02-23T00:47:47.190 INFO:tasks.workunit.client.0.smithi047.stderr:+ wait 82451 2024-02-23T00:47:47.190 INFO:tasks.workunit.client.0.smithi047.stderr:+ for t in "${!pids[@]}" 2024-02-23T00:47:47.190 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82423 2024-02-23T00:47:47.190 INFO:tasks.workunit.client.0.smithi047.stderr:+ wait 82423 2024-02-23T00:47:47.190 INFO:tasks.workunit.client.0.smithi047.stderr:+ for t in "${!pids[@]}" 2024-02-23T00:47:47.190 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82531 2024-02-23T00:47:47.191 INFO:tasks.workunit.client.0.smithi047.stderr:+ wait 82531 2024-02-23T00:47:47.191 INFO:tasks.workunit.client.0.smithi047.stderr:+ for t in "${!pids[@]}" 2024-02-23T00:47:47.191 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82389 2024-02-23T00:47:47.191 INFO:tasks.workunit.client.0.smithi047.stderr:+ wait 82389 2024-02-23T00:47:47.191 INFO:tasks.workunit.client.0.smithi047.stderr:+ for t in "${!pids[@]}" 2024-02-23T00:47:47.191 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82547 2024-02-23T00:47:47.191 INFO:tasks.workunit.client.0.smithi047.stderr:+ wait 82547 2024-02-23T00:47:47.191 INFO:tasks.workunit.client.0.smithi047.stderr:+ for t in "${!pids[@]}" 2024-02-23T00:47:47.191 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82499 2024-02-23T00:47:47.191 INFO:tasks.workunit.client.0.smithi047.stderr:+ wait 82499 2024-02-23T00:47:47.191 INFO:tasks.workunit.client.0.smithi047.stderr:+ for t in "${!pids[@]}" 2024-02-23T00:47:47.191 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82579 2024-02-23T00:47:47.191 INFO:tasks.workunit.client.0.smithi047.stderr:+ wait 82579 2024-02-23T00:47:47.191 INFO:tasks.workunit.client.0.smithi047.stderr:+ for t in "${!pids[@]}" 2024-02-23T00:47:47.191 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82539 2024-02-23T00:47:47.191 INFO:tasks.workunit.client.0.smithi047.stderr:+ wait 82539 2024-02-23T00:47:47.192 INFO:tasks.workunit.client.0.smithi047.stderr:+ for t in "${!pids[@]}" 2024-02-23T00:47:47.192 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82595 2024-02-23T00:47:47.192 INFO:tasks.workunit.client.0.smithi047.stderr:+ wait 82595 2024-02-23T00:47:47.193 INFO:tasks.workunit.client.0.smithi047.stderr:+ for t in "${!pids[@]}" 2024-02-23T00:47:47.193 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82373 2024-02-23T00:47:47.193 INFO:tasks.workunit.client.0.smithi047.stderr:+ wait 82373 2024-02-23T00:47:47.193 INFO:tasks.workunit.client.0.smithi047.stderr:+ for t in "${!pids[@]}" 2024-02-23T00:47:47.193 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82587 2024-02-23T00:47:47.193 INFO:tasks.workunit.client.0.smithi047.stderr:+ wait 82587 2024-02-23T00:47:47.193 INFO:tasks.workunit.client.0.smithi047.stderr:+ for t in "${!pids[@]}" 2024-02-23T00:47:47.193 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82555 2024-02-23T00:47:47.193 INFO:tasks.workunit.client.0.smithi047.stderr:+ wait 82555 2024-02-23T00:47:47.193 INFO:tasks.workunit.client.0.smithi047.stderr:+ for t in "${!pids[@]}" 2024-02-23T00:47:47.194 INFO:tasks.workunit.client.0.smithi047.stderr:+ pid=82460 2024-02-23T00:47:47.194 INFO:tasks.workunit.client.0.smithi047.stderr:+ wait 82460 2024-02-23T00:47:47.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.192+0000 7ff35b603700 1 -- 172.21.15.47:0/807057678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3540fcb40 msgr2=0x7ff3540fcf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:47.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.192+0000 7ff35b603700 1 --2- 172.21.15.47:0/807057678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3540fcb40 0x7ff3540fcf20 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 rx=0x7ff350004830 tx=0x7ff350009cb0).stop 2024-02-23T00:47:47.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.192+0000 7ff35b603700 1 -- 172.21.15.47:0/807057678 shutdown_connections 2024-02-23T00:47:47.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.192+0000 7ff35b603700 1 --2- 172.21.15.47:0/807057678 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff3540f7ba0 0x7ff354103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:47:47.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.192+0000 7ff35b603700 1 --2- 172.21.15.47:0/807057678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3540fcb40 0x7ff3540fcf20 unknown :-1 s=CLOSED pgs=2525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:47.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.192+0000 7ff35b603700 1 --2- 172.21.15.47:0/807057678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3540fd4a0 0x7ff3540f74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:47.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.192+0000 7ff35b603700 1 -- 172.21.15.47:0/807057678 >> 172.21.15.47:0/807057678 conn(0x7ff35406c370 msgr2=0x7ff35406c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:47.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.192+0000 7ff35b603700 1 -- 172.21.15.47:0/807057678 shutdown_connections 2024-02-23T00:47:47.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.192+0000 7ff35b603700 1 -- 172.21.15.47:0/807057678 wait complete. 2024-02-23T00:47:47.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.193+0000 7ff35b603700 1 librados: starting msgr at 2024-02-23T00:47:47.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.193+0000 7ff35b603700 1 librados: starting objecter 2024-02-23T00:47:47.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.193+0000 7ff35b603700 1 Processor -- start 2024-02-23T00:47:47.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.193+0000 7ff35b603700 1 -- start start 2024-02-23T00:47:47.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.193+0000 7ff35b603700 1 librados: setting wanted keys 2024-02-23T00:47:47.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.193+0000 7ff35b603700 1 librados: calling monclient init 2024-02-23T00:47:47.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.193+0000 7ff35b603700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3540f7ba0 0x7ff35410bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:47.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.193+0000 7ff35b603700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff3540fcb40 0x7ff35410c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:47.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.193+0000 7ff35b603700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3540fd4a0 0x7ff3541060f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:47.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.193+0000 7ff35b603700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3541065e0 con 0x7ff3540f7ba0 2024-02-23T00:47:47.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.193+0000 7ff35b603700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff354106750 con 0x7ff3540fd4a0 2024-02-23T00:47:47.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.193+0000 7ff35b603700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff354106910 con 0x7ff3540fcb40 2024-02-23T00:47:47.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.193+0000 7ff358b9e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff3540fcb40 0x7ff35410c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:47.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.193+0000 7ff358b9e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff3540fcb40 0x7ff35410c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:42724/0 (socket says 172.21.15.47:42724) 2024-02-23T00:47:47.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.193+0000 7ff358b9e700 1 -- 172.21.15.47:0/2130939846 learned_addr learned my addr 172.21.15.47:0/2130939846 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:47:47.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.193+0000 7ff35939f700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3540f7ba0 0x7ff35410bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:47.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.194+0000 7ff359ba0700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3540fd4a0 0x7ff3541060f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:47.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.194+0000 7ff35939f700 1 -- 172.21.15.47:0/2130939846 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff3540fcb40 msgr2=0x7ff35410c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:47.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.194+0000 7ff35939f700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff3540fcb40 0x7ff35410c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:47.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.194+0000 7ff35939f700 1 -- 172.21.15.47:0/2130939846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3540fd4a0 msgr2=0x7ff3541060f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:47.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.194+0000 7ff35939f700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3540fd4a0 0x7ff3541060f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:47.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.194+0000 7ff35939f700 1 -- 172.21.15.47:0/2130939846 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff350005040 con 0x7ff3540f7ba0 2024-02-23T00:47:47.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.194+0000 7ff358b9e700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff3540fcb40 0x7ff35410c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:47:47.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.194+0000 7ff359ba0700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3540fd4a0 0x7ff3541060f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:47:47.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.194+0000 7ff35939f700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3540f7ba0 0x7ff35410bf50 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 rx=0x7ff350005010 tx=0x7ff350009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:47.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.194+0000 7ff34a7fc700 10 client.?.objecter ms_handle_connect 0x7ff3540f7ba0 2024-02-23T00:47:47.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.194+0000 7ff34a7fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:47:47.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.194+0000 7ff34a7fc700 1 -- 172.21.15.47:0/2130939846 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35000f240 con 0x7ff3540f7ba0 2024-02-23T00:47:47.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.194+0000 7ff34a7fc700 1 -- 172.21.15.47:0/2130939846 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7ff350009630 con 0x7ff3540f7ba0 2024-02-23T00:47:47.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.194+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff354106b90 con 0x7ff3540f7ba0 2024-02-23T00:47:47.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.194+0000 7ff34a7fc700 1 -- 172.21.15.47:0/2130939846 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35000eca0 con 0x7ff3540f7ba0 2024-02-23T00:47:47.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.194+0000 7ff35b603700 10 client.30969.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:47:47.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.194+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff354112d80 con 0x7ff3540f7ba0 2024-02-23T00:47:47.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.194+0000 7ff35b603700 1 librados: init done 2024-02-23T00:47:47.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.196+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7ff35405fa60 con 0x7ff3540f7ba0 2024-02-23T00:47:47.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.197+0000 7ff34a7fc700 1 -- 172.21.15.47:0/2130939846 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7ff35000e450 con 0x7ff3540f7ba0 2024-02-23T00:47:47.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.197+0000 7ff34a7fc700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff340062cc0 0x7ff340065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:47.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.197+0000 7ff34a7fc700 1 -- 172.21.15.47:0/2130939846 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(395..395 src has 48..395) v4 ==== 9329+0+0 (secure 0 0 0) 0x7ff3500836a0 con 0x7ff3540f7ba0 2024-02-23T00:47:47.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.197+0000 7ff34a7fc700 10 client.30969.objecter ms_dispatch 0x7ff354001550 osd_map(395..395 src has 48..395) v4 2024-02-23T00:47:47.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.197+0000 7ff34a7fc700 3 client.30969.objecter handle_osd_map got epochs [395,395] > 0 2024-02-23T00:47:47.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.197+0000 7ff34a7fc700 3 client.30969.objecter handle_osd_map decoding full epoch 395 2024-02-23T00:47:47.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.197+0000 7ff358b9e700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff340062cc0 0x7ff340065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:47.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.197+0000 7ff34a7fc700 20 client.30969.objecter dump_active .. 0 homeless 2024-02-23T00:47:47.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.197+0000 7ff34a7fc700 1 -- 172.21.15.47:0/2130939846 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v395) v1 ==== 54+0+16 (secure 0 0 0) 0x7ff350083b80 con 0x7ff3540f7ba0 2024-02-23T00:47:47.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.198+0000 7ff34a7fc700 10 client.30969.objecter ms_handle_connect 0x7ff340062cc0 2024-02-23T00:47:47.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.198+0000 7ff358b9e700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff340062cc0 0x7ff340065180 secure :-1 s=READY pgs=3057 cs=0 l=1 rev1=1 rx=0x7ff344004fd0 tx=0x7ff34400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:47.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.198+0000 7ff35b603700 10 client.30969.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:47.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.198+0000 7ff35b603700 15 client.30969.objecter _session_command_op_assign -1 1 2024-02-23T00:47:47.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.198+0000 7ff35b603700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7ff354028540 0x7ff3540fd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:47.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.198+0000 7ff35b603700 20 client.30969.objecter _get_session s=0x7ff35405ee80 osd=0 3 2024-02-23T00:47:47.338 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:47:47.338 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:47:47.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.198+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff35405ee80 osd=0 3 2024-02-23T00:47:47.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.198+0000 7ff35b603700 20 client.30969.objecter _get_session s=0x7ff35405ee80 osd=0 3 2024-02-23T00:47:47.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.198+0000 7ff35b603700 15 client.30969.objecter _session_command_op_remove -1 1 2024-02-23T00:47:47.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.198+0000 7ff35b603700 20 client.30969.objecter get_session s=0x7ff35405ee80 osd=0 3 2024-02-23T00:47:47.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.198+0000 7ff35b603700 15 client.30969.objecter _session_command_op_assign 0 1 2024-02-23T00:47:47.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.198+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff35405ee80 osd=0 4 2024-02-23T00:47:47.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.198+0000 7ff35b603700 10 client.30969.objecter _send_command 1 2024-02-23T00:47:47.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.198+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff354104d90 con 0x7ff354028540 2024-02-23T00:47:47.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.198+0000 7ff359ba0700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7ff354028540 0x7ff3540fd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:47.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.198+0000 7ff34a7fc700 10 client.30969.objecter ms_handle_connect 0x7ff354028540 2024-02-23T00:47:47.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.198+0000 7ff359ba0700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7ff354028540 0x7ff3540fd8e0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:47.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.199+0000 7ff34a7fc700 1 -- 172.21.15.47:0/2130939846 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff354104d90 con 0x7ff354028540 2024-02-23T00:47:47.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.199+0000 7ff34a7fc700 10 client.30969.objecter ms_dispatch 0x7ff354001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:47:47.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.199+0000 7ff34a7fc700 10 client.30969.objecter _finish_command 1 = system:0 2024-02-23T00:47:47.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.199+0000 7ff34a7fc700 20 client.30969.objecter put_session s=0x7ff35405ee80 osd=0 4 2024-02-23T00:47:47.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.199+0000 7ff34a7fc700 15 client.30969.objecter _session_command_op_remove 0 1 2024-02-23T00:47:47.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.216+0000 7ff35b603700 10 client.30969.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:47:47.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.216+0000 7ff35b603700 15 client.30969.objecter _session_command_op_assign -1 2 2024-02-23T00:47:47.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.216+0000 7ff35b603700 20 client.30969.objecter _get_session s=0x7ff35405ee80 osd=0 3 2024-02-23T00:47:47.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.216+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff35405ee80 osd=0 3 2024-02-23T00:47:47.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.216+0000 7ff35b603700 20 client.30969.objecter _get_session s=0x7ff35405ee80 osd=0 3 2024-02-23T00:47:47.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.216+0000 7ff35b603700 15 client.30969.objecter _session_command_op_remove -1 2 2024-02-23T00:47:47.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.216+0000 7ff35b603700 20 client.30969.objecter get_session s=0x7ff35405ee80 osd=0 3 2024-02-23T00:47:47.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.216+0000 7ff35b603700 15 client.30969.objecter _session_command_op_assign 0 2 2024-02-23T00:47:47.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.216+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff35405ee80 osd=0 4 2024-02-23T00:47:47.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.216+0000 7ff35b603700 10 client.30969.objecter _send_command 2 2024-02-23T00:47:47.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.216+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ff35405a040 con 0x7ff354028540 2024-02-23T00:47:47.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.218+0000 7ff34a7fc700 1 -- 172.21.15.47:0/2130939846 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ff35405a040 con 0x7ff354028540 2024-02-23T00:47:47.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.218+0000 7ff34a7fc700 10 client.30969.objecter ms_dispatch 0x7ff354001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:47:47.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.218+0000 7ff34a7fc700 10 client.30969.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:47:47.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.218+0000 7ff34a7fc700 20 client.30969.objecter put_session s=0x7ff35405ee80 osd=0 4 2024-02-23T00:47:47.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.218+0000 7ff34a7fc700 15 client.30969.objecter _session_command_op_remove 0 2 2024-02-23T00:47:47.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.218+0000 7ff35b603700 10 client.30969.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:47.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.218+0000 7ff35b603700 15 client.30969.objecter _session_command_op_assign -1 3 2024-02-23T00:47:47.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.218+0000 7ff35b603700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7ff354027b90 0x7ff354027f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:47.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.218+0000 7ff35b603700 20 client.30969.objecter _get_session s=0x7ff3540fef50 osd=1 3 2024-02-23T00:47:47.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.218+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff3540fef50 osd=1 3 2024-02-23T00:47:47.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.218+0000 7ff35b603700 20 client.30969.objecter _get_session s=0x7ff3540fef50 osd=1 3 2024-02-23T00:47:47.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.218+0000 7ff35b603700 15 client.30969.objecter _session_command_op_remove -1 3 2024-02-23T00:47:47.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.218+0000 7ff35b603700 20 client.30969.objecter get_session s=0x7ff3540fef50 osd=1 3 2024-02-23T00:47:47.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.218+0000 7ff35b603700 15 client.30969.objecter _session_command_op_assign 1 3 2024-02-23T00:47:47.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.218+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff3540fef50 osd=1 4 2024-02-23T00:47:47.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.218+0000 7ff35b603700 10 client.30969.objecter _send_command 3 2024-02-23T00:47:47.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.218+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff3540ff0c0 con 0x7ff354027b90 2024-02-23T00:47:47.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.218+0000 7ff35939f700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7ff354027b90 0x7ff354027f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:47.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.219+0000 7ff35939f700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7ff354027b90 0x7ff354027f70 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:47.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.219+0000 7ff34a7fc700 10 client.30969.objecter ms_handle_connect 0x7ff354027b90 2024-02-23T00:47:47.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.220+0000 7ff34a7fc700 1 -- 172.21.15.47:0/2130939846 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff3540ff0c0 con 0x7ff354027b90 2024-02-23T00:47:47.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.220+0000 7ff34a7fc700 10 client.30969.objecter ms_dispatch 0x7ff354001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:47:47.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.220+0000 7ff34a7fc700 10 client.30969.objecter _finish_command 3 = system:0 2024-02-23T00:47:47.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.220+0000 7ff34a7fc700 20 client.30969.objecter put_session s=0x7ff3540fef50 osd=1 4 2024-02-23T00:47:47.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.220+0000 7ff34a7fc700 15 client.30969.objecter _session_command_op_remove 1 3 2024-02-23T00:47:47.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.235+0000 7ff35b603700 10 client.30969.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:47:47.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.235+0000 7ff35b603700 15 client.30969.objecter _session_command_op_assign -1 4 2024-02-23T00:47:47.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.235+0000 7ff35b603700 20 client.30969.objecter _get_session s=0x7ff3540fef50 osd=1 3 2024-02-23T00:47:47.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.235+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff3540fef50 osd=1 3 2024-02-23T00:47:47.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.235+0000 7ff35b603700 20 client.30969.objecter _get_session s=0x7ff3540fef50 osd=1 3 2024-02-23T00:47:47.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.235+0000 7ff35b603700 15 client.30969.objecter _session_command_op_remove -1 4 2024-02-23T00:47:47.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.235+0000 7ff35b603700 20 client.30969.objecter get_session s=0x7ff3540fef50 osd=1 3 2024-02-23T00:47:47.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.235+0000 7ff35b603700 15 client.30969.objecter _session_command_op_assign 1 4 2024-02-23T00:47:47.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.235+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff3540fef50 osd=1 4 2024-02-23T00:47:47.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.235+0000 7ff35b603700 10 client.30969.objecter _send_command 4 2024-02-23T00:47:47.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.235+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ff354027600 con 0x7ff354027b90 2024-02-23T00:47:47.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.237+0000 7ff34a7fc700 1 -- 172.21.15.47:0/2130939846 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ff354027600 con 0x7ff354027b90 2024-02-23T00:47:47.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.237+0000 7ff34a7fc700 10 client.30969.objecter ms_dispatch 0x7ff354001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:47:47.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.237+0000 7ff34a7fc700 10 client.30969.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:47:47.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.237+0000 7ff34a7fc700 20 client.30969.objecter put_session s=0x7ff3540fef50 osd=1 4 2024-02-23T00:47:47.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.237+0000 7ff34a7fc700 15 client.30969.objecter _session_command_op_remove 1 4 2024-02-23T00:47:47.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.237+0000 7ff35b603700 10 client.30969.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:47.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.237+0000 7ff35b603700 15 client.30969.objecter _session_command_op_assign -1 5 2024-02-23T00:47:47.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.237+0000 7ff35b603700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7ff3540feab0 0x7ff35411a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:47.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.237+0000 7ff35b603700 20 client.30969.objecter _get_session s=0x7ff354027a10 osd=2 3 2024-02-23T00:47:47.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.237+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff354027a10 osd=2 3 2024-02-23T00:47:47.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.237+0000 7ff35b603700 20 client.30969.objecter _get_session s=0x7ff354027a10 osd=2 3 2024-02-23T00:47:47.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.237+0000 7ff35b603700 15 client.30969.objecter _session_command_op_remove -1 5 2024-02-23T00:47:47.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.237+0000 7ff35b603700 20 client.30969.objecter get_session s=0x7ff354027a10 osd=2 3 2024-02-23T00:47:47.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.237+0000 7ff35b603700 15 client.30969.objecter _session_command_op_assign 2 5 2024-02-23T00:47:47.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.237+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff354027a10 osd=2 4 2024-02-23T00:47:47.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.237+0000 7ff35b603700 10 client.30969.objecter _send_command 5 2024-02-23T00:47:47.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.237+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff35411cd10 con 0x7ff3540feab0 2024-02-23T00:47:47.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.237+0000 7ff358b9e700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7ff3540feab0 0x7ff35411a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:47.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.237+0000 7ff34a7fc700 10 client.30969.objecter ms_handle_connect 0x7ff3540feab0 2024-02-23T00:47:47.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.237+0000 7ff358b9e700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7ff3540feab0 0x7ff35411a0c0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:47.345 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:47:47.345 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:47:47.345 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:47:47.345 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:47:47.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.238+0000 7ff34a7fc700 1 -- 172.21.15.47:0/2130939846 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff35411cd10 con 0x7ff3540feab0 2024-02-23T00:47:47.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.238+0000 7ff34a7fc700 10 client.30969.objecter ms_dispatch 0x7ff354001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:47:47.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.238+0000 7ff34a7fc700 10 client.30969.objecter _finish_command 5 = system:0 2024-02-23T00:47:47.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.238+0000 7ff34a7fc700 20 client.30969.objecter put_session s=0x7ff354027a10 osd=2 4 2024-02-23T00:47:47.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.238+0000 7ff34a7fc700 15 client.30969.objecter _session_command_op_remove 2 5 2024-02-23T00:47:47.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.252+0000 7ff35b603700 10 client.30969.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:47:47.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.252+0000 7ff35b603700 15 client.30969.objecter _session_command_op_assign -1 6 2024-02-23T00:47:47.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.252+0000 7ff35b603700 20 client.30969.objecter _get_session s=0x7ff354027a10 osd=2 3 2024-02-23T00:47:47.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.252+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff354027a10 osd=2 3 2024-02-23T00:47:47.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.252+0000 7ff35b603700 20 client.30969.objecter _get_session s=0x7ff354027a10 osd=2 3 2024-02-23T00:47:47.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.252+0000 7ff35b603700 15 client.30969.objecter _session_command_op_remove -1 6 2024-02-23T00:47:47.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.252+0000 7ff35b603700 20 client.30969.objecter get_session s=0x7ff354027a10 osd=2 3 2024-02-23T00:47:47.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.252+0000 7ff35b603700 15 client.30969.objecter _session_command_op_assign 2 6 2024-02-23T00:47:47.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.252+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff354027a10 osd=2 4 2024-02-23T00:47:47.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.252+0000 7ff35b603700 10 client.30969.objecter _send_command 6 2024-02-23T00:47:47.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.252+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ff3540fe010 con 0x7ff3540feab0 2024-02-23T00:47:47.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.254+0000 7ff34a7fc700 1 -- 172.21.15.47:0/2130939846 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ff3540fe010 con 0x7ff3540feab0 2024-02-23T00:47:47.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.254+0000 7ff34a7fc700 10 client.30969.objecter ms_dispatch 0x7ff354001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:47:47.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.254+0000 7ff34a7fc700 10 client.30969.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:47:47.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.254+0000 7ff34a7fc700 20 client.30969.objecter put_session s=0x7ff354027a10 osd=2 4 2024-02-23T00:47:47.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.254+0000 7ff34a7fc700 15 client.30969.objecter _session_command_op_remove 2 6 2024-02-23T00:47:47.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.254+0000 7ff35b603700 10 client.30969.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:47.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.254+0000 7ff35b603700 15 client.30969.objecter _session_command_op_assign -1 7 2024-02-23T00:47:47.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.254+0000 7ff35b603700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7ff354117c10 0x7ff35411b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:47.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.254+0000 7ff35b603700 20 client.30969.objecter _get_session s=0x7ff3540fe3f0 osd=3 3 2024-02-23T00:47:47.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.254+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff3540fe3f0 osd=3 3 2024-02-23T00:47:47.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.254+0000 7ff35b603700 20 client.30969.objecter _get_session s=0x7ff3540fe3f0 osd=3 3 2024-02-23T00:47:47.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.254+0000 7ff35b603700 15 client.30969.objecter _session_command_op_remove -1 7 2024-02-23T00:47:47.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.254+0000 7ff35b603700 20 client.30969.objecter get_session s=0x7ff3540fe3f0 osd=3 3 2024-02-23T00:47:47.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.254+0000 7ff35b603700 15 client.30969.objecter _session_command_op_assign 3 7 2024-02-23T00:47:47.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.254+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff3540fe3f0 osd=3 4 2024-02-23T00:47:47.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.254+0000 7ff35b603700 10 client.30969.objecter _send_command 7 2024-02-23T00:47:47.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.254+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff35411bce0 con 0x7ff354117c10 2024-02-23T00:47:47.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.254+0000 7ff359ba0700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7ff354117c10 0x7ff35411b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:47.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.254+0000 7ff34a7fc700 10 client.30969.objecter ms_handle_connect 0x7ff354117c10 2024-02-23T00:47:47.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.254+0000 7ff359ba0700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7ff354117c10 0x7ff35411b730 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:47.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.255+0000 7ff34a7fc700 1 -- 172.21.15.47:0/2130939846 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff35411bce0 con 0x7ff354117c10 2024-02-23T00:47:47.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.255+0000 7ff34a7fc700 10 client.30969.objecter ms_dispatch 0x7ff354001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:47:47.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.255+0000 7ff34a7fc700 10 client.30969.objecter _finish_command 7 = system:0 2024-02-23T00:47:47.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.255+0000 7ff34a7fc700 20 client.30969.objecter put_session s=0x7ff3540fe3f0 osd=3 4 2024-02-23T00:47:47.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.255+0000 7ff34a7fc700 15 client.30969.objecter _session_command_op_remove 3 7 2024-02-23T00:47:47.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.269+0000 7ff35b603700 10 client.30969.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:47:47.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.269+0000 7ff35b603700 15 client.30969.objecter _session_command_op_assign -1 8 2024-02-23T00:47:47.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.269+0000 7ff35b603700 20 client.30969.objecter _get_session s=0x7ff3540fe3f0 osd=3 3 2024-02-23T00:47:47.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.269+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff3540fe3f0 osd=3 3 2024-02-23T00:47:47.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.269+0000 7ff35b603700 20 client.30969.objecter _get_session s=0x7ff3540fe3f0 osd=3 3 2024-02-23T00:47:47.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.269+0000 7ff35b603700 15 client.30969.objecter _session_command_op_remove -1 8 2024-02-23T00:47:47.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.269+0000 7ff35b603700 20 client.30969.objecter get_session s=0x7ff3540fe3f0 osd=3 3 2024-02-23T00:47:47.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.269+0000 7ff35b603700 15 client.30969.objecter _session_command_op_assign 3 8 2024-02-23T00:47:47.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.269+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff3540fe3f0 osd=3 4 2024-02-23T00:47:47.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.269+0000 7ff35b603700 10 client.30969.objecter _send_command 8 2024-02-23T00:47:47.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.269+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ff354117170 con 0x7ff354117c10 2024-02-23T00:47:47.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.271+0000 7ff34a7fc700 1 -- 172.21.15.47:0/2130939846 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ff354117170 con 0x7ff354117c10 2024-02-23T00:47:47.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.271+0000 7ff34a7fc700 10 client.30969.objecter ms_dispatch 0x7ff354001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:47:47.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.271+0000 7ff34a7fc700 10 client.30969.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:47:47.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.271+0000 7ff34a7fc700 20 client.30969.objecter put_session s=0x7ff3540fe3f0 osd=3 4 2024-02-23T00:47:47.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.271+0000 7ff34a7fc700 15 client.30969.objecter _session_command_op_remove 3 8 2024-02-23T00:47:47.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.271+0000 7ff35b603700 10 client.30969.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:47.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.271+0000 7ff35b603700 15 client.30969.objecter _session_command_op_assign -1 9 2024-02-23T00:47:47.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.271+0000 7ff35b603700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7ff35411ee90 0x7ff354121280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:47.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.271+0000 7ff35b603700 20 client.30969.objecter _get_session s=0x7ff354117340 osd=4 3 2024-02-23T00:47:47.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.271+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff354117340 osd=4 3 2024-02-23T00:47:47.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.271+0000 7ff35b603700 20 client.30969.objecter _get_session s=0x7ff354117340 osd=4 3 2024-02-23T00:47:47.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.271+0000 7ff35b603700 15 client.30969.objecter _session_command_op_remove -1 9 2024-02-23T00:47:47.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.271+0000 7ff35b603700 20 client.30969.objecter get_session s=0x7ff354117340 osd=4 3 2024-02-23T00:47:47.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.271+0000 7ff35b603700 15 client.30969.objecter _session_command_op_assign 4 9 2024-02-23T00:47:47.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.271+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff354117340 osd=4 4 2024-02-23T00:47:47.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.271+0000 7ff35b603700 10 client.30969.objecter _send_command 9 2024-02-23T00:47:47.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.271+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff3541174b0 con 0x7ff35411ee90 2024-02-23T00:47:47.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.271+0000 7ff35939f700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7ff35411ee90 0x7ff354121280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:47.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.272+0000 7ff34a7fc700 10 client.30969.objecter ms_handle_connect 0x7ff35411ee90 2024-02-23T00:47:47.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.272+0000 7ff35939f700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7ff35411ee90 0x7ff354121280 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:47.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.272+0000 7ff34a7fc700 1 -- 172.21.15.47:0/2130939846 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff3541174b0 con 0x7ff35411ee90 2024-02-23T00:47:47.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.272+0000 7ff34a7fc700 10 client.30969.objecter ms_dispatch 0x7ff354001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:47:47.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.272+0000 7ff34a7fc700 10 client.30969.objecter _finish_command 9 = system:0 2024-02-23T00:47:47.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.272+0000 7ff34a7fc700 20 client.30969.objecter put_session s=0x7ff354117340 osd=4 4 2024-02-23T00:47:47.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.272+0000 7ff34a7fc700 15 client.30969.objecter _session_command_op_remove 4 9 2024-02-23T00:47:47.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.287+0000 7ff35b603700 10 client.30969.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:47:47.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.287+0000 7ff35b603700 15 client.30969.objecter _session_command_op_assign -1 10 2024-02-23T00:47:47.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.287+0000 7ff35b603700 20 client.30969.objecter _get_session s=0x7ff354117340 osd=4 3 2024-02-23T00:47:47.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.287+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff354117340 osd=4 3 2024-02-23T00:47:47.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.287+0000 7ff35b603700 20 client.30969.objecter _get_session s=0x7ff354117340 osd=4 3 2024-02-23T00:47:47.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.287+0000 7ff35b603700 15 client.30969.objecter _session_command_op_remove -1 10 2024-02-23T00:47:47.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.287+0000 7ff35b603700 20 client.30969.objecter get_session s=0x7ff354117340 osd=4 3 2024-02-23T00:47:47.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.287+0000 7ff35b603700 15 client.30969.objecter _session_command_op_assign 4 10 2024-02-23T00:47:47.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.287+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff354117340 osd=4 4 2024-02-23T00:47:47.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.287+0000 7ff35b603700 10 client.30969.objecter _send_command 10 2024-02-23T00:47:47.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.287+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ff3541174b0 con 0x7ff35411ee90 2024-02-23T00:47:47.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.289+0000 7ff34a7fc700 1 -- 172.21.15.47:0/2130939846 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ff3541174b0 con 0x7ff35411ee90 2024-02-23T00:47:47.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.289+0000 7ff34a7fc700 10 client.30969.objecter ms_dispatch 0x7ff354001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:47:47.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.289+0000 7ff34a7fc700 10 client.30969.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:47:47.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.289+0000 7ff34a7fc700 20 client.30969.objecter put_session s=0x7ff354117340 osd=4 4 2024-02-23T00:47:47.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.289+0000 7ff34a7fc700 15 client.30969.objecter _session_command_op_remove 4 10 2024-02-23T00:47:47.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.290+0000 7ff35b603700 10 client.30969.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:47.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.290+0000 7ff35b603700 15 client.30969.objecter _session_command_op_assign -1 11 2024-02-23T00:47:47.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.290+0000 7ff35b603700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7ff354121c80 0x7ff3541240d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:47.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.290+0000 7ff35b603700 20 client.30969.objecter _get_session s=0x7ff3541174b0 osd=5 3 2024-02-23T00:47:47.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.290+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff3541174b0 osd=5 3 2024-02-23T00:47:47.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.290+0000 7ff35b603700 20 client.30969.objecter _get_session s=0x7ff3541174b0 osd=5 3 2024-02-23T00:47:47.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.290+0000 7ff35b603700 15 client.30969.objecter _session_command_op_remove -1 11 2024-02-23T00:47:47.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.290+0000 7ff35b603700 20 client.30969.objecter get_session s=0x7ff3541174b0 osd=5 3 2024-02-23T00:47:47.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.290+0000 7ff35b603700 15 client.30969.objecter _session_command_op_assign 5 11 2024-02-23T00:47:47.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.290+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff3541174b0 osd=5 4 2024-02-23T00:47:47.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.290+0000 7ff35b603700 10 client.30969.objecter _send_command 11 2024-02-23T00:47:47.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.290+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff354126f10 con 0x7ff354121c80 2024-02-23T00:47:47.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.290+0000 7ff358b9e700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7ff354121c80 0x7ff3541240d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:47.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.290+0000 7ff34a7fc700 10 client.30969.objecter ms_handle_connect 0x7ff354121c80 2024-02-23T00:47:47.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.290+0000 7ff358b9e700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7ff354121c80 0x7ff3541240d0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:47.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.291+0000 7ff34a7fc700 1 -- 172.21.15.47:0/2130939846 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff354126f10 con 0x7ff354121c80 2024-02-23T00:47:47.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.291+0000 7ff34a7fc700 10 client.30969.objecter ms_dispatch 0x7ff354001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:47:47.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.291+0000 7ff34a7fc700 10 client.30969.objecter _finish_command 11 = system:0 2024-02-23T00:47:47.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.291+0000 7ff34a7fc700 20 client.30969.objecter put_session s=0x7ff3541174b0 osd=5 4 2024-02-23T00:47:47.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.291+0000 7ff34a7fc700 15 client.30969.objecter _session_command_op_remove 5 11 2024-02-23T00:47:47.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.305+0000 7ff35b603700 10 client.30969.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:47:47.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.305+0000 7ff35b603700 15 client.30969.objecter _session_command_op_assign -1 12 2024-02-23T00:47:47.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.305+0000 7ff35b603700 20 client.30969.objecter _get_session s=0x7ff3541174b0 osd=5 3 2024-02-23T00:47:47.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.305+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff3541174b0 osd=5 3 2024-02-23T00:47:47.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.305+0000 7ff35b603700 20 client.30969.objecter _get_session s=0x7ff3541174b0 osd=5 3 2024-02-23T00:47:47.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.305+0000 7ff35b603700 15 client.30969.objecter _session_command_op_remove -1 12 2024-02-23T00:47:47.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.305+0000 7ff35b603700 20 client.30969.objecter get_session s=0x7ff3541174b0 osd=5 3 2024-02-23T00:47:47.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.305+0000 7ff35b603700 15 client.30969.objecter _session_command_op_assign 5 12 2024-02-23T00:47:47.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.305+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff3541174b0 osd=5 4 2024-02-23T00:47:47.354 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:47:47.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.305+0000 7ff35b603700 10 client.30969.objecter _send_command 12 2024-02-23T00:47:47.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.305+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ff354126f10 con 0x7ff354121c80 2024-02-23T00:47:47.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.307+0000 7ff34a7fc700 1 -- 172.21.15.47:0/2130939846 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ff354126f10 con 0x7ff354121c80 2024-02-23T00:47:47.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.307+0000 7ff34a7fc700 10 client.30969.objecter ms_dispatch 0x7ff354001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:47:47.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.307+0000 7ff34a7fc700 10 client.30969.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:47:47.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.307+0000 7ff34a7fc700 20 client.30969.objecter put_session s=0x7ff3541174b0 osd=5 4 2024-02-23T00:47:47.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.307+0000 7ff34a7fc700 15 client.30969.objecter _session_command_op_remove 5 12 2024-02-23T00:47:47.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.307+0000 7ff35b603700 10 client.30969.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:47.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.307+0000 7ff35b603700 15 client.30969.objecter _session_command_op_assign -1 13 2024-02-23T00:47:47.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.307+0000 7ff35b603700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7ff354126240 0x7ff3541266a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:47.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.307+0000 7ff35b603700 20 client.30969.objecter _get_session s=0x7ff354126f10 osd=6 3 2024-02-23T00:47:47.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.307+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff354126f10 osd=6 3 2024-02-23T00:47:47.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.307+0000 7ff35b603700 20 client.30969.objecter _get_session s=0x7ff354126f10 osd=6 3 2024-02-23T00:47:47.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.307+0000 7ff35b603700 15 client.30969.objecter _session_command_op_remove -1 13 2024-02-23T00:47:47.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.307+0000 7ff35b603700 20 client.30969.objecter get_session s=0x7ff354126f10 osd=6 3 2024-02-23T00:47:47.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.308+0000 7ff35b603700 15 client.30969.objecter _session_command_op_assign 6 13 2024-02-23T00:47:47.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.308+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff354126f10 osd=6 4 2024-02-23T00:47:47.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.308+0000 7ff35b603700 10 client.30969.objecter _send_command 13 2024-02-23T00:47:47.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.308+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff35412a2c0 con 0x7ff354126240 2024-02-23T00:47:47.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.308+0000 7ff359ba0700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7ff354126240 0x7ff3541266a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:47.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.308+0000 7ff359ba0700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7ff354126240 0x7ff3541266a0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:47.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.308+0000 7ff34a7fc700 10 client.30969.objecter ms_handle_connect 0x7ff354126240 2024-02-23T00:47:47.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.309+0000 7ff34a7fc700 1 -- 172.21.15.47:0/2130939846 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff35412a2c0 con 0x7ff354126240 2024-02-23T00:47:47.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.309+0000 7ff34a7fc700 10 client.30969.objecter ms_dispatch 0x7ff354001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:47:47.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.309+0000 7ff34a7fc700 10 client.30969.objecter _finish_command 13 = system:0 2024-02-23T00:47:47.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.309+0000 7ff34a7fc700 20 client.30969.objecter put_session s=0x7ff354126f10 osd=6 4 2024-02-23T00:47:47.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.309+0000 7ff34a7fc700 15 client.30969.objecter _session_command_op_remove 6 13 2024-02-23T00:47:47.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.323+0000 7ff35b603700 10 client.30969.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:47:47.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.323+0000 7ff35b603700 15 client.30969.objecter _session_command_op_assign -1 14 2024-02-23T00:47:47.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.323+0000 7ff35b603700 20 client.30969.objecter _get_session s=0x7ff354126f10 osd=6 3 2024-02-23T00:47:47.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.323+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff354126f10 osd=6 3 2024-02-23T00:47:47.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.323+0000 7ff35b603700 20 client.30969.objecter _get_session s=0x7ff354126f10 osd=6 3 2024-02-23T00:47:47.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.323+0000 7ff35b603700 15 client.30969.objecter _session_command_op_remove -1 14 2024-02-23T00:47:47.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.323+0000 7ff35b603700 20 client.30969.objecter get_session s=0x7ff354126f10 osd=6 3 2024-02-23T00:47:47.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.323+0000 7ff35b603700 15 client.30969.objecter _session_command_op_assign 6 14 2024-02-23T00:47:47.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.323+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff354126f10 osd=6 4 2024-02-23T00:47:47.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.323+0000 7ff35b603700 10 client.30969.objecter _send_command 14 2024-02-23T00:47:47.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.323+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ff3541246d0 con 0x7ff354126240 2024-02-23T00:47:47.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.325+0000 7ff34a7fc700 1 -- 172.21.15.47:0/2130939846 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ff3541246d0 con 0x7ff354126240 2024-02-23T00:47:47.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.325+0000 7ff34a7fc700 10 client.30969.objecter ms_dispatch 0x7ff354001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:47:47.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.325+0000 7ff34a7fc700 10 client.30969.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:47:47.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.325+0000 7ff34a7fc700 20 client.30969.objecter put_session s=0x7ff354126f10 osd=6 4 2024-02-23T00:47:47.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.325+0000 7ff34a7fc700 15 client.30969.objecter _session_command_op_remove 6 14 2024-02-23T00:47:47.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.326+0000 7ff35b603700 10 client.30969.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:47.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.326+0000 7ff35b603700 15 client.30969.objecter _session_command_op_assign -1 15 2024-02-23T00:47:47.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.326+0000 7ff35b603700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7ff354124d50 0x7ff3541251b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:47.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.326+0000 7ff35b603700 20 client.30969.objecter _get_session s=0x7ff3541246d0 osd=7 3 2024-02-23T00:47:47.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.326+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff3541246d0 osd=7 3 2024-02-23T00:47:47.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.326+0000 7ff35b603700 20 client.30969.objecter _get_session s=0x7ff3541246d0 osd=7 3 2024-02-23T00:47:47.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.326+0000 7ff35b603700 15 client.30969.objecter _session_command_op_remove -1 15 2024-02-23T00:47:47.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.326+0000 7ff35b603700 20 client.30969.objecter get_session s=0x7ff3541246d0 osd=7 3 2024-02-23T00:47:47.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.326+0000 7ff35b603700 15 client.30969.objecter _session_command_op_assign 7 15 2024-02-23T00:47:47.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.326+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff3541246d0 osd=7 4 2024-02-23T00:47:47.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.326+0000 7ff35b603700 10 client.30969.objecter _send_command 15 2024-02-23T00:47:47.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.326+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff354125840 con 0x7ff354124d50 2024-02-23T00:47:47.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.326+0000 7ff35939f700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7ff354124d50 0x7ff3541251b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:47.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.326+0000 7ff34a7fc700 10 client.30969.objecter ms_handle_connect 0x7ff354124d50 2024-02-23T00:47:47.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.326+0000 7ff35939f700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7ff354124d50 0x7ff3541251b0 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:47.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.327+0000 7ff34a7fc700 1 -- 172.21.15.47:0/2130939846 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff354125840 con 0x7ff354124d50 2024-02-23T00:47:47.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.327+0000 7ff34a7fc700 10 client.30969.objecter ms_dispatch 0x7ff354001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:47:47.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.327+0000 7ff34a7fc700 10 client.30969.objecter _finish_command 15 = system:0 2024-02-23T00:47:47.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.327+0000 7ff34a7fc700 20 client.30969.objecter put_session s=0x7ff3541246d0 osd=7 4 2024-02-23T00:47:47.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.327+0000 7ff34a7fc700 15 client.30969.objecter _session_command_op_remove 7 15 2024-02-23T00:47:47.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.341+0000 7ff35b603700 10 client.30969.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:47:47.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.341+0000 7ff35b603700 15 client.30969.objecter _session_command_op_assign -1 16 2024-02-23T00:47:47.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.341+0000 7ff35b603700 20 client.30969.objecter _get_session s=0x7ff3541246d0 osd=7 3 2024-02-23T00:47:47.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.341+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff3541246d0 osd=7 3 2024-02-23T00:47:47.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.341+0000 7ff35b603700 20 client.30969.objecter _get_session s=0x7ff3541246d0 osd=7 3 2024-02-23T00:47:47.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.341+0000 7ff35b603700 15 client.30969.objecter _session_command_op_remove -1 16 2024-02-23T00:47:47.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.341+0000 7ff35b603700 20 client.30969.objecter get_session s=0x7ff3541246d0 osd=7 3 2024-02-23T00:47:47.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.341+0000 7ff35b603700 15 client.30969.objecter _session_command_op_assign 7 16 2024-02-23T00:47:47.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.341+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff3541246d0 osd=7 4 2024-02-23T00:47:47.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.341+0000 7ff35b603700 10 client.30969.objecter _send_command 16 2024-02-23T00:47:47.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.341+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ff354125840 con 0x7ff354124d50 2024-02-23T00:47:47.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.343+0000 7ff34a7fc700 1 -- 172.21.15.47:0/2130939846 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ff354125840 con 0x7ff354124d50 2024-02-23T00:47:47.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.343+0000 7ff34a7fc700 10 client.30969.objecter ms_dispatch 0x7ff354001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:47:47.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.343+0000 7ff34a7fc700 10 client.30969.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:47:47.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.343+0000 7ff34a7fc700 20 client.30969.objecter put_session s=0x7ff3541246d0 osd=7 4 2024-02-23T00:47:47.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.343+0000 7ff34a7fc700 15 client.30969.objecter _session_command_op_remove 7 16 2024-02-23T00:47:47.360 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:47:47.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 10 librados: watch_flush enter 2024-02-23T00:47:47.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 10 librados: watch_flush exit 2024-02-23T00:47:47.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 10 client.30969.objecter close_session for osd.0 2024-02-23T00:47:47.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7ff354028540 msgr2=0x7ff3540fd8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:47.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7ff354028540 0x7ff3540fd8e0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:47.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff35405ee80 osd=0 1 2024-02-23T00:47:47.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 10 client.30969.objecter close_session for osd.1 2024-02-23T00:47:47.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7ff354027b90 msgr2=0x7ff354027f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:47.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7ff354027b90 0x7ff354027f70 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:47.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff3540fef50 osd=1 1 2024-02-23T00:47:47.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 10 client.30969.objecter close_session for osd.2 2024-02-23T00:47:47.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7ff3540feab0 msgr2=0x7ff35411a0c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:47.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7ff3540feab0 0x7ff35411a0c0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:47.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff354027a10 osd=2 1 2024-02-23T00:47:47.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 10 client.30969.objecter close_session for osd.3 2024-02-23T00:47:47.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7ff354117c10 msgr2=0x7ff35411b730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:47.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff359ba0700 1 -- 172.21.15.47:0/2130939846 reap_dead start 2024-02-23T00:47:47.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7ff354117c10 0x7ff35411b730 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:47.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff3540fe3f0 osd=3 1 2024-02-23T00:47:47.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 10 client.30969.objecter close_session for osd.4 2024-02-23T00:47:47.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7ff35411ee90 msgr2=0x7ff354121280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:47.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7ff35411ee90 0x7ff354121280 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:47.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff354117340 osd=4 1 2024-02-23T00:47:47.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 10 client.30969.objecter close_session for osd.5 2024-02-23T00:47:47.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7ff354121c80 msgr2=0x7ff3541240d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:47.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7ff354121c80 0x7ff3541240d0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:47.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff3541174b0 osd=5 1 2024-02-23T00:47:47.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 10 client.30969.objecter close_session for osd.6 2024-02-23T00:47:47.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7ff354126240 msgr2=0x7ff3541266a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:47.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7ff354126240 0x7ff3541266a0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:47.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff354126f10 osd=6 1 2024-02-23T00:47:47.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 10 client.30969.objecter close_session for osd.7 2024-02-23T00:47:47.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7ff354124d50 msgr2=0x7ff3541251b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:47.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7ff354124d50 0x7ff3541251b0 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:47.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 20 client.30969.objecter put_session s=0x7ff3541246d0 osd=7 1 2024-02-23T00:47:47.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 20 client.30969.objecter shutdown clearing up homeless session... 2024-02-23T00:47:47.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 10 client.30969.objecter successfully canceled tick 2024-02-23T00:47:47.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff340062cc0 msgr2=0x7ff340065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:47.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff340062cc0 0x7ff340065180 secure :-1 s=READY pgs=3057 cs=0 l=1 rev1=1 rx=0x7ff344004fd0 tx=0x7ff34400b040).stop 2024-02-23T00:47:47.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3540f7ba0 msgr2=0x7ff35410bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:47.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 1 --2- 172.21.15.47:0/2130939846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3540f7ba0 0x7ff35410bf50 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 rx=0x7ff350005010 tx=0x7ff350009cb0).stop 2024-02-23T00:47:47.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff359ba0700 1 -- 172.21.15.47:0/2130939846 reap_dead start 2024-02-23T00:47:47.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff359ba0700 1 -- 172.21.15.47:0/2130939846 reap_dead start 2024-02-23T00:47:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 shutdown_connections 2024-02-23T00:47:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 >> 172.21.15.47:0/2130939846 conn(0x7ff35406c370 msgr2=0x7ff3540faf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 shutdown_connections 2024-02-23T00:47:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 1 -- 172.21.15.47:0/2130939846 wait complete. 2024-02-23T00:47:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:47.344+0000 7ff35b603700 1 librados: shutdown 2024-02-23T00:47:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:47:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:47:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:47:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:47:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:47:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:47:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:47:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:47:47.507 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:46.506109+0000 mon.a (mon.0) 2489 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:47:47.508 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:47.176688+0000 mon.a (mon.0) 2490 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi047-82386-65"}]': finished 2024-02-23T00:47:47.508 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:47.176774+0000 mon.a (mon.0) 2491 : cluster [DBG] osdmap e395: 8 total, 8 up, 7 in 2024-02-23T00:47:47.508 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:47.197857+0000 mon.a (mon.0) 2492 : audit [DBG] from='client.? 172.21.15.47:0/2130939846' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:47:47.508 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:46.506109+0000 mon.a (mon.0) 2489 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:47:47.508 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:47.176688+0000 mon.a (mon.0) 2490 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi047-82386-65"}]': finished 2024-02-23T00:47:47.508 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:47.176774+0000 mon.a (mon.0) 2491 : cluster [DBG] osdmap e395: 8 total, 8 up, 7 in 2024-02-23T00:47:47.508 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:47.197857+0000 mon.a (mon.0) 2492 : audit [DBG] from='client.? 172.21.15.47:0/2130939846' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:47:47.738 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset noscrub 2024-02-23T00:47:47.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:46.506109+0000 mon.a (mon.0) 2489 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:47:47.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:47.176688+0000 mon.a (mon.0) 2490 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi047-82386-65"}]': finished 2024-02-23T00:47:47.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:47.176774+0000 mon.a (mon.0) 2491 : cluster [DBG] osdmap e395: 8 total, 8 up, 7 in 2024-02-23T00:47:47.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:47.197857+0000 mon.a (mon.0) 2492 : audit [DBG] from='client.? 172.21.15.47:0/2130939846' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:47:48.170 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:47:48.171 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 20, 2024-02-23T00:47:48.171 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 600, 2024-02-23T00:47:48.171 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [ 2024-02-23T00:47:48.171 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.171 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(client.24775.0:603 20.1e e110/109 20:7c2d116d:test-rados-api-smithi047-82457-15::foo:head v 110'281, mlcod=110'279)", 2024-02-23T00:47:48.171 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:42:59.740311+0000", 2024-02-23T00:47:48.171 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 288.36791176200001, 2024-02-23T00:47:48.171 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.085153354000000001, 2024-02-23T00:47:48.171 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:47:48.171 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:47:48.171 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:47:48.171 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.172 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:47:48.172 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.740311+0000", 2024-02-23T00:47:48.172 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:47:48.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.172 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.172 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:47:48.172 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.740311+0000", 2024-02-23T00:47:48.172 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967296 2024-02-23T00:47:48.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.172 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.172 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:47:48.172 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.740311+0000", 2024-02-23T00:47:48.172 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3036e-05 2024-02-23T00:47:48.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.173 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.173 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:47:48.173 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.740324+0000", 2024-02-23T00:47:48.173 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.9549999999999997e-06 2024-02-23T00:47:48.173 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.173 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.173 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:47:48.173 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.740330+0000", 2024-02-23T00:47:48.173 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.2349999999999999e-06 2024-02-23T00:47:48.173 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.173 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.173 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:47:48.173 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.740335+0000", 2024-02-23T00:47:48.173 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.079788364000000001 2024-02-23T00:47:48.174 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.174 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.174 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:47:48.174 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.820123+0000", 2024-02-23T00:47:48.174 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0228e-05 2024-02-23T00:47:48.174 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.174 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.174 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:47:48.174 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.820143+0000", 2024-02-23T00:47:48.174 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0052875209999999999 2024-02-23T00:47:48.174 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.174 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.174 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:47:48.174 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.825431+0000", 2024-02-23T00:47:48.175 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.3232000000000003e-05 2024-02-23T00:47:48.175 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.175 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.175 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:47:48.175 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.825464+0000", 2024-02-23T00:47:48.175 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.085153354000000001 2024-02-23T00:47:48.175 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.175 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:47:48.175 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.175 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.175 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.175 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(client.24775.0:2069 20.13 e112/109 20:cde74feb:test-rados-api-smithi047-82457-17::big:head v 112'1, mlcod=0'0)", 2024-02-23T00:47:48.175 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:01.712327+0000", 2024-02-23T00:47:48.176 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 286.39589579199998, 2024-02-23T00:47:48.176 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.079036414999999999, 2024-02-23T00:47:48.176 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:47:48.176 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:47:48.176 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:47:48.176 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.176 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:47:48.176 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.712327+0000", 2024-02-23T00:47:48.176 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:47:48.176 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.176 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.176 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:47:48.176 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.712327+0000", 2024-02-23T00:47:48.176 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999986 2024-02-23T00:47:48.177 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.177 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.177 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:47:48.177 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.712325+0000", 2024-02-23T00:47:48.177 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.056795063999999999 2024-02-23T00:47:48.177 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.177 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.177 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:47:48.177 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.769120+0000", 2024-02-23T00:47:48.177 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.6740000000000002e-05 2024-02-23T00:47:48.177 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.177 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.177 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:47:48.177 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.769177+0000", 2024-02-23T00:47:48.178 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8029e-05 2024-02-23T00:47:48.178 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.178 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.178 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:47:48.178 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.769195+0000", 2024-02-23T00:47:48.178 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.7191000000000001e-05 2024-02-23T00:47:48.178 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.178 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.178 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:47:48.178 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.769262+0000", 2024-02-23T00:47:48.178 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.9514e-05 2024-02-23T00:47:48.178 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.178 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.178 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:47:48.178 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.769282+0000", 2024-02-23T00:47:48.179 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.022039333000000001 2024-02-23T00:47:48.179 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.179 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.179 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:47:48.179 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.791321+0000", 2024-02-23T00:47:48.179 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.2144999999999997e-05 2024-02-23T00:47:48.179 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.179 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.179 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:47:48.179 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.791363+0000", 2024-02-23T00:47:48.179 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.079036414999999999 2024-02-23T00:47:48.179 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.179 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:47:48.179 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.179 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.180 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(client.24775.0:2074 20.1a e112/109 20:5de0a470:test-rados-api-smithi047-82457-17::big3:head v 112'3, mlcod=112'1)", 2024-02-23T00:47:48.180 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:01.957298+0000", 2024-02-23T00:47:48.180 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 286.150924428, 2024-02-23T00:47:48.180 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.088898480000000002, 2024-02-23T00:47:48.180 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:47:48.180 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:47:48.180 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:47:48.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.180 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:47:48.180 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.957298+0000", 2024-02-23T00:47:48.180 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:47:48.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.180 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:47:48.181 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.957298+0000", 2024-02-23T00:47:48.181 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999976 2024-02-23T00:47:48.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.181 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.181 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:47:48.181 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.957296+0000", 2024-02-23T00:47:48.181 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.062746578999999997 2024-02-23T00:47:48.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.181 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.181 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:47:48.181 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:02.020043+0000", 2024-02-23T00:47:48.182 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4688000000000001e-05 2024-02-23T00:47:48.182 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.182 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.182 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:47:48.182 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:02.020057+0000", 2024-02-23T00:47:48.183 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.348e-06 2024-02-23T00:47:48.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.183 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.183 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:47:48.183 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:02.020064+0000", 2024-02-23T00:47:48.183 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.5720000000000003e-05 2024-02-23T00:47:48.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.183 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.183 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:47:48.183 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:02.020109+0000", 2024-02-23T00:47:48.183 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.2218000000000002e-05 2024-02-23T00:47:48.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.183 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.183 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:47:48.183 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:02.020141+0000", 2024-02-23T00:47:48.184 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.025988054999999999 2024-02-23T00:47:48.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.184 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.184 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:47:48.184 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:02.046130+0000", 2024-02-23T00:47:48.184 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.7034999999999998e-05 2024-02-23T00:47:48.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.184 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.184 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:47:48.184 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:02.046197+0000", 2024-02-23T00:47:48.184 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.088898480000000002 2024-02-23T00:47:48.184 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.184 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:47:48.184 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.185 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.185 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.185 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24992.0:2 37.15 37:ad551702:::test:head [create] snapc 0=[] ondisk+write+known_if_redirected e115)", 2024-02-23T00:47:48.185 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:04.221218+0000", 2024-02-23T00:47:48.185 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 283.88700474199999, 2024-02-23T00:47:48.185 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.13418903200000001, 2024-02-23T00:47:48.185 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:47:48.185 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:47:48.185 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:47:48.185 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24992", 2024-02-23T00:47:48.185 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/550771032", 2024-02-23T00:47:48.185 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2 2024-02-23T00:47:48.185 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.185 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:47:48.186 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.186 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:47:48.186 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.221218+0000", 2024-02-23T00:47:48.186 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:47:48.186 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.186 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.186 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:47:48.186 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.221218+0000", 2024-02-23T00:47:48.186 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:47:48.186 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.186 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.186 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:47:48.186 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.221216+0000", 2024-02-23T00:47:48.186 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1384e-05 2024-02-23T00:47:48.187 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.187 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.187 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:47:48.187 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.221227+0000", 2024-02-23T00:47:48.187 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2353e-05 2024-02-23T00:47:48.187 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.187 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.187 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:47:48.187 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.221240+0000", 2024-02-23T00:47:48.187 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.7479999999999993e-06 2024-02-23T00:47:48.187 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.187 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.187 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:47:48.187 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.221247+0000", 2024-02-23T00:47:48.187 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.015031549999999999 2024-02-23T00:47:48.188 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.188 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.188 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:47:48.188 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.236279+0000", 2024-02-23T00:47:48.188 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8679999999999999e-06 2024-02-23T00:47:48.188 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.188 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.188 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:47:48.188 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.236287+0000", 2024-02-23T00:47:48.188 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.11065734100000001 2024-02-23T00:47:48.188 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.188 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.188 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:47:48.189 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.346944+0000", 2024-02-23T00:47:48.189 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000113587 2024-02-23T00:47:48.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.189 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.189 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:47:48.189 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.347058+0000", 2024-02-23T00:47:48.189 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00013868100000000001 2024-02-23T00:47:48.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.189 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.189 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 2,4", 2024-02-23T00:47:48.189 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.347196+0000", 2024-02-23T00:47:48.189 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00086159699999999997 2024-02-23T00:47:48.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.189 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.190 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:47:48.190 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.348058+0000", 2024-02-23T00:47:48.190 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.003862533 2024-02-23T00:47:48.190 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.190 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.190 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:48.190 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.351921+0000", 2024-02-23T00:47:48.190 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0034370820000000002 2024-02-23T00:47:48.190 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.190 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.190 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:48.190 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.355358+0000", 2024-02-23T00:47:48.190 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.6704999999999999e-05 2024-02-23T00:47:48.190 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.191 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.191 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:47:48.191 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.355384+0000", 2024-02-23T00:47:48.191 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.2524e-05 2024-02-23T00:47:48.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.191 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.191 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:47:48.191 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.355407+0000", 2024-02-23T00:47:48.191 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.13418903200000001 2024-02-23T00:47:48.191 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.191 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:47:48.191 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.192 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.192 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.32284.0:2 78.6 78:602f83fe:::foo:head [append 0~128] snapc 0=[] ondisk+write+known_if_redirected e139)", 2024-02-23T00:47:48.192 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:31.219941+0000", 2024-02-23T00:47:48.192 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 256.88828109100001, 2024-02-23T00:47:48.192 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.028760101, 2024-02-23T00:47:48.192 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:47:48.192 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:47:48.192 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:47:48.192 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.32284", 2024-02-23T00:47:48.192 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1813864525", 2024-02-23T00:47:48.192 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2 2024-02-23T00:47:48.192 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.192 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:47:48.192 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.193 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:47:48.193 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.219941+0000", 2024-02-23T00:47:48.193 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:47:48.193 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.193 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.193 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:47:48.193 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.219941+0000", 2024-02-23T00:47:48.193 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:47:48.193 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.193 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.193 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:47:48.193 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.219939+0000", 2024-02-23T00:47:48.193 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.498e-06 2024-02-23T00:47:48.193 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.194 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.194 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:47:48.194 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.219947+0000", 2024-02-23T00:47:48.194 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7360000000000001e-05 2024-02-23T00:47:48.194 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.194 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.194 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:47:48.194 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.219984+0000", 2024-02-23T00:47:48.194 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1497e-05 2024-02-23T00:47:48.194 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.194 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.194 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:47:48.194 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.219996+0000", 2024-02-23T00:47:48.194 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00019599900000000001 2024-02-23T00:47:48.195 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.195 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.195 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:47:48.195 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.220192+0000", 2024-02-23T00:47:48.195 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.977e-06 2024-02-23T00:47:48.195 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.195 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.195 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:47:48.195 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.220200+0000", 2024-02-23T00:47:48.195 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.020613092999999999 2024-02-23T00:47:48.195 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.195 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.195 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:47:48.195 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.240813+0000", 2024-02-23T00:47:48.196 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012351699999999999 2024-02-23T00:47:48.196 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.196 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.196 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:47:48.196 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.240936+0000", 2024-02-23T00:47:48.196 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000158527 2024-02-23T00:47:48.196 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.196 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.196 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 3,5", 2024-02-23T00:47:48.196 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.241095+0000", 2024-02-23T00:47:48.196 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0038824100000000002 2024-02-23T00:47:48.196 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.196 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.196 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:48.197 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.244977+0000", 2024-02-23T00:47:48.197 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001243447 2024-02-23T00:47:48.197 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.197 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.197 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:48.197 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.246221+0000", 2024-02-23T00:47:48.197 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0023899310000000001 2024-02-23T00:47:48.197 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.197 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.197 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:47:48.197 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.248611+0000", 2024-02-23T00:47:48.197 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.4546999999999998e-05 2024-02-23T00:47:48.197 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.197 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.198 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:47:48.198 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.248665+0000", 2024-02-23T00:47:48.198 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.6310000000000003e-05 2024-02-23T00:47:48.198 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.198 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.198 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:47:48.198 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.248702+0000", 2024-02-23T00:47:48.198 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.028760101 2024-02-23T00:47:48.198 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.198 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:47:48.198 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.198 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.198 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.198 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(osd.3.0:1389 20.0 e139/109 -22:0000ebd9:::temp_20.0_0_14313_1:head v 0'0, mlcod=0'0)", 2024-02-23T00:47:48.198 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:32.131906+0000", 2024-02-23T00:47:48.198 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 255.97631677699999, 2024-02-23T00:47:48.199 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.025453091000000001, 2024-02-23T00:47:48.199 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:47:48.199 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:47:48.199 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:47:48.199 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.199 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:47:48.199 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.131906+0000", 2024-02-23T00:47:48.199 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:47:48.199 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.199 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.199 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:47:48.199 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.131906+0000", 2024-02-23T00:47:48.199 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999971 2024-02-23T00:47:48.199 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.200 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.200 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:47:48.200 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.131903+0000", 2024-02-23T00:47:48.200 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0055423579999999998 2024-02-23T00:47:48.200 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.200 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.200 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:47:48.200 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.137445+0000", 2024-02-23T00:47:48.200 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.7262000000000001e-05 2024-02-23T00:47:48.200 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.200 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.200 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:47:48.200 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.137473+0000", 2024-02-23T00:47:48.201 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0485e-05 2024-02-23T00:47:48.201 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.201 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.201 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:47:48.201 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.137483+0000", 2024-02-23T00:47:48.201 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00060361099999999999 2024-02-23T00:47:48.201 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.201 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.201 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:47:48.201 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.138087+0000", 2024-02-23T00:47:48.201 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.5690999999999998e-05 2024-02-23T00:47:48.201 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.201 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.201 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:47:48.201 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.138112+0000", 2024-02-23T00:47:48.201 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.019190893000000001 2024-02-23T00:47:48.202 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.202 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.202 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:47:48.202 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.157303+0000", 2024-02-23T00:47:48.202 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.5458000000000003e-05 2024-02-23T00:47:48.202 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.202 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.202 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:47:48.202 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.157359+0000", 2024-02-23T00:47:48.202 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.025453091000000001 2024-02-23T00:47:48.202 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.202 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:47:48.202 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.203 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.203 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.203 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(osd.3.0:1390 20.0 e139/109 -22:0000ebd9:::temp_20.0_0_14313_1:head v 0'0, mlcod=0'0)", 2024-02-23T00:47:48.203 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:32.180625+0000", 2024-02-23T00:47:48.203 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 255.92759767000001, 2024-02-23T00:47:48.203 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.066413153000000003, 2024-02-23T00:47:48.203 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:47:48.203 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:47:48.203 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:47:48.203 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.203 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:47:48.203 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.180625+0000", 2024-02-23T00:47:48.203 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:47:48.203 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.203 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.204 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:47:48.204 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.180625+0000", 2024-02-23T00:47:48.204 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999971 2024-02-23T00:47:48.204 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.204 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.204 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:47:48.204 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.180622+0000", 2024-02-23T00:47:48.204 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018534416000000001 2024-02-23T00:47:48.204 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.204 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.204 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:47:48.204 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.199157+0000", 2024-02-23T00:47:48.204 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0648999999999997e-05 2024-02-23T00:47:48.205 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.205 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.205 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:47:48.205 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.199187+0000", 2024-02-23T00:47:48.205 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.5439999999999999e-06 2024-02-23T00:47:48.205 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.205 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.205 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:47:48.205 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.199196+0000", 2024-02-23T00:47:48.205 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.5427999999999996e-05 2024-02-23T00:47:48.205 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.205 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.205 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:47:48.205 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.199281+0000", 2024-02-23T00:47:48.205 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.2827e-05 2024-02-23T00:47:48.206 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.206 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.206 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:47:48.206 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.199304+0000", 2024-02-23T00:47:48.206 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.047691698999999997 2024-02-23T00:47:48.206 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.206 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.206 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:47:48.206 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.246996+0000", 2024-02-23T00:47:48.206 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.2293999999999999e-05 2024-02-23T00:47:48.206 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.206 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.206 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:47:48.207 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.247038+0000", 2024-02-23T00:47:48.207 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.066413153000000003 2024-02-23T00:47:48.208 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.208 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:47:48.208 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.208 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.208 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.208 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.15270.0:181 75.0 75:00000000::::head [pgnls start_epoch 142] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e142)", 2024-02-23T00:47:48.208 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:34.500215+0000", 2024-02-23T00:47:48.208 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 253.608007265, 2024-02-23T00:47:48.208 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.026354064999999999, 2024-02-23T00:47:48.208 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:47:48.208 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:47:48.208 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:47:48.209 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.15270", 2024-02-23T00:47:48.209 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/4133111990", 2024-02-23T00:47:48.209 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 181 2024-02-23T00:47:48.209 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.209 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:47:48.209 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.209 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:47:48.209 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.500215+0000", 2024-02-23T00:47:48.209 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:47:48.209 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.209 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.209 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:47:48.209 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.500215+0000", 2024-02-23T00:47:48.209 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999971 2024-02-23T00:47:48.210 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.210 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.210 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:47:48.210 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.500212+0000", 2024-02-23T00:47:48.210 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.6798e-05 2024-02-23T00:47:48.210 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.210 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.210 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:47:48.210 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.500239+0000", 2024-02-23T00:47:48.210 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.9426e-05 2024-02-23T00:47:48.210 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.210 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.210 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:47:48.210 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.500259+0000", 2024-02-23T00:47:48.210 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.7950000000000002e-06 2024-02-23T00:47:48.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.211 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.211 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:47:48.211 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.500269+0000", 2024-02-23T00:47:48.211 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0034826060000000001 2024-02-23T00:47:48.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.211 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.211 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:47:48.211 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.503751+0000", 2024-02-23T00:47:48.211 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.1189999999999997e-06 2024-02-23T00:47:48.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.211 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.211 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-02-23T00:47:48.211 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.503759+0000", 2024-02-23T00:47:48.212 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.022616594 2024-02-23T00:47:48.212 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.212 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.212 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:47:48.212 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.526376+0000", 2024-02-23T00:47:48.212 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.6050000000000002e-05 2024-02-23T00:47:48.212 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.212 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.212 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:47:48.212 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.526412+0000", 2024-02-23T00:47:48.212 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000157455 2024-02-23T00:47:48.212 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.212 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.212 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:47:48.213 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.526569+0000", 2024-02-23T00:47:48.213 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.026354064999999999 2024-02-23T00:47:48.213 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.213 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:47:48.213 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.213 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.213 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.213 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.34306.0:2 82.3 82:c656db43:::oid_RoundTripPP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e142)", 2024-02-23T00:47:48.213 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:34.513354+0000", 2024-02-23T00:47:48.213 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 253.59486899000001, 2024-02-23T00:47:48.213 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.056878060000000001, 2024-02-23T00:47:48.213 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:47:48.213 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:47:48.213 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:47:48.213 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.34306", 2024-02-23T00:47:48.214 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1503027630", 2024-02-23T00:47:48.214 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2 2024-02-23T00:47:48.214 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.214 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:47:48.214 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.214 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:47:48.214 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.513354+0000", 2024-02-23T00:47:48.214 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:47:48.214 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.214 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.214 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:47:48.214 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.513354+0000", 2024-02-23T00:47:48.214 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:47:48.214 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.215 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.215 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:47:48.215 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.513352+0000", 2024-02-23T00:47:48.215 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.2949999999999996e-06 2024-02-23T00:47:48.215 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.215 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.215 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:47:48.215 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.513359+0000", 2024-02-23T00:47:48.215 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1875999999999999e-05 2024-02-23T00:47:48.215 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.215 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.215 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:47:48.215 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.513371+0000", 2024-02-23T00:47:48.215 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.4500000000000004e-06 2024-02-23T00:47:48.216 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.216 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.216 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:47:48.216 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.513379+0000", 2024-02-23T00:47:48.216 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.6609e-05 2024-02-23T00:47:48.216 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.216 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.216 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:47:48.216 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.513436+0000", 2024-02-23T00:47:48.216 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.9800000000000001e-06 2024-02-23T00:47:48.216 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.216 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.216 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-02-23T00:47:48.216 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.513443+0000", 2024-02-23T00:47:48.217 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0034703429999999999 2024-02-23T00:47:48.217 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.217 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.217 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:47:48.217 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.516913+0000", 2024-02-23T00:47:48.217 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.4170000000000003e-06 2024-02-23T00:47:48.217 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.217 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.217 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:47:48.217 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.516921+0000", 2024-02-23T00:47:48.217 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.045082443 2024-02-23T00:47:48.218 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.218 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.218 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:47:48.218 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.562003+0000", 2024-02-23T00:47:48.218 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000117848 2024-02-23T00:47:48.218 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.218 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.218 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:47:48.218 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.562121+0000", 2024-02-23T00:47:48.218 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00017905500000000001 2024-02-23T00:47:48.218 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.218 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.218 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 2,3", 2024-02-23T00:47:48.218 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.562300+0000", 2024-02-23T00:47:48.218 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00056806599999999999 2024-02-23T00:47:48.219 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.219 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.219 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:47:48.219 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.562868+0000", 2024-02-23T00:47:48.219 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0069143620000000003 2024-02-23T00:47:48.219 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.219 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.219 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:48.219 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.569782+0000", 2024-02-23T00:47:48.219 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00034649600000000001 2024-02-23T00:47:48.219 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.219 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.219 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:48.219 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.570129+0000", 2024-02-23T00:47:48.220 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.9905000000000003e-05 2024-02-23T00:47:48.220 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.220 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.220 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:47:48.220 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.570189+0000", 2024-02-23T00:47:48.220 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.2885e-05 2024-02-23T00:47:48.220 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.220 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.220 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:47:48.220 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.570232+0000", 2024-02-23T00:47:48.220 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.056878060000000001 2024-02-23T00:47:48.220 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.220 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:47:48.220 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.220 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.221 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.221 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(client.40517.0:2 115.1b e182/180 115:d8841f39:test-rados-api-smithi047-82457-26::foo:head v 182'1, mlcod=0'0)", 2024-02-23T00:47:48.221 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:14.496478+0000", 2024-02-23T00:47:48.221 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 213.61174493999999, 2024-02-23T00:47:48.221 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021581046999999999, 2024-02-23T00:47:48.221 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:47:48.221 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:47:48.221 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:47:48.221 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.221 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:47:48.221 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.496478+0000", 2024-02-23T00:47:48.221 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:47:48.221 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.221 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.222 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:47:48.222 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.496478+0000", 2024-02-23T00:47:48.222 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999971 2024-02-23T00:47:48.222 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.222 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.222 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:47:48.222 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.496475+0000", 2024-02-23T00:47:48.222 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0065093110000000003 2024-02-23T00:47:48.222 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.222 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.222 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:47:48.222 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.502984+0000", 2024-02-23T00:47:48.222 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.6535999999999997e-05 2024-02-23T00:47:48.222 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.222 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.223 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:47:48.223 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.503021+0000", 2024-02-23T00:47:48.223 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.129e-05 2024-02-23T00:47:48.223 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.223 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.223 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:47:48.223 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.503032+0000", 2024-02-23T00:47:48.223 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.5362000000000002e-05 2024-02-23T00:47:48.223 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.223 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.223 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:47:48.223 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.503097+0000", 2024-02-23T00:47:48.223 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.6917e-05 2024-02-23T00:47:48.223 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.223 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.224 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:47:48.224 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.503124+0000", 2024-02-23T00:47:48.224 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.014876454000000001 2024-02-23T00:47:48.224 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.224 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.224 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:47:48.224 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.518001+0000", 2024-02-23T00:47:48.224 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.7868999999999998e-05 2024-02-23T00:47:48.224 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.224 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.224 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:47:48.224 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.518059+0000", 2024-02-23T00:47:48.224 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021581046999999999 2024-02-23T00:47:48.224 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.225 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:47:48.225 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.225 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.225 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.225 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.40517.0:4 113.0s0 113:14a7a751:test-rados-api-smithi047-82457-26::foo.copy:head [copy-from ver 2] snapc 0=[] ondisk+write+known_if_redirected e182)", 2024-02-23T00:47:48.225 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:14.520153+0000", 2024-02-23T00:47:48.225 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 213.58806938399999, 2024-02-23T00:47:48.225 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.020744196999999999, 2024-02-23T00:47:48.225 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:47:48.225 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:47:48.225 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:47:48.225 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.40517", 2024-02-23T00:47:48.225 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/3108721855", 2024-02-23T00:47:48.225 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 4 2024-02-23T00:47:48.225 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.226 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:47:48.226 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.226 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:47:48.226 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.520153+0000", 2024-02-23T00:47:48.226 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:47:48.226 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.226 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.226 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:47:48.226 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.520153+0000", 2024-02-23T00:47:48.226 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999986 2024-02-23T00:47:48.226 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.226 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.226 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:47:48.226 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.520152+0000", 2024-02-23T00:47:48.227 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.614e-06 2024-02-23T00:47:48.227 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.227 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.227 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:47:48.227 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.520157+0000", 2024-02-23T00:47:48.227 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0224000000000002e-05 2024-02-23T00:47:48.227 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.227 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.227 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:47:48.227 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.520178+0000", 2024-02-23T00:47:48.227 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.8919999999999999e-06 2024-02-23T00:47:48.227 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.227 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.227 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:47:48.227 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.520183+0000", 2024-02-23T00:47:48.228 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.6939e-05 2024-02-23T00:47:48.228 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.228 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.228 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:47:48.228 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.520220+0000", 2024-02-23T00:47:48.228 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.5514999999999998e-05 2024-02-23T00:47:48.228 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.228 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.228 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:47:48.228 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.520295+0000", 2024-02-23T00:47:48.228 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.020602321 2024-02-23T00:47:48.228 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.228 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.229 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:47:48.229 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.540897+0000", 2024-02-23T00:47:48.229 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.020744196999999999 2024-02-23T00:47:48.229 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.229 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:47:48.229 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.229 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.229 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.229 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.40553.0:2 117.1a 117:5a7a12bd:::oid_SimpleStatPPNS_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e183)", 2024-02-23T00:47:48.229 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:15.129646+0000", 2024-02-23T00:47:48.229 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 212.97857614899999, 2024-02-23T00:47:48.229 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029333562000000001, 2024-02-23T00:47:48.229 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:47:48.229 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:47:48.229 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:47:48.230 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.40553", 2024-02-23T00:47:48.230 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1915740554", 2024-02-23T00:47:48.230 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2 2024-02-23T00:47:48.230 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.230 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:47:48.230 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.230 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:47:48.230 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:15.129646+0000", 2024-02-23T00:47:48.230 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:47:48.230 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.230 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.230 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:47:48.230 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:15.129646+0000", 2024-02-23T00:47:48.230 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:47:48.231 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.231 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.231 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:47:48.231 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:15.129645+0000", 2024-02-23T00:47:48.231 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.4470000000000006e-06 2024-02-23T00:47:48.231 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.231 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.231 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:47:48.231 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:15.129653+0000", 2024-02-23T00:47:48.231 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0852999999999999e-05 2024-02-23T00:47:48.231 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.231 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.231 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:47:48.231 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:15.129664+0000", 2024-02-23T00:47:48.232 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.4919999999999996e-06 2024-02-23T00:47:48.232 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.232 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.232 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:47:48.232 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:15.129671+0000", 2024-02-23T00:47:48.232 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.7908000000000002e-05 2024-02-23T00:47:48.232 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.232 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.232 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:47:48.232 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:15.129729+0000", 2024-02-23T00:47:48.232 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.6749999999999997e-06 2024-02-23T00:47:48.232 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.232 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.232 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:47:48.232 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:15.129739+0000", 2024-02-23T00:47:48.233 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.015631262 2024-02-23T00:47:48.233 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.233 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.233 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:47:48.233 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:15.145370+0000", 2024-02-23T00:47:48.233 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000156599 2024-02-23T00:47:48.233 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.233 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.233 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:47:48.233 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:15.145527+0000", 2024-02-23T00:47:48.233 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00017407699999999999 2024-02-23T00:47:48.233 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.233 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.233 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 5,7", 2024-02-23T00:47:48.234 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:15.145701+0000", 2024-02-23T00:47:48.234 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0025710630000000002 2024-02-23T00:47:48.234 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.234 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.234 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:47:48.234 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:15.148272+0000", 2024-02-23T00:47:48.234 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0045611039999999999 2024-02-23T00:47:48.234 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.234 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.234 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:48.234 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:15.152833+0000", 2024-02-23T00:47:48.234 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0060428050000000001 2024-02-23T00:47:48.234 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.234 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.235 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:47:48.235 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:15.158876+0000", 2024-02-23T00:47:48.235 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.2118000000000004e-05 2024-02-23T00:47:48.235 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.235 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.235 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:47:48.235 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:15.158938+0000", 2024-02-23T00:47:48.235 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.1950000000000003e-05 2024-02-23T00:47:48.235 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.235 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.235 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:47:48.235 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:15.158980+0000", 2024-02-23T00:47:48.235 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029333562000000001 2024-02-23T00:47:48.235 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.235 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:47:48.235 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.236 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.236 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.236 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.40529.0:9 114.5s0 114:b5491451:test-rados-api-smithi047-82496-18::foo:head [writefull 0~128] snapc 0=[] ondisk+write+known_if_redirected e186)", 2024-02-23T00:47:48.236 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:18.114404+0000", 2024-02-23T00:47:48.236 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 209.993818046, 2024-02-23T00:47:48.236 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.024854564999999999, 2024-02-23T00:47:48.236 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:47:48.236 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:47:48.236 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:47:48.236 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.40529", 2024-02-23T00:47:48.236 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/3414618047", 2024-02-23T00:47:48.236 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 9 2024-02-23T00:47:48.236 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.237 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:47:48.237 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.237 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:47:48.237 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:18.114404+0000", 2024-02-23T00:47:48.237 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:47:48.237 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.237 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.237 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:47:48.237 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:18.114404+0000", 2024-02-23T00:47:48.237 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999986 2024-02-23T00:47:48.237 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.237 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.237 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:47:48.237 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:18.114403+0000", 2024-02-23T00:47:48.237 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.5510000000000002e-06 2024-02-23T00:47:48.238 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.238 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.238 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:47:48.238 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:18.114411+0000", 2024-02-23T00:47:48.238 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2646999999999999e-05 2024-02-23T00:47:48.238 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.238 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.238 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:47:48.238 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:18.114423+0000", 2024-02-23T00:47:48.238 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.2630000000000003e-06 2024-02-23T00:47:48.238 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.238 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.238 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:47:48.238 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:18.114432+0000", 2024-02-23T00:47:48.239 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.9015000000000003e-05 2024-02-23T00:47:48.239 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.239 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.239 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:47:48.239 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:18.114491+0000", 2024-02-23T00:47:48.239 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.1619999999999997e-06 2024-02-23T00:47:48.239 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.239 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.239 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-02-23T00:47:48.239 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:18.114500+0000", 2024-02-23T00:47:48.239 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0058377619999999998 2024-02-23T00:47:48.239 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.239 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.239 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:47:48.239 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:18.120337+0000", 2024-02-23T00:47:48.240 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012195 2024-02-23T00:47:48.240 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.240 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.240 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:47:48.240 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:18.120459+0000", 2024-02-23T00:47:48.240 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00073385099999999999 2024-02-23T00:47:48.240 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.240 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.240 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-02-23T00:47:48.240 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:18.121193+0000", 2024-02-23T00:47:48.240 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000578678 2024-02-23T00:47:48.240 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.240 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.240 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-02-23T00:47:48.241 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:18.121772+0000", 2024-02-23T00:47:48.241 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.017397617000000001 2024-02-23T00:47:48.241 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.241 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.241 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:47:48.241 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:18.139170+0000", 2024-02-23T00:47:48.241 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.9493999999999994e-05 2024-02-23T00:47:48.241 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.241 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.241 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:47:48.241 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:18.139259+0000", 2024-02-23T00:47:48.241 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.024854564999999999 2024-02-23T00:47:48.241 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.241 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:47:48.242 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.242 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.242 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.242 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(osd.3.0:1452 62.c e199/126 62:303239da:test-rados-api-smithi047-82465-14::foo78:4 v 199'28, mlcod=195'24)", 2024-02-23T00:47:48.242 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:31.164164+0000", 2024-02-23T00:47:48.242 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 196.94405896699999, 2024-02-23T00:47:48.242 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023846743, 2024-02-23T00:47:48.242 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:47:48.242 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:47:48.242 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:47:48.242 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.242 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:47:48.242 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164164+0000", 2024-02-23T00:47:48.242 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:47:48.242 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.242 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.243 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:47:48.243 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164164+0000", 2024-02-23T00:47:48.243 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999943 2024-02-23T00:47:48.243 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.243 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.243 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:47:48.243 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164158+0000", 2024-02-23T00:47:48.243 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4290999999999999e-05 2024-02-23T00:47:48.243 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.243 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.243 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:47:48.243 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164172+0000", 2024-02-23T00:47:48.243 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0293e-05 2024-02-23T00:47:48.243 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.243 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.243 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:47:48.244 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164183+0000", 2024-02-23T00:47:48.244 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.575e-06 2024-02-23T00:47:48.244 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.244 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.244 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:47:48.244 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164188+0000", 2024-02-23T00:47:48.244 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.8688000000000001e-05 2024-02-23T00:47:48.244 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.244 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.244 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:47:48.244 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164287+0000", 2024-02-23T00:47:48.244 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.9468999999999999e-05 2024-02-23T00:47:48.244 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.245 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.245 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:47:48.245 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164306+0000", 2024-02-23T00:47:48.245 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023682110999999999 2024-02-23T00:47:48.245 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.245 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.245 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:47:48.245 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.187988+0000", 2024-02-23T00:47:48.245 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1946e-05 2024-02-23T00:47:48.245 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.245 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.245 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:47:48.245 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.188010+0000", 2024-02-23T00:47:48.245 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023846743 2024-02-23T00:47:48.246 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.246 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:47:48.246 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.246 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.246 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.246 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(osd.3.0:1453 62.c e199/126 62:37762822:test-rados-api-smithi047-82465-14::foo50:4 v 199'30, mlcod=195'24)", 2024-02-23T00:47:48.246 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:31.164203+0000", 2024-02-23T00:47:48.246 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 196.944019868, 2024-02-23T00:47:48.246 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023951410999999999, 2024-02-23T00:47:48.246 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:47:48.246 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:47:48.246 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:47:48.246 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.246 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:47:48.247 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164203+0000", 2024-02-23T00:47:48.247 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:47:48.247 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.247 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.247 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:47:48.247 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164203+0000", 2024-02-23T00:47:48.247 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:47:48.247 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.247 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.247 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:47:48.247 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164202+0000", 2024-02-23T00:47:48.247 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.4599999999999992e-06 2024-02-23T00:47:48.247 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.247 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.248 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:47:48.248 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164212+0000", 2024-02-23T00:47:48.248 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.5989999999999998e-06 2024-02-23T00:47:48.248 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.248 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.248 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:47:48.248 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164218+0000", 2024-02-23T00:47:48.248 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.1080000000000003e-06 2024-02-23T00:47:48.248 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.248 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.248 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:47:48.248 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164222+0000", 2024-02-23T00:47:48.248 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00036172199999999998 2024-02-23T00:47:48.249 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.249 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.249 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:47:48.249 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164584+0000", 2024-02-23T00:47:48.249 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.9641e-05 2024-02-23T00:47:48.249 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.249 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.249 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:47:48.249 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164604+0000", 2024-02-23T00:47:48.249 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023511941000000001 2024-02-23T00:47:48.249 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.249 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.249 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:47:48.249 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.188116+0000", 2024-02-23T00:47:48.249 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.8358000000000001e-05 2024-02-23T00:47:48.249 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.250 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.250 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:47:48.250 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.188154+0000", 2024-02-23T00:47:48.250 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023951410999999999 2024-02-23T00:47:48.250 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.250 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:47:48.250 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.250 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.250 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.250 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop_reply(osd.0.0:331 62.e e199/126 ondisk, result = 0)", 2024-02-23T00:47:48.250 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:31.169340+0000", 2024-02-23T00:47:48.250 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 196.938882568, 2024-02-23T00:47:48.250 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018945747999999998, 2024-02-23T00:47:48.250 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:47:48.250 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:47:48.250 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:47:48.251 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.251 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:47:48.251 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.169340+0000", 2024-02-23T00:47:48.251 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:47:48.251 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.251 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.251 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:47:48.251 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.169340+0000", 2024-02-23T00:47:48.251 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999986 2024-02-23T00:47:48.251 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.251 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.251 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:47:48.251 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.169339+0000", 2024-02-23T00:47:48.251 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.4970000000000001e-06 2024-02-23T00:47:48.251 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.251 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.252 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:47:48.252 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.169347+0000", 2024-02-23T00:47:48.252 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.2152999999999999e-05 2024-02-23T00:47:48.252 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.252 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.252 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:47:48.252 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.169369+0000", 2024-02-23T00:47:48.252 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0320000000000003e-06 2024-02-23T00:47:48.252 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.252 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.252 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:47:48.252 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.169377+0000", 2024-02-23T00:47:48.252 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018879638000000001 2024-02-23T00:47:48.252 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.252 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.253 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:47:48.253 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.188257+0000", 2024-02-23T00:47:48.253 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.3629999999999994e-06 2024-02-23T00:47:48.253 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.253 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.253 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:47:48.253 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.188265+0000", 2024-02-23T00:47:48.253 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0305999999999998e-05 2024-02-23T00:47:48.253 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.253 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.253 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:47:48.253 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.188286+0000", 2024-02-23T00:47:48.253 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018945747999999998 2024-02-23T00:47:48.253 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.253 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:47:48.253 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.254 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.254 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.254 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWrite(129.7s2 199/195 ECSubWrite(tid=1454, reqid=client.42730.0:8, at_version=199'4, trim_to=0'0, roll_forward_to=197'3))", 2024-02-23T00:47:48.254 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:31.170261+0000", 2024-02-23T00:47:48.254 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 196.93796161899999, 2024-02-23T00:47:48.254 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.017706718, 2024-02-23T00:47:48.254 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:47:48.254 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-02-23T00:47:48.254 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:47:48.254 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.254 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:47:48.254 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.170261+0000", 2024-02-23T00:47:48.254 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:47:48.254 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.254 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.255 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:47:48.255 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.170261+0000", 2024-02-23T00:47:48.255 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.999999 2024-02-23T00:47:48.255 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.255 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.255 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:47:48.255 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.170260+0000", 2024-02-23T00:47:48.255 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.4536e-05 2024-02-23T00:47:48.255 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.255 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.255 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:47:48.255 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.170284+0000", 2024-02-23T00:47:48.255 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.9298999999999999e-05 2024-02-23T00:47:48.255 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.255 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.256 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:47:48.256 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.170304+0000", 2024-02-23T00:47:48.256 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0399999999999993e-06 2024-02-23T00:47:48.256 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.256 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.256 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:47:48.256 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.170312+0000", 2024-02-23T00:47:48.256 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001240533 2024-02-23T00:47:48.256 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.256 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.256 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:47:48.256 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.171552+0000", 2024-02-23T00:47:48.256 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.201e-05 2024-02-23T00:47:48.256 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.256 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.257 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-02-23T00:47:48.257 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.171564+0000", 2024-02-23T00:47:48.257 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.016382556999999999 2024-02-23T00:47:48.257 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.257 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.257 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-02-23T00:47:48.257 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.187947+0000", 2024-02-23T00:47:48.257 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0727999999999999e-05 2024-02-23T00:47:48.257 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.257 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.257 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:47:48.257 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.187968+0000", 2024-02-23T00:47:48.257 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.017706718 2024-02-23T00:47:48.257 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.257 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:47:48.258 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.258 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.258 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.258 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop_reply(osd.0.0:331 62.e e199/126 ondisk, result = 0)", 2024-02-23T00:47:48.258 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:31.170335+0000", 2024-02-23T00:47:48.258 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 196.937887491, 2024-02-23T00:47:48.258 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018153419, 2024-02-23T00:47:48.258 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:47:48.258 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:47:48.258 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:47:48.258 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.258 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:47:48.258 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.170335+0000", 2024-02-23T00:47:48.258 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:47:48.258 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.258 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.259 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:47:48.259 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.170335+0000", 2024-02-23T00:47:48.259 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:47:48.259 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.259 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.259 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:47:48.259 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.170335+0000", 2024-02-23T00:47:48.259 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.856e-06 2024-02-23T00:47:48.259 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.259 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.259 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:47:48.259 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.170338+0000", 2024-02-23T00:47:48.259 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.5179999999999998e-06 2024-02-23T00:47:48.259 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.259 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.259 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:47:48.260 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.170346+0000", 2024-02-23T00:47:48.260 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.9649999999999999e-06 2024-02-23T00:47:48.260 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.260 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.260 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:47:48.260 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.170351+0000", 2024-02-23T00:47:48.260 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018060857 2024-02-23T00:47:48.260 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.260 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.260 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:47:48.260 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.188412+0000", 2024-02-23T00:47:48.260 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.7190000000000001e-06 2024-02-23T00:47:48.260 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.261 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.261 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:47:48.261 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.188419+0000", 2024-02-23T00:47:48.261 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.9029000000000006e-05 2024-02-23T00:47:48.261 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.261 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.261 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:47:48.261 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.188488+0000", 2024-02-23T00:47:48.261 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018153419 2024-02-23T00:47:48.261 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.261 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:47:48.261 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.261 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.261 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.261 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.41087.0:1 189.2s0 189:5455dc94:::oid_IsSafePP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e318)", 2024-02-23T00:47:48.262 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:46:30.187851+0000", 2024-02-23T00:47:48.262 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 77.920371364999994, 2024-02-23T00:47:48.262 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.57313715, 2024-02-23T00:47:48.262 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:47:48.262 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:47:48.262 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:47:48.262 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.41087", 2024-02-23T00:47:48.262 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/2888114840", 2024-02-23T00:47:48.262 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 1 2024-02-23T00:47:48.262 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.262 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:47:48.262 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.262 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:47:48.262 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:30.187851+0000", 2024-02-23T00:47:48.263 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:47:48.263 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.263 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.263 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:47:48.263 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:30.187851+0000", 2024-02-23T00:47:48.263 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999976 2024-02-23T00:47:48.263 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.263 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.263 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:47:48.263 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:30.187849+0000", 2024-02-23T00:47:48.263 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.1200000000000008e-06 2024-02-23T00:47:48.263 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.263 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.263 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:47:48.263 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:30.187858+0000", 2024-02-23T00:47:48.264 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5472e-05 2024-02-23T00:47:48.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.264 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.264 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:47:48.264 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:30.187874+0000", 2024-02-23T00:47:48.264 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.5690000000000005e-06 2024-02-23T00:47:48.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.264 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.264 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:47:48.264 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:30.187882+0000", 2024-02-23T00:47:48.264 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.7757999999999999e-05 2024-02-23T00:47:48.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.264 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.264 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:47:48.265 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:30.187930+0000", 2024-02-23T00:47:48.265 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.269e-06 2024-02-23T00:47:48.265 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.265 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.265 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:47:48.265 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:30.187933+0000", 2024-02-23T00:47:48.265 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5695471319999998 2024-02-23T00:47:48.265 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.265 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.265 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:47:48.265 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:31.757480+0000", 2024-02-23T00:47:48.265 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00013165799999999999 2024-02-23T00:47:48.265 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.265 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.266 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:47:48.266 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:31.757612+0000", 2024-02-23T00:47:48.266 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00041009 2024-02-23T00:47:48.266 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.266 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.266 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-02-23T00:47:48.266 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:31.758022+0000", 2024-02-23T00:47:48.266 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00061976200000000005 2024-02-23T00:47:48.266 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.266 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.266 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-02-23T00:47:48.266 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:31.758642+0000", 2024-02-23T00:47:48.266 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0023075489999999999 2024-02-23T00:47:48.266 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.267 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.267 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:47:48.267 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:31.760949+0000", 2024-02-23T00:47:48.267 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.8970000000000001e-05 2024-02-23T00:47:48.267 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.267 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.267 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:47:48.267 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:31.760988+0000", 2024-02-23T00:47:48.267 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.57313715 2024-02-23T00:47:48.267 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.267 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:47:48.267 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.267 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.267 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.268 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.41243.0:1 203.4s0 203:311cbe94:::oid_ExecuteClassPP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e377)", 2024-02-23T00:47:48.268 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:47:29.186078+0000", 2024-02-23T00:47:48.268 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 18.922144563, 2024-02-23T00:47:48.268 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.004199842, 2024-02-23T00:47:48.268 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:47:48.268 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:47:48.268 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:47:48.268 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.41243", 2024-02-23T00:47:48.268 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/3348293548", 2024-02-23T00:47:48.268 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 1 2024-02-23T00:47:48.268 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.268 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:47:48.268 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.268 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:47:48.269 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:29.186078+0000", 2024-02-23T00:47:48.269 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:47:48.269 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.269 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.269 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:47:48.269 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:29.186078+0000", 2024-02-23T00:47:48.269 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:47:48.269 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.269 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.269 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:47:48.269 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:29.186076+0000", 2024-02-23T00:47:48.269 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.2549999999999997e-06 2024-02-23T00:47:48.269 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.269 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.269 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:47:48.270 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:29.186084+0000", 2024-02-23T00:47:48.270 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2850000000000001e-05 2024-02-23T00:47:48.270 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.270 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.270 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:47:48.270 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:29.186097+0000", 2024-02-23T00:47:48.270 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.2300000000000008e-06 2024-02-23T00:47:48.270 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.270 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.270 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:47:48.270 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:29.186105+0000", 2024-02-23T00:47:48.270 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0018958600000000001 2024-02-23T00:47:48.270 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.270 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.270 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:47:48.271 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:29.188001+0000", 2024-02-23T00:47:48.271 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.1489999999999998e-06 2024-02-23T00:47:48.271 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.271 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.271 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:47:48.271 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:29.188004+0000", 2024-02-23T00:47:48.271 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.000917512 2024-02-23T00:47:48.271 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.271 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.271 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:47:48.271 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:30.188922+0000", 2024-02-23T00:47:48.271 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00013111000000000001 2024-02-23T00:47:48.271 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.271 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.271 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:47:48.272 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:30.189053+0000", 2024-02-23T00:47:48.272 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00040387600000000001 2024-02-23T00:47:48.272 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.272 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.272 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-02-23T00:47:48.272 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:30.189457+0000", 2024-02-23T00:47:48.272 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00043691699999999999 2024-02-23T00:47:48.272 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.272 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.272 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-02-23T00:47:48.272 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:30.189894+0000", 2024-02-23T00:47:48.272 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00031495399999999999 2024-02-23T00:47:48.272 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.272 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.273 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:47:48.273 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:30.190209+0000", 2024-02-23T00:47:48.273 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.9130999999999999e-05 2024-02-23T00:47:48.273 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:47:48.273 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:47:48.273 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:47:48.273 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:30.190278+0000", 2024-02-23T00:47:48.273 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.004199842 2024-02-23T00:47:48.273 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.273 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:47:48.273 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.273 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:47:48.273 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:47:48.274 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:47:48.510 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:47.280726+0000 mgr.y (mgr.14517) 365 : cluster [DBG] pgmap v568: 211 pgs: 1 peering, 1 active+remapped+backfill_toofull, 2 activating, 207 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/1125 objects misplaced (0.089%); 21 B/s, 9 objects/s recovering 2024-02-23T00:47:48.510 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:47.918619+0000 osd.1 (osd.1) 165 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:47:48.510 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:47.918750+0000 osd.1 (osd.1) 166 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:47:48.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:47.280726+0000 mgr.y (mgr.14517) 365 : cluster [DBG] pgmap v568: 211 pgs: 1 peering, 1 active+remapped+backfill_toofull, 2 activating, 207 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/1125 objects misplaced (0.089%); 21 B/s, 9 objects/s recovering 2024-02-23T00:47:48.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:47.918619+0000 osd.1 (osd.1) 165 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:47:48.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:47.918750+0000 osd.1 (osd.1) 166 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:47:48.885 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:47:48.888 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:48.888 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:47:48.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:47.280726+0000 mgr.y (mgr.14517) 365 : cluster [DBG] pgmap v568: 211 pgs: 1 peering, 1 active+remapped+backfill_toofull, 2 activating, 207 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/1125 objects misplaced (0.089%); 21 B/s, 9 objects/s recovering 2024-02-23T00:47:48.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:47.918619+0000 osd.1 (osd.1) 165 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:47:48.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:47.918750+0000 osd.1 (osd.1) 166 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:47:48.989 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:47:49.017 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:49.305 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.1 dump_ops_in_flight 2024-02-23T00:47:49.316 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:47:49.337 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:47:49.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:48.513659+0000 mon.a (mon.0) 2493 : cluster [DBG] osdmap e396: 8 total, 8 up, 7 in 2024-02-23T00:47:49.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:49.198342+0000 mon.a (mon.0) 2494 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:47:49.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:49.282128+0000 mon.a (mon.0) 2495 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "13"}]: dispatch 2024-02-23T00:47:49.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:48.513659+0000 mon.a (mon.0) 2493 : cluster [DBG] osdmap e396: 8 total, 8 up, 7 in 2024-02-23T00:47:49.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:49.198342+0000 mon.a (mon.0) 2494 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:47:49.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:49.282128+0000 mon.a (mon.0) 2495 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "13"}]: dispatch 2024-02-23T00:47:49.566 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:47:49.568 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:49.569 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:47:49.670 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:47:49.702 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:49.723 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:47:49.725 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:49.725 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:47:49.826 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:47:49.856 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:49.931 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:47:49.935 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:49.935 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:47:49.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:48.513659+0000 mon.a (mon.0) 2493 : cluster [DBG] osdmap e396: 8 total, 8 up, 7 in 2024-02-23T00:47:49.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:49.198342+0000 mon.a (mon.0) 2494 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:47:49.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:49.282128+0000 mon.a (mon.0) 2495 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "13"}]: dispatch 2024-02-23T00:47:50.036 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:47:50.068 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:50.089 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:47:50.091 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:50.092 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:47:50.193 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:47:50.221 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:50.240 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:47:50.243 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:50.243 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:47:50.344 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:47:50.371 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:50.390 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:47:50.393 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:50.393 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:47:50.494 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:47:50.526 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:50.670 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:47:50.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:48.811180+0000 osd.0 (osd.0) 157 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:47:50.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:48.811312+0000 osd.0 (osd.0) 158 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:47:50.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:49.281234+0000 mgr.y (mgr.14517) 366 : cluster [DBG] pgmap v570: 210 pgs: 1 peering, 1 active+remapped+backfill_toofull, 1 activating, 207 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/1125 objects misplaced (0.089%); 21 B/s, 9 objects/s recovering 2024-02-23T00:47:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:49.519824+0000 mon.a (mon.0) 2496 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:47:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:49.519893+0000 mon.a (mon.0) 2497 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "13"}]': finished 2024-02-23T00:47:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:49.519948+0000 mon.a (mon.0) 2498 : cluster [DBG] osdmap e397: 8 total, 8 up, 7 in 2024-02-23T00:47:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:49.522043+0000 mon.a (mon.0) 2499 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-49"}]: dispatch 2024-02-23T00:47:50.697 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:48.811180+0000 osd.0 (osd.0) 157 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:47:50.697 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:48.811312+0000 osd.0 (osd.0) 158 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:47:50.697 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:49.281234+0000 mgr.y (mgr.14517) 366 : cluster [DBG] pgmap v570: 210 pgs: 1 peering, 1 active+remapped+backfill_toofull, 1 activating, 207 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/1125 objects misplaced (0.089%); 21 B/s, 9 objects/s recovering 2024-02-23T00:47:50.697 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:49.519824+0000 mon.a (mon.0) 2496 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:47:50.697 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:49.519893+0000 mon.a (mon.0) 2497 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "13"}]': finished 2024-02-23T00:47:50.698 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:49.519948+0000 mon.a (mon.0) 2498 : cluster [DBG] osdmap e397: 8 total, 8 up, 7 in 2024-02-23T00:47:50.698 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:49.522043+0000 mon.a (mon.0) 2499 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-49"}]: dispatch 2024-02-23T00:47:50.698 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:47:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:47:50] "GET /metrics HTTP/1.1" 200 188714 "" "Prometheus/2.33.4" 2024-02-23T00:47:50.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:48.811180+0000 osd.0 (osd.0) 157 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:47:50.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:48.811312+0000 osd.0 (osd.0) 158 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:47:50.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:49.281234+0000 mgr.y (mgr.14517) 366 : cluster [DBG] pgmap v570: 210 pgs: 1 peering, 1 active+remapped+backfill_toofull, 1 activating, 207 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/1125 objects misplaced (0.089%); 21 B/s, 9 objects/s recovering 2024-02-23T00:47:50.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:49.519824+0000 mon.a (mon.0) 2496 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:47:50.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:49.519893+0000 mon.a (mon.0) 2497 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "13"}]': finished 2024-02-23T00:47:50.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:49.519948+0000 mon.a (mon.0) 2498 : cluster [DBG] osdmap e397: 8 total, 8 up, 7 in 2024-02-23T00:47:50.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:49.522043+0000 mon.a (mon.0) 2499 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-49"}]: dispatch 2024-02-23T00:47:51.278 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:47:51.281 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:51.281 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:47:51.382 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:47:51.412 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:51.440 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:47:51.443 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:51.443 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:47:51.535 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:50.520395+0000 mon.a (mon.0) 2500 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-49"}]': finished 2024-02-23T00:47:51.535 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:50.520480+0000 mon.a (mon.0) 2501 : cluster [DBG] osdmap e398: 8 total, 8 up, 7 in 2024-02-23T00:47:51.536 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:50.546327+0000 mon.a (mon.0) 2502 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:47:51.536 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:50.546671+0000 mon.a (mon.0) 2503 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-49"}]: dispatch 2024-02-23T00:47:51.536 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:51.470632+0000 mon.a (mon.0) 2504 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:47:51.536 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:51.516206+0000 mon.a (mon.0) 2505 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:47:51.536 INFO:tasks.workunit.client.0.smithi047.stdout:b4c720e9:::166:head 2024-02-23T00:47:51.536 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:e694b040:::167:head 2024-02-23T00:47:51.537 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:afa38db2:::168:head 2024-02-23T00:47:51.537 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:77ba9f53:::169:head 2024-02-23T00:47:51.537 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:87495034:::170:head 2024-02-23T00:47:51.537 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:7c96bf0e:::171:head 2024-02-23T00:47:51.537 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:dbe346cc:::172:head 2024-02-23T00:47:51.537 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:e943ec24:::173:head 2024-02-23T00:47:51.537 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:f97a9c0c:::174:head 2024-02-23T00:47:51.537 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:6f26e74d:::175:head 2024-02-23T00:47:51.537 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:4f95e106:::176:head 2024-02-23T00:47:51.537 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:0e6f2f8f:::177:head 2024-02-23T00:47:51.537 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:05db05f1:::178:head 2024-02-23T00:47:51.537 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:38a78d66:::179:head 2024-02-23T00:47:51.538 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:d095610b:::180:head 2024-02-23T00:47:51.538 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:a1a9d709:::181:head 2024-02-23T00:47:51.538 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:1e5d39db:::182:head 2024-02-23T00:47:51.538 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:f7df4fb9:::183:head 2024-02-23T00:47:51.538 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:03a7f161:::184:head 2024-02-23T00:47:51.538 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:ba70721e:::185:head 2024-02-23T00:47:51.538 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:28e5662d:::186:head 2024-02-23T00:47:51.538 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:973d52de:::187:head 2024-02-23T00:47:51.538 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:4303eb1c:::188:head 2024-02-23T00:47:51.538 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:b990b48e:::189:head 2024-02-23T00:47:51.538 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:29b8165b:::190:head 2024-02-23T00:47:51.538 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:3547f197:::191:head 2024-02-23T00:47:51.539 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:7e260936:::192:head 2024-02-23T00:47:51.539 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:1abec7b1:::193:head 2024-02-23T00:47:51.539 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:10fdda93:::194:head 2024-02-23T00:47:51.539 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:15817eea:::195:head 2024-02-23T00:47:51.539 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:770bab57:::196:head 2024-02-23T00:47:51.539 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:ed9e13e7:::197:head 2024-02-23T00:47:51.539 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:71471a8f:::198:head 2024-02-23T00:47:51.539 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: checking for 188:10fb1d02:::199:head 2024-02-23T00:47:51.539 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (8457 ms) 2024-02-23T00:47:51.539 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-02-23T00:47:51.539 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: got ls 1708649204,0 2024-02-23T00:47:51.539 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: first is 1708649204 2024-02-23T00:47:51.540 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: got ls 1708649204,0 2024-02-23T00:47:51.540 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: got ls 1708649204,0 2024-02-23T00:47:51.540 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: got ls 1708649204,1708649206,1708649207,0 2024-02-23T00:47:51.540 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: got ls 1708649204,1708649206,1708649207,0 2024-02-23T00:47:51.540 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: got ls 1708649204,1708649206,1708649207,0 2024-02-23T00:47:51.540 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: got ls 1708649204,1708649206,1708649207,1708649209,1708649210,0 2024-02-23T00:47:51.540 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: got ls 1708649204,1708649206,1708649207,1708649209,1708649210,0 2024-02-23T00:47:51.540 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: got ls 1708649204,1708649206,1708649207,1708649209,1708649210,0 2024-02-23T00:47:51.540 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: got ls 1708649204,1708649206,1708649207,1708649209,1708649210,1708649212,1708649213,0 2024-02-23T00:47:51.540 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: got ls 1708649204,1708649206,1708649207,1708649209,1708649210,1708649212,1708649213,0 2024-02-23T00:47:51.540 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: got ls 1708649204,1708649206,1708649207,1708649209,1708649210,1708649212,1708649213,0 2024-02-23T00:47:51.541 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: got ls 1708649207,1708649209,1708649210,1708649212,1708649213,1708649215,1708649216,0 2024-02-23T00:47:51.541 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: first now 1708649207, trimmed 2024-02-23T00:47:51.541 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (20984 ms) 2024-02-23T00:47:51.541 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-02-23T00:47:51.541 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: foo0 2024-02-23T00:47:51.541 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-02-23T00:47:51.541 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (13597 ms) 2024-02-23T00:47:51.541 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-02-23T00:47:51.541 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ProxyRead (17992 ms) 2024-02-23T00:47:51.541 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2024-02-23T00:47:51.545 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:47:51.577 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:51.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:50.520395+0000 mon.a (mon.0) 2500 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-49"}]': finished 2024-02-23T00:47:51.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:50.520480+0000 mon.a (mon.0) 2501 : cluster [DBG] osdmap e398: 8 total, 8 up, 7 in 2024-02-23T00:47:51.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:50.546327+0000 mon.a (mon.0) 2502 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:47:51.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:50.546671+0000 mon.a (mon.0) 2503 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-49"}]: dispatch 2024-02-23T00:47:51.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:51.470632+0000 mon.a (mon.0) 2504 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:47:51.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:51.516206+0000 mon.a (mon.0) 2505 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:47:51.814 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:47:51.835 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:47:51.838 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:51.838 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:47:51.939 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:47:51.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:50.520395+0000 mon.a (mon.0) 2500 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-49"}]': finished 2024-02-23T00:47:51.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:50.520480+0000 mon.a (mon.0) 2501 : cluster [DBG] osdmap e398: 8 total, 8 up, 7 in 2024-02-23T00:47:51.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:50.546327+0000 mon.a (mon.0) 2502 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:47:51.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:50.546671+0000 mon.a (mon.0) 2503 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-49"}]: dispatch 2024-02-23T00:47:51.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:51.470632+0000 mon.a (mon.0) 2504 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:47:51.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:51.516206+0000 mon.a (mon.0) 2505 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:47:51.966 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:51.986 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:47:51.988 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:51.988 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:47:52.089 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:47:52.115 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:52.133 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:47:52.136 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:52.136 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:47:52.237 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:47:52.267 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:52.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.337+0000 7fc159d72700 1 -- 172.21.15.47:0/733185105 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc1540f7400 msgr2=0x7fc154103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:52.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.337+0000 7fc159d72700 1 --2- 172.21.15.47:0/733185105 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc1540f7400 0x7fc154103a20 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 rx=0x7fc144009be0 tx=0x7fc144004d10).stop 2024-02-23T00:47:52.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.337+0000 7fc159d72700 1 -- 172.21.15.47:0/733185105 shutdown_connections 2024-02-23T00:47:52.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.337+0000 7fc159d72700 1 --2- 172.21.15.47:0/733185105 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc1540f7400 0x7fc154103a20 unknown :-1 s=CLOSED pgs=2657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:52.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.337+0000 7fc159d72700 1 --2- 172.21.15.47:0/733185105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1540fcbb0 0x7fc1540fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:52.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.337+0000 7fc159d72700 1 --2- 172.21.15.47:0/733185105 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1540fd510 0x7fc1540f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.337+0000 7fc159d72700 1 -- 172.21.15.47:0/733185105 >> 172.21.15.47:0/733185105 conn(0x7fc1540f2820 msgr2=0x7fc1540f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.337+0000 7fc159d72700 1 -- 172.21.15.47:0/733185105 shutdown_connections 2024-02-23T00:47:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.337+0000 7fc159d72700 1 -- 172.21.15.47:0/733185105 wait complete. 2024-02-23T00:47:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.338+0000 7fc159d72700 1 librados: starting msgr at 2024-02-23T00:47:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.338+0000 7fc159d72700 1 librados: starting objecter 2024-02-23T00:47:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.338+0000 7fc159d72700 1 Processor -- start 2024-02-23T00:47:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.338+0000 7fc159d72700 1 -- start start 2024-02-23T00:47:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.338+0000 7fc159d72700 1 librados: setting wanted keys 2024-02-23T00:47:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.338+0000 7fc159d72700 1 librados: calling monclient init 2024-02-23T00:47:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.338+0000 7fc159d72700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc1540f7400 0x7fc1540f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.338+0000 7fc159d72700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1540fcbb0 0x7fc1540f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.338+0000 7fc159d72700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1540fd510 0x7fc1540f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.338+0000 7fc159d72700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1541162c0 con 0x7fc1540fcbb0 2024-02-23T00:47:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.338+0000 7fc159d72700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc154116430 con 0x7fc1540fd510 2024-02-23T00:47:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.338+0000 7fc159d72700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1541165f0 con 0x7fc1540f7400 2024-02-23T00:47:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.338+0000 7fc1537fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc1540f7400 0x7fc1540f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.338+0000 7fc1537fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc1540f7400 0x7fc1540f89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:42748/0 (socket says 172.21.15.47:42748) 2024-02-23T00:47:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.338+0000 7fc1537fe700 1 -- 172.21.15.47:0/3584697987 learned_addr learned my addr 172.21.15.47:0/3584697987 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:47:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.338+0000 7fc152ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1540fcbb0 0x7fc1540f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.338+0000 7fc153fff700 1 --2- 172.21.15.47:0/3584697987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1540fd510 0x7fc1540f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.339+0000 7fc1537fe700 1 -- 172.21.15.47:0/3584697987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1540fd510 msgr2=0x7fc1540f9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.339+0000 7fc1537fe700 1 --2- 172.21.15.47:0/3584697987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1540fd510 0x7fc1540f9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.339+0000 7fc1537fe700 1 -- 172.21.15.47:0/3584697987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1540fcbb0 msgr2=0x7fc1540f8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.339+0000 7fc1537fe700 1 --2- 172.21.15.47:0/3584697987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1540fcbb0 0x7fc1540f8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.339+0000 7fc1537fe700 1 -- 172.21.15.47:0/3584697987 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc13c005040 con 0x7fc1540f7400 2024-02-23T00:47:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.339+0000 7fc152ffd700 1 --2- 172.21.15.47:0/3584697987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1540fcbb0 0x7fc1540f8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:47:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.339+0000 7fc1537fe700 1 --2- 172.21.15.47:0/3584697987 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc1540f7400 0x7fc1540f89b0 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 rx=0x7fc14c0027f0 tx=0x7fc14c002b00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:52.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.339+0000 7fc150ff9700 10 client.?.objecter ms_handle_connect 0x7fc1540f7400 2024-02-23T00:47:52.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.339+0000 7fc150ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:47:52.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.339+0000 7fc150ff9700 1 -- 172.21.15.47:0/3584697987 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c007b40 con 0x7fc1540f7400 2024-02-23T00:47:52.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.339+0000 7fc159d72700 1 -- 172.21.15.47:0/3584697987 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1440097c0 con 0x7fc1540f7400 2024-02-23T00:47:52.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.339+0000 7fc150ff9700 1 -- 172.21.15.47:0/3584697987 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fc14c00f040 con 0x7fc1540f7400 2024-02-23T00:47:52.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.339+0000 7fc150ff9700 1 -- 172.21.15.47:0/3584697987 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c014610 con 0x7fc1540f7400 2024-02-23T00:47:52.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.339+0000 7fc159d72700 10 client.41300.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:47:52.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.339+0000 7fc159d72700 1 -- 172.21.15.47:0/3584697987 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc154116c80 con 0x7fc1540f7400 2024-02-23T00:47:52.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.339+0000 7fc159d72700 1 librados: init done 2024-02-23T00:47:52.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.340+0000 7fc159d72700 1 -- 172.21.15.47:0/3584697987 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc15405fa60 con 0x7fc1540f7400 2024-02-23T00:47:52.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.340+0000 7fc150ff9700 1 -- 172.21.15.47:0/3584697987 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fc14c009110 con 0x7fc1540f7400 2024-02-23T00:47:52.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.341+0000 7fc150ff9700 1 --2- 172.21.15.47:0/3584697987 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc140062cc0 0x7fc140065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:52.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.341+0000 7fc150ff9700 1 -- 172.21.15.47:0/3584697987 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(399..399 src has 48..399) v4 ==== 8819+0+0 (secure 0 0 0) 0x7fc14c07d280 con 0x7fc1540f7400 2024-02-23T00:47:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.341+0000 7fc150ff9700 10 client.41300.objecter ms_dispatch 0x7fc154001550 osd_map(399..399 src has 48..399) v4 2024-02-23T00:47:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.341+0000 7fc150ff9700 3 client.41300.objecter handle_osd_map got epochs [399,399] > 0 2024-02-23T00:47:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.341+0000 7fc150ff9700 3 client.41300.objecter handle_osd_map decoding full epoch 399 2024-02-23T00:47:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.341+0000 7fc152ffd700 1 --2- 172.21.15.47:0/3584697987 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc140062cc0 0x7fc140065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.341+0000 7fc150ff9700 20 client.41300.objecter dump_active .. 0 homeless 2024-02-23T00:47:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.341+0000 7fc150ff9700 10 client.41300.objecter ms_handle_connect 0x7fc140062cc0 2024-02-23T00:47:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.341+0000 7fc152ffd700 1 --2- 172.21.15.47:0/3584697987 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc140062cc0 0x7fc140065180 secure :-1 s=READY pgs=3058 cs=0 l=1 rev1=1 rx=0x7fc1540fa150 tx=0x7fc13c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.345+0000 7fc150ff9700 1 -- 172.21.15.47:0/3584697987 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fc14c04ca20 con 0x7fc1540f7400 2024-02-23T00:47:52.390 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:47:52.393 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:52.394 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:47:52.394 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:47:52.495 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:47:52.528 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:52.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:52.559+0000 7fc159d72700 1 -- 172.21.15.47:0/3584697987 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7fc1540f9e90 con 0x7fc1540f7400 2024-02-23T00:47:52.754 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:51.281715+0000 mgr.y (mgr.14517) 367 : cluster [DBG] pgmap v573: 210 pgs: 1 peering, 1 active+remapped+backfill_toofull, 1 activating, 207 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/1125 objects misplaced (0.089%); 21 B/s, 9 objects/s recovering 2024-02-23T00:47:52.754 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:51.525254+0000 mon.a (mon.0) 2506 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-23T00:47:52.754 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:51.533406+0000 mon.a (mon.0) 2507 : cluster [DBG] osdmap e399: 8 total, 8 up, 7 in 2024-02-23T00:47:52.754 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:51.981536+0000 mon.a (mon.0) 2508 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:47:52.754 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:51.982583+0000 mon.a (mon.0) 2509 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:47:52.754 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:51.988736+0000 mon.a (mon.0) 2510 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:47:52.754 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:51.281715+0000 mgr.y (mgr.14517) 367 : cluster [DBG] pgmap v573: 210 pgs: 1 peering, 1 active+remapped+backfill_toofull, 1 activating, 207 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/1125 objects misplaced (0.089%); 21 B/s, 9 objects/s recovering 2024-02-23T00:47:52.755 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:51.525254+0000 mon.a (mon.0) 2506 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-23T00:47:52.755 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:51.533406+0000 mon.a (mon.0) 2507 : cluster [DBG] osdmap e399: 8 total, 8 up, 7 in 2024-02-23T00:47:52.755 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:51.981536+0000 mon.a (mon.0) 2508 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:47:52.755 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:51.982583+0000 mon.a (mon.0) 2509 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:47:52.755 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:51.988736+0000 mon.a (mon.0) 2510 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:47:52.944 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:47:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:47:52.944 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:47:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:47:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:47:52.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:51.281715+0000 mgr.y (mgr.14517) 367 : cluster [DBG] pgmap v573: 210 pgs: 1 peering, 1 active+remapped+backfill_toofull, 1 activating, 207 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/1125 objects misplaced (0.089%); 21 B/s, 9 objects/s recovering 2024-02-23T00:47:52.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:51.525254+0000 mon.a (mon.0) 2506 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-23T00:47:52.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:51.533406+0000 mon.a (mon.0) 2507 : cluster [DBG] osdmap e399: 8 total, 8 up, 7 in 2024-02-23T00:47:52.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:51.981536+0000 mon.a (mon.0) 2508 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:47:52.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:51.982583+0000 mon.a (mon.0) 2509 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:47:52.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:51.988736+0000 mon.a (mon.0) 2510 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:47:53.188 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:47:53.188 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:53.189 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:47:53.290 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:47:53.321 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:53.546 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:52.539119+0000 mon.a (mon.0) 2511 : cluster [DBG] osdmap e400: 8 total, 8 up, 7 in 2024-02-23T00:47:53.546 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:52.548632+0000 mon.a (mon.0) 2512 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:47:53.546 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:52.560726+0000 mgr.y (mgr.14517) 368 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:47:53.546 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:52.561031+0000 mon.c (mon.2) 296 : audit [INF] from='client.? 172.21.15.47:0/3584697987' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:47:53.546 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:52.561335+0000 mon.a (mon.0) 2513 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:47:53.546 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:53.283288+0000 mon.a (mon.0) 2514 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "12"}]: dispatch 2024-02-23T00:47:53.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:53.549+0000 7fc150ff9700 1 -- 172.21.15.47:0/3584697987 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v401) v1 ==== 91+0+0 (secure 0 0 0) 0x7fc14c050040 con 0x7fc1540f7400 2024-02-23T00:47:53.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:53.553+0000 7fc159d72700 10 librados: watch_flush enter 2024-02-23T00:47:53.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:53.553+0000 7fc159d72700 10 librados: watch_flush exit 2024-02-23T00:47:53.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:53.553+0000 7fc159d72700 20 client.41300.objecter shutdown clearing up homeless session... 2024-02-23T00:47:53.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:53.553+0000 7fc159d72700 10 client.41300.objecter successfully canceled tick 2024-02-23T00:47:53.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:53.556+0000 7fc159d72700 1 -- 172.21.15.47:0/3584697987 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc140062cc0 msgr2=0x7fc140065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:53.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:53.556+0000 7fc159d72700 1 --2- 172.21.15.47:0/3584697987 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc140062cc0 0x7fc140065180 secure :-1 s=READY pgs=3058 cs=0 l=1 rev1=1 rx=0x7fc1540fa150 tx=0x7fc13c00b040).stop 2024-02-23T00:47:53.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:53.556+0000 7fc159d72700 1 -- 172.21.15.47:0/3584697987 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc1540f7400 msgr2=0x7fc1540f89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:53.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:53.556+0000 7fc159d72700 1 --2- 172.21.15.47:0/3584697987 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc1540f7400 0x7fc1540f89b0 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 rx=0x7fc14c0027f0 tx=0x7fc14c002b00).stop 2024-02-23T00:47:53.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:53.556+0000 7fc159d72700 1 -- 172.21.15.47:0/3584697987 shutdown_connections 2024-02-23T00:47:53.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:53.556+0000 7fc159d72700 1 --2- 172.21.15.47:0/3584697987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1540fd510 0x7fc1540f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:53.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:53.556+0000 7fc159d72700 1 --2- 172.21.15.47:0/3584697987 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc140062cc0 0x7fc140065180 unknown :-1 s=CLOSED pgs=3058 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:53.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:53.556+0000 7fc159d72700 1 --2- 172.21.15.47:0/3584697987 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc1540f7400 0x7fc1540f89b0 unknown :-1 s=CLOSED pgs=2658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:53.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:53.556+0000 7fc159d72700 1 --2- 172.21.15.47:0/3584697987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1540fcbb0 0x7fc1540f8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:53.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:53.556+0000 7fc159d72700 1 -- 172.21.15.47:0/3584697987 >> 172.21.15.47:0/3584697987 conn(0x7fc1540f2820 msgr2=0x7fc154060dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:53.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:53.556+0000 7fc159d72700 1 -- 172.21.15.47:0/3584697987 shutdown_connections 2024-02-23T00:47:53.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:53.556+0000 7fc159d72700 1 -- 172.21.15.47:0/3584697987 wait complete. 2024-02-23T00:47:53.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:53.556+0000 7fc159d72700 1 librados: shutdown 2024-02-23T00:47:53.560 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is unset 2024-02-23T00:47:53.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:52.539119+0000 mon.a (mon.0) 2511 : cluster [DBG] osdmap e400: 8 total, 8 up, 7 in 2024-02-23T00:47:53.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:52.548632+0000 mon.a (mon.0) 2512 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:47:53.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:52.560726+0000 mgr.y (mgr.14517) 368 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:47:53.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:52.561031+0000 mon.c (mon.2) 296 : audit [INF] from='client.? 172.21.15.47:0/3584697987' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:47:53.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:52.561335+0000 mon.a (mon.0) 2513 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:47:53.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:53.283288+0000 mon.a (mon.0) 2514 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "12"}]: dispatch 2024-02-23T00:47:53.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:52.539119+0000 mon.a (mon.0) 2511 : cluster [DBG] osdmap e400: 8 total, 8 up, 7 in 2024-02-23T00:47:53.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:52.548632+0000 mon.a (mon.0) 2512 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:47:53.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:52.560726+0000 mgr.y (mgr.14517) 368 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:47:53.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:52.561031+0000 mon.c (mon.2) 296 : audit [INF] from='client.? 172.21.15.47:0/3584697987' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:47:53.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:52.561335+0000 mon.a (mon.0) 2513 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:47:53.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:53.283288+0000 mon.a (mon.0) 2514 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "12"}]: dispatch 2024-02-23T00:47:54.371 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:47:54.374 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:54.375 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:47:54.476 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:47:54.508 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:54.530 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:47:54.533 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:54.533 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:47:54.634 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:47:54.660 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:53.282411+0000 mgr.y (mgr.14517) 369 : cluster [DBG] pgmap v576: 209 pgs: 32 unknown, 1 peering, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 255 B/s rd, 0 op/s; 92 B/s, 0 objects/s recovering 2024-02-23T00:47:54.660 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:53.536646+0000 mon.a (mon.0) 2515 : cluster [INF] Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-02-23T00:47:54.660 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:53.537666+0000 mon.a (mon.0) 2516 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:47:54.660 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:53.545853+0000 mon.a (mon.0) 2517 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:47:54.660 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:53.545953+0000 mon.a (mon.0) 2518 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:47:54.660 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:53.546022+0000 mon.a (mon.0) 2519 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "12"}]': finished 2024-02-23T00:47:54.660 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:53.546076+0000 mon.a (mon.0) 2520 : cluster [DBG] osdmap e401: 8 total, 8 up, 7 in 2024-02-23T00:47:54.660 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:53.586332+0000 mon.a (mon.0) 2521 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:47:54.661 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:53.586693+0000 mon.a (mon.0) 2522 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-51"}]: dispatch 2024-02-23T00:47:54.661 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:53.668554+0000 osd.1 (osd.1) 167 : cluster [DBG] 3.0 scrub starts 2024-02-23T00:47:54.661 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:53.669528+0000 osd.1 (osd.1) 168 : cluster [DBG] 3.0 scrub ok 2024-02-23T00:47:54.661 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:54.033958+0000 osd.3 (osd.3) 182 : cluster [DBG] 3.2 scrub starts 2024-02-23T00:47:54.661 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:54.034832+0000 osd.3 (osd.3) 183 : cluster [DBG] 3.2 scrub ok 2024-02-23T00:47:54.661 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:53.282411+0000 mgr.y (mgr.14517) 369 : cluster [DBG] pgmap v576: 209 pgs: 32 unknown, 1 peering, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 255 B/s rd, 0 op/s; 92 B/s, 0 objects/s recovering 2024-02-23T00:47:54.661 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:53.536646+0000 mon.a (mon.0) 2515 : cluster [INF] Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-02-23T00:47:54.661 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:53.537666+0000 mon.a (mon.0) 2516 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:47:54.661 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:53.545853+0000 mon.a (mon.0) 2517 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:47:54.661 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:53.545953+0000 mon.a (mon.0) 2518 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:47:54.661 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:53.546022+0000 mon.a (mon.0) 2519 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "12"}]': finished 2024-02-23T00:47:54.662 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:53.546076+0000 mon.a (mon.0) 2520 : cluster [DBG] osdmap e401: 8 total, 8 up, 7 in 2024-02-23T00:47:54.662 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:53.586332+0000 mon.a (mon.0) 2521 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:47:54.662 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:53.586693+0000 mon.a (mon.0) 2522 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-51"}]: dispatch 2024-02-23T00:47:54.662 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:53.668554+0000 osd.1 (osd.1) 167 : cluster [DBG] 3.0 scrub starts 2024-02-23T00:47:54.662 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:53.669528+0000 osd.1 (osd.1) 168 : cluster [DBG] 3.0 scrub ok 2024-02-23T00:47:54.662 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:54.033958+0000 osd.3 (osd.3) 182 : cluster [DBG] 3.2 scrub starts 2024-02-23T00:47:54.662 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:54.034832+0000 osd.3 (osd.3) 183 : cluster [DBG] 3.2 scrub ok 2024-02-23T00:47:54.662 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:54.790 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:47:54.792 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:54.793 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:47:54.894 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:47:54.920 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:53.282411+0000 mgr.y (mgr.14517) 369 : cluster [DBG] pgmap v576: 209 pgs: 32 unknown, 1 peering, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 255 B/s rd, 0 op/s; 92 B/s, 0 objects/s recovering 2024-02-23T00:47:54.920 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:53.536646+0000 mon.a (mon.0) 2515 : cluster [INF] Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-02-23T00:47:54.920 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:53.537666+0000 mon.a (mon.0) 2516 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:47:54.920 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:53.545853+0000 mon.a (mon.0) 2517 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:47:54.921 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:53.545953+0000 mon.a (mon.0) 2518 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:47:54.921 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:53.546022+0000 mon.a (mon.0) 2519 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "12"}]': finished 2024-02-23T00:47:54.921 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:53.546076+0000 mon.a (mon.0) 2520 : cluster [DBG] osdmap e401: 8 total, 8 up, 7 in 2024-02-23T00:47:54.921 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:53.586332+0000 mon.a (mon.0) 2521 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:47:54.921 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:53.586693+0000 mon.a (mon.0) 2522 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-51"}]: dispatch 2024-02-23T00:47:54.921 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:53.668554+0000 osd.1 (osd.1) 167 : cluster [DBG] 3.0 scrub starts 2024-02-23T00:47:54.921 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:53.669528+0000 osd.1 (osd.1) 168 : cluster [DBG] 3.0 scrub ok 2024-02-23T00:47:54.921 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:54.033958+0000 osd.3 (osd.3) 182 : cluster [DBG] 3.2 scrub starts 2024-02-23T00:47:54.921 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:54.034832+0000 osd.3 (osd.3) 183 : cluster [DBG] 3.2 scrub ok 2024-02-23T00:47:54.922 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:54.941 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:47:54.943 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:54.944 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:47:55.045 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:47:55.075 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:55.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:54.428638+0000 osd.6 (osd.6) 157 : cluster [DBG] 62.7 scrub starts 2024-02-23T00:47:55.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:54.430462+0000 osd.6 (osd.6) 158 : cluster [DBG] 62.7 scrub ok 2024-02-23T00:47:55.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:54.551198+0000 mon.a (mon.0) 2523 : cluster [DBG] osdmap e402: 8 total, 8 up, 7 in 2024-02-23T00:47:55.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:55.177243+0000 mon.a (mon.0) 2524 : cluster [DBG] osdmap e403: 8 total, 8 up, 7 in 2024-02-23T00:47:55.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:55.181330+0000 mon.a (mon.0) 2525 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:47:55.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:54.428638+0000 osd.6 (osd.6) 157 : cluster [DBG] 62.7 scrub starts 2024-02-23T00:47:55.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:54.430462+0000 osd.6 (osd.6) 158 : cluster [DBG] 62.7 scrub ok 2024-02-23T00:47:55.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:54.551198+0000 mon.a (mon.0) 2523 : cluster [DBG] osdmap e402: 8 total, 8 up, 7 in 2024-02-23T00:47:55.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:55.177243+0000 mon.a (mon.0) 2524 : cluster [DBG] osdmap e403: 8 total, 8 up, 7 in 2024-02-23T00:47:55.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:55.181330+0000 mon.a (mon.0) 2525 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:47:55.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.869+0000 7f4c279e7700 1 -- 172.21.15.47:0/3696265892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c200f7300 msgr2=0x7f4c201040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:55.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.869+0000 7f4c279e7700 1 --2- 172.21.15.47:0/3696265892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c200f7300 0x7f4c201040a0 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 rx=0x7f4c100047e0 tx=0x7f4c10009cb0).stop 2024-02-23T00:47:55.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.869+0000 7f4c279e7700 1 -- 172.21.15.47:0/3696265892 shutdown_connections 2024-02-23T00:47:55.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.869+0000 7f4c279e7700 1 --2- 172.21.15.47:0/3696265892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c20104590 0x7f4c20106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:55.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.869+0000 7f4c279e7700 1 --2- 172.21.15.47:0/3696265892 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4c200f6a30 0x7f4c200f6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:55.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.869+0000 7f4c279e7700 1 --2- 172.21.15.47:0/3696265892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c200f7300 0x7f4c201040a0 unknown :-1 s=CLOSED pgs=2527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:55.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.869+0000 7f4c279e7700 1 -- 172.21.15.47:0/3696265892 >> 172.21.15.47:0/3696265892 conn(0x7f4c200f26f0 msgr2=0x7f4c200f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:55.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.869+0000 7f4c279e7700 1 -- 172.21.15.47:0/3696265892 shutdown_connections 2024-02-23T00:47:55.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.869+0000 7f4c279e7700 1 -- 172.21.15.47:0/3696265892 wait complete. 2024-02-23T00:47:55.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.869+0000 7f4c279e7700 1 librados: starting msgr at 2024-02-23T00:47:55.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.869+0000 7f4c279e7700 1 librados: starting objecter 2024-02-23T00:47:55.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.869+0000 7f4c279e7700 1 Processor -- start 2024-02-23T00:47:55.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.869+0000 7f4c279e7700 1 -- start start 2024-02-23T00:47:55.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.869+0000 7f4c279e7700 1 librados: setting wanted keys 2024-02-23T00:47:55.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.869+0000 7f4c279e7700 1 librados: calling monclient init 2024-02-23T00:47:55.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.870+0000 7f4c279e7700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c200f6a30 0x7f4c20110130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:55.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.870+0000 7f4c279e7700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4c200f7300 0x7f4c20110620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:55.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.870+0000 7f4c279e7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c20104590 0x7f4c2010a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:55.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.870+0000 7f4c279e7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2010a7c0 con 0x7f4c20104590 2024-02-23T00:47:55.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.870+0000 7f4c279e7700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2010a930 con 0x7f4c200f6a30 2024-02-23T00:47:55.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.870+0000 7f4c279e7700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2010aaf0 con 0x7f4c200f7300 2024-02-23T00:47:55.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.870+0000 7f4c24f82700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4c200f7300 0x7f4c20110620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:55.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.870+0000 7f4c25f84700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c20104590 0x7f4c2010a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:55.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.870+0000 7f4c24f82700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4c200f7300 0x7f4c20110620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:42762/0 (socket says 172.21.15.47:42762) 2024-02-23T00:47:55.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.870+0000 7f4c25783700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c200f6a30 0x7f4c20110130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:55.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.870+0000 7f4c24f82700 1 -- 172.21.15.47:0/2493717765 learned_addr learned my addr 172.21.15.47:0/2493717765 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:47:55.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.870+0000 7f4c24f82700 1 -- 172.21.15.47:0/2493717765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c200f6a30 msgr2=0x7f4c20110130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:55.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.870+0000 7f4c24f82700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c200f6a30 0x7f4c20110130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:55.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.870+0000 7f4c24f82700 1 -- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c20104590 msgr2=0x7f4c2010a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:55.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.870+0000 7f4c24f82700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c20104590 0x7f4c2010a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:55.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.870+0000 7f4c24f82700 1 -- 172.21.15.47:0/2493717765 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c1c005040 con 0x7f4c200f7300 2024-02-23T00:47:55.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.870+0000 7f4c25f84700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c20104590 0x7f4c2010a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:47:55.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.870+0000 7f4c24f82700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4c200f7300 0x7f4c20110620 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 rx=0x7f4c10007fd0 tx=0x7f4c10009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:55.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.870+0000 7f4c167fc700 10 client.?.objecter ms_handle_connect 0x7f4c200f7300 2024-02-23T00:47:55.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.870+0000 7f4c167fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:47:55.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.870+0000 7f4c167fc700 1 -- 172.21.15.47:0/2493717765 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c10018070 con 0x7f4c200f7300 2024-02-23T00:47:55.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.870+0000 7f4c167fc700 1 -- 172.21.15.47:0/2493717765 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f4c10013e30 con 0x7f4c200f7300 2024-02-23T00:47:55.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.870+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c10005040 con 0x7f4c200f7300 2024-02-23T00:47:55.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.870+0000 7f4c167fc700 1 -- 172.21.15.47:0/2493717765 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1001dac0 con 0x7f4c200f7300 2024-02-23T00:47:55.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.871+0000 7f4c279e7700 10 client.41306.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:47:55.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.871+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c2010af90 con 0x7f4c200f7300 2024-02-23T00:47:55.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.871+0000 7f4c279e7700 1 librados: init done 2024-02-23T00:47:55.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.871+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f4c2010b180 con 0x7f4c200f7300 2024-02-23T00:47:55.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.872+0000 7f4c167fc700 1 -- 172.21.15.47:0/2493717765 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f4c10009240 con 0x7f4c200f7300 2024-02-23T00:47:55.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.872+0000 7f4c167fc700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4c0c062cc0 0x7f4c0c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:55.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.872+0000 7f4c167fc700 1 -- 172.21.15.47:0/2493717765 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(403..403 src has 48..403) v4 ==== 9191+0+0 (secure 0 0 0) 0x7f4c1007f120 con 0x7f4c200f7300 2024-02-23T00:47:55.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.872+0000 7f4c167fc700 10 client.41306.objecter ms_dispatch 0x7f4c20001550 osd_map(403..403 src has 48..403) v4 2024-02-23T00:47:55.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.872+0000 7f4c167fc700 3 client.41306.objecter handle_osd_map got epochs [403,403] > 0 2024-02-23T00:47:55.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.872+0000 7f4c167fc700 3 client.41306.objecter handle_osd_map decoding full epoch 403 2024-02-23T00:47:55.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.872+0000 7f4c25783700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4c0c062cc0 0x7f4c0c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:55.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.872+0000 7f4c167fc700 20 client.41306.objecter dump_active .. 0 homeless 2024-02-23T00:47:55.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.872+0000 7f4c167fc700 1 -- 172.21.15.47:0/2493717765 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v403) v1 ==== 54+0+16 (secure 0 0 0) 0x7f4c1007f5c0 con 0x7f4c200f7300 2024-02-23T00:47:55.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.873+0000 7f4c167fc700 10 client.41306.objecter ms_handle_connect 0x7f4c0c062cc0 2024-02-23T00:47:55.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.873+0000 7f4c25783700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4c0c062cc0 0x7f4c0c065180 secure :-1 s=READY pgs=3059 cs=0 l=1 rev1=1 rx=0x7f4c1c0047e0 tx=0x7f4c1c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:55.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.873+0000 7f4c279e7700 10 client.41306.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:55.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.873+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_assign -1 1 2024-02-23T00:47:55.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.873+0000 7f4c279e7700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f4c200fe570 0x7f4c200fe950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:55.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.873+0000 7f4c279e7700 20 client.41306.objecter _get_session s=0x7f4c2005f990 osd=0 3 2024-02-23T00:47:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.873+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c2005f990 osd=0 3 2024-02-23T00:47:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.873+0000 7f4c279e7700 20 client.41306.objecter _get_session s=0x7f4c2005f990 osd=0 3 2024-02-23T00:47:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.873+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_remove -1 1 2024-02-23T00:47:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.873+0000 7f4c279e7700 20 client.41306.objecter get_session s=0x7f4c2005f990 osd=0 3 2024-02-23T00:47:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.873+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_assign 0 1 2024-02-23T00:47:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.873+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c2005f990 osd=0 4 2024-02-23T00:47:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.873+0000 7f4c279e7700 10 client.41306.objecter _send_command 1 2024-02-23T00:47:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.873+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c2005ee80 con 0x7f4c200fe570 2024-02-23T00:47:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.873+0000 7f4c25f84700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f4c200fe570 0x7f4c200fe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.873+0000 7f4c167fc700 10 client.41306.objecter ms_handle_connect 0x7f4c200fe570 2024-02-23T00:47:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.873+0000 7f4c25f84700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f4c200fe570 0x7f4c200fe950 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.874+0000 7f4c167fc700 1 -- 172.21.15.47:0/2493717765 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4c2005ee80 con 0x7f4c200fe570 2024-02-23T00:47:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.874+0000 7f4c167fc700 10 client.41306.objecter ms_dispatch 0x7f4c20001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:47:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.874+0000 7f4c167fc700 10 client.41306.objecter _finish_command 1 = system:0 2024-02-23T00:47:55.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.874+0000 7f4c167fc700 20 client.41306.objecter put_session s=0x7f4c2005f990 osd=0 4 2024-02-23T00:47:55.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.874+0000 7f4c167fc700 15 client.41306.objecter _session_command_op_remove 0 1 2024-02-23T00:47:55.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.889+0000 7f4c279e7700 10 client.41306.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:47:55.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.889+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_assign -1 2 2024-02-23T00:47:55.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.889+0000 7f4c279e7700 20 client.41306.objecter _get_session s=0x7f4c2005f990 osd=0 3 2024-02-23T00:47:55.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.889+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c2005f990 osd=0 3 2024-02-23T00:47:55.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.889+0000 7f4c279e7700 20 client.41306.objecter _get_session s=0x7f4c2005f990 osd=0 3 2024-02-23T00:47:55.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.889+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_remove -1 2 2024-02-23T00:47:55.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.889+0000 7f4c279e7700 20 client.41306.objecter get_session s=0x7f4c2005f990 osd=0 3 2024-02-23T00:47:55.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.889+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_assign 0 2 2024-02-23T00:47:55.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.889+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c2005f990 osd=0 4 2024-02-23T00:47:55.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.889+0000 7f4c279e7700 10 client.41306.objecter _send_command 2 2024-02-23T00:47:55.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.889+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f4c201090b0 con 0x7f4c200fe570 2024-02-23T00:47:55.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.890+0000 7f4c167fc700 1 -- 172.21.15.47:0/2493717765 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f4c201090b0 con 0x7f4c200fe570 2024-02-23T00:47:55.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.890+0000 7f4c167fc700 10 client.41306.objecter ms_dispatch 0x7f4c20001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:47:55.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.890+0000 7f4c167fc700 10 client.41306.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:47:55.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.890+0000 7f4c167fc700 20 client.41306.objecter put_session s=0x7f4c2005f990 osd=0 4 2024-02-23T00:47:55.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.890+0000 7f4c167fc700 15 client.41306.objecter _session_command_op_remove 0 2 2024-02-23T00:47:55.898 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:47:55.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.890+0000 7f4c279e7700 10 client.41306.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:55.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.890+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_assign -1 3 2024-02-23T00:47:55.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.890+0000 7f4c279e7700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f4c200598f0 0x7f4c20059d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:55.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.890+0000 7f4c279e7700 20 client.41306.objecter _get_session s=0x7f4c20028540 osd=1 3 2024-02-23T00:47:55.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.890+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c20028540 osd=1 3 2024-02-23T00:47:55.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.890+0000 7f4c279e7700 20 client.41306.objecter _get_session s=0x7f4c20028540 osd=1 3 2024-02-23T00:47:55.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.890+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_remove -1 3 2024-02-23T00:47:55.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.890+0000 7f4c279e7700 20 client.41306.objecter get_session s=0x7f4c20028540 osd=1 3 2024-02-23T00:47:55.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.890+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_assign 1 3 2024-02-23T00:47:55.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.890+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c20028540 osd=1 4 2024-02-23T00:47:55.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.890+0000 7f4c279e7700 10 client.41306.objecter _send_command 3 2024-02-23T00:47:55.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.890+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c200286b0 con 0x7f4c200598f0 2024-02-23T00:47:55.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.891+0000 7f4c25783700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f4c200598f0 0x7f4c20059d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:55.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.891+0000 7f4c167fc700 10 client.41306.objecter ms_handle_connect 0x7f4c200598f0 2024-02-23T00:47:55.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.891+0000 7f4c25783700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f4c200598f0 0x7f4c20059d10 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:55.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.891+0000 7f4c167fc700 1 -- 172.21.15.47:0/2493717765 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4c200286b0 con 0x7f4c200598f0 2024-02-23T00:47:55.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.891+0000 7f4c167fc700 10 client.41306.objecter ms_dispatch 0x7f4c20001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:47:55.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.891+0000 7f4c167fc700 10 client.41306.objecter _finish_command 3 = system:0 2024-02-23T00:47:55.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.891+0000 7f4c167fc700 20 client.41306.objecter put_session s=0x7f4c20028540 osd=1 4 2024-02-23T00:47:55.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.891+0000 7f4c167fc700 15 client.41306.objecter _session_command_op_remove 1 3 2024-02-23T00:47:55.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.906+0000 7f4c279e7700 10 client.41306.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:47:55.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.906+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_assign -1 4 2024-02-23T00:47:55.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.906+0000 7f4c279e7700 20 client.41306.objecter _get_session s=0x7f4c20028540 osd=1 3 2024-02-23T00:47:55.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.906+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c20028540 osd=1 3 2024-02-23T00:47:55.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.906+0000 7f4c279e7700 20 client.41306.objecter _get_session s=0x7f4c20028540 osd=1 3 2024-02-23T00:47:55.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.906+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_remove -1 4 2024-02-23T00:47:55.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.906+0000 7f4c279e7700 20 client.41306.objecter get_session s=0x7f4c20028540 osd=1 3 2024-02-23T00:47:55.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.906+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_assign 1 4 2024-02-23T00:47:55.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.906+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c20028540 osd=1 4 2024-02-23T00:47:55.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.906+0000 7f4c279e7700 10 client.41306.objecter _send_command 4 2024-02-23T00:47:55.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.906+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f4c20028820 con 0x7f4c200598f0 2024-02-23T00:47:55.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.908+0000 7f4c167fc700 1 -- 172.21.15.47:0/2493717765 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f4c1c00f040 con 0x7f4c200598f0 2024-02-23T00:47:55.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.908+0000 7f4c167fc700 10 client.41306.objecter ms_dispatch 0x7f4c20001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:47:55.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.908+0000 7f4c167fc700 10 client.41306.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:47:55.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.908+0000 7f4c167fc700 20 client.41306.objecter put_session s=0x7f4c20028540 osd=1 4 2024-02-23T00:47:55.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.908+0000 7f4c167fc700 15 client.41306.objecter _session_command_op_remove 1 4 2024-02-23T00:47:55.911 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:47:55.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.908+0000 7f4c279e7700 10 client.41306.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:55.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.908+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_assign -1 5 2024-02-23T00:47:55.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.908+0000 7f4c279e7700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f4c200280a0 0x7f4c2011e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:55.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.908+0000 7f4c279e7700 20 client.41306.objecter _get_session s=0x7f4c200595a0 osd=2 3 2024-02-23T00:47:55.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.908+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c200595a0 osd=2 3 2024-02-23T00:47:55.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.908+0000 7f4c279e7700 20 client.41306.objecter _get_session s=0x7f4c200595a0 osd=2 3 2024-02-23T00:47:55.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.908+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_remove -1 5 2024-02-23T00:47:55.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.908+0000 7f4c279e7700 20 client.41306.objecter get_session s=0x7f4c200595a0 osd=2 3 2024-02-23T00:47:55.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.908+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_assign 2 5 2024-02-23T00:47:55.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.908+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c200595a0 osd=2 4 2024-02-23T00:47:55.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.908+0000 7f4c279e7700 10 client.41306.objecter _send_command 5 2024-02-23T00:47:55.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.908+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c20120d60 con 0x7f4c200280a0 2024-02-23T00:47:55.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.908+0000 7f4c24f82700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f4c200280a0 0x7f4c2011e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:55.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.909+0000 7f4c167fc700 10 client.41306.objecter ms_handle_connect 0x7f4c200280a0 2024-02-23T00:47:55.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.909+0000 7f4c24f82700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f4c200280a0 0x7f4c2011e110 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:55.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.909+0000 7f4c167fc700 1 -- 172.21.15.47:0/2493717765 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4c20120d60 con 0x7f4c200280a0 2024-02-23T00:47:55.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.909+0000 7f4c167fc700 10 client.41306.objecter ms_dispatch 0x7f4c20001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:47:55.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.909+0000 7f4c167fc700 10 client.41306.objecter _finish_command 5 = system:0 2024-02-23T00:47:55.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.909+0000 7f4c167fc700 20 client.41306.objecter put_session s=0x7f4c200595a0 osd=2 4 2024-02-23T00:47:55.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.909+0000 7f4c167fc700 15 client.41306.objecter _session_command_op_remove 2 5 2024-02-23T00:47:55.925 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:47:55.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.924+0000 7f4c279e7700 10 client.41306.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:47:55.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.924+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_assign -1 6 2024-02-23T00:47:55.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.924+0000 7f4c279e7700 20 client.41306.objecter _get_session s=0x7f4c200595a0 osd=2 3 2024-02-23T00:47:55.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.924+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c200595a0 osd=2 3 2024-02-23T00:47:55.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.924+0000 7f4c279e7700 20 client.41306.objecter _get_session s=0x7f4c200595a0 osd=2 3 2024-02-23T00:47:55.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.924+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_remove -1 6 2024-02-23T00:47:55.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.924+0000 7f4c279e7700 20 client.41306.objecter get_session s=0x7f4c200595a0 osd=2 3 2024-02-23T00:47:55.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.924+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_assign 2 6 2024-02-23T00:47:55.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.924+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c200595a0 osd=2 4 2024-02-23T00:47:55.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.924+0000 7f4c279e7700 10 client.41306.objecter _send_command 6 2024-02-23T00:47:55.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.924+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f4c20027660 con 0x7f4c200280a0 2024-02-23T00:47:55.928 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:55.929 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:47:55.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.926+0000 7f4c167fc700 1 -- 172.21.15.47:0/2493717765 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f4c20027660 con 0x7f4c200280a0 2024-02-23T00:47:55.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.926+0000 7f4c167fc700 10 client.41306.objecter ms_dispatch 0x7f4c20001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:47:55.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.926+0000 7f4c167fc700 10 client.41306.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:47:55.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.926+0000 7f4c167fc700 20 client.41306.objecter put_session s=0x7f4c200595a0 osd=2 4 2024-02-23T00:47:55.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.926+0000 7f4c167fc700 15 client.41306.objecter _session_command_op_remove 2 6 2024-02-23T00:47:55.929 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:47:55.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.926+0000 7f4c279e7700 10 client.41306.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:55.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.926+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_assign -1 7 2024-02-23T00:47:55.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.926+0000 7f4c279e7700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f4c2011f780 0x7f4c2011fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:55.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.926+0000 7f4c279e7700 20 client.41306.objecter _get_session s=0x7f4c2011b750 osd=3 3 2024-02-23T00:47:55.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.926+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c2011b750 osd=3 3 2024-02-23T00:47:55.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.926+0000 7f4c279e7700 20 client.41306.objecter _get_session s=0x7f4c2011b750 osd=3 3 2024-02-23T00:47:55.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.926+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_remove -1 7 2024-02-23T00:47:55.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.926+0000 7f4c279e7700 20 client.41306.objecter get_session s=0x7f4c2011b750 osd=3 3 2024-02-23T00:47:55.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.926+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_assign 3 7 2024-02-23T00:47:55.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.926+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c2011b750 osd=3 4 2024-02-23T00:47:55.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.926+0000 7f4c279e7700 10 client.41306.objecter _send_command 7 2024-02-23T00:47:55.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.926+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c2011bee0 con 0x7f4c2011f780 2024-02-23T00:47:55.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.926+0000 7f4c25f84700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f4c2011f780 0x7f4c2011fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:55.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.927+0000 7f4c167fc700 10 client.41306.objecter ms_handle_connect 0x7f4c2011f780 2024-02-23T00:47:55.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.927+0000 7f4c25f84700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f4c2011f780 0x7f4c2011fbc0 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:55.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.927+0000 7f4c167fc700 1 -- 172.21.15.47:0/2493717765 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4c2011bee0 con 0x7f4c2011f780 2024-02-23T00:47:55.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.927+0000 7f4c167fc700 10 client.41306.objecter ms_dispatch 0x7f4c20001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:47:55.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.927+0000 7f4c167fc700 10 client.41306.objecter _finish_command 7 = system:0 2024-02-23T00:47:55.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.927+0000 7f4c167fc700 20 client.41306.objecter put_session s=0x7f4c2011b750 osd=3 4 2024-02-23T00:47:55.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.927+0000 7f4c167fc700 15 client.41306.objecter _session_command_op_remove 3 7 2024-02-23T00:47:55.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:54.428638+0000 osd.6 (osd.6) 157 : cluster [DBG] 62.7 scrub starts 2024-02-23T00:47:55.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:54.430462+0000 osd.6 (osd.6) 158 : cluster [DBG] 62.7 scrub ok 2024-02-23T00:47:55.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:54.551198+0000 mon.a (mon.0) 2523 : cluster [DBG] osdmap e402: 8 total, 8 up, 7 in 2024-02-23T00:47:55.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:55.177243+0000 mon.a (mon.0) 2524 : cluster [DBG] osdmap e403: 8 total, 8 up, 7 in 2024-02-23T00:47:55.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:55.181330+0000 mon.a (mon.0) 2525 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:47:55.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.942+0000 7f4c279e7700 10 client.41306.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:47:55.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.942+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_assign -1 8 2024-02-23T00:47:55.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.942+0000 7f4c279e7700 20 client.41306.objecter _get_session s=0x7f4c2011b750 osd=3 3 2024-02-23T00:47:55.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.942+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c2011b750 osd=3 3 2024-02-23T00:47:55.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.942+0000 7f4c279e7700 20 client.41306.objecter _get_session s=0x7f4c2011b750 osd=3 3 2024-02-23T00:47:55.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.942+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_remove -1 8 2024-02-23T00:47:55.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.942+0000 7f4c279e7700 20 client.41306.objecter get_session s=0x7f4c2011b750 osd=3 3 2024-02-23T00:47:55.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.942+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_assign 3 8 2024-02-23T00:47:55.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.942+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c2011b750 osd=3 4 2024-02-23T00:47:55.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.942+0000 7f4c279e7700 10 client.41306.objecter _send_command 8 2024-02-23T00:47:55.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.942+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f4c20120880 con 0x7f4c2011f780 2024-02-23T00:47:55.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.943+0000 7f4c167fc700 1 -- 172.21.15.47:0/2493717765 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f4c20120880 con 0x7f4c2011f780 2024-02-23T00:47:55.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.943+0000 7f4c167fc700 10 client.41306.objecter ms_dispatch 0x7f4c20001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:47:55.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.943+0000 7f4c167fc700 10 client.41306.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:47:55.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.943+0000 7f4c167fc700 20 client.41306.objecter put_session s=0x7f4c2011b750 osd=3 4 2024-02-23T00:47:55.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.943+0000 7f4c167fc700 15 client.41306.objecter _session_command_op_remove 3 8 2024-02-23T00:47:55.946 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:47:55.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.943+0000 7f4c279e7700 10 client.41306.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:55.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.943+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_assign -1 9 2024-02-23T00:47:55.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.943+0000 7f4c279e7700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f4c20122f10 0x7f4c20125300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:55.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.943+0000 7f4c279e7700 20 client.41306.objecter _get_session s=0x7f4c201200b0 osd=4 3 2024-02-23T00:47:55.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.943+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c201200b0 osd=4 3 2024-02-23T00:47:55.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.943+0000 7f4c279e7700 20 client.41306.objecter _get_session s=0x7f4c201200b0 osd=4 3 2024-02-23T00:47:55.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.943+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_remove -1 9 2024-02-23T00:47:55.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.943+0000 7f4c279e7700 20 client.41306.objecter get_session s=0x7f4c201200b0 osd=4 3 2024-02-23T00:47:55.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.943+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_assign 4 9 2024-02-23T00:47:55.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.943+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c201200b0 osd=4 4 2024-02-23T00:47:55.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.943+0000 7f4c279e7700 10 client.41306.objecter _send_command 9 2024-02-23T00:47:55.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.943+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c20120880 con 0x7f4c20122f10 2024-02-23T00:47:55.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.944+0000 7f4c25783700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f4c20122f10 0x7f4c20125300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:55.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.944+0000 7f4c167fc700 10 client.41306.objecter ms_handle_connect 0x7f4c20122f10 2024-02-23T00:47:55.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.944+0000 7f4c25783700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f4c20122f10 0x7f4c20125300 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:55.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.945+0000 7f4c167fc700 1 -- 172.21.15.47:0/2493717765 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4c20120880 con 0x7f4c20122f10 2024-02-23T00:47:55.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.945+0000 7f4c167fc700 10 client.41306.objecter ms_dispatch 0x7f4c20001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:47:55.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.945+0000 7f4c167fc700 10 client.41306.objecter _finish_command 9 = system:0 2024-02-23T00:47:55.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.945+0000 7f4c167fc700 20 client.41306.objecter put_session s=0x7f4c201200b0 osd=4 4 2024-02-23T00:47:55.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.945+0000 7f4c167fc700 15 client.41306.objecter _session_command_op_remove 4 9 2024-02-23T00:47:55.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.959+0000 7f4c279e7700 10 client.41306.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:47:55.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.959+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_assign -1 10 2024-02-23T00:47:55.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.959+0000 7f4c279e7700 20 client.41306.objecter _get_session s=0x7f4c201200b0 osd=4 3 2024-02-23T00:47:55.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.959+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c201200b0 osd=4 3 2024-02-23T00:47:55.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.959+0000 7f4c279e7700 20 client.41306.objecter _get_session s=0x7f4c201200b0 osd=4 3 2024-02-23T00:47:55.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.959+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_remove -1 10 2024-02-23T00:47:55.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.959+0000 7f4c279e7700 20 client.41306.objecter get_session s=0x7f4c201200b0 osd=4 3 2024-02-23T00:47:55.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.959+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_assign 4 10 2024-02-23T00:47:55.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.959+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c201200b0 osd=4 4 2024-02-23T00:47:55.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.959+0000 7f4c279e7700 10 client.41306.objecter _send_command 10 2024-02-23T00:47:55.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.959+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f4c20120ac0 con 0x7f4c20122f10 2024-02-23T00:47:55.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.961+0000 7f4c167fc700 1 -- 172.21.15.47:0/2493717765 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f4c20120ac0 con 0x7f4c20122f10 2024-02-23T00:47:55.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.961+0000 7f4c167fc700 10 client.41306.objecter ms_dispatch 0x7f4c20001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:47:55.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.961+0000 7f4c167fc700 10 client.41306.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:47:55.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.962+0000 7f4c167fc700 20 client.41306.objecter put_session s=0x7f4c201200b0 osd=4 4 2024-02-23T00:47:55.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.962+0000 7f4c167fc700 15 client.41306.objecter _session_command_op_remove 4 10 2024-02-23T00:47:55.965 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:47:55.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.962+0000 7f4c279e7700 10 client.41306.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.962+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_assign -1 11 2024-02-23T00:47:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.962+0000 7f4c279e7700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f4c20125ed0 0x7f4c20126310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.962+0000 7f4c279e7700 20 client.41306.objecter _get_session s=0x7f4c20125850 osd=5 3 2024-02-23T00:47:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.962+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c20125850 osd=5 3 2024-02-23T00:47:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.962+0000 7f4c279e7700 20 client.41306.objecter _get_session s=0x7f4c20125850 osd=5 3 2024-02-23T00:47:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.962+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_remove -1 11 2024-02-23T00:47:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.962+0000 7f4c279e7700 20 client.41306.objecter get_session s=0x7f4c20125850 osd=5 3 2024-02-23T00:47:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.962+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_assign 5 11 2024-02-23T00:47:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.962+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c20125850 osd=5 4 2024-02-23T00:47:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.962+0000 7f4c279e7700 10 client.41306.objecter _send_command 11 2024-02-23T00:47:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.962+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c2012b110 con 0x7f4c20125ed0 2024-02-23T00:47:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.962+0000 7f4c24f82700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f4c20125ed0 0x7f4c20126310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.963+0000 7f4c167fc700 10 client.41306.objecter ms_handle_connect 0x7f4c20125ed0 2024-02-23T00:47:55.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.963+0000 7f4c24f82700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f4c20125ed0 0x7f4c20126310 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:55.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.963+0000 7f4c167fc700 1 -- 172.21.15.47:0/2493717765 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4c2012b110 con 0x7f4c20125ed0 2024-02-23T00:47:55.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.963+0000 7f4c167fc700 10 client.41306.objecter ms_dispatch 0x7f4c20001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:47:55.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.963+0000 7f4c167fc700 10 client.41306.objecter _finish_command 11 = system:0 2024-02-23T00:47:55.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.963+0000 7f4c167fc700 20 client.41306.objecter put_session s=0x7f4c20125850 osd=5 4 2024-02-23T00:47:55.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.963+0000 7f4c167fc700 15 client.41306.objecter _session_command_op_remove 5 11 2024-02-23T00:47:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.978+0000 7f4c279e7700 10 client.41306.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:47:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.978+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_assign -1 12 2024-02-23T00:47:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.978+0000 7f4c279e7700 20 client.41306.objecter _get_session s=0x7f4c20125850 osd=5 3 2024-02-23T00:47:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.978+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c20125850 osd=5 3 2024-02-23T00:47:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.978+0000 7f4c279e7700 20 client.41306.objecter _get_session s=0x7f4c20125850 osd=5 3 2024-02-23T00:47:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.978+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_remove -1 12 2024-02-23T00:47:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.978+0000 7f4c279e7700 20 client.41306.objecter get_session s=0x7f4c20125850 osd=5 3 2024-02-23T00:47:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.978+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_assign 5 12 2024-02-23T00:47:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.978+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c20125850 osd=5 4 2024-02-23T00:47:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.978+0000 7f4c279e7700 10 client.41306.objecter _send_command 12 2024-02-23T00:47:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.978+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f4c20126850 con 0x7f4c20125ed0 2024-02-23T00:47:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.980+0000 7f4c167fc700 1 -- 172.21.15.47:0/2493717765 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f4c20126850 con 0x7f4c20125ed0 2024-02-23T00:47:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.980+0000 7f4c167fc700 10 client.41306.objecter ms_dispatch 0x7f4c20001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:47:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.980+0000 7f4c167fc700 10 client.41306.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:47:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.980+0000 7f4c167fc700 20 client.41306.objecter put_session s=0x7f4c20125850 osd=5 4 2024-02-23T00:47:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.980+0000 7f4c167fc700 15 client.41306.objecter _session_command_op_remove 5 12 2024-02-23T00:47:55.983 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:47:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.980+0000 7f4c279e7700 10 client.41306.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.980+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_assign -1 13 2024-02-23T00:47:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.980+0000 7f4c279e7700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f4c20126f20 0x7f4c20127360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.980+0000 7f4c279e7700 20 client.41306.objecter _get_session s=0x7f4c20126850 osd=6 3 2024-02-23T00:47:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.980+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c20126850 osd=6 3 2024-02-23T00:47:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.980+0000 7f4c279e7700 20 client.41306.objecter _get_session s=0x7f4c20126850 osd=6 3 2024-02-23T00:47:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.980+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_remove -1 13 2024-02-23T00:47:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.980+0000 7f4c279e7700 20 client.41306.objecter get_session s=0x7f4c20126850 osd=6 3 2024-02-23T00:47:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.980+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_assign 6 13 2024-02-23T00:47:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.980+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c20126850 osd=6 4 2024-02-23T00:47:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.980+0000 7f4c279e7700 10 client.41306.objecter _send_command 13 2024-02-23T00:47:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.980+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c20127a40 con 0x7f4c20126f20 2024-02-23T00:47:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.980+0000 7f4c25f84700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f4c20126f20 0x7f4c20127360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.980+0000 7f4c167fc700 10 client.41306.objecter ms_handle_connect 0x7f4c20126f20 2024-02-23T00:47:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.980+0000 7f4c25f84700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f4c20126f20 0x7f4c20127360 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.981+0000 7f4c167fc700 1 -- 172.21.15.47:0/2493717765 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4c20127a40 con 0x7f4c20126f20 2024-02-23T00:47:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.981+0000 7f4c167fc700 10 client.41306.objecter ms_dispatch 0x7f4c20001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:47:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.981+0000 7f4c167fc700 10 client.41306.objecter _finish_command 13 = system:0 2024-02-23T00:47:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.981+0000 7f4c167fc700 20 client.41306.objecter put_session s=0x7f4c20126850 osd=6 4 2024-02-23T00:47:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.981+0000 7f4c167fc700 15 client.41306.objecter _session_command_op_remove 6 13 2024-02-23T00:47:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.995+0000 7f4c279e7700 10 client.41306.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:47:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.995+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_assign -1 14 2024-02-23T00:47:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.995+0000 7f4c279e7700 20 client.41306.objecter _get_session s=0x7f4c20126850 osd=6 3 2024-02-23T00:47:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.995+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c20126850 osd=6 3 2024-02-23T00:47:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.995+0000 7f4c279e7700 20 client.41306.objecter _get_session s=0x7f4c20126850 osd=6 3 2024-02-23T00:47:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.995+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_remove -1 14 2024-02-23T00:47:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.995+0000 7f4c279e7700 20 client.41306.objecter get_session s=0x7f4c20126850 osd=6 3 2024-02-23T00:47:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.995+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_assign 6 14 2024-02-23T00:47:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.995+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c20126850 osd=6 4 2024-02-23T00:47:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.995+0000 7f4c279e7700 10 client.41306.objecter _send_command 14 2024-02-23T00:47:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.995+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f4c20127a40 con 0x7f4c20126f20 2024-02-23T00:47:56.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.997+0000 7f4c167fc700 1 -- 172.21.15.47:0/2493717765 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f4c20127a40 con 0x7f4c20126f20 2024-02-23T00:47:56.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.997+0000 7f4c167fc700 10 client.41306.objecter ms_dispatch 0x7f4c20001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:47:56.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.997+0000 7f4c167fc700 10 client.41306.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:47:56.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.997+0000 7f4c167fc700 20 client.41306.objecter put_session s=0x7f4c20126850 osd=6 4 2024-02-23T00:47:56.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.997+0000 7f4c167fc700 15 client.41306.objecter _session_command_op_remove 6 14 2024-02-23T00:47:56.000 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:47:56.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.998+0000 7f4c279e7700 10 client.41306.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:47:56.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.998+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_assign -1 15 2024-02-23T00:47:56.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.998+0000 7f4c279e7700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f4c2012c500 0x7f4c2012e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:56.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.998+0000 7f4c279e7700 20 client.41306.objecter _get_session s=0x7f4c20127a40 osd=7 3 2024-02-23T00:47:56.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.998+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c20127a40 osd=7 3 2024-02-23T00:47:56.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.998+0000 7f4c279e7700 20 client.41306.objecter _get_session s=0x7f4c20127a40 osd=7 3 2024-02-23T00:47:56.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.998+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_remove -1 15 2024-02-23T00:47:56.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.998+0000 7f4c279e7700 20 client.41306.objecter get_session s=0x7f4c20127a40 osd=7 3 2024-02-23T00:47:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.998+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_assign 7 15 2024-02-23T00:47:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.998+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c20127a40 osd=7 4 2024-02-23T00:47:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.998+0000 7f4c279e7700 10 client.41306.objecter _send_command 15 2024-02-23T00:47:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.998+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c201317b0 con 0x7f4c2012c500 2024-02-23T00:47:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.998+0000 7f4c25783700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f4c2012c500 0x7f4c2012e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.998+0000 7f4c167fc700 10 client.41306.objecter ms_handle_connect 0x7f4c2012c500 2024-02-23T00:47:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.998+0000 7f4c25783700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f4c2012c500 0x7f4c2012e970 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.999+0000 7f4c167fc700 1 -- 172.21.15.47:0/2493717765 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4c201317b0 con 0x7f4c2012c500 2024-02-23T00:47:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.999+0000 7f4c167fc700 10 client.41306.objecter ms_dispatch 0x7f4c20001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:47:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.999+0000 7f4c167fc700 10 client.41306.objecter _finish_command 15 = system:0 2024-02-23T00:47:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.999+0000 7f4c167fc700 20 client.41306.objecter put_session s=0x7f4c20127a40 osd=7 4 2024-02-23T00:47:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:55.999+0000 7f4c167fc700 15 client.41306.objecter _session_command_op_remove 7 15 2024-02-23T00:47:56.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.014+0000 7f4c279e7700 10 client.41306.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:47:56.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.014+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_assign -1 16 2024-02-23T00:47:56.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.014+0000 7f4c279e7700 20 client.41306.objecter _get_session s=0x7f4c20127a40 osd=7 3 2024-02-23T00:47:56.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.014+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c20127a40 osd=7 3 2024-02-23T00:47:56.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.014+0000 7f4c279e7700 20 client.41306.objecter _get_session s=0x7f4c20127a40 osd=7 3 2024-02-23T00:47:56.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.014+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_remove -1 16 2024-02-23T00:47:56.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.014+0000 7f4c279e7700 20 client.41306.objecter get_session s=0x7f4c20127a40 osd=7 3 2024-02-23T00:47:56.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.014+0000 7f4c279e7700 15 client.41306.objecter _session_command_op_assign 7 16 2024-02-23T00:47:56.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.014+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c20127a40 osd=7 4 2024-02-23T00:47:56.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.014+0000 7f4c279e7700 10 client.41306.objecter _send_command 16 2024-02-23T00:47:56.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.014+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f4c201317b0 con 0x7f4c2012c500 2024-02-23T00:47:56.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.015+0000 7f4c167fc700 1 -- 172.21.15.47:0/2493717765 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f4c201317b0 con 0x7f4c2012c500 2024-02-23T00:47:56.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.015+0000 7f4c167fc700 10 client.41306.objecter ms_dispatch 0x7f4c20001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:47:56.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.015+0000 7f4c167fc700 10 client.41306.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:47:56.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.015+0000 7f4c167fc700 20 client.41306.objecter put_session s=0x7f4c20127a40 osd=7 4 2024-02-23T00:47:56.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.015+0000 7f4c167fc700 15 client.41306.objecter _session_command_op_remove 7 16 2024-02-23T00:47:56.019 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:47:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 10 librados: watch_flush enter 2024-02-23T00:47:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 10 librados: watch_flush exit 2024-02-23T00:47:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 10 client.41306.objecter close_session for osd.0 2024-02-23T00:47:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f4c200fe570 msgr2=0x7f4c200fe950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f4c200fe570 0x7f4c200fe950 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c2005f990 osd=0 1 2024-02-23T00:47:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 10 client.41306.objecter close_session for osd.1 2024-02-23T00:47:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f4c200598f0 msgr2=0x7f4c20059d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f4c200598f0 0x7f4c20059d10 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c20028540 osd=1 1 2024-02-23T00:47:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 10 client.41306.objecter close_session for osd.2 2024-02-23T00:47:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f4c200280a0 msgr2=0x7f4c2011e110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f4c200280a0 0x7f4c2011e110 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:56.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c200595a0 osd=2 1 2024-02-23T00:47:56.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 10 client.41306.objecter close_session for osd.3 2024-02-23T00:47:56.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c25f84700 1 -- 172.21.15.47:0/2493717765 reap_dead start 2024-02-23T00:47:56.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f4c2011f780 msgr2=0x7f4c2011fbc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:56.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f4c2011f780 0x7f4c2011fbc0 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:56.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c2011b750 osd=3 1 2024-02-23T00:47:56.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 10 client.41306.objecter close_session for osd.4 2024-02-23T00:47:56.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f4c20122f10 msgr2=0x7f4c20125300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:56.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f4c20122f10 0x7f4c20125300 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:56.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c201200b0 osd=4 1 2024-02-23T00:47:56.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 10 client.41306.objecter close_session for osd.5 2024-02-23T00:47:56.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f4c20125ed0 msgr2=0x7f4c20126310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:56.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f4c20125ed0 0x7f4c20126310 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:56.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c20125850 osd=5 1 2024-02-23T00:47:56.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 10 client.41306.objecter close_session for osd.6 2024-02-23T00:47:56.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f4c20126f20 msgr2=0x7f4c20127360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:56.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f4c20126f20 0x7f4c20127360 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:56.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c20126850 osd=6 1 2024-02-23T00:47:56.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 10 client.41306.objecter close_session for osd.7 2024-02-23T00:47:56.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f4c2012c500 msgr2=0x7f4c2012e970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:56.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f4c2012c500 0x7f4c2012e970 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:56.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 20 client.41306.objecter put_session s=0x7f4c20127a40 osd=7 1 2024-02-23T00:47:56.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 20 client.41306.objecter shutdown clearing up homeless session... 2024-02-23T00:47:56.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 10 client.41306.objecter successfully canceled tick 2024-02-23T00:47:56.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c25f84700 1 -- 172.21.15.47:0/2493717765 reap_dead start 2024-02-23T00:47:56.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4c0c062cc0 msgr2=0x7f4c0c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:56.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4c0c062cc0 0x7f4c0c065180 secure :-1 s=READY pgs=3059 cs=0 l=1 rev1=1 rx=0x7f4c1c0047e0 tx=0x7f4c1c00b040).stop 2024-02-23T00:47:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4c200f7300 msgr2=0x7f4c20110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4c200f7300 0x7f4c20110620 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 rx=0x7f4c10007fd0 tx=0x7f4c10009cb0).stop 2024-02-23T00:47:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 shutdown_connections 2024-02-23T00:47:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4c200f7300 0x7f4c20110620 unknown :-1 s=CLOSED pgs=2659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 1 --2- 172.21.15.47:0/2493717765 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4c0c062cc0 0x7f4c0c065180 unknown :-1 s=CLOSED pgs=3059 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 >> 172.21.15.47:0/2493717765 conn(0x7f4c200f26f0 msgr2=0x7f4c200f4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 shutdown_connections 2024-02-23T00:47:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 1 -- 172.21.15.47:0/2493717765 wait complete. 2024-02-23T00:47:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:56.016+0000 7f4c279e7700 1 librados: shutdown 2024-02-23T00:47:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:47:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:47:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:47:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:47:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:47:56.025 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:47:56.025 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:47:56.025 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:47:56.030 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:47:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:56.409 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:47:56.412 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:56.413 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:47:56.513 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:47:56.542 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:56.566 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:47:56.568 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:56.569 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:47:56.671 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:47:56.699 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:54.953406+0000 osd.0 (osd.0) 159 : cluster [DBG] 18.e scrub starts 2024-02-23T00:47:56.699 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:54.954495+0000 osd.0 (osd.0) 160 : cluster [DBG] 18.e scrub ok 2024-02-23T00:47:56.699 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:55.283129+0000 mgr.y (mgr.14517) 370 : cluster [DBG] pgmap v580: 208 pgs: 32 unknown, 1 peering, 175 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-23T00:47:56.699 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:55.309844+0000 osd.5 (osd.5) 166 : cluster [DBG] 2.3 scrub starts 2024-02-23T00:47:56.699 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:55.310648+0000 osd.5 (osd.5) 167 : cluster [DBG] 2.3 scrub ok 2024-02-23T00:47:56.699 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:55.477645+0000 osd.6 (osd.6) 159 : cluster [DBG] 62.19 scrub starts 2024-02-23T00:47:56.699 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:55.479664+0000 osd.6 (osd.6) 160 : cluster [DBG] 62.19 scrub ok 2024-02-23T00:47:56.699 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:55.873068+0000 mon.c (mon.2) 297 : audit [DBG] from='client.? 172.21.15.47:0/2493717765' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:47:56.699 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:56.178280+0000 mon.a (mon.0) 2526 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:47:56.699 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:56.178400+0000 mon.a (mon.0) 2527 : cluster [DBG] osdmap e404: 8 total, 8 up, 7 in 2024-02-23T00:47:56.699 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:56.181964+0000 mon.a (mon.0) 2528 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:47:56.700 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:56.225355+0000 mon.a (mon.0) 2529 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:47:56.700 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:56.225721+0000 mon.a (mon.0) 2530 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-53"}]: dispatch 2024-02-23T00:47:56.700 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:56.303191+0000 mon.a (mon.0) 2531 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:47:56.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:54.953406+0000 osd.0 (osd.0) 159 : cluster [DBG] 18.e scrub starts 2024-02-23T00:47:56.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:54.954495+0000 osd.0 (osd.0) 160 : cluster [DBG] 18.e scrub ok 2024-02-23T00:47:56.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:55.283129+0000 mgr.y (mgr.14517) 370 : cluster [DBG] pgmap v580: 208 pgs: 32 unknown, 1 peering, 175 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-23T00:47:56.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:55.309844+0000 osd.5 (osd.5) 166 : cluster [DBG] 2.3 scrub starts 2024-02-23T00:47:56.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:55.310648+0000 osd.5 (osd.5) 167 : cluster [DBG] 2.3 scrub ok 2024-02-23T00:47:56.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:55.477645+0000 osd.6 (osd.6) 159 : cluster [DBG] 62.19 scrub starts 2024-02-23T00:47:56.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:55.479664+0000 osd.6 (osd.6) 160 : cluster [DBG] 62.19 scrub ok 2024-02-23T00:47:56.701 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:55.873068+0000 mon.c (mon.2) 297 : audit [DBG] from='client.? 172.21.15.47:0/2493717765' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:47:56.701 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:56.178280+0000 mon.a (mon.0) 2526 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:47:56.701 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:56.178400+0000 mon.a (mon.0) 2527 : cluster [DBG] osdmap e404: 8 total, 8 up, 7 in 2024-02-23T00:47:56.701 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:56.181964+0000 mon.a (mon.0) 2528 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:47:56.701 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:56.225355+0000 mon.a (mon.0) 2529 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:47:56.701 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:56.225721+0000 mon.a (mon.0) 2530 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-53"}]: dispatch 2024-02-23T00:47:56.701 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:56.303191+0000 mon.a (mon.0) 2531 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:47:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:56.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:54.953406+0000 osd.0 (osd.0) 159 : cluster [DBG] 18.e scrub starts 2024-02-23T00:47:56.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:54.954495+0000 osd.0 (osd.0) 160 : cluster [DBG] 18.e scrub ok 2024-02-23T00:47:56.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:55.283129+0000 mgr.y (mgr.14517) 370 : cluster [DBG] pgmap v580: 208 pgs: 32 unknown, 1 peering, 175 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-23T00:47:56.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:55.309844+0000 osd.5 (osd.5) 166 : cluster [DBG] 2.3 scrub starts 2024-02-23T00:47:56.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:55.310648+0000 osd.5 (osd.5) 167 : cluster [DBG] 2.3 scrub ok 2024-02-23T00:47:56.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:55.477645+0000 osd.6 (osd.6) 159 : cluster [DBG] 62.19 scrub starts 2024-02-23T00:47:56.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:55.479664+0000 osd.6 (osd.6) 160 : cluster [DBG] 62.19 scrub ok 2024-02-23T00:47:56.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:55.873068+0000 mon.c (mon.2) 297 : audit [DBG] from='client.? 172.21.15.47:0/2493717765' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:47:56.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:56.178280+0000 mon.a (mon.0) 2526 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:47:56.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:56.178400+0000 mon.a (mon.0) 2527 : cluster [DBG] osdmap e404: 8 total, 8 up, 7 in 2024-02-23T00:47:56.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:56.181964+0000 mon.a (mon.0) 2528 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:47:56.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:56.225355+0000 mon.a (mon.0) 2529 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:47:56.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:56.225721+0000 mon.a (mon.0) 2530 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-53"}]: dispatch 2024-02-23T00:47:56.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:56.303191+0000 mon.a (mon.0) 2531 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:47:56.950 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:47:56.953 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:56.954 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:47:57.054 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:47:57.083 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:57.104 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:47:57.106 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:57.107 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:47:57.208 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:47:57.235 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:57.254 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:47:57.257 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:57.257 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:47:57.358 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:47:57.387 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:57.465 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:47:57.466 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:47:57.636 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:47:57.638 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:57.639 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:47:57.664 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-23T00:47:57.740 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:47:57.767 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:57.787 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:47:57.789 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:57.789 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:47:57.890 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:47:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:58.188 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:57.179668+0000 mon.a (mon.0) 2532 : cluster [DBG] osdmap e405: 8 total, 8 up, 7 in 2024-02-23T00:47:58.188 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:57.283867+0000 mgr.y (mgr.14517) 371 : cluster [DBG] pgmap v583: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.9 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-02-23T00:47:58.212 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.1 dump_blocked_ops 2024-02-23T00:47:58.441 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:57.179668+0000 mon.a (mon.0) 2532 : cluster [DBG] osdmap e405: 8 total, 8 up, 7 in 2024-02-23T00:47:58.441 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:57.283867+0000 mgr.y (mgr.14517) 371 : cluster [DBG] pgmap v583: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.9 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-02-23T00:47:58.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:57.179668+0000 mon.a (mon.0) 2532 : cluster [DBG] osdmap e405: 8 total, 8 up, 7 in 2024-02-23T00:47:58.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:57.283867+0000 mgr.y (mgr.14517) 371 : cluster [DBG] pgmap v583: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.9 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-02-23T00:47:58.467 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:47:58.469 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:58.470 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:47:58.571 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:47:58.603 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:58.623 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:47:58.626 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:58.626 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:47:58.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.687+0000 7f78ec095700 1 -- 172.21.15.47:0/3324629826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78e40fdbb0 msgr2=0x7f78e40fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:58.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.687+0000 7f78ec095700 1 --2- 172.21.15.47:0/3324629826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78e40fdbb0 0x7f78e40fdf90 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 rx=0x7f78d80047b0 tx=0x7f78d8009cb0).stop 2024-02-23T00:47:58.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.687+0000 7f78ec095700 1 -- 172.21.15.47:0/3324629826 shutdown_connections 2024-02-23T00:47:58.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.687+0000 7f78ec095700 1 --2- 172.21.15.47:0/3324629826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78e40f8420 0x7f78e4104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:58.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.687+0000 7f78ec095700 1 --2- 172.21.15.47:0/3324629826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78e40fdbb0 0x7f78e40fdf90 unknown :-1 s=CLOSED pgs=2528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:58.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.687+0000 7f78ec095700 1 --2- 172.21.15.47:0/3324629826 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f78e40fe510 0x7f78e40f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:58.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.687+0000 7f78ec095700 1 -- 172.21.15.47:0/3324629826 >> 172.21.15.47:0/3324629826 conn(0x7f78e40f3800 msgr2=0x7f78e40f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:58.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.687+0000 7f78ec095700 1 -- 172.21.15.47:0/3324629826 shutdown_connections 2024-02-23T00:47:58.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.687+0000 7f78ec095700 1 -- 172.21.15.47:0/3324629826 wait complete. 2024-02-23T00:47:58.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.687+0000 7f78ec095700 1 librados: starting msgr at 2024-02-23T00:47:58.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.687+0000 7f78ec095700 1 librados: starting objecter 2024-02-23T00:47:58.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.687+0000 7f78ec095700 1 Processor -- start 2024-02-23T00:47:58.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.687+0000 7f78ec095700 1 -- start start 2024-02-23T00:47:58.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.687+0000 7f78ec095700 1 librados: setting wanted keys 2024-02-23T00:47:58.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.687+0000 7f78ec095700 1 librados: calling monclient init 2024-02-23T00:47:58.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.688+0000 7f78ec095700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78e40f8420 0x7f78e410df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:58.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.688+0000 7f78ec095700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78e40fdbb0 0x7f78e410e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:58.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.688+0000 7f78ec095700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f78e40fe510 0x7f78e41080f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:58.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.688+0000 7f78ec095700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e41085e0 con 0x7f78e40fdbb0 2024-02-23T00:47:58.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.688+0000 7f78ec095700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e4108750 con 0x7f78e40f8420 2024-02-23T00:47:58.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.688+0000 7f78ec095700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e4108910 con 0x7f78e40fe510 2024-02-23T00:47:58.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.688+0000 7f78ea632700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f78e40fe510 0x7f78e41080f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:58.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.688+0000 7f78e9630700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78e40fdbb0 0x7f78e410e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:58.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.688+0000 7f78ea632700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f78e40fe510 0x7f78e41080f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:48848/0 (socket says 172.21.15.47:48848) 2024-02-23T00:47:58.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.688+0000 7f78ea632700 1 -- 172.21.15.47:0/3149692638 learned_addr learned my addr 172.21.15.47:0/3149692638 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:47:58.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.688+0000 7f78e9e31700 1 --2- 172.21.15.47:0/3149692638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78e40f8420 0x7f78e410df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:58.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.688+0000 7f78ea632700 1 -- 172.21.15.47:0/3149692638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78e40f8420 msgr2=0x7f78e410df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:58.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.688+0000 7f78ea632700 1 --2- 172.21.15.47:0/3149692638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78e40f8420 0x7f78e410df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:58.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.688+0000 7f78ea632700 1 -- 172.21.15.47:0/3149692638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78e40fdbb0 msgr2=0x7f78e410e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:58.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.688+0000 7f78ea632700 1 --2- 172.21.15.47:0/3149692638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78e40fdbb0 0x7f78e410e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:58.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.688+0000 7f78ea632700 1 -- 172.21.15.47:0/3149692638 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e0005040 con 0x7f78e40fe510 2024-02-23T00:47:58.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.688+0000 7f78e9630700 1 --2- 172.21.15.47:0/3149692638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78e40fdbb0 0x7f78e410e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:47:58.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.688+0000 7f78ea632700 1 --2- 172.21.15.47:0/3149692638 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f78e40fe510 0x7f78e41080f0 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 rx=0x7f78dc00ec20 tx=0x7f78dc00b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:58.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.688+0000 7f78d6ffd700 10 client.?.objecter ms_handle_connect 0x7f78e40fe510 2024-02-23T00:47:58.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.688+0000 7f78d6ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:47:58.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.688+0000 7f78d6ffd700 1 -- 172.21.15.47:0/3149692638 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78dc009950 con 0x7f78e40fe510 2024-02-23T00:47:58.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.688+0000 7f78d6ffd700 1 -- 172.21.15.47:0/3149692638 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f78dc00f040 con 0x7f78e40fe510 2024-02-23T00:47:58.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.688+0000 7f78ec095700 1 -- 172.21.15.47:0/3149692638 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d8005040 con 0x7f78e40fe510 2024-02-23T00:47:58.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.688+0000 7f78d6ffd700 1 -- 172.21.15.47:0/3149692638 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78dc007d10 con 0x7f78e40fe510 2024-02-23T00:47:58.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.688+0000 7f78ec095700 10 client.41312.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:47:58.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.688+0000 7f78ec095700 1 -- 172.21.15.47:0/3149692638 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e4108ea0 con 0x7f78e40fe510 2024-02-23T00:47:58.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.688+0000 7f78ec095700 1 librados: init done 2024-02-23T00:47:58.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.689+0000 7f78ec095700 1 -- 172.21.15.47:0/3149692638 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78e405ee80 con 0x7f78e40fe510 2024-02-23T00:47:58.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.690+0000 7f78d6ffd700 1 -- 172.21.15.47:0/3149692638 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f78dc0092e0 con 0x7f78e40fe510 2024-02-23T00:47:58.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.690+0000 7f78d6ffd700 1 --2- 172.21.15.47:0/3149692638 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f78d0062cc0 0x7f78d0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:47:58.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.691+0000 7f78d6ffd700 1 -- 172.21.15.47:0/3149692638 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(406..406 src has 48..406) v4 ==== 9191+0+0 (secure 0 0 0) 0x7f78dc07d800 con 0x7f78e40fe510 2024-02-23T00:47:58.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.691+0000 7f78d6ffd700 10 client.41312.objecter ms_dispatch 0x7f78e4001550 osd_map(406..406 src has 48..406) v4 2024-02-23T00:47:58.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.691+0000 7f78d6ffd700 3 client.41312.objecter handle_osd_map got epochs [406,406] > 0 2024-02-23T00:47:58.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.691+0000 7f78d6ffd700 3 client.41312.objecter handle_osd_map decoding full epoch 406 2024-02-23T00:47:58.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.691+0000 7f78e9e31700 1 --2- 172.21.15.47:0/3149692638 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f78d0062cc0 0x7f78d0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:47:58.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.691+0000 7f78d6ffd700 20 client.41312.objecter dump_active .. 0 homeless 2024-02-23T00:47:58.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.691+0000 7f78d6ffd700 10 client.41312.objecter ms_handle_connect 0x7f78d0062cc0 2024-02-23T00:47:58.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.691+0000 7f78e9e31700 1 --2- 172.21.15.47:0/3149692638 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f78d0062cc0 0x7f78d0065180 secure :-1 s=READY pgs=3060 cs=0 l=1 rev1=1 rx=0x7f78d8004e20 tx=0x7f78d8009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:47:58.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.694+0000 7f78d6ffd700 1 -- 172.21.15.47:0/3149692638 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f78dc04ceb0 con 0x7f78e40fe510 2024-02-23T00:47:58.727 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:47:58.757 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:58.777 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:47:58.778 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:58.779 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:47:58.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.870+0000 7f78ec095700 1 -- 172.21.15.47:0/3149692638 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 1.0, "target": ["mon-mgr", ""]}) v1 -- 0x7f78e405fa60 con 0x7f78d0062cc0 2024-02-23T00:47:58.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.871+0000 7f78d6ffd700 1 -- 172.21.15.47:0/3149692638 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 no change) v1 ==== 17+0+298 (secure 0 0 0) 0x7f78e405fa60 con 0x7f78d0062cc0 2024-02-23T00:47:58.874 INFO:teuthology.orchestra.run.smithi047.stdout:moved 0 / 624 (0%) 2024-02-23T00:47:58.874 INFO:teuthology.orchestra.run.smithi047.stdout:avg 89.1429 2024-02-23T00:47:58.874 INFO:teuthology.orchestra.run.smithi047.stdout:stddev 10.0914 -> 10.0914 (expected baseline 8.74118) 2024-02-23T00:47:58.874 INFO:teuthology.orchestra.run.smithi047.stdout:min osd.5 with 71 -> 71 pgs (0.796474 -> 0.796474 * mean) 2024-02-23T00:47:58.874 INFO:teuthology.orchestra.run.smithi047.stdout:max osd.7 with 102 -> 102 pgs (1.14423 -> 1.14423 * mean) 2024-02-23T00:47:58.874 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:47:58.875 INFO:teuthology.orchestra.run.smithi047.stdout:oload 110 2024-02-23T00:47:58.875 INFO:teuthology.orchestra.run.smithi047.stdout:max_change 1 2024-02-23T00:47:58.875 INFO:teuthology.orchestra.run.smithi047.stdout:max_change_osds 4 2024-02-23T00:47:58.875 INFO:teuthology.orchestra.run.smithi047.stdout:average_utilization 0.0033 2024-02-23T00:47:58.875 INFO:teuthology.orchestra.run.smithi047.stdout:overload_utilization 0.0036 2024-02-23T00:47:58.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.873+0000 7f78ec095700 10 librados: watch_flush enter 2024-02-23T00:47:58.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.873+0000 7f78ec095700 10 librados: watch_flush exit 2024-02-23T00:47:58.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.873+0000 7f78ec095700 20 client.41312.objecter shutdown clearing up homeless session... 2024-02-23T00:47:58.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.873+0000 7f78ec095700 10 client.41312.objecter successfully canceled tick 2024-02-23T00:47:58.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.873+0000 7f78ec095700 1 -- 172.21.15.47:0/3149692638 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f78d0062cc0 msgr2=0x7f78d0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:58.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.873+0000 7f78ec095700 1 --2- 172.21.15.47:0/3149692638 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f78d0062cc0 0x7f78d0065180 secure :-1 s=READY pgs=3060 cs=0 l=1 rev1=1 rx=0x7f78d8004e20 tx=0x7f78d8009cb0).stop 2024-02-23T00:47:58.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.873+0000 7f78ec095700 1 -- 172.21.15.47:0/3149692638 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f78e40fe510 msgr2=0x7f78e41080f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:47:58.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.873+0000 7f78ec095700 1 --2- 172.21.15.47:0/3149692638 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f78e40fe510 0x7f78e41080f0 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 rx=0x7f78dc00ec20 tx=0x7f78dc00b040).stop 2024-02-23T00:47:58.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.874+0000 7f78ec095700 1 -- 172.21.15.47:0/3149692638 shutdown_connections 2024-02-23T00:47:58.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.874+0000 7f78ec095700 1 --2- 172.21.15.47:0/3149692638 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f78d0062cc0 0x7f78d0065180 unknown :-1 s=CLOSED pgs=3060 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:58.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.874+0000 7f78ec095700 1 --2- 172.21.15.47:0/3149692638 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f78e40fe510 0x7f78e41080f0 unknown :-1 s=CLOSED pgs=2660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:58.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.874+0000 7f78ec095700 1 --2- 172.21.15.47:0/3149692638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78e40f8420 0x7f78e410df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:58.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.874+0000 7f78ec095700 1 --2- 172.21.15.47:0/3149692638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78e40fdbb0 0x7f78e410e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:47:58.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.874+0000 7f78ec095700 1 -- 172.21.15.47:0/3149692638 >> 172.21.15.47:0/3149692638 conn(0x7f78e40f3800 msgr2=0x7f78e40fb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:47:58.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.874+0000 7f78ec095700 1 -- 172.21.15.47:0/3149692638 shutdown_connections 2024-02-23T00:47:58.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.874+0000 7f78ec095700 1 -- 172.21.15.47:0/3149692638 wait complete. 2024-02-23T00:47:58.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:47:58.874+0000 7f78ec095700 1 librados: shutdown 2024-02-23T00:47:58.878 INFO:teuthology.orchestra.run.smithi047.stderr:no change 2024-02-23T00:47:58.881 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:47:58.909 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:58.928 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:47:58.930 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:58.931 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:47:59.031 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:47:59.061 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:59.311 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:47:59.313 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:57.268002+0000 osd.5 (osd.5) 168 : cluster [DBG] 18.15 scrub starts 2024-02-23T00:47:59.313 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:57.272894+0000 osd.5 (osd.5) 169 : cluster [DBG] 18.15 scrub ok 2024-02-23T00:47:59.313 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:57.661647+0000 osd.1 (osd.1) 169 : cluster [DBG] 6.4 scrub starts 2024-02-23T00:47:59.313 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:57.665152+0000 osd.1 (osd.1) 170 : cluster [DBG] 6.4 scrub ok 2024-02-23T00:47:59.313 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:58.187333+0000 mon.a (mon.0) 2533 : cluster [DBG] osdmap e406: 8 total, 8 up, 7 in 2024-02-23T00:47:59.313 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:47:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:58.190687+0000 mon.a (mon.0) 2534 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:47:59.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:57.268002+0000 osd.5 (osd.5) 168 : cluster [DBG] 18.15 scrub starts 2024-02-23T00:47:59.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:57.272894+0000 osd.5 (osd.5) 169 : cluster [DBG] 18.15 scrub ok 2024-02-23T00:47:59.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:57.661647+0000 osd.1 (osd.1) 169 : cluster [DBG] 6.4 scrub starts 2024-02-23T00:47:59.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:57.665152+0000 osd.1 (osd.1) 170 : cluster [DBG] 6.4 scrub ok 2024-02-23T00:47:59.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:58.187333+0000 mon.a (mon.0) 2533 : cluster [DBG] osdmap e406: 8 total, 8 up, 7 in 2024-02-23T00:47:59.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:47:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:58.190687+0000 mon.a (mon.0) 2534 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:47:59.315 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:59.315 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:47:59.416 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:47:59.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:57.268002+0000 osd.5 (osd.5) 168 : cluster [DBG] 18.15 scrub starts 2024-02-23T00:47:59.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:57.272894+0000 osd.5 (osd.5) 169 : cluster [DBG] 18.15 scrub ok 2024-02-23T00:47:59.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:57.661647+0000 osd.1 (osd.1) 169 : cluster [DBG] 6.4 scrub starts 2024-02-23T00:47:59.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:57.665152+0000 osd.1 (osd.1) 170 : cluster [DBG] 6.4 scrub ok 2024-02-23T00:47:59.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:58.187333+0000 mon.a (mon.0) 2533 : cluster [DBG] osdmap e406: 8 total, 8 up, 7 in 2024-02-23T00:47:59.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:47:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:58.190687+0000 mon.a (mon.0) 2534 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:47:59.444 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:59.568 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:47:59.571 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:59.571 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:47:59.672 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:47:59.700 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:47:59.722 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:47:59.735 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:47:59.735 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:47:59.836 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:47:59.864 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:00.065 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:48:00.068 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:00.068 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:48:00.080 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-02-23T00:48:00.080 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd test-reweight-by-utilization 110 1.0 2024-02-23T00:48:00.134 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:48:00.168 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:48:00.169 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:48:00.232 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:00.435 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:48:00.437 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:58.872282+0000 mgr.y (mgr.14517) 372 : audit [DBG] from='client.41312 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 1.0, "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:48:00.437 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:58.979762+0000 osd.0 (osd.0) 161 : cluster [DBG] 4.1e scrub starts 2024-02-23T00:48:00.438 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:58.981224+0000 osd.0 (osd.0) 162 : cluster [DBG] 4.1e scrub ok 2024-02-23T00:48:00.438 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:59.074508+0000 osd.3 (osd.3) 184 : cluster [DBG] 5.13 scrub starts 2024-02-23T00:48:00.438 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:59.075443+0000 osd.3 (osd.3) 185 : cluster [DBG] 5.13 scrub ok 2024-02-23T00:48:00.438 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:59.189908+0000 mon.a (mon.0) 2535 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:48:00.438 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:59.190072+0000 mon.a (mon.0) 2536 : cluster [DBG] osdmap e407: 8 total, 8 up, 7 in 2024-02-23T00:48:00.438 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:59.229471+0000 mon.a (mon.0) 2537 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:48:00.438 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:47:59.229771+0000 mon.a (mon.0) 2538 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-55"}]: dispatch 2024-02-23T00:48:00.438 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:47:59.284451+0000 mgr.y (mgr.14517) 373 : cluster [DBG] pgmap v586: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s; 184 B/s, 0 objects/s recovering 2024-02-23T00:48:00.439 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:58.872282+0000 mgr.y (mgr.14517) 372 : audit [DBG] from='client.41312 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 1.0, "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:48:00.439 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:58.979762+0000 osd.0 (osd.0) 161 : cluster [DBG] 4.1e scrub starts 2024-02-23T00:48:00.439 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:58.981224+0000 osd.0 (osd.0) 162 : cluster [DBG] 4.1e scrub ok 2024-02-23T00:48:00.439 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:59.074508+0000 osd.3 (osd.3) 184 : cluster [DBG] 5.13 scrub starts 2024-02-23T00:48:00.439 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:59.075443+0000 osd.3 (osd.3) 185 : cluster [DBG] 5.13 scrub ok 2024-02-23T00:48:00.439 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:59.189908+0000 mon.a (mon.0) 2535 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:48:00.439 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:59.190072+0000 mon.a (mon.0) 2536 : cluster [DBG] osdmap e407: 8 total, 8 up, 7 in 2024-02-23T00:48:00.439 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:59.229471+0000 mon.a (mon.0) 2537 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:48:00.439 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:47:59.229771+0000 mon.a (mon.0) 2538 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-55"}]: dispatch 2024-02-23T00:48:00.440 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:47:59.284451+0000 mgr.y (mgr.14517) 373 : cluster [DBG] pgmap v586: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s; 184 B/s, 0 objects/s recovering 2024-02-23T00:48:00.440 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:00.440 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:48:00.541 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:48:00.570 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:00.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:58.872282+0000 mgr.y (mgr.14517) 372 : audit [DBG] from='client.41312 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 1.0, "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:48:00.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:58.979762+0000 osd.0 (osd.0) 161 : cluster [DBG] 4.1e scrub starts 2024-02-23T00:48:00.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:58.981224+0000 osd.0 (osd.0) 162 : cluster [DBG] 4.1e scrub ok 2024-02-23T00:48:00.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:59.074508+0000 osd.3 (osd.3) 184 : cluster [DBG] 5.13 scrub starts 2024-02-23T00:48:00.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:59.075443+0000 osd.3 (osd.3) 185 : cluster [DBG] 5.13 scrub ok 2024-02-23T00:48:00.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:59.189908+0000 mon.a (mon.0) 2535 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:48:00.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:59.190072+0000 mon.a (mon.0) 2536 : cluster [DBG] osdmap e407: 8 total, 8 up, 7 in 2024-02-23T00:48:00.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:59.229471+0000 mon.a (mon.0) 2537 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:48:00.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:47:59.229771+0000 mon.a (mon.0) 2538 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-55"}]: dispatch 2024-02-23T00:48:00.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:47:59.284451+0000 mgr.y (mgr.14517) 373 : cluster [DBG] pgmap v586: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s; 184 B/s, 0 objects/s recovering 2024-02-23T00:48:00.752 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:48:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:48:00] "GET /metrics HTTP/1.1" 200 188809 "" "Prometheus/2.33.4" 2024-02-23T00:48:01.327 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:48:01.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:00.198140+0000 mon.a (mon.0) 2539 : cluster [DBG] osdmap e408: 8 total, 8 up, 7 in 2024-02-23T00:48:01.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:00.198140+0000 mon.a (mon.0) 2539 : cluster [DBG] osdmap e408: 8 total, 8 up, 7 in 2024-02-23T00:48:01.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:00.198140+0000 mon.a (mon.0) 2539 : cluster [DBG] osdmap e408: 8 total, 8 up, 7 in 2024-02-23T00:48:01.740 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:48:02.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:00.697267+0000 osd.1 (osd.1) 171 : cluster [DBG] 4.f scrub starts 2024-02-23T00:48:02.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:00.698665+0000 osd.1 (osd.1) 172 : cluster [DBG] 4.f scrub ok 2024-02-23T00:48:02.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:01.200344+0000 mon.a (mon.0) 2540 : cluster [DBG] osdmap e409: 8 total, 8 up, 7 in 2024-02-23T00:48:02.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:01.204969+0000 mon.a (mon.0) 2541 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:48:02.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:01.284841+0000 mgr.y (mgr.14517) 374 : cluster [DBG] pgmap v589: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:48:02.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:02.200899+0000 mon.a (mon.0) 2542 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:48:02.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:02.201022+0000 mon.a (mon.0) 2543 : cluster [DBG] osdmap e410: 8 total, 8 up, 7 in 2024-02-23T00:48:02.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:02.202059+0000 mon.a (mon.0) 2544 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:48:02.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:00.697267+0000 osd.1 (osd.1) 171 : cluster [DBG] 4.f scrub starts 2024-02-23T00:48:02.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:00.698665+0000 osd.1 (osd.1) 172 : cluster [DBG] 4.f scrub ok 2024-02-23T00:48:02.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:01.200344+0000 mon.a (mon.0) 2540 : cluster [DBG] osdmap e409: 8 total, 8 up, 7 in 2024-02-23T00:48:02.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:01.204969+0000 mon.a (mon.0) 2541 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:48:02.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:01.284841+0000 mgr.y (mgr.14517) 374 : cluster [DBG] pgmap v589: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:48:02.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:02.200899+0000 mon.a (mon.0) 2542 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:48:02.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:02.201022+0000 mon.a (mon.0) 2543 : cluster [DBG] osdmap e410: 8 total, 8 up, 7 in 2024-02-23T00:48:02.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:02.202059+0000 mon.a (mon.0) 2544 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:48:02.527 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:00.697267+0000 osd.1 (osd.1) 171 : cluster [DBG] 4.f scrub starts 2024-02-23T00:48:02.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:00.698665+0000 osd.1 (osd.1) 172 : cluster [DBG] 4.f scrub ok 2024-02-23T00:48:02.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:01.200344+0000 mon.a (mon.0) 2540 : cluster [DBG] osdmap e409: 8 total, 8 up, 7 in 2024-02-23T00:48:02.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:01.204969+0000 mon.a (mon.0) 2541 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:48:02.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:01.284841+0000 mgr.y (mgr.14517) 374 : cluster [DBG] pgmap v589: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:48:02.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:02.200899+0000 mon.a (mon.0) 2542 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:48:02.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:02.201022+0000 mon.a (mon.0) 2543 : cluster [DBG] osdmap e410: 8 total, 8 up, 7 in 2024-02-23T00:48:02.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:02.202059+0000 mon.a (mon.0) 2544 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:48:02.943 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:48:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:48:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:48:02.944 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:48:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:48:03.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:01.583072+0000 osd.6 (osd.6) 161 : cluster [DBG] 4.5 scrub starts 2024-02-23T00:48:03.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:01.585197+0000 osd.6 (osd.6) 162 : cluster [DBG] 4.5 scrub ok 2024-02-23T00:48:03.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:02.262462+0000 mon.a (mon.0) 2545 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:48:03.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:02.262817+0000 mon.a (mon.0) 2546 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-57"}]: dispatch 2024-02-23T00:48:03.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:02.289658+0000 osd.5 (osd.5) 170 : cluster [DBG] 18.1e scrub starts 2024-02-23T00:48:03.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:02.292550+0000 osd.5 (osd.5) 171 : cluster [DBG] 18.1e scrub ok 2024-02-23T00:48:03.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:02.570024+0000 mgr.y (mgr.14517) 375 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:48:03.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:01.583072+0000 osd.6 (osd.6) 161 : cluster [DBG] 4.5 scrub starts 2024-02-23T00:48:03.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:01.585197+0000 osd.6 (osd.6) 162 : cluster [DBG] 4.5 scrub ok 2024-02-23T00:48:03.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:02.262462+0000 mon.a (mon.0) 2545 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:48:03.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:02.262817+0000 mon.a (mon.0) 2546 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-57"}]: dispatch 2024-02-23T00:48:03.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:02.289658+0000 osd.5 (osd.5) 170 : cluster [DBG] 18.1e scrub starts 2024-02-23T00:48:03.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:02.292550+0000 osd.5 (osd.5) 171 : cluster [DBG] 18.1e scrub ok 2024-02-23T00:48:03.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:02.570024+0000 mgr.y (mgr.14517) 375 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:48:03.365 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:48:03.368 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:03.369 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:48:03.470 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:48:03.498 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:03.525 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:48:03.528 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:03.529 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:48:03.629 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:48:03.659 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:01.583072+0000 osd.6 (osd.6) 161 : cluster [DBG] 4.5 scrub starts 2024-02-23T00:48:03.660 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:01.585197+0000 osd.6 (osd.6) 162 : cluster [DBG] 4.5 scrub ok 2024-02-23T00:48:03.660 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:02.262462+0000 mon.a (mon.0) 2545 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:48:03.660 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:02.262817+0000 mon.a (mon.0) 2546 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-57"}]: dispatch 2024-02-23T00:48:03.660 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:02.289658+0000 osd.5 (osd.5) 170 : cluster [DBG] 18.1e scrub starts 2024-02-23T00:48:03.660 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:02.292550+0000 osd.5 (osd.5) 171 : cluster [DBG] 18.1e scrub ok 2024-02-23T00:48:03.660 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:02.570024+0000 mgr.y (mgr.14517) 375 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:48:03.662 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:03.683 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:48:03.685 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:03.686 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:48:03.787 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:48:03.819 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:04.068 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:48:04.070 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:04.071 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:48:04.172 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:48:04.203 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:04.224 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:48:04.227 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:04.228 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:48:04.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:02.918458+0000 osd.0 (osd.0) 163 : cluster [DBG] 6.0 scrub starts 2024-02-23T00:48:04.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:02.920079+0000 osd.0 (osd.0) 164 : cluster [DBG] 6.0 scrub ok 2024-02-23T00:48:04.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:03.020040+0000 osd.3 (osd.3) 186 : cluster [DBG] 4.19 scrub starts 2024-02-23T00:48:04.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:03.023055+0000 osd.3 (osd.3) 187 : cluster [DBG] 4.19 scrub ok 2024-02-23T00:48:04.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:03.225944+0000 mon.a (mon.0) 2547 : cluster [DBG] osdmap e411: 8 total, 8 up, 7 in 2024-02-23T00:48:04.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:03.285234+0000 mgr.y (mgr.14517) 376 : cluster [DBG] pgmap v592: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-23T00:48:04.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:03.338941+0000 osd.5 (osd.5) 172 : cluster [DBG] 18.7 scrub starts 2024-02-23T00:48:04.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:03.345536+0000 osd.5 (osd.5) 173 : cluster [DBG] 18.7 scrub ok 2024-02-23T00:48:04.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:03.532625+0000 osd.4 (osd.4) 159 : cluster [DBG] 5.1c scrub starts 2024-02-23T00:48:04.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:03.533551+0000 osd.4 (osd.4) 160 : cluster [DBG] 5.1c scrub ok 2024-02-23T00:48:04.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:02.918458+0000 osd.0 (osd.0) 163 : cluster [DBG] 6.0 scrub starts 2024-02-23T00:48:04.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:02.920079+0000 osd.0 (osd.0) 164 : cluster [DBG] 6.0 scrub ok 2024-02-23T00:48:04.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:03.020040+0000 osd.3 (osd.3) 186 : cluster [DBG] 4.19 scrub starts 2024-02-23T00:48:04.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:03.023055+0000 osd.3 (osd.3) 187 : cluster [DBG] 4.19 scrub ok 2024-02-23T00:48:04.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:03.225944+0000 mon.a (mon.0) 2547 : cluster [DBG] osdmap e411: 8 total, 8 up, 7 in 2024-02-23T00:48:04.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:03.285234+0000 mgr.y (mgr.14517) 376 : cluster [DBG] pgmap v592: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-23T00:48:04.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:03.338941+0000 osd.5 (osd.5) 172 : cluster [DBG] 18.7 scrub starts 2024-02-23T00:48:04.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:03.345536+0000 osd.5 (osd.5) 173 : cluster [DBG] 18.7 scrub ok 2024-02-23T00:48:04.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:03.532625+0000 osd.4 (osd.4) 159 : cluster [DBG] 5.1c scrub starts 2024-02-23T00:48:04.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:03.533551+0000 osd.4 (osd.4) 160 : cluster [DBG] 5.1c scrub ok 2024-02-23T00:48:04.328 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:48:04.360 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:04.381 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:48:04.384 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:04.384 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:48:04.485 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:48:04.514 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:02.918458+0000 osd.0 (osd.0) 163 : cluster [DBG] 6.0 scrub starts 2024-02-23T00:48:04.514 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:02.920079+0000 osd.0 (osd.0) 164 : cluster [DBG] 6.0 scrub ok 2024-02-23T00:48:04.514 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:03.020040+0000 osd.3 (osd.3) 186 : cluster [DBG] 4.19 scrub starts 2024-02-23T00:48:04.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:03.023055+0000 osd.3 (osd.3) 187 : cluster [DBG] 4.19 scrub ok 2024-02-23T00:48:04.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:03.225944+0000 mon.a (mon.0) 2547 : cluster [DBG] osdmap e411: 8 total, 8 up, 7 in 2024-02-23T00:48:04.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:03.285234+0000 mgr.y (mgr.14517) 376 : cluster [DBG] pgmap v592: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-23T00:48:04.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:03.338941+0000 osd.5 (osd.5) 172 : cluster [DBG] 18.7 scrub starts 2024-02-23T00:48:04.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:03.345536+0000 osd.5 (osd.5) 173 : cluster [DBG] 18.7 scrub ok 2024-02-23T00:48:04.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:03.532625+0000 osd.4 (osd.4) 159 : cluster [DBG] 5.1c scrub starts 2024-02-23T00:48:04.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:03.533551+0000 osd.4 (osd.4) 160 : cluster [DBG] 5.1c scrub ok 2024-02-23T00:48:04.516 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:04.536 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:48:04.539 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:04.539 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:48:04.640 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:48:04.671 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:04.692 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:48:04.696 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:04.697 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:48:04.798 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:48:04.828 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:04.849 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:48:04.852 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:04.852 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:48:04.953 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:48:04.982 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:05.231 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:48:05.233 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:05.234 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:48:05.335 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:48:05.344 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:48:05.344 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:48:05.363 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:05.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:03.796185+0000 osd.1 (osd.1) 173 : cluster [DBG] 5.c scrub starts 2024-02-23T00:48:05.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:03.800724+0000 osd.1 (osd.1) 174 : cluster [DBG] 5.c scrub ok 2024-02-23T00:48:05.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:04.232367+0000 mon.a (mon.0) 2548 : cluster [DBG] osdmap e412: 8 total, 8 up, 7 in 2024-02-23T00:48:05.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:04.234985+0000 mon.a (mon.0) 2549 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:48:05.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:05.180607+0000 mon.a (mon.0) 2550 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:48:05.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:05.180794+0000 mon.a (mon.0) 2551 : cluster [DBG] osdmap e413: 8 total, 8 up, 7 in 2024-02-23T00:48:05.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:05.185436+0000 mon.a (mon.0) 2552 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:48:05.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:05.191690+0000 mon.a (mon.0) 2553 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-23T00:48:05.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:03.796185+0000 osd.1 (osd.1) 173 : cluster [DBG] 5.c scrub starts 2024-02-23T00:48:05.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:03.800724+0000 osd.1 (osd.1) 174 : cluster [DBG] 5.c scrub ok 2024-02-23T00:48:05.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:04.232367+0000 mon.a (mon.0) 2548 : cluster [DBG] osdmap e412: 8 total, 8 up, 7 in 2024-02-23T00:48:05.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:04.234985+0000 mon.a (mon.0) 2549 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:48:05.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:05.180607+0000 mon.a (mon.0) 2550 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:48:05.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:05.180794+0000 mon.a (mon.0) 2551 : cluster [DBG] osdmap e413: 8 total, 8 up, 7 in 2024-02-23T00:48:05.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:05.185436+0000 mon.a (mon.0) 2552 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:48:05.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:05.191690+0000 mon.a (mon.0) 2553 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-23T00:48:05.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:03.796185+0000 osd.1 (osd.1) 173 : cluster [DBG] 5.c scrub starts 2024-02-23T00:48:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:03.800724+0000 osd.1 (osd.1) 174 : cluster [DBG] 5.c scrub ok 2024-02-23T00:48:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:04.232367+0000 mon.a (mon.0) 2548 : cluster [DBG] osdmap e412: 8 total, 8 up, 7 in 2024-02-23T00:48:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:04.234985+0000 mon.a (mon.0) 2549 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:48:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:05.180607+0000 mon.a (mon.0) 2550 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:48:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:05.180794+0000 mon.a (mon.0) 2551 : cluster [DBG] osdmap e413: 8 total, 8 up, 7 in 2024-02-23T00:48:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:05.185436+0000 mon.a (mon.0) 2552 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:48:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:05.191690+0000 mon.a (mon.0) 2553 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-23T00:48:06.199 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.1 dump_historic_ops 2024-02-23T00:48:06.513 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:48:06.516 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:04.586128+0000 osd.6 (osd.6) 163 : cluster [DBG] 62.2 scrub starts 2024-02-23T00:48:06.516 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:04.587877+0000 osd.6 (osd.6) 164 : cluster [DBG] 62.2 scrub ok 2024-02-23T00:48:06.516 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:04.831408+0000 osd.1 (osd.1) 175 : cluster [DBG] 2.15 scrub starts 2024-02-23T00:48:06.516 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:04.832900+0000 osd.1 (osd.1) 176 : cluster [DBG] 2.15 scrub ok 2024-02-23T00:48:06.516 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:04.988498+0000 osd.3 (osd.3) 188 : cluster [DBG] 18.16 scrub starts 2024-02-23T00:48:06.516 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:04.989510+0000 osd.3 (osd.3) 189 : cluster [DBG] 18.16 scrub ok 2024-02-23T00:48:06.516 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:05.285594+0000 mgr.y (mgr.14517) 377 : cluster [DBG] pgmap v595: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-23T00:48:06.516 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:05.978116+0000 osd.3 (osd.3) 190 : cluster [DBG] 18.9 scrub starts 2024-02-23T00:48:06.516 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:05.979333+0000 osd.3 (osd.3) 191 : cluster [DBG] 18.9 scrub ok 2024-02-23T00:48:06.517 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:06.193449+0000 mon.a (mon.0) 2554 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-23T00:48:06.517 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:06.193529+0000 mon.a (mon.0) 2555 : cluster [DBG] osdmap e414: 8 total, 8 up, 7 in 2024-02-23T00:48:06.517 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:06.229790+0000 mon.a (mon.0) 2556 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:48:06.517 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:06.230181+0000 mon.a (mon.0) 2557 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-59"}]: dispatch 2024-02-23T00:48:06.517 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:04.586128+0000 osd.6 (osd.6) 163 : cluster [DBG] 62.2 scrub starts 2024-02-23T00:48:06.517 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:04.587877+0000 osd.6 (osd.6) 164 : cluster [DBG] 62.2 scrub ok 2024-02-23T00:48:06.517 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:04.831408+0000 osd.1 (osd.1) 175 : cluster [DBG] 2.15 scrub starts 2024-02-23T00:48:06.517 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:04.832900+0000 osd.1 (osd.1) 176 : cluster [DBG] 2.15 scrub ok 2024-02-23T00:48:06.517 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:04.988498+0000 osd.3 (osd.3) 188 : cluster [DBG] 18.16 scrub starts 2024-02-23T00:48:06.517 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:04.989510+0000 osd.3 (osd.3) 189 : cluster [DBG] 18.16 scrub ok 2024-02-23T00:48:06.518 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:05.285594+0000 mgr.y (mgr.14517) 377 : cluster [DBG] pgmap v595: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-23T00:48:06.518 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:05.978116+0000 osd.3 (osd.3) 190 : cluster [DBG] 18.9 scrub starts 2024-02-23T00:48:06.518 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:05.979333+0000 osd.3 (osd.3) 191 : cluster [DBG] 18.9 scrub ok 2024-02-23T00:48:06.518 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:06.193449+0000 mon.a (mon.0) 2554 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-23T00:48:06.518 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:06.193529+0000 mon.a (mon.0) 2555 : cluster [DBG] osdmap e414: 8 total, 8 up, 7 in 2024-02-23T00:48:06.518 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:06.229790+0000 mon.a (mon.0) 2556 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:48:06.518 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:06.230181+0000 mon.a (mon.0) 2557 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-59"}]: dispatch 2024-02-23T00:48:06.518 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:06.519 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:48:06.519 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:48:06.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.552+0000 7fa8b9ec4700 1 -- 172.21.15.47:0/552995164 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b40fe510 msgr2=0x7fa8b40f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:06.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.552+0000 7fa8b9ec4700 1 --2- 172.21.15.47:0/552995164 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b40fe510 0x7fa8b40f7d40 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 rx=0x7fa8a80047b0 tx=0x7fa8a8009cb0).stop 2024-02-23T00:48:06.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.552+0000 7fa8b9ec4700 1 -- 172.21.15.47:0/552995164 shutdown_connections 2024-02-23T00:48:06.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.552+0000 7fa8b9ec4700 1 --2- 172.21.15.47:0/552995164 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa8b40f8420 0x7fa8b4104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:06.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.552+0000 7fa8b9ec4700 1 --2- 172.21.15.47:0/552995164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8b40fdbb0 0x7fa8b40fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:06.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.552+0000 7fa8b9ec4700 1 --2- 172.21.15.47:0/552995164 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b40fe510 0x7fa8b40f7d40 unknown :-1 s=CLOSED pgs=2529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:06.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.552+0000 7fa8b9ec4700 1 -- 172.21.15.47:0/552995164 >> 172.21.15.47:0/552995164 conn(0x7fa8b40f3800 msgr2=0x7fa8b40f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:48:06.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.552+0000 7fa8b9ec4700 1 -- 172.21.15.47:0/552995164 shutdown_connections 2024-02-23T00:48:06.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.552+0000 7fa8b9ec4700 1 -- 172.21.15.47:0/552995164 wait complete. 2024-02-23T00:48:06.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.552+0000 7fa8b9ec4700 1 librados: starting msgr at 2024-02-23T00:48:06.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.552+0000 7fa8b9ec4700 1 librados: starting objecter 2024-02-23T00:48:06.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.553+0000 7fa8b9ec4700 1 Processor -- start 2024-02-23T00:48:06.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.553+0000 7fa8b9ec4700 1 -- start start 2024-02-23T00:48:06.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.553+0000 7fa8b9ec4700 1 librados: setting wanted keys 2024-02-23T00:48:06.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.553+0000 7fa8b9ec4700 1 librados: calling monclient init 2024-02-23T00:48:06.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.553+0000 7fa8b9ec4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8b40f8420 0x7fa8b410de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:06.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.553+0000 7fa8b9ec4700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa8b40fdbb0 0x7fa8b410e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:06.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.553+0000 7fa8b9ec4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b40fe510 0x7fa8b4108070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:06.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.553+0000 7fa8b9ec4700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b4108560 con 0x7fa8b40fe510 2024-02-23T00:48:06.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.553+0000 7fa8b9ec4700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b41086d0 con 0x7fa8b40f8420 2024-02-23T00:48:06.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.553+0000 7fa8b9ec4700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b4108890 con 0x7fa8b40fdbb0 2024-02-23T00:48:06.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.553+0000 7fa8b2ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa8b40fdbb0 0x7fa8b410e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:06.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.553+0000 7fa8b3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b40fe510 0x7fa8b4108070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:06.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.553+0000 7fa8b2ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa8b40fdbb0 0x7fa8b410e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:34134/0 (socket says 172.21.15.47:34134) 2024-02-23T00:48:06.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.553+0000 7fa8b2ffd700 1 -- 172.21.15.47:0/523225916 learned_addr learned my addr 172.21.15.47:0/523225916 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:48:06.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.553+0000 7fa8b3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b40fe510 0x7fa8b4108070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40700/0 (socket says 172.21.15.47:40700) 2024-02-23T00:48:06.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.553+0000 7fa8b37fe700 1 --2- 172.21.15.47:0/523225916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8b40f8420 0x7fa8b410de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:06.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.554+0000 7fa8b2ffd700 1 -- 172.21.15.47:0/523225916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8b40f8420 msgr2=0x7fa8b410de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:06.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.554+0000 7fa8b2ffd700 1 --2- 172.21.15.47:0/523225916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8b40f8420 0x7fa8b410de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:06.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.554+0000 7fa8b2ffd700 1 -- 172.21.15.47:0/523225916 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b40fe510 msgr2=0x7fa8b4108070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:06.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.554+0000 7fa8b2ffd700 1 --2- 172.21.15.47:0/523225916 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b40fe510 0x7fa8b4108070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:06.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.554+0000 7fa8b2ffd700 1 -- 172.21.15.47:0/523225916 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8ac0097c0 con 0x7fa8b40fdbb0 2024-02-23T00:48:06.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.554+0000 7fa8b37fe700 1 --2- 172.21.15.47:0/523225916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8b40f8420 0x7fa8b410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:48:06.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.554+0000 7fa8b3fff700 1 --2- 172.21.15.47:0/523225916 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b40fe510 0x7fa8b4108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:48:06.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.554+0000 7fa8b2ffd700 1 --2- 172.21.15.47:0/523225916 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa8b40fdbb0 0x7fa8b410e370 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 rx=0x7fa8a80044e0 tx=0x7fa8a8009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:06.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.554+0000 7fa8b0ff9700 10 client.?.objecter ms_handle_connect 0x7fa8b40fdbb0 2024-02-23T00:48:06.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.554+0000 7fa8b0ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:48:06.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.554+0000 7fa8b0ff9700 1 -- 172.21.15.47:0/523225916 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a8018070 con 0x7fa8b40fdbb0 2024-02-23T00:48:06.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.554+0000 7fa8b9ec4700 1 -- 172.21.15.47:0/523225916 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8a8005040 con 0x7fa8b40fdbb0 2024-02-23T00:48:06.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.554+0000 7fa8b0ff9700 1 -- 172.21.15.47:0/523225916 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fa8a80096d0 con 0x7fa8b40fdbb0 2024-02-23T00:48:06.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.554+0000 7fa8b0ff9700 1 -- 172.21.15.47:0/523225916 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a801dc10 con 0x7fa8b40fdbb0 2024-02-23T00:48:06.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.554+0000 7fa8b9ec4700 10 client.41318.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:48:06.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.554+0000 7fa8b9ec4700 1 -- 172.21.15.47:0/523225916 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8b4108db0 con 0x7fa8b40fdbb0 2024-02-23T00:48:06.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.554+0000 7fa8b9ec4700 1 librados: init done 2024-02-23T00:48:06.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.555+0000 7fa89a7fc700 1 -- 172.21.15.47:0/523225916 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa894004f20 con 0x7fa8b40fdbb0 2024-02-23T00:48:06.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.555+0000 7fa8b0ff9700 1 -- 172.21.15.47:0/523225916 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fa8a8009240 con 0x7fa8b40fdbb0 2024-02-23T00:48:06.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.556+0000 7fa8b0ff9700 1 --2- 172.21.15.47:0/523225916 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa89c062cc0 0x7fa89c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:06.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.556+0000 7fa8b0ff9700 1 -- 172.21.15.47:0/523225916 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(414..414 src has 48..414) v4 ==== 9220+0+0 (secure 0 0 0) 0x7fa8a807ec00 con 0x7fa8b40fdbb0 2024-02-23T00:48:06.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.556+0000 7fa8b0ff9700 10 client.41318.objecter ms_dispatch 0x7fa8b4001550 osd_map(414..414 src has 48..414) v4 2024-02-23T00:48:06.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.556+0000 7fa8b0ff9700 3 client.41318.objecter handle_osd_map got epochs [414,414] > 0 2024-02-23T00:48:06.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.556+0000 7fa8b0ff9700 3 client.41318.objecter handle_osd_map decoding full epoch 414 2024-02-23T00:48:06.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.556+0000 7fa8b37fe700 1 --2- 172.21.15.47:0/523225916 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa89c062cc0 0x7fa89c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:06.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.556+0000 7fa8b0ff9700 20 client.41318.objecter dump_active .. 0 homeless 2024-02-23T00:48:06.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.556+0000 7fa8b37fe700 1 --2- 172.21.15.47:0/523225916 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa89c062cc0 0x7fa89c065180 secure :-1 s=READY pgs=3061 cs=0 l=1 rev1=1 rx=0x7fa8a4004ae0 tx=0x7fa8a4009040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:06.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.556+0000 7fa8b0ff9700 10 client.41318.objecter ms_handle_connect 0x7fa89c062cc0 2024-02-23T00:48:06.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.561+0000 7fa8b0ff9700 1 -- 172.21.15.47:0/523225916 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fa8a804e210 con 0x7fa8b40fdbb0 2024-02-23T00:48:06.620 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:48:06.648 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:04.586128+0000 osd.6 (osd.6) 163 : cluster [DBG] 62.2 scrub starts 2024-02-23T00:48:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:04.587877+0000 osd.6 (osd.6) 164 : cluster [DBG] 62.2 scrub ok 2024-02-23T00:48:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:04.831408+0000 osd.1 (osd.1) 175 : cluster [DBG] 2.15 scrub starts 2024-02-23T00:48:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:04.832900+0000 osd.1 (osd.1) 176 : cluster [DBG] 2.15 scrub ok 2024-02-23T00:48:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:04.988498+0000 osd.3 (osd.3) 188 : cluster [DBG] 18.16 scrub starts 2024-02-23T00:48:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:04.989510+0000 osd.3 (osd.3) 189 : cluster [DBG] 18.16 scrub ok 2024-02-23T00:48:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:05.285594+0000 mgr.y (mgr.14517) 377 : cluster [DBG] pgmap v595: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-23T00:48:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:05.978116+0000 osd.3 (osd.3) 190 : cluster [DBG] 18.9 scrub starts 2024-02-23T00:48:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:05.979333+0000 osd.3 (osd.3) 191 : cluster [DBG] 18.9 scrub ok 2024-02-23T00:48:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:06.193449+0000 mon.a (mon.0) 2554 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-23T00:48:06.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:06.193529+0000 mon.a (mon.0) 2555 : cluster [DBG] osdmap e414: 8 total, 8 up, 7 in 2024-02-23T00:48:06.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:06.229790+0000 mon.a (mon.0) 2556 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:48:06.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:06.230181+0000 mon.a (mon.0) 2557 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-59"}]: dispatch 2024-02-23T00:48:06.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:06.745+0000 7fa89a7fc700 1 -- 172.21.15.47:0/523225916 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7fa8940051d0 con 0x7fa8b40fdbb0 2024-02-23T00:48:06.847 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:48:06.850 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:06.850 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:48:06.951 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:48:06.980 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:07.129 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:48:07.132 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:07.132 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:48:07.233 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:48:07.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.258+0000 7fa8b0ff9700 1 -- 172.21.15.47:0/523225916 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v415) v1 ==== 101+0+0 (secure 0 0 0) 0x7fa8a8051830 con 0x7fa8b40fdbb0 2024-02-23T00:48:07.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.261+0000 7fa89a7fc700 10 librados: watch_flush enter 2024-02-23T00:48:07.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.261+0000 7fa89a7fc700 10 librados: watch_flush exit 2024-02-23T00:48:07.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.261+0000 7fa89a7fc700 20 client.41318.objecter shutdown clearing up homeless session... 2024-02-23T00:48:07.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.261+0000 7fa89a7fc700 10 client.41318.objecter successfully canceled tick 2024-02-23T00:48:07.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.261+0000 7fa89a7fc700 1 -- 172.21.15.47:0/523225916 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa89c062cc0 msgr2=0x7fa89c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:07.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.262+0000 7fa89a7fc700 1 --2- 172.21.15.47:0/523225916 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa89c062cc0 0x7fa89c065180 secure :-1 s=READY pgs=3061 cs=0 l=1 rev1=1 rx=0x7fa8a4004ae0 tx=0x7fa8a4009040).stop 2024-02-23T00:48:07.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.262+0000 7fa89a7fc700 1 -- 172.21.15.47:0/523225916 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa8b40fdbb0 msgr2=0x7fa8b410e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:07.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.262+0000 7fa89a7fc700 1 --2- 172.21.15.47:0/523225916 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa8b40fdbb0 0x7fa8b410e370 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 rx=0x7fa8a80044e0 tx=0x7fa8a8009cb0).stop 2024-02-23T00:48:07.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.265+0000 7fa89a7fc700 1 -- 172.21.15.47:0/523225916 shutdown_connections 2024-02-23T00:48:07.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.265+0000 7fa89a7fc700 1 --2- 172.21.15.47:0/523225916 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b40fe510 0x7fa8b4108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:07.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.265+0000 7fa89a7fc700 1 --2- 172.21.15.47:0/523225916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8b40f8420 0x7fa8b410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:07.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.265+0000 7fa89a7fc700 1 --2- 172.21.15.47:0/523225916 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa89c062cc0 0x7fa89c065180 unknown :-1 s=CLOSED pgs=3061 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:07.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.265+0000 7fa89a7fc700 1 --2- 172.21.15.47:0/523225916 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa8b40fdbb0 0x7fa8b410e370 unknown :-1 s=CLOSED pgs=2661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:07.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.265+0000 7fa89a7fc700 1 -- 172.21.15.47:0/523225916 >> 172.21.15.47:0/523225916 conn(0x7fa8b40f3800 msgr2=0x7fa8b40fb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:48:07.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.266+0000 7fa89a7fc700 1 -- 172.21.15.47:0/523225916 shutdown_connections 2024-02-23T00:48:07.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.269+0000 7fa89a7fc700 1 -- 172.21.15.47:0/523225916 wait complete. 2024-02-23T00:48:07.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.269+0000 7fa89a7fc700 1 librados: shutdown 2024-02-23T00:48:07.271 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:07.272 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is unset 2024-02-23T00:48:07.345 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:48:07.348 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:07.348 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:48:07.449 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:48:07.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.448+0000 7f7cd9057700 1 -- 172.21.15.47:0/3651147282 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7cd40fdbb0 msgr2=0x7f7cd40fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:07.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.448+0000 7f7cd9057700 1 --2- 172.21.15.47:0/3651147282 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7cd40fdbb0 0x7f7cd40fdf90 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 rx=0x7f7cc4004870 tx=0x7f7cc4009cb0).stop 2024-02-23T00:48:07.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.448+0000 7f7cd9057700 1 -- 172.21.15.47:0/3651147282 shutdown_connections 2024-02-23T00:48:07.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.448+0000 7f7cd9057700 1 --2- 172.21.15.47:0/3651147282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cd40f8420 0x7f7cd4104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:07.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.448+0000 7f7cd9057700 1 --2- 172.21.15.47:0/3651147282 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7cd40fdbb0 0x7f7cd40fdf90 unknown :-1 s=CLOSED pgs=2662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:07.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.448+0000 7f7cd9057700 1 --2- 172.21.15.47:0/3651147282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cd40fe510 0x7f7cd40f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.448+0000 7f7cd9057700 1 -- 172.21.15.47:0/3651147282 >> 172.21.15.47:0/3651147282 conn(0x7f7cd40f3800 msgr2=0x7f7cd40f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:48:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.448+0000 7f7cd9057700 1 -- 172.21.15.47:0/3651147282 shutdown_connections 2024-02-23T00:48:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.448+0000 7f7cd9057700 1 -- 172.21.15.47:0/3651147282 wait complete. 2024-02-23T00:48:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.448+0000 7f7cd9057700 1 librados: starting msgr at 2024-02-23T00:48:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.448+0000 7f7cd9057700 1 librados: starting objecter 2024-02-23T00:48:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.448+0000 7f7cd9057700 1 Processor -- start 2024-02-23T00:48:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.449+0000 7f7cd9057700 1 -- start start 2024-02-23T00:48:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.449+0000 7f7cd9057700 1 librados: setting wanted keys 2024-02-23T00:48:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.449+0000 7f7cd9057700 1 librados: calling monclient init 2024-02-23T00:48:07.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.449+0000 7f7cd9057700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cd40f8420 0x7f7cd406b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:07.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.449+0000 7f7cd9057700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cd40fdbb0 0x7f7cd406b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:07.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.449+0000 7f7cd9057700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7cd40fe510 0x7f7cd406bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:07.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.449+0000 7f7cd9057700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd4116310 con 0x7f7cd40f8420 2024-02-23T00:48:07.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.449+0000 7f7cd9057700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd4116480 con 0x7f7cd40fdbb0 2024-02-23T00:48:07.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.449+0000 7f7cd9057700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd4116640 con 0x7f7cd40fe510 2024-02-23T00:48:07.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.449+0000 7f7cd2d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cd40f8420 0x7f7cd406b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:07.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.449+0000 7f7cd359e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7cd40fe510 0x7f7cd406bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:07.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.449+0000 7f7cd2d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cd40f8420 0x7f7cd406b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40722/0 (socket says 172.21.15.47:40722) 2024-02-23T00:48:07.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.449+0000 7f7cd2d9d700 1 -- 172.21.15.47:0/2331031863 learned_addr learned my addr 172.21.15.47:0/2331031863 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:48:07.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.449+0000 7f7cd359e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7cd40fe510 0x7f7cd406bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:34146/0 (socket says 172.21.15.47:34146) 2024-02-23T00:48:07.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.449+0000 7f7cd259c700 1 --2- 172.21.15.47:0/2331031863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cd40fdbb0 0x7f7cd406b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:07.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.449+0000 7f7cd359e700 1 -- 172.21.15.47:0/2331031863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cd40fdbb0 msgr2=0x7f7cd406b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:07.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.449+0000 7f7cd359e700 1 --2- 172.21.15.47:0/2331031863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cd40fdbb0 0x7f7cd406b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:07.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.449+0000 7f7cd359e700 1 -- 172.21.15.47:0/2331031863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cd40f8420 msgr2=0x7f7cd406b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:07.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.449+0000 7f7cd359e700 1 --2- 172.21.15.47:0/2331031863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cd40f8420 0x7f7cd406b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:07.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.449+0000 7f7cd359e700 1 -- 172.21.15.47:0/2331031863 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc4005040 con 0x7f7cd40fe510 2024-02-23T00:48:07.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.450+0000 7f7cd359e700 1 --2- 172.21.15.47:0/2331031863 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7cd40fe510 0x7f7cd406bab0 secure :-1 s=READY pgs=2663 cs=0 l=1 rev1=1 rx=0x7f7ccc00c8e0 tx=0x7f7ccc004ca0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:07.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.450+0000 7f7cbbfff700 10 client.?.objecter ms_handle_connect 0x7f7cd40fe510 2024-02-23T00:48:07.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.450+0000 7f7cbbfff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:48:07.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.450+0000 7f7cbbfff700 1 -- 172.21.15.47:0/2331031863 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc00e4a0 con 0x7f7cd40fe510 2024-02-23T00:48:07.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.450+0000 7f7cbbfff700 1 -- 172.21.15.47:0/2331031863 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f7ccc00ea90 con 0x7f7cd40fe510 2024-02-23T00:48:07.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.450+0000 7f7cd9057700 1 -- 172.21.15.47:0/2331031863 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cd4116920 con 0x7f7cd40fe510 2024-02-23T00:48:07.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.450+0000 7f7cbbfff700 1 -- 172.21.15.47:0/2331031863 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc0129b0 con 0x7f7cd40fe510 2024-02-23T00:48:07.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.450+0000 7f7cd9057700 10 client.41324.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:48:07.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.450+0000 7f7cd9057700 1 -- 172.21.15.47:0/2331031863 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cd4116e90 con 0x7f7cd40fe510 2024-02-23T00:48:07.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.450+0000 7f7cd9057700 1 librados: init done 2024-02-23T00:48:07.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.451+0000 7f7cd9057700 1 -- 172.21.15.47:0/2331031863 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cd4106ed0 con 0x7f7cd40fe510 2024-02-23T00:48:07.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.452+0000 7f7cbbfff700 1 -- 172.21.15.47:0/2331031863 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f7ccc00e600 con 0x7f7cd40fe510 2024-02-23T00:48:07.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.452+0000 7f7cbbfff700 1 --2- 172.21.15.47:0/2331031863 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7cbc062cc0 0x7f7cbc065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:07.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.452+0000 7f7cbbfff700 1 -- 172.21.15.47:0/2331031863 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(415..415 src has 48..415) v4 ==== 8835+0+0 (secure 0 0 0) 0x7f7ccc07cff0 con 0x7f7cd40fe510 2024-02-23T00:48:07.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.452+0000 7f7cbbfff700 10 client.41324.objecter ms_dispatch 0x7f7cd4001550 osd_map(415..415 src has 48..415) v4 2024-02-23T00:48:07.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.452+0000 7f7cbbfff700 3 client.41324.objecter handle_osd_map got epochs [415,415] > 0 2024-02-23T00:48:07.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.452+0000 7f7cbbfff700 3 client.41324.objecter handle_osd_map decoding full epoch 415 2024-02-23T00:48:07.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.452+0000 7f7cd2d9d700 1 --2- 172.21.15.47:0/2331031863 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7cbc062cc0 0x7f7cbc065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:07.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.452+0000 7f7cbbfff700 20 client.41324.objecter dump_active .. 0 homeless 2024-02-23T00:48:07.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.453+0000 7f7cbbfff700 10 client.41324.objecter ms_handle_connect 0x7f7cbc062cc0 2024-02-23T00:48:07.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.453+0000 7f7cd2d9d700 1 --2- 172.21.15.47:0/2331031863 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7cbc062cc0 0x7f7cbc065180 secure :-1 s=READY pgs=3062 cs=0 l=1 rev1=1 rx=0x7f7cc4009da0 tx=0x7f7cc4009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:07.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.457+0000 7f7cbbfff700 1 -- 172.21.15.47:0/2331031863 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f7ccc048f60 con 0x7f7cd40fe510 2024-02-23T00:48:07.481 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:07.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.636+0000 7f7cd9057700 1 -- 172.21.15.47:0/2331031863 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 1.0, "target": ["mon-mgr", ""]}) v1 -- 0x7f7cd405fa60 con 0x7f7cbc062cc0 2024-02-23T00:48:07.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.637+0000 7f7cbbfff700 1 -- 172.21.15.47:0/2331031863 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 no change) v1 ==== 17+0+294 (secure 0 0 0) 0x7f7cd405fa60 con 0x7f7cbc062cc0 2024-02-23T00:48:07.639 INFO:teuthology.orchestra.run.smithi047.stdout:moved 0 / 528 (0%) 2024-02-23T00:48:07.639 INFO:teuthology.orchestra.run.smithi047.stdout:avg 75.4286 2024-02-23T00:48:07.639 INFO:teuthology.orchestra.run.smithi047.stdout:stddev 8.39825 -> 8.39825 (expected baseline 8.04071) 2024-02-23T00:48:07.639 INFO:teuthology.orchestra.run.smithi047.stdout:min osd.5 with 62 -> 62 pgs (0.82197 -> 0.82197 * mean) 2024-02-23T00:48:07.639 INFO:teuthology.orchestra.run.smithi047.stdout:max osd.7 with 88 -> 88 pgs (1.16667 -> 1.16667 * mean) 2024-02-23T00:48:07.639 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:48:07.639 INFO:teuthology.orchestra.run.smithi047.stdout:oload 110 2024-02-23T00:48:07.639 INFO:teuthology.orchestra.run.smithi047.stdout:max_change 1 2024-02-23T00:48:07.640 INFO:teuthology.orchestra.run.smithi047.stdout:max_change_osds 4 2024-02-23T00:48:07.640 INFO:teuthology.orchestra.run.smithi047.stdout:average_utilization 0.0033 2024-02-23T00:48:07.640 INFO:teuthology.orchestra.run.smithi047.stdout:overload_utilization 0.0036 2024-02-23T00:48:07.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.639+0000 7f7cd9057700 10 librados: watch_flush enter 2024-02-23T00:48:07.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.639+0000 7f7cd9057700 10 librados: watch_flush exit 2024-02-23T00:48:07.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.639+0000 7f7cd9057700 20 client.41324.objecter shutdown clearing up homeless session... 2024-02-23T00:48:07.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.639+0000 7f7cd9057700 10 client.41324.objecter successfully canceled tick 2024-02-23T00:48:07.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.639+0000 7f7cd9057700 1 -- 172.21.15.47:0/2331031863 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7cbc062cc0 msgr2=0x7f7cbc065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:07.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.639+0000 7f7cd9057700 1 --2- 172.21.15.47:0/2331031863 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7cbc062cc0 0x7f7cbc065180 secure :-1 s=READY pgs=3062 cs=0 l=1 rev1=1 rx=0x7f7cc4009da0 tx=0x7f7cc4009cb0).stop 2024-02-23T00:48:07.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.639+0000 7f7cd9057700 1 -- 172.21.15.47:0/2331031863 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7cd40fe510 msgr2=0x7f7cd406bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:07.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.639+0000 7f7cd9057700 1 --2- 172.21.15.47:0/2331031863 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7cd40fe510 0x7f7cd406bab0 secure :-1 s=READY pgs=2663 cs=0 l=1 rev1=1 rx=0x7f7ccc00c8e0 tx=0x7f7ccc004ca0).stop 2024-02-23T00:48:07.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.639+0000 7f7cd9057700 1 -- 172.21.15.47:0/2331031863 shutdown_connections 2024-02-23T00:48:07.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.639+0000 7f7cd9057700 1 --2- 172.21.15.47:0/2331031863 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7cbc062cc0 0x7f7cbc065180 unknown :-1 s=CLOSED pgs=3062 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:07.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.639+0000 7f7cd9057700 1 --2- 172.21.15.47:0/2331031863 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7cd40fe510 0x7f7cd406bab0 unknown :-1 s=CLOSED pgs=2663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:07.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.639+0000 7f7cd9057700 1 --2- 172.21.15.47:0/2331031863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7cd40f8420 0x7f7cd406b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:07.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.639+0000 7f7cd9057700 1 --2- 172.21.15.47:0/2331031863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cd40fdbb0 0x7f7cd406b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:07.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.639+0000 7f7cd9057700 1 -- 172.21.15.47:0/2331031863 >> 172.21.15.47:0/2331031863 conn(0x7f7cd40f3800 msgr2=0x7f7cd40f5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:48:07.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.639+0000 7f7cd9057700 1 -- 172.21.15.47:0/2331031863 shutdown_connections 2024-02-23T00:48:07.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.639+0000 7f7cd9057700 1 -- 172.21.15.47:0/2331031863 wait complete. 2024-02-23T00:48:07.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:07.639+0000 7f7cd9057700 1 librados: shutdown 2024-02-23T00:48:07.643 INFO:teuthology.orchestra.run.smithi047.stderr:no change 2024-02-23T00:48:07.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:06.746720+0000 mon.c (mon.2) 298 : audit [INF] from='client.? 172.21.15.47:0/523225916' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:48:07.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:06.747018+0000 mon.a (mon.0) 2558 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:48:07.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:07.253613+0000 mon.a (mon.0) 2559 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:48:07.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:07.257820+0000 mon.a (mon.0) 2560 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:48:07.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:07.257887+0000 mon.a (mon.0) 2561 : cluster [DBG] osdmap e415: 8 total, 8 up, 7 in 2024-02-23T00:48:07.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:06.746720+0000 mon.c (mon.2) 298 : audit [INF] from='client.? 172.21.15.47:0/523225916' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:48:07.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:06.747018+0000 mon.a (mon.0) 2558 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:48:07.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:07.253613+0000 mon.a (mon.0) 2559 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:48:07.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:07.257820+0000 mon.a (mon.0) 2560 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:48:07.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:07.257887+0000 mon.a (mon.0) 2561 : cluster [DBG] osdmap e415: 8 total, 8 up, 7 in 2024-02-23T00:48:08.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:06.746720+0000 mon.c (mon.2) 298 : audit [INF] from='client.? 172.21.15.47:0/523225916' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:48:08.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:06.747018+0000 mon.a (mon.0) 2558 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:48:08.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:07.253613+0000 mon.a (mon.0) 2559 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:48:08.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:07.257820+0000 mon.a (mon.0) 2560 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:48:08.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:07.257887+0000 mon.a (mon.0) 2561 : cluster [DBG] osdmap e415: 8 total, 8 up, 7 in 2024-02-23T00:48:08.474 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-02-23T00:48:08.475 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight-by-utilization 1000 0.05 2024-02-23T00:48:08.724 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:06.906473+0000 osd.0 (osd.0) 165 : cluster [DBG] 18.1d scrub starts 2024-02-23T00:48:08.724 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:06.907962+0000 osd.0 (osd.0) 166 : cluster [DBG] 18.1d scrub ok 2024-02-23T00:48:08.724 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:07.285937+0000 mgr.y (mgr.14517) 378 : cluster [DBG] pgmap v598: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-23T00:48:08.724 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:07.637166+0000 mgr.y (mgr.14517) 379 : audit [DBG] from='client.41324 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 1.0, "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:48:08.725 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:07.939279+0000 osd.3 (osd.3) 192 : cluster [DBG] 4.17 deep-scrub starts 2024-02-23T00:48:08.725 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:07.941096+0000 osd.3 (osd.3) 193 : cluster [DBG] 4.17 deep-scrub ok 2024-02-23T00:48:08.725 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:08.259791+0000 mon.a (mon.0) 2562 : cluster [DBG] osdmap e416: 8 total, 8 up, 7 in 2024-02-23T00:48:08.725 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:08.263968+0000 mon.a (mon.0) 2563 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:48:08.734 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:48:08.737 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:08.738 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:48:08.839 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:48:08.869 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:09.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:06.906473+0000 osd.0 (osd.0) 165 : cluster [DBG] 18.1d scrub starts 2024-02-23T00:48:09.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:06.907962+0000 osd.0 (osd.0) 166 : cluster [DBG] 18.1d scrub ok 2024-02-23T00:48:09.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:07.285937+0000 mgr.y (mgr.14517) 378 : cluster [DBG] pgmap v598: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-23T00:48:09.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:07.637166+0000 mgr.y (mgr.14517) 379 : audit [DBG] from='client.41324 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 1.0, "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:48:09.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:07.939279+0000 osd.3 (osd.3) 192 : cluster [DBG] 4.17 deep-scrub starts 2024-02-23T00:48:09.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:07.941096+0000 osd.3 (osd.3) 193 : cluster [DBG] 4.17 deep-scrub ok 2024-02-23T00:48:09.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:08.259791+0000 mon.a (mon.0) 2562 : cluster [DBG] osdmap e416: 8 total, 8 up, 7 in 2024-02-23T00:48:09.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:08.263968+0000 mon.a (mon.0) 2563 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:48:09.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:06.906473+0000 osd.0 (osd.0) 165 : cluster [DBG] 18.1d scrub starts 2024-02-23T00:48:09.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:06.907962+0000 osd.0 (osd.0) 166 : cluster [DBG] 18.1d scrub ok 2024-02-23T00:48:09.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:07.285937+0000 mgr.y (mgr.14517) 378 : cluster [DBG] pgmap v598: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-23T00:48:09.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:07.637166+0000 mgr.y (mgr.14517) 379 : audit [DBG] from='client.41324 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 1.0, "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:48:09.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:07.939279+0000 osd.3 (osd.3) 192 : cluster [DBG] 4.17 deep-scrub starts 2024-02-23T00:48:09.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:07.941096+0000 osd.3 (osd.3) 193 : cluster [DBG] 4.17 deep-scrub ok 2024-02-23T00:48:09.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:08.259791+0000 mon.a (mon.0) 2562 : cluster [DBG] osdmap e416: 8 total, 8 up, 7 in 2024-02-23T00:48:09.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:08.263968+0000 mon.a (mon.0) 2563 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:48:09.819 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:48:09.822 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:09.823 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:48:09.826 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:48:09.924 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:48:09.954 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:10.269 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:09.261788+0000 mon.a (mon.0) 2564 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:48:10.269 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:09.261914+0000 mon.a (mon.0) 2565 : cluster [DBG] osdmap e417: 8 total, 8 up, 7 in 2024-02-23T00:48:10.269 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:09.264720+0000 mon.a (mon.0) 2566 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:48:10.269 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:09.270122+0000 mon.a (mon.0) 2567 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-23T00:48:10.269 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:09.286323+0000 mgr.y (mgr.14517) 380 : cluster [DBG] pgmap v601: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-23T00:48:10.269 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:09.798283+0000 osd.1 (osd.1) 177 : cluster [DBG] 2.9 scrub starts 2024-02-23T00:48:10.269 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:09.801247+0000 osd.1 (osd.1) 178 : cluster [DBG] 2.9 scrub ok 2024-02-23T00:48:10.405 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:48:10.407 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:10.408 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:48:10.509 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:48:10.538 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:09.261788+0000 mon.a (mon.0) 2564 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:48:10.538 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:09.261914+0000 mon.a (mon.0) 2565 : cluster [DBG] osdmap e417: 8 total, 8 up, 7 in 2024-02-23T00:48:10.538 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:09.264720+0000 mon.a (mon.0) 2566 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:48:10.538 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:09.270122+0000 mon.a (mon.0) 2567 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-23T00:48:10.539 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:09.286323+0000 mgr.y (mgr.14517) 380 : cluster [DBG] pgmap v601: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-23T00:48:10.539 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:09.798283+0000 osd.1 (osd.1) 177 : cluster [DBG] 2.9 scrub starts 2024-02-23T00:48:10.539 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:09.801247+0000 osd.1 (osd.1) 178 : cluster [DBG] 2.9 scrub ok 2024-02-23T00:48:10.540 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:10.557 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:48:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:48:10] "GET /metrics HTTP/1.1" 200 188809 "" "Prometheus/2.33.4" 2024-02-23T00:48:10.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:09.261788+0000 mon.a (mon.0) 2564 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:48:10.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:09.261914+0000 mon.a (mon.0) 2565 : cluster [DBG] osdmap e417: 8 total, 8 up, 7 in 2024-02-23T00:48:10.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:09.264720+0000 mon.a (mon.0) 2566 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:48:10.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:09.270122+0000 mon.a (mon.0) 2567 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-23T00:48:10.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:09.286323+0000 mgr.y (mgr.14517) 380 : cluster [DBG] pgmap v601: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-23T00:48:10.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:09.798283+0000 osd.1 (osd.1) 177 : cluster [DBG] 2.9 scrub starts 2024-02-23T00:48:10.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:09.801247+0000 osd.1 (osd.1) 178 : cluster [DBG] 2.9 scrub ok 2024-02-23T00:48:10.561 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:48:10.565 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:10.565 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:48:10.666 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:48:10.696 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:10.699 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:48:10.717 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:48:10.720 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:10.720 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:48:10.821 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:48:10.849 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:10.868 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:48:10.871 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:10.871 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:48:10.972 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:48:11.005 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:11.025 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:48:11.027 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:11.028 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:48:11.128 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:48:11.155 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:11.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.264+0000 7fe674649700 1 -- 172.21.15.47:0/574491581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe66c0fe510 msgr2=0x7fe66c0f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:11.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.264+0000 7fe674649700 1 --2- 172.21.15.47:0/574491581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe66c0fe510 0x7fe66c0f7d40 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 rx=0x7fe65c004870 tx=0x7fe65c009cb0).stop 2024-02-23T00:48:11.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.265+0000 7fe674649700 1 -- 172.21.15.47:0/574491581 shutdown_connections 2024-02-23T00:48:11.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.265+0000 7fe674649700 1 --2- 172.21.15.47:0/574491581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe66c0f8420 0x7fe66c104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:11.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.265+0000 7fe674649700 1 --2- 172.21.15.47:0/574491581 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe66c0fdbb0 0x7fe66c0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:11.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.265+0000 7fe674649700 1 --2- 172.21.15.47:0/574491581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe66c0fe510 0x7fe66c0f7d40 secure :-1 s=CLOSED pgs=2530 cs=0 l=1 rev1=1 rx=0x7fe65c004870 tx=0x7fe65c009cb0).stop 2024-02-23T00:48:11.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.265+0000 7fe674649700 1 -- 172.21.15.47:0/574491581 >> 172.21.15.47:0/574491581 conn(0x7fe66c0f3800 msgr2=0x7fe66c0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:48:11.267 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:09.896356+0000 osd.7 (osd.7) 197 : cluster [DBG] 62.1f scrub starts 2024-02-23T00:48:11.267 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:09.898580+0000 osd.7 (osd.7) 198 : cluster [DBG] 62.1f scrub ok 2024-02-23T00:48:11.267 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:10.267371+0000 mon.a (mon.0) 2568 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-23T00:48:11.268 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:10.267442+0000 mon.a (mon.0) 2569 : cluster [DBG] osdmap e418: 8 total, 8 up, 7 in 2024-02-23T00:48:11.268 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:10.492352+0000 osd.4 (osd.4) 161 : cluster [DBG] 5.1 deep-scrub starts 2024-02-23T00:48:11.268 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:10.493335+0000 osd.4 (osd.4) 162 : cluster [DBG] 5.1 deep-scrub ok 2024-02-23T00:48:11.268 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:10.932548+0000 osd.3 (osd.3) 194 : cluster [DBG] 18.a scrub starts 2024-02-23T00:48:11.268 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:10.933372+0000 osd.3 (osd.3) 195 : cluster [DBG] 18.a scrub ok 2024-02-23T00:48:11.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.267+0000 7fe674649700 1 -- 172.21.15.47:0/574491581 shutdown_connections 2024-02-23T00:48:11.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.267+0000 7fe674649700 1 -- 172.21.15.47:0/574491581 wait complete. 2024-02-23T00:48:11.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.268+0000 7fe674649700 1 librados: starting msgr at 2024-02-23T00:48:11.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.268+0000 7fe674649700 1 librados: starting objecter 2024-02-23T00:48:11.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.268+0000 7fe674649700 1 Processor -- start 2024-02-23T00:48:11.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.268+0000 7fe674649700 1 -- start start 2024-02-23T00:48:11.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.268+0000 7fe674649700 1 librados: setting wanted keys 2024-02-23T00:48:11.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.268+0000 7fe674649700 1 librados: calling monclient init 2024-02-23T00:48:11.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.268+0000 7fe674649700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe66c0f8420 0x7fe66c10e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:11.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.268+0000 7fe674649700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe66c0fdbb0 0x7fe66c10e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:11.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.268+0000 7fe674649700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe66c1081f0 0x7fe66c108670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:11.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.268+0000 7fe674649700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c108b60 con 0x7fe66c0f8420 2024-02-23T00:48:11.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.268+0000 7fe674649700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c108cd0 con 0x7fe66c1081f0 2024-02-23T00:48:11.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.268+0000 7fe674649700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c108e90 con 0x7fe66c0fdbb0 2024-02-23T00:48:11.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.269+0000 7fe6723e5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe66c0f8420 0x7fe66c10e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:11.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.269+0000 7fe6723e5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe66c0f8420 0x7fe66c10e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40742/0 (socket says 172.21.15.47:40742) 2024-02-23T00:48:11.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.269+0000 7fe6723e5700 1 -- 172.21.15.47:0/3395954453 learned_addr learned my addr 172.21.15.47:0/3395954453 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:48:11.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.269+0000 7fe672be6700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe66c1081f0 0x7fe66c108670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:11.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.269+0000 7fe6723e5700 1 -- 172.21.15.47:0/3395954453 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe66c0fdbb0 msgr2=0x7fe66c10e540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-23T00:48:11.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.269+0000 7fe6723e5700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe66c0fdbb0 0x7fe66c10e540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:48:11.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.269+0000 7fe6723e5700 1 -- 172.21.15.47:0/3395954453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe66c1081f0 msgr2=0x7fe66c108670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:11.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.269+0000 7fe6723e5700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe66c1081f0 0x7fe66c108670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:11.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.269+0000 7fe6723e5700 1 -- 172.21.15.47:0/3395954453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe668005040 con 0x7fe66c0f8420 2024-02-23T00:48:11.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.269+0000 7fe6723e5700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe66c0f8420 0x7fe66c10e050 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 rx=0x7fe6680048f0 tx=0x7fe668009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:11.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.271+0000 7fe6637fe700 10 client.?.objecter ms_handle_connect 0x7fe66c0f8420 2024-02-23T00:48:11.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.271+0000 7fe6637fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:48:11.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.271+0000 7fe6637fe700 1 -- 172.21.15.47:0/3395954453 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66800e660 con 0x7fe66c0f8420 2024-02-23T00:48:11.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.271+0000 7fe6637fe700 1 -- 172.21.15.47:0/3395954453 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fe66800f040 con 0x7fe66c0f8420 2024-02-23T00:48:11.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.271+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe65c005040 con 0x7fe66c0f8420 2024-02-23T00:48:11.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.271+0000 7fe6637fe700 1 -- 172.21.15.47:0/3395954453 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe668014930 con 0x7fe66c0f8420 2024-02-23T00:48:11.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.271+0000 7fe674649700 10 client.31002.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:48:11.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.271+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe66c06b1a0 con 0x7fe66c0f8420 2024-02-23T00:48:11.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.271+0000 7fe674649700 1 librados: init done 2024-02-23T00:48:11.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.273+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fe66c05ee80 con 0x7fe66c0f8420 2024-02-23T00:48:11.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.274+0000 7fe6637fe700 1 -- 172.21.15.47:0/3395954453 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fe66800e7c0 con 0x7fe66c0f8420 2024-02-23T00:48:11.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.275+0000 7fe6637fe700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe658062cc0 0x7fe658065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:11.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.275+0000 7fe6637fe700 1 -- 172.21.15.47:0/3395954453 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(419..419 src has 48..419) v4 ==== 9220+0+0 (secure 0 0 0) 0x7fe66807d390 con 0x7fe66c0f8420 2024-02-23T00:48:11.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.275+0000 7fe6637fe700 10 client.31002.objecter ms_dispatch 0x7fe66c001550 osd_map(419..419 src has 48..419) v4 2024-02-23T00:48:11.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.275+0000 7fe6637fe700 3 client.31002.objecter handle_osd_map got epochs [419,419] > 0 2024-02-23T00:48:11.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.275+0000 7fe6637fe700 3 client.31002.objecter handle_osd_map decoding full epoch 419 2024-02-23T00:48:11.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.275+0000 7fe671be4700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe658062cc0 0x7fe658065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:11.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.275+0000 7fe6637fe700 20 client.31002.objecter dump_active .. 0 homeless 2024-02-23T00:48:11.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.275+0000 7fe6637fe700 10 client.31002.objecter ms_handle_connect 0x7fe658062cc0 2024-02-23T00:48:11.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.275+0000 7fe671be4700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe658062cc0 0x7fe658065180 secure :-1 s=READY pgs=3063 cs=0 l=1 rev1=1 rx=0x7fe65c004840 tx=0x7fe65c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:11.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.278+0000 7fe6637fe700 1 -- 172.21.15.47:0/3395954453 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v419) v1 ==== 54+0+16 (secure 0 0 0) 0x7fe668084070 con 0x7fe66c0f8420 2024-02-23T00:48:11.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.278+0000 7fe674649700 10 client.31002.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:11.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.278+0000 7fe674649700 15 client.31002.objecter _session_command_op_assign -1 1 2024-02-23T00:48:11.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.278+0000 7fe674649700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fe66c028540 0x7fe66c0fe910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:11.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.278+0000 7fe674649700 20 client.31002.objecter _get_session s=0x7fe66c106e20 osd=0 3 2024-02-23T00:48:11.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.278+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c106e20 osd=0 3 2024-02-23T00:48:11.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.278+0000 7fe674649700 20 client.31002.objecter _get_session s=0x7fe66c106e20 osd=0 3 2024-02-23T00:48:11.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.278+0000 7fe674649700 15 client.31002.objecter _session_command_op_remove -1 1 2024-02-23T00:48:11.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.278+0000 7fe674649700 20 client.31002.objecter get_session s=0x7fe66c106e20 osd=0 3 2024-02-23T00:48:11.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.278+0000 7fe674649700 15 client.31002.objecter _session_command_op_assign 0 1 2024-02-23T00:48:11.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.278+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c106e20 osd=0 4 2024-02-23T00:48:11.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.278+0000 7fe674649700 10 client.31002.objecter _send_command 1 2024-02-23T00:48:11.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.278+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe66c1096d0 con 0x7fe66c028540 2024-02-23T00:48:11.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.278+0000 7fe672be6700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fe66c028540 0x7fe66c0fe910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:11.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.280+0000 7fe6637fe700 10 client.31002.objecter ms_handle_connect 0x7fe66c028540 2024-02-23T00:48:11.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.280+0000 7fe672be6700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fe66c028540 0x7fe66c0fe910 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:11.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.280+0000 7fe6637fe700 1 -- 172.21.15.47:0/3395954453 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe66c1096d0 con 0x7fe66c028540 2024-02-23T00:48:11.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.280+0000 7fe6637fe700 10 client.31002.objecter ms_dispatch 0x7fe66c001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:48:11.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.280+0000 7fe6637fe700 10 client.31002.objecter _finish_command 1 = system:0 2024-02-23T00:48:11.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.280+0000 7fe6637fe700 20 client.31002.objecter put_session s=0x7fe66c106e20 osd=0 4 2024-02-23T00:48:11.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.280+0000 7fe6637fe700 15 client.31002.objecter _session_command_op_remove 0 1 2024-02-23T00:48:11.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.296+0000 7fe674649700 10 client.31002.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:48:11.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.296+0000 7fe674649700 15 client.31002.objecter _session_command_op_assign -1 2 2024-02-23T00:48:11.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.296+0000 7fe674649700 20 client.31002.objecter _get_session s=0x7fe66c106e20 osd=0 3 2024-02-23T00:48:11.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.296+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c106e20 osd=0 3 2024-02-23T00:48:11.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.296+0000 7fe674649700 20 client.31002.objecter _get_session s=0x7fe66c106e20 osd=0 3 2024-02-23T00:48:11.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.296+0000 7fe674649700 15 client.31002.objecter _session_command_op_remove -1 2 2024-02-23T00:48:11.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.296+0000 7fe674649700 20 client.31002.objecter get_session s=0x7fe66c106e20 osd=0 3 2024-02-23T00:48:11.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.296+0000 7fe674649700 15 client.31002.objecter _session_command_op_assign 0 2 2024-02-23T00:48:11.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.296+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c106e20 osd=0 4 2024-02-23T00:48:11.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.296+0000 7fe674649700 10 client.31002.objecter _send_command 2 2024-02-23T00:48:11.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.296+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fe66c05a040 con 0x7fe66c028540 2024-02-23T00:48:11.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.297+0000 7fe6637fe700 1 -- 172.21.15.47:0/3395954453 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fe66c05a040 con 0x7fe66c028540 2024-02-23T00:48:11.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.297+0000 7fe6637fe700 10 client.31002.objecter ms_dispatch 0x7fe66c001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:48:11.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.297+0000 7fe6637fe700 10 client.31002.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:48:11.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.297+0000 7fe6637fe700 20 client.31002.objecter put_session s=0x7fe66c106e20 osd=0 4 2024-02-23T00:48:11.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.297+0000 7fe6637fe700 15 client.31002.objecter _session_command_op_remove 0 2 2024-02-23T00:48:11.299 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:48:11.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.297+0000 7fe674649700 10 client.31002.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:11.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.297+0000 7fe674649700 15 client.31002.objecter _session_command_op_assign -1 3 2024-02-23T00:48:11.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.297+0000 7fe674649700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fe66c027b90 0x7fe66c027fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:11.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.297+0000 7fe674649700 20 client.31002.objecter _get_session s=0x7fe66c0fff80 osd=1 3 2024-02-23T00:48:11.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.297+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c0fff80 osd=1 3 2024-02-23T00:48:11.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.297+0000 7fe674649700 20 client.31002.objecter _get_session s=0x7fe66c0fff80 osd=1 3 2024-02-23T00:48:11.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.297+0000 7fe674649700 15 client.31002.objecter _session_command_op_remove -1 3 2024-02-23T00:48:11.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.297+0000 7fe674649700 20 client.31002.objecter get_session s=0x7fe66c0fff80 osd=1 3 2024-02-23T00:48:11.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.297+0000 7fe674649700 15 client.31002.objecter _session_command_op_assign 1 3 2024-02-23T00:48:11.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.297+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c0fff80 osd=1 4 2024-02-23T00:48:11.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.297+0000 7fe674649700 10 client.31002.objecter _send_command 3 2024-02-23T00:48:11.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.297+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe66c1000f0 con 0x7fe66c027b90 2024-02-23T00:48:11.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.298+0000 7fe6723e5700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fe66c027b90 0x7fe66c027fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:11.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.298+0000 7fe6637fe700 10 client.31002.objecter ms_handle_connect 0x7fe66c027b90 2024-02-23T00:48:11.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.298+0000 7fe6723e5700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fe66c027b90 0x7fe66c027fb0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:11.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.298+0000 7fe6637fe700 1 -- 172.21.15.47:0/3395954453 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe66c1000f0 con 0x7fe66c027b90 2024-02-23T00:48:11.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.298+0000 7fe6637fe700 10 client.31002.objecter ms_dispatch 0x7fe66c001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:48:11.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.298+0000 7fe6637fe700 10 client.31002.objecter _finish_command 3 = system:0 2024-02-23T00:48:11.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.298+0000 7fe6637fe700 20 client.31002.objecter put_session s=0x7fe66c0fff80 osd=1 4 2024-02-23T00:48:11.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.298+0000 7fe6637fe700 15 client.31002.objecter _session_command_op_remove 1 3 2024-02-23T00:48:11.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.314+0000 7fe674649700 10 client.31002.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:48:11.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.314+0000 7fe674649700 15 client.31002.objecter _session_command_op_assign -1 4 2024-02-23T00:48:11.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.314+0000 7fe674649700 20 client.31002.objecter _get_session s=0x7fe66c0fff80 osd=1 3 2024-02-23T00:48:11.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.314+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c0fff80 osd=1 3 2024-02-23T00:48:11.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.314+0000 7fe674649700 20 client.31002.objecter _get_session s=0x7fe66c0fff80 osd=1 3 2024-02-23T00:48:11.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.314+0000 7fe674649700 15 client.31002.objecter _session_command_op_remove -1 4 2024-02-23T00:48:11.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.314+0000 7fe674649700 20 client.31002.objecter get_session s=0x7fe66c0fff80 osd=1 3 2024-02-23T00:48:11.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.314+0000 7fe674649700 15 client.31002.objecter _session_command_op_assign 1 4 2024-02-23T00:48:11.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.314+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c0fff80 osd=1 4 2024-02-23T00:48:11.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.314+0000 7fe674649700 10 client.31002.objecter _send_command 4 2024-02-23T00:48:11.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.314+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fe66c027600 con 0x7fe66c027b90 2024-02-23T00:48:11.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.316+0000 7fe6637fe700 1 -- 172.21.15.47:0/3395954453 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fe66c027600 con 0x7fe66c027b90 2024-02-23T00:48:11.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.316+0000 7fe6637fe700 10 client.31002.objecter ms_dispatch 0x7fe66c001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:48:11.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.316+0000 7fe6637fe700 10 client.31002.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:48:11.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.316+0000 7fe6637fe700 20 client.31002.objecter put_session s=0x7fe66c0fff80 osd=1 4 2024-02-23T00:48:11.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.316+0000 7fe6637fe700 15 client.31002.objecter _session_command_op_remove 1 4 2024-02-23T00:48:11.318 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:48:11.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.316+0000 7fe674649700 10 client.31002.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:11.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.316+0000 7fe674649700 15 client.31002.objecter _session_command_op_assign -1 5 2024-02-23T00:48:11.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.316+0000 7fe674649700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fe66c0ffb30 0x7fe66c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:11.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.316+0000 7fe674649700 20 client.31002.objecter _get_session s=0x7fe66c027a10 osd=2 3 2024-02-23T00:48:11.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.316+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c027a10 osd=2 3 2024-02-23T00:48:11.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.316+0000 7fe674649700 20 client.31002.objecter _get_session s=0x7fe66c027a10 osd=2 3 2024-02-23T00:48:11.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.316+0000 7fe674649700 15 client.31002.objecter _session_command_op_remove -1 5 2024-02-23T00:48:11.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.316+0000 7fe674649700 20 client.31002.objecter get_session s=0x7fe66c027a10 osd=2 3 2024-02-23T00:48:11.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.316+0000 7fe674649700 15 client.31002.objecter _session_command_op_assign 2 5 2024-02-23T00:48:11.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.316+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c027a10 osd=2 4 2024-02-23T00:48:11.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.316+0000 7fe674649700 10 client.31002.objecter _send_command 5 2024-02-23T00:48:11.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.316+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe66c06ccb0 con 0x7fe66c0ffb30 2024-02-23T00:48:11.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.317+0000 7fe671be4700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fe66c0ffb30 0x7fe66c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:11.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.317+0000 7fe6637fe700 10 client.31002.objecter ms_handle_connect 0x7fe66c0ffb30 2024-02-23T00:48:11.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.317+0000 7fe671be4700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fe66c0ffb30 0x7fe66c06c540 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:11.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.317+0000 7fe6637fe700 1 -- 172.21.15.47:0/3395954453 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe66c06ccb0 con 0x7fe66c0ffb30 2024-02-23T00:48:11.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.317+0000 7fe6637fe700 10 client.31002.objecter ms_dispatch 0x7fe66c001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:48:11.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.317+0000 7fe6637fe700 10 client.31002.objecter _finish_command 5 = system:0 2024-02-23T00:48:11.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.317+0000 7fe6637fe700 20 client.31002.objecter put_session s=0x7fe66c027a10 osd=2 4 2024-02-23T00:48:11.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.317+0000 7fe6637fe700 15 client.31002.objecter _session_command_op_remove 2 5 2024-02-23T00:48:11.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.332+0000 7fe674649700 10 client.31002.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:48:11.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.332+0000 7fe674649700 15 client.31002.objecter _session_command_op_assign -1 6 2024-02-23T00:48:11.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.332+0000 7fe674649700 20 client.31002.objecter _get_session s=0x7fe66c027a10 osd=2 3 2024-02-23T00:48:11.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.332+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c027a10 osd=2 3 2024-02-23T00:48:11.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.332+0000 7fe674649700 20 client.31002.objecter _get_session s=0x7fe66c027a10 osd=2 3 2024-02-23T00:48:11.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.332+0000 7fe674649700 15 client.31002.objecter _session_command_op_remove -1 6 2024-02-23T00:48:11.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.332+0000 7fe674649700 20 client.31002.objecter get_session s=0x7fe66c027a10 osd=2 3 2024-02-23T00:48:11.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.332+0000 7fe674649700 15 client.31002.objecter _session_command_op_assign 2 6 2024-02-23T00:48:11.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.332+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c027a10 osd=2 4 2024-02-23T00:48:11.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.332+0000 7fe674649700 10 client.31002.objecter _send_command 6 2024-02-23T00:48:11.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.332+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fe66c11f2c0 con 0x7fe66c0ffb30 2024-02-23T00:48:11.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.334+0000 7fe6637fe700 1 -- 172.21.15.47:0/3395954453 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fe66c11f2c0 con 0x7fe66c0ffb30 2024-02-23T00:48:11.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.334+0000 7fe6637fe700 10 client.31002.objecter ms_dispatch 0x7fe66c001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:48:11.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.334+0000 7fe6637fe700 10 client.31002.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:48:11.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.334+0000 7fe6637fe700 20 client.31002.objecter put_session s=0x7fe66c027a10 osd=2 4 2024-02-23T00:48:11.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.334+0000 7fe6637fe700 15 client.31002.objecter _session_command_op_remove 2 6 2024-02-23T00:48:11.336 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:48:11.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.334+0000 7fe674649700 10 client.31002.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:11.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.334+0000 7fe674649700 15 client.31002.objecter _session_command_op_assign -1 7 2024-02-23T00:48:11.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.334+0000 7fe674649700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fe66c0ff1b0 0x7fe66c122620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:11.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.334+0000 7fe674649700 20 client.31002.objecter _get_session s=0x7fe66c0ff040 osd=3 3 2024-02-23T00:48:11.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.334+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c0ff040 osd=3 3 2024-02-23T00:48:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.334+0000 7fe674649700 20 client.31002.objecter _get_session s=0x7fe66c0ff040 osd=3 3 2024-02-23T00:48:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.334+0000 7fe674649700 15 client.31002.objecter _session_command_op_remove -1 7 2024-02-23T00:48:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.334+0000 7fe674649700 20 client.31002.objecter get_session s=0x7fe66c0ff040 osd=3 3 2024-02-23T00:48:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.334+0000 7fe674649700 15 client.31002.objecter _session_command_op_assign 3 7 2024-02-23T00:48:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.334+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c0ff040 osd=3 4 2024-02-23T00:48:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.334+0000 7fe674649700 10 client.31002.objecter _send_command 7 2024-02-23T00:48:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.334+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe66c06c1f0 con 0x7fe66c0ff1b0 2024-02-23T00:48:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.334+0000 7fe672be6700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fe66c0ff1b0 0x7fe66c122620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.335+0000 7fe6637fe700 10 client.31002.objecter ms_handle_connect 0x7fe66c0ff1b0 2024-02-23T00:48:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.335+0000 7fe672be6700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fe66c0ff1b0 0x7fe66c122620 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.335+0000 7fe6637fe700 1 -- 172.21.15.47:0/3395954453 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe66c06c1f0 con 0x7fe66c0ff1b0 2024-02-23T00:48:11.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.335+0000 7fe6637fe700 10 client.31002.objecter ms_dispatch 0x7fe66c001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:48:11.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.335+0000 7fe6637fe700 10 client.31002.objecter _finish_command 7 = system:0 2024-02-23T00:48:11.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.335+0000 7fe6637fe700 20 client.31002.objecter put_session s=0x7fe66c0ff040 osd=3 4 2024-02-23T00:48:11.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.335+0000 7fe6637fe700 15 client.31002.objecter _session_command_op_remove 3 7 2024-02-23T00:48:11.350 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set noscrub 2024-02-23T00:48:11.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.350+0000 7fe674649700 10 client.31002.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:48:11.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.350+0000 7fe674649700 15 client.31002.objecter _session_command_op_assign -1 8 2024-02-23T00:48:11.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.350+0000 7fe674649700 20 client.31002.objecter _get_session s=0x7fe66c0ff040 osd=3 3 2024-02-23T00:48:11.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.350+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c0ff040 osd=3 3 2024-02-23T00:48:11.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.350+0000 7fe674649700 20 client.31002.objecter _get_session s=0x7fe66c0ff040 osd=3 3 2024-02-23T00:48:11.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.350+0000 7fe674649700 15 client.31002.objecter _session_command_op_remove -1 8 2024-02-23T00:48:11.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.350+0000 7fe674649700 20 client.31002.objecter get_session s=0x7fe66c0ff040 osd=3 3 2024-02-23T00:48:11.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.350+0000 7fe674649700 15 client.31002.objecter _session_command_op_assign 3 8 2024-02-23T00:48:11.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.350+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c0ff040 osd=3 4 2024-02-23T00:48:11.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.350+0000 7fe674649700 10 client.31002.objecter _send_command 8 2024-02-23T00:48:11.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.350+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fe66c06c360 con 0x7fe66c0ff1b0 2024-02-23T00:48:11.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.351+0000 7fe6637fe700 1 -- 172.21.15.47:0/3395954453 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fe66c06c360 con 0x7fe66c0ff1b0 2024-02-23T00:48:11.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.351+0000 7fe6637fe700 10 client.31002.objecter ms_dispatch 0x7fe66c001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:48:11.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.351+0000 7fe6637fe700 10 client.31002.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:48:11.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.352+0000 7fe6637fe700 20 client.31002.objecter put_session s=0x7fe66c0ff040 osd=3 4 2024-02-23T00:48:11.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.352+0000 7fe6637fe700 15 client.31002.objecter _session_command_op_remove 3 8 2024-02-23T00:48:11.355 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:48:11.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.352+0000 7fe674649700 10 client.31002.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:11.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.352+0000 7fe674649700 15 client.31002.objecter _session_command_op_assign -1 9 2024-02-23T00:48:11.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.352+0000 7fe674649700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fe66c11fc90 0x7fe66c1200d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:11.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.352+0000 7fe674649700 20 client.31002.objecter _get_session s=0x7fe66c06c360 osd=4 3 2024-02-23T00:48:11.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.352+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c06c360 osd=4 3 2024-02-23T00:48:11.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.352+0000 7fe674649700 20 client.31002.objecter _get_session s=0x7fe66c06c360 osd=4 3 2024-02-23T00:48:11.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.352+0000 7fe674649700 15 client.31002.objecter _session_command_op_remove -1 9 2024-02-23T00:48:11.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.352+0000 7fe674649700 20 client.31002.objecter get_session s=0x7fe66c06c360 osd=4 3 2024-02-23T00:48:11.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.352+0000 7fe674649700 15 client.31002.objecter _session_command_op_assign 4 9 2024-02-23T00:48:11.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.352+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c06c360 osd=4 4 2024-02-23T00:48:11.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.352+0000 7fe674649700 10 client.31002.objecter _send_command 9 2024-02-23T00:48:11.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.352+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe66c1285c0 con 0x7fe66c11fc90 2024-02-23T00:48:11.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.352+0000 7fe6723e5700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fe66c11fc90 0x7fe66c1200d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:11.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.353+0000 7fe6637fe700 10 client.31002.objecter ms_handle_connect 0x7fe66c11fc90 2024-02-23T00:48:11.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.353+0000 7fe6723e5700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fe66c11fc90 0x7fe66c1200d0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:11.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.353+0000 7fe6637fe700 1 -- 172.21.15.47:0/3395954453 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe66c1285c0 con 0x7fe66c11fc90 2024-02-23T00:48:11.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.353+0000 7fe6637fe700 10 client.31002.objecter ms_dispatch 0x7fe66c001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:48:11.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.353+0000 7fe6637fe700 10 client.31002.objecter _finish_command 9 = system:0 2024-02-23T00:48:11.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.353+0000 7fe6637fe700 20 client.31002.objecter put_session s=0x7fe66c06c360 osd=4 4 2024-02-23T00:48:11.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.353+0000 7fe6637fe700 15 client.31002.objecter _session_command_op_remove 4 9 2024-02-23T00:48:11.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.368+0000 7fe674649700 10 client.31002.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:48:11.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.368+0000 7fe674649700 15 client.31002.objecter _session_command_op_assign -1 10 2024-02-23T00:48:11.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.368+0000 7fe674649700 20 client.31002.objecter _get_session s=0x7fe66c06c360 osd=4 3 2024-02-23T00:48:11.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.368+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c06c360 osd=4 3 2024-02-23T00:48:11.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.368+0000 7fe674649700 20 client.31002.objecter _get_session s=0x7fe66c06c360 osd=4 3 2024-02-23T00:48:11.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.368+0000 7fe674649700 15 client.31002.objecter _session_command_op_remove -1 10 2024-02-23T00:48:11.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.368+0000 7fe674649700 20 client.31002.objecter get_session s=0x7fe66c06c360 osd=4 3 2024-02-23T00:48:11.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.368+0000 7fe674649700 15 client.31002.objecter _session_command_op_assign 4 10 2024-02-23T00:48:11.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.368+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c06c360 osd=4 4 2024-02-23T00:48:11.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.368+0000 7fe674649700 10 client.31002.objecter _send_command 10 2024-02-23T00:48:11.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.368+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fe66c127350 con 0x7fe66c11fc90 2024-02-23T00:48:11.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.370+0000 7fe6637fe700 1 -- 172.21.15.47:0/3395954453 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fe66c127350 con 0x7fe66c11fc90 2024-02-23T00:48:11.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.370+0000 7fe6637fe700 10 client.31002.objecter ms_dispatch 0x7fe66c001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:48:11.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.370+0000 7fe6637fe700 10 client.31002.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:48:11.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.370+0000 7fe6637fe700 20 client.31002.objecter put_session s=0x7fe66c06c360 osd=4 4 2024-02-23T00:48:11.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.370+0000 7fe6637fe700 15 client.31002.objecter _session_command_op_remove 4 10 2024-02-23T00:48:11.372 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:48:11.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.370+0000 7fe674649700 10 client.31002.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:11.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.370+0000 7fe674649700 15 client.31002.objecter _session_command_op_assign -1 11 2024-02-23T00:48:11.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.370+0000 7fe674649700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fe66c127a80 0x7fe66c127f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:11.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.370+0000 7fe674649700 20 client.31002.objecter _get_session s=0x7fe66c127350 osd=5 3 2024-02-23T00:48:11.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.370+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c127350 osd=5 3 2024-02-23T00:48:11.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.370+0000 7fe674649700 20 client.31002.objecter _get_session s=0x7fe66c127350 osd=5 3 2024-02-23T00:48:11.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.370+0000 7fe674649700 15 client.31002.objecter _session_command_op_remove -1 11 2024-02-23T00:48:11.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.370+0000 7fe674649700 20 client.31002.objecter get_session s=0x7fe66c127350 osd=5 3 2024-02-23T00:48:11.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.370+0000 7fe674649700 15 client.31002.objecter _session_command_op_assign 5 11 2024-02-23T00:48:11.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.370+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c127350 osd=5 4 2024-02-23T00:48:11.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.370+0000 7fe674649700 10 client.31002.objecter _send_command 11 2024-02-23T00:48:11.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.370+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe66c127000 con 0x7fe66c127a80 2024-02-23T00:48:11.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.371+0000 7fe671be4700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fe66c127a80 0x7fe66c127f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:11.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.371+0000 7fe671be4700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fe66c127a80 0x7fe66c127f00 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:11.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.371+0000 7fe6637fe700 10 client.31002.objecter ms_handle_connect 0x7fe66c127a80 2024-02-23T00:48:11.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.372+0000 7fe6637fe700 1 -- 172.21.15.47:0/3395954453 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe66c127000 con 0x7fe66c127a80 2024-02-23T00:48:11.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.372+0000 7fe6637fe700 10 client.31002.objecter ms_dispatch 0x7fe66c001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:48:11.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.372+0000 7fe6637fe700 10 client.31002.objecter _finish_command 11 = system:0 2024-02-23T00:48:11.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.372+0000 7fe6637fe700 20 client.31002.objecter put_session s=0x7fe66c127350 osd=5 4 2024-02-23T00:48:11.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.372+0000 7fe6637fe700 15 client.31002.objecter _session_command_op_remove 5 11 2024-02-23T00:48:11.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.387+0000 7fe674649700 10 client.31002.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:48:11.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.387+0000 7fe674649700 15 client.31002.objecter _session_command_op_assign -1 12 2024-02-23T00:48:11.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.387+0000 7fe674649700 20 client.31002.objecter _get_session s=0x7fe66c127350 osd=5 3 2024-02-23T00:48:11.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.387+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c127350 osd=5 3 2024-02-23T00:48:11.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.387+0000 7fe674649700 20 client.31002.objecter _get_session s=0x7fe66c127350 osd=5 3 2024-02-23T00:48:11.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.387+0000 7fe674649700 15 client.31002.objecter _session_command_op_remove -1 12 2024-02-23T00:48:11.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.387+0000 7fe674649700 20 client.31002.objecter get_session s=0x7fe66c127350 osd=5 3 2024-02-23T00:48:11.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.387+0000 7fe674649700 15 client.31002.objecter _session_command_op_assign 5 12 2024-02-23T00:48:11.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.387+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c127350 osd=5 4 2024-02-23T00:48:11.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.387+0000 7fe674649700 10 client.31002.objecter _send_command 12 2024-02-23T00:48:11.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.387+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fe66c1271a0 con 0x7fe66c127a80 2024-02-23T00:48:11.390 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:48:11.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.389+0000 7fe6637fe700 1 -- 172.21.15.47:0/3395954453 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fe65c009240 con 0x7fe66c127a80 2024-02-23T00:48:11.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.389+0000 7fe6637fe700 10 client.31002.objecter ms_dispatch 0x7fe66c001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:48:11.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.389+0000 7fe6637fe700 10 client.31002.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:48:11.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.389+0000 7fe6637fe700 20 client.31002.objecter put_session s=0x7fe66c127350 osd=5 4 2024-02-23T00:48:11.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.389+0000 7fe6637fe700 15 client.31002.objecter _session_command_op_remove 5 12 2024-02-23T00:48:11.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.389+0000 7fe674649700 10 client.31002.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:11.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.389+0000 7fe674649700 15 client.31002.objecter _session_command_op_assign -1 13 2024-02-23T00:48:11.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.389+0000 7fe674649700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fe66c1262a0 0x7fe66c126700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:11.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.389+0000 7fe674649700 20 client.31002.objecter _get_session s=0x7fe66c126fc0 osd=6 3 2024-02-23T00:48:11.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.389+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c126fc0 osd=6 3 2024-02-23T00:48:11.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.389+0000 7fe674649700 20 client.31002.objecter _get_session s=0x7fe66c126fc0 osd=6 3 2024-02-23T00:48:11.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.389+0000 7fe674649700 15 client.31002.objecter _session_command_op_remove -1 13 2024-02-23T00:48:11.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.389+0000 7fe674649700 20 client.31002.objecter get_session s=0x7fe66c126fc0 osd=6 3 2024-02-23T00:48:11.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.389+0000 7fe674649700 15 client.31002.objecter _session_command_op_assign 6 13 2024-02-23T00:48:11.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.389+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c126fc0 osd=6 4 2024-02-23T00:48:11.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.389+0000 7fe674649700 10 client.31002.objecter _send_command 13 2024-02-23T00:48:11.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.389+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe66c12ed00 con 0x7fe66c1262a0 2024-02-23T00:48:11.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.389+0000 7fe672be6700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fe66c1262a0 0x7fe66c126700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:11.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.390+0000 7fe6637fe700 10 client.31002.objecter ms_handle_connect 0x7fe66c1262a0 2024-02-23T00:48:11.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.390+0000 7fe672be6700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fe66c1262a0 0x7fe66c126700 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:11.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.390+0000 7fe6637fe700 1 -- 172.21.15.47:0/3395954453 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe66c12ed00 con 0x7fe66c1262a0 2024-02-23T00:48:11.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.390+0000 7fe6637fe700 10 client.31002.objecter ms_dispatch 0x7fe66c001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:48:11.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.390+0000 7fe6637fe700 10 client.31002.objecter _finish_command 13 = system:0 2024-02-23T00:48:11.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.390+0000 7fe6637fe700 20 client.31002.objecter put_session s=0x7fe66c126fc0 osd=6 4 2024-02-23T00:48:11.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.390+0000 7fe6637fe700 15 client.31002.objecter _session_command_op_remove 6 13 2024-02-23T00:48:11.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.405+0000 7fe674649700 10 client.31002.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:48:11.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.405+0000 7fe674649700 15 client.31002.objecter _session_command_op_assign -1 14 2024-02-23T00:48:11.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.405+0000 7fe674649700 20 client.31002.objecter _get_session s=0x7fe66c126fc0 osd=6 3 2024-02-23T00:48:11.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.405+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c126fc0 osd=6 3 2024-02-23T00:48:11.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.405+0000 7fe674649700 20 client.31002.objecter _get_session s=0x7fe66c126fc0 osd=6 3 2024-02-23T00:48:11.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.405+0000 7fe674649700 15 client.31002.objecter _session_command_op_remove -1 14 2024-02-23T00:48:11.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.405+0000 7fe674649700 20 client.31002.objecter get_session s=0x7fe66c126fc0 osd=6 3 2024-02-23T00:48:11.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.405+0000 7fe674649700 15 client.31002.objecter _session_command_op_assign 6 14 2024-02-23T00:48:11.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.405+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c126fc0 osd=6 4 2024-02-23T00:48:11.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.405+0000 7fe674649700 10 client.31002.objecter _send_command 14 2024-02-23T00:48:11.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.405+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fe66c12c5a0 con 0x7fe66c1262a0 2024-02-23T00:48:11.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.407+0000 7fe6637fe700 1 -- 172.21.15.47:0/3395954453 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fe66c12c5a0 con 0x7fe66c1262a0 2024-02-23T00:48:11.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.407+0000 7fe6637fe700 10 client.31002.objecter ms_dispatch 0x7fe66c001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:48:11.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.407+0000 7fe6637fe700 10 client.31002.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:48:11.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.407+0000 7fe6637fe700 20 client.31002.objecter put_session s=0x7fe66c126fc0 osd=6 4 2024-02-23T00:48:11.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.407+0000 7fe6637fe700 15 client.31002.objecter _session_command_op_remove 6 14 2024-02-23T00:48:11.409 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:48:11.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.407+0000 7fe674649700 10 client.31002.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:11.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.407+0000 7fe674649700 15 client.31002.objecter _session_command_op_assign -1 15 2024-02-23T00:48:11.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.407+0000 7fe674649700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fe66c12cc20 0x7fe66c12f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:11.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.407+0000 7fe674649700 20 client.31002.objecter _get_session s=0x7fe66c12c5a0 osd=7 3 2024-02-23T00:48:11.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.407+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c12c5a0 osd=7 3 2024-02-23T00:48:11.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.407+0000 7fe674649700 20 client.31002.objecter _get_session s=0x7fe66c12c5a0 osd=7 3 2024-02-23T00:48:11.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.407+0000 7fe674649700 15 client.31002.objecter _session_command_op_remove -1 15 2024-02-23T00:48:11.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.407+0000 7fe674649700 20 client.31002.objecter get_session s=0x7fe66c12c5a0 osd=7 3 2024-02-23T00:48:11.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.407+0000 7fe674649700 15 client.31002.objecter _session_command_op_assign 7 15 2024-02-23T00:48:11.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.407+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c12c5a0 osd=7 4 2024-02-23T00:48:11.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.407+0000 7fe674649700 10 client.31002.objecter _send_command 15 2024-02-23T00:48:11.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.407+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe66c131de0 con 0x7fe66c12cc20 2024-02-23T00:48:11.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.407+0000 7fe6723e5700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fe66c12cc20 0x7fe66c12f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:11.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.408+0000 7fe6637fe700 10 client.31002.objecter ms_handle_connect 0x7fe66c12cc20 2024-02-23T00:48:11.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.408+0000 7fe6723e5700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fe66c12cc20 0x7fe66c12f010 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:11.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.408+0000 7fe6637fe700 1 -- 172.21.15.47:0/3395954453 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe66c131de0 con 0x7fe66c12cc20 2024-02-23T00:48:11.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.408+0000 7fe6637fe700 10 client.31002.objecter ms_dispatch 0x7fe66c001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:48:11.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.408+0000 7fe6637fe700 10 client.31002.objecter _finish_command 15 = system:0 2024-02-23T00:48:11.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.408+0000 7fe6637fe700 20 client.31002.objecter put_session s=0x7fe66c12c5a0 osd=7 4 2024-02-23T00:48:11.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.408+0000 7fe6637fe700 15 client.31002.objecter _session_command_op_remove 7 15 2024-02-23T00:48:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.424+0000 7fe674649700 10 client.31002.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:48:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.424+0000 7fe674649700 15 client.31002.objecter _session_command_op_assign -1 16 2024-02-23T00:48:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.424+0000 7fe674649700 20 client.31002.objecter _get_session s=0x7fe66c12c5a0 osd=7 3 2024-02-23T00:48:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.424+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c12c5a0 osd=7 3 2024-02-23T00:48:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.424+0000 7fe674649700 20 client.31002.objecter _get_session s=0x7fe66c12c5a0 osd=7 3 2024-02-23T00:48:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.424+0000 7fe674649700 15 client.31002.objecter _session_command_op_remove -1 16 2024-02-23T00:48:11.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.424+0000 7fe674649700 20 client.31002.objecter get_session s=0x7fe66c12c5a0 osd=7 3 2024-02-23T00:48:11.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.424+0000 7fe674649700 15 client.31002.objecter _session_command_op_assign 7 16 2024-02-23T00:48:11.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.424+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c12c5a0 osd=7 4 2024-02-23T00:48:11.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.424+0000 7fe674649700 10 client.31002.objecter _send_command 16 2024-02-23T00:48:11.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.424+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fe66c131de0 con 0x7fe66c12cc20 2024-02-23T00:48:11.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.425+0000 7fe6637fe700 1 -- 172.21.15.47:0/3395954453 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fe66c131de0 con 0x7fe66c12cc20 2024-02-23T00:48:11.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe6637fe700 10 client.31002.objecter ms_dispatch 0x7fe66c001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:48:11.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe6637fe700 10 client.31002.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:48:11.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe6637fe700 20 client.31002.objecter put_session s=0x7fe66c12c5a0 osd=7 4 2024-02-23T00:48:11.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe6637fe700 15 client.31002.objecter _session_command_op_remove 7 16 2024-02-23T00:48:11.428 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:48:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 10 librados: watch_flush enter 2024-02-23T00:48:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 10 librados: watch_flush exit 2024-02-23T00:48:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 10 client.31002.objecter close_session for osd.0 2024-02-23T00:48:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fe66c028540 msgr2=0x7fe66c0fe910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fe66c028540 0x7fe66c0fe910 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c106e20 osd=0 1 2024-02-23T00:48:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 10 client.31002.objecter close_session for osd.1 2024-02-23T00:48:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fe66c027b90 msgr2=0x7fe66c027fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fe66c027b90 0x7fe66c027fb0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c0fff80 osd=1 1 2024-02-23T00:48:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 10 client.31002.objecter close_session for osd.2 2024-02-23T00:48:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fe66c0ffb30 msgr2=0x7fe66c06c540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fe66c0ffb30 0x7fe66c06c540 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c027a10 osd=2 1 2024-02-23T00:48:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 10 client.31002.objecter close_session for osd.3 2024-02-23T00:48:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe672be6700 1 -- 172.21.15.47:0/3395954453 reap_dead start 2024-02-23T00:48:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fe66c0ff1b0 msgr2=0x7fe66c122620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fe66c0ff1b0 0x7fe66c122620 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c0ff040 osd=3 1 2024-02-23T00:48:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 10 client.31002.objecter close_session for osd.4 2024-02-23T00:48:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fe66c11fc90 msgr2=0x7fe66c1200d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fe66c11fc90 0x7fe66c1200d0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c06c360 osd=4 1 2024-02-23T00:48:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 10 client.31002.objecter close_session for osd.5 2024-02-23T00:48:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fe66c127a80 msgr2=0x7fe66c127f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fe66c127a80 0x7fe66c127f00 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c127350 osd=5 1 2024-02-23T00:48:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 10 client.31002.objecter close_session for osd.6 2024-02-23T00:48:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fe66c1262a0 msgr2=0x7fe66c126700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fe66c1262a0 0x7fe66c126700 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c126fc0 osd=6 1 2024-02-23T00:48:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 10 client.31002.objecter close_session for osd.7 2024-02-23T00:48:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fe66c12cc20 msgr2=0x7fe66c12f010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fe66c12cc20 0x7fe66c12f010 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 20 client.31002.objecter put_session s=0x7fe66c12c5a0 osd=7 1 2024-02-23T00:48:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 20 client.31002.objecter shutdown clearing up homeless session... 2024-02-23T00:48:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 10 client.31002.objecter successfully canceled tick 2024-02-23T00:48:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe658062cc0 msgr2=0x7fe658065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe658062cc0 0x7fe658065180 secure :-1 s=READY pgs=3063 cs=0 l=1 rev1=1 rx=0x7fe65c004840 tx=0x7fe65c00b040).stop 2024-02-23T00:48:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe66c0f8420 msgr2=0x7fe66c10e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe674649700 1 --2- 172.21.15.47:0/3395954453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe66c0f8420 0x7fe66c10e050 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 rx=0x7fe6680048f0 tx=0x7fe668009cb0).stop 2024-02-23T00:48:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.426+0000 7fe672be6700 1 -- 172.21.15.47:0/3395954453 reap_dead start 2024-02-23T00:48:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.427+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 shutdown_connections 2024-02-23T00:48:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.427+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 >> 172.21.15.47:0/3395954453 conn(0x7fe66c0f3800 msgr2=0x7fe66c0fc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:48:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.427+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 shutdown_connections 2024-02-23T00:48:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.427+0000 7fe674649700 1 -- 172.21.15.47:0/3395954453 wait complete. 2024-02-23T00:48:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:11.427+0000 7fe674649700 1 librados: shutdown 2024-02-23T00:48:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:48:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:48:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:48:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:48:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:48:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:48:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:48:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:48:11.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:09.896356+0000 osd.7 (osd.7) 197 : cluster [DBG] 62.1f scrub starts 2024-02-23T00:48:11.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:09.898580+0000 osd.7 (osd.7) 198 : cluster [DBG] 62.1f scrub ok 2024-02-23T00:48:11.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:10.267371+0000 mon.a (mon.0) 2568 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-23T00:48:11.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:10.267442+0000 mon.a (mon.0) 2569 : cluster [DBG] osdmap e418: 8 total, 8 up, 7 in 2024-02-23T00:48:11.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:10.492352+0000 osd.4 (osd.4) 161 : cluster [DBG] 5.1 deep-scrub starts 2024-02-23T00:48:11.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:10.493335+0000 osd.4 (osd.4) 162 : cluster [DBG] 5.1 deep-scrub ok 2024-02-23T00:48:11.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:10.932548+0000 osd.3 (osd.3) 194 : cluster [DBG] 18.a scrub starts 2024-02-23T00:48:11.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:10.933372+0000 osd.3 (osd.3) 195 : cluster [DBG] 18.a scrub ok 2024-02-23T00:48:11.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:09.896356+0000 osd.7 (osd.7) 197 : cluster [DBG] 62.1f scrub starts 2024-02-23T00:48:11.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:09.898580+0000 osd.7 (osd.7) 198 : cluster [DBG] 62.1f scrub ok 2024-02-23T00:48:11.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:10.267371+0000 mon.a (mon.0) 2568 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-23T00:48:11.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:10.267442+0000 mon.a (mon.0) 2569 : cluster [DBG] osdmap e418: 8 total, 8 up, 7 in 2024-02-23T00:48:11.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:10.492352+0000 osd.4 (osd.4) 161 : cluster [DBG] 5.1 deep-scrub starts 2024-02-23T00:48:11.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:10.493335+0000 osd.4 (osd.4) 162 : cluster [DBG] 5.1 deep-scrub ok 2024-02-23T00:48:11.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:10.932548+0000 osd.3 (osd.3) 194 : cluster [DBG] 18.a scrub starts 2024-02-23T00:48:11.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:10.933372+0000 osd.3 (osd.3) 195 : cluster [DBG] 18.a scrub ok 2024-02-23T00:48:12.108 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:48:12.112 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:12.112 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:48:12.213 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:48:12.243 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:12.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:10.562771+0000 osd.6 (osd.6) 165 : cluster [DBG] 2.1a scrub starts 2024-02-23T00:48:12.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:10.563690+0000 osd.6 (osd.6) 166 : cluster [DBG] 2.1a scrub ok 2024-02-23T00:48:12.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:10.849331+0000 osd.0 (osd.0) 167 : cluster [DBG] 2.1f scrub starts 2024-02-23T00:48:12.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:10.850130+0000 osd.0 (osd.0) 168 : cluster [DBG] 2.1f scrub ok 2024-02-23T00:48:12.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:10.920404+0000 osd.7 (osd.7) 199 : cluster [DBG] 62.b deep-scrub starts 2024-02-23T00:48:12.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:10.926495+0000 osd.7 (osd.7) 200 : cluster [DBG] 62.b deep-scrub ok 2024-02-23T00:48:12.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:11.271146+0000 mon.a (mon.0) 2570 : cluster [DBG] osdmap e419: 8 total, 8 up, 7 in 2024-02-23T00:48:12.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:11.276745+0000 mon.a (mon.0) 2571 : audit [DBG] from='client.? 172.21.15.47:0/3395954453' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:48:12.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:11.286702+0000 mgr.y (mgr.14517) 381 : cluster [DBG] pgmap v604: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:48:12.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:11.303548+0000 mon.a (mon.0) 2572 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:48:12.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:11.826555+0000 osd.1 (osd.1) 179 : cluster [DBG] 4.7 scrub starts 2024-02-23T00:48:12.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:11.828538+0000 osd.1 (osd.1) 180 : cluster [DBG] 4.7 scrub ok 2024-02-23T00:48:12.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:10.562771+0000 osd.6 (osd.6) 165 : cluster [DBG] 2.1a scrub starts 2024-02-23T00:48:12.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:10.563690+0000 osd.6 (osd.6) 166 : cluster [DBG] 2.1a scrub ok 2024-02-23T00:48:12.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:10.849331+0000 osd.0 (osd.0) 167 : cluster [DBG] 2.1f scrub starts 2024-02-23T00:48:12.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:10.850130+0000 osd.0 (osd.0) 168 : cluster [DBG] 2.1f scrub ok 2024-02-23T00:48:12.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:10.920404+0000 osd.7 (osd.7) 199 : cluster [DBG] 62.b deep-scrub starts 2024-02-23T00:48:12.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:10.926495+0000 osd.7 (osd.7) 200 : cluster [DBG] 62.b deep-scrub ok 2024-02-23T00:48:12.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:11.271146+0000 mon.a (mon.0) 2570 : cluster [DBG] osdmap e419: 8 total, 8 up, 7 in 2024-02-23T00:48:12.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:11.276745+0000 mon.a (mon.0) 2571 : audit [DBG] from='client.? 172.21.15.47:0/3395954453' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:48:12.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:11.286702+0000 mgr.y (mgr.14517) 381 : cluster [DBG] pgmap v604: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:48:12.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:11.303548+0000 mon.a (mon.0) 2572 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:48:12.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:11.826555+0000 osd.1 (osd.1) 179 : cluster [DBG] 4.7 scrub starts 2024-02-23T00:48:12.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:11.828538+0000 osd.1 (osd.1) 180 : cluster [DBG] 4.7 scrub ok 2024-02-23T00:48:12.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:10.562771+0000 osd.6 (osd.6) 165 : cluster [DBG] 2.1a scrub starts 2024-02-23T00:48:12.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:10.563690+0000 osd.6 (osd.6) 166 : cluster [DBG] 2.1a scrub ok 2024-02-23T00:48:12.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:10.849331+0000 osd.0 (osd.0) 167 : cluster [DBG] 2.1f scrub starts 2024-02-23T00:48:12.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:10.850130+0000 osd.0 (osd.0) 168 : cluster [DBG] 2.1f scrub ok 2024-02-23T00:48:12.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:10.920404+0000 osd.7 (osd.7) 199 : cluster [DBG] 62.b deep-scrub starts 2024-02-23T00:48:12.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:10.926495+0000 osd.7 (osd.7) 200 : cluster [DBG] 62.b deep-scrub ok 2024-02-23T00:48:12.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:11.271146+0000 mon.a (mon.0) 2570 : cluster [DBG] osdmap e419: 8 total, 8 up, 7 in 2024-02-23T00:48:12.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:11.276745+0000 mon.a (mon.0) 2571 : audit [DBG] from='client.? 172.21.15.47:0/3395954453' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:48:12.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:11.286702+0000 mgr.y (mgr.14517) 381 : cluster [DBG] pgmap v604: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:48:12.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:11.303548+0000 mon.a (mon.0) 2572 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:48:12.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:11.826555+0000 osd.1 (osd.1) 179 : cluster [DBG] 4.7 scrub starts 2024-02-23T00:48:12.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:11.828538+0000 osd.1 (osd.1) 180 : cluster [DBG] 4.7 scrub ok 2024-02-23T00:48:12.694 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:48:12.696 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:12.697 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:48:12.798 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:48:12.828 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:48:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:48:12.828 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:48:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:48:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:48:12.830 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:12.852 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:48:12.854 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:12.855 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:48:12.956 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:48:12.987 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:13.147 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:48:13.150 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:13.150 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:48:13.166 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:48:13.252 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:48:13.283 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:12.278452+0000 mon.a (mon.0) 2573 : cluster [DBG] osdmap e420: 8 total, 8 up, 7 in 2024-02-23T00:48:13.283 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:12.361079+0000 osd.5 (osd.5) 174 : cluster [DBG] 4.4 scrub starts 2024-02-23T00:48:13.283 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:12.364535+0000 osd.5 (osd.5) 175 : cluster [DBG] 4.4 scrub ok 2024-02-23T00:48:13.283 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:12.581152+0000 mgr.y (mgr.14517) 382 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:48:13.283 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:12.865959+0000 osd.1 (osd.1) 181 : cluster [DBG] 2.4 scrub starts 2024-02-23T00:48:13.283 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:12.868927+0000 osd.1 (osd.1) 182 : cluster [DBG] 2.4 scrub ok 2024-02-23T00:48:13.285 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:13.307 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:48:13.310 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:13.310 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:48:13.411 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:48:13.448 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:13.470 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:48:13.473 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:13.474 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:48:13.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:12.278452+0000 mon.a (mon.0) 2573 : cluster [DBG] osdmap e420: 8 total, 8 up, 7 in 2024-02-23T00:48:13.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:12.361079+0000 osd.5 (osd.5) 174 : cluster [DBG] 4.4 scrub starts 2024-02-23T00:48:13.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:12.364535+0000 osd.5 (osd.5) 175 : cluster [DBG] 4.4 scrub ok 2024-02-23T00:48:13.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:12.581152+0000 mgr.y (mgr.14517) 382 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:48:13.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:12.865959+0000 osd.1 (osd.1) 181 : cluster [DBG] 2.4 scrub starts 2024-02-23T00:48:13.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:12.868927+0000 osd.1 (osd.1) 182 : cluster [DBG] 2.4 scrub ok 2024-02-23T00:48:13.575 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:48:13.604 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:12.278452+0000 mon.a (mon.0) 2573 : cluster [DBG] osdmap e420: 8 total, 8 up, 7 in 2024-02-23T00:48:13.604 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:12.361079+0000 osd.5 (osd.5) 174 : cluster [DBG] 4.4 scrub starts 2024-02-23T00:48:13.604 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:12.364535+0000 osd.5 (osd.5) 175 : cluster [DBG] 4.4 scrub ok 2024-02-23T00:48:13.604 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:12.581152+0000 mgr.y (mgr.14517) 382 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:48:13.604 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:12.865959+0000 osd.1 (osd.1) 181 : cluster [DBG] 2.4 scrub starts 2024-02-23T00:48:13.604 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:12.868927+0000 osd.1 (osd.1) 182 : cluster [DBG] 2.4 scrub ok 2024-02-23T00:48:13.606 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:13.626 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:48:13.628 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:13.629 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:48:13.730 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:48:13.759 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:13.779 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:48:13.781 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:13.782 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:48:13.883 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:48:13.912 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:13.931 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:48:13.933 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:13.934 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:48:14.035 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:48:14.066 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:14.166 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:48:14.166 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 20, 2024-02-23T00:48:14.166 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 600, 2024-02-23T00:48:14.166 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [ 2024-02-23T00:48:14.167 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.167 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24811.0:10 19.4 19:2094d267:test-rados-api-smithi047-82438-5::foo:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected e111)", 2024-02-23T00:48:14.167 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:00.849994+0000", 2024-02-23T00:48:14.167 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 313.29717968900002, 2024-02-23T00:48:14.167 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.13391424800000001, 2024-02-23T00:48:14.167 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:14.167 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:48:14.167 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:48:14.167 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24811", 2024-02-23T00:48:14.167 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/3614542636", 2024-02-23T00:48:14.167 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 10 2024-02-23T00:48:14.167 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.167 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:14.167 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.168 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:14.168 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.849994+0000", 2024-02-23T00:48:14.168 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:14.168 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.168 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.168 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:14.168 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.849994+0000", 2024-02-23T00:48:14.168 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:48:14.168 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.168 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.168 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:14.168 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.849992+0000", 2024-02-23T00:48:14.168 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1450999999999999e-05 2024-02-23T00:48:14.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.169 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.169 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:14.169 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.850004+0000", 2024-02-23T00:48:14.169 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3368e-05 2024-02-23T00:48:14.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.169 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.169 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:14.169 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.850017+0000", 2024-02-23T00:48:14.169 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0490000000000004e-06 2024-02-23T00:48:14.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.169 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.169 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:14.170 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.850025+0000", 2024-02-23T00:48:14.170 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.7912999999999999e-05 2024-02-23T00:48:14.170 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.170 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.170 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:14.170 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.850083+0000", 2024-02-23T00:48:14.170 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000110516 2024-02-23T00:48:14.170 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.170 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.170 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:14.170 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.850193+0000", 2024-02-23T00:48:14.170 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.133715062 2024-02-23T00:48:14.171 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.171 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.171 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:14.171 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.983908+0000", 2024-02-23T00:48:14.171 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.13391424800000001 2024-02-23T00:48:14.171 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.171 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:14.171 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.171 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.171 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.171 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(osd.6.0:245 20.12 e112/109 -22:4d27b8a4:::temp_20.12_0_24277_1:head v 0'0, mlcod=0'0)", 2024-02-23T00:48:14.171 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:01.452136+0000", 2024-02-23T00:48:14.171 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 312.695037668, 2024-02-23T00:48:14.171 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.033887135999999998, 2024-02-23T00:48:14.172 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:14.172 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:48:14.172 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:14.172 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.172 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:14.172 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.452136+0000", 2024-02-23T00:48:14.172 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:14.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.172 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.172 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:14.172 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.452136+0000", 2024-02-23T00:48:14.173 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:48:14.173 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.173 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.173 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:14.173 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.452134+0000", 2024-02-23T00:48:14.173 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018909622000000001 2024-02-23T00:48:14.173 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.173 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.173 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:14.173 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.471044+0000", 2024-02-23T00:48:14.173 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.2262000000000001e-05 2024-02-23T00:48:14.173 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.173 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.173 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:14.174 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.471066+0000", 2024-02-23T00:48:14.174 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0569999999999993e-06 2024-02-23T00:48:14.174 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.174 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.174 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:14.174 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.471074+0000", 2024-02-23T00:48:14.174 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.4327999999999999e-05 2024-02-23T00:48:14.174 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.174 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.175 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:14.175 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.471128+0000", 2024-02-23T00:48:14.175 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.9809e-05 2024-02-23T00:48:14.175 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.175 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.175 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:14.175 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.471148+0000", 2024-02-23T00:48:14.175 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.014816957 2024-02-23T00:48:14.175 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.175 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.175 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:48:14.175 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.485965+0000", 2024-02-23T00:48:14.175 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.8130000000000001e-05 2024-02-23T00:48:14.176 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.176 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.176 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:14.176 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.486023+0000", 2024-02-23T00:48:14.176 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.033887135999999998 2024-02-23T00:48:14.176 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.176 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:14.176 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.176 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.176 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.176 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24775.0:2070 20.6 20:63faae7e:test-rados-api-smithi047-82457-17::big2:head [writefull 0~25165828] snapc 0=[] ondisk+write+known_if_redirected e112)", 2024-02-23T00:48:14.176 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:01.792251+0000", 2024-02-23T00:48:14.176 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 312.354922762, 2024-02-23T00:48:14.177 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.10134280900000001, 2024-02-23T00:48:14.177 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:14.177 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:48:14.177 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:48:14.177 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24775", 2024-02-23T00:48:14.177 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1191937413", 2024-02-23T00:48:14.177 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2070 2024-02-23T00:48:14.177 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.177 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:14.177 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.177 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:14.177 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.792251+0000", 2024-02-23T00:48:14.177 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:14.177 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.178 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.178 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:14.178 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.792251+0000", 2024-02-23T00:48:14.178 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999962 2024-02-23T00:48:14.178 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.178 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.178 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:14.178 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.792247+0000", 2024-02-23T00:48:14.178 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.014557680999999999 2024-02-23T00:48:14.178 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.178 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.178 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:14.178 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.806805+0000", 2024-02-23T00:48:14.178 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.7353999999999999e-05 2024-02-23T00:48:14.179 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.179 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.179 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:14.179 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.806832+0000", 2024-02-23T00:48:14.179 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1474000000000001e-05 2024-02-23T00:48:14.179 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.179 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.179 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:14.179 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.806844+0000", 2024-02-23T00:48:14.179 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.9030000000000002e-05 2024-02-23T00:48:14.179 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.179 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.179 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:14.179 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.806903+0000", 2024-02-23T00:48:14.180 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014977100000000001 2024-02-23T00:48:14.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.180 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:14.180 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.807053+0000", 2024-02-23T00:48:14.180 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00017362000000000001 2024-02-23T00:48:14.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.180 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 3,7", 2024-02-23T00:48:14.180 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.807226+0000", 2024-02-23T00:48:14.180 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.052890589000000002 2024-02-23T00:48:14.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.180 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:48:14.181 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.860117+0000", 2024-02-23T00:48:14.181 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0059036360000000003 2024-02-23T00:48:14.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.181 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.181 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:48:14.181 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.866021+0000", 2024-02-23T00:48:14.181 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.027494049 2024-02-23T00:48:14.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.181 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.181 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:48:14.181 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.893515+0000", 2024-02-23T00:48:14.181 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.4363999999999998e-05 2024-02-23T00:48:14.182 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.182 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.182 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:48:14.182 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.893549+0000", 2024-02-23T00:48:14.182 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.4903e-05 2024-02-23T00:48:14.182 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.182 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.182 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:14.182 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.893594+0000", 2024-02-23T00:48:14.182 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.10134280900000001 2024-02-23T00:48:14.182 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.182 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:14.182 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.183 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.183 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24403.0:216 3.0 3:353ec8f9:::gateway.conf:head [getxattr epoch (1) in=5b] snapc 0=[] ondisk+read+known_if_redirected e113)", 2024-02-23T00:48:14.183 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:04.226432+0000", 2024-02-23T00:48:14.183 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 309.92074231100003, 2024-02-23T00:48:14.183 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.027281850999999999, 2024-02-23T00:48:14.183 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:14.183 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:48:14.183 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:48:14.183 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24403", 2024-02-23T00:48:14.183 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.196:0/519523863", 2024-02-23T00:48:14.183 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 216 2024-02-23T00:48:14.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.184 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:14.184 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.184 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:14.184 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.226432+0000", 2024-02-23T00:48:14.184 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:14.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.184 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.184 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:14.184 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.226432+0000", 2024-02-23T00:48:14.184 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999976 2024-02-23T00:48:14.185 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.185 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.185 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:14.185 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.226429+0000", 2024-02-23T00:48:14.185 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3016e-05 2024-02-23T00:48:14.185 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.185 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.185 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:14.185 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.226442+0000", 2024-02-23T00:48:14.185 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0083690269999999994 2024-02-23T00:48:14.185 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.185 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.185 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:14.185 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.234811+0000", 2024-02-23T00:48:14.186 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.7221999999999999e-05 2024-02-23T00:48:14.186 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.186 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.186 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:14.186 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.234828+0000", 2024-02-23T00:48:14.186 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018723701999999998 2024-02-23T00:48:14.186 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.186 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.186 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:14.186 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.253552+0000", 2024-02-23T00:48:14.186 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.2403000000000004e-05 2024-02-23T00:48:14.186 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.186 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.186 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:14.187 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.253634+0000", 2024-02-23T00:48:14.187 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8904000000000002e-05 2024-02-23T00:48:14.187 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.187 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.187 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:14.187 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.253713+0000", 2024-02-23T00:48:14.187 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.027281850999999999 2024-02-23T00:48:14.187 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.187 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:14.187 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.187 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.187 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.187 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.15144.0:3 38.f 38:f4b50b61:::1.obj:head [write 0~198] snapc 0=[] ondisk+write+known_if_redirected e113)", 2024-02-23T00:48:14.187 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:04.331121+0000", 2024-02-23T00:48:14.188 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 309.81605331899999, 2024-02-23T00:48:14.188 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.024663518999999998, 2024-02-23T00:48:14.188 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:14.188 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:48:14.188 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:48:14.188 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.15144", 2024-02-23T00:48:14.188 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/2599587504", 2024-02-23T00:48:14.188 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 3 2024-02-23T00:48:14.188 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.188 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:14.188 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.188 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:14.188 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.331121+0000", 2024-02-23T00:48:14.188 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:14.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.189 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.189 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:14.189 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.331121+0000", 2024-02-23T00:48:14.189 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:48:14.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.189 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.189 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:14.189 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.331119+0000", 2024-02-23T00:48:14.189 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.1100000000000003e-06 2024-02-23T00:48:14.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.190 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.190 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:14.190 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.331127+0000", 2024-02-23T00:48:14.190 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6103000000000002e-05 2024-02-23T00:48:14.190 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.190 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.190 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:14.190 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.331143+0000", 2024-02-23T00:48:14.190 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0609999999999996e-06 2024-02-23T00:48:14.190 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.190 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.190 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:14.190 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.331151+0000", 2024-02-23T00:48:14.191 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.4344000000000004e-05 2024-02-23T00:48:14.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.191 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.191 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:14.191 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.331235+0000", 2024-02-23T00:48:14.191 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000134793 2024-02-23T00:48:14.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.191 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.191 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:14.191 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.331370+0000", 2024-02-23T00:48:14.191 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00015517100000000001 2024-02-23T00:48:14.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.191 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.192 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 0,5", 2024-02-23T00:48:14.192 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.331525+0000", 2024-02-23T00:48:14.192 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00081146600000000005 2024-02-23T00:48:14.192 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.192 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.192 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:48:14.192 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.332337+0000", 2024-02-23T00:48:14.192 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0019837330000000001 2024-02-23T00:48:14.192 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.192 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.192 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:48:14.192 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.334320+0000", 2024-02-23T00:48:14.192 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021412520000000001 2024-02-23T00:48:14.192 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.193 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.193 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:48:14.193 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.355733+0000", 2024-02-23T00:48:14.193 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0108999999999999e-05 2024-02-23T00:48:14.193 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.193 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.193 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:48:14.193 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.355763+0000", 2024-02-23T00:48:14.193 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1069000000000001e-05 2024-02-23T00:48:14.193 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.193 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.193 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:14.194 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.355784+0000", 2024-02-23T00:48:14.194 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.024663518999999998 2024-02-23T00:48:14.194 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.194 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:14.194 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.194 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.194 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.194 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.25529.0:16 63.0 63:0725c07e:test-rados-api-smithi047-82496-9::foo:head [write 128~128] snapc 7=[7,6] ondisk+write+known_if_redirected e136)", 2024-02-23T00:48:14.194 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:27.581440+0000", 2024-02-23T00:48:14.194 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 286.565734249, 2024-02-23T00:48:14.194 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.031491658999999998, 2024-02-23T00:48:14.194 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:14.194 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:48:14.194 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:48:14.194 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.25529", 2024-02-23T00:48:14.195 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/2064770043", 2024-02-23T00:48:14.195 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 16 2024-02-23T00:48:14.195 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.195 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:14.195 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.195 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:14.195 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:27.581440+0000", 2024-02-23T00:48:14.195 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:14.195 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.195 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.195 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:14.195 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:27.581440+0000", 2024-02-23T00:48:14.195 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999962 2024-02-23T00:48:14.195 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.196 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.196 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:14.196 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:27.581436+0000", 2024-02-23T00:48:14.196 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.329e-06 2024-02-23T00:48:14.196 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.196 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.196 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:14.196 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:27.581445+0000", 2024-02-23T00:48:14.196 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6963999999999999e-05 2024-02-23T00:48:14.196 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.196 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.197 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:14.197 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:27.581462+0000", 2024-02-23T00:48:14.197 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.9680000000000006e-06 2024-02-23T00:48:14.197 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.197 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.197 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:14.197 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:27.581470+0000", 2024-02-23T00:48:14.197 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010562578 2024-02-23T00:48:14.197 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.197 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.197 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:14.197 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:27.592033+0000", 2024-02-23T00:48:14.197 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.9858000000000005e-05 2024-02-23T00:48:14.198 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.198 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.198 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:14.198 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:27.592133+0000", 2024-02-23T00:48:14.198 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00022296100000000001 2024-02-23T00:48:14.198 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.198 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.198 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 0,6", 2024-02-23T00:48:14.198 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:27.592356+0000", 2024-02-23T00:48:14.198 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.005479556 2024-02-23T00:48:14.198 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.198 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.198 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:48:14.199 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:27.597835+0000", 2024-02-23T00:48:14.199 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012266022 2024-02-23T00:48:14.199 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.199 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.199 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:48:14.199 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:27.610101+0000", 2024-02-23T00:48:14.199 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00274479 2024-02-23T00:48:14.199 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.199 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.199 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:48:14.199 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:27.612846+0000", 2024-02-23T00:48:14.199 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.5865999999999998e-05 2024-02-23T00:48:14.199 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.199 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.200 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:48:14.200 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:27.612892+0000", 2024-02-23T00:48:14.200 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.9390000000000001e-05 2024-02-23T00:48:14.200 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.200 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.200 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:14.200 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:27.612931+0000", 2024-02-23T00:48:14.200 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.031491658999999998 2024-02-23T00:48:14.200 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.200 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:14.200 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.200 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.200 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.200 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.25529.0:28 63.e 63:7037fd60:test-rados-api-smithi047-82496-10::foo:head [write 0~128] snapc a=[a] ondisk+write+known_if_redirected e139)", 2024-02-23T00:48:14.201 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:31.196152+0000", 2024-02-23T00:48:14.201 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 282.95102183699998, 2024-02-23T00:48:14.201 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.035570335000000002, 2024-02-23T00:48:14.201 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:14.201 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:48:14.201 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:48:14.201 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.25529", 2024-02-23T00:48:14.201 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/2064770043", 2024-02-23T00:48:14.201 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 28 2024-02-23T00:48:14.201 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.201 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:14.201 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.201 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:14.201 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.196152+0000", 2024-02-23T00:48:14.201 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:14.202 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.202 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.202 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:14.202 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.196152+0000", 2024-02-23T00:48:14.202 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:48:14.202 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.202 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.202 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:14.202 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.196150+0000", 2024-02-23T00:48:14.202 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.6059999999999999e-06 2024-02-23T00:48:14.202 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.202 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.202 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:14.203 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.196157+0000", 2024-02-23T00:48:14.203 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4922e-05 2024-02-23T00:48:14.203 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.203 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.203 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:14.203 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.196172+0000", 2024-02-23T00:48:14.203 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.1940000000000001e-06 2024-02-23T00:48:14.203 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.203 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.203 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:14.203 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.196179+0000", 2024-02-23T00:48:14.203 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0038786519999999998 2024-02-23T00:48:14.203 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.204 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.204 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:14.204 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.200058+0000", 2024-02-23T00:48:14.204 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00015678799999999999 2024-02-23T00:48:14.204 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.204 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.204 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:14.204 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.200214+0000", 2024-02-23T00:48:14.204 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00017207999999999999 2024-02-23T00:48:14.204 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.204 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.204 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 2,3", 2024-02-23T00:48:14.204 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.200386+0000", 2024-02-23T00:48:14.204 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0032031099999999999 2024-02-23T00:48:14.205 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.205 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.205 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:48:14.205 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.203590+0000", 2024-02-23T00:48:14.205 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.011119851 2024-02-23T00:48:14.205 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.205 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.205 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:48:14.205 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.214709+0000", 2024-02-23T00:48:14.205 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.01692623 2024-02-23T00:48:14.205 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.205 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.205 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:48:14.206 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.231636+0000", 2024-02-23T00:48:14.206 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.6619000000000002e-05 2024-02-23T00:48:14.206 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.206 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.206 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:48:14.206 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.231682+0000", 2024-02-23T00:48:14.206 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.0085e-05 2024-02-23T00:48:14.206 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.206 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.206 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:14.206 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.231722+0000", 2024-02-23T00:48:14.206 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.035570335000000002 2024-02-23T00:48:14.206 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.207 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:14.207 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.207 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.207 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.207 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.25529.0:35 63.e 63:7037fd60:test-rados-api-smithi047-82496-10::foo:head [write 128~128] snapc b=[b,a] ondisk+write+known_if_redirected e140)", 2024-02-23T00:48:14.207 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:32.199273+0000", 2024-02-23T00:48:14.207 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 281.94790108299998, 2024-02-23T00:48:14.207 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.106436984, 2024-02-23T00:48:14.207 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:14.207 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:48:14.207 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:48:14.207 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.25529", 2024-02-23T00:48:14.208 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/2064770043", 2024-02-23T00:48:14.208 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 35 2024-02-23T00:48:14.208 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.208 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:14.208 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.208 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:14.208 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.199273+0000", 2024-02-23T00:48:14.208 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:14.208 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.208 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.208 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:14.208 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.199273+0000", 2024-02-23T00:48:14.208 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999967 2024-02-23T00:48:14.209 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.209 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.209 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:14.209 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.199270+0000", 2024-02-23T00:48:14.209 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4188999999999999e-05 2024-02-23T00:48:14.209 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.209 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.209 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:14.209 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.199284+0000", 2024-02-23T00:48:14.209 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.9290999999999999e-05 2024-02-23T00:48:14.209 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.209 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.210 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:14.210 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.199303+0000", 2024-02-23T00:48:14.210 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.6670000000000008e-06 2024-02-23T00:48:14.210 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.210 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.210 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:14.210 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.199313+0000", 2024-02-23T00:48:14.210 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0039358450000000003 2024-02-23T00:48:14.210 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.210 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.210 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:14.211 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.203249+0000", 2024-02-23T00:48:14.211 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.1759999999999999e-06 2024-02-23T00:48:14.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.211 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.211 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-02-23T00:48:14.211 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.203258+0000", 2024-02-23T00:48:14.211 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.057894644000000002 2024-02-23T00:48:14.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.211 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.211 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:14.211 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.261152+0000", 2024-02-23T00:48:14.211 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00015342099999999999 2024-02-23T00:48:14.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.211 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.212 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:14.212 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.261306+0000", 2024-02-23T00:48:14.212 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00030064399999999998 2024-02-23T00:48:14.212 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.212 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.212 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 2,3", 2024-02-23T00:48:14.212 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.261606+0000", 2024-02-23T00:48:14.212 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.031497770000000001 2024-02-23T00:48:14.212 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.212 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.212 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:48:14.212 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.293104+0000", 2024-02-23T00:48:14.213 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00070708999999999995 2024-02-23T00:48:14.213 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.213 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.213 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:48:14.213 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.293811+0000", 2024-02-23T00:48:14.213 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.011786562 2024-02-23T00:48:14.213 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.213 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.213 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:48:14.213 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.305598+0000", 2024-02-23T00:48:14.213 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.5371999999999998e-05 2024-02-23T00:48:14.213 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.213 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.214 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:48:14.214 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.305653+0000", 2024-02-23T00:48:14.214 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.6471999999999997e-05 2024-02-23T00:48:14.214 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.214 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.214 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:14.214 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.305710+0000", 2024-02-23T00:48:14.214 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.106436984 2024-02-23T00:48:14.214 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.214 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:14.214 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.214 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.214 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.214 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWrite(79.2s1 140/139 ECSubWrite(tid=341, reqid=client.31603.0:1, at_version=140'1, trim_to=0'0, roll_forward_to=0'0))", 2024-02-23T00:48:14.215 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:32.291530+0000", 2024-02-23T00:48:14.215 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 281.85564410000001, 2024-02-23T00:48:14.215 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.022260156, 2024-02-23T00:48:14.215 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:14.215 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-02-23T00:48:14.215 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:14.215 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.215 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:14.215 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.291530+0000", 2024-02-23T00:48:14.215 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:14.215 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.215 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.216 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:14.216 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.291530+0000", 2024-02-23T00:48:14.216 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:48:14.216 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.216 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.216 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:14.216 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.291529+0000", 2024-02-23T00:48:14.216 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.4465999999999998e-05 2024-02-23T00:48:14.216 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.216 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.216 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:14.216 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.291564+0000", 2024-02-23T00:48:14.216 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.7619999999999998e-06 2024-02-23T00:48:14.217 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.217 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.217 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:14.217 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.291571+0000", 2024-02-23T00:48:14.217 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.585e-06 2024-02-23T00:48:14.217 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.217 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.217 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:14.217 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.291579+0000", 2024-02-23T00:48:14.217 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00090052800000000005 2024-02-23T00:48:14.218 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.218 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.218 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:14.218 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.292480+0000", 2024-02-23T00:48:14.218 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2649999999999999e-05 2024-02-23T00:48:14.218 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.218 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.218 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-02-23T00:48:14.218 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.292492+0000", 2024-02-23T00:48:14.218 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021268694000000001 2024-02-23T00:48:14.218 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.218 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.218 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-02-23T00:48:14.218 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.313761+0000", 2024-02-23T00:48:14.219 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8813999999999999e-05 2024-02-23T00:48:14.219 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.219 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.219 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:14.219 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.313790+0000", 2024-02-23T00:48:14.219 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.022260156 2024-02-23T00:48:14.219 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.219 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:14.219 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.219 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.219 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.219 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.25529.0:38 63.e 63:7037fd60:test-rados-api-smithi047-82496-10::foo:head [write 896~128] snapc b=[b,a] ondisk+write+known_if_redirected e140)", 2024-02-23T00:48:14.219 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:32.320224+0000", 2024-02-23T00:48:14.220 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 281.82695035500001, 2024-02-23T00:48:14.220 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029558615999999999, 2024-02-23T00:48:14.220 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:14.220 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:48:14.220 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:48:14.220 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.25529", 2024-02-23T00:48:14.220 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/2064770043", 2024-02-23T00:48:14.220 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 38 2024-02-23T00:48:14.220 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.220 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:14.220 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.220 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:14.220 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.320224+0000", 2024-02-23T00:48:14.221 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:14.221 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.221 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.221 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:14.221 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.320224+0000", 2024-02-23T00:48:14.221 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999986 2024-02-23T00:48:14.221 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.221 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.221 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:14.221 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.320222+0000", 2024-02-23T00:48:14.221 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0709999999999995e-06 2024-02-23T00:48:14.221 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.221 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.221 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:14.222 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.320230+0000", 2024-02-23T00:48:14.222 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1715999999999999e-05 2024-02-23T00:48:14.222 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.222 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.222 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:14.222 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.320242+0000", 2024-02-23T00:48:14.222 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.1199999999999999e-06 2024-02-23T00:48:14.222 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.222 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.222 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:14.222 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.320248+0000", 2024-02-23T00:48:14.222 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.4313999999999997e-05 2024-02-23T00:48:14.222 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.222 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.223 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:14.223 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.320282+0000", 2024-02-23T00:48:14.223 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.2701999999999997e-05 2024-02-23T00:48:14.223 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.223 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.223 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:14.223 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.320345+0000", 2024-02-23T00:48:14.223 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012842100000000001 2024-02-23T00:48:14.223 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.223 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.223 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 2,3", 2024-02-23T00:48:14.223 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.320474+0000", 2024-02-23T00:48:14.223 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.003900073 2024-02-23T00:48:14.224 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.224 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.224 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:48:14.224 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.324374+0000", 2024-02-23T00:48:14.224 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.011614757999999999 2024-02-23T00:48:14.224 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.224 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.224 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:48:14.224 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.335988+0000", 2024-02-23T00:48:14.224 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.013680285 2024-02-23T00:48:14.224 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.224 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.224 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:48:14.225 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.349669+0000", 2024-02-23T00:48:14.225 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.4411000000000004e-05 2024-02-23T00:48:14.225 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.225 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.225 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:48:14.225 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.349733+0000", 2024-02-23T00:48:14.225 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.9004e-05 2024-02-23T00:48:14.225 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.225 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.225 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:14.226 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.349782+0000", 2024-02-23T00:48:14.226 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029558615999999999 2024-02-23T00:48:14.226 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.226 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:14.226 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.226 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.226 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.226 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24775.0:2079 20.6 20:61acad7d:test-rados-api-smithi047-82457-17::big2.copy:head [copy-from ver 0] snapc 0=[] ondisk+write+known_if_redirected e139)", 2024-02-23T00:48:14.226 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:32.367186+0000", 2024-02-23T00:48:14.226 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 281.77998784099998, 2024-02-23T00:48:14.226 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.13614552299999999, 2024-02-23T00:48:14.226 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:14.226 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:48:14.226 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:48:14.226 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24775", 2024-02-23T00:48:14.227 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1191937413", 2024-02-23T00:48:14.227 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2079 2024-02-23T00:48:14.227 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.227 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:14.227 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.227 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:14.227 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.367186+0000", 2024-02-23T00:48:14.227 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:14.227 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.227 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.227 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:14.227 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.367186+0000", 2024-02-23T00:48:14.227 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999986 2024-02-23T00:48:14.227 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.228 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.228 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:14.228 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.367185+0000", 2024-02-23T00:48:14.228 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.5270000000000004e-06 2024-02-23T00:48:14.228 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.228 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.228 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:14.228 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.367194+0000", 2024-02-23T00:48:14.228 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.277e-05 2024-02-23T00:48:14.228 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.228 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.228 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:14.228 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.367207+0000", 2024-02-23T00:48:14.229 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.579e-06 2024-02-23T00:48:14.229 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.229 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.229 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:14.229 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.367213+0000", 2024-02-23T00:48:14.229 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000202981 2024-02-23T00:48:14.229 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.229 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.229 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:14.229 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.367416+0000", 2024-02-23T00:48:14.229 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012661399999999999 2024-02-23T00:48:14.229 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.229 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.230 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:14.230 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.367543+0000", 2024-02-23T00:48:14.230 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.11779671899999999 2024-02-23T00:48:14.230 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.230 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.230 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 3,7", 2024-02-23T00:48:14.230 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.485340+0000", 2024-02-23T00:48:14.230 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012007673999999999 2024-02-23T00:48:14.230 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.230 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.230 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:48:14.230 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.497347+0000", 2024-02-23T00:48:14.230 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00144239 2024-02-23T00:48:14.230 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.231 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.231 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:48:14.231 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.498790+0000", 2024-02-23T00:48:14.231 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0044293090000000002 2024-02-23T00:48:14.231 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.231 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.231 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:48:14.231 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.503219+0000", 2024-02-23T00:48:14.231 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.0754000000000001e-05 2024-02-23T00:48:14.231 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.231 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.231 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:48:14.232 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.503280+0000", 2024-02-23T00:48:14.232 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.1668000000000001e-05 2024-02-23T00:48:14.232 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.232 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.232 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:14.232 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.503332+0000", 2024-02-23T00:48:14.232 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.13614552299999999 2024-02-23T00:48:14.232 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.232 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:14.232 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.232 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.232 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.233 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24775.0:2080 20.5 20:a70ec860:test-rados-api-smithi047-82457-17::big3.copy:head [copy-from ver 0] snapc 0=[] ondisk+write+known_if_redirected e140)", 2024-02-23T00:48:14.233 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:32.508611+0000", 2024-02-23T00:48:14.233 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 281.63856273300001, 2024-02-23T00:48:14.233 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.18482968499999999, 2024-02-23T00:48:14.233 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:14.233 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:48:14.233 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:48:14.233 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24775", 2024-02-23T00:48:14.233 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1191937413", 2024-02-23T00:48:14.233 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2080 2024-02-23T00:48:14.233 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.233 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:14.233 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.233 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:14.234 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.508611+0000", 2024-02-23T00:48:14.234 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:14.234 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.234 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.234 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:14.234 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.508611+0000", 2024-02-23T00:48:14.234 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999986 2024-02-23T00:48:14.234 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.234 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.234 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:14.234 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.508610+0000", 2024-02-23T00:48:14.234 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.0729999999999994e-06 2024-02-23T00:48:14.234 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.234 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.235 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:14.235 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.508619+0000", 2024-02-23T00:48:14.235 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6867e-05 2024-02-23T00:48:14.235 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.235 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.235 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:14.235 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.508636+0000", 2024-02-23T00:48:14.235 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.9800000000000001e-06 2024-02-23T00:48:14.235 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.235 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.235 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:14.235 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.508642+0000", 2024-02-23T00:48:14.235 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.8140000000000003e-05 2024-02-23T00:48:14.235 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.235 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.236 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:14.236 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.508691+0000", 2024-02-23T00:48:14.236 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014609599999999999 2024-02-23T00:48:14.236 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.236 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.236 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:14.236 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.508837+0000", 2024-02-23T00:48:14.236 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.13144261099999999 2024-02-23T00:48:14.236 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.236 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.236 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 2,5", 2024-02-23T00:48:14.236 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.640279+0000", 2024-02-23T00:48:14.237 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0094432939999999996 2024-02-23T00:48:14.237 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.237 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.237 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:48:14.237 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.649723+0000", 2024-02-23T00:48:14.237 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.027088807999999999 2024-02-23T00:48:14.237 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.237 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.237 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:48:14.237 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.676811+0000", 2024-02-23T00:48:14.237 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.016523389999999999 2024-02-23T00:48:14.237 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.237 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.237 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:48:14.237 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.693335+0000", 2024-02-23T00:48:14.238 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.1526999999999996e-05 2024-02-23T00:48:14.238 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.238 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.238 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:48:14.238 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.693396+0000", 2024-02-23T00:48:14.238 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.4455e-05 2024-02-23T00:48:14.238 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.238 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.238 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:14.238 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.693441+0000", 2024-02-23T00:48:14.238 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.18482968499999999 2024-02-23T00:48:14.238 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.239 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:14.239 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.239 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.239 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.239 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.25529.0:42 63.e 63:7037fd60:test-rados-api-smithi047-82496-10::foo:head [write 128~128] snapc c=[c,b,a] ondisk+write+known_if_redirected e142)", 2024-02-23T00:48:14.239 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:34.496096+0000", 2024-02-23T00:48:14.239 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 279.65107766699998, 2024-02-23T00:48:14.239 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.061437618999999999, 2024-02-23T00:48:14.239 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:14.239 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:48:14.239 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:48:14.239 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.25529", 2024-02-23T00:48:14.239 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/2064770043", 2024-02-23T00:48:14.240 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 42 2024-02-23T00:48:14.240 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.240 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:14.240 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.240 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:14.240 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.496096+0000", 2024-02-23T00:48:14.240 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:14.240 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.240 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.240 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:14.240 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.496096+0000", 2024-02-23T00:48:14.240 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999986 2024-02-23T00:48:14.240 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.240 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.241 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:14.241 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.496095+0000", 2024-02-23T00:48:14.241 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.0389999999999999e-06 2024-02-23T00:48:14.241 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.241 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.241 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:14.241 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.496102+0000", 2024-02-23T00:48:14.241 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2e-05 2024-02-23T00:48:14.241 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.241 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.241 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:14.241 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.496114+0000", 2024-02-23T00:48:14.241 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0590000000000003e-06 2024-02-23T00:48:14.241 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.242 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.242 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:14.242 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.496122+0000", 2024-02-23T00:48:14.242 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0080011270000000002 2024-02-23T00:48:14.242 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.242 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.242 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:14.242 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.504123+0000", 2024-02-23T00:48:14.242 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0022672809999999999 2024-02-23T00:48:14.242 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.242 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.242 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:14.242 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.506390+0000", 2024-02-23T00:48:14.242 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00021441099999999999 2024-02-23T00:48:14.243 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.243 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.243 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 2,3", 2024-02-23T00:48:14.243 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.506605+0000", 2024-02-23T00:48:14.243 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0067776329999999999 2024-02-23T00:48:14.243 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.243 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.243 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:48:14.243 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.513382+0000", 2024-02-23T00:48:14.243 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.036179537999999997 2024-02-23T00:48:14.243 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.243 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.243 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:48:14.243 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.549562+0000", 2024-02-23T00:48:14.243 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0078643380000000002 2024-02-23T00:48:14.244 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.244 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.244 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:48:14.244 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.557426+0000", 2024-02-23T00:48:14.244 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.9908999999999998e-05 2024-02-23T00:48:14.244 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.244 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.244 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:48:14.244 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.557486+0000", 2024-02-23T00:48:14.244 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.7828999999999999e-05 2024-02-23T00:48:14.244 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.244 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.244 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:14.244 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.557534+0000", 2024-02-23T00:48:14.245 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.061437618999999999 2024-02-23T00:48:14.245 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.245 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:14.245 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.245 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.245 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.245 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(client.34321.0:2 83.6 e143/142 83:602f83fe:::foo:head v 143'1, mlcod=0'0)", 2024-02-23T00:48:14.245 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:35.131473+0000", 2024-02-23T00:48:14.245 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 279.01570064600003, 2024-02-23T00:48:14.245 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.025342482, 2024-02-23T00:48:14.245 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:14.245 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:48:14.245 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:14.246 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.246 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:14.246 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:35.131473+0000", 2024-02-23T00:48:14.246 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:14.246 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.246 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.246 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:14.246 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:35.131473+0000", 2024-02-23T00:48:14.246 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:48:14.246 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.246 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.246 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:14.246 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:35.131473+0000", 2024-02-23T00:48:14.246 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.3780000000000003e-06 2024-02-23T00:48:14.247 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.247 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.247 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:14.247 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:35.131477+0000", 2024-02-23T00:48:14.247 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.25e-06 2024-02-23T00:48:14.247 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.247 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.247 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:14.247 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:35.131481+0000", 2024-02-23T00:48:14.247 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.6960000000000002e-06 2024-02-23T00:48:14.247 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.247 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.248 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:14.248 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:35.131487+0000", 2024-02-23T00:48:14.248 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012547017000000001 2024-02-23T00:48:14.248 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.248 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.248 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:14.248 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:35.144034+0000", 2024-02-23T00:48:14.248 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0068000000000002e-05 2024-02-23T00:48:14.248 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.248 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.248 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:14.249 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:35.144054+0000", 2024-02-23T00:48:14.249 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012724839999999999 2024-02-23T00:48:14.249 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.249 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.249 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:48:14.249 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:35.156779+0000", 2024-02-23T00:48:14.249 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.6603000000000002e-05 2024-02-23T00:48:14.249 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.249 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.249 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:14.249 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:35.156816+0000", 2024-02-23T00:48:14.249 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.025342482 2024-02-23T00:48:14.249 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.249 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:14.250 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.250 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.250 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.250 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.25529.0:51 63.10 63:0b20e63d:test-rados-api-smithi047-82496-11::foo:head [write 0~1048576] snapc 10=[10] ondisk+write+known_if_redirected e146)", 2024-02-23T00:48:14.250 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:38.411127+0000", 2024-02-23T00:48:14.250 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 275.73604663200001, 2024-02-23T00:48:14.250 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.031541215999999997, 2024-02-23T00:48:14.250 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:14.250 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:48:14.250 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:48:14.250 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.25529", 2024-02-23T00:48:14.250 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/2064770043", 2024-02-23T00:48:14.251 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 51 2024-02-23T00:48:14.251 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.251 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:14.251 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.251 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:14.251 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.411127+0000", 2024-02-23T00:48:14.251 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:14.251 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.251 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.251 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:14.251 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.411127+0000", 2024-02-23T00:48:14.252 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:48:14.252 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.252 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.252 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:14.252 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.411125+0000", 2024-02-23T00:48:14.252 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00060817599999999999 2024-02-23T00:48:14.252 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.252 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.252 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:14.252 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.411734+0000", 2024-02-23T00:48:14.253 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0353999999999999e-05 2024-02-23T00:48:14.253 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.253 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.253 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:14.253 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.411754+0000", 2024-02-23T00:48:14.253 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.5280000000000003e-06 2024-02-23T00:48:14.253 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.253 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.253 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:14.253 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.411763+0000", 2024-02-23T00:48:14.253 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0017375159999999999 2024-02-23T00:48:14.253 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.253 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.254 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:14.254 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.413500+0000", 2024-02-23T00:48:14.254 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012374400000000001 2024-02-23T00:48:14.254 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.254 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.254 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:14.254 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.413624+0000", 2024-02-23T00:48:14.254 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000156458 2024-02-23T00:48:14.254 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.254 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.254 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 0,7", 2024-02-23T00:48:14.254 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.413780+0000", 2024-02-23T00:48:14.254 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0035642120000000002 2024-02-23T00:48:14.255 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.255 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.255 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:48:14.255 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.417344+0000", 2024-02-23T00:48:14.255 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0014813980000000001 2024-02-23T00:48:14.255 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.255 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.255 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:48:14.255 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.418826+0000", 2024-02-23T00:48:14.255 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021229049 2024-02-23T00:48:14.255 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.256 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.256 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:48:14.256 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.440055+0000", 2024-02-23T00:48:14.256 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0025668330000000001 2024-02-23T00:48:14.256 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.256 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.256 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:48:14.256 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.442622+0000", 2024-02-23T00:48:14.256 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.6690999999999997e-05 2024-02-23T00:48:14.256 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.256 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.256 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:14.257 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.442668+0000", 2024-02-23T00:48:14.257 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.031541215999999997 2024-02-23T00:48:14.257 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.257 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:14.257 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.257 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.257 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.257 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop_reply(osd.1.0:444 93.8 e198/154)", 2024-02-23T00:48:14.257 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:31.157526+0000", 2024-02-23T00:48:14.257 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 222.98964812700001, 2024-02-23T00:48:14.257 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021858437000000001, 2024-02-23T00:48:14.258 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:14.258 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "queued for pg", 2024-02-23T00:48:14.258 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:14.258 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.258 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:14.258 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.157526+0000", 2024-02-23T00:48:14.258 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:14.258 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.258 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.258 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:14.258 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.157526+0000", 2024-02-23T00:48:14.258 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999986 2024-02-23T00:48:14.258 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.259 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.259 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:14.259 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.157524+0000", 2024-02-23T00:48:14.259 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1851000000000001e-05 2024-02-23T00:48:14.259 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.259 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.259 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:14.259 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.157536+0000", 2024-02-23T00:48:14.259 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021730283999999999 2024-02-23T00:48:14.259 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.259 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.259 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:14.259 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.179266+0000", 2024-02-23T00:48:14.260 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.4079999999999998e-06 2024-02-23T00:48:14.260 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.260 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.260 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:14.260 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.179275+0000", 2024-02-23T00:48:14.260 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00010933 2024-02-23T00:48:14.260 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.260 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.260 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:14.260 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.179384+0000", 2024-02-23T00:48:14.260 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021858437000000001 2024-02-23T00:48:14.260 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.261 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:14.261 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.261 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.261 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.261 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWrite(131.5s1 199/198 ECSubWrite(tid=1455, reqid=client.40652.0:1, at_version=199'1, trim_to=0'0, roll_forward_to=0'0))", 2024-02-23T00:48:14.261 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:31.177786+0000", 2024-02-23T00:48:14.261 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 222.96938825800001, 2024-02-23T00:48:14.261 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.022289363, 2024-02-23T00:48:14.261 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:14.261 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-02-23T00:48:14.261 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:14.261 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.261 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:14.261 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.177786+0000", 2024-02-23T00:48:14.262 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:14.262 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.262 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.262 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:14.262 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.177786+0000", 2024-02-23T00:48:14.262 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967296 2024-02-23T00:48:14.262 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.262 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.262 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:14.262 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.177785+0000", 2024-02-23T00:48:14.262 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0548e-05 2024-02-23T00:48:14.262 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.262 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.263 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:14.263 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.177816+0000", 2024-02-23T00:48:14.263 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.0389999999999993e-06 2024-02-23T00:48:14.263 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.263 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.263 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:14.263 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.177825+0000", 2024-02-23T00:48:14.263 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3852999999999999e-05 2024-02-23T00:48:14.263 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.263 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.263 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:14.263 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.177839+0000", 2024-02-23T00:48:14.263 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0065961850000000001 2024-02-23T00:48:14.263 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.264 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.264 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:14.264 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.184435+0000", 2024-02-23T00:48:14.264 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2191000000000001e-05 2024-02-23T00:48:14.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.264 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.264 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-02-23T00:48:14.264 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.184447+0000", 2024-02-23T00:48:14.264 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.01559761 2024-02-23T00:48:14.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.265 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.265 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-02-23T00:48:14.265 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.200045+0000", 2024-02-23T00:48:14.265 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0137e-05 2024-02-23T00:48:14.265 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.265 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.265 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:14.265 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.200075+0000", 2024-02-23T00:48:14.265 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.022289363 2024-02-23T00:48:14.265 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.265 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:14.265 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.265 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.265 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.266 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(osd.6.0:446 62.1a e198/126 62:5b3390c0:test-rados-api-smithi047-82465-14::foo25:4 v 198'22, mlcod=159'16)", 2024-02-23T00:48:14.266 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:31.179336+0000", 2024-02-23T00:48:14.266 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 222.96783781100001, 2024-02-23T00:48:14.266 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.024145261000000001, 2024-02-23T00:48:14.266 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:14.266 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:48:14.266 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:14.266 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.266 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:14.266 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.179336+0000", 2024-02-23T00:48:14.266 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:14.266 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.267 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.267 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:14.267 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.179336+0000", 2024-02-23T00:48:14.267 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:48:14.267 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.267 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.267 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:14.267 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.179336+0000", 2024-02-23T00:48:14.267 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0668e-05 2024-02-23T00:48:14.267 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.267 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.267 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:14.267 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.179346+0000", 2024-02-23T00:48:14.268 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.3579999999999999e-06 2024-02-23T00:48:14.268 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.268 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.268 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:14.268 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.179353+0000", 2024-02-23T00:48:14.268 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.2529999999999998e-06 2024-02-23T00:48:14.268 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.268 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.268 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:14.268 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.179357+0000", 2024-02-23T00:48:14.268 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.011838892 2024-02-23T00:48:14.268 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.268 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.268 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:14.269 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.191196+0000", 2024-02-23T00:48:14.269 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.2711999999999999e-05 2024-02-23T00:48:14.269 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.269 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.269 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:14.269 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.191219+0000", 2024-02-23T00:48:14.269 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012223862 2024-02-23T00:48:14.269 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.269 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.269 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:48:14.269 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.203442+0000", 2024-02-23T00:48:14.269 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.8853999999999998e-05 2024-02-23T00:48:14.269 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.270 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.270 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:14.270 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.203481+0000", 2024-02-23T00:48:14.270 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.024145261000000001 2024-02-23T00:48:14.270 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.270 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:14.270 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.270 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.270 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.270 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.41147.0:1 193.1s0 193:90ca3134:::oid_FlushAsyncPP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e339)", 2024-02-23T00:48:14.270 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:46:51.185479+0000", 2024-02-23T00:48:14.270 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 82.961694710000003, 2024-02-23T00:48:14.270 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.013269247, 2024-02-23T00:48:14.271 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:14.271 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:48:14.271 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:48:14.271 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.41147", 2024-02-23T00:48:14.271 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1633196565", 2024-02-23T00:48:14.271 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 1 2024-02-23T00:48:14.271 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.271 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:14.271 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.271 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:14.271 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:51.185479+0000", 2024-02-23T00:48:14.271 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:14.271 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.271 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.272 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:14.272 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:51.185479+0000", 2024-02-23T00:48:14.272 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:48:14.272 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.272 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.272 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:14.272 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:51.185477+0000", 2024-02-23T00:48:14.272 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.5139999999999998e-06 2024-02-23T00:48:14.272 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.272 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.272 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:14.272 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:51.185484+0000", 2024-02-23T00:48:14.272 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.9850000000000006e-06 2024-02-23T00:48:14.273 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.273 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.273 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:14.273 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:51.185492+0000", 2024-02-23T00:48:14.273 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.4729999999999999e-06 2024-02-23T00:48:14.273 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.273 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.273 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:14.273 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:51.185497+0000", 2024-02-23T00:48:14.273 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.3026999999999999e-05 2024-02-23T00:48:14.273 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.273 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.273 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:14.273 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:51.185550+0000", 2024-02-23T00:48:14.274 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8739999999999999e-06 2024-02-23T00:48:14.274 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.274 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.274 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:48:14.274 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:51.185553+0000", 2024-02-23T00:48:14.274 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.009254662 2024-02-23T00:48:14.274 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.274 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.274 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:14.274 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:52.194808+0000", 2024-02-23T00:48:14.274 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00015087 2024-02-23T00:48:14.274 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.274 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.274 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:14.275 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:52.194959+0000", 2024-02-23T00:48:14.275 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00046873999999999999 2024-02-23T00:48:14.275 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.275 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.275 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-02-23T00:48:14.275 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:52.195427+0000", 2024-02-23T00:48:14.275 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000537624 2024-02-23T00:48:14.275 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.275 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.275 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-02-23T00:48:14.275 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:52.195965+0000", 2024-02-23T00:48:14.275 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0027150640000000001 2024-02-23T00:48:14.275 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.275 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.275 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:48:14.276 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:52.198680+0000", 2024-02-23T00:48:14.276 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.8254000000000001e-05 2024-02-23T00:48:14.276 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.276 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.276 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:14.276 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:52.198748+0000", 2024-02-23T00:48:14.276 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.013269247 2024-02-23T00:48:14.276 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.276 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:14.276 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.276 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.276 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.276 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24403.0:479 3.0 3:353ec8f9:::gateway.conf:head [getxattr epoch (1) in=5b] snapc 0=[] ondisk+read+known_if_redirected e374)", 2024-02-23T00:48:14.276 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:47:27.842443+0000", 2024-02-23T00:48:14.277 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 46.304731033000003, 2024-02-23T00:48:14.277 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.38625488699999999, 2024-02-23T00:48:14.277 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:14.277 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:48:14.277 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:48:14.277 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24403", 2024-02-23T00:48:14.277 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.196:0/519523863", 2024-02-23T00:48:14.277 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 479 2024-02-23T00:48:14.277 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.277 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:14.277 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.277 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:14.277 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:27.842443+0000", 2024-02-23T00:48:14.278 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:14.278 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.278 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.278 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:14.278 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:27.842443+0000", 2024-02-23T00:48:14.278 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999971 2024-02-23T00:48:14.278 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.278 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.278 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:14.278 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:27.842440+0000", 2024-02-23T00:48:14.278 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6144e-05 2024-02-23T00:48:14.278 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.278 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.278 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:14.278 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:27.842456+0000", 2024-02-23T00:48:14.279 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.4341e-05 2024-02-23T00:48:14.279 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.279 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.279 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:14.279 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:27.842480+0000", 2024-02-23T00:48:14.279 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.0359999999999995e-06 2024-02-23T00:48:14.279 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.279 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.279 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:14.279 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:27.842490+0000", 2024-02-23T00:48:14.279 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000117791 2024-02-23T00:48:14.279 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.279 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.279 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:14.280 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:27.842607+0000", 2024-02-23T00:48:14.280 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1094000000000001e-05 2024-02-23T00:48:14.280 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.280 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.280 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:48:14.280 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:27.842618+0000", 2024-02-23T00:48:14.280 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.38582140700000001 2024-02-23T00:48:14.280 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.280 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.280 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:14.280 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:28.228440+0000", 2024-02-23T00:48:14.280 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014037300000000001 2024-02-23T00:48:14.280 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.280 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.281 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:14.281 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:28.228580+0000", 2024-02-23T00:48:14.281 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000117533 2024-02-23T00:48:14.281 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:14.281 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:14.281 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:14.281 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:28.228698+0000", 2024-02-23T00:48:14.281 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.38625488699999999 2024-02-23T00:48:14.281 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.281 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:14.281 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.281 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:14.281 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:14.281 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:48:14.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:12.885523+0000 osd.7 (osd.7) 201 : cluster [DBG] 62.14 deep-scrub starts 2024-02-23T00:48:14.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:12.892454+0000 osd.7 (osd.7) 202 : cluster [DBG] 62.14 deep-scrub ok 2024-02-23T00:48:14.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:13.278284+0000 mon.a (mon.0) 2574 : cluster [DBG] osdmap e421: 8 total, 8 up, 7 in 2024-02-23T00:48:14.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:13.287064+0000 mgr.y (mgr.14517) 383 : cluster [DBG] pgmap v607: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-23T00:48:14.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:13.461134+0000 osd.4 (osd.4) 163 : cluster [DBG] 6.2 scrub starts 2024-02-23T00:48:14.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:13.462857+0000 osd.4 (osd.4) 164 : cluster [DBG] 6.2 scrub ok 2024-02-23T00:48:14.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:13.819012+0000 osd.1 (osd.1) 183 : cluster [DBG] 6.1 scrub starts 2024-02-23T00:48:14.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:13.821897+0000 osd.1 (osd.1) 184 : cluster [DBG] 6.1 scrub ok 2024-02-23T00:48:14.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:14.029546+0000 osd.3 (osd.3) 196 : cluster [DBG] 4.10 scrub starts 2024-02-23T00:48:14.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:14.033033+0000 osd.3 (osd.3) 197 : cluster [DBG] 4.10 scrub ok 2024-02-23T00:48:14.561 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:12.885523+0000 osd.7 (osd.7) 201 : cluster [DBG] 62.14 deep-scrub starts 2024-02-23T00:48:14.561 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:12.892454+0000 osd.7 (osd.7) 202 : cluster [DBG] 62.14 deep-scrub ok 2024-02-23T00:48:14.561 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:13.278284+0000 mon.a (mon.0) 2574 : cluster [DBG] osdmap e421: 8 total, 8 up, 7 in 2024-02-23T00:48:14.561 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:13.287064+0000 mgr.y (mgr.14517) 383 : cluster [DBG] pgmap v607: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-23T00:48:14.562 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:13.461134+0000 osd.4 (osd.4) 163 : cluster [DBG] 6.2 scrub starts 2024-02-23T00:48:14.562 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:13.462857+0000 osd.4 (osd.4) 164 : cluster [DBG] 6.2 scrub ok 2024-02-23T00:48:14.562 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:13.819012+0000 osd.1 (osd.1) 183 : cluster [DBG] 6.1 scrub starts 2024-02-23T00:48:14.562 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:13.821897+0000 osd.1 (osd.1) 184 : cluster [DBG] 6.1 scrub ok 2024-02-23T00:48:14.562 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:14.029546+0000 osd.3 (osd.3) 196 : cluster [DBG] 4.10 scrub starts 2024-02-23T00:48:14.562 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:14.033033+0000 osd.3 (osd.3) 197 : cluster [DBG] 4.10 scrub ok 2024-02-23T00:48:14.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:12.885523+0000 osd.7 (osd.7) 201 : cluster [DBG] 62.14 deep-scrub starts 2024-02-23T00:48:14.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:12.892454+0000 osd.7 (osd.7) 202 : cluster [DBG] 62.14 deep-scrub ok 2024-02-23T00:48:14.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:13.278284+0000 mon.a (mon.0) 2574 : cluster [DBG] osdmap e421: 8 total, 8 up, 7 in 2024-02-23T00:48:14.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:13.287064+0000 mgr.y (mgr.14517) 383 : cluster [DBG] pgmap v607: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-23T00:48:14.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:13.461134+0000 osd.4 (osd.4) 163 : cluster [DBG] 6.2 scrub starts 2024-02-23T00:48:14.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:13.462857+0000 osd.4 (osd.4) 164 : cluster [DBG] 6.2 scrub ok 2024-02-23T00:48:14.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:13.819012+0000 osd.1 (osd.1) 183 : cluster [DBG] 6.1 scrub starts 2024-02-23T00:48:14.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:13.821897+0000 osd.1 (osd.1) 184 : cluster [DBG] 6.1 scrub ok 2024-02-23T00:48:14.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:14.029546+0000 osd.3 (osd.3) 196 : cluster [DBG] 4.10 scrub starts 2024-02-23T00:48:14.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:14.033033+0000 osd.3 (osd.3) 197 : cluster [DBG] 4.10 scrub ok 2024-02-23T00:48:14.716 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:48:15.017 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:48:15.021 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:15.022 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:48:15.123 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:48:15.151 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:15.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:13.583660+0000 osd.6 (osd.6) 167 : cluster [DBG] 2.7 scrub starts 2024-02-23T00:48:15.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:13.584556+0000 osd.6 (osd.6) 168 : cluster [DBG] 2.7 scrub ok 2024-02-23T00:48:15.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:13.866769+0000 osd.0 (osd.0) 169 : cluster [DBG] 6.9 deep-scrub starts 2024-02-23T00:48:15.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:13.867694+0000 osd.0 (osd.0) 170 : cluster [DBG] 6.9 deep-scrub ok 2024-02-23T00:48:15.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:14.288994+0000 mon.a (mon.0) 2575 : cluster [DBG] osdmap e422: 8 total, 8 up, 7 in 2024-02-23T00:48:15.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:15.182721+0000 mon.a (mon.0) 2576 : cluster [DBG] osdmap e423: 8 total, 8 up, 7 in 2024-02-23T00:48:15.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:13.583660+0000 osd.6 (osd.6) 167 : cluster [DBG] 2.7 scrub starts 2024-02-23T00:48:15.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:13.584556+0000 osd.6 (osd.6) 168 : cluster [DBG] 2.7 scrub ok 2024-02-23T00:48:15.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:13.866769+0000 osd.0 (osd.0) 169 : cluster [DBG] 6.9 deep-scrub starts 2024-02-23T00:48:15.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:13.867694+0000 osd.0 (osd.0) 170 : cluster [DBG] 6.9 deep-scrub ok 2024-02-23T00:48:15.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:14.288994+0000 mon.a (mon.0) 2575 : cluster [DBG] osdmap e422: 8 total, 8 up, 7 in 2024-02-23T00:48:15.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:15.182721+0000 mon.a (mon.0) 2576 : cluster [DBG] osdmap e423: 8 total, 8 up, 7 in 2024-02-23T00:48:15.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:13.583660+0000 osd.6 (osd.6) 167 : cluster [DBG] 2.7 scrub starts 2024-02-23T00:48:15.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:13.584556+0000 osd.6 (osd.6) 168 : cluster [DBG] 2.7 scrub ok 2024-02-23T00:48:15.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:13.866769+0000 osd.0 (osd.0) 169 : cluster [DBG] 6.9 deep-scrub starts 2024-02-23T00:48:15.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:13.867694+0000 osd.0 (osd.0) 170 : cluster [DBG] 6.9 deep-scrub ok 2024-02-23T00:48:15.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:14.288994+0000 mon.a (mon.0) 2575 : cluster [DBG] osdmap e422: 8 total, 8 up, 7 in 2024-02-23T00:48:15.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:15.182721+0000 mon.a (mon.0) 2576 : cluster [DBG] osdmap e423: 8 total, 8 up, 7 in 2024-02-23T00:48:16.294 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:14.864258+0000 osd.0 (osd.0) 171 : cluster [DBG] 62.12 scrub starts 2024-02-23T00:48:16.294 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:14.866245+0000 osd.0 (osd.0) 172 : cluster [DBG] 62.12 scrub ok 2024-02-23T00:48:16.294 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:14.919534+0000 osd.7 (osd.7) 203 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:48:16.294 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:14.919744+0000 osd.7 (osd.7) 204 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:48:16.294 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:15.287518+0000 mgr.y (mgr.14517) 384 : cluster [DBG] pgmap v610: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-23T00:48:16.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:14.864258+0000 osd.0 (osd.0) 171 : cluster [DBG] 62.12 scrub starts 2024-02-23T00:48:16.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:14.866245+0000 osd.0 (osd.0) 172 : cluster [DBG] 62.12 scrub ok 2024-02-23T00:48:16.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:14.919534+0000 osd.7 (osd.7) 203 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:48:16.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:14.919744+0000 osd.7 (osd.7) 204 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:48:16.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:15.287518+0000 mgr.y (mgr.14517) 384 : cluster [DBG] pgmap v610: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-23T00:48:16.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:14.864258+0000 osd.0 (osd.0) 171 : cluster [DBG] 62.12 scrub starts 2024-02-23T00:48:16.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:14.866245+0000 osd.0 (osd.0) 172 : cluster [DBG] 62.12 scrub ok 2024-02-23T00:48:16.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:14.919534+0000 osd.7 (osd.7) 203 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:48:16.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:14.919744+0000 osd.7 (osd.7) 204 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:48:16.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:15.287518+0000 mgr.y (mgr.14517) 384 : cluster [DBG] pgmap v610: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-23T00:48:17.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.042+0000 7f254652f700 1 -- 172.21.15.47:0/282246323 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f25400fe510 msgr2=0x7f25400f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:17.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.042+0000 7f254652f700 1 --2- 172.21.15.47:0/282246323 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f25400fe510 0x7f25400f7d40 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 rx=0x7f2530004870 tx=0x7f2530009cb0).stop 2024-02-23T00:48:17.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.042+0000 7f254652f700 1 -- 172.21.15.47:0/282246323 shutdown_connections 2024-02-23T00:48:17.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.042+0000 7f254652f700 1 --2- 172.21.15.47:0/282246323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25400f8420 0x7f2540104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:17.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.042+0000 7f254652f700 1 --2- 172.21.15.47:0/282246323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25400fdbb0 0x7f25400fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:17.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.042+0000 7f254652f700 1 --2- 172.21.15.47:0/282246323 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f25400fe510 0x7f25400f7d40 unknown :-1 s=CLOSED pgs=2664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:17.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.042+0000 7f254652f700 1 -- 172.21.15.47:0/282246323 >> 172.21.15.47:0/282246323 conn(0x7f25400f3800 msgr2=0x7f25400f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:48:17.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.042+0000 7f254652f700 1 -- 172.21.15.47:0/282246323 shutdown_connections 2024-02-23T00:48:17.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.042+0000 7f254652f700 1 -- 172.21.15.47:0/282246323 wait complete. 2024-02-23T00:48:17.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.043+0000 7f254652f700 1 librados: starting msgr at 2024-02-23T00:48:17.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.043+0000 7f254652f700 1 librados: starting objecter 2024-02-23T00:48:17.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.043+0000 7f254652f700 1 Processor -- start 2024-02-23T00:48:17.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.043+0000 7f254652f700 1 -- start start 2024-02-23T00:48:17.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.043+0000 7f254652f700 1 librados: setting wanted keys 2024-02-23T00:48:17.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.043+0000 7f254652f700 1 librados: calling monclient init 2024-02-23T00:48:17.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.043+0000 7f254652f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25400f8420 0x7f254010de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:17.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.043+0000 7f254652f700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f25400fdbb0 0x7f254010e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:17.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.043+0000 7f254652f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25400fe510 0x7f2540107ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:17.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.043+0000 7f254652f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25401084e0 con 0x7f25400f8420 2024-02-23T00:48:17.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.044+0000 7f254652f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2540108650 con 0x7f25400fe510 2024-02-23T00:48:17.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.044+0000 7f254652f700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2540108810 con 0x7f25400fdbb0 2024-02-23T00:48:17.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.044+0000 7f253ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25400f8420 0x7f254010de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:17.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.044+0000 7f253f7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f25400fdbb0 0x7f254010e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:17.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.044+0000 7f253f7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f25400fdbb0 0x7f254010e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:55602/0 (socket says 172.21.15.47:55602) 2024-02-23T00:48:17.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.044+0000 7f253ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25400f8420 0x7f254010de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40356/0 (socket says 172.21.15.47:40356) 2024-02-23T00:48:17.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.044+0000 7f253f7fe700 1 -- 172.21.15.47:0/681971612 learned_addr learned my addr 172.21.15.47:0/681971612 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:48:17.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.044+0000 7f2544acc700 1 --2- 172.21.15.47:0/681971612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25400fe510 0x7f2540107ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:17.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.044+0000 7f253f7fe700 1 -- 172.21.15.47:0/681971612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25400fe510 msgr2=0x7f2540107ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:17.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.044+0000 7f253f7fe700 1 --2- 172.21.15.47:0/681971612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25400fe510 0x7f2540107ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:17.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.044+0000 7f253f7fe700 1 -- 172.21.15.47:0/681971612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25400f8420 msgr2=0x7f254010de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:17.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.044+0000 7f253f7fe700 1 --2- 172.21.15.47:0/681971612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25400f8420 0x7f254010de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:17.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.044+0000 7f253f7fe700 1 -- 172.21.15.47:0/681971612 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25380097c0 con 0x7f25400fdbb0 2024-02-23T00:48:17.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.044+0000 7f253ffff700 1 --2- 172.21.15.47:0/681971612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25400f8420 0x7f254010de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:48:17.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.044+0000 7f253f7fe700 1 --2- 172.21.15.47:0/681971612 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f25400fdbb0 0x7f254010e340 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 rx=0x7f2530009da0 tx=0x7f2530009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:17.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.044+0000 7f253d7fa700 10 client.?.objecter ms_handle_connect 0x7f25400fdbb0 2024-02-23T00:48:17.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.044+0000 7f253d7fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:48:17.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.044+0000 7f253d7fa700 1 -- 172.21.15.47:0/681971612 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253000e030 con 0x7f25400fdbb0 2024-02-23T00:48:17.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.044+0000 7f254652f700 1 -- 172.21.15.47:0/681971612 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2530005040 con 0x7f25400fdbb0 2024-02-23T00:48:17.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.044+0000 7f253d7fa700 1 -- 172.21.15.47:0/681971612 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f2530013e70 con 0x7f25400fdbb0 2024-02-23T00:48:17.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.044+0000 7f253d7fa700 1 -- 172.21.15.47:0/681971612 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2530018070 con 0x7f25400fdbb0 2024-02-23T00:48:17.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.044+0000 7f254652f700 10 client.41333.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:48:17.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.044+0000 7f254652f700 1 -- 172.21.15.47:0/681971612 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2540108d30 con 0x7f25400fdbb0 2024-02-23T00:48:17.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.044+0000 7f254652f700 1 librados: init done 2024-02-23T00:48:17.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.045+0000 7f254652f700 1 -- 172.21.15.47:0/681971612 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f254005ee80 con 0x7f25400fdbb0 2024-02-23T00:48:17.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.046+0000 7f253d7fa700 1 -- 172.21.15.47:0/681971612 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f2530009240 con 0x7f25400fdbb0 2024-02-23T00:48:17.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.046+0000 7f253d7fa700 1 --2- 172.21.15.47:0/681971612 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f252c062cc0 0x7f252c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:17.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.046+0000 7f253d7fa700 1 -- 172.21.15.47:0/681971612 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(423..423 src has 48..423) v4 ==== 9248+0+0 (secure 0 0 0) 0x7f253007e970 con 0x7f25400fdbb0 2024-02-23T00:48:17.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.046+0000 7f253d7fa700 10 client.41333.objecter ms_dispatch 0x7f2540001550 osd_map(423..423 src has 48..423) v4 2024-02-23T00:48:17.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.046+0000 7f253d7fa700 3 client.41333.objecter handle_osd_map got epochs [423,423] > 0 2024-02-23T00:48:17.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.046+0000 7f253d7fa700 3 client.41333.objecter handle_osd_map decoding full epoch 423 2024-02-23T00:48:17.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.046+0000 7f253ffff700 1 --2- 172.21.15.47:0/681971612 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f252c062cc0 0x7f252c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:17.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.046+0000 7f253d7fa700 20 client.41333.objecter dump_active .. 0 homeless 2024-02-23T00:48:17.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.046+0000 7f253d7fa700 10 client.41333.objecter ms_handle_connect 0x7f252c062cc0 2024-02-23T00:48:17.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.046+0000 7f253ffff700 1 --2- 172.21.15.47:0/681971612 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f252c062cc0 0x7f252c065180 secure :-1 s=READY pgs=3064 cs=0 l=1 rev1=1 rx=0x7f25280041d0 tx=0x7f2528009040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:17.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.050+0000 7f253d7fa700 1 -- 172.21.15.47:0/681971612 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f253004df70 con 0x7f25400fdbb0 2024-02-23T00:48:17.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.223+0000 7f254652f700 1 -- 172.21.15.47:0/681971612 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "osd reweight-by-utilization", "oload": 1000, "max_change": 0.05, "target": ["mon-mgr", ""]}) v1 -- 0x7f25401095d0 con 0x7f252c062cc0 2024-02-23T00:48:17.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.224+0000 7f253d7fa700 1 -- 172.21.15.47:0/681971612 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 no change) v1 ==== 17+0+302 (secure 0 0 0) 0x7f2528004bf0 con 0x7f252c062cc0 2024-02-23T00:48:17.226 INFO:teuthology.orchestra.run.smithi047.stdout:moved 0 / 624 (0%) 2024-02-23T00:48:17.226 INFO:teuthology.orchestra.run.smithi047.stdout:avg 89.1429 2024-02-23T00:48:17.226 INFO:teuthology.orchestra.run.smithi047.stdout:stddev 7.84545 -> 7.84545 (expected baseline 8.74118) 2024-02-23T00:48:17.227 INFO:teuthology.orchestra.run.smithi047.stdout:min osd.5 with 75 -> 75 pgs (0.841346 -> 0.841346 * mean) 2024-02-23T00:48:17.227 INFO:teuthology.orchestra.run.smithi047.stdout:max osd.7 with 100 -> 100 pgs (1.12179 -> 1.12179 * mean) 2024-02-23T00:48:17.227 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:48:17.227 INFO:teuthology.orchestra.run.smithi047.stdout:oload 1000 2024-02-23T00:48:17.227 INFO:teuthology.orchestra.run.smithi047.stdout:max_change 0.05 2024-02-23T00:48:17.227 INFO:teuthology.orchestra.run.smithi047.stdout:max_change_osds 4 2024-02-23T00:48:17.227 INFO:teuthology.orchestra.run.smithi047.stdout:average_utilization 0.0033 2024-02-23T00:48:17.227 INFO:teuthology.orchestra.run.smithi047.stdout:overload_utilization 0.0330 2024-02-23T00:48:17.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.225+0000 7f254652f700 10 librados: watch_flush enter 2024-02-23T00:48:17.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.225+0000 7f254652f700 10 librados: watch_flush exit 2024-02-23T00:48:17.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.225+0000 7f254652f700 20 client.41333.objecter shutdown clearing up homeless session... 2024-02-23T00:48:17.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.225+0000 7f254652f700 10 client.41333.objecter successfully canceled tick 2024-02-23T00:48:17.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.226+0000 7f254652f700 1 -- 172.21.15.47:0/681971612 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f252c062cc0 msgr2=0x7f252c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:17.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.226+0000 7f254652f700 1 --2- 172.21.15.47:0/681971612 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f252c062cc0 0x7f252c065180 secure :-1 s=READY pgs=3064 cs=0 l=1 rev1=1 rx=0x7f25280041d0 tx=0x7f2528009040).stop 2024-02-23T00:48:17.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.226+0000 7f254652f700 1 -- 172.21.15.47:0/681971612 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f25400fdbb0 msgr2=0x7f254010e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:17.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.226+0000 7f254652f700 1 --2- 172.21.15.47:0/681971612 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f25400fdbb0 0x7f254010e340 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 rx=0x7f2530009da0 tx=0x7f2530009cb0).stop 2024-02-23T00:48:17.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.226+0000 7f254652f700 1 -- 172.21.15.47:0/681971612 shutdown_connections 2024-02-23T00:48:17.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.226+0000 7f254652f700 1 --2- 172.21.15.47:0/681971612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25400fe510 0x7f2540107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:17.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.226+0000 7f254652f700 1 --2- 172.21.15.47:0/681971612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25400f8420 0x7f254010de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:17.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.226+0000 7f254652f700 1 --2- 172.21.15.47:0/681971612 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f252c062cc0 0x7f252c065180 unknown :-1 s=CLOSED pgs=3064 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:17.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.226+0000 7f254652f700 1 --2- 172.21.15.47:0/681971612 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f25400fdbb0 0x7f254010e340 unknown :-1 s=CLOSED pgs=2665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:17.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.226+0000 7f254652f700 1 -- 172.21.15.47:0/681971612 >> 172.21.15.47:0/681971612 conn(0x7f25400f3800 msgr2=0x7f25400fb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:48:17.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.226+0000 7f254652f700 1 -- 172.21.15.47:0/681971612 shutdown_connections 2024-02-23T00:48:17.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.226+0000 7f254652f700 1 -- 172.21.15.47:0/681971612 wait complete. 2024-02-23T00:48:17.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:17.226+0000 7f254652f700 1 librados: shutdown 2024-02-23T00:48:17.230 INFO:teuthology.orchestra.run.smithi047.stderr:no change 2024-02-23T00:48:17.486 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.2 dump_ops_in_flight 2024-02-23T00:48:17.553 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:15.592452+0000 osd.6 (osd.6) 169 : cluster [DBG] 18.10 scrub starts 2024-02-23T00:48:17.554 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:15.593480+0000 osd.6 (osd.6) 170 : cluster [DBG] 18.10 scrub ok 2024-02-23T00:48:17.554 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:15.864146+0000 osd.0 (osd.0) 173 : cluster [DBG] 62.13 scrub starts 2024-02-23T00:48:17.554 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:15.865958+0000 osd.0 (osd.0) 174 : cluster [DBG] 62.13 scrub ok 2024-02-23T00:48:17.554 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:16.380532+0000 osd.5 (osd.5) 176 : cluster [DBG] 18.c scrub starts 2024-02-23T00:48:17.554 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:16.383458+0000 osd.5 (osd.5) 177 : cluster [DBG] 18.c scrub ok 2024-02-23T00:48:17.554 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:16.787776+0000 osd.1 (osd.1) 185 : cluster [DBG] 5.1d scrub starts 2024-02-23T00:48:17.554 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:16.788802+0000 osd.1 (osd.1) 186 : cluster [DBG] 5.1d scrub ok 2024-02-23T00:48:17.554 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:17.050594+0000 osd.3 (osd.3) 198 : cluster [DBG] 4.0 scrub starts 2024-02-23T00:48:17.554 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:17.054713+0000 osd.3 (osd.3) 199 : cluster [DBG] 4.0 scrub ok 2024-02-23T00:48:17.554 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:15.592452+0000 osd.6 (osd.6) 169 : cluster [DBG] 18.10 scrub starts 2024-02-23T00:48:17.555 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:15.593480+0000 osd.6 (osd.6) 170 : cluster [DBG] 18.10 scrub ok 2024-02-23T00:48:17.555 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:15.864146+0000 osd.0 (osd.0) 173 : cluster [DBG] 62.13 scrub starts 2024-02-23T00:48:17.555 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:15.865958+0000 osd.0 (osd.0) 174 : cluster [DBG] 62.13 scrub ok 2024-02-23T00:48:17.555 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:16.380532+0000 osd.5 (osd.5) 176 : cluster [DBG] 18.c scrub starts 2024-02-23T00:48:17.555 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:16.383458+0000 osd.5 (osd.5) 177 : cluster [DBG] 18.c scrub ok 2024-02-23T00:48:17.555 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:16.787776+0000 osd.1 (osd.1) 185 : cluster [DBG] 5.1d scrub starts 2024-02-23T00:48:17.555 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:16.788802+0000 osd.1 (osd.1) 186 : cluster [DBG] 5.1d scrub ok 2024-02-23T00:48:17.555 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:17.050594+0000 osd.3 (osd.3) 198 : cluster [DBG] 4.0 scrub starts 2024-02-23T00:48:17.555 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:17.054713+0000 osd.3 (osd.3) 199 : cluster [DBG] 4.0 scrub ok 2024-02-23T00:48:17.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:15.592452+0000 osd.6 (osd.6) 169 : cluster [DBG] 18.10 scrub starts 2024-02-23T00:48:17.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:15.593480+0000 osd.6 (osd.6) 170 : cluster [DBG] 18.10 scrub ok 2024-02-23T00:48:17.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:15.864146+0000 osd.0 (osd.0) 173 : cluster [DBG] 62.13 scrub starts 2024-02-23T00:48:17.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:15.865958+0000 osd.0 (osd.0) 174 : cluster [DBG] 62.13 scrub ok 2024-02-23T00:48:17.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:16.380532+0000 osd.5 (osd.5) 176 : cluster [DBG] 18.c scrub starts 2024-02-23T00:48:17.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:16.383458+0000 osd.5 (osd.5) 177 : cluster [DBG] 18.c scrub ok 2024-02-23T00:48:17.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:16.787776+0000 osd.1 (osd.1) 185 : cluster [DBG] 5.1d scrub starts 2024-02-23T00:48:17.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:16.788802+0000 osd.1 (osd.1) 186 : cluster [DBG] 5.1d scrub ok 2024-02-23T00:48:17.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:17.050594+0000 osd.3 (osd.3) 198 : cluster [DBG] 4.0 scrub starts 2024-02-23T00:48:17.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:17.054713+0000 osd.3 (osd.3) 199 : cluster [DBG] 4.0 scrub ok 2024-02-23T00:48:17.711 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:48:17.901 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:48:17.905 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:17.905 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:48:18.006 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:48:18.038 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:18.060 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:48:18.062 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:18.063 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:48:18.164 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:48:18.194 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:18.215 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:48:18.218 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:18.218 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:48:18.319 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:48:18.351 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:18.372 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:48:18.375 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:18.375 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:48:18.476 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:48:18.503 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:17.224532+0000 mgr.y (mgr.14517) 385 : audit [DBG] from='client.41333 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 1000, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:48:18.503 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:17.288294+0000 mgr.y (mgr.14517) 386 : cluster [DBG] pgmap v611: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.2 KiB/s rd, 816 B/s wr, 3 op/s 2024-02-23T00:48:18.503 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:17.430451+0000 osd.4 (osd.4) 165 : cluster [DBG] 4.1 deep-scrub starts 2024-02-23T00:48:18.503 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:17.435899+0000 osd.4 (osd.4) 166 : cluster [DBG] 4.1 deep-scrub ok 2024-02-23T00:48:18.504 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:17.224532+0000 mgr.y (mgr.14517) 385 : audit [DBG] from='client.41333 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 1000, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:48:18.504 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:17.288294+0000 mgr.y (mgr.14517) 386 : cluster [DBG] pgmap v611: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.2 KiB/s rd, 816 B/s wr, 3 op/s 2024-02-23T00:48:18.504 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:17.430451+0000 osd.4 (osd.4) 165 : cluster [DBG] 4.1 deep-scrub starts 2024-02-23T00:48:18.504 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:17.435899+0000 osd.4 (osd.4) 166 : cluster [DBG] 4.1 deep-scrub ok 2024-02-23T00:48:18.505 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:18.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.512+0000 7fb2bafe6700 1 -- 172.21.15.47:0/1358857248 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb2b40f9610 msgr2=0x7fb2b4101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:18.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.512+0000 7fb2bafe6700 1 --2- 172.21.15.47:0/1358857248 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb2b40f9610 0x7fb2b4101d20 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 rx=0x7fb2a0004870 tx=0x7fb2a0009cb0).stop 2024-02-23T00:48:18.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.512+0000 7fb2bafe6700 1 -- 172.21.15.47:0/1358857248 shutdown_connections 2024-02-23T00:48:18.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.512+0000 7fb2bafe6700 1 --2- 172.21.15.47:0/1358857248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2b4102210 0x7fb2b4104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:18.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.512+0000 7fb2bafe6700 1 --2- 172.21.15.47:0/1358857248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2b40f8cb0 0x7fb2b40f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:18.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.512+0000 7fb2bafe6700 1 --2- 172.21.15.47:0/1358857248 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb2b40f9610 0x7fb2b4101d20 unknown :-1 s=CLOSED pgs=2666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:18.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.512+0000 7fb2bafe6700 1 -- 172.21.15.47:0/1358857248 >> 172.21.15.47:0/1358857248 conn(0x7fb2b40f2820 msgr2=0x7fb2b40f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:48:18.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.512+0000 7fb2bafe6700 1 -- 172.21.15.47:0/1358857248 shutdown_connections 2024-02-23T00:48:18.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.512+0000 7fb2bafe6700 1 -- 172.21.15.47:0/1358857248 wait complete. 2024-02-23T00:48:18.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.513+0000 7fb2bafe6700 1 librados: starting msgr at 2024-02-23T00:48:18.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.513+0000 7fb2bafe6700 1 librados: starting objecter 2024-02-23T00:48:18.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.513+0000 7fb2bafe6700 1 Processor -- start 2024-02-23T00:48:18.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.513+0000 7fb2bafe6700 1 -- start start 2024-02-23T00:48:18.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.513+0000 7fb2bafe6700 1 librados: setting wanted keys 2024-02-23T00:48:18.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.513+0000 7fb2bafe6700 1 librados: calling monclient init 2024-02-23T00:48:18.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.514+0000 7fb2bafe6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2b40f8cb0 0x7fb2b40f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:18.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.514+0000 7fb2bafe6700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2b40f9610 0x7fb2b40f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:18.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.514+0000 7fb2bafe6700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb2b4102210 0x7fb2b40f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:18.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.514+0000 7fb2bafe6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b41164d0 con 0x7fb2b40f8cb0 2024-02-23T00:48:18.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.514+0000 7fb2bafe6700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b4116640 con 0x7fb2b40f9610 2024-02-23T00:48:18.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.514+0000 7fb2bafe6700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b4116800 con 0x7fb2b4102210 2024-02-23T00:48:18.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.514+0000 7fb2b9583700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb2b4102210 0x7fb2b40f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:18.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.514+0000 7fb2b9583700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb2b4102210 0x7fb2b40f7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:55624/0 (socket says 172.21.15.47:55624) 2024-02-23T00:48:18.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.514+0000 7fb2b9583700 1 -- 172.21.15.47:0/865132933 learned_addr learned my addr 172.21.15.47:0/865132933 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:48:18.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.514+0000 7fb2b8d82700 1 --2- 172.21.15.47:0/865132933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2b40f8cb0 0x7fb2b40f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:18.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.514+0000 7fb2b9583700 1 -- 172.21.15.47:0/865132933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2b40f9610 msgr2=0x7fb2b40f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:18.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.514+0000 7fb2b9583700 1 --2- 172.21.15.47:0/865132933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2b40f9610 0x7fb2b40f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:48:18.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.514+0000 7fb2b9583700 1 -- 172.21.15.47:0/865132933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2b40f8cb0 msgr2=0x7fb2b40f6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:18.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.514+0000 7fb2b9583700 1 --2- 172.21.15.47:0/865132933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2b40f8cb0 0x7fb2b40f6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:18.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.514+0000 7fb2b9583700 1 -- 172.21.15.47:0/865132933 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ac0097c0 con 0x7fb2b4102210 2024-02-23T00:48:18.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.514+0000 7fb2b8d82700 1 --2- 172.21.15.47:0/865132933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2b40f8cb0 0x7fb2b40f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:48:18.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.514+0000 7fb2b9583700 1 --2- 172.21.15.47:0/865132933 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb2b4102210 0x7fb2b40f7640 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 rx=0x7fb2ac001df0 tx=0x7fb2ac004c50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:18.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.514+0000 7fb2b1ffb700 10 client.?.objecter ms_handle_connect 0x7fb2b4102210 2024-02-23T00:48:18.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.514+0000 7fb2b1ffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:48:18.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.515+0000 7fb2b1ffb700 1 -- 172.21.15.47:0/865132933 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac0099f0 con 0x7fb2b4102210 2024-02-23T00:48:18.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.515+0000 7fb2bafe6700 1 -- 172.21.15.47:0/865132933 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2a0005040 con 0x7fb2b4102210 2024-02-23T00:48:18.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.515+0000 7fb2b1ffb700 1 -- 172.21.15.47:0/865132933 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fb2ac009560 con 0x7fb2b4102210 2024-02-23T00:48:18.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.515+0000 7fb2b1ffb700 1 -- 172.21.15.47:0/865132933 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac007ce0 con 0x7fb2b4102210 2024-02-23T00:48:18.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.515+0000 7fb2bafe6700 10 client.41339.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:48:18.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.515+0000 7fb2bafe6700 1 -- 172.21.15.47:0/865132933 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b4108370 con 0x7fb2b4102210 2024-02-23T00:48:18.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.515+0000 7fb2bafe6700 1 librados: init done 2024-02-23T00:48:18.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.516+0000 7fb2bafe6700 1 -- 172.21.15.47:0/865132933 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2b405fa60 con 0x7fb2b4102210 2024-02-23T00:48:18.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.517+0000 7fb2b1ffb700 1 -- 172.21.15.47:0/865132933 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fb2ac0090d0 con 0x7fb2b4102210 2024-02-23T00:48:18.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.517+0000 7fb2b1ffb700 1 --2- 172.21.15.47:0/865132933 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb2a4062cc0 0x7fb2a4065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:18.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.517+0000 7fb2b1ffb700 1 -- 172.21.15.47:0/865132933 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(423..423 src has 48..423) v4 ==== 9248+0+0 (secure 0 0 0) 0x7fb2ac007e40 con 0x7fb2b4102210 2024-02-23T00:48:18.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.517+0000 7fb2b1ffb700 10 client.41339.objecter ms_dispatch 0x7fb2b4001550 osd_map(423..423 src has 48..423) v4 2024-02-23T00:48:18.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.517+0000 7fb2b1ffb700 3 client.41339.objecter handle_osd_map got epochs [423,423] > 0 2024-02-23T00:48:18.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.517+0000 7fb2b1ffb700 3 client.41339.objecter handle_osd_map decoding full epoch 423 2024-02-23T00:48:18.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.517+0000 7fb2b8d82700 1 --2- 172.21.15.47:0/865132933 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb2a4062cc0 0x7fb2a4065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:18.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.518+0000 7fb2b1ffb700 20 client.41339.objecter dump_active .. 0 homeless 2024-02-23T00:48:18.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.518+0000 7fb2b1ffb700 10 client.41339.objecter ms_handle_connect 0x7fb2a4062cc0 2024-02-23T00:48:18.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.518+0000 7fb2b8d82700 1 --2- 172.21.15.47:0/865132933 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb2a4062cc0 0x7fb2a4065180 secure :-1 s=READY pgs=3065 cs=0 l=1 rev1=1 rx=0x7fb2a80103c0 tx=0x7fb2a800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:18.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.521+0000 7fb2b1ffb700 1 -- 172.21.15.47:0/865132933 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb2ac04d920 con 0x7fb2b4102210 2024-02-23T00:48:18.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:17.224532+0000 mgr.y (mgr.14517) 385 : audit [DBG] from='client.41333 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 1000, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-02-23T00:48:18.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:17.288294+0000 mgr.y (mgr.14517) 386 : cluster [DBG] pgmap v611: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.2 KiB/s rd, 816 B/s wr, 3 op/s 2024-02-23T00:48:18.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:17.430451+0000 osd.4 (osd.4) 165 : cluster [DBG] 4.1 deep-scrub starts 2024-02-23T00:48:18.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:17.435899+0000 osd.4 (osd.4) 166 : cluster [DBG] 4.1 deep-scrub ok 2024-02-23T00:48:18.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:18.707+0000 7fb2bafe6700 1 -- 172.21.15.47:0/865132933 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7fb2b40f8140 con 0x7fb2b4102210 2024-02-23T00:48:18.756 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:48:18.769 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:18.769 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:48:18.870 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:48:18.902 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:18.924 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:48:18.926 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:18.927 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:48:19.028 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:48:19.057 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:19.182 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:48:19.185 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:19.185 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:48:19.286 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:48:19.306 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:17.897838+0000 osd.0 (osd.0) 175 : cluster [DBG] 18.8 scrub starts 2024-02-23T00:48:19.306 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:17.898717+0000 osd.0 (osd.0) 176 : cluster [DBG] 18.8 scrub ok 2024-02-23T00:48:19.306 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:18.709145+0000 mon.c (mon.2) 299 : audit [INF] from='client.? 172.21.15.47:0/865132933' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:48:19.306 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:18.709710+0000 mon.a (mon.0) 2577 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:48:19.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:17.897838+0000 osd.0 (osd.0) 175 : cluster [DBG] 18.8 scrub starts 2024-02-23T00:48:19.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:17.898717+0000 osd.0 (osd.0) 176 : cluster [DBG] 18.8 scrub ok 2024-02-23T00:48:19.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:18.709145+0000 mon.c (mon.2) 299 : audit [INF] from='client.? 172.21.15.47:0/865132933' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:48:19.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:18.709710+0000 mon.a (mon.0) 2577 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:48:19.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:19.306+0000 7fb2b1ffb700 1 -- 172.21.15.47:0/865132933 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v424) v1 ==== 87+0+0 (secure 0 0 0) 0x7fb2ac050f40 con 0x7fb2b4102210 2024-02-23T00:48:19.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:19.308+0000 7fb2bafe6700 10 librados: watch_flush enter 2024-02-23T00:48:19.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:19.308+0000 7fb2bafe6700 10 librados: watch_flush exit 2024-02-23T00:48:19.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:19.308+0000 7fb2bafe6700 20 client.41339.objecter shutdown clearing up homeless session... 2024-02-23T00:48:19.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:19.308+0000 7fb2bafe6700 10 client.41339.objecter successfully canceled tick 2024-02-23T00:48:19.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:19.308+0000 7fb2bafe6700 1 -- 172.21.15.47:0/865132933 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb2a4062cc0 msgr2=0x7fb2a4065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:19.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:19.308+0000 7fb2bafe6700 1 --2- 172.21.15.47:0/865132933 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb2a4062cc0 0x7fb2a4065180 secure :-1 s=READY pgs=3065 cs=0 l=1 rev1=1 rx=0x7fb2a80103c0 tx=0x7fb2a800b040).stop 2024-02-23T00:48:19.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:19.308+0000 7fb2bafe6700 1 -- 172.21.15.47:0/865132933 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb2b4102210 msgr2=0x7fb2b40f7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:19.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:19.308+0000 7fb2bafe6700 1 --2- 172.21.15.47:0/865132933 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb2b4102210 0x7fb2b40f7640 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 rx=0x7fb2ac001df0 tx=0x7fb2ac004c50).stop 2024-02-23T00:48:19.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:19.308+0000 7fb2bafe6700 1 -- 172.21.15.47:0/865132933 shutdown_connections 2024-02-23T00:48:19.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:19.308+0000 7fb2bafe6700 1 --2- 172.21.15.47:0/865132933 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb2a4062cc0 0x7fb2a4065180 unknown :-1 s=CLOSED pgs=3065 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:19.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:19.308+0000 7fb2bafe6700 1 --2- 172.21.15.47:0/865132933 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb2b4102210 0x7fb2b40f7640 unknown :-1 s=CLOSED pgs=2667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:19.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:19.308+0000 7fb2bafe6700 1 --2- 172.21.15.47:0/865132933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2b40f8cb0 0x7fb2b40f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:19.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:19.308+0000 7fb2bafe6700 1 --2- 172.21.15.47:0/865132933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2b40f9610 0x7fb2b40f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:48:19.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:19.308+0000 7fb2bafe6700 1 -- 172.21.15.47:0/865132933 >> 172.21.15.47:0/865132933 conn(0x7fb2b40f2820 msgr2=0x7fb2b40f3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:48:19.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:19.308+0000 7fb2bafe6700 1 -- 172.21.15.47:0/865132933 shutdown_connections 2024-02-23T00:48:19.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:19.308+0000 7fb2bafe6700 1 -- 172.21.15.47:0/865132933 wait complete. 2024-02-23T00:48:19.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:19.308+0000 7fb2bafe6700 1 librados: shutdown 2024-02-23T00:48:19.312 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is set 2024-02-23T00:48:19.320 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:19.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:17.897838+0000 osd.0 (osd.0) 175 : cluster [DBG] 18.8 scrub starts 2024-02-23T00:48:19.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:17.898717+0000 osd.0 (osd.0) 176 : cluster [DBG] 18.8 scrub ok 2024-02-23T00:48:19.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:18.709145+0000 mon.c (mon.2) 299 : audit [INF] from='client.? 172.21.15.47:0/865132933' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:48:19.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:18.709710+0000 mon.a (mon.0) 2577 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:48:20.171 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:48:20.174 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:20.175 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:48:20.276 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:48:20.306 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:20.431 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:48:20.433 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:18.900853+0000 osd.0 (osd.0) 177 : cluster [DBG] 62.e scrub starts 2024-02-23T00:48:20.433 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:18.902129+0000 osd.0 (osd.0) 178 : cluster [DBG] 62.e scrub ok 2024-02-23T00:48:20.433 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:19.197690+0000 osd.4 (osd.4) 167 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:48:20.434 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:19.197861+0000 osd.4 (osd.4) 168 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:48:20.434 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:19.288646+0000 mgr.y (mgr.14517) 387 : cluster [DBG] pgmap v612: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.8 KiB/s rd, 681 B/s wr, 2 op/s 2024-02-23T00:48:20.434 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:19.297468+0000 mon.a (mon.0) 2578 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:48:20.434 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:19.305461+0000 mon.a (mon.0) 2579 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:48:20.434 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:19.305523+0000 mon.a (mon.0) 2580 : cluster [DBG] osdmap e424: 8 total, 8 up, 7 in 2024-02-23T00:48:20.434 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:20.186003+0000 mon.a (mon.0) 2581 : cluster [DBG] osdmap e425: 8 total, 8 up, 7 in 2024-02-23T00:48:20.434 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:18.900853+0000 osd.0 (osd.0) 177 : cluster [DBG] 62.e scrub starts 2024-02-23T00:48:20.434 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:18.902129+0000 osd.0 (osd.0) 178 : cluster [DBG] 62.e scrub ok 2024-02-23T00:48:20.434 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:19.197690+0000 osd.4 (osd.4) 167 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:48:20.434 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:19.197861+0000 osd.4 (osd.4) 168 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:48:20.434 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:19.288646+0000 mgr.y (mgr.14517) 387 : cluster [DBG] pgmap v612: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.8 KiB/s rd, 681 B/s wr, 2 op/s 2024-02-23T00:48:20.435 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:19.297468+0000 mon.a (mon.0) 2578 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:48:20.435 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:19.305461+0000 mon.a (mon.0) 2579 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:48:20.435 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:19.305523+0000 mon.a (mon.0) 2580 : cluster [DBG] osdmap e424: 8 total, 8 up, 7 in 2024-02-23T00:48:20.435 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:20.186003+0000 mon.a (mon.0) 2581 : cluster [DBG] osdmap e425: 8 total, 8 up, 7 in 2024-02-23T00:48:20.435 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:20.436 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:48:20.536 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:48:20.567 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:18.900853+0000 osd.0 (osd.0) 177 : cluster [DBG] 62.e scrub starts 2024-02-23T00:48:20.567 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:18.902129+0000 osd.0 (osd.0) 178 : cluster [DBG] 62.e scrub ok 2024-02-23T00:48:20.567 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:19.197690+0000 osd.4 (osd.4) 167 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:48:20.567 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:19.197861+0000 osd.4 (osd.4) 168 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:48:20.567 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:19.288646+0000 mgr.y (mgr.14517) 387 : cluster [DBG] pgmap v612: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.8 KiB/s rd, 681 B/s wr, 2 op/s 2024-02-23T00:48:20.567 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:19.297468+0000 mon.a (mon.0) 2578 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:48:20.567 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:19.305461+0000 mon.a (mon.0) 2579 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:48:20.567 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:19.305523+0000 mon.a (mon.0) 2580 : cluster [DBG] osdmap e424: 8 total, 8 up, 7 in 2024-02-23T00:48:20.567 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:20.186003+0000 mon.a (mon.0) 2581 : cluster [DBG] osdmap e425: 8 total, 8 up, 7 in 2024-02-23T00:48:20.569 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:20.590 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:48:20.593 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:20.594 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:48:20.694 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:48:20.722 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:48:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:48:20] "GET /metrics HTTP/1.1" 200 188932 "" "Prometheus/2.33.4" 2024-02-23T00:48:20.725 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:20.985 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:48:21.275 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:48:21.278 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:21.278 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:48:21.379 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:48:21.410 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:21.556 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:48:21.556 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:48:21.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.507+0000 7fca29c82700 1 -- 172.21.15.47:0/4199242868 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fca240fcbb0 msgr2=0x7fca240fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:21.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.507+0000 7fca29c82700 1 --2- 172.21.15.47:0/4199242868 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fca240fcbb0 0x7fca240fcf90 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 rx=0x7fca140047b0 tx=0x7fca14009cb0).stop 2024-02-23T00:48:21.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.507+0000 7fca29c82700 1 -- 172.21.15.47:0/4199242868 shutdown_connections 2024-02-23T00:48:21.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.507+0000 7fca29c82700 1 --2- 172.21.15.47:0/4199242868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca240f7400 0x7fca24103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:21.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.507+0000 7fca29c82700 1 --2- 172.21.15.47:0/4199242868 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fca240fcbb0 0x7fca240fcf90 unknown :-1 s=CLOSED pgs=2668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:21.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.507+0000 7fca29c82700 1 --2- 172.21.15.47:0/4199242868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca240fd510 0x7fca240f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:21.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.507+0000 7fca29c82700 1 -- 172.21.15.47:0/4199242868 >> 172.21.15.47:0/4199242868 conn(0x7fca240f2820 msgr2=0x7fca240f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:48:21.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.507+0000 7fca29c82700 1 -- 172.21.15.47:0/4199242868 shutdown_connections 2024-02-23T00:48:21.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.507+0000 7fca29c82700 1 -- 172.21.15.47:0/4199242868 wait complete. 2024-02-23T00:48:21.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.507+0000 7fca29c82700 1 librados: starting msgr at 2024-02-23T00:48:21.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.507+0000 7fca29c82700 1 librados: starting objecter 2024-02-23T00:48:21.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.507+0000 7fca29c82700 1 Processor -- start 2024-02-23T00:48:21.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.508+0000 7fca29c82700 1 -- start start 2024-02-23T00:48:21.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.508+0000 7fca29c82700 1 librados: setting wanted keys 2024-02-23T00:48:21.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.508+0000 7fca29c82700 1 librados: calling monclient init 2024-02-23T00:48:21.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.508+0000 7fca29c82700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca240f7400 0x7fca240f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:21.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.508+0000 7fca29c82700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca240fcbb0 0x7fca240f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:21.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.508+0000 7fca29c82700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fca240fd510 0x7fca240f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:21.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.508+0000 7fca29c82700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca24116540 con 0x7fca240fcbb0 2024-02-23T00:48:21.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.508+0000 7fca29c82700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca241166b0 con 0x7fca240f7400 2024-02-23T00:48:21.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.508+0000 7fca29c82700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca24116870 con 0x7fca240fd510 2024-02-23T00:48:21.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.508+0000 7fca22ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca240fcbb0 0x7fca240f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:21.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.508+0000 7fca23fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fca240fd510 0x7fca240f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:21.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.508+0000 7fca22ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca240fcbb0 0x7fca240f8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40394/0 (socket says 172.21.15.47:40394) 2024-02-23T00:48:21.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.508+0000 7fca22ffd700 1 -- 172.21.15.47:0/2743933733 learned_addr learned my addr 172.21.15.47:0/2743933733 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:48:21.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.508+0000 7fca237fe700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca240f7400 0x7fca240f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:21.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.508+0000 7fca22ffd700 1 -- 172.21.15.47:0/2743933733 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fca240fd510 msgr2=0x7fca240f9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:21.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.508+0000 7fca22ffd700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fca240fd510 0x7fca240f9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:21.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.508+0000 7fca22ffd700 1 -- 172.21.15.47:0/2743933733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca240f7400 msgr2=0x7fca240f89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:21.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.508+0000 7fca22ffd700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca240f7400 0x7fca240f89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:21.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.508+0000 7fca22ffd700 1 -- 172.21.15.47:0/2743933733 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca0c005040 con 0x7fca240fcbb0 2024-02-23T00:48:21.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.508+0000 7fca22ffd700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca240fcbb0 0x7fca240f8ea0 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 rx=0x7fca0c009400 tx=0x7fca0c009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:21.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.508+0000 7fca20ff9700 10 client.?.objecter ms_handle_connect 0x7fca240fcbb0 2024-02-23T00:48:21.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.508+0000 7fca20ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:48:21.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.509+0000 7fca20ff9700 1 -- 172.21.15.47:0/2743933733 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c007a40 con 0x7fca240fcbb0 2024-02-23T00:48:21.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.509+0000 7fca20ff9700 1 -- 172.21.15.47:0/2743933733 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fca0c00f040 con 0x7fca240fcbb0 2024-02-23T00:48:21.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.509+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca14005040 con 0x7fca240fcbb0 2024-02-23T00:48:21.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.509+0000 7fca20ff9700 1 -- 172.21.15.47:0/2743933733 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c014900 con 0x7fca240fcbb0 2024-02-23T00:48:21.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.509+0000 7fca29c82700 10 client.31020.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:48:21.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.509+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca24116f00 con 0x7fca240fcbb0 2024-02-23T00:48:21.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.509+0000 7fca29c82700 1 librados: init done 2024-02-23T00:48:21.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.509+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fca2405fa60 con 0x7fca240fcbb0 2024-02-23T00:48:21.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.510+0000 7fca20ff9700 1 -- 172.21.15.47:0/2743933733 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fca0c014d30 con 0x7fca240fcbb0 2024-02-23T00:48:21.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.510+0000 7fca20ff9700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fca10062cc0 0x7fca10065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:21.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.511+0000 7fca20ff9700 1 -- 172.21.15.47:0/2743933733 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(425..425 src has 48..425) v4 ==== 9248+0+0 (secure 0 0 0) 0x7fca0c07d180 con 0x7fca240fcbb0 2024-02-23T00:48:21.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.511+0000 7fca20ff9700 10 client.31020.objecter ms_dispatch 0x7fca24001550 osd_map(425..425 src has 48..425) v4 2024-02-23T00:48:21.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.511+0000 7fca20ff9700 3 client.31020.objecter handle_osd_map got epochs [425,425] > 0 2024-02-23T00:48:21.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.511+0000 7fca20ff9700 3 client.31020.objecter handle_osd_map decoding full epoch 425 2024-02-23T00:48:21.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.511+0000 7fca237fe700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fca10062cc0 0x7fca10065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:21.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.511+0000 7fca20ff9700 20 client.31020.objecter dump_active .. 0 homeless 2024-02-23T00:48:21.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.511+0000 7fca20ff9700 1 -- 172.21.15.47:0/2743933733 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v425) v1 ==== 54+0+16 (secure 0 0 0) 0x7fca0c084070 con 0x7fca240fcbb0 2024-02-23T00:48:21.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.511+0000 7fca29c82700 10 client.31020.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:21.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.511+0000 7fca237fe700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fca10062cc0 0x7fca10065180 secure :-1 s=READY pgs=3066 cs=0 l=1 rev1=1 rx=0x7fca14009bc0 tx=0x7fca14009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:21.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.511+0000 7fca20ff9700 10 client.31020.objecter ms_handle_connect 0x7fca10062cc0 2024-02-23T00:48:21.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.511+0000 7fca29c82700 15 client.31020.objecter _session_command_op_assign -1 1 2024-02-23T00:48:21.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.511+0000 7fca29c82700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fca240ffed0 0x7fca241002d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:21.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.511+0000 7fca29c82700 20 client.31020.objecter _get_session s=0x7fca2405ee80 osd=0 3 2024-02-23T00:48:21.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.511+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca2405ee80 osd=0 3 2024-02-23T00:48:21.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.511+0000 7fca29c82700 20 client.31020.objecter _get_session s=0x7fca2405ee80 osd=0 3 2024-02-23T00:48:21.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.511+0000 7fca29c82700 15 client.31020.objecter _session_command_op_remove -1 1 2024-02-23T00:48:21.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.511+0000 7fca29c82700 20 client.31020.objecter get_session s=0x7fca2405ee80 osd=0 3 2024-02-23T00:48:21.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.511+0000 7fca29c82700 15 client.31020.objecter _session_command_op_assign 0 1 2024-02-23T00:48:21.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.511+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca2405ee80 osd=0 4 2024-02-23T00:48:21.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.511+0000 7fca29c82700 10 client.31020.objecter _send_command 1 2024-02-23T00:48:21.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.511+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca24105ea0 con 0x7fca240ffed0 2024-02-23T00:48:21.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.511+0000 7fca23fff700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fca240ffed0 0x7fca241002d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:21.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.511+0000 7fca20ff9700 10 client.31020.objecter ms_handle_connect 0x7fca240ffed0 2024-02-23T00:48:21.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.511+0000 7fca23fff700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fca240ffed0 0x7fca241002d0 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:21.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.512+0000 7fca20ff9700 1 -- 172.21.15.47:0/2743933733 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fca24105ea0 con 0x7fca240ffed0 2024-02-23T00:48:21.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.512+0000 7fca20ff9700 10 client.31020.objecter ms_dispatch 0x7fca24001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:48:21.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.512+0000 7fca20ff9700 10 client.31020.objecter _finish_command 1 = system:0 2024-02-23T00:48:21.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.512+0000 7fca20ff9700 20 client.31020.objecter put_session s=0x7fca2405ee80 osd=0 4 2024-02-23T00:48:21.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.512+0000 7fca20ff9700 15 client.31020.objecter _session_command_op_remove 0 1 2024-02-23T00:48:21.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.528+0000 7fca29c82700 10 client.31020.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:48:21.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.528+0000 7fca29c82700 15 client.31020.objecter _session_command_op_assign -1 2 2024-02-23T00:48:21.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.528+0000 7fca29c82700 20 client.31020.objecter _get_session s=0x7fca2405ee80 osd=0 3 2024-02-23T00:48:21.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.528+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca2405ee80 osd=0 3 2024-02-23T00:48:21.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.528+0000 7fca29c82700 20 client.31020.objecter _get_session s=0x7fca2405ee80 osd=0 3 2024-02-23T00:48:21.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.528+0000 7fca29c82700 15 client.31020.objecter _session_command_op_remove -1 2 2024-02-23T00:48:21.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.528+0000 7fca29c82700 20 client.31020.objecter get_session s=0x7fca2405ee80 osd=0 3 2024-02-23T00:48:21.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.528+0000 7fca29c82700 15 client.31020.objecter _session_command_op_assign 0 2 2024-02-23T00:48:21.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.528+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca2405ee80 osd=0 4 2024-02-23T00:48:21.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.528+0000 7fca29c82700 10 client.31020.objecter _send_command 2 2024-02-23T00:48:21.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.528+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fca240273c0 con 0x7fca240ffed0 2024-02-23T00:48:21.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.529+0000 7fca20ff9700 1 -- 172.21.15.47:0/2743933733 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fca240273c0 con 0x7fca240ffed0 2024-02-23T00:48:21.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.530+0000 7fca20ff9700 10 client.31020.objecter ms_dispatch 0x7fca24001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:48:21.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.530+0000 7fca20ff9700 10 client.31020.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:48:21.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.530+0000 7fca20ff9700 20 client.31020.objecter put_session s=0x7fca2405ee80 osd=0 4 2024-02-23T00:48:21.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.530+0000 7fca20ff9700 15 client.31020.objecter _session_command_op_remove 0 2 2024-02-23T00:48:21.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.530+0000 7fca29c82700 10 client.31020.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:21.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.530+0000 7fca29c82700 15 client.31020.objecter _session_command_op_assign -1 3 2024-02-23T00:48:21.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.530+0000 7fca29c82700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fca240598f0 0x7fca24059cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:21.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.530+0000 7fca29c82700 20 client.31020.objecter _get_session s=0x7fca24028710 osd=1 3 2024-02-23T00:48:21.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.530+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca24028710 osd=1 3 2024-02-23T00:48:21.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.530+0000 7fca29c82700 20 client.31020.objecter _get_session s=0x7fca24028710 osd=1 3 2024-02-23T00:48:21.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.530+0000 7fca29c82700 15 client.31020.objecter _session_command_op_remove -1 3 2024-02-23T00:48:21.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.530+0000 7fca29c82700 20 client.31020.objecter get_session s=0x7fca24028710 osd=1 3 2024-02-23T00:48:21.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.530+0000 7fca29c82700 15 client.31020.objecter _session_command_op_assign 1 3 2024-02-23T00:48:21.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.530+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca24028710 osd=1 4 2024-02-23T00:48:21.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.530+0000 7fca29c82700 10 client.31020.objecter _send_command 3 2024-02-23T00:48:21.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.530+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca2411d810 con 0x7fca240598f0 2024-02-23T00:48:21.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.530+0000 7fca237fe700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fca240598f0 0x7fca24059cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:21.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.530+0000 7fca20ff9700 10 client.31020.objecter ms_handle_connect 0x7fca240598f0 2024-02-23T00:48:21.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.530+0000 7fca237fe700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fca240598f0 0x7fca24059cf0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:21.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.531+0000 7fca20ff9700 1 -- 172.21.15.47:0/2743933733 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fca2411d810 con 0x7fca240598f0 2024-02-23T00:48:21.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.531+0000 7fca20ff9700 10 client.31020.objecter ms_dispatch 0x7fca24001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:48:21.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.531+0000 7fca20ff9700 10 client.31020.objecter _finish_command 3 = system:0 2024-02-23T00:48:21.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.531+0000 7fca20ff9700 20 client.31020.objecter put_session s=0x7fca24028710 osd=1 4 2024-02-23T00:48:21.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.531+0000 7fca20ff9700 15 client.31020.objecter _session_command_op_remove 1 3 2024-02-23T00:48:21.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.546+0000 7fca29c82700 10 client.31020.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:48:21.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.546+0000 7fca29c82700 15 client.31020.objecter _session_command_op_assign -1 4 2024-02-23T00:48:21.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.546+0000 7fca29c82700 20 client.31020.objecter _get_session s=0x7fca24028710 osd=1 3 2024-02-23T00:48:21.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.546+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca24028710 osd=1 3 2024-02-23T00:48:21.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.546+0000 7fca29c82700 20 client.31020.objecter _get_session s=0x7fca24028710 osd=1 3 2024-02-23T00:48:21.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.546+0000 7fca29c82700 15 client.31020.objecter _session_command_op_remove -1 4 2024-02-23T00:48:21.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.546+0000 7fca29c82700 20 client.31020.objecter get_session s=0x7fca24028710 osd=1 3 2024-02-23T00:48:21.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.546+0000 7fca29c82700 15 client.31020.objecter _session_command_op_assign 1 4 2024-02-23T00:48:21.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.546+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca24028710 osd=1 4 2024-02-23T00:48:21.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.546+0000 7fca29c82700 10 client.31020.objecter _send_command 4 2024-02-23T00:48:21.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.546+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fca24059360 con 0x7fca240598f0 2024-02-23T00:48:21.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.547+0000 7fca20ff9700 1 -- 172.21.15.47:0/2743933733 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fca24059360 con 0x7fca240598f0 2024-02-23T00:48:21.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.547+0000 7fca20ff9700 10 client.31020.objecter ms_dispatch 0x7fca24001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:48:21.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.547+0000 7fca20ff9700 10 client.31020.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:48:21.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.547+0000 7fca20ff9700 20 client.31020.objecter put_session s=0x7fca24028710 osd=1 4 2024-02-23T00:48:21.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.547+0000 7fca20ff9700 15 client.31020.objecter _session_command_op_remove 1 4 2024-02-23T00:48:21.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.547+0000 7fca29c82700 10 client.31020.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:21.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.547+0000 7fca29c82700 15 client.31020.objecter _session_command_op_assign -1 5 2024-02-23T00:48:21.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.547+0000 7fca29c82700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fca24028270 0x7fca2411c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:21.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.547+0000 7fca29c82700 20 client.31020.objecter _get_session s=0x7fca24059770 osd=2 3 2024-02-23T00:48:21.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.547+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca24059770 osd=2 3 2024-02-23T00:48:21.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.547+0000 7fca29c82700 20 client.31020.objecter _get_session s=0x7fca24059770 osd=2 3 2024-02-23T00:48:21.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.547+0000 7fca29c82700 15 client.31020.objecter _session_command_op_remove -1 5 2024-02-23T00:48:21.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.547+0000 7fca29c82700 20 client.31020.objecter get_session s=0x7fca24059770 osd=2 3 2024-02-23T00:48:21.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.547+0000 7fca29c82700 15 client.31020.objecter _session_command_op_assign 2 5 2024-02-23T00:48:21.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.547+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca24059770 osd=2 4 2024-02-23T00:48:21.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.547+0000 7fca29c82700 10 client.31020.objecter _send_command 5 2024-02-23T00:48:21.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.547+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca2411c7b0 con 0x7fca24028270 2024-02-23T00:48:21.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.547+0000 7fca22ffd700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fca24028270 0x7fca2411c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:21.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.548+0000 7fca20ff9700 10 client.31020.objecter ms_handle_connect 0x7fca24028270 2024-02-23T00:48:21.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.548+0000 7fca22ffd700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fca24028270 0x7fca2411c230 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:21.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.548+0000 7fca20ff9700 1 -- 172.21.15.47:0/2743933733 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fca2411c7b0 con 0x7fca24028270 2024-02-23T00:48:21.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.548+0000 7fca20ff9700 10 client.31020.objecter ms_dispatch 0x7fca24001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:48:21.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.548+0000 7fca20ff9700 10 client.31020.objecter _finish_command 5 = system:0 2024-02-23T00:48:21.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.548+0000 7fca20ff9700 20 client.31020.objecter put_session s=0x7fca24059770 osd=2 4 2024-02-23T00:48:21.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.548+0000 7fca20ff9700 15 client.31020.objecter _session_command_op_remove 2 5 2024-02-23T00:48:21.574 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:21.575 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:48:21.576 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:48:21.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.564+0000 7fca29c82700 10 client.31020.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:48:21.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.564+0000 7fca29c82700 15 client.31020.objecter _session_command_op_assign -1 6 2024-02-23T00:48:21.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.564+0000 7fca29c82700 20 client.31020.objecter _get_session s=0x7fca24059770 osd=2 3 2024-02-23T00:48:21.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.564+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca24059770 osd=2 3 2024-02-23T00:48:21.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.564+0000 7fca29c82700 20 client.31020.objecter _get_session s=0x7fca24059770 osd=2 3 2024-02-23T00:48:21.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.564+0000 7fca29c82700 15 client.31020.objecter _session_command_op_remove -1 6 2024-02-23T00:48:21.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.564+0000 7fca29c82700 20 client.31020.objecter get_session s=0x7fca24059770 osd=2 3 2024-02-23T00:48:21.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.564+0000 7fca29c82700 15 client.31020.objecter _session_command_op_assign 2 6 2024-02-23T00:48:21.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.564+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca24059770 osd=2 4 2024-02-23T00:48:21.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.564+0000 7fca29c82700 10 client.31020.objecter _send_command 6 2024-02-23T00:48:21.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.564+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fca24027890 con 0x7fca24028270 2024-02-23T00:48:21.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.565+0000 7fca20ff9700 1 -- 172.21.15.47:0/2743933733 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fca24027890 con 0x7fca24028270 2024-02-23T00:48:21.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.565+0000 7fca20ff9700 10 client.31020.objecter ms_dispatch 0x7fca24001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:48:21.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.565+0000 7fca20ff9700 10 client.31020.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:48:21.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.565+0000 7fca20ff9700 20 client.31020.objecter put_session s=0x7fca24059770 osd=2 4 2024-02-23T00:48:21.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.565+0000 7fca20ff9700 15 client.31020.objecter _session_command_op_remove 2 6 2024-02-23T00:48:21.578 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:48:21.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.566+0000 7fca29c82700 10 client.31020.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:21.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.566+0000 7fca29c82700 15 client.31020.objecter _session_command_op_assign -1 7 2024-02-23T00:48:21.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.566+0000 7fca29c82700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fca2411f990 0x7fca24121d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:21.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.566+0000 7fca29c82700 20 client.31020.objecter _get_session s=0x7fca2411d0f0 osd=3 3 2024-02-23T00:48:21.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.566+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca2411d0f0 osd=3 3 2024-02-23T00:48:21.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.566+0000 7fca29c82700 20 client.31020.objecter _get_session s=0x7fca2411d0f0 osd=3 3 2024-02-23T00:48:21.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.566+0000 7fca29c82700 15 client.31020.objecter _session_command_op_remove -1 7 2024-02-23T00:48:21.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.566+0000 7fca29c82700 20 client.31020.objecter get_session s=0x7fca2411d0f0 osd=3 3 2024-02-23T00:48:21.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.566+0000 7fca29c82700 15 client.31020.objecter _session_command_op_assign 3 7 2024-02-23T00:48:21.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.566+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca2411d0f0 osd=3 4 2024-02-23T00:48:21.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.566+0000 7fca29c82700 10 client.31020.objecter _send_command 7 2024-02-23T00:48:21.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.566+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca24124b00 con 0x7fca2411f990 2024-02-23T00:48:21.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.566+0000 7fca23fff700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fca2411f990 0x7fca24121d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:21.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.566+0000 7fca20ff9700 10 client.31020.objecter ms_handle_connect 0x7fca2411f990 2024-02-23T00:48:21.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.566+0000 7fca23fff700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fca2411f990 0x7fca24121d80 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:21.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.567+0000 7fca20ff9700 1 -- 172.21.15.47:0/2743933733 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fca24124b00 con 0x7fca2411f990 2024-02-23T00:48:21.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.567+0000 7fca20ff9700 10 client.31020.objecter ms_dispatch 0x7fca24001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:48:21.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.567+0000 7fca20ff9700 10 client.31020.objecter _finish_command 7 = system:0 2024-02-23T00:48:21.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.567+0000 7fca20ff9700 20 client.31020.objecter put_session s=0x7fca2411d0f0 osd=3 4 2024-02-23T00:48:21.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.567+0000 7fca20ff9700 15 client.31020.objecter _session_command_op_remove 3 7 2024-02-23T00:48:21.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.582+0000 7fca29c82700 10 client.31020.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:48:21.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.582+0000 7fca29c82700 15 client.31020.objecter _session_command_op_assign -1 8 2024-02-23T00:48:21.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.582+0000 7fca29c82700 20 client.31020.objecter _get_session s=0x7fca2411d0f0 osd=3 3 2024-02-23T00:48:21.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.582+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca2411d0f0 osd=3 3 2024-02-23T00:48:21.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.582+0000 7fca29c82700 20 client.31020.objecter _get_session s=0x7fca2411d0f0 osd=3 3 2024-02-23T00:48:21.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.582+0000 7fca29c82700 15 client.31020.objecter _session_command_op_remove -1 8 2024-02-23T00:48:21.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.582+0000 7fca29c82700 20 client.31020.objecter get_session s=0x7fca2411d0f0 osd=3 3 2024-02-23T00:48:21.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.582+0000 7fca29c82700 15 client.31020.objecter _session_command_op_assign 3 8 2024-02-23T00:48:21.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.582+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca2411d0f0 osd=3 4 2024-02-23T00:48:21.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.582+0000 7fca29c82700 10 client.31020.objecter _send_command 8 2024-02-23T00:48:21.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.582+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fca2411cb60 con 0x7fca2411f990 2024-02-23T00:48:21.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.583+0000 7fca20ff9700 1 -- 172.21.15.47:0/2743933733 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fca2411cb60 con 0x7fca2411f990 2024-02-23T00:48:21.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.583+0000 7fca20ff9700 10 client.31020.objecter ms_dispatch 0x7fca24001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:48:21.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.583+0000 7fca20ff9700 10 client.31020.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:48:21.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.583+0000 7fca20ff9700 20 client.31020.objecter put_session s=0x7fca2411d0f0 osd=3 4 2024-02-23T00:48:21.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.583+0000 7fca20ff9700 15 client.31020.objecter _session_command_op_remove 3 8 2024-02-23T00:48:21.587 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:48:21.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.583+0000 7fca29c82700 10 client.31020.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:21.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.583+0000 7fca29c82700 15 client.31020.objecter _session_command_op_assign -1 9 2024-02-23T00:48:21.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.583+0000 7fca29c82700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fca2411bd90 0x7fca241222c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:21.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.583+0000 7fca29c82700 20 client.31020.objecter _get_session s=0x7fca2411cd00 osd=4 3 2024-02-23T00:48:21.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.583+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca2411cd00 osd=4 3 2024-02-23T00:48:21.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.583+0000 7fca29c82700 20 client.31020.objecter _get_session s=0x7fca2411cd00 osd=4 3 2024-02-23T00:48:21.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.583+0000 7fca29c82700 15 client.31020.objecter _session_command_op_remove -1 9 2024-02-23T00:48:21.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.583+0000 7fca29c82700 20 client.31020.objecter get_session s=0x7fca2411cd00 osd=4 3 2024-02-23T00:48:21.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.583+0000 7fca29c82700 15 client.31020.objecter _session_command_op_assign 4 9 2024-02-23T00:48:21.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.583+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca2411cd00 osd=4 4 2024-02-23T00:48:21.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.583+0000 7fca29c82700 10 client.31020.objecter _send_command 9 2024-02-23T00:48:21.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.583+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca2411cf20 con 0x7fca2411bd90 2024-02-23T00:48:21.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.584+0000 7fca237fe700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fca2411bd90 0x7fca241222c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:21.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.584+0000 7fca237fe700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fca2411bd90 0x7fca241222c0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:21.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.584+0000 7fca20ff9700 10 client.31020.objecter ms_handle_connect 0x7fca2411bd90 2024-02-23T00:48:21.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.585+0000 7fca20ff9700 1 -- 172.21.15.47:0/2743933733 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fca2411cf20 con 0x7fca2411bd90 2024-02-23T00:48:21.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.585+0000 7fca20ff9700 10 client.31020.objecter ms_dispatch 0x7fca24001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:48:21.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.585+0000 7fca20ff9700 10 client.31020.objecter _finish_command 9 = system:0 2024-02-23T00:48:21.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.585+0000 7fca20ff9700 20 client.31020.objecter put_session s=0x7fca2411cd00 osd=4 4 2024-02-23T00:48:21.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.585+0000 7fca20ff9700 15 client.31020.objecter _session_command_op_remove 4 9 2024-02-23T00:48:21.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.599+0000 7fca29c82700 10 client.31020.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:48:21.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.599+0000 7fca29c82700 15 client.31020.objecter _session_command_op_assign -1 10 2024-02-23T00:48:21.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.599+0000 7fca29c82700 20 client.31020.objecter _get_session s=0x7fca2411cd00 osd=4 3 2024-02-23T00:48:21.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.599+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca2411cd00 osd=4 3 2024-02-23T00:48:21.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.599+0000 7fca29c82700 20 client.31020.objecter _get_session s=0x7fca2411cd00 osd=4 3 2024-02-23T00:48:21.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.599+0000 7fca29c82700 15 client.31020.objecter _session_command_op_remove -1 10 2024-02-23T00:48:21.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.599+0000 7fca29c82700 20 client.31020.objecter get_session s=0x7fca2411cd00 osd=4 3 2024-02-23T00:48:21.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.599+0000 7fca29c82700 15 client.31020.objecter _session_command_op_assign 4 10 2024-02-23T00:48:21.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.599+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca2411cd00 osd=4 4 2024-02-23T00:48:21.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.599+0000 7fca29c82700 10 client.31020.objecter _send_command 10 2024-02-23T00:48:21.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.599+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fca2411cf20 con 0x7fca2411bd90 2024-02-23T00:48:21.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.601+0000 7fca20ff9700 1 -- 172.21.15.47:0/2743933733 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fca2411cf20 con 0x7fca2411bd90 2024-02-23T00:48:21.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.601+0000 7fca20ff9700 10 client.31020.objecter ms_dispatch 0x7fca24001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:48:21.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.601+0000 7fca20ff9700 10 client.31020.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:48:21.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.601+0000 7fca20ff9700 20 client.31020.objecter put_session s=0x7fca2411cd00 osd=4 4 2024-02-23T00:48:21.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.601+0000 7fca20ff9700 15 client.31020.objecter _session_command_op_remove 4 10 2024-02-23T00:48:21.604 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:48:21.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.601+0000 7fca29c82700 10 client.31020.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:21.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.602+0000 7fca29c82700 15 client.31020.objecter _session_command_op_assign -1 11 2024-02-23T00:48:21.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.602+0000 7fca29c82700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fca24122d60 0x7fca241231a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:21.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.602+0000 7fca29c82700 20 client.31020.objecter _get_session s=0x7fca2411cf20 osd=5 3 2024-02-23T00:48:21.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.602+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca2411cf20 osd=5 3 2024-02-23T00:48:21.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.602+0000 7fca29c82700 20 client.31020.objecter _get_session s=0x7fca2411cf20 osd=5 3 2024-02-23T00:48:21.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.602+0000 7fca29c82700 15 client.31020.objecter _session_command_op_remove -1 11 2024-02-23T00:48:21.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.602+0000 7fca29c82700 20 client.31020.objecter get_session s=0x7fca2411cf20 osd=5 3 2024-02-23T00:48:21.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.602+0000 7fca29c82700 15 client.31020.objecter _session_command_op_assign 5 11 2024-02-23T00:48:21.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.602+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca2411cf20 osd=5 4 2024-02-23T00:48:21.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.602+0000 7fca29c82700 10 client.31020.objecter _send_command 11 2024-02-23T00:48:21.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.602+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca2412b420 con 0x7fca24122d60 2024-02-23T00:48:21.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.602+0000 7fca22ffd700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fca24122d60 0x7fca241231a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:21.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.602+0000 7fca22ffd700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fca24122d60 0x7fca241231a0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:21.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.602+0000 7fca20ff9700 10 client.31020.objecter ms_handle_connect 0x7fca24122d60 2024-02-23T00:48:21.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.603+0000 7fca20ff9700 1 -- 172.21.15.47:0/2743933733 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fca2412b420 con 0x7fca24122d60 2024-02-23T00:48:21.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.603+0000 7fca20ff9700 10 client.31020.objecter ms_dispatch 0x7fca24001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:48:21.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.603+0000 7fca20ff9700 10 client.31020.objecter _finish_command 11 = system:0 2024-02-23T00:48:21.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.603+0000 7fca20ff9700 20 client.31020.objecter put_session s=0x7fca2411cf20 osd=5 4 2024-02-23T00:48:21.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.603+0000 7fca20ff9700 15 client.31020.objecter _session_command_op_remove 5 11 2024-02-23T00:48:21.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.617+0000 7fca29c82700 10 client.31020.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:48:21.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.617+0000 7fca29c82700 15 client.31020.objecter _session_command_op_assign -1 12 2024-02-23T00:48:21.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.617+0000 7fca29c82700 20 client.31020.objecter _get_session s=0x7fca2411cf20 osd=5 3 2024-02-23T00:48:21.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.617+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca2411cf20 osd=5 3 2024-02-23T00:48:21.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.617+0000 7fca29c82700 20 client.31020.objecter _get_session s=0x7fca2411cf20 osd=5 3 2024-02-23T00:48:21.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.617+0000 7fca29c82700 15 client.31020.objecter _session_command_op_remove -1 12 2024-02-23T00:48:21.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.617+0000 7fca29c82700 20 client.31020.objecter get_session s=0x7fca2411cf20 osd=5 3 2024-02-23T00:48:21.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.617+0000 7fca29c82700 15 client.31020.objecter _session_command_op_assign 5 12 2024-02-23T00:48:21.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.617+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca2411cf20 osd=5 4 2024-02-23T00:48:21.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.617+0000 7fca29c82700 10 client.31020.objecter _send_command 12 2024-02-23T00:48:21.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.617+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fca24128cc0 con 0x7fca24122d60 2024-02-23T00:48:21.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.620+0000 7fca20ff9700 1 -- 172.21.15.47:0/2743933733 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fca24128cc0 con 0x7fca24122d60 2024-02-23T00:48:21.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.620+0000 7fca20ff9700 10 client.31020.objecter ms_dispatch 0x7fca24001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:48:21.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.620+0000 7fca20ff9700 10 client.31020.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:48:21.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.620+0000 7fca20ff9700 20 client.31020.objecter put_session s=0x7fca2411cf20 osd=5 4 2024-02-23T00:48:21.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.620+0000 7fca20ff9700 15 client.31020.objecter _session_command_op_remove 5 12 2024-02-23T00:48:21.623 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:48:21.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.620+0000 7fca29c82700 10 client.31020.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:21.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.620+0000 7fca29c82700 15 client.31020.objecter _session_command_op_assign -1 13 2024-02-23T00:48:21.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.620+0000 7fca29c82700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fca24129390 0x7fca241297d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:21.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.620+0000 7fca29c82700 20 client.31020.objecter _get_session s=0x7fca24128cc0 osd=6 3 2024-02-23T00:48:21.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.620+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca24128cc0 osd=6 3 2024-02-23T00:48:21.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.620+0000 7fca29c82700 20 client.31020.objecter _get_session s=0x7fca24128cc0 osd=6 3 2024-02-23T00:48:21.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.620+0000 7fca29c82700 15 client.31020.objecter _session_command_op_remove -1 13 2024-02-23T00:48:21.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.620+0000 7fca29c82700 20 client.31020.objecter get_session s=0x7fca24128cc0 osd=6 3 2024-02-23T00:48:21.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.620+0000 7fca29c82700 15 client.31020.objecter _session_command_op_assign 6 13 2024-02-23T00:48:21.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.620+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca24128cc0 osd=6 4 2024-02-23T00:48:21.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.620+0000 7fca29c82700 10 client.31020.objecter _send_command 13 2024-02-23T00:48:21.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.620+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca2412e770 con 0x7fca24129390 2024-02-23T00:48:21.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.620+0000 7fca23fff700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fca24129390 0x7fca241297d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:21.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.621+0000 7fca23fff700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fca24129390 0x7fca241297d0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:21.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.621+0000 7fca20ff9700 10 client.31020.objecter ms_handle_connect 0x7fca24129390 2024-02-23T00:48:21.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.621+0000 7fca20ff9700 1 -- 172.21.15.47:0/2743933733 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fca2412e770 con 0x7fca24129390 2024-02-23T00:48:21.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.621+0000 7fca20ff9700 10 client.31020.objecter ms_dispatch 0x7fca24001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:48:21.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.621+0000 7fca20ff9700 10 client.31020.objecter _finish_command 13 = system:0 2024-02-23T00:48:21.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.621+0000 7fca20ff9700 20 client.31020.objecter put_session s=0x7fca24128cc0 osd=6 4 2024-02-23T00:48:21.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.621+0000 7fca20ff9700 15 client.31020.objecter _session_command_op_remove 6 13 2024-02-23T00:48:21.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.636+0000 7fca29c82700 10 client.31020.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:48:21.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.636+0000 7fca29c82700 15 client.31020.objecter _session_command_op_assign -1 14 2024-02-23T00:48:21.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.636+0000 7fca29c82700 20 client.31020.objecter _get_session s=0x7fca24128cc0 osd=6 3 2024-02-23T00:48:21.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.636+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca24128cc0 osd=6 3 2024-02-23T00:48:21.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.636+0000 7fca29c82700 20 client.31020.objecter _get_session s=0x7fca24128cc0 osd=6 3 2024-02-23T00:48:21.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.636+0000 7fca29c82700 15 client.31020.objecter _session_command_op_remove -1 14 2024-02-23T00:48:21.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.636+0000 7fca29c82700 20 client.31020.objecter get_session s=0x7fca24128cc0 osd=6 3 2024-02-23T00:48:21.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.636+0000 7fca29c82700 15 client.31020.objecter _session_command_op_assign 6 14 2024-02-23T00:48:21.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.636+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca24128cc0 osd=6 4 2024-02-23T00:48:21.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.636+0000 7fca29c82700 10 client.31020.objecter _send_command 14 2024-02-23T00:48:21.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.636+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fca24129dd0 con 0x7fca24129390 2024-02-23T00:48:21.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.638+0000 7fca20ff9700 1 -- 172.21.15.47:0/2743933733 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fca24129dd0 con 0x7fca24129390 2024-02-23T00:48:21.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.638+0000 7fca20ff9700 10 client.31020.objecter ms_dispatch 0x7fca24001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:48:21.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.638+0000 7fca20ff9700 10 client.31020.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:48:21.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.638+0000 7fca20ff9700 20 client.31020.objecter put_session s=0x7fca24128cc0 osd=6 4 2024-02-23T00:48:21.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.638+0000 7fca20ff9700 15 client.31020.objecter _session_command_op_remove 6 14 2024-02-23T00:48:21.641 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:48:21.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.638+0000 7fca29c82700 10 client.31020.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:21.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.638+0000 7fca29c82700 15 client.31020.objecter _session_command_op_assign -1 15 2024-02-23T00:48:21.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.638+0000 7fca29c82700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fca2412a450 0x7fca2412a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:21.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.638+0000 7fca29c82700 20 client.31020.objecter _get_session s=0x7fca24129dd0 osd=7 3 2024-02-23T00:48:21.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.638+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca24129dd0 osd=7 3 2024-02-23T00:48:21.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.638+0000 7fca29c82700 20 client.31020.objecter _get_session s=0x7fca24129dd0 osd=7 3 2024-02-23T00:48:21.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.638+0000 7fca29c82700 15 client.31020.objecter _session_command_op_remove -1 15 2024-02-23T00:48:21.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.638+0000 7fca29c82700 20 client.31020.objecter get_session s=0x7fca24129dd0 osd=7 3 2024-02-23T00:48:21.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.638+0000 7fca29c82700 15 client.31020.objecter _session_command_op_assign 7 15 2024-02-23T00:48:21.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.638+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca24129dd0 osd=7 4 2024-02-23T00:48:21.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.638+0000 7fca29c82700 10 client.31020.objecter _send_command 15 2024-02-23T00:48:21.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.638+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca2412af10 con 0x7fca2412a450 2024-02-23T00:48:21.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.639+0000 7fca237fe700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fca2412a450 0x7fca2412a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:21.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.639+0000 7fca237fe700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fca2412a450 0x7fca2412a8d0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:21.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.639+0000 7fca20ff9700 10 client.31020.objecter ms_handle_connect 0x7fca2412a450 2024-02-23T00:48:21.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.640+0000 7fca20ff9700 1 -- 172.21.15.47:0/2743933733 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fca2412af10 con 0x7fca2412a450 2024-02-23T00:48:21.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.640+0000 7fca20ff9700 10 client.31020.objecter ms_dispatch 0x7fca24001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:48:21.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.640+0000 7fca20ff9700 10 client.31020.objecter _finish_command 15 = system:0 2024-02-23T00:48:21.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.640+0000 7fca20ff9700 20 client.31020.objecter put_session s=0x7fca24129dd0 osd=7 4 2024-02-23T00:48:21.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.640+0000 7fca20ff9700 15 client.31020.objecter _session_command_op_remove 7 15 2024-02-23T00:48:21.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.655+0000 7fca29c82700 10 client.31020.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:48:21.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.655+0000 7fca29c82700 15 client.31020.objecter _session_command_op_assign -1 16 2024-02-23T00:48:21.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.655+0000 7fca29c82700 20 client.31020.objecter _get_session s=0x7fca24129dd0 osd=7 3 2024-02-23T00:48:21.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.655+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca24129dd0 osd=7 3 2024-02-23T00:48:21.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.655+0000 7fca29c82700 20 client.31020.objecter _get_session s=0x7fca24129dd0 osd=7 3 2024-02-23T00:48:21.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.655+0000 7fca29c82700 15 client.31020.objecter _session_command_op_remove -1 16 2024-02-23T00:48:21.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.655+0000 7fca29c82700 20 client.31020.objecter get_session s=0x7fca24129dd0 osd=7 3 2024-02-23T00:48:21.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.655+0000 7fca29c82700 15 client.31020.objecter _session_command_op_assign 7 16 2024-02-23T00:48:21.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.655+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca24129dd0 osd=7 4 2024-02-23T00:48:21.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.655+0000 7fca29c82700 10 client.31020.objecter _send_command 16 2024-02-23T00:48:21.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.655+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fca2412af10 con 0x7fca2412a450 2024-02-23T00:48:21.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca20ff9700 1 -- 172.21.15.47:0/2743933733 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fca2412af10 con 0x7fca2412a450 2024-02-23T00:48:21.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca20ff9700 10 client.31020.objecter ms_dispatch 0x7fca24001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:48:21.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca20ff9700 10 client.31020.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:48:21.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca20ff9700 20 client.31020.objecter put_session s=0x7fca24129dd0 osd=7 4 2024-02-23T00:48:21.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca20ff9700 15 client.31020.objecter _session_command_op_remove 7 16 2024-02-23T00:48:21.660 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:48:21.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 10 librados: watch_flush enter 2024-02-23T00:48:21.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 10 librados: watch_flush exit 2024-02-23T00:48:21.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 10 client.31020.objecter close_session for osd.0 2024-02-23T00:48:21.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fca240ffed0 msgr2=0x7fca241002d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:21.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fca240ffed0 0x7fca241002d0 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:21.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca2405ee80 osd=0 1 2024-02-23T00:48:21.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 10 client.31020.objecter close_session for osd.1 2024-02-23T00:48:21.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fca240598f0 msgr2=0x7fca24059cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:21.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fca240598f0 0x7fca24059cf0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:21.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca24028710 osd=1 1 2024-02-23T00:48:21.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 10 client.31020.objecter close_session for osd.2 2024-02-23T00:48:21.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fca24028270 msgr2=0x7fca2411c230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:21.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fca24028270 0x7fca2411c230 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:21.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca24059770 osd=2 1 2024-02-23T00:48:21.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 10 client.31020.objecter close_session for osd.3 2024-02-23T00:48:21.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fca2411f990 msgr2=0x7fca24121d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:21.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fca2411f990 0x7fca24121d80 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:21.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca2411d0f0 osd=3 1 2024-02-23T00:48:21.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 10 client.31020.objecter close_session for osd.4 2024-02-23T00:48:21.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fca2411bd90 msgr2=0x7fca241222c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:21.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fca2411bd90 0x7fca241222c0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:21.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca2411cd00 osd=4 1 2024-02-23T00:48:21.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 10 client.31020.objecter close_session for osd.5 2024-02-23T00:48:21.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fca24122d60 msgr2=0x7fca241231a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:21.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fca24122d60 0x7fca241231a0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:21.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca2411cf20 osd=5 1 2024-02-23T00:48:21.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 10 client.31020.objecter close_session for osd.6 2024-02-23T00:48:21.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fca24129390 msgr2=0x7fca241297d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:21.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fca24129390 0x7fca241297d0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:21.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca24128cc0 osd=6 1 2024-02-23T00:48:21.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 10 client.31020.objecter close_session for osd.7 2024-02-23T00:48:21.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fca2412a450 msgr2=0x7fca2412a8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:21.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fca2412a450 0x7fca2412a8d0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:21.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 20 client.31020.objecter put_session s=0x7fca24129dd0 osd=7 1 2024-02-23T00:48:21.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 20 client.31020.objecter shutdown clearing up homeless session... 2024-02-23T00:48:21.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 10 client.31020.objecter successfully canceled tick 2024-02-23T00:48:21.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca23fff700 1 -- 172.21.15.47:0/2743933733 reap_dead start 2024-02-23T00:48:21.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca23fff700 1 -- 172.21.15.47:0/2743933733 reap_dead start 2024-02-23T00:48:21.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fca10062cc0 msgr2=0x7fca10065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:21.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fca10062cc0 0x7fca10065180 secure :-1 s=READY pgs=3066 cs=0 l=1 rev1=1 rx=0x7fca14009bc0 tx=0x7fca14009cb0).stop 2024-02-23T00:48:21.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca23fff700 1 -- 172.21.15.47:0/2743933733 reap_dead start 2024-02-23T00:48:21.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca23fff700 1 -- 172.21.15.47:0/2743933733 reap_dead start 2024-02-23T00:48:21.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca240fcbb0 msgr2=0x7fca240f8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:21.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca23fff700 1 -- 172.21.15.47:0/2743933733 reap_dead start 2024-02-23T00:48:21.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca240fcbb0 0x7fca240f8ea0 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 rx=0x7fca0c009400 tx=0x7fca0c009cb0).stop 2024-02-23T00:48:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 shutdown_connections 2024-02-23T00:48:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 1 --2- 172.21.15.47:0/2743933733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca240fcbb0 0x7fca240f8ea0 unknown :-1 s=CLOSED pgs=2532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.657+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 >> 172.21.15.47:0/2743933733 conn(0x7fca240f2820 msgr2=0x7fca24060eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:48:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.658+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 shutdown_connections 2024-02-23T00:48:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.658+0000 7fca29c82700 1 -- 172.21.15.47:0/2743933733 wait complete. 2024-02-23T00:48:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:21.658+0000 7fca29c82700 1 librados: shutdown 2024-02-23T00:48:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:48:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:48:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:48:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:48:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:48:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:48:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:48:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:48:21.677 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:48:21.709 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:21.730 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:48:21.732 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:21.733 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:48:21.833 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:48:21.864 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:21.937 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:48:21.939 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:21.940 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:48:22.041 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:48:22.070 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:22.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:21.289023+0000 mgr.y (mgr.14517) 388 : cluster [DBG] pgmap v615: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.8 KiB/s rd, 670 B/s wr, 2 op/s 2024-02-23T00:48:22.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:21.511855+0000 mon.a (mon.0) 2582 : audit [DBG] from='client.? 172.21.15.47:0/2743933733' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:48:22.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:21.289023+0000 mgr.y (mgr.14517) 388 : cluster [DBG] pgmap v615: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.8 KiB/s rd, 670 B/s wr, 2 op/s 2024-02-23T00:48:22.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:21.511855+0000 mon.a (mon.0) 2582 : audit [DBG] from='client.? 172.21.15.47:0/2743933733' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:48:22.594 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:21.289023+0000 mgr.y (mgr.14517) 388 : cluster [DBG] pgmap v615: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.8 KiB/s rd, 670 B/s wr, 2 op/s 2024-02-23T00:48:22.594 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:21.511855+0000 mon.a (mon.0) 2582 : audit [DBG] from='client.? 172.21.15.47:0/2743933733' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:48:22.594 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:48:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:48:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:48:22.619 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:48:22.622 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:22.622 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:48:22.723 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:48:22.754 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:22.943 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:48:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:48:23.010 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-23T00:48:23.403 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:48:23.406 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:23.407 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:48:23.507 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:48:23.535 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:23.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:22.592284+0000 mgr.y (mgr.14517) 389 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:48:23.798 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:22.592284+0000 mgr.y (mgr.14517) 389 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:48:23.798 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:22.592284+0000 mgr.y (mgr.14517) 389 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:48:23.832 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:48:24.190 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-23T00:48:24.190 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-23T00:48:24.190 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph pg dump --format=json 2024-02-23T00:48:24.364 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:23.289823+0000 mgr.y (mgr.14517) 390 : cluster [DBG] pgmap v616: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-23T00:48:24.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:23.289823+0000 mgr.y (mgr.14517) 390 : cluster [DBG] pgmap v616: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-23T00:48:24.384 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:48:24.387 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:24.388 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:48:24.488 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:48:24.517 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:24.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:23.289823+0000 mgr.y (mgr.14517) 390 : cluster [DBG] pgmap v616: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-23T00:48:24.732 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:48:24.732 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:48:24.968 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:48:24.971 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:24.971 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:48:25.072 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:48:25.101 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:25.397 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:48:25.551 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:48:25.556 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:25.556 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:48:25.557 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.2 dump_blocked_ops 2024-02-23T00:48:25.583 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:24.318154+0000 mon.a (mon.0) 2583 : cluster [DBG] osdmap e426: 8 total, 8 up, 7 in 2024-02-23T00:48:25.583 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:25.187271+0000 mon.a (mon.0) 2584 : cluster [DBG] osdmap e427: 8 total, 8 up, 7 in 2024-02-23T00:48:25.584 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:24.318154+0000 mon.a (mon.0) 2583 : cluster [DBG] osdmap e426: 8 total, 8 up, 7 in 2024-02-23T00:48:25.584 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:25.187271+0000 mon.a (mon.0) 2584 : cluster [DBG] osdmap e427: 8 total, 8 up, 7 in 2024-02-23T00:48:25.658 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:48:25.687 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:24.318154+0000 mon.a (mon.0) 2583 : cluster [DBG] osdmap e426: 8 total, 8 up, 7 in 2024-02-23T00:48:25.687 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:25.187271+0000 mon.a (mon.0) 2584 : cluster [DBG] osdmap e427: 8 total, 8 up, 7 in 2024-02-23T00:48:25.689 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:25.711 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:48:25.714 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:25.714 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:48:25.815 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:48:25.843 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:25.940 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:48:25.943 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:25.943 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:48:26.024 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:48:26.044 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:48:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:26.125 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:48:26.129 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:26.130 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:48:26.231 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:48:26.320 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:26.447 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:48:26.448 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:25.290197+0000 mgr.y (mgr.14517) 391 : cluster [DBG] pgmap v619: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 855 B/s rd, 0 op/s 2024-02-23T00:48:26.448 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:26.303189+0000 mon.a (mon.0) 2585 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:48:26.450 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:26.450 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:25.290197+0000 mgr.y (mgr.14517) 391 : cluster [DBG] pgmap v619: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 855 B/s rd, 0 op/s 2024-02-23T00:48:26.450 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:26.303189+0000 mon.a (mon.0) 2585 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:48:26.450 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:48:26.451 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:48:26.552 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:48:26.581 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:26.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:25.290197+0000 mgr.y (mgr.14517) 391 : cluster [DBG] pgmap v619: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 855 B/s rd, 0 op/s 2024-02-23T00:48:26.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:26.303189+0000 mon.a (mon.0) 2585 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:48:26.728 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:48:26.733 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:26.733 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:48:26.834 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:48:26.863 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:27.413 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:48:27.415 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:27.416 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:48:27.517 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:48:27.549 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:27.569 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:48:27.572 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:27.573 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:48:27.627 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:48:27.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.639+0000 7fec90a1c700 1 -- 172.21.15.47:0/2196281759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec8c0f8d10 msgr2=0x7fec8c0f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:27.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.639+0000 7fec90a1c700 1 --2- 172.21.15.47:0/2196281759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec8c0f8d10 0x7fec8c0f90f0 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 rx=0x7fec7c0047b0 tx=0x7fec7c009cb0).stop 2024-02-23T00:48:27.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.639+0000 7fec90a1c700 1 -- 172.21.15.47:0/2196281759 shutdown_connections 2024-02-23T00:48:27.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.639+0000 7fec90a1c700 1 --2- 172.21.15.47:0/2196281759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec8c102270 0x7fec8c104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:27.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.639+0000 7fec90a1c700 1 --2- 172.21.15.47:0/2196281759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec8c0f8d10 0x7fec8c0f90f0 unknown :-1 s=CLOSED pgs=2533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:27.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.639+0000 7fec90a1c700 1 --2- 172.21.15.47:0/2196281759 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fec8c0f9670 0x7fec8c101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:27.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.639+0000 7fec90a1c700 1 -- 172.21.15.47:0/2196281759 >> 172.21.15.47:0/2196281759 conn(0x7fec8c0f2820 msgr2=0x7fec8c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:48:27.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.639+0000 7fec90a1c700 1 -- 172.21.15.47:0/2196281759 shutdown_connections 2024-02-23T00:48:27.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.639+0000 7fec90a1c700 1 -- 172.21.15.47:0/2196281759 wait complete. 2024-02-23T00:48:27.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.640+0000 7fec90a1c700 1 librados: starting msgr at 2024-02-23T00:48:27.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.640+0000 7fec90a1c700 1 librados: starting objecter 2024-02-23T00:48:27.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.640+0000 7fec90a1c700 1 Processor -- start 2024-02-23T00:48:27.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.640+0000 7fec90a1c700 1 -- start start 2024-02-23T00:48:27.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.640+0000 7fec90a1c700 1 librados: setting wanted keys 2024-02-23T00:48:27.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.640+0000 7fec90a1c700 1 librados: calling monclient init 2024-02-23T00:48:27.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.640+0000 7fec90a1c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fec8c0f8d10 0x7fec8c10e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:27.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.640+0000 7fec90a1c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec8c0f9670 0x7fec8c10e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:27.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.640+0000 7fec90a1c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec8c102270 0x7fec8c108220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:27.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.640+0000 7fec90a1c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c108710 con 0x7fec8c102270 2024-02-23T00:48:27.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.640+0000 7fec90a1c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c108880 con 0x7fec8c0f9670 2024-02-23T00:48:27.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.640+0000 7fec90a1c700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c108a40 con 0x7fec8c0f8d10 2024-02-23T00:48:27.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.641+0000 7fec8a59c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fec8c0f8d10 0x7fec8c10e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:27.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.641+0000 7fec8a59c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fec8c0f8d10 0x7fec8c10e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52518/0 (socket says 172.21.15.47:52518) 2024-02-23T00:48:27.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.641+0000 7fec8ad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec8c102270 0x7fec8c108220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:27.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.641+0000 7fec8a59c700 1 -- 172.21.15.47:0/3353928822 learned_addr learned my addr 172.21.15.47:0/3353928822 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:48:27.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.641+0000 7fec89d9b700 1 --2- 172.21.15.47:0/3353928822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec8c0f9670 0x7fec8c10e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:27.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.641+0000 7fec8a59c700 1 -- 172.21.15.47:0/3353928822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec8c0f9670 msgr2=0x7fec8c10e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:27.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.641+0000 7fec8a59c700 1 --2- 172.21.15.47:0/3353928822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec8c0f9670 0x7fec8c10e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:27.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.641+0000 7fec8a59c700 1 -- 172.21.15.47:0/3353928822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec8c102270 msgr2=0x7fec8c108220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:27.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.641+0000 7fec8a59c700 1 --2- 172.21.15.47:0/3353928822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec8c102270 0x7fec8c108220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:27.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.641+0000 7fec8a59c700 1 -- 172.21.15.47:0/3353928822 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec80005040 con 0x7fec8c0f8d10 2024-02-23T00:48:27.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.641+0000 7fec8ad9d700 1 --2- 172.21.15.47:0/3353928822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec8c102270 0x7fec8c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:48:27.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.641+0000 7fec8a59c700 1 --2- 172.21.15.47:0/3353928822 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fec8c0f8d10 0x7fec8c10e080 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 rx=0x7fec7c009f20 tx=0x7fec7c009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:27.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.641+0000 7fec7b7fe700 10 client.?.objecter ms_handle_connect 0x7fec8c0f8d10 2024-02-23T00:48:27.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.641+0000 7fec7b7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:48:27.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.641+0000 7fec7b7fe700 1 -- 172.21.15.47:0/3353928822 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c018070 con 0x7fec8c0f8d10 2024-02-23T00:48:27.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.641+0000 7fec90a1c700 1 -- 172.21.15.47:0/3353928822 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec7c005040 con 0x7fec8c0f8d10 2024-02-23T00:48:27.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.641+0000 7fec7b7fe700 1 -- 172.21.15.47:0/3353928822 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fec7c013e50 con 0x7fec8c0f8d10 2024-02-23T00:48:27.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.641+0000 7fec7b7fe700 1 -- 172.21.15.47:0/3353928822 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c01dae0 con 0x7fec8c0f8d10 2024-02-23T00:48:27.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.641+0000 7fec90a1c700 10 client.41351.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:48:27.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.641+0000 7fec90a1c700 1 -- 172.21.15.47:0/3353928822 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec8c108ee0 con 0x7fec8c0f8d10 2024-02-23T00:48:27.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.641+0000 7fec90a1c700 1 librados: init done 2024-02-23T00:48:27.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.642+0000 7fec90a1c700 1 -- 172.21.15.47:0/3353928822 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec8c1090d0 con 0x7fec8c0f8d10 2024-02-23T00:48:27.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.643+0000 7fec7b7fe700 1 -- 172.21.15.47:0/3353928822 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fec7c009240 con 0x7fec8c0f8d10 2024-02-23T00:48:27.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.643+0000 7fec7b7fe700 1 --2- 172.21.15.47:0/3353928822 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fec74062cc0 0x7fec74065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:27.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.643+0000 7fec7b7fe700 1 -- 172.21.15.47:0/3353928822 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(427..427 src has 48..427) v4 ==== 9248+0+0 (secure 0 0 0) 0x7fec7c07e950 con 0x7fec8c0f8d10 2024-02-23T00:48:27.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.643+0000 7fec7b7fe700 10 client.41351.objecter ms_dispatch 0x7fec8c001550 osd_map(427..427 src has 48..427) v4 2024-02-23T00:48:27.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.643+0000 7fec7b7fe700 3 client.41351.objecter handle_osd_map got epochs [427,427] > 0 2024-02-23T00:48:27.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.643+0000 7fec7b7fe700 3 client.41351.objecter handle_osd_map decoding full epoch 427 2024-02-23T00:48:27.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.643+0000 7fec7b7fe700 20 client.41351.objecter dump_active .. 0 homeless 2024-02-23T00:48:27.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.645+0000 7fec89d9b700 1 --2- 172.21.15.47:0/3353928822 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fec74062cc0 0x7fec74065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:27.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.645+0000 7fec7b7fe700 10 client.41351.objecter ms_handle_connect 0x7fec74062cc0 2024-02-23T00:48:27.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.645+0000 7fec89d9b700 1 --2- 172.21.15.47:0/3353928822 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fec74062cc0 0x7fec74065180 secure :-1 s=READY pgs=3067 cs=0 l=1 rev1=1 rx=0x7fec8c1099c0 tx=0x7fec8000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:27.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.648+0000 7fec7b7fe700 1 -- 172.21.15.47:0/3353928822 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fec7c04df50 con 0x7fec8c0f8d10 2024-02-23T00:48:27.674 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:48:27.703 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:27.724 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:48:27.726 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:27.727 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:48:27.828 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:48:27.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:27.831+0000 7fec90a1c700 1 -- 172.21.15.47:0/3353928822 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7fec8c1070b0 con 0x7fec8c0f8d10 2024-02-23T00:48:27.856 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:27.875 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:48:27.878 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:27.879 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:48:27.980 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:48:28.010 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:28.258 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:48:28.261 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:28.261 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:48:28.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:28.324+0000 7fec7b7fe700 1 -- 172.21.15.47:0/3353928822 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v428) v1 ==== 97+0+0 (secure 0 0 0) 0x7fec7c0043d0 con 0x7fec8c0f8d10 2024-02-23T00:48:28.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:28.326+0000 7fec90a1c700 10 librados: watch_flush enter 2024-02-23T00:48:28.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:28.327+0000 7fec90a1c700 10 librados: watch_flush exit 2024-02-23T00:48:28.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:28.327+0000 7fec90a1c700 20 client.41351.objecter shutdown clearing up homeless session... 2024-02-23T00:48:28.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:28.327+0000 7fec90a1c700 10 client.41351.objecter successfully canceled tick 2024-02-23T00:48:28.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:28.327+0000 7fec90a1c700 1 -- 172.21.15.47:0/3353928822 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fec74062cc0 msgr2=0x7fec74065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:28.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:28.327+0000 7fec90a1c700 1 --2- 172.21.15.47:0/3353928822 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fec74062cc0 0x7fec74065180 secure :-1 s=READY pgs=3067 cs=0 l=1 rev1=1 rx=0x7fec8c1099c0 tx=0x7fec8000b040).stop 2024-02-23T00:48:28.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:28.327+0000 7fec90a1c700 1 -- 172.21.15.47:0/3353928822 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fec8c0f8d10 msgr2=0x7fec8c10e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:28.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:28.327+0000 7fec90a1c700 1 --2- 172.21.15.47:0/3353928822 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fec8c0f8d10 0x7fec8c10e080 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 rx=0x7fec7c009f20 tx=0x7fec7c009cb0).stop 2024-02-23T00:48:28.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:28.327+0000 7fec90a1c700 1 -- 172.21.15.47:0/3353928822 shutdown_connections 2024-02-23T00:48:28.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:28.327+0000 7fec90a1c700 1 --2- 172.21.15.47:0/3353928822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec8c102270 0x7fec8c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:28.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:28.327+0000 7fec90a1c700 1 --2- 172.21.15.47:0/3353928822 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fec74062cc0 0x7fec74065180 unknown :-1 s=CLOSED pgs=3067 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:28.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:28.327+0000 7fec90a1c700 1 --2- 172.21.15.47:0/3353928822 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fec8c0f8d10 0x7fec8c10e080 unknown :-1 s=CLOSED pgs=2669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:28.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:28.327+0000 7fec90a1c700 1 --2- 172.21.15.47:0/3353928822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec8c0f9670 0x7fec8c10e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:28.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:28.327+0000 7fec90a1c700 1 -- 172.21.15.47:0/3353928822 >> 172.21.15.47:0/3353928822 conn(0x7fec8c0f2820 msgr2=0x7fec8c0f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:48:28.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:28.327+0000 7fec90a1c700 1 -- 172.21.15.47:0/3353928822 shutdown_connections 2024-02-23T00:48:28.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:28.327+0000 7fec90a1c700 1 -- 172.21.15.47:0/3353928822 wait complete. 2024-02-23T00:48:28.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:28.327+0000 7fec90a1c700 1 librados: shutdown 2024-02-23T00:48:28.331 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is set 2024-02-23T00:48:28.362 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:48:28.391 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:28.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:27.291063+0000 mgr.y (mgr.14517) 392 : cluster [DBG] pgmap v620: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-23T00:48:28.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:27.832765+0000 mon.c (mon.2) 300 : audit [INF] from='client.? 172.21.15.47:0/3353928822' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:48:28.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:27.833139+0000 mon.a (mon.0) 2586 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:48:28.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:27.291063+0000 mgr.y (mgr.14517) 392 : cluster [DBG] pgmap v620: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-23T00:48:28.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:27.832765+0000 mon.c (mon.2) 300 : audit [INF] from='client.? 172.21.15.47:0/3353928822' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:48:28.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:27.833139+0000 mon.a (mon.0) 2586 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:48:28.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:27.291063+0000 mgr.y (mgr.14517) 392 : cluster [DBG] pgmap v620: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-23T00:48:28.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:27.832765+0000 mon.c (mon.2) 300 : audit [INF] from='client.? 172.21.15.47:0/3353928822' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:48:28.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:27.833139+0000 mon.a (mon.0) 2586 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:48:29.623 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:28.320063+0000 mon.a (mon.0) 2587 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:48:29.623 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:28.324398+0000 mon.a (mon.0) 2588 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:48:29.623 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:28.324475+0000 mon.a (mon.0) 2589 : cluster [DBG] osdmap e428: 8 total, 8 up, 7 in 2024-02-23T00:48:29.624 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:28.320063+0000 mon.a (mon.0) 2587 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:48:29.624 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:28.324398+0000 mon.a (mon.0) 2588 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:48:29.624 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:28.324475+0000 mon.a (mon.0) 2589 : cluster [DBG] osdmap e428: 8 total, 8 up, 7 in 2024-02-23T00:48:29.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:28.320063+0000 mon.a (mon.0) 2587 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:48:29.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:28.324398+0000 mon.a (mon.0) 2588 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:48:29.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:28.324475+0000 mon.a (mon.0) 2589 : cluster [DBG] osdmap e428: 8 total, 8 up, 7 in 2024-02-23T00:48:30.442 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:48:30.445 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:30.446 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:48:30.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:29.291471+0000 mgr.y (mgr.14517) 393 : cluster [DBG] pgmap v622: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:48:30.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:30.189293+0000 mon.a (mon.0) 2590 : cluster [DBG] osdmap e429: 8 total, 8 up, 7 in 2024-02-23T00:48:30.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:29.291471+0000 mgr.y (mgr.14517) 393 : cluster [DBG] pgmap v622: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:48:30.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:30.189293+0000 mon.a (mon.0) 2590 : cluster [DBG] osdmap e429: 8 total, 8 up, 7 in 2024-02-23T00:48:30.547 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:48:30.579 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:30.601 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:48:30.602 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:29.291471+0000 mgr.y (mgr.14517) 393 : cluster [DBG] pgmap v622: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:48:30.603 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:30.189293+0000 mon.a (mon.0) 2590 : cluster [DBG] osdmap e429: 8 total, 8 up, 7 in 2024-02-23T00:48:30.603 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:30.604 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:48:30.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.547+0000 7f8cebd2e700 1 -- 172.21.15.47:0/1129770790 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8ce40f8cb0 msgr2=0x7f8ce40f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:30.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.547+0000 7f8cebd2e700 1 --2- 172.21.15.47:0/1129770790 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8ce40f8cb0 0x7f8ce40f9090 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 rx=0x7f8ce0004870 tx=0x7f8ce0009cb0).stop 2024-02-23T00:48:30.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.547+0000 7f8cebd2e700 1 -- 172.21.15.47:0/1129770790 shutdown_connections 2024-02-23T00:48:30.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.547+0000 7f8cebd2e700 1 --2- 172.21.15.47:0/1129770790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ce4102210 0x7f8ce4104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:30.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.547+0000 7f8cebd2e700 1 --2- 172.21.15.47:0/1129770790 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8ce40f8cb0 0x7f8ce40f9090 unknown :-1 s=CLOSED pgs=2670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:30.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.547+0000 7f8cebd2e700 1 --2- 172.21.15.47:0/1129770790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ce40f9610 0x7f8ce4101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:30.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.547+0000 7f8cebd2e700 1 -- 172.21.15.47:0/1129770790 >> 172.21.15.47:0/1129770790 conn(0x7f8ce40f2820 msgr2=0x7f8ce40f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:48:30.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.547+0000 7f8cebd2e700 1 -- 172.21.15.47:0/1129770790 shutdown_connections 2024-02-23T00:48:30.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.547+0000 7f8cebd2e700 1 -- 172.21.15.47:0/1129770790 wait complete. 2024-02-23T00:48:30.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.548+0000 7f8cebd2e700 1 librados: starting msgr at 2024-02-23T00:48:30.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.548+0000 7f8cebd2e700 1 librados: starting objecter 2024-02-23T00:48:30.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.548+0000 7f8cebd2e700 1 Processor -- start 2024-02-23T00:48:30.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.548+0000 7f8cebd2e700 1 -- start start 2024-02-23T00:48:30.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.548+0000 7f8cebd2e700 1 librados: setting wanted keys 2024-02-23T00:48:30.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.548+0000 7f8cebd2e700 1 librados: calling monclient init 2024-02-23T00:48:30.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.548+0000 7f8cebd2e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8ce40f8cb0 0x7f8ce410e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:30.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.548+0000 7f8cebd2e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ce40f9610 0x7f8ce410e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:30.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.548+0000 7f8cebd2e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ce4102210 0x7f8ce41081c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:30.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.548+0000 7f8cebd2e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce41086b0 con 0x7f8ce40f9610 2024-02-23T00:48:30.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.548+0000 7f8cebd2e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce4108820 con 0x7f8ce4102210 2024-02-23T00:48:30.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.548+0000 7f8cebd2e700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce41089e0 con 0x7f8ce40f8cb0 2024-02-23T00:48:30.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.548+0000 7f8ce9aca700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8ce40f8cb0 0x7f8ce410e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:30.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.548+0000 7f8ce9aca700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8ce40f8cb0 0x7f8ce410e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52548/0 (socket says 172.21.15.47:52548) 2024-02-23T00:48:30.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.548+0000 7f8ce9aca700 1 -- 172.21.15.47:0/442408932 learned_addr learned my addr 172.21.15.47:0/442408932 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:48:30.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.548+0000 7f8ce92c9700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ce40f9610 0x7f8ce410e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:30.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.548+0000 7f8cea2cb700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ce4102210 0x7f8ce41081c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:30.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.548+0000 7f8ce9aca700 1 -- 172.21.15.47:0/442408932 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ce4102210 msgr2=0x7f8ce41081c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:30.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.548+0000 7f8ce9aca700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ce4102210 0x7f8ce41081c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:30.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.548+0000 7f8ce9aca700 1 -- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ce40f9610 msgr2=0x7f8ce410e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:30.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.548+0000 7f8ce9aca700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ce40f9610 0x7f8ce410e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:30.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.549+0000 7f8ce9aca700 1 -- 172.21.15.47:0/442408932 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd4005040 con 0x7f8ce40f8cb0 2024-02-23T00:48:30.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.549+0000 7f8ce9aca700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8ce40f8cb0 0x7f8ce410e020 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 rx=0x7f8ce0009da0 tx=0x7f8ce0009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:30.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.549+0000 7f8cea2cb700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ce4102210 0x7f8ce41081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:48:30.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.549+0000 7f8ce92c9700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ce40f9610 0x7f8ce410e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:48:30.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.549+0000 7f8cdaffd700 10 client.?.objecter ms_handle_connect 0x7f8ce40f8cb0 2024-02-23T00:48:30.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.549+0000 7f8cdaffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:48:30.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.549+0000 7f8cdaffd700 1 -- 172.21.15.47:0/442408932 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce000e030 con 0x7f8ce40f8cb0 2024-02-23T00:48:30.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.549+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ce0005040 con 0x7f8ce40f8cb0 2024-02-23T00:48:30.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.549+0000 7f8cdaffd700 1 -- 172.21.15.47:0/442408932 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f8ce0013e70 con 0x7f8ce40f8cb0 2024-02-23T00:48:30.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.549+0000 7f8cdaffd700 1 -- 172.21.15.47:0/442408932 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0018070 con 0x7f8ce40f8cb0 2024-02-23T00:48:30.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.549+0000 7f8cebd2e700 10 client.41357.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:48:30.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.549+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ce4108e80 con 0x7f8ce40f8cb0 2024-02-23T00:48:30.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.549+0000 7f8cebd2e700 1 librados: init done 2024-02-23T00:48:30.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.550+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f8ce4109070 con 0x7f8ce40f8cb0 2024-02-23T00:48:30.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.551+0000 7f8cdaffd700 1 -- 172.21.15.47:0/442408932 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f8ce0009240 con 0x7f8ce40f8cb0 2024-02-23T00:48:30.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.551+0000 7f8cdaffd700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8cd0062cc0 0x7f8cd0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:30.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.551+0000 7f8cdaffd700 1 -- 172.21.15.47:0/442408932 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(429..429 src has 48..429) v4 ==== 9248+0+0 (secure 0 0 0) 0x7f8ce007f060 con 0x7f8ce40f8cb0 2024-02-23T00:48:30.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.551+0000 7f8cdaffd700 10 client.41357.objecter ms_dispatch 0x7f8ce4001550 osd_map(429..429 src has 48..429) v4 2024-02-23T00:48:30.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.551+0000 7f8cdaffd700 3 client.41357.objecter handle_osd_map got epochs [429,429] > 0 2024-02-23T00:48:30.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.551+0000 7f8cdaffd700 3 client.41357.objecter handle_osd_map decoding full epoch 429 2024-02-23T00:48:30.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.551+0000 7f8ce92c9700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8cd0062cc0 0x7f8cd0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.551+0000 7f8cdaffd700 20 client.41357.objecter dump_active .. 0 homeless 2024-02-23T00:48:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.551+0000 7f8cdaffd700 1 -- 172.21.15.47:0/442408932 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v429) v1 ==== 54+0+16 (secure 0 0 0) 0x7f8ce007f500 con 0x7f8ce40f8cb0 2024-02-23T00:48:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.552+0000 7f8cebd2e700 10 client.41357.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.552+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_assign -1 1 2024-02-23T00:48:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.552+0000 7f8cdaffd700 10 client.41357.objecter ms_handle_connect 0x7f8cd0062cc0 2024-02-23T00:48:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.552+0000 7f8ce92c9700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8cd0062cc0 0x7f8cd0065180 secure :-1 s=READY pgs=3068 cs=0 l=1 rev1=1 rx=0x7f8ce4109960 tx=0x7f8cd400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.552+0000 7f8cebd2e700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f8ce40fc1a0 0x7f8ce40fc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.552+0000 7f8cebd2e700 20 client.41357.objecter _get_session s=0x7f8ce405fa60 osd=0 3 2024-02-23T00:48:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.552+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce405fa60 osd=0 3 2024-02-23T00:48:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.552+0000 7f8cebd2e700 20 client.41357.objecter _get_session s=0x7f8ce405fa60 osd=0 3 2024-02-23T00:48:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.552+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_remove -1 1 2024-02-23T00:48:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.552+0000 7f8cebd2e700 20 client.41357.objecter get_session s=0x7f8ce405fa60 osd=0 3 2024-02-23T00:48:30.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.552+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_assign 0 1 2024-02-23T00:48:30.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.552+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce405fa60 osd=0 4 2024-02-23T00:48:30.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.552+0000 7f8cebd2e700 10 client.41357.objecter _send_command 1 2024-02-23T00:48:30.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.552+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8ce405ee80 con 0x7f8ce40fc1a0 2024-02-23T00:48:30.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.552+0000 7f8cea2cb700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f8ce40fc1a0 0x7f8ce40fc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:30.677 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:48:30.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.552+0000 7f8cdaffd700 10 client.41357.objecter ms_handle_connect 0x7f8ce40fc1a0 2024-02-23T00:48:30.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.552+0000 7f8cea2cb700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f8ce40fc1a0 0x7f8ce40fc580 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:30.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.553+0000 7f8cdaffd700 1 -- 172.21.15.47:0/442408932 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f8ce405ee80 con 0x7f8ce40fc1a0 2024-02-23T00:48:30.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.553+0000 7f8cdaffd700 10 client.41357.objecter ms_dispatch 0x7f8ce4001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:48:30.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.553+0000 7f8cdaffd700 10 client.41357.objecter _finish_command 1 = system:0 2024-02-23T00:48:30.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.553+0000 7f8cdaffd700 20 client.41357.objecter put_session s=0x7f8ce405fa60 osd=0 4 2024-02-23T00:48:30.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.553+0000 7f8cdaffd700 15 client.41357.objecter _session_command_op_remove 0 1 2024-02-23T00:48:30.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.569+0000 7f8cebd2e700 10 client.41357.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:48:30.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.569+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_assign -1 2 2024-02-23T00:48:30.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.569+0000 7f8cebd2e700 20 client.41357.objecter _get_session s=0x7f8ce405fa60 osd=0 3 2024-02-23T00:48:30.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.569+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce405fa60 osd=0 3 2024-02-23T00:48:30.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.569+0000 7f8cebd2e700 20 client.41357.objecter _get_session s=0x7f8ce405fa60 osd=0 3 2024-02-23T00:48:30.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.569+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_remove -1 2 2024-02-23T00:48:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.569+0000 7f8cebd2e700 20 client.41357.objecter get_session s=0x7f8ce405fa60 osd=0 3 2024-02-23T00:48:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.569+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_assign 0 2 2024-02-23T00:48:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.569+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce405fa60 osd=0 4 2024-02-23T00:48:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.569+0000 7f8cebd2e700 10 client.41357.objecter _send_command 2 2024-02-23T00:48:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.569+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f8ce4107050 con 0x7f8ce40fc1a0 2024-02-23T00:48:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.571+0000 7f8cdaffd700 1 -- 172.21.15.47:0/442408932 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f8ce4107050 con 0x7f8ce40fc1a0 2024-02-23T00:48:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.571+0000 7f8cdaffd700 10 client.41357.objecter ms_dispatch 0x7f8ce4001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:48:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.571+0000 7f8cdaffd700 10 client.41357.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:48:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.571+0000 7f8cdaffd700 20 client.41357.objecter put_session s=0x7f8ce405fa60 osd=0 4 2024-02-23T00:48:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.571+0000 7f8cdaffd700 15 client.41357.objecter _session_command_op_remove 0 2 2024-02-23T00:48:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.571+0000 7f8cebd2e700 10 client.41357.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.571+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_assign -1 3 2024-02-23T00:48:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.571+0000 7f8cebd2e700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f8ce40598f0 0x7f8ce4059cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.571+0000 7f8cebd2e700 20 client.41357.objecter _get_session s=0x7f8ce4028540 osd=1 3 2024-02-23T00:48:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.571+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce4028540 osd=1 3 2024-02-23T00:48:30.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.571+0000 7f8cebd2e700 20 client.41357.objecter _get_session s=0x7f8ce4028540 osd=1 3 2024-02-23T00:48:30.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.571+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_remove -1 3 2024-02-23T00:48:30.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.571+0000 7f8cebd2e700 20 client.41357.objecter get_session s=0x7f8ce4028540 osd=1 3 2024-02-23T00:48:30.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.571+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_assign 1 3 2024-02-23T00:48:30.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.571+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce4028540 osd=1 4 2024-02-23T00:48:30.680 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:48:30.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.571+0000 7f8cebd2e700 10 client.41357.objecter _send_command 3 2024-02-23T00:48:30.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.571+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8ce40286b0 con 0x7f8ce40598f0 2024-02-23T00:48:30.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.571+0000 7f8ce9aca700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f8ce40598f0 0x7f8ce4059cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:30.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.572+0000 7f8cdaffd700 10 client.41357.objecter ms_handle_connect 0x7f8ce40598f0 2024-02-23T00:48:30.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.572+0000 7f8ce9aca700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f8ce40598f0 0x7f8ce4059cf0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:30.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.573+0000 7f8cdaffd700 1 -- 172.21.15.47:0/442408932 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f8ce40286b0 con 0x7f8ce40598f0 2024-02-23T00:48:30.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.573+0000 7f8cdaffd700 10 client.41357.objecter ms_dispatch 0x7f8ce4001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:48:30.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.573+0000 7f8cdaffd700 10 client.41357.objecter _finish_command 3 = system:0 2024-02-23T00:48:30.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.573+0000 7f8cdaffd700 20 client.41357.objecter put_session s=0x7f8ce4028540 osd=1 4 2024-02-23T00:48:30.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.573+0000 7f8cdaffd700 15 client.41357.objecter _session_command_op_remove 1 3 2024-02-23T00:48:30.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.588+0000 7f8cebd2e700 10 client.41357.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:48:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.588+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_assign -1 4 2024-02-23T00:48:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.588+0000 7f8cebd2e700 20 client.41357.objecter _get_session s=0x7f8ce4028540 osd=1 3 2024-02-23T00:48:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.588+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce4028540 osd=1 3 2024-02-23T00:48:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.588+0000 7f8cebd2e700 20 client.41357.objecter _get_session s=0x7f8ce4028540 osd=1 3 2024-02-23T00:48:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.588+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_remove -1 4 2024-02-23T00:48:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.588+0000 7f8cebd2e700 20 client.41357.objecter get_session s=0x7f8ce4028540 osd=1 3 2024-02-23T00:48:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.588+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_assign 1 4 2024-02-23T00:48:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.588+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce4028540 osd=1 4 2024-02-23T00:48:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.588+0000 7f8cebd2e700 10 client.41357.objecter _send_command 4 2024-02-23T00:48:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.588+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f8ce4028820 con 0x7f8ce40598f0 2024-02-23T00:48:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.589+0000 7f8cdaffd700 1 -- 172.21.15.47:0/442408932 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f8ce007f500 con 0x7f8ce40598f0 2024-02-23T00:48:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.589+0000 7f8cdaffd700 10 client.41357.objecter ms_dispatch 0x7f8ce4001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:48:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.589+0000 7f8cdaffd700 10 client.41357.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:48:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.589+0000 7f8cdaffd700 20 client.41357.objecter put_session s=0x7f8ce4028540 osd=1 4 2024-02-23T00:48:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.589+0000 7f8cdaffd700 15 client.41357.objecter _session_command_op_remove 1 4 2024-02-23T00:48:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.589+0000 7f8cebd2e700 10 client.41357.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.589+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_assign -1 5 2024-02-23T00:48:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.589+0000 7f8cebd2e700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f8ce40280a0 0x7f8ce411c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.590+0000 7f8cebd2e700 20 client.41357.objecter _get_session s=0x7f8ce40595a0 osd=2 3 2024-02-23T00:48:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.590+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce40595a0 osd=2 3 2024-02-23T00:48:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.590+0000 7f8cebd2e700 20 client.41357.objecter _get_session s=0x7f8ce40595a0 osd=2 3 2024-02-23T00:48:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.590+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_remove -1 5 2024-02-23T00:48:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.590+0000 7f8cebd2e700 20 client.41357.objecter get_session s=0x7f8ce40595a0 osd=2 3 2024-02-23T00:48:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.590+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_assign 2 5 2024-02-23T00:48:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.590+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce40595a0 osd=2 4 2024-02-23T00:48:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.590+0000 7f8cebd2e700 10 client.41357.objecter _send_command 5 2024-02-23T00:48:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.590+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8ce411ec50 con 0x7f8ce40280a0 2024-02-23T00:48:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.590+0000 7f8ce92c9700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f8ce40280a0 0x7f8ce411c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.590+0000 7f8cdaffd700 10 client.41357.objecter ms_handle_connect 0x7f8ce40280a0 2024-02-23T00:48:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.590+0000 7f8ce92c9700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f8ce40280a0 0x7f8ce411c000 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.590+0000 7f8cdaffd700 1 -- 172.21.15.47:0/442408932 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f8ce411ec50 con 0x7f8ce40280a0 2024-02-23T00:48:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.590+0000 7f8cdaffd700 10 client.41357.objecter ms_dispatch 0x7f8ce4001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:48:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.590+0000 7f8cdaffd700 10 client.41357.objecter _finish_command 5 = system:0 2024-02-23T00:48:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.590+0000 7f8cdaffd700 20 client.41357.objecter put_session s=0x7f8ce40595a0 osd=2 4 2024-02-23T00:48:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.590+0000 7f8cdaffd700 15 client.41357.objecter _session_command_op_remove 2 5 2024-02-23T00:48:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.605+0000 7f8cebd2e700 10 client.41357.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:48:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.605+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_assign -1 6 2024-02-23T00:48:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.605+0000 7f8cebd2e700 20 client.41357.objecter _get_session s=0x7f8ce40595a0 osd=2 3 2024-02-23T00:48:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.605+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce40595a0 osd=2 3 2024-02-23T00:48:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.605+0000 7f8cebd2e700 20 client.41357.objecter _get_session s=0x7f8ce40595a0 osd=2 3 2024-02-23T00:48:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.605+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_remove -1 6 2024-02-23T00:48:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.605+0000 7f8cebd2e700 20 client.41357.objecter get_session s=0x7f8ce40595a0 osd=2 3 2024-02-23T00:48:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.605+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_assign 2 6 2024-02-23T00:48:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.605+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce40595a0 osd=2 4 2024-02-23T00:48:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.605+0000 7f8cebd2e700 10 client.41357.objecter _send_command 6 2024-02-23T00:48:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.605+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f8ce4027600 con 0x7f8ce40280a0 2024-02-23T00:48:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.607+0000 7f8cdaffd700 1 -- 172.21.15.47:0/442408932 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f8ce4027600 con 0x7f8ce40280a0 2024-02-23T00:48:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.607+0000 7f8cdaffd700 10 client.41357.objecter ms_dispatch 0x7f8ce4001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:48:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.607+0000 7f8cdaffd700 10 client.41357.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:48:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.607+0000 7f8cdaffd700 20 client.41357.objecter put_session s=0x7f8ce40595a0 osd=2 4 2024-02-23T00:48:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.607+0000 7f8cdaffd700 15 client.41357.objecter _session_command_op_remove 2 6 2024-02-23T00:48:30.685 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:48:30.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.607+0000 7f8cebd2e700 10 client.41357.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:30.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.607+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_assign -1 7 2024-02-23T00:48:30.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.607+0000 7f8cebd2e700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f8ce4119b50 0x7f8ce411d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:30.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.607+0000 7f8cebd2e700 20 client.41357.objecter _get_session s=0x7f8ce4027a10 osd=3 3 2024-02-23T00:48:30.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.607+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce4027a10 osd=3 3 2024-02-23T00:48:30.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.607+0000 7f8cebd2e700 20 client.41357.objecter _get_session s=0x7f8ce4027a10 osd=3 3 2024-02-23T00:48:30.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.607+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_remove -1 7 2024-02-23T00:48:30.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.607+0000 7f8cebd2e700 20 client.41357.objecter get_session s=0x7f8ce4027a10 osd=3 3 2024-02-23T00:48:30.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.607+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_assign 3 7 2024-02-23T00:48:30.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.607+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce4027a10 osd=3 4 2024-02-23T00:48:30.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.607+0000 7f8cebd2e700 10 client.41357.objecter _send_command 7 2024-02-23T00:48:30.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.607+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8ce411dcc0 con 0x7f8ce4119b50 2024-02-23T00:48:30.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.607+0000 7f8cea2cb700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f8ce4119b50 0x7f8ce411d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:30.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.607+0000 7f8cdaffd700 10 client.41357.objecter ms_handle_connect 0x7f8ce4119b50 2024-02-23T00:48:30.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.607+0000 7f8cea2cb700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f8ce4119b50 0x7f8ce411d670 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:30.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.608+0000 7f8cdaffd700 1 -- 172.21.15.47:0/442408932 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f8ce411dcc0 con 0x7f8ce4119b50 2024-02-23T00:48:30.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.608+0000 7f8cdaffd700 10 client.41357.objecter ms_dispatch 0x7f8ce4001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:48:30.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.608+0000 7f8cdaffd700 10 client.41357.objecter _finish_command 7 = system:0 2024-02-23T00:48:30.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.608+0000 7f8cdaffd700 20 client.41357.objecter put_session s=0x7f8ce4027a10 osd=3 4 2024-02-23T00:48:30.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.608+0000 7f8cdaffd700 15 client.41357.objecter _session_command_op_remove 3 7 2024-02-23T00:48:30.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.622+0000 7f8cebd2e700 10 client.41357.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:48:30.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.622+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_assign -1 8 2024-02-23T00:48:30.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.622+0000 7f8cebd2e700 20 client.41357.objecter _get_session s=0x7f8ce4027a10 osd=3 3 2024-02-23T00:48:30.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.622+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce4027a10 osd=3 3 2024-02-23T00:48:30.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.622+0000 7f8cebd2e700 20 client.41357.objecter _get_session s=0x7f8ce4027a10 osd=3 3 2024-02-23T00:48:30.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.622+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_remove -1 8 2024-02-23T00:48:30.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.622+0000 7f8cebd2e700 20 client.41357.objecter get_session s=0x7f8ce4027a10 osd=3 3 2024-02-23T00:48:30.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.622+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_assign 3 8 2024-02-23T00:48:30.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.622+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce4027a10 osd=3 4 2024-02-23T00:48:30.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.622+0000 7f8cebd2e700 10 client.41357.objecter _send_command 8 2024-02-23T00:48:30.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.622+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f8ce41190b0 con 0x7f8ce4119b50 2024-02-23T00:48:30.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.624+0000 7f8cdaffd700 1 -- 172.21.15.47:0/442408932 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f8ce41190b0 con 0x7f8ce4119b50 2024-02-23T00:48:30.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.624+0000 7f8cdaffd700 10 client.41357.objecter ms_dispatch 0x7f8ce4001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:48:30.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.624+0000 7f8cdaffd700 10 client.41357.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:48:30.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.624+0000 7f8cdaffd700 20 client.41357.objecter put_session s=0x7f8ce4027a10 osd=3 4 2024-02-23T00:48:30.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.624+0000 7f8cdaffd700 15 client.41357.objecter _session_command_op_remove 3 8 2024-02-23T00:48:30.689 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:48:30.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.624+0000 7f8cebd2e700 10 client.41357.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:30.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.624+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_assign -1 9 2024-02-23T00:48:30.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.624+0000 7f8cebd2e700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f8ce40f6aa0 0x7f8ce40f6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:30.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.624+0000 7f8cebd2e700 20 client.41357.objecter _get_session s=0x7f8ce4118e70 osd=4 3 2024-02-23T00:48:30.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.624+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce4118e70 osd=4 3 2024-02-23T00:48:30.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.624+0000 7f8cebd2e700 20 client.41357.objecter _get_session s=0x7f8ce4118e70 osd=4 3 2024-02-23T00:48:30.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.624+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_remove -1 9 2024-02-23T00:48:30.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.624+0000 7f8cebd2e700 20 client.41357.objecter get_session s=0x7f8ce4118e70 osd=4 3 2024-02-23T00:48:30.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.624+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_assign 4 9 2024-02-23T00:48:30.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.624+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce4118e70 osd=4 4 2024-02-23T00:48:30.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.624+0000 7f8cebd2e700 10 client.41357.objecter _send_command 9 2024-02-23T00:48:30.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.624+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8ce4118fe0 con 0x7f8ce40f6aa0 2024-02-23T00:48:30.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.624+0000 7f8ce9aca700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f8ce40f6aa0 0x7f8ce40f6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:30.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.625+0000 7f8ce9aca700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f8ce40f6aa0 0x7f8ce40f6ee0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:30.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.625+0000 7f8cdaffd700 10 client.41357.objecter ms_handle_connect 0x7f8ce40f6aa0 2024-02-23T00:48:30.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.625+0000 7f8cdaffd700 1 -- 172.21.15.47:0/442408932 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f8ce4118fe0 con 0x7f8ce40f6aa0 2024-02-23T00:48:30.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.625+0000 7f8cdaffd700 10 client.41357.objecter ms_dispatch 0x7f8ce4001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:48:30.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.625+0000 7f8cdaffd700 10 client.41357.objecter _finish_command 9 = system:0 2024-02-23T00:48:30.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.625+0000 7f8cdaffd700 20 client.41357.objecter put_session s=0x7f8ce4118e70 osd=4 4 2024-02-23T00:48:30.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.625+0000 7f8cdaffd700 15 client.41357.objecter _session_command_op_remove 4 9 2024-02-23T00:48:30.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.640+0000 7f8cebd2e700 10 client.41357.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:48:30.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.640+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_assign -1 10 2024-02-23T00:48:30.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.640+0000 7f8cebd2e700 20 client.41357.objecter _get_session s=0x7f8ce4118e70 osd=4 3 2024-02-23T00:48:30.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.640+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce4118e70 osd=4 3 2024-02-23T00:48:30.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.640+0000 7f8cebd2e700 20 client.41357.objecter _get_session s=0x7f8ce4118e70 osd=4 3 2024-02-23T00:48:30.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.640+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_remove -1 10 2024-02-23T00:48:30.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.640+0000 7f8cebd2e700 20 client.41357.objecter get_session s=0x7f8ce4118e70 osd=4 3 2024-02-23T00:48:30.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.640+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_assign 4 10 2024-02-23T00:48:30.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.640+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce4118e70 osd=4 4 2024-02-23T00:48:30.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.640+0000 7f8cebd2e700 10 client.41357.objecter _send_command 10 2024-02-23T00:48:30.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.640+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f8ce4118fe0 con 0x7f8ce40f6aa0 2024-02-23T00:48:30.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.642+0000 7f8cdaffd700 1 -- 172.21.15.47:0/442408932 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f8ce4118fe0 con 0x7f8ce40f6aa0 2024-02-23T00:48:30.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.642+0000 7f8cdaffd700 10 client.41357.objecter ms_dispatch 0x7f8ce4001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:48:30.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.642+0000 7f8cdaffd700 10 client.41357.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:48:30.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.642+0000 7f8cdaffd700 20 client.41357.objecter put_session s=0x7f8ce4118e70 osd=4 4 2024-02-23T00:48:30.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.642+0000 7f8cdaffd700 15 client.41357.objecter _session_command_op_remove 4 10 2024-02-23T00:48:30.693 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:48:30.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.642+0000 7f8cebd2e700 10 client.41357.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:30.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.642+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_assign -1 11 2024-02-23T00:48:30.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.642+0000 7f8cebd2e700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f8ce40f78e0 0x7f8ce40f7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:30.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.642+0000 7f8cebd2e700 20 client.41357.objecter _get_session s=0x7f8ce4118fe0 osd=5 3 2024-02-23T00:48:30.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.642+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce4118fe0 osd=5 3 2024-02-23T00:48:30.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.642+0000 7f8cebd2e700 20 client.41357.objecter _get_session s=0x7f8ce4118fe0 osd=5 3 2024-02-23T00:48:30.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.642+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_remove -1 11 2024-02-23T00:48:30.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.642+0000 7f8cebd2e700 20 client.41357.objecter get_session s=0x7f8ce4118fe0 osd=5 3 2024-02-23T00:48:30.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.642+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_assign 5 11 2024-02-23T00:48:30.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.642+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce4118fe0 osd=5 4 2024-02-23T00:48:30.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.642+0000 7f8cebd2e700 10 client.41357.objecter _send_command 11 2024-02-23T00:48:30.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.642+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8ce40f8420 con 0x7f8ce40f78e0 2024-02-23T00:48:30.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.643+0000 7f8ce92c9700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f8ce40f78e0 0x7f8ce40f7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:30.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.643+0000 7f8ce92c9700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f8ce40f78e0 0x7f8ce40f7d40 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:30.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.643+0000 7f8cdaffd700 10 client.41357.objecter ms_handle_connect 0x7f8ce40f78e0 2024-02-23T00:48:30.694 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:48:30.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.644+0000 7f8cdaffd700 1 -- 172.21.15.47:0/442408932 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f8ce40f8420 con 0x7f8ce40f78e0 2024-02-23T00:48:30.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.644+0000 7f8cdaffd700 10 client.41357.objecter ms_dispatch 0x7f8ce4001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:48:30.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.644+0000 7f8cdaffd700 10 client.41357.objecter _finish_command 11 = system:0 2024-02-23T00:48:30.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.644+0000 7f8cdaffd700 20 client.41357.objecter put_session s=0x7f8ce4118fe0 osd=5 4 2024-02-23T00:48:30.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.644+0000 7f8cdaffd700 15 client.41357.objecter _session_command_op_remove 5 11 2024-02-23T00:48:30.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.658+0000 7f8cebd2e700 10 client.41357.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:48:30.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.658+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_assign -1 12 2024-02-23T00:48:30.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.658+0000 7f8cebd2e700 20 client.41357.objecter _get_session s=0x7f8ce4118fe0 osd=5 3 2024-02-23T00:48:30.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.658+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce4118fe0 osd=5 3 2024-02-23T00:48:30.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.658+0000 7f8cebd2e700 20 client.41357.objecter _get_session s=0x7f8ce4118fe0 osd=5 3 2024-02-23T00:48:30.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.658+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_remove -1 12 2024-02-23T00:48:30.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.658+0000 7f8cebd2e700 20 client.41357.objecter get_session s=0x7f8ce4118fe0 osd=5 3 2024-02-23T00:48:30.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.658+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_assign 5 12 2024-02-23T00:48:30.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.658+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce4118fe0 osd=5 4 2024-02-23T00:48:30.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.658+0000 7f8cebd2e700 10 client.41357.objecter _send_command 12 2024-02-23T00:48:30.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.658+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f8ce40f8420 con 0x7f8ce40f78e0 2024-02-23T00:48:30.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.660+0000 7f8cdaffd700 1 -- 172.21.15.47:0/442408932 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f8ce40f8420 con 0x7f8ce40f78e0 2024-02-23T00:48:30.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.660+0000 7f8cdaffd700 10 client.41357.objecter ms_dispatch 0x7f8ce4001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:48:30.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.660+0000 7f8cdaffd700 10 client.41357.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:48:30.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.660+0000 7f8cdaffd700 20 client.41357.objecter put_session s=0x7f8ce4118fe0 osd=5 4 2024-02-23T00:48:30.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.660+0000 7f8cdaffd700 15 client.41357.objecter _session_command_op_remove 5 12 2024-02-23T00:48:30.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.660+0000 7f8cebd2e700 10 client.41357.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:30.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.660+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_assign -1 13 2024-02-23T00:48:30.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.660+0000 7f8cebd2e700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f8ce4129200 0x7f8ce412b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:30.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.660+0000 7f8cebd2e700 20 client.41357.objecter _get_session s=0x7f8ce40f8420 osd=6 3 2024-02-23T00:48:30.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.660+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce40f8420 osd=6 3 2024-02-23T00:48:30.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.660+0000 7f8cebd2e700 20 client.41357.objecter _get_session s=0x7f8ce40f8420 osd=6 3 2024-02-23T00:48:30.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.660+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_remove -1 13 2024-02-23T00:48:30.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.660+0000 7f8cebd2e700 20 client.41357.objecter get_session s=0x7f8ce40f8420 osd=6 3 2024-02-23T00:48:30.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.660+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_assign 6 13 2024-02-23T00:48:30.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.660+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce40f8420 osd=6 4 2024-02-23T00:48:30.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.660+0000 7f8cebd2e700 10 client.41357.objecter _send_command 13 2024-02-23T00:48:30.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.660+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8ce412e350 con 0x7f8ce4129200 2024-02-23T00:48:30.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.660+0000 7f8cea2cb700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f8ce4129200 0x7f8ce412b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:30.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.661+0000 7f8cdaffd700 10 client.41357.objecter ms_handle_connect 0x7f8ce4129200 2024-02-23T00:48:30.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.661+0000 7f8cea2cb700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f8ce4129200 0x7f8ce412b5f0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:30.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.662+0000 7f8cdaffd700 1 -- 172.21.15.47:0/442408932 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f8ce412e350 con 0x7f8ce4129200 2024-02-23T00:48:30.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.662+0000 7f8cdaffd700 10 client.41357.objecter ms_dispatch 0x7f8ce4001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:48:30.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.662+0000 7f8cdaffd700 10 client.41357.objecter _finish_command 13 = system:0 2024-02-23T00:48:30.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.662+0000 7f8cdaffd700 20 client.41357.objecter put_session s=0x7f8ce40f8420 osd=6 4 2024-02-23T00:48:30.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.662+0000 7f8cdaffd700 15 client.41357.objecter _session_command_op_remove 6 13 2024-02-23T00:48:30.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.676+0000 7f8cebd2e700 10 client.41357.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:48:30.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.676+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_assign -1 14 2024-02-23T00:48:30.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.676+0000 7f8cebd2e700 20 client.41357.objecter _get_session s=0x7f8ce40f8420 osd=6 3 2024-02-23T00:48:30.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.676+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce40f8420 osd=6 3 2024-02-23T00:48:30.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.676+0000 7f8cebd2e700 20 client.41357.objecter _get_session s=0x7f8ce40f8420 osd=6 3 2024-02-23T00:48:30.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.676+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_remove -1 14 2024-02-23T00:48:30.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.676+0000 7f8cebd2e700 20 client.41357.objecter get_session s=0x7f8ce40f8420 osd=6 3 2024-02-23T00:48:30.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.676+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_assign 6 14 2024-02-23T00:48:30.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.676+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce40f8420 osd=6 4 2024-02-23T00:48:30.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.676+0000 7f8cebd2e700 10 client.41357.objecter _send_command 14 2024-02-23T00:48:30.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.676+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f8ce412e350 con 0x7f8ce4129200 2024-02-23T00:48:30.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.678+0000 7f8cdaffd700 1 -- 172.21.15.47:0/442408932 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f8ce412e350 con 0x7f8ce4129200 2024-02-23T00:48:30.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.678+0000 7f8cdaffd700 10 client.41357.objecter ms_dispatch 0x7f8ce4001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:48:30.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.678+0000 7f8cdaffd700 10 client.41357.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:48:30.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.678+0000 7f8cdaffd700 20 client.41357.objecter put_session s=0x7f8ce40f8420 osd=6 4 2024-02-23T00:48:30.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.678+0000 7f8cdaffd700 15 client.41357.objecter _session_command_op_remove 6 14 2024-02-23T00:48:30.699 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:48:30.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.678+0000 7f8cebd2e700 10 client.41357.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:30.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.678+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_assign -1 15 2024-02-23T00:48:30.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.678+0000 7f8cebd2e700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f8ce412d6a0 0x7f8ce412db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:30.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.678+0000 7f8cebd2e700 20 client.41357.objecter _get_session s=0x7f8ce412e350 osd=7 3 2024-02-23T00:48:30.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.678+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce412e350 osd=7 3 2024-02-23T00:48:30.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.678+0000 7f8cebd2e700 20 client.41357.objecter _get_session s=0x7f8ce412e350 osd=7 3 2024-02-23T00:48:30.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.678+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_remove -1 15 2024-02-23T00:48:30.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.678+0000 7f8cebd2e700 20 client.41357.objecter get_session s=0x7f8ce412e350 osd=7 3 2024-02-23T00:48:30.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.678+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_assign 7 15 2024-02-23T00:48:30.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.678+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce412e350 osd=7 4 2024-02-23T00:48:30.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.678+0000 7f8cebd2e700 10 client.41357.objecter _send_command 15 2024-02-23T00:48:30.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.678+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8ce4131680 con 0x7f8ce412d6a0 2024-02-23T00:48:30.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.679+0000 7f8ce9aca700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f8ce412d6a0 0x7f8ce412db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:30.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.679+0000 7f8cdaffd700 10 client.41357.objecter ms_handle_connect 0x7f8ce412d6a0 2024-02-23T00:48:30.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.679+0000 7f8ce9aca700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f8ce412d6a0 0x7f8ce412db00 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:30.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.680+0000 7f8cdaffd700 1 -- 172.21.15.47:0/442408932 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f8ce4131680 con 0x7f8ce412d6a0 2024-02-23T00:48:30.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.680+0000 7f8cdaffd700 10 client.41357.objecter ms_dispatch 0x7f8ce4001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:48:30.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.680+0000 7f8cdaffd700 10 client.41357.objecter _finish_command 15 = system:0 2024-02-23T00:48:30.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.680+0000 7f8cdaffd700 20 client.41357.objecter put_session s=0x7f8ce412e350 osd=7 4 2024-02-23T00:48:30.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.680+0000 7f8cdaffd700 15 client.41357.objecter _session_command_op_remove 7 15 2024-02-23T00:48:30.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.694+0000 7f8cebd2e700 10 client.41357.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:48:30.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.694+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_assign -1 16 2024-02-23T00:48:30.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.694+0000 7f8cebd2e700 20 client.41357.objecter _get_session s=0x7f8ce412e350 osd=7 3 2024-02-23T00:48:30.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.694+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce412e350 osd=7 3 2024-02-23T00:48:30.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.694+0000 7f8cebd2e700 20 client.41357.objecter _get_session s=0x7f8ce412e350 osd=7 3 2024-02-23T00:48:30.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.694+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_remove -1 16 2024-02-23T00:48:30.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.694+0000 7f8cebd2e700 20 client.41357.objecter get_session s=0x7f8ce412e350 osd=7 3 2024-02-23T00:48:30.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.694+0000 7f8cebd2e700 15 client.41357.objecter _session_command_op_assign 7 16 2024-02-23T00:48:30.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.694+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce412e350 osd=7 4 2024-02-23T00:48:30.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.694+0000 7f8cebd2e700 10 client.41357.objecter _send_command 16 2024-02-23T00:48:30.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.694+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f8ce412bb30 con 0x7f8ce412d6a0 2024-02-23T00:48:30.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.696+0000 7f8cdaffd700 1 -- 172.21.15.47:0/442408932 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f8ce412bb30 con 0x7f8ce412d6a0 2024-02-23T00:48:30.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.696+0000 7f8cdaffd700 10 client.41357.objecter ms_dispatch 0x7f8ce4001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:48:30.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.696+0000 7f8cdaffd700 10 client.41357.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:48:30.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.696+0000 7f8cdaffd700 20 client.41357.objecter put_session s=0x7f8ce412e350 osd=7 4 2024-02-23T00:48:30.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.696+0000 7f8cdaffd700 15 client.41357.objecter _session_command_op_remove 7 16 2024-02-23T00:48:30.703 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:48:30.704 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:48:30.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 10 librados: watch_flush enter 2024-02-23T00:48:30.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 10 librados: watch_flush exit 2024-02-23T00:48:30.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 10 client.41357.objecter close_session for osd.0 2024-02-23T00:48:30.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f8ce40fc1a0 msgr2=0x7f8ce40fc580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:30.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f8ce40fc1a0 0x7f8ce40fc580 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:30.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce405fa60 osd=0 1 2024-02-23T00:48:30.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 10 client.41357.objecter close_session for osd.1 2024-02-23T00:48:30.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f8ce40598f0 msgr2=0x7f8ce4059cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:30.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f8ce40598f0 0x7f8ce4059cf0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:30.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce4028540 osd=1 1 2024-02-23T00:48:30.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 10 client.41357.objecter close_session for osd.2 2024-02-23T00:48:30.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f8ce40280a0 msgr2=0x7f8ce411c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:30.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f8ce40280a0 0x7f8ce411c000 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:30.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce40595a0 osd=2 1 2024-02-23T00:48:30.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 10 client.41357.objecter close_session for osd.3 2024-02-23T00:48:30.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f8ce4119b50 msgr2=0x7f8ce411d670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:30.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f8ce4119b50 0x7f8ce411d670 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:30.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce4027a10 osd=3 1 2024-02-23T00:48:30.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 10 client.41357.objecter close_session for osd.4 2024-02-23T00:48:30.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f8ce40f6aa0 msgr2=0x7f8ce40f6ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:30.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f8ce40f6aa0 0x7f8ce40f6ee0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:30.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce4118e70 osd=4 1 2024-02-23T00:48:30.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 10 client.41357.objecter close_session for osd.5 2024-02-23T00:48:30.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f8ce40f78e0 msgr2=0x7f8ce40f7d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:30.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f8ce40f78e0 0x7f8ce40f7d40 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:30.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce4118fe0 osd=5 1 2024-02-23T00:48:30.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 10 client.41357.objecter close_session for osd.6 2024-02-23T00:48:30.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f8ce4129200 msgr2=0x7f8ce412b5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:30.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f8ce4129200 0x7f8ce412b5f0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:30.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce40f8420 osd=6 1 2024-02-23T00:48:30.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 10 client.41357.objecter close_session for osd.7 2024-02-23T00:48:30.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f8ce412d6a0 msgr2=0x7f8ce412db00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:30.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f8ce412d6a0 0x7f8ce412db00 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:30.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 20 client.41357.objecter put_session s=0x7f8ce412e350 osd=7 1 2024-02-23T00:48:30.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 20 client.41357.objecter shutdown clearing up homeless session... 2024-02-23T00:48:30.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 10 client.41357.objecter successfully canceled tick 2024-02-23T00:48:30.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cea2cb700 1 -- 172.21.15.47:0/442408932 reap_dead start 2024-02-23T00:48:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cea2cb700 1 -- 172.21.15.47:0/442408932 reap_dead start 2024-02-23T00:48:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cea2cb700 1 -- 172.21.15.47:0/442408932 reap_dead start 2024-02-23T00:48:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cea2cb700 1 -- 172.21.15.47:0/442408932 reap_dead start 2024-02-23T00:48:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cea2cb700 1 -- 172.21.15.47:0/442408932 reap_dead start 2024-02-23T00:48:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8cd0062cc0 msgr2=0x7f8cd0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8cd0062cc0 0x7f8cd0065180 secure :-1 s=READY pgs=3068 cs=0 l=1 rev1=1 rx=0x7f8ce4109960 tx=0x7f8cd400b040).stop 2024-02-23T00:48:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8ce40f8cb0 msgr2=0x7f8ce410e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8ce40f8cb0 0x7f8ce410e020 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 rx=0x7f8ce0009da0 tx=0x7f8ce0009cb0).stop 2024-02-23T00:48:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 shutdown_connections 2024-02-23T00:48:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8ce40f8cb0 0x7f8ce410e020 unknown :-1 s=CLOSED pgs=2671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 1 --2- 172.21.15.47:0/442408932 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8cd0062cc0 0x7f8cd0065180 unknown :-1 s=CLOSED pgs=3068 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 >> 172.21.15.47:0/442408932 conn(0x7f8ce40f2820 msgr2=0x7f8ce40f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:48:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 shutdown_connections 2024-02-23T00:48:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 1 -- 172.21.15.47:0/442408932 wait complete. 2024-02-23T00:48:30.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:30.697+0000 7f8cebd2e700 1 librados: shutdown 2024-02-23T00:48:30.710 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:48:30.710 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:48:30.710 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:48:30.710 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:48:30.710 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:48:30.710 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:48:30.710 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:48:30.710 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:48:30.726 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:48:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:48:30] "GET /metrics HTTP/1.1" 200 188971 "" "Prometheus/2.33.4" 2024-02-23T00:48:30.740 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:31.488 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:30.551875+0000 mon.c (mon.2) 301 : audit [DBG] from='client.? 172.21.15.47:0/442408932' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:48:31.488 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:30.551875+0000 mon.c (mon.2) 301 : audit [DBG] from='client.? 172.21.15.47:0/442408932' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:48:31.490 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:48:31.498 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:31.498 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:48:31.514 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset noscrub 2024-02-23T00:48:31.599 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:48:31.629 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:31.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:30.551875+0000 mon.c (mon.2) 301 : audit [DBG] from='client.? 172.21.15.47:0/442408932' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:48:32.599 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:31.291859+0000 mgr.y (mgr.14517) 394 : cluster [DBG] pgmap v624: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:48:32.599 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:48:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:48:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:48:32.779 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:31.291859+0000 mgr.y (mgr.14517) 394 : cluster [DBG] pgmap v624: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:48:32.779 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:31.291859+0000 mgr.y (mgr.14517) 394 : cluster [DBG] pgmap v624: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:48:32.779 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:48:32.783 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:32.783 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:48:32.884 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:48:32.913 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:48:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:48:32.915 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:32.936 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:48:32.939 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:32.940 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:48:33.041 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:48:33.070 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:33.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.485+0000 7f79c9611700 1 -- 172.21.15.47:0/3189148686 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f79c40fdbd0 msgr2=0x7f79c40fdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:33.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.485+0000 7f79c9611700 1 --2- 172.21.15.47:0/3189148686 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f79c40fdbd0 0x7f79c40fdfb0 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 rx=0x7f79b40047b0 tx=0x7f79b4009cb0).stop 2024-02-23T00:48:33.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.485+0000 7f79c9611700 1 -- 172.21.15.47:0/3189148686 shutdown_connections 2024-02-23T00:48:33.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.485+0000 7f79c9611700 1 --2- 172.21.15.47:0/3189148686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79c40f6a90 0x7f79c4104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:33.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.485+0000 7f79c9611700 1 --2- 172.21.15.47:0/3189148686 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f79c40fdbd0 0x7f79c40fdfb0 unknown :-1 s=CLOSED pgs=2672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:33.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.485+0000 7f79c9611700 1 --2- 172.21.15.47:0/3189148686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79c40fe530 0x7f79c40f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:33.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.485+0000 7f79c9611700 1 -- 172.21.15.47:0/3189148686 >> 172.21.15.47:0/3189148686 conn(0x7f79c406c370 msgr2=0x7f79c406c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:48:33.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.485+0000 7f79c9611700 1 -- 172.21.15.47:0/3189148686 shutdown_connections 2024-02-23T00:48:33.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.485+0000 7f79c9611700 1 -- 172.21.15.47:0/3189148686 wait complete. 2024-02-23T00:48:33.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.486+0000 7f79c9611700 1 librados: starting msgr at 2024-02-23T00:48:33.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.486+0000 7f79c9611700 1 librados: starting objecter 2024-02-23T00:48:33.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.486+0000 7f79c9611700 1 Processor -- start 2024-02-23T00:48:33.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.486+0000 7f79c9611700 1 -- start start 2024-02-23T00:48:33.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.486+0000 7f79c9611700 1 librados: setting wanted keys 2024-02-23T00:48:33.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.486+0000 7f79c9611700 1 librados: calling monclient init 2024-02-23T00:48:33.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.486+0000 7f79c9611700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f79c40f6a90 0x7f79c40f7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:33.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.487+0000 7f79c9611700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79c40fdbd0 0x7f79c40f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:33.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.487+0000 7f79c9611700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79c40fe530 0x7f79c40f8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:33.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.487+0000 7f79c9611700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c41164e0 con 0x7f79c40fdbd0 2024-02-23T00:48:33.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.487+0000 7f79c9611700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c4116650 con 0x7f79c40fe530 2024-02-23T00:48:33.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.487+0000 7f79c9611700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c4116810 con 0x7f79c40f6a90 2024-02-23T00:48:33.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.487+0000 7f79c2ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f79c40f6a90 0x7f79c40f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:33.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.487+0000 7f79c2ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f79c40f6a90 0x7f79c40f7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52562/0 (socket says 172.21.15.47:52562) 2024-02-23T00:48:33.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.487+0000 7f79c2ffd700 1 -- 172.21.15.47:0/742241190 learned_addr learned my addr 172.21.15.47:0/742241190 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:48:33.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.487+0000 7f79c27fc700 1 --2- 172.21.15.47:0/742241190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79c40fdbd0 0x7f79c40f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:33.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.487+0000 7f79c37fe700 1 --2- 172.21.15.47:0/742241190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79c40fe530 0x7f79c40f8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:33.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.487+0000 7f79c2ffd700 1 -- 172.21.15.47:0/742241190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79c40fe530 msgr2=0x7f79c40f8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:33.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.487+0000 7f79c2ffd700 1 --2- 172.21.15.47:0/742241190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79c40fe530 0x7f79c40f8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:33.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.487+0000 7f79c2ffd700 1 -- 172.21.15.47:0/742241190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79c40fdbd0 msgr2=0x7f79c40f7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:33.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.487+0000 7f79c2ffd700 1 --2- 172.21.15.47:0/742241190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79c40fdbd0 0x7f79c40f7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:33.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.487+0000 7f79c2ffd700 1 -- 172.21.15.47:0/742241190 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b8005040 con 0x7f79c40f6a90 2024-02-23T00:48:33.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.487+0000 7f79c37fe700 1 --2- 172.21.15.47:0/742241190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79c40fe530 0x7f79c40f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-23T00:48:33.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.487+0000 7f79c2ffd700 1 --2- 172.21.15.47:0/742241190 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f79c40f6a90 0x7f79c40f7680 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 rx=0x7f79b4009f20 tx=0x7f79b4009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:33.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.487+0000 7f79abfff700 10 client.?.objecter ms_handle_connect 0x7f79c40f6a90 2024-02-23T00:48:33.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.487+0000 7f79abfff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:48:33.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.487+0000 7f79abfff700 1 -- 172.21.15.47:0/742241190 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b4018070 con 0x7f79c40f6a90 2024-02-23T00:48:33.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.487+0000 7f79c9611700 1 -- 172.21.15.47:0/742241190 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b4005040 con 0x7f79c40f6a90 2024-02-23T00:48:33.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.487+0000 7f79abfff700 1 -- 172.21.15.47:0/742241190 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f79b40096d0 con 0x7f79c40f6a90 2024-02-23T00:48:33.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.487+0000 7f79abfff700 1 -- 172.21.15.47:0/742241190 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b401dc40 con 0x7f79c40f6a90 2024-02-23T00:48:33.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.487+0000 7f79c9611700 10 client.41363.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:48:33.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.487+0000 7f79c9611700 1 -- 172.21.15.47:0/742241190 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c4116d00 con 0x7f79c40f6a90 2024-02-23T00:48:33.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.487+0000 7f79c9611700 1 librados: init done 2024-02-23T00:48:33.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.488+0000 7f79c9611700 1 -- 172.21.15.47:0/742241190 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79c405f930 con 0x7f79c40f6a90 2024-02-23T00:48:33.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.489+0000 7f79abfff700 1 -- 172.21.15.47:0/742241190 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f79b4009240 con 0x7f79c40f6a90 2024-02-23T00:48:33.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.489+0000 7f79abfff700 1 --2- 172.21.15.47:0/742241190 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f79ac062cc0 0x7f79ac065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:33.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.489+0000 7f79abfff700 1 -- 172.21.15.47:0/742241190 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(429..429 src has 48..429) v4 ==== 9248+0+0 (secure 0 0 0) 0x7f79b407ea10 con 0x7f79c40f6a90 2024-02-23T00:48:33.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.489+0000 7f79abfff700 10 client.41363.objecter ms_dispatch 0x7f79c4001550 osd_map(429..429 src has 48..429) v4 2024-02-23T00:48:33.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.489+0000 7f79abfff700 3 client.41363.objecter handle_osd_map got epochs [429,429] > 0 2024-02-23T00:48:33.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.489+0000 7f79abfff700 3 client.41363.objecter handle_osd_map decoding full epoch 429 2024-02-23T00:48:33.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.489+0000 7f79c27fc700 1 --2- 172.21.15.47:0/742241190 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f79ac062cc0 0x7f79ac065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:33.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.489+0000 7f79abfff700 20 client.41363.objecter dump_active .. 0 homeless 2024-02-23T00:48:33.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.489+0000 7f79abfff700 10 client.41363.objecter ms_handle_connect 0x7f79ac062cc0 2024-02-23T00:48:33.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.489+0000 7f79c27fc700 1 --2- 172.21.15.47:0/742241190 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f79ac062cc0 0x7f79ac065180 secure :-1 s=READY pgs=3069 cs=0 l=1 rev1=1 rx=0x7f79c40f8e20 tx=0x7f79b800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:33.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.493+0000 7f79abfff700 1 -- 172.21.15.47:0/742241190 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f79b404e010 con 0x7f79c40f6a90 2024-02-23T00:48:33.599 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:48:33.600 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:32.597264+0000 mgr.y (mgr.14517) 395 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:48:33.601 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:32.597264+0000 mgr.y (mgr.14517) 395 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:48:33.602 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:33.602 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:48:33.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.661+0000 7f79c9611700 1 -- 172.21.15.47:0/742241190 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f79c40f8bd0 con 0x7f79ac062cc0 2024-02-23T00:48:33.671 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:48:33.672 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-02-23T00:48:33.672 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-02-23T00:48:33.672 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-02-23T00:48:33.672 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:48:33.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.676+0000 7f79abfff700 1 -- 172.21.15.47:0/742241190 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+531023 (secure 0 0 0) 0x7f79c40f8bd0 con 0x7f79ac062cc0 2024-02-23T00:48:33.686 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:48:33.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.680+0000 7f79c9611700 10 librados: watch_flush enter 2024-02-23T00:48:33.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.680+0000 7f79c9611700 10 librados: watch_flush exit 2024-02-23T00:48:33.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.680+0000 7f79c9611700 20 client.41363.objecter shutdown clearing up homeless session... 2024-02-23T00:48:33.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.680+0000 7f79c9611700 10 client.41363.objecter successfully canceled tick 2024-02-23T00:48:33.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.680+0000 7f79c9611700 1 -- 172.21.15.47:0/742241190 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f79ac062cc0 msgr2=0x7f79ac065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:33.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.680+0000 7f79c9611700 1 --2- 172.21.15.47:0/742241190 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f79ac062cc0 0x7f79ac065180 secure :-1 s=READY pgs=3069 cs=0 l=1 rev1=1 rx=0x7f79c40f8e20 tx=0x7f79b800b040).stop 2024-02-23T00:48:33.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.680+0000 7f79c9611700 1 -- 172.21.15.47:0/742241190 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f79c40f6a90 msgr2=0x7f79c40f7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:33.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.680+0000 7f79c9611700 1 --2- 172.21.15.47:0/742241190 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f79c40f6a90 0x7f79c40f7680 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 rx=0x7f79b4009f20 tx=0x7f79b4009cb0).stop 2024-02-23T00:48:33.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.680+0000 7f79c9611700 1 -- 172.21.15.47:0/742241190 shutdown_connections 2024-02-23T00:48:33.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.680+0000 7f79c9611700 1 --2- 172.21.15.47:0/742241190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79c40fe530 0x7f79c40f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:33.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.680+0000 7f79c9611700 1 --2- 172.21.15.47:0/742241190 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f79ac062cc0 0x7f79ac065180 unknown :-1 s=CLOSED pgs=3069 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:33.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.680+0000 7f79c9611700 1 --2- 172.21.15.47:0/742241190 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f79c40f6a90 0x7f79c40f7680 unknown :-1 s=CLOSED pgs=2673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:33.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.680+0000 7f79c9611700 1 --2- 172.21.15.47:0/742241190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79c40fdbd0 0x7f79c40f7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:33.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.680+0000 7f79c9611700 1 -- 172.21.15.47:0/742241190 >> 172.21.15.47:0/742241190 conn(0x7f79c406c370 msgr2=0x7f79c40fb930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:48:33.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.680+0000 7f79c9611700 1 -- 172.21.15.47:0/742241190 shutdown_connections 2024-02-23T00:48:33.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.680+0000 7f79c9611700 1 -- 172.21.15.47:0/742241190 wait complete. 2024-02-23T00:48:33.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:33.680+0000 7f79c9611700 1 librados: shutdown 2024-02-23T00:48:33.693 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-02-23T00:48:33.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:32.597264+0000 mgr.y (mgr.14517) 395 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:48:33.704 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:48:33.732 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:34.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:33.292669+0000 mgr.y (mgr.14517) 396 : cluster [DBG] pgmap v625: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:48:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:33.662699+0000 mgr.y (mgr.14517) 397 : audit [DBG] from='client.41363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:48:34.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:33.292669+0000 mgr.y (mgr.14517) 396 : cluster [DBG] pgmap v625: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:48:34.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:33.662699+0000 mgr.y (mgr.14517) 397 : audit [DBG] from='client.41363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:48:34.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:33.292669+0000 mgr.y (mgr.14517) 396 : cluster [DBG] pgmap v625: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:48:34.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:33.662699+0000 mgr.y (mgr.14517) 397 : audit [DBG] from='client.41363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:48:35.183 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:48:35.186 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:35.187 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:48:35.212 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.2 dump_historic_ops 2024-02-23T00:48:35.288 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:48:35.317 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:35.337 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:48:35.339 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:35.339 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:48:35.440 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:48:35.466 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:34.350421+0000 mon.a (mon.0) 2591 : cluster [DBG] osdmap e430: 8 total, 8 up, 7 in 2024-02-23T00:48:35.466 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:35.190838+0000 mon.a (mon.0) 2592 : cluster [DBG] osdmap e431: 8 total, 8 up, 7 in 2024-02-23T00:48:35.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:34.350421+0000 mon.a (mon.0) 2591 : cluster [DBG] osdmap e430: 8 total, 8 up, 7 in 2024-02-23T00:48:35.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:35.190838+0000 mon.a (mon.0) 2592 : cluster [DBG] osdmap e431: 8 total, 8 up, 7 in 2024-02-23T00:48:35.469 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:35.492 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:48:35.497 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:48:35.498 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:35.498 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:48:35.599 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:48:35.626 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:35.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:34.350421+0000 mon.a (mon.0) 2591 : cluster [DBG] osdmap e430: 8 total, 8 up, 7 in 2024-02-23T00:48:35.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:35.190838+0000 mon.a (mon.0) 2592 : cluster [DBG] osdmap e431: 8 total, 8 up, 7 in 2024-02-23T00:48:35.776 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:48:35.779 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:35.780 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:48:35.881 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:48:35.912 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:35.934 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:48:35.936 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:35.936 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:48:36.038 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:48:36.068 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:36.088 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:48:36.090 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:36.091 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:48:36.192 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:48:36.220 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:36.239 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:48:36.241 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:36.242 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:48:36.343 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:48:36.369 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:36.387 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:48:36.390 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:36.390 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:48:36.412 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":625,"stamp":"2024-02-23T00:48:33.292026+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4007297,"num_objects":387,"num_object_clones":3,"num_object_copies":1161,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":387,"num_whiteouts":0,"num_read":1774,"num_read_kb":1393,"num_write":1235,"num_write_kb":4547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":440,"num_bytes_recovered":3479,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":8,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2650,"ondisk_log_size":2650,"up":624,"acting":624,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":624,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":2167552,"kb_used_data":62372,"kb_used_omap":0,"kb_used_meta":2105088,"kb_avail":654048512,"statfs":{"total":671965249536,"available":669745676288,"internally_reserved":0,"allocated":63868928,"data_stored":43042470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2155610112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7,"num_objects":-2,"num_object_clones":-1,"num_object_copies":-6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":-1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002028"},"pg_stats":[{"pgid":"212.1f","version":"0'0","reported_seq":21,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.324394+0000","last_change":"2024-02-23T00:48:09.280894+0000","last_active":"2024-02-23T00:48:28.324394+0000","last_peered":"2024-02-23T00:48:28.324394+0000","last_clean":"2024-02-23T00:48:28.324394+0000","last_became_active":"2024-02-23T00:48:09.280465+0000","last_became_peered":"2024-02-23T00:48:09.280465+0000","last_unstale":"2024-02-23T00:48:28.324394+0000","last_undegraded":"2024-02-23T00:48:28.324394+0000","last_fullsized":"2024-02-23T00:48:28.324394+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":416,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_clean_scrub_stamp":"2024-02-23T00:48:08.254849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"212.1e","version":"0'0","reported_seq":21,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.466554+0000","last_change":"2024-02-23T00:48:09.286379+0000","last_active":"2024-02-23T00:48:28.466554+0000","last_peered":"2024-02-23T00:48:28.466554+0000","last_clean":"2024-02-23T00:48:28.466554+0000","last_became_active":"2024-02-23T00:48:09.285845+0000","last_became_peered":"2024-02-23T00:48:09.285845+0000","last_unstale":"2024-02-23T00:48:28.466554+0000","last_undegraded":"2024-02-23T00:48:28.466554+0000","last_fullsized":"2024-02-23T00:48:28.466554+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":416,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_clean_scrub_stamp":"2024-02-23T00:48:08.254849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"212.1d","version":"0'0","reported_seq":21,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.464868+0000","last_change":"2024-02-23T00:48:09.271461+0000","last_active":"2024-02-23T00:48:28.464868+0000","last_peered":"2024-02-23T00:48:28.464868+0000","last_clean":"2024-02-23T00:48:28.464868+0000","last_became_active":"2024-02-23T00:48:09.271202+0000","last_became_peered":"2024-02-23T00:48:09.271202+0000","last_unstale":"2024-02-23T00:48:28.464868+0000","last_undegraded":"2024-02-23T00:48:28.464868+0000","last_fullsized":"2024-02-23T00:48:28.464868+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":416,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_clean_scrub_stamp":"2024-02-23T00:48:08.254849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"212.1c","version":"426'4","reported_seq":25,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.465100+0000","last_change":"2024-02-23T00:48:09.281744+0000","last_active":"2024-02-23T00:48:28.465100+0000","last_peered":"2024-02-23T00:48:28.465100+0000","last_clean":"2024-02-23T00:48:28.465100+0000","last_became_active":"2024-02-23T00:48:09.281421+0000","last_became_peered":"2024-02-23T00:48:09.281421+0000","last_unstale":"2024-02-23T00:48:28.465100+0000","last_undegraded":"2024-02-23T00:48:28.465100+0000","last_fullsized":"2024-02-23T00:48:28.465100+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":416,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_clean_scrub_stamp":"2024-02-23T00:48:08.254849+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"212.1b","version":"0'0","reported_seq":21,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.465081+0000","last_change":"2024-02-23T00:48:09.281059+0000","last_active":"2024-02-23T00:48:28.465081+0000","last_peered":"2024-02-23T00:48:28.465081+0000","last_clean":"2024-02-23T00:48:28.465081+0000","last_became_active":"2024-02-23T00:48:09.280712+0000","last_became_peered":"2024-02-23T00:48:09.280712+0000","last_unstale":"2024-02-23T00:48:28.465081+0000","last_undegraded":"2024-02-23T00:48:28.465081+0000","last_fullsized":"2024-02-23T00:48:28.465081+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":416,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_clean_scrub_stamp":"2024-02-23T00:48:08.254849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"212.1a","version":"0'0","reported_seq":21,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.465563+0000","last_change":"2024-02-23T00:48:09.282631+0000","last_active":"2024-02-23T00:48:28.465563+0000","last_peered":"2024-02-23T00:48:28.465563+0000","last_clean":"2024-02-23T00:48:28.465563+0000","last_became_active":"2024-02-23T00:48:09.282098+0000","last_became_peered":"2024-02-23T00:48:09.282098+0000","last_unstale":"2024-02-23T00:48:28.465563+0000","last_undegraded":"2024-02-23T00:48:28.465563+0000","last_fullsized":"2024-02-23T00:48:28.465563+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":416,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_clean_scrub_stamp":"2024-02-23T00:48:08.254849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"212.19","version":"418'2","reported_seq":25,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.467455+0000","last_change":"2024-02-23T00:48:09.276058+0000","last_active":"2024-02-23T00:48:28.467455+0000","last_peered":"2024-02-23T00:48:28.467455+0000","last_clean":"2024-02-23T00:48:28.467455+0000","last_became_active":"2024-02-23T00:48:09.275654+0000","last_became_peered":"2024-02-23T00:48:09.275654+0000","last_unstale":"2024-02-23T00:48:28.467455+0000","last_undegraded":"2024-02-23T00:48:28.467455+0000","last_fullsized":"2024-02-23T00:48:28.467455+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":416,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_clean_scrub_stamp":"2024-02-23T00:48:08.254849+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"212.18","version":"0'0","reported_seq":21,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.464836+0000","last_change":"2024-02-23T00:48:09.275206+0000","last_active":"2024-02-23T00:48:28.464836+0000","last_peered":"2024-02-23T00:48:28.464836+0000","last_clean":"2024-02-23T00:48:28.464836+0000","last_became_active":"2024-02-23T00:48:09.274875+0000","last_became_peered":"2024-02-23T00:48:09.274875+0000","last_unstale":"2024-02-23T00:48:28.464836+0000","last_undegraded":"2024-02-23T00:48:28.464836+0000","last_fullsized":"2024-02-23T00:48:28.464836+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":416,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_clean_scrub_stamp":"2024-02-23T00:48:08.254849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"212.17","version":"0'0","reported_seq":21,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.466937+0000","last_change":"2024-02-23T00:48:09.281130+0000","last_active":"2024-02-23T00:48:28.466937+0000","last_peered":"2024-02-23T00:48:28.466937+0000","last_clean":"2024-02-23T00:48:28.466937+0000","last_became_active":"2024-02-23T00:48:09.280782+0000","last_became_peered":"2024-02-23T00:48:09.280782+0000","last_unstale":"2024-02-23T00:48:28.466937+0000","last_undegraded":"2024-02-23T00:48:28.466937+0000","last_fullsized":"2024-02-23T00:48:28.466937+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":416,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_clean_scrub_stamp":"2024-02-23T00:48:08.254849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"212.16","version":"0'0","reported_seq":21,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.468004+0000","last_change":"2024-02-23T00:48:09.279033+0000","last_active":"2024-02-23T00:48:28.468004+0000","last_peered":"2024-02-23T00:48:28.468004+0000","last_clean":"2024-02-23T00:48:28.468004+0000","last_became_active":"2024-02-23T00:48:09.278693+0000","last_became_peered":"2024-02-23T00:48:09.278693+0000","last_unstale":"2024-02-23T00:48:28.468004+0000","last_undegraded":"2024-02-23T00:48:28.468004+0000","last_fullsized":"2024-02-23T00:48:28.468004+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":416,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_clean_scrub_stamp":"2024-02-23T00:48:08.254849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"212.15","version":"0'0","reported_seq":21,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.324711+0000","last_change":"2024-02-23T00:48:09.274984+0000","last_active":"2024-02-23T00:48:28.324711+0000","last_peered":"2024-02-23T00:48:28.324711+0000","last_clean":"2024-02-23T00:48:28.324711+0000","last_became_active":"2024-02-23T00:48:09.274668+0000","last_became_peered":"2024-02-23T00:48:09.274668+0000","last_unstale":"2024-02-23T00:48:28.324711+0000","last_undegraded":"2024-02-23T00:48:28.324711+0000","last_fullsized":"2024-02-23T00:48:28.324711+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":416,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_clean_scrub_stamp":"2024-02-23T00:48:08.254849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"212.14","version":"0'0","reported_seq":21,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.323970+0000","last_change":"2024-02-23T00:48:09.271564+0000","last_active":"2024-02-23T00:48:28.323970+0000","last_peered":"2024-02-23T00:48:28.323970+0000","last_clean":"2024-02-23T00:48:28.323970+0000","last_became_active":"2024-02-23T00:48:09.271243+0000","last_became_peered":"2024-02-23T00:48:09.271243+0000","last_unstale":"2024-02-23T00:48:28.323970+0000","last_undegraded":"2024-02-23T00:48:28.323970+0000","last_fullsized":"2024-02-23T00:48:28.323970+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":416,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_clean_scrub_stamp":"2024-02-23T00:48:08.254849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"212.13","version":"0'0","reported_seq":21,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.323745+0000","last_change":"2024-02-23T00:48:09.280195+0000","last_active":"2024-02-23T00:48:28.323745+0000","last_peered":"2024-02-23T00:48:28.323745+0000","last_clean":"2024-02-23T00:48:28.323745+0000","last_became_active":"2024-02-23T00:48:09.279866+0000","last_became_peered":"2024-02-23T00:48:09.279866+0000","last_unstale":"2024-02-23T00:48:28.323745+0000","last_undegraded":"2024-02-23T00:48:28.323745+0000","last_fullsized":"2024-02-23T00:48:28.323745+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":416,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_clean_scrub_stamp":"2024-02-23T00:48:08.254849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"212.12","version":"0'0","reported_seq":21,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.465646+0000","last_change":"2024-02-23T00:48:09.282459+0000","last_active":"2024-02-23T00:48:28.465646+0000","last_peered":"2024-02-23T00:48:28.465646+0000","last_clean":"2024-02-23T00:48:28.465646+0000","last_became_active":"2024-02-23T00:48:09.280823+0000","last_became_peered":"2024-02-23T00:48:09.280823+0000","last_unstale":"2024-02-23T00:48:28.465646+0000","last_undegraded":"2024-02-23T00:48:28.465646+0000","last_fullsized":"2024-02-23T00:48:28.465646+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":416,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_clean_scrub_stamp":"2024-02-23T00:48:08.254849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"212.11","version":"0'0","reported_seq":21,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.465810+0000","last_change":"2024-02-23T00:48:09.284668+0000","last_active":"2024-02-23T00:48:28.465810+0000","last_peered":"2024-02-23T00:48:28.465810+0000","last_clean":"2024-02-23T00:48:28.465810+0000","last_became_active":"2024-02-23T00:48:09.284365+0000","last_became_peered":"2024-02-23T00:48:09.284365+0000","last_unstale":"2024-02-23T00:48:28.465810+0000","last_undegraded":"2024-02-23T00:48:28.465810+0000","last_fullsized":"2024-02-23T00:48:28.465810+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":416,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_clean_scrub_stamp":"2024-02-23T00:48:08.254849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"212.10","version":"0'0","reported_seq":21,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.465013+0000","last_change":"2024-02-23T00:48:09.281956+0000","last_active":"2024-02-23T00:48:28.465013+0000","last_peered":"2024-02-23T00:48:28.465013+0000","last_clean":"2024-02-23T00:48:28.465013+0000","last_became_active":"2024-02-23T00:48:09.281636+0000","last_became_peered":"2024-02-23T00:48:09.281636+0000","last_unstale":"2024-02-23T00:48:28.465013+0000","last_undegraded":"2024-02-23T00:48:28.465013+0000","last_fullsized":"2024-02-23T00:48:28.465013+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":416,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_clean_scrub_stamp":"2024-02-23T00:48:08.254849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"212.f","version":"0'0","reported_seq":21,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.468439+0000","last_change":"2024-02-23T00:48:09.277934+0000","last_active":"2024-02-23T00:48:28.468439+0000","last_peered":"2024-02-23T00:48:28.468439+0000","last_clean":"2024-02-23T00:48:28.468439+0000","last_became_active":"2024-02-23T00:48:09.277584+0000","last_became_peered":"2024-02-23T00:48:09.277584+0000","last_unstale":"2024-02-23T00:48:28.468439+0000","last_undegraded":"2024-02-23T00:48:28.468439+0000","last_fullsized":"2024-02-23T00:48:28.468439+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":416,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_clean_scrub_stamp":"2024-02-23T00:48:08.254849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"212.e","version":"0'0","reported_seq":21,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.466608+0000","last_change":"2024-02-23T00:48:09.286692+0000","last_active":"2024-02-23T00:48:28.466608+0000","last_peered":"2024-02-23T00:48:28.466608+0000","last_clean":"2024-02-23T00:48:28.466608+0000","last_became_active":"2024-02-23T00:48:09.286276+0000","last_became_peered":"2024-02-23T00:48:09.286276+0000","last_unstale":"2024-02-23T00:48:28.466608+0000","last_undegraded":"2024-02-23T00:48:28.466608+0000","last_fullsized":"2024-02-23T00:48:28.466608+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":416,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_clean_scrub_stamp":"2024-02-23T00:48:08.254849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"212.d","version":"0'0","reported_seq":21,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.325866+0000","last_change":"2024-02-23T00:48:09.270244+0000","last_active":"2024-02-23T00:48:28.325866+0000","last_peered":"2024-02-23T00:48:28.325866+0000","last_clean":"2024-02-23T00:48:28.325866+0000","last_became_active":"2024-02-23T00:48:09.269893+0000","last_became_peered":"2024-02-23T00:48:09.269893+0000","last_unstale":"2024-02-23T00:48:28.325866+0000","last_undegraded":"2024-02-23T00:48:28.325866+0000","last_fullsized":"2024-02-23T00:48:28.325866+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":416,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_clean_scrub_stamp":"2024-02-23T00:48:08.254849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"212.c","version":"0'0","reported_seq":21,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.323992+0000","last_change":"2024-02-23T00:48:09.280364+0000","last_active":"2024-02-23T00:48:28.323992+0000","last_peered":"2024-02-23T00:48:28.323992+0000","last_clean":"2024-02-23T00:48:28.323992+0000","last_became_active":"2024-02-23T00:48:09.279886+0000","last_became_peered":"2024-02-23T00:48:09.279886+0000","last_unstale":"2024-02-23T00:48:28.323992+0000","last_undegraded":"2024-02-23T00:48:28.323992+0000","last_fullsized":"2024-02-23T00:48:28.323992+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":416,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_clean_scrub_stamp":"2024-02-23T00:48:08.254849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,5],"acting":[4,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"212.b","version":"0'0","reported_seq":21,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.467334+0000","last_change":"2024-02-23T00:48:09.280087+0000","last_active":"2024-02-23T00:48:28.467334+0000","last_peered":"2024-02-23T00:48:28.467334+0000","last_clean":"2024-02-23T00:48:28.467334+0000","last_became_active":"2024-02-23T00:48:09.279787+0000","last_became_peered":"2024-02-23T00:48:09.279787+0000","last_unstale":"2024-02-23T00:48:28.467334+0000","last_undegraded":"2024-02-23T00:48:28.467334+0000","last_fullsized":"2024-02-23T00:48:28.467334+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":416,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_clean_scrub_stamp":"2024-02-23T00:48:08.254849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"212.a","version":"0'0","reported_seq":21,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.465733+0000","last_change":"2024-02-23T00:48:09.282249+0000","last_active":"2024-02-23T00:48:28.465733+0000","last_peered":"2024-02-23T00:48:28.465733+0000","last_clean":"2024-02-23T00:48:28.465733+0000","last_became_active":"2024-02-23T00:48:09.280349+0000","last_became_peered":"2024-02-23T00:48:09.280349+0000","last_unstale":"2024-02-23T00:48:28.465733+0000","last_undegraded":"2024-02-23T00:48:28.465733+0000","last_fullsized":"2024-02-23T00:48:28.465733+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":416,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_clean_scrub_stamp":"2024-02-23T00:48:08.254849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"212.9","version":"0'0","reported_seq":21,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.465413+0000","last_change":"2024-02-23T00:48:09.271969+0000","last_active":"2024-02-23T00:48:28.465413+0000","last_peered":"2024-02-23T00:48:28.465413+0000","last_clean":"2024-02-23T00:48:28.465413+0000","last_became_active":"2024-02-23T00:48:09.271628+0000","last_became_peered":"2024-02-23T00:48:09.271628+0000","last_unstale":"2024-02-23T00:48:28.465413+0000","last_undegraded":"2024-02-23T00:48:28.465413+0000","last_fullsized":"2024-02-23T00:48:28.465413+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":416,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_clean_scrub_stamp":"2024-02-23T00:48:08.254849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"212.8","version":"0'0","reported_seq":21,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.323180+0000","last_change":"2024-02-23T00:48:09.273352+0000","last_active":"2024-02-23T00:48:28.323180+0000","last_peered":"2024-02-23T00:48:28.323180+0000","last_clean":"2024-02-23T00:48:28.323180+0000","last_became_active":"2024-02-23T00:48:09.273049+0000","last_became_peered":"2024-02-23T00:48:09.273049+0000","last_unstale":"2024-02-23T00:48:28.323180+0000","last_undegraded":"2024-02-23T00:48:28.323180+0000","last_fullsized":"2024-02-23T00:48:28.323180+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":416,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_clean_scrub_stamp":"2024-02-23T00:48:08.254849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"212.7","version":"0'0","reported_seq":21,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.324513+0000","last_change":"2024-02-23T00:48:09.281392+0000","last_active":"2024-02-23T00:48:28.324513+0000","last_peered":"2024-02-23T00:48:28.324513+0000","last_clean":"2024-02-23T00:48:28.324513+0000","last_became_active":"2024-02-23T00:48:09.281072+0000","last_became_peered":"2024-02-23T00:48:09.281072+0000","last_unstale":"2024-02-23T00:48:28.324513+0000","last_undegraded":"2024-02-23T00:48:28.324513+0000","last_fullsized":"2024-02-23T00:48:28.324513+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":416,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_clean_scrub_stamp":"2024-02-23T00:48:08.254849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"212.6","version":"0'0","reported_seq":21,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.466668+0000","last_change":"2024-02-23T00:48:09.279534+0000","last_active":"2024-02-23T00:48:28.466668+0000","last_peered":"2024-02-23T00:48:28.466668+0000","last_clean":"2024-02-23T00:48:28.466668+0000","last_became_active":"2024-02-23T00:48:09.279236+0000","last_became_peered":"2024-02-23T00:48:09.279236+0000","last_unstale":"2024-02-23T00:48:28.466668+0000","last_undegraded":"2024-02-23T00:48:28.466668+0000","last_fullsized":"2024-02-23T00:48:28.466668+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":416,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_clean_scrub_stamp":"2024-02-23T00:48:08.254849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"212.5","version":"0'0","reported_seq":21,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.464730+0000","last_change":"2024-02-23T00:48:09.274129+0000","last_active":"2024-02-23T00:48:28.464730+0000","last_peered":"2024-02-23T00:48:28.464730+0000","last_clean":"2024-02-23T00:48:28.464730+0000","last_became_active":"2024-02-23T00:48:09.273743+0000","last_became_peered":"2024-02-23T00:48:09.273743+0000","last_unstale":"2024-02-23T00:48:28.464730+0000","last_undegraded":"2024-02-23T00:48:28.464730+0000","last_fullsized":"2024-02-23T00:48:28.464730+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":416,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_clean_scrub_stamp":"2024-02-23T00:48:08.254849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"212.4","version":"418'1","reported_seq":22,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.327278+0000","last_change":"2024-02-23T00:48:09.275089+0000","last_active":"2024-02-23T00:48:28.327278+0000","last_peered":"2024-02-23T00:48:28.327278+0000","last_clean":"2024-02-23T00:48:28.327278+0000","last_became_active":"2024-02-23T00:48:09.274798+0000","last_became_peered":"2024-02-23T00:48:09.274798+0000","last_unstale":"2024-02-23T00:48:28.327278+0000","last_undegraded":"2024-02-23T00:48:28.327278+0000","last_fullsized":"2024-02-23T00:48:28.327278+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":416,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_clean_scrub_stamp":"2024-02-23T00:48:08.254849+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"212.3","version":"426'5","reported_seq":29,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.323987+0000","last_change":"2024-02-23T00:48:09.280367+0000","last_active":"2024-02-23T00:48:28.323987+0000","last_peered":"2024-02-23T00:48:28.323987+0000","last_clean":"2024-02-23T00:48:28.323987+0000","last_became_active":"2024-02-23T00:48:09.280093+0000","last_became_peered":"2024-02-23T00:48:09.280093+0000","last_unstale":"2024-02-23T00:48:28.323987+0000","last_undegraded":"2024-02-23T00:48:28.323987+0000","last_fullsized":"2024-02-23T00:48:28.323987+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":416,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_clean_scrub_stamp":"2024-02-23T00:48:08.254849+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"212.2","version":"0'0","reported_seq":21,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.464424+0000","last_change":"2024-02-23T00:48:09.271841+0000","last_active":"2024-02-23T00:48:28.464424+0000","last_peered":"2024-02-23T00:48:28.464424+0000","last_clean":"2024-02-23T00:48:28.464424+0000","last_became_active":"2024-02-23T00:48:09.271557+0000","last_became_peered":"2024-02-23T00:48:09.271557+0000","last_unstale":"2024-02-23T00:48:28.464424+0000","last_undegraded":"2024-02-23T00:48:28.464424+0000","last_fullsized":"2024-02-23T00:48:28.464424+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":416,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_clean_scrub_stamp":"2024-02-23T00:48:08.254849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"212.1","version":"0'0","reported_seq":21,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.465705+0000","last_change":"2024-02-23T00:48:09.283401+0000","last_active":"2024-02-23T00:48:28.465705+0000","last_peered":"2024-02-23T00:48:28.465705+0000","last_clean":"2024-02-23T00:48:28.465705+0000","last_became_active":"2024-02-23T00:48:09.283075+0000","last_became_peered":"2024-02-23T00:48:09.283075+0000","last_unstale":"2024-02-23T00:48:28.465705+0000","last_undegraded":"2024-02-23T00:48:28.465705+0000","last_fullsized":"2024-02-23T00:48:28.465705+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":416,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_clean_scrub_stamp":"2024-02-23T00:48:08.254849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"212.0","version":"0'0","reported_seq":21,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.323266+0000","last_change":"2024-02-23T00:48:09.273033+0000","last_active":"2024-02-23T00:48:28.323266+0000","last_peered":"2024-02-23T00:48:28.323266+0000","last_clean":"2024-02-23T00:48:28.323266+0000","last_became_active":"2024-02-23T00:48:09.272740+0000","last_became_peered":"2024-02-23T00:48:09.272740+0000","last_unstale":"2024-02-23T00:48:28.323266+0000","last_undegraded":"2024-02-23T00:48:28.323266+0000","last_fullsized":"2024-02-23T00:48:28.323266+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":416,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:08.254849+0000","last_clean_scrub_stamp":"2024-02-23T00:48:08.254849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.0","version":"259'40","reported_seq":445,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.285765+0000","last_change":"2024-02-23T00:48:24.330350+0000","last_active":"2024-02-23T00:48:30.285765+0000","last_peered":"2024-02-23T00:48:30.285765+0000","last_clean":"2024-02-23T00:48:30.285765+0000","last_became_active":"2024-02-23T00:47:28.238034+0000","last_became_peered":"2024-02-23T00:47:28.238034+0000","last_unstale":"2024-02-23T00:48:30.285765+0000","last_undegraded":"2024-02-23T00:48:30.285765+0000","last_fullsized":"2024-02-23T00:48:30.285765+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'40","last_scrub_stamp":"2024-02-23T00:46:06.777995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:06.777995+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":31,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":1,"num_write":13,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":31,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.1","version":"230'87","reported_seq":445,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.283354+0000","last_change":"2024-02-23T00:48:24.324946+0000","last_active":"2024-02-23T00:48:30.283354+0000","last_peered":"2024-02-23T00:48:30.283354+0000","last_clean":"2024-02-23T00:48:30.283354+0000","last_became_active":"2024-02-23T00:43:17.851866+0000","last_became_peered":"2024-02-23T00:43:17.851866+0000","last_unstale":"2024-02-23T00:48:30.283354+0000","last_undegraded":"2024-02-23T00:48:30.283354+0000","last_fullsized":"2024-02-23T00:48:30.283354+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"230'87","last_scrub_stamp":"2024-02-23T00:47:16.460312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:47:16.460312+0000","log_size":87,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":63,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.2","version":"229'68","reported_seq":430,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.288134+0000","last_change":"2024-02-23T00:48:24.329097+0000","last_active":"2024-02-23T00:48:30.288134+0000","last_peered":"2024-02-23T00:48:30.288134+0000","last_clean":"2024-02-23T00:48:30.288134+0000","last_became_active":"2024-02-23T00:43:17.804792+0000","last_became_peered":"2024-02-23T00:43:17.804792+0000","last_unstale":"2024-02-23T00:48:30.288134+0000","last_undegraded":"2024-02-23T00:48:30.288134+0000","last_fullsized":"2024-02-23T00:48:30.288134+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'68","last_scrub_stamp":"2024-02-23T00:48:04.587882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:48:04.587882+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.3","version":"246'20","reported_seq":267,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.284151+0000","last_change":"2024-02-23T00:48:24.326241+0000","last_active":"2024-02-23T00:48:30.284151+0000","last_peered":"2024-02-23T00:48:30.284151+0000","last_clean":"2024-02-23T00:48:30.284151+0000","last_became_active":"2024-02-23T00:47:28.226936+0000","last_became_peered":"2024-02-23T00:47:28.226936+0000","last_unstale":"2024-02-23T00:48:30.284151+0000","last_undegraded":"2024-02-23T00:48:30.284151+0000","last_fullsized":"2024-02-23T00:48:30.284151+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"246'20","last_scrub_stamp":"2024-02-23T00:45:53.067737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:53.067737+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":24,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.4","version":"207'17","reported_seq":382,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.191148+0000","last_change":"2024-02-23T00:48:24.322046+0000","last_active":"2024-02-23T00:48:30.191148+0000","last_peered":"2024-02-23T00:48:30.191148+0000","last_clean":"2024-02-23T00:48:30.191148+0000","last_became_active":"2024-02-23T00:43:17.810343+0000","last_became_peered":"2024-02-23T00:43:17.810343+0000","last_unstale":"2024-02-23T00:48:30.191148+0000","last_undegraded":"2024-02-23T00:48:30.191148+0000","last_fullsized":"2024-02-23T00:48:30.191148+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"207'17","last_scrub_stamp":"2024-02-23T00:46:35.627480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:35.627480+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.5","version":"237'51","reported_seq":416,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.189602+0000","last_change":"2024-02-23T00:48:24.340870+0000","last_active":"2024-02-23T00:48:30.189602+0000","last_peered":"2024-02-23T00:48:30.189602+0000","last_clean":"2024-02-23T00:48:30.189602+0000","last_became_active":"2024-02-23T00:43:17.855339+0000","last_became_peered":"2024-02-23T00:43:17.855339+0000","last_unstale":"2024-02-23T00:48:30.189602+0000","last_undegraded":"2024-02-23T00:48:30.189602+0000","last_fullsized":"2024-02-23T00:48:30.189602+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'51","last_scrub_stamp":"2024-02-23T00:46:40.379159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:40.379159+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.6","version":"379'54","reported_seq":473,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.286705+0000","last_change":"2024-02-23T00:48:24.331179+0000","last_active":"2024-02-23T00:48:30.286705+0000","last_peered":"2024-02-23T00:48:30.286705+0000","last_clean":"2024-02-23T00:48:30.286705+0000","last_became_active":"2024-02-23T00:47:28.236471+0000","last_became_peered":"2024-02-23T00:47:28.236471+0000","last_unstale":"2024-02-23T00:48:30.286705+0000","last_undegraded":"2024-02-23T00:48:30.286705+0000","last_fullsized":"2024-02-23T00:48:30.286705+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'52","last_scrub_stamp":"2024-02-23T00:47:26.428479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:47:26.428479+0000","log_size":54,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":18,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":41,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.7","version":"347'69","reported_seq":450,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.289519+0000","last_change":"2024-02-23T00:48:24.328992+0000","last_active":"2024-02-23T00:48:30.289519+0000","last_peered":"2024-02-23T00:48:30.289519+0000","last_clean":"2024-02-23T00:48:30.289519+0000","last_became_active":"2024-02-23T00:47:28.240920+0000","last_became_peered":"2024-02-23T00:47:28.240920+0000","last_unstale":"2024-02-23T00:48:30.289519+0000","last_undegraded":"2024-02-23T00:48:30.289519+0000","last_fullsized":"2024-02-23T00:48:30.289519+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'69","last_scrub_stamp":"2024-02-23T00:47:54.430466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:47:54.430466+0000","log_size":69,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":52,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":21,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":52,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.8","version":"410'98","reported_seq":520,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.285500+0000","last_change":"2024-02-23T00:48:24.330083+0000","last_active":"2024-02-23T00:48:30.285500+0000","last_peered":"2024-02-23T00:48:30.285500+0000","last_clean":"2024-02-23T00:48:30.285500+0000","last_became_active":"2024-02-23T00:47:28.229780+0000","last_became_peered":"2024-02-23T00:47:28.229780+0000","last_unstale":"2024-02-23T00:48:30.285500+0000","last_undegraded":"2024-02-23T00:48:30.285500+0000","last_fullsized":"2024-02-23T00:48:30.285500+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'90","last_scrub_stamp":"2024-02-23T00:46:05.820651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:05.820651+0000","log_size":98,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":78,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":33,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":68,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.9","version":"259'39","reported_seq":443,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.188427+0000","last_change":"2024-02-23T00:48:24.339290+0000","last_active":"2024-02-23T00:48:30.188427+0000","last_peered":"2024-02-23T00:48:30.188427+0000","last_clean":"2024-02-23T00:48:30.188427+0000","last_became_active":"2024-02-23T00:47:28.202025+0000","last_became_peered":"2024-02-23T00:47:28.202025+0000","last_unstale":"2024-02-23T00:48:30.188427+0000","last_undegraded":"2024-02-23T00:48:30.188427+0000","last_fullsized":"2024-02-23T00:48:30.188427+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'39","last_scrub_stamp":"2024-02-23T00:46:10.439373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:10.439373+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":12,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":27,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.a","version":"347'91","reported_seq":518,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.286747+0000","last_change":"2024-02-23T00:48:24.329805+0000","last_active":"2024-02-23T00:48:30.286747+0000","last_peered":"2024-02-23T00:48:30.286747+0000","last_clean":"2024-02-23T00:48:30.286747+0000","last_became_active":"2024-02-23T00:47:28.241187+0000","last_became_peered":"2024-02-23T00:47:28.241187+0000","last_unstale":"2024-02-23T00:48:30.286747+0000","last_undegraded":"2024-02-23T00:48:30.286747+0000","last_fullsized":"2024-02-23T00:48:30.286747+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'91","last_scrub_stamp":"2024-02-23T00:47:25.464279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:47:25.464279+0000","log_size":91,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":68,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":28,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":68,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.b","version":"418'84","reported_seq":445,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.288539+0000","last_change":"2024-02-23T00:48:24.330763+0000","last_active":"2024-02-23T00:48:30.288539+0000","last_peered":"2024-02-23T00:48:30.288539+0000","last_clean":"2024-02-23T00:48:30.288539+0000","last_became_active":"2024-02-23T00:43:17.855321+0000","last_became_peered":"2024-02-23T00:43:17.855321+0000","last_unstale":"2024-02-23T00:48:30.288539+0000","last_undegraded":"2024-02-23T00:48:30.288539+0000","last_fullsized":"2024-02-23T00:48:30.288539+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"418'84","last_scrub_stamp":"2024-02-23T00:48:10.926499+0000","last_deep_scrub":"418'84","last_deep_scrub_stamp":"2024-02-23T00:48:10.926499+0000","last_clean_scrub_stamp":"2024-02-23T00:48:10.926499+0000","log_size":84,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":54,"num_objects":7,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":24,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.c","version":"401'36","reported_seq":401,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.192989+0000","last_change":"2024-02-23T00:48:24.320816+0000","last_active":"2024-02-23T00:48:30.192989+0000","last_peered":"2024-02-23T00:48:30.192989+0000","last_clean":"2024-02-23T00:48:30.192989+0000","last_became_active":"2024-02-23T00:43:17.829044+0000","last_became_peered":"2024-02-23T00:43:17.829044+0000","last_unstale":"2024-02-23T00:48:30.192989+0000","last_undegraded":"2024-02-23T00:48:30.192989+0000","last_fullsized":"2024-02-23T00:48:30.192989+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'34","last_scrub_stamp":"2024-02-23T00:45:57.874883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:57.874883+0000","log_size":36,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":12,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.d","version":"414'58","reported_seq":460,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.189361+0000","last_change":"2024-02-23T00:48:24.340793+0000","last_active":"2024-02-23T00:48:30.189361+0000","last_peered":"2024-02-23T00:48:30.189361+0000","last_clean":"2024-02-23T00:48:30.189361+0000","last_became_active":"2024-02-23T00:47:28.202691+0000","last_became_peered":"2024-02-23T00:47:28.202691+0000","last_unstale":"2024-02-23T00:48:30.189361+0000","last_undegraded":"2024-02-23T00:48:30.189361+0000","last_fullsized":"2024-02-23T00:48:30.189361+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'56","last_scrub_stamp":"2024-02-23T00:45:51.258745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:51.258745+0000","log_size":58,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":18,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":33,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.e","version":"211'51","reported_seq":414,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.190416+0000","last_change":"2024-02-23T00:48:24.321620+0000","last_active":"2024-02-23T00:48:30.190416+0000","last_peered":"2024-02-23T00:48:30.190416+0000","last_clean":"2024-02-23T00:48:30.190416+0000","last_became_active":"2024-02-23T00:43:17.845956+0000","last_became_peered":"2024-02-23T00:43:17.845956+0000","last_unstale":"2024-02-23T00:48:30.190416+0000","last_undegraded":"2024-02-23T00:48:30.190416+0000","last_fullsized":"2024-02-23T00:48:30.190416+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'51","last_scrub_stamp":"2024-02-23T00:48:18.902135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:48:18.902135+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.f","version":"241'136","reported_seq":678,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.190210+0000","last_change":"2024-02-23T00:48:24.341141+0000","last_active":"2024-02-23T00:48:30.190210+0000","last_peered":"2024-02-23T00:48:30.190210+0000","last_clean":"2024-02-23T00:48:30.190210+0000","last_became_active":"2024-02-23T00:47:47.183032+0000","last_became_peered":"2024-02-23T00:47:47.183032+0000","last_unstale":"2024-02-23T00:48:30.190210+0000","last_undegraded":"2024-02-23T00:48:30.190210+0000","last_fullsized":"2024-02-23T00:48:30.190210+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"241'136","last_scrub_stamp":"2024-02-23T00:46:35.318571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:35.318571+0000","log_size":136,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":88,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":40,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.10","version":"347'69","reported_seq":427,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.283355+0000","last_change":"2024-02-23T00:48:24.325380+0000","last_active":"2024-02-23T00:48:30.283355+0000","last_peered":"2024-02-23T00:48:30.283355+0000","last_clean":"2024-02-23T00:48:30.283355+0000","last_became_active":"2024-02-23T00:43:17.819865+0000","last_became_peered":"2024-02-23T00:43:17.819865+0000","last_unstale":"2024-02-23T00:48:30.283355+0000","last_undegraded":"2024-02-23T00:48:30.283355+0000","last_fullsized":"2024-02-23T00:48:30.283355+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'69","last_scrub_stamp":"2024-02-23T00:47:15.414418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:47:15.414418+0000","log_size":69,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":52,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":21,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.11","version":"347'53","reported_seq":408,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.283341+0000","last_change":"2024-02-23T00:48:24.324648+0000","last_active":"2024-02-23T00:48:30.283341+0000","last_peered":"2024-02-23T00:48:30.283341+0000","last_clean":"2024-02-23T00:48:30.283341+0000","last_became_active":"2024-02-23T00:43:17.852697+0000","last_became_peered":"2024-02-23T00:43:17.852697+0000","last_unstale":"2024-02-23T00:48:30.283341+0000","last_undegraded":"2024-02-23T00:48:30.283341+0000","last_fullsized":"2024-02-23T00:48:30.283341+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'53","last_scrub_stamp":"2024-02-23T00:47:11.380987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:47:11.380987+0000","log_size":53,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":49,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":17,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.12","version":"240'73","reported_seq":340,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.188439+0000","last_change":"2024-02-23T00:48:24.320722+0000","last_active":"2024-02-23T00:48:30.188439+0000","last_peered":"2024-02-23T00:48:30.188439+0000","last_clean":"2024-02-23T00:48:30.188439+0000","last_became_active":"2024-02-23T00:47:28.213692+0000","last_became_peered":"2024-02-23T00:47:28.213692+0000","last_unstale":"2024-02-23T00:48:30.188439+0000","last_undegraded":"2024-02-23T00:48:30.188439+0000","last_fullsized":"2024-02-23T00:48:30.188439+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'73","last_scrub_stamp":"2024-02-23T00:48:14.866255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:48:14.866255+0000","log_size":73,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":49,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":49,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.13","version":"426'57","reported_seq":427,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.192055+0000","last_change":"2024-02-23T00:48:26.324430+0000","last_active":"2024-02-23T00:48:30.192055+0000","last_peered":"2024-02-23T00:48:30.192055+0000","last_clean":"2024-02-23T00:48:30.192055+0000","last_became_active":"2024-02-23T00:43:17.857823+0000","last_became_peered":"2024-02-23T00:43:17.857823+0000","last_unstale":"2024-02-23T00:48:30.192055+0000","last_undegraded":"2024-02-23T00:48:30.192055+0000","last_fullsized":"2024-02-23T00:48:30.192055+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"422'55","last_scrub_stamp":"2024-02-23T00:48:15.865963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:48:15.865963+0000","log_size":57,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":40,"num_objects":5,"num_object_clones":1,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":19,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.14","version":"420'54","reported_seq":468,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.284458+0000","last_change":"2024-02-23T00:48:24.330694+0000","last_active":"2024-02-23T00:48:30.284458+0000","last_peered":"2024-02-23T00:48:30.284458+0000","last_clean":"2024-02-23T00:48:30.284458+0000","last_became_active":"2024-02-23T00:47:28.235465+0000","last_became_peered":"2024-02-23T00:47:28.235465+0000","last_unstale":"2024-02-23T00:48:30.284458+0000","last_undegraded":"2024-02-23T00:48:30.284458+0000","last_fullsized":"2024-02-23T00:48:30.284458+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"420'54","last_scrub_stamp":"2024-02-23T00:48:12.892464+0000","last_deep_scrub":"420'54","last_deep_scrub_stamp":"2024-02-23T00:48:12.892464+0000","last_clean_scrub_stamp":"2024-02-23T00:48:12.892464+0000","log_size":54,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":59,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":51,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.15","version":"347'19","reported_seq":411,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.192122+0000","last_change":"2024-02-23T00:48:24.319381+0000","last_active":"2024-02-23T00:48:30.192122+0000","last_peered":"2024-02-23T00:48:30.192122+0000","last_clean":"2024-02-23T00:48:30.192122+0000","last_became_active":"2024-02-23T00:47:28.219548+0000","last_became_peered":"2024-02-23T00:47:28.219548+0000","last_unstale":"2024-02-23T00:48:30.192122+0000","last_undegraded":"2024-02-23T00:48:30.192122+0000","last_fullsized":"2024-02-23T00:48:30.192122+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"299'18","last_scrub_stamp":"2024-02-23T00:46:11.830580+0000","last_deep_scrub":"299'18","last_deep_scrub_stamp":"2024-02-23T00:46:11.830580+0000","last_clean_scrub_stamp":"2024-02-23T00:46:11.830580+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.16","version":"407'30","reported_seq":435,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.189956+0000","last_change":"2024-02-23T00:48:24.341028+0000","last_active":"2024-02-23T00:48:30.189956+0000","last_peered":"2024-02-23T00:48:30.189956+0000","last_clean":"2024-02-23T00:48:30.189956+0000","last_became_active":"2024-02-23T00:47:28.199870+0000","last_became_peered":"2024-02-23T00:47:28.199870+0000","last_unstale":"2024-02-23T00:48:30.189956+0000","last_undegraded":"2024-02-23T00:48:30.189956+0000","last_fullsized":"2024-02-23T00:48:30.189956+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'23","last_scrub_stamp":"2024-02-23T00:45:56.354881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:56.354881+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":64,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":14,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.17","version":"375'63","reported_seq":434,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.289769+0000","last_change":"2024-02-23T00:48:24.329443+0000","last_active":"2024-02-23T00:48:30.289769+0000","last_peered":"2024-02-23T00:48:30.289769+0000","last_clean":"2024-02-23T00:48:30.289769+0000","last_became_active":"2024-02-23T00:43:17.843525+0000","last_became_peered":"2024-02-23T00:43:17.843525+0000","last_unstale":"2024-02-23T00:48:30.289769+0000","last_undegraded":"2024-02-23T00:48:30.289769+0000","last_fullsized":"2024-02-23T00:48:30.289769+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'63","last_scrub_stamp":"2024-02-23T00:47:27.229920+0000","last_deep_scrub":"375'63","last_deep_scrub_stamp":"2024-02-23T00:47:27.229920+0000","last_clean_scrub_stamp":"2024-02-23T00:47:27.229920+0000","log_size":63,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":20,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.19","version":"267'96","reported_seq":533,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.286731+0000","last_change":"2024-02-23T00:48:24.328640+0000","last_active":"2024-02-23T00:48:30.286731+0000","last_peered":"2024-02-23T00:48:30.286731+0000","last_clean":"2024-02-23T00:48:30.286731+0000","last_became_active":"2024-02-23T00:47:28.210660+0000","last_became_peered":"2024-02-23T00:47:28.210660+0000","last_unstale":"2024-02-23T00:48:30.286731+0000","last_undegraded":"2024-02-23T00:48:30.286731+0000","last_fullsized":"2024-02-23T00:48:30.286731+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'96","last_scrub_stamp":"2024-02-23T00:47:55.479671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:47:55.479671+0000","log_size":96,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":73,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":7,"num_read_kb":0,"num_write":30,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":73,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,7],"acting":[6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.18","version":"410'12","reported_seq":403,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.188130+0000","last_change":"2024-02-23T00:48:24.319311+0000","last_active":"2024-02-23T00:48:30.188130+0000","last_peered":"2024-02-23T00:48:30.188130+0000","last_clean":"2024-02-23T00:48:30.188130+0000","last_became_active":"2024-02-23T00:47:28.221906+0000","last_became_peered":"2024-02-23T00:47:28.221906+0000","last_unstale":"2024-02-23T00:48:30.188130+0000","last_undegraded":"2024-02-23T00:48:30.188130+0000","last_fullsized":"2024-02-23T00:48:30.188130+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"275'7","last_scrub_stamp":"2024-02-23T00:46:03.927783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:03.927783+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":15,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":5,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1b","version":"347'70","reported_seq":428,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.189130+0000","last_change":"2024-02-23T00:48:24.339930+0000","last_active":"2024-02-23T00:48:30.189130+0000","last_peered":"2024-02-23T00:48:30.189130+0000","last_clean":"2024-02-23T00:48:30.189130+0000","last_became_active":"2024-02-23T00:43:17.803043+0000","last_became_peered":"2024-02-23T00:43:17.803043+0000","last_unstale":"2024-02-23T00:48:30.189130+0000","last_undegraded":"2024-02-23T00:48:30.189130+0000","last_fullsized":"2024-02-23T00:48:30.189130+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"304'69","last_scrub_stamp":"2024-02-23T00:46:37.397467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.397467+0000","log_size":70,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4000060,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":22,"num_write_kb":3928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.1a","version":"347'35","reported_seq":431,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.288097+0000","last_change":"2024-02-23T00:48:24.327865+0000","last_active":"2024-02-23T00:48:30.288097+0000","last_peered":"2024-02-23T00:48:30.288097+0000","last_clean":"2024-02-23T00:48:30.288097+0000","last_became_active":"2024-02-23T00:45:44.178195+0000","last_became_peered":"2024-02-23T00:45:44.178195+0000","last_unstale":"2024-02-23T00:48:30.288097+0000","last_undegraded":"2024-02-23T00:48:30.288097+0000","last_fullsized":"2024-02-23T00:48:30.288097+0000","mapping_epoch":271,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":272,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'35","last_scrub_stamp":"2024-02-23T00:47:17.368393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:47:17.368393+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":30,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":11,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1d","version":"215'104","reported_seq":459,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.284189+0000","last_change":"2024-02-23T00:48:24.319151+0000","last_active":"2024-02-23T00:48:30.284189+0000","last_peered":"2024-02-23T00:48:30.284189+0000","last_clean":"2024-02-23T00:48:30.284189+0000","last_became_active":"2024-02-23T00:43:17.828194+0000","last_became_peered":"2024-02-23T00:43:17.828194+0000","last_unstale":"2024-02-23T00:48:30.284189+0000","last_undegraded":"2024-02-23T00:48:30.284189+0000","last_fullsized":"2024-02-23T00:48:30.284189+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'104","last_scrub_stamp":"2024-02-23T00:46:05.000615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:05.000615+0000","log_size":104,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":74,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":32,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"62.1c","version":"374'55","reported_seq":418,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.288534+0000","last_change":"2024-02-23T00:48:24.328328+0000","last_active":"2024-02-23T00:48:30.288534+0000","last_peered":"2024-02-23T00:48:30.288534+0000","last_clean":"2024-02-23T00:48:30.288534+0000","last_became_active":"2024-02-23T00:43:17.804718+0000","last_became_peered":"2024-02-23T00:43:17.804718+0000","last_unstale":"2024-02-23T00:48:30.288534+0000","last_undegraded":"2024-02-23T00:48:30.288534+0000","last_fullsized":"2024-02-23T00:48:30.288534+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"374'55","last_scrub_stamp":"2024-02-23T00:47:26.267291+0000","last_deep_scrub":"374'55","last_deep_scrub_stamp":"2024-02-23T00:47:26.267291+0000","last_clean_scrub_stamp":"2024-02-23T00:47:26.267291+0000","log_size":55,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1f","version":"414'43","reported_seq":426,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.287538+0000","last_change":"2024-02-23T00:48:24.331257+0000","last_active":"2024-02-23T00:48:30.287538+0000","last_peered":"2024-02-23T00:48:30.287538+0000","last_clean":"2024-02-23T00:48:30.287538+0000","last_became_active":"2024-02-23T00:47:28.243574+0000","last_became_peered":"2024-02-23T00:47:28.243574+0000","last_unstale":"2024-02-23T00:48:30.287538+0000","last_undegraded":"2024-02-23T00:48:30.287538+0000","last_fullsized":"2024-02-23T00:48:30.287538+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"414'43","last_scrub_stamp":"2024-02-23T00:48:09.898586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:48:09.898586+0000","log_size":43,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":43,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":15,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":35,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.1e","version":"245'51","reported_seq":452,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.286486+0000","last_change":"2024-02-23T00:48:24.331488+0000","last_active":"2024-02-23T00:48:30.286486+0000","last_peered":"2024-02-23T00:48:30.286486+0000","last_clean":"2024-02-23T00:48:30.286486+0000","last_became_active":"2024-02-23T00:47:28.244073+0000","last_became_peered":"2024-02-23T00:47:28.244073+0000","last_unstale":"2024-02-23T00:48:30.286486+0000","last_undegraded":"2024-02-23T00:48:30.286486+0000","last_fullsized":"2024-02-23T00:48:30.286486+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"245'51","last_scrub_stamp":"2024-02-23T00:45:58.723990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:58.723990+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":33,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"67'8","reported_seq":513,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.466405+0000","last_change":"2024-02-23T00:47:23.486066+0000","last_active":"2024-02-23T00:48:28.466405+0000","last_peered":"2024-02-23T00:48:28.466405+0000","last_clean":"2024-02-23T00:48:28.466405+0000","last_became_active":"2024-02-23T00:39:18.793414+0000","last_became_peered":"2024-02-23T00:39:18.793414+0000","last_unstale":"2024-02-23T00:48:28.466405+0000","last_undegraded":"2024-02-23T00:48:28.466405+0000","last_fullsized":"2024-02-23T00:48:28.466405+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:47:23.485949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:47:23.485949+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"68'27","reported_seq":449,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.465323+0000","last_change":"2024-02-23T00:47:20.244368+0000","last_active":"2024-02-23T00:48:28.465323+0000","last_peered":"2024-02-23T00:48:28.465323+0000","last_clean":"2024-02-23T00:48:28.465323+0000","last_became_active":"2024-02-23T00:39:16.859401+0000","last_became_peered":"2024-02-23T00:39:16.859401+0000","last_unstale":"2024-02-23T00:48:28.465323+0000","last_undegraded":"2024-02-23T00:48:28.465323+0000","last_fullsized":"2024-02-23T00:48:28.465323+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'27","last_scrub_stamp":"2024-02-23T00:47:20.244299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:47:20.244299+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":65,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.325347+0000","last_change":"2024-02-23T00:47:28.221354+0000","last_active":"2024-02-23T00:48:28.325347+0000","last_peered":"2024-02-23T00:48:28.325347+0000","last_clean":"2024-02-23T00:48:28.325347+0000","last_became_active":"2024-02-23T00:47:28.220876+0000","last_became_peered":"2024-02-23T00:47:28.220876+0000","last_unstale":"2024-02-23T00:48:28.325347+0000","last_undegraded":"2024-02-23T00:48:28.325347+0000","last_fullsized":"2024-02-23T00:48:28.325347+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.942091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:04.330426+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.942091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":408,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.465165+0000","last_change":"2024-02-23T00:47:11.224449+0000","last_active":"2024-02-23T00:48:28.465165+0000","last_peered":"2024-02-23T00:48:28.465165+0000","last_clean":"2024-02-23T00:48:28.465165+0000","last_became_active":"2024-02-23T00:43:32.282545+0000","last_became_peered":"2024-02-23T00:43:32.282545+0000","last_unstale":"2024-02-23T00:48:28.465165+0000","last_undegraded":"2024-02-23T00:48:28.465165+0000","last_fullsized":"2024-02-23T00:48:28.465165+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:11.224386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:47:11.224386+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":337,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.468155+0000","last_change":"2024-02-23T00:48:02.292630+0000","last_active":"2024-02-23T00:48:28.468155+0000","last_peered":"2024-02-23T00:48:28.468155+0000","last_clean":"2024-02-23T00:48:28.468155+0000","last_became_active":"2024-02-23T00:42:58.828846+0000","last_became_peered":"2024-02-23T00:42:58.828846+0000","last_unstale":"2024-02-23T00:48:28.468155+0000","last_undegraded":"2024-02-23T00:48:28.468155+0000","last_fullsized":"2024-02-23T00:48:28.468155+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:02.292555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:02.292555+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"68'15","reported_seq":426,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.326820+0000","last_change":"2024-02-23T00:46:45.625235+0000","last_active":"2024-02-23T00:48:28.326820+0000","last_peered":"2024-02-23T00:48:28.326820+0000","last_clean":"2024-02-23T00:48:28.326820+0000","last_became_active":"2024-02-23T00:39:16.783701+0000","last_became_peered":"2024-02-23T00:39:16.783701+0000","last_unstale":"2024-02-23T00:48:28.326820+0000","last_undegraded":"2024-02-23T00:48:28.326820+0000","last_fullsized":"2024-02-23T00:48:28.326820+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:46:45.625126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:45.625126+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":390,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.465109+0000","last_change":"2024-02-23T00:46:04.476277+0000","last_active":"2024-02-23T00:48:28.465109+0000","last_peered":"2024-02-23T00:48:28.465109+0000","last_clean":"2024-02-23T00:48:28.465109+0000","last_became_active":"2024-02-23T00:39:14.772493+0000","last_became_peered":"2024-02-23T00:39:14.772493+0000","last_unstale":"2024-02-23T00:48:28.465109+0000","last_undegraded":"2024-02-23T00:48:28.465109+0000","last_fullsized":"2024-02-23T00:48:28.465109+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.476156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.476156+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":64,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.323841+0000","last_change":"2024-02-23T00:47:28.232389+0000","last_active":"2024-02-23T00:48:28.323841+0000","last_peered":"2024-02-23T00:48:28.323841+0000","last_clean":"2024-02-23T00:48:28.323841+0000","last_became_active":"2024-02-23T00:47:28.231567+0000","last_became_peered":"2024-02-23T00:47:28.231567+0000","last_unstale":"2024-02-23T00:48:28.323841+0000","last_undegraded":"2024-02-23T00:48:28.323841+0000","last_fullsized":"2024-02-23T00:48:28.323841+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:45.674391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:08.335330+0000","last_clean_scrub_stamp":"2024-02-23T00:46:45.674391+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":418,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.323863+0000","last_change":"2024-02-23T00:48:13.867784+0000","last_active":"2024-02-23T00:48:28.323863+0000","last_peered":"2024-02-23T00:48:28.323863+0000","last_clean":"2024-02-23T00:48:28.323863+0000","last_became_active":"2024-02-23T00:47:28.214157+0000","last_became_peered":"2024-02-23T00:47:28.214157+0000","last_unstale":"2024-02-23T00:48:28.323863+0000","last_undegraded":"2024-02-23T00:48:28.323863+0000","last_fullsized":"2024-02-23T00:48:28.323863+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:13.867716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:13.867716+0000","last_clean_scrub_stamp":"2024-02-23T00:48:13.867716+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":338,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.325453+0000","last_change":"2024-02-23T00:48:06.908037+0000","last_active":"2024-02-23T00:48:28.325453+0000","last_peered":"2024-02-23T00:48:28.325453+0000","last_clean":"2024-02-23T00:48:28.325453+0000","last_became_active":"2024-02-23T00:42:58.867305+0000","last_became_peered":"2024-02-23T00:42:58.867305+0000","last_unstale":"2024-02-23T00:48:28.325453+0000","last_undegraded":"2024-02-23T00:48:28.325453+0000","last_fullsized":"2024-02-23T00:48:28.325453+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:06.907966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:06.907966+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"88'35","reported_seq":448,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.468270+0000","last_change":"2024-02-23T00:46:41.290546+0000","last_active":"2024-02-23T00:48:28.468270+0000","last_peered":"2024-02-23T00:48:28.468270+0000","last_clean":"2024-02-23T00:48:28.468270+0000","last_became_active":"2024-02-23T00:39:16.857413+0000","last_became_peered":"2024-02-23T00:39:16.857413+0000","last_unstale":"2024-02-23T00:48:28.468270+0000","last_undegraded":"2024-02-23T00:48:28.468270+0000","last_fullsized":"2024-02-23T00:48:28.468270+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'35","last_scrub_stamp":"2024-02-23T00:46:41.290478+0000","last_deep_scrub":"88'35","last_deep_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_clean_scrub_stamp":"2024-02-23T00:46:41.290478+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":64,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.466239+0000","last_change":"2024-02-23T00:47:28.232209+0000","last_active":"2024-02-23T00:48:28.466239+0000","last_peered":"2024-02-23T00:48:28.466239+0000","last_clean":"2024-02-23T00:48:28.466239+0000","last_became_active":"2024-02-23T00:47:28.231891+0000","last_became_peered":"2024-02-23T00:47:28.231891+0000","last_unstale":"2024-02-23T00:48:28.466239+0000","last_undegraded":"2024-02-23T00:48:28.466239+0000","last_fullsized":"2024-02-23T00:48:28.466239+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:05.910325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_clean_scrub_stamp":"2024-02-23T00:46:05.910325+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":65,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.326664+0000","last_change":"2024-02-23T00:47:28.220900+0000","last_active":"2024-02-23T00:48:28.326664+0000","last_peered":"2024-02-23T00:48:28.326664+0000","last_clean":"2024-02-23T00:48:28.326664+0000","last_became_active":"2024-02-23T00:47:28.220578+0000","last_became_peered":"2024-02-23T00:47:28.220578+0000","last_unstale":"2024-02-23T00:48:28.326664+0000","last_undegraded":"2024-02-23T00:48:28.326664+0000","last_fullsized":"2024-02-23T00:48:28.326664+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:34.726027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:34.726027+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":416,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.467893+0000","last_change":"2024-02-23T00:47:28.216027+0000","last_active":"2024-02-23T00:48:28.467893+0000","last_peered":"2024-02-23T00:48:28.467893+0000","last_clean":"2024-02-23T00:48:28.467893+0000","last_became_active":"2024-02-23T00:47:28.215636+0000","last_became_peered":"2024-02-23T00:47:28.215636+0000","last_unstale":"2024-02-23T00:48:28.467893+0000","last_undegraded":"2024-02-23T00:48:28.467893+0000","last_fullsized":"2024-02-23T00:48:28.467893+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:48.692679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:48.692679+0000","last_clean_scrub_stamp":"2024-02-23T00:46:48.692679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":332,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.465274+0000","last_change":"2024-02-23T00:46:40.354058+0000","last_active":"2024-02-23T00:48:28.465274+0000","last_peered":"2024-02-23T00:48:28.465274+0000","last_clean":"2024-02-23T00:48:28.465274+0000","last_became_active":"2024-02-23T00:42:57.639013+0000","last_became_peered":"2024-02-23T00:42:57.639013+0000","last_unstale":"2024-02-23T00:48:28.465274+0000","last_undegraded":"2024-02-23T00:48:28.465274+0000","last_fullsized":"2024-02-23T00:48:28.465274+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:40.353999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:46:40.353999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"67'17","reported_seq":471,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.324684+0000","last_change":"2024-02-23T00:47:28.219455+0000","last_active":"2024-02-23T00:48:28.324684+0000","last_peered":"2024-02-23T00:48:28.324684+0000","last_clean":"2024-02-23T00:48:28.324684+0000","last_became_active":"2024-02-23T00:47:28.200653+0000","last_became_peered":"2024-02-23T00:47:28.200653+0000","last_unstale":"2024-02-23T00:48:28.324684+0000","last_undegraded":"2024-02-23T00:48:28.324684+0000","last_fullsized":"2024-02-23T00:48:28.324684+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'17","last_scrub_stamp":"2024-02-23T00:47:15.357876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:47:15.357876+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":63,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.467503+0000","last_change":"2024-02-23T00:47:28.202200+0000","last_active":"2024-02-23T00:48:28.467503+0000","last_peered":"2024-02-23T00:48:28.467503+0000","last_clean":"2024-02-23T00:48:28.467503+0000","last_became_active":"2024-02-23T00:47:28.201926+0000","last_became_peered":"2024-02-23T00:47:28.201926+0000","last_unstale":"2024-02-23T00:48:28.467503+0000","last_undegraded":"2024-02-23T00:48:28.467503+0000","last_fullsized":"2024-02-23T00:48:28.467503+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:56.051575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:56.051575+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":391,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.465618+0000","last_change":"2024-02-23T00:48:03.801207+0000","last_active":"2024-02-23T00:48:28.465618+0000","last_peered":"2024-02-23T00:48:28.465618+0000","last_clean":"2024-02-23T00:48:28.465618+0000","last_became_active":"2024-02-23T00:39:18.799875+0000","last_became_peered":"2024-02-23T00:39:18.799875+0000","last_unstale":"2024-02-23T00:48:28.465618+0000","last_undegraded":"2024-02-23T00:48:28.465618+0000","last_fullsized":"2024-02-23T00:48:28.465618+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:03.801103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:48:03.801103+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":332,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.464815+0000","last_change":"2024-02-23T00:45:23.744990+0000","last_active":"2024-02-23T00:48:28.464815+0000","last_peered":"2024-02-23T00:48:28.464815+0000","last_clean":"2024-02-23T00:48:28.464815+0000","last_became_active":"2024-02-23T00:42:57.675306+0000","last_became_peered":"2024-02-23T00:42:57.675306+0000","last_unstale":"2024-02-23T00:48:28.464815+0000","last_undegraded":"2024-02-23T00:48:28.464815+0000","last_fullsized":"2024-02-23T00:48:28.464815+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:23.744846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.744846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"67'10","reported_seq":404,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.324270+0000","last_change":"2024-02-23T00:45:25.175258+0000","last_active":"2024-02-23T00:48:28.324270+0000","last_peered":"2024-02-23T00:48:28.324270+0000","last_clean":"2024-02-23T00:48:28.324270+0000","last_became_active":"2024-02-23T00:39:16.777530+0000","last_became_peered":"2024-02-23T00:39:16.777530+0000","last_unstale":"2024-02-23T00:48:28.324270+0000","last_undegraded":"2024-02-23T00:48:28.324270+0000","last_fullsized":"2024-02-23T00:48:28.324270+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'10","last_scrub_stamp":"2024-02-23T00:45:25.175173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.175173+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":390,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.464703+0000","last_change":"2024-02-23T00:45:10.693332+0000","last_active":"2024-02-23T00:48:28.464703+0000","last_peered":"2024-02-23T00:48:28.464703+0000","last_clean":"2024-02-23T00:48:28.464703+0000","last_became_active":"2024-02-23T00:39:14.780379+0000","last_became_peered":"2024-02-23T00:39:14.780379+0000","last_unstale":"2024-02-23T00:48:28.464703+0000","last_undegraded":"2024-02-23T00:48:28.464703+0000","last_fullsized":"2024-02-23T00:48:28.464703+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:10.693273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.575242+0000","last_clean_scrub_stamp":"2024-02-23T00:45:10.693273+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"376'9","reported_seq":101,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.468837+0000","last_change":"2024-02-23T00:47:28.336504+0000","last_active":"2024-02-23T00:48:28.468837+0000","last_peered":"2024-02-23T00:48:28.468837+0000","last_clean":"2024-02-23T00:48:28.468837+0000","last_became_active":"2024-02-23T00:47:28.231104+0000","last_became_peered":"2024-02-23T00:47:28.231104+0000","last_unstale":"2024-02-23T00:48:28.468837+0000","last_undegraded":"2024-02-23T00:48:28.468837+0000","last_fullsized":"2024-02-23T00:48:28.468837+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:46:49.665271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:49.665271+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":334,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.323570+0000","last_change":"2024-02-23T00:45:22.874761+0000","last_active":"2024-02-23T00:48:28.323570+0000","last_peered":"2024-02-23T00:48:28.323570+0000","last_clean":"2024-02-23T00:48:28.323570+0000","last_became_active":"2024-02-23T00:42:58.765106+0000","last_became_peered":"2024-02-23T00:42:58.765106+0000","last_unstale":"2024-02-23T00:48:28.323570+0000","last_undegraded":"2024-02-23T00:48:28.323570+0000","last_fullsized":"2024-02-23T00:48:28.323570+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:22.874696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:22.874696+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"67'15","reported_seq":418,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.465306+0000","last_change":"2024-02-23T00:48:00.698737+0000","last_active":"2024-02-23T00:48:28.465306+0000","last_peered":"2024-02-23T00:48:28.465306+0000","last_clean":"2024-02-23T00:48:28.465306+0000","last_became_active":"2024-02-23T00:39:16.789757+0000","last_became_peered":"2024-02-23T00:39:16.789757+0000","last_unstale":"2024-02-23T00:48:28.465306+0000","last_undegraded":"2024-02-23T00:48:28.465306+0000","last_fullsized":"2024-02-23T00:48:28.465306+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:48:00.698670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:48:00.698670+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":395,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.464471+0000","last_change":"2024-02-23T00:48:09.801324+0000","last_active":"2024-02-23T00:48:28.464471+0000","last_peered":"2024-02-23T00:48:28.464471+0000","last_clean":"2024-02-23T00:48:28.464471+0000","last_became_active":"2024-02-23T00:39:14.780526+0000","last_became_peered":"2024-02-23T00:39:14.780526+0000","last_unstale":"2024-02-23T00:48:28.464471+0000","last_undegraded":"2024-02-23T00:48:28.464471+0000","last_fullsized":"2024-02-23T00:48:28.464471+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:09.801251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:48:09.801251+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":511,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.324775+0000","last_change":"2024-02-23T00:47:24.353965+0000","last_active":"2024-02-23T00:48:28.324775+0000","last_peered":"2024-02-23T00:48:28.324775+0000","last_clean":"2024-02-23T00:48:28.324775+0000","last_became_active":"2024-02-23T00:39:18.800440+0000","last_became_peered":"2024-02-23T00:39:18.800440+0000","last_unstale":"2024-02-23T00:48:28.324775+0000","last_undegraded":"2024-02-23T00:48:28.324775+0000","last_fullsized":"2024-02-23T00:48:28.324775+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:47:24.353888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:47:24.353888+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":333,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.323441+0000","last_change":"2024-02-23T00:45:54.311565+0000","last_active":"2024-02-23T00:48:28.323441+0000","last_peered":"2024-02-23T00:48:28.323441+0000","last_clean":"2024-02-23T00:48:28.323441+0000","last_became_active":"2024-02-23T00:42:58.833924+0000","last_became_peered":"2024-02-23T00:42:58.833924+0000","last_unstale":"2024-02-23T00:48:28.323441+0000","last_undegraded":"2024-02-23T00:48:28.323441+0000","last_fullsized":"2024-02-23T00:48:28.323441+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:54.311442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:54.311442+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"336'23","reported_seq":475,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.324699+0000","last_change":"2024-02-23T00:47:28.259863+0000","last_active":"2024-02-23T00:48:28.324699+0000","last_peered":"2024-02-23T00:48:28.324699+0000","last_clean":"2024-02-23T00:48:28.324699+0000","last_became_active":"2024-02-23T00:47:28.201225+0000","last_became_peered":"2024-02-23T00:47:28.201225+0000","last_unstale":"2024-02-23T00:48:28.324699+0000","last_undegraded":"2024-02-23T00:48:28.324699+0000","last_fullsized":"2024-02-23T00:48:28.324699+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"336'23","last_scrub_stamp":"2024-02-23T00:46:48.320085+0000","last_deep_scrub":"336'23","last_deep_scrub_stamp":"2024-02-23T00:46:48.320085+0000","last_clean_scrub_stamp":"2024-02-23T00:46:48.320085+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":395,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.468264+0000","last_change":"2024-02-23T00:46:43.722544+0000","last_active":"2024-02-23T00:48:28.468264+0000","last_peered":"2024-02-23T00:48:28.468264+0000","last_clean":"2024-02-23T00:48:28.468264+0000","last_became_active":"2024-02-23T00:39:14.784121+0000","last_became_peered":"2024-02-23T00:39:14.784121+0000","last_unstale":"2024-02-23T00:48:28.468264+0000","last_undegraded":"2024-02-23T00:48:28.468264+0000","last_fullsized":"2024-02-23T00:48:28.468264+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:43.722480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:43.722480+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":392,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.468891+0000","last_change":"2024-02-23T00:47:18.254453+0000","last_active":"2024-02-23T00:48:28.468891+0000","last_peered":"2024-02-23T00:48:28.468891+0000","last_clean":"2024-02-23T00:48:28.468891+0000","last_became_active":"2024-02-23T00:39:18.800777+0000","last_became_peered":"2024-02-23T00:39:18.800777+0000","last_unstale":"2024-02-23T00:48:28.468891+0000","last_undegraded":"2024-02-23T00:48:28.468891+0000","last_fullsized":"2024-02-23T00:48:28.468891+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:18.254393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_clean_scrub_stamp":"2024-02-23T00:47:18.254393+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":332,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.464421+0000","last_change":"2024-02-23T00:46:35.407560+0000","last_active":"2024-02-23T00:48:28.464421+0000","last_peered":"2024-02-23T00:48:28.464421+0000","last_clean":"2024-02-23T00:48:28.464421+0000","last_became_active":"2024-02-23T00:42:57.653279+0000","last_became_peered":"2024-02-23T00:42:57.653279+0000","last_unstale":"2024-02-23T00:48:28.464421+0000","last_undegraded":"2024-02-23T00:48:28.464421+0000","last_fullsized":"2024-02-23T00:48:28.464421+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:35.407468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:46:35.407468+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":508,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.323266+0000","last_change":"2024-02-23T00:45:51.871688+0000","last_active":"2024-02-23T00:48:28.323266+0000","last_peered":"2024-02-23T00:48:28.323266+0000","last_clean":"2024-02-23T00:48:28.323266+0000","last_became_active":"2024-02-23T00:39:18.798536+0000","last_became_peered":"2024-02-23T00:39:18.798536+0000","last_unstale":"2024-02-23T00:48:28.323266+0000","last_undegraded":"2024-02-23T00:48:28.323266+0000","last_fullsized":"2024-02-23T00:48:28.323266+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:45:51.871618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:51.871618+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"423'28","reported_seq":438,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.323583+0000","last_change":"2024-02-23T00:48:17.435988+0000","last_active":"2024-02-23T00:48:28.323583+0000","last_peered":"2024-02-23T00:48:28.323583+0000","last_clean":"2024-02-23T00:48:28.323583+0000","last_became_active":"2024-02-23T00:39:16.780130+0000","last_became_peered":"2024-02-23T00:39:16.780130+0000","last_unstale":"2024-02-23T00:48:28.323583+0000","last_undegraded":"2024-02-23T00:48:28.323583+0000","last_fullsized":"2024-02-23T00:48:28.323583+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"423'28","last_scrub_stamp":"2024-02-23T00:48:17.435906+0000","last_deep_scrub":"423'28","last_deep_scrub_stamp":"2024-02-23T00:48:17.435906+0000","last_clean_scrub_stamp":"2024-02-23T00:48:17.435906+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":410,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.467372+0000","last_change":"2024-02-23T00:48:13.584696+0000","last_active":"2024-02-23T00:48:28.467372+0000","last_peered":"2024-02-23T00:48:28.467372+0000","last_clean":"2024-02-23T00:48:28.467372+0000","last_became_active":"2024-02-23T00:47:28.237083+0000","last_became_peered":"2024-02-23T00:47:28.237083+0000","last_unstale":"2024-02-23T00:48:28.467372+0000","last_undegraded":"2024-02-23T00:48:28.467372+0000","last_fullsized":"2024-02-23T00:48:28.467372+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:13.584566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:51.391939+0000","last_clean_scrub_stamp":"2024-02-23T00:48:13.584566+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":408,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.468236+0000","last_change":"2024-02-23T00:47:30.187702+0000","last_active":"2024-02-23T00:48:28.468236+0000","last_peered":"2024-02-23T00:48:28.468236+0000","last_clean":"2024-02-23T00:48:28.468236+0000","last_became_active":"2024-02-23T00:47:30.187386+0000","last_became_peered":"2024-02-23T00:47:30.187386+0000","last_unstale":"2024-02-23T00:48:28.468236+0000","last_undegraded":"2024-02-23T00:48:28.468236+0000","last_fullsized":"2024-02-23T00:48:28.468236+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:46.686764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:46.686764+0000","last_clean_scrub_stamp":"2024-02-23T00:46:46.686764+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":341,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.469044+0000","last_change":"2024-02-23T00:47:28.212626+0000","last_active":"2024-02-23T00:48:28.469044+0000","last_peered":"2024-02-23T00:48:28.469044+0000","last_clean":"2024-02-23T00:48:28.469044+0000","last_became_active":"2024-02-23T00:47:28.212307+0000","last_became_peered":"2024-02-23T00:47:28.212307+0000","last_unstale":"2024-02-23T00:48:28.469044+0000","last_undegraded":"2024-02-23T00:48:28.469044+0000","last_fullsized":"2024-02-23T00:48:28.469044+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:59.944587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:59.944587+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":394,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.323855+0000","last_change":"2024-02-23T00:48:10.493418+0000","last_active":"2024-02-23T00:48:28.323855+0000","last_peered":"2024-02-23T00:48:28.323855+0000","last_clean":"2024-02-23T00:48:28.323855+0000","last_became_active":"2024-02-23T00:39:18.798829+0000","last_became_peered":"2024-02-23T00:39:18.798829+0000","last_unstale":"2024-02-23T00:48:28.323855+0000","last_undegraded":"2024-02-23T00:48:28.323855+0000","last_fullsized":"2024-02-23T00:48:28.323855+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:10.493340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:10.493340+0000","last_clean_scrub_stamp":"2024-02-23T00:48:10.493340+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"251'40","reported_seq":468,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.323176+0000","last_change":"2024-02-23T00:48:17.054839+0000","last_active":"2024-02-23T00:48:28.323176+0000","last_peered":"2024-02-23T00:48:28.323176+0000","last_clean":"2024-02-23T00:48:28.323176+0000","last_became_active":"2024-02-23T00:39:16.859002+0000","last_became_peered":"2024-02-23T00:39:16.859002+0000","last_unstale":"2024-02-23T00:48:28.323176+0000","last_undegraded":"2024-02-23T00:48:28.323176+0000","last_fullsized":"2024-02-23T00:48:28.323176+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'40","last_scrub_stamp":"2024-02-23T00:48:17.054722+0000","last_deep_scrub":"251'40","last_deep_scrub_stamp":"2024-02-23T00:45:23.910546+0000","last_clean_scrub_stamp":"2024-02-23T00:48:17.054722+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":390,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.465228+0000","last_change":"2024-02-23T00:45:16.664785+0000","last_active":"2024-02-23T00:48:28.465228+0000","last_peered":"2024-02-23T00:48:28.465228+0000","last_clean":"2024-02-23T00:48:28.465228+0000","last_became_active":"2024-02-23T00:39:14.779193+0000","last_became_peered":"2024-02-23T00:39:14.779193+0000","last_unstale":"2024-02-23T00:48:28.465228+0000","last_undegraded":"2024-02-23T00:48:28.465228+0000","last_fullsized":"2024-02-23T00:48:28.465228+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:16.664682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:16.664682+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":417,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.323885+0000","last_change":"2024-02-23T00:48:13.462943+0000","last_active":"2024-02-23T00:48:28.323885+0000","last_peered":"2024-02-23T00:48:28.323885+0000","last_clean":"2024-02-23T00:48:28.323885+0000","last_became_active":"2024-02-23T00:47:33.257915+0000","last_became_peered":"2024-02-23T00:47:33.257915+0000","last_unstale":"2024-02-23T00:48:28.323885+0000","last_undegraded":"2024-02-23T00:48:28.323885+0000","last_fullsized":"2024-02-23T00:48:28.323885+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":381,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:13.462861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:48:13.462861+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":339,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.325765+0000","last_change":"2024-02-23T00:48:04.989583+0000","last_active":"2024-02-23T00:48:28.325765+0000","last_peered":"2024-02-23T00:48:28.325765+0000","last_clean":"2024-02-23T00:48:28.325765+0000","last_became_active":"2024-02-23T00:42:58.809283+0000","last_became_peered":"2024-02-23T00:42:58.809283+0000","last_unstale":"2024-02-23T00:48:28.325765+0000","last_undegraded":"2024-02-23T00:48:28.325765+0000","last_fullsized":"2024-02-23T00:48:28.325765+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:04.989514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:04.989514+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":390,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.468161+0000","last_change":"2024-02-23T00:46:45.294769+0000","last_active":"2024-02-23T00:48:28.468161+0000","last_peered":"2024-02-23T00:48:28.468161+0000","last_clean":"2024-02-23T00:48:28.468161+0000","last_became_active":"2024-02-23T00:39:18.796664+0000","last_became_peered":"2024-02-23T00:39:18.796664+0000","last_unstale":"2024-02-23T00:48:28.468161+0000","last_undegraded":"2024-02-23T00:48:28.468161+0000","last_fullsized":"2024-02-23T00:48:28.468161+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:45.294704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:45.294704+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"203'34","reported_seq":455,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.326588+0000","last_change":"2024-02-23T00:47:18.765648+0000","last_active":"2024-02-23T00:48:28.326588+0000","last_peered":"2024-02-23T00:48:28.326588+0000","last_clean":"2024-02-23T00:48:28.326588+0000","last_became_active":"2024-02-23T00:39:16.857278+0000","last_became_peered":"2024-02-23T00:39:16.857278+0000","last_unstale":"2024-02-23T00:48:28.326588+0000","last_undegraded":"2024-02-23T00:48:28.326588+0000","last_fullsized":"2024-02-23T00:48:28.326588+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'34","last_scrub_stamp":"2024-02-23T00:47:18.765582+0000","last_deep_scrub":"203'34","last_deep_scrub_stamp":"2024-02-23T00:44:35.643931+0000","last_clean_scrub_stamp":"2024-02-23T00:47:18.765582+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":395,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.468793+0000","last_change":"2024-02-23T00:47:17.448466+0000","last_active":"2024-02-23T00:48:28.468793+0000","last_peered":"2024-02-23T00:48:28.468793+0000","last_clean":"2024-02-23T00:48:28.468793+0000","last_became_active":"2024-02-23T00:39:14.783662+0000","last_became_peered":"2024-02-23T00:39:14.783662+0000","last_unstale":"2024-02-23T00:48:28.468793+0000","last_undegraded":"2024-02-23T00:48:28.468793+0000","last_fullsized":"2024-02-23T00:48:28.468793+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:17.448376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:47:17.448376+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":416,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.464574+0000","last_change":"2024-02-23T00:48:13.821965+0000","last_active":"2024-02-23T00:48:28.464574+0000","last_peered":"2024-02-23T00:48:28.464574+0000","last_clean":"2024-02-23T00:48:28.464574+0000","last_became_active":"2024-02-23T00:47:37.184240+0000","last_became_peered":"2024-02-23T00:47:37.184240+0000","last_unstale":"2024-02-23T00:48:28.464574+0000","last_undegraded":"2024-02-23T00:48:28.464574+0000","last_fullsized":"2024-02-23T00:48:28.464574+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":385,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:13.821901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:48:13.821901+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":337,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.465826+0000","last_change":"2024-02-23T00:47:57.273372+0000","last_active":"2024-02-23T00:48:28.465826+0000","last_peered":"2024-02-23T00:48:28.465826+0000","last_clean":"2024-02-23T00:48:28.465826+0000","last_became_active":"2024-02-23T00:42:58.726652+0000","last_became_peered":"2024-02-23T00:42:58.726652+0000","last_unstale":"2024-02-23T00:48:28.465826+0000","last_undegraded":"2024-02-23T00:48:28.465826+0000","last_fullsized":"2024-02-23T00:48:28.465826+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:57.272902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:47:57.272902+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":392,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.468952+0000","last_change":"2024-02-23T00:46:37.333068+0000","last_active":"2024-02-23T00:48:28.468952+0000","last_peered":"2024-02-23T00:48:28.468952+0000","last_clean":"2024-02-23T00:48:28.468952+0000","last_became_active":"2024-02-23T00:39:18.800378+0000","last_became_peered":"2024-02-23T00:39:18.800378+0000","last_unstale":"2024-02-23T00:48:28.468952+0000","last_undegraded":"2024-02-23T00:48:28.468952+0000","last_fullsized":"2024-02-23T00:48:28.468952+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.332973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:59.694033+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.332973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"95'18","reported_seq":442,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.325772+0000","last_change":"2024-02-23T00:47:28.261515+0000","last_active":"2024-02-23T00:48:28.325772+0000","last_peered":"2024-02-23T00:48:28.325772+0000","last_clean":"2024-02-23T00:48:28.325772+0000","last_became_active":"2024-02-23T00:47:28.215366+0000","last_became_peered":"2024-02-23T00:47:28.215366+0000","last_unstale":"2024-02-23T00:48:28.325772+0000","last_undegraded":"2024-02-23T00:48:28.325772+0000","last_fullsized":"2024-02-23T00:48:28.325772+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'18","last_scrub_stamp":"2024-02-23T00:47:20.795884+0000","last_deep_scrub":"95'18","last_deep_scrub_stamp":"2024-02-23T00:42:01.195545+0000","last_clean_scrub_stamp":"2024-02-23T00:47:20.795884+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":400,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.468057+0000","last_change":"2024-02-23T00:47:24.444461+0000","last_active":"2024-02-23T00:48:28.468057+0000","last_peered":"2024-02-23T00:48:28.468057+0000","last_clean":"2024-02-23T00:48:28.468057+0000","last_became_active":"2024-02-23T00:39:14.784578+0000","last_became_peered":"2024-02-23T00:39:14.784578+0000","last_unstale":"2024-02-23T00:48:28.468057+0000","last_undegraded":"2024-02-23T00:48:28.468057+0000","last_fullsized":"2024-02-23T00:48:28.468057+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:24.444390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:47:24.444390+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":397,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.323669+0000","last_change":"2024-02-23T00:46:44.613082+0000","last_active":"2024-02-23T00:48:28.323669+0000","last_peered":"2024-02-23T00:48:28.323669+0000","last_clean":"2024-02-23T00:48:28.323669+0000","last_became_active":"2024-02-23T00:39:14.775446+0000","last_became_peered":"2024-02-23T00:39:14.775446+0000","last_unstale":"2024-02-23T00:48:28.323669+0000","last_undegraded":"2024-02-23T00:48:28.323669+0000","last_fullsized":"2024-02-23T00:48:28.323669+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:44.613012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:44.613012+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"67'1","reported_seq":420,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.465681+0000","last_change":"2024-02-23T00:47:57.665268+0000","last_active":"2024-02-23T00:48:28.465681+0000","last_peered":"2024-02-23T00:48:28.465681+0000","last_clean":"2024-02-23T00:48:28.465681+0000","last_became_active":"2024-02-23T00:47:56.187940+0000","last_became_peered":"2024-02-23T00:47:56.187940+0000","last_unstale":"2024-02-23T00:48:28.465681+0000","last_undegraded":"2024-02-23T00:48:28.465681+0000","last_fullsized":"2024-02-23T00:48:28.465681+0000","mapping_epoch":65,"log_start":"67'1","ondisk_log_start":"67'1","created":65,"last_epoch_clean":404,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:47:57.665156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:47:57.665156+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":341,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.465249+0000","last_change":"2024-02-23T00:48:15.593613+0000","last_active":"2024-02-23T00:48:28.465249+0000","last_peered":"2024-02-23T00:48:28.465249+0000","last_clean":"2024-02-23T00:48:28.465249+0000","last_became_active":"2024-02-23T00:42:58.690858+0000","last_became_peered":"2024-02-23T00:42:58.690858+0000","last_unstale":"2024-02-23T00:48:28.465249+0000","last_undegraded":"2024-02-23T00:48:28.465249+0000","last_fullsized":"2024-02-23T00:48:28.465249+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:15.593488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:35.360054+0000","last_clean_scrub_stamp":"2024-02-23T00:48:15.593488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":400,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.467182+0000","last_change":"2024-02-23T00:47:28.231576+0000","last_active":"2024-02-23T00:48:28.467182+0000","last_peered":"2024-02-23T00:48:28.467182+0000","last_clean":"2024-02-23T00:48:28.467182+0000","last_became_active":"2024-02-23T00:47:28.231152+0000","last_became_peered":"2024-02-23T00:47:28.231152+0000","last_unstale":"2024-02-23T00:48:28.467182+0000","last_undegraded":"2024-02-23T00:48:28.467182+0000","last_fullsized":"2024-02-23T00:48:28.467182+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:49.704984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:49.704984+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"67'14","reported_seq":419,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.466329+0000","last_change":"2024-02-23T00:48:01.585300+0000","last_active":"2024-02-23T00:48:28.466329+0000","last_peered":"2024-02-23T00:48:28.466329+0000","last_clean":"2024-02-23T00:48:28.466329+0000","last_became_active":"2024-02-23T00:39:16.858880+0000","last_became_peered":"2024-02-23T00:39:16.858880+0000","last_unstale":"2024-02-23T00:48:28.466329+0000","last_undegraded":"2024-02-23T00:48:28.466329+0000","last_fullsized":"2024-02-23T00:48:28.466329+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'14","last_scrub_stamp":"2024-02-23T00:48:01.585203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:48:01.585203+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":405,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.468157+0000","last_change":"2024-02-23T00:47:14.531194+0000","last_active":"2024-02-23T00:48:28.468157+0000","last_peered":"2024-02-23T00:48:28.468157+0000","last_clean":"2024-02-23T00:48:28.468157+0000","last_became_active":"2024-02-23T00:39:08.248249+0000","last_became_peered":"2024-02-23T00:39:08.248249+0000","last_unstale":"2024-02-23T00:48:28.468157+0000","last_undegraded":"2024-02-23T00:48:28.468157+0000","last_fullsized":"2024-02-23T00:48:28.468157+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:14.531130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:49.977248+0000","last_clean_scrub_stamp":"2024-02-23T00:47:14.531130+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":405,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.466911+0000","last_change":"2024-02-23T00:47:55.310719+0000","last_active":"2024-02-23T00:48:28.466911+0000","last_peered":"2024-02-23T00:48:28.466911+0000","last_clean":"2024-02-23T00:48:28.466911+0000","last_became_active":"2024-02-23T00:47:28.240482+0000","last_became_peered":"2024-02-23T00:47:28.240482+0000","last_unstale":"2024-02-23T00:48:28.466911+0000","last_undegraded":"2024-02-23T00:48:28.466911+0000","last_fullsized":"2024-02-23T00:48:28.466911+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:55.310652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:47:55.310652+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":398,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.323452+0000","last_change":"2024-02-23T00:47:54.034897+0000","last_active":"2024-02-23T00:48:28.323452+0000","last_peered":"2024-02-23T00:48:28.323452+0000","last_clean":"2024-02-23T00:48:28.323452+0000","last_became_active":"2024-02-23T00:39:14.783489+0000","last_became_peered":"2024-02-23T00:39:14.783489+0000","last_unstale":"2024-02-23T00:48:28.323452+0000","last_undegraded":"2024-02-23T00:48:28.323452+0000","last_fullsized":"2024-02-23T00:48:28.323452+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:54.034836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:47:54.034836+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":426,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.468990+0000","last_change":"2024-02-23T00:47:45.183506+0000","last_active":"2024-02-23T00:48:28.468990+0000","last_peered":"2024-02-23T00:48:28.468990+0000","last_clean":"2024-02-23T00:48:28.468990+0000","last_became_active":"2024-02-23T00:47:45.182550+0000","last_became_peered":"2024-02-23T00:47:45.182550+0000","last_unstale":"2024-02-23T00:48:28.468990+0000","last_undegraded":"2024-02-23T00:48:28.468990+0000","last_fullsized":"2024-02-23T00:48:28.468990+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:45.207275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:46:45.207275+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":347,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.468004+0000","last_change":"2024-02-23T00:47:28.233784+0000","last_active":"2024-02-23T00:48:28.468004+0000","last_peered":"2024-02-23T00:48:28.468004+0000","last_clean":"2024-02-23T00:48:28.468004+0000","last_became_active":"2024-02-23T00:47:28.233475+0000","last_became_peered":"2024-02-23T00:47:28.233475+0000","last_unstale":"2024-02-23T00:48:28.468004+0000","last_undegraded":"2024-02-23T00:48:28.468004+0000","last_fullsized":"2024-02-23T00:48:28.468004+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:22.448945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:47:22.448945+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":64,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.468333+0000","last_change":"2024-02-23T00:47:28.209462+0000","last_active":"2024-02-23T00:48:28.468333+0000","last_peered":"2024-02-23T00:48:28.468333+0000","last_clean":"2024-02-23T00:48:28.468333+0000","last_became_active":"2024-02-23T00:47:28.209125+0000","last_became_peered":"2024-02-23T00:47:28.209125+0000","last_unstale":"2024-02-23T00:48:28.468333+0000","last_undegraded":"2024-02-23T00:48:28.468333+0000","last_fullsized":"2024-02-23T00:48:28.468333+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:09.924384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.264240+0000","last_clean_scrub_stamp":"2024-02-23T00:46:09.924384+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"68'19","reported_seq":435,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.465374+0000","last_change":"2024-02-23T00:48:11.828616+0000","last_active":"2024-02-23T00:48:28.465374+0000","last_peered":"2024-02-23T00:48:28.465374+0000","last_clean":"2024-02-23T00:48:28.465374+0000","last_became_active":"2024-02-23T00:39:16.787484+0000","last_became_peered":"2024-02-23T00:39:16.787484+0000","last_unstale":"2024-02-23T00:48:28.465374+0000","last_undegraded":"2024-02-23T00:48:28.465374+0000","last_fullsized":"2024-02-23T00:48:28.465374+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'19","last_scrub_stamp":"2024-02-23T00:48:11.828547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:48:11.828547+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":65,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.326702+0000","last_change":"2024-02-23T00:47:28.210014+0000","last_active":"2024-02-23T00:48:28.326702+0000","last_peered":"2024-02-23T00:48:28.326702+0000","last_clean":"2024-02-23T00:48:28.326702+0000","last_became_active":"2024-02-23T00:47:28.209575+0000","last_became_peered":"2024-02-23T00:47:28.209575+0000","last_unstale":"2024-02-23T00:48:28.326702+0000","last_undegraded":"2024-02-23T00:48:28.326702+0000","last_fullsized":"2024-02-23T00:48:28.326702+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:10.878204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:10.878204+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"68'5","reported_seq":954,"reported_epoch":429,"state":"active+clean","last_fresh":"2024-02-23T00:48:30.334759+0000","last_change":"2024-02-23T00:47:53.669608+0000","last_active":"2024-02-23T00:48:30.334759+0000","last_peered":"2024-02-23T00:48:30.334759+0000","last_clean":"2024-02-23T00:48:30.334759+0000","last_became_active":"2024-02-23T00:47:28.227928+0000","last_became_peered":"2024-02-23T00:47:28.227928+0000","last_unstale":"2024-02-23T00:48:30.334759+0000","last_undegraded":"2024-02-23T00:48:30.334759+0000","last_fullsized":"2024-02-23T00:48:30.334759+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'5","last_scrub_stamp":"2024-02-23T00:47:53.669533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:47:53.669533+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":547,"num_read_kb":542,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":425,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.468737+0000","last_change":"2024-02-23T00:47:52.552544+0000","last_active":"2024-02-23T00:48:28.468737+0000","last_peered":"2024-02-23T00:48:28.468737+0000","last_clean":"2024-02-23T00:48:28.468737+0000","last_became_active":"2024-02-23T00:47:52.552244+0000","last_became_peered":"2024-02-23T00:47:52.552244+0000","last_unstale":"2024-02-23T00:48:28.468737+0000","last_undegraded":"2024-02-23T00:48:28.468737+0000","last_fullsized":"2024-02-23T00:48:28.468737+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:36.725686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:46:36.725686+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":341,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.464351+0000","last_change":"2024-02-23T00:47:28.212866+0000","last_active":"2024-02-23T00:48:28.464351+0000","last_peered":"2024-02-23T00:48:28.464351+0000","last_clean":"2024-02-23T00:48:28.464351+0000","last_became_active":"2024-02-23T00:47:28.212516+0000","last_became_peered":"2024-02-23T00:47:28.212516+0000","last_unstale":"2024-02-23T00:48:28.464351+0000","last_undegraded":"2024-02-23T00:48:28.464351+0000","last_fullsized":"2024-02-23T00:48:28.464351+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:42.331547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:46:42.331547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":393,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.325390+0000","last_change":"2024-02-23T00:47:13.773330+0000","last_active":"2024-02-23T00:48:28.325390+0000","last_peered":"2024-02-23T00:48:28.325390+0000","last_clean":"2024-02-23T00:48:28.325390+0000","last_became_active":"2024-02-23T00:39:18.802134+0000","last_became_peered":"2024-02-23T00:39:18.802134+0000","last_unstale":"2024-02-23T00:48:28.325390+0000","last_undegraded":"2024-02-23T00:48:28.325390+0000","last_fullsized":"2024-02-23T00:48:28.325390+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:13.773262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:47:13.773262+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"376'36","reported_seq":522,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.466138+0000","last_change":"2024-02-23T00:48:12.364622+0000","last_active":"2024-02-23T00:48:28.466138+0000","last_peered":"2024-02-23T00:48:28.466138+0000","last_clean":"2024-02-23T00:48:28.466138+0000","last_became_active":"2024-02-23T00:47:28.240257+0000","last_became_peered":"2024-02-23T00:47:28.240257+0000","last_unstale":"2024-02-23T00:48:28.466138+0000","last_undegraded":"2024-02-23T00:48:28.466138+0000","last_fullsized":"2024-02-23T00:48:28.466138+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"376'36","last_scrub_stamp":"2024-02-23T00:48:12.364544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:48:12.364544+0000","log_size":36,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":396,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.465341+0000","last_change":"2024-02-23T00:47:17.212478+0000","last_active":"2024-02-23T00:48:28.465341+0000","last_peered":"2024-02-23T00:48:28.465341+0000","last_clean":"2024-02-23T00:48:28.465341+0000","last_became_active":"2024-02-23T00:39:14.784015+0000","last_became_peered":"2024-02-23T00:39:14.784015+0000","last_unstale":"2024-02-23T00:48:28.465341+0000","last_undegraded":"2024-02-23T00:48:28.465341+0000","last_fullsized":"2024-02-23T00:48:28.465341+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:17.212388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:47:17.212388+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":423,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.325614+0000","last_change":"2024-02-23T00:47:49.525743+0000","last_active":"2024-02-23T00:48:28.325614+0000","last_peered":"2024-02-23T00:48:28.325614+0000","last_clean":"2024-02-23T00:48:28.325614+0000","last_became_active":"2024-02-23T00:47:49.525421+0000","last_became_peered":"2024-02-23T00:47:49.525421+0000","last_unstale":"2024-02-23T00:48:28.325614+0000","last_undegraded":"2024-02-23T00:48:28.325614+0000","last_fullsized":"2024-02-23T00:48:28.325614+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":397,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:42.938209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:46:42.938209+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":65,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.323987+0000","last_change":"2024-02-23T00:47:28.232592+0000","last_active":"2024-02-23T00:48:28.323987+0000","last_peered":"2024-02-23T00:48:28.323987+0000","last_clean":"2024-02-23T00:48:28.323987+0000","last_became_active":"2024-02-23T00:47:28.232194+0000","last_became_peered":"2024-02-23T00:47:28.232194+0000","last_unstale":"2024-02-23T00:48:28.323987+0000","last_undegraded":"2024-02-23T00:48:28.323987+0000","last_fullsized":"2024-02-23T00:48:28.323987+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:21.604592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:47:21.604592+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3","version":"91'9","reported_seq":513,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.326900+0000","last_change":"2024-02-23T00:47:12.798383+0000","last_active":"2024-02-23T00:48:28.326900+0000","last_peered":"2024-02-23T00:48:28.326900+0000","last_clean":"2024-02-23T00:48:28.326900+0000","last_became_active":"2024-02-23T00:39:18.801297+0000","last_became_peered":"2024-02-23T00:39:18.801297+0000","last_unstale":"2024-02-23T00:48:28.326900+0000","last_undegraded":"2024-02-23T00:48:28.326900+0000","last_fullsized":"2024-02-23T00:48:28.326900+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'9","last_scrub_stamp":"2024-02-23T00:47:12.798314+0000","last_deep_scrub":"91'9","last_deep_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_clean_scrub_stamp":"2024-02-23T00:47:12.798314+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"68'16","reported_seq":418,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.464916+0000","last_change":"2024-02-23T00:45:11.713800+0000","last_active":"2024-02-23T00:48:28.464916+0000","last_peered":"2024-02-23T00:48:28.464916+0000","last_clean":"2024-02-23T00:48:28.464916+0000","last_became_active":"2024-02-23T00:39:16.787468+0000","last_became_peered":"2024-02-23T00:39:16.787468+0000","last_unstale":"2024-02-23T00:48:28.464916+0000","last_undegraded":"2024-02-23T00:48:28.464916+0000","last_fullsized":"2024-02-23T00:48:28.464916+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:45:11.713731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:11.713731+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":395,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.465415+0000","last_change":"2024-02-23T00:48:12.869003+0000","last_active":"2024-02-23T00:48:28.465415+0000","last_peered":"2024-02-23T00:48:28.465415+0000","last_clean":"2024-02-23T00:48:28.465415+0000","last_became_active":"2024-02-23T00:39:14.779952+0000","last_became_peered":"2024-02-23T00:39:14.779952+0000","last_unstale":"2024-02-23T00:48:28.465415+0000","last_undegraded":"2024-02-23T00:48:28.465415+0000","last_fullsized":"2024-02-23T00:48:28.465415+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:12.868930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:48:12.868930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"107'2","reported_seq":424,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.323420+0000","last_change":"2024-02-23T00:48:02.920168+0000","last_active":"2024-02-23T00:48:28.323420+0000","last_peered":"2024-02-23T00:48:28.323420+0000","last_clean":"2024-02-23T00:48:28.323420+0000","last_became_active":"2024-02-23T00:47:41.210074+0000","last_became_peered":"2024-02-23T00:47:41.210074+0000","last_unstale":"2024-02-23T00:48:28.323420+0000","last_undegraded":"2024-02-23T00:48:28.323420+0000","last_fullsized":"2024-02-23T00:48:28.323420+0000","mapping_epoch":375,"log_start":"107'2","ondisk_log_start":"107'2","created":65,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'2","last_scrub_stamp":"2024-02-23T00:48:02.920084+0000","last_deep_scrub":"107'2","last_deep_scrub_stamp":"2024-02-23T00:42:48.362739+0000","last_clean_scrub_stamp":"2024-02-23T00:48:02.920084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":345,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.466084+0000","last_change":"2024-02-23T00:47:28.242563+0000","last_active":"2024-02-23T00:48:28.466084+0000","last_peered":"2024-02-23T00:48:28.466084+0000","last_clean":"2024-02-23T00:48:28.466084+0000","last_became_active":"2024-02-23T00:47:28.242255+0000","last_became_peered":"2024-02-23T00:47:28.242255+0000","last_unstale":"2024-02-23T00:48:28.466084+0000","last_undegraded":"2024-02-23T00:48:28.466084+0000","last_fullsized":"2024-02-23T00:48:28.466084+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:10.415458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:47:10.415458+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":65,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.323679+0000","last_change":"2024-02-23T00:47:28.235122+0000","last_active":"2024-02-23T00:48:28.323679+0000","last_peered":"2024-02-23T00:48:28.323679+0000","last_clean":"2024-02-23T00:48:28.323679+0000","last_became_active":"2024-02-23T00:47:28.234034+0000","last_became_peered":"2024-02-23T00:47:28.234034+0000","last_unstale":"2024-02-23T00:48:28.323679+0000","last_undegraded":"2024-02-23T00:48:28.323679+0000","last_fullsized":"2024-02-23T00:48:28.323679+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:36.783823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_clean_scrub_stamp":"2024-02-23T00:46:36.783823+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"68'25","reported_seq":443,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.323825+0000","last_change":"2024-02-23T00:46:51.333974+0000","last_active":"2024-02-23T00:48:28.323825+0000","last_peered":"2024-02-23T00:48:28.323825+0000","last_clean":"2024-02-23T00:48:28.323825+0000","last_became_active":"2024-02-23T00:39:16.783926+0000","last_became_peered":"2024-02-23T00:39:16.783926+0000","last_unstale":"2024-02-23T00:48:28.323825+0000","last_undegraded":"2024-02-23T00:48:28.323825+0000","last_fullsized":"2024-02-23T00:48:28.323825+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'25","last_scrub_stamp":"2024-02-23T00:46:51.333896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:51.333896+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"95'4","reported_seq":411,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.324931+0000","last_change":"2024-02-23T00:47:27.346629+0000","last_active":"2024-02-23T00:48:28.324931+0000","last_peered":"2024-02-23T00:48:28.324931+0000","last_clean":"2024-02-23T00:48:28.324931+0000","last_became_active":"2024-02-23T00:39:14.779205+0000","last_became_peered":"2024-02-23T00:39:14.779205+0000","last_unstale":"2024-02-23T00:48:28.324931+0000","last_undegraded":"2024-02-23T00:48:28.324931+0000","last_fullsized":"2024-02-23T00:48:28.324931+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'4","last_scrub_stamp":"2024-02-23T00:47:27.346540+0000","last_deep_scrub":"95'4","last_deep_scrub_stamp":"2024-02-23T00:41:43.874397+0000","last_clean_scrub_stamp":"2024-02-23T00:47:27.346540+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":410,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.324490+0000","last_change":"2024-02-23T00:46:47.919416+0000","last_active":"2024-02-23T00:48:28.324490+0000","last_peered":"2024-02-23T00:48:28.324490+0000","last_clean":"2024-02-23T00:48:28.324490+0000","last_became_active":"2024-02-23T00:43:27.611660+0000","last_became_peered":"2024-02-23T00:43:27.611660+0000","last_unstale":"2024-02-23T00:48:28.324490+0000","last_undegraded":"2024-02-23T00:48:28.324490+0000","last_fullsized":"2024-02-23T00:48:28.324490+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:47.919353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:46:47.919353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":347,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.467658+0000","last_change":"2024-02-23T00:47:28.232007+0000","last_active":"2024-02-23T00:48:28.467658+0000","last_peered":"2024-02-23T00:48:28.467658+0000","last_clean":"2024-02-23T00:48:28.467658+0000","last_became_active":"2024-02-23T00:47:28.231670+0000","last_became_peered":"2024-02-23T00:47:28.231670+0000","last_unstale":"2024-02-23T00:48:28.467658+0000","last_undegraded":"2024-02-23T00:48:28.467658+0000","last_fullsized":"2024-02-23T00:48:28.467658+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:19.484659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:47:19.484659+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"68'21","reported_seq":436,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.325845+0000","last_change":"2024-02-23T00:46:46.645234+0000","last_active":"2024-02-23T00:48:28.325845+0000","last_peered":"2024-02-23T00:48:28.325845+0000","last_clean":"2024-02-23T00:48:28.325845+0000","last_became_active":"2024-02-23T00:39:16.858760+0000","last_became_peered":"2024-02-23T00:39:16.858760+0000","last_unstale":"2024-02-23T00:48:28.325845+0000","last_undegraded":"2024-02-23T00:48:28.325845+0000","last_fullsized":"2024-02-23T00:48:28.325845+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'21","last_scrub_stamp":"2024-02-23T00:46:46.645153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:46.645153+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":65,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.464436+0000","last_change":"2024-02-23T00:47:28.223061+0000","last_active":"2024-02-23T00:48:28.464436+0000","last_peered":"2024-02-23T00:48:28.464436+0000","last_clean":"2024-02-23T00:48:28.464436+0000","last_became_active":"2024-02-23T00:47:28.219764+0000","last_became_peered":"2024-02-23T00:47:28.219764+0000","last_unstale":"2024-02-23T00:48:28.464436+0000","last_undegraded":"2024-02-23T00:48:28.464436+0000","last_fullsized":"2024-02-23T00:48:28.464436+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:22.376063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:22.376063+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":394,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.325038+0000","last_change":"2024-02-23T00:46:45.913635+0000","last_active":"2024-02-23T00:48:28.325038+0000","last_peered":"2024-02-23T00:48:28.325038+0000","last_clean":"2024-02-23T00:48:28.325038+0000","last_became_active":"2024-02-23T00:39:18.798624+0000","last_became_peered":"2024-02-23T00:39:18.798624+0000","last_unstale":"2024-02-23T00:48:28.325038+0000","last_undegraded":"2024-02-23T00:48:28.325038+0000","last_fullsized":"2024-02-23T00:48:28.325038+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:45.913526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:45.913526+0000","last_clean_scrub_stamp":"2024-02-23T00:46:45.913526+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":338,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.323536+0000","last_change":"2024-02-23T00:47:24.792535+0000","last_active":"2024-02-23T00:48:28.323536+0000","last_peered":"2024-02-23T00:48:28.323536+0000","last_clean":"2024-02-23T00:48:28.323536+0000","last_became_active":"2024-02-23T00:42:58.825015+0000","last_became_peered":"2024-02-23T00:42:58.825015+0000","last_unstale":"2024-02-23T00:48:28.323536+0000","last_undegraded":"2024-02-23T00:48:28.323536+0000","last_fullsized":"2024-02-23T00:48:28.323536+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:24.792464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:47:24.792464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"107'16","reported_seq":424,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.324962+0000","last_change":"2024-02-23T00:48:07.941205+0000","last_active":"2024-02-23T00:48:28.324962+0000","last_peered":"2024-02-23T00:48:28.324962+0000","last_clean":"2024-02-23T00:48:28.324962+0000","last_became_active":"2024-02-23T00:39:16.858449+0000","last_became_peered":"2024-02-23T00:39:16.858449+0000","last_unstale":"2024-02-23T00:48:28.324962+0000","last_undegraded":"2024-02-23T00:48:28.324962+0000","last_fullsized":"2024-02-23T00:48:28.324962+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'16","last_scrub_stamp":"2024-02-23T00:48:07.941101+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-02-23T00:48:07.941101+0000","last_clean_scrub_stamp":"2024-02-23T00:48:07.941101+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":394,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.467210+0000","last_change":"2024-02-23T00:47:11.419515+0000","last_active":"2024-02-23T00:48:28.467210+0000","last_peered":"2024-02-23T00:48:28.467210+0000","last_clean":"2024-02-23T00:48:28.467210+0000","last_became_active":"2024-02-23T00:39:14.783164+0000","last_became_peered":"2024-02-23T00:39:14.783164+0000","last_unstale":"2024-02-23T00:48:28.467210+0000","last_undegraded":"2024-02-23T00:48:28.467210+0000","last_fullsized":"2024-02-23T00:48:28.467210+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:11.419452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:47:11.419452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":392,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.468599+0000","last_change":"2024-02-23T00:46:42.258031+0000","last_active":"2024-02-23T00:48:28.468599+0000","last_peered":"2024-02-23T00:48:28.468599+0000","last_clean":"2024-02-23T00:48:28.468599+0000","last_became_active":"2024-02-23T00:39:18.800297+0000","last_became_peered":"2024-02-23T00:39:18.800297+0000","last_unstale":"2024-02-23T00:48:28.468599+0000","last_undegraded":"2024-02-23T00:48:28.468599+0000","last_fullsized":"2024-02-23T00:48:28.468599+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:42.257958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:42.257958+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1","version":"110'7","reported_seq":347,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.326803+0000","last_change":"2024-02-23T00:45:18.947859+0000","last_active":"2024-02-23T00:48:28.326803+0000","last_peered":"2024-02-23T00:48:28.326803+0000","last_clean":"2024-02-23T00:48:28.326803+0000","last_became_active":"2024-02-23T00:42:58.777251+0000","last_became_peered":"2024-02-23T00:42:58.777251+0000","last_unstale":"2024-02-23T00:48:28.326803+0000","last_undegraded":"2024-02-23T00:48:28.326803+0000","last_fullsized":"2024-02-23T00:48:28.326803+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'7","last_scrub_stamp":"2024-02-23T00:45:18.947746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:18.947746+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"68'22","reported_seq":79,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.327193+0000","last_change":"2024-02-23T00:47:28.280794+0000","last_active":"2024-02-23T00:48:28.327193+0000","last_peered":"2024-02-23T00:48:28.327193+0000","last_clean":"2024-02-23T00:48:28.327193+0000","last_became_active":"2024-02-23T00:47:28.237898+0000","last_became_peered":"2024-02-23T00:47:28.237898+0000","last_unstale":"2024-02-23T00:48:28.327193+0000","last_undegraded":"2024-02-23T00:48:28.327193+0000","last_fullsized":"2024-02-23T00:48:28.327193+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'22","last_scrub_stamp":"2024-02-23T00:47:16.731630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:47:16.731630+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":37,"num_read_kb":30,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":396,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.465259+0000","last_change":"2024-02-23T00:46:46.201157+0000","last_active":"2024-02-23T00:48:28.465259+0000","last_peered":"2024-02-23T00:48:28.465259+0000","last_clean":"2024-02-23T00:48:28.465259+0000","last_became_active":"2024-02-23T00:39:14.785590+0000","last_became_peered":"2024-02-23T00:39:14.785590+0000","last_unstale":"2024-02-23T00:48:28.465259+0000","last_undegraded":"2024-02-23T00:48:28.465259+0000","last_fullsized":"2024-02-23T00:48:28.465259+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:46.201063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_clean_scrub_stamp":"2024-02-23T00:46:46.201063+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"336'9","reported_seq":512,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.466164+0000","last_change":"2024-02-23T00:46:49.161082+0000","last_active":"2024-02-23T00:48:28.466164+0000","last_peered":"2024-02-23T00:48:28.466164+0000","last_clean":"2024-02-23T00:48:28.466164+0000","last_became_active":"2024-02-23T00:39:18.803566+0000","last_became_peered":"2024-02-23T00:39:18.803566+0000","last_unstale":"2024-02-23T00:48:28.466164+0000","last_undegraded":"2024-02-23T00:48:28.466164+0000","last_fullsized":"2024-02-23T00:48:28.466164+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"336'9","last_scrub_stamp":"2024-02-23T00:46:49.160991+0000","last_deep_scrub":"336'9","last_deep_scrub_stamp":"2024-02-23T00:46:49.160991+0000","last_clean_scrub_stamp":"2024-02-23T00:46:49.160991+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":333,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.466396+0000","last_change":"2024-02-23T00:45:54.753371+0000","last_active":"2024-02-23T00:48:28.466396+0000","last_peered":"2024-02-23T00:48:28.466396+0000","last_clean":"2024-02-23T00:48:28.466396+0000","last_became_active":"2024-02-23T00:42:58.757424+0000","last_became_peered":"2024-02-23T00:42:58.757424+0000","last_unstale":"2024-02-23T00:48:28.466396+0000","last_undegraded":"2024-02-23T00:48:28.466396+0000","last_fullsized":"2024-02-23T00:48:28.466396+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:54.753267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:54.753267+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"68'15","reported_seq":425,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.465985+0000","last_change":"2024-02-23T00:47:15.239499+0000","last_active":"2024-02-23T00:48:28.465985+0000","last_peered":"2024-02-23T00:48:28.465985+0000","last_clean":"2024-02-23T00:48:28.465985+0000","last_became_active":"2024-02-23T00:39:16.858953+0000","last_became_peered":"2024-02-23T00:39:16.858953+0000","last_unstale":"2024-02-23T00:48:28.465985+0000","last_undegraded":"2024-02-23T00:48:28.465985+0000","last_fullsized":"2024-02-23T00:48:28.465985+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:47:15.239429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:47:15.239429+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":397,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.326739+0000","last_change":"2024-02-23T00:46:42.692530+0000","last_active":"2024-02-23T00:48:28.326739+0000","last_peered":"2024-02-23T00:48:28.326739+0000","last_clean":"2024-02-23T00:48:28.326739+0000","last_became_active":"2024-02-23T00:39:14.773772+0000","last_became_peered":"2024-02-23T00:39:14.773772+0000","last_unstale":"2024-02-23T00:48:28.326739+0000","last_undegraded":"2024-02-23T00:48:28.326739+0000","last_fullsized":"2024-02-23T00:48:28.326739+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:42.692434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:42.692434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"376'9","reported_seq":521,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.327519+0000","last_change":"2024-02-23T00:47:28.269858+0000","last_active":"2024-02-23T00:48:28.327519+0000","last_peered":"2024-02-23T00:48:28.327519+0000","last_clean":"2024-02-23T00:48:28.327519+0000","last_became_active":"2024-02-23T00:47:28.229316+0000","last_became_peered":"2024-02-23T00:47:28.229316+0000","last_unstale":"2024-02-23T00:48:28.327519+0000","last_undegraded":"2024-02-23T00:48:28.327519+0000","last_fullsized":"2024-02-23T00:48:28.327519+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:46:36.900988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:36.900988+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":332,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.464880+0000","last_change":"2024-02-23T00:46:51.254983+0000","last_active":"2024-02-23T00:48:28.464880+0000","last_peered":"2024-02-23T00:48:28.464880+0000","last_clean":"2024-02-23T00:48:28.464880+0000","last_became_active":"2024-02-23T00:42:57.652488+0000","last_became_peered":"2024-02-23T00:42:57.652488+0000","last_unstale":"2024-02-23T00:48:28.464880+0000","last_undegraded":"2024-02-23T00:48:28.464880+0000","last_fullsized":"2024-02-23T00:48:28.464880+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:51.254903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:51.254903+0000","last_clean_scrub_stamp":"2024-02-23T00:46:51.254903+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"67'9","reported_seq":423,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.464810+0000","last_change":"2024-02-23T00:47:28.646843+0000","last_active":"2024-02-23T00:48:28.464810+0000","last_peered":"2024-02-23T00:48:28.464810+0000","last_clean":"2024-02-23T00:48:28.464810+0000","last_became_active":"2024-02-23T00:47:28.225585+0000","last_became_peered":"2024-02-23T00:47:28.225585+0000","last_unstale":"2024-02-23T00:48:28.464810+0000","last_undegraded":"2024-02-23T00:48:28.464810+0000","last_fullsized":"2024-02-23T00:48:28.464810+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'9","last_scrub_stamp":"2024-02-23T00:46:36.406771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:36.406771+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":399,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.465985+0000","last_change":"2024-02-23T00:46:49.393060+0000","last_active":"2024-02-23T00:48:28.465985+0000","last_peered":"2024-02-23T00:48:28.465985+0000","last_clean":"2024-02-23T00:48:28.465985+0000","last_became_active":"2024-02-23T00:39:14.778455+0000","last_became_peered":"2024-02-23T00:39:14.778455+0000","last_unstale":"2024-02-23T00:48:28.465985+0000","last_undegraded":"2024-02-23T00:48:28.465985+0000","last_fullsized":"2024-02-23T00:48:28.465985+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:49.392993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:49.392993+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":403,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.324400+0000","last_change":"2024-02-23T00:47:59.075510+0000","last_active":"2024-02-23T00:48:28.324400+0000","last_peered":"2024-02-23T00:48:28.324400+0000","last_clean":"2024-02-23T00:48:28.324400+0000","last_became_active":"2024-02-23T00:47:44.225379+0000","last_became_peered":"2024-02-23T00:47:44.225379+0000","last_unstale":"2024-02-23T00:48:28.324400+0000","last_undegraded":"2024-02-23T00:48:28.324400+0000","last_fullsized":"2024-02-23T00:48:28.324400+0000","mapping_epoch":391,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":392,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:59.075447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_clean_scrub_stamp":"2024-02-23T00:47:59.075447+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":338,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.324826+0000","last_change":"2024-02-23T00:47:19.760299+0000","last_active":"2024-02-23T00:48:28.324826+0000","last_peered":"2024-02-23T00:48:28.324826+0000","last_clean":"2024-02-23T00:48:28.324826+0000","last_became_active":"2024-02-23T00:42:58.835862+0000","last_became_peered":"2024-02-23T00:42:58.835862+0000","last_unstale":"2024-02-23T00:48:28.324826+0000","last_undegraded":"2024-02-23T00:48:28.324826+0000","last_fullsized":"2024-02-23T00:48:28.324826+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:19.760235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:47:19.760235+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"67'11","reported_seq":408,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.324133+0000","last_change":"2024-02-23T00:45:13.315510+0000","last_active":"2024-02-23T00:48:28.324133+0000","last_peered":"2024-02-23T00:48:28.324133+0000","last_clean":"2024-02-23T00:48:28.324133+0000","last_became_active":"2024-02-23T00:39:16.783896+0000","last_became_peered":"2024-02-23T00:39:16.783896+0000","last_unstale":"2024-02-23T00:48:28.324133+0000","last_undegraded":"2024-02-23T00:48:28.324133+0000","last_fullsized":"2024-02-23T00:48:28.324133+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'11","last_scrub_stamp":"2024-02-23T00:45:13.315434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:13.315434+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"202'2","reported_seq":403,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.465077+0000","last_change":"2024-02-23T00:48:04.832975+0000","last_active":"2024-02-23T00:48:28.465077+0000","last_peered":"2024-02-23T00:48:28.465077+0000","last_clean":"2024-02-23T00:48:28.465077+0000","last_became_active":"2024-02-23T00:39:14.775804+0000","last_became_peered":"2024-02-23T00:39:14.775804+0000","last_unstale":"2024-02-23T00:48:28.465077+0000","last_undegraded":"2024-02-23T00:48:28.465077+0000","last_fullsized":"2024-02-23T00:48:28.465077+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"202'2","last_scrub_stamp":"2024-02-23T00:48:04.832904+0000","last_deep_scrub":"202'2","last_deep_scrub_stamp":"2024-02-23T00:44:34.720640+0000","last_clean_scrub_stamp":"2024-02-23T00:48:04.832904+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":386,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.464839+0000","last_change":"2024-02-23T00:47:22.371575+0000","last_active":"2024-02-23T00:48:28.464839+0000","last_peered":"2024-02-23T00:48:28.464839+0000","last_clean":"2024-02-23T00:48:28.464839+0000","last_became_active":"2024-02-23T00:39:18.800382+0000","last_became_peered":"2024-02-23T00:39:18.800382+0000","last_unstale":"2024-02-23T00:48:28.464839+0000","last_undegraded":"2024-02-23T00:48:28.464839+0000","last_fullsized":"2024-02-23T00:48:28.464839+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:22.371493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:47:22.371493+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":341,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.465055+0000","last_change":"2024-02-23T00:47:28.225228+0000","last_active":"2024-02-23T00:48:28.465055+0000","last_peered":"2024-02-23T00:48:28.465055+0000","last_clean":"2024-02-23T00:48:28.465055+0000","last_became_active":"2024-02-23T00:47:28.224961+0000","last_became_peered":"2024-02-23T00:47:28.224961+0000","last_unstale":"2024-02-23T00:48:28.465055+0000","last_undegraded":"2024-02-23T00:48:28.465055+0000","last_fullsized":"2024-02-23T00:48:28.465055+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:49.243955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:49.243955+0000","last_clean_scrub_stamp":"2024-02-23T00:46:49.243955+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"67'4","reported_seq":402,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.322993+0000","last_change":"2024-02-23T00:48:14.033151+0000","last_active":"2024-02-23T00:48:28.322993+0000","last_peered":"2024-02-23T00:48:28.322993+0000","last_clean":"2024-02-23T00:48:28.322993+0000","last_became_active":"2024-02-23T00:39:16.785830+0000","last_became_peered":"2024-02-23T00:39:16.785830+0000","last_unstale":"2024-02-23T00:48:28.322993+0000","last_undegraded":"2024-02-23T00:48:28.322993+0000","last_fullsized":"2024-02-23T00:48:28.322993+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'4","last_scrub_stamp":"2024-02-23T00:48:14.033041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:48:14.033041+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":408,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.468248+0000","last_change":"2024-02-23T00:47:28.224795+0000","last_active":"2024-02-23T00:48:28.468248+0000","last_peered":"2024-02-23T00:48:28.468248+0000","last_clean":"2024-02-23T00:48:28.468248+0000","last_became_active":"2024-02-23T00:47:28.224477+0000","last_became_peered":"2024-02-23T00:47:28.224477+0000","last_unstale":"2024-02-23T00:48:28.468248+0000","last_undegraded":"2024-02-23T00:48:28.468248+0000","last_fullsized":"2024-02-23T00:48:28.468248+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:35.268491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:35.268491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":404,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.465311+0000","last_change":"2024-02-23T00:47:44.230641+0000","last_active":"2024-02-23T00:48:28.465311+0000","last_peered":"2024-02-23T00:48:28.465311+0000","last_clean":"2024-02-23T00:48:28.465311+0000","last_became_active":"2024-02-23T00:47:44.230295+0000","last_became_peered":"2024-02-23T00:47:44.230295+0000","last_unstale":"2024-02-23T00:48:28.465311+0000","last_undegraded":"2024-02-23T00:48:28.465311+0000","last_fullsized":"2024-02-23T00:48:28.465311+0000","mapping_epoch":391,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":392,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:12.411247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:47:12.411247+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":333,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.467183+0000","last_change":"2024-02-23T00:45:56.706901+0000","last_active":"2024-02-23T00:48:28.467183+0000","last_peered":"2024-02-23T00:48:28.467183+0000","last_clean":"2024-02-23T00:48:28.467183+0000","last_became_active":"2024-02-23T00:42:58.741365+0000","last_became_peered":"2024-02-23T00:42:58.741365+0000","last_unstale":"2024-02-23T00:48:28.467183+0000","last_undegraded":"2024-02-23T00:48:28.467183+0000","last_fullsized":"2024-02-23T00:48:28.467183+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:56.706792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:56.706792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"68'17","reported_seq":430,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.326909+0000","last_change":"2024-02-23T00:46:49.921614+0000","last_active":"2024-02-23T00:48:28.326909+0000","last_peered":"2024-02-23T00:48:28.326909+0000","last_clean":"2024-02-23T00:48:28.326909+0000","last_became_active":"2024-02-23T00:39:16.857232+0000","last_became_peered":"2024-02-23T00:39:16.857232+0000","last_unstale":"2024-02-23T00:48:28.326909+0000","last_undegraded":"2024-02-23T00:48:28.326909+0000","last_fullsized":"2024-02-23T00:48:28.326909+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:46:49.921543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:49.921543+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":403,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.467143+0000","last_change":"2024-02-23T00:47:28.240576+0000","last_active":"2024-02-23T00:48:28.467143+0000","last_peered":"2024-02-23T00:48:28.467143+0000","last_clean":"2024-02-23T00:48:28.467143+0000","last_became_active":"2024-02-23T00:47:28.240162+0000","last_became_peered":"2024-02-23T00:47:28.240162+0000","last_unstale":"2024-02-23T00:48:28.467143+0000","last_undegraded":"2024-02-23T00:48:28.467143+0000","last_fullsized":"2024-02-23T00:48:28.467143+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:44.285083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:44.285083+0000","last_clean_scrub_stamp":"2024-02-23T00:46:44.285083+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":47,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.322816+0000","last_change":"2024-02-23T00:47:44.226489+0000","last_active":"2024-02-23T00:48:28.322816+0000","last_peered":"2024-02-23T00:48:28.322816+0000","last_clean":"2024-02-23T00:48:28.322816+0000","last_became_active":"2024-02-23T00:47:44.226097+0000","last_became_peered":"2024-02-23T00:47:44.226097+0000","last_unstale":"2024-02-23T00:48:28.322816+0000","last_undegraded":"2024-02-23T00:48:28.322816+0000","last_fullsized":"2024-02-23T00:48:28.322816+0000","mapping_epoch":391,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":392,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:39.715957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:39.715957+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":70,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.468794+0000","last_change":"2024-02-23T00:48:03.345663+0000","last_active":"2024-02-23T00:48:28.468794+0000","last_peered":"2024-02-23T00:48:28.468794+0000","last_clean":"2024-02-23T00:48:28.468794+0000","last_became_active":"2024-02-23T00:47:28.200688+0000","last_became_peered":"2024-02-23T00:47:28.200688+0000","last_unstale":"2024-02-23T00:48:28.468794+0000","last_undegraded":"2024-02-23T00:48:28.468794+0000","last_fullsized":"2024-02-23T00:48:28.468794+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:03.345545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:03.345545+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"68'16","reported_seq":425,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.326090+0000","last_change":"2024-02-23T00:47:58.981341+0000","last_active":"2024-02-23T00:48:28.326090+0000","last_peered":"2024-02-23T00:48:28.326090+0000","last_clean":"2024-02-23T00:48:28.326090+0000","last_became_active":"2024-02-23T00:39:16.858945+0000","last_became_peered":"2024-02-23T00:39:16.858945+0000","last_unstale":"2024-02-23T00:48:28.326090+0000","last_undegraded":"2024-02-23T00:48:28.326090+0000","last_fullsized":"2024-02-23T00:48:28.326090+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:47:58.981233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:47:58.981233+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":403,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.465418+0000","last_change":"2024-02-23T00:46:09.088334+0000","last_active":"2024-02-23T00:48:28.465418+0000","last_peered":"2024-02-23T00:48:28.465418+0000","last_clean":"2024-02-23T00:48:28.465418+0000","last_became_active":"2024-02-23T00:39:14.785363+0000","last_became_peered":"2024-02-23T00:39:14.785363+0000","last_unstale":"2024-02-23T00:48:28.465418+0000","last_undegraded":"2024-02-23T00:48:28.465418+0000","last_fullsized":"2024-02-23T00:48:28.465418+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"80'2","last_scrub_stamp":"2024-02-23T00:46:09.088249+0000","last_deep_scrub":"80'2","last_deep_scrub_stamp":"2024-02-23T00:40:15.942725+0000","last_clean_scrub_stamp":"2024-02-23T00:46:09.088249+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"93'9","reported_seq":513,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.467284+0000","last_change":"2024-02-23T00:46:37.230102+0000","last_active":"2024-02-23T00:48:28.467284+0000","last_peered":"2024-02-23T00:48:28.467284+0000","last_clean":"2024-02-23T00:48:28.467284+0000","last_became_active":"2024-02-23T00:39:18.796680+0000","last_became_peered":"2024-02-23T00:39:18.796680+0000","last_unstale":"2024-02-23T00:48:28.467284+0000","last_undegraded":"2024-02-23T00:48:28.467284+0000","last_fullsized":"2024-02-23T00:48:28.467284+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-02-23T00:46:37.230030+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-02-23T00:43:50.393713+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.230030+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":347,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.323723+0000","last_change":"2024-02-23T00:48:17.898787+0000","last_active":"2024-02-23T00:48:28.323723+0000","last_peered":"2024-02-23T00:48:28.323723+0000","last_clean":"2024-02-23T00:48:28.323723+0000","last_became_active":"2024-02-23T00:47:28.234641+0000","last_became_peered":"2024-02-23T00:47:28.234641+0000","last_unstale":"2024-02-23T00:48:28.323723+0000","last_undegraded":"2024-02-23T00:48:28.323723+0000","last_fullsized":"2024-02-23T00:48:28.323723+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:17.898720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:17.898720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"93'23","reported_seq":437,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.467397+0000","last_change":"2024-02-23T00:46:51.431911+0000","last_active":"2024-02-23T00:48:28.467397+0000","last_peered":"2024-02-23T00:48:28.467397+0000","last_clean":"2024-02-23T00:48:28.467397+0000","last_became_active":"2024-02-23T00:39:16.782660+0000","last_became_peered":"2024-02-23T00:39:16.782660+0000","last_unstale":"2024-02-23T00:48:28.467397+0000","last_undegraded":"2024-02-23T00:48:28.467397+0000","last_fullsized":"2024-02-23T00:48:28.467397+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'23","last_scrub_stamp":"2024-02-23T00:46:51.431838+0000","last_deep_scrub":"93'23","last_deep_scrub_stamp":"2024-02-23T00:41:35.954516+0000","last_clean_scrub_stamp":"2024-02-23T00:46:51.431838+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":398,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.326593+0000","last_change":"2024-02-23T00:47:20.033447+0000","last_active":"2024-02-23T00:48:28.326593+0000","last_peered":"2024-02-23T00:48:28.326593+0000","last_clean":"2024-02-23T00:48:28.326593+0000","last_became_active":"2024-02-23T00:39:14.781560+0000","last_became_peered":"2024-02-23T00:39:14.781560+0000","last_unstale":"2024-02-23T00:48:28.326593+0000","last_undegraded":"2024-02-23T00:48:28.326593+0000","last_fullsized":"2024-02-23T00:48:28.326593+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:20.033380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:47:20.033380+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":402,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.326175+0000","last_change":"2024-02-23T00:47:28.233334+0000","last_active":"2024-02-23T00:48:28.326175+0000","last_peered":"2024-02-23T00:48:28.326175+0000","last_clean":"2024-02-23T00:48:28.326175+0000","last_became_active":"2024-02-23T00:47:28.232797+0000","last_became_peered":"2024-02-23T00:47:28.232797+0000","last_unstale":"2024-02-23T00:48:28.326175+0000","last_undegraded":"2024-02-23T00:48:28.326175+0000","last_fullsized":"2024-02-23T00:48:28.326175+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:36.592887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:53.189881+0000","last_clean_scrub_stamp":"2024-02-23T00:46:36.592887+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":339,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.326633+0000","last_change":"2024-02-23T00:48:05.979406+0000","last_active":"2024-02-23T00:48:28.326633+0000","last_peered":"2024-02-23T00:48:28.326633+0000","last_clean":"2024-02-23T00:48:28.326633+0000","last_became_active":"2024-02-23T00:42:58.774720+0000","last_became_peered":"2024-02-23T00:42:58.774720+0000","last_unstale":"2024-02-23T00:48:28.326633+0000","last_undegraded":"2024-02-23T00:48:28.326633+0000","last_fullsized":"2024-02-23T00:48:28.326633+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:05.979339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:05.979339+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"67'15","reported_seq":118,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.465503+0000","last_change":"2024-02-23T00:47:30.032276+0000","last_active":"2024-02-23T00:48:28.465503+0000","last_peered":"2024-02-23T00:48:28.465503+0000","last_clean":"2024-02-23T00:48:28.465503+0000","last_became_active":"2024-02-23T00:47:28.211342+0000","last_became_peered":"2024-02-23T00:47:28.211342+0000","last_unstale":"2024-02-23T00:48:28.465503+0000","last_undegraded":"2024-02-23T00:48:28.465503+0000","last_fullsized":"2024-02-23T00:48:28.465503+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:46:03.951524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:03.951524+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":399,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.468019+0000","last_change":"2024-02-23T00:48:10.563771+0000","last_active":"2024-02-23T00:48:28.468019+0000","last_peered":"2024-02-23T00:48:28.468019+0000","last_clean":"2024-02-23T00:48:28.468019+0000","last_became_active":"2024-02-23T00:39:14.782494+0000","last_became_peered":"2024-02-23T00:39:14.782494+0000","last_unstale":"2024-02-23T00:48:28.468019+0000","last_undegraded":"2024-02-23T00:48:28.468019+0000","last_fullsized":"2024-02-23T00:48:28.468019+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:10.563695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:48:10.563695+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":391,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.465174+0000","last_change":"2024-02-23T00:48:16.788875+0000","last_active":"2024-02-23T00:48:28.465174+0000","last_peered":"2024-02-23T00:48:28.465174+0000","last_clean":"2024-02-23T00:48:28.465174+0000","last_became_active":"2024-02-23T00:39:18.801039+0000","last_became_peered":"2024-02-23T00:39:18.801039+0000","last_unstale":"2024-02-23T00:48:28.465174+0000","last_undegraded":"2024-02-23T00:48:28.465174+0000","last_fullsized":"2024-02-23T00:48:28.465174+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:16.788806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.641689+0000","last_clean_scrub_stamp":"2024-02-23T00:48:16.788806+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":339,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.323838+0000","last_change":"2024-02-23T00:48:10.933441+0000","last_active":"2024-02-23T00:48:28.323838+0000","last_peered":"2024-02-23T00:48:28.323838+0000","last_clean":"2024-02-23T00:48:28.323838+0000","last_became_active":"2024-02-23T00:42:58.770759+0000","last_became_peered":"2024-02-23T00:42:58.770759+0000","last_unstale":"2024-02-23T00:48:28.323838+0000","last_undegraded":"2024-02-23T00:48:28.323838+0000","last_fullsized":"2024-02-23T00:48:28.323838+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:10.933375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:10.933375+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"68'18","reported_seq":434,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.326217+0000","last_change":"2024-02-23T00:46:44.911346+0000","last_active":"2024-02-23T00:48:28.326217+0000","last_peered":"2024-02-23T00:48:28.326217+0000","last_clean":"2024-02-23T00:48:28.326217+0000","last_became_active":"2024-02-23T00:39:16.782803+0000","last_became_peered":"2024-02-23T00:39:16.782803+0000","last_unstale":"2024-02-23T00:48:28.326217+0000","last_undegraded":"2024-02-23T00:48:28.326217+0000","last_fullsized":"2024-02-23T00:48:28.326217+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'18","last_scrub_stamp":"2024-02-23T00:46:44.911278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:44.911278+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":398,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.324323+0000","last_change":"2024-02-23T00:47:16.017278+0000","last_active":"2024-02-23T00:48:28.324323+0000","last_peered":"2024-02-23T00:48:28.324323+0000","last_clean":"2024-02-23T00:48:28.324323+0000","last_became_active":"2024-02-23T00:39:14.782163+0000","last_became_peered":"2024-02-23T00:39:14.782163+0000","last_unstale":"2024-02-23T00:48:28.324323+0000","last_undegraded":"2024-02-23T00:48:28.324323+0000","last_fullsized":"2024-02-23T00:48:28.324323+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:16.017210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:47:16.017210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":401,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.324365+0000","last_change":"2024-02-23T00:48:03.533673+0000","last_active":"2024-02-23T00:48:28.324365+0000","last_peered":"2024-02-23T00:48:28.324365+0000","last_clean":"2024-02-23T00:48:28.324365+0000","last_became_active":"2024-02-23T00:47:28.215452+0000","last_became_peered":"2024-02-23T00:47:28.215452+0000","last_unstale":"2024-02-23T00:48:28.324365+0000","last_undegraded":"2024-02-23T00:48:28.324365+0000","last_fullsized":"2024-02-23T00:48:28.324365+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:03.533561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_clean_scrub_stamp":"2024-02-23T00:48:03.533561+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":341,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.467192+0000","last_change":"2024-02-23T00:47:28.204445+0000","last_active":"2024-02-23T00:48:28.467192+0000","last_peered":"2024-02-23T00:48:28.467192+0000","last_clean":"2024-02-23T00:48:28.467192+0000","last_became_active":"2024-02-23T00:47:28.203555+0000","last_became_peered":"2024-02-23T00:47:28.203555+0000","last_unstale":"2024-02-23T00:48:28.467192+0000","last_undegraded":"2024-02-23T00:48:28.467192+0000","last_fullsized":"2024-02-23T00:48:28.467192+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:00.943545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:46:00.943545+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"95'27","reported_seq":448,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.323610+0000","last_change":"2024-02-23T00:47:14.328310+0000","last_active":"2024-02-23T00:48:28.323610+0000","last_peered":"2024-02-23T00:48:28.323610+0000","last_clean":"2024-02-23T00:48:28.323610+0000","last_became_active":"2024-02-23T00:39:16.786791+0000","last_became_peered":"2024-02-23T00:39:16.786791+0000","last_unstale":"2024-02-23T00:48:28.323610+0000","last_undegraded":"2024-02-23T00:48:28.323610+0000","last_fullsized":"2024-02-23T00:48:28.323610+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'27","last_scrub_stamp":"2024-02-23T00:47:14.328242+0000","last_deep_scrub":"95'27","last_deep_scrub_stamp":"2024-02-23T00:41:55.919895+0000","last_clean_scrub_stamp":"2024-02-23T00:47:14.328242+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":404,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.467042+0000","last_change":"2024-02-23T00:47:28.209244+0000","last_active":"2024-02-23T00:48:28.467042+0000","last_peered":"2024-02-23T00:48:28.467042+0000","last_clean":"2024-02-23T00:48:28.467042+0000","last_became_active":"2024-02-23T00:47:28.208293+0000","last_became_peered":"2024-02-23T00:47:28.208293+0000","last_unstale":"2024-02-23T00:48:28.467042+0000","last_undegraded":"2024-02-23T00:48:28.467042+0000","last_fullsized":"2024-02-23T00:48:28.467042+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:47.689132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:47.689132+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":392,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.467057+0000","last_change":"2024-02-23T00:47:22.313110+0000","last_active":"2024-02-23T00:48:28.467057+0000","last_peered":"2024-02-23T00:48:28.467057+0000","last_clean":"2024-02-23T00:48:28.467057+0000","last_became_active":"2024-02-23T00:39:18.796431+0000","last_became_peered":"2024-02-23T00:39:18.796431+0000","last_unstale":"2024-02-23T00:48:28.467057+0000","last_undegraded":"2024-02-23T00:48:28.467057+0000","last_fullsized":"2024-02-23T00:48:28.467057+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:22.313004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:47:22.313004+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":337,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.466347+0000","last_change":"2024-02-23T00:48:16.383542+0000","last_active":"2024-02-23T00:48:28.466347+0000","last_peered":"2024-02-23T00:48:28.466347+0000","last_clean":"2024-02-23T00:48:28.466347+0000","last_became_active":"2024-02-23T00:42:58.832885+0000","last_became_peered":"2024-02-23T00:42:58.832885+0000","last_unstale":"2024-02-23T00:48:28.466347+0000","last_undegraded":"2024-02-23T00:48:28.466347+0000","last_fullsized":"2024-02-23T00:48:28.466347+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:16.383463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:16.383463+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"82'15","reported_seq":423,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.324327+0000","last_change":"2024-02-23T00:47:25.380130+0000","last_active":"2024-02-23T00:48:28.324327+0000","last_peered":"2024-02-23T00:48:28.324327+0000","last_clean":"2024-02-23T00:48:28.324327+0000","last_became_active":"2024-02-23T00:39:16.782762+0000","last_became_peered":"2024-02-23T00:39:16.782762+0000","last_unstale":"2024-02-23T00:48:28.324327+0000","last_undegraded":"2024-02-23T00:48:28.324327+0000","last_fullsized":"2024-02-23T00:48:28.324327+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'15","last_scrub_stamp":"2024-02-23T00:47:25.380060+0000","last_deep_scrub":"82'15","last_deep_scrub_stamp":"2024-02-23T00:41:50.975683+0000","last_clean_scrub_stamp":"2024-02-23T00:47:25.380060+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":395,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.468667+0000","last_change":"2024-02-23T00:46:08.794420+0000","last_active":"2024-02-23T00:48:28.468667+0000","last_peered":"2024-02-23T00:48:28.468667+0000","last_clean":"2024-02-23T00:48:28.468667+0000","last_became_active":"2024-02-23T00:39:14.783469+0000","last_became_peered":"2024-02-23T00:39:14.783469+0000","last_unstale":"2024-02-23T00:48:28.468667+0000","last_undegraded":"2024-02-23T00:48:28.468667+0000","last_fullsized":"2024-02-23T00:48:28.468667+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:08.794352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:08.794352+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":393,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.466053+0000","last_change":"2024-02-23T00:46:09.832330+0000","last_active":"2024-02-23T00:48:28.466053+0000","last_peered":"2024-02-23T00:48:28.466053+0000","last_clean":"2024-02-23T00:48:28.466053+0000","last_became_active":"2024-02-23T00:39:18.796952+0000","last_became_peered":"2024-02-23T00:39:18.796952+0000","last_unstale":"2024-02-23T00:48:28.466053+0000","last_undegraded":"2024-02-23T00:48:28.466053+0000","last_fullsized":"2024-02-23T00:48:28.466053+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:09.832253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:09.832253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":335,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.324212+0000","last_change":"2024-02-23T00:45:23.224771+0000","last_active":"2024-02-23T00:48:28.324212+0000","last_peered":"2024-02-23T00:48:28.324212+0000","last_clean":"2024-02-23T00:48:28.324212+0000","last_became_active":"2024-02-23T00:42:58.758147+0000","last_became_peered":"2024-02-23T00:42:58.758147+0000","last_unstale":"2024-02-23T00:48:28.324212+0000","last_undegraded":"2024-02-23T00:48:28.324212+0000","last_fullsized":"2024-02-23T00:48:28.324212+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:23.224663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:23.224663+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.224663+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"68'15","reported_seq":425,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.323390+0000","last_change":"2024-02-23T00:47:19.331746+0000","last_active":"2024-02-23T00:48:28.323390+0000","last_peered":"2024-02-23T00:48:28.323390+0000","last_clean":"2024-02-23T00:48:28.323390+0000","last_became_active":"2024-02-23T00:39:16.784606+0000","last_became_peered":"2024-02-23T00:39:16.784606+0000","last_unstale":"2024-02-23T00:48:28.323390+0000","last_undegraded":"2024-02-23T00:48:28.323390+0000","last_fullsized":"2024-02-23T00:48:28.323390+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:47:19.331675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:47:19.331675+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":398,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.325537+0000","last_change":"2024-02-23T00:47:24.067493+0000","last_active":"2024-02-23T00:48:28.325537+0000","last_peered":"2024-02-23T00:48:28.325537+0000","last_clean":"2024-02-23T00:48:28.325537+0000","last_became_active":"2024-02-23T00:39:14.785362+0000","last_became_peered":"2024-02-23T00:39:14.785362+0000","last_unstale":"2024-02-23T00:48:28.325537+0000","last_undegraded":"2024-02-23T00:48:28.325537+0000","last_fullsized":"2024-02-23T00:48:28.325537+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:24.067421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:47:24.067421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":386,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.464453+0000","last_change":"2024-02-23T00:46:47.226397+0000","last_active":"2024-02-23T00:48:28.464453+0000","last_peered":"2024-02-23T00:48:28.464453+0000","last_clean":"2024-02-23T00:48:28.464453+0000","last_became_active":"2024-02-23T00:39:18.796312+0000","last_became_peered":"2024-02-23T00:39:18.796312+0000","last_unstale":"2024-02-23T00:48:28.464453+0000","last_undegraded":"2024-02-23T00:48:28.464453+0000","last_fullsized":"2024-02-23T00:48:28.464453+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:47.226331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:47.226331+0000","last_clean_scrub_stamp":"2024-02-23T00:46:47.226331+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":338,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.325959+0000","last_change":"2024-02-23T00:47:54.954594+0000","last_active":"2024-02-23T00:48:28.325959+0000","last_peered":"2024-02-23T00:48:28.325959+0000","last_clean":"2024-02-23T00:48:28.325959+0000","last_became_active":"2024-02-23T00:42:58.838257+0000","last_became_peered":"2024-02-23T00:42:58.838257+0000","last_unstale":"2024-02-23T00:48:28.325959+0000","last_undegraded":"2024-02-23T00:48:28.325959+0000","last_fullsized":"2024-02-23T00:48:28.325959+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:54.954501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:47:54.954501+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"88'50","reported_seq":525,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.326996+0000","last_change":"2024-02-23T00:48:03.023256+0000","last_active":"2024-02-23T00:48:28.326996+0000","last_peered":"2024-02-23T00:48:28.326996+0000","last_clean":"2024-02-23T00:48:28.326996+0000","last_became_active":"2024-02-23T00:47:28.205150+0000","last_became_peered":"2024-02-23T00:47:28.205150+0000","last_unstale":"2024-02-23T00:48:28.326996+0000","last_undegraded":"2024-02-23T00:48:28.326996+0000","last_fullsized":"2024-02-23T00:48:28.326996+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'50","last_scrub_stamp":"2024-02-23T00:48:03.023069+0000","last_deep_scrub":"88'50","last_deep_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_clean_scrub_stamp":"2024-02-23T00:48:03.023069+0000","log_size":50,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":397,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.324258+0000","last_change":"2024-02-23T00:48:10.850211+0000","last_active":"2024-02-23T00:48:28.324258+0000","last_peered":"2024-02-23T00:48:28.324258+0000","last_clean":"2024-02-23T00:48:28.324258+0000","last_became_active":"2024-02-23T00:39:14.782000+0000","last_became_peered":"2024-02-23T00:39:14.782000+0000","last_unstale":"2024-02-23T00:48:28.324258+0000","last_undegraded":"2024-02-23T00:48:28.324258+0000","last_fullsized":"2024-02-23T00:48:28.324258+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:10.850134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_clean_scrub_stamp":"2024-02-23T00:48:10.850134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":387,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.323599+0000","last_change":"2024-02-23T00:46:49.271764+0000","last_active":"2024-02-23T00:48:28.323599+0000","last_peered":"2024-02-23T00:48:28.323599+0000","last_clean":"2024-02-23T00:48:28.323599+0000","last_became_active":"2024-02-23T00:39:18.799723+0000","last_became_peered":"2024-02-23T00:39:18.799723+0000","last_unstale":"2024-02-23T00:48:28.323599+0000","last_undegraded":"2024-02-23T00:48:28.323599+0000","last_fullsized":"2024-02-23T00:48:28.323599+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:49.271705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:49.271705+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":337,"reported_epoch":428,"state":"active+clean","last_fresh":"2024-02-23T00:48:28.468707+0000","last_change":"2024-02-23T00:47:19.231154+0000","last_active":"2024-02-23T00:48:28.468707+0000","last_peered":"2024-02-23T00:48:28.468707+0000","last_clean":"2024-02-23T00:48:28.468707+0000","last_became_active":"2024-02-23T00:42:58.824573+0000","last_became_peered":"2024-02-23T00:42:58.824573+0000","last_unstale":"2024-02-23T00:48:28.468707+0000","last_undegraded":"2024-02-23T00:48:28.468707+0000","last_fullsized":"2024-02-23T00:48:28.468707+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:19.231077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:47:19.231077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0],"acting":[5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":212,"num_pg":32,"stat_sum":{"num_bytes":12,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":10,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":36,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":96,"acting":96,"num_store_stats":7},{"poolid":62,"num_pg":32,"stat_sum":{"num_bytes":4001484,"num_objects":159,"num_object_clones":3,"num_object_copies":477,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":107,"num_read_kb":9,"num_write":596,"num_write_kb":4495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":853,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":8,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":13938688,"data_stored":12004529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1883,"ondisk_log_size":1883,"up":96,"acting":96,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":12,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":36,"acting":36,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1079,"num_read_kb":804,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":1486,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":666,"ondisk_log_size":666,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":547,"num_read_kb":542,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234099,"num_pgs":100,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":316632,"kb_used_data":11212,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93428520,"statfs":{"total":95995035648,"available":95670804480,"internally_reserved":0,"allocated":11481088,"data_stored":8434013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:48:29 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54500000000000004,"15min":0.54200000000000004},"min":{"1min":0.38100000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.75800000000000001,"5min":4.9420000000000002,"15min":4.9420000000000002},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54000000000000004,"15min":0.51800000000000002},"min":{"1min":0.32000000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.151,"5min":4.9729999999999999,"15min":4.9729999999999999},"last":0.373}]},{"osd":1,"last update":"Fri Feb 23 00:48:29 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.78500000000000003,"15min":0.623},"min":{"1min":0.37,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.25,"5min":10.786,"15min":10.786},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.78800000000000003,"15min":0.625},"min":{"1min":0.40500000000000003,"5min":0.29399999999999998,"15min":0.28100000000000003},"max":{"1min":1.107,"5min":11.885999999999999,"15min":11.885999999999999},"last":0.44900000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:48:29 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.60099999999999998,"15min":0.55500000000000005},"min":{"1min":0.38600000000000001,"5min":0.315,"15min":0.28999999999999998},"max":{"1min":0.73499999999999999,"5min":5.0789999999999997,"15min":5.0789999999999997},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58099999999999996,"15min":0.55200000000000005},"min":{"1min":0.46400000000000002,"5min":0.316,"15min":0.316},"max":{"1min":1.1180000000000001,"5min":5.1280000000000001,"15min":5.1280000000000001},"last":0.60099999999999998}]},{"osd":3,"last update":"Fri Feb 23 00:48:29 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.79400000000000004,"15min":0.62},"min":{"1min":0.41799999999999998,"5min":0.32200000000000001,"15min":0.32100000000000001},"max":{"1min":1.095,"5min":15.712999999999999,"15min":15.712999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.71099999999999997,"15min":0.61899999999999999},"min":{"1min":0.44900000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.72499999999999998,"5min":9.4090000000000007,"15min":9.4090000000000007},"last":0.64400000000000002}]},{"osd":4,"last update":"Fri Feb 23 00:48:29 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.45400000000000001,"15min":0.441},"min":{"1min":0.28899999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.63100000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":0.624},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.55200000000000005,"15min":0.49099999999999999},"min":{"1min":0.312,"5min":0.248,"15min":0.248},"max":{"1min":0.76500000000000001,"5min":11.236000000000001,"15min":11.236000000000001},"last":0.61099999999999999}]},{"osd":5,"last update":"Fri Feb 23 00:48:29 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.51100000000000001,"15min":0.49299999999999999},"min":{"1min":0.36199999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.68600000000000005,"5min":5.4000000000000004,"15min":5.4000000000000004},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.50800000000000001,"15min":0.497},"min":{"1min":0.373,"5min":0.29099999999999998,"15min":0.26700000000000002},"max":{"1min":0.71899999999999997,"5min":1.728,"15min":1.728},"last":0.51000000000000001}]},{"osd":6,"last update":"Fri Feb 23 00:48:29 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.51000000000000001,"15min":0.51200000000000001},"min":{"1min":0.33700000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.72599999999999998,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.625,"15min":0.54900000000000004},"min":{"1min":0.35199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.67000000000000004,"5min":13.395,"15min":13.395},"last":0.56999999999999995}]}]},{"osd":6,"up_from":47,"seq":201863463034,"num_pgs":83,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308536,"kb_used_data":7208,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93436616,"statfs":{"total":95995035648,"available":95679094784,"internally_reserved":0,"allocated":7380992,"data_stored":4433969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:47:59 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.95499999999999996,"15min":0.69399999999999995},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.86099999999999999,"5min":30.641999999999999,"15min":30.641999999999999},"last":0.625},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.99399999999999999,"15min":0.70999999999999996},"min":{"1min":0.27200000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":3.7360000000000002,"5min":29.292000000000002,"15min":29.292000000000002},"last":0.52000000000000002}]},{"osd":1,"last update":"Fri Feb 23 00:47:59 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.97399999999999998,"15min":0.69099999999999995},"min":{"1min":0.25900000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.77200000000000002,"5min":28.186,"15min":28.186},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":1.46,"15min":0.86899999999999999},"min":{"1min":0.316,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.76200000000000001,"5min":33.104999999999997,"15min":33.104999999999997},"last":0.61199999999999999}]},{"osd":2,"last update":"Fri Feb 23 00:47:59 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.997,"15min":0.72399999999999998},"min":{"1min":0.29399999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.69699999999999995,"5min":19.960000000000001,"15min":19.960000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.81000000000000005,"15min":0.65200000000000002},"min":{"1min":0.316,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.665,"5min":8.5860000000000003,"15min":8.5860000000000003},"last":0.67300000000000004}]},{"osd":3,"last update":"Fri Feb 23 00:47:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.73799999999999999,"15min":0.63200000000000001},"min":{"1min":0.36099999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78400000000000003,"5min":19.562000000000001,"15min":19.562000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.84099999999999997,"15min":0.67900000000000005},"min":{"1min":0.34999999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.6259999999999999,"5min":14.199,"15min":14.199},"last":0.64900000000000002}]},{"osd":4,"last update":"Fri Feb 23 00:47:59 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.58099999999999996,"15min":0.47299999999999998},"min":{"1min":0.249,"5min":0.218,"15min":0.218},"max":{"1min":1.6220000000000001,"5min":11.109999999999999,"15min":11.109999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.73999999999999999,"15min":0.54100000000000004},"min":{"1min":0.308,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.83099999999999996,"5min":16.734000000000002,"15min":16.734000000000002},"last":0.499}]},{"osd":5,"last update":"Fri Feb 23 00:47:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.64300000000000002,"15min":0.5},"min":{"1min":0.32100000000000001,"5min":0.249,"15min":0.249},"max":{"1min":1.784,"5min":14.154999999999999,"15min":14.154999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.70499999999999996,"15min":0.54000000000000004},"min":{"1min":0.33900000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":8.1400000000000006,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.57899999999999996}]},{"osd":7,"last update":"Fri Feb 23 00:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51600000000000001,"15min":0.52800000000000002},"min":{"1min":0.29399999999999998,"5min":0.29299999999999998,"15min":0.26800000000000002},"max":{"1min":0.67600000000000005,"5min":2.6949999999999998,"15min":8.548},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56000000000000005,"15min":0.60099999999999998},"min":{"1min":0.40300000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.70999999999999996,"5min":4.6680000000000001,"15min":15.618},"last":0.72799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574996,"num_pgs":97,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308380,"kb_used_data":11152,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93436772,"statfs":{"total":95995035648,"available":95679254528,"internally_reserved":0,"allocated":11419648,"data_stored":8435598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:47:45 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":1.258,"15min":0.66300000000000003},"min":{"1min":0.224,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.49299999999999999,"5min":78.361000000000004,"15min":78.361000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.52700000000000002,"15min":0.438},"min":{"1min":0.216,"5min":0.216,"15min":0.191},"max":{"1min":0.48799999999999999,"5min":5.8849999999999998,"15min":5.8849999999999998},"last":0.39800000000000002}]},{"osd":2,"last update":"Fri Feb 23 00:48:11 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":1.145,"15min":0.64900000000000002},"min":{"1min":0.23000000000000001,"5min":0.214,"15min":0.188},"max":{"1min":0.71399999999999997,"5min":48.606000000000002,"15min":48.606000000000002},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.84899999999999998,"15min":0.58199999999999996},"min":{"1min":0.27900000000000003,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.80500000000000005,"5min":23.911000000000001,"15min":23.911000000000001},"last":0.46800000000000003}]},{"osd":3,"last update":"Fri Feb 23 00:47:45 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.85099999999999998,"15min":0.60999999999999999},"min":{"1min":0.28899999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.61699999999999999,"5min":35.244999999999997,"15min":35.244999999999997},"last":0.435},{"interface":"front","average":{"1min":0.38200000000000001,"5min":1.1040000000000001,"15min":0.67300000000000004},"min":{"1min":0.28999999999999998,"5min":0.253,"15min":0.248},"max":{"1min":0.54800000000000004,"5min":56.853999999999999,"15min":56.853999999999999},"last":0.45100000000000001}]},{"osd":4,"last update":"Fri Feb 23 00:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":1.111,"15min":0.80900000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.29299999999999998},"max":{"1min":0.94199999999999995,"5min":37.646999999999998,"15min":37.646999999999998},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.79700000000000004,"15min":0.69399999999999995},"min":{"1min":0.37,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.002,"5min":9.6859999999999999,"15min":9.6859999999999999},"last":0.82099999999999995}]},{"osd":5,"last update":"Fri Feb 23 00:48:27 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63,"15min":0.82499999999999996},"min":{"1min":0.33000000000000002,"5min":0.32700000000000001,"15min":0.32600000000000001},"max":{"1min":0.91600000000000004,"5min":5.9500000000000002,"15min":48.792000000000002},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.68000000000000005,"15min":0.72999999999999998},"min":{"1min":0.44700000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83999999999999997,"5min":3.593,"15min":11.872999999999999},"last":0.84199999999999997}]},{"osd":6,"last update":"Fri Feb 23 00:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":1.1419999999999999,"15min":0.81000000000000005},"min":{"1min":0.38100000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.872,"5min":40.741,"15min":40.741},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.71699999999999997,"15min":0.68100000000000005},"min":{"1min":0.38400000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78500000000000003,"5min":6.0069999999999997,"15min":6.0069999999999997},"last":0.67700000000000005}]},{"osd":7,"last update":"Fri Feb 23 00:48:26 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.68300000000000005,"15min":0.71399999999999997},"min":{"1min":0.45000000000000001,"5min":0.39300000000000002,"15min":0.36699999999999999},"max":{"1min":0.92700000000000005,"5min":3.6230000000000002,"15min":20.225000000000001},"last":0.749},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.70099999999999996,"15min":0.72299999999999998},"min":{"1min":0.49199999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0369999999999999,"5min":5.9169999999999998,"15min":20.254999999999999},"last":0.86599999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738522,"num_pgs":90,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308528,"kb_used_data":7204,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93436624,"statfs":{"total":95995035648,"available":95679102976,"internally_reserved":0,"allocated":7376896,"data_stored":4434921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 23 00:47:49 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":1.028,"15min":0.59699999999999998},"min":{"1min":0.245,"5min":0.20999999999999999,"15min":0.19500000000000001},"max":{"1min":0.81399999999999995,"5min":28.856999999999999,"15min":28.856999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.38,"5min":1.083,"15min":0.61099999999999999},"min":{"1min":0.23599999999999999,"5min":0.218,"15min":0.20799999999999999},"max":{"1min":0.59899999999999998,"5min":20.577000000000002,"15min":20.577000000000002},"last":0.373}]},{"osd":2,"last update":"Fri Feb 23 00:48:10 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.93799999999999994,"15min":0.60299999999999998},"min":{"1min":0.28999999999999998,"5min":0.27200000000000002,"15min":0.20499999999999999},"max":{"1min":0.56799999999999995,"5min":13.863,"15min":13.863},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.379,"5min":0.76900000000000002,"15min":0.54300000000000004},"min":{"1min":0.29299999999999998,"5min":0.219,"15min":0.19800000000000001},"max":{"1min":0.51500000000000001,"5min":10.455,"15min":10.455},"last":0.34100000000000003}]},{"osd":3,"last update":"Fri Feb 23 00:47:39 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":1.415,"15min":0.79500000000000004},"min":{"1min":0.26200000000000001,"5min":0.255,"15min":0.23000000000000001},"max":{"1min":0.73099999999999998,"5min":35.566000000000003,"15min":35.566000000000003},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.42099999999999999,"5min":1.286,"15min":0.77800000000000002},"min":{"1min":0.28100000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.58999999999999997,"5min":20.702999999999999,"15min":20.702999999999999},"last":0.46100000000000002}]},{"osd":4,"last update":"Fri Feb 23 00:48:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.93899999999999995,"15min":0.78300000000000003},"min":{"1min":0.32700000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94699999999999995,"5min":10.584,"15min":10.584},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":1.0109999999999999,"15min":0.80700000000000005},"min":{"1min":0.41299999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.875,"5min":13.198,"15min":13.198},"last":0.65500000000000003}]},{"osd":5,"last update":"Fri Feb 23 00:47:32 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":1.006,"15min":0.73299999999999998},"min":{"1min":0.33800000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.90800000000000003,"5min":10.397,"15min":10.397},"last":0.75},{"interface":"front","average":{"1min":0.64200000000000002,"5min":1.0820000000000001,"15min":0.80100000000000005},"min":{"1min":0.52300000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92000000000000004,"5min":17.297000000000001,"15min":17.297000000000001},"last":0.79100000000000004}]},{"osd":6,"last update":"Fri Feb 23 00:48:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":1.0189999999999999,"15min":0.79300000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.38400000000000001},"max":{"1min":0.94399999999999995,"5min":17.373999999999999,"15min":17.373999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":1.026,"15min":0.76500000000000001},"min":{"1min":0.49199999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93400000000000005,"5min":13.973000000000001,"15min":13.973000000000001},"last":0.74199999999999999}]},{"osd":7,"last update":"Fri Feb 23 00:48:25 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.99199999999999999,"15min":0.79700000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.77600000000000002,"5min":17.399999999999999,"15min":17.399999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.626,"5min":1.0609999999999999,"15min":0.78200000000000003},"min":{"1min":0.438,"5min":0.433,"15min":0.43099999999999999},"max":{"1min":0.879,"5min":16.427,"15min":16.427},"last":0.70699999999999996}]}]},{"osd":2,"up_from":0,"seq":77309411471,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":26,"seq":111669149832,"num_pgs":92,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308596,"kb_used_data":7272,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93436556,"statfs":{"total":95995035648,"available":95679033344,"internally_reserved":0,"allocated":7446528,"data_stored":4434755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:47:38 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.747,"15min":0.53000000000000003},"min":{"1min":0.21199999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.63400000000000001,"5min":12.212999999999999,"15min":12.212999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.94599999999999995,"15min":0.61099999999999999},"min":{"1min":0.255,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.627,"5min":11.714,"15min":11.714},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri Feb 23 00:47:38 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":1.0089999999999999,"15min":0.64200000000000002},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.61199999999999999,"5min":11.742000000000001,"15min":11.742000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.443,"5min":0.92100000000000004,"15min":0.60599999999999998},"min":{"1min":0.26700000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":1.5249999999999999,"5min":8.2270000000000003,"15min":8.2270000000000003},"last":0.36899999999999999}]},{"osd":2,"last update":"Fri Feb 23 00:47:38 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":1.0760000000000001,"15min":0.68700000000000006},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":1.482,"5min":13.359999999999999,"15min":13.359999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.433,"5min":1.026,"15min":0.65700000000000003},"min":{"1min":0.29099999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.84499999999999997,"5min":21.122,"15min":21.122},"last":0.314}]},{"osd":4,"last update":"Fri Feb 23 00:48:05 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":1.0549999999999999,"15min":0.754},"min":{"1min":0.376,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73099999999999998,"5min":11.644,"15min":11.644},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":1.0189999999999999,"15min":0.75},"min":{"1min":0.39000000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76900000000000002,"5min":12.430999999999999,"15min":12.430999999999999},"last":0.67500000000000004}]},{"osd":5,"last update":"Fri Feb 23 00:48:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.78300000000000003,"15min":0.66200000000000003},"min":{"1min":0.372,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.4319999999999999,"5min":5.6360000000000001,"15min":5.6360000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":1.1100000000000001,"15min":0.79600000000000004},"min":{"1min":0.39400000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0029999999999999,"5min":11.648,"15min":11.648},"last":0.64200000000000002}]},{"osd":6,"last update":"Fri Feb 23 00:47:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.995,"15min":0.752},"min":{"1min":0.45200000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.7869999999999999,"5min":6.9420000000000002,"15min":6.9420000000000002},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.80000000000000004,"15min":0.68700000000000006},"min":{"1min":0.45100000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.4179999999999999,"5min":5.6680000000000001,"15min":5.6680000000000001},"last":0.68899999999999995}]},{"osd":7,"last update":"Fri Feb 23 00:48:25 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":1.167,"15min":0.82399999999999995},"min":{"1min":0.42099999999999999,"5min":0.40899999999999997,"15min":0.39000000000000001},"max":{"1min":0.97599999999999998,"5min":11.676,"15min":11.676},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":1.052,"15min":0.80600000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80400000000000005,"5min":7.9800000000000004,"15min":7.9800000000000004},"last":0.72299999999999998}]}]},{"osd":4,"up_from":32,"seq":137438953604,"num_pgs":87,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":312456,"kb_used_data":11128,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93432696,"statfs":{"total":95995035648,"available":95675080704,"internally_reserved":0,"allocated":11395072,"data_stored":8434033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:48:06 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.499,"15min":0.54300000000000004},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.746,"5min":3.5219999999999998,"15min":3.5219999999999998},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.62,"15min":0.59599999999999997},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.76600000000000001,"5min":8.6140000000000008,"15min":8.6140000000000008},"last":0.45300000000000001}]},{"osd":1,"last update":"Fri Feb 23 00:48:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.65800000000000003,"15min":0.59299999999999997},"min":{"1min":0.29699999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.69199999999999995,"5min":9.6110000000000007,"15min":9.6110000000000007},"last":2.6600000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.629,"15min":0.58899999999999997},"min":{"1min":0.29599999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.70499999999999996,"5min":6.6950000000000003,"15min":6.6950000000000003},"last":0.39100000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:48:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58299999999999996,"15min":0.58399999999999996},"min":{"1min":0.34599999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0960000000000001,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":0.438},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.38900000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0189999999999999,"5min":5.8440000000000003,"15min":5.8440000000000003},"last":0.377}]},{"osd":3,"last update":"Fri Feb 23 00:48:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.61399999999999999,"15min":0.59599999999999997},"min":{"1min":0.38400000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":6.5540000000000003,"15min":6.5540000000000003},"last":2.9119999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.372,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":4.0220000000000002,"5min":4.0220000000000002,"15min":4.1909999999999998},"last":0.48999999999999999}]},{"osd":5,"last update":"Fri Feb 23 00:48:30 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.40500000000000003,"15min":0.39700000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.46800000000000003,"15min":0.45300000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.28899999999999998},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.80800000000000005},"last":0.496}]},{"osd":6,"last update":"Fri Feb 23 00:48:03 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40699999999999997,"15min":0.41999999999999998},"min":{"1min":0.28799999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.75},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.41899999999999998,"15min":0.42399999999999999},"min":{"1min":0.28799999999999998,"5min":0.215,"15min":0.215},"max":{"1min":0.55400000000000005,"5min":0.72299999999999998,"15min":0.72699999999999998},"last":0.41499999999999998}]},{"osd":7,"last update":"Fri Feb 23 00:48:26 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.44400000000000001,"15min":0.47799999999999998},"min":{"1min":0.27600000000000002,"5min":0.26700000000000002,"15min":0.254},"max":{"1min":0.71299999999999997,"5min":0.77800000000000002,"15min":0.99099999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.50700000000000001,"15min":0.53400000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72899999999999998,"5min":0.78800000000000003,"15min":0.90200000000000002},"last":0.48099999999999998}]}]},{"osd":5,"up_from":39,"seq":167503724671,"num_pgs":75,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":304424,"kb_used_data":7196,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93440728,"statfs":{"total":95995035648,"available":95683305472,"internally_reserved":0,"allocated":7368704,"data_stored":4435181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:48:25 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.56299999999999994,"15min":0.59699999999999998},"min":{"1min":0.253,"5min":0.246,"15min":0.246},"max":{"1min":1.0009999999999999,"5min":3.7480000000000002,"15min":23.881},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.35299999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.95,"5min":2.9239999999999999,"15min":5.4649999999999999},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Feb 23 00:48:25 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.67800000000000005,"15min":0.64600000000000002},"min":{"1min":0.34999999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.93999999999999995,"5min":5.6509999999999998,"15min":18.175999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.65700000000000003,"15min":0.62},"min":{"1min":0.33400000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.905,"5min":4.2240000000000002,"15min":9.0950000000000006},"last":0.46500000000000002}]},{"osd":2,"last update":"Fri Feb 23 00:48:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.77100000000000002,"15min":0.69899999999999995},"min":{"1min":0.35299999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.282,"5min":10.590999999999999,"15min":22.282},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.72499999999999998,"15min":0.70799999999999996},"min":{"1min":0.41299999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.123,"5min":11.196999999999999,"15min":23.925000000000001},"last":0.621}]},{"osd":3,"last update":"Fri Feb 23 00:48:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.73599999999999999,"15min":0.61499999999999999},"min":{"1min":0.40200000000000002,"5min":0.312,"15min":0.312},"max":{"1min":1.23,"5min":6.5730000000000004,"15min":6.5730000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.72999999999999998,"15min":0.64100000000000001},"min":{"1min":0.41199999999999998,"5min":0.33600000000000002,"15min":0.33500000000000002},"max":{"1min":1.0640000000000001,"5min":6.7910000000000004,"15min":8.1760000000000002},"last":0.60199999999999998}]},{"osd":4,"last update":"Fri Feb 23 00:48:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.46500000000000002,"15min":0.42499999999999999},"min":{"1min":0.309,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":4.593,"5min":4.593,"15min":4.593},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.51200000000000001,"15min":0.45300000000000001},"min":{"1min":0.35799999999999998,"5min":0.27700000000000002,"15min":0.27300000000000002},"max":{"1min":1.5860000000000001,"5min":3.879,"15min":3.879},"last":0.52200000000000002}]},{"osd":6,"last update":"Fri Feb 23 00:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.502,"15min":0.46400000000000002},"min":{"1min":0.32100000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":4.6740000000000004,"5min":4.6740000000000004,"15min":4.6740000000000004},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.502,"5min":0.47199999999999998,"15min":0.46700000000000003},"min":{"1min":0.35399999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.57299999999999995}]},{"osd":7,"last update":"Fri Feb 23 00:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.50900000000000001,"15min":0.53800000000000003},"min":{"1min":0.33700000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.302,"5min":1.302,"15min":7.0300000000000002},"last":0.5},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.51000000000000001,"15min":0.53900000000000003},"min":{"1min":0.34699999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":2.016,"5min":2.016,"15min":9.0579999999999998},"last":0.54700000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":229376,"data_stored":537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4272128,"data_stored":4000622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":68,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":290816,"data_stored":680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4268032,"data_stored":4000639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":229376,"data_stored":533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4366336,"data_stored":4000844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":212,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":212,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":212,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":212,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":212,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":2,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":212,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":2,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":212,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-23T00:48:36.491 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:48:36.521 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:36.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:35.293039+0000 mgr.y (mgr.14517) 398 : cluster [DBG] pgmap v628: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:48:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:48:36.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:35.293039+0000 mgr.y (mgr.14517) 398 : cluster [DBG] pgmap v628: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:48:36.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:35.293039+0000 mgr.y (mgr.14517) 398 : cluster [DBG] pgmap v628: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:48:37.072 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:48:37.075 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:37.076 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:48:37.176 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:48:37.208 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:37.524 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:48:38.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:37.293868+0000 mgr.y (mgr.14517) 399 : cluster [DBG] pgmap v629: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-23T00:48:38.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:37.293868+0000 mgr.y (mgr.14517) 399 : cluster [DBG] pgmap v629: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-23T00:48:38.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:37.293868+0000 mgr.y (mgr.14517) 399 : cluster [DBG] pgmap v629: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-23T00:48:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:48:39.064 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:39.064 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:48:39.165 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:48:39.197 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:39.218 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:48:39.221 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:39.221 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:48:39.322 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:48:39.328 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:48:39.328 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 20, 2024-02-23T00:48:39.328 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 600, 2024-02-23T00:48:39.328 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [ 2024-02-23T00:48:39.328 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.329 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(client.15087.0:1 29.1a e111/109 29:5c6776db:::0.obj:head v 111'13, mlcod=110'11)", 2024-02-23T00:48:39.329 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:42:59.769352+0000", 2024-02-23T00:48:39.329 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 339.55565148199997, 2024-02-23T00:48:39.329 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.026973831, 2024-02-23T00:48:39.329 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:39.329 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:48:39.329 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:39.329 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.329 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:39.329 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.769352+0000", 2024-02-23T00:48:39.329 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:39.329 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.329 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.329 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:39.330 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.769352+0000", 2024-02-23T00:48:39.330 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:48:39.330 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.330 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.330 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:39.330 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.769352+0000", 2024-02-23T00:48:39.330 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5441999999999999e-05 2024-02-23T00:48:39.330 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.330 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.330 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:39.330 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.769367+0000", 2024-02-23T00:48:39.330 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.3549999999999999e-06 2024-02-23T00:48:39.331 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.331 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.331 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:39.331 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.769374+0000", 2024-02-23T00:48:39.331 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.3180000000000003e-06 2024-02-23T00:48:39.331 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.331 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.331 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:39.331 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.769381+0000", 2024-02-23T00:48:39.331 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0045879120000000004 2024-02-23T00:48:39.331 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.331 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.331 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:39.332 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.773969+0000", 2024-02-23T00:48:39.332 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.3527e-05 2024-02-23T00:48:39.332 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.332 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.332 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:39.332 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.773992+0000", 2024-02-23T00:48:39.332 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.022291986999999999 2024-02-23T00:48:39.332 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.332 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.332 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:48:39.332 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.796284+0000", 2024-02-23T00:48:39.332 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.1721000000000003e-05 2024-02-23T00:48:39.333 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.333 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.333 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:39.333 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.796326+0000", 2024-02-23T00:48:39.333 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.026973831 2024-02-23T00:48:39.333 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.333 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:39.333 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.333 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.333 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.333 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24832.0:15 21.15 21:ac8d7e44:test-rados-api-smithi047-82517-4::foo:head [watch unwatch cookie 94456142960576] snapc 0=[] ondisk+write+known_if_redirected e111)", 2024-02-23T00:48:39.333 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:00.909423+0000", 2024-02-23T00:48:39.333 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 338.41558009900001, 2024-02-23T00:48:39.334 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.048904532000000001, 2024-02-23T00:48:39.334 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:39.334 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:48:39.334 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:48:39.334 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24832", 2024-02-23T00:48:39.334 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/2893985577", 2024-02-23T00:48:39.334 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 15 2024-02-23T00:48:39.334 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.334 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:39.334 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.334 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:39.334 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.909423+0000", 2024-02-23T00:48:39.334 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:39.334 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.335 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.335 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:39.335 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.909423+0000", 2024-02-23T00:48:39.335 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999976 2024-02-23T00:48:39.335 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.335 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.335 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:39.335 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.909421+0000", 2024-02-23T00:48:39.335 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.5889999999999992e-06 2024-02-23T00:48:39.335 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.335 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.335 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:39.335 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.909430+0000", 2024-02-23T00:48:39.336 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.025760463000000001 2024-02-23T00:48:39.336 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.336 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.336 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:39.336 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.935191+0000", 2024-02-23T00:48:39.336 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002311156 2024-02-23T00:48:39.336 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.336 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.336 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:39.336 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.937502+0000", 2024-02-23T00:48:39.336 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00039955999999999998 2024-02-23T00:48:39.336 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.337 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.337 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:39.337 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.937902+0000", 2024-02-23T00:48:39.337 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.8650999999999998e-05 2024-02-23T00:48:39.337 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.337 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.337 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:39.337 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.937960+0000", 2024-02-23T00:48:39.337 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.020367645 2024-02-23T00:48:39.337 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.337 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.337 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:39.337 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.958328+0000", 2024-02-23T00:48:39.337 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.048904532000000001 2024-02-23T00:48:39.338 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.338 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:39.338 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.338 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.338 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.338 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24775.0:2059 20.10 20:0c23f028:test-rados-api-smithi047-82457-16::big:head [writefull 0~25165836] snapc 0=[] ondisk+write+known_if_redirected e112)", 2024-02-23T00:48:39.338 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:01.316014+0000", 2024-02-23T00:48:39.338 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 338.00898988799997, 2024-02-23T00:48:39.338 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.116455373, 2024-02-23T00:48:39.338 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:39.338 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:48:39.338 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:48:39.338 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24775", 2024-02-23T00:48:39.338 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1191937413", 2024-02-23T00:48:39.338 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2059 2024-02-23T00:48:39.339 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.339 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:39.339 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.339 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:39.339 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.316014+0000", 2024-02-23T00:48:39.339 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:39.339 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.339 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.339 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:39.339 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.316014+0000", 2024-02-23T00:48:39.339 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999948 2024-02-23T00:48:39.339 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.339 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.339 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:39.340 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.316008+0000", 2024-02-23T00:48:39.340 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.019319273000000001 2024-02-23T00:48:39.340 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.340 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.340 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:39.340 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.335328+0000", 2024-02-23T00:48:39.340 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.7900000000000001e-05 2024-02-23T00:48:39.340 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.340 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.340 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:39.340 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.335356+0000", 2024-02-23T00:48:39.340 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.3740000000000004e-06 2024-02-23T00:48:39.340 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.340 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.341 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:39.341 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.335365+0000", 2024-02-23T00:48:39.341 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.4500999999999998e-05 2024-02-23T00:48:39.341 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.341 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.341 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:39.341 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.335429+0000", 2024-02-23T00:48:39.341 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000157138 2024-02-23T00:48:39.341 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.341 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.341 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:39.341 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.335587+0000", 2024-02-23T00:48:39.341 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000189459 2024-02-23T00:48:39.341 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.341 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.342 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 5,6", 2024-02-23T00:48:39.342 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.335776+0000", 2024-02-23T00:48:39.342 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.028177522 2024-02-23T00:48:39.342 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.342 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.342 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:48:39.342 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.363954+0000", 2024-02-23T00:48:39.342 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.042488566999999998 2024-02-23T00:48:39.342 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.342 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.342 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:48:39.342 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.406442+0000", 2024-02-23T00:48:39.342 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.025953894000000002 2024-02-23T00:48:39.342 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.343 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.343 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:48:39.343 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.432396+0000", 2024-02-23T00:48:39.343 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.9501e-05 2024-02-23T00:48:39.343 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.343 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.343 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:48:39.343 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.432435+0000", 2024-02-23T00:48:39.343 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.3473999999999998e-05 2024-02-23T00:48:39.343 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.343 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.343 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:39.343 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.432469+0000", 2024-02-23T00:48:39.343 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.116455373 2024-02-23T00:48:39.344 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.344 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:39.344 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.344 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.344 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.344 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.14883.0:8 25.9 25:916e78a7:test-rados-api-smithi047-82496-4::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e115)", 2024-02-23T00:48:39.344 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:04.314816+0000", 2024-02-23T00:48:39.344 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 335.01018702599998, 2024-02-23T00:48:39.344 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029852838999999999, 2024-02-23T00:48:39.344 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:39.344 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:48:39.344 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:48:39.344 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.14883", 2024-02-23T00:48:39.345 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1494681915", 2024-02-23T00:48:39.345 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 8 2024-02-23T00:48:39.345 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.345 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:39.345 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.345 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:39.345 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.314816+0000", 2024-02-23T00:48:39.345 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:39.345 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.345 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.345 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:39.345 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.314816+0000", 2024-02-23T00:48:39.345 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:48:39.345 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.345 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.346 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:39.346 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.314815+0000", 2024-02-23T00:48:39.346 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00058973700000000001 2024-02-23T00:48:39.346 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.346 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.346 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:39.346 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.315404+0000", 2024-02-23T00:48:39.346 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0613e-05 2024-02-23T00:48:39.346 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.346 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.346 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:39.346 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.315415+0000", 2024-02-23T00:48:39.346 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.267e-06 2024-02-23T00:48:39.346 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.347 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.347 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:39.347 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.315422+0000", 2024-02-23T00:48:39.347 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014541700000000001 2024-02-23T00:48:39.347 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.347 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.347 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:39.347 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.315568+0000", 2024-02-23T00:48:39.347 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012594799999999999 2024-02-23T00:48:39.347 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.347 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.348 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:39.348 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.315694+0000", 2024-02-23T00:48:39.348 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00016006899999999999 2024-02-23T00:48:39.348 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.348 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.348 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 0,1", 2024-02-23T00:48:39.348 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.315854+0000", 2024-02-23T00:48:39.348 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0030685589999999998 2024-02-23T00:48:39.348 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.349 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.349 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:48:39.349 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.318922+0000", 2024-02-23T00:48:39.349 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0044993280000000004 2024-02-23T00:48:39.349 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.349 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.349 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:48:39.349 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.323422+0000", 2024-02-23T00:48:39.349 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021145840999999999 2024-02-23T00:48:39.349 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.349 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.349 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:48:39.349 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.344568+0000", 2024-02-23T00:48:39.350 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.4187999999999997e-05 2024-02-23T00:48:39.350 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.350 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.350 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:48:39.350 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.344622+0000", 2024-02-23T00:48:39.350 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.7571000000000002e-05 2024-02-23T00:48:39.350 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.350 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.350 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:39.350 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.344669+0000", 2024-02-23T00:48:39.350 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029852838999999999 2024-02-23T00:48:39.350 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.350 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:39.351 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.351 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.351 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.351 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(client.25120.0:2 41.1a e116/115 41:5c6776db:::0.obj:head v 116'1, mlcod=0'0)", 2024-02-23T00:48:39.351 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:05.279600+0000", 2024-02-23T00:48:39.351 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 334.045403072, 2024-02-23T00:48:39.351 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.019639007999999999, 2024-02-23T00:48:39.351 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:39.351 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:48:39.351 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:39.351 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.351 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:39.351 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:05.279600+0000", 2024-02-23T00:48:39.351 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:39.352 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.352 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.352 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:39.352 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:05.279600+0000", 2024-02-23T00:48:39.352 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:48:39.352 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.352 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.352 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:39.352 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:05.279600+0000", 2024-02-23T00:48:39.352 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.7436999999999999e-05 2024-02-23T00:48:39.352 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.352 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.352 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:39.353 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:05.279617+0000", 2024-02-23T00:48:39.353 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8709999999999997e-06 2024-02-23T00:48:39.353 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.353 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.353 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:39.353 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:05.279625+0000", 2024-02-23T00:48:39.353 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.5129999999999999e-06 2024-02-23T00:48:39.353 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.353 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.353 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:39.353 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:05.279633+0000", 2024-02-23T00:48:39.353 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.015584344999999999 2024-02-23T00:48:39.353 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.354 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.354 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:39.354 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:05.295217+0000", 2024-02-23T00:48:39.354 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.9414000000000001e-05 2024-02-23T00:48:39.354 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.354 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.354 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:39.354 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:05.295237+0000", 2024-02-23T00:48:39.354 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0039728480000000002 2024-02-23T00:48:39.354 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.354 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.354 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:48:39.354 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:05.299209+0000", 2024-02-23T00:48:39.354 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0018e-05 2024-02-23T00:48:39.355 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.355 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.355 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:39.355 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:05.299239+0000", 2024-02-23T00:48:39.355 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.019639007999999999 2024-02-23T00:48:39.355 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.355 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:39.355 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.355 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.355 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.355 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(osd.1.0:298 20.5 e140/109 -22:a70ec860:::temp_20.5_0_24170_1:head v 0'0, mlcod=112'1)", 2024-02-23T00:48:39.355 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:32.581619+0000", 2024-02-23T00:48:39.355 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 306.74338445500001, 2024-02-23T00:48:39.355 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.027222695000000002, 2024-02-23T00:48:39.356 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:39.356 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:48:39.356 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:39.356 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.356 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:39.356 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.581619+0000", 2024-02-23T00:48:39.356 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:39.356 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.356 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.356 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:39.356 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.581619+0000", 2024-02-23T00:48:39.356 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999976 2024-02-23T00:48:39.357 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.357 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.357 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:39.357 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.581616+0000", 2024-02-23T00:48:39.357 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0060049730000000003 2024-02-23T00:48:39.357 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.357 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.357 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:39.357 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.587621+0000", 2024-02-23T00:48:39.357 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.9785e-05 2024-02-23T00:48:39.357 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.357 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.357 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:39.358 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.587651+0000", 2024-02-23T00:48:39.358 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2072999999999999e-05 2024-02-23T00:48:39.358 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.358 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.358 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:39.358 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.587663+0000", 2024-02-23T00:48:39.358 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000194685 2024-02-23T00:48:39.358 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.358 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.358 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:39.358 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.587858+0000", 2024-02-23T00:48:39.358 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8759e-05 2024-02-23T00:48:39.358 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.358 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.359 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:39.359 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.587887+0000", 2024-02-23T00:48:39.359 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.020889690999999998 2024-02-23T00:48:39.359 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.359 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.359 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:48:39.359 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.608776+0000", 2024-02-23T00:48:39.359 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.5248999999999998e-05 2024-02-23T00:48:39.359 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.359 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.359 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:39.359 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.608842+0000", 2024-02-23T00:48:39.359 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.027222695000000002 2024-02-23T00:48:39.360 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.360 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:39.360 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.360 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.360 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.360 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(osd.1.0:299 20.5 e140/109 -22:a70ec860:::temp_20.5_0_24170_1:head v 0'0, mlcod=112'1)", 2024-02-23T00:48:39.360 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:32.615170+0000", 2024-02-23T00:48:39.360 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 306.70983311800001, 2024-02-23T00:48:39.360 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021461912, 2024-02-23T00:48:39.360 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:39.360 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:48:39.360 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:39.360 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.361 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:39.361 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.615170+0000", 2024-02-23T00:48:39.361 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:39.361 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.361 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.361 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:39.361 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.615170+0000", 2024-02-23T00:48:39.361 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999976 2024-02-23T00:48:39.361 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.361 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.361 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:39.361 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.615168+0000", 2024-02-23T00:48:39.361 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0063487930000000001 2024-02-23T00:48:39.362 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.362 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.362 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:39.362 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.621517+0000", 2024-02-23T00:48:39.362 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.3031000000000003e-05 2024-02-23T00:48:39.362 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.362 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.362 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:39.362 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.621550+0000", 2024-02-23T00:48:39.362 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2002e-05 2024-02-23T00:48:39.362 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.362 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.362 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:39.362 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.621562+0000", 2024-02-23T00:48:39.363 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.0396000000000007e-05 2024-02-23T00:48:39.363 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.363 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.363 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:39.363 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.621652+0000", 2024-02-23T00:48:39.363 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.4893e-05 2024-02-23T00:48:39.363 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.363 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.363 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:39.363 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.621677+0000", 2024-02-23T00:48:39.363 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.014898613999999999 2024-02-23T00:48:39.363 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.363 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.364 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:48:39.364 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.636576+0000", 2024-02-23T00:48:39.364 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.6641999999999997e-05 2024-02-23T00:48:39.364 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.364 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.364 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:39.364 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.636632+0000", 2024-02-23T00:48:39.364 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021461912 2024-02-23T00:48:39.364 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.364 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:39.364 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.364 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.364 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.365 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(client.25529.0:42 63.e e142/127 63:7037fd60:test-rados-api-smithi047-82496-10::foo:head v 142'13, mlcod=140'10)", 2024-02-23T00:48:39.365 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:34.508189+0000", 2024-02-23T00:48:39.365 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 304.81681434699999, 2024-02-23T00:48:39.365 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.046943154000000001, 2024-02-23T00:48:39.365 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:39.365 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:48:39.365 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:39.365 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.365 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:39.365 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.508189+0000", 2024-02-23T00:48:39.365 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:39.365 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.365 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.366 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:39.366 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.508189+0000", 2024-02-23T00:48:39.366 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.999999 2024-02-23T00:48:39.366 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.366 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.366 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:39.366 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.508188+0000", 2024-02-23T00:48:39.366 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0925e-05 2024-02-23T00:48:39.366 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.366 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.366 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:39.366 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.508199+0000", 2024-02-23T00:48:39.366 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.5159999999999994e-06 2024-02-23T00:48:39.366 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.367 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.367 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:39.367 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.508208+0000", 2024-02-23T00:48:39.367 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.1750000000000002e-06 2024-02-23T00:48:39.367 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.367 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.367 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:39.367 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.508214+0000", 2024-02-23T00:48:39.367 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00015588599999999999 2024-02-23T00:48:39.367 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.367 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.367 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:39.367 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.508370+0000", 2024-02-23T00:48:39.367 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1435000000000001e-05 2024-02-23T00:48:39.367 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.368 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.368 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:39.368 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.508391+0000", 2024-02-23T00:48:39.368 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.046710464 2024-02-23T00:48:39.368 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.368 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.368 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:48:39.368 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.555102+0000", 2024-02-23T00:48:39.368 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0552000000000002e-05 2024-02-23T00:48:39.368 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.368 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.368 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:39.368 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.555132+0000", 2024-02-23T00:48:39.369 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.046943154000000001 2024-02-23T00:48:39.369 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.369 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:39.369 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.369 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.369 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.369 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDScrubReserve(6.e RELEASE e144)", 2024-02-23T00:48:39.369 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:36.207155+0000", 2024-02-23T00:48:39.369 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 303.11784819600001, 2024-02-23T00:48:39.369 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.024392191000000001, 2024-02-23T00:48:39.369 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:39.369 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:48:39.369 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:39.369 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.369 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:39.370 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.207155+0000", 2024-02-23T00:48:39.370 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:39.370 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.370 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.370 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:39.370 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.207155+0000", 2024-02-23T00:48:39.370 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999986 2024-02-23T00:48:39.370 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.370 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.370 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:39.370 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.207154+0000", 2024-02-23T00:48:39.370 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.7485999999999996e-05 2024-02-23T00:48:39.370 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.370 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.371 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:39.371 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.207241+0000", 2024-02-23T00:48:39.371 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.394e-05 2024-02-23T00:48:39.371 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.371 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.371 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:39.371 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.207255+0000", 2024-02-23T00:48:39.371 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.6729999999999992e-06 2024-02-23T00:48:39.371 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.371 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.371 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:39.371 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.207263+0000", 2024-02-23T00:48:39.371 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.016180798 2024-02-23T00:48:39.371 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.372 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.372 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:39.372 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.223444+0000", 2024-02-23T00:48:39.372 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.7590000000000003e-06 2024-02-23T00:48:39.372 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.372 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.372 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-02-23T00:48:39.372 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.223452+0000", 2024-02-23T00:48:39.372 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.008038847 2024-02-23T00:48:39.372 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.372 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.372 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:39.372 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.231491+0000", 2024-02-23T00:48:39.373 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1171000000000001e-05 2024-02-23T00:48:39.373 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.373 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.373 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:39.373 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.231512+0000", 2024-02-23T00:48:39.373 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.4987000000000002e-05 2024-02-23T00:48:39.373 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.373 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.373 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:39.373 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.231547+0000", 2024-02-23T00:48:39.373 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.024392191000000001 2024-02-23T00:48:39.373 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.373 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:39.373 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.373 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.374 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.374 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWrite(79.6s2 145/139 ECSubWrite(tid=229, reqid=client.31603.0:7, at_version=145'1, trim_to=0'0, roll_forward_to=0'0))", 2024-02-23T00:48:39.374 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:37.220998+0000", 2024-02-23T00:48:39.374 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 302.10400589, 2024-02-23T00:48:39.374 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029900639999999999, 2024-02-23T00:48:39.374 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:39.374 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-02-23T00:48:39.374 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:39.374 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.374 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:39.374 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.220998+0000", 2024-02-23T00:48:39.374 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:39.374 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.374 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.375 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:39.375 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.220998+0000", 2024-02-23T00:48:39.375 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:48:39.375 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.375 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.375 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:39.375 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.220997+0000", 2024-02-23T00:48:39.375 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.4640000000000001e-05 2024-02-23T00:48:39.375 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.375 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.375 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:39.375 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.221022+0000", 2024-02-23T00:48:39.375 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.4220000000000006e-06 2024-02-23T00:48:39.375 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.376 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.376 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:39.376 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.221031+0000", 2024-02-23T00:48:39.376 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.4699999999999996e-06 2024-02-23T00:48:39.376 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.376 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.376 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:39.376 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.221039+0000", 2024-02-23T00:48:39.376 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0063503040000000002 2024-02-23T00:48:39.376 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.376 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.376 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:39.376 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.227389+0000", 2024-02-23T00:48:39.377 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.9499999999999996e-06 2024-02-23T00:48:39.377 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.377 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.377 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-02-23T00:48:39.377 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.227399+0000", 2024-02-23T00:48:39.377 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023482154000000002 2024-02-23T00:48:39.377 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.377 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.377 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-02-23T00:48:39.377 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.250881+0000", 2024-02-23T00:48:39.377 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.7016000000000001e-05 2024-02-23T00:48:39.377 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.377 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.377 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:39.378 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.250898+0000", 2024-02-23T00:48:39.378 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029900639999999999 2024-02-23T00:48:39.378 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.378 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:39.378 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.378 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.378 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.378 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(client.15270.0:248 62.14 e145/126 62:28b98b91:test-rados-api-smithi047-82465-12::bar:head v 145'1, mlcod=0'0)", 2024-02-23T00:48:39.378 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:37.222926+0000", 2024-02-23T00:48:39.378 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 302.10207795000002, 2024-02-23T00:48:39.378 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029711336000000001, 2024-02-23T00:48:39.378 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:39.378 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:48:39.379 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:39.379 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.379 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:39.379 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.222926+0000", 2024-02-23T00:48:39.379 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:39.379 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.379 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.379 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:39.379 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.222926+0000", 2024-02-23T00:48:39.379 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:48:39.379 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.379 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.379 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:39.379 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.222925+0000", 2024-02-23T00:48:39.380 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.8849999999999994e-06 2024-02-23T00:48:39.380 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.380 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.380 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:39.380 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.222935+0000", 2024-02-23T00:48:39.380 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.3560000000000004e-06 2024-02-23T00:48:39.380 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.380 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.380 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:39.380 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.222942+0000", 2024-02-23T00:48:39.380 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.0180000000000004e-06 2024-02-23T00:48:39.380 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.380 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.380 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:39.381 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.222949+0000", 2024-02-23T00:48:39.381 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001121143 2024-02-23T00:48:39.381 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.381 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.381 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:39.381 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.224070+0000", 2024-02-23T00:48:39.381 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5017e-05 2024-02-23T00:48:39.381 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.381 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.381 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:39.381 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.224085+0000", 2024-02-23T00:48:39.381 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.028523943999999999 2024-02-23T00:48:39.381 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.382 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.382 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:48:39.382 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.252609+0000", 2024-02-23T00:48:39.382 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.7416e-05 2024-02-23T00:48:39.382 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.382 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.382 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:39.382 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.252637+0000", 2024-02-23T00:48:39.382 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029711336000000001 2024-02-23T00:48:39.382 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.382 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:39.382 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.382 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.382 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.383 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(client.15270.0:252 62.18 e146/126 62:1be66eec:test-rados-api-smithi047-82465-12::foo:head v 146'3, mlcod=0'0)", 2024-02-23T00:48:39.383 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:38.406443+0000", 2024-02-23T00:48:39.383 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 300.918560845, 2024-02-23T00:48:39.383 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021146257000000002, 2024-02-23T00:48:39.383 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:39.383 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:48:39.383 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:39.383 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.383 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:39.383 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.406443+0000", 2024-02-23T00:48:39.383 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:39.383 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.383 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.383 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:39.384 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.406443+0000", 2024-02-23T00:48:39.384 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:48:39.384 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.384 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.384 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:39.384 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.406442+0000", 2024-02-23T00:48:39.384 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0117e-05 2024-02-23T00:48:39.384 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.384 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.384 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:39.384 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.406452+0000", 2024-02-23T00:48:39.384 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.5140000000000004e-06 2024-02-23T00:48:39.384 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.384 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.385 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:39.385 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.406460+0000", 2024-02-23T00:48:39.385 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8930000000000005e-06 2024-02-23T00:48:39.385 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.385 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.385 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:39.385 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.406468+0000", 2024-02-23T00:48:39.385 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010643227 2024-02-23T00:48:39.385 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.385 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.385 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:39.385 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.417111+0000", 2024-02-23T00:48:39.385 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1742999999999998e-05 2024-02-23T00:48:39.386 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.386 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.386 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:39.386 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.417133+0000", 2024-02-23T00:48:39.386 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010419154999999999 2024-02-23T00:48:39.386 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.386 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.386 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:48:39.386 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.427552+0000", 2024-02-23T00:48:39.386 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.6937000000000002e-05 2024-02-23T00:48:39.386 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.386 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.386 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:39.387 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.427589+0000", 2024-02-23T00:48:39.387 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021146257000000002 2024-02-23T00:48:39.387 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.387 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:39.387 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.387 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.387 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.387 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.40541.0:2 116.6 116:602f83fe:::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e181)", 2024-02-23T00:48:39.387 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:13.469852+0000", 2024-02-23T00:48:39.387 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 265.85515183299998, 2024-02-23T00:48:39.387 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.042655076, 2024-02-23T00:48:39.387 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:39.387 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:48:39.388 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:48:39.388 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.40541", 2024-02-23T00:48:39.388 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/865084253", 2024-02-23T00:48:39.388 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2 2024-02-23T00:48:39.388 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.388 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:39.388 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.388 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:39.388 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.469852+0000", 2024-02-23T00:48:39.388 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:39.388 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.388 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.388 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:39.388 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.469852+0000", 2024-02-23T00:48:39.389 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:48:39.389 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.389 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.389 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:39.389 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.469850+0000", 2024-02-23T00:48:39.389 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.166e-06 2024-02-23T00:48:39.389 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.389 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.389 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:39.389 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.469859+0000", 2024-02-23T00:48:39.389 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2982999999999999e-05 2024-02-23T00:48:39.389 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.389 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.390 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:39.390 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.469872+0000", 2024-02-23T00:48:39.390 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.6010000000000001e-06 2024-02-23T00:48:39.390 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.390 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.390 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:39.390 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.469881+0000", 2024-02-23T00:48:39.390 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.8393999999999996e-05 2024-02-23T00:48:39.390 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.390 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.390 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:39.390 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.469950+0000", 2024-02-23T00:48:39.390 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.5129999999999999e-06 2024-02-23T00:48:39.391 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.391 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.391 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-02-23T00:48:39.391 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.469957+0000", 2024-02-23T00:48:39.391 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.013799713 2024-02-23T00:48:39.391 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.391 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.391 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:39.391 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.483757+0000", 2024-02-23T00:48:39.391 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8869999999999992e-06 2024-02-23T00:48:39.391 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.391 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.391 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:48:39.391 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.483765+0000", 2024-02-23T00:48:39.391 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.015659230999999999 2024-02-23T00:48:39.392 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.392 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.392 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:39.392 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.499424+0000", 2024-02-23T00:48:39.392 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00013537900000000001 2024-02-23T00:48:39.392 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.392 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.392 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:39.392 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.499559+0000", 2024-02-23T00:48:39.392 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000169433 2024-02-23T00:48:39.392 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.392 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.392 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 5,6", 2024-02-23T00:48:39.393 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.499729+0000", 2024-02-23T00:48:39.393 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012107788 2024-02-23T00:48:39.393 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.393 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.393 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:48:39.393 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.511837+0000", 2024-02-23T00:48:39.393 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00053866299999999999 2024-02-23T00:48:39.393 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.393 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.393 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:48:39.393 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.512375+0000", 2024-02-23T00:48:39.393 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.4773000000000001e-05 2024-02-23T00:48:39.393 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.393 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.393 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:48:39.394 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.512410+0000", 2024-02-23T00:48:39.394 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.5303000000000002e-05 2024-02-23T00:48:39.394 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.394 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.394 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:48:39.394 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.512465+0000", 2024-02-23T00:48:39.394 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.1287e-05 2024-02-23T00:48:39.394 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.394 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.394 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:39.394 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.512507+0000", 2024-02-23T00:48:39.394 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.042655076 2024-02-23T00:48:39.394 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.394 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:39.394 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.395 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.395 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.395 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.40517.0:2 115.1b 115:d8841f39:test-rados-api-smithi047-82457-26::foo:head [writefull 0~8388608] snapc 0=[] ondisk+write+known_if_redirected e182)", 2024-02-23T00:48:39.395 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:14.489687+0000", 2024-02-23T00:48:39.395 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 264.83531643700002, 2024-02-23T00:48:39.395 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.02886706, 2024-02-23T00:48:39.395 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:39.395 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:48:39.395 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:48:39.395 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.40517", 2024-02-23T00:48:39.395 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/3108721855", 2024-02-23T00:48:39.395 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2 2024-02-23T00:48:39.395 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.395 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:39.395 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.396 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:39.396 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.489687+0000", 2024-02-23T00:48:39.396 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:39.396 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.396 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.396 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:39.396 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.489687+0000", 2024-02-23T00:48:39.396 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999976 2024-02-23T00:48:39.396 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.396 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.396 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:39.396 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.489684+0000", 2024-02-23T00:48:39.396 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0052556039999999997 2024-02-23T00:48:39.397 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.397 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.397 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:39.397 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.494940+0000", 2024-02-23T00:48:39.397 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.5494e-05 2024-02-23T00:48:39.397 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.397 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.397 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:39.397 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.494966+0000", 2024-02-23T00:48:39.397 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.4069999999999999e-06 2024-02-23T00:48:39.397 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.397 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.397 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:39.397 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.494975+0000", 2024-02-23T00:48:39.397 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.6199999999999996e-05 2024-02-23T00:48:39.398 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.398 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.398 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:39.398 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.495041+0000", 2024-02-23T00:48:39.398 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000167143 2024-02-23T00:48:39.398 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.398 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.398 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:39.398 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.495208+0000", 2024-02-23T00:48:39.398 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000190027 2024-02-23T00:48:39.398 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.398 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.398 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 0,1", 2024-02-23T00:48:39.398 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.495398+0000", 2024-02-23T00:48:39.399 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.013352564000000001 2024-02-23T00:48:39.399 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.399 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.399 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:48:39.399 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.508751+0000", 2024-02-23T00:48:39.399 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0094962550000000003 2024-02-23T00:48:39.399 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.399 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.399 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:48:39.399 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.518247+0000", 2024-02-23T00:48:39.399 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000230545 2024-02-23T00:48:39.399 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.400 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.400 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:48:39.400 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.518478+0000", 2024-02-23T00:48:39.400 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.8698000000000001e-05 2024-02-23T00:48:39.400 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.400 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.400 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:48:39.400 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.518516+0000", 2024-02-23T00:48:39.400 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7685999999999998e-05 2024-02-23T00:48:39.400 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.400 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.400 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:39.401 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.518554+0000", 2024-02-23T00:48:39.401 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.02886706 2024-02-23T00:48:39.401 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.401 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:39.401 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.401 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.401 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.401 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(osd.4.0:477 62.f e199/126 62:f64c5448:test-rados-api-smithi047-82465-14::foo96:4 v 199'80, mlcod=197'74)", 2024-02-23T00:48:39.401 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:31.164866+0000", 2024-02-23T00:48:39.401 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 248.160137846, 2024-02-23T00:48:39.401 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018575001000000001, 2024-02-23T00:48:39.401 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:39.401 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:48:39.401 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:39.402 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.402 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:39.402 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164866+0000", 2024-02-23T00:48:39.402 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:39.402 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.402 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.402 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:39.402 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164866+0000", 2024-02-23T00:48:39.402 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:48:39.402 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.402 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.402 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:39.402 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164865+0000", 2024-02-23T00:48:39.402 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1764e-05 2024-02-23T00:48:39.402 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.402 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.403 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:39.403 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164877+0000", 2024-02-23T00:48:39.403 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.5299999999999996e-06 2024-02-23T00:48:39.403 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.403 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.403 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:39.403 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164885+0000", 2024-02-23T00:48:39.403 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.5619999999999997e-06 2024-02-23T00:48:39.403 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.403 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.403 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:39.403 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164890+0000", 2024-02-23T00:48:39.403 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0022547829999999998 2024-02-23T00:48:39.403 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.403 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.404 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:39.404 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.167145+0000", 2024-02-23T00:48:39.404 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.8792000000000003e-05 2024-02-23T00:48:39.404 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.404 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.404 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:39.404 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.167193+0000", 2024-02-23T00:48:39.404 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.016211952000000002 2024-02-23T00:48:39.404 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.404 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.404 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:48:39.404 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.183405+0000", 2024-02-23T00:48:39.404 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.5181000000000001e-05 2024-02-23T00:48:39.405 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.405 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.405 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:39.405 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.183441+0000", 2024-02-23T00:48:39.405 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018575001000000001 2024-02-23T00:48:39.405 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.405 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:39.405 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.405 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.405 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.405 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.40754.0:2 145.6 145:602f83fe:::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e222)", 2024-02-23T00:48:39.405 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:54.146645+0000", 2024-02-23T00:48:39.405 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 225.17835872000001, 2024-02-23T00:48:39.405 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021068969, 2024-02-23T00:48:39.405 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:39.406 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:48:39.406 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:48:39.406 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.40754", 2024-02-23T00:48:39.406 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/2235208660", 2024-02-23T00:48:39.406 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2 2024-02-23T00:48:39.406 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.406 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:39.406 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.406 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:39.406 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.146645+0000", 2024-02-23T00:48:39.406 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:39.406 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.406 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.406 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:39.406 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.146645+0000", 2024-02-23T00:48:39.407 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999976 2024-02-23T00:48:39.407 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.407 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.407 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:39.407 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.146643+0000", 2024-02-23T00:48:39.407 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.918e-06 2024-02-23T00:48:39.407 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.407 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.407 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:39.407 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.146651+0000", 2024-02-23T00:48:39.407 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5065000000000001e-05 2024-02-23T00:48:39.407 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.408 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.408 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:39.408 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.146667+0000", 2024-02-23T00:48:39.408 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1534e-05 2024-02-23T00:48:39.408 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.408 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.408 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:39.408 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.146678+0000", 2024-02-23T00:48:39.408 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.5680000000000002e-05 2024-02-23T00:48:39.408 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.408 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.408 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:39.408 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.146734+0000", 2024-02-23T00:48:39.409 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.5449999999999998e-06 2024-02-23T00:48:39.409 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.409 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.409 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-02-23T00:48:39.409 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.146741+0000", 2024-02-23T00:48:39.409 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0075490319999999998 2024-02-23T00:48:39.409 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.409 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.409 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:39.409 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.154290+0000", 2024-02-23T00:48:39.409 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.8569999999999997e-06 2024-02-23T00:48:39.409 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.409 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.409 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:48:39.410 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.154297+0000", 2024-02-23T00:48:39.410 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0090633199999999997 2024-02-23T00:48:39.410 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.410 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.410 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:39.410 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.163361+0000", 2024-02-23T00:48:39.410 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000107849 2024-02-23T00:48:39.410 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.410 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.410 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:39.410 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.163468+0000", 2024-02-23T00:48:39.410 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00015853699999999999 2024-02-23T00:48:39.410 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.410 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.411 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 1,4", 2024-02-23T00:48:39.411 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.163627+0000", 2024-02-23T00:48:39.411 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00066408199999999998 2024-02-23T00:48:39.411 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.411 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.411 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:48:39.411 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.164291+0000", 2024-02-23T00:48:39.411 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000633827 2024-02-23T00:48:39.411 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.411 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.411 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:48:39.411 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.164925+0000", 2024-02-23T00:48:39.411 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002695514 2024-02-23T00:48:39.411 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.412 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.412 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:48:39.412 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.167620+0000", 2024-02-23T00:48:39.412 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.2404999999999997e-05 2024-02-23T00:48:39.412 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.412 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.412 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:48:39.412 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.167673+0000", 2024-02-23T00:48:39.412 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.0967000000000003e-05 2024-02-23T00:48:39.412 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.412 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.412 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:39.412 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.167714+0000", 2024-02-23T00:48:39.413 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021068969 2024-02-23T00:48:39.413 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.413 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:39.413 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.413 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.413 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.413 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.15270.0:2078 62.3 62:c3aa4723:test-rados-api-smithi047-82465-26::foo:head [writefull 0~8] snapc 0=[] ondisk+write+known_if_redirected e245)", 2024-02-23T00:48:39.413 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:45:17.141076+0000", 2024-02-23T00:48:39.413 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 202.18392729499999, 2024-02-23T00:48:39.413 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029775917999999998, 2024-02-23T00:48:39.413 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:39.413 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:48:39.413 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:48:39.414 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.15270", 2024-02-23T00:48:39.414 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/4133111990", 2024-02-23T00:48:39.414 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2078 2024-02-23T00:48:39.414 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.414 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:39.414 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.414 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:39.414 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.141076+0000", 2024-02-23T00:48:39.414 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:39.414 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.414 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.414 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:39.414 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.141076+0000", 2024-02-23T00:48:39.414 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999976 2024-02-23T00:48:39.415 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.415 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.415 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:39.415 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.141074+0000", 2024-02-23T00:48:39.415 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.3340000000000008e-06 2024-02-23T00:48:39.415 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.415 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.415 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:39.415 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.141083+0000", 2024-02-23T00:48:39.415 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3730999999999999e-05 2024-02-23T00:48:39.415 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.415 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.415 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:39.415 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.141097+0000", 2024-02-23T00:48:39.416 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.3159999999999997e-06 2024-02-23T00:48:39.416 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.416 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.416 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:39.416 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.141105+0000", 2024-02-23T00:48:39.416 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.1898999999999999e-05 2024-02-23T00:48:39.416 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.416 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.416 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:39.416 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.141157+0000", 2024-02-23T00:48:39.416 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00016548999999999999 2024-02-23T00:48:39.416 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.416 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.416 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:39.417 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.141323+0000", 2024-02-23T00:48:39.417 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00020201299999999999 2024-02-23T00:48:39.417 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.417 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.417 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 1,4", 2024-02-23T00:48:39.417 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.141525+0000", 2024-02-23T00:48:39.417 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0095168300000000004 2024-02-23T00:48:39.417 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.417 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.417 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:48:39.417 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.151042+0000", 2024-02-23T00:48:39.417 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018502253 2024-02-23T00:48:39.417 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.417 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.418 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:48:39.418 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.169544+0000", 2024-02-23T00:48:39.418 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001218156 2024-02-23T00:48:39.418 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.418 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.418 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:48:39.418 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.170762+0000", 2024-02-23T00:48:39.418 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.3652000000000002e-05 2024-02-23T00:48:39.418 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.418 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.418 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:48:39.418 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.170806+0000", 2024-02-23T00:48:39.418 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.6415000000000002e-05 2024-02-23T00:48:39.418 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.419 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.419 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:39.419 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.170852+0000", 2024-02-23T00:48:39.419 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029775917999999998 2024-02-23T00:48:39.419 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.419 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:39.419 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.419 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.419 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.419 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.30483.0:9 155.c 155:3197012c:::foo_rmxattr:head [rmxattr attr2 in=5b] snapc 0=[] ondisk+write+known_if_redirected e243)", 2024-02-23T00:48:39.419 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:45:17.146980+0000", 2024-02-23T00:48:39.419 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 202.17802327300001, 2024-02-23T00:48:39.419 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.022584933000000001, 2024-02-23T00:48:39.419 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:39.420 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:48:39.420 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:48:39.420 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.30483", 2024-02-23T00:48:39.420 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/3044143263", 2024-02-23T00:48:39.420 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 9 2024-02-23T00:48:39.420 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.420 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:39.420 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.420 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:39.420 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.146980+0000", 2024-02-23T00:48:39.420 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:39.420 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.420 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.420 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:39.421 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.146980+0000", 2024-02-23T00:48:39.421 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:48:39.421 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.421 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.421 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:39.421 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.146978+0000", 2024-02-23T00:48:39.421 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0020000000000006e-06 2024-02-23T00:48:39.421 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.421 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.421 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:39.421 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.146986+0000", 2024-02-23T00:48:39.421 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4829e-05 2024-02-23T00:48:39.421 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.421 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.422 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:39.422 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.147001+0000", 2024-02-23T00:48:39.422 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.8660000000000001e-06 2024-02-23T00:48:39.422 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.422 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.422 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:39.422 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.147011+0000", 2024-02-23T00:48:39.422 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0099999999999995e-05 2024-02-23T00:48:39.422 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.422 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.422 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:39.422 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.147091+0000", 2024-02-23T00:48:39.422 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00010793000000000001 2024-02-23T00:48:39.422 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.423 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.423 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:39.423 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.147199+0000", 2024-02-23T00:48:39.423 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.022366104000000001 2024-02-23T00:48:39.423 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.423 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.423 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:39.423 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.169565+0000", 2024-02-23T00:48:39.423 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.022584933000000001 2024-02-23T00:48:39.423 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.423 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:39.423 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.423 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.423 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.424 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24820.0:1124 15.0 15:00000000::::head [pgnls start_epoch 307] snapc 0=[] RETRY=1 ondisk+retry+read+ignore_overlay+known_if_redirected e337)", 2024-02-23T00:48:39.424 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:46:50.181661+0000", 2024-02-23T00:48:39.424 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 109.143342538, 2024-02-23T00:48:39.424 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.005913203, 2024-02-23T00:48:39.424 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:39.424 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:48:39.424 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:48:39.424 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24820", 2024-02-23T00:48:39.424 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/2979574659", 2024-02-23T00:48:39.424 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 1124 2024-02-23T00:48:39.424 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.424 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:39.425 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.425 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:39.425 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:50.181661+0000", 2024-02-23T00:48:39.425 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:39.425 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.425 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.425 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:39.425 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:50.181661+0000", 2024-02-23T00:48:39.425 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:48:39.425 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.425 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.425 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:39.425 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:50.181659+0000", 2024-02-23T00:48:39.425 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.6400000000000003e-06 2024-02-23T00:48:39.426 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.426 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.426 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:39.426 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:50.181668+0000", 2024-02-23T00:48:39.426 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1449e-05 2024-02-23T00:48:39.426 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.426 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.426 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:39.426 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:50.181679+0000", 2024-02-23T00:48:39.426 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.6820000000000003e-06 2024-02-23T00:48:39.426 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.426 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.426 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:39.426 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:50.181687+0000", 2024-02-23T00:48:39.426 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00045155199999999998 2024-02-23T00:48:39.427 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.427 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.427 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:39.427 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:50.182138+0000", 2024-02-23T00:48:39.427 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1856e-05 2024-02-23T00:48:39.427 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.427 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.427 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:48:39.427 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:50.182150+0000", 2024-02-23T00:48:39.427 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0051927949999999 2024-02-23T00:48:39.427 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.427 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.427 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:39.427 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:51.187343+0000", 2024-02-23T00:48:39.428 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.9544e-05 2024-02-23T00:48:39.428 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.428 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.428 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:39.428 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:51.187383+0000", 2024-02-23T00:48:39.428 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000191547 2024-02-23T00:48:39.428 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.428 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.428 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:39.428 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:51.187574+0000", 2024-02-23T00:48:39.428 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.005913203 2024-02-23T00:48:39.428 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.429 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:39.429 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.429 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.429 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.429 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24820.0:1124 15.0 15:00000000::::head [pgnls start_epoch 307] snapc 0=[] RETRY=2 ondisk+retry+read+ignore_overlay+known_if_redirected e338)", 2024-02-23T00:48:39.429 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:46:50.182461+0000", 2024-02-23T00:48:39.429 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 109.142542521, 2024-02-23T00:48:39.429 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.005281654, 2024-02-23T00:48:39.429 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:48:39.429 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:48:39.429 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:48:39.429 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24820", 2024-02-23T00:48:39.429 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/2979574659", 2024-02-23T00:48:39.429 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 1124 2024-02-23T00:48:39.430 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.430 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:48:39.430 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.430 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:48:39.430 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:50.182461+0000", 2024-02-23T00:48:39.430 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:48:39.430 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.430 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.430 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:48:39.430 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:50.182461+0000", 2024-02-23T00:48:39.430 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.999999 2024-02-23T00:48:39.430 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.430 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.430 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:48:39.430 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:50.182460+0000", 2024-02-23T00:48:39.431 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.8390000000000003e-06 2024-02-23T00:48:39.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.431 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.431 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:48:39.431 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:50.182466+0000", 2024-02-23T00:48:39.431 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0042e-05 2024-02-23T00:48:39.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.431 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.431 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:48:39.431 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:50.182476+0000", 2024-02-23T00:48:39.431 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.3230000000000003e-06 2024-02-23T00:48:39.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.431 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.432 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:48:39.432 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:50.182482+0000", 2024-02-23T00:48:39.432 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.3739000000000006e-05 2024-02-23T00:48:39.432 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.432 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.432 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:39.432 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:50.182566+0000", 2024-02-23T00:48:39.432 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.098e-06 2024-02-23T00:48:39.432 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.432 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.432 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:48:39.432 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:50.182575+0000", 2024-02-23T00:48:39.433 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.005027262 2024-02-23T00:48:39.433 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.433 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.433 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:48:39.433 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:51.187602+0000", 2024-02-23T00:48:39.433 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7106e-05 2024-02-23T00:48:39.433 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.433 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.433 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:48:39.433 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:51.187639+0000", 2024-02-23T00:48:39.433 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00010318 2024-02-23T00:48:39.433 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:48:39.433 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:48:39.433 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:48:39.434 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:51.187743+0000", 2024-02-23T00:48:39.434 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.005281654 2024-02-23T00:48:39.434 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.434 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:39.434 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.434 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:48:39.434 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:48:39.434 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:48:39.435 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:39.435 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:39.436 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:48:39.436 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:48:39.537 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:48:39.567 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:39.587 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:48:39.589 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:39.590 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:48:39.691 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:48:39.723 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:39.744 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:48:39.747 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:39.747 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:48:39.848 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:48:39.880 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:39.902 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:48:39.904 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:39.905 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:48:40.005 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:48:40.038 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:40.059 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:48:40.061 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:40.062 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:48:40.163 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:48:40.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.123+0000 7f0a26233700 1 -- 172.21.15.47:0/368953193 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a200f6880 msgr2=0x7f0a201049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:40.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.123+0000 7f0a26233700 1 --2- 172.21.15.47:0/368953193 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a200f6880 0x7f0a201049e0 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 rx=0x7f0a18009be0 tx=0x7f0a18004d10).stop 2024-02-23T00:48:40.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.123+0000 7f0a26233700 1 -- 172.21.15.47:0/368953193 shutdown_connections 2024-02-23T00:48:40.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.123+0000 7f0a26233700 1 --2- 172.21.15.47:0/368953193 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a200f6880 0x7f0a201049e0 unknown :-1 s=CLOSED pgs=2534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:40.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.123+0000 7f0a26233700 1 --2- 172.21.15.47:0/368953193 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a200fdc90 0x7f0a200fe070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:40.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.123+0000 7f0a26233700 1 --2- 172.21.15.47:0/368953193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a200fe560 0x7f0a200f61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:40.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.123+0000 7f0a26233700 1 -- 172.21.15.47:0/368953193 >> 172.21.15.47:0/368953193 conn(0x7f0a2006ffc0 msgr2=0x7f0a200f43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:48:40.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.123+0000 7f0a26233700 1 -- 172.21.15.47:0/368953193 shutdown_connections 2024-02-23T00:48:40.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.123+0000 7f0a26233700 1 -- 172.21.15.47:0/368953193 wait complete. 2024-02-23T00:48:40.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.124+0000 7f0a26233700 1 librados: starting msgr at 2024-02-23T00:48:40.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.124+0000 7f0a26233700 1 librados: starting objecter 2024-02-23T00:48:40.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.124+0000 7f0a26233700 1 Processor -- start 2024-02-23T00:48:40.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.124+0000 7f0a26233700 1 -- start start 2024-02-23T00:48:40.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.124+0000 7f0a26233700 1 librados: setting wanted keys 2024-02-23T00:48:40.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.124+0000 7f0a26233700 1 librados: calling monclient init 2024-02-23T00:48:40.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.124+0000 7f0a26233700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a200f6880 0x7f0a2010dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:40.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.124+0000 7f0a26233700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a200fdc90 0x7f0a2010e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:40.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.124+0000 7f0a26233700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a200fe560 0x7f0a20107f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:40.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.124+0000 7f0a26233700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a20108410 con 0x7f0a200fe560 2024-02-23T00:48:40.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.124+0000 7f0a26233700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a20108580 con 0x7f0a200fdc90 2024-02-23T00:48:40.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.124+0000 7f0a26233700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a20108740 con 0x7f0a200f6880 2024-02-23T00:48:40.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.125+0000 7f0a1f7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a200f6880 0x7f0a2010dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:40.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.125+0000 7f0a1f7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a200f6880 0x7f0a2010dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:56510/0 (socket says 172.21.15.47:56510) 2024-02-23T00:48:40.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.125+0000 7f0a1f7fe700 1 -- 172.21.15.47:0/1536735134 learned_addr learned my addr 172.21.15.47:0/1536735134 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:48:40.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.125+0000 7f0a1ffff700 1 --2- 172.21.15.47:0/1536735134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a200fe560 0x7f0a20107f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:40.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.125+0000 7f0a1effd700 1 --2- 172.21.15.47:0/1536735134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a200fdc90 0x7f0a2010e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:40.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.125+0000 7f0a1f7fe700 1 -- 172.21.15.47:0/1536735134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a200fdc90 msgr2=0x7f0a2010e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:40.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.125+0000 7f0a1f7fe700 1 --2- 172.21.15.47:0/1536735134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a200fdc90 0x7f0a2010e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:40.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.125+0000 7f0a1f7fe700 1 -- 172.21.15.47:0/1536735134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a200fe560 msgr2=0x7f0a20107f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:40.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.125+0000 7f0a1f7fe700 1 --2- 172.21.15.47:0/1536735134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a200fe560 0x7f0a20107f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:40.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.125+0000 7f0a1f7fe700 1 -- 172.21.15.47:0/1536735134 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a10005040 con 0x7f0a200f6880 2024-02-23T00:48:40.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.125+0000 7f0a1ffff700 1 --2- 172.21.15.47:0/1536735134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a200fe560 0x7f0a20107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:48:40.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.125+0000 7f0a1effd700 1 --2- 172.21.15.47:0/1536735134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a200fdc90 0x7f0a2010e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-23T00:48:40.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.125+0000 7f0a1f7fe700 1 --2- 172.21.15.47:0/1536735134 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a200f6880 0x7f0a2010dd80 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 rx=0x7f0a100048f0 tx=0x7f0a10009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:40.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.125+0000 7f0a1cff9700 10 client.?.objecter ms_handle_connect 0x7f0a200f6880 2024-02-23T00:48:40.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.125+0000 7f0a1cff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:48:40.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.125+0000 7f0a1cff9700 1 -- 172.21.15.47:0/1536735134 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10007b00 con 0x7f0a200f6880 2024-02-23T00:48:40.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.125+0000 7f0a1cff9700 1 -- 172.21.15.47:0/1536735134 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f0a1000f040 con 0x7f0a200f6880 2024-02-23T00:48:40.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.125+0000 7f0a26233700 1 -- 172.21.15.47:0/1536735134 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a180097c0 con 0x7f0a200f6880 2024-02-23T00:48:40.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.125+0000 7f0a1cff9700 1 -- 172.21.15.47:0/1536735134 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10014890 con 0x7f0a200f6880 2024-02-23T00:48:40.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.125+0000 7f0a26233700 10 client.41369.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:48:40.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.125+0000 7f0a26233700 1 -- 172.21.15.47:0/1536735134 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a20108d80 con 0x7f0a200f6880 2024-02-23T00:48:40.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.125+0000 7f0a26233700 1 librados: init done 2024-02-23T00:48:40.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.126+0000 7f0a26233700 1 -- 172.21.15.47:0/1536735134 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a2005f9b0 con 0x7f0a200f6880 2024-02-23T00:48:40.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.126+0000 7f0a1cff9700 1 -- 172.21.15.47:0/1536735134 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f0a10014cc0 con 0x7f0a200f6880 2024-02-23T00:48:40.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.127+0000 7f0a1cff9700 1 --2- 172.21.15.47:0/1536735134 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0a08062cc0 0x7f0a08065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:40.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.127+0000 7f0a1cff9700 1 -- 172.21.15.47:0/1536735134 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(431..431 src has 48..431) v4 ==== 9248+0+0 (secure 0 0 0) 0x7f0a1007d290 con 0x7f0a200f6880 2024-02-23T00:48:40.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.127+0000 7f0a1cff9700 10 client.41369.objecter ms_dispatch 0x7f0a20001550 osd_map(431..431 src has 48..431) v4 2024-02-23T00:48:40.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.127+0000 7f0a1cff9700 3 client.41369.objecter handle_osd_map got epochs [431,431] > 0 2024-02-23T00:48:40.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.127+0000 7f0a1cff9700 3 client.41369.objecter handle_osd_map decoding full epoch 431 2024-02-23T00:48:40.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.127+0000 7f0a1effd700 1 --2- 172.21.15.47:0/1536735134 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0a08062cc0 0x7f0a08065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:40.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.127+0000 7f0a1cff9700 20 client.41369.objecter dump_active .. 0 homeless 2024-02-23T00:48:40.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.127+0000 7f0a1cff9700 10 client.41369.objecter ms_handle_connect 0x7f0a08062cc0 2024-02-23T00:48:40.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.127+0000 7f0a1effd700 1 --2- 172.21.15.47:0/1536735134 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0a08062cc0 0x7f0a08065180 secure :-1 s=READY pgs=3070 cs=0 l=1 rev1=1 rx=0x7f0a14009a00 tx=0x7f0a1400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:40.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.131+0000 7f0a1cff9700 1 -- 172.21.15.47:0/1536735134 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f0a10049f60 con 0x7f0a200f6880 2024-02-23T00:48:40.196 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:40.321 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:48:40.334 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:40.335 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:48:40.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:40.334+0000 7f0a26233700 1 -- 172.21.15.47:0/1536735134 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7f0a20109400 con 0x7f0a200f6880 2024-02-23T00:48:40.436 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:48:40.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:39.294265+0000 mgr.y (mgr.14517) 400 : cluster [DBG] pgmap v630: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-23T00:48:40.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:40.191988+0000 mon.a (mon.0) 2593 : cluster [DBG] osdmap e432: 8 total, 8 up, 7 in 2024-02-23T00:48:40.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:40.336136+0000 mon.c (mon.2) 302 : audit [INF] from='client.? 172.21.15.47:0/1536735134' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:48:40.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:40.336471+0000 mon.a (mon.0) 2594 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:48:40.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:39.294265+0000 mgr.y (mgr.14517) 400 : cluster [DBG] pgmap v630: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-23T00:48:40.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:40.191988+0000 mon.a (mon.0) 2593 : cluster [DBG] osdmap e432: 8 total, 8 up, 7 in 2024-02-23T00:48:40.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:40.336136+0000 mon.c (mon.2) 302 : audit [INF] from='client.? 172.21.15.47:0/1536735134' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:48:40.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:40.336471+0000 mon.a (mon.0) 2594 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:48:40.464 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:39.294265+0000 mgr.y (mgr.14517) 400 : cluster [DBG] pgmap v630: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-23T00:48:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:40.191988+0000 mon.a (mon.0) 2593 : cluster [DBG] osdmap e432: 8 total, 8 up, 7 in 2024-02-23T00:48:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:40.336136+0000 mon.c (mon.2) 302 : audit [INF] from='client.? 172.21.15.47:0/1536735134' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:48:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:40.336471+0000 mon.a (mon.0) 2594 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:48:40.807 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:48:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:48:40] "GET /metrics HTTP/1.1" 200 188971 "" "Prometheus/2.33.4" 2024-02-23T00:48:40.814 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:48:40.816 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:48:40.818 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:40.818 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:48:40.838 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.3 dump_ops_in_flight 2024-02-23T00:48:40.919 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:48:40.951 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:40.972 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:48:40.975 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:40.975 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:48:41.076 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:48:41.105 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:41.126 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:48:41.128 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:41.129 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:48:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:41.194+0000 7f0a1cff9700 1 -- 172.21.15.47:0/1536735134 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v433) v1 ==== 91+0+0 (secure 0 0 0) 0x7f0a1004c890 con 0x7f0a200f6880 2024-02-23T00:48:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:41.196+0000 7f0a26233700 10 librados: watch_flush enter 2024-02-23T00:48:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:41.196+0000 7f0a26233700 10 librados: watch_flush exit 2024-02-23T00:48:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:41.196+0000 7f0a26233700 20 client.41369.objecter shutdown clearing up homeless session... 2024-02-23T00:48:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:41.196+0000 7f0a26233700 10 client.41369.objecter successfully canceled tick 2024-02-23T00:48:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:41.196+0000 7f0a26233700 1 -- 172.21.15.47:0/1536735134 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0a08062cc0 msgr2=0x7f0a08065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:41.196+0000 7f0a26233700 1 --2- 172.21.15.47:0/1536735134 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0a08062cc0 0x7f0a08065180 secure :-1 s=READY pgs=3070 cs=0 l=1 rev1=1 rx=0x7f0a14009a00 tx=0x7f0a1400b040).stop 2024-02-23T00:48:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:41.196+0000 7f0a26233700 1 -- 172.21.15.47:0/1536735134 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a200f6880 msgr2=0x7f0a2010dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:41.196+0000 7f0a26233700 1 --2- 172.21.15.47:0/1536735134 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a200f6880 0x7f0a2010dd80 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 rx=0x7f0a100048f0 tx=0x7f0a10009cb0).stop 2024-02-23T00:48:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:41.196+0000 7f0a26233700 1 -- 172.21.15.47:0/1536735134 shutdown_connections 2024-02-23T00:48:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:41.196+0000 7f0a26233700 1 --2- 172.21.15.47:0/1536735134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a200fe560 0x7f0a20107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:41.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:41.196+0000 7f0a26233700 1 --2- 172.21.15.47:0/1536735134 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0a08062cc0 0x7f0a08065180 unknown :-1 s=CLOSED pgs=3070 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:41.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:41.196+0000 7f0a26233700 1 --2- 172.21.15.47:0/1536735134 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a200f6880 0x7f0a2010dd80 unknown :-1 s=CLOSED pgs=2674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:41.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:41.196+0000 7f0a26233700 1 --2- 172.21.15.47:0/1536735134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a200fdc90 0x7f0a2010e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:41.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:41.196+0000 7f0a26233700 1 -- 172.21.15.47:0/1536735134 >> 172.21.15.47:0/1536735134 conn(0x7f0a2006ffc0 msgr2=0x7f0a2006d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:48:41.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:41.196+0000 7f0a26233700 1 -- 172.21.15.47:0/1536735134 shutdown_connections 2024-02-23T00:48:41.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:41.196+0000 7f0a26233700 1 -- 172.21.15.47:0/1536735134 wait complete. 2024-02-23T00:48:41.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:41.196+0000 7f0a26233700 1 librados: shutdown 2024-02-23T00:48:41.200 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is unset 2024-02-23T00:48:41.230 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:48:41.258 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:41.278 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:48:41.281 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:41.282 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:48:41.383 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:48:41.410 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:41.423 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-23T00:48:41.423 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-23T00:48:41.424 INFO:tasks.thrashosds.thrasher:Growing pool default.rgw.log 2024-02-23T00:48:41.424 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph pg dump --format=json 2024-02-23T00:48:41.428 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:48:41.430 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:41.431 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:48:41.532 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:48:41.560 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:41.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:40.521101+0000 osd.5 (osd.5) 178 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:48:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:40.521288+0000 osd.5 (osd.5) 179 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:48:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:41.189294+0000 mon.a (mon.0) 2595 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:48:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:41.193681+0000 mon.a (mon.0) 2596 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:48:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:41.193750+0000 mon.a (mon.0) 2597 : cluster [DBG] osdmap e433: 8 total, 8 up, 7 in 2024-02-23T00:48:41.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:40.521101+0000 osd.5 (osd.5) 178 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:48:41.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:40.521288+0000 osd.5 (osd.5) 179 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:48:41.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:41.189294+0000 mon.a (mon.0) 2595 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:48:41.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:41.193681+0000 mon.a (mon.0) 2596 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:48:41.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:41.193750+0000 mon.a (mon.0) 2597 : cluster [DBG] osdmap e433: 8 total, 8 up, 7 in 2024-02-23T00:48:41.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:40.521101+0000 osd.5 (osd.5) 178 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:48:41.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:40.521288+0000 osd.5 (osd.5) 179 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:48:41.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:41.189294+0000 mon.a (mon.0) 2595 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:48:41.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:41.193681+0000 mon.a (mon.0) 2596 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:48:41.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:41.193750+0000 mon.a (mon.0) 2597 : cluster [DBG] osdmap e433: 8 total, 8 up, 7 in 2024-02-23T00:48:42.409 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:48:42.411 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:42.412 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:48:42.446 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:48:42.513 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:48:42.545 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:42.565 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:48:42.568 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:42.568 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:48:42.669 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:48:42.693 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:48:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:48:42.694 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:48:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:48:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:48:42.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:41.294686+0000 mgr.y (mgr.14517) 401 : cluster [DBG] pgmap v633: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1006 B/s rd, 0 op/s 2024-02-23T00:48:42.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:41.685632+0000 osd.7 (osd.7) 205 : cluster [DBG] 18.2 scrub starts 2024-02-23T00:48:42.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:41.688725+0000 osd.7 (osd.7) 206 : cluster [DBG] 18.2 scrub ok 2024-02-23T00:48:42.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:41.716733+0000 osd.1 (osd.1) 187 : cluster [DBG] 2.a scrub starts 2024-02-23T00:48:42.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:41.719762+0000 osd.1 (osd.1) 188 : cluster [DBG] 2.a scrub ok 2024-02-23T00:48:42.695 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:41.294686+0000 mgr.y (mgr.14517) 401 : cluster [DBG] pgmap v633: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1006 B/s rd, 0 op/s 2024-02-23T00:48:42.695 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:41.685632+0000 osd.7 (osd.7) 205 : cluster [DBG] 18.2 scrub starts 2024-02-23T00:48:42.696 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:41.688725+0000 osd.7 (osd.7) 206 : cluster [DBG] 18.2 scrub ok 2024-02-23T00:48:42.696 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:41.716733+0000 osd.1 (osd.1) 187 : cluster [DBG] 2.a scrub starts 2024-02-23T00:48:42.696 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:41.719762+0000 osd.1 (osd.1) 188 : cluster [DBG] 2.a scrub ok 2024-02-23T00:48:42.696 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:41.294686+0000 mgr.y (mgr.14517) 401 : cluster [DBG] pgmap v633: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1006 B/s rd, 0 op/s 2024-02-23T00:48:42.696 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:41.685632+0000 osd.7 (osd.7) 205 : cluster [DBG] 18.2 scrub starts 2024-02-23T00:48:42.696 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:41.688725+0000 osd.7 (osd.7) 206 : cluster [DBG] 18.2 scrub ok 2024-02-23T00:48:42.696 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:41.716733+0000 osd.1 (osd.1) 187 : cluster [DBG] 2.a scrub starts 2024-02-23T00:48:42.696 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:41.719762+0000 osd.1 (osd.1) 188 : cluster [DBG] 2.a scrub ok 2024-02-23T00:48:42.698 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:43.247 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:48:43.250 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:43.250 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:48:43.351 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:48:43.383 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:43.405 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:48:43.408 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:43.409 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:48:43.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.481+0000 7f5e91e35700 1 -- 172.21.15.47:0/3545215286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e8c0fdbd0 msgr2=0x7f5e8c0fdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:43.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.481+0000 7f5e91e35700 1 --2- 172.21.15.47:0/3545215286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e8c0fdbd0 0x7f5e8c0fdfb0 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 rx=0x7f5e7c0047b0 tx=0x7f5e7c009cb0).stop 2024-02-23T00:48:43.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.481+0000 7f5e91e35700 1 -- 172.21.15.47:0/3545215286 shutdown_connections 2024-02-23T00:48:43.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.481+0000 7f5e91e35700 1 --2- 172.21.15.47:0/3545215286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e8c0f6a90 0x7f5e8c104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:43.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.481+0000 7f5e91e35700 1 --2- 172.21.15.47:0/3545215286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e8c0fdbd0 0x7f5e8c0fdfb0 unknown :-1 s=CLOSED pgs=2535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:43.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.481+0000 7f5e91e35700 1 --2- 172.21.15.47:0/3545215286 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5e8c0fe530 0x7f5e8c0f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:43.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.481+0000 7f5e91e35700 1 -- 172.21.15.47:0/3545215286 >> 172.21.15.47:0/3545215286 conn(0x7f5e8c06c370 msgr2=0x7f5e8c06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:48:43.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.481+0000 7f5e91e35700 1 -- 172.21.15.47:0/3545215286 shutdown_connections 2024-02-23T00:48:43.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.481+0000 7f5e91e35700 1 -- 172.21.15.47:0/3545215286 wait complete. 2024-02-23T00:48:43.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.481+0000 7f5e91e35700 1 librados: starting msgr at 2024-02-23T00:48:43.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.481+0000 7f5e91e35700 1 librados: starting objecter 2024-02-23T00:48:43.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.481+0000 7f5e91e35700 1 Processor -- start 2024-02-23T00:48:43.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.482+0000 7f5e91e35700 1 -- start start 2024-02-23T00:48:43.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.482+0000 7f5e91e35700 1 librados: setting wanted keys 2024-02-23T00:48:43.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.482+0000 7f5e91e35700 1 librados: calling monclient init 2024-02-23T00:48:43.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.482+0000 7f5e91e35700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e8c0f6a90 0x7f5e8c110200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:43.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.482+0000 7f5e91e35700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5e8c0fdbd0 0x7f5e8c1106f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:43.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.482+0000 7f5e91e35700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e8c0fe530 0x7f5e8c10a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:43.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.482+0000 7f5e91e35700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c10a890 con 0x7f5e8c0f6a90 2024-02-23T00:48:43.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.482+0000 7f5e91e35700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c10aa00 con 0x7f5e8c0fe530 2024-02-23T00:48:43.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.482+0000 7f5e91e35700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c10abc0 con 0x7f5e8c0fdbd0 2024-02-23T00:48:43.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.482+0000 7f5e8b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e8c0f6a90 0x7f5e8c110200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:43.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.482+0000 7f5e8affd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5e8c0fdbd0 0x7f5e8c1106f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:43.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.482+0000 7f5e8affd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5e8c0fdbd0 0x7f5e8c1106f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:56534/0 (socket says 172.21.15.47:56534) 2024-02-23T00:48:43.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.482+0000 7f5e8b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e8c0f6a90 0x7f5e8c110200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41140/0 (socket says 172.21.15.47:41140) 2024-02-23T00:48:43.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.482+0000 7f5e8affd700 1 -- 172.21.15.47:0/776364749 learned_addr learned my addr 172.21.15.47:0/776364749 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:48:43.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.482+0000 7f5e8bfff700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e8c0fe530 0x7f5e8c10a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:43.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.482+0000 7f5e8b7fe700 1 -- 172.21.15.47:0/776364749 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5e8c0fdbd0 msgr2=0x7f5e8c1106f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:43.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.482+0000 7f5e8b7fe700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5e8c0fdbd0 0x7f5e8c1106f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:43.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.482+0000 7f5e8b7fe700 1 -- 172.21.15.47:0/776364749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e8c0fe530 msgr2=0x7f5e8c10a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:43.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.482+0000 7f5e8b7fe700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e8c0fe530 0x7f5e8c10a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:43.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.482+0000 7f5e8b7fe700 1 -- 172.21.15.47:0/776364749 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e800097c0 con 0x7f5e8c0f6a90 2024-02-23T00:48:43.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.482+0000 7f5e8affd700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5e8c0fdbd0 0x7f5e8c1106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:48:43.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.482+0000 7f5e8bfff700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e8c0fe530 0x7f5e8c10a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-23T00:48:43.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.482+0000 7f5e8b7fe700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e8c0f6a90 0x7f5e8c110200 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 rx=0x7f5e7c004650 tx=0x7f5e7c009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:43.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.482+0000 7f5e88ff9700 10 client.?.objecter ms_handle_connect 0x7f5e8c0f6a90 2024-02-23T00:48:43.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.482+0000 7f5e88ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:48:43.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.483+0000 7f5e88ff9700 1 -- 172.21.15.47:0/776364749 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c018070 con 0x7f5e8c0f6a90 2024-02-23T00:48:43.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.483+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e7c005040 con 0x7f5e8c0f6a90 2024-02-23T00:48:43.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.483+0000 7f5e88ff9700 1 -- 172.21.15.47:0/776364749 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f5e7c0096d0 con 0x7f5e8c0f6a90 2024-02-23T00:48:43.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.483+0000 7f5e88ff9700 1 -- 172.21.15.47:0/776364749 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c01dbe0 con 0x7f5e8c0f6a90 2024-02-23T00:48:43.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.483+0000 7f5e91e35700 10 client.31050.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:48:43.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.483+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e8c10b060 con 0x7f5e8c0f6a90 2024-02-23T00:48:43.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.483+0000 7f5e91e35700 1 librados: init done 2024-02-23T00:48:43.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.484+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f5e8c10b250 con 0x7f5e8c0f6a90 2024-02-23T00:48:43.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.484+0000 7f5e88ff9700 1 -- 172.21.15.47:0/776364749 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f5e7c009240 con 0x7f5e8c0f6a90 2024-02-23T00:48:43.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.484+0000 7f5e88ff9700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5e74062cc0 0x7f5e74065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:43.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.485+0000 7f5e88ff9700 1 -- 172.21.15.47:0/776364749 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(433..433 src has 48..433) v4 ==== 9220+0+0 (secure 0 0 0) 0x7f5e7c07eb20 con 0x7f5e8c0f6a90 2024-02-23T00:48:43.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.485+0000 7f5e88ff9700 10 client.31050.objecter ms_dispatch 0x7f5e8c001550 osd_map(433..433 src has 48..433) v4 2024-02-23T00:48:43.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.485+0000 7f5e88ff9700 3 client.31050.objecter handle_osd_map got epochs [433,433] > 0 2024-02-23T00:48:43.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.485+0000 7f5e88ff9700 3 client.31050.objecter handle_osd_map decoding full epoch 433 2024-02-23T00:48:43.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.485+0000 7f5e8affd700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5e74062cc0 0x7f5e74065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:43.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.485+0000 7f5e88ff9700 20 client.31050.objecter dump_active .. 0 homeless 2024-02-23T00:48:43.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.485+0000 7f5e88ff9700 1 -- 172.21.15.47:0/776364749 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v433) v1 ==== 54+0+16 (secure 0 0 0) 0x7f5e7c083220 con 0x7f5e8c0f6a90 2024-02-23T00:48:43.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.485+0000 7f5e91e35700 10 client.31050.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:43.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.485+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_assign -1 1 2024-02-23T00:48:43.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.485+0000 7f5e88ff9700 10 client.31050.objecter ms_handle_connect 0x7f5e74062cc0 2024-02-23T00:48:43.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.485+0000 7f5e8affd700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5e74062cc0 0x7f5e74065180 secure :-1 s=READY pgs=3071 cs=0 l=1 rev1=1 rx=0x7f5e8c10bb40 tx=0x7f5e80004d10).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:43.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.485+0000 7f5e91e35700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f5e8c0f9d30 0x7f5e8c0fa110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:43.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.485+0000 7f5e91e35700 20 client.31050.objecter _get_session s=0x7f5e8c05f970 osd=0 3 2024-02-23T00:48:43.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.485+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c05f970 osd=0 3 2024-02-23T00:48:43.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.485+0000 7f5e91e35700 20 client.31050.objecter _get_session s=0x7f5e8c05f970 osd=0 3 2024-02-23T00:48:43.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.485+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_remove -1 1 2024-02-23T00:48:43.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.485+0000 7f5e91e35700 20 client.31050.objecter get_session s=0x7f5e8c05f970 osd=0 3 2024-02-23T00:48:43.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.485+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_assign 0 1 2024-02-23T00:48:43.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.485+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c05f970 osd=0 4 2024-02-23T00:48:43.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.485+0000 7f5e91e35700 10 client.31050.objecter _send_command 1 2024-02-23T00:48:43.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.485+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5e8c05ee80 con 0x7f5e8c0f9d30 2024-02-23T00:48:43.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.485+0000 7f5e8bfff700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f5e8c0f9d30 0x7f5e8c0fa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:43.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.485+0000 7f5e88ff9700 10 client.31050.objecter ms_handle_connect 0x7f5e8c0f9d30 2024-02-23T00:48:43.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.485+0000 7f5e8bfff700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f5e8c0f9d30 0x7f5e8c0fa110 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:43.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.486+0000 7f5e88ff9700 1 -- 172.21.15.47:0/776364749 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5e8c05ee80 con 0x7f5e8c0f9d30 2024-02-23T00:48:43.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.486+0000 7f5e88ff9700 10 client.31050.objecter ms_dispatch 0x7f5e8c001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:48:43.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.486+0000 7f5e88ff9700 10 client.31050.objecter _finish_command 1 = system:0 2024-02-23T00:48:43.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.486+0000 7f5e88ff9700 20 client.31050.objecter put_session s=0x7f5e8c05f970 osd=0 4 2024-02-23T00:48:43.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.486+0000 7f5e88ff9700 15 client.31050.objecter _session_command_op_remove 0 1 2024-02-23T00:48:43.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.501+0000 7f5e91e35700 10 client.31050.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:48:43.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.501+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_assign -1 2 2024-02-23T00:48:43.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.501+0000 7f5e91e35700 20 client.31050.objecter _get_session s=0x7f5e8c05f970 osd=0 3 2024-02-23T00:48:43.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.501+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c05f970 osd=0 3 2024-02-23T00:48:43.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.501+0000 7f5e91e35700 20 client.31050.objecter _get_session s=0x7f5e8c05f970 osd=0 3 2024-02-23T00:48:43.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.501+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_remove -1 2 2024-02-23T00:48:43.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.501+0000 7f5e91e35700 20 client.31050.objecter get_session s=0x7f5e8c05f970 osd=0 3 2024-02-23T00:48:43.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.501+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_assign 0 2 2024-02-23T00:48:43.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.501+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c05f970 osd=0 4 2024-02-23T00:48:43.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.501+0000 7f5e91e35700 10 client.31050.objecter _send_command 2 2024-02-23T00:48:43.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.501+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f5e8c106e90 con 0x7f5e8c0f9d30 2024-02-23T00:48:43.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.502+0000 7f5e88ff9700 1 -- 172.21.15.47:0/776364749 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f5e8c106e90 con 0x7f5e8c0f9d30 2024-02-23T00:48:43.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.502+0000 7f5e88ff9700 10 client.31050.objecter ms_dispatch 0x7f5e8c001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:48:43.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.502+0000 7f5e88ff9700 10 client.31050.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:48:43.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.502+0000 7f5e88ff9700 20 client.31050.objecter put_session s=0x7f5e8c05f970 osd=0 4 2024-02-23T00:48:43.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.502+0000 7f5e88ff9700 15 client.31050.objecter _session_command_op_remove 0 2 2024-02-23T00:48:43.505 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:48:43.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.503+0000 7f5e91e35700 10 client.31050.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:43.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.503+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_assign -1 3 2024-02-23T00:48:43.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.503+0000 7f5e91e35700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f5e8c0598f0 0x7f5e8c059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:43.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.503+0000 7f5e91e35700 20 client.31050.objecter _get_session s=0x7f5e8c028540 osd=1 3 2024-02-23T00:48:43.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.503+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c028540 osd=1 3 2024-02-23T00:48:43.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.503+0000 7f5e91e35700 20 client.31050.objecter _get_session s=0x7f5e8c028540 osd=1 3 2024-02-23T00:48:43.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.503+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_remove -1 3 2024-02-23T00:48:43.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.503+0000 7f5e91e35700 20 client.31050.objecter get_session s=0x7f5e8c028540 osd=1 3 2024-02-23T00:48:43.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.503+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_assign 1 3 2024-02-23T00:48:43.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.503+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c028540 osd=1 4 2024-02-23T00:48:43.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.503+0000 7f5e91e35700 10 client.31050.objecter _send_command 3 2024-02-23T00:48:43.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.503+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5e8c0286b0 con 0x7f5e8c0598f0 2024-02-23T00:48:43.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.503+0000 7f5e8b7fe700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f5e8c0598f0 0x7f5e8c059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:43.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.503+0000 7f5e88ff9700 10 client.31050.objecter ms_handle_connect 0x7f5e8c0598f0 2024-02-23T00:48:43.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.503+0000 7f5e8b7fe700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f5e8c0598f0 0x7f5e8c059cd0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:43.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.504+0000 7f5e88ff9700 1 -- 172.21.15.47:0/776364749 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5e8c0286b0 con 0x7f5e8c0598f0 2024-02-23T00:48:43.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.504+0000 7f5e88ff9700 10 client.31050.objecter ms_dispatch 0x7f5e8c001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:48:43.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.504+0000 7f5e88ff9700 10 client.31050.objecter _finish_command 3 = system:0 2024-02-23T00:48:43.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.504+0000 7f5e88ff9700 20 client.31050.objecter put_session s=0x7f5e8c028540 osd=1 4 2024-02-23T00:48:43.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.504+0000 7f5e88ff9700 15 client.31050.objecter _session_command_op_remove 1 3 2024-02-23T00:48:43.509 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:48:43.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.518+0000 7f5e91e35700 10 client.31050.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:48:43.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.518+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_assign -1 4 2024-02-23T00:48:43.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.518+0000 7f5e91e35700 20 client.31050.objecter _get_session s=0x7f5e8c028540 osd=1 3 2024-02-23T00:48:43.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.518+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c028540 osd=1 3 2024-02-23T00:48:43.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.518+0000 7f5e91e35700 20 client.31050.objecter _get_session s=0x7f5e8c028540 osd=1 3 2024-02-23T00:48:43.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.518+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_remove -1 4 2024-02-23T00:48:43.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.518+0000 7f5e91e35700 20 client.31050.objecter get_session s=0x7f5e8c028540 osd=1 3 2024-02-23T00:48:43.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.518+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_assign 1 4 2024-02-23T00:48:43.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.518+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c028540 osd=1 4 2024-02-23T00:48:43.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.518+0000 7f5e91e35700 10 client.31050.objecter _send_command 4 2024-02-23T00:48:43.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.518+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f5e8c028820 con 0x7f5e8c0598f0 2024-02-23T00:48:43.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.520+0000 7f5e88ff9700 1 -- 172.21.15.47:0/776364749 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f5e7c0516f0 con 0x7f5e8c0598f0 2024-02-23T00:48:43.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.520+0000 7f5e88ff9700 10 client.31050.objecter ms_dispatch 0x7f5e8c001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:48:43.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.520+0000 7f5e88ff9700 10 client.31050.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:48:43.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.520+0000 7f5e88ff9700 20 client.31050.objecter put_session s=0x7f5e8c028540 osd=1 4 2024-02-23T00:48:43.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.520+0000 7f5e88ff9700 15 client.31050.objecter _session_command_op_remove 1 4 2024-02-23T00:48:43.524 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:48:43.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.520+0000 7f5e91e35700 10 client.31050.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:43.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.520+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_assign -1 5 2024-02-23T00:48:43.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.520+0000 7f5e91e35700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f5e8c0280a0 0x7f5e8c11e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:43.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.520+0000 7f5e91e35700 20 client.31050.objecter _get_session s=0x7f5e8c0595a0 osd=2 3 2024-02-23T00:48:43.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.520+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c0595a0 osd=2 3 2024-02-23T00:48:43.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.520+0000 7f5e91e35700 20 client.31050.objecter _get_session s=0x7f5e8c0595a0 osd=2 3 2024-02-23T00:48:43.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.520+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_remove -1 5 2024-02-23T00:48:43.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.520+0000 7f5e91e35700 20 client.31050.objecter get_session s=0x7f5e8c0595a0 osd=2 3 2024-02-23T00:48:43.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.520+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_assign 2 5 2024-02-23T00:48:43.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.520+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c0595a0 osd=2 4 2024-02-23T00:48:43.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.520+0000 7f5e91e35700 10 client.31050.objecter _send_command 5 2024-02-23T00:48:43.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.520+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5e8c120e30 con 0x7f5e8c0280a0 2024-02-23T00:48:43.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.520+0000 7f5e8affd700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f5e8c0280a0 0x7f5e8c11e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:43.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.521+0000 7f5e88ff9700 10 client.31050.objecter ms_handle_connect 0x7f5e8c0280a0 2024-02-23T00:48:43.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.521+0000 7f5e8affd700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f5e8c0280a0 0x7f5e8c11e1e0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:43.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.521+0000 7f5e88ff9700 1 -- 172.21.15.47:0/776364749 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5e8c120e30 con 0x7f5e8c0280a0 2024-02-23T00:48:43.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.521+0000 7f5e88ff9700 10 client.31050.objecter ms_dispatch 0x7f5e8c001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:48:43.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.521+0000 7f5e88ff9700 10 client.31050.objecter _finish_command 5 = system:0 2024-02-23T00:48:43.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.521+0000 7f5e88ff9700 20 client.31050.objecter put_session s=0x7f5e8c0595a0 osd=2 4 2024-02-23T00:48:43.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.521+0000 7f5e88ff9700 15 client.31050.objecter _session_command_op_remove 2 5 2024-02-23T00:48:43.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.536+0000 7f5e91e35700 10 client.31050.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:48:43.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.536+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_assign -1 6 2024-02-23T00:48:43.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.536+0000 7f5e91e35700 20 client.31050.objecter _get_session s=0x7f5e8c0595a0 osd=2 3 2024-02-23T00:48:43.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.536+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c0595a0 osd=2 3 2024-02-23T00:48:43.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.536+0000 7f5e91e35700 20 client.31050.objecter _get_session s=0x7f5e8c0595a0 osd=2 3 2024-02-23T00:48:43.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.536+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_remove -1 6 2024-02-23T00:48:43.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.536+0000 7f5e91e35700 20 client.31050.objecter get_session s=0x7f5e8c0595a0 osd=2 3 2024-02-23T00:48:43.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.536+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_assign 2 6 2024-02-23T00:48:43.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.536+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c0595a0 osd=2 4 2024-02-23T00:48:43.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.536+0000 7f5e91e35700 10 client.31050.objecter _send_command 6 2024-02-23T00:48:43.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.536+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f5e8c027600 con 0x7f5e8c0280a0 2024-02-23T00:48:43.539 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:43.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.537+0000 7f5e88ff9700 1 -- 172.21.15.47:0/776364749 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f5e8c027600 con 0x7f5e8c0280a0 2024-02-23T00:48:43.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.537+0000 7f5e88ff9700 10 client.31050.objecter ms_dispatch 0x7f5e8c001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:48:43.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.537+0000 7f5e88ff9700 10 client.31050.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:48:43.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.537+0000 7f5e88ff9700 20 client.31050.objecter put_session s=0x7f5e8c0595a0 osd=2 4 2024-02-23T00:48:43.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.537+0000 7f5e88ff9700 15 client.31050.objecter _session_command_op_remove 2 6 2024-02-23T00:48:43.540 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:48:43.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.537+0000 7f5e91e35700 10 client.31050.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:43.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.537+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_assign -1 7 2024-02-23T00:48:43.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.537+0000 7f5e91e35700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f5e8c11bd30 0x7f5e8c11f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:43.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.537+0000 7f5e91e35700 20 client.31050.objecter _get_session s=0x7f5e8c0279b0 osd=3 3 2024-02-23T00:48:43.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.537+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c0279b0 osd=3 3 2024-02-23T00:48:43.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.537+0000 7f5e91e35700 20 client.31050.objecter _get_session s=0x7f5e8c0279b0 osd=3 3 2024-02-23T00:48:43.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.537+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_remove -1 7 2024-02-23T00:48:43.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.537+0000 7f5e91e35700 20 client.31050.objecter get_session s=0x7f5e8c0279b0 osd=3 3 2024-02-23T00:48:43.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.537+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_assign 3 7 2024-02-23T00:48:43.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.537+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c0279b0 osd=3 4 2024-02-23T00:48:43.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.537+0000 7f5e91e35700 10 client.31050.objecter _send_command 7 2024-02-23T00:48:43.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.537+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5e8c11fe90 con 0x7f5e8c11bd30 2024-02-23T00:48:43.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.537+0000 7f5e8bfff700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f5e8c11bd30 0x7f5e8c11f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:43.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.538+0000 7f5e88ff9700 10 client.31050.objecter ms_handle_connect 0x7f5e8c11bd30 2024-02-23T00:48:43.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.538+0000 7f5e8bfff700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f5e8c11bd30 0x7f5e8c11f850 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:43.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.538+0000 7f5e88ff9700 1 -- 172.21.15.47:0/776364749 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5e8c11fe90 con 0x7f5e8c11bd30 2024-02-23T00:48:43.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.538+0000 7f5e88ff9700 10 client.31050.objecter ms_dispatch 0x7f5e8c001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:48:43.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.538+0000 7f5e88ff9700 10 client.31050.objecter _finish_command 7 = system:0 2024-02-23T00:48:43.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.538+0000 7f5e88ff9700 20 client.31050.objecter put_session s=0x7f5e8c0279b0 osd=3 4 2024-02-23T00:48:43.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.538+0000 7f5e88ff9700 15 client.31050.objecter _session_command_op_remove 3 7 2024-02-23T00:48:43.552 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:48:43.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.554+0000 7f5e91e35700 10 client.31050.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:48:43.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.554+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_assign -1 8 2024-02-23T00:48:43.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.554+0000 7f5e91e35700 20 client.31050.objecter _get_session s=0x7f5e8c0279b0 osd=3 3 2024-02-23T00:48:43.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.554+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c0279b0 osd=3 3 2024-02-23T00:48:43.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.554+0000 7f5e91e35700 20 client.31050.objecter _get_session s=0x7f5e8c0279b0 osd=3 3 2024-02-23T00:48:43.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.554+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_remove -1 8 2024-02-23T00:48:43.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.554+0000 7f5e91e35700 20 client.31050.objecter get_session s=0x7f5e8c0279b0 osd=3 3 2024-02-23T00:48:43.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.554+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_assign 3 8 2024-02-23T00:48:43.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.554+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c0279b0 osd=3 4 2024-02-23T00:48:43.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.554+0000 7f5e91e35700 10 client.31050.objecter _send_command 8 2024-02-23T00:48:43.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.554+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f5e8c11b290 con 0x7f5e8c11bd30 2024-02-23T00:48:43.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:42.407234+0000 osd.6 (osd.6) 171 : cluster [DBG] 4.a scrub starts 2024-02-23T00:48:43.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:42.409675+0000 osd.6 (osd.6) 172 : cluster [DBG] 4.a scrub ok 2024-02-23T00:48:43.561 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:42.453155+0000 osd.5 (osd.5) 180 : cluster [DBG] 62.1d scrub starts 2024-02-23T00:48:43.561 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:42.456611+0000 osd.5 (osd.5) 181 : cluster [DBG] 62.1d scrub ok 2024-02-23T00:48:43.561 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:42.607075+0000 mgr.y (mgr.14517) 402 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:48:43.561 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:42.687793+0000 osd.7 (osd.7) 207 : cluster [DBG] 62.1e scrub starts 2024-02-23T00:48:43.561 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:42.689034+0000 osd.7 (osd.7) 208 : cluster [DBG] 62.1e scrub ok 2024-02-23T00:48:43.561 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:42.407234+0000 osd.6 (osd.6) 171 : cluster [DBG] 4.a scrub starts 2024-02-23T00:48:43.561 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:42.409675+0000 osd.6 (osd.6) 172 : cluster [DBG] 4.a scrub ok 2024-02-23T00:48:43.561 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:42.453155+0000 osd.5 (osd.5) 180 : cluster [DBG] 62.1d scrub starts 2024-02-23T00:48:43.561 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:42.456611+0000 osd.5 (osd.5) 181 : cluster [DBG] 62.1d scrub ok 2024-02-23T00:48:43.561 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:42.607075+0000 mgr.y (mgr.14517) 402 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:48:43.562 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:42.687793+0000 osd.7 (osd.7) 207 : cluster [DBG] 62.1e scrub starts 2024-02-23T00:48:43.562 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:42.689034+0000 osd.7 (osd.7) 208 : cluster [DBG] 62.1e scrub ok 2024-02-23T00:48:43.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.555+0000 7f5e88ff9700 1 -- 172.21.15.47:0/776364749 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f5e8c11b290 con 0x7f5e8c11bd30 2024-02-23T00:48:43.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.555+0000 7f5e88ff9700 10 client.31050.objecter ms_dispatch 0x7f5e8c001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:48:43.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.555+0000 7f5e88ff9700 10 client.31050.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:48:43.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.555+0000 7f5e88ff9700 20 client.31050.objecter put_session s=0x7f5e8c0279b0 osd=3 4 2024-02-23T00:48:43.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.555+0000 7f5e88ff9700 15 client.31050.objecter _session_command_op_remove 3 8 2024-02-23T00:48:43.563 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:48:43.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.556+0000 7f5e91e35700 10 client.31050.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:43.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.556+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_assign -1 9 2024-02-23T00:48:43.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.556+0000 7f5e91e35700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f5e8c11b400 0x7f5e8c124fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:43.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.556+0000 7f5e91e35700 20 client.31050.objecter _get_session s=0x7f5e8c11b290 osd=4 3 2024-02-23T00:48:43.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.556+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c11b290 osd=4 3 2024-02-23T00:48:43.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.556+0000 7f5e91e35700 20 client.31050.objecter _get_session s=0x7f5e8c11b290 osd=4 3 2024-02-23T00:48:43.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.556+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_remove -1 9 2024-02-23T00:48:43.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.556+0000 7f5e91e35700 20 client.31050.objecter get_session s=0x7f5e8c11b290 osd=4 3 2024-02-23T00:48:43.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.556+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_assign 4 9 2024-02-23T00:48:43.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.556+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c11b290 osd=4 4 2024-02-23T00:48:43.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.556+0000 7f5e91e35700 10 client.31050.objecter _send_command 9 2024-02-23T00:48:43.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.556+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5e8c127d60 con 0x7f5e8c11b400 2024-02-23T00:48:43.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.556+0000 7f5e8b7fe700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f5e8c11b400 0x7f5e8c124fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:43.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.556+0000 7f5e8b7fe700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f5e8c11b400 0x7f5e8c124fc0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:43.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.556+0000 7f5e88ff9700 10 client.31050.objecter ms_handle_connect 0x7f5e8c11b400 2024-02-23T00:48:43.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.557+0000 7f5e88ff9700 1 -- 172.21.15.47:0/776364749 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5e8c127d60 con 0x7f5e8c11b400 2024-02-23T00:48:43.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.557+0000 7f5e88ff9700 10 client.31050.objecter ms_dispatch 0x7f5e8c001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:48:43.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.557+0000 7f5e88ff9700 10 client.31050.objecter _finish_command 9 = system:0 2024-02-23T00:48:43.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.557+0000 7f5e88ff9700 20 client.31050.objecter put_session s=0x7f5e8c11b290 osd=4 4 2024-02-23T00:48:43.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.557+0000 7f5e88ff9700 15 client.31050.objecter _session_command_op_remove 4 9 2024-02-23T00:48:43.567 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:43.567 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:48:43.567 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:48:43.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.573+0000 7f5e91e35700 10 client.31050.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:48:43.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.573+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_assign -1 10 2024-02-23T00:48:43.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.573+0000 7f5e91e35700 20 client.31050.objecter _get_session s=0x7f5e8c11b290 osd=4 3 2024-02-23T00:48:43.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.573+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c11b290 osd=4 3 2024-02-23T00:48:43.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.573+0000 7f5e91e35700 20 client.31050.objecter _get_session s=0x7f5e8c11b290 osd=4 3 2024-02-23T00:48:43.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.573+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_remove -1 10 2024-02-23T00:48:43.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.573+0000 7f5e91e35700 20 client.31050.objecter get_session s=0x7f5e8c11b290 osd=4 3 2024-02-23T00:48:43.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.573+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_assign 4 10 2024-02-23T00:48:43.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.573+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c11b290 osd=4 4 2024-02-23T00:48:43.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.573+0000 7f5e91e35700 10 client.31050.objecter _send_command 10 2024-02-23T00:48:43.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.573+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f5e8c127d60 con 0x7f5e8c11b400 2024-02-23T00:48:43.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.575+0000 7f5e88ff9700 1 -- 172.21.15.47:0/776364749 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f5e8c127d60 con 0x7f5e8c11b400 2024-02-23T00:48:43.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.575+0000 7f5e88ff9700 10 client.31050.objecter ms_dispatch 0x7f5e8c001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:48:43.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.575+0000 7f5e88ff9700 10 client.31050.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:48:43.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.575+0000 7f5e88ff9700 20 client.31050.objecter put_session s=0x7f5e8c11b290 osd=4 4 2024-02-23T00:48:43.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.575+0000 7f5e88ff9700 15 client.31050.objecter _session_command_op_remove 4 10 2024-02-23T00:48:43.579 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:48:43.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.575+0000 7f5e91e35700 10 client.31050.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:43.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.575+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_assign -1 11 2024-02-23T00:48:43.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.575+0000 7f5e91e35700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f5e8c127100 0x7f5e8c127580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:43.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.575+0000 7f5e91e35700 20 client.31050.objecter _get_session s=0x7f5e8c127d60 osd=5 3 2024-02-23T00:48:43.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.575+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c127d60 osd=5 3 2024-02-23T00:48:43.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.575+0000 7f5e91e35700 20 client.31050.objecter _get_session s=0x7f5e8c127d60 osd=5 3 2024-02-23T00:48:43.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.575+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_remove -1 11 2024-02-23T00:48:43.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.575+0000 7f5e91e35700 20 client.31050.objecter get_session s=0x7f5e8c127d60 osd=5 3 2024-02-23T00:48:43.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.575+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_assign 5 11 2024-02-23T00:48:43.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.575+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c127d60 osd=5 4 2024-02-23T00:48:43.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.575+0000 7f5e91e35700 10 client.31050.objecter _send_command 11 2024-02-23T00:48:43.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.575+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5e8c12b090 con 0x7f5e8c127100 2024-02-23T00:48:43.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.576+0000 7f5e8affd700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f5e8c127100 0x7f5e8c127580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:43.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.576+0000 7f5e88ff9700 10 client.31050.objecter ms_handle_connect 0x7f5e8c127100 2024-02-23T00:48:43.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.576+0000 7f5e8affd700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f5e8c127100 0x7f5e8c127580 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:43.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.577+0000 7f5e88ff9700 1 -- 172.21.15.47:0/776364749 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5e8c12b090 con 0x7f5e8c127100 2024-02-23T00:48:43.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.577+0000 7f5e88ff9700 10 client.31050.objecter ms_dispatch 0x7f5e8c001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:48:43.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.577+0000 7f5e88ff9700 10 client.31050.objecter _finish_command 11 = system:0 2024-02-23T00:48:43.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.577+0000 7f5e88ff9700 20 client.31050.objecter put_session s=0x7f5e8c127d60 osd=5 4 2024-02-23T00:48:43.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.577+0000 7f5e88ff9700 15 client.31050.objecter _session_command_op_remove 5 11 2024-02-23T00:48:43.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.592+0000 7f5e91e35700 10 client.31050.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:48:43.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.592+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_assign -1 12 2024-02-23T00:48:43.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.592+0000 7f5e91e35700 20 client.31050.objecter _get_session s=0x7f5e8c127d60 osd=5 3 2024-02-23T00:48:43.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.592+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c127d60 osd=5 3 2024-02-23T00:48:43.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.592+0000 7f5e91e35700 20 client.31050.objecter _get_session s=0x7f5e8c127d60 osd=5 3 2024-02-23T00:48:43.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.592+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_remove -1 12 2024-02-23T00:48:43.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.592+0000 7f5e91e35700 20 client.31050.objecter get_session s=0x7f5e8c127d60 osd=5 3 2024-02-23T00:48:43.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.592+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_assign 5 12 2024-02-23T00:48:43.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.592+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c127d60 osd=5 4 2024-02-23T00:48:43.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.592+0000 7f5e91e35700 10 client.31050.objecter _send_command 12 2024-02-23T00:48:43.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.592+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f5e8c125520 con 0x7f5e8c127100 2024-02-23T00:48:43.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.593+0000 7f5e88ff9700 1 -- 172.21.15.47:0/776364749 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f5e8c125520 con 0x7f5e8c127100 2024-02-23T00:48:43.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.593+0000 7f5e88ff9700 10 client.31050.objecter ms_dispatch 0x7f5e8c001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:48:43.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.593+0000 7f5e88ff9700 10 client.31050.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:48:43.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.593+0000 7f5e88ff9700 20 client.31050.objecter put_session s=0x7f5e8c127d60 osd=5 4 2024-02-23T00:48:43.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.593+0000 7f5e88ff9700 15 client.31050.objecter _session_command_op_remove 5 12 2024-02-23T00:48:43.596 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:48:43.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.593+0000 7f5e91e35700 10 client.31050.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:43.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.593+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_assign -1 13 2024-02-23T00:48:43.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.593+0000 7f5e91e35700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f5e8c125ba0 0x7f5e8c125fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:43.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.593+0000 7f5e91e35700 20 client.31050.objecter _get_session s=0x7f5e8c125520 osd=6 3 2024-02-23T00:48:43.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.593+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c125520 osd=6 3 2024-02-23T00:48:43.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.593+0000 7f5e91e35700 20 client.31050.objecter _get_session s=0x7f5e8c125520 osd=6 3 2024-02-23T00:48:43.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.593+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_remove -1 13 2024-02-23T00:48:43.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.593+0000 7f5e91e35700 20 client.31050.objecter get_session s=0x7f5e8c125520 osd=6 3 2024-02-23T00:48:43.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.593+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_assign 6 13 2024-02-23T00:48:43.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.593+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c125520 osd=6 4 2024-02-23T00:48:43.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.593+0000 7f5e91e35700 10 client.31050.objecter _send_command 13 2024-02-23T00:48:43.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.593+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5e8c126700 con 0x7f5e8c125ba0 2024-02-23T00:48:43.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.594+0000 7f5e8bfff700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f5e8c125ba0 0x7f5e8c125fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:43.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.594+0000 7f5e88ff9700 10 client.31050.objecter ms_handle_connect 0x7f5e8c125ba0 2024-02-23T00:48:43.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.594+0000 7f5e8bfff700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f5e8c125ba0 0x7f5e8c125fe0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:43.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.595+0000 7f5e88ff9700 1 -- 172.21.15.47:0/776364749 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5e8c126700 con 0x7f5e8c125ba0 2024-02-23T00:48:43.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.595+0000 7f5e88ff9700 10 client.31050.objecter ms_dispatch 0x7f5e8c001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:48:43.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.595+0000 7f5e88ff9700 10 client.31050.objecter _finish_command 13 = system:0 2024-02-23T00:48:43.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.595+0000 7f5e88ff9700 20 client.31050.objecter put_session s=0x7f5e8c125520 osd=6 4 2024-02-23T00:48:43.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.595+0000 7f5e88ff9700 15 client.31050.objecter _session_command_op_remove 6 13 2024-02-23T00:48:43.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.609+0000 7f5e91e35700 10 client.31050.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:48:43.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.609+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_assign -1 14 2024-02-23T00:48:43.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.609+0000 7f5e91e35700 20 client.31050.objecter _get_session s=0x7f5e8c125520 osd=6 3 2024-02-23T00:48:43.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.609+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c125520 osd=6 3 2024-02-23T00:48:43.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.609+0000 7f5e91e35700 20 client.31050.objecter _get_session s=0x7f5e8c125520 osd=6 3 2024-02-23T00:48:43.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.609+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_remove -1 14 2024-02-23T00:48:43.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.609+0000 7f5e91e35700 20 client.31050.objecter get_session s=0x7f5e8c125520 osd=6 3 2024-02-23T00:48:43.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.609+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_assign 6 14 2024-02-23T00:48:43.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.609+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c125520 osd=6 4 2024-02-23T00:48:43.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.609+0000 7f5e91e35700 10 client.31050.objecter _send_command 14 2024-02-23T00:48:43.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.609+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f5e8c1266c0 con 0x7f5e8c125ba0 2024-02-23T00:48:43.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.611+0000 7f5e88ff9700 1 -- 172.21.15.47:0/776364749 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f5e8c1266c0 con 0x7f5e8c125ba0 2024-02-23T00:48:43.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.611+0000 7f5e88ff9700 10 client.31050.objecter ms_dispatch 0x7f5e8c001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:48:43.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.611+0000 7f5e88ff9700 10 client.31050.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:48:43.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.611+0000 7f5e88ff9700 20 client.31050.objecter put_session s=0x7f5e8c125520 osd=6 4 2024-02-23T00:48:43.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.611+0000 7f5e88ff9700 15 client.31050.objecter _session_command_op_remove 6 14 2024-02-23T00:48:43.614 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:48:43.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.611+0000 7f5e91e35700 10 client.31050.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:43.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.611+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_assign -1 15 2024-02-23T00:48:43.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.611+0000 7f5e91e35700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f5e8c12c430 0x7f5e8c12e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:43.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.611+0000 7f5e91e35700 20 client.31050.objecter _get_session s=0x7f5e8c1266c0 osd=7 3 2024-02-23T00:48:43.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.611+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c1266c0 osd=7 3 2024-02-23T00:48:43.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.611+0000 7f5e91e35700 20 client.31050.objecter _get_session s=0x7f5e8c1266c0 osd=7 3 2024-02-23T00:48:43.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.611+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_remove -1 15 2024-02-23T00:48:43.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.611+0000 7f5e91e35700 20 client.31050.objecter get_session s=0x7f5e8c1266c0 osd=7 3 2024-02-23T00:48:43.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.611+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_assign 7 15 2024-02-23T00:48:43.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.611+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c1266c0 osd=7 4 2024-02-23T00:48:43.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.611+0000 7f5e91e35700 10 client.31050.objecter _send_command 15 2024-02-23T00:48:43.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.611+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5e8c1314c0 con 0x7f5e8c12c430 2024-02-23T00:48:43.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.611+0000 7f5e8b7fe700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f5e8c12c430 0x7f5e8c12e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:43.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.612+0000 7f5e88ff9700 10 client.31050.objecter ms_handle_connect 0x7f5e8c12c430 2024-02-23T00:48:43.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.612+0000 7f5e8b7fe700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f5e8c12c430 0x7f5e8c12e820 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:43.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.612+0000 7f5e88ff9700 1 -- 172.21.15.47:0/776364749 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5e8c1314c0 con 0x7f5e8c12c430 2024-02-23T00:48:43.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.612+0000 7f5e88ff9700 10 client.31050.objecter ms_dispatch 0x7f5e8c001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:48:43.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.612+0000 7f5e88ff9700 10 client.31050.objecter _finish_command 15 = system:0 2024-02-23T00:48:43.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.612+0000 7f5e88ff9700 20 client.31050.objecter put_session s=0x7f5e8c1266c0 osd=7 4 2024-02-23T00:48:43.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.612+0000 7f5e88ff9700 15 client.31050.objecter _session_command_op_remove 7 15 2024-02-23T00:48:43.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.627+0000 7f5e91e35700 10 client.31050.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:48:43.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.627+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_assign -1 16 2024-02-23T00:48:43.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.627+0000 7f5e91e35700 20 client.31050.objecter _get_session s=0x7f5e8c1266c0 osd=7 3 2024-02-23T00:48:43.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.627+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c1266c0 osd=7 3 2024-02-23T00:48:43.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.627+0000 7f5e91e35700 20 client.31050.objecter _get_session s=0x7f5e8c1266c0 osd=7 3 2024-02-23T00:48:43.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.627+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_remove -1 16 2024-02-23T00:48:43.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.627+0000 7f5e91e35700 20 client.31050.objecter get_session s=0x7f5e8c1266c0 osd=7 3 2024-02-23T00:48:43.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.627+0000 7f5e91e35700 15 client.31050.objecter _session_command_op_assign 7 16 2024-02-23T00:48:43.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.627+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c1266c0 osd=7 4 2024-02-23T00:48:43.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.627+0000 7f5e91e35700 10 client.31050.objecter _send_command 16 2024-02-23T00:48:43.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.627+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f5e8c1314c0 con 0x7f5e8c12c430 2024-02-23T00:48:43.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e88ff9700 1 -- 172.21.15.47:0/776364749 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f5e8c1314c0 con 0x7f5e8c12c430 2024-02-23T00:48:43.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e88ff9700 10 client.31050.objecter ms_dispatch 0x7f5e8c001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:48:43.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e88ff9700 10 client.31050.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:48:43.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e88ff9700 20 client.31050.objecter put_session s=0x7f5e8c1266c0 osd=7 4 2024-02-23T00:48:43.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e88ff9700 15 client.31050.objecter _session_command_op_remove 7 16 2024-02-23T00:48:43.632 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:48:43.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 10 librados: watch_flush enter 2024-02-23T00:48:43.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 10 librados: watch_flush exit 2024-02-23T00:48:43.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 10 client.31050.objecter close_session for osd.0 2024-02-23T00:48:43.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f5e8c0f9d30 msgr2=0x7f5e8c0fa110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:43.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f5e8c0f9d30 0x7f5e8c0fa110 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:43.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c05f970 osd=0 1 2024-02-23T00:48:43.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 10 client.31050.objecter close_session for osd.1 2024-02-23T00:48:43.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f5e8c0598f0 msgr2=0x7f5e8c059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:43.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f5e8c0598f0 0x7f5e8c059cd0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:43.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c028540 osd=1 1 2024-02-23T00:48:43.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 10 client.31050.objecter close_session for osd.2 2024-02-23T00:48:43.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f5e8c0280a0 msgr2=0x7f5e8c11e1e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:43.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f5e8c0280a0 0x7f5e8c11e1e0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:43.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c0595a0 osd=2 1 2024-02-23T00:48:43.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 10 client.31050.objecter close_session for osd.3 2024-02-23T00:48:43.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f5e8c11bd30 msgr2=0x7f5e8c11f850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:43.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f5e8c11bd30 0x7f5e8c11f850 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:43.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c0279b0 osd=3 1 2024-02-23T00:48:43.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 10 client.31050.objecter close_session for osd.4 2024-02-23T00:48:43.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f5e8c11b400 msgr2=0x7f5e8c124fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:43.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e8bfff700 1 -- 172.21.15.47:0/776364749 reap_dead start 2024-02-23T00:48:43.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f5e8c11b400 0x7f5e8c124fc0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:43.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e8bfff700 1 -- 172.21.15.47:0/776364749 reap_dead start 2024-02-23T00:48:43.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c11b290 osd=4 1 2024-02-23T00:48:43.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 10 client.31050.objecter close_session for osd.5 2024-02-23T00:48:43.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f5e8c127100 msgr2=0x7f5e8c127580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:43.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f5e8c127100 0x7f5e8c127580 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:43.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c127d60 osd=5 1 2024-02-23T00:48:43.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 10 client.31050.objecter close_session for osd.6 2024-02-23T00:48:43.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f5e8c125ba0 msgr2=0x7f5e8c125fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:43.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f5e8c125ba0 0x7f5e8c125fe0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:43.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c125520 osd=6 1 2024-02-23T00:48:43.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 10 client.31050.objecter close_session for osd.7 2024-02-23T00:48:43.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f5e8c12c430 msgr2=0x7f5e8c12e820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:43.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f5e8c12c430 0x7f5e8c12e820 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:43.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 20 client.31050.objecter put_session s=0x7f5e8c1266c0 osd=7 1 2024-02-23T00:48:43.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 20 client.31050.objecter shutdown clearing up homeless session... 2024-02-23T00:48:43.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 10 client.31050.objecter successfully canceled tick 2024-02-23T00:48:43.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5e74062cc0 msgr2=0x7f5e74065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:43.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5e74062cc0 0x7f5e74065180 secure :-1 s=READY pgs=3071 cs=0 l=1 rev1=1 rx=0x7f5e8c10bb40 tx=0x7f5e80004d10).stop 2024-02-23T00:48:43.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e8c0f6a90 msgr2=0x7f5e8c110200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:43.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 1 --2- 172.21.15.47:0/776364749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e8c0f6a90 0x7f5e8c110200 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 rx=0x7f5e7c004650 tx=0x7f5e7c009cb0).stop 2024-02-23T00:48:43.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e8bfff700 1 -- 172.21.15.47:0/776364749 reap_dead start 2024-02-23T00:48:43.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e8bfff700 1 -- 172.21.15.47:0/776364749 reap_dead start 2024-02-23T00:48:43.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 shutdown_connections 2024-02-23T00:48:43.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.629+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 >> 172.21.15.47:0/776364749 conn(0x7f5e8c06c370 msgr2=0x7f5e8c102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:48:43.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.630+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 shutdown_connections 2024-02-23T00:48:43.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.630+0000 7f5e91e35700 1 -- 172.21.15.47:0/776364749 wait complete. 2024-02-23T00:48:43.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:43.630+0000 7f5e91e35700 1 librados: shutdown 2024-02-23T00:48:43.637 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:48:43.637 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:48:43.637 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:48:43.637 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:48:43.637 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:48:43.637 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:48:43.637 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:48:43.637 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:48:43.667 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:48:43.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:42.407234+0000 osd.6 (osd.6) 171 : cluster [DBG] 4.a scrub starts 2024-02-23T00:48:43.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:42.409675+0000 osd.6 (osd.6) 172 : cluster [DBG] 4.a scrub ok 2024-02-23T00:48:43.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:42.453155+0000 osd.5 (osd.5) 180 : cluster [DBG] 62.1d scrub starts 2024-02-23T00:48:43.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:42.456611+0000 osd.5 (osd.5) 181 : cluster [DBG] 62.1d scrub ok 2024-02-23T00:48:43.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:42.607075+0000 mgr.y (mgr.14517) 402 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:48:43.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:42.687793+0000 osd.7 (osd.7) 207 : cluster [DBG] 62.1e scrub starts 2024-02-23T00:48:43.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:42.689034+0000 osd.7 (osd.7) 208 : cluster [DBG] 62.1e scrub ok 2024-02-23T00:48:43.695 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:43.715 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:48:43.718 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:43.718 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:48:43.818 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:48:43.853 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:44.389 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-23T00:48:44.532 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:48:44.534 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:43.295539+0000 mgr.y (mgr.14517) 403 : cluster [DBG] pgmap v634: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-23T00:48:44.534 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:43.485902+0000 mon.a (mon.0) 2598 : audit [DBG] from='client.? 172.21.15.47:0/776364749' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:48:44.534 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:43.724831+0000 osd.7 (osd.7) 209 : cluster [DBG] 62.0 scrub starts 2024-02-23T00:48:44.534 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:43.728453+0000 osd.7 (osd.7) 210 : cluster [DBG] 62.0 scrub ok 2024-02-23T00:48:44.535 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:43.295539+0000 mgr.y (mgr.14517) 403 : cluster [DBG] pgmap v634: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-23T00:48:44.535 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:43.485902+0000 mon.a (mon.0) 2598 : audit [DBG] from='client.? 172.21.15.47:0/776364749' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:48:44.535 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:43.724831+0000 osd.7 (osd.7) 209 : cluster [DBG] 62.0 scrub starts 2024-02-23T00:48:44.535 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:43.728453+0000 osd.7 (osd.7) 210 : cluster [DBG] 62.0 scrub ok 2024-02-23T00:48:44.535 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:44.536 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:48:44.636 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:48:44.666 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:43.295539+0000 mgr.y (mgr.14517) 403 : cluster [DBG] pgmap v634: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-23T00:48:44.666 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:43.485902+0000 mon.a (mon.0) 2598 : audit [DBG] from='client.? 172.21.15.47:0/776364749' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:48:44.667 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:43.724831+0000 osd.7 (osd.7) 209 : cluster [DBG] 62.0 scrub starts 2024-02-23T00:48:44.667 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:43.728453+0000 osd.7 (osd.7) 210 : cluster [DBG] 62.0 scrub ok 2024-02-23T00:48:44.669 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:44.690 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:48:44.693 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:44.693 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:48:44.794 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:48:44.822 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:44.921 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:48:44.923 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:44.923 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:48:45.024 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:48:45.055 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:45.076 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:48:45.079 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:45.079 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:48:45.180 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:48:45.209 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:45.230 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:48:45.232 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:45.233 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:48:45.333 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:48:45.361 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:45.379 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:44.240712+0000 osd.4 (osd.4) 169 : cluster [DBG] 4.13 scrub starts 2024-02-23T00:48:45.379 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:44.242549+0000 osd.4 (osd.4) 170 : cluster [DBG] 4.13 scrub ok 2024-02-23T00:48:45.379 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:44.372309+0000 mon.a (mon.0) 2599 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:48:45.380 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:44.372666+0000 mon.a (mon.0) 2600 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-61"}]: dispatch 2024-02-23T00:48:45.380 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:44.695558+0000 osd.7 (osd.7) 211 : cluster [DBG] 2.b scrub starts 2024-02-23T00:48:45.380 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:44.698441+0000 osd.7 (osd.7) 212 : cluster [DBG] 2.b scrub ok 2024-02-23T00:48:45.380 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:44.727530+0000 osd.1 (osd.1) 189 : cluster [DBG] 2.6 scrub starts 2024-02-23T00:48:45.380 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:44.728515+0000 osd.1 (osd.1) 190 : cluster [DBG] 2.6 scrub ok 2024-02-23T00:48:45.380 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:45.294067+0000 osd.6 (osd.6) 173 : cluster [DBG] 4.1f scrub starts 2024-02-23T00:48:45.381 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:45.295829+0000 osd.6 (osd.6) 174 : cluster [DBG] 4.1f scrub ok 2024-02-23T00:48:45.383 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:48:45.385 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:45.386 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:48:45.487 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:48:45.516 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:45.614 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:48:45.616 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:45.617 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:48:45.632 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:44.240712+0000 osd.4 (osd.4) 169 : cluster [DBG] 4.13 scrub starts 2024-02-23T00:48:45.632 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:44.242549+0000 osd.4 (osd.4) 170 : cluster [DBG] 4.13 scrub ok 2024-02-23T00:48:45.632 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:44.372309+0000 mon.a (mon.0) 2599 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:48:45.632 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:44.372666+0000 mon.a (mon.0) 2600 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-61"}]: dispatch 2024-02-23T00:48:45.632 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:44.695558+0000 osd.7 (osd.7) 211 : cluster [DBG] 2.b scrub starts 2024-02-23T00:48:45.632 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:44.698441+0000 osd.7 (osd.7) 212 : cluster [DBG] 2.b scrub ok 2024-02-23T00:48:45.632 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:44.727530+0000 osd.1 (osd.1) 189 : cluster [DBG] 2.6 scrub starts 2024-02-23T00:48:45.633 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:44.728515+0000 osd.1 (osd.1) 190 : cluster [DBG] 2.6 scrub ok 2024-02-23T00:48:45.633 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:45.294067+0000 osd.6 (osd.6) 173 : cluster [DBG] 4.1f scrub starts 2024-02-23T00:48:45.633 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:45.295829+0000 osd.6 (osd.6) 174 : cluster [DBG] 4.1f scrub ok 2024-02-23T00:48:45.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:44.240712+0000 osd.4 (osd.4) 169 : cluster [DBG] 4.13 scrub starts 2024-02-23T00:48:45.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:44.242549+0000 osd.4 (osd.4) 170 : cluster [DBG] 4.13 scrub ok 2024-02-23T00:48:45.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:44.372309+0000 mon.a (mon.0) 2599 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:48:45.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:44.372666+0000 mon.a (mon.0) 2600 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-61"}]: dispatch 2024-02-23T00:48:45.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:44.695558+0000 osd.7 (osd.7) 211 : cluster [DBG] 2.b scrub starts 2024-02-23T00:48:45.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:44.698441+0000 osd.7 (osd.7) 212 : cluster [DBG] 2.b scrub ok 2024-02-23T00:48:45.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:44.727530+0000 osd.1 (osd.1) 189 : cluster [DBG] 2.6 scrub starts 2024-02-23T00:48:45.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:44.728515+0000 osd.1 (osd.1) 190 : cluster [DBG] 2.6 scrub ok 2024-02-23T00:48:45.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:45.294067+0000 osd.6 (osd.6) 173 : cluster [DBG] 4.1f scrub starts 2024-02-23T00:48:45.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:45.295829+0000 osd.6 (osd.6) 174 : cluster [DBG] 4.1f scrub ok 2024-02-23T00:48:45.719 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:48:45.746 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:45.996 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:48:45.998 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:45.999 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:48:46.099 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:48:46.131 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:46.153 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:48:46.155 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:46.156 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:48:46.257 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:48:46.285 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:46.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:45.282197+0000 osd.4 (osd.4) 171 : cluster [DBG] 18.d scrub starts 2024-02-23T00:48:46.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:45.283224+0000 osd.4 (osd.4) 172 : cluster [DBG] 18.d scrub ok 2024-02-23T00:48:46.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:45.296100+0000 mgr.y (mgr.14517) 404 : cluster [DBG] pgmap v635: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:48:46.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:45.377621+0000 mon.a (mon.0) 2601 : cluster [DBG] osdmap e434: 8 total, 8 up, 7 in 2024-02-23T00:48:46.534 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:48:46.535 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:45.282197+0000 osd.4 (osd.4) 171 : cluster [DBG] 18.d scrub starts 2024-02-23T00:48:46.536 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:45.283224+0000 osd.4 (osd.4) 172 : cluster [DBG] 18.d scrub ok 2024-02-23T00:48:46.536 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:45.296100+0000 mgr.y (mgr.14517) 404 : cluster [DBG] pgmap v635: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:48:46.536 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:45.377621+0000 mon.a (mon.0) 2601 : cluster [DBG] osdmap e434: 8 total, 8 up, 7 in 2024-02-23T00:48:46.536 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:45.282197+0000 osd.4 (osd.4) 171 : cluster [DBG] 18.d scrub starts 2024-02-23T00:48:46.537 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:45.283224+0000 osd.4 (osd.4) 172 : cluster [DBG] 18.d scrub ok 2024-02-23T00:48:46.537 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:45.296100+0000 mgr.y (mgr.14517) 404 : cluster [DBG] pgmap v635: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:48:46.537 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:45.377621+0000 mon.a (mon.0) 2601 : cluster [DBG] osdmap e434: 8 total, 8 up, 7 in 2024-02-23T00:48:46.537 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:46.538 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:48:46.639 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:48:46.666 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:47.018 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:48:47.023 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:47.023 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:48:47.027 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:48:47.124 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:48:47.152 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:47.310 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:48:47.313 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:47.313 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:48:47.414 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:48:47.443 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:47.592 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:48:47.596 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:46.381821+0000 mon.a (mon.0) 2602 : cluster [DBG] osdmap e435: 8 total, 8 up, 7 in 2024-02-23T00:48:47.596 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:46.386676+0000 mon.a (mon.0) 2603 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:48:47.596 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:46.381821+0000 mon.a (mon.0) 2602 : cluster [DBG] osdmap e435: 8 total, 8 up, 7 in 2024-02-23T00:48:47.596 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:46.386676+0000 mon.a (mon.0) 2603 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:48:47.597 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:47.597 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:48:47.611 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:48:47.612 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:48:47.635 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:48:47.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:46.381821+0000 mon.a (mon.0) 2602 : cluster [DBG] osdmap e435: 8 total, 8 up, 7 in 2024-02-23T00:48:47.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:46.386676+0000 mon.a (mon.0) 2603 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:48:47.698 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:48:47.730 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:47.751 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:48:47.754 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:47.754 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:48:47.855 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:48:47.884 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:48.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:47.296894+0000 mgr.y (mgr.14517) 405 : cluster [DBG] pgmap v638: 208 pgs: 32 creating+peering, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:48:48.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:47.386890+0000 mon.a (mon.0) 2604 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:48:48.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:47.387034+0000 mon.a (mon.0) 2605 : cluster [DBG] osdmap e436: 8 total, 8 up, 7 in 2024-02-23T00:48:48.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:47.389672+0000 mon.a (mon.0) 2606 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:48:48.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:47.393353+0000 mon.a (mon.0) 2607 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-23T00:48:48.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:47.419691+0000 osd.5 (osd.5) 182 : cluster [DBG] 18.b scrub starts 2024-02-23T00:48:48.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:47.420766+0000 osd.5 (osd.5) 183 : cluster [DBG] 18.b scrub ok 2024-02-23T00:48:48.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:47.516513+0000 osd.3 (osd.3) 200 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:48:48.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:47.516631+0000 osd.3 (osd.3) 201 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:48:48.754 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:47.296894+0000 mgr.y (mgr.14517) 405 : cluster [DBG] pgmap v638: 208 pgs: 32 creating+peering, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:48:48.754 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:47.386890+0000 mon.a (mon.0) 2604 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:48:48.754 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:47.387034+0000 mon.a (mon.0) 2605 : cluster [DBG] osdmap e436: 8 total, 8 up, 7 in 2024-02-23T00:48:48.754 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:47.389672+0000 mon.a (mon.0) 2606 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:48:48.754 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:47.393353+0000 mon.a (mon.0) 2607 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-23T00:48:48.754 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:47.419691+0000 osd.5 (osd.5) 182 : cluster [DBG] 18.b scrub starts 2024-02-23T00:48:48.754 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:47.420766+0000 osd.5 (osd.5) 183 : cluster [DBG] 18.b scrub ok 2024-02-23T00:48:48.754 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:47.516513+0000 osd.3 (osd.3) 200 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:48:48.754 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:47.516631+0000 osd.3 (osd.3) 201 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:48:48.755 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:47.296894+0000 mgr.y (mgr.14517) 405 : cluster [DBG] pgmap v638: 208 pgs: 32 creating+peering, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:48:48.755 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:47.386890+0000 mon.a (mon.0) 2604 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:48:48.755 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:47.387034+0000 mon.a (mon.0) 2605 : cluster [DBG] osdmap e436: 8 total, 8 up, 7 in 2024-02-23T00:48:48.755 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:47.389672+0000 mon.a (mon.0) 2606 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:48:48.755 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:47.393353+0000 mon.a (mon.0) 2607 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-23T00:48:48.755 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:47.419691+0000 osd.5 (osd.5) 182 : cluster [DBG] 18.b scrub starts 2024-02-23T00:48:48.755 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:47.420766+0000 osd.5 (osd.5) 183 : cluster [DBG] 18.b scrub ok 2024-02-23T00:48:48.755 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:47.516513+0000 osd.3 (osd.3) 200 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:48:48.755 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:47.516631+0000 osd.3 (osd.3) 201 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:48:48.776 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.3 dump_blocked_ops 2024-02-23T00:48:48.934 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:48:48.937 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:48.938 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:48:49.038 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:48:49.071 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:49.092 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:48:49.095 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:49.095 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:48:49.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.158+0000 7fca9afdb700 1 -- 172.21.15.47:0/2161688526 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fca940fe510 msgr2=0x7fca940f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:49.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.158+0000 7fca9afdb700 1 --2- 172.21.15.47:0/2161688526 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fca940fe510 0x7fca940f7d40 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 rx=0x7fca7c004870 tx=0x7fca7c009cb0).stop 2024-02-23T00:48:49.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.158+0000 7fca9afdb700 1 -- 172.21.15.47:0/2161688526 shutdown_connections 2024-02-23T00:48:49.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.158+0000 7fca9afdb700 1 --2- 172.21.15.47:0/2161688526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca940f8420 0x7fca94104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:49.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.158+0000 7fca9afdb700 1 --2- 172.21.15.47:0/2161688526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca940fdbb0 0x7fca940fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:49.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.158+0000 7fca9afdb700 1 --2- 172.21.15.47:0/2161688526 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fca940fe510 0x7fca940f7d40 unknown :-1 s=CLOSED pgs=2675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:49.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.158+0000 7fca9afdb700 1 -- 172.21.15.47:0/2161688526 >> 172.21.15.47:0/2161688526 conn(0x7fca940f3800 msgr2=0x7fca940f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:48:49.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.158+0000 7fca9afdb700 1 -- 172.21.15.47:0/2161688526 shutdown_connections 2024-02-23T00:48:49.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.158+0000 7fca9afdb700 1 -- 172.21.15.47:0/2161688526 wait complete. 2024-02-23T00:48:49.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.159+0000 7fca9afdb700 1 librados: starting msgr at 2024-02-23T00:48:49.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.159+0000 7fca9afdb700 1 librados: starting objecter 2024-02-23T00:48:49.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.159+0000 7fca9afdb700 1 Processor -- start 2024-02-23T00:48:49.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.159+0000 7fca9afdb700 1 -- start start 2024-02-23T00:48:49.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.159+0000 7fca9afdb700 1 librados: setting wanted keys 2024-02-23T00:48:49.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.159+0000 7fca9afdb700 1 librados: calling monclient init 2024-02-23T00:48:49.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.159+0000 7fca9afdb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca940f8420 0x7fca9410de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:49.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.159+0000 7fca9afdb700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fca940fdbb0 0x7fca9410e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:49.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.159+0000 7fca9afdb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca940fe510 0x7fca94107ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:49.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.159+0000 7fca9afdb700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca941084e0 con 0x7fca940f8420 2024-02-23T00:48:49.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.159+0000 7fca9afdb700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca94108650 con 0x7fca940fe510 2024-02-23T00:48:49.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.159+0000 7fca9afdb700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca94108810 con 0x7fca940fdbb0 2024-02-23T00:48:49.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.159+0000 7fca93fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fca940fdbb0 0x7fca9410e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:49.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.159+0000 7fca98d77700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca940f8420 0x7fca9410de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:49.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.159+0000 7fca93fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fca940fdbb0 0x7fca9410e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:43628/0 (socket says 172.21.15.47:43628) 2024-02-23T00:48:49.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.159+0000 7fca93fff700 1 -- 172.21.15.47:0/1850868744 learned_addr learned my addr 172.21.15.47:0/1850868744 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:48:49.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.160+0000 7fca99578700 1 --2- 172.21.15.47:0/1850868744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca940fe510 0x7fca94107ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:49.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.160+0000 7fca93fff700 1 -- 172.21.15.47:0/1850868744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca940fe510 msgr2=0x7fca94107ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:49.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.160+0000 7fca93fff700 1 --2- 172.21.15.47:0/1850868744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca940fe510 0x7fca94107ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:49.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.160+0000 7fca93fff700 1 -- 172.21.15.47:0/1850868744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca940f8420 msgr2=0x7fca9410de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:49.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.160+0000 7fca93fff700 1 --2- 172.21.15.47:0/1850868744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca940f8420 0x7fca9410de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:49.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.160+0000 7fca93fff700 1 -- 172.21.15.47:0/1850868744 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8c0097c0 con 0x7fca940fdbb0 2024-02-23T00:48:49.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.160+0000 7fca93fff700 1 --2- 172.21.15.47:0/1850868744 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fca940fdbb0 0x7fca9410e340 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 rx=0x7fca7c009da0 tx=0x7fca7c009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:49.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.160+0000 7fca91ffb700 10 client.?.objecter ms_handle_connect 0x7fca940fdbb0 2024-02-23T00:48:49.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.160+0000 7fca91ffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:48:49.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.160+0000 7fca91ffb700 1 -- 172.21.15.47:0/1850868744 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c00e030 con 0x7fca940fdbb0 2024-02-23T00:48:49.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.160+0000 7fca91ffb700 1 -- 172.21.15.47:0/1850868744 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fca7c013e70 con 0x7fca940fdbb0 2024-02-23T00:48:49.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.160+0000 7fca9afdb700 1 -- 172.21.15.47:0/1850868744 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca7c005040 con 0x7fca940fdbb0 2024-02-23T00:48:49.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.160+0000 7fca91ffb700 1 -- 172.21.15.47:0/1850868744 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c018070 con 0x7fca940fdbb0 2024-02-23T00:48:49.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.160+0000 7fca9afdb700 10 client.41381.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:48:49.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.160+0000 7fca9afdb700 1 -- 172.21.15.47:0/1850868744 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca94108d30 con 0x7fca940fdbb0 2024-02-23T00:48:49.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.160+0000 7fca9afdb700 1 librados: init done 2024-02-23T00:48:49.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.161+0000 7fca9afdb700 1 -- 172.21.15.47:0/1850868744 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca9405ee80 con 0x7fca940fdbb0 2024-02-23T00:48:49.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.162+0000 7fca91ffb700 1 -- 172.21.15.47:0/1850868744 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fca7c009240 con 0x7fca940fdbb0 2024-02-23T00:48:49.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.162+0000 7fca91ffb700 1 --2- 172.21.15.47:0/1850868744 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fca80062cc0 0x7fca80065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:49.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.162+0000 7fca91ffb700 1 -- 172.21.15.47:0/1850868744 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(437..437 src has 48..437) v4 ==== 9220+0+0 (secure 0 0 0) 0x7fca7c07e960 con 0x7fca940fdbb0 2024-02-23T00:48:49.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.162+0000 7fca91ffb700 10 client.41381.objecter ms_dispatch 0x7fca94001550 osd_map(437..437 src has 48..437) v4 2024-02-23T00:48:49.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.162+0000 7fca91ffb700 3 client.41381.objecter handle_osd_map got epochs [437,437] > 0 2024-02-23T00:48:49.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.162+0000 7fca91ffb700 3 client.41381.objecter handle_osd_map decoding full epoch 437 2024-02-23T00:48:49.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.162+0000 7fca98d77700 1 --2- 172.21.15.47:0/1850868744 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fca80062cc0 0x7fca80065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:49.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.162+0000 7fca91ffb700 20 client.41381.objecter dump_active .. 0 homeless 2024-02-23T00:48:49.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.163+0000 7fca91ffb700 10 client.41381.objecter ms_handle_connect 0x7fca80062cc0 2024-02-23T00:48:49.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.163+0000 7fca98d77700 1 --2- 172.21.15.47:0/1850868744 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fca80062cc0 0x7fca80065180 secure :-1 s=READY pgs=3072 cs=0 l=1 rev1=1 rx=0x7fca840041d0 tx=0x7fca84009040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:49.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.167+0000 7fca91ffb700 1 -- 172.21.15.47:0/1850868744 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fca7c04df70 con 0x7fca940fdbb0 2024-02-23T00:48:49.196 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:48:49.228 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:49.301 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:48:49.303 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:49.304 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:48:49.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.341+0000 7fca9afdb700 1 -- 172.21.15.47:0/1850868744 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fca941095d0 con 0x7fca80062cc0 2024-02-23T00:48:49.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.357+0000 7fca91ffb700 1 -- 172.21.15.47:0/1850868744 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+529515 (secure 0 0 0) 0x7fca840047f0 con 0x7fca80062cc0 2024-02-23T00:48:49.366 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:48:49.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.361+0000 7fca9afdb700 10 librados: watch_flush enter 2024-02-23T00:48:49.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.361+0000 7fca9afdb700 10 librados: watch_flush exit 2024-02-23T00:48:49.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.361+0000 7fca9afdb700 20 client.41381.objecter shutdown clearing up homeless session... 2024-02-23T00:48:49.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.361+0000 7fca9afdb700 10 client.41381.objecter successfully canceled tick 2024-02-23T00:48:49.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.361+0000 7fca9afdb700 1 -- 172.21.15.47:0/1850868744 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fca80062cc0 msgr2=0x7fca80065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:49.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.361+0000 7fca9afdb700 1 --2- 172.21.15.47:0/1850868744 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fca80062cc0 0x7fca80065180 secure :-1 s=READY pgs=3072 cs=0 l=1 rev1=1 rx=0x7fca840041d0 tx=0x7fca84009040).stop 2024-02-23T00:48:49.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.361+0000 7fca9afdb700 1 -- 172.21.15.47:0/1850868744 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fca940fdbb0 msgr2=0x7fca9410e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:49.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.361+0000 7fca9afdb700 1 --2- 172.21.15.47:0/1850868744 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fca940fdbb0 0x7fca9410e340 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 rx=0x7fca7c009da0 tx=0x7fca7c009cb0).stop 2024-02-23T00:48:49.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.361+0000 7fca9afdb700 1 -- 172.21.15.47:0/1850868744 shutdown_connections 2024-02-23T00:48:49.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.361+0000 7fca9afdb700 1 --2- 172.21.15.47:0/1850868744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca940fe510 0x7fca94107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:49.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.361+0000 7fca9afdb700 1 --2- 172.21.15.47:0/1850868744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca940f8420 0x7fca9410de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:49.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.361+0000 7fca9afdb700 1 --2- 172.21.15.47:0/1850868744 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fca80062cc0 0x7fca80065180 unknown :-1 s=CLOSED pgs=3072 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:49.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.361+0000 7fca9afdb700 1 --2- 172.21.15.47:0/1850868744 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fca940fdbb0 0x7fca9410e340 unknown :-1 s=CLOSED pgs=2676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:49.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.361+0000 7fca9afdb700 1 -- 172.21.15.47:0/1850868744 >> 172.21.15.47:0/1850868744 conn(0x7fca940f3800 msgr2=0x7fca940fb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:48:49.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.361+0000 7fca9afdb700 1 -- 172.21.15.47:0/1850868744 shutdown_connections 2024-02-23T00:48:49.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.361+0000 7fca9afdb700 1 -- 172.21.15.47:0/1850868744 wait complete. 2024-02-23T00:48:49.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:49.361+0000 7fca9afdb700 1 librados: shutdown 2024-02-23T00:48:49.373 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-02-23T00:48:49.405 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:48:49.437 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:49.457 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:48:49.459 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:49.460 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:48:49.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:48.253440+0000 osd.4 (osd.4) 173 : cluster [DBG] 4.c scrub starts 2024-02-23T00:48:49.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:48.254943+0000 osd.4 (osd.4) 174 : cluster [DBG] 4.c scrub ok 2024-02-23T00:48:49.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:48.389006+0000 mon.a (mon.0) 2608 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-23T00:48:49.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:48.389098+0000 mon.a (mon.0) 2609 : cluster [DBG] osdmap e437: 8 total, 8 up, 7 in 2024-02-23T00:48:49.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:49.265442+0000 osd.6 (osd.6) 175 : cluster [DBG] 5.2 scrub starts 2024-02-23T00:48:49.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:49.266290+0000 osd.6 (osd.6) 176 : cluster [DBG] 5.2 scrub ok 2024-02-23T00:48:49.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:48.253440+0000 osd.4 (osd.4) 173 : cluster [DBG] 4.c scrub starts 2024-02-23T00:48:49.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:48.254943+0000 osd.4 (osd.4) 174 : cluster [DBG] 4.c scrub ok 2024-02-23T00:48:49.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:48.389006+0000 mon.a (mon.0) 2608 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-23T00:48:49.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:48.389098+0000 mon.a (mon.0) 2609 : cluster [DBG] osdmap e437: 8 total, 8 up, 7 in 2024-02-23T00:48:49.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:49.265442+0000 osd.6 (osd.6) 175 : cluster [DBG] 5.2 scrub starts 2024-02-23T00:48:49.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:49.266290+0000 osd.6 (osd.6) 176 : cluster [DBG] 5.2 scrub ok 2024-02-23T00:48:49.561 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:48:49.590 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:49.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:48.253440+0000 osd.4 (osd.4) 173 : cluster [DBG] 4.c scrub starts 2024-02-23T00:48:49.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:48.254943+0000 osd.4 (osd.4) 174 : cluster [DBG] 4.c scrub ok 2024-02-23T00:48:49.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:48.389006+0000 mon.a (mon.0) 2608 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-23T00:48:49.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:48.389098+0000 mon.a (mon.0) 2609 : cluster [DBG] osdmap e437: 8 total, 8 up, 7 in 2024-02-23T00:48:49.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:49.265442+0000 osd.6 (osd.6) 175 : cluster [DBG] 5.2 scrub starts 2024-02-23T00:48:49.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:49.266290+0000 osd.6 (osd.6) 176 : cluster [DBG] 5.2 scrub ok 2024-02-23T00:48:49.947 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:48:49.950 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:49.951 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:48:49.956 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:48:49.956 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:48:50.051 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:48:50.083 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:50.104 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:48:50.106 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:50.107 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:48:50.208 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:48:50.241 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:50.269 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:48:50.272 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:50.272 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:48:50.373 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:48:50.402 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:50.422 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:48:50.424 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:50.425 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:48:50.526 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:48:50.552 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:48:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:48:50] "GET /metrics HTTP/1.1" 200 188972 "" "Prometheus/2.33.4" 2024-02-23T00:48:50.552 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:49.245622+0000 osd.4 (osd.4) 175 : cluster [DBG] 62.d scrub starts 2024-02-23T00:48:50.553 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:49.247396+0000 osd.4 (osd.4) 176 : cluster [DBG] 62.d scrub ok 2024-02-23T00:48:50.553 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:49.297311+0000 mgr.y (mgr.14517) 406 : cluster [DBG] pgmap v641: 208 pgs: 32 creating+peering, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:48:50.553 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:49.343232+0000 mgr.y (mgr.14517) 407 : audit [DBG] from='client.41381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:48:50.553 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:49.398755+0000 mon.a (mon.0) 2610 : cluster [DBG] osdmap e438: 8 total, 8 up, 7 in 2024-02-23T00:48:50.553 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:49.607186+0000 osd.1 (osd.1) 191 : cluster [DBG] 18.1b scrub starts 2024-02-23T00:48:50.553 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:49.610386+0000 osd.1 (osd.1) 192 : cluster [DBG] 18.1b scrub ok 2024-02-23T00:48:50.553 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:50.194182+0000 mon.a (mon.0) 2611 : cluster [DBG] osdmap e439: 8 total, 8 up, 7 in 2024-02-23T00:48:50.553 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:49.245622+0000 osd.4 (osd.4) 175 : cluster [DBG] 62.d scrub starts 2024-02-23T00:48:50.553 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:49.247396+0000 osd.4 (osd.4) 176 : cluster [DBG] 62.d scrub ok 2024-02-23T00:48:50.553 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:49.297311+0000 mgr.y (mgr.14517) 406 : cluster [DBG] pgmap v641: 208 pgs: 32 creating+peering, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:48:50.554 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:49.343232+0000 mgr.y (mgr.14517) 407 : audit [DBG] from='client.41381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:48:50.554 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:49.398755+0000 mon.a (mon.0) 2610 : cluster [DBG] osdmap e438: 8 total, 8 up, 7 in 2024-02-23T00:48:50.554 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:49.607186+0000 osd.1 (osd.1) 191 : cluster [DBG] 18.1b scrub starts 2024-02-23T00:48:50.554 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:49.610386+0000 osd.1 (osd.1) 192 : cluster [DBG] 18.1b scrub ok 2024-02-23T00:48:50.554 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:50.194182+0000 mon.a (mon.0) 2611 : cluster [DBG] osdmap e439: 8 total, 8 up, 7 in 2024-02-23T00:48:50.554 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:50.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:49.245622+0000 osd.4 (osd.4) 175 : cluster [DBG] 62.d scrub starts 2024-02-23T00:48:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:49.247396+0000 osd.4 (osd.4) 176 : cluster [DBG] 62.d scrub ok 2024-02-23T00:48:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:49.297311+0000 mgr.y (mgr.14517) 406 : cluster [DBG] pgmap v641: 208 pgs: 32 creating+peering, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:48:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:49.343232+0000 mgr.y (mgr.14517) 407 : audit [DBG] from='client.41381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:48:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:49.398755+0000 mon.a (mon.0) 2610 : cluster [DBG] osdmap e438: 8 total, 8 up, 7 in 2024-02-23T00:48:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:49.607186+0000 osd.1 (osd.1) 191 : cluster [DBG] 18.1b scrub starts 2024-02-23T00:48:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:49.610386+0000 osd.1 (osd.1) 192 : cluster [DBG] 18.1b scrub ok 2024-02-23T00:48:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:50.194182+0000 mon.a (mon.0) 2611 : cluster [DBG] osdmap e439: 8 total, 8 up, 7 in 2024-02-23T00:48:51.355 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":641,"stamp":"2024-02-23T00:48:49.297060+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4007280,"num_objects":383,"num_object_clones":2,"num_object_copies":1149,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":383,"num_whiteouts":0,"num_read":1776,"num_read_kb":1398,"num_write":1223,"num_write_kb":4543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":440,"num_bytes_recovered":3479,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":7,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2640,"ondisk_log_size":2640,"up":624,"acting":624,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":624,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":2168116,"kb_used_data":62936,"kb_used_omap":0,"kb_used_meta":2105088,"kb_avail":654047948,"statfs":{"total":671965249536,"available":669745098752,"internally_reserved":0,"allocated":64446464,"data_stored":43396228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2155610112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12,"num_objects":-3,"num_object_clones":0,"num_object_copies":-9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3,"num_whiteouts":0,"num_read":-9,"num_read_kb":-6,"num_write":-12,"num_write_kb":-4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001297"},"pg_stats":[{"pgid":"213.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:48:46.374300+0000","last_change":"2024-02-23T00:48:46.385932+0000","last_active":"2024-02-23T00:48:46.374300+0000","last_peered":"2024-02-23T00:48:46.374300+0000","last_clean":"2024-02-23T00:48:46.374300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:48:46.374300+0000","last_undegraded":"2024-02-23T00:48:46.374300+0000","last_fullsized":"2024-02-23T00:48:46.374300+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":435,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_clean_scrub_stamp":"2024-02-23T00:48:46.374300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"213.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:48:46.374300+0000","last_change":"2024-02-23T00:48:46.385577+0000","last_active":"2024-02-23T00:48:46.374300+0000","last_peered":"2024-02-23T00:48:46.374300+0000","last_clean":"2024-02-23T00:48:46.374300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:48:46.374300+0000","last_undegraded":"2024-02-23T00:48:46.374300+0000","last_fullsized":"2024-02-23T00:48:46.374300+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":435,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_clean_scrub_stamp":"2024-02-23T00:48:46.374300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"213.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:48:46.374300+0000","last_change":"2024-02-23T00:48:46.386736+0000","last_active":"2024-02-23T00:48:46.374300+0000","last_peered":"2024-02-23T00:48:46.374300+0000","last_clean":"2024-02-23T00:48:46.374300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:48:46.374300+0000","last_undegraded":"2024-02-23T00:48:46.374300+0000","last_fullsized":"2024-02-23T00:48:46.374300+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":435,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_clean_scrub_stamp":"2024-02-23T00:48:46.374300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"213.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:48:46.374300+0000","last_change":"2024-02-23T00:48:46.385542+0000","last_active":"2024-02-23T00:48:46.374300+0000","last_peered":"2024-02-23T00:48:46.374300+0000","last_clean":"2024-02-23T00:48:46.374300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:48:46.374300+0000","last_undegraded":"2024-02-23T00:48:46.374300+0000","last_fullsized":"2024-02-23T00:48:46.374300+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":435,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_clean_scrub_stamp":"2024-02-23T00:48:46.374300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"213.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:48:46.374300+0000","last_change":"2024-02-23T00:48:46.383814+0000","last_active":"2024-02-23T00:48:46.374300+0000","last_peered":"2024-02-23T00:48:46.374300+0000","last_clean":"2024-02-23T00:48:46.374300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:48:46.374300+0000","last_undegraded":"2024-02-23T00:48:46.374300+0000","last_fullsized":"2024-02-23T00:48:46.374300+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":435,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_clean_scrub_stamp":"2024-02-23T00:48:46.374300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"213.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:48:46.374300+0000","last_change":"2024-02-23T00:48:46.387266+0000","last_active":"2024-02-23T00:48:46.374300+0000","last_peered":"2024-02-23T00:48:46.374300+0000","last_clean":"2024-02-23T00:48:46.374300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:48:46.374300+0000","last_undegraded":"2024-02-23T00:48:46.374300+0000","last_fullsized":"2024-02-23T00:48:46.374300+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":435,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_clean_scrub_stamp":"2024-02-23T00:48:46.374300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"213.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:48:46.374300+0000","last_change":"2024-02-23T00:48:46.386637+0000","last_active":"2024-02-23T00:48:46.374300+0000","last_peered":"2024-02-23T00:48:46.374300+0000","last_clean":"2024-02-23T00:48:46.374300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:48:46.374300+0000","last_undegraded":"2024-02-23T00:48:46.374300+0000","last_fullsized":"2024-02-23T00:48:46.374300+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":435,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_clean_scrub_stamp":"2024-02-23T00:48:46.374300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"213.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:48:46.374300+0000","last_change":"2024-02-23T00:48:46.383443+0000","last_active":"2024-02-23T00:48:46.374300+0000","last_peered":"2024-02-23T00:48:46.374300+0000","last_clean":"2024-02-23T00:48:46.374300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:48:46.374300+0000","last_undegraded":"2024-02-23T00:48:46.374300+0000","last_fullsized":"2024-02-23T00:48:46.374300+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":435,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_clean_scrub_stamp":"2024-02-23T00:48:46.374300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"213.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:48:46.374300+0000","last_change":"2024-02-23T00:48:46.385516+0000","last_active":"2024-02-23T00:48:46.374300+0000","last_peered":"2024-02-23T00:48:46.374300+0000","last_clean":"2024-02-23T00:48:46.374300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:48:46.374300+0000","last_undegraded":"2024-02-23T00:48:46.374300+0000","last_fullsized":"2024-02-23T00:48:46.374300+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":435,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_clean_scrub_stamp":"2024-02-23T00:48:46.374300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"213.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:48:46.374300+0000","last_change":"2024-02-23T00:48:46.390702+0000","last_active":"2024-02-23T00:48:46.374300+0000","last_peered":"2024-02-23T00:48:46.374300+0000","last_clean":"2024-02-23T00:48:46.374300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:48:46.374300+0000","last_undegraded":"2024-02-23T00:48:46.374300+0000","last_fullsized":"2024-02-23T00:48:46.374300+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":435,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_clean_scrub_stamp":"2024-02-23T00:48:46.374300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"213.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:48:46.374300+0000","last_change":"2024-02-23T00:48:46.394040+0000","last_active":"2024-02-23T00:48:46.374300+0000","last_peered":"2024-02-23T00:48:46.374300+0000","last_clean":"2024-02-23T00:48:46.374300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:48:46.374300+0000","last_undegraded":"2024-02-23T00:48:46.374300+0000","last_fullsized":"2024-02-23T00:48:46.374300+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":435,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_clean_scrub_stamp":"2024-02-23T00:48:46.374300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"213.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:48:46.374300+0000","last_change":"2024-02-23T00:48:46.389848+0000","last_active":"2024-02-23T00:48:46.374300+0000","last_peered":"2024-02-23T00:48:46.374300+0000","last_clean":"2024-02-23T00:48:46.374300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:48:46.374300+0000","last_undegraded":"2024-02-23T00:48:46.374300+0000","last_fullsized":"2024-02-23T00:48:46.374300+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":435,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_clean_scrub_stamp":"2024-02-23T00:48:46.374300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,3],"acting":[7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"213.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:48:46.374300+0000","last_change":"2024-02-23T00:48:46.387285+0000","last_active":"2024-02-23T00:48:46.374300+0000","last_peered":"2024-02-23T00:48:46.374300+0000","last_clean":"2024-02-23T00:48:46.374300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:48:46.374300+0000","last_undegraded":"2024-02-23T00:48:46.374300+0000","last_fullsized":"2024-02-23T00:48:46.374300+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":435,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_clean_scrub_stamp":"2024-02-23T00:48:46.374300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"213.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:48:46.374300+0000","last_change":"2024-02-23T00:48:46.389709+0000","last_active":"2024-02-23T00:48:46.374300+0000","last_peered":"2024-02-23T00:48:46.374300+0000","last_clean":"2024-02-23T00:48:46.374300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:48:46.374300+0000","last_undegraded":"2024-02-23T00:48:46.374300+0000","last_fullsized":"2024-02-23T00:48:46.374300+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":435,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_clean_scrub_stamp":"2024-02-23T00:48:46.374300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"213.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:48:46.374300+0000","last_change":"2024-02-23T00:48:46.391763+0000","last_active":"2024-02-23T00:48:46.374300+0000","last_peered":"2024-02-23T00:48:46.374300+0000","last_clean":"2024-02-23T00:48:46.374300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:48:46.374300+0000","last_undegraded":"2024-02-23T00:48:46.374300+0000","last_fullsized":"2024-02-23T00:48:46.374300+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":435,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_clean_scrub_stamp":"2024-02-23T00:48:46.374300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"213.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:48:46.374300+0000","last_change":"2024-02-23T00:48:46.387241+0000","last_active":"2024-02-23T00:48:46.374300+0000","last_peered":"2024-02-23T00:48:46.374300+0000","last_clean":"2024-02-23T00:48:46.374300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:48:46.374300+0000","last_undegraded":"2024-02-23T00:48:46.374300+0000","last_fullsized":"2024-02-23T00:48:46.374300+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":435,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_clean_scrub_stamp":"2024-02-23T00:48:46.374300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"213.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:48:46.374300+0000","last_change":"2024-02-23T00:48:46.386294+0000","last_active":"2024-02-23T00:48:46.374300+0000","last_peered":"2024-02-23T00:48:46.374300+0000","last_clean":"2024-02-23T00:48:46.374300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:48:46.374300+0000","last_undegraded":"2024-02-23T00:48:46.374300+0000","last_fullsized":"2024-02-23T00:48:46.374300+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":435,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_clean_scrub_stamp":"2024-02-23T00:48:46.374300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7],"acting":[6,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"213.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:48:46.374300+0000","last_change":"2024-02-23T00:48:46.394440+0000","last_active":"2024-02-23T00:48:46.374300+0000","last_peered":"2024-02-23T00:48:46.374300+0000","last_clean":"2024-02-23T00:48:46.374300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:48:46.374300+0000","last_undegraded":"2024-02-23T00:48:46.374300+0000","last_fullsized":"2024-02-23T00:48:46.374300+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":435,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_clean_scrub_stamp":"2024-02-23T00:48:46.374300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"213.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:48:46.374300+0000","last_change":"2024-02-23T00:48:46.387832+0000","last_active":"2024-02-23T00:48:46.374300+0000","last_peered":"2024-02-23T00:48:46.374300+0000","last_clean":"2024-02-23T00:48:46.374300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:48:46.374300+0000","last_undegraded":"2024-02-23T00:48:46.374300+0000","last_fullsized":"2024-02-23T00:48:46.374300+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":435,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_clean_scrub_stamp":"2024-02-23T00:48:46.374300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"213.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:48:46.374300+0000","last_change":"2024-02-23T00:48:46.383450+0000","last_active":"2024-02-23T00:48:46.374300+0000","last_peered":"2024-02-23T00:48:46.374300+0000","last_clean":"2024-02-23T00:48:46.374300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:48:46.374300+0000","last_undegraded":"2024-02-23T00:48:46.374300+0000","last_fullsized":"2024-02-23T00:48:46.374300+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":435,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_clean_scrub_stamp":"2024-02-23T00:48:46.374300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"213.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:48:46.374300+0000","last_change":"2024-02-23T00:48:46.386640+0000","last_active":"2024-02-23T00:48:46.374300+0000","last_peered":"2024-02-23T00:48:46.374300+0000","last_clean":"2024-02-23T00:48:46.374300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:48:46.374300+0000","last_undegraded":"2024-02-23T00:48:46.374300+0000","last_fullsized":"2024-02-23T00:48:46.374300+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":435,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_clean_scrub_stamp":"2024-02-23T00:48:46.374300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"213.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:48:46.374300+0000","last_change":"2024-02-23T00:48:46.392263+0000","last_active":"2024-02-23T00:48:46.374300+0000","last_peered":"2024-02-23T00:48:46.374300+0000","last_clean":"2024-02-23T00:48:46.374300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:48:46.374300+0000","last_undegraded":"2024-02-23T00:48:46.374300+0000","last_fullsized":"2024-02-23T00:48:46.374300+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":435,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_clean_scrub_stamp":"2024-02-23T00:48:46.374300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"213.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:48:46.374300+0000","last_change":"2024-02-23T00:48:46.385202+0000","last_active":"2024-02-23T00:48:46.374300+0000","last_peered":"2024-02-23T00:48:46.374300+0000","last_clean":"2024-02-23T00:48:46.374300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:48:46.374300+0000","last_undegraded":"2024-02-23T00:48:46.374300+0000","last_fullsized":"2024-02-23T00:48:46.374300+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":435,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_clean_scrub_stamp":"2024-02-23T00:48:46.374300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"213.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:48:46.374300+0000","last_change":"2024-02-23T00:48:46.387839+0000","last_active":"2024-02-23T00:48:46.374300+0000","last_peered":"2024-02-23T00:48:46.374300+0000","last_clean":"2024-02-23T00:48:46.374300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:48:46.374300+0000","last_undegraded":"2024-02-23T00:48:46.374300+0000","last_fullsized":"2024-02-23T00:48:46.374300+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":435,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_clean_scrub_stamp":"2024-02-23T00:48:46.374300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"213.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:48:46.374300+0000","last_change":"2024-02-23T00:48:46.385094+0000","last_active":"2024-02-23T00:48:46.374300+0000","last_peered":"2024-02-23T00:48:46.374300+0000","last_clean":"2024-02-23T00:48:46.374300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:48:46.374300+0000","last_undegraded":"2024-02-23T00:48:46.374300+0000","last_fullsized":"2024-02-23T00:48:46.374300+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":435,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_clean_scrub_stamp":"2024-02-23T00:48:46.374300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"213.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:48:46.374300+0000","last_change":"2024-02-23T00:48:46.390358+0000","last_active":"2024-02-23T00:48:46.374300+0000","last_peered":"2024-02-23T00:48:46.374300+0000","last_clean":"2024-02-23T00:48:46.374300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:48:46.374300+0000","last_undegraded":"2024-02-23T00:48:46.374300+0000","last_fullsized":"2024-02-23T00:48:46.374300+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":435,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_clean_scrub_stamp":"2024-02-23T00:48:46.374300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"213.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:48:46.374300+0000","last_change":"2024-02-23T00:48:46.385072+0000","last_active":"2024-02-23T00:48:46.374300+0000","last_peered":"2024-02-23T00:48:46.374300+0000","last_clean":"2024-02-23T00:48:46.374300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:48:46.374300+0000","last_undegraded":"2024-02-23T00:48:46.374300+0000","last_fullsized":"2024-02-23T00:48:46.374300+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":435,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_clean_scrub_stamp":"2024-02-23T00:48:46.374300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"213.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:48:46.374300+0000","last_change":"2024-02-23T00:48:46.386677+0000","last_active":"2024-02-23T00:48:46.374300+0000","last_peered":"2024-02-23T00:48:46.374300+0000","last_clean":"2024-02-23T00:48:46.374300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:48:46.374300+0000","last_undegraded":"2024-02-23T00:48:46.374300+0000","last_fullsized":"2024-02-23T00:48:46.374300+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":435,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_clean_scrub_stamp":"2024-02-23T00:48:46.374300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"213.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:48:46.374300+0000","last_change":"2024-02-23T00:48:46.385925+0000","last_active":"2024-02-23T00:48:46.374300+0000","last_peered":"2024-02-23T00:48:46.374300+0000","last_clean":"2024-02-23T00:48:46.374300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:48:46.374300+0000","last_undegraded":"2024-02-23T00:48:46.374300+0000","last_fullsized":"2024-02-23T00:48:46.374300+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":435,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_clean_scrub_stamp":"2024-02-23T00:48:46.374300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"213.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:48:46.374300+0000","last_change":"2024-02-23T00:48:46.385910+0000","last_active":"2024-02-23T00:48:46.374300+0000","last_peered":"2024-02-23T00:48:46.374300+0000","last_clean":"2024-02-23T00:48:46.374300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:48:46.374300+0000","last_undegraded":"2024-02-23T00:48:46.374300+0000","last_fullsized":"2024-02-23T00:48:46.374300+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":435,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_clean_scrub_stamp":"2024-02-23T00:48:46.374300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"213.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:48:46.374300+0000","last_change":"2024-02-23T00:48:46.389243+0000","last_active":"2024-02-23T00:48:46.374300+0000","last_peered":"2024-02-23T00:48:46.374300+0000","last_clean":"2024-02-23T00:48:46.374300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:48:46.374300+0000","last_undegraded":"2024-02-23T00:48:46.374300+0000","last_fullsized":"2024-02-23T00:48:46.374300+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":435,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_clean_scrub_stamp":"2024-02-23T00:48:46.374300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"213.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:48:46.374300+0000","last_change":"2024-02-23T00:48:46.386620+0000","last_active":"2024-02-23T00:48:46.374300+0000","last_peered":"2024-02-23T00:48:46.374300+0000","last_clean":"2024-02-23T00:48:46.374300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:48:46.374300+0000","last_undegraded":"2024-02-23T00:48:46.374300+0000","last_fullsized":"2024-02-23T00:48:46.374300+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":435,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:46.374300+0000","last_clean_scrub_stamp":"2024-02-23T00:48:46.374300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,4],"acting":[1,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.0","version":"259'40","reported_seq":461,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.715073+0000","last_change":"2024-02-23T00:48:43.728566+0000","last_active":"2024-02-23T00:48:45.715073+0000","last_peered":"2024-02-23T00:48:45.715073+0000","last_clean":"2024-02-23T00:48:45.715073+0000","last_became_active":"2024-02-23T00:47:28.238034+0000","last_became_peered":"2024-02-23T00:47:28.238034+0000","last_unstale":"2024-02-23T00:48:45.715073+0000","last_undegraded":"2024-02-23T00:48:45.715073+0000","last_fullsized":"2024-02-23T00:48:45.715073+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'40","last_scrub_stamp":"2024-02-23T00:48:43.728461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:48:43.728461+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":31,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":1,"num_write":13,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":31,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.1","version":"230'87","reported_seq":456,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.380773+0000","last_change":"2024-02-23T00:48:34.427639+0000","last_active":"2024-02-23T00:48:45.380773+0000","last_peered":"2024-02-23T00:48:45.380773+0000","last_clean":"2024-02-23T00:48:45.380773+0000","last_became_active":"2024-02-23T00:43:17.851866+0000","last_became_peered":"2024-02-23T00:43:17.851866+0000","last_unstale":"2024-02-23T00:48:45.380773+0000","last_undegraded":"2024-02-23T00:48:45.380773+0000","last_fullsized":"2024-02-23T00:48:45.380773+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"230'87","last_scrub_stamp":"2024-02-23T00:47:16.460312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:47:16.460312+0000","log_size":87,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":63,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.2","version":"229'68","reported_seq":441,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.260849+0000","last_change":"2024-02-23T00:48:34.353806+0000","last_active":"2024-02-23T00:48:46.260849+0000","last_peered":"2024-02-23T00:48:46.260849+0000","last_clean":"2024-02-23T00:48:46.260849+0000","last_became_active":"2024-02-23T00:43:17.804792+0000","last_became_peered":"2024-02-23T00:43:17.804792+0000","last_unstale":"2024-02-23T00:48:46.260849+0000","last_undegraded":"2024-02-23T00:48:46.260849+0000","last_fullsized":"2024-02-23T00:48:46.260849+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'68","last_scrub_stamp":"2024-02-23T00:48:04.587882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:48:04.587882+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.3","version":"246'20","reported_seq":278,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.382100+0000","last_change":"2024-02-23T00:48:34.429041+0000","last_active":"2024-02-23T00:48:45.382100+0000","last_peered":"2024-02-23T00:48:45.382100+0000","last_clean":"2024-02-23T00:48:45.382100+0000","last_became_active":"2024-02-23T00:47:28.226936+0000","last_became_peered":"2024-02-23T00:47:28.226936+0000","last_unstale":"2024-02-23T00:48:45.382100+0000","last_undegraded":"2024-02-23T00:48:45.382100+0000","last_fullsized":"2024-02-23T00:48:45.382100+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"246'20","last_scrub_stamp":"2024-02-23T00:45:53.067737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:53.067737+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":24,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.4","version":"207'17","reported_seq":393,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.769896+0000","last_change":"2024-02-23T00:48:34.430260+0000","last_active":"2024-02-23T00:48:45.769896+0000","last_peered":"2024-02-23T00:48:45.769896+0000","last_clean":"2024-02-23T00:48:45.769896+0000","last_became_active":"2024-02-23T00:43:17.810343+0000","last_became_peered":"2024-02-23T00:43:17.810343+0000","last_unstale":"2024-02-23T00:48:45.769896+0000","last_undegraded":"2024-02-23T00:48:45.769896+0000","last_fullsized":"2024-02-23T00:48:45.769896+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"207'17","last_scrub_stamp":"2024-02-23T00:46:35.627480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:35.627480+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.5","version":"237'51","reported_seq":427,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.380256+0000","last_change":"2024-02-23T00:48:34.428018+0000","last_active":"2024-02-23T00:48:45.380256+0000","last_peered":"2024-02-23T00:48:45.380256+0000","last_clean":"2024-02-23T00:48:45.380256+0000","last_became_active":"2024-02-23T00:43:17.855339+0000","last_became_peered":"2024-02-23T00:43:17.855339+0000","last_unstale":"2024-02-23T00:48:45.380256+0000","last_undegraded":"2024-02-23T00:48:45.380256+0000","last_fullsized":"2024-02-23T00:48:45.380256+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'51","last_scrub_stamp":"2024-02-23T00:46:40.379159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:40.379159+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.6","version":"379'54","reported_seq":484,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.717741+0000","last_change":"2024-02-23T00:48:34.429532+0000","last_active":"2024-02-23T00:48:45.717741+0000","last_peered":"2024-02-23T00:48:45.717741+0000","last_clean":"2024-02-23T00:48:45.717741+0000","last_became_active":"2024-02-23T00:47:28.236471+0000","last_became_peered":"2024-02-23T00:47:28.236471+0000","last_unstale":"2024-02-23T00:48:45.717741+0000","last_undegraded":"2024-02-23T00:48:45.717741+0000","last_fullsized":"2024-02-23T00:48:45.717741+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'52","last_scrub_stamp":"2024-02-23T00:47:26.428479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:47:26.428479+0000","log_size":54,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":18,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":41,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.7","version":"347'69","reported_seq":461,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.262160+0000","last_change":"2024-02-23T00:48:34.354241+0000","last_active":"2024-02-23T00:48:46.262160+0000","last_peered":"2024-02-23T00:48:46.262160+0000","last_clean":"2024-02-23T00:48:46.262160+0000","last_became_active":"2024-02-23T00:47:28.240920+0000","last_became_peered":"2024-02-23T00:47:28.240920+0000","last_unstale":"2024-02-23T00:48:46.262160+0000","last_undegraded":"2024-02-23T00:48:46.262160+0000","last_fullsized":"2024-02-23T00:48:46.262160+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'69","last_scrub_stamp":"2024-02-23T00:47:54.430466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:47:54.430466+0000","log_size":69,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":52,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":21,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":52,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.8","version":"410'98","reported_seq":531,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.714970+0000","last_change":"2024-02-23T00:48:34.428204+0000","last_active":"2024-02-23T00:48:45.714970+0000","last_peered":"2024-02-23T00:48:45.714970+0000","last_clean":"2024-02-23T00:48:45.714970+0000","last_became_active":"2024-02-23T00:47:28.229780+0000","last_became_peered":"2024-02-23T00:47:28.229780+0000","last_unstale":"2024-02-23T00:48:45.714970+0000","last_undegraded":"2024-02-23T00:48:45.714970+0000","last_fullsized":"2024-02-23T00:48:45.714970+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'90","last_scrub_stamp":"2024-02-23T00:46:05.820651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:05.820651+0000","log_size":98,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":78,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":33,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":68,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.9","version":"259'39","reported_seq":454,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.378623+0000","last_change":"2024-02-23T00:48:34.426893+0000","last_active":"2024-02-23T00:48:45.378623+0000","last_peered":"2024-02-23T00:48:45.378623+0000","last_clean":"2024-02-23T00:48:45.378623+0000","last_became_active":"2024-02-23T00:47:28.202025+0000","last_became_peered":"2024-02-23T00:47:28.202025+0000","last_unstale":"2024-02-23T00:48:45.378623+0000","last_undegraded":"2024-02-23T00:48:45.378623+0000","last_fullsized":"2024-02-23T00:48:45.378623+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'39","last_scrub_stamp":"2024-02-23T00:46:10.439373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:10.439373+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":12,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":27,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.a","version":"347'91","reported_seq":529,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.715334+0000","last_change":"2024-02-23T00:48:34.428810+0000","last_active":"2024-02-23T00:48:45.715334+0000","last_peered":"2024-02-23T00:48:45.715334+0000","last_clean":"2024-02-23T00:48:45.715334+0000","last_became_active":"2024-02-23T00:47:28.241187+0000","last_became_peered":"2024-02-23T00:47:28.241187+0000","last_unstale":"2024-02-23T00:48:45.715334+0000","last_undegraded":"2024-02-23T00:48:45.715334+0000","last_fullsized":"2024-02-23T00:48:45.715334+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'91","last_scrub_stamp":"2024-02-23T00:47:25.464279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:47:25.464279+0000","log_size":91,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":68,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":28,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":68,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.b","version":"418'84","reported_seq":456,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.716203+0000","last_change":"2024-02-23T00:48:34.429123+0000","last_active":"2024-02-23T00:48:45.716203+0000","last_peered":"2024-02-23T00:48:45.716203+0000","last_clean":"2024-02-23T00:48:45.716203+0000","last_became_active":"2024-02-23T00:43:17.855321+0000","last_became_peered":"2024-02-23T00:43:17.855321+0000","last_unstale":"2024-02-23T00:48:45.716203+0000","last_undegraded":"2024-02-23T00:48:45.716203+0000","last_fullsized":"2024-02-23T00:48:45.716203+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"418'84","last_scrub_stamp":"2024-02-23T00:48:10.926499+0000","last_deep_scrub":"418'84","last_deep_scrub_stamp":"2024-02-23T00:48:10.926499+0000","last_clean_scrub_stamp":"2024-02-23T00:48:10.926499+0000","log_size":84,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":54,"num_objects":7,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":24,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.c","version":"401'36","reported_seq":412,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.378212+0000","last_change":"2024-02-23T00:48:34.351899+0000","last_active":"2024-02-23T00:48:45.378212+0000","last_peered":"2024-02-23T00:48:45.378212+0000","last_clean":"2024-02-23T00:48:45.378212+0000","last_became_active":"2024-02-23T00:43:17.829044+0000","last_became_peered":"2024-02-23T00:43:17.829044+0000","last_unstale":"2024-02-23T00:48:45.378212+0000","last_undegraded":"2024-02-23T00:48:45.378212+0000","last_fullsized":"2024-02-23T00:48:45.378212+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'34","last_scrub_stamp":"2024-02-23T00:45:57.874883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:57.874883+0000","log_size":36,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":12,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.d","version":"414'58","reported_seq":471,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.379135+0000","last_change":"2024-02-23T00:48:34.428000+0000","last_active":"2024-02-23T00:48:45.379135+0000","last_peered":"2024-02-23T00:48:45.379135+0000","last_clean":"2024-02-23T00:48:45.379135+0000","last_became_active":"2024-02-23T00:47:28.202691+0000","last_became_peered":"2024-02-23T00:47:28.202691+0000","last_unstale":"2024-02-23T00:48:45.379135+0000","last_undegraded":"2024-02-23T00:48:45.379135+0000","last_fullsized":"2024-02-23T00:48:45.379135+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'56","last_scrub_stamp":"2024-02-23T00:45:51.258745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:51.258745+0000","log_size":58,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":18,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":33,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.e","version":"211'51","reported_seq":425,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.769076+0000","last_change":"2024-02-23T00:48:34.430702+0000","last_active":"2024-02-23T00:48:45.769076+0000","last_peered":"2024-02-23T00:48:45.769076+0000","last_clean":"2024-02-23T00:48:45.769076+0000","last_became_active":"2024-02-23T00:43:17.845956+0000","last_became_peered":"2024-02-23T00:43:17.845956+0000","last_unstale":"2024-02-23T00:48:45.769076+0000","last_undegraded":"2024-02-23T00:48:45.769076+0000","last_fullsized":"2024-02-23T00:48:45.769076+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'51","last_scrub_stamp":"2024-02-23T00:48:18.902135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:48:18.902135+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.f","version":"241'136","reported_seq":689,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.381784+0000","last_change":"2024-02-23T00:48:34.428662+0000","last_active":"2024-02-23T00:48:45.381784+0000","last_peered":"2024-02-23T00:48:45.381784+0000","last_clean":"2024-02-23T00:48:45.381784+0000","last_became_active":"2024-02-23T00:47:47.183032+0000","last_became_peered":"2024-02-23T00:47:47.183032+0000","last_unstale":"2024-02-23T00:48:45.381784+0000","last_undegraded":"2024-02-23T00:48:45.381784+0000","last_fullsized":"2024-02-23T00:48:45.381784+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"241'136","last_scrub_stamp":"2024-02-23T00:46:35.318571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:35.318571+0000","log_size":136,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":88,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":40,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.10","version":"347'69","reported_seq":438,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.381055+0000","last_change":"2024-02-23T00:48:34.428746+0000","last_active":"2024-02-23T00:48:45.381055+0000","last_peered":"2024-02-23T00:48:45.381055+0000","last_clean":"2024-02-23T00:48:45.381055+0000","last_became_active":"2024-02-23T00:43:17.819865+0000","last_became_peered":"2024-02-23T00:43:17.819865+0000","last_unstale":"2024-02-23T00:48:45.381055+0000","last_undegraded":"2024-02-23T00:48:45.381055+0000","last_fullsized":"2024-02-23T00:48:45.381055+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'69","last_scrub_stamp":"2024-02-23T00:47:15.414418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:47:15.414418+0000","log_size":69,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":52,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":21,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.11","version":"347'53","reported_seq":419,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.380674+0000","last_change":"2024-02-23T00:48:34.427975+0000","last_active":"2024-02-23T00:48:45.380674+0000","last_peered":"2024-02-23T00:48:45.380674+0000","last_clean":"2024-02-23T00:48:45.380674+0000","last_became_active":"2024-02-23T00:43:17.852697+0000","last_became_peered":"2024-02-23T00:43:17.852697+0000","last_unstale":"2024-02-23T00:48:45.380674+0000","last_undegraded":"2024-02-23T00:48:45.380674+0000","last_fullsized":"2024-02-23T00:48:45.380674+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'53","last_scrub_stamp":"2024-02-23T00:47:11.380987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:47:11.380987+0000","log_size":53,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":49,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":17,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.12","version":"240'73","reported_seq":351,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.767934+0000","last_change":"2024-02-23T00:48:34.429079+0000","last_active":"2024-02-23T00:48:45.767934+0000","last_peered":"2024-02-23T00:48:45.767934+0000","last_clean":"2024-02-23T00:48:45.767934+0000","last_became_active":"2024-02-23T00:47:28.213692+0000","last_became_peered":"2024-02-23T00:47:28.213692+0000","last_unstale":"2024-02-23T00:48:45.767934+0000","last_undegraded":"2024-02-23T00:48:45.767934+0000","last_fullsized":"2024-02-23T00:48:45.767934+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'73","last_scrub_stamp":"2024-02-23T00:48:14.866255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:48:14.866255+0000","log_size":73,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":49,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":49,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.13","version":"430'59","reported_seq":443,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.771143+0000","last_change":"2024-02-23T00:48:36.432636+0000","last_active":"2024-02-23T00:48:45.771143+0000","last_peered":"2024-02-23T00:48:45.771143+0000","last_clean":"2024-02-23T00:48:45.771143+0000","last_became_active":"2024-02-23T00:43:17.857823+0000","last_became_peered":"2024-02-23T00:43:17.857823+0000","last_unstale":"2024-02-23T00:48:45.771143+0000","last_undegraded":"2024-02-23T00:48:45.771143+0000","last_fullsized":"2024-02-23T00:48:45.771143+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"422'55","last_scrub_stamp":"2024-02-23T00:48:15.865963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:48:15.865963+0000","log_size":59,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":35,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":19,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.14","version":"420'54","reported_seq":479,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.716251+0000","last_change":"2024-02-23T00:48:34.429363+0000","last_active":"2024-02-23T00:48:45.716251+0000","last_peered":"2024-02-23T00:48:45.716251+0000","last_clean":"2024-02-23T00:48:45.716251+0000","last_became_active":"2024-02-23T00:47:28.235465+0000","last_became_peered":"2024-02-23T00:47:28.235465+0000","last_unstale":"2024-02-23T00:48:45.716251+0000","last_undegraded":"2024-02-23T00:48:45.716251+0000","last_fullsized":"2024-02-23T00:48:45.716251+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"420'54","last_scrub_stamp":"2024-02-23T00:48:12.892464+0000","last_deep_scrub":"420'54","last_deep_scrub_stamp":"2024-02-23T00:48:12.892464+0000","last_clean_scrub_stamp":"2024-02-23T00:48:12.892464+0000","log_size":54,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":59,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":51,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.15","version":"347'19","reported_seq":422,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.378351+0000","last_change":"2024-02-23T00:48:34.351402+0000","last_active":"2024-02-23T00:48:45.378351+0000","last_peered":"2024-02-23T00:48:45.378351+0000","last_clean":"2024-02-23T00:48:45.378351+0000","last_became_active":"2024-02-23T00:47:28.219548+0000","last_became_peered":"2024-02-23T00:47:28.219548+0000","last_unstale":"2024-02-23T00:48:45.378351+0000","last_undegraded":"2024-02-23T00:48:45.378351+0000","last_fullsized":"2024-02-23T00:48:45.378351+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"299'18","last_scrub_stamp":"2024-02-23T00:46:11.830580+0000","last_deep_scrub":"299'18","last_deep_scrub_stamp":"2024-02-23T00:46:11.830580+0000","last_clean_scrub_stamp":"2024-02-23T00:46:11.830580+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.16","version":"407'30","reported_seq":446,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.380592+0000","last_change":"2024-02-23T00:48:34.430522+0000","last_active":"2024-02-23T00:48:45.380592+0000","last_peered":"2024-02-23T00:48:45.380592+0000","last_clean":"2024-02-23T00:48:45.380592+0000","last_became_active":"2024-02-23T00:47:28.199870+0000","last_became_peered":"2024-02-23T00:47:28.199870+0000","last_unstale":"2024-02-23T00:48:45.380592+0000","last_undegraded":"2024-02-23T00:48:45.380592+0000","last_fullsized":"2024-02-23T00:48:45.380592+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'23","last_scrub_stamp":"2024-02-23T00:45:56.354881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:45:56.354881+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":64,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":14,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.17","version":"375'63","reported_seq":445,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.260868+0000","last_change":"2024-02-23T00:48:34.354299+0000","last_active":"2024-02-23T00:48:46.260868+0000","last_peered":"2024-02-23T00:48:46.260868+0000","last_clean":"2024-02-23T00:48:46.260868+0000","last_became_active":"2024-02-23T00:43:17.843525+0000","last_became_peered":"2024-02-23T00:43:17.843525+0000","last_unstale":"2024-02-23T00:48:46.260868+0000","last_undegraded":"2024-02-23T00:48:46.260868+0000","last_fullsized":"2024-02-23T00:48:46.260868+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'63","last_scrub_stamp":"2024-02-23T00:47:27.229920+0000","last_deep_scrub":"375'63","last_deep_scrub_stamp":"2024-02-23T00:47:27.229920+0000","last_clean_scrub_stamp":"2024-02-23T00:47:27.229920+0000","log_size":63,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":20,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.19","version":"267'96","reported_seq":544,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.262358+0000","last_change":"2024-02-23T00:48:34.352913+0000","last_active":"2024-02-23T00:48:46.262358+0000","last_peered":"2024-02-23T00:48:46.262358+0000","last_clean":"2024-02-23T00:48:46.262358+0000","last_became_active":"2024-02-23T00:47:28.210660+0000","last_became_peered":"2024-02-23T00:47:28.210660+0000","last_unstale":"2024-02-23T00:48:46.262358+0000","last_undegraded":"2024-02-23T00:48:46.262358+0000","last_fullsized":"2024-02-23T00:48:46.262358+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'96","last_scrub_stamp":"2024-02-23T00:47:55.479671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:47:55.479671+0000","log_size":96,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":73,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":7,"num_read_kb":0,"num_write":30,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":73,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,7],"acting":[6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.18","version":"410'12","reported_seq":414,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.377508+0000","last_change":"2024-02-23T00:48:34.349418+0000","last_active":"2024-02-23T00:48:45.377508+0000","last_peered":"2024-02-23T00:48:45.377508+0000","last_clean":"2024-02-23T00:48:45.377508+0000","last_became_active":"2024-02-23T00:47:28.221906+0000","last_became_peered":"2024-02-23T00:47:28.221906+0000","last_unstale":"2024-02-23T00:48:45.377508+0000","last_undegraded":"2024-02-23T00:48:45.377508+0000","last_fullsized":"2024-02-23T00:48:45.377508+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"275'7","last_scrub_stamp":"2024-02-23T00:46:03.927783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:03.927783+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":15,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":5,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1b","version":"347'70","reported_seq":439,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.378846+0000","last_change":"2024-02-23T00:48:34.428804+0000","last_active":"2024-02-23T00:48:45.378846+0000","last_peered":"2024-02-23T00:48:45.378846+0000","last_clean":"2024-02-23T00:48:45.378846+0000","last_became_active":"2024-02-23T00:43:17.803043+0000","last_became_peered":"2024-02-23T00:43:17.803043+0000","last_unstale":"2024-02-23T00:48:45.378846+0000","last_undegraded":"2024-02-23T00:48:45.378846+0000","last_fullsized":"2024-02-23T00:48:45.378846+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"304'69","last_scrub_stamp":"2024-02-23T00:46:37.397467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.397467+0000","log_size":70,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4000060,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":22,"num_write_kb":3928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.1a","version":"347'35","reported_seq":442,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.263015+0000","last_change":"2024-02-23T00:48:34.353742+0000","last_active":"2024-02-23T00:48:46.263015+0000","last_peered":"2024-02-23T00:48:46.263015+0000","last_clean":"2024-02-23T00:48:46.263015+0000","last_became_active":"2024-02-23T00:45:44.178195+0000","last_became_peered":"2024-02-23T00:45:44.178195+0000","last_unstale":"2024-02-23T00:48:46.263015+0000","last_undegraded":"2024-02-23T00:48:46.263015+0000","last_fullsized":"2024-02-23T00:48:46.263015+0000","mapping_epoch":271,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":272,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'35","last_scrub_stamp":"2024-02-23T00:47:17.368393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:47:17.368393+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":30,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":11,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1d","version":"215'104","reported_seq":475,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.381772+0000","last_change":"2024-02-23T00:48:42.456689+0000","last_active":"2024-02-23T00:48:46.381772+0000","last_peered":"2024-02-23T00:48:46.381772+0000","last_clean":"2024-02-23T00:48:46.381772+0000","last_became_active":"2024-02-23T00:43:17.828194+0000","last_became_peered":"2024-02-23T00:43:17.828194+0000","last_unstale":"2024-02-23T00:48:46.381772+0000","last_undegraded":"2024-02-23T00:48:46.381772+0000","last_fullsized":"2024-02-23T00:48:46.381772+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'104","last_scrub_stamp":"2024-02-23T00:48:42.456616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:48:42.456616+0000","log_size":104,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":74,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":32,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"62.1c","version":"374'55","reported_seq":429,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.260876+0000","last_change":"2024-02-23T00:48:34.354038+0000","last_active":"2024-02-23T00:48:46.260876+0000","last_peered":"2024-02-23T00:48:46.260876+0000","last_clean":"2024-02-23T00:48:46.260876+0000","last_became_active":"2024-02-23T00:43:17.804718+0000","last_became_peered":"2024-02-23T00:43:17.804718+0000","last_unstale":"2024-02-23T00:48:46.260876+0000","last_undegraded":"2024-02-23T00:48:46.260876+0000","last_fullsized":"2024-02-23T00:48:46.260876+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"374'55","last_scrub_stamp":"2024-02-23T00:47:26.267291+0000","last_deep_scrub":"374'55","last_deep_scrub_stamp":"2024-02-23T00:47:26.267291+0000","last_clean_scrub_stamp":"2024-02-23T00:47:26.267291+0000","log_size":55,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1f","version":"414'43","reported_seq":437,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.717550+0000","last_change":"2024-02-23T00:48:34.430846+0000","last_active":"2024-02-23T00:48:45.717550+0000","last_peered":"2024-02-23T00:48:45.717550+0000","last_clean":"2024-02-23T00:48:45.717550+0000","last_became_active":"2024-02-23T00:47:28.243574+0000","last_became_peered":"2024-02-23T00:47:28.243574+0000","last_unstale":"2024-02-23T00:48:45.717550+0000","last_undegraded":"2024-02-23T00:48:45.717550+0000","last_fullsized":"2024-02-23T00:48:45.717550+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"414'43","last_scrub_stamp":"2024-02-23T00:48:09.898586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:48:09.898586+0000","log_size":43,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":43,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":15,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":35,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.1e","version":"245'51","reported_seq":468,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.717664+0000","last_change":"2024-02-23T00:48:42.689120+0000","last_active":"2024-02-23T00:48:45.717664+0000","last_peered":"2024-02-23T00:48:45.717664+0000","last_clean":"2024-02-23T00:48:45.717664+0000","last_became_active":"2024-02-23T00:47:28.244073+0000","last_became_peered":"2024-02-23T00:47:28.244073+0000","last_unstale":"2024-02-23T00:48:45.717664+0000","last_undegraded":"2024-02-23T00:48:45.717664+0000","last_fullsized":"2024-02-23T00:48:45.717664+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"245'51","last_scrub_stamp":"2024-02-23T00:48:42.689040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:48:42.689040+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":33,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"67'8","reported_seq":522,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.716014+0000","last_change":"2024-02-23T00:47:23.486066+0000","last_active":"2024-02-23T00:48:45.716014+0000","last_peered":"2024-02-23T00:48:45.716014+0000","last_clean":"2024-02-23T00:48:45.716014+0000","last_became_active":"2024-02-23T00:39:18.793414+0000","last_became_peered":"2024-02-23T00:39:18.793414+0000","last_unstale":"2024-02-23T00:48:45.716014+0000","last_undegraded":"2024-02-23T00:48:45.716014+0000","last_fullsized":"2024-02-23T00:48:45.716014+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:47:23.485949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:47:23.485949+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"68'27","reported_seq":455,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.382449+0000","last_change":"2024-02-23T00:47:20.244368+0000","last_active":"2024-02-23T00:48:46.382449+0000","last_peered":"2024-02-23T00:48:46.382449+0000","last_clean":"2024-02-23T00:48:46.382449+0000","last_became_active":"2024-02-23T00:39:16.859401+0000","last_became_peered":"2024-02-23T00:39:16.859401+0000","last_unstale":"2024-02-23T00:48:46.382449+0000","last_undegraded":"2024-02-23T00:48:46.382449+0000","last_fullsized":"2024-02-23T00:48:46.382449+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'27","last_scrub_stamp":"2024-02-23T00:47:20.244299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:47:20.244299+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":71,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.375738+0000","last_change":"2024-02-23T00:47:28.221354+0000","last_active":"2024-02-23T00:48:45.375738+0000","last_peered":"2024-02-23T00:48:45.375738+0000","last_clean":"2024-02-23T00:48:45.375738+0000","last_became_active":"2024-02-23T00:47:28.220876+0000","last_became_peered":"2024-02-23T00:47:28.220876+0000","last_unstale":"2024-02-23T00:48:45.375738+0000","last_undegraded":"2024-02-23T00:48:45.375738+0000","last_fullsized":"2024-02-23T00:48:45.375738+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.942091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:04.330426+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.942091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":414,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.382853+0000","last_change":"2024-02-23T00:47:11.224449+0000","last_active":"2024-02-23T00:48:46.382853+0000","last_peered":"2024-02-23T00:48:46.382853+0000","last_clean":"2024-02-23T00:48:46.382853+0000","last_became_active":"2024-02-23T00:43:32.282545+0000","last_became_peered":"2024-02-23T00:43:32.282545+0000","last_unstale":"2024-02-23T00:48:46.382853+0000","last_undegraded":"2024-02-23T00:48:46.382853+0000","last_fullsized":"2024-02-23T00:48:46.382853+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:11.224386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:47:11.224386+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":343,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.384316+0000","last_change":"2024-02-23T00:48:02.292630+0000","last_active":"2024-02-23T00:48:46.384316+0000","last_peered":"2024-02-23T00:48:46.384316+0000","last_clean":"2024-02-23T00:48:46.384316+0000","last_became_active":"2024-02-23T00:42:58.828846+0000","last_became_peered":"2024-02-23T00:42:58.828846+0000","last_unstale":"2024-02-23T00:48:46.384316+0000","last_undegraded":"2024-02-23T00:48:46.384316+0000","last_fullsized":"2024-02-23T00:48:46.384316+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:02.292555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:02.292555+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"68'15","reported_seq":432,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.771533+0000","last_change":"2024-02-23T00:46:45.625235+0000","last_active":"2024-02-23T00:48:45.771533+0000","last_peered":"2024-02-23T00:48:45.771533+0000","last_clean":"2024-02-23T00:48:45.771533+0000","last_became_active":"2024-02-23T00:39:16.783701+0000","last_became_peered":"2024-02-23T00:39:16.783701+0000","last_unstale":"2024-02-23T00:48:45.771533+0000","last_undegraded":"2024-02-23T00:48:45.771533+0000","last_fullsized":"2024-02-23T00:48:45.771533+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:46:45.625126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:45.625126+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":396,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.383811+0000","last_change":"2024-02-23T00:46:04.476277+0000","last_active":"2024-02-23T00:48:45.383811+0000","last_peered":"2024-02-23T00:48:45.383811+0000","last_clean":"2024-02-23T00:48:45.383811+0000","last_became_active":"2024-02-23T00:39:14.772493+0000","last_became_peered":"2024-02-23T00:39:14.772493+0000","last_unstale":"2024-02-23T00:48:45.383811+0000","last_undegraded":"2024-02-23T00:48:45.383811+0000","last_fullsized":"2024-02-23T00:48:45.383811+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:04.476156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:04.476156+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":70,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.381508+0000","last_change":"2024-02-23T00:47:28.232389+0000","last_active":"2024-02-23T00:48:45.381508+0000","last_peered":"2024-02-23T00:48:45.381508+0000","last_clean":"2024-02-23T00:48:45.381508+0000","last_became_active":"2024-02-23T00:47:28.231567+0000","last_became_peered":"2024-02-23T00:47:28.231567+0000","last_unstale":"2024-02-23T00:48:45.381508+0000","last_undegraded":"2024-02-23T00:48:45.381508+0000","last_fullsized":"2024-02-23T00:48:45.381508+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:45.674391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:08.335330+0000","last_clean_scrub_stamp":"2024-02-23T00:46:45.674391+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":424,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.768515+0000","last_change":"2024-02-23T00:48:13.867784+0000","last_active":"2024-02-23T00:48:45.768515+0000","last_peered":"2024-02-23T00:48:45.768515+0000","last_clean":"2024-02-23T00:48:45.768515+0000","last_became_active":"2024-02-23T00:47:28.214157+0000","last_became_peered":"2024-02-23T00:47:28.214157+0000","last_unstale":"2024-02-23T00:48:45.768515+0000","last_undegraded":"2024-02-23T00:48:45.768515+0000","last_fullsized":"2024-02-23T00:48:45.768515+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:13.867716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:13.867716+0000","last_clean_scrub_stamp":"2024-02-23T00:48:13.867716+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":344,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.770151+0000","last_change":"2024-02-23T00:48:06.908037+0000","last_active":"2024-02-23T00:48:45.770151+0000","last_peered":"2024-02-23T00:48:45.770151+0000","last_clean":"2024-02-23T00:48:45.770151+0000","last_became_active":"2024-02-23T00:42:58.867305+0000","last_became_peered":"2024-02-23T00:42:58.867305+0000","last_unstale":"2024-02-23T00:48:45.770151+0000","last_undegraded":"2024-02-23T00:48:45.770151+0000","last_fullsized":"2024-02-23T00:48:45.770151+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:06.907966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:06.907966+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"88'35","reported_seq":459,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.264139+0000","last_change":"2024-02-23T00:48:42.409822+0000","last_active":"2024-02-23T00:48:46.264139+0000","last_peered":"2024-02-23T00:48:46.264139+0000","last_clean":"2024-02-23T00:48:46.264139+0000","last_became_active":"2024-02-23T00:39:16.857413+0000","last_became_peered":"2024-02-23T00:39:16.857413+0000","last_unstale":"2024-02-23T00:48:46.264139+0000","last_undegraded":"2024-02-23T00:48:46.264139+0000","last_fullsized":"2024-02-23T00:48:46.264139+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'35","last_scrub_stamp":"2024-02-23T00:48:42.409686+0000","last_deep_scrub":"88'35","last_deep_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_clean_scrub_stamp":"2024-02-23T00:48:42.409686+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":70,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.383610+0000","last_change":"2024-02-23T00:47:28.232209+0000","last_active":"2024-02-23T00:48:46.383610+0000","last_peered":"2024-02-23T00:48:46.383610+0000","last_clean":"2024-02-23T00:48:46.383610+0000","last_became_active":"2024-02-23T00:47:28.231891+0000","last_became_peered":"2024-02-23T00:47:28.231891+0000","last_unstale":"2024-02-23T00:48:46.383610+0000","last_undegraded":"2024-02-23T00:48:46.383610+0000","last_fullsized":"2024-02-23T00:48:46.383610+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:05.910325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_clean_scrub_stamp":"2024-02-23T00:46:05.910325+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":71,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.771377+0000","last_change":"2024-02-23T00:47:28.220900+0000","last_active":"2024-02-23T00:48:45.771377+0000","last_peered":"2024-02-23T00:48:45.771377+0000","last_clean":"2024-02-23T00:48:45.771377+0000","last_became_active":"2024-02-23T00:47:28.220578+0000","last_became_peered":"2024-02-23T00:47:28.220578+0000","last_unstale":"2024-02-23T00:48:45.771377+0000","last_undegraded":"2024-02-23T00:48:45.771377+0000","last_fullsized":"2024-02-23T00:48:45.771377+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:34.726027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:34.726027+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":422,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.715164+0000","last_change":"2024-02-23T00:47:28.216027+0000","last_active":"2024-02-23T00:48:45.715164+0000","last_peered":"2024-02-23T00:48:45.715164+0000","last_clean":"2024-02-23T00:48:45.715164+0000","last_became_active":"2024-02-23T00:47:28.215636+0000","last_became_peered":"2024-02-23T00:47:28.215636+0000","last_unstale":"2024-02-23T00:48:45.715164+0000","last_undegraded":"2024-02-23T00:48:45.715164+0000","last_fullsized":"2024-02-23T00:48:45.715164+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:48.692679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:48.692679+0000","last_clean_scrub_stamp":"2024-02-23T00:46:48.692679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":338,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.382891+0000","last_change":"2024-02-23T00:46:40.354058+0000","last_active":"2024-02-23T00:48:45.382891+0000","last_peered":"2024-02-23T00:48:45.382891+0000","last_clean":"2024-02-23T00:48:45.382891+0000","last_became_active":"2024-02-23T00:42:57.639013+0000","last_became_peered":"2024-02-23T00:42:57.639013+0000","last_unstale":"2024-02-23T00:48:45.382891+0000","last_undegraded":"2024-02-23T00:48:45.382891+0000","last_fullsized":"2024-02-23T00:48:45.382891+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:40.353999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:46:40.353999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"67'17","reported_seq":477,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.380635+0000","last_change":"2024-02-23T00:47:28.219455+0000","last_active":"2024-02-23T00:48:45.380635+0000","last_peered":"2024-02-23T00:48:45.380635+0000","last_clean":"2024-02-23T00:48:45.380635+0000","last_became_active":"2024-02-23T00:47:28.200653+0000","last_became_peered":"2024-02-23T00:47:28.200653+0000","last_unstale":"2024-02-23T00:48:45.380635+0000","last_undegraded":"2024-02-23T00:48:45.380635+0000","last_fullsized":"2024-02-23T00:48:45.380635+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'17","last_scrub_stamp":"2024-02-23T00:47:15.357876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:47:15.357876+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":74,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.716040+0000","last_change":"2024-02-23T00:48:44.698517+0000","last_active":"2024-02-23T00:48:45.716040+0000","last_peered":"2024-02-23T00:48:45.716040+0000","last_clean":"2024-02-23T00:48:45.716040+0000","last_became_active":"2024-02-23T00:47:28.201926+0000","last_became_peered":"2024-02-23T00:47:28.201926+0000","last_unstale":"2024-02-23T00:48:45.716040+0000","last_undegraded":"2024-02-23T00:48:45.716040+0000","last_fullsized":"2024-02-23T00:48:45.716040+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:44.698446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:48:44.698446+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":397,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.383231+0000","last_change":"2024-02-23T00:48:03.801207+0000","last_active":"2024-02-23T00:48:45.383231+0000","last_peered":"2024-02-23T00:48:45.383231+0000","last_clean":"2024-02-23T00:48:45.383231+0000","last_became_active":"2024-02-23T00:39:18.799875+0000","last_became_peered":"2024-02-23T00:39:18.799875+0000","last_unstale":"2024-02-23T00:48:45.383231+0000","last_undegraded":"2024-02-23T00:48:45.383231+0000","last_fullsized":"2024-02-23T00:48:45.383231+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:03.801103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:48:03.801103+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":338,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.382267+0000","last_change":"2024-02-23T00:45:23.744990+0000","last_active":"2024-02-23T00:48:45.382267+0000","last_peered":"2024-02-23T00:48:45.382267+0000","last_clean":"2024-02-23T00:48:45.382267+0000","last_became_active":"2024-02-23T00:42:57.675306+0000","last_became_peered":"2024-02-23T00:42:57.675306+0000","last_unstale":"2024-02-23T00:48:45.382267+0000","last_undegraded":"2024-02-23T00:48:45.382267+0000","last_fullsized":"2024-02-23T00:48:45.382267+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:23.744846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:23.744846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"67'10","reported_seq":410,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.379696+0000","last_change":"2024-02-23T00:45:25.175258+0000","last_active":"2024-02-23T00:48:45.379696+0000","last_peered":"2024-02-23T00:48:45.379696+0000","last_clean":"2024-02-23T00:48:45.379696+0000","last_became_active":"2024-02-23T00:39:16.777530+0000","last_became_peered":"2024-02-23T00:39:16.777530+0000","last_unstale":"2024-02-23T00:48:45.379696+0000","last_undegraded":"2024-02-23T00:48:45.379696+0000","last_fullsized":"2024-02-23T00:48:45.379696+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'10","last_scrub_stamp":"2024-02-23T00:45:25.175173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:25.175173+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":401,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.382261+0000","last_change":"2024-02-23T00:48:41.719867+0000","last_active":"2024-02-23T00:48:45.382261+0000","last_peered":"2024-02-23T00:48:45.382261+0000","last_clean":"2024-02-23T00:48:45.382261+0000","last_became_active":"2024-02-23T00:39:14.780379+0000","last_became_peered":"2024-02-23T00:39:14.780379+0000","last_unstale":"2024-02-23T00:48:45.382261+0000","last_undegraded":"2024-02-23T00:48:45.382261+0000","last_fullsized":"2024-02-23T00:48:45.382261+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:41.719772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.575242+0000","last_clean_scrub_stamp":"2024-02-23T00:48:41.719772+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"376'9","reported_seq":110,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.716923+0000","last_change":"2024-02-23T00:47:28.336504+0000","last_active":"2024-02-23T00:48:45.716923+0000","last_peered":"2024-02-23T00:48:45.716923+0000","last_clean":"2024-02-23T00:48:45.716923+0000","last_became_active":"2024-02-23T00:47:28.231104+0000","last_became_peered":"2024-02-23T00:47:28.231104+0000","last_unstale":"2024-02-23T00:48:45.716923+0000","last_undegraded":"2024-02-23T00:48:45.716923+0000","last_fullsized":"2024-02-23T00:48:45.716923+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:46:49.665271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:49.665271+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":340,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.376961+0000","last_change":"2024-02-23T00:45:22.874761+0000","last_active":"2024-02-23T00:48:45.376961+0000","last_peered":"2024-02-23T00:48:45.376961+0000","last_clean":"2024-02-23T00:48:45.376961+0000","last_became_active":"2024-02-23T00:42:58.765106+0000","last_became_peered":"2024-02-23T00:42:58.765106+0000","last_unstale":"2024-02-23T00:48:45.376961+0000","last_undegraded":"2024-02-23T00:48:45.376961+0000","last_fullsized":"2024-02-23T00:48:45.376961+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:22.874696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:22.874696+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"67'15","reported_seq":424,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.384761+0000","last_change":"2024-02-23T00:48:00.698737+0000","last_active":"2024-02-23T00:48:45.384761+0000","last_peered":"2024-02-23T00:48:45.384761+0000","last_clean":"2024-02-23T00:48:45.384761+0000","last_became_active":"2024-02-23T00:39:16.789757+0000","last_became_peered":"2024-02-23T00:39:16.789757+0000","last_unstale":"2024-02-23T00:48:45.384761+0000","last_undegraded":"2024-02-23T00:48:45.384761+0000","last_fullsized":"2024-02-23T00:48:45.384761+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:48:00.698670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:48:00.698670+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":401,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.381039+0000","last_change":"2024-02-23T00:48:09.801324+0000","last_active":"2024-02-23T00:48:45.381039+0000","last_peered":"2024-02-23T00:48:45.381039+0000","last_clean":"2024-02-23T00:48:45.381039+0000","last_became_active":"2024-02-23T00:39:14.780526+0000","last_became_peered":"2024-02-23T00:39:14.780526+0000","last_unstale":"2024-02-23T00:48:45.381039+0000","last_undegraded":"2024-02-23T00:48:45.381039+0000","last_fullsized":"2024-02-23T00:48:45.381039+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:09.801251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:48:09.801251+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":520,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.380806+0000","last_change":"2024-02-23T00:47:24.353965+0000","last_active":"2024-02-23T00:48:45.380806+0000","last_peered":"2024-02-23T00:48:45.380806+0000","last_clean":"2024-02-23T00:48:45.380806+0000","last_became_active":"2024-02-23T00:39:18.800440+0000","last_became_peered":"2024-02-23T00:39:18.800440+0000","last_unstale":"2024-02-23T00:48:45.380806+0000","last_undegraded":"2024-02-23T00:48:45.380806+0000","last_fullsized":"2024-02-23T00:48:45.380806+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:47:24.353888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:47:24.353888+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":339,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.378682+0000","last_change":"2024-02-23T00:45:54.311565+0000","last_active":"2024-02-23T00:48:45.378682+0000","last_peered":"2024-02-23T00:48:45.378682+0000","last_clean":"2024-02-23T00:48:45.378682+0000","last_became_active":"2024-02-23T00:42:58.833924+0000","last_became_peered":"2024-02-23T00:42:58.833924+0000","last_unstale":"2024-02-23T00:48:45.378682+0000","last_undegraded":"2024-02-23T00:48:45.378682+0000","last_fullsized":"2024-02-23T00:48:45.378682+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:54.311442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:54.311442+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"336'23","reported_seq":481,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.380689+0000","last_change":"2024-02-23T00:47:28.259863+0000","last_active":"2024-02-23T00:48:45.380689+0000","last_peered":"2024-02-23T00:48:45.380689+0000","last_clean":"2024-02-23T00:48:45.380689+0000","last_became_active":"2024-02-23T00:47:28.201225+0000","last_became_peered":"2024-02-23T00:47:28.201225+0000","last_unstale":"2024-02-23T00:48:45.380689+0000","last_undegraded":"2024-02-23T00:48:45.380689+0000","last_fullsized":"2024-02-23T00:48:45.380689+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"336'23","last_scrub_stamp":"2024-02-23T00:46:48.320085+0000","last_deep_scrub":"336'23","last_deep_scrub_stamp":"2024-02-23T00:46:48.320085+0000","last_clean_scrub_stamp":"2024-02-23T00:46:48.320085+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":401,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.715475+0000","last_change":"2024-02-23T00:46:43.722544+0000","last_active":"2024-02-23T00:48:45.715475+0000","last_peered":"2024-02-23T00:48:45.715475+0000","last_clean":"2024-02-23T00:48:45.715475+0000","last_became_active":"2024-02-23T00:39:14.784121+0000","last_became_peered":"2024-02-23T00:39:14.784121+0000","last_unstale":"2024-02-23T00:48:45.715475+0000","last_undegraded":"2024-02-23T00:48:45.715475+0000","last_fullsized":"2024-02-23T00:48:45.715475+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:43.722480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:43.722480+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":398,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.385942+0000","last_change":"2024-02-23T00:47:18.254453+0000","last_active":"2024-02-23T00:48:46.385942+0000","last_peered":"2024-02-23T00:48:46.385942+0000","last_clean":"2024-02-23T00:48:46.385942+0000","last_became_active":"2024-02-23T00:39:18.800777+0000","last_became_peered":"2024-02-23T00:39:18.800777+0000","last_unstale":"2024-02-23T00:48:46.385942+0000","last_undegraded":"2024-02-23T00:48:46.385942+0000","last_fullsized":"2024-02-23T00:48:46.385942+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:18.254393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_clean_scrub_stamp":"2024-02-23T00:47:18.254393+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":338,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.381348+0000","last_change":"2024-02-23T00:46:35.407560+0000","last_active":"2024-02-23T00:48:45.381348+0000","last_peered":"2024-02-23T00:48:45.381348+0000","last_clean":"2024-02-23T00:48:45.381348+0000","last_became_active":"2024-02-23T00:42:57.653279+0000","last_became_peered":"2024-02-23T00:42:57.653279+0000","last_unstale":"2024-02-23T00:48:45.381348+0000","last_undegraded":"2024-02-23T00:48:45.381348+0000","last_fullsized":"2024-02-23T00:48:45.381348+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:35.407468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:46:35.407468+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":517,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.377870+0000","last_change":"2024-02-23T00:45:51.871688+0000","last_active":"2024-02-23T00:48:45.377870+0000","last_peered":"2024-02-23T00:48:45.377870+0000","last_clean":"2024-02-23T00:48:45.377870+0000","last_became_active":"2024-02-23T00:39:18.798536+0000","last_became_peered":"2024-02-23T00:39:18.798536+0000","last_unstale":"2024-02-23T00:48:45.377870+0000","last_undegraded":"2024-02-23T00:48:45.377870+0000","last_fullsized":"2024-02-23T00:48:45.377870+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:45:51.871618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:45:51.871618+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"423'28","reported_seq":444,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.378766+0000","last_change":"2024-02-23T00:48:17.435988+0000","last_active":"2024-02-23T00:48:45.378766+0000","last_peered":"2024-02-23T00:48:45.378766+0000","last_clean":"2024-02-23T00:48:45.378766+0000","last_became_active":"2024-02-23T00:39:16.780130+0000","last_became_peered":"2024-02-23T00:39:16.780130+0000","last_unstale":"2024-02-23T00:48:45.378766+0000","last_undegraded":"2024-02-23T00:48:45.378766+0000","last_fullsized":"2024-02-23T00:48:45.378766+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"423'28","last_scrub_stamp":"2024-02-23T00:48:17.435906+0000","last_deep_scrub":"423'28","last_deep_scrub_stamp":"2024-02-23T00:48:17.435906+0000","last_clean_scrub_stamp":"2024-02-23T00:48:17.435906+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":416,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.263183+0000","last_change":"2024-02-23T00:48:13.584696+0000","last_active":"2024-02-23T00:48:46.263183+0000","last_peered":"2024-02-23T00:48:46.263183+0000","last_clean":"2024-02-23T00:48:46.263183+0000","last_became_active":"2024-02-23T00:47:28.237083+0000","last_became_peered":"2024-02-23T00:47:28.237083+0000","last_unstale":"2024-02-23T00:48:46.263183+0000","last_undegraded":"2024-02-23T00:48:46.263183+0000","last_fullsized":"2024-02-23T00:48:46.263183+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:13.584566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:51.391939+0000","last_clean_scrub_stamp":"2024-02-23T00:48:13.584566+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":414,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.716659+0000","last_change":"2024-02-23T00:47:30.187702+0000","last_active":"2024-02-23T00:48:45.716659+0000","last_peered":"2024-02-23T00:48:45.716659+0000","last_clean":"2024-02-23T00:48:45.716659+0000","last_became_active":"2024-02-23T00:47:30.187386+0000","last_became_peered":"2024-02-23T00:47:30.187386+0000","last_unstale":"2024-02-23T00:48:45.716659+0000","last_undegraded":"2024-02-23T00:48:45.716659+0000","last_fullsized":"2024-02-23T00:48:45.716659+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:46.686764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:46.686764+0000","last_clean_scrub_stamp":"2024-02-23T00:46:46.686764+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":347,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.386275+0000","last_change":"2024-02-23T00:47:28.212626+0000","last_active":"2024-02-23T00:48:46.386275+0000","last_peered":"2024-02-23T00:48:46.386275+0000","last_clean":"2024-02-23T00:48:46.386275+0000","last_became_active":"2024-02-23T00:47:28.212307+0000","last_became_peered":"2024-02-23T00:47:28.212307+0000","last_unstale":"2024-02-23T00:48:46.386275+0000","last_undegraded":"2024-02-23T00:48:46.386275+0000","last_fullsized":"2024-02-23T00:48:46.386275+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:59.944587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:59.944587+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":400,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.378924+0000","last_change":"2024-02-23T00:48:10.493418+0000","last_active":"2024-02-23T00:48:45.378924+0000","last_peered":"2024-02-23T00:48:45.378924+0000","last_clean":"2024-02-23T00:48:45.378924+0000","last_became_active":"2024-02-23T00:39:18.798829+0000","last_became_peered":"2024-02-23T00:39:18.798829+0000","last_unstale":"2024-02-23T00:48:45.378924+0000","last_undegraded":"2024-02-23T00:48:45.378924+0000","last_fullsized":"2024-02-23T00:48:45.378924+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:10.493340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:10.493340+0000","last_clean_scrub_stamp":"2024-02-23T00:48:10.493340+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"251'40","reported_seq":474,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.377778+0000","last_change":"2024-02-23T00:48:17.054839+0000","last_active":"2024-02-23T00:48:45.377778+0000","last_peered":"2024-02-23T00:48:45.377778+0000","last_clean":"2024-02-23T00:48:45.377778+0000","last_became_active":"2024-02-23T00:39:16.859002+0000","last_became_peered":"2024-02-23T00:39:16.859002+0000","last_unstale":"2024-02-23T00:48:45.377778+0000","last_undegraded":"2024-02-23T00:48:45.377778+0000","last_fullsized":"2024-02-23T00:48:45.377778+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'40","last_scrub_stamp":"2024-02-23T00:48:17.054722+0000","last_deep_scrub":"251'40","last_deep_scrub_stamp":"2024-02-23T00:45:23.910546+0000","last_clean_scrub_stamp":"2024-02-23T00:48:17.054722+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":401,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.383564+0000","last_change":"2024-02-23T00:48:44.728592+0000","last_active":"2024-02-23T00:48:45.383564+0000","last_peered":"2024-02-23T00:48:45.383564+0000","last_clean":"2024-02-23T00:48:45.383564+0000","last_became_active":"2024-02-23T00:39:14.779193+0000","last_became_peered":"2024-02-23T00:39:14.779193+0000","last_unstale":"2024-02-23T00:48:45.383564+0000","last_undegraded":"2024-02-23T00:48:45.383564+0000","last_fullsized":"2024-02-23T00:48:45.383564+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:44.728520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:48:44.728520+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":423,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.381560+0000","last_change":"2024-02-23T00:48:13.462943+0000","last_active":"2024-02-23T00:48:45.381560+0000","last_peered":"2024-02-23T00:48:45.381560+0000","last_clean":"2024-02-23T00:48:45.381560+0000","last_became_active":"2024-02-23T00:47:33.257915+0000","last_became_peered":"2024-02-23T00:47:33.257915+0000","last_unstale":"2024-02-23T00:48:45.381560+0000","last_undegraded":"2024-02-23T00:48:45.381560+0000","last_fullsized":"2024-02-23T00:48:45.381560+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":381,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:13.462861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:48:13.462861+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":345,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.376100+0000","last_change":"2024-02-23T00:48:04.989583+0000","last_active":"2024-02-23T00:48:45.376100+0000","last_peered":"2024-02-23T00:48:45.376100+0000","last_clean":"2024-02-23T00:48:45.376100+0000","last_became_active":"2024-02-23T00:42:58.809283+0000","last_became_peered":"2024-02-23T00:42:58.809283+0000","last_unstale":"2024-02-23T00:48:45.376100+0000","last_undegraded":"2024-02-23T00:48:45.376100+0000","last_fullsized":"2024-02-23T00:48:45.376100+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:04.989514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:04.989514+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":396,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.263922+0000","last_change":"2024-02-23T00:46:45.294769+0000","last_active":"2024-02-23T00:48:46.263922+0000","last_peered":"2024-02-23T00:48:46.263922+0000","last_clean":"2024-02-23T00:48:46.263922+0000","last_became_active":"2024-02-23T00:39:18.796664+0000","last_became_peered":"2024-02-23T00:39:18.796664+0000","last_unstale":"2024-02-23T00:48:46.263922+0000","last_undegraded":"2024-02-23T00:48:46.263922+0000","last_fullsized":"2024-02-23T00:48:46.263922+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:45.294704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:45.294704+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"203'34","reported_seq":461,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.771289+0000","last_change":"2024-02-23T00:47:18.765648+0000","last_active":"2024-02-23T00:48:45.771289+0000","last_peered":"2024-02-23T00:48:45.771289+0000","last_clean":"2024-02-23T00:48:45.771289+0000","last_became_active":"2024-02-23T00:39:16.857278+0000","last_became_peered":"2024-02-23T00:39:16.857278+0000","last_unstale":"2024-02-23T00:48:45.771289+0000","last_undegraded":"2024-02-23T00:48:45.771289+0000","last_fullsized":"2024-02-23T00:48:45.771289+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'34","last_scrub_stamp":"2024-02-23T00:47:18.765582+0000","last_deep_scrub":"203'34","last_deep_scrub_stamp":"2024-02-23T00:44:35.643931+0000","last_clean_scrub_stamp":"2024-02-23T00:47:18.765582+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":401,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.716785+0000","last_change":"2024-02-23T00:47:17.448466+0000","last_active":"2024-02-23T00:48:45.716785+0000","last_peered":"2024-02-23T00:48:45.716785+0000","last_clean":"2024-02-23T00:48:45.716785+0000","last_became_active":"2024-02-23T00:39:14.783662+0000","last_became_peered":"2024-02-23T00:39:14.783662+0000","last_unstale":"2024-02-23T00:48:45.716785+0000","last_undegraded":"2024-02-23T00:48:45.716785+0000","last_fullsized":"2024-02-23T00:48:45.716785+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:17.448376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:47:17.448376+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":422,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.381101+0000","last_change":"2024-02-23T00:48:13.821965+0000","last_active":"2024-02-23T00:48:45.381101+0000","last_peered":"2024-02-23T00:48:45.381101+0000","last_clean":"2024-02-23T00:48:45.381101+0000","last_became_active":"2024-02-23T00:47:37.184240+0000","last_became_peered":"2024-02-23T00:47:37.184240+0000","last_unstale":"2024-02-23T00:48:45.381101+0000","last_undegraded":"2024-02-23T00:48:45.381101+0000","last_fullsized":"2024-02-23T00:48:45.381101+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":385,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:13.821901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:48:13.821901+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":343,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.383978+0000","last_change":"2024-02-23T00:47:57.273372+0000","last_active":"2024-02-23T00:48:46.383978+0000","last_peered":"2024-02-23T00:48:46.383978+0000","last_clean":"2024-02-23T00:48:46.383978+0000","last_became_active":"2024-02-23T00:42:58.726652+0000","last_became_peered":"2024-02-23T00:42:58.726652+0000","last_unstale":"2024-02-23T00:48:46.383978+0000","last_undegraded":"2024-02-23T00:48:46.383978+0000","last_fullsized":"2024-02-23T00:48:46.383978+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:57.272902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:47:57.272902+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":398,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.386029+0000","last_change":"2024-02-23T00:46:37.333068+0000","last_active":"2024-02-23T00:48:46.386029+0000","last_peered":"2024-02-23T00:48:46.386029+0000","last_clean":"2024-02-23T00:48:46.386029+0000","last_became_active":"2024-02-23T00:39:18.800378+0000","last_became_peered":"2024-02-23T00:39:18.800378+0000","last_unstale":"2024-02-23T00:48:46.386029+0000","last_undegraded":"2024-02-23T00:48:46.386029+0000","last_fullsized":"2024-02-23T00:48:46.386029+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:37.332973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:59.694033+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.332973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"95'18","reported_seq":448,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.769173+0000","last_change":"2024-02-23T00:47:28.261515+0000","last_active":"2024-02-23T00:48:45.769173+0000","last_peered":"2024-02-23T00:48:45.769173+0000","last_clean":"2024-02-23T00:48:45.769173+0000","last_became_active":"2024-02-23T00:47:28.215366+0000","last_became_peered":"2024-02-23T00:47:28.215366+0000","last_unstale":"2024-02-23T00:48:45.769173+0000","last_undegraded":"2024-02-23T00:48:45.769173+0000","last_fullsized":"2024-02-23T00:48:45.769173+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'18","last_scrub_stamp":"2024-02-23T00:47:20.795884+0000","last_deep_scrub":"95'18","last_deep_scrub_stamp":"2024-02-23T00:42:01.195545+0000","last_clean_scrub_stamp":"2024-02-23T00:47:20.795884+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":406,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.715332+0000","last_change":"2024-02-23T00:47:24.444461+0000","last_active":"2024-02-23T00:48:45.715332+0000","last_peered":"2024-02-23T00:48:45.715332+0000","last_clean":"2024-02-23T00:48:45.715332+0000","last_became_active":"2024-02-23T00:39:14.784578+0000","last_became_peered":"2024-02-23T00:39:14.784578+0000","last_unstale":"2024-02-23T00:48:45.715332+0000","last_undegraded":"2024-02-23T00:48:45.715332+0000","last_fullsized":"2024-02-23T00:48:45.715332+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:24.444390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:47:24.444390+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":403,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.768347+0000","last_change":"2024-02-23T00:46:44.613082+0000","last_active":"2024-02-23T00:48:45.768347+0000","last_peered":"2024-02-23T00:48:45.768347+0000","last_clean":"2024-02-23T00:48:45.768347+0000","last_became_active":"2024-02-23T00:39:14.775446+0000","last_became_peered":"2024-02-23T00:39:14.775446+0000","last_unstale":"2024-02-23T00:48:45.768347+0000","last_undegraded":"2024-02-23T00:48:45.768347+0000","last_fullsized":"2024-02-23T00:48:45.768347+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:44.613012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:44.613012+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"67'1","reported_seq":426,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.383288+0000","last_change":"2024-02-23T00:47:57.665268+0000","last_active":"2024-02-23T00:48:45.383288+0000","last_peered":"2024-02-23T00:48:45.383288+0000","last_clean":"2024-02-23T00:48:45.383288+0000","last_became_active":"2024-02-23T00:47:56.187940+0000","last_became_peered":"2024-02-23T00:47:56.187940+0000","last_unstale":"2024-02-23T00:48:45.383288+0000","last_undegraded":"2024-02-23T00:48:45.383288+0000","last_fullsized":"2024-02-23T00:48:45.383288+0000","mapping_epoch":65,"log_start":"67'1","ondisk_log_start":"67'1","created":65,"last_epoch_clean":404,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:47:57.665156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:47:57.665156+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":347,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.261771+0000","last_change":"2024-02-23T00:48:15.593613+0000","last_active":"2024-02-23T00:48:46.261771+0000","last_peered":"2024-02-23T00:48:46.261771+0000","last_clean":"2024-02-23T00:48:46.261771+0000","last_became_active":"2024-02-23T00:42:58.690858+0000","last_became_peered":"2024-02-23T00:42:58.690858+0000","last_unstale":"2024-02-23T00:48:46.261771+0000","last_undegraded":"2024-02-23T00:48:46.261771+0000","last_fullsized":"2024-02-23T00:48:46.261771+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:15.593488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:35.360054+0000","last_clean_scrub_stamp":"2024-02-23T00:48:15.593488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":406,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.716501+0000","last_change":"2024-02-23T00:47:28.231576+0000","last_active":"2024-02-23T00:48:45.716501+0000","last_peered":"2024-02-23T00:48:45.716501+0000","last_clean":"2024-02-23T00:48:45.716501+0000","last_became_active":"2024-02-23T00:47:28.231152+0000","last_became_peered":"2024-02-23T00:47:28.231152+0000","last_unstale":"2024-02-23T00:48:45.716501+0000","last_undegraded":"2024-02-23T00:48:45.716501+0000","last_fullsized":"2024-02-23T00:48:45.716501+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:49.704984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:49.704984+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"67'14","reported_seq":425,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.262020+0000","last_change":"2024-02-23T00:48:01.585300+0000","last_active":"2024-02-23T00:48:46.262020+0000","last_peered":"2024-02-23T00:48:46.262020+0000","last_clean":"2024-02-23T00:48:46.262020+0000","last_became_active":"2024-02-23T00:39:16.858880+0000","last_became_peered":"2024-02-23T00:39:16.858880+0000","last_unstale":"2024-02-23T00:48:46.262020+0000","last_undegraded":"2024-02-23T00:48:46.262020+0000","last_fullsized":"2024-02-23T00:48:46.262020+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'14","last_scrub_stamp":"2024-02-23T00:48:01.585203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:48:01.585203+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":411,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.715451+0000","last_change":"2024-02-23T00:47:14.531194+0000","last_active":"2024-02-23T00:48:45.715451+0000","last_peered":"2024-02-23T00:48:45.715451+0000","last_clean":"2024-02-23T00:48:45.715451+0000","last_became_active":"2024-02-23T00:39:08.248249+0000","last_became_peered":"2024-02-23T00:39:08.248249+0000","last_unstale":"2024-02-23T00:48:45.715451+0000","last_undegraded":"2024-02-23T00:48:45.715451+0000","last_fullsized":"2024-02-23T00:48:45.715451+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:14.531130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:49.977248+0000","last_clean_scrub_stamp":"2024-02-23T00:47:14.531130+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":411,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.385127+0000","last_change":"2024-02-23T00:47:55.310719+0000","last_active":"2024-02-23T00:48:46.385127+0000","last_peered":"2024-02-23T00:48:46.385127+0000","last_clean":"2024-02-23T00:48:46.385127+0000","last_became_active":"2024-02-23T00:47:28.240482+0000","last_became_peered":"2024-02-23T00:47:28.240482+0000","last_unstale":"2024-02-23T00:48:46.385127+0000","last_undegraded":"2024-02-23T00:48:46.385127+0000","last_fullsized":"2024-02-23T00:48:46.385127+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:55.310652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:47:55.310652+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":404,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.376824+0000","last_change":"2024-02-23T00:47:54.034897+0000","last_active":"2024-02-23T00:48:45.376824+0000","last_peered":"2024-02-23T00:48:45.376824+0000","last_clean":"2024-02-23T00:48:45.376824+0000","last_became_active":"2024-02-23T00:39:14.783489+0000","last_became_peered":"2024-02-23T00:39:14.783489+0000","last_unstale":"2024-02-23T00:48:45.376824+0000","last_undegraded":"2024-02-23T00:48:45.376824+0000","last_fullsized":"2024-02-23T00:48:45.376824+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:54.034836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:47:54.034836+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":432,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.386118+0000","last_change":"2024-02-23T00:47:45.183506+0000","last_active":"2024-02-23T00:48:46.386118+0000","last_peered":"2024-02-23T00:48:46.386118+0000","last_clean":"2024-02-23T00:48:46.386118+0000","last_became_active":"2024-02-23T00:47:45.182550+0000","last_became_peered":"2024-02-23T00:47:45.182550+0000","last_unstale":"2024-02-23T00:48:46.386118+0000","last_undegraded":"2024-02-23T00:48:46.386118+0000","last_fullsized":"2024-02-23T00:48:46.386118+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:45.207275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:46:45.207275+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":353,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.716475+0000","last_change":"2024-02-23T00:47:28.233784+0000","last_active":"2024-02-23T00:48:45.716475+0000","last_peered":"2024-02-23T00:48:45.716475+0000","last_clean":"2024-02-23T00:48:45.716475+0000","last_became_active":"2024-02-23T00:47:28.233475+0000","last_became_peered":"2024-02-23T00:47:28.233475+0000","last_unstale":"2024-02-23T00:48:45.716475+0000","last_undegraded":"2024-02-23T00:48:45.716475+0000","last_fullsized":"2024-02-23T00:48:45.716475+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:22.448945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:47:22.448945+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":70,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.384539+0000","last_change":"2024-02-23T00:47:28.209462+0000","last_active":"2024-02-23T00:48:46.384539+0000","last_peered":"2024-02-23T00:48:46.384539+0000","last_clean":"2024-02-23T00:48:46.384539+0000","last_became_active":"2024-02-23T00:47:28.209125+0000","last_became_peered":"2024-02-23T00:47:28.209125+0000","last_unstale":"2024-02-23T00:48:46.384539+0000","last_undegraded":"2024-02-23T00:48:46.384539+0000","last_fullsized":"2024-02-23T00:48:46.384539+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:09.924384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.264240+0000","last_clean_scrub_stamp":"2024-02-23T00:46:09.924384+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"68'19","reported_seq":441,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.384873+0000","last_change":"2024-02-23T00:48:11.828616+0000","last_active":"2024-02-23T00:48:45.384873+0000","last_peered":"2024-02-23T00:48:45.384873+0000","last_clean":"2024-02-23T00:48:45.384873+0000","last_became_active":"2024-02-23T00:39:16.787484+0000","last_became_peered":"2024-02-23T00:39:16.787484+0000","last_unstale":"2024-02-23T00:48:45.384873+0000","last_undegraded":"2024-02-23T00:48:45.384873+0000","last_fullsized":"2024-02-23T00:48:45.384873+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'19","last_scrub_stamp":"2024-02-23T00:48:11.828547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:48:11.828547+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":71,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.377262+0000","last_change":"2024-02-23T00:47:28.210014+0000","last_active":"2024-02-23T00:48:45.377262+0000","last_peered":"2024-02-23T00:48:45.377262+0000","last_clean":"2024-02-23T00:48:45.377262+0000","last_became_active":"2024-02-23T00:47:28.209575+0000","last_became_peered":"2024-02-23T00:47:28.209575+0000","last_unstale":"2024-02-23T00:48:45.377262+0000","last_undegraded":"2024-02-23T00:48:45.377262+0000","last_fullsized":"2024-02-23T00:48:45.377262+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:10.878204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:10.878204+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"68'5","reported_seq":975,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.358923+0000","last_change":"2024-02-23T00:47:53.669608+0000","last_active":"2024-02-23T00:48:46.358923+0000","last_peered":"2024-02-23T00:48:46.358923+0000","last_clean":"2024-02-23T00:48:46.358923+0000","last_became_active":"2024-02-23T00:47:28.227928+0000","last_became_peered":"2024-02-23T00:47:28.227928+0000","last_unstale":"2024-02-23T00:48:46.358923+0000","last_undegraded":"2024-02-23T00:48:46.358923+0000","last_fullsized":"2024-02-23T00:48:46.358923+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'5","last_scrub_stamp":"2024-02-23T00:47:53.669533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:47:53.669533+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":562,"num_read_kb":557,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":431,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.716701+0000","last_change":"2024-02-23T00:47:52.552544+0000","last_active":"2024-02-23T00:48:45.716701+0000","last_peered":"2024-02-23T00:48:45.716701+0000","last_clean":"2024-02-23T00:48:45.716701+0000","last_became_active":"2024-02-23T00:47:52.552244+0000","last_became_peered":"2024-02-23T00:47:52.552244+0000","last_unstale":"2024-02-23T00:48:45.716701+0000","last_undegraded":"2024-02-23T00:48:45.716701+0000","last_fullsized":"2024-02-23T00:48:45.716701+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:36.725686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:46:36.725686+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":347,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.380883+0000","last_change":"2024-02-23T00:47:28.212866+0000","last_active":"2024-02-23T00:48:45.380883+0000","last_peered":"2024-02-23T00:48:45.380883+0000","last_clean":"2024-02-23T00:48:45.380883+0000","last_became_active":"2024-02-23T00:47:28.212516+0000","last_became_peered":"2024-02-23T00:47:28.212516+0000","last_unstale":"2024-02-23T00:48:45.380883+0000","last_undegraded":"2024-02-23T00:48:45.380883+0000","last_fullsized":"2024-02-23T00:48:45.380883+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:42.331547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:46:42.331547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":399,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.770073+0000","last_change":"2024-02-23T00:47:13.773330+0000","last_active":"2024-02-23T00:48:45.770073+0000","last_peered":"2024-02-23T00:48:45.770073+0000","last_clean":"2024-02-23T00:48:45.770073+0000","last_became_active":"2024-02-23T00:39:18.802134+0000","last_became_peered":"2024-02-23T00:39:18.802134+0000","last_unstale":"2024-02-23T00:48:45.770073+0000","last_undegraded":"2024-02-23T00:48:45.770073+0000","last_fullsized":"2024-02-23T00:48:45.770073+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:13.773262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:47:13.773262+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"376'36","reported_seq":531,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.383420+0000","last_change":"2024-02-23T00:48:12.364622+0000","last_active":"2024-02-23T00:48:46.383420+0000","last_peered":"2024-02-23T00:48:46.383420+0000","last_clean":"2024-02-23T00:48:46.383420+0000","last_became_active":"2024-02-23T00:47:28.240257+0000","last_became_peered":"2024-02-23T00:47:28.240257+0000","last_unstale":"2024-02-23T00:48:46.383420+0000","last_undegraded":"2024-02-23T00:48:46.383420+0000","last_fullsized":"2024-02-23T00:48:46.383420+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"376'36","last_scrub_stamp":"2024-02-23T00:48:12.364544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:48:12.364544+0000","log_size":36,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":402,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.383041+0000","last_change":"2024-02-23T00:47:17.212478+0000","last_active":"2024-02-23T00:48:46.383041+0000","last_peered":"2024-02-23T00:48:46.383041+0000","last_clean":"2024-02-23T00:48:46.383041+0000","last_became_active":"2024-02-23T00:39:14.784015+0000","last_became_peered":"2024-02-23T00:39:14.784015+0000","last_unstale":"2024-02-23T00:48:46.383041+0000","last_undegraded":"2024-02-23T00:48:46.383041+0000","last_fullsized":"2024-02-23T00:48:46.383041+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:17.212388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:47:17.212388+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":429,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.376015+0000","last_change":"2024-02-23T00:47:49.525743+0000","last_active":"2024-02-23T00:48:45.376015+0000","last_peered":"2024-02-23T00:48:45.376015+0000","last_clean":"2024-02-23T00:48:45.376015+0000","last_became_active":"2024-02-23T00:47:49.525421+0000","last_became_peered":"2024-02-23T00:47:49.525421+0000","last_unstale":"2024-02-23T00:48:45.376015+0000","last_undegraded":"2024-02-23T00:48:45.376015+0000","last_fullsized":"2024-02-23T00:48:45.376015+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":397,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:42.938209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:46:42.938209+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":71,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.381655+0000","last_change":"2024-02-23T00:47:28.232592+0000","last_active":"2024-02-23T00:48:45.381655+0000","last_peered":"2024-02-23T00:48:45.381655+0000","last_clean":"2024-02-23T00:48:45.381655+0000","last_became_active":"2024-02-23T00:47:28.232194+0000","last_became_peered":"2024-02-23T00:47:28.232194+0000","last_unstale":"2024-02-23T00:48:45.381655+0000","last_undegraded":"2024-02-23T00:48:45.381655+0000","last_fullsized":"2024-02-23T00:48:45.381655+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:21.604592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:47:21.604592+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3","version":"91'9","reported_seq":522,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.771616+0000","last_change":"2024-02-23T00:47:12.798383+0000","last_active":"2024-02-23T00:48:45.771616+0000","last_peered":"2024-02-23T00:48:45.771616+0000","last_clean":"2024-02-23T00:48:45.771616+0000","last_became_active":"2024-02-23T00:39:18.801297+0000","last_became_peered":"2024-02-23T00:39:18.801297+0000","last_unstale":"2024-02-23T00:48:45.771616+0000","last_undegraded":"2024-02-23T00:48:45.771616+0000","last_fullsized":"2024-02-23T00:48:45.771616+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'9","last_scrub_stamp":"2024-02-23T00:47:12.798314+0000","last_deep_scrub":"91'9","last_deep_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_clean_scrub_stamp":"2024-02-23T00:47:12.798314+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"68'16","reported_seq":424,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.382422+0000","last_change":"2024-02-23T00:45:11.713800+0000","last_active":"2024-02-23T00:48:45.382422+0000","last_peered":"2024-02-23T00:48:45.382422+0000","last_clean":"2024-02-23T00:48:45.382422+0000","last_became_active":"2024-02-23T00:39:16.787468+0000","last_became_peered":"2024-02-23T00:39:16.787468+0000","last_unstale":"2024-02-23T00:48:45.382422+0000","last_undegraded":"2024-02-23T00:48:45.382422+0000","last_fullsized":"2024-02-23T00:48:45.382422+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:45:11.713731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:45:11.713731+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":401,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.383031+0000","last_change":"2024-02-23T00:48:12.869003+0000","last_active":"2024-02-23T00:48:45.383031+0000","last_peered":"2024-02-23T00:48:45.383031+0000","last_clean":"2024-02-23T00:48:45.383031+0000","last_became_active":"2024-02-23T00:39:14.779952+0000","last_became_peered":"2024-02-23T00:39:14.779952+0000","last_unstale":"2024-02-23T00:48:45.383031+0000","last_undegraded":"2024-02-23T00:48:45.383031+0000","last_fullsized":"2024-02-23T00:48:45.383031+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:12.868930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:48:12.868930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"107'2","reported_seq":430,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.768222+0000","last_change":"2024-02-23T00:48:02.920168+0000","last_active":"2024-02-23T00:48:45.768222+0000","last_peered":"2024-02-23T00:48:45.768222+0000","last_clean":"2024-02-23T00:48:45.768222+0000","last_became_active":"2024-02-23T00:47:41.210074+0000","last_became_peered":"2024-02-23T00:47:41.210074+0000","last_unstale":"2024-02-23T00:48:45.768222+0000","last_undegraded":"2024-02-23T00:48:45.768222+0000","last_fullsized":"2024-02-23T00:48:45.768222+0000","mapping_epoch":375,"log_start":"107'2","ondisk_log_start":"107'2","created":65,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'2","last_scrub_stamp":"2024-02-23T00:48:02.920084+0000","last_deep_scrub":"107'2","last_deep_scrub_stamp":"2024-02-23T00:42:48.362739+0000","last_clean_scrub_stamp":"2024-02-23T00:48:02.920084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":351,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.263074+0000","last_change":"2024-02-23T00:47:28.242563+0000","last_active":"2024-02-23T00:48:46.263074+0000","last_peered":"2024-02-23T00:48:46.263074+0000","last_clean":"2024-02-23T00:48:46.263074+0000","last_became_active":"2024-02-23T00:47:28.242255+0000","last_became_peered":"2024-02-23T00:47:28.242255+0000","last_unstale":"2024-02-23T00:48:46.263074+0000","last_undegraded":"2024-02-23T00:48:46.263074+0000","last_fullsized":"2024-02-23T00:48:46.263074+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:10.415458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:47:10.415458+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":71,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.768691+0000","last_change":"2024-02-23T00:47:28.235122+0000","last_active":"2024-02-23T00:48:45.768691+0000","last_peered":"2024-02-23T00:48:45.768691+0000","last_clean":"2024-02-23T00:48:45.768691+0000","last_became_active":"2024-02-23T00:47:28.234034+0000","last_became_peered":"2024-02-23T00:47:28.234034+0000","last_unstale":"2024-02-23T00:48:45.768691+0000","last_undegraded":"2024-02-23T00:48:45.768691+0000","last_fullsized":"2024-02-23T00:48:45.768691+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:36.783823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_clean_scrub_stamp":"2024-02-23T00:46:36.783823+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"68'25","reported_seq":449,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.378921+0000","last_change":"2024-02-23T00:46:51.333974+0000","last_active":"2024-02-23T00:48:45.378921+0000","last_peered":"2024-02-23T00:48:45.378921+0000","last_clean":"2024-02-23T00:48:45.378921+0000","last_became_active":"2024-02-23T00:39:16.783926+0000","last_became_peered":"2024-02-23T00:39:16.783926+0000","last_unstale":"2024-02-23T00:48:45.378921+0000","last_undegraded":"2024-02-23T00:48:45.378921+0000","last_fullsized":"2024-02-23T00:48:45.378921+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'25","last_scrub_stamp":"2024-02-23T00:46:51.333896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:51.333896+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"95'4","reported_seq":417,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.381903+0000","last_change":"2024-02-23T00:47:27.346629+0000","last_active":"2024-02-23T00:48:45.381903+0000","last_peered":"2024-02-23T00:48:45.381903+0000","last_clean":"2024-02-23T00:48:45.381903+0000","last_became_active":"2024-02-23T00:39:14.779205+0000","last_became_peered":"2024-02-23T00:39:14.779205+0000","last_unstale":"2024-02-23T00:48:45.381903+0000","last_undegraded":"2024-02-23T00:48:45.381903+0000","last_fullsized":"2024-02-23T00:48:45.381903+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'4","last_scrub_stamp":"2024-02-23T00:47:27.346540+0000","last_deep_scrub":"95'4","last_deep_scrub_stamp":"2024-02-23T00:41:43.874397+0000","last_clean_scrub_stamp":"2024-02-23T00:47:27.346540+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":416,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.376837+0000","last_change":"2024-02-23T00:46:47.919416+0000","last_active":"2024-02-23T00:48:45.376837+0000","last_peered":"2024-02-23T00:48:45.376837+0000","last_clean":"2024-02-23T00:48:45.376837+0000","last_became_active":"2024-02-23T00:43:27.611660+0000","last_became_peered":"2024-02-23T00:43:27.611660+0000","last_unstale":"2024-02-23T00:48:45.376837+0000","last_undegraded":"2024-02-23T00:48:45.376837+0000","last_fullsized":"2024-02-23T00:48:45.376837+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:47.919353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:46:47.919353+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":353,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.717743+0000","last_change":"2024-02-23T00:47:28.232007+0000","last_active":"2024-02-23T00:48:45.717743+0000","last_peered":"2024-02-23T00:48:45.717743+0000","last_clean":"2024-02-23T00:48:45.717743+0000","last_became_active":"2024-02-23T00:47:28.231670+0000","last_became_peered":"2024-02-23T00:47:28.231670+0000","last_unstale":"2024-02-23T00:48:45.717743+0000","last_undegraded":"2024-02-23T00:48:45.717743+0000","last_fullsized":"2024-02-23T00:48:45.717743+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:19.484659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:47:19.484659+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"68'21","reported_seq":442,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.769225+0000","last_change":"2024-02-23T00:46:46.645234+0000","last_active":"2024-02-23T00:48:45.769225+0000","last_peered":"2024-02-23T00:48:45.769225+0000","last_clean":"2024-02-23T00:48:45.769225+0000","last_became_active":"2024-02-23T00:39:16.858760+0000","last_became_peered":"2024-02-23T00:39:16.858760+0000","last_unstale":"2024-02-23T00:48:45.769225+0000","last_undegraded":"2024-02-23T00:48:45.769225+0000","last_fullsized":"2024-02-23T00:48:45.769225+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'21","last_scrub_stamp":"2024-02-23T00:46:46.645153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:46.645153+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":71,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.381414+0000","last_change":"2024-02-23T00:47:28.223061+0000","last_active":"2024-02-23T00:48:45.381414+0000","last_peered":"2024-02-23T00:48:45.381414+0000","last_clean":"2024-02-23T00:48:45.381414+0000","last_became_active":"2024-02-23T00:47:28.219764+0000","last_became_peered":"2024-02-23T00:47:28.219764+0000","last_unstale":"2024-02-23T00:48:45.381414+0000","last_undegraded":"2024-02-23T00:48:45.381414+0000","last_fullsized":"2024-02-23T00:48:45.381414+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:22.376063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:45:22.376063+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":400,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.379244+0000","last_change":"2024-02-23T00:46:45.913635+0000","last_active":"2024-02-23T00:48:45.379244+0000","last_peered":"2024-02-23T00:48:45.379244+0000","last_clean":"2024-02-23T00:48:45.379244+0000","last_became_active":"2024-02-23T00:39:18.798624+0000","last_became_peered":"2024-02-23T00:39:18.798624+0000","last_unstale":"2024-02-23T00:48:45.379244+0000","last_undegraded":"2024-02-23T00:48:45.379244+0000","last_fullsized":"2024-02-23T00:48:45.379244+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:45.913526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:45.913526+0000","last_clean_scrub_stamp":"2024-02-23T00:46:45.913526+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":344,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.768445+0000","last_change":"2024-02-23T00:47:24.792535+0000","last_active":"2024-02-23T00:48:45.768445+0000","last_peered":"2024-02-23T00:48:45.768445+0000","last_clean":"2024-02-23T00:48:45.768445+0000","last_became_active":"2024-02-23T00:42:58.825015+0000","last_became_peered":"2024-02-23T00:42:58.825015+0000","last_unstale":"2024-02-23T00:48:45.768445+0000","last_undegraded":"2024-02-23T00:48:45.768445+0000","last_fullsized":"2024-02-23T00:48:45.768445+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:24.792464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:47:24.792464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"107'16","reported_seq":430,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.379190+0000","last_change":"2024-02-23T00:48:07.941205+0000","last_active":"2024-02-23T00:48:45.379190+0000","last_peered":"2024-02-23T00:48:45.379190+0000","last_clean":"2024-02-23T00:48:45.379190+0000","last_became_active":"2024-02-23T00:39:16.858449+0000","last_became_peered":"2024-02-23T00:39:16.858449+0000","last_unstale":"2024-02-23T00:48:45.379190+0000","last_undegraded":"2024-02-23T00:48:45.379190+0000","last_fullsized":"2024-02-23T00:48:45.379190+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'16","last_scrub_stamp":"2024-02-23T00:48:07.941101+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-02-23T00:48:07.941101+0000","last_clean_scrub_stamp":"2024-02-23T00:48:07.941101+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":400,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.262849+0000","last_change":"2024-02-23T00:47:11.419515+0000","last_active":"2024-02-23T00:48:46.262849+0000","last_peered":"2024-02-23T00:48:46.262849+0000","last_clean":"2024-02-23T00:48:46.262849+0000","last_became_active":"2024-02-23T00:39:14.783164+0000","last_became_peered":"2024-02-23T00:39:14.783164+0000","last_unstale":"2024-02-23T00:48:46.262849+0000","last_undegraded":"2024-02-23T00:48:46.262849+0000","last_fullsized":"2024-02-23T00:48:46.262849+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:11.419452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:47:11.419452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":398,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.384648+0000","last_change":"2024-02-23T00:46:42.258031+0000","last_active":"2024-02-23T00:48:46.384648+0000","last_peered":"2024-02-23T00:48:46.384648+0000","last_clean":"2024-02-23T00:48:46.384648+0000","last_became_active":"2024-02-23T00:39:18.800297+0000","last_became_peered":"2024-02-23T00:39:18.800297+0000","last_unstale":"2024-02-23T00:48:46.384648+0000","last_undegraded":"2024-02-23T00:48:46.384648+0000","last_fullsized":"2024-02-23T00:48:46.384648+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:42.257958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:42.257958+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1","version":"110'7","reported_seq":353,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.378381+0000","last_change":"2024-02-23T00:45:18.947859+0000","last_active":"2024-02-23T00:48:45.378381+0000","last_peered":"2024-02-23T00:48:45.378381+0000","last_clean":"2024-02-23T00:48:45.378381+0000","last_became_active":"2024-02-23T00:42:58.777251+0000","last_became_peered":"2024-02-23T00:42:58.777251+0000","last_unstale":"2024-02-23T00:48:45.378381+0000","last_undegraded":"2024-02-23T00:48:45.378381+0000","last_fullsized":"2024-02-23T00:48:45.378381+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'7","last_scrub_stamp":"2024-02-23T00:45:18.947746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:18.947746+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"68'22","reported_seq":85,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.377715+0000","last_change":"2024-02-23T00:47:28.280794+0000","last_active":"2024-02-23T00:48:45.377715+0000","last_peered":"2024-02-23T00:48:45.377715+0000","last_clean":"2024-02-23T00:48:45.377715+0000","last_became_active":"2024-02-23T00:47:28.237898+0000","last_became_peered":"2024-02-23T00:47:28.237898+0000","last_unstale":"2024-02-23T00:48:45.377715+0000","last_undegraded":"2024-02-23T00:48:45.377715+0000","last_fullsized":"2024-02-23T00:48:45.377715+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'22","last_scrub_stamp":"2024-02-23T00:47:16.731630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:47:16.731630+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":37,"num_read_kb":30,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":402,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.382950+0000","last_change":"2024-02-23T00:46:46.201157+0000","last_active":"2024-02-23T00:48:46.382950+0000","last_peered":"2024-02-23T00:48:46.382950+0000","last_clean":"2024-02-23T00:48:46.382950+0000","last_became_active":"2024-02-23T00:39:14.785590+0000","last_became_peered":"2024-02-23T00:39:14.785590+0000","last_unstale":"2024-02-23T00:48:46.382950+0000","last_undegraded":"2024-02-23T00:48:46.382950+0000","last_fullsized":"2024-02-23T00:48:46.382950+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:46.201063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_clean_scrub_stamp":"2024-02-23T00:46:46.201063+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"336'9","reported_seq":521,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.384382+0000","last_change":"2024-02-23T00:46:49.161082+0000","last_active":"2024-02-23T00:48:46.384382+0000","last_peered":"2024-02-23T00:48:46.384382+0000","last_clean":"2024-02-23T00:48:46.384382+0000","last_became_active":"2024-02-23T00:39:18.803566+0000","last_became_peered":"2024-02-23T00:39:18.803566+0000","last_unstale":"2024-02-23T00:48:46.384382+0000","last_undegraded":"2024-02-23T00:48:46.384382+0000","last_fullsized":"2024-02-23T00:48:46.384382+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"336'9","last_scrub_stamp":"2024-02-23T00:46:49.160991+0000","last_deep_scrub":"336'9","last_deep_scrub_stamp":"2024-02-23T00:46:49.160991+0000","last_clean_scrub_stamp":"2024-02-23T00:46:49.160991+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":344,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.715567+0000","last_change":"2024-02-23T00:48:41.688804+0000","last_active":"2024-02-23T00:48:45.715567+0000","last_peered":"2024-02-23T00:48:45.715567+0000","last_clean":"2024-02-23T00:48:45.715567+0000","last_became_active":"2024-02-23T00:42:58.757424+0000","last_became_peered":"2024-02-23T00:42:58.757424+0000","last_unstale":"2024-02-23T00:48:45.715567+0000","last_undegraded":"2024-02-23T00:48:45.715567+0000","last_fullsized":"2024-02-23T00:48:45.715567+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:41.688728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:41.688728+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"68'15","reported_seq":431,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.384158+0000","last_change":"2024-02-23T00:47:15.239499+0000","last_active":"2024-02-23T00:48:46.384158+0000","last_peered":"2024-02-23T00:48:46.384158+0000","last_clean":"2024-02-23T00:48:46.384158+0000","last_became_active":"2024-02-23T00:39:16.858953+0000","last_became_peered":"2024-02-23T00:39:16.858953+0000","last_unstale":"2024-02-23T00:48:46.384158+0000","last_undegraded":"2024-02-23T00:48:46.384158+0000","last_fullsized":"2024-02-23T00:48:46.384158+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:47:15.239429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:47:15.239429+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":403,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.771453+0000","last_change":"2024-02-23T00:46:42.692530+0000","last_active":"2024-02-23T00:48:45.771453+0000","last_peered":"2024-02-23T00:48:45.771453+0000","last_clean":"2024-02-23T00:48:45.771453+0000","last_became_active":"2024-02-23T00:39:14.773772+0000","last_became_peered":"2024-02-23T00:39:14.773772+0000","last_unstale":"2024-02-23T00:48:45.771453+0000","last_undegraded":"2024-02-23T00:48:45.771453+0000","last_fullsized":"2024-02-23T00:48:45.771453+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:42.692434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:42.692434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"376'9","reported_seq":530,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.378733+0000","last_change":"2024-02-23T00:47:28.269858+0000","last_active":"2024-02-23T00:48:45.378733+0000","last_peered":"2024-02-23T00:48:45.378733+0000","last_clean":"2024-02-23T00:48:45.378733+0000","last_became_active":"2024-02-23T00:47:28.229316+0000","last_became_peered":"2024-02-23T00:47:28.229316+0000","last_unstale":"2024-02-23T00:48:45.378733+0000","last_undegraded":"2024-02-23T00:48:45.378733+0000","last_fullsized":"2024-02-23T00:48:45.378733+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:46:36.900988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:36.900988+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":338,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.382343+0000","last_change":"2024-02-23T00:46:51.254983+0000","last_active":"2024-02-23T00:48:45.382343+0000","last_peered":"2024-02-23T00:48:45.382343+0000","last_clean":"2024-02-23T00:48:45.382343+0000","last_became_active":"2024-02-23T00:42:57.652488+0000","last_became_peered":"2024-02-23T00:42:57.652488+0000","last_unstale":"2024-02-23T00:48:45.382343+0000","last_undegraded":"2024-02-23T00:48:45.382343+0000","last_fullsized":"2024-02-23T00:48:45.382343+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:51.254903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:51.254903+0000","last_clean_scrub_stamp":"2024-02-23T00:46:51.254903+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"67'9","reported_seq":429,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.382337+0000","last_change":"2024-02-23T00:47:28.646843+0000","last_active":"2024-02-23T00:48:45.382337+0000","last_peered":"2024-02-23T00:48:45.382337+0000","last_clean":"2024-02-23T00:48:45.382337+0000","last_became_active":"2024-02-23T00:47:28.225585+0000","last_became_peered":"2024-02-23T00:47:28.225585+0000","last_unstale":"2024-02-23T00:48:45.382337+0000","last_undegraded":"2024-02-23T00:48:45.382337+0000","last_fullsized":"2024-02-23T00:48:45.382337+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'9","last_scrub_stamp":"2024-02-23T00:46:36.406771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:36.406771+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":405,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.262964+0000","last_change":"2024-02-23T00:46:49.393060+0000","last_active":"2024-02-23T00:48:46.262964+0000","last_peered":"2024-02-23T00:48:46.262964+0000","last_clean":"2024-02-23T00:48:46.262964+0000","last_became_active":"2024-02-23T00:39:14.778455+0000","last_became_peered":"2024-02-23T00:39:14.778455+0000","last_unstale":"2024-02-23T00:48:46.262964+0000","last_undegraded":"2024-02-23T00:48:46.262964+0000","last_fullsized":"2024-02-23T00:48:46.262964+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:49.392993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:49.392993+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":409,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.376766+0000","last_change":"2024-02-23T00:47:59.075510+0000","last_active":"2024-02-23T00:48:45.376766+0000","last_peered":"2024-02-23T00:48:45.376766+0000","last_clean":"2024-02-23T00:48:45.376766+0000","last_became_active":"2024-02-23T00:47:44.225379+0000","last_became_peered":"2024-02-23T00:47:44.225379+0000","last_unstale":"2024-02-23T00:48:45.376766+0000","last_undegraded":"2024-02-23T00:48:45.376766+0000","last_fullsized":"2024-02-23T00:48:45.376766+0000","mapping_epoch":391,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":392,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:59.075447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_clean_scrub_stamp":"2024-02-23T00:47:59.075447+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":344,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.770807+0000","last_change":"2024-02-23T00:47:19.760299+0000","last_active":"2024-02-23T00:48:45.770807+0000","last_peered":"2024-02-23T00:48:45.770807+0000","last_clean":"2024-02-23T00:48:45.770807+0000","last_became_active":"2024-02-23T00:42:58.835862+0000","last_became_peered":"2024-02-23T00:42:58.835862+0000","last_unstale":"2024-02-23T00:48:45.770807+0000","last_undegraded":"2024-02-23T00:48:45.770807+0000","last_fullsized":"2024-02-23T00:48:45.770807+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:19.760235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:47:19.760235+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"67'11","reported_seq":419,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.380705+0000","last_change":"2024-02-23T00:48:44.242641+0000","last_active":"2024-02-23T00:48:45.380705+0000","last_peered":"2024-02-23T00:48:45.380705+0000","last_clean":"2024-02-23T00:48:45.380705+0000","last_became_active":"2024-02-23T00:39:16.783896+0000","last_became_peered":"2024-02-23T00:39:16.783896+0000","last_unstale":"2024-02-23T00:48:45.380705+0000","last_undegraded":"2024-02-23T00:48:45.380705+0000","last_fullsized":"2024-02-23T00:48:45.380705+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'11","last_scrub_stamp":"2024-02-23T00:48:44.242556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:48:44.242556+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"202'2","reported_seq":409,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.383777+0000","last_change":"2024-02-23T00:48:04.832975+0000","last_active":"2024-02-23T00:48:45.383777+0000","last_peered":"2024-02-23T00:48:45.383777+0000","last_clean":"2024-02-23T00:48:45.383777+0000","last_became_active":"2024-02-23T00:39:14.775804+0000","last_became_peered":"2024-02-23T00:39:14.775804+0000","last_unstale":"2024-02-23T00:48:45.383777+0000","last_undegraded":"2024-02-23T00:48:45.383777+0000","last_fullsized":"2024-02-23T00:48:45.383777+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"202'2","last_scrub_stamp":"2024-02-23T00:48:04.832904+0000","last_deep_scrub":"202'2","last_deep_scrub_stamp":"2024-02-23T00:44:34.720640+0000","last_clean_scrub_stamp":"2024-02-23T00:48:04.832904+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":392,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.382380+0000","last_change":"2024-02-23T00:47:22.371575+0000","last_active":"2024-02-23T00:48:45.382380+0000","last_peered":"2024-02-23T00:48:45.382380+0000","last_clean":"2024-02-23T00:48:45.382380+0000","last_became_active":"2024-02-23T00:39:18.800382+0000","last_became_peered":"2024-02-23T00:39:18.800382+0000","last_unstale":"2024-02-23T00:48:45.382380+0000","last_undegraded":"2024-02-23T00:48:45.382380+0000","last_fullsized":"2024-02-23T00:48:45.382380+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:22.371493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:47:22.371493+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":347,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.383718+0000","last_change":"2024-02-23T00:47:28.225228+0000","last_active":"2024-02-23T00:48:45.383718+0000","last_peered":"2024-02-23T00:48:45.383718+0000","last_clean":"2024-02-23T00:48:45.383718+0000","last_became_active":"2024-02-23T00:47:28.224961+0000","last_became_peered":"2024-02-23T00:47:28.224961+0000","last_unstale":"2024-02-23T00:48:45.383718+0000","last_undegraded":"2024-02-23T00:48:45.383718+0000","last_fullsized":"2024-02-23T00:48:45.383718+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:49.243955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:49.243955+0000","last_clean_scrub_stamp":"2024-02-23T00:46:49.243955+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"67'4","reported_seq":408,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.377606+0000","last_change":"2024-02-23T00:48:14.033151+0000","last_active":"2024-02-23T00:48:45.377606+0000","last_peered":"2024-02-23T00:48:45.377606+0000","last_clean":"2024-02-23T00:48:45.377606+0000","last_became_active":"2024-02-23T00:39:16.785830+0000","last_became_peered":"2024-02-23T00:39:16.785830+0000","last_unstale":"2024-02-23T00:48:45.377606+0000","last_undegraded":"2024-02-23T00:48:45.377606+0000","last_fullsized":"2024-02-23T00:48:45.377606+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'4","last_scrub_stamp":"2024-02-23T00:48:14.033041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:48:14.033041+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":414,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.384432+0000","last_change":"2024-02-23T00:47:28.224795+0000","last_active":"2024-02-23T00:48:46.384432+0000","last_peered":"2024-02-23T00:48:46.384432+0000","last_clean":"2024-02-23T00:48:46.384432+0000","last_became_active":"2024-02-23T00:47:28.224477+0000","last_became_peered":"2024-02-23T00:47:28.224477+0000","last_unstale":"2024-02-23T00:48:46.384432+0000","last_undegraded":"2024-02-23T00:48:46.384432+0000","last_fullsized":"2024-02-23T00:48:46.384432+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:35.268491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:35.268491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":410,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.262623+0000","last_change":"2024-02-23T00:47:44.230641+0000","last_active":"2024-02-23T00:48:46.262623+0000","last_peered":"2024-02-23T00:48:46.262623+0000","last_clean":"2024-02-23T00:48:46.262623+0000","last_became_active":"2024-02-23T00:47:44.230295+0000","last_became_peered":"2024-02-23T00:47:44.230295+0000","last_unstale":"2024-02-23T00:48:46.262623+0000","last_undegraded":"2024-02-23T00:48:46.262623+0000","last_fullsized":"2024-02-23T00:48:46.262623+0000","mapping_epoch":391,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":392,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:12.411247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:47:12.411247+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":339,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.718401+0000","last_change":"2024-02-23T00:45:56.706901+0000","last_active":"2024-02-23T00:48:45.718401+0000","last_peered":"2024-02-23T00:48:45.718401+0000","last_clean":"2024-02-23T00:48:45.718401+0000","last_became_active":"2024-02-23T00:42:58.741365+0000","last_became_peered":"2024-02-23T00:42:58.741365+0000","last_unstale":"2024-02-23T00:48:45.718401+0000","last_undegraded":"2024-02-23T00:48:45.718401+0000","last_fullsized":"2024-02-23T00:48:45.718401+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:45:56.706792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:45:56.706792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"68'17","reported_seq":436,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.378568+0000","last_change":"2024-02-23T00:46:49.921614+0000","last_active":"2024-02-23T00:48:45.378568+0000","last_peered":"2024-02-23T00:48:45.378568+0000","last_clean":"2024-02-23T00:48:45.378568+0000","last_became_active":"2024-02-23T00:39:16.857232+0000","last_became_peered":"2024-02-23T00:39:16.857232+0000","last_unstale":"2024-02-23T00:48:45.378568+0000","last_undegraded":"2024-02-23T00:48:45.378568+0000","last_fullsized":"2024-02-23T00:48:45.378568+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'17","last_scrub_stamp":"2024-02-23T00:46:49.921543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:49.921543+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":409,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.262964+0000","last_change":"2024-02-23T00:47:28.240576+0000","last_active":"2024-02-23T00:48:46.262964+0000","last_peered":"2024-02-23T00:48:46.262964+0000","last_clean":"2024-02-23T00:48:46.262964+0000","last_became_active":"2024-02-23T00:47:28.240162+0000","last_became_peered":"2024-02-23T00:47:28.240162+0000","last_unstale":"2024-02-23T00:48:46.262964+0000","last_undegraded":"2024-02-23T00:48:46.262964+0000","last_fullsized":"2024-02-23T00:48:46.262964+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:44.285083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:44.285083+0000","last_clean_scrub_stamp":"2024-02-23T00:46:44.285083+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":53,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.375228+0000","last_change":"2024-02-23T00:47:44.226489+0000","last_active":"2024-02-23T00:48:45.375228+0000","last_peered":"2024-02-23T00:48:45.375228+0000","last_clean":"2024-02-23T00:48:45.375228+0000","last_became_active":"2024-02-23T00:47:44.226097+0000","last_became_peered":"2024-02-23T00:47:44.226097+0000","last_unstale":"2024-02-23T00:48:45.375228+0000","last_undegraded":"2024-02-23T00:48:45.375228+0000","last_fullsized":"2024-02-23T00:48:45.375228+0000","mapping_epoch":391,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":392,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:39.715957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:39.715957+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":76,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.386172+0000","last_change":"2024-02-23T00:48:03.345663+0000","last_active":"2024-02-23T00:48:46.386172+0000","last_peered":"2024-02-23T00:48:46.386172+0000","last_clean":"2024-02-23T00:48:46.386172+0000","last_became_active":"2024-02-23T00:47:28.200688+0000","last_became_peered":"2024-02-23T00:47:28.200688+0000","last_unstale":"2024-02-23T00:48:46.386172+0000","last_undegraded":"2024-02-23T00:48:46.386172+0000","last_fullsized":"2024-02-23T00:48:46.386172+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:03.345545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:03.345545+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"68'16","reported_seq":431,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.769463+0000","last_change":"2024-02-23T00:47:58.981341+0000","last_active":"2024-02-23T00:48:45.769463+0000","last_peered":"2024-02-23T00:48:45.769463+0000","last_clean":"2024-02-23T00:48:45.769463+0000","last_became_active":"2024-02-23T00:39:16.858945+0000","last_became_peered":"2024-02-23T00:39:16.858945+0000","last_unstale":"2024-02-23T00:48:45.769463+0000","last_undegraded":"2024-02-23T00:48:45.769463+0000","last_fullsized":"2024-02-23T00:48:45.769463+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:47:58.981233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:47:58.981233+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":409,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.382469+0000","last_change":"2024-02-23T00:46:09.088334+0000","last_active":"2024-02-23T00:48:46.382469+0000","last_peered":"2024-02-23T00:48:46.382469+0000","last_clean":"2024-02-23T00:48:46.382469+0000","last_became_active":"2024-02-23T00:39:14.785363+0000","last_became_peered":"2024-02-23T00:39:14.785363+0000","last_unstale":"2024-02-23T00:48:46.382469+0000","last_undegraded":"2024-02-23T00:48:46.382469+0000","last_fullsized":"2024-02-23T00:48:46.382469+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"80'2","last_scrub_stamp":"2024-02-23T00:46:09.088249+0000","last_deep_scrub":"80'2","last_deep_scrub_stamp":"2024-02-23T00:40:15.942725+0000","last_clean_scrub_stamp":"2024-02-23T00:46:09.088249+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"93'9","reported_seq":522,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.263132+0000","last_change":"2024-02-23T00:46:37.230102+0000","last_active":"2024-02-23T00:48:46.263132+0000","last_peered":"2024-02-23T00:48:46.263132+0000","last_clean":"2024-02-23T00:48:46.263132+0000","last_became_active":"2024-02-23T00:39:18.796680+0000","last_became_peered":"2024-02-23T00:39:18.796680+0000","last_unstale":"2024-02-23T00:48:46.263132+0000","last_undegraded":"2024-02-23T00:48:46.263132+0000","last_fullsized":"2024-02-23T00:48:46.263132+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-02-23T00:46:37.230030+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-02-23T00:43:50.393713+0000","last_clean_scrub_stamp":"2024-02-23T00:46:37.230030+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":353,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.768775+0000","last_change":"2024-02-23T00:48:17.898787+0000","last_active":"2024-02-23T00:48:45.768775+0000","last_peered":"2024-02-23T00:48:45.768775+0000","last_clean":"2024-02-23T00:48:45.768775+0000","last_became_active":"2024-02-23T00:47:28.234641+0000","last_became_peered":"2024-02-23T00:47:28.234641+0000","last_unstale":"2024-02-23T00:48:45.768775+0000","last_undegraded":"2024-02-23T00:48:45.768775+0000","last_fullsized":"2024-02-23T00:48:45.768775+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:17.898720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:17.898720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"93'23","reported_seq":448,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.263387+0000","last_change":"2024-02-23T00:48:45.295904+0000","last_active":"2024-02-23T00:48:46.263387+0000","last_peered":"2024-02-23T00:48:46.263387+0000","last_clean":"2024-02-23T00:48:46.263387+0000","last_became_active":"2024-02-23T00:39:16.782660+0000","last_became_peered":"2024-02-23T00:39:16.782660+0000","last_unstale":"2024-02-23T00:48:46.263387+0000","last_undegraded":"2024-02-23T00:48:46.263387+0000","last_fullsized":"2024-02-23T00:48:46.263387+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'23","last_scrub_stamp":"2024-02-23T00:48:45.295833+0000","last_deep_scrub":"93'23","last_deep_scrub_stamp":"2024-02-23T00:41:35.954516+0000","last_clean_scrub_stamp":"2024-02-23T00:48:45.295833+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":404,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.377143+0000","last_change":"2024-02-23T00:47:20.033447+0000","last_active":"2024-02-23T00:48:45.377143+0000","last_peered":"2024-02-23T00:48:45.377143+0000","last_clean":"2024-02-23T00:48:45.377143+0000","last_became_active":"2024-02-23T00:39:14.781560+0000","last_became_peered":"2024-02-23T00:39:14.781560+0000","last_unstale":"2024-02-23T00:48:45.377143+0000","last_undegraded":"2024-02-23T00:48:45.377143+0000","last_fullsized":"2024-02-23T00:48:45.377143+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:20.033380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:47:20.033380+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":408,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.769544+0000","last_change":"2024-02-23T00:47:28.233334+0000","last_active":"2024-02-23T00:48:45.769544+0000","last_peered":"2024-02-23T00:48:45.769544+0000","last_clean":"2024-02-23T00:48:45.769544+0000","last_became_active":"2024-02-23T00:47:28.232797+0000","last_became_peered":"2024-02-23T00:47:28.232797+0000","last_unstale":"2024-02-23T00:48:45.769544+0000","last_undegraded":"2024-02-23T00:48:45.769544+0000","last_fullsized":"2024-02-23T00:48:45.769544+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:36.592887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:53.189881+0000","last_clean_scrub_stamp":"2024-02-23T00:46:36.592887+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":345,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.377183+0000","last_change":"2024-02-23T00:48:05.979406+0000","last_active":"2024-02-23T00:48:45.377183+0000","last_peered":"2024-02-23T00:48:45.377183+0000","last_clean":"2024-02-23T00:48:45.377183+0000","last_became_active":"2024-02-23T00:42:58.774720+0000","last_became_peered":"2024-02-23T00:42:58.774720+0000","last_unstale":"2024-02-23T00:48:45.377183+0000","last_undegraded":"2024-02-23T00:48:45.377183+0000","last_fullsized":"2024-02-23T00:48:45.377183+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:05.979339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:05.979339+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"67'15","reported_seq":124,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.383125+0000","last_change":"2024-02-23T00:47:30.032276+0000","last_active":"2024-02-23T00:48:45.383125+0000","last_peered":"2024-02-23T00:48:45.383125+0000","last_clean":"2024-02-23T00:48:45.383125+0000","last_became_active":"2024-02-23T00:47:28.211342+0000","last_became_peered":"2024-02-23T00:47:28.211342+0000","last_unstale":"2024-02-23T00:48:45.383125+0000","last_undegraded":"2024-02-23T00:48:45.383125+0000","last_fullsized":"2024-02-23T00:48:45.383125+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:46:03.951524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:03.951524+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":405,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.263345+0000","last_change":"2024-02-23T00:48:10.563771+0000","last_active":"2024-02-23T00:48:46.263345+0000","last_peered":"2024-02-23T00:48:46.263345+0000","last_clean":"2024-02-23T00:48:46.263345+0000","last_became_active":"2024-02-23T00:39:14.782494+0000","last_became_peered":"2024-02-23T00:39:14.782494+0000","last_unstale":"2024-02-23T00:48:46.263345+0000","last_undegraded":"2024-02-23T00:48:46.263345+0000","last_fullsized":"2024-02-23T00:48:46.263345+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:10.563695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:48:10.563695+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":397,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.383892+0000","last_change":"2024-02-23T00:48:16.788875+0000","last_active":"2024-02-23T00:48:45.383892+0000","last_peered":"2024-02-23T00:48:45.383892+0000","last_clean":"2024-02-23T00:48:45.383892+0000","last_became_active":"2024-02-23T00:39:18.801039+0000","last_became_peered":"2024-02-23T00:39:18.801039+0000","last_unstale":"2024-02-23T00:48:45.383892+0000","last_undegraded":"2024-02-23T00:48:45.383892+0000","last_fullsized":"2024-02-23T00:48:45.383892+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:16.788806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.641689+0000","last_clean_scrub_stamp":"2024-02-23T00:48:16.788806+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":345,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.377223+0000","last_change":"2024-02-23T00:48:10.933441+0000","last_active":"2024-02-23T00:48:45.377223+0000","last_peered":"2024-02-23T00:48:45.377223+0000","last_clean":"2024-02-23T00:48:45.377223+0000","last_became_active":"2024-02-23T00:42:58.770759+0000","last_became_peered":"2024-02-23T00:42:58.770759+0000","last_unstale":"2024-02-23T00:48:45.377223+0000","last_undegraded":"2024-02-23T00:48:45.377223+0000","last_fullsized":"2024-02-23T00:48:45.377223+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:10.933375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:10.933375+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"68'18","reported_seq":440,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.378627+0000","last_change":"2024-02-23T00:46:44.911346+0000","last_active":"2024-02-23T00:48:45.378627+0000","last_peered":"2024-02-23T00:48:45.378627+0000","last_clean":"2024-02-23T00:48:45.378627+0000","last_became_active":"2024-02-23T00:39:16.782803+0000","last_became_peered":"2024-02-23T00:39:16.782803+0000","last_unstale":"2024-02-23T00:48:45.378627+0000","last_undegraded":"2024-02-23T00:48:45.378627+0000","last_fullsized":"2024-02-23T00:48:45.378627+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'18","last_scrub_stamp":"2024-02-23T00:46:44.911278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:46:44.911278+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":404,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.376699+0000","last_change":"2024-02-23T00:47:16.017278+0000","last_active":"2024-02-23T00:48:45.376699+0000","last_peered":"2024-02-23T00:48:45.376699+0000","last_clean":"2024-02-23T00:48:45.376699+0000","last_became_active":"2024-02-23T00:39:14.782163+0000","last_became_peered":"2024-02-23T00:39:14.782163+0000","last_unstale":"2024-02-23T00:48:45.376699+0000","last_undegraded":"2024-02-23T00:48:45.376699+0000","last_fullsized":"2024-02-23T00:48:45.376699+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:16.017210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:47:16.017210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":407,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.379750+0000","last_change":"2024-02-23T00:48:03.533673+0000","last_active":"2024-02-23T00:48:45.379750+0000","last_peered":"2024-02-23T00:48:45.379750+0000","last_clean":"2024-02-23T00:48:45.379750+0000","last_became_active":"2024-02-23T00:47:28.215452+0000","last_became_peered":"2024-02-23T00:47:28.215452+0000","last_unstale":"2024-02-23T00:48:45.379750+0000","last_undegraded":"2024-02-23T00:48:45.379750+0000","last_fullsized":"2024-02-23T00:48:45.379750+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:03.533561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_clean_scrub_stamp":"2024-02-23T00:48:03.533561+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":347,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.385389+0000","last_change":"2024-02-23T00:47:28.204445+0000","last_active":"2024-02-23T00:48:46.385389+0000","last_peered":"2024-02-23T00:48:46.385389+0000","last_clean":"2024-02-23T00:48:46.385389+0000","last_became_active":"2024-02-23T00:47:28.203555+0000","last_became_peered":"2024-02-23T00:47:28.203555+0000","last_unstale":"2024-02-23T00:48:46.385389+0000","last_undegraded":"2024-02-23T00:48:46.385389+0000","last_fullsized":"2024-02-23T00:48:46.385389+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:00.943545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:46:00.943545+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"95'27","reported_seq":454,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.381307+0000","last_change":"2024-02-23T00:47:14.328310+0000","last_active":"2024-02-23T00:48:45.381307+0000","last_peered":"2024-02-23T00:48:45.381307+0000","last_clean":"2024-02-23T00:48:45.381307+0000","last_became_active":"2024-02-23T00:39:16.786791+0000","last_became_peered":"2024-02-23T00:39:16.786791+0000","last_unstale":"2024-02-23T00:48:45.381307+0000","last_undegraded":"2024-02-23T00:48:45.381307+0000","last_fullsized":"2024-02-23T00:48:45.381307+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'27","last_scrub_stamp":"2024-02-23T00:47:14.328242+0000","last_deep_scrub":"95'27","last_deep_scrub_stamp":"2024-02-23T00:41:55.919895+0000","last_clean_scrub_stamp":"2024-02-23T00:47:14.328242+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":410,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.716397+0000","last_change":"2024-02-23T00:47:28.209244+0000","last_active":"2024-02-23T00:48:45.716397+0000","last_peered":"2024-02-23T00:48:45.716397+0000","last_clean":"2024-02-23T00:48:45.716397+0000","last_became_active":"2024-02-23T00:47:28.208293+0000","last_became_peered":"2024-02-23T00:47:28.208293+0000","last_unstale":"2024-02-23T00:48:45.716397+0000","last_undegraded":"2024-02-23T00:48:45.716397+0000","last_fullsized":"2024-02-23T00:48:45.716397+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:47.689132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:47.689132+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":398,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.385225+0000","last_change":"2024-02-23T00:47:22.313110+0000","last_active":"2024-02-23T00:48:46.385225+0000","last_peered":"2024-02-23T00:48:46.385225+0000","last_clean":"2024-02-23T00:48:46.385225+0000","last_became_active":"2024-02-23T00:39:18.796431+0000","last_became_peered":"2024-02-23T00:39:18.796431+0000","last_unstale":"2024-02-23T00:48:46.385225+0000","last_undegraded":"2024-02-23T00:48:46.385225+0000","last_fullsized":"2024-02-23T00:48:46.385225+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:22.313004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:47:22.313004+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":343,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.384084+0000","last_change":"2024-02-23T00:48:16.383542+0000","last_active":"2024-02-23T00:48:46.384084+0000","last_peered":"2024-02-23T00:48:46.384084+0000","last_clean":"2024-02-23T00:48:46.384084+0000","last_became_active":"2024-02-23T00:42:58.832885+0000","last_became_peered":"2024-02-23T00:42:58.832885+0000","last_unstale":"2024-02-23T00:48:46.384084+0000","last_undegraded":"2024-02-23T00:48:46.384084+0000","last_fullsized":"2024-02-23T00:48:46.384084+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:16.383463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:16.383463+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"82'15","reported_seq":429,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.380850+0000","last_change":"2024-02-23T00:47:25.380130+0000","last_active":"2024-02-23T00:48:45.380850+0000","last_peered":"2024-02-23T00:48:45.380850+0000","last_clean":"2024-02-23T00:48:45.380850+0000","last_became_active":"2024-02-23T00:39:16.782762+0000","last_became_peered":"2024-02-23T00:39:16.782762+0000","last_unstale":"2024-02-23T00:48:45.380850+0000","last_undegraded":"2024-02-23T00:48:45.380850+0000","last_fullsized":"2024-02-23T00:48:45.380850+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'15","last_scrub_stamp":"2024-02-23T00:47:25.380060+0000","last_deep_scrub":"82'15","last_deep_scrub_stamp":"2024-02-23T00:41:50.975683+0000","last_clean_scrub_stamp":"2024-02-23T00:47:25.380060+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":401,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.716530+0000","last_change":"2024-02-23T00:46:08.794420+0000","last_active":"2024-02-23T00:48:45.716530+0000","last_peered":"2024-02-23T00:48:45.716530+0000","last_clean":"2024-02-23T00:48:45.716530+0000","last_became_active":"2024-02-23T00:39:14.783469+0000","last_became_peered":"2024-02-23T00:39:14.783469+0000","last_unstale":"2024-02-23T00:48:45.716530+0000","last_undegraded":"2024-02-23T00:48:45.716530+0000","last_fullsized":"2024-02-23T00:48:45.716530+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:08.794352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:46:08.794352+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":399,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.715514+0000","last_change":"2024-02-23T00:46:09.832330+0000","last_active":"2024-02-23T00:48:45.715514+0000","last_peered":"2024-02-23T00:48:45.715514+0000","last_clean":"2024-02-23T00:48:45.715514+0000","last_became_active":"2024-02-23T00:39:18.796952+0000","last_became_peered":"2024-02-23T00:39:18.796952+0000","last_unstale":"2024-02-23T00:48:45.715514+0000","last_undegraded":"2024-02-23T00:48:45.715514+0000","last_fullsized":"2024-02-23T00:48:45.715514+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:09.832253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:09.832253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":346,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.380322+0000","last_change":"2024-02-23T00:48:45.283304+0000","last_active":"2024-02-23T00:48:45.380322+0000","last_peered":"2024-02-23T00:48:45.380322+0000","last_clean":"2024-02-23T00:48:45.380322+0000","last_became_active":"2024-02-23T00:42:58.758147+0000","last_became_peered":"2024-02-23T00:42:58.758147+0000","last_unstale":"2024-02-23T00:48:45.380322+0000","last_undegraded":"2024-02-23T00:48:45.380322+0000","last_fullsized":"2024-02-23T00:48:45.380322+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:45.283230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:23.224663+0000","last_clean_scrub_stamp":"2024-02-23T00:48:45.283230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"68'15","reported_seq":431,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.378778+0000","last_change":"2024-02-23T00:47:19.331746+0000","last_active":"2024-02-23T00:48:45.378778+0000","last_peered":"2024-02-23T00:48:45.378778+0000","last_clean":"2024-02-23T00:48:45.378778+0000","last_became_active":"2024-02-23T00:39:16.784606+0000","last_became_peered":"2024-02-23T00:39:16.784606+0000","last_unstale":"2024-02-23T00:48:45.378778+0000","last_undegraded":"2024-02-23T00:48:45.378778+0000","last_fullsized":"2024-02-23T00:48:45.378778+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'15","last_scrub_stamp":"2024-02-23T00:47:19.331675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:47:19.331675+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":404,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.375932+0000","last_change":"2024-02-23T00:47:24.067493+0000","last_active":"2024-02-23T00:48:45.375932+0000","last_peered":"2024-02-23T00:48:45.375932+0000","last_clean":"2024-02-23T00:48:45.375932+0000","last_became_active":"2024-02-23T00:39:14.785362+0000","last_became_peered":"2024-02-23T00:39:14.785362+0000","last_unstale":"2024-02-23T00:48:45.375932+0000","last_undegraded":"2024-02-23T00:48:45.375932+0000","last_fullsized":"2024-02-23T00:48:45.375932+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:24.067421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:47:24.067421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":392,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.381006+0000","last_change":"2024-02-23T00:46:47.226397+0000","last_active":"2024-02-23T00:48:45.381006+0000","last_peered":"2024-02-23T00:48:45.381006+0000","last_clean":"2024-02-23T00:48:45.381006+0000","last_became_active":"2024-02-23T00:39:18.796312+0000","last_became_peered":"2024-02-23T00:39:18.796312+0000","last_unstale":"2024-02-23T00:48:45.381006+0000","last_undegraded":"2024-02-23T00:48:45.381006+0000","last_fullsized":"2024-02-23T00:48:45.381006+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:47.226331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:47.226331+0000","last_clean_scrub_stamp":"2024-02-23T00:46:47.226331+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":344,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.769313+0000","last_change":"2024-02-23T00:47:54.954594+0000","last_active":"2024-02-23T00:48:45.769313+0000","last_peered":"2024-02-23T00:48:45.769313+0000","last_clean":"2024-02-23T00:48:45.769313+0000","last_became_active":"2024-02-23T00:42:58.838257+0000","last_became_peered":"2024-02-23T00:42:58.838257+0000","last_unstale":"2024-02-23T00:48:45.769313+0000","last_undegraded":"2024-02-23T00:48:45.769313+0000","last_fullsized":"2024-02-23T00:48:45.769313+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:54.954501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:47:54.954501+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"88'50","reported_seq":531,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.378645+0000","last_change":"2024-02-23T00:48:03.023256+0000","last_active":"2024-02-23T00:48:45.378645+0000","last_peered":"2024-02-23T00:48:45.378645+0000","last_clean":"2024-02-23T00:48:45.378645+0000","last_became_active":"2024-02-23T00:47:28.205150+0000","last_became_peered":"2024-02-23T00:47:28.205150+0000","last_unstale":"2024-02-23T00:48:45.378645+0000","last_undegraded":"2024-02-23T00:48:45.378645+0000","last_fullsized":"2024-02-23T00:48:45.378645+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'50","last_scrub_stamp":"2024-02-23T00:48:03.023069+0000","last_deep_scrub":"88'50","last_deep_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_clean_scrub_stamp":"2024-02-23T00:48:03.023069+0000","log_size":50,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":403,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.771626+0000","last_change":"2024-02-23T00:48:10.850211+0000","last_active":"2024-02-23T00:48:45.771626+0000","last_peered":"2024-02-23T00:48:45.771626+0000","last_clean":"2024-02-23T00:48:45.771626+0000","last_became_active":"2024-02-23T00:39:14.782000+0000","last_became_peered":"2024-02-23T00:39:14.782000+0000","last_unstale":"2024-02-23T00:48:45.771626+0000","last_undegraded":"2024-02-23T00:48:45.771626+0000","last_fullsized":"2024-02-23T00:48:45.771626+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:10.850134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_clean_scrub_stamp":"2024-02-23T00:48:10.850134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":393,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:45.378902+0000","last_change":"2024-02-23T00:46:49.271764+0000","last_active":"2024-02-23T00:48:45.378902+0000","last_peered":"2024-02-23T00:48:45.378902+0000","last_clean":"2024-02-23T00:48:45.378902+0000","last_became_active":"2024-02-23T00:39:18.799723+0000","last_became_peered":"2024-02-23T00:39:18.799723+0000","last_unstale":"2024-02-23T00:48:45.378902+0000","last_undegraded":"2024-02-23T00:48:45.378902+0000","last_fullsized":"2024-02-23T00:48:45.378902+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:46:49.271705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:46:49.271705+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":343,"reported_epoch":434,"state":"active+clean","last_fresh":"2024-02-23T00:48:46.385336+0000","last_change":"2024-02-23T00:47:19.231154+0000","last_active":"2024-02-23T00:48:46.385336+0000","last_peered":"2024-02-23T00:48:46.385336+0000","last_clean":"2024-02-23T00:48:46.385336+0000","last_became_active":"2024-02-23T00:42:58.824573+0000","last_became_peered":"2024-02-23T00:42:58.824573+0000","last_unstale":"2024-02-23T00:48:46.385336+0000","last_undegraded":"2024-02-23T00:48:46.385336+0000","last_fullsized":"2024-02-23T00:48:46.385336+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:47:19.231077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:47:19.231077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0],"acting":[5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":213,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":7},{"poolid":62,"num_pg":32,"stat_sum":{"num_bytes":4001479,"num_objects":158,"num_object_clones":2,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":107,"num_read_kb":9,"num_write":596,"num_write_kb":4495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":853,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":7,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":13926400,"data_stored":12004505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1885,"ondisk_log_size":1885,"up":96,"acting":96,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":12,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":36,"acting":36,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1079,"num_read_kb":804,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":1486,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":666,"ondisk_log_size":666,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":562,"num_read_kb":557,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234102,"num_pgs":100,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":316736,"kb_used_data":11312,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93428416,"statfs":{"total":95995035648,"available":95670697984,"internally_reserved":0,"allocated":11583488,"data_stored":8491446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:48:29 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54500000000000004,"15min":0.54200000000000004},"min":{"1min":0.38100000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.75800000000000001,"5min":4.9420000000000002,"15min":4.9420000000000002},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54000000000000004,"15min":0.51800000000000002},"min":{"1min":0.32000000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.151,"5min":4.9729999999999999,"15min":4.9729999999999999},"last":0.55300000000000005}]},{"osd":1,"last update":"Fri Feb 23 00:48:29 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.78500000000000003,"15min":0.623},"min":{"1min":0.37,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.25,"5min":10.786,"15min":10.786},"last":0.62},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.78800000000000003,"15min":0.625},"min":{"1min":0.40500000000000003,"5min":0.29399999999999998,"15min":0.28100000000000003},"max":{"1min":1.107,"5min":11.885999999999999,"15min":11.885999999999999},"last":0.51900000000000002}]},{"osd":2,"last update":"Fri Feb 23 00:48:29 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.60099999999999998,"15min":0.55500000000000005},"min":{"1min":0.38600000000000001,"5min":0.315,"15min":0.28999999999999998},"max":{"1min":0.73499999999999999,"5min":5.0789999999999997,"15min":5.0789999999999997},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58099999999999996,"15min":0.55200000000000005},"min":{"1min":0.46400000000000002,"5min":0.316,"15min":0.316},"max":{"1min":1.1180000000000001,"5min":5.1280000000000001,"15min":5.1280000000000001},"last":0.54200000000000004}]},{"osd":3,"last update":"Fri Feb 23 00:48:29 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.79400000000000004,"15min":0.62},"min":{"1min":0.41799999999999998,"5min":0.32200000000000001,"15min":0.32100000000000001},"max":{"1min":1.095,"5min":15.712999999999999,"15min":15.712999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.71099999999999997,"15min":0.61899999999999999},"min":{"1min":0.44900000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.72499999999999998,"5min":9.4090000000000007,"15min":9.4090000000000007},"last":0.58299999999999996}]},{"osd":4,"last update":"Fri Feb 23 00:48:29 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.45400000000000001,"15min":0.441},"min":{"1min":0.28899999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.63100000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.55200000000000005,"15min":0.49099999999999999},"min":{"1min":0.312,"5min":0.248,"15min":0.248},"max":{"1min":0.76500000000000001,"5min":11.236000000000001,"15min":11.236000000000001},"last":0.45600000000000002}]},{"osd":5,"last update":"Fri Feb 23 00:48:29 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.51100000000000001,"15min":0.49299999999999999},"min":{"1min":0.36199999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.68600000000000005,"5min":5.4000000000000004,"15min":5.4000000000000004},"last":0.434},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.50800000000000001,"15min":0.497},"min":{"1min":0.373,"5min":0.29099999999999998,"15min":0.26700000000000002},"max":{"1min":0.71899999999999997,"5min":1.728,"15min":1.728},"last":0.443}]},{"osd":6,"last update":"Fri Feb 23 00:48:29 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.51000000000000001,"15min":0.51200000000000001},"min":{"1min":0.33700000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.72599999999999998,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.625,"15min":0.54900000000000004},"min":{"1min":0.35199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.67000000000000004,"5min":13.395,"15min":13.395},"last":0.52900000000000003}]}]},{"osd":6,"up_from":47,"seq":201863463037,"num_pgs":83,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308588,"kb_used_data":7268,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93436564,"statfs":{"total":95995035648,"available":95679041536,"internally_reserved":0,"allocated":7442432,"data_stored":4472189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:47:59 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.95499999999999996,"15min":0.69399999999999995},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.86099999999999999,"5min":30.641999999999999,"15min":30.641999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.99399999999999999,"15min":0.70999999999999996},"min":{"1min":0.27200000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":3.7360000000000002,"5min":29.292000000000002,"15min":29.292000000000002},"last":0.35399999999999998}]},{"osd":1,"last update":"Fri Feb 23 00:47:59 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.97399999999999998,"15min":0.69099999999999995},"min":{"1min":0.25900000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.77200000000000002,"5min":28.186,"15min":28.186},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":1.46,"15min":0.86899999999999999},"min":{"1min":0.316,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.76200000000000001,"5min":33.104999999999997,"15min":33.104999999999997},"last":0.39000000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:47:59 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.997,"15min":0.72399999999999998},"min":{"1min":0.29399999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.69699999999999995,"5min":19.960000000000001,"15min":19.960000000000001},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.81000000000000005,"15min":0.65200000000000002},"min":{"1min":0.316,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.665,"5min":8.5860000000000003,"15min":8.5860000000000003},"last":0.40699999999999997}]},{"osd":3,"last update":"Fri Feb 23 00:47:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.73799999999999999,"15min":0.63200000000000001},"min":{"1min":0.36099999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78400000000000003,"5min":19.562000000000001,"15min":19.562000000000001},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.84099999999999997,"15min":0.67900000000000005},"min":{"1min":0.34999999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.6259999999999999,"5min":14.199,"15min":14.199},"last":0.432}]},{"osd":4,"last update":"Fri Feb 23 00:47:59 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.58099999999999996,"15min":0.47299999999999998},"min":{"1min":0.249,"5min":0.218,"15min":0.218},"max":{"1min":1.6220000000000001,"5min":11.109999999999999,"15min":11.109999999999999},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.73999999999999999,"15min":0.54100000000000004},"min":{"1min":0.308,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.83099999999999996,"5min":16.734000000000002,"15min":16.734000000000002},"last":0.42099999999999999}]},{"osd":5,"last update":"Fri Feb 23 00:47:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.64300000000000002,"15min":0.5},"min":{"1min":0.32100000000000001,"5min":0.249,"15min":0.249},"max":{"1min":1.784,"5min":14.154999999999999,"15min":14.154999999999999},"last":0.372},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.70499999999999996,"15min":0.54000000000000004},"min":{"1min":0.33900000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":8.1400000000000006,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.45000000000000001}]},{"osd":7,"last update":"Fri Feb 23 00:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51600000000000001,"15min":0.52800000000000002},"min":{"1min":0.29399999999999998,"5min":0.29299999999999998,"15min":0.26800000000000002},"max":{"1min":0.67600000000000005,"5min":2.6949999999999998,"15min":8.548},"last":0.443},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56000000000000005,"15min":0.60099999999999998},"min":{"1min":0.40300000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.70999999999999996,"5min":4.6680000000000001,"15min":15.618},"last":0.47499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574999,"num_pgs":97,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308472,"kb_used_data":11240,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93436680,"statfs":{"total":95995035648,"available":95679160320,"internally_reserved":0,"allocated":11509760,"data_stored":8493013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:47:45 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":1.258,"15min":0.66300000000000003},"min":{"1min":0.224,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.49299999999999999,"5min":78.361000000000004,"15min":78.361000000000004},"last":0.318},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.52700000000000002,"15min":0.438},"min":{"1min":0.216,"5min":0.216,"15min":0.191},"max":{"1min":0.48799999999999999,"5min":5.8849999999999998,"15min":5.8849999999999998},"last":0.41499999999999998}]},{"osd":2,"last update":"Fri Feb 23 00:48:11 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":1.145,"15min":0.64900000000000002},"min":{"1min":0.23000000000000001,"5min":0.214,"15min":0.188},"max":{"1min":0.71399999999999997,"5min":48.606000000000002,"15min":48.606000000000002},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.84899999999999998,"15min":0.58199999999999996},"min":{"1min":0.27900000000000003,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.80500000000000005,"5min":23.911000000000001,"15min":23.911000000000001},"last":0.34899999999999998}]},{"osd":3,"last update":"Fri Feb 23 00:47:45 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.85099999999999998,"15min":0.60999999999999999},"min":{"1min":0.28899999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.61699999999999999,"5min":35.244999999999997,"15min":35.244999999999997},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":1.1040000000000001,"15min":0.67300000000000004},"min":{"1min":0.28999999999999998,"5min":0.253,"15min":0.248},"max":{"1min":0.54800000000000004,"5min":56.853999999999999,"15min":56.853999999999999},"last":0.378}]},{"osd":4,"last update":"Fri Feb 23 00:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":1.111,"15min":0.80900000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.29299999999999998},"max":{"1min":0.94199999999999995,"5min":37.646999999999998,"15min":37.646999999999998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.79700000000000004,"15min":0.69399999999999995},"min":{"1min":0.37,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.002,"5min":9.6859999999999999,"15min":9.6859999999999999},"last":0.64000000000000001}]},{"osd":5,"last update":"Fri Feb 23 00:48:27 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63,"15min":0.82499999999999996},"min":{"1min":0.33000000000000002,"5min":0.32700000000000001,"15min":0.32600000000000001},"max":{"1min":0.91600000000000004,"5min":5.9500000000000002,"15min":48.792000000000002},"last":0.755},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.68000000000000005,"15min":0.72999999999999998},"min":{"1min":0.44700000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83999999999999997,"5min":3.593,"15min":11.872999999999999},"last":0.69199999999999995}]},{"osd":6,"last update":"Fri Feb 23 00:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":1.1419999999999999,"15min":0.81000000000000005},"min":{"1min":0.38100000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.872,"5min":40.741,"15min":40.741},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.71699999999999997,"15min":0.68100000000000005},"min":{"1min":0.38400000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78500000000000003,"5min":6.0069999999999997,"15min":6.0069999999999997},"last":0.65100000000000002}]},{"osd":7,"last update":"Fri Feb 23 00:48:26 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.68300000000000005,"15min":0.71399999999999997},"min":{"1min":0.45000000000000001,"5min":0.39300000000000002,"15min":0.36699999999999999},"max":{"1min":0.92700000000000005,"5min":3.6230000000000002,"15min":20.225000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.70099999999999996,"15min":0.72299999999999998},"min":{"1min":0.49199999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0369999999999999,"5min":5.9169999999999998,"15min":20.254999999999999},"last":0.66800000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738525,"num_pgs":90,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308616,"kb_used_data":7288,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93436536,"statfs":{"total":95995035648,"available":95679012864,"internally_reserved":0,"allocated":7462912,"data_stored":4492334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 23 00:47:49 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":1.028,"15min":0.59699999999999998},"min":{"1min":0.245,"5min":0.20999999999999999,"15min":0.19500000000000001},"max":{"1min":0.81399999999999995,"5min":28.856999999999999,"15min":28.856999999999999},"last":0.31},{"interface":"front","average":{"1min":0.38,"5min":1.083,"15min":0.61099999999999999},"min":{"1min":0.23599999999999999,"5min":0.218,"15min":0.20799999999999999},"max":{"1min":0.59899999999999998,"5min":20.577000000000002,"15min":20.577000000000002},"last":0.42799999999999999}]},{"osd":2,"last update":"Fri Feb 23 00:48:10 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.93799999999999994,"15min":0.60299999999999998},"min":{"1min":0.28999999999999998,"5min":0.27200000000000002,"15min":0.20499999999999999},"max":{"1min":0.56799999999999995,"5min":13.863,"15min":13.863},"last":0.5},{"interface":"front","average":{"1min":0.379,"5min":0.76900000000000002,"15min":0.54300000000000004},"min":{"1min":0.29299999999999998,"5min":0.219,"15min":0.19800000000000001},"max":{"1min":0.51500000000000001,"5min":10.455,"15min":10.455},"last":0.32800000000000001}]},{"osd":3,"last update":"Fri Feb 23 00:48:39 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.68600000000000005,"15min":0.79100000000000004},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.52600000000000002,"5min":13.266,"15min":35.566000000000003},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.435,"5min":0.80700000000000005,"15min":0.77300000000000002},"min":{"1min":0.30199999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.59399999999999997,"5min":16.931999999999999,"15min":20.702999999999999},"last":0.59299999999999997}]},{"osd":4,"last update":"Fri Feb 23 00:48:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.93899999999999995,"15min":0.78300000000000003},"min":{"1min":0.32700000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94699999999999995,"5min":10.584,"15min":10.584},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":1.0109999999999999,"15min":0.80700000000000005},"min":{"1min":0.41299999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.875,"5min":13.198,"15min":13.198},"last":0.747}]},{"osd":5,"last update":"Fri Feb 23 00:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.90600000000000003,"15min":0.73099999999999998},"min":{"1min":0.36199999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.84199999999999997,"5min":10.397,"15min":10.397},"last":0.621},{"interface":"front","average":{"1min":0.64600000000000002,"5min":1.0049999999999999,"15min":0.79900000000000004},"min":{"1min":0.46600000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79900000000000004,"5min":17.297000000000001,"15min":17.297000000000001},"last":0.60199999999999998}]},{"osd":6,"last update":"Fri Feb 23 00:48:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":1.0189999999999999,"15min":0.79300000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.38400000000000001},"max":{"1min":0.94399999999999995,"5min":17.373999999999999,"15min":17.373999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":1.026,"15min":0.76500000000000001},"min":{"1min":0.49199999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93400000000000005,"5min":13.973000000000001,"15min":13.973000000000001},"last":0.68300000000000005}]},{"osd":7,"last update":"Fri Feb 23 00:48:25 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.99199999999999999,"15min":0.79700000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.77600000000000002,"5min":17.399999999999999,"15min":17.399999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.626,"5min":1.0609999999999999,"15min":0.78200000000000003},"min":{"1min":0.438,"5min":0.433,"15min":0.43099999999999999},"max":{"1min":0.879,"5min":16.427,"15min":16.427},"last":0.72799999999999998}]}]},{"osd":2,"up_from":0,"seq":77309411474,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":26,"seq":111669149835,"num_pgs":92,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308692,"kb_used_data":7364,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93436460,"statfs":{"total":95995035648,"available":95678935040,"internally_reserved":0,"allocated":7540736,"data_stored":4492178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:48:41 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.60199999999999998,"15min":0.52500000000000002},"min":{"1min":0.254,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.81999999999999995,"5min":12.212999999999999,"15min":12.212999999999999},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.73399999999999999,"15min":0.60899999999999999},"min":{"1min":0.28199999999999997,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.89900000000000002,"5min":11.714,"15min":11.714},"last":0.22800000000000001}]},{"osd":1,"last update":"Fri Feb 23 00:48:41 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.77500000000000002,"15min":0.63600000000000001},"min":{"1min":0.29999999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.56100000000000005,"5min":11.742000000000001,"15min":11.742000000000001},"last":0.311},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.746,"15min":0.60299999999999998},"min":{"1min":0.25800000000000001,"5min":0.223,"15min":0.19900000000000001},"max":{"1min":1.032,"5min":8.2270000000000003,"15min":8.2270000000000003},"last":0.24399999999999999}]},{"osd":2,"last update":"Fri Feb 23 00:48:41 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.874,"15min":0.68200000000000005},"min":{"1min":0.28999999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.79000000000000004,"5min":13.359999999999999,"15min":13.359999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.84299999999999997,"15min":0.65300000000000002},"min":{"1min":0.29999999999999999,"5min":0.26200000000000001,"15min":0.26100000000000001},"max":{"1min":1.3500000000000001,"5min":21.122,"15min":21.122},"last":0.30099999999999999}]},{"osd":4,"last update":"Fri Feb 23 00:48:05 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":1.0549999999999999,"15min":0.754},"min":{"1min":0.376,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73099999999999998,"5min":11.644,"15min":11.644},"last":0.378},{"interface":"front","average":{"1min":0.56599999999999995,"5min":1.0189999999999999,"15min":0.75},"min":{"1min":0.39000000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76900000000000002,"5min":12.430999999999999,"15min":12.430999999999999},"last":0.32700000000000001}]},{"osd":5,"last update":"Fri Feb 23 00:48:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.78300000000000003,"15min":0.66200000000000003},"min":{"1min":0.372,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.4319999999999999,"5min":5.6360000000000001,"15min":5.6360000000000001},"last":0.37},{"interface":"front","average":{"1min":0.60899999999999999,"5min":1.1100000000000001,"15min":0.79600000000000004},"min":{"1min":0.39400000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0029999999999999,"5min":11.648,"15min":11.648},"last":0.42799999999999999}]},{"osd":6,"last update":"Fri Feb 23 00:47:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.995,"15min":0.752},"min":{"1min":0.45200000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.7869999999999999,"5min":6.9420000000000002,"15min":6.9420000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.80000000000000004,"15min":0.68700000000000006},"min":{"1min":0.45100000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.4179999999999999,"5min":5.6680000000000001,"15min":5.6680000000000001},"last":0.439}]},{"osd":7,"last update":"Fri Feb 23 00:48:25 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":1.167,"15min":0.82399999999999995},"min":{"1min":0.42099999999999999,"5min":0.40899999999999997,"15min":0.39000000000000001},"max":{"1min":0.97599999999999998,"5min":11.676,"15min":11.676},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":1.052,"15min":0.80600000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80400000000000005,"5min":7.9800000000000004,"15min":7.9800000000000004},"last":0.48199999999999998}]}]},{"osd":4,"up_from":32,"seq":137438953607,"num_pgs":87,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":312536,"kb_used_data":11212,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93432616,"statfs":{"total":95995035648,"available":95674998784,"internally_reserved":0,"allocated":11481088,"data_stored":8481675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:48:06 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.499,"15min":0.54300000000000004},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.746,"5min":3.5219999999999998,"15min":3.5219999999999998},"last":0.502},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.62,"15min":0.59599999999999997},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.76600000000000001,"5min":8.6140000000000008,"15min":8.6140000000000008},"last":0.436}]},{"osd":1,"last update":"Fri Feb 23 00:48:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.65800000000000003,"15min":0.59299999999999997},"min":{"1min":0.29699999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.69199999999999995,"5min":9.6110000000000007,"15min":9.6110000000000007},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.629,"15min":0.58899999999999997},"min":{"1min":0.29599999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.70499999999999996,"5min":6.6950000000000003,"15min":6.6950000000000003},"last":0.53100000000000003}]},{"osd":2,"last update":"Fri Feb 23 00:48:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58299999999999996,"15min":0.58399999999999996},"min":{"1min":0.34599999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0960000000000001,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.38900000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0189999999999999,"5min":5.8440000000000003,"15min":5.8440000000000003},"last":0.51100000000000001}]},{"osd":3,"last update":"Fri Feb 23 00:48:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.61399999999999999,"15min":0.59599999999999997},"min":{"1min":0.38400000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":6.5540000000000003,"15min":6.5540000000000003},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.372,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":4.0220000000000002,"5min":4.0220000000000002,"15min":4.1909999999999998},"last":0.48699999999999999}]},{"osd":5,"last update":"Fri Feb 23 00:48:30 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.40500000000000003,"15min":0.39700000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.46800000000000003,"15min":0.45300000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.28899999999999998},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.80800000000000005},"last":0.40400000000000003}]},{"osd":6,"last update":"Fri Feb 23 00:48:03 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40699999999999997,"15min":0.41999999999999998},"min":{"1min":0.28799999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.75},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.41899999999999998,"15min":0.42399999999999999},"min":{"1min":0.28799999999999998,"5min":0.215,"15min":0.215},"max":{"1min":0.55400000000000005,"5min":0.72299999999999998,"15min":0.72699999999999998},"last":0.316}]},{"osd":7,"last update":"Fri Feb 23 00:48:26 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.44400000000000001,"15min":0.47799999999999998},"min":{"1min":0.27600000000000002,"5min":0.26700000000000002,"15min":0.254},"max":{"1min":0.71299999999999997,"5min":0.77800000000000002,"15min":0.99099999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.50700000000000001,"15min":0.53400000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72899999999999998,"5min":0.78800000000000003,"15min":0.90200000000000002},"last":0.377}]}]},{"osd":5,"up_from":39,"seq":167503724674,"num_pgs":75,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":304476,"kb_used_data":7252,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93440676,"statfs":{"total":95995035648,"available":95683252224,"internally_reserved":0,"allocated":7426048,"data_stored":4473393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:48:25 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.56299999999999994,"15min":0.59699999999999998},"min":{"1min":0.253,"5min":0.246,"15min":0.246},"max":{"1min":1.0009999999999999,"5min":3.7480000000000002,"15min":23.881},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.35299999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.95,"5min":2.9239999999999999,"15min":5.4649999999999999},"last":0.47999999999999998}]},{"osd":1,"last update":"Fri Feb 23 00:48:25 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.67800000000000005,"15min":0.64600000000000002},"min":{"1min":0.34999999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.93999999999999995,"5min":5.6509999999999998,"15min":18.175999999999998},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.65700000000000003,"15min":0.62},"min":{"1min":0.33400000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.905,"5min":4.2240000000000002,"15min":9.0950000000000006},"last":0.443}]},{"osd":2,"last update":"Fri Feb 23 00:48:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.77100000000000002,"15min":0.69899999999999995},"min":{"1min":0.35299999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.282,"5min":10.590999999999999,"15min":22.282},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.72499999999999998,"15min":0.70799999999999996},"min":{"1min":0.41299999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.123,"5min":11.196999999999999,"15min":23.925000000000001},"last":0.52300000000000002}]},{"osd":3,"last update":"Fri Feb 23 00:48:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.73599999999999999,"15min":0.61499999999999999},"min":{"1min":0.40200000000000002,"5min":0.312,"15min":0.312},"max":{"1min":1.23,"5min":6.5730000000000004,"15min":6.5730000000000004},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.72999999999999998,"15min":0.64100000000000001},"min":{"1min":0.41199999999999998,"5min":0.33600000000000002,"15min":0.33500000000000002},"max":{"1min":1.0640000000000001,"5min":6.7910000000000004,"15min":8.1760000000000002},"last":0.503}]},{"osd":4,"last update":"Fri Feb 23 00:48:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.46500000000000002,"15min":0.42499999999999999},"min":{"1min":0.309,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":4.593,"5min":4.593,"15min":4.593},"last":0.371},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.51200000000000001,"15min":0.45300000000000001},"min":{"1min":0.35799999999999998,"5min":0.27700000000000002,"15min":0.27300000000000002},"max":{"1min":1.5860000000000001,"5min":3.879,"15min":3.879},"last":0.34599999999999997}]},{"osd":6,"last update":"Fri Feb 23 00:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.502,"15min":0.46400000000000002},"min":{"1min":0.32100000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":4.6740000000000004,"5min":4.6740000000000004,"15min":4.6740000000000004},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.47199999999999998,"15min":0.46700000000000003},"min":{"1min":0.35399999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.46500000000000002}]},{"osd":7,"last update":"Fri Feb 23 00:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.50900000000000001,"15min":0.53800000000000003},"min":{"1min":0.33700000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.302,"5min":1.302,"15min":7.0300000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.51000000000000001,"15min":0.53900000000000003},"min":{"1min":0.34699999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":2.016,"5min":2.016,"15min":9.0579999999999998},"last":0.51100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":225280,"data_stored":529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4268032,"data_stored":4000614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":68,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":290816,"data_stored":680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4268032,"data_stored":4000639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":225280,"data_stored":525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4366336,"data_stored":4000844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":213,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":213,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":213,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":213,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":213,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":213,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":213,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-23T00:48:51.607 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:48:51.611 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:51.611 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:48:51.712 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:48:51.742 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:52.527 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:51.103375+0000 osd.3 (osd.3) 202 : cluster [DBG] 18.1a scrub starts 2024-02-23T00:48:52.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:51.104301+0000 osd.3 (osd.3) 203 : cluster [DBG] 18.1a scrub ok 2024-02-23T00:48:52.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:51.202976+0000 mon.a (mon.0) 2612 : cluster [DBG] osdmap e440: 8 total, 8 up, 7 in 2024-02-23T00:48:52.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:51.297877+0000 mgr.y (mgr.14517) 408 : cluster [DBG] pgmap v645: 208 pgs: 32 creating+peering, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:48:52.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:51.716350+0000 osd.7 (osd.7) 213 : cluster [DBG] 18.6 scrub starts 2024-02-23T00:48:52.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:51.717392+0000 osd.7 (osd.7) 214 : cluster [DBG] 18.6 scrub ok 2024-02-23T00:48:52.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:51.993519+0000 mon.a (mon.0) 2613 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:48:52.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:51.103375+0000 osd.3 (osd.3) 202 : cluster [DBG] 18.1a scrub starts 2024-02-23T00:48:52.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:51.104301+0000 osd.3 (osd.3) 203 : cluster [DBG] 18.1a scrub ok 2024-02-23T00:48:52.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:51.202976+0000 mon.a (mon.0) 2612 : cluster [DBG] osdmap e440: 8 total, 8 up, 7 in 2024-02-23T00:48:52.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:51.297877+0000 mgr.y (mgr.14517) 408 : cluster [DBG] pgmap v645: 208 pgs: 32 creating+peering, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:48:52.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:51.716350+0000 osd.7 (osd.7) 213 : cluster [DBG] 18.6 scrub starts 2024-02-23T00:48:52.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:51.717392+0000 osd.7 (osd.7) 214 : cluster [DBG] 18.6 scrub ok 2024-02-23T00:48:52.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:51.993519+0000 mon.a (mon.0) 2613 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:48:52.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:51.103375+0000 osd.3 (osd.3) 202 : cluster [DBG] 18.1a scrub starts 2024-02-23T00:48:52.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:51.104301+0000 osd.3 (osd.3) 203 : cluster [DBG] 18.1a scrub ok 2024-02-23T00:48:52.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:51.202976+0000 mon.a (mon.0) 2612 : cluster [DBG] osdmap e440: 8 total, 8 up, 7 in 2024-02-23T00:48:52.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:51.297877+0000 mgr.y (mgr.14517) 408 : cluster [DBG] pgmap v645: 208 pgs: 32 creating+peering, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:48:52.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:51.716350+0000 osd.7 (osd.7) 213 : cluster [DBG] 18.6 scrub starts 2024-02-23T00:48:52.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:51.717392+0000 osd.7 (osd.7) 214 : cluster [DBG] 18.6 scrub ok 2024-02-23T00:48:52.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:51.993519+0000 mon.a (mon.0) 2613 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:48:52.692 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:48:52.695 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:52.696 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:48:52.797 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:48:52.826 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:52.901 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:48:52.904 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:52.904 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:48:52.943 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:48:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:48:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:48:52.943 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:48:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:48:53.005 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:48:53.034 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:53.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:52.310800+0000 mon.a (mon.0) 2614 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:48:53.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:52.311474+0000 mon.a (mon.0) 2615 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:48:53.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:52.315737+0000 mon.a (mon.0) 2616 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:48:53.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:52.609304+0000 mgr.y (mgr.14517) 409 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:48:53.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:52.310800+0000 mon.a (mon.0) 2614 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:48:53.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:52.311474+0000 mon.a (mon.0) 2615 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:48:53.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:52.315737+0000 mon.a (mon.0) 2616 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:48:53.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:52.609304+0000 mgr.y (mgr.14517) 409 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:48:53.484 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:48:53.488 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:53.488 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:48:53.589 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:48:53.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:52.310800+0000 mon.a (mon.0) 2614 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:48:53.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:52.311474+0000 mon.a (mon.0) 2615 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:48:53.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:52.315737+0000 mon.a (mon.0) 2616 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:48:53.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:52.609304+0000 mgr.y (mgr.14517) 409 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:48:53.621 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:53.643 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:48:53.646 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:53.646 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:48:53.747 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:48:53.776 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:53.795 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:48:53.797 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:53.798 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:48:53.899 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:48:53.930 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:53.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.268+0000 7fd450ca3700 1 -- 172.21.15.47:0/954363141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd44c0f9670 msgr2=0x7fd44c101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:53.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.268+0000 7fd450ca3700 1 --2- 172.21.15.47:0/954363141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd44c0f9670 0x7fd44c101d80 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 rx=0x7fd440004870 tx=0x7fd440009cb0).stop 2024-02-23T00:48:53.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.268+0000 7fd450ca3700 1 -- 172.21.15.47:0/954363141 shutdown_connections 2024-02-23T00:48:53.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.268+0000 7fd450ca3700 1 --2- 172.21.15.47:0/954363141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd44c102270 0x7fd44c104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:53.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.268+0000 7fd450ca3700 1 --2- 172.21.15.47:0/954363141 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd44c0f8d10 0x7fd44c0f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:53.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.268+0000 7fd450ca3700 1 --2- 172.21.15.47:0/954363141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd44c0f9670 0x7fd44c101d80 unknown :-1 s=CLOSED pgs=2537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:53.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.268+0000 7fd450ca3700 1 -- 172.21.15.47:0/954363141 >> 172.21.15.47:0/954363141 conn(0x7fd44c0f2820 msgr2=0x7fd44c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:48:53.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.268+0000 7fd450ca3700 1 -- 172.21.15.47:0/954363141 shutdown_connections 2024-02-23T00:48:53.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.268+0000 7fd450ca3700 1 -- 172.21.15.47:0/954363141 wait complete. 2024-02-23T00:48:53.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.268+0000 7fd450ca3700 1 librados: starting msgr at 2024-02-23T00:48:53.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.268+0000 7fd450ca3700 1 librados: starting objecter 2024-02-23T00:48:53.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.268+0000 7fd450ca3700 1 Processor -- start 2024-02-23T00:48:53.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.269+0000 7fd450ca3700 1 -- start start 2024-02-23T00:48:53.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.269+0000 7fd450ca3700 1 librados: setting wanted keys 2024-02-23T00:48:53.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.269+0000 7fd450ca3700 1 librados: calling monclient init 2024-02-23T00:48:53.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.269+0000 7fd450ca3700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd44c0f8d10 0x7fd44c10e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:53.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.269+0000 7fd450ca3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd44c0f9670 0x7fd44c10e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:53.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.269+0000 7fd450ca3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd44c102270 0x7fd44c108220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:53.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.269+0000 7fd450ca3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c108710 con 0x7fd44c0f9670 2024-02-23T00:48:53.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.269+0000 7fd450ca3700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c108880 con 0x7fd44c102270 2024-02-23T00:48:53.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.269+0000 7fd450ca3700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c108a40 con 0x7fd44c0f8d10 2024-02-23T00:48:53.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.269+0000 7fd449d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd44c0f9670 0x7fd44c10e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:53.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.269+0000 7fd44a59c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd44c0f8d10 0x7fd44c10e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:53.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.269+0000 7fd449d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd44c0f9670 0x7fd44c10e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53830/0 (socket says 172.21.15.47:53830) 2024-02-23T00:48:53.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.269+0000 7fd449d9b700 1 -- 172.21.15.47:0/242100848 learned_addr learned my addr 172.21.15.47:0/242100848 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:48:53.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.269+0000 7fd44ad9d700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd44c102270 0x7fd44c108220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:53.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.269+0000 7fd44a59c700 1 -- 172.21.15.47:0/242100848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd44c102270 msgr2=0x7fd44c108220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:53.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.269+0000 7fd44a59c700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd44c102270 0x7fd44c108220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:53.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.269+0000 7fd44a59c700 1 -- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd44c0f9670 msgr2=0x7fd44c10e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:53.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.269+0000 7fd44a59c700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd44c0f9670 0x7fd44c10e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:53.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.269+0000 7fd44a59c700 1 -- 172.21.15.47:0/242100848 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd43c005040 con 0x7fd44c0f8d10 2024-02-23T00:48:53.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.269+0000 7fd449d9b700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd44c0f9670 0x7fd44c10e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:48:53.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.269+0000 7fd44a59c700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd44c0f8d10 0x7fd44c10e080 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 rx=0x7fd43c004920 tx=0x7fd43c009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:53.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.269+0000 7fd43b7fe700 10 client.?.objecter ms_handle_connect 0x7fd44c0f8d10 2024-02-23T00:48:53.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.269+0000 7fd43b7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:48:53.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.269+0000 7fd43b7fe700 1 -- 172.21.15.47:0/242100848 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c00e030 con 0x7fd44c0f8d10 2024-02-23T00:48:53.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.270+0000 7fd43b7fe700 1 -- 172.21.15.47:0/242100848 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fd43c00f040 con 0x7fd44c0f8d10 2024-02-23T00:48:53.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.270+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd440005040 con 0x7fd44c0f8d10 2024-02-23T00:48:53.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.270+0000 7fd43b7fe700 1 -- 172.21.15.47:0/242100848 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c014720 con 0x7fd44c0f8d10 2024-02-23T00:48:53.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.270+0000 7fd450ca3700 10 client.41387.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:48:53.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.270+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd44c109080 con 0x7fd44c0f8d10 2024-02-23T00:48:53.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.270+0000 7fd450ca3700 1 librados: init done 2024-02-23T00:48:53.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.270+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fd44c05fa60 con 0x7fd44c0f8d10 2024-02-23T00:48:53.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.271+0000 7fd43b7fe700 1 -- 172.21.15.47:0/242100848 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fd43c010500 con 0x7fd44c0f8d10 2024-02-23T00:48:53.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.271+0000 7fd43b7fe700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd434062cc0 0x7fd434065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:53.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.271+0000 7fd43b7fe700 1 -- 172.21.15.47:0/242100848 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(440..440 src has 48..440) v4 ==== 9276+0+0 (secure 0 0 0) 0x7fd43c07d6f0 con 0x7fd44c0f8d10 2024-02-23T00:48:53.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.271+0000 7fd43b7fe700 10 client.41387.objecter ms_dispatch 0x7fd44c001550 osd_map(440..440 src has 48..440) v4 2024-02-23T00:48:53.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.271+0000 7fd43b7fe700 3 client.41387.objecter handle_osd_map got epochs [440,440] > 0 2024-02-23T00:48:53.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.271+0000 7fd43b7fe700 3 client.41387.objecter handle_osd_map decoding full epoch 440 2024-02-23T00:48:53.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.271+0000 7fd449d9b700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd434062cc0 0x7fd434065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:53.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.271+0000 7fd43b7fe700 20 client.41387.objecter dump_active .. 0 homeless 2024-02-23T00:48:53.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.271+0000 7fd43b7fe700 1 -- 172.21.15.47:0/242100848 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v440) v1 ==== 54+0+16 (secure 0 0 0) 0x7fd43c084070 con 0x7fd44c0f8d10 2024-02-23T00:48:53.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.272+0000 7fd43b7fe700 10 client.41387.objecter ms_handle_connect 0x7fd434062cc0 2024-02-23T00:48:53.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.272+0000 7fd449d9b700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd434062cc0 0x7fd434065180 secure :-1 s=READY pgs=3073 cs=0 l=1 rev1=1 rx=0x7fd440007fd0 tx=0x7fd44000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:53.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.272+0000 7fd450ca3700 10 client.41387.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:53.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.272+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_assign -1 1 2024-02-23T00:48:53.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.272+0000 7fd450ca3700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fd44c028540 0x7fd44c0fbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:53.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.272+0000 7fd450ca3700 20 client.41387.objecter _get_session s=0x7fd44c05ee80 osd=0 3 2024-02-23T00:48:53.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.272+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c05ee80 osd=0 3 2024-02-23T00:48:53.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.272+0000 7fd450ca3700 20 client.41387.objecter _get_session s=0x7fd44c05ee80 osd=0 3 2024-02-23T00:48:53.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.272+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_remove -1 1 2024-02-23T00:48:53.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.272+0000 7fd450ca3700 20 client.41387.objecter get_session s=0x7fd44c05ee80 osd=0 3 2024-02-23T00:48:53.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.272+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_assign 0 1 2024-02-23T00:48:53.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.272+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c05ee80 osd=0 4 2024-02-23T00:48:53.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.272+0000 7fd450ca3700 10 client.41387.objecter _send_command 1 2024-02-23T00:48:53.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.272+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd44c107060 con 0x7fd44c028540 2024-02-23T00:48:53.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.272+0000 7fd44ad9d700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fd44c028540 0x7fd44c0fbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:53.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.272+0000 7fd43b7fe700 10 client.41387.objecter ms_handle_connect 0x7fd44c028540 2024-02-23T00:48:53.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.272+0000 7fd44ad9d700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fd44c028540 0x7fd44c0fbca0 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:53.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.273+0000 7fd43b7fe700 1 -- 172.21.15.47:0/242100848 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd44c107060 con 0x7fd44c028540 2024-02-23T00:48:53.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.273+0000 7fd43b7fe700 10 client.41387.objecter ms_dispatch 0x7fd44c001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:48:53.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.273+0000 7fd43b7fe700 10 client.41387.objecter _finish_command 1 = system:0 2024-02-23T00:48:53.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.273+0000 7fd43b7fe700 20 client.41387.objecter put_session s=0x7fd44c05ee80 osd=0 4 2024-02-23T00:48:53.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.273+0000 7fd43b7fe700 15 client.41387.objecter _session_command_op_remove 0 1 2024-02-23T00:48:53.960 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:48:53.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.288+0000 7fd450ca3700 10 client.41387.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:48:53.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.288+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_assign -1 2 2024-02-23T00:48:53.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.288+0000 7fd450ca3700 20 client.41387.objecter _get_session s=0x7fd44c05ee80 osd=0 3 2024-02-23T00:48:53.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.288+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c05ee80 osd=0 3 2024-02-23T00:48:53.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.288+0000 7fd450ca3700 20 client.41387.objecter _get_session s=0x7fd44c05ee80 osd=0 3 2024-02-23T00:48:53.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.288+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_remove -1 2 2024-02-23T00:48:53.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.288+0000 7fd450ca3700 20 client.41387.objecter get_session s=0x7fd44c05ee80 osd=0 3 2024-02-23T00:48:53.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.288+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_assign 0 2 2024-02-23T00:48:53.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.288+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c05ee80 osd=0 4 2024-02-23T00:48:53.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.288+0000 7fd450ca3700 10 client.41387.objecter _send_command 2 2024-02-23T00:48:53.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.288+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd44c05a040 con 0x7fd44c028540 2024-02-23T00:48:53.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.290+0000 7fd43b7fe700 1 -- 172.21.15.47:0/242100848 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd44c05a040 con 0x7fd44c028540 2024-02-23T00:48:53.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.290+0000 7fd43b7fe700 10 client.41387.objecter ms_dispatch 0x7fd44c001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:48:53.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.290+0000 7fd43b7fe700 10 client.41387.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:48:53.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.290+0000 7fd43b7fe700 20 client.41387.objecter put_session s=0x7fd44c05ee80 osd=0 4 2024-02-23T00:48:53.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.290+0000 7fd43b7fe700 15 client.41387.objecter _session_command_op_remove 0 2 2024-02-23T00:48:53.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.290+0000 7fd450ca3700 10 client.41387.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:53.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.290+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_assign -1 3 2024-02-23T00:48:53.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.290+0000 7fd450ca3700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fd44c027b90 0x7fd44c027f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:53.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.290+0000 7fd450ca3700 20 client.41387.objecter _get_session s=0x7fd44c0fd310 osd=1 3 2024-02-23T00:48:53.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.290+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c0fd310 osd=1 3 2024-02-23T00:48:53.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.290+0000 7fd450ca3700 20 client.41387.objecter _get_session s=0x7fd44c0fd310 osd=1 3 2024-02-23T00:48:53.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.290+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_remove -1 3 2024-02-23T00:48:53.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.290+0000 7fd450ca3700 20 client.41387.objecter get_session s=0x7fd44c0fd310 osd=1 3 2024-02-23T00:48:53.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.290+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_assign 1 3 2024-02-23T00:48:53.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.290+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c0fd310 osd=1 4 2024-02-23T00:48:53.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.290+0000 7fd450ca3700 10 client.41387.objecter _send_command 3 2024-02-23T00:48:53.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.290+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd44c0fd480 con 0x7fd44c027b90 2024-02-23T00:48:53.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.290+0000 7fd44a59c700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fd44c027b90 0x7fd44c027f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:53.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.290+0000 7fd43b7fe700 10 client.41387.objecter ms_handle_connect 0x7fd44c027b90 2024-02-23T00:48:53.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.290+0000 7fd44a59c700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fd44c027b90 0x7fd44c027f70 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:53.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.290+0000 7fd43b7fe700 1 -- 172.21.15.47:0/242100848 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd44c0fd480 con 0x7fd44c027b90 2024-02-23T00:48:53.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.290+0000 7fd43b7fe700 10 client.41387.objecter ms_dispatch 0x7fd44c001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:48:53.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.290+0000 7fd43b7fe700 10 client.41387.objecter _finish_command 3 = system:0 2024-02-23T00:48:53.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.290+0000 7fd43b7fe700 20 client.41387.objecter put_session s=0x7fd44c0fd310 osd=1 4 2024-02-23T00:48:53.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.290+0000 7fd43b7fe700 15 client.41387.objecter _session_command_op_remove 1 3 2024-02-23T00:48:53.963 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:48:53.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.306+0000 7fd450ca3700 10 client.41387.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:48:53.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.306+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_assign -1 4 2024-02-23T00:48:53.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.306+0000 7fd450ca3700 20 client.41387.objecter _get_session s=0x7fd44c0fd310 osd=1 3 2024-02-23T00:48:53.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.306+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c0fd310 osd=1 3 2024-02-23T00:48:53.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.306+0000 7fd450ca3700 20 client.41387.objecter _get_session s=0x7fd44c0fd310 osd=1 3 2024-02-23T00:48:53.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.306+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_remove -1 4 2024-02-23T00:48:53.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.306+0000 7fd450ca3700 20 client.41387.objecter get_session s=0x7fd44c0fd310 osd=1 3 2024-02-23T00:48:53.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.306+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_assign 1 4 2024-02-23T00:48:53.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.306+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c0fd310 osd=1 4 2024-02-23T00:48:53.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.306+0000 7fd450ca3700 10 client.41387.objecter _send_command 4 2024-02-23T00:48:53.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.306+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd44c027600 con 0x7fd44c027b90 2024-02-23T00:48:53.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.307+0000 7fd43b7fe700 1 -- 172.21.15.47:0/242100848 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd44c027600 con 0x7fd44c027b90 2024-02-23T00:48:53.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.307+0000 7fd43b7fe700 10 client.41387.objecter ms_dispatch 0x7fd44c001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:48:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.307+0000 7fd43b7fe700 10 client.41387.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:48:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.307+0000 7fd43b7fe700 20 client.41387.objecter put_session s=0x7fd44c0fd310 osd=1 4 2024-02-23T00:48:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.307+0000 7fd43b7fe700 15 client.41387.objecter _session_command_op_remove 1 4 2024-02-23T00:48:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.307+0000 7fd450ca3700 10 client.41387.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.307+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_assign -1 5 2024-02-23T00:48:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.307+0000 7fd450ca3700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fd44c0fce70 0x7fd44c11c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.307+0000 7fd450ca3700 20 client.41387.objecter _get_session s=0x7fd44c027a10 osd=2 3 2024-02-23T00:48:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.307+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c027a10 osd=2 3 2024-02-23T00:48:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.307+0000 7fd450ca3700 20 client.41387.objecter _get_session s=0x7fd44c027a10 osd=2 3 2024-02-23T00:48:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.307+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_remove -1 5 2024-02-23T00:48:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.307+0000 7fd450ca3700 20 client.41387.objecter get_session s=0x7fd44c027a10 osd=2 3 2024-02-23T00:48:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.307+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_assign 2 5 2024-02-23T00:48:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.307+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c027a10 osd=2 4 2024-02-23T00:48:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.307+0000 7fd450ca3700 10 client.41387.objecter _send_command 5 2024-02-23T00:48:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.307+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd44c11eeb0 con 0x7fd44c0fce70 2024-02-23T00:48:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.308+0000 7fd449d9b700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fd44c0fce70 0x7fd44c11c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.308+0000 7fd43b7fe700 10 client.41387.objecter ms_handle_connect 0x7fd44c0fce70 2024-02-23T00:48:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.308+0000 7fd449d9b700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fd44c0fce70 0x7fd44c11c0c0 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.308+0000 7fd43b7fe700 1 -- 172.21.15.47:0/242100848 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd44c11eeb0 con 0x7fd44c0fce70 2024-02-23T00:48:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.308+0000 7fd43b7fe700 10 client.41387.objecter ms_dispatch 0x7fd44c001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:48:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.308+0000 7fd43b7fe700 10 client.41387.objecter _finish_command 5 = system:0 2024-02-23T00:48:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.308+0000 7fd43b7fe700 20 client.41387.objecter put_session s=0x7fd44c027a10 osd=2 4 2024-02-23T00:48:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.308+0000 7fd43b7fe700 15 client.41387.objecter _session_command_op_remove 2 5 2024-02-23T00:48:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.323+0000 7fd450ca3700 10 client.41387.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:48:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.323+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_assign -1 6 2024-02-23T00:48:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.323+0000 7fd450ca3700 20 client.41387.objecter _get_session s=0x7fd44c027a10 osd=2 3 2024-02-23T00:48:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.323+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c027a10 osd=2 3 2024-02-23T00:48:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.323+0000 7fd450ca3700 20 client.41387.objecter _get_session s=0x7fd44c027a10 osd=2 3 2024-02-23T00:48:53.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.323+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_remove -1 6 2024-02-23T00:48:53.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.323+0000 7fd450ca3700 20 client.41387.objecter get_session s=0x7fd44c027a10 osd=2 3 2024-02-23T00:48:53.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.323+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_assign 2 6 2024-02-23T00:48:53.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.323+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c027a10 osd=2 4 2024-02-23T00:48:53.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.323+0000 7fd450ca3700 10 client.41387.objecter _send_command 6 2024-02-23T00:48:53.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.323+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd44c0fc3d0 con 0x7fd44c0fce70 2024-02-23T00:48:53.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.324+0000 7fd43b7fe700 1 -- 172.21.15.47:0/242100848 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd44c0fc3d0 con 0x7fd44c0fce70 2024-02-23T00:48:53.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.324+0000 7fd43b7fe700 10 client.41387.objecter ms_dispatch 0x7fd44c001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:48:53.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.324+0000 7fd43b7fe700 10 client.41387.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:48:53.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.324+0000 7fd43b7fe700 20 client.41387.objecter put_session s=0x7fd44c027a10 osd=2 4 2024-02-23T00:48:53.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.324+0000 7fd43b7fe700 15 client.41387.objecter _session_command_op_remove 2 6 2024-02-23T00:48:53.967 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:48:53.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.325+0000 7fd450ca3700 10 client.41387.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:53.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.325+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_assign -1 7 2024-02-23T00:48:53.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.325+0000 7fd450ca3700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fd44c119c10 0x7fd44c11c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:53.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.325+0000 7fd450ca3700 20 client.41387.objecter _get_session s=0x7fd44c0fc7e0 osd=3 3 2024-02-23T00:48:53.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.325+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c0fc7e0 osd=3 3 2024-02-23T00:48:53.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.325+0000 7fd450ca3700 20 client.41387.objecter _get_session s=0x7fd44c0fc7e0 osd=3 3 2024-02-23T00:48:53.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.325+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_remove -1 7 2024-02-23T00:48:53.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.325+0000 7fd450ca3700 20 client.41387.objecter get_session s=0x7fd44c0fc7e0 osd=3 3 2024-02-23T00:48:53.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.325+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_assign 3 7 2024-02-23T00:48:53.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.325+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c0fc7e0 osd=3 4 2024-02-23T00:48:53.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.325+0000 7fd450ca3700 10 client.41387.objecter _send_command 7 2024-02-23T00:48:53.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.325+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd44c11cc50 con 0x7fd44c119c10 2024-02-23T00:48:53.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.325+0000 7fd44ad9d700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fd44c119c10 0x7fd44c11c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:53.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.325+0000 7fd43b7fe700 10 client.41387.objecter ms_handle_connect 0x7fd44c119c10 2024-02-23T00:48:53.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.325+0000 7fd44ad9d700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fd44c119c10 0x7fd44c11c600 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:53.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.326+0000 7fd43b7fe700 1 -- 172.21.15.47:0/242100848 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd44c11cc50 con 0x7fd44c119c10 2024-02-23T00:48:53.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.326+0000 7fd43b7fe700 10 client.41387.objecter ms_dispatch 0x7fd44c001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:48:53.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.326+0000 7fd43b7fe700 10 client.41387.objecter _finish_command 7 = system:0 2024-02-23T00:48:53.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.326+0000 7fd43b7fe700 20 client.41387.objecter put_session s=0x7fd44c0fc7e0 osd=3 4 2024-02-23T00:48:53.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.326+0000 7fd43b7fe700 15 client.41387.objecter _session_command_op_remove 3 7 2024-02-23T00:48:53.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.340+0000 7fd450ca3700 10 client.41387.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:48:53.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.340+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_assign -1 8 2024-02-23T00:48:53.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.340+0000 7fd450ca3700 20 client.41387.objecter _get_session s=0x7fd44c0fc7e0 osd=3 3 2024-02-23T00:48:53.970 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:48:53.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.340+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c0fc7e0 osd=3 3 2024-02-23T00:48:53.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.340+0000 7fd450ca3700 20 client.41387.objecter _get_session s=0x7fd44c0fc7e0 osd=3 3 2024-02-23T00:48:53.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.340+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_remove -1 8 2024-02-23T00:48:53.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.340+0000 7fd450ca3700 20 client.41387.objecter get_session s=0x7fd44c0fc7e0 osd=3 3 2024-02-23T00:48:53.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.340+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_assign 3 8 2024-02-23T00:48:53.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.340+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c0fc7e0 osd=3 4 2024-02-23T00:48:53.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.340+0000 7fd450ca3700 10 client.41387.objecter _send_command 8 2024-02-23T00:48:53.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.340+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd44c119170 con 0x7fd44c119c10 2024-02-23T00:48:53.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.341+0000 7fd43b7fe700 1 -- 172.21.15.47:0/242100848 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd44c119170 con 0x7fd44c119c10 2024-02-23T00:48:53.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.341+0000 7fd43b7fe700 10 client.41387.objecter ms_dispatch 0x7fd44c001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:48:53.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.341+0000 7fd43b7fe700 10 client.41387.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:48:53.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.341+0000 7fd43b7fe700 20 client.41387.objecter put_session s=0x7fd44c0fc7e0 osd=3 4 2024-02-23T00:48:53.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.341+0000 7fd43b7fe700 15 client.41387.objecter _session_command_op_remove 3 8 2024-02-23T00:48:53.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.342+0000 7fd450ca3700 10 client.41387.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:53.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.342+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_assign -1 9 2024-02-23T00:48:53.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.342+0000 7fd450ca3700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fd44c1192e0 0x7fd44c123070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:53.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.342+0000 7fd450ca3700 20 client.41387.objecter _get_session s=0x7fd44c119170 osd=4 3 2024-02-23T00:48:53.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.342+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c119170 osd=4 3 2024-02-23T00:48:53.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.342+0000 7fd450ca3700 20 client.41387.objecter _get_session s=0x7fd44c119170 osd=4 3 2024-02-23T00:48:53.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.342+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_remove -1 9 2024-02-23T00:48:53.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.342+0000 7fd450ca3700 20 client.41387.objecter get_session s=0x7fd44c119170 osd=4 3 2024-02-23T00:48:53.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.342+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_assign 4 9 2024-02-23T00:48:53.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.342+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c119170 osd=4 4 2024-02-23T00:48:53.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.342+0000 7fd450ca3700 10 client.41387.objecter _send_command 9 2024-02-23T00:48:53.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.342+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd44c125e20 con 0x7fd44c1192e0 2024-02-23T00:48:53.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.342+0000 7fd44a59c700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fd44c1192e0 0x7fd44c123070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:53.972 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:48:53.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.342+0000 7fd44a59c700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fd44c1192e0 0x7fd44c123070 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:53.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.342+0000 7fd43b7fe700 10 client.41387.objecter ms_handle_connect 0x7fd44c1192e0 2024-02-23T00:48:53.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.343+0000 7fd43b7fe700 1 -- 172.21.15.47:0/242100848 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd44c125e20 con 0x7fd44c1192e0 2024-02-23T00:48:53.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.343+0000 7fd43b7fe700 10 client.41387.objecter ms_dispatch 0x7fd44c001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:48:53.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.343+0000 7fd43b7fe700 10 client.41387.objecter _finish_command 9 = system:0 2024-02-23T00:48:53.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.343+0000 7fd43b7fe700 20 client.41387.objecter put_session s=0x7fd44c119170 osd=4 4 2024-02-23T00:48:53.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.343+0000 7fd43b7fe700 15 client.41387.objecter _session_command_op_remove 4 9 2024-02-23T00:48:53.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.358+0000 7fd450ca3700 10 client.41387.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:48:53.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.358+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_assign -1 10 2024-02-23T00:48:53.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.358+0000 7fd450ca3700 20 client.41387.objecter _get_session s=0x7fd44c119170 osd=4 3 2024-02-23T00:48:53.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.358+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c119170 osd=4 3 2024-02-23T00:48:53.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.358+0000 7fd450ca3700 20 client.41387.objecter _get_session s=0x7fd44c119170 osd=4 3 2024-02-23T00:48:53.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.358+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_remove -1 10 2024-02-23T00:48:53.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.358+0000 7fd450ca3700 20 client.41387.objecter get_session s=0x7fd44c119170 osd=4 3 2024-02-23T00:48:53.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.358+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_assign 4 10 2024-02-23T00:48:53.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.358+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c119170 osd=4 4 2024-02-23T00:48:53.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.358+0000 7fd450ca3700 10 client.41387.objecter _send_command 10 2024-02-23T00:48:53.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.358+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd44c124c10 con 0x7fd44c1192e0 2024-02-23T00:48:53.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.361+0000 7fd43b7fe700 1 -- 172.21.15.47:0/242100848 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd44c124c10 con 0x7fd44c1192e0 2024-02-23T00:48:53.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.361+0000 7fd43b7fe700 10 client.41387.objecter ms_dispatch 0x7fd44c001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:48:53.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.361+0000 7fd43b7fe700 10 client.41387.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:48:53.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.361+0000 7fd43b7fe700 20 client.41387.objecter put_session s=0x7fd44c119170 osd=4 4 2024-02-23T00:48:53.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.361+0000 7fd43b7fe700 15 client.41387.objecter _session_command_op_remove 4 10 2024-02-23T00:48:53.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.361+0000 7fd450ca3700 10 client.41387.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:53.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.361+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_assign -1 11 2024-02-23T00:48:53.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.361+0000 7fd450ca3700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fd44c11e3e0 0x7fd44c11e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:53.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.361+0000 7fd450ca3700 20 client.41387.objecter _get_session s=0x7fd44c118f30 osd=5 3 2024-02-23T00:48:53.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.361+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c118f30 osd=5 3 2024-02-23T00:48:53.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.361+0000 7fd450ca3700 20 client.41387.objecter _get_session s=0x7fd44c118f30 osd=5 3 2024-02-23T00:48:53.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.361+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_remove -1 11 2024-02-23T00:48:53.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.361+0000 7fd450ca3700 20 client.41387.objecter get_session s=0x7fd44c118f30 osd=5 3 2024-02-23T00:48:53.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.361+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_assign 5 11 2024-02-23T00:48:53.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.361+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c118f30 osd=5 4 2024-02-23T00:48:53.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.361+0000 7fd450ca3700 10 client.41387.objecter _send_command 11 2024-02-23T00:48:53.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.361+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd44c12b3b0 con 0x7fd44c11e3e0 2024-02-23T00:48:53.975 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:48:53.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.361+0000 7fd449d9b700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fd44c11e3e0 0x7fd44c11e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:53.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.361+0000 7fd449d9b700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fd44c11e3e0 0x7fd44c11e840 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:53.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.361+0000 7fd43b7fe700 10 client.41387.objecter ms_handle_connect 0x7fd44c11e3e0 2024-02-23T00:48:53.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.362+0000 7fd43b7fe700 1 -- 172.21.15.47:0/242100848 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd44c12b3b0 con 0x7fd44c11e3e0 2024-02-23T00:48:53.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.362+0000 7fd43b7fe700 10 client.41387.objecter ms_dispatch 0x7fd44c001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:48:53.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.362+0000 7fd43b7fe700 10 client.41387.objecter _finish_command 11 = system:0 2024-02-23T00:48:53.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.362+0000 7fd43b7fe700 20 client.41387.objecter put_session s=0x7fd44c118f30 osd=5 4 2024-02-23T00:48:53.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.362+0000 7fd43b7fe700 15 client.41387.objecter _session_command_op_remove 5 11 2024-02-23T00:48:53.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.376+0000 7fd450ca3700 10 client.41387.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:48:53.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.376+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_assign -1 12 2024-02-23T00:48:53.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.376+0000 7fd450ca3700 20 client.41387.objecter _get_session s=0x7fd44c118f30 osd=5 3 2024-02-23T00:48:53.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.376+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c118f30 osd=5 3 2024-02-23T00:48:53.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.376+0000 7fd450ca3700 20 client.41387.objecter _get_session s=0x7fd44c118f30 osd=5 3 2024-02-23T00:48:53.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.376+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_remove -1 12 2024-02-23T00:48:53.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.376+0000 7fd450ca3700 20 client.41387.objecter get_session s=0x7fd44c118f30 osd=5 3 2024-02-23T00:48:53.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.376+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_assign 5 12 2024-02-23T00:48:53.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.376+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c118f30 osd=5 4 2024-02-23T00:48:53.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.376+0000 7fd450ca3700 10 client.41387.objecter _send_command 12 2024-02-23T00:48:53.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.376+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd44c123600 con 0x7fd44c11e3e0 2024-02-23T00:48:53.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.378+0000 7fd43b7fe700 1 -- 172.21.15.47:0/242100848 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd44c123600 con 0x7fd44c11e3e0 2024-02-23T00:48:53.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.378+0000 7fd43b7fe700 10 client.41387.objecter ms_dispatch 0x7fd44c001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:48:53.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.378+0000 7fd43b7fe700 10 client.41387.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:48:53.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.378+0000 7fd43b7fe700 20 client.41387.objecter put_session s=0x7fd44c118f30 osd=5 4 2024-02-23T00:48:53.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.378+0000 7fd43b7fe700 15 client.41387.objecter _session_command_op_remove 5 12 2024-02-23T00:48:53.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.378+0000 7fd450ca3700 10 client.41387.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:53.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.378+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_assign -1 13 2024-02-23T00:48:53.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.378+0000 7fd450ca3700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fd44c123c80 0x7fd44c1240e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:53.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.378+0000 7fd450ca3700 20 client.41387.objecter _get_session s=0x7fd44c123600 osd=6 3 2024-02-23T00:48:53.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.378+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c123600 osd=6 3 2024-02-23T00:48:53.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.378+0000 7fd450ca3700 20 client.41387.objecter _get_session s=0x7fd44c123600 osd=6 3 2024-02-23T00:48:53.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.378+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_remove -1 13 2024-02-23T00:48:53.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.378+0000 7fd450ca3700 20 client.41387.objecter get_session s=0x7fd44c123600 osd=6 3 2024-02-23T00:48:53.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.378+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_assign 6 13 2024-02-23T00:48:53.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.378+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c123600 osd=6 4 2024-02-23T00:48:53.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.378+0000 7fd450ca3700 10 client.41387.objecter _send_command 13 2024-02-23T00:48:53.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.378+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd44c124680 con 0x7fd44c123c80 2024-02-23T00:48:53.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.379+0000 7fd44ad9d700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fd44c123c80 0x7fd44c1240e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:53.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.379+0000 7fd43b7fe700 10 client.41387.objecter ms_handle_connect 0x7fd44c123c80 2024-02-23T00:48:53.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.379+0000 7fd44ad9d700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fd44c123c80 0x7fd44c1240e0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:53.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.380+0000 7fd43b7fe700 1 -- 172.21.15.47:0/242100848 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd44c124680 con 0x7fd44c123c80 2024-02-23T00:48:53.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.380+0000 7fd43b7fe700 10 client.41387.objecter ms_dispatch 0x7fd44c001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:48:53.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.380+0000 7fd43b7fe700 10 client.41387.objecter _finish_command 13 = system:0 2024-02-23T00:48:53.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.380+0000 7fd43b7fe700 20 client.41387.objecter put_session s=0x7fd44c123600 osd=6 4 2024-02-23T00:48:53.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.380+0000 7fd43b7fe700 15 client.41387.objecter _session_command_op_remove 6 13 2024-02-23T00:48:53.979 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:48:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.394+0000 7fd450ca3700 10 client.41387.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:48:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.394+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_assign -1 14 2024-02-23T00:48:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.394+0000 7fd450ca3700 20 client.41387.objecter _get_session s=0x7fd44c123600 osd=6 3 2024-02-23T00:48:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.394+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c123600 osd=6 3 2024-02-23T00:48:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.394+0000 7fd450ca3700 20 client.41387.objecter _get_session s=0x7fd44c123600 osd=6 3 2024-02-23T00:48:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.394+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_remove -1 14 2024-02-23T00:48:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.394+0000 7fd450ca3700 20 client.41387.objecter get_session s=0x7fd44c123600 osd=6 3 2024-02-23T00:48:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.394+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_assign 6 14 2024-02-23T00:48:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.394+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c123600 osd=6 4 2024-02-23T00:48:53.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.394+0000 7fd450ca3700 10 client.41387.objecter _send_command 14 2024-02-23T00:48:53.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.394+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd44c124680 con 0x7fd44c123c80 2024-02-23T00:48:53.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.397+0000 7fd43b7fe700 1 -- 172.21.15.47:0/242100848 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd44c124680 con 0x7fd44c123c80 2024-02-23T00:48:53.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.397+0000 7fd43b7fe700 10 client.41387.objecter ms_dispatch 0x7fd44c001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:48:53.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.397+0000 7fd43b7fe700 10 client.41387.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:48:53.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.397+0000 7fd43b7fe700 20 client.41387.objecter put_session s=0x7fd44c123600 osd=6 4 2024-02-23T00:48:53.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.397+0000 7fd43b7fe700 15 client.41387.objecter _session_command_op_remove 6 14 2024-02-23T00:48:53.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.397+0000 7fd450ca3700 10 client.41387.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:48:53.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.397+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_assign -1 15 2024-02-23T00:48:53.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.397+0000 7fd450ca3700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fd44c12c750 0x7fd44c12eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:53.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.397+0000 7fd450ca3700 20 client.41387.objecter _get_session s=0x7fd44c124680 osd=7 3 2024-02-23T00:48:53.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.397+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c124680 osd=7 3 2024-02-23T00:48:53.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.397+0000 7fd450ca3700 20 client.41387.objecter _get_session s=0x7fd44c124680 osd=7 3 2024-02-23T00:48:53.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.397+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_remove -1 15 2024-02-23T00:48:53.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.397+0000 7fd450ca3700 20 client.41387.objecter get_session s=0x7fd44c124680 osd=7 3 2024-02-23T00:48:53.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.397+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_assign 7 15 2024-02-23T00:48:53.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.397+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c124680 osd=7 4 2024-02-23T00:48:53.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.397+0000 7fd450ca3700 10 client.41387.objecter _send_command 15 2024-02-23T00:48:53.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.397+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd44c131790 con 0x7fd44c12c750 2024-02-23T00:48:53.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.397+0000 7fd44a59c700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fd44c12c750 0x7fd44c12eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:53.981 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:48:53.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.397+0000 7fd43b7fe700 10 client.41387.objecter ms_handle_connect 0x7fd44c12c750 2024-02-23T00:48:53.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.397+0000 7fd44a59c700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fd44c12c750 0x7fd44c12eb40 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:53.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.398+0000 7fd43b7fe700 1 -- 172.21.15.47:0/242100848 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd44c131790 con 0x7fd44c12c750 2024-02-23T00:48:53.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.398+0000 7fd43b7fe700 10 client.41387.objecter ms_dispatch 0x7fd44c001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:48:53.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.398+0000 7fd43b7fe700 10 client.41387.objecter _finish_command 15 = system:0 2024-02-23T00:48:53.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.398+0000 7fd43b7fe700 20 client.41387.objecter put_session s=0x7fd44c124680 osd=7 4 2024-02-23T00:48:53.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.398+0000 7fd43b7fe700 15 client.41387.objecter _session_command_op_remove 7 15 2024-02-23T00:48:53.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.413+0000 7fd450ca3700 10 client.41387.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:48:53.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.413+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_assign -1 16 2024-02-23T00:48:53.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.413+0000 7fd450ca3700 20 client.41387.objecter _get_session s=0x7fd44c124680 osd=7 3 2024-02-23T00:48:53.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.413+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c124680 osd=7 3 2024-02-23T00:48:53.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.413+0000 7fd450ca3700 20 client.41387.objecter _get_session s=0x7fd44c124680 osd=7 3 2024-02-23T00:48:53.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.413+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_remove -1 16 2024-02-23T00:48:53.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.413+0000 7fd450ca3700 20 client.41387.objecter get_session s=0x7fd44c124680 osd=7 3 2024-02-23T00:48:53.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.413+0000 7fd450ca3700 15 client.41387.objecter _session_command_op_assign 7 16 2024-02-23T00:48:53.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.413+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c124680 osd=7 4 2024-02-23T00:48:53.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.413+0000 7fd450ca3700 10 client.41387.objecter _send_command 16 2024-02-23T00:48:53.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.413+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd44c12f030 con 0x7fd44c12c750 2024-02-23T00:48:53.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd43b7fe700 1 -- 172.21.15.47:0/242100848 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd44c12f030 con 0x7fd44c12c750 2024-02-23T00:48:53.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd43b7fe700 10 client.41387.objecter ms_dispatch 0x7fd44c001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:48:53.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd43b7fe700 10 client.41387.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:48:53.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd43b7fe700 20 client.41387.objecter put_session s=0x7fd44c124680 osd=7 4 2024-02-23T00:48:53.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd43b7fe700 15 client.41387.objecter _session_command_op_remove 7 16 2024-02-23T00:48:53.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 10 librados: watch_flush enter 2024-02-23T00:48:53.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 10 librados: watch_flush exit 2024-02-23T00:48:53.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 10 client.41387.objecter close_session for osd.0 2024-02-23T00:48:53.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fd44c028540 msgr2=0x7fd44c0fbca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:53.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fd44c028540 0x7fd44c0fbca0 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:53.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c05ee80 osd=0 1 2024-02-23T00:48:53.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 10 client.41387.objecter close_session for osd.1 2024-02-23T00:48:53.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fd44c027b90 msgr2=0x7fd44c027f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:53.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fd44c027b90 0x7fd44c027f70 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:53.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c0fd310 osd=1 1 2024-02-23T00:48:53.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 10 client.41387.objecter close_session for osd.2 2024-02-23T00:48:53.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fd44c0fce70 msgr2=0x7fd44c11c0c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:53.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fd44c0fce70 0x7fd44c11c0c0 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:53.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c027a10 osd=2 1 2024-02-23T00:48:53.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 10 client.41387.objecter close_session for osd.3 2024-02-23T00:48:53.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fd44c119c10 msgr2=0x7fd44c11c600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:53.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd44ad9d700 1 -- 172.21.15.47:0/242100848 reap_dead start 2024-02-23T00:48:53.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fd44c119c10 0x7fd44c11c600 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:53.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c0fc7e0 osd=3 1 2024-02-23T00:48:53.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 10 client.41387.objecter close_session for osd.4 2024-02-23T00:48:53.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fd44c1192e0 msgr2=0x7fd44c123070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:53.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fd44c1192e0 0x7fd44c123070 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:53.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c119170 osd=4 1 2024-02-23T00:48:53.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 10 client.41387.objecter close_session for osd.5 2024-02-23T00:48:53.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fd44c11e3e0 msgr2=0x7fd44c11e840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:53.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fd44c11e3e0 0x7fd44c11e840 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:53.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c118f30 osd=5 1 2024-02-23T00:48:53.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 10 client.41387.objecter close_session for osd.6 2024-02-23T00:48:53.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fd44c123c80 msgr2=0x7fd44c1240e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:53.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fd44c123c80 0x7fd44c1240e0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:53.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c123600 osd=6 1 2024-02-23T00:48:53.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 10 client.41387.objecter close_session for osd.7 2024-02-23T00:48:53.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fd44c12c750 msgr2=0x7fd44c12eb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:53.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fd44c12c750 0x7fd44c12eb40 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:53.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 20 client.41387.objecter put_session s=0x7fd44c124680 osd=7 1 2024-02-23T00:48:53.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 20 client.41387.objecter shutdown clearing up homeless session... 2024-02-23T00:48:53.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 10 client.41387.objecter successfully canceled tick 2024-02-23T00:48:53.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd44ad9d700 1 -- 172.21.15.47:0/242100848 reap_dead start 2024-02-23T00:48:53.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd434062cc0 msgr2=0x7fd434065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:53.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd434062cc0 0x7fd434065180 secure :-1 s=READY pgs=3073 cs=0 l=1 rev1=1 rx=0x7fd440007fd0 tx=0x7fd44000b040).stop 2024-02-23T00:48:53.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd44c0f8d10 msgr2=0x7fd44c10e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:53.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd44c0f8d10 0x7fd44c10e080 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 rx=0x7fd43c004920 tx=0x7fd43c009cb0).stop 2024-02-23T00:48:53.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 shutdown_connections 2024-02-23T00:48:53.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd44c0f8d10 0x7fd44c10e080 unknown :-1 s=CLOSED pgs=2677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:53.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 1 --2- 172.21.15.47:0/242100848 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd434062cc0 0x7fd434065180 unknown :-1 s=CLOSED pgs=3073 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:53.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 >> 172.21.15.47:0/242100848 conn(0x7fd44c0f2820 msgr2=0x7fd44c0f3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:48:53.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 shutdown_connections 2024-02-23T00:48:53.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 1 -- 172.21.15.47:0/242100848 wait complete. 2024-02-23T00:48:53.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:53.415+0000 7fd450ca3700 1 librados: shutdown 2024-02-23T00:48:53.989 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:48:53.989 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:48:53.989 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:48:53.989 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:48:53.989 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:48:53.989 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:48:53.989 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:48:53.989 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:48:54.279 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:48:54.285 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:54.286 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:48:54.386 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:48:54.415 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:54.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:52.413316+0000 osd.5 (osd.5) 184 : cluster [DBG] 18.17 scrub starts 2024-02-23T00:48:54.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:52.416483+0000 osd.5 (osd.5) 185 : cluster [DBG] 18.17 scrub ok 2024-02-23T00:48:54.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:53.245128+0000 osd.6 (osd.6) 177 : cluster [DBG] 5.1f scrub starts 2024-02-23T00:48:54.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:53.246529+0000 osd.6 (osd.6) 178 : cluster [DBG] 5.1f scrub ok 2024-02-23T00:48:54.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:53.272625+0000 mon.c (mon.2) 303 : audit [DBG] from='client.? 172.21.15.47:0/242100848' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:48:54.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:53.298690+0000 mgr.y (mgr.14517) 410 : cluster [DBG] pgmap v646: 208 pgs: 1 active+clean+snaptrim, 207 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 3.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-02-23T00:48:54.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:53.694587+0000 osd.7 (osd.7) 215 : cluster [DBG] 62.8 scrub starts 2024-02-23T00:48:54.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:53.698461+0000 osd.7 (osd.7) 216 : cluster [DBG] 62.8 scrub ok 2024-02-23T00:48:54.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:52.413316+0000 osd.5 (osd.5) 184 : cluster [DBG] 18.17 scrub starts 2024-02-23T00:48:54.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:52.416483+0000 osd.5 (osd.5) 185 : cluster [DBG] 18.17 scrub ok 2024-02-23T00:48:54.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:53.245128+0000 osd.6 (osd.6) 177 : cluster [DBG] 5.1f scrub starts 2024-02-23T00:48:54.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:53.246529+0000 osd.6 (osd.6) 178 : cluster [DBG] 5.1f scrub ok 2024-02-23T00:48:54.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:53.272625+0000 mon.c (mon.2) 303 : audit [DBG] from='client.? 172.21.15.47:0/242100848' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:48:54.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:53.298690+0000 mgr.y (mgr.14517) 410 : cluster [DBG] pgmap v646: 208 pgs: 1 active+clean+snaptrim, 207 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 3.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-02-23T00:48:54.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:53.694587+0000 osd.7 (osd.7) 215 : cluster [DBG] 62.8 scrub starts 2024-02-23T00:48:54.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:53.698461+0000 osd.7 (osd.7) 216 : cluster [DBG] 62.8 scrub ok 2024-02-23T00:48:54.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:52.413316+0000 osd.5 (osd.5) 184 : cluster [DBG] 18.17 scrub starts 2024-02-23T00:48:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:52.416483+0000 osd.5 (osd.5) 185 : cluster [DBG] 18.17 scrub ok 2024-02-23T00:48:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:53.245128+0000 osd.6 (osd.6) 177 : cluster [DBG] 5.1f scrub starts 2024-02-23T00:48:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:53.246529+0000 osd.6 (osd.6) 178 : cluster [DBG] 5.1f scrub ok 2024-02-23T00:48:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:53.272625+0000 mon.c (mon.2) 303 : audit [DBG] from='client.? 172.21.15.47:0/242100848' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:48:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:53.298690+0000 mgr.y (mgr.14517) 410 : cluster [DBG] pgmap v646: 208 pgs: 1 active+clean+snaptrim, 207 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 3.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-02-23T00:48:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:53.694587+0000 osd.7 (osd.7) 215 : cluster [DBG] 62.8 scrub starts 2024-02-23T00:48:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:53.698461+0000 osd.7 (osd.7) 216 : cluster [DBG] 62.8 scrub ok 2024-02-23T00:48:54.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.995+0000 7fe309314700 1 -- 172.21.15.47:0/2151207090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3040f9ab0 msgr2=0x7fe3040f9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:54.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.995+0000 7fe309314700 1 --2- 172.21.15.47:0/2151207090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3040f9ab0 0x7fe3040f9e90 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 rx=0x7fe2f40047b0 tx=0x7fe2f4009cb0).stop 2024-02-23T00:48:54.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.996+0000 7fe309314700 1 -- 172.21.15.47:0/2151207090 shutdown_connections 2024-02-23T00:48:54.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.996+0000 7fe309314700 1 --2- 172.21.15.47:0/2151207090 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe3040f8440 0x7fe304104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:54.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.996+0000 7fe309314700 1 --2- 172.21.15.47:0/2151207090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3040f9ab0 0x7fe3040f9e90 unknown :-1 s=CLOSED pgs=2538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:54.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.996+0000 7fe309314700 1 --2- 172.21.15.47:0/2151207090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3040fa380 0x7fe3040f7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:54.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.996+0000 7fe309314700 1 -- 172.21.15.47:0/2151207090 >> 172.21.15.47:0/2151207090 conn(0x7fe3040f3800 msgr2=0x7fe3040f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:48:54.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.996+0000 7fe309314700 1 -- 172.21.15.47:0/2151207090 shutdown_connections 2024-02-23T00:48:54.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.996+0000 7fe309314700 1 -- 172.21.15.47:0/2151207090 wait complete. 2024-02-23T00:48:54.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.996+0000 7fe309314700 1 librados: starting msgr at 2024-02-23T00:48:54.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.996+0000 7fe309314700 1 librados: starting objecter 2024-02-23T00:48:54.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.996+0000 7fe309314700 1 Processor -- start 2024-02-23T00:48:54.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.996+0000 7fe309314700 1 -- start start 2024-02-23T00:48:54.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.996+0000 7fe309314700 1 librados: setting wanted keys 2024-02-23T00:48:54.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.996+0000 7fe309314700 1 librados: calling monclient init 2024-02-23T00:48:55.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.997+0000 7fe309314700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3040f8440 0x7fe30410e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:55.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.997+0000 7fe309314700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3040f9ab0 0x7fe30410e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:55.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.997+0000 7fe309314700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe3040fa380 0x7fe3041081a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:55.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.997+0000 7fe309314700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe304108690 con 0x7fe3040f8440 2024-02-23T00:48:55.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.997+0000 7fe309314700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe304108800 con 0x7fe3040f9ab0 2024-02-23T00:48:55.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.997+0000 7fe309314700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3041089c0 con 0x7fe3040fa380 2024-02-23T00:48:55.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.997+0000 7fe3037fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe3040fa380 0x7fe3041081a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:55.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.997+0000 7fe3037fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe3040fa380 0x7fe3041081a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:43676/0 (socket says 172.21.15.47:43676) 2024-02-23T00:48:55.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.997+0000 7fe3037fe700 1 -- 172.21.15.47:0/3895659704 learned_addr learned my addr 172.21.15.47:0/3895659704 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:48:55.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.997+0000 7fe302ffd700 1 --2- 172.21.15.47:0/3895659704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3040f8440 0x7fe30410e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:55.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.997+0000 7fe3027fc700 1 --2- 172.21.15.47:0/3895659704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3040f9ab0 0x7fe30410e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:55.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.997+0000 7fe3037fe700 1 -- 172.21.15.47:0/3895659704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3040f9ab0 msgr2=0x7fe30410e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:55.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.997+0000 7fe3037fe700 1 --2- 172.21.15.47:0/3895659704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3040f9ab0 0x7fe30410e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:55.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.997+0000 7fe3037fe700 1 -- 172.21.15.47:0/3895659704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3040f8440 msgr2=0x7fe30410e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:55.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.997+0000 7fe3037fe700 1 --2- 172.21.15.47:0/3895659704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3040f8440 0x7fe30410e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:55.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.997+0000 7fe3037fe700 1 -- 172.21.15.47:0/3895659704 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2fc0097c0 con 0x7fe3040fa380 2024-02-23T00:48:55.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.997+0000 7fe302ffd700 1 --2- 172.21.15.47:0/3895659704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3040f8440 0x7fe30410e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:48:55.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.997+0000 7fe3037fe700 1 --2- 172.21.15.47:0/3895659704 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe3040fa380 0x7fe3041081a0 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 rx=0x7fe2fc001df0 tx=0x7fe2fc004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:55.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.997+0000 7fe2ebfff700 10 client.?.objecter ms_handle_connect 0x7fe3040fa380 2024-02-23T00:48:55.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.997+0000 7fe2ebfff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:48:55.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.998+0000 7fe2ebfff700 1 -- 172.21.15.47:0/3895659704 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc009a10 con 0x7fe3040fa380 2024-02-23T00:48:55.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.998+0000 7fe309314700 1 -- 172.21.15.47:0/3895659704 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f4005040 con 0x7fe3040fa380 2024-02-23T00:48:55.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.998+0000 7fe2ebfff700 1 -- 172.21.15.47:0/3895659704 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fe2fc0090d0 con 0x7fe3040fa380 2024-02-23T00:48:55.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.998+0000 7fe2ebfff700 1 -- 172.21.15.47:0/3895659704 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc007be0 con 0x7fe3040fa380 2024-02-23T00:48:55.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.998+0000 7fe309314700 10 client.41393.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:48:55.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.998+0000 7fe309314700 1 -- 172.21.15.47:0/3895659704 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe304109000 con 0x7fe3040fa380 2024-02-23T00:48:55.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.998+0000 7fe309314700 1 librados: init done 2024-02-23T00:48:55.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.999+0000 7fe309314700 1 -- 172.21.15.47:0/3895659704 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe30405ee80 con 0x7fe3040fa380 2024-02-23T00:48:55.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.999+0000 7fe2ebfff700 1 -- 172.21.15.47:0/3895659704 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fe2fc00fb30 con 0x7fe3040fa380 2024-02-23T00:48:55.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:54.999+0000 7fe2ebfff700 1 --2- 172.21.15.47:0/3895659704 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe2ec062cc0 0x7fe2ec065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:48:55.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:55.000+0000 7fe2ebfff700 1 -- 172.21.15.47:0/3895659704 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(440..440 src has 48..440) v4 ==== 9276+0+0 (secure 0 0 0) 0x7fe2fc07df50 con 0x7fe3040fa380 2024-02-23T00:48:55.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:55.000+0000 7fe2ebfff700 10 client.41393.objecter ms_dispatch 0x7fe304001550 osd_map(440..440 src has 48..440) v4 2024-02-23T00:48:55.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:55.000+0000 7fe2ebfff700 3 client.41393.objecter handle_osd_map got epochs [440,440] > 0 2024-02-23T00:48:55.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:55.000+0000 7fe2ebfff700 3 client.41393.objecter handle_osd_map decoding full epoch 440 2024-02-23T00:48:55.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:55.000+0000 7fe302ffd700 1 --2- 172.21.15.47:0/3895659704 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe2ec062cc0 0x7fe2ec065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:48:55.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:55.000+0000 7fe2ebfff700 20 client.41393.objecter dump_active .. 0 homeless 2024-02-23T00:48:55.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:55.000+0000 7fe2ebfff700 10 client.41393.objecter ms_handle_connect 0x7fe2ec062cc0 2024-02-23T00:48:55.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:55.000+0000 7fe302ffd700 1 --2- 172.21.15.47:0/3895659704 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe2ec062cc0 0x7fe2ec065180 secure :-1 s=READY pgs=3074 cs=0 l=1 rev1=1 rx=0x7fe2f40044a0 tx=0x7fe2f4009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:48:55.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:55.004+0000 7fe2ebfff700 1 -- 172.21.15.47:0/3895659704 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe2fc04d5b0 con 0x7fe3040fa380 2024-02-23T00:48:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:55.187+0000 7fe309314700 1 -- 172.21.15.47:0/3895659704 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7fe304109680 con 0x7fe3040fa380 2024-02-23T00:48:55.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:55.196+0000 7fe2ebfff700 1 -- 172.21.15.47:0/3895659704 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v441) v1 ==== 101+0+0 (secure 0 0 0) 0x7fe2fc050bd0 con 0x7fe3040fa380 2024-02-23T00:48:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:55.199+0000 7fe309314700 10 librados: watch_flush enter 2024-02-23T00:48:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:55.199+0000 7fe309314700 10 librados: watch_flush exit 2024-02-23T00:48:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:55.199+0000 7fe309314700 20 client.41393.objecter shutdown clearing up homeless session... 2024-02-23T00:48:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:55.199+0000 7fe309314700 10 client.41393.objecter successfully canceled tick 2024-02-23T00:48:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:55.199+0000 7fe309314700 1 -- 172.21.15.47:0/3895659704 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe2ec062cc0 msgr2=0x7fe2ec065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:55.199+0000 7fe309314700 1 --2- 172.21.15.47:0/3895659704 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe2ec062cc0 0x7fe2ec065180 secure :-1 s=READY pgs=3074 cs=0 l=1 rev1=1 rx=0x7fe2f40044a0 tx=0x7fe2f4009cb0).stop 2024-02-23T00:48:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:55.199+0000 7fe309314700 1 -- 172.21.15.47:0/3895659704 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe3040fa380 msgr2=0x7fe3041081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:48:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:55.199+0000 7fe309314700 1 --2- 172.21.15.47:0/3895659704 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe3040fa380 0x7fe3041081a0 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 rx=0x7fe2fc001df0 tx=0x7fe2fc004d10).stop 2024-02-23T00:48:55.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:55.200+0000 7fe309314700 1 -- 172.21.15.47:0/3895659704 shutdown_connections 2024-02-23T00:48:55.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:55.200+0000 7fe309314700 1 --2- 172.21.15.47:0/3895659704 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe2ec062cc0 0x7fe2ec065180 unknown :-1 s=CLOSED pgs=3074 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:55.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:55.200+0000 7fe309314700 1 --2- 172.21.15.47:0/3895659704 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe3040fa380 0x7fe3041081a0 unknown :-1 s=CLOSED pgs=2678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:55.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:55.200+0000 7fe309314700 1 --2- 172.21.15.47:0/3895659704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3040f8440 0x7fe30410e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:55.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:55.200+0000 7fe309314700 1 --2- 172.21.15.47:0/3895659704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3040f9ab0 0x7fe30410e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:48:55.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:55.200+0000 7fe309314700 1 -- 172.21.15.47:0/3895659704 >> 172.21.15.47:0/3895659704 conn(0x7fe3040f3800 msgr2=0x7fe3040fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:48:55.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:55.200+0000 7fe309314700 1 -- 172.21.15.47:0/3895659704 shutdown_connections 2024-02-23T00:48:55.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:55.200+0000 7fe309314700 1 -- 172.21.15.47:0/3895659704 wait complete. 2024-02-23T00:48:55.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:48:55.200+0000 7fe309314700 1 librados: shutdown 2024-02-23T00:48:55.203 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is unset 2024-02-23T00:48:55.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:53.746325+0000 osd.0 (osd.0) 179 : cluster [DBG] 5.1e scrub starts 2024-02-23T00:48:55.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:53.747239+0000 osd.0 (osd.0) 180 : cluster [DBG] 5.1e scrub ok 2024-02-23T00:48:55.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:54.211400+0000 osd.4 (osd.4) 177 : cluster [DBG] 62.16 scrub starts 2024-02-23T00:48:55.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:54.213506+0000 osd.4 (osd.4) 178 : cluster [DBG] 62.16 scrub ok 2024-02-23T00:48:55.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:55.187982+0000 mon.c (mon.2) 304 : audit [INF] from='client.? 172.21.15.47:0/3895659704' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:48:55.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:55.188347+0000 mon.a (mon.0) 2617 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:48:55.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:55.191623+0000 mon.a (mon.0) 2618 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:48:55.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:55.195649+0000 mon.a (mon.0) 2619 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:48:55.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:55.195717+0000 mon.a (mon.0) 2620 : cluster [DBG] osdmap e441: 8 total, 8 up, 7 in 2024-02-23T00:48:55.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:53.746325+0000 osd.0 (osd.0) 179 : cluster [DBG] 5.1e scrub starts 2024-02-23T00:48:55.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:53.747239+0000 osd.0 (osd.0) 180 : cluster [DBG] 5.1e scrub ok 2024-02-23T00:48:55.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:54.211400+0000 osd.4 (osd.4) 177 : cluster [DBG] 62.16 scrub starts 2024-02-23T00:48:55.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:54.213506+0000 osd.4 (osd.4) 178 : cluster [DBG] 62.16 scrub ok 2024-02-23T00:48:55.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:55.187982+0000 mon.c (mon.2) 304 : audit [INF] from='client.? 172.21.15.47:0/3895659704' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:48:55.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:55.188347+0000 mon.a (mon.0) 2617 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:48:55.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:55.191623+0000 mon.a (mon.0) 2618 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:48:55.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:55.195649+0000 mon.a (mon.0) 2619 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:48:55.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:55.195717+0000 mon.a (mon.0) 2620 : cluster [DBG] osdmap e441: 8 total, 8 up, 7 in 2024-02-23T00:48:55.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:53.746325+0000 osd.0 (osd.0) 179 : cluster [DBG] 5.1e scrub starts 2024-02-23T00:48:55.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:53.747239+0000 osd.0 (osd.0) 180 : cluster [DBG] 5.1e scrub ok 2024-02-23T00:48:55.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:54.211400+0000 osd.4 (osd.4) 177 : cluster [DBG] 62.16 scrub starts 2024-02-23T00:48:55.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:54.213506+0000 osd.4 (osd.4) 178 : cluster [DBG] 62.16 scrub ok 2024-02-23T00:48:55.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:55.187982+0000 mon.c (mon.2) 304 : audit [INF] from='client.? 172.21.15.47:0/3895659704' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:48:55.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:55.188347+0000 mon.a (mon.0) 2617 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:48:55.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:55.191623+0000 mon.a (mon.0) 2618 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:48:55.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:55.195649+0000 mon.a (mon.0) 2619 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:48:55.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:55.195717+0000 mon.a (mon.0) 2620 : cluster [DBG] osdmap e441: 8 total, 8 up, 7 in 2024-02-23T00:48:56.165 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:48:56.168 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:56.168 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:48:56.269 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:48:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:56.365 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-23T00:48:56.365 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-23T00:48:56.365 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-02-23T00:48:56.365 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph tell osd.0 reset_purged_snaps_last 2024-02-23T00:48:56.391 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:54.705517+0000 osd.0 (osd.0) 181 : cluster [DBG] 5.8 scrub starts 2024-02-23T00:48:56.391 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:54.706622+0000 osd.0 (osd.0) 182 : cluster [DBG] 5.8 scrub ok 2024-02-23T00:48:56.391 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:55.299335+0000 mgr.y (mgr.14517) 411 : cluster [DBG] pgmap v648: 208 pgs: 1 active+clean+snaptrim, 207 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-23T00:48:56.391 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:54.705517+0000 osd.0 (osd.0) 181 : cluster [DBG] 5.8 scrub starts 2024-02-23T00:48:56.391 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:54.706622+0000 osd.0 (osd.0) 182 : cluster [DBG] 5.8 scrub ok 2024-02-23T00:48:56.392 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:55.299335+0000 mgr.y (mgr.14517) 411 : cluster [DBG] pgmap v648: 208 pgs: 1 active+clean+snaptrim, 207 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-23T00:48:56.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:54.705517+0000 osd.0 (osd.0) 181 : cluster [DBG] 5.8 scrub starts 2024-02-23T00:48:56.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:54.706622+0000 osd.0 (osd.0) 182 : cluster [DBG] 5.8 scrub ok 2024-02-23T00:48:56.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:55.299335+0000 mgr.y (mgr.14517) 411 : cluster [DBG] pgmap v648: 208 pgs: 1 active+clean+snaptrim, 207 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-23T00:48:56.729 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:48:56.729 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-02-23T00:48:56.729 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-02-23T00:48:56.730 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-02-23T00:48:56.730 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:48:57.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:55.585676+0000 osd.1 (osd.1) 193 : cluster [DBG] 4.2 scrub starts 2024-02-23T00:48:57.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:55.589202+0000 osd.1 (osd.1) 194 : cluster [DBG] 4.2 scrub ok 2024-02-23T00:48:57.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:48:56.303669+0000 mon.a (mon.0) 2621 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:48:57.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:55.585676+0000 osd.1 (osd.1) 193 : cluster [DBG] 4.2 scrub starts 2024-02-23T00:48:57.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:55.589202+0000 osd.1 (osd.1) 194 : cluster [DBG] 4.2 scrub ok 2024-02-23T00:48:57.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:48:56.303669+0000 mon.a (mon.0) 2621 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:48:57.482 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.3 dump_historic_ops 2024-02-23T00:48:57.552 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:48:57.556 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:57.556 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:48:57.581 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:48:57.658 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:48:57.687 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:55.585676+0000 osd.1 (osd.1) 193 : cluster [DBG] 4.2 scrub starts 2024-02-23T00:48:57.688 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:55.589202+0000 osd.1 (osd.1) 194 : cluster [DBG] 4.2 scrub ok 2024-02-23T00:48:57.688 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:48:56.303669+0000 mon.a (mon.0) 2621 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:48:57.690 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:57.711 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:48:57.714 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:57.714 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:48:57.815 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:48:57.844 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:57.863 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:48:57.866 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:57.867 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:48:57.967 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:48:57.999 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:58.020 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:48:58.022 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:58.023 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:48:58.123 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:48:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:58.161 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:48:58.403 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:48:58.408 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:58.409 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:48:58.409 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:56.701742+0000 osd.0 (osd.0) 183 : cluster [DBG] 62.4 scrub starts 2024-02-23T00:48:58.409 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:56.703238+0000 osd.0 (osd.0) 184 : cluster [DBG] 62.4 scrub ok 2024-02-23T00:48:58.409 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:57.166430+0000 osd.3 (osd.3) 204 : cluster [DBG] 18.1 scrub starts 2024-02-23T00:48:58.409 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:57.167975+0000 osd.3 (osd.3) 205 : cluster [DBG] 18.1 scrub ok 2024-02-23T00:48:58.409 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:57.300206+0000 mgr.y (mgr.14517) 412 : cluster [DBG] pgmap v649: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.8 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-02-23T00:48:58.409 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:57.427174+0000 osd.5 (osd.5) 186 : cluster [DBG] 5.6 scrub starts 2024-02-23T00:48:58.409 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:48:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:57.430122+0000 osd.5 (osd.5) 187 : cluster [DBG] 5.6 scrub ok 2024-02-23T00:48:58.409 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:56.701742+0000 osd.0 (osd.0) 183 : cluster [DBG] 62.4 scrub starts 2024-02-23T00:48:58.410 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:56.703238+0000 osd.0 (osd.0) 184 : cluster [DBG] 62.4 scrub ok 2024-02-23T00:48:58.410 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:57.166430+0000 osd.3 (osd.3) 204 : cluster [DBG] 18.1 scrub starts 2024-02-23T00:48:58.410 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:57.167975+0000 osd.3 (osd.3) 205 : cluster [DBG] 18.1 scrub ok 2024-02-23T00:48:58.410 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:57.300206+0000 mgr.y (mgr.14517) 412 : cluster [DBG] pgmap v649: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.8 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-02-23T00:48:58.410 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:57.427174+0000 osd.5 (osd.5) 186 : cluster [DBG] 5.6 scrub starts 2024-02-23T00:48:58.410 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:48:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:57.430122+0000 osd.5 (osd.5) 187 : cluster [DBG] 5.6 scrub ok 2024-02-23T00:48:58.411 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:48:58.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:56.701742+0000 osd.0 (osd.0) 183 : cluster [DBG] 62.4 scrub starts 2024-02-23T00:48:58.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:56.703238+0000 osd.0 (osd.0) 184 : cluster [DBG] 62.4 scrub ok 2024-02-23T00:48:58.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:57.166430+0000 osd.3 (osd.3) 204 : cluster [DBG] 18.1 scrub starts 2024-02-23T00:48:58.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:57.167975+0000 osd.3 (osd.3) 205 : cluster [DBG] 18.1 scrub ok 2024-02-23T00:48:58.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:57.300206+0000 mgr.y (mgr.14517) 412 : cluster [DBG] pgmap v649: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.8 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-02-23T00:48:58.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:57.427174+0000 osd.5 (osd.5) 186 : cluster [DBG] 5.6 scrub starts 2024-02-23T00:48:58.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:48:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:57.430122+0000 osd.5 (osd.5) 187 : cluster [DBG] 5.6 scrub ok 2024-02-23T00:48:58.455 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:48:58.509 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:48:58.541 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:58.563 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:48:58.565 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:58.566 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:48:58.600 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set noscrub 2024-02-23T00:48:58.667 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:48:58.696 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:58.716 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:48:58.718 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:58.719 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:48:58.819 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:48:58.851 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:58.872 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:48:58.875 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:58.876 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:48:58.977 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:48:59.004 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:59.354 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:48:59.358 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:59.359 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:48:59.459 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:48:59.488 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:48:59.687 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:48:59.689 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:48:59.690 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:48:59.790 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:48:59.818 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:00.468 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:49:00.473 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:00.473 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:49:00.574 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:49:00.605 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:00.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:59.205188+0000 osd.4 (osd.4) 179 : cluster [DBG] 18.19 scrub starts 2024-02-23T00:49:00.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:59.206245+0000 osd.4 (osd.4) 180 : cluster [DBG] 18.19 scrub ok 2024-02-23T00:49:00.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:59.300683+0000 mgr.y (mgr.14517) 413 : cluster [DBG] pgmap v650: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-23T00:49:00.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:00.197848+0000 mon.a (mon.0) 2622 : cluster [DBG] osdmap e442: 8 total, 8 up, 7 in 2024-02-23T00:49:00.708 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:59.205188+0000 osd.4 (osd.4) 179 : cluster [DBG] 18.19 scrub starts 2024-02-23T00:49:00.708 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:59.206245+0000 osd.4 (osd.4) 180 : cluster [DBG] 18.19 scrub ok 2024-02-23T00:49:00.708 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:59.300683+0000 mgr.y (mgr.14517) 413 : cluster [DBG] pgmap v650: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-23T00:49:00.709 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:00.197848+0000 mon.a (mon.0) 2622 : cluster [DBG] osdmap e442: 8 total, 8 up, 7 in 2024-02-23T00:49:00.709 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:59.205188+0000 osd.4 (osd.4) 179 : cluster [DBG] 18.19 scrub starts 2024-02-23T00:49:00.709 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:59.206245+0000 osd.4 (osd.4) 180 : cluster [DBG] 18.19 scrub ok 2024-02-23T00:49:00.709 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:59.300683+0000 mgr.y (mgr.14517) 413 : cluster [DBG] pgmap v650: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-23T00:49:00.709 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:00.197848+0000 mon.a (mon.0) 2622 : cluster [DBG] osdmap e442: 8 total, 8 up, 7 in 2024-02-23T00:49:00.709 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:49:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:49:00] "GET /metrics HTTP/1.1" 200 189093 "" "Prometheus/2.33.4" 2024-02-23T00:49:01.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:59.711851+0000 osd.0 (osd.0) 185 : cluster [DBG] 2.13 scrub starts 2024-02-23T00:49:01.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:48:59.712693+0000 osd.0 (osd.0) 186 : cluster [DBG] 2.13 scrub ok 2024-02-23T00:49:01.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:00.240623+0000 osd.4 (osd.4) 181 : cluster [DBG] 62.9 scrub starts 2024-02-23T00:49:01.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:00.243064+0000 osd.4 (osd.4) 182 : cluster [DBG] 62.9 scrub ok 2024-02-23T00:49:01.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:01.223613+0000 mon.a (mon.0) 2623 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:49:01.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:01.224047+0000 mon.a (mon.0) 2624 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-63"}]: dispatch 2024-02-23T00:49:01.635 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:49:01.640 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:59.711851+0000 osd.0 (osd.0) 185 : cluster [DBG] 2.13 scrub starts 2024-02-23T00:49:01.640 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:48:59.712693+0000 osd.0 (osd.0) 186 : cluster [DBG] 2.13 scrub ok 2024-02-23T00:49:01.640 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:00.240623+0000 osd.4 (osd.4) 181 : cluster [DBG] 62.9 scrub starts 2024-02-23T00:49:01.640 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:00.243064+0000 osd.4 (osd.4) 182 : cluster [DBG] 62.9 scrub ok 2024-02-23T00:49:01.641 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:01.223613+0000 mon.a (mon.0) 2623 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:49:01.641 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:01.224047+0000 mon.a (mon.0) 2624 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-63"}]: dispatch 2024-02-23T00:49:01.642 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:01.642 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:49:01.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:59.711851+0000 osd.0 (osd.0) 185 : cluster [DBG] 2.13 scrub starts 2024-02-23T00:49:01.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:48:59.712693+0000 osd.0 (osd.0) 186 : cluster [DBG] 2.13 scrub ok 2024-02-23T00:49:01.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:00.240623+0000 osd.4 (osd.4) 181 : cluster [DBG] 62.9 scrub starts 2024-02-23T00:49:01.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:00.243064+0000 osd.4 (osd.4) 182 : cluster [DBG] 62.9 scrub ok 2024-02-23T00:49:01.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:01.223613+0000 mon.a (mon.0) 2623 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:49:01.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:01.224047+0000 mon.a (mon.0) 2624 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-63"}]: dispatch 2024-02-23T00:49:01.743 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:49:01.776 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:01.797 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:49:01.799 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:01.800 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:49:01.900 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:49:01.932 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:02.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.045+0000 7f8eb52d1700 1 -- 172.21.15.47:0/2903985695 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8eb00f7280 msgr2=0x7f8eb0103880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:02.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.045+0000 7f8eb52d1700 1 --2- 172.21.15.47:0/2903985695 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8eb00f7280 0x7f8eb0103880 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 rx=0x7f8ea00047b0 tx=0x7f8ea0009cb0).stop 2024-02-23T00:49:02.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.046+0000 7f8eb52d1700 1 -- 172.21.15.47:0/2903985695 shutdown_connections 2024-02-23T00:49:02.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.046+0000 7f8eb52d1700 1 --2- 172.21.15.47:0/2903985695 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8eb00f7280 0x7f8eb0103880 unknown :-1 s=CLOSED pgs=2679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:02.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.046+0000 7f8eb52d1700 1 --2- 172.21.15.47:0/2903985695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8eb00fca10 0x7f8eb00fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:49:02.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.046+0000 7f8eb52d1700 1 --2- 172.21.15.47:0/2903985695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8eb00fd370 0x7f8eb00f6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:02.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.046+0000 7f8eb52d1700 1 -- 172.21.15.47:0/2903985695 >> 172.21.15.47:0/2903985695 conn(0x7f8eb00f2660 msgr2=0x7f8eb00f4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:02.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.046+0000 7f8eb52d1700 1 -- 172.21.15.47:0/2903985695 shutdown_connections 2024-02-23T00:49:02.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.046+0000 7f8eb52d1700 1 -- 172.21.15.47:0/2903985695 wait complete. 2024-02-23T00:49:02.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.046+0000 7f8eb52d1700 1 librados: starting msgr at 2024-02-23T00:49:02.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.046+0000 7f8eb52d1700 1 librados: starting objecter 2024-02-23T00:49:02.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.046+0000 7f8eb52d1700 1 Processor -- start 2024-02-23T00:49:02.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.046+0000 7f8eb52d1700 1 -- start start 2024-02-23T00:49:02.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.046+0000 7f8eb52d1700 1 librados: setting wanted keys 2024-02-23T00:49:02.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.046+0000 7f8eb52d1700 1 librados: calling monclient init 2024-02-23T00:49:02.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.047+0000 7f8eb52d1700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8eb00f7280 0x7f8eb0108db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:02.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.047+0000 7f8eb52d1700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8eb00fca10 0x7f8eb01092a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:02.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.047+0000 7f8eb52d1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8eb00fd370 0x7f8eb0109860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:02.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.047+0000 7f8eb52d1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb0113fa0 con 0x7f8eb00fd370 2024-02-23T00:49:02.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.047+0000 7f8eb52d1700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb0114110 con 0x7f8eb00f7280 2024-02-23T00:49:02.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.047+0000 7f8eb52d1700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb01142d0 con 0x7f8eb00fca10 2024-02-23T00:49:02.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.047+0000 7f8eae7fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8eb00fca10 0x7f8eb01092a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:02.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.047+0000 7f8eaf7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8eb00fd370 0x7f8eb0109860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:02.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.047+0000 7f8eae7fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8eb00fca10 0x7f8eb01092a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:45618/0 (socket says 172.21.15.47:45618) 2024-02-23T00:49:02.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.047+0000 7f8eae7fc700 1 -- 172.21.15.47:0/877427356 learned_addr learned my addr 172.21.15.47:0/877427356 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:49:02.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.047+0000 7f8eaeffd700 1 --2- 172.21.15.47:0/877427356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8eb00f7280 0x7f8eb0108db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:02.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.047+0000 7f8eae7fc700 1 -- 172.21.15.47:0/877427356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8eb00f7280 msgr2=0x7f8eb0108db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:02.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.047+0000 7f8eae7fc700 1 --2- 172.21.15.47:0/877427356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8eb00f7280 0x7f8eb0108db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:02.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.047+0000 7f8eae7fc700 1 -- 172.21.15.47:0/877427356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8eb00fd370 msgr2=0x7f8eb0109860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:02.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.047+0000 7f8eae7fc700 1 --2- 172.21.15.47:0/877427356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8eb00fd370 0x7f8eb0109860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:02.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.047+0000 7f8eae7fc700 1 -- 172.21.15.47:0/877427356 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea0005040 con 0x7f8eb00fca10 2024-02-23T00:49:02.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.047+0000 7f8eaeffd700 1 --2- 172.21.15.47:0/877427356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8eb00f7280 0x7f8eb0108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:49:02.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.047+0000 7f8eaf7fe700 1 --2- 172.21.15.47:0/877427356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8eb00fd370 0x7f8eb0109860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:49:02.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.047+0000 7f8eae7fc700 1 --2- 172.21.15.47:0/877427356 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8eb00fca10 0x7f8eb01092a0 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 rx=0x7f8ea400e970 tx=0x7f8ea400b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:02.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.047+0000 7f8e97fff700 10 client.?.objecter ms_handle_connect 0x7f8eb00fca10 2024-02-23T00:49:02.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.047+0000 7f8e97fff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:49:02.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.047+0000 7f8e97fff700 1 -- 172.21.15.47:0/877427356 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea40046e0 con 0x7f8eb00fca10 2024-02-23T00:49:02.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.047+0000 7f8e97fff700 1 -- 172.21.15.47:0/877427356 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f8ea400f040 con 0x7f8eb00fca10 2024-02-23T00:49:02.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.047+0000 7f8eb52d1700 1 -- 172.21.15.47:0/877427356 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eb0106c00 con 0x7f8eb00fca10 2024-02-23T00:49:02.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.047+0000 7f8e97fff700 1 -- 172.21.15.47:0/877427356 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea400c5b0 con 0x7f8eb00fca10 2024-02-23T00:49:02.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.048+0000 7f8eb52d1700 10 client.41399.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:49:02.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.048+0000 7f8eb52d1700 1 -- 172.21.15.47:0/877427356 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb01071c0 con 0x7f8eb00fca10 2024-02-23T00:49:02.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.048+0000 7f8eb52d1700 1 librados: init done 2024-02-23T00:49:02.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.048+0000 7f8eb52d1700 10 client.41399.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:02.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.048+0000 7f8eb52d1700 15 client.41399.objecter _session_command_op_assign -1 1 2024-02-23T00:49:02.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.048+0000 7f8eb52d1700 20 client.41399.objecter _get_session osd=-1 returning homeless 2024-02-23T00:49:02.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.048+0000 7f8eb52d1700 10 client.41399.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:49:02.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.048+0000 7f8eb52d1700 1 -- 172.21.15.47:0/877427356 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8eb0047540 con 0x7f8eb00fca10 2024-02-23T00:49:02.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.049+0000 7f8e97fff700 1 -- 172.21.15.47:0/877427356 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f8ea4007b40 con 0x7f8eb00fca10 2024-02-23T00:49:02.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.049+0000 7f8e97fff700 1 --2- 172.21.15.47:0/877427356 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8e98062cc0 0x7f8e98065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:02.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.049+0000 7f8e97fff700 1 -- 172.21.15.47:0/877427356 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(443..443 src has 48..443) v4 ==== 8835+0+0 (secure 0 0 0) 0x7f8ea407d040 con 0x7f8eb00fca10 2024-02-23T00:49:02.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.049+0000 7f8e97fff700 10 client.41399.objecter ms_dispatch 0x7f8eb0001550 osd_map(443..443 src has 48..443) v4 2024-02-23T00:49:02.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.049+0000 7f8e97fff700 3 client.41399.objecter handle_osd_map got epochs [443,443] > 0 2024-02-23T00:49:02.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.049+0000 7f8e97fff700 3 client.41399.objecter handle_osd_map decoding full epoch 443 2024-02-23T00:49:02.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.049+0000 7f8eaeffd700 1 --2- 172.21.15.47:0/877427356 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8e98062cc0 0x7f8e98065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:02.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.049+0000 7f8e97fff700 10 client.41399.objecter checking command 1 2024-02-23T00:49:02.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.049+0000 7f8e97fff700 1 --2- 172.21.15.47:0/877427356 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f8e9806e940 0x7f8e98070de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:02.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.049+0000 7f8e97fff700 20 client.41399.objecter _get_session s=0x7f8ea40046e0 osd=0 3 2024-02-23T00:49:02.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.049+0000 7f8e97fff700 20 client.41399.objecter put_session s=0x7f8ea40046e0 osd=0 3 2024-02-23T00:49:02.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.049+0000 7f8e97fff700 15 client.41399.objecter _session_command_op_remove -1 1 2024-02-23T00:49:02.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.049+0000 7f8e97fff700 20 client.41399.objecter _get_session s=0x7f8ea40046e0 osd=0 3 2024-02-23T00:49:02.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.049+0000 7f8e97fff700 20 client.41399.objecter get_session s=0x7f8ea40046e0 osd=0 3 2024-02-23T00:49:02.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.049+0000 7f8e97fff700 15 client.41399.objecter _session_command_op_assign 0 1 2024-02-23T00:49:02.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.049+0000 7f8e97fff700 20 client.41399.objecter put_session s=0x7f8ea40046e0 osd=0 4 2024-02-23T00:49:02.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.049+0000 7f8e97fff700 10 client.41399.objecter _send_command 1 2024-02-23T00:49:02.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.049+0000 7f8e97fff700 1 -- 172.21.15.47:0/877427356 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8e98071460 con 0x7f8e9806e940 2024-02-23T00:49:02.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.049+0000 7f8e97fff700 20 client.41399.objecter dump_active .. 0 homeless 2024-02-23T00:49:02.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.049+0000 7f8e97fff700 1 -- 172.21.15.47:0/877427356 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_get_version_reply(handle=1 version=443) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ea407f620 con 0x7f8eb00fca10 2024-02-23T00:49:02.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.050+0000 7f8eaf7fe700 1 --2- 172.21.15.47:0/877427356 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f8e9806e940 0x7f8e98070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:02.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.050+0000 7f8e97fff700 10 client.41399.objecter ms_handle_connect 0x7f8e98062cc0 2024-02-23T00:49:02.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.050+0000 7f8eaeffd700 1 --2- 172.21.15.47:0/877427356 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8e98062cc0 0x7f8e98065180 secure :-1 s=READY pgs=3075 cs=0 l=1 rev1=1 rx=0x7f8ea8001370 tx=0x7f8ea80096a0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:02.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.050+0000 7f8e97fff700 10 client.41399.objecter ms_handle_connect 0x7f8e9806e940 2024-02-23T00:49:02.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.050+0000 7f8eaf7fe700 1 --2- 172.21.15.47:0/877427356 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f8e9806e940 0x7f8e98070de0 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:02.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.050+0000 7f8e97fff700 1 -- 172.21.15.47:0/877427356 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f8e98071460 con 0x7f8e9806e940 2024-02-23T00:49:02.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.050+0000 7f8e97fff700 10 client.41399.objecter ms_dispatch 0x7f8eb0001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:49:02.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.050+0000 7f8e97fff700 10 client.41399.objecter _finish_command 1 = system:0 2024-02-23T00:49:02.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.050+0000 7f8e97fff700 20 client.41399.objecter put_session s=0x7f8ea40046e0 osd=0 4 2024-02-23T00:49:02.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.050+0000 7f8e97fff700 15 client.41399.objecter _session_command_op_remove 0 1 2024-02-23T00:49:02.058 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:02.058 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:49:02.058 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:49:02.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.068+0000 7f8eb52d1700 10 client.41399.objecter _submit_command 2 [{"prefix": "reset_purged_snaps_last"}] 2024-02-23T00:49:02.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.068+0000 7f8eb52d1700 15 client.41399.objecter _session_command_op_assign -1 2 2024-02-23T00:49:02.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.068+0000 7f8eb52d1700 20 client.41399.objecter _get_session s=0x7f8ea40046e0 osd=0 3 2024-02-23T00:49:02.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.068+0000 7f8eb52d1700 20 client.41399.objecter put_session s=0x7f8ea40046e0 osd=0 3 2024-02-23T00:49:02.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.068+0000 7f8eb52d1700 20 client.41399.objecter _get_session s=0x7f8ea40046e0 osd=0 3 2024-02-23T00:49:02.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.068+0000 7f8eb52d1700 15 client.41399.objecter _session_command_op_remove -1 2 2024-02-23T00:49:02.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.068+0000 7f8eb52d1700 20 client.41399.objecter get_session s=0x7f8ea40046e0 osd=0 3 2024-02-23T00:49:02.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.068+0000 7f8eb52d1700 15 client.41399.objecter _session_command_op_assign 0 2 2024-02-23T00:49:02.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.068+0000 7f8eb52d1700 20 client.41399.objecter put_session s=0x7f8ea40046e0 osd=0 4 2024-02-23T00:49:02.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.068+0000 7f8eb52d1700 10 client.41399.objecter _send_command 2 2024-02-23T00:49:02.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.068+0000 7f8eb52d1700 1 -- 172.21.15.47:0/877427356 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) v1 -- 0x7f8eb0105d30 con 0x7f8e9806e940 2024-02-23T00:49:02.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.069+0000 7f8e97fff700 1 -- 172.21.15.47:0/877427356 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f8eb0105d30 con 0x7f8e9806e940 2024-02-23T00:49:02.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.069+0000 7f8e97fff700 10 client.41399.objecter ms_dispatch 0x7f8eb0001550 command_reply(tid 2: 0 ) v1 2024-02-23T00:49:02.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.069+0000 7f8e97fff700 10 client.41399.objecter _finish_command 2 = system:0 2024-02-23T00:49:02.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.069+0000 7f8e97fff700 20 client.41399.objecter put_session s=0x7f8ea40046e0 osd=0 4 2024-02-23T00:49:02.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.069+0000 7f8e97fff700 15 client.41399.objecter _session_command_op_remove 0 2 2024-02-23T00:49:02.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.069+0000 7f8eb52d1700 10 librados: watch_flush enter 2024-02-23T00:49:02.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.069+0000 7f8eb52d1700 10 librados: watch_flush exit 2024-02-23T00:49:02.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.069+0000 7f8eb52d1700 10 client.41399.objecter close_session for osd.0 2024-02-23T00:49:02.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.069+0000 7f8eb52d1700 1 -- 172.21.15.47:0/877427356 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f8e9806e940 msgr2=0x7f8e98070de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:02.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.069+0000 7f8eb52d1700 1 --2- 172.21.15.47:0/877427356 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f8e9806e940 0x7f8e98070de0 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:02.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.069+0000 7f8eb52d1700 20 client.41399.objecter put_session s=0x7f8ea40046e0 osd=0 1 2024-02-23T00:49:02.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.069+0000 7f8eb52d1700 20 client.41399.objecter shutdown clearing up homeless session... 2024-02-23T00:49:02.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.069+0000 7f8eb52d1700 10 client.41399.objecter successfully canceled tick 2024-02-23T00:49:02.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.069+0000 7f8eb52d1700 1 -- 172.21.15.47:0/877427356 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8e98062cc0 msgr2=0x7f8e98065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:02.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.069+0000 7f8eb52d1700 1 --2- 172.21.15.47:0/877427356 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8e98062cc0 0x7f8e98065180 secure :-1 s=READY pgs=3075 cs=0 l=1 rev1=1 rx=0x7f8ea8001370 tx=0x7f8ea80096a0).stop 2024-02-23T00:49:02.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.069+0000 7f8eb52d1700 1 -- 172.21.15.47:0/877427356 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8eb00fca10 msgr2=0x7f8eb01092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:02.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.069+0000 7f8eb52d1700 1 --2- 172.21.15.47:0/877427356 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8eb00fca10 0x7f8eb01092a0 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 rx=0x7f8ea400e970 tx=0x7f8ea400b040).stop 2024-02-23T00:49:02.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.069+0000 7f8eaf7fe700 1 -- 172.21.15.47:0/877427356 reap_dead start 2024-02-23T00:49:02.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.069+0000 7f8eb52d1700 1 -- 172.21.15.47:0/877427356 shutdown_connections 2024-02-23T00:49:02.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.069+0000 7f8eb52d1700 1 -- 172.21.15.47:0/877427356 >> 172.21.15.47:0/877427356 conn(0x7f8eb00f2660 msgr2=0x7f8eb00fab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:02.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.069+0000 7f8eb52d1700 1 -- 172.21.15.47:0/877427356 shutdown_connections 2024-02-23T00:49:02.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.069+0000 7f8eb52d1700 1 -- 172.21.15.47:0/877427356 wait complete. 2024-02-23T00:49:02.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:02.069+0000 7f8eb52d1700 1 librados: shutdown 2024-02-23T00:49:02.093 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:49:02.159 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:49:02.189 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:02.210 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:49:02.212 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:02.213 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:49:02.313 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:49:02.342 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:02.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:00.674135+0000 osd.2 (osd.2) 161 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:49:02.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:00.674559+0000 osd.2 (osd.2) 162 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:49:02.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:01.301154+0000 mgr.y (mgr.14517) 414 : cluster [DBG] pgmap v652: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:49:02.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:01.363073+0000 mon.a (mon.0) 2625 : cluster [DBG] osdmap e443: 8 total, 8 up, 7 in 2024-02-23T00:49:02.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:02.302954+0000 osd.6 (osd.6) 179 : cluster [DBG] 2.14 scrub starts 2024-02-23T00:49:02.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:02.303785+0000 osd.6 (osd.6) 180 : cluster [DBG] 2.14 scrub ok 2024-02-23T00:49:02.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:00.674135+0000 osd.2 (osd.2) 161 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:49:02.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:00.674559+0000 osd.2 (osd.2) 162 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:49:02.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:01.301154+0000 mgr.y (mgr.14517) 414 : cluster [DBG] pgmap v652: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:49:02.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:01.363073+0000 mon.a (mon.0) 2625 : cluster [DBG] osdmap e443: 8 total, 8 up, 7 in 2024-02-23T00:49:02.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:02.302954+0000 osd.6 (osd.6) 179 : cluster [DBG] 2.14 scrub starts 2024-02-23T00:49:02.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:02.303785+0000 osd.6 (osd.6) 180 : cluster [DBG] 2.14 scrub ok 2024-02-23T00:49:02.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:00.674135+0000 osd.2 (osd.2) 161 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:49:02.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:00.674559+0000 osd.2 (osd.2) 162 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:49:02.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:01.301154+0000 mgr.y (mgr.14517) 414 : cluster [DBG] pgmap v652: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:49:02.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:01.363073+0000 mon.a (mon.0) 2625 : cluster [DBG] osdmap e443: 8 total, 8 up, 7 in 2024-02-23T00:49:02.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:02.302954+0000 osd.6 (osd.6) 179 : cluster [DBG] 2.14 scrub starts 2024-02-23T00:49:02.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:02.303785+0000 osd.6 (osd.6) 180 : cluster [DBG] 2.14 scrub ok 2024-02-23T00:49:02.943 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:49:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:49:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:49:02.944 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:49:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:49:03.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:02.367750+0000 mon.a (mon.0) 2626 : cluster [DBG] osdmap e444: 8 total, 8 up, 7 in 2024-02-23T00:49:03.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:02.369593+0000 mon.a (mon.0) 2627 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:49:03.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:02.433334+0000 osd.7 (osd.7) 217 : cluster [DBG] 2.1d scrub starts 2024-02-23T00:49:03.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:02.436238+0000 osd.7 (osd.7) 218 : cluster [DBG] 2.1d scrub ok 2024-02-23T00:49:03.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:02.480836+0000 osd.5 (osd.5) 188 : cluster [DBG] 2.18 scrub starts 2024-02-23T00:49:03.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:02.482205+0000 osd.5 (osd.5) 189 : cluster [DBG] 2.18 scrub ok 2024-02-23T00:49:03.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:02.600685+0000 osd.1 (osd.1) 195 : cluster [DBG] 2.10 scrub starts 2024-02-23T00:49:03.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:02.601721+0000 osd.1 (osd.1) 196 : cluster [DBG] 2.10 scrub ok 2024-02-23T00:49:03.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:02.620414+0000 mgr.y (mgr.14517) 415 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:49:03.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:02.367750+0000 mon.a (mon.0) 2626 : cluster [DBG] osdmap e444: 8 total, 8 up, 7 in 2024-02-23T00:49:03.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:02.369593+0000 mon.a (mon.0) 2627 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:49:03.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:02.433334+0000 osd.7 (osd.7) 217 : cluster [DBG] 2.1d scrub starts 2024-02-23T00:49:03.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:02.436238+0000 osd.7 (osd.7) 218 : cluster [DBG] 2.1d scrub ok 2024-02-23T00:49:03.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:02.480836+0000 osd.5 (osd.5) 188 : cluster [DBG] 2.18 scrub starts 2024-02-23T00:49:03.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:02.482205+0000 osd.5 (osd.5) 189 : cluster [DBG] 2.18 scrub ok 2024-02-23T00:49:03.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:02.600685+0000 osd.1 (osd.1) 195 : cluster [DBG] 2.10 scrub starts 2024-02-23T00:49:03.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:02.601721+0000 osd.1 (osd.1) 196 : cluster [DBG] 2.10 scrub ok 2024-02-23T00:49:03.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:02.620414+0000 mgr.y (mgr.14517) 415 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:49:03.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:02.367750+0000 mon.a (mon.0) 2626 : cluster [DBG] osdmap e444: 8 total, 8 up, 7 in 2024-02-23T00:49:03.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:02.369593+0000 mon.a (mon.0) 2627 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:49:03.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:02.433334+0000 osd.7 (osd.7) 217 : cluster [DBG] 2.1d scrub starts 2024-02-23T00:49:03.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:02.436238+0000 osd.7 (osd.7) 218 : cluster [DBG] 2.1d scrub ok 2024-02-23T00:49:03.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:02.480836+0000 osd.5 (osd.5) 188 : cluster [DBG] 2.18 scrub starts 2024-02-23T00:49:03.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:02.482205+0000 osd.5 (osd.5) 189 : cluster [DBG] 2.18 scrub ok 2024-02-23T00:49:03.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:02.600685+0000 osd.1 (osd.1) 195 : cluster [DBG] 2.10 scrub starts 2024-02-23T00:49:03.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:02.601721+0000 osd.1 (osd.1) 196 : cluster [DBG] 2.10 scrub ok 2024-02-23T00:49:03.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:02.620414+0000 mgr.y (mgr.14517) 415 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:49:04.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.423+0000 7fb6cd026700 1 -- 172.21.15.47:0/4112256805 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c000b920 con 0x7fb6c80f6a90 2024-02-23T00:49:04.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.423+0000 7fb6d028c700 1 -- 172.21.15.47:0/4112256805 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6c80f6a90 msgr2=0x7fb6c8104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:04.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.423+0000 7fb6d028c700 1 --2- 172.21.15.47:0/4112256805 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6c80f6a90 0x7fb6c8104a20 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 rx=0x7fb6c0009be0 tx=0x7fb6c0004d10).stop 2024-02-23T00:49:04.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.423+0000 7fb6d028c700 1 -- 172.21.15.47:0/4112256805 shutdown_connections 2024-02-23T00:49:04.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.423+0000 7fb6d028c700 1 --2- 172.21.15.47:0/4112256805 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6c80f6a90 0x7fb6c8104a20 unknown :-1 s=CLOSED pgs=2681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:04.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.423+0000 7fb6d028c700 1 --2- 172.21.15.47:0/4112256805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6c80fdbd0 0x7fb6c80fdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:04.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.423+0000 7fb6d028c700 1 --2- 172.21.15.47:0/4112256805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6c80fe530 0x7fb6c80f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:04.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.423+0000 7fb6d028c700 1 -- 172.21.15.47:0/4112256805 >> 172.21.15.47:0/4112256805 conn(0x7fb6c806c370 msgr2=0x7fb6c806c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:04.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.423+0000 7fb6d028c700 1 -- 172.21.15.47:0/4112256805 shutdown_connections 2024-02-23T00:49:04.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.423+0000 7fb6d028c700 1 -- 172.21.15.47:0/4112256805 wait complete. 2024-02-23T00:49:04.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.424+0000 7fb6d028c700 1 librados: starting msgr at 2024-02-23T00:49:04.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.424+0000 7fb6d028c700 1 librados: starting objecter 2024-02-23T00:49:04.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.424+0000 7fb6d028c700 1 Processor -- start 2024-02-23T00:49:04.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.424+0000 7fb6d028c700 1 -- start start 2024-02-23T00:49:04.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.424+0000 7fb6d028c700 1 librados: setting wanted keys 2024-02-23T00:49:04.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.424+0000 7fb6d028c700 1 librados: calling monclient init 2024-02-23T00:49:04.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.424+0000 7fb6d028c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6c80f6a90 0x7fb6c80f7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:04.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.424+0000 7fb6d028c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6c80fdbd0 0x7fb6c80f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:04.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.424+0000 7fb6d028c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6c80fe530 0x7fb6c80f8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:04.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.424+0000 7fb6d028c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c8116300 con 0x7fb6c80fdbd0 2024-02-23T00:49:04.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.424+0000 7fb6d028c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c8116470 con 0x7fb6c80f6a90 2024-02-23T00:49:04.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.424+0000 7fb6d028c700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c8116630 con 0x7fb6c80fe530 2024-02-23T00:49:04.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.424+0000 7fb6cd827700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6c80fdbd0 0x7fb6c80f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:04.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.424+0000 7fb6ce829700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6c80fe530 0x7fb6c80f8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:04.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.424+0000 7fb6ce829700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6c80fe530 0x7fb6c80f8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:45624/0 (socket says 172.21.15.47:45624) 2024-02-23T00:49:04.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.424+0000 7fb6ce829700 1 -- 172.21.15.47:0/1673070970 learned_addr learned my addr 172.21.15.47:0/1673070970 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:49:04.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.424+0000 7fb6cd827700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6c80fdbd0 0x7fb6c80f7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60038/0 (socket says 172.21.15.47:60038) 2024-02-23T00:49:04.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.424+0000 7fb6ce028700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6c80f6a90 0x7fb6c80f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:04.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.425+0000 7fb6cd827700 1 -- 172.21.15.47:0/1673070970 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6c80fe530 msgr2=0x7fb6c80f8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:04.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.425+0000 7fb6cd827700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6c80fe530 0x7fb6c80f8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:04.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.425+0000 7fb6cd827700 1 -- 172.21.15.47:0/1673070970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6c80f6a90 msgr2=0x7fb6c80f7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:04.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.425+0000 7fb6cd827700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6c80f6a90 0x7fb6c80f7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:04.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.425+0000 7fb6cd827700 1 -- 172.21.15.47:0/1673070970 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b8005040 con 0x7fb6c80fdbd0 2024-02-23T00:49:04.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.425+0000 7fb6ce829700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6c80fe530 0x7fb6c80f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:49:04.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.425+0000 7fb6cd827700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6c80fdbd0 0x7fb6c80f7b70 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 rx=0x7fb6b8004920 tx=0x7fb6b8009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:04.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.425+0000 7fb6bf779700 10 client.?.objecter ms_handle_connect 0x7fb6c80fdbd0 2024-02-23T00:49:04.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.425+0000 7fb6bf779700 10 client.?.objecter resend_mon_ops 2024-02-23T00:49:04.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.425+0000 7fb6bf779700 1 -- 172.21.15.47:0/1673070970 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b800e030 con 0x7fb6c80fdbd0 2024-02-23T00:49:04.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.425+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6c00097c0 con 0x7fb6c80fdbd0 2024-02-23T00:49:04.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.425+0000 7fb6bf779700 1 -- 172.21.15.47:0/1673070970 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fb6b800f040 con 0x7fb6c80fdbd0 2024-02-23T00:49:04.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.425+0000 7fb6bf779700 1 -- 172.21.15.47:0/1673070970 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8014720 con 0x7fb6c80fdbd0 2024-02-23T00:49:04.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.425+0000 7fb6d028c700 10 client.31080.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:49:04.457 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:49:04.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.425+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6c8116cc0 con 0x7fb6c80fdbd0 2024-02-23T00:49:04.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.425+0000 7fb6d028c700 1 librados: init done 2024-02-23T00:49:04.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.426+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fb6c805fa60 con 0x7fb6c80fdbd0 2024-02-23T00:49:04.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.427+0000 7fb6bf779700 1 -- 172.21.15.47:0/1673070970 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fb6b8010500 con 0x7fb6c80fdbd0 2024-02-23T00:49:04.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.427+0000 7fb6bf779700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb6b4062cc0 0x7fb6b4065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:04.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.427+0000 7fb6bf779700 1 -- 172.21.15.47:0/1673070970 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(446..446 src has 48..446) v4 ==== 9220+0+0 (secure 0 0 0) 0x7fb6b807dc70 con 0x7fb6c80fdbd0 2024-02-23T00:49:04.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.427+0000 7fb6bf779700 10 client.31080.objecter ms_dispatch 0x7fb6c8001550 osd_map(446..446 src has 48..446) v4 2024-02-23T00:49:04.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.427+0000 7fb6bf779700 3 client.31080.objecter handle_osd_map got epochs [446,446] > 0 2024-02-23T00:49:04.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.427+0000 7fb6bf779700 3 client.31080.objecter handle_osd_map decoding full epoch 446 2024-02-23T00:49:04.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.427+0000 7fb6ce028700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb6b4062cc0 0x7fb6b4065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:04.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.428+0000 7fb6bf779700 20 client.31080.objecter dump_active .. 0 homeless 2024-02-23T00:49:04.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.428+0000 7fb6bf779700 1 -- 172.21.15.47:0/1673070970 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v446) v1 ==== 54+0+16 (secure 0 0 0) 0x7fb6b807e110 con 0x7fb6c80fdbd0 2024-02-23T00:49:04.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.428+0000 7fb6d028c700 10 client.31080.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:04.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.428+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_assign -1 1 2024-02-23T00:49:04.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.428+0000 7fb6bf779700 10 client.31080.objecter ms_handle_connect 0x7fb6b4062cc0 2024-02-23T00:49:04.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.428+0000 7fb6ce028700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb6b4062cc0 0x7fb6b4065180 secure :-1 s=READY pgs=3076 cs=0 l=1 rev1=1 rx=0x7fb6c40048b0 tx=0x7fb6c400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:04.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.428+0000 7fb6d028c700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fb6c80feee0 0x7fb6c80ff2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:04.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.428+0000 7fb6d028c700 20 client.31080.objecter _get_session s=0x7fb6c805ee80 osd=0 3 2024-02-23T00:49:04.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.428+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c805ee80 osd=0 3 2024-02-23T00:49:04.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.428+0000 7fb6d028c700 20 client.31080.objecter _get_session s=0x7fb6c805ee80 osd=0 3 2024-02-23T00:49:04.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.428+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_remove -1 1 2024-02-23T00:49:04.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.428+0000 7fb6d028c700 20 client.31080.objecter get_session s=0x7fb6c805ee80 osd=0 3 2024-02-23T00:49:04.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.428+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_assign 0 1 2024-02-23T00:49:04.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.428+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c805ee80 osd=0 4 2024-02-23T00:49:04.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.428+0000 7fb6d028c700 10 client.31080.objecter _send_command 1 2024-02-23T00:49:04.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.428+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb6c8106e90 con 0x7fb6c80feee0 2024-02-23T00:49:04.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.428+0000 7fb6ce829700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fb6c80feee0 0x7fb6c80ff2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:04.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.428+0000 7fb6bf779700 10 client.31080.objecter ms_handle_connect 0x7fb6c80feee0 2024-02-23T00:49:04.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.428+0000 7fb6ce829700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fb6c80feee0 0x7fb6c80ff2c0 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:04.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.429+0000 7fb6bf779700 1 -- 172.21.15.47:0/1673070970 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb6c8106e90 con 0x7fb6c80feee0 2024-02-23T00:49:04.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.429+0000 7fb6bf779700 10 client.31080.objecter ms_dispatch 0x7fb6c8001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:49:04.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.429+0000 7fb6bf779700 10 client.31080.objecter _finish_command 1 = system:0 2024-02-23T00:49:04.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.429+0000 7fb6bf779700 20 client.31080.objecter put_session s=0x7fb6c805ee80 osd=0 4 2024-02-23T00:49:04.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.429+0000 7fb6bf779700 15 client.31080.objecter _session_command_op_remove 0 1 2024-02-23T00:49:04.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.445+0000 7fb6d028c700 10 client.31080.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:49:04.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.445+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_assign -1 2 2024-02-23T00:49:04.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.445+0000 7fb6d028c700 20 client.31080.objecter _get_session s=0x7fb6c805ee80 osd=0 3 2024-02-23T00:49:04.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.445+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c805ee80 osd=0 3 2024-02-23T00:49:04.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.445+0000 7fb6d028c700 20 client.31080.objecter _get_session s=0x7fb6c805ee80 osd=0 3 2024-02-23T00:49:04.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.445+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_remove -1 2 2024-02-23T00:49:04.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.445+0000 7fb6d028c700 20 client.31080.objecter get_session s=0x7fb6c805ee80 osd=0 3 2024-02-23T00:49:04.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.445+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_assign 0 2 2024-02-23T00:49:04.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.445+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c805ee80 osd=0 4 2024-02-23T00:49:04.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.445+0000 7fb6d028c700 10 client.31080.objecter _send_command 2 2024-02-23T00:49:04.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.445+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fb6c80273c0 con 0x7fb6c80feee0 2024-02-23T00:49:04.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.446+0000 7fb6bf779700 1 -- 172.21.15.47:0/1673070970 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fb6c80273c0 con 0x7fb6c80feee0 2024-02-23T00:49:04.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.446+0000 7fb6bf779700 10 client.31080.objecter ms_dispatch 0x7fb6c8001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:49:04.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.446+0000 7fb6bf779700 10 client.31080.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:49:04.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.446+0000 7fb6bf779700 20 client.31080.objecter put_session s=0x7fb6c805ee80 osd=0 4 2024-02-23T00:49:04.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.446+0000 7fb6bf779700 15 client.31080.objecter _session_command_op_remove 0 2 2024-02-23T00:49:04.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.447+0000 7fb6d028c700 10 client.31080.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:04.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.447+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_assign -1 3 2024-02-23T00:49:04.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.447+0000 7fb6d028c700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fb6c80598f0 0x7fb6c8059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:04.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.447+0000 7fb6d028c700 20 client.31080.objecter _get_session s=0x7fb6c8028710 osd=1 3 2024-02-23T00:49:04.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.447+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c8028710 osd=1 3 2024-02-23T00:49:04.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.447+0000 7fb6d028c700 20 client.31080.objecter _get_session s=0x7fb6c8028710 osd=1 3 2024-02-23T00:49:04.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.447+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_remove -1 3 2024-02-23T00:49:04.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.447+0000 7fb6d028c700 20 client.31080.objecter get_session s=0x7fb6c8028710 osd=1 3 2024-02-23T00:49:04.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.447+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_assign 1 3 2024-02-23T00:49:04.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.447+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c8028710 osd=1 4 2024-02-23T00:49:04.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.447+0000 7fb6d028c700 10 client.31080.objecter _send_command 3 2024-02-23T00:49:04.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.447+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb6c811d660 con 0x7fb6c80598f0 2024-02-23T00:49:04.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.447+0000 7fb6ce028700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fb6c80598f0 0x7fb6c8059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:04.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.447+0000 7fb6bf779700 10 client.31080.objecter ms_handle_connect 0x7fb6c80598f0 2024-02-23T00:49:04.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.447+0000 7fb6ce028700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fb6c80598f0 0x7fb6c8059cd0 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:04.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.448+0000 7fb6bf779700 1 -- 172.21.15.47:0/1673070970 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb6c811d660 con 0x7fb6c80598f0 2024-02-23T00:49:04.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.448+0000 7fb6bf779700 10 client.31080.objecter ms_dispatch 0x7fb6c8001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:49:04.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.448+0000 7fb6bf779700 10 client.31080.objecter _finish_command 3 = system:0 2024-02-23T00:49:04.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.448+0000 7fb6bf779700 20 client.31080.objecter put_session s=0x7fb6c8028710 osd=1 4 2024-02-23T00:49:04.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.448+0000 7fb6bf779700 15 client.31080.objecter _session_command_op_remove 1 3 2024-02-23T00:49:04.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.462+0000 7fb6d028c700 10 client.31080.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:49:04.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.462+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_assign -1 4 2024-02-23T00:49:04.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.462+0000 7fb6d028c700 20 client.31080.objecter _get_session s=0x7fb6c8028710 osd=1 3 2024-02-23T00:49:04.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.462+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c8028710 osd=1 3 2024-02-23T00:49:04.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.462+0000 7fb6d028c700 20 client.31080.objecter _get_session s=0x7fb6c8028710 osd=1 3 2024-02-23T00:49:04.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.462+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_remove -1 4 2024-02-23T00:49:04.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.462+0000 7fb6d028c700 20 client.31080.objecter get_session s=0x7fb6c8028710 osd=1 3 2024-02-23T00:49:04.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.462+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_assign 1 4 2024-02-23T00:49:04.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.462+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c8028710 osd=1 4 2024-02-23T00:49:04.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.462+0000 7fb6d028c700 10 client.31080.objecter _send_command 4 2024-02-23T00:49:04.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.462+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fb6c8059360 con 0x7fb6c80598f0 2024-02-23T00:49:04.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.464+0000 7fb6bf779700 1 -- 172.21.15.47:0/1673070970 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fb6c8059360 con 0x7fb6c80598f0 2024-02-23T00:49:04.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.464+0000 7fb6bf779700 10 client.31080.objecter ms_dispatch 0x7fb6c8001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:49:04.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.464+0000 7fb6bf779700 10 client.31080.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:49:04.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.464+0000 7fb6bf779700 20 client.31080.objecter put_session s=0x7fb6c8028710 osd=1 4 2024-02-23T00:49:04.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.464+0000 7fb6bf779700 15 client.31080.objecter _session_command_op_remove 1 4 2024-02-23T00:49:04.467 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:49:04.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.464+0000 7fb6d028c700 10 client.31080.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:04.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.464+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_assign -1 5 2024-02-23T00:49:04.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.464+0000 7fb6d028c700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fb6c8028270 0x7fb6c811c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:04.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.464+0000 7fb6d028c700 20 client.31080.objecter _get_session s=0x7fb6c8059770 osd=2 3 2024-02-23T00:49:04.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.464+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c8059770 osd=2 3 2024-02-23T00:49:04.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.464+0000 7fb6d028c700 20 client.31080.objecter _get_session s=0x7fb6c8059770 osd=2 3 2024-02-23T00:49:04.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.464+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_remove -1 5 2024-02-23T00:49:04.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.464+0000 7fb6d028c700 20 client.31080.objecter get_session s=0x7fb6c8059770 osd=2 3 2024-02-23T00:49:04.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.464+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_assign 2 5 2024-02-23T00:49:04.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.464+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c8059770 osd=2 4 2024-02-23T00:49:04.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.464+0000 7fb6d028c700 10 client.31080.objecter _send_command 5 2024-02-23T00:49:04.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.464+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb6c811c570 con 0x7fb6c8028270 2024-02-23T00:49:04.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.464+0000 7fb6cd827700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fb6c8028270 0x7fb6c811c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:04.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.464+0000 7fb6bf779700 10 client.31080.objecter ms_handle_connect 0x7fb6c8028270 2024-02-23T00:49:04.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.464+0000 7fb6cd827700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fb6c8028270 0x7fb6c811c080 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:04.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.465+0000 7fb6bf779700 1 -- 172.21.15.47:0/1673070970 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb6c811c570 con 0x7fb6c8028270 2024-02-23T00:49:04.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.465+0000 7fb6bf779700 10 client.31080.objecter ms_dispatch 0x7fb6c8001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:49:04.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.465+0000 7fb6bf779700 10 client.31080.objecter _finish_command 5 = system:0 2024-02-23T00:49:04.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.465+0000 7fb6bf779700 20 client.31080.objecter put_session s=0x7fb6c8059770 osd=2 4 2024-02-23T00:49:04.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.465+0000 7fb6bf779700 15 client.31080.objecter _session_command_op_remove 2 5 2024-02-23T00:49:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.479+0000 7fb6d028c700 10 client.31080.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:49:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.479+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_assign -1 6 2024-02-23T00:49:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.479+0000 7fb6d028c700 20 client.31080.objecter _get_session s=0x7fb6c8059770 osd=2 3 2024-02-23T00:49:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.479+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c8059770 osd=2 3 2024-02-23T00:49:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.479+0000 7fb6d028c700 20 client.31080.objecter _get_session s=0x7fb6c8059770 osd=2 3 2024-02-23T00:49:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.479+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_remove -1 6 2024-02-23T00:49:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.479+0000 7fb6d028c700 20 client.31080.objecter get_session s=0x7fb6c8059770 osd=2 3 2024-02-23T00:49:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.479+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_assign 2 6 2024-02-23T00:49:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.479+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c8059770 osd=2 4 2024-02-23T00:49:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.479+0000 7fb6d028c700 10 client.31080.objecter _send_command 6 2024-02-23T00:49:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.479+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fb6c80277d0 con 0x7fb6c8028270 2024-02-23T00:49:04.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.481+0000 7fb6bf779700 1 -- 172.21.15.47:0/1673070970 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fb6c80277d0 con 0x7fb6c8028270 2024-02-23T00:49:04.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.481+0000 7fb6bf779700 10 client.31080.objecter ms_dispatch 0x7fb6c8001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:49:04.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.481+0000 7fb6bf779700 10 client.31080.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:49:04.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.481+0000 7fb6bf779700 20 client.31080.objecter put_session s=0x7fb6c8059770 osd=2 4 2024-02-23T00:49:04.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.481+0000 7fb6bf779700 15 client.31080.objecter _session_command_op_remove 2 6 2024-02-23T00:49:04.484 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:49:04.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.481+0000 7fb6d028c700 10 client.31080.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:04.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.481+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_assign -1 7 2024-02-23T00:49:04.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.481+0000 7fb6d028c700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fb6c811f7e0 0x7fb6c8121bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.481+0000 7fb6d028c700 20 client.31080.objecter _get_session s=0x7fb6c8027be0 osd=3 3 2024-02-23T00:49:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.481+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c8027be0 osd=3 3 2024-02-23T00:49:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.481+0000 7fb6d028c700 20 client.31080.objecter _get_session s=0x7fb6c8027be0 osd=3 3 2024-02-23T00:49:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.481+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_remove -1 7 2024-02-23T00:49:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.481+0000 7fb6d028c700 20 client.31080.objecter get_session s=0x7fb6c8027be0 osd=3 3 2024-02-23T00:49:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.481+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_assign 3 7 2024-02-23T00:49:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.481+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c8027be0 osd=3 4 2024-02-23T00:49:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.481+0000 7fb6d028c700 10 client.31080.objecter _send_command 7 2024-02-23T00:49:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.481+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb6c8124820 con 0x7fb6c811f7e0 2024-02-23T00:49:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.481+0000 7fb6ce829700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fb6c811f7e0 0x7fb6c8121bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.481+0000 7fb6bf779700 10 client.31080.objecter ms_handle_connect 0x7fb6c811f7e0 2024-02-23T00:49:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.481+0000 7fb6ce829700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fb6c811f7e0 0x7fb6c8121bd0 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.482+0000 7fb6bf779700 1 -- 172.21.15.47:0/1673070970 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb6c8124820 con 0x7fb6c811f7e0 2024-02-23T00:49:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.482+0000 7fb6bf779700 10 client.31080.objecter ms_dispatch 0x7fb6c8001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:49:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.482+0000 7fb6bf779700 10 client.31080.objecter _finish_command 7 = system:0 2024-02-23T00:49:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.482+0000 7fb6bf779700 20 client.31080.objecter put_session s=0x7fb6c8027be0 osd=3 4 2024-02-23T00:49:04.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.482+0000 7fb6bf779700 15 client.31080.objecter _session_command_op_remove 3 7 2024-02-23T00:49:04.494 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:49:04.498 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:04.498 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:49:04.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.498+0000 7fb6d028c700 10 client.31080.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:49:04.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.498+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_assign -1 8 2024-02-23T00:49:04.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.498+0000 7fb6d028c700 20 client.31080.objecter _get_session s=0x7fb6c8027be0 osd=3 3 2024-02-23T00:49:04.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.498+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c8027be0 osd=3 3 2024-02-23T00:49:04.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.498+0000 7fb6d028c700 20 client.31080.objecter _get_session s=0x7fb6c8027be0 osd=3 3 2024-02-23T00:49:04.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.498+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_remove -1 8 2024-02-23T00:49:04.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.498+0000 7fb6d028c700 20 client.31080.objecter get_session s=0x7fb6c8027be0 osd=3 3 2024-02-23T00:49:04.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.498+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_assign 3 8 2024-02-23T00:49:04.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.498+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c8027be0 osd=3 4 2024-02-23T00:49:04.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.498+0000 7fb6d028c700 10 client.31080.objecter _send_command 8 2024-02-23T00:49:04.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.498+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fb6c811c9b0 con 0x7fb6c811f7e0 2024-02-23T00:49:04.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.500+0000 7fb6bf779700 1 -- 172.21.15.47:0/1673070970 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fb6c811c9b0 con 0x7fb6c811f7e0 2024-02-23T00:49:04.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.500+0000 7fb6bf779700 10 client.31080.objecter ms_dispatch 0x7fb6c8001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:49:04.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.500+0000 7fb6bf779700 10 client.31080.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:49:04.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.500+0000 7fb6bf779700 20 client.31080.objecter put_session s=0x7fb6c8027be0 osd=3 4 2024-02-23T00:49:04.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.500+0000 7fb6bf779700 15 client.31080.objecter _session_command_op_remove 3 8 2024-02-23T00:49:04.503 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:49:04.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.500+0000 7fb6d028c700 10 client.31080.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:04.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.500+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_assign -1 9 2024-02-23T00:49:04.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.500+0000 7fb6d028c700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fb6c811cb20 0x7fb6c8123240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:04.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.500+0000 7fb6d028c700 20 client.31080.objecter _get_session s=0x7fb6c811c9b0 osd=4 3 2024-02-23T00:49:04.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.500+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c811c9b0 osd=4 3 2024-02-23T00:49:04.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.500+0000 7fb6d028c700 20 client.31080.objecter _get_session s=0x7fb6c811c9b0 osd=4 3 2024-02-23T00:49:04.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.500+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_remove -1 9 2024-02-23T00:49:04.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.500+0000 7fb6d028c700 20 client.31080.objecter get_session s=0x7fb6c811c9b0 osd=4 3 2024-02-23T00:49:04.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.500+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_assign 4 9 2024-02-23T00:49:04.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.500+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c811c9b0 osd=4 4 2024-02-23T00:49:04.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.500+0000 7fb6d028c700 10 client.31080.objecter _send_command 9 2024-02-23T00:49:04.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.500+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb6c811bd30 con 0x7fb6c811cb20 2024-02-23T00:49:04.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.500+0000 7fb6ce028700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fb6c811cb20 0x7fb6c8123240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:04.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.500+0000 7fb6bf779700 10 client.31080.objecter ms_handle_connect 0x7fb6c811cb20 2024-02-23T00:49:04.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.500+0000 7fb6ce028700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fb6c811cb20 0x7fb6c8123240 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:04.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.501+0000 7fb6bf779700 1 -- 172.21.15.47:0/1673070970 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb6c811bd30 con 0x7fb6c811cb20 2024-02-23T00:49:04.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.501+0000 7fb6bf779700 10 client.31080.objecter ms_dispatch 0x7fb6c8001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:49:04.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.501+0000 7fb6bf779700 10 client.31080.objecter _finish_command 9 = system:0 2024-02-23T00:49:04.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.501+0000 7fb6bf779700 20 client.31080.objecter put_session s=0x7fb6c811c9b0 osd=4 4 2024-02-23T00:49:04.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.501+0000 7fb6bf779700 15 client.31080.objecter _session_command_op_remove 4 9 2024-02-23T00:49:04.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.516+0000 7fb6d028c700 10 client.31080.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:49:04.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.516+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_assign -1 10 2024-02-23T00:49:04.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.516+0000 7fb6d028c700 20 client.31080.objecter _get_session s=0x7fb6c811c9b0 osd=4 3 2024-02-23T00:49:04.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.516+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c811c9b0 osd=4 3 2024-02-23T00:49:04.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.516+0000 7fb6d028c700 20 client.31080.objecter _get_session s=0x7fb6c811c9b0 osd=4 3 2024-02-23T00:49:04.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.516+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_remove -1 10 2024-02-23T00:49:04.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.516+0000 7fb6d028c700 20 client.31080.objecter get_session s=0x7fb6c811c9b0 osd=4 3 2024-02-23T00:49:04.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.516+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_assign 4 10 2024-02-23T00:49:04.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.516+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c811c9b0 osd=4 4 2024-02-23T00:49:04.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.516+0000 7fb6d028c700 10 client.31080.objecter _send_command 10 2024-02-23T00:49:04.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.516+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fb6c811bd30 con 0x7fb6c811cb20 2024-02-23T00:49:04.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.518+0000 7fb6bf779700 1 -- 172.21.15.47:0/1673070970 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fb6c811bd30 con 0x7fb6c811cb20 2024-02-23T00:49:04.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.518+0000 7fb6bf779700 10 client.31080.objecter ms_dispatch 0x7fb6c8001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:49:04.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.518+0000 7fb6bf779700 10 client.31080.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:49:04.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.518+0000 7fb6bf779700 20 client.31080.objecter put_session s=0x7fb6c811c9b0 osd=4 4 2024-02-23T00:49:04.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.518+0000 7fb6bf779700 15 client.31080.objecter _session_command_op_remove 4 10 2024-02-23T00:49:04.521 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:49:04.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.518+0000 7fb6d028c700 10 client.31080.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:04.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.518+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_assign -1 11 2024-02-23T00:49:04.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.518+0000 7fb6d028c700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fb6c8123c40 0x7fb6c81240a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:04.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.518+0000 7fb6d028c700 20 client.31080.objecter _get_session s=0x7fb6c811bd30 osd=5 3 2024-02-23T00:49:04.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.518+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c811bd30 osd=5 3 2024-02-23T00:49:04.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.518+0000 7fb6d028c700 20 client.31080.objecter _get_session s=0x7fb6c811bd30 osd=5 3 2024-02-23T00:49:04.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.518+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_remove -1 11 2024-02-23T00:49:04.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.518+0000 7fb6d028c700 20 client.31080.objecter get_session s=0x7fb6c811bd30 osd=5 3 2024-02-23T00:49:04.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.518+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_assign 5 11 2024-02-23T00:49:04.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.518+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c811bd30 osd=5 4 2024-02-23T00:49:04.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.518+0000 7fb6d028c700 10 client.31080.objecter _send_command 11 2024-02-23T00:49:04.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.518+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb6c8124650 con 0x7fb6c8123c40 2024-02-23T00:49:04.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.518+0000 7fb6cd827700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fb6c8123c40 0x7fb6c81240a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:04.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.519+0000 7fb6bf779700 10 client.31080.objecter ms_handle_connect 0x7fb6c8123c40 2024-02-23T00:49:04.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.519+0000 7fb6cd827700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fb6c8123c40 0x7fb6c81240a0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:04.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.519+0000 7fb6bf779700 1 -- 172.21.15.47:0/1673070970 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb6c8124650 con 0x7fb6c8123c40 2024-02-23T00:49:04.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.519+0000 7fb6bf779700 10 client.31080.objecter ms_dispatch 0x7fb6c8001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:49:04.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.519+0000 7fb6bf779700 10 client.31080.objecter _finish_command 11 = system:0 2024-02-23T00:49:04.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.519+0000 7fb6bf779700 20 client.31080.objecter put_session s=0x7fb6c811bd30 osd=5 4 2024-02-23T00:49:04.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.519+0000 7fb6bf779700 15 client.31080.objecter _session_command_op_remove 5 11 2024-02-23T00:49:04.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.534+0000 7fb6d028c700 10 client.31080.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:49:04.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.534+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_assign -1 12 2024-02-23T00:49:04.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.534+0000 7fb6d028c700 20 client.31080.objecter _get_session s=0x7fb6c811bd30 osd=5 3 2024-02-23T00:49:04.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.534+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c811bd30 osd=5 3 2024-02-23T00:49:04.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.534+0000 7fb6d028c700 20 client.31080.objecter _get_session s=0x7fb6c811bd30 osd=5 3 2024-02-23T00:49:04.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.534+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_remove -1 12 2024-02-23T00:49:04.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.534+0000 7fb6d028c700 20 client.31080.objecter get_session s=0x7fb6c811bd30 osd=5 3 2024-02-23T00:49:04.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.534+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_assign 5 12 2024-02-23T00:49:04.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.534+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c811bd30 osd=5 4 2024-02-23T00:49:04.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.534+0000 7fb6d028c700 10 client.31080.objecter _send_command 12 2024-02-23T00:49:04.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.534+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fb6c8124650 con 0x7fb6c8123c40 2024-02-23T00:49:04.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.536+0000 7fb6bf779700 1 -- 172.21.15.47:0/1673070970 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fb6c8124650 con 0x7fb6c8123c40 2024-02-23T00:49:04.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.536+0000 7fb6bf779700 10 client.31080.objecter ms_dispatch 0x7fb6c8001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:49:04.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.536+0000 7fb6bf779700 10 client.31080.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:49:04.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.536+0000 7fb6bf779700 20 client.31080.objecter put_session s=0x7fb6c811bd30 osd=5 4 2024-02-23T00:49:04.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.536+0000 7fb6bf779700 15 client.31080.objecter _session_command_op_remove 5 12 2024-02-23T00:49:04.539 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:49:04.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.536+0000 7fb6d028c700 10 client.31080.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:04.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.536+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_assign -1 13 2024-02-23T00:49:04.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.536+0000 7fb6d028c700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fb6c8128f50 0x7fb6c812b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:04.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.536+0000 7fb6d028c700 20 client.31080.objecter _get_session s=0x7fb6c8124650 osd=6 3 2024-02-23T00:49:04.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.536+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c8124650 osd=6 3 2024-02-23T00:49:04.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.536+0000 7fb6d028c700 20 client.31080.objecter _get_session s=0x7fb6c8124650 osd=6 3 2024-02-23T00:49:04.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.536+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_remove -1 13 2024-02-23T00:49:04.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.536+0000 7fb6d028c700 20 client.31080.objecter get_session s=0x7fb6c8124650 osd=6 3 2024-02-23T00:49:04.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.536+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_assign 6 13 2024-02-23T00:49:04.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.536+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c8124650 osd=6 4 2024-02-23T00:49:04.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.536+0000 7fb6d028c700 10 client.31080.objecter _send_command 13 2024-02-23T00:49:04.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.536+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb6c812e200 con 0x7fb6c8128f50 2024-02-23T00:49:04.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.537+0000 7fb6ce829700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fb6c8128f50 0x7fb6c812b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:04.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.537+0000 7fb6bf779700 10 client.31080.objecter ms_handle_connect 0x7fb6c8128f50 2024-02-23T00:49:04.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.537+0000 7fb6ce829700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fb6c8128f50 0x7fb6c812b3c0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:04.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.538+0000 7fb6bf779700 1 -- 172.21.15.47:0/1673070970 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb6c812e200 con 0x7fb6c8128f50 2024-02-23T00:49:04.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.538+0000 7fb6bf779700 10 client.31080.objecter ms_dispatch 0x7fb6c8001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:49:04.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.538+0000 7fb6bf779700 10 client.31080.objecter _finish_command 13 = system:0 2024-02-23T00:49:04.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.538+0000 7fb6bf779700 20 client.31080.objecter put_session s=0x7fb6c8124650 osd=6 4 2024-02-23T00:49:04.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.538+0000 7fb6bf779700 15 client.31080.objecter _session_command_op_remove 6 13 2024-02-23T00:49:04.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.552+0000 7fb6d028c700 10 client.31080.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:49:04.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.552+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_assign -1 14 2024-02-23T00:49:04.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.552+0000 7fb6d028c700 20 client.31080.objecter _get_session s=0x7fb6c8124650 osd=6 3 2024-02-23T00:49:04.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.552+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c8124650 osd=6 3 2024-02-23T00:49:04.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.552+0000 7fb6d028c700 20 client.31080.objecter _get_session s=0x7fb6c8124650 osd=6 3 2024-02-23T00:49:04.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.552+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_remove -1 14 2024-02-23T00:49:04.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.552+0000 7fb6d028c700 20 client.31080.objecter get_session s=0x7fb6c8124650 osd=6 3 2024-02-23T00:49:04.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.552+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_assign 6 14 2024-02-23T00:49:04.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.552+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c8124650 osd=6 4 2024-02-23T00:49:04.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.552+0000 7fb6d028c700 10 client.31080.objecter _send_command 14 2024-02-23T00:49:04.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.552+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fb6c812cfd0 con 0x7fb6c8128f50 2024-02-23T00:49:04.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.554+0000 7fb6bf779700 1 -- 172.21.15.47:0/1673070970 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fb6c812cfd0 con 0x7fb6c8128f50 2024-02-23T00:49:04.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.554+0000 7fb6bf779700 10 client.31080.objecter ms_dispatch 0x7fb6c8001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:49:04.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.554+0000 7fb6bf779700 10 client.31080.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:49:04.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.554+0000 7fb6bf779700 20 client.31080.objecter put_session s=0x7fb6c8124650 osd=6 4 2024-02-23T00:49:04.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.554+0000 7fb6bf779700 15 client.31080.objecter _session_command_op_remove 6 14 2024-02-23T00:49:04.557 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:49:04.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.554+0000 7fb6d028c700 10 client.31080.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:04.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.554+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_assign -1 15 2024-02-23T00:49:04.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.554+0000 7fb6d028c700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fb6c812d6d0 0x7fb6c812db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:04.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.554+0000 7fb6d028c700 20 client.31080.objecter _get_session s=0x7fb6c812cfd0 osd=7 3 2024-02-23T00:49:04.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.554+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c812cfd0 osd=7 3 2024-02-23T00:49:04.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.554+0000 7fb6d028c700 20 client.31080.objecter _get_session s=0x7fb6c812cfd0 osd=7 3 2024-02-23T00:49:04.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.554+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_remove -1 15 2024-02-23T00:49:04.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.554+0000 7fb6d028c700 20 client.31080.objecter get_session s=0x7fb6c812cfd0 osd=7 3 2024-02-23T00:49:04.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.554+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_assign 7 15 2024-02-23T00:49:04.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.554+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c812cfd0 osd=7 4 2024-02-23T00:49:04.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.554+0000 7fb6d028c700 10 client.31080.objecter _send_command 15 2024-02-23T00:49:04.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.554+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb6c8131450 con 0x7fb6c812d6d0 2024-02-23T00:49:04.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.554+0000 7fb6ce028700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fb6c812d6d0 0x7fb6c812db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:04.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.554+0000 7fb6bf779700 10 client.31080.objecter ms_handle_connect 0x7fb6c812d6d0 2024-02-23T00:49:04.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.554+0000 7fb6ce028700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fb6c812d6d0 0x7fb6c812db50 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:04.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.555+0000 7fb6bf779700 1 -- 172.21.15.47:0/1673070970 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb6c8131450 con 0x7fb6c812d6d0 2024-02-23T00:49:04.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.555+0000 7fb6bf779700 10 client.31080.objecter ms_dispatch 0x7fb6c8001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:49:04.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.555+0000 7fb6bf779700 10 client.31080.objecter _finish_command 15 = system:0 2024-02-23T00:49:04.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.555+0000 7fb6bf779700 20 client.31080.objecter put_session s=0x7fb6c812cfd0 osd=7 4 2024-02-23T00:49:04.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.555+0000 7fb6bf779700 15 client.31080.objecter _session_command_op_remove 7 15 2024-02-23T00:49:04.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.569+0000 7fb6d028c700 10 client.31080.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:49:04.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.569+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_assign -1 16 2024-02-23T00:49:04.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.569+0000 7fb6d028c700 20 client.31080.objecter _get_session s=0x7fb6c812cfd0 osd=7 3 2024-02-23T00:49:04.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.569+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c812cfd0 osd=7 3 2024-02-23T00:49:04.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.569+0000 7fb6d028c700 20 client.31080.objecter _get_session s=0x7fb6c812cfd0 osd=7 3 2024-02-23T00:49:04.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.569+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_remove -1 16 2024-02-23T00:49:04.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.569+0000 7fb6d028c700 20 client.31080.objecter get_session s=0x7fb6c812cfd0 osd=7 3 2024-02-23T00:49:04.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.569+0000 7fb6d028c700 15 client.31080.objecter _session_command_op_assign 7 16 2024-02-23T00:49:04.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.569+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c812cfd0 osd=7 4 2024-02-23T00:49:04.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.569+0000 7fb6d028c700 10 client.31080.objecter _send_command 16 2024-02-23T00:49:04.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.569+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fb6c8131450 con 0x7fb6c812d6d0 2024-02-23T00:49:04.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.571+0000 7fb6bf779700 1 -- 172.21.15.47:0/1673070970 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fb6c8131450 con 0x7fb6c812d6d0 2024-02-23T00:49:04.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.571+0000 7fb6bf779700 10 client.31080.objecter ms_dispatch 0x7fb6c8001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:49:04.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.571+0000 7fb6bf779700 10 client.31080.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:49:04.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.571+0000 7fb6bf779700 20 client.31080.objecter put_session s=0x7fb6c812cfd0 osd=7 4 2024-02-23T00:49:04.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.571+0000 7fb6bf779700 15 client.31080.objecter _session_command_op_remove 7 16 2024-02-23T00:49:04.575 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:49:04.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 10 librados: watch_flush enter 2024-02-23T00:49:04.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 10 librados: watch_flush exit 2024-02-23T00:49:04.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 10 client.31080.objecter close_session for osd.0 2024-02-23T00:49:04.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fb6c80feee0 msgr2=0x7fb6c80ff2c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:04.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fb6c80feee0 0x7fb6c80ff2c0 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:04.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c805ee80 osd=0 1 2024-02-23T00:49:04.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 10 client.31080.objecter close_session for osd.1 2024-02-23T00:49:04.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fb6c80598f0 msgr2=0x7fb6c8059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:04.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fb6c80598f0 0x7fb6c8059cd0 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:04.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c8028710 osd=1 1 2024-02-23T00:49:04.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 10 client.31080.objecter close_session for osd.2 2024-02-23T00:49:04.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fb6c8028270 msgr2=0x7fb6c811c080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:04.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fb6c8028270 0x7fb6c811c080 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:04.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c8059770 osd=2 1 2024-02-23T00:49:04.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 10 client.31080.objecter close_session for osd.3 2024-02-23T00:49:04.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fb6c811f7e0 msgr2=0x7fb6c8121bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:04.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fb6c811f7e0 0x7fb6c8121bd0 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:04.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c8027be0 osd=3 1 2024-02-23T00:49:04.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 10 client.31080.objecter close_session for osd.4 2024-02-23T00:49:04.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fb6c811cb20 msgr2=0x7fb6c8123240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:04.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6ce829700 1 -- 172.21.15.47:0/1673070970 reap_dead start 2024-02-23T00:49:04.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fb6c811cb20 0x7fb6c8123240 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:04.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6ce829700 1 -- 172.21.15.47:0/1673070970 reap_dead start 2024-02-23T00:49:04.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c811c9b0 osd=4 1 2024-02-23T00:49:04.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 10 client.31080.objecter close_session for osd.5 2024-02-23T00:49:04.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fb6c8123c40 msgr2=0x7fb6c81240a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:04.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fb6c8123c40 0x7fb6c81240a0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:04.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c811bd30 osd=5 1 2024-02-23T00:49:04.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 10 client.31080.objecter close_session for osd.6 2024-02-23T00:49:04.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fb6c8128f50 msgr2=0x7fb6c812b3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:04.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fb6c8128f50 0x7fb6c812b3c0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:04.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c8124650 osd=6 1 2024-02-23T00:49:04.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 10 client.31080.objecter close_session for osd.7 2024-02-23T00:49:04.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fb6c812d6d0 msgr2=0x7fb6c812db50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:04.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fb6c812d6d0 0x7fb6c812db50 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:04.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 20 client.31080.objecter put_session s=0x7fb6c812cfd0 osd=7 1 2024-02-23T00:49:04.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 20 client.31080.objecter shutdown clearing up homeless session... 2024-02-23T00:49:04.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 10 client.31080.objecter successfully canceled tick 2024-02-23T00:49:04.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb6b4062cc0 msgr2=0x7fb6b4065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:04.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb6b4062cc0 0x7fb6b4065180 secure :-1 s=READY pgs=3076 cs=0 l=1 rev1=1 rx=0x7fb6c40048b0 tx=0x7fb6c400b040).stop 2024-02-23T00:49:04.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6c80fdbd0 msgr2=0x7fb6c80f7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:04.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6ce829700 1 -- 172.21.15.47:0/1673070970 reap_dead start 2024-02-23T00:49:04.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6c80fdbd0 0x7fb6c80f7b70 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 rx=0x7fb6b8004920 tx=0x7fb6b8009cb0).stop 2024-02-23T00:49:04.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 shutdown_connections 2024-02-23T00:49:04.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 1 --2- 172.21.15.47:0/1673070970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6c80fdbd0 0x7fb6c80f7b70 unknown :-1 s=CLOSED pgs=2539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:04.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 >> 172.21.15.47:0/1673070970 conn(0x7fb6c806c370 msgr2=0x7fb6c80f5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:04.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 shutdown_connections 2024-02-23T00:49:04.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 1 -- 172.21.15.47:0/1673070970 wait complete. 2024-02-23T00:49:04.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:04.572+0000 7fb6d028c700 1 librados: shutdown 2024-02-23T00:49:04.579 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:49:04.579 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:49:04.579 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:49:04.579 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:49:04.579 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:49:04.579 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:49:04.580 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:49:04.580 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:49:04.599 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:49:04.628 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:03.261661+0000 osd.4 (osd.4) 183 : cluster [DBG] 62.f scrub starts 2024-02-23T00:49:04.628 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:03.263929+0000 osd.4 (osd.4) 184 : cluster [DBG] 62.f scrub ok 2024-02-23T00:49:04.628 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:03.301607+0000 mgr.y (mgr.14517) 416 : cluster [DBG] pgmap v655: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:49:04.628 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:03.373812+0000 mon.a (mon.0) 2628 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:49:04.628 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:03.373967+0000 mon.a (mon.0) 2629 : cluster [DBG] osdmap e445: 8 total, 8 up, 7 in 2024-02-23T00:49:04.629 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:03.378267+0000 mon.a (mon.0) 2630 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:49:04.629 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:03.457790+0000 osd.5 (osd.5) 190 : cluster [DBG] 2.c scrub starts 2024-02-23T00:49:04.629 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:03.460736+0000 osd.5 (osd.5) 191 : cluster [DBG] 2.c scrub ok 2024-02-23T00:49:04.629 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:03.261661+0000 osd.4 (osd.4) 183 : cluster [DBG] 62.f scrub starts 2024-02-23T00:49:04.629 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:03.263929+0000 osd.4 (osd.4) 184 : cluster [DBG] 62.f scrub ok 2024-02-23T00:49:04.629 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:03.301607+0000 mgr.y (mgr.14517) 416 : cluster [DBG] pgmap v655: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:49:04.629 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:03.373812+0000 mon.a (mon.0) 2628 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:49:04.629 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:03.373967+0000 mon.a (mon.0) 2629 : cluster [DBG] osdmap e445: 8 total, 8 up, 7 in 2024-02-23T00:49:04.629 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:03.378267+0000 mon.a (mon.0) 2630 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:49:04.629 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:03.457790+0000 osd.5 (osd.5) 190 : cluster [DBG] 2.c scrub starts 2024-02-23T00:49:04.629 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:03.460736+0000 osd.5 (osd.5) 191 : cluster [DBG] 2.c scrub ok 2024-02-23T00:49:04.630 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:04.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:03.261661+0000 osd.4 (osd.4) 183 : cluster [DBG] 62.f scrub starts 2024-02-23T00:49:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:03.263929+0000 osd.4 (osd.4) 184 : cluster [DBG] 62.f scrub ok 2024-02-23T00:49:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:03.301607+0000 mgr.y (mgr.14517) 416 : cluster [DBG] pgmap v655: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:49:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:03.373812+0000 mon.a (mon.0) 2628 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:49:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:03.373967+0000 mon.a (mon.0) 2629 : cluster [DBG] osdmap e445: 8 total, 8 up, 7 in 2024-02-23T00:49:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:03.378267+0000 mon.a (mon.0) 2630 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:49:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:03.457790+0000 osd.5 (osd.5) 190 : cluster [DBG] 2.c scrub starts 2024-02-23T00:49:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:03.460736+0000 osd.5 (osd.5) 191 : cluster [DBG] 2.c scrub ok 2024-02-23T00:49:05.382 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:49:05.383 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:49:05.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:03.681830+0000 osd.0 (osd.0) 187 : cluster [DBG] 5.b deep-scrub starts 2024-02-23T00:49:05.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:03.682738+0000 osd.0 (osd.0) 188 : cluster [DBG] 5.b deep-scrub ok 2024-02-23T00:49:05.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:04.232914+0000 osd.4 (osd.4) 185 : cluster [DBG] 4.e scrub starts 2024-02-23T00:49:05.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:04.235036+0000 osd.4 (osd.4) 186 : cluster [DBG] 4.e scrub ok 2024-02-23T00:49:05.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:04.378217+0000 mon.a (mon.0) 2631 : cluster [DBG] osdmap e446: 8 total, 8 up, 7 in 2024-02-23T00:49:05.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:04.412782+0000 mon.a (mon.0) 2632 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:49:05.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:04.413123+0000 mon.a (mon.0) 2633 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-65"}]: dispatch 2024-02-23T00:49:05.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:04.427698+0000 mon.a (mon.0) 2634 : audit [DBG] from='client.? 172.21.15.47:0/1673070970' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:49:05.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:05.199362+0000 mon.a (mon.0) 2635 : cluster [DBG] osdmap e447: 8 total, 8 up, 7 in 2024-02-23T00:49:06.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:03.681830+0000 osd.0 (osd.0) 187 : cluster [DBG] 5.b deep-scrub starts 2024-02-23T00:49:06.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:03.682738+0000 osd.0 (osd.0) 188 : cluster [DBG] 5.b deep-scrub ok 2024-02-23T00:49:06.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:04.232914+0000 osd.4 (osd.4) 185 : cluster [DBG] 4.e scrub starts 2024-02-23T00:49:06.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:04.235036+0000 osd.4 (osd.4) 186 : cluster [DBG] 4.e scrub ok 2024-02-23T00:49:06.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:04.378217+0000 mon.a (mon.0) 2631 : cluster [DBG] osdmap e446: 8 total, 8 up, 7 in 2024-02-23T00:49:06.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:04.412782+0000 mon.a (mon.0) 2632 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:49:06.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:04.413123+0000 mon.a (mon.0) 2633 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-65"}]: dispatch 2024-02-23T00:49:06.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:04.427698+0000 mon.a (mon.0) 2634 : audit [DBG] from='client.? 172.21.15.47:0/1673070970' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:49:06.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:05.199362+0000 mon.a (mon.0) 2635 : cluster [DBG] osdmap e447: 8 total, 8 up, 7 in 2024-02-23T00:49:06.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:03.681830+0000 osd.0 (osd.0) 187 : cluster [DBG] 5.b deep-scrub starts 2024-02-23T00:49:06.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:03.682738+0000 osd.0 (osd.0) 188 : cluster [DBG] 5.b deep-scrub ok 2024-02-23T00:49:06.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:04.232914+0000 osd.4 (osd.4) 185 : cluster [DBG] 4.e scrub starts 2024-02-23T00:49:06.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:04.235036+0000 osd.4 (osd.4) 186 : cluster [DBG] 4.e scrub ok 2024-02-23T00:49:06.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:04.378217+0000 mon.a (mon.0) 2631 : cluster [DBG] osdmap e446: 8 total, 8 up, 7 in 2024-02-23T00:49:06.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:04.412782+0000 mon.a (mon.0) 2632 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:49:06.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:04.413123+0000 mon.a (mon.0) 2633 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-65"}]: dispatch 2024-02-23T00:49:06.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:04.427698+0000 mon.a (mon.0) 2634 : audit [DBG] from='client.? 172.21.15.47:0/1673070970' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:49:06.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:05.199362+0000 mon.a (mon.0) 2635 : cluster [DBG] osdmap e447: 8 total, 8 up, 7 in 2024-02-23T00:49:06.544 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.4 dump_ops_in_flight 2024-02-23T00:49:06.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:04.527131+0000 osd.1 (osd.1) 197 : cluster [DBG] 2.d deep-scrub starts 2024-02-23T00:49:06.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:04.527997+0000 osd.1 (osd.1) 198 : cluster [DBG] 2.d deep-scrub ok 2024-02-23T00:49:06.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:05.301957+0000 mgr.y (mgr.14517) 417 : cluster [DBG] pgmap v659: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-23T00:49:06.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:05.328344+0000 osd.6 (osd.6) 181 : cluster [DBG] 2.17 scrub starts 2024-02-23T00:49:06.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:05.329173+0000 osd.6 (osd.6) 182 : cluster [DBG] 2.17 scrub ok 2024-02-23T00:49:06.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:05.881876+0000 osd.1 (osd.1) 199 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:49:06.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:05.882022+0000 osd.1 (osd.1) 200 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:49:06.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:06.036908+0000 osd.3 (osd.3) 206 : cluster [DBG] 62.c deep-scrub starts 2024-02-23T00:49:06.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:06.040321+0000 osd.3 (osd.3) 207 : cluster [DBG] 62.c deep-scrub ok 2024-02-23T00:49:06.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:06.215796+0000 mon.a (mon.0) 2636 : cluster [DBG] osdmap e448: 8 total, 8 up, 7 in 2024-02-23T00:49:06.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:06.221108+0000 mon.a (mon.0) 2637 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:49:06.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:04.527131+0000 osd.1 (osd.1) 197 : cluster [DBG] 2.d deep-scrub starts 2024-02-23T00:49:06.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:04.527997+0000 osd.1 (osd.1) 198 : cluster [DBG] 2.d deep-scrub ok 2024-02-23T00:49:06.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:05.301957+0000 mgr.y (mgr.14517) 417 : cluster [DBG] pgmap v659: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-23T00:49:06.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:05.328344+0000 osd.6 (osd.6) 181 : cluster [DBG] 2.17 scrub starts 2024-02-23T00:49:06.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:05.329173+0000 osd.6 (osd.6) 182 : cluster [DBG] 2.17 scrub ok 2024-02-23T00:49:06.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:05.881876+0000 osd.1 (osd.1) 199 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:49:06.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:05.882022+0000 osd.1 (osd.1) 200 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:49:06.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:06.036908+0000 osd.3 (osd.3) 206 : cluster [DBG] 62.c deep-scrub starts 2024-02-23T00:49:06.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:06.040321+0000 osd.3 (osd.3) 207 : cluster [DBG] 62.c deep-scrub ok 2024-02-23T00:49:06.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:06.215796+0000 mon.a (mon.0) 2636 : cluster [DBG] osdmap e448: 8 total, 8 up, 7 in 2024-02-23T00:49:06.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:06.221108+0000 mon.a (mon.0) 2637 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:49:06.828 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:49:06.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:04.527131+0000 osd.1 (osd.1) 197 : cluster [DBG] 2.d deep-scrub starts 2024-02-23T00:49:06.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:04.527997+0000 osd.1 (osd.1) 198 : cluster [DBG] 2.d deep-scrub ok 2024-02-23T00:49:06.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:05.301957+0000 mgr.y (mgr.14517) 417 : cluster [DBG] pgmap v659: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-23T00:49:06.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:05.328344+0000 osd.6 (osd.6) 181 : cluster [DBG] 2.17 scrub starts 2024-02-23T00:49:06.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:05.329173+0000 osd.6 (osd.6) 182 : cluster [DBG] 2.17 scrub ok 2024-02-23T00:49:06.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:05.881876+0000 osd.1 (osd.1) 199 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:49:06.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:05.882022+0000 osd.1 (osd.1) 200 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:49:06.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:06.036908+0000 osd.3 (osd.3) 206 : cluster [DBG] 62.c deep-scrub starts 2024-02-23T00:49:06.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:06.040321+0000 osd.3 (osd.3) 207 : cluster [DBG] 62.c deep-scrub ok 2024-02-23T00:49:06.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:06.215796+0000 mon.a (mon.0) 2636 : cluster [DBG] osdmap e448: 8 total, 8 up, 7 in 2024-02-23T00:49:06.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:06.221108+0000 mon.a (mon.0) 2637 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:49:07.160 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:49:07.167 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:07.167 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:49:07.268 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:49:07.300 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:07.775 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph tell osd.1 reset_purged_snaps_last 2024-02-23T00:49:07.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:06.488710+0000 osd.7 (osd.7) 219 : cluster [DBG] 5.1a deep-scrub starts 2024-02-23T00:49:07.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:06.491784+0000 osd.7 (osd.7) 220 : cluster [DBG] 5.1a deep-scrub ok 2024-02-23T00:49:07.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:06.615756+0000 osd.6 (osd.6) 183 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:49:07.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:06.615915+0000 osd.6 (osd.6) 184 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:49:07.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:07.217845+0000 mon.a (mon.0) 2638 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:49:07.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:07.218018+0000 mon.a (mon.0) 2639 : cluster [DBG] osdmap e449: 8 total, 8 up, 7 in 2024-02-23T00:49:07.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:07.220274+0000 mon.a (mon.0) 2640 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:49:07.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:07.258104+0000 mon.a (mon.0) 2641 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:49:07.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:07.258535+0000 mon.a (mon.0) 2642 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-67"}]: dispatch 2024-02-23T00:49:08.002 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:06.488710+0000 osd.7 (osd.7) 219 : cluster [DBG] 5.1a deep-scrub starts 2024-02-23T00:49:08.003 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:06.491784+0000 osd.7 (osd.7) 220 : cluster [DBG] 5.1a deep-scrub ok 2024-02-23T00:49:08.003 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:06.615756+0000 osd.6 (osd.6) 183 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:49:08.003 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:06.615915+0000 osd.6 (osd.6) 184 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:49:08.003 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:07.217845+0000 mon.a (mon.0) 2638 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:49:08.003 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:07.218018+0000 mon.a (mon.0) 2639 : cluster [DBG] osdmap e449: 8 total, 8 up, 7 in 2024-02-23T00:49:08.003 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:07.220274+0000 mon.a (mon.0) 2640 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:49:08.003 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:07.258104+0000 mon.a (mon.0) 2641 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:49:08.003 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:07.258535+0000 mon.a (mon.0) 2642 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-67"}]: dispatch 2024-02-23T00:49:08.003 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:06.488710+0000 osd.7 (osd.7) 219 : cluster [DBG] 5.1a deep-scrub starts 2024-02-23T00:49:08.003 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:06.491784+0000 osd.7 (osd.7) 220 : cluster [DBG] 5.1a deep-scrub ok 2024-02-23T00:49:08.004 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:06.615756+0000 osd.6 (osd.6) 183 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:49:08.004 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:06.615915+0000 osd.6 (osd.6) 184 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:49:08.004 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:07.217845+0000 mon.a (mon.0) 2638 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:49:08.004 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:07.218018+0000 mon.a (mon.0) 2639 : cluster [DBG] osdmap e449: 8 total, 8 up, 7 in 2024-02-23T00:49:08.004 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:07.220274+0000 mon.a (mon.0) 2640 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:49:08.004 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:07.258104+0000 mon.a (mon.0) 2641 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:49:08.004 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:07.258535+0000 mon.a (mon.0) 2642 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-67"}]: dispatch 2024-02-23T00:49:08.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.258+0000 7f3981c3b700 1 -- 172.21.15.47:0/3213964980 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f397c102210 msgr2=0x7f397c104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:08.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.258+0000 7f3981c3b700 1 --2- 172.21.15.47:0/3213964980 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f397c102210 0x7f397c104600 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 rx=0x7f3974009be0 tx=0x7f3974004d10).stop 2024-02-23T00:49:08.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.258+0000 7f3981c3b700 1 -- 172.21.15.47:0/3213964980 shutdown_connections 2024-02-23T00:49:08.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.258+0000 7f3981c3b700 1 --2- 172.21.15.47:0/3213964980 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f397c102210 0x7f397c104600 unknown :-1 s=CLOSED pgs=2682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:08.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.258+0000 7f3981c3b700 1 --2- 172.21.15.47:0/3213964980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f397c0f8cb0 0x7f397c0f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:08.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.258+0000 7f3981c3b700 1 --2- 172.21.15.47:0/3213964980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f397c0f9610 0x7f397c101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:08.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.258+0000 7f3981c3b700 1 -- 172.21.15.47:0/3213964980 >> 172.21.15.47:0/3213964980 conn(0x7f397c0f2820 msgr2=0x7f397c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:08.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.258+0000 7f3981c3b700 1 -- 172.21.15.47:0/3213964980 shutdown_connections 2024-02-23T00:49:08.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.258+0000 7f3981c3b700 1 -- 172.21.15.47:0/3213964980 wait complete. 2024-02-23T00:49:08.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.259+0000 7f3981c3b700 1 librados: starting msgr at 2024-02-23T00:49:08.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.259+0000 7f3981c3b700 1 librados: starting objecter 2024-02-23T00:49:08.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.259+0000 7f3981c3b700 1 Processor -- start 2024-02-23T00:49:08.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.259+0000 7f3981c3b700 1 -- start start 2024-02-23T00:49:08.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.259+0000 7f3981c3b700 1 librados: setting wanted keys 2024-02-23T00:49:08.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.259+0000 7f3981c3b700 1 librados: calling monclient init 2024-02-23T00:49:08.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.259+0000 7f3981c3b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f397c0f8cb0 0x7f397c0f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:08.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.259+0000 7f3981c3b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f397c0f9610 0x7f397c0f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:08.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.259+0000 7f3981c3b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f397c102210 0x7f397c0f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:08.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.259+0000 7f3981c3b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397c116410 con 0x7f397c0f9610 2024-02-23T00:49:08.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.259+0000 7f3981c3b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397c116580 con 0x7f397c102210 2024-02-23T00:49:08.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.259+0000 7f3981c3b700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f397c116740 con 0x7f397c0f8cb0 2024-02-23T00:49:08.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.259+0000 7f397affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f397c0f9610 0x7f397c0f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:08.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.259+0000 7f397b7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f397c0f8cb0 0x7f397c0f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:08.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.259+0000 7f397b7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f397c0f8cb0 0x7f397c0f6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:36842/0 (socket says 172.21.15.47:36842) 2024-02-23T00:49:08.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.260+0000 7f397b7fe700 1 -- 172.21.15.47:0/459630730 learned_addr learned my addr 172.21.15.47:0/459630730 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:49:08.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.259+0000 7f397affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f397c0f9610 0x7f397c0f7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51136/0 (socket says 172.21.15.47:51136) 2024-02-23T00:49:08.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.260+0000 7f397bfff700 1 --2- 172.21.15.47:0/459630730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f397c102210 0x7f397c0f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:08.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.260+0000 7f397b7fe700 1 -- 172.21.15.47:0/459630730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f397c102210 msgr2=0x7f397c0f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:08.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.260+0000 7f397b7fe700 1 --2- 172.21.15.47:0/459630730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f397c102210 0x7f397c0f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:08.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.260+0000 7f397b7fe700 1 -- 172.21.15.47:0/459630730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f397c0f9610 msgr2=0x7f397c0f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.260+0000 7f397b7fe700 1 --2- 172.21.15.47:0/459630730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f397c0f9610 0x7f397c0f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.260+0000 7f397b7fe700 1 -- 172.21.15.47:0/459630730 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39740097c0 con 0x7f397c0f8cb0 2024-02-23T00:49:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.260+0000 7f397b7fe700 1 --2- 172.21.15.47:0/459630730 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f397c0f8cb0 0x7f397c0f6c60 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 rx=0x7f396c00e970 tx=0x7f396c00b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.260+0000 7f3978ff9700 10 client.?.objecter ms_handle_connect 0x7f397c0f8cb0 2024-02-23T00:49:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.260+0000 7f3978ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:49:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.260+0000 7f3978ff9700 1 -- 172.21.15.47:0/459630730 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c00ee60 con 0x7f397c0f8cb0 2024-02-23T00:49:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.260+0000 7f3978ff9700 1 -- 172.21.15.47:0/459630730 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f396c00f040 con 0x7f397c0f8cb0 2024-02-23T00:49:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.260+0000 7f3981c3b700 1 -- 172.21.15.47:0/459630730 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f397c116a20 con 0x7f397c0f8cb0 2024-02-23T00:49:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.260+0000 7f3978ff9700 1 -- 172.21.15.47:0/459630730 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c00c5b0 con 0x7f397c0f8cb0 2024-02-23T00:49:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.260+0000 7f3981c3b700 10 client.41411.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:49:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.260+0000 7f3981c3b700 1 -- 172.21.15.47:0/459630730 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f397c116f60 con 0x7f397c0f8cb0 2024-02-23T00:49:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.260+0000 7f3981c3b700 1 librados: init done 2024-02-23T00:49:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.262+0000 7f3981c3b700 1 -- 172.21.15.47:0/459630730 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f397c05fa60 con 0x7f397c0f8cb0 2024-02-23T00:49:08.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.262+0000 7f3978ff9700 1 -- 172.21.15.47:0/459630730 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f396c007b40 con 0x7f397c0f8cb0 2024-02-23T00:49:08.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.262+0000 7f3978ff9700 1 --2- 172.21.15.47:0/459630730 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3964062cc0 0x7f3964065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:08.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.263+0000 7f3978ff9700 1 -- 172.21.15.47:0/459630730 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(450..450 src has 48..450) v4 ==== 8835+0+0 (secure 0 0 0) 0x7f396c07d0e0 con 0x7f397c0f8cb0 2024-02-23T00:49:08.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.263+0000 7f3978ff9700 10 client.41411.objecter ms_dispatch 0x7f397c001550 osd_map(450..450 src has 48..450) v4 2024-02-23T00:49:08.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.263+0000 7f3978ff9700 3 client.41411.objecter handle_osd_map got epochs [450,450] > 0 2024-02-23T00:49:08.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.263+0000 7f3978ff9700 3 client.41411.objecter handle_osd_map decoding full epoch 450 2024-02-23T00:49:08.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.263+0000 7f3978ff9700 20 client.41411.objecter dump_active .. 0 homeless 2024-02-23T00:49:08.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.268+0000 7f397affd700 1 --2- 172.21.15.47:0/459630730 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3964062cc0 0x7f3964065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:08.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.269+0000 7f397affd700 1 --2- 172.21.15.47:0/459630730 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3964062cc0 0x7f3964065180 secure :-1 s=READY pgs=3077 cs=0 l=1 rev1=1 rx=0x7f397c0f83b0 tx=0x7f397000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:08.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.269+0000 7f3978ff9700 10 client.41411.objecter ms_handle_connect 0x7f3964062cc0 2024-02-23T00:49:08.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.274+0000 7f3978ff9700 1 -- 172.21.15.47:0/459630730 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f396c04af60 con 0x7f397c0f8cb0 2024-02-23T00:49:08.280 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:49:08.285 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:08.286 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:49:08.301 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:49:08.387 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:49:08.417 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:08.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:08.476+0000 7f3981c3b700 1 -- 172.21.15.47:0/459630730 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7f397c0f80f0 con 0x7f397c0f8cb0 2024-02-23T00:49:08.614 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:49:08.615 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:49:08.615 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:49:08.618 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:08.618 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:49:08.719 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:49:08.748 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:08.860 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:06.783087+0000 osd.0 (osd.0) 189 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:49:08.861 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:06.783296+0000 osd.0 (osd.0) 190 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:49:08.861 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:07.302614+0000 mgr.y (mgr.14517) 418 : cluster [DBG] pgmap v662: 208 pgs: 3 creating+activating, 19 creating+peering, 186 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 781 B/s wr, 2 op/s 2024-02-23T00:49:08.861 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:08.220750+0000 mon.a (mon.0) 2643 : cluster [DBG] osdmap e450: 8 total, 8 up, 7 in 2024-02-23T00:49:08.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:06.783087+0000 osd.0 (osd.0) 189 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:49:08.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:06.783296+0000 osd.0 (osd.0) 190 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:49:08.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:07.302614+0000 mgr.y (mgr.14517) 418 : cluster [DBG] pgmap v662: 208 pgs: 3 creating+activating, 19 creating+peering, 186 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 781 B/s wr, 2 op/s 2024-02-23T00:49:08.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:08.220750+0000 mon.a (mon.0) 2643 : cluster [DBG] osdmap e450: 8 total, 8 up, 7 in 2024-02-23T00:49:08.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:06.783087+0000 osd.0 (osd.0) 189 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:49:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:06.783296+0000 osd.0 (osd.0) 190 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:49:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:07.302614+0000 mgr.y (mgr.14517) 418 : cluster [DBG] pgmap v662: 208 pgs: 3 creating+activating, 19 creating+peering, 186 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 781 B/s wr, 2 op/s 2024-02-23T00:49:08.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:08.220750+0000 mon.a (mon.0) 2643 : cluster [DBG] osdmap e450: 8 total, 8 up, 7 in 2024-02-23T00:49:09.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:09.225+0000 7f3978ff9700 1 -- 172.21.15.47:0/459630730 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v451) v1 ==== 87+0+0 (secure 0 0 0) 0x7f396c019070 con 0x7f397c0f8cb0 2024-02-23T00:49:09.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:09.229+0000 7f3981c3b700 10 librados: watch_flush enter 2024-02-23T00:49:09.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:09.230+0000 7f3981c3b700 10 librados: watch_flush exit 2024-02-23T00:49:09.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:09.230+0000 7f3981c3b700 20 client.41411.objecter shutdown clearing up homeless session... 2024-02-23T00:49:09.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:09.230+0000 7f3981c3b700 10 client.41411.objecter successfully canceled tick 2024-02-23T00:49:09.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:09.230+0000 7f3981c3b700 1 -- 172.21.15.47:0/459630730 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3964062cc0 msgr2=0x7f3964065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:09.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:09.230+0000 7f3981c3b700 1 --2- 172.21.15.47:0/459630730 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3964062cc0 0x7f3964065180 secure :-1 s=READY pgs=3077 cs=0 l=1 rev1=1 rx=0x7f397c0f83b0 tx=0x7f397000b040).stop 2024-02-23T00:49:09.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:09.230+0000 7f3981c3b700 1 -- 172.21.15.47:0/459630730 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f397c0f8cb0 msgr2=0x7f397c0f6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:09.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:09.230+0000 7f3981c3b700 1 --2- 172.21.15.47:0/459630730 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f397c0f8cb0 0x7f397c0f6c60 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 rx=0x7f396c00e970 tx=0x7f396c00b040).stop 2024-02-23T00:49:09.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:09.230+0000 7f3981c3b700 1 -- 172.21.15.47:0/459630730 shutdown_connections 2024-02-23T00:49:09.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:09.230+0000 7f3981c3b700 1 --2- 172.21.15.47:0/459630730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f397c102210 0x7f397c0f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:09.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:09.230+0000 7f3981c3b700 1 --2- 172.21.15.47:0/459630730 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3964062cc0 0x7f3964065180 unknown :-1 s=CLOSED pgs=3077 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:09.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:09.230+0000 7f3981c3b700 1 --2- 172.21.15.47:0/459630730 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f397c0f8cb0 0x7f397c0f6c60 unknown :-1 s=CLOSED pgs=2683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:09.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:09.230+0000 7f3981c3b700 1 --2- 172.21.15.47:0/459630730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f397c0f9610 0x7f397c0f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:09.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:09.230+0000 7f3981c3b700 1 -- 172.21.15.47:0/459630730 >> 172.21.15.47:0/459630730 conn(0x7f397c0f2820 msgr2=0x7f397c0f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:09.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:09.230+0000 7f3981c3b700 1 -- 172.21.15.47:0/459630730 shutdown_connections 2024-02-23T00:49:09.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:09.230+0000 7f3981c3b700 1 -- 172.21.15.47:0/459630730 wait complete. 2024-02-23T00:49:09.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:09.230+0000 7f3981c3b700 1 librados: shutdown 2024-02-23T00:49:09.234 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is set 2024-02-23T00:49:09.415 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.4 dump_blocked_ops 2024-02-23T00:49:09.497 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:49:09.501 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:09.501 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:49:09.597 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:49:09.602 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:49:09.629 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:09.649 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:49:09.651 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:09.652 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:49:09.684 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:49:09.753 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:49:09.780 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:09.911 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:08.043672+0000 osd.3 (osd.3) 208 : cluster [DBG] 5.0 scrub starts 2024-02-23T00:49:09.911 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:08.047138+0000 osd.3 (osd.3) 209 : cluster [DBG] 5.0 scrub ok 2024-02-23T00:49:09.911 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:08.188399+0000 osd.4 (osd.4) 187 : cluster [DBG] 4.9 scrub starts 2024-02-23T00:49:09.911 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:08.190359+0000 osd.4 (osd.4) 188 : cluster [DBG] 4.9 scrub ok 2024-02-23T00:49:09.911 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:08.478100+0000 mon.c (mon.2) 305 : audit [INF] from='client.? 172.21.15.47:0/459630730' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:49:09.911 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:08.672567+0000 mon.a (mon.0) 2644 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:49:09.911 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:09.069546+0000 osd.3 (osd.3) 210 : cluster [DBG] 62.18 scrub starts 2024-02-23T00:49:09.912 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:09.072996+0000 osd.3 (osd.3) 211 : cluster [DBG] 62.18 scrub ok 2024-02-23T00:49:09.912 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:09.218453+0000 mon.a (mon.0) 2645 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:49:09.912 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:09.222981+0000 mon.a (mon.0) 2646 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:49:09.912 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:09.223032+0000 mon.a (mon.0) 2647 : cluster [DBG] osdmap e451: 8 total, 8 up, 7 in 2024-02-23T00:49:09.912 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:09.224718+0000 mon.a (mon.0) 2648 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:49:10.023 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:08.043672+0000 osd.3 (osd.3) 208 : cluster [DBG] 5.0 scrub starts 2024-02-23T00:49:10.023 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:08.047138+0000 osd.3 (osd.3) 209 : cluster [DBG] 5.0 scrub ok 2024-02-23T00:49:10.023 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:08.188399+0000 osd.4 (osd.4) 187 : cluster [DBG] 4.9 scrub starts 2024-02-23T00:49:10.023 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:08.190359+0000 osd.4 (osd.4) 188 : cluster [DBG] 4.9 scrub ok 2024-02-23T00:49:10.023 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:08.478100+0000 mon.c (mon.2) 305 : audit [INF] from='client.? 172.21.15.47:0/459630730' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:49:10.023 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:08.672567+0000 mon.a (mon.0) 2644 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:49:10.023 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:09.069546+0000 osd.3 (osd.3) 210 : cluster [DBG] 62.18 scrub starts 2024-02-23T00:49:10.023 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:09.072996+0000 osd.3 (osd.3) 211 : cluster [DBG] 62.18 scrub ok 2024-02-23T00:49:10.023 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:09.218453+0000 mon.a (mon.0) 2645 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:49:10.023 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:09.222981+0000 mon.a (mon.0) 2646 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:49:10.023 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:09.223032+0000 mon.a (mon.0) 2647 : cluster [DBG] osdmap e451: 8 total, 8 up, 7 in 2024-02-23T00:49:10.024 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:09.224718+0000 mon.a (mon.0) 2648 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:49:10.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:08.043672+0000 osd.3 (osd.3) 208 : cluster [DBG] 5.0 scrub starts 2024-02-23T00:49:10.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:08.047138+0000 osd.3 (osd.3) 209 : cluster [DBG] 5.0 scrub ok 2024-02-23T00:49:10.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:08.188399+0000 osd.4 (osd.4) 187 : cluster [DBG] 4.9 scrub starts 2024-02-23T00:49:10.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:08.190359+0000 osd.4 (osd.4) 188 : cluster [DBG] 4.9 scrub ok 2024-02-23T00:49:10.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:08.478100+0000 mon.c (mon.2) 305 : audit [INF] from='client.? 172.21.15.47:0/459630730' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:49:10.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:08.672567+0000 mon.a (mon.0) 2644 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:49:10.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:09.069546+0000 osd.3 (osd.3) 210 : cluster [DBG] 62.18 scrub starts 2024-02-23T00:49:10.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:09.072996+0000 osd.3 (osd.3) 211 : cluster [DBG] 62.18 scrub ok 2024-02-23T00:49:10.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:09.218453+0000 mon.a (mon.0) 2645 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:49:10.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:09.222981+0000 mon.a (mon.0) 2646 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:49:10.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:09.223032+0000 mon.a (mon.0) 2647 : cluster [DBG] osdmap e451: 8 total, 8 up, 7 in 2024-02-23T00:49:10.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:09.224718+0000 mon.a (mon.0) 2648 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:49:10.030 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:49:10.035 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:10.035 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:49:10.136 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:49:10.165 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:10.413 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:49:10.416 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:10.416 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:49:10.517 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:49:10.545 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:10.807 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:49:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:49:10] "GET /metrics HTTP/1.1" 200 189093 "" "Prometheus/2.33.4" 2024-02-23T00:49:10.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:09.303066+0000 mgr.y (mgr.14517) 419 : cluster [DBG] pgmap v665: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-23T00:49:10.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:10.226656+0000 mon.a (mon.0) 2649 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:49:10.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:10.226905+0000 mon.a (mon.0) 2650 : cluster [DBG] osdmap e452: 8 total, 8 up, 7 in 2024-02-23T00:49:10.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:10.231995+0000 mon.a (mon.0) 2651 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:49:10.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:10.235476+0000 mon.a (mon.0) 2652 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-23T00:49:10.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:09.303066+0000 mgr.y (mgr.14517) 419 : cluster [DBG] pgmap v665: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-23T00:49:10.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:10.226656+0000 mon.a (mon.0) 2649 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:49:10.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:10.226905+0000 mon.a (mon.0) 2650 : cluster [DBG] osdmap e452: 8 total, 8 up, 7 in 2024-02-23T00:49:10.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:10.231995+0000 mon.a (mon.0) 2651 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:49:10.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:10.235476+0000 mon.a (mon.0) 2652 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-23T00:49:10.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:09.303066+0000 mgr.y (mgr.14517) 419 : cluster [DBG] pgmap v665: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-23T00:49:10.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:10.226656+0000 mon.a (mon.0) 2649 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:49:10.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:10.226905+0000 mon.a (mon.0) 2650 : cluster [DBG] osdmap e452: 8 total, 8 up, 7 in 2024-02-23T00:49:10.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:10.231995+0000 mon.a (mon.0) 2651 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:49:10.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:10.235476+0000 mon.a (mon.0) 2652 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-23T00:49:11.093 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:49:11.096 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:11.097 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:49:11.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.146+0000 7fcf2cc9c700 1 -- 172.21.15.47:0/3518345736 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcf280f9270 msgr2=0x7fcf280f6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:11.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.146+0000 7fcf2cc9c700 1 --2- 172.21.15.47:0/3518345736 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcf280f9270 0x7fcf280f6c50 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 rx=0x7fcf1c0047b0 tx=0x7fcf1c009cb0).stop 2024-02-23T00:49:11.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.146+0000 7fcf2cc9c700 1 -- 172.21.15.47:0/3518345736 shutdown_connections 2024-02-23T00:49:11.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.146+0000 7fcf2cc9c700 1 --2- 172.21.15.47:0/3518345736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf280f7330 0x7fcf28103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:11.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.146+0000 7fcf2cc9c700 1 --2- 172.21.15.47:0/3518345736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf280f89a0 0x7fcf280f8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:11.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.146+0000 7fcf2cc9c700 1 --2- 172.21.15.47:0/3518345736 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcf280f9270 0x7fcf280f6c50 unknown :-1 s=CLOSED pgs=2684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:11.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.146+0000 7fcf2cc9c700 1 -- 172.21.15.47:0/3518345736 >> 172.21.15.47:0/3518345736 conn(0x7fcf280f26f0 msgr2=0x7fcf280f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:11.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.146+0000 7fcf2cc9c700 1 -- 172.21.15.47:0/3518345736 shutdown_connections 2024-02-23T00:49:11.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.146+0000 7fcf2cc9c700 1 -- 172.21.15.47:0/3518345736 wait complete. 2024-02-23T00:49:11.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.146+0000 7fcf2cc9c700 1 librados: starting msgr at 2024-02-23T00:49:11.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.146+0000 7fcf2cc9c700 1 librados: starting objecter 2024-02-23T00:49:11.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.147+0000 7fcf2cc9c700 1 Processor -- start 2024-02-23T00:49:11.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.147+0000 7fcf2cc9c700 1 -- start start 2024-02-23T00:49:11.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.147+0000 7fcf2cc9c700 1 librados: setting wanted keys 2024-02-23T00:49:11.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.147+0000 7fcf2cc9c700 1 librados: calling monclient init 2024-02-23T00:49:11.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.147+0000 7fcf2cc9c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf280f7330 0x7fcf2810df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:11.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.147+0000 7fcf2cc9c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcf280f89a0 0x7fcf2810e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:11.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.147+0000 7fcf2cc9c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf280f9270 0x7fcf28108130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:11.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.147+0000 7fcf2cc9c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf28108620 con 0x7fcf280f7330 2024-02-23T00:49:11.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.147+0000 7fcf2cc9c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf28108790 con 0x7fcf280f9270 2024-02-23T00:49:11.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.147+0000 7fcf2cc9c700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf28108950 con 0x7fcf280f89a0 2024-02-23T00:49:11.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.147+0000 7fcf2659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf280f7330 0x7fcf2810df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:11.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.147+0000 7fcf25d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcf280f89a0 0x7fcf2810e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:11.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.147+0000 7fcf2659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf280f7330 0x7fcf2810df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51146/0 (socket says 172.21.15.47:51146) 2024-02-23T00:49:11.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.147+0000 7fcf2659c700 1 -- 172.21.15.47:0/2527556800 learned_addr learned my addr 172.21.15.47:0/2527556800 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:49:11.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.147+0000 7fcf26d9d700 1 --2- 172.21.15.47:0/2527556800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf280f9270 0x7fcf28108130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:11.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.147+0000 7fcf25d9b700 1 -- 172.21.15.47:0/2527556800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf280f9270 msgr2=0x7fcf28108130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:11.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.147+0000 7fcf25d9b700 1 --2- 172.21.15.47:0/2527556800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf280f9270 0x7fcf28108130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:11.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.147+0000 7fcf25d9b700 1 -- 172.21.15.47:0/2527556800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf280f7330 msgr2=0x7fcf2810df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:11.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.147+0000 7fcf25d9b700 1 --2- 172.21.15.47:0/2527556800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf280f7330 0x7fcf2810df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:11.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.147+0000 7fcf25d9b700 1 -- 172.21.15.47:0/2527556800 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf200097c0 con 0x7fcf280f89a0 2024-02-23T00:49:11.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.147+0000 7fcf2659c700 1 --2- 172.21.15.47:0/2527556800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf280f7330 0x7fcf2810df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:49:11.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.147+0000 7fcf25d9b700 1 --2- 172.21.15.47:0/2527556800 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcf280f89a0 0x7fcf2810e480 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 rx=0x7fcf1c004650 tx=0x7fcf1c009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:11.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.147+0000 7fcf177fe700 10 client.?.objecter ms_handle_connect 0x7fcf280f89a0 2024-02-23T00:49:11.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.147+0000 7fcf177fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:49:11.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.147+0000 7fcf177fe700 1 -- 172.21.15.47:0/2527556800 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c01c070 con 0x7fcf280f89a0 2024-02-23T00:49:11.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.148+0000 7fcf177fe700 1 -- 172.21.15.47:0/2527556800 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fcf1c009620 con 0x7fcf280f89a0 2024-02-23T00:49:11.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.148+0000 7fcf2cc9c700 1 -- 172.21.15.47:0/2527556800 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf1c005040 con 0x7fcf280f89a0 2024-02-23T00:49:11.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.148+0000 7fcf177fe700 1 -- 172.21.15.47:0/2527556800 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c0123e0 con 0x7fcf280f89a0 2024-02-23T00:49:11.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.148+0000 7fcf2cc9c700 10 client.41417.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:49:11.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.148+0000 7fcf2cc9c700 1 -- 172.21.15.47:0/2527556800 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf28108df0 con 0x7fcf280f89a0 2024-02-23T00:49:11.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.148+0000 7fcf2cc9c700 1 librados: init done 2024-02-23T00:49:11.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.148+0000 7fcf2cc9c700 10 client.41417.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:11.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.148+0000 7fcf2cc9c700 15 client.41417.objecter _session_command_op_assign -1 1 2024-02-23T00:49:11.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.148+0000 7fcf2cc9c700 20 client.41417.objecter _get_session osd=-1 returning homeless 2024-02-23T00:49:11.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.148+0000 7fcf2cc9c700 10 client.41417.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:49:11.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.148+0000 7fcf2cc9c700 1 -- 172.21.15.47:0/2527556800 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcf2805ee80 con 0x7fcf280f89a0 2024-02-23T00:49:11.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.149+0000 7fcf177fe700 1 -- 172.21.15.47:0/2527556800 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fcf1c021420 con 0x7fcf280f89a0 2024-02-23T00:49:11.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.149+0000 7fcf177fe700 1 --2- 172.21.15.47:0/2527556800 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcf10062cc0 0x7fcf10065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:11.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.149+0000 7fcf177fe700 1 -- 172.21.15.47:0/2527556800 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(452..452 src has 48..452) v4 ==== 9220+0+0 (secure 0 0 0) 0x7fcf1c0830a0 con 0x7fcf280f89a0 2024-02-23T00:49:11.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.149+0000 7fcf177fe700 10 client.41417.objecter ms_dispatch 0x7fcf28001550 osd_map(452..452 src has 48..452) v4 2024-02-23T00:49:11.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.149+0000 7fcf177fe700 3 client.41417.objecter handle_osd_map got epochs [452,452] > 0 2024-02-23T00:49:11.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.149+0000 7fcf177fe700 3 client.41417.objecter handle_osd_map decoding full epoch 452 2024-02-23T00:49:11.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.149+0000 7fcf2659c700 1 --2- 172.21.15.47:0/2527556800 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcf10062cc0 0x7fcf10065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:11.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.149+0000 7fcf177fe700 10 client.41417.objecter checking command 1 2024-02-23T00:49:11.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.149+0000 7fcf177fe700 1 --2- 172.21.15.47:0/2527556800 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fcf1006f8f0 0x7fcf10071d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:11.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.149+0000 7fcf177fe700 20 client.41417.objecter _get_session s=0x7fcf1006f220 osd=1 3 2024-02-23T00:49:11.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.149+0000 7fcf177fe700 20 client.41417.objecter put_session s=0x7fcf1006f220 osd=1 3 2024-02-23T00:49:11.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.150+0000 7fcf177fe700 15 client.41417.objecter _session_command_op_remove -1 1 2024-02-23T00:49:11.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.150+0000 7fcf177fe700 20 client.41417.objecter _get_session s=0x7fcf1006f220 osd=1 3 2024-02-23T00:49:11.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.150+0000 7fcf177fe700 20 client.41417.objecter get_session s=0x7fcf1006f220 osd=1 3 2024-02-23T00:49:11.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.150+0000 7fcf177fe700 15 client.41417.objecter _session_command_op_assign 1 1 2024-02-23T00:49:11.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.150+0000 7fcf177fe700 20 client.41417.objecter put_session s=0x7fcf1006f220 osd=1 4 2024-02-23T00:49:11.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.150+0000 7fcf177fe700 10 client.41417.objecter _send_command 1 2024-02-23T00:49:11.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.150+0000 7fcf177fe700 1 -- 172.21.15.47:0/2527556800 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcf10072410 con 0x7fcf1006f8f0 2024-02-23T00:49:11.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.150+0000 7fcf177fe700 20 client.41417.objecter dump_active .. 0 homeless 2024-02-23T00:49:11.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.150+0000 7fcf177fe700 1 -- 172.21.15.47:0/2527556800 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_get_version_reply(handle=1 version=452) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf1c085600 con 0x7fcf280f89a0 2024-02-23T00:49:11.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.150+0000 7fcf26d9d700 1 --2- 172.21.15.47:0/2527556800 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fcf1006f8f0 0x7fcf10071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:11.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.150+0000 7fcf177fe700 10 client.41417.objecter ms_handle_connect 0x7fcf10062cc0 2024-02-23T00:49:11.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.150+0000 7fcf2659c700 1 --2- 172.21.15.47:0/2527556800 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcf10062cc0 0x7fcf10065180 secure :-1 s=READY pgs=3078 cs=0 l=1 rev1=1 rx=0x7fcf18004fd0 tx=0x7fcf1800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:11.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.150+0000 7fcf177fe700 10 client.41417.objecter ms_handle_connect 0x7fcf1006f8f0 2024-02-23T00:49:11.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.150+0000 7fcf26d9d700 1 --2- 172.21.15.47:0/2527556800 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fcf1006f8f0 0x7fcf10071d90 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:11.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.150+0000 7fcf177fe700 1 -- 172.21.15.47:0/2527556800 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcf10072410 con 0x7fcf1006f8f0 2024-02-23T00:49:11.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.150+0000 7fcf177fe700 10 client.41417.objecter ms_dispatch 0x7fcf28001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:49:11.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.150+0000 7fcf177fe700 10 client.41417.objecter _finish_command 1 = system:0 2024-02-23T00:49:11.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.150+0000 7fcf177fe700 20 client.41417.objecter put_session s=0x7fcf1006f220 osd=1 4 2024-02-23T00:49:11.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.150+0000 7fcf177fe700 15 client.41417.objecter _session_command_op_remove 1 1 2024-02-23T00:49:11.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.167+0000 7fcf2cc9c700 10 client.41417.objecter _submit_command 2 [{"prefix": "reset_purged_snaps_last"}] 2024-02-23T00:49:11.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.167+0000 7fcf2cc9c700 15 client.41417.objecter _session_command_op_assign -1 2 2024-02-23T00:49:11.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.167+0000 7fcf2cc9c700 20 client.41417.objecter _get_session s=0x7fcf1006f220 osd=1 3 2024-02-23T00:49:11.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.167+0000 7fcf2cc9c700 20 client.41417.objecter put_session s=0x7fcf1006f220 osd=1 3 2024-02-23T00:49:11.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.167+0000 7fcf2cc9c700 20 client.41417.objecter _get_session s=0x7fcf1006f220 osd=1 3 2024-02-23T00:49:11.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.167+0000 7fcf2cc9c700 15 client.41417.objecter _session_command_op_remove -1 2 2024-02-23T00:49:11.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.167+0000 7fcf2cc9c700 20 client.41417.objecter get_session s=0x7fcf1006f220 osd=1 3 2024-02-23T00:49:11.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.167+0000 7fcf2cc9c700 15 client.41417.objecter _session_command_op_assign 1 2 2024-02-23T00:49:11.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.167+0000 7fcf2cc9c700 20 client.41417.objecter put_session s=0x7fcf1006f220 osd=1 4 2024-02-23T00:49:11.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.167+0000 7fcf2cc9c700 10 client.41417.objecter _send_command 2 2024-02-23T00:49:11.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.167+0000 7fcf2cc9c700 1 -- 172.21.15.47:0/2527556800 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) v1 -- 0x7fcf28105f00 con 0x7fcf1006f8f0 2024-02-23T00:49:11.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.168+0000 7fcf177fe700 1 -- 172.21.15.47:0/2527556800 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fcf28105f00 con 0x7fcf1006f8f0 2024-02-23T00:49:11.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.168+0000 7fcf177fe700 10 client.41417.objecter ms_dispatch 0x7fcf28001550 command_reply(tid 2: 0 ) v1 2024-02-23T00:49:11.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.168+0000 7fcf177fe700 10 client.41417.objecter _finish_command 2 = system:0 2024-02-23T00:49:11.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.168+0000 7fcf177fe700 20 client.41417.objecter put_session s=0x7fcf1006f220 osd=1 4 2024-02-23T00:49:11.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.168+0000 7fcf177fe700 15 client.41417.objecter _session_command_op_remove 1 2 2024-02-23T00:49:11.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.168+0000 7fcf2cc9c700 10 librados: watch_flush enter 2024-02-23T00:49:11.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.168+0000 7fcf2cc9c700 10 librados: watch_flush exit 2024-02-23T00:49:11.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.168+0000 7fcf2cc9c700 10 client.41417.objecter close_session for osd.1 2024-02-23T00:49:11.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.168+0000 7fcf2cc9c700 1 -- 172.21.15.47:0/2527556800 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fcf1006f8f0 msgr2=0x7fcf10071d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:11.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.168+0000 7fcf2cc9c700 1 --2- 172.21.15.47:0/2527556800 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fcf1006f8f0 0x7fcf10071d90 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:11.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.168+0000 7fcf2cc9c700 20 client.41417.objecter put_session s=0x7fcf1006f220 osd=1 1 2024-02-23T00:49:11.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.168+0000 7fcf2cc9c700 20 client.41417.objecter shutdown clearing up homeless session... 2024-02-23T00:49:11.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.168+0000 7fcf2cc9c700 10 client.41417.objecter successfully canceled tick 2024-02-23T00:49:11.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.168+0000 7fcf2cc9c700 1 -- 172.21.15.47:0/2527556800 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcf10062cc0 msgr2=0x7fcf10065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:11.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.168+0000 7fcf2cc9c700 1 --2- 172.21.15.47:0/2527556800 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcf10062cc0 0x7fcf10065180 secure :-1 s=READY pgs=3078 cs=0 l=1 rev1=1 rx=0x7fcf18004fd0 tx=0x7fcf1800b040).stop 2024-02-23T00:49:11.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.168+0000 7fcf2cc9c700 1 -- 172.21.15.47:0/2527556800 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcf280f89a0 msgr2=0x7fcf2810e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:11.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.168+0000 7fcf2cc9c700 1 --2- 172.21.15.47:0/2527556800 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcf280f89a0 0x7fcf2810e480 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 rx=0x7fcf1c004650 tx=0x7fcf1c009cb0).stop 2024-02-23T00:49:11.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.168+0000 7fcf26d9d700 1 -- 172.21.15.47:0/2527556800 reap_dead start 2024-02-23T00:49:11.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.168+0000 7fcf2cc9c700 1 -- 172.21.15.47:0/2527556800 shutdown_connections 2024-02-23T00:49:11.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.168+0000 7fcf2cc9c700 1 -- 172.21.15.47:0/2527556800 >> 172.21.15.47:0/2527556800 conn(0x7fcf280f26f0 msgr2=0x7fcf28060fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:11.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.168+0000 7fcf2cc9c700 1 -- 172.21.15.47:0/2527556800 shutdown_connections 2024-02-23T00:49:11.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.168+0000 7fcf2cc9c700 1 -- 172.21.15.47:0/2527556800 wait complete. 2024-02-23T00:49:11.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:11.168+0000 7fcf2cc9c700 1 librados: shutdown 2024-02-23T00:49:11.198 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:49:11.234 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:11.349 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:49:11.350 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:49:11.447 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:49:11.788 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:49:11.791 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:11.791 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:49:11.892 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:49:11.922 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:12.044 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-23T00:49:12.054 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.4 dump_historic_ops 2024-02-23T00:49:12.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:11.228330+0000 mon.a (mon.0) 2653 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-23T00:49:12.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:11.228395+0000 mon.a (mon.0) 2654 : cluster [DBG] osdmap e453: 8 total, 8 up, 7 in 2024-02-23T00:49:12.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:11.303427+0000 mgr.y (mgr.14517) 420 : cluster [DBG] pgmap v668: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:49:12.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:11.303651+0000 mon.a (mon.0) 2655 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:49:12.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:11.228330+0000 mon.a (mon.0) 2653 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-23T00:49:12.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:11.228395+0000 mon.a (mon.0) 2654 : cluster [DBG] osdmap e453: 8 total, 8 up, 7 in 2024-02-23T00:49:12.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:11.303427+0000 mgr.y (mgr.14517) 420 : cluster [DBG] pgmap v668: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:49:12.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:11.303651+0000 mon.a (mon.0) 2655 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:49:12.330 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:49:12.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:11.228330+0000 mon.a (mon.0) 2653 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-23T00:49:12.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:11.228395+0000 mon.a (mon.0) 2654 : cluster [DBG] osdmap e453: 8 total, 8 up, 7 in 2024-02-23T00:49:12.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:11.303427+0000 mgr.y (mgr.14517) 420 : cluster [DBG] pgmap v668: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:49:12.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:11.303651+0000 mon.a (mon.0) 2655 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:49:12.538 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph tell osd.3 reset_purged_snaps_last 2024-02-23T00:49:12.868 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:49:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:49:12.868 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:49:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:49:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:49:12.971 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:49:12.974 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:12.974 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:49:13.075 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:49:13.105 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:13.177 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:49:13.179 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:13.180 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:49:13.281 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:49:13.307 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:13.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:12.234992+0000 mon.a (mon.0) 2656 : cluster [DBG] osdmap e454: 8 total, 8 up, 7 in 2024-02-23T00:49:13.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:12.631448+0000 mgr.y (mgr.14517) 421 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:49:13.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:12.234992+0000 mon.a (mon.0) 2656 : cluster [DBG] osdmap e454: 8 total, 8 up, 7 in 2024-02-23T00:49:13.542 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:12.631448+0000 mgr.y (mgr.14517) 421 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:49:13.542 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:12.234992+0000 mon.a (mon.0) 2656 : cluster [DBG] osdmap e454: 8 total, 8 up, 7 in 2024-02-23T00:49:13.542 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:12.631448+0000 mgr.y (mgr.14517) 421 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:49:13.594 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:49:13.655 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:49:13.658 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:13.658 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:49:13.759 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:49:13.786 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:13.910 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:49:13.913 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:13.913 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:49:14.014 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:49:14.042 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:14.139 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:49:14.142 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:14.142 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:49:14.152 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:49:14.152 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:49:14.243 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:49:14.270 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:14.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.260+0000 7f35e2c41700 1 -- 172.21.15.47:0/540930939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35dc0fd510 msgr2=0x7f35dc0f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:14.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.260+0000 7f35e2c41700 1 --2- 172.21.15.47:0/540930939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35dc0fd510 0x7f35dc0f6d40 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 rx=0x7f35c8004870 tx=0x7f35c8009cb0).stop 2024-02-23T00:49:14.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.260+0000 7f35e2c41700 1 -- 172.21.15.47:0/540930939 shutdown_connections 2024-02-23T00:49:14.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.260+0000 7f35e2c41700 1 --2- 172.21.15.47:0/540930939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35dc0f7400 0x7f35dc103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:14.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.260+0000 7f35e2c41700 1 --2- 172.21.15.47:0/540930939 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35dc0fcbb0 0x7f35dc0fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:14.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.260+0000 7f35e2c41700 1 --2- 172.21.15.47:0/540930939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35dc0fd510 0x7f35dc0f6d40 unknown :-1 s=CLOSED pgs=2540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:14.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.260+0000 7f35e2c41700 1 -- 172.21.15.47:0/540930939 >> 172.21.15.47:0/540930939 conn(0x7f35dc0f2820 msgr2=0x7f35dc0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:14.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.260+0000 7f35e2c41700 1 -- 172.21.15.47:0/540930939 shutdown_connections 2024-02-23T00:49:14.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.260+0000 7f35e2c41700 1 -- 172.21.15.47:0/540930939 wait complete. 2024-02-23T00:49:14.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.260+0000 7f35e2c41700 1 librados: starting msgr at 2024-02-23T00:49:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.260+0000 7f35e2c41700 1 librados: starting objecter 2024-02-23T00:49:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.260+0000 7f35e2c41700 1 Processor -- start 2024-02-23T00:49:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.260+0000 7f35e2c41700 1 -- start start 2024-02-23T00:49:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.260+0000 7f35e2c41700 1 librados: setting wanted keys 2024-02-23T00:49:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.260+0000 7f35e2c41700 1 librados: calling monclient init 2024-02-23T00:49:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.261+0000 7f35e2c41700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35dc0f7400 0x7f35dc0f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.261+0000 7f35e2c41700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35dc0fcbb0 0x7f35dc0f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.261+0000 7f35e2c41700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35dc0fd510 0x7f35dc0f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.261+0000 7f35e2c41700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc116530 con 0x7f35dc0fd510 2024-02-23T00:49:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.261+0000 7f35e2c41700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc116670 con 0x7f35dc0f7400 2024-02-23T00:49:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.261+0000 7f35e2c41700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc116830 con 0x7f35dc0fcbb0 2024-02-23T00:49:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.261+0000 7f35dbfff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35dc0fcbb0 0x7f35dc0f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.261+0000 7f35e11de700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35dc0fd510 0x7f35dc0f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.261+0000 7f35e09dd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35dc0f7400 0x7f35dc0f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.261+0000 7f35dbfff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35dc0fcbb0 0x7f35dc0f8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:36860/0 (socket says 172.21.15.47:36860) 2024-02-23T00:49:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.261+0000 7f35dbfff700 1 -- 172.21.15.47:0/1343224562 learned_addr learned my addr 172.21.15.47:0/1343224562 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:49:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.261+0000 7f35e11de700 1 -- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35dc0fcbb0 msgr2=0x7f35dc0f8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.261+0000 7f35e11de700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35dc0fcbb0 0x7f35dc0f8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.261+0000 7f35e11de700 1 -- 172.21.15.47:0/1343224562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35dc0f7400 msgr2=0x7f35dc0f89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.261+0000 7f35e11de700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35dc0f7400 0x7f35dc0f89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.261+0000 7f35e11de700 1 -- 172.21.15.47:0/1343224562 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d0005040 con 0x7f35dc0fd510 2024-02-23T00:49:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.261+0000 7f35dbfff700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35dc0fcbb0 0x7f35dc0f8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:49:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.261+0000 7f35e11de700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35dc0fd510 0x7f35dc0f9390 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 rx=0x7f35d400d940 tx=0x7f35d4004e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.261+0000 7f35d9ffb700 10 client.?.objecter ms_handle_connect 0x7f35dc0fd510 2024-02-23T00:49:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.261+0000 7f35d9ffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:49:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.261+0000 7f35d9ffb700 1 -- 172.21.15.47:0/1343224562 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d400b4f0 con 0x7f35dc0fd510 2024-02-23T00:49:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.261+0000 7f35d9ffb700 1 -- 172.21.15.47:0/1343224562 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f35d400f040 con 0x7f35dc0fd510 2024-02-23T00:49:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.261+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35c8005040 con 0x7f35dc0fd510 2024-02-23T00:49:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.261+0000 7f35d9ffb700 1 -- 172.21.15.47:0/1343224562 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d40138b0 con 0x7f35dc0fd510 2024-02-23T00:49:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.262+0000 7f35e2c41700 10 client.31098.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:49:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.262+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35dc116ec0 con 0x7f35dc0fd510 2024-02-23T00:49:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.262+0000 7f35e2c41700 1 librados: init done 2024-02-23T00:49:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.262+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f35dc05fa60 con 0x7f35dc0fd510 2024-02-23T00:49:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.263+0000 7f35d9ffb700 1 -- 172.21.15.47:0/1343224562 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f35d4013ce0 con 0x7f35dc0fd510 2024-02-23T00:49:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.263+0000 7f35d9ffb700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f35cc062cc0 0x7f35cc065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.263+0000 7f35d9ffb700 1 -- 172.21.15.47:0/1343224562 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(454..454 src has 48..454) v4 ==== 9220+0+0 (secure 0 0 0) 0x7f35d407d0e0 con 0x7f35dc0fd510 2024-02-23T00:49:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.263+0000 7f35d9ffb700 10 client.31098.objecter ms_dispatch 0x7f35dc001550 osd_map(454..454 src has 48..454) v4 2024-02-23T00:49:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.263+0000 7f35d9ffb700 3 client.31098.objecter handle_osd_map got epochs [454,454] > 0 2024-02-23T00:49:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.263+0000 7f35d9ffb700 3 client.31098.objecter handle_osd_map decoding full epoch 454 2024-02-23T00:49:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.263+0000 7f35e09dd700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f35cc062cc0 0x7f35cc065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.264+0000 7f35d9ffb700 20 client.31098.objecter dump_active .. 0 homeless 2024-02-23T00:49:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.264+0000 7f35d9ffb700 1 -- 172.21.15.47:0/1343224562 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v454) v1 ==== 54+0+16 (secure 0 0 0) 0x7f35d4084070 con 0x7f35dc0fd510 2024-02-23T00:49:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.264+0000 7f35e2c41700 10 client.31098.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.264+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_assign -1 1 2024-02-23T00:49:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.264+0000 7f35e2c41700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f35dc0ffed0 0x7f35dc1002d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.264+0000 7f35e2c41700 20 client.31098.objecter _get_session s=0x7f35dc05ee80 osd=0 3 2024-02-23T00:49:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.264+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc05ee80 osd=0 3 2024-02-23T00:49:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.264+0000 7f35e2c41700 20 client.31098.objecter _get_session s=0x7f35dc05ee80 osd=0 3 2024-02-23T00:49:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.264+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_remove -1 1 2024-02-23T00:49:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.264+0000 7f35e2c41700 20 client.31098.objecter get_session s=0x7f35dc05ee80 osd=0 3 2024-02-23T00:49:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.264+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_assign 0 1 2024-02-23T00:49:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.264+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc05ee80 osd=0 4 2024-02-23T00:49:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.264+0000 7f35d9ffb700 10 client.31098.objecter ms_handle_connect 0x7f35cc062cc0 2024-02-23T00:49:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.264+0000 7f35e2c41700 10 client.31098.objecter _send_command 1 2024-02-23T00:49:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.264+0000 7f35e09dd700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f35cc062cc0 0x7f35cc065180 secure :-1 s=READY pgs=3079 cs=0 l=1 rev1=1 rx=0x7f35d0009350 tx=0x7f35d000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.264+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35dc105ea0 con 0x7f35dc0ffed0 2024-02-23T00:49:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.264+0000 7f35dbfff700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f35dc0ffed0 0x7f35dc1002d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.264+0000 7f35dbfff700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f35dc0ffed0 0x7f35dc1002d0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.264+0000 7f35d9ffb700 10 client.31098.objecter ms_handle_connect 0x7f35dc0ffed0 2024-02-23T00:49:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.265+0000 7f35d9ffb700 1 -- 172.21.15.47:0/1343224562 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f35dc105ea0 con 0x7f35dc0ffed0 2024-02-23T00:49:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.265+0000 7f35d9ffb700 10 client.31098.objecter ms_dispatch 0x7f35dc001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:49:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.265+0000 7f35d9ffb700 10 client.31098.objecter _finish_command 1 = system:0 2024-02-23T00:49:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.265+0000 7f35d9ffb700 20 client.31098.objecter put_session s=0x7f35dc05ee80 osd=0 4 2024-02-23T00:49:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.265+0000 7f35d9ffb700 15 client.31098.objecter _session_command_op_remove 0 1 2024-02-23T00:49:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.280+0000 7f35e2c41700 10 client.31098.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:49:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.280+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_assign -1 2 2024-02-23T00:49:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.280+0000 7f35e2c41700 20 client.31098.objecter _get_session s=0x7f35dc05ee80 osd=0 3 2024-02-23T00:49:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.280+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc05ee80 osd=0 3 2024-02-23T00:49:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.280+0000 7f35e2c41700 20 client.31098.objecter _get_session s=0x7f35dc05ee80 osd=0 3 2024-02-23T00:49:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.280+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_remove -1 2 2024-02-23T00:49:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.280+0000 7f35e2c41700 20 client.31098.objecter get_session s=0x7f35dc05ee80 osd=0 3 2024-02-23T00:49:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.280+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_assign 0 2 2024-02-23T00:49:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.280+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc05ee80 osd=0 4 2024-02-23T00:49:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.280+0000 7f35e2c41700 10 client.31098.objecter _send_command 2 2024-02-23T00:49:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.280+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f35dc0273c0 con 0x7f35dc0ffed0 2024-02-23T00:49:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.281+0000 7f35d9ffb700 1 -- 172.21.15.47:0/1343224562 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f35dc0273c0 con 0x7f35dc0ffed0 2024-02-23T00:49:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.281+0000 7f35d9ffb700 10 client.31098.objecter ms_dispatch 0x7f35dc001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:49:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.281+0000 7f35d9ffb700 10 client.31098.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:49:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.281+0000 7f35d9ffb700 20 client.31098.objecter put_session s=0x7f35dc05ee80 osd=0 4 2024-02-23T00:49:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.281+0000 7f35d9ffb700 15 client.31098.objecter _session_command_op_remove 0 2 2024-02-23T00:49:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.282+0000 7f35e2c41700 10 client.31098.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.282+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_assign -1 3 2024-02-23T00:49:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.282+0000 7f35e2c41700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f35dc0598f0 0x7f35dc059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.282+0000 7f35e2c41700 20 client.31098.objecter _get_session s=0x7f35dc028710 osd=1 3 2024-02-23T00:49:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.282+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc028710 osd=1 3 2024-02-23T00:49:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.282+0000 7f35e2c41700 20 client.31098.objecter _get_session s=0x7f35dc028710 osd=1 3 2024-02-23T00:49:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.282+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_remove -1 3 2024-02-23T00:49:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.282+0000 7f35e2c41700 20 client.31098.objecter get_session s=0x7f35dc028710 osd=1 3 2024-02-23T00:49:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.282+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_assign 1 3 2024-02-23T00:49:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.282+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc028710 osd=1 4 2024-02-23T00:49:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.282+0000 7f35e2c41700 10 client.31098.objecter _send_command 3 2024-02-23T00:49:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.282+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35dc11d890 con 0x7f35dc0598f0 2024-02-23T00:49:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.282+0000 7f35e09dd700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f35dc0598f0 0x7f35dc059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:14.317 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:49:14.317 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:49:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.282+0000 7f35d9ffb700 10 client.31098.objecter ms_handle_connect 0x7f35dc0598f0 2024-02-23T00:49:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.282+0000 7f35e09dd700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f35dc0598f0 0x7f35dc059cd0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.283+0000 7f35d9ffb700 1 -- 172.21.15.47:0/1343224562 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f35dc11d890 con 0x7f35dc0598f0 2024-02-23T00:49:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.283+0000 7f35d9ffb700 10 client.31098.objecter ms_dispatch 0x7f35dc001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:49:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.283+0000 7f35d9ffb700 10 client.31098.objecter _finish_command 3 = system:0 2024-02-23T00:49:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.283+0000 7f35d9ffb700 20 client.31098.objecter put_session s=0x7f35dc028710 osd=1 4 2024-02-23T00:49:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.283+0000 7f35d9ffb700 15 client.31098.objecter _session_command_op_remove 1 3 2024-02-23T00:49:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.298+0000 7f35e2c41700 10 client.31098.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:49:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.298+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_assign -1 4 2024-02-23T00:49:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.298+0000 7f35e2c41700 20 client.31098.objecter _get_session s=0x7f35dc028710 osd=1 3 2024-02-23T00:49:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.298+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc028710 osd=1 3 2024-02-23T00:49:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.298+0000 7f35e2c41700 20 client.31098.objecter _get_session s=0x7f35dc028710 osd=1 3 2024-02-23T00:49:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.298+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_remove -1 4 2024-02-23T00:49:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.298+0000 7f35e2c41700 20 client.31098.objecter get_session s=0x7f35dc028710 osd=1 3 2024-02-23T00:49:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.298+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_assign 1 4 2024-02-23T00:49:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.298+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc028710 osd=1 4 2024-02-23T00:49:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.298+0000 7f35e2c41700 10 client.31098.objecter _send_command 4 2024-02-23T00:49:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.298+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f35dc059360 con 0x7f35dc0598f0 2024-02-23T00:49:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.299+0000 7f35d9ffb700 1 -- 172.21.15.47:0/1343224562 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f35dc059360 con 0x7f35dc0598f0 2024-02-23T00:49:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.299+0000 7f35d9ffb700 10 client.31098.objecter ms_dispatch 0x7f35dc001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:49:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.299+0000 7f35d9ffb700 10 client.31098.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:49:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.299+0000 7f35d9ffb700 20 client.31098.objecter put_session s=0x7f35dc028710 osd=1 4 2024-02-23T00:49:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.299+0000 7f35d9ffb700 15 client.31098.objecter _session_command_op_remove 1 4 2024-02-23T00:49:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.299+0000 7f35e2c41700 10 client.31098.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.299+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_assign -1 5 2024-02-23T00:49:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.299+0000 7f35e2c41700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f35dc028210 0x7f35dc11c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.299+0000 7f35e2c41700 20 client.31098.objecter _get_session s=0x7f35dc059710 osd=2 3 2024-02-23T00:49:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.299+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc059710 osd=2 3 2024-02-23T00:49:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.299+0000 7f35e2c41700 20 client.31098.objecter _get_session s=0x7f35dc059710 osd=2 3 2024-02-23T00:49:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.299+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_remove -1 5 2024-02-23T00:49:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.299+0000 7f35e2c41700 20 client.31098.objecter get_session s=0x7f35dc059710 osd=2 3 2024-02-23T00:49:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.299+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_assign 2 5 2024-02-23T00:49:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.299+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc059710 osd=2 4 2024-02-23T00:49:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.299+0000 7f35e2c41700 10 client.31098.objecter _send_command 5 2024-02-23T00:49:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.299+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35dc11c7a0 con 0x7f35dc028210 2024-02-23T00:49:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.299+0000 7f35dbfff700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f35dc028210 0x7f35dc11c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.300+0000 7f35d9ffb700 10 client.31098.objecter ms_handle_connect 0x7f35dc028210 2024-02-23T00:49:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.300+0000 7f35dbfff700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f35dc028210 0x7f35dc11c2b0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.300+0000 7f35d9ffb700 1 -- 172.21.15.47:0/1343224562 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f35dc11c7a0 con 0x7f35dc028210 2024-02-23T00:49:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.300+0000 7f35d9ffb700 10 client.31098.objecter ms_dispatch 0x7f35dc001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:49:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.300+0000 7f35d9ffb700 10 client.31098.objecter _finish_command 5 = system:0 2024-02-23T00:49:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.300+0000 7f35d9ffb700 20 client.31098.objecter put_session s=0x7f35dc059710 osd=2 4 2024-02-23T00:49:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.300+0000 7f35d9ffb700 15 client.31098.objecter _session_command_op_remove 2 5 2024-02-23T00:49:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.315+0000 7f35e2c41700 10 client.31098.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:49:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.315+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_assign -1 6 2024-02-23T00:49:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.315+0000 7f35e2c41700 20 client.31098.objecter _get_session s=0x7f35dc059710 osd=2 3 2024-02-23T00:49:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.315+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc059710 osd=2 3 2024-02-23T00:49:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.315+0000 7f35e2c41700 20 client.31098.objecter _get_session s=0x7f35dc059710 osd=2 3 2024-02-23T00:49:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.315+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_remove -1 6 2024-02-23T00:49:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.315+0000 7f35e2c41700 20 client.31098.objecter get_session s=0x7f35dc059710 osd=2 3 2024-02-23T00:49:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.315+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_assign 2 6 2024-02-23T00:49:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.315+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc059710 osd=2 4 2024-02-23T00:49:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.315+0000 7f35e2c41700 10 client.31098.objecter _send_command 6 2024-02-23T00:49:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.315+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f35dc027770 con 0x7f35dc028210 2024-02-23T00:49:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.317+0000 7f35d9ffb700 1 -- 172.21.15.47:0/1343224562 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f35dc027770 con 0x7f35dc028210 2024-02-23T00:49:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.317+0000 7f35d9ffb700 10 client.31098.objecter ms_dispatch 0x7f35dc001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:49:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.317+0000 7f35d9ffb700 10 client.31098.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:49:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.317+0000 7f35d9ffb700 20 client.31098.objecter put_session s=0x7f35dc059710 osd=2 4 2024-02-23T00:49:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.317+0000 7f35d9ffb700 15 client.31098.objecter _session_command_op_remove 2 6 2024-02-23T00:49:14.322 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:49:14.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.317+0000 7f35e2c41700 10 client.31098.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:14.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.317+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_assign -1 7 2024-02-23T00:49:14.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.317+0000 7f35e2c41700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f35dc11fa10 0x7f35dc121e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:14.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.317+0000 7f35e2c41700 20 client.31098.objecter _get_session s=0x7f35dc027b80 osd=3 3 2024-02-23T00:49:14.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.317+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc027b80 osd=3 3 2024-02-23T00:49:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.317+0000 7f35e2c41700 20 client.31098.objecter _get_session s=0x7f35dc027b80 osd=3 3 2024-02-23T00:49:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.317+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_remove -1 7 2024-02-23T00:49:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.317+0000 7f35e2c41700 20 client.31098.objecter get_session s=0x7f35dc027b80 osd=3 3 2024-02-23T00:49:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.317+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_assign 3 7 2024-02-23T00:49:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.317+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc027b80 osd=3 4 2024-02-23T00:49:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.317+0000 7f35e2c41700 10 client.31098.objecter _send_command 7 2024-02-23T00:49:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.317+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35dc124a50 con 0x7f35dc11fa10 2024-02-23T00:49:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.317+0000 7f35e11de700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f35dc11fa10 0x7f35dc121e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.318+0000 7f35d9ffb700 10 client.31098.objecter ms_handle_connect 0x7f35dc11fa10 2024-02-23T00:49:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.318+0000 7f35e11de700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f35dc11fa10 0x7f35dc121e00 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.318+0000 7f35d9ffb700 1 -- 172.21.15.47:0/1343224562 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f35dc124a50 con 0x7f35dc11fa10 2024-02-23T00:49:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.318+0000 7f35d9ffb700 10 client.31098.objecter ms_dispatch 0x7f35dc001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:49:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.318+0000 7f35d9ffb700 10 client.31098.objecter _finish_command 7 = system:0 2024-02-23T00:49:14.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.318+0000 7f35d9ffb700 20 client.31098.objecter put_session s=0x7f35dc027b80 osd=3 4 2024-02-23T00:49:14.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.318+0000 7f35d9ffb700 15 client.31098.objecter _session_command_op_remove 3 7 2024-02-23T00:49:14.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.333+0000 7f35e2c41700 10 client.31098.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:49:14.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.333+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_assign -1 8 2024-02-23T00:49:14.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.333+0000 7f35e2c41700 20 client.31098.objecter _get_session s=0x7f35dc027b80 osd=3 3 2024-02-23T00:49:14.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.333+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc027b80 osd=3 3 2024-02-23T00:49:14.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.333+0000 7f35e2c41700 20 client.31098.objecter _get_session s=0x7f35dc027b80 osd=3 3 2024-02-23T00:49:14.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.333+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_remove -1 8 2024-02-23T00:49:14.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.333+0000 7f35e2c41700 20 client.31098.objecter get_session s=0x7f35dc027b80 osd=3 3 2024-02-23T00:49:14.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.333+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_assign 3 8 2024-02-23T00:49:14.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.333+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc027b80 osd=3 4 2024-02-23T00:49:14.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.333+0000 7f35e2c41700 10 client.31098.objecter _send_command 8 2024-02-23T00:49:14.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.333+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f35dc11cb80 con 0x7f35dc11fa10 2024-02-23T00:49:14.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.334+0000 7f35d9ffb700 1 -- 172.21.15.47:0/1343224562 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f35dc11cb80 con 0x7f35dc11fa10 2024-02-23T00:49:14.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.334+0000 7f35d9ffb700 10 client.31098.objecter ms_dispatch 0x7f35dc001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:49:14.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.334+0000 7f35d9ffb700 10 client.31098.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:49:14.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.334+0000 7f35d9ffb700 20 client.31098.objecter put_session s=0x7f35dc027b80 osd=3 4 2024-02-23T00:49:14.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.334+0000 7f35d9ffb700 15 client.31098.objecter _session_command_op_remove 3 8 2024-02-23T00:49:14.337 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:49:14.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.335+0000 7f35e2c41700 10 client.31098.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:14.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.335+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_assign -1 9 2024-02-23T00:49:14.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.335+0000 7f35e2c41700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f35dc11ccf0 0x7f35dc123470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:14.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.335+0000 7f35e2c41700 20 client.31098.objecter _get_session s=0x7f35dc11cb80 osd=4 3 2024-02-23T00:49:14.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.335+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc11cb80 osd=4 3 2024-02-23T00:49:14.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.335+0000 7f35e2c41700 20 client.31098.objecter _get_session s=0x7f35dc11cb80 osd=4 3 2024-02-23T00:49:14.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.335+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_remove -1 9 2024-02-23T00:49:14.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.335+0000 7f35e2c41700 20 client.31098.objecter get_session s=0x7f35dc11cb80 osd=4 3 2024-02-23T00:49:14.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.335+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_assign 4 9 2024-02-23T00:49:14.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.335+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc11cb80 osd=4 4 2024-02-23T00:49:14.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.335+0000 7f35e2c41700 10 client.31098.objecter _send_command 9 2024-02-23T00:49:14.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.335+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35dc11bfb0 con 0x7f35dc11ccf0 2024-02-23T00:49:14.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.335+0000 7f35e09dd700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f35dc11ccf0 0x7f35dc123470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:14.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.335+0000 7f35d9ffb700 10 client.31098.objecter ms_handle_connect 0x7f35dc11ccf0 2024-02-23T00:49:14.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.335+0000 7f35e09dd700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f35dc11ccf0 0x7f35dc123470 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:14.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.336+0000 7f35d9ffb700 1 -- 172.21.15.47:0/1343224562 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f35dc11bfb0 con 0x7f35dc11ccf0 2024-02-23T00:49:14.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.336+0000 7f35d9ffb700 10 client.31098.objecter ms_dispatch 0x7f35dc001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:49:14.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.336+0000 7f35d9ffb700 10 client.31098.objecter _finish_command 9 = system:0 2024-02-23T00:49:14.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.336+0000 7f35d9ffb700 20 client.31098.objecter put_session s=0x7f35dc11cb80 osd=4 4 2024-02-23T00:49:14.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.336+0000 7f35d9ffb700 15 client.31098.objecter _session_command_op_remove 4 9 2024-02-23T00:49:14.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.351+0000 7f35e2c41700 10 client.31098.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:49:14.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.351+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_assign -1 10 2024-02-23T00:49:14.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.351+0000 7f35e2c41700 20 client.31098.objecter _get_session s=0x7f35dc11cb80 osd=4 3 2024-02-23T00:49:14.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.351+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc11cb80 osd=4 3 2024-02-23T00:49:14.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.351+0000 7f35e2c41700 20 client.31098.objecter _get_session s=0x7f35dc11cb80 osd=4 3 2024-02-23T00:49:14.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.351+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_remove -1 10 2024-02-23T00:49:14.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.351+0000 7f35e2c41700 20 client.31098.objecter get_session s=0x7f35dc11cb80 osd=4 3 2024-02-23T00:49:14.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.351+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_assign 4 10 2024-02-23T00:49:14.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.351+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc11cb80 osd=4 4 2024-02-23T00:49:14.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.351+0000 7f35e2c41700 10 client.31098.objecter _send_command 10 2024-02-23T00:49:14.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.351+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f35dc11bfb0 con 0x7f35dc11ccf0 2024-02-23T00:49:14.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.353+0000 7f35d9ffb700 1 -- 172.21.15.47:0/1343224562 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f35dc11bfb0 con 0x7f35dc11ccf0 2024-02-23T00:49:14.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.353+0000 7f35d9ffb700 10 client.31098.objecter ms_dispatch 0x7f35dc001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:49:14.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.353+0000 7f35d9ffb700 10 client.31098.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:49:14.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.353+0000 7f35d9ffb700 20 client.31098.objecter put_session s=0x7f35dc11cb80 osd=4 4 2024-02-23T00:49:14.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.353+0000 7f35d9ffb700 15 client.31098.objecter _session_command_op_remove 4 10 2024-02-23T00:49:14.356 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:49:14.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.353+0000 7f35e2c41700 10 client.31098.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:14.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.353+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_assign -1 11 2024-02-23T00:49:14.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.353+0000 7f35e2c41700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f35dc123e70 0x7f35dc1242b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:14.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.353+0000 7f35e2c41700 20 client.31098.objecter _get_session s=0x7f35dc11bfb0 osd=5 3 2024-02-23T00:49:14.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.353+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc11bfb0 osd=5 3 2024-02-23T00:49:14.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.353+0000 7f35e2c41700 20 client.31098.objecter _get_session s=0x7f35dc11bfb0 osd=5 3 2024-02-23T00:49:14.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.353+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_remove -1 11 2024-02-23T00:49:14.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.353+0000 7f35e2c41700 20 client.31098.objecter get_session s=0x7f35dc11bfb0 osd=5 3 2024-02-23T00:49:14.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.353+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_assign 5 11 2024-02-23T00:49:14.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.353+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc11bfb0 osd=5 4 2024-02-23T00:49:14.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.353+0000 7f35e2c41700 10 client.31098.objecter _send_command 11 2024-02-23T00:49:14.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.353+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35dc1248d0 con 0x7f35dc123e70 2024-02-23T00:49:14.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.353+0000 7f35dbfff700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f35dc123e70 0x7f35dc1242b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:14.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.354+0000 7f35d9ffb700 10 client.31098.objecter ms_handle_connect 0x7f35dc123e70 2024-02-23T00:49:14.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.354+0000 7f35dbfff700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f35dc123e70 0x7f35dc1242b0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:14.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.354+0000 7f35d9ffb700 1 -- 172.21.15.47:0/1343224562 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f35c8027040 con 0x7f35dc123e70 2024-02-23T00:49:14.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.354+0000 7f35d9ffb700 10 client.31098.objecter ms_dispatch 0x7f35dc001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:49:14.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.354+0000 7f35d9ffb700 10 client.31098.objecter _finish_command 11 = system:0 2024-02-23T00:49:14.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.354+0000 7f35d9ffb700 20 client.31098.objecter put_session s=0x7f35dc11bfb0 osd=5 4 2024-02-23T00:49:14.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.354+0000 7f35d9ffb700 15 client.31098.objecter _session_command_op_remove 5 11 2024-02-23T00:49:14.368 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:49:14.370 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:14.371 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:49:14.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.369+0000 7f35e2c41700 10 client.31098.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:49:14.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.369+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_assign -1 12 2024-02-23T00:49:14.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.369+0000 7f35e2c41700 20 client.31098.objecter _get_session s=0x7f35dc11bfb0 osd=5 3 2024-02-23T00:49:14.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.369+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc11bfb0 osd=5 3 2024-02-23T00:49:14.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.369+0000 7f35e2c41700 20 client.31098.objecter _get_session s=0x7f35dc11bfb0 osd=5 3 2024-02-23T00:49:14.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.369+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_remove -1 12 2024-02-23T00:49:14.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.369+0000 7f35e2c41700 20 client.31098.objecter get_session s=0x7f35dc11bfb0 osd=5 3 2024-02-23T00:49:14.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.369+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_assign 5 12 2024-02-23T00:49:14.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.369+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc11bfb0 osd=5 4 2024-02-23T00:49:14.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.369+0000 7f35e2c41700 10 client.31098.objecter _send_command 12 2024-02-23T00:49:14.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.369+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f35dc128be0 con 0x7f35dc123e70 2024-02-23T00:49:14.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.370+0000 7f35d9ffb700 1 -- 172.21.15.47:0/1343224562 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f35dc128be0 con 0x7f35dc123e70 2024-02-23T00:49:14.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.370+0000 7f35d9ffb700 10 client.31098.objecter ms_dispatch 0x7f35dc001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:49:14.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.370+0000 7f35d9ffb700 10 client.31098.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:49:14.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.370+0000 7f35d9ffb700 20 client.31098.objecter put_session s=0x7f35dc11bfb0 osd=5 4 2024-02-23T00:49:14.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.370+0000 7f35d9ffb700 15 client.31098.objecter _session_command_op_remove 5 12 2024-02-23T00:49:14.374 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:49:14.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.371+0000 7f35e2c41700 10 client.31098.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:14.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.371+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_assign -1 13 2024-02-23T00:49:14.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.371+0000 7f35e2c41700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f35dc12aa30 0x7f35dc128be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:14.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.371+0000 7f35e2c41700 20 client.31098.objecter _get_session s=0x7f35dc12a360 osd=6 3 2024-02-23T00:49:14.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.371+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc12a360 osd=6 3 2024-02-23T00:49:14.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.371+0000 7f35e2c41700 20 client.31098.objecter _get_session s=0x7f35dc12a360 osd=6 3 2024-02-23T00:49:14.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.371+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_remove -1 13 2024-02-23T00:49:14.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.371+0000 7f35e2c41700 20 client.31098.objecter get_session s=0x7f35dc12a360 osd=6 3 2024-02-23T00:49:14.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.371+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_assign 6 13 2024-02-23T00:49:14.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.371+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc12a360 osd=6 4 2024-02-23T00:49:14.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.371+0000 7f35e2c41700 10 client.31098.objecter _send_command 13 2024-02-23T00:49:14.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.371+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35dc12afd0 con 0x7f35dc12aa30 2024-02-23T00:49:14.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.371+0000 7f35e11de700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f35dc12aa30 0x7f35dc128be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:14.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.371+0000 7f35e11de700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f35dc12aa30 0x7f35dc128be0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:14.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.371+0000 7f35d9ffb700 10 client.31098.objecter ms_handle_connect 0x7f35dc12aa30 2024-02-23T00:49:14.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.372+0000 7f35d9ffb700 1 -- 172.21.15.47:0/1343224562 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f35dc12afd0 con 0x7f35dc12aa30 2024-02-23T00:49:14.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.372+0000 7f35d9ffb700 10 client.31098.objecter ms_dispatch 0x7f35dc001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:49:14.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.372+0000 7f35d9ffb700 10 client.31098.objecter _finish_command 13 = system:0 2024-02-23T00:49:14.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.372+0000 7f35d9ffb700 20 client.31098.objecter put_session s=0x7f35dc12a360 osd=6 4 2024-02-23T00:49:14.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.372+0000 7f35d9ffb700 15 client.31098.objecter _session_command_op_remove 6 13 2024-02-23T00:49:14.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.386+0000 7f35e2c41700 10 client.31098.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:49:14.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.386+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_assign -1 14 2024-02-23T00:49:14.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.386+0000 7f35e2c41700 20 client.31098.objecter _get_session s=0x7f35dc12a360 osd=6 3 2024-02-23T00:49:14.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.386+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc12a360 osd=6 3 2024-02-23T00:49:14.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.386+0000 7f35e2c41700 20 client.31098.objecter _get_session s=0x7f35dc12a360 osd=6 3 2024-02-23T00:49:14.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.386+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_remove -1 14 2024-02-23T00:49:14.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.386+0000 7f35e2c41700 20 client.31098.objecter get_session s=0x7f35dc12a360 osd=6 3 2024-02-23T00:49:14.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.386+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_assign 6 14 2024-02-23T00:49:14.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.386+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc12a360 osd=6 4 2024-02-23T00:49:14.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.386+0000 7f35e2c41700 10 client.31098.objecter _send_command 14 2024-02-23T00:49:14.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.386+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f35dc12afd0 con 0x7f35dc12aa30 2024-02-23T00:49:14.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.388+0000 7f35d9ffb700 1 -- 172.21.15.47:0/1343224562 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f35dc12afd0 con 0x7f35dc12aa30 2024-02-23T00:49:14.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.388+0000 7f35d9ffb700 10 client.31098.objecter ms_dispatch 0x7f35dc001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:49:14.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.388+0000 7f35d9ffb700 10 client.31098.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:49:14.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.388+0000 7f35d9ffb700 20 client.31098.objecter put_session s=0x7f35dc12a360 osd=6 4 2024-02-23T00:49:14.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.388+0000 7f35d9ffb700 15 client.31098.objecter _session_command_op_remove 6 14 2024-02-23T00:49:14.391 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:49:14.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.388+0000 7f35e2c41700 10 client.31098.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:14.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.388+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_assign -1 15 2024-02-23T00:49:14.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.388+0000 7f35e2c41700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f35dc1295e0 0x7f35dc129a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:14.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.388+0000 7f35e2c41700 20 client.31098.objecter _get_session s=0x7f35dc12afd0 osd=7 3 2024-02-23T00:49:14.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.388+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc12afd0 osd=7 3 2024-02-23T00:49:14.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.388+0000 7f35e2c41700 20 client.31098.objecter _get_session s=0x7f35dc12afd0 osd=7 3 2024-02-23T00:49:14.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.388+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_remove -1 15 2024-02-23T00:49:14.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.388+0000 7f35e2c41700 20 client.31098.objecter get_session s=0x7f35dc12afd0 osd=7 3 2024-02-23T00:49:14.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.388+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_assign 7 15 2024-02-23T00:49:14.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.388+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc12afd0 osd=7 4 2024-02-23T00:49:14.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.388+0000 7f35e2c41700 10 client.31098.objecter _send_command 15 2024-02-23T00:49:14.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.388+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35dc129fa0 con 0x7f35dc1295e0 2024-02-23T00:49:14.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.389+0000 7f35e09dd700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f35dc1295e0 0x7f35dc129a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:14.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.389+0000 7f35d9ffb700 10 client.31098.objecter ms_handle_connect 0x7f35dc1295e0 2024-02-23T00:49:14.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.389+0000 7f35e09dd700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f35dc1295e0 0x7f35dc129a60 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:14.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.390+0000 7f35d9ffb700 1 -- 172.21.15.47:0/1343224562 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f35dc129fa0 con 0x7f35dc1295e0 2024-02-23T00:49:14.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.390+0000 7f35d9ffb700 10 client.31098.objecter ms_dispatch 0x7f35dc001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:49:14.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.390+0000 7f35d9ffb700 10 client.31098.objecter _finish_command 15 = system:0 2024-02-23T00:49:14.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.390+0000 7f35d9ffb700 20 client.31098.objecter put_session s=0x7f35dc12afd0 osd=7 4 2024-02-23T00:49:14.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.390+0000 7f35d9ffb700 15 client.31098.objecter _session_command_op_remove 7 15 2024-02-23T00:49:14.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.405+0000 7f35e2c41700 10 client.31098.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:49:14.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.405+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_assign -1 16 2024-02-23T00:49:14.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.405+0000 7f35e2c41700 20 client.31098.objecter _get_session s=0x7f35dc12afd0 osd=7 3 2024-02-23T00:49:14.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.405+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc12afd0 osd=7 3 2024-02-23T00:49:14.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.405+0000 7f35e2c41700 20 client.31098.objecter _get_session s=0x7f35dc12afd0 osd=7 3 2024-02-23T00:49:14.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.405+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_remove -1 16 2024-02-23T00:49:14.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.405+0000 7f35e2c41700 20 client.31098.objecter get_session s=0x7f35dc12afd0 osd=7 3 2024-02-23T00:49:14.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.405+0000 7f35e2c41700 15 client.31098.objecter _session_command_op_assign 7 16 2024-02-23T00:49:14.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.405+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc12afd0 osd=7 4 2024-02-23T00:49:14.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.405+0000 7f35e2c41700 10 client.31098.objecter _send_command 16 2024-02-23T00:49:14.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.405+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f35dc129fa0 con 0x7f35dc1295e0 2024-02-23T00:49:14.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35d9ffb700 1 -- 172.21.15.47:0/1343224562 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f35dc129fa0 con 0x7f35dc1295e0 2024-02-23T00:49:14.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35d9ffb700 10 client.31098.objecter ms_dispatch 0x7f35dc001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:49:14.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35d9ffb700 10 client.31098.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:49:14.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35d9ffb700 20 client.31098.objecter put_session s=0x7f35dc12afd0 osd=7 4 2024-02-23T00:49:14.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35d9ffb700 15 client.31098.objecter _session_command_op_remove 7 16 2024-02-23T00:49:14.410 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:49:14.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 10 librados: watch_flush enter 2024-02-23T00:49:14.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 10 librados: watch_flush exit 2024-02-23T00:49:14.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 10 client.31098.objecter close_session for osd.0 2024-02-23T00:49:14.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f35dc0ffed0 msgr2=0x7f35dc1002d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:14.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f35dc0ffed0 0x7f35dc1002d0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:14.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc05ee80 osd=0 1 2024-02-23T00:49:14.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 10 client.31098.objecter close_session for osd.1 2024-02-23T00:49:14.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f35dc0598f0 msgr2=0x7f35dc059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:14.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f35dc0598f0 0x7f35dc059cd0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:14.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc028710 osd=1 1 2024-02-23T00:49:14.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 10 client.31098.objecter close_session for osd.2 2024-02-23T00:49:14.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f35dc028210 msgr2=0x7f35dc11c2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:14.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f35dc028210 0x7f35dc11c2b0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:14.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc059710 osd=2 1 2024-02-23T00:49:14.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 10 client.31098.objecter close_session for osd.3 2024-02-23T00:49:14.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f35dc11fa10 msgr2=0x7f35dc121e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:14.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f35dc11fa10 0x7f35dc121e00 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:14.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc027b80 osd=3 1 2024-02-23T00:49:14.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 10 client.31098.objecter close_session for osd.4 2024-02-23T00:49:14.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f35dc11ccf0 msgr2=0x7f35dc123470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:14.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f35dc11ccf0 0x7f35dc123470 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:14.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e11de700 1 -- 172.21.15.47:0/1343224562 reap_dead start 2024-02-23T00:49:14.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc11cb80 osd=4 1 2024-02-23T00:49:14.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 10 client.31098.objecter close_session for osd.5 2024-02-23T00:49:14.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f35dc123e70 msgr2=0x7f35dc1242b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:14.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f35dc123e70 0x7f35dc1242b0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:14.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e11de700 1 -- 172.21.15.47:0/1343224562 reap_dead start 2024-02-23T00:49:14.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc11bfb0 osd=5 1 2024-02-23T00:49:14.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 10 client.31098.objecter close_session for osd.6 2024-02-23T00:49:14.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f35dc12aa30 msgr2=0x7f35dc128be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:14.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f35dc12aa30 0x7f35dc128be0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:14.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc12a360 osd=6 1 2024-02-23T00:49:14.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 10 client.31098.objecter close_session for osd.7 2024-02-23T00:49:14.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f35dc1295e0 msgr2=0x7f35dc129a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:14.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f35dc1295e0 0x7f35dc129a60 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:14.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e11de700 1 -- 172.21.15.47:0/1343224562 reap_dead start 2024-02-23T00:49:14.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 20 client.31098.objecter put_session s=0x7f35dc12afd0 osd=7 1 2024-02-23T00:49:14.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 20 client.31098.objecter shutdown clearing up homeless session... 2024-02-23T00:49:14.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 10 client.31098.objecter successfully canceled tick 2024-02-23T00:49:14.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f35cc062cc0 msgr2=0x7f35cc065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:14.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f35cc062cc0 0x7f35cc065180 secure :-1 s=READY pgs=3079 cs=0 l=1 rev1=1 rx=0x7f35d0009350 tx=0x7f35d000b040).stop 2024-02-23T00:49:14.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35dc0fd510 msgr2=0x7f35dc0f9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:14.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35dc0fd510 0x7f35dc0f9390 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 rx=0x7f35d400d940 tx=0x7f35d4004e30).stop 2024-02-23T00:49:14.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 shutdown_connections 2024-02-23T00:49:14.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f35cc062cc0 0x7f35cc065180 unknown :-1 s=CLOSED pgs=3079 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:14.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 1 --2- 172.21.15.47:0/1343224562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35dc0fd510 0x7f35dc0f9390 unknown :-1 s=CLOSED pgs=2541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:14.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 >> 172.21.15.47:0/1343224562 conn(0x7f35dc0f2820 msgr2=0x7f35dc060ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:14.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 shutdown_connections 2024-02-23T00:49:14.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 1 -- 172.21.15.47:0/1343224562 wait complete. 2024-02-23T00:49:14.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:14.407+0000 7f35e2c41700 1 librados: shutdown 2024-02-23T00:49:14.415 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:49:14.415 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:49:14.415 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:49:14.415 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:49:14.415 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:49:14.415 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:49:14.415 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:49:14.415 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:49:14.435 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:13.304010+0000 mgr.y (mgr.14517) 422 : cluster [DBG] pgmap v670: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-02-23T00:49:14.435 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:13.304010+0000 mgr.y (mgr.14517) 422 : cluster [DBG] pgmap v670: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-02-23T00:49:14.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:13.304010+0000 mgr.y (mgr.14517) 422 : cluster [DBG] pgmap v670: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-02-23T00:49:14.472 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:49:14.503 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:14.844 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.5 dump_ops_in_flight 2024-02-23T00:49:14.982 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:49:14.985 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:14.985 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:49:15.087 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:49:15.106 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:49:15.114 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:15.133 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:49:15.136 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:15.136 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:49:15.237 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:49:15.268 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:15.379 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:14.264706+0000 mon.a (mon.0) 2657 : audit [DBG] from='client.? 172.21.15.47:0/1343224562' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:49:15.379 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:15.201846+0000 mon.a (mon.0) 2658 : cluster [DBG] osdmap e455: 8 total, 8 up, 7 in 2024-02-23T00:49:15.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:14.264706+0000 mon.a (mon.0) 2657 : audit [DBG] from='client.? 172.21.15.47:0/1343224562' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:49:15.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:15.201846+0000 mon.a (mon.0) 2658 : cluster [DBG] osdmap e455: 8 total, 8 up, 7 in 2024-02-23T00:49:15.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:14.264706+0000 mon.a (mon.0) 2657 : audit [DBG] from='client.? 172.21.15.47:0/1343224562' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:49:15.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:15.201846+0000 mon.a (mon.0) 2658 : cluster [DBG] osdmap e455: 8 total, 8 up, 7 in 2024-02-23T00:49:15.949 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:49:15.952 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:15.953 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:49:16.054 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:49:16.084 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:16.332 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:49:16.335 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:16.335 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:49:16.436 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:49:16.463 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:16.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.516+0000 7ffa79a9e700 1 -- 172.21.15.47:0/2136115891 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffa740fe510 msgr2=0x7ffa740f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:16.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.516+0000 7ffa79a9e700 1 --2- 172.21.15.47:0/2136115891 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffa740fe510 0x7ffa740f7d40 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 rx=0x7ffa680047b0 tx=0x7ffa68009cb0).stop 2024-02-23T00:49:16.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.517+0000 7ffa79a9e700 1 -- 172.21.15.47:0/2136115891 shutdown_connections 2024-02-23T00:49:16.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.517+0000 7ffa79a9e700 1 --2- 172.21.15.47:0/2136115891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa740f8420 0x7ffa74104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:16.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.517+0000 7ffa79a9e700 1 --2- 172.21.15.47:0/2136115891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa740fdbb0 0x7ffa740fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:16.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.517+0000 7ffa79a9e700 1 --2- 172.21.15.47:0/2136115891 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffa740fe510 0x7ffa740f7d40 unknown :-1 s=CLOSED pgs=2686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:16.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.517+0000 7ffa79a9e700 1 -- 172.21.15.47:0/2136115891 >> 172.21.15.47:0/2136115891 conn(0x7ffa740f3800 msgr2=0x7ffa740f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:16.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.517+0000 7ffa79a9e700 1 -- 172.21.15.47:0/2136115891 shutdown_connections 2024-02-23T00:49:16.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.517+0000 7ffa79a9e700 1 -- 172.21.15.47:0/2136115891 wait complete. 2024-02-23T00:49:16.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.517+0000 7ffa79a9e700 1 librados: starting msgr at 2024-02-23T00:49:16.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.517+0000 7ffa79a9e700 1 librados: starting objecter 2024-02-23T00:49:16.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.517+0000 7ffa79a9e700 1 Processor -- start 2024-02-23T00:49:16.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.518+0000 7ffa79a9e700 1 -- start start 2024-02-23T00:49:16.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.518+0000 7ffa79a9e700 1 librados: setting wanted keys 2024-02-23T00:49:16.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.518+0000 7ffa79a9e700 1 librados: calling monclient init 2024-02-23T00:49:16.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.518+0000 7ffa79a9e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffa740f8420 0x7ffa7410df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:16.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.518+0000 7ffa79a9e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa740fdbb0 0x7ffa7410e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:16.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.518+0000 7ffa79a9e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa740fe510 0x7ffa741080e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:16.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.518+0000 7ffa79a9e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa741085d0 con 0x7ffa740fe510 2024-02-23T00:49:16.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.518+0000 7ffa79a9e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa74108740 con 0x7ffa740fdbb0 2024-02-23T00:49:16.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.518+0000 7ffa79a9e700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa74108900 con 0x7ffa740f8420 2024-02-23T00:49:16.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.518+0000 7ffa737fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffa740f8420 0x7ffa7410df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:16.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.518+0000 7ffa737fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffa740f8420 0x7ffa7410df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:36016/0 (socket says 172.21.15.47:36016) 2024-02-23T00:49:16.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.518+0000 7ffa73fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa740fe510 0x7ffa741080e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:16.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.518+0000 7ffa737fe700 1 -- 172.21.15.47:0/104788251 learned_addr learned my addr 172.21.15.47:0/104788251 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:49:16.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.518+0000 7ffa72ffd700 1 --2- 172.21.15.47:0/104788251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa740fdbb0 0x7ffa7410e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:16.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.518+0000 7ffa737fe700 1 -- 172.21.15.47:0/104788251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa740fdbb0 msgr2=0x7ffa7410e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:16.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.518+0000 7ffa737fe700 1 --2- 172.21.15.47:0/104788251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa740fdbb0 0x7ffa7410e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:16.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.518+0000 7ffa737fe700 1 -- 172.21.15.47:0/104788251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa740fe510 msgr2=0x7ffa741080e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:16.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.518+0000 7ffa737fe700 1 --2- 172.21.15.47:0/104788251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa740fe510 0x7ffa741080e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:16.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.518+0000 7ffa737fe700 1 -- 172.21.15.47:0/104788251 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa64005040 con 0x7ffa740f8420 2024-02-23T00:49:16.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.518+0000 7ffa73fff700 1 --2- 172.21.15.47:0/104788251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa740fe510 0x7ffa741080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:49:16.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.518+0000 7ffa737fe700 1 --2- 172.21.15.47:0/104788251 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffa740f8420 0x7ffa7410df40 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 rx=0x7ffa64009400 tx=0x7ffa64009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:16.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.518+0000 7ffa70f74700 10 client.?.objecter ms_handle_connect 0x7ffa740f8420 2024-02-23T00:49:16.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.518+0000 7ffa70f74700 10 client.?.objecter resend_mon_ops 2024-02-23T00:49:16.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.519+0000 7ffa70f74700 1 -- 172.21.15.47:0/104788251 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa64007a40 con 0x7ffa740f8420 2024-02-23T00:49:16.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.519+0000 7ffa79a9e700 1 -- 172.21.15.47:0/104788251 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa68005040 con 0x7ffa740f8420 2024-02-23T00:49:16.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.519+0000 7ffa70f74700 1 -- 172.21.15.47:0/104788251 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7ffa6400f040 con 0x7ffa740f8420 2024-02-23T00:49:16.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.519+0000 7ffa70f74700 1 -- 172.21.15.47:0/104788251 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa64014900 con 0x7ffa740f8420 2024-02-23T00:49:16.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.519+0000 7ffa79a9e700 10 client.41429.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:49:16.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.519+0000 7ffa79a9e700 1 -- 172.21.15.47:0/104788251 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa74108f40 con 0x7ffa740f8420 2024-02-23T00:49:16.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.519+0000 7ffa79a9e700 1 librados: init done 2024-02-23T00:49:16.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.520+0000 7ffa79a9e700 1 -- 172.21.15.47:0/104788251 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa74106e90 con 0x7ffa740f8420 2024-02-23T00:49:16.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.521+0000 7ffa70f74700 1 -- 172.21.15.47:0/104788251 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7ffa64014d30 con 0x7ffa740f8420 2024-02-23T00:49:16.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.521+0000 7ffa70f74700 1 --2- 172.21.15.47:0/104788251 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ffa5c062cc0 0x7ffa5c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:16.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.521+0000 7ffa70f74700 1 -- 172.21.15.47:0/104788251 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(455..455 src has 48..455) v4 ==== 9220+0+0 (secure 0 0 0) 0x7ffa6407d6b0 con 0x7ffa740f8420 2024-02-23T00:49:16.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.521+0000 7ffa70f74700 10 client.41429.objecter ms_dispatch 0x7ffa74001550 osd_map(455..455 src has 48..455) v4 2024-02-23T00:49:16.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.521+0000 7ffa70f74700 3 client.41429.objecter handle_osd_map got epochs [455,455] > 0 2024-02-23T00:49:16.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.521+0000 7ffa70f74700 3 client.41429.objecter handle_osd_map decoding full epoch 455 2024-02-23T00:49:16.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.521+0000 7ffa70f74700 20 client.41429.objecter dump_active .. 0 homeless 2024-02-23T00:49:16.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.522+0000 7ffa72ffd700 1 --2- 172.21.15.47:0/104788251 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ffa5c062cc0 0x7ffa5c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:16.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.522+0000 7ffa70f74700 10 client.41429.objecter ms_handle_connect 0x7ffa5c062cc0 2024-02-23T00:49:16.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.522+0000 7ffa72ffd700 1 --2- 172.21.15.47:0/104788251 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ffa5c062cc0 0x7ffa5c065180 secure :-1 s=READY pgs=3080 cs=0 l=1 rev1=1 rx=0x7ffa74109880 tx=0x7ffa68009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:16.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.525+0000 7ffa70f74700 1 -- 172.21.15.47:0/104788251 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ffa6404cd40 con 0x7ffa740f8420 2024-02-23T00:49:16.556 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:15.304393+0000 mgr.y (mgr.14517) 423 : cluster [DBG] pgmap v672: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-02-23T00:49:16.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:15.304393+0000 mgr.y (mgr.14517) 423 : cluster [DBG] pgmap v672: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-02-23T00:49:16.570 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:49:16.594 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:49:16.596 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:15.304393+0000 mgr.y (mgr.14517) 423 : cluster [DBG] pgmap v672: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-02-23T00:49:16.597 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:16.597 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:49:16.698 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:49:16.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:16.725+0000 7ffa79a9e700 1 -- 172.21.15.47:0/104788251 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7ffa741095c0 con 0x7ffa740f8420 2024-02-23T00:49:16.730 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:16.853 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:49:16.855 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:16.856 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:49:16.859 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:49:16.859 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-02-23T00:49:16.859 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 0 2024-02-23T00:49:16.859 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:49:16.957 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:49:16.987 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:17.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:17.277+0000 7ffa70f74700 1 -- 172.21.15.47:0/104788251 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v456) v1 ==== 97+0+0 (secure 0 0 0) 0x7ffa64022090 con 0x7ffa740f8420 2024-02-23T00:49:17.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:17.278+0000 7ffa79a9e700 10 librados: watch_flush enter 2024-02-23T00:49:17.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:17.278+0000 7ffa79a9e700 10 librados: watch_flush exit 2024-02-23T00:49:17.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:17.278+0000 7ffa79a9e700 20 client.41429.objecter shutdown clearing up homeless session... 2024-02-23T00:49:17.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:17.278+0000 7ffa79a9e700 10 client.41429.objecter successfully canceled tick 2024-02-23T00:49:17.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:17.278+0000 7ffa79a9e700 1 -- 172.21.15.47:0/104788251 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ffa5c062cc0 msgr2=0x7ffa5c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:17.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:17.278+0000 7ffa79a9e700 1 --2- 172.21.15.47:0/104788251 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ffa5c062cc0 0x7ffa5c065180 secure :-1 s=READY pgs=3080 cs=0 l=1 rev1=1 rx=0x7ffa74109880 tx=0x7ffa68009cb0).stop 2024-02-23T00:49:17.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:17.278+0000 7ffa79a9e700 1 -- 172.21.15.47:0/104788251 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffa740f8420 msgr2=0x7ffa7410df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:17.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:17.278+0000 7ffa79a9e700 1 --2- 172.21.15.47:0/104788251 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffa740f8420 0x7ffa7410df40 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 rx=0x7ffa64009400 tx=0x7ffa64009cb0).stop 2024-02-23T00:49:17.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:17.279+0000 7ffa79a9e700 1 -- 172.21.15.47:0/104788251 shutdown_connections 2024-02-23T00:49:17.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:17.279+0000 7ffa79a9e700 1 --2- 172.21.15.47:0/104788251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa740fe510 0x7ffa741080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:17.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:17.279+0000 7ffa79a9e700 1 --2- 172.21.15.47:0/104788251 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ffa5c062cc0 0x7ffa5c065180 unknown :-1 s=CLOSED pgs=3080 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:17.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:17.279+0000 7ffa79a9e700 1 --2- 172.21.15.47:0/104788251 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffa740f8420 0x7ffa7410df40 unknown :-1 s=CLOSED pgs=2687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:17.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:17.279+0000 7ffa79a9e700 1 --2- 172.21.15.47:0/104788251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa740fdbb0 0x7ffa7410e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:17.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:17.279+0000 7ffa79a9e700 1 -- 172.21.15.47:0/104788251 >> 172.21.15.47:0/104788251 conn(0x7ffa740f3800 msgr2=0x7ffa740fb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:17.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:17.279+0000 7ffa79a9e700 1 -- 172.21.15.47:0/104788251 shutdown_connections 2024-02-23T00:49:17.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:17.279+0000 7ffa79a9e700 1 -- 172.21.15.47:0/104788251 wait complete. 2024-02-23T00:49:17.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:17.279+0000 7ffa79a9e700 1 librados: shutdown 2024-02-23T00:49:17.283 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is set 2024-02-23T00:49:17.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:16.727381+0000 mon.c (mon.2) 306 : audit [INF] from='client.? 172.21.15.47:0/104788251' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:49:17.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:16.727741+0000 mon.a (mon.0) 2659 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:49:17.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:16.727381+0000 mon.c (mon.2) 306 : audit [INF] from='client.? 172.21.15.47:0/104788251' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:49:17.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:16.727741+0000 mon.a (mon.0) 2659 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:49:17.624 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:16.727381+0000 mon.c (mon.2) 306 : audit [INF] from='client.? 172.21.15.47:0/104788251' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:49:17.624 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:16.727741+0000 mon.a (mon.0) 2659 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:49:17.641 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.5 dump_blocked_ops 2024-02-23T00:49:17.736 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:49:17.739 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:17.740 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:49:17.841 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:49:17.871 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:17.909 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:49:17.966 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:49:18.407 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:17.269917+0000 mon.a (mon.0) 2660 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:49:18.407 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:17.277532+0000 mon.a (mon.0) 2661 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:49:18.407 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:17.277617+0000 mon.a (mon.0) 2662 : cluster [DBG] osdmap e456: 8 total, 8 up, 7 in 2024-02-23T00:49:18.407 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:17.304756+0000 mgr.y (mgr.14517) 424 : cluster [DBG] pgmap v674: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.0 KiB/s rd, 2.5 KiB/s wr, 6 op/s 2024-02-23T00:49:18.420 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:49:18.424 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:18.424 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:49:18.525 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:49:18.552 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:17.269917+0000 mon.a (mon.0) 2660 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:49:18.552 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:17.277532+0000 mon.a (mon.0) 2661 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:49:18.552 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:17.277617+0000 mon.a (mon.0) 2662 : cluster [DBG] osdmap e456: 8 total, 8 up, 7 in 2024-02-23T00:49:18.552 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:17.304756+0000 mgr.y (mgr.14517) 424 : cluster [DBG] pgmap v674: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.0 KiB/s rd, 2.5 KiB/s wr, 6 op/s 2024-02-23T00:49:18.552 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:17.269917+0000 mon.a (mon.0) 2660 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:49:18.552 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:17.277532+0000 mon.a (mon.0) 2661 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:49:18.553 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:17.277617+0000 mon.a (mon.0) 2662 : cluster [DBG] osdmap e456: 8 total, 8 up, 7 in 2024-02-23T00:49:18.553 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:17.304756+0000 mgr.y (mgr.14517) 424 : cluster [DBG] pgmap v674: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.0 KiB/s rd, 2.5 KiB/s wr, 6 op/s 2024-02-23T00:49:18.554 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:18.881 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:49:18.884 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:18.885 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:49:18.986 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:49:19.016 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:19.044 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:49:19.046 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:19.046 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:49:19.147 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:49:19.178 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:19.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.388+0000 7f6bf27cf700 1 -- 172.21.15.47:0/4183879138 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6bec0f9610 msgr2=0x7f6bec101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:19.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.388+0000 7f6bf27cf700 1 --2- 172.21.15.47:0/4183879138 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6bec0f9610 0x7f6bec101d20 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 rx=0x7f6bd4004870 tx=0x7f6bd4009cb0).stop 2024-02-23T00:49:19.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.388+0000 7f6bf27cf700 1 -- 172.21.15.47:0/4183879138 shutdown_connections 2024-02-23T00:49:19.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.388+0000 7f6bf27cf700 1 --2- 172.21.15.47:0/4183879138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bec102210 0x7f6bec104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:19.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.388+0000 7f6bf27cf700 1 --2- 172.21.15.47:0/4183879138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6bec0f8cb0 0x7f6bec0f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:19.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.388+0000 7f6bf27cf700 1 --2- 172.21.15.47:0/4183879138 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6bec0f9610 0x7f6bec101d20 unknown :-1 s=CLOSED pgs=2688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:19.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.388+0000 7f6bf27cf700 1 -- 172.21.15.47:0/4183879138 >> 172.21.15.47:0/4183879138 conn(0x7f6bec0f2820 msgr2=0x7f6bec0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:19.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.388+0000 7f6bf27cf700 1 -- 172.21.15.47:0/4183879138 shutdown_connections 2024-02-23T00:49:19.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.388+0000 7f6bf27cf700 1 -- 172.21.15.47:0/4183879138 wait complete. 2024-02-23T00:49:19.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.388+0000 7f6bf27cf700 1 librados: starting msgr at 2024-02-23T00:49:19.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.388+0000 7f6bf27cf700 1 librados: starting objecter 2024-02-23T00:49:19.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.389+0000 7f6bf27cf700 1 Processor -- start 2024-02-23T00:49:19.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.389+0000 7f6bf27cf700 1 -- start start 2024-02-23T00:49:19.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.389+0000 7f6bf27cf700 1 librados: setting wanted keys 2024-02-23T00:49:19.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.389+0000 7f6bf27cf700 1 librados: calling monclient init 2024-02-23T00:49:19.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.389+0000 7f6bf27cf700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6bec0f8cb0 0x7f6bec10e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:19.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.389+0000 7f6bf27cf700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6bec0f9610 0x7f6bec10e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:19.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.389+0000 7f6bf27cf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bec102210 0x7f6bec108240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:19.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.389+0000 7f6bf27cf700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec108730 con 0x7f6bec102210 2024-02-23T00:49:19.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.389+0000 7f6bf27cf700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec1088a0 con 0x7f6bec0f9610 2024-02-23T00:49:19.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.389+0000 7f6bf27cf700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec108a60 con 0x7f6bec0f8cb0 2024-02-23T00:49:19.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.389+0000 7f6bebfff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6bec0f8cb0 0x7f6bec10e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:19.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.389+0000 7f6bf0d6c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bec102210 0x7f6bec108240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:19.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.389+0000 7f6bebfff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6bec0f8cb0 0x7f6bec10e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:36048/0 (socket says 172.21.15.47:36048) 2024-02-23T00:49:19.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.389+0000 7f6bf0d6c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bec102210 0x7f6bec108240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35340/0 (socket says 172.21.15.47:35340) 2024-02-23T00:49:19.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.389+0000 7f6bebfff700 1 -- 172.21.15.47:0/2205807906 learned_addr learned my addr 172.21.15.47:0/2205807906 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:49:19.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.389+0000 7f6beb7fe700 1 --2- 172.21.15.47:0/2205807906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6bec0f9610 0x7f6bec10e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:19.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.389+0000 7f6bf0d6c700 1 -- 172.21.15.47:0/2205807906 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6bec0f8cb0 msgr2=0x7f6bec10e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:19.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.389+0000 7f6bf0d6c700 1 --2- 172.21.15.47:0/2205807906 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6bec0f8cb0 0x7f6bec10e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:19.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.389+0000 7f6bf0d6c700 1 -- 172.21.15.47:0/2205807906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6bec0f9610 msgr2=0x7f6bec10e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:19.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.389+0000 7f6bf0d6c700 1 --2- 172.21.15.47:0/2205807906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6bec0f9610 0x7f6bec10e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:19.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.390+0000 7f6bf0d6c700 1 -- 172.21.15.47:0/2205807906 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be40097c0 con 0x7f6bec102210 2024-02-23T00:49:19.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.390+0000 7f6bebfff700 1 --2- 172.21.15.47:0/2205807906 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6bec0f8cb0 0x7f6bec10e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:49:19.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.390+0000 7f6bf0d6c700 1 --2- 172.21.15.47:0/2205807906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bec102210 0x7f6bec108240 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 rx=0x7f6bec060200 tx=0x7f6be4004c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:19.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.390+0000 7f6be97fa700 10 client.?.objecter ms_handle_connect 0x7f6bec102210 2024-02-23T00:49:19.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.390+0000 7f6be97fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:49:19.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.390+0000 7f6be97fa700 1 -- 172.21.15.47:0/2205807906 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be40079b0 con 0x7f6bec102210 2024-02-23T00:49:19.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.390+0000 7f6bf27cf700 1 -- 172.21.15.47:0/2205807906 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd4005040 con 0x7f6bec102210 2024-02-23T00:49:19.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.390+0000 7f6be97fa700 1 -- 172.21.15.47:0/2205807906 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f6be40090d0 con 0x7f6bec102210 2024-02-23T00:49:19.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.390+0000 7f6be97fa700 1 -- 172.21.15.47:0/2205807906 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be400fb90 con 0x7f6bec102210 2024-02-23T00:49:19.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.390+0000 7f6bf27cf700 10 client.31110.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:49:19.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.390+0000 7f6bf27cf700 1 -- 172.21.15.47:0/2205807906 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bec114e80 con 0x7f6bec102210 2024-02-23T00:49:19.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.390+0000 7f6bf27cf700 1 librados: init done 2024-02-23T00:49:19.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.391+0000 7f6bf27cf700 10 client.31110.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:19.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.391+0000 7f6bf27cf700 15 client.31110.objecter _session_command_op_assign -1 1 2024-02-23T00:49:19.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.391+0000 7f6bf27cf700 20 client.31110.objecter _get_session osd=-1 returning homeless 2024-02-23T00:49:19.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.391+0000 7f6bf27cf700 10 client.31110.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:49:19.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.391+0000 7f6bf27cf700 1 -- 172.21.15.47:0/2205807906 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6bec05fa60 con 0x7f6bec102210 2024-02-23T00:49:19.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.391+0000 7f6be97fa700 1 -- 172.21.15.47:0/2205807906 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f6be4007b10 con 0x7f6bec102210 2024-02-23T00:49:19.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.392+0000 7f6be97fa700 1 --2- 172.21.15.47:0/2205807906 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6bd8062cc0 0x7f6bd8065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:19.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.392+0000 7f6be97fa700 1 -- 172.21.15.47:0/2205807906 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(456..456 src has 48..456) v4 ==== 9220+0+0 (secure 0 0 0) 0x7f6be407e3c0 con 0x7f6bec102210 2024-02-23T00:49:19.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.392+0000 7f6be97fa700 10 client.31110.objecter ms_dispatch 0x7f6bec001550 osd_map(456..456 src has 48..456) v4 2024-02-23T00:49:19.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.392+0000 7f6be97fa700 3 client.31110.objecter handle_osd_map got epochs [456,456] > 0 2024-02-23T00:49:19.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.392+0000 7f6be97fa700 3 client.31110.objecter handle_osd_map decoding full epoch 456 2024-02-23T00:49:19.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.392+0000 7f6bebfff700 1 --2- 172.21.15.47:0/2205807906 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6bd8062cc0 0x7f6bd8065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:19.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.392+0000 7f6be97fa700 10 client.31110.objecter checking command 1 2024-02-23T00:49:19.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.392+0000 7f6be97fa700 1 --2- 172.21.15.47:0/2205807906 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f6bd806f870 0x7f6bd8071d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:19.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.392+0000 7f6be97fa700 20 client.31110.objecter _get_session s=0x7f6bd806f1a0 osd=3 3 2024-02-23T00:49:19.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.392+0000 7f6be97fa700 20 client.31110.objecter put_session s=0x7f6bd806f1a0 osd=3 3 2024-02-23T00:49:19.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.392+0000 7f6be97fa700 15 client.31110.objecter _session_command_op_remove -1 1 2024-02-23T00:49:19.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.392+0000 7f6be97fa700 20 client.31110.objecter _get_session s=0x7f6bd806f1a0 osd=3 3 2024-02-23T00:49:19.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.392+0000 7f6be97fa700 20 client.31110.objecter get_session s=0x7f6bd806f1a0 osd=3 3 2024-02-23T00:49:19.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.392+0000 7f6be97fa700 15 client.31110.objecter _session_command_op_assign 3 1 2024-02-23T00:49:19.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.392+0000 7f6be97fa700 20 client.31110.objecter put_session s=0x7f6bd806f1a0 osd=3 4 2024-02-23T00:49:19.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.392+0000 7f6be97fa700 10 client.31110.objecter _send_command 1 2024-02-23T00:49:19.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.392+0000 7f6be97fa700 1 -- 172.21.15.47:0/2205807906 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6bd8072390 con 0x7f6bd806f870 2024-02-23T00:49:19.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.392+0000 7f6be97fa700 20 client.31110.objecter dump_active .. 0 homeless 2024-02-23T00:49:19.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.392+0000 7f6be97fa700 1 -- 172.21.15.47:0/2205807906 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=456) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6be407e730 con 0x7f6bec102210 2024-02-23T00:49:19.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.392+0000 7f6beb7fe700 1 --2- 172.21.15.47:0/2205807906 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f6bd806f870 0x7f6bd8071d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:19.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.392+0000 7f6bebfff700 1 --2- 172.21.15.47:0/2205807906 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6bd8062cc0 0x7f6bd8065180 secure :-1 s=READY pgs=3081 cs=0 l=1 rev1=1 rx=0x7f6bdc004fd0 tx=0x7f6bdc00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:19.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.392+0000 7f6be97fa700 10 client.31110.objecter ms_handle_connect 0x7f6bd8062cc0 2024-02-23T00:49:19.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.392+0000 7f6be97fa700 10 client.31110.objecter ms_handle_connect 0x7f6bd806f870 2024-02-23T00:49:19.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.392+0000 7f6beb7fe700 1 --2- 172.21.15.47:0/2205807906 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f6bd806f870 0x7f6bd8071d10 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:19.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.393+0000 7f6be97fa700 1 -- 172.21.15.47:0/2205807906 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f6bd8072390 con 0x7f6bd806f870 2024-02-23T00:49:19.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.393+0000 7f6be97fa700 10 client.31110.objecter ms_dispatch 0x7f6bec001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:49:19.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.393+0000 7f6be97fa700 10 client.31110.objecter _finish_command 1 = system:0 2024-02-23T00:49:19.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.393+0000 7f6be97fa700 20 client.31110.objecter put_session s=0x7f6bd806f1a0 osd=3 4 2024-02-23T00:49:19.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.393+0000 7f6be97fa700 15 client.31110.objecter _session_command_op_remove 3 1 2024-02-23T00:49:19.408 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:49:19.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.412+0000 7f6bf27cf700 10 client.31110.objecter _submit_command 2 [{"prefix": "reset_purged_snaps_last"}] 2024-02-23T00:49:19.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.412+0000 7f6bf27cf700 15 client.31110.objecter _session_command_op_assign -1 2 2024-02-23T00:49:19.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.412+0000 7f6bf27cf700 20 client.31110.objecter _get_session s=0x7f6bd806f1a0 osd=3 3 2024-02-23T00:49:19.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.412+0000 7f6bf27cf700 20 client.31110.objecter put_session s=0x7f6bd806f1a0 osd=3 3 2024-02-23T00:49:19.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.412+0000 7f6bf27cf700 20 client.31110.objecter _get_session s=0x7f6bd806f1a0 osd=3 3 2024-02-23T00:49:19.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.412+0000 7f6bf27cf700 15 client.31110.objecter _session_command_op_remove -1 2 2024-02-23T00:49:19.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.412+0000 7f6bf27cf700 20 client.31110.objecter get_session s=0x7f6bd806f1a0 osd=3 3 2024-02-23T00:49:19.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.412+0000 7f6bf27cf700 15 client.31110.objecter _session_command_op_assign 3 2 2024-02-23T00:49:19.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.412+0000 7f6bf27cf700 20 client.31110.objecter put_session s=0x7f6bd806f1a0 osd=3 4 2024-02-23T00:49:19.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.412+0000 7f6bf27cf700 10 client.31110.objecter _send_command 2 2024-02-23T00:49:19.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.412+0000 7f6bf27cf700 1 -- 172.21.15.47:0/2205807906 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) v1 -- 0x7f6bec05ee80 con 0x7f6bd806f870 2024-02-23T00:49:19.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.412+0000 7f6be97fa700 1 -- 172.21.15.47:0/2205807906 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f6bec05ee80 con 0x7f6bd806f870 2024-02-23T00:49:19.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.412+0000 7f6be97fa700 10 client.31110.objecter ms_dispatch 0x7f6bec001550 command_reply(tid 2: 0 ) v1 2024-02-23T00:49:19.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.412+0000 7f6be97fa700 10 client.31110.objecter _finish_command 2 = system:0 2024-02-23T00:49:19.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.412+0000 7f6be97fa700 20 client.31110.objecter put_session s=0x7f6bd806f1a0 osd=3 4 2024-02-23T00:49:19.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.412+0000 7f6be97fa700 15 client.31110.objecter _session_command_op_remove 3 2 2024-02-23T00:49:19.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.413+0000 7f6bf27cf700 10 librados: watch_flush enter 2024-02-23T00:49:19.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.413+0000 7f6bf27cf700 10 librados: watch_flush exit 2024-02-23T00:49:19.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.413+0000 7f6bf27cf700 10 client.31110.objecter close_session for osd.3 2024-02-23T00:49:19.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.413+0000 7f6bf27cf700 1 -- 172.21.15.47:0/2205807906 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f6bd806f870 msgr2=0x7f6bd8071d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:19.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.413+0000 7f6bf27cf700 1 --2- 172.21.15.47:0/2205807906 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f6bd806f870 0x7f6bd8071d10 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:19.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.413+0000 7f6bf27cf700 20 client.31110.objecter put_session s=0x7f6bd806f1a0 osd=3 1 2024-02-23T00:49:19.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.413+0000 7f6bf27cf700 20 client.31110.objecter shutdown clearing up homeless session... 2024-02-23T00:49:19.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.413+0000 7f6bf27cf700 10 client.31110.objecter successfully canceled tick 2024-02-23T00:49:19.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.413+0000 7f6bf27cf700 1 -- 172.21.15.47:0/2205807906 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6bd8062cc0 msgr2=0x7f6bd8065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:19.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.413+0000 7f6bf27cf700 1 --2- 172.21.15.47:0/2205807906 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6bd8062cc0 0x7f6bd8065180 secure :-1 s=READY pgs=3081 cs=0 l=1 rev1=1 rx=0x7f6bdc004fd0 tx=0x7f6bdc00b040).stop 2024-02-23T00:49:19.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.413+0000 7f6bf27cf700 1 -- 172.21.15.47:0/2205807906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bec102210 msgr2=0x7f6bec108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:19.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.413+0000 7f6bf27cf700 1 --2- 172.21.15.47:0/2205807906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bec102210 0x7f6bec108240 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 rx=0x7f6bec060200 tx=0x7f6be4004c50).stop 2024-02-23T00:49:19.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.413+0000 7f6bf0d6c700 1 -- 172.21.15.47:0/2205807906 reap_dead start 2024-02-23T00:49:19.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.413+0000 7f6bf27cf700 1 -- 172.21.15.47:0/2205807906 shutdown_connections 2024-02-23T00:49:19.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.413+0000 7f6bf27cf700 1 -- 172.21.15.47:0/2205807906 >> 172.21.15.47:0/2205807906 conn(0x7f6bec0f2820 msgr2=0x7f6bec0f34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:19.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.413+0000 7f6bf27cf700 1 -- 172.21.15.47:0/2205807906 shutdown_connections 2024-02-23T00:49:19.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.413+0000 7f6bf27cf700 1 -- 172.21.15.47:0/2205807906 wait complete. 2024-02-23T00:49:19.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:19.413+0000 7f6bf27cf700 1 librados: shutdown 2024-02-23T00:49:19.427 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:49:19.430 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:19.430 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:49:19.531 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:49:19.559 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:19.721 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:49:19.722 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-02-23T00:49:19.722 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-02-23T00:49:19.722 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-02-23T00:49:19.722 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:49:19.758 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:49:19.762 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:19.762 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:49:19.863 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:49:19.891 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:19.964 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:49:19.966 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:19.966 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:49:20.067 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:49:20.093 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:20.284 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset noscrub 2024-02-23T00:49:20.506 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.5 dump_historic_ops 2024-02-23T00:49:20.542 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:49:20.545 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:20.545 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:49:20.638 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:49:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:49:20] "GET /metrics HTTP/1.1" 200 189177 "" "Prometheus/2.33.4" 2024-02-23T00:49:20.638 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:19.305127+0000 mgr.y (mgr.14517) 425 : cluster [DBG] pgmap v675: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-23T00:49:20.638 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:20.203744+0000 mon.a (mon.0) 2663 : cluster [DBG] osdmap e457: 8 total, 8 up, 7 in 2024-02-23T00:49:20.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:19.305127+0000 mgr.y (mgr.14517) 425 : cluster [DBG] pgmap v675: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-23T00:49:20.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:20.203744+0000 mon.a (mon.0) 2663 : cluster [DBG] osdmap e457: 8 total, 8 up, 7 in 2024-02-23T00:49:20.646 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:49:20.669 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:19.305127+0000 mgr.y (mgr.14517) 425 : cluster [DBG] pgmap v675: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-23T00:49:20.669 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:20.203744+0000 mon.a (mon.0) 2663 : cluster [DBG] osdmap e457: 8 total, 8 up, 7 in 2024-02-23T00:49:20.671 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:20.694 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:49:20.696 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:20.697 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:49:20.797 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:49:20.813 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:49:20.826 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:20.974 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:49:20.977 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:20.977 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:49:21.078 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:49:21.110 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:21.129 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:49:21.131 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:21.132 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:49:21.147 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph tell osd.4 reset_purged_snaps_last 2024-02-23T00:49:21.232 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:49:21.261 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:21.615 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:49:21.618 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:21.619 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:49:21.625 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:49:21.719 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:49:21.747 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:22.420 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:21.305504+0000 mgr.y (mgr.14517) 426 : cluster [DBG] pgmap v677: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1006 B/s rd, 670 B/s wr, 2 op/s 2024-02-23T00:49:22.420 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:21.305504+0000 mgr.y (mgr.14517) 426 : cluster [DBG] pgmap v677: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1006 B/s rd, 670 B/s wr, 2 op/s 2024-02-23T00:49:22.528 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:49:22.528 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:49:22.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:21.305504+0000 mgr.y (mgr.14517) 426 : cluster [DBG] pgmap v677: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1006 B/s rd, 670 B/s wr, 2 op/s 2024-02-23T00:49:22.831 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:49:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:49:22.831 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:49:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:49:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:49:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.369+0000 7f54be16a700 1 -- 172.21.15.47:0/3607842042 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f54b80f7400 msgr2=0x7f54b8103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.369+0000 7f54be16a700 1 --2- 172.21.15.47:0/3607842042 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f54b80f7400 0x7f54b8103a20 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 rx=0x7f54b0009be0 tx=0x7f54b0004d10).stop 2024-02-23T00:49:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.370+0000 7f54be16a700 1 -- 172.21.15.47:0/3607842042 shutdown_connections 2024-02-23T00:49:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.370+0000 7f54be16a700 1 --2- 172.21.15.47:0/3607842042 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f54b80f7400 0x7f54b8103a20 unknown :-1 s=CLOSED pgs=2689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.370+0000 7f54be16a700 1 --2- 172.21.15.47:0/3607842042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54b80fcbb0 0x7f54b80fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.370+0000 7f54be16a700 1 --2- 172.21.15.47:0/3607842042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54b80fd510 0x7f54b80f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.370+0000 7f54be16a700 1 -- 172.21.15.47:0/3607842042 >> 172.21.15.47:0/3607842042 conn(0x7f54b80f2820 msgr2=0x7f54b80f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.370+0000 7f54be16a700 1 -- 172.21.15.47:0/3607842042 shutdown_connections 2024-02-23T00:49:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.370+0000 7f54be16a700 1 -- 172.21.15.47:0/3607842042 wait complete. 2024-02-23T00:49:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.370+0000 7f54be16a700 1 librados: starting msgr at 2024-02-23T00:49:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.370+0000 7f54be16a700 1 librados: starting objecter 2024-02-23T00:49:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.370+0000 7f54be16a700 1 Processor -- start 2024-02-23T00:49:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.370+0000 7f54be16a700 1 -- start start 2024-02-23T00:49:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.370+0000 7f54be16a700 1 librados: setting wanted keys 2024-02-23T00:49:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.370+0000 7f54be16a700 1 librados: calling monclient init 2024-02-23T00:49:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.371+0000 7f54be16a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54b80f7400 0x7f54b810cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:22.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.371+0000 7f54be16a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54b80fcbb0 0x7f54b810d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:22.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.371+0000 7f54be16a700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f54b80fd510 0x7f54b8106f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:22.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.371+0000 7f54be16a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b8107470 con 0x7f54b80f7400 2024-02-23T00:49:22.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.371+0000 7f54be16a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b81075e0 con 0x7f54b80fcbb0 2024-02-23T00:49:22.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.371+0000 7f54be16a700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b81077a0 con 0x7f54b80fd510 2024-02-23T00:49:22.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.371+0000 7f54b77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54b80f7400 0x7f54b810cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:22.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.371+0000 7f54b7fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f54b80fd510 0x7f54b8106f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:22.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.371+0000 7f54b77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54b80f7400 0x7f54b810cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35358/0 (socket says 172.21.15.47:35358) 2024-02-23T00:49:22.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.371+0000 7f54b7fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f54b80fd510 0x7f54b8106f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:36066/0 (socket says 172.21.15.47:36066) 2024-02-23T00:49:22.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.371+0000 7f54b77fe700 1 -- 172.21.15.47:0/1184928469 learned_addr learned my addr 172.21.15.47:0/1184928469 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:49:22.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.371+0000 7f54b6ffd700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54b80fcbb0 0x7f54b810d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:22.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.371+0000 7f54b77fe700 1 -- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f54b80fd510 msgr2=0x7f54b8106f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.371+0000 7f54b77fe700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f54b80fd510 0x7f54b8106f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.371+0000 7f54b77fe700 1 -- 172.21.15.47:0/1184928469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54b80fcbb0 msgr2=0x7f54b810d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.371+0000 7f54b77fe700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54b80fcbb0 0x7f54b810d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.371+0000 7f54b77fe700 1 -- 172.21.15.47:0/1184928469 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a8005040 con 0x7f54b80f7400 2024-02-23T00:49:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.371+0000 7f54b7fff700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f54b80fd510 0x7f54b8106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:49:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.371+0000 7f54b77fe700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54b80f7400 0x7f54b810cde0 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 rx=0x7f54a80048f0 tx=0x7f54a8009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.371+0000 7f54b4ff9700 10 client.?.objecter ms_handle_connect 0x7f54b80f7400 2024-02-23T00:49:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.371+0000 7f54b4ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:49:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.371+0000 7f54b4ff9700 1 -- 172.21.15.47:0/1184928469 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a8007b00 con 0x7f54b80f7400 2024-02-23T00:49:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.372+0000 7f54b4ff9700 1 -- 172.21.15.47:0/1184928469 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f54a8012ba0 con 0x7f54b80f7400 2024-02-23T00:49:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.372+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b00097c0 con 0x7f54b80f7400 2024-02-23T00:49:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.372+0000 7f54b4ff9700 1 -- 172.21.15.47:0/1184928469 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a8010c80 con 0x7f54b80f7400 2024-02-23T00:49:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.372+0000 7f54be16a700 10 client.31116.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:49:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.372+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b8107d80 con 0x7f54b80f7400 2024-02-23T00:49:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.372+0000 7f54be16a700 1 librados: init done 2024-02-23T00:49:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.372+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f54b805fa60 con 0x7f54b80f7400 2024-02-23T00:49:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.373+0000 7f54b4ff9700 1 -- 172.21.15.47:0/1184928469 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f54a8012d10 con 0x7f54b80f7400 2024-02-23T00:49:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.373+0000 7f54b4ff9700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f54a0062cc0 0x7f54a0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.374+0000 7f54b4ff9700 1 -- 172.21.15.47:0/1184928469 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(457..457 src has 48..457) v4 ==== 9220+0+0 (secure 0 0 0) 0x7f54a807fd60 con 0x7f54b80f7400 2024-02-23T00:49:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.374+0000 7f54b4ff9700 10 client.31116.objecter ms_dispatch 0x7f54b8001550 osd_map(457..457 src has 48..457) v4 2024-02-23T00:49:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.374+0000 7f54b4ff9700 3 client.31116.objecter handle_osd_map got epochs [457,457] > 0 2024-02-23T00:49:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.374+0000 7f54b4ff9700 3 client.31116.objecter handle_osd_map decoding full epoch 457 2024-02-23T00:49:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.374+0000 7f54b6ffd700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f54a0062cc0 0x7f54a0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.374+0000 7f54b4ff9700 20 client.31116.objecter dump_active .. 0 homeless 2024-02-23T00:49:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.374+0000 7f54b4ff9700 1 -- 172.21.15.47:0/1184928469 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v457) v1 ==== 54+0+16 (secure 0 0 0) 0x7f54a80823f0 con 0x7f54b80f7400 2024-02-23T00:49:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.374+0000 7f54b4ff9700 10 client.31116.objecter ms_handle_connect 0x7f54a0062cc0 2024-02-23T00:49:22.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.374+0000 7f54b6ffd700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f54a0062cc0 0x7f54a0065180 secure :-1 s=READY pgs=3082 cs=0 l=1 rev1=1 rx=0x7f54b8108720 tx=0x7f54ac00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:22.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.374+0000 7f54be16a700 10 client.31116.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:22.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.374+0000 7f54be16a700 15 client.31116.objecter _session_command_op_assign -1 1 2024-02-23T00:49:22.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.374+0000 7f54be16a700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f54b8061380 0x7f54b8061760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:22.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.374+0000 7f54be16a700 20 client.31116.objecter _get_session s=0x7f54b8105ea0 osd=0 3 2024-02-23T00:49:22.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.374+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b8105ea0 osd=0 3 2024-02-23T00:49:22.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.374+0000 7f54be16a700 20 client.31116.objecter _get_session s=0x7f54b8105ea0 osd=0 3 2024-02-23T00:49:22.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.374+0000 7f54be16a700 15 client.31116.objecter _session_command_op_remove -1 1 2024-02-23T00:49:22.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.374+0000 7f54be16a700 20 client.31116.objecter get_session s=0x7f54b8105ea0 osd=0 3 2024-02-23T00:49:22.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.374+0000 7f54be16a700 15 client.31116.objecter _session_command_op_assign 0 1 2024-02-23T00:49:22.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.374+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b8105ea0 osd=0 4 2024-02-23T00:49:22.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.374+0000 7f54be16a700 10 client.31116.objecter _send_command 1 2024-02-23T00:49:22.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.374+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54b8061c50 con 0x7f54b8061380 2024-02-23T00:49:22.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.374+0000 7f54b7fff700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f54b8061380 0x7f54b8061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.374+0000 7f54b4ff9700 10 client.31116.objecter ms_handle_connect 0x7f54b8061380 2024-02-23T00:49:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.374+0000 7f54b7fff700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f54b8061380 0x7f54b8061760 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.375+0000 7f54b4ff9700 1 -- 172.21.15.47:0/1184928469 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f54b8061c50 con 0x7f54b8061380 2024-02-23T00:49:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.375+0000 7f54b4ff9700 10 client.31116.objecter ms_dispatch 0x7f54b8001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:49:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.375+0000 7f54b4ff9700 10 client.31116.objecter _finish_command 1 = system:0 2024-02-23T00:49:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.375+0000 7f54b4ff9700 20 client.31116.objecter put_session s=0x7f54b8105ea0 osd=0 4 2024-02-23T00:49:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.375+0000 7f54b4ff9700 15 client.31116.objecter _session_command_op_remove 0 1 2024-02-23T00:49:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.393+0000 7f54be16a700 10 client.31116.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:49:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.393+0000 7f54be16a700 15 client.31116.objecter _session_command_op_assign -1 2 2024-02-23T00:49:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.393+0000 7f54be16a700 20 client.31116.objecter _get_session s=0x7f54b8105ea0 osd=0 3 2024-02-23T00:49:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.393+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b8105ea0 osd=0 3 2024-02-23T00:49:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.393+0000 7f54be16a700 20 client.31116.objecter _get_session s=0x7f54b8105ea0 osd=0 3 2024-02-23T00:49:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.393+0000 7f54be16a700 15 client.31116.objecter _session_command_op_remove -1 2 2024-02-23T00:49:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.393+0000 7f54be16a700 20 client.31116.objecter get_session s=0x7f54b8105ea0 osd=0 3 2024-02-23T00:49:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.393+0000 7f54be16a700 15 client.31116.objecter _session_command_op_assign 0 2 2024-02-23T00:49:22.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.393+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b8105ea0 osd=0 4 2024-02-23T00:49:22.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.393+0000 7f54be16a700 10 client.31116.objecter _send_command 2 2024-02-23T00:49:22.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.393+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f54b80f4b70 con 0x7f54b8061380 2024-02-23T00:49:22.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.395+0000 7f54b4ff9700 1 -- 172.21.15.47:0/1184928469 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f54b80f4b70 con 0x7f54b8061380 2024-02-23T00:49:22.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.395+0000 7f54b4ff9700 10 client.31116.objecter ms_dispatch 0x7f54b8001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:49:22.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.395+0000 7f54b4ff9700 10 client.31116.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:49:22.892 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:49:22.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.395+0000 7f54b4ff9700 20 client.31116.objecter put_session s=0x7f54b8105ea0 osd=0 4 2024-02-23T00:49:22.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.395+0000 7f54b4ff9700 15 client.31116.objecter _session_command_op_remove 0 2 2024-02-23T00:49:22.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.395+0000 7f54be16a700 10 client.31116.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:22.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.395+0000 7f54be16a700 15 client.31116.objecter _session_command_op_assign -1 3 2024-02-23T00:49:22.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.395+0000 7f54be16a700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f54b8059e00 0x7f54b80273c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:22.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.395+0000 7f54be16a700 20 client.31116.objecter _get_session s=0x7f54b80f4f80 osd=1 3 2024-02-23T00:49:22.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.395+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b80f4f80 osd=1 3 2024-02-23T00:49:22.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.395+0000 7f54be16a700 20 client.31116.objecter _get_session s=0x7f54b80f4f80 osd=1 3 2024-02-23T00:49:22.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.395+0000 7f54be16a700 15 client.31116.objecter _session_command_op_remove -1 3 2024-02-23T00:49:22.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.395+0000 7f54be16a700 20 client.31116.objecter get_session s=0x7f54b80f4f80 osd=1 3 2024-02-23T00:49:22.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.395+0000 7f54be16a700 15 client.31116.objecter _session_command_op_assign 1 3 2024-02-23T00:49:22.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.395+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b80f4f80 osd=1 4 2024-02-23T00:49:22.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.395+0000 7f54be16a700 10 client.31116.objecter _send_command 3 2024-02-23T00:49:22.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.395+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54b80278b0 con 0x7f54b8059e00 2024-02-23T00:49:22.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.395+0000 7f54b77fe700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f54b8059e00 0x7f54b80273c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:22.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.396+0000 7f54b77fe700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f54b8059e00 0x7f54b80273c0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:22.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.396+0000 7f54b4ff9700 10 client.31116.objecter ms_handle_connect 0x7f54b8059e00 2024-02-23T00:49:22.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.396+0000 7f54b4ff9700 1 -- 172.21.15.47:0/1184928469 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f54b80278b0 con 0x7f54b8059e00 2024-02-23T00:49:22.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.396+0000 7f54b4ff9700 10 client.31116.objecter ms_dispatch 0x7f54b8001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:49:22.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.396+0000 7f54b4ff9700 10 client.31116.objecter _finish_command 3 = system:0 2024-02-23T00:49:22.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.396+0000 7f54b4ff9700 20 client.31116.objecter put_session s=0x7f54b80f4f80 osd=1 4 2024-02-23T00:49:22.894 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:49:22.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.396+0000 7f54b4ff9700 15 client.31116.objecter _session_command_op_remove 1 3 2024-02-23T00:49:22.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.415+0000 7f54be16a700 10 client.31116.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:49:22.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.415+0000 7f54be16a700 15 client.31116.objecter _session_command_op_assign -1 4 2024-02-23T00:49:22.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.415+0000 7f54be16a700 20 client.31116.objecter _get_session s=0x7f54b80f4f80 osd=1 3 2024-02-23T00:49:22.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.415+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b80f4f80 osd=1 3 2024-02-23T00:49:22.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.415+0000 7f54be16a700 20 client.31116.objecter _get_session s=0x7f54b80f4f80 osd=1 3 2024-02-23T00:49:22.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.415+0000 7f54be16a700 15 client.31116.objecter _session_command_op_remove -1 4 2024-02-23T00:49:22.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.415+0000 7f54be16a700 20 client.31116.objecter get_session s=0x7f54b80f4f80 osd=1 3 2024-02-23T00:49:22.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.415+0000 7f54be16a700 15 client.31116.objecter _session_command_op_assign 1 4 2024-02-23T00:49:22.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.415+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b80f4f80 osd=1 4 2024-02-23T00:49:22.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.415+0000 7f54be16a700 10 client.31116.objecter _send_command 4 2024-02-23T00:49:22.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.415+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f54b8059360 con 0x7f54b8059e00 2024-02-23T00:49:22.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.416+0000 7f54b4ff9700 1 -- 172.21.15.47:0/1184928469 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f54b8059360 con 0x7f54b8059e00 2024-02-23T00:49:22.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.416+0000 7f54b4ff9700 10 client.31116.objecter ms_dispatch 0x7f54b8001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:49:22.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.416+0000 7f54b4ff9700 10 client.31116.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:49:22.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.416+0000 7f54b4ff9700 20 client.31116.objecter put_session s=0x7f54b80f4f80 osd=1 4 2024-02-23T00:49:22.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.416+0000 7f54b4ff9700 15 client.31116.objecter _session_command_op_remove 1 4 2024-02-23T00:49:22.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.417+0000 7f54be16a700 10 client.31116.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:22.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.417+0000 7f54be16a700 15 client.31116.objecter _session_command_op_assign -1 5 2024-02-23T00:49:22.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.417+0000 7f54be16a700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f54b8117c30 0x7f54b811a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:22.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.417+0000 7f54be16a700 20 client.31116.objecter _get_session s=0x7f54b8059770 osd=2 3 2024-02-23T00:49:22.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.417+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b8059770 osd=2 3 2024-02-23T00:49:22.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.417+0000 7f54be16a700 20 client.31116.objecter _get_session s=0x7f54b8059770 osd=2 3 2024-02-23T00:49:22.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.417+0000 7f54be16a700 15 client.31116.objecter _session_command_op_remove -1 5 2024-02-23T00:49:22.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.417+0000 7f54be16a700 20 client.31116.objecter get_session s=0x7f54b8059770 osd=2 3 2024-02-23T00:49:22.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.417+0000 7f54be16a700 15 client.31116.objecter _session_command_op_assign 2 5 2024-02-23T00:49:22.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.417+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b8059770 osd=2 4 2024-02-23T00:49:22.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.417+0000 7f54be16a700 10 client.31116.objecter _send_command 5 2024-02-23T00:49:22.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.417+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54b8028730 con 0x7f54b8117c30 2024-02-23T00:49:22.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.417+0000 7f54b6ffd700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f54b8117c30 0x7f54b811a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:22.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.418+0000 7f54b4ff9700 10 client.31116.objecter ms_handle_connect 0x7f54b8117c30 2024-02-23T00:49:22.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.418+0000 7f54b6ffd700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f54b8117c30 0x7f54b811a020 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:22.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.418+0000 7f54b4ff9700 1 -- 172.21.15.47:0/1184928469 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f54b8028730 con 0x7f54b8117c30 2024-02-23T00:49:22.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.418+0000 7f54b4ff9700 10 client.31116.objecter ms_dispatch 0x7f54b8001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:49:22.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.418+0000 7f54b4ff9700 10 client.31116.objecter _finish_command 5 = system:0 2024-02-23T00:49:22.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.418+0000 7f54b4ff9700 20 client.31116.objecter put_session s=0x7f54b8059770 osd=2 4 2024-02-23T00:49:22.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.418+0000 7f54b4ff9700 15 client.31116.objecter _session_command_op_remove 2 5 2024-02-23T00:49:22.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.436+0000 7f54be16a700 10 client.31116.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:49:22.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.436+0000 7f54be16a700 15 client.31116.objecter _session_command_op_assign -1 6 2024-02-23T00:49:22.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.436+0000 7f54be16a700 20 client.31116.objecter _get_session s=0x7f54b8059770 osd=2 3 2024-02-23T00:49:22.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.436+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b8059770 osd=2 3 2024-02-23T00:49:22.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.436+0000 7f54be16a700 20 client.31116.objecter _get_session s=0x7f54b8059770 osd=2 3 2024-02-23T00:49:22.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.436+0000 7f54be16a700 15 client.31116.objecter _session_command_op_remove -1 6 2024-02-23T00:49:22.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.436+0000 7f54be16a700 20 client.31116.objecter get_session s=0x7f54b8059770 osd=2 3 2024-02-23T00:49:22.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.436+0000 7f54be16a700 15 client.31116.objecter _session_command_op_assign 2 6 2024-02-23T00:49:22.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.436+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b8059770 osd=2 4 2024-02-23T00:49:22.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.436+0000 7f54be16a700 10 client.31116.objecter _send_command 6 2024-02-23T00:49:22.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.436+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f54b8027c90 con 0x7f54b8117c30 2024-02-23T00:49:22.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.437+0000 7f54b4ff9700 1 -- 172.21.15.47:0/1184928469 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f54b8027c90 con 0x7f54b8117c30 2024-02-23T00:49:22.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.437+0000 7f54b4ff9700 10 client.31116.objecter ms_dispatch 0x7f54b8001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:49:22.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.437+0000 7f54b4ff9700 10 client.31116.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:49:22.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.437+0000 7f54b4ff9700 20 client.31116.objecter put_session s=0x7f54b8059770 osd=2 4 2024-02-23T00:49:22.899 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:49:22.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.437+0000 7f54b4ff9700 15 client.31116.objecter _session_command_op_remove 2 6 2024-02-23T00:49:22.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.437+0000 7f54be16a700 10 client.31116.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:22.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.437+0000 7f54be16a700 15 client.31116.objecter _session_command_op_assign -1 7 2024-02-23T00:49:22.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.438+0000 7f54be16a700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f54b80fe670 0x7f54b811d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:22.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.438+0000 7f54be16a700 20 client.31116.objecter _get_session s=0x7f54b8028070 osd=3 3 2024-02-23T00:49:22.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.438+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b8028070 osd=3 3 2024-02-23T00:49:22.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.438+0000 7f54be16a700 20 client.31116.objecter _get_session s=0x7f54b8028070 osd=3 3 2024-02-23T00:49:22.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.438+0000 7f54be16a700 15 client.31116.objecter _session_command_op_remove -1 7 2024-02-23T00:49:22.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.438+0000 7f54be16a700 20 client.31116.objecter get_session s=0x7f54b8028070 osd=3 3 2024-02-23T00:49:22.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.438+0000 7f54be16a700 15 client.31116.objecter _session_command_op_assign 3 7 2024-02-23T00:49:22.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.438+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b8028070 osd=3 4 2024-02-23T00:49:22.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.438+0000 7f54be16a700 10 client.31116.objecter _send_command 7 2024-02-23T00:49:22.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.438+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54b8120390 con 0x7f54b80fe670 2024-02-23T00:49:22.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.438+0000 7f54b7fff700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f54b80fe670 0x7f54b811d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:22.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.438+0000 7f54b4ff9700 10 client.31116.objecter ms_handle_connect 0x7f54b80fe670 2024-02-23T00:49:22.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.438+0000 7f54b7fff700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f54b80fe670 0x7f54b811d6a0 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:22.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.439+0000 7f54b4ff9700 1 -- 172.21.15.47:0/1184928469 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f54b8120390 con 0x7f54b80fe670 2024-02-23T00:49:22.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.439+0000 7f54b4ff9700 10 client.31116.objecter ms_dispatch 0x7f54b8001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:49:22.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.439+0000 7f54b4ff9700 10 client.31116.objecter _finish_command 7 = system:0 2024-02-23T00:49:22.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.439+0000 7f54b4ff9700 20 client.31116.objecter put_session s=0x7f54b8028070 osd=3 4 2024-02-23T00:49:22.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.439+0000 7f54b4ff9700 15 client.31116.objecter _session_command_op_remove 3 7 2024-02-23T00:49:22.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.458+0000 7f54be16a700 10 client.31116.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:49:22.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.458+0000 7f54be16a700 15 client.31116.objecter _session_command_op_assign -1 8 2024-02-23T00:49:22.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.458+0000 7f54be16a700 20 client.31116.objecter _get_session s=0x7f54b8028070 osd=3 3 2024-02-23T00:49:22.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.458+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b8028070 osd=3 3 2024-02-23T00:49:22.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.458+0000 7f54be16a700 20 client.31116.objecter _get_session s=0x7f54b8028070 osd=3 3 2024-02-23T00:49:22.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.458+0000 7f54be16a700 15 client.31116.objecter _session_command_op_remove -1 8 2024-02-23T00:49:22.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.458+0000 7f54be16a700 20 client.31116.objecter get_session s=0x7f54b8028070 osd=3 3 2024-02-23T00:49:22.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.458+0000 7f54be16a700 15 client.31116.objecter _session_command_op_assign 3 8 2024-02-23T00:49:22.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.458+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b8028070 osd=3 4 2024-02-23T00:49:22.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.458+0000 7f54be16a700 10 client.31116.objecter _send_command 8 2024-02-23T00:49:22.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.458+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f54b80fdbd0 con 0x7f54b80fe670 2024-02-23T00:49:22.902 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:49:22.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.460+0000 7f54b4ff9700 1 -- 172.21.15.47:0/1184928469 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f54b80fdbd0 con 0x7f54b80fe670 2024-02-23T00:49:22.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.460+0000 7f54b4ff9700 10 client.31116.objecter ms_dispatch 0x7f54b8001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:49:22.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.460+0000 7f54b4ff9700 10 client.31116.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:49:22.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.460+0000 7f54b4ff9700 20 client.31116.objecter put_session s=0x7f54b8028070 osd=3 4 2024-02-23T00:49:22.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.460+0000 7f54b4ff9700 15 client.31116.objecter _session_command_op_remove 3 8 2024-02-23T00:49:22.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.460+0000 7f54be16a700 10 client.31116.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:22.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.460+0000 7f54be16a700 15 client.31116.objecter _session_command_op_assign -1 9 2024-02-23T00:49:22.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.460+0000 7f54be16a700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f54b80fdd70 0x7f54b811ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:22.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.460+0000 7f54be16a700 20 client.31116.objecter _get_session s=0x7f54b80fdbd0 osd=4 3 2024-02-23T00:49:22.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.460+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b80fdbd0 osd=4 3 2024-02-23T00:49:22.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.460+0000 7f54be16a700 20 client.31116.objecter _get_session s=0x7f54b80fdbd0 osd=4 3 2024-02-23T00:49:22.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.460+0000 7f54be16a700 15 client.31116.objecter _session_command_op_remove -1 9 2024-02-23T00:49:22.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.460+0000 7f54be16a700 20 client.31116.objecter get_session s=0x7f54b80fdbd0 osd=4 3 2024-02-23T00:49:22.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.460+0000 7f54be16a700 15 client.31116.objecter _session_command_op_assign 4 9 2024-02-23T00:49:22.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.460+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b80fdbd0 osd=4 4 2024-02-23T00:49:22.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.460+0000 7f54be16a700 10 client.31116.objecter _send_command 9 2024-02-23T00:49:22.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.460+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54b811b430 con 0x7f54b80fdd70 2024-02-23T00:49:22.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.460+0000 7f54b77fe700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f54b80fdd70 0x7f54b811ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:22.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.461+0000 7f54b4ff9700 10 client.31116.objecter ms_handle_connect 0x7f54b80fdd70 2024-02-23T00:49:22.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.461+0000 7f54b77fe700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f54b80fdd70 0x7f54b811ed60 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:22.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.461+0000 7f54b4ff9700 1 -- 172.21.15.47:0/1184928469 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f54b811b430 con 0x7f54b80fdd70 2024-02-23T00:49:22.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.461+0000 7f54b4ff9700 10 client.31116.objecter ms_dispatch 0x7f54b8001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:49:22.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.461+0000 7f54b4ff9700 10 client.31116.objecter _finish_command 9 = system:0 2024-02-23T00:49:22.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.461+0000 7f54b4ff9700 20 client.31116.objecter put_session s=0x7f54b80fdbd0 osd=4 4 2024-02-23T00:49:22.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.461+0000 7f54b4ff9700 15 client.31116.objecter _session_command_op_remove 4 9 2024-02-23T00:49:22.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.480+0000 7f54be16a700 10 client.31116.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:49:22.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.480+0000 7f54be16a700 15 client.31116.objecter _session_command_op_assign -1 10 2024-02-23T00:49:22.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.480+0000 7f54be16a700 20 client.31116.objecter _get_session s=0x7f54b80fdbd0 osd=4 3 2024-02-23T00:49:22.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.480+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b80fdbd0 osd=4 3 2024-02-23T00:49:22.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.480+0000 7f54be16a700 20 client.31116.objecter _get_session s=0x7f54b80fdbd0 osd=4 3 2024-02-23T00:49:22.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.480+0000 7f54be16a700 15 client.31116.objecter _session_command_op_remove -1 10 2024-02-23T00:49:22.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.480+0000 7f54be16a700 20 client.31116.objecter get_session s=0x7f54b80fdbd0 osd=4 3 2024-02-23T00:49:22.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.480+0000 7f54be16a700 15 client.31116.objecter _session_command_op_assign 4 10 2024-02-23T00:49:22.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.480+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b80fdbd0 osd=4 4 2024-02-23T00:49:22.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.480+0000 7f54be16a700 10 client.31116.objecter _send_command 10 2024-02-23T00:49:22.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.480+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f54b811b430 con 0x7f54b80fdd70 2024-02-23T00:49:22.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.482+0000 7f54b4ff9700 1 -- 172.21.15.47:0/1184928469 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f54b811b430 con 0x7f54b80fdd70 2024-02-23T00:49:22.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.482+0000 7f54b4ff9700 10 client.31116.objecter ms_dispatch 0x7f54b8001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:49:22.906 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:49:22.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.482+0000 7f54b4ff9700 10 client.31116.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:49:22.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.482+0000 7f54b4ff9700 20 client.31116.objecter put_session s=0x7f54b80fdbd0 osd=4 4 2024-02-23T00:49:22.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.482+0000 7f54b4ff9700 15 client.31116.objecter _session_command_op_remove 4 10 2024-02-23T00:49:22.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.482+0000 7f54be16a700 10 client.31116.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:22.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.482+0000 7f54be16a700 15 client.31116.objecter _session_command_op_assign -1 11 2024-02-23T00:49:22.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.482+0000 7f54be16a700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f54b811f760 0x7f54b811fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:22.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.482+0000 7f54be16a700 20 client.31116.objecter _get_session s=0x7f54b811b430 osd=5 3 2024-02-23T00:49:22.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.482+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b811b430 osd=5 3 2024-02-23T00:49:22.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.482+0000 7f54be16a700 20 client.31116.objecter _get_session s=0x7f54b811b430 osd=5 3 2024-02-23T00:49:22.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.482+0000 7f54be16a700 15 client.31116.objecter _session_command_op_remove -1 11 2024-02-23T00:49:22.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.482+0000 7f54be16a700 20 client.31116.objecter get_session s=0x7f54b811b430 osd=5 3 2024-02-23T00:49:22.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.482+0000 7f54be16a700 15 client.31116.objecter _session_command_op_assign 5 11 2024-02-23T00:49:22.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.482+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b811b430 osd=5 4 2024-02-23T00:49:22.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.482+0000 7f54be16a700 10 client.31116.objecter _send_command 11 2024-02-23T00:49:22.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.482+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54b8126d10 con 0x7f54b811f760 2024-02-23T00:49:22.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.482+0000 7f54b6ffd700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f54b811f760 0x7f54b811fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:22.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.483+0000 7f54b4ff9700 10 client.31116.objecter ms_handle_connect 0x7f54b811f760 2024-02-23T00:49:22.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.483+0000 7f54b6ffd700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f54b811f760 0x7f54b811fbc0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:22.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.484+0000 7f54b4ff9700 1 -- 172.21.15.47:0/1184928469 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f54b8126d10 con 0x7f54b811f760 2024-02-23T00:49:22.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.484+0000 7f54b4ff9700 10 client.31116.objecter ms_dispatch 0x7f54b8001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:49:22.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.484+0000 7f54b4ff9700 10 client.31116.objecter _finish_command 11 = system:0 2024-02-23T00:49:22.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.484+0000 7f54b4ff9700 20 client.31116.objecter put_session s=0x7f54b811b430 osd=5 4 2024-02-23T00:49:22.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.484+0000 7f54b4ff9700 15 client.31116.objecter _session_command_op_remove 5 11 2024-02-23T00:49:22.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.501+0000 7f54be16a700 10 client.31116.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:49:22.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.501+0000 7f54be16a700 15 client.31116.objecter _session_command_op_assign -1 12 2024-02-23T00:49:22.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.501+0000 7f54be16a700 20 client.31116.objecter _get_session s=0x7f54b811b430 osd=5 3 2024-02-23T00:49:22.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.501+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b811b430 osd=5 3 2024-02-23T00:49:22.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.501+0000 7f54be16a700 20 client.31116.objecter _get_session s=0x7f54b811b430 osd=5 3 2024-02-23T00:49:22.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.501+0000 7f54be16a700 15 client.31116.objecter _session_command_op_remove -1 12 2024-02-23T00:49:22.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.501+0000 7f54be16a700 20 client.31116.objecter get_session s=0x7f54b811b430 osd=5 3 2024-02-23T00:49:22.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.501+0000 7f54be16a700 15 client.31116.objecter _session_command_op_assign 5 12 2024-02-23T00:49:22.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.501+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b811b430 osd=5 4 2024-02-23T00:49:22.909 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:49:22.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.501+0000 7f54be16a700 10 client.31116.objecter _send_command 12 2024-02-23T00:49:22.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.501+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f54b81245b0 con 0x7f54b811f760 2024-02-23T00:49:22.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.503+0000 7f54b4ff9700 1 -- 172.21.15.47:0/1184928469 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f54b81245b0 con 0x7f54b811f760 2024-02-23T00:49:22.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.503+0000 7f54b4ff9700 10 client.31116.objecter ms_dispatch 0x7f54b8001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:49:22.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.503+0000 7f54b4ff9700 10 client.31116.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:49:22.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.503+0000 7f54b4ff9700 20 client.31116.objecter put_session s=0x7f54b811b430 osd=5 4 2024-02-23T00:49:22.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.503+0000 7f54b4ff9700 15 client.31116.objecter _session_command_op_remove 5 12 2024-02-23T00:49:22.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.503+0000 7f54be16a700 10 client.31116.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:22.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.503+0000 7f54be16a700 15 client.31116.objecter _session_command_op_assign -1 13 2024-02-23T00:49:22.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.503+0000 7f54be16a700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f54b8124c30 0x7f54b8127020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:22.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.503+0000 7f54be16a700 20 client.31116.objecter _get_session s=0x7f54b81245b0 osd=6 3 2024-02-23T00:49:22.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.503+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b81245b0 osd=6 3 2024-02-23T00:49:22.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.503+0000 7f54be16a700 20 client.31116.objecter _get_session s=0x7f54b81245b0 osd=6 3 2024-02-23T00:49:22.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.503+0000 7f54be16a700 15 client.31116.objecter _session_command_op_remove -1 13 2024-02-23T00:49:22.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.503+0000 7f54be16a700 20 client.31116.objecter get_session s=0x7f54b81245b0 osd=6 3 2024-02-23T00:49:22.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.503+0000 7f54be16a700 15 client.31116.objecter _session_command_op_assign 6 13 2024-02-23T00:49:22.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.503+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b81245b0 osd=6 4 2024-02-23T00:49:22.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.503+0000 7f54be16a700 10 client.31116.objecter _send_command 13 2024-02-23T00:49:22.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.503+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54b8129e10 con 0x7f54b8124c30 2024-02-23T00:49:22.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.504+0000 7f54b7fff700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f54b8124c30 0x7f54b8127020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:22.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.504+0000 7f54b4ff9700 10 client.31116.objecter ms_handle_connect 0x7f54b8124c30 2024-02-23T00:49:22.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.504+0000 7f54b7fff700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f54b8124c30 0x7f54b8127020 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:22.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.505+0000 7f54b4ff9700 1 -- 172.21.15.47:0/1184928469 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f54b8129e10 con 0x7f54b8124c30 2024-02-23T00:49:22.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.505+0000 7f54b4ff9700 10 client.31116.objecter ms_dispatch 0x7f54b8001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:49:22.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.505+0000 7f54b4ff9700 10 client.31116.objecter _finish_command 13 = system:0 2024-02-23T00:49:22.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.505+0000 7f54b4ff9700 20 client.31116.objecter put_session s=0x7f54b81245b0 osd=6 4 2024-02-23T00:49:22.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.505+0000 7f54b4ff9700 15 client.31116.objecter _session_command_op_remove 6 13 2024-02-23T00:49:22.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.521+0000 7f54be16a700 10 client.31116.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:49:22.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.521+0000 7f54be16a700 15 client.31116.objecter _session_command_op_assign -1 14 2024-02-23T00:49:22.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.521+0000 7f54be16a700 20 client.31116.objecter _get_session s=0x7f54b81245b0 osd=6 3 2024-02-23T00:49:22.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.521+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b81245b0 osd=6 3 2024-02-23T00:49:22.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.521+0000 7f54be16a700 20 client.31116.objecter _get_session s=0x7f54b81245b0 osd=6 3 2024-02-23T00:49:22.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.521+0000 7f54be16a700 15 client.31116.objecter _session_command_op_remove -1 14 2024-02-23T00:49:22.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.521+0000 7f54be16a700 20 client.31116.objecter get_session s=0x7f54b81245b0 osd=6 3 2024-02-23T00:49:22.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.521+0000 7f54be16a700 15 client.31116.objecter _session_command_op_assign 6 14 2024-02-23T00:49:22.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.521+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b81245b0 osd=6 4 2024-02-23T00:49:22.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.521+0000 7f54be16a700 10 client.31116.objecter _send_command 14 2024-02-23T00:49:22.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.521+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f54b8129e10 con 0x7f54b8124c30 2024-02-23T00:49:22.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.524+0000 7f54b4ff9700 1 -- 172.21.15.47:0/1184928469 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f54b8129e10 con 0x7f54b8124c30 2024-02-23T00:49:22.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.524+0000 7f54b4ff9700 10 client.31116.objecter ms_dispatch 0x7f54b8001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:49:22.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.524+0000 7f54b4ff9700 10 client.31116.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:49:22.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.524+0000 7f54b4ff9700 20 client.31116.objecter put_session s=0x7f54b81245b0 osd=6 4 2024-02-23T00:49:22.912 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:49:22.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.524+0000 7f54b4ff9700 15 client.31116.objecter _session_command_op_remove 6 14 2024-02-23T00:49:22.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.524+0000 7f54be16a700 10 client.31116.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:22.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.524+0000 7f54be16a700 15 client.31116.objecter _session_command_op_assign -1 15 2024-02-23T00:49:22.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.524+0000 7f54be16a700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f54b8127b10 0x7f54b8127f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:22.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.524+0000 7f54be16a700 20 client.31116.objecter _get_session s=0x7f54b8129e10 osd=7 3 2024-02-23T00:49:22.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.524+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b8129e10 osd=7 3 2024-02-23T00:49:22.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.524+0000 7f54be16a700 20 client.31116.objecter _get_session s=0x7f54b8129e10 osd=7 3 2024-02-23T00:49:22.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.524+0000 7f54be16a700 15 client.31116.objecter _session_command_op_remove -1 15 2024-02-23T00:49:22.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.524+0000 7f54be16a700 20 client.31116.objecter get_session s=0x7f54b8129e10 osd=7 3 2024-02-23T00:49:22.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.524+0000 7f54be16a700 15 client.31116.objecter _session_command_op_assign 7 15 2024-02-23T00:49:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.524+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b8129e10 osd=7 4 2024-02-23T00:49:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.524+0000 7f54be16a700 10 client.31116.objecter _send_command 15 2024-02-23T00:49:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.524+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54b812d190 con 0x7f54b8127b10 2024-02-23T00:49:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.524+0000 7f54b77fe700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f54b8127b10 0x7f54b8127f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.525+0000 7f54b4ff9700 10 client.31116.objecter ms_handle_connect 0x7f54b8127b10 2024-02-23T00:49:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.525+0000 7f54b77fe700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f54b8127b10 0x7f54b8127f70 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.525+0000 7f54b4ff9700 1 -- 172.21.15.47:0/1184928469 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f54b812d190 con 0x7f54b8127b10 2024-02-23T00:49:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.525+0000 7f54b4ff9700 10 client.31116.objecter ms_dispatch 0x7f54b8001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:49:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.525+0000 7f54b4ff9700 10 client.31116.objecter _finish_command 15 = system:0 2024-02-23T00:49:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.525+0000 7f54b4ff9700 20 client.31116.objecter put_session s=0x7f54b8129e10 osd=7 4 2024-02-23T00:49:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.525+0000 7f54b4ff9700 15 client.31116.objecter _session_command_op_remove 7 15 2024-02-23T00:49:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.541+0000 7f54be16a700 10 client.31116.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:49:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.541+0000 7f54be16a700 15 client.31116.objecter _session_command_op_assign -1 16 2024-02-23T00:49:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.541+0000 7f54be16a700 20 client.31116.objecter _get_session s=0x7f54b8129e10 osd=7 3 2024-02-23T00:49:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.541+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b8129e10 osd=7 3 2024-02-23T00:49:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.541+0000 7f54be16a700 20 client.31116.objecter _get_session s=0x7f54b8129e10 osd=7 3 2024-02-23T00:49:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.541+0000 7f54be16a700 15 client.31116.objecter _session_command_op_remove -1 16 2024-02-23T00:49:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.541+0000 7f54be16a700 20 client.31116.objecter get_session s=0x7f54b8129e10 osd=7 3 2024-02-23T00:49:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.541+0000 7f54be16a700 15 client.31116.objecter _session_command_op_assign 7 16 2024-02-23T00:49:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.541+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b8129e10 osd=7 4 2024-02-23T00:49:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.541+0000 7f54be16a700 10 client.31116.objecter _send_command 16 2024-02-23T00:49:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.541+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f54b8128570 con 0x7f54b8127b10 2024-02-23T00:49:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54b4ff9700 1 -- 172.21.15.47:0/1184928469 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f54b8128570 con 0x7f54b8127b10 2024-02-23T00:49:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54b4ff9700 10 client.31116.objecter ms_dispatch 0x7f54b8001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:49:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54b4ff9700 10 client.31116.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:49:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54b4ff9700 20 client.31116.objecter put_session s=0x7f54b8129e10 osd=7 4 2024-02-23T00:49:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54b4ff9700 15 client.31116.objecter _session_command_op_remove 7 16 2024-02-23T00:49:22.916 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:49:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 10 librados: watch_flush enter 2024-02-23T00:49:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 10 librados: watch_flush exit 2024-02-23T00:49:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 10 client.31116.objecter close_session for osd.0 2024-02-23T00:49:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f54b8061380 msgr2=0x7f54b8061760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f54b8061380 0x7f54b8061760 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b8105ea0 osd=0 1 2024-02-23T00:49:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 10 client.31116.objecter close_session for osd.1 2024-02-23T00:49:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f54b8059e00 msgr2=0x7f54b80273c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f54b8059e00 0x7f54b80273c0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b80f4f80 osd=1 1 2024-02-23T00:49:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 10 client.31116.objecter close_session for osd.2 2024-02-23T00:49:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f54b8117c30 msgr2=0x7f54b811a020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f54b8117c30 0x7f54b811a020 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b8059770 osd=2 1 2024-02-23T00:49:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 10 client.31116.objecter close_session for osd.3 2024-02-23T00:49:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f54b80fe670 msgr2=0x7f54b811d6a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54b7fff700 1 -- 172.21.15.47:0/1184928469 reap_dead start 2024-02-23T00:49:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f54b80fe670 0x7f54b811d6a0 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:22.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b8028070 osd=3 1 2024-02-23T00:49:22.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 10 client.31116.objecter close_session for osd.4 2024-02-23T00:49:22.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f54b80fdd70 msgr2=0x7f54b811ed60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:22.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f54b80fdd70 0x7f54b811ed60 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:22.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b80fdbd0 osd=4 1 2024-02-23T00:49:22.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 10 client.31116.objecter close_session for osd.5 2024-02-23T00:49:22.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f54b811f760 msgr2=0x7f54b811fbc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:22.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f54b811f760 0x7f54b811fbc0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:22.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b811b430 osd=5 1 2024-02-23T00:49:22.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 10 client.31116.objecter close_session for osd.6 2024-02-23T00:49:22.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f54b8124c30 msgr2=0x7f54b8127020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:22.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f54b8124c30 0x7f54b8127020 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:22.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b81245b0 osd=6 1 2024-02-23T00:49:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 10 client.31116.objecter close_session for osd.7 2024-02-23T00:49:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f54b8127b10 msgr2=0x7f54b8127f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f54b8127b10 0x7f54b8127f70 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 20 client.31116.objecter put_session s=0x7f54b8129e10 osd=7 1 2024-02-23T00:49:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 20 client.31116.objecter shutdown clearing up homeless session... 2024-02-23T00:49:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 10 client.31116.objecter successfully canceled tick 2024-02-23T00:49:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54b7fff700 1 -- 172.21.15.47:0/1184928469 reap_dead start 2024-02-23T00:49:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f54a0062cc0 msgr2=0x7f54a0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f54a0062cc0 0x7f54a0065180 secure :-1 s=READY pgs=3082 cs=0 l=1 rev1=1 rx=0x7f54b8108720 tx=0x7f54ac00b040).stop 2024-02-23T00:49:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54b80f7400 msgr2=0x7f54b810cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54b80f7400 0x7f54b810cde0 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 rx=0x7f54a80048f0 tx=0x7f54a8009cb0).stop 2024-02-23T00:49:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 shutdown_connections 2024-02-23T00:49:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54b80f7400 0x7f54b810cde0 unknown :-1 s=CLOSED pgs=2543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 1 --2- 172.21.15.47:0/1184928469 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f54a0062cc0 0x7f54a0065180 unknown :-1 s=CLOSED pgs=3082 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 >> 172.21.15.47:0/1184928469 conn(0x7f54b80f2820 msgr2=0x7f54b80f4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:22.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 shutdown_connections 2024-02-23T00:49:22.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.543+0000 7f54be16a700 1 -- 172.21.15.47:0/1184928469 wait complete. 2024-02-23T00:49:22.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:22.544+0000 7f54be16a700 1 librados: shutdown 2024-02-23T00:49:22.920 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:49:22.920 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:49:22.920 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:49:22.920 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:49:22.920 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:49:22.920 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:49:22.920 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:49:22.920 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:49:23.102 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:49:23.205 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.6 dump_ops_in_flight 2024-02-23T00:49:23.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:22.374891+0000 mon.a (mon.0) 2664 : audit [DBG] from='client.? 172.21.15.47:0/1184928469' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:49:23.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:22.637494+0000 mgr.y (mgr.14517) 427 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:49:23.471 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:49:23.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:22.374891+0000 mon.a (mon.0) 2664 : audit [DBG] from='client.? 172.21.15.47:0/1184928469' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:49:23.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:22.637494+0000 mgr.y (mgr.14517) 427 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:49:23.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:22.374891+0000 mon.a (mon.0) 2664 : audit [DBG] from='client.? 172.21.15.47:0/1184928469' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:49:23.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:22.637494+0000 mgr.y (mgr.14517) 427 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:49:23.730 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:49:23.733 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:23.734 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:49:23.834 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:49:23.873 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:24.225 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:49:24.228 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:24.228 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:49:24.329 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:49:24.358 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:24.478 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:49:24.481 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:23.306461+0000 mgr.y (mgr.14517) 428 : cluster [DBG] pgmap v678: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 7.7 KiB/s rd, 511 B/s wr, 13 op/s 2024-02-23T00:49:24.482 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:23.306461+0000 mgr.y (mgr.14517) 428 : cluster [DBG] pgmap v678: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 7.7 KiB/s rd, 511 B/s wr, 13 op/s 2024-02-23T00:49:24.482 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:24.482 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:49:24.583 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:49:24.611 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:23.306461+0000 mgr.y (mgr.14517) 428 : cluster [DBG] pgmap v678: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 7.7 KiB/s rd, 511 B/s wr, 13 op/s 2024-02-23T00:49:24.613 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:24.812 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:49:24.814 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:24.815 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:49:24.916 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:49:24.946 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:25.352 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:49:25.352 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:49:25.395 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:49:25.398 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:25.398 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:49:25.499 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:49:25.528 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:25.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.063+0000 7f560ad0a700 1 -- 172.21.15.47:0/36074205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56040fdbb0 msgr2=0x7f56040fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:25.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.063+0000 7f560ad0a700 1 --2- 172.21.15.47:0/36074205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56040fdbb0 0x7f56040fdf90 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 rx=0x7f55f8004870 tx=0x7f55f8009cb0).stop 2024-02-23T00:49:25.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.063+0000 7f560ad0a700 1 -- 172.21.15.47:0/36074205 shutdown_connections 2024-02-23T00:49:25.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.063+0000 7f560ad0a700 1 --2- 172.21.15.47:0/36074205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56040f8420 0x7f5604104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:25.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.063+0000 7f560ad0a700 1 --2- 172.21.15.47:0/36074205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56040fdbb0 0x7f56040fdf90 unknown :-1 s=CLOSED pgs=2544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:25.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.063+0000 7f560ad0a700 1 --2- 172.21.15.47:0/36074205 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f56040fe510 0x7f56040f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:25.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.063+0000 7f560ad0a700 1 -- 172.21.15.47:0/36074205 >> 172.21.15.47:0/36074205 conn(0x7f56040f3800 msgr2=0x7f56040f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:25.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.063+0000 7f560ad0a700 1 -- 172.21.15.47:0/36074205 shutdown_connections 2024-02-23T00:49:25.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.063+0000 7f560ad0a700 1 -- 172.21.15.47:0/36074205 wait complete. 2024-02-23T00:49:25.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.063+0000 7f560ad0a700 1 librados: starting msgr at 2024-02-23T00:49:25.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.063+0000 7f560ad0a700 1 librados: starting objecter 2024-02-23T00:49:25.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.064+0000 7f560ad0a700 1 Processor -- start 2024-02-23T00:49:25.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.064+0000 7f560ad0a700 1 -- start start 2024-02-23T00:49:25.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.064+0000 7f560ad0a700 1 librados: setting wanted keys 2024-02-23T00:49:25.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.064+0000 7f560ad0a700 1 librados: calling monclient init 2024-02-23T00:49:25.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.064+0000 7f560ad0a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56040f8420 0x7f560410dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:25.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.064+0000 7f560ad0a700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f56040fdbb0 0x7f560410e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:25.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.064+0000 7f560ad0a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56040fe510 0x7f5604108080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:25.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.064+0000 7f560ad0a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5604108570 con 0x7f56040fe510 2024-02-23T00:49:25.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.064+0000 7f560ad0a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56041086e0 con 0x7f56040f8420 2024-02-23T00:49:25.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.064+0000 7f560ad0a700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56041088a0 con 0x7f56040fdbb0 2024-02-23T00:49:25.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.064+0000 7f5603fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f56040fdbb0 0x7f560410e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:25.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.064+0000 7f56092a7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56040fe510 0x7f5604108080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:25.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.064+0000 7f5603fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f56040fdbb0 0x7f560410e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:36078/0 (socket says 172.21.15.47:36078) 2024-02-23T00:49:25.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.064+0000 7f5603fff700 1 -- 172.21.15.47:0/1103652126 learned_addr learned my addr 172.21.15.47:0/1103652126 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:49:25.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.064+0000 7f5608aa6700 1 --2- 172.21.15.47:0/1103652126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56040f8420 0x7f560410dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:25.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.064+0000 7f5603fff700 1 -- 172.21.15.47:0/1103652126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56040f8420 msgr2=0x7f560410dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:25.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.064+0000 7f5603fff700 1 --2- 172.21.15.47:0/1103652126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56040f8420 0x7f560410dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:25.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.064+0000 7f5603fff700 1 -- 172.21.15.47:0/1103652126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56040fe510 msgr2=0x7f5604108080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:25.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.064+0000 7f5603fff700 1 --2- 172.21.15.47:0/1103652126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56040fe510 0x7f5604108080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:25.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.064+0000 7f5603fff700 1 -- 172.21.15.47:0/1103652126 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f0005040 con 0x7f56040fdbb0 2024-02-23T00:49:25.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.064+0000 7f56092a7700 1 --2- 172.21.15.47:0/1103652126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56040fe510 0x7f5604108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:49:25.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.064+0000 7f5603fff700 1 --2- 172.21.15.47:0/1103652126 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f56040fdbb0 0x7f560410e3d0 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 rx=0x7f55f00048f0 tx=0x7f55f0009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:25.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.064+0000 7f5601ffb700 10 client.?.objecter ms_handle_connect 0x7f56040fdbb0 2024-02-23T00:49:25.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.064+0000 7f5601ffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:49:25.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.064+0000 7f5601ffb700 1 -- 172.21.15.47:0/1103652126 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f0007b00 con 0x7f56040fdbb0 2024-02-23T00:49:25.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.065+0000 7f560ad0a700 1 -- 172.21.15.47:0/1103652126 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f8005040 con 0x7f56040fdbb0 2024-02-23T00:49:25.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.065+0000 7f5601ffb700 1 -- 172.21.15.47:0/1103652126 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f55f000f040 con 0x7f56040fdbb0 2024-02-23T00:49:25.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.065+0000 7f5601ffb700 1 -- 172.21.15.47:0/1103652126 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f0014820 con 0x7f56040fdbb0 2024-02-23T00:49:25.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.065+0000 7f560ad0a700 10 client.41447.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:49:25.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.065+0000 7f560ad0a700 1 -- 172.21.15.47:0/1103652126 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5604108ee0 con 0x7f56040fdbb0 2024-02-23T00:49:25.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.065+0000 7f560ad0a700 1 librados: init done 2024-02-23T00:49:25.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.066+0000 7f560ad0a700 1 -- 172.21.15.47:0/1103652126 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f560405ee80 con 0x7f56040fdbb0 2024-02-23T00:49:25.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.066+0000 7f5601ffb700 1 -- 172.21.15.47:0/1103652126 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f55f0014c50 con 0x7f56040fdbb0 2024-02-23T00:49:25.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.066+0000 7f5601ffb700 1 --2- 172.21.15.47:0/1103652126 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f55f4062cc0 0x7f55f4065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:25.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.066+0000 7f5601ffb700 1 -- 172.21.15.47:0/1103652126 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(457..457 src has 48..457) v4 ==== 9220+0+0 (secure 0 0 0) 0x7f55f007d1c0 con 0x7f56040fdbb0 2024-02-23T00:49:25.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.066+0000 7f5601ffb700 10 client.41447.objecter ms_dispatch 0x7f5604001550 osd_map(457..457 src has 48..457) v4 2024-02-23T00:49:25.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.066+0000 7f5601ffb700 3 client.41447.objecter handle_osd_map got epochs [457,457] > 0 2024-02-23T00:49:25.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.066+0000 7f5601ffb700 3 client.41447.objecter handle_osd_map decoding full epoch 457 2024-02-23T00:49:25.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.066+0000 7f5608aa6700 1 --2- 172.21.15.47:0/1103652126 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f55f4062cc0 0x7f55f4065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:25.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.066+0000 7f5601ffb700 20 client.41447.objecter dump_active .. 0 homeless 2024-02-23T00:49:25.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.067+0000 7f5601ffb700 10 client.41447.objecter ms_handle_connect 0x7f55f4062cc0 2024-02-23T00:49:25.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.067+0000 7f5608aa6700 1 --2- 172.21.15.47:0/1103652126 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f55f4062cc0 0x7f55f4065180 secure :-1 s=READY pgs=3083 cs=0 l=1 rev1=1 rx=0x7f55f8004ee0 tx=0x7f55f800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:25.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.071+0000 7f5601ffb700 1 -- 172.21.15.47:0/1103652126 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f55f0049f60 con 0x7f56040fdbb0 2024-02-23T00:49:25.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.256+0000 7f560ad0a700 1 -- 172.21.15.47:0/1103652126 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7f56041095f0 con 0x7f56040fdbb0 2024-02-23T00:49:25.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.376+0000 7f5601ffb700 1 -- 172.21.15.47:0/1103652126 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v458) v1 ==== 91+0+0 (secure 0 0 0) 0x7f55f004c7d0 con 0x7f56040fdbb0 2024-02-23T00:49:25.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.378+0000 7f560ad0a700 10 librados: watch_flush enter 2024-02-23T00:49:25.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.378+0000 7f560ad0a700 10 librados: watch_flush exit 2024-02-23T00:49:25.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.378+0000 7f560ad0a700 20 client.41447.objecter shutdown clearing up homeless session... 2024-02-23T00:49:25.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.378+0000 7f560ad0a700 10 client.41447.objecter successfully canceled tick 2024-02-23T00:49:25.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.378+0000 7f560ad0a700 1 -- 172.21.15.47:0/1103652126 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f55f4062cc0 msgr2=0x7f55f4065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:25.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.378+0000 7f560ad0a700 1 --2- 172.21.15.47:0/1103652126 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f55f4062cc0 0x7f55f4065180 secure :-1 s=READY pgs=3083 cs=0 l=1 rev1=1 rx=0x7f55f8004ee0 tx=0x7f55f800b040).stop 2024-02-23T00:49:25.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.378+0000 7f560ad0a700 1 -- 172.21.15.47:0/1103652126 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f56040fdbb0 msgr2=0x7f560410e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:25.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.378+0000 7f560ad0a700 1 --2- 172.21.15.47:0/1103652126 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f56040fdbb0 0x7f560410e3d0 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 rx=0x7f55f00048f0 tx=0x7f55f0009cb0).stop 2024-02-23T00:49:25.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.378+0000 7f560ad0a700 1 -- 172.21.15.47:0/1103652126 shutdown_connections 2024-02-23T00:49:25.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.378+0000 7f560ad0a700 1 --2- 172.21.15.47:0/1103652126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56040fe510 0x7f5604108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:25.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.378+0000 7f560ad0a700 1 --2- 172.21.15.47:0/1103652126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56040f8420 0x7f560410dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:25.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.378+0000 7f560ad0a700 1 --2- 172.21.15.47:0/1103652126 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f55f4062cc0 0x7f55f4065180 unknown :-1 s=CLOSED pgs=3083 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:25.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.378+0000 7f560ad0a700 1 --2- 172.21.15.47:0/1103652126 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f56040fdbb0 0x7f560410e3d0 unknown :-1 s=CLOSED pgs=2690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:25.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.378+0000 7f560ad0a700 1 -- 172.21.15.47:0/1103652126 >> 172.21.15.47:0/1103652126 conn(0x7f56040f3800 msgr2=0x7f56040fb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:25.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.378+0000 7f560ad0a700 1 -- 172.21.15.47:0/1103652126 shutdown_connections 2024-02-23T00:49:25.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.378+0000 7f560ad0a700 1 -- 172.21.15.47:0/1103652126 wait complete. 2024-02-23T00:49:25.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:25.378+0000 7f560ad0a700 1 librados: shutdown 2024-02-23T00:49:25.549 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is unset 2024-02-23T00:49:25.687 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:25.258247+0000 mon.c (mon.2) 307 : audit [INF] from='client.? 172.21.15.47:0/1103652126' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:49:25.687 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:25.258518+0000 mon.a (mon.0) 2665 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:49:25.695 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:49:25.721 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:25.258247+0000 mon.c (mon.2) 307 : audit [INF] from='client.? 172.21.15.47:0/1103652126' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:49:25.721 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:25.258518+0000 mon.a (mon.0) 2665 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:49:25.721 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:25.258247+0000 mon.c (mon.2) 307 : audit [INF] from='client.? 172.21.15.47:0/1103652126' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:49:25.721 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:25.258518+0000 mon.a (mon.0) 2665 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:49:25.727 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:49:25.731 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:25.731 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:49:25.832 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:49:25.861 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:25.900 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:49:25.909 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:25.910 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:49:26.011 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:49:26.040 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:26.068 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:49:26.071 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:26.071 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:49:26.086 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.6 dump_blocked_ops 2024-02-23T00:49:26.172 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:49:26.199 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:26.217 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:49:26.220 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:26.220 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:49:26.321 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:49:26.349 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:26.370 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:49:26.372 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:26.373 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:49:26.402 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:49:26.473 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:49:26.502 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:26.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:25.306833+0000 mgr.y (mgr.14517) 429 : cluster [DBG] pgmap v679: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 7.0 KiB/s rd, 0 B/s wr, 11 op/s 2024-02-23T00:49:26.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:25.368824+0000 mon.a (mon.0) 2666 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:49:26.643 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:25.376860+0000 mon.a (mon.0) 2667 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:49:26.643 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:25.376943+0000 mon.a (mon.0) 2668 : cluster [DBG] osdmap e458: 8 total, 8 up, 7 in 2024-02-23T00:49:26.643 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:25.954774+0000 osd.3 (osd.3) 212 : cluster [DBG] 2.e scrub starts 2024-02-23T00:49:26.643 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:25.955708+0000 osd.3 (osd.3) 213 : cluster [DBG] 2.e scrub ok 2024-02-23T00:49:26.655 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:25.306833+0000 mgr.y (mgr.14517) 429 : cluster [DBG] pgmap v679: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 7.0 KiB/s rd, 0 B/s wr, 11 op/s 2024-02-23T00:49:26.655 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:25.368824+0000 mon.a (mon.0) 2666 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:49:26.655 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:25.376860+0000 mon.a (mon.0) 2667 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:49:26.655 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:25.376943+0000 mon.a (mon.0) 2668 : cluster [DBG] osdmap e458: 8 total, 8 up, 7 in 2024-02-23T00:49:26.655 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:25.954774+0000 osd.3 (osd.3) 212 : cluster [DBG] 2.e scrub starts 2024-02-23T00:49:26.655 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:25.955708+0000 osd.3 (osd.3) 213 : cluster [DBG] 2.e scrub ok 2024-02-23T00:49:26.656 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:25.306833+0000 mgr.y (mgr.14517) 429 : cluster [DBG] pgmap v679: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 7.0 KiB/s rd, 0 B/s wr, 11 op/s 2024-02-23T00:49:26.656 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:25.368824+0000 mon.a (mon.0) 2666 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:49:26.656 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:25.376860+0000 mon.a (mon.0) 2667 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:49:26.656 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:25.376943+0000 mon.a (mon.0) 2668 : cluster [DBG] osdmap e458: 8 total, 8 up, 7 in 2024-02-23T00:49:26.656 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:25.954774+0000 osd.3 (osd.3) 212 : cluster [DBG] 2.e scrub starts 2024-02-23T00:49:26.656 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:25.955708+0000 osd.3 (osd.3) 213 : cluster [DBG] 2.e scrub ok 2024-02-23T00:49:27.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.005+0000 7f2fc9ad5700 1 -- 172.21.15.47:0/3263140365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fc40fd510 msgr2=0x7f2fc40f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:27.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.005+0000 7f2fc9ad5700 1 --2- 172.21.15.47:0/3263140365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fc40fd510 0x7f2fc40f6d40 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 rx=0x7f2fac004870 tx=0x7f2fac009cb0).stop 2024-02-23T00:49:27.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.005+0000 7f2fc9ad5700 1 -- 172.21.15.47:0/3263140365 shutdown_connections 2024-02-23T00:49:27.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.005+0000 7f2fc9ad5700 1 --2- 172.21.15.47:0/3263140365 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2fc40f7400 0x7f2fc4103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:27.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.005+0000 7f2fc9ad5700 1 --2- 172.21.15.47:0/3263140365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2fc40fcbb0 0x7f2fc40fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:27.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.005+0000 7f2fc9ad5700 1 --2- 172.21.15.47:0/3263140365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fc40fd510 0x7f2fc40f6d40 unknown :-1 s=CLOSED pgs=2545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:27.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.005+0000 7f2fc9ad5700 1 -- 172.21.15.47:0/3263140365 >> 172.21.15.47:0/3263140365 conn(0x7f2fc40f2820 msgr2=0x7f2fc40f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:27.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.005+0000 7f2fc9ad5700 1 -- 172.21.15.47:0/3263140365 shutdown_connections 2024-02-23T00:49:27.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.005+0000 7f2fc9ad5700 1 -- 172.21.15.47:0/3263140365 wait complete. 2024-02-23T00:49:27.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.005+0000 7f2fc9ad5700 1 librados: starting msgr at 2024-02-23T00:49:27.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.005+0000 7f2fc9ad5700 1 librados: starting objecter 2024-02-23T00:49:27.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.005+0000 7f2fc9ad5700 1 Processor -- start 2024-02-23T00:49:27.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.006+0000 7f2fc9ad5700 1 -- start start 2024-02-23T00:49:27.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.006+0000 7f2fc9ad5700 1 librados: setting wanted keys 2024-02-23T00:49:27.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.006+0000 7f2fc9ad5700 1 librados: calling monclient init 2024-02-23T00:49:27.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.006+0000 7f2fc9ad5700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2fc40f7400 0x7f2fc40f91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:27.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.006+0000 7f2fc9ad5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fc40fcbb0 0x7f2fc40f96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:27.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.006+0000 7f2fc9ad5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2fc40fd510 0x7f2fc40f9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:27.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.006+0000 7f2fc9ad5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc4116510 con 0x7f2fc40fcbb0 2024-02-23T00:49:27.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.006+0000 7f2fc9ad5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc4116680 con 0x7f2fc40fd510 2024-02-23T00:49:27.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.006+0000 7f2fc9ad5700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc4116840 con 0x7f2fc40f7400 2024-02-23T00:49:27.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.006+0000 7f2fc2ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fc40fcbb0 0x7f2fc40f96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:27.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.006+0000 7f2fc37fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2fc40f7400 0x7f2fc40f91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:27.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.006+0000 7f2fc2ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fc40fcbb0 0x7f2fc40f96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45582/0 (socket says 172.21.15.47:45582) 2024-02-23T00:49:27.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.006+0000 7f2fc2ffd700 1 -- 172.21.15.47:0/3332432451 learned_addr learned my addr 172.21.15.47:0/3332432451 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:49:27.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.006+0000 7f2fc3fff700 1 --2- 172.21.15.47:0/3332432451 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2fc40fd510 0x7f2fc40f9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:27.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.006+0000 7f2fc2ffd700 1 -- 172.21.15.47:0/3332432451 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2fc40f7400 msgr2=0x7f2fc40f91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:27.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.006+0000 7f2fc2ffd700 1 --2- 172.21.15.47:0/3332432451 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2fc40f7400 0x7f2fc40f91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:27.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.006+0000 7f2fc2ffd700 1 -- 172.21.15.47:0/3332432451 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2fc40fd510 msgr2=0x7f2fc40f9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:27.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.006+0000 7f2fc2ffd700 1 --2- 172.21.15.47:0/3332432451 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2fc40fd510 0x7f2fc40f9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:27.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.006+0000 7f2fc2ffd700 1 -- 172.21.15.47:0/3332432451 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fac005040 con 0x7f2fc40fcbb0 2024-02-23T00:49:27.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.006+0000 7f2fc2ffd700 1 --2- 172.21.15.47:0/3332432451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fc40fcbb0 0x7f2fc40f96c0 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 rx=0x7f2fac000e50 tx=0x7f2fac009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:27.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.006+0000 7f2fc0f74700 10 client.?.objecter ms_handle_connect 0x7f2fc40fcbb0 2024-02-23T00:49:27.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.006+0000 7f2fc0f74700 10 client.?.objecter resend_mon_ops 2024-02-23T00:49:27.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.006+0000 7f2fc0f74700 1 -- 172.21.15.47:0/3332432451 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac00e030 con 0x7f2fc40fcbb0 2024-02-23T00:49:27.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.006+0000 7f2fc0f74700 1 -- 172.21.15.47:0/3332432451 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f2fac013df0 con 0x7f2fc40fcbb0 2024-02-23T00:49:27.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.006+0000 7f2fc9ad5700 1 -- 172.21.15.47:0/3332432451 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc4116ac0 con 0x7f2fc40fcbb0 2024-02-23T00:49:27.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.006+0000 7f2fc0f74700 1 -- 172.21.15.47:0/3332432451 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac018070 con 0x7f2fc40fcbb0 2024-02-23T00:49:27.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.007+0000 7f2fc9ad5700 10 client.31128.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:49:27.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.007+0000 7f2fc9ad5700 1 -- 172.21.15.47:0/3332432451 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc4116e70 con 0x7f2fc40fcbb0 2024-02-23T00:49:27.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.007+0000 7f2fc9ad5700 1 librados: init done 2024-02-23T00:49:27.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.008+0000 7f2fc9ad5700 10 client.31128.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:27.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.008+0000 7f2fc9ad5700 15 client.31128.objecter _session_command_op_assign -1 1 2024-02-23T00:49:27.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.008+0000 7f2fc9ad5700 20 client.31128.objecter _get_session osd=-1 returning homeless 2024-02-23T00:49:27.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.008+0000 7f2fc9ad5700 10 client.31128.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:49:27.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.008+0000 7f2fc9ad5700 1 -- 172.21.15.47:0/3332432451 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2fc405fa60 con 0x7f2fc40fcbb0 2024-02-23T00:49:27.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.008+0000 7f2fc0f74700 1 -- 172.21.15.47:0/3332432451 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f2fac009240 con 0x7f2fc40fcbb0 2024-02-23T00:49:27.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.008+0000 7f2fc0f74700 1 --2- 172.21.15.47:0/3332432451 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2fb0062cc0 0x7f2fb0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:27.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.008+0000 7f2fc0f74700 1 -- 172.21.15.47:0/3332432451 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(458..458 src has 48..458) v4 ==== 9220+0+0 (secure 0 0 0) 0x7f2fac07e8b0 con 0x7f2fc40fcbb0 2024-02-23T00:49:27.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.008+0000 7f2fc0f74700 10 client.31128.objecter ms_dispatch 0x7f2fc4001550 osd_map(458..458 src has 48..458) v4 2024-02-23T00:49:27.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.008+0000 7f2fc0f74700 3 client.31128.objecter handle_osd_map got epochs [458,458] > 0 2024-02-23T00:49:27.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.008+0000 7f2fc0f74700 3 client.31128.objecter handle_osd_map decoding full epoch 458 2024-02-23T00:49:27.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.008+0000 7f2fc37fe700 1 --2- 172.21.15.47:0/3332432451 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2fb0062cc0 0x7f2fb0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:27.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.008+0000 7f2fc0f74700 10 client.31128.objecter checking command 1 2024-02-23T00:49:27.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.008+0000 7f2fc0f74700 1 --2- 172.21.15.47:0/3332432451 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2fb006f870 0x7f2fb0071d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:27.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.008+0000 7f2fc0f74700 20 client.31128.objecter _get_session s=0x7f2fb006f1a0 osd=4 3 2024-02-23T00:49:27.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.008+0000 7f2fc0f74700 20 client.31128.objecter put_session s=0x7f2fb006f1a0 osd=4 3 2024-02-23T00:49:27.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.008+0000 7f2fc0f74700 15 client.31128.objecter _session_command_op_remove -1 1 2024-02-23T00:49:27.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.008+0000 7f2fc0f74700 20 client.31128.objecter _get_session s=0x7f2fb006f1a0 osd=4 3 2024-02-23T00:49:27.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.008+0000 7f2fc0f74700 20 client.31128.objecter get_session s=0x7f2fb006f1a0 osd=4 3 2024-02-23T00:49:27.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.008+0000 7f2fc0f74700 15 client.31128.objecter _session_command_op_assign 4 1 2024-02-23T00:49:27.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.008+0000 7f2fc0f74700 20 client.31128.objecter put_session s=0x7f2fb006f1a0 osd=4 4 2024-02-23T00:49:27.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.008+0000 7f2fc0f74700 10 client.31128.objecter _send_command 1 2024-02-23T00:49:27.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.008+0000 7f2fc0f74700 1 -- 172.21.15.47:0/3332432451 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2fb0072390 con 0x7f2fb006f870 2024-02-23T00:49:27.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.008+0000 7f2fc0f74700 20 client.31128.objecter dump_active .. 0 homeless 2024-02-23T00:49:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.008+0000 7f2fc0f74700 1 -- 172.21.15.47:0/3332432451 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=458) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fac083220 con 0x7f2fc40fcbb0 2024-02-23T00:49:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.009+0000 7f2fc0f74700 10 client.31128.objecter ms_handle_connect 0x7f2fb0062cc0 2024-02-23T00:49:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.009+0000 7f2fc37fe700 1 --2- 172.21.15.47:0/3332432451 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2fb0062cc0 0x7f2fb0065180 secure :-1 s=READY pgs=3084 cs=0 l=1 rev1=1 rx=0x7f2fb40048b0 tx=0x7f2fb400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.009+0000 7f2fc3fff700 1 --2- 172.21.15.47:0/3332432451 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2fb006f870 0x7f2fb0071d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.009+0000 7f2fc0f74700 10 client.31128.objecter ms_handle_connect 0x7f2fb006f870 2024-02-23T00:49:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.009+0000 7f2fc3fff700 1 --2- 172.21.15.47:0/3332432451 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2fb006f870 0x7f2fb0071d10 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.010+0000 7f2fc0f74700 1 -- 172.21.15.47:0/3332432451 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2fb0072390 con 0x7f2fb006f870 2024-02-23T00:49:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.010+0000 7f2fc0f74700 10 client.31128.objecter ms_dispatch 0x7f2fc4001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:49:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.010+0000 7f2fc0f74700 10 client.31128.objecter _finish_command 1 = system:0 2024-02-23T00:49:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.010+0000 7f2fc0f74700 20 client.31128.objecter put_session s=0x7f2fb006f1a0 osd=4 4 2024-02-23T00:49:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.010+0000 7f2fc0f74700 15 client.31128.objecter _session_command_op_remove 4 1 2024-02-23T00:49:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.027+0000 7f2fc9ad5700 10 client.31128.objecter _submit_command 2 [{"prefix": "reset_purged_snaps_last"}] 2024-02-23T00:49:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.027+0000 7f2fc9ad5700 15 client.31128.objecter _session_command_op_assign -1 2 2024-02-23T00:49:27.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.027+0000 7f2fc9ad5700 20 client.31128.objecter _get_session s=0x7f2fb006f1a0 osd=4 3 2024-02-23T00:49:27.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.027+0000 7f2fc9ad5700 20 client.31128.objecter put_session s=0x7f2fb006f1a0 osd=4 3 2024-02-23T00:49:27.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.027+0000 7f2fc9ad5700 20 client.31128.objecter _get_session s=0x7f2fb006f1a0 osd=4 3 2024-02-23T00:49:27.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.027+0000 7f2fc9ad5700 15 client.31128.objecter _session_command_op_remove -1 2 2024-02-23T00:49:27.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.027+0000 7f2fc9ad5700 20 client.31128.objecter get_session s=0x7f2fb006f1a0 osd=4 3 2024-02-23T00:49:27.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.027+0000 7f2fc9ad5700 15 client.31128.objecter _session_command_op_assign 4 2 2024-02-23T00:49:27.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.027+0000 7f2fc9ad5700 20 client.31128.objecter put_session s=0x7f2fb006f1a0 osd=4 4 2024-02-23T00:49:27.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.027+0000 7f2fc9ad5700 10 client.31128.objecter _send_command 2 2024-02-23T00:49:27.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.027+0000 7f2fc9ad5700 1 -- 172.21.15.47:0/3332432451 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) v1 -- 0x7f2fc405ee80 con 0x7f2fb006f870 2024-02-23T00:49:27.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.027+0000 7f2fc0f74700 1 -- 172.21.15.47:0/3332432451 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f2fc405ee80 con 0x7f2fb006f870 2024-02-23T00:49:27.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.027+0000 7f2fc0f74700 10 client.31128.objecter ms_dispatch 0x7f2fc4001550 command_reply(tid 2: 0 ) v1 2024-02-23T00:49:27.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.027+0000 7f2fc0f74700 10 client.31128.objecter _finish_command 2 = system:0 2024-02-23T00:49:27.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.027+0000 7f2fc0f74700 20 client.31128.objecter put_session s=0x7f2fb006f1a0 osd=4 4 2024-02-23T00:49:27.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.027+0000 7f2fc0f74700 15 client.31128.objecter _session_command_op_remove 4 2 2024-02-23T00:49:27.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.027+0000 7f2fc9ad5700 10 librados: watch_flush enter 2024-02-23T00:49:27.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.027+0000 7f2fc9ad5700 10 librados: watch_flush exit 2024-02-23T00:49:27.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.027+0000 7f2fc9ad5700 10 client.31128.objecter close_session for osd.4 2024-02-23T00:49:27.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.027+0000 7f2fc9ad5700 1 -- 172.21.15.47:0/3332432451 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2fb006f870 msgr2=0x7f2fb0071d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:27.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.027+0000 7f2fc9ad5700 1 --2- 172.21.15.47:0/3332432451 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2fb006f870 0x7f2fb0071d10 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:27.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.027+0000 7f2fc9ad5700 20 client.31128.objecter put_session s=0x7f2fb006f1a0 osd=4 1 2024-02-23T00:49:27.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.027+0000 7f2fc9ad5700 20 client.31128.objecter shutdown clearing up homeless session... 2024-02-23T00:49:27.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.027+0000 7f2fc9ad5700 10 client.31128.objecter successfully canceled tick 2024-02-23T00:49:27.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.028+0000 7f2fc9ad5700 1 -- 172.21.15.47:0/3332432451 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2fb0062cc0 msgr2=0x7f2fb0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:27.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.028+0000 7f2fc9ad5700 1 --2- 172.21.15.47:0/3332432451 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2fb0062cc0 0x7f2fb0065180 secure :-1 s=READY pgs=3084 cs=0 l=1 rev1=1 rx=0x7f2fb40048b0 tx=0x7f2fb400b040).stop 2024-02-23T00:49:27.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.028+0000 7f2fc9ad5700 1 -- 172.21.15.47:0/3332432451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fc40fcbb0 msgr2=0x7f2fc40f96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:27.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.028+0000 7f2fc9ad5700 1 --2- 172.21.15.47:0/3332432451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fc40fcbb0 0x7f2fc40f96c0 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 rx=0x7f2fac000e50 tx=0x7f2fac009cb0).stop 2024-02-23T00:49:27.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.028+0000 7f2fc3fff700 1 -- 172.21.15.47:0/3332432451 reap_dead start 2024-02-23T00:49:27.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.028+0000 7f2fc9ad5700 1 -- 172.21.15.47:0/3332432451 shutdown_connections 2024-02-23T00:49:27.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.028+0000 7f2fc9ad5700 1 -- 172.21.15.47:0/3332432451 >> 172.21.15.47:0/3332432451 conn(0x7f2fc40f2820 msgr2=0x7f2fc40f4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:27.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.028+0000 7f2fc9ad5700 1 -- 172.21.15.47:0/3332432451 shutdown_connections 2024-02-23T00:49:27.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.028+0000 7f2fc9ad5700 1 -- 172.21.15.47:0/3332432451 wait complete. 2024-02-23T00:49:27.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:27.028+0000 7f2fc9ad5700 1 librados: shutdown 2024-02-23T00:49:27.552 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:49:27.556 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:27.556 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:49:27.585 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:26.429779+0000 osd.4 (osd.4) 189 : cluster [DBG] 62.5 scrub starts 2024-02-23T00:49:27.586 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:26.430984+0000 osd.4 (osd.4) 190 : cluster [DBG] 62.5 scrub ok 2024-02-23T00:49:27.586 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:26.514467+0000 osd.1 (osd.1) 201 : cluster [DBG] 4.1c scrub starts 2024-02-23T00:49:27.586 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:26.517955+0000 osd.1 (osd.1) 202 : cluster [DBG] 4.1c scrub ok 2024-02-23T00:49:27.586 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:26.669754+0000 osd.5 (osd.5) 192 : cluster [DBG] 2.16 scrub starts 2024-02-23T00:49:27.586 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:26.672652+0000 osd.5 (osd.5) 193 : cluster [DBG] 2.16 scrub ok 2024-02-23T00:49:27.589 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:49:27.657 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:49:27.708 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:27.711 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:26.429779+0000 osd.4 (osd.4) 189 : cluster [DBG] 62.5 scrub starts 2024-02-23T00:49:27.711 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:26.430984+0000 osd.4 (osd.4) 190 : cluster [DBG] 62.5 scrub ok 2024-02-23T00:49:27.711 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:26.514467+0000 osd.1 (osd.1) 201 : cluster [DBG] 4.1c scrub starts 2024-02-23T00:49:27.711 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:26.517955+0000 osd.1 (osd.1) 202 : cluster [DBG] 4.1c scrub ok 2024-02-23T00:49:27.711 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:26.669754+0000 osd.5 (osd.5) 192 : cluster [DBG] 2.16 scrub starts 2024-02-23T00:49:27.711 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:26.672652+0000 osd.5 (osd.5) 193 : cluster [DBG] 2.16 scrub ok 2024-02-23T00:49:27.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:26.429779+0000 osd.4 (osd.4) 189 : cluster [DBG] 62.5 scrub starts 2024-02-23T00:49:27.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:26.430984+0000 osd.4 (osd.4) 190 : cluster [DBG] 62.5 scrub ok 2024-02-23T00:49:27.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:26.514467+0000 osd.1 (osd.1) 201 : cluster [DBG] 4.1c scrub starts 2024-02-23T00:49:27.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:26.517955+0000 osd.1 (osd.1) 202 : cluster [DBG] 4.1c scrub ok 2024-02-23T00:49:27.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:26.669754+0000 osd.5 (osd.5) 192 : cluster [DBG] 2.16 scrub starts 2024-02-23T00:49:27.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:26.672652+0000 osd.5 (osd.5) 193 : cluster [DBG] 2.16 scrub ok 2024-02-23T00:49:27.906 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:49:27.908 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:27.909 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:49:28.010 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:49:28.042 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:28.125 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:49:28.125 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:49:28.143 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:49:28.145 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:28.146 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:49:28.153 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph tell osd.5 reset_purged_snaps_last 2024-02-23T00:49:28.247 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:49:28.276 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:28.400 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:49:28.403 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:28.403 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:49:28.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:27.307584+0000 mgr.y (mgr.14517) 430 : cluster [DBG] pgmap v681: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 55 KiB/s rd, 383 B/s wr, 92 op/s 2024-02-23T00:49:28.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:27.400409+0000 osd.7 (osd.7) 221 : cluster [DBG] 2.1c scrub starts 2024-02-23T00:49:28.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:27.403301+0000 osd.7 (osd.7) 222 : cluster [DBG] 2.1c scrub ok 2024-02-23T00:49:28.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:27.459051+0000 osd.4 (osd.4) 191 : cluster [DBG] 5.18 scrub starts 2024-02-23T00:49:28.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:27.459886+0000 osd.4 (osd.4) 192 : cluster [DBG] 5.18 scrub ok 2024-02-23T00:49:28.504 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:49:28.535 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:28.608 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:49:28.610 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:28.610 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:49:28.639 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:27.307584+0000 mgr.y (mgr.14517) 430 : cluster [DBG] pgmap v681: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 55 KiB/s rd, 383 B/s wr, 92 op/s 2024-02-23T00:49:28.639 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:27.400409+0000 osd.7 (osd.7) 221 : cluster [DBG] 2.1c scrub starts 2024-02-23T00:49:28.639 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:27.403301+0000 osd.7 (osd.7) 222 : cluster [DBG] 2.1c scrub ok 2024-02-23T00:49:28.639 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:27.459051+0000 osd.4 (osd.4) 191 : cluster [DBG] 5.18 scrub starts 2024-02-23T00:49:28.639 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:27.459886+0000 osd.4 (osd.4) 192 : cluster [DBG] 5.18 scrub ok 2024-02-23T00:49:28.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:27.307584+0000 mgr.y (mgr.14517) 430 : cluster [DBG] pgmap v681: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 55 KiB/s rd, 383 B/s wr, 92 op/s 2024-02-23T00:49:28.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:27.400409+0000 osd.7 (osd.7) 221 : cluster [DBG] 2.1c scrub starts 2024-02-23T00:49:28.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:27.403301+0000 osd.7 (osd.7) 222 : cluster [DBG] 2.1c scrub ok 2024-02-23T00:49:28.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:27.459051+0000 osd.4 (osd.4) 191 : cluster [DBG] 5.18 scrub starts 2024-02-23T00:49:28.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:27.459886+0000 osd.4 (osd.4) 192 : cluster [DBG] 5.18 scrub ok 2024-02-23T00:49:28.712 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:49:28.741 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:28.890 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:49:28.895 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:28.896 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:49:28.904 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.6 dump_historic_ops 2024-02-23T00:49:28.997 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:49:29.014 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:49:29.024 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:29.042 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:49:29.045 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:29.045 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:49:29.146 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:49:29.175 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:29.195 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:49:29.198 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:29.198 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:49:29.218 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:49:29.299 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:49:29.333 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:29.355 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:49:29.358 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:29.358 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:49:29.459 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:49:29.486 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:29.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:27.763402+0000 osd.0 (osd.0) 191 : cluster [DBG] 4.b scrub starts 2024-02-23T00:49:29.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:27.765042+0000 osd.0 (osd.0) 192 : cluster [DBG] 4.b scrub ok 2024-02-23T00:49:29.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:28.346460+0000 osd.6 (osd.6) 185 : cluster [DBG] 2.11 scrub starts 2024-02-23T00:49:29.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:28.347277+0000 osd.6 (osd.6) 186 : cluster [DBG] 2.11 scrub ok 2024-02-23T00:49:29.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:28.355746+0000 osd.7 (osd.7) 223 : cluster [DBG] 5.4 scrub starts 2024-02-23T00:49:29.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:28.358641+0000 osd.7 (osd.7) 224 : cluster [DBG] 5.4 scrub ok 2024-02-23T00:49:29.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:28.471560+0000 osd.1 (osd.1) 203 : cluster [DBG] 62.3 scrub starts 2024-02-23T00:49:29.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:28.475040+0000 osd.1 (osd.1) 204 : cluster [DBG] 62.3 scrub ok 2024-02-23T00:49:29.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:28.479654+0000 osd.4 (osd.4) 193 : cluster [DBG] 62.1b scrub starts 2024-02-23T00:49:29.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:28.481695+0000 osd.4 (osd.4) 194 : cluster [DBG] 62.1b scrub ok 2024-02-23T00:49:29.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:27.763402+0000 osd.0 (osd.0) 191 : cluster [DBG] 4.b scrub starts 2024-02-23T00:49:29.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:27.765042+0000 osd.0 (osd.0) 192 : cluster [DBG] 4.b scrub ok 2024-02-23T00:49:29.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:28.346460+0000 osd.6 (osd.6) 185 : cluster [DBG] 2.11 scrub starts 2024-02-23T00:49:29.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:28.347277+0000 osd.6 (osd.6) 186 : cluster [DBG] 2.11 scrub ok 2024-02-23T00:49:29.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:28.355746+0000 osd.7 (osd.7) 223 : cluster [DBG] 5.4 scrub starts 2024-02-23T00:49:29.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:28.358641+0000 osd.7 (osd.7) 224 : cluster [DBG] 5.4 scrub ok 2024-02-23T00:49:29.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:28.471560+0000 osd.1 (osd.1) 203 : cluster [DBG] 62.3 scrub starts 2024-02-23T00:49:29.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:28.475040+0000 osd.1 (osd.1) 204 : cluster [DBG] 62.3 scrub ok 2024-02-23T00:49:29.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:28.479654+0000 osd.4 (osd.4) 193 : cluster [DBG] 62.1b scrub starts 2024-02-23T00:49:29.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:28.481695+0000 osd.4 (osd.4) 194 : cluster [DBG] 62.1b scrub ok 2024-02-23T00:49:29.573 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-23T00:49:29.634 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:49:29.635 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:27.763402+0000 osd.0 (osd.0) 191 : cluster [DBG] 4.b scrub starts 2024-02-23T00:49:29.635 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:27.765042+0000 osd.0 (osd.0) 192 : cluster [DBG] 4.b scrub ok 2024-02-23T00:49:29.635 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:28.346460+0000 osd.6 (osd.6) 185 : cluster [DBG] 2.11 scrub starts 2024-02-23T00:49:29.636 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:28.347277+0000 osd.6 (osd.6) 186 : cluster [DBG] 2.11 scrub ok 2024-02-23T00:49:29.636 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:28.355746+0000 osd.7 (osd.7) 223 : cluster [DBG] 5.4 scrub starts 2024-02-23T00:49:29.636 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:28.358641+0000 osd.7 (osd.7) 224 : cluster [DBG] 5.4 scrub ok 2024-02-23T00:49:29.636 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:28.471560+0000 osd.1 (osd.1) 203 : cluster [DBG] 62.3 scrub starts 2024-02-23T00:49:29.636 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:28.475040+0000 osd.1 (osd.1) 204 : cluster [DBG] 62.3 scrub ok 2024-02-23T00:49:29.636 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:28.479654+0000 osd.4 (osd.4) 193 : cluster [DBG] 62.1b scrub starts 2024-02-23T00:49:29.636 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:28.481695+0000 osd.4 (osd.4) 194 : cluster [DBG] 62.1b scrub ok 2024-02-23T00:49:29.637 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:29.637 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:49:29.738 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:49:29.765 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:29.839 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:49:29.841 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:29.842 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:49:29.943 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:49:29.970 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:30.068 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:49:30.070 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:30.071 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:49:30.172 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:49:30.202 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:30.352 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:49:30.354 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:30.355 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:49:30.456 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:49:30.483 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:30.502 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:49:30.506 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:30.506 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:49:30.607 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:49:30.633 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:30.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:29.308035+0000 mgr.y (mgr.14517) 431 : cluster [DBG] pgmap v682: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 49 KiB/s rd, 337 B/s wr, 81 op/s 2024-02-23T00:49:30.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:30.206394+0000 mon.a (mon.0) 2669 : cluster [DBG] osdmap e459: 8 total, 8 up, 7 in 2024-02-23T00:49:30.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:29.308035+0000 mgr.y (mgr.14517) 431 : cluster [DBG] pgmap v682: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 49 KiB/s rd, 337 B/s wr, 81 op/s 2024-02-23T00:49:30.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:30.206394+0000 mon.a (mon.0) 2669 : cluster [DBG] osdmap e459: 8 total, 8 up, 7 in 2024-02-23T00:49:30.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:29.308035+0000 mgr.y (mgr.14517) 431 : cluster [DBG] pgmap v682: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 49 KiB/s rd, 337 B/s wr, 81 op/s 2024-02-23T00:49:30.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:30.206394+0000 mon.a (mon.0) 2669 : cluster [DBG] osdmap e459: 8 total, 8 up, 7 in 2024-02-23T00:49:30.808 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:49:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:49:30] "GET /metrics HTTP/1.1" 200 189207 "" "Prometheus/2.33.4" 2024-02-23T00:49:30.910 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:49:30.913 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:30.913 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:49:31.014 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:49:31.040 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:31.188 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:49:31.191 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:31.191 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:49:31.292 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:49:31.319 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:31.346 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:49:31.348 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:31.349 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:49:31.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:30.394007+0000 osd.7 (osd.7) 225 : cluster [DBG] 6.5 scrub starts 2024-02-23T00:49:31.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:30.396861+0000 osd.7 (osd.7) 226 : cluster [DBG] 6.5 scrub ok 2024-02-23T00:49:31.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:30.498604+0000 osd.4 (osd.4) 195 : cluster [DBG] 5.a scrub starts 2024-02-23T00:49:31.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:30.499765+0000 osd.4 (osd.4) 196 : cluster [DBG] 5.a scrub ok 2024-02-23T00:49:31.450 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:49:31.474 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:49:31.474 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:49:31.476 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:31.496 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:49:31.498 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:31.499 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:49:31.600 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:49:31.632 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:30.394007+0000 osd.7 (osd.7) 225 : cluster [DBG] 6.5 scrub starts 2024-02-23T00:49:31.632 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:30.396861+0000 osd.7 (osd.7) 226 : cluster [DBG] 6.5 scrub ok 2024-02-23T00:49:31.632 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:30.498604+0000 osd.4 (osd.4) 195 : cluster [DBG] 5.a scrub starts 2024-02-23T00:49:31.632 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:30.499765+0000 osd.4 (osd.4) 196 : cluster [DBG] 5.a scrub ok 2024-02-23T00:49:31.633 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:30.394007+0000 osd.7 (osd.7) 225 : cluster [DBG] 6.5 scrub starts 2024-02-23T00:49:31.633 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:30.396861+0000 osd.7 (osd.7) 226 : cluster [DBG] 6.5 scrub ok 2024-02-23T00:49:31.633 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:30.498604+0000 osd.4 (osd.4) 195 : cluster [DBG] 5.a scrub starts 2024-02-23T00:49:31.633 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:30.499765+0000 osd.4 (osd.4) 196 : cluster [DBG] 5.a scrub ok 2024-02-23T00:49:31.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.619+0000 7fa9736f6700 1 -- 172.21.15.47:0/3565184399 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa96c0f8d10 msgr2=0x7fa96c0f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:31.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.619+0000 7fa9736f6700 1 --2- 172.21.15.47:0/3565184399 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa96c0f8d10 0x7fa96c0f90f0 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 rx=0x7fa968004870 tx=0x7fa968009cb0).stop 2024-02-23T00:49:31.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.619+0000 7fa9736f6700 1 -- 172.21.15.47:0/3565184399 shutdown_connections 2024-02-23T00:49:31.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.619+0000 7fa9736f6700 1 --2- 172.21.15.47:0/3565184399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa96c102270 0x7fa96c104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:49:31.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.619+0000 7fa9736f6700 1 --2- 172.21.15.47:0/3565184399 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa96c0f8d10 0x7fa96c0f90f0 unknown :-1 s=CLOSED pgs=2691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:31.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.619+0000 7fa9736f6700 1 --2- 172.21.15.47:0/3565184399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa96c0f9670 0x7fa96c101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:31.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.619+0000 7fa9736f6700 1 -- 172.21.15.47:0/3565184399 >> 172.21.15.47:0/3565184399 conn(0x7fa96c0f2820 msgr2=0x7fa96c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:31.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.619+0000 7fa9736f6700 1 -- 172.21.15.47:0/3565184399 shutdown_connections 2024-02-23T00:49:31.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.619+0000 7fa9736f6700 1 -- 172.21.15.47:0/3565184399 wait complete. 2024-02-23T00:49:31.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.619+0000 7fa9736f6700 1 librados: starting msgr at 2024-02-23T00:49:31.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.619+0000 7fa9736f6700 1 librados: starting objecter 2024-02-23T00:49:31.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.619+0000 7fa9736f6700 1 Processor -- start 2024-02-23T00:49:31.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.619+0000 7fa9736f6700 1 -- start start 2024-02-23T00:49:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.619+0000 7fa9736f6700 1 librados: setting wanted keys 2024-02-23T00:49:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.619+0000 7fa9736f6700 1 librados: calling monclient init 2024-02-23T00:49:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.620+0000 7fa9736f6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa96c0f8d10 0x7fa96c0f6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.620+0000 7fa9736f6700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa96c0f9670 0x7fa96c0f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.620+0000 7fa9736f6700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa96c102270 0x7fa96c0f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.620+0000 7fa9736f6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c116620 con 0x7fa96c0f8d10 2024-02-23T00:49:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.620+0000 7fa9736f6700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c116790 con 0x7fa96c102270 2024-02-23T00:49:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.620+0000 7fa9736f6700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c116950 con 0x7fa96c0f9670 2024-02-23T00:49:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.620+0000 7fa970c91700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa96c0f9670 0x7fa96c0f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.620+0000 7fa970c91700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa96c0f9670 0x7fa96c0f71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:54086/0 (socket says 172.21.15.47:54086) 2024-02-23T00:49:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.620+0000 7fa971492700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa96c0f8d10 0x7fa96c0f6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.620+0000 7fa970c91700 1 -- 172.21.15.47:0/3268240606 learned_addr learned my addr 172.21.15.47:0/3268240606 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:49:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.620+0000 7fa971c93700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa96c102270 0x7fa96c0f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.620+0000 7fa970c91700 1 -- 172.21.15.47:0/3268240606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa96c102270 msgr2=0x7fa96c0f76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.620+0000 7fa970c91700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa96c102270 0x7fa96c0f76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.620+0000 7fa970c91700 1 -- 172.21.15.47:0/3268240606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa96c0f8d10 msgr2=0x7fa96c0f6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.620+0000 7fa970c91700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa96c0f8d10 0x7fa96c0f6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.620+0000 7fa970c91700 1 -- 172.21.15.47:0/3268240606 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa95c005040 con 0x7fa96c0f9670 2024-02-23T00:49:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.620+0000 7fa971c93700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa96c102270 0x7fa96c0f76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:49:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.620+0000 7fa970c91700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa96c0f9670 0x7fa96c0f71b0 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 rx=0x7fa95c0048f0 tx=0x7fa95c009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.620+0000 7fa9627fc700 10 client.?.objecter ms_handle_connect 0x7fa96c0f9670 2024-02-23T00:49:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.620+0000 7fa9627fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:49:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.620+0000 7fa9627fc700 1 -- 172.21.15.47:0/3268240606 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c007b00 con 0x7fa96c0f9670 2024-02-23T00:49:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.620+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa968005040 con 0x7fa96c0f9670 2024-02-23T00:49:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.620+0000 7fa9627fc700 1 -- 172.21.15.47:0/3268240606 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fa95c00f040 con 0x7fa96c0f9670 2024-02-23T00:49:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.620+0000 7fa9627fc700 1 -- 172.21.15.47:0/3268240606 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c014890 con 0x7fa96c0f9670 2024-02-23T00:49:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.620+0000 7fa9736f6700 10 client.41459.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:49:31.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.621+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa96c116f90 con 0x7fa96c0f9670 2024-02-23T00:49:31.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.621+0000 7fa9736f6700 1 librados: init done 2024-02-23T00:49:31.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.621+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fa96c05fa60 con 0x7fa96c0f9670 2024-02-23T00:49:31.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.622+0000 7fa9627fc700 1 -- 172.21.15.47:0/3268240606 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fa95c014cc0 con 0x7fa96c0f9670 2024-02-23T00:49:31.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.622+0000 7fa9627fc700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa958062cc0 0x7fa958065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:31.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.622+0000 7fa9627fc700 1 -- 172.21.15.47:0/3268240606 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(459..459 src has 48..459) v4 ==== 9220+0+0 (secure 0 0 0) 0x7fa95c07d330 con 0x7fa96c0f9670 2024-02-23T00:49:31.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.622+0000 7fa9627fc700 10 client.41459.objecter ms_dispatch 0x7fa96c001550 osd_map(459..459 src has 48..459) v4 2024-02-23T00:49:31.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.622+0000 7fa9627fc700 3 client.41459.objecter handle_osd_map got epochs [459,459] > 0 2024-02-23T00:49:31.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.622+0000 7fa9627fc700 3 client.41459.objecter handle_osd_map decoding full epoch 459 2024-02-23T00:49:31.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.622+0000 7fa971492700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa958062cc0 0x7fa958065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:31.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.623+0000 7fa9627fc700 20 client.41459.objecter dump_active .. 0 homeless 2024-02-23T00:49:31.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.623+0000 7fa9627fc700 1 -- 172.21.15.47:0/3268240606 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v459) v1 ==== 54+0+16 (secure 0 0 0) 0x7fa95c084070 con 0x7fa96c0f9670 2024-02-23T00:49:31.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.623+0000 7fa971492700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa958062cc0 0x7fa958065180 secure :-1 s=READY pgs=3085 cs=0 l=1 rev1=1 rx=0x7fa968004ee0 tx=0x7fa96800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:31.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.623+0000 7fa9736f6700 10 client.41459.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:31.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.623+0000 7fa9627fc700 10 client.41459.objecter ms_handle_connect 0x7fa958062cc0 2024-02-23T00:49:31.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.623+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_assign -1 1 2024-02-23T00:49:31.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.623+0000 7fa9736f6700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fa96c028540 0x7fa96c0fbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:31.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.623+0000 7fa9736f6700 20 client.41459.objecter _get_session s=0x7fa96c05ee80 osd=0 3 2024-02-23T00:49:31.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.623+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c05ee80 osd=0 3 2024-02-23T00:49:31.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.623+0000 7fa9736f6700 20 client.41459.objecter _get_session s=0x7fa96c05ee80 osd=0 3 2024-02-23T00:49:31.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.623+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_remove -1 1 2024-02-23T00:49:31.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.623+0000 7fa9736f6700 20 client.41459.objecter get_session s=0x7fa96c05ee80 osd=0 3 2024-02-23T00:49:31.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.623+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_assign 0 1 2024-02-23T00:49:31.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.623+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c05ee80 osd=0 4 2024-02-23T00:49:31.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.623+0000 7fa9736f6700 10 client.41459.objecter _send_command 1 2024-02-23T00:49:31.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.623+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa96c107060 con 0x7fa96c028540 2024-02-23T00:49:31.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.623+0000 7fa971c93700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fa96c028540 0x7fa96c0fbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:31.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.623+0000 7fa9627fc700 10 client.41459.objecter ms_handle_connect 0x7fa96c028540 2024-02-23T00:49:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.623+0000 7fa971c93700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fa96c028540 0x7fa96c0fbd30 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.624+0000 7fa9627fc700 1 -- 172.21.15.47:0/3268240606 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa96c107060 con 0x7fa96c028540 2024-02-23T00:49:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.624+0000 7fa9627fc700 10 client.41459.objecter ms_dispatch 0x7fa96c001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:49:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.624+0000 7fa9627fc700 10 client.41459.objecter _finish_command 1 = system:0 2024-02-23T00:49:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.624+0000 7fa9627fc700 20 client.41459.objecter put_session s=0x7fa96c05ee80 osd=0 4 2024-02-23T00:49:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.624+0000 7fa9627fc700 15 client.41459.objecter _session_command_op_remove 0 1 2024-02-23T00:49:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:31.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.640+0000 7fa9736f6700 10 client.41459.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:49:31.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.640+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_assign -1 2 2024-02-23T00:49:31.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.640+0000 7fa9736f6700 20 client.41459.objecter _get_session s=0x7fa96c05ee80 osd=0 3 2024-02-23T00:49:31.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.640+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c05ee80 osd=0 3 2024-02-23T00:49:31.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.640+0000 7fa9736f6700 20 client.41459.objecter _get_session s=0x7fa96c05ee80 osd=0 3 2024-02-23T00:49:31.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.640+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_remove -1 2 2024-02-23T00:49:31.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.640+0000 7fa9736f6700 20 client.41459.objecter get_session s=0x7fa96c05ee80 osd=0 3 2024-02-23T00:49:31.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.640+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_assign 0 2 2024-02-23T00:49:31.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.640+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c05ee80 osd=0 4 2024-02-23T00:49:31.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.640+0000 7fa9736f6700 10 client.41459.objecter _send_command 2 2024-02-23T00:49:31.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.640+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa96c05a040 con 0x7fa96c028540 2024-02-23T00:49:31.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.642+0000 7fa9627fc700 1 -- 172.21.15.47:0/3268240606 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa96c05a040 con 0x7fa96c028540 2024-02-23T00:49:31.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.642+0000 7fa9627fc700 10 client.41459.objecter ms_dispatch 0x7fa96c001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:49:31.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.642+0000 7fa9627fc700 10 client.41459.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:49:31.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.642+0000 7fa9627fc700 20 client.41459.objecter put_session s=0x7fa96c05ee80 osd=0 4 2024-02-23T00:49:31.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.642+0000 7fa9627fc700 15 client.41459.objecter _session_command_op_remove 0 2 2024-02-23T00:49:31.646 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:49:31.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.643+0000 7fa9736f6700 10 client.41459.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:31.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.643+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_assign -1 3 2024-02-23T00:49:31.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.643+0000 7fa9736f6700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fa96c027b90 0x7fa96c027f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:31.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.643+0000 7fa9736f6700 20 client.41459.objecter _get_session s=0x7fa96c0fd3a0 osd=1 3 2024-02-23T00:49:31.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.643+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c0fd3a0 osd=1 3 2024-02-23T00:49:31.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.643+0000 7fa9736f6700 20 client.41459.objecter _get_session s=0x7fa96c0fd3a0 osd=1 3 2024-02-23T00:49:31.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.643+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_remove -1 3 2024-02-23T00:49:31.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.643+0000 7fa9736f6700 20 client.41459.objecter get_session s=0x7fa96c0fd3a0 osd=1 3 2024-02-23T00:49:31.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.643+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_assign 1 3 2024-02-23T00:49:31.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.643+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c0fd3a0 osd=1 4 2024-02-23T00:49:31.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.643+0000 7fa9736f6700 10 client.41459.objecter _send_command 3 2024-02-23T00:49:31.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.643+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa96c0fd510 con 0x7fa96c027b90 2024-02-23T00:49:31.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.643+0000 7fa971492700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fa96c027b90 0x7fa96c027f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:31.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.643+0000 7fa9627fc700 10 client.41459.objecter ms_handle_connect 0x7fa96c027b90 2024-02-23T00:49:31.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.643+0000 7fa971492700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fa96c027b90 0x7fa96c027f70 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:31.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.644+0000 7fa9627fc700 1 -- 172.21.15.47:0/3268240606 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa96c0fd510 con 0x7fa96c027b90 2024-02-23T00:49:31.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.644+0000 7fa9627fc700 10 client.41459.objecter ms_dispatch 0x7fa96c001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:49:31.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.644+0000 7fa9627fc700 10 client.41459.objecter _finish_command 3 = system:0 2024-02-23T00:49:31.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.644+0000 7fa9627fc700 20 client.41459.objecter put_session s=0x7fa96c0fd3a0 osd=1 4 2024-02-23T00:49:31.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.644+0000 7fa9627fc700 15 client.41459.objecter _session_command_op_remove 1 3 2024-02-23T00:49:31.656 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:49:31.658 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:31.658 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:49:31.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.661+0000 7fa9736f6700 10 client.41459.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:49:31.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.661+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_assign -1 4 2024-02-23T00:49:31.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.661+0000 7fa9736f6700 20 client.41459.objecter _get_session s=0x7fa96c0fd3a0 osd=1 3 2024-02-23T00:49:31.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.661+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c0fd3a0 osd=1 3 2024-02-23T00:49:31.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.661+0000 7fa9736f6700 20 client.41459.objecter _get_session s=0x7fa96c0fd3a0 osd=1 3 2024-02-23T00:49:31.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.661+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_remove -1 4 2024-02-23T00:49:31.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.661+0000 7fa9736f6700 20 client.41459.objecter get_session s=0x7fa96c0fd3a0 osd=1 3 2024-02-23T00:49:31.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.661+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_assign 1 4 2024-02-23T00:49:31.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.661+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c0fd3a0 osd=1 4 2024-02-23T00:49:31.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.661+0000 7fa9736f6700 10 client.41459.objecter _send_command 4 2024-02-23T00:49:31.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.661+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa96c027600 con 0x7fa96c027b90 2024-02-23T00:49:31.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.663+0000 7fa9627fc700 1 -- 172.21.15.47:0/3268240606 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa96c027600 con 0x7fa96c027b90 2024-02-23T00:49:31.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.663+0000 7fa9627fc700 10 client.41459.objecter ms_dispatch 0x7fa96c001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:49:31.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.663+0000 7fa9627fc700 10 client.41459.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:49:31.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.663+0000 7fa9627fc700 20 client.41459.objecter put_session s=0x7fa96c0fd3a0 osd=1 4 2024-02-23T00:49:31.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.663+0000 7fa9627fc700 15 client.41459.objecter _session_command_op_remove 1 4 2024-02-23T00:49:31.666 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:49:31.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.663+0000 7fa9736f6700 10 client.41459.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:31.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.663+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_assign -1 5 2024-02-23T00:49:31.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.663+0000 7fa9736f6700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa96c0fcf00 0x7fa96c11e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:31.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.663+0000 7fa9736f6700 20 client.41459.objecter _get_session s=0x7fa96c027a10 osd=2 3 2024-02-23T00:49:31.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.663+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c027a10 osd=2 3 2024-02-23T00:49:31.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.663+0000 7fa9736f6700 20 client.41459.objecter _get_session s=0x7fa96c027a10 osd=2 3 2024-02-23T00:49:31.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.663+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_remove -1 5 2024-02-23T00:49:31.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.663+0000 7fa9736f6700 20 client.41459.objecter get_session s=0x7fa96c027a10 osd=2 3 2024-02-23T00:49:31.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.663+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_assign 2 5 2024-02-23T00:49:31.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.663+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c027a10 osd=2 4 2024-02-23T00:49:31.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.663+0000 7fa9736f6700 10 client.41459.objecter _send_command 5 2024-02-23T00:49:31.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.663+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa96c121060 con 0x7fa96c0fcf00 2024-02-23T00:49:31.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.663+0000 7fa970c91700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa96c0fcf00 0x7fa96c11e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:31.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.663+0000 7fa9627fc700 10 client.41459.objecter ms_handle_connect 0x7fa96c0fcf00 2024-02-23T00:49:31.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.663+0000 7fa970c91700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa96c0fcf00 0x7fa96c11e330 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:31.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.664+0000 7fa9627fc700 1 -- 172.21.15.47:0/3268240606 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa96c121060 con 0x7fa96c0fcf00 2024-02-23T00:49:31.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.664+0000 7fa9627fc700 10 client.41459.objecter ms_dispatch 0x7fa96c001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:49:31.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.664+0000 7fa9627fc700 10 client.41459.objecter _finish_command 5 = system:0 2024-02-23T00:49:31.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.664+0000 7fa9627fc700 20 client.41459.objecter put_session s=0x7fa96c027a10 osd=2 4 2024-02-23T00:49:31.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.664+0000 7fa9627fc700 15 client.41459.objecter _session_command_op_remove 2 5 2024-02-23T00:49:31.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.678+0000 7fa9736f6700 10 client.41459.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:49:31.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.678+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_assign -1 6 2024-02-23T00:49:31.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.678+0000 7fa9736f6700 20 client.41459.objecter _get_session s=0x7fa96c027a10 osd=2 3 2024-02-23T00:49:31.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.678+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c027a10 osd=2 3 2024-02-23T00:49:31.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.678+0000 7fa9736f6700 20 client.41459.objecter _get_session s=0x7fa96c027a10 osd=2 3 2024-02-23T00:49:31.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.678+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_remove -1 6 2024-02-23T00:49:31.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.678+0000 7fa9736f6700 20 client.41459.objecter get_session s=0x7fa96c027a10 osd=2 3 2024-02-23T00:49:31.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.678+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_assign 2 6 2024-02-23T00:49:31.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.678+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c027a10 osd=2 4 2024-02-23T00:49:31.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.678+0000 7fa9736f6700 10 client.41459.objecter _send_command 6 2024-02-23T00:49:31.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.678+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa96c0fc460 con 0x7fa96c0fcf00 2024-02-23T00:49:31.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.680+0000 7fa9627fc700 1 -- 172.21.15.47:0/3268240606 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa96c0fc460 con 0x7fa96c0fcf00 2024-02-23T00:49:31.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.680+0000 7fa9627fc700 10 client.41459.objecter ms_dispatch 0x7fa96c001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:49:31.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.680+0000 7fa9627fc700 10 client.41459.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:49:31.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.680+0000 7fa9627fc700 20 client.41459.objecter put_session s=0x7fa96c027a10 osd=2 4 2024-02-23T00:49:31.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.680+0000 7fa9627fc700 15 client.41459.objecter _session_command_op_remove 2 6 2024-02-23T00:49:31.683 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:49:31.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.680+0000 7fa9736f6700 10 client.41459.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:31.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.680+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_assign -1 7 2024-02-23T00:49:31.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.680+0000 7fa9736f6700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fa96c11be80 0x7fa96c11f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:31.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.680+0000 7fa9736f6700 20 client.41459.objecter _get_session s=0x7fa96c0fc810 osd=3 3 2024-02-23T00:49:31.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.680+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c0fc810 osd=3 3 2024-02-23T00:49:31.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.680+0000 7fa9736f6700 20 client.41459.objecter _get_session s=0x7fa96c0fc810 osd=3 3 2024-02-23T00:49:31.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.680+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_remove -1 7 2024-02-23T00:49:31.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.680+0000 7fa9736f6700 20 client.41459.objecter get_session s=0x7fa96c0fc810 osd=3 3 2024-02-23T00:49:31.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.680+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_assign 3 7 2024-02-23T00:49:31.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.680+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c0fc810 osd=3 4 2024-02-23T00:49:31.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.680+0000 7fa9736f6700 10 client.41459.objecter _send_command 7 2024-02-23T00:49:31.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.680+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa96c120030 con 0x7fa96c11be80 2024-02-23T00:49:31.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.680+0000 7fa971c93700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fa96c11be80 0x7fa96c11f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:31.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.680+0000 7fa9627fc700 10 client.41459.objecter ms_handle_connect 0x7fa96c11be80 2024-02-23T00:49:31.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.680+0000 7fa971c93700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fa96c11be80 0x7fa96c11f9a0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:31.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.681+0000 7fa9627fc700 1 -- 172.21.15.47:0/3268240606 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa96c120030 con 0x7fa96c11be80 2024-02-23T00:49:31.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.681+0000 7fa9627fc700 10 client.41459.objecter ms_dispatch 0x7fa96c001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:49:31.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.681+0000 7fa9627fc700 10 client.41459.objecter _finish_command 7 = system:0 2024-02-23T00:49:31.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.681+0000 7fa9627fc700 20 client.41459.objecter put_session s=0x7fa96c0fc810 osd=3 4 2024-02-23T00:49:31.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.681+0000 7fa9627fc700 15 client.41459.objecter _session_command_op_remove 3 7 2024-02-23T00:49:31.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.695+0000 7fa9736f6700 10 client.41459.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:49:31.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.695+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_assign -1 8 2024-02-23T00:49:31.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.695+0000 7fa9736f6700 20 client.41459.objecter _get_session s=0x7fa96c0fc810 osd=3 3 2024-02-23T00:49:31.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.695+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c0fc810 osd=3 3 2024-02-23T00:49:31.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.695+0000 7fa9736f6700 20 client.41459.objecter _get_session s=0x7fa96c0fc810 osd=3 3 2024-02-23T00:49:31.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.695+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_remove -1 8 2024-02-23T00:49:31.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.695+0000 7fa9736f6700 20 client.41459.objecter get_session s=0x7fa96c0fc810 osd=3 3 2024-02-23T00:49:31.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.695+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_assign 3 8 2024-02-23T00:49:31.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.695+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c0fc810 osd=3 4 2024-02-23T00:49:31.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.695+0000 7fa9736f6700 10 client.41459.objecter _send_command 8 2024-02-23T00:49:31.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.695+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa96c11b3e0 con 0x7fa96c11be80 2024-02-23T00:49:31.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.697+0000 7fa9627fc700 1 -- 172.21.15.47:0/3268240606 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa96c11b3e0 con 0x7fa96c11be80 2024-02-23T00:49:31.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.697+0000 7fa9627fc700 10 client.41459.objecter ms_dispatch 0x7fa96c001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:49:31.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.697+0000 7fa9627fc700 10 client.41459.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:49:31.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.697+0000 7fa9627fc700 20 client.41459.objecter put_session s=0x7fa96c0fc810 osd=3 4 2024-02-23T00:49:31.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.697+0000 7fa9627fc700 15 client.41459.objecter _session_command_op_remove 3 8 2024-02-23T00:49:31.700 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:49:31.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.697+0000 7fa9736f6700 10 client.41459.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:31.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.697+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_assign -1 9 2024-02-23T00:49:31.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.697+0000 7fa9736f6700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fa96c1231e0 0x7fa96c1255d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:31.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.697+0000 7fa9736f6700 20 client.41459.objecter _get_session s=0x7fa96c11b1a0 osd=4 3 2024-02-23T00:49:31.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.697+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c11b1a0 osd=4 3 2024-02-23T00:49:31.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.697+0000 7fa9736f6700 20 client.41459.objecter _get_session s=0x7fa96c11b1a0 osd=4 3 2024-02-23T00:49:31.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.697+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_remove -1 9 2024-02-23T00:49:31.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.697+0000 7fa9736f6700 20 client.41459.objecter get_session s=0x7fa96c11b1a0 osd=4 3 2024-02-23T00:49:31.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.697+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_assign 4 9 2024-02-23T00:49:31.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.697+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c11b1a0 osd=4 4 2024-02-23T00:49:31.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.697+0000 7fa9736f6700 10 client.41459.objecter _send_command 9 2024-02-23T00:49:31.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.697+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa96c11b3a0 con 0x7fa96c1231e0 2024-02-23T00:49:31.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.697+0000 7fa971492700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fa96c1231e0 0x7fa96c1255d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:31.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.698+0000 7fa9627fc700 10 client.41459.objecter ms_handle_connect 0x7fa96c1231e0 2024-02-23T00:49:31.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.698+0000 7fa971492700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fa96c1231e0 0x7fa96c1255d0 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:31.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.698+0000 7fa9627fc700 1 -- 172.21.15.47:0/3268240606 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa96c11b3a0 con 0x7fa96c1231e0 2024-02-23T00:49:31.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.698+0000 7fa9627fc700 10 client.41459.objecter ms_dispatch 0x7fa96c001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:49:31.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.698+0000 7fa9627fc700 10 client.41459.objecter _finish_command 9 = system:0 2024-02-23T00:49:31.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.698+0000 7fa9627fc700 20 client.41459.objecter put_session s=0x7fa96c11b1a0 osd=4 4 2024-02-23T00:49:31.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.698+0000 7fa9627fc700 15 client.41459.objecter _session_command_op_remove 4 9 2024-02-23T00:49:31.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.713+0000 7fa9736f6700 10 client.41459.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:49:31.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.713+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_assign -1 10 2024-02-23T00:49:31.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.713+0000 7fa9736f6700 20 client.41459.objecter _get_session s=0x7fa96c11b1a0 osd=4 3 2024-02-23T00:49:31.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.713+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c11b1a0 osd=4 3 2024-02-23T00:49:31.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.713+0000 7fa9736f6700 20 client.41459.objecter _get_session s=0x7fa96c11b1a0 osd=4 3 2024-02-23T00:49:31.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.713+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_remove -1 10 2024-02-23T00:49:31.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.713+0000 7fa9736f6700 20 client.41459.objecter get_session s=0x7fa96c11b1a0 osd=4 3 2024-02-23T00:49:31.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.713+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_assign 4 10 2024-02-23T00:49:31.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.713+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c11b1a0 osd=4 4 2024-02-23T00:49:31.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.713+0000 7fa9736f6700 10 client.41459.objecter _send_command 10 2024-02-23T00:49:31.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.713+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa96c11b3a0 con 0x7fa96c1231e0 2024-02-23T00:49:31.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.715+0000 7fa9627fc700 1 -- 172.21.15.47:0/3268240606 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa96c11b3a0 con 0x7fa96c1231e0 2024-02-23T00:49:31.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.715+0000 7fa9627fc700 10 client.41459.objecter ms_dispatch 0x7fa96c001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:49:31.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.715+0000 7fa9627fc700 10 client.41459.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:49:31.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.715+0000 7fa9627fc700 20 client.41459.objecter put_session s=0x7fa96c11b1a0 osd=4 4 2024-02-23T00:49:31.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.715+0000 7fa9627fc700 15 client.41459.objecter _session_command_op_remove 4 10 2024-02-23T00:49:31.718 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:49:31.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.715+0000 7fa9736f6700 10 client.41459.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:31.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.715+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_assign -1 11 2024-02-23T00:49:31.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.715+0000 7fa9736f6700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fa96c125fd0 0x7fa96c126430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:31.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.715+0000 7fa9736f6700 20 client.41459.objecter _get_session s=0x7fa96c11b3a0 osd=5 3 2024-02-23T00:49:31.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.715+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c11b3a0 osd=5 3 2024-02-23T00:49:31.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.715+0000 7fa9736f6700 20 client.41459.objecter _get_session s=0x7fa96c11b3a0 osd=5 3 2024-02-23T00:49:31.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.715+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_remove -1 11 2024-02-23T00:49:31.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.715+0000 7fa9736f6700 20 client.41459.objecter get_session s=0x7fa96c11b3a0 osd=5 3 2024-02-23T00:49:31.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.715+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_assign 5 11 2024-02-23T00:49:31.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.715+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c11b3a0 osd=5 4 2024-02-23T00:49:31.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.715+0000 7fa9736f6700 10 client.41459.objecter _send_command 11 2024-02-23T00:49:31.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.715+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa96c12b3e0 con 0x7fa96c125fd0 2024-02-23T00:49:31.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.715+0000 7fa970c91700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fa96c125fd0 0x7fa96c126430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:31.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.715+0000 7fa9627fc700 10 client.41459.objecter ms_handle_connect 0x7fa96c125fd0 2024-02-23T00:49:31.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.715+0000 7fa970c91700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fa96c125fd0 0x7fa96c126430 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:31.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.716+0000 7fa9627fc700 1 -- 172.21.15.47:0/3268240606 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa96c12b3e0 con 0x7fa96c125fd0 2024-02-23T00:49:31.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.716+0000 7fa9627fc700 10 client.41459.objecter ms_dispatch 0x7fa96c001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:49:31.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.716+0000 7fa9627fc700 10 client.41459.objecter _finish_command 11 = system:0 2024-02-23T00:49:31.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.716+0000 7fa9627fc700 20 client.41459.objecter put_session s=0x7fa96c11b3a0 osd=5 4 2024-02-23T00:49:31.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.716+0000 7fa9627fc700 15 client.41459.objecter _session_command_op_remove 5 11 2024-02-23T00:49:31.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.731+0000 7fa9736f6700 10 client.41459.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:49:31.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.731+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_assign -1 12 2024-02-23T00:49:31.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.731+0000 7fa9736f6700 20 client.41459.objecter _get_session s=0x7fa96c11b3a0 osd=5 3 2024-02-23T00:49:31.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.731+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c11b3a0 osd=5 3 2024-02-23T00:49:31.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.731+0000 7fa9736f6700 20 client.41459.objecter _get_session s=0x7fa96c11b3a0 osd=5 3 2024-02-23T00:49:31.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.731+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_remove -1 12 2024-02-23T00:49:31.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.731+0000 7fa9736f6700 20 client.41459.objecter get_session s=0x7fa96c11b3a0 osd=5 3 2024-02-23T00:49:31.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.731+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_assign 5 12 2024-02-23T00:49:31.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.731+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c11b3a0 osd=5 4 2024-02-23T00:49:31.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.731+0000 7fa9736f6700 10 client.41459.objecter _send_command 12 2024-02-23T00:49:31.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.731+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa96c126a30 con 0x7fa96c125fd0 2024-02-23T00:49:31.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.733+0000 7fa9627fc700 1 -- 172.21.15.47:0/3268240606 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa96c126a30 con 0x7fa96c125fd0 2024-02-23T00:49:31.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.733+0000 7fa9627fc700 10 client.41459.objecter ms_dispatch 0x7fa96c001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:49:31.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.733+0000 7fa9627fc700 10 client.41459.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:49:31.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.733+0000 7fa9627fc700 20 client.41459.objecter put_session s=0x7fa96c11b3a0 osd=5 4 2024-02-23T00:49:31.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.733+0000 7fa9627fc700 15 client.41459.objecter _session_command_op_remove 5 12 2024-02-23T00:49:31.736 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:49:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.733+0000 7fa9736f6700 10 client.41459.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.733+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_assign -1 13 2024-02-23T00:49:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.733+0000 7fa9736f6700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fa96c127100 0x7fa96c127520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.733+0000 7fa9736f6700 20 client.41459.objecter _get_session s=0x7fa96c126a30 osd=6 3 2024-02-23T00:49:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.733+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c126a30 osd=6 3 2024-02-23T00:49:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.733+0000 7fa9736f6700 20 client.41459.objecter _get_session s=0x7fa96c126a30 osd=6 3 2024-02-23T00:49:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.733+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_remove -1 13 2024-02-23T00:49:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.733+0000 7fa9736f6700 20 client.41459.objecter get_session s=0x7fa96c126a30 osd=6 3 2024-02-23T00:49:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.733+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_assign 6 13 2024-02-23T00:49:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.733+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c126a30 osd=6 4 2024-02-23T00:49:31.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.733+0000 7fa9736f6700 10 client.41459.objecter _send_command 13 2024-02-23T00:49:31.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.733+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa96c127c00 con 0x7fa96c127100 2024-02-23T00:49:31.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.733+0000 7fa971c93700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fa96c127100 0x7fa96c127520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:31.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.734+0000 7fa9627fc700 10 client.41459.objecter ms_handle_connect 0x7fa96c127100 2024-02-23T00:49:31.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.734+0000 7fa971c93700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fa96c127100 0x7fa96c127520 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:31.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.734+0000 7fa9627fc700 1 -- 172.21.15.47:0/3268240606 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa96c127c00 con 0x7fa96c127100 2024-02-23T00:49:31.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.734+0000 7fa9627fc700 10 client.41459.objecter ms_dispatch 0x7fa96c001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:49:31.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.734+0000 7fa9627fc700 10 client.41459.objecter _finish_command 13 = system:0 2024-02-23T00:49:31.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.734+0000 7fa9627fc700 20 client.41459.objecter put_session s=0x7fa96c126a30 osd=6 4 2024-02-23T00:49:31.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.734+0000 7fa9627fc700 15 client.41459.objecter _session_command_op_remove 6 13 2024-02-23T00:49:31.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.749+0000 7fa9736f6700 10 client.41459.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:49:31.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.749+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_assign -1 14 2024-02-23T00:49:31.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.749+0000 7fa9736f6700 20 client.41459.objecter _get_session s=0x7fa96c126a30 osd=6 3 2024-02-23T00:49:31.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.749+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c126a30 osd=6 3 2024-02-23T00:49:31.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.749+0000 7fa9736f6700 20 client.41459.objecter _get_session s=0x7fa96c126a30 osd=6 3 2024-02-23T00:49:31.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.749+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_remove -1 14 2024-02-23T00:49:31.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.749+0000 7fa9736f6700 20 client.41459.objecter get_session s=0x7fa96c126a30 osd=6 3 2024-02-23T00:49:31.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.749+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_assign 6 14 2024-02-23T00:49:31.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.749+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c126a30 osd=6 4 2024-02-23T00:49:31.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.749+0000 7fa9736f6700 10 client.41459.objecter _send_command 14 2024-02-23T00:49:31.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.749+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa96c127c00 con 0x7fa96c127100 2024-02-23T00:49:31.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.750+0000 7fa9627fc700 1 -- 172.21.15.47:0/3268240606 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa96c127c00 con 0x7fa96c127100 2024-02-23T00:49:31.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.750+0000 7fa9627fc700 10 client.41459.objecter ms_dispatch 0x7fa96c001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:49:31.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.750+0000 7fa9627fc700 10 client.41459.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:49:31.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.750+0000 7fa9627fc700 20 client.41459.objecter put_session s=0x7fa96c126a30 osd=6 4 2024-02-23T00:49:31.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.750+0000 7fa9627fc700 15 client.41459.objecter _session_command_op_remove 6 14 2024-02-23T00:49:31.754 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:49:31.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.751+0000 7fa9736f6700 10 client.41459.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:31.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.751+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_assign -1 15 2024-02-23T00:49:31.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.751+0000 7fa9736f6700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fa96c12c780 0x7fa96c12eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:31.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.751+0000 7fa9736f6700 20 client.41459.objecter _get_session s=0x7fa96c127c00 osd=7 3 2024-02-23T00:49:31.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.751+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c127c00 osd=7 3 2024-02-23T00:49:31.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.751+0000 7fa9736f6700 20 client.41459.objecter _get_session s=0x7fa96c127c00 osd=7 3 2024-02-23T00:49:31.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.751+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_remove -1 15 2024-02-23T00:49:31.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.751+0000 7fa9736f6700 20 client.41459.objecter get_session s=0x7fa96c127c00 osd=7 3 2024-02-23T00:49:31.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.751+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_assign 7 15 2024-02-23T00:49:31.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.751+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c127c00 osd=7 4 2024-02-23T00:49:31.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.751+0000 7fa9736f6700 10 client.41459.objecter _send_command 15 2024-02-23T00:49:31.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.751+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa96c131810 con 0x7fa96c12c780 2024-02-23T00:49:31.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.751+0000 7fa971492700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fa96c12c780 0x7fa96c12eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:31.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.751+0000 7fa971492700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fa96c12c780 0x7fa96c12eb70 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:31.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.751+0000 7fa9627fc700 10 client.41459.objecter ms_handle_connect 0x7fa96c12c780 2024-02-23T00:49:31.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.752+0000 7fa9627fc700 1 -- 172.21.15.47:0/3268240606 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa96c131810 con 0x7fa96c12c780 2024-02-23T00:49:31.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.752+0000 7fa9627fc700 10 client.41459.objecter ms_dispatch 0x7fa96c001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:49:31.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.752+0000 7fa9627fc700 10 client.41459.objecter _finish_command 15 = system:0 2024-02-23T00:49:31.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.752+0000 7fa9627fc700 20 client.41459.objecter put_session s=0x7fa96c127c00 osd=7 4 2024-02-23T00:49:31.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.752+0000 7fa9627fc700 15 client.41459.objecter _session_command_op_remove 7 15 2024-02-23T00:49:31.760 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:49:31.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.767+0000 7fa9736f6700 10 client.41459.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:49:31.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.767+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_assign -1 16 2024-02-23T00:49:31.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.767+0000 7fa9736f6700 20 client.41459.objecter _get_session s=0x7fa96c127c00 osd=7 3 2024-02-23T00:49:31.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.767+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c127c00 osd=7 3 2024-02-23T00:49:31.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.767+0000 7fa9736f6700 20 client.41459.objecter _get_session s=0x7fa96c127c00 osd=7 3 2024-02-23T00:49:31.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.767+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_remove -1 16 2024-02-23T00:49:31.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.767+0000 7fa9736f6700 20 client.41459.objecter get_session s=0x7fa96c127c00 osd=7 3 2024-02-23T00:49:31.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.767+0000 7fa9736f6700 15 client.41459.objecter _session_command_op_assign 7 16 2024-02-23T00:49:31.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.767+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c127c00 osd=7 4 2024-02-23T00:49:31.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.767+0000 7fa9736f6700 10 client.41459.objecter _send_command 16 2024-02-23T00:49:31.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.767+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa96c12f060 con 0x7fa96c12c780 2024-02-23T00:49:31.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.768+0000 7fa9627fc700 1 -- 172.21.15.47:0/3268240606 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa96c12f060 con 0x7fa96c12c780 2024-02-23T00:49:31.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.768+0000 7fa9627fc700 10 client.41459.objecter ms_dispatch 0x7fa96c001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:49:31.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.768+0000 7fa9627fc700 10 client.41459.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:49:31.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.768+0000 7fa9627fc700 20 client.41459.objecter put_session s=0x7fa96c127c00 osd=7 4 2024-02-23T00:49:31.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.768+0000 7fa9627fc700 15 client.41459.objecter _session_command_op_remove 7 16 2024-02-23T00:49:31.771 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:49:31.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 10 librados: watch_flush enter 2024-02-23T00:49:31.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 10 librados: watch_flush exit 2024-02-23T00:49:31.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 10 client.41459.objecter close_session for osd.0 2024-02-23T00:49:31.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fa96c028540 msgr2=0x7fa96c0fbd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:31.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fa96c028540 0x7fa96c0fbd30 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:31.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c05ee80 osd=0 1 2024-02-23T00:49:31.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 10 client.41459.objecter close_session for osd.1 2024-02-23T00:49:31.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fa96c027b90 msgr2=0x7fa96c027f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:31.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fa96c027b90 0x7fa96c027f70 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:31.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c0fd3a0 osd=1 1 2024-02-23T00:49:31.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 10 client.41459.objecter close_session for osd.2 2024-02-23T00:49:31.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa96c0fcf00 msgr2=0x7fa96c11e330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:31.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa96c0fcf00 0x7fa96c11e330 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:31.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c027a10 osd=2 1 2024-02-23T00:49:31.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 10 client.41459.objecter close_session for osd.3 2024-02-23T00:49:31.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa971c93700 1 -- 172.21.15.47:0/3268240606 reap_dead start 2024-02-23T00:49:31.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fa96c11be80 msgr2=0x7fa96c11f9a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:31.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fa96c11be80 0x7fa96c11f9a0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:31.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c0fc810 osd=3 1 2024-02-23T00:49:31.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 10 client.41459.objecter close_session for osd.4 2024-02-23T00:49:31.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fa96c1231e0 msgr2=0x7fa96c1255d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:31.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fa96c1231e0 0x7fa96c1255d0 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:31.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c11b1a0 osd=4 1 2024-02-23T00:49:31.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 10 client.41459.objecter close_session for osd.5 2024-02-23T00:49:31.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fa96c125fd0 msgr2=0x7fa96c126430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:31.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fa96c125fd0 0x7fa96c126430 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:31.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c11b3a0 osd=5 1 2024-02-23T00:49:31.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 10 client.41459.objecter close_session for osd.6 2024-02-23T00:49:31.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fa96c127100 msgr2=0x7fa96c127520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:31.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fa96c127100 0x7fa96c127520 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:31.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c126a30 osd=6 1 2024-02-23T00:49:31.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 10 client.41459.objecter close_session for osd.7 2024-02-23T00:49:31.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fa96c12c780 msgr2=0x7fa96c12eb70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:31.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fa96c12c780 0x7fa96c12eb70 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:31.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 20 client.41459.objecter put_session s=0x7fa96c127c00 osd=7 1 2024-02-23T00:49:31.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 20 client.41459.objecter shutdown clearing up homeless session... 2024-02-23T00:49:31.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 10 client.41459.objecter successfully canceled tick 2024-02-23T00:49:31.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa971c93700 1 -- 172.21.15.47:0/3268240606 reap_dead start 2024-02-23T00:49:31.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa958062cc0 msgr2=0x7fa958065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:31.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa958062cc0 0x7fa958065180 secure :-1 s=READY pgs=3085 cs=0 l=1 rev1=1 rx=0x7fa968004ee0 tx=0x7fa96800b040).stop 2024-02-23T00:49:31.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa96c0f9670 msgr2=0x7fa96c0f71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:31.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa96c0f9670 0x7fa96c0f71b0 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 rx=0x7fa95c0048f0 tx=0x7fa95c009cb0).stop 2024-02-23T00:49:31.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 shutdown_connections 2024-02-23T00:49:31.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa96c0f9670 0x7fa96c0f71b0 unknown :-1 s=CLOSED pgs=2692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:31.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 1 --2- 172.21.15.47:0/3268240606 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa958062cc0 0x7fa958065180 unknown :-1 s=CLOSED pgs=3085 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:31.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 >> 172.21.15.47:0/3268240606 conn(0x7fa96c0f2820 msgr2=0x7fa96c0f34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:31.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 shutdown_connections 2024-02-23T00:49:31.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 1 -- 172.21.15.47:0/3268240606 wait complete. 2024-02-23T00:49:31.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:31.769+0000 7fa9736f6700 1 librados: shutdown 2024-02-23T00:49:31.777 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:49:31.777 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:49:31.777 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:49:31.777 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:49:31.777 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:49:31.777 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:49:31.777 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:49:31.777 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:49:31.787 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:31.840 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:49:31.851 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:31.852 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:49:31.953 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:49:31.980 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:32.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.156+0000 7fc1f25ae700 1 -- 172.21.15.47:0/1487397016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1ec0fe510 msgr2=0x7fc1ec0f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:32.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.156+0000 7fc1f25ae700 1 --2- 172.21.15.47:0/1487397016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1ec0fe510 0x7fc1ec0f7d40 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 rx=0x7fc1e00047b0 tx=0x7fc1e0009cb0).stop 2024-02-23T00:49:32.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.156+0000 7fc1f25ae700 1 -- 172.21.15.47:0/1487397016 shutdown_connections 2024-02-23T00:49:32.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.156+0000 7fc1f25ae700 1 --2- 172.21.15.47:0/1487397016 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc1ec0f8420 0x7fc1ec104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:32.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.156+0000 7fc1f25ae700 1 --2- 172.21.15.47:0/1487397016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1ec0fdbb0 0x7fc1ec0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:32.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.156+0000 7fc1f25ae700 1 --2- 172.21.15.47:0/1487397016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1ec0fe510 0x7fc1ec0f7d40 unknown :-1 s=CLOSED pgs=2547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:32.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.156+0000 7fc1f25ae700 1 -- 172.21.15.47:0/1487397016 >> 172.21.15.47:0/1487397016 conn(0x7fc1ec0f3800 msgr2=0x7fc1ec0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:32.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.156+0000 7fc1f25ae700 1 -- 172.21.15.47:0/1487397016 shutdown_connections 2024-02-23T00:49:32.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.156+0000 7fc1f25ae700 1 -- 172.21.15.47:0/1487397016 wait complete. 2024-02-23T00:49:32.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.157+0000 7fc1f25ae700 1 librados: starting msgr at 2024-02-23T00:49:32.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.157+0000 7fc1f25ae700 1 librados: starting objecter 2024-02-23T00:49:32.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.157+0000 7fc1f25ae700 1 Processor -- start 2024-02-23T00:49:32.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.157+0000 7fc1f25ae700 1 -- start start 2024-02-23T00:49:32.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.157+0000 7fc1f25ae700 1 librados: setting wanted keys 2024-02-23T00:49:32.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.157+0000 7fc1f25ae700 1 librados: calling monclient init 2024-02-23T00:49:32.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.157+0000 7fc1f25ae700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1ec0f8420 0x7fc1ec1100c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:32.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.157+0000 7fc1f25ae700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc1ec0fdbb0 0x7fc1ec1105b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:32.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.157+0000 7fc1f25ae700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1ec0fe510 0x7fc1ec10a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:32.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.157+0000 7fc1f25ae700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ec10a750 con 0x7fc1ec0f8420 2024-02-23T00:49:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.157+0000 7fc1f25ae700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ec10a8c0 con 0x7fc1ec0fe510 2024-02-23T00:49:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.157+0000 7fc1f25ae700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ec10aa80 con 0x7fc1ec0fdbb0 2024-02-23T00:49:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.158+0000 7fc1ebfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1ec0f8420 0x7fc1ec1100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.158+0000 7fc1ebfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1ec0f8420 0x7fc1ec1100c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45602/0 (socket says 172.21.15.47:45602) 2024-02-23T00:49:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.158+0000 7fc1eb7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc1ec0fdbb0 0x7fc1ec1105b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.158+0000 7fc1ebfff700 1 -- 172.21.15.47:0/3712106706 learned_addr learned my addr 172.21.15.47:0/3712106706 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:49:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.158+0000 7fc1f0b4b700 1 --2- 172.21.15.47:0/3712106706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1ec0fe510 0x7fc1ec10a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.158+0000 7fc1ebfff700 1 -- 172.21.15.47:0/3712106706 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc1ec0fdbb0 msgr2=0x7fc1ec1105b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:32.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.158+0000 7fc1ebfff700 1 --2- 172.21.15.47:0/3712106706 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc1ec0fdbb0 0x7fc1ec1105b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:32.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.158+0000 7fc1ebfff700 1 -- 172.21.15.47:0/3712106706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1ec0fe510 msgr2=0x7fc1ec10a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:32.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.158+0000 7fc1ebfff700 1 --2- 172.21.15.47:0/3712106706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1ec0fe510 0x7fc1ec10a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:32.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.158+0000 7fc1ebfff700 1 -- 172.21.15.47:0/3712106706 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e40097c0 con 0x7fc1ec0f8420 2024-02-23T00:49:32.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.158+0000 7fc1eb7fe700 1 --2- 172.21.15.47:0/3712106706 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc1ec0fdbb0 0x7fc1ec1105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:49:32.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.158+0000 7fc1ebfff700 1 --2- 172.21.15.47:0/3712106706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1ec0f8420 0x7fc1ec1100c0 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 rx=0x7fc1dc0041d0 tx=0x7fc1dc009f60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:32.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.158+0000 7fc1e97fa700 10 client.?.objecter ms_handle_connect 0x7fc1ec0f8420 2024-02-23T00:49:32.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.158+0000 7fc1e97fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:49:32.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.158+0000 7fc1e97fa700 1 -- 172.21.15.47:0/3712106706 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc007b40 con 0x7fc1ec0f8420 2024-02-23T00:49:32.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.158+0000 7fc1e97fa700 1 -- 172.21.15.47:0/3712106706 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fc1dc00f040 con 0x7fc1ec0f8420 2024-02-23T00:49:32.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.158+0000 7fc1f25ae700 1 -- 172.21.15.47:0/3712106706 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1e0005040 con 0x7fc1ec0f8420 2024-02-23T00:49:32.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.158+0000 7fc1e97fa700 1 -- 172.21.15.47:0/3712106706 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc014730 con 0x7fc1ec0f8420 2024-02-23T00:49:32.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.158+0000 7fc1f25ae700 10 client.31140.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:49:32.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.158+0000 7fc1f25ae700 1 -- 172.21.15.47:0/3712106706 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1ec10b0c0 con 0x7fc1ec0f8420 2024-02-23T00:49:32.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.158+0000 7fc1f25ae700 1 librados: init done 2024-02-23T00:49:32.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.159+0000 7fc1f25ae700 10 client.31140.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:32.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.159+0000 7fc1f25ae700 15 client.31140.objecter _session_command_op_assign -1 1 2024-02-23T00:49:32.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.159+0000 7fc1f25ae700 20 client.31140.objecter _get_session osd=-1 returning homeless 2024-02-23T00:49:32.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.159+0000 7fc1f25ae700 10 client.31140.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:49:32.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.159+0000 7fc1f25ae700 1 -- 172.21.15.47:0/3712106706 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc1ec05ee80 con 0x7fc1ec0f8420 2024-02-23T00:49:32.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.160+0000 7fc1e97fa700 1 -- 172.21.15.47:0/3712106706 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fc1dc014ae0 con 0x7fc1ec0f8420 2024-02-23T00:49:32.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.160+0000 7fc1e97fa700 1 --2- 172.21.15.47:0/3712106706 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc1d4062cc0 0x7fc1d4065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:32.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.160+0000 7fc1e97fa700 1 -- 172.21.15.47:0/3712106706 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(459..459 src has 48..459) v4 ==== 9220+0+0 (secure 0 0 0) 0x7fc1dc07da20 con 0x7fc1ec0f8420 2024-02-23T00:49:32.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.160+0000 7fc1e97fa700 10 client.31140.objecter ms_dispatch 0x7fc1ec001550 osd_map(459..459 src has 48..459) v4 2024-02-23T00:49:32.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.160+0000 7fc1e97fa700 3 client.31140.objecter handle_osd_map got epochs [459,459] > 0 2024-02-23T00:49:32.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.160+0000 7fc1e97fa700 3 client.31140.objecter handle_osd_map decoding full epoch 459 2024-02-23T00:49:32.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.160+0000 7fc1eb7fe700 1 --2- 172.21.15.47:0/3712106706 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc1d4062cc0 0x7fc1d4065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:32.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.160+0000 7fc1e97fa700 10 client.31140.objecter checking command 1 2024-02-23T00:49:32.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.160+0000 7fc1e97fa700 1 --2- 172.21.15.47:0/3712106706 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fc1d406f870 0x7fc1d4071d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.160+0000 7fc1e97fa700 20 client.31140.objecter _get_session s=0x7fc1d406f1a0 osd=5 3 2024-02-23T00:49:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.160+0000 7fc1e97fa700 20 client.31140.objecter put_session s=0x7fc1d406f1a0 osd=5 3 2024-02-23T00:49:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.160+0000 7fc1e97fa700 15 client.31140.objecter _session_command_op_remove -1 1 2024-02-23T00:49:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.160+0000 7fc1e97fa700 20 client.31140.objecter _get_session s=0x7fc1d406f1a0 osd=5 3 2024-02-23T00:49:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.160+0000 7fc1e97fa700 20 client.31140.objecter get_session s=0x7fc1d406f1a0 osd=5 3 2024-02-23T00:49:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.160+0000 7fc1e97fa700 15 client.31140.objecter _session_command_op_assign 5 1 2024-02-23T00:49:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.160+0000 7fc1e97fa700 20 client.31140.objecter put_session s=0x7fc1d406f1a0 osd=5 4 2024-02-23T00:49:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.160+0000 7fc1e97fa700 10 client.31140.objecter _send_command 1 2024-02-23T00:49:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.160+0000 7fc1e97fa700 1 -- 172.21.15.47:0/3712106706 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc1d4072390 con 0x7fc1d406f870 2024-02-23T00:49:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.160+0000 7fc1e97fa700 20 client.31140.objecter dump_active .. 0 homeless 2024-02-23T00:49:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.160+0000 7fc1e97fa700 1 -- 172.21.15.47:0/3712106706 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=459) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1dc07dd90 con 0x7fc1ec0f8420 2024-02-23T00:49:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.160+0000 7fc1e97fa700 10 client.31140.objecter ms_handle_connect 0x7fc1d4062cc0 2024-02-23T00:49:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.160+0000 7fc1eb7fe700 1 --2- 172.21.15.47:0/3712106706 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc1d4062cc0 0x7fc1d4065180 secure :-1 s=READY pgs=3086 cs=0 l=1 rev1=1 rx=0x7fc1e00044a0 tx=0x7fc1e0009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.160+0000 7fc1f0b4b700 1 --2- 172.21.15.47:0/3712106706 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fc1d406f870 0x7fc1d4071d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.161+0000 7fc1e97fa700 10 client.31140.objecter ms_handle_connect 0x7fc1d406f870 2024-02-23T00:49:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.161+0000 7fc1f0b4b700 1 --2- 172.21.15.47:0/3712106706 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fc1d406f870 0x7fc1d4071d10 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:32.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.163+0000 7fc1e97fa700 1 -- 172.21.15.47:0/3712106706 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc1d4072390 con 0x7fc1d406f870 2024-02-23T00:49:32.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.163+0000 7fc1e97fa700 10 client.31140.objecter ms_dispatch 0x7fc1ec001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:49:32.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.163+0000 7fc1e97fa700 10 client.31140.objecter _finish_command 1 = system:0 2024-02-23T00:49:32.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.163+0000 7fc1e97fa700 20 client.31140.objecter put_session s=0x7fc1d406f1a0 osd=5 4 2024-02-23T00:49:32.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.163+0000 7fc1e97fa700 15 client.31140.objecter _session_command_op_remove 5 1 2024-02-23T00:49:32.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.180+0000 7fc1f25ae700 10 client.31140.objecter _submit_command 2 [{"prefix": "reset_purged_snaps_last"}] 2024-02-23T00:49:32.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.180+0000 7fc1f25ae700 15 client.31140.objecter _session_command_op_assign -1 2 2024-02-23T00:49:32.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.180+0000 7fc1f25ae700 20 client.31140.objecter _get_session s=0x7fc1d406f1a0 osd=5 3 2024-02-23T00:49:32.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.180+0000 7fc1f25ae700 20 client.31140.objecter put_session s=0x7fc1d406f1a0 osd=5 3 2024-02-23T00:49:32.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.180+0000 7fc1f25ae700 20 client.31140.objecter _get_session s=0x7fc1d406f1a0 osd=5 3 2024-02-23T00:49:32.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.180+0000 7fc1f25ae700 15 client.31140.objecter _session_command_op_remove -1 2 2024-02-23T00:49:32.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.180+0000 7fc1f25ae700 20 client.31140.objecter get_session s=0x7fc1d406f1a0 osd=5 3 2024-02-23T00:49:32.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.180+0000 7fc1f25ae700 15 client.31140.objecter _session_command_op_assign 5 2 2024-02-23T00:49:32.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.180+0000 7fc1f25ae700 20 client.31140.objecter put_session s=0x7fc1d406f1a0 osd=5 4 2024-02-23T00:49:32.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.180+0000 7fc1f25ae700 10 client.31140.objecter _send_command 2 2024-02-23T00:49:32.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.180+0000 7fc1f25ae700 1 -- 172.21.15.47:0/3712106706 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) v1 -- 0x7fc1ec106ed0 con 0x7fc1d406f870 2024-02-23T00:49:32.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.181+0000 7fc1e97fa700 1 -- 172.21.15.47:0/3712106706 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fc1ec106ed0 con 0x7fc1d406f870 2024-02-23T00:49:32.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.181+0000 7fc1e97fa700 10 client.31140.objecter ms_dispatch 0x7fc1ec001550 command_reply(tid 2: 0 ) v1 2024-02-23T00:49:32.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.181+0000 7fc1e97fa700 10 client.31140.objecter _finish_command 2 = system:0 2024-02-23T00:49:32.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.181+0000 7fc1e97fa700 20 client.31140.objecter put_session s=0x7fc1d406f1a0 osd=5 4 2024-02-23T00:49:32.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.181+0000 7fc1e97fa700 15 client.31140.objecter _session_command_op_remove 5 2 2024-02-23T00:49:32.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.181+0000 7fc1f25ae700 10 librados: watch_flush enter 2024-02-23T00:49:32.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.181+0000 7fc1f25ae700 10 librados: watch_flush exit 2024-02-23T00:49:32.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.181+0000 7fc1f25ae700 10 client.31140.objecter close_session for osd.5 2024-02-23T00:49:32.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.181+0000 7fc1f25ae700 1 -- 172.21.15.47:0/3712106706 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fc1d406f870 msgr2=0x7fc1d4071d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:32.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.181+0000 7fc1f25ae700 1 --2- 172.21.15.47:0/3712106706 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fc1d406f870 0x7fc1d4071d10 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:32.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.181+0000 7fc1f25ae700 20 client.31140.objecter put_session s=0x7fc1d406f1a0 osd=5 1 2024-02-23T00:49:32.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.181+0000 7fc1f25ae700 20 client.31140.objecter shutdown clearing up homeless session... 2024-02-23T00:49:32.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.181+0000 7fc1f25ae700 10 client.31140.objecter successfully canceled tick 2024-02-23T00:49:32.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.181+0000 7fc1f25ae700 1 -- 172.21.15.47:0/3712106706 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc1d4062cc0 msgr2=0x7fc1d4065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:32.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.181+0000 7fc1f25ae700 1 --2- 172.21.15.47:0/3712106706 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc1d4062cc0 0x7fc1d4065180 secure :-1 s=READY pgs=3086 cs=0 l=1 rev1=1 rx=0x7fc1e00044a0 tx=0x7fc1e0009cb0).stop 2024-02-23T00:49:32.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.181+0000 7fc1f25ae700 1 -- 172.21.15.47:0/3712106706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1ec0f8420 msgr2=0x7fc1ec1100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:32.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.181+0000 7fc1f25ae700 1 --2- 172.21.15.47:0/3712106706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1ec0f8420 0x7fc1ec1100c0 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 rx=0x7fc1dc0041d0 tx=0x7fc1dc009f60).stop 2024-02-23T00:49:32.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.181+0000 7fc1f0b4b700 1 -- 172.21.15.47:0/3712106706 reap_dead start 2024-02-23T00:49:32.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.181+0000 7fc1f25ae700 1 -- 172.21.15.47:0/3712106706 shutdown_connections 2024-02-23T00:49:32.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.181+0000 7fc1f25ae700 1 -- 172.21.15.47:0/3712106706 >> 172.21.15.47:0/3712106706 conn(0x7fc1ec0f3800 msgr2=0x7fc1ec06af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:32.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.181+0000 7fc1f25ae700 1 -- 172.21.15.47:0/3712106706 shutdown_connections 2024-02-23T00:49:32.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.181+0000 7fc1f25ae700 1 -- 172.21.15.47:0/3712106706 wait complete. 2024-02-23T00:49:32.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:32.181+0000 7fc1f25ae700 1 librados: shutdown 2024-02-23T00:49:32.229 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:49:32.233 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:32.233 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:49:32.248 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.7 dump_ops_in_flight 2024-02-23T00:49:32.334 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:49:32.364 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:32.422 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:49:32.424 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:32.425 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:49:32.526 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:49:32.554 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:32.564 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:49:32.574 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:49:32.578 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:32.579 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:49:32.679 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:49:32.707 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:32.815 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:49:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:49:32.815 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:49:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:49:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:49:33.066 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:31.308467+0000 mgr.y (mgr.14517) 432 : cluster [DBG] pgmap v684: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 48 KiB/s rd, 383 B/s wr, 81 op/s 2024-02-23T00:49:33.067 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:31.478277+0000 osd.4 (osd.4) 197 : cluster [DBG] 4.d scrub starts 2024-02-23T00:49:33.067 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:31.480244+0000 osd.4 (osd.4) 198 : cluster [DBG] 4.d scrub ok 2024-02-23T00:49:33.067 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:31.623638+0000 mon.c (mon.2) 308 : audit [DBG] from='client.? 172.21.15.47:0/3268240606' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:49:33.067 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:32.287722+0000 mon.a (mon.0) 2670 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:49:33.067 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:32.288081+0000 mon.a (mon.0) 2671 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-69"}]: dispatch 2024-02-23T00:49:33.068 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:49:33.070 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:33.071 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:49:33.172 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:49:33.200 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:33.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:31.308467+0000 mgr.y (mgr.14517) 432 : cluster [DBG] pgmap v684: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 48 KiB/s rd, 383 B/s wr, 81 op/s 2024-02-23T00:49:33.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:31.478277+0000 osd.4 (osd.4) 197 : cluster [DBG] 4.d scrub starts 2024-02-23T00:49:33.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:31.480244+0000 osd.4 (osd.4) 198 : cluster [DBG] 4.d scrub ok 2024-02-23T00:49:33.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:31.623638+0000 mon.c (mon.2) 308 : audit [DBG] from='client.? 172.21.15.47:0/3268240606' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:49:33.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:32.287722+0000 mon.a (mon.0) 2670 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:49:33.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:32.288081+0000 mon.a (mon.0) 2671 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-69"}]: dispatch 2024-02-23T00:49:33.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:31.308467+0000 mgr.y (mgr.14517) 432 : cluster [DBG] pgmap v684: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 48 KiB/s rd, 383 B/s wr, 81 op/s 2024-02-23T00:49:33.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:31.478277+0000 osd.4 (osd.4) 197 : cluster [DBG] 4.d scrub starts 2024-02-23T00:49:33.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:31.480244+0000 osd.4 (osd.4) 198 : cluster [DBG] 4.d scrub ok 2024-02-23T00:49:33.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:31.623638+0000 mon.c (mon.2) 308 : audit [DBG] from='client.? 172.21.15.47:0/3268240606' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:49:33.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:32.287722+0000 mon.a (mon.0) 2670 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:49:33.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:32.288081+0000 mon.a (mon.0) 2671 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-69"}]: dispatch 2024-02-23T00:49:33.401 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:49:33.403 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:33.404 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:49:33.505 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:49:33.536 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:34.019 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph tell osd.6 reset_purged_snaps_last 2024-02-23T00:49:34.045 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:49:34.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:31.767609+0000 osd.0 (osd.0) 193 : cluster [DBG] 3.1 scrub starts 2024-02-23T00:49:34.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:31.768468+0000 osd.0 (osd.0) 194 : cluster [DBG] 3.1 scrub ok 2024-02-23T00:49:34.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:32.389839+0000 mon.a (mon.0) 2672 : cluster [DBG] osdmap e460: 8 total, 8 up, 7 in 2024-02-23T00:49:34.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:32.432821+0000 osd.4 (osd.4) 199 : cluster [DBG] 4.18 scrub starts 2024-02-23T00:49:34.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:32.434274+0000 osd.4 (osd.4) 200 : cluster [DBG] 4.18 scrub ok 2024-02-23T00:49:34.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:32.644443+0000 mgr.y (mgr.14517) 433 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:49:34.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:33.308828+0000 mgr.y (mgr.14517) 434 : cluster [DBG] pgmap v686: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 49 KiB/s rd, 0 B/s wr, 78 op/s 2024-02-23T00:49:34.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:33.392748+0000 mon.a (mon.0) 2673 : cluster [DBG] osdmap e461: 8 total, 8 up, 7 in 2024-02-23T00:49:34.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:33.397087+0000 mon.a (mon.0) 2674 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:49:34.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:31.767609+0000 osd.0 (osd.0) 193 : cluster [DBG] 3.1 scrub starts 2024-02-23T00:49:34.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:31.768468+0000 osd.0 (osd.0) 194 : cluster [DBG] 3.1 scrub ok 2024-02-23T00:49:34.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:32.389839+0000 mon.a (mon.0) 2672 : cluster [DBG] osdmap e460: 8 total, 8 up, 7 in 2024-02-23T00:49:34.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:32.432821+0000 osd.4 (osd.4) 199 : cluster [DBG] 4.18 scrub starts 2024-02-23T00:49:34.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:32.434274+0000 osd.4 (osd.4) 200 : cluster [DBG] 4.18 scrub ok 2024-02-23T00:49:34.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:32.644443+0000 mgr.y (mgr.14517) 433 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:49:34.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:33.308828+0000 mgr.y (mgr.14517) 434 : cluster [DBG] pgmap v686: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 49 KiB/s rd, 0 B/s wr, 78 op/s 2024-02-23T00:49:34.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:33.392748+0000 mon.a (mon.0) 2673 : cluster [DBG] osdmap e461: 8 total, 8 up, 7 in 2024-02-23T00:49:34.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:33.397087+0000 mon.a (mon.0) 2674 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:49:34.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:31.767609+0000 osd.0 (osd.0) 193 : cluster [DBG] 3.1 scrub starts 2024-02-23T00:49:34.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:31.768468+0000 osd.0 (osd.0) 194 : cluster [DBG] 3.1 scrub ok 2024-02-23T00:49:34.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:32.389839+0000 mon.a (mon.0) 2672 : cluster [DBG] osdmap e460: 8 total, 8 up, 7 in 2024-02-23T00:49:34.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:32.432821+0000 osd.4 (osd.4) 199 : cluster [DBG] 4.18 scrub starts 2024-02-23T00:49:34.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:32.434274+0000 osd.4 (osd.4) 200 : cluster [DBG] 4.18 scrub ok 2024-02-23T00:49:34.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:32.644443+0000 mgr.y (mgr.14517) 433 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:49:34.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:33.308828+0000 mgr.y (mgr.14517) 434 : cluster [DBG] pgmap v686: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 49 KiB/s rd, 0 B/s wr, 78 op/s 2024-02-23T00:49:34.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:33.392748+0000 mon.a (mon.0) 2673 : cluster [DBG] osdmap e461: 8 total, 8 up, 7 in 2024-02-23T00:49:34.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:33.397087+0000 mon.a (mon.0) 2674 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:49:34.414 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:49:34.418 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:34.419 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:49:34.472 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:49:34.472 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-02-23T00:49:34.472 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 0 2024-02-23T00:49:34.473 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:49:34.519 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:49:34.547 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:34.600 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:49:34.603 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:34.603 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:49:34.704 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:49:34.733 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:34.967 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:49:34.981 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:49:34.986 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:33.460876+0000 osd.6 (osd.6) 187 : cluster [DBG] 5.11 scrub starts 2024-02-23T00:49:34.986 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:33.461902+0000 osd.6 (osd.6) 188 : cluster [DBG] 5.11 scrub ok 2024-02-23T00:49:34.986 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:33.464737+0000 osd.4 (osd.4) 201 : cluster [DBG] 2.f scrub starts 2024-02-23T00:49:34.987 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:33.465692+0000 osd.4 (osd.4) 202 : cluster [DBG] 2.f scrub ok 2024-02-23T00:49:34.987 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:33.527866+0000 osd.1 (osd.1) 205 : cluster [DBG] 18.18 scrub starts 2024-02-23T00:49:34.987 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:33.529129+0000 osd.1 (osd.1) 206 : cluster [DBG] 18.18 scrub ok 2024-02-23T00:49:34.987 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:33.730387+0000 osd.5 (osd.5) 194 : cluster [DBG] 2.12 scrub starts 2024-02-23T00:49:34.987 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:33.731183+0000 osd.5 (osd.5) 195 : cluster [DBG] 2.12 scrub ok 2024-02-23T00:49:34.987 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:33.835452+0000 osd.4 (osd.4) 203 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:49:34.987 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:33.835569+0000 osd.4 (osd.4) 204 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:49:34.988 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:33.945833+0000 osd.3 (osd.3) 214 : cluster [DBG] 62.15 scrub starts 2024-02-23T00:49:34.988 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:33.948043+0000 osd.3 (osd.3) 215 : cluster [DBG] 62.15 scrub ok 2024-02-23T00:49:34.988 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:34.393910+0000 mon.a (mon.0) 2675 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:49:34.988 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:34.394002+0000 mon.a (mon.0) 2676 : cluster [DBG] osdmap e462: 8 total, 8 up, 7 in 2024-02-23T00:49:34.988 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:34.396313+0000 mon.a (mon.0) 2677 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:49:34.989 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:33.460876+0000 osd.6 (osd.6) 187 : cluster [DBG] 5.11 scrub starts 2024-02-23T00:49:34.989 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:33.461902+0000 osd.6 (osd.6) 188 : cluster [DBG] 5.11 scrub ok 2024-02-23T00:49:34.989 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:33.464737+0000 osd.4 (osd.4) 201 : cluster [DBG] 2.f scrub starts 2024-02-23T00:49:34.989 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:33.465692+0000 osd.4 (osd.4) 202 : cluster [DBG] 2.f scrub ok 2024-02-23T00:49:34.989 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:33.527866+0000 osd.1 (osd.1) 205 : cluster [DBG] 18.18 scrub starts 2024-02-23T00:49:34.990 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:33.529129+0000 osd.1 (osd.1) 206 : cluster [DBG] 18.18 scrub ok 2024-02-23T00:49:34.990 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:33.730387+0000 osd.5 (osd.5) 194 : cluster [DBG] 2.12 scrub starts 2024-02-23T00:49:34.990 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:33.731183+0000 osd.5 (osd.5) 195 : cluster [DBG] 2.12 scrub ok 2024-02-23T00:49:34.990 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:33.835452+0000 osd.4 (osd.4) 203 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:49:34.990 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:33.835569+0000 osd.4 (osd.4) 204 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:49:34.990 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:33.945833+0000 osd.3 (osd.3) 214 : cluster [DBG] 62.15 scrub starts 2024-02-23T00:49:34.990 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:33.948043+0000 osd.3 (osd.3) 215 : cluster [DBG] 62.15 scrub ok 2024-02-23T00:49:34.990 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:34.393910+0000 mon.a (mon.0) 2675 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:49:34.991 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:34.394002+0000 mon.a (mon.0) 2676 : cluster [DBG] osdmap e462: 8 total, 8 up, 7 in 2024-02-23T00:49:34.991 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:34.396313+0000 mon.a (mon.0) 2677 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:49:34.991 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:34.992 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:49:35.072 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:49:35.093 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:49:35.125 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:33.460876+0000 osd.6 (osd.6) 187 : cluster [DBG] 5.11 scrub starts 2024-02-23T00:49:35.125 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:33.461902+0000 osd.6 (osd.6) 188 : cluster [DBG] 5.11 scrub ok 2024-02-23T00:49:35.125 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:33.464737+0000 osd.4 (osd.4) 201 : cluster [DBG] 2.f scrub starts 2024-02-23T00:49:35.125 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:33.465692+0000 osd.4 (osd.4) 202 : cluster [DBG] 2.f scrub ok 2024-02-23T00:49:35.125 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:33.527866+0000 osd.1 (osd.1) 205 : cluster [DBG] 18.18 scrub starts 2024-02-23T00:49:35.125 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:33.529129+0000 osd.1 (osd.1) 206 : cluster [DBG] 18.18 scrub ok 2024-02-23T00:49:35.125 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:33.730387+0000 osd.5 (osd.5) 194 : cluster [DBG] 2.12 scrub starts 2024-02-23T00:49:35.125 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:33.731183+0000 osd.5 (osd.5) 195 : cluster [DBG] 2.12 scrub ok 2024-02-23T00:49:35.126 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:33.835452+0000 osd.4 (osd.4) 203 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:49:35.126 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:33.835569+0000 osd.4 (osd.4) 204 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:49:35.126 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:33.945833+0000 osd.3 (osd.3) 214 : cluster [DBG] 62.15 scrub starts 2024-02-23T00:49:35.126 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:33.948043+0000 osd.3 (osd.3) 215 : cluster [DBG] 62.15 scrub ok 2024-02-23T00:49:35.126 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:34.393910+0000 mon.a (mon.0) 2675 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:49:35.126 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:34.394002+0000 mon.a (mon.0) 2676 : cluster [DBG] osdmap e462: 8 total, 8 up, 7 in 2024-02-23T00:49:35.126 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:34.396313+0000 mon.a (mon.0) 2677 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:49:35.143 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:35.144 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.7 dump_blocked_ops 2024-02-23T00:49:35.291 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:49:35.293 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:35.294 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:49:35.395 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:49:35.418 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:49:35.423 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:36.273 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:49:36.276 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:36.277 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:49:36.303 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:49:36.377 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:49:36.406 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:36.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:34.512548+0000 osd.1 (osd.1) 207 : cluster [DBG] 18.1c scrub starts 2024-02-23T00:49:36.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:34.515492+0000 osd.1 (osd.1) 208 : cluster [DBG] 18.1c scrub ok 2024-02-23T00:49:36.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:34.706344+0000 osd.5 (osd.5) 196 : cluster [DBG] 5.7 scrub starts 2024-02-23T00:49:36.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:34.709248+0000 osd.5 (osd.5) 197 : cluster [DBG] 5.7 scrub ok 2024-02-23T00:49:36.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:35.207132+0000 mon.a (mon.0) 2678 : cluster [DBG] osdmap e463: 8 total, 8 up, 7 in 2024-02-23T00:49:36.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:35.309219+0000 mgr.y (mgr.14517) 435 : cluster [DBG] pgmap v690: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:49:36.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:34.512548+0000 osd.1 (osd.1) 207 : cluster [DBG] 18.1c scrub starts 2024-02-23T00:49:36.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:34.515492+0000 osd.1 (osd.1) 208 : cluster [DBG] 18.1c scrub ok 2024-02-23T00:49:36.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:34.706344+0000 osd.5 (osd.5) 196 : cluster [DBG] 5.7 scrub starts 2024-02-23T00:49:36.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:34.709248+0000 osd.5 (osd.5) 197 : cluster [DBG] 5.7 scrub ok 2024-02-23T00:49:36.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:35.207132+0000 mon.a (mon.0) 2678 : cluster [DBG] osdmap e463: 8 total, 8 up, 7 in 2024-02-23T00:49:36.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:35.309219+0000 mgr.y (mgr.14517) 435 : cluster [DBG] pgmap v690: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:49:36.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:34.512548+0000 osd.1 (osd.1) 207 : cluster [DBG] 18.1c scrub starts 2024-02-23T00:49:36.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:34.515492+0000 osd.1 (osd.1) 208 : cluster [DBG] 18.1c scrub ok 2024-02-23T00:49:36.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:34.706344+0000 osd.5 (osd.5) 196 : cluster [DBG] 5.7 scrub starts 2024-02-23T00:49:36.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:34.709248+0000 osd.5 (osd.5) 197 : cluster [DBG] 5.7 scrub ok 2024-02-23T00:49:36.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:35.207132+0000 mon.a (mon.0) 2678 : cluster [DBG] osdmap e463: 8 total, 8 up, 7 in 2024-02-23T00:49:36.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:35.309219+0000 mgr.y (mgr.14517) 435 : cluster [DBG] pgmap v690: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:49:36.955 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:49:36.958 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:36.959 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:49:37.059 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:49:37.088 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:37.181 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:49:37.182 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-02-23T00:49:37.182 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-02-23T00:49:37.182 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-02-23T00:49:37.182 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:49:37.186 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:49:37.188 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:37.189 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:49:37.290 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:49:37.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:35.753058+0000 osd.5 (osd.5) 198 : cluster [DBG] 6.7 scrub starts 2024-02-23T00:49:37.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:35.756066+0000 osd.5 (osd.5) 199 : cluster [DBG] 6.7 scrub ok 2024-02-23T00:49:37.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:35.876970+0000 osd.3 (osd.3) 216 : cluster [DBG] 2.1 scrub starts 2024-02-23T00:49:37.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:35.877796+0000 osd.3 (osd.3) 217 : cluster [DBG] 2.1 scrub ok 2024-02-23T00:49:37.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:36.216439+0000 mon.a (mon.0) 2679 : cluster [DBG] osdmap e464: 8 total, 8 up, 7 in 2024-02-23T00:49:37.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:37.058400+0000 mon.a (mon.0) 2680 : audit [INF] from='client.? 172.21.15.47:0/2025082108' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:49:37.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:35.753058+0000 osd.5 (osd.5) 198 : cluster [DBG] 6.7 scrub starts 2024-02-23T00:49:37.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:35.756066+0000 osd.5 (osd.5) 199 : cluster [DBG] 6.7 scrub ok 2024-02-23T00:49:37.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:35.876970+0000 osd.3 (osd.3) 216 : cluster [DBG] 2.1 scrub starts 2024-02-23T00:49:37.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:35.877796+0000 osd.3 (osd.3) 217 : cluster [DBG] 2.1 scrub ok 2024-02-23T00:49:37.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:36.216439+0000 mon.a (mon.0) 2679 : cluster [DBG] osdmap e464: 8 total, 8 up, 7 in 2024-02-23T00:49:37.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:37.058400+0000 mon.a (mon.0) 2680 : audit [INF] from='client.? 172.21.15.47:0/2025082108' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:49:37.321 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:37.575 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:35.753058+0000 osd.5 (osd.5) 198 : cluster [DBG] 6.7 scrub starts 2024-02-23T00:49:37.575 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:35.756066+0000 osd.5 (osd.5) 199 : cluster [DBG] 6.7 scrub ok 2024-02-23T00:49:37.575 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:35.876970+0000 osd.3 (osd.3) 216 : cluster [DBG] 2.1 scrub starts 2024-02-23T00:49:37.575 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:35.877796+0000 osd.3 (osd.3) 217 : cluster [DBG] 2.1 scrub ok 2024-02-23T00:49:37.575 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:36.216439+0000 mon.a (mon.0) 2679 : cluster [DBG] osdmap e464: 8 total, 8 up, 7 in 2024-02-23T00:49:37.576 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:37.058400+0000 mon.a (mon.0) 2680 : audit [INF] from='client.? 172.21.15.47:0/2025082108' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:49:37.871 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:49:37.874 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:37.874 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:49:37.924 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.7 dump_historic_ops 2024-02-23T00:49:37.975 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:49:38.003 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:38.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.864+0000 7f9abe274700 1 -- 172.21.15.47:0/1136017785 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9ab80fd510 msgr2=0x7f9ab80f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:38.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.864+0000 7f9abe274700 1 --2- 172.21.15.47:0/1136017785 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9ab80fd510 0x7f9ab80f6d40 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 rx=0x7f9aac004870 tx=0x7f9aac009cb0).stop 2024-02-23T00:49:38.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.865+0000 7f9abe274700 1 -- 172.21.15.47:0/1136017785 shutdown_connections 2024-02-23T00:49:38.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.865+0000 7f9abe274700 1 --2- 172.21.15.47:0/1136017785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ab80f7400 0x7f9ab8103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.865+0000 7f9abe274700 1 --2- 172.21.15.47:0/1136017785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ab80fcbb0 0x7f9ab80fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.865+0000 7f9abe274700 1 --2- 172.21.15.47:0/1136017785 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9ab80fd510 0x7f9ab80f6d40 unknown :-1 s=CLOSED pgs=2693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.865+0000 7f9abe274700 1 -- 172.21.15.47:0/1136017785 >> 172.21.15.47:0/1136017785 conn(0x7f9ab80f2820 msgr2=0x7f9ab80f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.865+0000 7f9abe274700 1 -- 172.21.15.47:0/1136017785 shutdown_connections 2024-02-23T00:49:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.865+0000 7f9abe274700 1 -- 172.21.15.47:0/1136017785 wait complete. 2024-02-23T00:49:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.865+0000 7f9abe274700 1 librados: starting msgr at 2024-02-23T00:49:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.865+0000 7f9abe274700 1 librados: starting objecter 2024-02-23T00:49:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.865+0000 7f9abe274700 1 Processor -- start 2024-02-23T00:49:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.866+0000 7f9abe274700 1 -- start start 2024-02-23T00:49:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.866+0000 7f9abe274700 1 librados: setting wanted keys 2024-02-23T00:49:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.866+0000 7f9abe274700 1 librados: calling monclient init 2024-02-23T00:49:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.866+0000 7f9abe274700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9ab80f7400 0x7f9ab8110220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.866+0000 7f9abe274700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ab80fcbb0 0x7f9ab8110710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.866+0000 7f9abe274700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ab80fd510 0x7f9ab810a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:38.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.866+0000 7f9abe274700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab810a8b0 con 0x7f9ab80fd510 2024-02-23T00:49:38.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.866+0000 7f9abe274700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab810aa20 con 0x7f9ab80fcbb0 2024-02-23T00:49:38.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.866+0000 7f9abe274700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab810abe0 con 0x7f9ab80f7400 2024-02-23T00:49:38.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.866+0000 7f9abc811700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ab80fd510 0x7f9ab810a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:38.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.866+0000 7f9ab7fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9ab80f7400 0x7f9ab8110220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:38.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.866+0000 7f9ab7fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9ab80f7400 0x7f9ab8110220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:50976/0 (socket says 172.21.15.47:50976) 2024-02-23T00:49:38.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.866+0000 7f9ab7fff700 1 -- 172.21.15.47:0/2025082108 learned_addr learned my addr 172.21.15.47:0/2025082108 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:49:38.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.866+0000 7f9ab77fe700 1 --2- 172.21.15.47:0/2025082108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ab80fcbb0 0x7f9ab8110710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:38.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.866+0000 7f9abc811700 1 -- 172.21.15.47:0/2025082108 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9ab80f7400 msgr2=0x7f9ab8110220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:38.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.866+0000 7f9abc811700 1 --2- 172.21.15.47:0/2025082108 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9ab80f7400 0x7f9ab8110220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:38.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.866+0000 7f9abc811700 1 -- 172.21.15.47:0/2025082108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ab80fcbb0 msgr2=0x7f9ab8110710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:38.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.866+0000 7f9abc811700 1 --2- 172.21.15.47:0/2025082108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ab80fcbb0 0x7f9ab8110710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:38.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.867+0000 7f9abc811700 1 -- 172.21.15.47:0/2025082108 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab00097c0 con 0x7f9ab80fd510 2024-02-23T00:49:38.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.867+0000 7f9ab7fff700 1 --2- 172.21.15.47:0/2025082108 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9ab80f7400 0x7f9ab8110220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:49:38.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.867+0000 7f9ab77fe700 1 --2- 172.21.15.47:0/2025082108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ab80fcbb0 0x7f9ab8110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:49:38.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.867+0000 7f9abc811700 1 --2- 172.21.15.47:0/2025082108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ab80fd510 0x7f9ab810a3c0 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 rx=0x7f9ab0001cb0 tx=0x7f9ab0004c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:38.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.867+0000 7f9ab5775700 10 client.?.objecter ms_handle_connect 0x7f9ab80fd510 2024-02-23T00:49:38.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.867+0000 7f9ab5775700 10 client.?.objecter resend_mon_ops 2024-02-23T00:49:38.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.867+0000 7f9ab5775700 1 -- 172.21.15.47:0/2025082108 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab000fa30 con 0x7f9ab80fd510 2024-02-23T00:49:38.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.867+0000 7f9abe274700 1 -- 172.21.15.47:0/2025082108 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aac005040 con 0x7f9ab80fd510 2024-02-23T00:49:38.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.867+0000 7f9ab5775700 1 -- 172.21.15.47:0/2025082108 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f9ab00090d0 con 0x7f9ab80fd510 2024-02-23T00:49:38.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.867+0000 7f9ab5775700 1 -- 172.21.15.47:0/2025082108 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0007b40 con 0x7f9ab80fd510 2024-02-23T00:49:38.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.867+0000 7f9abe274700 10 client.31146.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:49:38.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.867+0000 7f9abe274700 1 -- 172.21.15.47:0/2025082108 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab810b1d0 con 0x7f9ab80fd510 2024-02-23T00:49:38.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.867+0000 7f9abe274700 1 librados: init done 2024-02-23T00:49:38.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.868+0000 7f9abe274700 1 -- 172.21.15.47:0/2025082108 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ab805fa60 con 0x7f9ab80fd510 2024-02-23T00:49:38.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.868+0000 7f9ab5775700 1 -- 172.21.15.47:0/2025082108 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f9ab0009240 con 0x7f9ab80fd510 2024-02-23T00:49:38.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.869+0000 7f9ab5775700 1 --2- 172.21.15.47:0/2025082108 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9aa0062cc0 0x7f9aa0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:38.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.869+0000 7f9ab5775700 1 -- 172.21.15.47:0/2025082108 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(464..464 src has 48..464) v4 ==== 9220+0+0 (secure 0 0 0) 0x7f9ab007dfa0 con 0x7f9ab80fd510 2024-02-23T00:49:38.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.869+0000 7f9ab5775700 10 client.31146.objecter ms_dispatch 0x7f9ab8001550 osd_map(464..464 src has 48..464) v4 2024-02-23T00:49:38.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.869+0000 7f9ab5775700 3 client.31146.objecter handle_osd_map got epochs [464,464] > 0 2024-02-23T00:49:38.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.869+0000 7f9ab5775700 3 client.31146.objecter handle_osd_map decoding full epoch 464 2024-02-23T00:49:38.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.869+0000 7f9ab7fff700 1 --2- 172.21.15.47:0/2025082108 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9aa0062cc0 0x7f9aa0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:38.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.869+0000 7f9ab5775700 20 client.31146.objecter dump_active .. 0 homeless 2024-02-23T00:49:38.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.869+0000 7f9ab5775700 10 client.31146.objecter ms_handle_connect 0x7f9aa0062cc0 2024-02-23T00:49:38.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.869+0000 7f9ab7fff700 1 --2- 172.21.15.47:0/2025082108 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9aa0062cc0 0x7f9aa0065180 secure :-1 s=READY pgs=3087 cs=0 l=1 rev1=1 rx=0x7f9aa8004fd0 tx=0x7f9aa800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:38.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:36.873+0000 7f9ab5775700 1 -- 172.21.15.47:0/2025082108 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f9ab004d5b0 con 0x7f9ab80fd510 2024-02-23T00:49:38.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:37.056+0000 7f9abe274700 1 -- 172.21.15.47:0/2025082108 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7f9ab8105f10 con 0x7f9ab80fd510 2024-02-23T00:49:38.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:37.217+0000 7f9ab5775700 1 -- 172.21.15.47:0/2025082108 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v465) v1 ==== 101+0+0 (secure 0 0 0) 0x7f9ab0050bd0 con 0x7f9ab80fd510 2024-02-23T00:49:38.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:37.220+0000 7f9abe274700 10 librados: watch_flush enter 2024-02-23T00:49:38.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:37.220+0000 7f9abe274700 10 librados: watch_flush exit 2024-02-23T00:49:38.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:37.220+0000 7f9abe274700 20 client.31146.objecter shutdown clearing up homeless session... 2024-02-23T00:49:38.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:37.220+0000 7f9abe274700 10 client.31146.objecter successfully canceled tick 2024-02-23T00:49:38.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:37.220+0000 7f9abe274700 1 -- 172.21.15.47:0/2025082108 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9aa0062cc0 msgr2=0x7f9aa0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:38.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:37.220+0000 7f9abe274700 1 --2- 172.21.15.47:0/2025082108 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9aa0062cc0 0x7f9aa0065180 secure :-1 s=READY pgs=3087 cs=0 l=1 rev1=1 rx=0x7f9aa8004fd0 tx=0x7f9aa800b040).stop 2024-02-23T00:49:38.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:37.220+0000 7f9abe274700 1 -- 172.21.15.47:0/2025082108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ab80fd510 msgr2=0x7f9ab810a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:38.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:37.220+0000 7f9abe274700 1 --2- 172.21.15.47:0/2025082108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ab80fd510 0x7f9ab810a3c0 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 rx=0x7f9ab0001cb0 tx=0x7f9ab0004c50).stop 2024-02-23T00:49:38.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:37.222+0000 7f9abe274700 1 -- 172.21.15.47:0/2025082108 shutdown_connections 2024-02-23T00:49:38.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:37.222+0000 7f9abe274700 1 --2- 172.21.15.47:0/2025082108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ab80fd510 0x7f9ab810a3c0 unknown :-1 s=CLOSED pgs=2549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:38.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:37.222+0000 7f9abe274700 1 --2- 172.21.15.47:0/2025082108 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9aa0062cc0 0x7f9aa0065180 unknown :-1 s=CLOSED pgs=3087 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:38.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:37.222+0000 7f9abe274700 1 --2- 172.21.15.47:0/2025082108 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9ab80f7400 0x7f9ab8110220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:38.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:37.222+0000 7f9abe274700 1 --2- 172.21.15.47:0/2025082108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ab80fcbb0 0x7f9ab8110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:38.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:37.222+0000 7f9abe274700 1 -- 172.21.15.47:0/2025082108 >> 172.21.15.47:0/2025082108 conn(0x7f9ab80f2820 msgr2=0x7f9ab8060db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:38.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:37.222+0000 7f9abe274700 1 -- 172.21.15.47:0/2025082108 shutdown_connections 2024-02-23T00:49:38.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:37.222+0000 7f9abe274700 1 -- 172.21.15.47:0/2025082108 wait complete. 2024-02-23T00:49:38.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:37.222+0000 7f9abe274700 1 librados: shutdown 2024-02-23T00:49:38.036 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is unset 2024-02-23T00:49:38.190 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:49:38.202 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:49:38.204 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:38.205 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:49:38.306 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:49:38.333 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:38.465 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:36.821681+0000 osd.0 (osd.0) 195 : cluster [DBG] 4.16 scrub starts 2024-02-23T00:49:38.465 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:36.823205+0000 osd.0 (osd.0) 196 : cluster [DBG] 4.16 scrub ok 2024-02-23T00:49:38.465 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:36.866565+0000 osd.3 (osd.3) 218 : cluster [DBG] 6.6 scrub starts 2024-02-23T00:49:38.465 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:36.867479+0000 osd.3 (osd.3) 219 : cluster [DBG] 6.6 scrub ok 2024-02-23T00:49:38.466 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:37.214222+0000 mon.a (mon.0) 2681 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:49:38.466 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:37.218326+0000 mon.a (mon.0) 2682 : audit [INF] from='client.? 172.21.15.47:0/2025082108' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:49:38.466 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:37.218389+0000 mon.a (mon.0) 2683 : cluster [DBG] osdmap e465: 8 total, 8 up, 7 in 2024-02-23T00:49:38.466 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:37.309818+0000 mgr.y (mgr.14517) 436 : cluster [DBG] pgmap v693: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.6 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-02-23T00:49:38.466 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:37.328306+0000 osd.7 (osd.7) 227 : cluster [DBG] 2.8 scrub starts 2024-02-23T00:49:38.466 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:37.331266+0000 osd.7 (osd.7) 228 : cluster [DBG] 2.8 scrub ok 2024-02-23T00:49:38.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:36.821681+0000 osd.0 (osd.0) 195 : cluster [DBG] 4.16 scrub starts 2024-02-23T00:49:38.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:36.823205+0000 osd.0 (osd.0) 196 : cluster [DBG] 4.16 scrub ok 2024-02-23T00:49:38.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:36.866565+0000 osd.3 (osd.3) 218 : cluster [DBG] 6.6 scrub starts 2024-02-23T00:49:38.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:36.867479+0000 osd.3 (osd.3) 219 : cluster [DBG] 6.6 scrub ok 2024-02-23T00:49:38.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:37.214222+0000 mon.a (mon.0) 2681 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:49:38.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:37.218326+0000 mon.a (mon.0) 2682 : audit [INF] from='client.? 172.21.15.47:0/2025082108' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:49:38.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:37.218389+0000 mon.a (mon.0) 2683 : cluster [DBG] osdmap e465: 8 total, 8 up, 7 in 2024-02-23T00:49:38.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:37.309818+0000 mgr.y (mgr.14517) 436 : cluster [DBG] pgmap v693: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.6 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-02-23T00:49:38.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:37.328306+0000 osd.7 (osd.7) 227 : cluster [DBG] 2.8 scrub starts 2024-02-23T00:49:38.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:37.331266+0000 osd.7 (osd.7) 228 : cluster [DBG] 2.8 scrub ok 2024-02-23T00:49:38.660 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:36.821681+0000 osd.0 (osd.0) 195 : cluster [DBG] 4.16 scrub starts 2024-02-23T00:49:38.660 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:36.823205+0000 osd.0 (osd.0) 196 : cluster [DBG] 4.16 scrub ok 2024-02-23T00:49:38.660 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:36.866565+0000 osd.3 (osd.3) 218 : cluster [DBG] 6.6 scrub starts 2024-02-23T00:49:38.660 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:36.867479+0000 osd.3 (osd.3) 219 : cluster [DBG] 6.6 scrub ok 2024-02-23T00:49:38.661 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:37.214222+0000 mon.a (mon.0) 2681 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:49:38.661 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:37.218326+0000 mon.a (mon.0) 2682 : audit [INF] from='client.? 172.21.15.47:0/2025082108' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:49:38.661 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:37.218389+0000 mon.a (mon.0) 2683 : cluster [DBG] osdmap e465: 8 total, 8 up, 7 in 2024-02-23T00:49:38.661 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:37.309818+0000 mgr.y (mgr.14517) 436 : cluster [DBG] pgmap v693: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.6 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-02-23T00:49:38.661 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:37.328306+0000 osd.7 (osd.7) 227 : cluster [DBG] 2.8 scrub starts 2024-02-23T00:49:38.661 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:37.331266+0000 osd.7 (osd.7) 228 : cluster [DBG] 2.8 scrub ok 2024-02-23T00:49:38.983 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:49:38.985 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:38.986 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:49:39.087 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:49:39.117 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:39.404 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:37.534303+0000 osd.1 (osd.1) 209 : cluster [DBG] 5.19 scrub starts 2024-02-23T00:49:39.404 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:37.537207+0000 osd.1 (osd.1) 210 : cluster [DBG] 5.19 scrub ok 2024-02-23T00:49:39.404 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:37.718640+0000 osd.5 (osd.5) 200 : cluster [DBG] 5.16 scrub starts 2024-02-23T00:49:39.405 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:37.721559+0000 osd.5 (osd.5) 201 : cluster [DBG] 5.16 scrub ok 2024-02-23T00:49:39.405 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:37.786274+0000 osd.0 (osd.0) 197 : cluster [DBG] 5.5 scrub starts 2024-02-23T00:49:39.405 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:37.786974+0000 osd.0 (osd.0) 198 : cluster [DBG] 5.5 scrub ok 2024-02-23T00:49:39.405 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:37.880176+0000 osd.3 (osd.3) 220 : cluster [DBG] 5.17 scrub starts 2024-02-23T00:49:39.405 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:37.881179+0000 osd.3 (osd.3) 221 : cluster [DBG] 5.17 scrub ok 2024-02-23T00:49:39.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:37.534303+0000 osd.1 (osd.1) 209 : cluster [DBG] 5.19 scrub starts 2024-02-23T00:49:39.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:37.537207+0000 osd.1 (osd.1) 210 : cluster [DBG] 5.19 scrub ok 2024-02-23T00:49:39.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:37.718640+0000 osd.5 (osd.5) 200 : cluster [DBG] 5.16 scrub starts 2024-02-23T00:49:39.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:37.721559+0000 osd.5 (osd.5) 201 : cluster [DBG] 5.16 scrub ok 2024-02-23T00:49:39.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:37.786274+0000 osd.0 (osd.0) 197 : cluster [DBG] 5.5 scrub starts 2024-02-23T00:49:39.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:37.786974+0000 osd.0 (osd.0) 198 : cluster [DBG] 5.5 scrub ok 2024-02-23T00:49:39.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:37.880176+0000 osd.3 (osd.3) 220 : cluster [DBG] 5.17 scrub starts 2024-02-23T00:49:39.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:37.881179+0000 osd.3 (osd.3) 221 : cluster [DBG] 5.17 scrub ok 2024-02-23T00:49:39.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:37.534303+0000 osd.1 (osd.1) 209 : cluster [DBG] 5.19 scrub starts 2024-02-23T00:49:39.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:37.537207+0000 osd.1 (osd.1) 210 : cluster [DBG] 5.19 scrub ok 2024-02-23T00:49:39.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:37.718640+0000 osd.5 (osd.5) 200 : cluster [DBG] 5.16 scrub starts 2024-02-23T00:49:39.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:37.721559+0000 osd.5 (osd.5) 201 : cluster [DBG] 5.16 scrub ok 2024-02-23T00:49:39.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:37.786274+0000 osd.0 (osd.0) 197 : cluster [DBG] 5.5 scrub starts 2024-02-23T00:49:39.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:37.786974+0000 osd.0 (osd.0) 198 : cluster [DBG] 5.5 scrub ok 2024-02-23T00:49:39.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:37.880176+0000 osd.3 (osd.3) 220 : cluster [DBG] 5.17 scrub starts 2024-02-23T00:49:39.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:37.881179+0000 osd.3 (osd.3) 221 : cluster [DBG] 5.17 scrub ok 2024-02-23T00:49:39.949 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:49:39.949 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-02-23T00:49:39.949 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-02-23T00:49:39.949 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-02-23T00:49:39.949 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.949 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(osd.3.0:1295 20.1e e112/109 -22:7b6591d8:::temp_20.1e_0_14313_1:head v 0'0, mlcod=112'1002)", 2024-02-23T00:49:39.949 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:01.568631+0000", 2024-02-23T00:49:39.949 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 398.37601441300001, 2024-02-23T00:49:39.950 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034138363999999997, 2024-02-23T00:49:39.950 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:49:39.950 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:49:39.950 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:49:39.950 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.950 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:49:39.950 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.568631+0000", 2024-02-23T00:49:39.950 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:49:39.950 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.950 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.950 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:49:39.950 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.568631+0000", 2024-02-23T00:49:39.951 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999804 2024-02-23T00:49:39.951 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.951 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.951 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:49:39.951 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.568612+0000", 2024-02-23T00:49:39.951 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.020189048000000001 2024-02-23T00:49:39.951 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.951 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.951 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:49:39.951 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.588801+0000", 2024-02-23T00:49:39.951 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1197000000000001e-05 2024-02-23T00:49:39.951 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.951 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.952 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:49:39.952 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.588822+0000", 2024-02-23T00:49:39.952 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4791999999999999e-05 2024-02-23T00:49:39.952 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.952 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.952 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:49:39.952 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.588847+0000", 2024-02-23T00:49:39.952 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9911999999999998e-05 2024-02-23T00:49:39.952 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.952 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.952 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:49:39.952 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.588906+0000", 2024-02-23T00:49:39.953 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5961000000000001e-05 2024-02-23T00:49:39.953 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.953 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.953 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:49:39.953 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.588932+0000", 2024-02-23T00:49:39.953 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01379406 2024-02-23T00:49:39.953 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.953 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.953 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:49:39.953 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.602726+0000", 2024-02-23T00:49:39.954 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3027e-05 2024-02-23T00:49:39.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.954 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.954 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:49:39.954 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.602769+0000", 2024-02-23T00:49:39.954 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034138363999999997 2024-02-23T00:49:39.954 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:39.954 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:49:39.954 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:39.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.954 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.955 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.24775.0:2069 20.13 e112/109 20:cde74feb:test-rados-api-smithi047-82457-17::big:head v 112'1, mlcod=0'0)", 2024-02-23T00:49:39.955 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:01.712338+0000", 2024-02-23T00:49:39.955 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 398.232307327, 2024-02-23T00:49:39.955 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.057972005, 2024-02-23T00:49:39.955 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:49:39.955 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:49:39.955 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:49:39.955 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.956 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:49:39.956 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.712338+0000", 2024-02-23T00:49:39.956 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:49:39.956 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.956 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.956 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:49:39.956 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.712338+0000", 2024-02-23T00:49:39.956 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:49:39.956 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.956 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.956 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:49:39.956 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.712336+0000", 2024-02-23T00:49:39.957 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012853547 2024-02-23T00:49:39.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.957 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.957 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:49:39.957 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.725190+0000", 2024-02-23T00:49:39.957 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.3782999999999998e-05 2024-02-23T00:49:39.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.957 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.957 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:49:39.957 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.725224+0000", 2024-02-23T00:49:39.957 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3610000000000001e-06 2024-02-23T00:49:39.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.957 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.957 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:49:39.958 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.725232+0000", 2024-02-23T00:49:39.958 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5358000000000002e-05 2024-02-23T00:49:39.958 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.958 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.958 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:49:39.958 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.725277+0000", 2024-02-23T00:49:39.958 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.6744999999999999e-05 2024-02-23T00:49:39.958 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.958 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.958 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:49:39.958 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.725304+0000", 2024-02-23T00:49:39.958 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.04497164 2024-02-23T00:49:39.958 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.958 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.959 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:49:39.959 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.770276+0000", 2024-02-23T00:49:39.959 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4561000000000003e-05 2024-02-23T00:49:39.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.959 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.959 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:49:39.959 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.770310+0000", 2024-02-23T00:49:39.959 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.057972005 2024-02-23T00:49:39.959 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:39.959 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:49:39.959 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:39.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.959 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.959 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.24775.0:2070 20.6 e112/109 20:63faae7e:test-rados-api-smithi047-82457-17::big2:head v 112'4, mlcod=110'2)", 2024-02-23T00:49:39.959 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:01.807537+0000", 2024-02-23T00:49:39.960 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 398.13710896200001, 2024-02-23T00:49:39.960 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.085855267999999998, 2024-02-23T00:49:39.960 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:49:39.960 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:49:39.960 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:49:39.960 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.960 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:49:39.960 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.807537+0000", 2024-02-23T00:49:39.960 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:49:39.960 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.960 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.960 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:49:39.960 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.807537+0000", 2024-02-23T00:49:39.960 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999976 2024-02-23T00:49:39.961 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.961 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.961 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:49:39.961 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.807534+0000", 2024-02-23T00:49:39.961 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.057621555999999997 2024-02-23T00:49:39.961 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.961 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.961 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:49:39.961 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.865156+0000", 2024-02-23T00:49:39.961 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1656999999999999e-05 2024-02-23T00:49:39.961 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.961 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.961 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:49:39.961 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.865178+0000", 2024-02-23T00:49:39.962 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0491e-05 2024-02-23T00:49:39.962 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.962 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.962 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:49:39.962 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.865188+0000", 2024-02-23T00:49:39.962 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2298999999999993e-05 2024-02-23T00:49:39.962 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.962 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.962 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:49:39.962 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.865260+0000", 2024-02-23T00:49:39.962 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7316000000000001e-05 2024-02-23T00:49:39.962 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.962 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.963 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:49:39.963 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.865288+0000", 2024-02-23T00:49:39.963 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028062336 2024-02-23T00:49:39.963 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.963 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.963 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:49:39.963 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.893350+0000", 2024-02-23T00:49:39.963 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1810000000000001e-05 2024-02-23T00:49:39.963 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.963 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.964 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:49:39.964 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.893392+0000", 2024-02-23T00:49:39.964 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.085855267999999998 2024-02-23T00:49:39.964 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:39.964 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:49:39.964 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:39.964 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.964 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.964 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.24775.0:2074 20.1a e112/109 20:5de0a470:test-rados-api-smithi047-82457-17::big3:head v 112'3, mlcod=112'1)", 2024-02-23T00:49:39.965 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:01.957292+0000", 2024-02-23T00:49:39.965 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 397.98735401800002, 2024-02-23T00:49:39.965 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.060300126000000003, 2024-02-23T00:49:39.965 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:49:39.965 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:49:39.965 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:49:39.966 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.966 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:49:39.966 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.957292+0000", 2024-02-23T00:49:39.966 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:49:39.966 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.966 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.966 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:49:39.966 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.957292+0000", 2024-02-23T00:49:39.966 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:49:39.966 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.966 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.966 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:49:39.966 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.957291+0000", 2024-02-23T00:49:39.967 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0093502140000000008 2024-02-23T00:49:39.967 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.967 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.967 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:49:39.967 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.966641+0000", 2024-02-23T00:49:39.967 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0536999999999999e-05 2024-02-23T00:49:39.967 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.967 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.967 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:49:39.967 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.966652+0000", 2024-02-23T00:49:39.967 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9869999999999996e-06 2024-02-23T00:49:39.967 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.967 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.968 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:49:39.968 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.966658+0000", 2024-02-23T00:49:39.968 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1517999999999998e-05 2024-02-23T00:49:39.968 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.968 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.968 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:49:39.968 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.966699+0000", 2024-02-23T00:49:39.968 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.3855000000000001e-05 2024-02-23T00:49:39.968 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.968 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.968 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:49:39.968 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.966723+0000", 2024-02-23T00:49:39.968 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.050836219000000002 2024-02-23T00:49:39.969 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.969 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.969 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:49:39.969 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.017559+0000", 2024-02-23T00:49:39.969 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.2558999999999998e-05 2024-02-23T00:49:39.969 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.969 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.969 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:49:39.969 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.017592+0000", 2024-02-23T00:49:39.969 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.060300126000000003 2024-02-23T00:49:39.969 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:39.969 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:49:39.969 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:39.970 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.970 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.970 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.15144.0:2 38.1a 38:5c6776db:::0.obj:head [write 0~103] snapc 0=[] ondisk+write+known_if_redirected e113)", 2024-02-23T00:49:39.970 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:03.198372+0000", 2024-02-23T00:49:39.970 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 396.74627383799998, 2024-02-23T00:49:39.970 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1182399249999999, 2024-02-23T00:49:39.970 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:49:39.970 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:49:39.970 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:49:39.970 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.15144", 2024-02-23T00:49:39.970 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/2599587504", 2024-02-23T00:49:39.970 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 2 2024-02-23T00:49:39.971 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.971 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:49:39.971 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.971 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:49:39.971 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.198372+0000", 2024-02-23T00:49:39.971 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:49:39.971 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.971 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.971 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:49:39.971 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.198372+0000", 2024-02-23T00:49:39.971 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:49:39.971 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.971 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.971 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:49:39.971 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.198370+0000", 2024-02-23T00:49:39.972 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0789999999999997e-06 2024-02-23T00:49:39.972 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.972 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.972 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:49:39.972 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.198376+0000", 2024-02-23T00:49:39.972 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6660000000000003e-06 2024-02-23T00:49:39.972 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.972 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.972 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:49:39.972 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.198386+0000", 2024-02-23T00:49:39.972 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.5639999999999996e-06 2024-02-23T00:49:39.972 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.972 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.972 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:49:39.973 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.198392+0000", 2024-02-23T00:49:39.973 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8449999999999999e-05 2024-02-23T00:49:39.973 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.973 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.973 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:49:39.973 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.198440+0000", 2024-02-23T00:49:39.973 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1109999999999996e-06 2024-02-23T00:49:39.973 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.973 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.973 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:49:39.973 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.198446+0000", 2024-02-23T00:49:39.973 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1068513440000001 2024-02-23T00:49:39.973 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.973 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.974 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:49:39.974 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.305297+0000", 2024-02-23T00:49:39.974 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012047799999999999 2024-02-23T00:49:39.974 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.974 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.974 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:49:39.974 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.305418+0000", 2024-02-23T00:49:39.974 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000148796 2024-02-23T00:49:39.974 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.974 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.974 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 4,6", 2024-02-23T00:49:39.974 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.305567+0000", 2024-02-23T00:49:39.974 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0045039290000000003 2024-02-23T00:49:39.974 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.975 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.975 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:49:39.975 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.310071+0000", 2024-02-23T00:49:39.975 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0063369849999999998 2024-02-23T00:49:39.975 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.975 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.975 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:49:39.975 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.316408+0000", 2024-02-23T00:49:39.975 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000112998 2024-02-23T00:49:39.975 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.975 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.975 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:49:39.975 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.316521+0000", 2024-02-23T00:49:39.975 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9656e-05 2024-02-23T00:49:39.976 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.976 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.976 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:49:39.976 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.316570+0000", 2024-02-23T00:49:39.976 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1307999999999998e-05 2024-02-23T00:49:39.976 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.976 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.976 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:49:39.976 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.316612+0000", 2024-02-23T00:49:39.976 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1182399249999999 2024-02-23T00:49:39.976 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:39.976 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:49:39.976 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:39.976 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.977 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.977 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.25111.0:1 46.4s0 46:241ec7d8:test-rados-api-smithi047-82512-8::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e117)", 2024-02-23T00:49:39.977 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:06.589186+0000", 2024-02-23T00:49:39.977 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 393.35545958099999, 2024-02-23T00:49:39.977 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.40034441, 2024-02-23T00:49:39.977 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:49:39.977 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:49:39.977 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:49:39.977 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.25111", 2024-02-23T00:49:39.977 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/851664869", 2024-02-23T00:49:39.977 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1 2024-02-23T00:49:39.977 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.977 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:49:39.977 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.977 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:49:39.978 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:06.589186+0000", 2024-02-23T00:49:39.978 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:49:39.978 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.978 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.978 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:49:39.978 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:06.589186+0000", 2024-02-23T00:49:39.978 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:49:39.978 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.978 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.978 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:49:39.978 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:06.589185+0000", 2024-02-23T00:49:39.978 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0599999999999996e-06 2024-02-23T00:49:39.978 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.978 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.978 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:49:39.979 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:06.589191+0000", 2024-02-23T00:49:39.979 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000303713 2024-02-23T00:49:39.979 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.979 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.979 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:49:39.979 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:06.589494+0000", 2024-02-23T00:49:39.979 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0910000000000004e-06 2024-02-23T00:49:39.979 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.979 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.979 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:49:39.979 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:06.589501+0000", 2024-02-23T00:49:39.979 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6595999999999999e-05 2024-02-23T00:49:39.979 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.979 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.980 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:49:39.980 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:06.589548+0000", 2024-02-23T00:49:39.980 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.3579999999999999e-06 2024-02-23T00:49:39.980 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.980 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.980 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:49:39.980 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:06.589551+0000", 2024-02-23T00:49:39.980 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.392918256 2024-02-23T00:49:39.980 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.980 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.980 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:49:39.980 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.982470+0000", 2024-02-23T00:49:39.980 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013435300000000001 2024-02-23T00:49:39.980 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.981 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.981 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:49:39.981 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.982604+0000", 2024-02-23T00:49:39.981 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00045436599999999999 2024-02-23T00:49:39.981 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.981 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.981 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:49:39.981 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.983058+0000", 2024-02-23T00:49:39.981 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00046995799999999998 2024-02-23T00:49:39.981 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.981 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.981 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:49:39.981 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.983528+0000", 2024-02-23T00:49:39.981 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0059363580000000001 2024-02-23T00:49:39.982 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.982 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.982 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:49:39.982 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.989465+0000", 2024-02-23T00:49:39.982 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5662999999999998e-05 2024-02-23T00:49:39.982 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.982 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.982 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:49:39.982 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.989530+0000", 2024-02-23T00:49:39.982 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.40034441 2024-02-23T00:49:39.982 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:39.982 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:49:39.982 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:39.982 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.982 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.983 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(osd.3.0:1389 20.0 e139/109 -22:0000ebd9:::temp_20.0_0_14313_1:head v 0'0, mlcod=0'0)", 2024-02-23T00:49:39.983 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:32.123601+0000", 2024-02-23T00:49:39.983 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 367.82104422899999, 2024-02-23T00:49:39.983 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.024989610999999998, 2024-02-23T00:49:39.983 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:49:39.983 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:49:39.983 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:49:39.983 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.983 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:49:39.983 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.123601+0000", 2024-02-23T00:49:39.983 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:49:39.983 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.983 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.983 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:49:39.984 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.123601+0000", 2024-02-23T00:49:39.984 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:49:39.984 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.984 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.984 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:49:39.984 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.123599+0000", 2024-02-23T00:49:39.984 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013123701999999999 2024-02-23T00:49:39.984 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.984 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.984 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:49:39.984 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.136723+0000", 2024-02-23T00:49:39.984 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2133999999999998e-05 2024-02-23T00:49:39.984 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.984 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.984 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:49:39.985 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.136745+0000", 2024-02-23T00:49:39.985 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.4460000000000001e-06 2024-02-23T00:49:39.985 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.985 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.985 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:49:39.985 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.136754+0000", 2024-02-23T00:49:39.985 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8553000000000001e-05 2024-02-23T00:49:39.985 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.985 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.985 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:49:39.985 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.136802+0000", 2024-02-23T00:49:39.985 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9707999999999998e-05 2024-02-23T00:49:39.985 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.985 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.986 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:49:39.986 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.136822+0000", 2024-02-23T00:49:39.986 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011702494000000001 2024-02-23T00:49:39.986 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.986 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.986 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:49:39.986 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.148524+0000", 2024-02-23T00:49:39.986 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6593000000000004e-05 2024-02-23T00:49:39.986 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.986 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.986 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:49:39.986 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.148591+0000", 2024-02-23T00:49:39.986 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.024989610999999998 2024-02-23T00:49:39.986 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:39.986 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:49:39.987 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:39.987 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.987 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.987 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(osd.3.0:1390 20.0 e139/109 -22:0000ebd9:::temp_20.0_0_14313_1:head v 0'0, mlcod=0'0)", 2024-02-23T00:49:39.987 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:32.171787+0000", 2024-02-23T00:49:39.987 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 367.77285878200001, 2024-02-23T00:49:39.987 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.049768688999999998, 2024-02-23T00:49:39.987 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:49:39.987 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:49:39.987 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:49:39.987 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.987 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:49:39.987 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.171787+0000", 2024-02-23T00:49:39.987 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:49:39.988 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.988 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.988 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:49:39.988 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.171787+0000", 2024-02-23T00:49:39.988 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:49:39.988 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.988 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.988 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:49:39.988 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.171785+0000", 2024-02-23T00:49:39.988 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037314238 2024-02-23T00:49:39.988 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.988 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.988 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:49:39.988 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.209100+0000", 2024-02-23T00:49:39.988 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6101999999999999e-05 2024-02-23T00:49:39.989 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.989 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.989 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:49:39.989 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.209116+0000", 2024-02-23T00:49:39.989 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.3939999999999998e-06 2024-02-23T00:49:39.989 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.989 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.989 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:49:39.989 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.209123+0000", 2024-02-23T00:49:39.989 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0012550479999999999 2024-02-23T00:49:39.989 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.989 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.989 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:49:39.989 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.210378+0000", 2024-02-23T00:49:39.990 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7702000000000001e-05 2024-02-23T00:49:39.990 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.990 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.990 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:49:39.990 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.210396+0000", 2024-02-23T00:49:39.990 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011131066 2024-02-23T00:49:39.990 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.990 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.990 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:49:39.990 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.221527+0000", 2024-02-23T00:49:39.990 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.8481000000000001e-05 2024-02-23T00:49:39.990 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.990 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.990 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:49:39.990 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.221555+0000", 2024-02-23T00:49:39.991 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.049768688999999998 2024-02-23T00:49:39.991 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:39.991 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:49:39.991 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:39.991 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.991 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.991 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(osd.3.0:1391 20.0 e140/109 -22:0000ebd9:::temp_20.0_0_14313_1:head v 0'0, mlcod=0'0)", 2024-02-23T00:49:39.991 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:32.305524+0000", 2024-02-23T00:49:39.991 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 367.63912141999998, 2024-02-23T00:49:39.991 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.045602061999999999, 2024-02-23T00:49:39.991 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:49:39.991 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:49:39.991 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:49:39.991 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.992 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:49:39.992 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.305524+0000", 2024-02-23T00:49:39.992 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:49:39.992 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.992 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.992 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:49:39.992 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.305524+0000", 2024-02-23T00:49:39.992 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:49:39.992 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.992 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.992 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:49:39.992 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.305523+0000", 2024-02-23T00:49:39.992 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033906463999999997 2024-02-23T00:49:39.992 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.993 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.993 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:49:39.993 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.339430+0000", 2024-02-23T00:49:39.993 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8787000000000002e-05 2024-02-23T00:49:39.993 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.993 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.993 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:49:39.993 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.339449+0000", 2024-02-23T00:49:39.993 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5659999999999996e-06 2024-02-23T00:49:39.993 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.993 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.993 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:49:39.993 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.339458+0000", 2024-02-23T00:49:39.993 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5281999999999998e-05 2024-02-23T00:49:39.993 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.994 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.994 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:49:39.994 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.339503+0000", 2024-02-23T00:49:39.994 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9097e-05 2024-02-23T00:49:39.994 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.994 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.994 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:49:39.994 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.339523+0000", 2024-02-23T00:49:39.994 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01157158 2024-02-23T00:49:39.994 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.994 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.994 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:49:39.994 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.351094+0000", 2024-02-23T00:49:39.994 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.2014999999999998e-05 2024-02-23T00:49:39.995 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.995 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.995 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:49:39.995 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.351126+0000", 2024-02-23T00:49:39.995 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.045602061999999999 2024-02-23T00:49:39.995 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:39.995 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:49:39.995 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:39.995 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.995 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.995 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(osd.1.0:294 20.6 e140/109 -22:61acad7d:::temp_20.6_0_24170_1:head v 0'0, mlcod=112'6)", 2024-02-23T00:49:39.995 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:32.414966+0000", 2024-02-23T00:49:39.995 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 367.52967981199998, 2024-02-23T00:49:39.995 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.025904508, 2024-02-23T00:49:39.995 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:49:39.996 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:49:39.996 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:49:39.996 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.996 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:49:39.996 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.414966+0000", 2024-02-23T00:49:39.996 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:49:39.996 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.996 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.996 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:49:39.996 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.414966+0000", 2024-02-23T00:49:39.996 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:49:39.996 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.996 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.997 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:49:39.997 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.414965+0000", 2024-02-23T00:49:39.997 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750562999999999 2024-02-23T00:49:39.997 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.997 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.997 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:49:39.997 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.430715+0000", 2024-02-23T00:49:39.997 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0077000000000001e-05 2024-02-23T00:49:39.997 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.997 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.997 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:49:39.997 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.430735+0000", 2024-02-23T00:49:39.997 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1629999999999996e-06 2024-02-23T00:49:39.997 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.997 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.998 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:49:39.998 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.430743+0000", 2024-02-23T00:49:39.998 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.4369000000000002e-05 2024-02-23T00:49:39.998 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.998 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.998 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:49:39.998 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.430788+0000", 2024-02-23T00:49:39.998 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8938999999999999e-05 2024-02-23T00:49:39.998 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.998 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.998 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:49:39.998 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.430807+0000", 2024-02-23T00:49:39.998 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010016872 2024-02-23T00:49:39.998 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.998 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.999 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:49:39.999 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.440824+0000", 2024-02-23T00:49:39.999 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6726e-05 2024-02-23T00:49:39.999 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.999 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.999 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:49:39.999 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.440870+0000", 2024-02-23T00:49:39.999 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.025904508 2024-02-23T00:49:39.999 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:39.999 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:49:39.999 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:39.999 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:39.999 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:39.999 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.31603.0:4 79.5s0 79:a89d8add:test-rados-api-smithi047-82487-11::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e143)", 2024-02-23T00:49:40.000 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:34.937176+0000", 2024-02-23T00:49:40.000 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 365.00746921400003, 2024-02-23T00:49:40.000 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.101250728, 2024-02-23T00:49:40.000 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:49:40.000 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:49:40.000 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:49:40.000 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.31603", 2024-02-23T00:49:40.000 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1361361966", 2024-02-23T00:49:40.000 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 4 2024-02-23T00:49:40.000 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.000 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:49:40.000 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.000 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:49:40.000 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:34.937176+0000", 2024-02-23T00:49:40.001 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:49:40.001 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.001 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.001 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:49:40.001 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:34.937176+0000", 2024-02-23T00:49:40.001 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:49:40.001 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.001 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.001 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:49:40.001 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:34.937174+0000", 2024-02-23T00:49:40.001 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9299999999999992e-06 2024-02-23T00:49:40.001 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.001 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.001 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:49:40.001 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:34.937183+0000", 2024-02-23T00:49:40.002 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2823000000000001e-05 2024-02-23T00:49:40.002 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.002 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.002 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:49:40.002 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:34.937196+0000", 2024-02-23T00:49:40.002 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5089999999999999e-06 2024-02-23T00:49:40.002 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.002 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.002 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:49:40.002 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:34.937203+0000", 2024-02-23T00:49:40.002 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6110999999999999e-05 2024-02-23T00:49:40.002 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.002 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.002 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:49:40.003 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:34.937259+0000", 2024-02-23T00:49:40.003 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.3068999999999999e-05 2024-02-23T00:49:40.003 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.003 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.003 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:49:40.003 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:34.937352+0000", 2024-02-23T00:49:40.003 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028850399999999997 2024-02-23T00:49:40.003 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.003 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.003 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:49:40.003 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:34.937640+0000", 2024-02-23T00:49:40.003 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028285199999999999 2024-02-23T00:49:40.003 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.003 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.004 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:49:40.004 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:34.937923+0000", 2024-02-23T00:49:40.004 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.100441616 2024-02-23T00:49:40.004 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.004 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.004 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:49:40.004 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:35.038365+0000", 2024-02-23T00:49:40.004 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.2336999999999996e-05 2024-02-23T00:49:40.004 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.004 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.004 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:49:40.004 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:35.038427+0000", 2024-02-23T00:49:40.004 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.101250728 2024-02-23T00:49:40.004 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:40.005 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:49:40.005 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:40.005 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.005 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.005 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.15270.0:248 62.14 62:28b98b91:test-rados-api-smithi047-82465-12::bar:head [writefull 0~8] snapc 0=[] ondisk+write+known_if_redirected e145)", 2024-02-23T00:49:40.005 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:37.219883+0000", 2024-02-23T00:49:40.005 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 362.72476292300001, 2024-02-23T00:49:40.005 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037348011, 2024-02-23T00:49:40.005 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:49:40.005 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:49:40.005 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:49:40.005 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.15270", 2024-02-23T00:49:40.005 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/4133111990", 2024-02-23T00:49:40.005 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 248 2024-02-23T00:49:40.006 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.006 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:49:40.006 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.006 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:49:40.006 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.219883+0000", 2024-02-23T00:49:40.006 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:49:40.006 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.006 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.006 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:49:40.006 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.219883+0000", 2024-02-23T00:49:40.006 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:49:40.006 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.006 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.006 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:49:40.007 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.219881+0000", 2024-02-23T00:49:40.007 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4849999999999997e-06 2024-02-23T00:49:40.007 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.007 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.007 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:49:40.007 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.219888+0000", 2024-02-23T00:49:40.007 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0907000000000001e-05 2024-02-23T00:49:40.007 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.007 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.007 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:49:40.007 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.219899+0000", 2024-02-23T00:49:40.007 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1180000000000003e-06 2024-02-23T00:49:40.007 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.007 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.008 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:49:40.008 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.219906+0000", 2024-02-23T00:49:40.008 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1885999999999998e-05 2024-02-23T00:49:40.008 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.008 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.008 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:49:40.008 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.219958+0000", 2024-02-23T00:49:40.008 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000102293 2024-02-23T00:49:40.008 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.008 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.008 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:49:40.008 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.220060+0000", 2024-02-23T00:49:40.008 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4350000000000003e-05 2024-02-23T00:49:40.008 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.009 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.009 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 2,4", 2024-02-23T00:49:40.009 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.220154+0000", 2024-02-23T00:49:40.009 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00034315799999999998 2024-02-23T00:49:40.009 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.009 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.009 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:49:40.009 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.220497+0000", 2024-02-23T00:49:40.009 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00019797599999999999 2024-02-23T00:49:40.009 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.009 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.009 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:49:40.009 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.220695+0000", 2024-02-23T00:49:40.009 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036432616000000001 2024-02-23T00:49:40.010 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.010 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.010 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:49:40.010 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.257128+0000", 2024-02-23T00:49:40.010 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.5760000000000001e-05 2024-02-23T00:49:40.010 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.010 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.010 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:49:40.010 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.257184+0000", 2024-02-23T00:49:40.010 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6805999999999999e-05 2024-02-23T00:49:40.010 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.010 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.010 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:49:40.010 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.257231+0000", 2024-02-23T00:49:40.010 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037348011 2024-02-23T00:49:40.011 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:40.011 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:49:40.011 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:40.011 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.011 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.011 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.39416.0:13 95.2s0 95:442961eb:test-rados-api-smithi047-82487-16::foo:head [write 8192~8192] snapc 7=[7,6] ondisk+write+known_if_redirected e161)", 2024-02-23T00:49:40.011 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:53.113989+0000", 2024-02-23T00:49:40.011 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 346.83065646699998, 2024-02-23T00:49:40.011 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.024944128999999999, 2024-02-23T00:49:40.011 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:49:40.011 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:49:40.011 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:49:40.011 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.39416", 2024-02-23T00:49:40.011 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/2338570187", 2024-02-23T00:49:40.012 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 13 2024-02-23T00:49:40.012 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.012 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:49:40.012 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.012 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:49:40.012 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:53.113989+0000", 2024-02-23T00:49:40.012 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:49:40.012 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.012 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.012 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:49:40.012 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:53.113989+0000", 2024-02-23T00:49:40.012 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:49:40.012 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.012 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.013 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:49:40.013 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:53.113987+0000", 2024-02-23T00:49:40.013 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4941999999999999e-05 2024-02-23T00:49:40.013 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.013 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.013 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:49:40.013 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:53.114012+0000", 2024-02-23T00:49:40.013 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4773000000000001e-05 2024-02-23T00:49:40.013 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.013 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.013 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:49:40.013 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:53.114027+0000", 2024-02-23T00:49:40.013 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.8349999999999993e-06 2024-02-23T00:49:40.013 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.013 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.014 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:49:40.014 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:53.114036+0000", 2024-02-23T00:49:40.014 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015537468 2024-02-23T00:49:40.014 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.014 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.014 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:49:40.014 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:53.129573+0000", 2024-02-23T00:49:40.014 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103847 2024-02-23T00:49:40.014 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.014 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.014 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:49:40.014 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:53.129677+0000", 2024-02-23T00:49:40.014 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00069602900000000003 2024-02-23T00:49:40.014 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.014 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.015 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:49:40.015 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:53.130373+0000", 2024-02-23T00:49:40.015 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001455729 2024-02-23T00:49:40.015 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.015 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.015 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:49:40.015 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:53.131829+0000", 2024-02-23T00:49:40.015 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0070283079999999996 2024-02-23T00:49:40.015 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.015 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.015 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:49:40.015 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:53.138857+0000", 2024-02-23T00:49:40.015 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6155000000000005e-05 2024-02-23T00:49:40.015 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.015 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.016 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:49:40.016 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:53.138933+0000", 2024-02-23T00:49:40.016 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.024944128999999999 2024-02-23T00:49:40.016 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:40.016 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:49:40.016 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:40.016 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.016 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.016 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.40529.0:1 114.4s0 114:2a73ef27:test-rados-api-smithi047-82496-16::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e180)", 2024-02-23T00:49:40.016 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:44:12.479624+0000", 2024-02-23T00:49:40.016 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 327.46502177799999, 2024-02-23T00:49:40.016 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.035619211, 2024-02-23T00:49:40.016 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:49:40.016 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:49:40.017 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:49:40.017 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.40529", 2024-02-23T00:49:40.017 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/3414618047", 2024-02-23T00:49:40.017 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1 2024-02-23T00:49:40.017 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.017 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:49:40.017 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.017 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:49:40.017 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:12.479624+0000", 2024-02-23T00:49:40.017 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:49:40.017 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.018 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.018 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:49:40.018 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:12.479624+0000", 2024-02-23T00:49:40.018 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:49:40.018 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.018 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.018 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:49:40.018 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:12.479623+0000", 2024-02-23T00:49:40.018 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5939999999999996e-06 2024-02-23T00:49:40.018 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.018 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.018 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:49:40.019 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:12.479630+0000", 2024-02-23T00:49:40.019 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6150000000000002e-06 2024-02-23T00:49:40.019 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.019 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.019 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:49:40.019 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:12.479637+0000", 2024-02-23T00:49:40.019 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4399999999999996e-06 2024-02-23T00:49:40.019 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.019 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.019 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:49:40.019 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:12.479642+0000", 2024-02-23T00:49:40.019 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9409999999999997e-05 2024-02-23T00:49:40.019 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.020 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.020 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:49:40.020 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:12.479682+0000", 2024-02-23T00:49:40.020 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.311e-06 2024-02-23T00:49:40.020 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.020 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.020 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:49:40.020 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:12.479684+0000", 2024-02-23T00:49:40.020 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0225209479999999 2024-02-23T00:49:40.020 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.020 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.021 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:49:40.021 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:13.502205+0000", 2024-02-23T00:49:40.021 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000244198 2024-02-23T00:49:40.021 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.021 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.021 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:49:40.021 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:13.502449+0000", 2024-02-23T00:49:40.021 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00046503000000000001 2024-02-23T00:49:40.021 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.021 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.021 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:49:40.021 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:13.502914+0000", 2024-02-23T00:49:40.021 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0018608260000000001 2024-02-23T00:49:40.021 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.022 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.022 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:49:40.022 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:13.504775+0000", 2024-02-23T00:49:40.022 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010417872999999999 2024-02-23T00:49:40.022 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.022 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.022 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:49:40.022 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:13.515193+0000", 2024-02-23T00:49:40.022 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0229999999999998e-05 2024-02-23T00:49:40.022 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.022 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.022 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:49:40.022 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:13.515243+0000", 2024-02-23T00:49:40.023 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.035619211 2024-02-23T00:49:40.023 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:40.023 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:49:40.023 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:40.023 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.023 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.023 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.40529.0:4 114.4s0 114:34880124:test-rados-api-smithi047-82496-17::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e183)", 2024-02-23T00:49:40.023 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:44:15.132272+0000", 2024-02-23T00:49:40.023 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 324.81237387200002, 2024-02-23T00:49:40.023 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.026014500999999999, 2024-02-23T00:49:40.023 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:49:40.023 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:49:40.023 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:49:40.023 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.40529", 2024-02-23T00:49:40.023 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/3414618047", 2024-02-23T00:49:40.024 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 4 2024-02-23T00:49:40.024 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.024 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:49:40.024 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.024 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:49:40.024 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:15.132272+0000", 2024-02-23T00:49:40.024 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:49:40.024 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.024 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.024 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:49:40.024 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:15.132272+0000", 2024-02-23T00:49:40.024 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999976 2024-02-23T00:49:40.024 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.024 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.025 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:49:40.025 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:15.132269+0000", 2024-02-23T00:49:40.025 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9840000000000001e-06 2024-02-23T00:49:40.025 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.025 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.025 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:49:40.025 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:15.132277+0000", 2024-02-23T00:49:40.025 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1032999999999999e-05 2024-02-23T00:49:40.025 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.025 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.025 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:49:40.025 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:15.132288+0000", 2024-02-23T00:49:40.025 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.6419999999999996e-06 2024-02-23T00:49:40.025 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.025 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.026 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:49:40.026 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:15.132297+0000", 2024-02-23T00:49:40.026 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00036041099999999999 2024-02-23T00:49:40.026 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.026 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.026 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:49:40.026 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:15.132658+0000", 2024-02-23T00:49:40.026 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0790000000000001e-06 2024-02-23T00:49:40.026 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.026 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.026 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op must wait for map", 2024-02-23T00:49:40.026 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:15.132666+0000", 2024-02-23T00:49:40.026 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015467946 2024-02-23T00:49:40.026 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.027 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.027 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:49:40.027 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:15.148134+0000", 2024-02-23T00:49:40.027 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016813900000000001 2024-02-23T00:49:40.027 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.027 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.027 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:49:40.027 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:15.148302+0000", 2024-02-23T00:49:40.027 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00059727800000000002 2024-02-23T00:49:40.027 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.027 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.027 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:49:40.027 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:15.148899+0000", 2024-02-23T00:49:40.027 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0090316800000000003 2024-02-23T00:49:40.028 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.028 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.028 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:49:40.028 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:15.157931+0000", 2024-02-23T00:49:40.028 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2560999999999998e-05 2024-02-23T00:49:40.028 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.028 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.028 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:49:40.028 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:15.158013+0000", 2024-02-23T00:49:40.028 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00027295299999999999 2024-02-23T00:49:40.028 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.028 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.028 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:49:40.029 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:15.158286+0000", 2024-02-23T00:49:40.029 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.026014500999999999 2024-02-23T00:49:40.029 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:40.029 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:49:40.029 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:40.029 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.029 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.029 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.41009.0:1 179.6s0 179:602f83fe:::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e292)", 2024-02-23T00:49:40.029 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:46:04.517108+0000", 2024-02-23T00:49:40.029 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 215.42753768200001, 2024-02-23T00:49:40.029 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.66709289299999996, 2024-02-23T00:49:40.029 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:49:40.029 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:49:40.029 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:49:40.030 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.41009", 2024-02-23T00:49:40.030 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/2639020958", 2024-02-23T00:49:40.030 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1 2024-02-23T00:49:40.030 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.030 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:49:40.030 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.030 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:49:40.030 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:04.517108+0000", 2024-02-23T00:49:40.030 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:49:40.030 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.030 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.030 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:49:40.030 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:04.517108+0000", 2024-02-23T00:49:40.030 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:49:40.030 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.031 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.031 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:49:40.031 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:04.517107+0000", 2024-02-23T00:49:40.031 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0429999999999998e-06 2024-02-23T00:49:40.031 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.031 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.031 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:49:40.031 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:04.517112+0000", 2024-02-23T00:49:40.031 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5920000000000003e-06 2024-02-23T00:49:40.031 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.031 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.031 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:49:40.031 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:04.517119+0000", 2024-02-23T00:49:40.031 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1099999999999997e-06 2024-02-23T00:49:40.032 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.032 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.032 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:49:40.032 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:04.517126+0000", 2024-02-23T00:49:40.032 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.8010000000000001e-05 2024-02-23T00:49:40.032 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.032 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.032 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:49:40.032 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:04.517154+0000", 2024-02-23T00:49:40.032 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.593e-06 2024-02-23T00:49:40.032 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.032 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.032 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:49:40.032 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:04.517157+0000", 2024-02-23T00:49:40.032 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.66365052700000005 2024-02-23T00:49:40.033 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.033 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.033 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:49:40.033 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:05.180807+0000", 2024-02-23T00:49:40.033 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111835 2024-02-23T00:49:40.033 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.033 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.033 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:49:40.033 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:05.180919+0000", 2024-02-23T00:49:40.033 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040999 2024-02-23T00:49:40.033 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.033 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.033 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:49:40.033 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:05.181329+0000", 2024-02-23T00:49:40.033 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0011668410000000001 2024-02-23T00:49:40.034 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.034 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.034 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:49:40.034 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:05.182496+0000", 2024-02-23T00:49:40.034 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0016674039999999999 2024-02-23T00:49:40.034 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.034 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.034 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:49:40.034 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:05.184163+0000", 2024-02-23T00:49:40.034 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7271000000000003e-05 2024-02-23T00:49:40.034 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.034 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.034 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:49:40.035 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:05.184201+0000", 2024-02-23T00:49:40.035 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.66709289299999996 2024-02-23T00:49:40.035 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:40.035 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:49:40.035 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:40.035 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.035 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.035 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.30741.0:1 187.6s0 187:72c88bc2:::oid_IsCompletePP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e311)", 2024-02-23T00:49:40.035 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:46:23.195582+0000", 2024-02-23T00:49:40.035 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 196.74906383800001, 2024-02-23T00:49:40.035 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0060393860000001, 2024-02-23T00:49:40.035 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:49:40.035 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:49:40.035 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:49:40.036 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.30741", 2024-02-23T00:49:40.036 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1977736697", 2024-02-23T00:49:40.036 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1 2024-02-23T00:49:40.036 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.036 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:49:40.036 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.036 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:49:40.036 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:23.195582+0000", 2024-02-23T00:49:40.036 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:49:40.036 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.036 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.036 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:49:40.036 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:23.195582+0000", 2024-02-23T00:49:40.036 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:49:40.037 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.037 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.037 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:49:40.037 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:23.195580+0000", 2024-02-23T00:49:40.037 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.7450000000000001e-06 2024-02-23T00:49:40.037 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.037 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.037 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:49:40.037 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:23.195586+0000", 2024-02-23T00:49:40.037 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.6580000000000008e-06 2024-02-23T00:49:40.037 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.037 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.037 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:49:40.037 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:23.195595+0000", 2024-02-23T00:49:40.038 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1179999999999997e-06 2024-02-23T00:49:40.038 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.038 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.038 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:49:40.038 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:23.195601+0000", 2024-02-23T00:49:40.038 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.4499999999999997e-05 2024-02-23T00:49:40.038 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.038 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.038 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:49:40.038 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:23.195645+0000", 2024-02-23T00:49:40.038 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1960000000000001e-06 2024-02-23T00:49:40.038 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.038 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.038 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:49:40.039 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:23.195650+0000", 2024-02-23T00:49:40.039 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0044928049999999 2024-02-23T00:49:40.039 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.039 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.039 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:49:40.039 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:24.200142+0000", 2024-02-23T00:49:40.039 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000152762 2024-02-23T00:49:40.039 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.039 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.039 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:49:40.039 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:24.200295+0000", 2024-02-23T00:49:40.039 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00054695799999999995 2024-02-23T00:49:40.039 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.039 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.040 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:49:40.040 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:24.200842+0000", 2024-02-23T00:49:40.040 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00047161300000000001 2024-02-23T00:49:40.040 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.040 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.040 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:49:40.040 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:24.201314+0000", 2024-02-23T00:49:40.040 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026416699999999999 2024-02-23T00:49:40.040 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.040 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.040 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:49:40.040 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:24.201578+0000", 2024-02-23T00:49:40.040 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3269000000000002e-05 2024-02-23T00:49:40.040 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.041 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.041 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:49:40.041 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:24.201621+0000", 2024-02-23T00:49:40.041 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0060393860000001 2024-02-23T00:49:40.041 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:40.041 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:49:40.041 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:40.041 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.041 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.041 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.30861.0:1 199.2s0 199:482bd08b:::oid_SimpleStatPP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e356)", 2024-02-23T00:49:40.041 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:47:08.178993+0000", 2024-02-23T00:49:40.041 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 151.76565246000001, 2024-02-23T00:49:40.041 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1427090879999999, 2024-02-23T00:49:40.041 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:49:40.041 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:49:40.042 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:49:40.042 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.30861", 2024-02-23T00:49:40.042 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/3942253766", 2024-02-23T00:49:40.042 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1 2024-02-23T00:49:40.042 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.042 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:49:40.042 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.042 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:49:40.042 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:08.178993+0000", 2024-02-23T00:49:40.042 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:49:40.042 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.042 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.042 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:49:40.042 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:08.178993+0000", 2024-02-23T00:49:40.042 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:49:40.043 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.043 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.043 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:49:40.043 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:08.178992+0000", 2024-02-23T00:49:40.043 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.4229999999999998e-06 2024-02-23T00:49:40.043 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.043 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.043 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:49:40.043 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:08.178997+0000", 2024-02-23T00:49:40.043 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5320000000000001e-06 2024-02-23T00:49:40.043 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.043 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.043 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:49:40.043 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:08.179004+0000", 2024-02-23T00:49:40.044 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.815e-06 2024-02-23T00:49:40.044 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.044 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.044 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:49:40.044 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:08.179011+0000", 2024-02-23T00:49:40.044 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0550000000000003e-05 2024-02-23T00:49:40.044 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.044 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.044 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:49:40.044 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:08.179051+0000", 2024-02-23T00:49:40.044 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9099999999999999e-06 2024-02-23T00:49:40.044 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.044 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.044 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:49:40.045 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:08.179053+0000", 2024-02-23T00:49:40.045 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.141086343 2024-02-23T00:49:40.045 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.045 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.045 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:49:40.045 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:09.320140+0000", 2024-02-23T00:49:40.045 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156467 2024-02-23T00:49:40.045 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.045 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.045 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:49:40.045 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:09.320296+0000", 2024-02-23T00:49:40.045 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00042973700000000002 2024-02-23T00:49:40.045 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.045 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.045 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:49:40.046 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:09.320726+0000", 2024-02-23T00:49:40.046 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00049958999999999995 2024-02-23T00:49:40.046 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.046 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.046 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:49:40.046 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:09.321226+0000", 2024-02-23T00:49:40.046 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000398384 2024-02-23T00:49:40.046 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.046 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.046 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:49:40.046 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:09.321624+0000", 2024-02-23T00:49:40.046 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.8273000000000001e-05 2024-02-23T00:49:40.046 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.046 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.046 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:49:40.047 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:09.321702+0000", 2024-02-23T00:49:40.047 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1427090879999999 2024-02-23T00:49:40.047 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:40.047 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:49:40.047 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:40.047 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.047 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.047 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.15270.0:4850 196.18 196:18000000:%01:::head [pgnls start_epoch 358] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e358)", 2024-02-23T00:49:40.047 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:47:10.195882+0000", 2024-02-23T00:49:40.047 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 149.748763239, 2024-02-23T00:49:40.047 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.024687572000000001, 2024-02-23T00:49:40.047 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:49:40.047 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:49:40.047 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:49:40.048 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.15270", 2024-02-23T00:49:40.048 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/4133111990", 2024-02-23T00:49:40.048 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 4850 2024-02-23T00:49:40.048 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.048 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:49:40.048 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.048 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:49:40.048 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:10.195882+0000", 2024-02-23T00:49:40.048 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:49:40.048 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.048 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.048 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:49:40.048 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:10.195882+0000", 2024-02-23T00:49:40.048 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999976 2024-02-23T00:49:40.048 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.049 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.049 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:49:40.049 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:10.195880+0000", 2024-02-23T00:49:40.049 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3458999999999999e-05 2024-02-23T00:49:40.049 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.049 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.049 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:49:40.049 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:10.195893+0000", 2024-02-23T00:49:40.049 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0264000000000001e-05 2024-02-23T00:49:40.049 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.049 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.049 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:49:40.049 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:10.195913+0000", 2024-02-23T00:49:40.049 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2982e-05 2024-02-23T00:49:40.050 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.050 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.050 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:49:40.050 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:10.195926+0000", 2024-02-23T00:49:40.050 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6429999999999999e-05 2024-02-23T00:49:40.050 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.050 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.050 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:49:40.050 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:10.195993+0000", 2024-02-23T00:49:40.050 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1660000000000002e-06 2024-02-23T00:49:40.050 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.050 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.050 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op must wait for map", 2024-02-23T00:49:40.050 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:10.195998+0000", 2024-02-23T00:49:40.050 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.024425932000000001 2024-02-23T00:49:40.051 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.051 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.051 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:49:40.051 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:10.220424+0000", 2024-02-23T00:49:40.051 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7039000000000002e-05 2024-02-23T00:49:40.051 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.051 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.051 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:49:40.051 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:10.220461+0000", 2024-02-23T00:49:40.051 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000108857 2024-02-23T00:49:40.051 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.051 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.051 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:49:40.052 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:10.220570+0000", 2024-02-23T00:49:40.052 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.024687572000000001 2024-02-23T00:49:40.052 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:40.052 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:49:40.052 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:40.052 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.052 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.052 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.24311.0:1285 5.d 5:b4812045:::notify.4:head [watch reconnect cookie 94587070401920 gen 1] snapc 0=[] ondisk+write+known_if_redirected e375)", 2024-02-23T00:49:40.052 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:47:27.942599+0000", 2024-02-23T00:49:40.052 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 132.002046447, 2024-02-23T00:49:40.052 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.31385020000000002, 2024-02-23T00:49:40.052 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:49:40.053 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:49:40.053 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:49:40.053 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.24311", 2024-02-23T00:49:40.053 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/3723202188", 2024-02-23T00:49:40.053 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1285 2024-02-23T00:49:40.053 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.053 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:49:40.053 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.053 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:49:40.053 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.942599+0000", 2024-02-23T00:49:40.053 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:49:40.053 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.053 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.053 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:49:40.054 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.942599+0000", 2024-02-23T00:49:40.054 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:49:40.054 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.054 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.054 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:49:40.054 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.942598+0000", 2024-02-23T00:49:40.054 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7109999999999998e-06 2024-02-23T00:49:40.054 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.054 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.054 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:49:40.054 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.942605+0000", 2024-02-23T00:49:40.054 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2098e-05 2024-02-23T00:49:40.054 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.055 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.055 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:49:40.055 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.942617+0000", 2024-02-23T00:49:40.055 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.8320000000000002e-06 2024-02-23T00:49:40.055 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.055 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.055 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:49:40.055 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.942623+0000", 2024-02-23T00:49:40.055 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.5760000000000001e-05 2024-02-23T00:49:40.055 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.055 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.055 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:49:40.055 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.942678+0000", 2024-02-23T00:49:40.055 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.559e-06 2024-02-23T00:49:40.056 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.056 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.056 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:49:40.056 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.942686+0000", 2024-02-23T00:49:40.056 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.288747688 2024-02-23T00:49:40.056 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.056 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.056 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:49:40.056 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:28.231434+0000", 2024-02-23T00:49:40.056 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000296077 2024-02-23T00:49:40.056 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.056 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.056 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for degraded object", 2024-02-23T00:49:40.057 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:28.231730+0000", 2024-02-23T00:49:40.057 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.018676325000000001 2024-02-23T00:49:40.057 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.057 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.057 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:49:40.057 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:28.250406+0000", 2024-02-23T00:49:40.057 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2424999999999998e-05 2024-02-23T00:49:40.057 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.057 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.057 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:49:40.057 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:28.250488+0000", 2024-02-23T00:49:40.057 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0059609060000000002 2024-02-23T00:49:40.057 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:49:40.057 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:49:40.058 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:49:40.058 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:28.256449+0000", 2024-02-23T00:49:40.058 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.31385020000000002 2024-02-23T00:49:40.058 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:40.058 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:49:40.058 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:40.058 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:49:40.058 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:49:40.058 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:49:40.059 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:40.059 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:49:40.060 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:49:40.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.079+0000 7f6e3219e700 1 -- 172.21.15.47:0/3138300297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e2c0fe510 msgr2=0x7f6e2c0f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:40.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.079+0000 7f6e3219e700 1 --2- 172.21.15.47:0/3138300297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e2c0fe510 0x7f6e2c0f7d40 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 rx=0x7f6e200047b0 tx=0x7f6e20009cb0).stop 2024-02-23T00:49:40.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.079+0000 7f6e3219e700 1 -- 172.21.15.47:0/3138300297 shutdown_connections 2024-02-23T00:49:40.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.079+0000 7f6e3219e700 1 --2- 172.21.15.47:0/3138300297 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6e2c0f8420 0x7f6e2c104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:40.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.079+0000 7f6e3219e700 1 --2- 172.21.15.47:0/3138300297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e2c0fdbb0 0x7f6e2c0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:40.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.079+0000 7f6e3219e700 1 --2- 172.21.15.47:0/3138300297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e2c0fe510 0x7f6e2c0f7d40 unknown :-1 s=CLOSED pgs=2550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:40.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.079+0000 7f6e3219e700 1 -- 172.21.15.47:0/3138300297 >> 172.21.15.47:0/3138300297 conn(0x7f6e2c0f3800 msgr2=0x7f6e2c0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:40.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.079+0000 7f6e3219e700 1 -- 172.21.15.47:0/3138300297 shutdown_connections 2024-02-23T00:49:40.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.079+0000 7f6e3219e700 1 -- 172.21.15.47:0/3138300297 wait complete. 2024-02-23T00:49:40.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.080+0000 7f6e3219e700 1 librados: starting msgr at 2024-02-23T00:49:40.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.080+0000 7f6e3219e700 1 librados: starting objecter 2024-02-23T00:49:40.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.080+0000 7f6e3219e700 1 Processor -- start 2024-02-23T00:49:40.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.080+0000 7f6e3219e700 1 -- start start 2024-02-23T00:49:40.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.080+0000 7f6e3219e700 1 librados: setting wanted keys 2024-02-23T00:49:40.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.080+0000 7f6e3219e700 1 librados: calling monclient init 2024-02-23T00:49:40.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.081+0000 7f6e3219e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6e2c0f8420 0x7f6e2c10ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:40.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.081+0000 7f6e3219e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e2c0fdbb0 0x7f6e2c10e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:40.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.081+0000 7f6e3219e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e2c0fe510 0x7f6e2c108070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:40.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.081+0000 7f6e3219e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c108560 con 0x7f6e2c0fe510 2024-02-23T00:49:40.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.081+0000 7f6e3219e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c1086d0 con 0x7f6e2c0fdbb0 2024-02-23T00:49:40.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.081+0000 7f6e3219e700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c108890 con 0x7f6e2c0f8420 2024-02-23T00:49:40.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.081+0000 7f6e2b7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6e2c0f8420 0x7f6e2c10ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:40.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.081+0000 7f6e2b7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6e2c0f8420 0x7f6e2c10ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:50992/0 (socket says 172.21.15.47:50992) 2024-02-23T00:49:40.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.081+0000 7f6e2b7fe700 1 -- 172.21.15.47:0/1557728024 learned_addr learned my addr 172.21.15.47:0/1557728024 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:49:40.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.081+0000 7f6e2bfff700 1 --2- 172.21.15.47:0/1557728024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e2c0fe510 0x7f6e2c108070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:40.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.081+0000 7f6e2affd700 1 --2- 172.21.15.47:0/1557728024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e2c0fdbb0 0x7f6e2c10e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:40.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.081+0000 7f6e2b7fe700 1 -- 172.21.15.47:0/1557728024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e2c0fdbb0 msgr2=0x7f6e2c10e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:40.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.081+0000 7f6e2b7fe700 1 --2- 172.21.15.47:0/1557728024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e2c0fdbb0 0x7f6e2c10e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:40.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.081+0000 7f6e2b7fe700 1 -- 172.21.15.47:0/1557728024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e2c0fe510 msgr2=0x7f6e2c108070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:40.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.081+0000 7f6e2b7fe700 1 --2- 172.21.15.47:0/1557728024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e2c0fe510 0x7f6e2c108070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:40.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.081+0000 7f6e2b7fe700 1 -- 172.21.15.47:0/1557728024 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e240097c0 con 0x7f6e2c0f8420 2024-02-23T00:49:40.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.081+0000 7f6e2affd700 1 --2- 172.21.15.47:0/1557728024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e2c0fdbb0 0x7f6e2c10e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:49:40.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.081+0000 7f6e2bfff700 1 --2- 172.21.15.47:0/1557728024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e2c0fe510 0x7f6e2c108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:49:40.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.082+0000 7f6e2b7fe700 1 --2- 172.21.15.47:0/1557728024 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6e2c0f8420 0x7f6e2c10ded0 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 rx=0x7f6e1c0041d0 tx=0x7f6e1c009f60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:40.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.082+0000 7f6e28ff9700 10 client.?.objecter ms_handle_connect 0x7f6e2c0f8420 2024-02-23T00:49:40.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.082+0000 7f6e28ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:49:40.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.082+0000 7f6e28ff9700 1 -- 172.21.15.47:0/1557728024 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c007b40 con 0x7f6e2c0f8420 2024-02-23T00:49:40.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.082+0000 7f6e28ff9700 1 -- 172.21.15.47:0/1557728024 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f6e1c00f040 con 0x7f6e2c0f8420 2024-02-23T00:49:40.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.082+0000 7f6e3219e700 1 -- 172.21.15.47:0/1557728024 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e20005040 con 0x7f6e2c0f8420 2024-02-23T00:49:40.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.082+0000 7f6e28ff9700 1 -- 172.21.15.47:0/1557728024 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c014730 con 0x7f6e2c0f8420 2024-02-23T00:49:40.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.082+0000 7f6e3219e700 10 client.41477.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:49:40.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.082+0000 7f6e3219e700 1 -- 172.21.15.47:0/1557728024 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e2c114d00 con 0x7f6e2c0f8420 2024-02-23T00:49:40.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.082+0000 7f6e3219e700 1 librados: init done 2024-02-23T00:49:40.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.083+0000 7f6e3219e700 10 client.41477.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:40.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.083+0000 7f6e3219e700 15 client.41477.objecter _session_command_op_assign -1 1 2024-02-23T00:49:40.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.083+0000 7f6e3219e700 20 client.41477.objecter _get_session osd=-1 returning homeless 2024-02-23T00:49:40.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.083+0000 7f6e3219e700 10 client.41477.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:49:40.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.083+0000 7f6e3219e700 1 -- 172.21.15.47:0/1557728024 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6e2c05ee80 con 0x7f6e2c0f8420 2024-02-23T00:49:40.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.083+0000 7f6e28ff9700 1 -- 172.21.15.47:0/1557728024 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f6e1c009110 con 0x7f6e2c0f8420 2024-02-23T00:49:40.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.083+0000 7f6e28ff9700 1 --2- 172.21.15.47:0/1557728024 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6e14062cc0 0x7f6e14065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:40.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.084+0000 7f6e28ff9700 1 -- 172.21.15.47:0/1557728024 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(465..465 src has 48..465) v4 ==== 9220+0+0 (secure 0 0 0) 0x7f6e1c07d6f0 con 0x7f6e2c0f8420 2024-02-23T00:49:40.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.084+0000 7f6e28ff9700 10 client.41477.objecter ms_dispatch 0x7f6e2c001550 osd_map(465..465 src has 48..465) v4 2024-02-23T00:49:40.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.084+0000 7f6e28ff9700 3 client.41477.objecter handle_osd_map got epochs [465,465] > 0 2024-02-23T00:49:40.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.084+0000 7f6e28ff9700 3 client.41477.objecter handle_osd_map decoding full epoch 465 2024-02-23T00:49:40.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.084+0000 7f6e2affd700 1 --2- 172.21.15.47:0/1557728024 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6e14062cc0 0x7f6e14065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:40.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.084+0000 7f6e28ff9700 10 client.41477.objecter checking command 1 2024-02-23T00:49:40.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.084+0000 7f6e28ff9700 1 --2- 172.21.15.47:0/1557728024 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f6e1406f8f0 0x7f6e14071d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:40.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.084+0000 7f6e28ff9700 20 client.41477.objecter _get_session s=0x7f6e1406f220 osd=6 3 2024-02-23T00:49:40.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.084+0000 7f6e28ff9700 20 client.41477.objecter put_session s=0x7f6e1406f220 osd=6 3 2024-02-23T00:49:40.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.084+0000 7f6e28ff9700 15 client.41477.objecter _session_command_op_remove -1 1 2024-02-23T00:49:40.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.084+0000 7f6e28ff9700 20 client.41477.objecter _get_session s=0x7f6e1406f220 osd=6 3 2024-02-23T00:49:40.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.084+0000 7f6e28ff9700 20 client.41477.objecter get_session s=0x7f6e1406f220 osd=6 3 2024-02-23T00:49:40.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.084+0000 7f6e28ff9700 15 client.41477.objecter _session_command_op_assign 6 1 2024-02-23T00:49:40.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.084+0000 7f6e28ff9700 20 client.41477.objecter put_session s=0x7f6e1406f220 osd=6 4 2024-02-23T00:49:40.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.084+0000 7f6e28ff9700 10 client.41477.objecter _send_command 1 2024-02-23T00:49:40.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.084+0000 7f6e28ff9700 1 -- 172.21.15.47:0/1557728024 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6e14072410 con 0x7f6e1406f8f0 2024-02-23T00:49:40.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.084+0000 7f6e28ff9700 20 client.41477.objecter dump_active .. 0 homeless 2024-02-23T00:49:40.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.084+0000 7f6e28ff9700 1 -- 172.21.15.47:0/1557728024 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_get_version_reply(handle=1 version=465) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e1c07fd10 con 0x7f6e2c0f8420 2024-02-23T00:49:40.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.084+0000 7f6e28ff9700 10 client.41477.objecter ms_handle_connect 0x7f6e14062cc0 2024-02-23T00:49:40.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.084+0000 7f6e2affd700 1 --2- 172.21.15.47:0/1557728024 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6e14062cc0 0x7f6e14065180 secure :-1 s=READY pgs=3088 cs=0 l=1 rev1=1 rx=0x7f6e20013830 tx=0x7f6e20009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:40.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.084+0000 7f6e2bfff700 1 --2- 172.21.15.47:0/1557728024 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f6e1406f8f0 0x7f6e14071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:40.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.084+0000 7f6e28ff9700 10 client.41477.objecter ms_handle_connect 0x7f6e1406f8f0 2024-02-23T00:49:40.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.084+0000 7f6e2bfff700 1 --2- 172.21.15.47:0/1557728024 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f6e1406f8f0 0x7f6e14071d90 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:40.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.085+0000 7f6e28ff9700 1 -- 172.21.15.47:0/1557728024 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f6e14072410 con 0x7f6e1406f8f0 2024-02-23T00:49:40.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.085+0000 7f6e28ff9700 10 client.41477.objecter ms_dispatch 0x7f6e2c001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:49:40.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.085+0000 7f6e28ff9700 10 client.41477.objecter _finish_command 1 = system:0 2024-02-23T00:49:40.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.085+0000 7f6e28ff9700 20 client.41477.objecter put_session s=0x7f6e1406f220 osd=6 4 2024-02-23T00:49:40.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.085+0000 7f6e28ff9700 15 client.41477.objecter _session_command_op_remove 6 1 2024-02-23T00:49:40.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.102+0000 7f6e3219e700 10 client.41477.objecter _submit_command 2 [{"prefix": "reset_purged_snaps_last"}] 2024-02-23T00:49:40.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.102+0000 7f6e3219e700 15 client.41477.objecter _session_command_op_assign -1 2 2024-02-23T00:49:40.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.102+0000 7f6e3219e700 20 client.41477.objecter _get_session s=0x7f6e1406f220 osd=6 3 2024-02-23T00:49:40.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.102+0000 7f6e3219e700 20 client.41477.objecter put_session s=0x7f6e1406f220 osd=6 3 2024-02-23T00:49:40.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.102+0000 7f6e3219e700 20 client.41477.objecter _get_session s=0x7f6e1406f220 osd=6 3 2024-02-23T00:49:40.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.102+0000 7f6e3219e700 15 client.41477.objecter _session_command_op_remove -1 2 2024-02-23T00:49:40.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.102+0000 7f6e3219e700 20 client.41477.objecter get_session s=0x7f6e1406f220 osd=6 3 2024-02-23T00:49:40.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.102+0000 7f6e3219e700 15 client.41477.objecter _session_command_op_assign 6 2 2024-02-23T00:49:40.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.102+0000 7f6e3219e700 20 client.41477.objecter put_session s=0x7f6e1406f220 osd=6 4 2024-02-23T00:49:40.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.102+0000 7f6e3219e700 10 client.41477.objecter _send_command 2 2024-02-23T00:49:40.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.102+0000 7f6e3219e700 1 -- 172.21.15.47:0/1557728024 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) v1 -- 0x7f6e2c106ed0 con 0x7f6e1406f8f0 2024-02-23T00:49:40.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.103+0000 7f6e28ff9700 1 -- 172.21.15.47:0/1557728024 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f6e2c106ed0 con 0x7f6e1406f8f0 2024-02-23T00:49:40.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.103+0000 7f6e28ff9700 10 client.41477.objecter ms_dispatch 0x7f6e2c001550 command_reply(tid 2: 0 ) v1 2024-02-23T00:49:40.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.103+0000 7f6e28ff9700 10 client.41477.objecter _finish_command 2 = system:0 2024-02-23T00:49:40.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.103+0000 7f6e28ff9700 20 client.41477.objecter put_session s=0x7f6e1406f220 osd=6 4 2024-02-23T00:49:40.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.103+0000 7f6e28ff9700 15 client.41477.objecter _session_command_op_remove 6 2 2024-02-23T00:49:40.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.103+0000 7f6e3219e700 10 librados: watch_flush enter 2024-02-23T00:49:40.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.103+0000 7f6e3219e700 10 librados: watch_flush exit 2024-02-23T00:49:40.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.103+0000 7f6e3219e700 10 client.41477.objecter close_session for osd.6 2024-02-23T00:49:40.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.103+0000 7f6e3219e700 1 -- 172.21.15.47:0/1557728024 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f6e1406f8f0 msgr2=0x7f6e14071d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:40.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.103+0000 7f6e3219e700 1 --2- 172.21.15.47:0/1557728024 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f6e1406f8f0 0x7f6e14071d90 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:40.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.103+0000 7f6e3219e700 20 client.41477.objecter put_session s=0x7f6e1406f220 osd=6 1 2024-02-23T00:49:40.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.103+0000 7f6e3219e700 20 client.41477.objecter shutdown clearing up homeless session... 2024-02-23T00:49:40.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.103+0000 7f6e3219e700 10 client.41477.objecter successfully canceled tick 2024-02-23T00:49:40.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.103+0000 7f6e3219e700 1 -- 172.21.15.47:0/1557728024 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6e14062cc0 msgr2=0x7f6e14065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:40.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.103+0000 7f6e3219e700 1 --2- 172.21.15.47:0/1557728024 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6e14062cc0 0x7f6e14065180 secure :-1 s=READY pgs=3088 cs=0 l=1 rev1=1 rx=0x7f6e20013830 tx=0x7f6e20009cb0).stop 2024-02-23T00:49:40.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.103+0000 7f6e3219e700 1 -- 172.21.15.47:0/1557728024 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6e2c0f8420 msgr2=0x7f6e2c10ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:40.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.103+0000 7f6e3219e700 1 --2- 172.21.15.47:0/1557728024 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6e2c0f8420 0x7f6e2c10ded0 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 rx=0x7f6e1c0041d0 tx=0x7f6e1c009f60).stop 2024-02-23T00:49:40.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.103+0000 7f6e2bfff700 1 -- 172.21.15.47:0/1557728024 reap_dead start 2024-02-23T00:49:40.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.103+0000 7f6e3219e700 1 -- 172.21.15.47:0/1557728024 shutdown_connections 2024-02-23T00:49:40.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.103+0000 7f6e3219e700 1 -- 172.21.15.47:0/1557728024 >> 172.21.15.47:0/1557728024 conn(0x7f6e2c0f3800 msgr2=0x7f6e2c0fb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:40.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.103+0000 7f6e3219e700 1 -- 172.21.15.47:0/1557728024 shutdown_connections 2024-02-23T00:49:40.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.104+0000 7f6e3219e700 1 -- 172.21.15.47:0/1557728024 wait complete. 2024-02-23T00:49:40.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:40.104+0000 7f6e3219e700 1 librados: shutdown 2024-02-23T00:49:40.161 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:49:40.190 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:40.208 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:49:40.211 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:40.211 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:49:40.312 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:49:40.340 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:40.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:39.310327+0000 mgr.y (mgr.14517) 437 : cluster [DBG] pgmap v694: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.0 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-02-23T00:49:40.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:39.632904+0000 osd.5 (osd.5) 202 : cluster [DBG] 5.15 scrub starts 2024-02-23T00:49:40.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:39.636402+0000 osd.5 (osd.5) 203 : cluster [DBG] 5.15 scrub ok 2024-02-23T00:49:40.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:39.310327+0000 mgr.y (mgr.14517) 437 : cluster [DBG] pgmap v694: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.0 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-02-23T00:49:40.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:39.632904+0000 osd.5 (osd.5) 202 : cluster [DBG] 5.15 scrub starts 2024-02-23T00:49:40.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:39.636402+0000 osd.5 (osd.5) 203 : cluster [DBG] 5.15 scrub ok 2024-02-23T00:49:40.637 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:39.310327+0000 mgr.y (mgr.14517) 437 : cluster [DBG] pgmap v694: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.0 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-02-23T00:49:40.637 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:39.632904+0000 osd.5 (osd.5) 202 : cluster [DBG] 5.15 scrub starts 2024-02-23T00:49:40.637 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:39.636402+0000 osd.5 (osd.5) 203 : cluster [DBG] 5.15 scrub ok 2024-02-23T00:49:40.655 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.0 dump_ops_in_flight 2024-02-23T00:49:40.687 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:49:40.690 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:40.690 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:49:40.791 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:49:40.807 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:49:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:49:40] "GET /metrics HTTP/1.1" 200 189207 "" "Prometheus/2.33.4" 2024-02-23T00:49:40.820 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:40.967 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set noscrub 2024-02-23T00:49:41.078 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:49:41.081 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:41.082 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:49:41.182 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:49:41.210 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:41.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:39.570003+0000 osd.1 (osd.1) 211 : cluster [DBG] 4.12 scrub starts 2024-02-23T00:49:41.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:39.573468+0000 osd.1 (osd.1) 212 : cluster [DBG] 4.12 scrub ok 2024-02-23T00:49:41.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:40.177149+0000 osd.7 (osd.7) 229 : cluster [DBG] 5.d scrub starts 2024-02-23T00:49:41.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:40.180612+0000 osd.7 (osd.7) 230 : cluster [DBG] 5.d scrub ok 2024-02-23T00:49:41.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:40.383147+0000 osd.4 (osd.4) 205 : cluster [DBG] 5.e scrub starts 2024-02-23T00:49:41.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:40.384603+0000 osd.4 (osd.4) 206 : cluster [DBG] 5.e scrub ok 2024-02-23T00:49:41.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:39.570003+0000 osd.1 (osd.1) 211 : cluster [DBG] 4.12 scrub starts 2024-02-23T00:49:41.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:39.573468+0000 osd.1 (osd.1) 212 : cluster [DBG] 4.12 scrub ok 2024-02-23T00:49:41.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:40.177149+0000 osd.7 (osd.7) 229 : cluster [DBG] 5.d scrub starts 2024-02-23T00:49:41.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:40.180612+0000 osd.7 (osd.7) 230 : cluster [DBG] 5.d scrub ok 2024-02-23T00:49:41.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:40.383147+0000 osd.4 (osd.4) 205 : cluster [DBG] 5.e scrub starts 2024-02-23T00:49:41.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:40.384603+0000 osd.4 (osd.4) 206 : cluster [DBG] 5.e scrub ok 2024-02-23T00:49:41.634 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph tell osd.7 reset_purged_snaps_last 2024-02-23T00:49:41.658 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:49:41.663 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:41.663 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:49:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:39.570003+0000 osd.1 (osd.1) 211 : cluster [DBG] 4.12 scrub starts 2024-02-23T00:49:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:39.573468+0000 osd.1 (osd.1) 212 : cluster [DBG] 4.12 scrub ok 2024-02-23T00:49:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:40.177149+0000 osd.7 (osd.7) 229 : cluster [DBG] 5.d scrub starts 2024-02-23T00:49:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:40.180612+0000 osd.7 (osd.7) 230 : cluster [DBG] 5.d scrub ok 2024-02-23T00:49:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:40.383147+0000 osd.4 (osd.4) 205 : cluster [DBG] 5.e scrub starts 2024-02-23T00:49:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:40.384603+0000 osd.4 (osd.4) 206 : cluster [DBG] 5.e scrub ok 2024-02-23T00:49:41.695 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:49:41.701 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:49:41.764 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:49:41.793 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:42.049 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:49:42.051 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:42.052 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:49:42.153 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:49:42.186 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:42.207 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:49:42.210 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:42.211 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:49:42.306 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:40.530373+0000 osd.1 (osd.1) 213 : cluster [DBG] 18.11 scrub starts 2024-02-23T00:49:42.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:40.533335+0000 osd.1 (osd.1) 214 : cluster [DBG] 18.11 scrub ok 2024-02-23T00:49:42.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:40.769847+0000 osd.0 (osd.0) 199 : cluster [DBG] 18.4 scrub starts 2024-02-23T00:49:42.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:40.770687+0000 osd.0 (osd.0) 200 : cluster [DBG] 18.4 scrub ok 2024-02-23T00:49:42.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:41.159907+0000 osd.7 (osd.7) 231 : cluster [DBG] 6.3 scrub starts 2024-02-23T00:49:42.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:41.162887+0000 osd.7 (osd.7) 232 : cluster [DBG] 6.3 scrub ok 2024-02-23T00:49:42.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:41.304418+0000 mon.a (mon.0) 2684 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:49:42.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:41.315807+0000 mgr.y (mgr.14517) 438 : cluster [DBG] pgmap v695: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.6 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-23T00:49:42.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:41.638639+0000 osd.5 (osd.5) 204 : cluster [DBG] 2.2 scrub starts 2024-02-23T00:49:42.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:41.639573+0000 osd.5 (osd.5) 205 : cluster [DBG] 2.2 scrub ok 2024-02-23T00:49:42.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:40.530373+0000 osd.1 (osd.1) 213 : cluster [DBG] 18.11 scrub starts 2024-02-23T00:49:42.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:40.533335+0000 osd.1 (osd.1) 214 : cluster [DBG] 18.11 scrub ok 2024-02-23T00:49:42.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:40.769847+0000 osd.0 (osd.0) 199 : cluster [DBG] 18.4 scrub starts 2024-02-23T00:49:42.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:40.770687+0000 osd.0 (osd.0) 200 : cluster [DBG] 18.4 scrub ok 2024-02-23T00:49:42.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:41.159907+0000 osd.7 (osd.7) 231 : cluster [DBG] 6.3 scrub starts 2024-02-23T00:49:42.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:41.162887+0000 osd.7 (osd.7) 232 : cluster [DBG] 6.3 scrub ok 2024-02-23T00:49:42.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:41.304418+0000 mon.a (mon.0) 2684 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:49:42.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:41.315807+0000 mgr.y (mgr.14517) 438 : cluster [DBG] pgmap v695: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.6 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-23T00:49:42.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:41.638639+0000 osd.5 (osd.5) 204 : cluster [DBG] 2.2 scrub starts 2024-02-23T00:49:42.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:41.639573+0000 osd.5 (osd.5) 205 : cluster [DBG] 2.2 scrub ok 2024-02-23T00:49:42.312 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:49:42.345 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:42.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:40.530373+0000 osd.1 (osd.1) 213 : cluster [DBG] 18.11 scrub starts 2024-02-23T00:49:42.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:40.533335+0000 osd.1 (osd.1) 214 : cluster [DBG] 18.11 scrub ok 2024-02-23T00:49:42.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:40.769847+0000 osd.0 (osd.0) 199 : cluster [DBG] 18.4 scrub starts 2024-02-23T00:49:42.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:40.770687+0000 osd.0 (osd.0) 200 : cluster [DBG] 18.4 scrub ok 2024-02-23T00:49:42.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:41.159907+0000 osd.7 (osd.7) 231 : cluster [DBG] 6.3 scrub starts 2024-02-23T00:49:42.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:41.162887+0000 osd.7 (osd.7) 232 : cluster [DBG] 6.3 scrub ok 2024-02-23T00:49:42.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:41.304418+0000 mon.a (mon.0) 2684 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:49:42.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:41.315807+0000 mgr.y (mgr.14517) 438 : cluster [DBG] pgmap v695: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.6 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-23T00:49:42.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:41.638639+0000 osd.5 (osd.5) 204 : cluster [DBG] 2.2 scrub starts 2024-02-23T00:49:42.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:41.639573+0000 osd.5 (osd.5) 205 : cluster [DBG] 2.2 scrub ok 2024-02-23T00:49:42.944 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:49:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:49:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:49:42.944 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:49:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:49:43.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:41.774590+0000 osd.0 (osd.0) 201 : cluster [DBG] 5.3 scrub starts 2024-02-23T00:49:43.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:41.776107+0000 osd.0 (osd.0) 202 : cluster [DBG] 5.3 scrub ok 2024-02-23T00:49:43.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:41.924601+0000 osd.7 (osd.7) 233 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:49:43.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:41.924912+0000 osd.7 (osd.7) 234 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:49:43.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:42.147992+0000 osd.7 (osd.7) 235 : cluster [DBG] 6.8 deep-scrub starts 2024-02-23T00:49:43.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:42.150870+0000 osd.7 (osd.7) 236 : cluster [DBG] 6.8 deep-scrub ok 2024-02-23T00:49:43.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:42.310955+0000 mon.a (mon.0) 2685 : cluster [DBG] osdmap e466: 8 total, 8 up, 7 in 2024-02-23T00:49:43.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:42.655313+0000 mgr.y (mgr.14517) 439 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:49:43.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:42.869778+0000 osd.3 (osd.3) 222 : cluster [DBG] 5.10 scrub starts 2024-02-23T00:49:43.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:42.872744+0000 osd.3 (osd.3) 223 : cluster [DBG] 5.10 scrub ok 2024-02-23T00:49:43.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:43.218610+0000 mon.a (mon.0) 2686 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "217.4", "id": [0, 3]}]: dispatch 2024-02-23T00:49:43.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:41.774590+0000 osd.0 (osd.0) 201 : cluster [DBG] 5.3 scrub starts 2024-02-23T00:49:43.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:41.776107+0000 osd.0 (osd.0) 202 : cluster [DBG] 5.3 scrub ok 2024-02-23T00:49:43.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:41.924601+0000 osd.7 (osd.7) 233 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:49:43.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:41.924912+0000 osd.7 (osd.7) 234 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:49:43.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:42.147992+0000 osd.7 (osd.7) 235 : cluster [DBG] 6.8 deep-scrub starts 2024-02-23T00:49:43.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:42.150870+0000 osd.7 (osd.7) 236 : cluster [DBG] 6.8 deep-scrub ok 2024-02-23T00:49:43.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:42.310955+0000 mon.a (mon.0) 2685 : cluster [DBG] osdmap e466: 8 total, 8 up, 7 in 2024-02-23T00:49:43.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:42.655313+0000 mgr.y (mgr.14517) 439 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:49:43.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:42.869778+0000 osd.3 (osd.3) 222 : cluster [DBG] 5.10 scrub starts 2024-02-23T00:49:43.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:42.872744+0000 osd.3 (osd.3) 223 : cluster [DBG] 5.10 scrub ok 2024-02-23T00:49:43.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:43.218610+0000 mon.a (mon.0) 2686 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "217.4", "id": [0, 3]}]: dispatch 2024-02-23T00:49:43.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:41.774590+0000 osd.0 (osd.0) 201 : cluster [DBG] 5.3 scrub starts 2024-02-23T00:49:43.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:41.776107+0000 osd.0 (osd.0) 202 : cluster [DBG] 5.3 scrub ok 2024-02-23T00:49:43.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:41.924601+0000 osd.7 (osd.7) 233 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:49:43.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:41.924912+0000 osd.7 (osd.7) 234 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:49:43.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:42.147992+0000 osd.7 (osd.7) 235 : cluster [DBG] 6.8 deep-scrub starts 2024-02-23T00:49:43.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:42.150870+0000 osd.7 (osd.7) 236 : cluster [DBG] 6.8 deep-scrub ok 2024-02-23T00:49:43.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:42.310955+0000 mon.a (mon.0) 2685 : cluster [DBG] osdmap e466: 8 total, 8 up, 7 in 2024-02-23T00:49:43.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:42.655313+0000 mgr.y (mgr.14517) 439 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:49:43.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:42.869778+0000 osd.3 (osd.3) 222 : cluster [DBG] 5.10 scrub starts 2024-02-23T00:49:43.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:42.872744+0000 osd.3 (osd.3) 223 : cluster [DBG] 5.10 scrub ok 2024-02-23T00:49:43.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:43.218610+0000 mon.a (mon.0) 2686 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "217.4", "id": [0, 3]}]: dispatch 2024-02-23T00:49:44.316 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:49:44.421 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:42.319002+0000 osd.6 (osd.6) 189 : cluster [DBG] 62.17 scrub starts 2024-02-23T00:49:44.422 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:42.320931+0000 osd.6 (osd.6) 190 : cluster [DBG] 62.17 scrub ok 2024-02-23T00:49:44.422 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:42.728953+0000 osd.0 (osd.0) 203 : cluster [DBG] 4.6 scrub starts 2024-02-23T00:49:44.422 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:42.730665+0000 osd.0 (osd.0) 204 : cluster [DBG] 4.6 scrub ok 2024-02-23T00:49:44.422 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:43.183949+0000 osd.7 (osd.7) 237 : cluster [DBG] 5.9 deep-scrub starts 2024-02-23T00:49:44.422 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:43.187984+0000 osd.7 (osd.7) 238 : cluster [DBG] 5.9 deep-scrub ok 2024-02-23T00:49:44.422 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:43.312579+0000 mon.a (mon.0) 2687 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "217.4", "id": [0, 3]}]': finished 2024-02-23T00:49:44.422 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:43.312656+0000 mon.a (mon.0) 2688 : cluster [DBG] osdmap e467: 8 total, 8 up, 7 in 2024-02-23T00:49:44.422 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:43.316244+0000 mgr.y (mgr.14517) 440 : cluster [DBG] pgmap v698: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.3 KiB/s rd, 503 B/s wr, 4 op/s 2024-02-23T00:49:44.423 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:42.319002+0000 osd.6 (osd.6) 189 : cluster [DBG] 62.17 scrub starts 2024-02-23T00:49:44.423 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:42.320931+0000 osd.6 (osd.6) 190 : cluster [DBG] 62.17 scrub ok 2024-02-23T00:49:44.423 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:42.728953+0000 osd.0 (osd.0) 203 : cluster [DBG] 4.6 scrub starts 2024-02-23T00:49:44.423 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:42.730665+0000 osd.0 (osd.0) 204 : cluster [DBG] 4.6 scrub ok 2024-02-23T00:49:44.423 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:43.183949+0000 osd.7 (osd.7) 237 : cluster [DBG] 5.9 deep-scrub starts 2024-02-23T00:49:44.423 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:43.187984+0000 osd.7 (osd.7) 238 : cluster [DBG] 5.9 deep-scrub ok 2024-02-23T00:49:44.423 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:43.312579+0000 mon.a (mon.0) 2687 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "217.4", "id": [0, 3]}]': finished 2024-02-23T00:49:44.423 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:43.312656+0000 mon.a (mon.0) 2688 : cluster [DBG] osdmap e467: 8 total, 8 up, 7 in 2024-02-23T00:49:44.423 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:43.316244+0000 mgr.y (mgr.14517) 440 : cluster [DBG] pgmap v698: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.3 KiB/s rd, 503 B/s wr, 4 op/s 2024-02-23T00:49:44.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:42.319002+0000 osd.6 (osd.6) 189 : cluster [DBG] 62.17 scrub starts 2024-02-23T00:49:44.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:42.320931+0000 osd.6 (osd.6) 190 : cluster [DBG] 62.17 scrub ok 2024-02-23T00:49:44.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:42.728953+0000 osd.0 (osd.0) 203 : cluster [DBG] 4.6 scrub starts 2024-02-23T00:49:44.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:42.730665+0000 osd.0 (osd.0) 204 : cluster [DBG] 4.6 scrub ok 2024-02-23T00:49:44.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:43.183949+0000 osd.7 (osd.7) 237 : cluster [DBG] 5.9 deep-scrub starts 2024-02-23T00:49:44.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:43.187984+0000 osd.7 (osd.7) 238 : cluster [DBG] 5.9 deep-scrub ok 2024-02-23T00:49:44.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:43.312579+0000 mon.a (mon.0) 2687 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "217.4", "id": [0, 3]}]': finished 2024-02-23T00:49:44.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:43.312656+0000 mon.a (mon.0) 2688 : cluster [DBG] osdmap e467: 8 total, 8 up, 7 in 2024-02-23T00:49:44.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:43.316244+0000 mgr.y (mgr.14517) 440 : cluster [DBG] pgmap v698: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.3 KiB/s rd, 503 B/s wr, 4 op/s 2024-02-23T00:49:45.478 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:44.142060+0000 osd.7 (osd.7) 239 : cluster [DBG] 18.1f scrub starts 2024-02-23T00:49:45.478 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:44.143679+0000 osd.7 (osd.7) 240 : cluster [DBG] 18.1f scrub ok 2024-02-23T00:49:45.478 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:44.318142+0000 mon.a (mon.0) 2689 : cluster [DBG] osdmap e468: 8 total, 8 up, 7 in 2024-02-23T00:49:45.478 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:44.435460+0000 mon.c (mon.2) 309 : audit [DBG] from='client.? 172.21.15.47:0/477861393' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:49:45.478 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:44.597976+0000 osd.5 (osd.5) 206 : cluster [DBG] 18.f deep-scrub starts 2024-02-23T00:49:45.478 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:44.601082+0000 osd.5 (osd.5) 207 : cluster [DBG] 18.f deep-scrub ok 2024-02-23T00:49:45.478 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:45.209878+0000 mon.a (mon.0) 2690 : cluster [DBG] osdmap e469: 8 total, 8 up, 7 in 2024-02-23T00:49:45.478 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:45.259284+0000 osd.6 (osd.6) 191 : cluster [DBG] 62.1c scrub starts 2024-02-23T00:49:45.478 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:45.261347+0000 osd.6 (osd.6) 192 : cluster [DBG] 62.1c scrub ok 2024-02-23T00:49:45.479 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:44.142060+0000 osd.7 (osd.7) 239 : cluster [DBG] 18.1f scrub starts 2024-02-23T00:49:45.479 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:44.143679+0000 osd.7 (osd.7) 240 : cluster [DBG] 18.1f scrub ok 2024-02-23T00:49:45.479 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:44.318142+0000 mon.a (mon.0) 2689 : cluster [DBG] osdmap e468: 8 total, 8 up, 7 in 2024-02-23T00:49:45.479 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:44.435460+0000 mon.c (mon.2) 309 : audit [DBG] from='client.? 172.21.15.47:0/477861393' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:49:45.479 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:44.597976+0000 osd.5 (osd.5) 206 : cluster [DBG] 18.f deep-scrub starts 2024-02-23T00:49:45.479 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:44.601082+0000 osd.5 (osd.5) 207 : cluster [DBG] 18.f deep-scrub ok 2024-02-23T00:49:45.479 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:45.209878+0000 mon.a (mon.0) 2690 : cluster [DBG] osdmap e469: 8 total, 8 up, 7 in 2024-02-23T00:49:45.479 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:45.259284+0000 osd.6 (osd.6) 191 : cluster [DBG] 62.1c scrub starts 2024-02-23T00:49:45.479 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:45.261347+0000 osd.6 (osd.6) 192 : cluster [DBG] 62.1c scrub ok 2024-02-23T00:49:45.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:44.142060+0000 osd.7 (osd.7) 239 : cluster [DBG] 18.1f scrub starts 2024-02-23T00:49:45.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:44.143679+0000 osd.7 (osd.7) 240 : cluster [DBG] 18.1f scrub ok 2024-02-23T00:49:45.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:44.318142+0000 mon.a (mon.0) 2689 : cluster [DBG] osdmap e468: 8 total, 8 up, 7 in 2024-02-23T00:49:45.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:44.435460+0000 mon.c (mon.2) 309 : audit [DBG] from='client.? 172.21.15.47:0/477861393' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:49:45.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:44.597976+0000 osd.5 (osd.5) 206 : cluster [DBG] 18.f deep-scrub starts 2024-02-23T00:49:45.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:44.601082+0000 osd.5 (osd.5) 207 : cluster [DBG] 18.f deep-scrub ok 2024-02-23T00:49:45.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:45.209878+0000 mon.a (mon.0) 2690 : cluster [DBG] osdmap e469: 8 total, 8 up, 7 in 2024-02-23T00:49:45.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:45.259284+0000 osd.6 (osd.6) 191 : cluster [DBG] 62.1c scrub starts 2024-02-23T00:49:45.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:45.261347+0000 osd.6 (osd.6) 192 : cluster [DBG] 62.1c scrub ok 2024-02-23T00:49:45.792 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:49:45.792 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:49:46.398 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:49:46.402 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:46.403 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:49:46.504 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:49:46.536 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:46.559 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:49:46.564 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:46.564 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:49:46.665 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:49:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:45.316588+0000 mgr.y (mgr.14517) 441 : cluster [DBG] pgmap v701: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:49:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:45.866818+0000 osd.3 (osd.3) 224 : cluster [DBG] 5.14 scrub starts 2024-02-23T00:49:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:45.868285+0000 osd.3 (osd.3) 225 : cluster [DBG] 5.14 scrub ok 2024-02-23T00:49:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:46.260894+0000 mon.a (mon.0) 2691 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:49:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:46.261247+0000 mon.a (mon.0) 2692 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-71"}]: dispatch 2024-02-23T00:49:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:46.305239+0000 osd.6 (osd.6) 193 : cluster [DBG] 62.1a scrub starts 2024-02-23T00:49:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:46.306970+0000 osd.6 (osd.6) 194 : cluster [DBG] 62.1a scrub ok 2024-02-23T00:49:46.695 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:45.316588+0000 mgr.y (mgr.14517) 441 : cluster [DBG] pgmap v701: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:49:46.695 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:45.866818+0000 osd.3 (osd.3) 224 : cluster [DBG] 5.14 scrub starts 2024-02-23T00:49:46.695 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:45.868285+0000 osd.3 (osd.3) 225 : cluster [DBG] 5.14 scrub ok 2024-02-23T00:49:46.695 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:46.260894+0000 mon.a (mon.0) 2691 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:49:46.695 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:46.261247+0000 mon.a (mon.0) 2692 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-71"}]: dispatch 2024-02-23T00:49:46.695 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:46.305239+0000 osd.6 (osd.6) 193 : cluster [DBG] 62.1a scrub starts 2024-02-23T00:49:46.695 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:46.306970+0000 osd.6 (osd.6) 194 : cluster [DBG] 62.1a scrub ok 2024-02-23T00:49:46.696 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:45.316588+0000 mgr.y (mgr.14517) 441 : cluster [DBG] pgmap v701: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:49:46.696 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:45.866818+0000 osd.3 (osd.3) 224 : cluster [DBG] 5.14 scrub starts 2024-02-23T00:49:46.696 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:45.868285+0000 osd.3 (osd.3) 225 : cluster [DBG] 5.14 scrub ok 2024-02-23T00:49:46.696 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:46.260894+0000 mon.a (mon.0) 2691 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:49:46.696 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:46.261247+0000 mon.a (mon.0) 2692 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-71"}]: dispatch 2024-02-23T00:49:46.696 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:46.305239+0000 osd.6 (osd.6) 193 : cluster [DBG] 62.1a scrub starts 2024-02-23T00:49:46.696 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:46.306970+0000 osd.6 (osd.6) 194 : cluster [DBG] 62.1a scrub ok 2024-02-23T00:49:46.698 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:46.900 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:49:46.903 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:46.903 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:49:47.004 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:49:47.032 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:47.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.431+0000 7f88af160700 1 -- 172.21.15.47:0/2971206817 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f88a80fd510 msgr2=0x7f88a80f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:47.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.431+0000 7f88af160700 1 --2- 172.21.15.47:0/2971206817 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f88a80fd510 0x7f88a80f6d40 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 rx=0x7f8898004870 tx=0x7f8898009cb0).stop 2024-02-23T00:49:47.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.431+0000 7f88af160700 1 -- 172.21.15.47:0/2971206817 shutdown_connections 2024-02-23T00:49:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.431+0000 7f88af160700 1 --2- 172.21.15.47:0/2971206817 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88a80f7400 0x7f88a8103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.431+0000 7f88af160700 1 --2- 172.21.15.47:0/2971206817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88a80fcbb0 0x7f88a80fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.431+0000 7f88af160700 1 --2- 172.21.15.47:0/2971206817 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f88a80fd510 0x7f88a80f6d40 unknown :-1 s=CLOSED pgs=2695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.431+0000 7f88af160700 1 -- 172.21.15.47:0/2971206817 >> 172.21.15.47:0/2971206817 conn(0x7f88a80f2820 msgr2=0x7f88a80f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.431+0000 7f88af160700 1 -- 172.21.15.47:0/2971206817 shutdown_connections 2024-02-23T00:49:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.431+0000 7f88af160700 1 -- 172.21.15.47:0/2971206817 wait complete. 2024-02-23T00:49:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.432+0000 7f88af160700 1 librados: starting msgr at 2024-02-23T00:49:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.432+0000 7f88af160700 1 librados: starting objecter 2024-02-23T00:49:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.432+0000 7f88af160700 1 Processor -- start 2024-02-23T00:49:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.432+0000 7f88af160700 1 -- start start 2024-02-23T00:49:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.432+0000 7f88af160700 1 librados: setting wanted keys 2024-02-23T00:49:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.432+0000 7f88af160700 1 librados: calling monclient init 2024-02-23T00:49:47.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.432+0000 7f88af160700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88a80f7400 0x7f88a810e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.432+0000 7f88af160700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88a80fcbb0 0x7f88a810e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.432+0000 7f88af160700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f88a80fd510 0x7f88a81081e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.432+0000 7f88af160700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a81086d0 con 0x7f88a80f7400 2024-02-23T00:49:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.432+0000 7f88af160700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a8108840 con 0x7f88a80fcbb0 2024-02-23T00:49:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.432+0000 7f88af160700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88a8108a00 con 0x7f88a80fd510 2024-02-23T00:49:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.432+0000 7f88ad6fd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f88a80fd510 0x7f88a81081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.432+0000 7f88acefc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88a80f7400 0x7f88a810e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.432+0000 7f88ad6fd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f88a80fd510 0x7f88a81081e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:51006/0 (socket says 172.21.15.47:51006) 2024-02-23T00:49:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.432+0000 7f88ad6fd700 1 -- 172.21.15.47:0/477861393 learned_addr learned my addr 172.21.15.47:0/477861393 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:49:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.433+0000 7f88a7fff700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88a80fcbb0 0x7f88a810e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.433+0000 7f88ad6fd700 1 -- 172.21.15.47:0/477861393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88a80fcbb0 msgr2=0x7f88a810e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.433+0000 7f88ad6fd700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88a80fcbb0 0x7f88a810e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.433+0000 7f88ad6fd700 1 -- 172.21.15.47:0/477861393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88a80f7400 msgr2=0x7f88a810e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.433+0000 7f88ad6fd700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88a80f7400 0x7f88a810e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.433+0000 7f88ad6fd700 1 -- 172.21.15.47:0/477861393 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a00097c0 con 0x7f88a80fd510 2024-02-23T00:49:47.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.433+0000 7f88acefc700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88a80f7400 0x7f88a810e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:49:47.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.433+0000 7f88ad6fd700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f88a80fd510 0x7f88a81081e0 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 rx=0x7f88a0001df0 tx=0x7f88a0004c50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:47.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.433+0000 7f88a5ffb700 10 client.?.objecter ms_handle_connect 0x7f88a80fd510 2024-02-23T00:49:47.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.433+0000 7f88a5ffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:49:47.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.433+0000 7f88a5ffb700 1 -- 172.21.15.47:0/477861393 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a0009a10 con 0x7f88a80fd510 2024-02-23T00:49:47.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.433+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8898005040 con 0x7f88a80fd510 2024-02-23T00:49:47.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.433+0000 7f88a5ffb700 1 -- 172.21.15.47:0/477861393 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f88a00090d0 con 0x7f88a80fd510 2024-02-23T00:49:47.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.433+0000 7f88a5ffb700 1 -- 172.21.15.47:0/477861393 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a0007b50 con 0x7f88a80fd510 2024-02-23T00:49:47.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.433+0000 7f88af160700 10 client.41483.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:49:47.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.433+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a8109010 con 0x7f88a80fd510 2024-02-23T00:49:47.368 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:49:47.368 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:49:47.368 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:49:47.368 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:49:47.368 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:49:47.368 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:49:47.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.433+0000 7f88af160700 1 librados: init done 2024-02-23T00:49:47.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.434+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f88a805fa60 con 0x7f88a80fd510 2024-02-23T00:49:47.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.435+0000 7f88a5ffb700 1 -- 172.21.15.47:0/477861393 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f88a0009240 con 0x7f88a80fd510 2024-02-23T00:49:47.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.435+0000 7f88a5ffb700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8894062cc0 0x7f8894065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:47.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.435+0000 7f88a5ffb700 1 -- 172.21.15.47:0/477861393 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(468..468 src has 48..468) v4 ==== 9249+0+0 (secure 0 0 0) 0x7f88a007e020 con 0x7f88a80fd510 2024-02-23T00:49:47.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.435+0000 7f88a5ffb700 10 client.41483.objecter ms_dispatch 0x7f88a8001550 osd_map(468..468 src has 48..468) v4 2024-02-23T00:49:47.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.435+0000 7f88a5ffb700 3 client.41483.objecter handle_osd_map got epochs [468,468] > 0 2024-02-23T00:49:47.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.435+0000 7f88a5ffb700 3 client.41483.objecter handle_osd_map decoding full epoch 468 2024-02-23T00:49:47.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.435+0000 7f88acefc700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8894062cc0 0x7f8894065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:47.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.435+0000 7f88a5ffb700 20 client.41483.objecter dump_active .. 0 homeless 2024-02-23T00:49:47.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.435+0000 7f88a5ffb700 1 -- 172.21.15.47:0/477861393 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v468) v1 ==== 54+0+16 (secure 0 0 0) 0x7f88a0085070 con 0x7f88a80fd510 2024-02-23T00:49:47.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.436+0000 7f88a5ffb700 10 client.41483.objecter ms_handle_connect 0x7f8894062cc0 2024-02-23T00:49:47.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.436+0000 7f88acefc700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8894062cc0 0x7f8894065180 secure :-1 s=READY pgs=3089 cs=0 l=1 rev1=1 rx=0x7f88900103c0 tx=0x7f889000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:47.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.436+0000 7f88af160700 10 client.41483.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:47.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.436+0000 7f88af160700 15 client.41483.objecter _session_command_op_assign -1 1 2024-02-23T00:49:47.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.436+0000 7f88af160700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f88a80ffed0 0x7f88a81002b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.436+0000 7f88af160700 20 client.41483.objecter _get_session s=0x7f88a805ee80 osd=0 3 2024-02-23T00:49:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.436+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a805ee80 osd=0 3 2024-02-23T00:49:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.436+0000 7f88af160700 20 client.41483.objecter _get_session s=0x7f88a805ee80 osd=0 3 2024-02-23T00:49:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.436+0000 7f88af160700 15 client.41483.objecter _session_command_op_remove -1 1 2024-02-23T00:49:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.436+0000 7f88af160700 20 client.41483.objecter get_session s=0x7f88a805ee80 osd=0 3 2024-02-23T00:49:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.436+0000 7f88af160700 15 client.41483.objecter _session_command_op_assign 0 1 2024-02-23T00:49:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.436+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a805ee80 osd=0 4 2024-02-23T00:49:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.436+0000 7f88af160700 10 client.41483.objecter _send_command 1 2024-02-23T00:49:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.436+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f88a8105ee0 con 0x7f88a80ffed0 2024-02-23T00:49:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.436+0000 7f88a7fff700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f88a80ffed0 0x7f88a81002b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.437+0000 7f88a5ffb700 10 client.41483.objecter ms_handle_connect 0x7f88a80ffed0 2024-02-23T00:49:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.437+0000 7f88a7fff700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f88a80ffed0 0x7f88a81002b0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.437+0000 7f88a5ffb700 1 -- 172.21.15.47:0/477861393 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f88a8105ee0 con 0x7f88a80ffed0 2024-02-23T00:49:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.437+0000 7f88a5ffb700 10 client.41483.objecter ms_dispatch 0x7f88a8001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:49:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.437+0000 7f88a5ffb700 10 client.41483.objecter _finish_command 1 = system:0 2024-02-23T00:49:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.437+0000 7f88a5ffb700 20 client.41483.objecter put_session s=0x7f88a805ee80 osd=0 4 2024-02-23T00:49:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.437+0000 7f88a5ffb700 15 client.41483.objecter _session_command_op_remove 0 1 2024-02-23T00:49:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.452+0000 7f88af160700 10 client.41483.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:49:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.452+0000 7f88af160700 15 client.41483.objecter _session_command_op_assign -1 2 2024-02-23T00:49:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.452+0000 7f88af160700 20 client.41483.objecter _get_session s=0x7f88a805ee80 osd=0 3 2024-02-23T00:49:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.452+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a805ee80 osd=0 3 2024-02-23T00:49:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.452+0000 7f88af160700 20 client.41483.objecter _get_session s=0x7f88a805ee80 osd=0 3 2024-02-23T00:49:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.452+0000 7f88af160700 15 client.41483.objecter _session_command_op_remove -1 2 2024-02-23T00:49:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.452+0000 7f88af160700 20 client.41483.objecter get_session s=0x7f88a805ee80 osd=0 3 2024-02-23T00:49:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.452+0000 7f88af160700 15 client.41483.objecter _session_command_op_assign 0 2 2024-02-23T00:49:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.452+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a805ee80 osd=0 4 2024-02-23T00:49:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.452+0000 7f88af160700 10 client.41483.objecter _send_command 2 2024-02-23T00:49:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.452+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f88a80273c0 con 0x7f88a80ffed0 2024-02-23T00:49:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.454+0000 7f88a5ffb700 1 -- 172.21.15.47:0/477861393 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f88a80273c0 con 0x7f88a80ffed0 2024-02-23T00:49:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.454+0000 7f88a5ffb700 10 client.41483.objecter ms_dispatch 0x7f88a8001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:49:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.454+0000 7f88a5ffb700 10 client.41483.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:49:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.454+0000 7f88a5ffb700 20 client.41483.objecter put_session s=0x7f88a805ee80 osd=0 4 2024-02-23T00:49:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.454+0000 7f88a5ffb700 15 client.41483.objecter _session_command_op_remove 0 2 2024-02-23T00:49:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.454+0000 7f88af160700 10 client.41483.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.454+0000 7f88af160700 15 client.41483.objecter _session_command_op_assign -1 3 2024-02-23T00:49:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.454+0000 7f88af160700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f88a80598f0 0x7f88a8059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.454+0000 7f88af160700 20 client.41483.objecter _get_session s=0x7f88a8028710 osd=1 3 2024-02-23T00:49:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.454+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a8028710 osd=1 3 2024-02-23T00:49:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.454+0000 7f88af160700 20 client.41483.objecter _get_session s=0x7f88a8028710 osd=1 3 2024-02-23T00:49:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.454+0000 7f88af160700 15 client.41483.objecter _session_command_op_remove -1 3 2024-02-23T00:49:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.454+0000 7f88af160700 20 client.41483.objecter get_session s=0x7f88a8028710 osd=1 3 2024-02-23T00:49:47.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.454+0000 7f88af160700 15 client.41483.objecter _session_command_op_assign 1 3 2024-02-23T00:49:47.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.454+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a8028710 osd=1 4 2024-02-23T00:49:47.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.454+0000 7f88af160700 10 client.41483.objecter _send_command 3 2024-02-23T00:49:47.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.454+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f88a811b650 con 0x7f88a80598f0 2024-02-23T00:49:47.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.454+0000 7f88acefc700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f88a80598f0 0x7f88a8059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:47.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.454+0000 7f88a5ffb700 10 client.41483.objecter ms_handle_connect 0x7f88a80598f0 2024-02-23T00:49:47.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.454+0000 7f88acefc700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f88a80598f0 0x7f88a8059cd0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:47.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.455+0000 7f88a5ffb700 1 -- 172.21.15.47:0/477861393 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f88a811b650 con 0x7f88a80598f0 2024-02-23T00:49:47.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.455+0000 7f88a5ffb700 10 client.41483.objecter ms_dispatch 0x7f88a8001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:49:47.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.455+0000 7f88a5ffb700 10 client.41483.objecter _finish_command 3 = system:0 2024-02-23T00:49:47.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.455+0000 7f88a5ffb700 20 client.41483.objecter put_session s=0x7f88a8028710 osd=1 4 2024-02-23T00:49:47.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.455+0000 7f88a5ffb700 15 client.41483.objecter _session_command_op_remove 1 3 2024-02-23T00:49:47.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.470+0000 7f88af160700 10 client.41483.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:49:47.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.470+0000 7f88af160700 15 client.41483.objecter _session_command_op_assign -1 4 2024-02-23T00:49:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.470+0000 7f88af160700 20 client.41483.objecter _get_session s=0x7f88a8028710 osd=1 3 2024-02-23T00:49:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.470+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a8028710 osd=1 3 2024-02-23T00:49:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.470+0000 7f88af160700 20 client.41483.objecter _get_session s=0x7f88a8028710 osd=1 3 2024-02-23T00:49:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.470+0000 7f88af160700 15 client.41483.objecter _session_command_op_remove -1 4 2024-02-23T00:49:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.470+0000 7f88af160700 20 client.41483.objecter get_session s=0x7f88a8028710 osd=1 3 2024-02-23T00:49:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.470+0000 7f88af160700 15 client.41483.objecter _session_command_op_assign 1 4 2024-02-23T00:49:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.470+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a8028710 osd=1 4 2024-02-23T00:49:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.470+0000 7f88af160700 10 client.41483.objecter _send_command 4 2024-02-23T00:49:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.470+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f88a8059360 con 0x7f88a80598f0 2024-02-23T00:49:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.472+0000 7f88a5ffb700 1 -- 172.21.15.47:0/477861393 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f88a8059360 con 0x7f88a80598f0 2024-02-23T00:49:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.472+0000 7f88a5ffb700 10 client.41483.objecter ms_dispatch 0x7f88a8001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:49:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.472+0000 7f88a5ffb700 10 client.41483.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:49:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.472+0000 7f88a5ffb700 20 client.41483.objecter put_session s=0x7f88a8028710 osd=1 4 2024-02-23T00:49:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.472+0000 7f88a5ffb700 15 client.41483.objecter _session_command_op_remove 1 4 2024-02-23T00:49:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.472+0000 7f88af160700 10 client.41483.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.472+0000 7f88af160700 15 client.41483.objecter _session_command_op_assign -1 5 2024-02-23T00:49:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.472+0000 7f88af160700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f88a8028270 0x7f88a811af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.472+0000 7f88af160700 20 client.41483.objecter _get_session s=0x7f88a8059770 osd=2 3 2024-02-23T00:49:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.472+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a8059770 osd=2 3 2024-02-23T00:49:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.472+0000 7f88af160700 20 client.41483.objecter _get_session s=0x7f88a8059770 osd=2 3 2024-02-23T00:49:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.472+0000 7f88af160700 15 client.41483.objecter _session_command_op_remove -1 5 2024-02-23T00:49:47.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.472+0000 7f88af160700 20 client.41483.objecter get_session s=0x7f88a8059770 osd=2 3 2024-02-23T00:49:47.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.472+0000 7f88af160700 15 client.41483.objecter _session_command_op_assign 2 5 2024-02-23T00:49:47.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.472+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a8059770 osd=2 4 2024-02-23T00:49:47.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.472+0000 7f88af160700 10 client.41483.objecter _send_command 5 2024-02-23T00:49:47.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.472+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f88a811b3f0 con 0x7f88a8028270 2024-02-23T00:49:47.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.472+0000 7f88a7fff700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f88a8028270 0x7f88a811af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:47.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.472+0000 7f88a5ffb700 10 client.41483.objecter ms_handle_connect 0x7f88a8028270 2024-02-23T00:49:47.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.472+0000 7f88a7fff700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f88a8028270 0x7f88a811af00 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:47.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.473+0000 7f88a5ffb700 1 -- 172.21.15.47:0/477861393 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f88a811b3f0 con 0x7f88a8028270 2024-02-23T00:49:47.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.473+0000 7f88a5ffb700 10 client.41483.objecter ms_dispatch 0x7f88a8001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:49:47.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.473+0000 7f88a5ffb700 10 client.41483.objecter _finish_command 5 = system:0 2024-02-23T00:49:47.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.473+0000 7f88a5ffb700 20 client.41483.objecter put_session s=0x7f88a8059770 osd=2 4 2024-02-23T00:49:47.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.473+0000 7f88a5ffb700 15 client.41483.objecter _session_command_op_remove 2 5 2024-02-23T00:49:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.488+0000 7f88af160700 10 client.41483.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:49:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.488+0000 7f88af160700 15 client.41483.objecter _session_command_op_assign -1 6 2024-02-23T00:49:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.488+0000 7f88af160700 20 client.41483.objecter _get_session s=0x7f88a8059770 osd=2 3 2024-02-23T00:49:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.488+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a8059770 osd=2 3 2024-02-23T00:49:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.488+0000 7f88af160700 20 client.41483.objecter _get_session s=0x7f88a8059770 osd=2 3 2024-02-23T00:49:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.488+0000 7f88af160700 15 client.41483.objecter _session_command_op_remove -1 6 2024-02-23T00:49:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.488+0000 7f88af160700 20 client.41483.objecter get_session s=0x7f88a8059770 osd=2 3 2024-02-23T00:49:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.488+0000 7f88af160700 15 client.41483.objecter _session_command_op_assign 2 6 2024-02-23T00:49:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.488+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a8059770 osd=2 4 2024-02-23T00:49:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.488+0000 7f88af160700 10 client.41483.objecter _send_command 6 2024-02-23T00:49:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.488+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f88a80277d0 con 0x7f88a8028270 2024-02-23T00:49:47.378 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:49:47.378 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:49:47.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.489+0000 7f88a5ffb700 1 -- 172.21.15.47:0/477861393 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f88a80277d0 con 0x7f88a8028270 2024-02-23T00:49:47.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.489+0000 7f88a5ffb700 10 client.41483.objecter ms_dispatch 0x7f88a8001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:49:47.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.489+0000 7f88a5ffb700 10 client.41483.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:49:47.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.489+0000 7f88a5ffb700 20 client.41483.objecter put_session s=0x7f88a8059770 osd=2 4 2024-02-23T00:49:47.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.489+0000 7f88a5ffb700 15 client.41483.objecter _session_command_op_remove 2 6 2024-02-23T00:49:47.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.489+0000 7f88af160700 10 client.41483.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:47.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.489+0000 7f88af160700 15 client.41483.objecter _session_command_op_assign -1 7 2024-02-23T00:49:47.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.489+0000 7f88af160700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f88a811fbd0 0x7f88a8121fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:47.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.489+0000 7f88af160700 20 client.41483.objecter _get_session s=0x7f88a8027be0 osd=3 3 2024-02-23T00:49:47.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.489+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a8027be0 osd=3 3 2024-02-23T00:49:47.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.489+0000 7f88af160700 20 client.41483.objecter _get_session s=0x7f88a8027be0 osd=3 3 2024-02-23T00:49:47.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.489+0000 7f88af160700 15 client.41483.objecter _session_command_op_remove -1 7 2024-02-23T00:49:47.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.489+0000 7f88af160700 20 client.41483.objecter get_session s=0x7f88a8027be0 osd=3 3 2024-02-23T00:49:47.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.489+0000 7f88af160700 15 client.41483.objecter _session_command_op_assign 3 7 2024-02-23T00:49:47.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.489+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a8027be0 osd=3 4 2024-02-23T00:49:47.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.489+0000 7f88af160700 10 client.41483.objecter _send_command 7 2024-02-23T00:49:47.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.489+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f88a80fa7b0 con 0x7f88a811fbd0 2024-02-23T00:49:47.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.489+0000 7f88ad6fd700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f88a811fbd0 0x7f88a8121fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:47.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.490+0000 7f88a5ffb700 10 client.41483.objecter ms_handle_connect 0x7f88a811fbd0 2024-02-23T00:49:47.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.490+0000 7f88ad6fd700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f88a811fbd0 0x7f88a8121fc0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:47.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.490+0000 7f88a5ffb700 1 -- 172.21.15.47:0/477861393 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f88a80fa7b0 con 0x7f88a811fbd0 2024-02-23T00:49:47.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.490+0000 7f88a5ffb700 10 client.41483.objecter ms_dispatch 0x7f88a8001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:49:47.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.490+0000 7f88a5ffb700 10 client.41483.objecter _finish_command 7 = system:0 2024-02-23T00:49:47.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.490+0000 7f88a5ffb700 20 client.41483.objecter put_session s=0x7f88a8027be0 osd=3 4 2024-02-23T00:49:47.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.490+0000 7f88a5ffb700 15 client.41483.objecter _session_command_op_remove 3 7 2024-02-23T00:49:47.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.505+0000 7f88af160700 10 client.41483.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:49:47.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.505+0000 7f88af160700 15 client.41483.objecter _session_command_op_assign -1 8 2024-02-23T00:49:47.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.505+0000 7f88af160700 20 client.41483.objecter _get_session s=0x7f88a8027be0 osd=3 3 2024-02-23T00:49:47.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.505+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a8027be0 osd=3 3 2024-02-23T00:49:47.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.505+0000 7f88af160700 20 client.41483.objecter _get_session s=0x7f88a8027be0 osd=3 3 2024-02-23T00:49:47.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.505+0000 7f88af160700 15 client.41483.objecter _session_command_op_remove -1 8 2024-02-23T00:49:47.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.505+0000 7f88af160700 20 client.41483.objecter get_session s=0x7f88a8027be0 osd=3 3 2024-02-23T00:49:47.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.505+0000 7f88af160700 15 client.41483.objecter _session_command_op_assign 3 8 2024-02-23T00:49:47.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.505+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a8027be0 osd=3 4 2024-02-23T00:49:47.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.505+0000 7f88af160700 10 client.41483.objecter _send_command 8 2024-02-23T00:49:47.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.505+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f88a80f9200 con 0x7f88a811fbd0 2024-02-23T00:49:47.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.506+0000 7f88a5ffb700 1 -- 172.21.15.47:0/477861393 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f88a80f9200 con 0x7f88a811fbd0 2024-02-23T00:49:47.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.506+0000 7f88a5ffb700 10 client.41483.objecter ms_dispatch 0x7f88a8001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:49:47.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.506+0000 7f88a5ffb700 10 client.41483.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:49:47.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.506+0000 7f88a5ffb700 20 client.41483.objecter put_session s=0x7f88a8027be0 osd=3 4 2024-02-23T00:49:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.506+0000 7f88a5ffb700 15 client.41483.objecter _session_command_op_remove 3 8 2024-02-23T00:49:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.506+0000 7f88af160700 10 client.41483.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.506+0000 7f88af160700 15 client.41483.objecter _session_command_op_assign -1 9 2024-02-23T00:49:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.506+0000 7f88af160700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f88a80f98e0 0x7f88a8125640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.506+0000 7f88af160700 20 client.41483.objecter _get_session s=0x7f88a80f9200 osd=4 3 2024-02-23T00:49:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.506+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a80f9200 osd=4 3 2024-02-23T00:49:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.506+0000 7f88af160700 20 client.41483.objecter _get_session s=0x7f88a80f9200 osd=4 3 2024-02-23T00:49:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.506+0000 7f88af160700 15 client.41483.objecter _session_command_op_remove -1 9 2024-02-23T00:49:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.506+0000 7f88af160700 20 client.41483.objecter get_session s=0x7f88a80f9200 osd=4 3 2024-02-23T00:49:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.506+0000 7f88af160700 15 client.41483.objecter _session_command_op_assign 4 9 2024-02-23T00:49:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.506+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a80f9200 osd=4 4 2024-02-23T00:49:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.506+0000 7f88af160700 10 client.41483.objecter _send_command 9 2024-02-23T00:49:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.506+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f88a80f9ef0 con 0x7f88a80f98e0 2024-02-23T00:49:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.507+0000 7f88acefc700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f88a80f98e0 0x7f88a8125640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:47.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.507+0000 7f88a5ffb700 10 client.41483.objecter ms_handle_connect 0x7f88a80f98e0 2024-02-23T00:49:47.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.507+0000 7f88acefc700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f88a80f98e0 0x7f88a8125640 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:47.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.508+0000 7f88a5ffb700 1 -- 172.21.15.47:0/477861393 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f88a80f9ef0 con 0x7f88a80f98e0 2024-02-23T00:49:47.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.508+0000 7f88a5ffb700 10 client.41483.objecter ms_dispatch 0x7f88a8001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:49:47.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.508+0000 7f88a5ffb700 10 client.41483.objecter _finish_command 9 = system:0 2024-02-23T00:49:47.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.508+0000 7f88a5ffb700 20 client.41483.objecter put_session s=0x7f88a80f9200 osd=4 4 2024-02-23T00:49:47.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.508+0000 7f88a5ffb700 15 client.41483.objecter _session_command_op_remove 4 9 2024-02-23T00:49:47.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.523+0000 7f88af160700 10 client.41483.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:49:47.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.523+0000 7f88af160700 15 client.41483.objecter _session_command_op_assign -1 10 2024-02-23T00:49:47.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.523+0000 7f88af160700 20 client.41483.objecter _get_session s=0x7f88a80f9200 osd=4 3 2024-02-23T00:49:47.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.523+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a80f9200 osd=4 3 2024-02-23T00:49:47.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.523+0000 7f88af160700 20 client.41483.objecter _get_session s=0x7f88a80f9200 osd=4 3 2024-02-23T00:49:47.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.523+0000 7f88af160700 15 client.41483.objecter _session_command_op_remove -1 10 2024-02-23T00:49:47.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.523+0000 7f88af160700 20 client.41483.objecter get_session s=0x7f88a80f9200 osd=4 3 2024-02-23T00:49:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.523+0000 7f88af160700 15 client.41483.objecter _session_command_op_assign 4 10 2024-02-23T00:49:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.523+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a80f9200 osd=4 4 2024-02-23T00:49:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.523+0000 7f88af160700 10 client.41483.objecter _send_command 10 2024-02-23T00:49:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.523+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f88a80f9ef0 con 0x7f88a80f98e0 2024-02-23T00:49:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.525+0000 7f88a5ffb700 1 -- 172.21.15.47:0/477861393 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f88a80f9ef0 con 0x7f88a80f98e0 2024-02-23T00:49:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.525+0000 7f88a5ffb700 10 client.41483.objecter ms_dispatch 0x7f88a8001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:49:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.525+0000 7f88a5ffb700 10 client.41483.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:49:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.525+0000 7f88a5ffb700 20 client.41483.objecter put_session s=0x7f88a80f9200 osd=4 4 2024-02-23T00:49:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.525+0000 7f88a5ffb700 15 client.41483.objecter _session_command_op_remove 4 10 2024-02-23T00:49:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.525+0000 7f88af160700 10 client.41483.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.525+0000 7f88af160700 15 client.41483.objecter _session_command_op_assign -1 11 2024-02-23T00:49:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.525+0000 7f88af160700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f88a8126040 0x7f88a8128430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.525+0000 7f88af160700 20 client.41483.objecter _get_session s=0x7f88a80f9ef0 osd=5 3 2024-02-23T00:49:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.525+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a80f9ef0 osd=5 3 2024-02-23T00:49:47.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.525+0000 7f88af160700 20 client.41483.objecter _get_session s=0x7f88a80f9ef0 osd=5 3 2024-02-23T00:49:47.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.525+0000 7f88af160700 15 client.41483.objecter _session_command_op_remove -1 11 2024-02-23T00:49:47.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.525+0000 7f88af160700 20 client.41483.objecter get_session s=0x7f88a80f9ef0 osd=5 3 2024-02-23T00:49:47.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.525+0000 7f88af160700 15 client.41483.objecter _session_command_op_assign 5 11 2024-02-23T00:49:47.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.525+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a80f9ef0 osd=5 4 2024-02-23T00:49:47.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.525+0000 7f88af160700 10 client.41483.objecter _send_command 11 2024-02-23T00:49:47.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.525+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f88a812b250 con 0x7f88a8126040 2024-02-23T00:49:47.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.525+0000 7f88a7fff700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f88a8126040 0x7f88a8128430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:47.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.526+0000 7f88a5ffb700 10 client.41483.objecter ms_handle_connect 0x7f88a8126040 2024-02-23T00:49:47.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.526+0000 7f88a7fff700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f88a8126040 0x7f88a8128430 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:47.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.526+0000 7f88a5ffb700 1 -- 172.21.15.47:0/477861393 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f88a812b250 con 0x7f88a8126040 2024-02-23T00:49:47.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.526+0000 7f88a5ffb700 10 client.41483.objecter ms_dispatch 0x7f88a8001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:49:47.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.526+0000 7f88a5ffb700 10 client.41483.objecter _finish_command 11 = system:0 2024-02-23T00:49:47.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.526+0000 7f88a5ffb700 20 client.41483.objecter put_session s=0x7f88a80f9ef0 osd=5 4 2024-02-23T00:49:47.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.526+0000 7f88a5ffb700 15 client.41483.objecter _session_command_op_remove 5 11 2024-02-23T00:49:47.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.541+0000 7f88af160700 10 client.41483.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:49:47.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.541+0000 7f88af160700 15 client.41483.objecter _session_command_op_assign -1 12 2024-02-23T00:49:47.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.541+0000 7f88af160700 20 client.41483.objecter _get_session s=0x7f88a80f9ef0 osd=5 3 2024-02-23T00:49:47.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.541+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a80f9ef0 osd=5 3 2024-02-23T00:49:47.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.541+0000 7f88af160700 20 client.41483.objecter _get_session s=0x7f88a80f9ef0 osd=5 3 2024-02-23T00:49:47.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.541+0000 7f88af160700 15 client.41483.objecter _session_command_op_remove -1 12 2024-02-23T00:49:47.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.541+0000 7f88af160700 20 client.41483.objecter get_session s=0x7f88a80f9ef0 osd=5 3 2024-02-23T00:49:47.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.541+0000 7f88af160700 15 client.41483.objecter _session_command_op_assign 5 12 2024-02-23T00:49:47.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.541+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a80f9ef0 osd=5 4 2024-02-23T00:49:47.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.541+0000 7f88af160700 10 client.41483.objecter _send_command 12 2024-02-23T00:49:47.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.541+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f88a812b250 con 0x7f88a8126040 2024-02-23T00:49:47.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.543+0000 7f88a5ffb700 1 -- 172.21.15.47:0/477861393 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f88a812b250 con 0x7f88a8126040 2024-02-23T00:49:47.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.543+0000 7f88a5ffb700 10 client.41483.objecter ms_dispatch 0x7f88a8001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:49:47.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.543+0000 7f88a5ffb700 10 client.41483.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:49:47.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.543+0000 7f88a5ffb700 20 client.41483.objecter put_session s=0x7f88a80f9ef0 osd=5 4 2024-02-23T00:49:47.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.543+0000 7f88a5ffb700 15 client.41483.objecter _session_command_op_remove 5 12 2024-02-23T00:49:47.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.543+0000 7f88af160700 10 client.41483.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:47.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.543+0000 7f88af160700 15 client.41483.objecter _session_command_op_assign -1 13 2024-02-23T00:49:47.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.543+0000 7f88af160700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f88a812a580 0x7f88a812a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:47.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.543+0000 7f88af160700 20 client.41483.objecter _get_session s=0x7f88a812b250 osd=6 3 2024-02-23T00:49:47.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.543+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a812b250 osd=6 3 2024-02-23T00:49:47.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.543+0000 7f88af160700 20 client.41483.objecter _get_session s=0x7f88a812b250 osd=6 3 2024-02-23T00:49:47.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.543+0000 7f88af160700 15 client.41483.objecter _session_command_op_remove -1 13 2024-02-23T00:49:47.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.543+0000 7f88af160700 20 client.41483.objecter get_session s=0x7f88a812b250 osd=6 3 2024-02-23T00:49:47.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.543+0000 7f88af160700 15 client.41483.objecter _session_command_op_assign 6 13 2024-02-23T00:49:47.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.543+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a812b250 osd=6 4 2024-02-23T00:49:47.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.543+0000 7f88af160700 10 client.41483.objecter _send_command 13 2024-02-23T00:49:47.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.543+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f88a812e5e0 con 0x7f88a812a580 2024-02-23T00:49:47.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.543+0000 7f88ad6fd700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f88a812a580 0x7f88a812a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:47.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.544+0000 7f88a5ffb700 10 client.41483.objecter ms_handle_connect 0x7f88a812a580 2024-02-23T00:49:47.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.544+0000 7f88ad6fd700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f88a812a580 0x7f88a812a9e0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:47.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.544+0000 7f88a5ffb700 1 -- 172.21.15.47:0/477861393 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f88a812e5e0 con 0x7f88a812a580 2024-02-23T00:49:47.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.544+0000 7f88a5ffb700 10 client.41483.objecter ms_dispatch 0x7f88a8001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:49:47.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.544+0000 7f88a5ffb700 10 client.41483.objecter _finish_command 13 = system:0 2024-02-23T00:49:47.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.544+0000 7f88a5ffb700 20 client.41483.objecter put_session s=0x7f88a812b250 osd=6 4 2024-02-23T00:49:47.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.544+0000 7f88a5ffb700 15 client.41483.objecter _session_command_op_remove 6 13 2024-02-23T00:49:47.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.559+0000 7f88af160700 10 client.41483.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:49:47.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.559+0000 7f88af160700 15 client.41483.objecter _session_command_op_assign -1 14 2024-02-23T00:49:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.559+0000 7f88af160700 20 client.41483.objecter _get_session s=0x7f88a812b250 osd=6 3 2024-02-23T00:49:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.559+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a812b250 osd=6 3 2024-02-23T00:49:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.559+0000 7f88af160700 20 client.41483.objecter _get_session s=0x7f88a812b250 osd=6 3 2024-02-23T00:49:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.559+0000 7f88af160700 15 client.41483.objecter _session_command_op_remove -1 14 2024-02-23T00:49:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.559+0000 7f88af160700 20 client.41483.objecter get_session s=0x7f88a812b250 osd=6 3 2024-02-23T00:49:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.559+0000 7f88af160700 15 client.41483.objecter _session_command_op_assign 6 14 2024-02-23T00:49:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.559+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a812b250 osd=6 4 2024-02-23T00:49:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.559+0000 7f88af160700 10 client.41483.objecter _send_command 14 2024-02-23T00:49:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.559+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f88a8128a10 con 0x7f88a812a580 2024-02-23T00:49:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.561+0000 7f88a5ffb700 1 -- 172.21.15.47:0/477861393 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f88a8128a10 con 0x7f88a812a580 2024-02-23T00:49:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.561+0000 7f88a5ffb700 10 client.41483.objecter ms_dispatch 0x7f88a8001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:49:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.561+0000 7f88a5ffb700 10 client.41483.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:49:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.561+0000 7f88a5ffb700 20 client.41483.objecter put_session s=0x7f88a812b250 osd=6 4 2024-02-23T00:49:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.561+0000 7f88a5ffb700 15 client.41483.objecter _session_command_op_remove 6 14 2024-02-23T00:49:47.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.561+0000 7f88af160700 10 client.41483.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:47.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.561+0000 7f88af160700 15 client.41483.objecter _session_command_op_assign -1 15 2024-02-23T00:49:47.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.561+0000 7f88af160700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f88a8129090 0x7f88a81294d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:47.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.561+0000 7f88af160700 20 client.41483.objecter _get_session s=0x7f88a8128a10 osd=7 3 2024-02-23T00:49:47.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.561+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a8128a10 osd=7 3 2024-02-23T00:49:47.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.561+0000 7f88af160700 20 client.41483.objecter _get_session s=0x7f88a8128a10 osd=7 3 2024-02-23T00:49:47.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.561+0000 7f88af160700 15 client.41483.objecter _session_command_op_remove -1 15 2024-02-23T00:49:47.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.561+0000 7f88af160700 20 client.41483.objecter get_session s=0x7f88a8128a10 osd=7 3 2024-02-23T00:49:47.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.561+0000 7f88af160700 15 client.41483.objecter _session_command_op_assign 7 15 2024-02-23T00:49:47.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.561+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a8128a10 osd=7 4 2024-02-23T00:49:47.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.561+0000 7f88af160700 10 client.41483.objecter _send_command 15 2024-02-23T00:49:47.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.561+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f88a8129b80 con 0x7f88a8129090 2024-02-23T00:49:47.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.561+0000 7f88acefc700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f88a8129090 0x7f88a81294d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:47.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.561+0000 7f88a5ffb700 10 client.41483.objecter ms_handle_connect 0x7f88a8129090 2024-02-23T00:49:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.561+0000 7f88acefc700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f88a8129090 0x7f88a81294d0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.562+0000 7f88a5ffb700 1 -- 172.21.15.47:0/477861393 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f88a8129b80 con 0x7f88a8129090 2024-02-23T00:49:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.562+0000 7f88a5ffb700 10 client.41483.objecter ms_dispatch 0x7f88a8001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:49:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.562+0000 7f88a5ffb700 10 client.41483.objecter _finish_command 15 = system:0 2024-02-23T00:49:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.562+0000 7f88a5ffb700 20 client.41483.objecter put_session s=0x7f88a8128a10 osd=7 4 2024-02-23T00:49:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.562+0000 7f88a5ffb700 15 client.41483.objecter _session_command_op_remove 7 15 2024-02-23T00:49:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.577+0000 7f88af160700 10 client.41483.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:49:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.577+0000 7f88af160700 15 client.41483.objecter _session_command_op_assign -1 16 2024-02-23T00:49:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.577+0000 7f88af160700 20 client.41483.objecter _get_session s=0x7f88a8128a10 osd=7 3 2024-02-23T00:49:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.577+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a8128a10 osd=7 3 2024-02-23T00:49:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.577+0000 7f88af160700 20 client.41483.objecter _get_session s=0x7f88a8128a10 osd=7 3 2024-02-23T00:49:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.577+0000 7f88af160700 15 client.41483.objecter _session_command_op_remove -1 16 2024-02-23T00:49:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.577+0000 7f88af160700 20 client.41483.objecter get_session s=0x7f88a8128a10 osd=7 3 2024-02-23T00:49:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.577+0000 7f88af160700 15 client.41483.objecter _session_command_op_assign 7 16 2024-02-23T00:49:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.577+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a8128a10 osd=7 4 2024-02-23T00:49:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.577+0000 7f88af160700 10 client.41483.objecter _send_command 16 2024-02-23T00:49:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.577+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f88a8129dc0 con 0x7f88a8129090 2024-02-23T00:49:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.578+0000 7f88a5ffb700 1 -- 172.21.15.47:0/477861393 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f88a8129dc0 con 0x7f88a8129090 2024-02-23T00:49:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.578+0000 7f88a5ffb700 10 client.41483.objecter ms_dispatch 0x7f88a8001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:49:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.578+0000 7f88a5ffb700 10 client.41483.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:49:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.578+0000 7f88a5ffb700 20 client.41483.objecter put_session s=0x7f88a8128a10 osd=7 4 2024-02-23T00:49:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.578+0000 7f88a5ffb700 15 client.41483.objecter _session_command_op_remove 7 16 2024-02-23T00:49:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 10 librados: watch_flush enter 2024-02-23T00:49:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 10 librados: watch_flush exit 2024-02-23T00:49:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 10 client.41483.objecter close_session for osd.0 2024-02-23T00:49:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f88a80ffed0 msgr2=0x7f88a81002b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f88a80ffed0 0x7f88a81002b0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:47.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a805ee80 osd=0 1 2024-02-23T00:49:47.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 10 client.41483.objecter close_session for osd.1 2024-02-23T00:49:47.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f88a80598f0 msgr2=0x7f88a8059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:47.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f88a80598f0 0x7f88a8059cd0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:47.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a8028710 osd=1 1 2024-02-23T00:49:47.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 10 client.41483.objecter close_session for osd.2 2024-02-23T00:49:47.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f88a8028270 msgr2=0x7f88a811af00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:47.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f88a8028270 0x7f88a811af00 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:47.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a8059770 osd=2 1 2024-02-23T00:49:47.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 10 client.41483.objecter close_session for osd.3 2024-02-23T00:49:47.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f88a811fbd0 msgr2=0x7f88a8121fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:47.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f88a811fbd0 0x7f88a8121fc0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:47.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88ad6fd700 1 -- 172.21.15.47:0/477861393 reap_dead start 2024-02-23T00:49:47.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a8027be0 osd=3 1 2024-02-23T00:49:47.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 10 client.41483.objecter close_session for osd.4 2024-02-23T00:49:47.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f88a80f98e0 msgr2=0x7f88a8125640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:47.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f88a80f98e0 0x7f88a8125640 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:47.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a80f9200 osd=4 1 2024-02-23T00:49:47.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 10 client.41483.objecter close_session for osd.5 2024-02-23T00:49:47.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88ad6fd700 1 -- 172.21.15.47:0/477861393 reap_dead start 2024-02-23T00:49:47.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f88a8126040 msgr2=0x7f88a8128430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:47.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f88a8126040 0x7f88a8128430 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:47.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a80f9ef0 osd=5 1 2024-02-23T00:49:47.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 10 client.41483.objecter close_session for osd.6 2024-02-23T00:49:47.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f88a812a580 msgr2=0x7f88a812a9e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:47.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f88a812a580 0x7f88a812a9e0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:47.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a812b250 osd=6 1 2024-02-23T00:49:47.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 10 client.41483.objecter close_session for osd.7 2024-02-23T00:49:47.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f88a8129090 msgr2=0x7f88a81294d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:47.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f88a8129090 0x7f88a81294d0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:47.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 20 client.41483.objecter put_session s=0x7f88a8128a10 osd=7 1 2024-02-23T00:49:47.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 20 client.41483.objecter shutdown clearing up homeless session... 2024-02-23T00:49:47.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 10 client.41483.objecter successfully canceled tick 2024-02-23T00:49:47.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8894062cc0 msgr2=0x7f8894065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:47.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8894062cc0 0x7f8894065180 secure :-1 s=READY pgs=3089 cs=0 l=1 rev1=1 rx=0x7f88900103c0 tx=0x7f889000b040).stop 2024-02-23T00:49:47.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f88a80fd510 msgr2=0x7f88a81081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:47.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 1 --2- 172.21.15.47:0/477861393 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f88a80fd510 0x7f88a81081e0 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 rx=0x7f88a0001df0 tx=0x7f88a0004c50).stop 2024-02-23T00:49:47.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88ad6fd700 1 -- 172.21.15.47:0/477861393 reap_dead start 2024-02-23T00:49:47.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 shutdown_connections 2024-02-23T00:49:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 >> 172.21.15.47:0/477861393 conn(0x7f88a80f2820 msgr2=0x7f88a80f43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 shutdown_connections 2024-02-23T00:49:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 1 -- 172.21.15.47:0/477861393 wait complete. 2024-02-23T00:49:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:44.579+0000 7f88af160700 1 librados: shutdown 2024-02-23T00:49:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:49:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:49:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:49:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:49:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:49:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:49:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:49:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:49:47.583 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:49:47.586 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:47.586 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:49:47.687 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:49:47.717 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:47.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.781+0000 7f89936a1700 1 -- 172.21.15.47:0/3090491324 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f898c0f9670 msgr2=0x7f898c101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:47.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.781+0000 7f89936a1700 1 --2- 172.21.15.47:0/3090491324 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f898c0f9670 0x7f898c101d80 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 rx=0x7f897c004870 tx=0x7f897c009cb0).stop 2024-02-23T00:49:47.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.781+0000 7f89936a1700 1 -- 172.21.15.47:0/3090491324 shutdown_connections 2024-02-23T00:49:47.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.781+0000 7f89936a1700 1 --2- 172.21.15.47:0/3090491324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f898c102270 0x7f898c104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:47.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.781+0000 7f89936a1700 1 --2- 172.21.15.47:0/3090491324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f898c0f8d10 0x7f898c0f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:47.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.781+0000 7f89936a1700 1 --2- 172.21.15.47:0/3090491324 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f898c0f9670 0x7f898c101d80 unknown :-1 s=CLOSED pgs=2697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:47.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.781+0000 7f89936a1700 1 -- 172.21.15.47:0/3090491324 >> 172.21.15.47:0/3090491324 conn(0x7f898c0f2820 msgr2=0x7f898c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:47.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.782+0000 7f89936a1700 1 -- 172.21.15.47:0/3090491324 shutdown_connections 2024-02-23T00:49:47.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.782+0000 7f89936a1700 1 -- 172.21.15.47:0/3090491324 wait complete. 2024-02-23T00:49:47.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.782+0000 7f89936a1700 1 librados: starting msgr at 2024-02-23T00:49:47.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.782+0000 7f89936a1700 1 librados: starting objecter 2024-02-23T00:49:47.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.782+0000 7f89936a1700 1 Processor -- start 2024-02-23T00:49:47.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.782+0000 7f89936a1700 1 -- start start 2024-02-23T00:49:47.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.782+0000 7f89936a1700 1 librados: setting wanted keys 2024-02-23T00:49:47.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.782+0000 7f89936a1700 1 librados: calling monclient init 2024-02-23T00:49:47.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.782+0000 7f89936a1700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f898c0f8d10 0x7f898c1101f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.782+0000 7f89936a1700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f898c0f9670 0x7f898c1106e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.782+0000 7f89936a1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f898c102270 0x7f898c10a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.782+0000 7f89936a1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c10a8c0 con 0x7f898c102270 2024-02-23T00:49:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.782+0000 7f89936a1700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c10aa30 con 0x7f898c0f9670 2024-02-23T00:49:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.782+0000 7f89936a1700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f898c10aba0 con 0x7f898c0f8d10 2024-02-23T00:49:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.782+0000 7f899143d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f898c0f8d10 0x7f898c1101f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.783+0000 7f899143d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f898c0f8d10 0x7f898c1101f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:47894/0 (socket says 172.21.15.47:47894) 2024-02-23T00:49:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.783+0000 7f899143d700 1 -- 172.21.15.47:0/489758656 learned_addr learned my addr 172.21.15.47:0/489758656 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:49:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.783+0000 7f8991c3e700 1 --2- 172.21.15.47:0/489758656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f898c102270 0x7f898c10a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.783+0000 7f8990c3c700 1 --2- 172.21.15.47:0/489758656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f898c0f9670 0x7f898c1106e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.783+0000 7f899143d700 1 -- 172.21.15.47:0/489758656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f898c0f9670 msgr2=0x7f898c1106e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.783+0000 7f899143d700 1 --2- 172.21.15.47:0/489758656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f898c0f9670 0x7f898c1106e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.783+0000 7f899143d700 1 -- 172.21.15.47:0/489758656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f898c102270 msgr2=0x7f898c10a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:47.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.783+0000 7f899143d700 1 --2- 172.21.15.47:0/489758656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f898c102270 0x7f898c10a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:47.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.783+0000 7f899143d700 1 -- 172.21.15.47:0/489758656 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8988005040 con 0x7f898c0f8d10 2024-02-23T00:49:47.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.783+0000 7f899143d700 1 --2- 172.21.15.47:0/489758656 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f898c0f8d10 0x7f898c1101f0 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 rx=0x7f89880048f0 tx=0x7f8988009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:47.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.783+0000 7f89827fc700 10 client.?.objecter ms_handle_connect 0x7f898c0f8d10 2024-02-23T00:49:47.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.783+0000 7f89827fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:49:47.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.783+0000 7f89827fc700 1 -- 172.21.15.47:0/489758656 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8988007b00 con 0x7f898c0f8d10 2024-02-23T00:49:47.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.783+0000 7f89936a1700 1 -- 172.21.15.47:0/489758656 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f897c005040 con 0x7f898c0f8d10 2024-02-23T00:49:47.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.783+0000 7f89827fc700 1 -- 172.21.15.47:0/489758656 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f898800f040 con 0x7f898c0f8d10 2024-02-23T00:49:47.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.783+0000 7f89827fc700 1 -- 172.21.15.47:0/489758656 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8988014890 con 0x7f898c0f8d10 2024-02-23T00:49:47.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.783+0000 7f89936a1700 10 client.41489.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:49:47.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.783+0000 7f89936a1700 1 -- 172.21.15.47:0/489758656 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f898c10b1e0 con 0x7f898c0f8d10 2024-02-23T00:49:47.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.783+0000 7f89936a1700 1 librados: init done 2024-02-23T00:49:47.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.784+0000 7f89936a1700 1 -- 172.21.15.47:0/489758656 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f898c05fa60 con 0x7f898c0f8d10 2024-02-23T00:49:47.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.785+0000 7f89827fc700 1 -- 172.21.15.47:0/489758656 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f8988014cc0 con 0x7f898c0f8d10 2024-02-23T00:49:47.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.785+0000 7f89827fc700 1 --2- 172.21.15.47:0/489758656 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8978062cc0 0x7f8978065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:47.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.785+0000 7f89827fc700 1 -- 172.21.15.47:0/489758656 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(470..470 src has 48..470) v4 ==== 8835+0+0 (secure 0 0 0) 0x7f898807d0e0 con 0x7f898c0f8d10 2024-02-23T00:49:47.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.785+0000 7f89827fc700 10 client.41489.objecter ms_dispatch 0x7f898c001550 osd_map(470..470 src has 48..470) v4 2024-02-23T00:49:47.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.785+0000 7f89827fc700 3 client.41489.objecter handle_osd_map got epochs [470,470] > 0 2024-02-23T00:49:47.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.785+0000 7f89827fc700 3 client.41489.objecter handle_osd_map decoding full epoch 470 2024-02-23T00:49:47.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.785+0000 7f8990c3c700 1 --2- 172.21.15.47:0/489758656 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8978062cc0 0x7f8978065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:47.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.785+0000 7f89827fc700 20 client.41489.objecter dump_active .. 0 homeless 2024-02-23T00:49:47.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.786+0000 7f89827fc700 10 client.41489.objecter ms_handle_connect 0x7f8978062cc0 2024-02-23T00:49:47.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.786+0000 7f8990c3c700 1 --2- 172.21.15.47:0/489758656 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8978062cc0 0x7f8978065180 secure :-1 s=READY pgs=3090 cs=0 l=1 rev1=1 rx=0x7f897c004ee0 tx=0x7f897c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:47.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.790+0000 7f89827fc700 1 -- 172.21.15.47:0/489758656 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f8988049f60 con 0x7f898c0f8d10 2024-02-23T00:49:47.844 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:49:47.846 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:46.159499+0000 osd.7 (osd.7) 241 : cluster [DBG] 2.0 scrub starts 2024-02-23T00:49:47.846 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:46.162440+0000 osd.7 (osd.7) 242 : cluster [DBG] 2.0 scrub ok 2024-02-23T00:49:47.847 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:46.362844+0000 osd.4 (osd.4) 207 : cluster [DBG] 4.1a scrub starts 2024-02-23T00:49:47.847 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:46.364411+0000 osd.4 (osd.4) 208 : cluster [DBG] 4.1a scrub ok 2024-02-23T00:49:47.847 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:46.642487+0000 osd.0 (osd.0) 205 : cluster [DBG] 4.3 deep-scrub starts 2024-02-23T00:49:47.847 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:46.644907+0000 osd.0 (osd.0) 206 : cluster [DBG] 4.3 deep-scrub ok 2024-02-23T00:49:47.847 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:47.212590+0000 mon.a (mon.0) 2693 : cluster [DBG] osdmap e470: 8 total, 8 up, 7 in 2024-02-23T00:49:47.847 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:46.159499+0000 osd.7 (osd.7) 241 : cluster [DBG] 2.0 scrub starts 2024-02-23T00:49:47.847 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:46.162440+0000 osd.7 (osd.7) 242 : cluster [DBG] 2.0 scrub ok 2024-02-23T00:49:47.847 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:46.362844+0000 osd.4 (osd.4) 207 : cluster [DBG] 4.1a scrub starts 2024-02-23T00:49:47.847 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:46.364411+0000 osd.4 (osd.4) 208 : cluster [DBG] 4.1a scrub ok 2024-02-23T00:49:47.847 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:46.642487+0000 osd.0 (osd.0) 205 : cluster [DBG] 4.3 deep-scrub starts 2024-02-23T00:49:47.847 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:46.644907+0000 osd.0 (osd.0) 206 : cluster [DBG] 4.3 deep-scrub ok 2024-02-23T00:49:47.848 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:47.212590+0000 mon.a (mon.0) 2693 : cluster [DBG] osdmap e470: 8 total, 8 up, 7 in 2024-02-23T00:49:47.848 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:47.848 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:49:47.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:46.159499+0000 osd.7 (osd.7) 241 : cluster [DBG] 2.0 scrub starts 2024-02-23T00:49:47.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:46.162440+0000 osd.7 (osd.7) 242 : cluster [DBG] 2.0 scrub ok 2024-02-23T00:49:47.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:46.362844+0000 osd.4 (osd.4) 207 : cluster [DBG] 4.1a scrub starts 2024-02-23T00:49:47.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:46.364411+0000 osd.4 (osd.4) 208 : cluster [DBG] 4.1a scrub ok 2024-02-23T00:49:47.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:46.642487+0000 osd.0 (osd.0) 205 : cluster [DBG] 4.3 deep-scrub starts 2024-02-23T00:49:47.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:46.644907+0000 osd.0 (osd.0) 206 : cluster [DBG] 4.3 deep-scrub ok 2024-02-23T00:49:47.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:47.212590+0000 mon.a (mon.0) 2693 : cluster [DBG] osdmap e470: 8 total, 8 up, 7 in 2024-02-23T00:49:47.949 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:49:47.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:47.976+0000 7f89936a1700 1 -- 172.21.15.47:0/489758656 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7f898c10b8b0 con 0x7f898c0f8d10 2024-02-23T00:49:47.982 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:48.004 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:49:48.007 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:48.007 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:49:48.108 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:49:48.138 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:48.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:48.217+0000 7f89827fc700 1 -- 172.21.15.47:0/489758656 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v471) v1 ==== 87+0+0 (secure 0 0 0) 0x7f898804c870 con 0x7f898c0f8d10 2024-02-23T00:49:48.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:48.220+0000 7f89936a1700 10 librados: watch_flush enter 2024-02-23T00:49:48.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:48.220+0000 7f89936a1700 10 librados: watch_flush exit 2024-02-23T00:49:48.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:48.220+0000 7f89936a1700 20 client.41489.objecter shutdown clearing up homeless session... 2024-02-23T00:49:48.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:48.220+0000 7f89936a1700 10 client.41489.objecter successfully canceled tick 2024-02-23T00:49:48.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:48.222+0000 7f89936a1700 1 -- 172.21.15.47:0/489758656 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8978062cc0 msgr2=0x7f8978065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:48.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:48.222+0000 7f89936a1700 1 --2- 172.21.15.47:0/489758656 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8978062cc0 0x7f8978065180 secure :-1 s=READY pgs=3090 cs=0 l=1 rev1=1 rx=0x7f897c004ee0 tx=0x7f897c00b040).stop 2024-02-23T00:49:48.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:48.222+0000 7f89936a1700 1 -- 172.21.15.47:0/489758656 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f898c0f8d10 msgr2=0x7f898c1101f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:48.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:48.222+0000 7f89936a1700 1 --2- 172.21.15.47:0/489758656 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f898c0f8d10 0x7f898c1101f0 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 rx=0x7f89880048f0 tx=0x7f8988009cb0).stop 2024-02-23T00:49:48.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:48.223+0000 7f89936a1700 1 -- 172.21.15.47:0/489758656 shutdown_connections 2024-02-23T00:49:48.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:48.223+0000 7f89936a1700 1 --2- 172.21.15.47:0/489758656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f898c102270 0x7f898c10a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:48.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:48.223+0000 7f89936a1700 1 --2- 172.21.15.47:0/489758656 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8978062cc0 0x7f8978065180 unknown :-1 s=CLOSED pgs=3090 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:48.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:48.223+0000 7f89936a1700 1 --2- 172.21.15.47:0/489758656 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f898c0f8d10 0x7f898c1101f0 unknown :-1 s=CLOSED pgs=2698 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:48.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:48.223+0000 7f89936a1700 1 --2- 172.21.15.47:0/489758656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f898c0f9670 0x7f898c1106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:48.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:48.224+0000 7f89936a1700 1 -- 172.21.15.47:0/489758656 >> 172.21.15.47:0/489758656 conn(0x7f898c0f2820 msgr2=0x7f898c0f4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:48.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:48.224+0000 7f89936a1700 1 -- 172.21.15.47:0/489758656 shutdown_connections 2024-02-23T00:49:48.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:48.224+0000 7f89936a1700 1 -- 172.21.15.47:0/489758656 wait complete. 2024-02-23T00:49:48.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:48.224+0000 7f89936a1700 1 librados: shutdown 2024-02-23T00:49:48.228 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is set 2024-02-23T00:49:48.488 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:49:48.491 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:48.491 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:49:48.504 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.0 dump_blocked_ops 2024-02-23T00:49:48.592 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:49:48.625 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:48.646 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:49:48.649 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:48.649 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:49:48.750 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:49:48.780 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:48.801 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:49:48.804 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:48.804 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:49:48.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:46.460793+0000 osd.1 (osd.1) 215 : cluster [DBG] 18.5 deep-scrub starts 2024-02-23T00:49:48.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:46.462011+0000 osd.1 (osd.1) 216 : cluster [DBG] 18.5 deep-scrub ok 2024-02-23T00:49:48.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:46.611588+0000 osd.5 (osd.5) 208 : cluster [DBG] 5.1b scrub starts 2024-02-23T00:49:48.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:46.612457+0000 osd.5 (osd.5) 209 : cluster [DBG] 5.1b scrub ok 2024-02-23T00:49:48.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:47.317183+0000 mgr.y (mgr.14517) 442 : cluster [DBG] pgmap v703: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:49:48.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:47.646610+0000 osd.5 (osd.5) 210 : cluster [DBG] 4.8 deep-scrub starts 2024-02-23T00:49:48.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:47.653170+0000 osd.5 (osd.5) 211 : cluster [DBG] 4.8 deep-scrub ok 2024-02-23T00:49:48.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:47.785810+0000 osd.3 (osd.3) 226 : cluster [DBG] 4.11 scrub starts 2024-02-23T00:49:48.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:47.788040+0000 osd.3 (osd.3) 227 : cluster [DBG] 4.11 scrub ok 2024-02-23T00:49:48.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:47.978171+0000 mon.c (mon.2) 310 : audit [INF] from='client.? 172.21.15.47:0/489758656' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:49:48.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:47.978453+0000 mon.a (mon.0) 2694 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:49:48.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:48.210589+0000 mon.a (mon.0) 2695 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:49:48.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:48.217215+0000 mon.a (mon.0) 2696 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:49:48.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:48.217285+0000 mon.a (mon.0) 2697 : cluster [DBG] osdmap e471: 8 total, 8 up, 7 in 2024-02-23T00:49:48.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:48.223365+0000 mon.a (mon.0) 2698 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:49:48.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:46.460793+0000 osd.1 (osd.1) 215 : cluster [DBG] 18.5 deep-scrub starts 2024-02-23T00:49:48.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:46.462011+0000 osd.1 (osd.1) 216 : cluster [DBG] 18.5 deep-scrub ok 2024-02-23T00:49:48.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:46.611588+0000 osd.5 (osd.5) 208 : cluster [DBG] 5.1b scrub starts 2024-02-23T00:49:48.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:46.612457+0000 osd.5 (osd.5) 209 : cluster [DBG] 5.1b scrub ok 2024-02-23T00:49:48.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:47.317183+0000 mgr.y (mgr.14517) 442 : cluster [DBG] pgmap v703: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:49:48.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:47.646610+0000 osd.5 (osd.5) 210 : cluster [DBG] 4.8 deep-scrub starts 2024-02-23T00:49:48.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:47.653170+0000 osd.5 (osd.5) 211 : cluster [DBG] 4.8 deep-scrub ok 2024-02-23T00:49:48.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:47.785810+0000 osd.3 (osd.3) 226 : cluster [DBG] 4.11 scrub starts 2024-02-23T00:49:48.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:47.788040+0000 osd.3 (osd.3) 227 : cluster [DBG] 4.11 scrub ok 2024-02-23T00:49:48.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:47.978171+0000 mon.c (mon.2) 310 : audit [INF] from='client.? 172.21.15.47:0/489758656' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:49:48.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:47.978453+0000 mon.a (mon.0) 2694 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:49:48.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:48.210589+0000 mon.a (mon.0) 2695 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:49:48.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:48.217215+0000 mon.a (mon.0) 2696 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:49:48.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:48.217285+0000 mon.a (mon.0) 2697 : cluster [DBG] osdmap e471: 8 total, 8 up, 7 in 2024-02-23T00:49:48.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:48.223365+0000 mon.a (mon.0) 2698 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:49:48.906 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:49:48.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:46.460793+0000 osd.1 (osd.1) 215 : cluster [DBG] 18.5 deep-scrub starts 2024-02-23T00:49:48.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:46.462011+0000 osd.1 (osd.1) 216 : cluster [DBG] 18.5 deep-scrub ok 2024-02-23T00:49:48.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:46.611588+0000 osd.5 (osd.5) 208 : cluster [DBG] 5.1b scrub starts 2024-02-23T00:49:48.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:46.612457+0000 osd.5 (osd.5) 209 : cluster [DBG] 5.1b scrub ok 2024-02-23T00:49:48.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:47.317183+0000 mgr.y (mgr.14517) 442 : cluster [DBG] pgmap v703: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:49:48.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:47.646610+0000 osd.5 (osd.5) 210 : cluster [DBG] 4.8 deep-scrub starts 2024-02-23T00:49:48.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:47.653170+0000 osd.5 (osd.5) 211 : cluster [DBG] 4.8 deep-scrub ok 2024-02-23T00:49:48.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:47.785810+0000 osd.3 (osd.3) 226 : cluster [DBG] 4.11 scrub starts 2024-02-23T00:49:48.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:47.788040+0000 osd.3 (osd.3) 227 : cluster [DBG] 4.11 scrub ok 2024-02-23T00:49:48.933 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:47.978171+0000 mon.c (mon.2) 310 : audit [INF] from='client.? 172.21.15.47:0/489758656' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:49:48.933 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:47.978453+0000 mon.a (mon.0) 2694 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:49:48.933 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:48.210589+0000 mon.a (mon.0) 2695 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:49:48.933 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:48.217215+0000 mon.a (mon.0) 2696 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:49:48.933 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:48.217285+0000 mon.a (mon.0) 2697 : cluster [DBG] osdmap e471: 8 total, 8 up, 7 in 2024-02-23T00:49:48.933 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:48.223365+0000 mon.a (mon.0) 2698 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:49:48.933 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:48.954 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:49:48.957 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:48.957 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:49:49.058 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:49:49.086 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:49.105 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:49:49.107 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:49.108 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:49:49.209 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:49:49.246 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:49.264 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:49:49.267 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:49.267 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:49:49.368 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:49:49.397 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:49.946 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:49:49.948 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:49.949 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:49:50.049 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:49:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:50.177 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:49:50.180 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:50.180 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:49:50.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.187+0000 7f27eb975700 1 -- 172.21.15.47:0/83850750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27e40fe510 msgr2=0x7f27e40f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:50.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.187+0000 7f27eb975700 1 --2- 172.21.15.47:0/83850750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27e40fe510 0x7f27e40f7d40 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 rx=0x7f27d4004870 tx=0x7f27d4009cb0).stop 2024-02-23T00:49:50.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.187+0000 7f27eb975700 1 -- 172.21.15.47:0/83850750 shutdown_connections 2024-02-23T00:49:50.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.187+0000 7f27eb975700 1 --2- 172.21.15.47:0/83850750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27e40f8420 0x7f27e4104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:50.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.187+0000 7f27eb975700 1 --2- 172.21.15.47:0/83850750 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f27e40fdbb0 0x7f27e40fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:50.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.187+0000 7f27eb975700 1 --2- 172.21.15.47:0/83850750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27e40fe510 0x7f27e40f7d40 unknown :-1 s=CLOSED pgs=2551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:50.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.187+0000 7f27eb975700 1 -- 172.21.15.47:0/83850750 >> 172.21.15.47:0/83850750 conn(0x7f27e40f3800 msgr2=0x7f27e40f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:50.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.187+0000 7f27eb975700 1 -- 172.21.15.47:0/83850750 shutdown_connections 2024-02-23T00:49:50.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.187+0000 7f27eb975700 1 -- 172.21.15.47:0/83850750 wait complete. 2024-02-23T00:49:50.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.188+0000 7f27eb975700 1 librados: starting msgr at 2024-02-23T00:49:50.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.188+0000 7f27eb975700 1 librados: starting objecter 2024-02-23T00:49:50.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.188+0000 7f27eb975700 1 Processor -- start 2024-02-23T00:49:50.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.188+0000 7f27eb975700 1 -- start start 2024-02-23T00:49:50.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.188+0000 7f27eb975700 1 librados: setting wanted keys 2024-02-23T00:49:50.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.188+0000 7f27eb975700 1 librados: calling monclient init 2024-02-23T00:49:50.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.188+0000 7f27eb975700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27e40f8420 0x7f27e410dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:50.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.188+0000 7f27eb975700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27e40fdbb0 0x7f27e410e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:50.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.188+0000 7f27eb975700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f27e40fe510 0x7f27e4108080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:50.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.188+0000 7f27eb975700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e4108570 con 0x7f27e40f8420 2024-02-23T00:49:50.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.188+0000 7f27eb975700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e41086e0 con 0x7f27e40fdbb0 2024-02-23T00:49:50.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.188+0000 7f27eb975700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e41088a0 con 0x7f27e40fe510 2024-02-23T00:49:50.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.188+0000 7f27e9f12700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f27e40fe510 0x7f27e4108080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:50.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.188+0000 7f27e9f12700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f27e40fe510 0x7f27e4108080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:47918/0 (socket says 172.21.15.47:47918) 2024-02-23T00:49:50.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.188+0000 7f27e9f12700 1 -- 172.21.15.47:0/2559840393 learned_addr learned my addr 172.21.15.47:0/2559840393 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:49:50.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.189+0000 7f27e8f10700 1 --2- 172.21.15.47:0/2559840393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27e40fdbb0 0x7f27e410e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:50.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.189+0000 7f27e9711700 1 --2- 172.21.15.47:0/2559840393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27e40f8420 0x7f27e410dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:50.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.189+0000 7f27e9f12700 1 -- 172.21.15.47:0/2559840393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27e40fdbb0 msgr2=0x7f27e410e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:50.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.189+0000 7f27e9f12700 1 --2- 172.21.15.47:0/2559840393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27e40fdbb0 0x7f27e410e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:50.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.189+0000 7f27e9f12700 1 -- 172.21.15.47:0/2559840393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27e40f8420 msgr2=0x7f27e410dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:50.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.189+0000 7f27e9f12700 1 --2- 172.21.15.47:0/2559840393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27e40f8420 0x7f27e410dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:50.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.189+0000 7f27e9f12700 1 -- 172.21.15.47:0/2559840393 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d4005040 con 0x7f27e40fe510 2024-02-23T00:49:50.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.189+0000 7f27e9711700 1 --2- 172.21.15.47:0/2559840393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27e40f8420 0x7f27e410dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:49:50.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.189+0000 7f27e9f12700 1 --2- 172.21.15.47:0/2559840393 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f27e40fe510 0x7f27e4108080 secure :-1 s=READY pgs=2699 cs=0 l=1 rev1=1 rx=0x7f27dc00ec20 tx=0x7f27dc00b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:50.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.189+0000 7f27da7fc700 10 client.?.objecter ms_handle_connect 0x7f27e40fe510 2024-02-23T00:49:50.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.189+0000 7f27da7fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:49:50.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.189+0000 7f27da7fc700 1 -- 172.21.15.47:0/2559840393 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc009950 con 0x7f27e40fe510 2024-02-23T00:49:50.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.189+0000 7f27eb975700 1 -- 172.21.15.47:0/2559840393 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e4108b20 con 0x7f27e40fe510 2024-02-23T00:49:50.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.189+0000 7f27eb975700 10 client.41495.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:49:50.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.189+0000 7f27da7fc700 1 -- 172.21.15.47:0/2559840393 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f27dc00f040 con 0x7f27e40fe510 2024-02-23T00:49:50.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.189+0000 7f27eb975700 1 -- 172.21.15.47:0/2559840393 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e4114d10 con 0x7f27e40fe510 2024-02-23T00:49:50.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.189+0000 7f27eb975700 1 librados: init done 2024-02-23T00:49:50.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.189+0000 7f27da7fc700 1 -- 172.21.15.47:0/2559840393 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc007e10 con 0x7f27e40fe510 2024-02-23T00:49:50.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.190+0000 7f27eb975700 10 client.41495.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:50.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.190+0000 7f27eb975700 15 client.41495.objecter _session_command_op_assign -1 1 2024-02-23T00:49:50.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.190+0000 7f27eb975700 20 client.41495.objecter _get_session osd=-1 returning homeless 2024-02-23T00:49:50.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.190+0000 7f27eb975700 10 client.41495.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:49:50.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.190+0000 7f27eb975700 1 -- 172.21.15.47:0/2559840393 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f27e405ee80 con 0x7f27e40fe510 2024-02-23T00:49:50.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.190+0000 7f27da7fc700 1 -- 172.21.15.47:0/2559840393 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f27dc00c9a0 con 0x7f27e40fe510 2024-02-23T00:49:50.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.191+0000 7f27da7fc700 1 --2- 172.21.15.47:0/2559840393 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f27d0062cc0 0x7f27d0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:50.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.191+0000 7f27da7fc700 1 -- 172.21.15.47:0/2559840393 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(472..472 src has 48..472) v4 ==== 9220+0+0 (secure 0 0 0) 0x7f27dc07d600 con 0x7f27e40fe510 2024-02-23T00:49:50.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.191+0000 7f27da7fc700 10 client.41495.objecter ms_dispatch 0x7f27e4001550 osd_map(472..472 src has 48..472) v4 2024-02-23T00:49:50.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.191+0000 7f27da7fc700 3 client.41495.objecter handle_osd_map got epochs [472,472] > 0 2024-02-23T00:49:50.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.191+0000 7f27da7fc700 3 client.41495.objecter handle_osd_map decoding full epoch 472 2024-02-23T00:49:50.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.191+0000 7f27e9711700 1 --2- 172.21.15.47:0/2559840393 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f27d0062cc0 0x7f27d0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:50.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.191+0000 7f27da7fc700 10 client.41495.objecter checking command 1 2024-02-23T00:49:50.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.191+0000 7f27da7fc700 1 --2- 172.21.15.47:0/2559840393 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f27d006f870 0x7f27d0071d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:50.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.191+0000 7f27da7fc700 20 client.41495.objecter _get_session s=0x7f27d006f1a0 osd=7 3 2024-02-23T00:49:50.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.191+0000 7f27da7fc700 20 client.41495.objecter put_session s=0x7f27d006f1a0 osd=7 3 2024-02-23T00:49:50.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.191+0000 7f27da7fc700 15 client.41495.objecter _session_command_op_remove -1 1 2024-02-23T00:49:50.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.191+0000 7f27da7fc700 20 client.41495.objecter _get_session s=0x7f27d006f1a0 osd=7 3 2024-02-23T00:49:50.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.191+0000 7f27da7fc700 20 client.41495.objecter get_session s=0x7f27d006f1a0 osd=7 3 2024-02-23T00:49:50.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.191+0000 7f27da7fc700 15 client.41495.objecter _session_command_op_assign 7 1 2024-02-23T00:49:50.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.191+0000 7f27da7fc700 20 client.41495.objecter put_session s=0x7f27d006f1a0 osd=7 4 2024-02-23T00:49:50.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.191+0000 7f27da7fc700 10 client.41495.objecter _send_command 1 2024-02-23T00:49:50.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.191+0000 7f27da7fc700 1 -- 172.21.15.47:0/2559840393 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f27d0072390 con 0x7f27d006f870 2024-02-23T00:49:50.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.191+0000 7f27da7fc700 20 client.41495.objecter dump_active .. 0 homeless 2024-02-23T00:49:50.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.191+0000 7f27da7fc700 1 -- 172.21.15.47:0/2559840393 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_get_version_reply(handle=1 version=472) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27dc07fb40 con 0x7f27e40fe510 2024-02-23T00:49:50.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.191+0000 7f27da7fc700 10 client.41495.objecter ms_handle_connect 0x7f27d0062cc0 2024-02-23T00:49:50.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.191+0000 7f27e9711700 1 --2- 172.21.15.47:0/2559840393 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f27d0062cc0 0x7f27d0065180 secure :-1 s=READY pgs=3091 cs=0 l=1 rev1=1 rx=0x7f27e0007fd0 tx=0x7f27e000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:50.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.191+0000 7f27e8f10700 1 --2- 172.21.15.47:0/2559840393 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f27d006f870 0x7f27d0071d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:50.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.192+0000 7f27e8f10700 1 --2- 172.21.15.47:0/2559840393 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f27d006f870 0x7f27d0071d10 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:50.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.192+0000 7f27da7fc700 10 client.41495.objecter ms_handle_connect 0x7f27d006f870 2024-02-23T00:49:50.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.192+0000 7f27da7fc700 1 -- 172.21.15.47:0/2559840393 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f27d0072390 con 0x7f27d006f870 2024-02-23T00:49:50.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.192+0000 7f27da7fc700 10 client.41495.objecter ms_dispatch 0x7f27e4001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:49:50.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.192+0000 7f27da7fc700 10 client.41495.objecter _finish_command 1 = system:0 2024-02-23T00:49:50.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.192+0000 7f27da7fc700 20 client.41495.objecter put_session s=0x7f27d006f1a0 osd=7 4 2024-02-23T00:49:50.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.192+0000 7f27da7fc700 15 client.41495.objecter _session_command_op_remove 7 1 2024-02-23T00:49:50.222 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:49:50.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.227+0000 7f27eb975700 10 client.41495.objecter _submit_command 2 [{"prefix": "reset_purged_snaps_last"}] 2024-02-23T00:49:50.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.227+0000 7f27eb975700 15 client.41495.objecter _session_command_op_assign -1 2 2024-02-23T00:49:50.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.227+0000 7f27eb975700 20 client.41495.objecter _get_session s=0x7f27d006f1a0 osd=7 3 2024-02-23T00:49:50.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.227+0000 7f27eb975700 20 client.41495.objecter put_session s=0x7f27d006f1a0 osd=7 3 2024-02-23T00:49:50.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.227+0000 7f27eb975700 20 client.41495.objecter _get_session s=0x7f27d006f1a0 osd=7 3 2024-02-23T00:49:50.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.227+0000 7f27eb975700 15 client.41495.objecter _session_command_op_remove -1 2 2024-02-23T00:49:50.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.227+0000 7f27eb975700 20 client.41495.objecter get_session s=0x7f27d006f1a0 osd=7 3 2024-02-23T00:49:50.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.227+0000 7f27eb975700 15 client.41495.objecter _session_command_op_assign 7 2 2024-02-23T00:49:50.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.227+0000 7f27eb975700 20 client.41495.objecter put_session s=0x7f27d006f1a0 osd=7 4 2024-02-23T00:49:50.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.227+0000 7f27eb975700 10 client.41495.objecter _send_command 2 2024-02-23T00:49:50.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.227+0000 7f27eb975700 1 -- 172.21.15.47:0/2559840393 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) v1 -- 0x7f27e4106e20 con 0x7f27d006f870 2024-02-23T00:49:50.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.227+0000 7f27da7fc700 1 -- 172.21.15.47:0/2559840393 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7f27e4106e20 con 0x7f27d006f870 2024-02-23T00:49:50.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.227+0000 7f27da7fc700 10 client.41495.objecter ms_dispatch 0x7f27e4001550 command_reply(tid 2: 0 ) v1 2024-02-23T00:49:50.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.227+0000 7f27da7fc700 10 client.41495.objecter _finish_command 2 = system:0 2024-02-23T00:49:50.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.227+0000 7f27da7fc700 20 client.41495.objecter put_session s=0x7f27d006f1a0 osd=7 4 2024-02-23T00:49:50.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.227+0000 7f27da7fc700 15 client.41495.objecter _session_command_op_remove 7 2 2024-02-23T00:49:50.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.228+0000 7f27eb975700 10 librados: watch_flush enter 2024-02-23T00:49:50.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.228+0000 7f27eb975700 10 librados: watch_flush exit 2024-02-23T00:49:50.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.228+0000 7f27eb975700 10 client.41495.objecter close_session for osd.7 2024-02-23T00:49:50.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.228+0000 7f27eb975700 1 -- 172.21.15.47:0/2559840393 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f27d006f870 msgr2=0x7f27d0071d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:50.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.228+0000 7f27eb975700 1 --2- 172.21.15.47:0/2559840393 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f27d006f870 0x7f27d0071d10 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:50.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.228+0000 7f27eb975700 20 client.41495.objecter put_session s=0x7f27d006f1a0 osd=7 1 2024-02-23T00:49:50.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.228+0000 7f27eb975700 20 client.41495.objecter shutdown clearing up homeless session... 2024-02-23T00:49:50.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.228+0000 7f27eb975700 10 client.41495.objecter successfully canceled tick 2024-02-23T00:49:50.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.228+0000 7f27eb975700 1 -- 172.21.15.47:0/2559840393 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f27d0062cc0 msgr2=0x7f27d0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:50.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.228+0000 7f27eb975700 1 --2- 172.21.15.47:0/2559840393 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f27d0062cc0 0x7f27d0065180 secure :-1 s=READY pgs=3091 cs=0 l=1 rev1=1 rx=0x7f27e0007fd0 tx=0x7f27e000b040).stop 2024-02-23T00:49:50.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.228+0000 7f27eb975700 1 -- 172.21.15.47:0/2559840393 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f27e40fe510 msgr2=0x7f27e4108080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:50.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.228+0000 7f27eb975700 1 --2- 172.21.15.47:0/2559840393 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f27e40fe510 0x7f27e4108080 secure :-1 s=READY pgs=2699 cs=0 l=1 rev1=1 rx=0x7f27dc00ec20 tx=0x7f27dc00b040).stop 2024-02-23T00:49:50.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.228+0000 7f27e9f12700 1 -- 172.21.15.47:0/2559840393 reap_dead start 2024-02-23T00:49:50.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.230+0000 7f27eb975700 1 -- 172.21.15.47:0/2559840393 shutdown_connections 2024-02-23T00:49:50.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.230+0000 7f27eb975700 1 -- 172.21.15.47:0/2559840393 >> 172.21.15.47:0/2559840393 conn(0x7f27e40f3800 msgr2=0x7f27e40fc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:50.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.230+0000 7f27eb975700 1 -- 172.21.15.47:0/2559840393 shutdown_connections 2024-02-23T00:49:50.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.230+0000 7f27eb975700 1 -- 172.21.15.47:0/2559840393 wait complete. 2024-02-23T00:49:50.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:50.230+0000 7f27eb975700 1 librados: shutdown 2024-02-23T00:49:50.281 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:49:50.310 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:50.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:49.218957+0000 mon.a (mon.0) 2699 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:49:50.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:49.219044+0000 mon.a (mon.0) 2700 : cluster [DBG] osdmap e472: 8 total, 8 up, 7 in 2024-02-23T00:49:50.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:49.223767+0000 mon.a (mon.0) 2701 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:49:50.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:49.317730+0000 mgr.y (mgr.14517) 443 : cluster [DBG] pgmap v706: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:49:50.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:50.210891+0000 mon.a (mon.0) 2702 : cluster [DBG] osdmap e473: 8 total, 8 up, 7 in 2024-02-23T00:49:50.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:49.218957+0000 mon.a (mon.0) 2699 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:49:50.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:49.219044+0000 mon.a (mon.0) 2700 : cluster [DBG] osdmap e472: 8 total, 8 up, 7 in 2024-02-23T00:49:50.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:49.223767+0000 mon.a (mon.0) 2701 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:49:50.458 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:49.317730+0000 mgr.y (mgr.14517) 443 : cluster [DBG] pgmap v706: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:49:50.458 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:50.210891+0000 mon.a (mon.0) 2702 : cluster [DBG] osdmap e473: 8 total, 8 up, 7 in 2024-02-23T00:49:50.510 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:49:50.513 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:50.513 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:49:50.614 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:49:50.643 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:49.218957+0000 mon.a (mon.0) 2699 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:49:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:49.219044+0000 mon.a (mon.0) 2700 : cluster [DBG] osdmap e472: 8 total, 8 up, 7 in 2024-02-23T00:49:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:49.223767+0000 mon.a (mon.0) 2701 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:49:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:49.317730+0000 mgr.y (mgr.14517) 443 : cluster [DBG] pgmap v706: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:49:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:50.210891+0000 mon.a (mon.0) 2702 : cluster [DBG] osdmap e473: 8 total, 8 up, 7 in 2024-02-23T00:49:50.767 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:49:50.768 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:49:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:49:50] "GET /metrics HTTP/1.1" 200 189338 "" "Prometheus/2.33.4" 2024-02-23T00:49:50.769 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:50.770 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:49:50.870 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:49:50.903 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:50.924 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:49:50.926 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:50.927 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:49:51.028 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:49:51.056 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:51.305 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:49:51.306 INFO:journalctl@ceph.osd.0.smithi047.stdout:Feb 23 00:49:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-0[48294]: debug 2024-02-23T00:49:51.249+0000 7f8844538700 -1 snap_mapper.add_oid found existing snaps mapped on 62:947b8ab5:test-rados-api-smithi047-82465-74::foo:24, removing 2024-02-23T00:49:51.307 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:51.308 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:49:51.408 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:49:51.436 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:51.483 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-23T00:49:51.694 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:49:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:49:51.248+0000 7f2cd0485700 -1 snap_mapper.add_oid found existing snaps mapped on 62:947b8ab5:test-rados-api-smithi047-82465-74::foo:24, removing 2024-02-23T00:49:51.694 INFO:journalctl@ceph.osd.4.smithi196.stdout:Feb 23 00:49:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-4[43838]: debug 2024-02-23T00:49:51.248+0000 7f1aa6108700 -1 snap_mapper.add_oid found existing snaps mapped on 62:947b8ab5:test-rados-api-smithi047-82465-74::foo:24, removing 2024-02-23T00:49:51.921 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:49:51.985 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:49:51.989 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:51.989 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:49:52.090 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:49:52.126 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:52.146 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:49:52.149 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:52.150 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:49:52.225 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:50.544908+0000 osd.5 (osd.5) 212 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:49:52.226 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:50.545162+0000 osd.5 (osd.5) 213 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:49:52.226 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:51.220508+0000 mon.a (mon.0) 2703 : cluster [DBG] osdmap e474: 8 total, 8 up, 7 in 2024-02-23T00:49:52.226 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:51.267789+0000 mon.a (mon.0) 2704 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:49:52.226 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:51.268125+0000 mon.a (mon.0) 2705 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-73"}]: dispatch 2024-02-23T00:49:52.226 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:51.318125+0000 mgr.y (mgr.14517) 444 : cluster [DBG] pgmap v709: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:49:52.250 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:49:52.285 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:52.434 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:49:52.436 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:52.437 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:49:52.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:50.544908+0000 osd.5 (osd.5) 212 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:49:52.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:50.545162+0000 osd.5 (osd.5) 213 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:49:52.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:51.220508+0000 mon.a (mon.0) 2703 : cluster [DBG] osdmap e474: 8 total, 8 up, 7 in 2024-02-23T00:49:52.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:51.267789+0000 mon.a (mon.0) 2704 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:49:52.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:51.268125+0000 mon.a (mon.0) 2705 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-73"}]: dispatch 2024-02-23T00:49:52.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:51.318125+0000 mgr.y (mgr.14517) 444 : cluster [DBG] pgmap v709: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:49:52.537 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:49:52.552 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:49:52.552 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:49:52.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:50.544908+0000 osd.5 (osd.5) 212 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:49:52.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:50.545162+0000 osd.5 (osd.5) 213 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:49:52.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:51.220508+0000 mon.a (mon.0) 2703 : cluster [DBG] osdmap e474: 8 total, 8 up, 7 in 2024-02-23T00:49:52.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:51.267789+0000 mon.a (mon.0) 2704 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:49:52.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:51.268125+0000 mon.a (mon.0) 2705 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-73"}]: dispatch 2024-02-23T00:49:52.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:51.318125+0000 mgr.y (mgr.14517) 444 : cluster [DBG] pgmap v709: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:49:52.568 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:52.588 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:49:52.591 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:52.591 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:49:52.692 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:49:52.724 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:52.746 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:49:52.748 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:52.749 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:49:52.850 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:49:52.882 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:52.943 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:49:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:49:52.944 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:49:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:49:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:49:52.980 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:49:52.983 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:52.983 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:49:53.084 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:49:53.112 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:53.461 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:49:53.463 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:53.463 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:49:53.473 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.0 dump_historic_ops 2024-02-23T00:49:53.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:52.225816+0000 mon.a (mon.0) 2706 : cluster [DBG] osdmap e475: 8 total, 8 up, 7 in 2024-02-23T00:49:53.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:52.666377+0000 mgr.y (mgr.14517) 445 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:49:53.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:52.680477+0000 mon.a (mon.0) 2707 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:49:53.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:52.967547+0000 mon.a (mon.0) 2708 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:49:53.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:52.968544+0000 mon.a (mon.0) 2709 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:49:53.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:52.969242+0000 mon.a (mon.0) 2710 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:49:53.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:52.970035+0000 mon.a (mon.0) 2711 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:49:53.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:52.976530+0000 mon.a (mon.0) 2712 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:49:53.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:52.225816+0000 mon.a (mon.0) 2706 : cluster [DBG] osdmap e475: 8 total, 8 up, 7 in 2024-02-23T00:49:53.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:52.666377+0000 mgr.y (mgr.14517) 445 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:49:53.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:52.680477+0000 mon.a (mon.0) 2707 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:49:53.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:52.967547+0000 mon.a (mon.0) 2708 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:49:53.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:52.968544+0000 mon.a (mon.0) 2709 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:49:53.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:52.969242+0000 mon.a (mon.0) 2710 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:49:53.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:52.970035+0000 mon.a (mon.0) 2711 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:49:53.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:52.976530+0000 mon.a (mon.0) 2712 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:49:53.564 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:49:53.592 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:53.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:52.225816+0000 mon.a (mon.0) 2706 : cluster [DBG] osdmap e475: 8 total, 8 up, 7 in 2024-02-23T00:49:53.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:52.666377+0000 mgr.y (mgr.14517) 445 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:49:53.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:52.680477+0000 mon.a (mon.0) 2707 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:49:53.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:52.967547+0000 mon.a (mon.0) 2708 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:49:53.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:52.968544+0000 mon.a (mon.0) 2709 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:49:53.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:52.969242+0000 mon.a (mon.0) 2710 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:49:53.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:52.970035+0000 mon.a (mon.0) 2711 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:49:53.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:52.976530+0000 mon.a (mon.0) 2712 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:49:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:49:53.967 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:49:54.010 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:49:54.472 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:49:52.970672+0000 mgr.y (mgr.14517) 446 : cephadm [INF] Adjusting osd_memory_target on smithi196 to 3256M 2024-02-23T00:49:54.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:53.229555+0000 mon.a (mon.0) 2713 : cluster [DBG] osdmap e476: 8 total, 8 up, 7 in 2024-02-23T00:49:54.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:53.230879+0000 mon.a (mon.0) 2714 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:49:54.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:53.318564+0000 mgr.y (mgr.14517) 447 : cluster [DBG] pgmap v712: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-23T00:49:54.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:54.058971+0000 mon.a (mon.0) 2715 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:49:54.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:54.059602+0000 mon.a (mon.0) 2716 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:49:54.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:54.060120+0000 mon.a (mon.0) 2717 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:49:54.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:54.060662+0000 mon.a (mon.0) 2718 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:49:54.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:54.064961+0000 mon.a (mon.0) 2719 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:49:54.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:54.066578+0000 mon.a (mon.0) 2720 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:49:54.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:54.067134+0000 mon.a (mon.0) 2721 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:49:54.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:54.071890+0000 mon.a (mon.0) 2722 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:49:54.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:49:52.970672+0000 mgr.y (mgr.14517) 446 : cephadm [INF] Adjusting osd_memory_target on smithi196 to 3256M 2024-02-23T00:49:54.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:53.229555+0000 mon.a (mon.0) 2713 : cluster [DBG] osdmap e476: 8 total, 8 up, 7 in 2024-02-23T00:49:54.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:53.230879+0000 mon.a (mon.0) 2714 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:49:54.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:53.318564+0000 mgr.y (mgr.14517) 447 : cluster [DBG] pgmap v712: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-23T00:49:54.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:54.058971+0000 mon.a (mon.0) 2715 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:49:54.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:54.059602+0000 mon.a (mon.0) 2716 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:49:54.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:54.060120+0000 mon.a (mon.0) 2717 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:49:54.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:54.060662+0000 mon.a (mon.0) 2718 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:49:54.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:54.064961+0000 mon.a (mon.0) 2719 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:49:54.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:54.066578+0000 mon.a (mon.0) 2720 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:49:54.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:54.067134+0000 mon.a (mon.0) 2721 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:49:54.475 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:54.071890+0000 mon.a (mon.0) 2722 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:49:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:49:52.970672+0000 mgr.y (mgr.14517) 446 : cephadm [INF] Adjusting osd_memory_target on smithi196 to 3256M 2024-02-23T00:49:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:53.229555+0000 mon.a (mon.0) 2713 : cluster [DBG] osdmap e476: 8 total, 8 up, 7 in 2024-02-23T00:49:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:53.230879+0000 mon.a (mon.0) 2714 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:49:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:53.318564+0000 mgr.y (mgr.14517) 447 : cluster [DBG] pgmap v712: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-23T00:49:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:54.058971+0000 mon.a (mon.0) 2715 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:49:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:54.059602+0000 mon.a (mon.0) 2716 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:49:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:54.060120+0000 mon.a (mon.0) 2717 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:49:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:54.060662+0000 mon.a (mon.0) 2718 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-23T00:49:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:54.064961+0000 mon.a (mon.0) 2719 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:49:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:54.066578+0000 mon.a (mon.0) 2720 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:49:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:54.067134+0000 mon.a (mon.0) 2721 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:49:54.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:54.071890+0000 mon.a (mon.0) 2722 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:49:55.445 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:49:55.448 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cephadm 2024-02-23T00:49:54.061077+0000 mgr.y (mgr.14517) 448 : cephadm [INF] Adjusting osd_memory_target on smithi047 to 3257M 2024-02-23T00:49:55.448 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:54.237428+0000 mon.a (mon.0) 2723 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:49:55.448 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:54.237614+0000 mon.a (mon.0) 2724 : cluster [DBG] osdmap e477: 8 total, 8 up, 7 in 2024-02-23T00:49:55.448 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:54.242639+0000 mon.a (mon.0) 2725 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:49:55.449 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cephadm 2024-02-23T00:49:54.061077+0000 mgr.y (mgr.14517) 448 : cephadm [INF] Adjusting osd_memory_target on smithi047 to 3257M 2024-02-23T00:49:55.449 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:54.237428+0000 mon.a (mon.0) 2723 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:49:55.449 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:54.237614+0000 mon.a (mon.0) 2724 : cluster [DBG] osdmap e477: 8 total, 8 up, 7 in 2024-02-23T00:49:55.449 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:54.242639+0000 mon.a (mon.0) 2725 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:49:55.449 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:55.450 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:49:55.550 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:49:55.578 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cephadm 2024-02-23T00:49:54.061077+0000 mgr.y (mgr.14517) 448 : cephadm [INF] Adjusting osd_memory_target on smithi047 to 3257M 2024-02-23T00:49:55.578 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:54.237428+0000 mon.a (mon.0) 2723 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:49:55.578 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:54.237614+0000 mon.a (mon.0) 2724 : cluster [DBG] osdmap e477: 8 total, 8 up, 7 in 2024-02-23T00:49:55.579 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:54.242639+0000 mon.a (mon.0) 2725 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:49:55.581 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:55.601 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:49:55.604 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:55.604 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:49:55.705 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:49:55.737 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:56.087 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:49:56.090 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:56.090 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:49:56.191 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:49:56.224 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:56.246 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:49:56.248 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:56.248 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:49:56.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:55.237788+0000 mon.a (mon.0) 2726 : cluster [DBG] osdmap e478: 8 total, 8 up, 7 in 2024-02-23T00:49:56.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:55.318979+0000 mgr.y (mgr.14517) 449 : cluster [DBG] pgmap v715: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-23T00:49:56.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:55.237788+0000 mon.a (mon.0) 2726 : cluster [DBG] osdmap e478: 8 total, 8 up, 7 in 2024-02-23T00:49:56.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:55.318979+0000 mgr.y (mgr.14517) 449 : cluster [DBG] pgmap v715: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-23T00:49:56.349 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:49:56.379 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:56.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:55.237788+0000 mon.a (mon.0) 2726 : cluster [DBG] osdmap e478: 8 total, 8 up, 7 in 2024-02-23T00:49:56.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:55.318979+0000 mgr.y (mgr.14517) 449 : cluster [DBG] pgmap v715: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-23T00:49:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.182+0000 7f3f7f68d700 1 -- 172.21.15.47:0/2594495936 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3f780fcbb0 msgr2=0x7f3f780fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.182+0000 7f3f7f68d700 1 --2- 172.21.15.47:0/2594495936 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3f780fcbb0 0x7f3f780fcf90 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 rx=0x7f3f70009be0 tx=0x7f3f70004d10).stop 2024-02-23T00:49:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.182+0000 7f3f7f68d700 1 -- 172.21.15.47:0/2594495936 shutdown_connections 2024-02-23T00:49:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.182+0000 7f3f7f68d700 1 --2- 172.21.15.47:0/2594495936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f780f7400 0x7f3f78103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.182+0000 7f3f7f68d700 1 --2- 172.21.15.47:0/2594495936 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3f780fcbb0 0x7f3f780fcf90 unknown :-1 s=CLOSED pgs=2700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.182+0000 7f3f7f68d700 1 --2- 172.21.15.47:0/2594495936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f780fd510 0x7f3f780f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.182+0000 7f3f7f68d700 1 -- 172.21.15.47:0/2594495936 >> 172.21.15.47:0/2594495936 conn(0x7f3f780f2820 msgr2=0x7f3f780f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.182+0000 7f3f7f68d700 1 -- 172.21.15.47:0/2594495936 shutdown_connections 2024-02-23T00:49:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.182+0000 7f3f7f68d700 1 -- 172.21.15.47:0/2594495936 wait complete. 2024-02-23T00:49:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.183+0000 7f3f7f68d700 1 librados: starting msgr at 2024-02-23T00:49:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.183+0000 7f3f7f68d700 1 librados: starting objecter 2024-02-23T00:49:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.183+0000 7f3f7f68d700 1 Processor -- start 2024-02-23T00:49:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.183+0000 7f3f7f68d700 1 -- start start 2024-02-23T00:49:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.183+0000 7f3f7f68d700 1 librados: setting wanted keys 2024-02-23T00:49:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.183+0000 7f3f7f68d700 1 librados: calling monclient init 2024-02-23T00:49:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.183+0000 7f3f7f68d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3f780f7400 0x7f3f7810e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.183+0000 7f3f7f68d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f780fcbb0 0x7f3f7810e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.183+0000 7f3f7f68d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f780fd510 0x7f3f78108210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.183+0000 7f3f7f68d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f78108700 con 0x7f3f780fcbb0 2024-02-23T00:49:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.183+0000 7f3f7f68d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f78108870 con 0x7f3f780fd510 2024-02-23T00:49:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.183+0000 7f3f7f68d700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f78108a30 con 0x7f3f780f7400 2024-02-23T00:49:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.184+0000 7f3f7d429700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3f780f7400 0x7f3f7810e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.184+0000 7f3f7d429700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3f780f7400 0x7f3f7810e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:45204/0 (socket says 172.21.15.47:45204) 2024-02-23T00:49:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.184+0000 7f3f7d429700 1 -- 172.21.15.47:0/3505902396 learned_addr learned my addr 172.21.15.47:0/3505902396 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:49:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.184+0000 7f3f7cc28700 1 --2- 172.21.15.47:0/3505902396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f780fcbb0 0x7f3f7810e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.184+0000 7f3f7dc2a700 1 --2- 172.21.15.47:0/3505902396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f780fd510 0x7f3f78108210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.184+0000 7f3f7d429700 1 -- 172.21.15.47:0/3505902396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f780fd510 msgr2=0x7f3f78108210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.184+0000 7f3f7d429700 1 --2- 172.21.15.47:0/3505902396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f780fd510 0x7f3f78108210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.184+0000 7f3f7d429700 1 -- 172.21.15.47:0/3505902396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f780fcbb0 msgr2=0x7f3f7810e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.184+0000 7f3f7d429700 1 --2- 172.21.15.47:0/3505902396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f780fcbb0 0x7f3f7810e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.184+0000 7f3f7d429700 1 -- 172.21.15.47:0/3505902396 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f700097c0 con 0x7f3f780f7400 2024-02-23T00:49:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.184+0000 7f3f7cc28700 1 --2- 172.21.15.47:0/3505902396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f780fcbb0 0x7f3f7810e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:49:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.184+0000 7f3f7d429700 1 --2- 172.21.15.47:0/3505902396 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3f780f7400 0x7f3f7810e070 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 rx=0x7f3f70004750 tx=0x7f3f70004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.184+0000 7f3f767fc700 10 client.?.objecter ms_handle_connect 0x7f3f780f7400 2024-02-23T00:49:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.184+0000 7f3f767fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:49:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.184+0000 7f3f767fc700 1 -- 172.21.15.47:0/3505902396 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7000e040 con 0x7f3f780f7400 2024-02-23T00:49:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.184+0000 7f3f767fc700 1 -- 172.21.15.47:0/3505902396 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f3f70018040 con 0x7f3f780f7400 2024-02-23T00:49:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.184+0000 7f3f7f68d700 1 -- 172.21.15.47:0/3505902396 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f78108cb0 con 0x7f3f780f7400 2024-02-23T00:49:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.184+0000 7f3f767fc700 1 -- 172.21.15.47:0/3505902396 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f70013c00 con 0x7f3f780f7400 2024-02-23T00:49:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.184+0000 7f3f7f68d700 10 client.41501.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:49:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.184+0000 7f3f7f68d700 1 -- 172.21.15.47:0/3505902396 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f78114ea0 con 0x7f3f780f7400 2024-02-23T00:49:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.184+0000 7f3f7f68d700 1 librados: init done 2024-02-23T00:49:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.185+0000 7f3f7f68d700 1 -- 172.21.15.47:0/3505902396 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f7805fa60 con 0x7f3f780f7400 2024-02-23T00:49:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.186+0000 7f3f767fc700 1 -- 172.21.15.47:0/3505902396 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f3f70007910 con 0x7f3f780f7400 2024-02-23T00:49:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.186+0000 7f3f767fc700 1 --2- 172.21.15.47:0/3505902396 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3f60062cc0 0x7f3f60065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.186+0000 7f3f767fc700 1 -- 172.21.15.47:0/3505902396 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(479..479 src has 48..479) v4 ==== 9220+0+0 (secure 0 0 0) 0x7f3f7007e170 con 0x7f3f780f7400 2024-02-23T00:49:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.186+0000 7f3f767fc700 10 client.41501.objecter ms_dispatch 0x7f3f78001550 osd_map(479..479 src has 48..479) v4 2024-02-23T00:49:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.186+0000 7f3f767fc700 3 client.41501.objecter handle_osd_map got epochs [479,479] > 0 2024-02-23T00:49:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.186+0000 7f3f767fc700 3 client.41501.objecter handle_osd_map decoding full epoch 479 2024-02-23T00:49:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.186+0000 7f3f7cc28700 1 --2- 172.21.15.47:0/3505902396 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3f60062cc0 0x7f3f60065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.186+0000 7f3f767fc700 20 client.41501.objecter dump_active .. 0 homeless 2024-02-23T00:49:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.186+0000 7f3f767fc700 10 client.41501.objecter ms_handle_connect 0x7f3f60062cc0 2024-02-23T00:49:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.186+0000 7f3f7cc28700 1 --2- 172.21.15.47:0/3505902396 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3f60062cc0 0x7f3f60065180 secure :-1 s=READY pgs=3092 cs=0 l=1 rev1=1 rx=0x7f3f781099b0 tx=0x7f3f6c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.190+0000 7f3f767fc700 1 -- 172.21.15.47:0/3505902396 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f3f7004d800 con 0x7f3f780f7400 2024-02-23T00:49:57.260 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:56.259607+0000 mon.a (mon.0) 2727 : cluster [DBG] osdmap e479: 8 total, 8 up, 7 in 2024-02-23T00:49:57.260 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:56.271223+0000 mon.a (mon.0) 2728 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "219.12"}]: dispatch 2024-02-23T00:49:57.260 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:56.271441+0000 mgr.y (mgr.14517) 450 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "219.12"}]: dispatch 2024-02-23T00:49:57.260 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:56.304918+0000 mon.a (mon.0) 2729 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:49:57.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:57.378+0000 7f3f7f68d700 1 -- 172.21.15.47:0/3505902396 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7f3f78105ea0 con 0x7f3f780f7400 2024-02-23T00:49:57.529 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:49:57.530 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:56.259607+0000 mon.a (mon.0) 2727 : cluster [DBG] osdmap e479: 8 total, 8 up, 7 in 2024-02-23T00:49:57.530 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:56.271223+0000 mon.a (mon.0) 2728 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "219.12"}]: dispatch 2024-02-23T00:49:57.530 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:56.271441+0000 mgr.y (mgr.14517) 450 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "219.12"}]: dispatch 2024-02-23T00:49:57.530 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:56.304918+0000 mon.a (mon.0) 2729 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:49:57.532 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:57.533 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:49:57.634 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:49:57.664 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:56.259607+0000 mon.a (mon.0) 2727 : cluster [DBG] osdmap e479: 8 total, 8 up, 7 in 2024-02-23T00:49:57.664 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:56.271223+0000 mon.a (mon.0) 2728 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "219.12"}]: dispatch 2024-02-23T00:49:57.664 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:56.271441+0000 mgr.y (mgr.14517) 450 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "219.12"}]: dispatch 2024-02-23T00:49:57.664 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:56.304918+0000 mon.a (mon.0) 2729 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:49:57.666 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:57.688 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:49:57.691 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:57.692 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:49:57.792 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:49:57.822 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:57.843 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:49:57.845 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:57.846 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:49:57.947 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:49:57.978 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:57.999 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:49:58.002 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:58.002 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:49:58.103 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:49:58.133 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:58.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.092+0000 7ffb01c9f700 1 -- 172.21.15.47:0/3263633124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffafc0f8420 msgr2=0x7ffafc104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:58.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.092+0000 7ffb01c9f700 1 --2- 172.21.15.47:0/3263633124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffafc0f8420 0x7ffafc104a20 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 rx=0x7ffaf4009be0 tx=0x7ffaf4004d10).stop 2024-02-23T00:49:58.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.092+0000 7ffb01c9f700 1 -- 172.21.15.47:0/3263633124 shutdown_connections 2024-02-23T00:49:58.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.092+0000 7ffb01c9f700 1 --2- 172.21.15.47:0/3263633124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffafc0f8420 0x7ffafc104a20 unknown :-1 s=CLOSED pgs=2552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:58.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.092+0000 7ffb01c9f700 1 --2- 172.21.15.47:0/3263633124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffafc0fdbb0 0x7ffafc0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:58.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.092+0000 7ffb01c9f700 1 --2- 172.21.15.47:0/3263633124 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffafc0fe510 0x7ffafc0f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:58.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.092+0000 7ffb01c9f700 1 -- 172.21.15.47:0/3263633124 >> 172.21.15.47:0/3263633124 conn(0x7ffafc0f3800 msgr2=0x7ffafc0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:58.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.092+0000 7ffb01c9f700 1 -- 172.21.15.47:0/3263633124 shutdown_connections 2024-02-23T00:49:58.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.092+0000 7ffb01c9f700 1 -- 172.21.15.47:0/3263633124 wait complete. 2024-02-23T00:49:58.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.093+0000 7ffb01c9f700 1 librados: starting msgr at 2024-02-23T00:49:58.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.093+0000 7ffb01c9f700 1 librados: starting objecter 2024-02-23T00:49:58.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.093+0000 7ffb01c9f700 1 Processor -- start 2024-02-23T00:49:58.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.093+0000 7ffb01c9f700 1 -- start start 2024-02-23T00:49:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.093+0000 7ffb01c9f700 1 librados: setting wanted keys 2024-02-23T00:49:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.093+0000 7ffb01c9f700 1 librados: calling monclient init 2024-02-23T00:49:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.093+0000 7ffb01c9f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffafc0f8420 0x7ffafc10dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.093+0000 7ffb01c9f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffafc0fdbb0 0x7ffafc10e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.093+0000 7ffb01c9f700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffafc0fe510 0x7ffafc107f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.093+0000 7ffb01c9f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc1083f0 con 0x7ffafc0f8420 2024-02-23T00:49:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.093+0000 7ffb01c9f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc108560 con 0x7ffafc0fdbb0 2024-02-23T00:49:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.093+0000 7ffb01c9f700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc108720 con 0x7ffafc0fe510 2024-02-23T00:49:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.093+0000 7ffafb7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffafc0f8420 0x7ffafc10dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.093+0000 7ffafb7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffafc0f8420 0x7ffafc10dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39990/0 (socket says 172.21.15.47:39990) 2024-02-23T00:49:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.093+0000 7ffafb7fe700 1 -- 172.21.15.47:0/1305516132 learned_addr learned my addr 172.21.15.47:0/1305516132 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:49:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.093+0000 7ffafbfff700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffafc0fe510 0x7ffafc107f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.093+0000 7ffafaffd700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffafc0fdbb0 0x7ffafc10e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.093+0000 7ffafb7fe700 1 -- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffafc0fe510 msgr2=0x7ffafc107f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.094+0000 7ffafb7fe700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffafc0fe510 0x7ffafc107f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.094+0000 7ffafb7fe700 1 -- 172.21.15.47:0/1305516132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffafc0fdbb0 msgr2=0x7ffafc10e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.094+0000 7ffafb7fe700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffafc0fdbb0 0x7ffafc10e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.094+0000 7ffafb7fe700 1 -- 172.21.15.47:0/1305516132 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae4005040 con 0x7ffafc0f8420 2024-02-23T00:49:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.094+0000 7ffafaffd700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffafc0fdbb0 0x7ffafc10e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:49:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.094+0000 7ffafb7fe700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffafc0f8420 0x7ffafc10dd60 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 rx=0x7ffaec00e970 tx=0x7ffaec00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.094+0000 7ffaf8ff9700 10 client.?.objecter ms_handle_connect 0x7ffafc0f8420 2024-02-23T00:49:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.094+0000 7ffaf8ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:49:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.094+0000 7ffaf8ff9700 1 -- 172.21.15.47:0/1305516132 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaec00ee60 con 0x7ffafc0f8420 2024-02-23T00:49:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.094+0000 7ffaf8ff9700 1 -- 172.21.15.47:0/1305516132 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7ffaec00f040 con 0x7ffafc0f8420 2024-02-23T00:49:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.094+0000 7ffaf8ff9700 1 -- 172.21.15.47:0/1305516132 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaec00c5b0 con 0x7ffafc0f8420 2024-02-23T00:49:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.094+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaf40097c0 con 0x7ffafc0f8420 2024-02-23T00:49:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.094+0000 7ffb01c9f700 10 client.31182.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:49:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.094+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffafc108d10 con 0x7ffafc0f8420 2024-02-23T00:49:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.094+0000 7ffb01c9f700 1 librados: init done 2024-02-23T00:49:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.095+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7ffafc106ed0 con 0x7ffafc0f8420 2024-02-23T00:49:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.095+0000 7ffaf8ff9700 1 -- 172.21.15.47:0/1305516132 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7ffaec007b40 con 0x7ffafc0f8420 2024-02-23T00:49:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.096+0000 7ffaf8ff9700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ffae8062cc0 0x7ffae8065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.096+0000 7ffaf8ff9700 1 -- 172.21.15.47:0/1305516132 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(479..479 src has 48..479) v4 ==== 9220+0+0 (secure 0 0 0) 0x7ffaec07d290 con 0x7ffafc0f8420 2024-02-23T00:49:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.096+0000 7ffaf8ff9700 10 client.31182.objecter ms_dispatch 0x7ffafc001550 osd_map(479..479 src has 48..479) v4 2024-02-23T00:49:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.096+0000 7ffaf8ff9700 3 client.31182.objecter handle_osd_map got epochs [479,479] > 0 2024-02-23T00:49:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.096+0000 7ffaf8ff9700 3 client.31182.objecter handle_osd_map decoding full epoch 479 2024-02-23T00:49:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.096+0000 7ffafaffd700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ffae8062cc0 0x7ffae8065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.096+0000 7ffaf8ff9700 20 client.31182.objecter dump_active .. 0 homeless 2024-02-23T00:49:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.096+0000 7ffaf8ff9700 1 -- 172.21.15.47:0/1305516132 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v479) v1 ==== 54+0+16 (secure 0 0 0) 0x7ffaec084070 con 0x7ffafc0f8420 2024-02-23T00:49:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.096+0000 7ffaf8ff9700 10 client.31182.objecter ms_handle_connect 0x7ffae8062cc0 2024-02-23T00:49:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.096+0000 7ffafaffd700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ffae8062cc0 0x7ffae8065180 secure :-1 s=READY pgs=3093 cs=0 l=1 rev1=1 rx=0x7ffafc1096a0 tx=0x7ffae4009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.096+0000 7ffb01c9f700 10 client.31182.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.096+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_assign -1 1 2024-02-23T00:49:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.096+0000 7ffb01c9f700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7ffafc0598f0 0x7ffafc059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.096+0000 7ffb01c9f700 20 client.31182.objecter _get_session s=0x7ffafc1093e0 osd=0 3 2024-02-23T00:49:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.096+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc1093e0 osd=0 3 2024-02-23T00:49:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.096+0000 7ffb01c9f700 20 client.31182.objecter _get_session s=0x7ffafc1093e0 osd=0 3 2024-02-23T00:49:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.096+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_remove -1 1 2024-02-23T00:49:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.096+0000 7ffb01c9f700 20 client.31182.objecter get_session s=0x7ffafc1093e0 osd=0 3 2024-02-23T00:49:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.096+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_assign 0 1 2024-02-23T00:49:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.096+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc1093e0 osd=0 4 2024-02-23T00:49:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.096+0000 7ffb01c9f700 10 client.31182.objecter _send_command 1 2024-02-23T00:49:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.096+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffafc05fa60 con 0x7ffafc0598f0 2024-02-23T00:49:58.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.096+0000 7ffafbfff700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7ffafc0598f0 0x7ffafc059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:58.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.097+0000 7ffaf8ff9700 10 client.31182.objecter ms_handle_connect 0x7ffafc0598f0 2024-02-23T00:49:58.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.097+0000 7ffafbfff700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7ffafc0598f0 0x7ffafc059cd0 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:58.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.097+0000 7ffaf8ff9700 1 -- 172.21.15.47:0/1305516132 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ffafc05fa60 con 0x7ffafc0598f0 2024-02-23T00:49:58.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.097+0000 7ffaf8ff9700 10 client.31182.objecter ms_dispatch 0x7ffafc001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:49:58.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.097+0000 7ffaf8ff9700 10 client.31182.objecter _finish_command 1 = system:0 2024-02-23T00:49:58.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.097+0000 7ffaf8ff9700 20 client.31182.objecter put_session s=0x7ffafc1093e0 osd=0 4 2024-02-23T00:49:58.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.097+0000 7ffaf8ff9700 15 client.31182.objecter _session_command_op_remove 0 1 2024-02-23T00:49:58.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.112+0000 7ffb01c9f700 10 client.31182.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:49:58.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.112+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_assign -1 2 2024-02-23T00:49:58.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.112+0000 7ffb01c9f700 20 client.31182.objecter _get_session s=0x7ffafc1093e0 osd=0 3 2024-02-23T00:49:58.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.112+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc1093e0 osd=0 3 2024-02-23T00:49:58.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.112+0000 7ffb01c9f700 20 client.31182.objecter _get_session s=0x7ffafc1093e0 osd=0 3 2024-02-23T00:49:58.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.112+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_remove -1 2 2024-02-23T00:49:58.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.112+0000 7ffb01c9f700 20 client.31182.objecter get_session s=0x7ffafc1093e0 osd=0 3 2024-02-23T00:49:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.112+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_assign 0 2 2024-02-23T00:49:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.112+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc1093e0 osd=0 4 2024-02-23T00:49:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.112+0000 7ffb01c9f700 10 client.31182.objecter _send_command 2 2024-02-23T00:49:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.112+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ffafc05fbd0 con 0x7ffafc0598f0 2024-02-23T00:49:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.114+0000 7ffaf8ff9700 1 -- 172.21.15.47:0/1305516132 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ffafc05fbd0 con 0x7ffafc0598f0 2024-02-23T00:49:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.114+0000 7ffaf8ff9700 10 client.31182.objecter ms_dispatch 0x7ffafc001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:49:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.114+0000 7ffaf8ff9700 10 client.31182.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:49:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.114+0000 7ffaf8ff9700 20 client.31182.objecter put_session s=0x7ffafc1093e0 osd=0 4 2024-02-23T00:49:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.114+0000 7ffaf8ff9700 15 client.31182.objecter _session_command_op_remove 0 2 2024-02-23T00:49:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.114+0000 7ffb01c9f700 10 client.31182.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.114+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_assign -1 3 2024-02-23T00:49:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.114+0000 7ffb01c9f700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7ffafc028540 0x7ffafc118c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.114+0000 7ffb01c9f700 20 client.31182.objecter _get_session s=0x7ffafc0595a0 osd=1 3 2024-02-23T00:49:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.114+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc0595a0 osd=1 3 2024-02-23T00:49:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.114+0000 7ffb01c9f700 20 client.31182.objecter _get_session s=0x7ffafc0595a0 osd=1 3 2024-02-23T00:49:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.114+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_remove -1 3 2024-02-23T00:49:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.114+0000 7ffb01c9f700 20 client.31182.objecter get_session s=0x7ffafc0595a0 osd=1 3 2024-02-23T00:49:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.114+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_assign 1 3 2024-02-23T00:49:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.114+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc0595a0 osd=1 4 2024-02-23T00:49:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.114+0000 7ffb01c9f700 10 client.31182.objecter _send_command 3 2024-02-23T00:49:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.114+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffafc059710 con 0x7ffafc028540 2024-02-23T00:49:58.153 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:49:58.153 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:49:58.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.114+0000 7ffafb7fe700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7ffafc028540 0x7ffafc118c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:58.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.114+0000 7ffaf8ff9700 10 client.31182.objecter ms_handle_connect 0x7ffafc028540 2024-02-23T00:49:58.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.114+0000 7ffafb7fe700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7ffafc028540 0x7ffafc118c10 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:58.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.115+0000 7ffaf8ff9700 1 -- 172.21.15.47:0/1305516132 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ffafc059710 con 0x7ffafc028540 2024-02-23T00:49:58.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.115+0000 7ffaf8ff9700 10 client.31182.objecter ms_dispatch 0x7ffafc001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:49:58.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.115+0000 7ffaf8ff9700 10 client.31182.objecter _finish_command 3 = system:0 2024-02-23T00:49:58.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.115+0000 7ffaf8ff9700 20 client.31182.objecter put_session s=0x7ffafc0595a0 osd=1 4 2024-02-23T00:49:58.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.115+0000 7ffaf8ff9700 15 client.31182.objecter _session_command_op_remove 1 3 2024-02-23T00:49:58.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.130+0000 7ffb01c9f700 10 client.31182.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:49:58.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.130+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_assign -1 4 2024-02-23T00:49:58.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.130+0000 7ffb01c9f700 20 client.31182.objecter _get_session s=0x7ffafc0595a0 osd=1 3 2024-02-23T00:49:58.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.130+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc0595a0 osd=1 3 2024-02-23T00:49:58.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.130+0000 7ffb01c9f700 20 client.31182.objecter _get_session s=0x7ffafc0595a0 osd=1 3 2024-02-23T00:49:58.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.130+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_remove -1 4 2024-02-23T00:49:58.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.130+0000 7ffb01c9f700 20 client.31182.objecter get_session s=0x7ffafc0595a0 osd=1 3 2024-02-23T00:49:58.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.130+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_assign 1 4 2024-02-23T00:49:58.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.130+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc0595a0 osd=1 4 2024-02-23T00:49:58.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.130+0000 7ffb01c9f700 10 client.31182.objecter _send_command 4 2024-02-23T00:49:58.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.130+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ffafc0ff870 con 0x7ffafc028540 2024-02-23T00:49:58.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.131+0000 7ffaf8ff9700 1 -- 172.21.15.47:0/1305516132 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ffafc0ff870 con 0x7ffafc028540 2024-02-23T00:49:58.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.131+0000 7ffaf8ff9700 10 client.31182.objecter ms_dispatch 0x7ffafc001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:49:58.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.131+0000 7ffaf8ff9700 10 client.31182.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:49:58.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.131+0000 7ffaf8ff9700 20 client.31182.objecter put_session s=0x7ffafc0595a0 osd=1 4 2024-02-23T00:49:58.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.131+0000 7ffaf8ff9700 15 client.31182.objecter _session_command_op_remove 1 4 2024-02-23T00:49:58.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.131+0000 7ffb01c9f700 10 client.31182.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:58.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.131+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_assign -1 5 2024-02-23T00:49:58.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.131+0000 7ffb01c9f700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7ffafc0fed00 0x7ffafc06c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:58.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.131+0000 7ffb01c9f700 20 client.31182.objecter _get_session s=0x7ffafc0feb90 osd=2 3 2024-02-23T00:49:58.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.131+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc0feb90 osd=2 3 2024-02-23T00:49:58.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.131+0000 7ffb01c9f700 20 client.31182.objecter _get_session s=0x7ffafc0feb90 osd=2 3 2024-02-23T00:49:58.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.131+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_remove -1 5 2024-02-23T00:49:58.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.131+0000 7ffb01c9f700 20 client.31182.objecter get_session s=0x7ffafc0feb90 osd=2 3 2024-02-23T00:49:58.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.131+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_assign 2 5 2024-02-23T00:49:58.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.131+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc0feb90 osd=2 4 2024-02-23T00:49:58.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.131+0000 7ffb01c9f700 10 client.31182.objecter _send_command 5 2024-02-23T00:49:58.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.131+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffafc0281a0 con 0x7ffafc0fed00 2024-02-23T00:49:58.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.132+0000 7ffafaffd700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7ffafc0fed00 0x7ffafc06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:58.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.132+0000 7ffaf8ff9700 10 client.31182.objecter ms_handle_connect 0x7ffafc0fed00 2024-02-23T00:49:58.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.132+0000 7ffafaffd700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7ffafc0fed00 0x7ffafc06c090 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:58.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.132+0000 7ffaf8ff9700 1 -- 172.21.15.47:0/1305516132 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ffafc0281a0 con 0x7ffafc0fed00 2024-02-23T00:49:58.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.132+0000 7ffaf8ff9700 10 client.31182.objecter ms_dispatch 0x7ffafc001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:49:58.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.132+0000 7ffaf8ff9700 10 client.31182.objecter _finish_command 5 = system:0 2024-02-23T00:49:58.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.132+0000 7ffaf8ff9700 20 client.31182.objecter put_session s=0x7ffafc0feb90 osd=2 4 2024-02-23T00:49:58.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.132+0000 7ffaf8ff9700 15 client.31182.objecter _session_command_op_remove 2 5 2024-02-23T00:49:58.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.148+0000 7ffb01c9f700 10 client.31182.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:49:58.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.148+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_assign -1 6 2024-02-23T00:49:58.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.148+0000 7ffb01c9f700 20 client.31182.objecter _get_session s=0x7ffafc0feb90 osd=2 3 2024-02-23T00:49:58.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.148+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc0feb90 osd=2 3 2024-02-23T00:49:58.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.148+0000 7ffb01c9f700 20 client.31182.objecter _get_session s=0x7ffafc0feb90 osd=2 3 2024-02-23T00:49:58.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.148+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_remove -1 6 2024-02-23T00:49:58.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.148+0000 7ffb01c9f700 20 client.31182.objecter get_session s=0x7ffafc0feb90 osd=2 3 2024-02-23T00:49:58.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.148+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_assign 2 6 2024-02-23T00:49:58.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.148+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc0feb90 osd=2 4 2024-02-23T00:49:58.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.148+0000 7ffb01c9f700 10 client.31182.objecter _send_command 6 2024-02-23T00:49:58.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.148+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ffafc028370 con 0x7ffafc0fed00 2024-02-23T00:49:58.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.149+0000 7ffaf8ff9700 1 -- 172.21.15.47:0/1305516132 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ffafc028370 con 0x7ffafc0fed00 2024-02-23T00:49:58.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.149+0000 7ffaf8ff9700 10 client.31182.objecter ms_dispatch 0x7ffafc001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:49:58.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.149+0000 7ffaf8ff9700 10 client.31182.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:49:58.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.149+0000 7ffaf8ff9700 20 client.31182.objecter put_session s=0x7ffafc0feb90 osd=2 4 2024-02-23T00:49:58.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.149+0000 7ffaf8ff9700 15 client.31182.objecter _session_command_op_remove 2 6 2024-02-23T00:49:58.158 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:49:58.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.149+0000 7ffb01c9f700 10 client.31182.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:58.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.149+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_assign -1 7 2024-02-23T00:49:58.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.149+0000 7ffb01c9f700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7ffafc027770 0x7ffafc121a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:58.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.149+0000 7ffb01c9f700 20 client.31182.objecter _get_session s=0x7ffafc027600 osd=3 3 2024-02-23T00:49:58.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.149+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc027600 osd=3 3 2024-02-23T00:49:58.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.149+0000 7ffb01c9f700 20 client.31182.objecter _get_session s=0x7ffafc027600 osd=3 3 2024-02-23T00:49:58.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.149+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_remove -1 7 2024-02-23T00:49:58.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.149+0000 7ffb01c9f700 20 client.31182.objecter get_session s=0x7ffafc027600 osd=3 3 2024-02-23T00:49:58.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.149+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_assign 3 7 2024-02-23T00:49:58.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.149+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc027600 osd=3 4 2024-02-23T00:49:58.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.149+0000 7ffb01c9f700 10 client.31182.objecter _send_command 7 2024-02-23T00:49:58.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.149+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffafc124780 con 0x7ffafc027770 2024-02-23T00:49:58.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.149+0000 7ffafbfff700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7ffafc027770 0x7ffafc121a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:58.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.150+0000 7ffaf8ff9700 10 client.31182.objecter ms_handle_connect 0x7ffafc027770 2024-02-23T00:49:58.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.150+0000 7ffafbfff700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7ffafc027770 0x7ffafc121a40 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:58.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.150+0000 7ffaf8ff9700 1 -- 172.21.15.47:0/1305516132 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ffafc124780 con 0x7ffafc027770 2024-02-23T00:49:58.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.150+0000 7ffaf8ff9700 10 client.31182.objecter ms_dispatch 0x7ffafc001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:49:58.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.150+0000 7ffaf8ff9700 10 client.31182.objecter _finish_command 7 = system:0 2024-02-23T00:49:58.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.150+0000 7ffaf8ff9700 20 client.31182.objecter put_session s=0x7ffafc027600 osd=3 4 2024-02-23T00:49:58.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.150+0000 7ffaf8ff9700 15 client.31182.objecter _session_command_op_remove 3 7 2024-02-23T00:49:58.162 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:49:58.162 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:49:58.162 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:49:58.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.165+0000 7ffb01c9f700 10 client.31182.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:49:58.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.165+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_assign -1 8 2024-02-23T00:49:58.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.165+0000 7ffb01c9f700 20 client.31182.objecter _get_session s=0x7ffafc027600 osd=3 3 2024-02-23T00:49:58.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.165+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc027600 osd=3 3 2024-02-23T00:49:58.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.165+0000 7ffb01c9f700 20 client.31182.objecter _get_session s=0x7ffafc027600 osd=3 3 2024-02-23T00:49:58.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.165+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_remove -1 8 2024-02-23T00:49:58.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.165+0000 7ffb01c9f700 20 client.31182.objecter get_session s=0x7ffafc027600 osd=3 3 2024-02-23T00:49:58.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.165+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_assign 3 8 2024-02-23T00:49:58.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.165+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc027600 osd=3 4 2024-02-23T00:49:58.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.165+0000 7ffb01c9f700 10 client.31182.objecter _send_command 8 2024-02-23T00:49:58.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.165+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ffafc06c7c0 con 0x7ffafc027770 2024-02-23T00:49:58.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.166+0000 7ffaf8ff9700 1 -- 172.21.15.47:0/1305516132 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ffafc06c7c0 con 0x7ffafc027770 2024-02-23T00:49:58.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.166+0000 7ffaf8ff9700 10 client.31182.objecter ms_dispatch 0x7ffafc001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:49:58.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.166+0000 7ffaf8ff9700 10 client.31182.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:49:58.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.166+0000 7ffaf8ff9700 20 client.31182.objecter put_session s=0x7ffafc027600 osd=3 4 2024-02-23T00:49:58.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.166+0000 7ffaf8ff9700 15 client.31182.objecter _session_command_op_remove 3 8 2024-02-23T00:49:58.169 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:49:58.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.166+0000 7ffb01c9f700 10 client.31182.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:58.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.166+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_assign -1 9 2024-02-23T00:49:58.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.166+0000 7ffb01c9f700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7ffafc06c960 0x7ffafc121f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:58.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.166+0000 7ffb01c9f700 20 client.31182.objecter _get_session s=0x7ffafc06c7c0 osd=4 3 2024-02-23T00:49:58.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.166+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc06c7c0 osd=4 3 2024-02-23T00:49:58.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.166+0000 7ffb01c9f700 20 client.31182.objecter _get_session s=0x7ffafc06c7c0 osd=4 3 2024-02-23T00:49:58.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.166+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_remove -1 9 2024-02-23T00:49:58.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.166+0000 7ffb01c9f700 20 client.31182.objecter get_session s=0x7ffafc06c7c0 osd=4 3 2024-02-23T00:49:58.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.166+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_assign 4 9 2024-02-23T00:49:58.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.166+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc06c7c0 osd=4 4 2024-02-23T00:49:58.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.166+0000 7ffb01c9f700 10 client.31182.objecter _send_command 9 2024-02-23T00:49:58.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.166+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffafc06be60 con 0x7ffafc06c960 2024-02-23T00:49:58.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.167+0000 7ffafb7fe700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7ffafc06c960 0x7ffafc121f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:58.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.167+0000 7ffaf8ff9700 10 client.31182.objecter ms_handle_connect 0x7ffafc06c960 2024-02-23T00:49:58.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.167+0000 7ffafb7fe700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7ffafc06c960 0x7ffafc121f80 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:58.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.168+0000 7ffaf8ff9700 1 -- 172.21.15.47:0/1305516132 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ffafc06be60 con 0x7ffafc06c960 2024-02-23T00:49:58.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.168+0000 7ffaf8ff9700 10 client.31182.objecter ms_dispatch 0x7ffafc001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:49:58.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.168+0000 7ffaf8ff9700 10 client.31182.objecter _finish_command 9 = system:0 2024-02-23T00:49:58.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.168+0000 7ffaf8ff9700 20 client.31182.objecter put_session s=0x7ffafc06c7c0 osd=4 4 2024-02-23T00:49:58.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.168+0000 7ffaf8ff9700 15 client.31182.objecter _session_command_op_remove 4 9 2024-02-23T00:49:58.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.183+0000 7ffb01c9f700 10 client.31182.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:49:58.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.183+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_assign -1 10 2024-02-23T00:49:58.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.183+0000 7ffb01c9f700 20 client.31182.objecter _get_session s=0x7ffafc06c7c0 osd=4 3 2024-02-23T00:49:58.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.183+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc06c7c0 osd=4 3 2024-02-23T00:49:58.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.183+0000 7ffb01c9f700 20 client.31182.objecter _get_session s=0x7ffafc06c7c0 osd=4 3 2024-02-23T00:49:58.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.183+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_remove -1 10 2024-02-23T00:49:58.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.183+0000 7ffb01c9f700 20 client.31182.objecter get_session s=0x7ffafc06c7c0 osd=4 3 2024-02-23T00:49:58.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.183+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_assign 4 10 2024-02-23T00:49:58.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.183+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc06c7c0 osd=4 4 2024-02-23T00:49:58.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.183+0000 7ffb01c9f700 10 client.31182.objecter _send_command 10 2024-02-23T00:49:58.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.183+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ffafc06be60 con 0x7ffafc06c960 2024-02-23T00:49:58.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.185+0000 7ffaf8ff9700 1 -- 172.21.15.47:0/1305516132 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ffafc06be60 con 0x7ffafc06c960 2024-02-23T00:49:58.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.185+0000 7ffaf8ff9700 10 client.31182.objecter ms_dispatch 0x7ffafc001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:49:58.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.185+0000 7ffaf8ff9700 10 client.31182.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:49:58.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.185+0000 7ffaf8ff9700 20 client.31182.objecter put_session s=0x7ffafc06c7c0 osd=4 4 2024-02-23T00:49:58.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.185+0000 7ffaf8ff9700 15 client.31182.objecter _session_command_op_remove 4 10 2024-02-23T00:49:58.188 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:49:58.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.185+0000 7ffb01c9f700 10 client.31182.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:58.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.185+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_assign -1 11 2024-02-23T00:49:58.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.185+0000 7ffb01c9f700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7ffafc122980 0x7ffafc122de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:58.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.185+0000 7ffb01c9f700 20 client.31182.objecter _get_session s=0x7ffafc06be60 osd=5 3 2024-02-23T00:49:58.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.185+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc06be60 osd=5 3 2024-02-23T00:49:58.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.185+0000 7ffb01c9f700 20 client.31182.objecter _get_session s=0x7ffafc06be60 osd=5 3 2024-02-23T00:49:58.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.185+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_remove -1 11 2024-02-23T00:49:58.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.185+0000 7ffb01c9f700 20 client.31182.objecter get_session s=0x7ffafc06be60 osd=5 3 2024-02-23T00:49:58.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.185+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_assign 5 11 2024-02-23T00:49:58.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.185+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc06be60 osd=5 4 2024-02-23T00:49:58.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.185+0000 7ffb01c9f700 10 client.31182.objecter _send_command 11 2024-02-23T00:49:58.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.185+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffafc12b100 con 0x7ffafc122980 2024-02-23T00:49:58.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.185+0000 7ffafaffd700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7ffafc122980 0x7ffafc122de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:58.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.186+0000 7ffaf8ff9700 10 client.31182.objecter ms_handle_connect 0x7ffafc122980 2024-02-23T00:49:58.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.186+0000 7ffafaffd700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7ffafc122980 0x7ffafc122de0 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:58.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.186+0000 7ffaf8ff9700 1 -- 172.21.15.47:0/1305516132 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ffafc12b100 con 0x7ffafc122980 2024-02-23T00:49:58.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.186+0000 7ffaf8ff9700 10 client.31182.objecter ms_dispatch 0x7ffafc001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:49:58.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.186+0000 7ffaf8ff9700 10 client.31182.objecter _finish_command 11 = system:0 2024-02-23T00:49:58.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.186+0000 7ffaf8ff9700 20 client.31182.objecter put_session s=0x7ffafc06be60 osd=5 4 2024-02-23T00:49:58.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.186+0000 7ffaf8ff9700 15 client.31182.objecter _session_command_op_remove 5 11 2024-02-23T00:49:58.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.201+0000 7ffb01c9f700 10 client.31182.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:49:58.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.201+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_assign -1 12 2024-02-23T00:49:58.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.201+0000 7ffb01c9f700 20 client.31182.objecter _get_session s=0x7ffafc06be60 osd=5 3 2024-02-23T00:49:58.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.201+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc06be60 osd=5 3 2024-02-23T00:49:58.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.201+0000 7ffb01c9f700 20 client.31182.objecter _get_session s=0x7ffafc06be60 osd=5 3 2024-02-23T00:49:58.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.201+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_remove -1 12 2024-02-23T00:49:58.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.201+0000 7ffb01c9f700 20 client.31182.objecter get_session s=0x7ffafc06be60 osd=5 3 2024-02-23T00:49:58.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.201+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_assign 5 12 2024-02-23T00:49:58.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.201+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc06be60 osd=5 4 2024-02-23T00:49:58.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.201+0000 7ffb01c9f700 10 client.31182.objecter _send_command 12 2024-02-23T00:49:58.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.201+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ffafc1289a0 con 0x7ffafc122980 2024-02-23T00:49:58.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.203+0000 7ffaf8ff9700 1 -- 172.21.15.47:0/1305516132 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ffafc1289a0 con 0x7ffafc122980 2024-02-23T00:49:58.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.203+0000 7ffaf8ff9700 10 client.31182.objecter ms_dispatch 0x7ffafc001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:49:58.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.203+0000 7ffaf8ff9700 10 client.31182.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:49:58.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.203+0000 7ffaf8ff9700 20 client.31182.objecter put_session s=0x7ffafc06be60 osd=5 4 2024-02-23T00:49:58.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.203+0000 7ffaf8ff9700 15 client.31182.objecter _session_command_op_remove 5 12 2024-02-23T00:49:58.206 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:49:58.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.203+0000 7ffb01c9f700 10 client.31182.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:58.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.203+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_assign -1 13 2024-02-23T00:49:58.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.203+0000 7ffb01c9f700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7ffafc129020 0x7ffafc129460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:58.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.203+0000 7ffb01c9f700 20 client.31182.objecter _get_session s=0x7ffafc1289a0 osd=6 3 2024-02-23T00:49:58.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.203+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc1289a0 osd=6 3 2024-02-23T00:49:58.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.203+0000 7ffb01c9f700 20 client.31182.objecter _get_session s=0x7ffafc1289a0 osd=6 3 2024-02-23T00:49:58.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.203+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_remove -1 13 2024-02-23T00:49:58.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.203+0000 7ffb01c9f700 20 client.31182.objecter get_session s=0x7ffafc1289a0 osd=6 3 2024-02-23T00:49:58.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.203+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_assign 6 13 2024-02-23T00:49:58.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.203+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc1289a0 osd=6 4 2024-02-23T00:49:58.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.203+0000 7ffb01c9f700 10 client.31182.objecter _send_command 13 2024-02-23T00:49:58.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.203+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffafc12e430 con 0x7ffafc129020 2024-02-23T00:49:58.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.203+0000 7ffafbfff700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7ffafc129020 0x7ffafc129460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:58.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.204+0000 7ffaf8ff9700 10 client.31182.objecter ms_handle_connect 0x7ffafc129020 2024-02-23T00:49:58.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.204+0000 7ffafbfff700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7ffafc129020 0x7ffafc129460 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:58.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.204+0000 7ffaf8ff9700 1 -- 172.21.15.47:0/1305516132 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ffafc12e430 con 0x7ffafc129020 2024-02-23T00:49:58.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.204+0000 7ffaf8ff9700 10 client.31182.objecter ms_dispatch 0x7ffafc001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:49:58.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.204+0000 7ffaf8ff9700 10 client.31182.objecter _finish_command 13 = system:0 2024-02-23T00:49:58.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.204+0000 7ffaf8ff9700 20 client.31182.objecter put_session s=0x7ffafc1289a0 osd=6 4 2024-02-23T00:49:58.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.204+0000 7ffaf8ff9700 15 client.31182.objecter _session_command_op_remove 6 13 2024-02-23T00:49:58.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.219+0000 7ffb01c9f700 10 client.31182.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:49:58.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.219+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_assign -1 14 2024-02-23T00:49:58.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.219+0000 7ffb01c9f700 20 client.31182.objecter _get_session s=0x7ffafc1289a0 osd=6 3 2024-02-23T00:49:58.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.219+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc1289a0 osd=6 3 2024-02-23T00:49:58.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.219+0000 7ffb01c9f700 20 client.31182.objecter _get_session s=0x7ffafc1289a0 osd=6 3 2024-02-23T00:49:58.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.219+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_remove -1 14 2024-02-23T00:49:58.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.219+0000 7ffb01c9f700 20 client.31182.objecter get_session s=0x7ffafc1289a0 osd=6 3 2024-02-23T00:49:58.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.219+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_assign 6 14 2024-02-23T00:49:58.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.219+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc1289a0 osd=6 4 2024-02-23T00:49:58.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.219+0000 7ffb01c9f700 10 client.31182.objecter _send_command 14 2024-02-23T00:49:58.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.219+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ffafc129a60 con 0x7ffafc129020 2024-02-23T00:49:58.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.221+0000 7ffaf8ff9700 1 -- 172.21.15.47:0/1305516132 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ffafc129a60 con 0x7ffafc129020 2024-02-23T00:49:58.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.221+0000 7ffaf8ff9700 10 client.31182.objecter ms_dispatch 0x7ffafc001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:49:58.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.221+0000 7ffaf8ff9700 10 client.31182.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:49:58.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.221+0000 7ffaf8ff9700 20 client.31182.objecter put_session s=0x7ffafc1289a0 osd=6 4 2024-02-23T00:49:58.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.221+0000 7ffaf8ff9700 15 client.31182.objecter _session_command_op_remove 6 14 2024-02-23T00:49:58.224 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:49:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.221+0000 7ffb01c9f700 10 client.31182.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:49:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.221+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_assign -1 15 2024-02-23T00:49:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.221+0000 7ffb01c9f700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7ffafc12a0e0 0x7ffafc12a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:49:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.221+0000 7ffb01c9f700 20 client.31182.objecter _get_session s=0x7ffafc129a60 osd=7 3 2024-02-23T00:49:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.221+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc129a60 osd=7 3 2024-02-23T00:49:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.221+0000 7ffb01c9f700 20 client.31182.objecter _get_session s=0x7ffafc129a60 osd=7 3 2024-02-23T00:49:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.221+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_remove -1 15 2024-02-23T00:49:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.221+0000 7ffb01c9f700 20 client.31182.objecter get_session s=0x7ffafc129a60 osd=7 3 2024-02-23T00:49:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.221+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_assign 7 15 2024-02-23T00:49:58.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.221+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc129a60 osd=7 4 2024-02-23T00:49:58.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.221+0000 7ffb01c9f700 10 client.31182.objecter _send_command 15 2024-02-23T00:49:58.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.221+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffafc12abe0 con 0x7ffafc12a0e0 2024-02-23T00:49:58.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.222+0000 7ffafb7fe700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7ffafc12a0e0 0x7ffafc12a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:49:58.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.222+0000 7ffaf8ff9700 10 client.31182.objecter ms_handle_connect 0x7ffafc12a0e0 2024-02-23T00:49:58.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.222+0000 7ffafb7fe700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7ffafc12a0e0 0x7ffafc12a500 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:49:58.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.223+0000 7ffaf8ff9700 1 -- 172.21.15.47:0/1305516132 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ffafc12abe0 con 0x7ffafc12a0e0 2024-02-23T00:49:58.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.223+0000 7ffaf8ff9700 10 client.31182.objecter ms_dispatch 0x7ffafc001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:49:58.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.223+0000 7ffaf8ff9700 10 client.31182.objecter _finish_command 15 = system:0 2024-02-23T00:49:58.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.223+0000 7ffaf8ff9700 20 client.31182.objecter put_session s=0x7ffafc129a60 osd=7 4 2024-02-23T00:49:58.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.223+0000 7ffaf8ff9700 15 client.31182.objecter _session_command_op_remove 7 15 2024-02-23T00:49:58.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.238+0000 7ffb01c9f700 10 client.31182.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:49:58.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.238+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_assign -1 16 2024-02-23T00:49:58.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.238+0000 7ffb01c9f700 20 client.31182.objecter _get_session s=0x7ffafc129a60 osd=7 3 2024-02-23T00:49:58.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.238+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc129a60 osd=7 3 2024-02-23T00:49:58.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.238+0000 7ffb01c9f700 20 client.31182.objecter _get_session s=0x7ffafc129a60 osd=7 3 2024-02-23T00:49:58.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.238+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_remove -1 16 2024-02-23T00:49:58.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.238+0000 7ffb01c9f700 20 client.31182.objecter get_session s=0x7ffafc129a60 osd=7 3 2024-02-23T00:49:58.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.238+0000 7ffb01c9f700 15 client.31182.objecter _session_command_op_assign 7 16 2024-02-23T00:49:58.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.238+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc129a60 osd=7 4 2024-02-23T00:49:58.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.238+0000 7ffb01c9f700 10 client.31182.objecter _send_command 16 2024-02-23T00:49:58.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.238+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ffafc12abe0 con 0x7ffafc12a0e0 2024-02-23T00:49:58.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.239+0000 7ffaf8ff9700 1 -- 172.21.15.47:0/1305516132 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ffafc12abe0 con 0x7ffafc12a0e0 2024-02-23T00:49:58.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.239+0000 7ffaf8ff9700 10 client.31182.objecter ms_dispatch 0x7ffafc001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:49:58.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.239+0000 7ffaf8ff9700 10 client.31182.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:49:58.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.239+0000 7ffaf8ff9700 20 client.31182.objecter put_session s=0x7ffafc129a60 osd=7 4 2024-02-23T00:49:58.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.239+0000 7ffaf8ff9700 15 client.31182.objecter _session_command_op_remove 7 16 2024-02-23T00:49:58.243 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:49:58.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 10 librados: watch_flush enter 2024-02-23T00:49:58.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 10 librados: watch_flush exit 2024-02-23T00:49:58.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 10 client.31182.objecter close_session for osd.0 2024-02-23T00:49:58.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7ffafc0598f0 msgr2=0x7ffafc059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:58.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7ffafc0598f0 0x7ffafc059cd0 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:58.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc1093e0 osd=0 1 2024-02-23T00:49:58.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 10 client.31182.objecter close_session for osd.1 2024-02-23T00:49:58.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7ffafc028540 msgr2=0x7ffafc118c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7ffafc028540 0x7ffafc118c10 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc0595a0 osd=1 1 2024-02-23T00:49:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 10 client.31182.objecter close_session for osd.2 2024-02-23T00:49:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7ffafc0fed00 msgr2=0x7ffafc06c090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7ffafc0fed00 0x7ffafc06c090 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc0feb90 osd=2 1 2024-02-23T00:49:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 10 client.31182.objecter close_session for osd.3 2024-02-23T00:49:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffafbfff700 1 -- 172.21.15.47:0/1305516132 reap_dead start 2024-02-23T00:49:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7ffafc027770 msgr2=0x7ffafc121a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7ffafc027770 0x7ffafc121a40 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc027600 osd=3 1 2024-02-23T00:49:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 10 client.31182.objecter close_session for osd.4 2024-02-23T00:49:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7ffafc06c960 msgr2=0x7ffafc121f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7ffafc06c960 0x7ffafc121f80 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:58.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc06c7c0 osd=4 1 2024-02-23T00:49:58.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 10 client.31182.objecter close_session for osd.5 2024-02-23T00:49:58.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7ffafc122980 msgr2=0x7ffafc122de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:58.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7ffafc122980 0x7ffafc122de0 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:58.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc06be60 osd=5 1 2024-02-23T00:49:58.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 10 client.31182.objecter close_session for osd.6 2024-02-23T00:49:58.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7ffafc129020 msgr2=0x7ffafc129460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:58.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7ffafc129020 0x7ffafc129460 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:58.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc1289a0 osd=6 1 2024-02-23T00:49:58.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 10 client.31182.objecter close_session for osd.7 2024-02-23T00:49:58.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7ffafc12a0e0 msgr2=0x7ffafc12a500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:58.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7ffafc12a0e0 0x7ffafc12a500 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:58.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 20 client.31182.objecter put_session s=0x7ffafc129a60 osd=7 1 2024-02-23T00:49:58.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 20 client.31182.objecter shutdown clearing up homeless session... 2024-02-23T00:49:58.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 10 client.31182.objecter successfully canceled tick 2024-02-23T00:49:58.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffafbfff700 1 -- 172.21.15.47:0/1305516132 reap_dead start 2024-02-23T00:49:58.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ffae8062cc0 msgr2=0x7ffae8065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:58.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ffae8062cc0 0x7ffae8065180 secure :-1 s=READY pgs=3093 cs=0 l=1 rev1=1 rx=0x7ffafc1096a0 tx=0x7ffae4009cb0).stop 2024-02-23T00:49:58.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffafc0f8420 msgr2=0x7ffafc10dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:58.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffafc0f8420 0x7ffafc10dd60 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 rx=0x7ffaec00e970 tx=0x7ffaec00b040).stop 2024-02-23T00:49:58.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 shutdown_connections 2024-02-23T00:49:58.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffafc0f8420 0x7ffafc10dd60 unknown :-1 s=CLOSED pgs=2553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:58.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 1 --2- 172.21.15.47:0/1305516132 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ffae8062cc0 0x7ffae8065180 unknown :-1 s=CLOSED pgs=3093 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:58.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 >> 172.21.15.47:0/1305516132 conn(0x7ffafc0f3800 msgr2=0x7ffafc0f5150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:58.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 shutdown_connections 2024-02-23T00:49:58.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 1 -- 172.21.15.47:0/1305516132 wait complete. 2024-02-23T00:49:58.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.240+0000 7ffb01c9f700 1 librados: shutdown 2024-02-23T00:49:58.248 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:49:58.248 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:49:58.248 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:49:58.248 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:49:58.248 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:49:58.248 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:49:58.248 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:49:58.248 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:49:58.263 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:49:58.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.267+0000 7f3f767fc700 1 -- 172.21.15.47:0/3505902396 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v480) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3f70021030 con 0x7f3f780f7400 2024-02-23T00:49:58.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.268+0000 7f3f7f68d700 10 librados: watch_flush enter 2024-02-23T00:49:58.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.268+0000 7f3f7f68d700 10 librados: watch_flush exit 2024-02-23T00:49:58.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.268+0000 7f3f7f68d700 20 client.41501.objecter shutdown clearing up homeless session... 2024-02-23T00:49:58.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.268+0000 7f3f7f68d700 10 client.41501.objecter successfully canceled tick 2024-02-23T00:49:58.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.269+0000 7f3f7f68d700 1 -- 172.21.15.47:0/3505902396 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3f60062cc0 msgr2=0x7f3f60065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:58.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.269+0000 7f3f7f68d700 1 --2- 172.21.15.47:0/3505902396 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3f60062cc0 0x7f3f60065180 secure :-1 s=READY pgs=3092 cs=0 l=1 rev1=1 rx=0x7f3f781099b0 tx=0x7f3f6c00b040).stop 2024-02-23T00:49:58.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.269+0000 7f3f7f68d700 1 -- 172.21.15.47:0/3505902396 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3f780f7400 msgr2=0x7f3f7810e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:49:58.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.269+0000 7f3f7f68d700 1 --2- 172.21.15.47:0/3505902396 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3f780f7400 0x7f3f7810e070 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 rx=0x7f3f70004750 tx=0x7f3f70004d10).stop 2024-02-23T00:49:58.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.269+0000 7f3f7f68d700 1 -- 172.21.15.47:0/3505902396 shutdown_connections 2024-02-23T00:49:58.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.269+0000 7f3f7f68d700 1 --2- 172.21.15.47:0/3505902396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f780fd510 0x7f3f78108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:58.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.269+0000 7f3f7f68d700 1 --2- 172.21.15.47:0/3505902396 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3f60062cc0 0x7f3f60065180 unknown :-1 s=CLOSED pgs=3092 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:58.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.269+0000 7f3f7f68d700 1 --2- 172.21.15.47:0/3505902396 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3f780f7400 0x7f3f7810e070 unknown :-1 s=CLOSED pgs=2701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:58.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.269+0000 7f3f7f68d700 1 --2- 172.21.15.47:0/3505902396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f780fcbb0 0x7f3f7810e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:49:58.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.269+0000 7f3f7f68d700 1 -- 172.21.15.47:0/3505902396 >> 172.21.15.47:0/3505902396 conn(0x7f3f780f2820 msgr2=0x7f3f78060e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:49:58.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.269+0000 7f3f7f68d700 1 -- 172.21.15.47:0/3505902396 shutdown_connections 2024-02-23T00:49:58.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.269+0000 7f3f7f68d700 1 -- 172.21.15.47:0/3505902396 wait complete. 2024-02-23T00:49:58.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:49:58.269+0000 7f3f7f68d700 1 librados: shutdown 2024-02-23T00:49:58.273 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is set 2024-02-23T00:49:58.293 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:49:58.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:57.319738+0000 mgr.y (mgr.14517) 451 : cluster [DBG] pgmap v717: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-23T00:49:58.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:57.380279+0000 mon.c (mon.2) 311 : audit [INF] from='client.? 172.21.15.47:0/3505902396' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:49:58.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:57.380569+0000 mon.a (mon.0) 2730 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:49:58.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:58.097078+0000 mon.a (mon.0) 2731 : audit [DBG] from='client.? 172.21.15.47:0/1305516132' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:49:58.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:57.319738+0000 mgr.y (mgr.14517) 451 : cluster [DBG] pgmap v717: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-23T00:49:58.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:57.380279+0000 mon.c (mon.2) 311 : audit [INF] from='client.? 172.21.15.47:0/3505902396' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:49:58.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:57.380569+0000 mon.a (mon.0) 2730 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:49:58.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:58.097078+0000 mon.a (mon.0) 2731 : audit [DBG] from='client.? 172.21.15.47:0/1305516132' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:49:58.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:57.319738+0000 mgr.y (mgr.14517) 451 : cluster [DBG] pgmap v717: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-23T00:49:58.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:57.380279+0000 mon.c (mon.2) 311 : audit [INF] from='client.? 172.21.15.47:0/3505902396' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:49:58.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:57.380569+0000 mon.a (mon.0) 2730 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:49:58.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:58.097078+0000 mon.a (mon.0) 2731 : audit [DBG] from='client.? 172.21.15.47:0/1305516132' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:49:58.939 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-23T00:49:58.939 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight 0 1 2024-02-23T00:49:59.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:58.144503+0000 osd.7 (osd.7) 243 : cluster [DBG] 219.12 deep-scrub starts 2024-02-23T00:49:59.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:58.148109+0000 osd.7 (osd.7) 244 : cluster [DBG] 219.12 deep-scrub ok 2024-02-23T00:49:59.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:58.257884+0000 mon.a (mon.0) 2732 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:49:59.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:49:58.266092+0000 mon.a (mon.0) 2733 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:49:59.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:49:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:58.266205+0000 mon.a (mon.0) 2734 : cluster [DBG] osdmap e480: 8 total, 8 up, 7 in 2024-02-23T00:49:59.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:58.144503+0000 osd.7 (osd.7) 243 : cluster [DBG] 219.12 deep-scrub starts 2024-02-23T00:49:59.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:58.148109+0000 osd.7 (osd.7) 244 : cluster [DBG] 219.12 deep-scrub ok 2024-02-23T00:49:59.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:58.257884+0000 mon.a (mon.0) 2732 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:49:59.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:49:58.266092+0000 mon.a (mon.0) 2733 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:49:59.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:49:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:58.266205+0000 mon.a (mon.0) 2734 : cluster [DBG] osdmap e480: 8 total, 8 up, 7 in 2024-02-23T00:49:59.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:58.144503+0000 osd.7 (osd.7) 243 : cluster [DBG] 219.12 deep-scrub starts 2024-02-23T00:49:59.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:58.148109+0000 osd.7 (osd.7) 244 : cluster [DBG] 219.12 deep-scrub ok 2024-02-23T00:49:59.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:58.257884+0000 mon.a (mon.0) 2732 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:49:59.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:49:58.266092+0000 mon.a (mon.0) 2733 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:49:59.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:49:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:58.266205+0000 mon.a (mon.0) 2734 : cluster [DBG] osdmap e480: 8 total, 8 up, 7 in 2024-02-23T00:50:00.229 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:50:00.229 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 20, 2024-02-23T00:50:00.229 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 600, 2024-02-23T00:50:00.229 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [ 2024-02-23T00:50:00.229 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.229 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(client.24775.0:603 20.1e e110/109 20:7c2d116d:test-rados-api-smithi047-82457-15::foo:head v 110'281, mlcod=110'279)", 2024-02-23T00:50:00.230 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:42:59.740311+0000", 2024-02-23T00:50:00.230 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 420.48464144899998, 2024-02-23T00:50:00.230 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.085153354000000001, 2024-02-23T00:50:00.230 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:50:00.230 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:50:00.230 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:50:00.230 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.230 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:50:00.230 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.740311+0000", 2024-02-23T00:50:00.230 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:50:00.230 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.230 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.230 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:50:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.740311+0000", 2024-02-23T00:50:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967296 2024-02-23T00:50:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:50:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.740311+0000", 2024-02-23T00:50:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3036e-05 2024-02-23T00:50:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:50:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.740324+0000", 2024-02-23T00:50:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.9549999999999997e-06 2024-02-23T00:50:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:50:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.740330+0000", 2024-02-23T00:50:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.2349999999999999e-06 2024-02-23T00:50:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:50:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.740335+0000", 2024-02-23T00:50:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.079788364000000001 2024-02-23T00:50:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:50:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.820123+0000", 2024-02-23T00:50:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0228e-05 2024-02-23T00:50:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:50:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.820143+0000", 2024-02-23T00:50:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0052875209999999999 2024-02-23T00:50:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:50:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.825431+0000", 2024-02-23T00:50:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.3232000000000003e-05 2024-02-23T00:50:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:50:00.234 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:42:59.825464+0000", 2024-02-23T00:50:00.234 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.085153354000000001 2024-02-23T00:50:00.234 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.234 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:50:00.234 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.234 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.234 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.234 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(client.24775.0:2069 20.13 e112/109 20:cde74feb:test-rados-api-smithi047-82457-17::big:head v 112'1, mlcod=0'0)", 2024-02-23T00:50:00.234 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:01.712327+0000", 2024-02-23T00:50:00.234 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 418.51262547900001, 2024-02-23T00:50:00.234 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.079036414999999999, 2024-02-23T00:50:00.234 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:50:00.234 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:50:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:50:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:50:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.712327+0000", 2024-02-23T00:50:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:50:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:50:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.712327+0000", 2024-02-23T00:50:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999986 2024-02-23T00:50:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:50:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.712325+0000", 2024-02-23T00:50:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.056795063999999999 2024-02-23T00:50:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:50:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.769120+0000", 2024-02-23T00:50:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.6740000000000002e-05 2024-02-23T00:50:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:50:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.769177+0000", 2024-02-23T00:50:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8029e-05 2024-02-23T00:50:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.237 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.237 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:50:00.237 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.769195+0000", 2024-02-23T00:50:00.237 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.7191000000000001e-05 2024-02-23T00:50:00.237 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.237 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.237 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:50:00.237 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.769262+0000", 2024-02-23T00:50:00.237 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.9514e-05 2024-02-23T00:50:00.238 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.238 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.238 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:50:00.238 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.769282+0000", 2024-02-23T00:50:00.238 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.022039333000000001 2024-02-23T00:50:00.238 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.238 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.238 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:50:00.238 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.791321+0000", 2024-02-23T00:50:00.238 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.2144999999999997e-05 2024-02-23T00:50:00.238 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.238 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:50:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.791363+0000", 2024-02-23T00:50:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.079036414999999999 2024-02-23T00:50:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:50:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(client.24775.0:2074 20.1a e112/109 20:5de0a470:test-rados-api-smithi047-82457-17::big3:head v 112'3, mlcod=112'1)", 2024-02-23T00:50:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:01.957298+0000", 2024-02-23T00:50:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 418.26765411500003, 2024-02-23T00:50:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.088898480000000002, 2024-02-23T00:50:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:50:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:50:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:50:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:50:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.957298+0000", 2024-02-23T00:50:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:50:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:50:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.957298+0000", 2024-02-23T00:50:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999976 2024-02-23T00:50:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:50:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.957296+0000", 2024-02-23T00:50:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.062746578999999997 2024-02-23T00:50:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:50:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:02.020043+0000", 2024-02-23T00:50:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4688000000000001e-05 2024-02-23T00:50:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:50:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:02.020057+0000", 2024-02-23T00:50:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.348e-06 2024-02-23T00:50:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:50:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:02.020064+0000", 2024-02-23T00:50:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.5720000000000003e-05 2024-02-23T00:50:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:50:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:02.020109+0000", 2024-02-23T00:50:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.2218000000000002e-05 2024-02-23T00:50:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:50:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:02.020141+0000", 2024-02-23T00:50:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.025988054999999999 2024-02-23T00:50:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:50:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:02.046130+0000", 2024-02-23T00:50:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.7034999999999998e-05 2024-02-23T00:50:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:50:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:02.046197+0000", 2024-02-23T00:50:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.088898480000000002 2024-02-23T00:50:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:50:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24992.0:2 37.15 37:ad551702:::test:head [create] snapc 0=[] ondisk+write+known_if_redirected e115)", 2024-02-23T00:50:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:04.221218+0000", 2024-02-23T00:50:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 416.00373442900002, 2024-02-23T00:50:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.13418903200000001, 2024-02-23T00:50:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:50:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:50:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:50:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24992", 2024-02-23T00:50:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/550771032", 2024-02-23T00:50:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2 2024-02-23T00:50:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:50:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:50:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.221218+0000", 2024-02-23T00:50:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:50:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:50:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.221218+0000", 2024-02-23T00:50:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:50:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:50:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.221216+0000", 2024-02-23T00:50:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1384e-05 2024-02-23T00:50:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.246 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.246 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:50:00.246 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.221227+0000", 2024-02-23T00:50:00.246 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2353e-05 2024-02-23T00:50:00.246 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.246 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.246 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:50:00.246 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.221240+0000", 2024-02-23T00:50:00.246 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.7479999999999993e-06 2024-02-23T00:50:00.246 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.246 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.246 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:50:00.246 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.221247+0000", 2024-02-23T00:50:00.246 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.015031549999999999 2024-02-23T00:50:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:50:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.236279+0000", 2024-02-23T00:50:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8679999999999999e-06 2024-02-23T00:50:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:50:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.236287+0000", 2024-02-23T00:50:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.11065734100000001 2024-02-23T00:50:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:50:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.346944+0000", 2024-02-23T00:50:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000113587 2024-02-23T00:50:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:50:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.347058+0000", 2024-02-23T00:50:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00013868100000000001 2024-02-23T00:50:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 2,4", 2024-02-23T00:50:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.347196+0000", 2024-02-23T00:50:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00086159699999999997 2024-02-23T00:50:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:50:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.348058+0000", 2024-02-23T00:50:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.003862533 2024-02-23T00:50:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:50:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.351921+0000", 2024-02-23T00:50:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0034370820000000002 2024-02-23T00:50:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:50:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.355358+0000", 2024-02-23T00:50:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.6704999999999999e-05 2024-02-23T00:50:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:50:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.355384+0000", 2024-02-23T00:50:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.2524e-05 2024-02-23T00:50:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:50:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.355407+0000", 2024-02-23T00:50:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.13418903200000001 2024-02-23T00:50:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:50:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.32284.0:2 78.6 78:602f83fe:::foo:head [append 0~128] snapc 0=[] ondisk+write+known_if_redirected e139)", 2024-02-23T00:50:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:31.219941+0000", 2024-02-23T00:50:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 389.00501077799998, 2024-02-23T00:50:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.028760101, 2024-02-23T00:50:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:50:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:50:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:50:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.32284", 2024-02-23T00:50:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1813864525", 2024-02-23T00:50:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2 2024-02-23T00:50:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:50:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:50:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.219941+0000", 2024-02-23T00:50:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:50:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:50:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.219941+0000", 2024-02-23T00:50:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:50:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:50:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.219939+0000", 2024-02-23T00:50:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.498e-06 2024-02-23T00:50:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:50:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.219947+0000", 2024-02-23T00:50:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7360000000000001e-05 2024-02-23T00:50:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:50:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.219984+0000", 2024-02-23T00:50:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1497e-05 2024-02-23T00:50:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:50:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.219996+0000", 2024-02-23T00:50:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00019599900000000001 2024-02-23T00:50:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:50:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.220192+0000", 2024-02-23T00:50:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.977e-06 2024-02-23T00:50:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:50:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.220200+0000", 2024-02-23T00:50:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.020613092999999999 2024-02-23T00:50:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:50:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.240813+0000", 2024-02-23T00:50:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012351699999999999 2024-02-23T00:50:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:50:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.240936+0000", 2024-02-23T00:50:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000158527 2024-02-23T00:50:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 3,5", 2024-02-23T00:50:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.241095+0000", 2024-02-23T00:50:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0038824100000000002 2024-02-23T00:50:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:50:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.244977+0000", 2024-02-23T00:50:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001243447 2024-02-23T00:50:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:50:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.246221+0000", 2024-02-23T00:50:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0023899310000000001 2024-02-23T00:50:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:50:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.248611+0000", 2024-02-23T00:50:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.4546999999999998e-05 2024-02-23T00:50:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:50:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.248665+0000", 2024-02-23T00:50:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.6310000000000003e-05 2024-02-23T00:50:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:50:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.248702+0000", 2024-02-23T00:50:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.028760101 2024-02-23T00:50:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:50:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(osd.3.0:1389 20.0 e139/109 -22:0000ebd9:::temp_20.0_0_14313_1:head v 0'0, mlcod=0'0)", 2024-02-23T00:50:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:32.131906+0000", 2024-02-23T00:50:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 388.093046464, 2024-02-23T00:50:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.025453091000000001, 2024-02-23T00:50:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:50:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:50:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:50:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:50:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.131906+0000", 2024-02-23T00:50:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:50:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:50:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.131906+0000", 2024-02-23T00:50:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999971 2024-02-23T00:50:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:50:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.131903+0000", 2024-02-23T00:50:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0055423579999999998 2024-02-23T00:50:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:50:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.137445+0000", 2024-02-23T00:50:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.7262000000000001e-05 2024-02-23T00:50:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:50:00.260 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.137473+0000", 2024-02-23T00:50:00.260 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0485e-05 2024-02-23T00:50:00.260 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.260 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.260 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:50:00.260 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.137483+0000", 2024-02-23T00:50:00.260 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00060361099999999999 2024-02-23T00:50:00.260 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.260 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.260 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:50:00.260 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.138087+0000", 2024-02-23T00:50:00.261 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.5690999999999998e-05 2024-02-23T00:50:00.261 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.261 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.261 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:50:00.261 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.138112+0000", 2024-02-23T00:50:00.261 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.019190893000000001 2024-02-23T00:50:00.261 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.261 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.261 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:50:00.261 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.157303+0000", 2024-02-23T00:50:00.262 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.5458000000000003e-05 2024-02-23T00:50:00.262 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.262 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.262 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:50:00.262 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.157359+0000", 2024-02-23T00:50:00.262 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.025453091000000001 2024-02-23T00:50:00.262 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.262 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:50:00.262 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.262 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.262 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.262 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(osd.3.0:1390 20.0 e139/109 -22:0000ebd9:::temp_20.0_0_14313_1:head v 0'0, mlcod=0'0)", 2024-02-23T00:50:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:32.180625+0000", 2024-02-23T00:50:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 388.04432735699999, 2024-02-23T00:50:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.066413153000000003, 2024-02-23T00:50:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:50:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:50:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:50:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:50:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.180625+0000", 2024-02-23T00:50:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:50:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:50:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.180625+0000", 2024-02-23T00:50:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999971 2024-02-23T00:50:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:50:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.180622+0000", 2024-02-23T00:50:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018534416000000001 2024-02-23T00:50:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:50:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.199157+0000", 2024-02-23T00:50:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0648999999999997e-05 2024-02-23T00:50:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.265 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.265 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:50:00.265 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.199187+0000", 2024-02-23T00:50:00.265 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.5439999999999999e-06 2024-02-23T00:50:00.265 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.265 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.265 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:50:00.265 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.199196+0000", 2024-02-23T00:50:00.265 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.5427999999999996e-05 2024-02-23T00:50:00.265 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.265 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.265 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:50:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.199281+0000", 2024-02-23T00:50:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.2827e-05 2024-02-23T00:50:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:50:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.199304+0000", 2024-02-23T00:50:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.047691698999999997 2024-02-23T00:50:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:50:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.246996+0000", 2024-02-23T00:50:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.2293999999999999e-05 2024-02-23T00:50:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:50:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.247038+0000", 2024-02-23T00:50:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.066413153000000003 2024-02-23T00:50:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:50:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.15270.0:181 75.0 75:00000000::::head [pgnls start_epoch 142] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e142)", 2024-02-23T00:50:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:34.500215+0000", 2024-02-23T00:50:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 385.724736952, 2024-02-23T00:50:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.026354064999999999, 2024-02-23T00:50:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:50:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:50:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:50:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.15270", 2024-02-23T00:50:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/4133111990", 2024-02-23T00:50:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 181 2024-02-23T00:50:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:50:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:50:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.500215+0000", 2024-02-23T00:50:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:50:00.269 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.269 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.269 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:50:00.269 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.500215+0000", 2024-02-23T00:50:00.269 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999971 2024-02-23T00:50:00.269 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.269 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.269 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:50:00.269 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.500212+0000", 2024-02-23T00:50:00.269 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.6798e-05 2024-02-23T00:50:00.270 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.270 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.270 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:50:00.270 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.500239+0000", 2024-02-23T00:50:00.270 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.9426e-05 2024-02-23T00:50:00.270 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.270 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.270 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:50:00.270 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.500259+0000", 2024-02-23T00:50:00.270 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.7950000000000002e-06 2024-02-23T00:50:00.270 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.270 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.270 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:50:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.500269+0000", 2024-02-23T00:50:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0034826060000000001 2024-02-23T00:50:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:50:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.503751+0000", 2024-02-23T00:50:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.1189999999999997e-06 2024-02-23T00:50:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-02-23T00:50:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.503759+0000", 2024-02-23T00:50:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.022616594 2024-02-23T00:50:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.272 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:50:00.272 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.526376+0000", 2024-02-23T00:50:00.272 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.6050000000000002e-05 2024-02-23T00:50:00.272 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.272 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.272 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:50:00.272 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.526412+0000", 2024-02-23T00:50:00.272 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000157455 2024-02-23T00:50:00.272 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.272 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:50:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.526569+0000", 2024-02-23T00:50:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.026354064999999999 2024-02-23T00:50:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:50:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.34306.0:2 82.3 82:c656db43:::oid_RoundTripPP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e142)", 2024-02-23T00:50:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:34.513354+0000", 2024-02-23T00:50:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 385.71159867699998, 2024-02-23T00:50:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.056878060000000001, 2024-02-23T00:50:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:50:00.274 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:50:00.274 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:50:00.274 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.34306", 2024-02-23T00:50:00.274 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1503027630", 2024-02-23T00:50:00.274 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2 2024-02-23T00:50:00.274 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.274 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:50:00.274 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.274 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:50:00.274 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.513354+0000", 2024-02-23T00:50:00.274 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:50:00.274 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.274 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:50:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.513354+0000", 2024-02-23T00:50:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:50:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:50:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.513352+0000", 2024-02-23T00:50:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.2949999999999996e-06 2024-02-23T00:50:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:50:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.513359+0000", 2024-02-23T00:50:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1875999999999999e-05 2024-02-23T00:50:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:50:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.513371+0000", 2024-02-23T00:50:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.4500000000000004e-06 2024-02-23T00:50:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:50:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.513379+0000", 2024-02-23T00:50:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.6609e-05 2024-02-23T00:50:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.277 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:50:00.277 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.513436+0000", 2024-02-23T00:50:00.277 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.9800000000000001e-06 2024-02-23T00:50:00.277 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.277 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.277 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-02-23T00:50:00.277 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.513443+0000", 2024-02-23T00:50:00.277 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0034703429999999999 2024-02-23T00:50:00.277 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.277 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.277 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:50:00.277 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.516913+0000", 2024-02-23T00:50:00.277 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.4170000000000003e-06 2024-02-23T00:50:00.277 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:50:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.516921+0000", 2024-02-23T00:50:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.045082443 2024-02-23T00:50:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:50:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.562003+0000", 2024-02-23T00:50:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000117848 2024-02-23T00:50:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:50:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.562121+0000", 2024-02-23T00:50:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00017905500000000001 2024-02-23T00:50:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 2,3", 2024-02-23T00:50:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.562300+0000", 2024-02-23T00:50:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00056806599999999999 2024-02-23T00:50:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:50:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.562868+0000", 2024-02-23T00:50:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0069143620000000003 2024-02-23T00:50:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:50:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.569782+0000", 2024-02-23T00:50:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00034649600000000001 2024-02-23T00:50:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:50:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.570129+0000", 2024-02-23T00:50:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.9905000000000003e-05 2024-02-23T00:50:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:50:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.570189+0000", 2024-02-23T00:50:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.2885e-05 2024-02-23T00:50:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:50:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.570232+0000", 2024-02-23T00:50:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.056878060000000001 2024-02-23T00:50:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:50:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(client.40517.0:2 115.1b e182/180 115:d8841f39:test-rados-api-smithi047-82457-26::foo:head v 182'1, mlcod=0'0)", 2024-02-23T00:50:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:14.496478+0000", 2024-02-23T00:50:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 345.72847462700003, 2024-02-23T00:50:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021581046999999999, 2024-02-23T00:50:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:50:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:50:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:50:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:50:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.496478+0000", 2024-02-23T00:50:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:50:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:50:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.496478+0000", 2024-02-23T00:50:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999971 2024-02-23T00:50:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:50:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.496475+0000", 2024-02-23T00:50:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0065093110000000003 2024-02-23T00:50:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:50:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.502984+0000", 2024-02-23T00:50:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.6535999999999997e-05 2024-02-23T00:50:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:50:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.503021+0000", 2024-02-23T00:50:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.129e-05 2024-02-23T00:50:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:50:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.503032+0000", 2024-02-23T00:50:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.5362000000000002e-05 2024-02-23T00:50:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:50:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.503097+0000", 2024-02-23T00:50:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.6917e-05 2024-02-23T00:50:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:50:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.503124+0000", 2024-02-23T00:50:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.014876454000000001 2024-02-23T00:50:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.285 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:50:00.285 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.518001+0000", 2024-02-23T00:50:00.285 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.7868999999999998e-05 2024-02-23T00:50:00.285 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.285 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.285 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:50:00.285 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.518059+0000", 2024-02-23T00:50:00.285 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021581046999999999 2024-02-23T00:50:00.285 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.285 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:50:00.285 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.285 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.40517.0:4 113.0s0 113:14a7a751:test-rados-api-smithi047-82457-26::foo.copy:head [copy-from ver 2] snapc 0=[] ondisk+write+known_if_redirected e182)", 2024-02-23T00:50:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:14.520153+0000", 2024-02-23T00:50:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 345.70479907100002, 2024-02-23T00:50:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.020744196999999999, 2024-02-23T00:50:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:50:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:50:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:50:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.40517", 2024-02-23T00:50:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/3108721855", 2024-02-23T00:50:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 4 2024-02-23T00:50:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:50:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:50:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.520153+0000", 2024-02-23T00:50:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:50:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:50:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.520153+0000", 2024-02-23T00:50:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999986 2024-02-23T00:50:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:50:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.520152+0000", 2024-02-23T00:50:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.614e-06 2024-02-23T00:50:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.288 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.288 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:50:00.288 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.520157+0000", 2024-02-23T00:50:00.288 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0224000000000002e-05 2024-02-23T00:50:00.288 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.288 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.288 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:50:00.288 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.520178+0000", 2024-02-23T00:50:00.288 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.8919999999999999e-06 2024-02-23T00:50:00.288 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.288 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.288 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:50:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.520183+0000", 2024-02-23T00:50:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.6939e-05 2024-02-23T00:50:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:50:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.520220+0000", 2024-02-23T00:50:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.5514999999999998e-05 2024-02-23T00:50:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:50:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.520295+0000", 2024-02-23T00:50:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.020602321 2024-02-23T00:50:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:50:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.540897+0000", 2024-02-23T00:50:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.020744196999999999 2024-02-23T00:50:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:50:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.40553.0:2 117.1a 117:5a7a12bd:::oid_SimpleStatPPNS_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e183)", 2024-02-23T00:50:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:15.129646+0000", 2024-02-23T00:50:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 345.09530583600002, 2024-02-23T00:50:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029333562000000001, 2024-02-23T00:50:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:50:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:50:00.291 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:50:00.291 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.40553", 2024-02-23T00:50:00.291 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1915740554", 2024-02-23T00:50:00.291 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2 2024-02-23T00:50:00.291 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.291 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:50:00.291 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.291 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:50:00.291 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:15.129646+0000", 2024-02-23T00:50:00.291 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:50:00.291 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.291 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.291 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:50:00.291 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:15.129646+0000", 2024-02-23T00:50:00.292 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:50:00.292 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.292 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.292 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:50:00.292 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:15.129645+0000", 2024-02-23T00:50:00.292 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.4470000000000006e-06 2024-02-23T00:50:00.292 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.292 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.292 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:50:00.292 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:15.129653+0000", 2024-02-23T00:50:00.292 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0852999999999999e-05 2024-02-23T00:50:00.292 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.292 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.292 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:50:00.293 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:15.129664+0000", 2024-02-23T00:50:00.293 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.4919999999999996e-06 2024-02-23T00:50:00.293 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.293 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.293 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:50:00.293 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:15.129671+0000", 2024-02-23T00:50:00.293 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.7908000000000002e-05 2024-02-23T00:50:00.293 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.293 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.293 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:50:00.293 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:15.129729+0000", 2024-02-23T00:50:00.293 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.6749999999999997e-06 2024-02-23T00:50:00.293 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.293 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.294 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:50:00.294 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:15.129739+0000", 2024-02-23T00:50:00.294 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.015631262 2024-02-23T00:50:00.294 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.294 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.294 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:50:00.294 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:15.145370+0000", 2024-02-23T00:50:00.294 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000156599 2024-02-23T00:50:00.294 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.294 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.294 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:50:00.294 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:15.145527+0000", 2024-02-23T00:50:00.294 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00017407699999999999 2024-02-23T00:50:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 5,7", 2024-02-23T00:50:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:15.145701+0000", 2024-02-23T00:50:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0025710630000000002 2024-02-23T00:50:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:50:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:15.148272+0000", 2024-02-23T00:50:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0045611039999999999 2024-02-23T00:50:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:50:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:15.152833+0000", 2024-02-23T00:50:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0060428050000000001 2024-02-23T00:50:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:50:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:15.158876+0000", 2024-02-23T00:50:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.2118000000000004e-05 2024-02-23T00:50:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:50:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:15.158938+0000", 2024-02-23T00:50:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.1950000000000003e-05 2024-02-23T00:50:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:50:00.297 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:15.158980+0000", 2024-02-23T00:50:00.297 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029333562000000001 2024-02-23T00:50:00.297 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.297 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:50:00.297 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.297 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.297 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.297 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.40529.0:9 114.5s0 114:b5491451:test-rados-api-smithi047-82496-18::foo:head [writefull 0~128] snapc 0=[] ondisk+write+known_if_redirected e186)", 2024-02-23T00:50:00.297 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:18.114404+0000", 2024-02-23T00:50:00.297 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 342.11054773299998, 2024-02-23T00:50:00.297 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.024854564999999999, 2024-02-23T00:50:00.297 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:50:00.297 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:50:00.297 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:50:00.297 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.40529", 2024-02-23T00:50:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/3414618047", 2024-02-23T00:50:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 9 2024-02-23T00:50:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:50:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:50:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:18.114404+0000", 2024-02-23T00:50:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:50:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:50:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:18.114404+0000", 2024-02-23T00:50:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999986 2024-02-23T00:50:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:50:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:18.114403+0000", 2024-02-23T00:50:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.5510000000000002e-06 2024-02-23T00:50:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:50:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:18.114411+0000", 2024-02-23T00:50:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2646999999999999e-05 2024-02-23T00:50:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:50:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:18.114423+0000", 2024-02-23T00:50:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.2630000000000003e-06 2024-02-23T00:50:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:50:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:18.114432+0000", 2024-02-23T00:50:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.9015000000000003e-05 2024-02-23T00:50:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:50:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:18.114491+0000", 2024-02-23T00:50:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.1619999999999997e-06 2024-02-23T00:50:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-02-23T00:50:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:18.114500+0000", 2024-02-23T00:50:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0058377619999999998 2024-02-23T00:50:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:50:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:18.120337+0000", 2024-02-23T00:50:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012195 2024-02-23T00:50:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:50:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:18.120459+0000", 2024-02-23T00:50:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00073385099999999999 2024-02-23T00:50:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-02-23T00:50:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:18.121193+0000", 2024-02-23T00:50:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000578678 2024-02-23T00:50:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.302 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.302 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-02-23T00:50:00.302 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:18.121772+0000", 2024-02-23T00:50:00.302 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.017397617000000001 2024-02-23T00:50:00.302 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.302 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.302 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:50:00.302 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:18.139170+0000", 2024-02-23T00:50:00.302 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.9493999999999994e-05 2024-02-23T00:50:00.302 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.302 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.302 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:50:00.302 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:18.139259+0000", 2024-02-23T00:50:00.302 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.024854564999999999 2024-02-23T00:50:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:50:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(osd.3.0:1452 62.c e199/126 62:303239da:test-rados-api-smithi047-82465-14::foo78:4 v 199'28, mlcod=195'24)", 2024-02-23T00:50:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:31.164164+0000", 2024-02-23T00:50:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 329.06078865400002, 2024-02-23T00:50:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023846743, 2024-02-23T00:50:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:50:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:50:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:50:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:50:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164164+0000", 2024-02-23T00:50:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:50:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:50:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164164+0000", 2024-02-23T00:50:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999943 2024-02-23T00:50:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:50:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164158+0000", 2024-02-23T00:50:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4290999999999999e-05 2024-02-23T00:50:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:50:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164172+0000", 2024-02-23T00:50:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0293e-05 2024-02-23T00:50:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:50:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164183+0000", 2024-02-23T00:50:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.575e-06 2024-02-23T00:50:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:50:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164188+0000", 2024-02-23T00:50:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.8688000000000001e-05 2024-02-23T00:50:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:50:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164287+0000", 2024-02-23T00:50:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.9468999999999999e-05 2024-02-23T00:50:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:50:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164306+0000", 2024-02-23T00:50:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023682110999999999 2024-02-23T00:50:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:50:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.187988+0000", 2024-02-23T00:50:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1946e-05 2024-02-23T00:50:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:50:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.188010+0000", 2024-02-23T00:50:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023846743 2024-02-23T00:50:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:50:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(osd.3.0:1453 62.c e199/126 62:37762822:test-rados-api-smithi047-82465-14::foo50:4 v 199'30, mlcod=195'24)", 2024-02-23T00:50:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:31.164203+0000", 2024-02-23T00:50:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 329.06074955499997, 2024-02-23T00:50:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023951410999999999, 2024-02-23T00:50:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:50:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:50:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:50:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:50:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164203+0000", 2024-02-23T00:50:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:50:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:50:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164203+0000", 2024-02-23T00:50:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:50:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:50:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164202+0000", 2024-02-23T00:50:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.4599999999999992e-06 2024-02-23T00:50:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:50:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164212+0000", 2024-02-23T00:50:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.5989999999999998e-06 2024-02-23T00:50:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:50:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164218+0000", 2024-02-23T00:50:00.310 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.1080000000000003e-06 2024-02-23T00:50:00.310 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.310 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.310 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:50:00.310 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164222+0000", 2024-02-23T00:50:00.310 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00036172199999999998 2024-02-23T00:50:00.310 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.310 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.310 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:50:00.310 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164584+0000", 2024-02-23T00:50:00.310 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.9641e-05 2024-02-23T00:50:00.310 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.310 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:50:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164604+0000", 2024-02-23T00:50:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023511941000000001 2024-02-23T00:50:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:50:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.188116+0000", 2024-02-23T00:50:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.8358000000000001e-05 2024-02-23T00:50:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:50:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.188154+0000", 2024-02-23T00:50:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023951410999999999 2024-02-23T00:50:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:50:00.312 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.312 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.312 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.312 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop_reply(osd.0.0:331 62.e e199/126 ondisk, result = 0)", 2024-02-23T00:50:00.312 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:31.169340+0000", 2024-02-23T00:50:00.312 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 329.05561225500003, 2024-02-23T00:50:00.312 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018945747999999998, 2024-02-23T00:50:00.312 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:50:00.312 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:50:00.312 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:50:00.312 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.312 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:50:00.312 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.169340+0000", 2024-02-23T00:50:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:50:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:50:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.169340+0000", 2024-02-23T00:50:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999986 2024-02-23T00:50:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:50:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.169339+0000", 2024-02-23T00:50:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.4970000000000001e-06 2024-02-23T00:50:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:50:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.169347+0000", 2024-02-23T00:50:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.2152999999999999e-05 2024-02-23T00:50:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:50:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.169369+0000", 2024-02-23T00:50:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0320000000000003e-06 2024-02-23T00:50:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:50:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.169377+0000", 2024-02-23T00:50:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018879638000000001 2024-02-23T00:50:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:50:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.188257+0000", 2024-02-23T00:50:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.3629999999999994e-06 2024-02-23T00:50:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:50:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.188265+0000", 2024-02-23T00:50:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0305999999999998e-05 2024-02-23T00:50:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:50:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.188286+0000", 2024-02-23T00:50:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018945747999999998 2024-02-23T00:50:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:50:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWrite(129.7s2 199/195 ECSubWrite(tid=1454, reqid=client.42730.0:8, at_version=199'4, trim_to=0'0, roll_forward_to=197'3))", 2024-02-23T00:50:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:31.170261+0000", 2024-02-23T00:50:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 329.054691306, 2024-02-23T00:50:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.017706718, 2024-02-23T00:50:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:50:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-02-23T00:50:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:50:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:50:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.170261+0000", 2024-02-23T00:50:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:50:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:50:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.170261+0000", 2024-02-23T00:50:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.999999 2024-02-23T00:50:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:50:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.170260+0000", 2024-02-23T00:50:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.4536e-05 2024-02-23T00:50:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:50:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.170284+0000", 2024-02-23T00:50:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.9298999999999999e-05 2024-02-23T00:50:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:50:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.170304+0000", 2024-02-23T00:50:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0399999999999993e-06 2024-02-23T00:50:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:50:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.170312+0000", 2024-02-23T00:50:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001240533 2024-02-23T00:50:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:50:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.171552+0000", 2024-02-23T00:50:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.201e-05 2024-02-23T00:50:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-02-23T00:50:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.171564+0000", 2024-02-23T00:50:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.016382556999999999 2024-02-23T00:50:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-02-23T00:50:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.187947+0000", 2024-02-23T00:50:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0727999999999999e-05 2024-02-23T00:50:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:50:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.187968+0000", 2024-02-23T00:50:00.320 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.017706718 2024-02-23T00:50:00.320 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.320 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:50:00.320 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.320 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.320 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.320 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop_reply(osd.0.0:331 62.e e199/126 ondisk, result = 0)", 2024-02-23T00:50:00.320 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:31.170335+0000", 2024-02-23T00:50:00.320 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 329.054617178, 2024-02-23T00:50:00.320 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018153419, 2024-02-23T00:50:00.320 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:50:00.320 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:50:00.320 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:50:00.320 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:50:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.170335+0000", 2024-02-23T00:50:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:50:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:50:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.170335+0000", 2024-02-23T00:50:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:50:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:50:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.170335+0000", 2024-02-23T00:50:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.856e-06 2024-02-23T00:50:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.322 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.322 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:50:00.322 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.170338+0000", 2024-02-23T00:50:00.322 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.5179999999999998e-06 2024-02-23T00:50:00.322 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.322 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.322 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:50:00.322 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.170346+0000", 2024-02-23T00:50:00.322 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.9649999999999999e-06 2024-02-23T00:50:00.322 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.322 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.322 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:50:00.322 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.170351+0000", 2024-02-23T00:50:00.323 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018060857 2024-02-23T00:50:00.323 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.323 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.323 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:50:00.323 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.188412+0000", 2024-02-23T00:50:00.323 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.7190000000000001e-06 2024-02-23T00:50:00.323 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.323 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.323 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:50:00.323 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.188419+0000", 2024-02-23T00:50:00.323 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.9029000000000006e-05 2024-02-23T00:50:00.323 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.324 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.324 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:50:00.324 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.188488+0000", 2024-02-23T00:50:00.324 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018153419 2024-02-23T00:50:00.324 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.324 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:50:00.324 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.324 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.324 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.324 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.41087.0:1 189.2s0 189:5455dc94:::oid_IsSafePP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e318)", 2024-02-23T00:50:00.324 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:46:30.187851+0000", 2024-02-23T00:50:00.324 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 210.037101052, 2024-02-23T00:50:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.57313715, 2024-02-23T00:50:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:50:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:50:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:50:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.41087", 2024-02-23T00:50:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/2888114840", 2024-02-23T00:50:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 1 2024-02-23T00:50:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:50:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:50:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:30.187851+0000", 2024-02-23T00:50:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:50:00.326 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.326 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.326 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:50:00.326 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:30.187851+0000", 2024-02-23T00:50:00.326 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999976 2024-02-23T00:50:00.326 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.326 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.326 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:50:00.326 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:30.187849+0000", 2024-02-23T00:50:00.326 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.1200000000000008e-06 2024-02-23T00:50:00.326 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.327 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.327 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:50:00.327 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:30.187858+0000", 2024-02-23T00:50:00.327 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5472e-05 2024-02-23T00:50:00.327 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.327 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.327 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:50:00.327 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:30.187874+0000", 2024-02-23T00:50:00.327 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.5690000000000005e-06 2024-02-23T00:50:00.327 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.327 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.327 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:50:00.328 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:30.187882+0000", 2024-02-23T00:50:00.328 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.7757999999999999e-05 2024-02-23T00:50:00.328 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.328 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.328 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:50:00.328 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:30.187930+0000", 2024-02-23T00:50:00.328 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.269e-06 2024-02-23T00:50:00.328 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.328 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.328 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:50:00.329 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:30.187933+0000", 2024-02-23T00:50:00.329 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5695471319999998 2024-02-23T00:50:00.329 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.329 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.329 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:50:00.329 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:31.757480+0000", 2024-02-23T00:50:00.329 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00013165799999999999 2024-02-23T00:50:00.329 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.329 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.329 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:50:00.329 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:31.757612+0000", 2024-02-23T00:50:00.329 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00041009 2024-02-23T00:50:00.330 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.330 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.330 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-02-23T00:50:00.330 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:31.758022+0000", 2024-02-23T00:50:00.330 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00061976200000000005 2024-02-23T00:50:00.330 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.330 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.330 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-02-23T00:50:00.330 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:31.758642+0000", 2024-02-23T00:50:00.330 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0023075489999999999 2024-02-23T00:50:00.330 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.330 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:50:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:31.760949+0000", 2024-02-23T00:50:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.8970000000000001e-05 2024-02-23T00:50:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:50:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:31.760988+0000", 2024-02-23T00:50:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.57313715 2024-02-23T00:50:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:50:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.41243.0:1 203.4s0 203:311cbe94:::oid_ExecuteClassPP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e377)", 2024-02-23T00:50:00.332 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:47:29.186078+0000", 2024-02-23T00:50:00.332 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 151.03887424999999, 2024-02-23T00:50:00.332 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.004199842, 2024-02-23T00:50:00.332 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:50:00.332 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:50:00.332 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:50:00.332 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.41243", 2024-02-23T00:50:00.332 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/3348293548", 2024-02-23T00:50:00.332 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 1 2024-02-23T00:50:00.332 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.332 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:50:00.333 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.333 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:50:00.333 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:29.186078+0000", 2024-02-23T00:50:00.333 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:50:00.333 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.333 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.333 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:50:00.333 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:29.186078+0000", 2024-02-23T00:50:00.333 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:50:00.333 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.333 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.334 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:50:00.334 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:29.186076+0000", 2024-02-23T00:50:00.334 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.2549999999999997e-06 2024-02-23T00:50:00.334 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.334 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.334 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:50:00.334 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:29.186084+0000", 2024-02-23T00:50:00.334 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2850000000000001e-05 2024-02-23T00:50:00.334 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.334 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.334 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:50:00.335 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:29.186097+0000", 2024-02-23T00:50:00.335 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.2300000000000008e-06 2024-02-23T00:50:00.335 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.335 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.335 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:50:00.335 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:29.186105+0000", 2024-02-23T00:50:00.335 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0018958600000000001 2024-02-23T00:50:00.335 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.335 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.335 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:50:00.335 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:29.188001+0000", 2024-02-23T00:50:00.335 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.1489999999999998e-06 2024-02-23T00:50:00.336 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.336 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.336 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:50:00.336 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:29.188004+0000", 2024-02-23T00:50:00.336 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.000917512 2024-02-23T00:50:00.336 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.336 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.336 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:50:00.336 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:30.188922+0000", 2024-02-23T00:50:00.336 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00013111000000000001 2024-02-23T00:50:00.336 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.337 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.337 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:50:00.337 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:30.189053+0000", 2024-02-23T00:50:00.337 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00040387600000000001 2024-02-23T00:50:00.337 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.337 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.337 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-02-23T00:50:00.337 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:30.189457+0000", 2024-02-23T00:50:00.337 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00043691699999999999 2024-02-23T00:50:00.337 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.337 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.337 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-02-23T00:50:00.337 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:30.189894+0000", 2024-02-23T00:50:00.338 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00031495399999999999 2024-02-23T00:50:00.338 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.338 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.338 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:50:00.338 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:30.190209+0000", 2024-02-23T00:50:00.338 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.9130999999999999e-05 2024-02-23T00:50:00.338 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:50:00.338 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:50:00.338 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:50:00.338 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:30.190278+0000", 2024-02-23T00:50:00.338 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.004199842 2024-02-23T00:50:00.338 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.338 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:50:00.338 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.338 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:50:00.339 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:50:00.339 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:50:00.339 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:00.340 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:50:00.340 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:50:00.441 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:50:00.473 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:00.497 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:50:00.499 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:00.500 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:50:00.557 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:50:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:50:00] "GET /metrics HTTP/1.1" 200 189428 "" "Prometheus/2.33.4" 2024-02-23T00:50:00.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:49:59.320234+0000 mgr.y (mgr.14517) 452 : cluster [DBG] pgmap v719: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1006 B/s rd, 1006 B/s wr, 2 op/s 2024-02-23T00:50:00.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:00.000228+0000 mon.a (mon.0) 2735 : cluster [WRN] Health detail: HEALTH_WARN noscrub,nodeep-scrub flag(s) set; 1 pool(s) do not have an application enabled 2024-02-23T00:50:00.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:00.000309+0000 mon.a (mon.0) 2736 : cluster [WRN] [WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set 2024-02-23T00:50:00.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:00.000356+0000 mon.a (mon.0) 2737 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-23T00:50:00.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:00.000407+0000 mon.a (mon.0) 2738 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-23T00:50:00.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:00.000423+0000 mon.a (mon.0) 2739 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-23T00:50:00.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:00.213703+0000 mon.a (mon.0) 2740 : cluster [DBG] osdmap e481: 8 total, 8 up, 7 in 2024-02-23T00:50:00.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:49:59.320234+0000 mgr.y (mgr.14517) 452 : cluster [DBG] pgmap v719: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1006 B/s rd, 1006 B/s wr, 2 op/s 2024-02-23T00:50:00.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:00.000228+0000 mon.a (mon.0) 2735 : cluster [WRN] Health detail: HEALTH_WARN noscrub,nodeep-scrub flag(s) set; 1 pool(s) do not have an application enabled 2024-02-23T00:50:00.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:00.000309+0000 mon.a (mon.0) 2736 : cluster [WRN] [WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set 2024-02-23T00:50:00.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:00.000356+0000 mon.a (mon.0) 2737 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-23T00:50:00.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:00.000407+0000 mon.a (mon.0) 2738 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-23T00:50:00.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:00.000423+0000 mon.a (mon.0) 2739 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-23T00:50:00.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:00.213703+0000 mon.a (mon.0) 2740 : cluster [DBG] osdmap e481: 8 total, 8 up, 7 in 2024-02-23T00:50:00.601 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:50:00.628 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:49:59.320234+0000 mgr.y (mgr.14517) 452 : cluster [DBG] pgmap v719: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1006 B/s rd, 1006 B/s wr, 2 op/s 2024-02-23T00:50:00.628 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:00.000228+0000 mon.a (mon.0) 2735 : cluster [WRN] Health detail: HEALTH_WARN noscrub,nodeep-scrub flag(s) set; 1 pool(s) do not have an application enabled 2024-02-23T00:50:00.628 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:00.000309+0000 mon.a (mon.0) 2736 : cluster [WRN] [WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set 2024-02-23T00:50:00.628 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:00.000356+0000 mon.a (mon.0) 2737 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-23T00:50:00.628 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:00.000407+0000 mon.a (mon.0) 2738 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-23T00:50:00.628 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:00.000423+0000 mon.a (mon.0) 2739 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-23T00:50:00.628 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:00.213703+0000 mon.a (mon.0) 2740 : cluster [DBG] osdmap e481: 8 total, 8 up, 7 in 2024-02-23T00:50:00.630 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:00.650 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:50:00.652 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:00.653 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:50:00.754 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:50:00.783 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:00.803 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:50:00.806 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:00.806 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:50:00.907 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:50:00.942 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:01.496 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:50:01.500 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:01.500 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:50:01.527 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:50:01.601 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:50:01.634 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:01.656 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:50:01.658 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:01.659 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:50:01.760 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:50:01.795 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:01.817 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:50:01.825 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:01.825 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:50:01.926 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:50:01.956 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:02.405 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:50:02.408 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:02.409 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:50:02.509 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:50:02.539 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:02.557 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:50:02.560 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:02.561 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:50:02.662 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:50:02.679 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:50:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:50:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:50:02.679 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:01.320688+0000 mgr.y (mgr.14517) 453 : cluster [DBG] pgmap v721: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 853 B/s wr, 2 op/s 2024-02-23T00:50:02.689 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:01.320688+0000 mgr.y (mgr.14517) 453 : cluster [DBG] pgmap v721: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 853 B/s wr, 2 op/s 2024-02-23T00:50:02.689 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:01.320688+0000 mgr.y (mgr.14517) 453 : cluster [DBG] pgmap v721: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 853 B/s wr, 2 op/s 2024-02-23T00:50:02.691 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:02.820 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:50:02.941 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:50:02.943 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:50:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:50:02.944 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:02.944 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:50:02.947 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.1 dump_ops_in_flight 2024-02-23T00:50:03.045 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:50:03.076 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:03.098 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:50:03.101 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:03.102 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:50:03.203 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:50:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:03.514 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset noscrub 2024-02-23T00:50:03.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:02.677233+0000 mgr.y (mgr.14517) 454 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:50:03.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:02.677233+0000 mgr.y (mgr.14517) 454 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:50:03.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:02.677233+0000 mgr.y (mgr.14517) 454 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:50:03.781 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:50:03.784 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:03.784 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:50:03.885 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:50:03.918 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:03.938 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:50:03.942 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:03.942 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:50:04.042 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:50:04.044 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:50:04.076 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:04.098 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:50:04.100 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:04.100 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:50:04.201 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:50:04.231 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:04.251 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:50:04.253 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:04.254 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:50:04.355 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:50:04.388 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:04.407 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:50:04.409 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:04.410 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:50:04.511 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:50:04.540 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:03.321552+0000 mgr.y (mgr.14517) 455 : cluster [DBG] pgmap v722: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.4 KiB/s rd, 724 B/s wr, 2 op/s 2024-02-23T00:50:04.542 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:04.563 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:50:04.565 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:04.566 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:50:04.666 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:50:04.666 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:50:04.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.690+0000 7ff808223700 1 -- 172.21.15.47:0/4155481212 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff8000f9610 msgr2=0x7ff800101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:04.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.690+0000 7ff808223700 1 --2- 172.21.15.47:0/4155481212 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff8000f9610 0x7ff800101d20 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 rx=0x7ff7f0004870 tx=0x7ff7f0009cb0).stop 2024-02-23T00:50:04.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.690+0000 7ff808223700 1 -- 172.21.15.47:0/4155481212 shutdown_connections 2024-02-23T00:50:04.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.690+0000 7ff808223700 1 --2- 172.21.15.47:0/4155481212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff800102210 0x7ff800104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:04.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.690+0000 7ff808223700 1 --2- 172.21.15.47:0/4155481212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8000f8cb0 0x7ff8000f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:04.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.690+0000 7ff808223700 1 --2- 172.21.15.47:0/4155481212 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff8000f9610 0x7ff800101d20 unknown :-1 s=CLOSED pgs=2702 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:04.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.690+0000 7ff808223700 1 -- 172.21.15.47:0/4155481212 >> 172.21.15.47:0/4155481212 conn(0x7ff8000f2820 msgr2=0x7ff8000f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:04.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.690+0000 7ff808223700 1 -- 172.21.15.47:0/4155481212 shutdown_connections 2024-02-23T00:50:04.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.690+0000 7ff808223700 1 -- 172.21.15.47:0/4155481212 wait complete. 2024-02-23T00:50:04.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.690+0000 7ff808223700 1 librados: starting msgr at 2024-02-23T00:50:04.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.690+0000 7ff808223700 1 librados: starting objecter 2024-02-23T00:50:04.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.691+0000 7ff808223700 1 Processor -- start 2024-02-23T00:50:04.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.691+0000 7ff808223700 1 -- start start 2024-02-23T00:50:04.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.691+0000 7ff808223700 1 librados: setting wanted keys 2024-02-23T00:50:04.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.691+0000 7ff808223700 1 librados: calling monclient init 2024-02-23T00:50:04.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.691+0000 7ff808223700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8000f8cb0 0x7ff8000f86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:04.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.691+0000 7ff808223700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8000f9610 0x7ff8000f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:04.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.691+0000 7ff808223700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff800102210 0x7ff8000f7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:04.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.691+0000 7ff808223700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8000f7950 con 0x7ff8000f8cb0 2024-02-23T00:50:04.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.691+0000 7ff808223700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8000f7ac0 con 0x7ff8000f9610 2024-02-23T00:50:04.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.691+0000 7ff808223700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff80010c4f0 con 0x7ff800102210 2024-02-23T00:50:04.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.691+0000 7ff805fbf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8000f8cb0 0x7ff8000f86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:04.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.691+0000 7ff805fbf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8000f8cb0 0x7ff8000f86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40002/0 (socket says 172.21.15.47:40002) 2024-02-23T00:50:04.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.691+0000 7ff805fbf700 1 -- 172.21.15.47:0/4104135238 learned_addr learned my addr 172.21.15.47:0/4104135238 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:50:04.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.691+0000 7ff8067c0700 1 --2- 172.21.15.47:0/4104135238 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff800102210 0x7ff8000f7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:04.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.692+0000 7ff805fbf700 1 -- 172.21.15.47:0/4104135238 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff800102210 msgr2=0x7ff8000f7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:04.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.692+0000 7ff805fbf700 1 --2- 172.21.15.47:0/4104135238 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff800102210 0x7ff8000f7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:04.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.692+0000 7ff805fbf700 1 -- 172.21.15.47:0/4104135238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8000f9610 msgr2=0x7ff8000f6c60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-23T00:50:04.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.692+0000 7ff805fbf700 1 --2- 172.21.15.47:0/4104135238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8000f9610 0x7ff8000f6c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:50:04.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.692+0000 7ff805fbf700 1 -- 172.21.15.47:0/4104135238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f0005040 con 0x7ff8000f8cb0 2024-02-23T00:50:04.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.692+0000 7ff8067c0700 1 --2- 172.21.15.47:0/4104135238 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff800102210 0x7ff8000f7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:50:04.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.692+0000 7ff805fbf700 1 --2- 172.21.15.47:0/4104135238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8000f8cb0 0x7ff8000f86c0 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 rx=0x7ff7fc00e970 tx=0x7ff7fc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:04.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.692+0000 7ff7f6ffd700 10 client.?.objecter ms_handle_connect 0x7ff8000f8cb0 2024-02-23T00:50:04.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.692+0000 7ff7f6ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:50:04.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.692+0000 7ff7f6ffd700 1 -- 172.21.15.47:0/4104135238 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc00ee60 con 0x7ff8000f8cb0 2024-02-23T00:50:04.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.692+0000 7ff808223700 1 -- 172.21.15.47:0/4104135238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff80010c7d0 con 0x7ff8000f8cb0 2024-02-23T00:50:04.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.692+0000 7ff808223700 10 client.31188.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:50:04.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.692+0000 7ff808223700 1 -- 172.21.15.47:0/4104135238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff80010cd10 con 0x7ff8000f8cb0 2024-02-23T00:50:04.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.692+0000 7ff808223700 1 librados: init done 2024-02-23T00:50:04.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.692+0000 7ff7f6ffd700 1 -- 172.21.15.47:0/4104135238 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7ff7fc00f040 con 0x7ff8000f8cb0 2024-02-23T00:50:04.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.692+0000 7ff7f6ffd700 1 -- 172.21.15.47:0/4104135238 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc00c5b0 con 0x7ff8000f8cb0 2024-02-23T00:50:04.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.694+0000 7ff808223700 1 -- 172.21.15.47:0/4104135238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7e4004f50 con 0x7ff8000f8cb0 2024-02-23T00:50:04.699 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:04.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.694+0000 7ff7f6ffd700 1 -- 172.21.15.47:0/4104135238 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7ff7fc007b40 con 0x7ff8000f8cb0 2024-02-23T00:50:04.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.694+0000 7ff7f6ffd700 1 --2- 172.21.15.47:0/4104135238 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff7ec062cc0 0x7ff7ec065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:04.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.694+0000 7ff7f6ffd700 1 -- 172.21.15.47:0/4104135238 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(481..481 src has 48..481) v4 ==== 9220+0+0 (secure 0 0 0) 0x7ff7fc07d1f0 con 0x7ff8000f8cb0 2024-02-23T00:50:04.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.694+0000 7ff7f6ffd700 10 client.31188.objecter ms_dispatch 0x7ff800001550 osd_map(481..481 src has 48..481) v4 2024-02-23T00:50:04.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.694+0000 7ff7f6ffd700 3 client.31188.objecter handle_osd_map got epochs [481,481] > 0 2024-02-23T00:50:04.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.694+0000 7ff7f6ffd700 3 client.31188.objecter handle_osd_map decoding full epoch 481 2024-02-23T00:50:04.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.694+0000 7ff8057be700 1 --2- 172.21.15.47:0/4104135238 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff7ec062cc0 0x7ff7ec065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:04.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.695+0000 7ff7f6ffd700 20 client.31188.objecter dump_active .. 0 homeless 2024-02-23T00:50:04.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.695+0000 7ff7f6ffd700 10 client.31188.objecter ms_handle_connect 0x7ff7ec062cc0 2024-02-23T00:50:04.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.695+0000 7ff8057be700 1 --2- 172.21.15.47:0/4104135238 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff7ec062cc0 0x7ff7ec065180 secure :-1 s=READY pgs=3094 cs=0 l=1 rev1=1 rx=0x7ff7f0009da0 tx=0x7ff7f0009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:04.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.699+0000 7ff7f6ffd700 1 -- 172.21.15.47:0/4104135238 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ff7fc04c930 con 0x7ff8000f8cb0 2024-02-23T00:50:04.717 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:50:04.720 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:04.720 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:50:04.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:03.321552+0000 mgr.y (mgr.14517) 455 : cluster [DBG] pgmap v722: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.4 KiB/s rd, 724 B/s wr, 2 op/s 2024-02-23T00:50:04.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:03.321552+0000 mgr.y (mgr.14517) 455 : cluster [DBG] pgmap v722: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.4 KiB/s rd, 724 B/s wr, 2 op/s 2024-02-23T00:50:04.821 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:50:04.852 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:04.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:04.893+0000 7ff808223700 1 -- 172.21.15.47:0/4104135238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd reweight", "id": 0, "weight": 1.0} v 0) v1 -- 0x7ff7e4005200 con 0x7ff8000f8cb0 2024-02-23T00:50:05.000 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:50:05.004 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:05.004 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:50:05.105 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:50:05.133 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:05.383 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:50:05.385 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:05.386 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:50:05.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:05.395+0000 7ff7f6ffd700 1 -- 172.21.15.47:0/4104135238 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 0, "weight": 1.0}]=0 reweighted osd.0 to 1 (10000) v482) v1 ==== 113+0+0 (secure 0 0 0) 0x7ff7fc019070 con 0x7ff8000f8cb0 2024-02-23T00:50:05.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:05.397+0000 7ff808223700 10 librados: watch_flush enter 2024-02-23T00:50:05.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:05.397+0000 7ff808223700 10 librados: watch_flush exit 2024-02-23T00:50:05.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:05.397+0000 7ff808223700 20 client.31188.objecter shutdown clearing up homeless session... 2024-02-23T00:50:05.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:05.397+0000 7ff808223700 10 client.31188.objecter successfully canceled tick 2024-02-23T00:50:05.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:05.397+0000 7ff808223700 1 -- 172.21.15.47:0/4104135238 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff7ec062cc0 msgr2=0x7ff7ec065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:05.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:05.397+0000 7ff808223700 1 --2- 172.21.15.47:0/4104135238 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff7ec062cc0 0x7ff7ec065180 secure :-1 s=READY pgs=3094 cs=0 l=1 rev1=1 rx=0x7ff7f0009da0 tx=0x7ff7f0009cb0).stop 2024-02-23T00:50:05.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:05.397+0000 7ff808223700 1 -- 172.21.15.47:0/4104135238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8000f8cb0 msgr2=0x7ff8000f86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:05.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:05.397+0000 7ff808223700 1 --2- 172.21.15.47:0/4104135238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8000f8cb0 0x7ff8000f86c0 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 rx=0x7ff7fc00e970 tx=0x7ff7fc00b040).stop 2024-02-23T00:50:05.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:05.397+0000 7ff808223700 1 -- 172.21.15.47:0/4104135238 shutdown_connections 2024-02-23T00:50:05.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:05.397+0000 7ff808223700 1 --2- 172.21.15.47:0/4104135238 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff7ec062cc0 0x7ff7ec065180 unknown :-1 s=CLOSED pgs=3094 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:05.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:05.397+0000 7ff808223700 1 --2- 172.21.15.47:0/4104135238 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff800102210 0x7ff8000f7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:05.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:05.397+0000 7ff808223700 1 --2- 172.21.15.47:0/4104135238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8000f8cb0 0x7ff8000f86c0 unknown :-1 s=CLOSED pgs=2554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:05.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:05.397+0000 7ff808223700 1 --2- 172.21.15.47:0/4104135238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8000f9610 0x7ff8000f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:50:05.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:05.397+0000 7ff808223700 1 -- 172.21.15.47:0/4104135238 >> 172.21.15.47:0/4104135238 conn(0x7ff8000f2820 msgr2=0x7ff8000f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:05.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:05.397+0000 7ff808223700 1 -- 172.21.15.47:0/4104135238 shutdown_connections 2024-02-23T00:50:05.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:05.397+0000 7ff808223700 1 -- 172.21.15.47:0/4104135238 wait complete. 2024-02-23T00:50:05.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:05.397+0000 7ff808223700 1 librados: shutdown 2024-02-23T00:50:05.402 INFO:teuthology.orchestra.run.smithi047.stderr:reweighted osd.0 to 1 (10000) 2024-02-23T00:50:05.487 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:50:05.522 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:05.543 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:50:05.546 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:05.547 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:50:05.648 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:50:05.674 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:04.895394+0000 mon.a (mon.0) 2741 : audit [INF] from='client.? 172.21.15.47:0/4104135238' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-23T00:50:05.675 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:04.895394+0000 mon.a (mon.0) 2741 : audit [INF] from='client.? 172.21.15.47:0/4104135238' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-23T00:50:05.677 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:05.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:04.895394+0000 mon.a (mon.0) 2741 : audit [INF] from='client.? 172.21.15.47:0/4104135238' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-23T00:50:06.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:05.322144+0000 mgr.y (mgr.14517) 456 : cluster [DBG] pgmap v723: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:50:06.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:05.393322+0000 mon.a (mon.0) 2742 : audit [INF] from='client.? 172.21.15.47:0/4104135238' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-02-23T00:50:06.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:05.393383+0000 mon.a (mon.0) 2743 : cluster [DBG] osdmap e482: 8 total, 8 up, 7 in 2024-02-23T00:50:06.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:05.322144+0000 mgr.y (mgr.14517) 456 : cluster [DBG] pgmap v723: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:50:06.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:05.393322+0000 mon.a (mon.0) 2742 : audit [INF] from='client.? 172.21.15.47:0/4104135238' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-02-23T00:50:06.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:05.393383+0000 mon.a (mon.0) 2743 : cluster [DBG] osdmap e482: 8 total, 8 up, 7 in 2024-02-23T00:50:06.628 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:50:06.631 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:06.632 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:50:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:05.322144+0000 mgr.y (mgr.14517) 456 : cluster [DBG] pgmap v723: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:50:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:05.393322+0000 mon.a (mon.0) 2742 : audit [INF] from='client.? 172.21.15.47:0/4104135238' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-02-23T00:50:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:05.393383+0000 mon.a (mon.0) 2743 : cluster [DBG] osdmap e482: 8 total, 8 up, 7 in 2024-02-23T00:50:06.732 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:50:06.764 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:06.786 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:50:06.789 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:06.789 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:50:06.849 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:50:06.890 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:50:06.923 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:06.943 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:50:06.951 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:06.951 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:50:07.052 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:50:07.084 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:07.105 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:50:07.108 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:07.108 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:50:07.209 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:50:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:07.473 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:50:07.476 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight 1 1 2024-02-23T00:50:07.477 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:07.477 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:50:07.579 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:50:07.610 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:07.760 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:50:07.765 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:07.766 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:50:07.867 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:50:07.899 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:07.921 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:50:07.923 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:07.924 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:50:08.026 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:50:08.056 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:08.076 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:50:08.079 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:08.079 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:50:08.180 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:50:08.209 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:08.229 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:50:08.231 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:08.232 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:50:08.333 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:50:08.359 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:08.378 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:50:08.380 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:08.381 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:50:08.481 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:50:08.513 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:08.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.147+0000 7f9020225700 1 -- 172.21.15.47:0/1406297262 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f90180fd510 msgr2=0x7f90180f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:08.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.147+0000 7f9020225700 1 --2- 172.21.15.47:0/1406297262 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f90180fd510 0x7f90180f6d40 secure :-1 s=READY pgs=2703 cs=0 l=1 rev1=1 rx=0x7f900c004870 tx=0x7f900c009cb0).stop 2024-02-23T00:50:08.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.147+0000 7f9020225700 1 -- 172.21.15.47:0/1406297262 shutdown_connections 2024-02-23T00:50:08.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.147+0000 7f9020225700 1 --2- 172.21.15.47:0/1406297262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90180f7400 0x7f9018103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:08.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.147+0000 7f9020225700 1 --2- 172.21.15.47:0/1406297262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90180fcbb0 0x7f90180fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:08.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.147+0000 7f9020225700 1 --2- 172.21.15.47:0/1406297262 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f90180fd510 0x7f90180f6d40 unknown :-1 s=CLOSED pgs=2703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:08.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.147+0000 7f9020225700 1 -- 172.21.15.47:0/1406297262 >> 172.21.15.47:0/1406297262 conn(0x7f90180f2820 msgr2=0x7f90180f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:08.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.148+0000 7f9020225700 1 -- 172.21.15.47:0/1406297262 shutdown_connections 2024-02-23T00:50:08.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.148+0000 7f9020225700 1 -- 172.21.15.47:0/1406297262 wait complete. 2024-02-23T00:50:08.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.148+0000 7f9020225700 1 librados: starting msgr at 2024-02-23T00:50:08.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.148+0000 7f9020225700 1 librados: starting objecter 2024-02-23T00:50:08.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.148+0000 7f9020225700 1 Processor -- start 2024-02-23T00:50:08.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.148+0000 7f9020225700 1 -- start start 2024-02-23T00:50:08.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.148+0000 7f9020225700 1 librados: setting wanted keys 2024-02-23T00:50:08.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.148+0000 7f9020225700 1 librados: calling monclient init 2024-02-23T00:50:08.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.149+0000 7f9020225700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90180f7400 0x7f9018109140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:08.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.149+0000 7f9020225700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f90180fcbb0 0x7f9018109630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:08.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.149+0000 7f9020225700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90180fd510 0x7f9018109c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:08.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.149+0000 7f9020225700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9018114320 con 0x7f90180fd510 2024-02-23T00:50:08.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.149+0000 7f9020225700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9018114490 con 0x7f90180f7400 2024-02-23T00:50:08.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.149+0000 7f9020225700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9018114650 con 0x7f90180fcbb0 2024-02-23T00:50:08.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.149+0000 7f901d7c0700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f90180fcbb0 0x7f9018109630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:08.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.149+0000 7f901d7c0700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f90180fcbb0 0x7f9018109630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:34144/0 (socket says 172.21.15.47:34144) 2024-02-23T00:50:08.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.149+0000 7f901d7c0700 1 -- 172.21.15.47:0/2147264050 learned_addr learned my addr 172.21.15.47:0/2147264050 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:50:08.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.149+0000 7f901e7c2700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90180fd510 0x7f9018109c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:08.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.149+0000 7f901dfc1700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90180f7400 0x7f9018109140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:08.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.149+0000 7f901d7c0700 1 -- 172.21.15.47:0/2147264050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90180f7400 msgr2=0x7f9018109140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:08.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.149+0000 7f901d7c0700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90180f7400 0x7f9018109140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:08.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.149+0000 7f901d7c0700 1 -- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90180fd510 msgr2=0x7f9018109c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:08.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.149+0000 7f901d7c0700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90180fd510 0x7f9018109c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:08.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.149+0000 7f901d7c0700 1 -- 172.21.15.47:0/2147264050 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9014005040 con 0x7f90180fcbb0 2024-02-23T00:50:08.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.149+0000 7f901d7c0700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f90180fcbb0 0x7f9018109630 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 rx=0x7f900c009da0 tx=0x7f900c009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:08.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.149+0000 7f900affd700 10 client.?.objecter ms_handle_connect 0x7f90180fcbb0 2024-02-23T00:50:08.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.149+0000 7f900affd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:50:08.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.149+0000 7f900affd700 1 -- 172.21.15.47:0/2147264050 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900c00e030 con 0x7f90180fcbb0 2024-02-23T00:50:08.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.149+0000 7f900affd700 1 -- 172.21.15.47:0/2147264050 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f900c013d70 con 0x7f90180fcbb0 2024-02-23T00:50:08.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.149+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f900c005040 con 0x7f90180fcbb0 2024-02-23T00:50:08.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.150+0000 7f900affd700 1 -- 172.21.15.47:0/2147264050 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900c018070 con 0x7f90180fcbb0 2024-02-23T00:50:08.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.150+0000 7f9020225700 10 client.41519.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:50:08.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.150+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90181071a0 con 0x7f90180fcbb0 2024-02-23T00:50:08.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.150+0000 7f9020225700 1 librados: init done 2024-02-23T00:50:08.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.150+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f901805fa60 con 0x7f90180fcbb0 2024-02-23T00:50:08.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.151+0000 7f900affd700 1 -- 172.21.15.47:0/2147264050 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f900c009240 con 0x7f90180fcbb0 2024-02-23T00:50:08.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.151+0000 7f900affd700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9004062cc0 0x7f9004065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:08.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.151+0000 7f900affd700 1 -- 172.21.15.47:0/2147264050 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(482..482 src has 48..482) v4 ==== 9220+0+0 (secure 0 0 0) 0x7f900c07e770 con 0x7f90180fcbb0 2024-02-23T00:50:08.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.151+0000 7f900affd700 10 client.41519.objecter ms_dispatch 0x7f9018001550 osd_map(482..482 src has 48..482) v4 2024-02-23T00:50:08.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.151+0000 7f901dfc1700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9004062cc0 0x7f9004065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:08.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.151+0000 7f900affd700 3 client.41519.objecter handle_osd_map got epochs [482,482] > 0 2024-02-23T00:50:08.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.151+0000 7f900affd700 3 client.41519.objecter handle_osd_map decoding full epoch 482 2024-02-23T00:50:08.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.152+0000 7f900affd700 20 client.41519.objecter dump_active .. 0 homeless 2024-02-23T00:50:08.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.152+0000 7f900affd700 1 -- 172.21.15.47:0/2147264050 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v482) v1 ==== 54+0+16 (secure 0 0 0) 0x7f900c085070 con 0x7f90180fcbb0 2024-02-23T00:50:08.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.152+0000 7f900affd700 10 client.41519.objecter ms_handle_connect 0x7f9004062cc0 2024-02-23T00:50:08.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.152+0000 7f901dfc1700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9004062cc0 0x7f9004065180 secure :-1 s=READY pgs=3095 cs=0 l=1 rev1=1 rx=0x7f9014009350 tx=0x7f901400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:08.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.152+0000 7f9020225700 10 client.41519.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:08.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.152+0000 7f9020225700 15 client.41519.objecter _session_command_op_assign -1 1 2024-02-23T00:50:08.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.152+0000 7f9020225700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f90180598f0 0x7f9018059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:08.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.152+0000 7f9020225700 20 client.41519.objecter _get_session s=0x7f901805ee80 osd=0 3 2024-02-23T00:50:08.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.152+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f901805ee80 osd=0 3 2024-02-23T00:50:08.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.152+0000 7f9020225700 20 client.41519.objecter _get_session s=0x7f901805ee80 osd=0 3 2024-02-23T00:50:08.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.152+0000 7f9020225700 15 client.41519.objecter _session_command_op_remove -1 1 2024-02-23T00:50:08.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.152+0000 7f9020225700 20 client.41519.objecter get_session s=0x7f901805ee80 osd=0 3 2024-02-23T00:50:08.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.152+0000 7f9020225700 15 client.41519.objecter _session_command_op_assign 0 1 2024-02-23T00:50:08.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.152+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f901805ee80 osd=0 4 2024-02-23T00:50:08.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.152+0000 7f9020225700 10 client.41519.objecter _send_command 1 2024-02-23T00:50:08.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.152+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9018105ea0 con 0x7f90180598f0 2024-02-23T00:50:08.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.152+0000 7f901e7c2700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f90180598f0 0x7f9018059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:08.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.152+0000 7f901e7c2700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f90180598f0 0x7f9018059cd0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:08.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.152+0000 7f900affd700 10 client.41519.objecter ms_handle_connect 0x7f90180598f0 2024-02-23T00:50:08.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.153+0000 7f900affd700 1 -- 172.21.15.47:0/2147264050 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f9018105ea0 con 0x7f90180598f0 2024-02-23T00:50:08.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.153+0000 7f900affd700 10 client.41519.objecter ms_dispatch 0x7f9018001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:50:08.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.153+0000 7f900affd700 10 client.41519.objecter _finish_command 1 = system:0 2024-02-23T00:50:08.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.153+0000 7f900affd700 20 client.41519.objecter put_session s=0x7f901805ee80 osd=0 4 2024-02-23T00:50:08.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.153+0000 7f900affd700 15 client.41519.objecter _session_command_op_remove 0 1 2024-02-23T00:50:08.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.168+0000 7f9020225700 10 client.41519.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:50:08.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.168+0000 7f9020225700 15 client.41519.objecter _session_command_op_assign -1 2 2024-02-23T00:50:08.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.168+0000 7f9020225700 20 client.41519.objecter _get_session s=0x7f901805ee80 osd=0 3 2024-02-23T00:50:08.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.168+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f901805ee80 osd=0 3 2024-02-23T00:50:08.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.168+0000 7f9020225700 20 client.41519.objecter _get_session s=0x7f901805ee80 osd=0 3 2024-02-23T00:50:08.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.168+0000 7f9020225700 15 client.41519.objecter _session_command_op_remove -1 2 2024-02-23T00:50:08.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.168+0000 7f9020225700 20 client.41519.objecter get_session s=0x7f901805ee80 osd=0 3 2024-02-23T00:50:08.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.168+0000 7f9020225700 15 client.41519.objecter _session_command_op_assign 0 2 2024-02-23T00:50:08.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.168+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f901805ee80 osd=0 4 2024-02-23T00:50:08.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.168+0000 7f9020225700 10 client.41519.objecter _send_command 2 2024-02-23T00:50:08.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.168+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f90180273c0 con 0x7f90180598f0 2024-02-23T00:50:08.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.169+0000 7f900affd700 1 -- 172.21.15.47:0/2147264050 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f90180273c0 con 0x7f90180598f0 2024-02-23T00:50:08.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.169+0000 7f900affd700 10 client.41519.objecter ms_dispatch 0x7f9018001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:50:08.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.169+0000 7f900affd700 10 client.41519.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:50:08.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.169+0000 7f900affd700 20 client.41519.objecter put_session s=0x7f901805ee80 osd=0 4 2024-02-23T00:50:08.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.169+0000 7f900affd700 15 client.41519.objecter _session_command_op_remove 0 2 2024-02-23T00:50:08.576 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:50:08.577 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:50:08.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.170+0000 7f9020225700 10 client.41519.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:08.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.170+0000 7f9020225700 15 client.41519.objecter _session_command_op_assign -1 3 2024-02-23T00:50:08.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.170+0000 7f9020225700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f90180fe180 0x7f90180fe560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:08.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.170+0000 7f9020225700 20 client.41519.objecter _get_session s=0x7f9018028710 osd=1 3 2024-02-23T00:50:08.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.170+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f9018028710 osd=1 3 2024-02-23T00:50:08.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.170+0000 7f9020225700 20 client.41519.objecter _get_session s=0x7f9018028710 osd=1 3 2024-02-23T00:50:08.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.170+0000 7f9020225700 15 client.41519.objecter _session_command_op_remove -1 3 2024-02-23T00:50:08.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.170+0000 7f9020225700 20 client.41519.objecter get_session s=0x7f9018028710 osd=1 3 2024-02-23T00:50:08.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.170+0000 7f9020225700 15 client.41519.objecter _session_command_op_assign 1 3 2024-02-23T00:50:08.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.170+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f9018028710 osd=1 4 2024-02-23T00:50:08.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.170+0000 7f9020225700 10 client.41519.objecter _send_command 3 2024-02-23T00:50:08.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.170+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f90181089a0 con 0x7f90180fe180 2024-02-23T00:50:08.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.170+0000 7f901dfc1700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f90180fe180 0x7f90180fe560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:08.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.170+0000 7f900affd700 10 client.41519.objecter ms_handle_connect 0x7f90180fe180 2024-02-23T00:50:08.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.170+0000 7f901dfc1700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f90180fe180 0x7f90180fe560 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:08.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.171+0000 7f900affd700 1 -- 172.21.15.47:0/2147264050 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f90181089a0 con 0x7f90180fe180 2024-02-23T00:50:08.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.171+0000 7f900affd700 10 client.41519.objecter ms_dispatch 0x7f9018001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:50:08.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.171+0000 7f900affd700 10 client.41519.objecter _finish_command 3 = system:0 2024-02-23T00:50:08.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.171+0000 7f900affd700 20 client.41519.objecter put_session s=0x7f9018028710 osd=1 4 2024-02-23T00:50:08.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.171+0000 7f900affd700 15 client.41519.objecter _session_command_op_remove 1 3 2024-02-23T00:50:08.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.186+0000 7f9020225700 10 client.41519.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:50:08.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.186+0000 7f9020225700 15 client.41519.objecter _session_command_op_assign -1 4 2024-02-23T00:50:08.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.186+0000 7f9020225700 20 client.41519.objecter _get_session s=0x7f9018028710 osd=1 3 2024-02-23T00:50:08.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.186+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f9018028710 osd=1 3 2024-02-23T00:50:08.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.186+0000 7f9020225700 20 client.41519.objecter _get_session s=0x7f9018028710 osd=1 3 2024-02-23T00:50:08.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.186+0000 7f9020225700 15 client.41519.objecter _session_command_op_remove -1 4 2024-02-23T00:50:08.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.186+0000 7f9020225700 20 client.41519.objecter get_session s=0x7f9018028710 osd=1 3 2024-02-23T00:50:08.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.186+0000 7f9020225700 15 client.41519.objecter _session_command_op_assign 1 4 2024-02-23T00:50:08.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.186+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f9018028710 osd=1 4 2024-02-23T00:50:08.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.186+0000 7f9020225700 10 client.41519.objecter _send_command 4 2024-02-23T00:50:08.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.186+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f9018108d80 con 0x7f90180fe180 2024-02-23T00:50:08.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.187+0000 7f900affd700 1 -- 172.21.15.47:0/2147264050 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f9018108d80 con 0x7f90180fe180 2024-02-23T00:50:08.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.187+0000 7f900affd700 10 client.41519.objecter ms_dispatch 0x7f9018001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:50:08.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.187+0000 7f900affd700 10 client.41519.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:50:08.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.187+0000 7f900affd700 20 client.41519.objecter put_session s=0x7f9018028710 osd=1 4 2024-02-23T00:50:08.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.187+0000 7f900affd700 15 client.41519.objecter _session_command_op_remove 1 4 2024-02-23T00:50:08.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.187+0000 7f9020225700 10 client.41519.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:08.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.187+0000 7f9020225700 15 client.41519.objecter _session_command_op_assign -1 5 2024-02-23T00:50:08.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.187+0000 7f9020225700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f9018108540 0x7f9018027d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:08.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.187+0000 7f9020225700 20 client.41519.objecter _get_session s=0x7f9018108f50 osd=2 3 2024-02-23T00:50:08.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.187+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f9018108f50 osd=2 3 2024-02-23T00:50:08.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.187+0000 7f9020225700 20 client.41519.objecter _get_session s=0x7f9018108f50 osd=2 3 2024-02-23T00:50:08.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.187+0000 7f9020225700 15 client.41519.objecter _session_command_op_remove -1 5 2024-02-23T00:50:08.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.187+0000 7f9020225700 20 client.41519.objecter get_session s=0x7f9018108f50 osd=2 3 2024-02-23T00:50:08.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.187+0000 7f9020225700 15 client.41519.objecter _session_command_op_assign 2 5 2024-02-23T00:50:08.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.187+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f9018108f50 osd=2 4 2024-02-23T00:50:08.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.187+0000 7f9020225700 10 client.41519.objecter _send_command 5 2024-02-23T00:50:08.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.187+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9018028250 con 0x7f9018108540 2024-02-23T00:50:08.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.187+0000 7f901d7c0700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f9018108540 0x7f9018027d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:08.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.188+0000 7f900affd700 10 client.41519.objecter ms_handle_connect 0x7f9018108540 2024-02-23T00:50:08.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.188+0000 7f901d7c0700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f9018108540 0x7f9018027d60 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:08.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.188+0000 7f900affd700 1 -- 172.21.15.47:0/2147264050 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f9018028250 con 0x7f9018108540 2024-02-23T00:50:08.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.188+0000 7f900affd700 10 client.41519.objecter ms_dispatch 0x7f9018001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:50:08.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.188+0000 7f900affd700 10 client.41519.objecter _finish_command 5 = system:0 2024-02-23T00:50:08.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.188+0000 7f900affd700 20 client.41519.objecter put_session s=0x7f9018108f50 osd=2 4 2024-02-23T00:50:08.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.188+0000 7f900affd700 15 client.41519.objecter _session_command_op_remove 2 5 2024-02-23T00:50:08.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.203+0000 7f9020225700 10 client.41519.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:50:08.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.203+0000 7f9020225700 15 client.41519.objecter _session_command_op_assign -1 6 2024-02-23T00:50:08.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.203+0000 7f9020225700 20 client.41519.objecter _get_session s=0x7f9018108f50 osd=2 3 2024-02-23T00:50:08.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.203+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f9018108f50 osd=2 3 2024-02-23T00:50:08.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.203+0000 7f9020225700 20 client.41519.objecter _get_session s=0x7f9018108f50 osd=2 3 2024-02-23T00:50:08.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.203+0000 7f9020225700 15 client.41519.objecter _session_command_op_remove -1 6 2024-02-23T00:50:08.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.203+0000 7f9020225700 20 client.41519.objecter get_session s=0x7f9018108f50 osd=2 3 2024-02-23T00:50:08.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.203+0000 7f9020225700 15 client.41519.objecter _session_command_op_assign 2 6 2024-02-23T00:50:08.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.203+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f9018108f50 osd=2 4 2024-02-23T00:50:08.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.203+0000 7f9020225700 10 client.41519.objecter _send_command 6 2024-02-23T00:50:08.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.203+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f901811a900 con 0x7f9018108540 2024-02-23T00:50:08.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.204+0000 7f900affd700 1 -- 172.21.15.47:0/2147264050 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f901811a900 con 0x7f9018108540 2024-02-23T00:50:08.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.204+0000 7f900affd700 10 client.41519.objecter ms_dispatch 0x7f9018001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:50:08.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.204+0000 7f900affd700 10 client.41519.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:50:08.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.204+0000 7f900affd700 20 client.41519.objecter put_session s=0x7f9018108f50 osd=2 4 2024-02-23T00:50:08.582 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:50:08.582 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:50:08.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.204+0000 7f900affd700 15 client.41519.objecter _session_command_op_remove 2 6 2024-02-23T00:50:08.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.205+0000 7f9020225700 10 client.41519.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:08.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.205+0000 7f9020225700 15 client.41519.objecter _session_command_op_assign -1 7 2024-02-23T00:50:08.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.205+0000 7f9020225700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f9018027940 0x7f901811dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:08.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.205+0000 7f9020225700 20 client.41519.objecter _get_session s=0x7f90180277d0 osd=3 3 2024-02-23T00:50:08.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.205+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f90180277d0 osd=3 3 2024-02-23T00:50:08.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.205+0000 7f9020225700 20 client.41519.objecter _get_session s=0x7f90180277d0 osd=3 3 2024-02-23T00:50:08.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.205+0000 7f9020225700 15 client.41519.objecter _session_command_op_remove -1 7 2024-02-23T00:50:08.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.205+0000 7f9020225700 20 client.41519.objecter get_session s=0x7f90180277d0 osd=3 3 2024-02-23T00:50:08.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.205+0000 7f9020225700 15 client.41519.objecter _session_command_op_assign 3 7 2024-02-23T00:50:08.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.205+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f90180277d0 osd=3 4 2024-02-23T00:50:08.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.205+0000 7f9020225700 10 client.41519.objecter _send_command 7 2024-02-23T00:50:08.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.205+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9018108190 con 0x7f9018027940 2024-02-23T00:50:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.205+0000 7f901e7c2700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f9018027940 0x7f901811dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.205+0000 7f900affd700 10 client.41519.objecter ms_handle_connect 0x7f9018027940 2024-02-23T00:50:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.205+0000 7f901e7c2700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f9018027940 0x7f901811dc00 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.206+0000 7f900affd700 1 -- 172.21.15.47:0/2147264050 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f9018108190 con 0x7f9018027940 2024-02-23T00:50:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.206+0000 7f900affd700 10 client.41519.objecter ms_dispatch 0x7f9018001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:50:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.206+0000 7f900affd700 10 client.41519.objecter _finish_command 7 = system:0 2024-02-23T00:50:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.206+0000 7f900affd700 20 client.41519.objecter put_session s=0x7f90180277d0 osd=3 4 2024-02-23T00:50:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.206+0000 7f900affd700 15 client.41519.objecter _session_command_op_remove 3 7 2024-02-23T00:50:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.221+0000 7f9020225700 10 client.41519.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:50:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.221+0000 7f9020225700 15 client.41519.objecter _session_command_op_assign -1 8 2024-02-23T00:50:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.221+0000 7f9020225700 20 client.41519.objecter _get_session s=0x7f90180277d0 osd=3 3 2024-02-23T00:50:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.221+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f90180277d0 osd=3 3 2024-02-23T00:50:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.221+0000 7f9020225700 20 client.41519.objecter _get_session s=0x7f90180277d0 osd=3 3 2024-02-23T00:50:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.221+0000 7f9020225700 15 client.41519.objecter _session_command_op_remove -1 8 2024-02-23T00:50:08.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.221+0000 7f9020225700 20 client.41519.objecter get_session s=0x7f90180277d0 osd=3 3 2024-02-23T00:50:08.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.221+0000 7f9020225700 15 client.41519.objecter _session_command_op_assign 3 8 2024-02-23T00:50:08.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.221+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f90180277d0 osd=3 4 2024-02-23T00:50:08.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.221+0000 7f9020225700 10 client.41519.objecter _send_command 8 2024-02-23T00:50:08.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.221+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f9018108360 con 0x7f9018027940 2024-02-23T00:50:08.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.223+0000 7f900affd700 1 -- 172.21.15.47:0/2147264050 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f9018108360 con 0x7f9018027940 2024-02-23T00:50:08.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.223+0000 7f900affd700 10 client.41519.objecter ms_dispatch 0x7f9018001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:50:08.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.223+0000 7f900affd700 10 client.41519.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:50:08.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.223+0000 7f900affd700 20 client.41519.objecter put_session s=0x7f90180277d0 osd=3 4 2024-02-23T00:50:08.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.223+0000 7f900affd700 15 client.41519.objecter _session_command_op_remove 3 8 2024-02-23T00:50:08.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.223+0000 7f9020225700 10 client.41519.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:08.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.223+0000 7f9020225700 15 client.41519.objecter _session_command_op_assign -1 9 2024-02-23T00:50:08.586 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:50:08.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.223+0000 7f9020225700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f901811b240 0x7f901811b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:08.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.223+0000 7f9020225700 20 client.41519.objecter _get_session s=0x7f9018108360 osd=4 3 2024-02-23T00:50:08.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.223+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f9018108360 osd=4 3 2024-02-23T00:50:08.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.223+0000 7f9020225700 20 client.41519.objecter _get_session s=0x7f9018108360 osd=4 3 2024-02-23T00:50:08.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.223+0000 7f9020225700 15 client.41519.objecter _session_command_op_remove -1 9 2024-02-23T00:50:08.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.223+0000 7f9020225700 20 client.41519.objecter get_session s=0x7f9018108360 osd=4 3 2024-02-23T00:50:08.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.223+0000 7f9020225700 15 client.41519.objecter _session_command_op_assign 4 9 2024-02-23T00:50:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.223+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f9018108360 osd=4 4 2024-02-23T00:50:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.223+0000 7f9020225700 10 client.41519.objecter _send_command 9 2024-02-23T00:50:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.223+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9018123ad0 con 0x7f901811b240 2024-02-23T00:50:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.223+0000 7f901dfc1700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f901811b240 0x7f901811b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.224+0000 7f900affd700 10 client.41519.objecter ms_handle_connect 0x7f901811b240 2024-02-23T00:50:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.224+0000 7f901dfc1700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f901811b240 0x7f901811b6a0 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.224+0000 7f900affd700 1 -- 172.21.15.47:0/2147264050 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f9018123ad0 con 0x7f901811b240 2024-02-23T00:50:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.224+0000 7f900affd700 10 client.41519.objecter ms_dispatch 0x7f9018001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:50:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.224+0000 7f900affd700 10 client.41519.objecter _finish_command 9 = system:0 2024-02-23T00:50:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.224+0000 7f900affd700 20 client.41519.objecter put_session s=0x7f9018108360 osd=4 4 2024-02-23T00:50:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.224+0000 7f900affd700 15 client.41519.objecter _session_command_op_remove 4 9 2024-02-23T00:50:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.241+0000 7f9020225700 10 client.41519.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:50:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.241+0000 7f9020225700 15 client.41519.objecter _session_command_op_assign -1 10 2024-02-23T00:50:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.241+0000 7f9020225700 20 client.41519.objecter _get_session s=0x7f9018108360 osd=4 3 2024-02-23T00:50:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.241+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f9018108360 osd=4 3 2024-02-23T00:50:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.241+0000 7f9020225700 20 client.41519.objecter _get_session s=0x7f9018108360 osd=4 3 2024-02-23T00:50:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.241+0000 7f9020225700 15 client.41519.objecter _session_command_op_remove -1 10 2024-02-23T00:50:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.241+0000 7f9020225700 20 client.41519.objecter get_session s=0x7f9018108360 osd=4 3 2024-02-23T00:50:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.241+0000 7f9020225700 15 client.41519.objecter _session_command_op_assign 4 10 2024-02-23T00:50:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.241+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f9018108360 osd=4 4 2024-02-23T00:50:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.241+0000 7f9020225700 10 client.41519.objecter _send_command 10 2024-02-23T00:50:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.241+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f9018123ad0 con 0x7f901811b240 2024-02-23T00:50:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.243+0000 7f900affd700 1 -- 172.21.15.47:0/2147264050 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f9018123ad0 con 0x7f901811b240 2024-02-23T00:50:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.243+0000 7f900affd700 10 client.41519.objecter ms_dispatch 0x7f9018001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:50:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.243+0000 7f900affd700 10 client.41519.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:50:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.243+0000 7f900affd700 20 client.41519.objecter put_session s=0x7f9018108360 osd=4 4 2024-02-23T00:50:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.243+0000 7f900affd700 15 client.41519.objecter _session_command_op_remove 4 10 2024-02-23T00:50:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.243+0000 7f9020225700 10 client.41519.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.243+0000 7f9020225700 15 client.41519.objecter _session_command_op_assign -1 11 2024-02-23T00:50:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.243+0000 7f9020225700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f9018121880 0x7f9018121cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.243+0000 7f9020225700 20 client.41519.objecter _get_session s=0x7f9018123ad0 osd=5 3 2024-02-23T00:50:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.243+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f9018123ad0 osd=5 3 2024-02-23T00:50:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.243+0000 7f9020225700 20 client.41519.objecter _get_session s=0x7f9018123ad0 osd=5 3 2024-02-23T00:50:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.243+0000 7f9020225700 15 client.41519.objecter _session_command_op_remove -1 11 2024-02-23T00:50:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.243+0000 7f9020225700 20 client.41519.objecter get_session s=0x7f9018123ad0 osd=5 3 2024-02-23T00:50:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.243+0000 7f9020225700 15 client.41519.objecter _session_command_op_assign 5 11 2024-02-23T00:50:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.243+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f9018123ad0 osd=5 4 2024-02-23T00:50:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.243+0000 7f9020225700 10 client.41519.objecter _send_command 11 2024-02-23T00:50:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.243+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9018126e50 con 0x7f9018121880 2024-02-23T00:50:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.244+0000 7f901d7c0700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f9018121880 0x7f9018121cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.244+0000 7f901d7c0700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f9018121880 0x7f9018121cc0 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:08.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.244+0000 7f900affd700 10 client.41519.objecter ms_handle_connect 0x7f9018121880 2024-02-23T00:50:08.590 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:50:08.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.245+0000 7f900affd700 1 -- 172.21.15.47:0/2147264050 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f9018126e50 con 0x7f9018121880 2024-02-23T00:50:08.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.245+0000 7f900affd700 10 client.41519.objecter ms_dispatch 0x7f9018001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:50:08.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.245+0000 7f900affd700 10 client.41519.objecter _finish_command 11 = system:0 2024-02-23T00:50:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.245+0000 7f900affd700 20 client.41519.objecter put_session s=0x7f9018123ad0 osd=5 4 2024-02-23T00:50:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.245+0000 7f900affd700 15 client.41519.objecter _session_command_op_remove 5 11 2024-02-23T00:50:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.259+0000 7f9020225700 10 client.41519.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:50:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.259+0000 7f9020225700 15 client.41519.objecter _session_command_op_assign -1 12 2024-02-23T00:50:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.259+0000 7f9020225700 20 client.41519.objecter _get_session s=0x7f9018123ad0 osd=5 3 2024-02-23T00:50:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.259+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f9018123ad0 osd=5 3 2024-02-23T00:50:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.259+0000 7f9020225700 20 client.41519.objecter _get_session s=0x7f9018123ad0 osd=5 3 2024-02-23T00:50:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.259+0000 7f9020225700 15 client.41519.objecter _session_command_op_remove -1 12 2024-02-23T00:50:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.259+0000 7f9020225700 20 client.41519.objecter get_session s=0x7f9018123ad0 osd=5 3 2024-02-23T00:50:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.259+0000 7f9020225700 15 client.41519.objecter _session_command_op_assign 5 12 2024-02-23T00:50:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.259+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f9018123ad0 osd=5 4 2024-02-23T00:50:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.259+0000 7f9020225700 10 client.41519.objecter _send_command 12 2024-02-23T00:50:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.259+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f90181222c0 con 0x7f9018121880 2024-02-23T00:50:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.261+0000 7f900affd700 1 -- 172.21.15.47:0/2147264050 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f90181222c0 con 0x7f9018121880 2024-02-23T00:50:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.261+0000 7f900affd700 10 client.41519.objecter ms_dispatch 0x7f9018001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:50:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.261+0000 7f900affd700 10 client.41519.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:50:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.261+0000 7f900affd700 20 client.41519.objecter put_session s=0x7f9018123ad0 osd=5 4 2024-02-23T00:50:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.261+0000 7f900affd700 15 client.41519.objecter _session_command_op_remove 5 12 2024-02-23T00:50:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.261+0000 7f9020225700 10 client.41519.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.261+0000 7f9020225700 15 client.41519.objecter _session_command_op_assign -1 13 2024-02-23T00:50:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.261+0000 7f9020225700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f9018122940 0x7f9018122da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.261+0000 7f9020225700 20 client.41519.objecter _get_session s=0x7f90181222c0 osd=6 3 2024-02-23T00:50:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.261+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f90181222c0 osd=6 3 2024-02-23T00:50:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.261+0000 7f9020225700 20 client.41519.objecter _get_session s=0x7f90181222c0 osd=6 3 2024-02-23T00:50:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.261+0000 7f9020225700 15 client.41519.objecter _session_command_op_remove -1 13 2024-02-23T00:50:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.261+0000 7f9020225700 20 client.41519.objecter get_session s=0x7f90181222c0 osd=6 3 2024-02-23T00:50:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.261+0000 7f9020225700 15 client.41519.objecter _session_command_op_assign 6 13 2024-02-23T00:50:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.261+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f90181222c0 osd=6 4 2024-02-23T00:50:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.261+0000 7f9020225700 10 client.41519.objecter _send_command 13 2024-02-23T00:50:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.261+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9018123430 con 0x7f9018122940 2024-02-23T00:50:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.261+0000 7f901e7c2700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f9018122940 0x7f9018122da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.262+0000 7f901e7c2700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f9018122940 0x7f9018122da0 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.262+0000 7f900affd700 10 client.41519.objecter ms_handle_connect 0x7f9018122940 2024-02-23T00:50:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.262+0000 7f900affd700 1 -- 172.21.15.47:0/2147264050 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f9018123430 con 0x7f9018122940 2024-02-23T00:50:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.263+0000 7f900affd700 10 client.41519.objecter ms_dispatch 0x7f9018001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:50:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.263+0000 7f900affd700 10 client.41519.objecter _finish_command 13 = system:0 2024-02-23T00:50:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.263+0000 7f900affd700 20 client.41519.objecter put_session s=0x7f90181222c0 osd=6 4 2024-02-23T00:50:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.263+0000 7f900affd700 15 client.41519.objecter _session_command_op_remove 6 13 2024-02-23T00:50:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.277+0000 7f9020225700 10 client.41519.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:50:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.277+0000 7f9020225700 15 client.41519.objecter _session_command_op_assign -1 14 2024-02-23T00:50:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.277+0000 7f9020225700 20 client.41519.objecter _get_session s=0x7f90181222c0 osd=6 3 2024-02-23T00:50:08.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.277+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f90181222c0 osd=6 3 2024-02-23T00:50:08.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.277+0000 7f9020225700 20 client.41519.objecter _get_session s=0x7f90181222c0 osd=6 3 2024-02-23T00:50:08.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.277+0000 7f9020225700 15 client.41519.objecter _session_command_op_remove -1 14 2024-02-23T00:50:08.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.277+0000 7f9020225700 20 client.41519.objecter get_session s=0x7f90181222c0 osd=6 3 2024-02-23T00:50:08.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.277+0000 7f9020225700 15 client.41519.objecter _session_command_op_assign 6 14 2024-02-23T00:50:08.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.277+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f90181222c0 osd=6 4 2024-02-23T00:50:08.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.277+0000 7f9020225700 10 client.41519.objecter _send_command 14 2024-02-23T00:50:08.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.277+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f9018123430 con 0x7f9018122940 2024-02-23T00:50:08.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.279+0000 7f900affd700 1 -- 172.21.15.47:0/2147264050 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f9018123430 con 0x7f9018122940 2024-02-23T00:50:08.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.279+0000 7f900affd700 10 client.41519.objecter ms_dispatch 0x7f9018001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:50:08.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.279+0000 7f900affd700 10 client.41519.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:50:08.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.279+0000 7f900affd700 20 client.41519.objecter put_session s=0x7f90181222c0 osd=6 4 2024-02-23T00:50:08.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.279+0000 7f900affd700 15 client.41519.objecter _session_command_op_remove 6 14 2024-02-23T00:50:08.595 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:50:08.595 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:50:08.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.279+0000 7f9020225700 10 client.41519.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:08.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.279+0000 7f9020225700 15 client.41519.objecter _session_command_op_assign -1 15 2024-02-23T00:50:08.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.279+0000 7f9020225700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f90181281f0 0x7f901812a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:08.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.279+0000 7f9020225700 20 client.41519.objecter _get_session s=0x7f9018123430 osd=7 3 2024-02-23T00:50:08.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.279+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f9018123430 osd=7 3 2024-02-23T00:50:08.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.279+0000 7f9020225700 20 client.41519.objecter _get_session s=0x7f9018123430 osd=7 3 2024-02-23T00:50:08.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.279+0000 7f9020225700 15 client.41519.objecter _session_command_op_remove -1 15 2024-02-23T00:50:08.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.279+0000 7f9020225700 20 client.41519.objecter get_session s=0x7f9018123430 osd=7 3 2024-02-23T00:50:08.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.279+0000 7f9020225700 15 client.41519.objecter _session_command_op_assign 7 15 2024-02-23T00:50:08.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.279+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f9018123430 osd=7 4 2024-02-23T00:50:08.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.279+0000 7f9020225700 10 client.41519.objecter _send_command 15 2024-02-23T00:50:08.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.279+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f901812d230 con 0x7f90181281f0 2024-02-23T00:50:08.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.279+0000 7f901dfc1700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f90181281f0 0x7f901812a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:08.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.280+0000 7f900affd700 10 client.41519.objecter ms_handle_connect 0x7f90181281f0 2024-02-23T00:50:08.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.280+0000 7f901dfc1700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f90181281f0 0x7f901812a5e0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:08.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.280+0000 7f900affd700 1 -- 172.21.15.47:0/2147264050 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f901812d230 con 0x7f90181281f0 2024-02-23T00:50:08.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.280+0000 7f900affd700 10 client.41519.objecter ms_dispatch 0x7f9018001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:50:08.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.280+0000 7f900affd700 10 client.41519.objecter _finish_command 15 = system:0 2024-02-23T00:50:08.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.280+0000 7f900affd700 20 client.41519.objecter put_session s=0x7f9018123430 osd=7 4 2024-02-23T00:50:08.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.280+0000 7f900affd700 15 client.41519.objecter _session_command_op_remove 7 15 2024-02-23T00:50:08.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.295+0000 7f9020225700 10 client.41519.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:50:08.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.295+0000 7f9020225700 15 client.41519.objecter _session_command_op_assign -1 16 2024-02-23T00:50:08.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.295+0000 7f9020225700 20 client.41519.objecter _get_session s=0x7f9018123430 osd=7 3 2024-02-23T00:50:08.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.295+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f9018123430 osd=7 3 2024-02-23T00:50:08.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.295+0000 7f9020225700 20 client.41519.objecter _get_session s=0x7f9018123430 osd=7 3 2024-02-23T00:50:08.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.295+0000 7f9020225700 15 client.41519.objecter _session_command_op_remove -1 16 2024-02-23T00:50:08.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.295+0000 7f9020225700 20 client.41519.objecter get_session s=0x7f9018123430 osd=7 3 2024-02-23T00:50:08.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.295+0000 7f9020225700 15 client.41519.objecter _session_command_op_assign 7 16 2024-02-23T00:50:08.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.295+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f9018123430 osd=7 4 2024-02-23T00:50:08.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.295+0000 7f9020225700 10 client.41519.objecter _send_command 16 2024-02-23T00:50:08.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.295+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f901812aad0 con 0x7f90181281f0 2024-02-23T00:50:08.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f900affd700 1 -- 172.21.15.47:0/2147264050 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f901812aad0 con 0x7f90181281f0 2024-02-23T00:50:08.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f900affd700 10 client.41519.objecter ms_dispatch 0x7f9018001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:50:08.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f900affd700 10 client.41519.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:50:08.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f900affd700 20 client.41519.objecter put_session s=0x7f9018123430 osd=7 4 2024-02-23T00:50:08.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f900affd700 15 client.41519.objecter _session_command_op_remove 7 16 2024-02-23T00:50:08.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 10 librados: watch_flush enter 2024-02-23T00:50:08.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 10 librados: watch_flush exit 2024-02-23T00:50:08.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 10 client.41519.objecter close_session for osd.0 2024-02-23T00:50:08.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f90180598f0 msgr2=0x7f9018059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:08.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f90180598f0 0x7f9018059cd0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:08.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f901805ee80 osd=0 1 2024-02-23T00:50:08.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 10 client.41519.objecter close_session for osd.1 2024-02-23T00:50:08.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f90180fe180 msgr2=0x7f90180fe560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:08.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f90180fe180 0x7f90180fe560 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:08.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f9018028710 osd=1 1 2024-02-23T00:50:08.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 10 client.41519.objecter close_session for osd.2 2024-02-23T00:50:08.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f9018108540 msgr2=0x7f9018027d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:08.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f9018108540 0x7f9018027d60 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:08.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f9018108f50 osd=2 1 2024-02-23T00:50:08.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 10 client.41519.objecter close_session for osd.3 2024-02-23T00:50:08.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f9018027940 msgr2=0x7f901811dc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:08.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f901e7c2700 1 -- 172.21.15.47:0/2147264050 reap_dead start 2024-02-23T00:50:08.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f9018027940 0x7f901811dc00 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:08.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f90180277d0 osd=3 1 2024-02-23T00:50:08.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 10 client.41519.objecter close_session for osd.4 2024-02-23T00:50:08.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f901811b240 msgr2=0x7f901811b6a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:08.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f901811b240 0x7f901811b6a0 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:08.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f9018108360 osd=4 1 2024-02-23T00:50:08.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 10 client.41519.objecter close_session for osd.5 2024-02-23T00:50:08.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f9018121880 msgr2=0x7f9018121cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:08.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f9018121880 0x7f9018121cc0 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:08.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f9018123ad0 osd=5 1 2024-02-23T00:50:08.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 10 client.41519.objecter close_session for osd.6 2024-02-23T00:50:08.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f9018122940 msgr2=0x7f9018122da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:08.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f9018122940 0x7f9018122da0 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:08.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f90181222c0 osd=6 1 2024-02-23T00:50:08.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 10 client.41519.objecter close_session for osd.7 2024-02-23T00:50:08.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f90181281f0 msgr2=0x7f901812a5e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:08.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f90181281f0 0x7f901812a5e0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:08.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 20 client.41519.objecter put_session s=0x7f9018123430 osd=7 1 2024-02-23T00:50:08.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 20 client.41519.objecter shutdown clearing up homeless session... 2024-02-23T00:50:08.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 10 client.41519.objecter successfully canceled tick 2024-02-23T00:50:08.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f901e7c2700 1 -- 172.21.15.47:0/2147264050 reap_dead start 2024-02-23T00:50:08.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9004062cc0 msgr2=0x7f9004065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:08.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9004062cc0 0x7f9004065180 secure :-1 s=READY pgs=3095 cs=0 l=1 rev1=1 rx=0x7f9014009350 tx=0x7f901400b040).stop 2024-02-23T00:50:08.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f90180fcbb0 msgr2=0x7f9018109630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:08.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f90180fcbb0 0x7f9018109630 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 rx=0x7f900c009da0 tx=0x7f900c009cb0).stop 2024-02-23T00:50:08.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 shutdown_connections 2024-02-23T00:50:08.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f90180fcbb0 0x7f9018109630 unknown :-1 s=CLOSED pgs=2704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:08.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 1 --2- 172.21.15.47:0/2147264050 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9004062cc0 0x7f9004065180 unknown :-1 s=CLOSED pgs=3095 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:08.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.297+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 >> 172.21.15.47:0/2147264050 conn(0x7f90180f2820 msgr2=0x7f90180fa940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:08.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.298+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 shutdown_connections 2024-02-23T00:50:08.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.298+0000 7f9020225700 1 -- 172.21.15.47:0/2147264050 wait complete. 2024-02-23T00:50:08.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:08.298+0000 7f9020225700 1 librados: shutdown 2024-02-23T00:50:08.602 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:50:08.602 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:50:08.602 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:50:08.602 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:50:08.602 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:50:08.602 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:50:08.602 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:50:08.602 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:50:08.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:07.323073+0000 mgr.y (mgr.14517) 457 : cluster [DBG] pgmap v725: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:50:08.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:08.152729+0000 mon.c (mon.2) 312 : audit [DBG] from='client.? 172.21.15.47:0/2147264050' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:50:08.766 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:50:08.768 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:07.323073+0000 mgr.y (mgr.14517) 457 : cluster [DBG] pgmap v725: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:50:08.768 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:08.152729+0000 mon.c (mon.2) 312 : audit [DBG] from='client.? 172.21.15.47:0/2147264050' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:50:08.769 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:07.323073+0000 mgr.y (mgr.14517) 457 : cluster [DBG] pgmap v725: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:50:08.769 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:08.152729+0000 mon.c (mon.2) 312 : audit [DBG] from='client.? 172.21.15.47:0/2147264050' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:50:08.769 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:08.771 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:50:08.872 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:50:08.901 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:09.350 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:50:09.353 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:09.354 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:50:09.455 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:50:09.487 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:09.507 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:50:09.510 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:09.511 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:50:09.612 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:50:09.646 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:10.099 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:50:10.102 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:10.103 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:50:10.204 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:50:10.237 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:10.260 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:50:10.262 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:10.263 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:50:10.364 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:50:10.395 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:10.450 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:50:10.453 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:10.454 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:50:10.479 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:50:10.554 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:50:10.584 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:09.323708+0000 mgr.y (mgr.14517) 458 : cluster [DBG] pgmap v726: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-23T00:50:10.584 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:10.217091+0000 mon.a (mon.0) 2744 : cluster [DBG] osdmap e483: 8 total, 8 up, 7 in 2024-02-23T00:50:10.587 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:10.607 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:50:10.610 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:10.610 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:50:10.711 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:50:10.737 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:09.323708+0000 mgr.y (mgr.14517) 458 : cluster [DBG] pgmap v726: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-23T00:50:10.737 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:10.217091+0000 mon.a (mon.0) 2744 : cluster [DBG] osdmap e483: 8 total, 8 up, 7 in 2024-02-23T00:50:10.738 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:09.323708+0000 mgr.y (mgr.14517) 458 : cluster [DBG] pgmap v726: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-23T00:50:10.738 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:10.217091+0000 mon.a (mon.0) 2744 : cluster [DBG] osdmap e483: 8 total, 8 up, 7 in 2024-02-23T00:50:10.738 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:50:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:50:10] "GET /metrics HTTP/1.1" 200 189428 "" "Prometheus/2.33.4" 2024-02-23T00:50:10.739 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:50:10.814 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:10.814 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:50:10.915 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:50:10.943 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:11.092 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:50:11.095 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:11.095 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:50:11.196 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:50:11.224 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:11.369 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:50:11.472 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:50:11.476 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:11.476 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:50:11.577 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:50:11.604 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:11.624 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:50:11.635 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:11.635 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:50:11.736 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:50:11.769 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:11.790 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:50:11.793 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:11.794 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:50:11.895 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:50:11.924 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:12.123 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:50:12.127 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:12.128 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:50:12.229 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:50:12.259 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:12.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.470+0000 7f2bf47e5700 1 -- 172.21.15.47:0/2812432747 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2bec0fe510 msgr2=0x7f2bec0f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:12.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.470+0000 7f2bf47e5700 1 --2- 172.21.15.47:0/2812432747 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2bec0fe510 0x7f2bec0f7d40 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 rx=0x7f2bdc0047b0 tx=0x7f2bdc009cb0).stop 2024-02-23T00:50:12.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.470+0000 7f2bf47e5700 1 -- 172.21.15.47:0/2812432747 shutdown_connections 2024-02-23T00:50:12.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.470+0000 7f2bf47e5700 1 --2- 172.21.15.47:0/2812432747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bec0f8420 0x7f2bec104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:12.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.470+0000 7f2bf47e5700 1 --2- 172.21.15.47:0/2812432747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bec0fdbb0 0x7f2bec0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:12.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.470+0000 7f2bf47e5700 1 --2- 172.21.15.47:0/2812432747 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2bec0fe510 0x7f2bec0f7d40 unknown :-1 s=CLOSED pgs=2705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:12.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.470+0000 7f2bf47e5700 1 -- 172.21.15.47:0/2812432747 >> 172.21.15.47:0/2812432747 conn(0x7f2bec0f3800 msgr2=0x7f2bec0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:12.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.470+0000 7f2bf47e5700 1 -- 172.21.15.47:0/2812432747 shutdown_connections 2024-02-23T00:50:12.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.470+0000 7f2bf47e5700 1 -- 172.21.15.47:0/2812432747 wait complete. 2024-02-23T00:50:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.470+0000 7f2bf47e5700 1 librados: starting msgr at 2024-02-23T00:50:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.470+0000 7f2bf47e5700 1 librados: starting objecter 2024-02-23T00:50:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.471+0000 7f2bf47e5700 1 Processor -- start 2024-02-23T00:50:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.471+0000 7f2bf47e5700 1 -- start start 2024-02-23T00:50:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.471+0000 7f2bf47e5700 1 librados: setting wanted keys 2024-02-23T00:50:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.471+0000 7f2bf47e5700 1 librados: calling monclient init 2024-02-23T00:50:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.471+0000 7f2bf47e5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bec0f8420 0x7f2bec10de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.471+0000 7f2bf47e5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bec0fdbb0 0x7f2bec10e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.471+0000 7f2bf47e5700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2bec0fe510 0x7f2bec108020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.471+0000 7f2bf47e5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec108510 con 0x7f2bec0fdbb0 2024-02-23T00:50:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.471+0000 7f2bf47e5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec108680 con 0x7f2bec0f8420 2024-02-23T00:50:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.471+0000 7f2bf47e5700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec108840 con 0x7f2bec0fe510 2024-02-23T00:50:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.471+0000 7f2bf2d82700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2bec0fe510 0x7f2bec108020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:12.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.471+0000 7f2bf1d80700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bec0fdbb0 0x7f2bec10e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:12.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.471+0000 7f2bf1d80700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bec0fdbb0 0x7f2bec10e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33248/0 (socket says 172.21.15.47:33248) 2024-02-23T00:50:12.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.471+0000 7f2bf2d82700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2bec0fe510 0x7f2bec108020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:34172/0 (socket says 172.21.15.47:34172) 2024-02-23T00:50:12.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.471+0000 7f2bf1d80700 1 -- 172.21.15.47:0/1320095870 learned_addr learned my addr 172.21.15.47:0/1320095870 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:50:12.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.471+0000 7f2bf2581700 1 --2- 172.21.15.47:0/1320095870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bec0f8420 0x7f2bec10de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:12.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.471+0000 7f2bf2d82700 1 -- 172.21.15.47:0/1320095870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bec0f8420 msgr2=0x7f2bec10de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:12.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.471+0000 7f2bf2d82700 1 --2- 172.21.15.47:0/1320095870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bec0f8420 0x7f2bec10de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:12.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.471+0000 7f2bf2d82700 1 -- 172.21.15.47:0/1320095870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bec0fdbb0 msgr2=0x7f2bec10e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:12.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.471+0000 7f2bf2d82700 1 --2- 172.21.15.47:0/1320095870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bec0fdbb0 0x7f2bec10e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:12.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.471+0000 7f2bf2d82700 1 -- 172.21.15.47:0/1320095870 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be40097c0 con 0x7f2bec0fe510 2024-02-23T00:50:12.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.471+0000 7f2bf2581700 1 --2- 172.21.15.47:0/1320095870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bec0f8420 0x7f2bec10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:50:12.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.471+0000 7f2bf1d80700 1 --2- 172.21.15.47:0/1320095870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bec0fdbb0 0x7f2bec10e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:50:12.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.471+0000 7f2bf2d82700 1 --2- 172.21.15.47:0/1320095870 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2bec0fe510 0x7f2bec108020 secure :-1 s=READY pgs=2706 cs=0 l=1 rev1=1 rx=0x7f2be4001df0 tx=0x7f2be4004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.471+0000 7f2be37fe700 10 client.?.objecter ms_handle_connect 0x7f2bec0fe510 2024-02-23T00:50:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.471+0000 7f2be37fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:50:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.472+0000 7f2be37fe700 1 -- 172.21.15.47:0/1320095870 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be40099f0 con 0x7f2bec0fe510 2024-02-23T00:50:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.472+0000 7f2be37fe700 1 -- 172.21.15.47:0/1320095870 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f2be40090d0 con 0x7f2bec0fe510 2024-02-23T00:50:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.472+0000 7f2bf47e5700 1 -- 172.21.15.47:0/1320095870 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bdc005040 con 0x7f2bec0fe510 2024-02-23T00:50:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.472+0000 7f2be37fe700 1 -- 172.21.15.47:0/1320095870 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be4007c20 con 0x7f2bec0fe510 2024-02-23T00:50:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.472+0000 7f2bf47e5700 10 client.41525.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:50:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.472+0000 7f2bf47e5700 1 -- 172.21.15.47:0/1320095870 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bec108e80 con 0x7f2bec0fe510 2024-02-23T00:50:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.472+0000 7f2bf47e5700 1 librados: init done 2024-02-23T00:50:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.473+0000 7f2bf47e5700 1 -- 172.21.15.47:0/1320095870 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bec05ee80 con 0x7f2bec0fe510 2024-02-23T00:50:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.473+0000 7f2be37fe700 1 -- 172.21.15.47:0/1320095870 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f2be400fb80 con 0x7f2bec0fe510 2024-02-23T00:50:12.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.473+0000 7f2be37fe700 1 --2- 172.21.15.47:0/1320095870 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2bd8062cc0 0x7f2bd8065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:12.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.473+0000 7f2be37fe700 1 -- 172.21.15.47:0/1320095870 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(483..483 src has 48..483) v4 ==== 9220+0+0 (secure 0 0 0) 0x7f2be407dee0 con 0x7f2bec0fe510 2024-02-23T00:50:12.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.473+0000 7f2be37fe700 10 client.41525.objecter ms_dispatch 0x7f2bec001550 osd_map(483..483 src has 48..483) v4 2024-02-23T00:50:12.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.473+0000 7f2be37fe700 3 client.41525.objecter handle_osd_map got epochs [483,483] > 0 2024-02-23T00:50:12.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.473+0000 7f2be37fe700 3 client.41525.objecter handle_osd_map decoding full epoch 483 2024-02-23T00:50:12.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.473+0000 7f2bf2581700 1 --2- 172.21.15.47:0/1320095870 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2bd8062cc0 0x7f2bd8065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:12.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.473+0000 7f2be37fe700 20 client.41525.objecter dump_active .. 0 homeless 2024-02-23T00:50:12.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.474+0000 7f2be37fe700 10 client.41525.objecter ms_handle_connect 0x7f2bd8062cc0 2024-02-23T00:50:12.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.474+0000 7f2bf2581700 1 --2- 172.21.15.47:0/1320095870 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2bd8062cc0 0x7f2bd8065180 secure :-1 s=READY pgs=3096 cs=0 l=1 rev1=1 rx=0x7f2be8009250 tx=0x7f2be8009de0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:12.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.478+0000 7f2be37fe700 1 -- 172.21.15.47:0/1320095870 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f2be404d570 con 0x7f2bec0fe510 2024-02-23T00:50:12.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:11.324027+0000 mgr.y (mgr.14517) 459 : cluster [DBG] pgmap v728: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:50:12.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:11.324027+0000 mgr.y (mgr.14517) 459 : cluster [DBG] pgmap v728: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:50:12.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:12.659+0000 7f2bf47e5700 1 -- 172.21.15.47:0/1320095870 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7f2bec05fa60 con 0x7f2bec0fe510 2024-02-23T00:50:12.690 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:50:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:50:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:50:12.690 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:11.324027+0000 mgr.y (mgr.14517) 459 : cluster [DBG] pgmap v728: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:50:12.943 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:50:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:50:13.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:13.409+0000 7f2be37fe700 1 -- 172.21.15.47:0/1320095870 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v484) v1 ==== 91+0+0 (secure 0 0 0) 0x7f2be4050b90 con 0x7f2bec0fe510 2024-02-23T00:50:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:13.411+0000 7f2bf47e5700 10 librados: watch_flush enter 2024-02-23T00:50:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:13.411+0000 7f2bf47e5700 10 librados: watch_flush exit 2024-02-23T00:50:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:13.411+0000 7f2bf47e5700 20 client.41525.objecter shutdown clearing up homeless session... 2024-02-23T00:50:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:13.411+0000 7f2bf47e5700 10 client.41525.objecter successfully canceled tick 2024-02-23T00:50:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:13.411+0000 7f2bf47e5700 1 -- 172.21.15.47:0/1320095870 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2bd8062cc0 msgr2=0x7f2bd8065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:13.411+0000 7f2bf47e5700 1 --2- 172.21.15.47:0/1320095870 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2bd8062cc0 0x7f2bd8065180 secure :-1 s=READY pgs=3096 cs=0 l=1 rev1=1 rx=0x7f2be8009250 tx=0x7f2be8009de0).stop 2024-02-23T00:50:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:13.411+0000 7f2bf47e5700 1 -- 172.21.15.47:0/1320095870 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2bec0fe510 msgr2=0x7f2bec108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:13.411+0000 7f2bf47e5700 1 --2- 172.21.15.47:0/1320095870 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2bec0fe510 0x7f2bec108020 secure :-1 s=READY pgs=2706 cs=0 l=1 rev1=1 rx=0x7f2be4001df0 tx=0x7f2be4004d10).stop 2024-02-23T00:50:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:13.412+0000 7f2bf47e5700 1 -- 172.21.15.47:0/1320095870 shutdown_connections 2024-02-23T00:50:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:13.412+0000 7f2bf47e5700 1 --2- 172.21.15.47:0/1320095870 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2bd8062cc0 0x7f2bd8065180 unknown :-1 s=CLOSED pgs=3096 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:13.412+0000 7f2bf47e5700 1 --2- 172.21.15.47:0/1320095870 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2bec0fe510 0x7f2bec108020 unknown :-1 s=CLOSED pgs=2706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:13.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:13.412+0000 7f2bf47e5700 1 --2- 172.21.15.47:0/1320095870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bec0f8420 0x7f2bec10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:13.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:13.412+0000 7f2bf47e5700 1 --2- 172.21.15.47:0/1320095870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bec0fdbb0 0x7f2bec10e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:13.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:13.412+0000 7f2bf47e5700 1 -- 172.21.15.47:0/1320095870 >> 172.21.15.47:0/1320095870 conn(0x7f2bec0f3800 msgr2=0x7f2bec0fb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:13.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:13.412+0000 7f2bf47e5700 1 -- 172.21.15.47:0/1320095870 shutdown_connections 2024-02-23T00:50:13.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:13.412+0000 7f2bf47e5700 1 -- 172.21.15.47:0/1320095870 wait complete. 2024-02-23T00:50:13.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:13.412+0000 7f2bf47e5700 1 librados: shutdown 2024-02-23T00:50:13.416 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is unset 2024-02-23T00:50:13.684 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:50:13.684 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:50:13.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:12.660968+0000 mon.c (mon.2) 313 : audit [INF] from='client.? 172.21.15.47:0/1320095870' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:50:13.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:12.661314+0000 mon.a (mon.0) 2745 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:50:13.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:12.688074+0000 mgr.y (mgr.14517) 460 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:50:13.709 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:12.660968+0000 mon.c (mon.2) 313 : audit [INF] from='client.? 172.21.15.47:0/1320095870' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:50:13.709 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:12.661314+0000 mon.a (mon.0) 2745 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:50:13.709 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:12.688074+0000 mgr.y (mgr.14517) 460 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:50:13.710 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:12.660968+0000 mon.c (mon.2) 313 : audit [INF] from='client.? 172.21.15.47:0/1320095870' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:50:13.710 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:12.661314+0000 mon.a (mon.0) 2745 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:50:13.710 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:12.688074+0000 mgr.y (mgr.14517) 460 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:50:14.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:13.324861+0000 mgr.y (mgr.14517) 461 : cluster [DBG] pgmap v729: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:50:14.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:13.400504+0000 mon.a (mon.0) 2746 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:50:14.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:13.407793+0000 mon.a (mon.0) 2747 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:50:14.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:13.407859+0000 mon.a (mon.0) 2748 : cluster [DBG] osdmap e484: 8 total, 8 up, 7 in 2024-02-23T00:50:14.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:13.548553+0000 osd.4 (osd.4) 209 : cluster [DBG] 18.12 scrub starts 2024-02-23T00:50:14.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:13.549525+0000 osd.4 (osd.4) 210 : cluster [DBG] 18.12 scrub ok 2024-02-23T00:50:14.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:13.641383+0000 osd.0 (osd.0) 207 : cluster [DBG] 18.0 scrub starts 2024-02-23T00:50:14.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:13.642355+0000 osd.0 (osd.0) 208 : cluster [DBG] 18.0 scrub ok 2024-02-23T00:50:14.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:13.859748+0000 osd.3 (osd.3) 228 : cluster [DBG] 4.1d scrub starts 2024-02-23T00:50:14.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:13.861216+0000 osd.3 (osd.3) 229 : cluster [DBG] 4.1d scrub ok 2024-02-23T00:50:14.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:13.324861+0000 mgr.y (mgr.14517) 461 : cluster [DBG] pgmap v729: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:50:14.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:13.400504+0000 mon.a (mon.0) 2746 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:50:14.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:13.407793+0000 mon.a (mon.0) 2747 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:50:14.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:13.407859+0000 mon.a (mon.0) 2748 : cluster [DBG] osdmap e484: 8 total, 8 up, 7 in 2024-02-23T00:50:14.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:13.548553+0000 osd.4 (osd.4) 209 : cluster [DBG] 18.12 scrub starts 2024-02-23T00:50:14.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:13.549525+0000 osd.4 (osd.4) 210 : cluster [DBG] 18.12 scrub ok 2024-02-23T00:50:14.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:13.641383+0000 osd.0 (osd.0) 207 : cluster [DBG] 18.0 scrub starts 2024-02-23T00:50:14.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:13.642355+0000 osd.0 (osd.0) 208 : cluster [DBG] 18.0 scrub ok 2024-02-23T00:50:14.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:13.859748+0000 osd.3 (osd.3) 228 : cluster [DBG] 4.1d scrub starts 2024-02-23T00:50:14.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:13.861216+0000 osd.3 (osd.3) 229 : cluster [DBG] 4.1d scrub ok 2024-02-23T00:50:14.610 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:50:14.615 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:14.616 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:50:14.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:13.324861+0000 mgr.y (mgr.14517) 461 : cluster [DBG] pgmap v729: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:50:14.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:13.400504+0000 mon.a (mon.0) 2746 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:50:14.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:13.407793+0000 mon.a (mon.0) 2747 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:50:14.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:13.407859+0000 mon.a (mon.0) 2748 : cluster [DBG] osdmap e484: 8 total, 8 up, 7 in 2024-02-23T00:50:14.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:13.548553+0000 osd.4 (osd.4) 209 : cluster [DBG] 18.12 scrub starts 2024-02-23T00:50:14.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:13.549525+0000 osd.4 (osd.4) 210 : cluster [DBG] 18.12 scrub ok 2024-02-23T00:50:14.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:13.641383+0000 osd.0 (osd.0) 207 : cluster [DBG] 18.0 scrub starts 2024-02-23T00:50:14.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:13.642355+0000 osd.0 (osd.0) 208 : cluster [DBG] 18.0 scrub ok 2024-02-23T00:50:14.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:13.859748+0000 osd.3 (osd.3) 228 : cluster [DBG] 4.1d scrub starts 2024-02-23T00:50:14.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:13.861216+0000 osd.3 (osd.3) 229 : cluster [DBG] 4.1d scrub ok 2024-02-23T00:50:14.717 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:50:14.751 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:14.903 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:50:14.906 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:14.906 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:50:15.007 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:50:15.038 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:15.112 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:50:15.115 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:15.115 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:50:15.216 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:50:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.226+0000 7fe7710eb700 1 -- 172.21.15.47:0/3059378280 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe76c0f9570 msgr2=0x7fe76c101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.226+0000 7fe7710eb700 1 --2- 172.21.15.47:0/3059378280 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe76c0f9570 0x7fe76c101c80 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 rx=0x7fe760004870 tx=0x7fe760009cb0).stop 2024-02-23T00:50:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.226+0000 7fe7710eb700 1 -- 172.21.15.47:0/3059378280 shutdown_connections 2024-02-23T00:50:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.226+0000 7fe7710eb700 1 --2- 172.21.15.47:0/3059378280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe76c102170 0x7fe76c104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.226+0000 7fe7710eb700 1 --2- 172.21.15.47:0/3059378280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe76c0f8c10 0x7fe76c0f8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.226+0000 7fe7710eb700 1 --2- 172.21.15.47:0/3059378280 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe76c0f9570 0x7fe76c101c80 unknown :-1 s=CLOSED pgs=2707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.226+0000 7fe7710eb700 1 -- 172.21.15.47:0/3059378280 >> 172.21.15.47:0/3059378280 conn(0x7fe76c0f26e0 msgr2=0x7fe76c0f4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.226+0000 7fe7710eb700 1 -- 172.21.15.47:0/3059378280 shutdown_connections 2024-02-23T00:50:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.226+0000 7fe7710eb700 1 -- 172.21.15.47:0/3059378280 wait complete. 2024-02-23T00:50:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.227+0000 7fe7710eb700 1 librados: starting msgr at 2024-02-23T00:50:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.227+0000 7fe7710eb700 1 librados: starting objecter 2024-02-23T00:50:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.227+0000 7fe7710eb700 1 Processor -- start 2024-02-23T00:50:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.227+0000 7fe7710eb700 1 -- start start 2024-02-23T00:50:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.227+0000 7fe7710eb700 1 librados: setting wanted keys 2024-02-23T00:50:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.227+0000 7fe7710eb700 1 librados: calling monclient init 2024-02-23T00:50:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.227+0000 7fe7710eb700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe76c0f8c10 0x7fe76c0f6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.227+0000 7fe7710eb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe76c0f9570 0x7fe76c0f70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.227+0000 7fe7710eb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe76c102170 0x7fe76c0f75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.227+0000 7fe7710eb700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76c116500 con 0x7fe76c0f9570 2024-02-23T00:50:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.227+0000 7fe7710eb700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76c116670 con 0x7fe76c102170 2024-02-23T00:50:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.227+0000 7fe7710eb700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe76c116830 con 0x7fe76c0f8c10 2024-02-23T00:50:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.228+0000 7fe76ad9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe76c0f8c10 0x7fe76c0f6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.228+0000 7fe76ad9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe76c0f8c10 0x7fe76c0f6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:34182/0 (socket says 172.21.15.47:34182) 2024-02-23T00:50:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.228+0000 7fe76a59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe76c0f9570 0x7fe76c0f70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.228+0000 7fe76ad9d700 1 -- 172.21.15.47:0/4050586715 learned_addr learned my addr 172.21.15.47:0/4050586715 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:50:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.228+0000 7fe76b59e700 1 --2- 172.21.15.47:0/4050586715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe76c102170 0x7fe76c0f75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.228+0000 7fe76ad9d700 1 -- 172.21.15.47:0/4050586715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe76c102170 msgr2=0x7fe76c0f75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.228+0000 7fe76ad9d700 1 --2- 172.21.15.47:0/4050586715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe76c102170 0x7fe76c0f75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.228+0000 7fe76ad9d700 1 -- 172.21.15.47:0/4050586715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe76c0f9570 msgr2=0x7fe76c0f70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.228+0000 7fe76ad9d700 1 --2- 172.21.15.47:0/4050586715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe76c0f9570 0x7fe76c0f70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.228+0000 7fe76ad9d700 1 -- 172.21.15.47:0/4050586715 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7640097c0 con 0x7fe76c0f8c10 2024-02-23T00:50:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.228+0000 7fe76a59c700 1 --2- 172.21.15.47:0/4050586715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe76c0f9570 0x7fe76c0f70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:50:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.228+0000 7fe76b59e700 1 --2- 172.21.15.47:0/4050586715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe76c102170 0x7fe76c0f75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:50:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.228+0000 7fe76ad9d700 1 --2- 172.21.15.47:0/4050586715 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe76c0f8c10 0x7fe76c0f6bc0 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 rx=0x7fe75c00e990 tx=0x7fe75c00b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.228+0000 7fe753fff700 10 client.?.objecter ms_handle_connect 0x7fe76c0f8c10 2024-02-23T00:50:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.228+0000 7fe753fff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:50:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.228+0000 7fe753fff700 1 -- 172.21.15.47:0/4050586715 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c00ee80 con 0x7fe76c0f8c10 2024-02-23T00:50:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.228+0000 7fe7710eb700 1 -- 172.21.15.47:0/4050586715 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe760005040 con 0x7fe76c0f8c10 2024-02-23T00:50:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.228+0000 7fe753fff700 1 -- 172.21.15.47:0/4050586715 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fe75c00f040 con 0x7fe76c0f8c10 2024-02-23T00:50:15.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.228+0000 7fe753fff700 1 -- 172.21.15.47:0/4050586715 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c00c840 con 0x7fe76c0f8c10 2024-02-23T00:50:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.228+0000 7fe7710eb700 10 client.41531.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:50:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.228+0000 7fe7710eb700 1 -- 172.21.15.47:0/4050586715 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe76c116de0 con 0x7fe76c0f8c10 2024-02-23T00:50:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.228+0000 7fe7710eb700 1 librados: init done 2024-02-23T00:50:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.229+0000 7fe7710eb700 1 -- 172.21.15.47:0/4050586715 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe76c05f950 con 0x7fe76c0f8c10 2024-02-23T00:50:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.230+0000 7fe753fff700 1 -- 172.21.15.47:0/4050586715 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fe75c00cc70 con 0x7fe76c0f8c10 2024-02-23T00:50:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.230+0000 7fe753fff700 1 --2- 172.21.15.47:0/4050586715 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe754062cc0 0x7fe754065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.230+0000 7fe753fff700 1 -- 172.21.15.47:0/4050586715 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(485..485 src has 48..485) v4 ==== 9220+0+0 (secure 0 0 0) 0x7fe75c07d2a0 con 0x7fe76c0f8c10 2024-02-23T00:50:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.230+0000 7fe753fff700 10 client.41531.objecter ms_dispatch 0x7fe76c001550 osd_map(485..485 src has 48..485) v4 2024-02-23T00:50:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.230+0000 7fe753fff700 3 client.41531.objecter handle_osd_map got epochs [485,485] > 0 2024-02-23T00:50:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.230+0000 7fe753fff700 3 client.41531.objecter handle_osd_map decoding full epoch 485 2024-02-23T00:50:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.231+0000 7fe76a59c700 1 --2- 172.21.15.47:0/4050586715 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe754062cc0 0x7fe754065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.231+0000 7fe753fff700 20 client.41531.objecter dump_active .. 0 homeless 2024-02-23T00:50:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.231+0000 7fe753fff700 10 client.41531.objecter ms_handle_connect 0x7fe754062cc0 2024-02-23T00:50:15.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.231+0000 7fe76a59c700 1 --2- 172.21.15.47:0/4050586715 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe754062cc0 0x7fe754065180 secure :-1 s=READY pgs=3097 cs=0 l=1 rev1=1 rx=0x7fe760004ee0 tx=0x7fe76000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:15.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.235+0000 7fe753fff700 1 -- 172.21.15.47:0/4050586715 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe75c049f60 con 0x7fe76c0f8c10 2024-02-23T00:50:15.248 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:15.269 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:50:15.272 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:15.272 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:50:15.373 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:50:15.401 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:15.435 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:50:15.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:15.434+0000 7fe7710eb700 1 -- 172.21.15.47:0/4050586715 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd reweight", "id": 1, "weight": 1.0} v 0) v1 -- 0x7fe76c0f80a0 con 0x7fe76c0f8c10 2024-02-23T00:50:15.439 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:15.439 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:50:15.540 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:50:15.568 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:15.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:14.319298+0000 osd.1 (osd.1) 217 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:50:15.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:14.319442+0000 osd.1 (osd.1) 218 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:50:15.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:14.411929+0000 mon.a (mon.0) 2749 : cluster [DBG] osdmap e485: 8 total, 8 up, 7 in 2024-02-23T00:50:15.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:14.631765+0000 osd.3 (osd.3) 230 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:50:15.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:14.631880+0000 osd.3 (osd.3) 231 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:50:15.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:14.800787+0000 osd.5 (osd.5) 214 : cluster [DBG] 6.a scrub starts 2024-02-23T00:50:15.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:14.801908+0000 osd.5 (osd.5) 215 : cluster [DBG] 6.a scrub ok 2024-02-23T00:50:15.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:14.319298+0000 osd.1 (osd.1) 217 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:50:15.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:14.319442+0000 osd.1 (osd.1) 218 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:50:15.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:14.411929+0000 mon.a (mon.0) 2749 : cluster [DBG] osdmap e485: 8 total, 8 up, 7 in 2024-02-23T00:50:15.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:14.631765+0000 osd.3 (osd.3) 230 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:50:15.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:14.631880+0000 osd.3 (osd.3) 231 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:50:15.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:14.800787+0000 osd.5 (osd.5) 214 : cluster [DBG] 6.a scrub starts 2024-02-23T00:50:15.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:14.801908+0000 osd.5 (osd.5) 215 : cluster [DBG] 6.a scrub ok 2024-02-23T00:50:15.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:14.319298+0000 osd.1 (osd.1) 217 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:50:15.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:14.319442+0000 osd.1 (osd.1) 218 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:50:15.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:14.411929+0000 mon.a (mon.0) 2749 : cluster [DBG] osdmap e485: 8 total, 8 up, 7 in 2024-02-23T00:50:15.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:14.631765+0000 osd.3 (osd.3) 230 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:50:15.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:14.631880+0000 osd.3 (osd.3) 231 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:50:15.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:14.800787+0000 osd.5 (osd.5) 214 : cluster [DBG] 6.a scrub starts 2024-02-23T00:50:15.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:14.801908+0000 osd.5 (osd.5) 215 : cluster [DBG] 6.a scrub ok 2024-02-23T00:50:15.896 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.1 dump_blocked_ops 2024-02-23T00:50:15.916 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:50:15.919 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:15.919 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:50:15.968 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:50:16.021 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:50:16.053 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:16.074 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:50:16.076 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:16.077 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:50:16.177 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:50:16.207 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:16.228 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:50:16.230 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:16.231 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:50:16.332 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:50:16.360 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:16.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:16.425+0000 7fe753fff700 1 -- 172.21.15.47:0/4050586715 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 1, "weight": 1.0}]=0 reweighted osd.1 to 1 (10000) v487) v1 ==== 113+0+0 (secure 0 0 0) 0x7fe75c04c8b0 con 0x7fe76c0f8c10 2024-02-23T00:50:16.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:16.428+0000 7fe7710eb700 10 librados: watch_flush enter 2024-02-23T00:50:16.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:16.428+0000 7fe7710eb700 10 librados: watch_flush exit 2024-02-23T00:50:16.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:16.428+0000 7fe7710eb700 20 client.41531.objecter shutdown clearing up homeless session... 2024-02-23T00:50:16.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:16.428+0000 7fe7710eb700 10 client.41531.objecter successfully canceled tick 2024-02-23T00:50:16.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:16.428+0000 7fe7710eb700 1 -- 172.21.15.47:0/4050586715 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe754062cc0 msgr2=0x7fe754065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:16.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:16.428+0000 7fe7710eb700 1 --2- 172.21.15.47:0/4050586715 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe754062cc0 0x7fe754065180 secure :-1 s=READY pgs=3097 cs=0 l=1 rev1=1 rx=0x7fe760004ee0 tx=0x7fe76000b040).stop 2024-02-23T00:50:16.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:16.429+0000 7fe7710eb700 1 -- 172.21.15.47:0/4050586715 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe76c0f8c10 msgr2=0x7fe76c0f6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:16.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:16.429+0000 7fe7710eb700 1 --2- 172.21.15.47:0/4050586715 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe76c0f8c10 0x7fe76c0f6bc0 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 rx=0x7fe75c00e990 tx=0x7fe75c00b040).stop 2024-02-23T00:50:16.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:16.429+0000 7fe7710eb700 1 -- 172.21.15.47:0/4050586715 shutdown_connections 2024-02-23T00:50:16.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:16.429+0000 7fe7710eb700 1 --2- 172.21.15.47:0/4050586715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe76c102170 0x7fe76c0f75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:16.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:16.429+0000 7fe7710eb700 1 --2- 172.21.15.47:0/4050586715 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe754062cc0 0x7fe754065180 unknown :-1 s=CLOSED pgs=3097 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:16.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:16.429+0000 7fe7710eb700 1 --2- 172.21.15.47:0/4050586715 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe76c0f8c10 0x7fe76c0f6bc0 unknown :-1 s=CLOSED pgs=2708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:16.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:16.429+0000 7fe7710eb700 1 --2- 172.21.15.47:0/4050586715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe76c0f9570 0x7fe76c0f70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:16.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:16.429+0000 7fe7710eb700 1 -- 172.21.15.47:0/4050586715 >> 172.21.15.47:0/4050586715 conn(0x7fe76c0f26e0 msgr2=0x7fe76c0f3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:16.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:16.429+0000 7fe7710eb700 1 -- 172.21.15.47:0/4050586715 shutdown_connections 2024-02-23T00:50:16.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:16.429+0000 7fe7710eb700 1 -- 172.21.15.47:0/4050586715 wait complete. 2024-02-23T00:50:16.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:16.429+0000 7fe7710eb700 1 librados: shutdown 2024-02-23T00:50:16.433 INFO:teuthology.orchestra.run.smithi047.stderr:reweighted osd.1 to 1 (10000) 2024-02-23T00:50:16.443 INFO:journalctl@ceph.osd.4.smithi196.stdout:Feb 23 00:50:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-4[43838]: debug 2024-02-23T00:50:16.285+0000 7f1aa2901700 -1 snap_mapper.add_oid found existing snaps mapped on 219:4d905415:test-rados-api-smithi047-82465-76::foo:2, removing 2024-02-23T00:50:16.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:50:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:50:16.285+0000 7f189573a700 -1 snap_mapper.add_oid found existing snaps mapped on 219:4d905415:test-rados-api-smithi047-82465-76::foo:2, removing 2024-02-23T00:50:16.488 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:50:16.492 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:16.492 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:50:16.515 INFO:journalctl@ceph.osd.3.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-3[62151]: debug 2024-02-23T00:50:16.284+0000 7f77f5dd5700 -1 snap_mapper.add_oid found existing snaps mapped on 219:4d905415:test-rados-api-smithi047-82465-76::foo:2, removing 2024-02-23T00:50:16.593 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:50:16.625 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:16.647 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-23T00:50:16.749 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:50:16.753 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:16.753 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:50:16.854 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:50:16.884 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:16.904 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:50:16.906 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:16.907 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:50:17.008 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:50:17.035 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:17.054 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:50:17.057 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:17.057 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:50:17.158 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:50:17.183 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:14.605884+0000 osd.2 (osd.2) 163 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:50:17.184 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:14.606186+0000 osd.2 (osd.2) 164 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:50:17.184 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:15.103044+0000 osd.0 (osd.0) 209 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:50:17.184 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:15.103298+0000 osd.0 (osd.0) 210 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:50:17.184 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:15.325227+0000 mgr.y (mgr.14517) 462 : cluster [DBG] pgmap v732: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:50:17.184 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:15.418529+0000 mon.a (mon.0) 2750 : cluster [DBG] osdmap e486: 8 total, 8 up, 7 in 2024-02-23T00:50:17.184 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:15.436292+0000 mon.c (mon.2) 314 : audit [INF] from='client.? 172.21.15.47:0/4050586715' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-23T00:50:17.184 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:15.436616+0000 mon.a (mon.0) 2751 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-23T00:50:17.184 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:15.588414+0000 osd.4 (osd.4) 211 : cluster [DBG] 4.1b scrub starts 2024-02-23T00:50:17.184 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:15.590087+0000 osd.4 (osd.4) 212 : cluster [DBG] 4.1b scrub ok 2024-02-23T00:50:17.184 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:15.711192+0000 osd.1 (osd.1) 219 : cluster [DBG] 18.3 scrub starts 2024-02-23T00:50:17.184 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:15.714197+0000 osd.1 (osd.1) 220 : cluster [DBG] 18.3 scrub ok 2024-02-23T00:50:17.184 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:15.921626+0000 osd.3 (osd.3) 232 : cluster [DBG] 6.b scrub starts 2024-02-23T00:50:17.184 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:15.922741+0000 osd.3 (osd.3) 233 : cluster [DBG] 6.b scrub ok 2024-02-23T00:50:17.185 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:16.287067+0000 mon.a (mon.0) 2752 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:50:17.185 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:16.287474+0000 mon.a (mon.0) 2753 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-75"}]: dispatch 2024-02-23T00:50:17.185 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:17.204 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:50:17.207 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:17.207 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:50:17.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:14.605884+0000 osd.2 (osd.2) 163 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:50:17.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:14.606186+0000 osd.2 (osd.2) 164 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:50:17.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:15.103044+0000 osd.0 (osd.0) 209 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:50:17.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:15.103298+0000 osd.0 (osd.0) 210 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:50:17.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:15.325227+0000 mgr.y (mgr.14517) 462 : cluster [DBG] pgmap v732: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:50:17.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:15.418529+0000 mon.a (mon.0) 2750 : cluster [DBG] osdmap e486: 8 total, 8 up, 7 in 2024-02-23T00:50:17.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:15.436292+0000 mon.c (mon.2) 314 : audit [INF] from='client.? 172.21.15.47:0/4050586715' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-23T00:50:17.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:15.436616+0000 mon.a (mon.0) 2751 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-23T00:50:17.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:15.588414+0000 osd.4 (osd.4) 211 : cluster [DBG] 4.1b scrub starts 2024-02-23T00:50:17.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:15.590087+0000 osd.4 (osd.4) 212 : cluster [DBG] 4.1b scrub ok 2024-02-23T00:50:17.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:15.711192+0000 osd.1 (osd.1) 219 : cluster [DBG] 18.3 scrub starts 2024-02-23T00:50:17.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:15.714197+0000 osd.1 (osd.1) 220 : cluster [DBG] 18.3 scrub ok 2024-02-23T00:50:17.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:15.921626+0000 osd.3 (osd.3) 232 : cluster [DBG] 6.b scrub starts 2024-02-23T00:50:17.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:15.922741+0000 osd.3 (osd.3) 233 : cluster [DBG] 6.b scrub ok 2024-02-23T00:50:17.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:16.287067+0000 mon.a (mon.0) 2752 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:50:17.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:16.287474+0000 mon.a (mon.0) 2753 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-75"}]: dispatch 2024-02-23T00:50:17.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:14.605884+0000 osd.2 (osd.2) 163 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:50:17.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:14.606186+0000 osd.2 (osd.2) 164 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:50:17.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:15.103044+0000 osd.0 (osd.0) 209 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:50:17.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:15.103298+0000 osd.0 (osd.0) 210 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:50:17.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:15.325227+0000 mgr.y (mgr.14517) 462 : cluster [DBG] pgmap v732: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:50:17.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:15.418529+0000 mon.a (mon.0) 2750 : cluster [DBG] osdmap e486: 8 total, 8 up, 7 in 2024-02-23T00:50:17.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:15.436292+0000 mon.c (mon.2) 314 : audit [INF] from='client.? 172.21.15.47:0/4050586715' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-23T00:50:17.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:15.436616+0000 mon.a (mon.0) 2751 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-23T00:50:17.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:15.588414+0000 osd.4 (osd.4) 211 : cluster [DBG] 4.1b scrub starts 2024-02-23T00:50:17.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:15.590087+0000 osd.4 (osd.4) 212 : cluster [DBG] 4.1b scrub ok 2024-02-23T00:50:17.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:15.711192+0000 osd.1 (osd.1) 219 : cluster [DBG] 18.3 scrub starts 2024-02-23T00:50:17.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:15.714197+0000 osd.1 (osd.1) 220 : cluster [DBG] 18.3 scrub ok 2024-02-23T00:50:17.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:15.921626+0000 osd.3 (osd.3) 232 : cluster [DBG] 6.b scrub starts 2024-02-23T00:50:17.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:15.922741+0000 osd.3 (osd.3) 233 : cluster [DBG] 6.b scrub ok 2024-02-23T00:50:17.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:16.287067+0000 mon.a (mon.0) 2752 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:50:17.310 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:16.287474+0000 mon.a (mon.0) 2753 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-75"}]: dispatch 2024-02-23T00:50:17.310 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:50:17.337 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:17.355 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:50:17.358 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:17.358 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:50:17.374 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight 3 1 2024-02-23T00:50:17.459 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:50:17.485 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:17.503 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:50:17.506 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:17.506 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:50:17.607 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:50:17.634 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:17.757 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:50:17.760 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:17.760 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:50:17.861 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:50:17.889 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:16.254998+0000 osd.7 (osd.7) 245 : cluster [DBG] 18.13 scrub starts 2024-02-23T00:50:17.889 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:16.257892+0000 osd.7 (osd.7) 246 : cluster [DBG] 18.13 scrub ok 2024-02-23T00:50:17.889 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:16.420157+0000 mon.a (mon.0) 2754 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-23T00:50:17.889 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:16.420258+0000 mon.a (mon.0) 2755 : cluster [DBG] osdmap e487: 8 total, 8 up, 7 in 2024-02-23T00:50:17.889 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:16.439049+0000 osd.6 (osd.6) 195 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:50:17.889 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:16.439200+0000 osd.6 (osd.6) 196 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:50:17.889 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:17.326039+0000 mgr.y (mgr.14517) 463 : cluster [DBG] pgmap v735: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-02-23T00:50:17.890 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:17.422475+0000 mon.a (mon.0) 2756 : cluster [DBG] osdmap e488: 8 total, 8 up, 7 in 2024-02-23T00:50:17.890 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:17.430429+0000 mon.a (mon.0) 2757 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:50:17.890 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:17.585936+0000 osd.0 (osd.0) 211 : cluster [DBG] 6.0 scrub starts 2024-02-23T00:50:17.890 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:17.587587+0000 osd.0 (osd.0) 212 : cluster [DBG] 6.0 scrub ok 2024-02-23T00:50:17.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:16.254998+0000 osd.7 (osd.7) 245 : cluster [DBG] 18.13 scrub starts 2024-02-23T00:50:17.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:16.257892+0000 osd.7 (osd.7) 246 : cluster [DBG] 18.13 scrub ok 2024-02-23T00:50:17.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:16.420157+0000 mon.a (mon.0) 2754 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-23T00:50:17.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:16.420258+0000 mon.a (mon.0) 2755 : cluster [DBG] osdmap e487: 8 total, 8 up, 7 in 2024-02-23T00:50:17.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:16.439049+0000 osd.6 (osd.6) 195 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:50:17.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:16.439200+0000 osd.6 (osd.6) 196 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:50:17.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:17.326039+0000 mgr.y (mgr.14517) 463 : cluster [DBG] pgmap v735: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-02-23T00:50:17.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:17.422475+0000 mon.a (mon.0) 2756 : cluster [DBG] osdmap e488: 8 total, 8 up, 7 in 2024-02-23T00:50:17.891 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:17.430429+0000 mon.a (mon.0) 2757 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:50:17.891 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:17.585936+0000 osd.0 (osd.0) 211 : cluster [DBG] 6.0 scrub starts 2024-02-23T00:50:17.891 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:17.587587+0000 osd.0 (osd.0) 212 : cluster [DBG] 6.0 scrub ok 2024-02-23T00:50:17.891 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:17.919 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:50:17.927 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:17.927 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:50:18.028 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:50:18.055 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:18.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:16.254998+0000 osd.7 (osd.7) 245 : cluster [DBG] 18.13 scrub starts 2024-02-23T00:50:18.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:16.257892+0000 osd.7 (osd.7) 246 : cluster [DBG] 18.13 scrub ok 2024-02-23T00:50:18.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:16.420157+0000 mon.a (mon.0) 2754 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-23T00:50:18.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:16.420258+0000 mon.a (mon.0) 2755 : cluster [DBG] osdmap e487: 8 total, 8 up, 7 in 2024-02-23T00:50:18.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:16.439049+0000 osd.6 (osd.6) 195 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:50:18.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:16.439200+0000 osd.6 (osd.6) 196 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:50:18.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:17.326039+0000 mgr.y (mgr.14517) 463 : cluster [DBG] pgmap v735: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-02-23T00:50:18.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:17.422475+0000 mon.a (mon.0) 2756 : cluster [DBG] osdmap e488: 8 total, 8 up, 7 in 2024-02-23T00:50:18.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:17.430429+0000 mon.a (mon.0) 2757 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:50:18.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:17.585936+0000 osd.0 (osd.0) 211 : cluster [DBG] 6.0 scrub starts 2024-02-23T00:50:18.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:17.587587+0000 osd.0 (osd.0) 212 : cluster [DBG] 6.0 scrub ok 2024-02-23T00:50:18.382 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:50:18.385 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:18.386 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:50:18.487 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:50:18.518 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:18.539 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:50:18.542 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:18.543 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:50:18.643 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:50:18.674 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:18.693 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:50:18.696 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:18.697 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:50:18.797 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:50:18.828 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:18.900 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:50:18.900 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:50:18.901 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:18.901 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:50:18.901 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:50:18.963 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:50:18.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.961+0000 7fe7a41f3700 1 -- 172.21.15.47:0/946425373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe79c0f6a30 msgr2=0x7fe79c0f6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:18.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.961+0000 7fe7a41f3700 1 --2- 172.21.15.47:0/946425373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe79c0f6a30 0x7fe79c0f6e10 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 rx=0x7fe7980047e0 tx=0x7fe798009cb0).stop 2024-02-23T00:50:18.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.961+0000 7fe7a41f3700 1 -- 172.21.15.47:0/946425373 shutdown_connections 2024-02-23T00:50:18.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.961+0000 7fe7a41f3700 1 --2- 172.21.15.47:0/946425373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe79c104590 0x7fe79c106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:18.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.961+0000 7fe7a41f3700 1 --2- 172.21.15.47:0/946425373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe79c0f6a30 0x7fe79c0f6e10 unknown :-1 s=CLOSED pgs=2555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:18.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.961+0000 7fe7a41f3700 1 --2- 172.21.15.47:0/946425373 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe79c0f7300 0x7fe79c1040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:18.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.961+0000 7fe7a41f3700 1 -- 172.21.15.47:0/946425373 >> 172.21.15.47:0/946425373 conn(0x7fe79c0f26f0 msgr2=0x7fe79c0f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:18.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.962+0000 7fe7a41f3700 1 -- 172.21.15.47:0/946425373 shutdown_connections 2024-02-23T00:50:18.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.962+0000 7fe7a41f3700 1 -- 172.21.15.47:0/946425373 wait complete. 2024-02-23T00:50:18.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.962+0000 7fe7a41f3700 1 librados: starting msgr at 2024-02-23T00:50:18.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.962+0000 7fe7a41f3700 1 librados: starting objecter 2024-02-23T00:50:18.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.962+0000 7fe7a41f3700 1 Processor -- start 2024-02-23T00:50:18.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.962+0000 7fe7a41f3700 1 -- start start 2024-02-23T00:50:18.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.962+0000 7fe7a41f3700 1 librados: setting wanted keys 2024-02-23T00:50:18.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.962+0000 7fe7a41f3700 1 librados: calling monclient init 2024-02-23T00:50:18.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.963+0000 7fe7a41f3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe79c0f6a30 0x7fe79c110110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:18.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.963+0000 7fe7a41f3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe79c0f7300 0x7fe79c110600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:18.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.963+0000 7fe7a41f3700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe79c104590 0x7fe79c10a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:18.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.963+0000 7fe7a41f3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c10a7a0 con 0x7fe79c0f6a30 2024-02-23T00:50:18.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.963+0000 7fe7a41f3700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c10a910 con 0x7fe79c0f7300 2024-02-23T00:50:18.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.963+0000 7fe7a41f3700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c10aad0 con 0x7fe79c104590 2024-02-23T00:50:18.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.963+0000 7fe7a1f8f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe79c0f6a30 0x7fe79c110110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:18.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.963+0000 7fe7a1f8f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe79c0f6a30 0x7fe79c110110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39582/0 (socket says 172.21.15.47:39582) 2024-02-23T00:50:18.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.963+0000 7fe7a1f8f700 1 -- 172.21.15.47:0/3988359399 learned_addr learned my addr 172.21.15.47:0/3988359399 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:50:18.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.963+0000 7fe7a2790700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe79c104590 0x7fe79c10a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:18.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.963+0000 7fe7a178e700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe79c0f7300 0x7fe79c110600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:18.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.963+0000 7fe7a1f8f700 1 -- 172.21.15.47:0/3988359399 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe79c104590 msgr2=0x7fe79c10a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:18.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.963+0000 7fe7a1f8f700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe79c104590 0x7fe79c10a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:18.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.963+0000 7fe7a1f8f700 1 -- 172.21.15.47:0/3988359399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe79c0f7300 msgr2=0x7fe79c110600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:18.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.963+0000 7fe7a1f8f700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe79c0f7300 0x7fe79c110600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:18.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.963+0000 7fe7a1f8f700 1 -- 172.21.15.47:0/3988359399 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe78c005040 con 0x7fe79c0f6a30 2024-02-23T00:50:18.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.963+0000 7fe7a178e700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe79c0f7300 0x7fe79c110600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:50:18.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.963+0000 7fe7a1f8f700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe79c0f6a30 0x7fe79c110110 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 rx=0x7fe798007fd0 tx=0x7fe798009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:18.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.963+0000 7fe792ffd700 10 client.?.objecter ms_handle_connect 0x7fe79c0f6a30 2024-02-23T00:50:18.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.963+0000 7fe792ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:50:18.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.964+0000 7fe792ffd700 1 -- 172.21.15.47:0/3988359399 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe798018070 con 0x7fe79c0f6a30 2024-02-23T00:50:18.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.964+0000 7fe792ffd700 1 -- 172.21.15.47:0/3988359399 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fe7980096d0 con 0x7fe79c0f6a30 2024-02-23T00:50:18.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.964+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe798005040 con 0x7fe79c0f6a30 2024-02-23T00:50:18.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.964+0000 7fe792ffd700 1 -- 172.21.15.47:0/3988359399 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79801dba0 con 0x7fe79c0f6a30 2024-02-23T00:50:18.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.964+0000 7fe7a41f3700 10 client.31212.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:50:18.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.964+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79c10af70 con 0x7fe79c0f6a30 2024-02-23T00:50:18.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.964+0000 7fe7a41f3700 1 librados: init done 2024-02-23T00:50:18.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.965+0000 7fe792ffd700 1 -- 172.21.15.47:0/3988359399 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fe798009240 con 0x7fe79c0f6a30 2024-02-23T00:50:18.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.966+0000 7fe792ffd700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe788062d10 0x7fe7880651d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:18.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.966+0000 7fe792ffd700 1 -- 172.21.15.47:0/3988359399 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(489..489 src has 48..489) v4 ==== 9220+0+0 (secure 0 0 0) 0x7fe79807eaf0 con 0x7fe79c0f6a30 2024-02-23T00:50:18.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.966+0000 7fe792ffd700 10 client.31212.objecter ms_dispatch 0x7fe79c001550 osd_map(489..489 src has 48..489) v4 2024-02-23T00:50:18.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.966+0000 7fe792ffd700 3 client.31212.objecter handle_osd_map got epochs [489,489] > 0 2024-02-23T00:50:18.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.966+0000 7fe792ffd700 3 client.31212.objecter handle_osd_map decoding full epoch 489 2024-02-23T00:50:18.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.966+0000 7fe7a178e700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe788062d10 0x7fe7880651d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:18.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.966+0000 7fe792ffd700 20 client.31212.objecter dump_active .. 0 homeless 2024-02-23T00:50:18.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.967+0000 7fe792ffd700 10 client.31212.objecter ms_handle_connect 0x7fe788062d10 2024-02-23T00:50:18.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.967+0000 7fe7a178e700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe788062d10 0x7fe7880651d0 secure :-1 s=READY pgs=3098 cs=0 l=1 rev1=1 rx=0x7fe78c009f20 tx=0x7fe78c009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:18.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.968+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fe79c10b160 con 0x7fe79c0f6a30 2024-02-23T00:50:18.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.968+0000 7fe792ffd700 1 -- 172.21.15.47:0/3988359399 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v489) v1 ==== 54+0+16 (secure 0 0 0) 0x7fe79804e100 con 0x7fe79c0f6a30 2024-02-23T00:50:18.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.968+0000 7fe7a41f3700 10 client.31212.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:18.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.968+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_assign -1 1 2024-02-23T00:50:18.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.968+0000 7fe7a41f3700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fe79c0fe570 0x7fe79c0fe950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:18.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.968+0000 7fe7a41f3700 20 client.31212.objecter _get_session s=0x7fe79c05f990 osd=0 3 2024-02-23T00:50:18.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.968+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c05f990 osd=0 3 2024-02-23T00:50:18.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.968+0000 7fe7a41f3700 20 client.31212.objecter _get_session s=0x7fe79c05f990 osd=0 3 2024-02-23T00:50:18.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.968+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_remove -1 1 2024-02-23T00:50:18.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.968+0000 7fe7a41f3700 20 client.31212.objecter get_session s=0x7fe79c05f990 osd=0 3 2024-02-23T00:50:18.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.968+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_assign 0 1 2024-02-23T00:50:18.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.968+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c05f990 osd=0 4 2024-02-23T00:50:18.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.968+0000 7fe7a41f3700 10 client.31212.objecter _send_command 1 2024-02-23T00:50:18.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.968+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe79c05ee80 con 0x7fe79c0fe570 2024-02-23T00:50:18.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.968+0000 7fe7a2790700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fe79c0fe570 0x7fe79c0fe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:18.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.969+0000 7fe792ffd700 10 client.31212.objecter ms_handle_connect 0x7fe79c0fe570 2024-02-23T00:50:18.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.969+0000 7fe7a2790700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fe79c0fe570 0x7fe79c0fe950 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:18.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.969+0000 7fe792ffd700 1 -- 172.21.15.47:0/3988359399 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe79c05ee80 con 0x7fe79c0fe570 2024-02-23T00:50:18.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.969+0000 7fe792ffd700 10 client.31212.objecter ms_dispatch 0x7fe79c001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:50:18.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.969+0000 7fe792ffd700 10 client.31212.objecter _finish_command 1 = system:0 2024-02-23T00:50:18.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.969+0000 7fe792ffd700 20 client.31212.objecter put_session s=0x7fe79c05f990 osd=0 4 2024-02-23T00:50:18.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.969+0000 7fe792ffd700 15 client.31212.objecter _session_command_op_remove 0 1 2024-02-23T00:50:18.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.985+0000 7fe7a41f3700 10 client.31212.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:50:18.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.985+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_assign -1 2 2024-02-23T00:50:18.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.985+0000 7fe7a41f3700 20 client.31212.objecter _get_session s=0x7fe79c05f990 osd=0 3 2024-02-23T00:50:18.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.985+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c05f990 osd=0 3 2024-02-23T00:50:18.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.985+0000 7fe7a41f3700 20 client.31212.objecter _get_session s=0x7fe79c05f990 osd=0 3 2024-02-23T00:50:18.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.985+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_remove -1 2 2024-02-23T00:50:18.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.985+0000 7fe7a41f3700 20 client.31212.objecter get_session s=0x7fe79c05f990 osd=0 3 2024-02-23T00:50:18.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.985+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_assign 0 2 2024-02-23T00:50:18.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.985+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c05f990 osd=0 4 2024-02-23T00:50:18.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.985+0000 7fe7a41f3700 10 client.31212.objecter _send_command 2 2024-02-23T00:50:18.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.985+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fe79c1090b0 con 0x7fe79c0fe570 2024-02-23T00:50:18.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.987+0000 7fe792ffd700 1 -- 172.21.15.47:0/3988359399 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fe79c1090b0 con 0x7fe79c0fe570 2024-02-23T00:50:18.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.987+0000 7fe792ffd700 10 client.31212.objecter ms_dispatch 0x7fe79c001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:50:18.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.987+0000 7fe792ffd700 10 client.31212.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:50:18.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.987+0000 7fe792ffd700 20 client.31212.objecter put_session s=0x7fe79c05f990 osd=0 4 2024-02-23T00:50:18.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.987+0000 7fe792ffd700 15 client.31212.objecter _session_command_op_remove 0 2 2024-02-23T00:50:18.992 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:50:18.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.987+0000 7fe7a41f3700 10 client.31212.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:18.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.987+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_assign -1 3 2024-02-23T00:50:18.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.987+0000 7fe7a41f3700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fe79c0598f0 0x7fe79c059cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:18.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.987+0000 7fe7a41f3700 20 client.31212.objecter _get_session s=0x7fe79c028540 osd=1 3 2024-02-23T00:50:18.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.987+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c028540 osd=1 3 2024-02-23T00:50:18.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.987+0000 7fe7a41f3700 20 client.31212.objecter _get_session s=0x7fe79c028540 osd=1 3 2024-02-23T00:50:18.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.987+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_remove -1 3 2024-02-23T00:50:18.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.987+0000 7fe7a41f3700 20 client.31212.objecter get_session s=0x7fe79c028540 osd=1 3 2024-02-23T00:50:18.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.987+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_assign 1 3 2024-02-23T00:50:18.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.987+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c028540 osd=1 4 2024-02-23T00:50:18.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.987+0000 7fe7a41f3700 10 client.31212.objecter _send_command 3 2024-02-23T00:50:18.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.987+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe79c0286b0 con 0x7fe79c0598f0 2024-02-23T00:50:18.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.988+0000 7fe7a1f8f700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fe79c0598f0 0x7fe79c059cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:18.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.988+0000 7fe792ffd700 10 client.31212.objecter ms_handle_connect 0x7fe79c0598f0 2024-02-23T00:50:18.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.988+0000 7fe7a1f8f700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fe79c0598f0 0x7fe79c059cf0 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:18.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.988+0000 7fe792ffd700 1 -- 172.21.15.47:0/3988359399 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe79c0286b0 con 0x7fe79c0598f0 2024-02-23T00:50:18.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.988+0000 7fe792ffd700 10 client.31212.objecter ms_dispatch 0x7fe79c001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:50:18.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.988+0000 7fe792ffd700 10 client.31212.objecter _finish_command 3 = system:0 2024-02-23T00:50:18.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.988+0000 7fe792ffd700 20 client.31212.objecter put_session s=0x7fe79c028540 osd=1 4 2024-02-23T00:50:18.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:18.988+0000 7fe792ffd700 15 client.31212.objecter _session_command_op_remove 1 3 2024-02-23T00:50:19.002 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:50:19.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.003+0000 7fe7a41f3700 10 client.31212.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:50:19.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.003+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_assign -1 4 2024-02-23T00:50:19.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.003+0000 7fe7a41f3700 20 client.31212.objecter _get_session s=0x7fe79c028540 osd=1 3 2024-02-23T00:50:19.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.003+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c028540 osd=1 3 2024-02-23T00:50:19.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.003+0000 7fe7a41f3700 20 client.31212.objecter _get_session s=0x7fe79c028540 osd=1 3 2024-02-23T00:50:19.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.003+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_remove -1 4 2024-02-23T00:50:19.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.003+0000 7fe7a41f3700 20 client.31212.objecter get_session s=0x7fe79c028540 osd=1 3 2024-02-23T00:50:19.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.003+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_assign 1 4 2024-02-23T00:50:19.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.003+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c028540 osd=1 4 2024-02-23T00:50:19.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.003+0000 7fe7a41f3700 10 client.31212.objecter _send_command 4 2024-02-23T00:50:19.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.003+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fe79c059360 con 0x7fe79c0598f0 2024-02-23T00:50:19.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.005+0000 7fe792ffd700 1 -- 172.21.15.47:0/3988359399 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fe79c059360 con 0x7fe79c0598f0 2024-02-23T00:50:19.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.005+0000 7fe792ffd700 10 client.31212.objecter ms_dispatch 0x7fe79c001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:50:19.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.005+0000 7fe792ffd700 10 client.31212.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:50:19.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.005+0000 7fe792ffd700 20 client.31212.objecter put_session s=0x7fe79c028540 osd=1 4 2024-02-23T00:50:19.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.005+0000 7fe792ffd700 15 client.31212.objecter _session_command_op_remove 1 4 2024-02-23T00:50:19.008 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:50:19.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.006+0000 7fe7a41f3700 10 client.31212.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:19.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.006+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_assign -1 5 2024-02-23T00:50:19.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.006+0000 7fe7a41f3700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fe79c0280a0 0x7fe79c11e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:19.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.006+0000 7fe7a41f3700 20 client.31212.objecter _get_session s=0x7fe79c059770 osd=2 3 2024-02-23T00:50:19.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.006+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c059770 osd=2 3 2024-02-23T00:50:19.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.006+0000 7fe7a41f3700 20 client.31212.objecter _get_session s=0x7fe79c059770 osd=2 3 2024-02-23T00:50:19.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.006+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_remove -1 5 2024-02-23T00:50:19.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.006+0000 7fe7a41f3700 20 client.31212.objecter get_session s=0x7fe79c059770 osd=2 3 2024-02-23T00:50:19.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.006+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_assign 2 5 2024-02-23T00:50:19.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.006+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c059770 osd=2 4 2024-02-23T00:50:19.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.006+0000 7fe7a41f3700 10 client.31212.objecter _send_command 5 2024-02-23T00:50:19.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.006+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe79c120d40 con 0x7fe79c0280a0 2024-02-23T00:50:19.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.006+0000 7fe7a178e700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fe79c0280a0 0x7fe79c11e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:19.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.006+0000 7fe792ffd700 10 client.31212.objecter ms_handle_connect 0x7fe79c0280a0 2024-02-23T00:50:19.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.006+0000 7fe7a178e700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fe79c0280a0 0x7fe79c11e0f0 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:19.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.007+0000 7fe792ffd700 1 -- 172.21.15.47:0/3988359399 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe79c120d40 con 0x7fe79c0280a0 2024-02-23T00:50:19.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.007+0000 7fe792ffd700 10 client.31212.objecter ms_dispatch 0x7fe79c001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:50:19.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.007+0000 7fe792ffd700 10 client.31212.objecter _finish_command 5 = system:0 2024-02-23T00:50:19.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.007+0000 7fe792ffd700 20 client.31212.objecter put_session s=0x7fe79c059770 osd=2 4 2024-02-23T00:50:19.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.007+0000 7fe792ffd700 15 client.31212.objecter _session_command_op_remove 2 5 2024-02-23T00:50:19.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.024+0000 7fe7a41f3700 10 client.31212.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:50:19.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.024+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_assign -1 6 2024-02-23T00:50:19.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.024+0000 7fe7a41f3700 20 client.31212.objecter _get_session s=0x7fe79c059770 osd=2 3 2024-02-23T00:50:19.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.024+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c059770 osd=2 3 2024-02-23T00:50:19.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.024+0000 7fe7a41f3700 20 client.31212.objecter _get_session s=0x7fe79c059770 osd=2 3 2024-02-23T00:50:19.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.024+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_remove -1 6 2024-02-23T00:50:19.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.024+0000 7fe7a41f3700 20 client.31212.objecter get_session s=0x7fe79c059770 osd=2 3 2024-02-23T00:50:19.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.024+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_assign 2 6 2024-02-23T00:50:19.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.024+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c059770 osd=2 4 2024-02-23T00:50:19.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.024+0000 7fe7a41f3700 10 client.31212.objecter _send_command 6 2024-02-23T00:50:19.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.024+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fe79c027660 con 0x7fe79c0280a0 2024-02-23T00:50:19.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.026+0000 7fe792ffd700 1 -- 172.21.15.47:0/3988359399 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fe79c027660 con 0x7fe79c0280a0 2024-02-23T00:50:19.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.026+0000 7fe792ffd700 10 client.31212.objecter ms_dispatch 0x7fe79c001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:50:19.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.026+0000 7fe792ffd700 10 client.31212.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:50:19.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.026+0000 7fe792ffd700 20 client.31212.objecter put_session s=0x7fe79c059770 osd=2 4 2024-02-23T00:50:19.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.026+0000 7fe792ffd700 15 client.31212.objecter _session_command_op_remove 2 6 2024-02-23T00:50:19.029 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:50:19.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.026+0000 7fe7a41f3700 10 client.31212.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:19.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.026+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_assign -1 7 2024-02-23T00:50:19.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.026+0000 7fe7a41f3700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fe79c11f760 0x7fe79c11fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:19.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.026+0000 7fe7a41f3700 20 client.31212.objecter _get_session s=0x7fe79c11b730 osd=3 3 2024-02-23T00:50:19.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.026+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c11b730 osd=3 3 2024-02-23T00:50:19.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.026+0000 7fe7a41f3700 20 client.31212.objecter _get_session s=0x7fe79c11b730 osd=3 3 2024-02-23T00:50:19.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.026+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_remove -1 7 2024-02-23T00:50:19.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.026+0000 7fe7a41f3700 20 client.31212.objecter get_session s=0x7fe79c11b730 osd=3 3 2024-02-23T00:50:19.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.026+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_assign 3 7 2024-02-23T00:50:19.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.026+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c11b730 osd=3 4 2024-02-23T00:50:19.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.026+0000 7fe7a41f3700 10 client.31212.objecter _send_command 7 2024-02-23T00:50:19.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.026+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe79c11bef0 con 0x7fe79c11f760 2024-02-23T00:50:19.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.026+0000 7fe7a2790700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fe79c11f760 0x7fe79c11fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:19.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.026+0000 7fe792ffd700 10 client.31212.objecter ms_handle_connect 0x7fe79c11f760 2024-02-23T00:50:19.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.026+0000 7fe7a2790700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fe79c11f760 0x7fe79c11fba0 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:19.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.027+0000 7fe792ffd700 1 -- 172.21.15.47:0/3988359399 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe79c11bef0 con 0x7fe79c11f760 2024-02-23T00:50:19.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.027+0000 7fe792ffd700 10 client.31212.objecter ms_dispatch 0x7fe79c001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:50:19.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.027+0000 7fe792ffd700 10 client.31212.objecter _finish_command 7 = system:0 2024-02-23T00:50:19.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.027+0000 7fe792ffd700 20 client.31212.objecter put_session s=0x7fe79c11b730 osd=3 4 2024-02-23T00:50:19.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.027+0000 7fe792ffd700 15 client.31212.objecter _session_command_op_remove 3 7 2024-02-23T00:50:19.032 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:17.598687+0000 osd.4 (osd.4) 213 : cluster [DBG] 6.2 scrub starts 2024-02-23T00:50:19.032 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:17.599718+0000 osd.4 (osd.4) 214 : cluster [DBG] 6.2 scrub ok 2024-02-23T00:50:19.032 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:17.858005+0000 osd.5 (osd.5) 216 : cluster [DBG] 4.15 scrub starts 2024-02-23T00:50:19.032 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:17.861486+0000 osd.5 (osd.5) 217 : cluster [DBG] 4.15 scrub ok 2024-02-23T00:50:19.032 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:18.307386+0000 osd.6 (osd.6) 197 : cluster [DBG] 18.14 scrub starts 2024-02-23T00:50:19.032 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:18.308344+0000 osd.6 (osd.6) 198 : cluster [DBG] 18.14 scrub ok 2024-02-23T00:50:19.032 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:18.423969+0000 mon.a (mon.0) 2758 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:50:19.032 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:18.424034+0000 mon.a (mon.0) 2759 : cluster [DBG] osdmap e489: 8 total, 8 up, 7 in 2024-02-23T00:50:19.032 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:18.432080+0000 mon.a (mon.0) 2760 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:50:19.032 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:18.434902+0000 mon.a (mon.0) 2761 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-23T00:50:19.033 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:17.598687+0000 osd.4 (osd.4) 213 : cluster [DBG] 6.2 scrub starts 2024-02-23T00:50:19.033 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:17.599718+0000 osd.4 (osd.4) 214 : cluster [DBG] 6.2 scrub ok 2024-02-23T00:50:19.033 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:17.858005+0000 osd.5 (osd.5) 216 : cluster [DBG] 4.15 scrub starts 2024-02-23T00:50:19.033 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:17.861486+0000 osd.5 (osd.5) 217 : cluster [DBG] 4.15 scrub ok 2024-02-23T00:50:19.033 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:18.307386+0000 osd.6 (osd.6) 197 : cluster [DBG] 18.14 scrub starts 2024-02-23T00:50:19.033 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:18.308344+0000 osd.6 (osd.6) 198 : cluster [DBG] 18.14 scrub ok 2024-02-23T00:50:19.033 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:18.423969+0000 mon.a (mon.0) 2758 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:50:19.033 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:18.424034+0000 mon.a (mon.0) 2759 : cluster [DBG] osdmap e489: 8 total, 8 up, 7 in 2024-02-23T00:50:19.033 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:18.432080+0000 mon.a (mon.0) 2760 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:50:19.033 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:18.434902+0000 mon.a (mon.0) 2761 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-23T00:50:19.034 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:19.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.043+0000 7fe7a41f3700 10 client.31212.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:50:19.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.043+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_assign -1 8 2024-02-23T00:50:19.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.043+0000 7fe7a41f3700 20 client.31212.objecter _get_session s=0x7fe79c11b730 osd=3 3 2024-02-23T00:50:19.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.043+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c11b730 osd=3 3 2024-02-23T00:50:19.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.043+0000 7fe7a41f3700 20 client.31212.objecter _get_session s=0x7fe79c11b730 osd=3 3 2024-02-23T00:50:19.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.043+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_remove -1 8 2024-02-23T00:50:19.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.043+0000 7fe7a41f3700 20 client.31212.objecter get_session s=0x7fe79c11b730 osd=3 3 2024-02-23T00:50:19.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.043+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_assign 3 8 2024-02-23T00:50:19.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.043+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c11b730 osd=3 4 2024-02-23T00:50:19.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.043+0000 7fe7a41f3700 10 client.31212.objecter _send_command 8 2024-02-23T00:50:19.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.043+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fe79c120890 con 0x7fe79c11f760 2024-02-23T00:50:19.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.044+0000 7fe792ffd700 1 -- 172.21.15.47:0/3988359399 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fe79c120890 con 0x7fe79c11f760 2024-02-23T00:50:19.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.044+0000 7fe792ffd700 10 client.31212.objecter ms_dispatch 0x7fe79c001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:50:19.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.044+0000 7fe792ffd700 10 client.31212.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:50:19.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.044+0000 7fe792ffd700 20 client.31212.objecter put_session s=0x7fe79c11b730 osd=3 4 2024-02-23T00:50:19.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.044+0000 7fe792ffd700 15 client.31212.objecter _session_command_op_remove 3 8 2024-02-23T00:50:19.048 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:50:19.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.045+0000 7fe7a41f3700 10 client.31212.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:19.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.045+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_assign -1 9 2024-02-23T00:50:19.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.045+0000 7fe7a41f3700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fe79c122ec0 0x7fe79c1252b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:19.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.045+0000 7fe7a41f3700 20 client.31212.objecter _get_session s=0x7fe79c1200c0 osd=4 3 2024-02-23T00:50:19.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.045+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c1200c0 osd=4 3 2024-02-23T00:50:19.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.045+0000 7fe7a41f3700 20 client.31212.objecter _get_session s=0x7fe79c1200c0 osd=4 3 2024-02-23T00:50:19.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.045+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_remove -1 9 2024-02-23T00:50:19.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.045+0000 7fe7a41f3700 20 client.31212.objecter get_session s=0x7fe79c1200c0 osd=4 3 2024-02-23T00:50:19.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.045+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_assign 4 9 2024-02-23T00:50:19.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.045+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c1200c0 osd=4 4 2024-02-23T00:50:19.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.045+0000 7fe7a41f3700 10 client.31212.objecter _send_command 9 2024-02-23T00:50:19.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.045+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe79c127f00 con 0x7fe79c122ec0 2024-02-23T00:50:19.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.045+0000 7fe7a1f8f700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fe79c122ec0 0x7fe79c1252b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:19.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.046+0000 7fe792ffd700 10 client.31212.objecter ms_handle_connect 0x7fe79c122ec0 2024-02-23T00:50:19.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.046+0000 7fe7a1f8f700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fe79c122ec0 0x7fe79c1252b0 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:19.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.046+0000 7fe792ffd700 1 -- 172.21.15.47:0/3988359399 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe79c127f00 con 0x7fe79c122ec0 2024-02-23T00:50:19.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.046+0000 7fe792ffd700 10 client.31212.objecter ms_dispatch 0x7fe79c001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:50:19.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.046+0000 7fe792ffd700 10 client.31212.objecter _finish_command 9 = system:0 2024-02-23T00:50:19.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.046+0000 7fe792ffd700 20 client.31212.objecter put_session s=0x7fe79c1200c0 osd=4 4 2024-02-23T00:50:19.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.046+0000 7fe792ffd700 15 client.31212.objecter _session_command_op_remove 4 9 2024-02-23T00:50:19.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.062+0000 7fe7a41f3700 10 client.31212.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:50:19.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.062+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_assign -1 10 2024-02-23T00:50:19.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.062+0000 7fe7a41f3700 20 client.31212.objecter _get_session s=0x7fe79c1200c0 osd=4 3 2024-02-23T00:50:19.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.062+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c1200c0 osd=4 3 2024-02-23T00:50:19.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.062+0000 7fe7a41f3700 20 client.31212.objecter _get_session s=0x7fe79c1200c0 osd=4 3 2024-02-23T00:50:19.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.062+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_remove -1 10 2024-02-23T00:50:19.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.062+0000 7fe7a41f3700 20 client.31212.objecter get_session s=0x7fe79c1200c0 osd=4 3 2024-02-23T00:50:19.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.062+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_assign 4 10 2024-02-23T00:50:19.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.062+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c1200c0 osd=4 4 2024-02-23T00:50:19.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.062+0000 7fe7a41f3700 10 client.31212.objecter _send_command 10 2024-02-23T00:50:19.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.062+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fe79c1257a0 con 0x7fe79c122ec0 2024-02-23T00:50:19.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.064+0000 7fe792ffd700 1 -- 172.21.15.47:0/3988359399 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fe79c1257a0 con 0x7fe79c122ec0 2024-02-23T00:50:19.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.064+0000 7fe792ffd700 10 client.31212.objecter ms_dispatch 0x7fe79c001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:50:19.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.064+0000 7fe792ffd700 10 client.31212.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:50:19.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.064+0000 7fe792ffd700 20 client.31212.objecter put_session s=0x7fe79c1200c0 osd=4 4 2024-02-23T00:50:19.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.064+0000 7fe792ffd700 15 client.31212.objecter _session_command_op_remove 4 10 2024-02-23T00:50:19.067 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:50:19.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.064+0000 7fe7a41f3700 10 client.31212.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:19.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.064+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_assign -1 11 2024-02-23T00:50:19.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.064+0000 7fe7a41f3700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fe79c125e20 0x7fe79c128210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:19.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.064+0000 7fe7a41f3700 20 client.31212.objecter _get_session s=0x7fe79c1257a0 osd=5 3 2024-02-23T00:50:19.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.064+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c1257a0 osd=5 3 2024-02-23T00:50:19.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.064+0000 7fe7a41f3700 20 client.31212.objecter _get_session s=0x7fe79c1257a0 osd=5 3 2024-02-23T00:50:19.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.064+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_remove -1 11 2024-02-23T00:50:19.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.064+0000 7fe7a41f3700 20 client.31212.objecter get_session s=0x7fe79c1257a0 osd=5 3 2024-02-23T00:50:19.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.064+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_assign 5 11 2024-02-23T00:50:19.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.064+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c1257a0 osd=5 4 2024-02-23T00:50:19.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.064+0000 7fe7a41f3700 10 client.31212.objecter _send_command 11 2024-02-23T00:50:19.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.064+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe79c12b030 con 0x7fe79c125e20 2024-02-23T00:50:19.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.065+0000 7fe7a178e700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fe79c125e20 0x7fe79c128210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:19.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.065+0000 7fe792ffd700 10 client.31212.objecter ms_handle_connect 0x7fe79c125e20 2024-02-23T00:50:19.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.065+0000 7fe7a178e700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fe79c125e20 0x7fe79c128210 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:19.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.066+0000 7fe792ffd700 1 -- 172.21.15.47:0/3988359399 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe79c12b030 con 0x7fe79c125e20 2024-02-23T00:50:19.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.066+0000 7fe792ffd700 10 client.31212.objecter ms_dispatch 0x7fe79c001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:50:19.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.066+0000 7fe792ffd700 10 client.31212.objecter _finish_command 11 = system:0 2024-02-23T00:50:19.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.066+0000 7fe792ffd700 20 client.31212.objecter put_session s=0x7fe79c1257a0 osd=5 4 2024-02-23T00:50:19.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.066+0000 7fe792ffd700 15 client.31212.objecter _session_command_op_remove 5 11 2024-02-23T00:50:19.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.080+0000 7fe7a41f3700 10 client.31212.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:50:19.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.080+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_assign -1 12 2024-02-23T00:50:19.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.080+0000 7fe7a41f3700 20 client.31212.objecter _get_session s=0x7fe79c1257a0 osd=5 3 2024-02-23T00:50:19.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.080+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c1257a0 osd=5 3 2024-02-23T00:50:19.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.080+0000 7fe7a41f3700 20 client.31212.objecter _get_session s=0x7fe79c1257a0 osd=5 3 2024-02-23T00:50:19.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.080+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_remove -1 12 2024-02-23T00:50:19.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.080+0000 7fe7a41f3700 20 client.31212.objecter get_session s=0x7fe79c1257a0 osd=5 3 2024-02-23T00:50:19.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.080+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_assign 5 12 2024-02-23T00:50:19.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.080+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c1257a0 osd=5 4 2024-02-23T00:50:19.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.080+0000 7fe7a41f3700 10 client.31212.objecter _send_command 12 2024-02-23T00:50:19.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.080+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fe79c12b030 con 0x7fe79c125e20 2024-02-23T00:50:19.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.082+0000 7fe792ffd700 1 -- 172.21.15.47:0/3988359399 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fe79c12b030 con 0x7fe79c125e20 2024-02-23T00:50:19.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.082+0000 7fe792ffd700 10 client.31212.objecter ms_dispatch 0x7fe79c001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:50:19.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.082+0000 7fe792ffd700 10 client.31212.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:50:19.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.082+0000 7fe792ffd700 20 client.31212.objecter put_session s=0x7fe79c1257a0 osd=5 4 2024-02-23T00:50:19.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.082+0000 7fe792ffd700 15 client.31212.objecter _session_command_op_remove 5 12 2024-02-23T00:50:19.086 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:50:19.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.083+0000 7fe7a41f3700 10 client.31212.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:19.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.083+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_assign -1 13 2024-02-23T00:50:19.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.083+0000 7fe7a41f3700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fe79c12a360 0x7fe79c12a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:19.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.083+0000 7fe7a41f3700 20 client.31212.objecter _get_session s=0x7fe79c12b030 osd=6 3 2024-02-23T00:50:19.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.083+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c12b030 osd=6 3 2024-02-23T00:50:19.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.083+0000 7fe7a41f3700 20 client.31212.objecter _get_session s=0x7fe79c12b030 osd=6 3 2024-02-23T00:50:19.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.083+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_remove -1 13 2024-02-23T00:50:19.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.083+0000 7fe7a41f3700 20 client.31212.objecter get_session s=0x7fe79c12b030 osd=6 3 2024-02-23T00:50:19.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.083+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_assign 6 13 2024-02-23T00:50:19.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.083+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c12b030 osd=6 4 2024-02-23T00:50:19.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.083+0000 7fe7a41f3700 10 client.31212.objecter _send_command 13 2024-02-23T00:50:19.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.083+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe79c12e3e0 con 0x7fe79c12a360 2024-02-23T00:50:19.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.083+0000 7fe7a2790700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fe79c12a360 0x7fe79c12a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:19.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.083+0000 7fe7a2790700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fe79c12a360 0x7fe79c12a7c0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:19.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.083+0000 7fe792ffd700 10 client.31212.objecter ms_handle_connect 0x7fe79c12a360 2024-02-23T00:50:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.084+0000 7fe792ffd700 1 -- 172.21.15.47:0/3988359399 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe79c12e3e0 con 0x7fe79c12a360 2024-02-23T00:50:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.084+0000 7fe792ffd700 10 client.31212.objecter ms_dispatch 0x7fe79c001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:50:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.084+0000 7fe792ffd700 10 client.31212.objecter _finish_command 13 = system:0 2024-02-23T00:50:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.084+0000 7fe792ffd700 20 client.31212.objecter put_session s=0x7fe79c12b030 osd=6 4 2024-02-23T00:50:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.084+0000 7fe792ffd700 15 client.31212.objecter _session_command_op_remove 6 13 2024-02-23T00:50:19.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.099+0000 7fe7a41f3700 10 client.31212.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:50:19.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.099+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_assign -1 14 2024-02-23T00:50:19.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.099+0000 7fe7a41f3700 20 client.31212.objecter _get_session s=0x7fe79c12b030 osd=6 3 2024-02-23T00:50:19.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.099+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c12b030 osd=6 3 2024-02-23T00:50:19.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.099+0000 7fe7a41f3700 20 client.31212.objecter _get_session s=0x7fe79c12b030 osd=6 3 2024-02-23T00:50:19.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.099+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_remove -1 14 2024-02-23T00:50:19.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.099+0000 7fe7a41f3700 20 client.31212.objecter get_session s=0x7fe79c12b030 osd=6 3 2024-02-23T00:50:19.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.099+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_assign 6 14 2024-02-23T00:50:19.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.099+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c12b030 osd=6 4 2024-02-23T00:50:19.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.099+0000 7fe7a41f3700 10 client.31212.objecter _send_command 14 2024-02-23T00:50:19.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.099+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fe79c1287f0 con 0x7fe79c12a360 2024-02-23T00:50:19.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.101+0000 7fe792ffd700 1 -- 172.21.15.47:0/3988359399 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fe79c1287f0 con 0x7fe79c12a360 2024-02-23T00:50:19.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.101+0000 7fe792ffd700 10 client.31212.objecter ms_dispatch 0x7fe79c001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:50:19.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.101+0000 7fe792ffd700 10 client.31212.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:50:19.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.101+0000 7fe792ffd700 20 client.31212.objecter put_session s=0x7fe79c12b030 osd=6 4 2024-02-23T00:50:19.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.101+0000 7fe792ffd700 15 client.31212.objecter _session_command_op_remove 6 14 2024-02-23T00:50:19.104 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:50:19.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.101+0000 7fe7a41f3700 10 client.31212.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:19.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.101+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_assign -1 15 2024-02-23T00:50:19.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.101+0000 7fe7a41f3700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fe79c128e70 0x7fe79c1292d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:19.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.101+0000 7fe7a41f3700 20 client.31212.objecter _get_session s=0x7fe79c1287f0 osd=7 3 2024-02-23T00:50:19.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.101+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c1287f0 osd=7 3 2024-02-23T00:50:19.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.101+0000 7fe7a41f3700 20 client.31212.objecter _get_session s=0x7fe79c1287f0 osd=7 3 2024-02-23T00:50:19.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.101+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_remove -1 15 2024-02-23T00:50:19.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.101+0000 7fe7a41f3700 20 client.31212.objecter get_session s=0x7fe79c1287f0 osd=7 3 2024-02-23T00:50:19.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.101+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_assign 7 15 2024-02-23T00:50:19.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.101+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c1287f0 osd=7 4 2024-02-23T00:50:19.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.101+0000 7fe7a41f3700 10 client.31212.objecter _send_command 15 2024-02-23T00:50:19.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.101+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe79c129960 con 0x7fe79c128e70 2024-02-23T00:50:19.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.101+0000 7fe7a1f8f700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fe79c128e70 0x7fe79c1292d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:19.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.101+0000 7fe792ffd700 10 client.31212.objecter ms_handle_connect 0x7fe79c128e70 2024-02-23T00:50:19.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.101+0000 7fe7a1f8f700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fe79c128e70 0x7fe79c1292d0 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:19.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.102+0000 7fe792ffd700 1 -- 172.21.15.47:0/3988359399 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe79c129960 con 0x7fe79c128e70 2024-02-23T00:50:19.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.102+0000 7fe792ffd700 10 client.31212.objecter ms_dispatch 0x7fe79c001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:50:19.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.102+0000 7fe792ffd700 10 client.31212.objecter _finish_command 15 = system:0 2024-02-23T00:50:19.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.102+0000 7fe792ffd700 20 client.31212.objecter put_session s=0x7fe79c1287f0 osd=7 4 2024-02-23T00:50:19.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.102+0000 7fe792ffd700 15 client.31212.objecter _session_command_op_remove 7 15 2024-02-23T00:50:19.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.117+0000 7fe7a41f3700 10 client.31212.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:50:19.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.117+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_assign -1 16 2024-02-23T00:50:19.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.117+0000 7fe7a41f3700 20 client.31212.objecter _get_session s=0x7fe79c1287f0 osd=7 3 2024-02-23T00:50:19.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.117+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c1287f0 osd=7 3 2024-02-23T00:50:19.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.117+0000 7fe7a41f3700 20 client.31212.objecter _get_session s=0x7fe79c1287f0 osd=7 3 2024-02-23T00:50:19.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.117+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_remove -1 16 2024-02-23T00:50:19.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.117+0000 7fe7a41f3700 20 client.31212.objecter get_session s=0x7fe79c1287f0 osd=7 3 2024-02-23T00:50:19.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.117+0000 7fe7a41f3700 15 client.31212.objecter _session_command_op_assign 7 16 2024-02-23T00:50:19.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.117+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c1287f0 osd=7 4 2024-02-23T00:50:19.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.117+0000 7fe7a41f3700 10 client.31212.objecter _send_command 16 2024-02-23T00:50:19.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.117+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fe79c129960 con 0x7fe79c128e70 2024-02-23T00:50:19.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe792ffd700 1 -- 172.21.15.47:0/3988359399 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fe79c129960 con 0x7fe79c128e70 2024-02-23T00:50:19.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe792ffd700 10 client.31212.objecter ms_dispatch 0x7fe79c001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:50:19.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe792ffd700 10 client.31212.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:50:19.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe792ffd700 20 client.31212.objecter put_session s=0x7fe79c1287f0 osd=7 4 2024-02-23T00:50:19.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe792ffd700 15 client.31212.objecter _session_command_op_remove 7 16 2024-02-23T00:50:19.122 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:50:19.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 10 librados: watch_flush enter 2024-02-23T00:50:19.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 10 librados: watch_flush exit 2024-02-23T00:50:19.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 10 client.31212.objecter close_session for osd.0 2024-02-23T00:50:19.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fe79c0fe570 msgr2=0x7fe79c0fe950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:19.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fe79c0fe570 0x7fe79c0fe950 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:19.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c05f990 osd=0 1 2024-02-23T00:50:19.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 10 client.31212.objecter close_session for osd.1 2024-02-23T00:50:19.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fe79c0598f0 msgr2=0x7fe79c059cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:19.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fe79c0598f0 0x7fe79c059cf0 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:19.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c028540 osd=1 1 2024-02-23T00:50:19.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 10 client.31212.objecter close_session for osd.2 2024-02-23T00:50:19.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fe79c0280a0 msgr2=0x7fe79c11e0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:19.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fe79c0280a0 0x7fe79c11e0f0 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:19.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c059770 osd=2 1 2024-02-23T00:50:19.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 10 client.31212.objecter close_session for osd.3 2024-02-23T00:50:19.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fe79c11f760 msgr2=0x7fe79c11fba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:19.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fe79c11f760 0x7fe79c11fba0 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:19.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c11b730 osd=3 1 2024-02-23T00:50:19.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 10 client.31212.objecter close_session for osd.4 2024-02-23T00:50:19.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fe79c122ec0 msgr2=0x7fe79c1252b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:19.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fe79c122ec0 0x7fe79c1252b0 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:19.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a2790700 1 -- 172.21.15.47:0/3988359399 reap_dead start 2024-02-23T00:50:19.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a2790700 1 -- 172.21.15.47:0/3988359399 reap_dead start 2024-02-23T00:50:19.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c1200c0 osd=4 1 2024-02-23T00:50:19.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 10 client.31212.objecter close_session for osd.5 2024-02-23T00:50:19.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fe79c125e20 msgr2=0x7fe79c128210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:19.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fe79c125e20 0x7fe79c128210 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:19.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c1257a0 osd=5 1 2024-02-23T00:50:19.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 10 client.31212.objecter close_session for osd.6 2024-02-23T00:50:19.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fe79c12a360 msgr2=0x7fe79c12a7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:19.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fe79c12a360 0x7fe79c12a7c0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:19.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c12b030 osd=6 1 2024-02-23T00:50:19.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 10 client.31212.objecter close_session for osd.7 2024-02-23T00:50:19.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fe79c128e70 msgr2=0x7fe79c1292d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:19.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fe79c128e70 0x7fe79c1292d0 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:19.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 20 client.31212.objecter put_session s=0x7fe79c1287f0 osd=7 1 2024-02-23T00:50:19.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 20 client.31212.objecter shutdown clearing up homeless session... 2024-02-23T00:50:19.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 10 client.31212.objecter successfully canceled tick 2024-02-23T00:50:19.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe788062d10 msgr2=0x7fe7880651d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:19.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe788062d10 0x7fe7880651d0 secure :-1 s=READY pgs=3098 cs=0 l=1 rev1=1 rx=0x7fe78c009f20 tx=0x7fe78c009cb0).stop 2024-02-23T00:50:19.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe79c0f6a30 msgr2=0x7fe79c110110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:19.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 1 --2- 172.21.15.47:0/3988359399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe79c0f6a30 0x7fe79c110110 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 rx=0x7fe798007fd0 tx=0x7fe798009cb0).stop 2024-02-23T00:50:19.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a2790700 1 -- 172.21.15.47:0/3988359399 reap_dead start 2024-02-23T00:50:19.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 shutdown_connections 2024-02-23T00:50:19.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.119+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 >> 172.21.15.47:0/3988359399 conn(0x7fe79c0f26f0 msgr2=0x7fe79c0f4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:19.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.120+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 shutdown_connections 2024-02-23T00:50:19.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.120+0000 7fe7a41f3700 1 -- 172.21.15.47:0/3988359399 wait complete. 2024-02-23T00:50:19.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:19.120+0000 7fe7a41f3700 1 librados: shutdown 2024-02-23T00:50:19.131 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:50:19.131 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:50:19.131 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:50:19.131 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:50:19.131 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:50:19.131 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:50:19.131 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:50:19.132 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:50:19.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:17.598687+0000 osd.4 (osd.4) 213 : cluster [DBG] 6.2 scrub starts 2024-02-23T00:50:19.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:17.599718+0000 osd.4 (osd.4) 214 : cluster [DBG] 6.2 scrub ok 2024-02-23T00:50:19.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:17.858005+0000 osd.5 (osd.5) 216 : cluster [DBG] 4.15 scrub starts 2024-02-23T00:50:19.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:17.861486+0000 osd.5 (osd.5) 217 : cluster [DBG] 4.15 scrub ok 2024-02-23T00:50:19.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:18.307386+0000 osd.6 (osd.6) 197 : cluster [DBG] 18.14 scrub starts 2024-02-23T00:50:19.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:18.308344+0000 osd.6 (osd.6) 198 : cluster [DBG] 18.14 scrub ok 2024-02-23T00:50:19.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:18.423969+0000 mon.a (mon.0) 2758 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:50:19.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:18.424034+0000 mon.a (mon.0) 2759 : cluster [DBG] osdmap e489: 8 total, 8 up, 7 in 2024-02-23T00:50:19.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:18.432080+0000 mon.a (mon.0) 2760 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:50:19.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:18.434902+0000 mon.a (mon.0) 2761 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-23T00:50:19.583 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:50:19.589 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:19.589 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:50:19.690 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:50:19.718 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:20.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:18.628306+0000 osd.4 (osd.4) 215 : cluster [DBG] 5.1c scrub starts 2024-02-23T00:50:20.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:18.629199+0000 osd.4 (osd.4) 216 : cluster [DBG] 5.1c scrub ok 2024-02-23T00:50:20.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:18.695747+0000 osd.1 (osd.1) 221 : cluster [DBG] 5.12 scrub starts 2024-02-23T00:50:20.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:18.698678+0000 osd.1 (osd.1) 222 : cluster [DBG] 5.12 scrub ok 2024-02-23T00:50:20.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:18.969428+0000 mon.a (mon.0) 2762 : audit [DBG] from='client.? 172.21.15.47:0/3988359399' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:50:20.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:19.326405+0000 mgr.y (mgr.14517) 464 : cluster [DBG] pgmap v738: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-02-23T00:50:20.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:19.425207+0000 mon.a (mon.0) 2763 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-23T00:50:20.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:19.425279+0000 mon.a (mon.0) 2764 : cluster [DBG] osdmap e490: 8 total, 8 up, 7 in 2024-02-23T00:50:20.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:19.427053+0000 mon.a (mon.0) 2765 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_tier","val": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:50:20.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:18.628306+0000 osd.4 (osd.4) 215 : cluster [DBG] 5.1c scrub starts 2024-02-23T00:50:20.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:18.629199+0000 osd.4 (osd.4) 216 : cluster [DBG] 5.1c scrub ok 2024-02-23T00:50:20.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:18.695747+0000 osd.1 (osd.1) 221 : cluster [DBG] 5.12 scrub starts 2024-02-23T00:50:20.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:18.698678+0000 osd.1 (osd.1) 222 : cluster [DBG] 5.12 scrub ok 2024-02-23T00:50:20.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:18.969428+0000 mon.a (mon.0) 2762 : audit [DBG] from='client.? 172.21.15.47:0/3988359399' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:50:20.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:19.326405+0000 mgr.y (mgr.14517) 464 : cluster [DBG] pgmap v738: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-02-23T00:50:20.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:19.425207+0000 mon.a (mon.0) 2763 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-23T00:50:20.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:19.425279+0000 mon.a (mon.0) 2764 : cluster [DBG] osdmap e490: 8 total, 8 up, 7 in 2024-02-23T00:50:20.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:19.427053+0000 mon.a (mon.0) 2765 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_tier","val": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:50:20.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:18.628306+0000 osd.4 (osd.4) 215 : cluster [DBG] 5.1c scrub starts 2024-02-23T00:50:20.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:18.629199+0000 osd.4 (osd.4) 216 : cluster [DBG] 5.1c scrub ok 2024-02-23T00:50:20.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:18.695747+0000 osd.1 (osd.1) 221 : cluster [DBG] 5.12 scrub starts 2024-02-23T00:50:20.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:18.698678+0000 osd.1 (osd.1) 222 : cluster [DBG] 5.12 scrub ok 2024-02-23T00:50:20.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:18.969428+0000 mon.a (mon.0) 2762 : audit [DBG] from='client.? 172.21.15.47:0/3988359399' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:50:20.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:19.326405+0000 mgr.y (mgr.14517) 464 : cluster [DBG] pgmap v738: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-02-23T00:50:20.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:19.425207+0000 mon.a (mon.0) 2763 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-23T00:50:20.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:19.425279+0000 mon.a (mon.0) 2764 : cluster [DBG] osdmap e490: 8 total, 8 up, 7 in 2024-02-23T00:50:20.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:19.427053+0000 mon.a (mon.0) 2765 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_tier","val": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:50:20.725 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:50:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:50:20] "GET /metrics HTTP/1.1" 200 189423 "" "Prometheus/2.33.4" 2024-02-23T00:50:20.768 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:50:20.770 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:20.771 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:50:20.872 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:50:20.901 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:21.024 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:50:21.029 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:21.029 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:50:21.130 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:50:21.158 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:21.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:19.824902+0000 osd.5 (osd.5) 218 : cluster [DBG] 5.f scrub starts 2024-02-23T00:50:21.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:19.827809+0000 osd.5 (osd.5) 219 : cluster [DBG] 5.f scrub ok 2024-02-23T00:50:21.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:19.966111+0000 osd.3 (osd.3) 234 : cluster [DBG] 2.1e scrub starts 2024-02-23T00:50:21.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:19.967005+0000 osd.3 (osd.3) 235 : cluster [DBG] 2.1e scrub ok 2024-02-23T00:50:21.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:20.219762+0000 mon.a (mon.0) 2766 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_tier","val": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:50:21.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:20.219816+0000 mon.a (mon.0) 2767 : cluster [DBG] osdmap e491: 8 total, 8 up, 7 in 2024-02-23T00:50:21.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:20.221031+0000 mon.a (mon.0) 2768 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-23T00:50:21.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:20.273363+0000 osd.6 (osd.6) 199 : cluster [DBG] 62.19 scrub starts 2024-02-23T00:50:21.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:20.275310+0000 osd.6 (osd.6) 200 : cluster [DBG] 62.19 scrub ok 2024-02-23T00:50:21.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:19.824902+0000 osd.5 (osd.5) 218 : cluster [DBG] 5.f scrub starts 2024-02-23T00:50:21.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:19.827809+0000 osd.5 (osd.5) 219 : cluster [DBG] 5.f scrub ok 2024-02-23T00:50:21.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:19.966111+0000 osd.3 (osd.3) 234 : cluster [DBG] 2.1e scrub starts 2024-02-23T00:50:21.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:19.967005+0000 osd.3 (osd.3) 235 : cluster [DBG] 2.1e scrub ok 2024-02-23T00:50:21.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:20.219762+0000 mon.a (mon.0) 2766 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_tier","val": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:50:21.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:20.219816+0000 mon.a (mon.0) 2767 : cluster [DBG] osdmap e491: 8 total, 8 up, 7 in 2024-02-23T00:50:21.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:20.221031+0000 mon.a (mon.0) 2768 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-23T00:50:21.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:20.273363+0000 osd.6 (osd.6) 199 : cluster [DBG] 62.19 scrub starts 2024-02-23T00:50:21.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:20.275310+0000 osd.6 (osd.6) 200 : cluster [DBG] 62.19 scrub ok 2024-02-23T00:50:21.607 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:50:21.610 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:21.610 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:50:21.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:19.824902+0000 osd.5 (osd.5) 218 : cluster [DBG] 5.f scrub starts 2024-02-23T00:50:21.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:19.827809+0000 osd.5 (osd.5) 219 : cluster [DBG] 5.f scrub ok 2024-02-23T00:50:21.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:19.966111+0000 osd.3 (osd.3) 234 : cluster [DBG] 2.1e scrub starts 2024-02-23T00:50:21.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:19.967005+0000 osd.3 (osd.3) 235 : cluster [DBG] 2.1e scrub ok 2024-02-23T00:50:21.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:20.219762+0000 mon.a (mon.0) 2766 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_tier","val": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:50:21.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:20.219816+0000 mon.a (mon.0) 2767 : cluster [DBG] osdmap e491: 8 total, 8 up, 7 in 2024-02-23T00:50:21.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:20.221031+0000 mon.a (mon.0) 2768 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-23T00:50:21.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:20.273363+0000 osd.6 (osd.6) 199 : cluster [DBG] 62.19 scrub starts 2024-02-23T00:50:21.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:20.275310+0000 osd.6 (osd.6) 200 : cluster [DBG] 62.19 scrub ok 2024-02-23T00:50:21.711 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:50:21.743 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:21.765 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:50:21.768 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:21.768 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:50:21.869 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:50:21.898 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:21.918 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:50:21.921 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:21.921 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:50:21.962 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:50:22.021 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:50:22.049 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:22.069 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:50:22.071 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:22.071 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:50:22.172 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:50:22.199 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:22.218 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:50:22.221 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:22.221 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:50:22.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:21.221379+0000 mon.a (mon.0) 2769 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-23T00:50:22.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:21.221443+0000 mon.a (mon.0) 2770 : cluster [DBG] osdmap e492: 8 total, 8 up, 7 in 2024-02-23T00:50:22.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:21.223653+0000 mon.a (mon.0) 2771 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-23T00:50:22.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:21.292094+0000 osd.7 (osd.7) 247 : cluster [DBG] 62.6 scrub starts 2024-02-23T00:50:22.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:21.294448+0000 osd.7 (osd.7) 248 : cluster [DBG] 62.6 scrub ok 2024-02-23T00:50:22.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:21.326891+0000 mgr.y (mgr.14517) 465 : cluster [DBG] pgmap v742: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:50:22.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:22.222589+0000 mon.a (mon.0) 2772 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-23T00:50:22.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:22.222657+0000 mon.a (mon.0) 2773 : cluster [DBG] osdmap e493: 8 total, 8 up, 7 in 2024-02-23T00:50:22.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:21.221379+0000 mon.a (mon.0) 2769 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-23T00:50:22.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:21.221443+0000 mon.a (mon.0) 2770 : cluster [DBG] osdmap e492: 8 total, 8 up, 7 in 2024-02-23T00:50:22.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:21.223653+0000 mon.a (mon.0) 2771 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-23T00:50:22.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:21.292094+0000 osd.7 (osd.7) 247 : cluster [DBG] 62.6 scrub starts 2024-02-23T00:50:22.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:21.294448+0000 osd.7 (osd.7) 248 : cluster [DBG] 62.6 scrub ok 2024-02-23T00:50:22.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:21.326891+0000 mgr.y (mgr.14517) 465 : cluster [DBG] pgmap v742: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:50:22.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:22.222589+0000 mon.a (mon.0) 2772 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-23T00:50:22.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:22.222657+0000 mon.a (mon.0) 2773 : cluster [DBG] osdmap e493: 8 total, 8 up, 7 in 2024-02-23T00:50:22.322 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:50:22.346 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:21.221379+0000 mon.a (mon.0) 2769 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-23T00:50:22.346 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:21.221443+0000 mon.a (mon.0) 2770 : cluster [DBG] osdmap e492: 8 total, 8 up, 7 in 2024-02-23T00:50:22.346 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:21.223653+0000 mon.a (mon.0) 2771 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-23T00:50:22.346 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:21.292094+0000 osd.7 (osd.7) 247 : cluster [DBG] 62.6 scrub starts 2024-02-23T00:50:22.347 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:21.294448+0000 osd.7 (osd.7) 248 : cluster [DBG] 62.6 scrub ok 2024-02-23T00:50:22.347 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:21.326891+0000 mgr.y (mgr.14517) 465 : cluster [DBG] pgmap v742: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:50:22.347 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:22.222589+0000 mon.a (mon.0) 2772 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-23T00:50:22.347 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:22.222657+0000 mon.a (mon.0) 2773 : cluster [DBG] osdmap e493: 8 total, 8 up, 7 in 2024-02-23T00:50:22.348 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:22.366 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:50:22.368 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:22.369 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:50:22.470 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:50:22.497 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:22.515 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:50:22.517 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:22.518 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:50:22.619 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:50:22.643 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:50:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:50:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:50:22.645 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:22.663 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:50:22.665 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:22.666 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:50:22.767 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:50:22.797 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:22.944 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:50:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:50:22.946 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:50:22.948 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:22.950 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:50:23.051 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:50:23.078 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:23.278 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:50:23.281 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:23.281 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:50:23.382 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:50:23.411 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:23.431 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:50:23.433 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:23.434 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:50:23.535 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:50:23.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:21.744867+0000 osd.5 (osd.5) 220 : cluster [DBG] 2.3 scrub starts 2024-02-23T00:50:23.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:21.745952+0000 osd.5 (osd.5) 221 : cluster [DBG] 2.3 scrub ok 2024-02-23T00:50:23.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:22.243346+0000 mon.a (mon.0) 2774 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-23T00:50:23.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:22.296608+0000 osd.7 (osd.7) 249 : cluster [DBG] 2.5 scrub starts 2024-02-23T00:50:23.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:22.299530+0000 osd.7 (osd.7) 250 : cluster [DBG] 2.5 scrub ok 2024-02-23T00:50:23.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:21.744867+0000 osd.5 (osd.5) 220 : cluster [DBG] 2.3 scrub starts 2024-02-23T00:50:23.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:21.745952+0000 osd.5 (osd.5) 221 : cluster [DBG] 2.3 scrub ok 2024-02-23T00:50:23.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:22.243346+0000 mon.a (mon.0) 2774 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-23T00:50:23.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:22.296608+0000 osd.7 (osd.7) 249 : cluster [DBG] 2.5 scrub starts 2024-02-23T00:50:23.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:22.299530+0000 osd.7 (osd.7) 250 : cluster [DBG] 2.5 scrub ok 2024-02-23T00:50:23.568 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:23.664 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:50:23.664 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-02-23T00:50:23.664 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-02-23T00:50:23.664 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-02-23T00:50:23.665 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:50:23.668 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:50:23.671 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:23.671 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:50:23.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:21.744867+0000 osd.5 (osd.5) 220 : cluster [DBG] 2.3 scrub starts 2024-02-23T00:50:23.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:21.745952+0000 osd.5 (osd.5) 221 : cluster [DBG] 2.3 scrub ok 2024-02-23T00:50:23.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:22.243346+0000 mon.a (mon.0) 2774 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-23T00:50:23.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:22.296608+0000 osd.7 (osd.7) 249 : cluster [DBG] 2.5 scrub starts 2024-02-23T00:50:23.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:22.299530+0000 osd.7 (osd.7) 250 : cluster [DBG] 2.5 scrub ok 2024-02-23T00:50:23.772 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:50:23.800 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:24.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:22.698941+0000 mgr.y (mgr.14517) 466 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:50:24.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:22.720347+0000 osd.1 (osd.1) 223 : cluster [DBG] 62.11 scrub starts 2024-02-23T00:50:24.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:22.723812+0000 osd.1 (osd.1) 224 : cluster [DBG] 62.11 scrub ok 2024-02-23T00:50:24.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:23.237941+0000 mon.a (mon.0) 2775 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-23T00:50:24.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:23.238014+0000 mon.a (mon.0) 2776 : cluster [DBG] osdmap e494: 8 total, 8 up, 7 in 2024-02-23T00:50:24.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:23.255194+0000 mon.a (mon.0) 2777 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-23T00:50:24.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:23.327333+0000 mgr.y (mgr.14517) 467 : cluster [DBG] pgmap v745: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-02-23T00:50:24.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:23.538472+0000 osd.0 (osd.0) 213 : cluster [DBG] 4.1e scrub starts 2024-02-23T00:50:24.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:23.540223+0000 osd.0 (osd.0) 214 : cluster [DBG] 4.1e scrub ok 2024-02-23T00:50:24.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:23.724033+0000 osd.5 (osd.5) 222 : cluster [DBG] 18.1e scrub starts 2024-02-23T00:50:24.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:23.727164+0000 osd.5 (osd.5) 223 : cluster [DBG] 18.1e scrub ok 2024-02-23T00:50:24.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:22.698941+0000 mgr.y (mgr.14517) 466 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:50:24.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:22.720347+0000 osd.1 (osd.1) 223 : cluster [DBG] 62.11 scrub starts 2024-02-23T00:50:24.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:22.723812+0000 osd.1 (osd.1) 224 : cluster [DBG] 62.11 scrub ok 2024-02-23T00:50:24.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:23.237941+0000 mon.a (mon.0) 2775 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-23T00:50:24.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:23.238014+0000 mon.a (mon.0) 2776 : cluster [DBG] osdmap e494: 8 total, 8 up, 7 in 2024-02-23T00:50:24.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:23.255194+0000 mon.a (mon.0) 2777 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-23T00:50:24.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:23.327333+0000 mgr.y (mgr.14517) 467 : cluster [DBG] pgmap v745: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-02-23T00:50:24.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:23.538472+0000 osd.0 (osd.0) 213 : cluster [DBG] 4.1e scrub starts 2024-02-23T00:50:24.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:23.540223+0000 osd.0 (osd.0) 214 : cluster [DBG] 4.1e scrub ok 2024-02-23T00:50:24.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:23.724033+0000 osd.5 (osd.5) 222 : cluster [DBG] 18.1e scrub starts 2024-02-23T00:50:24.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:23.727164+0000 osd.5 (osd.5) 223 : cluster [DBG] 18.1e scrub ok 2024-02-23T00:50:24.439 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.1 dump_historic_ops 2024-02-23T00:50:24.554 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:50:24.557 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:24.557 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:50:24.658 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:50:24.690 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:24.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:22.698941+0000 mgr.y (mgr.14517) 466 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:50:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:22.720347+0000 osd.1 (osd.1) 223 : cluster [DBG] 62.11 scrub starts 2024-02-23T00:50:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:22.723812+0000 osd.1 (osd.1) 224 : cluster [DBG] 62.11 scrub ok 2024-02-23T00:50:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:23.237941+0000 mon.a (mon.0) 2775 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-23T00:50:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:23.238014+0000 mon.a (mon.0) 2776 : cluster [DBG] osdmap e494: 8 total, 8 up, 7 in 2024-02-23T00:50:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:23.255194+0000 mon.a (mon.0) 2777 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-23T00:50:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:23.327333+0000 mgr.y (mgr.14517) 467 : cluster [DBG] pgmap v745: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-02-23T00:50:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:23.538472+0000 osd.0 (osd.0) 213 : cluster [DBG] 4.1e scrub starts 2024-02-23T00:50:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:23.540223+0000 osd.0 (osd.0) 214 : cluster [DBG] 4.1e scrub ok 2024-02-23T00:50:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:23.724033+0000 osd.5 (osd.5) 222 : cluster [DBG] 18.1e scrub starts 2024-02-23T00:50:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:23.727164+0000 osd.5 (osd.5) 223 : cluster [DBG] 18.1e scrub ok 2024-02-23T00:50:24.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.867+0000 7f4c8fd35700 1 -- 172.21.15.47:0/3619905295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c880f8420 msgr2=0x7f4c88104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:24.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.867+0000 7f4c8fd35700 1 --2- 172.21.15.47:0/3619905295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c880f8420 0x7f4c88104a20 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 rx=0x7f4c80009be0 tx=0x7f4c80004d10).stop 2024-02-23T00:50:24.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.867+0000 7f4c8fd35700 1 -- 172.21.15.47:0/3619905295 shutdown_connections 2024-02-23T00:50:24.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.867+0000 7f4c8fd35700 1 --2- 172.21.15.47:0/3619905295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c880f8420 0x7f4c88104a20 unknown :-1 s=CLOSED pgs=2557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:24.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.867+0000 7f4c8fd35700 1 --2- 172.21.15.47:0/3619905295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c880fdbb0 0x7f4c880fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:24.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.867+0000 7f4c8fd35700 1 --2- 172.21.15.47:0/3619905295 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4c880fe510 0x7f4c880f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:24.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.867+0000 7f4c8fd35700 1 -- 172.21.15.47:0/3619905295 >> 172.21.15.47:0/3619905295 conn(0x7f4c880f3800 msgr2=0x7f4c880f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:24.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.867+0000 7f4c8fd35700 1 -- 172.21.15.47:0/3619905295 shutdown_connections 2024-02-23T00:50:24.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.867+0000 7f4c8fd35700 1 -- 172.21.15.47:0/3619905295 wait complete. 2024-02-23T00:50:24.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.867+0000 7f4c8fd35700 1 librados: starting msgr at 2024-02-23T00:50:24.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.867+0000 7f4c8fd35700 1 librados: starting objecter 2024-02-23T00:50:24.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.868+0000 7f4c8fd35700 1 Processor -- start 2024-02-23T00:50:24.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.868+0000 7f4c8fd35700 1 -- start start 2024-02-23T00:50:24.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.868+0000 7f4c8fd35700 1 librados: setting wanted keys 2024-02-23T00:50:24.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.868+0000 7f4c8fd35700 1 librados: calling monclient init 2024-02-23T00:50:24.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.868+0000 7f4c8fd35700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c880f8420 0x7f4c8810dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:24.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.868+0000 7f4c8fd35700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4c880fdbb0 0x7f4c8810e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:24.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.868+0000 7f4c8fd35700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c880fe510 0x7f4c88107e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:24.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.868+0000 7f4c8fd35700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c88108340 con 0x7f4c880f8420 2024-02-23T00:50:24.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.868+0000 7f4c8fd35700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c881084b0 con 0x7f4c880fe510 2024-02-23T00:50:24.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.868+0000 7f4c8fd35700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c88108670 con 0x7f4c880fdbb0 2024-02-23T00:50:24.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.868+0000 7f4c8dad1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c880f8420 0x7f4c8810dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:24.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.868+0000 7f4c8dad1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c880f8420 0x7f4c8810dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39602/0 (socket says 172.21.15.47:39602) 2024-02-23T00:50:24.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.868+0000 7f4c8dad1700 1 -- 172.21.15.47:0/3692222110 learned_addr learned my addr 172.21.15.47:0/3692222110 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:50:24.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.868+0000 7f4c8d2d0700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4c880fdbb0 0x7f4c8810e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:24.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.868+0000 7f4c8e2d2700 1 --2- 172.21.15.47:0/3692222110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c880fe510 0x7f4c88107e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:24.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.868+0000 7f4c8dad1700 1 -- 172.21.15.47:0/3692222110 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4c880fdbb0 msgr2=0x7f4c8810e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:24.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.868+0000 7f4c8dad1700 1 --2- 172.21.15.47:0/3692222110 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4c880fdbb0 0x7f4c8810e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:24.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.868+0000 7f4c8dad1700 1 -- 172.21.15.47:0/3692222110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c880fe510 msgr2=0x7f4c88107e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:24.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.868+0000 7f4c8dad1700 1 --2- 172.21.15.47:0/3692222110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c880fe510 0x7f4c88107e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:24.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.868+0000 7f4c8dad1700 1 -- 172.21.15.47:0/3692222110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c84005040 con 0x7f4c880f8420 2024-02-23T00:50:24.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.868+0000 7f4c8e2d2700 1 --2- 172.21.15.47:0/3692222110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c880fe510 0x7f4c88107e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:50:24.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.869+0000 7f4c8dad1700 1 --2- 172.21.15.47:0/3692222110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c880f8420 0x7f4c8810dc60 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 rx=0x7f4c780041d0 tx=0x7f4c78009f60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:24.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.869+0000 7f4c7effd700 10 client.?.objecter ms_handle_connect 0x7f4c880f8420 2024-02-23T00:50:24.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.869+0000 7f4c7effd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:50:24.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.869+0000 7f4c7effd700 1 -- 172.21.15.47:0/3692222110 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c78007b40 con 0x7f4c880f8420 2024-02-23T00:50:24.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.869+0000 7f4c8fd35700 1 -- 172.21.15.47:0/3692222110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c800097c0 con 0x7f4c880f8420 2024-02-23T00:50:24.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.869+0000 7f4c7effd700 1 -- 172.21.15.47:0/3692222110 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f4c7800f040 con 0x7f4c880f8420 2024-02-23T00:50:24.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.869+0000 7f4c7effd700 1 -- 172.21.15.47:0/3692222110 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c78014690 con 0x7f4c880f8420 2024-02-23T00:50:24.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.869+0000 7f4c8fd35700 10 client.31218.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:50:24.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.869+0000 7f4c8fd35700 1 -- 172.21.15.47:0/3692222110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c88114a90 con 0x7f4c880f8420 2024-02-23T00:50:24.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.869+0000 7f4c8fd35700 1 librados: init done 2024-02-23T00:50:24.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.870+0000 7f4c7effd700 1 -- 172.21.15.47:0/3692222110 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f4c78009110 con 0x7f4c880f8420 2024-02-23T00:50:24.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.870+0000 7f4c8fd35700 1 -- 172.21.15.47:0/3692222110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c88106ed0 con 0x7f4c880f8420 2024-02-23T00:50:24.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.870+0000 7f4c7effd700 1 --2- 172.21.15.47:0/3692222110 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4c74062cc0 0x7f4c74065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:24.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.870+0000 7f4c7effd700 1 -- 172.21.15.47:0/3692222110 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(495..495 src has 48..495) v4 ==== 9287+0+0 (secure 0 0 0) 0x7f4c7807d730 con 0x7f4c880f8420 2024-02-23T00:50:24.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.870+0000 7f4c7effd700 10 client.31218.objecter ms_dispatch 0x7f4c88001550 osd_map(495..495 src has 48..495) v4 2024-02-23T00:50:24.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.870+0000 7f4c7effd700 3 client.31218.objecter handle_osd_map got epochs [495,495] > 0 2024-02-23T00:50:24.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.870+0000 7f4c7effd700 3 client.31218.objecter handle_osd_map decoding full epoch 495 2024-02-23T00:50:24.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.870+0000 7f4c8d2d0700 1 --2- 172.21.15.47:0/3692222110 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4c74062cc0 0x7f4c74065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:24.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.871+0000 7f4c7effd700 20 client.31218.objecter dump_active .. 0 homeless 2024-02-23T00:50:24.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.871+0000 7f4c7effd700 10 client.31218.objecter ms_handle_connect 0x7f4c74062cc0 2024-02-23T00:50:24.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.871+0000 7f4c8d2d0700 1 --2- 172.21.15.47:0/3692222110 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4c74062cc0 0x7f4c74065180 secure :-1 s=READY pgs=3099 cs=0 l=1 rev1=1 rx=0x7f4c881095a0 tx=0x7f4c8400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:24.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:24.875+0000 7f4c7effd700 1 -- 172.21.15.47:0/3692222110 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f4c7804cd80 con 0x7f4c880f8420 2024-02-23T00:50:25.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:25.060+0000 7f4c8fd35700 1 -- 172.21.15.47:0/3692222110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7f4c8805fac0 con 0x7f4c880f8420 2024-02-23T00:50:25.244 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:50:25.245 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:50:25.245 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:50:25.249 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:25.250 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:50:25.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:25.251+0000 7f4c7effd700 1 -- 172.21.15.47:0/3692222110 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v496) v1 ==== 101+0+0 (secure 0 0 0) 0x7f4c780503a0 con 0x7f4c880f8420 2024-02-23T00:50:25.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:25.253+0000 7f4c8fd35700 10 librados: watch_flush enter 2024-02-23T00:50:25.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:25.253+0000 7f4c8fd35700 10 librados: watch_flush exit 2024-02-23T00:50:25.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:25.253+0000 7f4c8fd35700 20 client.31218.objecter shutdown clearing up homeless session... 2024-02-23T00:50:25.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:25.253+0000 7f4c8fd35700 10 client.31218.objecter successfully canceled tick 2024-02-23T00:50:25.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:25.255+0000 7f4c8fd35700 1 -- 172.21.15.47:0/3692222110 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4c74062cc0 msgr2=0x7f4c74065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:25.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:25.255+0000 7f4c8fd35700 1 --2- 172.21.15.47:0/3692222110 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4c74062cc0 0x7f4c74065180 secure :-1 s=READY pgs=3099 cs=0 l=1 rev1=1 rx=0x7f4c881095a0 tx=0x7f4c8400b040).stop 2024-02-23T00:50:25.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:25.255+0000 7f4c8fd35700 1 -- 172.21.15.47:0/3692222110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c880f8420 msgr2=0x7f4c8810dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:25.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:25.255+0000 7f4c8fd35700 1 --2- 172.21.15.47:0/3692222110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c880f8420 0x7f4c8810dc60 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 rx=0x7f4c780041d0 tx=0x7f4c78009f60).stop 2024-02-23T00:50:25.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:25.257+0000 7f4c8fd35700 1 -- 172.21.15.47:0/3692222110 shutdown_connections 2024-02-23T00:50:25.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:25.257+0000 7f4c8fd35700 1 --2- 172.21.15.47:0/3692222110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c880fe510 0x7f4c88107e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:25.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:25.257+0000 7f4c8fd35700 1 --2- 172.21.15.47:0/3692222110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c880f8420 0x7f4c8810dc60 unknown :-1 s=CLOSED pgs=2558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:25.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:25.257+0000 7f4c8fd35700 1 --2- 172.21.15.47:0/3692222110 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4c74062cc0 0x7f4c74065180 secure :-1 s=CLOSED pgs=3099 cs=0 l=1 rev1=1 rx=0x7f4c881095a0 tx=0x7f4c8400b040).stop 2024-02-23T00:50:25.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:25.257+0000 7f4c8fd35700 1 --2- 172.21.15.47:0/3692222110 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4c880fdbb0 0x7f4c8810e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:25.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:25.257+0000 7f4c8fd35700 1 -- 172.21.15.47:0/3692222110 >> 172.21.15.47:0/3692222110 conn(0x7f4c880f3800 msgr2=0x7f4c880f50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:25.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:25.258+0000 7f4c8fd35700 1 -- 172.21.15.47:0/3692222110 shutdown_connections 2024-02-23T00:50:25.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:25.260+0000 7f4c8fd35700 1 -- 172.21.15.47:0/3692222110 wait complete. 2024-02-23T00:50:25.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:25.260+0000 7f4c8fd35700 1 librados: shutdown 2024-02-23T00:50:25.265 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is unset 2024-02-23T00:50:25.350 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:50:25.385 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:25.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:24.249044+0000 mon.a (mon.0) 2778 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-23T00:50:25.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:24.249110+0000 mon.a (mon.0) 2779 : cluster [DBG] osdmap e495: 8 total, 8 up, 7 in 2024-02-23T00:50:25.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:24.260939+0000 mon.a (mon.0) 2780 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-23T00:50:25.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:24.294453+0000 osd.7 (osd.7) 251 : cluster [DBG] 1.0 scrub starts 2024-02-23T00:50:25.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:24.297568+0000 osd.7 (osd.7) 252 : cluster [DBG] 1.0 scrub ok 2024-02-23T00:50:25.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:24.530478+0000 osd.4 (osd.4) 217 : cluster [DBG] 4.1 scrub starts 2024-02-23T00:50:25.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:24.532259+0000 osd.4 (osd.4) 218 : cluster [DBG] 4.1 scrub ok 2024-02-23T00:50:25.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:24.678915+0000 osd.5 (osd.5) 224 : cluster [DBG] 18.15 scrub starts 2024-02-23T00:50:25.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:24.682034+0000 osd.5 (osd.5) 225 : cluster [DBG] 18.15 scrub ok 2024-02-23T00:50:25.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:25.061687+0000 mon.a (mon.0) 2781 : audit [INF] from='client.? 172.21.15.47:0/3692222110' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:50:25.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:24.249044+0000 mon.a (mon.0) 2778 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-23T00:50:25.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:24.249110+0000 mon.a (mon.0) 2779 : cluster [DBG] osdmap e495: 8 total, 8 up, 7 in 2024-02-23T00:50:25.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:24.260939+0000 mon.a (mon.0) 2780 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-23T00:50:25.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:24.294453+0000 osd.7 (osd.7) 251 : cluster [DBG] 1.0 scrub starts 2024-02-23T00:50:25.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:24.297568+0000 osd.7 (osd.7) 252 : cluster [DBG] 1.0 scrub ok 2024-02-23T00:50:25.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:24.530478+0000 osd.4 (osd.4) 217 : cluster [DBG] 4.1 scrub starts 2024-02-23T00:50:25.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:24.532259+0000 osd.4 (osd.4) 218 : cluster [DBG] 4.1 scrub ok 2024-02-23T00:50:25.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:24.678915+0000 osd.5 (osd.5) 224 : cluster [DBG] 18.15 scrub starts 2024-02-23T00:50:25.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:24.682034+0000 osd.5 (osd.5) 225 : cluster [DBG] 18.15 scrub ok 2024-02-23T00:50:25.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:25.061687+0000 mon.a (mon.0) 2781 : audit [INF] from='client.? 172.21.15.47:0/3692222110' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:50:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:24.249044+0000 mon.a (mon.0) 2778 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-23T00:50:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:24.249110+0000 mon.a (mon.0) 2779 : cluster [DBG] osdmap e495: 8 total, 8 up, 7 in 2024-02-23T00:50:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:24.260939+0000 mon.a (mon.0) 2780 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-23T00:50:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:24.294453+0000 osd.7 (osd.7) 251 : cluster [DBG] 1.0 scrub starts 2024-02-23T00:50:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:24.297568+0000 osd.7 (osd.7) 252 : cluster [DBG] 1.0 scrub ok 2024-02-23T00:50:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:24.530478+0000 osd.4 (osd.4) 217 : cluster [DBG] 4.1 scrub starts 2024-02-23T00:50:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:24.532259+0000 osd.4 (osd.4) 218 : cluster [DBG] 4.1 scrub ok 2024-02-23T00:50:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:24.678915+0000 osd.5 (osd.5) 224 : cluster [DBG] 18.15 scrub starts 2024-02-23T00:50:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:24.682034+0000 osd.5 (osd.5) 225 : cluster [DBG] 18.15 scrub ok 2024-02-23T00:50:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:25.061687+0000 mon.a (mon.0) 2781 : audit [INF] from='client.? 172.21.15.47:0/3692222110' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:50:26.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:25.077828+0000 osd.3 (osd.3) 236 : cluster [DBG] 4.14 scrub starts 2024-02-23T00:50:26.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:25.079786+0000 osd.3 (osd.3) 237 : cluster [DBG] 4.14 scrub ok 2024-02-23T00:50:26.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:25.244491+0000 mon.a (mon.0) 2782 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:50:26.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:25.249448+0000 mon.a (mon.0) 2783 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-23T00:50:26.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:25.249547+0000 mon.a (mon.0) 2784 : audit [INF] from='client.? 172.21.15.47:0/3692222110' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:50:26.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:25.249602+0000 mon.a (mon.0) 2785 : cluster [DBG] osdmap e496: 8 total, 8 up, 7 in 2024-02-23T00:50:26.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:25.294399+0000 mon.a (mon.0) 2786 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:50:26.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:25.294713+0000 mon.a (mon.0) 2787 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-77"}]: dispatch 2024-02-23T00:50:26.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:25.327734+0000 mgr.y (mgr.14517) 468 : cluster [DBG] pgmap v748: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-23T00:50:26.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:25.574222+0000 osd.4 (osd.4) 219 : cluster [DBG] 5.1 scrub starts 2024-02-23T00:50:26.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:25.575233+0000 osd.4 (osd.4) 220 : cluster [DBG] 5.1 scrub ok 2024-02-23T00:50:26.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:25.077828+0000 osd.3 (osd.3) 236 : cluster [DBG] 4.14 scrub starts 2024-02-23T00:50:26.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:25.079786+0000 osd.3 (osd.3) 237 : cluster [DBG] 4.14 scrub ok 2024-02-23T00:50:26.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:25.244491+0000 mon.a (mon.0) 2782 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:50:26.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:25.249448+0000 mon.a (mon.0) 2783 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-23T00:50:26.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:25.249547+0000 mon.a (mon.0) 2784 : audit [INF] from='client.? 172.21.15.47:0/3692222110' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:50:26.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:25.249602+0000 mon.a (mon.0) 2785 : cluster [DBG] osdmap e496: 8 total, 8 up, 7 in 2024-02-23T00:50:26.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:25.294399+0000 mon.a (mon.0) 2786 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:50:26.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:25.294713+0000 mon.a (mon.0) 2787 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-77"}]: dispatch 2024-02-23T00:50:26.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:25.327734+0000 mgr.y (mgr.14517) 468 : cluster [DBG] pgmap v748: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-23T00:50:26.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:25.574222+0000 osd.4 (osd.4) 219 : cluster [DBG] 5.1 scrub starts 2024-02-23T00:50:26.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:25.575233+0000 osd.4 (osd.4) 220 : cluster [DBG] 5.1 scrub ok 2024-02-23T00:50:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.678+0000 7f8801ae5700 1 -- 172.21.15.47:0/4062296137 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87fc0f8420 msgr2=0x7f87fc104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.678+0000 7f8801ae5700 1 --2- 172.21.15.47:0/4062296137 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87fc0f8420 0x7f87fc104a20 secure :-1 s=READY pgs=2709 cs=0 l=1 rev1=1 rx=0x7f87f4009be0 tx=0x7f87f4004d10).stop 2024-02-23T00:50:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.678+0000 7f8801ae5700 1 -- 172.21.15.47:0/4062296137 shutdown_connections 2024-02-23T00:50:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.678+0000 7f8801ae5700 1 --2- 172.21.15.47:0/4062296137 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87fc0f8420 0x7f87fc104a20 unknown :-1 s=CLOSED pgs=2709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.678+0000 7f8801ae5700 1 --2- 172.21.15.47:0/4062296137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87fc0fdbb0 0x7f87fc0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.678+0000 7f8801ae5700 1 --2- 172.21.15.47:0/4062296137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87fc0fe510 0x7f87fc0f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.678+0000 7f8801ae5700 1 -- 172.21.15.47:0/4062296137 >> 172.21.15.47:0/4062296137 conn(0x7f87fc0f3800 msgr2=0x7f87fc0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.678+0000 7f8801ae5700 1 -- 172.21.15.47:0/4062296137 shutdown_connections 2024-02-23T00:50:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.678+0000 7f8801ae5700 1 -- 172.21.15.47:0/4062296137 wait complete. 2024-02-23T00:50:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.679+0000 7f8801ae5700 1 librados: starting msgr at 2024-02-23T00:50:26.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.679+0000 7f8801ae5700 1 librados: starting objecter 2024-02-23T00:50:26.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.679+0000 7f8801ae5700 1 Processor -- start 2024-02-23T00:50:26.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.679+0000 7f8801ae5700 1 -- start start 2024-02-23T00:50:26.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.679+0000 7f8801ae5700 1 librados: setting wanted keys 2024-02-23T00:50:26.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.679+0000 7f8801ae5700 1 librados: calling monclient init 2024-02-23T00:50:26.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.680+0000 7f8801ae5700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87fc0f8420 0x7f87fc10dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:26.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.680+0000 7f8801ae5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87fc0fdbb0 0x7f87fc10e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:26.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.680+0000 7f8801ae5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87fc0fe510 0x7f87fc107d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:26.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.680+0000 7f8801ae5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc108280 con 0x7f87fc0fdbb0 2024-02-23T00:50:26.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.680+0000 7f8801ae5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc1083f0 con 0x7f87fc0fe510 2024-02-23T00:50:26.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.680+0000 7f8801ae5700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc1085b0 con 0x7f87fc0f8420 2024-02-23T00:50:26.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.680+0000 7f87fb7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87fc0f8420 0x7f87fc10dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:26.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.680+0000 7f87fb7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87fc0f8420 0x7f87fc10dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:45566/0 (socket says 172.21.15.47:45566) 2024-02-23T00:50:26.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.680+0000 7f87fb7fe700 1 -- 172.21.15.47:0/2443547124 learned_addr learned my addr 172.21.15.47:0/2443547124 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:50:26.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.680+0000 7f87f2ffd700 1 --2- 172.21.15.47:0/2443547124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87fc0fdbb0 0x7f87fc10e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:26.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.680+0000 7f87fbfff700 1 --2- 172.21.15.47:0/2443547124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87fc0fe510 0x7f87fc107d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:26.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.680+0000 7f87fb7fe700 1 -- 172.21.15.47:0/2443547124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87fc0fe510 msgr2=0x7f87fc107d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:26.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.680+0000 7f87fb7fe700 1 --2- 172.21.15.47:0/2443547124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87fc0fe510 0x7f87fc107d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:26.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.680+0000 7f87fb7fe700 1 -- 172.21.15.47:0/2443547124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87fc0fdbb0 msgr2=0x7f87fc10e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:26.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.680+0000 7f87fb7fe700 1 --2- 172.21.15.47:0/2443547124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87fc0fdbb0 0x7f87fc10e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:26.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.680+0000 7f87fb7fe700 1 -- 172.21.15.47:0/2443547124 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f40097c0 con 0x7f87fc0f8420 2024-02-23T00:50:26.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.680+0000 7f87f2ffd700 1 --2- 172.21.15.47:0/2443547124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87fc0fdbb0 0x7f87fc10e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:50:26.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.680+0000 7f87fb7fe700 1 --2- 172.21.15.47:0/2443547124 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87fc0f8420 0x7f87fc10dba0 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 rx=0x7f87ec0027f0 tx=0x7f87ec002b00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:26.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.680+0000 7f87f9775700 10 client.?.objecter ms_handle_connect 0x7f87fc0f8420 2024-02-23T00:50:26.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.680+0000 7f87f9775700 10 client.?.objecter resend_mon_ops 2024-02-23T00:50:26.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.680+0000 7f87f9775700 1 -- 172.21.15.47:0/2443547124 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ec007b40 con 0x7f87fc0f8420 2024-02-23T00:50:26.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.680+0000 7f87f9775700 1 -- 172.21.15.47:0/2443547124 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f87ec00f040 con 0x7f87fc0f8420 2024-02-23T00:50:26.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.680+0000 7f8801ae5700 1 -- 172.21.15.47:0/2443547124 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87fc108890 con 0x7f87fc0f8420 2024-02-23T00:50:26.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.680+0000 7f87f9775700 1 -- 172.21.15.47:0/2443547124 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ec00d660 con 0x7f87fc0f8420 2024-02-23T00:50:26.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.681+0000 7f8801ae5700 10 client.41549.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:50:26.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.681+0000 7f8801ae5700 1 -- 172.21.15.47:0/2443547124 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87fc108c10 con 0x7f87fc0f8420 2024-02-23T00:50:26.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.681+0000 7f8801ae5700 1 librados: init done 2024-02-23T00:50:26.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.681+0000 7f8801ae5700 1 -- 172.21.15.47:0/2443547124 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87fc106ed0 con 0x7f87fc0f8420 2024-02-23T00:50:26.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.682+0000 7f87f9775700 1 -- 172.21.15.47:0/2443547124 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f87ec009110 con 0x7f87fc0f8420 2024-02-23T00:50:26.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.683+0000 7f87f9775700 1 --2- 172.21.15.47:0/2443547124 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f87e8062cc0 0x7f87e8065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:26.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.683+0000 7f87f9775700 1 -- 172.21.15.47:0/2443547124 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(497..497 src has 48..497) v4 ==== 8835+0+0 (secure 0 0 0) 0x7f87ec07d740 con 0x7f87fc0f8420 2024-02-23T00:50:26.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.683+0000 7f87f9775700 10 client.41549.objecter ms_dispatch 0x7f87fc001550 osd_map(497..497 src has 48..497) v4 2024-02-23T00:50:26.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.683+0000 7f87f9775700 3 client.41549.objecter handle_osd_map got epochs [497,497] > 0 2024-02-23T00:50:26.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.683+0000 7f87f9775700 3 client.41549.objecter handle_osd_map decoding full epoch 497 2024-02-23T00:50:26.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.683+0000 7f87f2ffd700 1 --2- 172.21.15.47:0/2443547124 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f87e8062cc0 0x7f87e8065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:26.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.683+0000 7f87f9775700 20 client.41549.objecter dump_active .. 0 homeless 2024-02-23T00:50:26.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.683+0000 7f87f9775700 10 client.41549.objecter ms_handle_connect 0x7f87e8062cc0 2024-02-23T00:50:26.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.683+0000 7f87f2ffd700 1 --2- 172.21.15.47:0/2443547124 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f87e8062cc0 0x7f87e8065180 secure :-1 s=READY pgs=3100 cs=0 l=1 rev1=1 rx=0x7f87fc1094e0 tx=0x7f87e4009d60).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:26.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.687+0000 7f87f9775700 1 -- 172.21.15.47:0/2443547124 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f87ec04ce70 con 0x7f87fc0f8420 2024-02-23T00:50:26.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:25.077828+0000 osd.3 (osd.3) 236 : cluster [DBG] 4.14 scrub starts 2024-02-23T00:50:26.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:25.079786+0000 osd.3 (osd.3) 237 : cluster [DBG] 4.14 scrub ok 2024-02-23T00:50:26.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:25.244491+0000 mon.a (mon.0) 2782 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:50:26.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:25.249448+0000 mon.a (mon.0) 2783 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-77","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-23T00:50:26.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:25.249547+0000 mon.a (mon.0) 2784 : audit [INF] from='client.? 172.21.15.47:0/3692222110' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:50:26.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:25.249602+0000 mon.a (mon.0) 2785 : cluster [DBG] osdmap e496: 8 total, 8 up, 7 in 2024-02-23T00:50:26.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:25.294399+0000 mon.a (mon.0) 2786 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:50:26.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:25.294713+0000 mon.a (mon.0) 2787 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-77"}]: dispatch 2024-02-23T00:50:26.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:25.327734+0000 mgr.y (mgr.14517) 468 : cluster [DBG] pgmap v748: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-23T00:50:26.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:25.574222+0000 osd.4 (osd.4) 219 : cluster [DBG] 5.1 scrub starts 2024-02-23T00:50:26.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:25.575233+0000 osd.4 (osd.4) 220 : cluster [DBG] 5.1 scrub ok 2024-02-23T00:50:26.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:26.880+0000 7f8801ae5700 1 -- 172.21.15.47:0/2443547124 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd reweight", "id": 3, "weight": 1.0} v 0) v1 -- 0x7f87fc05faf0 con 0x7f87fc0f8420 2024-02-23T00:50:27.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:27.267+0000 7f87f9775700 1 -- 172.21.15.47:0/2443547124 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 3, "weight": 1.0}]=0 reweighted osd.3 to 1 (10000) v498) v1 ==== 113+0+0 (secure 0 0 0) 0x7f87ec0093c0 con 0x7f87fc0f8420 2024-02-23T00:50:27.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:27.271+0000 7f8801ae5700 10 librados: watch_flush enter 2024-02-23T00:50:27.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:27.271+0000 7f8801ae5700 10 librados: watch_flush exit 2024-02-23T00:50:27.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:27.271+0000 7f8801ae5700 20 client.41549.objecter shutdown clearing up homeless session... 2024-02-23T00:50:27.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:27.271+0000 7f8801ae5700 10 client.41549.objecter successfully canceled tick 2024-02-23T00:50:27.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:27.272+0000 7f8801ae5700 1 -- 172.21.15.47:0/2443547124 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f87e8062cc0 msgr2=0x7f87e8065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:27.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:27.272+0000 7f8801ae5700 1 --2- 172.21.15.47:0/2443547124 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f87e8062cc0 0x7f87e8065180 secure :-1 s=READY pgs=3100 cs=0 l=1 rev1=1 rx=0x7f87fc1094e0 tx=0x7f87e4009d60).stop 2024-02-23T00:50:27.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:27.272+0000 7f8801ae5700 1 -- 172.21.15.47:0/2443547124 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87fc0f8420 msgr2=0x7f87fc10dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:27.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:27.272+0000 7f8801ae5700 1 --2- 172.21.15.47:0/2443547124 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87fc0f8420 0x7f87fc10dba0 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 rx=0x7f87ec0027f0 tx=0x7f87ec002b00).stop 2024-02-23T00:50:27.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:27.272+0000 7f8801ae5700 1 -- 172.21.15.47:0/2443547124 shutdown_connections 2024-02-23T00:50:27.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:27.272+0000 7f8801ae5700 1 --2- 172.21.15.47:0/2443547124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87fc0fe510 0x7f87fc107d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:27.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:27.272+0000 7f8801ae5700 1 --2- 172.21.15.47:0/2443547124 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f87e8062cc0 0x7f87e8065180 unknown :-1 s=CLOSED pgs=3100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:27.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:27.273+0000 7f8801ae5700 1 --2- 172.21.15.47:0/2443547124 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87fc0f8420 0x7f87fc10dba0 unknown :-1 s=CLOSED pgs=2710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:27.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:27.273+0000 7f8801ae5700 1 --2- 172.21.15.47:0/2443547124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87fc0fdbb0 0x7f87fc10e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:27.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:27.273+0000 7f8801ae5700 1 -- 172.21.15.47:0/2443547124 >> 172.21.15.47:0/2443547124 conn(0x7f87fc0f3800 msgr2=0x7f87fc0f50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:27.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:27.273+0000 7f8801ae5700 1 -- 172.21.15.47:0/2443547124 shutdown_connections 2024-02-23T00:50:27.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:27.273+0000 7f8801ae5700 1 -- 172.21.15.47:0/2443547124 wait complete. 2024-02-23T00:50:27.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:27.273+0000 7f8801ae5700 1 librados: shutdown 2024-02-23T00:50:27.278 INFO:teuthology.orchestra.run.smithi047.stderr:reweighted osd.3 to 1 (10000) 2024-02-23T00:50:27.475 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:26.263383+0000 mon.a (mon.0) 2788 : cluster [DBG] osdmap e497: 8 total, 8 up, 7 in 2024-02-23T00:50:27.475 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:26.625886+0000 osd.5 (osd.5) 226 : cluster [DBG] 18.c scrub starts 2024-02-23T00:50:27.475 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:26.630082+0000 osd.5 (osd.5) 227 : cluster [DBG] 18.c scrub ok 2024-02-23T00:50:27.475 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:26.714607+0000 mon.a (mon.0) 2789 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:50:27.475 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:26.882149+0000 mon.c (mon.2) 315 : audit [INF] from='client.? 172.21.15.47:0/2443547124' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-23T00:50:27.475 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:26.882520+0000 mon.a (mon.0) 2790 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-23T00:50:27.476 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:26.263383+0000 mon.a (mon.0) 2788 : cluster [DBG] osdmap e497: 8 total, 8 up, 7 in 2024-02-23T00:50:27.476 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:26.625886+0000 osd.5 (osd.5) 226 : cluster [DBG] 18.c scrub starts 2024-02-23T00:50:27.476 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:26.630082+0000 osd.5 (osd.5) 227 : cluster [DBG] 18.c scrub ok 2024-02-23T00:50:27.476 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:26.714607+0000 mon.a (mon.0) 2789 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:50:27.476 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:26.882149+0000 mon.c (mon.2) 315 : audit [INF] from='client.? 172.21.15.47:0/2443547124' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-23T00:50:27.476 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:26.882520+0000 mon.a (mon.0) 2790 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-23T00:50:27.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:26.263383+0000 mon.a (mon.0) 2788 : cluster [DBG] osdmap e497: 8 total, 8 up, 7 in 2024-02-23T00:50:27.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:26.625886+0000 osd.5 (osd.5) 226 : cluster [DBG] 18.c scrub starts 2024-02-23T00:50:27.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:26.630082+0000 osd.5 (osd.5) 227 : cluster [DBG] 18.c scrub ok 2024-02-23T00:50:27.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:26.714607+0000 mon.a (mon.0) 2789 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:50:27.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:26.882149+0000 mon.c (mon.2) 315 : audit [INF] from='client.? 172.21.15.47:0/2443547124' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-23T00:50:27.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:26.882520+0000 mon.a (mon.0) 2790 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-23T00:50:28.136 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:50:28.139 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:28.140 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:50:28.241 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:50:28.273 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:28.312 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:50:28.314 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:28.315 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:50:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.317+0000 7f060197b700 1 -- 172.21.15.47:0/3660096615 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05fc0fd4a0 msgr2=0x7f05fc0f74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.317+0000 7f060197b700 1 --2- 172.21.15.47:0/3660096615 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05fc0fd4a0 0x7f05fc0f74c0 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 rx=0x7f05e4004870 tx=0x7f05e4009cb0).stop 2024-02-23T00:50:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.318+0000 7f060197b700 1 -- 172.21.15.47:0/3660096615 shutdown_connections 2024-02-23T00:50:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.318+0000 7f060197b700 1 --2- 172.21.15.47:0/3660096615 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f05fc0f7ba0 0x7f05fc103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.318+0000 7f060197b700 1 --2- 172.21.15.47:0/3660096615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05fc0fcb40 0x7f05fc0fcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.318+0000 7f060197b700 1 --2- 172.21.15.47:0/3660096615 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05fc0fd4a0 0x7f05fc0f74c0 unknown :-1 s=CLOSED pgs=2559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.318+0000 7f060197b700 1 -- 172.21.15.47:0/3660096615 >> 172.21.15.47:0/3660096615 conn(0x7f05fc06c370 msgr2=0x7f05fc06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.318+0000 7f060197b700 1 -- 172.21.15.47:0/3660096615 shutdown_connections 2024-02-23T00:50:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.318+0000 7f060197b700 1 -- 172.21.15.47:0/3660096615 wait complete. 2024-02-23T00:50:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.318+0000 7f060197b700 1 librados: starting msgr at 2024-02-23T00:50:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.318+0000 7f060197b700 1 librados: starting objecter 2024-02-23T00:50:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.318+0000 7f060197b700 1 Processor -- start 2024-02-23T00:50:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.318+0000 7f060197b700 1 -- start start 2024-02-23T00:50:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.318+0000 7f060197b700 1 librados: setting wanted keys 2024-02-23T00:50:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.318+0000 7f060197b700 1 librados: calling monclient init 2024-02-23T00:50:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.319+0000 7f060197b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05fc0f7ba0 0x7f05fc10df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.319+0000 7f060197b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f05fc0fcb40 0x7f05fc10e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.319+0000 7f060197b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05fc0fd4a0 0x7f05fc1080b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.319+0000 7f060197b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc1085a0 con 0x7f05fc0f7ba0 2024-02-23T00:50:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.319+0000 7f060197b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc108710 con 0x7f05fc0fd4a0 2024-02-23T00:50:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.319+0000 7f060197b700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc1088d0 con 0x7f05fc0fcb40 2024-02-23T00:50:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.319+0000 7f05fa7fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f05fc0fcb40 0x7f05fc10e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.319+0000 7f05fa7fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f05fc0fcb40 0x7f05fc10e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:45580/0 (socket says 172.21.15.47:45580) 2024-02-23T00:50:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.319+0000 7f05fa7fc700 1 -- 172.21.15.47:0/2141433997 learned_addr learned my addr 172.21.15.47:0/2141433997 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:50:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.319+0000 7f05faffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05fc0f7ba0 0x7f05fc10df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.319+0000 7f05fb7fe700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05fc0fd4a0 0x7f05fc1080b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.319+0000 7f05fa7fc700 1 -- 172.21.15.47:0/2141433997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05fc0fd4a0 msgr2=0x7f05fc1080b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.319+0000 7f05fa7fc700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05fc0fd4a0 0x7f05fc1080b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:28.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.319+0000 7f05fa7fc700 1 -- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05fc0f7ba0 msgr2=0x7f05fc10df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:28.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.319+0000 7f05fa7fc700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05fc0f7ba0 0x7f05fc10df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:28.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.319+0000 7f05fa7fc700 1 -- 172.21.15.47:0/2141433997 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e4005040 con 0x7f05fc0fcb40 2024-02-23T00:50:28.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.319+0000 7f05faffd700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05fc0f7ba0 0x7f05fc10df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:50:28.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.319+0000 7f05fa7fc700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f05fc0fcb40 0x7f05fc10e400 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 rx=0x7f05e4009da0 tx=0x7f05e4009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:28.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.319+0000 7f05f3fff700 10 client.?.objecter ms_handle_connect 0x7f05fc0fcb40 2024-02-23T00:50:28.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.319+0000 7f05f3fff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:50:28.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.319+0000 7f05f3fff700 1 -- 172.21.15.47:0/2141433997 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e400e030 con 0x7f05fc0fcb40 2024-02-23T00:50:28.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.319+0000 7f05f3fff700 1 -- 172.21.15.47:0/2141433997 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f05e4013d70 con 0x7f05fc0fcb40 2024-02-23T00:50:28.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.319+0000 7f05f3fff700 1 -- 172.21.15.47:0/2141433997 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e4018070 con 0x7f05fc0fcb40 2024-02-23T00:50:28.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.319+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05fc108b50 con 0x7f05fc0fcb40 2024-02-23T00:50:28.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.319+0000 7f060197b700 10 client.41555.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:50:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.319+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05fc114d40 con 0x7f05fc0fcb40 2024-02-23T00:50:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.319+0000 7f060197b700 1 librados: init done 2024-02-23T00:50:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.320+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f05fc108f90 con 0x7f05fc0fcb40 2024-02-23T00:50:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.321+0000 7f05f3fff700 1 -- 172.21.15.47:0/2141433997 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f05e4009240 con 0x7f05fc0fcb40 2024-02-23T00:50:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.321+0000 7f05f3fff700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f05e8062cc0 0x7f05e8065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.321+0000 7f05f3fff700 1 -- 172.21.15.47:0/2141433997 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(499..499 src has 48..499) v4 ==== 9220+0+0 (secure 0 0 0) 0x7f05e407e800 con 0x7f05fc0fcb40 2024-02-23T00:50:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.321+0000 7f05f3fff700 10 client.41555.objecter ms_dispatch 0x7f05fc001550 osd_map(499..499 src has 48..499) v4 2024-02-23T00:50:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.321+0000 7f05f3fff700 3 client.41555.objecter handle_osd_map got epochs [499,499] > 0 2024-02-23T00:50:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.321+0000 7f05f3fff700 3 client.41555.objecter handle_osd_map decoding full epoch 499 2024-02-23T00:50:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.321+0000 7f05faffd700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f05e8062cc0 0x7f05e8065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.322+0000 7f05f3fff700 20 client.41555.objecter dump_active .. 0 homeless 2024-02-23T00:50:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.322+0000 7f05f3fff700 1 -- 172.21.15.47:0/2141433997 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v499) v1 ==== 54+0+16 (secure 0 0 0) 0x7f05e4085070 con 0x7f05fc0fcb40 2024-02-23T00:50:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.322+0000 7f060197b700 10 client.41555.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.322+0000 7f060197b700 15 client.41555.objecter _session_command_op_assign -1 1 2024-02-23T00:50:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.322+0000 7f060197b700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f05fc028540 0x7f05fc0fe8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.322+0000 7f060197b700 20 client.41555.objecter _get_session s=0x7f05fc05f930 osd=0 3 2024-02-23T00:50:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.322+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc05f930 osd=0 3 2024-02-23T00:50:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.322+0000 7f060197b700 20 client.41555.objecter _get_session s=0x7f05fc05f930 osd=0 3 2024-02-23T00:50:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.322+0000 7f060197b700 15 client.41555.objecter _session_command_op_remove -1 1 2024-02-23T00:50:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.322+0000 7f060197b700 20 client.41555.objecter get_session s=0x7f05fc05f930 osd=0 3 2024-02-23T00:50:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.322+0000 7f060197b700 15 client.41555.objecter _session_command_op_assign 0 1 2024-02-23T00:50:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.322+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc05f930 osd=0 4 2024-02-23T00:50:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.322+0000 7f060197b700 10 client.41555.objecter _send_command 1 2024-02-23T00:50:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.322+0000 7f05f3fff700 10 client.41555.objecter ms_handle_connect 0x7f05e8062cc0 2024-02-23T00:50:28.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.322+0000 7f05faffd700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f05e8062cc0 0x7f05e8065180 secure :-1 s=READY pgs=3101 cs=0 l=1 rev1=1 rx=0x7f05ec004fd0 tx=0x7f05ec00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:28.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.322+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f05fc05ee80 con 0x7f05fc028540 2024-02-23T00:50:28.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.322+0000 7f05fb7fe700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f05fc028540 0x7f05fc0fe8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:28.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.322+0000 7f05f3fff700 10 client.41555.objecter ms_handle_connect 0x7f05fc028540 2024-02-23T00:50:28.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.322+0000 7f05fb7fe700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f05fc028540 0x7f05fc0fe8d0 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:28.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.323+0000 7f05f3fff700 1 -- 172.21.15.47:0/2141433997 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f05fc05ee80 con 0x7f05fc028540 2024-02-23T00:50:28.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.323+0000 7f05f3fff700 10 client.41555.objecter ms_dispatch 0x7f05fc001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:50:28.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.323+0000 7f05f3fff700 10 client.41555.objecter _finish_command 1 = system:0 2024-02-23T00:50:28.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.323+0000 7f05f3fff700 20 client.41555.objecter put_session s=0x7f05fc05f930 osd=0 4 2024-02-23T00:50:28.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.323+0000 7f05f3fff700 15 client.41555.objecter _session_command_op_remove 0 1 2024-02-23T00:50:28.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.338+0000 7f060197b700 10 client.41555.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:50:28.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.338+0000 7f060197b700 15 client.41555.objecter _session_command_op_assign -1 2 2024-02-23T00:50:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.338+0000 7f060197b700 20 client.41555.objecter _get_session s=0x7f05fc05f930 osd=0 3 2024-02-23T00:50:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.338+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc05f930 osd=0 3 2024-02-23T00:50:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.338+0000 7f060197b700 20 client.41555.objecter _get_session s=0x7f05fc05f930 osd=0 3 2024-02-23T00:50:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.338+0000 7f060197b700 15 client.41555.objecter _session_command_op_remove -1 2 2024-02-23T00:50:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.338+0000 7f060197b700 20 client.41555.objecter get_session s=0x7f05fc05f930 osd=0 3 2024-02-23T00:50:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.338+0000 7f060197b700 15 client.41555.objecter _session_command_op_assign 0 2 2024-02-23T00:50:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.338+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc05f930 osd=0 4 2024-02-23T00:50:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.338+0000 7f060197b700 10 client.41555.objecter _send_command 2 2024-02-23T00:50:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.338+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f05fc105e30 con 0x7f05fc028540 2024-02-23T00:50:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.339+0000 7f05f3fff700 1 -- 172.21.15.47:0/2141433997 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f05fc105e30 con 0x7f05fc028540 2024-02-23T00:50:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.339+0000 7f05f3fff700 10 client.41555.objecter ms_dispatch 0x7f05fc001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:50:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.339+0000 7f05f3fff700 10 client.41555.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:50:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.339+0000 7f05f3fff700 20 client.41555.objecter put_session s=0x7f05fc05f930 osd=0 4 2024-02-23T00:50:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.339+0000 7f05f3fff700 15 client.41555.objecter _session_command_op_remove 0 2 2024-02-23T00:50:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.339+0000 7f060197b700 10 client.41555.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:28.362 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:50:28.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.339+0000 7f060197b700 15 client.41555.objecter _session_command_op_assign -1 3 2024-02-23T00:50:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.339+0000 7f060197b700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f05fc0ff440 0x7f05fc0ff820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.339+0000 7f060197b700 20 client.41555.objecter _get_session s=0x7f05fc0fedc0 osd=1 3 2024-02-23T00:50:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.339+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc0fedc0 osd=1 3 2024-02-23T00:50:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.339+0000 7f060197b700 20 client.41555.objecter _get_session s=0x7f05fc0fedc0 osd=1 3 2024-02-23T00:50:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.339+0000 7f060197b700 15 client.41555.objecter _session_command_op_remove -1 3 2024-02-23T00:50:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.339+0000 7f060197b700 20 client.41555.objecter get_session s=0x7f05fc0fedc0 osd=1 3 2024-02-23T00:50:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.339+0000 7f060197b700 15 client.41555.objecter _session_command_op_assign 1 3 2024-02-23T00:50:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.339+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc0fedc0 osd=1 4 2024-02-23T00:50:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.339+0000 7f060197b700 10 client.41555.objecter _send_command 3 2024-02-23T00:50:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.339+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f05fc0ffd10 con 0x7f05fc0ff440 2024-02-23T00:50:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.340+0000 7f05faffd700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f05fc0ff440 0x7f05fc0ff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.340+0000 7f05f3fff700 10 client.41555.objecter ms_handle_connect 0x7f05fc0ff440 2024-02-23T00:50:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.340+0000 7f05faffd700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f05fc0ff440 0x7f05fc0ff820 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:28.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.340+0000 7f05f3fff700 1 -- 172.21.15.47:0/2141433997 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f05fc0ffd10 con 0x7f05fc0ff440 2024-02-23T00:50:28.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.340+0000 7f05f3fff700 10 client.41555.objecter ms_dispatch 0x7f05fc001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:50:28.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.340+0000 7f05f3fff700 10 client.41555.objecter _finish_command 3 = system:0 2024-02-23T00:50:28.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.340+0000 7f05f3fff700 20 client.41555.objecter put_session s=0x7f05fc0fedc0 osd=1 4 2024-02-23T00:50:28.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.340+0000 7f05f3fff700 15 client.41555.objecter _session_command_op_remove 1 3 2024-02-23T00:50:28.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.355+0000 7f060197b700 10 client.41555.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:50:28.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.355+0000 7f060197b700 15 client.41555.objecter _session_command_op_assign -1 4 2024-02-23T00:50:28.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.355+0000 7f060197b700 20 client.41555.objecter _get_session s=0x7f05fc0fedc0 osd=1 3 2024-02-23T00:50:28.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.355+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc0fedc0 osd=1 3 2024-02-23T00:50:28.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.355+0000 7f060197b700 20 client.41555.objecter _get_session s=0x7f05fc0fedc0 osd=1 3 2024-02-23T00:50:28.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.355+0000 7f060197b700 15 client.41555.objecter _session_command_op_remove -1 4 2024-02-23T00:50:28.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.355+0000 7f060197b700 20 client.41555.objecter get_session s=0x7f05fc0fedc0 osd=1 3 2024-02-23T00:50:28.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.355+0000 7f060197b700 15 client.41555.objecter _session_command_op_assign 1 4 2024-02-23T00:50:28.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.355+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc0fedc0 osd=1 4 2024-02-23T00:50:28.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.355+0000 7f060197b700 10 client.41555.objecter _send_command 4 2024-02-23T00:50:28.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.355+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f05fc118ef0 con 0x7f05fc0ff440 2024-02-23T00:50:28.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.357+0000 7f05f3fff700 1 -- 172.21.15.47:0/2141433997 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f05fc118ef0 con 0x7f05fc0ff440 2024-02-23T00:50:28.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.357+0000 7f05f3fff700 10 client.41555.objecter ms_dispatch 0x7f05fc001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:50:28.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.357+0000 7f05f3fff700 10 client.41555.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:50:28.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.357+0000 7f05f3fff700 20 client.41555.objecter put_session s=0x7f05fc0fedc0 osd=1 4 2024-02-23T00:50:28.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.357+0000 7f05f3fff700 15 client.41555.objecter _session_command_op_remove 1 4 2024-02-23T00:50:28.365 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:50:28.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.357+0000 7f060197b700 10 client.41555.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:28.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.357+0000 7f060197b700 15 client.41555.objecter _session_command_op_assign -1 5 2024-02-23T00:50:28.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.357+0000 7f060197b700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f05fc0594d0 0x7f05fc11c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:28.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.357+0000 7f060197b700 20 client.41555.objecter _get_session s=0x7f05fc059360 osd=2 3 2024-02-23T00:50:28.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.357+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc059360 osd=2 3 2024-02-23T00:50:28.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.357+0000 7f060197b700 20 client.41555.objecter _get_session s=0x7f05fc059360 osd=2 3 2024-02-23T00:50:28.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.357+0000 7f060197b700 15 client.41555.objecter _session_command_op_remove -1 5 2024-02-23T00:50:28.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.357+0000 7f060197b700 20 client.41555.objecter get_session s=0x7f05fc059360 osd=2 3 2024-02-23T00:50:28.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.357+0000 7f060197b700 15 client.41555.objecter _session_command_op_assign 2 5 2024-02-23T00:50:28.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.357+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc059360 osd=2 4 2024-02-23T00:50:28.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.357+0000 7f060197b700 10 client.41555.objecter _send_command 5 2024-02-23T00:50:28.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.357+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f05fc028370 con 0x7f05fc0594d0 2024-02-23T00:50:28.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.357+0000 7f05fa7fc700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f05fc0594d0 0x7f05fc11c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:28.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.357+0000 7f05f3fff700 10 client.41555.objecter ms_handle_connect 0x7f05fc0594d0 2024-02-23T00:50:28.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.357+0000 7f05fa7fc700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f05fc0594d0 0x7f05fc11c250 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:28.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.358+0000 7f05f3fff700 1 -- 172.21.15.47:0/2141433997 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f05fc028370 con 0x7f05fc0594d0 2024-02-23T00:50:28.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.358+0000 7f05f3fff700 10 client.41555.objecter ms_dispatch 0x7f05fc001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:50:28.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.358+0000 7f05f3fff700 10 client.41555.objecter _finish_command 5 = system:0 2024-02-23T00:50:28.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.358+0000 7f05f3fff700 20 client.41555.objecter put_session s=0x7f05fc059360 osd=2 4 2024-02-23T00:50:28.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.358+0000 7f05f3fff700 15 client.41555.objecter _session_command_op_remove 2 5 2024-02-23T00:50:28.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.372+0000 7f060197b700 10 client.41555.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:50:28.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.372+0000 7f060197b700 15 client.41555.objecter _session_command_op_assign -1 6 2024-02-23T00:50:28.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.372+0000 7f060197b700 20 client.41555.objecter _get_session s=0x7f05fc059360 osd=2 3 2024-02-23T00:50:28.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.372+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc059360 osd=2 3 2024-02-23T00:50:28.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.372+0000 7f060197b700 20 client.41555.objecter _get_session s=0x7f05fc059360 osd=2 3 2024-02-23T00:50:28.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.372+0000 7f060197b700 15 client.41555.objecter _session_command_op_remove -1 6 2024-02-23T00:50:28.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.372+0000 7f060197b700 20 client.41555.objecter get_session s=0x7f05fc059360 osd=2 3 2024-02-23T00:50:28.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.372+0000 7f060197b700 15 client.41555.objecter _session_command_op_assign 2 6 2024-02-23T00:50:28.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.372+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc059360 osd=2 4 2024-02-23T00:50:28.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.372+0000 7f060197b700 10 client.41555.objecter _send_command 6 2024-02-23T00:50:28.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.372+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f05fc027600 con 0x7f05fc0594d0 2024-02-23T00:50:28.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.374+0000 7f05f3fff700 1 -- 172.21.15.47:0/2141433997 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f05fc027600 con 0x7f05fc0594d0 2024-02-23T00:50:28.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.374+0000 7f05f3fff700 10 client.41555.objecter ms_dispatch 0x7f05fc001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:50:28.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.374+0000 7f05f3fff700 10 client.41555.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:50:28.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.374+0000 7f05f3fff700 20 client.41555.objecter put_session s=0x7f05fc059360 osd=2 4 2024-02-23T00:50:28.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.374+0000 7f05f3fff700 15 client.41555.objecter _session_command_op_remove 2 6 2024-02-23T00:50:28.376 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:50:28.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.374+0000 7f060197b700 10 client.41555.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:28.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.374+0000 7f060197b700 15 client.41555.objecter _session_command_op_assign -1 7 2024-02-23T00:50:28.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.374+0000 7f060197b700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f05fc119d40 0x7f05fc11d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:28.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.374+0000 7f060197b700 20 client.41555.objecter _get_session s=0x7f05fc027a10 osd=3 3 2024-02-23T00:50:28.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.374+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc027a10 osd=3 3 2024-02-23T00:50:28.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.374+0000 7f060197b700 20 client.41555.objecter _get_session s=0x7f05fc027a10 osd=3 3 2024-02-23T00:50:28.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.374+0000 7f060197b700 15 client.41555.objecter _session_command_op_remove -1 7 2024-02-23T00:50:28.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.374+0000 7f060197b700 20 client.41555.objecter get_session s=0x7f05fc027a10 osd=3 3 2024-02-23T00:50:28.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.374+0000 7f060197b700 15 client.41555.objecter _session_command_op_assign 3 7 2024-02-23T00:50:28.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.374+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc027a10 osd=3 4 2024-02-23T00:50:28.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.374+0000 7f060197b700 10 client.41555.objecter _send_command 7 2024-02-23T00:50:28.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.374+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f05fc11dee0 con 0x7f05fc119d40 2024-02-23T00:50:28.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.374+0000 7f05fb7fe700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f05fc119d40 0x7f05fc11d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:28.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.375+0000 7f05f3fff700 10 client.41555.objecter ms_handle_connect 0x7f05fc119d40 2024-02-23T00:50:28.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.375+0000 7f05fb7fe700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f05fc119d40 0x7f05fc11d8c0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:28.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.375+0000 7f05f3fff700 1 -- 172.21.15.47:0/2141433997 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f05fc11dee0 con 0x7f05fc119d40 2024-02-23T00:50:28.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.375+0000 7f05f3fff700 10 client.41555.objecter ms_dispatch 0x7f05fc001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:50:28.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.375+0000 7f05f3fff700 10 client.41555.objecter _finish_command 7 = system:0 2024-02-23T00:50:28.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.375+0000 7f05f3fff700 20 client.41555.objecter put_session s=0x7f05fc027a10 osd=3 4 2024-02-23T00:50:28.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.375+0000 7f05f3fff700 15 client.41555.objecter _session_command_op_remove 3 7 2024-02-23T00:50:28.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.390+0000 7f060197b700 10 client.41555.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:50:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.390+0000 7f060197b700 15 client.41555.objecter _session_command_op_assign -1 8 2024-02-23T00:50:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.390+0000 7f060197b700 20 client.41555.objecter _get_session s=0x7f05fc027a10 osd=3 3 2024-02-23T00:50:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.390+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc027a10 osd=3 3 2024-02-23T00:50:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.390+0000 7f060197b700 20 client.41555.objecter _get_session s=0x7f05fc027a10 osd=3 3 2024-02-23T00:50:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.390+0000 7f060197b700 15 client.41555.objecter _session_command_op_remove -1 8 2024-02-23T00:50:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.390+0000 7f060197b700 20 client.41555.objecter get_session s=0x7f05fc027a10 osd=3 3 2024-02-23T00:50:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.390+0000 7f060197b700 15 client.41555.objecter _session_command_op_assign 3 8 2024-02-23T00:50:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.390+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc027a10 osd=3 4 2024-02-23T00:50:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.390+0000 7f060197b700 10 client.41555.objecter _send_command 8 2024-02-23T00:50:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.390+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f05fc1192a0 con 0x7f05fc119d40 2024-02-23T00:50:28.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.391+0000 7f05f3fff700 1 -- 172.21.15.47:0/2141433997 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f05fc1192a0 con 0x7f05fc119d40 2024-02-23T00:50:28.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.391+0000 7f05f3fff700 10 client.41555.objecter ms_dispatch 0x7f05fc001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:50:28.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.391+0000 7f05f3fff700 10 client.41555.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:50:28.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.391+0000 7f05f3fff700 20 client.41555.objecter put_session s=0x7f05fc027a10 osd=3 4 2024-02-23T00:50:28.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.391+0000 7f05f3fff700 15 client.41555.objecter _session_command_op_remove 3 8 2024-02-23T00:50:28.393 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:50:28.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.391+0000 7f060197b700 10 client.41555.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:28.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.391+0000 7f060197b700 15 client.41555.objecter _session_command_op_assign -1 9 2024-02-23T00:50:28.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.391+0000 7f060197b700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f05fc0f8750 0x7f05fc0f8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:28.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.391+0000 7f060197b700 20 client.41555.objecter _get_session s=0x7f05fc119470 osd=4 3 2024-02-23T00:50:28.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.391+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc119470 osd=4 3 2024-02-23T00:50:28.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.391+0000 7f060197b700 20 client.41555.objecter _get_session s=0x7f05fc119470 osd=4 3 2024-02-23T00:50:28.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.391+0000 7f060197b700 15 client.41555.objecter _session_command_op_remove -1 9 2024-02-23T00:50:28.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.391+0000 7f060197b700 20 client.41555.objecter get_session s=0x7f05fc119470 osd=4 3 2024-02-23T00:50:28.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.391+0000 7f060197b700 15 client.41555.objecter _session_command_op_assign 4 9 2024-02-23T00:50:28.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.391+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc119470 osd=4 4 2024-02-23T00:50:28.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.391+0000 7f060197b700 10 client.41555.objecter _send_command 9 2024-02-23T00:50:28.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.391+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f05fc1195e0 con 0x7f05fc0f8750 2024-02-23T00:50:28.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.392+0000 7f05faffd700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f05fc0f8750 0x7f05fc0f8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:28.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.392+0000 7f05f3fff700 10 client.41555.objecter ms_handle_connect 0x7f05fc0f8750 2024-02-23T00:50:28.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.392+0000 7f05faffd700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f05fc0f8750 0x7f05fc0f8b90 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:28.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.393+0000 7f05f3fff700 1 -- 172.21.15.47:0/2141433997 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f05fc1195e0 con 0x7f05fc0f8750 2024-02-23T00:50:28.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.393+0000 7f05f3fff700 10 client.41555.objecter ms_dispatch 0x7f05fc001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:50:28.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.393+0000 7f05f3fff700 10 client.41555.objecter _finish_command 9 = system:0 2024-02-23T00:50:28.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.393+0000 7f05f3fff700 20 client.41555.objecter put_session s=0x7f05fc119470 osd=4 4 2024-02-23T00:50:28.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.393+0000 7f05f3fff700 15 client.41555.objecter _session_command_op_remove 4 9 2024-02-23T00:50:28.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.407+0000 7f060197b700 10 client.41555.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:50:28.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.407+0000 7f060197b700 15 client.41555.objecter _session_command_op_assign -1 10 2024-02-23T00:50:28.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.407+0000 7f060197b700 20 client.41555.objecter _get_session s=0x7f05fc119470 osd=4 3 2024-02-23T00:50:28.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.407+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc119470 osd=4 3 2024-02-23T00:50:28.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.407+0000 7f060197b700 20 client.41555.objecter _get_session s=0x7f05fc119470 osd=4 3 2024-02-23T00:50:28.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.407+0000 7f060197b700 15 client.41555.objecter _session_command_op_remove -1 10 2024-02-23T00:50:28.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.407+0000 7f060197b700 20 client.41555.objecter get_session s=0x7f05fc119470 osd=4 3 2024-02-23T00:50:28.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.407+0000 7f060197b700 15 client.41555.objecter _session_command_op_assign 4 10 2024-02-23T00:50:28.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.407+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc119470 osd=4 4 2024-02-23T00:50:28.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.407+0000 7f060197b700 10 client.41555.objecter _send_command 10 2024-02-23T00:50:28.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.407+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f05fc1195e0 con 0x7f05fc0f8750 2024-02-23T00:50:28.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.409+0000 7f05f3fff700 1 -- 172.21.15.47:0/2141433997 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f05fc1195e0 con 0x7f05fc0f8750 2024-02-23T00:50:28.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.409+0000 7f05f3fff700 10 client.41555.objecter ms_dispatch 0x7f05fc001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:50:28.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.410+0000 7f05f3fff700 10 client.41555.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:50:28.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.410+0000 7f05f3fff700 20 client.41555.objecter put_session s=0x7f05fc119470 osd=4 4 2024-02-23T00:50:28.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.410+0000 7f05f3fff700 15 client.41555.objecter _session_command_op_remove 4 10 2024-02-23T00:50:28.412 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:50:28.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.410+0000 7f060197b700 10 client.41555.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:28.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.410+0000 7f060197b700 15 client.41555.objecter _session_command_op_assign -1 11 2024-02-23T00:50:28.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.410+0000 7f060197b700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f05fc0f9590 0x7f05fc0f99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:28.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.410+0000 7f060197b700 20 client.41555.objecter _get_session s=0x7f05fc1195e0 osd=5 3 2024-02-23T00:50:28.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.410+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc1195e0 osd=5 3 2024-02-23T00:50:28.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.410+0000 7f060197b700 20 client.41555.objecter _get_session s=0x7f05fc1195e0 osd=5 3 2024-02-23T00:50:28.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.410+0000 7f060197b700 15 client.41555.objecter _session_command_op_remove -1 11 2024-02-23T00:50:28.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.410+0000 7f060197b700 20 client.41555.objecter get_session s=0x7f05fc1195e0 osd=5 3 2024-02-23T00:50:28.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.410+0000 7f060197b700 15 client.41555.objecter _session_command_op_assign 5 11 2024-02-23T00:50:28.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.410+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc1195e0 osd=5 4 2024-02-23T00:50:28.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.410+0000 7f060197b700 10 client.41555.objecter _send_command 11 2024-02-23T00:50:28.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.410+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f05fc0fa080 con 0x7f05fc0f9590 2024-02-23T00:50:28.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.410+0000 7f05fa7fc700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f05fc0f9590 0x7f05fc0f99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:28.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.411+0000 7f05fa7fc700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f05fc0f9590 0x7f05fc0f99d0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:28.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.411+0000 7f05f3fff700 10 client.41555.objecter ms_handle_connect 0x7f05fc0f9590 2024-02-23T00:50:28.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.411+0000 7f05f3fff700 1 -- 172.21.15.47:0/2141433997 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f05fc0fa080 con 0x7f05fc0f9590 2024-02-23T00:50:28.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.411+0000 7f05f3fff700 10 client.41555.objecter ms_dispatch 0x7f05fc001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:50:28.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.411+0000 7f05f3fff700 10 client.41555.objecter _finish_command 11 = system:0 2024-02-23T00:50:28.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.411+0000 7f05f3fff700 20 client.41555.objecter put_session s=0x7f05fc1195e0 osd=5 4 2024-02-23T00:50:28.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.411+0000 7f05f3fff700 15 client.41555.objecter _session_command_op_remove 5 11 2024-02-23T00:50:28.415 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:50:28.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.426+0000 7f060197b700 10 client.41555.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:50:28.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.426+0000 7f060197b700 15 client.41555.objecter _session_command_op_assign -1 12 2024-02-23T00:50:28.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.426+0000 7f060197b700 20 client.41555.objecter _get_session s=0x7f05fc1195e0 osd=5 3 2024-02-23T00:50:28.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.426+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc1195e0 osd=5 3 2024-02-23T00:50:28.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.426+0000 7f060197b700 20 client.41555.objecter _get_session s=0x7f05fc1195e0 osd=5 3 2024-02-23T00:50:28.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.426+0000 7f060197b700 15 client.41555.objecter _session_command_op_remove -1 12 2024-02-23T00:50:28.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.426+0000 7f060197b700 20 client.41555.objecter get_session s=0x7f05fc1195e0 osd=5 3 2024-02-23T00:50:28.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.426+0000 7f060197b700 15 client.41555.objecter _session_command_op_assign 5 12 2024-02-23T00:50:28.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.426+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc1195e0 osd=5 4 2024-02-23T00:50:28.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.426+0000 7f060197b700 10 client.41555.objecter _send_command 12 2024-02-23T00:50:28.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.426+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f05fc0fa080 con 0x7f05fc0f9590 2024-02-23T00:50:28.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.428+0000 7f05f3fff700 1 -- 172.21.15.47:0/2141433997 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f05fc0fa080 con 0x7f05fc0f9590 2024-02-23T00:50:28.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.428+0000 7f05f3fff700 10 client.41555.objecter ms_dispatch 0x7f05fc001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:50:28.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.428+0000 7f05f3fff700 10 client.41555.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:50:28.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.428+0000 7f05f3fff700 20 client.41555.objecter put_session s=0x7f05fc1195e0 osd=5 4 2024-02-23T00:50:28.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.428+0000 7f05f3fff700 15 client.41555.objecter _session_command_op_remove 5 12 2024-02-23T00:50:28.430 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:50:28.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.428+0000 7f060197b700 10 client.41555.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:28.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.428+0000 7f060197b700 15 client.41555.objecter _session_command_op_assign -1 13 2024-02-23T00:50:28.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.428+0000 7f060197b700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f05fc129360 0x7f05fc12b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:28.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.428+0000 7f060197b700 20 client.41555.objecter _get_session s=0x7f05fc0fa080 osd=6 3 2024-02-23T00:50:28.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.428+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc0fa080 osd=6 3 2024-02-23T00:50:28.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.428+0000 7f060197b700 20 client.41555.objecter _get_session s=0x7f05fc0fa080 osd=6 3 2024-02-23T00:50:28.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.428+0000 7f060197b700 15 client.41555.objecter _session_command_op_remove -1 13 2024-02-23T00:50:28.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.428+0000 7f060197b700 20 client.41555.objecter get_session s=0x7f05fc0fa080 osd=6 3 2024-02-23T00:50:28.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.428+0000 7f060197b700 15 client.41555.objecter _session_command_op_assign 6 13 2024-02-23T00:50:28.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.428+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc0fa080 osd=6 4 2024-02-23T00:50:28.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.428+0000 7f060197b700 10 client.41555.objecter _send_command 13 2024-02-23T00:50:28.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.428+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f05fc12e430 con 0x7f05fc129360 2024-02-23T00:50:28.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.428+0000 7f05fb7fe700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f05fc129360 0x7f05fc12b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:28.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.429+0000 7f05f3fff700 10 client.41555.objecter ms_handle_connect 0x7f05fc129360 2024-02-23T00:50:28.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.429+0000 7f05fb7fe700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f05fc129360 0x7f05fc12b750 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:28.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.429+0000 7f05f3fff700 1 -- 172.21.15.47:0/2141433997 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f05fc12e430 con 0x7f05fc129360 2024-02-23T00:50:28.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.429+0000 7f05f3fff700 10 client.41555.objecter ms_dispatch 0x7f05fc001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:50:28.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.429+0000 7f05f3fff700 10 client.41555.objecter _finish_command 13 = system:0 2024-02-23T00:50:28.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.429+0000 7f05f3fff700 20 client.41555.objecter put_session s=0x7f05fc0fa080 osd=6 4 2024-02-23T00:50:28.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.429+0000 7f05f3fff700 15 client.41555.objecter _session_command_op_remove 6 13 2024-02-23T00:50:28.444 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:27.267530+0000 mon.a (mon.0) 2791 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-23T00:50:28.444 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:27.267620+0000 mon.a (mon.0) 2792 : cluster [DBG] osdmap e498: 8 total, 8 up, 7 in 2024-02-23T00:50:28.444 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:27.273908+0000 mon.a (mon.0) 2793 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:50:28.444 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:27.328193+0000 mgr.y (mgr.14517) 469 : cluster [DBG] pgmap v751: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 10 KiB/s rd, 11 KiB/s wr, 37 op/s 2024-02-23T00:50:28.444 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:27.627060+0000 osd.1 (osd.1) 225 : cluster [DBG] 62.10 scrub starts 2024-02-23T00:50:28.444 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:27.628790+0000 osd.1 (osd.1) 226 : cluster [DBG] 62.10 scrub ok 2024-02-23T00:50:28.444 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:28.059146+0000 osd.3 (osd.3) 238 : cluster [DBG] 2.1b scrub starts 2024-02-23T00:50:28.444 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:28.060105+0000 osd.3 (osd.3) 239 : cluster [DBG] 2.1b scrub ok 2024-02-23T00:50:28.445 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:27.267530+0000 mon.a (mon.0) 2791 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-23T00:50:28.445 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:27.267620+0000 mon.a (mon.0) 2792 : cluster [DBG] osdmap e498: 8 total, 8 up, 7 in 2024-02-23T00:50:28.445 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:27.273908+0000 mon.a (mon.0) 2793 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:50:28.445 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:27.328193+0000 mgr.y (mgr.14517) 469 : cluster [DBG] pgmap v751: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 10 KiB/s rd, 11 KiB/s wr, 37 op/s 2024-02-23T00:50:28.445 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:27.627060+0000 osd.1 (osd.1) 225 : cluster [DBG] 62.10 scrub starts 2024-02-23T00:50:28.445 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:27.628790+0000 osd.1 (osd.1) 226 : cluster [DBG] 62.10 scrub ok 2024-02-23T00:50:28.445 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:28.059146+0000 osd.3 (osd.3) 238 : cluster [DBG] 2.1b scrub starts 2024-02-23T00:50:28.445 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:28.060105+0000 osd.3 (osd.3) 239 : cluster [DBG] 2.1b scrub ok 2024-02-23T00:50:28.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.445+0000 7f060197b700 10 client.41555.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:50:28.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.445+0000 7f060197b700 15 client.41555.objecter _session_command_op_assign -1 14 2024-02-23T00:50:28.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.445+0000 7f060197b700 20 client.41555.objecter _get_session s=0x7f05fc0fa080 osd=6 3 2024-02-23T00:50:28.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.445+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc0fa080 osd=6 3 2024-02-23T00:50:28.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.445+0000 7f060197b700 20 client.41555.objecter _get_session s=0x7f05fc0fa080 osd=6 3 2024-02-23T00:50:28.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.445+0000 7f060197b700 15 client.41555.objecter _session_command_op_remove -1 14 2024-02-23T00:50:28.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.445+0000 7f060197b700 20 client.41555.objecter get_session s=0x7f05fc0fa080 osd=6 3 2024-02-23T00:50:28.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.445+0000 7f060197b700 15 client.41555.objecter _session_command_op_assign 6 14 2024-02-23T00:50:28.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.445+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc0fa080 osd=6 4 2024-02-23T00:50:28.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.445+0000 7f060197b700 10 client.41555.objecter _send_command 14 2024-02-23T00:50:28.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.445+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f05fc12e430 con 0x7f05fc129360 2024-02-23T00:50:28.447 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:28.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.447+0000 7f05f3fff700 1 -- 172.21.15.47:0/2141433997 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f05fc12e430 con 0x7f05fc129360 2024-02-23T00:50:28.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.447+0000 7f05f3fff700 10 client.41555.objecter ms_dispatch 0x7f05fc001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:50:28.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.447+0000 7f05f3fff700 10 client.41555.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:50:28.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.447+0000 7f05f3fff700 20 client.41555.objecter put_session s=0x7f05fc0fa080 osd=6 4 2024-02-23T00:50:28.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.447+0000 7f05f3fff700 15 client.41555.objecter _session_command_op_remove 6 14 2024-02-23T00:50:28.449 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:50:28.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.447+0000 7f060197b700 10 client.41555.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:28.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.447+0000 7f060197b700 15 client.41555.objecter _session_command_op_assign -1 15 2024-02-23T00:50:28.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.447+0000 7f060197b700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f05fc12c1a0 0x7f05fc12c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:28.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.447+0000 7f060197b700 20 client.41555.objecter _get_session s=0x7f05fc12e430 osd=7 3 2024-02-23T00:50:28.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.447+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc12e430 osd=7 3 2024-02-23T00:50:28.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.447+0000 7f060197b700 20 client.41555.objecter _get_session s=0x7f05fc12e430 osd=7 3 2024-02-23T00:50:28.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.447+0000 7f060197b700 15 client.41555.objecter _session_command_op_remove -1 15 2024-02-23T00:50:28.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.447+0000 7f060197b700 20 client.41555.objecter get_session s=0x7f05fc12e430 osd=7 3 2024-02-23T00:50:28.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.447+0000 7f060197b700 15 client.41555.objecter _session_command_op_assign 7 15 2024-02-23T00:50:28.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.447+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc12e430 osd=7 4 2024-02-23T00:50:28.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.447+0000 7f060197b700 10 client.41555.objecter _send_command 15 2024-02-23T00:50:28.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.447+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f05fc131760 con 0x7f05fc12c1a0 2024-02-23T00:50:28.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.447+0000 7f05faffd700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f05fc12c1a0 0x7f05fc12c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:28.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.448+0000 7f05f3fff700 10 client.41555.objecter ms_handle_connect 0x7f05fc12c1a0 2024-02-23T00:50:28.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.448+0000 7f05faffd700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f05fc12c1a0 0x7f05fc12c600 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:28.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.448+0000 7f05f3fff700 1 -- 172.21.15.47:0/2141433997 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f05fc131760 con 0x7f05fc12c1a0 2024-02-23T00:50:28.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.448+0000 7f05f3fff700 10 client.41555.objecter ms_dispatch 0x7f05fc001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:50:28.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.448+0000 7f05f3fff700 10 client.41555.objecter _finish_command 15 = system:0 2024-02-23T00:50:28.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.448+0000 7f05f3fff700 20 client.41555.objecter put_session s=0x7f05fc12e430 osd=7 4 2024-02-23T00:50:28.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.448+0000 7f05f3fff700 15 client.41555.objecter _session_command_op_remove 7 15 2024-02-23T00:50:28.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.464+0000 7f060197b700 10 client.41555.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:50:28.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.464+0000 7f060197b700 15 client.41555.objecter _session_command_op_assign -1 16 2024-02-23T00:50:28.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.464+0000 7f060197b700 20 client.41555.objecter _get_session s=0x7f05fc12e430 osd=7 3 2024-02-23T00:50:28.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.464+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc12e430 osd=7 3 2024-02-23T00:50:28.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.464+0000 7f060197b700 20 client.41555.objecter _get_session s=0x7f05fc12e430 osd=7 3 2024-02-23T00:50:28.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.464+0000 7f060197b700 15 client.41555.objecter _session_command_op_remove -1 16 2024-02-23T00:50:28.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.464+0000 7f060197b700 20 client.41555.objecter get_session s=0x7f05fc12e430 osd=7 3 2024-02-23T00:50:28.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.464+0000 7f060197b700 15 client.41555.objecter _session_command_op_assign 7 16 2024-02-23T00:50:28.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.464+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc12e430 osd=7 4 2024-02-23T00:50:28.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.464+0000 7f060197b700 10 client.41555.objecter _send_command 16 2024-02-23T00:50:28.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.464+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f05fc12cc00 con 0x7f05fc12c1a0 2024-02-23T00:50:28.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.466+0000 7f05f3fff700 1 -- 172.21.15.47:0/2141433997 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f05fc12cc00 con 0x7f05fc12c1a0 2024-02-23T00:50:28.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.466+0000 7f05f3fff700 10 client.41555.objecter ms_dispatch 0x7f05fc001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:50:28.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.466+0000 7f05f3fff700 10 client.41555.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:50:28.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.466+0000 7f05f3fff700 20 client.41555.objecter put_session s=0x7f05fc12e430 osd=7 4 2024-02-23T00:50:28.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.466+0000 7f05f3fff700 15 client.41555.objecter _session_command_op_remove 7 16 2024-02-23T00:50:28.468 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:50:28.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 10 librados: watch_flush enter 2024-02-23T00:50:28.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 10 librados: watch_flush exit 2024-02-23T00:50:28.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 10 client.41555.objecter close_session for osd.0 2024-02-23T00:50:28.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f05fc028540 msgr2=0x7f05fc0fe8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:28.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f05fc028540 0x7f05fc0fe8d0 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:28.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc05f930 osd=0 1 2024-02-23T00:50:28.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 10 client.41555.objecter close_session for osd.1 2024-02-23T00:50:28.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f05fc0ff440 msgr2=0x7f05fc0ff820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:28.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f05fc0ff440 0x7f05fc0ff820 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:28.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc0fedc0 osd=1 1 2024-02-23T00:50:28.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 10 client.41555.objecter close_session for osd.2 2024-02-23T00:50:28.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f05fc0594d0 msgr2=0x7f05fc11c250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:28.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f05fc0594d0 0x7f05fc11c250 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:28.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc059360 osd=2 1 2024-02-23T00:50:28.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 10 client.41555.objecter close_session for osd.3 2024-02-23T00:50:28.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f05fb7fe700 1 -- 172.21.15.47:0/2141433997 reap_dead start 2024-02-23T00:50:28.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f05fc119d40 msgr2=0x7f05fc11d8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:28.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f05fc119d40 0x7f05fc11d8c0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:28.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc027a10 osd=3 1 2024-02-23T00:50:28.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 10 client.41555.objecter close_session for osd.4 2024-02-23T00:50:28.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f05fc0f8750 msgr2=0x7f05fc0f8b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:28.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f05fc0f8750 0x7f05fc0f8b90 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:28.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc119470 osd=4 1 2024-02-23T00:50:28.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 10 client.41555.objecter close_session for osd.5 2024-02-23T00:50:28.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f05fc0f9590 msgr2=0x7f05fc0f99d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:28.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f05fc0f9590 0x7f05fc0f99d0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:28.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc1195e0 osd=5 1 2024-02-23T00:50:28.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 10 client.41555.objecter close_session for osd.6 2024-02-23T00:50:28.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f05fc129360 msgr2=0x7f05fc12b750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:28.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f05fc129360 0x7f05fc12b750 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:28.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc0fa080 osd=6 1 2024-02-23T00:50:28.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 10 client.41555.objecter close_session for osd.7 2024-02-23T00:50:28.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f05fc12c1a0 msgr2=0x7f05fc12c600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:28.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f05fc12c1a0 0x7f05fc12c600 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:28.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 20 client.41555.objecter put_session s=0x7f05fc12e430 osd=7 1 2024-02-23T00:50:28.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 20 client.41555.objecter shutdown clearing up homeless session... 2024-02-23T00:50:28.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 10 client.41555.objecter successfully canceled tick 2024-02-23T00:50:28.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f05fb7fe700 1 -- 172.21.15.47:0/2141433997 reap_dead start 2024-02-23T00:50:28.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f05e8062cc0 msgr2=0x7f05e8065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:28.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f05e8062cc0 0x7f05e8065180 secure :-1 s=READY pgs=3101 cs=0 l=1 rev1=1 rx=0x7f05ec004fd0 tx=0x7f05ec00b040).stop 2024-02-23T00:50:28.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f05fc0fcb40 msgr2=0x7f05fc10e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:28.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f05fc0fcb40 0x7f05fc10e400 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 rx=0x7f05e4009da0 tx=0x7f05e4009cb0).stop 2024-02-23T00:50:28.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 shutdown_connections 2024-02-23T00:50:28.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f05fc0fcb40 0x7f05fc10e400 unknown :-1 s=CLOSED pgs=2711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:28.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 1 --2- 172.21.15.47:0/2141433997 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f05e8062cc0 0x7f05e8065180 unknown :-1 s=CLOSED pgs=3101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:28.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 >> 172.21.15.47:0/2141433997 conn(0x7f05fc06c370 msgr2=0x7f05fc0fb180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:28.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 shutdown_connections 2024-02-23T00:50:28.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 1 -- 172.21.15.47:0/2141433997 wait complete. 2024-02-23T00:50:28.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:28.467+0000 7f060197b700 1 librados: shutdown 2024-02-23T00:50:28.473 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:50:28.474 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:50:28.474 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:50:28.474 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:50:28.474 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:50:28.474 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:50:28.474 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:50:28.474 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:50:28.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:27.267530+0000 mon.a (mon.0) 2791 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-23T00:50:28.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:27.267620+0000 mon.a (mon.0) 2792 : cluster [DBG] osdmap e498: 8 total, 8 up, 7 in 2024-02-23T00:50:28.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:27.273908+0000 mon.a (mon.0) 2793 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:50:28.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:27.328193+0000 mgr.y (mgr.14517) 469 : cluster [DBG] pgmap v751: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 10 KiB/s rd, 11 KiB/s wr, 37 op/s 2024-02-23T00:50:28.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:27.627060+0000 osd.1 (osd.1) 225 : cluster [DBG] 62.10 scrub starts 2024-02-23T00:50:28.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:27.628790+0000 osd.1 (osd.1) 226 : cluster [DBG] 62.10 scrub ok 2024-02-23T00:50:28.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:28.059146+0000 osd.3 (osd.3) 238 : cluster [DBG] 2.1b scrub starts 2024-02-23T00:50:28.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:28.060105+0000 osd.3 (osd.3) 239 : cluster [DBG] 2.1b scrub ok 2024-02-23T00:50:29.238 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight 4 1 2024-02-23T00:50:29.464 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:28.273276+0000 mon.a (mon.0) 2794 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:50:29.464 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:28.273387+0000 mon.a (mon.0) 2795 : cluster [DBG] osdmap e499: 8 total, 8 up, 7 in 2024-02-23T00:50:29.465 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:28.280390+0000 mon.a (mon.0) 2796 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:50:29.465 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:28.283649+0000 mon.a (mon.0) 2797 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-79","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-23T00:50:29.465 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:28.299171+0000 osd.6 (osd.6) 201 : cluster [DBG] 62.7 deep-scrub starts 2024-02-23T00:50:29.465 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:28.305017+0000 osd.6 (osd.6) 202 : cluster [DBG] 62.7 deep-scrub ok 2024-02-23T00:50:29.465 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:28.321673+0000 mon.c (mon.2) 316 : audit [DBG] from='client.? 172.21.15.47:0/2141433997' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:50:29.465 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:28.478617+0000 osd.0 (osd.0) 215 : cluster [DBG] 18.e deep-scrub starts 2024-02-23T00:50:29.465 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:28.479507+0000 osd.0 (osd.0) 216 : cluster [DBG] 18.e deep-scrub ok 2024-02-23T00:50:29.465 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:28.630355+0000 osd.4 (osd.4) 221 : cluster [DBG] 62.d scrub starts 2024-02-23T00:50:29.465 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:28.632134+0000 osd.4 (osd.4) 222 : cluster [DBG] 62.d scrub ok 2024-02-23T00:50:29.465 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:28.674902+0000 osd.5 (osd.5) 228 : cluster [DBG] 18.7 scrub starts 2024-02-23T00:50:29.465 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:28.678034+0000 osd.5 (osd.5) 229 : cluster [DBG] 18.7 scrub ok 2024-02-23T00:50:29.465 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:29.028767+0000 osd.3 (osd.3) 240 : cluster [DBG] 2.19 scrub starts 2024-02-23T00:50:29.465 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:29.029721+0000 osd.3 (osd.3) 241 : cluster [DBG] 2.19 scrub ok 2024-02-23T00:50:29.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:28.273276+0000 mon.a (mon.0) 2794 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:50:29.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:28.273387+0000 mon.a (mon.0) 2795 : cluster [DBG] osdmap e499: 8 total, 8 up, 7 in 2024-02-23T00:50:29.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:28.280390+0000 mon.a (mon.0) 2796 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:50:29.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:28.283649+0000 mon.a (mon.0) 2797 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-79","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-23T00:50:29.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:28.299171+0000 osd.6 (osd.6) 201 : cluster [DBG] 62.7 deep-scrub starts 2024-02-23T00:50:29.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:28.305017+0000 osd.6 (osd.6) 202 : cluster [DBG] 62.7 deep-scrub ok 2024-02-23T00:50:29.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:28.321673+0000 mon.c (mon.2) 316 : audit [DBG] from='client.? 172.21.15.47:0/2141433997' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:50:29.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:28.478617+0000 osd.0 (osd.0) 215 : cluster [DBG] 18.e deep-scrub starts 2024-02-23T00:50:29.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:28.479507+0000 osd.0 (osd.0) 216 : cluster [DBG] 18.e deep-scrub ok 2024-02-23T00:50:29.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:28.630355+0000 osd.4 (osd.4) 221 : cluster [DBG] 62.d scrub starts 2024-02-23T00:50:29.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:28.632134+0000 osd.4 (osd.4) 222 : cluster [DBG] 62.d scrub ok 2024-02-23T00:50:29.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:28.674902+0000 osd.5 (osd.5) 228 : cluster [DBG] 18.7 scrub starts 2024-02-23T00:50:29.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:28.678034+0000 osd.5 (osd.5) 229 : cluster [DBG] 18.7 scrub ok 2024-02-23T00:50:29.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:29.028767+0000 osd.3 (osd.3) 240 : cluster [DBG] 2.19 scrub starts 2024-02-23T00:50:29.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:29.029721+0000 osd.3 (osd.3) 241 : cluster [DBG] 2.19 scrub ok 2024-02-23T00:50:29.497 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set noscrub 2024-02-23T00:50:29.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:28.273276+0000 mon.a (mon.0) 2794 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:50:29.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:28.273387+0000 mon.a (mon.0) 2795 : cluster [DBG] osdmap e499: 8 total, 8 up, 7 in 2024-02-23T00:50:29.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:28.280390+0000 mon.a (mon.0) 2796 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:50:29.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:28.283649+0000 mon.a (mon.0) 2797 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-79","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-23T00:50:29.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:28.299171+0000 osd.6 (osd.6) 201 : cluster [DBG] 62.7 deep-scrub starts 2024-02-23T00:50:29.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:28.305017+0000 osd.6 (osd.6) 202 : cluster [DBG] 62.7 deep-scrub ok 2024-02-23T00:50:29.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:28.321673+0000 mon.c (mon.2) 316 : audit [DBG] from='client.? 172.21.15.47:0/2141433997' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:50:29.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:28.478617+0000 osd.0 (osd.0) 215 : cluster [DBG] 18.e deep-scrub starts 2024-02-23T00:50:29.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:28.479507+0000 osd.0 (osd.0) 216 : cluster [DBG] 18.e deep-scrub ok 2024-02-23T00:50:29.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:28.630355+0000 osd.4 (osd.4) 221 : cluster [DBG] 62.d scrub starts 2024-02-23T00:50:29.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:28.632134+0000 osd.4 (osd.4) 222 : cluster [DBG] 62.d scrub ok 2024-02-23T00:50:29.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:28.674902+0000 osd.5 (osd.5) 228 : cluster [DBG] 18.7 scrub starts 2024-02-23T00:50:29.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:28.678034+0000 osd.5 (osd.5) 229 : cluster [DBG] 18.7 scrub ok 2024-02-23T00:50:29.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:29.028767+0000 osd.3 (osd.3) 240 : cluster [DBG] 2.19 scrub starts 2024-02-23T00:50:29.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:29.029721+0000 osd.3 (osd.3) 241 : cluster [DBG] 2.19 scrub ok 2024-02-23T00:50:29.697 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:50:29.702 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:29.702 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:50:29.803 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:50:29.832 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:30.082 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:50:30.085 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:30.086 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:50:30.186 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:50:30.214 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:30.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:29.273840+0000 mon.a (mon.0) 2798 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-79","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-23T00:50:30.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:29.273907+0000 mon.a (mon.0) 2799 : cluster [DBG] osdmap e500: 8 total, 8 up, 7 in 2024-02-23T00:50:30.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:29.276851+0000 mon.a (mon.0) 2800 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-79","var": "dedup_tier","val": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:50:30.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:29.328681+0000 mgr.y (mgr.14517) 470 : cluster [DBG] pgmap v754: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 10 KiB/s rd, 11 KiB/s wr, 37 op/s 2024-02-23T00:50:30.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:29.597585+0000 osd.4 (osd.4) 223 : cluster [DBG] 4.13 scrub starts 2024-02-23T00:50:30.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:29.599477+0000 osd.4 (osd.4) 224 : cluster [DBG] 4.13 scrub ok 2024-02-23T00:50:30.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:30.068427+0000 osd.3 (osd.3) 242 : cluster [DBG] 5.13 scrub starts 2024-02-23T00:50:30.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:30.069321+0000 osd.3 (osd.3) 243 : cluster [DBG] 5.13 scrub ok 2024-02-23T00:50:30.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:30.222279+0000 mon.a (mon.0) 2801 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-79","var": "dedup_tier","val": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:50:30.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:30.222354+0000 mon.a (mon.0) 2802 : cluster [DBG] osdmap e501: 8 total, 8 up, 7 in 2024-02-23T00:50:30.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:30.225506+0000 mon.a (mon.0) 2803 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-23T00:50:30.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:29.273840+0000 mon.a (mon.0) 2798 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-79","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-23T00:50:30.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:29.273907+0000 mon.a (mon.0) 2799 : cluster [DBG] osdmap e500: 8 total, 8 up, 7 in 2024-02-23T00:50:30.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:29.276851+0000 mon.a (mon.0) 2800 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-79","var": "dedup_tier","val": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:50:30.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:29.328681+0000 mgr.y (mgr.14517) 470 : cluster [DBG] pgmap v754: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 10 KiB/s rd, 11 KiB/s wr, 37 op/s 2024-02-23T00:50:30.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:29.597585+0000 osd.4 (osd.4) 223 : cluster [DBG] 4.13 scrub starts 2024-02-23T00:50:30.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:29.599477+0000 osd.4 (osd.4) 224 : cluster [DBG] 4.13 scrub ok 2024-02-23T00:50:30.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:30.068427+0000 osd.3 (osd.3) 242 : cluster [DBG] 5.13 scrub starts 2024-02-23T00:50:30.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:30.069321+0000 osd.3 (osd.3) 243 : cluster [DBG] 5.13 scrub ok 2024-02-23T00:50:30.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:30.222279+0000 mon.a (mon.0) 2801 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-79","var": "dedup_tier","val": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:50:30.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:30.222354+0000 mon.a (mon.0) 2802 : cluster [DBG] osdmap e501: 8 total, 8 up, 7 in 2024-02-23T00:50:30.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:30.225506+0000 mon.a (mon.0) 2803 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-23T00:50:30.458 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:50:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:50:30] "GET /metrics HTTP/1.1" 200 186989 "" "Prometheus/2.33.4" 2024-02-23T00:50:30.463 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:50:30.463 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:50:30.466 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:50:30.469 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:30.469 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:50:30.570 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:50:30.600 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:29.273840+0000 mon.a (mon.0) 2798 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-79","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-23T00:50:30.601 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:29.273907+0000 mon.a (mon.0) 2799 : cluster [DBG] osdmap e500: 8 total, 8 up, 7 in 2024-02-23T00:50:30.601 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:29.276851+0000 mon.a (mon.0) 2800 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-79","var": "dedup_tier","val": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:50:30.601 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:29.328681+0000 mgr.y (mgr.14517) 470 : cluster [DBG] pgmap v754: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 10 KiB/s rd, 11 KiB/s wr, 37 op/s 2024-02-23T00:50:30.601 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:29.597585+0000 osd.4 (osd.4) 223 : cluster [DBG] 4.13 scrub starts 2024-02-23T00:50:30.601 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:29.599477+0000 osd.4 (osd.4) 224 : cluster [DBG] 4.13 scrub ok 2024-02-23T00:50:30.601 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:30.068427+0000 osd.3 (osd.3) 242 : cluster [DBG] 5.13 scrub starts 2024-02-23T00:50:30.601 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:30.069321+0000 osd.3 (osd.3) 243 : cluster [DBG] 5.13 scrub ok 2024-02-23T00:50:30.601 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:30.222279+0000 mon.a (mon.0) 2801 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-79","var": "dedup_tier","val": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:50:30.601 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:30.222354+0000 mon.a (mon.0) 2802 : cluster [DBG] osdmap e501: 8 total, 8 up, 7 in 2024-02-23T00:50:30.601 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:30.225506+0000 mon.a (mon.0) 2803 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-23T00:50:30.602 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:30.624 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:50:30.626 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:30.627 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:50:30.710 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:50:30.711 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:50:30.728 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:50:30.758 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:31.207 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:50:31.209 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:31.210 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:50:31.213 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.2 dump_ops_in_flight 2024-02-23T00:50:31.311 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:50:31.340 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:31.470 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:50:31.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:30.144413+0000 osd.7 (osd.7) 253 : cluster [DBG] 62.a scrub starts 2024-02-23T00:50:31.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:30.147868+0000 osd.7 (osd.7) 254 : cluster [DBG] 62.a scrub ok 2024-02-23T00:50:31.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:30.280503+0000 osd.6 (osd.6) 203 : cluster [DBG] 62.2 scrub starts 2024-02-23T00:50:31.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:30.283811+0000 osd.6 (osd.6) 204 : cluster [DBG] 62.2 scrub ok 2024-02-23T00:50:31.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:30.759620+0000 osd.5 (osd.5) 230 : cluster [DBG] 4.4 scrub starts 2024-02-23T00:50:31.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:30.763676+0000 osd.5 (osd.5) 231 : cluster [DBG] 4.4 scrub ok 2024-02-23T00:50:31.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:31.223595+0000 mon.a (mon.0) 2804 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-23T00:50:31.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:31.223669+0000 mon.a (mon.0) 2805 : cluster [DBG] osdmap e502: 8 total, 8 up, 7 in 2024-02-23T00:50:31.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:31.224619+0000 mon.a (mon.0) 2806 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-79","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-23T00:50:31.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:30.144413+0000 osd.7 (osd.7) 253 : cluster [DBG] 62.a scrub starts 2024-02-23T00:50:31.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:30.147868+0000 osd.7 (osd.7) 254 : cluster [DBG] 62.a scrub ok 2024-02-23T00:50:31.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:30.280503+0000 osd.6 (osd.6) 203 : cluster [DBG] 62.2 scrub starts 2024-02-23T00:50:31.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:30.283811+0000 osd.6 (osd.6) 204 : cluster [DBG] 62.2 scrub ok 2024-02-23T00:50:31.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:30.759620+0000 osd.5 (osd.5) 230 : cluster [DBG] 4.4 scrub starts 2024-02-23T00:50:31.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:30.763676+0000 osd.5 (osd.5) 231 : cluster [DBG] 4.4 scrub ok 2024-02-23T00:50:31.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:31.223595+0000 mon.a (mon.0) 2804 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-23T00:50:31.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:31.223669+0000 mon.a (mon.0) 2805 : cluster [DBG] osdmap e502: 8 total, 8 up, 7 in 2024-02-23T00:50:31.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:31.224619+0000 mon.a (mon.0) 2806 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-79","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-23T00:50:31.475 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:31.475 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:50:31.576 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:50:31.605 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:31.678 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:50:31.680 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:31.681 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:50:31.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:30.144413+0000 osd.7 (osd.7) 253 : cluster [DBG] 62.a scrub starts 2024-02-23T00:50:31.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:30.147868+0000 osd.7 (osd.7) 254 : cluster [DBG] 62.a scrub ok 2024-02-23T00:50:31.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:30.280503+0000 osd.6 (osd.6) 203 : cluster [DBG] 62.2 scrub starts 2024-02-23T00:50:31.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:30.283811+0000 osd.6 (osd.6) 204 : cluster [DBG] 62.2 scrub ok 2024-02-23T00:50:31.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:30.759620+0000 osd.5 (osd.5) 230 : cluster [DBG] 4.4 scrub starts 2024-02-23T00:50:31.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:30.763676+0000 osd.5 (osd.5) 231 : cluster [DBG] 4.4 scrub ok 2024-02-23T00:50:31.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:31.223595+0000 mon.a (mon.0) 2804 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-23T00:50:31.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:31.223669+0000 mon.a (mon.0) 2805 : cluster [DBG] osdmap e502: 8 total, 8 up, 7 in 2024-02-23T00:50:31.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:31.224619+0000 mon.a (mon.0) 2806 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-79","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-23T00:50:31.782 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:50:31.814 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:31.835 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:50:31.838 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:31.838 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:50:31.939 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:50:31.968 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:32.090 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:50:32.219 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:50:32.222 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:32.222 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:50:32.323 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:50:32.352 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:32.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:31.329078+0000 mgr.y (mgr.14517) 471 : cluster [DBG] pgmap v757: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:50:32.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:31.599287+0000 osd.0 (osd.0) 217 : cluster [DBG] 62.e scrub starts 2024-02-23T00:50:32.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:31.600813+0000 osd.0 (osd.0) 218 : cluster [DBG] 62.e scrub ok 2024-02-23T00:50:32.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:32.225758+0000 mon.a (mon.0) 2807 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-79","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-23T00:50:32.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:32.225826+0000 mon.a (mon.0) 2808 : cluster [DBG] osdmap e503: 8 total, 8 up, 7 in 2024-02-23T00:50:32.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:31.329078+0000 mgr.y (mgr.14517) 471 : cluster [DBG] pgmap v757: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:50:32.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:31.599287+0000 osd.0 (osd.0) 217 : cluster [DBG] 62.e scrub starts 2024-02-23T00:50:32.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:31.600813+0000 osd.0 (osd.0) 218 : cluster [DBG] 62.e scrub ok 2024-02-23T00:50:32.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:32.225758+0000 mon.a (mon.0) 2807 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-79","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-23T00:50:32.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:32.225826+0000 mon.a (mon.0) 2808 : cluster [DBG] osdmap e503: 8 total, 8 up, 7 in 2024-02-23T00:50:32.694 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:50:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:50:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:50:32.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:31.329078+0000 mgr.y (mgr.14517) 471 : cluster [DBG] pgmap v757: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:50:32.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:31.599287+0000 osd.0 (osd.0) 217 : cluster [DBG] 62.e scrub starts 2024-02-23T00:50:32.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:31.600813+0000 osd.0 (osd.0) 218 : cluster [DBG] 62.e scrub ok 2024-02-23T00:50:32.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:32.225758+0000 mon.a (mon.0) 2807 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-79","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-23T00:50:32.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:32.225826+0000 mon.a (mon.0) 2808 : cluster [DBG] osdmap e503: 8 total, 8 up, 7 in 2024-02-23T00:50:33.193 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:50:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:50:33.460 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:32.620585+0000 osd.0 (osd.0) 219 : cluster [DBG] 18.1d scrub starts 2024-02-23T00:50:33.460 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:32.622021+0000 osd.0 (osd.0) 220 : cluster [DBG] 18.1d scrub ok 2024-02-23T00:50:33.460 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:33.227683+0000 mon.a (mon.0) 2809 : cluster [DBG] osdmap e504: 8 total, 8 up, 7 in 2024-02-23T00:50:33.460 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:32.620585+0000 osd.0 (osd.0) 219 : cluster [DBG] 18.1d scrub starts 2024-02-23T00:50:33.460 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:32.622021+0000 osd.0 (osd.0) 220 : cluster [DBG] 18.1d scrub ok 2024-02-23T00:50:33.460 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:33.227683+0000 mon.a (mon.0) 2809 : cluster [DBG] osdmap e504: 8 total, 8 up, 7 in 2024-02-23T00:50:33.607 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:50:33.610 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:33.611 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:50:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:32.620585+0000 osd.0 (osd.0) 219 : cluster [DBG] 18.1d scrub starts 2024-02-23T00:50:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:32.622021+0000 osd.0 (osd.0) 220 : cluster [DBG] 18.1d scrub ok 2024-02-23T00:50:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:33.227683+0000 mon.a (mon.0) 2809 : cluster [DBG] osdmap e504: 8 total, 8 up, 7 in 2024-02-23T00:50:33.711 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:50:33.748 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:34.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.148+0000 7f41f71ec700 1 -- 172.21.15.47:0/2785034582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41f0102210 msgr2=0x7f41f0104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:34.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.148+0000 7f41f71ec700 1 --2- 172.21.15.47:0/2785034582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41f0102210 0x7f41f0104600 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 rx=0x7f41e8009c30 tx=0x7f41e8004c50).stop 2024-02-23T00:50:34.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.149+0000 7f41f71ec700 1 -- 172.21.15.47:0/2785034582 shutdown_connections 2024-02-23T00:50:34.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.149+0000 7f41f71ec700 1 --2- 172.21.15.47:0/2785034582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41f0102210 0x7f41f0104600 unknown :-1 s=CLOSED pgs=2560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:34.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.149+0000 7f41f71ec700 1 --2- 172.21.15.47:0/2785034582 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f41f00f8cb0 0x7f41f00f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:34.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.149+0000 7f41f71ec700 1 --2- 172.21.15.47:0/2785034582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41f00f9610 0x7f41f0101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:34.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.149+0000 7f41f71ec700 1 -- 172.21.15.47:0/2785034582 >> 172.21.15.47:0/2785034582 conn(0x7f41f00f2820 msgr2=0x7f41f00f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:34.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.149+0000 7f41f71ec700 1 -- 172.21.15.47:0/2785034582 shutdown_connections 2024-02-23T00:50:34.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.149+0000 7f41f71ec700 1 -- 172.21.15.47:0/2785034582 wait complete. 2024-02-23T00:50:34.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.149+0000 7f41f71ec700 1 librados: starting msgr at 2024-02-23T00:50:34.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.149+0000 7f41f71ec700 1 librados: starting objecter 2024-02-23T00:50:34.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.150+0000 7f41f71ec700 1 Processor -- start 2024-02-23T00:50:34.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.150+0000 7f41f71ec700 1 -- start start 2024-02-23T00:50:34.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.150+0000 7f41f71ec700 1 librados: setting wanted keys 2024-02-23T00:50:34.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.150+0000 7f41f71ec700 1 librados: calling monclient init 2024-02-23T00:50:34.153 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:50:34.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.150+0000 7f41f71ec700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41f00f8cb0 0x7f41f010ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:34.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.150+0000 7f41f71ec700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41f00f9610 0x7f41f010f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:34.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.150+0000 7f41f71ec700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f41f0102210 0x7f41f010a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:34.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.150+0000 7f41f71ec700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f010aef0 con 0x7f41f00f8cb0 2024-02-23T00:50:34.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.150+0000 7f41f71ec700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f010b060 con 0x7f41f00f9610 2024-02-23T00:50:34.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.150+0000 7f41f71ec700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f010b220 con 0x7f41f0102210 2024-02-23T00:50:34.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.150+0000 7f41f4f88700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41f00f8cb0 0x7f41f010ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:34.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.150+0000 7f41f4f88700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41f00f8cb0 0x7f41f010ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47206/0 (socket says 172.21.15.47:47206) 2024-02-23T00:50:34.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.150+0000 7f41f4f88700 1 -- 172.21.15.47:0/692219574 learned_addr learned my addr 172.21.15.47:0/692219574 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:50:34.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.150+0000 7f41f5789700 1 --2- 172.21.15.47:0/692219574 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f41f0102210 0x7f41f010a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:34.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.150+0000 7f41effff700 1 --2- 172.21.15.47:0/692219574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41f00f9610 0x7f41f010f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:34.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.150+0000 7f41f4f88700 1 -- 172.21.15.47:0/692219574 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f41f0102210 msgr2=0x7f41f010a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:34.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.150+0000 7f41f4f88700 1 --2- 172.21.15.47:0/692219574 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f41f0102210 0x7f41f010a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:34.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.150+0000 7f41f4f88700 1 -- 172.21.15.47:0/692219574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41f00f9610 msgr2=0x7f41f010f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:34.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.150+0000 7f41f4f88700 1 --2- 172.21.15.47:0/692219574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41f00f9610 0x7f41f010f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:34.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.151+0000 7f41f4f88700 1 -- 172.21.15.47:0/692219574 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e80097c0 con 0x7f41f00f8cb0 2024-02-23T00:50:34.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.151+0000 7f41f4f88700 1 --2- 172.21.15.47:0/692219574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41f00f8cb0 0x7f41f010ef10 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 rx=0x7f41e4002770 tx=0x7f41e4009e60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:34.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.151+0000 7f41edffb700 10 client.?.objecter ms_handle_connect 0x7f41f00f8cb0 2024-02-23T00:50:34.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.151+0000 7f41edffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:50:34.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.151+0000 7f41edffb700 1 -- 172.21.15.47:0/692219574 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e4007b40 con 0x7f41f00f8cb0 2024-02-23T00:50:34.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.151+0000 7f41f71ec700 1 -- 172.21.15.47:0/692219574 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f010b500 con 0x7f41f00f8cb0 2024-02-23T00:50:34.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.151+0000 7f41edffb700 1 -- 172.21.15.47:0/692219574 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f41e400f040 con 0x7f41f00f8cb0 2024-02-23T00:50:34.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.151+0000 7f41edffb700 1 -- 172.21.15.47:0/692219574 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e40146e0 con 0x7f41f00f8cb0 2024-02-23T00:50:34.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.151+0000 7f41f71ec700 10 client.31236.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:50:34.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.151+0000 7f41f71ec700 1 -- 172.21.15.47:0/692219574 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f0117010 con 0x7f41f00f8cb0 2024-02-23T00:50:34.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.151+0000 7f41f71ec700 1 librados: init done 2024-02-23T00:50:34.157 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:50:34.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.152+0000 7f41edffb700 1 -- 172.21.15.47:0/692219574 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f41e4009110 con 0x7f41f00f8cb0 2024-02-23T00:50:34.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.153+0000 7f41edffb700 1 --2- 172.21.15.47:0/692219574 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f41e0062cc0 0x7f41e0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:34.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.153+0000 7f41f71ec700 1 -- 172.21.15.47:0/692219574 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41f005fa60 con 0x7f41f00f8cb0 2024-02-23T00:50:34.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.153+0000 7f41edffb700 1 -- 172.21.15.47:0/692219574 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(504..504 src has 48..504) v4 ==== 9287+0+0 (secure 0 0 0) 0x7f41e407d250 con 0x7f41f00f8cb0 2024-02-23T00:50:34.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.153+0000 7f41edffb700 10 client.31236.objecter ms_dispatch 0x7f41f0001550 osd_map(504..504 src has 48..504) v4 2024-02-23T00:50:34.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.153+0000 7f41edffb700 3 client.31236.objecter handle_osd_map got epochs [504,504] > 0 2024-02-23T00:50:34.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.153+0000 7f41edffb700 3 client.31236.objecter handle_osd_map decoding full epoch 504 2024-02-23T00:50:34.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.153+0000 7f41effff700 1 --2- 172.21.15.47:0/692219574 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f41e0062cc0 0x7f41e0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:34.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.153+0000 7f41edffb700 20 client.31236.objecter dump_active .. 0 homeless 2024-02-23T00:50:34.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.153+0000 7f41edffb700 10 client.31236.objecter ms_handle_connect 0x7f41e0062cc0 2024-02-23T00:50:34.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.153+0000 7f41effff700 1 --2- 172.21.15.47:0/692219574 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f41e0062cc0 0x7f41e0065180 secure :-1 s=READY pgs=3102 cs=0 l=1 rev1=1 rx=0x7f41dc009a00 tx=0x7f41dc00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:34.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.158+0000 7f41edffb700 1 -- 172.21.15.47:0/692219574 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f41e404c820 con 0x7f41f00f8cb0 2024-02-23T00:50:34.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.362+0000 7f41f71ec700 1 -- 172.21.15.47:0/692219574 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7f41f005ee80 con 0x7f41f00f8cb0 2024-02-23T00:50:34.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:32.710101+0000 mgr.y (mgr.14517) 472 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:50:34.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:33.329645+0000 mgr.y (mgr.14517) 473 : cluster [DBG] pgmap v760: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:50:34.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:34.054864+0000 osd.3 (osd.3) 244 : cluster [DBG] 3.2 scrub starts 2024-02-23T00:50:34.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:34.055919+0000 osd.3 (osd.3) 245 : cluster [DBG] 3.2 scrub ok 2024-02-23T00:50:34.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:34.228005+0000 mon.a (mon.0) 2810 : cluster [DBG] osdmap e505: 8 total, 8 up, 7 in 2024-02-23T00:50:34.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:34.284064+0000 mon.a (mon.0) 2811 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:50:34.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:34.284428+0000 mon.a (mon.0) 2812 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-79"}]: dispatch 2024-02-23T00:50:34.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:32.710101+0000 mgr.y (mgr.14517) 472 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:50:34.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:33.329645+0000 mgr.y (mgr.14517) 473 : cluster [DBG] pgmap v760: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:50:34.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:34.054864+0000 osd.3 (osd.3) 244 : cluster [DBG] 3.2 scrub starts 2024-02-23T00:50:34.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:34.055919+0000 osd.3 (osd.3) 245 : cluster [DBG] 3.2 scrub ok 2024-02-23T00:50:34.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:34.228005+0000 mon.a (mon.0) 2810 : cluster [DBG] osdmap e505: 8 total, 8 up, 7 in 2024-02-23T00:50:34.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:34.284064+0000 mon.a (mon.0) 2811 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:50:34.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:34.284428+0000 mon.a (mon.0) 2812 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-79"}]: dispatch 2024-02-23T00:50:34.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.600+0000 7faaa16de700 1 -- 172.21.15.47:0/3270220577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa9c0f6aa0 msgr2=0x7faa9c0f6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.600+0000 7faaa16de700 1 --2- 172.21.15.47:0/3270220577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa9c0f6aa0 0x7faa9c0f6e80 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 rx=0x7faa8c004870 tx=0x7faa8c009cb0).stop 2024-02-23T00:50:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.600+0000 7faaa16de700 1 -- 172.21.15.47:0/3270220577 shutdown_connections 2024-02-23T00:50:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.600+0000 7faaa16de700 1 --2- 172.21.15.47:0/3270220577 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faa9c104600 0x7faa9c1069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.600+0000 7faaa16de700 1 --2- 172.21.15.47:0/3270220577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa9c0f6aa0 0x7faa9c0f6e80 unknown :-1 s=CLOSED pgs=2562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.600+0000 7faaa16de700 1 --2- 172.21.15.47:0/3270220577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa9c0f7370 0x7faa9c104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.600+0000 7faaa16de700 1 -- 172.21.15.47:0/3270220577 >> 172.21.15.47:0/3270220577 conn(0x7faa9c0f2820 msgr2=0x7faa9c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.600+0000 7faaa16de700 1 -- 172.21.15.47:0/3270220577 shutdown_connections 2024-02-23T00:50:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.600+0000 7faaa16de700 1 -- 172.21.15.47:0/3270220577 wait complete. 2024-02-23T00:50:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.600+0000 7faaa16de700 1 librados: starting msgr at 2024-02-23T00:50:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.600+0000 7faaa16de700 1 librados: starting objecter 2024-02-23T00:50:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.600+0000 7faaa16de700 1 Processor -- start 2024-02-23T00:50:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.601+0000 7faaa16de700 1 -- start start 2024-02-23T00:50:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.601+0000 7faaa16de700 1 librados: setting wanted keys 2024-02-23T00:50:34.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.601+0000 7faaa16de700 1 librados: calling monclient init 2024-02-23T00:50:34.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.601+0000 7faaa16de700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faa9c0f6aa0 0x7faa9c1102a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:34.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.601+0000 7faaa16de700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa9c0f7370 0x7faa9c110790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:34.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.601+0000 7faaa16de700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa9c104600 0x7faa9c10a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:34.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.601+0000 7faaa16de700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9c10a930 con 0x7faa9c104600 2024-02-23T00:50:34.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.601+0000 7faaa16de700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9c10aaa0 con 0x7faa9c0f7370 2024-02-23T00:50:34.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.601+0000 7faaa16de700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa9c10ac60 con 0x7faa9c0f6aa0 2024-02-23T00:50:34.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.601+0000 7faa92ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faa9c0f6aa0 0x7faa9c1102a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:34.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.601+0000 7faa9b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa9c104600 0x7faa9c10a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:34.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.601+0000 7faa92ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faa9c0f6aa0 0x7faa9c1102a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:45630/0 (socket says 172.21.15.47:45630) 2024-02-23T00:50:34.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.601+0000 7faa92ffd700 1 -- 172.21.15.47:0/856486572 learned_addr learned my addr 172.21.15.47:0/856486572 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:50:34.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.601+0000 7faa9b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa9c104600 0x7faa9c10a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47230/0 (socket says 172.21.15.47:47230) 2024-02-23T00:50:34.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.601+0000 7faa9affd700 1 --2- 172.21.15.47:0/856486572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa9c0f7370 0x7faa9c110790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:34.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.601+0000 7faa92ffd700 1 -- 172.21.15.47:0/856486572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa9c0f7370 msgr2=0x7faa9c110790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:34.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.601+0000 7faa92ffd700 1 --2- 172.21.15.47:0/856486572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa9c0f7370 0x7faa9c110790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:34.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.601+0000 7faa92ffd700 1 -- 172.21.15.47:0/856486572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa9c104600 msgr2=0x7faa9c10a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:34.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.601+0000 7faa92ffd700 1 --2- 172.21.15.47:0/856486572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa9c104600 0x7faa9c10a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:34.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.601+0000 7faa92ffd700 1 -- 172.21.15.47:0/856486572 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa8c005040 con 0x7faa9c0f6aa0 2024-02-23T00:50:34.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.601+0000 7faa9b7fe700 1 --2- 172.21.15.47:0/856486572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa9c104600 0x7faa9c10a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:50:34.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.601+0000 7faa92ffd700 1 --2- 172.21.15.47:0/856486572 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faa9c0f6aa0 0x7faa9c1102a0 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 rx=0x7faa8c009da0 tx=0x7faa8c009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:34.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.601+0000 7faa98ff9700 10 client.?.objecter ms_handle_connect 0x7faa9c0f6aa0 2024-02-23T00:50:34.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.601+0000 7faa98ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:50:34.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.601+0000 7faa98ff9700 1 -- 172.21.15.47:0/856486572 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8c00e030 con 0x7faa9c0f6aa0 2024-02-23T00:50:34.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.601+0000 7faaa16de700 1 -- 172.21.15.47:0/856486572 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa9c10aee0 con 0x7faa9c0f6aa0 2024-02-23T00:50:34.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.601+0000 7faa98ff9700 1 -- 172.21.15.47:0/856486572 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7faa8c013e70 con 0x7faa9c0f6aa0 2024-02-23T00:50:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.601+0000 7faa98ff9700 1 -- 172.21.15.47:0/856486572 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8c018070 con 0x7faa9c0f6aa0 2024-02-23T00:50:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.602+0000 7faaa16de700 10 client.41567.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:50:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.602+0000 7faaa16de700 1 -- 172.21.15.47:0/856486572 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa9c1170d0 con 0x7faa9c0f6aa0 2024-02-23T00:50:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.602+0000 7faaa16de700 1 librados: init done 2024-02-23T00:50:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.602+0000 7faaa16de700 1 -- 172.21.15.47:0/856486572 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa9c05fa60 con 0x7faa9c0f6aa0 2024-02-23T00:50:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.603+0000 7faa98ff9700 1 -- 172.21.15.47:0/856486572 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7faa8c009240 con 0x7faa9c0f6aa0 2024-02-23T00:50:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.603+0000 7faa98ff9700 1 --2- 172.21.15.47:0/856486572 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7faa88062cc0 0x7faa88065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.603+0000 7faa9affd700 1 --2- 172.21.15.47:0/856486572 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7faa88062cc0 0x7faa88065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.603+0000 7faa98ff9700 1 -- 172.21.15.47:0/856486572 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(505..505 src has 48..505) v4 ==== 9287+0+0 (secure 0 0 0) 0x7faa8c07e900 con 0x7faa9c0f6aa0 2024-02-23T00:50:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.603+0000 7faa98ff9700 10 client.41567.objecter ms_dispatch 0x7faa9c001550 osd_map(505..505 src has 48..505) v4 2024-02-23T00:50:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.603+0000 7faa98ff9700 3 client.41567.objecter handle_osd_map got epochs [505,505] > 0 2024-02-23T00:50:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.603+0000 7faa98ff9700 3 client.41567.objecter handle_osd_map decoding full epoch 505 2024-02-23T00:50:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.604+0000 7faa98ff9700 20 client.41567.objecter dump_active .. 0 homeless 2024-02-23T00:50:34.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.604+0000 7faa98ff9700 10 client.41567.objecter ms_handle_connect 0x7faa88062cc0 2024-02-23T00:50:34.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.604+0000 7faa9affd700 1 --2- 172.21.15.47:0/856486572 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7faa88062cc0 0x7faa88065180 secure :-1 s=READY pgs=3103 cs=0 l=1 rev1=1 rx=0x7faa9c10bbe0 tx=0x7faa84009de0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:34.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.608+0000 7faa98ff9700 1 -- 172.21.15.47:0/856486572 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7faa8c04ded0 con 0x7faa9c0f6aa0 2024-02-23T00:50:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:32.710101+0000 mgr.y (mgr.14517) 472 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:50:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:33.329645+0000 mgr.y (mgr.14517) 473 : cluster [DBG] pgmap v760: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:50:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:34.054864+0000 osd.3 (osd.3) 244 : cluster [DBG] 3.2 scrub starts 2024-02-23T00:50:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:34.055919+0000 osd.3 (osd.3) 245 : cluster [DBG] 3.2 scrub ok 2024-02-23T00:50:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:34.228005+0000 mon.a (mon.0) 2810 : cluster [DBG] osdmap e505: 8 total, 8 up, 7 in 2024-02-23T00:50:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:34.284064+0000 mon.a (mon.0) 2811 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:50:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:34.284428+0000 mon.a (mon.0) 2812 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-79"}]: dispatch 2024-02-23T00:50:34.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:34.795+0000 7faaa16de700 1 -- 172.21.15.47:0/856486572 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd reweight", "id": 4, "weight": 1.0} v 0) v1 -- 0x7faa9c1091e0 con 0x7faa9c0f6aa0 2024-02-23T00:50:34.899 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:50:34.901 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:34.902 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:50:35.003 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:50:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:35.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.299+0000 7f41edffb700 1 -- 172.21.15.47:0/692219574 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v506) v1 ==== 87+0+0 (secure 0 0 0) 0x7f41f005ee80 con 0x7f41f00f8cb0 2024-02-23T00:50:35.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.302+0000 7f41f71ec700 10 librados: watch_flush enter 2024-02-23T00:50:35.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.302+0000 7f41f71ec700 10 librados: watch_flush exit 2024-02-23T00:50:35.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.302+0000 7f41f71ec700 20 client.31236.objecter shutdown clearing up homeless session... 2024-02-23T00:50:35.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.302+0000 7f41f71ec700 10 client.31236.objecter successfully canceled tick 2024-02-23T00:50:35.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.303+0000 7faa98ff9700 1 -- 172.21.15.47:0/856486572 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 4, "weight": 1.0}]=0 reweighted osd.4 to 1 (10000) v506) v1 ==== 113+0+0 (secure 0 0 0) 0x7faa8c0514f0 con 0x7faa9c0f6aa0 2024-02-23T00:50:35.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.306+0000 7faaa16de700 10 librados: watch_flush enter 2024-02-23T00:50:35.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.307+0000 7faaa16de700 10 librados: watch_flush exit 2024-02-23T00:50:35.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.307+0000 7faaa16de700 20 client.41567.objecter shutdown clearing up homeless session... 2024-02-23T00:50:35.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.307+0000 7faaa16de700 10 client.41567.objecter successfully canceled tick 2024-02-23T00:50:35.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.305+0000 7f41f71ec700 1 -- 172.21.15.47:0/692219574 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f41e0062cc0 msgr2=0x7f41e0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:35.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.305+0000 7f41f71ec700 1 --2- 172.21.15.47:0/692219574 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f41e0062cc0 0x7f41e0065180 secure :-1 s=READY pgs=3102 cs=0 l=1 rev1=1 rx=0x7f41dc009a00 tx=0x7f41dc00b040).stop 2024-02-23T00:50:35.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.305+0000 7f41f71ec700 1 -- 172.21.15.47:0/692219574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41f00f8cb0 msgr2=0x7f41f010ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:35.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.305+0000 7f41f71ec700 1 --2- 172.21.15.47:0/692219574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41f00f8cb0 0x7f41f010ef10 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 rx=0x7f41e4002770 tx=0x7f41e4009e60).stop 2024-02-23T00:50:35.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.308+0000 7f41f71ec700 1 -- 172.21.15.47:0/692219574 shutdown_connections 2024-02-23T00:50:35.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.308+0000 7f41f71ec700 1 --2- 172.21.15.47:0/692219574 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f41e0062cc0 0x7f41e0065180 unknown :-1 s=CLOSED pgs=3102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:35.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.308+0000 7f41f71ec700 1 --2- 172.21.15.47:0/692219574 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f41f0102210 0x7f41f010a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:35.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.308+0000 7f41f71ec700 1 --2- 172.21.15.47:0/692219574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41f00f8cb0 0x7f41f010ef10 unknown :-1 s=CLOSED pgs=2561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:35.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.308+0000 7f41f71ec700 1 --2- 172.21.15.47:0/692219574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41f00f9610 0x7f41f010f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:35.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.308+0000 7f41f71ec700 1 -- 172.21.15.47:0/692219574 >> 172.21.15.47:0/692219574 conn(0x7f41f00f2820 msgr2=0x7f41f00f4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:35.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.308+0000 7f41f71ec700 1 -- 172.21.15.47:0/692219574 shutdown_connections 2024-02-23T00:50:35.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.308+0000 7f41f71ec700 1 -- 172.21.15.47:0/692219574 wait complete. 2024-02-23T00:50:35.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.308+0000 7f41f71ec700 1 librados: shutdown 2024-02-23T00:50:35.313 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is set 2024-02-23T00:50:35.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.311+0000 7faaa16de700 1 -- 172.21.15.47:0/856486572 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7faa88062cc0 msgr2=0x7faa88065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:35.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.311+0000 7faaa16de700 1 --2- 172.21.15.47:0/856486572 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7faa88062cc0 0x7faa88065180 secure :-1 s=READY pgs=3103 cs=0 l=1 rev1=1 rx=0x7faa9c10bbe0 tx=0x7faa84009de0).stop 2024-02-23T00:50:35.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.311+0000 7faaa16de700 1 -- 172.21.15.47:0/856486572 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faa9c0f6aa0 msgr2=0x7faa9c1102a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:35.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.311+0000 7faaa16de700 1 --2- 172.21.15.47:0/856486572 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faa9c0f6aa0 0x7faa9c1102a0 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 rx=0x7faa8c009da0 tx=0x7faa8c009cb0).stop 2024-02-23T00:50:35.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.311+0000 7faaa16de700 1 -- 172.21.15.47:0/856486572 shutdown_connections 2024-02-23T00:50:35.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.311+0000 7faaa16de700 1 --2- 172.21.15.47:0/856486572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa9c104600 0x7faa9c10a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:35.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.311+0000 7faaa16de700 1 --2- 172.21.15.47:0/856486572 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7faa88062cc0 0x7faa88065180 unknown :-1 s=CLOSED pgs=3103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:35.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.311+0000 7faaa16de700 1 --2- 172.21.15.47:0/856486572 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faa9c0f6aa0 0x7faa9c1102a0 unknown :-1 s=CLOSED pgs=2712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:35.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.311+0000 7faaa16de700 1 --2- 172.21.15.47:0/856486572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa9c0f7370 0x7faa9c110790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:35.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.311+0000 7faaa16de700 1 -- 172.21.15.47:0/856486572 >> 172.21.15.47:0/856486572 conn(0x7faa9c0f2820 msgr2=0x7faa9c106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:35.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.312+0000 7faaa16de700 1 -- 172.21.15.47:0/856486572 shutdown_connections 2024-02-23T00:50:35.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.312+0000 7faaa16de700 1 -- 172.21.15.47:0/856486572 wait complete. 2024-02-23T00:50:35.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:35.312+0000 7faaa16de700 1 librados: shutdown 2024-02-23T00:50:35.324 INFO:teuthology.orchestra.run.smithi047.stderr:reweighted osd.4 to 1 (10000) 2024-02-23T00:50:36.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:34.186562+0000 osd.7 (osd.7) 255 : cluster [DBG] 62.14 scrub starts 2024-02-23T00:50:36.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:34.190007+0000 osd.7 (osd.7) 256 : cluster [DBG] 62.14 scrub ok 2024-02-23T00:50:36.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:34.362805+0000 mon.a (mon.0) 2813 : audit [INF] from='client.? 172.21.15.47:0/692219574' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:50:36.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:34.599503+0000 osd.0 (osd.0) 221 : cluster [DBG] 2.1f scrub starts 2024-02-23T00:50:36.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:34.600588+0000 osd.0 (osd.0) 222 : cluster [DBG] 2.1f scrub ok 2024-02-23T00:50:36.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:34.796658+0000 mon.c (mon.2) 317 : audit [INF] from='client.? 172.21.15.47:0/856486572' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-23T00:50:36.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:34.796968+0000 mon.a (mon.0) 2814 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-23T00:50:36.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:35.042139+0000 osd.3 (osd.3) 246 : cluster [DBG] 4.10 scrub starts 2024-02-23T00:50:36.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:35.045607+0000 osd.3 (osd.3) 247 : cluster [DBG] 4.10 scrub ok 2024-02-23T00:50:36.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:34.186562+0000 osd.7 (osd.7) 255 : cluster [DBG] 62.14 scrub starts 2024-02-23T00:50:36.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:34.190007+0000 osd.7 (osd.7) 256 : cluster [DBG] 62.14 scrub ok 2024-02-23T00:50:36.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:34.362805+0000 mon.a (mon.0) 2813 : audit [INF] from='client.? 172.21.15.47:0/692219574' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:50:36.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:34.599503+0000 osd.0 (osd.0) 221 : cluster [DBG] 2.1f scrub starts 2024-02-23T00:50:36.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:34.600588+0000 osd.0 (osd.0) 222 : cluster [DBG] 2.1f scrub ok 2024-02-23T00:50:36.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:34.796658+0000 mon.c (mon.2) 317 : audit [INF] from='client.? 172.21.15.47:0/856486572' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-23T00:50:36.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:34.796968+0000 mon.a (mon.0) 2814 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-23T00:50:36.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:35.042139+0000 osd.3 (osd.3) 246 : cluster [DBG] 4.10 scrub starts 2024-02-23T00:50:36.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:35.045607+0000 osd.3 (osd.3) 247 : cluster [DBG] 4.10 scrub ok 2024-02-23T00:50:36.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:34.186562+0000 osd.7 (osd.7) 255 : cluster [DBG] 62.14 scrub starts 2024-02-23T00:50:36.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:34.190007+0000 osd.7 (osd.7) 256 : cluster [DBG] 62.14 scrub ok 2024-02-23T00:50:36.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:34.362805+0000 mon.a (mon.0) 2813 : audit [INF] from='client.? 172.21.15.47:0/692219574' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:50:36.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:34.599503+0000 osd.0 (osd.0) 221 : cluster [DBG] 2.1f scrub starts 2024-02-23T00:50:36.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:34.600588+0000 osd.0 (osd.0) 222 : cluster [DBG] 2.1f scrub ok 2024-02-23T00:50:36.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:34.796658+0000 mon.c (mon.2) 317 : audit [INF] from='client.? 172.21.15.47:0/856486572' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-23T00:50:36.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:34.796968+0000 mon.a (mon.0) 2814 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-23T00:50:36.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:35.042139+0000 osd.3 (osd.3) 246 : cluster [DBG] 4.10 scrub starts 2024-02-23T00:50:36.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:35.045607+0000 osd.3 (osd.3) 247 : cluster [DBG] 4.10 scrub ok 2024-02-23T00:50:36.584 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:50:36.586 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:36.587 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:50:36.688 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:50:36.720 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:36.742 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:50:36.745 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:36.745 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:50:36.846 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:50:36.876 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:35.295175+0000 mon.a (mon.0) 2815 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:50:36.876 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:35.300105+0000 mon.a (mon.0) 2816 : audit [INF] from='client.? 172.21.15.47:0/692219574' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:50:36.877 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:35.300201+0000 mon.a (mon.0) 2817 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-02-23T00:50:36.877 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:35.300270+0000 mon.a (mon.0) 2818 : cluster [DBG] osdmap e506: 8 total, 8 up, 7 in 2024-02-23T00:50:36.877 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:35.330164+0000 mgr.y (mgr.14517) 474 : cluster [DBG] pgmap v763: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:50:36.877 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:36.303737+0000 mon.a (mon.0) 2819 : cluster [DBG] osdmap e507: 8 total, 8 up, 7 in 2024-02-23T00:50:36.877 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:36.305569+0000 mon.a (mon.0) 2820 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:50:36.877 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:35.295175+0000 mon.a (mon.0) 2815 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:50:36.877 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:35.300105+0000 mon.a (mon.0) 2816 : audit [INF] from='client.? 172.21.15.47:0/692219574' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:50:36.877 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:35.300201+0000 mon.a (mon.0) 2817 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-02-23T00:50:36.877 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:35.300270+0000 mon.a (mon.0) 2818 : cluster [DBG] osdmap e506: 8 total, 8 up, 7 in 2024-02-23T00:50:36.877 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:35.330164+0000 mgr.y (mgr.14517) 474 : cluster [DBG] pgmap v763: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:50:36.877 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:36.303737+0000 mon.a (mon.0) 2819 : cluster [DBG] osdmap e507: 8 total, 8 up, 7 in 2024-02-23T00:50:36.877 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:36.305569+0000 mon.a (mon.0) 2820 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:50:36.879 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:37.130 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:50:37.133 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:37.134 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:50:37.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:35.295175+0000 mon.a (mon.0) 2815 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:50:37.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:35.300105+0000 mon.a (mon.0) 2816 : audit [INF] from='client.? 172.21.15.47:0/692219574' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:50:37.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:35.300201+0000 mon.a (mon.0) 2817 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-02-23T00:50:37.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:35.300270+0000 mon.a (mon.0) 2818 : cluster [DBG] osdmap e506: 8 total, 8 up, 7 in 2024-02-23T00:50:37.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:35.330164+0000 mgr.y (mgr.14517) 474 : cluster [DBG] pgmap v763: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:50:37.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:36.303737+0000 mon.a (mon.0) 2819 : cluster [DBG] osdmap e507: 8 total, 8 up, 7 in 2024-02-23T00:50:37.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:36.305569+0000 mon.a (mon.0) 2820 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:50:37.235 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:50:37.265 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:37.922 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:50:37.922 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:50:38.186 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight 5 1 2024-02-23T00:50:38.214 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-23T00:50:38.506 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:37.305740+0000 mon.a (mon.0) 2821 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:50:38.506 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:37.305923+0000 mon.a (mon.0) 2822 : cluster [DBG] osdmap e508: 8 total, 8 up, 7 in 2024-02-23T00:50:38.506 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:37.319207+0000 mon.a (mon.0) 2823 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:50:38.506 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:37.321751+0000 mon.a (mon.0) 2824 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-23T00:50:38.507 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:37.330534+0000 mgr.y (mgr.14517) 475 : cluster [DBG] pgmap v766: 208 pgs: 32 creating+peering, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.0 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-02-23T00:50:38.507 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:37.305740+0000 mon.a (mon.0) 2821 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:50:38.507 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:37.305923+0000 mon.a (mon.0) 2822 : cluster [DBG] osdmap e508: 8 total, 8 up, 7 in 2024-02-23T00:50:38.507 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:37.319207+0000 mon.a (mon.0) 2823 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:50:38.507 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:37.321751+0000 mon.a (mon.0) 2824 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-23T00:50:38.507 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:37.330534+0000 mgr.y (mgr.14517) 475 : cluster [DBG] pgmap v766: 208 pgs: 32 creating+peering, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.0 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-02-23T00:50:38.515 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:50:38.519 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:38.519 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:50:38.620 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:50:38.648 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:38.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:37.305740+0000 mon.a (mon.0) 2821 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:50:38.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:37.305923+0000 mon.a (mon.0) 2822 : cluster [DBG] osdmap e508: 8 total, 8 up, 7 in 2024-02-23T00:50:38.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:37.319207+0000 mon.a (mon.0) 2823 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:50:38.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:37.321751+0000 mon.a (mon.0) 2824 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-23T00:50:38.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:37.330534+0000 mgr.y (mgr.14517) 475 : cluster [DBG] pgmap v766: 208 pgs: 32 creating+peering, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.0 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-02-23T00:50:38.897 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:50:38.900 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:38.900 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:50:38.906 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.2 dump_blocked_ops 2024-02-23T00:50:39.001 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:50:39.032 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:39.649 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:38.311484+0000 mon.a (mon.0) 2825 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-23T00:50:39.649 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:38.311549+0000 mon.a (mon.0) 2826 : cluster [DBG] osdmap e509: 8 total, 8 up, 7 in 2024-02-23T00:50:39.649 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:38.314438+0000 mon.a (mon.0) 2827 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-81","var": "dedup_tier","val": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:50:39.649 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:38.311484+0000 mon.a (mon.0) 2825 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-23T00:50:39.649 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:38.311549+0000 mon.a (mon.0) 2826 : cluster [DBG] osdmap e509: 8 total, 8 up, 7 in 2024-02-23T00:50:39.649 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:38.314438+0000 mon.a (mon.0) 2827 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-81","var": "dedup_tier","val": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:50:39.682 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:50:39.687 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:39.687 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:50:39.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:38.311484+0000 mon.a (mon.0) 2825 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-23T00:50:39.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:38.311549+0000 mon.a (mon.0) 2826 : cluster [DBG] osdmap e509: 8 total, 8 up, 7 in 2024-02-23T00:50:39.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:38.314438+0000 mon.a (mon.0) 2827 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-81","var": "dedup_tier","val": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:50:39.788 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:50:39.820 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:39.842 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:50:39.845 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:39.845 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:50:39.946 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:50:39.976 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:39.996 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:50:39.998 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:39.999 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:50:40.100 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:50:40.131 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:40.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:39.312119+0000 mon.a (mon.0) 2828 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-81","var": "dedup_tier","val": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:50:40.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:39.312204+0000 mon.a (mon.0) 2829 : cluster [DBG] osdmap e510: 8 total, 8 up, 7 in 2024-02-23T00:50:40.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:39.315537+0000 mon.a (mon.0) 2830 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-23T00:50:40.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:39.330895+0000 mgr.y (mgr.14517) 476 : cluster [DBG] pgmap v769: 208 pgs: 32 creating+peering, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.0 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-02-23T00:50:40.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:40.224862+0000 mon.a (mon.0) 2831 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-23T00:50:40.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:40.224931+0000 mon.a (mon.0) 2832 : cluster [DBG] osdmap e511: 8 total, 8 up, 7 in 2024-02-23T00:50:40.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:40.226023+0000 mon.a (mon.0) 2833 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-23T00:50:40.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:39.312119+0000 mon.a (mon.0) 2828 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-81","var": "dedup_tier","val": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:50:40.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:39.312204+0000 mon.a (mon.0) 2829 : cluster [DBG] osdmap e510: 8 total, 8 up, 7 in 2024-02-23T00:50:40.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:39.315537+0000 mon.a (mon.0) 2830 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-23T00:50:40.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:39.330895+0000 mgr.y (mgr.14517) 476 : cluster [DBG] pgmap v769: 208 pgs: 32 creating+peering, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.0 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-02-23T00:50:40.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:40.224862+0000 mon.a (mon.0) 2831 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-23T00:50:40.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:40.224931+0000 mon.a (mon.0) 2832 : cluster [DBG] osdmap e511: 8 total, 8 up, 7 in 2024-02-23T00:50:40.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:40.226023+0000 mon.a (mon.0) 2833 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-23T00:50:40.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.471+0000 7f5753531700 1 -- 172.21.15.47:0/146514889 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f574c0f9620 msgr2=0x7f574c101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:40.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.471+0000 7f5753531700 1 --2- 172.21.15.47:0/146514889 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f574c0f9620 0x7f574c101dc0 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 rx=0x7f5740004870 tx=0x7f5740009cb0).stop 2024-02-23T00:50:40.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.471+0000 7f5753531700 1 -- 172.21.15.47:0/146514889 shutdown_connections 2024-02-23T00:50:40.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.471+0000 7f5753531700 1 --2- 172.21.15.47:0/146514889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f574c1022b0 0x7f574c1046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:40.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.471+0000 7f5753531700 1 --2- 172.21.15.47:0/146514889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f574c0f8d50 0x7f574c0f9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:40.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.471+0000 7f5753531700 1 --2- 172.21.15.47:0/146514889 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f574c0f9620 0x7f574c101dc0 unknown :-1 s=CLOSED pgs=2713 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:40.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.471+0000 7f5753531700 1 -- 172.21.15.47:0/146514889 >> 172.21.15.47:0/146514889 conn(0x7f574c0f2820 msgr2=0x7f574c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:40.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.472+0000 7f5753531700 1 -- 172.21.15.47:0/146514889 shutdown_connections 2024-02-23T00:50:40.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.472+0000 7f5753531700 1 -- 172.21.15.47:0/146514889 wait complete. 2024-02-23T00:50:40.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.472+0000 7f5753531700 1 librados: starting msgr at 2024-02-23T00:50:40.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.472+0000 7f5753531700 1 librados: starting objecter 2024-02-23T00:50:40.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.472+0000 7f5753531700 1 Processor -- start 2024-02-23T00:50:40.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.472+0000 7f5753531700 1 -- start start 2024-02-23T00:50:40.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.472+0000 7f5753531700 1 librados: setting wanted keys 2024-02-23T00:50:40.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.472+0000 7f5753531700 1 librados: calling monclient init 2024-02-23T00:50:40.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.472+0000 7f5753531700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f574c0f8d50 0x7f574c10e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.472+0000 7f5753531700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f574c0f9620 0x7f574c10e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.473+0000 7f5753531700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f574c1022b0 0x7f574c108200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.473+0000 7f5753531700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c1086f0 con 0x7f574c0f8d50 2024-02-23T00:50:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.473+0000 7f5753531700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c108860 con 0x7f574c0f9620 2024-02-23T00:50:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.473+0000 7f5753531700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574c108a20 con 0x7f574c1022b0 2024-02-23T00:50:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.473+0000 7f57512cd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f574c0f8d50 0x7f574c10e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.473+0000 7f5751ace700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f574c1022b0 0x7f574c108200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.473+0000 7f57512cd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f574c0f8d50 0x7f574c10e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42716/0 (socket says 172.21.15.47:42716) 2024-02-23T00:50:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.473+0000 7f57512cd700 1 -- 172.21.15.47:0/2477356923 learned_addr learned my addr 172.21.15.47:0/2477356923 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:50:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.473+0000 7f5750acc700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f574c0f9620 0x7f574c10e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.473+0000 7f5751ace700 1 -- 172.21.15.47:0/2477356923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f574c0f9620 msgr2=0x7f574c10e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.473+0000 7f5751ace700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f574c0f9620 0x7f574c10e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.473+0000 7f5751ace700 1 -- 172.21.15.47:0/2477356923 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f574c0f8d50 msgr2=0x7f574c10e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.473+0000 7f5751ace700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f574c0f8d50 0x7f574c10e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.473+0000 7f5751ace700 1 -- 172.21.15.47:0/2477356923 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5748005040 con 0x7f574c1022b0 2024-02-23T00:50:40.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.473+0000 7f57512cd700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f574c0f8d50 0x7f574c10e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:50:40.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.473+0000 7f5751ace700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f574c1022b0 0x7f574c108200 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 rx=0x7f574400fb50 tx=0x7f5744004c50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:40.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.473+0000 7f573e7fc700 10 client.?.objecter ms_handle_connect 0x7f574c1022b0 2024-02-23T00:50:40.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.473+0000 7f573e7fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:50:40.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.473+0000 7f573e7fc700 1 -- 172.21.15.47:0/2477356923 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57440077a0 con 0x7f574c1022b0 2024-02-23T00:50:40.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.473+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5740005040 con 0x7f574c1022b0 2024-02-23T00:50:40.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.473+0000 7f573e7fc700 1 -- 172.21.15.47:0/2477356923 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f574400fe60 con 0x7f574c1022b0 2024-02-23T00:50:40.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.473+0000 7f573e7fc700 1 -- 172.21.15.47:0/2477356923 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5744015700 con 0x7f574c1022b0 2024-02-23T00:50:40.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.473+0000 7f5753531700 10 client.41573.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:50:40.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.473+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574c109030 con 0x7f574c1022b0 2024-02-23T00:50:40.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.473+0000 7f5753531700 1 librados: init done 2024-02-23T00:50:40.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.474+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f574c05fa60 con 0x7f574c1022b0 2024-02-23T00:50:40.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.475+0000 7f573e7fc700 1 -- 172.21.15.47:0/2477356923 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f5744015b30 con 0x7f574c1022b0 2024-02-23T00:50:40.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.475+0000 7f573e7fc700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5738062cc0 0x7f5738065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:40.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.475+0000 7f573e7fc700 1 -- 172.21.15.47:0/2477356923 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(511..511 src has 48..511) v4 ==== 9271+0+0 (secure 0 0 0) 0x7f574407d9b0 con 0x7f574c1022b0 2024-02-23T00:50:40.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.475+0000 7f573e7fc700 10 client.41573.objecter ms_dispatch 0x7f574c001550 osd_map(511..511 src has 48..511) v4 2024-02-23T00:50:40.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.475+0000 7f573e7fc700 3 client.41573.objecter handle_osd_map got epochs [511,511] > 0 2024-02-23T00:50:40.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.475+0000 7f573e7fc700 3 client.41573.objecter handle_osd_map decoding full epoch 511 2024-02-23T00:50:40.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.475+0000 7f57512cd700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5738062cc0 0x7f5738065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:40.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.475+0000 7f573e7fc700 20 client.41573.objecter dump_active .. 0 homeless 2024-02-23T00:50:40.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.475+0000 7f573e7fc700 1 -- 172.21.15.47:0/2477356923 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v511) v1 ==== 54+0+16 (secure 0 0 0) 0x7f5744082220 con 0x7f574c1022b0 2024-02-23T00:50:40.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.476+0000 7f573e7fc700 10 client.41573.objecter ms_handle_connect 0x7f5738062cc0 2024-02-23T00:50:40.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.476+0000 7f57512cd700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5738062cc0 0x7f5738065180 secure :-1 s=READY pgs=3104 cs=0 l=1 rev1=1 rx=0x7f5748009810 tx=0x7f574800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:40.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.476+0000 7f5753531700 10 client.41573.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:40.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.476+0000 7f5753531700 15 client.41573.objecter _session_command_op_assign -1 1 2024-02-23T00:50:40.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.476+0000 7f5753531700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f574c028540 0x7f574c0fbd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:40.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.476+0000 7f5753531700 20 client.41573.objecter _get_session s=0x7f574c05ee80 osd=0 3 2024-02-23T00:50:40.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.476+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c05ee80 osd=0 3 2024-02-23T00:50:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.476+0000 7f5753531700 20 client.41573.objecter _get_session s=0x7f574c05ee80 osd=0 3 2024-02-23T00:50:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.476+0000 7f5753531700 15 client.41573.objecter _session_command_op_remove -1 1 2024-02-23T00:50:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.476+0000 7f5753531700 20 client.41573.objecter get_session s=0x7f574c05ee80 osd=0 3 2024-02-23T00:50:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.476+0000 7f5753531700 15 client.41573.objecter _session_command_op_assign 0 1 2024-02-23T00:50:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.476+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c05ee80 osd=0 4 2024-02-23T00:50:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.476+0000 7f5753531700 10 client.41573.objecter _send_command 1 2024-02-23T00:50:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.476+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f574c107050 con 0x7f574c028540 2024-02-23T00:50:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.476+0000 7f5750acc700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f574c028540 0x7f574c0fbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.476+0000 7f573e7fc700 10 client.41573.objecter ms_handle_connect 0x7f574c028540 2024-02-23T00:50:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.476+0000 7f5750acc700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f574c028540 0x7f574c0fbd70 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:40.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.477+0000 7f573e7fc700 1 -- 172.21.15.47:0/2477356923 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f574c107050 con 0x7f574c028540 2024-02-23T00:50:40.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.477+0000 7f573e7fc700 10 client.41573.objecter ms_dispatch 0x7f574c001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:50:40.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.477+0000 7f573e7fc700 10 client.41573.objecter _finish_command 1 = system:0 2024-02-23T00:50:40.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.477+0000 7f573e7fc700 20 client.41573.objecter put_session s=0x7f574c05ee80 osd=0 4 2024-02-23T00:50:40.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.477+0000 7f573e7fc700 15 client.41573.objecter _session_command_op_remove 0 1 2024-02-23T00:50:40.487 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:50:40.489 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:40.490 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:50:40.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.493+0000 7f5753531700 10 client.41573.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:50:40.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.493+0000 7f5753531700 15 client.41573.objecter _session_command_op_assign -1 2 2024-02-23T00:50:40.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.493+0000 7f5753531700 20 client.41573.objecter _get_session s=0x7f574c05ee80 osd=0 3 2024-02-23T00:50:40.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.493+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c05ee80 osd=0 3 2024-02-23T00:50:40.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.493+0000 7f5753531700 20 client.41573.objecter _get_session s=0x7f574c05ee80 osd=0 3 2024-02-23T00:50:40.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.493+0000 7f5753531700 15 client.41573.objecter _session_command_op_remove -1 2 2024-02-23T00:50:40.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.493+0000 7f5753531700 20 client.41573.objecter get_session s=0x7f574c05ee80 osd=0 3 2024-02-23T00:50:40.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.493+0000 7f5753531700 15 client.41573.objecter _session_command_op_assign 0 2 2024-02-23T00:50:40.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.493+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c05ee80 osd=0 4 2024-02-23T00:50:40.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.493+0000 7f5753531700 10 client.41573.objecter _send_command 2 2024-02-23T00:50:40.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.493+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f574c05a040 con 0x7f574c028540 2024-02-23T00:50:40.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.494+0000 7f573e7fc700 1 -- 172.21.15.47:0/2477356923 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f574c05a040 con 0x7f574c028540 2024-02-23T00:50:40.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.494+0000 7f573e7fc700 10 client.41573.objecter ms_dispatch 0x7f574c001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:50:40.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.494+0000 7f573e7fc700 10 client.41573.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:50:40.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.494+0000 7f573e7fc700 20 client.41573.objecter put_session s=0x7f574c05ee80 osd=0 4 2024-02-23T00:50:40.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.494+0000 7f573e7fc700 15 client.41573.objecter _session_command_op_remove 0 2 2024-02-23T00:50:40.498 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:50:40.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.495+0000 7f5753531700 10 client.41573.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:40.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.495+0000 7f5753531700 15 client.41573.objecter _session_command_op_assign -1 3 2024-02-23T00:50:40.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.495+0000 7f5753531700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f574c027b90 0x7f574c027fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:40.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.495+0000 7f5753531700 20 client.41573.objecter _get_session s=0x7f574c0fd3e0 osd=1 3 2024-02-23T00:50:40.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.495+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c0fd3e0 osd=1 3 2024-02-23T00:50:40.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.495+0000 7f5753531700 20 client.41573.objecter _get_session s=0x7f574c0fd3e0 osd=1 3 2024-02-23T00:50:40.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.495+0000 7f5753531700 15 client.41573.objecter _session_command_op_remove -1 3 2024-02-23T00:50:40.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.495+0000 7f5753531700 20 client.41573.objecter get_session s=0x7f574c0fd3e0 osd=1 3 2024-02-23T00:50:40.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.495+0000 7f5753531700 15 client.41573.objecter _session_command_op_assign 1 3 2024-02-23T00:50:40.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.495+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c0fd3e0 osd=1 4 2024-02-23T00:50:40.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.495+0000 7f5753531700 10 client.41573.objecter _send_command 3 2024-02-23T00:50:40.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.495+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f574c0fd550 con 0x7f574c027b90 2024-02-23T00:50:40.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.495+0000 7f57512cd700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f574c027b90 0x7f574c027fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:40.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.495+0000 7f573e7fc700 10 client.41573.objecter ms_handle_connect 0x7f574c027b90 2024-02-23T00:50:40.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.495+0000 7f57512cd700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f574c027b90 0x7f574c027fb0 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:40.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.496+0000 7f573e7fc700 1 -- 172.21.15.47:0/2477356923 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f574c0fd550 con 0x7f574c027b90 2024-02-23T00:50:40.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.496+0000 7f573e7fc700 10 client.41573.objecter ms_dispatch 0x7f574c001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:50:40.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.496+0000 7f573e7fc700 10 client.41573.objecter _finish_command 3 = system:0 2024-02-23T00:50:40.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.496+0000 7f573e7fc700 20 client.41573.objecter put_session s=0x7f574c0fd3e0 osd=1 4 2024-02-23T00:50:40.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.496+0000 7f573e7fc700 15 client.41573.objecter _session_command_op_remove 1 3 2024-02-23T00:50:40.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.514+0000 7f5753531700 10 client.41573.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:50:40.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.514+0000 7f5753531700 15 client.41573.objecter _session_command_op_assign -1 4 2024-02-23T00:50:40.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.514+0000 7f5753531700 20 client.41573.objecter _get_session s=0x7f574c0fd3e0 osd=1 3 2024-02-23T00:50:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.514+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c0fd3e0 osd=1 3 2024-02-23T00:50:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.514+0000 7f5753531700 20 client.41573.objecter _get_session s=0x7f574c0fd3e0 osd=1 3 2024-02-23T00:50:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.514+0000 7f5753531700 15 client.41573.objecter _session_command_op_remove -1 4 2024-02-23T00:50:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.514+0000 7f5753531700 20 client.41573.objecter get_session s=0x7f574c0fd3e0 osd=1 3 2024-02-23T00:50:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.514+0000 7f5753531700 15 client.41573.objecter _session_command_op_assign 1 4 2024-02-23T00:50:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.514+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c0fd3e0 osd=1 4 2024-02-23T00:50:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.514+0000 7f5753531700 10 client.41573.objecter _send_command 4 2024-02-23T00:50:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.514+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f574c027600 con 0x7f574c027b90 2024-02-23T00:50:40.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.515+0000 7f573e7fc700 1 -- 172.21.15.47:0/2477356923 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f574c027600 con 0x7f574c027b90 2024-02-23T00:50:40.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.515+0000 7f573e7fc700 10 client.41573.objecter ms_dispatch 0x7f574c001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:50:40.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.515+0000 7f573e7fc700 10 client.41573.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:50:40.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.515+0000 7f573e7fc700 20 client.41573.objecter put_session s=0x7f574c0fd3e0 osd=1 4 2024-02-23T00:50:40.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.515+0000 7f573e7fc700 15 client.41573.objecter _session_command_op_remove 1 4 2024-02-23T00:50:40.519 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:50:40.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.516+0000 7f5753531700 10 client.41573.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:40.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.516+0000 7f5753531700 15 client.41573.objecter _session_command_op_assign -1 5 2024-02-23T00:50:40.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.516+0000 7f5753531700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f574c0fcf40 0x7f574c11c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:40.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.516+0000 7f5753531700 20 client.41573.objecter _get_session s=0x7f574c027a10 osd=2 3 2024-02-23T00:50:40.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.516+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c027a10 osd=2 3 2024-02-23T00:50:40.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.516+0000 7f5753531700 20 client.41573.objecter _get_session s=0x7f574c027a10 osd=2 3 2024-02-23T00:50:40.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.516+0000 7f5753531700 15 client.41573.objecter _session_command_op_remove -1 5 2024-02-23T00:50:40.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.516+0000 7f5753531700 20 client.41573.objecter get_session s=0x7f574c027a10 osd=2 3 2024-02-23T00:50:40.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.516+0000 7f5753531700 15 client.41573.objecter _session_command_op_assign 2 5 2024-02-23T00:50:40.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.516+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c027a10 osd=2 4 2024-02-23T00:50:40.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.516+0000 7f5753531700 10 client.41573.objecter _send_command 5 2024-02-23T00:50:40.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.516+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f574c11ee00 con 0x7f574c0fcf40 2024-02-23T00:50:40.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.516+0000 7f5750acc700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f574c0fcf40 0x7f574c11c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:40.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.516+0000 7f573e7fc700 10 client.41573.objecter ms_handle_connect 0x7f574c0fcf40 2024-02-23T00:50:40.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.516+0000 7f5750acc700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f574c0fcf40 0x7f574c11c040 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:40.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.517+0000 7f573e7fc700 1 -- 172.21.15.47:0/2477356923 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f574c11ee00 con 0x7f574c0fcf40 2024-02-23T00:50:40.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.517+0000 7f573e7fc700 10 client.41573.objecter ms_dispatch 0x7f574c001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:50:40.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.517+0000 7f573e7fc700 10 client.41573.objecter _finish_command 5 = system:0 2024-02-23T00:50:40.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.517+0000 7f573e7fc700 20 client.41573.objecter put_session s=0x7f574c027a10 osd=2 4 2024-02-23T00:50:40.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.517+0000 7f573e7fc700 15 client.41573.objecter _session_command_op_remove 2 5 2024-02-23T00:50:40.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.532+0000 7f5753531700 10 client.41573.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:50:40.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.532+0000 7f5753531700 15 client.41573.objecter _session_command_op_assign -1 6 2024-02-23T00:50:40.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.532+0000 7f5753531700 20 client.41573.objecter _get_session s=0x7f574c027a10 osd=2 3 2024-02-23T00:50:40.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.532+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c027a10 osd=2 3 2024-02-23T00:50:40.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.532+0000 7f5753531700 20 client.41573.objecter _get_session s=0x7f574c027a10 osd=2 3 2024-02-23T00:50:40.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.532+0000 7f5753531700 15 client.41573.objecter _session_command_op_remove -1 6 2024-02-23T00:50:40.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.532+0000 7f5753531700 20 client.41573.objecter get_session s=0x7f574c027a10 osd=2 3 2024-02-23T00:50:40.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.532+0000 7f5753531700 15 client.41573.objecter _session_command_op_assign 2 6 2024-02-23T00:50:40.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.532+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c027a10 osd=2 4 2024-02-23T00:50:40.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.532+0000 7f5753531700 10 client.41573.objecter _send_command 6 2024-02-23T00:50:40.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.532+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f574c0fc4a0 con 0x7f574c0fcf40 2024-02-23T00:50:40.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.533+0000 7f573e7fc700 1 -- 172.21.15.47:0/2477356923 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f574c0fc4a0 con 0x7f574c0fcf40 2024-02-23T00:50:40.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.533+0000 7f573e7fc700 10 client.41573.objecter ms_dispatch 0x7f574c001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:50:40.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.533+0000 7f573e7fc700 10 client.41573.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:50:40.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.533+0000 7f573e7fc700 20 client.41573.objecter put_session s=0x7f574c027a10 osd=2 4 2024-02-23T00:50:40.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.533+0000 7f573e7fc700 15 client.41573.objecter _session_command_op_remove 2 6 2024-02-23T00:50:40.536 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:50:40.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.533+0000 7f5753531700 10 client.41573.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:40.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.533+0000 7f5753531700 15 client.41573.objecter _session_command_op_assign -1 7 2024-02-23T00:50:40.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.533+0000 7f5753531700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f574c11d700 0x7f574c11db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:40.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.533+0000 7f5753531700 20 client.41573.objecter _get_session s=0x7f574c119680 osd=3 3 2024-02-23T00:50:40.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.533+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c119680 osd=3 3 2024-02-23T00:50:40.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.533+0000 7f5753531700 20 client.41573.objecter _get_session s=0x7f574c119680 osd=3 3 2024-02-23T00:50:40.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.533+0000 7f5753531700 15 client.41573.objecter _session_command_op_remove -1 7 2024-02-23T00:50:40.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.533+0000 7f5753531700 20 client.41573.objecter get_session s=0x7f574c119680 osd=3 3 2024-02-23T00:50:40.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.533+0000 7f5753531700 15 client.41573.objecter _session_command_op_assign 3 7 2024-02-23T00:50:40.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.533+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c119680 osd=3 4 2024-02-23T00:50:40.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.533+0000 7f5753531700 10 client.41573.objecter _send_command 7 2024-02-23T00:50:40.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.533+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f574c119dd0 con 0x7f574c11d700 2024-02-23T00:50:40.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.534+0000 7f5751ace700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f574c11d700 0x7f574c11db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:40.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.534+0000 7f573e7fc700 10 client.41573.objecter ms_handle_connect 0x7f574c11d700 2024-02-23T00:50:40.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.534+0000 7f5751ace700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f574c11d700 0x7f574c11db40 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:40.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.534+0000 7f573e7fc700 1 -- 172.21.15.47:0/2477356923 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f574c119dd0 con 0x7f574c11d700 2024-02-23T00:50:40.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.534+0000 7f573e7fc700 10 client.41573.objecter ms_dispatch 0x7f574c001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:50:40.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.534+0000 7f573e7fc700 10 client.41573.objecter _finish_command 7 = system:0 2024-02-23T00:50:40.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.534+0000 7f573e7fc700 20 client.41573.objecter put_session s=0x7f574c119680 osd=3 4 2024-02-23T00:50:40.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.534+0000 7f573e7fc700 15 client.41573.objecter _session_command_op_remove 3 7 2024-02-23T00:50:40.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.550+0000 7f5753531700 10 client.41573.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:50:40.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.550+0000 7f5753531700 15 client.41573.objecter _session_command_op_assign -1 8 2024-02-23T00:50:40.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.550+0000 7f5753531700 20 client.41573.objecter _get_session s=0x7f574c119680 osd=3 3 2024-02-23T00:50:40.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.550+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c119680 osd=3 3 2024-02-23T00:50:40.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.550+0000 7f5753531700 20 client.41573.objecter _get_session s=0x7f574c119680 osd=3 3 2024-02-23T00:50:40.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.550+0000 7f5753531700 15 client.41573.objecter _session_command_op_remove -1 8 2024-02-23T00:50:40.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.550+0000 7f5753531700 20 client.41573.objecter get_session s=0x7f574c119680 osd=3 3 2024-02-23T00:50:40.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.550+0000 7f5753531700 15 client.41573.objecter _session_command_op_assign 3 8 2024-02-23T00:50:40.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.550+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c119680 osd=3 4 2024-02-23T00:50:40.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.550+0000 7f5753531700 10 client.41573.objecter _send_command 8 2024-02-23T00:50:40.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.550+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f574c11e800 con 0x7f574c11d700 2024-02-23T00:50:40.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.552+0000 7f573e7fc700 1 -- 172.21.15.47:0/2477356923 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f574c11e800 con 0x7f574c11d700 2024-02-23T00:50:40.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.552+0000 7f573e7fc700 10 client.41573.objecter ms_dispatch 0x7f574c001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:50:40.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.552+0000 7f573e7fc700 10 client.41573.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:50:40.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.552+0000 7f573e7fc700 20 client.41573.objecter put_session s=0x7f574c119680 osd=3 4 2024-02-23T00:50:40.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.552+0000 7f573e7fc700 15 client.41573.objecter _session_command_op_remove 3 8 2024-02-23T00:50:40.555 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:50:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.552+0000 7f5753531700 10 client.41573.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.552+0000 7f5753531700 15 client.41573.objecter _session_command_op_assign -1 9 2024-02-23T00:50:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.552+0000 7f5753531700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f574c0f6b40 0x7f574c0f6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.552+0000 7f5753531700 20 client.41573.objecter _get_session s=0x7f574c11e030 osd=4 3 2024-02-23T00:50:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.552+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c11e030 osd=4 3 2024-02-23T00:50:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.552+0000 7f5753531700 20 client.41573.objecter _get_session s=0x7f574c11e030 osd=4 3 2024-02-23T00:50:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.552+0000 7f5753531700 15 client.41573.objecter _session_command_op_remove -1 9 2024-02-23T00:50:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.552+0000 7f5753531700 20 client.41573.objecter get_session s=0x7f574c11e030 osd=4 3 2024-02-23T00:50:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.552+0000 7f5753531700 15 client.41573.objecter _session_command_op_assign 4 9 2024-02-23T00:50:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.552+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c11e030 osd=4 4 2024-02-23T00:50:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.552+0000 7f5753531700 10 client.41573.objecter _send_command 9 2024-02-23T00:50:40.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.552+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f574c11e770 con 0x7f574c0f6b40 2024-02-23T00:50:40.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.553+0000 7f57512cd700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f574c0f6b40 0x7f574c0f6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:40.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.553+0000 7f573e7fc700 10 client.41573.objecter ms_handle_connect 0x7f574c0f6b40 2024-02-23T00:50:40.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.553+0000 7f57512cd700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f574c0f6b40 0x7f574c0f6f60 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:40.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.554+0000 7f573e7fc700 1 -- 172.21.15.47:0/2477356923 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f574c11e770 con 0x7f574c0f6b40 2024-02-23T00:50:40.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.554+0000 7f573e7fc700 10 client.41573.objecter ms_dispatch 0x7f574c001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:50:40.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.554+0000 7f573e7fc700 10 client.41573.objecter _finish_command 9 = system:0 2024-02-23T00:50:40.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.554+0000 7f573e7fc700 20 client.41573.objecter put_session s=0x7f574c11e030 osd=4 4 2024-02-23T00:50:40.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.554+0000 7f573e7fc700 15 client.41573.objecter _session_command_op_remove 4 9 2024-02-23T00:50:40.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.579+0000 7f5753531700 10 client.41573.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:50:40.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.579+0000 7f5753531700 15 client.41573.objecter _session_command_op_assign -1 10 2024-02-23T00:50:40.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.579+0000 7f5753531700 20 client.41573.objecter _get_session s=0x7f574c11e030 osd=4 3 2024-02-23T00:50:40.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.579+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c11e030 osd=4 3 2024-02-23T00:50:40.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.579+0000 7f5753531700 20 client.41573.objecter _get_session s=0x7f574c11e030 osd=4 3 2024-02-23T00:50:40.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.579+0000 7f5753531700 15 client.41573.objecter _session_command_op_remove -1 10 2024-02-23T00:50:40.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.579+0000 7f5753531700 20 client.41573.objecter get_session s=0x7f574c11e030 osd=4 3 2024-02-23T00:50:40.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.579+0000 7f5753531700 15 client.41573.objecter _session_command_op_assign 4 10 2024-02-23T00:50:40.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.579+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c11e030 osd=4 4 2024-02-23T00:50:40.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.579+0000 7f5753531700 10 client.41573.objecter _send_command 10 2024-02-23T00:50:40.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.579+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f574c11e770 con 0x7f574c0f6b40 2024-02-23T00:50:40.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.581+0000 7f573e7fc700 1 -- 172.21.15.47:0/2477356923 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f574c11e770 con 0x7f574c0f6b40 2024-02-23T00:50:40.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.581+0000 7f573e7fc700 10 client.41573.objecter ms_dispatch 0x7f574c001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:50:40.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.581+0000 7f573e7fc700 10 client.41573.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:50:40.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.581+0000 7f573e7fc700 20 client.41573.objecter put_session s=0x7f574c11e030 osd=4 4 2024-02-23T00:50:40.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.581+0000 7f573e7fc700 15 client.41573.objecter _session_command_op_remove 4 10 2024-02-23T00:50:40.585 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:50:40.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.582+0000 7f5753531700 10 client.41573.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:40.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.582+0000 7f5753531700 15 client.41573.objecter _session_command_op_assign -1 11 2024-02-23T00:50:40.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.582+0000 7f5753531700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f574c0f7960 0x7f574c0f7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:40.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.582+0000 7f5753531700 20 client.41573.objecter _get_session s=0x7f574c11e770 osd=5 3 2024-02-23T00:50:40.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.582+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c11e770 osd=5 3 2024-02-23T00:50:40.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.582+0000 7f5753531700 20 client.41573.objecter _get_session s=0x7f574c11e770 osd=5 3 2024-02-23T00:50:40.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.582+0000 7f5753531700 15 client.41573.objecter _session_command_op_remove -1 11 2024-02-23T00:50:40.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.582+0000 7f5753531700 20 client.41573.objecter get_session s=0x7f574c11e770 osd=5 3 2024-02-23T00:50:40.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.582+0000 7f5753531700 15 client.41573.objecter _session_command_op_assign 5 11 2024-02-23T00:50:40.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.582+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c11e770 osd=5 4 2024-02-23T00:50:40.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.582+0000 7f5753531700 10 client.41573.objecter _send_command 11 2024-02-23T00:50:40.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.582+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f574c0f8290 con 0x7f574c0f7960 2024-02-23T00:50:40.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.583+0000 7f5750acc700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f574c0f7960 0x7f574c0f7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:40.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.583+0000 7f573e7fc700 10 client.41573.objecter ms_handle_connect 0x7f574c0f7960 2024-02-23T00:50:40.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.583+0000 7f5750acc700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f574c0f7960 0x7f574c0f7da0 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:40.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.584+0000 7f573e7fc700 1 -- 172.21.15.47:0/2477356923 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f574c0f8290 con 0x7f574c0f7960 2024-02-23T00:50:40.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.584+0000 7f573e7fc700 10 client.41573.objecter ms_dispatch 0x7f574c001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:50:40.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.584+0000 7f573e7fc700 10 client.41573.objecter _finish_command 11 = system:0 2024-02-23T00:50:40.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.584+0000 7f573e7fc700 20 client.41573.objecter put_session s=0x7f574c11e770 osd=5 4 2024-02-23T00:50:40.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.584+0000 7f573e7fc700 15 client.41573.objecter _session_command_op_remove 5 11 2024-02-23T00:50:40.590 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:50:40.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.600+0000 7f5753531700 10 client.41573.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:50:40.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.600+0000 7f5753531700 15 client.41573.objecter _session_command_op_assign -1 12 2024-02-23T00:50:40.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.600+0000 7f5753531700 20 client.41573.objecter _get_session s=0x7f574c11e770 osd=5 3 2024-02-23T00:50:40.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.600+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c11e770 osd=5 3 2024-02-23T00:50:40.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.600+0000 7f5753531700 20 client.41573.objecter _get_session s=0x7f574c11e770 osd=5 3 2024-02-23T00:50:40.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.600+0000 7f5753531700 15 client.41573.objecter _session_command_op_remove -1 12 2024-02-23T00:50:40.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.600+0000 7f5753531700 20 client.41573.objecter get_session s=0x7f574c11e770 osd=5 3 2024-02-23T00:50:40.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.600+0000 7f5753531700 15 client.41573.objecter _session_command_op_assign 5 12 2024-02-23T00:50:40.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.600+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c11e770 osd=5 4 2024-02-23T00:50:40.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.600+0000 7f5753531700 10 client.41573.objecter _send_command 12 2024-02-23T00:50:40.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.600+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f574c0f8290 con 0x7f574c0f7960 2024-02-23T00:50:40.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.602+0000 7f573e7fc700 1 -- 172.21.15.47:0/2477356923 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f574c0f8290 con 0x7f574c0f7960 2024-02-23T00:50:40.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.602+0000 7f573e7fc700 10 client.41573.objecter ms_dispatch 0x7f574c001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:50:40.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.602+0000 7f573e7fc700 10 client.41573.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:50:40.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.602+0000 7f573e7fc700 20 client.41573.objecter put_session s=0x7f574c11e770 osd=5 4 2024-02-23T00:50:40.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.602+0000 7f573e7fc700 15 client.41573.objecter _session_command_op_remove 5 12 2024-02-23T00:50:40.606 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:50:40.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.603+0000 7f5753531700 10 client.41573.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:40.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.603+0000 7f5753531700 15 client.41573.objecter _session_command_op_assign -1 13 2024-02-23T00:50:40.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.603+0000 7f5753531700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f574c1293b0 0x7f574c12b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:40.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.603+0000 7f5753531700 20 client.41573.objecter _get_session s=0x7f574c0f8290 osd=6 3 2024-02-23T00:50:40.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.603+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c0f8290 osd=6 3 2024-02-23T00:50:40.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.603+0000 7f5753531700 20 client.41573.objecter _get_session s=0x7f574c0f8290 osd=6 3 2024-02-23T00:50:40.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.603+0000 7f5753531700 15 client.41573.objecter _session_command_op_remove -1 13 2024-02-23T00:50:40.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.603+0000 7f5753531700 20 client.41573.objecter get_session s=0x7f574c0f8290 osd=6 3 2024-02-23T00:50:40.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.603+0000 7f5753531700 15 client.41573.objecter _session_command_op_assign 6 13 2024-02-23T00:50:40.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.603+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c0f8290 osd=6 4 2024-02-23T00:50:40.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.603+0000 7f5753531700 10 client.41573.objecter _send_command 13 2024-02-23T00:50:40.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.603+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f574c12e540 con 0x7f574c1293b0 2024-02-23T00:50:40.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.604+0000 7f5751ace700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f574c1293b0 0x7f574c12b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:40.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.605+0000 7f573e7fc700 10 client.41573.objecter ms_handle_connect 0x7f574c1293b0 2024-02-23T00:50:40.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.605+0000 7f5751ace700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f574c1293b0 0x7f574c12b7a0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:40.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.606+0000 7f573e7fc700 1 -- 172.21.15.47:0/2477356923 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f574c12e540 con 0x7f574c1293b0 2024-02-23T00:50:40.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.606+0000 7f573e7fc700 10 client.41573.objecter ms_dispatch 0x7f574c001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:50:40.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.606+0000 7f573e7fc700 10 client.41573.objecter _finish_command 13 = system:0 2024-02-23T00:50:40.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.606+0000 7f573e7fc700 20 client.41573.objecter put_session s=0x7f574c0f8290 osd=6 4 2024-02-23T00:50:40.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.606+0000 7f573e7fc700 15 client.41573.objecter _session_command_op_remove 6 13 2024-02-23T00:50:40.614 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:50:40.615 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:50:40.622 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:50:40.624 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:40.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.630+0000 7f5753531700 10 client.41573.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:50:40.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.630+0000 7f5753531700 15 client.41573.objecter _session_command_op_assign -1 14 2024-02-23T00:50:40.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.630+0000 7f5753531700 20 client.41573.objecter _get_session s=0x7f574c0f8290 osd=6 3 2024-02-23T00:50:40.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.630+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c0f8290 osd=6 3 2024-02-23T00:50:40.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.630+0000 7f5753531700 20 client.41573.objecter _get_session s=0x7f574c0f8290 osd=6 3 2024-02-23T00:50:40.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.630+0000 7f5753531700 15 client.41573.objecter _session_command_op_remove -1 14 2024-02-23T00:50:40.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.630+0000 7f5753531700 20 client.41573.objecter get_session s=0x7f574c0f8290 osd=6 3 2024-02-23T00:50:40.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.630+0000 7f5753531700 15 client.41573.objecter _session_command_op_assign 6 14 2024-02-23T00:50:40.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.630+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c0f8290 osd=6 4 2024-02-23T00:50:40.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.630+0000 7f5753531700 10 client.41573.objecter _send_command 14 2024-02-23T00:50:40.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.630+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f574c12e540 con 0x7f574c1293b0 2024-02-23T00:50:40.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.640+0000 7f573e7fc700 1 -- 172.21.15.47:0/2477356923 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f574c12e540 con 0x7f574c1293b0 2024-02-23T00:50:40.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.640+0000 7f573e7fc700 10 client.41573.objecter ms_dispatch 0x7f574c001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:50:40.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.640+0000 7f573e7fc700 10 client.41573.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:50:40.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.640+0000 7f573e7fc700 20 client.41573.objecter put_session s=0x7f574c0f8290 osd=6 4 2024-02-23T00:50:40.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.640+0000 7f573e7fc700 15 client.41573.objecter _session_command_op_remove 6 14 2024-02-23T00:50:40.648 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:50:40.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.646+0000 7f5753531700 10 client.41573.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:40.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.646+0000 7f5753531700 15 client.41573.objecter _session_command_op_assign -1 15 2024-02-23T00:50:40.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.646+0000 7f5753531700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f574c12d8b0 0x7f574c12dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:40.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.646+0000 7f5753531700 20 client.41573.objecter _get_session s=0x7f574c12e540 osd=7 3 2024-02-23T00:50:40.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.646+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c12e540 osd=7 3 2024-02-23T00:50:40.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.646+0000 7f5753531700 20 client.41573.objecter _get_session s=0x7f574c12e540 osd=7 3 2024-02-23T00:50:40.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.646+0000 7f5753531700 15 client.41573.objecter _session_command_op_remove -1 15 2024-02-23T00:50:40.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.646+0000 7f5753531700 20 client.41573.objecter get_session s=0x7f574c12e540 osd=7 3 2024-02-23T00:50:40.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.646+0000 7f5753531700 15 client.41573.objecter _session_command_op_assign 7 15 2024-02-23T00:50:40.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.646+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c12e540 osd=7 4 2024-02-23T00:50:40.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.646+0000 7f5753531700 10 client.41573.objecter _send_command 15 2024-02-23T00:50:40.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.646+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f574c131870 con 0x7f574c12d8b0 2024-02-23T00:50:40.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.646+0000 7f57512cd700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f574c12d8b0 0x7f574c12dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:40.651 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:50:40.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.648+0000 7f57512cd700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f574c12d8b0 0x7f574c12dd10 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:40.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.648+0000 7f573e7fc700 10 client.41573.objecter ms_handle_connect 0x7f574c12d8b0 2024-02-23T00:50:40.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.650+0000 7f573e7fc700 1 -- 172.21.15.47:0/2477356923 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f574c131870 con 0x7f574c12d8b0 2024-02-23T00:50:40.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.650+0000 7f573e7fc700 10 client.41573.objecter ms_dispatch 0x7f574c001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:50:40.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.650+0000 7f573e7fc700 10 client.41573.objecter _finish_command 15 = system:0 2024-02-23T00:50:40.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.650+0000 7f573e7fc700 20 client.41573.objecter put_session s=0x7f574c12e540 osd=7 4 2024-02-23T00:50:40.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.650+0000 7f573e7fc700 15 client.41573.objecter _session_command_op_remove 7 15 2024-02-23T00:50:40.654 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:40.654 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:50:40.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.672+0000 7f5753531700 10 client.41573.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:50:40.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.672+0000 7f5753531700 15 client.41573.objecter _session_command_op_assign -1 16 2024-02-23T00:50:40.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.672+0000 7f5753531700 20 client.41573.objecter _get_session s=0x7f574c12e540 osd=7 3 2024-02-23T00:50:40.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.672+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c12e540 osd=7 3 2024-02-23T00:50:40.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.672+0000 7f5753531700 20 client.41573.objecter _get_session s=0x7f574c12e540 osd=7 3 2024-02-23T00:50:40.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.672+0000 7f5753531700 15 client.41573.objecter _session_command_op_remove -1 16 2024-02-23T00:50:40.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.672+0000 7f5753531700 20 client.41573.objecter get_session s=0x7f574c12e540 osd=7 3 2024-02-23T00:50:40.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.672+0000 7f5753531700 15 client.41573.objecter _session_command_op_assign 7 16 2024-02-23T00:50:40.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.672+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c12e540 osd=7 4 2024-02-23T00:50:40.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.672+0000 7f5753531700 10 client.41573.objecter _send_command 16 2024-02-23T00:50:40.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.672+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f574c12bd00 con 0x7f574c12d8b0 2024-02-23T00:50:40.676 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:50:40.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f573e7fc700 1 -- 172.21.15.47:0/2477356923 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f574c12bd00 con 0x7f574c12d8b0 2024-02-23T00:50:40.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f573e7fc700 10 client.41573.objecter ms_dispatch 0x7f574c001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:50:40.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f573e7fc700 10 client.41573.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:50:40.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f573e7fc700 20 client.41573.objecter put_session s=0x7f574c12e540 osd=7 4 2024-02-23T00:50:40.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f573e7fc700 15 client.41573.objecter _session_command_op_remove 7 16 2024-02-23T00:50:40.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 10 librados: watch_flush enter 2024-02-23T00:50:40.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 10 librados: watch_flush exit 2024-02-23T00:50:40.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 10 client.41573.objecter close_session for osd.0 2024-02-23T00:50:40.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f574c028540 msgr2=0x7f574c0fbd70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:40.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f574c028540 0x7f574c0fbd70 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:40.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c05ee80 osd=0 1 2024-02-23T00:50:40.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 10 client.41573.objecter close_session for osd.1 2024-02-23T00:50:40.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f574c027b90 msgr2=0x7f574c027fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:40.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f574c027b90 0x7f574c027fb0 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:40.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c0fd3e0 osd=1 1 2024-02-23T00:50:40.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 10 client.41573.objecter close_session for osd.2 2024-02-23T00:50:40.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f574c0fcf40 msgr2=0x7f574c11c040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:40.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f574c0fcf40 0x7f574c11c040 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:40.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c027a10 osd=2 1 2024-02-23T00:50:40.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 10 client.41573.objecter close_session for osd.3 2024-02-23T00:50:40.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f574c11d700 msgr2=0x7f574c11db40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:40.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f574c11d700 0x7f574c11db40 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:40.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5751ace700 1 -- 172.21.15.47:0/2477356923 reap_dead start 2024-02-23T00:50:40.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c119680 osd=3 1 2024-02-23T00:50:40.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 10 client.41573.objecter close_session for osd.4 2024-02-23T00:50:40.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f574c0f6b40 msgr2=0x7f574c0f6f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:40.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f574c0f6b40 0x7f574c0f6f60 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:40.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5751ace700 1 -- 172.21.15.47:0/2477356923 reap_dead start 2024-02-23T00:50:40.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c11e030 osd=4 1 2024-02-23T00:50:40.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 10 client.41573.objecter close_session for osd.5 2024-02-23T00:50:40.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f574c0f7960 msgr2=0x7f574c0f7da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:40.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f574c0f7960 0x7f574c0f7da0 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:40.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c11e770 osd=5 1 2024-02-23T00:50:40.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 10 client.41573.objecter close_session for osd.6 2024-02-23T00:50:40.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f574c1293b0 msgr2=0x7f574c12b7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:40.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f574c1293b0 0x7f574c12b7a0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:40.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c0f8290 osd=6 1 2024-02-23T00:50:40.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 10 client.41573.objecter close_session for osd.7 2024-02-23T00:50:40.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f574c12d8b0 msgr2=0x7f574c12dd10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:40.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f574c12d8b0 0x7f574c12dd10 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:40.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 20 client.41573.objecter put_session s=0x7f574c12e540 osd=7 1 2024-02-23T00:50:40.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 20 client.41573.objecter shutdown clearing up homeless session... 2024-02-23T00:50:40.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 10 client.41573.objecter successfully canceled tick 2024-02-23T00:50:40.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5738062cc0 msgr2=0x7f5738065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:40.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.674+0000 7f5753531700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5738062cc0 0x7f5738065180 secure :-1 s=READY pgs=3104 cs=0 l=1 rev1=1 rx=0x7f5748009810 tx=0x7f574800b040).stop 2024-02-23T00:50:40.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.675+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f574c1022b0 msgr2=0x7f574c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:40.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.675+0000 7f5751ace700 1 -- 172.21.15.47:0/2477356923 reap_dead start 2024-02-23T00:50:40.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.675+0000 7f5753531700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f574c1022b0 0x7f574c108200 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 rx=0x7f574400fb50 tx=0x7f5744004c50).stop 2024-02-23T00:50:40.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.675+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 shutdown_connections 2024-02-23T00:50:40.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.675+0000 7f5753531700 1 --2- 172.21.15.47:0/2477356923 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f574c1022b0 0x7f574c108200 unknown :-1 s=CLOSED pgs=2714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:40.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.675+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 >> 172.21.15.47:0/2477356923 conn(0x7f574c0f2820 msgr2=0x7f574c0f3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:40.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.675+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 shutdown_connections 2024-02-23T00:50:40.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.675+0000 7f5753531700 1 -- 172.21.15.47:0/2477356923 wait complete. 2024-02-23T00:50:40.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:40.675+0000 7f5753531700 1 librados: shutdown 2024-02-23T00:50:40.682 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:50:40.682 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:50:40.682 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:50:40.682 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:50:40.682 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:50:40.682 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:50:40.683 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:50:40.683 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:50:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:39.312119+0000 mon.a (mon.0) 2828 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-81","var": "dedup_tier","val": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:50:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:39.312204+0000 mon.a (mon.0) 2829 : cluster [DBG] osdmap e510: 8 total, 8 up, 7 in 2024-02-23T00:50:40.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:39.315537+0000 mon.a (mon.0) 2830 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-23T00:50:40.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:39.330895+0000 mgr.y (mgr.14517) 476 : cluster [DBG] pgmap v769: 208 pgs: 32 creating+peering, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.0 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-02-23T00:50:40.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:40.224862+0000 mon.a (mon.0) 2831 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-23T00:50:40.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:40.224931+0000 mon.a (mon.0) 2832 : cluster [DBG] osdmap e511: 8 total, 8 up, 7 in 2024-02-23T00:50:40.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:40.226023+0000 mon.a (mon.0) 2833 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-23T00:50:40.755 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:50:40.782 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:50:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:50:40] "GET /metrics HTTP/1.1" 200 186989 "" "Prometheus/2.33.4" 2024-02-23T00:50:40.783 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:40.882 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:50:40.884 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:40.885 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:50:40.986 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:50:41.018 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:41.039 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:50:41.042 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:41.042 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:50:41.143 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:50:41.172 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:50:41.204 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:41.205 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:50:41.305 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:50:41.334 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:41.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:40.476543+0000 mon.c (mon.2) 318 : audit [DBG] from='client.? 172.21.15.47:0/2477356923' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:50:41.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:41.226051+0000 mon.a (mon.0) 2834 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-23T00:50:41.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:41.226120+0000 mon.a (mon.0) 2835 : cluster [DBG] osdmap e512: 8 total, 8 up, 7 in 2024-02-23T00:50:41.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:41.305423+0000 mon.a (mon.0) 2836 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:50:41.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:40.476543+0000 mon.c (mon.2) 318 : audit [DBG] from='client.? 172.21.15.47:0/2477356923' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:50:41.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:41.226051+0000 mon.a (mon.0) 2834 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-23T00:50:41.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:41.226120+0000 mon.a (mon.0) 2835 : cluster [DBG] osdmap e512: 8 total, 8 up, 7 in 2024-02-23T00:50:41.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:41.305423+0000 mon.a (mon.0) 2836 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:50:41.583 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:50:41.601 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:41.601 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:50:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:40.476543+0000 mon.c (mon.2) 318 : audit [DBG] from='client.? 172.21.15.47:0/2477356923' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:50:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:41.226051+0000 mon.a (mon.0) 2834 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-23T00:50:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:41.226120+0000 mon.a (mon.0) 2835 : cluster [DBG] osdmap e512: 8 total, 8 up, 7 in 2024-02-23T00:50:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:41.305423+0000 mon.a (mon.0) 2836 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:50:41.703 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:50:41.735 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:41.756 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:50:41.759 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:41.759 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:50:41.861 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:50:41.891 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:41.910 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:50:41.912 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:41.913 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:50:42.014 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:50:42.042 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:42.061 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:50:42.064 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:42.064 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:50:42.165 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:50:42.194 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:42.268 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:50:42.271 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:42.272 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:50:42.373 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:50:42.402 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:42.475 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:50:42.477 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:42.478 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:50:42.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:41.331264+0000 mgr.y (mgr.14517) 477 : cluster [DBG] pgmap v772: 208 pgs: 32 creating+peering, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:50:42.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:42.227762+0000 mon.a (mon.0) 2837 : cluster [DBG] osdmap e513: 8 total, 8 up, 7 in 2024-02-23T00:50:42.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:41.331264+0000 mgr.y (mgr.14517) 477 : cluster [DBG] pgmap v772: 208 pgs: 32 creating+peering, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:50:42.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:42.227762+0000 mon.a (mon.0) 2837 : cluster [DBG] osdmap e513: 8 total, 8 up, 7 in 2024-02-23T00:50:42.579 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:50:42.606 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:50:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:50:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:50:42.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:41.331264+0000 mgr.y (mgr.14517) 477 : cluster [DBG] pgmap v772: 208 pgs: 32 creating+peering, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:50:42.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:42.227762+0000 mon.a (mon.0) 2837 : cluster [DBG] osdmap e513: 8 total, 8 up, 7 in 2024-02-23T00:50:42.608 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:42.627 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:50:42.629 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:42.629 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:50:42.730 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:50:42.757 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:42.776 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:50:42.779 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:42.779 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:50:42.880 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:50:42.909 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:42.943 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:50:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:50:44.242 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:42.721296+0000 mgr.y (mgr.14517) 478 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:50:44.242 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:43.229251+0000 mon.a (mon.0) 2838 : cluster [DBG] osdmap e514: 8 total, 8 up, 7 in 2024-02-23T00:50:44.242 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:43.262337+0000 mon.a (mon.0) 2839 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:50:44.242 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:43.262707+0000 mon.a (mon.0) 2840 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-81"}]: dispatch 2024-02-23T00:50:44.242 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:43.331593+0000 mgr.y (mgr.14517) 479 : cluster [DBG] pgmap v775: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-02-23T00:50:44.338 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:50:44.341 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:44.341 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:50:44.443 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:50:44.475 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:44.496 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:50:44.497 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:42.721296+0000 mgr.y (mgr.14517) 478 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:50:44.497 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:43.229251+0000 mon.a (mon.0) 2838 : cluster [DBG] osdmap e514: 8 total, 8 up, 7 in 2024-02-23T00:50:44.498 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:43.262337+0000 mon.a (mon.0) 2839 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:50:44.498 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:43.262707+0000 mon.a (mon.0) 2840 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-81"}]: dispatch 2024-02-23T00:50:44.498 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:43.331593+0000 mgr.y (mgr.14517) 479 : cluster [DBG] pgmap v775: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-02-23T00:50:44.498 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:44.499 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:50:44.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.529+0000 7f30860da700 1 -- 172.21.15.47:0/2481114301 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f30800f9610 msgr2=0x7f3080101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:44.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.529+0000 7f30860da700 1 --2- 172.21.15.47:0/2481114301 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f30800f9610 0x7f3080101d20 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 rx=0x7f3074004870 tx=0x7f3074009cb0).stop 2024-02-23T00:50:44.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.530+0000 7f30860da700 1 -- 172.21.15.47:0/2481114301 shutdown_connections 2024-02-23T00:50:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.530+0000 7f30860da700 1 --2- 172.21.15.47:0/2481114301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3080102210 0x7f3080104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.530+0000 7f30860da700 1 --2- 172.21.15.47:0/2481114301 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30800f8cb0 0x7f30800f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.530+0000 7f30860da700 1 --2- 172.21.15.47:0/2481114301 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f30800f9610 0x7f3080101d20 unknown :-1 s=CLOSED pgs=2715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.530+0000 7f30860da700 1 -- 172.21.15.47:0/2481114301 >> 172.21.15.47:0/2481114301 conn(0x7f30800f2820 msgr2=0x7f30800f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.530+0000 7f30860da700 1 -- 172.21.15.47:0/2481114301 shutdown_connections 2024-02-23T00:50:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.530+0000 7f30860da700 1 -- 172.21.15.47:0/2481114301 wait complete. 2024-02-23T00:50:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.530+0000 7f30860da700 1 librados: starting msgr at 2024-02-23T00:50:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.530+0000 7f30860da700 1 librados: starting objecter 2024-02-23T00:50:44.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.530+0000 7f30860da700 1 Processor -- start 2024-02-23T00:50:44.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.531+0000 7f30860da700 1 -- start start 2024-02-23T00:50:44.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.531+0000 7f30860da700 1 librados: setting wanted keys 2024-02-23T00:50:44.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.531+0000 7f30860da700 1 librados: calling monclient init 2024-02-23T00:50:44.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.531+0000 7f30860da700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30800f8cb0 0x7f30800f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:44.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.531+0000 7f30860da700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f30800f9610 0x7f30800f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:44.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.531+0000 7f30860da700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3080102210 0x7f30800f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:44.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.531+0000 7f30860da700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3080116590 con 0x7f30800f8cb0 2024-02-23T00:50:44.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.531+0000 7f30860da700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3080116700 con 0x7f3080102210 2024-02-23T00:50:44.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.531+0000 7f30860da700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30801168c0 con 0x7f30800f9610 2024-02-23T00:50:44.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.531+0000 7f307effd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f30800f9610 0x7f30800f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:44.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.531+0000 7f307f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30800f8cb0 0x7f30800f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:44.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.531+0000 7f307effd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f30800f9610 0x7f30800f7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:58206/0 (socket says 172.21.15.47:58206) 2024-02-23T00:50:44.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.531+0000 7f307effd700 1 -- 172.21.15.47:0/1196851123 learned_addr learned my addr 172.21.15.47:0/1196851123 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:50:44.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.531+0000 7f307ffff700 1 --2- 172.21.15.47:0/1196851123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3080102210 0x7f30800f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:44.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.531+0000 7f307effd700 1 -- 172.21.15.47:0/1196851123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3080102210 msgr2=0x7f30800f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:44.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.531+0000 7f307effd700 1 --2- 172.21.15.47:0/1196851123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3080102210 0x7f30800f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.531+0000 7f307effd700 1 -- 172.21.15.47:0/1196851123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30800f8cb0 msgr2=0x7f30800f6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.531+0000 7f307effd700 1 --2- 172.21.15.47:0/1196851123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30800f8cb0 0x7f30800f6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.531+0000 7f307effd700 1 -- 172.21.15.47:0/1196851123 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3074005040 con 0x7f30800f9610 2024-02-23T00:50:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.532+0000 7f307effd700 1 --2- 172.21.15.47:0/1196851123 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f30800f9610 0x7f30800f7150 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 rx=0x7f30740043d0 tx=0x7f3074009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.532+0000 7f307cff9700 10 client.?.objecter ms_handle_connect 0x7f30800f9610 2024-02-23T00:50:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.532+0000 7f307cff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:50:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.532+0000 7f307cff9700 1 -- 172.21.15.47:0/1196851123 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307400e030 con 0x7f30800f9610 2024-02-23T00:50:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.532+0000 7f307cff9700 1 -- 172.21.15.47:0/1196851123 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f3074013e70 con 0x7f30800f9610 2024-02-23T00:50:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.532+0000 7f30860da700 1 -- 172.21.15.47:0/1196851123 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3080116b40 con 0x7f30800f9610 2024-02-23T00:50:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.532+0000 7f307cff9700 1 -- 172.21.15.47:0/1196851123 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074018070 con 0x7f30800f9610 2024-02-23T00:50:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.532+0000 7f30860da700 10 client.41579.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:50:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.532+0000 7f30860da700 1 -- 172.21.15.47:0/1196851123 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3080116f40 con 0x7f30800f9610 2024-02-23T00:50:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.532+0000 7f30860da700 1 librados: init done 2024-02-23T00:50:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.533+0000 7f30860da700 1 -- 172.21.15.47:0/1196851123 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f308005fa60 con 0x7f30800f9610 2024-02-23T00:50:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.533+0000 7f307cff9700 1 -- 172.21.15.47:0/1196851123 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f3074009240 con 0x7f30800f9610 2024-02-23T00:50:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.534+0000 7f307cff9700 1 --2- 172.21.15.47:0/1196851123 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3068062cc0 0x7f3068065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.534+0000 7f307f7fe700 1 --2- 172.21.15.47:0/1196851123 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3068062cc0 0x7f3068065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.534+0000 7f307cff9700 1 -- 172.21.15.47:0/1196851123 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(515..515 src has 48..515) v4 ==== 8835+0+0 (secure 0 0 0) 0x7f307407e720 con 0x7f30800f9610 2024-02-23T00:50:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.534+0000 7f307cff9700 10 client.41579.objecter ms_dispatch 0x7f3080001550 osd_map(515..515 src has 48..515) v4 2024-02-23T00:50:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.534+0000 7f307cff9700 3 client.41579.objecter handle_osd_map got epochs [515,515] > 0 2024-02-23T00:50:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.534+0000 7f307cff9700 3 client.41579.objecter handle_osd_map decoding full epoch 515 2024-02-23T00:50:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.534+0000 7f307cff9700 20 client.41579.objecter dump_active .. 0 homeless 2024-02-23T00:50:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.535+0000 7f307cff9700 10 client.41579.objecter ms_handle_connect 0x7f3068062cc0 2024-02-23T00:50:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.535+0000 7f307f7fe700 1 --2- 172.21.15.47:0/1196851123 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3068062cc0 0x7f3068065180 secure :-1 s=READY pgs=3105 cs=0 l=1 rev1=1 rx=0x7f30700048b0 tx=0x7f307000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:44.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.538+0000 7f307cff9700 1 -- 172.21.15.47:0/1196851123 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f307404def0 con 0x7f30800f9610 2024-02-23T00:50:44.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:42.721296+0000 mgr.y (mgr.14517) 478 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:50:44.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:43.229251+0000 mon.a (mon.0) 2838 : cluster [DBG] osdmap e514: 8 total, 8 up, 7 in 2024-02-23T00:50:44.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:43.262337+0000 mon.a (mon.0) 2839 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:50:44.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:43.262707+0000 mon.a (mon.0) 2840 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-81"}]: dispatch 2024-02-23T00:50:44.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:43.331593+0000 mgr.y (mgr.14517) 479 : cluster [DBG] pgmap v775: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-02-23T00:50:44.600 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:50:44.630 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:44.650 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:50:44.652 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:44.653 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:50:44.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:44.723+0000 7f30860da700 1 -- 172.21.15.47:0/1196851123 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7f30801070a0 con 0x7f30800f9610 2024-02-23T00:50:44.754 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:50:44.783 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:44.931 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:50:44.934 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:44.936 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:50:45.036 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:50:45.065 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:45.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.238+0000 7f307cff9700 1 -- 172.21.15.47:0/1196851123 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v516) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3074051510 con 0x7f30800f9610 2024-02-23T00:50:45.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.242+0000 7f30860da700 10 librados: watch_flush enter 2024-02-23T00:50:45.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.242+0000 7f30860da700 10 librados: watch_flush exit 2024-02-23T00:50:45.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.242+0000 7f30860da700 20 client.41579.objecter shutdown clearing up homeless session... 2024-02-23T00:50:45.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.242+0000 7f30860da700 10 client.41579.objecter successfully canceled tick 2024-02-23T00:50:45.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.243+0000 7f30860da700 1 -- 172.21.15.47:0/1196851123 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3068062cc0 msgr2=0x7f3068065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:45.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.243+0000 7f30860da700 1 --2- 172.21.15.47:0/1196851123 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3068062cc0 0x7f3068065180 secure :-1 s=READY pgs=3105 cs=0 l=1 rev1=1 rx=0x7f30700048b0 tx=0x7f307000b040).stop 2024-02-23T00:50:45.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.243+0000 7f30860da700 1 -- 172.21.15.47:0/1196851123 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f30800f9610 msgr2=0x7f30800f7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:45.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.243+0000 7f30860da700 1 --2- 172.21.15.47:0/1196851123 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f30800f9610 0x7f30800f7150 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 rx=0x7f30740043d0 tx=0x7f3074009cb0).stop 2024-02-23T00:50:45.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.243+0000 7f30860da700 1 -- 172.21.15.47:0/1196851123 shutdown_connections 2024-02-23T00:50:45.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.243+0000 7f30860da700 1 --2- 172.21.15.47:0/1196851123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3080102210 0x7f30800f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:45.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.244+0000 7f30860da700 1 --2- 172.21.15.47:0/1196851123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30800f8cb0 0x7f30800f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:45.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.244+0000 7f30860da700 1 --2- 172.21.15.47:0/1196851123 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3068062cc0 0x7f3068065180 unknown :-1 s=CLOSED pgs=3105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:45.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.244+0000 7f30860da700 1 --2- 172.21.15.47:0/1196851123 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f30800f9610 0x7f30800f7150 unknown :-1 s=CLOSED pgs=2716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:45.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.244+0000 7f30860da700 1 -- 172.21.15.47:0/1196851123 >> 172.21.15.47:0/1196851123 conn(0x7f30800f2820 msgr2=0x7f30800f3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:45.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.244+0000 7f30860da700 1 -- 172.21.15.47:0/1196851123 shutdown_connections 2024-02-23T00:50:45.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.244+0000 7f30860da700 1 -- 172.21.15.47:0/1196851123 wait complete. 2024-02-23T00:50:45.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.245+0000 7f30860da700 1 librados: shutdown 2024-02-23T00:50:45.249 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is set 2024-02-23T00:50:45.463 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:44.234822+0000 mon.a (mon.0) 2841 : cluster [DBG] osdmap e515: 8 total, 8 up, 7 in 2024-02-23T00:50:45.464 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:44.725257+0000 mon.c (mon.2) 319 : audit [INF] from='client.? 172.21.15.47:0/1196851123' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:50:45.464 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:44.725616+0000 mon.a (mon.0) 2842 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:50:45.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:44.234822+0000 mon.a (mon.0) 2841 : cluster [DBG] osdmap e515: 8 total, 8 up, 7 in 2024-02-23T00:50:45.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:44.725257+0000 mon.c (mon.2) 319 : audit [INF] from='client.? 172.21.15.47:0/1196851123' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:50:45.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:44.725616+0000 mon.a (mon.0) 2842 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:50:45.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.610+0000 7f33c2919700 1 -- 172.21.15.47:0/2921338915 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f33bc104440 msgr2=0x7f33bc106830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:45.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.610+0000 7f33c2919700 1 --2- 172.21.15.47:0/2921338915 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f33bc104440 0x7f33bc106830 secure :-1 s=READY pgs=2717 cs=0 l=1 rev1=1 rx=0x7f33b4009be0 tx=0x7f33b4004d10).stop 2024-02-23T00:50:45.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.610+0000 7f33c2919700 1 -- 172.21.15.47:0/2921338915 shutdown_connections 2024-02-23T00:50:45.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.610+0000 7f33c2919700 1 --2- 172.21.15.47:0/2921338915 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f33bc104440 0x7f33bc106830 unknown :-1 s=CLOSED pgs=2717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:45.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.610+0000 7f33c2919700 1 --2- 172.21.15.47:0/2921338915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33bc06b170 0x7f33bc06b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:45.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.610+0000 7f33c2919700 1 --2- 172.21.15.47:0/2921338915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33bc06ba40 0x7f33bc103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:45.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.610+0000 7f33c2919700 1 -- 172.21.15.47:0/2921338915 >> 172.21.15.47:0/2921338915 conn(0x7f33bc0f3800 msgr2=0x7f33bc0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:45.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.611+0000 7f33c2919700 1 -- 172.21.15.47:0/2921338915 shutdown_connections 2024-02-23T00:50:45.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.611+0000 7f33c2919700 1 -- 172.21.15.47:0/2921338915 wait complete. 2024-02-23T00:50:45.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.611+0000 7f33c2919700 1 librados: starting msgr at 2024-02-23T00:50:45.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.611+0000 7f33c2919700 1 librados: starting objecter 2024-02-23T00:50:45.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.611+0000 7f33c2919700 1 Processor -- start 2024-02-23T00:50:45.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.611+0000 7f33c2919700 1 -- start start 2024-02-23T00:50:45.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.611+0000 7f33c2919700 1 librados: setting wanted keys 2024-02-23T00:50:45.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.611+0000 7f33c2919700 1 librados: calling monclient init 2024-02-23T00:50:45.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.612+0000 7f33c2919700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33bc06b170 0x7f33bc110030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:45.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.612+0000 7f33c2919700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33bc06ba40 0x7f33bc110520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:45.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.612+0000 7f33c2919700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f33bc104440 0x7f33bc10a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:45.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.612+0000 7f33c2919700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33bc10a710 con 0x7f33bc06ba40 2024-02-23T00:50:45.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.612+0000 7f33c2919700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33bc10a880 con 0x7f33bc06b170 2024-02-23T00:50:45.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.612+0000 7f33c2919700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33bc10aa40 con 0x7f33bc104440 2024-02-23T00:50:45.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.612+0000 7f33bb7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33bc06ba40 0x7f33bc110520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:45.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.612+0000 7f33bb7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33bc06ba40 0x7f33bc110520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42766/0 (socket says 172.21.15.47:42766) 2024-02-23T00:50:45.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.612+0000 7f33bb7fe700 1 -- 172.21.15.47:0/4071644021 learned_addr learned my addr 172.21.15.47:0/4071644021 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:50:45.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.612+0000 7f33c0eb6700 1 --2- 172.21.15.47:0/4071644021 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f33bc104440 0x7f33bc10a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:45.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.612+0000 7f33bbfff700 1 --2- 172.21.15.47:0/4071644021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33bc06b170 0x7f33bc110030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:45.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.612+0000 7f33bb7fe700 1 -- 172.21.15.47:0/4071644021 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f33bc104440 msgr2=0x7f33bc10a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:45.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.612+0000 7f33bb7fe700 1 --2- 172.21.15.47:0/4071644021 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f33bc104440 0x7f33bc10a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:45.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.612+0000 7f33bb7fe700 1 -- 172.21.15.47:0/4071644021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33bc06b170 msgr2=0x7f33bc110030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:45.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.612+0000 7f33bb7fe700 1 --2- 172.21.15.47:0/4071644021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33bc06b170 0x7f33bc110030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:45.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.612+0000 7f33bb7fe700 1 -- 172.21.15.47:0/4071644021 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b0005040 con 0x7f33bc06ba40 2024-02-23T00:50:45.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.612+0000 7f33bb7fe700 1 --2- 172.21.15.47:0/4071644021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33bc06ba40 0x7f33bc110520 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 rx=0x7f33b0004920 tx=0x7f33b0009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:45.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.612+0000 7f33b97fa700 10 client.?.objecter ms_handle_connect 0x7f33bc06ba40 2024-02-23T00:50:45.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.612+0000 7f33b97fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:50:45.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.612+0000 7f33b97fa700 1 -- 172.21.15.47:0/4071644021 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b0014410 con 0x7f33bc06ba40 2024-02-23T00:50:45.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.613+0000 7f33c2919700 1 -- 172.21.15.47:0/4071644021 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b40097c0 con 0x7f33bc06ba40 2024-02-23T00:50:45.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.613+0000 7f33b97fa700 1 -- 172.21.15.47:0/4071644021 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f33b000f040 con 0x7f33bc06ba40 2024-02-23T00:50:45.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.613+0000 7f33b97fa700 1 -- 172.21.15.47:0/4071644021 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b0013870 con 0x7f33bc06ba40 2024-02-23T00:50:45.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.613+0000 7f33c2919700 10 client.31260.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:50:45.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.613+0000 7f33c2919700 1 -- 172.21.15.47:0/4071644021 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33bc116e60 con 0x7f33bc06ba40 2024-02-23T00:50:45.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.613+0000 7f33c2919700 1 librados: init done 2024-02-23T00:50:45.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.613+0000 7f33c2919700 1 -- 172.21.15.47:0/4071644021 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33bc05ee80 con 0x7f33bc06ba40 2024-02-23T00:50:45.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.614+0000 7f33b97fa700 1 -- 172.21.15.47:0/4071644021 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f33b0013ca0 con 0x7f33bc06ba40 2024-02-23T00:50:45.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.614+0000 7f33b97fa700 1 --2- 172.21.15.47:0/4071644021 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f33a4062cc0 0x7f33a4065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:45.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.614+0000 7f33b97fa700 1 -- 172.21.15.47:0/4071644021 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(516..516 src has 463..516) v4 ==== 9207+0+0 (secure 0 0 0) 0x7f33b007d210 con 0x7f33bc06ba40 2024-02-23T00:50:45.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.614+0000 7f33b97fa700 10 client.31260.objecter ms_dispatch 0x7f33bc001550 osd_map(516..516 src has 463..516) v4 2024-02-23T00:50:45.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.614+0000 7f33b97fa700 3 client.31260.objecter handle_osd_map got epochs [516,516] > 0 2024-02-23T00:50:45.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.614+0000 7f33b97fa700 3 client.31260.objecter handle_osd_map decoding full epoch 516 2024-02-23T00:50:45.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.614+0000 7f33bbfff700 1 --2- 172.21.15.47:0/4071644021 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f33a4062cc0 0x7f33a4065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:45.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.615+0000 7f33b97fa700 20 client.31260.objecter dump_active .. 0 homeless 2024-02-23T00:50:45.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.615+0000 7f33b97fa700 10 client.31260.objecter ms_handle_connect 0x7f33a4062cc0 2024-02-23T00:50:45.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.615+0000 7f33bbfff700 1 --2- 172.21.15.47:0/4071644021 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f33a4062cc0 0x7f33a4065180 secure :-1 s=READY pgs=3106 cs=0 l=1 rev1=1 rx=0x7f33ac004de0 tx=0x7f33ac00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:45.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.623+0000 7f33b97fa700 1 -- 172.21.15.47:0/4071644021 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f33b0049f60 con 0x7f33bc06ba40 2024-02-23T00:50:45.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:44.234822+0000 mon.a (mon.0) 2841 : cluster [DBG] osdmap e515: 8 total, 8 up, 7 in 2024-02-23T00:50:45.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:44.725257+0000 mon.c (mon.2) 319 : audit [INF] from='client.? 172.21.15.47:0/1196851123' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:50:45.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:44.725616+0000 mon.a (mon.0) 2842 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:50:45.743 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:50:45.743 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-02-23T00:50:45.743 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-02-23T00:50:45.744 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-02-23T00:50:45.744 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:50:45.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:45.821+0000 7f33c2919700 1 -- 172.21.15.47:0/4071644021 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd reweight", "id": 5, "weight": 1.0} v 0) v1 -- 0x7f33bc10b6b0 con 0x7f33bc06ba40 2024-02-23T00:50:46.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:46.246+0000 7f33b97fa700 1 -- 172.21.15.47:0/4071644021 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 5, "weight": 1.0}]=0 reweighted osd.5 to 1 (10000) v517) v1 ==== 113+0+0 (secure 0 0 0) 0x7f33b004c830 con 0x7f33bc06ba40 2024-02-23T00:50:46.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:46.249+0000 7f33c2919700 10 librados: watch_flush enter 2024-02-23T00:50:46.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:46.250+0000 7f33c2919700 10 librados: watch_flush exit 2024-02-23T00:50:46.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:46.250+0000 7f33c2919700 20 client.31260.objecter shutdown clearing up homeless session... 2024-02-23T00:50:46.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:46.250+0000 7f33c2919700 10 client.31260.objecter successfully canceled tick 2024-02-23T00:50:46.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:46.250+0000 7f33c2919700 1 -- 172.21.15.47:0/4071644021 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f33a4062cc0 msgr2=0x7f33a4065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:46.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:46.250+0000 7f33c2919700 1 --2- 172.21.15.47:0/4071644021 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f33a4062cc0 0x7f33a4065180 secure :-1 s=READY pgs=3106 cs=0 l=1 rev1=1 rx=0x7f33ac004de0 tx=0x7f33ac00b040).stop 2024-02-23T00:50:46.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:46.250+0000 7f33c2919700 1 -- 172.21.15.47:0/4071644021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33bc06ba40 msgr2=0x7f33bc110520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:46.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:46.250+0000 7f33c2919700 1 --2- 172.21.15.47:0/4071644021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33bc06ba40 0x7f33bc110520 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 rx=0x7f33b0004920 tx=0x7f33b0009cb0).stop 2024-02-23T00:50:46.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:46.250+0000 7f33c2919700 1 -- 172.21.15.47:0/4071644021 shutdown_connections 2024-02-23T00:50:46.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:46.250+0000 7f33c2919700 1 --2- 172.21.15.47:0/4071644021 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f33a4062cc0 0x7f33a4065180 unknown :-1 s=CLOSED pgs=3106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:46.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:46.250+0000 7f33c2919700 1 --2- 172.21.15.47:0/4071644021 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f33bc104440 0x7f33bc10a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:46.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:46.250+0000 7f33c2919700 1 --2- 172.21.15.47:0/4071644021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33bc06b170 0x7f33bc110030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:46.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:46.250+0000 7f33c2919700 1 --2- 172.21.15.47:0/4071644021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33bc06ba40 0x7f33bc110520 unknown :-1 s=CLOSED pgs=2563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:46.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:46.250+0000 7f33c2919700 1 -- 172.21.15.47:0/4071644021 >> 172.21.15.47:0/4071644021 conn(0x7f33bc0f3800 msgr2=0x7f33bc0f4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:46.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:46.251+0000 7f33c2919700 1 -- 172.21.15.47:0/4071644021 shutdown_connections 2024-02-23T00:50:46.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:46.251+0000 7f33c2919700 1 -- 172.21.15.47:0/4071644021 wait complete. 2024-02-23T00:50:46.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:46.252+0000 7f33c2919700 1 librados: shutdown 2024-02-23T00:50:46.257 INFO:teuthology.orchestra.run.smithi047.stderr:reweighted osd.5 to 1 (10000) 2024-02-23T00:50:46.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:44.969025+0000 osd.4 (osd.4) 225 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:50:46.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:44.969195+0000 osd.4 (osd.4) 226 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:50:46.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:45.231946+0000 mon.a (mon.0) 2843 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:50:46.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:45.238915+0000 mon.a (mon.0) 2844 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:50:46.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:45.238998+0000 mon.a (mon.0) 2845 : cluster [DBG] osdmap e516: 8 total, 8 up, 7 in 2024-02-23T00:50:46.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:45.242336+0000 mon.a (mon.0) 2846 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:50:46.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:45.332043+0000 mgr.y (mgr.14517) 480 : cluster [DBG] pgmap v778: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:50:46.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:45.822755+0000 mon.a (mon.0) 2847 : audit [INF] from='client.? 172.21.15.47:0/4071644021' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-23T00:50:46.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:44.969025+0000 osd.4 (osd.4) 225 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:50:46.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:44.969195+0000 osd.4 (osd.4) 226 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:50:46.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:45.231946+0000 mon.a (mon.0) 2843 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:50:46.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:45.238915+0000 mon.a (mon.0) 2844 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:50:46.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:45.238998+0000 mon.a (mon.0) 2845 : cluster [DBG] osdmap e516: 8 total, 8 up, 7 in 2024-02-23T00:50:46.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:45.242336+0000 mon.a (mon.0) 2846 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:50:46.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:45.332043+0000 mgr.y (mgr.14517) 480 : cluster [DBG] pgmap v778: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:50:46.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:45.822755+0000 mon.a (mon.0) 2847 : audit [INF] from='client.? 172.21.15.47:0/4071644021' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-23T00:50:46.363 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:50:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:44.969025+0000 osd.4 (osd.4) 225 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:50:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:44.969195+0000 osd.4 (osd.4) 226 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:50:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:45.231946+0000 mon.a (mon.0) 2843 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:50:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:45.238915+0000 mon.a (mon.0) 2844 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:50:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:45.238998+0000 mon.a (mon.0) 2845 : cluster [DBG] osdmap e516: 8 total, 8 up, 7 in 2024-02-23T00:50:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:45.242336+0000 mon.a (mon.0) 2846 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:50:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:45.332043+0000 mgr.y (mgr.14517) 480 : cluster [DBG] pgmap v778: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:50:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:45.822755+0000 mon.a (mon.0) 2847 : audit [INF] from='client.? 172.21.15.47:0/4071644021' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-23T00:50:46.914 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:50:46.916 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:46.917 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:50:47.018 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:50:47.050 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:47.071 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:50:47.074 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:47.074 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:50:47.175 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:50:47.205 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:47.226 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:50:47.228 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:47.229 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:50:47.330 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:50:47.361 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:47.437 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:50:47.439 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:47.440 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:50:47.441 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight 6 1 2024-02-23T00:50:47.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:46.243781+0000 mon.a (mon.0) 2848 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:50:47.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:46.243932+0000 mon.a (mon.0) 2849 : audit [INF] from='client.? 172.21.15.47:0/4071644021' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-23T00:50:47.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:46.243997+0000 mon.a (mon.0) 2850 : cluster [DBG] osdmap e517: 8 total, 8 up, 7 in 2024-02-23T00:50:47.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:46.248332+0000 mon.a (mon.0) 2851 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:50:47.506 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:46.243781+0000 mon.a (mon.0) 2848 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:50:47.507 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:46.243932+0000 mon.a (mon.0) 2849 : audit [INF] from='client.? 172.21.15.47:0/4071644021' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-23T00:50:47.507 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:46.243997+0000 mon.a (mon.0) 2850 : cluster [DBG] osdmap e517: 8 total, 8 up, 7 in 2024-02-23T00:50:47.507 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:46.248332+0000 mon.a (mon.0) 2851 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:50:47.507 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:46.243781+0000 mon.a (mon.0) 2848 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:50:47.507 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:46.243932+0000 mon.a (mon.0) 2849 : audit [INF] from='client.? 172.21.15.47:0/4071644021' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-23T00:50:47.507 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:46.243997+0000 mon.a (mon.0) 2850 : cluster [DBG] osdmap e517: 8 total, 8 up, 7 in 2024-02-23T00:50:47.507 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:46.248332+0000 mon.a (mon.0) 2851 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:50:47.542 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:50:47.570 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:47.820 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:50:47.828 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:47.828 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:50:47.836 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.2 dump_historic_ops 2024-02-23T00:50:47.851 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:50:47.930 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:50:47.957 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:48.134 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:50:48.156 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:50:48.161 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:48.161 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:50:48.262 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:50:48.298 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:48.319 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:50:48.321 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:48.322 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:50:48.381 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:47.250358+0000 mon.a (mon.0) 2852 : cluster [DBG] osdmap e518: 8 total, 8 up, 7 in 2024-02-23T00:50:48.381 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:47.255991+0000 mon.a (mon.0) 2853 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-23T00:50:48.381 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:47.332607+0000 mgr.y (mgr.14517) 481 : cluster [DBG] pgmap v781: 240 pgs: 32 unknown, 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-02-23T00:50:48.382 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:47.250358+0000 mon.a (mon.0) 2852 : cluster [DBG] osdmap e518: 8 total, 8 up, 7 in 2024-02-23T00:50:48.382 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:47.255991+0000 mon.a (mon.0) 2853 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-23T00:50:48.382 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:47.332607+0000 mgr.y (mgr.14517) 481 : cluster [DBG] pgmap v781: 240 pgs: 32 unknown, 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-02-23T00:50:48.422 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:50:48.451 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:48.471 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:50:48.474 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:48.474 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:50:48.545 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset noscrub 2024-02-23T00:50:48.574 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:50:48.600 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:47.250358+0000 mon.a (mon.0) 2852 : cluster [DBG] osdmap e518: 8 total, 8 up, 7 in 2024-02-23T00:50:48.600 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:47.255991+0000 mon.a (mon.0) 2853 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-23T00:50:48.600 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:47.332607+0000 mgr.y (mgr.14517) 481 : cluster [DBG] pgmap v781: 240 pgs: 32 unknown, 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-02-23T00:50:48.602 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:48.621 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:50:48.623 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:48.624 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:50:48.725 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:50:48.754 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:48.877 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:50:48.880 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:48.881 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:50:48.981 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:50:49.010 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:49.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:48.255140+0000 mon.a (mon.0) 2854 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-23T00:50:49.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:48.255293+0000 mon.a (mon.0) 2855 : cluster [DBG] osdmap e519: 8 total, 8 up, 7 in 2024-02-23T00:50:49.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:48.258765+0000 mon.a (mon.0) 2856 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-83","var": "dedup_tier","val": "test-rados-api-smithi047-82465-85-test-flush"}]: dispatch 2024-02-23T00:50:49.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:48.255140+0000 mon.a (mon.0) 2854 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-23T00:50:49.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:48.255293+0000 mon.a (mon.0) 2855 : cluster [DBG] osdmap e519: 8 total, 8 up, 7 in 2024-02-23T00:50:49.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:48.258765+0000 mon.a (mon.0) 2856 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-83","var": "dedup_tier","val": "test-rados-api-smithi047-82465-85-test-flush"}]: dispatch 2024-02-23T00:50:49.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:48.255140+0000 mon.a (mon.0) 2854 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-23T00:50:49.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:48.255293+0000 mon.a (mon.0) 2855 : cluster [DBG] osdmap e519: 8 total, 8 up, 7 in 2024-02-23T00:50:49.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:48.258765+0000 mon.a (mon.0) 2856 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-83","var": "dedup_tier","val": "test-rados-api-smithi047-82465-85-test-flush"}]: dispatch 2024-02-23T00:50:50.063 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:50:50.066 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:50.066 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:50:50.167 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:50:50.200 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:50.274 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:50:50.277 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:50.277 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:50:50.378 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:50:50.408 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:50.427 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:50:50.430 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:50.430 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:50:50.528 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:49.255534+0000 mon.a (mon.0) 2857 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-83","var": "dedup_tier","val": "test-rados-api-smithi047-82465-85-test-flush"}]': finished 2024-02-23T00:50:50.528 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:49.255606+0000 mon.a (mon.0) 2858 : cluster [DBG] osdmap e520: 8 total, 8 up, 7 in 2024-02-23T00:50:50.529 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:49.258789+0000 mon.a (mon.0) 2859 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-23T00:50:50.529 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:49.333125+0000 mgr.y (mgr.14517) 482 : cluster [DBG] pgmap v784: 240 pgs: 32 unknown, 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-02-23T00:50:50.529 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:50.257578+0000 mon.a (mon.0) 2860 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-23T00:50:50.529 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:50.257647+0000 mon.a (mon.0) 2861 : cluster [DBG] osdmap e521: 8 total, 8 up, 7 in 2024-02-23T00:50:50.529 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:50.258945+0000 mon.a (mon.0) 2862 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-23T00:50:50.529 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:49.255534+0000 mon.a (mon.0) 2857 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-83","var": "dedup_tier","val": "test-rados-api-smithi047-82465-85-test-flush"}]': finished 2024-02-23T00:50:50.529 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:49.255606+0000 mon.a (mon.0) 2858 : cluster [DBG] osdmap e520: 8 total, 8 up, 7 in 2024-02-23T00:50:50.529 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:49.258789+0000 mon.a (mon.0) 2859 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-23T00:50:50.529 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:49.333125+0000 mgr.y (mgr.14517) 482 : cluster [DBG] pgmap v784: 240 pgs: 32 unknown, 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-02-23T00:50:50.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:50.257578+0000 mon.a (mon.0) 2860 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-23T00:50:50.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:50.257647+0000 mon.a (mon.0) 2861 : cluster [DBG] osdmap e521: 8 total, 8 up, 7 in 2024-02-23T00:50:50.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:50.258945+0000 mon.a (mon.0) 2862 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-23T00:50:50.530 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:50:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:50:50] "GET /metrics HTTP/1.1" 200 189541 "" "Prometheus/2.33.4" 2024-02-23T00:50:50.531 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:50:50.563 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:50.638 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:50:50.638 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:50:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:49.255534+0000 mon.a (mon.0) 2857 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-83","var": "dedup_tier","val": "test-rados-api-smithi047-82465-85-test-flush"}]': finished 2024-02-23T00:50:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:49.255606+0000 mon.a (mon.0) 2858 : cluster [DBG] osdmap e520: 8 total, 8 up, 7 in 2024-02-23T00:50:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:49.258789+0000 mon.a (mon.0) 2859 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-23T00:50:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:49.333125+0000 mgr.y (mgr.14517) 482 : cluster [DBG] pgmap v784: 240 pgs: 32 unknown, 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-02-23T00:50:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:50.257578+0000 mon.a (mon.0) 2860 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-23T00:50:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:50.257647+0000 mon.a (mon.0) 2861 : cluster [DBG] osdmap e521: 8 total, 8 up, 7 in 2024-02-23T00:50:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:50.258945+0000 mon.a (mon.0) 2862 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-23T00:50:50.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.747+0000 7febd8137700 1 -- 172.21.15.47:0/628665239 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febd00f6aa0 msgr2=0x7febd00f6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:50.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.747+0000 7febd8137700 1 --2- 172.21.15.47:0/628665239 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febd00f6aa0 0x7febd00f6e80 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 rx=0x7febcc0047b0 tx=0x7febcc009cb0).stop 2024-02-23T00:50:50.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.747+0000 7febd8137700 1 -- 172.21.15.47:0/628665239 shutdown_connections 2024-02-23T00:50:50.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.747+0000 7febd8137700 1 --2- 172.21.15.47:0/628665239 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7febd00fbfe0 0x7febd00fc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:50.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.747+0000 7febd8137700 1 --2- 172.21.15.47:0/628665239 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febd00f6aa0 0x7febd00f6e80 unknown :-1 s=CLOSED pgs=2564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:50.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.747+0000 7febd8137700 1 --2- 172.21.15.47:0/628665239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febd00f7400 0x7febd00fbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:50.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.747+0000 7febd8137700 1 -- 172.21.15.47:0/628665239 >> 172.21.15.47:0/628665239 conn(0x7febd00f2820 msgr2=0x7febd00f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:50.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.747+0000 7febd8137700 1 -- 172.21.15.47:0/628665239 shutdown_connections 2024-02-23T00:50:50.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.747+0000 7febd8137700 1 -- 172.21.15.47:0/628665239 wait complete. 2024-02-23T00:50:50.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.748+0000 7febd8137700 1 librados: starting msgr at 2024-02-23T00:50:50.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.748+0000 7febd8137700 1 librados: starting objecter 2024-02-23T00:50:50.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.748+0000 7febd8137700 1 Processor -- start 2024-02-23T00:50:50.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.748+0000 7febd8137700 1 -- start start 2024-02-23T00:50:50.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.748+0000 7febd8137700 1 librados: setting wanted keys 2024-02-23T00:50:50.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.748+0000 7febd8137700 1 librados: calling monclient init 2024-02-23T00:50:50.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.748+0000 7febd8137700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febd00f6aa0 0x7febd0114580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:50.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.748+0000 7febd8137700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febd00f7400 0x7febd0114a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:50.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.748+0000 7febd8137700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7febd00fbfe0 0x7febd010e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:50.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.748+0000 7febd8137700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd010eca0 con 0x7febd00f6aa0 2024-02-23T00:50:50.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.748+0000 7febd8137700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd010ee10 con 0x7febd00f7400 2024-02-23T00:50:50.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.748+0000 7febd8137700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd010efd0 con 0x7febd00fbfe0 2024-02-23T00:50:50.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.748+0000 7febd66d4700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7febd00fbfe0 0x7febd010e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:50.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.748+0000 7febd5ed3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febd00f6aa0 0x7febd0114580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:50.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.748+0000 7febd66d4700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7febd00fbfe0 0x7febd010e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:47514/0 (socket says 172.21.15.47:47514) 2024-02-23T00:50:50.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.748+0000 7febd66d4700 1 -- 172.21.15.47:0/3579779368 learned_addr learned my addr 172.21.15.47:0/3579779368 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:50:50.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.749+0000 7febd56d2700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febd00f7400 0x7febd0114a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:50.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.749+0000 7febd66d4700 1 -- 172.21.15.47:0/3579779368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febd00f7400 msgr2=0x7febd0114a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:50.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.749+0000 7febd66d4700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febd00f7400 0x7febd0114a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:50.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.749+0000 7febd66d4700 1 -- 172.21.15.47:0/3579779368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febd00f6aa0 msgr2=0x7febd0114580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:50.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.749+0000 7febd66d4700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febd00f6aa0 0x7febd0114580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:50.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.749+0000 7febd66d4700 1 -- 172.21.15.47:0/3579779368 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc80097c0 con 0x7febd00fbfe0 2024-02-23T00:50:50.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.749+0000 7febd56d2700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febd00f7400 0x7febd0114a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:50:50.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.749+0000 7febd5ed3700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febd00f6aa0 0x7febd0114580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:50:50.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.749+0000 7febd66d4700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7febd00fbfe0 0x7febd010e770 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 rx=0x7febc8001df0 tx=0x7febc8004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:50.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.749+0000 7febc6ffd700 10 client.?.objecter ms_handle_connect 0x7febd00fbfe0 2024-02-23T00:50:50.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.749+0000 7febc6ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:50:50.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.749+0000 7febc6ffd700 1 -- 172.21.15.47:0/3579779368 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc8009a10 con 0x7febd00fbfe0 2024-02-23T00:50:50.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.749+0000 7febc6ffd700 1 -- 172.21.15.47:0/3579779368 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7febc8009560 con 0x7febd00fbfe0 2024-02-23T00:50:50.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.749+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febcc005040 con 0x7febd00fbfe0 2024-02-23T00:50:50.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.749+0000 7febc6ffd700 1 -- 172.21.15.47:0/3579779368 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc8007ce0 con 0x7febd00fbfe0 2024-02-23T00:50:50.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.749+0000 7febd8137700 10 client.41591.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:50:50.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.749+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd010f5c0 con 0x7febd00fbfe0 2024-02-23T00:50:50.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.749+0000 7febd8137700 1 librados: init done 2024-02-23T00:50:50.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.750+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7febd005fa60 con 0x7febd00fbfe0 2024-02-23T00:50:50.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.750+0000 7febc6ffd700 1 -- 172.21.15.47:0/3579779368 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7febc80090d0 con 0x7febd00fbfe0 2024-02-23T00:50:50.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.751+0000 7febc6ffd700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7febbc062cc0 0x7febbc065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:50.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.751+0000 7febc6ffd700 1 -- 172.21.15.47:0/3579779368 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(521..521 src has 463..521) v4 ==== 9654+0+0 (secure 0 0 0) 0x7febc807e1a0 con 0x7febd00fbfe0 2024-02-23T00:50:50.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.751+0000 7febc6ffd700 10 client.41591.objecter ms_dispatch 0x7febd0001550 osd_map(521..521 src has 463..521) v4 2024-02-23T00:50:50.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.751+0000 7febc6ffd700 3 client.41591.objecter handle_osd_map got epochs [521,521] > 0 2024-02-23T00:50:50.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.751+0000 7febc6ffd700 3 client.41591.objecter handle_osd_map decoding full epoch 521 2024-02-23T00:50:50.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.751+0000 7febd5ed3700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7febbc062cc0 0x7febbc065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:50.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.751+0000 7febc6ffd700 20 client.41591.objecter dump_active .. 0 homeless 2024-02-23T00:50:50.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.751+0000 7febc6ffd700 1 -- 172.21.15.47:0/3579779368 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v521) v1 ==== 54+0+16 (secure 0 0 0) 0x7febc8085070 con 0x7febd00fbfe0 2024-02-23T00:50:50.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.751+0000 7febc6ffd700 10 client.41591.objecter ms_handle_connect 0x7febbc062cc0 2024-02-23T00:50:50.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.751+0000 7febd5ed3700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7febbc062cc0 0x7febbc065180 secure :-1 s=READY pgs=3107 cs=0 l=1 rev1=1 rx=0x7febcc004e20 tx=0x7febcc009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:50.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.751+0000 7febd8137700 10 client.41591.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:50.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.751+0000 7febd8137700 15 client.41591.objecter _session_command_op_assign -1 1 2024-02-23T00:50:50.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.751+0000 7febd8137700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7febd0105de0 0x7febd01061c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:50.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.751+0000 7febd8137700 20 client.41591.objecter _get_session s=0x7febd005ee80 osd=0 3 2024-02-23T00:50:50.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.751+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd005ee80 osd=0 3 2024-02-23T00:50:50.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.751+0000 7febd8137700 20 client.41591.objecter _get_session s=0x7febd005ee80 osd=0 3 2024-02-23T00:50:50.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.751+0000 7febd8137700 15 client.41591.objecter _session_command_op_remove -1 1 2024-02-23T00:50:50.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.751+0000 7febd8137700 20 client.41591.objecter get_session s=0x7febd005ee80 osd=0 3 2024-02-23T00:50:50.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.751+0000 7febd8137700 15 client.41591.objecter _session_command_op_assign 0 1 2024-02-23T00:50:50.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.751+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd005ee80 osd=0 4 2024-02-23T00:50:50.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.751+0000 7febd8137700 10 client.41591.objecter _send_command 1 2024-02-23T00:50:50.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.751+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7febd010d590 con 0x7febd0105de0 2024-02-23T00:50:50.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.752+0000 7febd56d2700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7febd0105de0 0x7febd01061c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:50.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.752+0000 7febd56d2700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7febd0105de0 0x7febd01061c0 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:50.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.752+0000 7febc6ffd700 10 client.41591.objecter ms_handle_connect 0x7febd0105de0 2024-02-23T00:50:50.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.752+0000 7febc6ffd700 1 -- 172.21.15.47:0/3579779368 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7febd010d590 con 0x7febd0105de0 2024-02-23T00:50:50.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.752+0000 7febc6ffd700 10 client.41591.objecter ms_dispatch 0x7febd0001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:50:50.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.752+0000 7febc6ffd700 10 client.41591.objecter _finish_command 1 = system:0 2024-02-23T00:50:50.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.752+0000 7febc6ffd700 20 client.41591.objecter put_session s=0x7febd005ee80 osd=0 4 2024-02-23T00:50:50.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.752+0000 7febc6ffd700 15 client.41591.objecter _session_command_op_remove 0 1 2024-02-23T00:50:50.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.768+0000 7febd8137700 10 client.41591.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:50:50.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.768+0000 7febd8137700 15 client.41591.objecter _session_command_op_assign -1 2 2024-02-23T00:50:50.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.768+0000 7febd8137700 20 client.41591.objecter _get_session s=0x7febd005ee80 osd=0 3 2024-02-23T00:50:50.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.768+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd005ee80 osd=0 3 2024-02-23T00:50:50.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.768+0000 7febd8137700 20 client.41591.objecter _get_session s=0x7febd005ee80 osd=0 3 2024-02-23T00:50:50.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.768+0000 7febd8137700 15 client.41591.objecter _session_command_op_remove -1 2 2024-02-23T00:50:50.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.768+0000 7febd8137700 20 client.41591.objecter get_session s=0x7febd005ee80 osd=0 3 2024-02-23T00:50:50.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.768+0000 7febd8137700 15 client.41591.objecter _session_command_op_assign 0 2 2024-02-23T00:50:50.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.768+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd005ee80 osd=0 4 2024-02-23T00:50:50.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.768+0000 7febd8137700 10 client.41591.objecter _send_command 2 2024-02-23T00:50:50.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.768+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7febd00273c0 con 0x7febd0105de0 2024-02-23T00:50:50.816 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:50:50.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.769+0000 7febc6ffd700 1 -- 172.21.15.47:0/3579779368 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7febd00273c0 con 0x7febd0105de0 2024-02-23T00:50:50.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.769+0000 7febc6ffd700 10 client.41591.objecter ms_dispatch 0x7febd0001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:50:50.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.769+0000 7febc6ffd700 10 client.41591.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:50:50.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.769+0000 7febc6ffd700 20 client.41591.objecter put_session s=0x7febd005ee80 osd=0 4 2024-02-23T00:50:50.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.769+0000 7febc6ffd700 15 client.41591.objecter _session_command_op_remove 0 2 2024-02-23T00:50:50.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.769+0000 7febd8137700 10 client.41591.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:50.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.769+0000 7febd8137700 15 client.41591.objecter _session_command_op_assign -1 3 2024-02-23T00:50:50.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.769+0000 7febd8137700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7febd00598f0 0x7febd0059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:50.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.769+0000 7febd8137700 20 client.41591.objecter _get_session s=0x7febd0028710 osd=1 3 2024-02-23T00:50:50.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.769+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd0028710 osd=1 3 2024-02-23T00:50:50.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.769+0000 7febd8137700 20 client.41591.objecter _get_session s=0x7febd0028710 osd=1 3 2024-02-23T00:50:50.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.769+0000 7febd8137700 15 client.41591.objecter _session_command_op_remove -1 3 2024-02-23T00:50:50.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.769+0000 7febd8137700 20 client.41591.objecter get_session s=0x7febd0028710 osd=1 3 2024-02-23T00:50:50.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.769+0000 7febd8137700 15 client.41591.objecter _session_command_op_assign 1 3 2024-02-23T00:50:50.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.769+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd0028710 osd=1 4 2024-02-23T00:50:50.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.769+0000 7febd8137700 10 client.41591.objecter _send_command 3 2024-02-23T00:50:50.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.769+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7febd0121bc0 con 0x7febd00598f0 2024-02-23T00:50:50.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.770+0000 7febd5ed3700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7febd00598f0 0x7febd0059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:50.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.770+0000 7febd5ed3700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7febd00598f0 0x7febd0059cd0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:50.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.770+0000 7febc6ffd700 10 client.41591.objecter ms_handle_connect 0x7febd00598f0 2024-02-23T00:50:50.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.770+0000 7febc6ffd700 1 -- 172.21.15.47:0/3579779368 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7febd0121bc0 con 0x7febd00598f0 2024-02-23T00:50:50.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.770+0000 7febc6ffd700 10 client.41591.objecter ms_dispatch 0x7febd0001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:50:50.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.770+0000 7febc6ffd700 10 client.41591.objecter _finish_command 3 = system:0 2024-02-23T00:50:50.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.770+0000 7febc6ffd700 20 client.41591.objecter put_session s=0x7febd0028710 osd=1 4 2024-02-23T00:50:50.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.770+0000 7febc6ffd700 15 client.41591.objecter _session_command_op_remove 1 3 2024-02-23T00:50:50.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.785+0000 7febd8137700 10 client.41591.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:50:50.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.786+0000 7febd8137700 15 client.41591.objecter _session_command_op_assign -1 4 2024-02-23T00:50:50.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.786+0000 7febd8137700 20 client.41591.objecter _get_session s=0x7febd0028710 osd=1 3 2024-02-23T00:50:50.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.786+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd0028710 osd=1 3 2024-02-23T00:50:50.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.786+0000 7febd8137700 20 client.41591.objecter _get_session s=0x7febd0028710 osd=1 3 2024-02-23T00:50:50.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.786+0000 7febd8137700 15 client.41591.objecter _session_command_op_remove -1 4 2024-02-23T00:50:50.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.786+0000 7febd8137700 20 client.41591.objecter get_session s=0x7febd0028710 osd=1 3 2024-02-23T00:50:50.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.786+0000 7febd8137700 15 client.41591.objecter _session_command_op_assign 1 4 2024-02-23T00:50:50.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.786+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd0028710 osd=1 4 2024-02-23T00:50:50.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.786+0000 7febd8137700 10 client.41591.objecter _send_command 4 2024-02-23T00:50:50.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.786+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7febd0059360 con 0x7febd00598f0 2024-02-23T00:50:50.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.787+0000 7febc6ffd700 1 -- 172.21.15.47:0/3579779368 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7febd0059360 con 0x7febd00598f0 2024-02-23T00:50:50.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.787+0000 7febc6ffd700 10 client.41591.objecter ms_dispatch 0x7febd0001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:50:50.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.787+0000 7febc6ffd700 10 client.41591.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:50:50.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.787+0000 7febc6ffd700 20 client.41591.objecter put_session s=0x7febd0028710 osd=1 4 2024-02-23T00:50:50.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.787+0000 7febc6ffd700 15 client.41591.objecter _session_command_op_remove 1 4 2024-02-23T00:50:50.820 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:50:50.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.787+0000 7febd8137700 10 client.41591.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:50.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.787+0000 7febd8137700 15 client.41591.objecter _session_command_op_assign -1 5 2024-02-23T00:50:50.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.787+0000 7febd8137700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7febd01205e0 0x7febd01209e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:50.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.787+0000 7febd8137700 20 client.41591.objecter _get_session s=0x7febd0027d60 osd=2 3 2024-02-23T00:50:50.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.787+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd0027d60 osd=2 3 2024-02-23T00:50:50.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.787+0000 7febd8137700 20 client.41591.objecter _get_session s=0x7febd0027d60 osd=2 3 2024-02-23T00:50:50.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.787+0000 7febd8137700 15 client.41591.objecter _session_command_op_remove -1 5 2024-02-23T00:50:50.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.787+0000 7febd8137700 20 client.41591.objecter get_session s=0x7febd0027d60 osd=2 3 2024-02-23T00:50:50.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.787+0000 7febd8137700 15 client.41591.objecter _session_command_op_assign 2 5 2024-02-23T00:50:50.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.787+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd0027d60 osd=2 4 2024-02-23T00:50:50.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.787+0000 7febd8137700 10 client.41591.objecter _send_command 5 2024-02-23T00:50:50.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.787+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7febd00283e0 con 0x7febd01205e0 2024-02-23T00:50:50.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.787+0000 7febd56d2700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7febd01205e0 0x7febd01209e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:50.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.788+0000 7febc6ffd700 10 client.41591.objecter ms_handle_connect 0x7febd01205e0 2024-02-23T00:50:50.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.788+0000 7febd56d2700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7febd01205e0 0x7febd01209e0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:50.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.788+0000 7febc6ffd700 1 -- 172.21.15.47:0/3579779368 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7febd00283e0 con 0x7febd01205e0 2024-02-23T00:50:50.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.788+0000 7febc6ffd700 10 client.41591.objecter ms_dispatch 0x7febd0001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:50:50.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.788+0000 7febc6ffd700 10 client.41591.objecter _finish_command 5 = system:0 2024-02-23T00:50:50.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.788+0000 7febc6ffd700 20 client.41591.objecter put_session s=0x7febd0027d60 osd=2 4 2024-02-23T00:50:50.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.788+0000 7febc6ffd700 15 client.41591.objecter _session_command_op_remove 2 5 2024-02-23T00:50:50.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.804+0000 7febd8137700 10 client.41591.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:50:50.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.804+0000 7febd8137700 15 client.41591.objecter _session_command_op_assign -1 6 2024-02-23T00:50:50.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.804+0000 7febd8137700 20 client.41591.objecter _get_session s=0x7febd0027d60 osd=2 3 2024-02-23T00:50:50.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.804+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd0027d60 osd=2 3 2024-02-23T00:50:50.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.804+0000 7febd8137700 20 client.41591.objecter _get_session s=0x7febd0027d60 osd=2 3 2024-02-23T00:50:50.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.804+0000 7febd8137700 15 client.41591.objecter _session_command_op_remove -1 6 2024-02-23T00:50:50.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.804+0000 7febd8137700 20 client.41591.objecter get_session s=0x7febd0027d60 osd=2 3 2024-02-23T00:50:50.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.804+0000 7febd8137700 15 client.41591.objecter _session_command_op_assign 2 6 2024-02-23T00:50:50.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.804+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd0027d60 osd=2 4 2024-02-23T00:50:50.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.804+0000 7febd8137700 10 client.41591.objecter _send_command 6 2024-02-23T00:50:50.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.804+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7febd0028580 con 0x7febd01205e0 2024-02-23T00:50:50.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.806+0000 7febc6ffd700 1 -- 172.21.15.47:0/3579779368 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7febd0028580 con 0x7febd01205e0 2024-02-23T00:50:50.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.806+0000 7febc6ffd700 10 client.41591.objecter ms_dispatch 0x7febd0001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:50:50.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.806+0000 7febc6ffd700 10 client.41591.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:50:50.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.806+0000 7febc6ffd700 20 client.41591.objecter put_session s=0x7febd0027d60 osd=2 4 2024-02-23T00:50:50.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.806+0000 7febc6ffd700 15 client.41591.objecter _session_command_op_remove 2 6 2024-02-23T00:50:50.823 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:50:50.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.806+0000 7febd8137700 10 client.41591.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:50.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.806+0000 7febd8137700 15 client.41591.objecter _session_command_op_assign -1 7 2024-02-23T00:50:50.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.806+0000 7febd8137700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7febd0123d40 0x7febd0126130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:50.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.806+0000 7febd8137700 20 client.41591.objecter _get_session s=0x7febd0027a10 osd=3 3 2024-02-23T00:50:50.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.806+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd0027a10 osd=3 3 2024-02-23T00:50:50.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.806+0000 7febd8137700 20 client.41591.objecter _get_session s=0x7febd0027a10 osd=3 3 2024-02-23T00:50:50.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.806+0000 7febd8137700 15 client.41591.objecter _session_command_op_remove -1 7 2024-02-23T00:50:50.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.806+0000 7febd8137700 20 client.41591.objecter get_session s=0x7febd0027a10 osd=3 3 2024-02-23T00:50:50.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.806+0000 7febd8137700 15 client.41591.objecter _session_command_op_assign 3 7 2024-02-23T00:50:50.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.806+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd0027a10 osd=3 4 2024-02-23T00:50:50.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.806+0000 7febd8137700 10 client.41591.objecter _send_command 7 2024-02-23T00:50:50.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.806+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7febd0128d80 con 0x7febd0123d40 2024-02-23T00:50:50.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.806+0000 7febd66d4700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7febd0123d40 0x7febd0126130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:50.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.807+0000 7febc6ffd700 10 client.41591.objecter ms_handle_connect 0x7febd0123d40 2024-02-23T00:50:50.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.807+0000 7febd66d4700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7febd0123d40 0x7febd0126130 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:50.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.807+0000 7febc6ffd700 1 -- 172.21.15.47:0/3579779368 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7febd0128d80 con 0x7febd0123d40 2024-02-23T00:50:50.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.807+0000 7febc6ffd700 10 client.41591.objecter ms_dispatch 0x7febd0001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:50:50.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.807+0000 7febc6ffd700 10 client.41591.objecter _finish_command 7 = system:0 2024-02-23T00:50:50.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.807+0000 7febc6ffd700 20 client.41591.objecter put_session s=0x7febd0027a10 osd=3 4 2024-02-23T00:50:50.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.807+0000 7febc6ffd700 15 client.41591.objecter _session_command_op_remove 3 7 2024-02-23T00:50:50.826 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:50.826 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:50:50.827 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:50:50.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.823+0000 7febd8137700 10 client.41591.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:50:50.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.823+0000 7febd8137700 15 client.41591.objecter _session_command_op_assign -1 8 2024-02-23T00:50:50.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.823+0000 7febd8137700 20 client.41591.objecter _get_session s=0x7febd0027a10 osd=3 3 2024-02-23T00:50:50.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.823+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd0027a10 osd=3 3 2024-02-23T00:50:50.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.823+0000 7febd8137700 20 client.41591.objecter _get_session s=0x7febd0027a10 osd=3 3 2024-02-23T00:50:50.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.823+0000 7febd8137700 15 client.41591.objecter _session_command_op_remove -1 8 2024-02-23T00:50:50.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.823+0000 7febd8137700 20 client.41591.objecter get_session s=0x7febd0027a10 osd=3 3 2024-02-23T00:50:50.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.823+0000 7febd8137700 15 client.41591.objecter _session_command_op_assign 3 8 2024-02-23T00:50:50.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.823+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd0027a10 osd=3 4 2024-02-23T00:50:50.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.823+0000 7febd8137700 10 client.41591.objecter _send_command 8 2024-02-23T00:50:50.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.823+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7febd0121110 con 0x7febd0123d40 2024-02-23T00:50:50.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.825+0000 7febc6ffd700 1 -- 172.21.15.47:0/3579779368 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7febd0121110 con 0x7febd0123d40 2024-02-23T00:50:50.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.825+0000 7febc6ffd700 10 client.41591.objecter ms_dispatch 0x7febd0001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:50:50.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.825+0000 7febc6ffd700 10 client.41591.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:50:50.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.825+0000 7febc6ffd700 20 client.41591.objecter put_session s=0x7febd0027a10 osd=3 4 2024-02-23T00:50:50.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.825+0000 7febc6ffd700 15 client.41591.objecter _session_command_op_remove 3 8 2024-02-23T00:50:50.829 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:50:50.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.825+0000 7febd8137700 10 client.41591.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:50.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.825+0000 7febd8137700 15 client.41591.objecter _session_command_op_assign -1 9 2024-02-23T00:50:50.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.825+0000 7febd8137700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7febd0120140 0x7febd01277a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:50.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.825+0000 7febd8137700 20 client.41591.objecter _get_session s=0x7febd0121110 osd=4 3 2024-02-23T00:50:50.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.825+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd0121110 osd=4 3 2024-02-23T00:50:50.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.825+0000 7febd8137700 20 client.41591.objecter _get_session s=0x7febd0121110 osd=4 3 2024-02-23T00:50:50.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.825+0000 7febd8137700 15 client.41591.objecter _session_command_op_remove -1 9 2024-02-23T00:50:50.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.825+0000 7febd8137700 20 client.41591.objecter get_session s=0x7febd0121110 osd=4 3 2024-02-23T00:50:50.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.825+0000 7febd8137700 15 client.41591.objecter _session_command_op_assign 4 9 2024-02-23T00:50:50.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.825+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd0121110 osd=4 4 2024-02-23T00:50:50.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.825+0000 7febd8137700 10 client.41591.objecter _send_command 9 2024-02-23T00:50:50.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.825+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7febd0121430 con 0x7febd0120140 2024-02-23T00:50:50.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.826+0000 7febd5ed3700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7febd0120140 0x7febd01277a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:50.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.826+0000 7febc6ffd700 10 client.41591.objecter ms_handle_connect 0x7febd0120140 2024-02-23T00:50:50.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.826+0000 7febd5ed3700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7febd0120140 0x7febd01277a0 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:50.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.827+0000 7febc6ffd700 1 -- 172.21.15.47:0/3579779368 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7febd0121430 con 0x7febd0120140 2024-02-23T00:50:50.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.827+0000 7febc6ffd700 10 client.41591.objecter ms_dispatch 0x7febd0001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:50:50.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.827+0000 7febc6ffd700 10 client.41591.objecter _finish_command 9 = system:0 2024-02-23T00:50:50.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.827+0000 7febc6ffd700 20 client.41591.objecter put_session s=0x7febd0121110 osd=4 4 2024-02-23T00:50:50.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.827+0000 7febc6ffd700 15 client.41591.objecter _session_command_op_remove 4 9 2024-02-23T00:50:50.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.843+0000 7febd8137700 10 client.41591.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:50:50.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.843+0000 7febd8137700 15 client.41591.objecter _session_command_op_assign -1 10 2024-02-23T00:50:50.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.843+0000 7febd8137700 20 client.41591.objecter _get_session s=0x7febd0121110 osd=4 3 2024-02-23T00:50:50.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.843+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd0121110 osd=4 3 2024-02-23T00:50:50.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.843+0000 7febd8137700 20 client.41591.objecter _get_session s=0x7febd0121110 osd=4 3 2024-02-23T00:50:50.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.843+0000 7febd8137700 15 client.41591.objecter _session_command_op_remove -1 10 2024-02-23T00:50:50.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.843+0000 7febd8137700 20 client.41591.objecter get_session s=0x7febd0121110 osd=4 3 2024-02-23T00:50:50.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.843+0000 7febd8137700 15 client.41591.objecter _session_command_op_assign 4 10 2024-02-23T00:50:50.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.843+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd0121110 osd=4 4 2024-02-23T00:50:50.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.843+0000 7febd8137700 10 client.41591.objecter _send_command 10 2024-02-23T00:50:50.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.843+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7febd0121430 con 0x7febd0120140 2024-02-23T00:50:50.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.844+0000 7febc6ffd700 1 -- 172.21.15.47:0/3579779368 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7febd0121430 con 0x7febd0120140 2024-02-23T00:50:50.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.845+0000 7febc6ffd700 10 client.41591.objecter ms_dispatch 0x7febd0001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:50:50.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.845+0000 7febc6ffd700 10 client.41591.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:50:50.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.845+0000 7febc6ffd700 20 client.41591.objecter put_session s=0x7febd0121110 osd=4 4 2024-02-23T00:50:50.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.845+0000 7febc6ffd700 15 client.41591.objecter _session_command_op_remove 4 10 2024-02-23T00:50:50.848 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:50:50.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.845+0000 7febd8137700 10 client.41591.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:50.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.845+0000 7febd8137700 15 client.41591.objecter _session_command_op_assign -1 11 2024-02-23T00:50:50.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.845+0000 7febd8137700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7febd01281a0 0x7febd0128600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:50.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.845+0000 7febd8137700 20 client.41591.objecter _get_session s=0x7febd0121430 osd=5 3 2024-02-23T00:50:50.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.845+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd0121430 osd=5 3 2024-02-23T00:50:50.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.845+0000 7febd8137700 20 client.41591.objecter _get_session s=0x7febd0121430 osd=5 3 2024-02-23T00:50:50.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.845+0000 7febd8137700 15 client.41591.objecter _session_command_op_remove -1 11 2024-02-23T00:50:50.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.845+0000 7febd8137700 20 client.41591.objecter get_session s=0x7febd0121430 osd=5 3 2024-02-23T00:50:50.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.845+0000 7febd8137700 15 client.41591.objecter _session_command_op_assign 5 11 2024-02-23T00:50:50.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.845+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd0121430 osd=5 4 2024-02-23T00:50:50.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.845+0000 7febd8137700 10 client.41591.objecter _send_command 11 2024-02-23T00:50:50.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.845+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7febd012f670 con 0x7febd01281a0 2024-02-23T00:50:50.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.845+0000 7febd56d2700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7febd01281a0 0x7febd0128600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:50.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.845+0000 7febc6ffd700 10 client.41591.objecter ms_handle_connect 0x7febd01281a0 2024-02-23T00:50:50.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.845+0000 7febd56d2700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7febd01281a0 0x7febd0128600 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:50.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.846+0000 7febc6ffd700 1 -- 172.21.15.47:0/3579779368 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7febd012f670 con 0x7febd01281a0 2024-02-23T00:50:50.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.846+0000 7febc6ffd700 10 client.41591.objecter ms_dispatch 0x7febd0001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:50:50.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.846+0000 7febc6ffd700 10 client.41591.objecter _finish_command 11 = system:0 2024-02-23T00:50:50.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.846+0000 7febc6ffd700 20 client.41591.objecter put_session s=0x7febd0121430 osd=5 4 2024-02-23T00:50:50.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.846+0000 7febc6ffd700 15 client.41591.objecter _session_command_op_remove 5 11 2024-02-23T00:50:50.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.877+0000 7febd8137700 10 client.41591.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:50:50.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.877+0000 7febd8137700 15 client.41591.objecter _session_command_op_assign -1 12 2024-02-23T00:50:50.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.877+0000 7febd8137700 20 client.41591.objecter _get_session s=0x7febd0121430 osd=5 3 2024-02-23T00:50:50.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.877+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd0121430 osd=5 3 2024-02-23T00:50:50.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.877+0000 7febd8137700 20 client.41591.objecter _get_session s=0x7febd0121430 osd=5 3 2024-02-23T00:50:50.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.877+0000 7febd8137700 15 client.41591.objecter _session_command_op_remove -1 12 2024-02-23T00:50:50.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.877+0000 7febd8137700 20 client.41591.objecter get_session s=0x7febd0121430 osd=5 3 2024-02-23T00:50:50.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.877+0000 7febd8137700 15 client.41591.objecter _session_command_op_assign 5 12 2024-02-23T00:50:50.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.877+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd0121430 osd=5 4 2024-02-23T00:50:50.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.877+0000 7febd8137700 10 client.41591.objecter _send_command 12 2024-02-23T00:50:50.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.877+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7febd012cf10 con 0x7febd01281a0 2024-02-23T00:50:50.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.878+0000 7febc6ffd700 1 -- 172.21.15.47:0/3579779368 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7febd012cf10 con 0x7febd01281a0 2024-02-23T00:50:50.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.878+0000 7febc6ffd700 10 client.41591.objecter ms_dispatch 0x7febd0001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:50:50.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.878+0000 7febc6ffd700 10 client.41591.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:50:50.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.878+0000 7febc6ffd700 20 client.41591.objecter put_session s=0x7febd0121430 osd=5 4 2024-02-23T00:50:50.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.878+0000 7febc6ffd700 15 client.41591.objecter _session_command_op_remove 5 12 2024-02-23T00:50:50.881 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:50:50.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.878+0000 7febd8137700 10 client.41591.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:50.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.878+0000 7febd8137700 15 client.41591.objecter _session_command_op_assign -1 13 2024-02-23T00:50:50.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.879+0000 7febd8137700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7febd012d590 0x7febd012f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:50.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.879+0000 7febd8137700 20 client.41591.objecter _get_session s=0x7febd012cf10 osd=6 3 2024-02-23T00:50:50.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.879+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd012cf10 osd=6 3 2024-02-23T00:50:50.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.879+0000 7febd8137700 20 client.41591.objecter _get_session s=0x7febd012cf10 osd=6 3 2024-02-23T00:50:50.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.879+0000 7febd8137700 15 client.41591.objecter _session_command_op_remove -1 13 2024-02-23T00:50:50.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.879+0000 7febd8137700 20 client.41591.objecter get_session s=0x7febd012cf10 osd=6 3 2024-02-23T00:50:50.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.879+0000 7febd8137700 15 client.41591.objecter _session_command_op_assign 6 13 2024-02-23T00:50:50.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.879+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd012cf10 osd=6 4 2024-02-23T00:50:50.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.879+0000 7febd8137700 10 client.41591.objecter _send_command 13 2024-02-23T00:50:50.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.879+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7febd01327a0 con 0x7febd012d590 2024-02-23T00:50:50.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.879+0000 7febd66d4700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7febd012d590 0x7febd012f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:50.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.879+0000 7febc6ffd700 10 client.41591.objecter ms_handle_connect 0x7febd012d590 2024-02-23T00:50:50.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.879+0000 7febd66d4700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7febd012d590 0x7febd012f980 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:50.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.880+0000 7febc6ffd700 1 -- 172.21.15.47:0/3579779368 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7febd01327a0 con 0x7febd012d590 2024-02-23T00:50:50.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.880+0000 7febc6ffd700 10 client.41591.objecter ms_dispatch 0x7febd0001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:50:50.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.880+0000 7febc6ffd700 10 client.41591.objecter _finish_command 13 = system:0 2024-02-23T00:50:50.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.880+0000 7febc6ffd700 20 client.41591.objecter put_session s=0x7febd012cf10 osd=6 4 2024-02-23T00:50:50.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.880+0000 7febc6ffd700 15 client.41591.objecter _session_command_op_remove 6 13 2024-02-23T00:50:50.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.896+0000 7febd8137700 10 client.41591.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:50:50.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.896+0000 7febd8137700 15 client.41591.objecter _session_command_op_assign -1 14 2024-02-23T00:50:50.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.896+0000 7febd8137700 20 client.41591.objecter _get_session s=0x7febd012cf10 osd=6 3 2024-02-23T00:50:50.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.896+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd012cf10 osd=6 3 2024-02-23T00:50:50.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.896+0000 7febd8137700 20 client.41591.objecter _get_session s=0x7febd012cf10 osd=6 3 2024-02-23T00:50:50.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.896+0000 7febd8137700 15 client.41591.objecter _session_command_op_remove -1 14 2024-02-23T00:50:50.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.896+0000 7febd8137700 20 client.41591.objecter get_session s=0x7febd012cf10 osd=6 3 2024-02-23T00:50:50.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.896+0000 7febd8137700 15 client.41591.objecter _session_command_op_assign 6 14 2024-02-23T00:50:50.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.896+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd012cf10 osd=6 4 2024-02-23T00:50:50.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.896+0000 7febd8137700 10 client.41591.objecter _send_command 14 2024-02-23T00:50:50.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.896+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7febd01327a0 con 0x7febd012d590 2024-02-23T00:50:50.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.898+0000 7febc6ffd700 1 -- 172.21.15.47:0/3579779368 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7febd01327a0 con 0x7febd012d590 2024-02-23T00:50:50.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.898+0000 7febc6ffd700 10 client.41591.objecter ms_dispatch 0x7febd0001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:50:50.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.898+0000 7febc6ffd700 10 client.41591.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:50:50.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.898+0000 7febc6ffd700 20 client.41591.objecter put_session s=0x7febd012cf10 osd=6 4 2024-02-23T00:50:50.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.898+0000 7febc6ffd700 15 client.41591.objecter _session_command_op_remove 6 14 2024-02-23T00:50:50.902 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:50:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.899+0000 7febd8137700 10 client.41591.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:50:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.899+0000 7febd8137700 15 client.41591.objecter _session_command_op_assign -1 15 2024-02-23T00:50:50.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.899+0000 7febd8137700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7febd0131ad0 0x7febd0131f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:50.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.899+0000 7febd8137700 20 client.41591.objecter _get_session s=0x7febd01327a0 osd=7 3 2024-02-23T00:50:50.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.899+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd01327a0 osd=7 3 2024-02-23T00:50:50.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.899+0000 7febd8137700 20 client.41591.objecter _get_session s=0x7febd01327a0 osd=7 3 2024-02-23T00:50:50.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.899+0000 7febd8137700 15 client.41591.objecter _session_command_op_remove -1 15 2024-02-23T00:50:50.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.899+0000 7febd8137700 20 client.41591.objecter get_session s=0x7febd01327a0 osd=7 3 2024-02-23T00:50:50.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.899+0000 7febd8137700 15 client.41591.objecter _session_command_op_assign 7 15 2024-02-23T00:50:50.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.899+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd01327a0 osd=7 4 2024-02-23T00:50:50.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.899+0000 7febd8137700 10 client.41591.objecter _send_command 15 2024-02-23T00:50:50.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.899+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7febd0135ad0 con 0x7febd0131ad0 2024-02-23T00:50:50.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.899+0000 7febd5ed3700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7febd0131ad0 0x7febd0131f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:50.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.899+0000 7febd5ed3700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7febd0131ad0 0x7febd0131f50 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.899+0000 7febc6ffd700 10 client.41591.objecter ms_handle_connect 0x7febd0131ad0 2024-02-23T00:50:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.900+0000 7febc6ffd700 1 -- 172.21.15.47:0/3579779368 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7febd0135ad0 con 0x7febd0131ad0 2024-02-23T00:50:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.900+0000 7febc6ffd700 10 client.41591.objecter ms_dispatch 0x7febd0001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:50:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.900+0000 7febc6ffd700 10 client.41591.objecter _finish_command 15 = system:0 2024-02-23T00:50:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.900+0000 7febc6ffd700 20 client.41591.objecter put_session s=0x7febd01327a0 osd=7 4 2024-02-23T00:50:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.900+0000 7febc6ffd700 15 client.41591.objecter _session_command_op_remove 7 15 2024-02-23T00:50:50.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.915+0000 7febd8137700 10 client.41591.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:50:50.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.915+0000 7febd8137700 15 client.41591.objecter _session_command_op_assign -1 16 2024-02-23T00:50:50.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.915+0000 7febd8137700 20 client.41591.objecter _get_session s=0x7febd01327a0 osd=7 3 2024-02-23T00:50:50.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.915+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd01327a0 osd=7 3 2024-02-23T00:50:50.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.915+0000 7febd8137700 20 client.41591.objecter _get_session s=0x7febd01327a0 osd=7 3 2024-02-23T00:50:50.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.915+0000 7febd8137700 15 client.41591.objecter _session_command_op_remove -1 16 2024-02-23T00:50:50.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.915+0000 7febd8137700 20 client.41591.objecter get_session s=0x7febd01327a0 osd=7 3 2024-02-23T00:50:50.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.915+0000 7febd8137700 15 client.41591.objecter _session_command_op_assign 7 16 2024-02-23T00:50:50.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.915+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd01327a0 osd=7 4 2024-02-23T00:50:50.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.915+0000 7febd8137700 10 client.41591.objecter _send_command 16 2024-02-23T00:50:50.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.915+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7febd012ff60 con 0x7febd0131ad0 2024-02-23T00:50:50.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.824+0000 7fe0818b0700 1 -- 172.21.15.47:0/2046106325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe07c0f8cb0 msgr2=0x7fe07c0f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:50.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.824+0000 7fe0818b0700 1 --2- 172.21.15.47:0/2046106325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe07c0f8cb0 0x7fe07c0f9090 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 rx=0x7fe06c0047b0 tx=0x7fe06c009cb0).stop 2024-02-23T00:50:50.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.824+0000 7fe0818b0700 1 -- 172.21.15.47:0/2046106325 shutdown_connections 2024-02-23T00:50:50.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.824+0000 7fe0818b0700 1 --2- 172.21.15.47:0/2046106325 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe07c102210 0x7fe07c104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:50.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.824+0000 7fe0818b0700 1 --2- 172.21.15.47:0/2046106325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe07c0f8cb0 0x7fe07c0f9090 unknown :-1 s=CLOSED pgs=2565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:50.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.824+0000 7fe0818b0700 1 --2- 172.21.15.47:0/2046106325 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe07c0f9610 0x7fe07c101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:50.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.824+0000 7fe0818b0700 1 -- 172.21.15.47:0/2046106325 >> 172.21.15.47:0/2046106325 conn(0x7fe07c0f2820 msgr2=0x7fe07c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.825+0000 7fe0818b0700 1 -- 172.21.15.47:0/2046106325 shutdown_connections 2024-02-23T00:50:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.825+0000 7fe0818b0700 1 -- 172.21.15.47:0/2046106325 wait complete. 2024-02-23T00:50:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.825+0000 7fe0818b0700 1 librados: starting msgr at 2024-02-23T00:50:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.825+0000 7fe0818b0700 1 librados: starting objecter 2024-02-23T00:50:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.825+0000 7fe0818b0700 1 Processor -- start 2024-02-23T00:50:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.825+0000 7fe0818b0700 1 -- start start 2024-02-23T00:50:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.825+0000 7fe0818b0700 1 librados: setting wanted keys 2024-02-23T00:50:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.825+0000 7fe0818b0700 1 librados: calling monclient init 2024-02-23T00:50:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.826+0000 7fe0818b0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe07c0f8cb0 0x7fe07c10e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.826+0000 7fe0818b0700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe07c0f9610 0x7fe07c10e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.826+0000 7fe0818b0700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe07c102210 0x7fe07c108260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.826+0000 7fe0818b0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07c108750 con 0x7fe07c0f8cb0 2024-02-23T00:50:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.826+0000 7fe0818b0700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07c1088c0 con 0x7fe07c102210 2024-02-23T00:50:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.826+0000 7fe0818b0700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe07c108a80 con 0x7fe07c0f9610 2024-02-23T00:50:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.826+0000 7fe07a7fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe07c0f9610 0x7fe07c10e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.826+0000 7fe07affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe07c0f8cb0 0x7fe07c10e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.826+0000 7fe07a7fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe07c0f9610 0x7fe07c10e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:47532/0 (socket says 172.21.15.47:47532) 2024-02-23T00:50:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.826+0000 7fe07a7fc700 1 -- 172.21.15.47:0/839380814 learned_addr learned my addr 172.21.15.47:0/839380814 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:50:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.826+0000 7fe07b7fe700 1 --2- 172.21.15.47:0/839380814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe07c102210 0x7fe07c108260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.826+0000 7fe07a7fc700 1 -- 172.21.15.47:0/839380814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe07c102210 msgr2=0x7fe07c108260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.826+0000 7fe07a7fc700 1 --2- 172.21.15.47:0/839380814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe07c102210 0x7fe07c108260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.826+0000 7fe07a7fc700 1 -- 172.21.15.47:0/839380814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe07c0f8cb0 msgr2=0x7fe07c10e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.826+0000 7fe07a7fc700 1 --2- 172.21.15.47:0/839380814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe07c0f8cb0 0x7fe07c10e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.826+0000 7fe07a7fc700 1 -- 172.21.15.47:0/839380814 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe070005040 con 0x7fe07c0f9610 2024-02-23T00:50:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.826+0000 7fe07affd700 1 --2- 172.21.15.47:0/839380814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe07c0f8cb0 0x7fe07c10e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:50:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.826+0000 7fe07a7fc700 1 --2- 172.21.15.47:0/839380814 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe07c0f9610 0x7fe07c10e5b0 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 rx=0x7fe070009400 tx=0x7fe070009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.826+0000 7fe063fff700 10 client.?.objecter ms_handle_connect 0x7fe07c0f9610 2024-02-23T00:50:50.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.826+0000 7fe063fff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:50:50.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.827+0000 7fe063fff700 1 -- 172.21.15.47:0/839380814 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe070007a40 con 0x7fe07c0f9610 2024-02-23T00:50:50.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.827+0000 7fe063fff700 1 -- 172.21.15.47:0/839380814 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fe07000f040 con 0x7fe07c0f9610 2024-02-23T00:50:50.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.827+0000 7fe063fff700 1 -- 172.21.15.47:0/839380814 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe070014900 con 0x7fe07c0f9610 2024-02-23T00:50:50.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.827+0000 7fe0818b0700 1 -- 172.21.15.47:0/839380814 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe06c005040 con 0x7fe07c0f9610 2024-02-23T00:50:50.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.827+0000 7fe0818b0700 10 client.41597.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:50:50.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.827+0000 7fe0818b0700 1 -- 172.21.15.47:0/839380814 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe07c109070 con 0x7fe07c0f9610 2024-02-23T00:50:50.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.827+0000 7fe0818b0700 1 librados: init done 2024-02-23T00:50:50.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.828+0000 7fe0818b0700 1 -- 172.21.15.47:0/839380814 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe07c05fa60 con 0x7fe07c0f9610 2024-02-23T00:50:50.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.828+0000 7fe063fff700 1 -- 172.21.15.47:0/839380814 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fe070014d30 con 0x7fe07c0f9610 2024-02-23T00:50:50.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.828+0000 7fe063fff700 1 --2- 172.21.15.47:0/839380814 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe064062cc0 0x7fe064065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:50.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.829+0000 7fe063fff700 1 -- 172.21.15.47:0/839380814 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(521..521 src has 463..521) v4 ==== 9654+0+0 (secure 0 0 0) 0x7fe07007d410 con 0x7fe07c0f9610 2024-02-23T00:50:50.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.829+0000 7fe063fff700 10 client.41597.objecter ms_dispatch 0x7fe07c001550 osd_map(521..521 src has 463..521) v4 2024-02-23T00:50:50.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.829+0000 7fe063fff700 3 client.41597.objecter handle_osd_map got epochs [521,521] > 0 2024-02-23T00:50:50.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.829+0000 7fe063fff700 3 client.41597.objecter handle_osd_map decoding full epoch 521 2024-02-23T00:50:50.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.829+0000 7fe07affd700 1 --2- 172.21.15.47:0/839380814 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe064062cc0 0x7fe064065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:50.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.829+0000 7fe063fff700 20 client.41597.objecter dump_active .. 0 homeless 2024-02-23T00:50:50.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.829+0000 7fe07affd700 1 --2- 172.21.15.47:0/839380814 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe064062cc0 0x7fe064065180 secure :-1 s=READY pgs=3108 cs=0 l=1 rev1=1 rx=0x7fe06c004e20 tx=0x7fe06c009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:50.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.829+0000 7fe063fff700 10 client.41597.objecter ms_handle_connect 0x7fe064062cc0 2024-02-23T00:50:50.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.833+0000 7fe063fff700 1 -- 172.21.15.47:0/839380814 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe070049f60 con 0x7fe07c0f9610 2024-02-23T00:50:50.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febc6ffd700 1 -- 172.21.15.47:0/3579779368 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7febd012ff60 con 0x7febd0131ad0 2024-02-23T00:50:50.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febc6ffd700 10 client.41591.objecter ms_dispatch 0x7febd0001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:50:50.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febc6ffd700 10 client.41591.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:50:50.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febc6ffd700 20 client.41591.objecter put_session s=0x7febd01327a0 osd=7 4 2024-02-23T00:50:50.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febc6ffd700 15 client.41591.objecter _session_command_op_remove 7 16 2024-02-23T00:50:50.925 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:50:50.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 10 librados: watch_flush enter 2024-02-23T00:50:50.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 10 librados: watch_flush exit 2024-02-23T00:50:50.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 10 client.41591.objecter close_session for osd.0 2024-02-23T00:50:50.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7febd0105de0 msgr2=0x7febd01061c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:50.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7febd0105de0 0x7febd01061c0 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:50.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd005ee80 osd=0 1 2024-02-23T00:50:50.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 10 client.41591.objecter close_session for osd.1 2024-02-23T00:50:50.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7febd00598f0 msgr2=0x7febd0059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:50.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7febd00598f0 0x7febd0059cd0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:50.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd0028710 osd=1 1 2024-02-23T00:50:50.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 10 client.41591.objecter close_session for osd.2 2024-02-23T00:50:50.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7febd01205e0 msgr2=0x7febd01209e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:50.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7febd01205e0 0x7febd01209e0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:50.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd0027d60 osd=2 1 2024-02-23T00:50:50.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 10 client.41591.objecter close_session for osd.3 2024-02-23T00:50:50.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7febd0123d40 msgr2=0x7febd0126130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:50.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7febd0123d40 0x7febd0126130 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:50.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd0027a10 osd=3 1 2024-02-23T00:50:50.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 10 client.41591.objecter close_session for osd.4 2024-02-23T00:50:50.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7febd0120140 msgr2=0x7febd01277a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:50.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd66d4700 1 -- 172.21.15.47:0/3579779368 reap_dead start 2024-02-23T00:50:50.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7febd0120140 0x7febd01277a0 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:50.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd66d4700 1 -- 172.21.15.47:0/3579779368 reap_dead start 2024-02-23T00:50:50.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd0121110 osd=4 1 2024-02-23T00:50:50.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 10 client.41591.objecter close_session for osd.5 2024-02-23T00:50:50.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7febd01281a0 msgr2=0x7febd0128600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:50.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7febd01281a0 0x7febd0128600 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:50.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd0121430 osd=5 1 2024-02-23T00:50:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 10 client.41591.objecter close_session for osd.6 2024-02-23T00:50:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7febd012d590 msgr2=0x7febd012f980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7febd012d590 0x7febd012f980 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd012cf10 osd=6 1 2024-02-23T00:50:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 10 client.41591.objecter close_session for osd.7 2024-02-23T00:50:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7febd0131ad0 msgr2=0x7febd0131f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7febd0131ad0 0x7febd0131f50 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 20 client.41591.objecter put_session s=0x7febd01327a0 osd=7 1 2024-02-23T00:50:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 20 client.41591.objecter shutdown clearing up homeless session... 2024-02-23T00:50:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 10 client.41591.objecter successfully canceled tick 2024-02-23T00:50:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7febbc062cc0 msgr2=0x7febbc065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7febbc062cc0 0x7febbc065180 secure :-1 s=READY pgs=3107 cs=0 l=1 rev1=1 rx=0x7febcc004e20 tx=0x7febcc009cb0).stop 2024-02-23T00:50:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7febd00fbfe0 msgr2=0x7febd010e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 1 --2- 172.21.15.47:0/3579779368 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7febd00fbfe0 0x7febd010e770 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 rx=0x7febc8001df0 tx=0x7febc8004d10).stop 2024-02-23T00:50:50.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd66d4700 1 -- 172.21.15.47:0/3579779368 reap_dead start 2024-02-23T00:50:50.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd66d4700 1 -- 172.21.15.47:0/3579779368 reap_dead start 2024-02-23T00:50:50.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 shutdown_connections 2024-02-23T00:50:50.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.917+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 >> 172.21.15.47:0/3579779368 conn(0x7febd00f2820 msgr2=0x7febd00f9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:50.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.918+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 shutdown_connections 2024-02-23T00:50:50.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.918+0000 7febd8137700 1 -- 172.21.15.47:0/3579779368 wait complete. 2024-02-23T00:50:50.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:50.918+0000 7febd8137700 1 librados: shutdown 2024-02-23T00:50:50.930 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:50:50.930 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:50:50.930 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:50:50.930 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:50:50.930 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:50:50.930 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:50:50.930 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:50:50.931 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:50:50.931 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:50:50.959 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:51.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:51.052+0000 7fe0818b0700 1 -- 172.21.15.47:0/839380814 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd reweight", "id": 6, "weight": 1.0} v 0) v1 -- 0x7fe07c107050 con 0x7fe07c0f9610 2024-02-23T00:50:51.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:51.258+0000 7fe063fff700 1 -- 172.21.15.47:0/839380814 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 6, "weight": 1.0}]=0 reweighted osd.6 to 1 (10000) v522) v1 ==== 113+0+0 (secure 0 0 0) 0x7fe07004c870 con 0x7fe07c0f9610 2024-02-23T00:50:51.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:51.260+0000 7fe0818b0700 10 librados: watch_flush enter 2024-02-23T00:50:51.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:51.261+0000 7fe0818b0700 10 librados: watch_flush exit 2024-02-23T00:50:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:51.261+0000 7fe0818b0700 20 client.41597.objecter shutdown clearing up homeless session... 2024-02-23T00:50:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:51.261+0000 7fe0818b0700 10 client.41597.objecter successfully canceled tick 2024-02-23T00:50:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:51.261+0000 7fe0818b0700 1 -- 172.21.15.47:0/839380814 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe064062cc0 msgr2=0x7fe064065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:51.261+0000 7fe0818b0700 1 --2- 172.21.15.47:0/839380814 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe064062cc0 0x7fe064065180 secure :-1 s=READY pgs=3108 cs=0 l=1 rev1=1 rx=0x7fe06c004e20 tx=0x7fe06c009cb0).stop 2024-02-23T00:50:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:51.261+0000 7fe0818b0700 1 -- 172.21.15.47:0/839380814 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe07c0f9610 msgr2=0x7fe07c10e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:51.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:51.261+0000 7fe0818b0700 1 --2- 172.21.15.47:0/839380814 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe07c0f9610 0x7fe07c10e5b0 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 rx=0x7fe070009400 tx=0x7fe070009cb0).stop 2024-02-23T00:50:51.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:51.262+0000 7fe0818b0700 1 -- 172.21.15.47:0/839380814 shutdown_connections 2024-02-23T00:50:51.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:51.262+0000 7fe0818b0700 1 --2- 172.21.15.47:0/839380814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe07c102210 0x7fe07c108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:51.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:51.262+0000 7fe0818b0700 1 --2- 172.21.15.47:0/839380814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe07c0f8cb0 0x7fe07c10e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:51.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:51.262+0000 7fe0818b0700 1 --2- 172.21.15.47:0/839380814 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe064062cc0 0x7fe064065180 unknown :-1 s=CLOSED pgs=3108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:51.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:51.262+0000 7fe0818b0700 1 --2- 172.21.15.47:0/839380814 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe07c0f9610 0x7fe07c10e5b0 unknown :-1 s=CLOSED pgs=2719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:51.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:51.262+0000 7fe0818b0700 1 -- 172.21.15.47:0/839380814 >> 172.21.15.47:0/839380814 conn(0x7fe07c0f2820 msgr2=0x7fe07c0f34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:51.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:51.262+0000 7fe0818b0700 1 -- 172.21.15.47:0/839380814 shutdown_connections 2024-02-23T00:50:51.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:51.262+0000 7fe0818b0700 1 -- 172.21.15.47:0/839380814 wait complete. 2024-02-23T00:50:51.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:51.263+0000 7fe0818b0700 1 librados: shutdown 2024-02-23T00:50:51.266 INFO:teuthology.orchestra.run.smithi047.stderr:reweighted osd.6 to 1 (10000) 2024-02-23T00:50:51.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:50.724037+0000 osd.5 (osd.5) 232 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:50:51.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:50.724209+0000 osd.5 (osd.5) 233 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:50:51.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:50.752253+0000 mon.c (mon.2) 320 : audit [DBG] from='client.? 172.21.15.47:0/3579779368' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:50:51.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:51.053951+0000 mon.c (mon.2) 321 : audit [INF] from='client.? 172.21.15.47:0/839380814' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-23T00:50:51.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:51.054451+0000 mon.a (mon.0) 2863 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-23T00:50:51.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:51.258726+0000 mon.a (mon.0) 2864 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-23T00:50:51.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:51.258780+0000 mon.a (mon.0) 2865 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-02-23T00:50:51.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:51.258814+0000 mon.a (mon.0) 2866 : cluster [DBG] osdmap e522: 8 total, 8 up, 7 in 2024-02-23T00:50:51.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:50.724037+0000 osd.5 (osd.5) 232 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:50:51.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:50.724209+0000 osd.5 (osd.5) 233 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:50:51.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:50.752253+0000 mon.c (mon.2) 320 : audit [DBG] from='client.? 172.21.15.47:0/3579779368' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:50:51.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:51.053951+0000 mon.c (mon.2) 321 : audit [INF] from='client.? 172.21.15.47:0/839380814' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-23T00:50:51.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:51.054451+0000 mon.a (mon.0) 2863 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-23T00:50:51.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:51.258726+0000 mon.a (mon.0) 2864 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-23T00:50:51.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:51.258780+0000 mon.a (mon.0) 2865 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-02-23T00:50:51.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:51.258814+0000 mon.a (mon.0) 2866 : cluster [DBG] osdmap e522: 8 total, 8 up, 7 in 2024-02-23T00:50:51.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:50.724037+0000 osd.5 (osd.5) 232 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:50:51.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:50.724209+0000 osd.5 (osd.5) 233 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:50:51.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:50.752253+0000 mon.c (mon.2) 320 : audit [DBG] from='client.? 172.21.15.47:0/3579779368' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:50:51.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:51.053951+0000 mon.c (mon.2) 321 : audit [INF] from='client.? 172.21.15.47:0/839380814' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-23T00:50:51.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:51.054451+0000 mon.a (mon.0) 2863 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-23T00:50:51.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:51.258726+0000 mon.a (mon.0) 2864 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-23T00:50:51.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:51.258780+0000 mon.a (mon.0) 2865 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-02-23T00:50:51.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:51.258814+0000 mon.a (mon.0) 2866 : cluster [DBG] osdmap e522: 8 total, 8 up, 7 in 2024-02-23T00:50:51.910 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:50:51.912 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:51.913 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:50:52.013 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:50:52.046 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:52.068 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:50:52.070 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:52.070 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:50:52.171 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:50:52.203 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:52.304 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:51.333520+0000 mgr.y (mgr.14517) 483 : cluster [DBG] pgmap v787: 240 pgs: 32 unknown, 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:50:52.304 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:52.260059+0000 mon.a (mon.0) 2867 : cluster [DBG] osdmap e523: 8 total, 8 up, 7 in 2024-02-23T00:50:52.304 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:52.295053+0000 mon.a (mon.0) 2868 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:50:52.305 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:52.295408+0000 mon.a (mon.0) 2869 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-83"}]: dispatch 2024-02-23T00:50:52.404 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:50:52.409 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:52.410 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:50:52.511 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:50:52.538 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:51.333520+0000 mgr.y (mgr.14517) 483 : cluster [DBG] pgmap v787: 240 pgs: 32 unknown, 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:50:52.538 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:52.260059+0000 mon.a (mon.0) 2867 : cluster [DBG] osdmap e523: 8 total, 8 up, 7 in 2024-02-23T00:50:52.538 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:52.295053+0000 mon.a (mon.0) 2868 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:50:52.538 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:52.295408+0000 mon.a (mon.0) 2869 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-83"}]: dispatch 2024-02-23T00:50:52.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:51.333520+0000 mgr.y (mgr.14517) 483 : cluster [DBG] pgmap v787: 240 pgs: 32 unknown, 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:50:52.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:52.260059+0000 mon.a (mon.0) 2867 : cluster [DBG] osdmap e523: 8 total, 8 up, 7 in 2024-02-23T00:50:52.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:52.295053+0000 mon.a (mon.0) 2868 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:50:52.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:52.295408+0000 mon.a (mon.0) 2869 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-83"}]: dispatch 2024-02-23T00:50:52.540 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:52.693 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:50:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:50:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:50:52.747 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight 7 1 2024-02-23T00:50:53.194 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:50:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:50:53.684 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:50:54.419 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:50:54.421 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:54.422 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:50:54.523 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:50:54.550 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:54.563 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:52.724549+0000 mgr.y (mgr.14517) 484 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:50:54.564 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:53.305885+0000 mon.a (mon.0) 2870 : cluster [DBG] osdmap e524: 8 total, 8 up, 7 in 2024-02-23T00:50:54.564 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:53.333869+0000 mgr.y (mgr.14517) 485 : cluster [DBG] pgmap v790: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:50:54.564 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:54.076656+0000 mon.a (mon.0) 2871 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:50:54.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:52.724549+0000 mgr.y (mgr.14517) 484 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:50:54.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:53.305885+0000 mon.a (mon.0) 2870 : cluster [DBG] osdmap e524: 8 total, 8 up, 7 in 2024-02-23T00:50:54.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:53.333869+0000 mgr.y (mgr.14517) 485 : cluster [DBG] pgmap v790: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:50:54.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:54.076656+0000 mon.a (mon.0) 2871 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:50:54.568 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:50:54.570 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:52.724549+0000 mgr.y (mgr.14517) 484 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:50:54.570 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:53.305885+0000 mon.a (mon.0) 2870 : cluster [DBG] osdmap e524: 8 total, 8 up, 7 in 2024-02-23T00:50:54.570 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:53.333869+0000 mgr.y (mgr.14517) 485 : cluster [DBG] pgmap v790: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:50:54.570 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:54.076656+0000 mon.a (mon.0) 2871 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:50:54.571 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:54.571 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:50:54.672 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:50:54.702 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:54.725 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:50:54.728 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:54.728 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:50:54.829 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:50:54.839 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:50:54.839 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:50:54.856 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:54.874 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:50:54.877 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:54.877 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:50:54.978 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:50:55.008 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:55.047 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:50:55.054 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:55.055 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:50:55.155 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:50:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:55.561 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.3 dump_ops_in_flight 2024-02-23T00:50:55.627 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:54.312389+0000 mon.a (mon.0) 2872 : cluster [DBG] osdmap e525: 8 total, 8 up, 7 in 2024-02-23T00:50:55.627 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:54.317992+0000 mon.a (mon.0) 2873 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-86","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:50:55.627 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:54.312389+0000 mon.a (mon.0) 2872 : cluster [DBG] osdmap e525: 8 total, 8 up, 7 in 2024-02-23T00:50:55.627 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:54.317992+0000 mon.a (mon.0) 2873 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-86","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:50:55.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:54.312389+0000 mon.a (mon.0) 2872 : cluster [DBG] osdmap e525: 8 total, 8 up, 7 in 2024-02-23T00:50:55.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:54.317992+0000 mon.a (mon.0) 2873 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-86","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:50:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.985+0000 7eff1d233700 1 -- 172.21.15.47:0/2819520095 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7eff181022d0 msgr2=0x7eff181046c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.985+0000 7eff1d233700 1 --2- 172.21.15.47:0/2819520095 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7eff181022d0 0x7eff181046c0 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 rx=0x7eff10009be0 tx=0x7eff10004d10).stop 2024-02-23T00:50:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.985+0000 7eff1d233700 1 -- 172.21.15.47:0/2819520095 shutdown_connections 2024-02-23T00:50:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.985+0000 7eff1d233700 1 --2- 172.21.15.47:0/2819520095 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7eff181022d0 0x7eff181046c0 unknown :-1 s=CLOSED pgs=2720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.985+0000 7eff1d233700 1 --2- 172.21.15.47:0/2819520095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff180f8d70 0x7eff180f9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.985+0000 7eff1d233700 1 --2- 172.21.15.47:0/2819520095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff180f9640 0x7eff18101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.985+0000 7eff1d233700 1 -- 172.21.15.47:0/2819520095 >> 172.21.15.47:0/2819520095 conn(0x7eff180f2820 msgr2=0x7eff180f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.985+0000 7eff1d233700 1 -- 172.21.15.47:0/2819520095 shutdown_connections 2024-02-23T00:50:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.985+0000 7eff1d233700 1 -- 172.21.15.47:0/2819520095 wait complete. 2024-02-23T00:50:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.985+0000 7eff1d233700 1 librados: starting msgr at 2024-02-23T00:50:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.985+0000 7eff1d233700 1 librados: starting objecter 2024-02-23T00:50:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.986+0000 7eff1d233700 1 Processor -- start 2024-02-23T00:50:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.986+0000 7eff1d233700 1 -- start start 2024-02-23T00:50:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.986+0000 7eff1d233700 1 librados: setting wanted keys 2024-02-23T00:50:56.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.986+0000 7eff1d233700 1 librados: calling monclient init 2024-02-23T00:50:56.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.986+0000 7eff1d233700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff180f8d70 0x7eff1810dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:56.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.986+0000 7eff1d233700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7eff180f9640 0x7eff1810e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:56.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.986+0000 7eff1d233700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff181022d0 0x7eff18107fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:56.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.986+0000 7eff1d233700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff181084b0 con 0x7eff180f8d70 2024-02-23T00:50:56.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.986+0000 7eff1d233700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff18108620 con 0x7eff181022d0 2024-02-23T00:50:56.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.986+0000 7eff1d233700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff18108790 con 0x7eff180f9640 2024-02-23T00:50:56.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.986+0000 7eff16d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff180f8d70 0x7eff1810dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:56.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.986+0000 7eff1659c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7eff180f9640 0x7eff1810e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:56.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.986+0000 7eff16d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff180f8d70 0x7eff1810dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50638/0 (socket says 172.21.15.47:50638) 2024-02-23T00:50:56.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.986+0000 7eff1659c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7eff180f9640 0x7eff1810e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:47548/0 (socket says 172.21.15.47:47548) 2024-02-23T00:50:56.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.986+0000 7eff16d9d700 1 -- 172.21.15.47:0/3330810981 learned_addr learned my addr 172.21.15.47:0/3330810981 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:50:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.986+0000 7eff1759e700 1 --2- 172.21.15.47:0/3330810981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff181022d0 0x7eff18107fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.987+0000 7eff16d9d700 1 -- 172.21.15.47:0/3330810981 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7eff180f9640 msgr2=0x7eff1810e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.987+0000 7eff16d9d700 1 --2- 172.21.15.47:0/3330810981 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7eff180f9640 0x7eff1810e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.987+0000 7eff16d9d700 1 -- 172.21.15.47:0/3330810981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff181022d0 msgr2=0x7eff18107fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.987+0000 7eff16d9d700 1 --2- 172.21.15.47:0/3330810981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff181022d0 0x7eff18107fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.987+0000 7eff16d9d700 1 -- 172.21.15.47:0/3330810981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff0c005040 con 0x7eff180f8d70 2024-02-23T00:50:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.987+0000 7eff1659c700 1 --2- 172.21.15.47:0/3330810981 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7eff180f9640 0x7eff1810e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:50:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.987+0000 7eff16d9d700 1 --2- 172.21.15.47:0/3330810981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff180f8d70 0x7eff1810dde0 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 rx=0x7eff0c0048f0 tx=0x7eff0c009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.987+0000 7efefffff700 10 client.?.objecter ms_handle_connect 0x7eff180f8d70 2024-02-23T00:50:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.987+0000 7efefffff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:50:56.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.987+0000 7efefffff700 1 -- 172.21.15.47:0/3330810981 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c007b00 con 0x7eff180f8d70 2024-02-23T00:50:56.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.987+0000 7eff1d233700 1 -- 172.21.15.47:0/3330810981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff100097c0 con 0x7eff180f8d70 2024-02-23T00:50:56.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.987+0000 7efefffff700 1 -- 172.21.15.47:0/3330810981 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7eff0c00f040 con 0x7eff180f8d70 2024-02-23T00:50:56.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.987+0000 7efefffff700 1 -- 172.21.15.47:0/3330810981 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c014820 con 0x7eff180f8d70 2024-02-23T00:50:56.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.987+0000 7eff1d233700 10 client.31278.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:50:56.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.987+0000 7eff1d233700 1 -- 172.21.15.47:0/3330810981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff18108dd0 con 0x7eff180f8d70 2024-02-23T00:50:56.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.987+0000 7eff1d233700 1 librados: init done 2024-02-23T00:50:56.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.988+0000 7eff1d233700 1 -- 172.21.15.47:0/3330810981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff1805fa60 con 0x7eff180f8d70 2024-02-23T00:50:56.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.989+0000 7efefffff700 1 -- 172.21.15.47:0/3330810981 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7eff0c014c50 con 0x7eff180f8d70 2024-02-23T00:50:56.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.989+0000 7efefffff700 1 --2- 172.21.15.47:0/3330810981 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7eff00062cc0 0x7eff00065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:56.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.989+0000 7efefffff700 1 -- 172.21.15.47:0/3330810981 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(526..526 src has 463..526) v4 ==== 9220+0+0 (secure 0 0 0) 0x7eff0c07d1d0 con 0x7eff180f8d70 2024-02-23T00:50:56.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.989+0000 7efefffff700 10 client.31278.objecter ms_dispatch 0x7eff18001550 osd_map(526..526 src has 463..526) v4 2024-02-23T00:50:56.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.989+0000 7efefffff700 3 client.31278.objecter handle_osd_map got epochs [526,526] > 0 2024-02-23T00:50:56.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.989+0000 7efefffff700 3 client.31278.objecter handle_osd_map decoding full epoch 526 2024-02-23T00:50:56.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.989+0000 7eff1659c700 1 --2- 172.21.15.47:0/3330810981 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7eff00062cc0 0x7eff00065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:56.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.989+0000 7efefffff700 20 client.31278.objecter dump_active .. 0 homeless 2024-02-23T00:50:56.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.989+0000 7efefffff700 10 client.31278.objecter ms_handle_connect 0x7eff00062cc0 2024-02-23T00:50:56.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.989+0000 7eff1659c700 1 --2- 172.21.15.47:0/3330810981 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7eff00062cc0 0x7eff00065180 secure :-1 s=READY pgs=3109 cs=0 l=1 rev1=1 rx=0x7eff18109760 tx=0x7eff0800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:56.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:55.993+0000 7efefffff700 1 -- 172.21.15.47:0/3330810981 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7eff0c049f60 con 0x7eff180f8d70 2024-02-23T00:50:56.008 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:50:56.009 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:50:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:50:56.036 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:56.037 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:50:56.138 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:50:56.171 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:56.192 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:50:56.194 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:56.195 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:50:56.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:56.195+0000 7eff1d233700 1 -- 172.21.15.47:0/3330810981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7eff181094a0 con 0x7eff180f8d70 2024-02-23T00:50:56.296 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:50:56.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:56.314+0000 7efefffff700 1 -- 172.21.15.47:0/3330810981 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v527) v1 ==== 91+0+0 (secure 0 0 0) 0x7eff0c04c7e0 con 0x7eff180f8d70 2024-02-23T00:50:56.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:56.318+0000 7eff1d233700 10 librados: watch_flush enter 2024-02-23T00:50:56.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:56.318+0000 7eff1d233700 10 librados: watch_flush exit 2024-02-23T00:50:56.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:56.318+0000 7eff1d233700 20 client.31278.objecter shutdown clearing up homeless session... 2024-02-23T00:50:56.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:56.318+0000 7eff1d233700 10 client.31278.objecter successfully canceled tick 2024-02-23T00:50:56.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:56.318+0000 7eff1d233700 1 -- 172.21.15.47:0/3330810981 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7eff00062cc0 msgr2=0x7eff00065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:56.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:56.318+0000 7eff1d233700 1 --2- 172.21.15.47:0/3330810981 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7eff00062cc0 0x7eff00065180 secure :-1 s=READY pgs=3109 cs=0 l=1 rev1=1 rx=0x7eff18109760 tx=0x7eff0800b040).stop 2024-02-23T00:50:56.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:56.318+0000 7eff1d233700 1 -- 172.21.15.47:0/3330810981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff180f8d70 msgr2=0x7eff1810dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:56.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:56.318+0000 7eff1d233700 1 --2- 172.21.15.47:0/3330810981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff180f8d70 0x7eff1810dde0 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 rx=0x7eff0c0048f0 tx=0x7eff0c009cb0).stop 2024-02-23T00:50:56.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:56.318+0000 7eff1d233700 1 -- 172.21.15.47:0/3330810981 shutdown_connections 2024-02-23T00:50:56.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:56.318+0000 7eff1d233700 1 --2- 172.21.15.47:0/3330810981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff181022d0 0x7eff18107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:56.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:56.318+0000 7eff1d233700 1 --2- 172.21.15.47:0/3330810981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff180f8d70 0x7eff1810dde0 unknown :-1 s=CLOSED pgs=2566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:56.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:56.318+0000 7eff1d233700 1 --2- 172.21.15.47:0/3330810981 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7eff00062cc0 0x7eff00065180 secure :-1 s=CLOSED pgs=3109 cs=0 l=1 rev1=1 rx=0x7eff18109760 tx=0x7eff0800b040).stop 2024-02-23T00:50:56.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:56.318+0000 7eff1d233700 1 --2- 172.21.15.47:0/3330810981 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7eff180f9640 0x7eff1810e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:56.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:56.318+0000 7eff1d233700 1 -- 172.21.15.47:0/3330810981 >> 172.21.15.47:0/3330810981 conn(0x7eff180f2820 msgr2=0x7eff180f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:56.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:56.319+0000 7eff1d233700 1 -- 172.21.15.47:0/3330810981 shutdown_connections 2024-02-23T00:50:56.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:56.319+0000 7eff1d233700 1 -- 172.21.15.47:0/3330810981 wait complete. 2024-02-23T00:50:56.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:56.319+0000 7eff1d233700 1 librados: shutdown 2024-02-23T00:50:56.323 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is unset 2024-02-23T00:50:56.333 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:56.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:55.312760+0000 mon.a (mon.0) 2874 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-86","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:50:56.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:55.312840+0000 mon.a (mon.0) 2875 : cluster [DBG] osdmap e526: 8 total, 8 up, 7 in 2024-02-23T00:50:56.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:55.315373+0000 mon.a (mon.0) 2876 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:50:56.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:55.321539+0000 mon.a (mon.0) 2877 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-23T00:50:56.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:55.334338+0000 mgr.y (mgr.14517) 486 : cluster [DBG] pgmap v793: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:50:56.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:55.919989+0000 mon.a (mon.0) 2878 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:50:56.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:55.920727+0000 mon.a (mon.0) 2879 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:50:56.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:55.925968+0000 mon.a (mon.0) 2880 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:50:56.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:56.197535+0000 mon.a (mon.0) 2881 : audit [INF] from='client.? 172.21.15.47:0/3330810981' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:50:56.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:56.309829+0000 mon.a (mon.0) 2882 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:50:56.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:56.314482+0000 mon.a (mon.0) 2883 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-23T00:50:56.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:56.314550+0000 mon.a (mon.0) 2884 : audit [INF] from='client.? 172.21.15.47:0/3330810981' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:50:56.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:56.314597+0000 mon.a (mon.0) 2885 : cluster [DBG] osdmap e527: 8 total, 8 up, 7 in 2024-02-23T00:50:56.719 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:55.312760+0000 mon.a (mon.0) 2874 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-86","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:50:56.719 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:55.312840+0000 mon.a (mon.0) 2875 : cluster [DBG] osdmap e526: 8 total, 8 up, 7 in 2024-02-23T00:50:56.719 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:55.315373+0000 mon.a (mon.0) 2876 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:50:56.719 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:55.321539+0000 mon.a (mon.0) 2877 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-23T00:50:56.719 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:55.334338+0000 mgr.y (mgr.14517) 486 : cluster [DBG] pgmap v793: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:50:56.719 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:55.919989+0000 mon.a (mon.0) 2878 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:50:56.719 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:55.920727+0000 mon.a (mon.0) 2879 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:50:56.719 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:55.925968+0000 mon.a (mon.0) 2880 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:50:56.719 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:56.197535+0000 mon.a (mon.0) 2881 : audit [INF] from='client.? 172.21.15.47:0/3330810981' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:50:56.720 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:56.309829+0000 mon.a (mon.0) 2882 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:50:56.720 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:56.314482+0000 mon.a (mon.0) 2883 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-23T00:50:56.720 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:56.314550+0000 mon.a (mon.0) 2884 : audit [INF] from='client.? 172.21.15.47:0/3330810981' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:50:56.720 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:56.314597+0000 mon.a (mon.0) 2885 : cluster [DBG] osdmap e527: 8 total, 8 up, 7 in 2024-02-23T00:50:56.720 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:55.312760+0000 mon.a (mon.0) 2874 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-86","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:50:56.720 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:55.312840+0000 mon.a (mon.0) 2875 : cluster [DBG] osdmap e526: 8 total, 8 up, 7 in 2024-02-23T00:50:56.720 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:55.315373+0000 mon.a (mon.0) 2876 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:50:56.720 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:55.321539+0000 mon.a (mon.0) 2877 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-23T00:50:56.720 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:55.334338+0000 mgr.y (mgr.14517) 486 : cluster [DBG] pgmap v793: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:50:56.720 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:55.919989+0000 mon.a (mon.0) 2878 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:50:56.720 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:55.920727+0000 mon.a (mon.0) 2879 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:50:56.721 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:55.925968+0000 mon.a (mon.0) 2880 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:50:56.721 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:56.197535+0000 mon.a (mon.0) 2881 : audit [INF] from='client.? 172.21.15.47:0/3330810981' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:50:56.721 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:56.309829+0000 mon.a (mon.0) 2882 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:50:56.721 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:56.314482+0000 mon.a (mon.0) 2883 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-23T00:50:56.721 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:56.314550+0000 mon.a (mon.0) 2884 : audit [INF] from='client.? 172.21.15.47:0/3330810981' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:50:56.721 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:56.314597+0000 mon.a (mon.0) 2885 : cluster [DBG] osdmap e527: 8 total, 8 up, 7 in 2024-02-23T00:50:57.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:56.164799+0000 osd.7 (osd.7) 257 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:50:57.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:56.165037+0000 osd.7 (osd.7) 258 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:50:57.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:56.474950+0000 osd.1 (osd.1) 227 : cluster [DBG] 62.1 scrub starts 2024-02-23T00:50:57.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:56.478429+0000 osd.1 (osd.1) 228 : cluster [DBG] 62.1 scrub ok 2024-02-23T00:50:57.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:56.773091+0000 osd.5 (osd.5) 234 : cluster [DBG] 18.b scrub starts 2024-02-23T00:50:57.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:56.774377+0000 osd.5 (osd.5) 235 : cluster [DBG] 18.b scrub ok 2024-02-23T00:50:57.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:56.164799+0000 osd.7 (osd.7) 257 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:50:57.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:56.165037+0000 osd.7 (osd.7) 258 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:50:57.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:56.474950+0000 osd.1 (osd.1) 227 : cluster [DBG] 62.1 scrub starts 2024-02-23T00:50:57.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:56.478429+0000 osd.1 (osd.1) 228 : cluster [DBG] 62.1 scrub ok 2024-02-23T00:50:57.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:56.773091+0000 osd.5 (osd.5) 234 : cluster [DBG] 18.b scrub starts 2024-02-23T00:50:57.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:56.774377+0000 osd.5 (osd.5) 235 : cluster [DBG] 18.b scrub ok 2024-02-23T00:50:57.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:56.164799+0000 osd.7 (osd.7) 257 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:50:57.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:56.165037+0000 osd.7 (osd.7) 258 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:50:57.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:56.474950+0000 osd.1 (osd.1) 227 : cluster [DBG] 62.1 scrub starts 2024-02-23T00:50:57.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:56.478429+0000 osd.1 (osd.1) 228 : cluster [DBG] 62.1 scrub ok 2024-02-23T00:50:57.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:56.773091+0000 osd.5 (osd.5) 234 : cluster [DBG] 18.b scrub starts 2024-02-23T00:50:57.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:56.774377+0000 osd.5 (osd.5) 235 : cluster [DBG] 18.b scrub ok 2024-02-23T00:50:57.791 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:50:57.793 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:57.794 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:50:57.895 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:50:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:50:58.132 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:58.132 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:50:58.235 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:50:58.267 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:58.288 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:50:58.291 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:58.291 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:50:58.392 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:50:58.420 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:58.439 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:50:58.441 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:58.441 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:50:58.504 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:50:58.542 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:50:58.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:57.335511+0000 mgr.y (mgr.14517) 487 : cluster [DBG] pgmap v795: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-02-23T00:50:58.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:57.337677+0000 mon.a (mon.0) 2886 : cluster [DBG] osdmap e528: 8 total, 8 up, 7 in 2024-02-23T00:50:58.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:57.453231+0000 osd.1 (osd.1) 229 : cluster [DBG] 3.0 scrub starts 2024-02-23T00:50:58.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:57.456050+0000 osd.1 (osd.1) 230 : cluster [DBG] 3.0 scrub ok 2024-02-23T00:50:58.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:57.767381+0000 osd.5 (osd.5) 236 : cluster [DBG] 2.c scrub starts 2024-02-23T00:50:58.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:57.770445+0000 osd.5 (osd.5) 237 : cluster [DBG] 2.c scrub ok 2024-02-23T00:50:58.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:58.149591+0000 osd.3 (osd.3) 248 : cluster [DBG] 4.19 scrub starts 2024-02-23T00:50:58.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:58.151489+0000 osd.3 (osd.3) 249 : cluster [DBG] 4.19 scrub ok 2024-02-23T00:50:58.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:57.335511+0000 mgr.y (mgr.14517) 487 : cluster [DBG] pgmap v795: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-02-23T00:50:58.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:57.337677+0000 mon.a (mon.0) 2886 : cluster [DBG] osdmap e528: 8 total, 8 up, 7 in 2024-02-23T00:50:58.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:57.453231+0000 osd.1 (osd.1) 229 : cluster [DBG] 3.0 scrub starts 2024-02-23T00:50:58.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:57.456050+0000 osd.1 (osd.1) 230 : cluster [DBG] 3.0 scrub ok 2024-02-23T00:50:58.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:57.767381+0000 osd.5 (osd.5) 236 : cluster [DBG] 2.c scrub starts 2024-02-23T00:50:58.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:57.770445+0000 osd.5 (osd.5) 237 : cluster [DBG] 2.c scrub ok 2024-02-23T00:50:58.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:58.149591+0000 osd.3 (osd.3) 248 : cluster [DBG] 4.19 scrub starts 2024-02-23T00:50:58.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:58.151489+0000 osd.3 (osd.3) 249 : cluster [DBG] 4.19 scrub ok 2024-02-23T00:50:58.567 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:57.335511+0000 mgr.y (mgr.14517) 487 : cluster [DBG] pgmap v795: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-02-23T00:50:58.567 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:57.337677+0000 mon.a (mon.0) 2886 : cluster [DBG] osdmap e528: 8 total, 8 up, 7 in 2024-02-23T00:50:58.568 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:57.453231+0000 osd.1 (osd.1) 229 : cluster [DBG] 3.0 scrub starts 2024-02-23T00:50:58.568 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:57.456050+0000 osd.1 (osd.1) 230 : cluster [DBG] 3.0 scrub ok 2024-02-23T00:50:58.568 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:57.767381+0000 osd.5 (osd.5) 236 : cluster [DBG] 2.c scrub starts 2024-02-23T00:50:58.568 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:57.770445+0000 osd.5 (osd.5) 237 : cluster [DBG] 2.c scrub ok 2024-02-23T00:50:58.568 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:58.149591+0000 osd.3 (osd.3) 248 : cluster [DBG] 4.19 scrub starts 2024-02-23T00:50:58.568 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:58.151489+0000 osd.3 (osd.3) 249 : cluster [DBG] 4.19 scrub ok 2024-02-23T00:50:58.569 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:58.588 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:50:58.591 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:58.591 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:50:58.692 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:50:58.723 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:58.782 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:50:58.795 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:50:58.795 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:50:58.896 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:50:58.925 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:50:58.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.990+0000 7fd681453700 1 -- 172.21.15.47:0/2056408614 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd67c0f89a0 msgr2=0x7fd67c0f8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:58.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.990+0000 7fd681453700 1 --2- 172.21.15.47:0/2056408614 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd67c0f89a0 0x7fd67c0f8d80 secure :-1 s=READY pgs=2721 cs=0 l=1 rev1=1 rx=0x7fd670004870 tx=0x7fd670009cb0).stop 2024-02-23T00:50:58.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.990+0000 7fd681453700 1 -- 172.21.15.47:0/2056408614 shutdown_connections 2024-02-23T00:50:58.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.990+0000 7fd681453700 1 --2- 172.21.15.47:0/2056408614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd67c0f7330 0x7fd67c103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:58.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.990+0000 7fd681453700 1 --2- 172.21.15.47:0/2056408614 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd67c0f89a0 0x7fd67c0f8d80 unknown :-1 s=CLOSED pgs=2721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:58.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.990+0000 7fd681453700 1 --2- 172.21.15.47:0/2056408614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd67c0f9270 0x7fd67c0f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:58.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.990+0000 7fd681453700 1 -- 172.21.15.47:0/2056408614 >> 172.21.15.47:0/2056408614 conn(0x7fd67c0f26f0 msgr2=0x7fd67c0f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:58.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.990+0000 7fd681453700 1 -- 172.21.15.47:0/2056408614 shutdown_connections 2024-02-23T00:50:58.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.990+0000 7fd681453700 1 -- 172.21.15.47:0/2056408614 wait complete. 2024-02-23T00:50:58.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.991+0000 7fd681453700 1 librados: starting msgr at 2024-02-23T00:50:58.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.991+0000 7fd681453700 1 librados: starting objecter 2024-02-23T00:50:58.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.991+0000 7fd681453700 1 Processor -- start 2024-02-23T00:50:58.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.991+0000 7fd681453700 1 -- start start 2024-02-23T00:50:58.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.991+0000 7fd681453700 1 librados: setting wanted keys 2024-02-23T00:50:58.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.991+0000 7fd681453700 1 librados: calling monclient init 2024-02-23T00:50:58.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.991+0000 7fd681453700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd67c0f7330 0x7fd67c0f99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:58.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.991+0000 7fd681453700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd67c0f89a0 0x7fd67c0f9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:58.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.991+0000 7fd681453700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd67c0f9270 0x7fd67c0fa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:58.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.991+0000 7fd681453700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67c1164a0 con 0x7fd67c0f9270 2024-02-23T00:50:58.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.991+0000 7fd681453700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67c116610 con 0x7fd67c0f7330 2024-02-23T00:50:58.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.991+0000 7fd681453700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd67c1167d0 con 0x7fd67c0f89a0 2024-02-23T00:50:58.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.991+0000 7fd67a7fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd67c0f89a0 0x7fd67c0f9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:58.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.991+0000 7fd67a7fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd67c0f89a0 0x7fd67c0f9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:47936/0 (socket says 172.21.15.47:47936) 2024-02-23T00:50:58.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.991+0000 7fd67a7fc700 1 -- 172.21.15.47:0/2317769983 learned_addr learned my addr 172.21.15.47:0/2317769983 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:50:58.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.991+0000 7fd67b7fe700 1 --2- 172.21.15.47:0/2317769983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd67c0f9270 0x7fd67c0fa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:58.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.991+0000 7fd67affd700 1 --2- 172.21.15.47:0/2317769983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd67c0f7330 0x7fd67c0f99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:58.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.992+0000 7fd67a7fc700 1 -- 172.21.15.47:0/2317769983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd67c0f7330 msgr2=0x7fd67c0f99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:58.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.992+0000 7fd67a7fc700 1 --2- 172.21.15.47:0/2317769983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd67c0f7330 0x7fd67c0f99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:58.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.992+0000 7fd67a7fc700 1 -- 172.21.15.47:0/2317769983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd67c0f9270 msgr2=0x7fd67c0fa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:58.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.992+0000 7fd67a7fc700 1 --2- 172.21.15.47:0/2317769983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd67c0f9270 0x7fd67c0fa380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:58.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.992+0000 7fd67a7fc700 1 -- 172.21.15.47:0/2317769983 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6740097c0 con 0x7fd67c0f89a0 2024-02-23T00:50:58.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.992+0000 7fd67b7fe700 1 --2- 172.21.15.47:0/2317769983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd67c0f9270 0x7fd67c0fa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:50:59.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.992+0000 7fd67a7fc700 1 --2- 172.21.15.47:0/2317769983 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd67c0f89a0 0x7fd67c0f9e90 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 rx=0x7fd66c00e970 tx=0x7fd66c00b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:59.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.992+0000 7fd663fff700 10 client.?.objecter ms_handle_connect 0x7fd67c0f89a0 2024-02-23T00:50:59.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.992+0000 7fd663fff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:50:59.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.992+0000 7fd663fff700 1 -- 172.21.15.47:0/2317769983 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c004660 con 0x7fd67c0f89a0 2024-02-23T00:50:59.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.992+0000 7fd681453700 1 -- 172.21.15.47:0/2317769983 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd670005040 con 0x7fd67c0f89a0 2024-02-23T00:50:59.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.992+0000 7fd663fff700 1 -- 172.21.15.47:0/2317769983 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fd66c00f040 con 0x7fd67c0f89a0 2024-02-23T00:50:59.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.992+0000 7fd663fff700 1 -- 172.21.15.47:0/2317769983 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c0146f0 con 0x7fd67c0f89a0 2024-02-23T00:50:59.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.992+0000 7fd681453700 10 client.41609.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:50:59.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.992+0000 7fd681453700 1 -- 172.21.15.47:0/2317769983 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd67c116ee0 con 0x7fd67c0f89a0 2024-02-23T00:50:59.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.992+0000 7fd681453700 1 librados: init done 2024-02-23T00:50:59.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.993+0000 7fd681453700 1 -- 172.21.15.47:0/2317769983 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd67c05ee80 con 0x7fd67c0f89a0 2024-02-23T00:50:59.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.993+0000 7fd663fff700 1 -- 172.21.15.47:0/2317769983 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fd66c007b40 con 0x7fd67c0f89a0 2024-02-23T00:50:59.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.994+0000 7fd663fff700 1 --2- 172.21.15.47:0/2317769983 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd664062cc0 0x7fd664065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:50:59.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.994+0000 7fd663fff700 1 -- 172.21.15.47:0/2317769983 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(529..529 src has 463..529) v4 ==== 9220+0+0 (secure 0 0 0) 0x7fd66c07d150 con 0x7fd67c0f89a0 2024-02-23T00:50:59.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.994+0000 7fd663fff700 10 client.41609.objecter ms_dispatch 0x7fd67c001550 osd_map(529..529 src has 463..529) v4 2024-02-23T00:50:59.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.994+0000 7fd663fff700 3 client.41609.objecter handle_osd_map got epochs [529,529] > 0 2024-02-23T00:50:59.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.994+0000 7fd663fff700 3 client.41609.objecter handle_osd_map decoding full epoch 529 2024-02-23T00:50:59.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.994+0000 7fd67affd700 1 --2- 172.21.15.47:0/2317769983 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd664062cc0 0x7fd664065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:50:59.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.994+0000 7fd663fff700 20 client.41609.objecter dump_active .. 0 homeless 2024-02-23T00:50:59.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.994+0000 7fd663fff700 10 client.41609.objecter ms_handle_connect 0x7fd664062cc0 2024-02-23T00:50:59.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.994+0000 7fd67affd700 1 --2- 172.21.15.47:0/2317769983 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd664062cc0 0x7fd664065180 secure :-1 s=READY pgs=3110 cs=0 l=1 rev1=1 rx=0x7fd670004ee0 tx=0x7fd67000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:50:59.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:58.998+0000 7fd663fff700 1 -- 172.21.15.47:0/2317769983 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fd66c04c810 con 0x7fd67c0f89a0 2024-02-23T00:50:59.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:59.184+0000 7fd681453700 1 -- 172.21.15.47:0/2317769983 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd reweight", "id": 7, "weight": 1.0} v 0) v1 -- 0x7fd67c0fae80 con 0x7fd67c0f89a0 2024-02-23T00:50:59.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:59.349+0000 7fd663fff700 1 -- 172.21.15.47:0/2317769983 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 7, "weight": 1.0}]=0 reweighted osd.7 to 1 (10000) v530) v1 ==== 113+0+0 (secure 0 0 0) 0x7fd66c019070 con 0x7fd67c0f89a0 2024-02-23T00:50:59.353 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (22349 ms) 2024-02-23T00:50:59.353 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-02-23T00:50:59.353 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (3017 ms) 2024-02-23T00:50:59.353 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-02-23T00:50:59.353 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: require_osd_release = pacific 2024-02-23T00:50:59.353 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (2631 ms) 2024-02-23T00:50:59.353 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-02-23T00:50:59.353 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (3026 ms) 2024-02-23T00:50:59.353 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-02-23T00:50:59.353 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: require_osd_release = pacific 2024-02-23T00:50:59.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (3019 ms) 2024-02-23T00:50:59.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-02-23T00:50:59.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: require_osd_release = pacific 2024-02-23T00:50:59.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (4035 ms) 2024-02-23T00:50:59.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-02-23T00:50:59.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: require_osd_release = pacific 2024-02-23T00:50:59.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (38117 ms) 2024-02-23T00:50:59.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-02-23T00:50:59.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: require_osd_release = pacific 2024-02-23T00:50:59.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (15985 ms) 2024-02-23T00:50:59.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-02-23T00:50:59.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: require_osd_release = pacific 2024-02-23T00:50:59.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (3842 ms) 2024-02-23T00:50:59.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-02-23T00:50:59.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: require_osd_release = pacific 2024-02-23T00:50:59.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (3018 ms) 2024-02-23T00:50:59.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-02-23T00:50:59.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: require_osd_release = pacific 2024-02-23T00:50:59.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (24172 ms) 2024-02-23T00:50:59.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-02-23T00:50:59.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: require_osd_release = pacific 2024-02-23T00:50:59.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (14817 ms) 2024-02-23T00:50:59.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-02-23T00:50:59.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: require_osd_release = pacific 2024-02-23T00:50:59.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (5017 ms) 2024-02-23T00:50:59.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-02-23T00:50:59.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: require_osd_release = pacific 2024-02-23T00:50:59.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: waiting for scrubs... 2024-02-23T00:50:59.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: done waiting 2024-02-23T00:50:59.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (24193 ms) 2024-02-23T00:50:59.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-02-23T00:50:59.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: require_osd_release = pacific 2024-02-23T00:50:59.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (9841 ms) 2024-02-23T00:50:59.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-02-23T00:50:59.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: require_osd_release = pacific 2024-02-23T00:50:59.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (9037 ms) 2024-02-23T00:50:59.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-02-23T00:50:59.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: require_osd_release = pacific 2024-02-23T00:50:59.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (8937 ms) 2024-02-23T00:50:59.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-02-23T00:50:59.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: require_osd_release = pacific 2024-02-23T00:50:59.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (9068 ms) 2024-02-23T00:50:59.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-02-23T00:50:59.357 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: require_osd_release = pacific 2024-02-23T00:50:59.357 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (6043 ms) 2024-02-23T00:50:59.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:59.353+0000 7fd681453700 10 librados: watch_flush enter 2024-02-23T00:50:59.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:59.355+0000 7fd681453700 10 librados: watch_flush exit 2024-02-23T00:50:59.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:59.355+0000 7fd681453700 20 client.41609.objecter shutdown clearing up homeless session... 2024-02-23T00:50:59.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:59.355+0000 7fd681453700 10 client.41609.objecter successfully canceled tick 2024-02-23T00:50:59.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:59.356+0000 7fd681453700 1 -- 172.21.15.47:0/2317769983 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd664062cc0 msgr2=0x7fd664065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:59.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:59.356+0000 7fd681453700 1 --2- 172.21.15.47:0/2317769983 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd664062cc0 0x7fd664065180 secure :-1 s=READY pgs=3110 cs=0 l=1 rev1=1 rx=0x7fd670004ee0 tx=0x7fd67000b040).stop 2024-02-23T00:50:59.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:59.356+0000 7fd681453700 1 -- 172.21.15.47:0/2317769983 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd67c0f89a0 msgr2=0x7fd67c0f9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:50:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:59.356+0000 7fd681453700 1 --2- 172.21.15.47:0/2317769983 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd67c0f89a0 0x7fd67c0f9e90 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 rx=0x7fd66c00e970 tx=0x7fd66c00b040).stop 2024-02-23T00:50:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:59.357+0000 7fd681453700 1 -- 172.21.15.47:0/2317769983 shutdown_connections 2024-02-23T00:50:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:59.357+0000 7fd681453700 1 --2- 172.21.15.47:0/2317769983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd67c0f9270 0x7fd67c0fa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:59.357+0000 7fd681453700 1 --2- 172.21.15.47:0/2317769983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd67c0f7330 0x7fd67c0f99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:59.357+0000 7fd681453700 1 --2- 172.21.15.47:0/2317769983 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd664062cc0 0x7fd664065180 secure :-1 s=CLOSED pgs=3110 cs=0 l=1 rev1=1 rx=0x7fd670004ee0 tx=0x7fd67000b040).stop 2024-02-23T00:50:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:59.357+0000 7fd681453700 1 --2- 172.21.15.47:0/2317769983 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd67c0f89a0 0x7fd67c0f9e90 unknown :-1 s=CLOSED pgs=2722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:50:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:59.357+0000 7fd681453700 1 -- 172.21.15.47:0/2317769983 >> 172.21.15.47:0/2317769983 conn(0x7fd67c0f26f0 msgr2=0x7fd67c0fc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:50:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:59.359+0000 7fd681453700 1 -- 172.21.15.47:0/2317769983 shutdown_connections 2024-02-23T00:50:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:59.359+0000 7fd681453700 1 -- 172.21.15.47:0/2317769983 wait complete. 2024-02-23T00:50:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:50:59.359+0000 7fd681453700 1 librados: shutdown 2024-02-23T00:50:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:reweighted osd.7 to 1 (10000) 2024-02-23T00:50:59.519 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:58.154791+0000 osd.7 (osd.7) 259 : cluster [DBG] 62.1f scrub starts 2024-02-23T00:50:59.519 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:58.157134+0000 osd.7 (osd.7) 260 : cluster [DBG] 62.1f scrub ok 2024-02-23T00:50:59.519 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:58.339258+0000 mon.a (mon.0) 2887 : cluster [DBG] osdmap e529: 8 total, 8 up, 7 in 2024-02-23T00:50:59.519 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:58.373046+0000 mon.a (mon.0) 2888 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:50:59.519 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:58.373421+0000 mon.a (mon.0) 2889 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-86"}]: dispatch 2024-02-23T00:50:59.519 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:58.790798+0000 osd.5 (osd.5) 238 : cluster [DBG] 62.1d scrub starts 2024-02-23T00:50:59.519 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:58.794266+0000 osd.5 (osd.5) 239 : cluster [DBG] 62.1d scrub ok 2024-02-23T00:50:59.519 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:59.185667+0000 mon.c (mon.2) 322 : audit [INF] from='client.? 172.21.15.47:0/2317769983' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-23T00:50:59.520 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:59.185970+0000 mon.a (mon.0) 2890 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-23T00:50:59.520 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:59.199102+0000 osd.6 (osd.6) 205 : cluster [DBG] 2.7 scrub starts 2024-02-23T00:50:59.520 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:50:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:59.199894+0000 osd.6 (osd.6) 206 : cluster [DBG] 2.7 scrub ok 2024-02-23T00:50:59.520 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:58.154791+0000 osd.7 (osd.7) 259 : cluster [DBG] 62.1f scrub starts 2024-02-23T00:50:59.520 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:58.157134+0000 osd.7 (osd.7) 260 : cluster [DBG] 62.1f scrub ok 2024-02-23T00:50:59.520 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:58.339258+0000 mon.a (mon.0) 2887 : cluster [DBG] osdmap e529: 8 total, 8 up, 7 in 2024-02-23T00:50:59.520 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:58.373046+0000 mon.a (mon.0) 2888 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:50:59.520 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:58.373421+0000 mon.a (mon.0) 2889 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-86"}]: dispatch 2024-02-23T00:50:59.520 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:58.790798+0000 osd.5 (osd.5) 238 : cluster [DBG] 62.1d scrub starts 2024-02-23T00:50:59.520 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:58.794266+0000 osd.5 (osd.5) 239 : cluster [DBG] 62.1d scrub ok 2024-02-23T00:50:59.520 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:59.185667+0000 mon.c (mon.2) 322 : audit [INF] from='client.? 172.21.15.47:0/2317769983' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-23T00:50:59.521 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:59.185970+0000 mon.a (mon.0) 2890 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-23T00:50:59.521 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:59.199102+0000 osd.6 (osd.6) 205 : cluster [DBG] 2.7 scrub starts 2024-02-23T00:50:59.521 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:50:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:59.199894+0000 osd.6 (osd.6) 206 : cluster [DBG] 2.7 scrub ok 2024-02-23T00:50:59.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:58.154791+0000 osd.7 (osd.7) 259 : cluster [DBG] 62.1f scrub starts 2024-02-23T00:50:59.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:58.157134+0000 osd.7 (osd.7) 260 : cluster [DBG] 62.1f scrub ok 2024-02-23T00:50:59.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:58.339258+0000 mon.a (mon.0) 2887 : cluster [DBG] osdmap e529: 8 total, 8 up, 7 in 2024-02-23T00:50:59.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:58.373046+0000 mon.a (mon.0) 2888 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:50:59.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:58.373421+0000 mon.a (mon.0) 2889 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-86"}]: dispatch 2024-02-23T00:50:59.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:58.790798+0000 osd.5 (osd.5) 238 : cluster [DBG] 62.1d scrub starts 2024-02-23T00:50:59.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:58.794266+0000 osd.5 (osd.5) 239 : cluster [DBG] 62.1d scrub ok 2024-02-23T00:50:59.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:59.185667+0000 mon.c (mon.2) 322 : audit [INF] from='client.? 172.21.15.47:0/2317769983' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-23T00:50:59.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:59.185970+0000 mon.a (mon.0) 2890 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-23T00:50:59.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:59.199102+0000 osd.6 (osd.6) 205 : cluster [DBG] 2.7 scrub starts 2024-02-23T00:50:59.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:50:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:59.199894+0000 osd.6 (osd.6) 206 : cluster [DBG] 2.7 scrub ok 2024-02-23T00:50:59.705 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-23T00:51:00.204 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-02-23T00:51:00.204 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph pg dump --format=json 2024-02-23T00:51:00.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:59.129202+0000 osd.7 (osd.7) 261 : cluster [DBG] 62.b scrub starts 2024-02-23T00:51:00.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:59.132666+0000 osd.7 (osd.7) 262 : cluster [DBG] 62.b scrub ok 2024-02-23T00:51:00.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:59.336047+0000 mgr.y (mgr.14517) 488 : cluster [DBG] pgmap v798: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-02-23T00:51:00.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:50:59.349566+0000 mon.a (mon.0) 2891 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-02-23T00:51:00.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:50:59.349650+0000 mon.a (mon.0) 2892 : cluster [DBG] osdmap e530: 8 total, 8 up, 7 in 2024-02-23T00:51:00.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:00.230788+0000 osd.3 (osd.3) 250 : cluster [DBG] 18.9 scrub starts 2024-02-23T00:51:00.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:00.232258+0000 osd.3 (osd.3) 251 : cluster [DBG] 18.9 scrub ok 2024-02-23T00:51:00.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:00.237084+0000 mon.a (mon.0) 2893 : cluster [DBG] osdmap e531: 8 total, 8 up, 7 in 2024-02-23T00:51:00.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:00.244798+0000 mon.a (mon.0) 2894 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-88","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:51:00.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:59.129202+0000 osd.7 (osd.7) 261 : cluster [DBG] 62.b scrub starts 2024-02-23T00:51:00.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:59.132666+0000 osd.7 (osd.7) 262 : cluster [DBG] 62.b scrub ok 2024-02-23T00:51:00.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:59.336047+0000 mgr.y (mgr.14517) 488 : cluster [DBG] pgmap v798: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-02-23T00:51:00.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:50:59.349566+0000 mon.a (mon.0) 2891 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-02-23T00:51:00.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:50:59.349650+0000 mon.a (mon.0) 2892 : cluster [DBG] osdmap e530: 8 total, 8 up, 7 in 2024-02-23T00:51:00.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:00.230788+0000 osd.3 (osd.3) 250 : cluster [DBG] 18.9 scrub starts 2024-02-23T00:51:00.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:00.232258+0000 osd.3 (osd.3) 251 : cluster [DBG] 18.9 scrub ok 2024-02-23T00:51:00.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:00.237084+0000 mon.a (mon.0) 2893 : cluster [DBG] osdmap e531: 8 total, 8 up, 7 in 2024-02-23T00:51:00.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:00.244798+0000 mon.a (mon.0) 2894 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-88","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:51:00.575 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:51:00.579 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:00.579 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:51:00.680 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:51:00.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:59.129202+0000 osd.7 (osd.7) 261 : cluster [DBG] 62.b scrub starts 2024-02-23T00:51:00.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:59.132666+0000 osd.7 (osd.7) 262 : cluster [DBG] 62.b scrub ok 2024-02-23T00:51:00.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:59.336047+0000 mgr.y (mgr.14517) 488 : cluster [DBG] pgmap v798: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-02-23T00:51:00.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:50:59.349566+0000 mon.a (mon.0) 2891 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-02-23T00:51:00.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:50:59.349650+0000 mon.a (mon.0) 2892 : cluster [DBG] osdmap e530: 8 total, 8 up, 7 in 2024-02-23T00:51:00.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:00.230788+0000 osd.3 (osd.3) 250 : cluster [DBG] 18.9 scrub starts 2024-02-23T00:51:00.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:00.232258+0000 osd.3 (osd.3) 251 : cluster [DBG] 18.9 scrub ok 2024-02-23T00:51:00.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:00.237084+0000 mon.a (mon.0) 2893 : cluster [DBG] osdmap e531: 8 total, 8 up, 7 in 2024-02-23T00:51:00.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:00.244798+0000 mon.a (mon.0) 2894 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-88","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:51:00.706 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:51:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:51:00] "GET /metrics HTTP/1.1" 200 189660 "" "Prometheus/2.33.4" 2024-02-23T00:51:00.708 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:01.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.621+0000 7f71c45c3700 1 -- 172.21.15.47:0/3348013658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71bc1021a0 msgr2=0x7f71bc104590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:01.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.621+0000 7f71c45c3700 1 --2- 172.21.15.47:0/3348013658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71bc1021a0 0x7f71bc104590 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 rx=0x7f71b4009c30 tx=0x7f71b4004c50).stop 2024-02-23T00:51:01.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.621+0000 7f71c45c3700 1 -- 172.21.15.47:0/3348013658 shutdown_connections 2024-02-23T00:51:01.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.621+0000 7f71c45c3700 1 --2- 172.21.15.47:0/3348013658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71bc1021a0 0x7f71bc104590 unknown :-1 s=CLOSED pgs=2567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:01.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.621+0000 7f71c45c3700 1 --2- 172.21.15.47:0/3348013658 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f71bc0f8c40 0x7f71bc0f9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:01.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.621+0000 7f71c45c3700 1 --2- 172.21.15.47:0/3348013658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71bc0f9510 0x7f71bc101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:01.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.621+0000 7f71c45c3700 1 -- 172.21.15.47:0/3348013658 >> 172.21.15.47:0/3348013658 conn(0x7f71bc0f26f0 msgr2=0x7f71bc0f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:01.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.621+0000 7f71c45c3700 1 -- 172.21.15.47:0/3348013658 shutdown_connections 2024-02-23T00:51:01.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.621+0000 7f71c45c3700 1 -- 172.21.15.47:0/3348013658 wait complete. 2024-02-23T00:51:01.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.621+0000 7f71c45c3700 1 librados: starting msgr at 2024-02-23T00:51:01.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.621+0000 7f71c45c3700 1 librados: starting objecter 2024-02-23T00:51:01.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.622+0000 7f71c45c3700 1 Processor -- start 2024-02-23T00:51:01.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.622+0000 7f71c45c3700 1 -- start start 2024-02-23T00:51:01.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.622+0000 7f71c45c3700 1 librados: setting wanted keys 2024-02-23T00:51:01.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.622+0000 7f71c45c3700 1 librados: calling monclient init 2024-02-23T00:51:01.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.622+0000 7f71c45c3700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f71bc0f8c40 0x7f71bc10dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:01.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.622+0000 7f71c45c3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71bc0f9510 0x7f71bc10e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:01.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.622+0000 7f71c45c3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71bc1021a0 0x7f71bc107f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:01.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.622+0000 7f71c45c3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc108420 con 0x7f71bc1021a0 2024-02-23T00:51:01.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.622+0000 7f71c45c3700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc108590 con 0x7f71bc0f9510 2024-02-23T00:51:01.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.622+0000 7f71c45c3700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc108750 con 0x7f71bc0f8c40 2024-02-23T00:51:01.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.622+0000 7f71c235f700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f71bc0f8c40 0x7f71bc10dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:01.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.622+0000 7f71c2b60700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71bc1021a0 0x7f71bc107f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:01.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.622+0000 7f71c2b60700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71bc1021a0 0x7f71bc107f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36188/0 (socket says 172.21.15.47:36188) 2024-02-23T00:51:01.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.622+0000 7f71c235f700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f71bc0f8c40 0x7f71bc10dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:47948/0 (socket says 172.21.15.47:47948) 2024-02-23T00:51:01.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.622+0000 7f71c2b60700 1 -- 172.21.15.47:0/3648999758 learned_addr learned my addr 172.21.15.47:0/3648999758 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:51:01.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.622+0000 7f71c1b5e700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71bc0f9510 0x7f71bc10e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:01.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.622+0000 7f71c2b60700 1 -- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f71bc0f8c40 msgr2=0x7f71bc10dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:01.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.622+0000 7f71c2b60700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f71bc0f8c40 0x7f71bc10dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:01.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.622+0000 7f71c2b60700 1 -- 172.21.15.47:0/3648999758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71bc0f9510 msgr2=0x7f71bc10e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:01.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.622+0000 7f71c2b60700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71bc0f9510 0x7f71bc10e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:01.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.622+0000 7f71c2b60700 1 -- 172.21.15.47:0/3648999758 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b8005040 con 0x7f71bc1021a0 2024-02-23T00:51:01.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.622+0000 7f71c235f700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f71bc0f8c40 0x7f71bc10dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:51:01.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.622+0000 7f71c2b60700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71bc1021a0 0x7f71bc107f30 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 rx=0x7f71b4004890 tx=0x7f71b4004c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:01.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.622+0000 7f71b37fe700 10 client.?.objecter ms_handle_connect 0x7f71bc1021a0 2024-02-23T00:51:01.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.623+0000 7f71b37fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:51:01.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.623+0000 7f71b37fe700 1 -- 172.21.15.47:0/3648999758 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b400e040 con 0x7f71bc1021a0 2024-02-23T00:51:01.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.623+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b40097c0 con 0x7f71bc1021a0 2024-02-23T00:51:01.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.623+0000 7f71b37fe700 1 -- 172.21.15.47:0/3648999758 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f71b4018040 con 0x7f71bc1021a0 2024-02-23T00:51:01.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.623+0000 7f71b37fe700 1 -- 172.21.15.47:0/3648999758 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b4013dd0 con 0x7f71bc1021a0 2024-02-23T00:51:01.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.623+0000 7f71c45c3700 10 client.31290.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:51:01.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.623+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71bc108bf0 con 0x7f71bc1021a0 2024-02-23T00:51:01.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.623+0000 7f71c45c3700 1 librados: init done 2024-02-23T00:51:01.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.623+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f71bc05f990 con 0x7f71bc1021a0 2024-02-23T00:51:01.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.624+0000 7f71b37fe700 1 -- 172.21.15.47:0/3648999758 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f71b40135b0 con 0x7f71bc1021a0 2024-02-23T00:51:01.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.625+0000 7f71b37fe700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f71a8062cc0 0x7f71a8065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:01.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.625+0000 7f71b37fe700 1 -- 172.21.15.47:0/3648999758 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(532..532 src has 463..532) v4 ==== 9220+0+0 (secure 0 0 0) 0x7f71b407eb10 con 0x7f71bc1021a0 2024-02-23T00:51:01.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.625+0000 7f71b37fe700 10 client.31290.objecter ms_dispatch 0x7f71bc001550 osd_map(532..532 src has 463..532) v4 2024-02-23T00:51:01.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.625+0000 7f71b37fe700 3 client.31290.objecter handle_osd_map got epochs [532,532] > 0 2024-02-23T00:51:01.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.625+0000 7f71b37fe700 3 client.31290.objecter handle_osd_map decoding full epoch 532 2024-02-23T00:51:01.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.625+0000 7f71c235f700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f71a8062cc0 0x7f71a8065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:01.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.625+0000 7f71b37fe700 20 client.31290.objecter dump_active .. 0 homeless 2024-02-23T00:51:01.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.625+0000 7f71b37fe700 1 -- 172.21.15.47:0/3648999758 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v532) v1 ==== 54+0+16 (secure 0 0 0) 0x7f71b407f0a0 con 0x7f71bc1021a0 2024-02-23T00:51:01.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.625+0000 7f71b37fe700 10 client.31290.objecter ms_handle_connect 0x7f71a8062cc0 2024-02-23T00:51:01.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.625+0000 7f71c45c3700 10 client.31290.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:01.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.625+0000 7f71c235f700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f71a8062cc0 0x7f71a8065180 secure :-1 s=READY pgs=3111 cs=0 l=1 rev1=1 rx=0x7f71b8004c80 tx=0x7f71b800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:01.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.625+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_assign -1 1 2024-02-23T00:51:01.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.625+0000 7f71c45c3700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f71bc0fc140 0x7f71bc0fc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:01.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.625+0000 7f71c45c3700 20 client.31290.objecter _get_session s=0x7f71bc05ee80 osd=0 3 2024-02-23T00:51:01.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.625+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc05ee80 osd=0 3 2024-02-23T00:51:01.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.625+0000 7f71c45c3700 20 client.31290.objecter _get_session s=0x7f71bc05ee80 osd=0 3 2024-02-23T00:51:01.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.625+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_remove -1 1 2024-02-23T00:51:01.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.625+0000 7f71c45c3700 20 client.31290.objecter get_session s=0x7f71bc05ee80 osd=0 3 2024-02-23T00:51:01.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.625+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_assign 0 1 2024-02-23T00:51:01.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.625+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc05ee80 osd=0 4 2024-02-23T00:51:01.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.625+0000 7f71c45c3700 10 client.31290.objecter _send_command 1 2024-02-23T00:51:01.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.625+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f71bc106eb0 con 0x7f71bc0fc140 2024-02-23T00:51:01.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.625+0000 7f71c1b5e700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f71bc0fc140 0x7f71bc0fc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:01.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.625+0000 7f71b37fe700 10 client.31290.objecter ms_handle_connect 0x7f71bc0fc140 2024-02-23T00:51:01.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.625+0000 7f71c1b5e700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f71bc0fc140 0x7f71bc0fc520 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:01.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.626+0000 7f71b37fe700 1 -- 172.21.15.47:0/3648999758 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f71bc106eb0 con 0x7f71bc0fc140 2024-02-23T00:51:01.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.626+0000 7f71b37fe700 10 client.31290.objecter ms_dispatch 0x7f71bc001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:51:01.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.626+0000 7f71b37fe700 10 client.31290.objecter _finish_command 1 = system:0 2024-02-23T00:51:01.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.626+0000 7f71b37fe700 20 client.31290.objecter put_session s=0x7f71bc05ee80 osd=0 4 2024-02-23T00:51:01.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.626+0000 7f71b37fe700 15 client.31290.objecter _session_command_op_remove 0 1 2024-02-23T00:51:01.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.641+0000 7f71c45c3700 10 client.31290.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:51:01.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.641+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_assign -1 2 2024-02-23T00:51:01.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.641+0000 7f71c45c3700 20 client.31290.objecter _get_session s=0x7f71bc05ee80 osd=0 3 2024-02-23T00:51:01.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.641+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc05ee80 osd=0 3 2024-02-23T00:51:01.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.641+0000 7f71c45c3700 20 client.31290.objecter _get_session s=0x7f71bc05ee80 osd=0 3 2024-02-23T00:51:01.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.641+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_remove -1 2 2024-02-23T00:51:01.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.641+0000 7f71c45c3700 20 client.31290.objecter get_session s=0x7f71bc05ee80 osd=0 3 2024-02-23T00:51:01.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.641+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_assign 0 2 2024-02-23T00:51:01.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.641+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc05ee80 osd=0 4 2024-02-23T00:51:01.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.641+0000 7f71c45c3700 10 client.31290.objecter _send_command 2 2024-02-23T00:51:01.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.641+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f71bc0273c0 con 0x7f71bc0fc140 2024-02-23T00:51:01.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.642+0000 7f71b37fe700 1 -- 172.21.15.47:0/3648999758 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f71bc0273c0 con 0x7f71bc0fc140 2024-02-23T00:51:01.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.643+0000 7f71b37fe700 10 client.31290.objecter ms_dispatch 0x7f71bc001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:51:01.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.643+0000 7f71b37fe700 10 client.31290.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:51:01.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.643+0000 7f71b37fe700 20 client.31290.objecter put_session s=0x7f71bc05ee80 osd=0 4 2024-02-23T00:51:01.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.643+0000 7f71b37fe700 15 client.31290.objecter _session_command_op_remove 0 2 2024-02-23T00:51:01.645 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:51:01.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.643+0000 7f71c45c3700 10 client.31290.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:01.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.643+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_assign -1 3 2024-02-23T00:51:01.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.643+0000 7f71c45c3700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f71bc0598f0 0x7f71bc059d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.643+0000 7f71c45c3700 20 client.31290.objecter _get_session s=0x7f71bc028710 osd=1 3 2024-02-23T00:51:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.643+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc028710 osd=1 3 2024-02-23T00:51:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.643+0000 7f71c45c3700 20 client.31290.objecter _get_session s=0x7f71bc028710 osd=1 3 2024-02-23T00:51:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.643+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_remove -1 3 2024-02-23T00:51:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.643+0000 7f71c45c3700 20 client.31290.objecter get_session s=0x7f71bc028710 osd=1 3 2024-02-23T00:51:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.643+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_assign 1 3 2024-02-23T00:51:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.643+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc028710 osd=1 4 2024-02-23T00:51:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.643+0000 7f71c45c3700 10 client.31290.objecter _send_command 3 2024-02-23T00:51:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.643+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f71bc11b340 con 0x7f71bc0598f0 2024-02-23T00:51:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.643+0000 7f71c235f700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f71bc0598f0 0x7f71bc059d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.643+0000 7f71b37fe700 10 client.31290.objecter ms_handle_connect 0x7f71bc0598f0 2024-02-23T00:51:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.643+0000 7f71c235f700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f71bc0598f0 0x7f71bc059d10 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.644+0000 7f71b37fe700 1 -- 172.21.15.47:0/3648999758 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f71bc11b340 con 0x7f71bc0598f0 2024-02-23T00:51:01.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.644+0000 7f71b37fe700 10 client.31290.objecter ms_dispatch 0x7f71bc001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:51:01.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.644+0000 7f71b37fe700 10 client.31290.objecter _finish_command 3 = system:0 2024-02-23T00:51:01.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.644+0000 7f71b37fe700 20 client.31290.objecter put_session s=0x7f71bc028710 osd=1 4 2024-02-23T00:51:01.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.644+0000 7f71b37fe700 15 client.31290.objecter _session_command_op_remove 1 3 2024-02-23T00:51:01.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.658+0000 7f71c45c3700 10 client.31290.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:51:01.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.658+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_assign -1 4 2024-02-23T00:51:01.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.658+0000 7f71c45c3700 20 client.31290.objecter _get_session s=0x7f71bc028710 osd=1 3 2024-02-23T00:51:01.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.658+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc028710 osd=1 3 2024-02-23T00:51:01.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.658+0000 7f71c45c3700 20 client.31290.objecter _get_session s=0x7f71bc028710 osd=1 3 2024-02-23T00:51:01.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.658+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_remove -1 4 2024-02-23T00:51:01.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.658+0000 7f71c45c3700 20 client.31290.objecter get_session s=0x7f71bc028710 osd=1 3 2024-02-23T00:51:01.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.658+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_assign 1 4 2024-02-23T00:51:01.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.658+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc028710 osd=1 4 2024-02-23T00:51:01.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.658+0000 7f71c45c3700 10 client.31290.objecter _send_command 4 2024-02-23T00:51:01.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.658+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f71bc059360 con 0x7f71bc0598f0 2024-02-23T00:51:01.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.660+0000 7f71b37fe700 1 -- 172.21.15.47:0/3648999758 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f71bc059360 con 0x7f71bc0598f0 2024-02-23T00:51:01.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.660+0000 7f71b37fe700 10 client.31290.objecter ms_dispatch 0x7f71bc001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:51:01.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.660+0000 7f71b37fe700 10 client.31290.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:51:01.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.660+0000 7f71b37fe700 20 client.31290.objecter put_session s=0x7f71bc028710 osd=1 4 2024-02-23T00:51:01.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.660+0000 7f71b37fe700 15 client.31290.objecter _session_command_op_remove 1 4 2024-02-23T00:51:01.663 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:51:01.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.660+0000 7f71c45c3700 10 client.31290.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:01.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.660+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_assign -1 5 2024-02-23T00:51:01.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.660+0000 7f71c45c3700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f71bc028270 0x7f71bc119d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:01.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.660+0000 7f71c45c3700 20 client.31290.objecter _get_session s=0x7f71bc059770 osd=2 3 2024-02-23T00:51:01.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.660+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc059770 osd=2 3 2024-02-23T00:51:01.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.660+0000 7f71c45c3700 20 client.31290.objecter _get_session s=0x7f71bc059770 osd=2 3 2024-02-23T00:51:01.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.660+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_remove -1 5 2024-02-23T00:51:01.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.660+0000 7f71c45c3700 20 client.31290.objecter get_session s=0x7f71bc059770 osd=2 3 2024-02-23T00:51:01.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.660+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_assign 2 5 2024-02-23T00:51:01.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.660+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc059770 osd=2 4 2024-02-23T00:51:01.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.660+0000 7f71c45c3700 10 client.31290.objecter _send_command 5 2024-02-23T00:51:01.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.660+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f71bc11a250 con 0x7f71bc028270 2024-02-23T00:51:01.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.660+0000 7f71c1b5e700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f71bc028270 0x7f71bc119d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:01.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.661+0000 7f71b37fe700 10 client.31290.objecter ms_handle_connect 0x7f71bc028270 2024-02-23T00:51:01.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.661+0000 7f71c1b5e700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f71bc028270 0x7f71bc119d60 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:01.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.661+0000 7f71b37fe700 1 -- 172.21.15.47:0/3648999758 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f71bc11a250 con 0x7f71bc028270 2024-02-23T00:51:01.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.661+0000 7f71b37fe700 10 client.31290.objecter ms_dispatch 0x7f71bc001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:51:01.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.661+0000 7f71b37fe700 10 client.31290.objecter _finish_command 5 = system:0 2024-02-23T00:51:01.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.661+0000 7f71b37fe700 20 client.31290.objecter put_session s=0x7f71bc059770 osd=2 4 2024-02-23T00:51:01.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.661+0000 7f71b37fe700 15 client.31290.objecter _session_command_op_remove 2 5 2024-02-23T00:51:01.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.676+0000 7f71c45c3700 10 client.31290.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:51:01.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.676+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_assign -1 6 2024-02-23T00:51:01.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.676+0000 7f71c45c3700 20 client.31290.objecter _get_session s=0x7f71bc059770 osd=2 3 2024-02-23T00:51:01.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.676+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc059770 osd=2 3 2024-02-23T00:51:01.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.676+0000 7f71c45c3700 20 client.31290.objecter _get_session s=0x7f71bc059770 osd=2 3 2024-02-23T00:51:01.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.676+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_remove -1 6 2024-02-23T00:51:01.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.676+0000 7f71c45c3700 20 client.31290.objecter get_session s=0x7f71bc059770 osd=2 3 2024-02-23T00:51:01.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.676+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_assign 2 6 2024-02-23T00:51:01.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.676+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc059770 osd=2 4 2024-02-23T00:51:01.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.676+0000 7f71c45c3700 10 client.31290.objecter _send_command 6 2024-02-23T00:51:01.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.676+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f71bc0277d0 con 0x7f71bc028270 2024-02-23T00:51:01.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.677+0000 7f71b37fe700 1 -- 172.21.15.47:0/3648999758 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f71bc0277d0 con 0x7f71bc028270 2024-02-23T00:51:01.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.677+0000 7f71b37fe700 10 client.31290.objecter ms_dispatch 0x7f71bc001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:51:01.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.677+0000 7f71b37fe700 10 client.31290.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:51:01.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.677+0000 7f71b37fe700 20 client.31290.objecter put_session s=0x7f71bc059770 osd=2 4 2024-02-23T00:51:01.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.677+0000 7f71b37fe700 15 client.31290.objecter _session_command_op_remove 2 6 2024-02-23T00:51:01.680 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:51:01.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.677+0000 7f71c45c3700 10 client.31290.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:01.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.677+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_assign -1 7 2024-02-23T00:51:01.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.677+0000 7f71c45c3700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f71bc11d4c0 0x7f71bc11f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:01.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.677+0000 7f71c45c3700 20 client.31290.objecter _get_session s=0x7f71bc027be0 osd=3 3 2024-02-23T00:51:01.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.677+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc027be0 osd=3 3 2024-02-23T00:51:01.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.677+0000 7f71c45c3700 20 client.31290.objecter _get_session s=0x7f71bc027be0 osd=3 3 2024-02-23T00:51:01.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.677+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_remove -1 7 2024-02-23T00:51:01.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.677+0000 7f71c45c3700 20 client.31290.objecter get_session s=0x7f71bc027be0 osd=3 3 2024-02-23T00:51:01.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.677+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_assign 3 7 2024-02-23T00:51:01.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.677+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc027be0 osd=3 4 2024-02-23T00:51:01.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.677+0000 7f71c45c3700 10 client.31290.objecter _send_command 7 2024-02-23T00:51:01.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.677+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f71bc122500 con 0x7f71bc11d4c0 2024-02-23T00:51:01.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.677+0000 7f71c2b60700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f71bc11d4c0 0x7f71bc11f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:01.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.678+0000 7f71b37fe700 10 client.31290.objecter ms_handle_connect 0x7f71bc11d4c0 2024-02-23T00:51:01.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.678+0000 7f71c2b60700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f71bc11d4c0 0x7f71bc11f8b0 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:01.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.678+0000 7f71b37fe700 1 -- 172.21.15.47:0/3648999758 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f71bc122500 con 0x7f71bc11d4c0 2024-02-23T00:51:01.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.678+0000 7f71b37fe700 10 client.31290.objecter ms_dispatch 0x7f71bc001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:51:01.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.678+0000 7f71b37fe700 10 client.31290.objecter _finish_command 7 = system:0 2024-02-23T00:51:01.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.678+0000 7f71b37fe700 20 client.31290.objecter put_session s=0x7f71bc027be0 osd=3 4 2024-02-23T00:51:01.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.678+0000 7f71b37fe700 15 client.31290.objecter _session_command_op_remove 3 7 2024-02-23T00:51:01.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.693+0000 7f71c45c3700 10 client.31290.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:51:01.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.693+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_assign -1 8 2024-02-23T00:51:01.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.693+0000 7f71c45c3700 20 client.31290.objecter _get_session s=0x7f71bc027be0 osd=3 3 2024-02-23T00:51:01.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.693+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc027be0 osd=3 3 2024-02-23T00:51:01.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.693+0000 7f71c45c3700 20 client.31290.objecter _get_session s=0x7f71bc027be0 osd=3 3 2024-02-23T00:51:01.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.693+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_remove -1 8 2024-02-23T00:51:01.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.693+0000 7f71c45c3700 20 client.31290.objecter get_session s=0x7f71bc027be0 osd=3 3 2024-02-23T00:51:01.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.693+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_assign 3 8 2024-02-23T00:51:01.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.693+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc027be0 osd=3 4 2024-02-23T00:51:01.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.693+0000 7f71c45c3700 10 client.31290.objecter _send_command 8 2024-02-23T00:51:01.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.693+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f71bc11a660 con 0x7f71bc11d4c0 2024-02-23T00:51:01.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.694+0000 7f71b37fe700 1 -- 172.21.15.47:0/3648999758 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f71bc11a660 con 0x7f71bc11d4c0 2024-02-23T00:51:01.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.694+0000 7f71b37fe700 10 client.31290.objecter ms_dispatch 0x7f71bc001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:51:01.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.694+0000 7f71b37fe700 10 client.31290.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:51:01.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.694+0000 7f71b37fe700 20 client.31290.objecter put_session s=0x7f71bc027be0 osd=3 4 2024-02-23T00:51:01.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.694+0000 7f71b37fe700 15 client.31290.objecter _session_command_op_remove 3 8 2024-02-23T00:51:01.697 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:51:01.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.694+0000 7f71c45c3700 10 client.31290.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:01.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.694+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_assign -1 9 2024-02-23T00:51:01.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.694+0000 7f71c45c3700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f71bc11a7d0 0x7f71bc120f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:01.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.694+0000 7f71c45c3700 20 client.31290.objecter _get_session s=0x7f71bc11a660 osd=4 3 2024-02-23T00:51:01.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.694+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc11a660 osd=4 3 2024-02-23T00:51:01.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.694+0000 7f71c45c3700 20 client.31290.objecter _get_session s=0x7f71bc11a660 osd=4 3 2024-02-23T00:51:01.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.694+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_remove -1 9 2024-02-23T00:51:01.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.694+0000 7f71c45c3700 20 client.31290.objecter get_session s=0x7f71bc11a660 osd=4 3 2024-02-23T00:51:01.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.694+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_assign 4 9 2024-02-23T00:51:01.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.694+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc11a660 osd=4 4 2024-02-23T00:51:01.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.694+0000 7f71c45c3700 10 client.31290.objecter _send_command 9 2024-02-23T00:51:01.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.694+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f71bc119a80 con 0x7f71bc11a7d0 2024-02-23T00:51:01.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.695+0000 7f71c235f700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f71bc11a7d0 0x7f71bc120f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:01.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.695+0000 7f71b37fe700 10 client.31290.objecter ms_handle_connect 0x7f71bc11a7d0 2024-02-23T00:51:01.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.695+0000 7f71c235f700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f71bc11a7d0 0x7f71bc120f20 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:01.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.696+0000 7f71b37fe700 1 -- 172.21.15.47:0/3648999758 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f71bc119a80 con 0x7f71bc11a7d0 2024-02-23T00:51:01.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.696+0000 7f71b37fe700 10 client.31290.objecter ms_dispatch 0x7f71bc001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:51:01.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.696+0000 7f71b37fe700 10 client.31290.objecter _finish_command 9 = system:0 2024-02-23T00:51:01.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.696+0000 7f71b37fe700 20 client.31290.objecter put_session s=0x7f71bc11a660 osd=4 4 2024-02-23T00:51:01.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.696+0000 7f71b37fe700 15 client.31290.objecter _session_command_op_remove 4 9 2024-02-23T00:51:01.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.711+0000 7f71c45c3700 10 client.31290.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:51:01.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.711+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_assign -1 10 2024-02-23T00:51:01.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.711+0000 7f71c45c3700 20 client.31290.objecter _get_session s=0x7f71bc11a660 osd=4 3 2024-02-23T00:51:01.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.711+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc11a660 osd=4 3 2024-02-23T00:51:01.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.711+0000 7f71c45c3700 20 client.31290.objecter _get_session s=0x7f71bc11a660 osd=4 3 2024-02-23T00:51:01.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.711+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_remove -1 10 2024-02-23T00:51:01.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.711+0000 7f71c45c3700 20 client.31290.objecter get_session s=0x7f71bc11a660 osd=4 3 2024-02-23T00:51:01.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.711+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_assign 4 10 2024-02-23T00:51:01.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.711+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc11a660 osd=4 4 2024-02-23T00:51:01.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.711+0000 7f71c45c3700 10 client.31290.objecter _send_command 10 2024-02-23T00:51:01.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.711+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f71bc119a80 con 0x7f71bc11a7d0 2024-02-23T00:51:01.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.713+0000 7f71b37fe700 1 -- 172.21.15.47:0/3648999758 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f71bc119a80 con 0x7f71bc11a7d0 2024-02-23T00:51:01.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.713+0000 7f71b37fe700 10 client.31290.objecter ms_dispatch 0x7f71bc001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:51:01.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.713+0000 7f71b37fe700 10 client.31290.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:51:01.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.713+0000 7f71b37fe700 20 client.31290.objecter put_session s=0x7f71bc11a660 osd=4 4 2024-02-23T00:51:01.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.713+0000 7f71b37fe700 15 client.31290.objecter _session_command_op_remove 4 10 2024-02-23T00:51:01.716 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:51:01.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.713+0000 7f71c45c3700 10 client.31290.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:01.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.713+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_assign -1 11 2024-02-23T00:51:01.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.713+0000 7f71c45c3700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f71bc121920 0x7f71bc121d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:01.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.713+0000 7f71c45c3700 20 client.31290.objecter _get_session s=0x7f71bc119a80 osd=5 3 2024-02-23T00:51:01.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.713+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc119a80 osd=5 3 2024-02-23T00:51:01.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.713+0000 7f71c45c3700 20 client.31290.objecter _get_session s=0x7f71bc119a80 osd=5 3 2024-02-23T00:51:01.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.713+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_remove -1 11 2024-02-23T00:51:01.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.713+0000 7f71c45c3700 20 client.31290.objecter get_session s=0x7f71bc119a80 osd=5 3 2024-02-23T00:51:01.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.713+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_assign 5 11 2024-02-23T00:51:01.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.713+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc119a80 osd=5 4 2024-02-23T00:51:01.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.713+0000 7f71c45c3700 10 client.31290.objecter _send_command 11 2024-02-23T00:51:01.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.713+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f71bc122370 con 0x7f71bc121920 2024-02-23T00:51:01.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.714+0000 7f71c1b5e700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f71bc121920 0x7f71bc121d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:01.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.714+0000 7f71b37fe700 10 client.31290.objecter ms_handle_connect 0x7f71bc121920 2024-02-23T00:51:01.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.714+0000 7f71c1b5e700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f71bc121920 0x7f71bc121d80 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:01.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.715+0000 7f71b37fe700 1 -- 172.21.15.47:0/3648999758 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f71bc122370 con 0x7f71bc121920 2024-02-23T00:51:01.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.715+0000 7f71b37fe700 10 client.31290.objecter ms_dispatch 0x7f71bc001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:51:01.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.715+0000 7f71b37fe700 10 client.31290.objecter _finish_command 11 = system:0 2024-02-23T00:51:01.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.715+0000 7f71b37fe700 20 client.31290.objecter put_session s=0x7f71bc119a80 osd=5 4 2024-02-23T00:51:01.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.715+0000 7f71b37fe700 15 client.31290.objecter _session_command_op_remove 5 11 2024-02-23T00:51:01.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.729+0000 7f71c45c3700 10 client.31290.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:51:01.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.729+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_assign -1 12 2024-02-23T00:51:01.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.729+0000 7f71c45c3700 20 client.31290.objecter _get_session s=0x7f71bc119a80 osd=5 3 2024-02-23T00:51:01.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.729+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc119a80 osd=5 3 2024-02-23T00:51:01.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.729+0000 7f71c45c3700 20 client.31290.objecter _get_session s=0x7f71bc119a80 osd=5 3 2024-02-23T00:51:01.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.729+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_remove -1 12 2024-02-23T00:51:01.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.729+0000 7f71c45c3700 20 client.31290.objecter get_session s=0x7f71bc119a80 osd=5 3 2024-02-23T00:51:01.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.729+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_assign 5 12 2024-02-23T00:51:01.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.729+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc119a80 osd=5 4 2024-02-23T00:51:01.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.729+0000 7f71c45c3700 10 client.31290.objecter _send_command 12 2024-02-23T00:51:01.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.729+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f71bc122370 con 0x7f71bc121920 2024-02-23T00:51:01.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.731+0000 7f71b37fe700 1 -- 172.21.15.47:0/3648999758 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f71bc122370 con 0x7f71bc121920 2024-02-23T00:51:01.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.731+0000 7f71b37fe700 10 client.31290.objecter ms_dispatch 0x7f71bc001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:51:01.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.731+0000 7f71b37fe700 10 client.31290.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:51:01.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.731+0000 7f71b37fe700 20 client.31290.objecter put_session s=0x7f71bc119a80 osd=5 4 2024-02-23T00:51:01.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.731+0000 7f71b37fe700 15 client.31290.objecter _session_command_op_remove 5 12 2024-02-23T00:51:01.735 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:51:01.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.731+0000 7f71c45c3700 10 client.31290.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:01.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.731+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_assign -1 13 2024-02-23T00:51:01.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.731+0000 7f71c45c3700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f71bc126bf0 0x7f71bc127030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:01.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.731+0000 7f71c45c3700 20 client.31290.objecter _get_session s=0x7f71bc122370 osd=6 3 2024-02-23T00:51:01.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.731+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc122370 osd=6 3 2024-02-23T00:51:01.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.731+0000 7f71c45c3700 20 client.31290.objecter _get_session s=0x7f71bc122370 osd=6 3 2024-02-23T00:51:01.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.731+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_remove -1 13 2024-02-23T00:51:01.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.731+0000 7f71c45c3700 20 client.31290.objecter get_session s=0x7f71bc122370 osd=6 3 2024-02-23T00:51:01.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.731+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_assign 6 13 2024-02-23T00:51:01.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.731+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc122370 osd=6 4 2024-02-23T00:51:01.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.731+0000 7f71c45c3700 10 client.31290.objecter _send_command 13 2024-02-23T00:51:01.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.731+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f71bc12bfb0 con 0x7f71bc126bf0 2024-02-23T00:51:01.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.732+0000 7f71c2b60700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f71bc126bf0 0x7f71bc127030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:01.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.732+0000 7f71b37fe700 10 client.31290.objecter ms_handle_connect 0x7f71bc126bf0 2024-02-23T00:51:01.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.732+0000 7f71c2b60700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f71bc126bf0 0x7f71bc127030 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:01.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.733+0000 7f71b37fe700 1 -- 172.21.15.47:0/3648999758 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f71bc12bfb0 con 0x7f71bc126bf0 2024-02-23T00:51:01.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.733+0000 7f71b37fe700 10 client.31290.objecter ms_dispatch 0x7f71bc001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:51:01.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.733+0000 7f71b37fe700 10 client.31290.objecter _finish_command 13 = system:0 2024-02-23T00:51:01.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.733+0000 7f71b37fe700 20 client.31290.objecter put_session s=0x7f71bc122370 osd=6 4 2024-02-23T00:51:01.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.733+0000 7f71b37fe700 15 client.31290.objecter _session_command_op_remove 6 13 2024-02-23T00:51:01.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.748+0000 7f71c45c3700 10 client.31290.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:51:01.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.748+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_assign -1 14 2024-02-23T00:51:01.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.748+0000 7f71c45c3700 20 client.31290.objecter _get_session s=0x7f71bc122370 osd=6 3 2024-02-23T00:51:01.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.748+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc122370 osd=6 3 2024-02-23T00:51:01.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.748+0000 7f71c45c3700 20 client.31290.objecter _get_session s=0x7f71bc122370 osd=6 3 2024-02-23T00:51:01.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.748+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_remove -1 14 2024-02-23T00:51:01.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.748+0000 7f71c45c3700 20 client.31290.objecter get_session s=0x7f71bc122370 osd=6 3 2024-02-23T00:51:01.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.748+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_assign 6 14 2024-02-23T00:51:01.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.748+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc122370 osd=6 4 2024-02-23T00:51:01.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.748+0000 7f71c45c3700 10 client.31290.objecter _send_command 14 2024-02-23T00:51:01.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.748+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f71bc127630 con 0x7f71bc126bf0 2024-02-23T00:51:01.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.750+0000 7f71b37fe700 1 -- 172.21.15.47:0/3648999758 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f71bc127630 con 0x7f71bc126bf0 2024-02-23T00:51:01.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.750+0000 7f71b37fe700 10 client.31290.objecter ms_dispatch 0x7f71bc001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:51:01.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.750+0000 7f71b37fe700 10 client.31290.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:51:01.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.750+0000 7f71b37fe700 20 client.31290.objecter put_session s=0x7f71bc122370 osd=6 4 2024-02-23T00:51:01.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.750+0000 7f71b37fe700 15 client.31290.objecter _session_command_op_remove 6 14 2024-02-23T00:51:01.753 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:51:01.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.750+0000 7f71c45c3700 10 client.31290.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:01.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.750+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_assign -1 15 2024-02-23T00:51:01.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.750+0000 7f71c45c3700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f71bc127cb0 0x7f71bc128110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:01.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.750+0000 7f71c45c3700 20 client.31290.objecter _get_session s=0x7f71bc127630 osd=7 3 2024-02-23T00:51:01.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.750+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc127630 osd=7 3 2024-02-23T00:51:01.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.750+0000 7f71c45c3700 20 client.31290.objecter _get_session s=0x7f71bc127630 osd=7 3 2024-02-23T00:51:01.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.750+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_remove -1 15 2024-02-23T00:51:01.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.750+0000 7f71c45c3700 20 client.31290.objecter get_session s=0x7f71bc127630 osd=7 3 2024-02-23T00:51:01.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.750+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_assign 7 15 2024-02-23T00:51:01.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.750+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc127630 osd=7 4 2024-02-23T00:51:01.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.750+0000 7f71c45c3700 10 client.31290.objecter _send_command 15 2024-02-23T00:51:01.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.750+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f71bc128600 con 0x7f71bc127cb0 2024-02-23T00:51:01.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.751+0000 7f71c235f700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f71bc127cb0 0x7f71bc128110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:01.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.751+0000 7f71b37fe700 10 client.31290.objecter ms_handle_connect 0x7f71bc127cb0 2024-02-23T00:51:01.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.751+0000 7f71c235f700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f71bc127cb0 0x7f71bc128110 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:01.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.752+0000 7f71b37fe700 1 -- 172.21.15.47:0/3648999758 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f71bc128600 con 0x7f71bc127cb0 2024-02-23T00:51:01.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.752+0000 7f71b37fe700 10 client.31290.objecter ms_dispatch 0x7f71bc001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:51:01.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.752+0000 7f71b37fe700 10 client.31290.objecter _finish_command 15 = system:0 2024-02-23T00:51:01.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.752+0000 7f71b37fe700 20 client.31290.objecter put_session s=0x7f71bc127630 osd=7 4 2024-02-23T00:51:01.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.752+0000 7f71b37fe700 15 client.31290.objecter _session_command_op_remove 7 15 2024-02-23T00:51:01.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.767+0000 7f71c45c3700 10 client.31290.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:51:01.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.767+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_assign -1 16 2024-02-23T00:51:01.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.767+0000 7f71c45c3700 20 client.31290.objecter _get_session s=0x7f71bc127630 osd=7 3 2024-02-23T00:51:01.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.767+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc127630 osd=7 3 2024-02-23T00:51:01.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.767+0000 7f71c45c3700 20 client.31290.objecter _get_session s=0x7f71bc127630 osd=7 3 2024-02-23T00:51:01.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.767+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_remove -1 16 2024-02-23T00:51:01.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.767+0000 7f71c45c3700 20 client.31290.objecter get_session s=0x7f71bc127630 osd=7 3 2024-02-23T00:51:01.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.767+0000 7f71c45c3700 15 client.31290.objecter _session_command_op_assign 7 16 2024-02-23T00:51:01.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.767+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc127630 osd=7 4 2024-02-23T00:51:01.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.767+0000 7f71c45c3700 10 client.31290.objecter _send_command 16 2024-02-23T00:51:01.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.767+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f71bc128600 con 0x7f71bc127cb0 2024-02-23T00:51:01.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.768+0000 7f71b37fe700 1 -- 172.21.15.47:0/3648999758 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f71bc128600 con 0x7f71bc127cb0 2024-02-23T00:51:01.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.768+0000 7f71b37fe700 10 client.31290.objecter ms_dispatch 0x7f71bc001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:51:01.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.768+0000 7f71b37fe700 10 client.31290.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:51:01.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.768+0000 7f71b37fe700 20 client.31290.objecter put_session s=0x7f71bc127630 osd=7 4 2024-02-23T00:51:01.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.768+0000 7f71b37fe700 15 client.31290.objecter _session_command_op_remove 7 16 2024-02-23T00:51:01.771 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:51:01.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 10 librados: watch_flush enter 2024-02-23T00:51:01.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 10 librados: watch_flush exit 2024-02-23T00:51:01.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 10 client.31290.objecter close_session for osd.0 2024-02-23T00:51:01.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f71bc0fc140 msgr2=0x7f71bc0fc520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:01.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f71bc0fc140 0x7f71bc0fc520 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:01.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc05ee80 osd=0 1 2024-02-23T00:51:01.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 10 client.31290.objecter close_session for osd.1 2024-02-23T00:51:01.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f71bc0598f0 msgr2=0x7f71bc059d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:01.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f71bc0598f0 0x7f71bc059d10 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:01.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc028710 osd=1 1 2024-02-23T00:51:01.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 10 client.31290.objecter close_session for osd.2 2024-02-23T00:51:01.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f71bc028270 msgr2=0x7f71bc119d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:01.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f71bc028270 0x7f71bc119d60 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:01.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc059770 osd=2 1 2024-02-23T00:51:01.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 10 client.31290.objecter close_session for osd.3 2024-02-23T00:51:01.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f71bc11d4c0 msgr2=0x7f71bc11f8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:01.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c2b60700 1 -- 172.21.15.47:0/3648999758 reap_dead start 2024-02-23T00:51:01.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f71bc11d4c0 0x7f71bc11f8b0 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:01.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc027be0 osd=3 1 2024-02-23T00:51:01.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 10 client.31290.objecter close_session for osd.4 2024-02-23T00:51:01.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f71bc11a7d0 msgr2=0x7f71bc120f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:01.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f71bc11a7d0 0x7f71bc120f20 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:01.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc11a660 osd=4 1 2024-02-23T00:51:01.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 10 client.31290.objecter close_session for osd.5 2024-02-23T00:51:01.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f71bc121920 msgr2=0x7f71bc121d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:01.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f71bc121920 0x7f71bc121d80 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:01.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc119a80 osd=5 1 2024-02-23T00:51:01.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 10 client.31290.objecter close_session for osd.6 2024-02-23T00:51:01.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f71bc126bf0 msgr2=0x7f71bc127030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:01.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f71bc126bf0 0x7f71bc127030 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:01.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc122370 osd=6 1 2024-02-23T00:51:01.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 10 client.31290.objecter close_session for osd.7 2024-02-23T00:51:01.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f71bc127cb0 msgr2=0x7f71bc128110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:01.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f71bc127cb0 0x7f71bc128110 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:01.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 20 client.31290.objecter put_session s=0x7f71bc127630 osd=7 1 2024-02-23T00:51:01.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 20 client.31290.objecter shutdown clearing up homeless session... 2024-02-23T00:51:01.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 10 client.31290.objecter successfully canceled tick 2024-02-23T00:51:01.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c2b60700 1 -- 172.21.15.47:0/3648999758 reap_dead start 2024-02-23T00:51:01.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f71a8062cc0 msgr2=0x7f71a8065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:01.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f71a8062cc0 0x7f71a8065180 secure :-1 s=READY pgs=3111 cs=0 l=1 rev1=1 rx=0x7f71b8004c80 tx=0x7f71b800b040).stop 2024-02-23T00:51:01.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71bc1021a0 msgr2=0x7f71bc107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:01.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71bc1021a0 0x7f71bc107f30 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 rx=0x7f71b4004890 tx=0x7f71b4004c50).stop 2024-02-23T00:51:01.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 shutdown_connections 2024-02-23T00:51:01.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f71a8062cc0 0x7f71a8065180 unknown :-1 s=CLOSED pgs=3111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:01.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 1 --2- 172.21.15.47:0/3648999758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71bc1021a0 0x7f71bc107f30 unknown :-1 s=CLOSED pgs=2568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:01.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 >> 172.21.15.47:0/3648999758 conn(0x7f71bc0f26f0 msgr2=0x7f71bc0f32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:01.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 shutdown_connections 2024-02-23T00:51:01.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 1 -- 172.21.15.47:0/3648999758 wait complete. 2024-02-23T00:51:01.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:01.769+0000 7f71c45c3700 1 librados: shutdown 2024-02-23T00:51:01.776 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:51:01.776 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:51:01.776 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:51:01.776 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:51:01.776 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:51:01.776 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:51:01.776 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:51:01.776 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:51:01.860 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:51:01.861 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-02-23T00:51:01.861 INFO:teuthology.orchestra.run.smithi047.stdout: "num_ops": 0 2024-02-23T00:51:01.861 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:51:02.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:01.200364+0000 osd.3 (osd.3) 252 : cluster [DBG] 4.17 scrub starts 2024-02-23T00:51:02.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:01.201733+0000 osd.3 (osd.3) 253 : cluster [DBG] 4.17 scrub ok 2024-02-23T00:51:02.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:01.217923+0000 osd.6 (osd.6) 207 : cluster [DBG] 4.5 scrub starts 2024-02-23T00:51:02.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:01.219818+0000 osd.6 (osd.6) 208 : cluster [DBG] 4.5 scrub ok 2024-02-23T00:51:02.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:01.238908+0000 mon.a (mon.0) 2895 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-88","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:51:02.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:01.238993+0000 mon.a (mon.0) 2896 : cluster [DBG] osdmap e532: 8 total, 8 up, 7 in 2024-02-23T00:51:02.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:01.259810+0000 mon.a (mon.0) 2897 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-88", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:51:02.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:01.336401+0000 mgr.y (mgr.14517) 489 : cluster [DBG] pgmap v802: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:51:02.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:01.486866+0000 osd.4 (osd.4) 227 : cluster [DBG] 4.c scrub starts 2024-02-23T00:51:02.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:01.488366+0000 osd.4 (osd.4) 228 : cluster [DBG] 4.c scrub ok 2024-02-23T00:51:02.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:01.625404+0000 mon.a (mon.0) 2898 : audit [DBG] from='client.? 172.21.15.47:0/3648999758' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:51:02.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:01.200364+0000 osd.3 (osd.3) 252 : cluster [DBG] 4.17 scrub starts 2024-02-23T00:51:02.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:01.201733+0000 osd.3 (osd.3) 253 : cluster [DBG] 4.17 scrub ok 2024-02-23T00:51:02.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:01.217923+0000 osd.6 (osd.6) 207 : cluster [DBG] 4.5 scrub starts 2024-02-23T00:51:02.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:01.219818+0000 osd.6 (osd.6) 208 : cluster [DBG] 4.5 scrub ok 2024-02-23T00:51:02.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:01.238908+0000 mon.a (mon.0) 2895 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-88","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:51:02.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:01.238993+0000 mon.a (mon.0) 2896 : cluster [DBG] osdmap e532: 8 total, 8 up, 7 in 2024-02-23T00:51:02.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:01.259810+0000 mon.a (mon.0) 2897 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-88", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:51:02.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:01.336401+0000 mgr.y (mgr.14517) 489 : cluster [DBG] pgmap v802: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:51:02.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:01.486866+0000 osd.4 (osd.4) 227 : cluster [DBG] 4.c scrub starts 2024-02-23T00:51:02.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:01.488366+0000 osd.4 (osd.4) 228 : cluster [DBG] 4.c scrub ok 2024-02-23T00:51:02.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:01.625404+0000 mon.a (mon.0) 2898 : audit [DBG] from='client.? 172.21.15.47:0/3648999758' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:51:02.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:01.200364+0000 osd.3 (osd.3) 252 : cluster [DBG] 4.17 scrub starts 2024-02-23T00:51:02.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:01.201733+0000 osd.3 (osd.3) 253 : cluster [DBG] 4.17 scrub ok 2024-02-23T00:51:02.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:01.217923+0000 osd.6 (osd.6) 207 : cluster [DBG] 4.5 scrub starts 2024-02-23T00:51:02.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:01.219818+0000 osd.6 (osd.6) 208 : cluster [DBG] 4.5 scrub ok 2024-02-23T00:51:02.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:01.238908+0000 mon.a (mon.0) 2895 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-88","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:51:02.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:01.238993+0000 mon.a (mon.0) 2896 : cluster [DBG] osdmap e532: 8 total, 8 up, 7 in 2024-02-23T00:51:02.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:01.259810+0000 mon.a (mon.0) 2897 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-88", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:51:02.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:01.336401+0000 mgr.y (mgr.14517) 489 : cluster [DBG] pgmap v802: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:51:02.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:01.486866+0000 osd.4 (osd.4) 227 : cluster [DBG] 4.c scrub starts 2024-02-23T00:51:02.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:01.488366+0000 osd.4 (osd.4) 228 : cluster [DBG] 4.c scrub ok 2024-02-23T00:51:02.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:01.625404+0000 mon.a (mon.0) 2898 : audit [DBG] from='client.? 172.21.15.47:0/3648999758' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:51:02.758 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:51:02.764 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:02.764 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:51:02.865 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:51:02.894 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:51:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:51:02.895 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:51:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:51:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:51:02.897 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:02.918 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:51:02.921 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:02.921 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:51:03.022 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:51:03.052 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:03.071 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:51:03.074 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:03.075 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:51:03.175 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:51:03.208 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:03.249 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:02.193505+0000 osd.7 (osd.7) 263 : cluster [DBG] 62.0 scrub starts 2024-02-23T00:51:03.249 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:02.196963+0000 osd.7 (osd.7) 264 : cluster [DBG] 62.0 scrub ok 2024-02-23T00:51:03.249 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:02.243937+0000 mon.a (mon.0) 2899 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-88", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:51:03.249 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:02.243987+0000 mon.a (mon.0) 2900 : cluster [DBG] osdmap e533: 8 total, 8 up, 7 in 2024-02-23T00:51:03.249 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:02.244599+0000 mon.a (mon.0) 2901 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-88", "mode": "writeback"}]: dispatch 2024-02-23T00:51:03.251 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:02.193505+0000 osd.7 (osd.7) 263 : cluster [DBG] 62.0 scrub starts 2024-02-23T00:51:03.251 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:02.196963+0000 osd.7 (osd.7) 264 : cluster [DBG] 62.0 scrub ok 2024-02-23T00:51:03.251 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:02.243937+0000 mon.a (mon.0) 2899 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-88", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:51:03.251 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:02.243987+0000 mon.a (mon.0) 2900 : cluster [DBG] osdmap e533: 8 total, 8 up, 7 in 2024-02-23T00:51:03.251 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:02.244599+0000 mon.a (mon.0) 2901 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-88", "mode": "writeback"}]: dispatch 2024-02-23T00:51:03.283 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:51:03.286 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:03.287 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:51:03.301 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.3 dump_blocked_ops 2024-02-23T00:51:03.358 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:51:03.363 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:51:03.388 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:51:03.419 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:03.440 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:51:03.442 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:03.443 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:51:03.544 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:51:03.571 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:02.193505+0000 osd.7 (osd.7) 263 : cluster [DBG] 62.0 scrub starts 2024-02-23T00:51:03.571 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:02.196963+0000 osd.7 (osd.7) 264 : cluster [DBG] 62.0 scrub ok 2024-02-23T00:51:03.571 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:02.243937+0000 mon.a (mon.0) 2899 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-88", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:51:03.571 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:02.243987+0000 mon.a (mon.0) 2900 : cluster [DBG] osdmap e533: 8 total, 8 up, 7 in 2024-02-23T00:51:03.571 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:02.244599+0000 mon.a (mon.0) 2901 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-88", "mode": "writeback"}]: dispatch 2024-02-23T00:51:03.573 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:03.592 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:51:03.595 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:03.595 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:51:03.696 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:51:03.724 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:03.743 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:51:03.745 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:03.746 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:51:03.847 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:51:03.876 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:03.897 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:51:03.899 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:03.900 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:51:04.001 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:51:04.029 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:04.048 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:51:04.050 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:04.051 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:51:04.152 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:51:04.180 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:04.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:02.376989+0000 osd.1 (osd.1) 231 : cluster [DBG] 6.4 scrub starts 2024-02-23T00:51:04.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:02.380717+0000 osd.1 (osd.1) 232 : cluster [DBG] 6.4 scrub ok 2024-02-23T00:51:04.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:02.730336+0000 mgr.y (mgr.14517) 490 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:51:04.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:03.165106+0000 osd.3 (osd.3) 254 : cluster [DBG] 18.16 scrub starts 2024-02-23T00:51:04.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:03.166135+0000 osd.3 (osd.3) 255 : cluster [DBG] 18.16 scrub ok 2024-02-23T00:51:04.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:03.240504+0000 mon.a (mon.0) 2902 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:51:04.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:03.244645+0000 mon.a (mon.0) 2903 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-88", "mode": "writeback"}]': finished 2024-02-23T00:51:04.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:03.244709+0000 mon.a (mon.0) 2904 : cluster [DBG] osdmap e534: 8 total, 8 up, 7 in 2024-02-23T00:51:04.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:03.245560+0000 mon.a (mon.0) 2905 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-88"}]: dispatch 2024-02-23T00:51:04.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:03.336775+0000 mgr.y (mgr.14517) 491 : cluster [DBG] pgmap v805: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-23T00:51:04.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:03.459164+0000 osd.4 (osd.4) 229 : cluster [DBG] 18.d scrub starts 2024-02-23T00:51:04.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:03.460295+0000 osd.4 (osd.4) 230 : cluster [DBG] 18.d scrub ok 2024-02-23T00:51:04.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:03.872318+0000 osd.5 (osd.5) 240 : cluster [DBG] 18.17 scrub starts 2024-02-23T00:51:04.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:03.875269+0000 osd.5 (osd.5) 241 : cluster [DBG] 18.17 scrub ok 2024-02-23T00:51:04.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:04.246019+0000 mon.a (mon.0) 2906 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-88"}]': finished 2024-02-23T00:51:04.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:04.246089+0000 mon.a (mon.0) 2907 : cluster [DBG] osdmap e535: 8 total, 8 up, 7 in 2024-02-23T00:51:04.529 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:51:04.531 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:02.376989+0000 osd.1 (osd.1) 231 : cluster [DBG] 6.4 scrub starts 2024-02-23T00:51:04.531 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:02.380717+0000 osd.1 (osd.1) 232 : cluster [DBG] 6.4 scrub ok 2024-02-23T00:51:04.531 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:02.730336+0000 mgr.y (mgr.14517) 490 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:51:04.531 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:03.165106+0000 osd.3 (osd.3) 254 : cluster [DBG] 18.16 scrub starts 2024-02-23T00:51:04.531 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:03.166135+0000 osd.3 (osd.3) 255 : cluster [DBG] 18.16 scrub ok 2024-02-23T00:51:04.531 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:03.240504+0000 mon.a (mon.0) 2902 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:51:04.531 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:03.244645+0000 mon.a (mon.0) 2903 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-88", "mode": "writeback"}]': finished 2024-02-23T00:51:04.531 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:03.244709+0000 mon.a (mon.0) 2904 : cluster [DBG] osdmap e534: 8 total, 8 up, 7 in 2024-02-23T00:51:04.531 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:03.245560+0000 mon.a (mon.0) 2905 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-88"}]: dispatch 2024-02-23T00:51:04.532 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:03.336775+0000 mgr.y (mgr.14517) 491 : cluster [DBG] pgmap v805: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-23T00:51:04.532 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:03.459164+0000 osd.4 (osd.4) 229 : cluster [DBG] 18.d scrub starts 2024-02-23T00:51:04.532 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:03.460295+0000 osd.4 (osd.4) 230 : cluster [DBG] 18.d scrub ok 2024-02-23T00:51:04.532 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:03.872318+0000 osd.5 (osd.5) 240 : cluster [DBG] 18.17 scrub starts 2024-02-23T00:51:04.532 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:03.875269+0000 osd.5 (osd.5) 241 : cluster [DBG] 18.17 scrub ok 2024-02-23T00:51:04.532 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:04.246019+0000 mon.a (mon.0) 2906 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-88"}]': finished 2024-02-23T00:51:04.532 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:04.246089+0000 mon.a (mon.0) 2907 : cluster [DBG] osdmap e535: 8 total, 8 up, 7 in 2024-02-23T00:51:04.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:02.376989+0000 osd.1 (osd.1) 231 : cluster [DBG] 6.4 scrub starts 2024-02-23T00:51:04.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:02.380717+0000 osd.1 (osd.1) 232 : cluster [DBG] 6.4 scrub ok 2024-02-23T00:51:04.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:02.730336+0000 mgr.y (mgr.14517) 490 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:51:04.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:03.165106+0000 osd.3 (osd.3) 254 : cluster [DBG] 18.16 scrub starts 2024-02-23T00:51:04.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:03.166135+0000 osd.3 (osd.3) 255 : cluster [DBG] 18.16 scrub ok 2024-02-23T00:51:04.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:03.240504+0000 mon.a (mon.0) 2902 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:51:04.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:03.244645+0000 mon.a (mon.0) 2903 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-88", "mode": "writeback"}]': finished 2024-02-23T00:51:04.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:03.244709+0000 mon.a (mon.0) 2904 : cluster [DBG] osdmap e534: 8 total, 8 up, 7 in 2024-02-23T00:51:04.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:03.245560+0000 mon.a (mon.0) 2905 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-88"}]: dispatch 2024-02-23T00:51:04.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:03.336775+0000 mgr.y (mgr.14517) 491 : cluster [DBG] pgmap v805: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-23T00:51:04.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:03.459164+0000 osd.4 (osd.4) 229 : cluster [DBG] 18.d scrub starts 2024-02-23T00:51:04.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:03.460295+0000 osd.4 (osd.4) 230 : cluster [DBG] 18.d scrub ok 2024-02-23T00:51:04.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:03.872318+0000 osd.5 (osd.5) 240 : cluster [DBG] 18.17 scrub starts 2024-02-23T00:51:04.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:03.875269+0000 osd.5 (osd.5) 241 : cluster [DBG] 18.17 scrub ok 2024-02-23T00:51:04.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:04.246019+0000 mon.a (mon.0) 2906 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-88"}]': finished 2024-02-23T00:51:04.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:04.246089+0000 mon.a (mon.0) 2907 : cluster [DBG] osdmap e535: 8 total, 8 up, 7 in 2024-02-23T00:51:04.534 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:04.534 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:51:04.635 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:51:04.665 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:04.733 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:51:05.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:04.247819+0000 mon.a (mon.0) 2908 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-88","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-23T00:51:05.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:05.233593+0000 mon.a (mon.0) 2909 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:51:05.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:05.237857+0000 mon.a (mon.0) 2910 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-88","var": "hit_set_type","val": "bloom"}]': finished 2024-02-23T00:51:05.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:05.237918+0000 mon.a (mon.0) 2911 : cluster [DBG] osdmap e536: 8 total, 8 up, 7 in 2024-02-23T00:51:05.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:05.238533+0000 mon.a (mon.0) 2912 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-88","var": "hit_set_count","val": "1"}]: dispatch 2024-02-23T00:51:05.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:04.247819+0000 mon.a (mon.0) 2908 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-88","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-23T00:51:05.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:05.233593+0000 mon.a (mon.0) 2909 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:51:05.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:05.237857+0000 mon.a (mon.0) 2910 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-88","var": "hit_set_type","val": "bloom"}]': finished 2024-02-23T00:51:05.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:05.237918+0000 mon.a (mon.0) 2911 : cluster [DBG] osdmap e536: 8 total, 8 up, 7 in 2024-02-23T00:51:05.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:05.238533+0000 mon.a (mon.0) 2912 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-88","var": "hit_set_count","val": "1"}]: dispatch 2024-02-23T00:51:05.314 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:51:05.318 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:05.319 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:51:05.419 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:51:05.451 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:05.472 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:51:05.474 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:05.475 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:51:05.576 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:51:05.606 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:04.247819+0000 mon.a (mon.0) 2908 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-88","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-23T00:51:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:05.233593+0000 mon.a (mon.0) 2909 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:51:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:05.237857+0000 mon.a (mon.0) 2910 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-88","var": "hit_set_type","val": "bloom"}]': finished 2024-02-23T00:51:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:05.237918+0000 mon.a (mon.0) 2911 : cluster [DBG] osdmap e536: 8 total, 8 up, 7 in 2024-02-23T00:51:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:05.238533+0000 mon.a (mon.0) 2912 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-88","var": "hit_set_count","val": "1"}]: dispatch 2024-02-23T00:51:05.755 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:51:05.757 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:05.757 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:51:05.858 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:51:05.887 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:06.262 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:05.217596+0000 osd.3 (osd.3) 256 : cluster [DBG] 4.0 scrub starts 2024-02-23T00:51:06.262 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:05.221568+0000 osd.3 (osd.3) 257 : cluster [DBG] 4.0 scrub ok 2024-02-23T00:51:06.262 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:05.337177+0000 mgr.y (mgr.14517) 492 : cluster [DBG] pgmap v808: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-23T00:51:06.262 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:06.239820+0000 mon.a (mon.0) 2913 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-88","var": "hit_set_count","val": "1"}]': finished 2024-02-23T00:51:06.262 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:06.239897+0000 mon.a (mon.0) 2914 : cluster [DBG] osdmap e537: 8 total, 8 up, 7 in 2024-02-23T00:51:06.262 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:06.241451+0000 mon.a (mon.0) 2915 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-88","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:51:06.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.330+0000 7f5e5db16700 1 -- 172.21.15.47:0/1174298247 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5e580f8420 msgr2=0x7f5e58104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:06.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.330+0000 7f5e5db16700 1 --2- 172.21.15.47:0/1174298247 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5e580f8420 0x7f5e58104a20 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 rx=0x7f5e50009be0 tx=0x7f5e50004d10).stop 2024-02-23T00:51:06.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.330+0000 7f5e5db16700 1 -- 172.21.15.47:0/1174298247 shutdown_connections 2024-02-23T00:51:06.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.330+0000 7f5e5db16700 1 --2- 172.21.15.47:0/1174298247 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5e580f8420 0x7f5e58104a20 unknown :-1 s=CLOSED pgs=2723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:06.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.330+0000 7f5e5db16700 1 --2- 172.21.15.47:0/1174298247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e580fdbb0 0x7f5e580fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:06.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.330+0000 7f5e5db16700 1 --2- 172.21.15.47:0/1174298247 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e580fe510 0x7f5e580f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:06.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.330+0000 7f5e5db16700 1 -- 172.21.15.47:0/1174298247 >> 172.21.15.47:0/1174298247 conn(0x7f5e580f3800 msgr2=0x7f5e580f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:06.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.330+0000 7f5e5db16700 1 -- 172.21.15.47:0/1174298247 shutdown_connections 2024-02-23T00:51:06.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.330+0000 7f5e5db16700 1 -- 172.21.15.47:0/1174298247 wait complete. 2024-02-23T00:51:06.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.330+0000 7f5e5db16700 1 librados: starting msgr at 2024-02-23T00:51:06.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.330+0000 7f5e5db16700 1 librados: starting objecter 2024-02-23T00:51:06.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.330+0000 7f5e5db16700 1 Processor -- start 2024-02-23T00:51:06.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.330+0000 7f5e5db16700 1 -- start start 2024-02-23T00:51:06.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.330+0000 7f5e5db16700 1 librados: setting wanted keys 2024-02-23T00:51:06.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.330+0000 7f5e5db16700 1 librados: calling monclient init 2024-02-23T00:51:06.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.331+0000 7f5e5db16700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5e580f8420 0x7f5e5810fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:06.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.331+0000 7f5e5db16700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e580fdbb0 0x7f5e58110350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:06.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.331+0000 7f5e5db16700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e580fe510 0x7f5e5810a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:06.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.331+0000 7f5e5db16700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5810a4f0 con 0x7f5e580fe510 2024-02-23T00:51:06.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.331+0000 7f5e5db16700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5810a660 con 0x7f5e580fdbb0 2024-02-23T00:51:06.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.331+0000 7f5e5db16700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5810a820 con 0x7f5e580f8420 2024-02-23T00:51:06.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.331+0000 7f5e577fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5e580f8420 0x7f5e5810fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:06.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.331+0000 7f5e57fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e580fe510 0x7f5e5810a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:06.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.331+0000 7f5e577fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5e580f8420 0x7f5e5810fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:47962/0 (socket says 172.21.15.47:47962) 2024-02-23T00:51:06.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.331+0000 7f5e577fe700 1 -- 172.21.15.47:0/976307319 learned_addr learned my addr 172.21.15.47:0/976307319 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:51:06.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.331+0000 7f5e56ffd700 1 --2- 172.21.15.47:0/976307319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e580fdbb0 0x7f5e58110350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:06.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.331+0000 7f5e577fe700 1 -- 172.21.15.47:0/976307319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e580fdbb0 msgr2=0x7f5e58110350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:06.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.331+0000 7f5e577fe700 1 --2- 172.21.15.47:0/976307319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e580fdbb0 0x7f5e58110350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:06.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.331+0000 7f5e577fe700 1 -- 172.21.15.47:0/976307319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e580fe510 msgr2=0x7f5e5810a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:06.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.331+0000 7f5e577fe700 1 --2- 172.21.15.47:0/976307319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e580fe510 0x7f5e5810a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:06.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.331+0000 7f5e577fe700 1 -- 172.21.15.47:0/976307319 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4c005040 con 0x7f5e580f8420 2024-02-23T00:51:06.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.331+0000 7f5e57fff700 1 --2- 172.21.15.47:0/976307319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e580fe510 0x7f5e5810a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:51:06.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.331+0000 7f5e56ffd700 1 --2- 172.21.15.47:0/976307319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e580fdbb0 0x7f5e58110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:51:06.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.331+0000 7f5e577fe700 1 --2- 172.21.15.47:0/976307319 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5e580f8420 0x7f5e5810fe60 secure :-1 s=READY pgs=2724 cs=0 l=1 rev1=1 rx=0x7f5e480027f0 tx=0x7f5e48002b00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:06.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.331+0000 7f5e54ff9700 10 client.?.objecter ms_handle_connect 0x7f5e580f8420 2024-02-23T00:51:06.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.331+0000 7f5e54ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:51:06.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.332+0000 7f5e54ff9700 1 -- 172.21.15.47:0/976307319 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e48007b40 con 0x7f5e580f8420 2024-02-23T00:51:06.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.332+0000 7f5e54ff9700 1 -- 172.21.15.47:0/976307319 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f5e4800f040 con 0x7f5e580f8420 2024-02-23T00:51:06.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.332+0000 7f5e5db16700 1 -- 172.21.15.47:0/976307319 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e500097c0 con 0x7f5e580f8420 2024-02-23T00:51:06.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.332+0000 7f5e54ff9700 1 -- 172.21.15.47:0/976307319 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4800d660 con 0x7f5e580f8420 2024-02-23T00:51:06.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.332+0000 7f5e5db16700 10 client.41621.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:51:06.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.332+0000 7f5e5db16700 1 -- 172.21.15.47:0/976307319 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e5810ae60 con 0x7f5e580f8420 2024-02-23T00:51:06.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.332+0000 7f5e5db16700 1 librados: init done 2024-02-23T00:51:06.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.333+0000 7f5e5db16700 1 -- 172.21.15.47:0/976307319 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e58106ed0 con 0x7f5e580f8420 2024-02-23T00:51:06.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.333+0000 7f5e54ff9700 1 -- 172.21.15.47:0/976307319 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f5e48009110 con 0x7f5e580f8420 2024-02-23T00:51:06.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.334+0000 7f5e54ff9700 1 --2- 172.21.15.47:0/976307319 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5e40062cc0 0x7f5e40065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:06.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.334+0000 7f5e54ff9700 1 -- 172.21.15.47:0/976307319 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(537..537 src has 463..537) v4 ==== 9254+0+0 (secure 0 0 0) 0x7f5e4807d590 con 0x7f5e580f8420 2024-02-23T00:51:06.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.334+0000 7f5e54ff9700 10 client.41621.objecter ms_dispatch 0x7f5e58001550 osd_map(537..537 src has 463..537) v4 2024-02-23T00:51:06.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.334+0000 7f5e54ff9700 3 client.41621.objecter handle_osd_map got epochs [537,537] > 0 2024-02-23T00:51:06.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.334+0000 7f5e54ff9700 3 client.41621.objecter handle_osd_map decoding full epoch 537 2024-02-23T00:51:06.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.334+0000 7f5e54ff9700 20 client.41621.objecter dump_active .. 0 homeless 2024-02-23T00:51:06.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.334+0000 7f5e56ffd700 1 --2- 172.21.15.47:0/976307319 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5e40062cc0 0x7f5e40065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:06.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.335+0000 7f5e54ff9700 10 client.41621.objecter ms_handle_connect 0x7f5e40062cc0 2024-02-23T00:51:06.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.335+0000 7f5e56ffd700 1 --2- 172.21.15.47:0/976307319 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5e40062cc0 0x7f5e40065180 secure :-1 s=READY pgs=3112 cs=0 l=1 rev1=1 rx=0x7f5e4c007fd0 tx=0x7f5e4c009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:06.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.339+0000 7f5e54ff9700 1 -- 172.21.15.47:0/976307319 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f5e4804cbd0 con 0x7f5e580f8420 2024-02-23T00:51:06.434 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:51:06.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.515+0000 7f5e5db16700 1 -- 172.21.15.47:0/976307319 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5e5805fb30 con 0x7f5e40062cc0 2024-02-23T00:51:06.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.531+0000 7f5e54ff9700 1 -- 172.21.15.47:0/976307319 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+531241 (secure 0 0 0) 0x7f5e5805fb30 con 0x7f5e40062cc0 2024-02-23T00:51:06.541 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:51:06.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.535+0000 7f5e5db16700 10 librados: watch_flush enter 2024-02-23T00:51:06.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.535+0000 7f5e5db16700 10 librados: watch_flush exit 2024-02-23T00:51:06.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.535+0000 7f5e5db16700 20 client.41621.objecter shutdown clearing up homeless session... 2024-02-23T00:51:06.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.535+0000 7f5e5db16700 10 client.41621.objecter successfully canceled tick 2024-02-23T00:51:06.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.535+0000 7f5e5db16700 1 -- 172.21.15.47:0/976307319 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5e40062cc0 msgr2=0x7f5e40065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:06.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.535+0000 7f5e5db16700 1 --2- 172.21.15.47:0/976307319 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5e40062cc0 0x7f5e40065180 secure :-1 s=READY pgs=3112 cs=0 l=1 rev1=1 rx=0x7f5e4c007fd0 tx=0x7f5e4c009cb0).stop 2024-02-23T00:51:06.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.535+0000 7f5e5db16700 1 -- 172.21.15.47:0/976307319 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5e580f8420 msgr2=0x7f5e5810fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:06.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.535+0000 7f5e5db16700 1 --2- 172.21.15.47:0/976307319 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5e580f8420 0x7f5e5810fe60 secure :-1 s=READY pgs=2724 cs=0 l=1 rev1=1 rx=0x7f5e480027f0 tx=0x7f5e48002b00).stop 2024-02-23T00:51:06.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.535+0000 7f5e5db16700 1 -- 172.21.15.47:0/976307319 shutdown_connections 2024-02-23T00:51:06.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.535+0000 7f5e5db16700 1 --2- 172.21.15.47:0/976307319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e580fe510 0x7f5e5810a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:06.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.535+0000 7f5e5db16700 1 --2- 172.21.15.47:0/976307319 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5e40062cc0 0x7f5e40065180 unknown :-1 s=CLOSED pgs=3112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:06.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.535+0000 7f5e5db16700 1 --2- 172.21.15.47:0/976307319 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5e580f8420 0x7f5e5810fe60 unknown :-1 s=CLOSED pgs=2724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:06.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.535+0000 7f5e5db16700 1 --2- 172.21.15.47:0/976307319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e580fdbb0 0x7f5e58110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:06.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.535+0000 7f5e5db16700 1 -- 172.21.15.47:0/976307319 >> 172.21.15.47:0/976307319 conn(0x7f5e580f3800 msgr2=0x7f5e581030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:06.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.535+0000 7f5e5db16700 1 -- 172.21.15.47:0/976307319 shutdown_connections 2024-02-23T00:51:06.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.536+0000 7f5e5db16700 1 -- 172.21.15.47:0/976307319 wait complete. 2024-02-23T00:51:06.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.536+0000 7f5e5db16700 1 librados: shutdown 2024-02-23T00:51:06.547 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-02-23T00:51:06.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:05.217596+0000 osd.3 (osd.3) 256 : cluster [DBG] 4.0 scrub starts 2024-02-23T00:51:06.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:05.221568+0000 osd.3 (osd.3) 257 : cluster [DBG] 4.0 scrub ok 2024-02-23T00:51:06.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:05.337177+0000 mgr.y (mgr.14517) 492 : cluster [DBG] pgmap v808: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-23T00:51:06.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:06.239820+0000 mon.a (mon.0) 2913 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-88","var": "hit_set_count","val": "1"}]': finished 2024-02-23T00:51:06.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:06.239897+0000 mon.a (mon.0) 2914 : cluster [DBG] osdmap e537: 8 total, 8 up, 7 in 2024-02-23T00:51:06.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:06.241451+0000 mon.a (mon.0) 2915 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-88","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:51:06.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.618+0000 7f115f156700 1 -- 172.21.15.47:0/3317763200 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11580fcbb0 msgr2=0x7f11580fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:06.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.618+0000 7f115f156700 1 --2- 172.21.15.47:0/3317763200 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11580fcbb0 0x7f11580fcf90 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 rx=0x7f114c0047b0 tx=0x7f114c009cb0).stop 2024-02-23T00:51:06.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.618+0000 7f115f156700 1 -- 172.21.15.47:0/3317763200 shutdown_connections 2024-02-23T00:51:06.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.618+0000 7f115f156700 1 --2- 172.21.15.47:0/3317763200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11580f7400 0x7f1158103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:06.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.618+0000 7f115f156700 1 --2- 172.21.15.47:0/3317763200 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11580fcbb0 0x7f11580fcf90 unknown :-1 s=CLOSED pgs=2725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:06.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.618+0000 7f115f156700 1 --2- 172.21.15.47:0/3317763200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11580fd510 0x7f11580f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:06.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.618+0000 7f115f156700 1 -- 172.21.15.47:0/3317763200 >> 172.21.15.47:0/3317763200 conn(0x7f11580f2820 msgr2=0x7f11580f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:06.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.618+0000 7f115f156700 1 -- 172.21.15.47:0/3317763200 shutdown_connections 2024-02-23T00:51:06.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.618+0000 7f115f156700 1 -- 172.21.15.47:0/3317763200 wait complete. 2024-02-23T00:51:06.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.619+0000 7f115f156700 1 librados: starting msgr at 2024-02-23T00:51:06.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.619+0000 7f115f156700 1 librados: starting objecter 2024-02-23T00:51:06.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.619+0000 7f115f156700 1 Processor -- start 2024-02-23T00:51:06.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.619+0000 7f115f156700 1 -- start start 2024-02-23T00:51:06.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.619+0000 7f115f156700 1 librados: setting wanted keys 2024-02-23T00:51:06.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.619+0000 7f115f156700 1 librados: calling monclient init 2024-02-23T00:51:06.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.619+0000 7f115f156700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11580f7400 0x7f115810e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:06.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.619+0000 7f115f156700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11580fcbb0 0x7f115810e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:06.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.619+0000 7f115f156700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11580fd510 0x7f1158108230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:06.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.619+0000 7f115f156700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1158108720 con 0x7f11580fd510 2024-02-23T00:51:06.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.619+0000 7f115f156700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1158108890 con 0x7f11580fcbb0 2024-02-23T00:51:06.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.619+0000 7f115f156700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1158108a50 con 0x7f11580f7400 2024-02-23T00:51:06.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.619+0000 7f115cef2700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11580f7400 0x7f115810e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:06.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.619+0000 7f115d6f3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11580fd510 0x7f1158108230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:06.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.619+0000 7f115cef2700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11580f7400 0x7f115810e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:58916/0 (socket says 172.21.15.47:58916) 2024-02-23T00:51:06.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.619+0000 7f115cef2700 1 -- 172.21.15.47:0/3464190293 learned_addr learned my addr 172.21.15.47:0/3464190293 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:51:06.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.619+0000 7f1157fff700 1 --2- 172.21.15.47:0/3464190293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11580fcbb0 0x7f115810e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:06.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.620+0000 7f115cef2700 1 -- 172.21.15.47:0/3464190293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11580fcbb0 msgr2=0x7f115810e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:06.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.620+0000 7f115cef2700 1 --2- 172.21.15.47:0/3464190293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11580fcbb0 0x7f115810e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:06.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.620+0000 7f115cef2700 1 -- 172.21.15.47:0/3464190293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11580fd510 msgr2=0x7f1158108230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:06.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.620+0000 7f115cef2700 1 --2- 172.21.15.47:0/3464190293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11580fd510 0x7f1158108230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:06.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.620+0000 7f115cef2700 1 -- 172.21.15.47:0/3464190293 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1144005040 con 0x7f11580f7400 2024-02-23T00:51:06.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.620+0000 7f115d6f3700 1 --2- 172.21.15.47:0/3464190293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11580fd510 0x7f1158108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:51:06.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.620+0000 7f115cef2700 1 --2- 172.21.15.47:0/3464190293 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11580f7400 0x7f115810e090 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 rx=0x7f114c009f20 tx=0x7f114c009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:06.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.620+0000 7f1155ffb700 10 client.?.objecter ms_handle_connect 0x7f11580f7400 2024-02-23T00:51:06.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.620+0000 7f1155ffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:51:06.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.620+0000 7f1155ffb700 1 -- 172.21.15.47:0/3464190293 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c018070 con 0x7f11580f7400 2024-02-23T00:51:06.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.620+0000 7f115f156700 1 -- 172.21.15.47:0/3464190293 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f114c005040 con 0x7f11580f7400 2024-02-23T00:51:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.620+0000 7f1155ffb700 1 -- 172.21.15.47:0/3464190293 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f114c013dd0 con 0x7f11580f7400 2024-02-23T00:51:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.620+0000 7f1155ffb700 1 -- 172.21.15.47:0/3464190293 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c01d9f0 con 0x7f11580f7400 2024-02-23T00:51:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.620+0000 7f115f156700 10 client.41627.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:51:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.620+0000 7f115f156700 1 -- 172.21.15.47:0/3464190293 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1158108ef0 con 0x7f11580f7400 2024-02-23T00:51:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.620+0000 7f115f156700 1 librados: init done 2024-02-23T00:51:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.621+0000 7f115f156700 1 -- 172.21.15.47:0/3464190293 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11581090e0 con 0x7f11580f7400 2024-02-23T00:51:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.621+0000 7f1155ffb700 1 -- 172.21.15.47:0/3464190293 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f114c009240 con 0x7f11580f7400 2024-02-23T00:51:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.622+0000 7f1155ffb700 1 --2- 172.21.15.47:0/3464190293 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1148062cc0 0x7f1148065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.622+0000 7f1155ffb700 1 -- 172.21.15.47:0/3464190293 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(537..537 src has 463..537) v4 ==== 9254+0+0 (secure 0 0 0) 0x7f114c07e8f0 con 0x7f11580f7400 2024-02-23T00:51:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.622+0000 7f1155ffb700 10 client.41627.objecter ms_dispatch 0x7f1158001550 osd_map(537..537 src has 463..537) v4 2024-02-23T00:51:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.622+0000 7f1155ffb700 3 client.41627.objecter handle_osd_map got epochs [537,537] > 0 2024-02-23T00:51:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.622+0000 7f1155ffb700 3 client.41627.objecter handle_osd_map decoding full epoch 537 2024-02-23T00:51:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.622+0000 7f1157fff700 1 --2- 172.21.15.47:0/3464190293 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1148062cc0 0x7f1148065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.622+0000 7f1155ffb700 20 client.41627.objecter dump_active .. 0 homeless 2024-02-23T00:51:06.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.622+0000 7f1155ffb700 10 client.41627.objecter ms_handle_connect 0x7f1148062cc0 2024-02-23T00:51:06.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.622+0000 7f1157fff700 1 --2- 172.21.15.47:0/3464190293 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1148062cc0 0x7f1148065180 secure :-1 s=READY pgs=3113 cs=0 l=1 rev1=1 rx=0x7f11581099d0 tx=0x7f1144009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:06.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.627+0000 7f1155ffb700 1 -- 172.21.15.47:0/3464190293 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f114c04dee0 con 0x7f11580f7400 2024-02-23T00:51:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:05.217596+0000 osd.3 (osd.3) 256 : cluster [DBG] 4.0 scrub starts 2024-02-23T00:51:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:05.221568+0000 osd.3 (osd.3) 257 : cluster [DBG] 4.0 scrub ok 2024-02-23T00:51:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:05.337177+0000 mgr.y (mgr.14517) 492 : cluster [DBG] pgmap v808: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-23T00:51:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:06.239820+0000 mon.a (mon.0) 2913 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-88","var": "hit_set_count","val": "1"}]': finished 2024-02-23T00:51:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:06.239897+0000 mon.a (mon.0) 2914 : cluster [DBG] osdmap e537: 8 total, 8 up, 7 in 2024-02-23T00:51:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:06.241451+0000 mon.a (mon.0) 2915 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-88","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:51:06.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:06.818+0000 7f115f156700 1 -- 172.21.15.47:0/3464190293 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7f1158105ea0 con 0x7f11580f7400 2024-02-23T00:51:07.179 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:51:07.204 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":809,"stamp":"2024-02-23T00:51:06.236868+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4026236,"num_objects":417,"num_object_clones":11,"num_object_copies":1251,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":417,"num_whiteouts":0,"num_read":2444,"num_read_kb":2010,"num_write":1690,"num_write_kb":4667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":440,"num_bytes_recovered":3479,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":21,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3122,"ondisk_log_size":3122,"up":624,"acting":624,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":575,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":2175360,"kb_used_data":28456,"kb_used_omap":0,"kb_used_meta":2146048,"kb_avail":654040704,"statfs":{"total":671965249536,"available":669737680896,"internally_reserved":0,"allocated":29138944,"data_stored":17546148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2197553152},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.996526"},"pg_stats":[{"pgid":"226.1f","version":"0'0","reported_seq":9,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.256240+0000","last_change":"2024-02-23T00:51:01.256240+0000","last_active":"2024-02-23T00:51:01.256240+0000","last_peered":"2024-02-23T00:51:01.256240+0000","last_clean":"2024-02-23T00:51:01.256240+0000","last_became_active":"2024-02-23T00:51:01.255954+0000","last_became_peered":"2024-02-23T00:51:01.255954+0000","last_unstale":"2024-02-23T00:51:01.256240+0000","last_undegraded":"2024-02-23T00:51:01.256240+0000","last_fullsized":"2024-02-23T00:51:01.256240+0000","mapping_epoch":531,"log_start":"0'0","ondisk_log_start":"0'0","created":531,"last_epoch_clean":532,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.231061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"226.1e","version":"0'0","reported_seq":9,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.254869+0000","last_change":"2024-02-23T00:51:01.254869+0000","last_active":"2024-02-23T00:51:01.254869+0000","last_peered":"2024-02-23T00:51:01.254869+0000","last_clean":"2024-02-23T00:51:01.254869+0000","last_became_active":"2024-02-23T00:51:01.254532+0000","last_became_peered":"2024-02-23T00:51:01.254532+0000","last_unstale":"2024-02-23T00:51:01.254869+0000","last_undegraded":"2024-02-23T00:51:01.254869+0000","last_fullsized":"2024-02-23T00:51:01.254869+0000","mapping_epoch":531,"log_start":"0'0","ondisk_log_start":"0'0","created":531,"last_epoch_clean":532,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.231061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"226.1d","version":"0'0","reported_seq":9,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.255008+0000","last_change":"2024-02-23T00:51:01.255008+0000","last_active":"2024-02-23T00:51:01.255008+0000","last_peered":"2024-02-23T00:51:01.255008+0000","last_clean":"2024-02-23T00:51:01.255008+0000","last_became_active":"2024-02-23T00:51:01.254693+0000","last_became_peered":"2024-02-23T00:51:01.254693+0000","last_unstale":"2024-02-23T00:51:01.255008+0000","last_undegraded":"2024-02-23T00:51:01.255008+0000","last_fullsized":"2024-02-23T00:51:01.255008+0000","mapping_epoch":531,"log_start":"0'0","ondisk_log_start":"0'0","created":531,"last_epoch_clean":532,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.231061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"226.1c","version":"0'0","reported_seq":9,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.256309+0000","last_change":"2024-02-23T00:51:01.256309+0000","last_active":"2024-02-23T00:51:01.256309+0000","last_peered":"2024-02-23T00:51:01.256309+0000","last_clean":"2024-02-23T00:51:01.256309+0000","last_became_active":"2024-02-23T00:51:01.255790+0000","last_became_peered":"2024-02-23T00:51:01.255790+0000","last_unstale":"2024-02-23T00:51:01.256309+0000","last_undegraded":"2024-02-23T00:51:01.256309+0000","last_fullsized":"2024-02-23T00:51:01.256309+0000","mapping_epoch":531,"log_start":"0'0","ondisk_log_start":"0'0","created":531,"last_epoch_clean":532,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.231061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"226.1b","version":"0'0","reported_seq":9,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.243392+0000","last_change":"2024-02-23T00:51:01.243392+0000","last_active":"2024-02-23T00:51:01.243392+0000","last_peered":"2024-02-23T00:51:01.243392+0000","last_clean":"2024-02-23T00:51:01.243392+0000","last_became_active":"2024-02-23T00:51:01.243172+0000","last_became_peered":"2024-02-23T00:51:01.243172+0000","last_unstale":"2024-02-23T00:51:01.243392+0000","last_undegraded":"2024-02-23T00:51:01.243392+0000","last_fullsized":"2024-02-23T00:51:01.243392+0000","mapping_epoch":531,"log_start":"0'0","ondisk_log_start":"0'0","created":531,"last_epoch_clean":532,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.231061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"226.1a","version":"0'0","reported_seq":9,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.257002+0000","last_change":"2024-02-23T00:51:01.257002+0000","last_active":"2024-02-23T00:51:01.257002+0000","last_peered":"2024-02-23T00:51:01.257002+0000","last_clean":"2024-02-23T00:51:01.257002+0000","last_became_active":"2024-02-23T00:51:01.256680+0000","last_became_peered":"2024-02-23T00:51:01.256680+0000","last_unstale":"2024-02-23T00:51:01.257002+0000","last_undegraded":"2024-02-23T00:51:01.257002+0000","last_fullsized":"2024-02-23T00:51:01.257002+0000","mapping_epoch":531,"log_start":"0'0","ondisk_log_start":"0'0","created":531,"last_epoch_clean":532,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.231061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"226.19","version":"0'0","reported_seq":9,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.261921+0000","last_change":"2024-02-23T00:51:01.261921+0000","last_active":"2024-02-23T00:51:01.261921+0000","last_peered":"2024-02-23T00:51:01.261921+0000","last_clean":"2024-02-23T00:51:01.261921+0000","last_became_active":"2024-02-23T00:51:01.261545+0000","last_became_peered":"2024-02-23T00:51:01.261545+0000","last_unstale":"2024-02-23T00:51:01.261921+0000","last_undegraded":"2024-02-23T00:51:01.261921+0000","last_fullsized":"2024-02-23T00:51:01.261921+0000","mapping_epoch":531,"log_start":"0'0","ondisk_log_start":"0'0","created":531,"last_epoch_clean":532,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.231061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"226.18","version":"0'0","reported_seq":9,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.256896+0000","last_change":"2024-02-23T00:51:01.256896+0000","last_active":"2024-02-23T00:51:01.256896+0000","last_peered":"2024-02-23T00:51:01.256896+0000","last_clean":"2024-02-23T00:51:01.256896+0000","last_became_active":"2024-02-23T00:51:01.256574+0000","last_became_peered":"2024-02-23T00:51:01.256574+0000","last_unstale":"2024-02-23T00:51:01.256896+0000","last_undegraded":"2024-02-23T00:51:01.256896+0000","last_fullsized":"2024-02-23T00:51:01.256896+0000","mapping_epoch":531,"log_start":"0'0","ondisk_log_start":"0'0","created":531,"last_epoch_clean":532,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.231061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,7],"acting":[5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"226.17","version":"0'0","reported_seq":9,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.255889+0000","last_change":"2024-02-23T00:51:01.255889+0000","last_active":"2024-02-23T00:51:01.255889+0000","last_peered":"2024-02-23T00:51:01.255889+0000","last_clean":"2024-02-23T00:51:01.255889+0000","last_became_active":"2024-02-23T00:51:01.255591+0000","last_became_peered":"2024-02-23T00:51:01.255591+0000","last_unstale":"2024-02-23T00:51:01.255889+0000","last_undegraded":"2024-02-23T00:51:01.255889+0000","last_fullsized":"2024-02-23T00:51:01.255889+0000","mapping_epoch":531,"log_start":"0'0","ondisk_log_start":"0'0","created":531,"last_epoch_clean":532,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.231061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"226.16","version":"0'0","reported_seq":9,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.251014+0000","last_change":"2024-02-23T00:51:01.251014+0000","last_active":"2024-02-23T00:51:01.251014+0000","last_peered":"2024-02-23T00:51:01.251014+0000","last_clean":"2024-02-23T00:51:01.251014+0000","last_became_active":"2024-02-23T00:51:01.250696+0000","last_became_peered":"2024-02-23T00:51:01.250696+0000","last_unstale":"2024-02-23T00:51:01.251014+0000","last_undegraded":"2024-02-23T00:51:01.251014+0000","last_fullsized":"2024-02-23T00:51:01.251014+0000","mapping_epoch":531,"log_start":"0'0","ondisk_log_start":"0'0","created":531,"last_epoch_clean":532,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.231061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"226.15","version":"0'0","reported_seq":9,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.258255+0000","last_change":"2024-02-23T00:51:01.258255+0000","last_active":"2024-02-23T00:51:01.258255+0000","last_peered":"2024-02-23T00:51:01.258255+0000","last_clean":"2024-02-23T00:51:01.258255+0000","last_became_active":"2024-02-23T00:51:01.257907+0000","last_became_peered":"2024-02-23T00:51:01.257907+0000","last_unstale":"2024-02-23T00:51:01.258255+0000","last_undegraded":"2024-02-23T00:51:01.258255+0000","last_fullsized":"2024-02-23T00:51:01.258255+0000","mapping_epoch":531,"log_start":"0'0","ondisk_log_start":"0'0","created":531,"last_epoch_clean":532,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.231061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"226.14","version":"0'0","reported_seq":9,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.243501+0000","last_change":"2024-02-23T00:51:01.243501+0000","last_active":"2024-02-23T00:51:01.243501+0000","last_peered":"2024-02-23T00:51:01.243501+0000","last_clean":"2024-02-23T00:51:01.243501+0000","last_became_active":"2024-02-23T00:51:01.243169+0000","last_became_peered":"2024-02-23T00:51:01.243169+0000","last_unstale":"2024-02-23T00:51:01.243501+0000","last_undegraded":"2024-02-23T00:51:01.243501+0000","last_fullsized":"2024-02-23T00:51:01.243501+0000","mapping_epoch":531,"log_start":"0'0","ondisk_log_start":"0'0","created":531,"last_epoch_clean":532,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.231061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"226.13","version":"0'0","reported_seq":9,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.259307+0000","last_change":"2024-02-23T00:51:01.259307+0000","last_active":"2024-02-23T00:51:01.259307+0000","last_peered":"2024-02-23T00:51:01.259307+0000","last_clean":"2024-02-23T00:51:01.259307+0000","last_became_active":"2024-02-23T00:51:01.258958+0000","last_became_peered":"2024-02-23T00:51:01.258958+0000","last_unstale":"2024-02-23T00:51:01.259307+0000","last_undegraded":"2024-02-23T00:51:01.259307+0000","last_fullsized":"2024-02-23T00:51:01.259307+0000","mapping_epoch":531,"log_start":"0'0","ondisk_log_start":"0'0","created":531,"last_epoch_clean":532,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.231061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"226.12","version":"0'0","reported_seq":9,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.254634+0000","last_change":"2024-02-23T00:51:01.254634+0000","last_active":"2024-02-23T00:51:01.254634+0000","last_peered":"2024-02-23T00:51:01.254634+0000","last_clean":"2024-02-23T00:51:01.254634+0000","last_became_active":"2024-02-23T00:51:01.254315+0000","last_became_peered":"2024-02-23T00:51:01.254315+0000","last_unstale":"2024-02-23T00:51:01.254634+0000","last_undegraded":"2024-02-23T00:51:01.254634+0000","last_fullsized":"2024-02-23T00:51:01.254634+0000","mapping_epoch":531,"log_start":"0'0","ondisk_log_start":"0'0","created":531,"last_epoch_clean":532,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.231061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"226.11","version":"0'0","reported_seq":9,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.259343+0000","last_change":"2024-02-23T00:51:01.259343+0000","last_active":"2024-02-23T00:51:01.259343+0000","last_peered":"2024-02-23T00:51:01.259343+0000","last_clean":"2024-02-23T00:51:01.259343+0000","last_became_active":"2024-02-23T00:51:01.258851+0000","last_became_peered":"2024-02-23T00:51:01.258851+0000","last_unstale":"2024-02-23T00:51:01.259343+0000","last_undegraded":"2024-02-23T00:51:01.259343+0000","last_fullsized":"2024-02-23T00:51:01.259343+0000","mapping_epoch":531,"log_start":"0'0","ondisk_log_start":"0'0","created":531,"last_epoch_clean":532,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.231061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"226.10","version":"0'0","reported_seq":9,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.261958+0000","last_change":"2024-02-23T00:51:01.261958+0000","last_active":"2024-02-23T00:51:01.261958+0000","last_peered":"2024-02-23T00:51:01.261958+0000","last_clean":"2024-02-23T00:51:01.261958+0000","last_became_active":"2024-02-23T00:51:01.261643+0000","last_became_peered":"2024-02-23T00:51:01.261643+0000","last_unstale":"2024-02-23T00:51:01.261958+0000","last_undegraded":"2024-02-23T00:51:01.261958+0000","last_fullsized":"2024-02-23T00:51:01.261958+0000","mapping_epoch":531,"log_start":"0'0","ondisk_log_start":"0'0","created":531,"last_epoch_clean":532,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.231061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"226.f","version":"0'0","reported_seq":10,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.253412+0000","last_change":"2024-02-23T00:51:01.253179+0000","last_active":"2024-02-23T00:51:01.253412+0000","last_peered":"2024-02-23T00:51:01.253412+0000","last_clean":"2024-02-23T00:51:01.253412+0000","last_became_active":"2024-02-23T00:51:01.252868+0000","last_became_peered":"2024-02-23T00:51:01.252868+0000","last_unstale":"2024-02-23T00:51:01.253412+0000","last_undegraded":"2024-02-23T00:51:01.253412+0000","last_fullsized":"2024-02-23T00:51:01.253412+0000","mapping_epoch":531,"log_start":"0'0","ondisk_log_start":"0'0","created":531,"last_epoch_clean":532,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.231061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"226.e","version":"0'0","reported_seq":9,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.255312+0000","last_change":"2024-02-23T00:51:01.255312+0000","last_active":"2024-02-23T00:51:01.255312+0000","last_peered":"2024-02-23T00:51:01.255312+0000","last_clean":"2024-02-23T00:51:01.255312+0000","last_became_active":"2024-02-23T00:51:01.254882+0000","last_became_peered":"2024-02-23T00:51:01.254882+0000","last_unstale":"2024-02-23T00:51:01.255312+0000","last_undegraded":"2024-02-23T00:51:01.255312+0000","last_fullsized":"2024-02-23T00:51:01.255312+0000","mapping_epoch":531,"log_start":"0'0","ondisk_log_start":"0'0","created":531,"last_epoch_clean":532,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.231061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"226.d","version":"0'0","reported_seq":10,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.254081+0000","last_change":"2024-02-23T00:51:01.253907+0000","last_active":"2024-02-23T00:51:01.254081+0000","last_peered":"2024-02-23T00:51:01.254081+0000","last_clean":"2024-02-23T00:51:01.254081+0000","last_became_active":"2024-02-23T00:51:01.253560+0000","last_became_peered":"2024-02-23T00:51:01.253560+0000","last_unstale":"2024-02-23T00:51:01.254081+0000","last_undegraded":"2024-02-23T00:51:01.254081+0000","last_fullsized":"2024-02-23T00:51:01.254081+0000","mapping_epoch":531,"log_start":"0'0","ondisk_log_start":"0'0","created":531,"last_epoch_clean":532,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.231061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"226.c","version":"0'0","reported_seq":9,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.254664+0000","last_change":"2024-02-23T00:51:01.254664+0000","last_active":"2024-02-23T00:51:01.254664+0000","last_peered":"2024-02-23T00:51:01.254664+0000","last_clean":"2024-02-23T00:51:01.254664+0000","last_became_active":"2024-02-23T00:51:01.254331+0000","last_became_peered":"2024-02-23T00:51:01.254331+0000","last_unstale":"2024-02-23T00:51:01.254664+0000","last_undegraded":"2024-02-23T00:51:01.254664+0000","last_fullsized":"2024-02-23T00:51:01.254664+0000","mapping_epoch":531,"log_start":"0'0","ondisk_log_start":"0'0","created":531,"last_epoch_clean":532,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.231061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"226.b","version":"0'0","reported_seq":9,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.259694+0000","last_change":"2024-02-23T00:51:01.259694+0000","last_active":"2024-02-23T00:51:01.259694+0000","last_peered":"2024-02-23T00:51:01.259694+0000","last_clean":"2024-02-23T00:51:01.259694+0000","last_became_active":"2024-02-23T00:51:01.259404+0000","last_became_peered":"2024-02-23T00:51:01.259404+0000","last_unstale":"2024-02-23T00:51:01.259694+0000","last_undegraded":"2024-02-23T00:51:01.259694+0000","last_fullsized":"2024-02-23T00:51:01.259694+0000","mapping_epoch":531,"log_start":"0'0","ondisk_log_start":"0'0","created":531,"last_epoch_clean":532,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.231061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"226.a","version":"0'0","reported_seq":9,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.252682+0000","last_change":"2024-02-23T00:51:01.252682+0000","last_active":"2024-02-23T00:51:01.252682+0000","last_peered":"2024-02-23T00:51:01.252682+0000","last_clean":"2024-02-23T00:51:01.252682+0000","last_became_active":"2024-02-23T00:51:01.252367+0000","last_became_peered":"2024-02-23T00:51:01.252367+0000","last_unstale":"2024-02-23T00:51:01.252682+0000","last_undegraded":"2024-02-23T00:51:01.252682+0000","last_fullsized":"2024-02-23T00:51:01.252682+0000","mapping_epoch":531,"log_start":"0'0","ondisk_log_start":"0'0","created":531,"last_epoch_clean":532,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.231061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"226.9","version":"0'0","reported_seq":9,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.253753+0000","last_change":"2024-02-23T00:51:01.253753+0000","last_active":"2024-02-23T00:51:01.253753+0000","last_peered":"2024-02-23T00:51:01.253753+0000","last_clean":"2024-02-23T00:51:01.253753+0000","last_became_active":"2024-02-23T00:51:01.253419+0000","last_became_peered":"2024-02-23T00:51:01.253419+0000","last_unstale":"2024-02-23T00:51:01.253753+0000","last_undegraded":"2024-02-23T00:51:01.253753+0000","last_fullsized":"2024-02-23T00:51:01.253753+0000","mapping_epoch":531,"log_start":"0'0","ondisk_log_start":"0'0","created":531,"last_epoch_clean":532,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.231061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"226.8","version":"0'0","reported_seq":9,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.255202+0000","last_change":"2024-02-23T00:51:01.255202+0000","last_active":"2024-02-23T00:51:01.255202+0000","last_peered":"2024-02-23T00:51:01.255202+0000","last_clean":"2024-02-23T00:51:01.255202+0000","last_became_active":"2024-02-23T00:51:01.254880+0000","last_became_peered":"2024-02-23T00:51:01.254880+0000","last_unstale":"2024-02-23T00:51:01.255202+0000","last_undegraded":"2024-02-23T00:51:01.255202+0000","last_fullsized":"2024-02-23T00:51:01.255202+0000","mapping_epoch":531,"log_start":"0'0","ondisk_log_start":"0'0","created":531,"last_epoch_clean":532,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.231061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"226.7","version":"0'0","reported_seq":9,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.255038+0000","last_change":"2024-02-23T00:51:01.255038+0000","last_active":"2024-02-23T00:51:01.255038+0000","last_peered":"2024-02-23T00:51:01.255038+0000","last_clean":"2024-02-23T00:51:01.255038+0000","last_became_active":"2024-02-23T00:51:01.254094+0000","last_became_peered":"2024-02-23T00:51:01.254094+0000","last_unstale":"2024-02-23T00:51:01.255038+0000","last_undegraded":"2024-02-23T00:51:01.255038+0000","last_fullsized":"2024-02-23T00:51:01.255038+0000","mapping_epoch":531,"log_start":"0'0","ondisk_log_start":"0'0","created":531,"last_epoch_clean":532,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.231061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"226.6","version":"0'0","reported_seq":9,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.260302+0000","last_change":"2024-02-23T00:51:01.260302+0000","last_active":"2024-02-23T00:51:01.260302+0000","last_peered":"2024-02-23T00:51:01.260302+0000","last_clean":"2024-02-23T00:51:01.260302+0000","last_became_active":"2024-02-23T00:51:01.259948+0000","last_became_peered":"2024-02-23T00:51:01.259948+0000","last_unstale":"2024-02-23T00:51:01.260302+0000","last_undegraded":"2024-02-23T00:51:01.260302+0000","last_fullsized":"2024-02-23T00:51:01.260302+0000","mapping_epoch":531,"log_start":"0'0","ondisk_log_start":"0'0","created":531,"last_epoch_clean":532,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.231061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"226.5","version":"0'0","reported_seq":9,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.255954+0000","last_change":"2024-02-23T00:51:01.255954+0000","last_active":"2024-02-23T00:51:01.255954+0000","last_peered":"2024-02-23T00:51:01.255954+0000","last_clean":"2024-02-23T00:51:01.255954+0000","last_became_active":"2024-02-23T00:51:01.255662+0000","last_became_peered":"2024-02-23T00:51:01.255662+0000","last_unstale":"2024-02-23T00:51:01.255954+0000","last_undegraded":"2024-02-23T00:51:01.255954+0000","last_fullsized":"2024-02-23T00:51:01.255954+0000","mapping_epoch":531,"log_start":"0'0","ondisk_log_start":"0'0","created":531,"last_epoch_clean":532,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.231061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"226.4","version":"0'0","reported_seq":9,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.253584+0000","last_change":"2024-02-23T00:51:01.253584+0000","last_active":"2024-02-23T00:51:01.253584+0000","last_peered":"2024-02-23T00:51:01.253584+0000","last_clean":"2024-02-23T00:51:01.253584+0000","last_became_active":"2024-02-23T00:51:01.253264+0000","last_became_peered":"2024-02-23T00:51:01.253264+0000","last_unstale":"2024-02-23T00:51:01.253584+0000","last_undegraded":"2024-02-23T00:51:01.253584+0000","last_fullsized":"2024-02-23T00:51:01.253584+0000","mapping_epoch":531,"log_start":"0'0","ondisk_log_start":"0'0","created":531,"last_epoch_clean":532,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.231061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"226.3","version":"0'0","reported_seq":9,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.262040+0000","last_change":"2024-02-23T00:51:01.262040+0000","last_active":"2024-02-23T00:51:01.262040+0000","last_peered":"2024-02-23T00:51:01.262040+0000","last_clean":"2024-02-23T00:51:01.262040+0000","last_became_active":"2024-02-23T00:51:01.261659+0000","last_became_peered":"2024-02-23T00:51:01.261659+0000","last_unstale":"2024-02-23T00:51:01.262040+0000","last_undegraded":"2024-02-23T00:51:01.262040+0000","last_fullsized":"2024-02-23T00:51:01.262040+0000","mapping_epoch":531,"log_start":"0'0","ondisk_log_start":"0'0","created":531,"last_epoch_clean":532,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.231061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"226.2","version":"0'0","reported_seq":9,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.256277+0000","last_change":"2024-02-23T00:51:01.256277+0000","last_active":"2024-02-23T00:51:01.256277+0000","last_peered":"2024-02-23T00:51:01.256277+0000","last_clean":"2024-02-23T00:51:01.256277+0000","last_became_active":"2024-02-23T00:51:01.255573+0000","last_became_peered":"2024-02-23T00:51:01.255573+0000","last_unstale":"2024-02-23T00:51:01.256277+0000","last_undegraded":"2024-02-23T00:51:01.256277+0000","last_fullsized":"2024-02-23T00:51:01.256277+0000","mapping_epoch":531,"log_start":"0'0","ondisk_log_start":"0'0","created":531,"last_epoch_clean":532,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.231061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"226.1","version":"0'0","reported_seq":9,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.252712+0000","last_change":"2024-02-23T00:51:01.252712+0000","last_active":"2024-02-23T00:51:01.252712+0000","last_peered":"2024-02-23T00:51:01.252712+0000","last_clean":"2024-02-23T00:51:01.252712+0000","last_became_active":"2024-02-23T00:51:01.252352+0000","last_became_peered":"2024-02-23T00:51:01.252352+0000","last_unstale":"2024-02-23T00:51:01.252712+0000","last_undegraded":"2024-02-23T00:51:01.252712+0000","last_fullsized":"2024-02-23T00:51:01.252712+0000","mapping_epoch":531,"log_start":"0'0","ondisk_log_start":"0'0","created":531,"last_epoch_clean":532,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.231061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"226.0","version":"0'0","reported_seq":9,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.254845+0000","last_change":"2024-02-23T00:51:01.254845+0000","last_active":"2024-02-23T00:51:01.254845+0000","last_peered":"2024-02-23T00:51:01.254845+0000","last_clean":"2024-02-23T00:51:01.254845+0000","last_became_active":"2024-02-23T00:51:01.254504+0000","last_became_peered":"2024-02-23T00:51:01.254504+0000","last_unstale":"2024-02-23T00:51:01.254845+0000","last_undegraded":"2024-02-23T00:51:01.254845+0000","last_fullsized":"2024-02-23T00:51:01.254845+0000","mapping_epoch":531,"log_start":"0'0","ondisk_log_start":"0'0","created":531,"last_epoch_clean":532,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:00.231061+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.231061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.0","version":"259'40","reported_seq":568,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:02.197067+0000","last_change":"2024-02-23T00:51:02.197067+0000","last_active":"2024-02-23T00:51:02.197067+0000","last_peered":"2024-02-23T00:51:02.197067+0000","last_clean":"2024-02-23T00:51:02.197067+0000","last_became_active":"2024-02-23T00:47:28.238034+0000","last_became_peered":"2024-02-23T00:47:28.238034+0000","last_unstale":"2024-02-23T00:51:02.197067+0000","last_undegraded":"2024-02-23T00:51:02.197067+0000","last_fullsized":"2024-02-23T00:51:02.197067+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'40","last_scrub_stamp":"2024-02-23T00:51:02.196968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:51:02.196968+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":31,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":1,"num_write":13,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":31,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.1","version":"495'89","reported_seq":563,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.239726+0000","last_change":"2024-02-23T00:50:56.478515+0000","last_active":"2024-02-23T00:51:00.239726+0000","last_peered":"2024-02-23T00:51:00.239726+0000","last_clean":"2024-02-23T00:51:00.239726+0000","last_became_active":"2024-02-23T00:43:17.851866+0000","last_became_peered":"2024-02-23T00:43:17.851866+0000","last_unstale":"2024-02-23T00:51:00.239726+0000","last_undegraded":"2024-02-23T00:51:00.239726+0000","last_fullsized":"2024-02-23T00:51:00.239726+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"495'89","last_scrub_stamp":"2024-02-23T00:50:56.478433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:56.478433+0000","log_size":89,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":63,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7,"num_read_kb":1,"num_write":30,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.2","version":"514'72","reported_seq":553,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.242194+0000","last_change":"2024-02-23T00:50:30.283941+0000","last_active":"2024-02-23T00:51:00.242194+0000","last_peered":"2024-02-23T00:51:00.242194+0000","last_clean":"2024-02-23T00:51:00.242194+0000","last_became_active":"2024-02-23T00:43:17.804792+0000","last_became_peered":"2024-02-23T00:43:17.804792+0000","last_unstale":"2024-02-23T00:51:00.242194+0000","last_undegraded":"2024-02-23T00:51:00.242194+0000","last_fullsized":"2024-02-23T00:51:00.242194+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'68","last_scrub_stamp":"2024-02-23T00:50:30.283820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:30.283820+0000","log_size":72,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1265,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":9,"num_read_kb":2,"num_write":27,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.3","version":"505'27","reported_seq":391,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.240533+0000","last_change":"2024-02-23T00:49:28.475170+0000","last_active":"2024-02-23T00:51:00.240533+0000","last_peered":"2024-02-23T00:51:00.240533+0000","last_clean":"2024-02-23T00:51:00.240533+0000","last_became_active":"2024-02-23T00:47:28.226936+0000","last_became_peered":"2024-02-23T00:47:28.226936+0000","last_unstale":"2024-02-23T00:51:00.240533+0000","last_undegraded":"2024-02-23T00:51:00.240533+0000","last_fullsized":"2024-02-23T00:51:00.240533+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"246'20","last_scrub_stamp":"2024-02-23T00:49:28.475056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:49:28.475056+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":4,"num_write":16,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":24,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.4","version":"514'24","reported_seq":507,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.240372+0000","last_change":"2024-02-23T00:48:56.703323+0000","last_active":"2024-02-23T00:51:00.240372+0000","last_peered":"2024-02-23T00:51:00.240372+0000","last_clean":"2024-02-23T00:51:00.240372+0000","last_became_active":"2024-02-23T00:43:17.810343+0000","last_became_peered":"2024-02-23T00:43:17.810343+0000","last_unstale":"2024-02-23T00:51:00.240372+0000","last_undegraded":"2024-02-23T00:51:00.240372+0000","last_fullsized":"2024-02-23T00:51:00.240372+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"207'17","last_scrub_stamp":"2024-02-23T00:48:56.703243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:48:56.703243+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":14,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.5","version":"495'55","reported_seq":538,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.237100+0000","last_change":"2024-02-23T00:49:26.431068+0000","last_active":"2024-02-23T00:51:00.237100+0000","last_peered":"2024-02-23T00:51:00.237100+0000","last_clean":"2024-02-23T00:51:00.237100+0000","last_became_active":"2024-02-23T00:43:17.855339+0000","last_became_peered":"2024-02-23T00:43:17.855339+0000","last_unstale":"2024-02-23T00:51:00.237100+0000","last_undegraded":"2024-02-23T00:51:00.237100+0000","last_fullsized":"2024-02-23T00:51:00.237100+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'51","last_scrub_stamp":"2024-02-23T00:49:26.430990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:49:26.430990+0000","log_size":55,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":2,"num_write":21,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.6","version":"514'55","reported_seq":593,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.248832+0000","last_change":"2024-02-23T00:50:21.294652+0000","last_active":"2024-02-23T00:51:00.248832+0000","last_peered":"2024-02-23T00:51:00.248832+0000","last_clean":"2024-02-23T00:51:00.248832+0000","last_became_active":"2024-02-23T00:47:28.236471+0000","last_became_peered":"2024-02-23T00:47:28.236471+0000","last_unstale":"2024-02-23T00:51:00.248832+0000","last_undegraded":"2024-02-23T00:51:00.248832+0000","last_fullsized":"2024-02-23T00:51:00.248832+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"379'54","last_scrub_stamp":"2024-02-23T00:50:21.294457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:21.294457+0000","log_size":55,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1272,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":7,"num_read_kb":1,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":41,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.7","version":"514'77","reported_seq":576,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.242510+0000","last_change":"2024-02-23T00:50:28.305136+0000","last_active":"2024-02-23T00:51:00.242510+0000","last_peered":"2024-02-23T00:51:00.242510+0000","last_clean":"2024-02-23T00:51:00.242510+0000","last_became_active":"2024-02-23T00:47:28.240920+0000","last_became_peered":"2024-02-23T00:47:28.240920+0000","last_unstale":"2024-02-23T00:51:00.242510+0000","last_undegraded":"2024-02-23T00:51:00.242510+0000","last_fullsized":"2024-02-23T00:51:00.242510+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"499'76","last_scrub_stamp":"2024-02-23T00:50:28.305026+0000","last_deep_scrub":"499'76","last_deep_scrub_stamp":"2024-02-23T00:50:28.305026+0000","last_clean_scrub_stamp":"2024-02-23T00:50:28.305026+0000","log_size":77,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":573,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":26,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":52,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.8","version":"514'134","reported_seq":672,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.247292+0000","last_change":"2024-02-23T00:48:53.698549+0000","last_active":"2024-02-23T00:51:00.247292+0000","last_peered":"2024-02-23T00:51:00.247292+0000","last_clean":"2024-02-23T00:51:00.247292+0000","last_became_active":"2024-02-23T00:47:28.229780+0000","last_became_peered":"2024-02-23T00:47:28.229780+0000","last_unstale":"2024-02-23T00:51:00.247292+0000","last_undegraded":"2024-02-23T00:51:00.247292+0000","last_fullsized":"2024-02-23T00:51:00.247292+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"440'114","last_scrub_stamp":"2024-02-23T00:48:53.698466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:48:53.698466+0000","log_size":134,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":816,"num_objects":14,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":9,"num_read_kb":3,"num_write":51,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":68,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":4,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.9","version":"529'69","reported_seq":594,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.236639+0000","last_change":"2024-02-23T00:49:00.243151+0000","last_active":"2024-02-23T00:51:00.236639+0000","last_peered":"2024-02-23T00:51:00.236639+0000","last_clean":"2024-02-23T00:51:00.236639+0000","last_became_active":"2024-02-23T00:47:28.202025+0000","last_became_peered":"2024-02-23T00:47:28.202025+0000","last_unstale":"2024-02-23T00:51:00.236639+0000","last_undegraded":"2024-02-23T00:51:00.236639+0000","last_fullsized":"2024-02-23T00:51:00.236639+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'39","last_scrub_stamp":"2024-02-23T00:49:00.243068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:49:00.243068+0000","log_size":69,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1285,"num_objects":10,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":29,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":27,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":6,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.a","version":"505'92","reported_seq":637,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.247905+0000","last_change":"2024-02-23T00:50:30.147956+0000","last_active":"2024-02-23T00:51:00.247905+0000","last_peered":"2024-02-23T00:51:00.247905+0000","last_clean":"2024-02-23T00:51:00.247905+0000","last_became_active":"2024-02-23T00:47:28.241187+0000","last_became_peered":"2024-02-23T00:47:28.241187+0000","last_unstale":"2024-02-23T00:51:00.247905+0000","last_undegraded":"2024-02-23T00:51:00.247905+0000","last_fullsized":"2024-02-23T00:51:00.247905+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'91","last_scrub_stamp":"2024-02-23T00:50:30.147873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:30.147873+0000","log_size":92,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1283,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":30,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":68,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.b","version":"514'91","reported_seq":570,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.247782+0000","last_change":"2024-02-23T00:50:59.132747+0000","last_active":"2024-02-23T00:51:00.247782+0000","last_peered":"2024-02-23T00:51:00.247782+0000","last_clean":"2024-02-23T00:51:00.247782+0000","last_became_active":"2024-02-23T00:43:17.855321+0000","last_became_peered":"2024-02-23T00:43:17.855321+0000","last_unstale":"2024-02-23T00:51:00.247782+0000","last_undegraded":"2024-02-23T00:51:00.247782+0000","last_fullsized":"2024-02-23T00:51:00.247782+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"514'91","last_scrub_stamp":"2024-02-23T00:50:59.132670+0000","last_deep_scrub":"418'84","last_deep_scrub_stamp":"2024-02-23T00:48:10.926499+0000","last_clean_scrub_stamp":"2024-02-23T00:50:59.132670+0000","log_size":91,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":621,"num_objects":8,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":13,"num_read_kb":6,"num_write":35,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.c","version":"495'43","reported_seq":530,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.237296+0000","last_change":"2024-02-23T00:49:06.040399+0000","last_active":"2024-02-23T00:51:01.237296+0000","last_peered":"2024-02-23T00:51:01.237296+0000","last_clean":"2024-02-23T00:51:01.237296+0000","last_became_active":"2024-02-23T00:43:17.829044+0000","last_became_peered":"2024-02-23T00:43:17.829044+0000","last_unstale":"2024-02-23T00:51:01.237296+0000","last_undegraded":"2024-02-23T00:51:01.237296+0000","last_fullsized":"2024-02-23T00:51:01.237296+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"447'39","last_scrub_stamp":"2024-02-23T00:49:06.040326+0000","last_deep_scrub":"447'39","last_deep_scrub_stamp":"2024-02-23T00:49:06.040326+0000","last_clean_scrub_stamp":"2024-02-23T00:49:06.040326+0000","log_size":43,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":3,"num_write":18,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.d","version":"414'58","reported_seq":583,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.236988+0000","last_change":"2024-02-23T00:50:28.632286+0000","last_active":"2024-02-23T00:51:00.236988+0000","last_peered":"2024-02-23T00:51:00.236988+0000","last_clean":"2024-02-23T00:51:00.236988+0000","last_became_active":"2024-02-23T00:47:28.202691+0000","last_became_peered":"2024-02-23T00:47:28.202691+0000","last_unstale":"2024-02-23T00:51:00.236988+0000","last_undegraded":"2024-02-23T00:51:00.236988+0000","last_fullsized":"2024-02-23T00:51:00.236988+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"414'58","last_scrub_stamp":"2024-02-23T00:50:28.632146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:28.632146+0000","log_size":58,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":18,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":33,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.e","version":"514'60","reported_seq":542,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.241222+0000","last_change":"2024-02-23T00:50:31.600892+0000","last_active":"2024-02-23T00:51:00.241222+0000","last_peered":"2024-02-23T00:51:00.241222+0000","last_clean":"2024-02-23T00:51:00.241222+0000","last_became_active":"2024-02-23T00:43:17.845956+0000","last_became_peered":"2024-02-23T00:43:17.845956+0000","last_unstale":"2024-02-23T00:51:00.241222+0000","last_undegraded":"2024-02-23T00:51:00.241222+0000","last_fullsized":"2024-02-23T00:51:00.241222+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"495'58","last_scrub_stamp":"2024-02-23T00:50:31.600819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:31.600819+0000","log_size":60,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":43,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":24,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.f","version":"496'138","reported_seq":798,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.237445+0000","last_change":"2024-02-23T00:49:03.264065+0000","last_active":"2024-02-23T00:51:00.237445+0000","last_peered":"2024-02-23T00:51:00.237445+0000","last_clean":"2024-02-23T00:51:00.237445+0000","last_became_active":"2024-02-23T00:47:47.183032+0000","last_became_peered":"2024-02-23T00:47:47.183032+0000","last_unstale":"2024-02-23T00:51:00.237445+0000","last_undegraded":"2024-02-23T00:51:00.237445+0000","last_fullsized":"2024-02-23T00:51:00.237445+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"241'136","last_scrub_stamp":"2024-02-23T00:49:03.263943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:49:03.263943+0000","log_size":138,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":88,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":8,"num_read_kb":1,"num_write":43,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.10","version":"514'76","reported_seq":550,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.239733+0000","last_change":"2024-02-23T00:50:27.628865+0000","last_active":"2024-02-23T00:51:00.239733+0000","last_peered":"2024-02-23T00:51:00.239733+0000","last_clean":"2024-02-23T00:51:00.239733+0000","last_became_active":"2024-02-23T00:43:17.819865+0000","last_became_peered":"2024-02-23T00:43:17.819865+0000","last_unstale":"2024-02-23T00:51:00.239733+0000","last_undegraded":"2024-02-23T00:51:00.239733+0000","last_fullsized":"2024-02-23T00:51:00.239733+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"496'75","last_scrub_stamp":"2024-02-23T00:50:27.628796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:27.628796+0000","log_size":76,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1389,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":3,"num_write":32,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.11","version":"347'53","reported_seq":524,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.239633+0000","last_change":"2024-02-23T00:50:22.723895+0000","last_active":"2024-02-23T00:51:00.239633+0000","last_peered":"2024-02-23T00:51:00.239633+0000","last_clean":"2024-02-23T00:51:00.239633+0000","last_became_active":"2024-02-23T00:43:17.852697+0000","last_became_peered":"2024-02-23T00:43:17.852697+0000","last_unstale":"2024-02-23T00:51:00.239633+0000","last_undegraded":"2024-02-23T00:51:00.239633+0000","last_fullsized":"2024-02-23T00:51:00.239633+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'53","last_scrub_stamp":"2024-02-23T00:50:22.723817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:22.723817+0000","log_size":53,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":49,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":17,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.12","version":"240'73","reported_seq":453,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.241555+0000","last_change":"2024-02-23T00:48:51.203754+0000","last_active":"2024-02-23T00:51:00.241555+0000","last_peered":"2024-02-23T00:51:00.241555+0000","last_clean":"2024-02-23T00:51:00.241555+0000","last_became_active":"2024-02-23T00:47:28.213692+0000","last_became_peered":"2024-02-23T00:47:28.213692+0000","last_unstale":"2024-02-23T00:51:00.241555+0000","last_undegraded":"2024-02-23T00:51:00.241555+0000","last_fullsized":"2024-02-23T00:51:00.241555+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'73","last_scrub_stamp":"2024-02-23T00:48:14.866255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:48:14.866255+0000","log_size":73,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":49,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":49,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.13","version":"495'63","reported_seq":551,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.239783+0000","last_change":"2024-02-23T00:48:51.206467+0000","last_active":"2024-02-23T00:51:00.239783+0000","last_peered":"2024-02-23T00:51:00.239783+0000","last_clean":"2024-02-23T00:51:00.239783+0000","last_became_active":"2024-02-23T00:43:17.857823+0000","last_became_peered":"2024-02-23T00:43:17.857823+0000","last_unstale":"2024-02-23T00:51:00.239783+0000","last_undegraded":"2024-02-23T00:51:00.239783+0000","last_fullsized":"2024-02-23T00:51:00.239783+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"422'55","last_scrub_stamp":"2024-02-23T00:48:15.865963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:48:15.865963+0000","log_size":63,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":45,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":8,"num_read_kb":4,"num_write":24,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.14","version":"514'62","reported_seq":594,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.248450+0000","last_change":"2024-02-23T00:50:34.190105+0000","last_active":"2024-02-23T00:51:00.248450+0000","last_peered":"2024-02-23T00:51:00.248450+0000","last_clean":"2024-02-23T00:51:00.248450+0000","last_became_active":"2024-02-23T00:47:28.235465+0000","last_became_peered":"2024-02-23T00:47:28.235465+0000","last_unstale":"2024-02-23T00:51:00.248450+0000","last_undegraded":"2024-02-23T00:51:00.248450+0000","last_fullsized":"2024-02-23T00:51:00.248450+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"496'61","last_scrub_stamp":"2024-02-23T00:50:34.190011+0000","last_deep_scrub":"420'54","last_deep_scrub_stamp":"2024-02-23T00:48:12.892464+0000","last_clean_scrub_stamp":"2024-02-23T00:50:34.190011+0000","log_size":62,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2369,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":13,"num_read_kb":5,"num_write":31,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":51,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.15","version":"505'22","reported_seq":533,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.240062+0000","last_change":"2024-02-23T00:49:33.948127+0000","last_active":"2024-02-23T00:51:01.240062+0000","last_peered":"2024-02-23T00:51:01.240062+0000","last_clean":"2024-02-23T00:51:01.240062+0000","last_became_active":"2024-02-23T00:47:28.219548+0000","last_became_peered":"2024-02-23T00:47:28.219548+0000","last_unstale":"2024-02-23T00:51:01.240062+0000","last_undegraded":"2024-02-23T00:51:01.240062+0000","last_fullsized":"2024-02-23T00:51:01.240062+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'19","last_scrub_stamp":"2024-02-23T00:49:33.948049+0000","last_deep_scrub":"299'18","last_deep_scrub_stamp":"2024-02-23T00:46:11.830580+0000","last_clean_scrub_stamp":"2024-02-23T00:49:33.948049+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3465,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":12,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.16","version":"407'30","reported_seq":553,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.237287+0000","last_change":"2024-02-23T00:48:54.213604+0000","last_active":"2024-02-23T00:51:00.237287+0000","last_peered":"2024-02-23T00:51:00.237287+0000","last_clean":"2024-02-23T00:51:00.237287+0000","last_became_active":"2024-02-23T00:47:28.199870+0000","last_became_peered":"2024-02-23T00:47:28.199870+0000","last_unstale":"2024-02-23T00:51:00.237287+0000","last_undegraded":"2024-02-23T00:51:00.237287+0000","last_fullsized":"2024-02-23T00:51:00.237287+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"407'30","last_scrub_stamp":"2024-02-23T00:48:54.213512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:48:54.213512+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":64,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":14,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.17","version":"375'63","reported_seq":552,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.243789+0000","last_change":"2024-02-23T00:49:42.321041+0000","last_active":"2024-02-23T00:51:00.243789+0000","last_peered":"2024-02-23T00:51:00.243789+0000","last_clean":"2024-02-23T00:51:00.243789+0000","last_became_active":"2024-02-23T00:43:17.843525+0000","last_became_peered":"2024-02-23T00:43:17.843525+0000","last_unstale":"2024-02-23T00:51:00.243789+0000","last_undegraded":"2024-02-23T00:51:00.243789+0000","last_fullsized":"2024-02-23T00:51:00.243789+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'63","last_scrub_stamp":"2024-02-23T00:49:42.320947+0000","last_deep_scrub":"375'63","last_deep_scrub_stamp":"2024-02-23T00:47:27.229920+0000","last_clean_scrub_stamp":"2024-02-23T00:49:42.320947+0000","log_size":63,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":20,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.19","version":"494'98","reported_seq":653,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.243157+0000","last_change":"2024-02-23T00:50:20.275389+0000","last_active":"2024-02-23T00:51:00.243157+0000","last_peered":"2024-02-23T00:51:00.243157+0000","last_clean":"2024-02-23T00:51:00.243157+0000","last_became_active":"2024-02-23T00:47:28.210660+0000","last_became_peered":"2024-02-23T00:47:28.210660+0000","last_unstale":"2024-02-23T00:51:00.243157+0000","last_undegraded":"2024-02-23T00:51:00.243157+0000","last_fullsized":"2024-02-23T00:51:00.243157+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'96","last_scrub_stamp":"2024-02-23T00:50:20.275315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:20.275315+0000","log_size":98,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":73,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":9,"num_read_kb":1,"num_write":33,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":73,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,7],"acting":[6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.18","version":"523'15","reported_seq":525,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.236820+0000","last_change":"2024-02-23T00:49:09.073073+0000","last_active":"2024-02-23T00:51:01.236820+0000","last_peered":"2024-02-23T00:51:01.236820+0000","last_clean":"2024-02-23T00:51:01.236820+0000","last_became_active":"2024-02-23T00:47:28.221906+0000","last_became_peered":"2024-02-23T00:47:28.221906+0000","last_unstale":"2024-02-23T00:51:01.236820+0000","last_undegraded":"2024-02-23T00:51:01.236820+0000","last_fullsized":"2024-02-23T00:51:01.236820+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"410'12","last_scrub_stamp":"2024-02-23T00:49:09.073000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:49:09.073000+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":15,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":1,"num_write":11,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":5,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1b","version":"347'70","reported_seq":546,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.236815+0000","last_change":"2024-02-23T00:49:28.481771+0000","last_active":"2024-02-23T00:51:00.236815+0000","last_peered":"2024-02-23T00:51:00.236815+0000","last_clean":"2024-02-23T00:51:00.236815+0000","last_became_active":"2024-02-23T00:43:17.803043+0000","last_became_peered":"2024-02-23T00:43:17.803043+0000","last_unstale":"2024-02-23T00:51:00.236815+0000","last_undegraded":"2024-02-23T00:51:00.236815+0000","last_fullsized":"2024-02-23T00:51:00.236815+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'70","last_scrub_stamp":"2024-02-23T00:49:28.481700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:49:28.481700+0000","log_size":70,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4000060,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":22,"num_write_kb":3928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.1a","version":"347'35","reported_seq":549,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.242149+0000","last_change":"2024-02-23T00:49:46.307053+0000","last_active":"2024-02-23T00:51:00.242149+0000","last_peered":"2024-02-23T00:51:00.242149+0000","last_clean":"2024-02-23T00:51:00.242149+0000","last_became_active":"2024-02-23T00:45:44.178195+0000","last_became_peered":"2024-02-23T00:45:44.178195+0000","last_unstale":"2024-02-23T00:51:00.242149+0000","last_undegraded":"2024-02-23T00:51:00.242149+0000","last_fullsized":"2024-02-23T00:51:00.242149+0000","mapping_epoch":271,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":272,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'35","last_scrub_stamp":"2024-02-23T00:49:46.306975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.306975+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":30,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":11,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1d","version":"532'106","reported_seq":584,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.238811+0000","last_change":"2024-02-23T00:50:58.794356+0000","last_active":"2024-02-23T00:51:01.238811+0000","last_peered":"2024-02-23T00:51:01.238811+0000","last_clean":"2024-02-23T00:51:01.238811+0000","last_became_active":"2024-02-23T00:43:17.828194+0000","last_became_peered":"2024-02-23T00:43:17.828194+0000","last_unstale":"2024-02-23T00:51:01.238811+0000","last_undegraded":"2024-02-23T00:51:01.238811+0000","last_fullsized":"2024-02-23T00:51:01.238811+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"505'105","last_scrub_stamp":"2024-02-23T00:50:58.794271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:58.794271+0000","log_size":106,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2384,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":7,"num_read_kb":0,"num_write":35,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"62.1c","version":"514'70","reported_seq":553,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.242327+0000","last_change":"2024-02-23T00:49:45.261438+0000","last_active":"2024-02-23T00:51:00.242327+0000","last_peered":"2024-02-23T00:51:00.242327+0000","last_clean":"2024-02-23T00:51:00.242327+0000","last_became_active":"2024-02-23T00:43:17.804718+0000","last_became_peered":"2024-02-23T00:43:17.804718+0000","last_unstale":"2024-02-23T00:51:00.242327+0000","last_undegraded":"2024-02-23T00:51:00.242327+0000","last_fullsized":"2024-02-23T00:51:00.242327+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"446'61","last_scrub_stamp":"2024-02-23T00:49:45.261352+0000","last_deep_scrub":"374'55","last_deep_scrub_stamp":"2024-02-23T00:47:26.267291+0000","last_clean_scrub_stamp":"2024-02-23T00:49:45.261352+0000","log_size":70,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1594,"num_objects":7,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":15,"num_read_kb":8,"num_write":31,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1f","version":"464'55","reported_seq":555,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.248951+0000","last_change":"2024-02-23T00:50:58.157230+0000","last_active":"2024-02-23T00:51:00.248951+0000","last_peered":"2024-02-23T00:51:00.248951+0000","last_clean":"2024-02-23T00:51:00.248951+0000","last_became_active":"2024-02-23T00:47:28.243574+0000","last_became_peered":"2024-02-23T00:47:28.243574+0000","last_unstale":"2024-02-23T00:51:00.248951+0000","last_undegraded":"2024-02-23T00:51:00.248951+0000","last_fullsized":"2024-02-23T00:51:00.248951+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"464'55","last_scrub_stamp":"2024-02-23T00:50:58.157140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:58.157140+0000","log_size":55,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":73,"num_objects":9,"num_object_clones":2,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":18,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":35,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":4,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.1e","version":"505'62","reported_seq":581,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.248797+0000","last_change":"2024-02-23T00:48:51.478270+0000","last_active":"2024-02-23T00:51:00.248797+0000","last_peered":"2024-02-23T00:51:00.248797+0000","last_clean":"2024-02-23T00:51:00.248797+0000","last_became_active":"2024-02-23T00:47:28.244073+0000","last_became_peered":"2024-02-23T00:47:28.244073+0000","last_unstale":"2024-02-23T00:51:00.248797+0000","last_undegraded":"2024-02-23T00:51:00.248797+0000","last_fullsized":"2024-02-23T00:51:00.248797+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"245'51","last_scrub_stamp":"2024-02-23T00:48:42.689040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:48:42.689040+0000","log_size":62,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1248,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":11,"num_read_kb":4,"num_write":29,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":33,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"466'9","reported_seq":652,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.247899+0000","last_change":"2024-02-23T00:49:43.188066+0000","last_active":"2024-02-23T00:51:00.247899+0000","last_peered":"2024-02-23T00:51:00.247899+0000","last_clean":"2024-02-23T00:51:00.247899+0000","last_became_active":"2024-02-23T00:39:18.793414+0000","last_became_peered":"2024-02-23T00:39:18.793414+0000","last_unstale":"2024-02-23T00:51:00.247899+0000","last_undegraded":"2024-02-23T00:51:00.247899+0000","last_fullsized":"2024-02-23T00:51:00.247899+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'9","last_scrub_stamp":"2024-02-23T00:49:43.187989+0000","last_deep_scrub":"466'9","last_deep_scrub_stamp":"2024-02-23T00:49:43.187989+0000","last_clean_scrub_stamp":"2024-02-23T00:49:43.187989+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"470'42","reported_seq":578,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.238904+0000","last_change":"2024-02-23T00:49:47.653272+0000","last_active":"2024-02-23T00:51:00.238904+0000","last_peered":"2024-02-23T00:51:00.238904+0000","last_clean":"2024-02-23T00:51:00.238904+0000","last_became_active":"2024-02-23T00:39:16.859401+0000","last_became_peered":"2024-02-23T00:39:16.859401+0000","last_unstale":"2024-02-23T00:51:00.238904+0000","last_undegraded":"2024-02-23T00:51:00.238904+0000","last_fullsized":"2024-02-23T00:51:00.238904+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"470'42","last_scrub_stamp":"2024-02-23T00:49:47.653176+0000","last_deep_scrub":"470'42","last_deep_scrub_stamp":"2024-02-23T00:49:47.653176+0000","last_clean_scrub_stamp":"2024-02-23T00:49:47.653176+0000","log_size":42,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":174,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.239171+0000","last_change":"2024-02-23T00:49:25.955780+0000","last_active":"2024-02-23T00:51:01.239171+0000","last_peered":"2024-02-23T00:51:01.239171+0000","last_clean":"2024-02-23T00:51:01.239171+0000","last_became_active":"2024-02-23T00:47:28.220876+0000","last_became_peered":"2024-02-23T00:47:28.220876+0000","last_unstale":"2024-02-23T00:51:01.239171+0000","last_undegraded":"2024-02-23T00:51:01.239171+0000","last_fullsized":"2024-02-23T00:51:01.239171+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:25.955711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:04.330426+0000","last_clean_scrub_stamp":"2024-02-23T00:49:25.955711+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":516,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.239938+0000","last_change":"2024-02-23T00:50:14.801990+0000","last_active":"2024-02-23T00:51:00.239938+0000","last_peered":"2024-02-23T00:51:00.239938+0000","last_clean":"2024-02-23T00:51:00.239938+0000","last_became_active":"2024-02-23T00:43:32.282545+0000","last_became_peered":"2024-02-23T00:43:32.282545+0000","last_unstale":"2024-02-23T00:51:00.239938+0000","last_undegraded":"2024-02-23T00:51:00.239938+0000","last_fullsized":"2024-02-23T00:51:00.239938+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:14.801915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:50:14.801915+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":445,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.241195+0000","last_change":"2024-02-23T00:50:23.727290+0000","last_active":"2024-02-23T00:51:00.241195+0000","last_peered":"2024-02-23T00:51:00.241195+0000","last_clean":"2024-02-23T00:51:00.241195+0000","last_became_active":"2024-02-23T00:42:58.828846+0000","last_became_peered":"2024-02-23T00:42:58.828846+0000","last_unstale":"2024-02-23T00:51:00.241195+0000","last_undegraded":"2024-02-23T00:51:00.241195+0000","last_fullsized":"2024-02-23T00:51:00.241195+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:23.727173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:23.727173+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"457'21","reported_seq":543,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.240190+0000","last_change":"2024-02-23T00:49:27.765115+0000","last_active":"2024-02-23T00:51:00.240190+0000","last_peered":"2024-02-23T00:51:00.240190+0000","last_clean":"2024-02-23T00:51:00.240190+0000","last_became_active":"2024-02-23T00:39:16.783701+0000","last_became_peered":"2024-02-23T00:39:16.783701+0000","last_unstale":"2024-02-23T00:51:00.240190+0000","last_undegraded":"2024-02-23T00:51:00.240190+0000","last_fullsized":"2024-02-23T00:51:00.240190+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-23T00:49:27.765048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:27.765048+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":496,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.242435+0000","last_change":"2024-02-23T00:49:04.528117+0000","last_active":"2024-02-23T00:51:00.242435+0000","last_peered":"2024-02-23T00:51:00.242435+0000","last_clean":"2024-02-23T00:51:00.242435+0000","last_became_active":"2024-02-23T00:39:14.772493+0000","last_became_peered":"2024-02-23T00:39:14.772493+0000","last_unstale":"2024-02-23T00:51:00.242435+0000","last_undegraded":"2024-02-23T00:51:00.242435+0000","last_fullsized":"2024-02-23T00:51:00.242435+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:04.528006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:04.528006+0000","last_clean_scrub_stamp":"2024-02-23T00:49:04.528006+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":172,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.236804+0000","last_change":"2024-02-23T00:49:30.499854+0000","last_active":"2024-02-23T00:51:00.236804+0000","last_peered":"2024-02-23T00:51:00.236804+0000","last_clean":"2024-02-23T00:51:00.236804+0000","last_became_active":"2024-02-23T00:47:28.231567+0000","last_became_peered":"2024-02-23T00:47:28.231567+0000","last_unstale":"2024-02-23T00:51:00.236804+0000","last_undegraded":"2024-02-23T00:51:00.236804+0000","last_fullsized":"2024-02-23T00:51:00.236804+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:30.499769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:08.335330+0000","last_clean_scrub_stamp":"2024-02-23T00:49:30.499769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":521,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.239470+0000","last_change":"2024-02-23T00:48:13.867784+0000","last_active":"2024-02-23T00:51:00.239470+0000","last_peered":"2024-02-23T00:51:00.239470+0000","last_clean":"2024-02-23T00:51:00.239470+0000","last_became_active":"2024-02-23T00:47:28.214157+0000","last_became_peered":"2024-02-23T00:47:28.214157+0000","last_unstale":"2024-02-23T00:51:00.239470+0000","last_undegraded":"2024-02-23T00:51:00.239470+0000","last_fullsized":"2024-02-23T00:51:00.239470+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:13.867716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:13.867716+0000","last_clean_scrub_stamp":"2024-02-23T00:48:13.867716+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":446,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.241995+0000","last_change":"2024-02-23T00:50:32.622097+0000","last_active":"2024-02-23T00:51:00.241995+0000","last_peered":"2024-02-23T00:51:00.241995+0000","last_clean":"2024-02-23T00:51:00.241995+0000","last_became_active":"2024-02-23T00:42:58.867305+0000","last_became_peered":"2024-02-23T00:42:58.867305+0000","last_unstale":"2024-02-23T00:51:00.241995+0000","last_undegraded":"2024-02-23T00:51:00.241995+0000","last_fullsized":"2024-02-23T00:51:00.241995+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:32.622026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:32.622026+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"457'51","reported_seq":580,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.242698+0000","last_change":"2024-02-23T00:48:42.409822+0000","last_active":"2024-02-23T00:51:00.242698+0000","last_peered":"2024-02-23T00:51:00.242698+0000","last_clean":"2024-02-23T00:51:00.242698+0000","last_became_active":"2024-02-23T00:39:16.857413+0000","last_became_peered":"2024-02-23T00:39:16.857413+0000","last_unstale":"2024-02-23T00:51:00.242698+0000","last_undegraded":"2024-02-23T00:51:00.242698+0000","last_fullsized":"2024-02-23T00:51:00.242698+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'35","last_scrub_stamp":"2024-02-23T00:48:42.409686+0000","last_deep_scrub":"88'35","last_deep_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_clean_scrub_stamp":"2024-02-23T00:48:42.409686+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":68,"num_read_kb":56,"num_write":42,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":177,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.241160+0000","last_change":"2024-02-23T00:50:57.770551+0000","last_active":"2024-02-23T00:51:00.241160+0000","last_peered":"2024-02-23T00:51:00.241160+0000","last_clean":"2024-02-23T00:51:00.241160+0000","last_became_active":"2024-02-23T00:47:28.231891+0000","last_became_peered":"2024-02-23T00:47:28.231891+0000","last_unstale":"2024-02-23T00:51:00.241160+0000","last_undegraded":"2024-02-23T00:51:00.241160+0000","last_fullsized":"2024-02-23T00:51:00.241160+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:57.770450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_clean_scrub_stamp":"2024-02-23T00:50:57.770450+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":173,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.240019+0000","last_change":"2024-02-23T00:49:03.682811+0000","last_active":"2024-02-23T00:51:00.240019+0000","last_peered":"2024-02-23T00:51:00.240019+0000","last_clean":"2024-02-23T00:51:00.240019+0000","last_became_active":"2024-02-23T00:47:28.220578+0000","last_became_peered":"2024-02-23T00:47:28.220578+0000","last_unstale":"2024-02-23T00:51:00.240019+0000","last_undegraded":"2024-02-23T00:51:00.240019+0000","last_fullsized":"2024-02-23T00:51:00.240019+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:03.682745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:03.682745+0000","last_clean_scrub_stamp":"2024-02-23T00:49:03.682745+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":524,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.247391+0000","last_change":"2024-02-23T00:49:42.150972+0000","last_active":"2024-02-23T00:51:00.247391+0000","last_peered":"2024-02-23T00:51:00.247391+0000","last_clean":"2024-02-23T00:51:00.247391+0000","last_became_active":"2024-02-23T00:47:28.215636+0000","last_became_peered":"2024-02-23T00:47:28.215636+0000","last_unstale":"2024-02-23T00:51:00.247391+0000","last_undegraded":"2024-02-23T00:51:00.247391+0000","last_fullsized":"2024-02-23T00:51:00.247391+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:42.150875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:42.150875+0000","last_clean_scrub_stamp":"2024-02-23T00:49:42.150875+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":438,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.240095+0000","last_change":"2024-02-23T00:49:34.515593+0000","last_active":"2024-02-23T00:51:00.240095+0000","last_peered":"2024-02-23T00:51:00.240095+0000","last_clean":"2024-02-23T00:51:00.240095+0000","last_became_active":"2024-02-23T00:42:57.639013+0000","last_became_peered":"2024-02-23T00:42:57.639013+0000","last_unstale":"2024-02-23T00:51:00.240095+0000","last_undegraded":"2024-02-23T00:51:00.240095+0000","last_fullsized":"2024-02-23T00:51:00.240095+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:34.515496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:49:34.515496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"457'31","reported_seq":600,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.238300+0000","last_change":"2024-02-23T00:49:31.480372+0000","last_active":"2024-02-23T00:51:00.238300+0000","last_peered":"2024-02-23T00:51:00.238300+0000","last_clean":"2024-02-23T00:51:00.238300+0000","last_became_active":"2024-02-23T00:47:28.200653+0000","last_became_peered":"2024-02-23T00:47:28.200653+0000","last_unstale":"2024-02-23T00:51:00.238300+0000","last_undegraded":"2024-02-23T00:51:00.238300+0000","last_fullsized":"2024-02-23T00:51:00.238300+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:49:31.480253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:31.480253+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":171,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.247614+0000","last_change":"2024-02-23T00:48:44.698517+0000","last_active":"2024-02-23T00:51:00.247614+0000","last_peered":"2024-02-23T00:51:00.247614+0000","last_clean":"2024-02-23T00:51:00.247614+0000","last_became_active":"2024-02-23T00:47:28.201926+0000","last_became_peered":"2024-02-23T00:47:28.201926+0000","last_unstale":"2024-02-23T00:51:00.247614+0000","last_undegraded":"2024-02-23T00:51:00.247614+0000","last_fullsized":"2024-02-23T00:51:00.247614+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:44.698446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:48:44.698446+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":492,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.240481+0000","last_change":"2024-02-23T00:48:03.801207+0000","last_active":"2024-02-23T00:51:00.240481+0000","last_peered":"2024-02-23T00:51:00.240481+0000","last_clean":"2024-02-23T00:51:00.240481+0000","last_became_active":"2024-02-23T00:39:18.799875+0000","last_became_peered":"2024-02-23T00:39:18.799875+0000","last_unstale":"2024-02-23T00:51:00.240481+0000","last_undegraded":"2024-02-23T00:51:00.240481+0000","last_fullsized":"2024-02-23T00:51:00.240481+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:03.801103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:48:03.801103+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":438,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.240675+0000","last_change":"2024-02-23T00:48:49.610457+0000","last_active":"2024-02-23T00:51:00.240675+0000","last_peered":"2024-02-23T00:51:00.240675+0000","last_clean":"2024-02-23T00:51:00.240675+0000","last_became_active":"2024-02-23T00:42:57.675306+0000","last_became_peered":"2024-02-23T00:42:57.675306+0000","last_unstale":"2024-02-23T00:51:00.240675+0000","last_undegraded":"2024-02-23T00:51:00.240675+0000","last_fullsized":"2024-02-23T00:51:00.240675+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:49.610391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:49.610391+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"457'20","reported_seq":532,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.488450+0000","last_change":"2024-02-23T00:51:01.488450+0000","last_active":"2024-02-23T00:51:01.488450+0000","last_peered":"2024-02-23T00:51:01.488450+0000","last_clean":"2024-02-23T00:51:01.488450+0000","last_became_active":"2024-02-23T00:39:16.777530+0000","last_became_peered":"2024-02-23T00:39:16.777530+0000","last_unstale":"2024-02-23T00:51:01.488450+0000","last_undegraded":"2024-02-23T00:51:01.488450+0000","last_fullsized":"2024-02-23T00:51:01.488450+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'20","last_scrub_stamp":"2024-02-23T00:51:01.488371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:01.488371+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":496,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.240751+0000","last_change":"2024-02-23T00:48:41.719867+0000","last_active":"2024-02-23T00:51:00.240751+0000","last_peered":"2024-02-23T00:51:00.240751+0000","last_clean":"2024-02-23T00:51:00.240751+0000","last_became_active":"2024-02-23T00:39:14.780379+0000","last_became_peered":"2024-02-23T00:39:14.780379+0000","last_unstale":"2024-02-23T00:51:00.240751+0000","last_undegraded":"2024-02-23T00:51:00.240751+0000","last_fullsized":"2024-02-23T00:51:00.240751+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:41.719772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.575242+0000","last_clean_scrub_stamp":"2024-02-23T00:48:41.719772+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"376'9","reported_seq":239,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.248892+0000","last_change":"2024-02-23T00:49:40.180695+0000","last_active":"2024-02-23T00:51:00.248892+0000","last_peered":"2024-02-23T00:51:00.248892+0000","last_clean":"2024-02-23T00:51:00.248892+0000","last_became_active":"2024-02-23T00:47:28.231104+0000","last_became_peered":"2024-02-23T00:47:28.231104+0000","last_unstale":"2024-02-23T00:51:00.248892+0000","last_undegraded":"2024-02-23T00:51:00.248892+0000","last_fullsized":"2024-02-23T00:51:00.248892+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"376'9","last_scrub_stamp":"2024-02-23T00:49:40.180616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:40.180616+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":443,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.238133+0000","last_change":"2024-02-23T00:48:51.104378+0000","last_active":"2024-02-23T00:51:01.238133+0000","last_peered":"2024-02-23T00:51:01.238133+0000","last_clean":"2024-02-23T00:51:01.238133+0000","last_became_active":"2024-02-23T00:42:58.765106+0000","last_became_peered":"2024-02-23T00:42:58.765106+0000","last_unstale":"2024-02-23T00:51:01.238133+0000","last_undegraded":"2024-02-23T00:51:01.238133+0000","last_fullsized":"2024-02-23T00:51:01.238133+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:51.104305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:51.104305+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"457'27","reported_seq":537,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.241756+0000","last_change":"2024-02-23T00:48:00.698737+0000","last_active":"2024-02-23T00:51:00.241756+0000","last_peered":"2024-02-23T00:51:00.241756+0000","last_clean":"2024-02-23T00:51:00.241756+0000","last_became_active":"2024-02-23T00:39:16.789757+0000","last_became_peered":"2024-02-23T00:39:16.789757+0000","last_unstale":"2024-02-23T00:51:00.241756+0000","last_undegraded":"2024-02-23T00:51:00.241756+0000","last_fullsized":"2024-02-23T00:51:00.241756+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:48:00.698670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:48:00.698670+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":496,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.240150+0000","last_change":"2024-02-23T00:48:09.801324+0000","last_active":"2024-02-23T00:51:00.240150+0000","last_peered":"2024-02-23T00:51:00.240150+0000","last_clean":"2024-02-23T00:51:00.240150+0000","last_became_active":"2024-02-23T00:39:14.780526+0000","last_became_peered":"2024-02-23T00:39:14.780526+0000","last_unstale":"2024-02-23T00:51:00.240150+0000","last_undegraded":"2024-02-23T00:51:00.240150+0000","last_fullsized":"2024-02-23T00:51:00.240150+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:09.801251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:48:09.801251+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":649,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.237485+0000","last_change":"2024-02-23T00:49:40.384676+0000","last_active":"2024-02-23T00:51:00.237485+0000","last_peered":"2024-02-23T00:51:00.237485+0000","last_clean":"2024-02-23T00:51:00.237485+0000","last_became_active":"2024-02-23T00:39:18.800440+0000","last_became_peered":"2024-02-23T00:39:18.800440+0000","last_unstale":"2024-02-23T00:51:00.237485+0000","last_undegraded":"2024-02-23T00:51:00.237485+0000","last_fullsized":"2024-02-23T00:51:00.237485+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:49:40.384608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:40.384608+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":443,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.236746+0000","last_change":"2024-02-23T00:48:59.206327+0000","last_active":"2024-02-23T00:51:00.236746+0000","last_peered":"2024-02-23T00:51:00.236746+0000","last_clean":"2024-02-23T00:51:00.236746+0000","last_became_active":"2024-02-23T00:42:58.833924+0000","last_became_peered":"2024-02-23T00:42:58.833924+0000","last_unstale":"2024-02-23T00:51:00.236746+0000","last_undegraded":"2024-02-23T00:51:00.236746+0000","last_fullsized":"2024-02-23T00:51:00.236746+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:59.206249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:59.206249+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"457'31","reported_seq":595,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.237398+0000","last_change":"2024-02-23T00:49:04.235169+0000","last_active":"2024-02-23T00:51:00.237398+0000","last_peered":"2024-02-23T00:51:00.237398+0000","last_clean":"2024-02-23T00:51:00.237398+0000","last_became_active":"2024-02-23T00:47:28.201225+0000","last_became_peered":"2024-02-23T00:47:28.201225+0000","last_unstale":"2024-02-23T00:51:00.237398+0000","last_undegraded":"2024-02-23T00:51:00.237398+0000","last_fullsized":"2024-02-23T00:51:00.237398+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"336'23","last_scrub_stamp":"2024-02-23T00:49:04.235046+0000","last_deep_scrub":"336'23","last_deep_scrub_stamp":"2024-02-23T00:46:48.320085+0000","last_clean_scrub_stamp":"2024-02-23T00:49:04.235046+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":503,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.247626+0000","last_change":"2024-02-23T00:49:37.331334+0000","last_active":"2024-02-23T00:51:00.247626+0000","last_peered":"2024-02-23T00:51:00.247626+0000","last_clean":"2024-02-23T00:51:00.247626+0000","last_became_active":"2024-02-23T00:39:14.784121+0000","last_became_peered":"2024-02-23T00:39:14.784121+0000","last_unstale":"2024-02-23T00:51:00.247626+0000","last_undegraded":"2024-02-23T00:51:00.247626+0000","last_fullsized":"2024-02-23T00:51:00.247626+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:37.331270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:37.331270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":500,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.241975+0000","last_change":"2024-02-23T00:50:19.827889+0000","last_active":"2024-02-23T00:51:00.241975+0000","last_peered":"2024-02-23T00:51:00.241975+0000","last_clean":"2024-02-23T00:51:00.241975+0000","last_became_active":"2024-02-23T00:39:18.800777+0000","last_became_peered":"2024-02-23T00:39:18.800777+0000","last_unstale":"2024-02-23T00:51:00.241975+0000","last_undegraded":"2024-02-23T00:51:00.241975+0000","last_fullsized":"2024-02-23T00:51:00.241975+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:19.827814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_clean_scrub_stamp":"2024-02-23T00:50:19.827814+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":438,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.240095+0000","last_change":"2024-02-23T00:49:33.529224+0000","last_active":"2024-02-23T00:51:00.240095+0000","last_peered":"2024-02-23T00:51:00.240095+0000","last_clean":"2024-02-23T00:51:00.240095+0000","last_became_active":"2024-02-23T00:42:57.653279+0000","last_became_peered":"2024-02-23T00:42:57.653279+0000","last_unstale":"2024-02-23T00:51:00.240095+0000","last_undegraded":"2024-02-23T00:51:00.240095+0000","last_fullsized":"2024-02-23T00:51:00.240095+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:33.529135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:49:33.529135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":647,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.236984+0000","last_change":"2024-02-23T00:49:08.047225+0000","last_active":"2024-02-23T00:51:01.236984+0000","last_peered":"2024-02-23T00:51:01.236984+0000","last_clean":"2024-02-23T00:51:01.236984+0000","last_became_active":"2024-02-23T00:39:18.798536+0000","last_became_peered":"2024-02-23T00:39:18.798536+0000","last_unstale":"2024-02-23T00:51:01.236984+0000","last_undegraded":"2024-02-23T00:51:01.236984+0000","last_fullsized":"2024-02-23T00:51:01.236984+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:49:08.047142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:08.047142+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"457'42","reported_seq":567,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.236949+0000","last_change":"2024-02-23T00:50:24.532387+0000","last_active":"2024-02-23T00:51:00.236949+0000","last_peered":"2024-02-23T00:51:00.236949+0000","last_clean":"2024-02-23T00:51:00.236949+0000","last_became_active":"2024-02-23T00:39:16.780130+0000","last_became_peered":"2024-02-23T00:39:16.780130+0000","last_unstale":"2024-02-23T00:51:00.236949+0000","last_undegraded":"2024-02-23T00:51:00.236949+0000","last_fullsized":"2024-02-23T00:51:00.236949+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'42","last_scrub_stamp":"2024-02-23T00:50:24.532271+0000","last_deep_scrub":"423'28","last_deep_scrub_stamp":"2024-02-23T00:48:17.435906+0000","last_clean_scrub_stamp":"2024-02-23T00:50:24.532271+0000","log_size":42,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":46,"num_write":34,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":518,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.243358+0000","last_change":"2024-02-23T00:50:59.199966+0000","last_active":"2024-02-23T00:51:00.243358+0000","last_peered":"2024-02-23T00:51:00.243358+0000","last_clean":"2024-02-23T00:51:00.243358+0000","last_became_active":"2024-02-23T00:47:28.237083+0000","last_became_peered":"2024-02-23T00:47:28.237083+0000","last_unstale":"2024-02-23T00:51:00.243358+0000","last_undegraded":"2024-02-23T00:51:00.243358+0000","last_fullsized":"2024-02-23T00:51:00.243358+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:59.199899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:51.391939+0000","last_clean_scrub_stamp":"2024-02-23T00:50:59.199899+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":516,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.248481+0000","last_change":"2024-02-23T00:49:41.163033+0000","last_active":"2024-02-23T00:51:00.248481+0000","last_peered":"2024-02-23T00:51:00.248481+0000","last_clean":"2024-02-23T00:51:00.248481+0000","last_became_active":"2024-02-23T00:47:30.187386+0000","last_became_peered":"2024-02-23T00:47:30.187386+0000","last_unstale":"2024-02-23T00:51:00.248481+0000","last_undegraded":"2024-02-23T00:51:00.248481+0000","last_fullsized":"2024-02-23T00:51:00.248481+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:41.162895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:46.686764+0000","last_clean_scrub_stamp":"2024-02-23T00:49:41.162895+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":449,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.242275+0000","last_change":"2024-02-23T00:48:52.416575+0000","last_active":"2024-02-23T00:51:00.242275+0000","last_peered":"2024-02-23T00:51:00.242275+0000","last_clean":"2024-02-23T00:51:00.242275+0000","last_became_active":"2024-02-23T00:47:28.212307+0000","last_became_peered":"2024-02-23T00:47:28.212307+0000","last_unstale":"2024-02-23T00:51:00.242275+0000","last_undegraded":"2024-02-23T00:51:00.242275+0000","last_fullsized":"2024-02-23T00:51:00.242275+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:52.416489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:52.416489+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":502,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.237685+0000","last_change":"2024-02-23T00:50:25.575319+0000","last_active":"2024-02-23T00:51:00.237685+0000","last_peered":"2024-02-23T00:51:00.237685+0000","last_clean":"2024-02-23T00:51:00.237685+0000","last_became_active":"2024-02-23T00:39:18.798829+0000","last_became_peered":"2024-02-23T00:39:18.798829+0000","last_unstale":"2024-02-23T00:51:00.237685+0000","last_undegraded":"2024-02-23T00:51:00.237685+0000","last_fullsized":"2024-02-23T00:51:00.237685+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:25.575238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:10.493340+0000","last_clean_scrub_stamp":"2024-02-23T00:50:25.575238+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"457'52","reported_seq":590,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.236980+0000","last_change":"2024-02-23T00:48:17.054839+0000","last_active":"2024-02-23T00:51:01.236980+0000","last_peered":"2024-02-23T00:51:01.236980+0000","last_clean":"2024-02-23T00:51:01.236980+0000","last_became_active":"2024-02-23T00:39:16.859002+0000","last_became_peered":"2024-02-23T00:39:16.859002+0000","last_unstale":"2024-02-23T00:51:01.236980+0000","last_undegraded":"2024-02-23T00:51:01.236980+0000","last_fullsized":"2024-02-23T00:51:01.236980+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'40","last_scrub_stamp":"2024-02-23T00:48:17.054722+0000","last_deep_scrub":"251'40","last_deep_scrub_stamp":"2024-02-23T00:45:23.910546+0000","last_clean_scrub_stamp":"2024-02-23T00:48:17.054722+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":76,"num_read_kb":62,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":496,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.240965+0000","last_change":"2024-02-23T00:48:44.728592+0000","last_active":"2024-02-23T00:51:00.240965+0000","last_peered":"2024-02-23T00:51:00.240965+0000","last_clean":"2024-02-23T00:51:00.240965+0000","last_became_active":"2024-02-23T00:39:14.779193+0000","last_became_peered":"2024-02-23T00:39:14.779193+0000","last_unstale":"2024-02-23T00:51:00.240965+0000","last_undegraded":"2024-02-23T00:51:00.240965+0000","last_fullsized":"2024-02-23T00:51:00.240965+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:44.728520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:48:44.728520+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":525,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.236914+0000","last_change":"2024-02-23T00:50:17.599788+0000","last_active":"2024-02-23T00:51:00.236914+0000","last_peered":"2024-02-23T00:51:00.236914+0000","last_clean":"2024-02-23T00:51:00.236914+0000","last_became_active":"2024-02-23T00:47:33.257915+0000","last_became_peered":"2024-02-23T00:47:33.257915+0000","last_unstale":"2024-02-23T00:51:00.236914+0000","last_undegraded":"2024-02-23T00:51:00.236914+0000","last_fullsized":"2024-02-23T00:51:00.236914+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":381,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:17.599724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:50:17.599724+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":443,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.239566+0000","last_change":"2024-02-23T00:48:04.989583+0000","last_active":"2024-02-23T00:51:01.239566+0000","last_peered":"2024-02-23T00:51:01.239566+0000","last_clean":"2024-02-23T00:51:01.239566+0000","last_became_active":"2024-02-23T00:42:58.809283+0000","last_became_peered":"2024-02-23T00:42:58.809283+0000","last_unstale":"2024-02-23T00:51:01.239566+0000","last_undegraded":"2024-02-23T00:51:01.239566+0000","last_fullsized":"2024-02-23T00:51:01.239566+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:04.989514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:04.989514+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":498,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.242576+0000","last_change":"2024-02-23T00:48:49.266366+0000","last_active":"2024-02-23T00:51:00.242576+0000","last_peered":"2024-02-23T00:51:00.242576+0000","last_clean":"2024-02-23T00:51:00.242576+0000","last_became_active":"2024-02-23T00:39:18.796664+0000","last_became_peered":"2024-02-23T00:39:18.796664+0000","last_unstale":"2024-02-23T00:51:00.242576+0000","last_undegraded":"2024-02-23T00:51:00.242576+0000","last_fullsized":"2024-02-23T00:51:00.242576+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:49.266295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:48:49.266295+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"457'44","reported_seq":578,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.239931+0000","last_change":"2024-02-23T00:49:46.644982+0000","last_active":"2024-02-23T00:51:00.239931+0000","last_peered":"2024-02-23T00:51:00.239931+0000","last_clean":"2024-02-23T00:51:00.239931+0000","last_became_active":"2024-02-23T00:39:16.857278+0000","last_became_peered":"2024-02-23T00:39:16.857278+0000","last_unstale":"2024-02-23T00:51:00.239931+0000","last_undegraded":"2024-02-23T00:51:00.239931+0000","last_fullsized":"2024-02-23T00:51:00.239931+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'44","last_scrub_stamp":"2024-02-23T00:49:46.644911+0000","last_deep_scrub":"457'44","last_deep_scrub_stamp":"2024-02-23T00:49:46.644911+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.644911+0000","log_size":44,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":503,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.248785+0000","last_change":"2024-02-23T00:50:22.299591+0000","last_active":"2024-02-23T00:51:00.248785+0000","last_peered":"2024-02-23T00:51:00.248785+0000","last_clean":"2024-02-23T00:51:00.248785+0000","last_became_active":"2024-02-23T00:39:14.783662+0000","last_became_peered":"2024-02-23T00:39:14.783662+0000","last_unstale":"2024-02-23T00:51:00.248785+0000","last_undegraded":"2024-02-23T00:51:00.248785+0000","last_fullsized":"2024-02-23T00:51:00.248785+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:22.299534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:22.299534+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":517,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.240247+0000","last_change":"2024-02-23T00:48:13.821965+0000","last_active":"2024-02-23T00:51:00.240247+0000","last_peered":"2024-02-23T00:51:00.240247+0000","last_clean":"2024-02-23T00:51:00.240247+0000","last_became_active":"2024-02-23T00:47:37.184240+0000","last_became_peered":"2024-02-23T00:47:37.184240+0000","last_unstale":"2024-02-23T00:51:00.240247+0000","last_undegraded":"2024-02-23T00:51:00.240247+0000","last_fullsized":"2024-02-23T00:51:00.240247+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":385,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:13.821901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:48:13.821901+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":445,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.240871+0000","last_change":"2024-02-23T00:50:24.682130+0000","last_active":"2024-02-23T00:51:00.240871+0000","last_peered":"2024-02-23T00:51:00.240871+0000","last_clean":"2024-02-23T00:51:00.240871+0000","last_became_active":"2024-02-23T00:42:58.726652+0000","last_became_peered":"2024-02-23T00:42:58.726652+0000","last_unstale":"2024-02-23T00:51:00.240871+0000","last_undegraded":"2024-02-23T00:51:00.240871+0000","last_fullsized":"2024-02-23T00:51:00.240871+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:24.682039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:24.682039+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":500,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.242103+0000","last_change":"2024-02-23T00:49:34.709320+0000","last_active":"2024-02-23T00:51:00.242103+0000","last_peered":"2024-02-23T00:51:00.242103+0000","last_clean":"2024-02-23T00:51:00.242103+0000","last_became_active":"2024-02-23T00:39:18.800378+0000","last_became_peered":"2024-02-23T00:39:18.800378+0000","last_unstale":"2024-02-23T00:51:00.242103+0000","last_undegraded":"2024-02-23T00:51:00.242103+0000","last_fullsized":"2024-02-23T00:51:00.242103+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:34.709251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:59.694033+0000","last_clean_scrub_stamp":"2024-02-23T00:49:34.709251+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"457'30","reported_seq":568,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.241377+0000","last_change":"2024-02-23T00:49:42.730745+0000","last_active":"2024-02-23T00:51:00.241377+0000","last_peered":"2024-02-23T00:51:00.241377+0000","last_clean":"2024-02-23T00:51:00.241377+0000","last_became_active":"2024-02-23T00:47:28.215366+0000","last_became_peered":"2024-02-23T00:47:28.215366+0000","last_unstale":"2024-02-23T00:51:00.241377+0000","last_undegraded":"2024-02-23T00:51:00.241377+0000","last_fullsized":"2024-02-23T00:51:00.241377+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'30","last_scrub_stamp":"2024-02-23T00:49:42.730670+0000","last_deep_scrub":"95'18","last_deep_scrub_stamp":"2024-02-23T00:42:01.195545+0000","last_clean_scrub_stamp":"2024-02-23T00:49:42.730670+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":508,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.247516+0000","last_change":"2024-02-23T00:49:46.162522+0000","last_active":"2024-02-23T00:51:00.247516+0000","last_peered":"2024-02-23T00:51:00.247516+0000","last_clean":"2024-02-23T00:51:00.247516+0000","last_became_active":"2024-02-23T00:39:14.784578+0000","last_became_peered":"2024-02-23T00:39:14.784578+0000","last_unstale":"2024-02-23T00:51:00.247516+0000","last_undegraded":"2024-02-23T00:51:00.247516+0000","last_fullsized":"2024-02-23T00:51:00.247516+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:46.162444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.162444+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":505,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.239244+0000","last_change":"2024-02-23T00:49:31.768539+0000","last_active":"2024-02-23T00:51:00.239244+0000","last_peered":"2024-02-23T00:51:00.239244+0000","last_clean":"2024-02-23T00:51:00.239244+0000","last_became_active":"2024-02-23T00:39:14.775446+0000","last_became_peered":"2024-02-23T00:39:14.775446+0000","last_unstale":"2024-02-23T00:51:00.239244+0000","last_undegraded":"2024-02-23T00:51:00.239244+0000","last_fullsized":"2024-02-23T00:51:00.239244+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:31.768472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:31.768472+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"67'1","reported_seq":523,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.240567+0000","last_change":"2024-02-23T00:50:59.395820+0000","last_active":"2024-02-23T00:51:00.240567+0000","last_peered":"2024-02-23T00:51:00.240567+0000","last_clean":"2024-02-23T00:51:00.240567+0000","last_became_active":"2024-02-23T00:47:56.187940+0000","last_became_peered":"2024-02-23T00:47:56.187940+0000","last_unstale":"2024-02-23T00:51:00.240567+0000","last_undegraded":"2024-02-23T00:51:00.240567+0000","last_fullsized":"2024-02-23T00:51:00.240567+0000","mapping_epoch":65,"log_start":"67'1","ondisk_log_start":"67'1","created":65,"last_epoch_clean":404,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:47:57.665156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:47:57.665156+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":444,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.241776+0000","last_change":"2024-02-23T00:48:15.593613+0000","last_active":"2024-02-23T00:51:00.241776+0000","last_peered":"2024-02-23T00:51:00.241776+0000","last_clean":"2024-02-23T00:51:00.241776+0000","last_became_active":"2024-02-23T00:42:58.690858+0000","last_became_peered":"2024-02-23T00:42:58.690858+0000","last_unstale":"2024-02-23T00:51:00.241776+0000","last_undegraded":"2024-02-23T00:51:00.241776+0000","last_fullsized":"2024-02-23T00:51:00.241776+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:15.593488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:35.360054+0000","last_clean_scrub_stamp":"2024-02-23T00:48:15.593488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":508,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.248646+0000","last_change":"2024-02-23T00:49:28.358727+0000","last_active":"2024-02-23T00:51:00.248646+0000","last_peered":"2024-02-23T00:51:00.248646+0000","last_clean":"2024-02-23T00:51:00.248646+0000","last_became_active":"2024-02-23T00:47:28.231152+0000","last_became_peered":"2024-02-23T00:47:28.231152+0000","last_unstale":"2024-02-23T00:51:00.248646+0000","last_undegraded":"2024-02-23T00:51:00.248646+0000","last_fullsized":"2024-02-23T00:51:00.248646+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:28.358647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:28.358647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"457'26","reported_seq":545,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.219937+0000","last_change":"2024-02-23T00:51:01.219903+0000","last_active":"2024-02-23T00:51:01.219937+0000","last_peered":"2024-02-23T00:51:01.219937+0000","last_clean":"2024-02-23T00:51:01.219937+0000","last_became_active":"2024-02-23T00:39:16.858880+0000","last_became_peered":"2024-02-23T00:39:16.858880+0000","last_unstale":"2024-02-23T00:51:01.219937+0000","last_undegraded":"2024-02-23T00:51:01.219937+0000","last_fullsized":"2024-02-23T00:51:01.219937+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-23T00:51:01.219824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:01.219824+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":513,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.247556+0000","last_change":"2024-02-23T00:50:24.297660+0000","last_active":"2024-02-23T00:51:00.247556+0000","last_peered":"2024-02-23T00:51:00.247556+0000","last_clean":"2024-02-23T00:51:00.247556+0000","last_became_active":"2024-02-23T00:39:08.248249+0000","last_became_peered":"2024-02-23T00:39:08.248249+0000","last_unstale":"2024-02-23T00:51:00.247556+0000","last_undegraded":"2024-02-23T00:51:00.247556+0000","last_fullsized":"2024-02-23T00:51:00.247556+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:24.297572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:49.977248+0000","last_clean_scrub_stamp":"2024-02-23T00:50:24.297572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":513,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.240099+0000","last_change":"2024-02-23T00:50:21.746072+0000","last_active":"2024-02-23T00:51:00.240099+0000","last_peered":"2024-02-23T00:51:00.240099+0000","last_clean":"2024-02-23T00:51:00.240099+0000","last_became_active":"2024-02-23T00:47:28.240482+0000","last_became_peered":"2024-02-23T00:47:28.240482+0000","last_unstale":"2024-02-23T00:51:00.240099+0000","last_undegraded":"2024-02-23T00:51:00.240099+0000","last_fullsized":"2024-02-23T00:51:00.240099+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:21.745961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:21.745961+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":507,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.237998+0000","last_change":"2024-02-23T00:50:34.055990+0000","last_active":"2024-02-23T00:51:01.237998+0000","last_peered":"2024-02-23T00:51:01.237998+0000","last_clean":"2024-02-23T00:51:01.237998+0000","last_became_active":"2024-02-23T00:39:14.783489+0000","last_became_peered":"2024-02-23T00:39:14.783489+0000","last_unstale":"2024-02-23T00:51:01.237998+0000","last_undegraded":"2024-02-23T00:51:01.237998+0000","last_fullsized":"2024-02-23T00:51:01.237998+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:34.055923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:34.055923+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":534,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.242187+0000","last_change":"2024-02-23T00:49:35.756168+0000","last_active":"2024-02-23T00:51:00.242187+0000","last_peered":"2024-02-23T00:51:00.242187+0000","last_clean":"2024-02-23T00:51:00.242187+0000","last_became_active":"2024-02-23T00:47:45.182550+0000","last_became_peered":"2024-02-23T00:47:45.182550+0000","last_unstale":"2024-02-23T00:51:00.242187+0000","last_undegraded":"2024-02-23T00:51:00.242187+0000","last_fullsized":"2024-02-23T00:51:00.242187+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:35.756073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:49:35.756073+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":457,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.248326+0000","last_change":"2024-02-23T00:50:16.257963+0000","last_active":"2024-02-23T00:51:00.248326+0000","last_peered":"2024-02-23T00:51:00.248326+0000","last_clean":"2024-02-23T00:51:00.248326+0000","last_became_active":"2024-02-23T00:47:28.233475+0000","last_became_peered":"2024-02-23T00:47:28.233475+0000","last_unstale":"2024-02-23T00:51:00.248326+0000","last_undegraded":"2024-02-23T00:51:00.248326+0000","last_fullsized":"2024-02-23T00:51:00.248326+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:16.257896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:16.257896+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":172,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.241464+0000","last_change":"2024-02-23T00:48:57.430209+0000","last_active":"2024-02-23T00:51:00.241464+0000","last_peered":"2024-02-23T00:51:00.241464+0000","last_clean":"2024-02-23T00:51:00.241464+0000","last_became_active":"2024-02-23T00:47:28.209125+0000","last_became_peered":"2024-02-23T00:47:28.209125+0000","last_unstale":"2024-02-23T00:51:00.241464+0000","last_undegraded":"2024-02-23T00:51:00.241464+0000","last_fullsized":"2024-02-23T00:51:00.241464+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:57.430129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.264240+0000","last_clean_scrub_stamp":"2024-02-23T00:48:57.430129+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"457'23","reported_seq":542,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.241900+0000","last_change":"2024-02-23T00:48:11.828616+0000","last_active":"2024-02-23T00:51:00.241900+0000","last_peered":"2024-02-23T00:51:00.241900+0000","last_clean":"2024-02-23T00:51:00.241900+0000","last_became_active":"2024-02-23T00:39:16.787484+0000","last_became_peered":"2024-02-23T00:39:16.787484+0000","last_unstale":"2024-02-23T00:51:00.241900+0000","last_undegraded":"2024-02-23T00:51:00.241900+0000","last_fullsized":"2024-02-23T00:51:00.241900+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'19","last_scrub_stamp":"2024-02-23T00:48:11.828547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:48:11.828547+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":48,"num_read_kb":36,"num_write":26,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":174,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.237066+0000","last_change":"2024-02-23T00:49:35.877867+0000","last_active":"2024-02-23T00:51:01.237066+0000","last_peered":"2024-02-23T00:51:01.237066+0000","last_clean":"2024-02-23T00:51:01.237066+0000","last_became_active":"2024-02-23T00:47:28.209575+0000","last_became_peered":"2024-02-23T00:47:28.209575+0000","last_unstale":"2024-02-23T00:51:01.237066+0000","last_undegraded":"2024-02-23T00:51:01.237066+0000","last_fullsized":"2024-02-23T00:51:01.237066+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:35.877800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:35.877800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"68'5","reported_seq":1210,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.246855+0000","last_change":"2024-02-23T00:50:57.456201+0000","last_active":"2024-02-23T00:51:01.246855+0000","last_peered":"2024-02-23T00:51:01.246855+0000","last_clean":"2024-02-23T00:51:01.246855+0000","last_became_active":"2024-02-23T00:47:28.227928+0000","last_became_peered":"2024-02-23T00:47:28.227928+0000","last_unstale":"2024-02-23T00:51:01.246855+0000","last_undegraded":"2024-02-23T00:51:01.246855+0000","last_fullsized":"2024-02-23T00:51:01.246855+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'5","last_scrub_stamp":"2024-02-23T00:50:57.456059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:57.456059+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":697,"num_read_kb":692,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":533,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.248760+0000","last_change":"2024-02-23T00:49:30.396935+0000","last_active":"2024-02-23T00:51:00.248760+0000","last_peered":"2024-02-23T00:51:00.248760+0000","last_clean":"2024-02-23T00:51:00.248760+0000","last_became_active":"2024-02-23T00:47:52.552244+0000","last_became_peered":"2024-02-23T00:47:52.552244+0000","last_unstale":"2024-02-23T00:51:00.248760+0000","last_undegraded":"2024-02-23T00:51:00.248760+0000","last_fullsized":"2024-02-23T00:51:00.248760+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:30.396865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:49:30.396865+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":447,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.239874+0000","last_change":"2024-02-23T00:49:40.533441+0000","last_active":"2024-02-23T00:51:00.239874+0000","last_peered":"2024-02-23T00:51:00.239874+0000","last_clean":"2024-02-23T00:51:00.239874+0000","last_became_active":"2024-02-23T00:47:28.212516+0000","last_became_peered":"2024-02-23T00:47:28.212516+0000","last_unstale":"2024-02-23T00:51:00.239874+0000","last_undegraded":"2024-02-23T00:51:00.239874+0000","last_fullsized":"2024-02-23T00:51:00.239874+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:40.533344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:49:40.533344+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":501,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.241915+0000","last_change":"2024-02-23T00:49:37.787041+0000","last_active":"2024-02-23T00:51:00.241915+0000","last_peered":"2024-02-23T00:51:00.241915+0000","last_clean":"2024-02-23T00:51:00.241915+0000","last_became_active":"2024-02-23T00:39:18.802134+0000","last_became_peered":"2024-02-23T00:39:18.802134+0000","last_unstale":"2024-02-23T00:51:00.241915+0000","last_undegraded":"2024-02-23T00:51:00.241915+0000","last_fullsized":"2024-02-23T00:51:00.241915+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:37.786978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:37.786978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"457'48","reported_seq":678,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.240998+0000","last_change":"2024-02-23T00:50:30.763802+0000","last_active":"2024-02-23T00:51:00.240998+0000","last_peered":"2024-02-23T00:51:00.240998+0000","last_clean":"2024-02-23T00:51:00.240998+0000","last_became_active":"2024-02-23T00:47:28.240257+0000","last_became_peered":"2024-02-23T00:47:28.240257+0000","last_unstale":"2024-02-23T00:51:00.240998+0000","last_undegraded":"2024-02-23T00:51:00.240998+0000","last_fullsized":"2024-02-23T00:51:00.240998+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'48","last_scrub_stamp":"2024-02-23T00:50:30.763686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:30.763686+0000","log_size":48,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":78,"num_read_kb":62,"num_write":44,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":504,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.240085+0000","last_change":"2024-02-23T00:49:41.639660+0000","last_active":"2024-02-23T00:51:00.240085+0000","last_peered":"2024-02-23T00:51:00.240085+0000","last_clean":"2024-02-23T00:51:00.240085+0000","last_became_active":"2024-02-23T00:39:14.784015+0000","last_became_peered":"2024-02-23T00:39:14.784015+0000","last_unstale":"2024-02-23T00:51:00.240085+0000","last_undegraded":"2024-02-23T00:51:00.240085+0000","last_fullsized":"2024-02-23T00:51:00.240085+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:41.639582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:41.639582+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":532,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.239437+0000","last_change":"2024-02-23T00:49:36.867589+0000","last_active":"2024-02-23T00:51:01.239437+0000","last_peered":"2024-02-23T00:51:01.239437+0000","last_clean":"2024-02-23T00:51:01.239437+0000","last_became_active":"2024-02-23T00:47:49.525421+0000","last_became_peered":"2024-02-23T00:47:49.525421+0000","last_unstale":"2024-02-23T00:51:01.239437+0000","last_undegraded":"2024-02-23T00:51:01.239437+0000","last_fullsized":"2024-02-23T00:51:01.239437+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":397,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:36.867488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:49:36.867488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":175,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.237063+0000","last_change":"2024-02-23T00:50:13.549604+0000","last_active":"2024-02-23T00:51:00.237063+0000","last_peered":"2024-02-23T00:51:00.237063+0000","last_clean":"2024-02-23T00:51:00.237063+0000","last_became_active":"2024-02-23T00:47:28.232194+0000","last_became_peered":"2024-02-23T00:47:28.232194+0000","last_unstale":"2024-02-23T00:51:00.237063+0000","last_undegraded":"2024-02-23T00:51:00.237063+0000","last_fullsized":"2024-02-23T00:51:00.237063+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:13.549529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:13.549529+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3","version":"91'9","reported_seq":651,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.240282+0000","last_change":"2024-02-23T00:49:41.776249+0000","last_active":"2024-02-23T00:51:00.240282+0000","last_peered":"2024-02-23T00:51:00.240282+0000","last_clean":"2024-02-23T00:51:00.240282+0000","last_became_active":"2024-02-23T00:39:18.801297+0000","last_became_peered":"2024-02-23T00:39:18.801297+0000","last_unstale":"2024-02-23T00:51:00.240282+0000","last_undegraded":"2024-02-23T00:51:00.240282+0000","last_fullsized":"2024-02-23T00:51:00.240282+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'9","last_scrub_stamp":"2024-02-23T00:49:41.776117+0000","last_deep_scrub":"91'9","last_deep_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_clean_scrub_stamp":"2024-02-23T00:49:41.776117+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"457'26","reported_seq":539,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.241070+0000","last_change":"2024-02-23T00:48:55.589320+0000","last_active":"2024-02-23T00:51:00.241070+0000","last_peered":"2024-02-23T00:51:00.241070+0000","last_clean":"2024-02-23T00:51:00.241070+0000","last_became_active":"2024-02-23T00:39:16.787468+0000","last_became_peered":"2024-02-23T00:39:16.787468+0000","last_unstale":"2024-02-23T00:51:00.241070+0000","last_undegraded":"2024-02-23T00:51:00.241070+0000","last_fullsized":"2024-02-23T00:51:00.241070+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:48:55.589209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:48:55.589209+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":36,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":496,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.240242+0000","last_change":"2024-02-23T00:48:12.869003+0000","last_active":"2024-02-23T00:51:00.240242+0000","last_peered":"2024-02-23T00:51:00.240242+0000","last_clean":"2024-02-23T00:51:00.240242+0000","last_became_active":"2024-02-23T00:39:14.779952+0000","last_became_peered":"2024-02-23T00:39:14.779952+0000","last_unstale":"2024-02-23T00:51:00.240242+0000","last_undegraded":"2024-02-23T00:51:00.240242+0000","last_fullsized":"2024-02-23T00:51:00.240242+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:12.868930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:48:12.868930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"107'2","reported_seq":534,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.239592+0000","last_change":"2024-02-23T00:50:17.587665+0000","last_active":"2024-02-23T00:51:00.239592+0000","last_peered":"2024-02-23T00:51:00.239592+0000","last_clean":"2024-02-23T00:51:00.239592+0000","last_became_active":"2024-02-23T00:47:41.210074+0000","last_became_peered":"2024-02-23T00:47:41.210074+0000","last_unstale":"2024-02-23T00:51:00.239592+0000","last_undegraded":"2024-02-23T00:51:00.239592+0000","last_fullsized":"2024-02-23T00:51:00.239592+0000","mapping_epoch":375,"log_start":"107'2","ondisk_log_start":"107'2","created":65,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'2","last_scrub_stamp":"2024-02-23T00:50:17.587591+0000","last_deep_scrub":"107'2","last_deep_scrub_stamp":"2024-02-23T00:42:48.362739+0000","last_clean_scrub_stamp":"2024-02-23T00:50:17.587591+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":453,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.242522+0000","last_change":"2024-02-23T00:50:18.308436+0000","last_active":"2024-02-23T00:51:00.242522+0000","last_peered":"2024-02-23T00:51:00.242522+0000","last_clean":"2024-02-23T00:51:00.242522+0000","last_became_active":"2024-02-23T00:47:28.242255+0000","last_became_peered":"2024-02-23T00:47:28.242255+0000","last_unstale":"2024-02-23T00:51:00.242522+0000","last_undegraded":"2024-02-23T00:51:00.242522+0000","last_fullsized":"2024-02-23T00:51:00.242522+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:18.308350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:18.308350+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":173,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.240078+0000","last_change":"2024-02-23T00:48:54.706708+0000","last_active":"2024-02-23T00:51:00.240078+0000","last_peered":"2024-02-23T00:51:00.240078+0000","last_clean":"2024-02-23T00:51:00.240078+0000","last_became_active":"2024-02-23T00:47:28.234034+0000","last_became_peered":"2024-02-23T00:47:28.234034+0000","last_unstale":"2024-02-23T00:51:00.240078+0000","last_undegraded":"2024-02-23T00:51:00.240078+0000","last_fullsized":"2024-02-23T00:51:00.240078+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:54.706627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_clean_scrub_stamp":"2024-02-23T00:48:54.706627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"457'31","reported_seq":560,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.237564+0000","last_change":"2024-02-23T00:49:08.190437+0000","last_active":"2024-02-23T00:51:00.237564+0000","last_peered":"2024-02-23T00:51:00.237564+0000","last_clean":"2024-02-23T00:51:00.237564+0000","last_became_active":"2024-02-23T00:39:16.783926+0000","last_became_peered":"2024-02-23T00:39:16.783926+0000","last_unstale":"2024-02-23T00:51:00.237564+0000","last_undegraded":"2024-02-23T00:51:00.237564+0000","last_fullsized":"2024-02-23T00:51:00.237564+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'25","last_scrub_stamp":"2024-02-23T00:49:08.190363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:08.190363+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":58,"num_read_kb":47,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"95'4","reported_seq":519,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.237694+0000","last_change":"2024-02-23T00:49:33.465769+0000","last_active":"2024-02-23T00:51:00.237694+0000","last_peered":"2024-02-23T00:51:00.237694+0000","last_clean":"2024-02-23T00:51:00.237694+0000","last_became_active":"2024-02-23T00:39:14.779205+0000","last_became_peered":"2024-02-23T00:39:14.779205+0000","last_unstale":"2024-02-23T00:51:00.237694+0000","last_undegraded":"2024-02-23T00:51:00.237694+0000","last_fullsized":"2024-02-23T00:51:00.237694+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'4","last_scrub_stamp":"2024-02-23T00:49:33.465697+0000","last_deep_scrub":"95'4","last_deep_scrub_stamp":"2024-02-23T00:41:43.874397+0000","last_clean_scrub_stamp":"2024-02-23T00:49:33.465697+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":519,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.238950+0000","last_change":"2024-02-23T00:50:15.922829+0000","last_active":"2024-02-23T00:51:01.238950+0000","last_peered":"2024-02-23T00:51:01.238950+0000","last_clean":"2024-02-23T00:51:01.238950+0000","last_became_active":"2024-02-23T00:43:27.611660+0000","last_became_peered":"2024-02-23T00:43:27.611660+0000","last_unstale":"2024-02-23T00:51:01.238950+0000","last_undegraded":"2024-02-23T00:51:01.238950+0000","last_fullsized":"2024-02-23T00:51:01.238950+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:15.922745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:50:15.922745+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":455,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.249047+0000","last_change":"2024-02-23T00:49:44.143768+0000","last_active":"2024-02-23T00:51:00.249047+0000","last_peered":"2024-02-23T00:51:00.249047+0000","last_clean":"2024-02-23T00:51:00.249047+0000","last_became_active":"2024-02-23T00:47:28.231670+0000","last_became_peered":"2024-02-23T00:47:28.231670+0000","last_unstale":"2024-02-23T00:51:00.249047+0000","last_undegraded":"2024-02-23T00:51:00.249047+0000","last_fullsized":"2024-02-23T00:51:00.249047+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:44.143684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:49:44.143684+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"457'27","reported_seq":553,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.241753+0000","last_change":"2024-02-23T00:49:36.823322+0000","last_active":"2024-02-23T00:51:00.241753+0000","last_peered":"2024-02-23T00:51:00.241753+0000","last_clean":"2024-02-23T00:51:00.241753+0000","last_became_active":"2024-02-23T00:39:16.858760+0000","last_became_peered":"2024-02-23T00:39:16.858760+0000","last_unstale":"2024-02-23T00:51:00.241753+0000","last_undegraded":"2024-02-23T00:51:00.241753+0000","last_fullsized":"2024-02-23T00:51:00.241753+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-23T00:49:36.823214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:36.823214+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":171,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.240192+0000","last_change":"2024-02-23T00:49:02.601790+0000","last_active":"2024-02-23T00:51:00.240192+0000","last_peered":"2024-02-23T00:51:00.240192+0000","last_clean":"2024-02-23T00:51:00.240192+0000","last_became_active":"2024-02-23T00:47:28.219764+0000","last_became_peered":"2024-02-23T00:47:28.219764+0000","last_unstale":"2024-02-23T00:51:00.240192+0000","last_undegraded":"2024-02-23T00:51:00.240192+0000","last_fullsized":"2024-02-23T00:51:00.240192+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:02.601726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:02.601726+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":503,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.237840+0000","last_change":"2024-02-23T00:49:37.881251+0000","last_active":"2024-02-23T00:51:01.237840+0000","last_peered":"2024-02-23T00:51:01.237840+0000","last_clean":"2024-02-23T00:51:01.237840+0000","last_became_active":"2024-02-23T00:39:18.798624+0000","last_became_peered":"2024-02-23T00:39:18.798624+0000","last_unstale":"2024-02-23T00:51:01.237840+0000","last_undegraded":"2024-02-23T00:51:01.237840+0000","last_fullsized":"2024-02-23T00:51:01.237840+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:37.881183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:45.913526+0000","last_clean_scrub_stamp":"2024-02-23T00:49:37.881183+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":446,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.239810+0000","last_change":"2024-02-23T00:50:13.642424+0000","last_active":"2024-02-23T00:51:00.239810+0000","last_peered":"2024-02-23T00:51:00.239810+0000","last_clean":"2024-02-23T00:51:00.239810+0000","last_became_active":"2024-02-23T00:42:58.825015+0000","last_became_peered":"2024-02-23T00:42:58.825015+0000","last_unstale":"2024-02-23T00:51:00.239810+0000","last_undegraded":"2024-02-23T00:51:00.239810+0000","last_fullsized":"2024-02-23T00:51:00.239810+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:13.642359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:13.642359+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"457'22","reported_seq":542,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.237794+0000","last_change":"2024-02-23T00:51:01.201807+0000","last_active":"2024-02-23T00:51:01.237794+0000","last_peered":"2024-02-23T00:51:01.237794+0000","last_clean":"2024-02-23T00:51:01.237794+0000","last_became_active":"2024-02-23T00:39:16.858449+0000","last_became_peered":"2024-02-23T00:39:16.858449+0000","last_unstale":"2024-02-23T00:51:01.237794+0000","last_undegraded":"2024-02-23T00:51:01.237794+0000","last_fullsized":"2024-02-23T00:51:01.237794+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'22","last_scrub_stamp":"2024-02-23T00:51:01.201737+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-02-23T00:48:07.941101+0000","last_clean_scrub_stamp":"2024-02-23T00:51:01.201737+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":30,"num_read_kb":26,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":502,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.243339+0000","last_change":"2024-02-23T00:49:28.347354+0000","last_active":"2024-02-23T00:51:00.243339+0000","last_peered":"2024-02-23T00:51:00.243339+0000","last_clean":"2024-02-23T00:51:00.243339+0000","last_became_active":"2024-02-23T00:39:14.783164+0000","last_became_peered":"2024-02-23T00:39:14.783164+0000","last_unstale":"2024-02-23T00:51:00.243339+0000","last_undegraded":"2024-02-23T00:51:00.243339+0000","last_fullsized":"2024-02-23T00:51:00.243339+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:28.347281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:28.347281+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":500,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.241409+0000","last_change":"2024-02-23T00:49:37.721639+0000","last_active":"2024-02-23T00:51:00.241409+0000","last_peered":"2024-02-23T00:51:00.241409+0000","last_clean":"2024-02-23T00:51:00.241409+0000","last_became_active":"2024-02-23T00:39:18.800297+0000","last_became_peered":"2024-02-23T00:39:18.800297+0000","last_unstale":"2024-02-23T00:51:00.241409+0000","last_undegraded":"2024-02-23T00:51:00.241409+0000","last_fullsized":"2024-02-23T00:51:00.241409+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:37.721563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:37.721563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1","version":"110'7","reported_seq":456,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.237235+0000","last_change":"2024-02-23T00:48:57.168061+0000","last_active":"2024-02-23T00:51:01.237235+0000","last_peered":"2024-02-23T00:51:01.237235+0000","last_clean":"2024-02-23T00:51:01.237235+0000","last_became_active":"2024-02-23T00:42:58.777251+0000","last_became_peered":"2024-02-23T00:42:58.777251+0000","last_unstale":"2024-02-23T00:51:01.237235+0000","last_undegraded":"2024-02-23T00:51:01.237235+0000","last_fullsized":"2024-02-23T00:51:01.237235+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'7","last_scrub_stamp":"2024-02-23T00:48:57.167981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:57.167981+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"457'32","reported_seq":203,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.237217+0000","last_change":"2024-02-23T00:50:25.079859+0000","last_active":"2024-02-23T00:51:01.237217+0000","last_peered":"2024-02-23T00:51:01.237217+0000","last_clean":"2024-02-23T00:51:01.237217+0000","last_became_active":"2024-02-23T00:47:28.237898+0000","last_became_peered":"2024-02-23T00:47:28.237898+0000","last_unstale":"2024-02-23T00:51:01.237217+0000","last_undegraded":"2024-02-23T00:51:01.237217+0000","last_fullsized":"2024-02-23T00:51:01.237217+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-23T00:50:25.079789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:25.079789+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":52,"num_read_kb":45,"num_write":32,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":504,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.240011+0000","last_change":"2024-02-23T00:49:33.731301+0000","last_active":"2024-02-23T00:51:00.240011+0000","last_peered":"2024-02-23T00:51:00.240011+0000","last_clean":"2024-02-23T00:51:00.240011+0000","last_became_active":"2024-02-23T00:39:14.785590+0000","last_became_peered":"2024-02-23T00:39:14.785590+0000","last_unstale":"2024-02-23T00:51:00.240011+0000","last_undegraded":"2024-02-23T00:51:00.240011+0000","last_fullsized":"2024-02-23T00:51:00.240011+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:33.731192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_clean_scrub_stamp":"2024-02-23T00:49:33.731192+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"336'9","reported_seq":650,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.241208+0000","last_change":"2024-02-23T00:49:39.636503+0000","last_active":"2024-02-23T00:51:00.241208+0000","last_peered":"2024-02-23T00:51:00.241208+0000","last_clean":"2024-02-23T00:51:00.241208+0000","last_became_active":"2024-02-23T00:39:18.803566+0000","last_became_peered":"2024-02-23T00:39:18.803566+0000","last_unstale":"2024-02-23T00:51:00.241208+0000","last_undegraded":"2024-02-23T00:51:00.241208+0000","last_fullsized":"2024-02-23T00:51:00.241208+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"336'9","last_scrub_stamp":"2024-02-23T00:49:39.636409+0000","last_deep_scrub":"336'9","last_deep_scrub_stamp":"2024-02-23T00:46:49.160991+0000","last_clean_scrub_stamp":"2024-02-23T00:49:39.636409+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":441,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.248050+0000","last_change":"2024-02-23T00:48:41.688804+0000","last_active":"2024-02-23T00:51:00.248050+0000","last_peered":"2024-02-23T00:51:00.248050+0000","last_clean":"2024-02-23T00:51:00.248050+0000","last_became_active":"2024-02-23T00:42:58.757424+0000","last_became_peered":"2024-02-23T00:42:58.757424+0000","last_unstale":"2024-02-23T00:51:00.248050+0000","last_undegraded":"2024-02-23T00:51:00.248050+0000","last_fullsized":"2024-02-23T00:51:00.248050+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:41.688728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:41.688728+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"457'21","reported_seq":542,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.241030+0000","last_change":"2024-02-23T00:50:17.861556+0000","last_active":"2024-02-23T00:51:00.241030+0000","last_peered":"2024-02-23T00:51:00.241030+0000","last_clean":"2024-02-23T00:51:00.241030+0000","last_became_active":"2024-02-23T00:39:16.858953+0000","last_became_peered":"2024-02-23T00:39:16.858953+0000","last_unstale":"2024-02-23T00:51:00.241030+0000","last_undegraded":"2024-02-23T00:51:00.241030+0000","last_fullsized":"2024-02-23T00:51:00.241030+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-23T00:50:17.861490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:17.861490+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":505,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.240098+0000","last_change":"2024-02-23T00:48:59.712766+0000","last_active":"2024-02-23T00:51:00.240098+0000","last_peered":"2024-02-23T00:51:00.240098+0000","last_clean":"2024-02-23T00:51:00.240098+0000","last_became_active":"2024-02-23T00:39:14.773772+0000","last_became_peered":"2024-02-23T00:39:14.773772+0000","last_unstale":"2024-02-23T00:51:00.240098+0000","last_undegraded":"2024-02-23T00:51:00.240098+0000","last_fullsized":"2024-02-23T00:51:00.240098+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:59.712699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:48:59.712699+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"376'9","reported_seq":660,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.237707+0000","last_change":"2024-02-23T00:49:45.868359+0000","last_active":"2024-02-23T00:51:01.237707+0000","last_peered":"2024-02-23T00:51:01.237707+0000","last_clean":"2024-02-23T00:51:01.237707+0000","last_became_active":"2024-02-23T00:47:28.229316+0000","last_became_peered":"2024-02-23T00:47:28.229316+0000","last_unstale":"2024-02-23T00:51:01.237707+0000","last_undegraded":"2024-02-23T00:51:01.237707+0000","last_fullsized":"2024-02-23T00:51:01.237707+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"376'9","last_scrub_stamp":"2024-02-23T00:49:45.868289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:45.868289+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":438,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.240748+0000","last_change":"2024-02-23T00:50:15.714265+0000","last_active":"2024-02-23T00:51:00.240748+0000","last_peered":"2024-02-23T00:51:00.240748+0000","last_clean":"2024-02-23T00:51:00.240748+0000","last_became_active":"2024-02-23T00:42:57.652488+0000","last_became_peered":"2024-02-23T00:42:57.652488+0000","last_unstale":"2024-02-23T00:51:00.240748+0000","last_undegraded":"2024-02-23T00:51:00.240748+0000","last_fullsized":"2024-02-23T00:51:00.240748+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:15.714200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:51.254903+0000","last_clean_scrub_stamp":"2024-02-23T00:50:15.714200+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"457'15","reported_seq":538,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.240898+0000","last_change":"2024-02-23T00:49:39.573557+0000","last_active":"2024-02-23T00:51:00.240898+0000","last_peered":"2024-02-23T00:51:00.240898+0000","last_clean":"2024-02-23T00:51:00.240898+0000","last_became_active":"2024-02-23T00:47:28.225585+0000","last_became_peered":"2024-02-23T00:47:28.225585+0000","last_unstale":"2024-02-23T00:51:00.240898+0000","last_undegraded":"2024-02-23T00:51:00.240898+0000","last_fullsized":"2024-02-23T00:51:00.240898+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'15","last_scrub_stamp":"2024-02-23T00:49:39.573472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:39.573472+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":507,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.242422+0000","last_change":"2024-02-23T00:49:02.303864+0000","last_active":"2024-02-23T00:51:00.242422+0000","last_peered":"2024-02-23T00:51:00.242422+0000","last_clean":"2024-02-23T00:51:00.242422+0000","last_became_active":"2024-02-23T00:39:14.778455+0000","last_became_peered":"2024-02-23T00:39:14.778455+0000","last_unstale":"2024-02-23T00:51:00.242422+0000","last_undegraded":"2024-02-23T00:51:00.242422+0000","last_fullsized":"2024-02-23T00:51:00.242422+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:02.303789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:02.303789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":512,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.238873+0000","last_change":"2024-02-23T00:50:30.069404+0000","last_active":"2024-02-23T00:51:01.238873+0000","last_peered":"2024-02-23T00:51:01.238873+0000","last_clean":"2024-02-23T00:51:01.238873+0000","last_became_active":"2024-02-23T00:47:44.225379+0000","last_became_peered":"2024-02-23T00:47:44.225379+0000","last_unstale":"2024-02-23T00:51:01.238873+0000","last_undegraded":"2024-02-23T00:51:01.238873+0000","last_fullsized":"2024-02-23T00:51:01.238873+0000","mapping_epoch":391,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":392,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:30.069325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_clean_scrub_stamp":"2024-02-23T00:50:30.069325+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":446,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.240567+0000","last_change":"2024-02-23T00:49:40.770764+0000","last_active":"2024-02-23T00:51:00.240567+0000","last_peered":"2024-02-23T00:51:00.240567+0000","last_clean":"2024-02-23T00:51:00.240567+0000","last_became_active":"2024-02-23T00:42:58.835862+0000","last_became_peered":"2024-02-23T00:42:58.835862+0000","last_unstale":"2024-02-23T00:51:00.240567+0000","last_undegraded":"2024-02-23T00:51:00.240567+0000","last_fullsized":"2024-02-23T00:51:00.240567+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:40.770694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:49:40.770694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"457'19","reported_seq":533,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.237188+0000","last_change":"2024-02-23T00:50:29.599563+0000","last_active":"2024-02-23T00:51:00.237188+0000","last_peered":"2024-02-23T00:51:00.237188+0000","last_clean":"2024-02-23T00:51:00.237188+0000","last_became_active":"2024-02-23T00:39:16.783896+0000","last_became_peered":"2024-02-23T00:39:16.783896+0000","last_unstale":"2024-02-23T00:51:00.237188+0000","last_undegraded":"2024-02-23T00:51:00.237188+0000","last_fullsized":"2024-02-23T00:51:00.237188+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'19","last_scrub_stamp":"2024-02-23T00:50:29.599483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:29.599483+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"202'2","reported_seq":504,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.242394+0000","last_change":"2024-02-23T00:48:04.832975+0000","last_active":"2024-02-23T00:51:00.242394+0000","last_peered":"2024-02-23T00:51:00.242394+0000","last_clean":"2024-02-23T00:51:00.242394+0000","last_became_active":"2024-02-23T00:39:14.775804+0000","last_became_peered":"2024-02-23T00:39:14.775804+0000","last_unstale":"2024-02-23T00:51:00.242394+0000","last_undegraded":"2024-02-23T00:51:00.242394+0000","last_fullsized":"2024-02-23T00:51:00.242394+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"202'2","last_scrub_stamp":"2024-02-23T00:48:04.832904+0000","last_deep_scrub":"202'2","last_deep_scrub_stamp":"2024-02-23T00:44:34.720640+0000","last_clean_scrub_stamp":"2024-02-23T00:48:04.832904+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":492,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.240988+0000","last_change":"2024-02-23T00:50:18.698805+0000","last_active":"2024-02-23T00:51:00.240988+0000","last_peered":"2024-02-23T00:51:00.240988+0000","last_clean":"2024-02-23T00:51:00.240988+0000","last_became_active":"2024-02-23T00:39:18.800382+0000","last_became_peered":"2024-02-23T00:39:18.800382+0000","last_unstale":"2024-02-23T00:51:00.240988+0000","last_undegraded":"2024-02-23T00:51:00.240988+0000","last_fullsized":"2024-02-23T00:51:00.240988+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:18.698687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:50:18.698687+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":447,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.242339+0000","last_change":"2024-02-23T00:49:46.462085+0000","last_active":"2024-02-23T00:51:00.242339+0000","last_peered":"2024-02-23T00:51:00.242339+0000","last_clean":"2024-02-23T00:51:00.242339+0000","last_became_active":"2024-02-23T00:47:28.224961+0000","last_became_peered":"2024-02-23T00:47:28.224961+0000","last_unstale":"2024-02-23T00:51:00.242339+0000","last_undegraded":"2024-02-23T00:51:00.242339+0000","last_fullsized":"2024-02-23T00:51:00.242339+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:46.462015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:46.462015+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.462015+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"457'8","reported_seq":517,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.236865+0000","last_change":"2024-02-23T00:50:35.045697+0000","last_active":"2024-02-23T00:51:01.236865+0000","last_peered":"2024-02-23T00:51:01.236865+0000","last_clean":"2024-02-23T00:51:01.236865+0000","last_became_active":"2024-02-23T00:39:16.785830+0000","last_became_peered":"2024-02-23T00:39:16.785830+0000","last_unstale":"2024-02-23T00:51:01.236865+0000","last_undegraded":"2024-02-23T00:51:01.236865+0000","last_fullsized":"2024-02-23T00:51:01.236865+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'8","last_scrub_stamp":"2024-02-23T00:50:35.045612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:35.045612+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":516,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.241316+0000","last_change":"2024-02-23T00:49:26.672761+0000","last_active":"2024-02-23T00:51:00.241316+0000","last_peered":"2024-02-23T00:51:00.241316+0000","last_clean":"2024-02-23T00:51:00.241316+0000","last_became_active":"2024-02-23T00:47:28.224477+0000","last_became_peered":"2024-02-23T00:47:28.224477+0000","last_unstale":"2024-02-23T00:51:00.241316+0000","last_undegraded":"2024-02-23T00:51:00.241316+0000","last_fullsized":"2024-02-23T00:51:00.241316+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:26.672673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:26.672673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":512,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.243717+0000","last_change":"2024-02-23T00:49:33.461971+0000","last_active":"2024-02-23T00:51:00.243717+0000","last_peered":"2024-02-23T00:51:00.243717+0000","last_clean":"2024-02-23T00:51:00.243717+0000","last_became_active":"2024-02-23T00:47:44.230295+0000","last_became_peered":"2024-02-23T00:47:44.230295+0000","last_unstale":"2024-02-23T00:51:00.243717+0000","last_undegraded":"2024-02-23T00:51:00.243717+0000","last_fullsized":"2024-02-23T00:51:00.243717+0000","mapping_epoch":391,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":392,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:33.461907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:33.461907+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":441,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.248990+0000","last_change":"2024-02-23T00:48:51.717473+0000","last_active":"2024-02-23T00:51:00.248990+0000","last_peered":"2024-02-23T00:51:00.248990+0000","last_clean":"2024-02-23T00:51:00.248990+0000","last_became_active":"2024-02-23T00:42:58.741365+0000","last_became_peered":"2024-02-23T00:42:58.741365+0000","last_unstale":"2024-02-23T00:51:00.248990+0000","last_undegraded":"2024-02-23T00:51:00.248990+0000","last_fullsized":"2024-02-23T00:51:00.248990+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:51.717397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:51.717397+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"457'25","reported_seq":551,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.237324+0000","last_change":"2024-02-23T00:49:47.788110+0000","last_active":"2024-02-23T00:51:01.237324+0000","last_peered":"2024-02-23T00:51:01.237324+0000","last_clean":"2024-02-23T00:51:01.237324+0000","last_became_active":"2024-02-23T00:39:16.857232+0000","last_became_peered":"2024-02-23T00:39:16.857232+0000","last_unstale":"2024-02-23T00:51:01.237324+0000","last_undegraded":"2024-02-23T00:51:01.237324+0000","last_fullsized":"2024-02-23T00:51:01.237324+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-23T00:49:47.788044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:47.788044+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":511,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.243270+0000","last_change":"2024-02-23T00:49:05.329264+0000","last_active":"2024-02-23T00:51:00.243270+0000","last_peered":"2024-02-23T00:51:00.243270+0000","last_clean":"2024-02-23T00:51:00.243270+0000","last_became_active":"2024-02-23T00:47:28.240162+0000","last_became_peered":"2024-02-23T00:47:28.240162+0000","last_unstale":"2024-02-23T00:51:00.243270+0000","last_undegraded":"2024-02-23T00:51:00.243270+0000","last_fullsized":"2024-02-23T00:51:00.243270+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:05.329178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:44.285083+0000","last_clean_scrub_stamp":"2024-02-23T00:49:05.329178+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":156,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.236700+0000","last_change":"2024-02-23T00:49:42.872820+0000","last_active":"2024-02-23T00:51:01.236700+0000","last_peered":"2024-02-23T00:51:01.236700+0000","last_clean":"2024-02-23T00:51:01.236700+0000","last_became_active":"2024-02-23T00:47:44.226097+0000","last_became_peered":"2024-02-23T00:47:44.226097+0000","last_unstale":"2024-02-23T00:51:01.236700+0000","last_undegraded":"2024-02-23T00:51:01.236700+0000","last_fullsized":"2024-02-23T00:51:01.236700+0000","mapping_epoch":391,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":392,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:42.872748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:42.872748+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":178,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.241733+0000","last_change":"2024-02-23T00:50:28.678155+0000","last_active":"2024-02-23T00:51:00.241733+0000","last_peered":"2024-02-23T00:51:00.241733+0000","last_clean":"2024-02-23T00:51:00.241733+0000","last_became_active":"2024-02-23T00:47:28.200688+0000","last_became_peered":"2024-02-23T00:47:28.200688+0000","last_unstale":"2024-02-23T00:51:00.241733+0000","last_undegraded":"2024-02-23T00:51:00.241733+0000","last_fullsized":"2024-02-23T00:51:00.241733+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:28.678041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:28.678041+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"457'26","reported_seq":548,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.242108+0000","last_change":"2024-02-23T00:50:23.540361+0000","last_active":"2024-02-23T00:51:00.242108+0000","last_peered":"2024-02-23T00:51:00.242108+0000","last_clean":"2024-02-23T00:51:00.242108+0000","last_became_active":"2024-02-23T00:39:16.858945+0000","last_became_peered":"2024-02-23T00:39:16.858945+0000","last_unstale":"2024-02-23T00:51:00.242108+0000","last_undegraded":"2024-02-23T00:51:00.242108+0000","last_fullsized":"2024-02-23T00:51:00.242108+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-23T00:50:23.540232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:23.540232+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":36,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":511,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.239017+0000","last_change":"2024-02-23T00:49:02.482297+0000","last_active":"2024-02-23T00:51:00.239017+0000","last_peered":"2024-02-23T00:51:00.239017+0000","last_clean":"2024-02-23T00:51:00.239017+0000","last_became_active":"2024-02-23T00:39:14.785363+0000","last_became_peered":"2024-02-23T00:39:14.785363+0000","last_unstale":"2024-02-23T00:51:00.239017+0000","last_undegraded":"2024-02-23T00:51:00.239017+0000","last_fullsized":"2024-02-23T00:51:00.239017+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"80'2","last_scrub_stamp":"2024-02-23T00:49:02.482210+0000","last_deep_scrub":"80'2","last_deep_scrub_stamp":"2024-02-23T00:40:15.942725+0000","last_clean_scrub_stamp":"2024-02-23T00:49:02.482210+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"93'9","reported_seq":651,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.243448+0000","last_change":"2024-02-23T00:48:53.246622+0000","last_active":"2024-02-23T00:51:00.243448+0000","last_peered":"2024-02-23T00:51:00.243448+0000","last_clean":"2024-02-23T00:51:00.243448+0000","last_became_active":"2024-02-23T00:39:18.796680+0000","last_became_peered":"2024-02-23T00:39:18.796680+0000","last_unstale":"2024-02-23T00:51:00.243448+0000","last_undegraded":"2024-02-23T00:51:00.243448+0000","last_fullsized":"2024-02-23T00:51:00.243448+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-02-23T00:48:53.246535+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-02-23T00:43:50.393713+0000","last_clean_scrub_stamp":"2024-02-23T00:48:53.246535+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":450,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.240169+0000","last_change":"2024-02-23T00:48:17.898787+0000","last_active":"2024-02-23T00:51:00.240169+0000","last_peered":"2024-02-23T00:51:00.240169+0000","last_clean":"2024-02-23T00:51:00.240169+0000","last_became_active":"2024-02-23T00:47:28.234641+0000","last_became_peered":"2024-02-23T00:47:28.234641+0000","last_unstale":"2024-02-23T00:51:00.240169+0000","last_undegraded":"2024-02-23T00:51:00.240169+0000","last_fullsized":"2024-02-23T00:51:00.240169+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:17.898720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:17.898720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"457'31","reported_seq":557,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.243538+0000","last_change":"2024-02-23T00:48:45.295904+0000","last_active":"2024-02-23T00:51:00.243538+0000","last_peered":"2024-02-23T00:51:00.243538+0000","last_clean":"2024-02-23T00:51:00.243538+0000","last_became_active":"2024-02-23T00:39:16.782660+0000","last_became_peered":"2024-02-23T00:39:16.782660+0000","last_unstale":"2024-02-23T00:51:00.243538+0000","last_undegraded":"2024-02-23T00:51:00.243538+0000","last_fullsized":"2024-02-23T00:51:00.243538+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'23","last_scrub_stamp":"2024-02-23T00:48:45.295833+0000","last_deep_scrub":"93'23","last_deep_scrub_stamp":"2024-02-23T00:41:35.954516+0000","last_clean_scrub_stamp":"2024-02-23T00:48:45.295833+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":507,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.236933+0000","last_change":"2024-02-23T00:50:29.029794+0000","last_active":"2024-02-23T00:51:01.236933+0000","last_peered":"2024-02-23T00:51:01.236933+0000","last_clean":"2024-02-23T00:51:01.236933+0000","last_became_active":"2024-02-23T00:39:14.781560+0000","last_became_peered":"2024-02-23T00:39:14.781560+0000","last_unstale":"2024-02-23T00:51:01.236933+0000","last_undegraded":"2024-02-23T00:51:01.236933+0000","last_fullsized":"2024-02-23T00:51:01.236933+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:29.029725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:29.029725+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":510,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.241957+0000","last_change":"2024-02-23T00:48:53.747314+0000","last_active":"2024-02-23T00:51:00.241957+0000","last_peered":"2024-02-23T00:51:00.241957+0000","last_clean":"2024-02-23T00:51:00.241957+0000","last_became_active":"2024-02-23T00:47:28.232797+0000","last_became_peered":"2024-02-23T00:47:28.232797+0000","last_unstale":"2024-02-23T00:51:00.241957+0000","last_undegraded":"2024-02-23T00:51:00.241957+0000","last_fullsized":"2024-02-23T00:51:00.241957+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:53.747243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:53.189881+0000","last_clean_scrub_stamp":"2024-02-23T00:48:53.747243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":448,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.236960+0000","last_change":"2024-02-23T00:51:00.232343+0000","last_active":"2024-02-23T00:51:01.236960+0000","last_peered":"2024-02-23T00:51:01.236960+0000","last_clean":"2024-02-23T00:51:01.236960+0000","last_became_active":"2024-02-23T00:42:58.774720+0000","last_became_peered":"2024-02-23T00:42:58.774720+0000","last_unstale":"2024-02-23T00:51:01.236960+0000","last_undegraded":"2024-02-23T00:51:01.236960+0000","last_fullsized":"2024-02-23T00:51:01.236960+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.232261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.232261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"457'27","reported_seq":242,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.240337+0000","last_change":"2024-02-23T00:49:26.518048+0000","last_active":"2024-02-23T00:51:00.240337+0000","last_peered":"2024-02-23T00:51:00.240337+0000","last_clean":"2024-02-23T00:51:00.240337+0000","last_became_active":"2024-02-23T00:47:28.211342+0000","last_became_peered":"2024-02-23T00:47:28.211342+0000","last_unstale":"2024-02-23T00:51:00.240337+0000","last_undegraded":"2024-02-23T00:51:00.240337+0000","last_fullsized":"2024-02-23T00:51:00.240337+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-23T00:49:26.517965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:26.517965+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":502,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.242425+0000","last_change":"2024-02-23T00:48:10.563771+0000","last_active":"2024-02-23T00:51:00.242425+0000","last_peered":"2024-02-23T00:51:00.242425+0000","last_clean":"2024-02-23T00:51:00.242425+0000","last_became_active":"2024-02-23T00:39:14.782494+0000","last_became_peered":"2024-02-23T00:39:14.782494+0000","last_unstale":"2024-02-23T00:51:00.242425+0000","last_undegraded":"2024-02-23T00:51:00.242425+0000","last_fullsized":"2024-02-23T00:51:00.242425+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:10.563695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:48:10.563695+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":492,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.242506+0000","last_change":"2024-02-23T00:48:16.788875+0000","last_active":"2024-02-23T00:51:00.242506+0000","last_peered":"2024-02-23T00:51:00.242506+0000","last_clean":"2024-02-23T00:51:00.242506+0000","last_became_active":"2024-02-23T00:39:18.801039+0000","last_became_peered":"2024-02-23T00:39:18.801039+0000","last_unstale":"2024-02-23T00:51:00.242506+0000","last_undegraded":"2024-02-23T00:51:00.242506+0000","last_fullsized":"2024-02-23T00:51:00.242506+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:16.788806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.641689+0000","last_clean_scrub_stamp":"2024-02-23T00:48:16.788806+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":443,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.238442+0000","last_change":"2024-02-23T00:48:10.933441+0000","last_active":"2024-02-23T00:51:01.238442+0000","last_peered":"2024-02-23T00:51:01.238442+0000","last_clean":"2024-02-23T00:51:01.238442+0000","last_became_active":"2024-02-23T00:42:58.770759+0000","last_became_peered":"2024-02-23T00:42:58.770759+0000","last_unstale":"2024-02-23T00:51:01.238442+0000","last_undegraded":"2024-02-23T00:51:01.238442+0000","last_fullsized":"2024-02-23T00:51:01.238442+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:10.933375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:10.933375+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"457'24","reported_seq":552,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.240259+0000","last_change":"2024-02-23T00:50:13.861288+0000","last_active":"2024-02-23T00:51:01.240259+0000","last_peered":"2024-02-23T00:51:01.240259+0000","last_clean":"2024-02-23T00:51:01.240259+0000","last_became_active":"2024-02-23T00:39:16.782803+0000","last_became_peered":"2024-02-23T00:39:16.782803+0000","last_unstale":"2024-02-23T00:51:01.240259+0000","last_undegraded":"2024-02-23T00:51:01.240259+0000","last_fullsized":"2024-02-23T00:51:01.240259+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'24","last_scrub_stamp":"2024-02-23T00:50:13.861220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:13.861220+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":46,"num_read_kb":36,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":507,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.238793+0000","last_change":"2024-02-23T00:50:28.060186+0000","last_active":"2024-02-23T00:51:01.238793+0000","last_peered":"2024-02-23T00:51:01.238793+0000","last_clean":"2024-02-23T00:51:01.238793+0000","last_became_active":"2024-02-23T00:39:14.782163+0000","last_became_peered":"2024-02-23T00:39:14.782163+0000","last_unstale":"2024-02-23T00:51:01.238793+0000","last_undegraded":"2024-02-23T00:51:01.238793+0000","last_fullsized":"2024-02-23T00:51:01.238793+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:28.060109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:28.060109+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":509,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.237357+0000","last_change":"2024-02-23T00:50:18.629287+0000","last_active":"2024-02-23T00:51:00.237357+0000","last_peered":"2024-02-23T00:51:00.237357+0000","last_clean":"2024-02-23T00:51:00.237357+0000","last_became_active":"2024-02-23T00:47:28.215452+0000","last_became_peered":"2024-02-23T00:47:28.215452+0000","last_unstale":"2024-02-23T00:51:00.237357+0000","last_undegraded":"2024-02-23T00:51:00.237357+0000","last_fullsized":"2024-02-23T00:51:00.237357+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:18.629203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_clean_scrub_stamp":"2024-02-23T00:50:18.629203+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":454,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.240307+0000","last_change":"2024-02-23T00:50:56.774466+0000","last_active":"2024-02-23T00:51:00.240307+0000","last_peered":"2024-02-23T00:51:00.240307+0000","last_clean":"2024-02-23T00:51:00.240307+0000","last_became_active":"2024-02-23T00:47:28.203555+0000","last_became_peered":"2024-02-23T00:47:28.203555+0000","last_unstale":"2024-02-23T00:51:00.240307+0000","last_undegraded":"2024-02-23T00:51:00.240307+0000","last_fullsized":"2024-02-23T00:51:00.240307+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:56.774382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:56.774382+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"457'33","reported_seq":565,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.236538+0000","last_change":"2024-02-23T00:49:46.364498+0000","last_active":"2024-02-23T00:51:00.236538+0000","last_peered":"2024-02-23T00:51:00.236538+0000","last_clean":"2024-02-23T00:51:00.236538+0000","last_became_active":"2024-02-23T00:39:16.786791+0000","last_became_peered":"2024-02-23T00:39:16.786791+0000","last_unstale":"2024-02-23T00:51:00.236538+0000","last_undegraded":"2024-02-23T00:51:00.236538+0000","last_fullsized":"2024-02-23T00:51:00.236538+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'33","last_scrub_stamp":"2024-02-23T00:49:46.364418+0000","last_deep_scrub":"95'27","last_deep_scrub_stamp":"2024-02-23T00:41:55.919895+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.364418+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":512,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.248549+0000","last_change":"2024-02-23T00:49:27.403374+0000","last_active":"2024-02-23T00:51:00.248549+0000","last_peered":"2024-02-23T00:51:00.248549+0000","last_clean":"2024-02-23T00:51:00.248549+0000","last_became_active":"2024-02-23T00:47:28.208293+0000","last_became_peered":"2024-02-23T00:47:28.208293+0000","last_unstale":"2024-02-23T00:51:00.248549+0000","last_undegraded":"2024-02-23T00:51:00.248549+0000","last_fullsized":"2024-02-23T00:51:00.248549+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:27.403305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:27.403305+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":500,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.240168+0000","last_change":"2024-02-23T00:49:46.612538+0000","last_active":"2024-02-23T00:51:00.240168+0000","last_peered":"2024-02-23T00:51:00.240168+0000","last_clean":"2024-02-23T00:51:00.240168+0000","last_became_active":"2024-02-23T00:39:18.796431+0000","last_became_peered":"2024-02-23T00:39:18.796431+0000","last_unstale":"2024-02-23T00:51:00.240168+0000","last_undegraded":"2024-02-23T00:51:00.240168+0000","last_fullsized":"2024-02-23T00:51:00.240168+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:46.612461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.612461+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":445,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.241479+0000","last_change":"2024-02-23T00:50:26.630228+0000","last_active":"2024-02-23T00:51:00.241479+0000","last_peered":"2024-02-23T00:51:00.241479+0000","last_clean":"2024-02-23T00:51:00.241479+0000","last_became_active":"2024-02-23T00:42:58.832885+0000","last_became_peered":"2024-02-23T00:42:58.832885+0000","last_unstale":"2024-02-23T00:51:00.241479+0000","last_undegraded":"2024-02-23T00:51:00.241479+0000","last_fullsized":"2024-02-23T00:51:00.241479+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:26.630088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:26.630088+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"457'17","reported_seq":534,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.237391+0000","last_change":"2024-02-23T00:50:15.590233+0000","last_active":"2024-02-23T00:51:00.237391+0000","last_peered":"2024-02-23T00:51:00.237391+0000","last_clean":"2024-02-23T00:51:00.237391+0000","last_became_active":"2024-02-23T00:39:16.782762+0000","last_became_peered":"2024-02-23T00:39:16.782762+0000","last_unstale":"2024-02-23T00:51:00.237391+0000","last_undegraded":"2024-02-23T00:51:00.237391+0000","last_fullsized":"2024-02-23T00:51:00.237391+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'17","last_scrub_stamp":"2024-02-23T00:50:15.590096+0000","last_deep_scrub":"82'15","last_deep_scrub_stamp":"2024-02-23T00:41:50.975683+0000","last_clean_scrub_stamp":"2024-02-23T00:50:15.590096+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":21,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":503,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.248663+0000","last_change":"2024-02-23T00:49:02.436312+0000","last_active":"2024-02-23T00:51:00.248663+0000","last_peered":"2024-02-23T00:51:00.248663+0000","last_clean":"2024-02-23T00:51:00.248663+0000","last_became_active":"2024-02-23T00:39:14.783469+0000","last_became_peered":"2024-02-23T00:39:14.783469+0000","last_unstale":"2024-02-23T00:51:00.248663+0000","last_undegraded":"2024-02-23T00:51:00.248663+0000","last_fullsized":"2024-02-23T00:51:00.248663+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:02.436243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:02.436243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":501,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.247779+0000","last_change":"2024-02-23T00:49:06.491877+0000","last_active":"2024-02-23T00:51:00.247779+0000","last_peered":"2024-02-23T00:51:00.247779+0000","last_clean":"2024-02-23T00:51:00.247779+0000","last_became_active":"2024-02-23T00:39:18.796952+0000","last_became_peered":"2024-02-23T00:39:18.796952+0000","last_unstale":"2024-02-23T00:51:00.247779+0000","last_undegraded":"2024-02-23T00:51:00.247779+0000","last_fullsized":"2024-02-23T00:51:00.247779+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:06.491793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:06.491793+0000","last_clean_scrub_stamp":"2024-02-23T00:49:06.491793+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":443,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.237206+0000","last_change":"2024-02-23T00:48:45.283304+0000","last_active":"2024-02-23T00:51:00.237206+0000","last_peered":"2024-02-23T00:51:00.237206+0000","last_clean":"2024-02-23T00:51:00.237206+0000","last_became_active":"2024-02-23T00:42:58.758147+0000","last_became_peered":"2024-02-23T00:42:58.758147+0000","last_unstale":"2024-02-23T00:51:00.237206+0000","last_undegraded":"2024-02-23T00:51:00.237206+0000","last_fullsized":"2024-02-23T00:51:00.237206+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:45.283230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:23.224663+0000","last_clean_scrub_stamp":"2024-02-23T00:48:45.283230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"457'21","reported_seq":542,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.236530+0000","last_change":"2024-02-23T00:49:32.434383+0000","last_active":"2024-02-23T00:51:00.236530+0000","last_peered":"2024-02-23T00:51:00.236530+0000","last_clean":"2024-02-23T00:51:00.236530+0000","last_became_active":"2024-02-23T00:39:16.784606+0000","last_became_peered":"2024-02-23T00:39:16.784606+0000","last_unstale":"2024-02-23T00:51:00.236530+0000","last_undegraded":"2024-02-23T00:51:00.236530+0000","last_fullsized":"2024-02-23T00:51:00.236530+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-23T00:49:32.434282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:32.434282+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":507,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.239351+0000","last_change":"2024-02-23T00:50:19.967076+0000","last_active":"2024-02-23T00:51:01.239351+0000","last_peered":"2024-02-23T00:51:01.239351+0000","last_clean":"2024-02-23T00:51:01.239351+0000","last_became_active":"2024-02-23T00:39:14.785362+0000","last_became_peered":"2024-02-23T00:39:14.785362+0000","last_unstale":"2024-02-23T00:51:01.239351+0000","last_undegraded":"2024-02-23T00:51:01.239351+0000","last_fullsized":"2024-02-23T00:51:01.239351+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:19.967009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:19.967009+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":492,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.240075+0000","last_change":"2024-02-23T00:49:37.537279+0000","last_active":"2024-02-23T00:51:00.240075+0000","last_peered":"2024-02-23T00:51:00.240075+0000","last_clean":"2024-02-23T00:51:00.240075+0000","last_became_active":"2024-02-23T00:39:18.796312+0000","last_became_peered":"2024-02-23T00:39:18.796312+0000","last_unstale":"2024-02-23T00:51:00.240075+0000","last_undegraded":"2024-02-23T00:51:00.240075+0000","last_fullsized":"2024-02-23T00:51:00.240075+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:37.537211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:47.226331+0000","last_clean_scrub_stamp":"2024-02-23T00:49:37.537211+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":446,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.241859+0000","last_change":"2024-02-23T00:50:28.479578+0000","last_active":"2024-02-23T00:51:00.241859+0000","last_peered":"2024-02-23T00:51:00.241859+0000","last_clean":"2024-02-23T00:51:00.241859+0000","last_became_active":"2024-02-23T00:42:58.838257+0000","last_became_peered":"2024-02-23T00:42:58.838257+0000","last_unstale":"2024-02-23T00:51:00.241859+0000","last_undegraded":"2024-02-23T00:51:00.241859+0000","last_fullsized":"2024-02-23T00:51:00.241859+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:28.479512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:50:28.479512+0000","last_clean_scrub_stamp":"2024-02-23T00:50:28.479512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"457'62","reported_seq":652,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-23T00:51:01.237371+0000","last_change":"2024-02-23T00:50:58.151569+0000","last_active":"2024-02-23T00:51:01.237371+0000","last_peered":"2024-02-23T00:51:01.237371+0000","last_clean":"2024-02-23T00:51:01.237371+0000","last_became_active":"2024-02-23T00:47:28.205150+0000","last_became_peered":"2024-02-23T00:47:28.205150+0000","last_unstale":"2024-02-23T00:51:01.237371+0000","last_undegraded":"2024-02-23T00:51:01.237371+0000","last_fullsized":"2024-02-23T00:51:01.237371+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'62","last_scrub_stamp":"2024-02-23T00:50:58.151494+0000","last_deep_scrub":"88'50","last_deep_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_clean_scrub_stamp":"2024-02-23T00:50:58.151494+0000","log_size":62,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":92,"num_read_kb":79,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":505,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.242429+0000","last_change":"2024-02-23T00:50:34.600693+0000","last_active":"2024-02-23T00:51:00.242429+0000","last_peered":"2024-02-23T00:51:00.242429+0000","last_clean":"2024-02-23T00:51:00.242429+0000","last_became_active":"2024-02-23T00:39:14.782000+0000","last_became_peered":"2024-02-23T00:39:14.782000+0000","last_unstale":"2024-02-23T00:51:00.242429+0000","last_undegraded":"2024-02-23T00:51:00.242429+0000","last_fullsized":"2024-02-23T00:51:00.242429+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:34.600592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_clean_scrub_stamp":"2024-02-23T00:50:34.600592+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":495,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.237314+0000","last_change":"2024-02-23T00:49:27.459981+0000","last_active":"2024-02-23T00:51:00.237314+0000","last_peered":"2024-02-23T00:51:00.237314+0000","last_clean":"2024-02-23T00:51:00.237314+0000","last_became_active":"2024-02-23T00:39:18.799723+0000","last_became_peered":"2024-02-23T00:39:18.799723+0000","last_unstale":"2024-02-23T00:51:00.237314+0000","last_undegraded":"2024-02-23T00:51:00.237314+0000","last_fullsized":"2024-02-23T00:51:00.237314+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:27.459893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:27.459893+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":445,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-23T00:51:00.241564+0000","last_change":"2024-02-23T00:49:44.601162+0000","last_active":"2024-02-23T00:51:00.241564+0000","last_peered":"2024-02-23T00:51:00.241564+0000","last_clean":"2024-02-23T00:51:00.241564+0000","last_became_active":"2024-02-23T00:42:58.824573+0000","last_became_peered":"2024-02-23T00:42:58.824573+0000","last_unstale":"2024-02-23T00:51:00.241564+0000","last_undegraded":"2024-02-23T00:51:00.241564+0000","last_fullsized":"2024-02-23T00:51:00.241564+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:44.601086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:44.601086+0000","last_clean_scrub_stamp":"2024-02-23T00:49:44.601086+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0],"acting":[5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":62,"num_pg":32,"stat_sum":{"num_bytes":4020435,"num_objects":192,"num_object_clones":11,"num_object_copies":576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":192,"num_whiteouts":0,"num_read":211,"num_read_kb":57,"num_write":777,"num_write_kb":4619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":853,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":21,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":14331904,"data_stored":12061457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2077,"ondisk_log_size":2077,"up":96,"acting":96,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":12,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":36,"acting":36,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":226,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":7},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1508,"num_read_kb":1233,"num_write":895,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":1486,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":955,"ondisk_log_size":955,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":697,"num_read_kb":692,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234129,"num_pgs":106,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":315908,"kb_used_data":6396,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93429244,"statfs":{"total":95995035648,"available":95671545856,"internally_reserved":0,"allocated":6549504,"data_stored":4798296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:50:32 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51500000000000001,"15min":0.53500000000000003},"min":{"1min":0.29499999999999998,"5min":0.27300000000000002,"15min":0.26900000000000002},"max":{"1min":0.64100000000000001,"5min":0.76400000000000001,"15min":4.9420000000000002},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.376,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.78400000000000003,"5min":1.151,"15min":4.9729999999999999},"last":0.36799999999999999}]},{"osd":1,"last update":"Fri Feb 23 00:50:32 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.52500000000000002,"15min":0.61699999999999999},"min":{"1min":0.30599999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.79700000000000004,"5min":1.25,"15min":10.786},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52300000000000002,"15min":0.62},"min":{"1min":0.39100000000000001,"5min":0.317,"15min":0.28100000000000003},"max":{"1min":0.78400000000000003,"5min":1.107,"15min":11.885999999999999},"last":0.39200000000000002}]},{"osd":2,"last update":"Fri Feb 23 00:50:32 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.52700000000000002,"15min":0.55400000000000005},"min":{"1min":0.377,"5min":0.36099999999999999,"15min":0.28999999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":5.0789999999999997},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.497,"5min":0.53000000000000003,"15min":0.54600000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.316},"max":{"1min":0.66900000000000004,"5min":1.1180000000000001,"15min":5.1280000000000001},"last":0.40600000000000003}]},{"osd":3,"last update":"Fri Feb 23 00:50:32 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52300000000000002,"15min":0.61899999999999999},"min":{"1min":0.35399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.75800000000000001,"5min":1.095,"15min":15.712999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.55800000000000005,"15min":0.61499999999999999},"min":{"1min":0.40200000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.76800000000000002,"5min":1.1659999999999999,"15min":9.4090000000000007},"last":0.35799999999999998}]},{"osd":4,"last update":"Fri Feb 23 00:50:32 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.439,"15min":0.439},"min":{"1min":0.31,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.63600000000000001,"5min":1.6819999999999999,"15min":2.0350000000000001},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.46100000000000002,"15min":0.48799999999999999},"min":{"1min":0.32000000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.63600000000000001,"5min":0.86699999999999999,"15min":11.236000000000001},"last":0.376}]},{"osd":5,"last update":"Fri Feb 23 00:50:32 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48099999999999998,"15min":0.49199999999999999},"min":{"1min":0.32300000000000001,"5min":0.27700000000000002,"15min":0.27000000000000002},"max":{"1min":0.72299999999999998,"5min":0.73199999999999998,"15min":5.4000000000000004},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.50900000000000001,"15min":0.497},"min":{"1min":0.35199999999999998,"5min":0.29099999999999998,"15min":0.26700000000000002},"max":{"1min":0.69499999999999995,"5min":1.728,"15min":1.728},"last":0.41499999999999998}]},{"osd":6,"last update":"Fri Feb 23 00:50:32 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51500000000000001,"15min":0.51000000000000001},"min":{"1min":0.36099999999999999,"5min":0.32200000000000001,"15min":0.317},"max":{"1min":0.67800000000000005,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.435},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52000000000000002,"15min":0.54800000000000004},"min":{"1min":0.34000000000000002,"5min":0.32700000000000001,"15min":0.32400000000000001},"max":{"1min":0.70499999999999996,"5min":1.2370000000000001,"15min":13.395},"last":0.48999999999999999}]}]},{"osd":6,"up_from":47,"seq":201863463064,"num_pgs":84,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308188,"kb_used_data":2372,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93436964,"statfs":{"total":95995035648,"available":95679451136,"internally_reserved":0,"allocated":2428928,"data_stored":783290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.49199999999999999,"15min":0.68500000000000005},"min":{"1min":0.26000000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.68799999999999994,"5min":0.86099999999999999,"15min":30.641999999999999},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.52600000000000002,"15min":0.69799999999999995},"min":{"1min":0.27000000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.63600000000000001,"5min":3.7360000000000002,"15min":29.292000000000002},"last":0.23899999999999999}]},{"osd":1,"last update":"Fri Feb 23 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51500000000000001,"15min":0.68500000000000005},"min":{"1min":0.38600000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.72699999999999998,"5min":0.77200000000000002,"15min":28.186},"last":0.308},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.51400000000000001,"15min":0.86399999999999999},"min":{"1min":0.34499999999999997,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.083,"5min":1.083,"15min":33.104999999999997},"last":0.39400000000000002}]},{"osd":2,"last update":"Fri Feb 23 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52400000000000002,"15min":0.71799999999999997},"min":{"1min":0.35299999999999998,"5min":0.29399999999999998,"15min":0.27400000000000002},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":19.960000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.53200000000000003,"15min":0.64700000000000002},"min":{"1min":0.35199999999999998,"5min":0.316,"15min":0.29899999999999999},"max":{"1min":0.65100000000000002,"5min":1.665,"15min":8.5860000000000003},"last":0.40400000000000003}]},{"osd":3,"last update":"Fri Feb 23 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53900000000000003,"15min":0.627},"min":{"1min":0.32800000000000001,"5min":0.29199999999999998,"15min":0.28599999999999998},"max":{"1min":0.71799999999999997,"5min":0.80000000000000004,"15min":19.562000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55800000000000005,"15min":0.67000000000000004},"min":{"1min":0.35699999999999998,"5min":0.33500000000000002,"15min":0.33000000000000002},"max":{"1min":0.67200000000000004,"5min":2.6259999999999999,"15min":14.199},"last":0.42399999999999999}]},{"osd":4,"last update":"Fri Feb 23 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.45100000000000001,"15min":0.47699999999999998},"min":{"1min":0.27900000000000003,"5min":0.245,"15min":0.218},"max":{"1min":0.61599999999999999,"5min":1.6220000000000001,"15min":11.109999999999999},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.45700000000000002,"15min":0.54400000000000004},"min":{"1min":0.27300000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.69299999999999995,"5min":0.83099999999999996,"15min":16.734000000000002},"last":0.34300000000000003}]},{"osd":5,"last update":"Fri Feb 23 00:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.48599999999999999,"15min":0.50800000000000001},"min":{"1min":0.308,"5min":0.308,"15min":0.249},"max":{"1min":0.64500000000000002,"5min":1.784,"15min":14.154999999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.57599999999999996,"15min":0.54800000000000004},"min":{"1min":0.318,"5min":0.27900000000000003,"15min":0.26400000000000001},"max":{"1min":0.73899999999999999,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.376}]},{"osd":7,"last update":"Fri Feb 23 00:50:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.51400000000000001,"15min":0.52900000000000003},"min":{"1min":0.36499999999999999,"5min":0.29399999999999998,"15min":0.26800000000000002},"max":{"1min":0.67600000000000005,"5min":1.726,"15min":8.548},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54600000000000004,"15min":0.60399999999999998},"min":{"1min":0.36399999999999999,"5min":0.34200000000000003,"15min":0.309},"max":{"1min":0.749,"5min":0.88500000000000001,"15min":15.618},"last":0.433}]}]},{"osd":1,"up_from":13,"seq":55834575026,"num_pgs":85,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":312112,"kb_used_data":6296,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93433040,"statfs":{"total":95995035648,"available":95675432960,"internally_reserved":0,"allocated":6447104,"data_stored":4793836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.39000000000000001,"15min":0.68400000000000005},"min":{"1min":0.23799999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.56999999999999995,"5min":6.202,"15min":78.361000000000004},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.379,"5min":0.36899999999999999,"15min":0.44400000000000001},"min":{"1min":0.26900000000000002,"5min":0.216,"15min":0.191},"max":{"1min":0.64300000000000002,"5min":0.69999999999999996,"15min":5.8849999999999998},"last":0.24199999999999999}]},{"osd":2,"last update":"Fri Feb 23 00:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.41699999999999998,"15min":0.66700000000000004},"min":{"1min":0.22700000000000001,"5min":0.20499999999999999,"15min":0.188},"max":{"1min":7.5789999999999997,"5min":7.5789999999999997,"15min":48.606000000000002},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.40500000000000003,"15min":0.57699999999999996},"min":{"1min":0.28000000000000003,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.63,"5min":1.3460000000000001,"15min":23.911000000000001},"last":0.32000000000000001}]},{"osd":3,"last update":"Fri Feb 23 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.40500000000000003,"15min":0.59499999999999997},"min":{"1min":0.27800000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":35.244999999999997},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.437,"5min":0.47699999999999998,"15min":0.68700000000000006},"min":{"1min":0.28899999999999998,"5min":0.26900000000000002,"15min":0.248},"max":{"1min":0.73999999999999999,"5min":7.641,"15min":56.853999999999999},"last":0.33100000000000002}]},{"osd":4,"last update":"Fri Feb 23 00:50:05 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.627,"15min":0.82399999999999995},"min":{"1min":0.38,"5min":0.32700000000000001,"15min":0.29299999999999998},"max":{"1min":7.6829999999999998,"5min":7.6829999999999998,"15min":37.646999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.64100000000000001,"15min":0.70899999999999996},"min":{"1min":0.38200000000000001,"5min":0.35899999999999999,"15min":0.31900000000000001},"max":{"1min":6.0519999999999996,"5min":6.0519999999999996,"15min":9.6859999999999999},"last":0.54100000000000004}]},{"osd":5,"last update":"Fri Feb 23 00:50:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.65300000000000002,"15min":0.84099999999999997},"min":{"1min":0.40600000000000003,"5min":0.309,"15min":0.309},"max":{"1min":7.742,"5min":7.742,"15min":48.792000000000002},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.61899999999999999,"15min":0.71899999999999997},"min":{"1min":0.45200000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":11.872999999999999},"last":0.59899999999999998}]},{"osd":6,"last update":"Fri Feb 23 00:50:05 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.63,"15min":0.82499999999999996},"min":{"1min":0.37,"5min":0.318,"15min":0.318},"max":{"1min":7.7130000000000001,"5min":7.7130000000000001,"15min":40.741},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.58899999999999997,"15min":0.67400000000000004},"min":{"1min":0.442,"5min":0.38400000000000001,"15min":0.34000000000000002},"max":{"1min":0.82899999999999996,"5min":0.872,"15min":6.0069999999999997},"last":0.63300000000000001}]},{"osd":7,"last update":"Fri Feb 23 00:50:31 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.60699999999999998,"15min":0.71399999999999997},"min":{"1min":0.45300000000000001,"5min":0.39300000000000002,"15min":0.36699999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":20.225000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.66000000000000003,"15min":0.74099999999999999},"min":{"1min":0.46200000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":6.0949999999999998,"5min":6.0949999999999998,"15min":20.254999999999999},"last":0.58699999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738552,"num_pgs":85,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":307788,"kb_used_data":2372,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93437364,"statfs":{"total":95995035648,"available":95679860736,"internally_reserved":0,"allocated":2428928,"data_stored":791778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 23 00:50:52 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.35199999999999998,"15min":0.60299999999999998},"min":{"1min":0.224,"5min":0.20999999999999999,"15min":0.19500000000000001},"max":{"1min":0.68200000000000005,"5min":0.81399999999999995,"15min":28.856999999999999},"last":0.246},{"interface":"front","average":{"1min":0.373,"5min":0.39500000000000002,"15min":0.629},"min":{"1min":0.249,"5min":0.23300000000000001,"15min":0.20799999999999999},"max":{"1min":0.59099999999999997,"5min":2.3650000000000002,"15min":20.577000000000002},"last":0.22800000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:50:15 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.40200000000000002,"15min":0.61099999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.20499999999999999},"max":{"1min":2.3889999999999998,"5min":2.3889999999999998,"15min":13.863},"last":0.317},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.34999999999999998,"15min":0.54100000000000004},"min":{"1min":0.245,"5min":0.219,"15min":0.19800000000000001},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":10.455},"last":0.26500000000000001}]},{"osd":3,"last update":"Fri Feb 23 00:50:46 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.377,"15min":0.77900000000000003},"min":{"1min":0.246,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.64300000000000002,"5min":0.73099999999999998,"15min":35.566000000000003},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.438,"15min":0.76900000000000002},"min":{"1min":0.27700000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.66700000000000004,"5min":2.4049999999999998,"15min":20.702999999999999},"last":0.27800000000000002}]},{"osd":4,"last update":"Fri Feb 23 00:50:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56200000000000006,"15min":0.76300000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.88300000000000001,"5min":0.94699999999999995,"15min":10.584},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.75600000000000001,"15min":0.79500000000000004},"min":{"1min":0.39500000000000002,"5min":0.38100000000000001,"15min":0.35099999999999998},"max":{"1min":2.5019999999999998,"5min":13.198,"15min":13.198},"last":0.59999999999999998}]},{"osd":5,"last update":"Fri Feb 23 00:50:35 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54500000000000004,"15min":0.72099999999999997},"min":{"1min":0.34799999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81799999999999995,"5min":0.90800000000000003,"15min":10.397},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61299999999999999,"15min":0.78800000000000003},"min":{"1min":0.40999999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":17.297000000000001},"last":0.58299999999999996}]},{"osd":6,"last update":"Fri Feb 23 00:50:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59899999999999998,"15min":0.78700000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.85499999999999998,"5min":0.94399999999999995,"15min":17.373999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.77300000000000002,"15min":0.77000000000000002},"min":{"1min":0.437,"5min":0.38400000000000001,"15min":0.36799999999999999},"max":{"1min":2.556,"5min":13.973000000000001,"15min":13.973000000000001},"last":0.57399999999999995}]},{"osd":7,"last update":"Fri Feb 23 00:50:27 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59899999999999998,"15min":0.78600000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.95199999999999996,"5min":0.96099999999999997,"15min":17.399999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.64500000000000002,"15min":0.78700000000000003},"min":{"1min":0.433,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93400000000000005,"5min":2.5289999999999999,"15min":16.427},"last":0.46100000000000002}]}]},{"osd":2,"up_from":0,"seq":77309411501,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":26,"seq":111669149862,"num_pgs":80,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":311932,"kb_used_data":2420,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93433220,"statfs":{"total":95995035648,"available":95675617280,"internally_reserved":0,"allocated":2478080,"data_stored":796015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:50:45 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.441,"15min":0.53800000000000003},"min":{"1min":0.246,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":8.3300000000000001,"5min":8.3300000000000001,"15min":12.212999999999999},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.499,"15min":0.61099999999999999},"min":{"1min":0.23100000000000001,"5min":0.22800000000000001,"15min":0.21199999999999999},"max":{"1min":2.9740000000000002,"5min":5.9800000000000004,"15min":11.714},"last":0.38400000000000001}]},{"osd":1,"last update":"Fri Feb 23 00:50:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55700000000000005,"15min":0.63200000000000001},"min":{"1min":0.27200000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":3.056,"5min":6.851,"15min":11.742000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.58099999999999996,"15min":0.60199999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.19900000000000001},"max":{"1min":2.7709999999999999,"5min":8.2270000000000003,"15min":8.2270000000000003},"last":0.34100000000000003}]},{"osd":2,"last update":"Fri Feb 23 00:50:45 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.625,"15min":0.68999999999999995},"min":{"1min":0.29199999999999998,"5min":0.255,"15min":0.255},"max":{"1min":3.242,"5min":8.0269999999999992,"15min":13.359999999999999},"last":0.442},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.52600000000000002,"15min":0.66600000000000004},"min":{"1min":0.27600000000000002,"5min":0.27500000000000002,"15min":0.26100000000000001},"max":{"1min":8.5820000000000007,"5min":8.5820000000000007,"15min":21.122},"last":0.372}]},{"osd":4,"last update":"Fri Feb 23 00:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.67500000000000004,"15min":0.749},"min":{"1min":0.38400000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.276,"5min":11.558,"15min":11.644},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.78800000000000003,"15min":0.74399999999999999},"min":{"1min":0.37,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.70799999999999996,"5min":12.430999999999999,"15min":12.430999999999999},"last":0.52300000000000002}]},{"osd":5,"last update":"Fri Feb 23 00:50:35 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":0.69899999999999995,"15min":0.68899999999999995},"min":{"1min":0.375,"5min":0.37,"15min":0.35099999999999998},"max":{"1min":10.231999999999999,"5min":10.231999999999999,"15min":10.231999999999999},"last":0.497},{"interface":"front","average":{"1min":0.746,"5min":0.76200000000000001,"15min":0.80100000000000005},"min":{"1min":0.38300000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":3.1829999999999998,"5min":11.648,"15min":11.648},"last":0.63400000000000001}]},{"osd":6,"last update":"Fri Feb 23 00:50:53 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76300000000000001,"15min":0.75600000000000001},"min":{"1min":0.42399999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.9510000000000001,"5min":6.9420000000000002,"15min":6.9420000000000002},"last":0.58499999999999996},{"interface":"front","average":{"1min":1.2050000000000001,"5min":0.751,"15min":0.72299999999999998},"min":{"1min":0.39200000000000002,"5min":0.376,"15min":0.376},"max":{"1min":10.965999999999999,"5min":10.965999999999999,"15min":10.965999999999999},"last":0.53400000000000003}]},{"osd":7,"last update":"Fri Feb 23 00:50:29 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.77500000000000002,"15min":0.82399999999999995},"min":{"1min":0.44500000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.3160000000000001,"5min":11.676,"15min":11.676},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.79300000000000004,"15min":0.80200000000000005},"min":{"1min":0.47699999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80900000000000005,"5min":7.9800000000000004,"15min":7.9800000000000004},"last":0.62}]}]},{"osd":4,"up_from":32,"seq":137438953634,"num_pgs":73,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":311692,"kb_used_data":6276,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93433460,"statfs":{"total":95995035648,"available":95675863040,"internally_reserved":0,"allocated":6426624,"data_stored":4791157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:50:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.50600000000000001,"15min":0.53200000000000003},"min":{"1min":0.34699999999999998,"5min":0.223,"15min":0.223},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":3.5219999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.52500000000000002,"15min":0.59099999999999997},"min":{"1min":0.33200000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.8,"5min":1.8,"15min":8.6140000000000008},"last":0.68600000000000005}]},{"osd":1,"last update":"Fri Feb 23 00:50:10 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51400000000000001,"15min":0.58599999999999997},"min":{"1min":0.32000000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65200000000000002,"5min":2.6600000000000001,"15min":9.6110000000000007},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.52600000000000002,"15min":0.58699999999999997},"min":{"1min":0.34200000000000003,"5min":0.255,"15min":0.255},"max":{"1min":2.6920000000000002,"5min":2.6920000000000002,"15min":6.6950000000000003},"last":0.61399999999999999}]},{"osd":2,"last update":"Fri Feb 23 00:50:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52700000000000002,"15min":0.57599999999999996},"min":{"1min":0.307,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.89900000000000002,"5min":1.0960000000000001,"15min":3.3069999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.53800000000000003,"15min":0.57399999999999995},"min":{"1min":0.36299999999999999,"5min":0.33600000000000002,"15min":0.32500000000000001},"max":{"1min":0.73899999999999999,"5min":1.0189999999999999,"15min":5.8440000000000003},"last":0.63700000000000001}]},{"osd":3,"last update":"Fri Feb 23 00:50:10 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.67500000000000004,"15min":0.63600000000000001},"min":{"1min":0.38500000000000001,"5min":0.316,"15min":0.316},"max":{"1min":9.1280000000000001,"5min":9.1280000000000001,"15min":9.1280000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.58599999999999997,"15min":0.60899999999999999},"min":{"1min":0.36599999999999999,"5min":0.33100000000000002,"15min":0.29699999999999999},"max":{"1min":2.1699999999999999,"5min":4.0220000000000002,"15min":4.1909999999999998},"last":0.73299999999999998}]},{"osd":5,"last update":"Fri Feb 23 00:50:35 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.45600000000000002,"15min":0.41599999999999998},"min":{"1min":0.26700000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.51500000000000001,"15min":0.47199999999999998},"min":{"1min":0.35399999999999998,"5min":0.29499999999999998,"15min":0.28899999999999998},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.52900000000000003}]},{"osd":6,"last update":"Fri Feb 23 00:50:05 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.443,"15min":0.42899999999999999},"min":{"1min":0.29099999999999998,"5min":0.28399999999999997,"15min":0.253},"max":{"1min":0.73999999999999999,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.496},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.46800000000000003,"15min":0.438},"min":{"1min":0.317,"5min":0.27000000000000002,"15min":0.215},"max":{"1min":2.2120000000000002,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":0.50600000000000001}]},{"osd":7,"last update":"Fri Feb 23 00:50:31 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.497,"15min":0.48599999999999999},"min":{"1min":0.30599999999999999,"5min":0.26700000000000002,"15min":0.254},"max":{"1min":2.294,"5min":2.294,"15min":2.294},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.379,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":0.56399999999999995}]}]},{"osd":5,"up_from":39,"seq":167503724701,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":307740,"kb_used_data":2324,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93437412,"statfs":{"total":95995035648,"available":95679909888,"internally_reserved":0,"allocated":2379776,"data_stored":791776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:50:27 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.54500000000000004,"15min":0.61399999999999999},"min":{"1min":0.35799999999999998,"5min":0.253,"15min":0.246},"max":{"1min":5.2220000000000004,"5min":5.2220000000000004,"15min":23.881},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.57199999999999995,"15min":0.59399999999999997},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.27000000000000002},"max":{"1min":3.9249999999999998,"5min":3.9249999999999998,"15min":5.4649999999999999},"last":0.57099999999999995}]},{"osd":1,"last update":"Fri Feb 23 00:50:27 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.55200000000000005,"15min":0.65700000000000003},"min":{"1min":0.41299999999999998,"5min":0.254,"15min":0.254},"max":{"1min":3.9460000000000002,"5min":3.9460000000000002,"15min":18.175999999999998},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.56699999999999995,"15min":0.63100000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":3.976,"5min":3.976,"15min":9.0950000000000006},"last":0.54000000000000004}]},{"osd":2,"last update":"Fri Feb 23 00:50:27 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.56699999999999995,"15min":0.71099999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":4.5899999999999999,"5min":4.5899999999999999,"15min":22.282},"last":0.751},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.59099999999999997,"15min":0.72699999999999998},"min":{"1min":0.36699999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":3.8540000000000001,"5min":3.8540000000000001,"15min":23.925000000000001},"last":0.63}]},{"osd":3,"last update":"Fri Feb 23 00:50:27 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54800000000000004,"15min":0.61899999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.1060000000000001,"5min":1.23,"15min":6.5730000000000004},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.60399999999999998,"15min":0.65500000000000003},"min":{"1min":0.41799999999999998,"5min":0.378,"15min":0.33500000000000002},"max":{"1min":3.9980000000000002,"5min":3.9980000000000002,"15min":8.1760000000000002},"last":0.52300000000000002}]},{"osd":4,"last update":"Fri Feb 23 00:50:27 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.48399999999999999,"15min":0.43099999999999999},"min":{"1min":0.29999999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.68100000000000005,"5min":4.593,"15min":4.593},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.47899999999999998,"15min":0.46300000000000002},"min":{"1min":0.27900000000000003,"5min":0.27700000000000002,"15min":0.27300000000000002},"max":{"1min":0.68400000000000005,"5min":1.5860000000000001,"15min":3.879},"last":0.379}]},{"osd":6,"last update":"Fri Feb 23 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51300000000000001,"15min":0.47599999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.27900000000000003},"max":{"1min":0.71099999999999997,"5min":4.6740000000000004,"15min":4.6740000000000004},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.49099999999999999,"15min":0.47699999999999998},"min":{"1min":0.35999999999999999,"5min":0.28000000000000003,"15min":0.27100000000000002},"max":{"1min":0.73199999999999998,"5min":1.159,"15min":1.159},"last":0.41399999999999998}]},{"osd":7,"last update":"Fri Feb 23 00:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54100000000000004,"15min":0.54200000000000004},"min":{"1min":0.34200000000000003,"5min":0.32500000000000001,"15min":0.30399999999999999},"max":{"1min":0.79200000000000004,"5min":1.302,"15min":7.0300000000000002},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.52800000000000002,"15min":0.53800000000000003},"min":{"1min":0.36699999999999999,"5min":0.314,"15min":0.29099999999999998},"max":{"1min":0.73599999999999999,"5min":2.016,"15min":9.0579999999999998},"last":0.51100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":303104,"data_stored":6419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4317184,"data_stored":4007883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":68,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":344064,"data_stored":11278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4308992,"data_stored":4006776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":6143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":331776,"data_stored":8750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4444160,"data_stored":4014140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":226,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":226,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":226,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":226,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":226,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":226,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":226,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-23T00:51:07.214 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph pg dump --format=json 2024-02-23T00:51:07.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:07.242+0000 7f1155ffb700 1 -- 172.21.15.47:0/3464190293 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v538) v1 ==== 101+0+0 (secure 0 0 0) 0x7f114c0043d0 con 0x7f11580f7400 2024-02-23T00:51:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:07.246+0000 7f115f156700 10 librados: watch_flush enter 2024-02-23T00:51:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:07.246+0000 7f115f156700 10 librados: watch_flush exit 2024-02-23T00:51:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:07.246+0000 7f115f156700 20 client.41627.objecter shutdown clearing up homeless session... 2024-02-23T00:51:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:07.246+0000 7f115f156700 10 client.41627.objecter successfully canceled tick 2024-02-23T00:51:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:07.246+0000 7f115f156700 1 -- 172.21.15.47:0/3464190293 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1148062cc0 msgr2=0x7f1148065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:07.246+0000 7f115f156700 1 --2- 172.21.15.47:0/3464190293 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1148062cc0 0x7f1148065180 secure :-1 s=READY pgs=3113 cs=0 l=1 rev1=1 rx=0x7f11581099d0 tx=0x7f1144009cb0).stop 2024-02-23T00:51:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:07.246+0000 7f115f156700 1 -- 172.21.15.47:0/3464190293 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11580f7400 msgr2=0x7f115810e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:07.246+0000 7f115f156700 1 --2- 172.21.15.47:0/3464190293 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11580f7400 0x7f115810e090 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 rx=0x7f114c009f20 tx=0x7f114c009cb0).stop 2024-02-23T00:51:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:07.246+0000 7f115f156700 1 -- 172.21.15.47:0/3464190293 shutdown_connections 2024-02-23T00:51:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:07.246+0000 7f115f156700 1 --2- 172.21.15.47:0/3464190293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11580fd510 0x7f1158108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:07.246+0000 7f115f156700 1 --2- 172.21.15.47:0/3464190293 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1148062cc0 0x7f1148065180 unknown :-1 s=CLOSED pgs=3113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:07.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:07.246+0000 7f115f156700 1 --2- 172.21.15.47:0/3464190293 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11580f7400 0x7f115810e090 unknown :-1 s=CLOSED pgs=2726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:07.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:07.246+0000 7f115f156700 1 --2- 172.21.15.47:0/3464190293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11580fcbb0 0x7f115810e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:07.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:07.246+0000 7f115f156700 1 -- 172.21.15.47:0/3464190293 >> 172.21.15.47:0/3464190293 conn(0x7f11580f2820 msgr2=0x7f1158060eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:07.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:07.246+0000 7f115f156700 1 -- 172.21.15.47:0/3464190293 shutdown_connections 2024-02-23T00:51:07.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:07.246+0000 7f115f156700 1 -- 172.21.15.47:0/3464190293 wait complete. 2024-02-23T00:51:07.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:07.246+0000 7f115f156700 1 librados: shutdown 2024-02-23T00:51:07.250 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is unset 2024-02-23T00:51:07.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:06.517469+0000 mgr.y (mgr.14517) 493 : audit [DBG] from='client.41621 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:51:07.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:06.819792+0000 mon.c (mon.2) 323 : audit [INF] from='client.? 172.21.15.47:0/3464190293' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:51:07.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:06.820094+0000 mon.a (mon.0) 2916 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:51:07.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:07.236719+0000 mon.a (mon.0) 2917 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:51:07.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:07.241399+0000 mon.a (mon.0) 2918 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-88","var": "hit_set_period","val": "600"}]': finished 2024-02-23T00:51:07.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:07.241484+0000 mon.a (mon.0) 2919 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:51:07.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:07.241539+0000 mon.a (mon.0) 2920 : cluster [DBG] osdmap e538: 8 total, 8 up, 7 in 2024-02-23T00:51:07.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:07.246622+0000 mon.a (mon.0) 2921 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-88","var": "target_max_objects","val": "250"}]: dispatch 2024-02-23T00:51:07.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:06.517469+0000 mgr.y (mgr.14517) 493 : audit [DBG] from='client.41621 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:51:07.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:06.819792+0000 mon.c (mon.2) 323 : audit [INF] from='client.? 172.21.15.47:0/3464190293' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:51:07.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:06.820094+0000 mon.a (mon.0) 2916 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:51:07.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:07.236719+0000 mon.a (mon.0) 2917 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:51:07.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:07.241399+0000 mon.a (mon.0) 2918 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-88","var": "hit_set_period","val": "600"}]': finished 2024-02-23T00:51:07.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:07.241484+0000 mon.a (mon.0) 2919 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:51:07.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:07.241539+0000 mon.a (mon.0) 2920 : cluster [DBG] osdmap e538: 8 total, 8 up, 7 in 2024-02-23T00:51:07.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:07.246622+0000 mon.a (mon.0) 2921 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-88","var": "target_max_objects","val": "250"}]: dispatch 2024-02-23T00:51:07.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:06.517469+0000 mgr.y (mgr.14517) 493 : audit [DBG] from='client.41621 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:51:07.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:06.819792+0000 mon.c (mon.2) 323 : audit [INF] from='client.? 172.21.15.47:0/3464190293' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:51:07.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:06.820094+0000 mon.a (mon.0) 2916 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:51:07.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:07.236719+0000 mon.a (mon.0) 2917 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:51:07.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:07.241399+0000 mon.a (mon.0) 2918 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-88","var": "hit_set_period","val": "600"}]': finished 2024-02-23T00:51:07.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:07.241484+0000 mon.a (mon.0) 2919 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:51:07.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:07.241539+0000 mon.a (mon.0) 2920 : cluster [DBG] osdmap e538: 8 total, 8 up, 7 in 2024-02-23T00:51:07.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:07.246622+0000 mon.a (mon.0) 2921 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-88","var": "target_max_objects","val": "250"}]: dispatch 2024-02-23T00:51:07.768 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:51:07.771 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:07.771 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:51:07.872 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:51:07.901 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:08.249 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:51:08.252 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:08.252 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:51:08.353 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:51:08.380 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:08.518 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:07.231204+0000 osd.7 (osd.7) 265 : cluster [DBG] 62.1e scrub starts 2024-02-23T00:51:08.519 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:07.232716+0000 osd.7 (osd.7) 266 : cluster [DBG] 62.1e scrub ok 2024-02-23T00:51:08.519 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:07.337958+0000 mgr.y (mgr.14517) 494 : cluster [DBG] pgmap v811: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:08.519 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:07.563244+0000 osd.4 (osd.4) 231 : cluster [DBG] 62.16 scrub starts 2024-02-23T00:51:08.519 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:07.565241+0000 osd.4 (osd.4) 232 : cluster [DBG] 62.16 scrub ok 2024-02-23T00:51:08.519 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:08.242343+0000 mon.a (mon.0) 2922 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-88","var": "target_max_objects","val": "250"}]': finished 2024-02-23T00:51:08.519 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:08.242393+0000 mon.a (mon.0) 2923 : cluster [DBG] osdmap e539: 8 total, 8 up, 7 in 2024-02-23T00:51:08.519 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:07.231204+0000 osd.7 (osd.7) 265 : cluster [DBG] 62.1e scrub starts 2024-02-23T00:51:08.519 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:07.232716+0000 osd.7 (osd.7) 266 : cluster [DBG] 62.1e scrub ok 2024-02-23T00:51:08.519 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:07.337958+0000 mgr.y (mgr.14517) 494 : cluster [DBG] pgmap v811: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:08.519 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:07.563244+0000 osd.4 (osd.4) 231 : cluster [DBG] 62.16 scrub starts 2024-02-23T00:51:08.520 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:07.565241+0000 osd.4 (osd.4) 232 : cluster [DBG] 62.16 scrub ok 2024-02-23T00:51:08.520 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:08.242343+0000 mon.a (mon.0) 2922 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-88","var": "target_max_objects","val": "250"}]': finished 2024-02-23T00:51:08.520 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:08.242393+0000 mon.a (mon.0) 2923 : cluster [DBG] osdmap e539: 8 total, 8 up, 7 in 2024-02-23T00:51:08.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:07.231204+0000 osd.7 (osd.7) 265 : cluster [DBG] 62.1e scrub starts 2024-02-23T00:51:08.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:07.232716+0000 osd.7 (osd.7) 266 : cluster [DBG] 62.1e scrub ok 2024-02-23T00:51:08.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:07.337958+0000 mgr.y (mgr.14517) 494 : cluster [DBG] pgmap v811: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:08.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:07.563244+0000 osd.4 (osd.4) 231 : cluster [DBG] 62.16 scrub starts 2024-02-23T00:51:08.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:07.565241+0000 osd.4 (osd.4) 232 : cluster [DBG] 62.16 scrub ok 2024-02-23T00:51:08.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:08.242343+0000 mon.a (mon.0) 2922 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-88","var": "target_max_objects","val": "250"}]': finished 2024-02-23T00:51:08.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:08.242393+0000 mon.a (mon.0) 2923 : cluster [DBG] osdmap e539: 8 total, 8 up, 7 in 2024-02-23T00:51:09.525 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:08.196915+0000 osd.7 (osd.7) 267 : cluster [DBG] 2.1d scrub starts 2024-02-23T00:51:09.525 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:08.199745+0000 osd.7 (osd.7) 268 : cluster [DBG] 2.1d scrub ok 2024-02-23T00:51:09.525 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:08.234827+0000 osd.6 (osd.6) 209 : cluster [DBG] 2.1a scrub starts 2024-02-23T00:51:09.525 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:08.235731+0000 osd.6 (osd.6) 210 : cluster [DBG] 2.1a scrub ok 2024-02-23T00:51:09.526 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:08.269775+0000 mon.a (mon.0) 2924 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:51:09.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:08.196915+0000 osd.7 (osd.7) 267 : cluster [DBG] 2.1d scrub starts 2024-02-23T00:51:09.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:08.199745+0000 osd.7 (osd.7) 268 : cluster [DBG] 2.1d scrub ok 2024-02-23T00:51:09.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:08.234827+0000 osd.6 (osd.6) 209 : cluster [DBG] 2.1a scrub starts 2024-02-23T00:51:09.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:08.235731+0000 osd.6 (osd.6) 210 : cluster [DBG] 2.1a scrub ok 2024-02-23T00:51:09.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:08.269775+0000 mon.a (mon.0) 2924 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:51:09.531 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:51:09.534 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:09.535 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:51:09.636 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:51:09.668 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:09.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:08.196915+0000 osd.7 (osd.7) 267 : cluster [DBG] 2.1d scrub starts 2024-02-23T00:51:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:08.199745+0000 osd.7 (osd.7) 268 : cluster [DBG] 2.1d scrub ok 2024-02-23T00:51:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:08.234827+0000 osd.6 (osd.6) 209 : cluster [DBG] 2.1a scrub starts 2024-02-23T00:51:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:08.235731+0000 osd.6 (osd.6) 210 : cluster [DBG] 2.1a scrub ok 2024-02-23T00:51:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:08.269775+0000 mon.a (mon.0) 2924 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:51:10.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.072+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3363159228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f380fcbb0 msgr2=0x7f7f380fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:10.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.072+0000 7f7f3ce23700 1 --2- 172.21.15.47:0/3363159228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f380fcbb0 0x7f7f380fcf90 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 rx=0x7f7f28004870 tx=0x7f7f28009cb0).stop 2024-02-23T00:51:10.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.072+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3363159228 shutdown_connections 2024-02-23T00:51:10.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.072+0000 7f7f3ce23700 1 --2- 172.21.15.47:0/3363159228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f380f7400 0x7f7f38103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:10.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.072+0000 7f7f3ce23700 1 --2- 172.21.15.47:0/3363159228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f380fcbb0 0x7f7f380fcf90 unknown :-1 s=CLOSED pgs=2569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:10.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.072+0000 7f7f3ce23700 1 --2- 172.21.15.47:0/3363159228 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7f380fd510 0x7f7f380f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:10.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.072+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3363159228 >> 172.21.15.47:0/3363159228 conn(0x7f7f380f2820 msgr2=0x7f7f380f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:10.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.072+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3363159228 shutdown_connections 2024-02-23T00:51:10.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.072+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3363159228 wait complete. 2024-02-23T00:51:10.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.073+0000 7f7f3ce23700 1 librados: starting msgr at 2024-02-23T00:51:10.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.073+0000 7f7f3ce23700 1 librados: starting objecter 2024-02-23T00:51:10.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.073+0000 7f7f3ce23700 1 Processor -- start 2024-02-23T00:51:10.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.073+0000 7f7f3ce23700 1 -- start start 2024-02-23T00:51:10.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.073+0000 7f7f3ce23700 1 librados: setting wanted keys 2024-02-23T00:51:10.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.073+0000 7f7f3ce23700 1 librados: calling monclient init 2024-02-23T00:51:10.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.073+0000 7f7f3ce23700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f380f7400 0x7f7f3810e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:10.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.073+0000 7f7f3ce23700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f380fcbb0 0x7f7f3810e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:10.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.073+0000 7f7f3ce23700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7f380fd510 0x7f7f38108200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:10.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.073+0000 7f7f3ce23700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f381086f0 con 0x7f7f380f7400 2024-02-23T00:51:10.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.073+0000 7f7f3ce23700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f38108860 con 0x7f7f380fcbb0 2024-02-23T00:51:10.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.073+0000 7f7f3ce23700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f38108a20 con 0x7f7f380fd510 2024-02-23T00:51:10.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.073+0000 7f7f3659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f380f7400 0x7f7f3810e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:10.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.074+0000 7f7f36d9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7f380fd510 0x7f7f38108200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:10.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.074+0000 7f7f3659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f380f7400 0x7f7f3810e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35106/0 (socket says 172.21.15.47:35106) 2024-02-23T00:51:10.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.074+0000 7f7f3659c700 1 -- 172.21.15.47:0/3815867910 learned_addr learned my addr 172.21.15.47:0/3815867910 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:51:10.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.074+0000 7f7f35d9b700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f380fcbb0 0x7f7f3810e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:10.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.074+0000 7f7f36d9d700 1 -- 172.21.15.47:0/3815867910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f380fcbb0 msgr2=0x7f7f3810e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:10.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.074+0000 7f7f36d9d700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f380fcbb0 0x7f7f3810e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:10.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.074+0000 7f7f36d9d700 1 -- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f380f7400 msgr2=0x7f7f3810e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:10.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.074+0000 7f7f36d9d700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f380f7400 0x7f7f3810e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:10.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.074+0000 7f7f36d9d700 1 -- 172.21.15.47:0/3815867910 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f2c005040 con 0x7f7f380fd510 2024-02-23T00:51:10.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.074+0000 7f7f3659c700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f380f7400 0x7f7f3810e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:51:10.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.074+0000 7f7f36d9d700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7f380fd510 0x7f7f38108200 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 rx=0x7f7f3000d8d0 tx=0x7f7f30004db0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:10.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.074+0000 7f7f277fe700 10 client.?.objecter ms_handle_connect 0x7f7f380fd510 2024-02-23T00:51:10.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.074+0000 7f7f277fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:51:10.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.074+0000 7f7f277fe700 1 -- 172.21.15.47:0/3815867910 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f30004020 con 0x7f7f380fd510 2024-02-23T00:51:10.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.074+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f28005040 con 0x7f7f380fd510 2024-02-23T00:51:10.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.074+0000 7f7f277fe700 1 -- 172.21.15.47:0/3815867910 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f7f3000f040 con 0x7f7f380fd510 2024-02-23T00:51:10.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.074+0000 7f7f277fe700 1 -- 172.21.15.47:0/3815867910 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f30003430 con 0x7f7f380fd510 2024-02-23T00:51:10.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.074+0000 7f7f3ce23700 10 client.41633.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:51:10.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.074+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f38109060 con 0x7f7f380fd510 2024-02-23T00:51:10.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.074+0000 7f7f3ce23700 1 librados: init done 2024-02-23T00:51:10.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.075+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f7f3805fa60 con 0x7f7f380fd510 2024-02-23T00:51:10.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.076+0000 7f7f277fe700 1 -- 172.21.15.47:0/3815867910 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f7f3000b510 con 0x7f7f380fd510 2024-02-23T00:51:10.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.076+0000 7f7f277fe700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7f20062cb0 0x7f7f20065170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:10.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.076+0000 7f7f277fe700 1 -- 172.21.15.47:0/3815867910 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(540..540 src has 463..540) v4 ==== 9254+0+0 (secure 0 0 0) 0x7f7f3007dcc0 con 0x7f7f380fd510 2024-02-23T00:51:10.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.076+0000 7f7f277fe700 10 client.41633.objecter ms_dispatch 0x7f7f38001550 osd_map(540..540 src has 463..540) v4 2024-02-23T00:51:10.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.076+0000 7f7f277fe700 3 client.41633.objecter handle_osd_map got epochs [540,540] > 0 2024-02-23T00:51:10.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.076+0000 7f7f277fe700 3 client.41633.objecter handle_osd_map decoding full epoch 540 2024-02-23T00:51:10.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.076+0000 7f7f3659c700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7f20062cb0 0x7f7f20065170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:10.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.076+0000 7f7f277fe700 20 client.41633.objecter dump_active .. 0 homeless 2024-02-23T00:51:10.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.076+0000 7f7f277fe700 1 -- 172.21.15.47:0/3815867910 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v540) v1 ==== 54+0+16 (secure 0 0 0) 0x7f7f3007e160 con 0x7f7f380fd510 2024-02-23T00:51:10.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.077+0000 7f7f3ce23700 10 client.41633.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:10.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.077+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_assign -1 1 2024-02-23T00:51:10.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.077+0000 7f7f3ce23700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f7f380ffed0 0x7f7f381002b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:10.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.077+0000 7f7f3ce23700 20 client.41633.objecter _get_session s=0x7f7f3805ee80 osd=0 3 2024-02-23T00:51:10.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.077+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f3805ee80 osd=0 3 2024-02-23T00:51:10.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.077+0000 7f7f3ce23700 20 client.41633.objecter _get_session s=0x7f7f3805ee80 osd=0 3 2024-02-23T00:51:10.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.077+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_remove -1 1 2024-02-23T00:51:10.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.077+0000 7f7f3ce23700 20 client.41633.objecter get_session s=0x7f7f3805ee80 osd=0 3 2024-02-23T00:51:10.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.077+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_assign 0 1 2024-02-23T00:51:10.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.077+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f3805ee80 osd=0 4 2024-02-23T00:51:10.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.077+0000 7f7f3ce23700 10 client.41633.objecter _send_command 1 2024-02-23T00:51:10.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.077+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7f38105ea0 con 0x7f7f380ffed0 2024-02-23T00:51:10.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.077+0000 7f7f277fe700 10 client.41633.objecter ms_handle_connect 0x7f7f20062cb0 2024-02-23T00:51:10.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.077+0000 7f7f3659c700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7f20062cb0 0x7f7f20065170 secure :-1 s=READY pgs=3114 cs=0 l=1 rev1=1 rx=0x7f7f28007fd0 tx=0x7f7f2800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:10.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.077+0000 7f7f35d9b700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f7f380ffed0 0x7f7f381002b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:10.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.077+0000 7f7f277fe700 10 client.41633.objecter ms_handle_connect 0x7f7f380ffed0 2024-02-23T00:51:10.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.077+0000 7f7f35d9b700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f7f380ffed0 0x7f7f381002b0 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:10.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.078+0000 7f7f277fe700 1 -- 172.21.15.47:0/3815867910 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7f38105ea0 con 0x7f7f380ffed0 2024-02-23T00:51:10.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.078+0000 7f7f277fe700 10 client.41633.objecter ms_dispatch 0x7f7f38001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:51:10.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.078+0000 7f7f277fe700 10 client.41633.objecter _finish_command 1 = system:0 2024-02-23T00:51:10.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.078+0000 7f7f277fe700 20 client.41633.objecter put_session s=0x7f7f3805ee80 osd=0 4 2024-02-23T00:51:10.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.078+0000 7f7f277fe700 15 client.41633.objecter _session_command_op_remove 0 1 2024-02-23T00:51:10.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.100+0000 7f7f3ce23700 10 client.41633.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:51:10.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.100+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_assign -1 2 2024-02-23T00:51:10.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.100+0000 7f7f3ce23700 20 client.41633.objecter _get_session s=0x7f7f3805ee80 osd=0 3 2024-02-23T00:51:10.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.100+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f3805ee80 osd=0 3 2024-02-23T00:51:10.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.100+0000 7f7f3ce23700 20 client.41633.objecter _get_session s=0x7f7f3805ee80 osd=0 3 2024-02-23T00:51:10.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.100+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_remove -1 2 2024-02-23T00:51:10.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.100+0000 7f7f3ce23700 20 client.41633.objecter get_session s=0x7f7f3805ee80 osd=0 3 2024-02-23T00:51:10.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.100+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_assign 0 2 2024-02-23T00:51:10.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.100+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f3805ee80 osd=0 4 2024-02-23T00:51:10.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.100+0000 7f7f3ce23700 10 client.41633.objecter _send_command 2 2024-02-23T00:51:10.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.100+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f7f380273c0 con 0x7f7f380ffed0 2024-02-23T00:51:10.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.102+0000 7f7f277fe700 1 -- 172.21.15.47:0/3815867910 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f7f380273c0 con 0x7f7f380ffed0 2024-02-23T00:51:10.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.102+0000 7f7f277fe700 10 client.41633.objecter ms_dispatch 0x7f7f38001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:51:10.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.102+0000 7f7f277fe700 10 client.41633.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:51:10.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.102+0000 7f7f277fe700 20 client.41633.objecter put_session s=0x7f7f3805ee80 osd=0 4 2024-02-23T00:51:10.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.102+0000 7f7f277fe700 15 client.41633.objecter _session_command_op_remove 0 2 2024-02-23T00:51:10.105 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:51:10.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.102+0000 7f7f3ce23700 10 client.41633.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:10.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.102+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_assign -1 3 2024-02-23T00:51:10.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.102+0000 7f7f3ce23700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f7f380598f0 0x7f7f38059cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:10.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.102+0000 7f7f3ce23700 20 client.41633.objecter _get_session s=0x7f7f38028710 osd=1 3 2024-02-23T00:51:10.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.102+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f38028710 osd=1 3 2024-02-23T00:51:10.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.102+0000 7f7f3ce23700 20 client.41633.objecter _get_session s=0x7f7f38028710 osd=1 3 2024-02-23T00:51:10.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.102+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_remove -1 3 2024-02-23T00:51:10.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.102+0000 7f7f3ce23700 20 client.41633.objecter get_session s=0x7f7f38028710 osd=1 3 2024-02-23T00:51:10.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.102+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_assign 1 3 2024-02-23T00:51:10.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.102+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f38028710 osd=1 4 2024-02-23T00:51:10.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.102+0000 7f7f3ce23700 10 client.41633.objecter _send_command 3 2024-02-23T00:51:10.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.102+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7f3811b670 con 0x7f7f380598f0 2024-02-23T00:51:10.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.102+0000 7f7f3659c700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f7f380598f0 0x7f7f38059cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:10.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.103+0000 7f7f277fe700 10 client.41633.objecter ms_handle_connect 0x7f7f380598f0 2024-02-23T00:51:10.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.103+0000 7f7f3659c700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f7f380598f0 0x7f7f38059cf0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:10.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.103+0000 7f7f277fe700 1 -- 172.21.15.47:0/3815867910 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7f3811b670 con 0x7f7f380598f0 2024-02-23T00:51:10.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.103+0000 7f7f277fe700 10 client.41633.objecter ms_dispatch 0x7f7f38001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:51:10.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.103+0000 7f7f277fe700 10 client.41633.objecter _finish_command 3 = system:0 2024-02-23T00:51:10.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.103+0000 7f7f277fe700 20 client.41633.objecter put_session s=0x7f7f38028710 osd=1 4 2024-02-23T00:51:10.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.103+0000 7f7f277fe700 15 client.41633.objecter _session_command_op_remove 1 3 2024-02-23T00:51:10.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.118+0000 7f7f3ce23700 10 client.41633.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:51:10.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.118+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_assign -1 4 2024-02-23T00:51:10.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.118+0000 7f7f3ce23700 20 client.41633.objecter _get_session s=0x7f7f38028710 osd=1 3 2024-02-23T00:51:10.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.118+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f38028710 osd=1 3 2024-02-23T00:51:10.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.118+0000 7f7f3ce23700 20 client.41633.objecter _get_session s=0x7f7f38028710 osd=1 3 2024-02-23T00:51:10.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.118+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_remove -1 4 2024-02-23T00:51:10.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.118+0000 7f7f3ce23700 20 client.41633.objecter get_session s=0x7f7f38028710 osd=1 3 2024-02-23T00:51:10.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.118+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_assign 1 4 2024-02-23T00:51:10.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.118+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f38028710 osd=1 4 2024-02-23T00:51:10.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.118+0000 7f7f3ce23700 10 client.41633.objecter _send_command 4 2024-02-23T00:51:10.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.118+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f7f38059360 con 0x7f7f380598f0 2024-02-23T00:51:10.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.120+0000 7f7f277fe700 1 -- 172.21.15.47:0/3815867910 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f7f38059360 con 0x7f7f380598f0 2024-02-23T00:51:10.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.120+0000 7f7f277fe700 10 client.41633.objecter ms_dispatch 0x7f7f38001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:51:10.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.120+0000 7f7f277fe700 10 client.41633.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:51:10.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.120+0000 7f7f277fe700 20 client.41633.objecter put_session s=0x7f7f38028710 osd=1 4 2024-02-23T00:51:10.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.120+0000 7f7f277fe700 15 client.41633.objecter _session_command_op_remove 1 4 2024-02-23T00:51:10.123 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:51:10.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.120+0000 7f7f3ce23700 10 client.41633.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:10.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.120+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_assign -1 5 2024-02-23T00:51:10.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.120+0000 7f7f3ce23700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f7f38028270 0x7f7f3811a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:10.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.120+0000 7f7f3ce23700 20 client.41633.objecter _get_session s=0x7f7f38059770 osd=2 3 2024-02-23T00:51:10.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.120+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f38059770 osd=2 3 2024-02-23T00:51:10.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.120+0000 7f7f3ce23700 20 client.41633.objecter _get_session s=0x7f7f38059770 osd=2 3 2024-02-23T00:51:10.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.120+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_remove -1 5 2024-02-23T00:51:10.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.120+0000 7f7f3ce23700 20 client.41633.objecter get_session s=0x7f7f38059770 osd=2 3 2024-02-23T00:51:10.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.120+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_assign 2 5 2024-02-23T00:51:10.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.120+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f38059770 osd=2 4 2024-02-23T00:51:10.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.120+0000 7f7f3ce23700 10 client.41633.objecter _send_command 5 2024-02-23T00:51:10.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.120+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7f3811a580 con 0x7f7f38028270 2024-02-23T00:51:10.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.120+0000 7f7f35d9b700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f7f38028270 0x7f7f3811a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:10.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.120+0000 7f7f277fe700 10 client.41633.objecter ms_handle_connect 0x7f7f38028270 2024-02-23T00:51:10.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.120+0000 7f7f35d9b700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f7f38028270 0x7f7f3811a090 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:10.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.121+0000 7f7f277fe700 1 -- 172.21.15.47:0/3815867910 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7f3811a580 con 0x7f7f38028270 2024-02-23T00:51:10.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.121+0000 7f7f277fe700 10 client.41633.objecter ms_dispatch 0x7f7f38001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:51:10.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.121+0000 7f7f277fe700 10 client.41633.objecter _finish_command 5 = system:0 2024-02-23T00:51:10.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.121+0000 7f7f277fe700 20 client.41633.objecter put_session s=0x7f7f38059770 osd=2 4 2024-02-23T00:51:10.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.121+0000 7f7f277fe700 15 client.41633.objecter _session_command_op_remove 2 5 2024-02-23T00:51:10.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.135+0000 7f7f3ce23700 10 client.41633.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:51:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.135+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_assign -1 6 2024-02-23T00:51:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.135+0000 7f7f3ce23700 20 client.41633.objecter _get_session s=0x7f7f38059770 osd=2 3 2024-02-23T00:51:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.135+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f38059770 osd=2 3 2024-02-23T00:51:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.135+0000 7f7f3ce23700 20 client.41633.objecter _get_session s=0x7f7f38059770 osd=2 3 2024-02-23T00:51:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.135+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_remove -1 6 2024-02-23T00:51:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.135+0000 7f7f3ce23700 20 client.41633.objecter get_session s=0x7f7f38059770 osd=2 3 2024-02-23T00:51:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.135+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_assign 2 6 2024-02-23T00:51:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.135+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f38059770 osd=2 4 2024-02-23T00:51:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.135+0000 7f7f3ce23700 10 client.41633.objecter _send_command 6 2024-02-23T00:51:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.135+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f7f380277d0 con 0x7f7f38028270 2024-02-23T00:51:10.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.137+0000 7f7f277fe700 1 -- 172.21.15.47:0/3815867910 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f7f380277d0 con 0x7f7f38028270 2024-02-23T00:51:10.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.137+0000 7f7f277fe700 10 client.41633.objecter ms_dispatch 0x7f7f38001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:51:10.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.137+0000 7f7f277fe700 10 client.41633.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:51:10.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.137+0000 7f7f277fe700 20 client.41633.objecter put_session s=0x7f7f38059770 osd=2 4 2024-02-23T00:51:10.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.137+0000 7f7f277fe700 15 client.41633.objecter _session_command_op_remove 2 6 2024-02-23T00:51:10.139 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:51:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.137+0000 7f7f3ce23700 10 client.41633.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.137+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_assign -1 7 2024-02-23T00:51:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.137+0000 7f7f3ce23700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f7f380f87f0 0x7f7f380f8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.137+0000 7f7f3ce23700 20 client.41633.objecter _get_session s=0x7f7f3811aef0 osd=3 3 2024-02-23T00:51:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.137+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f3811aef0 osd=3 3 2024-02-23T00:51:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.137+0000 7f7f3ce23700 20 client.41633.objecter _get_session s=0x7f7f3811aef0 osd=3 3 2024-02-23T00:51:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.137+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_remove -1 7 2024-02-23T00:51:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.137+0000 7f7f3ce23700 20 client.41633.objecter get_session s=0x7f7f3811aef0 osd=3 3 2024-02-23T00:51:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.137+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_assign 3 7 2024-02-23T00:51:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.137+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f3811aef0 osd=3 4 2024-02-23T00:51:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.137+0000 7f7f3ce23700 10 client.41633.objecter _send_command 7 2024-02-23T00:51:10.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.137+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7f380fa720 con 0x7f7f380f87f0 2024-02-23T00:51:10.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.137+0000 7f7f36d9d700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f7f380f87f0 0x7f7f380f8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:10.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.137+0000 7f7f277fe700 10 client.41633.objecter ms_handle_connect 0x7f7f380f87f0 2024-02-23T00:51:10.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.137+0000 7f7f36d9d700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f7f380f87f0 0x7f7f380f8c50 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:10.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.138+0000 7f7f277fe700 1 -- 172.21.15.47:0/3815867910 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7f380fa720 con 0x7f7f380f87f0 2024-02-23T00:51:10.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.138+0000 7f7f277fe700 10 client.41633.objecter ms_dispatch 0x7f7f38001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:51:10.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.138+0000 7f7f277fe700 10 client.41633.objecter _finish_command 7 = system:0 2024-02-23T00:51:10.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.138+0000 7f7f277fe700 20 client.41633.objecter put_session s=0x7f7f3811aef0 osd=3 4 2024-02-23T00:51:10.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.138+0000 7f7f277fe700 15 client.41633.objecter _session_command_op_remove 3 7 2024-02-23T00:51:10.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.152+0000 7f7f3ce23700 10 client.41633.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:51:10.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.152+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_assign -1 8 2024-02-23T00:51:10.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.152+0000 7f7f3ce23700 20 client.41633.objecter _get_session s=0x7f7f3811aef0 osd=3 3 2024-02-23T00:51:10.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.152+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f3811aef0 osd=3 3 2024-02-23T00:51:10.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.152+0000 7f7f3ce23700 20 client.41633.objecter _get_session s=0x7f7f3811aef0 osd=3 3 2024-02-23T00:51:10.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.152+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_remove -1 8 2024-02-23T00:51:10.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.152+0000 7f7f3ce23700 20 client.41633.objecter get_session s=0x7f7f3811aef0 osd=3 3 2024-02-23T00:51:10.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.152+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_assign 3 8 2024-02-23T00:51:10.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.152+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f3811aef0 osd=3 4 2024-02-23T00:51:10.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.152+0000 7f7f3ce23700 10 client.41633.objecter _send_command 8 2024-02-23T00:51:10.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.152+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f7f380fa2c0 con 0x7f7f380f87f0 2024-02-23T00:51:10.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.154+0000 7f7f277fe700 1 -- 172.21.15.47:0/3815867910 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f7f380fa2c0 con 0x7f7f380f87f0 2024-02-23T00:51:10.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.154+0000 7f7f277fe700 10 client.41633.objecter ms_dispatch 0x7f7f38001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:51:10.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.154+0000 7f7f277fe700 10 client.41633.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:51:10.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.154+0000 7f7f277fe700 20 client.41633.objecter put_session s=0x7f7f3811aef0 osd=3 4 2024-02-23T00:51:10.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.154+0000 7f7f277fe700 15 client.41633.objecter _session_command_op_remove 3 8 2024-02-23T00:51:10.158 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:51:10.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.154+0000 7f7f3ce23700 10 client.41633.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:10.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.154+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_assign -1 9 2024-02-23T00:51:10.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.154+0000 7f7f3ce23700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f7f380f97c0 0x7f7f380f9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:10.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.154+0000 7f7f3ce23700 20 client.41633.objecter _get_session s=0x7f7f380f9140 osd=4 3 2024-02-23T00:51:10.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.154+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f380f9140 osd=4 3 2024-02-23T00:51:10.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.154+0000 7f7f3ce23700 20 client.41633.objecter _get_session s=0x7f7f380f9140 osd=4 3 2024-02-23T00:51:10.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.154+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_remove -1 9 2024-02-23T00:51:10.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.154+0000 7f7f3ce23700 20 client.41633.objecter get_session s=0x7f7f380f9140 osd=4 3 2024-02-23T00:51:10.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.154+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_assign 4 9 2024-02-23T00:51:10.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.154+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f380f9140 osd=4 4 2024-02-23T00:51:10.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.154+0000 7f7f3ce23700 10 client.41633.objecter _send_command 9 2024-02-23T00:51:10.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.154+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7f380fa0f0 con 0x7f7f380f97c0 2024-02-23T00:51:10.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.154+0000 7f7f3659c700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f7f380f97c0 0x7f7f380f9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:10.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.155+0000 7f7f277fe700 10 client.41633.objecter ms_handle_connect 0x7f7f380f97c0 2024-02-23T00:51:10.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.155+0000 7f7f3659c700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f7f380f97c0 0x7f7f380f9c00 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:10.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.155+0000 7f7f277fe700 1 -- 172.21.15.47:0/3815867910 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7f380fa0f0 con 0x7f7f380f97c0 2024-02-23T00:51:10.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.155+0000 7f7f277fe700 10 client.41633.objecter ms_dispatch 0x7f7f38001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:51:10.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.155+0000 7f7f277fe700 10 client.41633.objecter _finish_command 9 = system:0 2024-02-23T00:51:10.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.155+0000 7f7f277fe700 20 client.41633.objecter put_session s=0x7f7f380f9140 osd=4 4 2024-02-23T00:51:10.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.155+0000 7f7f277fe700 15 client.41633.objecter _session_command_op_remove 4 9 2024-02-23T00:51:10.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.170+0000 7f7f3ce23700 10 client.41633.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:51:10.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.170+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_assign -1 10 2024-02-23T00:51:10.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.170+0000 7f7f3ce23700 20 client.41633.objecter _get_session s=0x7f7f380f9140 osd=4 3 2024-02-23T00:51:10.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.170+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f380f9140 osd=4 3 2024-02-23T00:51:10.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.170+0000 7f7f3ce23700 20 client.41633.objecter _get_session s=0x7f7f380f9140 osd=4 3 2024-02-23T00:51:10.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.170+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_remove -1 10 2024-02-23T00:51:10.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.170+0000 7f7f3ce23700 20 client.41633.objecter get_session s=0x7f7f380f9140 osd=4 3 2024-02-23T00:51:10.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.170+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_assign 4 10 2024-02-23T00:51:10.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.170+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f380f9140 osd=4 4 2024-02-23T00:51:10.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.170+0000 7f7f3ce23700 10 client.41633.objecter _send_command 10 2024-02-23T00:51:10.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.170+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f7f380fa0f0 con 0x7f7f380f97c0 2024-02-23T00:51:10.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.172+0000 7f7f277fe700 1 -- 172.21.15.47:0/3815867910 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f7f380fa0f0 con 0x7f7f380f97c0 2024-02-23T00:51:10.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.172+0000 7f7f277fe700 10 client.41633.objecter ms_dispatch 0x7f7f38001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:51:10.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.172+0000 7f7f277fe700 10 client.41633.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:51:10.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.172+0000 7f7f277fe700 20 client.41633.objecter put_session s=0x7f7f380f9140 osd=4 4 2024-02-23T00:51:10.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.172+0000 7f7f277fe700 15 client.41633.objecter _session_command_op_remove 4 10 2024-02-23T00:51:10.175 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:51:10.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.172+0000 7f7f3ce23700 10 client.41633.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:10.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.172+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_assign -1 11 2024-02-23T00:51:10.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.172+0000 7f7f3ce23700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f7f38126130 0x7f7f38128520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:10.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.172+0000 7f7f3ce23700 20 client.41633.objecter _get_session s=0x7f7f380fa0f0 osd=5 3 2024-02-23T00:51:10.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.172+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f380fa0f0 osd=5 3 2024-02-23T00:51:10.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.172+0000 7f7f3ce23700 20 client.41633.objecter _get_session s=0x7f7f380fa0f0 osd=5 3 2024-02-23T00:51:10.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.172+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_remove -1 11 2024-02-23T00:51:10.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.172+0000 7f7f3ce23700 20 client.41633.objecter get_session s=0x7f7f380fa0f0 osd=5 3 2024-02-23T00:51:10.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.172+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_assign 5 11 2024-02-23T00:51:10.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.172+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f380fa0f0 osd=5 4 2024-02-23T00:51:10.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.172+0000 7f7f3ce23700 10 client.41633.objecter _send_command 11 2024-02-23T00:51:10.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.172+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7f3812b200 con 0x7f7f38126130 2024-02-23T00:51:10.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.173+0000 7f7f35d9b700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f7f38126130 0x7f7f38128520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:10.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.173+0000 7f7f277fe700 10 client.41633.objecter ms_handle_connect 0x7f7f38126130 2024-02-23T00:51:10.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.173+0000 7f7f35d9b700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f7f38126130 0x7f7f38128520 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:10.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.174+0000 7f7f277fe700 1 -- 172.21.15.47:0/3815867910 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7f3812b200 con 0x7f7f38126130 2024-02-23T00:51:10.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.174+0000 7f7f277fe700 10 client.41633.objecter ms_dispatch 0x7f7f38001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:51:10.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.174+0000 7f7f277fe700 10 client.41633.objecter _finish_command 11 = system:0 2024-02-23T00:51:10.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.174+0000 7f7f277fe700 20 client.41633.objecter put_session s=0x7f7f380fa0f0 osd=5 4 2024-02-23T00:51:10.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.174+0000 7f7f277fe700 15 client.41633.objecter _session_command_op_remove 5 11 2024-02-23T00:51:10.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.188+0000 7f7f3ce23700 10 client.41633.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:51:10.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.188+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_assign -1 12 2024-02-23T00:51:10.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.188+0000 7f7f3ce23700 20 client.41633.objecter _get_session s=0x7f7f380fa0f0 osd=5 3 2024-02-23T00:51:10.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.188+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f380fa0f0 osd=5 3 2024-02-23T00:51:10.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.188+0000 7f7f3ce23700 20 client.41633.objecter _get_session s=0x7f7f380fa0f0 osd=5 3 2024-02-23T00:51:10.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.188+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_remove -1 12 2024-02-23T00:51:10.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.188+0000 7f7f3ce23700 20 client.41633.objecter get_session s=0x7f7f380fa0f0 osd=5 3 2024-02-23T00:51:10.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.188+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_assign 5 12 2024-02-23T00:51:10.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.188+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f380fa0f0 osd=5 4 2024-02-23T00:51:10.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.188+0000 7f7f3ce23700 10 client.41633.objecter _send_command 12 2024-02-23T00:51:10.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.188+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f7f3812b200 con 0x7f7f38126130 2024-02-23T00:51:10.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.190+0000 7f7f277fe700 1 -- 172.21.15.47:0/3815867910 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f7f3812b200 con 0x7f7f38126130 2024-02-23T00:51:10.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.190+0000 7f7f277fe700 10 client.41633.objecter ms_dispatch 0x7f7f38001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:51:10.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.190+0000 7f7f277fe700 10 client.41633.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:51:10.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.190+0000 7f7f277fe700 20 client.41633.objecter put_session s=0x7f7f380fa0f0 osd=5 4 2024-02-23T00:51:10.194 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:51:10.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.190+0000 7f7f277fe700 15 client.41633.objecter _session_command_op_remove 5 12 2024-02-23T00:51:10.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.191+0000 7f7f3ce23700 10 client.41633.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:10.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.191+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_assign -1 13 2024-02-23T00:51:10.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.191+0000 7f7f3ce23700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f7f38128f20 0x7f7f38129380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:10.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.191+0000 7f7f3ce23700 20 client.41633.objecter _get_session s=0x7f7f3812b200 osd=6 3 2024-02-23T00:51:10.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.191+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f3812b200 osd=6 3 2024-02-23T00:51:10.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.191+0000 7f7f3ce23700 20 client.41633.objecter _get_session s=0x7f7f3812b200 osd=6 3 2024-02-23T00:51:10.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.191+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_remove -1 13 2024-02-23T00:51:10.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.191+0000 7f7f3ce23700 20 client.41633.objecter get_session s=0x7f7f3812b200 osd=6 3 2024-02-23T00:51:10.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.191+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_assign 6 13 2024-02-23T00:51:10.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.191+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f3812b200 osd=6 4 2024-02-23T00:51:10.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.191+0000 7f7f3ce23700 10 client.41633.objecter _send_command 13 2024-02-23T00:51:10.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.191+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7f3812e580 con 0x7f7f38128f20 2024-02-23T00:51:10.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.191+0000 7f7f36d9d700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f7f38128f20 0x7f7f38129380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:10.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.191+0000 7f7f277fe700 10 client.41633.objecter ms_handle_connect 0x7f7f38128f20 2024-02-23T00:51:10.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.191+0000 7f7f36d9d700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f7f38128f20 0x7f7f38129380 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:10.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.192+0000 7f7f277fe700 1 -- 172.21.15.47:0/3815867910 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7f3812e580 con 0x7f7f38128f20 2024-02-23T00:51:10.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.192+0000 7f7f277fe700 10 client.41633.objecter ms_dispatch 0x7f7f38001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:51:10.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.192+0000 7f7f277fe700 10 client.41633.objecter _finish_command 13 = system:0 2024-02-23T00:51:10.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.192+0000 7f7f277fe700 20 client.41633.objecter put_session s=0x7f7f3812b200 osd=6 4 2024-02-23T00:51:10.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.192+0000 7f7f277fe700 15 client.41633.objecter _session_command_op_remove 6 13 2024-02-23T00:51:10.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.207+0000 7f7f3ce23700 10 client.41633.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:51:10.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.207+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_assign -1 14 2024-02-23T00:51:10.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.207+0000 7f7f3ce23700 20 client.41633.objecter _get_session s=0x7f7f3812b200 osd=6 3 2024-02-23T00:51:10.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.207+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f3812b200 osd=6 3 2024-02-23T00:51:10.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.207+0000 7f7f3ce23700 20 client.41633.objecter _get_session s=0x7f7f3812b200 osd=6 3 2024-02-23T00:51:10.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.207+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_remove -1 14 2024-02-23T00:51:10.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.207+0000 7f7f3ce23700 20 client.41633.objecter get_session s=0x7f7f3812b200 osd=6 3 2024-02-23T00:51:10.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.207+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_assign 6 14 2024-02-23T00:51:10.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.207+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f3812b200 osd=6 4 2024-02-23T00:51:10.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.207+0000 7f7f3ce23700 10 client.41633.objecter _send_command 14 2024-02-23T00:51:10.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.207+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f7f38129980 con 0x7f7f38128f20 2024-02-23T00:51:10.211 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:51:10.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.208+0000 7f7f277fe700 1 -- 172.21.15.47:0/3815867910 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f7f38129980 con 0x7f7f38128f20 2024-02-23T00:51:10.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.208+0000 7f7f277fe700 10 client.41633.objecter ms_dispatch 0x7f7f38001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:51:10.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.208+0000 7f7f277fe700 10 client.41633.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:51:10.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.208+0000 7f7f277fe700 20 client.41633.objecter put_session s=0x7f7f3812b200 osd=6 4 2024-02-23T00:51:10.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.208+0000 7f7f277fe700 15 client.41633.objecter _session_command_op_remove 6 14 2024-02-23T00:51:10.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.209+0000 7f7f3ce23700 10 client.41633.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:10.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.209+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_assign -1 15 2024-02-23T00:51:10.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.209+0000 7f7f3ce23700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f7f3812a000 0x7f7f3812a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:10.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.209+0000 7f7f3ce23700 20 client.41633.objecter _get_session s=0x7f7f38129980 osd=7 3 2024-02-23T00:51:10.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.209+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f38129980 osd=7 3 2024-02-23T00:51:10.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.209+0000 7f7f3ce23700 20 client.41633.objecter _get_session s=0x7f7f38129980 osd=7 3 2024-02-23T00:51:10.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.209+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_remove -1 15 2024-02-23T00:51:10.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.209+0000 7f7f3ce23700 20 client.41633.objecter get_session s=0x7f7f38129980 osd=7 3 2024-02-23T00:51:10.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.209+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_assign 7 15 2024-02-23T00:51:10.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.209+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f38129980 osd=7 4 2024-02-23T00:51:10.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.209+0000 7f7f3ce23700 10 client.41633.objecter _send_command 15 2024-02-23T00:51:10.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.209+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7f3812aaf0 con 0x7f7f3812a000 2024-02-23T00:51:10.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.209+0000 7f7f3659c700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f7f3812a000 0x7f7f3812a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:10.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.210+0000 7f7f277fe700 10 client.41633.objecter ms_handle_connect 0x7f7f3812a000 2024-02-23T00:51:10.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.210+0000 7f7f3659c700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f7f3812a000 0x7f7f3812a460 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:10.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.211+0000 7f7f277fe700 1 -- 172.21.15.47:0/3815867910 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7f3812aaf0 con 0x7f7f3812a000 2024-02-23T00:51:10.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.211+0000 7f7f277fe700 10 client.41633.objecter ms_dispatch 0x7f7f38001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:51:10.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.211+0000 7f7f277fe700 10 client.41633.objecter _finish_command 15 = system:0 2024-02-23T00:51:10.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.211+0000 7f7f277fe700 20 client.41633.objecter put_session s=0x7f7f38129980 osd=7 4 2024-02-23T00:51:10.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.211+0000 7f7f277fe700 15 client.41633.objecter _session_command_op_remove 7 15 2024-02-23T00:51:10.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.225+0000 7f7f3ce23700 10 client.41633.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:51:10.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.225+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_assign -1 16 2024-02-23T00:51:10.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.225+0000 7f7f3ce23700 20 client.41633.objecter _get_session s=0x7f7f38129980 osd=7 3 2024-02-23T00:51:10.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.225+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f38129980 osd=7 3 2024-02-23T00:51:10.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.225+0000 7f7f3ce23700 20 client.41633.objecter _get_session s=0x7f7f38129980 osd=7 3 2024-02-23T00:51:10.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.225+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_remove -1 16 2024-02-23T00:51:10.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.225+0000 7f7f3ce23700 20 client.41633.objecter get_session s=0x7f7f38129980 osd=7 3 2024-02-23T00:51:10.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.225+0000 7f7f3ce23700 15 client.41633.objecter _session_command_op_assign 7 16 2024-02-23T00:51:10.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.225+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f38129980 osd=7 4 2024-02-23T00:51:10.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.225+0000 7f7f3ce23700 10 client.41633.objecter _send_command 16 2024-02-23T00:51:10.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.225+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f7f3812aaf0 con 0x7f7f3812a000 2024-02-23T00:51:10.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.227+0000 7f7f277fe700 1 -- 172.21.15.47:0/3815867910 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f7f3812aaf0 con 0x7f7f3812a000 2024-02-23T00:51:10.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.227+0000 7f7f277fe700 10 client.41633.objecter ms_dispatch 0x7f7f38001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:51:10.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.227+0000 7f7f277fe700 10 client.41633.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:51:10.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.227+0000 7f7f277fe700 20 client.41633.objecter put_session s=0x7f7f38129980 osd=7 4 2024-02-23T00:51:10.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.227+0000 7f7f277fe700 15 client.41633.objecter _session_command_op_remove 7 16 2024-02-23T00:51:10.232 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:51:10.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.227+0000 7f7f3ce23700 10 librados: watch_flush enter 2024-02-23T00:51:10.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 10 librados: watch_flush exit 2024-02-23T00:51:10.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 10 client.41633.objecter close_session for osd.0 2024-02-23T00:51:10.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f7f380ffed0 msgr2=0x7f7f381002b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:10.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f7f380ffed0 0x7f7f381002b0 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:10.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f3805ee80 osd=0 1 2024-02-23T00:51:10.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 10 client.41633.objecter close_session for osd.1 2024-02-23T00:51:10.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f7f380598f0 msgr2=0x7f7f38059cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:10.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f7f380598f0 0x7f7f38059cf0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:10.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f38028710 osd=1 1 2024-02-23T00:51:10.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 10 client.41633.objecter close_session for osd.2 2024-02-23T00:51:10.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f7f38028270 msgr2=0x7f7f3811a090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:10.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f7f38028270 0x7f7f3811a090 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:10.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f38059770 osd=2 1 2024-02-23T00:51:10.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 10 client.41633.objecter close_session for osd.3 2024-02-23T00:51:10.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f36d9d700 1 -- 172.21.15.47:0/3815867910 reap_dead start 2024-02-23T00:51:10.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f7f380f87f0 msgr2=0x7f7f380f8c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:10.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f7f380f87f0 0x7f7f380f8c50 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:10.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f3811aef0 osd=3 1 2024-02-23T00:51:10.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 10 client.41633.objecter close_session for osd.4 2024-02-23T00:51:10.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f7f380f97c0 msgr2=0x7f7f380f9c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:10.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f7f380f97c0 0x7f7f380f9c00 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:10.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f380f9140 osd=4 1 2024-02-23T00:51:10.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 10 client.41633.objecter close_session for osd.5 2024-02-23T00:51:10.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f7f38126130 msgr2=0x7f7f38128520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:10.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f7f38126130 0x7f7f38128520 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:10.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f380fa0f0 osd=5 1 2024-02-23T00:51:10.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 10 client.41633.objecter close_session for osd.6 2024-02-23T00:51:10.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f7f38128f20 msgr2=0x7f7f38129380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:10.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f7f38128f20 0x7f7f38129380 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:10.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f3812b200 osd=6 1 2024-02-23T00:51:10.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 10 client.41633.objecter close_session for osd.7 2024-02-23T00:51:10.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f7f3812a000 msgr2=0x7f7f3812a460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:10.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f7f3812a000 0x7f7f3812a460 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:10.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 20 client.41633.objecter put_session s=0x7f7f38129980 osd=7 1 2024-02-23T00:51:10.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 20 client.41633.objecter shutdown clearing up homeless session... 2024-02-23T00:51:10.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 10 client.41633.objecter successfully canceled tick 2024-02-23T00:51:10.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f36d9d700 1 -- 172.21.15.47:0/3815867910 reap_dead start 2024-02-23T00:51:10.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7f20062cb0 msgr2=0x7f7f20065170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:10.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7f20062cb0 0x7f7f20065170 secure :-1 s=READY pgs=3114 cs=0 l=1 rev1=1 rx=0x7f7f28007fd0 tx=0x7f7f2800b040).stop 2024-02-23T00:51:10.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7f380fd510 msgr2=0x7f7f38108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:10.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7f380fd510 0x7f7f38108200 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 rx=0x7f7f3000d8d0 tx=0x7f7f30004db0).stop 2024-02-23T00:51:10.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 shutdown_connections 2024-02-23T00:51:10.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7f380fd510 0x7f7f38108200 unknown :-1 s=CLOSED pgs=2727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:10.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 1 --2- 172.21.15.47:0/3815867910 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7f20062cb0 0x7f7f20065170 unknown :-1 s=CLOSED pgs=3114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:10.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 >> 172.21.15.47:0/3815867910 conn(0x7f7f380f2820 msgr2=0x7f7f38060e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:10.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 shutdown_connections 2024-02-23T00:51:10.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 1 -- 172.21.15.47:0/3815867910 wait complete. 2024-02-23T00:51:10.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:10.228+0000 7f7f3ce23700 1 librados: shutdown 2024-02-23T00:51:10.238 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:51:10.238 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:51:10.238 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:51:10.239 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:51:10.239 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:51:10.239 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:51:10.239 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:51:10.239 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:51:10.250 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set noscrub 2024-02-23T00:51:10.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:09.272540+0000 mon.a (mon.0) 2925 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:51:10.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:09.272608+0000 mon.a (mon.0) 2926 : cluster [DBG] osdmap e540: 8 total, 8 up, 7 in 2024-02-23T00:51:10.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:09.277331+0000 mon.a (mon.0) 2927 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-88"}]: dispatch 2024-02-23T00:51:10.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:09.338386+0000 mgr.y (mgr.14517) 495 : cluster [DBG] pgmap v814: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:10.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:09.341024+0000 osd.1 (osd.1) 233 : cluster [DBG] 5.c deep-scrub starts 2024-02-23T00:51:10.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:09.343880+0000 osd.1 (osd.1) 234 : cluster [DBG] 5.c deep-scrub ok 2024-02-23T00:51:10.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:09.473117+0000 osd.0 (osd.0) 223 : cluster [DBG] 62.13 deep-scrub starts 2024-02-23T00:51:10.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:09.477287+0000 osd.0 (osd.0) 224 : cluster [DBG] 62.13 deep-scrub ok 2024-02-23T00:51:10.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:09.826316+0000 osd.5 (osd.5) 242 : cluster [DBG] 2.18 scrub starts 2024-02-23T00:51:10.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:09.827850+0000 osd.5 (osd.5) 243 : cluster [DBG] 2.18 scrub ok 2024-02-23T00:51:10.457 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:10.076901+0000 mon.c (mon.2) 324 : audit [DBG] from='client.? 172.21.15.47:0/3815867910' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:51:10.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:09.272540+0000 mon.a (mon.0) 2925 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:51:10.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:09.272608+0000 mon.a (mon.0) 2926 : cluster [DBG] osdmap e540: 8 total, 8 up, 7 in 2024-02-23T00:51:10.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:09.277331+0000 mon.a (mon.0) 2927 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-88"}]: dispatch 2024-02-23T00:51:10.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:09.338386+0000 mgr.y (mgr.14517) 495 : cluster [DBG] pgmap v814: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:10.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:09.341024+0000 osd.1 (osd.1) 233 : cluster [DBG] 5.c deep-scrub starts 2024-02-23T00:51:10.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:09.343880+0000 osd.1 (osd.1) 234 : cluster [DBG] 5.c deep-scrub ok 2024-02-23T00:51:10.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:09.473117+0000 osd.0 (osd.0) 223 : cluster [DBG] 62.13 deep-scrub starts 2024-02-23T00:51:10.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:09.477287+0000 osd.0 (osd.0) 224 : cluster [DBG] 62.13 deep-scrub ok 2024-02-23T00:51:10.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:09.826316+0000 osd.5 (osd.5) 242 : cluster [DBG] 2.18 scrub starts 2024-02-23T00:51:10.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:09.827850+0000 osd.5 (osd.5) 243 : cluster [DBG] 2.18 scrub ok 2024-02-23T00:51:10.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:10.076901+0000 mon.c (mon.2) 324 : audit [DBG] from='client.? 172.21.15.47:0/3815867910' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:51:10.625 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:51:10.628 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:10.629 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:51:10.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:09.272540+0000 mon.a (mon.0) 2925 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:51:10.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:09.272608+0000 mon.a (mon.0) 2926 : cluster [DBG] osdmap e540: 8 total, 8 up, 7 in 2024-02-23T00:51:10.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:09.277331+0000 mon.a (mon.0) 2927 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-88"}]: dispatch 2024-02-23T00:51:10.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:09.338386+0000 mgr.y (mgr.14517) 495 : cluster [DBG] pgmap v814: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:10.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:09.341024+0000 osd.1 (osd.1) 233 : cluster [DBG] 5.c deep-scrub starts 2024-02-23T00:51:10.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:09.343880+0000 osd.1 (osd.1) 234 : cluster [DBG] 5.c deep-scrub ok 2024-02-23T00:51:10.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:09.473117+0000 osd.0 (osd.0) 223 : cluster [DBG] 62.13 deep-scrub starts 2024-02-23T00:51:10.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:09.477287+0000 osd.0 (osd.0) 224 : cluster [DBG] 62.13 deep-scrub ok 2024-02-23T00:51:10.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:09.826316+0000 osd.5 (osd.5) 242 : cluster [DBG] 2.18 scrub starts 2024-02-23T00:51:10.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:09.827850+0000 osd.5 (osd.5) 243 : cluster [DBG] 2.18 scrub ok 2024-02-23T00:51:10.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:10.076901+0000 mon.c (mon.2) 324 : audit [DBG] from='client.? 172.21.15.47:0/3815867910' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:51:10.729 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:51:10.756 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:51:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:51:10] "GET /metrics HTTP/1.1" 200 189660 "" "Prometheus/2.33.4" 2024-02-23T00:51:10.758 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:10.831 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:51:10.835 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:10.836 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:51:10.937 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:51:10.969 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:10.990 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:51:10.993 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:10.993 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:51:11.094 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:51:11.124 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:11.144 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:51:11.146 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:11.147 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:51:11.248 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:51:11.282 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:11.632 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:51:11.634 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:11.635 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:51:11.735 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:51:11.767 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:11.789 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:51:11.792 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:11.792 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:51:11.829 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:51:11.867 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:51:11.867 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:51:11.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:10.210452+0000 osd.7 (osd.7) 269 : cluster [DBG] 2.b scrub starts 2024-02-23T00:51:11.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:10.213333+0000 osd.7 (osd.7) 270 : cluster [DBG] 2.b scrub ok 2024-02-23T00:51:11.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:10.278527+0000 mon.a (mon.0) 2928 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-88"}]': finished 2024-02-23T00:51:11.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:10.278595+0000 mon.a (mon.0) 2929 : cluster [DBG] osdmap e541: 8 total, 8 up, 7 in 2024-02-23T00:51:11.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:10.297751+0000 osd.6 (osd.6) 211 : cluster [DBG] 18.10 scrub starts 2024-02-23T00:51:11.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:10.298735+0000 osd.6 (osd.6) 212 : cluster [DBG] 18.10 scrub ok 2024-02-23T00:51:11.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:10.340809+0000 osd.3 (osd.3) 258 : cluster [DBG] 18.a scrub starts 2024-02-23T00:51:11.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:10.341764+0000 osd.3 (osd.3) 259 : cluster [DBG] 18.a scrub ok 2024-02-23T00:51:11.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:10.506200+0000 osd.0 (osd.0) 225 : cluster [DBG] 62.12 scrub starts 2024-02-23T00:51:11.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:10.508374+0000 osd.0 (osd.0) 226 : cluster [DBG] 62.12 scrub ok 2024-02-23T00:51:11.889 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:10.210452+0000 osd.7 (osd.7) 269 : cluster [DBG] 2.b scrub starts 2024-02-23T00:51:11.889 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:10.213333+0000 osd.7 (osd.7) 270 : cluster [DBG] 2.b scrub ok 2024-02-23T00:51:11.889 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:10.278527+0000 mon.a (mon.0) 2928 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-88"}]': finished 2024-02-23T00:51:11.889 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:10.278595+0000 mon.a (mon.0) 2929 : cluster [DBG] osdmap e541: 8 total, 8 up, 7 in 2024-02-23T00:51:11.889 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:10.297751+0000 osd.6 (osd.6) 211 : cluster [DBG] 18.10 scrub starts 2024-02-23T00:51:11.889 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:10.298735+0000 osd.6 (osd.6) 212 : cluster [DBG] 18.10 scrub ok 2024-02-23T00:51:11.890 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:10.340809+0000 osd.3 (osd.3) 258 : cluster [DBG] 18.a scrub starts 2024-02-23T00:51:11.890 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:10.341764+0000 osd.3 (osd.3) 259 : cluster [DBG] 18.a scrub ok 2024-02-23T00:51:11.890 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:10.506200+0000 osd.0 (osd.0) 225 : cluster [DBG] 62.12 scrub starts 2024-02-23T00:51:11.890 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:10.508374+0000 osd.0 (osd.0) 226 : cluster [DBG] 62.12 scrub ok 2024-02-23T00:51:11.893 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:51:11.920 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:12.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:10.210452+0000 osd.7 (osd.7) 269 : cluster [DBG] 2.b scrub starts 2024-02-23T00:51:12.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:10.213333+0000 osd.7 (osd.7) 270 : cluster [DBG] 2.b scrub ok 2024-02-23T00:51:12.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:10.278527+0000 mon.a (mon.0) 2928 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-88"}]': finished 2024-02-23T00:51:12.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:10.278595+0000 mon.a (mon.0) 2929 : cluster [DBG] osdmap e541: 8 total, 8 up, 7 in 2024-02-23T00:51:12.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:10.297751+0000 osd.6 (osd.6) 211 : cluster [DBG] 18.10 scrub starts 2024-02-23T00:51:12.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:10.298735+0000 osd.6 (osd.6) 212 : cluster [DBG] 18.10 scrub ok 2024-02-23T00:51:12.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:10.340809+0000 osd.3 (osd.3) 258 : cluster [DBG] 18.a scrub starts 2024-02-23T00:51:12.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:10.341764+0000 osd.3 (osd.3) 259 : cluster [DBG] 18.a scrub ok 2024-02-23T00:51:12.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:10.506200+0000 osd.0 (osd.0) 225 : cluster [DBG] 62.12 scrub starts 2024-02-23T00:51:12.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:10.508374+0000 osd.0 (osd.0) 226 : cluster [DBG] 62.12 scrub ok 2024-02-23T00:51:12.272 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:51:12.944 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:51:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:51:12.944 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:51:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:51:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:51:12.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:11.278909+0000 mon.a (mon.0) 2930 : cluster [DBG] osdmap e542: 8 total, 8 up, 7 in 2024-02-23T00:51:12.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:11.338722+0000 mgr.y (mgr.14517) 496 : cluster [DBG] pgmap v817: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:51:12.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:11.523968+0000 osd.0 (osd.0) 227 : cluster [DBG] 6.9 scrub starts 2024-02-23T00:51:12.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:11.526577+0000 osd.0 (osd.0) 228 : cluster [DBG] 6.9 scrub ok 2024-02-23T00:51:12.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:11.742600+0000 mon.a (mon.0) 2931 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:51:12.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:12.280698+0000 mon.a (mon.0) 2932 : cluster [DBG] osdmap e543: 8 total, 8 up, 7 in 2024-02-23T00:51:12.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:12.282795+0000 mon.a (mon.0) 2933 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:51:12.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:12.492053+0000 osd.0 (osd.0) 229 : cluster [DBG] 18.8 scrub starts 2024-02-23T00:51:12.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:12.492939+0000 osd.0 (osd.0) 230 : cluster [DBG] 18.8 scrub ok 2024-02-23T00:51:13.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:11.278909+0000 mon.a (mon.0) 2930 : cluster [DBG] osdmap e542: 8 total, 8 up, 7 in 2024-02-23T00:51:13.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:11.338722+0000 mgr.y (mgr.14517) 496 : cluster [DBG] pgmap v817: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:51:13.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:11.523968+0000 osd.0 (osd.0) 227 : cluster [DBG] 6.9 scrub starts 2024-02-23T00:51:13.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:11.526577+0000 osd.0 (osd.0) 228 : cluster [DBG] 6.9 scrub ok 2024-02-23T00:51:13.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:11.742600+0000 mon.a (mon.0) 2931 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:51:13.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:12.280698+0000 mon.a (mon.0) 2932 : cluster [DBG] osdmap e543: 8 total, 8 up, 7 in 2024-02-23T00:51:13.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:12.282795+0000 mon.a (mon.0) 2933 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:51:13.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:12.492053+0000 osd.0 (osd.0) 229 : cluster [DBG] 18.8 scrub starts 2024-02-23T00:51:13.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:12.492939+0000 osd.0 (osd.0) 230 : cluster [DBG] 18.8 scrub ok 2024-02-23T00:51:13.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:11.278909+0000 mon.a (mon.0) 2930 : cluster [DBG] osdmap e542: 8 total, 8 up, 7 in 2024-02-23T00:51:13.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:11.338722+0000 mgr.y (mgr.14517) 496 : cluster [DBG] pgmap v817: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:51:13.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:11.523968+0000 osd.0 (osd.0) 227 : cluster [DBG] 6.9 scrub starts 2024-02-23T00:51:13.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:11.526577+0000 osd.0 (osd.0) 228 : cluster [DBG] 6.9 scrub ok 2024-02-23T00:51:13.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:11.742600+0000 mon.a (mon.0) 2931 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:51:13.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:12.280698+0000 mon.a (mon.0) 2932 : cluster [DBG] osdmap e543: 8 total, 8 up, 7 in 2024-02-23T00:51:13.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:12.282795+0000 mon.a (mon.0) 2933 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:51:13.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:12.492053+0000 osd.0 (osd.0) 229 : cluster [DBG] 18.8 scrub starts 2024-02-23T00:51:13.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:12.492939+0000 osd.0 (osd.0) 230 : cluster [DBG] 18.8 scrub ok 2024-02-23T00:51:13.192 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.3 dump_historic_ops 2024-02-23T00:51:13.550 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:51:13.552 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:13.553 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:51:13.647 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:51:13.654 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:51:13.719 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:14.002 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:12.334733+0000 osd.3 (osd.3) 260 : cluster [DBG] 18.1a deep-scrub starts 2024-02-23T00:51:14.002 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:12.335748+0000 osd.3 (osd.3) 261 : cluster [DBG] 18.1a deep-scrub ok 2024-02-23T00:51:14.002 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:12.706712+0000 osd.5 (osd.5) 244 : cluster [DBG] 5.6 scrub starts 2024-02-23T00:51:14.002 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:12.709613+0000 osd.5 (osd.5) 245 : cluster [DBG] 5.6 scrub ok 2024-02-23T00:51:14.002 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:12.741040+0000 mgr.y (mgr.14517) 497 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:51:14.002 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:13.282112+0000 mon.a (mon.0) 2934 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:51:14.002 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:13.282339+0000 mon.a (mon.0) 2935 : cluster [DBG] osdmap e544: 8 total, 8 up, 7 in 2024-02-23T00:51:14.002 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:13.296450+0000 mon.a (mon.0) 2936 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-90", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:51:14.002 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:13.324166+0000 osd.6 (osd.6) 213 : cluster [DBG] 4.a scrub starts 2024-02-23T00:51:14.002 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:13.326526+0000 osd.6 (osd.6) 214 : cluster [DBG] 4.a scrub ok 2024-02-23T00:51:14.002 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:13.339096+0000 mgr.y (mgr.14517) 498 : cluster [DBG] pgmap v820: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:14.003 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:12.334733+0000 osd.3 (osd.3) 260 : cluster [DBG] 18.1a deep-scrub starts 2024-02-23T00:51:14.003 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:12.335748+0000 osd.3 (osd.3) 261 : cluster [DBG] 18.1a deep-scrub ok 2024-02-23T00:51:14.003 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:12.706712+0000 osd.5 (osd.5) 244 : cluster [DBG] 5.6 scrub starts 2024-02-23T00:51:14.003 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:12.709613+0000 osd.5 (osd.5) 245 : cluster [DBG] 5.6 scrub ok 2024-02-23T00:51:14.003 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:12.741040+0000 mgr.y (mgr.14517) 497 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:51:14.003 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:13.282112+0000 mon.a (mon.0) 2934 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:51:14.003 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:13.282339+0000 mon.a (mon.0) 2935 : cluster [DBG] osdmap e544: 8 total, 8 up, 7 in 2024-02-23T00:51:14.003 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:13.296450+0000 mon.a (mon.0) 2936 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-90", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:51:14.003 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:13.324166+0000 osd.6 (osd.6) 213 : cluster [DBG] 4.a scrub starts 2024-02-23T00:51:14.003 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:13.326526+0000 osd.6 (osd.6) 214 : cluster [DBG] 4.a scrub ok 2024-02-23T00:51:14.003 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:13.339096+0000 mgr.y (mgr.14517) 498 : cluster [DBG] pgmap v820: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:14.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:12.334733+0000 osd.3 (osd.3) 260 : cluster [DBG] 18.1a deep-scrub starts 2024-02-23T00:51:14.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:12.335748+0000 osd.3 (osd.3) 261 : cluster [DBG] 18.1a deep-scrub ok 2024-02-23T00:51:14.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:12.706712+0000 osd.5 (osd.5) 244 : cluster [DBG] 5.6 scrub starts 2024-02-23T00:51:14.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:12.709613+0000 osd.5 (osd.5) 245 : cluster [DBG] 5.6 scrub ok 2024-02-23T00:51:14.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:12.741040+0000 mgr.y (mgr.14517) 497 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:51:14.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:13.282112+0000 mon.a (mon.0) 2934 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:51:14.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:13.282339+0000 mon.a (mon.0) 2935 : cluster [DBG] osdmap e544: 8 total, 8 up, 7 in 2024-02-23T00:51:14.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:13.296450+0000 mon.a (mon.0) 2936 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-90", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:51:14.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:13.324166+0000 osd.6 (osd.6) 213 : cluster [DBG] 4.a scrub starts 2024-02-23T00:51:14.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:13.326526+0000 osd.6 (osd.6) 214 : cluster [DBG] 4.a scrub ok 2024-02-23T00:51:14.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:13.339096+0000 mgr.y (mgr.14517) 498 : cluster [DBG] pgmap v820: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:14.369 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:51:14.372 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:14.373 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:51:14.473 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:51:14.507 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:14.527 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:51:14.530 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:14.530 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:51:14.631 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:51:14.661 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:14.681 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:51:14.684 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:14.684 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:51:14.748 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:13.261975+0000 osd.7 (osd.7) 271 : cluster [DBG] 5.1a scrub starts 2024-02-23T00:51:14.748 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:13.264864+0000 osd.7 (osd.7) 272 : cluster [DBG] 5.1a scrub ok 2024-02-23T00:51:14.748 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:13.351278+0000 osd.3 (osd.3) 262 : cluster [DBG] 18.1 scrub starts 2024-02-23T00:51:14.748 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:13.352605+0000 osd.3 (osd.3) 263 : cluster [DBG] 18.1 scrub ok 2024-02-23T00:51:14.749 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:13.372150+0000 osd.1 (osd.1) 235 : cluster [DBG] 2.9 scrub starts 2024-02-23T00:51:14.749 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:13.375033+0000 osd.1 (osd.1) 236 : cluster [DBG] 2.9 scrub ok 2024-02-23T00:51:14.749 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:14.286329+0000 mon.a (mon.0) 2937 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-90", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:51:14.749 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:14.286396+0000 mon.a (mon.0) 2938 : cluster [DBG] osdmap e545: 8 total, 8 up, 7 in 2024-02-23T00:51:14.749 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:14.287099+0000 mon.a (mon.0) 2939 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-90"}]: dispatch 2024-02-23T00:51:14.784 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:51:14.800 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:51:14.811 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:13.261975+0000 osd.7 (osd.7) 271 : cluster [DBG] 5.1a scrub starts 2024-02-23T00:51:14.811 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:13.264864+0000 osd.7 (osd.7) 272 : cluster [DBG] 5.1a scrub ok 2024-02-23T00:51:14.811 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:13.351278+0000 osd.3 (osd.3) 262 : cluster [DBG] 18.1 scrub starts 2024-02-23T00:51:14.812 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:13.352605+0000 osd.3 (osd.3) 263 : cluster [DBG] 18.1 scrub ok 2024-02-23T00:51:14.812 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:13.372150+0000 osd.1 (osd.1) 235 : cluster [DBG] 2.9 scrub starts 2024-02-23T00:51:14.812 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:13.375033+0000 osd.1 (osd.1) 236 : cluster [DBG] 2.9 scrub ok 2024-02-23T00:51:14.812 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:14.286329+0000 mon.a (mon.0) 2937 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-90", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:51:14.812 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:14.286396+0000 mon.a (mon.0) 2938 : cluster [DBG] osdmap e545: 8 total, 8 up, 7 in 2024-02-23T00:51:14.812 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:14.287099+0000 mon.a (mon.0) 2939 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-90"}]: dispatch 2024-02-23T00:51:14.812 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:14.487274+0000 osd.0 (osd.0) 231 : cluster [DBG] 5.1e scrub starts 2024-02-23T00:51:14.812 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:14.488197+0000 osd.0 (osd.0) 232 : cluster [DBG] 5.1e scrub ok 2024-02-23T00:51:14.813 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:14.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.825+0000 7fddfb62d700 1 -- 172.21.15.47:0/1945819265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddf40fcbb0 msgr2=0x7fddf40fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:14.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.825+0000 7fddfb62d700 1 --2- 172.21.15.47:0/1945819265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddf40fcbb0 0x7fddf40fcf90 secure :-1 s=READY pgs=3976 cs=0 l=1 rev1=1 rx=0x7fddf0004870 tx=0x7fddf0009cb0).stop 2024-02-23T00:51:14.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.826+0000 7fddfb62d700 1 -- 172.21.15.47:0/1945819265 shutdown_connections 2024-02-23T00:51:14.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.826+0000 7fddfb62d700 1 --2- 172.21.15.47:0/1945819265 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fddf40f7400 0x7fddf4103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:51:14.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.826+0000 7fddfb62d700 1 --2- 172.21.15.47:0/1945819265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddf40fcbb0 0x7fddf40fcf90 unknown :-1 s=CLOSED pgs=3976 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:14.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.826+0000 7fddfb62d700 1 --2- 172.21.15.47:0/1945819265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddf40fd510 0x7fddf40f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:14.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.826+0000 7fddfb62d700 1 -- 172.21.15.47:0/1945819265 >> 172.21.15.47:0/1945819265 conn(0x7fddf40f2820 msgr2=0x7fddf40f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:14.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.826+0000 7fddfb62d700 1 -- 172.21.15.47:0/1945819265 shutdown_connections 2024-02-23T00:51:14.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.826+0000 7fddfb62d700 1 -- 172.21.15.47:0/1945819265 wait complete. 2024-02-23T00:51:14.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.826+0000 7fddfb62d700 1 librados: starting msgr at 2024-02-23T00:51:14.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.826+0000 7fddfb62d700 1 librados: starting objecter 2024-02-23T00:51:14.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.826+0000 7fddfb62d700 1 Processor -- start 2024-02-23T00:51:14.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.828+0000 7fddfb62d700 1 -- start start 2024-02-23T00:51:14.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.828+0000 7fddfb62d700 1 librados: setting wanted keys 2024-02-23T00:51:14.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.828+0000 7fddfb62d700 1 librados: calling monclient init 2024-02-23T00:51:14.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.828+0000 7fddfb62d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddf40f7400 0x7fddf410e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:14.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.828+0000 7fddfb62d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fddf40fcbb0 0x7fddf410e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:14.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.828+0000 7fddfb62d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddf40fd510 0x7fddf41082c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:14.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.828+0000 7fddfb62d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf41087b0 con 0x7fddf40f7400 2024-02-23T00:51:14.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.828+0000 7fddfb62d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf4108920 con 0x7fddf40fd510 2024-02-23T00:51:14.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.828+0000 7fddfb62d700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf4108ae0 con 0x7fddf40fcbb0 2024-02-23T00:51:14.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.828+0000 7fddf93c9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddf40f7400 0x7fddf410e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:14.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.828+0000 7fddf93c9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddf40f7400 0x7fddf410e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35134/0 (socket says 172.21.15.47:35134) 2024-02-23T00:51:14.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.828+0000 7fddf93c9700 1 -- 172.21.15.47:0/231014530 learned_addr learned my addr 172.21.15.47:0/231014530 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:51:14.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.828+0000 7fddf9bca700 1 --2- 172.21.15.47:0/231014530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddf40fd510 0x7fddf41082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:14.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.829+0000 7fddf93c9700 1 -- 172.21.15.47:0/231014530 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fddf40fcbb0 msgr2=0x7fddf410e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:14.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.829+0000 7fddf93c9700 1 --2- 172.21.15.47:0/231014530 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fddf40fcbb0 0x7fddf410e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:51:14.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.829+0000 7fddf93c9700 1 -- 172.21.15.47:0/231014530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddf40fd510 msgr2=0x7fddf41082c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:14.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.829+0000 7fddf93c9700 1 --2- 172.21.15.47:0/231014530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddf40fd510 0x7fddf41082c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:14.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.829+0000 7fddf93c9700 1 -- 172.21.15.47:0/231014530 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf0005040 con 0x7fddf40f7400 2024-02-23T00:51:14.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.829+0000 7fddf93c9700 1 --2- 172.21.15.47:0/231014530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddf40f7400 0x7fddf410e120 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 rx=0x7fddf0009da0 tx=0x7fddf0009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:14.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.829+0000 7fddea7fc700 10 client.?.objecter ms_handle_connect 0x7fddf40f7400 2024-02-23T00:51:14.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.829+0000 7fddea7fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:51:14.832 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:51:14.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.833+0000 7fddea7fc700 1 -- 172.21.15.47:0/231014530 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf000e030 con 0x7fddf40f7400 2024-02-23T00:51:14.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.833+0000 7fddea7fc700 1 -- 172.21.15.47:0/231014530 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fddf0013e70 con 0x7fddf40f7400 2024-02-23T00:51:14.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.833+0000 7fddea7fc700 1 -- 172.21.15.47:0/231014530 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf0018070 con 0x7fddf40f7400 2024-02-23T00:51:14.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.833+0000 7fddfb62d700 1 -- 172.21.15.47:0/231014530 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf4108d60 con 0x7fddf40f7400 2024-02-23T00:51:14.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.833+0000 7fddfb62d700 10 client.31314.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:51:14.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.833+0000 7fddfb62d700 1 -- 172.21.15.47:0/231014530 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddf4062700 con 0x7fddf40f7400 2024-02-23T00:51:14.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.833+0000 7fddfb62d700 1 librados: init done 2024-02-23T00:51:14.836 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:14.836 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:51:14.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.834+0000 7fddfb62d700 1 -- 172.21.15.47:0/231014530 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddf41091a0 con 0x7fddf40f7400 2024-02-23T00:51:14.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.838+0000 7fddea7fc700 1 -- 172.21.15.47:0/231014530 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fddf0007d50 con 0x7fddf40f7400 2024-02-23T00:51:14.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.838+0000 7fddea7fc700 1 --2- 172.21.15.47:0/231014530 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdde0062d10 0x7fdde00651d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:14.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.838+0000 7fddea7fc700 1 -- 172.21.15.47:0/231014530 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(545..545 src has 463..545) v4 ==== 9228+0+0 (secure 0 0 0) 0x7fddf007ee70 con 0x7fddf40f7400 2024-02-23T00:51:14.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.838+0000 7fddea7fc700 10 client.31314.objecter ms_dispatch 0x7fddf4001550 osd_map(545..545 src has 463..545) v4 2024-02-23T00:51:14.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.838+0000 7fddea7fc700 3 client.31314.objecter handle_osd_map got epochs [545,545] > 0 2024-02-23T00:51:14.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.838+0000 7fddea7fc700 3 client.31314.objecter handle_osd_map decoding full epoch 545 2024-02-23T00:51:14.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.838+0000 7fddea7fc700 20 client.31314.objecter dump_active .. 0 homeless 2024-02-23T00:51:14.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.839+0000 7fddf8bc8700 1 --2- 172.21.15.47:0/231014530 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdde0062d10 0x7fdde00651d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:14.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.839+0000 7fddea7fc700 10 client.31314.objecter ms_handle_connect 0x7fdde0062d10 2024-02-23T00:51:14.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.839+0000 7fddf8bc8700 1 --2- 172.21.15.47:0/231014530 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdde0062d10 0x7fdde00651d0 secure :-1 s=READY pgs=3115 cs=0 l=1 rev1=1 rx=0x7fddf4109a60 tx=0x7fdde400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:14.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:14.842+0000 7fddea7fc700 1 -- 172.21.15.47:0/231014530 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fddf004e460 con 0x7fddf40f7400 2024-02-23T00:51:14.937 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:51:14.964 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:14.982 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:51:14.985 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:14.985 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:51:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:15.026+0000 7fddfb62d700 1 -- 172.21.15.47:0/231014530 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7fddf405fa60 con 0x7fddf40f7400 2024-02-23T00:51:15.047 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:14.487274+0000 osd.0 (osd.0) 231 : cluster [DBG] 5.1e scrub starts 2024-02-23T00:51:15.047 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:14.488197+0000 osd.0 (osd.0) 232 : cluster [DBG] 5.1e scrub ok 2024-02-23T00:51:15.047 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:13.261975+0000 osd.7 (osd.7) 271 : cluster [DBG] 5.1a scrub starts 2024-02-23T00:51:15.047 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:13.264864+0000 osd.7 (osd.7) 272 : cluster [DBG] 5.1a scrub ok 2024-02-23T00:51:15.047 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:13.351278+0000 osd.3 (osd.3) 262 : cluster [DBG] 18.1 scrub starts 2024-02-23T00:51:15.048 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:13.352605+0000 osd.3 (osd.3) 263 : cluster [DBG] 18.1 scrub ok 2024-02-23T00:51:15.048 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:13.372150+0000 osd.1 (osd.1) 235 : cluster [DBG] 2.9 scrub starts 2024-02-23T00:51:15.048 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:13.375033+0000 osd.1 (osd.1) 236 : cluster [DBG] 2.9 scrub ok 2024-02-23T00:51:15.048 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:14.286329+0000 mon.a (mon.0) 2937 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-90", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:51:15.048 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:14.286396+0000 mon.a (mon.0) 2938 : cluster [DBG] osdmap e545: 8 total, 8 up, 7 in 2024-02-23T00:51:15.048 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:14.287099+0000 mon.a (mon.0) 2939 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-90"}]: dispatch 2024-02-23T00:51:15.048 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:14.487274+0000 osd.0 (osd.0) 231 : cluster [DBG] 5.1e scrub starts 2024-02-23T00:51:15.048 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:14.488197+0000 osd.0 (osd.0) 232 : cluster [DBG] 5.1e scrub ok 2024-02-23T00:51:15.086 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:51:15.112 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:15.131 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:51:15.133 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:15.134 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:51:15.234 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:51:15.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:15.240+0000 7fddea7fc700 1 -- 172.21.15.47:0/231014530 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v546) v1 ==== 87+0+0 (secure 0 0 0) 0x7fddf0051a80 con 0x7fddf40f7400 2024-02-23T00:51:15.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:15.244+0000 7fddfb62d700 10 librados: watch_flush enter 2024-02-23T00:51:15.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:15.244+0000 7fddfb62d700 10 librados: watch_flush exit 2024-02-23T00:51:15.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:15.244+0000 7fddfb62d700 20 client.31314.objecter shutdown clearing up homeless session... 2024-02-23T00:51:15.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:15.244+0000 7fddfb62d700 10 client.31314.objecter successfully canceled tick 2024-02-23T00:51:15.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:15.244+0000 7fddfb62d700 1 -- 172.21.15.47:0/231014530 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdde0062d10 msgr2=0x7fdde00651d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:15.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:15.244+0000 7fddfb62d700 1 --2- 172.21.15.47:0/231014530 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdde0062d10 0x7fdde00651d0 secure :-1 s=READY pgs=3115 cs=0 l=1 rev1=1 rx=0x7fddf4109a60 tx=0x7fdde400b040).stop 2024-02-23T00:51:15.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:15.244+0000 7fddfb62d700 1 -- 172.21.15.47:0/231014530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddf40f7400 msgr2=0x7fddf410e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:15.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:15.244+0000 7fddfb62d700 1 --2- 172.21.15.47:0/231014530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddf40f7400 0x7fddf410e120 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 rx=0x7fddf0009da0 tx=0x7fddf0009cb0).stop 2024-02-23T00:51:15.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:15.244+0000 7fddfb62d700 1 -- 172.21.15.47:0/231014530 shutdown_connections 2024-02-23T00:51:15.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:15.244+0000 7fddfb62d700 1 --2- 172.21.15.47:0/231014530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddf40fd510 0x7fddf41082c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:15.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:15.244+0000 7fddfb62d700 1 --2- 172.21.15.47:0/231014530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddf40f7400 0x7fddf410e120 unknown :-1 s=CLOSED pgs=2570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:15.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:15.244+0000 7fddfb62d700 1 --2- 172.21.15.47:0/231014530 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdde0062d10 0x7fdde00651d0 secure :-1 s=CLOSED pgs=3115 cs=0 l=1 rev1=1 rx=0x7fddf4109a60 tx=0x7fdde400b040).stop 2024-02-23T00:51:15.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:15.244+0000 7fddfb62d700 1 --2- 172.21.15.47:0/231014530 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fddf40fcbb0 0x7fddf410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:51:15.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:15.244+0000 7fddfb62d700 1 -- 172.21.15.47:0/231014530 >> 172.21.15.47:0/231014530 conn(0x7fddf40f2820 msgr2=0x7fddf40f4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:15.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:15.245+0000 7fddfb62d700 1 -- 172.21.15.47:0/231014530 shutdown_connections 2024-02-23T00:51:15.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:15.245+0000 7fddfb62d700 1 -- 172.21.15.47:0/231014530 wait complete. 2024-02-23T00:51:15.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:15.245+0000 7fddfb62d700 1 librados: shutdown 2024-02-23T00:51:15.248 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is set 2024-02-23T00:51:15.263 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:15.284 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:51:15.286 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:15.287 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:51:15.388 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:51:15.419 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:15.517 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:51:15.520 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:15.521 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:51:15.622 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:51:15.650 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:15.723 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:51:15.725 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:15.726 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:51:15.827 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:51:15.856 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:15.977 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:14.309958+0000 osd.3 (osd.3) 264 : cluster [DBG] 62.18 deep-scrub starts 2024-02-23T00:51:15.978 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:14.315398+0000 osd.3 (osd.3) 265 : cluster [DBG] 62.18 deep-scrub ok 2024-02-23T00:51:15.978 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:14.656977+0000 osd.5 (osd.5) 246 : cluster [DBG] 2.12 scrub starts 2024-02-23T00:51:15.978 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:14.658288+0000 osd.5 (osd.5) 247 : cluster [DBG] 2.12 scrub ok 2024-02-23T00:51:15.978 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:15.027140+0000 mon.a (mon.0) 2940 : audit [INF] from='client.? 172.21.15.47:0/231014530' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:51:15.978 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:15.235786+0000 mon.a (mon.0) 2941 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:51:15.978 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:15.240205+0000 mon.a (mon.0) 2942 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-90"}]': finished 2024-02-23T00:51:15.978 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:15.240274+0000 mon.a (mon.0) 2943 : audit [INF] from='client.? 172.21.15.47:0/231014530' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:51:15.978 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:15.240323+0000 mon.a (mon.0) 2944 : cluster [DBG] osdmap e546: 8 total, 8 up, 7 in 2024-02-23T00:51:15.978 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:15.246409+0000 mon.a (mon.0) 2945 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-90", "mode": "writeback"}]: dispatch 2024-02-23T00:51:15.978 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:15.339648+0000 mgr.y (mgr.14517) 499 : cluster [DBG] pgmap v823: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:15.979 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:14.309958+0000 osd.3 (osd.3) 264 : cluster [DBG] 62.18 deep-scrub starts 2024-02-23T00:51:15.979 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:14.315398+0000 osd.3 (osd.3) 265 : cluster [DBG] 62.18 deep-scrub ok 2024-02-23T00:51:15.979 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:14.656977+0000 osd.5 (osd.5) 246 : cluster [DBG] 2.12 scrub starts 2024-02-23T00:51:15.979 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:14.658288+0000 osd.5 (osd.5) 247 : cluster [DBG] 2.12 scrub ok 2024-02-23T00:51:15.979 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:15.027140+0000 mon.a (mon.0) 2940 : audit [INF] from='client.? 172.21.15.47:0/231014530' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:51:15.979 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:15.235786+0000 mon.a (mon.0) 2941 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:51:15.979 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:15.240205+0000 mon.a (mon.0) 2942 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-90"}]': finished 2024-02-23T00:51:15.979 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:15.240274+0000 mon.a (mon.0) 2943 : audit [INF] from='client.? 172.21.15.47:0/231014530' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:51:15.979 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:15.240323+0000 mon.a (mon.0) 2944 : cluster [DBG] osdmap e546: 8 total, 8 up, 7 in 2024-02-23T00:51:15.979 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:15.246409+0000 mon.a (mon.0) 2945 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-90", "mode": "writeback"}]: dispatch 2024-02-23T00:51:15.979 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:15.339648+0000 mgr.y (mgr.14517) 499 : cluster [DBG] pgmap v823: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:15.980 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:51:15.983 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:15.983 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:51:16.084 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:51:16.111 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:14.309958+0000 osd.3 (osd.3) 264 : cluster [DBG] 62.18 deep-scrub starts 2024-02-23T00:51:16.111 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:14.315398+0000 osd.3 (osd.3) 265 : cluster [DBG] 62.18 deep-scrub ok 2024-02-23T00:51:16.111 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:14.656977+0000 osd.5 (osd.5) 246 : cluster [DBG] 2.12 scrub starts 2024-02-23T00:51:16.111 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:14.658288+0000 osd.5 (osd.5) 247 : cluster [DBG] 2.12 scrub ok 2024-02-23T00:51:16.111 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:15.027140+0000 mon.a (mon.0) 2940 : audit [INF] from='client.? 172.21.15.47:0/231014530' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:51:16.112 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:15.235786+0000 mon.a (mon.0) 2941 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:51:16.112 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:15.240205+0000 mon.a (mon.0) 2942 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-6", "overlaypool": "test-rados-api-smithi047-82465-90"}]': finished 2024-02-23T00:51:16.112 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:15.240274+0000 mon.a (mon.0) 2943 : audit [INF] from='client.? 172.21.15.47:0/231014530' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:51:16.112 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:15.240323+0000 mon.a (mon.0) 2944 : cluster [DBG] osdmap e546: 8 total, 8 up, 7 in 2024-02-23T00:51:16.112 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:15.246409+0000 mon.a (mon.0) 2945 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-90", "mode": "writeback"}]: dispatch 2024-02-23T00:51:16.112 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:15.339648+0000 mgr.y (mgr.14517) 499 : cluster [DBG] pgmap v823: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:16.113 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:16.132 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:51:16.135 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:16.136 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:51:16.239 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:51:16.272 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:16.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.320+0000 7fe68aed7700 1 -- 172.21.15.47:0/2176570664 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe6840f7400 msgr2=0x7fe684103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:16.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.320+0000 7fe68aed7700 1 --2- 172.21.15.47:0/2176570664 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe6840f7400 0x7fe684103a20 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 rx=0x7fe67c009be0 tx=0x7fe67c004d10).stop 2024-02-23T00:51:16.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.320+0000 7fe68aed7700 1 -- 172.21.15.47:0/2176570664 shutdown_connections 2024-02-23T00:51:16.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.320+0000 7fe68aed7700 1 --2- 172.21.15.47:0/2176570664 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe6840f7400 0x7fe684103a20 unknown :-1 s=CLOSED pgs=2728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:16.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.320+0000 7fe68aed7700 1 --2- 172.21.15.47:0/2176570664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6840fcbb0 0x7fe6840fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:16.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.320+0000 7fe68aed7700 1 --2- 172.21.15.47:0/2176570664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6840fd510 0x7fe6840f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:16.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.320+0000 7fe68aed7700 1 -- 172.21.15.47:0/2176570664 >> 172.21.15.47:0/2176570664 conn(0x7fe6840f2820 msgr2=0x7fe6840f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:16.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.320+0000 7fe68aed7700 1 -- 172.21.15.47:0/2176570664 shutdown_connections 2024-02-23T00:51:16.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.320+0000 7fe68aed7700 1 -- 172.21.15.47:0/2176570664 wait complete. 2024-02-23T00:51:16.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.321+0000 7fe68aed7700 1 librados: starting msgr at 2024-02-23T00:51:16.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.321+0000 7fe68aed7700 1 librados: starting objecter 2024-02-23T00:51:16.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.321+0000 7fe68aed7700 1 Processor -- start 2024-02-23T00:51:16.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.321+0000 7fe68aed7700 1 -- start start 2024-02-23T00:51:16.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.321+0000 7fe68aed7700 1 librados: setting wanted keys 2024-02-23T00:51:16.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.321+0000 7fe68aed7700 1 librados: calling monclient init 2024-02-23T00:51:16.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.321+0000 7fe68aed7700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6840f7400 0x7fe68410cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:16.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.321+0000 7fe68aed7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6840fcbb0 0x7fe68410d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:16.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.321+0000 7fe68aed7700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe6840fd510 0x7fe684106ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:16.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.321+0000 7fe68aed7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6841073c0 con 0x7fe6840fcbb0 2024-02-23T00:51:16.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.321+0000 7fe68aed7700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe684107530 con 0x7fe6840f7400 2024-02-23T00:51:16.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.321+0000 7fe68aed7700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6841076f0 con 0x7fe6840fd510 2024-02-23T00:51:16.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.321+0000 7fe689474700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe6840fd510 0x7fe684106ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:16.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.321+0000 7fe683fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6840fcbb0 0x7fe68410d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:16.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.321+0000 7fe689474700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe6840fd510 0x7fe684106ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:58976/0 (socket says 172.21.15.47:58976) 2024-02-23T00:51:16.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.321+0000 7fe689474700 1 -- 172.21.15.47:0/731791069 learned_addr learned my addr 172.21.15.47:0/731791069 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:51:16.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.321+0000 7fe688c73700 1 --2- 172.21.15.47:0/731791069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6840f7400 0x7fe68410cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:16.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.322+0000 7fe683fff700 1 -- 172.21.15.47:0/731791069 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe6840fd510 msgr2=0x7fe684106ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:16.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.322+0000 7fe683fff700 1 --2- 172.21.15.47:0/731791069 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe6840fd510 0x7fe684106ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:16.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.322+0000 7fe683fff700 1 -- 172.21.15.47:0/731791069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6840f7400 msgr2=0x7fe68410cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:16.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.322+0000 7fe683fff700 1 --2- 172.21.15.47:0/731791069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6840f7400 0x7fe68410cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:16.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.322+0000 7fe683fff700 1 -- 172.21.15.47:0/731791069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66c005040 con 0x7fe6840fcbb0 2024-02-23T00:51:16.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.322+0000 7fe689474700 1 --2- 172.21.15.47:0/731791069 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe6840fd510 0x7fe684106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:51:16.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.322+0000 7fe683fff700 1 --2- 172.21.15.47:0/731791069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6840fcbb0 0x7fe68410d220 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 rx=0x7fe66c004920 tx=0x7fe66c009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:16.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.322+0000 7fe681ffb700 10 client.?.objecter ms_handle_connect 0x7fe6840fcbb0 2024-02-23T00:51:16.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.322+0000 7fe681ffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:51:16.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.322+0000 7fe681ffb700 1 -- 172.21.15.47:0/731791069 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c00e030 con 0x7fe6840fcbb0 2024-02-23T00:51:16.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.322+0000 7fe681ffb700 1 -- 172.21.15.47:0/731791069 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fe66c00f040 con 0x7fe6840fcbb0 2024-02-23T00:51:16.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.322+0000 7fe68aed7700 1 -- 172.21.15.47:0/731791069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe67c0097c0 con 0x7fe6840fcbb0 2024-02-23T00:51:16.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.322+0000 7fe681ffb700 1 -- 172.21.15.47:0/731791069 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c014720 con 0x7fe6840fcbb0 2024-02-23T00:51:16.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.322+0000 7fe68aed7700 10 client.31320.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:51:16.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.322+0000 7fe68aed7700 1 -- 172.21.15.47:0/731791069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe684107d30 con 0x7fe6840fcbb0 2024-02-23T00:51:16.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.322+0000 7fe68aed7700 1 librados: init done 2024-02-23T00:51:16.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.323+0000 7fe68aed7700 1 -- 172.21.15.47:0/731791069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe68405fa60 con 0x7fe6840fcbb0 2024-02-23T00:51:16.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.324+0000 7fe681ffb700 1 -- 172.21.15.47:0/731791069 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fe66c010500 con 0x7fe6840fcbb0 2024-02-23T00:51:16.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.325+0000 7fe681ffb700 1 --2- 172.21.15.47:0/731791069 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe670062cc0 0x7fe670065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:16.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.325+0000 7fe681ffb700 1 -- 172.21.15.47:0/731791069 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(547..547 src has 463..547) v4 ==== 9228+0+0 (secure 0 0 0) 0x7fe66c07d6c0 con 0x7fe6840fcbb0 2024-02-23T00:51:16.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.325+0000 7fe681ffb700 10 client.31320.objecter ms_dispatch 0x7fe684001550 osd_map(547..547 src has 463..547) v4 2024-02-23T00:51:16.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.325+0000 7fe681ffb700 3 client.31320.objecter handle_osd_map got epochs [547,547] > 0 2024-02-23T00:51:16.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.325+0000 7fe681ffb700 3 client.31320.objecter handle_osd_map decoding full epoch 547 2024-02-23T00:51:16.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.325+0000 7fe681ffb700 20 client.31320.objecter dump_active .. 0 homeless 2024-02-23T00:51:16.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.325+0000 7fe688c73700 1 --2- 172.21.15.47:0/731791069 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe670062cc0 0x7fe670065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:16.329 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:51:16.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.325+0000 7fe681ffb700 10 client.31320.objecter ms_handle_connect 0x7fe670062cc0 2024-02-23T00:51:16.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.325+0000 7fe688c73700 1 --2- 172.21.15.47:0/731791069 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe670062cc0 0x7fe670065180 secure :-1 s=READY pgs=3116 cs=0 l=1 rev1=1 rx=0x7fe674004de0 tx=0x7fe67400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:16.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.330+0000 7fe681ffb700 1 -- 172.21.15.47:0/731791069 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe66c04ccd0 con 0x7fe6840fcbb0 2024-02-23T00:51:16.333 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:16.333 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:51:16.434 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:51:16.465 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:16.485 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:51:16.487 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:16.487 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:51:16.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.498+0000 7fe68aed7700 1 -- 172.21.15.47:0/731791069 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe6841083b0 con 0x7fe670062cc0 2024-02-23T00:51:16.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.512+0000 7fe681ffb700 1 -- 172.21.15.47:0/731791069 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+527568 (secure 0 0 0) 0x7fe6841083b0 con 0x7fe670062cc0 2024-02-23T00:51:16.521 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:51:16.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.516+0000 7fe68aed7700 10 librados: watch_flush enter 2024-02-23T00:51:16.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.516+0000 7fe68aed7700 10 librados: watch_flush exit 2024-02-23T00:51:16.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.516+0000 7fe68aed7700 20 client.31320.objecter shutdown clearing up homeless session... 2024-02-23T00:51:16.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.516+0000 7fe68aed7700 10 client.31320.objecter successfully canceled tick 2024-02-23T00:51:16.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.516+0000 7fe68aed7700 1 -- 172.21.15.47:0/731791069 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe670062cc0 msgr2=0x7fe670065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:16.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.516+0000 7fe68aed7700 1 --2- 172.21.15.47:0/731791069 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe670062cc0 0x7fe670065180 secure :-1 s=READY pgs=3116 cs=0 l=1 rev1=1 rx=0x7fe674004de0 tx=0x7fe67400b040).stop 2024-02-23T00:51:16.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.516+0000 7fe68aed7700 1 -- 172.21.15.47:0/731791069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6840fcbb0 msgr2=0x7fe68410d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:16.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.516+0000 7fe68aed7700 1 --2- 172.21.15.47:0/731791069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6840fcbb0 0x7fe68410d220 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 rx=0x7fe66c004920 tx=0x7fe66c009cb0).stop 2024-02-23T00:51:16.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.516+0000 7fe68aed7700 1 -- 172.21.15.47:0/731791069 shutdown_connections 2024-02-23T00:51:16.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.516+0000 7fe68aed7700 1 --2- 172.21.15.47:0/731791069 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe670062cc0 0x7fe670065180 unknown :-1 s=CLOSED pgs=3116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:16.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.516+0000 7fe68aed7700 1 --2- 172.21.15.47:0/731791069 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe6840fd510 0x7fe684106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:16.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.516+0000 7fe68aed7700 1 --2- 172.21.15.47:0/731791069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6840f7400 0x7fe68410cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:16.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.516+0000 7fe68aed7700 1 --2- 172.21.15.47:0/731791069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6840fcbb0 0x7fe68410d220 unknown :-1 s=CLOSED pgs=2571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:16.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.516+0000 7fe68aed7700 1 -- 172.21.15.47:0/731791069 >> 172.21.15.47:0/731791069 conn(0x7fe6840f2820 msgr2=0x7fe6840f4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:16.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.516+0000 7fe68aed7700 1 -- 172.21.15.47:0/731791069 shutdown_connections 2024-02-23T00:51:16.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.516+0000 7fe68aed7700 1 -- 172.21.15.47:0/731791069 wait complete. 2024-02-23T00:51:16.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:16.516+0000 7fe68aed7700 1 librados: shutdown 2024-02-23T00:51:16.527 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-02-23T00:51:16.588 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:51:16.618 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:16.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:15.406876+0000 osd.6 (osd.6) 215 : cluster [DBG] 4.1f scrub starts 2024-02-23T00:51:16.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:15.408778+0000 osd.6 (osd.6) 216 : cluster [DBG] 4.1f scrub ok 2024-02-23T00:51:16.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:16.243800+0000 mon.a (mon.0) 2946 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:51:16.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:16.251326+0000 mon.a (mon.0) 2947 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-90", "mode": "writeback"}]': finished 2024-02-23T00:51:16.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:16.251386+0000 mon.a (mon.0) 2948 : cluster [DBG] osdmap e547: 8 total, 8 up, 7 in 2024-02-23T00:51:16.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:16.253916+0000 mon.a (mon.0) 2949 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-90","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:51:16.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:16.498795+0000 mgr.y (mgr.14517) 500 : audit [DBG] from='client.31320 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:51:16.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:15.406876+0000 osd.6 (osd.6) 215 : cluster [DBG] 4.1f scrub starts 2024-02-23T00:51:16.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:15.408778+0000 osd.6 (osd.6) 216 : cluster [DBG] 4.1f scrub ok 2024-02-23T00:51:16.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:16.243800+0000 mon.a (mon.0) 2946 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:51:16.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:16.251326+0000 mon.a (mon.0) 2947 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-90", "mode": "writeback"}]': finished 2024-02-23T00:51:16.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:16.251386+0000 mon.a (mon.0) 2948 : cluster [DBG] osdmap e547: 8 total, 8 up, 7 in 2024-02-23T00:51:16.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:16.253916+0000 mon.a (mon.0) 2949 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-90","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:51:16.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:16.498795+0000 mgr.y (mgr.14517) 500 : audit [DBG] from='client.31320 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:51:17.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:15.406876+0000 osd.6 (osd.6) 215 : cluster [DBG] 4.1f scrub starts 2024-02-23T00:51:17.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:15.408778+0000 osd.6 (osd.6) 216 : cluster [DBG] 4.1f scrub ok 2024-02-23T00:51:17.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:16.243800+0000 mon.a (mon.0) 2946 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:51:17.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:16.251326+0000 mon.a (mon.0) 2947 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-90", "mode": "writeback"}]': finished 2024-02-23T00:51:17.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:16.251386+0000 mon.a (mon.0) 2948 : cluster [DBG] osdmap e547: 8 total, 8 up, 7 in 2024-02-23T00:51:17.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:16.253916+0000 mon.a (mon.0) 2949 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-90","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:51:17.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:16.498795+0000 mgr.y (mgr.14517) 500 : audit [DBG] from='client.31320 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:51:17.765 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:51:17.769 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:51:17.776 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:17.777 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:51:17.786 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":824,"stamp":"2024-02-23T00:51:16.248391+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4026236,"num_objects":417,"num_object_clones":11,"num_object_copies":1251,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":417,"num_whiteouts":0,"num_read":2455,"num_read_kb":2020,"num_write":1690,"num_write_kb":4667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":440,"num_bytes_recovered":3479,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":21,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3128,"ondisk_log_size":3128,"up":528,"acting":528,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":586,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":2175664,"kb_used_data":29560,"kb_used_omap":0,"kb_used_meta":2146048,"kb_avail":654040400,"statfs":{"total":671965249536,"available":669737369600,"internally_reserved":0,"allocated":30269440,"data_stored":18220368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2197553152},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.970538"},"pg_stats":[{"pgid":"227.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:12.275803+0000","last_change":"2024-02-23T00:51:12.275803+0000","last_active":"2024-02-23T00:51:12.275803+0000","last_peered":"2024-02-23T00:51:12.275803+0000","last_clean":"2024-02-23T00:51:12.275803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:12.275803+0000","last_undegraded":"2024-02-23T00:51:12.275803+0000","last_fullsized":"2024-02-23T00:51:12.275803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.275803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"227.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:12.275803+0000","last_change":"2024-02-23T00:51:12.275803+0000","last_active":"2024-02-23T00:51:12.275803+0000","last_peered":"2024-02-23T00:51:12.275803+0000","last_clean":"2024-02-23T00:51:12.275803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:12.275803+0000","last_undegraded":"2024-02-23T00:51:12.275803+0000","last_fullsized":"2024-02-23T00:51:12.275803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.275803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"227.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:12.275803+0000","last_change":"2024-02-23T00:51:12.275803+0000","last_active":"2024-02-23T00:51:12.275803+0000","last_peered":"2024-02-23T00:51:12.275803+0000","last_clean":"2024-02-23T00:51:12.275803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:12.275803+0000","last_undegraded":"2024-02-23T00:51:12.275803+0000","last_fullsized":"2024-02-23T00:51:12.275803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.275803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"227.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:12.275803+0000","last_change":"2024-02-23T00:51:12.275803+0000","last_active":"2024-02-23T00:51:12.275803+0000","last_peered":"2024-02-23T00:51:12.275803+0000","last_clean":"2024-02-23T00:51:12.275803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:12.275803+0000","last_undegraded":"2024-02-23T00:51:12.275803+0000","last_fullsized":"2024-02-23T00:51:12.275803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.275803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"227.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:12.275803+0000","last_change":"2024-02-23T00:51:12.275803+0000","last_active":"2024-02-23T00:51:12.275803+0000","last_peered":"2024-02-23T00:51:12.275803+0000","last_clean":"2024-02-23T00:51:12.275803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:12.275803+0000","last_undegraded":"2024-02-23T00:51:12.275803+0000","last_fullsized":"2024-02-23T00:51:12.275803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.275803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"227.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:12.275803+0000","last_change":"2024-02-23T00:51:12.275803+0000","last_active":"2024-02-23T00:51:12.275803+0000","last_peered":"2024-02-23T00:51:12.275803+0000","last_clean":"2024-02-23T00:51:12.275803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:12.275803+0000","last_undegraded":"2024-02-23T00:51:12.275803+0000","last_fullsized":"2024-02-23T00:51:12.275803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.275803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"227.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:12.275803+0000","last_change":"2024-02-23T00:51:12.275803+0000","last_active":"2024-02-23T00:51:12.275803+0000","last_peered":"2024-02-23T00:51:12.275803+0000","last_clean":"2024-02-23T00:51:12.275803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:12.275803+0000","last_undegraded":"2024-02-23T00:51:12.275803+0000","last_fullsized":"2024-02-23T00:51:12.275803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.275803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"227.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:12.275803+0000","last_change":"2024-02-23T00:51:12.275803+0000","last_active":"2024-02-23T00:51:12.275803+0000","last_peered":"2024-02-23T00:51:12.275803+0000","last_clean":"2024-02-23T00:51:12.275803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:12.275803+0000","last_undegraded":"2024-02-23T00:51:12.275803+0000","last_fullsized":"2024-02-23T00:51:12.275803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.275803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"227.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:12.275803+0000","last_change":"2024-02-23T00:51:12.275803+0000","last_active":"2024-02-23T00:51:12.275803+0000","last_peered":"2024-02-23T00:51:12.275803+0000","last_clean":"2024-02-23T00:51:12.275803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:12.275803+0000","last_undegraded":"2024-02-23T00:51:12.275803+0000","last_fullsized":"2024-02-23T00:51:12.275803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.275803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"227.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:12.275803+0000","last_change":"2024-02-23T00:51:12.275803+0000","last_active":"2024-02-23T00:51:12.275803+0000","last_peered":"2024-02-23T00:51:12.275803+0000","last_clean":"2024-02-23T00:51:12.275803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:12.275803+0000","last_undegraded":"2024-02-23T00:51:12.275803+0000","last_fullsized":"2024-02-23T00:51:12.275803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.275803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"227.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:12.275803+0000","last_change":"2024-02-23T00:51:12.275803+0000","last_active":"2024-02-23T00:51:12.275803+0000","last_peered":"2024-02-23T00:51:12.275803+0000","last_clean":"2024-02-23T00:51:12.275803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:12.275803+0000","last_undegraded":"2024-02-23T00:51:12.275803+0000","last_fullsized":"2024-02-23T00:51:12.275803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.275803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"227.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:12.275803+0000","last_change":"2024-02-23T00:51:12.275803+0000","last_active":"2024-02-23T00:51:12.275803+0000","last_peered":"2024-02-23T00:51:12.275803+0000","last_clean":"2024-02-23T00:51:12.275803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:12.275803+0000","last_undegraded":"2024-02-23T00:51:12.275803+0000","last_fullsized":"2024-02-23T00:51:12.275803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.275803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"227.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:12.275803+0000","last_change":"2024-02-23T00:51:12.275803+0000","last_active":"2024-02-23T00:51:12.275803+0000","last_peered":"2024-02-23T00:51:12.275803+0000","last_clean":"2024-02-23T00:51:12.275803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:12.275803+0000","last_undegraded":"2024-02-23T00:51:12.275803+0000","last_fullsized":"2024-02-23T00:51:12.275803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.275803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"227.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:12.275803+0000","last_change":"2024-02-23T00:51:12.275803+0000","last_active":"2024-02-23T00:51:12.275803+0000","last_peered":"2024-02-23T00:51:12.275803+0000","last_clean":"2024-02-23T00:51:12.275803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:12.275803+0000","last_undegraded":"2024-02-23T00:51:12.275803+0000","last_fullsized":"2024-02-23T00:51:12.275803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.275803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"227.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:12.275803+0000","last_change":"2024-02-23T00:51:12.275803+0000","last_active":"2024-02-23T00:51:12.275803+0000","last_peered":"2024-02-23T00:51:12.275803+0000","last_clean":"2024-02-23T00:51:12.275803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:12.275803+0000","last_undegraded":"2024-02-23T00:51:12.275803+0000","last_fullsized":"2024-02-23T00:51:12.275803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.275803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"227.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:12.275803+0000","last_change":"2024-02-23T00:51:12.275803+0000","last_active":"2024-02-23T00:51:12.275803+0000","last_peered":"2024-02-23T00:51:12.275803+0000","last_clean":"2024-02-23T00:51:12.275803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:12.275803+0000","last_undegraded":"2024-02-23T00:51:12.275803+0000","last_fullsized":"2024-02-23T00:51:12.275803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.275803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"227.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:12.275803+0000","last_change":"2024-02-23T00:51:12.275803+0000","last_active":"2024-02-23T00:51:12.275803+0000","last_peered":"2024-02-23T00:51:12.275803+0000","last_clean":"2024-02-23T00:51:12.275803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:12.275803+0000","last_undegraded":"2024-02-23T00:51:12.275803+0000","last_fullsized":"2024-02-23T00:51:12.275803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.275803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"227.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:12.275803+0000","last_change":"2024-02-23T00:51:12.275803+0000","last_active":"2024-02-23T00:51:12.275803+0000","last_peered":"2024-02-23T00:51:12.275803+0000","last_clean":"2024-02-23T00:51:12.275803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:12.275803+0000","last_undegraded":"2024-02-23T00:51:12.275803+0000","last_fullsized":"2024-02-23T00:51:12.275803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.275803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"227.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:12.275803+0000","last_change":"2024-02-23T00:51:12.275803+0000","last_active":"2024-02-23T00:51:12.275803+0000","last_peered":"2024-02-23T00:51:12.275803+0000","last_clean":"2024-02-23T00:51:12.275803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:12.275803+0000","last_undegraded":"2024-02-23T00:51:12.275803+0000","last_fullsized":"2024-02-23T00:51:12.275803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.275803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"227.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:12.275803+0000","last_change":"2024-02-23T00:51:12.275803+0000","last_active":"2024-02-23T00:51:12.275803+0000","last_peered":"2024-02-23T00:51:12.275803+0000","last_clean":"2024-02-23T00:51:12.275803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:12.275803+0000","last_undegraded":"2024-02-23T00:51:12.275803+0000","last_fullsized":"2024-02-23T00:51:12.275803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.275803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"227.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:12.275803+0000","last_change":"2024-02-23T00:51:12.275803+0000","last_active":"2024-02-23T00:51:12.275803+0000","last_peered":"2024-02-23T00:51:12.275803+0000","last_clean":"2024-02-23T00:51:12.275803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:12.275803+0000","last_undegraded":"2024-02-23T00:51:12.275803+0000","last_fullsized":"2024-02-23T00:51:12.275803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.275803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"227.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:12.275803+0000","last_change":"2024-02-23T00:51:12.275803+0000","last_active":"2024-02-23T00:51:12.275803+0000","last_peered":"2024-02-23T00:51:12.275803+0000","last_clean":"2024-02-23T00:51:12.275803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:12.275803+0000","last_undegraded":"2024-02-23T00:51:12.275803+0000","last_fullsized":"2024-02-23T00:51:12.275803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.275803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"227.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:12.275803+0000","last_change":"2024-02-23T00:51:12.275803+0000","last_active":"2024-02-23T00:51:12.275803+0000","last_peered":"2024-02-23T00:51:12.275803+0000","last_clean":"2024-02-23T00:51:12.275803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:12.275803+0000","last_undegraded":"2024-02-23T00:51:12.275803+0000","last_fullsized":"2024-02-23T00:51:12.275803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.275803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"227.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:12.275803+0000","last_change":"2024-02-23T00:51:12.275803+0000","last_active":"2024-02-23T00:51:12.275803+0000","last_peered":"2024-02-23T00:51:12.275803+0000","last_clean":"2024-02-23T00:51:12.275803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:12.275803+0000","last_undegraded":"2024-02-23T00:51:12.275803+0000","last_fullsized":"2024-02-23T00:51:12.275803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.275803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"227.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:12.275803+0000","last_change":"2024-02-23T00:51:12.275803+0000","last_active":"2024-02-23T00:51:12.275803+0000","last_peered":"2024-02-23T00:51:12.275803+0000","last_clean":"2024-02-23T00:51:12.275803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:12.275803+0000","last_undegraded":"2024-02-23T00:51:12.275803+0000","last_fullsized":"2024-02-23T00:51:12.275803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.275803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"227.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:12.275803+0000","last_change":"2024-02-23T00:51:12.275803+0000","last_active":"2024-02-23T00:51:12.275803+0000","last_peered":"2024-02-23T00:51:12.275803+0000","last_clean":"2024-02-23T00:51:12.275803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:12.275803+0000","last_undegraded":"2024-02-23T00:51:12.275803+0000","last_fullsized":"2024-02-23T00:51:12.275803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.275803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"227.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:12.275803+0000","last_change":"2024-02-23T00:51:12.275803+0000","last_active":"2024-02-23T00:51:12.275803+0000","last_peered":"2024-02-23T00:51:12.275803+0000","last_clean":"2024-02-23T00:51:12.275803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:12.275803+0000","last_undegraded":"2024-02-23T00:51:12.275803+0000","last_fullsized":"2024-02-23T00:51:12.275803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.275803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"227.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:12.275803+0000","last_change":"2024-02-23T00:51:12.275803+0000","last_active":"2024-02-23T00:51:12.275803+0000","last_peered":"2024-02-23T00:51:12.275803+0000","last_clean":"2024-02-23T00:51:12.275803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:12.275803+0000","last_undegraded":"2024-02-23T00:51:12.275803+0000","last_fullsized":"2024-02-23T00:51:12.275803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.275803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"227.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:12.275803+0000","last_change":"2024-02-23T00:51:12.275803+0000","last_active":"2024-02-23T00:51:12.275803+0000","last_peered":"2024-02-23T00:51:12.275803+0000","last_clean":"2024-02-23T00:51:12.275803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:12.275803+0000","last_undegraded":"2024-02-23T00:51:12.275803+0000","last_fullsized":"2024-02-23T00:51:12.275803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.275803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"227.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:12.275803+0000","last_change":"2024-02-23T00:51:12.275803+0000","last_active":"2024-02-23T00:51:12.275803+0000","last_peered":"2024-02-23T00:51:12.275803+0000","last_clean":"2024-02-23T00:51:12.275803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:12.275803+0000","last_undegraded":"2024-02-23T00:51:12.275803+0000","last_fullsized":"2024-02-23T00:51:12.275803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.275803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"227.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:12.275803+0000","last_change":"2024-02-23T00:51:12.275803+0000","last_active":"2024-02-23T00:51:12.275803+0000","last_peered":"2024-02-23T00:51:12.275803+0000","last_clean":"2024-02-23T00:51:12.275803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:12.275803+0000","last_undegraded":"2024-02-23T00:51:12.275803+0000","last_fullsized":"2024-02-23T00:51:12.275803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.275803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"227.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:12.275803+0000","last_change":"2024-02-23T00:51:12.275803+0000","last_active":"2024-02-23T00:51:12.275803+0000","last_peered":"2024-02-23T00:51:12.275803+0000","last_clean":"2024-02-23T00:51:12.275803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:12.275803+0000","last_undegraded":"2024-02-23T00:51:12.275803+0000","last_fullsized":"2024-02-23T00:51:12.275803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.275803+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.275803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.0","version":"259'40","reported_seq":578,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.285368+0000","last_change":"2024-02-23T00:51:02.197067+0000","last_active":"2024-02-23T00:51:10.285368+0000","last_peered":"2024-02-23T00:51:10.285368+0000","last_clean":"2024-02-23T00:51:10.285368+0000","last_became_active":"2024-02-23T00:47:28.238034+0000","last_became_peered":"2024-02-23T00:47:28.238034+0000","last_unstale":"2024-02-23T00:51:10.285368+0000","last_undegraded":"2024-02-23T00:51:10.285368+0000","last_fullsized":"2024-02-23T00:51:10.285368+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'40","last_scrub_stamp":"2024-02-23T00:51:02.196968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:51:02.196968+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":31,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":1,"num_write":13,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":31,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.1","version":"495'89","reported_seq":572,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.282328+0000","last_change":"2024-02-23T00:50:56.478515+0000","last_active":"2024-02-23T00:51:10.282328+0000","last_peered":"2024-02-23T00:51:10.282328+0000","last_clean":"2024-02-23T00:51:10.282328+0000","last_became_active":"2024-02-23T00:43:17.851866+0000","last_became_peered":"2024-02-23T00:43:17.851866+0000","last_unstale":"2024-02-23T00:51:10.282328+0000","last_undegraded":"2024-02-23T00:51:10.282328+0000","last_fullsized":"2024-02-23T00:51:10.282328+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"495'89","last_scrub_stamp":"2024-02-23T00:50:56.478433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:56.478433+0000","log_size":89,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":63,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7,"num_read_kb":1,"num_write":30,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.2","version":"514'72","reported_seq":563,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.289401+0000","last_change":"2024-02-23T00:50:30.283941+0000","last_active":"2024-02-23T00:51:10.289401+0000","last_peered":"2024-02-23T00:51:10.289401+0000","last_clean":"2024-02-23T00:51:10.289401+0000","last_became_active":"2024-02-23T00:43:17.804792+0000","last_became_peered":"2024-02-23T00:43:17.804792+0000","last_unstale":"2024-02-23T00:51:10.289401+0000","last_undegraded":"2024-02-23T00:51:10.289401+0000","last_fullsized":"2024-02-23T00:51:10.289401+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'68","last_scrub_stamp":"2024-02-23T00:50:30.283820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:30.283820+0000","log_size":72,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1265,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":9,"num_read_kb":2,"num_write":27,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.3","version":"505'27","reported_seq":400,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.283246+0000","last_change":"2024-02-23T00:49:28.475170+0000","last_active":"2024-02-23T00:51:10.283246+0000","last_peered":"2024-02-23T00:51:10.283246+0000","last_clean":"2024-02-23T00:51:10.283246+0000","last_became_active":"2024-02-23T00:47:28.226936+0000","last_became_peered":"2024-02-23T00:47:28.226936+0000","last_unstale":"2024-02-23T00:51:10.283246+0000","last_undegraded":"2024-02-23T00:51:10.283246+0000","last_fullsized":"2024-02-23T00:51:10.283246+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"246'20","last_scrub_stamp":"2024-02-23T00:49:28.475056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:49:28.475056+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":4,"num_write":16,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":24,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.4","version":"514'24","reported_seq":517,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.296605+0000","last_change":"2024-02-23T00:48:56.703323+0000","last_active":"2024-02-23T00:51:10.296605+0000","last_peered":"2024-02-23T00:51:10.296605+0000","last_clean":"2024-02-23T00:51:10.296605+0000","last_became_active":"2024-02-23T00:43:17.810343+0000","last_became_peered":"2024-02-23T00:43:17.810343+0000","last_unstale":"2024-02-23T00:51:10.296605+0000","last_undegraded":"2024-02-23T00:51:10.296605+0000","last_fullsized":"2024-02-23T00:51:10.296605+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"207'17","last_scrub_stamp":"2024-02-23T00:48:56.703243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:48:56.703243+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":14,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.5","version":"495'55","reported_seq":547,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.294317+0000","last_change":"2024-02-23T00:49:26.431068+0000","last_active":"2024-02-23T00:51:10.294317+0000","last_peered":"2024-02-23T00:51:10.294317+0000","last_clean":"2024-02-23T00:51:10.294317+0000","last_became_active":"2024-02-23T00:43:17.855339+0000","last_became_peered":"2024-02-23T00:43:17.855339+0000","last_unstale":"2024-02-23T00:51:10.294317+0000","last_undegraded":"2024-02-23T00:51:10.294317+0000","last_fullsized":"2024-02-23T00:51:10.294317+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'51","last_scrub_stamp":"2024-02-23T00:49:26.430990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:49:26.430990+0000","log_size":55,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":2,"num_write":21,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.6","version":"514'55","reported_seq":603,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.286274+0000","last_change":"2024-02-23T00:50:21.294652+0000","last_active":"2024-02-23T00:51:10.286274+0000","last_peered":"2024-02-23T00:51:10.286274+0000","last_clean":"2024-02-23T00:51:10.286274+0000","last_became_active":"2024-02-23T00:47:28.236471+0000","last_became_peered":"2024-02-23T00:47:28.236471+0000","last_unstale":"2024-02-23T00:51:10.286274+0000","last_undegraded":"2024-02-23T00:51:10.286274+0000","last_fullsized":"2024-02-23T00:51:10.286274+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"379'54","last_scrub_stamp":"2024-02-23T00:50:21.294457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:21.294457+0000","log_size":55,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1272,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":7,"num_read_kb":1,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":41,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.7","version":"514'77","reported_seq":586,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.290207+0000","last_change":"2024-02-23T00:50:28.305136+0000","last_active":"2024-02-23T00:51:10.290207+0000","last_peered":"2024-02-23T00:51:10.290207+0000","last_clean":"2024-02-23T00:51:10.290207+0000","last_became_active":"2024-02-23T00:47:28.240920+0000","last_became_peered":"2024-02-23T00:47:28.240920+0000","last_unstale":"2024-02-23T00:51:10.290207+0000","last_undegraded":"2024-02-23T00:51:10.290207+0000","last_fullsized":"2024-02-23T00:51:10.290207+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"499'76","last_scrub_stamp":"2024-02-23T00:50:28.305026+0000","last_deep_scrub":"499'76","last_deep_scrub_stamp":"2024-02-23T00:50:28.305026+0000","last_clean_scrub_stamp":"2024-02-23T00:50:28.305026+0000","log_size":77,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":573,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":26,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":52,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.8","version":"514'134","reported_seq":682,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.285152+0000","last_change":"2024-02-23T00:48:53.698549+0000","last_active":"2024-02-23T00:51:10.285152+0000","last_peered":"2024-02-23T00:51:10.285152+0000","last_clean":"2024-02-23T00:51:10.285152+0000","last_became_active":"2024-02-23T00:47:28.229780+0000","last_became_peered":"2024-02-23T00:47:28.229780+0000","last_unstale":"2024-02-23T00:51:10.285152+0000","last_undegraded":"2024-02-23T00:51:10.285152+0000","last_fullsized":"2024-02-23T00:51:10.285152+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"440'114","last_scrub_stamp":"2024-02-23T00:48:53.698466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:48:53.698466+0000","log_size":134,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":816,"num_objects":14,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":9,"num_read_kb":3,"num_write":51,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":68,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":4,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.9","version":"529'69","reported_seq":603,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.293589+0000","last_change":"2024-02-23T00:49:00.243151+0000","last_active":"2024-02-23T00:51:10.293589+0000","last_peered":"2024-02-23T00:51:10.293589+0000","last_clean":"2024-02-23T00:51:10.293589+0000","last_became_active":"2024-02-23T00:47:28.202025+0000","last_became_peered":"2024-02-23T00:47:28.202025+0000","last_unstale":"2024-02-23T00:51:10.293589+0000","last_undegraded":"2024-02-23T00:51:10.293589+0000","last_fullsized":"2024-02-23T00:51:10.293589+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'39","last_scrub_stamp":"2024-02-23T00:49:00.243068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:49:00.243068+0000","log_size":69,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1285,"num_objects":10,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":29,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":27,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":6,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.a","version":"505'92","reported_seq":647,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.285590+0000","last_change":"2024-02-23T00:50:30.147956+0000","last_active":"2024-02-23T00:51:10.285590+0000","last_peered":"2024-02-23T00:51:10.285590+0000","last_clean":"2024-02-23T00:51:10.285590+0000","last_became_active":"2024-02-23T00:47:28.241187+0000","last_became_peered":"2024-02-23T00:47:28.241187+0000","last_unstale":"2024-02-23T00:51:10.285590+0000","last_undegraded":"2024-02-23T00:51:10.285590+0000","last_fullsized":"2024-02-23T00:51:10.285590+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'91","last_scrub_stamp":"2024-02-23T00:50:30.147873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:30.147873+0000","log_size":92,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1283,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":30,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":68,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.b","version":"514'91","reported_seq":580,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.285601+0000","last_change":"2024-02-23T00:50:59.132747+0000","last_active":"2024-02-23T00:51:10.285601+0000","last_peered":"2024-02-23T00:51:10.285601+0000","last_clean":"2024-02-23T00:51:10.285601+0000","last_became_active":"2024-02-23T00:43:17.855321+0000","last_became_peered":"2024-02-23T00:43:17.855321+0000","last_unstale":"2024-02-23T00:51:10.285601+0000","last_undegraded":"2024-02-23T00:51:10.285601+0000","last_fullsized":"2024-02-23T00:51:10.285601+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"514'91","last_scrub_stamp":"2024-02-23T00:50:59.132670+0000","last_deep_scrub":"418'84","last_deep_scrub_stamp":"2024-02-23T00:48:10.926499+0000","last_clean_scrub_stamp":"2024-02-23T00:50:59.132670+0000","log_size":91,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":621,"num_objects":8,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":13,"num_read_kb":6,"num_write":35,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.c","version":"495'43","reported_seq":539,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.278001+0000","last_change":"2024-02-23T00:49:06.040399+0000","last_active":"2024-02-23T00:51:10.278001+0000","last_peered":"2024-02-23T00:51:10.278001+0000","last_clean":"2024-02-23T00:51:10.278001+0000","last_became_active":"2024-02-23T00:43:17.829044+0000","last_became_peered":"2024-02-23T00:43:17.829044+0000","last_unstale":"2024-02-23T00:51:10.278001+0000","last_undegraded":"2024-02-23T00:51:10.278001+0000","last_fullsized":"2024-02-23T00:51:10.278001+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"447'39","last_scrub_stamp":"2024-02-23T00:49:06.040326+0000","last_deep_scrub":"447'39","last_deep_scrub_stamp":"2024-02-23T00:49:06.040326+0000","last_clean_scrub_stamp":"2024-02-23T00:49:06.040326+0000","log_size":43,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":3,"num_write":18,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.d","version":"414'58","reported_seq":592,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.294318+0000","last_change":"2024-02-23T00:50:28.632286+0000","last_active":"2024-02-23T00:51:10.294318+0000","last_peered":"2024-02-23T00:51:10.294318+0000","last_clean":"2024-02-23T00:51:10.294318+0000","last_became_active":"2024-02-23T00:47:28.202691+0000","last_became_peered":"2024-02-23T00:47:28.202691+0000","last_unstale":"2024-02-23T00:51:10.294318+0000","last_undegraded":"2024-02-23T00:51:10.294318+0000","last_fullsized":"2024-02-23T00:51:10.294318+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"414'58","last_scrub_stamp":"2024-02-23T00:50:28.632146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:28.632146+0000","log_size":58,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":18,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":33,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.e","version":"514'60","reported_seq":552,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.297116+0000","last_change":"2024-02-23T00:50:31.600892+0000","last_active":"2024-02-23T00:51:10.297116+0000","last_peered":"2024-02-23T00:51:10.297116+0000","last_clean":"2024-02-23T00:51:10.297116+0000","last_became_active":"2024-02-23T00:43:17.845956+0000","last_became_peered":"2024-02-23T00:43:17.845956+0000","last_unstale":"2024-02-23T00:51:10.297116+0000","last_undegraded":"2024-02-23T00:51:10.297116+0000","last_fullsized":"2024-02-23T00:51:10.297116+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"495'58","last_scrub_stamp":"2024-02-23T00:50:31.600819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:31.600819+0000","log_size":60,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":43,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":24,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.f","version":"496'138","reported_seq":807,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.294947+0000","last_change":"2024-02-23T00:49:03.264065+0000","last_active":"2024-02-23T00:51:10.294947+0000","last_peered":"2024-02-23T00:51:10.294947+0000","last_clean":"2024-02-23T00:51:10.294947+0000","last_became_active":"2024-02-23T00:47:47.183032+0000","last_became_peered":"2024-02-23T00:47:47.183032+0000","last_unstale":"2024-02-23T00:51:10.294947+0000","last_undegraded":"2024-02-23T00:51:10.294947+0000","last_fullsized":"2024-02-23T00:51:10.294947+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"241'136","last_scrub_stamp":"2024-02-23T00:49:03.263943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:49:03.263943+0000","log_size":138,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":88,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":8,"num_read_kb":1,"num_write":43,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.10","version":"514'76","reported_seq":559,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.282205+0000","last_change":"2024-02-23T00:50:27.628865+0000","last_active":"2024-02-23T00:51:10.282205+0000","last_peered":"2024-02-23T00:51:10.282205+0000","last_clean":"2024-02-23T00:51:10.282205+0000","last_became_active":"2024-02-23T00:43:17.819865+0000","last_became_peered":"2024-02-23T00:43:17.819865+0000","last_unstale":"2024-02-23T00:51:10.282205+0000","last_undegraded":"2024-02-23T00:51:10.282205+0000","last_fullsized":"2024-02-23T00:51:10.282205+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"496'75","last_scrub_stamp":"2024-02-23T00:50:27.628796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:27.628796+0000","log_size":76,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1389,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":3,"num_write":32,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.11","version":"347'53","reported_seq":533,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.282332+0000","last_change":"2024-02-23T00:50:22.723895+0000","last_active":"2024-02-23T00:51:10.282332+0000","last_peered":"2024-02-23T00:51:10.282332+0000","last_clean":"2024-02-23T00:51:10.282332+0000","last_became_active":"2024-02-23T00:43:17.852697+0000","last_became_peered":"2024-02-23T00:43:17.852697+0000","last_unstale":"2024-02-23T00:51:10.282332+0000","last_undegraded":"2024-02-23T00:51:10.282332+0000","last_fullsized":"2024-02-23T00:51:10.282332+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'53","last_scrub_stamp":"2024-02-23T00:50:22.723817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:22.723817+0000","log_size":53,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":49,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":17,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.12","version":"240'73","reported_seq":468,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.508469+0000","last_change":"2024-02-23T00:51:10.508444+0000","last_active":"2024-02-23T00:51:10.508469+0000","last_peered":"2024-02-23T00:51:10.508469+0000","last_clean":"2024-02-23T00:51:10.508469+0000","last_became_active":"2024-02-23T00:47:28.213692+0000","last_became_peered":"2024-02-23T00:47:28.213692+0000","last_unstale":"2024-02-23T00:51:10.508469+0000","last_undegraded":"2024-02-23T00:51:10.508469+0000","last_fullsized":"2024-02-23T00:51:10.508469+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'73","last_scrub_stamp":"2024-02-23T00:51:10.508378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:51:10.508378+0000","log_size":73,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":49,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":49,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.13","version":"540'68","reported_seq":571,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.296095+0000","last_change":"2024-02-23T00:51:09.477358+0000","last_active":"2024-02-23T00:51:10.296095+0000","last_peered":"2024-02-23T00:51:10.296095+0000","last_clean":"2024-02-23T00:51:10.296095+0000","last_became_active":"2024-02-23T00:43:17.857823+0000","last_became_peered":"2024-02-23T00:43:17.857823+0000","last_unstale":"2024-02-23T00:51:10.296095+0000","last_undegraded":"2024-02-23T00:51:10.296095+0000","last_fullsized":"2024-02-23T00:51:10.296095+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"540'68","last_scrub_stamp":"2024-02-23T00:51:09.477292+0000","last_deep_scrub":"540'68","last_deep_scrub_stamp":"2024-02-23T00:51:09.477292+0000","last_clean_scrub_stamp":"2024-02-23T00:51:09.477292+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":45,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":8,"num_read_kb":4,"num_write":24,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.14","version":"514'62","reported_seq":604,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.285748+0000","last_change":"2024-02-23T00:50:34.190105+0000","last_active":"2024-02-23T00:51:10.285748+0000","last_peered":"2024-02-23T00:51:10.285748+0000","last_clean":"2024-02-23T00:51:10.285748+0000","last_became_active":"2024-02-23T00:47:28.235465+0000","last_became_peered":"2024-02-23T00:47:28.235465+0000","last_unstale":"2024-02-23T00:51:10.285748+0000","last_undegraded":"2024-02-23T00:51:10.285748+0000","last_fullsized":"2024-02-23T00:51:10.285748+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"496'61","last_scrub_stamp":"2024-02-23T00:50:34.190011+0000","last_deep_scrub":"420'54","last_deep_scrub_stamp":"2024-02-23T00:48:12.892464+0000","last_clean_scrub_stamp":"2024-02-23T00:50:34.190011+0000","log_size":62,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2369,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":13,"num_read_kb":5,"num_write":31,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":51,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.15","version":"505'22","reported_seq":542,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.278900+0000","last_change":"2024-02-23T00:49:33.948127+0000","last_active":"2024-02-23T00:51:10.278900+0000","last_peered":"2024-02-23T00:51:10.278900+0000","last_clean":"2024-02-23T00:51:10.278900+0000","last_became_active":"2024-02-23T00:47:28.219548+0000","last_became_peered":"2024-02-23T00:47:28.219548+0000","last_unstale":"2024-02-23T00:51:10.278900+0000","last_undegraded":"2024-02-23T00:51:10.278900+0000","last_fullsized":"2024-02-23T00:51:10.278900+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'19","last_scrub_stamp":"2024-02-23T00:49:33.948049+0000","last_deep_scrub":"299'18","last_deep_scrub_stamp":"2024-02-23T00:46:11.830580+0000","last_clean_scrub_stamp":"2024-02-23T00:49:33.948049+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3465,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":12,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.16","version":"407'30","reported_seq":567,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.294577+0000","last_change":"2024-02-23T00:51:07.565330+0000","last_active":"2024-02-23T00:51:10.294577+0000","last_peered":"2024-02-23T00:51:10.294577+0000","last_clean":"2024-02-23T00:51:10.294577+0000","last_became_active":"2024-02-23T00:47:28.199870+0000","last_became_peered":"2024-02-23T00:47:28.199870+0000","last_unstale":"2024-02-23T00:51:10.294577+0000","last_undegraded":"2024-02-23T00:51:10.294577+0000","last_fullsized":"2024-02-23T00:51:10.294577+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"407'30","last_scrub_stamp":"2024-02-23T00:51:07.565245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:51:07.565245+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":64,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":14,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.17","version":"375'63","reported_seq":562,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.290422+0000","last_change":"2024-02-23T00:49:42.321041+0000","last_active":"2024-02-23T00:51:10.290422+0000","last_peered":"2024-02-23T00:51:10.290422+0000","last_clean":"2024-02-23T00:51:10.290422+0000","last_became_active":"2024-02-23T00:43:17.843525+0000","last_became_peered":"2024-02-23T00:43:17.843525+0000","last_unstale":"2024-02-23T00:51:10.290422+0000","last_undegraded":"2024-02-23T00:51:10.290422+0000","last_fullsized":"2024-02-23T00:51:10.290422+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'63","last_scrub_stamp":"2024-02-23T00:49:42.320947+0000","last_deep_scrub":"375'63","last_deep_scrub_stamp":"2024-02-23T00:47:27.229920+0000","last_clean_scrub_stamp":"2024-02-23T00:49:42.320947+0000","log_size":63,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":20,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.19","version":"494'98","reported_seq":663,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.289236+0000","last_change":"2024-02-23T00:50:20.275389+0000","last_active":"2024-02-23T00:51:10.289236+0000","last_peered":"2024-02-23T00:51:10.289236+0000","last_clean":"2024-02-23T00:51:10.289236+0000","last_became_active":"2024-02-23T00:47:28.210660+0000","last_became_peered":"2024-02-23T00:47:28.210660+0000","last_unstale":"2024-02-23T00:51:10.289236+0000","last_undegraded":"2024-02-23T00:51:10.289236+0000","last_fullsized":"2024-02-23T00:51:10.289236+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'96","last_scrub_stamp":"2024-02-23T00:50:20.275315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:20.275315+0000","log_size":98,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":73,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":9,"num_read_kb":1,"num_write":33,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":73,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,7],"acting":[6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.18","version":"523'15","reported_seq":534,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.278073+0000","last_change":"2024-02-23T00:49:09.073073+0000","last_active":"2024-02-23T00:51:10.278073+0000","last_peered":"2024-02-23T00:51:10.278073+0000","last_clean":"2024-02-23T00:51:10.278073+0000","last_became_active":"2024-02-23T00:47:28.221906+0000","last_became_peered":"2024-02-23T00:47:28.221906+0000","last_unstale":"2024-02-23T00:51:10.278073+0000","last_undegraded":"2024-02-23T00:51:10.278073+0000","last_fullsized":"2024-02-23T00:51:10.278073+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"410'12","last_scrub_stamp":"2024-02-23T00:49:09.073000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:49:09.073000+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":15,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":1,"num_write":11,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":5,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1b","version":"347'70","reported_seq":555,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.294763+0000","last_change":"2024-02-23T00:49:28.481771+0000","last_active":"2024-02-23T00:51:10.294763+0000","last_peered":"2024-02-23T00:51:10.294763+0000","last_clean":"2024-02-23T00:51:10.294763+0000","last_became_active":"2024-02-23T00:43:17.803043+0000","last_became_peered":"2024-02-23T00:43:17.803043+0000","last_unstale":"2024-02-23T00:51:10.294763+0000","last_undegraded":"2024-02-23T00:51:10.294763+0000","last_fullsized":"2024-02-23T00:51:10.294763+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'70","last_scrub_stamp":"2024-02-23T00:49:28.481700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:49:28.481700+0000","log_size":70,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4000060,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":22,"num_write_kb":3928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.1a","version":"347'35","reported_seq":559,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.289374+0000","last_change":"2024-02-23T00:49:46.307053+0000","last_active":"2024-02-23T00:51:10.289374+0000","last_peered":"2024-02-23T00:51:10.289374+0000","last_clean":"2024-02-23T00:51:10.289374+0000","last_became_active":"2024-02-23T00:45:44.178195+0000","last_became_peered":"2024-02-23T00:45:44.178195+0000","last_unstale":"2024-02-23T00:51:10.289374+0000","last_undegraded":"2024-02-23T00:51:10.289374+0000","last_fullsized":"2024-02-23T00:51:10.289374+0000","mapping_epoch":271,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":272,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'35","last_scrub_stamp":"2024-02-23T00:49:46.306975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.306975+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":30,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":11,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1d","version":"539'107","reported_seq":597,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.279476+0000","last_change":"2024-02-23T00:50:58.794356+0000","last_active":"2024-02-23T00:51:10.279476+0000","last_peered":"2024-02-23T00:51:10.279476+0000","last_clean":"2024-02-23T00:51:10.279476+0000","last_became_active":"2024-02-23T00:43:17.828194+0000","last_became_peered":"2024-02-23T00:43:17.828194+0000","last_unstale":"2024-02-23T00:51:10.279476+0000","last_undegraded":"2024-02-23T00:51:10.279476+0000","last_fullsized":"2024-02-23T00:51:10.279476+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"505'105","last_scrub_stamp":"2024-02-23T00:50:58.794271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:58.794271+0000","log_size":107,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2384,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":8,"num_read_kb":0,"num_write":35,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"62.1c","version":"514'70","reported_seq":563,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.289829+0000","last_change":"2024-02-23T00:49:45.261438+0000","last_active":"2024-02-23T00:51:10.289829+0000","last_peered":"2024-02-23T00:51:10.289829+0000","last_clean":"2024-02-23T00:51:10.289829+0000","last_became_active":"2024-02-23T00:43:17.804718+0000","last_became_peered":"2024-02-23T00:43:17.804718+0000","last_unstale":"2024-02-23T00:51:10.289829+0000","last_undegraded":"2024-02-23T00:51:10.289829+0000","last_fullsized":"2024-02-23T00:51:10.289829+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"446'61","last_scrub_stamp":"2024-02-23T00:49:45.261352+0000","last_deep_scrub":"374'55","last_deep_scrub_stamp":"2024-02-23T00:47:26.267291+0000","last_clean_scrub_stamp":"2024-02-23T00:49:45.261352+0000","log_size":70,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1594,"num_objects":7,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":15,"num_read_kb":8,"num_write":31,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1f","version":"464'55","reported_seq":565,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.286340+0000","last_change":"2024-02-23T00:50:58.157230+0000","last_active":"2024-02-23T00:51:10.286340+0000","last_peered":"2024-02-23T00:51:10.286340+0000","last_clean":"2024-02-23T00:51:10.286340+0000","last_became_active":"2024-02-23T00:47:28.243574+0000","last_became_peered":"2024-02-23T00:47:28.243574+0000","last_unstale":"2024-02-23T00:51:10.286340+0000","last_undegraded":"2024-02-23T00:51:10.286340+0000","last_fullsized":"2024-02-23T00:51:10.286340+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"464'55","last_scrub_stamp":"2024-02-23T00:50:58.157140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:58.157140+0000","log_size":55,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":73,"num_objects":9,"num_object_clones":2,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":18,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":35,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":4,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.1e","version":"505'62","reported_seq":596,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.286272+0000","last_change":"2024-02-23T00:51:07.232814+0000","last_active":"2024-02-23T00:51:10.286272+0000","last_peered":"2024-02-23T00:51:10.286272+0000","last_clean":"2024-02-23T00:51:10.286272+0000","last_became_active":"2024-02-23T00:47:28.244073+0000","last_became_peered":"2024-02-23T00:47:28.244073+0000","last_unstale":"2024-02-23T00:51:10.286272+0000","last_undegraded":"2024-02-23T00:51:10.286272+0000","last_fullsized":"2024-02-23T00:51:10.286272+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"505'62","last_scrub_stamp":"2024-02-23T00:51:07.232724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:51:07.232724+0000","log_size":62,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1248,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":11,"num_read_kb":4,"num_write":29,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":33,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"466'9","reported_seq":664,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.285502+0000","last_change":"2024-02-23T00:49:43.188066+0000","last_active":"2024-02-23T00:51:10.285502+0000","last_peered":"2024-02-23T00:51:10.285502+0000","last_clean":"2024-02-23T00:51:10.285502+0000","last_became_active":"2024-02-23T00:39:18.793414+0000","last_became_peered":"2024-02-23T00:39:18.793414+0000","last_unstale":"2024-02-23T00:51:10.285502+0000","last_undegraded":"2024-02-23T00:51:10.285502+0000","last_fullsized":"2024-02-23T00:51:10.285502+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'9","last_scrub_stamp":"2024-02-23T00:49:43.187989+0000","last_deep_scrub":"466'9","last_deep_scrub_stamp":"2024-02-23T00:49:43.187989+0000","last_clean_scrub_stamp":"2024-02-23T00:49:43.187989+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"470'42","reported_seq":588,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.278664+0000","last_change":"2024-02-23T00:49:47.653272+0000","last_active":"2024-02-23T00:51:10.278664+0000","last_peered":"2024-02-23T00:51:10.278664+0000","last_clean":"2024-02-23T00:51:10.278664+0000","last_became_active":"2024-02-23T00:39:16.859401+0000","last_became_peered":"2024-02-23T00:39:16.859401+0000","last_unstale":"2024-02-23T00:51:10.278664+0000","last_undegraded":"2024-02-23T00:51:10.278664+0000","last_fullsized":"2024-02-23T00:51:10.278664+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"470'42","last_scrub_stamp":"2024-02-23T00:49:47.653176+0000","last_deep_scrub":"470'42","last_deep_scrub_stamp":"2024-02-23T00:49:47.653176+0000","last_clean_scrub_stamp":"2024-02-23T00:49:47.653176+0000","log_size":42,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":183,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.278839+0000","last_change":"2024-02-23T00:49:25.955780+0000","last_active":"2024-02-23T00:51:10.278839+0000","last_peered":"2024-02-23T00:51:10.278839+0000","last_clean":"2024-02-23T00:51:10.278839+0000","last_became_active":"2024-02-23T00:47:28.220876+0000","last_became_peered":"2024-02-23T00:47:28.220876+0000","last_unstale":"2024-02-23T00:51:10.278839+0000","last_undegraded":"2024-02-23T00:51:10.278839+0000","last_fullsized":"2024-02-23T00:51:10.278839+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:25.955711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:04.330426+0000","last_clean_scrub_stamp":"2024-02-23T00:49:25.955711+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":526,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.279079+0000","last_change":"2024-02-23T00:50:14.801990+0000","last_active":"2024-02-23T00:51:10.279079+0000","last_peered":"2024-02-23T00:51:10.279079+0000","last_clean":"2024-02-23T00:51:10.279079+0000","last_became_active":"2024-02-23T00:43:32.282545+0000","last_became_peered":"2024-02-23T00:43:32.282545+0000","last_unstale":"2024-02-23T00:51:10.279079+0000","last_undegraded":"2024-02-23T00:51:10.279079+0000","last_fullsized":"2024-02-23T00:51:10.279079+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:14.801915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:50:14.801915+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":455,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.279686+0000","last_change":"2024-02-23T00:50:23.727290+0000","last_active":"2024-02-23T00:51:10.279686+0000","last_peered":"2024-02-23T00:51:10.279686+0000","last_clean":"2024-02-23T00:51:10.279686+0000","last_became_active":"2024-02-23T00:42:58.828846+0000","last_became_peered":"2024-02-23T00:42:58.828846+0000","last_unstale":"2024-02-23T00:51:10.279686+0000","last_undegraded":"2024-02-23T00:51:10.279686+0000","last_fullsized":"2024-02-23T00:51:10.279686+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:23.727173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:23.727173+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"457'21","reported_seq":553,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.296448+0000","last_change":"2024-02-23T00:49:27.765115+0000","last_active":"2024-02-23T00:51:10.296448+0000","last_peered":"2024-02-23T00:51:10.296448+0000","last_clean":"2024-02-23T00:51:10.296448+0000","last_became_active":"2024-02-23T00:39:16.783701+0000","last_became_peered":"2024-02-23T00:39:16.783701+0000","last_unstale":"2024-02-23T00:51:10.296448+0000","last_undegraded":"2024-02-23T00:51:10.296448+0000","last_fullsized":"2024-02-23T00:51:10.296448+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-23T00:49:27.765048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:27.765048+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":505,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.283705+0000","last_change":"2024-02-23T00:49:04.528117+0000","last_active":"2024-02-23T00:51:10.283705+0000","last_peered":"2024-02-23T00:51:10.283705+0000","last_clean":"2024-02-23T00:51:10.283705+0000","last_became_active":"2024-02-23T00:39:14.772493+0000","last_became_peered":"2024-02-23T00:39:14.772493+0000","last_unstale":"2024-02-23T00:51:10.283705+0000","last_undegraded":"2024-02-23T00:51:10.283705+0000","last_fullsized":"2024-02-23T00:51:10.283705+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:04.528006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:04.528006+0000","last_clean_scrub_stamp":"2024-02-23T00:49:04.528006+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":181,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.294028+0000","last_change":"2024-02-23T00:49:30.499854+0000","last_active":"2024-02-23T00:51:10.294028+0000","last_peered":"2024-02-23T00:51:10.294028+0000","last_clean":"2024-02-23T00:51:10.294028+0000","last_became_active":"2024-02-23T00:47:28.231567+0000","last_became_peered":"2024-02-23T00:47:28.231567+0000","last_unstale":"2024-02-23T00:51:10.294028+0000","last_undegraded":"2024-02-23T00:51:10.294028+0000","last_fullsized":"2024-02-23T00:51:10.294028+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:30.499769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:08.335330+0000","last_clean_scrub_stamp":"2024-02-23T00:49:30.499769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":536,"reported_epoch":542,"state":"active+clean","last_fresh":"2024-02-23T00:51:11.526671+0000","last_change":"2024-02-23T00:51:11.526671+0000","last_active":"2024-02-23T00:51:11.526671+0000","last_peered":"2024-02-23T00:51:11.526671+0000","last_clean":"2024-02-23T00:51:11.526671+0000","last_became_active":"2024-02-23T00:47:28.214157+0000","last_became_peered":"2024-02-23T00:47:28.214157+0000","last_unstale":"2024-02-23T00:51:11.526671+0000","last_undegraded":"2024-02-23T00:51:11.526671+0000","last_fullsized":"2024-02-23T00:51:11.526671+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:11.526584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:13.867716+0000","last_clean_scrub_stamp":"2024-02-23T00:51:11.526584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":456,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.297202+0000","last_change":"2024-02-23T00:50:32.622097+0000","last_active":"2024-02-23T00:51:10.297202+0000","last_peered":"2024-02-23T00:51:10.297202+0000","last_clean":"2024-02-23T00:51:10.297202+0000","last_became_active":"2024-02-23T00:42:58.867305+0000","last_became_peered":"2024-02-23T00:42:58.867305+0000","last_unstale":"2024-02-23T00:51:10.297202+0000","last_undegraded":"2024-02-23T00:51:10.297202+0000","last_fullsized":"2024-02-23T00:51:10.297202+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:32.622026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:32.622026+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"457'51","reported_seq":590,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.290073+0000","last_change":"2024-02-23T00:48:42.409822+0000","last_active":"2024-02-23T00:51:10.290073+0000","last_peered":"2024-02-23T00:51:10.290073+0000","last_clean":"2024-02-23T00:51:10.290073+0000","last_became_active":"2024-02-23T00:39:16.857413+0000","last_became_peered":"2024-02-23T00:39:16.857413+0000","last_unstale":"2024-02-23T00:51:10.290073+0000","last_undegraded":"2024-02-23T00:51:10.290073+0000","last_fullsized":"2024-02-23T00:51:10.290073+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'35","last_scrub_stamp":"2024-02-23T00:48:42.409686+0000","last_deep_scrub":"88'35","last_deep_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_clean_scrub_stamp":"2024-02-23T00:48:42.409686+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":68,"num_read_kb":56,"num_write":42,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":187,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.279342+0000","last_change":"2024-02-23T00:50:57.770551+0000","last_active":"2024-02-23T00:51:10.279342+0000","last_peered":"2024-02-23T00:51:10.279342+0000","last_clean":"2024-02-23T00:51:10.279342+0000","last_became_active":"2024-02-23T00:47:28.231891+0000","last_became_peered":"2024-02-23T00:47:28.231891+0000","last_unstale":"2024-02-23T00:51:10.279342+0000","last_undegraded":"2024-02-23T00:51:10.279342+0000","last_fullsized":"2024-02-23T00:51:10.279342+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:57.770450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_clean_scrub_stamp":"2024-02-23T00:50:57.770450+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":183,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.296333+0000","last_change":"2024-02-23T00:49:03.682811+0000","last_active":"2024-02-23T00:51:10.296333+0000","last_peered":"2024-02-23T00:51:10.296333+0000","last_clean":"2024-02-23T00:51:10.296333+0000","last_became_active":"2024-02-23T00:47:28.220578+0000","last_became_peered":"2024-02-23T00:47:28.220578+0000","last_unstale":"2024-02-23T00:51:10.296333+0000","last_undegraded":"2024-02-23T00:51:10.296333+0000","last_fullsized":"2024-02-23T00:51:10.296333+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:03.682745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:03.682745+0000","last_clean_scrub_stamp":"2024-02-23T00:49:03.682745+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":534,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.285319+0000","last_change":"2024-02-23T00:49:42.150972+0000","last_active":"2024-02-23T00:51:10.285319+0000","last_peered":"2024-02-23T00:51:10.285319+0000","last_clean":"2024-02-23T00:51:10.285319+0000","last_became_active":"2024-02-23T00:47:28.215636+0000","last_became_peered":"2024-02-23T00:47:28.215636+0000","last_unstale":"2024-02-23T00:51:10.285319+0000","last_undegraded":"2024-02-23T00:51:10.285319+0000","last_fullsized":"2024-02-23T00:51:10.285319+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:42.150875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:42.150875+0000","last_clean_scrub_stamp":"2024-02-23T00:49:42.150875+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":447,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.283341+0000","last_change":"2024-02-23T00:49:34.515593+0000","last_active":"2024-02-23T00:51:10.283341+0000","last_peered":"2024-02-23T00:51:10.283341+0000","last_clean":"2024-02-23T00:51:10.283341+0000","last_became_active":"2024-02-23T00:42:57.639013+0000","last_became_peered":"2024-02-23T00:42:57.639013+0000","last_unstale":"2024-02-23T00:51:10.283341+0000","last_undegraded":"2024-02-23T00:51:10.283341+0000","last_fullsized":"2024-02-23T00:51:10.283341+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:34.515496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:49:34.515496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"457'31","reported_seq":609,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.294577+0000","last_change":"2024-02-23T00:49:31.480372+0000","last_active":"2024-02-23T00:51:10.294577+0000","last_peered":"2024-02-23T00:51:10.294577+0000","last_clean":"2024-02-23T00:51:10.294577+0000","last_became_active":"2024-02-23T00:47:28.200653+0000","last_became_peered":"2024-02-23T00:47:28.200653+0000","last_unstale":"2024-02-23T00:51:10.294577+0000","last_undegraded":"2024-02-23T00:51:10.294577+0000","last_fullsized":"2024-02-23T00:51:10.294577+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:49:31.480253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:31.480253+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":186,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.285433+0000","last_change":"2024-02-23T00:51:10.213423+0000","last_active":"2024-02-23T00:51:10.285433+0000","last_peered":"2024-02-23T00:51:10.285433+0000","last_clean":"2024-02-23T00:51:10.285433+0000","last_became_active":"2024-02-23T00:47:28.201926+0000","last_became_peered":"2024-02-23T00:47:28.201926+0000","last_unstale":"2024-02-23T00:51:10.285433+0000","last_undegraded":"2024-02-23T00:51:10.285433+0000","last_fullsized":"2024-02-23T00:51:10.285433+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:10.213337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:10.213337+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":506,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.283543+0000","last_change":"2024-02-23T00:51:09.343951+0000","last_active":"2024-02-23T00:51:10.283543+0000","last_peered":"2024-02-23T00:51:10.283543+0000","last_clean":"2024-02-23T00:51:10.283543+0000","last_became_active":"2024-02-23T00:39:18.799875+0000","last_became_peered":"2024-02-23T00:39:18.799875+0000","last_unstale":"2024-02-23T00:51:10.283543+0000","last_undegraded":"2024-02-23T00:51:10.283543+0000","last_fullsized":"2024-02-23T00:51:10.283543+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:09.343884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:09.343884+0000","last_clean_scrub_stamp":"2024-02-23T00:51:09.343884+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":447,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.283344+0000","last_change":"2024-02-23T00:48:49.610457+0000","last_active":"2024-02-23T00:51:10.283344+0000","last_peered":"2024-02-23T00:51:10.283344+0000","last_clean":"2024-02-23T00:51:10.283344+0000","last_became_active":"2024-02-23T00:42:57.675306+0000","last_became_peered":"2024-02-23T00:42:57.675306+0000","last_unstale":"2024-02-23T00:51:10.283344+0000","last_undegraded":"2024-02-23T00:51:10.283344+0000","last_fullsized":"2024-02-23T00:51:10.283344+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:49.610391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:49.610391+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"457'20","reported_seq":541,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.294185+0000","last_change":"2024-02-23T00:51:01.488450+0000","last_active":"2024-02-23T00:51:10.294185+0000","last_peered":"2024-02-23T00:51:10.294185+0000","last_clean":"2024-02-23T00:51:10.294185+0000","last_became_active":"2024-02-23T00:39:16.777530+0000","last_became_peered":"2024-02-23T00:39:16.777530+0000","last_unstale":"2024-02-23T00:51:10.294185+0000","last_undegraded":"2024-02-23T00:51:10.294185+0000","last_fullsized":"2024-02-23T00:51:10.294185+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'20","last_scrub_stamp":"2024-02-23T00:51:01.488371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:01.488371+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":505,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.282740+0000","last_change":"2024-02-23T00:48:41.719867+0000","last_active":"2024-02-23T00:51:10.282740+0000","last_peered":"2024-02-23T00:51:10.282740+0000","last_clean":"2024-02-23T00:51:10.282740+0000","last_became_active":"2024-02-23T00:39:14.780379+0000","last_became_peered":"2024-02-23T00:39:14.780379+0000","last_unstale":"2024-02-23T00:51:10.282740+0000","last_undegraded":"2024-02-23T00:51:10.282740+0000","last_fullsized":"2024-02-23T00:51:10.282740+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:41.719772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.575242+0000","last_clean_scrub_stamp":"2024-02-23T00:48:41.719772+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"376'9","reported_seq":251,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.286746+0000","last_change":"2024-02-23T00:49:40.180695+0000","last_active":"2024-02-23T00:51:10.286746+0000","last_peered":"2024-02-23T00:51:10.286746+0000","last_clean":"2024-02-23T00:51:10.286746+0000","last_became_active":"2024-02-23T00:47:28.231104+0000","last_became_peered":"2024-02-23T00:47:28.231104+0000","last_unstale":"2024-02-23T00:51:10.286746+0000","last_undegraded":"2024-02-23T00:51:10.286746+0000","last_fullsized":"2024-02-23T00:51:10.286746+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"376'9","last_scrub_stamp":"2024-02-23T00:49:40.180616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:40.180616+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":452,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.276367+0000","last_change":"2024-02-23T00:48:51.104378+0000","last_active":"2024-02-23T00:51:10.276367+0000","last_peered":"2024-02-23T00:51:10.276367+0000","last_clean":"2024-02-23T00:51:10.276367+0000","last_became_active":"2024-02-23T00:42:58.765106+0000","last_became_peered":"2024-02-23T00:42:58.765106+0000","last_unstale":"2024-02-23T00:51:10.276367+0000","last_undegraded":"2024-02-23T00:51:10.276367+0000","last_fullsized":"2024-02-23T00:51:10.276367+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:51.104305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:51.104305+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"457'27","reported_seq":546,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.283917+0000","last_change":"2024-02-23T00:48:00.698737+0000","last_active":"2024-02-23T00:51:10.283917+0000","last_peered":"2024-02-23T00:51:10.283917+0000","last_clean":"2024-02-23T00:51:10.283917+0000","last_became_active":"2024-02-23T00:39:16.789757+0000","last_became_peered":"2024-02-23T00:39:16.789757+0000","last_unstale":"2024-02-23T00:51:10.283917+0000","last_undegraded":"2024-02-23T00:51:10.283917+0000","last_fullsized":"2024-02-23T00:51:10.283917+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:48:00.698670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:48:00.698670+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":505,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.282667+0000","last_change":"2024-02-23T00:48:09.801324+0000","last_active":"2024-02-23T00:51:10.282667+0000","last_peered":"2024-02-23T00:51:10.282667+0000","last_clean":"2024-02-23T00:51:10.282667+0000","last_became_active":"2024-02-23T00:39:14.780526+0000","last_became_peered":"2024-02-23T00:39:14.780526+0000","last_unstale":"2024-02-23T00:51:10.282667+0000","last_undegraded":"2024-02-23T00:51:10.282667+0000","last_fullsized":"2024-02-23T00:51:10.282667+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:09.801251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:48:09.801251+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":660,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.294674+0000","last_change":"2024-02-23T00:49:40.384676+0000","last_active":"2024-02-23T00:51:10.294674+0000","last_peered":"2024-02-23T00:51:10.294674+0000","last_clean":"2024-02-23T00:51:10.294674+0000","last_became_active":"2024-02-23T00:39:18.800440+0000","last_became_peered":"2024-02-23T00:39:18.800440+0000","last_unstale":"2024-02-23T00:51:10.294674+0000","last_undegraded":"2024-02-23T00:51:10.294674+0000","last_fullsized":"2024-02-23T00:51:10.294674+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:49:40.384608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:40.384608+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":452,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.293672+0000","last_change":"2024-02-23T00:48:59.206327+0000","last_active":"2024-02-23T00:51:10.293672+0000","last_peered":"2024-02-23T00:51:10.293672+0000","last_clean":"2024-02-23T00:51:10.293672+0000","last_became_active":"2024-02-23T00:42:58.833924+0000","last_became_peered":"2024-02-23T00:42:58.833924+0000","last_unstale":"2024-02-23T00:51:10.293672+0000","last_undegraded":"2024-02-23T00:51:10.293672+0000","last_fullsized":"2024-02-23T00:51:10.293672+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:59.206249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:59.206249+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"457'31","reported_seq":604,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.294620+0000","last_change":"2024-02-23T00:49:04.235169+0000","last_active":"2024-02-23T00:51:10.294620+0000","last_peered":"2024-02-23T00:51:10.294620+0000","last_clean":"2024-02-23T00:51:10.294620+0000","last_became_active":"2024-02-23T00:47:28.201225+0000","last_became_peered":"2024-02-23T00:47:28.201225+0000","last_unstale":"2024-02-23T00:51:10.294620+0000","last_undegraded":"2024-02-23T00:51:10.294620+0000","last_fullsized":"2024-02-23T00:51:10.294620+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"336'23","last_scrub_stamp":"2024-02-23T00:49:04.235046+0000","last_deep_scrub":"336'23","last_deep_scrub_stamp":"2024-02-23T00:46:48.320085+0000","last_clean_scrub_stamp":"2024-02-23T00:49:04.235046+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":513,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.285582+0000","last_change":"2024-02-23T00:49:37.331334+0000","last_active":"2024-02-23T00:51:10.285582+0000","last_peered":"2024-02-23T00:51:10.285582+0000","last_clean":"2024-02-23T00:51:10.285582+0000","last_became_active":"2024-02-23T00:39:14.784121+0000","last_became_peered":"2024-02-23T00:39:14.784121+0000","last_unstale":"2024-02-23T00:51:10.285582+0000","last_undegraded":"2024-02-23T00:51:10.285582+0000","last_fullsized":"2024-02-23T00:51:10.285582+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:37.331270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:37.331270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":510,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.280148+0000","last_change":"2024-02-23T00:50:19.827889+0000","last_active":"2024-02-23T00:51:10.280148+0000","last_peered":"2024-02-23T00:51:10.280148+0000","last_clean":"2024-02-23T00:51:10.280148+0000","last_became_active":"2024-02-23T00:39:18.800777+0000","last_became_peered":"2024-02-23T00:39:18.800777+0000","last_unstale":"2024-02-23T00:51:10.280148+0000","last_undegraded":"2024-02-23T00:51:10.280148+0000","last_fullsized":"2024-02-23T00:51:10.280148+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:19.827814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_clean_scrub_stamp":"2024-02-23T00:50:19.827814+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":447,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.282412+0000","last_change":"2024-02-23T00:49:33.529224+0000","last_active":"2024-02-23T00:51:10.282412+0000","last_peered":"2024-02-23T00:51:10.282412+0000","last_clean":"2024-02-23T00:51:10.282412+0000","last_became_active":"2024-02-23T00:42:57.653279+0000","last_became_peered":"2024-02-23T00:42:57.653279+0000","last_unstale":"2024-02-23T00:51:10.282412+0000","last_undegraded":"2024-02-23T00:51:10.282412+0000","last_fullsized":"2024-02-23T00:51:10.282412+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:33.529135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:49:33.529135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":658,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.278408+0000","last_change":"2024-02-23T00:49:08.047225+0000","last_active":"2024-02-23T00:51:10.278408+0000","last_peered":"2024-02-23T00:51:10.278408+0000","last_clean":"2024-02-23T00:51:10.278408+0000","last_became_active":"2024-02-23T00:39:18.798536+0000","last_became_peered":"2024-02-23T00:39:18.798536+0000","last_unstale":"2024-02-23T00:51:10.278408+0000","last_undegraded":"2024-02-23T00:51:10.278408+0000","last_fullsized":"2024-02-23T00:51:10.278408+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:49:08.047142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:08.047142+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"457'42","reported_seq":576,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.293877+0000","last_change":"2024-02-23T00:50:24.532387+0000","last_active":"2024-02-23T00:51:10.293877+0000","last_peered":"2024-02-23T00:51:10.293877+0000","last_clean":"2024-02-23T00:51:10.293877+0000","last_became_active":"2024-02-23T00:39:16.780130+0000","last_became_peered":"2024-02-23T00:39:16.780130+0000","last_unstale":"2024-02-23T00:51:10.293877+0000","last_undegraded":"2024-02-23T00:51:10.293877+0000","last_fullsized":"2024-02-23T00:51:10.293877+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'42","last_scrub_stamp":"2024-02-23T00:50:24.532271+0000","last_deep_scrub":"423'28","last_deep_scrub_stamp":"2024-02-23T00:48:17.435906+0000","last_clean_scrub_stamp":"2024-02-23T00:50:24.532271+0000","log_size":42,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":46,"num_write":34,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":528,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.290554+0000","last_change":"2024-02-23T00:50:59.199966+0000","last_active":"2024-02-23T00:51:10.290554+0000","last_peered":"2024-02-23T00:51:10.290554+0000","last_clean":"2024-02-23T00:51:10.290554+0000","last_became_active":"2024-02-23T00:47:28.237083+0000","last_became_peered":"2024-02-23T00:47:28.237083+0000","last_unstale":"2024-02-23T00:51:10.290554+0000","last_undegraded":"2024-02-23T00:51:10.290554+0000","last_fullsized":"2024-02-23T00:51:10.290554+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:59.199899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:51.391939+0000","last_clean_scrub_stamp":"2024-02-23T00:50:59.199899+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":526,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.286193+0000","last_change":"2024-02-23T00:49:41.163033+0000","last_active":"2024-02-23T00:51:10.286193+0000","last_peered":"2024-02-23T00:51:10.286193+0000","last_clean":"2024-02-23T00:51:10.286193+0000","last_became_active":"2024-02-23T00:47:30.187386+0000","last_became_peered":"2024-02-23T00:47:30.187386+0000","last_unstale":"2024-02-23T00:51:10.286193+0000","last_undegraded":"2024-02-23T00:51:10.286193+0000","last_fullsized":"2024-02-23T00:51:10.286193+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:41.162895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:46.686764+0000","last_clean_scrub_stamp":"2024-02-23T00:49:41.162895+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":464,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.280320+0000","last_change":"2024-02-23T00:51:03.875340+0000","last_active":"2024-02-23T00:51:10.280320+0000","last_peered":"2024-02-23T00:51:10.280320+0000","last_clean":"2024-02-23T00:51:10.280320+0000","last_became_active":"2024-02-23T00:47:28.212307+0000","last_became_peered":"2024-02-23T00:47:28.212307+0000","last_unstale":"2024-02-23T00:51:10.280320+0000","last_undegraded":"2024-02-23T00:51:10.280320+0000","last_fullsized":"2024-02-23T00:51:10.280320+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:03.875273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:03.875273+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":511,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.294115+0000","last_change":"2024-02-23T00:50:25.575319+0000","last_active":"2024-02-23T00:51:10.294115+0000","last_peered":"2024-02-23T00:51:10.294115+0000","last_clean":"2024-02-23T00:51:10.294115+0000","last_became_active":"2024-02-23T00:39:18.798829+0000","last_became_peered":"2024-02-23T00:39:18.798829+0000","last_unstale":"2024-02-23T00:51:10.294115+0000","last_undegraded":"2024-02-23T00:51:10.294115+0000","last_fullsized":"2024-02-23T00:51:10.294115+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:25.575238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:10.493340+0000","last_clean_scrub_stamp":"2024-02-23T00:50:25.575238+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"457'52","reported_seq":604,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.278320+0000","last_change":"2024-02-23T00:51:05.221646+0000","last_active":"2024-02-23T00:51:10.278320+0000","last_peered":"2024-02-23T00:51:10.278320+0000","last_clean":"2024-02-23T00:51:10.278320+0000","last_became_active":"2024-02-23T00:39:16.859002+0000","last_became_peered":"2024-02-23T00:39:16.859002+0000","last_unstale":"2024-02-23T00:51:10.278320+0000","last_undegraded":"2024-02-23T00:51:10.278320+0000","last_fullsized":"2024-02-23T00:51:10.278320+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'52","last_scrub_stamp":"2024-02-23T00:51:05.221572+0000","last_deep_scrub":"251'40","last_deep_scrub_stamp":"2024-02-23T00:45:23.910546+0000","last_clean_scrub_stamp":"2024-02-23T00:51:05.221572+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":76,"num_read_kb":62,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":505,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.283937+0000","last_change":"2024-02-23T00:48:44.728592+0000","last_active":"2024-02-23T00:51:10.283937+0000","last_peered":"2024-02-23T00:51:10.283937+0000","last_clean":"2024-02-23T00:51:10.283937+0000","last_became_active":"2024-02-23T00:39:14.779193+0000","last_became_peered":"2024-02-23T00:39:14.779193+0000","last_unstale":"2024-02-23T00:51:10.283937+0000","last_undegraded":"2024-02-23T00:51:10.283937+0000","last_fullsized":"2024-02-23T00:51:10.283937+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:44.728520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:48:44.728520+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":534,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.294102+0000","last_change":"2024-02-23T00:50:17.599788+0000","last_active":"2024-02-23T00:51:10.294102+0000","last_peered":"2024-02-23T00:51:10.294102+0000","last_clean":"2024-02-23T00:51:10.294102+0000","last_became_active":"2024-02-23T00:47:33.257915+0000","last_became_peered":"2024-02-23T00:47:33.257915+0000","last_unstale":"2024-02-23T00:51:10.294102+0000","last_undegraded":"2024-02-23T00:51:10.294102+0000","last_fullsized":"2024-02-23T00:51:10.294102+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":381,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:17.599724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:50:17.599724+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":457,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.279176+0000","last_change":"2024-02-23T00:51:03.166243+0000","last_active":"2024-02-23T00:51:10.279176+0000","last_peered":"2024-02-23T00:51:10.279176+0000","last_clean":"2024-02-23T00:51:10.279176+0000","last_became_active":"2024-02-23T00:42:58.809283+0000","last_became_peered":"2024-02-23T00:42:58.809283+0000","last_unstale":"2024-02-23T00:51:10.279176+0000","last_undegraded":"2024-02-23T00:51:10.279176+0000","last_fullsized":"2024-02-23T00:51:10.279176+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:03.166140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:03.166140+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":508,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.289904+0000","last_change":"2024-02-23T00:48:49.266366+0000","last_active":"2024-02-23T00:51:10.289904+0000","last_peered":"2024-02-23T00:51:10.289904+0000","last_clean":"2024-02-23T00:51:10.289904+0000","last_became_active":"2024-02-23T00:39:18.796664+0000","last_became_peered":"2024-02-23T00:39:18.796664+0000","last_unstale":"2024-02-23T00:51:10.289904+0000","last_undegraded":"2024-02-23T00:51:10.289904+0000","last_fullsized":"2024-02-23T00:51:10.289904+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:49.266295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:48:49.266295+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"457'44","reported_seq":588,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.296258+0000","last_change":"2024-02-23T00:49:46.644982+0000","last_active":"2024-02-23T00:51:10.296258+0000","last_peered":"2024-02-23T00:51:10.296258+0000","last_clean":"2024-02-23T00:51:10.296258+0000","last_became_active":"2024-02-23T00:39:16.857278+0000","last_became_peered":"2024-02-23T00:39:16.857278+0000","last_unstale":"2024-02-23T00:51:10.296258+0000","last_undegraded":"2024-02-23T00:51:10.296258+0000","last_fullsized":"2024-02-23T00:51:10.296258+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'44","last_scrub_stamp":"2024-02-23T00:49:46.644911+0000","last_deep_scrub":"457'44","last_deep_scrub_stamp":"2024-02-23T00:49:46.644911+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.644911+0000","log_size":44,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":513,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.286655+0000","last_change":"2024-02-23T00:50:22.299591+0000","last_active":"2024-02-23T00:51:10.286655+0000","last_peered":"2024-02-23T00:51:10.286655+0000","last_clean":"2024-02-23T00:51:10.286655+0000","last_became_active":"2024-02-23T00:39:14.783662+0000","last_became_peered":"2024-02-23T00:39:14.783662+0000","last_unstale":"2024-02-23T00:51:10.286655+0000","last_undegraded":"2024-02-23T00:51:10.286655+0000","last_fullsized":"2024-02-23T00:51:10.286655+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:22.299534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:22.299534+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":526,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.282696+0000","last_change":"2024-02-23T00:48:13.821965+0000","last_active":"2024-02-23T00:51:10.282696+0000","last_peered":"2024-02-23T00:51:10.282696+0000","last_clean":"2024-02-23T00:51:10.282696+0000","last_became_active":"2024-02-23T00:47:37.184240+0000","last_became_peered":"2024-02-23T00:47:37.184240+0000","last_unstale":"2024-02-23T00:51:10.282696+0000","last_undegraded":"2024-02-23T00:51:10.282696+0000","last_fullsized":"2024-02-23T00:51:10.282696+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":385,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:13.821901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:48:13.821901+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":455,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.279537+0000","last_change":"2024-02-23T00:50:24.682130+0000","last_active":"2024-02-23T00:51:10.279537+0000","last_peered":"2024-02-23T00:51:10.279537+0000","last_clean":"2024-02-23T00:51:10.279537+0000","last_became_active":"2024-02-23T00:42:58.726652+0000","last_became_peered":"2024-02-23T00:42:58.726652+0000","last_unstale":"2024-02-23T00:51:10.279537+0000","last_undegraded":"2024-02-23T00:51:10.279537+0000","last_fullsized":"2024-02-23T00:51:10.279537+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:24.682039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:24.682039+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":510,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.280207+0000","last_change":"2024-02-23T00:49:34.709320+0000","last_active":"2024-02-23T00:51:10.280207+0000","last_peered":"2024-02-23T00:51:10.280207+0000","last_clean":"2024-02-23T00:51:10.280207+0000","last_became_active":"2024-02-23T00:39:18.800378+0000","last_became_peered":"2024-02-23T00:39:18.800378+0000","last_unstale":"2024-02-23T00:51:10.280207+0000","last_undegraded":"2024-02-23T00:51:10.280207+0000","last_fullsized":"2024-02-23T00:51:10.280207+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:34.709251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:59.694033+0000","last_clean_scrub_stamp":"2024-02-23T00:49:34.709251+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"457'30","reported_seq":578,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.297277+0000","last_change":"2024-02-23T00:49:42.730745+0000","last_active":"2024-02-23T00:51:10.297277+0000","last_peered":"2024-02-23T00:51:10.297277+0000","last_clean":"2024-02-23T00:51:10.297277+0000","last_became_active":"2024-02-23T00:47:28.215366+0000","last_became_peered":"2024-02-23T00:47:28.215366+0000","last_unstale":"2024-02-23T00:51:10.297277+0000","last_undegraded":"2024-02-23T00:51:10.297277+0000","last_fullsized":"2024-02-23T00:51:10.297277+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'30","last_scrub_stamp":"2024-02-23T00:49:42.730670+0000","last_deep_scrub":"95'18","last_deep_scrub_stamp":"2024-02-23T00:42:01.195545+0000","last_clean_scrub_stamp":"2024-02-23T00:49:42.730670+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":518,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.285485+0000","last_change":"2024-02-23T00:49:46.162522+0000","last_active":"2024-02-23T00:51:10.285485+0000","last_peered":"2024-02-23T00:51:10.285485+0000","last_clean":"2024-02-23T00:51:10.285485+0000","last_became_active":"2024-02-23T00:39:14.784578+0000","last_became_peered":"2024-02-23T00:39:14.784578+0000","last_unstale":"2024-02-23T00:51:10.285485+0000","last_undegraded":"2024-02-23T00:51:10.285485+0000","last_fullsized":"2024-02-23T00:51:10.285485+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:46.162444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.162444+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":515,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.296321+0000","last_change":"2024-02-23T00:49:31.768539+0000","last_active":"2024-02-23T00:51:10.296321+0000","last_peered":"2024-02-23T00:51:10.296321+0000","last_clean":"2024-02-23T00:51:10.296321+0000","last_became_active":"2024-02-23T00:39:14.775446+0000","last_became_peered":"2024-02-23T00:39:14.775446+0000","last_unstale":"2024-02-23T00:51:10.296321+0000","last_undegraded":"2024-02-23T00:51:10.296321+0000","last_fullsized":"2024-02-23T00:51:10.296321+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:31.768472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:31.768472+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"67'1","reported_seq":537,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.283596+0000","last_change":"2024-02-23T00:51:02.380789+0000","last_active":"2024-02-23T00:51:10.283596+0000","last_peered":"2024-02-23T00:51:10.283596+0000","last_clean":"2024-02-23T00:51:10.283596+0000","last_became_active":"2024-02-23T00:47:56.187940+0000","last_became_peered":"2024-02-23T00:47:56.187940+0000","last_unstale":"2024-02-23T00:51:10.283596+0000","last_undegraded":"2024-02-23T00:51:10.283596+0000","last_fullsized":"2024-02-23T00:51:10.283596+0000","mapping_epoch":65,"log_start":"67'1","ondisk_log_start":"67'1","created":65,"last_epoch_clean":404,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:51:02.380721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:51:02.380721+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":459,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.298846+0000","last_change":"2024-02-23T00:51:10.298816+0000","last_active":"2024-02-23T00:51:10.298846+0000","last_peered":"2024-02-23T00:51:10.298846+0000","last_clean":"2024-02-23T00:51:10.298846+0000","last_became_active":"2024-02-23T00:42:58.690858+0000","last_became_peered":"2024-02-23T00:42:58.690858+0000","last_unstale":"2024-02-23T00:51:10.298846+0000","last_undegraded":"2024-02-23T00:51:10.298846+0000","last_fullsized":"2024-02-23T00:51:10.298846+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:10.298740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:35.360054+0000","last_clean_scrub_stamp":"2024-02-23T00:51:10.298740+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":518,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.286044+0000","last_change":"2024-02-23T00:49:28.358727+0000","last_active":"2024-02-23T00:51:10.286044+0000","last_peered":"2024-02-23T00:51:10.286044+0000","last_clean":"2024-02-23T00:51:10.286044+0000","last_became_active":"2024-02-23T00:47:28.231152+0000","last_became_peered":"2024-02-23T00:47:28.231152+0000","last_unstale":"2024-02-23T00:51:10.286044+0000","last_undegraded":"2024-02-23T00:51:10.286044+0000","last_fullsized":"2024-02-23T00:51:10.286044+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:28.358647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:28.358647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"457'26","reported_seq":555,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.290453+0000","last_change":"2024-02-23T00:51:01.219903+0000","last_active":"2024-02-23T00:51:10.290453+0000","last_peered":"2024-02-23T00:51:10.290453+0000","last_clean":"2024-02-23T00:51:10.290453+0000","last_became_active":"2024-02-23T00:39:16.858880+0000","last_became_peered":"2024-02-23T00:39:16.858880+0000","last_unstale":"2024-02-23T00:51:10.290453+0000","last_undegraded":"2024-02-23T00:51:10.290453+0000","last_fullsized":"2024-02-23T00:51:10.290453+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-23T00:51:01.219824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:01.219824+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":523,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.285691+0000","last_change":"2024-02-23T00:50:24.297660+0000","last_active":"2024-02-23T00:51:10.285691+0000","last_peered":"2024-02-23T00:51:10.285691+0000","last_clean":"2024-02-23T00:51:10.285691+0000","last_became_active":"2024-02-23T00:39:08.248249+0000","last_became_peered":"2024-02-23T00:39:08.248249+0000","last_unstale":"2024-02-23T00:51:10.285691+0000","last_undegraded":"2024-02-23T00:51:10.285691+0000","last_fullsized":"2024-02-23T00:51:10.285691+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:24.297572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:49.977248+0000","last_clean_scrub_stamp":"2024-02-23T00:50:24.297572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":523,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.279186+0000","last_change":"2024-02-23T00:50:21.746072+0000","last_active":"2024-02-23T00:51:10.279186+0000","last_peered":"2024-02-23T00:51:10.279186+0000","last_clean":"2024-02-23T00:51:10.279186+0000","last_became_active":"2024-02-23T00:47:28.240482+0000","last_became_peered":"2024-02-23T00:47:28.240482+0000","last_unstale":"2024-02-23T00:51:10.279186+0000","last_undegraded":"2024-02-23T00:51:10.279186+0000","last_fullsized":"2024-02-23T00:51:10.279186+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:21.745961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:21.745961+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":516,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.276230+0000","last_change":"2024-02-23T00:50:34.055990+0000","last_active":"2024-02-23T00:51:10.276230+0000","last_peered":"2024-02-23T00:51:10.276230+0000","last_clean":"2024-02-23T00:51:10.276230+0000","last_became_active":"2024-02-23T00:39:14.783489+0000","last_became_peered":"2024-02-23T00:39:14.783489+0000","last_unstale":"2024-02-23T00:51:10.276230+0000","last_undegraded":"2024-02-23T00:51:10.276230+0000","last_fullsized":"2024-02-23T00:51:10.276230+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:34.055923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:34.055923+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":544,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.280251+0000","last_change":"2024-02-23T00:49:35.756168+0000","last_active":"2024-02-23T00:51:10.280251+0000","last_peered":"2024-02-23T00:51:10.280251+0000","last_clean":"2024-02-23T00:51:10.280251+0000","last_became_active":"2024-02-23T00:47:45.182550+0000","last_became_peered":"2024-02-23T00:47:45.182550+0000","last_unstale":"2024-02-23T00:51:10.280251+0000","last_undegraded":"2024-02-23T00:51:10.280251+0000","last_fullsized":"2024-02-23T00:51:10.280251+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:35.756073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:49:35.756073+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":467,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.285974+0000","last_change":"2024-02-23T00:50:16.257963+0000","last_active":"2024-02-23T00:51:10.285974+0000","last_peered":"2024-02-23T00:51:10.285974+0000","last_clean":"2024-02-23T00:51:10.285974+0000","last_became_active":"2024-02-23T00:47:28.233475+0000","last_became_peered":"2024-02-23T00:47:28.233475+0000","last_unstale":"2024-02-23T00:51:10.285974+0000","last_undegraded":"2024-02-23T00:51:10.285974+0000","last_fullsized":"2024-02-23T00:51:10.285974+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:16.257896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:16.257896+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":182,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.279806+0000","last_change":"2024-02-23T00:48:57.430209+0000","last_active":"2024-02-23T00:51:10.279806+0000","last_peered":"2024-02-23T00:51:10.279806+0000","last_clean":"2024-02-23T00:51:10.279806+0000","last_became_active":"2024-02-23T00:47:28.209125+0000","last_became_peered":"2024-02-23T00:47:28.209125+0000","last_unstale":"2024-02-23T00:51:10.279806+0000","last_undegraded":"2024-02-23T00:51:10.279806+0000","last_fullsized":"2024-02-23T00:51:10.279806+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:57.430129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.264240+0000","last_clean_scrub_stamp":"2024-02-23T00:48:57.430129+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"457'23","reported_seq":551,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.283980+0000","last_change":"2024-02-23T00:48:11.828616+0000","last_active":"2024-02-23T00:51:10.283980+0000","last_peered":"2024-02-23T00:51:10.283980+0000","last_clean":"2024-02-23T00:51:10.283980+0000","last_became_active":"2024-02-23T00:39:16.787484+0000","last_became_peered":"2024-02-23T00:39:16.787484+0000","last_unstale":"2024-02-23T00:51:10.283980+0000","last_undegraded":"2024-02-23T00:51:10.283980+0000","last_fullsized":"2024-02-23T00:51:10.283980+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'19","last_scrub_stamp":"2024-02-23T00:48:11.828547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:48:11.828547+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":48,"num_read_kb":36,"num_write":26,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":183,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.276368+0000","last_change":"2024-02-23T00:49:35.877867+0000","last_active":"2024-02-23T00:51:10.276368+0000","last_peered":"2024-02-23T00:51:10.276368+0000","last_clean":"2024-02-23T00:51:10.276368+0000","last_became_active":"2024-02-23T00:47:28.209575+0000","last_became_peered":"2024-02-23T00:47:28.209575+0000","last_unstale":"2024-02-23T00:51:10.276368+0000","last_undegraded":"2024-02-23T00:51:10.276368+0000","last_fullsized":"2024-02-23T00:51:10.276368+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:35.877800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:35.877800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"68'5","reported_seq":1229,"reported_epoch":542,"state":"active+clean","last_fresh":"2024-02-23T00:51:11.278543+0000","last_change":"2024-02-23T00:50:57.456201+0000","last_active":"2024-02-23T00:51:11.278543+0000","last_peered":"2024-02-23T00:51:11.278543+0000","last_clean":"2024-02-23T00:51:11.278543+0000","last_became_active":"2024-02-23T00:47:28.227928+0000","last_became_peered":"2024-02-23T00:47:28.227928+0000","last_unstale":"2024-02-23T00:51:11.278543+0000","last_undegraded":"2024-02-23T00:51:11.278543+0000","last_fullsized":"2024-02-23T00:51:11.278543+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'5","last_scrub_stamp":"2024-02-23T00:50:57.456059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:57.456059+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":707,"num_read_kb":702,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":543,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.286553+0000","last_change":"2024-02-23T00:49:30.396935+0000","last_active":"2024-02-23T00:51:10.286553+0000","last_peered":"2024-02-23T00:51:10.286553+0000","last_clean":"2024-02-23T00:51:10.286553+0000","last_became_active":"2024-02-23T00:47:52.552244+0000","last_became_peered":"2024-02-23T00:47:52.552244+0000","last_unstale":"2024-02-23T00:51:10.286553+0000","last_undegraded":"2024-02-23T00:51:10.286553+0000","last_fullsized":"2024-02-23T00:51:10.286553+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:30.396865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:49:30.396865+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":456,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.282443+0000","last_change":"2024-02-23T00:49:40.533441+0000","last_active":"2024-02-23T00:51:10.282443+0000","last_peered":"2024-02-23T00:51:10.282443+0000","last_clean":"2024-02-23T00:51:10.282443+0000","last_became_active":"2024-02-23T00:47:28.212516+0000","last_became_peered":"2024-02-23T00:47:28.212516+0000","last_unstale":"2024-02-23T00:51:10.282443+0000","last_undegraded":"2024-02-23T00:51:10.282443+0000","last_fullsized":"2024-02-23T00:51:10.282443+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:40.533344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:49:40.533344+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":511,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.297198+0000","last_change":"2024-02-23T00:49:37.787041+0000","last_active":"2024-02-23T00:51:10.297198+0000","last_peered":"2024-02-23T00:51:10.297198+0000","last_clean":"2024-02-23T00:51:10.297198+0000","last_became_active":"2024-02-23T00:39:18.802134+0000","last_became_peered":"2024-02-23T00:39:18.802134+0000","last_unstale":"2024-02-23T00:51:10.297198+0000","last_undegraded":"2024-02-23T00:51:10.297198+0000","last_fullsized":"2024-02-23T00:51:10.297198+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:37.786978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:37.786978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"457'48","reported_seq":690,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.279142+0000","last_change":"2024-02-23T00:50:30.763802+0000","last_active":"2024-02-23T00:51:10.279142+0000","last_peered":"2024-02-23T00:51:10.279142+0000","last_clean":"2024-02-23T00:51:10.279142+0000","last_became_active":"2024-02-23T00:47:28.240257+0000","last_became_peered":"2024-02-23T00:47:28.240257+0000","last_unstale":"2024-02-23T00:51:10.279142+0000","last_undegraded":"2024-02-23T00:51:10.279142+0000","last_fullsized":"2024-02-23T00:51:10.279142+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'48","last_scrub_stamp":"2024-02-23T00:50:30.763686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:30.763686+0000","log_size":48,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":78,"num_read_kb":62,"num_write":44,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":514,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.279196+0000","last_change":"2024-02-23T00:49:41.639660+0000","last_active":"2024-02-23T00:51:10.279196+0000","last_peered":"2024-02-23T00:51:10.279196+0000","last_clean":"2024-02-23T00:51:10.279196+0000","last_became_active":"2024-02-23T00:39:14.784015+0000","last_became_peered":"2024-02-23T00:39:14.784015+0000","last_unstale":"2024-02-23T00:51:10.279196+0000","last_undegraded":"2024-02-23T00:51:10.279196+0000","last_fullsized":"2024-02-23T00:51:10.279196+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:41.639582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:41.639582+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":541,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.279076+0000","last_change":"2024-02-23T00:49:36.867589+0000","last_active":"2024-02-23T00:51:10.279076+0000","last_peered":"2024-02-23T00:51:10.279076+0000","last_clean":"2024-02-23T00:51:10.279076+0000","last_became_active":"2024-02-23T00:47:49.525421+0000","last_became_peered":"2024-02-23T00:47:49.525421+0000","last_unstale":"2024-02-23T00:51:10.279076+0000","last_undegraded":"2024-02-23T00:51:10.279076+0000","last_fullsized":"2024-02-23T00:51:10.279076+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":397,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:36.867488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:49:36.867488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":184,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.294383+0000","last_change":"2024-02-23T00:50:13.549604+0000","last_active":"2024-02-23T00:51:10.294383+0000","last_peered":"2024-02-23T00:51:10.294383+0000","last_clean":"2024-02-23T00:51:10.294383+0000","last_became_active":"2024-02-23T00:47:28.232194+0000","last_became_peered":"2024-02-23T00:47:28.232194+0000","last_unstale":"2024-02-23T00:51:10.294383+0000","last_undegraded":"2024-02-23T00:51:10.294383+0000","last_fullsized":"2024-02-23T00:51:10.294383+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:13.549529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:13.549529+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3","version":"91'9","reported_seq":663,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.296513+0000","last_change":"2024-02-23T00:49:41.776249+0000","last_active":"2024-02-23T00:51:10.296513+0000","last_peered":"2024-02-23T00:51:10.296513+0000","last_clean":"2024-02-23T00:51:10.296513+0000","last_became_active":"2024-02-23T00:39:18.801297+0000","last_became_peered":"2024-02-23T00:39:18.801297+0000","last_unstale":"2024-02-23T00:51:10.296513+0000","last_undegraded":"2024-02-23T00:51:10.296513+0000","last_fullsized":"2024-02-23T00:51:10.296513+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'9","last_scrub_stamp":"2024-02-23T00:49:41.776117+0000","last_deep_scrub":"91'9","last_deep_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_clean_scrub_stamp":"2024-02-23T00:49:41.776117+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"457'26","reported_seq":548,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.283045+0000","last_change":"2024-02-23T00:48:55.589320+0000","last_active":"2024-02-23T00:51:10.283045+0000","last_peered":"2024-02-23T00:51:10.283045+0000","last_clean":"2024-02-23T00:51:10.283045+0000","last_became_active":"2024-02-23T00:39:16.787468+0000","last_became_peered":"2024-02-23T00:39:16.787468+0000","last_unstale":"2024-02-23T00:51:10.283045+0000","last_undegraded":"2024-02-23T00:51:10.283045+0000","last_fullsized":"2024-02-23T00:51:10.283045+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:48:55.589209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:48:55.589209+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":36,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":505,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.283437+0000","last_change":"2024-02-23T00:48:12.869003+0000","last_active":"2024-02-23T00:51:10.283437+0000","last_peered":"2024-02-23T00:51:10.283437+0000","last_clean":"2024-02-23T00:51:10.283437+0000","last_became_active":"2024-02-23T00:39:14.779952+0000","last_became_peered":"2024-02-23T00:39:14.779952+0000","last_unstale":"2024-02-23T00:51:10.283437+0000","last_undegraded":"2024-02-23T00:51:10.283437+0000","last_fullsized":"2024-02-23T00:51:10.283437+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:12.868930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:48:12.868930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"107'2","reported_seq":544,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.295847+0000","last_change":"2024-02-23T00:50:17.587665+0000","last_active":"2024-02-23T00:51:10.295847+0000","last_peered":"2024-02-23T00:51:10.295847+0000","last_clean":"2024-02-23T00:51:10.295847+0000","last_became_active":"2024-02-23T00:47:41.210074+0000","last_became_peered":"2024-02-23T00:47:41.210074+0000","last_unstale":"2024-02-23T00:51:10.295847+0000","last_undegraded":"2024-02-23T00:51:10.295847+0000","last_fullsized":"2024-02-23T00:51:10.295847+0000","mapping_epoch":375,"log_start":"107'2","ondisk_log_start":"107'2","created":65,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'2","last_scrub_stamp":"2024-02-23T00:50:17.587591+0000","last_deep_scrub":"107'2","last_deep_scrub_stamp":"2024-02-23T00:42:48.362739+0000","last_clean_scrub_stamp":"2024-02-23T00:50:17.587591+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":463,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.290101+0000","last_change":"2024-02-23T00:50:18.308436+0000","last_active":"2024-02-23T00:51:10.290101+0000","last_peered":"2024-02-23T00:51:10.290101+0000","last_clean":"2024-02-23T00:51:10.290101+0000","last_became_active":"2024-02-23T00:47:28.242255+0000","last_became_peered":"2024-02-23T00:47:28.242255+0000","last_unstale":"2024-02-23T00:51:10.290101+0000","last_undegraded":"2024-02-23T00:51:10.290101+0000","last_fullsized":"2024-02-23T00:51:10.290101+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:18.308350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:18.308350+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":183,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.296238+0000","last_change":"2024-02-23T00:48:54.706708+0000","last_active":"2024-02-23T00:51:10.296238+0000","last_peered":"2024-02-23T00:51:10.296238+0000","last_clean":"2024-02-23T00:51:10.296238+0000","last_became_active":"2024-02-23T00:47:28.234034+0000","last_became_peered":"2024-02-23T00:47:28.234034+0000","last_unstale":"2024-02-23T00:51:10.296238+0000","last_undegraded":"2024-02-23T00:51:10.296238+0000","last_fullsized":"2024-02-23T00:51:10.296238+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:54.706627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_clean_scrub_stamp":"2024-02-23T00:48:54.706627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"457'31","reported_seq":569,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.294163+0000","last_change":"2024-02-23T00:49:08.190437+0000","last_active":"2024-02-23T00:51:10.294163+0000","last_peered":"2024-02-23T00:51:10.294163+0000","last_clean":"2024-02-23T00:51:10.294163+0000","last_became_active":"2024-02-23T00:39:16.783926+0000","last_became_peered":"2024-02-23T00:39:16.783926+0000","last_unstale":"2024-02-23T00:51:10.294163+0000","last_undegraded":"2024-02-23T00:51:10.294163+0000","last_fullsized":"2024-02-23T00:51:10.294163+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'25","last_scrub_stamp":"2024-02-23T00:49:08.190363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:08.190363+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":58,"num_read_kb":47,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"95'4","reported_seq":528,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.295016+0000","last_change":"2024-02-23T00:49:33.465769+0000","last_active":"2024-02-23T00:51:10.295016+0000","last_peered":"2024-02-23T00:51:10.295016+0000","last_clean":"2024-02-23T00:51:10.295016+0000","last_became_active":"2024-02-23T00:39:14.779205+0000","last_became_peered":"2024-02-23T00:39:14.779205+0000","last_unstale":"2024-02-23T00:51:10.295016+0000","last_undegraded":"2024-02-23T00:51:10.295016+0000","last_fullsized":"2024-02-23T00:51:10.295016+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'4","last_scrub_stamp":"2024-02-23T00:49:33.465697+0000","last_deep_scrub":"95'4","last_deep_scrub_stamp":"2024-02-23T00:41:43.874397+0000","last_clean_scrub_stamp":"2024-02-23T00:49:33.465697+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":528,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.277470+0000","last_change":"2024-02-23T00:50:15.922829+0000","last_active":"2024-02-23T00:51:10.277470+0000","last_peered":"2024-02-23T00:51:10.277470+0000","last_clean":"2024-02-23T00:51:10.277470+0000","last_became_active":"2024-02-23T00:43:27.611660+0000","last_became_peered":"2024-02-23T00:43:27.611660+0000","last_unstale":"2024-02-23T00:51:10.277470+0000","last_undegraded":"2024-02-23T00:51:10.277470+0000","last_fullsized":"2024-02-23T00:51:10.277470+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:15.922745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:50:15.922745+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":465,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.286522+0000","last_change":"2024-02-23T00:49:44.143768+0000","last_active":"2024-02-23T00:51:10.286522+0000","last_peered":"2024-02-23T00:51:10.286522+0000","last_clean":"2024-02-23T00:51:10.286522+0000","last_became_active":"2024-02-23T00:47:28.231670+0000","last_became_peered":"2024-02-23T00:47:28.231670+0000","last_unstale":"2024-02-23T00:51:10.286522+0000","last_undegraded":"2024-02-23T00:51:10.286522+0000","last_fullsized":"2024-02-23T00:51:10.286522+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:44.143684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:49:44.143684+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"457'27","reported_seq":563,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.297229+0000","last_change":"2024-02-23T00:49:36.823322+0000","last_active":"2024-02-23T00:51:10.297229+0000","last_peered":"2024-02-23T00:51:10.297229+0000","last_clean":"2024-02-23T00:51:10.297229+0000","last_became_active":"2024-02-23T00:39:16.858760+0000","last_became_peered":"2024-02-23T00:39:16.858760+0000","last_unstale":"2024-02-23T00:51:10.297229+0000","last_undegraded":"2024-02-23T00:51:10.297229+0000","last_fullsized":"2024-02-23T00:51:10.297229+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-23T00:49:36.823214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:36.823214+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":180,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.282433+0000","last_change":"2024-02-23T00:49:02.601790+0000","last_active":"2024-02-23T00:51:10.282433+0000","last_peered":"2024-02-23T00:51:10.282433+0000","last_clean":"2024-02-23T00:51:10.282433+0000","last_became_active":"2024-02-23T00:47:28.219764+0000","last_became_peered":"2024-02-23T00:47:28.219764+0000","last_unstale":"2024-02-23T00:51:10.282433+0000","last_undegraded":"2024-02-23T00:51:10.282433+0000","last_fullsized":"2024-02-23T00:51:10.282433+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:02.601726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:02.601726+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":512,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.279538+0000","last_change":"2024-02-23T00:49:37.881251+0000","last_active":"2024-02-23T00:51:10.279538+0000","last_peered":"2024-02-23T00:51:10.279538+0000","last_clean":"2024-02-23T00:51:10.279538+0000","last_became_active":"2024-02-23T00:39:18.798624+0000","last_became_peered":"2024-02-23T00:39:18.798624+0000","last_unstale":"2024-02-23T00:51:10.279538+0000","last_undegraded":"2024-02-23T00:51:10.279538+0000","last_fullsized":"2024-02-23T00:51:10.279538+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:37.881183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:45.913526+0000","last_clean_scrub_stamp":"2024-02-23T00:49:37.881183+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":456,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.296018+0000","last_change":"2024-02-23T00:50:13.642424+0000","last_active":"2024-02-23T00:51:10.296018+0000","last_peered":"2024-02-23T00:51:10.296018+0000","last_clean":"2024-02-23T00:51:10.296018+0000","last_became_active":"2024-02-23T00:42:58.825015+0000","last_became_peered":"2024-02-23T00:42:58.825015+0000","last_unstale":"2024-02-23T00:51:10.296018+0000","last_undegraded":"2024-02-23T00:51:10.296018+0000","last_fullsized":"2024-02-23T00:51:10.296018+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:13.642359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:13.642359+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"457'22","reported_seq":551,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.279498+0000","last_change":"2024-02-23T00:51:01.201807+0000","last_active":"2024-02-23T00:51:10.279498+0000","last_peered":"2024-02-23T00:51:10.279498+0000","last_clean":"2024-02-23T00:51:10.279498+0000","last_became_active":"2024-02-23T00:39:16.858449+0000","last_became_peered":"2024-02-23T00:39:16.858449+0000","last_unstale":"2024-02-23T00:51:10.279498+0000","last_undegraded":"2024-02-23T00:51:10.279498+0000","last_fullsized":"2024-02-23T00:51:10.279498+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'22","last_scrub_stamp":"2024-02-23T00:51:01.201737+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-02-23T00:48:07.941101+0000","last_clean_scrub_stamp":"2024-02-23T00:51:01.201737+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":30,"num_read_kb":26,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":512,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.289476+0000","last_change":"2024-02-23T00:49:28.347354+0000","last_active":"2024-02-23T00:51:10.289476+0000","last_peered":"2024-02-23T00:51:10.289476+0000","last_clean":"2024-02-23T00:51:10.289476+0000","last_became_active":"2024-02-23T00:39:14.783164+0000","last_became_peered":"2024-02-23T00:39:14.783164+0000","last_unstale":"2024-02-23T00:51:10.289476+0000","last_undegraded":"2024-02-23T00:51:10.289476+0000","last_fullsized":"2024-02-23T00:51:10.289476+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:28.347281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:28.347281+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":510,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.279915+0000","last_change":"2024-02-23T00:49:37.721639+0000","last_active":"2024-02-23T00:51:10.279915+0000","last_peered":"2024-02-23T00:51:10.279915+0000","last_clean":"2024-02-23T00:51:10.279915+0000","last_became_active":"2024-02-23T00:39:18.800297+0000","last_became_peered":"2024-02-23T00:39:18.800297+0000","last_unstale":"2024-02-23T00:51:10.279915+0000","last_undegraded":"2024-02-23T00:51:10.279915+0000","last_fullsized":"2024-02-23T00:51:10.279915+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:37.721563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:37.721563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1","version":"110'7","reported_seq":465,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.276555+0000","last_change":"2024-02-23T00:48:57.168061+0000","last_active":"2024-02-23T00:51:10.276555+0000","last_peered":"2024-02-23T00:51:10.276555+0000","last_clean":"2024-02-23T00:51:10.276555+0000","last_became_active":"2024-02-23T00:42:58.777251+0000","last_became_peered":"2024-02-23T00:42:58.777251+0000","last_unstale":"2024-02-23T00:51:10.276555+0000","last_undegraded":"2024-02-23T00:51:10.276555+0000","last_fullsized":"2024-02-23T00:51:10.276555+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'7","last_scrub_stamp":"2024-02-23T00:48:57.167981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:57.167981+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"457'32","reported_seq":212,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.277904+0000","last_change":"2024-02-23T00:50:25.079859+0000","last_active":"2024-02-23T00:51:10.277904+0000","last_peered":"2024-02-23T00:51:10.277904+0000","last_clean":"2024-02-23T00:51:10.277904+0000","last_became_active":"2024-02-23T00:47:28.237898+0000","last_became_peered":"2024-02-23T00:47:28.237898+0000","last_unstale":"2024-02-23T00:51:10.277904+0000","last_undegraded":"2024-02-23T00:51:10.277904+0000","last_fullsized":"2024-02-23T00:51:10.277904+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-23T00:50:25.079789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:25.079789+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":52,"num_read_kb":45,"num_write":32,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":514,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.279123+0000","last_change":"2024-02-23T00:49:33.731301+0000","last_active":"2024-02-23T00:51:10.279123+0000","last_peered":"2024-02-23T00:51:10.279123+0000","last_clean":"2024-02-23T00:51:10.279123+0000","last_became_active":"2024-02-23T00:39:14.785590+0000","last_became_peered":"2024-02-23T00:39:14.785590+0000","last_unstale":"2024-02-23T00:51:10.279123+0000","last_undegraded":"2024-02-23T00:51:10.279123+0000","last_fullsized":"2024-02-23T00:51:10.279123+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:33.731192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_clean_scrub_stamp":"2024-02-23T00:49:33.731192+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"336'9","reported_seq":662,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.279839+0000","last_change":"2024-02-23T00:49:39.636503+0000","last_active":"2024-02-23T00:51:10.279839+0000","last_peered":"2024-02-23T00:51:10.279839+0000","last_clean":"2024-02-23T00:51:10.279839+0000","last_became_active":"2024-02-23T00:39:18.803566+0000","last_became_peered":"2024-02-23T00:39:18.803566+0000","last_unstale":"2024-02-23T00:51:10.279839+0000","last_undegraded":"2024-02-23T00:51:10.279839+0000","last_fullsized":"2024-02-23T00:51:10.279839+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"336'9","last_scrub_stamp":"2024-02-23T00:49:39.636409+0000","last_deep_scrub":"336'9","last_deep_scrub_stamp":"2024-02-23T00:46:49.160991+0000","last_clean_scrub_stamp":"2024-02-23T00:49:39.636409+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":451,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.285911+0000","last_change":"2024-02-23T00:48:41.688804+0000","last_active":"2024-02-23T00:51:10.285911+0000","last_peered":"2024-02-23T00:51:10.285911+0000","last_clean":"2024-02-23T00:51:10.285911+0000","last_became_active":"2024-02-23T00:42:58.757424+0000","last_became_peered":"2024-02-23T00:42:58.757424+0000","last_unstale":"2024-02-23T00:51:10.285911+0000","last_undegraded":"2024-02-23T00:51:10.285911+0000","last_fullsized":"2024-02-23T00:51:10.285911+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:41.688728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:41.688728+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"457'21","reported_seq":552,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.279645+0000","last_change":"2024-02-23T00:50:17.861556+0000","last_active":"2024-02-23T00:51:10.279645+0000","last_peered":"2024-02-23T00:51:10.279645+0000","last_clean":"2024-02-23T00:51:10.279645+0000","last_became_active":"2024-02-23T00:39:16.858953+0000","last_became_peered":"2024-02-23T00:39:16.858953+0000","last_unstale":"2024-02-23T00:51:10.279645+0000","last_undegraded":"2024-02-23T00:51:10.279645+0000","last_fullsized":"2024-02-23T00:51:10.279645+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-23T00:50:17.861490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:17.861490+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":515,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.296435+0000","last_change":"2024-02-23T00:48:59.712766+0000","last_active":"2024-02-23T00:51:10.296435+0000","last_peered":"2024-02-23T00:51:10.296435+0000","last_clean":"2024-02-23T00:51:10.296435+0000","last_became_active":"2024-02-23T00:39:14.773772+0000","last_became_peered":"2024-02-23T00:39:14.773772+0000","last_unstale":"2024-02-23T00:51:10.296435+0000","last_undegraded":"2024-02-23T00:51:10.296435+0000","last_fullsized":"2024-02-23T00:51:10.296435+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:59.712699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:48:59.712699+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"376'9","reported_seq":671,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.278472+0000","last_change":"2024-02-23T00:49:45.868359+0000","last_active":"2024-02-23T00:51:10.278472+0000","last_peered":"2024-02-23T00:51:10.278472+0000","last_clean":"2024-02-23T00:51:10.278472+0000","last_became_active":"2024-02-23T00:47:28.229316+0000","last_became_peered":"2024-02-23T00:47:28.229316+0000","last_unstale":"2024-02-23T00:51:10.278472+0000","last_undegraded":"2024-02-23T00:51:10.278472+0000","last_fullsized":"2024-02-23T00:51:10.278472+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"376'9","last_scrub_stamp":"2024-02-23T00:49:45.868289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:45.868289+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":447,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.283351+0000","last_change":"2024-02-23T00:50:15.714265+0000","last_active":"2024-02-23T00:51:10.283351+0000","last_peered":"2024-02-23T00:51:10.283351+0000","last_clean":"2024-02-23T00:51:10.283351+0000","last_became_active":"2024-02-23T00:42:57.652488+0000","last_became_peered":"2024-02-23T00:42:57.652488+0000","last_unstale":"2024-02-23T00:51:10.283351+0000","last_undegraded":"2024-02-23T00:51:10.283351+0000","last_fullsized":"2024-02-23T00:51:10.283351+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:15.714200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:51.254903+0000","last_clean_scrub_stamp":"2024-02-23T00:50:15.714200+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"457'15","reported_seq":547,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.282882+0000","last_change":"2024-02-23T00:49:39.573557+0000","last_active":"2024-02-23T00:51:10.282882+0000","last_peered":"2024-02-23T00:51:10.282882+0000","last_clean":"2024-02-23T00:51:10.282882+0000","last_became_active":"2024-02-23T00:47:28.225585+0000","last_became_peered":"2024-02-23T00:47:28.225585+0000","last_unstale":"2024-02-23T00:51:10.282882+0000","last_undegraded":"2024-02-23T00:51:10.282882+0000","last_fullsized":"2024-02-23T00:51:10.282882+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'15","last_scrub_stamp":"2024-02-23T00:49:39.573472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:39.573472+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":517,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.290065+0000","last_change":"2024-02-23T00:49:02.303864+0000","last_active":"2024-02-23T00:51:10.290065+0000","last_peered":"2024-02-23T00:51:10.290065+0000","last_clean":"2024-02-23T00:51:10.290065+0000","last_became_active":"2024-02-23T00:39:14.778455+0000","last_became_peered":"2024-02-23T00:39:14.778455+0000","last_unstale":"2024-02-23T00:51:10.290065+0000","last_undegraded":"2024-02-23T00:51:10.290065+0000","last_fullsized":"2024-02-23T00:51:10.290065+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:02.303789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:02.303789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":521,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.277345+0000","last_change":"2024-02-23T00:50:30.069404+0000","last_active":"2024-02-23T00:51:10.277345+0000","last_peered":"2024-02-23T00:51:10.277345+0000","last_clean":"2024-02-23T00:51:10.277345+0000","last_became_active":"2024-02-23T00:47:44.225379+0000","last_became_peered":"2024-02-23T00:47:44.225379+0000","last_unstale":"2024-02-23T00:51:10.277345+0000","last_undegraded":"2024-02-23T00:51:10.277345+0000","last_fullsized":"2024-02-23T00:51:10.277345+0000","mapping_epoch":391,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":392,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:30.069325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_clean_scrub_stamp":"2024-02-23T00:50:30.069325+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":456,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.296683+0000","last_change":"2024-02-23T00:49:40.770764+0000","last_active":"2024-02-23T00:51:10.296683+0000","last_peered":"2024-02-23T00:51:10.296683+0000","last_clean":"2024-02-23T00:51:10.296683+0000","last_became_active":"2024-02-23T00:42:58.835862+0000","last_became_peered":"2024-02-23T00:42:58.835862+0000","last_unstale":"2024-02-23T00:51:10.296683+0000","last_undegraded":"2024-02-23T00:51:10.296683+0000","last_fullsized":"2024-02-23T00:51:10.296683+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:40.770694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:49:40.770694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"457'19","reported_seq":542,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.295024+0000","last_change":"2024-02-23T00:50:29.599563+0000","last_active":"2024-02-23T00:51:10.295024+0000","last_peered":"2024-02-23T00:51:10.295024+0000","last_clean":"2024-02-23T00:51:10.295024+0000","last_became_active":"2024-02-23T00:39:16.783896+0000","last_became_peered":"2024-02-23T00:39:16.783896+0000","last_unstale":"2024-02-23T00:51:10.295024+0000","last_undegraded":"2024-02-23T00:51:10.295024+0000","last_fullsized":"2024-02-23T00:51:10.295024+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'19","last_scrub_stamp":"2024-02-23T00:50:29.599483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:29.599483+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"202'2","reported_seq":513,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.283640+0000","last_change":"2024-02-23T00:48:04.832975+0000","last_active":"2024-02-23T00:51:10.283640+0000","last_peered":"2024-02-23T00:51:10.283640+0000","last_clean":"2024-02-23T00:51:10.283640+0000","last_became_active":"2024-02-23T00:39:14.775804+0000","last_became_peered":"2024-02-23T00:39:14.775804+0000","last_unstale":"2024-02-23T00:51:10.283640+0000","last_undegraded":"2024-02-23T00:51:10.283640+0000","last_fullsized":"2024-02-23T00:51:10.283640+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"202'2","last_scrub_stamp":"2024-02-23T00:48:04.832904+0000","last_deep_scrub":"202'2","last_deep_scrub_stamp":"2024-02-23T00:44:34.720640+0000","last_clean_scrub_stamp":"2024-02-23T00:48:04.832904+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":501,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.282963+0000","last_change":"2024-02-23T00:50:18.698805+0000","last_active":"2024-02-23T00:51:10.282963+0000","last_peered":"2024-02-23T00:51:10.282963+0000","last_clean":"2024-02-23T00:51:10.282963+0000","last_became_active":"2024-02-23T00:39:18.800382+0000","last_became_peered":"2024-02-23T00:39:18.800382+0000","last_unstale":"2024-02-23T00:51:10.282963+0000","last_undegraded":"2024-02-23T00:51:10.282963+0000","last_fullsized":"2024-02-23T00:51:10.282963+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:18.698687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:50:18.698687+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":456,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.283617+0000","last_change":"2024-02-23T00:49:46.462085+0000","last_active":"2024-02-23T00:51:10.283617+0000","last_peered":"2024-02-23T00:51:10.283617+0000","last_clean":"2024-02-23T00:51:10.283617+0000","last_became_active":"2024-02-23T00:47:28.224961+0000","last_became_peered":"2024-02-23T00:47:28.224961+0000","last_unstale":"2024-02-23T00:51:10.283617+0000","last_undegraded":"2024-02-23T00:51:10.283617+0000","last_fullsized":"2024-02-23T00:51:10.283617+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:46.462015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:46.462015+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.462015+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"457'8","reported_seq":526,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.278167+0000","last_change":"2024-02-23T00:50:35.045697+0000","last_active":"2024-02-23T00:51:10.278167+0000","last_peered":"2024-02-23T00:51:10.278167+0000","last_clean":"2024-02-23T00:51:10.278167+0000","last_became_active":"2024-02-23T00:39:16.785830+0000","last_became_peered":"2024-02-23T00:39:16.785830+0000","last_unstale":"2024-02-23T00:51:10.278167+0000","last_undegraded":"2024-02-23T00:51:10.278167+0000","last_fullsized":"2024-02-23T00:51:10.278167+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'8","last_scrub_stamp":"2024-02-23T00:50:35.045612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:35.045612+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":526,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.279929+0000","last_change":"2024-02-23T00:49:26.672761+0000","last_active":"2024-02-23T00:51:10.279929+0000","last_peered":"2024-02-23T00:51:10.279929+0000","last_clean":"2024-02-23T00:51:10.279929+0000","last_became_active":"2024-02-23T00:47:28.224477+0000","last_became_peered":"2024-02-23T00:47:28.224477+0000","last_unstale":"2024-02-23T00:51:10.279929+0000","last_undegraded":"2024-02-23T00:51:10.279929+0000","last_fullsized":"2024-02-23T00:51:10.279929+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:26.672673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:26.672673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":522,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.289377+0000","last_change":"2024-02-23T00:49:33.461971+0000","last_active":"2024-02-23T00:51:10.289377+0000","last_peered":"2024-02-23T00:51:10.289377+0000","last_clean":"2024-02-23T00:51:10.289377+0000","last_became_active":"2024-02-23T00:47:44.230295+0000","last_became_peered":"2024-02-23T00:47:44.230295+0000","last_unstale":"2024-02-23T00:51:10.289377+0000","last_undegraded":"2024-02-23T00:51:10.289377+0000","last_fullsized":"2024-02-23T00:51:10.289377+0000","mapping_epoch":391,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":392,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:33.461907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:33.461907+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":451,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.286473+0000","last_change":"2024-02-23T00:48:51.717473+0000","last_active":"2024-02-23T00:51:10.286473+0000","last_peered":"2024-02-23T00:51:10.286473+0000","last_clean":"2024-02-23T00:51:10.286473+0000","last_became_active":"2024-02-23T00:42:58.741365+0000","last_became_peered":"2024-02-23T00:42:58.741365+0000","last_unstale":"2024-02-23T00:51:10.286473+0000","last_undegraded":"2024-02-23T00:51:10.286473+0000","last_fullsized":"2024-02-23T00:51:10.286473+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:51.717397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:51.717397+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"457'25","reported_seq":560,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.276785+0000","last_change":"2024-02-23T00:49:47.788110+0000","last_active":"2024-02-23T00:51:10.276785+0000","last_peered":"2024-02-23T00:51:10.276785+0000","last_clean":"2024-02-23T00:51:10.276785+0000","last_became_active":"2024-02-23T00:39:16.857232+0000","last_became_peered":"2024-02-23T00:39:16.857232+0000","last_unstale":"2024-02-23T00:51:10.276785+0000","last_undegraded":"2024-02-23T00:51:10.276785+0000","last_fullsized":"2024-02-23T00:51:10.276785+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-23T00:49:47.788044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:47.788044+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":521,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.290493+0000","last_change":"2024-02-23T00:49:05.329264+0000","last_active":"2024-02-23T00:51:10.290493+0000","last_peered":"2024-02-23T00:51:10.290493+0000","last_clean":"2024-02-23T00:51:10.290493+0000","last_became_active":"2024-02-23T00:47:28.240162+0000","last_became_peered":"2024-02-23T00:47:28.240162+0000","last_unstale":"2024-02-23T00:51:10.290493+0000","last_undegraded":"2024-02-23T00:51:10.290493+0000","last_fullsized":"2024-02-23T00:51:10.290493+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:05.329178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:44.285083+0000","last_clean_scrub_stamp":"2024-02-23T00:49:05.329178+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":165,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.277810+0000","last_change":"2024-02-23T00:49:42.872820+0000","last_active":"2024-02-23T00:51:10.277810+0000","last_peered":"2024-02-23T00:51:10.277810+0000","last_clean":"2024-02-23T00:51:10.277810+0000","last_became_active":"2024-02-23T00:47:44.226097+0000","last_became_peered":"2024-02-23T00:47:44.226097+0000","last_unstale":"2024-02-23T00:51:10.277810+0000","last_undegraded":"2024-02-23T00:51:10.277810+0000","last_fullsized":"2024-02-23T00:51:10.277810+0000","mapping_epoch":391,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":392,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:42.872748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:42.872748+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":188,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.280024+0000","last_change":"2024-02-23T00:50:28.678155+0000","last_active":"2024-02-23T00:51:10.280024+0000","last_peered":"2024-02-23T00:51:10.280024+0000","last_clean":"2024-02-23T00:51:10.280024+0000","last_became_active":"2024-02-23T00:47:28.200688+0000","last_became_peered":"2024-02-23T00:47:28.200688+0000","last_unstale":"2024-02-23T00:51:10.280024+0000","last_undegraded":"2024-02-23T00:51:10.280024+0000","last_fullsized":"2024-02-23T00:51:10.280024+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:28.678041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:28.678041+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"457'26","reported_seq":558,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.297535+0000","last_change":"2024-02-23T00:50:23.540361+0000","last_active":"2024-02-23T00:51:10.297535+0000","last_peered":"2024-02-23T00:51:10.297535+0000","last_clean":"2024-02-23T00:51:10.297535+0000","last_became_active":"2024-02-23T00:39:16.858945+0000","last_became_peered":"2024-02-23T00:39:16.858945+0000","last_unstale":"2024-02-23T00:51:10.297535+0000","last_undegraded":"2024-02-23T00:51:10.297535+0000","last_fullsized":"2024-02-23T00:51:10.297535+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-23T00:50:23.540232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:23.540232+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":36,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":526,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.278673+0000","last_change":"2024-02-23T00:51:09.827935+0000","last_active":"2024-02-23T00:51:10.278673+0000","last_peered":"2024-02-23T00:51:10.278673+0000","last_clean":"2024-02-23T00:51:10.278673+0000","last_became_active":"2024-02-23T00:39:14.785363+0000","last_became_peered":"2024-02-23T00:39:14.785363+0000","last_unstale":"2024-02-23T00:51:10.278673+0000","last_undegraded":"2024-02-23T00:51:10.278673+0000","last_fullsized":"2024-02-23T00:51:10.278673+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"80'2","last_scrub_stamp":"2024-02-23T00:51:09.827857+0000","last_deep_scrub":"80'2","last_deep_scrub_stamp":"2024-02-23T00:40:15.942725+0000","last_clean_scrub_stamp":"2024-02-23T00:51:09.827857+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"93'9","reported_seq":663,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.290609+0000","last_change":"2024-02-23T00:48:53.246622+0000","last_active":"2024-02-23T00:51:10.290609+0000","last_peered":"2024-02-23T00:51:10.290609+0000","last_clean":"2024-02-23T00:51:10.290609+0000","last_became_active":"2024-02-23T00:39:18.796680+0000","last_became_peered":"2024-02-23T00:39:18.796680+0000","last_unstale":"2024-02-23T00:51:10.290609+0000","last_undegraded":"2024-02-23T00:51:10.290609+0000","last_fullsized":"2024-02-23T00:51:10.290609+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-02-23T00:48:53.246535+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-02-23T00:43:50.393713+0000","last_clean_scrub_stamp":"2024-02-23T00:48:53.246535+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":460,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.296284+0000","last_change":"2024-02-23T00:48:17.898787+0000","last_active":"2024-02-23T00:51:10.296284+0000","last_peered":"2024-02-23T00:51:10.296284+0000","last_clean":"2024-02-23T00:51:10.296284+0000","last_became_active":"2024-02-23T00:47:28.234641+0000","last_became_peered":"2024-02-23T00:47:28.234641+0000","last_unstale":"2024-02-23T00:51:10.296284+0000","last_undegraded":"2024-02-23T00:51:10.296284+0000","last_fullsized":"2024-02-23T00:51:10.296284+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:17.898720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:17.898720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"457'31","reported_seq":567,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.290702+0000","last_change":"2024-02-23T00:48:45.295904+0000","last_active":"2024-02-23T00:51:10.290702+0000","last_peered":"2024-02-23T00:51:10.290702+0000","last_clean":"2024-02-23T00:51:10.290702+0000","last_became_active":"2024-02-23T00:39:16.782660+0000","last_became_peered":"2024-02-23T00:39:16.782660+0000","last_unstale":"2024-02-23T00:51:10.290702+0000","last_undegraded":"2024-02-23T00:51:10.290702+0000","last_fullsized":"2024-02-23T00:51:10.290702+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'23","last_scrub_stamp":"2024-02-23T00:48:45.295833+0000","last_deep_scrub":"93'23","last_deep_scrub_stamp":"2024-02-23T00:41:35.954516+0000","last_clean_scrub_stamp":"2024-02-23T00:48:45.295833+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":516,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.276146+0000","last_change":"2024-02-23T00:50:29.029794+0000","last_active":"2024-02-23T00:51:10.276146+0000","last_peered":"2024-02-23T00:51:10.276146+0000","last_clean":"2024-02-23T00:51:10.276146+0000","last_became_active":"2024-02-23T00:39:14.781560+0000","last_became_peered":"2024-02-23T00:39:14.781560+0000","last_unstale":"2024-02-23T00:51:10.276146+0000","last_undegraded":"2024-02-23T00:51:10.276146+0000","last_fullsized":"2024-02-23T00:51:10.276146+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:29.029725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:29.029725+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":520,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.297559+0000","last_change":"2024-02-23T00:48:53.747314+0000","last_active":"2024-02-23T00:51:10.297559+0000","last_peered":"2024-02-23T00:51:10.297559+0000","last_clean":"2024-02-23T00:51:10.297559+0000","last_became_active":"2024-02-23T00:47:28.232797+0000","last_became_peered":"2024-02-23T00:47:28.232797+0000","last_unstale":"2024-02-23T00:51:10.297559+0000","last_undegraded":"2024-02-23T00:51:10.297559+0000","last_fullsized":"2024-02-23T00:51:10.297559+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:53.747243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:53.189881+0000","last_clean_scrub_stamp":"2024-02-23T00:48:53.747243+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":457,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.276235+0000","last_change":"2024-02-23T00:51:00.232343+0000","last_active":"2024-02-23T00:51:10.276235+0000","last_peered":"2024-02-23T00:51:10.276235+0000","last_clean":"2024-02-23T00:51:10.276235+0000","last_became_active":"2024-02-23T00:42:58.774720+0000","last_became_peered":"2024-02-23T00:42:58.774720+0000","last_unstale":"2024-02-23T00:51:10.276235+0000","last_undegraded":"2024-02-23T00:51:10.276235+0000","last_fullsized":"2024-02-23T00:51:10.276235+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.232261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.232261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"457'27","reported_seq":251,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.283460+0000","last_change":"2024-02-23T00:49:26.518048+0000","last_active":"2024-02-23T00:51:10.283460+0000","last_peered":"2024-02-23T00:51:10.283460+0000","last_clean":"2024-02-23T00:51:10.283460+0000","last_became_active":"2024-02-23T00:47:28.211342+0000","last_became_peered":"2024-02-23T00:47:28.211342+0000","last_unstale":"2024-02-23T00:51:10.283460+0000","last_undegraded":"2024-02-23T00:51:10.283460+0000","last_fullsized":"2024-02-23T00:51:10.283460+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-23T00:49:26.517965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:26.517965+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":517,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.289690+0000","last_change":"2024-02-23T00:51:08.235807+0000","last_active":"2024-02-23T00:51:10.289690+0000","last_peered":"2024-02-23T00:51:10.289690+0000","last_clean":"2024-02-23T00:51:10.289690+0000","last_became_active":"2024-02-23T00:39:14.782494+0000","last_became_peered":"2024-02-23T00:39:14.782494+0000","last_unstale":"2024-02-23T00:51:10.289690+0000","last_undegraded":"2024-02-23T00:51:10.289690+0000","last_fullsized":"2024-02-23T00:51:10.289690+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:08.235735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:08.235735+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":501,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.283812+0000","last_change":"2024-02-23T00:48:16.788875+0000","last_active":"2024-02-23T00:51:10.283812+0000","last_peered":"2024-02-23T00:51:10.283812+0000","last_clean":"2024-02-23T00:51:10.283812+0000","last_became_active":"2024-02-23T00:39:18.801039+0000","last_became_peered":"2024-02-23T00:39:18.801039+0000","last_unstale":"2024-02-23T00:51:10.283812+0000","last_undegraded":"2024-02-23T00:51:10.283812+0000","last_fullsized":"2024-02-23T00:51:10.283812+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:16.788806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.641689+0000","last_clean_scrub_stamp":"2024-02-23T00:48:16.788806+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":457,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.341858+0000","last_change":"2024-02-23T00:51:10.341831+0000","last_active":"2024-02-23T00:51:10.341858+0000","last_peered":"2024-02-23T00:51:10.341858+0000","last_clean":"2024-02-23T00:51:10.341858+0000","last_became_active":"2024-02-23T00:42:58.770759+0000","last_became_peered":"2024-02-23T00:42:58.770759+0000","last_unstale":"2024-02-23T00:51:10.341858+0000","last_undegraded":"2024-02-23T00:51:10.341858+0000","last_fullsized":"2024-02-23T00:51:10.341858+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:10.341768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:10.341768+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"457'24","reported_seq":561,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.279113+0000","last_change":"2024-02-23T00:50:13.861288+0000","last_active":"2024-02-23T00:51:10.279113+0000","last_peered":"2024-02-23T00:51:10.279113+0000","last_clean":"2024-02-23T00:51:10.279113+0000","last_became_active":"2024-02-23T00:39:16.782803+0000","last_became_peered":"2024-02-23T00:39:16.782803+0000","last_unstale":"2024-02-23T00:51:10.279113+0000","last_undegraded":"2024-02-23T00:51:10.279113+0000","last_fullsized":"2024-02-23T00:51:10.279113+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'24","last_scrub_stamp":"2024-02-23T00:50:13.861220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:13.861220+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":46,"num_read_kb":36,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":516,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.277264+0000","last_change":"2024-02-23T00:50:28.060186+0000","last_active":"2024-02-23T00:51:10.277264+0000","last_peered":"2024-02-23T00:51:10.277264+0000","last_clean":"2024-02-23T00:51:10.277264+0000","last_became_active":"2024-02-23T00:39:14.782163+0000","last_became_peered":"2024-02-23T00:39:14.782163+0000","last_unstale":"2024-02-23T00:51:10.277264+0000","last_undegraded":"2024-02-23T00:51:10.277264+0000","last_fullsized":"2024-02-23T00:51:10.277264+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:28.060109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:28.060109+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":518,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.294263+0000","last_change":"2024-02-23T00:50:18.629287+0000","last_active":"2024-02-23T00:51:10.294263+0000","last_peered":"2024-02-23T00:51:10.294263+0000","last_clean":"2024-02-23T00:51:10.294263+0000","last_became_active":"2024-02-23T00:47:28.215452+0000","last_became_peered":"2024-02-23T00:47:28.215452+0000","last_unstale":"2024-02-23T00:51:10.294263+0000","last_undegraded":"2024-02-23T00:51:10.294263+0000","last_fullsized":"2024-02-23T00:51:10.294263+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:18.629203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_clean_scrub_stamp":"2024-02-23T00:50:18.629203+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":464,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.279399+0000","last_change":"2024-02-23T00:50:56.774466+0000","last_active":"2024-02-23T00:51:10.279399+0000","last_peered":"2024-02-23T00:51:10.279399+0000","last_clean":"2024-02-23T00:51:10.279399+0000","last_became_active":"2024-02-23T00:47:28.203555+0000","last_became_peered":"2024-02-23T00:47:28.203555+0000","last_unstale":"2024-02-23T00:51:10.279399+0000","last_undegraded":"2024-02-23T00:51:10.279399+0000","last_fullsized":"2024-02-23T00:51:10.279399+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:56.774382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:56.774382+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"457'33","reported_seq":574,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.293683+0000","last_change":"2024-02-23T00:49:46.364498+0000","last_active":"2024-02-23T00:51:10.293683+0000","last_peered":"2024-02-23T00:51:10.293683+0000","last_clean":"2024-02-23T00:51:10.293683+0000","last_became_active":"2024-02-23T00:39:16.786791+0000","last_became_peered":"2024-02-23T00:39:16.786791+0000","last_unstale":"2024-02-23T00:51:10.293683+0000","last_undegraded":"2024-02-23T00:51:10.293683+0000","last_fullsized":"2024-02-23T00:51:10.293683+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'33","last_scrub_stamp":"2024-02-23T00:49:46.364418+0000","last_deep_scrub":"95'27","last_deep_scrub_stamp":"2024-02-23T00:41:55.919895+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.364418+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":522,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.285925+0000","last_change":"2024-02-23T00:49:27.403374+0000","last_active":"2024-02-23T00:51:10.285925+0000","last_peered":"2024-02-23T00:51:10.285925+0000","last_clean":"2024-02-23T00:51:10.285925+0000","last_became_active":"2024-02-23T00:47:28.208293+0000","last_became_peered":"2024-02-23T00:47:28.208293+0000","last_unstale":"2024-02-23T00:51:10.285925+0000","last_undegraded":"2024-02-23T00:51:10.285925+0000","last_fullsized":"2024-02-23T00:51:10.285925+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:27.403305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:27.403305+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":510,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.279267+0000","last_change":"2024-02-23T00:49:46.612538+0000","last_active":"2024-02-23T00:51:10.279267+0000","last_peered":"2024-02-23T00:51:10.279267+0000","last_clean":"2024-02-23T00:51:10.279267+0000","last_became_active":"2024-02-23T00:39:18.796431+0000","last_became_peered":"2024-02-23T00:39:18.796431+0000","last_unstale":"2024-02-23T00:51:10.279267+0000","last_undegraded":"2024-02-23T00:51:10.279267+0000","last_fullsized":"2024-02-23T00:51:10.279267+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:46.612461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.612461+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":455,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.279586+0000","last_change":"2024-02-23T00:50:26.630228+0000","last_active":"2024-02-23T00:51:10.279586+0000","last_peered":"2024-02-23T00:51:10.279586+0000","last_clean":"2024-02-23T00:51:10.279586+0000","last_became_active":"2024-02-23T00:42:58.832885+0000","last_became_peered":"2024-02-23T00:42:58.832885+0000","last_unstale":"2024-02-23T00:51:10.279586+0000","last_undegraded":"2024-02-23T00:51:10.279586+0000","last_fullsized":"2024-02-23T00:51:10.279586+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:26.630088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:26.630088+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"457'17","reported_seq":543,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.295237+0000","last_change":"2024-02-23T00:50:15.590233+0000","last_active":"2024-02-23T00:51:10.295237+0000","last_peered":"2024-02-23T00:51:10.295237+0000","last_clean":"2024-02-23T00:51:10.295237+0000","last_became_active":"2024-02-23T00:39:16.782762+0000","last_became_peered":"2024-02-23T00:39:16.782762+0000","last_unstale":"2024-02-23T00:51:10.295237+0000","last_undegraded":"2024-02-23T00:51:10.295237+0000","last_fullsized":"2024-02-23T00:51:10.295237+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'17","last_scrub_stamp":"2024-02-23T00:50:15.590096+0000","last_deep_scrub":"82'15","last_deep_scrub_stamp":"2024-02-23T00:41:50.975683+0000","last_clean_scrub_stamp":"2024-02-23T00:50:15.590096+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":21,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":518,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.286365+0000","last_change":"2024-02-23T00:51:08.199826+0000","last_active":"2024-02-23T00:51:10.286365+0000","last_peered":"2024-02-23T00:51:10.286365+0000","last_clean":"2024-02-23T00:51:10.286365+0000","last_became_active":"2024-02-23T00:39:14.783469+0000","last_became_peered":"2024-02-23T00:39:14.783469+0000","last_unstale":"2024-02-23T00:51:10.286365+0000","last_undegraded":"2024-02-23T00:51:10.286365+0000","last_fullsized":"2024-02-23T00:51:10.286365+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:08.199749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:08.199749+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":511,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.285684+0000","last_change":"2024-02-23T00:49:06.491877+0000","last_active":"2024-02-23T00:51:10.285684+0000","last_peered":"2024-02-23T00:51:10.285684+0000","last_clean":"2024-02-23T00:51:10.285684+0000","last_became_active":"2024-02-23T00:39:18.796952+0000","last_became_peered":"2024-02-23T00:39:18.796952+0000","last_unstale":"2024-02-23T00:51:10.285684+0000","last_undegraded":"2024-02-23T00:51:10.285684+0000","last_fullsized":"2024-02-23T00:51:10.285684+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:06.491793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:06.491793+0000","last_clean_scrub_stamp":"2024-02-23T00:49:06.491793+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":457,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.294414+0000","last_change":"2024-02-23T00:51:03.460376+0000","last_active":"2024-02-23T00:51:10.294414+0000","last_peered":"2024-02-23T00:51:10.294414+0000","last_clean":"2024-02-23T00:51:10.294414+0000","last_became_active":"2024-02-23T00:42:58.758147+0000","last_became_peered":"2024-02-23T00:42:58.758147+0000","last_unstale":"2024-02-23T00:51:10.294414+0000","last_undegraded":"2024-02-23T00:51:10.294414+0000","last_fullsized":"2024-02-23T00:51:10.294414+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:03.460302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:23.224663+0000","last_clean_scrub_stamp":"2024-02-23T00:51:03.460302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"457'21","reported_seq":551,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.293482+0000","last_change":"2024-02-23T00:49:32.434383+0000","last_active":"2024-02-23T00:51:10.293482+0000","last_peered":"2024-02-23T00:51:10.293482+0000","last_clean":"2024-02-23T00:51:10.293482+0000","last_became_active":"2024-02-23T00:39:16.784606+0000","last_became_peered":"2024-02-23T00:39:16.784606+0000","last_unstale":"2024-02-23T00:51:10.293482+0000","last_undegraded":"2024-02-23T00:51:10.293482+0000","last_fullsized":"2024-02-23T00:51:10.293482+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-23T00:49:32.434282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:32.434282+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":516,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.279003+0000","last_change":"2024-02-23T00:50:19.967076+0000","last_active":"2024-02-23T00:51:10.279003+0000","last_peered":"2024-02-23T00:51:10.279003+0000","last_clean":"2024-02-23T00:51:10.279003+0000","last_became_active":"2024-02-23T00:39:14.785362+0000","last_became_peered":"2024-02-23T00:39:14.785362+0000","last_unstale":"2024-02-23T00:51:10.279003+0000","last_undegraded":"2024-02-23T00:51:10.279003+0000","last_fullsized":"2024-02-23T00:51:10.279003+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:19.967009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:19.967009+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":501,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.282643+0000","last_change":"2024-02-23T00:49:37.537279+0000","last_active":"2024-02-23T00:51:10.282643+0000","last_peered":"2024-02-23T00:51:10.282643+0000","last_clean":"2024-02-23T00:51:10.282643+0000","last_became_active":"2024-02-23T00:39:18.796312+0000","last_became_peered":"2024-02-23T00:39:18.796312+0000","last_unstale":"2024-02-23T00:51:10.282643+0000","last_undegraded":"2024-02-23T00:51:10.282643+0000","last_fullsized":"2024-02-23T00:51:10.282643+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:37.537211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:47.226331+0000","last_clean_scrub_stamp":"2024-02-23T00:49:37.537211+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":456,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.297361+0000","last_change":"2024-02-23T00:50:28.479578+0000","last_active":"2024-02-23T00:51:10.297361+0000","last_peered":"2024-02-23T00:51:10.297361+0000","last_clean":"2024-02-23T00:51:10.297361+0000","last_became_active":"2024-02-23T00:42:58.838257+0000","last_became_peered":"2024-02-23T00:42:58.838257+0000","last_unstale":"2024-02-23T00:51:10.297361+0000","last_undegraded":"2024-02-23T00:51:10.297361+0000","last_fullsized":"2024-02-23T00:51:10.297361+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:28.479512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:50:28.479512+0000","last_clean_scrub_stamp":"2024-02-23T00:50:28.479512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"457'62","reported_seq":661,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.276880+0000","last_change":"2024-02-23T00:50:58.151569+0000","last_active":"2024-02-23T00:51:10.276880+0000","last_peered":"2024-02-23T00:51:10.276880+0000","last_clean":"2024-02-23T00:51:10.276880+0000","last_became_active":"2024-02-23T00:47:28.205150+0000","last_became_peered":"2024-02-23T00:47:28.205150+0000","last_unstale":"2024-02-23T00:51:10.276880+0000","last_undegraded":"2024-02-23T00:51:10.276880+0000","last_fullsized":"2024-02-23T00:51:10.276880+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'62","last_scrub_stamp":"2024-02-23T00:50:58.151494+0000","last_deep_scrub":"88'50","last_deep_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_clean_scrub_stamp":"2024-02-23T00:50:58.151494+0000","log_size":62,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":92,"num_read_kb":79,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":515,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.297367+0000","last_change":"2024-02-23T00:50:34.600693+0000","last_active":"2024-02-23T00:51:10.297367+0000","last_peered":"2024-02-23T00:51:10.297367+0000","last_clean":"2024-02-23T00:51:10.297367+0000","last_became_active":"2024-02-23T00:39:14.782000+0000","last_became_peered":"2024-02-23T00:39:14.782000+0000","last_unstale":"2024-02-23T00:51:10.297367+0000","last_undegraded":"2024-02-23T00:51:10.297367+0000","last_fullsized":"2024-02-23T00:51:10.297367+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:34.600592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_clean_scrub_stamp":"2024-02-23T00:50:34.600592+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":504,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.293703+0000","last_change":"2024-02-23T00:49:27.459981+0000","last_active":"2024-02-23T00:51:10.293703+0000","last_peered":"2024-02-23T00:51:10.293703+0000","last_clean":"2024-02-23T00:51:10.293703+0000","last_became_active":"2024-02-23T00:39:18.799723+0000","last_became_peered":"2024-02-23T00:39:18.799723+0000","last_unstale":"2024-02-23T00:51:10.293703+0000","last_undegraded":"2024-02-23T00:51:10.293703+0000","last_fullsized":"2024-02-23T00:51:10.293703+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:27.459893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:27.459893+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":455,"reported_epoch":541,"state":"active+clean","last_fresh":"2024-02-23T00:51:10.279960+0000","last_change":"2024-02-23T00:49:44.601162+0000","last_active":"2024-02-23T00:51:10.279960+0000","last_peered":"2024-02-23T00:51:10.279960+0000","last_clean":"2024-02-23T00:51:10.279960+0000","last_became_active":"2024-02-23T00:42:58.824573+0000","last_became_peered":"2024-02-23T00:42:58.824573+0000","last_unstale":"2024-02-23T00:51:10.279960+0000","last_undegraded":"2024-02-23T00:51:10.279960+0000","last_fullsized":"2024-02-23T00:51:10.279960+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:44.601086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:44.601086+0000","last_clean_scrub_stamp":"2024-02-23T00:49:44.601086+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0],"acting":[5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":62,"num_pg":32,"stat_sum":{"num_bytes":4020435,"num_objects":192,"num_object_clones":11,"num_object_copies":576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":192,"num_whiteouts":0,"num_read":212,"num_read_kb":57,"num_write":777,"num_write_kb":4619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":853,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":21,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":14331904,"data_stored":12061457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2083,"ondisk_log_size":2083,"up":96,"acting":96,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":12,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":36,"acting":36,"num_store_stats":8},{"poolid":227,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1508,"num_read_kb":1233,"num_write":895,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":1486,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":955,"ondisk_log_size":955,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":707,"num_read_kb":702,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234131,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":316052,"kb_used_data":6540,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93429100,"statfs":{"total":95995035648,"available":95671398400,"internally_reserved":0,"allocated":6696960,"data_stored":4887836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:50:32 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51500000000000001,"15min":0.53500000000000003},"min":{"1min":0.29499999999999998,"5min":0.27300000000000002,"15min":0.26900000000000002},"max":{"1min":0.64100000000000001,"5min":0.76400000000000001,"15min":4.9420000000000002},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.376,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.78400000000000003,"5min":1.151,"15min":4.9729999999999999},"last":0.56599999999999995}]},{"osd":1,"last update":"Fri Feb 23 00:50:32 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.52500000000000002,"15min":0.61699999999999999},"min":{"1min":0.30599999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.79700000000000004,"5min":1.25,"15min":10.786},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52300000000000002,"15min":0.62},"min":{"1min":0.39100000000000001,"5min":0.317,"15min":0.28100000000000003},"max":{"1min":0.78400000000000003,"5min":1.107,"15min":11.885999999999999},"last":0.59399999999999997}]},{"osd":2,"last update":"Fri Feb 23 00:50:32 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.52700000000000002,"15min":0.55400000000000005},"min":{"1min":0.377,"5min":0.36099999999999999,"15min":0.28999999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":5.0789999999999997},"last":0.497},{"interface":"front","average":{"1min":0.497,"5min":0.53000000000000003,"15min":0.54600000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.316},"max":{"1min":0.66900000000000004,"5min":1.1180000000000001,"15min":5.1280000000000001},"last":0.58399999999999996}]},{"osd":3,"last update":"Fri Feb 23 00:50:32 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52300000000000002,"15min":0.61899999999999999},"min":{"1min":0.35399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.75800000000000001,"5min":1.095,"15min":15.712999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.55800000000000005,"15min":0.61499999999999999},"min":{"1min":0.40200000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.76800000000000002,"5min":1.1659999999999999,"15min":9.4090000000000007},"last":0.51200000000000001}]},{"osd":4,"last update":"Fri Feb 23 00:50:32 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.439,"15min":0.439},"min":{"1min":0.31,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.63600000000000001,"5min":1.6819999999999999,"15min":2.0350000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.441,"5min":0.46100000000000002,"15min":0.48799999999999999},"min":{"1min":0.32000000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.63600000000000001,"5min":0.86699999999999999,"15min":11.236000000000001},"last":0.39400000000000002}]},{"osd":5,"last update":"Fri Feb 23 00:50:32 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48099999999999998,"15min":0.49199999999999999},"min":{"1min":0.32300000000000001,"5min":0.27700000000000002,"15min":0.27000000000000002},"max":{"1min":0.72299999999999998,"5min":0.73199999999999998,"15min":5.4000000000000004},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.50900000000000001,"15min":0.497},"min":{"1min":0.35199999999999998,"5min":0.29099999999999998,"15min":0.26700000000000002},"max":{"1min":0.69499999999999995,"5min":1.728,"15min":1.728},"last":0.62}]},{"osd":6,"last update":"Fri Feb 23 00:50:32 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51500000000000001,"15min":0.51000000000000001},"min":{"1min":0.36099999999999999,"5min":0.32200000000000001,"15min":0.317},"max":{"1min":0.67800000000000005,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52000000000000002,"15min":0.54800000000000004},"min":{"1min":0.34000000000000002,"5min":0.32700000000000001,"15min":0.32400000000000001},"max":{"1min":0.70499999999999996,"5min":1.2370000000000001,"15min":13.395},"last":0.53600000000000003}]}]},{"osd":6,"up_from":47,"seq":201863463066,"num_pgs":92,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":307948,"kb_used_data":2532,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93437204,"statfs":{"total":95995035648,"available":95679696896,"internally_reserved":0,"allocated":2592768,"data_stored":882638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:51:07 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.505,"15min":0.68200000000000005},"min":{"1min":0.22500000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":30.641999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.53600000000000003,"15min":0.69699999999999995},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.577,"5min":3.7360000000000002,"15min":29.292000000000002},"last":0.47899999999999998}]},{"osd":1,"last update":"Fri Feb 23 00:51:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51500000000000001,"15min":0.68300000000000005},"min":{"1min":0.308,"5min":0.247,"15min":0.247},"max":{"1min":0.77100000000000002,"5min":0.77200000000000002,"15min":28.186},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.52900000000000003,"15min":0.86299999999999999},"min":{"1min":0.39400000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.752,"5min":1.083,"15min":33.104999999999997},"last":0.53400000000000003}]},{"osd":2,"last update":"Fri Feb 23 00:51:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52700000000000002,"15min":0.71399999999999997},"min":{"1min":0.36699999999999999,"5min":0.29399999999999998,"15min":0.27400000000000002},"max":{"1min":0.69599999999999995,"5min":1.1160000000000001,"15min":19.960000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53400000000000003,"15min":0.64600000000000002},"min":{"1min":0.40400000000000003,"5min":0.316,"15min":0.29899999999999999},"max":{"1min":0.79000000000000004,"5min":1.665,"15min":8.5860000000000003},"last":0.499}]},{"osd":3,"last update":"Fri Feb 23 00:51:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54300000000000004,"15min":0.626},"min":{"1min":0.40999999999999998,"5min":0.29199999999999998,"15min":0.28599999999999998},"max":{"1min":0.78000000000000003,"5min":0.78400000000000003,"15min":19.562000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56899999999999995,"15min":0.67000000000000004},"min":{"1min":0.42399999999999999,"5min":0.33500000000000002,"15min":0.33000000000000002},"max":{"1min":0.80700000000000005,"5min":2.6259999999999999,"15min":14.199},"last":0.438}]},{"osd":4,"last update":"Fri Feb 23 00:51:07 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.44700000000000001,"15min":0.47999999999999998},"min":{"1min":0.317,"5min":0.245,"15min":0.218},"max":{"1min":0.61499999999999999,"5min":1.6220000000000001,"15min":11.109999999999999},"last":0.376},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.45900000000000002,"15min":0.54800000000000004},"min":{"1min":0.32700000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.751,"5min":0.83099999999999996,"15min":16.734000000000002},"last":0.39300000000000002}]},{"osd":5,"last update":"Fri Feb 23 00:51:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48299999999999998,"15min":0.51300000000000001},"min":{"1min":0.34499999999999997,"5min":0.308,"15min":0.249},"max":{"1min":0.72399999999999998,"5min":1.784,"15min":14.154999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.58499999999999996,"15min":0.55500000000000005},"min":{"1min":0.37,"5min":0.318,"15min":0.26400000000000001},"max":{"1min":0.68500000000000005,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.57499999999999996}]},{"osd":7,"last update":"Fri Feb 23 00:50:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.51400000000000001,"15min":0.52900000000000003},"min":{"1min":0.36499999999999999,"5min":0.29399999999999998,"15min":0.26800000000000002},"max":{"1min":0.67600000000000005,"5min":1.726,"15min":8.548},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54600000000000004,"15min":0.60399999999999998},"min":{"1min":0.36399999999999999,"5min":0.34200000000000003,"15min":0.309},"max":{"1min":0.749,"5min":0.88500000000000001,"15min":15.618},"last":0.45500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575028,"num_pgs":84,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":311876,"kb_used_data":6460,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93433276,"statfs":{"total":95995035648,"available":95675674624,"internally_reserved":0,"allocated":6615040,"data_stored":4892784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.39000000000000001,"15min":0.68400000000000005},"min":{"1min":0.23799999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.56999999999999995,"5min":6.202,"15min":78.361000000000004},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.379,"5min":0.36899999999999999,"15min":0.44400000000000001},"min":{"1min":0.26900000000000002,"5min":0.216,"15min":0.191},"max":{"1min":0.64300000000000002,"5min":0.69999999999999996,"15min":5.8849999999999998},"last":0.28499999999999998}]},{"osd":2,"last update":"Fri Feb 23 00:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.41699999999999998,"15min":0.66700000000000004},"min":{"1min":0.22700000000000001,"5min":0.20499999999999999,"15min":0.188},"max":{"1min":7.5789999999999997,"5min":7.5789999999999997,"15min":48.606000000000002},"last":0.379},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.40500000000000003,"15min":0.57699999999999996},"min":{"1min":0.28000000000000003,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.63,"5min":1.3460000000000001,"15min":23.911000000000001},"last":0.40200000000000002}]},{"osd":3,"last update":"Fri Feb 23 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.40500000000000003,"15min":0.59499999999999997},"min":{"1min":0.27800000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":35.244999999999997},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.437,"5min":0.47699999999999998,"15min":0.68700000000000006},"min":{"1min":0.28899999999999998,"5min":0.26900000000000002,"15min":0.248},"max":{"1min":0.73999999999999999,"5min":7.641,"15min":56.853999999999999},"last":0.30099999999999999}]},{"osd":4,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.63300000000000001,"15min":0.82299999999999995},"min":{"1min":0.40100000000000002,"5min":0.32700000000000001,"15min":0.29299999999999998},"max":{"1min":0.97999999999999998,"5min":7.6829999999999998,"15min":37.646999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.64700000000000002,"15min":0.70799999999999996},"min":{"1min":0.48099999999999998,"5min":0.35899999999999999,"15min":0.31900000000000001},"max":{"1min":0.91400000000000003,"5min":6.0519999999999996,"15min":9.6859999999999999},"last":0.64300000000000002}]},{"osd":5,"last update":"Fri Feb 23 00:50:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.65300000000000002,"15min":0.84099999999999997},"min":{"1min":0.40600000000000003,"5min":0.309,"15min":0.309},"max":{"1min":7.742,"5min":7.742,"15min":48.792000000000002},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.61899999999999999,"15min":0.71899999999999997},"min":{"1min":0.45200000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":11.872999999999999},"last":0.68000000000000005}]},{"osd":6,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.64100000000000001,"15min":0.82199999999999995},"min":{"1min":0.40000000000000002,"5min":0.37,"15min":0.318},"max":{"1min":0.94199999999999995,"5min":7.7130000000000001,"15min":40.741},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60399999999999998,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.38400000000000001,"15min":0.34000000000000002},"max":{"1min":0.82199999999999995,"5min":0.85999999999999999,"15min":6.0069999999999997},"last":0.60299999999999998}]},{"osd":7,"last update":"Fri Feb 23 00:50:31 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.60699999999999998,"15min":0.71399999999999997},"min":{"1min":0.45300000000000001,"5min":0.39300000000000002,"15min":0.36699999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":20.225000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.66000000000000003,"15min":0.74099999999999999},"min":{"1min":0.46200000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":6.0949999999999998,"5min":6.0949999999999998,"15min":20.254999999999999},"last":0.70799999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738554,"num_pgs":74,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":307932,"kb_used_data":2516,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93437220,"statfs":{"total":95995035648,"available":95679713280,"internally_reserved":0,"allocated":2576384,"data_stored":881318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 23 00:50:52 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.35199999999999998,"15min":0.60299999999999998},"min":{"1min":0.224,"5min":0.20999999999999999,"15min":0.19500000000000001},"max":{"1min":0.68200000000000005,"5min":0.81399999999999995,"15min":28.856999999999999},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.373,"5min":0.39500000000000002,"15min":0.629},"min":{"1min":0.249,"5min":0.23300000000000001,"15min":0.20799999999999999},"max":{"1min":0.59099999999999997,"5min":2.3650000000000002,"15min":20.577000000000002},"last":0.34499999999999997}]},{"osd":2,"last update":"Fri Feb 23 00:50:15 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.40200000000000002,"15min":0.61099999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.20499999999999999},"max":{"1min":2.3889999999999998,"5min":2.3889999999999998,"15min":13.863},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.34999999999999998,"15min":0.54100000000000004},"min":{"1min":0.245,"5min":0.219,"15min":0.19800000000000001},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":10.455},"last":0.31}]},{"osd":3,"last update":"Fri Feb 23 00:50:46 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.377,"15min":0.77900000000000003},"min":{"1min":0.246,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.64300000000000002,"5min":0.73099999999999998,"15min":35.566000000000003},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.438,"15min":0.76900000000000002},"min":{"1min":0.27700000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.66700000000000004,"5min":2.4049999999999998,"15min":20.702999999999999},"last":0.39200000000000002}]},{"osd":4,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55900000000000005,"15min":0.752},"min":{"1min":0.40899999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.755,"5min":0.94699999999999995,"15min":10.584},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.61599999999999999,"15min":0.78100000000000003},"min":{"1min":0.42899999999999999,"5min":0.38100000000000001,"15min":0.35099999999999998},"max":{"1min":0.78900000000000003,"5min":2.5019999999999998,"15min":13.198},"last":0.495}]},{"osd":5,"last update":"Fri Feb 23 00:50:35 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54500000000000004,"15min":0.72099999999999997},"min":{"1min":0.34799999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81799999999999995,"5min":0.90800000000000003,"15min":10.397},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61299999999999999,"15min":0.78800000000000003},"min":{"1min":0.40999999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":17.297000000000001},"last":0.56299999999999994}]},{"osd":6,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.58899999999999997,"15min":0.78000000000000003},"min":{"1min":0.39400000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.76400000000000001,"5min":0.94399999999999995,"15min":17.373999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.61299999999999999,"15min":0.76200000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.36799999999999999},"max":{"1min":0.72199999999999998,"5min":2.556,"15min":13.973000000000001},"last":0.54500000000000004}]},{"osd":7,"last update":"Fri Feb 23 00:50:27 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59899999999999998,"15min":0.78600000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.95199999999999996,"5min":0.96099999999999997,"15min":17.399999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.64500000000000002,"15min":0.78700000000000003},"min":{"1min":0.433,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93400000000000005,"5min":2.5289999999999999,"15min":16.427},"last":0.58799999999999997}]}]},{"osd":2,"up_from":0,"seq":77309411503,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":26,"seq":111669149864,"num_pgs":90,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":312096,"kb_used_data":2584,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93433056,"statfs":{"total":95995035648,"available":95675449344,"internally_reserved":0,"allocated":2646016,"data_stored":894963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:50:45 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.441,"15min":0.53800000000000003},"min":{"1min":0.246,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":8.3300000000000001,"5min":8.3300000000000001,"15min":12.212999999999999},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.499,"15min":0.61099999999999999},"min":{"1min":0.23100000000000001,"5min":0.22800000000000001,"15min":0.21199999999999999},"max":{"1min":2.9740000000000002,"5min":5.9800000000000004,"15min":11.714},"last":0.35099999999999998}]},{"osd":1,"last update":"Fri Feb 23 00:50:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55700000000000005,"15min":0.63200000000000001},"min":{"1min":0.27200000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":3.056,"5min":6.851,"15min":11.742000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.58099999999999996,"15min":0.60199999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.19900000000000001},"max":{"1min":2.7709999999999999,"5min":8.2270000000000003,"15min":8.2270000000000003},"last":0.32300000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:50:45 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.625,"15min":0.68999999999999995},"min":{"1min":0.29199999999999998,"5min":0.255,"15min":0.255},"max":{"1min":3.242,"5min":8.0269999999999992,"15min":13.359999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.52600000000000002,"15min":0.66600000000000004},"min":{"1min":0.27600000000000002,"5min":0.27500000000000002,"15min":0.26100000000000001},"max":{"1min":8.5820000000000007,"5min":8.5820000000000007,"15min":21.122},"last":0.38400000000000001}]},{"osd":4,"last update":"Fri Feb 23 00:51:10 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.58899999999999997,"15min":0.755},"min":{"1min":0.40300000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":3.1419999999999999,"5min":3.1419999999999999,"15min":11.644},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.74199999999999999,"15min":0.753},"min":{"1min":0.36899999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":2.9020000000000001,"5min":12.430999999999999,"15min":12.430999999999999},"last":0.46500000000000002}]},{"osd":5,"last update":"Fri Feb 23 00:50:35 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":0.69899999999999995,"15min":0.68899999999999995},"min":{"1min":0.375,"5min":0.37,"15min":0.35099999999999998},"max":{"1min":10.231999999999999,"5min":10.231999999999999,"15min":10.231999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.76200000000000001,"15min":0.80100000000000005},"min":{"1min":0.38300000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":3.1829999999999998,"5min":11.648,"15min":11.648},"last":0.56699999999999995}]},{"osd":6,"last update":"Fri Feb 23 00:50:53 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76300000000000001,"15min":0.75600000000000001},"min":{"1min":0.42399999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.9510000000000001,"5min":6.9420000000000002,"15min":6.9420000000000002},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.2050000000000001,"5min":0.751,"15min":0.72299999999999998},"min":{"1min":0.39200000000000002,"5min":0.376,"15min":0.376},"max":{"1min":10.965999999999999,"5min":10.965999999999999,"15min":10.965999999999999},"last":0.61299999999999999}]},{"osd":7,"last update":"Fri Feb 23 00:50:29 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.77500000000000002,"15min":0.82399999999999995},"min":{"1min":0.44500000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.3160000000000001,"5min":11.676,"15min":11.676},"last":0.63},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.79300000000000004,"15min":0.80200000000000005},"min":{"1min":0.47699999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80900000000000005,"5min":7.9800000000000004,"15min":7.9800000000000004},"last":0.68899999999999995}]}]},{"osd":4,"up_from":32,"seq":137438953636,"num_pgs":85,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":311856,"kb_used_data":6440,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93433296,"statfs":{"total":95995035648,"available":95675695104,"internally_reserved":0,"allocated":6594560,"data_stored":4890105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.504,"15min":0.52600000000000002},"min":{"1min":0.32200000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.65300000000000002,"5min":1.3819999999999999,"15min":3.5219999999999998},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.52400000000000002,"15min":0.58499999999999996},"min":{"1min":0.373,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.72799999999999998,"5min":1.8,"15min":8.6140000000000008},"last":0.51100000000000001}]},{"osd":1,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51600000000000001,"15min":0.57999999999999996},"min":{"1min":0.31900000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.70699999999999996,"5min":2.6600000000000001,"15min":9.6110000000000007},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.53500000000000003,"15min":0.58499999999999996},"min":{"1min":0.31900000000000001,"5min":0.26300000000000001,"15min":0.255},"max":{"1min":0.70099999999999996,"5min":2.6920000000000002,"15min":6.6950000000000003},"last":0.377}]},{"osd":2,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.51900000000000002,"15min":0.56999999999999995},"min":{"1min":0.34699999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.65700000000000003,"5min":1.0960000000000001,"15min":3.3069999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54000000000000004,"15min":0.57099999999999995},"min":{"1min":0.33700000000000002,"5min":0.33600000000000002,"15min":0.32500000000000001},"max":{"1min":0.68700000000000006,"5min":1.0189999999999999,"15min":5.8440000000000003},"last":0.61399999999999999}]},{"osd":3,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.67300000000000004,"15min":0.63400000000000001},"min":{"1min":0.38500000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.71999999999999997,"5min":9.1280000000000001,"15min":9.1280000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.58199999999999996,"15min":0.60399999999999998},"min":{"1min":0.39200000000000002,"5min":0.34599999999999997,"15min":0.29699999999999999},"max":{"1min":0.73299999999999998,"5min":4.0220000000000002,"15min":4.1909999999999998},"last":0.47899999999999998}]},{"osd":5,"last update":"Fri Feb 23 00:50:35 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.45600000000000002,"15min":0.41599999999999998},"min":{"1min":0.26700000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.51500000000000001,"15min":0.47199999999999998},"min":{"1min":0.35399999999999998,"5min":0.29499999999999998,"15min":0.28899999999999998},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.49299999999999999}]},{"osd":6,"last update":"Fri Feb 23 00:51:05 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.44900000000000001,"15min":0.435},"min":{"1min":0.34000000000000002,"5min":0.28399999999999997,"15min":0.253},"max":{"1min":0.67200000000000004,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47199999999999998,"15min":0.443},"min":{"1min":0.34699999999999998,"5min":0.28799999999999998,"15min":0.215},"max":{"1min":0.64100000000000001,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":0.57199999999999995}]},{"osd":7,"last update":"Fri Feb 23 00:50:31 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.497,"15min":0.48599999999999999},"min":{"1min":0.30599999999999999,"5min":0.26700000000000002,"15min":0.254},"max":{"1min":2.294,"5min":2.294,"15min":2.294},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.379,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":0.63}]}]},{"osd":5,"up_from":39,"seq":167503724703,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":307904,"kb_used_data":2488,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93437248,"statfs":{"total":95995035648,"available":95679741952,"internally_reserved":0,"allocated":2547712,"data_stored":890724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:50:27 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.54500000000000004,"15min":0.61399999999999999},"min":{"1min":0.35799999999999998,"5min":0.253,"15min":0.246},"max":{"1min":5.2220000000000004,"5min":5.2220000000000004,"15min":23.881},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.57199999999999995,"15min":0.59399999999999997},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.27000000000000002},"max":{"1min":3.9249999999999998,"5min":3.9249999999999998,"15min":5.4649999999999999},"last":0.55700000000000005}]},{"osd":1,"last update":"Fri Feb 23 00:50:27 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.55200000000000005,"15min":0.65700000000000003},"min":{"1min":0.41299999999999998,"5min":0.254,"15min":0.254},"max":{"1min":3.9460000000000002,"5min":3.9460000000000002,"15min":18.175999999999998},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.56699999999999995,"15min":0.63100000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":3.976,"5min":3.976,"15min":9.0950000000000006},"last":0.40100000000000002}]},{"osd":2,"last update":"Fri Feb 23 00:50:27 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.56699999999999995,"15min":0.71099999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":4.5899999999999999,"5min":4.5899999999999999,"15min":22.282},"last":0.37},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.59099999999999997,"15min":0.72699999999999998},"min":{"1min":0.36699999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":3.8540000000000001,"5min":3.8540000000000001,"15min":23.925000000000001},"last":0.53600000000000003}]},{"osd":3,"last update":"Fri Feb 23 00:50:27 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54800000000000004,"15min":0.61899999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.1060000000000001,"5min":1.23,"15min":6.5730000000000004},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.60399999999999998,"15min":0.65500000000000003},"min":{"1min":0.41799999999999998,"5min":0.378,"15min":0.33500000000000002},"max":{"1min":3.9980000000000002,"5min":3.9980000000000002,"15min":8.1760000000000002},"last":0.61199999999999999}]},{"osd":4,"last update":"Fri Feb 23 00:50:27 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.48399999999999999,"15min":0.43099999999999999},"min":{"1min":0.29999999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.68100000000000005,"5min":4.593,"15min":4.593},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.47899999999999998,"15min":0.46300000000000002},"min":{"1min":0.27900000000000003,"5min":0.27700000000000002,"15min":0.27300000000000002},"max":{"1min":0.68400000000000005,"5min":1.5860000000000001,"15min":3.879},"last":0.46200000000000002}]},{"osd":6,"last update":"Fri Feb 23 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51300000000000001,"15min":0.47599999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.27900000000000003},"max":{"1min":0.71099999999999997,"5min":4.6740000000000004,"15min":4.6740000000000004},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.49099999999999999,"15min":0.47699999999999998},"min":{"1min":0.35999999999999999,"5min":0.28000000000000003,"15min":0.27100000000000002},"max":{"1min":0.73199999999999998,"5min":1.159,"15min":1.159},"last":0.57299999999999995}]},{"osd":7,"last update":"Fri Feb 23 00:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54100000000000004,"15min":0.54200000000000004},"min":{"1min":0.34200000000000003,"5min":0.32500000000000001,"15min":0.30399999999999999},"max":{"1min":0.79200000000000004,"5min":1.302,"15min":7.0300000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.52800000000000002,"15min":0.53800000000000003},"min":{"1min":0.36699999999999999,"5min":0.314,"15min":0.29099999999999998},"max":{"1min":0.73599999999999999,"5min":2.016,"15min":9.0579999999999998},"last":0.505}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":303104,"data_stored":6419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4317184,"data_stored":4007883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":68,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":344064,"data_stored":11278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4308992,"data_stored":4006776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":6143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":331776,"data_stored":8750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4444160,"data_stored":4014140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-23T00:51:17.795 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph status --format=json 2024-02-23T00:51:17.877 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:51:17.909 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:17.930 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:51:17.933 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:17.934 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:51:18.034 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:51:18.062 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:18.287 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:17.252505+0000 mon.a (mon.0) 2950 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-90","var": "hit_set_count","val": "2"}]': finished 2024-02-23T00:51:18.288 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:17.252565+0000 mon.a (mon.0) 2951 : cluster [DBG] osdmap e548: 8 total, 8 up, 7 in 2024-02-23T00:51:18.288 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:17.254783+0000 mon.a (mon.0) 2952 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-90","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:51:18.288 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:17.340184+0000 mgr.y (mgr.14517) 501 : cluster [DBG] pgmap v826: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:18.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:17.252505+0000 mon.a (mon.0) 2950 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-90","var": "hit_set_count","val": "2"}]': finished 2024-02-23T00:51:18.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:17.252565+0000 mon.a (mon.0) 2951 : cluster [DBG] osdmap e548: 8 total, 8 up, 7 in 2024-02-23T00:51:18.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:17.254783+0000 mon.a (mon.0) 2952 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-90","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:51:18.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:17.340184+0000 mgr.y (mgr.14517) 501 : cluster [DBG] pgmap v826: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:18.612 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:51:18.615 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:18.616 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:51:18.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:17.252505+0000 mon.a (mon.0) 2950 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-90","var": "hit_set_count","val": "2"}]': finished 2024-02-23T00:51:18.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:17.252565+0000 mon.a (mon.0) 2951 : cluster [DBG] osdmap e548: 8 total, 8 up, 7 in 2024-02-23T00:51:18.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:17.254783+0000 mon.a (mon.0) 2952 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-90","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:51:18.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:17.340184+0000 mgr.y (mgr.14517) 501 : cluster [DBG] pgmap v826: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:18.716 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:51:18.748 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:18.769 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:51:18.772 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:18.772 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:51:18.863 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:51:18.863 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:51:18.873 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:51:18.904 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:19.154 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:51:19.157 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:19.157 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:51:19.258 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:51:19.288 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:19.308 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:51:19.311 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:19.311 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:51:19.412 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:51:19.438 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:18.258138+0000 mon.a (mon.0) 2953 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-90","var": "hit_set_period","val": "600"}]': finished 2024-02-23T00:51:19.439 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:18.258230+0000 mon.a (mon.0) 2954 : cluster [DBG] osdmap e549: 8 total, 8 up, 7 in 2024-02-23T00:51:19.439 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:18.260730+0000 mon.a (mon.0) 2955 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-90","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-23T00:51:19.439 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:18.258138+0000 mon.a (mon.0) 2953 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-90","var": "hit_set_period","val": "600"}]': finished 2024-02-23T00:51:19.439 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:18.258230+0000 mon.a (mon.0) 2954 : cluster [DBG] osdmap e549: 8 total, 8 up, 7 in 2024-02-23T00:51:19.439 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:18.260730+0000 mon.a (mon.0) 2955 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-90","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-23T00:51:19.441 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:19.564 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.4 dump_ops_in_flight 2024-02-23T00:51:19.565 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:51:19.567 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:19.568 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:51:19.596 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:18.258138+0000 mon.a (mon.0) 2953 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-90","var": "hit_set_period","val": "600"}]': finished 2024-02-23T00:51:19.596 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:18.258230+0000 mon.a (mon.0) 2954 : cluster [DBG] osdmap e549: 8 total, 8 up, 7 in 2024-02-23T00:51:19.596 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:18.260730+0000 mon.a (mon.0) 2955 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-90","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-23T00:51:19.669 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:51:19.697 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:19.716 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:51:19.719 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:19.719 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:51:19.820 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:51:19.840 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:51:19.849 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:19.869 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:51:19.872 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:19.872 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:51:19.973 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:51:20.002 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:20.022 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:51:20.028 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:20.028 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:51:20.129 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:51:20.158 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:20.178 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:51:20.180 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:20.181 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:51:20.282 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:51:20.309 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:20.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:19.254832+0000 mon.a (mon.0) 2956 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:51:20.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:19.259052+0000 mon.a (mon.0) 2957 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-90","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-23T00:51:20.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:19.259120+0000 mon.a (mon.0) 2958 : cluster [DBG] osdmap e550: 8 total, 8 up, 7 in 2024-02-23T00:51:20.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:19.261638+0000 mon.a (mon.0) 2959 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-90","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-23T00:51:20.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:19.340614+0000 mgr.y (mgr.14517) 502 : cluster [DBG] pgmap v829: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:20.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:20.260103+0000 mon.a (mon.0) 2960 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-90","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-23T00:51:20.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:20.260176+0000 mon.a (mon.0) 2961 : cluster [DBG] osdmap e551: 8 total, 8 up, 7 in 2024-02-23T00:51:20.552 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:51:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:51:20] "GET /metrics HTTP/1.1" 200 187230 "" "Prometheus/2.33.4" 2024-02-23T00:51:20.552 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:19.254832+0000 mon.a (mon.0) 2956 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:51:20.552 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:19.259052+0000 mon.a (mon.0) 2957 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-90","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-23T00:51:20.552 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:19.259120+0000 mon.a (mon.0) 2958 : cluster [DBG] osdmap e550: 8 total, 8 up, 7 in 2024-02-23T00:51:20.552 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:19.261638+0000 mon.a (mon.0) 2959 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-90","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-23T00:51:20.552 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:19.340614+0000 mgr.y (mgr.14517) 502 : cluster [DBG] pgmap v829: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:20.552 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:20.260103+0000 mon.a (mon.0) 2960 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-90","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-23T00:51:20.553 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:20.260176+0000 mon.a (mon.0) 2961 : cluster [DBG] osdmap e551: 8 total, 8 up, 7 in 2024-02-23T00:51:20.553 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:19.254832+0000 mon.a (mon.0) 2956 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:51:20.553 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:19.259052+0000 mon.a (mon.0) 2957 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-90","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-23T00:51:20.553 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:19.259120+0000 mon.a (mon.0) 2958 : cluster [DBG] osdmap e550: 8 total, 8 up, 7 in 2024-02-23T00:51:20.553 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:19.261638+0000 mon.a (mon.0) 2959 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-90","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-23T00:51:20.553 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:19.340614+0000 mgr.y (mgr.14517) 502 : cluster [DBG] pgmap v829: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:20.553 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:20.260103+0000 mon.a (mon.0) 2960 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-90","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-23T00:51:20.553 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:20.260176+0000 mon.a (mon.0) 2961 : cluster [DBG] osdmap e551: 8 total, 8 up, 7 in 2024-02-23T00:51:20.557 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:51:20.560 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:20.560 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:51:20.589 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:51:20.636 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-23T00:51:20.662 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:51:20.707 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:20.831 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:51:20.834 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:20.835 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:51:20.936 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:51:20.965 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:21.038 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:51:21.042 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:21.042 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:51:21.143 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:51:21.172 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:21.311 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:20.286373+0000 mon.a (mon.0) 2962 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:51:21.320 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:51:21.325 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:21.325 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:51:21.426 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:51:21.456 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:21.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:20.286373+0000 mon.a (mon.0) 2962 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:51:21.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:20.286373+0000 mon.a (mon.0) 2962 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:51:21.594 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:51:21.595 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-02-23T00:51:21.595 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 0 2024-02-23T00:51:21.595 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:51:21.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.309+0000 7efec9d77700 1 -- 172.21.15.47:0/1010175611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efec40f8440 msgr2=0x7efec4104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:21.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.309+0000 7efec9d77700 1 --2- 172.21.15.47:0/1010175611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efec40f8440 0x7efec4104b10 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 rx=0x7efebc009c30 tx=0x7efebc004c50).stop 2024-02-23T00:51:21.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.309+0000 7efec9d77700 1 -- 172.21.15.47:0/1010175611 shutdown_connections 2024-02-23T00:51:21.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.309+0000 7efec9d77700 1 --2- 172.21.15.47:0/1010175611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efec40f8440 0x7efec4104b10 unknown :-1 s=CLOSED pgs=2572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:21.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.309+0000 7efec9d77700 1 --2- 172.21.15.47:0/1010175611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efec40f9ab0 0x7efec40f9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:21.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.309+0000 7efec9d77700 1 --2- 172.21.15.47:0/1010175611 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efec40fa380 0x7efec40f7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:21.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.309+0000 7efec9d77700 1 -- 172.21.15.47:0/1010175611 >> 172.21.15.47:0/1010175611 conn(0x7efec40f3800 msgr2=0x7efec40f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:21.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.309+0000 7efec9d77700 1 -- 172.21.15.47:0/1010175611 shutdown_connections 2024-02-23T00:51:21.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.309+0000 7efec9d77700 1 -- 172.21.15.47:0/1010175611 wait complete. 2024-02-23T00:51:21.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.309+0000 7efec9d77700 1 librados: starting msgr at 2024-02-23T00:51:21.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.309+0000 7efec9d77700 1 librados: starting objecter 2024-02-23T00:51:21.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.310+0000 7efec9d77700 1 Processor -- start 2024-02-23T00:51:21.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.310+0000 7efec9d77700 1 -- start start 2024-02-23T00:51:21.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.310+0000 7efec9d77700 1 librados: setting wanted keys 2024-02-23T00:51:21.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.310+0000 7efec9d77700 1 librados: calling monclient init 2024-02-23T00:51:21.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.310+0000 7efec9d77700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efec40f8440 0x7efec406b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:21.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.310+0000 7efec9d77700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efec40f9ab0 0x7efec406b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:21.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.310+0000 7efec9d77700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efec40fa380 0x7efec406bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:21.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.310+0000 7efec9d77700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec4116350 con 0x7efec40fa380 2024-02-23T00:51:21.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.310+0000 7efec9d77700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec41164c0 con 0x7efec40f8440 2024-02-23T00:51:21.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.310+0000 7efec9d77700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec4116680 con 0x7efec40f9ab0 2024-02-23T00:51:21.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.310+0000 7efec3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efec40fa380 0x7efec406bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:21.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.310+0000 7efec2ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efec40f9ab0 0x7efec406b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:21.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.310+0000 7efec3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efec40fa380 0x7efec406bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41724/0 (socket says 172.21.15.47:41724) 2024-02-23T00:51:21.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.310+0000 7efec3fff700 1 -- 172.21.15.47:0/3102566360 learned_addr learned my addr 172.21.15.47:0/3102566360 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:51:21.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.310+0000 7efec37fe700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efec40f8440 0x7efec406b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:21.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.310+0000 7efec3fff700 1 -- 172.21.15.47:0/3102566360 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efec40f9ab0 msgr2=0x7efec406b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:21.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.310+0000 7efec3fff700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efec40f9ab0 0x7efec406b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:21.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.310+0000 7efec3fff700 1 -- 172.21.15.47:0/3102566360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efec40f8440 msgr2=0x7efec406b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:21.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.310+0000 7efec3fff700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efec40f8440 0x7efec406b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:21.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.311+0000 7efec3fff700 1 -- 172.21.15.47:0/3102566360 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb8005040 con 0x7efec40fa380 2024-02-23T00:51:21.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.311+0000 7efec3fff700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efec40fa380 0x7efec406bab0 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 rx=0x7efebc00bf80 tx=0x7efebc004c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:21.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.311+0000 7efec0ff9700 10 client.?.objecter ms_handle_connect 0x7efec40fa380 2024-02-23T00:51:21.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.311+0000 7efec0ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:51:21.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.311+0000 7efec0ff9700 1 -- 172.21.15.47:0/3102566360 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc00e040 con 0x7efec40fa380 2024-02-23T00:51:21.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.311+0000 7efec0ff9700 1 -- 172.21.15.47:0/3102566360 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7efebc018040 con 0x7efec40fa380 2024-02-23T00:51:21.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.311+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efebc0097c0 con 0x7efec40fa380 2024-02-23T00:51:21.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.311+0000 7efec0ff9700 1 -- 172.21.15.47:0/3102566360 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc013cf0 con 0x7efec40fa380 2024-02-23T00:51:21.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.311+0000 7efec9d77700 10 client.31326.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:51:21.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.311+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec4116bf0 con 0x7efec40fa380 2024-02-23T00:51:21.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.311+0000 7efec9d77700 1 librados: init done 2024-02-23T00:51:21.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.312+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7efec405ee80 con 0x7efec40fa380 2024-02-23T00:51:21.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.312+0000 7efec0ff9700 1 -- 172.21.15.47:0/3102566360 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7efebc013420 con 0x7efec40fa380 2024-02-23T00:51:21.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.313+0000 7efec0ff9700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7efeac062cc0 0x7efeac065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:21.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.313+0000 7efec0ff9700 1 -- 172.21.15.47:0/3102566360 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(552..552 src has 463..552) v4 ==== 9228+0+0 (secure 0 0 0) 0x7efebc07ede0 con 0x7efec40fa380 2024-02-23T00:51:21.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.313+0000 7efec0ff9700 10 client.31326.objecter ms_dispatch 0x7efec4001550 osd_map(552..552 src has 463..552) v4 2024-02-23T00:51:21.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.313+0000 7efec0ff9700 3 client.31326.objecter handle_osd_map got epochs [552,552] > 0 2024-02-23T00:51:21.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.313+0000 7efec0ff9700 3 client.31326.objecter handle_osd_map decoding full epoch 552 2024-02-23T00:51:21.717 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:51:21.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.313+0000 7efec37fe700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7efeac062cc0 0x7efeac065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:21.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.313+0000 7efec0ff9700 20 client.31326.objecter dump_active .. 0 homeless 2024-02-23T00:51:21.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.313+0000 7efec0ff9700 1 -- 172.21.15.47:0/3102566360 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v552) v1 ==== 54+0+16 (secure 0 0 0) 0x7efebc07f2a0 con 0x7efec40fa380 2024-02-23T00:51:21.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.313+0000 7efec0ff9700 10 client.31326.objecter ms_handle_connect 0x7efeac062cc0 2024-02-23T00:51:21.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.313+0000 7efec37fe700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7efeac062cc0 0x7efeac065180 secure :-1 s=READY pgs=3117 cs=0 l=1 rev1=1 rx=0x7efeb40048b0 tx=0x7efeb400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:21.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.313+0000 7efec9d77700 10 client.31326.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:21.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.313+0000 7efec9d77700 15 client.31326.objecter _session_command_op_assign -1 1 2024-02-23T00:51:21.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.313+0000 7efec9d77700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7efec40fefb0 0x7efec40ff3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:21.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.313+0000 7efec9d77700 20 client.31326.objecter _get_session s=0x7efec4106fc0 osd=0 3 2024-02-23T00:51:21.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.313+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec4106fc0 osd=0 3 2024-02-23T00:51:21.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.313+0000 7efec9d77700 20 client.31326.objecter _get_session s=0x7efec4106fc0 osd=0 3 2024-02-23T00:51:21.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.313+0000 7efec9d77700 15 client.31326.objecter _session_command_op_remove -1 1 2024-02-23T00:51:21.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.313+0000 7efec9d77700 20 client.31326.objecter get_session s=0x7efec4106fc0 osd=0 3 2024-02-23T00:51:21.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.313+0000 7efec9d77700 15 client.31326.objecter _session_command_op_assign 0 1 2024-02-23T00:51:21.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.313+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec4106fc0 osd=0 4 2024-02-23T00:51:21.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.313+0000 7efec9d77700 10 client.31326.objecter _send_command 1 2024-02-23T00:51:21.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.313+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efec40ff8a0 con 0x7efec40fefb0 2024-02-23T00:51:21.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.313+0000 7efec2ffd700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7efec40fefb0 0x7efec40ff3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:21.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.314+0000 7efec0ff9700 10 client.31326.objecter ms_handle_connect 0x7efec40fefb0 2024-02-23T00:51:21.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.314+0000 7efec2ffd700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7efec40fefb0 0x7efec40ff3b0 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:21.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.314+0000 7efec0ff9700 1 -- 172.21.15.47:0/3102566360 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efec40ff8a0 con 0x7efec40fefb0 2024-02-23T00:51:21.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.314+0000 7efec0ff9700 10 client.31326.objecter ms_dispatch 0x7efec4001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:51:21.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.314+0000 7efec0ff9700 10 client.31326.objecter _finish_command 1 = system:0 2024-02-23T00:51:21.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.314+0000 7efec0ff9700 20 client.31326.objecter put_session s=0x7efec4106fc0 osd=0 4 2024-02-23T00:51:21.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.314+0000 7efec0ff9700 15 client.31326.objecter _session_command_op_remove 0 1 2024-02-23T00:51:21.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.329+0000 7efec9d77700 10 client.31326.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:51:21.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.329+0000 7efec9d77700 15 client.31326.objecter _session_command_op_assign -1 2 2024-02-23T00:51:21.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.329+0000 7efec9d77700 20 client.31326.objecter _get_session s=0x7efec4106fc0 osd=0 3 2024-02-23T00:51:21.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.329+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec4106fc0 osd=0 3 2024-02-23T00:51:21.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.329+0000 7efec9d77700 20 client.31326.objecter _get_session s=0x7efec4106fc0 osd=0 3 2024-02-23T00:51:21.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.329+0000 7efec9d77700 15 client.31326.objecter _session_command_op_remove -1 2 2024-02-23T00:51:21.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.329+0000 7efec9d77700 20 client.31326.objecter get_session s=0x7efec4106fc0 osd=0 3 2024-02-23T00:51:21.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.329+0000 7efec9d77700 15 client.31326.objecter _session_command_op_assign 0 2 2024-02-23T00:51:21.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.329+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec4106fc0 osd=0 4 2024-02-23T00:51:21.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.329+0000 7efec9d77700 10 client.31326.objecter _send_command 2 2024-02-23T00:51:21.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.329+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7efec40ffa10 con 0x7efec40fefb0 2024-02-23T00:51:21.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.331+0000 7efec0ff9700 1 -- 172.21.15.47:0/3102566360 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7efec40ffa10 con 0x7efec40fefb0 2024-02-23T00:51:21.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.331+0000 7efec0ff9700 10 client.31326.objecter ms_dispatch 0x7efec4001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:51:21.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.331+0000 7efec0ff9700 10 client.31326.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:51:21.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.331+0000 7efec0ff9700 20 client.31326.objecter put_session s=0x7efec4106fc0 osd=0 4 2024-02-23T00:51:21.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.331+0000 7efec0ff9700 15 client.31326.objecter _session_command_op_remove 0 2 2024-02-23T00:51:21.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.331+0000 7efec9d77700 10 client.31326.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:21.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.331+0000 7efec9d77700 15 client.31326.objecter _session_command_op_assign -1 3 2024-02-23T00:51:21.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.331+0000 7efec9d77700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7efec4028540 0x7efec411ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:21.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.331+0000 7efec9d77700 20 client.31326.objecter _get_session s=0x7efec40f4c30 osd=1 3 2024-02-23T00:51:21.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.331+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec40f4c30 osd=1 3 2024-02-23T00:51:21.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.331+0000 7efec9d77700 20 client.31326.objecter _get_session s=0x7efec40f4c30 osd=1 3 2024-02-23T00:51:21.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.331+0000 7efec9d77700 15 client.31326.objecter _session_command_op_remove -1 3 2024-02-23T00:51:21.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.331+0000 7efec9d77700 20 client.31326.objecter get_session s=0x7efec40f4c30 osd=1 3 2024-02-23T00:51:21.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.331+0000 7efec9d77700 15 client.31326.objecter _session_command_op_assign 1 3 2024-02-23T00:51:21.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.331+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec40f4c30 osd=1 4 2024-02-23T00:51:21.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.331+0000 7efec9d77700 10 client.31326.objecter _send_command 3 2024-02-23T00:51:21.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.331+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7efec40f4da0 con 0x7efec4028540 2024-02-23T00:51:21.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.331+0000 7efec37fe700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7efec4028540 0x7efec411ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:21.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.332+0000 7efec0ff9700 10 client.31326.objecter ms_handle_connect 0x7efec4028540 2024-02-23T00:51:21.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.332+0000 7efec37fe700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7efec4028540 0x7efec411ae30 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:21.722 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:51:21.722 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:51:21.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.332+0000 7efec0ff9700 1 -- 172.21.15.47:0/3102566360 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efec40f4da0 con 0x7efec4028540 2024-02-23T00:51:21.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.332+0000 7efec0ff9700 10 client.31326.objecter ms_dispatch 0x7efec4001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:51:21.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.332+0000 7efec0ff9700 10 client.31326.objecter _finish_command 3 = system:0 2024-02-23T00:51:21.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.332+0000 7efec0ff9700 20 client.31326.objecter put_session s=0x7efec40f4c30 osd=1 4 2024-02-23T00:51:21.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.332+0000 7efec0ff9700 15 client.31326.objecter _session_command_op_remove 1 3 2024-02-23T00:51:21.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.347+0000 7efec9d77700 10 client.31326.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:51:21.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.347+0000 7efec9d77700 15 client.31326.objecter _session_command_op_assign -1 4 2024-02-23T00:51:21.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.347+0000 7efec9d77700 20 client.31326.objecter _get_session s=0x7efec40f4c30 osd=1 3 2024-02-23T00:51:21.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.347+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec40f4c30 osd=1 3 2024-02-23T00:51:21.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.347+0000 7efec9d77700 20 client.31326.objecter _get_session s=0x7efec40f4c30 osd=1 3 2024-02-23T00:51:21.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.347+0000 7efec9d77700 15 client.31326.objecter _session_command_op_remove -1 4 2024-02-23T00:51:21.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.347+0000 7efec9d77700 20 client.31326.objecter get_session s=0x7efec40f4c30 osd=1 3 2024-02-23T00:51:21.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.347+0000 7efec9d77700 15 client.31326.objecter _session_command_op_assign 1 4 2024-02-23T00:51:21.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.347+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec40f4c30 osd=1 4 2024-02-23T00:51:21.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.347+0000 7efec9d77700 10 client.31326.objecter _send_command 4 2024-02-23T00:51:21.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.347+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7efec405a040 con 0x7efec4028540 2024-02-23T00:51:21.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.349+0000 7efec0ff9700 1 -- 172.21.15.47:0/3102566360 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7efec405a040 con 0x7efec4028540 2024-02-23T00:51:21.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.349+0000 7efec0ff9700 10 client.31326.objecter ms_dispatch 0x7efec4001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:51:21.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.349+0000 7efec0ff9700 10 client.31326.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:51:21.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.349+0000 7efec0ff9700 20 client.31326.objecter put_session s=0x7efec40f4c30 osd=1 4 2024-02-23T00:51:21.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.349+0000 7efec0ff9700 15 client.31326.objecter _session_command_op_remove 1 4 2024-02-23T00:51:21.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.349+0000 7efec9d77700 10 client.31326.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:21.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.349+0000 7efec9d77700 15 client.31326.objecter _session_command_op_assign -1 5 2024-02-23T00:51:21.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.349+0000 7efec9d77700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7efec40594d0 0x7efec411e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:21.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.349+0000 7efec9d77700 20 client.31326.objecter _get_session s=0x7efec4059360 osd=2 3 2024-02-23T00:51:21.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.349+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec4059360 osd=2 3 2024-02-23T00:51:21.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.349+0000 7efec9d77700 20 client.31326.objecter _get_session s=0x7efec4059360 osd=2 3 2024-02-23T00:51:21.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.349+0000 7efec9d77700 15 client.31326.objecter _session_command_op_remove -1 5 2024-02-23T00:51:21.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.349+0000 7efec9d77700 20 client.31326.objecter get_session s=0x7efec4059360 osd=2 3 2024-02-23T00:51:21.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.349+0000 7efec9d77700 15 client.31326.objecter _session_command_op_assign 2 5 2024-02-23T00:51:21.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.349+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec4059360 osd=2 4 2024-02-23T00:51:21.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.349+0000 7efec9d77700 10 client.31326.objecter _send_command 5 2024-02-23T00:51:21.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.349+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7efec40280a0 con 0x7efec40594d0 2024-02-23T00:51:21.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.349+0000 7efec2ffd700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7efec40594d0 0x7efec411e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:21.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.349+0000 7efec0ff9700 10 client.31326.objecter ms_handle_connect 0x7efec40594d0 2024-02-23T00:51:21.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.349+0000 7efec2ffd700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7efec40594d0 0x7efec411e4b0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:21.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.350+0000 7efec0ff9700 1 -- 172.21.15.47:0/3102566360 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efec40280a0 con 0x7efec40594d0 2024-02-23T00:51:21.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.350+0000 7efec0ff9700 10 client.31326.objecter ms_dispatch 0x7efec4001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:51:21.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.350+0000 7efec0ff9700 10 client.31326.objecter _finish_command 5 = system:0 2024-02-23T00:51:21.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.350+0000 7efec0ff9700 20 client.31326.objecter put_session s=0x7efec4059360 osd=2 4 2024-02-23T00:51:21.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.350+0000 7efec0ff9700 15 client.31326.objecter _session_command_op_remove 2 5 2024-02-23T00:51:21.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.365+0000 7efec9d77700 10 client.31326.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:51:21.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.365+0000 7efec9d77700 15 client.31326.objecter _session_command_op_assign -1 6 2024-02-23T00:51:21.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.365+0000 7efec9d77700 20 client.31326.objecter _get_session s=0x7efec4059360 osd=2 3 2024-02-23T00:51:21.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.365+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec4059360 osd=2 3 2024-02-23T00:51:21.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.365+0000 7efec9d77700 20 client.31326.objecter _get_session s=0x7efec4059360 osd=2 3 2024-02-23T00:51:21.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.365+0000 7efec9d77700 15 client.31326.objecter _session_command_op_remove -1 6 2024-02-23T00:51:21.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.365+0000 7efec9d77700 20 client.31326.objecter get_session s=0x7efec4059360 osd=2 3 2024-02-23T00:51:21.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.365+0000 7efec9d77700 15 client.31326.objecter _session_command_op_assign 2 6 2024-02-23T00:51:21.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.365+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec4059360 osd=2 4 2024-02-23T00:51:21.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.365+0000 7efec9d77700 10 client.31326.objecter _send_command 6 2024-02-23T00:51:21.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.365+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7efec411e9a0 con 0x7efec40594d0 2024-02-23T00:51:21.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.367+0000 7efec0ff9700 1 -- 172.21.15.47:0/3102566360 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7efec411e9a0 con 0x7efec40594d0 2024-02-23T00:51:21.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.367+0000 7efec0ff9700 10 client.31326.objecter ms_dispatch 0x7efec4001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:51:21.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.367+0000 7efec0ff9700 10 client.31326.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:51:21.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.367+0000 7efec0ff9700 20 client.31326.objecter put_session s=0x7efec4059360 osd=2 4 2024-02-23T00:51:21.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.367+0000 7efec0ff9700 15 client.31326.objecter _session_command_op_remove 2 6 2024-02-23T00:51:21.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.367+0000 7efec9d77700 10 client.31326.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:21.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.367+0000 7efec9d77700 15 client.31326.objecter _session_command_op_assign -1 7 2024-02-23T00:51:21.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.367+0000 7efec9d77700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7efec4027770 0x7efec411fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:21.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.367+0000 7efec9d77700 20 client.31326.objecter _get_session s=0x7efec4027600 osd=3 3 2024-02-23T00:51:21.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.367+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec4027600 osd=3 3 2024-02-23T00:51:21.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.367+0000 7efec9d77700 20 client.31326.objecter _get_session s=0x7efec4027600 osd=3 3 2024-02-23T00:51:21.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.367+0000 7efec9d77700 15 client.31326.objecter _session_command_op_remove -1 7 2024-02-23T00:51:21.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.367+0000 7efec9d77700 20 client.31326.objecter get_session s=0x7efec4027600 osd=3 3 2024-02-23T00:51:21.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.367+0000 7efec9d77700 15 client.31326.objecter _session_command_op_assign 3 7 2024-02-23T00:51:21.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.367+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec4027600 osd=3 4 2024-02-23T00:51:21.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.367+0000 7efec9d77700 10 client.31326.objecter _send_command 7 2024-02-23T00:51:21.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.367+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7efec411c1d0 con 0x7efec4027770 2024-02-23T00:51:21.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.367+0000 7efec3fff700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7efec4027770 0x7efec411fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:21.727 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:51:21.727 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:51:21.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.367+0000 7efec0ff9700 10 client.31326.objecter ms_handle_connect 0x7efec4027770 2024-02-23T00:51:21.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.367+0000 7efec3fff700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7efec4027770 0x7efec411fcc0 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:21.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.368+0000 7efec0ff9700 1 -- 172.21.15.47:0/3102566360 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efec411c1d0 con 0x7efec4027770 2024-02-23T00:51:21.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.368+0000 7efec0ff9700 10 client.31326.objecter ms_dispatch 0x7efec4001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:51:21.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.368+0000 7efec0ff9700 10 client.31326.objecter _finish_command 7 = system:0 2024-02-23T00:51:21.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.368+0000 7efec0ff9700 20 client.31326.objecter put_session s=0x7efec4027600 osd=3 4 2024-02-23T00:51:21.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.368+0000 7efec0ff9700 15 client.31326.objecter _session_command_op_remove 3 7 2024-02-23T00:51:21.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.383+0000 7efec9d77700 10 client.31326.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:51:21.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.383+0000 7efec9d77700 15 client.31326.objecter _session_command_op_assign -1 8 2024-02-23T00:51:21.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.383+0000 7efec9d77700 20 client.31326.objecter _get_session s=0x7efec4027600 osd=3 3 2024-02-23T00:51:21.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.383+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec4027600 osd=3 3 2024-02-23T00:51:21.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.383+0000 7efec9d77700 20 client.31326.objecter _get_session s=0x7efec4027600 osd=3 3 2024-02-23T00:51:21.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.383+0000 7efec9d77700 15 client.31326.objecter _session_command_op_remove -1 8 2024-02-23T00:51:21.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.383+0000 7efec9d77700 20 client.31326.objecter get_session s=0x7efec4027600 osd=3 3 2024-02-23T00:51:21.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.383+0000 7efec9d77700 15 client.31326.objecter _session_command_op_assign 3 8 2024-02-23T00:51:21.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.383+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec4027600 osd=3 4 2024-02-23T00:51:21.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.383+0000 7efec9d77700 10 client.31326.objecter _send_command 8 2024-02-23T00:51:21.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.383+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7efec411b560 con 0x7efec4027770 2024-02-23T00:51:21.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.385+0000 7efec0ff9700 1 -- 172.21.15.47:0/3102566360 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7efec411b560 con 0x7efec4027770 2024-02-23T00:51:21.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.385+0000 7efec0ff9700 10 client.31326.objecter ms_dispatch 0x7efec4001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:51:21.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.385+0000 7efec0ff9700 10 client.31326.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:51:21.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.385+0000 7efec0ff9700 20 client.31326.objecter put_session s=0x7efec4027600 osd=3 4 2024-02-23T00:51:21.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.385+0000 7efec0ff9700 15 client.31326.objecter _session_command_op_remove 3 8 2024-02-23T00:51:21.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.385+0000 7efec9d77700 10 client.31326.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:21.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.385+0000 7efec9d77700 15 client.31326.objecter _session_command_op_assign -1 9 2024-02-23T00:51:21.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.385+0000 7efec9d77700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7efec4120e90 0x7efec41252f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:21.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.385+0000 7efec9d77700 20 client.31326.objecter _get_session s=0x7efec411b7a0 osd=4 3 2024-02-23T00:51:21.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.385+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec411b7a0 osd=4 3 2024-02-23T00:51:21.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.385+0000 7efec9d77700 20 client.31326.objecter _get_session s=0x7efec411b7a0 osd=4 3 2024-02-23T00:51:21.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.385+0000 7efec9d77700 15 client.31326.objecter _session_command_op_remove -1 9 2024-02-23T00:51:21.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.385+0000 7efec9d77700 20 client.31326.objecter get_session s=0x7efec411b7a0 osd=4 3 2024-02-23T00:51:21.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.385+0000 7efec9d77700 15 client.31326.objecter _session_command_op_assign 4 9 2024-02-23T00:51:21.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.385+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec411b7a0 osd=4 4 2024-02-23T00:51:21.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.385+0000 7efec9d77700 10 client.31326.objecter _send_command 9 2024-02-23T00:51:21.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.385+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7efec4127f40 con 0x7efec4120e90 2024-02-23T00:51:21.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.385+0000 7efec37fe700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7efec4120e90 0x7efec41252f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:21.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.385+0000 7efec37fe700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7efec4120e90 0x7efec41252f0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:21.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.386+0000 7efec0ff9700 10 client.31326.objecter ms_handle_connect 0x7efec4120e90 2024-02-23T00:51:21.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.386+0000 7efec0ff9700 1 -- 172.21.15.47:0/3102566360 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efec4127f40 con 0x7efec4120e90 2024-02-23T00:51:21.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.386+0000 7efec0ff9700 10 client.31326.objecter ms_dispatch 0x7efec4001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:51:21.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.386+0000 7efec0ff9700 10 client.31326.objecter _finish_command 9 = system:0 2024-02-23T00:51:21.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.386+0000 7efec0ff9700 20 client.31326.objecter put_session s=0x7efec411b7a0 osd=4 4 2024-02-23T00:51:21.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.386+0000 7efec0ff9700 15 client.31326.objecter _session_command_op_remove 4 9 2024-02-23T00:51:21.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.401+0000 7efec9d77700 10 client.31326.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:51:21.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.401+0000 7efec9d77700 15 client.31326.objecter _session_command_op_assign -1 10 2024-02-23T00:51:21.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.401+0000 7efec9d77700 20 client.31326.objecter _get_session s=0x7efec411b7a0 osd=4 3 2024-02-23T00:51:21.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.401+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec411b7a0 osd=4 3 2024-02-23T00:51:21.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.401+0000 7efec9d77700 20 client.31326.objecter _get_session s=0x7efec411b7a0 osd=4 3 2024-02-23T00:51:21.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.401+0000 7efec9d77700 15 client.31326.objecter _session_command_op_remove -1 10 2024-02-23T00:51:21.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.401+0000 7efec9d77700 20 client.31326.objecter get_session s=0x7efec411b7a0 osd=4 3 2024-02-23T00:51:21.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.401+0000 7efec9d77700 15 client.31326.objecter _session_command_op_assign 4 10 2024-02-23T00:51:21.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.401+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec411b7a0 osd=4 4 2024-02-23T00:51:21.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.401+0000 7efec9d77700 10 client.31326.objecter _send_command 10 2024-02-23T00:51:21.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.401+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7efec41257e0 con 0x7efec4120e90 2024-02-23T00:51:21.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.403+0000 7efec0ff9700 1 -- 172.21.15.47:0/3102566360 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7efec41257e0 con 0x7efec4120e90 2024-02-23T00:51:21.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.403+0000 7efec0ff9700 10 client.31326.objecter ms_dispatch 0x7efec4001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:51:21.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.403+0000 7efec0ff9700 10 client.31326.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:51:21.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.403+0000 7efec0ff9700 20 client.31326.objecter put_session s=0x7efec411b7a0 osd=4 4 2024-02-23T00:51:21.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.403+0000 7efec0ff9700 15 client.31326.objecter _session_command_op_remove 4 10 2024-02-23T00:51:21.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.403+0000 7efec9d77700 10 client.31326.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:21.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.403+0000 7efec9d77700 15 client.31326.objecter _session_command_op_assign -1 11 2024-02-23T00:51:21.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.403+0000 7efec9d77700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7efec4125e90 0x7efec4126310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:21.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.403+0000 7efec9d77700 20 client.31326.objecter _get_session s=0x7efec41257e0 osd=5 3 2024-02-23T00:51:21.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.403+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec41257e0 osd=5 3 2024-02-23T00:51:21.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.403+0000 7efec9d77700 20 client.31326.objecter _get_session s=0x7efec41257e0 osd=5 3 2024-02-23T00:51:21.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.403+0000 7efec9d77700 15 client.31326.objecter _session_command_op_remove -1 11 2024-02-23T00:51:21.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.403+0000 7efec9d77700 20 client.31326.objecter get_session s=0x7efec41257e0 osd=5 3 2024-02-23T00:51:21.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.403+0000 7efec9d77700 15 client.31326.objecter _session_command_op_assign 5 11 2024-02-23T00:51:21.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.403+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec41257e0 osd=5 4 2024-02-23T00:51:21.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.403+0000 7efec9d77700 10 client.31326.objecter _send_command 11 2024-02-23T00:51:21.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.403+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7efec412b2a0 con 0x7efec4125e90 2024-02-23T00:51:21.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.403+0000 7efec2ffd700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7efec4125e90 0x7efec4126310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:21.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.404+0000 7efec0ff9700 10 client.31326.objecter ms_handle_connect 0x7efec4125e90 2024-02-23T00:51:21.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.404+0000 7efec2ffd700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7efec4125e90 0x7efec4126310 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:21.734 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:51:21.734 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:51:21.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.404+0000 7efec0ff9700 1 -- 172.21.15.47:0/3102566360 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efec412b2a0 con 0x7efec4125e90 2024-02-23T00:51:21.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.404+0000 7efec0ff9700 10 client.31326.objecter ms_dispatch 0x7efec4001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:51:21.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.404+0000 7efec0ff9700 10 client.31326.objecter _finish_command 11 = system:0 2024-02-23T00:51:21.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.404+0000 7efec0ff9700 20 client.31326.objecter put_session s=0x7efec41257e0 osd=5 4 2024-02-23T00:51:21.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.404+0000 7efec0ff9700 15 client.31326.objecter _session_command_op_remove 5 11 2024-02-23T00:51:21.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.419+0000 7efec9d77700 10 client.31326.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:51:21.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.419+0000 7efec9d77700 15 client.31326.objecter _session_command_op_assign -1 12 2024-02-23T00:51:21.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.419+0000 7efec9d77700 20 client.31326.objecter _get_session s=0x7efec41257e0 osd=5 3 2024-02-23T00:51:21.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.419+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec41257e0 osd=5 3 2024-02-23T00:51:21.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.419+0000 7efec9d77700 20 client.31326.objecter _get_session s=0x7efec41257e0 osd=5 3 2024-02-23T00:51:21.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.419+0000 7efec9d77700 15 client.31326.objecter _session_command_op_remove -1 12 2024-02-23T00:51:21.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.419+0000 7efec9d77700 20 client.31326.objecter get_session s=0x7efec41257e0 osd=5 3 2024-02-23T00:51:21.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.419+0000 7efec9d77700 15 client.31326.objecter _session_command_op_assign 5 12 2024-02-23T00:51:21.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.419+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec41257e0 osd=5 4 2024-02-23T00:51:21.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.419+0000 7efec9d77700 10 client.31326.objecter _send_command 12 2024-02-23T00:51:21.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.419+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7efec4126910 con 0x7efec4125e90 2024-02-23T00:51:21.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.421+0000 7efec0ff9700 1 -- 172.21.15.47:0/3102566360 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7efec4126910 con 0x7efec4125e90 2024-02-23T00:51:21.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.421+0000 7efec0ff9700 10 client.31326.objecter ms_dispatch 0x7efec4001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:51:21.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.421+0000 7efec0ff9700 10 client.31326.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:51:21.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.421+0000 7efec0ff9700 20 client.31326.objecter put_session s=0x7efec41257e0 osd=5 4 2024-02-23T00:51:21.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.421+0000 7efec0ff9700 15 client.31326.objecter _session_command_op_remove 5 12 2024-02-23T00:51:21.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.421+0000 7efec9d77700 10 client.31326.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:21.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.421+0000 7efec9d77700 15 client.31326.objecter _session_command_op_assign -1 13 2024-02-23T00:51:21.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.421+0000 7efec9d77700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7efec4126fc0 0x7efec4127420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:21.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.421+0000 7efec9d77700 20 client.31326.objecter _get_session s=0x7efec4126910 osd=6 3 2024-02-23T00:51:21.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.421+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec4126910 osd=6 3 2024-02-23T00:51:21.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.421+0000 7efec9d77700 20 client.31326.objecter _get_session s=0x7efec4126910 osd=6 3 2024-02-23T00:51:21.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.421+0000 7efec9d77700 15 client.31326.objecter _session_command_op_remove -1 13 2024-02-23T00:51:21.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.421+0000 7efec9d77700 20 client.31326.objecter get_session s=0x7efec4126910 osd=6 3 2024-02-23T00:51:21.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.421+0000 7efec9d77700 15 client.31326.objecter _session_command_op_assign 6 13 2024-02-23T00:51:21.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.421+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec4126910 osd=6 4 2024-02-23T00:51:21.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.421+0000 7efec9d77700 10 client.31326.objecter _send_command 13 2024-02-23T00:51:21.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.421+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7efec4127ad0 con 0x7efec4126fc0 2024-02-23T00:51:21.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.421+0000 7efec3fff700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7efec4126fc0 0x7efec4127420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:21.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.421+0000 7efec0ff9700 10 client.31326.objecter ms_handle_connect 0x7efec4126fc0 2024-02-23T00:51:21.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.421+0000 7efec3fff700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7efec4126fc0 0x7efec4127420 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:21.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.422+0000 7efec0ff9700 1 -- 172.21.15.47:0/3102566360 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efec4127ad0 con 0x7efec4126fc0 2024-02-23T00:51:21.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.422+0000 7efec0ff9700 10 client.31326.objecter ms_dispatch 0x7efec4001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:51:21.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.422+0000 7efec0ff9700 10 client.31326.objecter _finish_command 13 = system:0 2024-02-23T00:51:21.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.422+0000 7efec0ff9700 20 client.31326.objecter put_session s=0x7efec4126910 osd=6 4 2024-02-23T00:51:21.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.422+0000 7efec0ff9700 15 client.31326.objecter _session_command_op_remove 6 13 2024-02-23T00:51:21.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.437+0000 7efec9d77700 10 client.31326.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:51:21.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.437+0000 7efec9d77700 15 client.31326.objecter _session_command_op_assign -1 14 2024-02-23T00:51:21.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.437+0000 7efec9d77700 20 client.31326.objecter _get_session s=0x7efec4126910 osd=6 3 2024-02-23T00:51:21.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.437+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec4126910 osd=6 3 2024-02-23T00:51:21.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.437+0000 7efec9d77700 20 client.31326.objecter _get_session s=0x7efec4126910 osd=6 3 2024-02-23T00:51:21.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.437+0000 7efec9d77700 15 client.31326.objecter _session_command_op_remove -1 14 2024-02-23T00:51:21.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.437+0000 7efec9d77700 20 client.31326.objecter get_session s=0x7efec4126910 osd=6 3 2024-02-23T00:51:21.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.437+0000 7efec9d77700 15 client.31326.objecter _session_command_op_assign 6 14 2024-02-23T00:51:21.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.437+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec4126910 osd=6 4 2024-02-23T00:51:21.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.437+0000 7efec9d77700 10 client.31326.objecter _send_command 14 2024-02-23T00:51:21.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.437+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7efec4127ad0 con 0x7efec4126fc0 2024-02-23T00:51:21.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.438+0000 7efec0ff9700 1 -- 172.21.15.47:0/3102566360 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7efec4127ad0 con 0x7efec4126fc0 2024-02-23T00:51:21.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.438+0000 7efec0ff9700 10 client.31326.objecter ms_dispatch 0x7efec4001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:51:21.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.438+0000 7efec0ff9700 10 client.31326.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:51:21.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.438+0000 7efec0ff9700 20 client.31326.objecter put_session s=0x7efec4126910 osd=6 4 2024-02-23T00:51:21.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.438+0000 7efec0ff9700 15 client.31326.objecter _session_command_op_remove 6 14 2024-02-23T00:51:21.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.439+0000 7efec9d77700 10 client.31326.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:21.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.439+0000 7efec9d77700 15 client.31326.objecter _session_command_op_assign -1 15 2024-02-23T00:51:21.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.439+0000 7efec9d77700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7efec412c640 0x7efec412ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:21.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.439+0000 7efec9d77700 20 client.31326.objecter _get_session s=0x7efec4127ad0 osd=7 3 2024-02-23T00:51:21.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.439+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec4127ad0 osd=7 3 2024-02-23T00:51:21.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.439+0000 7efec9d77700 20 client.31326.objecter _get_session s=0x7efec4127ad0 osd=7 3 2024-02-23T00:51:21.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.439+0000 7efec9d77700 15 client.31326.objecter _session_command_op_remove -1 15 2024-02-23T00:51:21.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.439+0000 7efec9d77700 20 client.31326.objecter get_session s=0x7efec4127ad0 osd=7 3 2024-02-23T00:51:21.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.439+0000 7efec9d77700 15 client.31326.objecter _session_command_op_assign 7 15 2024-02-23T00:51:21.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.439+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec4127ad0 osd=7 4 2024-02-23T00:51:21.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.439+0000 7efec9d77700 10 client.31326.objecter _send_command 15 2024-02-23T00:51:21.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.439+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7efec41316d0 con 0x7efec412c640 2024-02-23T00:51:21.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.439+0000 7efec37fe700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7efec412c640 0x7efec412ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:21.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.439+0000 7efec0ff9700 10 client.31326.objecter ms_handle_connect 0x7efec412c640 2024-02-23T00:51:21.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.439+0000 7efec37fe700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7efec412c640 0x7efec412ea30 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:21.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.440+0000 7efec0ff9700 1 -- 172.21.15.47:0/3102566360 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efec41316d0 con 0x7efec412c640 2024-02-23T00:51:21.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.440+0000 7efec0ff9700 10 client.31326.objecter ms_dispatch 0x7efec4001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:51:21.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.440+0000 7efec0ff9700 10 client.31326.objecter _finish_command 15 = system:0 2024-02-23T00:51:21.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.440+0000 7efec0ff9700 20 client.31326.objecter put_session s=0x7efec4127ad0 osd=7 4 2024-02-23T00:51:21.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.440+0000 7efec0ff9700 15 client.31326.objecter _session_command_op_remove 7 15 2024-02-23T00:51:21.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.456+0000 7efec9d77700 10 client.31326.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:51:21.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.456+0000 7efec9d77700 15 client.31326.objecter _session_command_op_assign -1 16 2024-02-23T00:51:21.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.456+0000 7efec9d77700 20 client.31326.objecter _get_session s=0x7efec4127ad0 osd=7 3 2024-02-23T00:51:21.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.456+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec4127ad0 osd=7 3 2024-02-23T00:51:21.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.456+0000 7efec9d77700 20 client.31326.objecter _get_session s=0x7efec4127ad0 osd=7 3 2024-02-23T00:51:21.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.456+0000 7efec9d77700 15 client.31326.objecter _session_command_op_remove -1 16 2024-02-23T00:51:21.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.456+0000 7efec9d77700 20 client.31326.objecter get_session s=0x7efec4127ad0 osd=7 3 2024-02-23T00:51:21.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.456+0000 7efec9d77700 15 client.31326.objecter _session_command_op_assign 7 16 2024-02-23T00:51:21.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.456+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec4127ad0 osd=7 4 2024-02-23T00:51:21.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.456+0000 7efec9d77700 10 client.31326.objecter _send_command 16 2024-02-23T00:51:21.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.456+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7efec412ef20 con 0x7efec412c640 2024-02-23T00:51:21.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.457+0000 7efec0ff9700 1 -- 172.21.15.47:0/3102566360 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7efec412ef20 con 0x7efec412c640 2024-02-23T00:51:21.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.457+0000 7efec0ff9700 10 client.31326.objecter ms_dispatch 0x7efec4001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:51:21.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.457+0000 7efec0ff9700 10 client.31326.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:51:21.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.457+0000 7efec0ff9700 20 client.31326.objecter put_session s=0x7efec4127ad0 osd=7 4 2024-02-23T00:51:21.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.457+0000 7efec0ff9700 15 client.31326.objecter _session_command_op_remove 7 16 2024-02-23T00:51:21.741 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:51:21.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 10 librados: watch_flush enter 2024-02-23T00:51:21.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 10 librados: watch_flush exit 2024-02-23T00:51:21.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 10 client.31326.objecter close_session for osd.0 2024-02-23T00:51:21.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7efec40fefb0 msgr2=0x7efec40ff3b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:21.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7efec40fefb0 0x7efec40ff3b0 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:21.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec4106fc0 osd=0 1 2024-02-23T00:51:21.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 10 client.31326.objecter close_session for osd.1 2024-02-23T00:51:21.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7efec4028540 msgr2=0x7efec411ae30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:21.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7efec4028540 0x7efec411ae30 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:21.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec40f4c30 osd=1 1 2024-02-23T00:51:21.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 10 client.31326.objecter close_session for osd.2 2024-02-23T00:51:21.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7efec40594d0 msgr2=0x7efec411e4b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:21.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7efec40594d0 0x7efec411e4b0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:21.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec4059360 osd=2 1 2024-02-23T00:51:21.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 10 client.31326.objecter close_session for osd.3 2024-02-23T00:51:21.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7efec4027770 msgr2=0x7efec411fcc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:21.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec3fff700 1 -- 172.21.15.47:0/3102566360 reap_dead start 2024-02-23T00:51:21.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7efec4027770 0x7efec411fcc0 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:21.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec4027600 osd=3 1 2024-02-23T00:51:21.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 10 client.31326.objecter close_session for osd.4 2024-02-23T00:51:21.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7efec4120e90 msgr2=0x7efec41252f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:21.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7efec4120e90 0x7efec41252f0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:21.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec411b7a0 osd=4 1 2024-02-23T00:51:21.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 10 client.31326.objecter close_session for osd.5 2024-02-23T00:51:21.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7efec4125e90 msgr2=0x7efec4126310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:21.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7efec4125e90 0x7efec4126310 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:21.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec41257e0 osd=5 1 2024-02-23T00:51:21.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 10 client.31326.objecter close_session for osd.6 2024-02-23T00:51:21.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7efec4126fc0 msgr2=0x7efec4127420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:21.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7efec4126fc0 0x7efec4127420 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:21.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec4126910 osd=6 1 2024-02-23T00:51:21.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 10 client.31326.objecter close_session for osd.7 2024-02-23T00:51:21.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7efec412c640 msgr2=0x7efec412ea30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:21.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7efec412c640 0x7efec412ea30 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:21.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 20 client.31326.objecter put_session s=0x7efec4127ad0 osd=7 1 2024-02-23T00:51:21.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 20 client.31326.objecter shutdown clearing up homeless session... 2024-02-23T00:51:21.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 10 client.31326.objecter successfully canceled tick 2024-02-23T00:51:21.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec3fff700 1 -- 172.21.15.47:0/3102566360 reap_dead start 2024-02-23T00:51:21.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7efeac062cc0 msgr2=0x7efeac065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:21.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7efeac062cc0 0x7efeac065180 secure :-1 s=READY pgs=3117 cs=0 l=1 rev1=1 rx=0x7efeb40048b0 tx=0x7efeb400b040).stop 2024-02-23T00:51:21.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efec40fa380 msgr2=0x7efec406bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:21.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efec40fa380 0x7efec406bab0 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 rx=0x7efebc00bf80 tx=0x7efebc004c50).stop 2024-02-23T00:51:21.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 shutdown_connections 2024-02-23T00:51:21.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7efeac062cc0 0x7efeac065180 unknown :-1 s=CLOSED pgs=3117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:21.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 1 --2- 172.21.15.47:0/3102566360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efec40fa380 0x7efec406bab0 unknown :-1 s=CLOSED pgs=2573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:21.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 >> 172.21.15.47:0/3102566360 conn(0x7efec40f3800 msgr2=0x7efec40f40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:21.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 shutdown_connections 2024-02-23T00:51:21.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 1 -- 172.21.15.47:0/3102566360 wait complete. 2024-02-23T00:51:21.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:21.458+0000 7efec9d77700 1 librados: shutdown 2024-02-23T00:51:21.745 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:51:21.745 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:51:21.745 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:51:21.745 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:51:21.746 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:51:21.746 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:51:21.746 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:51:21.746 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:51:22.236 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.4 dump_blocked_ops 2024-02-23T00:51:22.306 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:51:22.309 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:22.309 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:51:22.410 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:51:22.437 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:22.457 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:51:22.460 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:22.460 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:51:22.512 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:51:22.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:20.841099+0000 osd.2 (osd.2) 165 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:51:22.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:20.841361+0000 osd.2 (osd.2) 166 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:51:22.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:21.275888+0000 mon.a (mon.0) 2963 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:51:22.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:21.275948+0000 mon.a (mon.0) 2964 : cluster [DBG] osdmap e552: 8 total, 8 up, 7 in 2024-02-23T00:51:22.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:21.278095+0000 mon.a (mon.0) 2965 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-90"}]: dispatch 2024-02-23T00:51:22.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:21.313609+0000 mon.a (mon.0) 2966 : audit [DBG] from='client.? 172.21.15.47:0/3102566360' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:51:22.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:21.340953+0000 mgr.y (mgr.14517) 503 : cluster [DBG] pgmap v832: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:51:22.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:20.841099+0000 osd.2 (osd.2) 165 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:51:22.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:20.841361+0000 osd.2 (osd.2) 166 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:51:22.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:21.275888+0000 mon.a (mon.0) 2963 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:51:22.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:21.275948+0000 mon.a (mon.0) 2964 : cluster [DBG] osdmap e552: 8 total, 8 up, 7 in 2024-02-23T00:51:22.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:21.278095+0000 mon.a (mon.0) 2965 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-90"}]: dispatch 2024-02-23T00:51:22.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:21.313609+0000 mon.a (mon.0) 2966 : audit [DBG] from='client.? 172.21.15.47:0/3102566360' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:51:22.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:21.340953+0000 mgr.y (mgr.14517) 503 : cluster [DBG] pgmap v832: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:51:22.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:20.841099+0000 osd.2 (osd.2) 165 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:51:22.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:20.841361+0000 osd.2 (osd.2) 166 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:51:22.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:21.275888+0000 mon.a (mon.0) 2963 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]': finished 2024-02-23T00:51:22.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:21.275948+0000 mon.a (mon.0) 2964 : cluster [DBG] osdmap e552: 8 total, 8 up, 7 in 2024-02-23T00:51:22.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:21.278095+0000 mon.a (mon.0) 2965 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-90"}]: dispatch 2024-02-23T00:51:22.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:21.313609+0000 mon.a (mon.0) 2966 : audit [DBG] from='client.? 172.21.15.47:0/3102566360' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:51:22.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:21.340953+0000 mgr.y (mgr.14517) 503 : cluster [DBG] pgmap v832: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:51:22.562 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:51:22.590 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:22.610 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:51:22.613 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:22.613 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:51:22.646 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:51:22.714 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:51:22.742 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:22.796 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:51:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:51:22.796 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:51:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:51:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:51:22.869 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:51:22.872 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:22.872 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:51:22.973 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:51:23.003 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:23.032 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:51:23.035 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:23.035 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:51:23.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.125+0000 7f147e731700 1 -- 172.21.15.47:0/3609090063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14780f7400 msgr2=0x7f1478103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:23.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.125+0000 7f147e731700 1 --2- 172.21.15.47:0/3609090063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14780f7400 0x7f1478103a20 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 rx=0x7f1470009be0 tx=0x7f1470004d10).stop 2024-02-23T00:51:23.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.125+0000 7f147e731700 1 -- 172.21.15.47:0/3609090063 shutdown_connections 2024-02-23T00:51:23.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.125+0000 7f147e731700 1 --2- 172.21.15.47:0/3609090063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14780f7400 0x7f1478103a20 unknown :-1 s=CLOSED pgs=2574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:23.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.125+0000 7f147e731700 1 --2- 172.21.15.47:0/3609090063 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f14780fcbb0 0x7f14780fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:23.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.125+0000 7f147e731700 1 --2- 172.21.15.47:0/3609090063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14780fd510 0x7f14780f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:23.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.125+0000 7f147e731700 1 -- 172.21.15.47:0/3609090063 >> 172.21.15.47:0/3609090063 conn(0x7f14780f2820 msgr2=0x7f14780f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:23.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.125+0000 7f147e731700 1 -- 172.21.15.47:0/3609090063 shutdown_connections 2024-02-23T00:51:23.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.125+0000 7f147e731700 1 -- 172.21.15.47:0/3609090063 wait complete. 2024-02-23T00:51:23.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.126+0000 7f147e731700 1 librados: starting msgr at 2024-02-23T00:51:23.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.126+0000 7f147e731700 1 librados: starting objecter 2024-02-23T00:51:23.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.126+0000 7f147e731700 1 Processor -- start 2024-02-23T00:51:23.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.126+0000 7f147e731700 1 -- start start 2024-02-23T00:51:23.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.126+0000 7f147e731700 1 librados: setting wanted keys 2024-02-23T00:51:23.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.126+0000 7f147e731700 1 librados: calling monclient init 2024-02-23T00:51:23.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.126+0000 7f147e731700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14780f7400 0x7f147810cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:23.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.126+0000 7f147e731700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f14780fcbb0 0x7f147810d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:23.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.126+0000 7f147e731700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14780fd510 0x7f1478106ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:23.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.126+0000 7f147e731700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14781073e0 con 0x7f14780fd510 2024-02-23T00:51:23.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.126+0000 7f147e731700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1478107550 con 0x7f14780f7400 2024-02-23T00:51:23.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.126+0000 7f147e731700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1478107710 con 0x7f14780fcbb0 2024-02-23T00:51:23.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.126+0000 7f14777fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f14780fcbb0 0x7f147810d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:23.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.126+0000 7f147ccce700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14780fd510 0x7f1478106ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:23.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.126+0000 7f14777fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f14780fcbb0 0x7f147810d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:49964/0 (socket says 172.21.15.47:49964) 2024-02-23T00:51:23.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.126+0000 7f14777fe700 1 -- 172.21.15.47:0/1705794836 learned_addr learned my addr 172.21.15.47:0/1705794836 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:51:23.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.126+0000 7f147ccce700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14780fd510 0x7f1478106ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41746/0 (socket says 172.21.15.47:41746) 2024-02-23T00:51:23.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.126+0000 7f1477fff700 1 --2- 172.21.15.47:0/1705794836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14780f7400 0x7f147810cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:23.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.127+0000 7f14777fe700 1 -- 172.21.15.47:0/1705794836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14780f7400 msgr2=0x7f147810cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:23.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.127+0000 7f14777fe700 1 --2- 172.21.15.47:0/1705794836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14780f7400 0x7f147810cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:23.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.127+0000 7f14777fe700 1 -- 172.21.15.47:0/1705794836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14780fd510 msgr2=0x7f1478106ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:23.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.127+0000 7f14777fe700 1 --2- 172.21.15.47:0/1705794836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14780fd510 0x7f1478106ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:23.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.127+0000 7f14777fe700 1 -- 172.21.15.47:0/1705794836 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1468005040 con 0x7f14780fcbb0 2024-02-23T00:51:23.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.127+0000 7f147ccce700 1 --2- 172.21.15.47:0/1705794836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14780fd510 0x7f1478106ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:51:23.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.127+0000 7f14777fe700 1 --2- 172.21.15.47:0/1705794836 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f14780fcbb0 0x7f147810d240 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 rx=0x7f146c00e970 tx=0x7f146c00b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:23.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.127+0000 7f14757fa700 10 client.?.objecter ms_handle_connect 0x7f14780fcbb0 2024-02-23T00:51:23.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.127+0000 7f14757fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:51:23.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.127+0000 7f14757fa700 1 -- 172.21.15.47:0/1705794836 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c004660 con 0x7f14780fcbb0 2024-02-23T00:51:23.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.127+0000 7f14757fa700 1 -- 172.21.15.47:0/1705794836 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f146c00f040 con 0x7f14780fcbb0 2024-02-23T00:51:23.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.127+0000 7f147e731700 1 -- 172.21.15.47:0/1705794836 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14700097c0 con 0x7f14780fcbb0 2024-02-23T00:51:23.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.127+0000 7f14757fa700 1 -- 172.21.15.47:0/1705794836 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c0146f0 con 0x7f14780fcbb0 2024-02-23T00:51:23.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.127+0000 7f147e731700 10 client.41651.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:51:23.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.127+0000 7f147e731700 1 -- 172.21.15.47:0/1705794836 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1478113b80 con 0x7f14780fcbb0 2024-02-23T00:51:23.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.127+0000 7f147e731700 1 librados: init done 2024-02-23T00:51:23.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.128+0000 7f147e731700 1 -- 172.21.15.47:0/1705794836 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f147805fa60 con 0x7f14780fcbb0 2024-02-23T00:51:23.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.129+0000 7f14757fa700 1 -- 172.21.15.47:0/1705794836 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f146c007b40 con 0x7f14780fcbb0 2024-02-23T00:51:23.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.129+0000 7f14757fa700 1 --2- 172.21.15.47:0/1705794836 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1460062cc0 0x7f1460065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:23.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.129+0000 7f14757fa700 1 -- 172.21.15.47:0/1705794836 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(553..553 src has 463..553) v4 ==== 9220+0+0 (secure 0 0 0) 0x7f146c07d090 con 0x7f14780fcbb0 2024-02-23T00:51:23.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.129+0000 7f14757fa700 10 client.41651.objecter ms_dispatch 0x7f1478001550 osd_map(553..553 src has 463..553) v4 2024-02-23T00:51:23.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.129+0000 7f14757fa700 3 client.41651.objecter handle_osd_map got epochs [553,553] > 0 2024-02-23T00:51:23.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.129+0000 7f14757fa700 3 client.41651.objecter handle_osd_map decoding full epoch 553 2024-02-23T00:51:23.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.129+0000 7f1477fff700 1 --2- 172.21.15.47:0/1705794836 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1460062cc0 0x7f1460065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:23.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.129+0000 7f14757fa700 20 client.41651.objecter dump_active .. 0 homeless 2024-02-23T00:51:23.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.130+0000 7f14757fa700 10 client.41651.objecter ms_handle_connect 0x7f1460062cc0 2024-02-23T00:51:23.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.130+0000 7f1477fff700 1 --2- 172.21.15.47:0/1705794836 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1460062cc0 0x7f1460065180 secure :-1 s=READY pgs=3118 cs=0 l=1 rev1=1 rx=0x7f1468009350 tx=0x7f146800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:23.136 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:51:23.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.134+0000 7f14757fa700 1 -- 172.21.15.47:0/1705794836 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f146c04c750 con 0x7f14780fcbb0 2024-02-23T00:51:23.164 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:23.288 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:51:23.291 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:23.292 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:51:23.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.382+0000 7f147e731700 1 -- 172.21.15.47:0/1705794836 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f1478105ea0 con 0x7f14780fcbb0 2024-02-23T00:51:23.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.382+0000 7f14757fa700 1 -- 172.21.15.47:0/1705794836 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+2378 (secure 0 0 0) 0x7f146c019070 con 0x7f14780fcbb0 2024-02-23T00:51:23.385 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:51:23.385 INFO:teuthology.orchestra.run.smithi047.stdout:{"fsid":"fce2617e-d1e2-11ee-95c0-87774f69a715","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noscrub flag(s) set","count":7},"muted":false},"POOL_APP_NOT_ENABLED":{"severity":"HEALTH_WARN","summary":{"message":"1 pool(s) do not have an application enabled","count":1},"muted":false}},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":968,"monmap":{"epoch":3,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":554,"num_osds":8,"num_up_osds":8,"osd_up_since":1708648745,"num_in_osds":7,"osd_in_since":1708649247,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":208}],"num_pgs":208,"num_pools":9,"num_objects":417,"data_bytes":4026236,"bytes_used":2228449280,"bytes_avail":669736800256,"bytes_total":671965249536},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"services":{"dashboard":"https://172.21.15.47:8443/","prometheus":"http://172.21.15.47:9283/"}},"servicemap":{"epoch":23,"modified":"2024-02-23T00:43:27.194339+0000","services":{"rgw":{"daemons":{"summary":"","14460":{"start_epoch":12,"start_stamp":"2024-02-23T00:39:21.905359+0000","gid":14460,"addr":"172.21.15.47:0/1573276039","metadata":{"arch":"x86_64","ceph_release":"pacific","ceph_version":"ceph version 16.2.14-550-geb66ed92 (eb66ed921e744301e6863be6353618d63967ea59) pacific (stable)","ceph_version_short":"16.2.14-550-geb66ed92","container_hostname":"smithi047","container_image":"quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:44277403522cf37edeee63b04643fd6c6e9416f13c3731a771e3cc8e53b7d010","cpu":"Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz","distro":"centos","distro_description":"CentOS Stream 8","distro_version":"8","frontend_config#0":"beast port=80","frontend_type#0":"beast","hostname":"smithi047","id":"foo.a","kernel_description":"#1 SMP Fri Feb 9 14:58:00 UTC 2024","kernel_version":"4.18.0-540.el8.x86_64","mem_swap_kb":"0","mem_total_kb":"32543848","num_handles":"1","os":"Linux","pid":"7","realm_id":"","realm_name":"","zone_id":"52970d38-d172-4ec0-ad23-3afdfeba31e4","zone_name":"default","zonegroup_id":"343d5407-1492-440c-b380-09e4750e9af7","zonegroup_name":"default"},"task_status":{}}}}}},"progress_events":{}} 2024-02-23T00:51:23.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.386+0000 7f147e731700 10 librados: watch_flush enter 2024-02-23T00:51:23.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.386+0000 7f147e731700 10 librados: watch_flush exit 2024-02-23T00:51:23.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.386+0000 7f147e731700 20 client.41651.objecter shutdown clearing up homeless session... 2024-02-23T00:51:23.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.386+0000 7f147e731700 10 client.41651.objecter successfully canceled tick 2024-02-23T00:51:23.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.386+0000 7f147e731700 1 -- 172.21.15.47:0/1705794836 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1460062cc0 msgr2=0x7f1460065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:23.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.386+0000 7f147e731700 1 --2- 172.21.15.47:0/1705794836 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1460062cc0 0x7f1460065180 secure :-1 s=READY pgs=3118 cs=0 l=1 rev1=1 rx=0x7f1468009350 tx=0x7f146800b040).stop 2024-02-23T00:51:23.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.386+0000 7f147e731700 1 -- 172.21.15.47:0/1705794836 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f14780fcbb0 msgr2=0x7f147810d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:23.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.386+0000 7f147e731700 1 --2- 172.21.15.47:0/1705794836 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f14780fcbb0 0x7f147810d240 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 rx=0x7f146c00e970 tx=0x7f146c00b040).stop 2024-02-23T00:51:23.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.386+0000 7f147e731700 1 -- 172.21.15.47:0/1705794836 shutdown_connections 2024-02-23T00:51:23.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.386+0000 7f147e731700 1 --2- 172.21.15.47:0/1705794836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14780fd510 0x7f1478106ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:23.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.386+0000 7f147e731700 1 --2- 172.21.15.47:0/1705794836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14780f7400 0x7f147810cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:23.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.386+0000 7f147e731700 1 --2- 172.21.15.47:0/1705794836 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1460062cc0 0x7f1460065180 unknown :-1 s=CLOSED pgs=3118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:23.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.386+0000 7f147e731700 1 --2- 172.21.15.47:0/1705794836 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f14780fcbb0 0x7f147810d240 unknown :-1 s=CLOSED pgs=2729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:23.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.386+0000 7f147e731700 1 -- 172.21.15.47:0/1705794836 >> 172.21.15.47:0/1705794836 conn(0x7f14780f2820 msgr2=0x7f14780f4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:23.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.386+0000 7f147e731700 1 -- 172.21.15.47:0/1705794836 shutdown_connections 2024-02-23T00:51:23.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.386+0000 7f147e731700 1 -- 172.21.15.47:0/1705794836 wait complete. 2024-02-23T00:51:23.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:23.387+0000 7f147e731700 1 librados: shutdown 2024-02-23T00:51:23.394 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:51:23.419 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:22.280648+0000 mon.a (mon.0) 2967 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-90"}]': finished 2024-02-23T00:51:23.419 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:22.280716+0000 mon.a (mon.0) 2968 : cluster [DBG] osdmap e553: 8 total, 8 up, 7 in 2024-02-23T00:51:23.421 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:23.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:22.280648+0000 mon.a (mon.0) 2967 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-90"}]': finished 2024-02-23T00:51:23.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:22.280716+0000 mon.a (mon.0) 2968 : cluster [DBG] osdmap e553: 8 total, 8 up, 7 in 2024-02-23T00:51:23.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:22.280648+0000 mon.a (mon.0) 2967 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-90"}]': finished 2024-02-23T00:51:23.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:22.280716+0000 mon.a (mon.0) 2968 : cluster [DBG] osdmap e553: 8 total, 8 up, 7 in 2024-02-23T00:51:24.277 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:51:24.481 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:51:24.482 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-02-23T00:51:24.482 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-02-23T00:51:24.482 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-02-23T00:51:24.482 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:51:24.499 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:22.751904+0000 mgr.y (mgr.14517) 504 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:51:24.499 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:23.284207+0000 mon.a (mon.0) 2969 : cluster [DBG] osdmap e554: 8 total, 8 up, 7 in 2024-02-23T00:51:24.499 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:23.341352+0000 mgr.y (mgr.14517) 505 : cluster [DBG] pgmap v835: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:51:24.499 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:23.383691+0000 mon.c (mon.2) 325 : audit [DBG] from='client.? 172.21.15.47:0/1705794836' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-23T00:51:24.500 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:22.751904+0000 mgr.y (mgr.14517) 504 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:51:24.500 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:23.284207+0000 mon.a (mon.0) 2969 : cluster [DBG] osdmap e554: 8 total, 8 up, 7 in 2024-02-23T00:51:24.500 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:23.341352+0000 mgr.y (mgr.14517) 505 : cluster [DBG] pgmap v835: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:51:24.500 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:23.383691+0000 mon.c (mon.2) 325 : audit [DBG] from='client.? 172.21.15.47:0/1705794836' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-23T00:51:24.526 INFO:tasks.cephadm.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-02-23T00:51:24.526 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph pg dump --format=json 2024-02-23T00:51:24.672 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:51:24.673 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:22.751904+0000 mgr.y (mgr.14517) 504 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:51:24.673 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:23.284207+0000 mon.a (mon.0) 2969 : cluster [DBG] osdmap e554: 8 total, 8 up, 7 in 2024-02-23T00:51:24.673 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:23.341352+0000 mgr.y (mgr.14517) 505 : cluster [DBG] pgmap v835: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:51:24.673 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:23.383691+0000 mon.c (mon.2) 325 : audit [DBG] from='client.? 172.21.15.47:0/1705794836' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-23T00:51:24.674 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:24.675 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:51:24.776 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:51:24.803 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:24.928 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:51:24.932 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:24.932 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:51:24.975 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:51:25.033 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:51:25.041 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.4 dump_historic_ops 2024-02-23T00:51:25.063 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:25.083 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:51:25.086 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:25.086 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:51:25.187 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:51:25.215 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:25.235 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:51:25.238 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:25.238 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:51:25.290 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:24.290689+0000 mon.a (mon.0) 2970 : cluster [DBG] osdmap e555: 8 total, 8 up, 7 in 2024-02-23T00:51:25.340 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:51:25.344 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:51:25.369 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:25.434 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:24.290689+0000 mon.a (mon.0) 2970 : cluster [DBG] osdmap e555: 8 total, 8 up, 7 in 2024-02-23T00:51:25.434 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:24.295575+0000 mon.a (mon.0) 2971 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:51:25.434 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:25.242426+0000 mon.a (mon.0) 2972 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:51:25.434 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:25.242536+0000 mon.a (mon.0) 2973 : cluster [DBG] osdmap e556: 8 total, 8 up, 7 in 2024-02-23T00:51:25.434 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:25.255255+0000 mon.a (mon.0) 2974 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:51:25.434 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:25.258521+0000 mon.a (mon.0) 2975 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-92","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-23T00:51:25.435 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:24.290689+0000 mon.a (mon.0) 2970 : cluster [DBG] osdmap e555: 8 total, 8 up, 7 in 2024-02-23T00:51:25.435 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:24.295575+0000 mon.a (mon.0) 2971 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:51:25.435 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:25.242426+0000 mon.a (mon.0) 2972 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:51:25.435 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:25.242536+0000 mon.a (mon.0) 2973 : cluster [DBG] osdmap e556: 8 total, 8 up, 7 in 2024-02-23T00:51:25.435 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:25.255255+0000 mon.a (mon.0) 2974 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:51:25.435 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:25.258521+0000 mon.a (mon.0) 2975 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-92","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-23T00:51:25.620 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:24.295575+0000 mon.a (mon.0) 2971 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:51:25.620 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:25.242426+0000 mon.a (mon.0) 2972 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:51:25.620 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:25.242536+0000 mon.a (mon.0) 2973 : cluster [DBG] osdmap e556: 8 total, 8 up, 7 in 2024-02-23T00:51:25.620 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:25.255255+0000 mon.a (mon.0) 2974 : audit [DBG] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-23T00:51:25.621 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:25.258521+0000 mon.a (mon.0) 2975 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-92","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-23T00:51:26.297 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:25.341767+0000 mgr.y (mgr.14517) 506 : cluster [DBG] pgmap v838: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:51:26.297 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:25.833690+0000 mon.a (mon.0) 2976 : audit [INF] from='client.? 172.21.15.47:0/2834308300' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:51:26.297 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:26.239574+0000 mon.a (mon.0) 2977 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:51:26.297 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:26.243712+0000 mon.a (mon.0) 2978 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-92","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-23T00:51:26.297 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:26.243794+0000 mon.a (mon.0) 2979 : audit [INF] from='client.? 172.21.15.47:0/2834308300' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:51:26.297 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:26.243855+0000 mon.a (mon.0) 2980 : cluster [DBG] osdmap e557: 8 total, 8 up, 7 in 2024-02-23T00:51:26.297 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:26.244461+0000 mon.a (mon.0) 2981 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-23T00:51:26.418 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:51:26.421 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:26.422 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:51:26.522 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:51:26.551 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:25.341767+0000 mgr.y (mgr.14517) 506 : cluster [DBG] pgmap v838: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:51:26.551 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:25.833690+0000 mon.a (mon.0) 2976 : audit [INF] from='client.? 172.21.15.47:0/2834308300' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:51:26.551 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:26.239574+0000 mon.a (mon.0) 2977 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:51:26.551 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:26.243712+0000 mon.a (mon.0) 2978 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-92","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-23T00:51:26.551 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:26.243794+0000 mon.a (mon.0) 2979 : audit [INF] from='client.? 172.21.15.47:0/2834308300' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:51:26.551 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:26.243855+0000 mon.a (mon.0) 2980 : cluster [DBG] osdmap e557: 8 total, 8 up, 7 in 2024-02-23T00:51:26.551 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:26.244461+0000 mon.a (mon.0) 2981 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-23T00:51:26.553 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:26.651 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:51:26.655 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:26.655 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:51:26.655 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:25.341767+0000 mgr.y (mgr.14517) 506 : cluster [DBG] pgmap v838: 208 pgs: 32 unknown, 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:51:26.656 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:25.833690+0000 mon.a (mon.0) 2976 : audit [INF] from='client.? 172.21.15.47:0/2834308300' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:51:26.656 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:26.239574+0000 mon.a (mon.0) 2977 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:51:26.656 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:26.243712+0000 mon.a (mon.0) 2978 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-92","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-23T00:51:26.656 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:26.243794+0000 mon.a (mon.0) 2979 : audit [INF] from='client.? 172.21.15.47:0/2834308300' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:51:26.656 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:26.243855+0000 mon.a (mon.0) 2980 : cluster [DBG] osdmap e557: 8 total, 8 up, 7 in 2024-02-23T00:51:26.656 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:26.244461+0000 mon.a (mon.0) 2981 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-23T00:51:26.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.636+0000 7f479bb59700 1 -- 172.21.15.47:0/2600407500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47940fd510 msgr2=0x7f47940f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:26.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.636+0000 7f479bb59700 1 --2- 172.21.15.47:0/2600407500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47940fd510 0x7f47940f6d40 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 rx=0x7f47840047b0 tx=0x7f4784009cb0).stop 2024-02-23T00:51:26.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.636+0000 7f479bb59700 1 -- 172.21.15.47:0/2600407500 shutdown_connections 2024-02-23T00:51:26.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.636+0000 7f479bb59700 1 --2- 172.21.15.47:0/2600407500 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47940f7400 0x7f4794103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:26.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.636+0000 7f479bb59700 1 --2- 172.21.15.47:0/2600407500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47940fcbb0 0x7f47940fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:26.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.636+0000 7f479bb59700 1 --2- 172.21.15.47:0/2600407500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47940fd510 0x7f47940f6d40 unknown :-1 s=CLOSED pgs=2575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:26.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.636+0000 7f479bb59700 1 -- 172.21.15.47:0/2600407500 >> 172.21.15.47:0/2600407500 conn(0x7f47940f2820 msgr2=0x7f47940f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:26.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.636+0000 7f479bb59700 1 -- 172.21.15.47:0/2600407500 shutdown_connections 2024-02-23T00:51:26.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.636+0000 7f479bb59700 1 -- 172.21.15.47:0/2600407500 wait complete. 2024-02-23T00:51:26.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.637+0000 7f479bb59700 1 librados: starting msgr at 2024-02-23T00:51:26.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.637+0000 7f479bb59700 1 librados: starting objecter 2024-02-23T00:51:26.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.637+0000 7f479bb59700 1 Processor -- start 2024-02-23T00:51:26.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.637+0000 7f479bb59700 1 -- start start 2024-02-23T00:51:26.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.637+0000 7f479bb59700 1 librados: setting wanted keys 2024-02-23T00:51:26.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.637+0000 7f479bb59700 1 librados: calling monclient init 2024-02-23T00:51:26.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.637+0000 7f479bb59700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47940f7400 0x7f479410e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:26.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.637+0000 7f479bb59700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47940fcbb0 0x7f479410e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:26.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.637+0000 7f479bb59700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47940fd510 0x7f4794108230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:26.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.637+0000 7f479bb59700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4794108720 con 0x7f47940f7400 2024-02-23T00:51:26.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.637+0000 7f479bb59700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4794108890 con 0x7f47940fcbb0 2024-02-23T00:51:26.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.637+0000 7f479bb59700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4794108a50 con 0x7f47940fd510 2024-02-23T00:51:26.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.637+0000 7f47998f5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47940f7400 0x7f479410e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:26.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.637+0000 7f47998f5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47940f7400 0x7f479410e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41766/0 (socket says 172.21.15.47:41766) 2024-02-23T00:51:26.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.637+0000 7f47998f5700 1 -- 172.21.15.47:0/2834308300 learned_addr learned my addr 172.21.15.47:0/2834308300 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:51:26.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.637+0000 7f47990f4700 1 --2- 172.21.15.47:0/2834308300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47940fcbb0 0x7f479410e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:26.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.637+0000 7f479a0f6700 1 --2- 172.21.15.47:0/2834308300 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47940fd510 0x7f4794108230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:26.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.638+0000 7f47998f5700 1 -- 172.21.15.47:0/2834308300 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47940fd510 msgr2=0x7f4794108230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:26.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.638+0000 7f47998f5700 1 --2- 172.21.15.47:0/2834308300 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47940fd510 0x7f4794108230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:26.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.638+0000 7f47998f5700 1 -- 172.21.15.47:0/2834308300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47940fcbb0 msgr2=0x7f479410e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:26.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.638+0000 7f47998f5700 1 --2- 172.21.15.47:0/2834308300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47940fcbb0 0x7f479410e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:26.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.638+0000 7f47998f5700 1 -- 172.21.15.47:0/2834308300 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f478c0097c0 con 0x7f47940f7400 2024-02-23T00:51:26.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.638+0000 7f47998f5700 1 --2- 172.21.15.47:0/2834308300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47940f7400 0x7f479410e040 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 rx=0x7f479000e970 tx=0x7f479000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:26.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.638+0000 7f478affd700 10 client.?.objecter ms_handle_connect 0x7f47940f7400 2024-02-23T00:51:26.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.638+0000 7f478affd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:51:26.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.638+0000 7f478affd700 1 -- 172.21.15.47:0/2834308300 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4790004660 con 0x7f47940f7400 2024-02-23T00:51:26.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.638+0000 7f479bb59700 1 -- 172.21.15.47:0/2834308300 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4784005040 con 0x7f47940f7400 2024-02-23T00:51:26.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.638+0000 7f478affd700 1 -- 172.21.15.47:0/2834308300 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f479000f040 con 0x7f47940f7400 2024-02-23T00:51:26.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.638+0000 7f478affd700 1 -- 172.21.15.47:0/2834308300 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47900146f0 con 0x7f47940f7400 2024-02-23T00:51:26.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.638+0000 7f479bb59700 10 client.31338.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:51:26.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.638+0000 7f479bb59700 1 -- 172.21.15.47:0/2834308300 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4794114e70 con 0x7f47940f7400 2024-02-23T00:51:26.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.638+0000 7f479bb59700 1 librados: init done 2024-02-23T00:51:26.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.639+0000 7f479bb59700 1 -- 172.21.15.47:0/2834308300 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f479405fa60 con 0x7f47940f7400 2024-02-23T00:51:26.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.639+0000 7f478affd700 1 -- 172.21.15.47:0/2834308300 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f4790007b40 con 0x7f47940f7400 2024-02-23T00:51:26.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.640+0000 7f478affd700 1 --2- 172.21.15.47:0/2834308300 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4780062cc0 0x7f4780065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:26.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.640+0000 7f478affd700 1 -- 172.21.15.47:0/2834308300 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(556..556 src has 463..556) v4 ==== 9220+0+0 (secure 0 0 0) 0x7f479007d140 con 0x7f47940f7400 2024-02-23T00:51:26.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.640+0000 7f478affd700 10 client.31338.objecter ms_dispatch 0x7f4794001550 osd_map(556..556 src has 463..556) v4 2024-02-23T00:51:26.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.640+0000 7f478affd700 3 client.31338.objecter handle_osd_map got epochs [556,556] > 0 2024-02-23T00:51:26.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.640+0000 7f478affd700 3 client.31338.objecter handle_osd_map decoding full epoch 556 2024-02-23T00:51:26.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.640+0000 7f47990f4700 1 --2- 172.21.15.47:0/2834308300 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4780062cc0 0x7f4780065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:26.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.640+0000 7f478affd700 20 client.31338.objecter dump_active .. 0 homeless 2024-02-23T00:51:26.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.640+0000 7f478affd700 10 client.31338.objecter ms_handle_connect 0x7f4780062cc0 2024-02-23T00:51:26.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.640+0000 7f47990f4700 1 --2- 172.21.15.47:0/2834308300 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4780062cc0 0x7f4780065180 secure :-1 s=READY pgs=3119 cs=0 l=1 rev1=1 rx=0x7f4784009bc0 tx=0x7f4784009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:26.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.646+0000 7f478affd700 1 -- 172.21.15.47:0/2834308300 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f479004c800 con 0x7f47940f7400 2024-02-23T00:51:26.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:25.832+0000 7f479bb59700 1 -- 172.21.15.47:0/2834308300 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7f4794105ee0 con 0x7f47940f7400 2024-02-23T00:51:26.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:26.242+0000 7f478affd700 1 -- 172.21.15.47:0/2834308300 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v557) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4790019070 con 0x7f47940f7400 2024-02-23T00:51:26.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:26.245+0000 7f479bb59700 10 librados: watch_flush enter 2024-02-23T00:51:26.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:26.245+0000 7f479bb59700 10 librados: watch_flush exit 2024-02-23T00:51:26.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:26.245+0000 7f479bb59700 20 client.31338.objecter shutdown clearing up homeless session... 2024-02-23T00:51:26.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:26.245+0000 7f479bb59700 10 client.31338.objecter successfully canceled tick 2024-02-23T00:51:26.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:26.245+0000 7f479bb59700 1 -- 172.21.15.47:0/2834308300 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4780062cc0 msgr2=0x7f4780065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:26.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:26.245+0000 7f479bb59700 1 --2- 172.21.15.47:0/2834308300 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4780062cc0 0x7f4780065180 secure :-1 s=READY pgs=3119 cs=0 l=1 rev1=1 rx=0x7f4784009bc0 tx=0x7f4784009cb0).stop 2024-02-23T00:51:26.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:26.245+0000 7f479bb59700 1 -- 172.21.15.47:0/2834308300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47940f7400 msgr2=0x7f479410e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:26.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:26.245+0000 7f479bb59700 1 --2- 172.21.15.47:0/2834308300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47940f7400 0x7f479410e040 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 rx=0x7f479000e970 tx=0x7f479000b040).stop 2024-02-23T00:51:26.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:26.245+0000 7f479bb59700 1 -- 172.21.15.47:0/2834308300 shutdown_connections 2024-02-23T00:51:26.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:26.245+0000 7f479bb59700 1 --2- 172.21.15.47:0/2834308300 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4780062cc0 0x7f4780065180 unknown :-1 s=CLOSED pgs=3119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:26.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:26.245+0000 7f479bb59700 1 --2- 172.21.15.47:0/2834308300 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47940fd510 0x7f4794108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:26.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:26.245+0000 7f479bb59700 1 --2- 172.21.15.47:0/2834308300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47940f7400 0x7f479410e040 unknown :-1 s=CLOSED pgs=2576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:26.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:26.245+0000 7f479bb59700 1 --2- 172.21.15.47:0/2834308300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47940fcbb0 0x7f479410e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:26.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:26.245+0000 7f479bb59700 1 -- 172.21.15.47:0/2834308300 >> 172.21.15.47:0/2834308300 conn(0x7f47940f2820 msgr2=0x7f4794060eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:26.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:26.245+0000 7f479bb59700 1 -- 172.21.15.47:0/2834308300 shutdown_connections 2024-02-23T00:51:26.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:26.245+0000 7f479bb59700 1 -- 172.21.15.47:0/2834308300 wait complete. 2024-02-23T00:51:26.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:26.245+0000 7f479bb59700 1 librados: shutdown 2024-02-23T00:51:26.749 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is set 2024-02-23T00:51:26.756 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:51:26.785 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:26.910 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:51:26.913 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:26.914 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:51:26.981 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:51:27.014 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:51:27.043 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:27.082 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:51:27.084 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:27.085 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:51:27.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.089+0000 7f56569be700 1 -- 172.21.15.47:0/2897747461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56500f9610 msgr2=0x7f5650101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:27.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.089+0000 7f56569be700 1 --2- 172.21.15.47:0/2897747461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56500f9610 0x7f5650101d20 secure :-1 s=READY pgs=3977 cs=0 l=1 rev1=1 rx=0x7f5644004870 tx=0x7f5644009cb0).stop 2024-02-23T00:51:27.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.089+0000 7f56569be700 1 -- 172.21.15.47:0/2897747461 shutdown_connections 2024-02-23T00:51:27.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.089+0000 7f56569be700 1 --2- 172.21.15.47:0/2897747461 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5650102210 0x7f5650104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:27.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.089+0000 7f56569be700 1 --2- 172.21.15.47:0/2897747461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56500f8cb0 0x7f56500f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:27.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.089+0000 7f56569be700 1 --2- 172.21.15.47:0/2897747461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56500f9610 0x7f5650101d20 unknown :-1 s=CLOSED pgs=3977 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:27.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.089+0000 7f56569be700 1 -- 172.21.15.47:0/2897747461 >> 172.21.15.47:0/2897747461 conn(0x7f56500f2820 msgr2=0x7f56500f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:27.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.089+0000 7f56569be700 1 -- 172.21.15.47:0/2897747461 shutdown_connections 2024-02-23T00:51:27.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.089+0000 7f56569be700 1 -- 172.21.15.47:0/2897747461 wait complete. 2024-02-23T00:51:27.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.090+0000 7f56569be700 1 librados: starting msgr at 2024-02-23T00:51:27.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.090+0000 7f56569be700 1 librados: starting objecter 2024-02-23T00:51:27.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.090+0000 7f56569be700 1 Processor -- start 2024-02-23T00:51:27.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.090+0000 7f56569be700 1 -- start start 2024-02-23T00:51:27.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.090+0000 7f56569be700 1 librados: setting wanted keys 2024-02-23T00:51:27.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.090+0000 7f56569be700 1 librados: calling monclient init 2024-02-23T00:51:27.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.090+0000 7f56569be700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f56500f8cb0 0x7f56500f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:27.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.090+0000 7f56569be700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56500f9610 0x7f56500f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:27.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.090+0000 7f56569be700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5650102210 0x7f56500f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:27.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.090+0000 7f56569be700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5650116530 con 0x7f56500f9610 2024-02-23T00:51:27.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.090+0000 7f56569be700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56501166a0 con 0x7f5650102210 2024-02-23T00:51:27.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.090+0000 7f56569be700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5650116860 con 0x7f56500f8cb0 2024-02-23T00:51:27.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.090+0000 7f564ffff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f56500f8cb0 0x7f56500f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:27.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.090+0000 7f564ffff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f56500f8cb0 0x7f56500f6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:43008/0 (socket says 172.21.15.47:43008) 2024-02-23T00:51:27.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.090+0000 7f564ffff700 1 -- 172.21.15.47:0/3834819343 learned_addr learned my addr 172.21.15.47:0/3834819343 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:51:27.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.090+0000 7f564f7fe700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56500f9610 0x7f56500f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:27.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.090+0000 7f5654f5b700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5650102210 0x7f56500f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:27.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.091+0000 7f564ffff700 1 -- 172.21.15.47:0/3834819343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5650102210 msgr2=0x7f56500f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:27.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.091+0000 7f564ffff700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5650102210 0x7f56500f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:27.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.091+0000 7f564ffff700 1 -- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56500f9610 msgr2=0x7f56500f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:27.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.091+0000 7f564ffff700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56500f9610 0x7f56500f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:27.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.091+0000 7f564ffff700 1 -- 172.21.15.47:0/3834819343 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5644005040 con 0x7f56500f8cb0 2024-02-23T00:51:27.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.091+0000 7f564f7fe700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56500f9610 0x7f56500f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:51:27.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.091+0000 7f564ffff700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f56500f8cb0 0x7f56500f6c60 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 rx=0x7f564000e970 tx=0x7f564000b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:27.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.091+0000 7f564d7fa700 10 client.?.objecter ms_handle_connect 0x7f56500f8cb0 2024-02-23T00:51:27.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.091+0000 7f564d7fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:51:27.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.091+0000 7f564d7fa700 1 -- 172.21.15.47:0/3834819343 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5640004660 con 0x7f56500f8cb0 2024-02-23T00:51:27.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.091+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5650116b40 con 0x7f56500f8cb0 2024-02-23T00:51:27.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.091+0000 7f564d7fa700 1 -- 172.21.15.47:0/3834819343 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f564000f040 con 0x7f56500f8cb0 2024-02-23T00:51:27.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.091+0000 7f564d7fa700 1 -- 172.21.15.47:0/3834819343 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56400146f0 con 0x7f56500f8cb0 2024-02-23T00:51:27.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.091+0000 7f56569be700 10 client.41663.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:51:27.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.091+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5650117050 con 0x7f56500f8cb0 2024-02-23T00:51:27.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.091+0000 7f56569be700 1 librados: init done 2024-02-23T00:51:27.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.092+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f565005fa60 con 0x7f56500f8cb0 2024-02-23T00:51:27.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.092+0000 7f564d7fa700 1 -- 172.21.15.47:0/3834819343 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f56400133c0 con 0x7f56500f8cb0 2024-02-23T00:51:27.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.093+0000 7f564d7fa700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5638062cc0 0x7f5638065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:27.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.093+0000 7f564d7fa700 1 -- 172.21.15.47:0/3834819343 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(557..557 src has 463..557) v4 ==== 9236+0+0 (secure 0 0 0) 0x7f564007d200 con 0x7f56500f8cb0 2024-02-23T00:51:27.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.093+0000 7f564d7fa700 10 client.41663.objecter ms_dispatch 0x7f5650001550 osd_map(557..557 src has 463..557) v4 2024-02-23T00:51:27.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.093+0000 7f564d7fa700 3 client.41663.objecter handle_osd_map got epochs [557,557] > 0 2024-02-23T00:51:27.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.093+0000 7f564d7fa700 3 client.41663.objecter handle_osd_map decoding full epoch 557 2024-02-23T00:51:27.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.093+0000 7f564f7fe700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5638062cc0 0x7f5638065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:27.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.093+0000 7f564d7fa700 20 client.41663.objecter dump_active .. 0 homeless 2024-02-23T00:51:27.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.093+0000 7f564d7fa700 1 -- 172.21.15.47:0/3834819343 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v557) v1 ==== 54+0+16 (secure 0 0 0) 0x7f5640084070 con 0x7f56500f8cb0 2024-02-23T00:51:27.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.093+0000 7f56569be700 10 client.41663.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:27.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.093+0000 7f56569be700 15 client.41663.objecter _session_command_op_assign -1 1 2024-02-23T00:51:27.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.093+0000 7f564d7fa700 10 client.41663.objecter ms_handle_connect 0x7f5638062cc0 2024-02-23T00:51:27.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.093+0000 7f564f7fe700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5638062cc0 0x7f5638065180 secure :-1 s=READY pgs=3120 cs=0 l=1 rev1=1 rx=0x7f5644004ee0 tx=0x7f564400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:27.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.093+0000 7f56569be700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f5650028540 0x7f56500fbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:27.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.093+0000 7f56569be700 20 client.41663.objecter _get_session s=0x7f565005ee80 osd=0 3 2024-02-23T00:51:27.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.093+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f565005ee80 osd=0 3 2024-02-23T00:51:27.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.093+0000 7f56569be700 20 client.41663.objecter _get_session s=0x7f565005ee80 osd=0 3 2024-02-23T00:51:27.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.093+0000 7f56569be700 15 client.41663.objecter _session_command_op_remove -1 1 2024-02-23T00:51:27.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.093+0000 7f56569be700 20 client.41663.objecter get_session s=0x7f565005ee80 osd=0 3 2024-02-23T00:51:27.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.093+0000 7f56569be700 15 client.41663.objecter _session_command_op_assign 0 1 2024-02-23T00:51:27.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.093+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f565005ee80 osd=0 4 2024-02-23T00:51:27.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.093+0000 7f56569be700 10 client.41663.objecter _send_command 1 2024-02-23T00:51:27.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.093+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5650107050 con 0x7f5650028540 2024-02-23T00:51:27.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.094+0000 7f5654f5b700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f5650028540 0x7f56500fbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:27.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.094+0000 7f564d7fa700 10 client.41663.objecter ms_handle_connect 0x7f5650028540 2024-02-23T00:51:27.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.094+0000 7f5654f5b700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f5650028540 0x7f56500fbc40 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:27.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.094+0000 7f564d7fa700 1 -- 172.21.15.47:0/3834819343 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5650107050 con 0x7f5650028540 2024-02-23T00:51:27.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.094+0000 7f564d7fa700 10 client.41663.objecter ms_dispatch 0x7f5650001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:51:27.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.094+0000 7f564d7fa700 10 client.41663.objecter _finish_command 1 = system:0 2024-02-23T00:51:27.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.094+0000 7f564d7fa700 20 client.41663.objecter put_session s=0x7f565005ee80 osd=0 4 2024-02-23T00:51:27.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.094+0000 7f564d7fa700 15 client.41663.objecter _session_command_op_remove 0 1 2024-02-23T00:51:27.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.110+0000 7f56569be700 10 client.41663.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:51:27.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.110+0000 7f56569be700 15 client.41663.objecter _session_command_op_assign -1 2 2024-02-23T00:51:27.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.110+0000 7f56569be700 20 client.41663.objecter _get_session s=0x7f565005ee80 osd=0 3 2024-02-23T00:51:27.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.110+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f565005ee80 osd=0 3 2024-02-23T00:51:27.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.110+0000 7f56569be700 20 client.41663.objecter _get_session s=0x7f565005ee80 osd=0 3 2024-02-23T00:51:27.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.110+0000 7f56569be700 15 client.41663.objecter _session_command_op_remove -1 2 2024-02-23T00:51:27.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.110+0000 7f56569be700 20 client.41663.objecter get_session s=0x7f565005ee80 osd=0 3 2024-02-23T00:51:27.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.110+0000 7f56569be700 15 client.41663.objecter _session_command_op_assign 0 2 2024-02-23T00:51:27.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.110+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f565005ee80 osd=0 4 2024-02-23T00:51:27.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.110+0000 7f56569be700 10 client.41663.objecter _send_command 2 2024-02-23T00:51:27.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.110+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f565005a040 con 0x7f5650028540 2024-02-23T00:51:27.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.112+0000 7f564d7fa700 1 -- 172.21.15.47:0/3834819343 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f565005a040 con 0x7f5650028540 2024-02-23T00:51:27.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.112+0000 7f564d7fa700 10 client.41663.objecter ms_dispatch 0x7f5650001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:51:27.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.112+0000 7f564d7fa700 10 client.41663.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:51:27.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.112+0000 7f564d7fa700 20 client.41663.objecter put_session s=0x7f565005ee80 osd=0 4 2024-02-23T00:51:27.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.112+0000 7f564d7fa700 15 client.41663.objecter _session_command_op_remove 0 2 2024-02-23T00:51:27.115 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:51:27.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.112+0000 7f56569be700 10 client.41663.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:27.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.112+0000 7f56569be700 15 client.41663.objecter _session_command_op_assign -1 3 2024-02-23T00:51:27.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.112+0000 7f56569be700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f5650027b90 0x7f5650027fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:27.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.112+0000 7f56569be700 20 client.41663.objecter _get_session s=0x7f56500fd2b0 osd=1 3 2024-02-23T00:51:27.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.112+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f56500fd2b0 osd=1 3 2024-02-23T00:51:27.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.112+0000 7f56569be700 20 client.41663.objecter _get_session s=0x7f56500fd2b0 osd=1 3 2024-02-23T00:51:27.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.112+0000 7f56569be700 15 client.41663.objecter _session_command_op_remove -1 3 2024-02-23T00:51:27.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.112+0000 7f56569be700 20 client.41663.objecter get_session s=0x7f56500fd2b0 osd=1 3 2024-02-23T00:51:27.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.112+0000 7f56569be700 15 client.41663.objecter _session_command_op_assign 1 3 2024-02-23T00:51:27.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.112+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f56500fd2b0 osd=1 4 2024-02-23T00:51:27.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.112+0000 7f56569be700 10 client.41663.objecter _send_command 3 2024-02-23T00:51:27.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.112+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56500fd420 con 0x7f5650027b90 2024-02-23T00:51:27.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.116+0000 7f564ffff700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f5650027b90 0x7f5650027fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:27.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.116+0000 7f564d7fa700 10 client.41663.objecter ms_handle_connect 0x7f5650027b90 2024-02-23T00:51:27.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.116+0000 7f564ffff700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f5650027b90 0x7f5650027fd0 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:27.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.117+0000 7f564d7fa700 1 -- 172.21.15.47:0/3834819343 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f56500fd420 con 0x7f5650027b90 2024-02-23T00:51:27.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.117+0000 7f564d7fa700 10 client.41663.objecter ms_dispatch 0x7f5650001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:51:27.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.117+0000 7f564d7fa700 10 client.41663.objecter _finish_command 3 = system:0 2024-02-23T00:51:27.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.117+0000 7f564d7fa700 20 client.41663.objecter put_session s=0x7f56500fd2b0 osd=1 4 2024-02-23T00:51:27.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.117+0000 7f564d7fa700 15 client.41663.objecter _session_command_op_remove 1 3 2024-02-23T00:51:27.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.132+0000 7f56569be700 10 client.41663.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:51:27.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.132+0000 7f56569be700 15 client.41663.objecter _session_command_op_assign -1 4 2024-02-23T00:51:27.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.132+0000 7f56569be700 20 client.41663.objecter _get_session s=0x7f56500fd2b0 osd=1 3 2024-02-23T00:51:27.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.132+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f56500fd2b0 osd=1 3 2024-02-23T00:51:27.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.132+0000 7f56569be700 20 client.41663.objecter _get_session s=0x7f56500fd2b0 osd=1 3 2024-02-23T00:51:27.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.132+0000 7f56569be700 15 client.41663.objecter _session_command_op_remove -1 4 2024-02-23T00:51:27.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.132+0000 7f56569be700 20 client.41663.objecter get_session s=0x7f56500fd2b0 osd=1 3 2024-02-23T00:51:27.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.132+0000 7f56569be700 15 client.41663.objecter _session_command_op_assign 1 4 2024-02-23T00:51:27.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.132+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f56500fd2b0 osd=1 4 2024-02-23T00:51:27.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.132+0000 7f56569be700 10 client.41663.objecter _send_command 4 2024-02-23T00:51:27.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.132+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f5650027600 con 0x7f5650027b90 2024-02-23T00:51:27.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.133+0000 7f564d7fa700 1 -- 172.21.15.47:0/3834819343 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f5650027600 con 0x7f5650027b90 2024-02-23T00:51:27.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.133+0000 7f564d7fa700 10 client.41663.objecter ms_dispatch 0x7f5650001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:51:27.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.133+0000 7f564d7fa700 10 client.41663.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:51:27.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.133+0000 7f564d7fa700 20 client.41663.objecter put_session s=0x7f56500fd2b0 osd=1 4 2024-02-23T00:51:27.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.133+0000 7f564d7fa700 15 client.41663.objecter _session_command_op_remove 1 4 2024-02-23T00:51:27.136 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:51:27.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.133+0000 7f56569be700 10 client.41663.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:27.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.133+0000 7f56569be700 15 client.41663.objecter _session_command_op_assign -1 5 2024-02-23T00:51:27.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.133+0000 7f56569be700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f56500fce10 0x7f565011e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:27.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.133+0000 7f56569be700 20 client.41663.objecter _get_session s=0x7f5650027a10 osd=2 3 2024-02-23T00:51:27.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.133+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f5650027a10 osd=2 3 2024-02-23T00:51:27.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.133+0000 7f56569be700 20 client.41663.objecter _get_session s=0x7f5650027a10 osd=2 3 2024-02-23T00:51:27.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.133+0000 7f56569be700 15 client.41663.objecter _session_command_op_remove -1 5 2024-02-23T00:51:27.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.133+0000 7f56569be700 20 client.41663.objecter get_session s=0x7f5650027a10 osd=2 3 2024-02-23T00:51:27.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.133+0000 7f56569be700 15 client.41663.objecter _session_command_op_assign 2 5 2024-02-23T00:51:27.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.133+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f5650027a10 osd=2 4 2024-02-23T00:51:27.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.133+0000 7f56569be700 10 client.41663.objecter _send_command 5 2024-02-23T00:51:27.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.133+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5650121120 con 0x7f56500fce10 2024-02-23T00:51:27.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.133+0000 7f564f7fe700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f56500fce10 0x7f565011e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:27.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.134+0000 7f564d7fa700 10 client.41663.objecter ms_handle_connect 0x7f56500fce10 2024-02-23T00:51:27.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.134+0000 7f564f7fe700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f56500fce10 0x7f565011e3f0 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:27.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.134+0000 7f564d7fa700 1 -- 172.21.15.47:0/3834819343 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5650121120 con 0x7f56500fce10 2024-02-23T00:51:27.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.134+0000 7f564d7fa700 10 client.41663.objecter ms_dispatch 0x7f5650001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:51:27.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.134+0000 7f564d7fa700 10 client.41663.objecter _finish_command 5 = system:0 2024-02-23T00:51:27.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.134+0000 7f564d7fa700 20 client.41663.objecter put_session s=0x7f5650027a10 osd=2 4 2024-02-23T00:51:27.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.134+0000 7f564d7fa700 15 client.41663.objecter _session_command_op_remove 2 5 2024-02-23T00:51:27.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.149+0000 7f56569be700 10 client.41663.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:51:27.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.149+0000 7f56569be700 15 client.41663.objecter _session_command_op_assign -1 6 2024-02-23T00:51:27.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.149+0000 7f56569be700 20 client.41663.objecter _get_session s=0x7f5650027a10 osd=2 3 2024-02-23T00:51:27.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.149+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f5650027a10 osd=2 3 2024-02-23T00:51:27.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.149+0000 7f56569be700 20 client.41663.objecter _get_session s=0x7f5650027a10 osd=2 3 2024-02-23T00:51:27.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.149+0000 7f56569be700 15 client.41663.objecter _session_command_op_remove -1 6 2024-02-23T00:51:27.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.149+0000 7f56569be700 20 client.41663.objecter get_session s=0x7f5650027a10 osd=2 3 2024-02-23T00:51:27.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.149+0000 7f56569be700 15 client.41663.objecter _session_command_op_assign 2 6 2024-02-23T00:51:27.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.149+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f5650027a10 osd=2 4 2024-02-23T00:51:27.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.149+0000 7f56569be700 10 client.41663.objecter _send_command 6 2024-02-23T00:51:27.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.149+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f56500fc370 con 0x7f56500fce10 2024-02-23T00:51:27.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.150+0000 7f564d7fa700 1 -- 172.21.15.47:0/3834819343 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f56500fc370 con 0x7f56500fce10 2024-02-23T00:51:27.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.150+0000 7f564d7fa700 10 client.41663.objecter ms_dispatch 0x7f5650001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:51:27.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.150+0000 7f564d7fa700 10 client.41663.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:51:27.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.150+0000 7f564d7fa700 20 client.41663.objecter put_session s=0x7f5650027a10 osd=2 4 2024-02-23T00:51:27.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.150+0000 7f564d7fa700 15 client.41663.objecter _session_command_op_remove 2 6 2024-02-23T00:51:27.153 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:51:27.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.150+0000 7f56569be700 10 client.41663.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:27.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.150+0000 7f56569be700 15 client.41663.objecter _session_command_op_assign -1 7 2024-02-23T00:51:27.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.150+0000 7f56569be700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f565011bf90 0x7f565011fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:27.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.150+0000 7f56569be700 20 client.41663.objecter _get_session s=0x7f56500fc780 osd=3 3 2024-02-23T00:51:27.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.150+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f56500fc780 osd=3 3 2024-02-23T00:51:27.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.150+0000 7f56569be700 20 client.41663.objecter _get_session s=0x7f56500fc780 osd=3 3 2024-02-23T00:51:27.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.150+0000 7f56569be700 15 client.41663.objecter _session_command_op_remove -1 7 2024-02-23T00:51:27.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.150+0000 7f56569be700 20 client.41663.objecter get_session s=0x7f56500fc780 osd=3 3 2024-02-23T00:51:27.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.150+0000 7f56569be700 15 client.41663.objecter _session_command_op_assign 3 7 2024-02-23T00:51:27.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.150+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f56500fc780 osd=3 4 2024-02-23T00:51:27.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.150+0000 7f56569be700 10 client.41663.objecter _send_command 7 2024-02-23T00:51:27.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.150+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56501200f0 con 0x7f565011bf90 2024-02-23T00:51:27.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.150+0000 7f5654f5b700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f565011bf90 0x7f565011fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:27.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.151+0000 7f564d7fa700 10 client.41663.objecter ms_handle_connect 0x7f565011bf90 2024-02-23T00:51:27.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.151+0000 7f5654f5b700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f565011bf90 0x7f565011fa60 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:27.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.151+0000 7f564d7fa700 1 -- 172.21.15.47:0/3834819343 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f56501200f0 con 0x7f565011bf90 2024-02-23T00:51:27.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.151+0000 7f564d7fa700 10 client.41663.objecter ms_dispatch 0x7f5650001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:51:27.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.151+0000 7f564d7fa700 10 client.41663.objecter _finish_command 7 = system:0 2024-02-23T00:51:27.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.151+0000 7f564d7fa700 20 client.41663.objecter put_session s=0x7f56500fc780 osd=3 4 2024-02-23T00:51:27.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.151+0000 7f564d7fa700 15 client.41663.objecter _session_command_op_remove 3 7 2024-02-23T00:51:27.158 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:51:27.158 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-02-23T00:51:27.158 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-02-23T00:51:27.158 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-02-23T00:51:27.158 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.158 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.24769.0:15 8.14 8:2afeb74e:test-rados-api-smithi047-82428-7::foo:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected e112)", 2024-02-23T00:51:27.158 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:00.985676+0000", 2024-02-23T00:51:27.158 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 506.16804489399999, 2024-02-23T00:51:27.158 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013198675, 2024-02-23T00:51:27.158 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:27.158 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:27.159 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:27.159 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.24769", 2024-02-23T00:51:27.159 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/820656190", 2024-02-23T00:51:27.159 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 15 2024-02-23T00:51:27.159 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.159 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:27.159 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.159 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:27.159 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.985676+0000", 2024-02-23T00:51:27.159 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:27.159 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.159 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.159 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:27.159 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.985676+0000", 2024-02-23T00:51:27.159 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:51:27.160 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.160 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.160 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:27.160 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.985675+0000", 2024-02-23T00:51:27.160 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6940000000000003e-06 2024-02-23T00:51:27.160 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.160 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.160 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:27.160 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.985680+0000", 2024-02-23T00:51:27.160 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.9999999999999999e-06 2024-02-23T00:51:27.160 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.160 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.160 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:27.160 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.985687+0000", 2024-02-23T00:51:27.161 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3930000000000001e-06 2024-02-23T00:51:27.161 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.161 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.161 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:27.161 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.985691+0000", 2024-02-23T00:51:27.161 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.5889999999999997e-05 2024-02-23T00:51:27.161 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.161 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.161 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:27.161 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.985727+0000", 2024-02-23T00:51:27.161 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9975000000000006e-05 2024-02-23T00:51:27.161 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.161 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.161 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:27.162 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.985807+0000", 2024-02-23T00:51:27.162 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000125964 2024-02-23T00:51:27.162 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.162 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.162 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0,2", 2024-02-23T00:51:27.162 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.985933+0000", 2024-02-23T00:51:27.162 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000320188 2024-02-23T00:51:27.162 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.162 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.162 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:51:27.162 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.986253+0000", 2024-02-23T00:51:27.162 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0063198509999999996 2024-02-23T00:51:27.162 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.162 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.163 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:51:27.163 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.992573+0000", 2024-02-23T00:51:27.163 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061739760000000003 2024-02-23T00:51:27.163 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.163 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.163 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:51:27.163 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.998747+0000", 2024-02-23T00:51:27.163 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0014999999999999e-05 2024-02-23T00:51:27.163 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.163 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.163 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:27.163 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.998827+0000", 2024-02-23T00:51:27.163 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7908999999999998e-05 2024-02-23T00:51:27.163 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.164 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.164 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:27.164 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.998875+0000", 2024-02-23T00:51:27.164 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013198675 2024-02-23T00:51:27.164 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.164 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:27.164 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.164 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.164 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.164 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.15165.0:8 29.10 e112/109 29:0cdc28ab:::47.obj:head v 112'27, mlcod=112'25)", 2024-02-23T00:51:27.164 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:01.975830+0000", 2024-02-23T00:51:27.164 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 505.17789129400001, 2024-02-23T00:51:27.164 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022123151000000001, 2024-02-23T00:51:27.164 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:27.165 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:27.165 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:27.165 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.165 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:27.165 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.975830+0000", 2024-02-23T00:51:27.165 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:27.165 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.165 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.165 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:27.165 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.975830+0000", 2024-02-23T00:51:27.166 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:51:27.166 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.166 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.166 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:27.166 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.975828+0000", 2024-02-23T00:51:27.166 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5201000000000001e-05 2024-02-23T00:51:27.166 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.166 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.166 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:27.166 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.975843+0000", 2024-02-23T00:51:27.166 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9936999999999999e-05 2024-02-23T00:51:27.166 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.166 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.166 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:27.166 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.975863+0000", 2024-02-23T00:51:27.167 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.5790000000000004e-06 2024-02-23T00:51:27.167 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.167 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.167 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:27.167 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.975872+0000", 2024-02-23T00:51:27.167 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.2378000000000005e-05 2024-02-23T00:51:27.167 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.167 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.167 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:27.167 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.975934+0000", 2024-02-23T00:51:27.167 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7061000000000001e-05 2024-02-23T00:51:27.167 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.167 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.167 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:27.167 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.975961+0000", 2024-02-23T00:51:27.168 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021960121999999999 2024-02-23T00:51:27.168 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.168 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.168 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:27.168 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.997922+0000", 2024-02-23T00:51:27.168 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.1309999999999997e-05 2024-02-23T00:51:27.168 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.168 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.168 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:27.168 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.997953+0000", 2024-02-23T00:51:27.168 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022123151000000001 2024-02-23T00:51:27.168 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.168 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:27.168 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.169 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.169 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.169 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.15165.0:9 29.17 e112/109 29:eb69ee62:::27.obj:head v 112'18, mlcod=112'16)", 2024-02-23T00:51:27.169 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:01.998811+0000", 2024-02-23T00:51:27.169 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 505.15491009599998, 2024-02-23T00:51:27.169 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.018678802000000001, 2024-02-23T00:51:27.169 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:27.169 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:27.169 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:27.169 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.169 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:27.169 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.998811+0000", 2024-02-23T00:51:27.170 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:27.170 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.170 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.170 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:27.170 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.998811+0000", 2024-02-23T00:51:27.170 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:51:27.170 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.170 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.170 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:27.170 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.998810+0000", 2024-02-23T00:51:27.170 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1049999999999995e-06 2024-02-23T00:51:27.170 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.170 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.170 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:27.171 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.998818+0000", 2024-02-23T00:51:27.171 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2082999999999999e-05 2024-02-23T00:51:27.171 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.171 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.171 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:27.171 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.998830+0000", 2024-02-23T00:51:27.171 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8040000000000004e-06 2024-02-23T00:51:27.171 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.171 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.171 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:27.171 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.998837+0000", 2024-02-23T00:51:27.171 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0862000000000003e-05 2024-02-23T00:51:27.171 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.171 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.171 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:27.172 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.998878+0000", 2024-02-23T00:51:27.172 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8145999999999999e-05 2024-02-23T00:51:27.172 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.172 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.172 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:27.172 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.998896+0000", 2024-02-23T00:51:27.172 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.018492277000000001 2024-02-23T00:51:27.172 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.172 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.172 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:27.172 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.017388+0000", 2024-02-23T00:51:27.172 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000101394 2024-02-23T00:51:27.172 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.173 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.173 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:27.173 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.017490+0000", 2024-02-23T00:51:27.173 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.018678802000000001 2024-02-23T00:51:27.173 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.173 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:27.173 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.173 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.173 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.173 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.15165.0:35 29.2 29:452ae4ca:::6.obj:head [write 0~225] snapc 0=[] ondisk+write+known_if_redirected e112)", 2024-02-23T00:51:27.173 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:02.058811+0000", 2024-02-23T00:51:27.173 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 505.09490985999997, 2024-02-23T00:51:27.173 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0050541459999999998, 2024-02-23T00:51:27.173 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:27.173 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:27.174 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:27.174 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.15165", 2024-02-23T00:51:27.174 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1835198847", 2024-02-23T00:51:27.174 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 35 2024-02-23T00:51:27.174 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.174 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:27.174 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.174 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:27.174 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.058811+0000", 2024-02-23T00:51:27.174 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:27.174 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.174 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.174 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:27.174 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.058811+0000", 2024-02-23T00:51:27.175 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:51:27.175 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.175 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.175 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:27.175 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.058809+0000", 2024-02-23T00:51:27.175 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0716e-05 2024-02-23T00:51:27.175 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.175 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.175 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:27.175 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.058820+0000", 2024-02-23T00:51:27.175 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6276000000000001e-05 2024-02-23T00:51:27.175 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.175 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.175 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:27.176 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.058836+0000", 2024-02-23T00:51:27.176 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3060000000000003e-06 2024-02-23T00:51:27.176 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.176 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.176 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:27.176 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.058842+0000", 2024-02-23T00:51:27.176 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.2403e-05 2024-02-23T00:51:27.176 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.176 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.176 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:27.176 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.058885+0000", 2024-02-23T00:51:27.176 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011220100000000001 2024-02-23T00:51:27.176 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.176 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.177 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:27.177 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.058997+0000", 2024-02-23T00:51:27.177 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011846899999999999 2024-02-23T00:51:27.177 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.177 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.177 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0,2", 2024-02-23T00:51:27.177 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.059115+0000", 2024-02-23T00:51:27.177 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000424592 2024-02-23T00:51:27.177 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.178 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.178 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:51:27.178 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.059540+0000", 2024-02-23T00:51:27.178 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000268719 2024-02-23T00:51:27.178 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.178 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.178 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:51:27.178 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.059809+0000", 2024-02-23T00:51:27.178 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.003963622 2024-02-23T00:51:27.178 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.178 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.178 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:51:27.178 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.063772+0000", 2024-02-23T00:51:27.179 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.2289e-05 2024-02-23T00:51:27.179 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.179 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.179 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:27.179 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.063825+0000", 2024-02-23T00:51:27.179 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0559999999999998e-05 2024-02-23T00:51:27.179 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.179 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.179 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:27.179 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.063865+0000", 2024-02-23T00:51:27.179 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0050541459999999998 2024-02-23T00:51:27.179 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.179 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:27.179 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.180 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.180 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.180 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.15099.0:3 34.9 34:9294e480:nspace::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e114)", 2024-02-23T00:51:27.180 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:03.239954+0000", 2024-02-23T00:51:27.180 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 503.91376655599998, 2024-02-23T00:51:27.180 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0064735499999999998, 2024-02-23T00:51:27.180 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:27.180 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:27.180 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:27.180 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.15099", 2024-02-23T00:51:27.180 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/3592004948", 2024-02-23T00:51:27.180 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 3 2024-02-23T00:51:27.180 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.180 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:27.181 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.181 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:27.181 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.239954+0000", 2024-02-23T00:51:27.181 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:27.181 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.181 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.181 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:27.181 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.239954+0000", 2024-02-23T00:51:27.181 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:51:27.181 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.181 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.181 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:27.181 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.239953+0000", 2024-02-23T00:51:27.181 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.327e-06 2024-02-23T00:51:27.181 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.182 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.182 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:27.182 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.239958+0000", 2024-02-23T00:51:27.182 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.7379999999999994e-06 2024-02-23T00:51:27.182 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.182 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.182 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:27.182 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.239966+0000", 2024-02-23T00:51:27.182 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4240000000000001e-06 2024-02-23T00:51:27.182 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.182 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.182 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:27.182 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.239972+0000", 2024-02-23T00:51:27.182 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.2553999999999999e-05 2024-02-23T00:51:27.182 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.183 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.183 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:27.183 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.240014+0000", 2024-02-23T00:51:27.183 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1692000000000002e-05 2024-02-23T00:51:27.183 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.183 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.183 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:27.183 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.240106+0000", 2024-02-23T00:51:27.183 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103417 2024-02-23T00:51:27.183 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.183 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.183 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0,3", 2024-02-23T00:51:27.183 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.240209+0000", 2024-02-23T00:51:27.183 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00036850300000000001 2024-02-23T00:51:27.183 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.184 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.184 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:51:27.184 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.240578+0000", 2024-02-23T00:51:27.184 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0052171199999999996 2024-02-23T00:51:27.184 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.184 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.184 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:51:27.184 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.245795+0000", 2024-02-23T00:51:27.184 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00057471099999999999 2024-02-23T00:51:27.184 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.184 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.184 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:51:27.184 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.246370+0000", 2024-02-23T00:51:27.184 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.2317999999999998e-05 2024-02-23T00:51:27.185 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.185 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.185 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:27.185 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.246402+0000", 2024-02-23T00:51:27.185 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.6115e-05 2024-02-23T00:51:27.185 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.185 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.185 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:27.185 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.246428+0000", 2024-02-23T00:51:27.185 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0064735499999999998 2024-02-23T00:51:27.185 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.185 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:27.185 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.185 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.185 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.186 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.25120.0:2 41.1a 41:5c6776db:::0.obj:head [write 0~103] snapc 0=[] ondisk+write+known_if_redirected e115)", 2024-02-23T00:51:27.186 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:04.257379+0000", 2024-02-23T00:51:27.186 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 502.896342296, 2024-02-23T00:51:27.186 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.042556126, 2024-02-23T00:51:27.186 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:27.186 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:27.186 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:27.186 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.25120", 2024-02-23T00:51:27.186 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/484148516", 2024-02-23T00:51:27.186 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 2 2024-02-23T00:51:27.186 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.186 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:27.186 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.186 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:27.187 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.257379+0000", 2024-02-23T00:51:27.187 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:27.187 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.187 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.187 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:27.187 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.257379+0000", 2024-02-23T00:51:27.187 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999976 2024-02-23T00:51:27.187 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.187 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.187 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:27.187 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.257376+0000", 2024-02-23T00:51:27.187 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1412e-05 2024-02-23T00:51:27.187 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.187 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.187 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:27.188 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.257388+0000", 2024-02-23T00:51:27.188 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5347999999999999e-05 2024-02-23T00:51:27.188 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.188 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.188 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:27.188 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.257403+0000", 2024-02-23T00:51:27.188 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0669999999999992e-06 2024-02-23T00:51:27.188 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.188 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.188 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:27.188 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.257411+0000", 2024-02-23T00:51:27.188 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012477369 2024-02-23T00:51:27.188 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.188 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.188 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:27.189 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.269889+0000", 2024-02-23T00:51:27.189 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5549999999999997e-06 2024-02-23T00:51:27.189 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.189 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.189 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:51:27.189 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.269896+0000", 2024-02-23T00:51:27.189 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.98403268899999996 2024-02-23T00:51:27.189 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.189 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.189 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:27.189 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:05.253929+0000", 2024-02-23T00:51:27.189 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8703999999999996e-05 2024-02-23T00:51:27.189 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.189 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.190 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:27.190 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:05.254028+0000", 2024-02-23T00:51:27.190 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011678199999999999 2024-02-23T00:51:27.190 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.190 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.190 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1,2", 2024-02-23T00:51:27.190 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:05.254144+0000", 2024-02-23T00:51:27.190 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0017152720000000001 2024-02-23T00:51:27.190 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.190 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.190 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:51:27.190 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:05.255860+0000", 2024-02-23T00:51:27.190 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035976222000000002 2024-02-23T00:51:27.190 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.191 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.191 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:51:27.191 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:05.291836+0000", 2024-02-23T00:51:27.191 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0080063189999999996 2024-02-23T00:51:27.191 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.191 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.191 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:51:27.191 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:05.299842+0000", 2024-02-23T00:51:27.191 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1570000000000003e-05 2024-02-23T00:51:27.191 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.191 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.191 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:27.191 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:05.299894+0000", 2024-02-23T00:51:27.191 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1026000000000003e-05 2024-02-23T00:51:27.191 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.192 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.192 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:27.192 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:05.299935+0000", 2024-02-23T00:51:27.192 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.042556126 2024-02-23T00:51:27.192 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.192 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:27.192 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.192 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.192 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.192 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.25132.0:1 45.1s0 45:88c56992:test-rados-api-smithi047-82394-16::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e117)", 2024-02-23T00:51:27.192 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:06.598180+0000", 2024-02-23T00:51:27.192 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 500.55554121900002, 2024-02-23T00:51:27.192 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.414473189, 2024-02-23T00:51:27.192 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:27.193 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:27.193 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:27.193 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.25132", 2024-02-23T00:51:27.193 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1728200087", 2024-02-23T00:51:27.193 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1 2024-02-23T00:51:27.193 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.193 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:27.193 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.193 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:27.193 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:06.598180+0000", 2024-02-23T00:51:27.193 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:27.193 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.193 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.193 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:27.194 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:06.598180+0000", 2024-02-23T00:51:27.194 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:51:27.194 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.194 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.194 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:27.194 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:06.598178+0000", 2024-02-23T00:51:27.194 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.221e-06 2024-02-23T00:51:27.194 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.194 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.194 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:27.194 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:06.598185+0000", 2024-02-23T00:51:27.194 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1851000000000001e-05 2024-02-23T00:51:27.194 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.194 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.195 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:27.195 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:06.598197+0000", 2024-02-23T00:51:27.195 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9759999999999995e-06 2024-02-23T00:51:27.195 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.195 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.195 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:27.195 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:06.598205+0000", 2024-02-23T00:51:27.195 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1051999999999998e-05 2024-02-23T00:51:27.195 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.195 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.195 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:27.195 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:06.598266+0000", 2024-02-23T00:51:27.195 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5469999999999999e-06 2024-02-23T00:51:27.195 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.195 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.196 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:51:27.196 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:06.598269+0000", 2024-02-23T00:51:27.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.404963942 2024-02-23T00:51:27.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.196 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.196 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:27.196 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:08.003233+0000", 2024-02-23T00:51:27.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000114925 2024-02-23T00:51:27.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.196 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.196 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:27.196 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:08.003348+0000", 2024-02-23T00:51:27.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038947999999999999 2024-02-23T00:51:27.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.197 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.197 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:51:27.197 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:08.003737+0000", 2024-02-23T00:51:27.197 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00066348700000000004 2024-02-23T00:51:27.197 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.197 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.197 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:51:27.197 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:08.004401+0000", 2024-02-23T00:51:27.197 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0082121090000000004 2024-02-23T00:51:27.197 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.197 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.197 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:27.197 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:08.012613+0000", 2024-02-23T00:51:27.197 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0266999999999999e-05 2024-02-23T00:51:27.198 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.198 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.198 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:27.198 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:08.012653+0000", 2024-02-23T00:51:27.198 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.414473189 2024-02-23T00:51:27.198 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.198 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:27.198 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.198 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.198 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.198 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.15147.0:1 44.0s0 44:1ecacb5d:test-rados-api-smithi047-82504-7::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e117)", 2024-02-23T00:51:27.198 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:07.476730+0000", 2024-02-23T00:51:27.198 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 499.67699057099998, 2024-02-23T00:51:27.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.52719580399999999, 2024-02-23T00:51:27.199 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:27.199 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:27.199 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:27.199 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.15147", 2024-02-23T00:51:27.199 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/640613617", 2024-02-23T00:51:27.199 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1 2024-02-23T00:51:27.199 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.199 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:27.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.199 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:27.199 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.476730+0000", 2024-02-23T00:51:27.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:27.199 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:27.200 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.476730+0000", 2024-02-23T00:51:27.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999976 2024-02-23T00:51:27.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:27.200 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.476728+0000", 2024-02-23T00:51:27.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4539999999999996e-06 2024-02-23T00:51:27.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:27.200 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.476738+0000", 2024-02-23T00:51:27.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1059e-05 2024-02-23T00:51:27.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.201 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:27.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.476749+0000", 2024-02-23T00:51:27.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.4679999999999992e-06 2024-02-23T00:51:27.201 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.201 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:27.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.476757+0000", 2024-02-23T00:51:27.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5839000000000004e-05 2024-02-23T00:51:27.201 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.201 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:27.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.476823+0000", 2024-02-23T00:51:27.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0300000000000001e-06 2024-02-23T00:51:27.202 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.202 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.202 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:51:27.202 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.476828+0000", 2024-02-23T00:51:27.202 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.48895222900000002 2024-02-23T00:51:27.202 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.202 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.202 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:27.202 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.965780+0000", 2024-02-23T00:51:27.202 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012051400000000001 2024-02-23T00:51:27.202 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.202 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.202 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:27.202 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.965901+0000", 2024-02-23T00:51:27.203 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040527500000000003 2024-02-23T00:51:27.203 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.203 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:51:27.203 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.966306+0000", 2024-02-23T00:51:27.203 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0058061459999999999 2024-02-23T00:51:27.203 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.203 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:51:27.203 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.972112+0000", 2024-02-23T00:51:27.203 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.031746051999999997 2024-02-23T00:51:27.203 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.203 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:27.203 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:08.003858+0000", 2024-02-23T00:51:27.204 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7968999999999999e-05 2024-02-23T00:51:27.204 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.204 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.204 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:27.204 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:08.003926+0000", 2024-02-23T00:51:27.204 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.52719580399999999 2024-02-23T00:51:27.204 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.204 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:27.204 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.204 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.204 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.204 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.31603.0:1 79.2s0 79:55492aac:test-rados-api-smithi047-82487-10::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e139)", 2024-02-23T00:51:27.204 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:31.194267+0000", 2024-02-23T00:51:27.204 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 475.95945366799998, 2024-02-23T00:51:27.205 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1197622819999999, 2024-02-23T00:51:27.205 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:27.205 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:27.205 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:27.205 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.31603", 2024-02-23T00:51:27.205 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1361361966", 2024-02-23T00:51:27.205 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1 2024-02-23T00:51:27.205 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.205 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:27.205 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.205 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:27.205 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:31.194267+0000", 2024-02-23T00:51:27.205 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:27.205 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.206 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.206 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:27.206 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:31.194267+0000", 2024-02-23T00:51:27.206 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:51:27.206 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.206 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.206 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:27.206 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:31.194266+0000", 2024-02-23T00:51:27.206 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.2689999999999999e-06 2024-02-23T00:51:27.206 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.206 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.206 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:27.207 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:31.194271+0000", 2024-02-23T00:51:27.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.6400000000000003e-06 2024-02-23T00:51:27.207 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.207 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.207 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:27.207 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:31.194280+0000", 2024-02-23T00:51:27.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4630000000000004e-06 2024-02-23T00:51:27.207 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.207 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.207 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:27.207 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:31.194288+0000", 2024-02-23T00:51:27.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7209999999999998e-05 2024-02-23T00:51:27.207 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.207 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.208 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:27.208 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:31.194325+0000", 2024-02-23T00:51:27.208 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7709999999999998e-06 2024-02-23T00:51:27.208 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.208 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.208 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:51:27.208 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:31.194328+0000", 2024-02-23T00:51:27.208 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.077833724 2024-02-23T00:51:27.208 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.208 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.208 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:27.208 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.272161+0000", 2024-02-23T00:51:27.208 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013754699999999999 2024-02-23T00:51:27.208 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.209 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.209 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:27.209 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.272299+0000", 2024-02-23T00:51:27.209 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00036407399999999998 2024-02-23T00:51:27.209 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.209 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.209 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:51:27.209 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.272663+0000", 2024-02-23T00:51:27.209 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031999099999999999 2024-02-23T00:51:27.209 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.209 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.209 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:51:27.209 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.272983+0000", 2024-02-23T00:51:27.209 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.040999727999999999 2024-02-23T00:51:27.209 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.210 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.210 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:27.210 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.313983+0000", 2024-02-23T00:51:27.210 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7034999999999999e-05 2024-02-23T00:51:27.210 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.210 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.210 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:27.210 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.314030+0000", 2024-02-23T00:51:27.210 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1197622819999999 2024-02-23T00:51:27.210 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.210 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:27.210 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.210 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.210 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.210 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.24775.0:2077 20.1d 20:bda0dbd6:test-rados-api-smithi047-82457-17::small.copy:head [copy-from ver 0] snapc 0=[] ondisk+write+known_if_redirected e112)", 2024-02-23T00:51:27.211 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:32.070997+0000", 2024-02-23T00:51:27.211 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 475.08272427499998, 2024-02-23T00:51:27.211 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083743199999999993, 2024-02-23T00:51:27.211 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:27.211 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:27.211 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:27.211 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.24775", 2024-02-23T00:51:27.211 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1191937413", 2024-02-23T00:51:27.211 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 2077 2024-02-23T00:51:27.211 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.211 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:27.211 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.211 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:27.211 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.070997+0000", 2024-02-23T00:51:27.212 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:27.212 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.212 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:27.212 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.070997+0000", 2024-02-23T00:51:27.212 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:51:27.212 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.212 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:27.212 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.070995+0000", 2024-02-23T00:51:27.212 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2429000000000001e-05 2024-02-23T00:51:27.212 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.212 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:27.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.071007+0000", 2024-02-23T00:51:27.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5676999999999999e-05 2024-02-23T00:51:27.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:27.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.071023+0000", 2024-02-23T00:51:27.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.9829999999999999e-06 2024-02-23T00:51:27.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:27.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.071030+0000", 2024-02-23T00:51:27.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.3088e-05 2024-02-23T00:51:27.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.214 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:27.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.071123+0000", 2024-02-23T00:51:27.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015489200000000001 2024-02-23T00:51:27.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.214 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:27.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.071278+0000", 2024-02-23T00:51:27.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097897200000000005 2024-02-23T00:51:27.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.214 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0,7", 2024-02-23T00:51:27.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.072257+0000", 2024-02-23T00:51:27.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00039726399999999998 2024-02-23T00:51:27.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.215 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:51:27.215 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.072654+0000", 2024-02-23T00:51:27.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000233985 2024-02-23T00:51:27.215 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.215 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.215 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:51:27.215 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.072888+0000", 2024-02-23T00:51:27.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0064254150000000003 2024-02-23T00:51:27.215 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.215 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.215 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:51:27.215 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.079313+0000", 2024-02-23T00:51:27.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.1680000000000002e-05 2024-02-23T00:51:27.215 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.215 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.216 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:27.216 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.079345+0000", 2024-02-23T00:51:27.216 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5998000000000001e-05 2024-02-23T00:51:27.216 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.216 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:27.216 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.079371+0000", 2024-02-23T00:51:27.216 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083743199999999993 2024-02-23T00:51:27.216 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.216 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:27.216 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.216 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.217 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(osd.1.0:425 62.11 e195/126 62:8f236abd:test-rados-api-smithi047-82465-14::foo31:7 v 195'33, mlcod=193'29)", 2024-02-23T00:51:27.217 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:44:27.133318+0000", 2024-02-23T00:51:27.217 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 420.02040274299998, 2024-02-23T00:51:27.217 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010179812, 2024-02-23T00:51:27.217 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:27.217 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:27.217 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:27.217 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.217 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:27.217 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:27.133318+0000", 2024-02-23T00:51:27.217 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:27.217 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.217 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.217 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:27.217 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:27.133318+0000", 2024-02-23T00:51:27.218 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:51:27.218 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.218 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.218 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:27.218 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:27.133317+0000", 2024-02-23T00:51:27.218 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1359999999999999e-06 2024-02-23T00:51:27.218 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.218 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.218 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:27.218 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:27.133325+0000", 2024-02-23T00:51:27.218 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3862e-05 2024-02-23T00:51:27.218 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.218 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.218 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:27.218 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:27.133338+0000", 2024-02-23T00:51:27.219 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1419999999999998e-06 2024-02-23T00:51:27.219 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.219 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.219 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:27.219 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:27.133345+0000", 2024-02-23T00:51:27.219 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3120000000000006e-05 2024-02-23T00:51:27.220 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.220 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.220 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:27.220 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:27.133408+0000", 2024-02-23T00:51:27.220 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1288e-05 2024-02-23T00:51:27.220 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.220 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.220 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:27.220 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:27.133429+0000", 2024-02-23T00:51:27.220 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010047270000000001 2024-02-23T00:51:27.220 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.221 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.221 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:27.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:27.143476+0000", 2024-02-23T00:51:27.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1761000000000001e-05 2024-02-23T00:51:27.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.221 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.221 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:27.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:27.143498+0000", 2024-02-23T00:51:27.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010179812 2024-02-23T00:51:27.221 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.221 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:27.221 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.221 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.221 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(osd.3.0:1452 62.c e199/126 62:303239da:test-rados-api-smithi047-82465-14::foo78:4 v 199'28, mlcod=195'24)", 2024-02-23T00:51:27.222 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:44:31.163303+0000", 2024-02-23T00:51:27.222 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 415.99041809599998, 2024-02-23T00:51:27.222 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0058507990000000003, 2024-02-23T00:51:27.222 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:27.222 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:27.222 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:27.222 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.222 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:27.222 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.163303+0000", 2024-02-23T00:51:27.222 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:27.222 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.222 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.222 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:27.222 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.163303+0000", 2024-02-23T00:51:27.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999995 2024-02-23T00:51:27.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.223 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:27.223 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.163302+0000", 2024-02-23T00:51:27.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5060000000000001e-06 2024-02-23T00:51:27.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.223 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:27.223 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.163309+0000", 2024-02-23T00:51:27.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9638000000000001e-05 2024-02-23T00:51:27.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.223 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:27.223 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.163329+0000", 2024-02-23T00:51:27.224 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4399999999999999e-06 2024-02-23T00:51:27.224 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.224 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.224 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:27.224 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.163336+0000", 2024-02-23T00:51:27.224 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1777000000000002e-05 2024-02-23T00:51:27.224 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.224 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.224 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:27.224 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.163388+0000", 2024-02-23T00:51:27.224 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.3034e-05 2024-02-23T00:51:27.224 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.224 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.224 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:27.225 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.163411+0000", 2024-02-23T00:51:27.225 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.005707714 2024-02-23T00:51:27.225 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.225 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.225 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:27.225 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.169119+0000", 2024-02-23T00:51:27.225 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.5179999999999999e-05 2024-02-23T00:51:27.225 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.225 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.225 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:27.225 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.169154+0000", 2024-02-23T00:51:27.225 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0058507990000000003 2024-02-23T00:51:27.225 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.225 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:27.226 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.226 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.226 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.226 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(osd.3.0:1453 62.c e199/126 62:37762822:test-rados-api-smithi047-82465-14::foo50:4 v 199'30, mlcod=195'24)", 2024-02-23T00:51:27.226 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:44:31.163348+0000", 2024-02-23T00:51:27.226 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 415.99037283400003, 2024-02-23T00:51:27.226 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0060680919999999998, 2024-02-23T00:51:27.226 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:27.226 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:27.226 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:27.226 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.226 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:27.226 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.163348+0000", 2024-02-23T00:51:27.226 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:27.226 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.227 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.227 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:27.227 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.163348+0000", 2024-02-23T00:51:27.227 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999995 2024-02-23T00:51:27.227 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.227 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.227 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:27.227 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.163348+0000", 2024-02-23T00:51:27.227 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.3500000000000003e-06 2024-02-23T00:51:27.227 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.227 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.227 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:27.227 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.163357+0000", 2024-02-23T00:51:27.227 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6599999999999998e-06 2024-02-23T00:51:27.228 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.228 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.228 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:27.228 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.163364+0000", 2024-02-23T00:51:27.228 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.2200000000000003e-06 2024-02-23T00:51:27.228 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.228 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.228 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:27.228 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.163368+0000", 2024-02-23T00:51:27.228 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00048882099999999996 2024-02-23T00:51:27.228 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.228 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.228 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:27.228 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.163857+0000", 2024-02-23T00:51:27.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2456e-05 2024-02-23T00:51:27.229 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:27.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.163879+0000", 2024-02-23T00:51:27.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0055061169999999996 2024-02-23T00:51:27.229 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:27.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.169385+0000", 2024-02-23T00:51:27.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.0917000000000002e-05 2024-02-23T00:51:27.229 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:27.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:31.169416+0000", 2024-02-23T00:51:27.230 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0060680919999999998 2024-02-23T00:51:27.230 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.230 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:27.230 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.230 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.230 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.15270.0:1536 139.9 139:93182763:test-rados-api-smithi047-82465-18::bar:head [stat,delete] snapc 0=[] ondisk+write+known_if_redirected e214)", 2024-02-23T00:51:27.230 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:44:46.557748+0000", 2024-02-23T00:51:27.230 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 400.595973383, 2024-02-23T00:51:27.230 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010162584000000001, 2024-02-23T00:51:27.230 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:27.230 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:51:27.230 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:27.230 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.15270", 2024-02-23T00:51:27.231 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/4133111990", 2024-02-23T00:51:27.231 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1536 2024-02-23T00:51:27.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.231 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:27.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:27.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:46.557748+0000", 2024-02-23T00:51:27.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:27.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:27.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:46.557748+0000", 2024-02-23T00:51:27.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999976 2024-02-23T00:51:27.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.232 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.232 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:27.232 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:46.557745+0000", 2024-02-23T00:51:27.232 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2969999999999999e-05 2024-02-23T00:51:27.232 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.232 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.232 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:27.232 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:46.557758+0000", 2024-02-23T00:51:27.232 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7716e-05 2024-02-23T00:51:27.232 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.232 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.232 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:27.232 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:46.557786+0000", 2024-02-23T00:51:27.232 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.065e-05 2024-02-23T00:51:27.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:27.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:46.557797+0000", 2024-02-23T00:51:27.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8850000000000007e-05 2024-02-23T00:51:27.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:27.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:46.557866+0000", 2024-02-23T00:51:27.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6620000000000005e-06 2024-02-23T00:51:27.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op must wait for map", 2024-02-23T00:51:27.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:46.557873+0000", 2024-02-23T00:51:27.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0052936110000000001 2024-02-23T00:51:27.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:27.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:46.563167+0000", 2024-02-23T00:51:27.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029901100000000001 2024-02-23T00:51:27.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for blocked object", 2024-02-23T00:51:27.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:46.563466+0000", 2024-02-23T00:51:27.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014262630000000001 2024-02-23T00:51:27.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:27.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:46.564892+0000", 2024-02-23T00:51:27.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000108467 2024-02-23T00:51:27.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:27.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:46.565001+0000", 2024-02-23T00:51:27.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0029096719999999999 2024-02-23T00:51:27.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:27.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:46.567910+0000", 2024-02-23T00:51:27.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010162584000000001 2024-02-23T00:51:27.235 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.235 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:27.235 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.236 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.236 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.40808.0:1 154.6s0 154:602f83fe:::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e236)", 2024-02-23T00:51:27.236 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:45:08.147420+0000", 2024-02-23T00:51:27.236 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 379.00630100900003, 2024-02-23T00:51:27.236 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0216719649999999, 2024-02-23T00:51:27.236 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:27.236 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:27.236 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:27.236 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.40808", 2024-02-23T00:51:27.236 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/2440014932", 2024-02-23T00:51:27.236 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1 2024-02-23T00:51:27.236 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.237 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:27.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.237 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:27.237 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:08.147420+0000", 2024-02-23T00:51:27.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:27.237 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.237 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:27.237 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:08.147420+0000", 2024-02-23T00:51:27.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999976 2024-02-23T00:51:27.237 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.237 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:27.237 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:08.147418+0000", 2024-02-23T00:51:27.238 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0395e-05 2024-02-23T00:51:27.238 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.238 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.238 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:27.238 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:08.147428+0000", 2024-02-23T00:51:27.238 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6918e-05 2024-02-23T00:51:27.238 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.238 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.238 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:27.238 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:08.147445+0000", 2024-02-23T00:51:27.238 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0013999999999998e-05 2024-02-23T00:51:27.238 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.238 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.238 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:27.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:08.147465+0000", 2024-02-23T00:51:27.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4432000000000003e-05 2024-02-23T00:51:27.239 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:27.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:08.147520+0000", 2024-02-23T00:51:27.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.473e-06 2024-02-23T00:51:27.239 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:51:27.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:08.147523+0000", 2024-02-23T00:51:27.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0146129070000001 2024-02-23T00:51:27.239 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:27.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.162136+0000", 2024-02-23T00:51:27.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000107526 2024-02-23T00:51:27.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:27.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.162243+0000", 2024-02-23T00:51:27.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00048819800000000002 2024-02-23T00:51:27.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:51:27.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.162732+0000", 2024-02-23T00:51:27.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00091639300000000005 2024-02-23T00:51:27.241 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:51:27.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.163648+0000", 2024-02-23T00:51:27.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0053861350000000002 2024-02-23T00:51:27.241 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:27.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.169034+0000", 2024-02-23T00:51:27.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.7728e-05 2024-02-23T00:51:27.241 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:27.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.169092+0000", 2024-02-23T00:51:27.242 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0216719649999999 2024-02-23T00:51:27.242 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.242 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:27.242 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.242 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.242 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.242 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.30483.0:2 155.7 155:e19d0d35:::oid_RmXattrPP_:head [append 0~128] snapc 0=[] ondisk+write+known_if_redirected e237)", 2024-02-23T00:51:27.242 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:45:09.145192+0000", 2024-02-23T00:51:27.242 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 378.00852886799998, 2024-02-23T00:51:27.242 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.023838841, 2024-02-23T00:51:27.242 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:27.242 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:27.242 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:27.242 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.30483", 2024-02-23T00:51:27.242 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/3044143263", 2024-02-23T00:51:27.243 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 2 2024-02-23T00:51:27.243 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.243 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:27.243 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.243 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:27.243 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.145192+0000", 2024-02-23T00:51:27.243 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:27.243 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.243 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.243 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:27.243 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.145192+0000", 2024-02-23T00:51:27.243 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999976 2024-02-23T00:51:27.243 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.243 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.243 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:27.244 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.145190+0000", 2024-02-23T00:51:27.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1411e-05 2024-02-23T00:51:27.244 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:27.244 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.145201+0000", 2024-02-23T00:51:27.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.2265999999999999e-05 2024-02-23T00:51:27.244 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:27.244 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.145233+0000", 2024-02-23T00:51:27.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0781000000000001e-05 2024-02-23T00:51:27.244 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.245 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:27.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.145244+0000", 2024-02-23T00:51:27.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9168e-05 2024-02-23T00:51:27.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.245 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.245 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:27.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.145303+0000", 2024-02-23T00:51:27.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5449999999999992e-06 2024-02-23T00:51:27.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.245 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.245 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:51:27.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.145313+0000", 2024-02-23T00:51:27.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.016702787 2024-02-23T00:51:27.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.246 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:27.246 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.162016+0000", 2024-02-23T00:51:27.246 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013284700000000001 2024-02-23T00:51:27.246 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.246 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:27.246 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.162148+0000", 2024-02-23T00:51:27.246 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017232599999999999 2024-02-23T00:51:27.246 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.246 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0,2", 2024-02-23T00:51:27.246 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.162321+0000", 2024-02-23T00:51:27.246 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00092248799999999995 2024-02-23T00:51:27.247 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.247 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.247 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:51:27.247 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.163243+0000", 2024-02-23T00:51:27.247 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00090715999999999995 2024-02-23T00:51:27.247 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.247 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.247 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:51:27.247 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.164150+0000", 2024-02-23T00:51:27.247 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00481891 2024-02-23T00:51:27.247 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.247 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.247 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:51:27.247 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.168969+0000", 2024-02-23T00:51:27.248 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4189000000000001e-05 2024-02-23T00:51:27.248 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.248 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.248 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:27.248 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.169003+0000", 2024-02-23T00:51:27.248 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7521999999999998e-05 2024-02-23T00:51:27.248 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.248 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.248 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:27.248 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:09.169031+0000", 2024-02-23T00:51:27.248 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.023838841 2024-02-23T00:51:27.248 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.248 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:27.248 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.249 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.249 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.249 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.30483.0:4 155.7 155:e19d0d35:::oid_RmXattrPP_:head [rmxattr attr1 in=5b] snapc 0=[] ondisk+write+known_if_redirected e237)", 2024-02-23T00:51:27.249 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:45:11.156695+0000", 2024-02-23T00:51:27.249 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 375.99702596899999, 2024-02-23T00:51:27.249 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010290539, 2024-02-23T00:51:27.249 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:27.249 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:27.249 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:27.249 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.30483", 2024-02-23T00:51:27.249 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/3044143263", 2024-02-23T00:51:27.249 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 4 2024-02-23T00:51:27.249 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.250 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:27.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.250 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:27.250 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:11.156695+0000", 2024-02-23T00:51:27.250 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:27.250 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.250 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:27.250 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:11.156695+0000", 2024-02-23T00:51:27.250 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:51:27.250 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.250 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:27.250 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:11.156694+0000", 2024-02-23T00:51:27.250 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.9450000000000004e-06 2024-02-23T00:51:27.251 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.251 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.251 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:27.251 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:11.156701+0000", 2024-02-23T00:51:27.251 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5896000000000002e-05 2024-02-23T00:51:27.251 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.251 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.251 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:27.251 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:11.156717+0000", 2024-02-23T00:51:27.251 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1049999999999995e-06 2024-02-23T00:51:27.251 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.251 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.251 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:27.252 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:11.156725+0000", 2024-02-23T00:51:27.252 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8067999999999999e-05 2024-02-23T00:51:27.252 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.252 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.252 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:27.252 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:11.156793+0000", 2024-02-23T00:51:27.252 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3289999999999997e-05 2024-02-23T00:51:27.252 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.252 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.252 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:27.252 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:11.156876+0000", 2024-02-23T00:51:27.252 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000113872 2024-02-23T00:51:27.252 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.252 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.253 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0,2", 2024-02-23T00:51:27.253 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:11.156990+0000", 2024-02-23T00:51:27.253 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0073364820000000001 2024-02-23T00:51:27.253 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.253 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.253 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:51:27.253 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:11.164326+0000", 2024-02-23T00:51:27.253 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.0790000000000002e-05 2024-02-23T00:51:27.253 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.253 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.253 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:51:27.253 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:11.164357+0000", 2024-02-23T00:51:27.253 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0025387470000000001 2024-02-23T00:51:27.253 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.254 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.254 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:51:27.254 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:11.166896+0000", 2024-02-23T00:51:27.254 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9968e-05 2024-02-23T00:51:27.254 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.254 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.254 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:27.254 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:11.166946+0000", 2024-02-23T00:51:27.254 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9716999999999999e-05 2024-02-23T00:51:27.254 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.254 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.254 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:27.254 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:11.166986+0000", 2024-02-23T00:51:27.255 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010290539 2024-02-23T00:51:27.255 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.255 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:27.255 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.255 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.255 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.255 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.40928.0:1 170.7s0 170:fd8eaea7:nspace::oid_SimpleWritePP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e266)", 2024-02-23T00:51:27.255 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:45:38.149323+0000", 2024-02-23T00:51:27.255 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 349.00439817199998, 2024-02-23T00:51:27.255 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0017361360000001, 2024-02-23T00:51:27.255 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:27.255 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:27.255 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:27.255 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.40928", 2024-02-23T00:51:27.256 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/3947607643", 2024-02-23T00:51:27.256 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1 2024-02-23T00:51:27.256 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.256 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:27.256 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.256 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:27.256 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:38.149323+0000", 2024-02-23T00:51:27.256 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:27.256 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.256 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.256 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:27.256 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:38.149323+0000", 2024-02-23T00:51:27.256 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999971 2024-02-23T00:51:27.256 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.257 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.257 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:27.257 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:38.149320+0000", 2024-02-23T00:51:27.257 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0796e-05 2024-02-23T00:51:27.257 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.257 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.257 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:27.257 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:38.149331+0000", 2024-02-23T00:51:27.257 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6829000000000001e-05 2024-02-23T00:51:27.257 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.257 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.257 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:27.257 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:38.149348+0000", 2024-02-23T00:51:27.257 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0499000000000001e-05 2024-02-23T00:51:27.258 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.258 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.258 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:27.258 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:38.149358+0000", 2024-02-23T00:51:27.258 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8974000000000002e-05 2024-02-23T00:51:27.258 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.258 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.258 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:27.258 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:38.149407+0000", 2024-02-23T00:51:27.258 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3039999999999998e-06 2024-02-23T00:51:27.258 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.258 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.258 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:51:27.259 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:38.149412+0000", 2024-02-23T00:51:27.259 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.99911529799999999 2024-02-23T00:51:27.259 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.259 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.259 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:27.259 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:39.148527+0000", 2024-02-23T00:51:27.259 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6186000000000004e-05 2024-02-23T00:51:27.259 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.259 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.259 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:27.259 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:39.148623+0000", 2024-02-23T00:51:27.259 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040401199999999998 2024-02-23T00:51:27.259 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.259 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.259 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:51:27.260 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:39.149027+0000", 2024-02-23T00:51:27.260 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00044522000000000002 2024-02-23T00:51:27.260 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.260 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.260 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:51:27.260 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:39.149472+0000", 2024-02-23T00:51:27.260 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001539931 2024-02-23T00:51:27.260 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.260 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.260 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:27.260 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:39.151012+0000", 2024-02-23T00:51:27.260 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6737000000000003e-05 2024-02-23T00:51:27.260 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.261 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.261 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:27.261 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:39.151059+0000", 2024-02-23T00:51:27.261 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0017361360000001 2024-02-23T00:51:27.261 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.261 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:27.261 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.261 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.261 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.261 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(osd.5.0:371 175.b e288/279 175:d3d11105:test-rados-api-smithi047-82465-34::foo:14 v 288'11, mlcod=285'8)", 2024-02-23T00:51:27.261 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:46:00.171325+0000", 2024-02-23T00:51:27.261 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 326.98239615099999, 2024-02-23T00:51:27.261 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0065955129999999999, 2024-02-23T00:51:27.261 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:27.261 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:27.262 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:27.262 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.262 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:27.262 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:00.171325+0000", 2024-02-23T00:51:27.262 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:27.262 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.262 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.262 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:27.262 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:00.171325+0000", 2024-02-23T00:51:27.262 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999995 2024-02-23T00:51:27.262 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.262 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.262 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:27.263 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:00.171324+0000", 2024-02-23T00:51:27.263 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.8379999999999998e-06 2024-02-23T00:51:27.263 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.263 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.263 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:27.263 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:00.171330+0000", 2024-02-23T00:51:27.263 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0871e-05 2024-02-23T00:51:27.263 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.263 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.263 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:27.263 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:00.171341+0000", 2024-02-23T00:51:27.263 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4309999999999997e-06 2024-02-23T00:51:27.263 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.263 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.264 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:27.264 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:00.171349+0000", 2024-02-23T00:51:27.264 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012786699999999999 2024-02-23T00:51:27.264 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.264 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.264 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:27.264 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:00.171476+0000", 2024-02-23T00:51:27.264 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4640000000000001e-05 2024-02-23T00:51:27.264 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.264 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.264 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:27.264 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:00.171501+0000", 2024-02-23T00:51:27.264 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0063964579999999998 2024-02-23T00:51:27.264 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.264 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.265 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:27.265 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:00.177897+0000", 2024-02-23T00:51:27.265 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2878e-05 2024-02-23T00:51:27.265 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.265 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.265 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:27.265 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:00.177920+0000", 2024-02-23T00:51:27.265 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0065955129999999999 2024-02-23T00:51:27.265 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.265 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:27.265 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.265 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.265 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.265 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.15270.0:5372 204.6 204:678263ef:test-rados-api-smithi047-82465-50::foo:head [read 0~1] snapc 0=[] ondisk+read+known_if_redirected e382)", 2024-02-23T00:51:27.266 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:47:34.181741+0000", 2024-02-23T00:51:27.266 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 232.971980288, 2024-02-23T00:51:27.266 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010581304999999999, 2024-02-23T00:51:27.266 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:27.266 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:51:27.266 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:27.266 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.15270", 2024-02-23T00:51:27.266 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/4133111990", 2024-02-23T00:51:27.266 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 5372 2024-02-23T00:51:27.266 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.266 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:27.266 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.266 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:27.266 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:34.181741+0000", 2024-02-23T00:51:27.266 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:27.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.267 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.267 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:27.267 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:34.181741+0000", 2024-02-23T00:51:27.267 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999976 2024-02-23T00:51:27.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.267 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.267 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:27.267 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:34.181738+0000", 2024-02-23T00:51:27.267 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2337000000000001e-05 2024-02-23T00:51:27.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.267 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.267 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:27.267 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:34.181750+0000", 2024-02-23T00:51:27.267 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4881e-05 2024-02-23T00:51:27.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.268 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.268 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:27.268 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:34.181765+0000", 2024-02-23T00:51:27.268 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.628e-06 2024-02-23T00:51:27.268 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.268 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.268 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:27.268 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:34.181775+0000", 2024-02-23T00:51:27.268 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3414000000000001e-05 2024-02-23T00:51:27.268 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.268 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.268 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:27.268 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:34.181818+0000", 2024-02-23T00:51:27.268 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00019916700000000001 2024-02-23T00:51:27.269 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.269 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.269 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for blocked object", 2024-02-23T00:51:27.269 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:34.182018+0000", 2024-02-23T00:51:27.269 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0070079749999999996 2024-02-23T00:51:27.269 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.269 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.269 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:27.269 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:34.189026+0000", 2024-02-23T00:51:27.269 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000105541 2024-02-23T00:51:27.269 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.269 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.269 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for rw locks", 2024-02-23T00:51:27.269 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:34.189131+0000", 2024-02-23T00:51:27.269 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0010998570000000001 2024-02-23T00:51:27.269 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.270 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.270 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:27.270 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:34.190231+0000", 2024-02-23T00:51:27.270 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000101282 2024-02-23T00:51:27.270 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.270 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.270 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:27.270 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:34.190332+0000", 2024-02-23T00:51:27.270 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0019897859999999999 2024-02-23T00:51:27.270 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:27.270 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:27.270 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:27.270 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:34.192322+0000", 2024-02-23T00:51:27.270 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010581304999999999 2024-02-23T00:51:27.270 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.270 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:27.271 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.271 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:27.271 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:27.271 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:51:27.271 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:51:27.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.166+0000 7f56569be700 10 client.41663.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:51:27.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.166+0000 7f56569be700 15 client.41663.objecter _session_command_op_assign -1 8 2024-02-23T00:51:27.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.166+0000 7f56569be700 20 client.41663.objecter _get_session s=0x7f56500fc780 osd=3 3 2024-02-23T00:51:27.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.166+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f56500fc780 osd=3 3 2024-02-23T00:51:27.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.166+0000 7f56569be700 20 client.41663.objecter _get_session s=0x7f56500fc780 osd=3 3 2024-02-23T00:51:27.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.166+0000 7f56569be700 15 client.41663.objecter _session_command_op_remove -1 8 2024-02-23T00:51:27.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.166+0000 7f56569be700 20 client.41663.objecter get_session s=0x7f56500fc780 osd=3 3 2024-02-23T00:51:27.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.166+0000 7f56569be700 15 client.41663.objecter _session_command_op_assign 3 8 2024-02-23T00:51:27.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.166+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f56500fc780 osd=3 4 2024-02-23T00:51:27.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.166+0000 7f56569be700 10 client.41663.objecter _send_command 8 2024-02-23T00:51:27.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.166+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f565011b4a0 con 0x7f565011bf90 2024-02-23T00:51:27.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.167+0000 7f564d7fa700 1 -- 172.21.15.47:0/3834819343 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f565011b4a0 con 0x7f565011bf90 2024-02-23T00:51:27.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.167+0000 7f564d7fa700 10 client.41663.objecter ms_dispatch 0x7f5650001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:51:27.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.167+0000 7f564d7fa700 10 client.41663.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:51:27.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.167+0000 7f564d7fa700 20 client.41663.objecter put_session s=0x7f56500fc780 osd=3 4 2024-02-23T00:51:27.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.167+0000 7f564d7fa700 15 client.41663.objecter _session_command_op_remove 3 8 2024-02-23T00:51:27.274 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:51:27.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.167+0000 7f56569be700 10 client.41663.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:27.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.167+0000 7f56569be700 15 client.41663.objecter _session_command_op_assign -1 9 2024-02-23T00:51:27.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.167+0000 7f56569be700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f56501232d0 0x7f56501256c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:27.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.167+0000 7f56569be700 20 client.41663.objecter _get_session s=0x7f565011b640 osd=4 3 2024-02-23T00:51:27.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.167+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f565011b640 osd=4 3 2024-02-23T00:51:27.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.167+0000 7f56569be700 20 client.41663.objecter _get_session s=0x7f565011b640 osd=4 3 2024-02-23T00:51:27.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.167+0000 7f56569be700 15 client.41663.objecter _session_command_op_remove -1 9 2024-02-23T00:51:27.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.167+0000 7f56569be700 20 client.41663.objecter get_session s=0x7f565011b640 osd=4 3 2024-02-23T00:51:27.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.167+0000 7f56569be700 15 client.41663.objecter _session_command_op_assign 4 9 2024-02-23T00:51:27.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.167+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f565011b640 osd=4 4 2024-02-23T00:51:27.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.167+0000 7f56569be700 10 client.41663.objecter _send_command 9 2024-02-23T00:51:27.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.167+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f565011b840 con 0x7f56501232d0 2024-02-23T00:51:27.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.168+0000 7f564ffff700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f56501232d0 0x7f56501256c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:27.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.168+0000 7f564d7fa700 10 client.41663.objecter ms_handle_connect 0x7f56501232d0 2024-02-23T00:51:27.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.168+0000 7f564ffff700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f56501232d0 0x7f56501256c0 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:27.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.169+0000 7f564d7fa700 1 -- 172.21.15.47:0/3834819343 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f565011b840 con 0x7f56501232d0 2024-02-23T00:51:27.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.169+0000 7f564d7fa700 10 client.41663.objecter ms_dispatch 0x7f5650001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:51:27.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.169+0000 7f564d7fa700 10 client.41663.objecter _finish_command 9 = system:0 2024-02-23T00:51:27.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.169+0000 7f564d7fa700 20 client.41663.objecter put_session s=0x7f565011b640 osd=4 4 2024-02-23T00:51:27.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.169+0000 7f564d7fa700 15 client.41663.objecter _session_command_op_remove 4 9 2024-02-23T00:51:27.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.183+0000 7f56569be700 10 client.41663.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:51:27.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.183+0000 7f56569be700 15 client.41663.objecter _session_command_op_assign -1 10 2024-02-23T00:51:27.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.183+0000 7f56569be700 20 client.41663.objecter _get_session s=0x7f565011b640 osd=4 3 2024-02-23T00:51:27.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.183+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f565011b640 osd=4 3 2024-02-23T00:51:27.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.183+0000 7f56569be700 20 client.41663.objecter _get_session s=0x7f565011b640 osd=4 3 2024-02-23T00:51:27.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.183+0000 7f56569be700 15 client.41663.objecter _session_command_op_remove -1 10 2024-02-23T00:51:27.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.183+0000 7f56569be700 20 client.41663.objecter get_session s=0x7f565011b640 osd=4 3 2024-02-23T00:51:27.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.183+0000 7f56569be700 15 client.41663.objecter _session_command_op_assign 4 10 2024-02-23T00:51:27.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.183+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f565011b640 osd=4 4 2024-02-23T00:51:27.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.183+0000 7f56569be700 10 client.41663.objecter _send_command 10 2024-02-23T00:51:27.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.183+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f565011b840 con 0x7f56501232d0 2024-02-23T00:51:27.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.185+0000 7f564d7fa700 1 -- 172.21.15.47:0/3834819343 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f565011b840 con 0x7f56501232d0 2024-02-23T00:51:27.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.185+0000 7f564d7fa700 10 client.41663.objecter ms_dispatch 0x7f5650001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:51:27.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.185+0000 7f564d7fa700 10 client.41663.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:51:27.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.185+0000 7f564d7fa700 20 client.41663.objecter put_session s=0x7f565011b640 osd=4 4 2024-02-23T00:51:27.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.185+0000 7f564d7fa700 15 client.41663.objecter _session_command_op_remove 4 10 2024-02-23T00:51:27.278 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:51:27.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.185+0000 7f56569be700 10 client.41663.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:27.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.185+0000 7f56569be700 15 client.41663.objecter _session_command_op_assign -1 11 2024-02-23T00:51:27.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.185+0000 7f56569be700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f56501260c0 0x7f5650128530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:27.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.185+0000 7f56569be700 20 client.41663.objecter _get_session s=0x7f565011b840 osd=5 3 2024-02-23T00:51:27.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.185+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f565011b840 osd=5 3 2024-02-23T00:51:27.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.185+0000 7f56569be700 20 client.41663.objecter _get_session s=0x7f565011b840 osd=5 3 2024-02-23T00:51:27.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.185+0000 7f56569be700 15 client.41663.objecter _session_command_op_remove -1 11 2024-02-23T00:51:27.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.185+0000 7f56569be700 20 client.41663.objecter get_session s=0x7f565011b840 osd=5 3 2024-02-23T00:51:27.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.185+0000 7f56569be700 15 client.41663.objecter _session_command_op_assign 5 11 2024-02-23T00:51:27.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.185+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f565011b840 osd=5 4 2024-02-23T00:51:27.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.185+0000 7f56569be700 10 client.41663.objecter _send_command 11 2024-02-23T00:51:27.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.185+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f565012b370 con 0x7f56501260c0 2024-02-23T00:51:27.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.186+0000 7f564f7fe700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f56501260c0 0x7f5650128530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:27.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.186+0000 7f564d7fa700 10 client.41663.objecter ms_handle_connect 0x7f56501260c0 2024-02-23T00:51:27.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.186+0000 7f564f7fe700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f56501260c0 0x7f5650128530 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:27.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.187+0000 7f564d7fa700 1 -- 172.21.15.47:0/3834819343 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f565012b370 con 0x7f56501260c0 2024-02-23T00:51:27.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.187+0000 7f564d7fa700 10 client.41663.objecter ms_dispatch 0x7f5650001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:51:27.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.187+0000 7f564d7fa700 10 client.41663.objecter _finish_command 11 = system:0 2024-02-23T00:51:27.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.187+0000 7f564d7fa700 20 client.41663.objecter put_session s=0x7f565011b840 osd=5 4 2024-02-23T00:51:27.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.187+0000 7f564d7fa700 15 client.41663.objecter _session_command_op_remove 5 11 2024-02-23T00:51:27.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.202+0000 7f56569be700 10 client.41663.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:51:27.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.202+0000 7f56569be700 15 client.41663.objecter _session_command_op_assign -1 12 2024-02-23T00:51:27.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.202+0000 7f56569be700 20 client.41663.objecter _get_session s=0x7f565011b840 osd=5 3 2024-02-23T00:51:27.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.202+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f565011b840 osd=5 3 2024-02-23T00:51:27.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.202+0000 7f56569be700 20 client.41663.objecter _get_session s=0x7f565011b840 osd=5 3 2024-02-23T00:51:27.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.202+0000 7f56569be700 15 client.41663.objecter _session_command_op_remove -1 12 2024-02-23T00:51:27.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.202+0000 7f56569be700 20 client.41663.objecter get_session s=0x7f565011b840 osd=5 3 2024-02-23T00:51:27.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.202+0000 7f56569be700 15 client.41663.objecter _session_command_op_assign 5 12 2024-02-23T00:51:27.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.202+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f565011b840 osd=5 4 2024-02-23T00:51:27.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.202+0000 7f56569be700 10 client.41663.objecter _send_command 12 2024-02-23T00:51:27.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.202+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f565012a140 con 0x7f56501260c0 2024-02-23T00:51:27.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.204+0000 7f564d7fa700 1 -- 172.21.15.47:0/3834819343 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f565012a140 con 0x7f56501260c0 2024-02-23T00:51:27.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.204+0000 7f564d7fa700 10 client.41663.objecter ms_dispatch 0x7f5650001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:51:27.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.204+0000 7f564d7fa700 10 client.41663.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:51:27.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.204+0000 7f564d7fa700 20 client.41663.objecter put_session s=0x7f565011b840 osd=5 4 2024-02-23T00:51:27.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.204+0000 7f564d7fa700 15 client.41663.objecter _session_command_op_remove 5 12 2024-02-23T00:51:27.282 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:51:27.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.204+0000 7f56569be700 10 client.41663.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:27.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.204+0000 7f56569be700 15 client.41663.objecter _session_command_op_assign -1 13 2024-02-23T00:51:27.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.204+0000 7f56569be700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f565012a810 0x7f565012ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:27.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.204+0000 7f56569be700 20 client.41663.objecter _get_session s=0x7f565012a140 osd=6 3 2024-02-23T00:51:27.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.204+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f565012a140 osd=6 3 2024-02-23T00:51:27.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.204+0000 7f56569be700 20 client.41663.objecter _get_session s=0x7f565012a140 osd=6 3 2024-02-23T00:51:27.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.204+0000 7f56569be700 15 client.41663.objecter _session_command_op_remove -1 13 2024-02-23T00:51:27.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.204+0000 7f56569be700 20 client.41663.objecter get_session s=0x7f565012a140 osd=6 3 2024-02-23T00:51:27.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.204+0000 7f56569be700 15 client.41663.objecter _session_command_op_assign 6 13 2024-02-23T00:51:27.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.204+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f565012a140 osd=6 4 2024-02-23T00:51:27.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.204+0000 7f56569be700 10 client.41663.objecter _send_command 13 2024-02-23T00:51:27.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.204+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f565012e590 con 0x7f565012a810 2024-02-23T00:51:27.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.204+0000 7f5654f5b700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f565012a810 0x7f565012ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:27.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.205+0000 7f5654f5b700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f565012a810 0x7f565012ac70 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:27.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.205+0000 7f564d7fa700 10 client.41663.objecter ms_handle_connect 0x7f565012a810 2024-02-23T00:51:27.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.206+0000 7f564d7fa700 1 -- 172.21.15.47:0/3834819343 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f565012e590 con 0x7f565012a810 2024-02-23T00:51:27.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.206+0000 7f564d7fa700 10 client.41663.objecter ms_dispatch 0x7f5650001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:51:27.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.206+0000 7f564d7fa700 10 client.41663.objecter _finish_command 13 = system:0 2024-02-23T00:51:27.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.206+0000 7f564d7fa700 20 client.41663.objecter put_session s=0x7f565012a140 osd=6 4 2024-02-23T00:51:27.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.206+0000 7f564d7fa700 15 client.41663.objecter _session_command_op_remove 6 13 2024-02-23T00:51:27.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.220+0000 7f56569be700 10 client.41663.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:51:27.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.220+0000 7f56569be700 15 client.41663.objecter _session_command_op_assign -1 14 2024-02-23T00:51:27.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.220+0000 7f56569be700 20 client.41663.objecter _get_session s=0x7f565012a140 osd=6 3 2024-02-23T00:51:27.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.220+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f565012a140 osd=6 3 2024-02-23T00:51:27.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.220+0000 7f56569be700 20 client.41663.objecter _get_session s=0x7f565012a140 osd=6 3 2024-02-23T00:51:27.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.220+0000 7f56569be700 15 client.41663.objecter _session_command_op_remove -1 14 2024-02-23T00:51:27.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.220+0000 7f56569be700 20 client.41663.objecter get_session s=0x7f565012a140 osd=6 3 2024-02-23T00:51:27.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.220+0000 7f56569be700 15 client.41663.objecter _session_command_op_assign 6 14 2024-02-23T00:51:27.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.220+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f565012a140 osd=6 4 2024-02-23T00:51:27.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.220+0000 7f56569be700 10 client.41663.objecter _send_command 14 2024-02-23T00:51:27.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.220+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f565012d380 con 0x7f565012a810 2024-02-23T00:51:27.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.222+0000 7f564d7fa700 1 -- 172.21.15.47:0/3834819343 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f565012d380 con 0x7f565012a810 2024-02-23T00:51:27.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.222+0000 7f564d7fa700 10 client.41663.objecter ms_dispatch 0x7f5650001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:51:27.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.222+0000 7f564d7fa700 10 client.41663.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:51:27.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.222+0000 7f564d7fa700 20 client.41663.objecter put_session s=0x7f565012a140 osd=6 4 2024-02-23T00:51:27.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.222+0000 7f564d7fa700 15 client.41663.objecter _session_command_op_remove 6 14 2024-02-23T00:51:27.286 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:51:27.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.222+0000 7f56569be700 10 client.41663.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:27.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.222+0000 7f56569be700 15 client.41663.objecter _session_command_op_assign -1 15 2024-02-23T00:51:27.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.222+0000 7f56569be700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f565012da50 0x7f565012deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:27.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.222+0000 7f56569be700 20 client.41663.objecter _get_session s=0x7f565012d380 osd=7 3 2024-02-23T00:51:27.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.222+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f565012d380 osd=7 3 2024-02-23T00:51:27.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.222+0000 7f56569be700 20 client.41663.objecter _get_session s=0x7f565012d380 osd=7 3 2024-02-23T00:51:27.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.222+0000 7f56569be700 15 client.41663.objecter _session_command_op_remove -1 15 2024-02-23T00:51:27.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.222+0000 7f56569be700 20 client.41663.objecter get_session s=0x7f565012d380 osd=7 3 2024-02-23T00:51:27.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.222+0000 7f56569be700 15 client.41663.objecter _session_command_op_assign 7 15 2024-02-23T00:51:27.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.222+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f565012d380 osd=7 4 2024-02-23T00:51:27.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.222+0000 7f56569be700 10 client.41663.objecter _send_command 15 2024-02-23T00:51:27.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.222+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56501317b0 con 0x7f565012da50 2024-02-23T00:51:27.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.222+0000 7f564ffff700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f565012da50 0x7f565012deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:27.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.223+0000 7f564d7fa700 10 client.41663.objecter ms_handle_connect 0x7f565012da50 2024-02-23T00:51:27.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.223+0000 7f564ffff700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f565012da50 0x7f565012deb0 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:27.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.223+0000 7f564d7fa700 1 -- 172.21.15.47:0/3834819343 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f56501317b0 con 0x7f565012da50 2024-02-23T00:51:27.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.223+0000 7f564d7fa700 10 client.41663.objecter ms_dispatch 0x7f5650001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:51:27.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.223+0000 7f564d7fa700 10 client.41663.objecter _finish_command 15 = system:0 2024-02-23T00:51:27.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.223+0000 7f564d7fa700 20 client.41663.objecter put_session s=0x7f565012d380 osd=7 4 2024-02-23T00:51:27.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.223+0000 7f564d7fa700 15 client.41663.objecter _session_command_op_remove 7 15 2024-02-23T00:51:27.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.238+0000 7f56569be700 10 client.41663.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:51:27.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.238+0000 7f56569be700 15 client.41663.objecter _session_command_op_assign -1 16 2024-02-23T00:51:27.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.238+0000 7f56569be700 20 client.41663.objecter _get_session s=0x7f565012d380 osd=7 3 2024-02-23T00:51:27.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.238+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f565012d380 osd=7 3 2024-02-23T00:51:27.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.238+0000 7f56569be700 20 client.41663.objecter _get_session s=0x7f565012d380 osd=7 3 2024-02-23T00:51:27.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.238+0000 7f56569be700 15 client.41663.objecter _session_command_op_remove -1 16 2024-02-23T00:51:27.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.238+0000 7f56569be700 20 client.41663.objecter get_session s=0x7f565012d380 osd=7 3 2024-02-23T00:51:27.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.238+0000 7f56569be700 15 client.41663.objecter _session_command_op_assign 7 16 2024-02-23T00:51:27.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.238+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f565012d380 osd=7 4 2024-02-23T00:51:27.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.238+0000 7f56569be700 10 client.41663.objecter _send_command 16 2024-02-23T00:51:27.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.238+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f56501305a0 con 0x7f565012da50 2024-02-23T00:51:27.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.239+0000 7f564d7fa700 1 -- 172.21.15.47:0/3834819343 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f56501305a0 con 0x7f565012da50 2024-02-23T00:51:27.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.239+0000 7f564d7fa700 10 client.41663.objecter ms_dispatch 0x7f5650001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:51:27.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.239+0000 7f564d7fa700 10 client.41663.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:51:27.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.239+0000 7f564d7fa700 20 client.41663.objecter put_session s=0x7f565012d380 osd=7 4 2024-02-23T00:51:27.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.239+0000 7f564d7fa700 15 client.41663.objecter _session_command_op_remove 7 16 2024-02-23T00:51:27.289 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:51:27.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 10 librados: watch_flush enter 2024-02-23T00:51:27.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 10 librados: watch_flush exit 2024-02-23T00:51:27.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 10 client.41663.objecter close_session for osd.0 2024-02-23T00:51:27.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f5650028540 msgr2=0x7f56500fbc40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:27.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f5650028540 0x7f56500fbc40 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:27.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f565005ee80 osd=0 1 2024-02-23T00:51:27.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 10 client.41663.objecter close_session for osd.1 2024-02-23T00:51:27.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f5650027b90 msgr2=0x7f5650027fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:27.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f5650027b90 0x7f5650027fd0 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:27.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f56500fd2b0 osd=1 1 2024-02-23T00:51:27.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 10 client.41663.objecter close_session for osd.2 2024-02-23T00:51:27.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f56500fce10 msgr2=0x7f565011e3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:27.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f56500fce10 0x7f565011e3f0 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:27.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f5650027a10 osd=2 1 2024-02-23T00:51:27.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 10 client.41663.objecter close_session for osd.3 2024-02-23T00:51:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f565011bf90 msgr2=0x7f565011fa60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f5654f5b700 1 -- 172.21.15.47:0/3834819343 reap_dead start 2024-02-23T00:51:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f565011bf90 0x7f565011fa60 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f56500fc780 osd=3 1 2024-02-23T00:51:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 10 client.41663.objecter close_session for osd.4 2024-02-23T00:51:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f56501232d0 msgr2=0x7f56501256c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f56501232d0 0x7f56501256c0 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f565011b640 osd=4 1 2024-02-23T00:51:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 10 client.41663.objecter close_session for osd.5 2024-02-23T00:51:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f56501260c0 msgr2=0x7f5650128530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f56501260c0 0x7f5650128530 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f565011b840 osd=5 1 2024-02-23T00:51:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 10 client.41663.objecter close_session for osd.6 2024-02-23T00:51:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f565012a810 msgr2=0x7f565012ac70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f565012a810 0x7f565012ac70 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:27.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f565012a140 osd=6 1 2024-02-23T00:51:27.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 10 client.41663.objecter close_session for osd.7 2024-02-23T00:51:27.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f565012da50 msgr2=0x7f565012deb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:27.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f565012da50 0x7f565012deb0 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:27.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 20 client.41663.objecter put_session s=0x7f565012d380 osd=7 1 2024-02-23T00:51:27.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 20 client.41663.objecter shutdown clearing up homeless session... 2024-02-23T00:51:27.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 10 client.41663.objecter successfully canceled tick 2024-02-23T00:51:27.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f5654f5b700 1 -- 172.21.15.47:0/3834819343 reap_dead start 2024-02-23T00:51:27.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5638062cc0 msgr2=0x7f5638065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:27.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5638062cc0 0x7f5638065180 secure :-1 s=READY pgs=3120 cs=0 l=1 rev1=1 rx=0x7f5644004ee0 tx=0x7f564400b040).stop 2024-02-23T00:51:27.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f56500f8cb0 msgr2=0x7f56500f6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:27.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f56500f8cb0 0x7f56500f6c60 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 rx=0x7f564000e970 tx=0x7f564000b040).stop 2024-02-23T00:51:27.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 shutdown_connections 2024-02-23T00:51:27.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f56500f8cb0 0x7f56500f6c60 unknown :-1 s=CLOSED pgs=2730 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:27.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 1 --2- 172.21.15.47:0/3834819343 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5638062cc0 0x7f5638065180 unknown :-1 s=CLOSED pgs=3120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:27.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 >> 172.21.15.47:0/3834819343 conn(0x7f56500f2820 msgr2=0x7f56500f3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:27.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 shutdown_connections 2024-02-23T00:51:27.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 1 -- 172.21.15.47:0/3834819343 wait complete. 2024-02-23T00:51:27.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:27.240+0000 7f56569be700 1 librados: shutdown 2024-02-23T00:51:27.294 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:51:27.294 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:51:27.294 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:51:27.294 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:51:27.294 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:51:27.294 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:51:27.294 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:51:27.294 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:51:27.300 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:27.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:26.305734+0000 mon.a (mon.0) 2982 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:51:27.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:27.094309+0000 mon.c (mon.2) 326 : audit [DBG] from='client.? 172.21.15.47:0/3834819343' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:51:27.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:27.245136+0000 mon.a (mon.0) 2983 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-23T00:51:27.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:27.245211+0000 mon.a (mon.0) 2984 : cluster [DBG] osdmap e558: 8 total, 8 up, 7 in 2024-02-23T00:51:27.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:27.248214+0000 mon.a (mon.0) 2985 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-92","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-23T00:51:27.448 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:51:27.451 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:27.451 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:51:27.552 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:51:27.590 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:26.305734+0000 mon.a (mon.0) 2982 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:51:27.590 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:27.094309+0000 mon.c (mon.2) 326 : audit [DBG] from='client.? 172.21.15.47:0/3834819343' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:51:27.590 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:27.245136+0000 mon.a (mon.0) 2983 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-23T00:51:27.590 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:27.245211+0000 mon.a (mon.0) 2984 : cluster [DBG] osdmap e558: 8 total, 8 up, 7 in 2024-02-23T00:51:27.591 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:27.248214+0000 mon.a (mon.0) 2985 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-92","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-23T00:51:27.591 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:26.305734+0000 mon.a (mon.0) 2982 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:51:27.591 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:27.094309+0000 mon.c (mon.2) 326 : audit [DBG] from='client.? 172.21.15.47:0/3834819343' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:51:27.591 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:27.245136+0000 mon.a (mon.0) 2983 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-23T00:51:27.591 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:27.245211+0000 mon.a (mon.0) 2984 : cluster [DBG] osdmap e558: 8 total, 8 up, 7 in 2024-02-23T00:51:27.591 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:27.248214+0000 mon.a (mon.0) 2985 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-92","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-23T00:51:27.592 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:27.900 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.5 dump_ops_in_flight 2024-02-23T00:51:27.920 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:51:27.923 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:27.923 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:51:28.024 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:51:28.051 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:28.070 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:51:28.072 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:28.072 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:51:28.171 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:51:28.173 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:51:28.201 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:28.449 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:51:28.451 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:27.342462+0000 mgr.y (mgr.14517) 507 : cluster [DBG] pgmap v841: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:28.451 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:28.246380+0000 mon.a (mon.0) 2986 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-92","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-23T00:51:28.451 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:28.246451+0000 mon.a (mon.0) 2987 : cluster [DBG] osdmap e559: 8 total, 8 up, 7 in 2024-02-23T00:51:28.451 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:28.268513+0000 mon.a (mon.0) 2988 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:51:28.451 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:28.268865+0000 mon.a (mon.0) 2989 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-92"}]: dispatch 2024-02-23T00:51:28.452 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:28.452 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:51:28.553 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:51:28.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:27.342462+0000 mgr.y (mgr.14517) 507 : cluster [DBG] pgmap v841: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:28.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:28.246380+0000 mon.a (mon.0) 2986 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-92","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-23T00:51:28.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:28.246451+0000 mon.a (mon.0) 2987 : cluster [DBG] osdmap e559: 8 total, 8 up, 7 in 2024-02-23T00:51:28.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:28.268513+0000 mon.a (mon.0) 2988 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:51:28.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:28.268865+0000 mon.a (mon.0) 2989 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-92"}]: dispatch 2024-02-23T00:51:28.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:27.342462+0000 mgr.y (mgr.14517) 507 : cluster [DBG] pgmap v841: 208 pgs: 208 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:28.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:28.246380+0000 mon.a (mon.0) 2986 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-92","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-23T00:51:28.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:28.246451+0000 mon.a (mon.0) 2987 : cluster [DBG] osdmap e559: 8 total, 8 up, 7 in 2024-02-23T00:51:28.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:28.268513+0000 mon.a (mon.0) 2988 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-6"}]: dispatch 2024-02-23T00:51:28.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:28.268865+0000 mon.a (mon.0) 2989 : audit [INF] from='client.? 172.21.15.47:0/4133111990' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-6", "tierpool": "test-rados-api-smithi047-82465-92"}]: dispatch 2024-02-23T00:51:28.584 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:28.933 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:51:28.937 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:28.937 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:51:29.038 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:51:29.067 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:29.190 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:51:29.194 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:29.194 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:51:29.295 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:51:29.329 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:29.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:27.379140+0000 osd.6 (osd.6) 217 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:51:29.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:27.379322+0000 osd.6 (osd.6) 218 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:51:29.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:29.247416+0000 mon.a (mon.0) 2990 : cluster [DBG] osdmap e560: 8 total, 8 up, 7 in 2024-02-23T00:51:29.486 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:27.379140+0000 osd.6 (osd.6) 217 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:51:29.486 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:27.379322+0000 osd.6 (osd.6) 218 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:51:29.487 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:29.247416+0000 mon.a (mon.0) 2990 : cluster [DBG] osdmap e560: 8 total, 8 up, 7 in 2024-02-23T00:51:29.487 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:27.379140+0000 osd.6 (osd.6) 217 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:51:29.487 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:27.379322+0000 osd.6 (osd.6) 218 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:51:29.487 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:29.247416+0000 mon.a (mon.0) 2990 : cluster [DBG] osdmap e560: 8 total, 8 up, 7 in 2024-02-23T00:51:29.487 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:51:29.490 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:29.490 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:51:29.524 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset noscrub 2024-02-23T00:51:29.591 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:51:29.621 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:29.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.707+0000 7f215e176700 1 -- 172.21.15.47:0/657736305 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21580f8420 msgr2=0x7f2158104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:29.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.707+0000 7f215e176700 1 --2- 172.21.15.47:0/657736305 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21580f8420 0x7f2158104a20 secure :-1 s=READY pgs=2731 cs=0 l=1 rev1=1 rx=0x7f2150009be0 tx=0x7f2150004d10).stop 2024-02-23T00:51:29.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.707+0000 7f215e176700 1 -- 172.21.15.47:0/657736305 shutdown_connections 2024-02-23T00:51:29.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.707+0000 7f215e176700 1 --2- 172.21.15.47:0/657736305 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21580f8420 0x7f2158104a20 unknown :-1 s=CLOSED pgs=2731 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:29.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.707+0000 7f215e176700 1 --2- 172.21.15.47:0/657736305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21580fdbb0 0x7f21580fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:29.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.707+0000 7f215e176700 1 --2- 172.21.15.47:0/657736305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21580fe510 0x7f21580f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:29.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.707+0000 7f215e176700 1 -- 172.21.15.47:0/657736305 >> 172.21.15.47:0/657736305 conn(0x7f21580f3800 msgr2=0x7f21580f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:29.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.707+0000 7f215e176700 1 -- 172.21.15.47:0/657736305 shutdown_connections 2024-02-23T00:51:29.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.707+0000 7f215e176700 1 -- 172.21.15.47:0/657736305 wait complete. 2024-02-23T00:51:29.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.708+0000 7f215e176700 1 librados: starting msgr at 2024-02-23T00:51:29.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.708+0000 7f215e176700 1 librados: starting objecter 2024-02-23T00:51:29.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.708+0000 7f215e176700 1 Processor -- start 2024-02-23T00:51:29.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.708+0000 7f215e176700 1 -- start start 2024-02-23T00:51:29.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.708+0000 7f215e176700 1 librados: setting wanted keys 2024-02-23T00:51:29.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.708+0000 7f215e176700 1 librados: calling monclient init 2024-02-23T00:51:29.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.708+0000 7f215e176700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21580f8420 0x7f215810dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:29.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.708+0000 7f215e176700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21580fdbb0 0x7f215810e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:29.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.708+0000 7f215e176700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21580fe510 0x7f2158107e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:29.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.708+0000 7f215e176700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2158108340 con 0x7f21580fe510 2024-02-23T00:51:29.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.708+0000 7f215e176700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21581084b0 con 0x7f21580fdbb0 2024-02-23T00:51:29.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.708+0000 7f215e176700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2158108670 con 0x7f21580f8420 2024-02-23T00:51:29.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.709+0000 7f21577fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21580f8420 0x7f215810dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:29.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.709+0000 7f21577fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21580f8420 0x7f215810dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:43026/0 (socket says 172.21.15.47:43026) 2024-02-23T00:51:29.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.709+0000 7f21577fe700 1 -- 172.21.15.47:0/2464634772 learned_addr learned my addr 172.21.15.47:0/2464634772 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:51:29.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.709+0000 7f2157fff700 1 --2- 172.21.15.47:0/2464634772 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21580fe510 0x7f2158107e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:29.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.709+0000 7f2156ffd700 1 --2- 172.21.15.47:0/2464634772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21580fdbb0 0x7f215810e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:29.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.709+0000 7f21577fe700 1 -- 172.21.15.47:0/2464634772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21580fdbb0 msgr2=0x7f215810e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:29.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.709+0000 7f21577fe700 1 --2- 172.21.15.47:0/2464634772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21580fdbb0 0x7f215810e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:29.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.709+0000 7f21577fe700 1 -- 172.21.15.47:0/2464634772 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21580fe510 msgr2=0x7f2158107e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:29.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.709+0000 7f21577fe700 1 --2- 172.21.15.47:0/2464634772 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21580fe510 0x7f2158107e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:29.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.709+0000 7f21577fe700 1 -- 172.21.15.47:0/2464634772 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f214c005040 con 0x7f21580f8420 2024-02-23T00:51:29.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.709+0000 7f2157fff700 1 --2- 172.21.15.47:0/2464634772 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21580fe510 0x7f2158107e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:51:29.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.709+0000 7f21577fe700 1 --2- 172.21.15.47:0/2464634772 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21580f8420 0x7f215810dc60 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 rx=0x7f21480041d0 tx=0x7f2148009f60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:29.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.709+0000 7f2154ff9700 10 client.?.objecter ms_handle_connect 0x7f21580f8420 2024-02-23T00:51:29.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.709+0000 7f2154ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:51:29.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.709+0000 7f2154ff9700 1 -- 172.21.15.47:0/2464634772 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2148007b40 con 0x7f21580f8420 2024-02-23T00:51:29.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.709+0000 7f2154ff9700 1 -- 172.21.15.47:0/2464634772 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f214800f040 con 0x7f21580f8420 2024-02-23T00:51:29.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.709+0000 7f2154ff9700 1 -- 172.21.15.47:0/2464634772 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2148014730 con 0x7f21580f8420 2024-02-23T00:51:29.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.709+0000 7f215e176700 1 -- 172.21.15.47:0/2464634772 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21500097c0 con 0x7f21580f8420 2024-02-23T00:51:29.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.709+0000 7f215e176700 10 client.41669.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:51:29.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.709+0000 7f215e176700 1 -- 172.21.15.47:0/2464634772 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2158108c60 con 0x7f21580f8420 2024-02-23T00:51:29.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.709+0000 7f215e176700 1 librados: init done 2024-02-23T00:51:29.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.711+0000 7f2154ff9700 1 -- 172.21.15.47:0/2464634772 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f2148009110 con 0x7f21580f8420 2024-02-23T00:51:29.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.711+0000 7f215e176700 1 -- 172.21.15.47:0/2464634772 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2158106ed0 con 0x7f21580f8420 2024-02-23T00:51:29.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.711+0000 7f2154ff9700 1 --2- 172.21.15.47:0/2464634772 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2140062cc0 0x7f2140065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:29.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.711+0000 7f2154ff9700 1 -- 172.21.15.47:0/2464634772 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(560..560 src has 463..560) v4 ==== 8835+0+0 (secure 0 0 0) 0x7f214807d500 con 0x7f21580f8420 2024-02-23T00:51:29.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.711+0000 7f2154ff9700 10 client.41669.objecter ms_dispatch 0x7f2158001550 osd_map(560..560 src has 463..560) v4 2024-02-23T00:51:29.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.711+0000 7f2154ff9700 3 client.41669.objecter handle_osd_map got epochs [560,560] > 0 2024-02-23T00:51:29.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.711+0000 7f2154ff9700 3 client.41669.objecter handle_osd_map decoding full epoch 560 2024-02-23T00:51:29.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.711+0000 7f2156ffd700 1 --2- 172.21.15.47:0/2464634772 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2140062cc0 0x7f2140065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:29.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.711+0000 7f2154ff9700 20 client.41669.objecter dump_active .. 0 homeless 2024-02-23T00:51:29.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.712+0000 7f2154ff9700 10 client.41669.objecter ms_handle_connect 0x7f2140062cc0 2024-02-23T00:51:29.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.712+0000 7f2156ffd700 1 --2- 172.21.15.47:0/2464634772 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2140062cc0 0x7f2140065180 secure :-1 s=READY pgs=3121 cs=0 l=1 rev1=1 rx=0x7f21581095a0 tx=0x7f214c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:29.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.716+0000 7f2154ff9700 1 -- 172.21.15.47:0/2464634772 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f214804cc90 con 0x7f21580f8420 2024-02-23T00:51:29.719 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:51:29.722 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:29.722 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:51:29.823 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:51:29.841 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:51:29.854 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:29.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.916+0000 7f215e176700 1 -- 172.21.15.47:0/2464634772 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f215805fba0 con 0x7f2140062cc0 2024-02-23T00:51:29.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.929+0000 7f2154ff9700 1 -- 172.21.15.47:0/2464634772 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+457155 (secure 0 0 0) 0x7f215805fba0 con 0x7f2140062cc0 2024-02-23T00:51:29.938 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:51:29.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.932+0000 7f215e176700 10 librados: watch_flush enter 2024-02-23T00:51:29.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.932+0000 7f215e176700 10 librados: watch_flush exit 2024-02-23T00:51:29.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.932+0000 7f215e176700 20 client.41669.objecter shutdown clearing up homeless session... 2024-02-23T00:51:29.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.932+0000 7f215e176700 10 client.41669.objecter successfully canceled tick 2024-02-23T00:51:29.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.932+0000 7f215e176700 1 -- 172.21.15.47:0/2464634772 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2140062cc0 msgr2=0x7f2140065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:29.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.932+0000 7f215e176700 1 --2- 172.21.15.47:0/2464634772 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2140062cc0 0x7f2140065180 secure :-1 s=READY pgs=3121 cs=0 l=1 rev1=1 rx=0x7f21581095a0 tx=0x7f214c00b040).stop 2024-02-23T00:51:29.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.933+0000 7f215e176700 1 -- 172.21.15.47:0/2464634772 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21580f8420 msgr2=0x7f215810dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:29.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.933+0000 7f215e176700 1 --2- 172.21.15.47:0/2464634772 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21580f8420 0x7f215810dc60 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 rx=0x7f21480041d0 tx=0x7f2148009f60).stop 2024-02-23T00:51:29.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.933+0000 7f215e176700 1 -- 172.21.15.47:0/2464634772 shutdown_connections 2024-02-23T00:51:29.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.933+0000 7f215e176700 1 --2- 172.21.15.47:0/2464634772 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21580fe510 0x7f2158107e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:29.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.933+0000 7f215e176700 1 --2- 172.21.15.47:0/2464634772 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2140062cc0 0x7f2140065180 unknown :-1 s=CLOSED pgs=3121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:29.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.933+0000 7f215e176700 1 --2- 172.21.15.47:0/2464634772 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21580f8420 0x7f215810dc60 unknown :-1 s=CLOSED pgs=2732 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:29.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.933+0000 7f215e176700 1 --2- 172.21.15.47:0/2464634772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21580fdbb0 0x7f215810e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:29.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.933+0000 7f215e176700 1 -- 172.21.15.47:0/2464634772 >> 172.21.15.47:0/2464634772 conn(0x7f21580f3800 msgr2=0x7f21580f50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:29.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.933+0000 7f215e176700 1 -- 172.21.15.47:0/2464634772 shutdown_connections 2024-02-23T00:51:29.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.933+0000 7f215e176700 1 -- 172.21.15.47:0/2464634772 wait complete. 2024-02-23T00:51:29.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:29.933+0000 7f215e176700 1 librados: shutdown 2024-02-23T00:51:29.943 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-02-23T00:51:30.105 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:51:30.109 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:30.109 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:51:30.210 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:51:30.240 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:30.343 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:51:30.345 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:30.345 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:51:30.379 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:29.342824+0000 mgr.y (mgr.14517) 508 : cluster [DBG] pgmap v844: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:30.379 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:30.256187+0000 mon.a (mon.0) 2991 : cluster [DBG] osdmap e561: 8 total, 8 up, 7 in 2024-02-23T00:51:30.379 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:30.287548+0000 mon.a (mon.0) 2992 : audit [INF] from='client.? 172.21.15.47:0/956675132' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi047-82465-94"}]: dispatch 2024-02-23T00:51:30.379 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:30.288716+0000 mon.a (mon.0) 2993 : audit [INF] from='client.? 172.21.15.47:0/956675132' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi047-82465-94"}]: dispatch 2024-02-23T00:51:30.380 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:30.289042+0000 mon.a (mon.0) 2994 : audit [INF] from='client.? 172.21.15.47:0/956675132' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi047-82465-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:51:30.446 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:51:30.476 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:30.495 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:51:30.498 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:30.498 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:51:30.600 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:51:30.606 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:51:30.606 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:51:30.628 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:30.647 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:51:30.649 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:30.650 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:51:30.678 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:29.342824+0000 mgr.y (mgr.14517) 508 : cluster [DBG] pgmap v844: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:30.678 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:30.256187+0000 mon.a (mon.0) 2991 : cluster [DBG] osdmap e561: 8 total, 8 up, 7 in 2024-02-23T00:51:30.678 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:30.287548+0000 mon.a (mon.0) 2992 : audit [INF] from='client.? 172.21.15.47:0/956675132' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi047-82465-94"}]: dispatch 2024-02-23T00:51:30.678 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:30.288716+0000 mon.a (mon.0) 2993 : audit [INF] from='client.? 172.21.15.47:0/956675132' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi047-82465-94"}]: dispatch 2024-02-23T00:51:30.679 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:30.289042+0000 mon.a (mon.0) 2994 : audit [INF] from='client.? 172.21.15.47:0/956675132' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi047-82465-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:51:30.679 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:51:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:51:30] "GET /metrics HTTP/1.1" 200 189781 "" "Prometheus/2.33.4" 2024-02-23T00:51:30.679 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:29.342824+0000 mgr.y (mgr.14517) 508 : cluster [DBG] pgmap v844: 176 pgs: 176 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:30.680 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:30.256187+0000 mon.a (mon.0) 2991 : cluster [DBG] osdmap e561: 8 total, 8 up, 7 in 2024-02-23T00:51:30.680 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:30.287548+0000 mon.a (mon.0) 2992 : audit [INF] from='client.? 172.21.15.47:0/956675132' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi047-82465-94"}]: dispatch 2024-02-23T00:51:30.680 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:30.288716+0000 mon.a (mon.0) 2993 : audit [INF] from='client.? 172.21.15.47:0/956675132' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi047-82465-94"}]: dispatch 2024-02-23T00:51:30.680 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:30.289042+0000 mon.a (mon.0) 2994 : audit [INF] from='client.? 172.21.15.47:0/956675132' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi047-82465-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:51:30.700 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":844,"stamp":"2024-02-23T00:51:29.342625+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4026236,"num_objects":418,"num_object_clones":11,"num_object_copies":1254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":418,"num_whiteouts":0,"num_read":2471,"num_read_kb":2035,"num_write":1692,"num_write_kb":4667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":440,"num_bytes_recovered":3479,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":21,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3133,"ondisk_log_size":3133,"up":528,"acting":528,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":591,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":2177368,"kb_used_data":31264,"kb_used_omap":0,"kb_used_meta":2146048,"kb_avail":654038696,"statfs":{"total":671965249536,"available":669735624704,"internally_reserved":0,"allocated":32014336,"data_stored":19243216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2197553152},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001110"},"pg_stats":[{"pgid":"62.0","version":"259'40","reported_seq":593,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.245717+0000","last_change":"2024-02-23T00:51:02.197067+0000","last_active":"2024-02-23T00:51:25.245717+0000","last_peered":"2024-02-23T00:51:25.245717+0000","last_clean":"2024-02-23T00:51:25.245717+0000","last_became_active":"2024-02-23T00:47:28.238034+0000","last_became_peered":"2024-02-23T00:47:28.238034+0000","last_unstale":"2024-02-23T00:51:25.245717+0000","last_undegraded":"2024-02-23T00:51:25.245717+0000","last_fullsized":"2024-02-23T00:51:25.245717+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'40","last_scrub_stamp":"2024-02-23T00:51:02.196968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:51:02.196968+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":31,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":1,"num_write":13,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":31,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.1","version":"495'89","reported_seq":587,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:27.116486+0000","last_change":"2024-02-23T00:50:56.478515+0000","last_active":"2024-02-23T00:51:27.116486+0000","last_peered":"2024-02-23T00:51:27.116486+0000","last_clean":"2024-02-23T00:51:27.116486+0000","last_became_active":"2024-02-23T00:43:17.851866+0000","last_became_peered":"2024-02-23T00:43:17.851866+0000","last_unstale":"2024-02-23T00:51:27.116486+0000","last_undegraded":"2024-02-23T00:51:27.116486+0000","last_fullsized":"2024-02-23T00:51:27.116486+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"495'89","last_scrub_stamp":"2024-02-23T00:50:56.478433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:56.478433+0000","log_size":89,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":63,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7,"num_read_kb":1,"num_write":30,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.2","version":"514'72","reported_seq":578,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.259145+0000","last_change":"2024-02-23T00:50:30.283941+0000","last_active":"2024-02-23T00:51:25.259145+0000","last_peered":"2024-02-23T00:51:25.259145+0000","last_clean":"2024-02-23T00:51:25.259145+0000","last_became_active":"2024-02-23T00:43:17.804792+0000","last_became_peered":"2024-02-23T00:43:17.804792+0000","last_unstale":"2024-02-23T00:51:25.259145+0000","last_undegraded":"2024-02-23T00:51:25.259145+0000","last_fullsized":"2024-02-23T00:51:25.259145+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'68","last_scrub_stamp":"2024-02-23T00:50:30.283820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:30.283820+0000","log_size":72,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1265,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":9,"num_read_kb":2,"num_write":27,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.3","version":"505'27","reported_seq":415,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:27.115383+0000","last_change":"2024-02-23T00:49:28.475170+0000","last_active":"2024-02-23T00:51:27.115383+0000","last_peered":"2024-02-23T00:51:27.115383+0000","last_clean":"2024-02-23T00:51:27.115383+0000","last_became_active":"2024-02-23T00:47:28.226936+0000","last_became_peered":"2024-02-23T00:47:28.226936+0000","last_unstale":"2024-02-23T00:51:27.115383+0000","last_undegraded":"2024-02-23T00:51:27.115383+0000","last_fullsized":"2024-02-23T00:51:27.115383+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"246'20","last_scrub_stamp":"2024-02-23T00:49:28.475056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:49:28.475056+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":4,"num_write":16,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":24,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.4","version":"514'24","reported_seq":531,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.244662+0000","last_change":"2024-02-23T00:48:56.703323+0000","last_active":"2024-02-23T00:51:25.244662+0000","last_peered":"2024-02-23T00:51:25.244662+0000","last_clean":"2024-02-23T00:51:25.244662+0000","last_became_active":"2024-02-23T00:43:17.810343+0000","last_became_peered":"2024-02-23T00:43:17.810343+0000","last_unstale":"2024-02-23T00:51:25.244662+0000","last_undegraded":"2024-02-23T00:51:25.244662+0000","last_fullsized":"2024-02-23T00:51:25.244662+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"207'17","last_scrub_stamp":"2024-02-23T00:48:56.703243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:48:56.703243+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":14,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.5","version":"495'55","reported_seq":560,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.241338+0000","last_change":"2024-02-23T00:49:26.431068+0000","last_active":"2024-02-23T00:51:25.241338+0000","last_peered":"2024-02-23T00:51:25.241338+0000","last_clean":"2024-02-23T00:51:25.241338+0000","last_became_active":"2024-02-23T00:43:17.855339+0000","last_became_peered":"2024-02-23T00:43:17.855339+0000","last_unstale":"2024-02-23T00:51:25.241338+0000","last_undegraded":"2024-02-23T00:51:25.241338+0000","last_fullsized":"2024-02-23T00:51:25.241338+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'51","last_scrub_stamp":"2024-02-23T00:49:26.430990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:49:26.430990+0000","log_size":55,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":2,"num_write":21,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.6","version":"514'55","reported_seq":618,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.248402+0000","last_change":"2024-02-23T00:50:21.294652+0000","last_active":"2024-02-23T00:51:25.248402+0000","last_peered":"2024-02-23T00:51:25.248402+0000","last_clean":"2024-02-23T00:51:25.248402+0000","last_became_active":"2024-02-23T00:47:28.236471+0000","last_became_peered":"2024-02-23T00:47:28.236471+0000","last_unstale":"2024-02-23T00:51:25.248402+0000","last_undegraded":"2024-02-23T00:51:25.248402+0000","last_fullsized":"2024-02-23T00:51:25.248402+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"379'54","last_scrub_stamp":"2024-02-23T00:50:21.294457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:21.294457+0000","log_size":55,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1272,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":7,"num_read_kb":1,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":41,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.7","version":"514'77","reported_seq":601,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.254768+0000","last_change":"2024-02-23T00:50:28.305136+0000","last_active":"2024-02-23T00:51:25.254768+0000","last_peered":"2024-02-23T00:51:25.254768+0000","last_clean":"2024-02-23T00:51:25.254768+0000","last_became_active":"2024-02-23T00:47:28.240920+0000","last_became_peered":"2024-02-23T00:47:28.240920+0000","last_unstale":"2024-02-23T00:51:25.254768+0000","last_undegraded":"2024-02-23T00:51:25.254768+0000","last_fullsized":"2024-02-23T00:51:25.254768+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"499'76","last_scrub_stamp":"2024-02-23T00:50:28.305026+0000","last_deep_scrub":"499'76","last_deep_scrub_stamp":"2024-02-23T00:50:28.305026+0000","last_clean_scrub_stamp":"2024-02-23T00:50:28.305026+0000","log_size":77,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":573,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":26,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":52,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.8","version":"514'134","reported_seq":697,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.245704+0000","last_change":"2024-02-23T00:48:53.698549+0000","last_active":"2024-02-23T00:51:25.245704+0000","last_peered":"2024-02-23T00:51:25.245704+0000","last_clean":"2024-02-23T00:51:25.245704+0000","last_became_active":"2024-02-23T00:47:28.229780+0000","last_became_peered":"2024-02-23T00:47:28.229780+0000","last_unstale":"2024-02-23T00:51:25.245704+0000","last_undegraded":"2024-02-23T00:51:25.245704+0000","last_fullsized":"2024-02-23T00:51:25.245704+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"440'114","last_scrub_stamp":"2024-02-23T00:48:53.698466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:48:53.698466+0000","log_size":134,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":816,"num_objects":14,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":9,"num_read_kb":3,"num_write":51,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":68,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":4,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.9","version":"529'69","reported_seq":616,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.241263+0000","last_change":"2024-02-23T00:49:00.243151+0000","last_active":"2024-02-23T00:51:25.241263+0000","last_peered":"2024-02-23T00:51:25.241263+0000","last_clean":"2024-02-23T00:51:25.241263+0000","last_became_active":"2024-02-23T00:47:28.202025+0000","last_became_peered":"2024-02-23T00:47:28.202025+0000","last_unstale":"2024-02-23T00:51:25.241263+0000","last_undegraded":"2024-02-23T00:51:25.241263+0000","last_fullsized":"2024-02-23T00:51:25.241263+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'39","last_scrub_stamp":"2024-02-23T00:49:00.243068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:49:00.243068+0000","log_size":69,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1285,"num_objects":10,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":29,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":27,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":6,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.a","version":"505'92","reported_seq":662,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.246273+0000","last_change":"2024-02-23T00:50:30.147956+0000","last_active":"2024-02-23T00:51:25.246273+0000","last_peered":"2024-02-23T00:51:25.246273+0000","last_clean":"2024-02-23T00:51:25.246273+0000","last_became_active":"2024-02-23T00:47:28.241187+0000","last_became_peered":"2024-02-23T00:47:28.241187+0000","last_unstale":"2024-02-23T00:51:25.246273+0000","last_undegraded":"2024-02-23T00:51:25.246273+0000","last_fullsized":"2024-02-23T00:51:25.246273+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'91","last_scrub_stamp":"2024-02-23T00:50:30.147873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:30.147873+0000","log_size":92,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1283,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":30,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":68,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.b","version":"514'91","reported_seq":595,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.247493+0000","last_change":"2024-02-23T00:50:59.132747+0000","last_active":"2024-02-23T00:51:25.247493+0000","last_peered":"2024-02-23T00:51:25.247493+0000","last_clean":"2024-02-23T00:51:25.247493+0000","last_became_active":"2024-02-23T00:43:17.855321+0000","last_became_peered":"2024-02-23T00:43:17.855321+0000","last_unstale":"2024-02-23T00:51:25.247493+0000","last_undegraded":"2024-02-23T00:51:25.247493+0000","last_fullsized":"2024-02-23T00:51:25.247493+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"514'91","last_scrub_stamp":"2024-02-23T00:50:59.132670+0000","last_deep_scrub":"418'84","last_deep_scrub_stamp":"2024-02-23T00:48:10.926499+0000","last_clean_scrub_stamp":"2024-02-23T00:50:59.132670+0000","log_size":91,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":621,"num_objects":8,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":13,"num_read_kb":6,"num_write":35,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.c","version":"495'43","reported_seq":555,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:26.242806+0000","last_change":"2024-02-23T00:49:06.040399+0000","last_active":"2024-02-23T00:51:26.242806+0000","last_peered":"2024-02-23T00:51:26.242806+0000","last_clean":"2024-02-23T00:51:26.242806+0000","last_became_active":"2024-02-23T00:43:17.829044+0000","last_became_peered":"2024-02-23T00:43:17.829044+0000","last_unstale":"2024-02-23T00:51:26.242806+0000","last_undegraded":"2024-02-23T00:51:26.242806+0000","last_fullsized":"2024-02-23T00:51:26.242806+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"447'39","last_scrub_stamp":"2024-02-23T00:49:06.040326+0000","last_deep_scrub":"447'39","last_deep_scrub_stamp":"2024-02-23T00:49:06.040326+0000","last_clean_scrub_stamp":"2024-02-23T00:49:06.040326+0000","log_size":43,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":3,"num_write":18,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.d","version":"414'58","reported_seq":605,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.241241+0000","last_change":"2024-02-23T00:50:28.632286+0000","last_active":"2024-02-23T00:51:25.241241+0000","last_peered":"2024-02-23T00:51:25.241241+0000","last_clean":"2024-02-23T00:51:25.241241+0000","last_became_active":"2024-02-23T00:47:28.202691+0000","last_became_peered":"2024-02-23T00:47:28.202691+0000","last_unstale":"2024-02-23T00:51:25.241241+0000","last_undegraded":"2024-02-23T00:51:25.241241+0000","last_fullsized":"2024-02-23T00:51:25.241241+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"414'58","last_scrub_stamp":"2024-02-23T00:50:28.632146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:28.632146+0000","log_size":58,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":18,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":33,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.e","version":"514'60","reported_seq":566,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.245032+0000","last_change":"2024-02-23T00:50:31.600892+0000","last_active":"2024-02-23T00:51:25.245032+0000","last_peered":"2024-02-23T00:51:25.245032+0000","last_clean":"2024-02-23T00:51:25.245032+0000","last_became_active":"2024-02-23T00:43:17.845956+0000","last_became_peered":"2024-02-23T00:43:17.845956+0000","last_unstale":"2024-02-23T00:51:25.245032+0000","last_undegraded":"2024-02-23T00:51:25.245032+0000","last_fullsized":"2024-02-23T00:51:25.245032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"495'58","last_scrub_stamp":"2024-02-23T00:50:31.600819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:31.600819+0000","log_size":60,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":43,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":24,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.f","version":"496'138","reported_seq":820,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.244582+0000","last_change":"2024-02-23T00:49:03.264065+0000","last_active":"2024-02-23T00:51:25.244582+0000","last_peered":"2024-02-23T00:51:25.244582+0000","last_clean":"2024-02-23T00:51:25.244582+0000","last_became_active":"2024-02-23T00:47:47.183032+0000","last_became_peered":"2024-02-23T00:47:47.183032+0000","last_unstale":"2024-02-23T00:51:25.244582+0000","last_undegraded":"2024-02-23T00:51:25.244582+0000","last_fullsized":"2024-02-23T00:51:25.244582+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"241'136","last_scrub_stamp":"2024-02-23T00:49:03.263943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:49:03.263943+0000","log_size":138,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":88,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":8,"num_read_kb":1,"num_write":43,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.10","version":"514'76","reported_seq":574,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:27.114502+0000","last_change":"2024-02-23T00:50:27.628865+0000","last_active":"2024-02-23T00:51:27.114502+0000","last_peered":"2024-02-23T00:51:27.114502+0000","last_clean":"2024-02-23T00:51:27.114502+0000","last_became_active":"2024-02-23T00:43:17.819865+0000","last_became_peered":"2024-02-23T00:43:17.819865+0000","last_unstale":"2024-02-23T00:51:27.114502+0000","last_undegraded":"2024-02-23T00:51:27.114502+0000","last_fullsized":"2024-02-23T00:51:27.114502+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"496'75","last_scrub_stamp":"2024-02-23T00:50:27.628796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:27.628796+0000","log_size":76,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1389,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":3,"num_write":32,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.11","version":"347'53","reported_seq":548,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:27.116400+0000","last_change":"2024-02-23T00:50:22.723895+0000","last_active":"2024-02-23T00:51:27.116400+0000","last_peered":"2024-02-23T00:51:27.116400+0000","last_clean":"2024-02-23T00:51:27.116400+0000","last_became_active":"2024-02-23T00:43:17.852697+0000","last_became_peered":"2024-02-23T00:43:17.852697+0000","last_unstale":"2024-02-23T00:51:27.116400+0000","last_undegraded":"2024-02-23T00:51:27.116400+0000","last_fullsized":"2024-02-23T00:51:27.116400+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'53","last_scrub_stamp":"2024-02-23T00:50:22.723817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:22.723817+0000","log_size":53,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":49,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":17,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.12","version":"240'73","reported_seq":482,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.244565+0000","last_change":"2024-02-23T00:51:10.508444+0000","last_active":"2024-02-23T00:51:25.244565+0000","last_peered":"2024-02-23T00:51:25.244565+0000","last_clean":"2024-02-23T00:51:25.244565+0000","last_became_active":"2024-02-23T00:47:28.213692+0000","last_became_peered":"2024-02-23T00:47:28.213692+0000","last_unstale":"2024-02-23T00:51:25.244565+0000","last_undegraded":"2024-02-23T00:51:25.244565+0000","last_fullsized":"2024-02-23T00:51:25.244565+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'73","last_scrub_stamp":"2024-02-23T00:51:10.508378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:51:10.508378+0000","log_size":73,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":49,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":49,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.13","version":"540'68","reported_seq":585,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.244621+0000","last_change":"2024-02-23T00:51:09.477358+0000","last_active":"2024-02-23T00:51:25.244621+0000","last_peered":"2024-02-23T00:51:25.244621+0000","last_clean":"2024-02-23T00:51:25.244621+0000","last_became_active":"2024-02-23T00:43:17.857823+0000","last_became_peered":"2024-02-23T00:43:17.857823+0000","last_unstale":"2024-02-23T00:51:25.244621+0000","last_undegraded":"2024-02-23T00:51:25.244621+0000","last_fullsized":"2024-02-23T00:51:25.244621+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"540'68","last_scrub_stamp":"2024-02-23T00:51:09.477292+0000","last_deep_scrub":"540'68","last_deep_scrub_stamp":"2024-02-23T00:51:09.477292+0000","last_clean_scrub_stamp":"2024-02-23T00:51:09.477292+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":45,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":8,"num_read_kb":4,"num_write":24,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.14","version":"514'62","reported_seq":619,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.246345+0000","last_change":"2024-02-23T00:50:34.190105+0000","last_active":"2024-02-23T00:51:25.246345+0000","last_peered":"2024-02-23T00:51:25.246345+0000","last_clean":"2024-02-23T00:51:25.246345+0000","last_became_active":"2024-02-23T00:47:28.235465+0000","last_became_peered":"2024-02-23T00:47:28.235465+0000","last_unstale":"2024-02-23T00:51:25.246345+0000","last_undegraded":"2024-02-23T00:51:25.246345+0000","last_fullsized":"2024-02-23T00:51:25.246345+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"496'61","last_scrub_stamp":"2024-02-23T00:50:34.190011+0000","last_deep_scrub":"420'54","last_deep_scrub_stamp":"2024-02-23T00:48:12.892464+0000","last_clean_scrub_stamp":"2024-02-23T00:50:34.190011+0000","log_size":62,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2369,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":13,"num_read_kb":5,"num_write":31,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":51,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.15","version":"505'22","reported_seq":558,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:26.242844+0000","last_change":"2024-02-23T00:49:33.948127+0000","last_active":"2024-02-23T00:51:26.242844+0000","last_peered":"2024-02-23T00:51:26.242844+0000","last_clean":"2024-02-23T00:51:26.242844+0000","last_became_active":"2024-02-23T00:47:28.219548+0000","last_became_peered":"2024-02-23T00:47:28.219548+0000","last_unstale":"2024-02-23T00:51:26.242844+0000","last_undegraded":"2024-02-23T00:51:26.242844+0000","last_fullsized":"2024-02-23T00:51:26.242844+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'19","last_scrub_stamp":"2024-02-23T00:49:33.948049+0000","last_deep_scrub":"299'18","last_deep_scrub_stamp":"2024-02-23T00:46:11.830580+0000","last_clean_scrub_stamp":"2024-02-23T00:49:33.948049+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3465,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":12,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.16","version":"407'30","reported_seq":580,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.243530+0000","last_change":"2024-02-23T00:51:07.565330+0000","last_active":"2024-02-23T00:51:25.243530+0000","last_peered":"2024-02-23T00:51:25.243530+0000","last_clean":"2024-02-23T00:51:25.243530+0000","last_became_active":"2024-02-23T00:47:28.199870+0000","last_became_peered":"2024-02-23T00:47:28.199870+0000","last_unstale":"2024-02-23T00:51:25.243530+0000","last_undegraded":"2024-02-23T00:51:25.243530+0000","last_fullsized":"2024-02-23T00:51:25.243530+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"407'30","last_scrub_stamp":"2024-02-23T00:51:07.565245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:51:07.565245+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":64,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":14,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.17","version":"375'63","reported_seq":577,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.254935+0000","last_change":"2024-02-23T00:49:42.321041+0000","last_active":"2024-02-23T00:51:25.254935+0000","last_peered":"2024-02-23T00:51:25.254935+0000","last_clean":"2024-02-23T00:51:25.254935+0000","last_became_active":"2024-02-23T00:43:17.843525+0000","last_became_peered":"2024-02-23T00:43:17.843525+0000","last_unstale":"2024-02-23T00:51:25.254935+0000","last_undegraded":"2024-02-23T00:51:25.254935+0000","last_fullsized":"2024-02-23T00:51:25.254935+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'63","last_scrub_stamp":"2024-02-23T00:49:42.320947+0000","last_deep_scrub":"375'63","last_deep_scrub_stamp":"2024-02-23T00:47:27.229920+0000","last_clean_scrub_stamp":"2024-02-23T00:49:42.320947+0000","log_size":63,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":20,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.19","version":"494'98","reported_seq":678,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.253211+0000","last_change":"2024-02-23T00:50:20.275389+0000","last_active":"2024-02-23T00:51:25.253211+0000","last_peered":"2024-02-23T00:51:25.253211+0000","last_clean":"2024-02-23T00:51:25.253211+0000","last_became_active":"2024-02-23T00:47:28.210660+0000","last_became_peered":"2024-02-23T00:47:28.210660+0000","last_unstale":"2024-02-23T00:51:25.253211+0000","last_undegraded":"2024-02-23T00:51:25.253211+0000","last_fullsized":"2024-02-23T00:51:25.253211+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'96","last_scrub_stamp":"2024-02-23T00:50:20.275315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:20.275315+0000","log_size":98,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":73,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":9,"num_read_kb":1,"num_write":33,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":73,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,7],"acting":[6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.18","version":"545'18","reported_seq":558,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:26.242186+0000","last_change":"2024-02-23T00:51:14.315460+0000","last_active":"2024-02-23T00:51:26.242186+0000","last_peered":"2024-02-23T00:51:26.242186+0000","last_clean":"2024-02-23T00:51:26.242186+0000","last_became_active":"2024-02-23T00:47:28.221906+0000","last_became_peered":"2024-02-23T00:47:28.221906+0000","last_unstale":"2024-02-23T00:51:26.242186+0000","last_undegraded":"2024-02-23T00:51:26.242186+0000","last_fullsized":"2024-02-23T00:51:26.242186+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"545'18","last_scrub_stamp":"2024-02-23T00:51:14.315402+0000","last_deep_scrub":"545'18","last_deep_scrub_stamp":"2024-02-23T00:51:14.315402+0000","last_clean_scrub_stamp":"2024-02-23T00:51:14.315402+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":15,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":1,"num_write":11,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":5,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1b","version":"347'70","reported_seq":568,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.241813+0000","last_change":"2024-02-23T00:49:28.481771+0000","last_active":"2024-02-23T00:51:25.241813+0000","last_peered":"2024-02-23T00:51:25.241813+0000","last_clean":"2024-02-23T00:51:25.241813+0000","last_became_active":"2024-02-23T00:43:17.803043+0000","last_became_peered":"2024-02-23T00:43:17.803043+0000","last_unstale":"2024-02-23T00:51:25.241813+0000","last_undegraded":"2024-02-23T00:51:25.241813+0000","last_fullsized":"2024-02-23T00:51:25.241813+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'70","last_scrub_stamp":"2024-02-23T00:49:28.481700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:49:28.481700+0000","log_size":70,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4000060,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":22,"num_write_kb":3928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.1a","version":"347'35","reported_seq":574,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.259056+0000","last_change":"2024-02-23T00:49:46.307053+0000","last_active":"2024-02-23T00:51:25.259056+0000","last_peered":"2024-02-23T00:51:25.259056+0000","last_clean":"2024-02-23T00:51:25.259056+0000","last_became_active":"2024-02-23T00:45:44.178195+0000","last_became_peered":"2024-02-23T00:45:44.178195+0000","last_unstale":"2024-02-23T00:51:25.259056+0000","last_undegraded":"2024-02-23T00:51:25.259056+0000","last_fullsized":"2024-02-23T00:51:25.259056+0000","mapping_epoch":271,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":272,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'35","last_scrub_stamp":"2024-02-23T00:49:46.306975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.306975+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":30,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":11,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1d","version":"539'107","reported_seq":612,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.245416+0000","last_change":"2024-02-23T00:50:58.794356+0000","last_active":"2024-02-23T00:51:25.245416+0000","last_peered":"2024-02-23T00:51:25.245416+0000","last_clean":"2024-02-23T00:51:25.245416+0000","last_became_active":"2024-02-23T00:43:17.828194+0000","last_became_peered":"2024-02-23T00:43:17.828194+0000","last_unstale":"2024-02-23T00:51:25.245416+0000","last_undegraded":"2024-02-23T00:51:25.245416+0000","last_fullsized":"2024-02-23T00:51:25.245416+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"505'105","last_scrub_stamp":"2024-02-23T00:50:58.794271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:58.794271+0000","log_size":107,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2384,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":8,"num_read_kb":0,"num_write":35,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"62.1c","version":"514'70","reported_seq":578,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.254705+0000","last_change":"2024-02-23T00:49:45.261438+0000","last_active":"2024-02-23T00:51:25.254705+0000","last_peered":"2024-02-23T00:51:25.254705+0000","last_clean":"2024-02-23T00:51:25.254705+0000","last_became_active":"2024-02-23T00:43:17.804718+0000","last_became_peered":"2024-02-23T00:43:17.804718+0000","last_unstale":"2024-02-23T00:51:25.254705+0000","last_undegraded":"2024-02-23T00:51:25.254705+0000","last_fullsized":"2024-02-23T00:51:25.254705+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"446'61","last_scrub_stamp":"2024-02-23T00:49:45.261352+0000","last_deep_scrub":"374'55","last_deep_scrub_stamp":"2024-02-23T00:47:26.267291+0000","last_clean_scrub_stamp":"2024-02-23T00:49:45.261352+0000","log_size":70,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1594,"num_objects":7,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":15,"num_read_kb":8,"num_write":31,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1f","version":"551'57","reported_seq":584,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.247573+0000","last_change":"2024-02-23T00:50:58.157230+0000","last_active":"2024-02-23T00:51:25.247573+0000","last_peered":"2024-02-23T00:51:25.247573+0000","last_clean":"2024-02-23T00:51:25.247573+0000","last_became_active":"2024-02-23T00:47:28.243574+0000","last_became_peered":"2024-02-23T00:47:28.243574+0000","last_unstale":"2024-02-23T00:51:25.247573+0000","last_undegraded":"2024-02-23T00:51:25.247573+0000","last_fullsized":"2024-02-23T00:51:25.247573+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"464'55","last_scrub_stamp":"2024-02-23T00:50:58.157140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:50:58.157140+0000","log_size":57,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":73,"num_objects":10,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":35,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":4,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.1e","version":"505'62","reported_seq":611,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.248312+0000","last_change":"2024-02-23T00:51:07.232814+0000","last_active":"2024-02-23T00:51:25.248312+0000","last_peered":"2024-02-23T00:51:25.248312+0000","last_clean":"2024-02-23T00:51:25.248312+0000","last_became_active":"2024-02-23T00:47:28.244073+0000","last_became_peered":"2024-02-23T00:47:28.244073+0000","last_unstale":"2024-02-23T00:51:25.248312+0000","last_undegraded":"2024-02-23T00:51:25.248312+0000","last_fullsized":"2024-02-23T00:51:25.248312+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"505'62","last_scrub_stamp":"2024-02-23T00:51:07.232724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:16.533032+0000","last_clean_scrub_stamp":"2024-02-23T00:51:07.232724+0000","log_size":62,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1248,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":11,"num_read_kb":4,"num_write":29,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":33,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"466'9","reported_seq":682,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.248323+0000","last_change":"2024-02-23T00:49:43.188066+0000","last_active":"2024-02-23T00:51:25.248323+0000","last_peered":"2024-02-23T00:51:25.248323+0000","last_clean":"2024-02-23T00:51:25.248323+0000","last_became_active":"2024-02-23T00:39:18.793414+0000","last_became_peered":"2024-02-23T00:39:18.793414+0000","last_unstale":"2024-02-23T00:51:25.248323+0000","last_undegraded":"2024-02-23T00:51:25.248323+0000","last_fullsized":"2024-02-23T00:51:25.248323+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'9","last_scrub_stamp":"2024-02-23T00:49:43.187989+0000","last_deep_scrub":"466'9","last_deep_scrub_stamp":"2024-02-23T00:49:43.187989+0000","last_clean_scrub_stamp":"2024-02-23T00:49:43.187989+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"470'42","reported_seq":603,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.244811+0000","last_change":"2024-02-23T00:49:47.653272+0000","last_active":"2024-02-23T00:51:25.244811+0000","last_peered":"2024-02-23T00:51:25.244811+0000","last_clean":"2024-02-23T00:51:25.244811+0000","last_became_active":"2024-02-23T00:39:16.859401+0000","last_became_peered":"2024-02-23T00:39:16.859401+0000","last_unstale":"2024-02-23T00:51:25.244811+0000","last_undegraded":"2024-02-23T00:51:25.244811+0000","last_fullsized":"2024-02-23T00:51:25.244811+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"470'42","last_scrub_stamp":"2024-02-23T00:49:47.653176+0000","last_deep_scrub":"470'42","last_deep_scrub_stamp":"2024-02-23T00:49:47.653176+0000","last_clean_scrub_stamp":"2024-02-23T00:49:47.653176+0000","log_size":42,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":199,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:26.243074+0000","last_change":"2024-02-23T00:49:25.955780+0000","last_active":"2024-02-23T00:51:26.243074+0000","last_peered":"2024-02-23T00:51:26.243074+0000","last_clean":"2024-02-23T00:51:26.243074+0000","last_became_active":"2024-02-23T00:47:28.220876+0000","last_became_peered":"2024-02-23T00:47:28.220876+0000","last_unstale":"2024-02-23T00:51:26.243074+0000","last_undegraded":"2024-02-23T00:51:26.243074+0000","last_fullsized":"2024-02-23T00:51:26.243074+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:25.955711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:04.330426+0000","last_clean_scrub_stamp":"2024-02-23T00:49:25.955711+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":541,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.246547+0000","last_change":"2024-02-23T00:50:14.801990+0000","last_active":"2024-02-23T00:51:25.246547+0000","last_peered":"2024-02-23T00:51:25.246547+0000","last_clean":"2024-02-23T00:51:25.246547+0000","last_became_active":"2024-02-23T00:43:32.282545+0000","last_became_peered":"2024-02-23T00:43:32.282545+0000","last_unstale":"2024-02-23T00:51:25.246547+0000","last_undegraded":"2024-02-23T00:51:25.246547+0000","last_fullsized":"2024-02-23T00:51:25.246547+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:14.801915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:50:14.801915+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":470,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.247757+0000","last_change":"2024-02-23T00:50:23.727290+0000","last_active":"2024-02-23T00:51:25.247757+0000","last_peered":"2024-02-23T00:51:25.247757+0000","last_clean":"2024-02-23T00:51:25.247757+0000","last_became_active":"2024-02-23T00:42:58.828846+0000","last_became_peered":"2024-02-23T00:42:58.828846+0000","last_unstale":"2024-02-23T00:51:25.247757+0000","last_undegraded":"2024-02-23T00:51:25.247757+0000","last_fullsized":"2024-02-23T00:51:25.247757+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:23.727173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:23.727173+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"457'21","reported_seq":567,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.245198+0000","last_change":"2024-02-23T00:49:27.765115+0000","last_active":"2024-02-23T00:51:25.245198+0000","last_peered":"2024-02-23T00:51:25.245198+0000","last_clean":"2024-02-23T00:51:25.245198+0000","last_became_active":"2024-02-23T00:39:16.783701+0000","last_became_peered":"2024-02-23T00:39:16.783701+0000","last_unstale":"2024-02-23T00:51:25.245198+0000","last_undegraded":"2024-02-23T00:51:25.245198+0000","last_fullsized":"2024-02-23T00:51:25.245198+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-23T00:49:27.765048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:27.765048+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":520,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:27.116830+0000","last_change":"2024-02-23T00:49:04.528117+0000","last_active":"2024-02-23T00:51:27.116830+0000","last_peered":"2024-02-23T00:51:27.116830+0000","last_clean":"2024-02-23T00:51:27.116830+0000","last_became_active":"2024-02-23T00:39:14.772493+0000","last_became_peered":"2024-02-23T00:39:14.772493+0000","last_unstale":"2024-02-23T00:51:27.116830+0000","last_undegraded":"2024-02-23T00:51:27.116830+0000","last_fullsized":"2024-02-23T00:51:27.116830+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:04.528006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:04.528006+0000","last_clean_scrub_stamp":"2024-02-23T00:49:04.528006+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":194,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.242477+0000","last_change":"2024-02-23T00:49:30.499854+0000","last_active":"2024-02-23T00:51:25.242477+0000","last_peered":"2024-02-23T00:51:25.242477+0000","last_clean":"2024-02-23T00:51:25.242477+0000","last_became_active":"2024-02-23T00:47:28.231567+0000","last_became_peered":"2024-02-23T00:47:28.231567+0000","last_unstale":"2024-02-23T00:51:25.242477+0000","last_undegraded":"2024-02-23T00:51:25.242477+0000","last_fullsized":"2024-02-23T00:51:25.242477+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:30.499769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:08.335330+0000","last_clean_scrub_stamp":"2024-02-23T00:49:30.499769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":550,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.245830+0000","last_change":"2024-02-23T00:51:11.526671+0000","last_active":"2024-02-23T00:51:25.245830+0000","last_peered":"2024-02-23T00:51:25.245830+0000","last_clean":"2024-02-23T00:51:25.245830+0000","last_became_active":"2024-02-23T00:47:28.214157+0000","last_became_peered":"2024-02-23T00:47:28.214157+0000","last_unstale":"2024-02-23T00:51:25.245830+0000","last_undegraded":"2024-02-23T00:51:25.245830+0000","last_fullsized":"2024-02-23T00:51:25.245830+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:11.526584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:13.867716+0000","last_clean_scrub_stamp":"2024-02-23T00:51:11.526584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":470,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.245919+0000","last_change":"2024-02-23T00:50:32.622097+0000","last_active":"2024-02-23T00:51:25.245919+0000","last_peered":"2024-02-23T00:51:25.245919+0000","last_clean":"2024-02-23T00:51:25.245919+0000","last_became_active":"2024-02-23T00:42:58.867305+0000","last_became_peered":"2024-02-23T00:42:58.867305+0000","last_unstale":"2024-02-23T00:51:25.245919+0000","last_undegraded":"2024-02-23T00:51:25.245919+0000","last_fullsized":"2024-02-23T00:51:25.245919+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:32.622026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:32.622026+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"457'51","reported_seq":610,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.259700+0000","last_change":"2024-02-23T00:51:13.326646+0000","last_active":"2024-02-23T00:51:25.259700+0000","last_peered":"2024-02-23T00:51:25.259700+0000","last_clean":"2024-02-23T00:51:25.259700+0000","last_became_active":"2024-02-23T00:39:16.857413+0000","last_became_peered":"2024-02-23T00:39:16.857413+0000","last_unstale":"2024-02-23T00:51:25.259700+0000","last_undegraded":"2024-02-23T00:51:25.259700+0000","last_fullsized":"2024-02-23T00:51:25.259700+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'51","last_scrub_stamp":"2024-02-23T00:51:13.326535+0000","last_deep_scrub":"88'35","last_deep_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_clean_scrub_stamp":"2024-02-23T00:51:13.326535+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":68,"num_read_kb":56,"num_write":42,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":202,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.247262+0000","last_change":"2024-02-23T00:50:57.770551+0000","last_active":"2024-02-23T00:51:25.247262+0000","last_peered":"2024-02-23T00:51:25.247262+0000","last_clean":"2024-02-23T00:51:25.247262+0000","last_became_active":"2024-02-23T00:47:28.231891+0000","last_became_peered":"2024-02-23T00:47:28.231891+0000","last_unstale":"2024-02-23T00:51:25.247262+0000","last_undegraded":"2024-02-23T00:51:25.247262+0000","last_fullsized":"2024-02-23T00:51:25.247262+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:57.770450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_clean_scrub_stamp":"2024-02-23T00:50:57.770450+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":197,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.245009+0000","last_change":"2024-02-23T00:49:03.682811+0000","last_active":"2024-02-23T00:51:25.245009+0000","last_peered":"2024-02-23T00:51:25.245009+0000","last_clean":"2024-02-23T00:51:25.245009+0000","last_became_active":"2024-02-23T00:47:28.220578+0000","last_became_peered":"2024-02-23T00:47:28.220578+0000","last_unstale":"2024-02-23T00:51:25.245009+0000","last_undegraded":"2024-02-23T00:51:25.245009+0000","last_fullsized":"2024-02-23T00:51:25.245009+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:03.682745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:03.682745+0000","last_clean_scrub_stamp":"2024-02-23T00:49:03.682745+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":549,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.245809+0000","last_change":"2024-02-23T00:49:42.150972+0000","last_active":"2024-02-23T00:51:25.245809+0000","last_peered":"2024-02-23T00:51:25.245809+0000","last_clean":"2024-02-23T00:51:25.245809+0000","last_became_active":"2024-02-23T00:47:28.215636+0000","last_became_peered":"2024-02-23T00:47:28.215636+0000","last_unstale":"2024-02-23T00:51:25.245809+0000","last_undegraded":"2024-02-23T00:51:25.245809+0000","last_fullsized":"2024-02-23T00:51:25.245809+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:42.150875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:42.150875+0000","last_clean_scrub_stamp":"2024-02-23T00:49:42.150875+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":462,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:27.117126+0000","last_change":"2024-02-23T00:49:34.515593+0000","last_active":"2024-02-23T00:51:27.117126+0000","last_peered":"2024-02-23T00:51:27.117126+0000","last_clean":"2024-02-23T00:51:27.117126+0000","last_became_active":"2024-02-23T00:42:57.639013+0000","last_became_peered":"2024-02-23T00:42:57.639013+0000","last_unstale":"2024-02-23T00:51:27.117126+0000","last_undegraded":"2024-02-23T00:51:27.117126+0000","last_fullsized":"2024-02-23T00:51:27.117126+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:34.515496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:49:34.515496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"457'31","reported_seq":622,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.243384+0000","last_change":"2024-02-23T00:49:31.480372+0000","last_active":"2024-02-23T00:51:25.243384+0000","last_peered":"2024-02-23T00:51:25.243384+0000","last_clean":"2024-02-23T00:51:25.243384+0000","last_became_active":"2024-02-23T00:47:28.200653+0000","last_became_peered":"2024-02-23T00:47:28.200653+0000","last_unstale":"2024-02-23T00:51:25.243384+0000","last_undegraded":"2024-02-23T00:51:25.243384+0000","last_fullsized":"2024-02-23T00:51:25.243384+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:49:31.480253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:31.480253+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":201,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.247340+0000","last_change":"2024-02-23T00:51:10.213423+0000","last_active":"2024-02-23T00:51:25.247340+0000","last_peered":"2024-02-23T00:51:25.247340+0000","last_clean":"2024-02-23T00:51:25.247340+0000","last_became_active":"2024-02-23T00:47:28.201926+0000","last_became_peered":"2024-02-23T00:47:28.201926+0000","last_unstale":"2024-02-23T00:51:25.247340+0000","last_undegraded":"2024-02-23T00:51:25.247340+0000","last_fullsized":"2024-02-23T00:51:25.247340+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:10.213337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:10.213337+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":521,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:27.116978+0000","last_change":"2024-02-23T00:51:09.343951+0000","last_active":"2024-02-23T00:51:27.116978+0000","last_peered":"2024-02-23T00:51:27.116978+0000","last_clean":"2024-02-23T00:51:27.116978+0000","last_became_active":"2024-02-23T00:39:18.799875+0000","last_became_peered":"2024-02-23T00:39:18.799875+0000","last_unstale":"2024-02-23T00:51:27.116978+0000","last_undegraded":"2024-02-23T00:51:27.116978+0000","last_fullsized":"2024-02-23T00:51:27.116978+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:09.343884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:09.343884+0000","last_clean_scrub_stamp":"2024-02-23T00:51:09.343884+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":462,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:27.115552+0000","last_change":"2024-02-23T00:48:49.610457+0000","last_active":"2024-02-23T00:51:27.115552+0000","last_peered":"2024-02-23T00:51:27.115552+0000","last_clean":"2024-02-23T00:51:27.115552+0000","last_became_active":"2024-02-23T00:42:57.675306+0000","last_became_peered":"2024-02-23T00:42:57.675306+0000","last_unstale":"2024-02-23T00:51:27.115552+0000","last_undegraded":"2024-02-23T00:51:27.115552+0000","last_fullsized":"2024-02-23T00:51:27.115552+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:49.610391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:49.610391+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"457'20","reported_seq":554,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.242053+0000","last_change":"2024-02-23T00:51:01.488450+0000","last_active":"2024-02-23T00:51:25.242053+0000","last_peered":"2024-02-23T00:51:25.242053+0000","last_clean":"2024-02-23T00:51:25.242053+0000","last_became_active":"2024-02-23T00:39:16.777530+0000","last_became_peered":"2024-02-23T00:39:16.777530+0000","last_unstale":"2024-02-23T00:51:25.242053+0000","last_undegraded":"2024-02-23T00:51:25.242053+0000","last_fullsized":"2024-02-23T00:51:25.242053+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'20","last_scrub_stamp":"2024-02-23T00:51:01.488371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:01.488371+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":520,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:27.115032+0000","last_change":"2024-02-23T00:48:41.719867+0000","last_active":"2024-02-23T00:51:27.115032+0000","last_peered":"2024-02-23T00:51:27.115032+0000","last_clean":"2024-02-23T00:51:27.115032+0000","last_became_active":"2024-02-23T00:39:14.780379+0000","last_became_peered":"2024-02-23T00:39:14.780379+0000","last_unstale":"2024-02-23T00:51:27.115032+0000","last_undegraded":"2024-02-23T00:51:27.115032+0000","last_fullsized":"2024-02-23T00:51:27.115032+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:41.719772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.575242+0000","last_clean_scrub_stamp":"2024-02-23T00:48:41.719772+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"376'9","reported_seq":269,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.247053+0000","last_change":"2024-02-23T00:49:40.180695+0000","last_active":"2024-02-23T00:51:25.247053+0000","last_peered":"2024-02-23T00:51:25.247053+0000","last_clean":"2024-02-23T00:51:25.247053+0000","last_became_active":"2024-02-23T00:47:28.231104+0000","last_became_peered":"2024-02-23T00:47:28.231104+0000","last_unstale":"2024-02-23T00:51:25.247053+0000","last_undegraded":"2024-02-23T00:51:25.247053+0000","last_fullsized":"2024-02-23T00:51:25.247053+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"376'9","last_scrub_stamp":"2024-02-23T00:49:40.180616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:40.180616+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":473,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:26.242582+0000","last_change":"2024-02-23T00:51:12.335819+0000","last_active":"2024-02-23T00:51:26.242582+0000","last_peered":"2024-02-23T00:51:26.242582+0000","last_clean":"2024-02-23T00:51:26.242582+0000","last_became_active":"2024-02-23T00:42:58.765106+0000","last_became_peered":"2024-02-23T00:42:58.765106+0000","last_unstale":"2024-02-23T00:51:26.242582+0000","last_undegraded":"2024-02-23T00:51:26.242582+0000","last_fullsized":"2024-02-23T00:51:26.242582+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.335752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.335752+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.335752+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"457'27","reported_seq":561,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:27.115944+0000","last_change":"2024-02-23T00:48:00.698737+0000","last_active":"2024-02-23T00:51:27.115944+0000","last_peered":"2024-02-23T00:51:27.115944+0000","last_clean":"2024-02-23T00:51:27.115944+0000","last_became_active":"2024-02-23T00:39:16.789757+0000","last_became_peered":"2024-02-23T00:39:16.789757+0000","last_unstale":"2024-02-23T00:51:27.115944+0000","last_undegraded":"2024-02-23T00:51:27.115944+0000","last_fullsized":"2024-02-23T00:51:27.115944+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:48:00.698670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:48:00.698670+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":525,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:27.116813+0000","last_change":"2024-02-23T00:51:13.375106+0000","last_active":"2024-02-23T00:51:27.116813+0000","last_peered":"2024-02-23T00:51:27.116813+0000","last_clean":"2024-02-23T00:51:27.116813+0000","last_became_active":"2024-02-23T00:39:14.780526+0000","last_became_peered":"2024-02-23T00:39:14.780526+0000","last_unstale":"2024-02-23T00:51:27.116813+0000","last_undegraded":"2024-02-23T00:51:27.116813+0000","last_fullsized":"2024-02-23T00:51:27.116813+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:13.375039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:13.375039+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":676,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.243831+0000","last_change":"2024-02-23T00:49:40.384676+0000","last_active":"2024-02-23T00:51:25.243831+0000","last_peered":"2024-02-23T00:51:25.243831+0000","last_clean":"2024-02-23T00:51:25.243831+0000","last_became_active":"2024-02-23T00:39:18.800440+0000","last_became_peered":"2024-02-23T00:39:18.800440+0000","last_unstale":"2024-02-23T00:51:25.243831+0000","last_undegraded":"2024-02-23T00:51:25.243831+0000","last_fullsized":"2024-02-23T00:51:25.243831+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:49:40.384608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:40.384608+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":465,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.241339+0000","last_change":"2024-02-23T00:48:59.206327+0000","last_active":"2024-02-23T00:51:25.241339+0000","last_peered":"2024-02-23T00:51:25.241339+0000","last_clean":"2024-02-23T00:51:25.241339+0000","last_became_active":"2024-02-23T00:42:58.833924+0000","last_became_peered":"2024-02-23T00:42:58.833924+0000","last_unstale":"2024-02-23T00:51:25.241339+0000","last_undegraded":"2024-02-23T00:51:25.241339+0000","last_fullsized":"2024-02-23T00:51:25.241339+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:59.206249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:59.206249+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"457'31","reported_seq":617,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.243632+0000","last_change":"2024-02-23T00:49:04.235169+0000","last_active":"2024-02-23T00:51:25.243632+0000","last_peered":"2024-02-23T00:51:25.243632+0000","last_clean":"2024-02-23T00:51:25.243632+0000","last_became_active":"2024-02-23T00:47:28.201225+0000","last_became_peered":"2024-02-23T00:47:28.201225+0000","last_unstale":"2024-02-23T00:51:25.243632+0000","last_undegraded":"2024-02-23T00:51:25.243632+0000","last_fullsized":"2024-02-23T00:51:25.243632+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"336'23","last_scrub_stamp":"2024-02-23T00:49:04.235046+0000","last_deep_scrub":"336'23","last_deep_scrub_stamp":"2024-02-23T00:46:48.320085+0000","last_clean_scrub_stamp":"2024-02-23T00:49:04.235046+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":528,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.246000+0000","last_change":"2024-02-23T00:49:37.331334+0000","last_active":"2024-02-23T00:51:25.246000+0000","last_peered":"2024-02-23T00:51:25.246000+0000","last_clean":"2024-02-23T00:51:25.246000+0000","last_became_active":"2024-02-23T00:39:14.784121+0000","last_became_peered":"2024-02-23T00:39:14.784121+0000","last_unstale":"2024-02-23T00:51:25.246000+0000","last_undegraded":"2024-02-23T00:51:25.246000+0000","last_fullsized":"2024-02-23T00:51:25.246000+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:37.331270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:37.331270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":525,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.248551+0000","last_change":"2024-02-23T00:50:19.827889+0000","last_active":"2024-02-23T00:51:25.248551+0000","last_peered":"2024-02-23T00:51:25.248551+0000","last_clean":"2024-02-23T00:51:25.248551+0000","last_became_active":"2024-02-23T00:39:18.800777+0000","last_became_peered":"2024-02-23T00:39:18.800777+0000","last_unstale":"2024-02-23T00:51:25.248551+0000","last_undegraded":"2024-02-23T00:51:25.248551+0000","last_fullsized":"2024-02-23T00:51:25.248551+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:19.827814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_clean_scrub_stamp":"2024-02-23T00:50:19.827814+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":462,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:27.114721+0000","last_change":"2024-02-23T00:49:33.529224+0000","last_active":"2024-02-23T00:51:27.114721+0000","last_peered":"2024-02-23T00:51:27.114721+0000","last_clean":"2024-02-23T00:51:27.114721+0000","last_became_active":"2024-02-23T00:42:57.653279+0000","last_became_peered":"2024-02-23T00:42:57.653279+0000","last_unstale":"2024-02-23T00:51:27.114721+0000","last_undegraded":"2024-02-23T00:51:27.114721+0000","last_fullsized":"2024-02-23T00:51:27.114721+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:33.529135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:49:33.529135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":677,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:26.242382+0000","last_change":"2024-02-23T00:49:08.047225+0000","last_active":"2024-02-23T00:51:26.242382+0000","last_peered":"2024-02-23T00:51:26.242382+0000","last_clean":"2024-02-23T00:51:26.242382+0000","last_became_active":"2024-02-23T00:39:18.798536+0000","last_became_peered":"2024-02-23T00:39:18.798536+0000","last_unstale":"2024-02-23T00:51:26.242382+0000","last_undegraded":"2024-02-23T00:51:26.242382+0000","last_fullsized":"2024-02-23T00:51:26.242382+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:49:08.047142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:08.047142+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"457'42","reported_seq":589,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.241473+0000","last_change":"2024-02-23T00:50:24.532387+0000","last_active":"2024-02-23T00:51:25.241473+0000","last_peered":"2024-02-23T00:51:25.241473+0000","last_clean":"2024-02-23T00:51:25.241473+0000","last_became_active":"2024-02-23T00:39:16.780130+0000","last_became_peered":"2024-02-23T00:39:16.780130+0000","last_unstale":"2024-02-23T00:51:25.241473+0000","last_undegraded":"2024-02-23T00:51:25.241473+0000","last_fullsized":"2024-02-23T00:51:25.241473+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'42","last_scrub_stamp":"2024-02-23T00:50:24.532271+0000","last_deep_scrub":"423'28","last_deep_scrub_stamp":"2024-02-23T00:48:17.435906+0000","last_clean_scrub_stamp":"2024-02-23T00:50:24.532271+0000","log_size":42,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":46,"num_write":34,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":543,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.255198+0000","last_change":"2024-02-23T00:50:59.199966+0000","last_active":"2024-02-23T00:51:25.255198+0000","last_peered":"2024-02-23T00:51:25.255198+0000","last_clean":"2024-02-23T00:51:25.255198+0000","last_became_active":"2024-02-23T00:47:28.237083+0000","last_became_peered":"2024-02-23T00:47:28.237083+0000","last_unstale":"2024-02-23T00:51:25.255198+0000","last_undegraded":"2024-02-23T00:51:25.255198+0000","last_fullsized":"2024-02-23T00:51:25.255198+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:59.199899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:51.391939+0000","last_clean_scrub_stamp":"2024-02-23T00:50:59.199899+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":541,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.249535+0000","last_change":"2024-02-23T00:49:41.163033+0000","last_active":"2024-02-23T00:51:25.249535+0000","last_peered":"2024-02-23T00:51:25.249535+0000","last_clean":"2024-02-23T00:51:25.249535+0000","last_became_active":"2024-02-23T00:47:30.187386+0000","last_became_peered":"2024-02-23T00:47:30.187386+0000","last_unstale":"2024-02-23T00:51:25.249535+0000","last_undegraded":"2024-02-23T00:51:25.249535+0000","last_fullsized":"2024-02-23T00:51:25.249535+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:41.162895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:46.686764+0000","last_clean_scrub_stamp":"2024-02-23T00:49:41.162895+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":479,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.248782+0000","last_change":"2024-02-23T00:51:03.875340+0000","last_active":"2024-02-23T00:51:25.248782+0000","last_peered":"2024-02-23T00:51:25.248782+0000","last_clean":"2024-02-23T00:51:25.248782+0000","last_became_active":"2024-02-23T00:47:28.212307+0000","last_became_peered":"2024-02-23T00:47:28.212307+0000","last_unstale":"2024-02-23T00:51:25.248782+0000","last_undegraded":"2024-02-23T00:51:25.248782+0000","last_fullsized":"2024-02-23T00:51:25.248782+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:03.875273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:03.875273+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":524,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.241761+0000","last_change":"2024-02-23T00:50:25.575319+0000","last_active":"2024-02-23T00:51:25.241761+0000","last_peered":"2024-02-23T00:51:25.241761+0000","last_clean":"2024-02-23T00:51:25.241761+0000","last_became_active":"2024-02-23T00:39:18.798829+0000","last_became_peered":"2024-02-23T00:39:18.798829+0000","last_unstale":"2024-02-23T00:51:25.241761+0000","last_undegraded":"2024-02-23T00:51:25.241761+0000","last_fullsized":"2024-02-23T00:51:25.241761+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:25.575238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:10.493340+0000","last_clean_scrub_stamp":"2024-02-23T00:50:25.575238+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"457'52","reported_seq":620,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:26.242364+0000","last_change":"2024-02-23T00:51:05.221646+0000","last_active":"2024-02-23T00:51:26.242364+0000","last_peered":"2024-02-23T00:51:26.242364+0000","last_clean":"2024-02-23T00:51:26.242364+0000","last_became_active":"2024-02-23T00:39:16.859002+0000","last_became_peered":"2024-02-23T00:39:16.859002+0000","last_unstale":"2024-02-23T00:51:26.242364+0000","last_undegraded":"2024-02-23T00:51:26.242364+0000","last_fullsized":"2024-02-23T00:51:26.242364+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'52","last_scrub_stamp":"2024-02-23T00:51:05.221572+0000","last_deep_scrub":"251'40","last_deep_scrub_stamp":"2024-02-23T00:45:23.910546+0000","last_clean_scrub_stamp":"2024-02-23T00:51:05.221572+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":76,"num_read_kb":62,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":520,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:27.115730+0000","last_change":"2024-02-23T00:48:44.728592+0000","last_active":"2024-02-23T00:51:27.115730+0000","last_peered":"2024-02-23T00:51:27.115730+0000","last_clean":"2024-02-23T00:51:27.115730+0000","last_became_active":"2024-02-23T00:39:14.779193+0000","last_became_peered":"2024-02-23T00:39:14.779193+0000","last_unstale":"2024-02-23T00:51:27.115730+0000","last_undegraded":"2024-02-23T00:51:27.115730+0000","last_fullsized":"2024-02-23T00:51:27.115730+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:44.728520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:48:44.728520+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":547,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.242530+0000","last_change":"2024-02-23T00:50:17.599788+0000","last_active":"2024-02-23T00:51:25.242530+0000","last_peered":"2024-02-23T00:51:25.242530+0000","last_clean":"2024-02-23T00:51:25.242530+0000","last_became_active":"2024-02-23T00:47:33.257915+0000","last_became_peered":"2024-02-23T00:47:33.257915+0000","last_unstale":"2024-02-23T00:51:25.242530+0000","last_undegraded":"2024-02-23T00:51:25.242530+0000","last_fullsized":"2024-02-23T00:51:25.242530+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":381,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:17.599724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:50:17.599724+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":473,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:26.243370+0000","last_change":"2024-02-23T00:51:03.166243+0000","last_active":"2024-02-23T00:51:26.243370+0000","last_peered":"2024-02-23T00:51:26.243370+0000","last_clean":"2024-02-23T00:51:26.243370+0000","last_became_active":"2024-02-23T00:42:58.809283+0000","last_became_peered":"2024-02-23T00:42:58.809283+0000","last_unstale":"2024-02-23T00:51:26.243370+0000","last_undegraded":"2024-02-23T00:51:26.243370+0000","last_fullsized":"2024-02-23T00:51:26.243370+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:03.166140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:03.166140+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":523,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.259615+0000","last_change":"2024-02-23T00:48:49.266366+0000","last_active":"2024-02-23T00:51:25.259615+0000","last_peered":"2024-02-23T00:51:25.259615+0000","last_clean":"2024-02-23T00:51:25.259615+0000","last_became_active":"2024-02-23T00:39:18.796664+0000","last_became_peered":"2024-02-23T00:39:18.796664+0000","last_unstale":"2024-02-23T00:51:25.259615+0000","last_undegraded":"2024-02-23T00:51:25.259615+0000","last_fullsized":"2024-02-23T00:51:25.259615+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:49.266295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:48:49.266295+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"457'44","reported_seq":602,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.244916+0000","last_change":"2024-02-23T00:49:46.644982+0000","last_active":"2024-02-23T00:51:25.244916+0000","last_peered":"2024-02-23T00:51:25.244916+0000","last_clean":"2024-02-23T00:51:25.244916+0000","last_became_active":"2024-02-23T00:39:16.857278+0000","last_became_peered":"2024-02-23T00:39:16.857278+0000","last_unstale":"2024-02-23T00:51:25.244916+0000","last_undegraded":"2024-02-23T00:51:25.244916+0000","last_fullsized":"2024-02-23T00:51:25.244916+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'44","last_scrub_stamp":"2024-02-23T00:49:46.644911+0000","last_deep_scrub":"457'44","last_deep_scrub_stamp":"2024-02-23T00:49:46.644911+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.644911+0000","log_size":44,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":528,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.246903+0000","last_change":"2024-02-23T00:50:22.299591+0000","last_active":"2024-02-23T00:51:25.246903+0000","last_peered":"2024-02-23T00:51:25.246903+0000","last_clean":"2024-02-23T00:51:25.246903+0000","last_became_active":"2024-02-23T00:39:14.783662+0000","last_became_peered":"2024-02-23T00:39:14.783662+0000","last_unstale":"2024-02-23T00:51:25.246903+0000","last_undegraded":"2024-02-23T00:51:25.246903+0000","last_fullsized":"2024-02-23T00:51:25.246903+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:22.299534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:22.299534+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":541,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:27.116897+0000","last_change":"2024-02-23T00:48:13.821965+0000","last_active":"2024-02-23T00:51:27.116897+0000","last_peered":"2024-02-23T00:51:27.116897+0000","last_clean":"2024-02-23T00:51:27.116897+0000","last_became_active":"2024-02-23T00:47:37.184240+0000","last_became_peered":"2024-02-23T00:47:37.184240+0000","last_unstale":"2024-02-23T00:51:27.116897+0000","last_undegraded":"2024-02-23T00:51:27.116897+0000","last_fullsized":"2024-02-23T00:51:27.116897+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":385,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:13.821901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:48:13.821901+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":470,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.245915+0000","last_change":"2024-02-23T00:50:24.682130+0000","last_active":"2024-02-23T00:51:25.245915+0000","last_peered":"2024-02-23T00:51:25.245915+0000","last_clean":"2024-02-23T00:51:25.245915+0000","last_became_active":"2024-02-23T00:42:58.726652+0000","last_became_peered":"2024-02-23T00:42:58.726652+0000","last_unstale":"2024-02-23T00:51:25.245915+0000","last_undegraded":"2024-02-23T00:51:25.245915+0000","last_fullsized":"2024-02-23T00:51:25.245915+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:24.682039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:24.682039+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":525,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.248621+0000","last_change":"2024-02-23T00:49:34.709320+0000","last_active":"2024-02-23T00:51:25.248621+0000","last_peered":"2024-02-23T00:51:25.248621+0000","last_clean":"2024-02-23T00:51:25.248621+0000","last_became_active":"2024-02-23T00:39:18.800378+0000","last_became_peered":"2024-02-23T00:39:18.800378+0000","last_unstale":"2024-02-23T00:51:25.248621+0000","last_undegraded":"2024-02-23T00:51:25.248621+0000","last_fullsized":"2024-02-23T00:51:25.248621+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:34.709251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:59.694033+0000","last_clean_scrub_stamp":"2024-02-23T00:49:34.709251+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"457'30","reported_seq":592,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.245205+0000","last_change":"2024-02-23T00:49:42.730745+0000","last_active":"2024-02-23T00:51:25.245205+0000","last_peered":"2024-02-23T00:51:25.245205+0000","last_clean":"2024-02-23T00:51:25.245205+0000","last_became_active":"2024-02-23T00:47:28.215366+0000","last_became_peered":"2024-02-23T00:47:28.215366+0000","last_unstale":"2024-02-23T00:51:25.245205+0000","last_undegraded":"2024-02-23T00:51:25.245205+0000","last_fullsized":"2024-02-23T00:51:25.245205+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'30","last_scrub_stamp":"2024-02-23T00:49:42.730670+0000","last_deep_scrub":"95'18","last_deep_scrub_stamp":"2024-02-23T00:42:01.195545+0000","last_clean_scrub_stamp":"2024-02-23T00:49:42.730670+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":533,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.245902+0000","last_change":"2024-02-23T00:49:46.162522+0000","last_active":"2024-02-23T00:51:25.245902+0000","last_peered":"2024-02-23T00:51:25.245902+0000","last_clean":"2024-02-23T00:51:25.245902+0000","last_became_active":"2024-02-23T00:39:14.784578+0000","last_became_peered":"2024-02-23T00:39:14.784578+0000","last_unstale":"2024-02-23T00:51:25.245902+0000","last_undegraded":"2024-02-23T00:51:25.245902+0000","last_fullsized":"2024-02-23T00:51:25.245902+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:46.162444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.162444+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":529,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.244455+0000","last_change":"2024-02-23T00:49:31.768539+0000","last_active":"2024-02-23T00:51:25.244455+0000","last_peered":"2024-02-23T00:51:25.244455+0000","last_clean":"2024-02-23T00:51:25.244455+0000","last_became_active":"2024-02-23T00:39:14.775446+0000","last_became_peered":"2024-02-23T00:39:14.775446+0000","last_unstale":"2024-02-23T00:51:25.244455+0000","last_undegraded":"2024-02-23T00:51:25.244455+0000","last_fullsized":"2024-02-23T00:51:25.244455+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:31.768472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:31.768472+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"67'1","reported_seq":552,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:27.117425+0000","last_change":"2024-02-23T00:51:02.380789+0000","last_active":"2024-02-23T00:51:27.117425+0000","last_peered":"2024-02-23T00:51:27.117425+0000","last_clean":"2024-02-23T00:51:27.117425+0000","last_became_active":"2024-02-23T00:47:56.187940+0000","last_became_peered":"2024-02-23T00:47:56.187940+0000","last_unstale":"2024-02-23T00:51:27.117425+0000","last_undegraded":"2024-02-23T00:51:27.117425+0000","last_fullsized":"2024-02-23T00:51:27.117425+0000","mapping_epoch":65,"log_start":"67'1","ondisk_log_start":"67'1","created":65,"last_epoch_clean":404,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:51:02.380721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:51:02.380721+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":474,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.254154+0000","last_change":"2024-02-23T00:51:10.298816+0000","last_active":"2024-02-23T00:51:25.254154+0000","last_peered":"2024-02-23T00:51:25.254154+0000","last_clean":"2024-02-23T00:51:25.254154+0000","last_became_active":"2024-02-23T00:42:58.690858+0000","last_became_peered":"2024-02-23T00:42:58.690858+0000","last_unstale":"2024-02-23T00:51:25.254154+0000","last_undegraded":"2024-02-23T00:51:25.254154+0000","last_fullsized":"2024-02-23T00:51:25.254154+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:10.298740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:35.360054+0000","last_clean_scrub_stamp":"2024-02-23T00:51:10.298740+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":533,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.246718+0000","last_change":"2024-02-23T00:49:28.358727+0000","last_active":"2024-02-23T00:51:25.246718+0000","last_peered":"2024-02-23T00:51:25.246718+0000","last_clean":"2024-02-23T00:51:25.246718+0000","last_became_active":"2024-02-23T00:47:28.231152+0000","last_became_peered":"2024-02-23T00:47:28.231152+0000","last_unstale":"2024-02-23T00:51:25.246718+0000","last_undegraded":"2024-02-23T00:51:25.246718+0000","last_fullsized":"2024-02-23T00:51:25.246718+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:28.358647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:28.358647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"457'26","reported_seq":570,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.258584+0000","last_change":"2024-02-23T00:51:01.219903+0000","last_active":"2024-02-23T00:51:25.258584+0000","last_peered":"2024-02-23T00:51:25.258584+0000","last_clean":"2024-02-23T00:51:25.258584+0000","last_became_active":"2024-02-23T00:39:16.858880+0000","last_became_peered":"2024-02-23T00:39:16.858880+0000","last_unstale":"2024-02-23T00:51:25.258584+0000","last_undegraded":"2024-02-23T00:51:25.258584+0000","last_fullsized":"2024-02-23T00:51:25.258584+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-23T00:51:01.219824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:01.219824+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":538,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.245905+0000","last_change":"2024-02-23T00:50:24.297660+0000","last_active":"2024-02-23T00:51:25.245905+0000","last_peered":"2024-02-23T00:51:25.245905+0000","last_clean":"2024-02-23T00:51:25.245905+0000","last_became_active":"2024-02-23T00:39:08.248249+0000","last_became_peered":"2024-02-23T00:39:08.248249+0000","last_unstale":"2024-02-23T00:51:25.245905+0000","last_undegraded":"2024-02-23T00:51:25.245905+0000","last_fullsized":"2024-02-23T00:51:25.245905+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:24.297572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:49.977248+0000","last_clean_scrub_stamp":"2024-02-23T00:50:24.297572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":538,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.245572+0000","last_change":"2024-02-23T00:50:21.746072+0000","last_active":"2024-02-23T00:51:25.245572+0000","last_peered":"2024-02-23T00:51:25.245572+0000","last_clean":"2024-02-23T00:51:25.245572+0000","last_became_active":"2024-02-23T00:47:28.240482+0000","last_became_peered":"2024-02-23T00:47:28.240482+0000","last_unstale":"2024-02-23T00:51:25.245572+0000","last_undegraded":"2024-02-23T00:51:25.245572+0000","last_fullsized":"2024-02-23T00:51:25.245572+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:21.745961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:21.745961+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":532,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:26.242489+0000","last_change":"2024-02-23T00:50:34.055990+0000","last_active":"2024-02-23T00:51:26.242489+0000","last_peered":"2024-02-23T00:51:26.242489+0000","last_clean":"2024-02-23T00:51:26.242489+0000","last_became_active":"2024-02-23T00:39:14.783489+0000","last_became_peered":"2024-02-23T00:39:14.783489+0000","last_unstale":"2024-02-23T00:51:26.242489+0000","last_undegraded":"2024-02-23T00:51:26.242489+0000","last_fullsized":"2024-02-23T00:51:26.242489+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:34.055923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:34.055923+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":559,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.248700+0000","last_change":"2024-02-23T00:49:35.756168+0000","last_active":"2024-02-23T00:51:25.248700+0000","last_peered":"2024-02-23T00:51:25.248700+0000","last_clean":"2024-02-23T00:51:25.248700+0000","last_became_active":"2024-02-23T00:47:45.182550+0000","last_became_peered":"2024-02-23T00:47:45.182550+0000","last_unstale":"2024-02-23T00:51:25.248700+0000","last_undegraded":"2024-02-23T00:51:25.248700+0000","last_fullsized":"2024-02-23T00:51:25.248700+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:35.756073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:49:35.756073+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":482,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.249373+0000","last_change":"2024-02-23T00:50:16.257963+0000","last_active":"2024-02-23T00:51:25.249373+0000","last_peered":"2024-02-23T00:51:25.249373+0000","last_clean":"2024-02-23T00:51:25.249373+0000","last_became_active":"2024-02-23T00:47:28.233475+0000","last_became_peered":"2024-02-23T00:47:28.233475+0000","last_unstale":"2024-02-23T00:51:25.249373+0000","last_undegraded":"2024-02-23T00:51:25.249373+0000","last_fullsized":"2024-02-23T00:51:25.249373+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:16.257896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:16.257896+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":202,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.247816+0000","last_change":"2024-02-23T00:51:12.709695+0000","last_active":"2024-02-23T00:51:25.247816+0000","last_peered":"2024-02-23T00:51:25.247816+0000","last_clean":"2024-02-23T00:51:25.247816+0000","last_became_active":"2024-02-23T00:47:28.209125+0000","last_became_peered":"2024-02-23T00:47:28.209125+0000","last_unstale":"2024-02-23T00:51:25.247816+0000","last_undegraded":"2024-02-23T00:51:25.247816+0000","last_fullsized":"2024-02-23T00:51:25.247816+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.709617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.264240+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.709617+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"457'23","reported_seq":566,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:27.116042+0000","last_change":"2024-02-23T00:48:11.828616+0000","last_active":"2024-02-23T00:51:27.116042+0000","last_peered":"2024-02-23T00:51:27.116042+0000","last_clean":"2024-02-23T00:51:27.116042+0000","last_became_active":"2024-02-23T00:39:16.787484+0000","last_became_peered":"2024-02-23T00:39:16.787484+0000","last_unstale":"2024-02-23T00:51:27.116042+0000","last_undegraded":"2024-02-23T00:51:27.116042+0000","last_fullsized":"2024-02-23T00:51:27.116042+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'19","last_scrub_stamp":"2024-02-23T00:48:11.828547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:48:11.828547+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":48,"num_read_kb":36,"num_write":26,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":199,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:26.243074+0000","last_change":"2024-02-23T00:49:35.877867+0000","last_active":"2024-02-23T00:51:26.243074+0000","last_peered":"2024-02-23T00:51:26.243074+0000","last_clean":"2024-02-23T00:51:26.243074+0000","last_became_active":"2024-02-23T00:47:28.209575+0000","last_became_peered":"2024-02-23T00:47:28.209575+0000","last_unstale":"2024-02-23T00:51:26.243074+0000","last_undegraded":"2024-02-23T00:51:26.243074+0000","last_fullsized":"2024-02-23T00:51:26.243074+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:35.877800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:35.877800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"68'5","reported_seq":1259,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:27.115069+0000","last_change":"2024-02-23T00:50:57.456201+0000","last_active":"2024-02-23T00:51:27.115069+0000","last_peered":"2024-02-23T00:51:27.115069+0000","last_clean":"2024-02-23T00:51:27.115069+0000","last_became_active":"2024-02-23T00:47:28.227928+0000","last_became_peered":"2024-02-23T00:47:28.227928+0000","last_unstale":"2024-02-23T00:51:27.115069+0000","last_undegraded":"2024-02-23T00:51:27.115069+0000","last_fullsized":"2024-02-23T00:51:27.115069+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'5","last_scrub_stamp":"2024-02-23T00:50:57.456059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:57.456059+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":722,"num_read_kb":717,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":558,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.246820+0000","last_change":"2024-02-23T00:49:30.396935+0000","last_active":"2024-02-23T00:51:25.246820+0000","last_peered":"2024-02-23T00:51:25.246820+0000","last_clean":"2024-02-23T00:51:25.246820+0000","last_became_active":"2024-02-23T00:47:52.552244+0000","last_became_peered":"2024-02-23T00:47:52.552244+0000","last_unstale":"2024-02-23T00:51:25.246820+0000","last_undegraded":"2024-02-23T00:51:25.246820+0000","last_fullsized":"2024-02-23T00:51:25.246820+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:30.396865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:49:30.396865+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":471,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:27.116614+0000","last_change":"2024-02-23T00:49:40.533441+0000","last_active":"2024-02-23T00:51:27.116614+0000","last_peered":"2024-02-23T00:51:27.116614+0000","last_clean":"2024-02-23T00:51:27.116614+0000","last_became_active":"2024-02-23T00:47:28.212516+0000","last_became_peered":"2024-02-23T00:47:28.212516+0000","last_unstale":"2024-02-23T00:51:27.116614+0000","last_undegraded":"2024-02-23T00:51:27.116614+0000","last_fullsized":"2024-02-23T00:51:27.116614+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:40.533344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:49:40.533344+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":525,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.245916+0000","last_change":"2024-02-23T00:49:37.787041+0000","last_active":"2024-02-23T00:51:25.245916+0000","last_peered":"2024-02-23T00:51:25.245916+0000","last_clean":"2024-02-23T00:51:25.245916+0000","last_became_active":"2024-02-23T00:39:18.802134+0000","last_became_peered":"2024-02-23T00:39:18.802134+0000","last_unstale":"2024-02-23T00:51:25.245916+0000","last_undegraded":"2024-02-23T00:51:25.245916+0000","last_fullsized":"2024-02-23T00:51:25.245916+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:37.786978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:37.786978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"457'48","reported_seq":708,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.247103+0000","last_change":"2024-02-23T00:50:30.763802+0000","last_active":"2024-02-23T00:51:25.247103+0000","last_peered":"2024-02-23T00:51:25.247103+0000","last_clean":"2024-02-23T00:51:25.247103+0000","last_became_active":"2024-02-23T00:47:28.240257+0000","last_became_peered":"2024-02-23T00:47:28.240257+0000","last_unstale":"2024-02-23T00:51:25.247103+0000","last_undegraded":"2024-02-23T00:51:25.247103+0000","last_fullsized":"2024-02-23T00:51:25.247103+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'48","last_scrub_stamp":"2024-02-23T00:50:30.763686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:30.763686+0000","log_size":48,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":78,"num_read_kb":62,"num_write":44,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":529,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.246699+0000","last_change":"2024-02-23T00:49:41.639660+0000","last_active":"2024-02-23T00:51:25.246699+0000","last_peered":"2024-02-23T00:51:25.246699+0000","last_clean":"2024-02-23T00:51:25.246699+0000","last_became_active":"2024-02-23T00:39:14.784015+0000","last_became_peered":"2024-02-23T00:39:14.784015+0000","last_unstale":"2024-02-23T00:51:25.246699+0000","last_undegraded":"2024-02-23T00:51:25.246699+0000","last_fullsized":"2024-02-23T00:51:25.246699+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:41.639582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:41.639582+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":557,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:26.243262+0000","last_change":"2024-02-23T00:49:36.867589+0000","last_active":"2024-02-23T00:51:26.243262+0000","last_peered":"2024-02-23T00:51:26.243262+0000","last_clean":"2024-02-23T00:51:26.243262+0000","last_became_active":"2024-02-23T00:47:49.525421+0000","last_became_peered":"2024-02-23T00:47:49.525421+0000","last_unstale":"2024-02-23T00:51:26.243262+0000","last_undegraded":"2024-02-23T00:51:26.243262+0000","last_fullsized":"2024-02-23T00:51:26.243262+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":397,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:36.867488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:49:36.867488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":197,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.242617+0000","last_change":"2024-02-23T00:50:13.549604+0000","last_active":"2024-02-23T00:51:25.242617+0000","last_peered":"2024-02-23T00:51:25.242617+0000","last_clean":"2024-02-23T00:51:25.242617+0000","last_became_active":"2024-02-23T00:47:28.232194+0000","last_became_peered":"2024-02-23T00:47:28.232194+0000","last_unstale":"2024-02-23T00:51:25.242617+0000","last_undegraded":"2024-02-23T00:51:25.242617+0000","last_fullsized":"2024-02-23T00:51:25.242617+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:13.549529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:13.549529+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3","version":"91'9","reported_seq":680,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.245303+0000","last_change":"2024-02-23T00:49:41.776249+0000","last_active":"2024-02-23T00:51:25.245303+0000","last_peered":"2024-02-23T00:51:25.245303+0000","last_clean":"2024-02-23T00:51:25.245303+0000","last_became_active":"2024-02-23T00:39:18.801297+0000","last_became_peered":"2024-02-23T00:39:18.801297+0000","last_unstale":"2024-02-23T00:51:25.245303+0000","last_undegraded":"2024-02-23T00:51:25.245303+0000","last_fullsized":"2024-02-23T00:51:25.245303+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'9","last_scrub_stamp":"2024-02-23T00:49:41.776117+0000","last_deep_scrub":"91'9","last_deep_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_clean_scrub_stamp":"2024-02-23T00:49:41.776117+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"457'26","reported_seq":563,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:27.115279+0000","last_change":"2024-02-23T00:48:55.589320+0000","last_active":"2024-02-23T00:51:27.115279+0000","last_peered":"2024-02-23T00:51:27.115279+0000","last_clean":"2024-02-23T00:51:27.115279+0000","last_became_active":"2024-02-23T00:39:16.787468+0000","last_became_peered":"2024-02-23T00:39:16.787468+0000","last_unstale":"2024-02-23T00:51:27.115279+0000","last_undegraded":"2024-02-23T00:51:27.115279+0000","last_fullsized":"2024-02-23T00:51:27.115279+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:48:55.589209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:48:55.589209+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":36,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":520,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:27.117231+0000","last_change":"2024-02-23T00:48:12.869003+0000","last_active":"2024-02-23T00:51:27.117231+0000","last_peered":"2024-02-23T00:51:27.117231+0000","last_clean":"2024-02-23T00:51:27.117231+0000","last_became_active":"2024-02-23T00:39:14.779952+0000","last_became_peered":"2024-02-23T00:39:14.779952+0000","last_unstale":"2024-02-23T00:51:27.117231+0000","last_undegraded":"2024-02-23T00:51:27.117231+0000","last_fullsized":"2024-02-23T00:51:27.117231+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:12.868930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:48:12.868930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"107'2","reported_seq":558,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.248672+0000","last_change":"2024-02-23T00:50:17.587665+0000","last_active":"2024-02-23T00:51:25.248672+0000","last_peered":"2024-02-23T00:51:25.248672+0000","last_clean":"2024-02-23T00:51:25.248672+0000","last_became_active":"2024-02-23T00:47:41.210074+0000","last_became_peered":"2024-02-23T00:47:41.210074+0000","last_unstale":"2024-02-23T00:51:25.248672+0000","last_undegraded":"2024-02-23T00:51:25.248672+0000","last_fullsized":"2024-02-23T00:51:25.248672+0000","mapping_epoch":375,"log_start":"107'2","ondisk_log_start":"107'2","created":65,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'2","last_scrub_stamp":"2024-02-23T00:50:17.587591+0000","last_deep_scrub":"107'2","last_deep_scrub_stamp":"2024-02-23T00:42:48.362739+0000","last_clean_scrub_stamp":"2024-02-23T00:50:17.587591+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":478,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.254957+0000","last_change":"2024-02-23T00:50:18.308436+0000","last_active":"2024-02-23T00:51:25.254957+0000","last_peered":"2024-02-23T00:51:25.254957+0000","last_clean":"2024-02-23T00:51:25.254957+0000","last_became_active":"2024-02-23T00:47:28.242255+0000","last_became_peered":"2024-02-23T00:47:28.242255+0000","last_unstale":"2024-02-23T00:51:25.254957+0000","last_undegraded":"2024-02-23T00:51:25.254957+0000","last_fullsized":"2024-02-23T00:51:25.254957+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:18.308350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:18.308350+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":197,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.249136+0000","last_change":"2024-02-23T00:48:54.706708+0000","last_active":"2024-02-23T00:51:25.249136+0000","last_peered":"2024-02-23T00:51:25.249136+0000","last_clean":"2024-02-23T00:51:25.249136+0000","last_became_active":"2024-02-23T00:47:28.234034+0000","last_became_peered":"2024-02-23T00:47:28.234034+0000","last_unstale":"2024-02-23T00:51:25.249136+0000","last_undegraded":"2024-02-23T00:51:25.249136+0000","last_fullsized":"2024-02-23T00:51:25.249136+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:54.706627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_clean_scrub_stamp":"2024-02-23T00:48:54.706627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"457'31","reported_seq":582,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.241691+0000","last_change":"2024-02-23T00:49:08.190437+0000","last_active":"2024-02-23T00:51:25.241691+0000","last_peered":"2024-02-23T00:51:25.241691+0000","last_clean":"2024-02-23T00:51:25.241691+0000","last_became_active":"2024-02-23T00:39:16.783926+0000","last_became_peered":"2024-02-23T00:39:16.783926+0000","last_unstale":"2024-02-23T00:51:25.241691+0000","last_undegraded":"2024-02-23T00:51:25.241691+0000","last_fullsized":"2024-02-23T00:51:25.241691+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'25","last_scrub_stamp":"2024-02-23T00:49:08.190363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:08.190363+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":58,"num_read_kb":47,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"95'4","reported_seq":541,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.244779+0000","last_change":"2024-02-23T00:49:33.465769+0000","last_active":"2024-02-23T00:51:25.244779+0000","last_peered":"2024-02-23T00:51:25.244779+0000","last_clean":"2024-02-23T00:51:25.244779+0000","last_became_active":"2024-02-23T00:39:14.779205+0000","last_became_peered":"2024-02-23T00:39:14.779205+0000","last_unstale":"2024-02-23T00:51:25.244779+0000","last_undegraded":"2024-02-23T00:51:25.244779+0000","last_fullsized":"2024-02-23T00:51:25.244779+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'4","last_scrub_stamp":"2024-02-23T00:49:33.465697+0000","last_deep_scrub":"95'4","last_deep_scrub_stamp":"2024-02-23T00:41:43.874397+0000","last_clean_scrub_stamp":"2024-02-23T00:49:33.465697+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":544,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:26.242924+0000","last_change":"2024-02-23T00:50:15.922829+0000","last_active":"2024-02-23T00:51:26.242924+0000","last_peered":"2024-02-23T00:51:26.242924+0000","last_clean":"2024-02-23T00:51:26.242924+0000","last_became_active":"2024-02-23T00:43:27.611660+0000","last_became_peered":"2024-02-23T00:43:27.611660+0000","last_unstale":"2024-02-23T00:51:26.242924+0000","last_undegraded":"2024-02-23T00:51:26.242924+0000","last_fullsized":"2024-02-23T00:51:26.242924+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:15.922745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:50:15.922745+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":480,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.248137+0000","last_change":"2024-02-23T00:49:44.143768+0000","last_active":"2024-02-23T00:51:25.248137+0000","last_peered":"2024-02-23T00:51:25.248137+0000","last_clean":"2024-02-23T00:51:25.248137+0000","last_became_active":"2024-02-23T00:47:28.231670+0000","last_became_peered":"2024-02-23T00:47:28.231670+0000","last_unstale":"2024-02-23T00:51:25.248137+0000","last_undegraded":"2024-02-23T00:51:25.248137+0000","last_fullsized":"2024-02-23T00:51:25.248137+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:44.143684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:49:44.143684+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"457'27","reported_seq":577,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.245246+0000","last_change":"2024-02-23T00:49:36.823322+0000","last_active":"2024-02-23T00:51:25.245246+0000","last_peered":"2024-02-23T00:51:25.245246+0000","last_clean":"2024-02-23T00:51:25.245246+0000","last_became_active":"2024-02-23T00:39:16.858760+0000","last_became_peered":"2024-02-23T00:39:16.858760+0000","last_unstale":"2024-02-23T00:51:25.245246+0000","last_undegraded":"2024-02-23T00:51:25.245246+0000","last_fullsized":"2024-02-23T00:51:25.245246+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-23T00:49:36.823214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:36.823214+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":195,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:27.114804+0000","last_change":"2024-02-23T00:49:02.601790+0000","last_active":"2024-02-23T00:51:27.114804+0000","last_peered":"2024-02-23T00:51:27.114804+0000","last_clean":"2024-02-23T00:51:27.114804+0000","last_became_active":"2024-02-23T00:47:28.219764+0000","last_became_peered":"2024-02-23T00:47:28.219764+0000","last_unstale":"2024-02-23T00:51:27.114804+0000","last_undegraded":"2024-02-23T00:51:27.114804+0000","last_fullsized":"2024-02-23T00:51:27.114804+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:02.601726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:02.601726+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":528,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:26.243698+0000","last_change":"2024-02-23T00:49:37.881251+0000","last_active":"2024-02-23T00:51:26.243698+0000","last_peered":"2024-02-23T00:51:26.243698+0000","last_clean":"2024-02-23T00:51:26.243698+0000","last_became_active":"2024-02-23T00:39:18.798624+0000","last_became_peered":"2024-02-23T00:39:18.798624+0000","last_unstale":"2024-02-23T00:51:26.243698+0000","last_undegraded":"2024-02-23T00:51:26.243698+0000","last_fullsized":"2024-02-23T00:51:26.243698+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:37.881183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:45.913526+0000","last_clean_scrub_stamp":"2024-02-23T00:49:37.881183+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":470,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.248879+0000","last_change":"2024-02-23T00:50:13.642424+0000","last_active":"2024-02-23T00:51:25.248879+0000","last_peered":"2024-02-23T00:51:25.248879+0000","last_clean":"2024-02-23T00:51:25.248879+0000","last_became_active":"2024-02-23T00:42:58.825015+0000","last_became_peered":"2024-02-23T00:42:58.825015+0000","last_unstale":"2024-02-23T00:51:25.248879+0000","last_undegraded":"2024-02-23T00:51:25.248879+0000","last_fullsized":"2024-02-23T00:51:25.248879+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:13.642359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:13.642359+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"457'22","reported_seq":567,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:26.243612+0000","last_change":"2024-02-23T00:51:01.201807+0000","last_active":"2024-02-23T00:51:26.243612+0000","last_peered":"2024-02-23T00:51:26.243612+0000","last_clean":"2024-02-23T00:51:26.243612+0000","last_became_active":"2024-02-23T00:39:16.858449+0000","last_became_peered":"2024-02-23T00:39:16.858449+0000","last_unstale":"2024-02-23T00:51:26.243612+0000","last_undegraded":"2024-02-23T00:51:26.243612+0000","last_fullsized":"2024-02-23T00:51:26.243612+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'22","last_scrub_stamp":"2024-02-23T00:51:01.201737+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-02-23T00:48:07.941101+0000","last_clean_scrub_stamp":"2024-02-23T00:51:01.201737+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":30,"num_read_kb":26,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":527,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.253603+0000","last_change":"2024-02-23T00:49:28.347354+0000","last_active":"2024-02-23T00:51:25.253603+0000","last_peered":"2024-02-23T00:51:25.253603+0000","last_clean":"2024-02-23T00:51:25.253603+0000","last_became_active":"2024-02-23T00:39:14.783164+0000","last_became_peered":"2024-02-23T00:39:14.783164+0000","last_unstale":"2024-02-23T00:51:25.253603+0000","last_undegraded":"2024-02-23T00:51:25.253603+0000","last_fullsized":"2024-02-23T00:51:25.253603+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:28.347281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:28.347281+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":525,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.247842+0000","last_change":"2024-02-23T00:49:37.721639+0000","last_active":"2024-02-23T00:51:25.247842+0000","last_peered":"2024-02-23T00:51:25.247842+0000","last_clean":"2024-02-23T00:51:25.247842+0000","last_became_active":"2024-02-23T00:39:18.800297+0000","last_became_peered":"2024-02-23T00:39:18.800297+0000","last_unstale":"2024-02-23T00:51:25.247842+0000","last_undegraded":"2024-02-23T00:51:25.247842+0000","last_fullsized":"2024-02-23T00:51:25.247842+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:37.721563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:37.721563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1","version":"110'7","reported_seq":486,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:26.243292+0000","last_change":"2024-02-23T00:51:13.352686+0000","last_active":"2024-02-23T00:51:26.243292+0000","last_peered":"2024-02-23T00:51:26.243292+0000","last_clean":"2024-02-23T00:51:26.243292+0000","last_became_active":"2024-02-23T00:42:58.777251+0000","last_became_peered":"2024-02-23T00:42:58.777251+0000","last_unstale":"2024-02-23T00:51:26.243292+0000","last_undegraded":"2024-02-23T00:51:26.243292+0000","last_fullsized":"2024-02-23T00:51:26.243292+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'7","last_scrub_stamp":"2024-02-23T00:51:13.352611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:13.352611+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"457'32","reported_seq":228,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:26.242710+0000","last_change":"2024-02-23T00:50:25.079859+0000","last_active":"2024-02-23T00:51:26.242710+0000","last_peered":"2024-02-23T00:51:26.242710+0000","last_clean":"2024-02-23T00:51:26.242710+0000","last_became_active":"2024-02-23T00:47:28.237898+0000","last_became_peered":"2024-02-23T00:47:28.237898+0000","last_unstale":"2024-02-23T00:51:26.242710+0000","last_undegraded":"2024-02-23T00:51:26.242710+0000","last_fullsized":"2024-02-23T00:51:26.242710+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-23T00:50:25.079789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:25.079789+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":52,"num_read_kb":45,"num_write":32,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":534,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.246625+0000","last_change":"2024-02-23T00:51:14.658413+0000","last_active":"2024-02-23T00:51:25.246625+0000","last_peered":"2024-02-23T00:51:25.246625+0000","last_clean":"2024-02-23T00:51:25.246625+0000","last_became_active":"2024-02-23T00:39:14.785590+0000","last_became_peered":"2024-02-23T00:39:14.785590+0000","last_unstale":"2024-02-23T00:51:25.246625+0000","last_undegraded":"2024-02-23T00:51:25.246625+0000","last_fullsized":"2024-02-23T00:51:25.246625+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:14.658297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_clean_scrub_stamp":"2024-02-23T00:51:14.658297+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"336'9","reported_seq":680,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.246208+0000","last_change":"2024-02-23T00:49:39.636503+0000","last_active":"2024-02-23T00:51:25.246208+0000","last_peered":"2024-02-23T00:51:25.246208+0000","last_clean":"2024-02-23T00:51:25.246208+0000","last_became_active":"2024-02-23T00:39:18.803566+0000","last_became_peered":"2024-02-23T00:39:18.803566+0000","last_unstale":"2024-02-23T00:51:25.246208+0000","last_undegraded":"2024-02-23T00:51:25.246208+0000","last_fullsized":"2024-02-23T00:51:25.246208+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"336'9","last_scrub_stamp":"2024-02-23T00:49:39.636409+0000","last_deep_scrub":"336'9","last_deep_scrub_stamp":"2024-02-23T00:46:49.160991+0000","last_clean_scrub_stamp":"2024-02-23T00:49:39.636409+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":466,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.247279+0000","last_change":"2024-02-23T00:48:41.688804+0000","last_active":"2024-02-23T00:51:25.247279+0000","last_peered":"2024-02-23T00:51:25.247279+0000","last_clean":"2024-02-23T00:51:25.247279+0000","last_became_active":"2024-02-23T00:42:58.757424+0000","last_became_peered":"2024-02-23T00:42:58.757424+0000","last_unstale":"2024-02-23T00:51:25.247279+0000","last_undegraded":"2024-02-23T00:51:25.247279+0000","last_fullsized":"2024-02-23T00:51:25.247279+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:41.688728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:41.688728+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"457'21","reported_seq":567,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.246058+0000","last_change":"2024-02-23T00:50:17.861556+0000","last_active":"2024-02-23T00:51:25.246058+0000","last_peered":"2024-02-23T00:51:25.246058+0000","last_clean":"2024-02-23T00:51:25.246058+0000","last_became_active":"2024-02-23T00:39:16.858953+0000","last_became_peered":"2024-02-23T00:39:16.858953+0000","last_unstale":"2024-02-23T00:51:25.246058+0000","last_undegraded":"2024-02-23T00:51:25.246058+0000","last_fullsized":"2024-02-23T00:51:25.246058+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-23T00:50:17.861490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:17.861490+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":529,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.245088+0000","last_change":"2024-02-23T00:48:59.712766+0000","last_active":"2024-02-23T00:51:25.245088+0000","last_peered":"2024-02-23T00:51:25.245088+0000","last_clean":"2024-02-23T00:51:25.245088+0000","last_became_active":"2024-02-23T00:39:14.773772+0000","last_became_peered":"2024-02-23T00:39:14.773772+0000","last_unstale":"2024-02-23T00:51:25.245088+0000","last_undegraded":"2024-02-23T00:51:25.245088+0000","last_fullsized":"2024-02-23T00:51:25.245088+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:59.712699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:48:59.712699+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"376'9","reported_seq":690,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:26.243357+0000","last_change":"2024-02-23T00:49:45.868359+0000","last_active":"2024-02-23T00:51:26.243357+0000","last_peered":"2024-02-23T00:51:26.243357+0000","last_clean":"2024-02-23T00:51:26.243357+0000","last_became_active":"2024-02-23T00:47:28.229316+0000","last_became_peered":"2024-02-23T00:47:28.229316+0000","last_unstale":"2024-02-23T00:51:26.243357+0000","last_undegraded":"2024-02-23T00:51:26.243357+0000","last_fullsized":"2024-02-23T00:51:26.243357+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"376'9","last_scrub_stamp":"2024-02-23T00:49:45.868289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:45.868289+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":462,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:27.115655+0000","last_change":"2024-02-23T00:50:15.714265+0000","last_active":"2024-02-23T00:51:27.115655+0000","last_peered":"2024-02-23T00:51:27.115655+0000","last_clean":"2024-02-23T00:51:27.115655+0000","last_became_active":"2024-02-23T00:42:57.652488+0000","last_became_peered":"2024-02-23T00:42:57.652488+0000","last_unstale":"2024-02-23T00:51:27.115655+0000","last_undegraded":"2024-02-23T00:51:27.115655+0000","last_fullsized":"2024-02-23T00:51:27.115655+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:15.714200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:51.254903+0000","last_clean_scrub_stamp":"2024-02-23T00:50:15.714200+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"457'15","reported_seq":562,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:27.115178+0000","last_change":"2024-02-23T00:49:39.573557+0000","last_active":"2024-02-23T00:51:27.115178+0000","last_peered":"2024-02-23T00:51:27.115178+0000","last_clean":"2024-02-23T00:51:27.115178+0000","last_became_active":"2024-02-23T00:47:28.225585+0000","last_became_peered":"2024-02-23T00:47:28.225585+0000","last_unstale":"2024-02-23T00:51:27.115178+0000","last_undegraded":"2024-02-23T00:51:27.115178+0000","last_fullsized":"2024-02-23T00:51:27.115178+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'15","last_scrub_stamp":"2024-02-23T00:49:39.573472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:39.573472+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":532,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.254869+0000","last_change":"2024-02-23T00:49:02.303864+0000","last_active":"2024-02-23T00:51:25.254869+0000","last_peered":"2024-02-23T00:51:25.254869+0000","last_clean":"2024-02-23T00:51:25.254869+0000","last_became_active":"2024-02-23T00:39:14.778455+0000","last_became_peered":"2024-02-23T00:39:14.778455+0000","last_unstale":"2024-02-23T00:51:25.254869+0000","last_undegraded":"2024-02-23T00:51:25.254869+0000","last_fullsized":"2024-02-23T00:51:25.254869+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:02.303789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:02.303789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":537,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:26.242836+0000","last_change":"2024-02-23T00:50:30.069404+0000","last_active":"2024-02-23T00:51:26.242836+0000","last_peered":"2024-02-23T00:51:26.242836+0000","last_clean":"2024-02-23T00:51:26.242836+0000","last_became_active":"2024-02-23T00:47:44.225379+0000","last_became_peered":"2024-02-23T00:47:44.225379+0000","last_unstale":"2024-02-23T00:51:26.242836+0000","last_undegraded":"2024-02-23T00:51:26.242836+0000","last_fullsized":"2024-02-23T00:51:26.242836+0000","mapping_epoch":391,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":392,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:30.069325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_clean_scrub_stamp":"2024-02-23T00:50:30.069325+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":470,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.245019+0000","last_change":"2024-02-23T00:49:40.770764+0000","last_active":"2024-02-23T00:51:25.245019+0000","last_peered":"2024-02-23T00:51:25.245019+0000","last_clean":"2024-02-23T00:51:25.245019+0000","last_became_active":"2024-02-23T00:42:58.835862+0000","last_became_peered":"2024-02-23T00:42:58.835862+0000","last_unstale":"2024-02-23T00:51:25.245019+0000","last_undegraded":"2024-02-23T00:51:25.245019+0000","last_fullsized":"2024-02-23T00:51:25.245019+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:40.770694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:49:40.770694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"457'19","reported_seq":555,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.242008+0000","last_change":"2024-02-23T00:50:29.599563+0000","last_active":"2024-02-23T00:51:25.242008+0000","last_peered":"2024-02-23T00:51:25.242008+0000","last_clean":"2024-02-23T00:51:25.242008+0000","last_became_active":"2024-02-23T00:39:16.783896+0000","last_became_peered":"2024-02-23T00:39:16.783896+0000","last_unstale":"2024-02-23T00:51:25.242008+0000","last_undegraded":"2024-02-23T00:51:25.242008+0000","last_fullsized":"2024-02-23T00:51:25.242008+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'19","last_scrub_stamp":"2024-02-23T00:50:29.599483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:29.599483+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"202'2","reported_seq":528,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:27.116750+0000","last_change":"2024-02-23T00:48:04.832975+0000","last_active":"2024-02-23T00:51:27.116750+0000","last_peered":"2024-02-23T00:51:27.116750+0000","last_clean":"2024-02-23T00:51:27.116750+0000","last_became_active":"2024-02-23T00:39:14.775804+0000","last_became_peered":"2024-02-23T00:39:14.775804+0000","last_unstale":"2024-02-23T00:51:27.116750+0000","last_undegraded":"2024-02-23T00:51:27.116750+0000","last_fullsized":"2024-02-23T00:51:27.116750+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"202'2","last_scrub_stamp":"2024-02-23T00:48:04.832904+0000","last_deep_scrub":"202'2","last_deep_scrub_stamp":"2024-02-23T00:44:34.720640+0000","last_clean_scrub_stamp":"2024-02-23T00:48:04.832904+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":516,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:27.114538+0000","last_change":"2024-02-23T00:50:18.698805+0000","last_active":"2024-02-23T00:51:27.114538+0000","last_peered":"2024-02-23T00:51:27.114538+0000","last_clean":"2024-02-23T00:51:27.114538+0000","last_became_active":"2024-02-23T00:39:18.800382+0000","last_became_peered":"2024-02-23T00:39:18.800382+0000","last_unstale":"2024-02-23T00:51:27.114538+0000","last_undegraded":"2024-02-23T00:51:27.114538+0000","last_fullsized":"2024-02-23T00:51:27.114538+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:18.698687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:50:18.698687+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":471,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:27.116663+0000","last_change":"2024-02-23T00:49:46.462085+0000","last_active":"2024-02-23T00:51:27.116663+0000","last_peered":"2024-02-23T00:51:27.116663+0000","last_clean":"2024-02-23T00:51:27.116663+0000","last_became_active":"2024-02-23T00:47:28.224961+0000","last_became_peered":"2024-02-23T00:47:28.224961+0000","last_unstale":"2024-02-23T00:51:27.116663+0000","last_undegraded":"2024-02-23T00:51:27.116663+0000","last_fullsized":"2024-02-23T00:51:27.116663+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:46.462015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:46.462015+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.462015+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"457'8","reported_seq":542,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:26.242244+0000","last_change":"2024-02-23T00:50:35.045697+0000","last_active":"2024-02-23T00:51:26.242244+0000","last_peered":"2024-02-23T00:51:26.242244+0000","last_clean":"2024-02-23T00:51:26.242244+0000","last_became_active":"2024-02-23T00:39:16.785830+0000","last_became_peered":"2024-02-23T00:39:16.785830+0000","last_unstale":"2024-02-23T00:51:26.242244+0000","last_undegraded":"2024-02-23T00:51:26.242244+0000","last_fullsized":"2024-02-23T00:51:26.242244+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'8","last_scrub_stamp":"2024-02-23T00:50:35.045612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:35.045612+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":541,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.247734+0000","last_change":"2024-02-23T00:49:26.672761+0000","last_active":"2024-02-23T00:51:25.247734+0000","last_peered":"2024-02-23T00:51:25.247734+0000","last_clean":"2024-02-23T00:51:25.247734+0000","last_became_active":"2024-02-23T00:47:28.224477+0000","last_became_peered":"2024-02-23T00:47:28.224477+0000","last_unstale":"2024-02-23T00:51:25.247734+0000","last_undegraded":"2024-02-23T00:51:25.247734+0000","last_fullsized":"2024-02-23T00:51:25.247734+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:26.672673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:26.672673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":537,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.253459+0000","last_change":"2024-02-23T00:49:33.461971+0000","last_active":"2024-02-23T00:51:25.253459+0000","last_peered":"2024-02-23T00:51:25.253459+0000","last_clean":"2024-02-23T00:51:25.253459+0000","last_became_active":"2024-02-23T00:47:44.230295+0000","last_became_peered":"2024-02-23T00:47:44.230295+0000","last_unstale":"2024-02-23T00:51:25.253459+0000","last_undegraded":"2024-02-23T00:51:25.253459+0000","last_fullsized":"2024-02-23T00:51:25.253459+0000","mapping_epoch":391,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":392,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:33.461907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:33.461907+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":466,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.248640+0000","last_change":"2024-02-23T00:48:51.717473+0000","last_active":"2024-02-23T00:51:25.248640+0000","last_peered":"2024-02-23T00:51:25.248640+0000","last_clean":"2024-02-23T00:51:25.248640+0000","last_became_active":"2024-02-23T00:42:58.741365+0000","last_became_peered":"2024-02-23T00:42:58.741365+0000","last_unstale":"2024-02-23T00:51:25.248640+0000","last_undegraded":"2024-02-23T00:51:25.248640+0000","last_fullsized":"2024-02-23T00:51:25.248640+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:51.717397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:51.717397+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"457'25","reported_seq":576,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:26.243507+0000","last_change":"2024-02-23T00:49:47.788110+0000","last_active":"2024-02-23T00:51:26.243507+0000","last_peered":"2024-02-23T00:51:26.243507+0000","last_clean":"2024-02-23T00:51:26.243507+0000","last_became_active":"2024-02-23T00:39:16.857232+0000","last_became_peered":"2024-02-23T00:39:16.857232+0000","last_unstale":"2024-02-23T00:51:26.243507+0000","last_undegraded":"2024-02-23T00:51:26.243507+0000","last_fullsized":"2024-02-23T00:51:26.243507+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-23T00:49:47.788044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:47.788044+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":536,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.255022+0000","last_change":"2024-02-23T00:49:05.329264+0000","last_active":"2024-02-23T00:51:25.255022+0000","last_peered":"2024-02-23T00:51:25.255022+0000","last_clean":"2024-02-23T00:51:25.255022+0000","last_became_active":"2024-02-23T00:47:28.240162+0000","last_became_peered":"2024-02-23T00:47:28.240162+0000","last_unstale":"2024-02-23T00:51:25.255022+0000","last_undegraded":"2024-02-23T00:51:25.255022+0000","last_fullsized":"2024-02-23T00:51:25.255022+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:05.329178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:44.285083+0000","last_clean_scrub_stamp":"2024-02-23T00:49:05.329178+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":181,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:26.242055+0000","last_change":"2024-02-23T00:49:42.872820+0000","last_active":"2024-02-23T00:51:26.242055+0000","last_peered":"2024-02-23T00:51:26.242055+0000","last_clean":"2024-02-23T00:51:26.242055+0000","last_became_active":"2024-02-23T00:47:44.226097+0000","last_became_peered":"2024-02-23T00:47:44.226097+0000","last_unstale":"2024-02-23T00:51:26.242055+0000","last_undegraded":"2024-02-23T00:51:26.242055+0000","last_fullsized":"2024-02-23T00:51:26.242055+0000","mapping_epoch":391,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":392,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:42.872748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:42.872748+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":203,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.248354+0000","last_change":"2024-02-23T00:50:28.678155+0000","last_active":"2024-02-23T00:51:25.248354+0000","last_peered":"2024-02-23T00:51:25.248354+0000","last_clean":"2024-02-23T00:51:25.248354+0000","last_became_active":"2024-02-23T00:47:28.200688+0000","last_became_peered":"2024-02-23T00:47:28.200688+0000","last_unstale":"2024-02-23T00:51:25.248354+0000","last_undegraded":"2024-02-23T00:51:25.248354+0000","last_fullsized":"2024-02-23T00:51:25.248354+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:28.678041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:28.678041+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"457'26","reported_seq":572,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.245801+0000","last_change":"2024-02-23T00:50:23.540361+0000","last_active":"2024-02-23T00:51:25.245801+0000","last_peered":"2024-02-23T00:51:25.245801+0000","last_clean":"2024-02-23T00:51:25.245801+0000","last_became_active":"2024-02-23T00:39:16.858945+0000","last_became_peered":"2024-02-23T00:39:16.858945+0000","last_unstale":"2024-02-23T00:51:25.245801+0000","last_undegraded":"2024-02-23T00:51:25.245801+0000","last_fullsized":"2024-02-23T00:51:25.245801+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-23T00:50:23.540232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:23.540232+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":36,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":541,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.244897+0000","last_change":"2024-02-23T00:51:09.827935+0000","last_active":"2024-02-23T00:51:25.244897+0000","last_peered":"2024-02-23T00:51:25.244897+0000","last_clean":"2024-02-23T00:51:25.244897+0000","last_became_active":"2024-02-23T00:39:14.785363+0000","last_became_peered":"2024-02-23T00:39:14.785363+0000","last_unstale":"2024-02-23T00:51:25.244897+0000","last_undegraded":"2024-02-23T00:51:25.244897+0000","last_fullsized":"2024-02-23T00:51:25.244897+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"80'2","last_scrub_stamp":"2024-02-23T00:51:09.827857+0000","last_deep_scrub":"80'2","last_deep_scrub_stamp":"2024-02-23T00:40:15.942725+0000","last_clean_scrub_stamp":"2024-02-23T00:51:09.827857+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"93'9","reported_seq":681,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.255314+0000","last_change":"2024-02-23T00:48:53.246622+0000","last_active":"2024-02-23T00:51:25.255314+0000","last_peered":"2024-02-23T00:51:25.255314+0000","last_clean":"2024-02-23T00:51:25.255314+0000","last_became_active":"2024-02-23T00:39:18.796680+0000","last_became_peered":"2024-02-23T00:39:18.796680+0000","last_unstale":"2024-02-23T00:51:25.255314+0000","last_undegraded":"2024-02-23T00:51:25.255314+0000","last_fullsized":"2024-02-23T00:51:25.255314+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-02-23T00:48:53.246535+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-02-23T00:43:50.393713+0000","last_clean_scrub_stamp":"2024-02-23T00:48:53.246535+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":479,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.249217+0000","last_change":"2024-02-23T00:51:12.493007+0000","last_active":"2024-02-23T00:51:25.249217+0000","last_peered":"2024-02-23T00:51:25.249217+0000","last_clean":"2024-02-23T00:51:25.249217+0000","last_became_active":"2024-02-23T00:47:28.234641+0000","last_became_peered":"2024-02-23T00:47:28.234641+0000","last_unstale":"2024-02-23T00:51:25.249217+0000","last_undegraded":"2024-02-23T00:51:25.249217+0000","last_fullsized":"2024-02-23T00:51:25.249217+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.492942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.492942+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"457'31","reported_seq":587,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.255410+0000","last_change":"2024-02-23T00:51:15.408847+0000","last_active":"2024-02-23T00:51:25.255410+0000","last_peered":"2024-02-23T00:51:25.255410+0000","last_clean":"2024-02-23T00:51:25.255410+0000","last_became_active":"2024-02-23T00:39:16.782660+0000","last_became_peered":"2024-02-23T00:39:16.782660+0000","last_unstale":"2024-02-23T00:51:25.255410+0000","last_undegraded":"2024-02-23T00:51:25.255410+0000","last_fullsized":"2024-02-23T00:51:25.255410+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:51:15.408782+0000","last_deep_scrub":"93'23","last_deep_scrub_stamp":"2024-02-23T00:41:35.954516+0000","last_clean_scrub_stamp":"2024-02-23T00:51:15.408782+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":532,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:26.242309+0000","last_change":"2024-02-23T00:50:29.029794+0000","last_active":"2024-02-23T00:51:26.242309+0000","last_peered":"2024-02-23T00:51:26.242309+0000","last_clean":"2024-02-23T00:51:26.242309+0000","last_became_active":"2024-02-23T00:39:14.781560+0000","last_became_peered":"2024-02-23T00:39:14.781560+0000","last_unstale":"2024-02-23T00:51:26.242309+0000","last_undegraded":"2024-02-23T00:51:26.242309+0000","last_fullsized":"2024-02-23T00:51:26.242309+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:29.029725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:29.029725+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":539,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.245875+0000","last_change":"2024-02-23T00:51:14.488316+0000","last_active":"2024-02-23T00:51:25.245875+0000","last_peered":"2024-02-23T00:51:25.245875+0000","last_clean":"2024-02-23T00:51:25.245875+0000","last_became_active":"2024-02-23T00:47:28.232797+0000","last_became_peered":"2024-02-23T00:47:28.232797+0000","last_unstale":"2024-02-23T00:51:25.245875+0000","last_undegraded":"2024-02-23T00:51:25.245875+0000","last_fullsized":"2024-02-23T00:51:25.245875+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:14.488205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:53.189881+0000","last_clean_scrub_stamp":"2024-02-23T00:51:14.488205+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":473,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:26.243568+0000","last_change":"2024-02-23T00:51:00.232343+0000","last_active":"2024-02-23T00:51:26.243568+0000","last_peered":"2024-02-23T00:51:26.243568+0000","last_clean":"2024-02-23T00:51:26.243568+0000","last_became_active":"2024-02-23T00:42:58.774720+0000","last_became_peered":"2024-02-23T00:42:58.774720+0000","last_unstale":"2024-02-23T00:51:26.243568+0000","last_undegraded":"2024-02-23T00:51:26.243568+0000","last_fullsized":"2024-02-23T00:51:26.243568+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.232261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.232261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"457'27","reported_seq":266,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:27.117302+0000","last_change":"2024-02-23T00:49:26.518048+0000","last_active":"2024-02-23T00:51:27.117302+0000","last_peered":"2024-02-23T00:51:27.117302+0000","last_clean":"2024-02-23T00:51:27.117302+0000","last_became_active":"2024-02-23T00:47:28.211342+0000","last_became_peered":"2024-02-23T00:47:28.211342+0000","last_unstale":"2024-02-23T00:51:27.117302+0000","last_undegraded":"2024-02-23T00:51:27.117302+0000","last_fullsized":"2024-02-23T00:51:27.117302+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-23T00:49:26.517965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:26.517965+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":532,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.259481+0000","last_change":"2024-02-23T00:51:08.235807+0000","last_active":"2024-02-23T00:51:25.259481+0000","last_peered":"2024-02-23T00:51:25.259481+0000","last_clean":"2024-02-23T00:51:25.259481+0000","last_became_active":"2024-02-23T00:39:14.782494+0000","last_became_peered":"2024-02-23T00:39:14.782494+0000","last_unstale":"2024-02-23T00:51:25.259481+0000","last_undegraded":"2024-02-23T00:51:25.259481+0000","last_fullsized":"2024-02-23T00:51:25.259481+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:08.235735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:08.235735+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":516,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:27.114320+0000","last_change":"2024-02-23T00:48:16.788875+0000","last_active":"2024-02-23T00:51:27.114320+0000","last_peered":"2024-02-23T00:51:27.114320+0000","last_clean":"2024-02-23T00:51:27.114320+0000","last_became_active":"2024-02-23T00:39:18.801039+0000","last_became_peered":"2024-02-23T00:39:18.801039+0000","last_unstale":"2024-02-23T00:51:27.114320+0000","last_undegraded":"2024-02-23T00:51:27.114320+0000","last_fullsized":"2024-02-23T00:51:27.114320+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:16.788806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.641689+0000","last_clean_scrub_stamp":"2024-02-23T00:48:16.788806+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":473,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:26.242896+0000","last_change":"2024-02-23T00:51:10.341831+0000","last_active":"2024-02-23T00:51:26.242896+0000","last_peered":"2024-02-23T00:51:26.242896+0000","last_clean":"2024-02-23T00:51:26.242896+0000","last_became_active":"2024-02-23T00:42:58.770759+0000","last_became_peered":"2024-02-23T00:42:58.770759+0000","last_unstale":"2024-02-23T00:51:26.242896+0000","last_undegraded":"2024-02-23T00:51:26.242896+0000","last_fullsized":"2024-02-23T00:51:26.242896+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:10.341768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:10.341768+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"457'24","reported_seq":577,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:26.243196+0000","last_change":"2024-02-23T00:50:13.861288+0000","last_active":"2024-02-23T00:51:26.243196+0000","last_peered":"2024-02-23T00:51:26.243196+0000","last_clean":"2024-02-23T00:51:26.243196+0000","last_became_active":"2024-02-23T00:39:16.782803+0000","last_became_peered":"2024-02-23T00:39:16.782803+0000","last_unstale":"2024-02-23T00:51:26.243196+0000","last_undegraded":"2024-02-23T00:51:26.243196+0000","last_fullsized":"2024-02-23T00:51:26.243196+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'24","last_scrub_stamp":"2024-02-23T00:50:13.861220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:13.861220+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":46,"num_read_kb":36,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":532,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:26.242814+0000","last_change":"2024-02-23T00:50:28.060186+0000","last_active":"2024-02-23T00:51:26.242814+0000","last_peered":"2024-02-23T00:51:26.242814+0000","last_clean":"2024-02-23T00:51:26.242814+0000","last_became_active":"2024-02-23T00:39:14.782163+0000","last_became_peered":"2024-02-23T00:39:14.782163+0000","last_unstale":"2024-02-23T00:51:26.242814+0000","last_undegraded":"2024-02-23T00:51:26.242814+0000","last_fullsized":"2024-02-23T00:51:26.242814+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:28.060109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:28.060109+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":531,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.242137+0000","last_change":"2024-02-23T00:50:18.629287+0000","last_active":"2024-02-23T00:51:25.242137+0000","last_peered":"2024-02-23T00:51:25.242137+0000","last_clean":"2024-02-23T00:51:25.242137+0000","last_became_active":"2024-02-23T00:47:28.215452+0000","last_became_peered":"2024-02-23T00:47:28.215452+0000","last_unstale":"2024-02-23T00:51:25.242137+0000","last_undegraded":"2024-02-23T00:51:25.242137+0000","last_fullsized":"2024-02-23T00:51:25.242137+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:18.629203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_clean_scrub_stamp":"2024-02-23T00:50:18.629203+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":479,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.245783+0000","last_change":"2024-02-23T00:50:56.774466+0000","last_active":"2024-02-23T00:51:25.245783+0000","last_peered":"2024-02-23T00:51:25.245783+0000","last_clean":"2024-02-23T00:51:25.245783+0000","last_became_active":"2024-02-23T00:47:28.203555+0000","last_became_peered":"2024-02-23T00:47:28.203555+0000","last_unstale":"2024-02-23T00:51:25.245783+0000","last_undegraded":"2024-02-23T00:51:25.245783+0000","last_fullsized":"2024-02-23T00:51:25.245783+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:56.774382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:56.774382+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"457'33","reported_seq":587,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.242296+0000","last_change":"2024-02-23T00:49:46.364498+0000","last_active":"2024-02-23T00:51:25.242296+0000","last_peered":"2024-02-23T00:51:25.242296+0000","last_clean":"2024-02-23T00:51:25.242296+0000","last_became_active":"2024-02-23T00:39:16.786791+0000","last_became_peered":"2024-02-23T00:39:16.786791+0000","last_unstale":"2024-02-23T00:51:25.242296+0000","last_undegraded":"2024-02-23T00:51:25.242296+0000","last_fullsized":"2024-02-23T00:51:25.242296+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'33","last_scrub_stamp":"2024-02-23T00:49:46.364418+0000","last_deep_scrub":"95'27","last_deep_scrub_stamp":"2024-02-23T00:41:55.919895+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.364418+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":537,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.246571+0000","last_change":"2024-02-23T00:49:27.403374+0000","last_active":"2024-02-23T00:51:25.246571+0000","last_peered":"2024-02-23T00:51:25.246571+0000","last_clean":"2024-02-23T00:51:25.246571+0000","last_became_active":"2024-02-23T00:47:28.208293+0000","last_became_peered":"2024-02-23T00:47:28.208293+0000","last_unstale":"2024-02-23T00:51:25.246571+0000","last_undegraded":"2024-02-23T00:51:25.246571+0000","last_fullsized":"2024-02-23T00:51:25.246571+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:27.403305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:27.403305+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":525,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.245644+0000","last_change":"2024-02-23T00:49:46.612538+0000","last_active":"2024-02-23T00:51:25.245644+0000","last_peered":"2024-02-23T00:51:25.245644+0000","last_clean":"2024-02-23T00:51:25.245644+0000","last_became_active":"2024-02-23T00:39:18.796431+0000","last_became_peered":"2024-02-23T00:39:18.796431+0000","last_unstale":"2024-02-23T00:51:25.245644+0000","last_undegraded":"2024-02-23T00:51:25.245644+0000","last_fullsized":"2024-02-23T00:51:25.245644+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:46.612461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.612461+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":470,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.247443+0000","last_change":"2024-02-23T00:50:26.630228+0000","last_active":"2024-02-23T00:51:25.247443+0000","last_peered":"2024-02-23T00:51:25.247443+0000","last_clean":"2024-02-23T00:51:25.247443+0000","last_became_active":"2024-02-23T00:42:58.832885+0000","last_became_peered":"2024-02-23T00:42:58.832885+0000","last_unstale":"2024-02-23T00:51:25.247443+0000","last_undegraded":"2024-02-23T00:51:25.247443+0000","last_fullsized":"2024-02-23T00:51:25.247443+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:26.630088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:26.630088+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"457'17","reported_seq":556,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.242157+0000","last_change":"2024-02-23T00:50:15.590233+0000","last_active":"2024-02-23T00:51:25.242157+0000","last_peered":"2024-02-23T00:51:25.242157+0000","last_clean":"2024-02-23T00:51:25.242157+0000","last_became_active":"2024-02-23T00:39:16.782762+0000","last_became_peered":"2024-02-23T00:39:16.782762+0000","last_unstale":"2024-02-23T00:51:25.242157+0000","last_undegraded":"2024-02-23T00:51:25.242157+0000","last_fullsized":"2024-02-23T00:51:25.242157+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'17","last_scrub_stamp":"2024-02-23T00:50:15.590096+0000","last_deep_scrub":"82'15","last_deep_scrub_stamp":"2024-02-23T00:41:50.975683+0000","last_clean_scrub_stamp":"2024-02-23T00:50:15.590096+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":21,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":533,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.246663+0000","last_change":"2024-02-23T00:51:08.199826+0000","last_active":"2024-02-23T00:51:25.246663+0000","last_peered":"2024-02-23T00:51:25.246663+0000","last_clean":"2024-02-23T00:51:25.246663+0000","last_became_active":"2024-02-23T00:39:14.783469+0000","last_became_peered":"2024-02-23T00:39:14.783469+0000","last_unstale":"2024-02-23T00:51:25.246663+0000","last_undegraded":"2024-02-23T00:51:25.246663+0000","last_fullsized":"2024-02-23T00:51:25.246663+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:08.199749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:08.199749+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":531,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.246360+0000","last_change":"2024-02-23T00:51:13.264970+0000","last_active":"2024-02-23T00:51:25.246360+0000","last_peered":"2024-02-23T00:51:25.246360+0000","last_clean":"2024-02-23T00:51:25.246360+0000","last_became_active":"2024-02-23T00:39:18.796952+0000","last_became_peered":"2024-02-23T00:39:18.796952+0000","last_unstale":"2024-02-23T00:51:25.246360+0000","last_undegraded":"2024-02-23T00:51:25.246360+0000","last_fullsized":"2024-02-23T00:51:25.246360+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:13.264868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:06.491793+0000","last_clean_scrub_stamp":"2024-02-23T00:51:13.264868+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":470,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.241427+0000","last_change":"2024-02-23T00:51:03.460376+0000","last_active":"2024-02-23T00:51:25.241427+0000","last_peered":"2024-02-23T00:51:25.241427+0000","last_clean":"2024-02-23T00:51:25.241427+0000","last_became_active":"2024-02-23T00:42:58.758147+0000","last_became_peered":"2024-02-23T00:42:58.758147+0000","last_unstale":"2024-02-23T00:51:25.241427+0000","last_undegraded":"2024-02-23T00:51:25.241427+0000","last_fullsized":"2024-02-23T00:51:25.241427+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:03.460302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:23.224663+0000","last_clean_scrub_stamp":"2024-02-23T00:51:03.460302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"457'21","reported_seq":564,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.241371+0000","last_change":"2024-02-23T00:49:32.434383+0000","last_active":"2024-02-23T00:51:25.241371+0000","last_peered":"2024-02-23T00:51:25.241371+0000","last_clean":"2024-02-23T00:51:25.241371+0000","last_became_active":"2024-02-23T00:39:16.784606+0000","last_became_peered":"2024-02-23T00:39:16.784606+0000","last_unstale":"2024-02-23T00:51:25.241371+0000","last_undegraded":"2024-02-23T00:51:25.241371+0000","last_fullsized":"2024-02-23T00:51:25.241371+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-23T00:49:32.434282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:32.434282+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":532,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:26.243249+0000","last_change":"2024-02-23T00:50:19.967076+0000","last_active":"2024-02-23T00:51:26.243249+0000","last_peered":"2024-02-23T00:51:26.243249+0000","last_clean":"2024-02-23T00:51:26.243249+0000","last_became_active":"2024-02-23T00:39:14.785362+0000","last_became_peered":"2024-02-23T00:39:14.785362+0000","last_unstale":"2024-02-23T00:51:26.243249+0000","last_undegraded":"2024-02-23T00:51:26.243249+0000","last_fullsized":"2024-02-23T00:51:26.243249+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:19.967009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:19.967009+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":516,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:27.115390+0000","last_change":"2024-02-23T00:49:37.537279+0000","last_active":"2024-02-23T00:51:27.115390+0000","last_peered":"2024-02-23T00:51:27.115390+0000","last_clean":"2024-02-23T00:51:27.115390+0000","last_became_active":"2024-02-23T00:39:18.796312+0000","last_became_peered":"2024-02-23T00:39:18.796312+0000","last_unstale":"2024-02-23T00:51:27.115390+0000","last_undegraded":"2024-02-23T00:51:27.115390+0000","last_fullsized":"2024-02-23T00:51:27.115390+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:37.537211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:47.226331+0000","last_clean_scrub_stamp":"2024-02-23T00:49:37.537211+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":470,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.245396+0000","last_change":"2024-02-23T00:50:28.479578+0000","last_active":"2024-02-23T00:51:25.245396+0000","last_peered":"2024-02-23T00:51:25.245396+0000","last_clean":"2024-02-23T00:51:25.245396+0000","last_became_active":"2024-02-23T00:42:58.838257+0000","last_became_peered":"2024-02-23T00:42:58.838257+0000","last_unstale":"2024-02-23T00:51:25.245396+0000","last_undegraded":"2024-02-23T00:51:25.245396+0000","last_fullsized":"2024-02-23T00:51:25.245396+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:28.479512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:50:28.479512+0000","last_clean_scrub_stamp":"2024-02-23T00:50:28.479512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"457'62","reported_seq":677,"reported_epoch":557,"state":"active+clean","last_fresh":"2024-02-23T00:51:26.243569+0000","last_change":"2024-02-23T00:50:58.151569+0000","last_active":"2024-02-23T00:51:26.243569+0000","last_peered":"2024-02-23T00:51:26.243569+0000","last_clean":"2024-02-23T00:51:26.243569+0000","last_became_active":"2024-02-23T00:47:28.205150+0000","last_became_peered":"2024-02-23T00:47:28.205150+0000","last_unstale":"2024-02-23T00:51:26.243569+0000","last_undegraded":"2024-02-23T00:51:26.243569+0000","last_fullsized":"2024-02-23T00:51:26.243569+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'62","last_scrub_stamp":"2024-02-23T00:50:58.151494+0000","last_deep_scrub":"88'50","last_deep_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_clean_scrub_stamp":"2024-02-23T00:50:58.151494+0000","log_size":62,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":92,"num_read_kb":79,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":529,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.245688+0000","last_change":"2024-02-23T00:50:34.600693+0000","last_active":"2024-02-23T00:51:25.245688+0000","last_peered":"2024-02-23T00:51:25.245688+0000","last_clean":"2024-02-23T00:51:25.245688+0000","last_became_active":"2024-02-23T00:39:14.782000+0000","last_became_peered":"2024-02-23T00:39:14.782000+0000","last_unstale":"2024-02-23T00:51:25.245688+0000","last_undegraded":"2024-02-23T00:51:25.245688+0000","last_fullsized":"2024-02-23T00:51:25.245688+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:34.600592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_clean_scrub_stamp":"2024-02-23T00:50:34.600592+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":517,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.241647+0000","last_change":"2024-02-23T00:49:27.459981+0000","last_active":"2024-02-23T00:51:25.241647+0000","last_peered":"2024-02-23T00:51:25.241647+0000","last_clean":"2024-02-23T00:51:25.241647+0000","last_became_active":"2024-02-23T00:39:18.799723+0000","last_became_peered":"2024-02-23T00:39:18.799723+0000","last_unstale":"2024-02-23T00:51:25.241647+0000","last_undegraded":"2024-02-23T00:51:25.241647+0000","last_fullsized":"2024-02-23T00:51:25.241647+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:27.459893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:27.459893+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":470,"reported_epoch":556,"state":"active+clean","last_fresh":"2024-02-23T00:51:25.248209+0000","last_change":"2024-02-23T00:49:44.601162+0000","last_active":"2024-02-23T00:51:25.248209+0000","last_peered":"2024-02-23T00:51:25.248209+0000","last_clean":"2024-02-23T00:51:25.248209+0000","last_became_active":"2024-02-23T00:42:58.824573+0000","last_became_peered":"2024-02-23T00:42:58.824573+0000","last_unstale":"2024-02-23T00:51:25.248209+0000","last_undegraded":"2024-02-23T00:51:25.248209+0000","last_fullsized":"2024-02-23T00:51:25.248209+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:44.601086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:44.601086+0000","last_clean_scrub_stamp":"2024-02-23T00:49:44.601086+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0],"acting":[5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":62,"num_pg":32,"stat_sum":{"num_bytes":4020435,"num_objects":193,"num_object_clones":11,"num_object_copies":579,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":193,"num_whiteouts":0,"num_read":213,"num_read_kb":57,"num_write":779,"num_write_kb":4619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":853,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":21,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":14331904,"data_stored":12061457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2088,"ondisk_log_size":2088,"up":96,"acting":96,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":12,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":36,"acting":36,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1508,"num_read_kb":1233,"num_write":895,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":1486,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":955,"ondisk_log_size":955,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":722,"num_read_kb":717,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234134,"num_pgs":91,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":316300,"kb_used_data":6788,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93428852,"statfs":{"total":95995035648,"available":95671144448,"internally_reserved":0,"allocated":6950912,"data_stored":5035140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:50:32 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51500000000000001,"15min":0.53500000000000003},"min":{"1min":0.29499999999999998,"5min":0.27300000000000002,"15min":0.26900000000000002},"max":{"1min":0.64100000000000001,"5min":0.76400000000000001,"15min":4.9420000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.376,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.78400000000000003,"5min":1.151,"15min":4.9729999999999999},"last":0.45100000000000001}]},{"osd":1,"last update":"Fri Feb 23 00:50:32 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.52500000000000002,"15min":0.61699999999999999},"min":{"1min":0.30599999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.79700000000000004,"5min":1.25,"15min":10.786},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52300000000000002,"15min":0.62},"min":{"1min":0.39100000000000001,"5min":0.317,"15min":0.28100000000000003},"max":{"1min":0.78400000000000003,"5min":1.107,"15min":11.885999999999999},"last":0.56899999999999995}]},{"osd":2,"last update":"Fri Feb 23 00:50:32 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.52700000000000002,"15min":0.55400000000000005},"min":{"1min":0.377,"5min":0.36099999999999999,"15min":0.28999999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":5.0789999999999997},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.53000000000000003,"15min":0.54600000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.316},"max":{"1min":0.66900000000000004,"5min":1.1180000000000001,"15min":5.1280000000000001},"last":0.59099999999999997}]},{"osd":3,"last update":"Fri Feb 23 00:50:32 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52300000000000002,"15min":0.61899999999999999},"min":{"1min":0.35399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.75800000000000001,"5min":1.095,"15min":15.712999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.55800000000000005,"15min":0.61499999999999999},"min":{"1min":0.40200000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.76800000000000002,"5min":1.1659999999999999,"15min":9.4090000000000007},"last":0.64700000000000002}]},{"osd":4,"last update":"Fri Feb 23 00:50:32 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.439,"15min":0.439},"min":{"1min":0.31,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.63600000000000001,"5min":1.6819999999999999,"15min":2.0350000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.46100000000000002,"15min":0.48799999999999999},"min":{"1min":0.32000000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.63600000000000001,"5min":0.86699999999999999,"15min":11.236000000000001},"last":0.58199999999999996}]},{"osd":5,"last update":"Fri Feb 23 00:50:32 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48099999999999998,"15min":0.49199999999999999},"min":{"1min":0.32300000000000001,"5min":0.27700000000000002,"15min":0.27000000000000002},"max":{"1min":0.72299999999999998,"5min":0.73199999999999998,"15min":5.4000000000000004},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.50900000000000001,"15min":0.497},"min":{"1min":0.35199999999999998,"5min":0.29099999999999998,"15min":0.26700000000000002},"max":{"1min":0.69499999999999995,"5min":1.728,"15min":1.728},"last":0.624}]},{"osd":6,"last update":"Fri Feb 23 00:50:32 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51500000000000001,"15min":0.51000000000000001},"min":{"1min":0.36099999999999999,"5min":0.32200000000000001,"15min":0.317},"max":{"1min":0.67800000000000005,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52000000000000002,"15min":0.54800000000000004},"min":{"1min":0.34000000000000002,"5min":0.32700000000000001,"15min":0.32400000000000001},"max":{"1min":0.70499999999999996,"5min":1.2370000000000001,"15min":13.395},"last":0.60799999999999998}]}]},{"osd":6,"up_from":47,"seq":201863463069,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308196,"kb_used_data":2780,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93436956,"statfs":{"total":95995035648,"available":95679442944,"internally_reserved":0,"allocated":2846720,"data_stored":1029942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:51:07 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.505,"15min":0.68200000000000005},"min":{"1min":0.22500000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":30.641999999999999},"last":0.437},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.53600000000000003,"15min":0.69699999999999995},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.577,"5min":3.7360000000000002,"15min":29.292000000000002},"last":0.48199999999999998}]},{"osd":1,"last update":"Fri Feb 23 00:51:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51500000000000001,"15min":0.68300000000000005},"min":{"1min":0.308,"5min":0.247,"15min":0.247},"max":{"1min":0.77100000000000002,"5min":0.77200000000000002,"15min":28.186},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.52900000000000003,"15min":0.86299999999999999},"min":{"1min":0.39400000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.752,"5min":1.083,"15min":33.104999999999997},"last":0.44900000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:51:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52700000000000002,"15min":0.71399999999999997},"min":{"1min":0.36699999999999999,"5min":0.29399999999999998,"15min":0.27400000000000002},"max":{"1min":0.69599999999999995,"5min":1.1160000000000001,"15min":19.960000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53400000000000003,"15min":0.64600000000000002},"min":{"1min":0.40400000000000003,"5min":0.316,"15min":0.29899999999999999},"max":{"1min":0.79000000000000004,"5min":1.665,"15min":8.5860000000000003},"last":0.49099999999999999}]},{"osd":3,"last update":"Fri Feb 23 00:51:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54300000000000004,"15min":0.626},"min":{"1min":0.40999999999999998,"5min":0.29199999999999998,"15min":0.28599999999999998},"max":{"1min":0.78000000000000003,"5min":0.78400000000000003,"15min":19.562000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56899999999999995,"15min":0.67000000000000004},"min":{"1min":0.42399999999999999,"5min":0.33500000000000002,"15min":0.33000000000000002},"max":{"1min":0.80700000000000005,"5min":2.6259999999999999,"15min":14.199},"last":0.41199999999999998}]},{"osd":4,"last update":"Fri Feb 23 00:51:07 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.44700000000000001,"15min":0.47999999999999998},"min":{"1min":0.317,"5min":0.245,"15min":0.218},"max":{"1min":0.61499999999999999,"5min":1.6220000000000001,"15min":11.109999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.45900000000000002,"15min":0.54800000000000004},"min":{"1min":0.32700000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.751,"5min":0.83099999999999996,"15min":16.734000000000002},"last":0.32000000000000001}]},{"osd":5,"last update":"Fri Feb 23 00:51:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48299999999999998,"15min":0.51300000000000001},"min":{"1min":0.34499999999999997,"5min":0.308,"15min":0.249},"max":{"1min":0.72399999999999998,"5min":1.784,"15min":14.154999999999999},"last":0.378},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.58499999999999996,"15min":0.55500000000000005},"min":{"1min":0.37,"5min":0.318,"15min":0.26400000000000001},"max":{"1min":0.68500000000000005,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.501}]},{"osd":7,"last update":"Fri Feb 23 00:51:23 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.51000000000000001,"15min":0.53100000000000003},"min":{"1min":0.38600000000000001,"5min":0.29399999999999998,"15min":0.26800000000000002},"max":{"1min":0.63100000000000001,"5min":1.726,"15min":8.548},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53900000000000003,"15min":0.60699999999999998},"min":{"1min":0.40300000000000002,"5min":0.34200000000000003,"15min":0.309},"max":{"1min":0.71199999999999997,"5min":0.88500000000000001,"15min":15.618},"last":0.40300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575031,"num_pgs":96,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":312104,"kb_used_data":6688,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93433048,"statfs":{"total":95995035648,"available":95675441152,"internally_reserved":0,"allocated":6848512,"data_stored":5030680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.39000000000000001,"15min":0.68400000000000005},"min":{"1min":0.23799999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.56999999999999995,"5min":6.202,"15min":78.361000000000004},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.379,"5min":0.36899999999999999,"15min":0.44400000000000001},"min":{"1min":0.26900000000000002,"5min":0.216,"15min":0.191},"max":{"1min":0.64300000000000002,"5min":0.69999999999999996,"15min":5.8849999999999998},"last":0.30399999999999999}]},{"osd":2,"last update":"Fri Feb 23 00:51:13 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.436,"15min":0.67100000000000004},"min":{"1min":0.24399999999999999,"5min":0.20499999999999999,"15min":0.188},"max":{"1min":2.3010000000000002,"5min":7.5789999999999997,"15min":48.606000000000002},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.40799999999999997,"15min":0.57599999999999996},"min":{"1min":0.29399999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.63500000000000001,"5min":1,"15min":23.911000000000001},"last":0.34699999999999998}]},{"osd":3,"last update":"Fri Feb 23 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.40500000000000003,"15min":0.59499999999999997},"min":{"1min":0.27800000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":35.244999999999997},"last":0.374},{"interface":"front","average":{"1min":0.437,"5min":0.47699999999999998,"15min":0.68700000000000006},"min":{"1min":0.28899999999999998,"5min":0.26900000000000002,"15min":0.248},"max":{"1min":0.73999999999999999,"5min":7.641,"15min":56.853999999999999},"last":0.49199999999999999}]},{"osd":4,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.63300000000000001,"15min":0.82299999999999995},"min":{"1min":0.40100000000000002,"5min":0.32700000000000001,"15min":0.29299999999999998},"max":{"1min":0.97999999999999998,"5min":7.6829999999999998,"15min":37.646999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.64700000000000002,"15min":0.70799999999999996},"min":{"1min":0.48099999999999998,"5min":0.35899999999999999,"15min":0.31900000000000001},"max":{"1min":0.91400000000000003,"5min":6.0519999999999996,"15min":9.6859999999999999},"last":0.44900000000000001}]},{"osd":5,"last update":"Fri Feb 23 00:50:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.65300000000000002,"15min":0.84099999999999997},"min":{"1min":0.40600000000000003,"5min":0.309,"15min":0.309},"max":{"1min":7.742,"5min":7.742,"15min":48.792000000000002},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.61899999999999999,"15min":0.71899999999999997},"min":{"1min":0.45200000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":11.872999999999999},"last":0.504}]},{"osd":6,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.64100000000000001,"15min":0.82199999999999995},"min":{"1min":0.40000000000000002,"5min":0.37,"15min":0.318},"max":{"1min":0.94199999999999995,"5min":7.7130000000000001,"15min":40.741},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60399999999999998,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.38400000000000001,"15min":0.34000000000000002},"max":{"1min":0.82199999999999995,"5min":0.85999999999999999,"15min":6.0069999999999997},"last":0.47199999999999998}]},{"osd":7,"last update":"Fri Feb 23 00:50:31 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.60699999999999998,"15min":0.71399999999999997},"min":{"1min":0.45300000000000001,"5min":0.39300000000000002,"15min":0.36699999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":20.225000000000001},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.66000000000000003,"15min":0.74099999999999999},"min":{"1min":0.46200000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":6.0949999999999998,"5min":6.0949999999999998,"15min":20.254999999999999},"last":0.48199999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738557,"num_pgs":96,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308180,"kb_used_data":2764,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93436972,"statfs":{"total":95995035648,"available":95679459328,"internally_reserved":0,"allocated":2830336,"data_stored":1028622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 23 00:50:52 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.35199999999999998,"15min":0.60299999999999998},"min":{"1min":0.224,"5min":0.20999999999999999,"15min":0.19500000000000001},"max":{"1min":0.68200000000000005,"5min":0.81399999999999995,"15min":28.856999999999999},"last":0.25},{"interface":"front","average":{"1min":0.373,"5min":0.39500000000000002,"15min":0.629},"min":{"1min":0.249,"5min":0.23300000000000001,"15min":0.20799999999999999},"max":{"1min":0.59099999999999997,"5min":2.3650000000000002,"15min":20.577000000000002},"last":0.41299999999999998}]},{"osd":2,"last update":"Fri Feb 23 00:51:18 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.40400000000000003,"15min":0.60999999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.20499999999999999},"max":{"1min":0.61099999999999999,"5min":2.3889999999999998,"15min":13.863},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.35099999999999998,"15min":0.53800000000000003},"min":{"1min":0.252,"5min":0.223,"15min":0.19800000000000001},"max":{"1min":0.504,"5min":0.65000000000000002,"15min":10.455},"last":0.42199999999999999}]},{"osd":3,"last update":"Fri Feb 23 00:50:46 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.377,"15min":0.77900000000000003},"min":{"1min":0.246,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.64300000000000002,"5min":0.73099999999999998,"15min":35.566000000000003},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.438,"15min":0.76900000000000002},"min":{"1min":0.27700000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.66700000000000004,"5min":2.4049999999999998,"15min":20.702999999999999},"last":0.30599999999999999}]},{"osd":4,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55900000000000005,"15min":0.752},"min":{"1min":0.40899999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.755,"5min":0.94699999999999995,"15min":10.584},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.61599999999999999,"15min":0.78100000000000003},"min":{"1min":0.42899999999999999,"5min":0.38100000000000001,"15min":0.35099999999999998},"max":{"1min":0.78900000000000003,"5min":2.5019999999999998,"15min":13.198},"last":0.35999999999999999}]},{"osd":5,"last update":"Fri Feb 23 00:50:35 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54500000000000004,"15min":0.72099999999999997},"min":{"1min":0.34799999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81799999999999995,"5min":0.90800000000000003,"15min":10.397},"last":0.371},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61299999999999999,"15min":0.78800000000000003},"min":{"1min":0.40999999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":17.297000000000001},"last":0.44400000000000001}]},{"osd":6,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.58899999999999997,"15min":0.78000000000000003},"min":{"1min":0.39400000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.76400000000000001,"5min":0.94399999999999995,"15min":17.373999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.61299999999999999,"15min":0.76200000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.36799999999999999},"max":{"1min":0.72199999999999998,"5min":2.556,"15min":13.973000000000001},"last":0.436}]},{"osd":7,"last update":"Fri Feb 23 00:51:27 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.59099999999999997,"15min":0.77800000000000002},"min":{"1min":0.379,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83399999999999996,"5min":0.96099999999999997,"15min":17.399999999999999},"last":0.496},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.63100000000000001,"15min":0.78100000000000003},"min":{"1min":0.46000000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.752,"5min":2.5289999999999999,"15min":16.427},"last":0.47999999999999998}]}]},{"osd":2,"up_from":0,"seq":77309411506,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":26,"seq":111669149867,"num_pgs":92,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":312340,"kb_used_data":2828,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93432812,"statfs":{"total":95995035648,"available":95675199488,"internally_reserved":0,"allocated":2895872,"data_stored":1042643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:50:45 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.441,"15min":0.53800000000000003},"min":{"1min":0.246,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":8.3300000000000001,"5min":8.3300000000000001,"15min":12.212999999999999},"last":0.216},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.499,"15min":0.61099999999999999},"min":{"1min":0.23100000000000001,"5min":0.22800000000000001,"15min":0.21199999999999999},"max":{"1min":2.9740000000000002,"5min":5.9800000000000004,"15min":11.714},"last":0.28000000000000003}]},{"osd":1,"last update":"Fri Feb 23 00:50:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55700000000000005,"15min":0.63200000000000001},"min":{"1min":0.27200000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":3.056,"5min":6.851,"15min":11.742000000000001},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.58099999999999996,"15min":0.60199999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.19900000000000001},"max":{"1min":2.7709999999999999,"5min":8.2270000000000003,"15min":8.2270000000000003},"last":0.44900000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:50:45 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.625,"15min":0.68999999999999995},"min":{"1min":0.29199999999999998,"5min":0.255,"15min":0.255},"max":{"1min":3.242,"5min":8.0269999999999992,"15min":13.359999999999999},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.52600000000000002,"15min":0.66600000000000004},"min":{"1min":0.27600000000000002,"5min":0.27500000000000002,"15min":0.26100000000000001},"max":{"1min":8.5820000000000007,"5min":8.5820000000000007,"15min":21.122},"last":0.30399999999999999}]},{"osd":4,"last update":"Fri Feb 23 00:51:10 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.58899999999999997,"15min":0.755},"min":{"1min":0.40300000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":3.1419999999999999,"5min":3.1419999999999999,"15min":11.644},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.74199999999999999,"15min":0.753},"min":{"1min":0.36899999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":2.9020000000000001,"5min":12.430999999999999,"15min":12.430999999999999},"last":0.52000000000000002}]},{"osd":5,"last update":"Fri Feb 23 00:50:35 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":0.69899999999999995,"15min":0.68899999999999995},"min":{"1min":0.375,"5min":0.37,"15min":0.35099999999999998},"max":{"1min":10.231999999999999,"5min":10.231999999999999,"15min":10.231999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.76200000000000001,"15min":0.80100000000000005},"min":{"1min":0.38300000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":3.1829999999999998,"5min":11.648,"15min":11.648},"last":0.57799999999999996}]},{"osd":6,"last update":"Fri Feb 23 00:50:53 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76300000000000001,"15min":0.75600000000000001},"min":{"1min":0.42399999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.9510000000000001,"5min":6.9420000000000002,"15min":6.9420000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":1.2050000000000001,"5min":0.751,"15min":0.72299999999999998},"min":{"1min":0.39200000000000002,"5min":0.376,"15min":0.376},"max":{"1min":10.965999999999999,"5min":10.965999999999999,"15min":10.965999999999999},"last":0.48999999999999999}]},{"osd":7,"last update":"Fri Feb 23 00:50:29 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.77500000000000002,"15min":0.82399999999999995},"min":{"1min":0.44500000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.3160000000000001,"5min":11.676,"15min":11.676},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.79300000000000004,"15min":0.80200000000000005},"min":{"1min":0.47699999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80900000000000005,"5min":7.9800000000000004,"15min":7.9800000000000004},"last":0.56999999999999995}]}]},{"osd":4,"up_from":32,"seq":137438953639,"num_pgs":74,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":312100,"kb_used_data":6684,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93433052,"statfs":{"total":95995035648,"available":95675445248,"internally_reserved":0,"allocated":6844416,"data_stored":5037785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.504,"15min":0.52600000000000002},"min":{"1min":0.32200000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.65300000000000002,"5min":1.3819999999999999,"15min":3.5219999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.52400000000000002,"15min":0.58499999999999996},"min":{"1min":0.373,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.72799999999999998,"5min":1.8,"15min":8.6140000000000008},"last":0.505}]},{"osd":1,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51600000000000001,"15min":0.57999999999999996},"min":{"1min":0.31900000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.70699999999999996,"5min":2.6600000000000001,"15min":9.6110000000000007},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.53500000000000003,"15min":0.58499999999999996},"min":{"1min":0.31900000000000001,"5min":0.26300000000000001,"15min":0.255},"max":{"1min":0.70099999999999996,"5min":2.6920000000000002,"15min":6.6950000000000003},"last":0.38800000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.51900000000000002,"15min":0.56999999999999995},"min":{"1min":0.34699999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.65700000000000003,"5min":1.0960000000000001,"15min":3.3069999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54000000000000004,"15min":0.57099999999999995},"min":{"1min":0.33700000000000002,"5min":0.33600000000000002,"15min":0.32500000000000001},"max":{"1min":0.68700000000000006,"5min":1.0189999999999999,"15min":5.8440000000000003},"last":0.48099999999999998}]},{"osd":3,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.67300000000000004,"15min":0.63400000000000001},"min":{"1min":0.38500000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.71999999999999997,"5min":9.1280000000000001,"15min":9.1280000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.58199999999999996,"15min":0.60399999999999998},"min":{"1min":0.39200000000000002,"5min":0.34599999999999997,"15min":0.29699999999999999},"max":{"1min":0.73299999999999998,"5min":4.0220000000000002,"15min":4.1909999999999998},"last":0.434}]},{"osd":5,"last update":"Fri Feb 23 00:50:35 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.45600000000000002,"15min":0.41599999999999998},"min":{"1min":0.26700000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.51500000000000001,"15min":0.47199999999999998},"min":{"1min":0.35399999999999998,"5min":0.29499999999999998,"15min":0.28899999999999998},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.441}]},{"osd":6,"last update":"Fri Feb 23 00:51:05 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.44900000000000001,"15min":0.435},"min":{"1min":0.34000000000000002,"5min":0.28399999999999997,"15min":0.253},"max":{"1min":0.67200000000000004,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47199999999999998,"15min":0.443},"min":{"1min":0.34699999999999998,"5min":0.28799999999999998,"15min":0.215},"max":{"1min":0.64100000000000001,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":0.39900000000000002}]},{"osd":7,"last update":"Fri Feb 23 00:50:31 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.497,"15min":0.48599999999999999},"min":{"1min":0.30599999999999999,"5min":0.26700000000000002,"15min":0.254},"max":{"1min":2.294,"5min":2.294,"15min":2.294},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.379,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":0.45200000000000001}]}]},{"osd":5,"up_from":39,"seq":167503724706,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308148,"kb_used_data":2732,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93437004,"statfs":{"total":95995035648,"available":95679492096,"internally_reserved":0,"allocated":2797568,"data_stored":1038404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:50:27 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.54500000000000004,"15min":0.61399999999999999},"min":{"1min":0.35799999999999998,"5min":0.253,"15min":0.246},"max":{"1min":5.2220000000000004,"5min":5.2220000000000004,"15min":23.881},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.57199999999999995,"15min":0.59399999999999997},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.27000000000000002},"max":{"1min":3.9249999999999998,"5min":3.9249999999999998,"15min":5.4649999999999999},"last":0.40999999999999998}]},{"osd":1,"last update":"Fri Feb 23 00:50:27 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.55200000000000005,"15min":0.65700000000000003},"min":{"1min":0.41299999999999998,"5min":0.254,"15min":0.254},"max":{"1min":3.9460000000000002,"5min":3.9460000000000002,"15min":18.175999999999998},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.56699999999999995,"15min":0.63100000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":3.976,"5min":3.976,"15min":9.0950000000000006},"last":0.46600000000000003}]},{"osd":2,"last update":"Fri Feb 23 00:50:27 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.56699999999999995,"15min":0.71099999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":4.5899999999999999,"5min":4.5899999999999999,"15min":22.282},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.59099999999999997,"15min":0.72699999999999998},"min":{"1min":0.36699999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":3.8540000000000001,"5min":3.8540000000000001,"15min":23.925000000000001},"last":0.54800000000000004}]},{"osd":3,"last update":"Fri Feb 23 00:50:27 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54800000000000004,"15min":0.61899999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.1060000000000001,"5min":1.23,"15min":6.5730000000000004},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.60399999999999998,"15min":0.65500000000000003},"min":{"1min":0.41799999999999998,"5min":0.378,"15min":0.33500000000000002},"max":{"1min":3.9980000000000002,"5min":3.9980000000000002,"15min":8.1760000000000002},"last":0.58399999999999996}]},{"osd":4,"last update":"Fri Feb 23 00:50:27 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.48399999999999999,"15min":0.43099999999999999},"min":{"1min":0.29999999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.68100000000000005,"5min":4.593,"15min":4.593},"last":0.433},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.47899999999999998,"15min":0.46300000000000002},"min":{"1min":0.27900000000000003,"5min":0.27700000000000002,"15min":0.27300000000000002},"max":{"1min":0.68400000000000005,"5min":1.5860000000000001,"15min":3.879},"last":0.42099999999999999}]},{"osd":6,"last update":"Fri Feb 23 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51300000000000001,"15min":0.47599999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.27900000000000003},"max":{"1min":0.71099999999999997,"5min":4.6740000000000004,"15min":4.6740000000000004},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.49099999999999999,"15min":0.47699999999999998},"min":{"1min":0.35999999999999999,"5min":0.28000000000000003,"15min":0.27100000000000002},"max":{"1min":0.73199999999999998,"5min":1.159,"15min":1.159},"last":0.44700000000000001}]},{"osd":7,"last update":"Fri Feb 23 00:51:26 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.39000000000000001,"5min":0.32500000000000001,"15min":0.30399999999999999},"max":{"1min":0.74099999999999999,"5min":1.302,"15min":7.0300000000000002},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52600000000000002,"15min":0.53900000000000003},"min":{"1min":0.36099999999999999,"5min":0.314,"15min":0.29099999999999998},"max":{"1min":0.64600000000000002,"5min":2.016,"15min":9.0579999999999998},"last":0.36099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":303104,"data_stored":6419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4317184,"data_stored":4007883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":68,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":344064,"data_stored":11278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4308992,"data_stored":4006776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":6143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":331776,"data_stored":8750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4444160,"data_stored":4014140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-23T00:51:30.751 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:51:30.781 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:30.835 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:51:30.837 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:30.838 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:51:30.939 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:51:30.967 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:31.316 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:51:31.319 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:31.319 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:51:31.324 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.5 dump_blocked_ops 2024-02-23T00:51:31.420 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:51:31.447 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:31.466 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:51:31.468 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:29.918401+0000 mgr.y (mgr.14517) 509 : audit [DBG] from='client.41669 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:51:31.468 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:31.257196+0000 mon.a (mon.0) 2995 : audit [INF] from='client.? 172.21.15.47:0/956675132' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi047-82465-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:51:31.468 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:31.257260+0000 mon.a (mon.0) 2996 : cluster [DBG] osdmap e562: 8 total, 8 up, 7 in 2024-02-23T00:51:31.468 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:31.257662+0000 mon.a (mon.0) 2997 : audit [INF] from='client.? 172.21.15.47:0/956675132' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi047-82465-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi047-82465-94"}]: dispatch 2024-02-23T00:51:31.469 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:29.918401+0000 mgr.y (mgr.14517) 509 : audit [DBG] from='client.41669 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:51:31.469 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:31.257196+0000 mon.a (mon.0) 2995 : audit [INF] from='client.? 172.21.15.47:0/956675132' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi047-82465-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:51:31.469 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:31.257260+0000 mon.a (mon.0) 2996 : cluster [DBG] osdmap e562: 8 total, 8 up, 7 in 2024-02-23T00:51:31.469 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:31.257662+0000 mon.a (mon.0) 2997 : audit [INF] from='client.? 172.21.15.47:0/956675132' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi047-82465-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi047-82465-94"}]: dispatch 2024-02-23T00:51:31.469 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:31.470 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:51:31.505 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:51:31.571 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:51:31.597 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:29.918401+0000 mgr.y (mgr.14517) 509 : audit [DBG] from='client.41669 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:51:31.597 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:31.257196+0000 mon.a (mon.0) 2995 : audit [INF] from='client.? 172.21.15.47:0/956675132' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi047-82465-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:51:31.597 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:31.257260+0000 mon.a (mon.0) 2996 : cluster [DBG] osdmap e562: 8 total, 8 up, 7 in 2024-02-23T00:51:31.597 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:31.257662+0000 mon.a (mon.0) 2997 : audit [INF] from='client.? 172.21.15.47:0/956675132' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi047-82465-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi047-82465-94"}]: dispatch 2024-02-23T00:51:31.598 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:31.619 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:51:31.622 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:31.623 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:51:31.637 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:51:31.724 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:51:31.751 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:32.202 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:51:32.206 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:32.206 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:51:32.307 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:51:32.334 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:32.432 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:51:32.435 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:32.435 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:51:32.536 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:51:32.563 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:32.620 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:51:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:51:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:51:32.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.624+0000 7fc9635b3700 1 -- 172.21.15.47:0/2227482983 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc95c0f8420 msgr2=0x7fc95c104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:32.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.624+0000 7fc9635b3700 1 --2- 172.21.15.47:0/2227482983 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc95c0f8420 0x7fc95c104a20 secure :-1 s=READY pgs=2734 cs=0 l=1 rev1=1 rx=0x7fc954009be0 tx=0x7fc954004d10).stop 2024-02-23T00:51:32.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.624+0000 7fc9635b3700 1 -- 172.21.15.47:0/2227482983 shutdown_connections 2024-02-23T00:51:32.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.624+0000 7fc9635b3700 1 --2- 172.21.15.47:0/2227482983 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc95c0f8420 0x7fc95c104a20 unknown :-1 s=CLOSED pgs=2734 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:32.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.624+0000 7fc9635b3700 1 --2- 172.21.15.47:0/2227482983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc95c0fdbb0 0x7fc95c0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:32.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.624+0000 7fc9635b3700 1 --2- 172.21.15.47:0/2227482983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc95c0fe510 0x7fc95c0f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:32.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.624+0000 7fc9635b3700 1 -- 172.21.15.47:0/2227482983 >> 172.21.15.47:0/2227482983 conn(0x7fc95c0f3800 msgr2=0x7fc95c0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:32.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.624+0000 7fc9635b3700 1 -- 172.21.15.47:0/2227482983 shutdown_connections 2024-02-23T00:51:32.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.624+0000 7fc9635b3700 1 -- 172.21.15.47:0/2227482983 wait complete. 2024-02-23T00:51:32.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.625+0000 7fc9635b3700 1 librados: starting msgr at 2024-02-23T00:51:32.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.625+0000 7fc9635b3700 1 librados: starting objecter 2024-02-23T00:51:32.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.625+0000 7fc9635b3700 1 Processor -- start 2024-02-23T00:51:32.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.626+0000 7fc9635b3700 1 -- start start 2024-02-23T00:51:32.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.626+0000 7fc9635b3700 1 librados: setting wanted keys 2024-02-23T00:51:32.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.626+0000 7fc9635b3700 1 librados: calling monclient init 2024-02-23T00:51:32.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.626+0000 7fc9635b3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc95c0f8420 0x7fc95c10db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:32.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.626+0000 7fc9635b3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc95c0fdbb0 0x7fc95c10dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:32.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.626+0000 7fc9635b3700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc95c0fe510 0x7fc95c107ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:32.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.626+0000 7fc9635b3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c1083e0 con 0x7fc95c0f8420 2024-02-23T00:51:32.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.626+0000 7fc9635b3700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c108550 con 0x7fc95c0fdbb0 2024-02-23T00:51:32.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.626+0000 7fc9635b3700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c1086c0 con 0x7fc95c0fe510 2024-02-23T00:51:32.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.626+0000 7fc96134f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc95c0f8420 0x7fc95c10db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:32.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.626+0000 7fc961b50700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc95c0fe510 0x7fc95c107ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:32.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.626+0000 7fc96134f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc95c0f8420 0x7fc95c10db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52638/0 (socket says 172.21.15.47:52638) 2024-02-23T00:51:32.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.626+0000 7fc96134f700 1 -- 172.21.15.47:0/3837237073 learned_addr learned my addr 172.21.15.47:0/3837237073 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:51:32.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.626+0000 7fc961b50700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc95c0fe510 0x7fc95c107ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:43058/0 (socket says 172.21.15.47:43058) 2024-02-23T00:51:32.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.626+0000 7fc960b4e700 1 --2- 172.21.15.47:0/3837237073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc95c0fdbb0 0x7fc95c10dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:32.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.626+0000 7fc961b50700 1 -- 172.21.15.47:0/3837237073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc95c0fdbb0 msgr2=0x7fc95c10dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:32.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.626+0000 7fc961b50700 1 --2- 172.21.15.47:0/3837237073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc95c0fdbb0 0x7fc95c10dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:32.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.626+0000 7fc961b50700 1 -- 172.21.15.47:0/3837237073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc95c0f8420 msgr2=0x7fc95c10db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:32.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.626+0000 7fc961b50700 1 --2- 172.21.15.47:0/3837237073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc95c0f8420 0x7fc95c10db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:32.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.626+0000 7fc961b50700 1 -- 172.21.15.47:0/3837237073 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9540097c0 con 0x7fc95c0fe510 2024-02-23T00:51:32.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.626+0000 7fc960b4e700 1 --2- 172.21.15.47:0/3837237073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc95c0fdbb0 0x7fc95c10dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:51:32.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.627+0000 7fc961b50700 1 --2- 172.21.15.47:0/3837237073 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc95c0fe510 0x7fc95c107ef0 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 rx=0x7fc954009bb0 tx=0x7fc954004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:32.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.627+0000 7fc9527fc700 10 client.?.objecter ms_handle_connect 0x7fc95c0fe510 2024-02-23T00:51:32.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.627+0000 7fc9527fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:51:32.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.627+0000 7fc9527fc700 1 -- 172.21.15.47:0/3837237073 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95400e040 con 0x7fc95c0fe510 2024-02-23T00:51:32.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.627+0000 7fc9527fc700 1 -- 172.21.15.47:0/3837237073 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fc954018040 con 0x7fc95c0fe510 2024-02-23T00:51:32.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.627+0000 7fc9635b3700 1 -- 172.21.15.47:0/3837237073 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc95c108940 con 0x7fc95c0fe510 2024-02-23T00:51:32.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.627+0000 7fc9527fc700 1 -- 172.21.15.47:0/3837237073 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95401d400 con 0x7fc95c0fe510 2024-02-23T00:51:32.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.627+0000 7fc9635b3700 10 client.41681.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:51:32.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.627+0000 7fc9635b3700 1 -- 172.21.15.47:0/3837237073 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc95c06af10 con 0x7fc95c0fe510 2024-02-23T00:51:32.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.627+0000 7fc9635b3700 1 librados: init done 2024-02-23T00:51:32.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.628+0000 7fc9527fc700 1 -- 172.21.15.47:0/3837237073 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fc954013630 con 0x7fc95c0fe510 2024-02-23T00:51:32.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.628+0000 7fc9527fc700 1 --2- 172.21.15.47:0/3837237073 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc948062cc0 0x7fc948065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:32.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.629+0000 7fc9527fc700 1 -- 172.21.15.47:0/3837237073 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(563..563 src has 482..563) v4 ==== 8780+0+0 (secure 0 0 0) 0x7fc95407ecc0 con 0x7fc95c0fe510 2024-02-23T00:51:32.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.629+0000 7fc9527fc700 10 client.41681.objecter ms_dispatch 0x7fc95c001550 osd_map(563..563 src has 482..563) v4 2024-02-23T00:51:32.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.629+0000 7fc9527fc700 3 client.41681.objecter handle_osd_map got epochs [563,563] > 0 2024-02-23T00:51:32.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.629+0000 7fc9527fc700 3 client.41681.objecter handle_osd_map decoding full epoch 563 2024-02-23T00:51:32.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.629+0000 7fc96134f700 1 --2- 172.21.15.47:0/3837237073 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc948062cc0 0x7fc948065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:32.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.629+0000 7fc9527fc700 20 client.41681.objecter dump_active .. 0 homeless 2024-02-23T00:51:32.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.629+0000 7fc9635b3700 1 -- 172.21.15.47:0/3837237073 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc95c05ee80 con 0x7fc95c0fe510 2024-02-23T00:51:32.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.629+0000 7fc9527fc700 10 client.41681.objecter ms_handle_connect 0x7fc948062cc0 2024-02-23T00:51:32.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.629+0000 7fc96134f700 1 --2- 172.21.15.47:0/3837237073 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc948062cc0 0x7fc948065180 secure :-1 s=READY pgs=3123 cs=0 l=1 rev1=1 rx=0x7fc958004990 tx=0x7fc95800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:32.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.634+0000 7fc9527fc700 1 -- 172.21.15.47:0/3837237073 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fc95404e490 con 0x7fc95c0fe510 2024-02-23T00:51:32.684 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:51:32.707 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:31.343210+0000 mgr.y (mgr.14517) 510 : cluster [DBG] pgmap v847: 144 pgs: 144 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:51:32.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:32.830+0000 7fc9635b3700 1 -- 172.21.15.47:0/3837237073 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7fc95c106f00 con 0x7fc95c0fe510 2024-02-23T00:51:32.926 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:51:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:51:32.926 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:31.343210+0000 mgr.y (mgr.14517) 510 : cluster [DBG] pgmap v847: 144 pgs: 144 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:51:32.970 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:31.343210+0000 mgr.y (mgr.14517) 510 : cluster [DBG] pgmap v847: 144 pgs: 144 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:51:33.212 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:51:33.215 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:33.215 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:51:33.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:33.260+0000 7fc9527fc700 1 -- 172.21.15.47:0/3837237073 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v564) v1 ==== 91+0+0 (secure 0 0 0) 0x7fc954004810 con 0x7fc95c0fe510 2024-02-23T00:51:33.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:33.263+0000 7fc9635b3700 10 librados: watch_flush enter 2024-02-23T00:51:33.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:33.263+0000 7fc9635b3700 10 librados: watch_flush exit 2024-02-23T00:51:33.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:33.263+0000 7fc9635b3700 20 client.41681.objecter shutdown clearing up homeless session... 2024-02-23T00:51:33.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:33.263+0000 7fc9635b3700 10 client.41681.objecter successfully canceled tick 2024-02-23T00:51:33.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:33.263+0000 7fc9635b3700 1 -- 172.21.15.47:0/3837237073 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc948062cc0 msgr2=0x7fc948065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:33.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:33.263+0000 7fc9635b3700 1 --2- 172.21.15.47:0/3837237073 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc948062cc0 0x7fc948065180 secure :-1 s=READY pgs=3123 cs=0 l=1 rev1=1 rx=0x7fc958004990 tx=0x7fc95800b040).stop 2024-02-23T00:51:33.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:33.263+0000 7fc9635b3700 1 -- 172.21.15.47:0/3837237073 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc95c0fe510 msgr2=0x7fc95c107ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:33.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:33.263+0000 7fc9635b3700 1 --2- 172.21.15.47:0/3837237073 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc95c0fe510 0x7fc95c107ef0 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 rx=0x7fc954009bb0 tx=0x7fc954004d10).stop 2024-02-23T00:51:33.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:33.264+0000 7fc9635b3700 1 -- 172.21.15.47:0/3837237073 shutdown_connections 2024-02-23T00:51:33.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:33.264+0000 7fc9635b3700 1 --2- 172.21.15.47:0/3837237073 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc948062cc0 0x7fc948065180 unknown :-1 s=CLOSED pgs=3123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:33.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:33.264+0000 7fc9635b3700 1 --2- 172.21.15.47:0/3837237073 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc95c0fe510 0x7fc95c107ef0 unknown :-1 s=CLOSED pgs=2735 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:33.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:33.264+0000 7fc9635b3700 1 --2- 172.21.15.47:0/3837237073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc95c0f8420 0x7fc95c10db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:33.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:33.264+0000 7fc9635b3700 1 --2- 172.21.15.47:0/3837237073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc95c0fdbb0 0x7fc95c10dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:33.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:33.264+0000 7fc9635b3700 1 -- 172.21.15.47:0/3837237073 >> 172.21.15.47:0/3837237073 conn(0x7fc95c0f3800 msgr2=0x7fc95c0f51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:33.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:33.264+0000 7fc9635b3700 1 -- 172.21.15.47:0/3837237073 shutdown_connections 2024-02-23T00:51:33.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:33.264+0000 7fc9635b3700 1 -- 172.21.15.47:0/3837237073 wait complete. 2024-02-23T00:51:33.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:33.265+0000 7fc9635b3700 1 librados: shutdown 2024-02-23T00:51:33.268 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is unset 2024-02-23T00:51:33.316 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:51:33.350 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:33.500 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:51:33.500 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:51:33.599 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:51:33.602 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:33.602 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:51:33.703 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:51:33.708 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph pg dump --format=json 2024-02-23T00:51:33.734 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:33.755 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:51:33.761 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:33.762 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:51:33.862 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:51:33.877 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:32.698835+0000 mon.a (mon.0) 2998 : cluster [DBG] osdmap e563: 8 total, 8 up, 7 in 2024-02-23T00:51:33.877 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:32.831700+0000 mon.c (mon.2) 327 : audit [INF] from='client.? 172.21.15.47:0/3837237073' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:51:33.878 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:32.832026+0000 mon.a (mon.0) 2999 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:51:33.878 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:33.255730+0000 mon.a (mon.0) 3000 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:51:33.878 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:33.259976+0000 mon.a (mon.0) 3001 : audit [INF] from='client.? 172.21.15.47:0/956675132' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi047-82465-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi047-82465-94"}]': finished 2024-02-23T00:51:33.878 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:33.260063+0000 mon.a (mon.0) 3002 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:51:33.878 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:33.260118+0000 mon.a (mon.0) 3003 : cluster [DBG] osdmap e564: 8 total, 8 up, 7 in 2024-02-23T00:51:33.878 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:33.661579+0000 osd.0 (osd.0) 233 : cluster [DBG] 2.13 scrub starts 2024-02-23T00:51:33.878 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:33.662350+0000 osd.0 (osd.0) 234 : cluster [DBG] 2.13 scrub ok 2024-02-23T00:51:33.888 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:32.698835+0000 mon.a (mon.0) 2998 : cluster [DBG] osdmap e563: 8 total, 8 up, 7 in 2024-02-23T00:51:33.888 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:32.831700+0000 mon.c (mon.2) 327 : audit [INF] from='client.? 172.21.15.47:0/3837237073' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:51:33.888 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:32.832026+0000 mon.a (mon.0) 2999 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:51:33.888 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:33.255730+0000 mon.a (mon.0) 3000 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:51:33.888 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:33.259976+0000 mon.a (mon.0) 3001 : audit [INF] from='client.? 172.21.15.47:0/956675132' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi047-82465-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi047-82465-94"}]': finished 2024-02-23T00:51:33.889 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:33.260063+0000 mon.a (mon.0) 3002 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:51:33.889 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:33.260118+0000 mon.a (mon.0) 3003 : cluster [DBG] osdmap e564: 8 total, 8 up, 7 in 2024-02-23T00:51:33.889 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:33.661579+0000 osd.0 (osd.0) 233 : cluster [DBG] 2.13 scrub starts 2024-02-23T00:51:33.889 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:33.662350+0000 osd.0 (osd.0) 234 : cluster [DBG] 2.13 scrub ok 2024-02-23T00:51:33.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:32.698835+0000 mon.a (mon.0) 2998 : cluster [DBG] osdmap e563: 8 total, 8 up, 7 in 2024-02-23T00:51:33.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:32.831700+0000 mon.c (mon.2) 327 : audit [INF] from='client.? 172.21.15.47:0/3837237073' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:51:33.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:32.832026+0000 mon.a (mon.0) 2999 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:51:33.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:33.255730+0000 mon.a (mon.0) 3000 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:51:33.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:33.259976+0000 mon.a (mon.0) 3001 : audit [INF] from='client.? 172.21.15.47:0/956675132' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi047-82465-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi047-82465-94"}]': finished 2024-02-23T00:51:33.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:33.260063+0000 mon.a (mon.0) 3002 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:51:33.889 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:33.260118+0000 mon.a (mon.0) 3003 : cluster [DBG] osdmap e564: 8 total, 8 up, 7 in 2024-02-23T00:51:33.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:33.661579+0000 osd.0 (osd.0) 233 : cluster [DBG] 2.13 scrub starts 2024-02-23T00:51:33.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:33.662350+0000 osd.0 (osd.0) 234 : cluster [DBG] 2.13 scrub ok 2024-02-23T00:51:33.890 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:34.089 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:51:34.090 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:34.091 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:51:34.169 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.5 dump_historic_ops 2024-02-23T00:51:34.191 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:51:34.238 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:34.259 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:51:34.261 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:34.262 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:51:34.362 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:51:34.390 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:34.415 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:51:34.418 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:34.418 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:51:34.454 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:51:34.520 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:51:34.548 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:34.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.544+0000 7f291359d700 1 -- 172.21.15.47:0/3521288614 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f290c0fd510 msgr2=0x7f290c0f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:34.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.544+0000 7f291359d700 1 --2- 172.21.15.47:0/3521288614 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f290c0fd510 0x7f290c0f6d40 secure :-1 s=READY pgs=2736 cs=0 l=1 rev1=1 rx=0x7f28fc004870 tx=0x7f28fc009cb0).stop 2024-02-23T00:51:34.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.544+0000 7f291359d700 1 -- 172.21.15.47:0/3521288614 shutdown_connections 2024-02-23T00:51:34.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.544+0000 7f291359d700 1 --2- 172.21.15.47:0/3521288614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f290c0f7400 0x7f290c103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:34.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.544+0000 7f291359d700 1 --2- 172.21.15.47:0/3521288614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f290c0fcbb0 0x7f290c0fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:34.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.544+0000 7f291359d700 1 --2- 172.21.15.47:0/3521288614 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f290c0fd510 0x7f290c0f6d40 unknown :-1 s=CLOSED pgs=2736 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:34.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.544+0000 7f291359d700 1 -- 172.21.15.47:0/3521288614 >> 172.21.15.47:0/3521288614 conn(0x7f290c0f2820 msgr2=0x7f290c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:34.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.544+0000 7f291359d700 1 -- 172.21.15.47:0/3521288614 shutdown_connections 2024-02-23T00:51:34.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.544+0000 7f291359d700 1 -- 172.21.15.47:0/3521288614 wait complete. 2024-02-23T00:51:34.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.545+0000 7f291359d700 1 librados: starting msgr at 2024-02-23T00:51:34.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.545+0000 7f291359d700 1 librados: starting objecter 2024-02-23T00:51:34.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.545+0000 7f291359d700 1 Processor -- start 2024-02-23T00:51:34.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.545+0000 7f291359d700 1 -- start start 2024-02-23T00:51:34.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.545+0000 7f291359d700 1 librados: setting wanted keys 2024-02-23T00:51:34.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.545+0000 7f291359d700 1 librados: calling monclient init 2024-02-23T00:51:34.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.545+0000 7f291359d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f290c0f7400 0x7f290c10cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:34.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.546+0000 7f291359d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f290c0fcbb0 0x7f290c10d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:34.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.546+0000 7f291359d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f290c0fd510 0x7f290c107180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:34.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.546+0000 7f291359d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290c107670 con 0x7f290c0fd510 2024-02-23T00:51:34.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.546+0000 7f291359d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290c1077e0 con 0x7f290c0f7400 2024-02-23T00:51:34.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.546+0000 7f291359d700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f290c107950 con 0x7f290c0fcbb0 2024-02-23T00:51:34.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.546+0000 7f2911339700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f290c0f7400 0x7f290c10cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:34.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.546+0000 7f2910b38700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f290c0fcbb0 0x7f290c10d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:34.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.546+0000 7f2911b3a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f290c0fd510 0x7f290c107180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:34.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.546+0000 7f2911339700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f290c0f7400 0x7f290c10cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.47:36702/0 (socket says 172.21.15.47:36702) 2024-02-23T00:51:34.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.546+0000 7f2911339700 1 -- 172.21.15.47:0/835569047 learned_addr learned my addr 172.21.15.47:0/835569047 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:51:34.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.546+0000 7f2910b38700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f290c0fcbb0 0x7f290c10d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:43094/0 (socket says 172.21.15.47:43094) 2024-02-23T00:51:34.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.546+0000 7f2911339700 1 -- 172.21.15.47:0/835569047 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f290c0fcbb0 msgr2=0x7f290c10d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:34.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.546+0000 7f2911339700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f290c0fcbb0 0x7f290c10d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:34.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.546+0000 7f2911339700 1 -- 172.21.15.47:0/835569047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f290c0fd510 msgr2=0x7f290c107180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:34.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.546+0000 7f2911339700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f290c0fd510 0x7f290c107180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:34.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.546+0000 7f2911339700 1 -- 172.21.15.47:0/835569047 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28fc005040 con 0x7f290c0f7400 2024-02-23T00:51:34.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.546+0000 7f2911339700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f290c0f7400 0x7f290c10cfa0 secure :-1 s=READY pgs=3978 cs=0 l=1 rev1=1 rx=0x7f290800e970 tx=0x7f290800b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:34.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.546+0000 7f29027fc700 10 client.?.objecter ms_handle_connect 0x7f290c0f7400 2024-02-23T00:51:34.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.546+0000 7f29027fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:51:34.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.546+0000 7f29027fc700 1 -- 172.21.15.47:0/835569047 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908004660 con 0x7f290c0f7400 2024-02-23T00:51:34.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.546+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f290c107c30 con 0x7f290c0f7400 2024-02-23T00:51:34.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.546+0000 7f29027fc700 1 -- 172.21.15.47:0/835569047 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f290800f040 con 0x7f290c0f7400 2024-02-23T00:51:34.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.546+0000 7f29027fc700 1 -- 172.21.15.47:0/835569047 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29080146f0 con 0x7f290c0f7400 2024-02-23T00:51:34.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.546+0000 7f291359d700 10 client.43711.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:51:34.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.546+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f290c113e10 con 0x7f290c0f7400 2024-02-23T00:51:34.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.546+0000 7f291359d700 1 librados: init done 2024-02-23T00:51:34.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.547+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f290c05fa60 con 0x7f290c0f7400 2024-02-23T00:51:34.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.548+0000 7f29027fc700 1 -- 172.21.15.47:0/835569047 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f2908007b40 con 0x7f290c0f7400 2024-02-23T00:51:34.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.548+0000 7f29027fc700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f28f8062cc0 0x7f28f8065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:34.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.548+0000 7f29027fc700 1 -- 172.21.15.47:0/835569047 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(565..565 src has 482..565) v4 ==== 9201+0+0 (secure 0 0 0) 0x7f290807d330 con 0x7f290c0f7400 2024-02-23T00:51:34.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.548+0000 7f29027fc700 10 client.43711.objecter ms_dispatch 0x7f290c001550 osd_map(565..565 src has 482..565) v4 2024-02-23T00:51:34.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.548+0000 7f29027fc700 3 client.43711.objecter handle_osd_map got epochs [565,565] > 0 2024-02-23T00:51:34.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.548+0000 7f29027fc700 3 client.43711.objecter handle_osd_map decoding full epoch 565 2024-02-23T00:51:34.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.548+0000 7f2910b38700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f28f8062cc0 0x7f28f8065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:34.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.548+0000 7f29027fc700 20 client.43711.objecter dump_active .. 0 homeless 2024-02-23T00:51:34.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.548+0000 7f29027fc700 1 -- 172.21.15.47:0/835569047 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v565) v1 ==== 54+0+16 (secure 0 0 0) 0x7f290807d750 con 0x7f290c0f7400 2024-02-23T00:51:34.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.549+0000 7f291359d700 10 client.43711.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:34.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.549+0000 7f291359d700 15 client.43711.objecter _session_command_op_assign -1 1 2024-02-23T00:51:34.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.549+0000 7f291359d700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f290c028540 0x7f290c0fd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:34.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.549+0000 7f291359d700 20 client.43711.objecter _get_session s=0x7f290c05ee80 osd=0 3 2024-02-23T00:51:34.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.549+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c05ee80 osd=0 3 2024-02-23T00:51:34.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.549+0000 7f291359d700 20 client.43711.objecter _get_session s=0x7f290c05ee80 osd=0 3 2024-02-23T00:51:34.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.549+0000 7f291359d700 15 client.43711.objecter _session_command_op_remove -1 1 2024-02-23T00:51:34.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.549+0000 7f291359d700 20 client.43711.objecter get_session s=0x7f290c05ee80 osd=0 3 2024-02-23T00:51:34.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.549+0000 7f291359d700 15 client.43711.objecter _session_command_op_assign 0 1 2024-02-23T00:51:34.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.549+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c05ee80 osd=0 4 2024-02-23T00:51:34.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.549+0000 7f291359d700 10 client.43711.objecter _send_command 1 2024-02-23T00:51:34.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.549+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f290c105ee0 con 0x7f290c028540 2024-02-23T00:51:34.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.549+0000 7f29027fc700 10 client.43711.objecter ms_handle_connect 0x7f28f8062cc0 2024-02-23T00:51:34.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.549+0000 7f2911b3a700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f290c028540 0x7f290c0fd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:34.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.549+0000 7f2910b38700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f28f8062cc0 0x7f28f8065180 secure :-1 s=READY pgs=3125 cs=0 l=1 rev1=1 rx=0x7f28fc004ee0 tx=0x7f28fc00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:34.626 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:51:34.626 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:51:34.626 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:51:34.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.549+0000 7f29027fc700 10 client.43711.objecter ms_handle_connect 0x7f290c028540 2024-02-23T00:51:34.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.549+0000 7f2911b3a700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f290c028540 0x7f290c0fd9b0 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:34.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.550+0000 7f29027fc700 1 -- 172.21.15.47:0/835569047 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f290c105ee0 con 0x7f290c028540 2024-02-23T00:51:34.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.550+0000 7f29027fc700 10 client.43711.objecter ms_dispatch 0x7f290c001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:51:34.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.550+0000 7f29027fc700 10 client.43711.objecter _finish_command 1 = system:0 2024-02-23T00:51:34.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.550+0000 7f29027fc700 20 client.43711.objecter put_session s=0x7f290c05ee80 osd=0 4 2024-02-23T00:51:34.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.550+0000 7f29027fc700 15 client.43711.objecter _session_command_op_remove 0 1 2024-02-23T00:51:34.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.565+0000 7f291359d700 10 client.43711.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:51:34.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.565+0000 7f291359d700 15 client.43711.objecter _session_command_op_assign -1 2 2024-02-23T00:51:34.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.565+0000 7f291359d700 20 client.43711.objecter _get_session s=0x7f290c05ee80 osd=0 3 2024-02-23T00:51:34.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.565+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c05ee80 osd=0 3 2024-02-23T00:51:34.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.565+0000 7f291359d700 20 client.43711.objecter _get_session s=0x7f290c05ee80 osd=0 3 2024-02-23T00:51:34.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.565+0000 7f291359d700 15 client.43711.objecter _session_command_op_remove -1 2 2024-02-23T00:51:34.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.565+0000 7f291359d700 20 client.43711.objecter get_session s=0x7f290c05ee80 osd=0 3 2024-02-23T00:51:34.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.565+0000 7f291359d700 15 client.43711.objecter _session_command_op_assign 0 2 2024-02-23T00:51:34.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.565+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c05ee80 osd=0 4 2024-02-23T00:51:34.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.565+0000 7f291359d700 10 client.43711.objecter _send_command 2 2024-02-23T00:51:34.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.565+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f290c05a040 con 0x7f290c028540 2024-02-23T00:51:34.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.566+0000 7f29027fc700 1 -- 172.21.15.47:0/835569047 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f290c05a040 con 0x7f290c028540 2024-02-23T00:51:34.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.566+0000 7f29027fc700 10 client.43711.objecter ms_dispatch 0x7f290c001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:51:34.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.566+0000 7f29027fc700 10 client.43711.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:51:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.566+0000 7f29027fc700 20 client.43711.objecter put_session s=0x7f290c05ee80 osd=0 4 2024-02-23T00:51:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.566+0000 7f29027fc700 15 client.43711.objecter _session_command_op_remove 0 2 2024-02-23T00:51:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.567+0000 7f291359d700 10 client.43711.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.567+0000 7f291359d700 15 client.43711.objecter _session_command_op_assign -1 3 2024-02-23T00:51:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.567+0000 7f291359d700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f290c0594d0 0x7f290c0fdea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.567+0000 7f291359d700 20 client.43711.objecter _get_session s=0x7f290c059360 osd=1 3 2024-02-23T00:51:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.567+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c059360 osd=1 3 2024-02-23T00:51:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.567+0000 7f291359d700 20 client.43711.objecter _get_session s=0x7f290c059360 osd=1 3 2024-02-23T00:51:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.567+0000 7f291359d700 15 client.43711.objecter _session_command_op_remove -1 3 2024-02-23T00:51:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.567+0000 7f291359d700 20 client.43711.objecter get_session s=0x7f290c059360 osd=1 3 2024-02-23T00:51:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.567+0000 7f291359d700 15 client.43711.objecter _session_command_op_assign 1 3 2024-02-23T00:51:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.567+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c059360 osd=1 4 2024-02-23T00:51:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.567+0000 7f291359d700 10 client.43711.objecter _send_command 3 2024-02-23T00:51:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.567+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f290c061a90 con 0x7f290c0594d0 2024-02-23T00:51:34.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.567+0000 7f2911339700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f290c0594d0 0x7f290c0fdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:34.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.567+0000 7f29027fc700 10 client.43711.objecter ms_handle_connect 0x7f290c0594d0 2024-02-23T00:51:34.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.567+0000 7f2911339700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f290c0594d0 0x7f290c0fdea0 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:34.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.568+0000 7f29027fc700 1 -- 172.21.15.47:0/835569047 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f290c061a90 con 0x7f290c0594d0 2024-02-23T00:51:34.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.568+0000 7f29027fc700 10 client.43711.objecter ms_dispatch 0x7f290c001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:51:34.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.568+0000 7f29027fc700 10 client.43711.objecter _finish_command 3 = system:0 2024-02-23T00:51:34.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.568+0000 7f29027fc700 20 client.43711.objecter put_session s=0x7f290c059360 osd=1 4 2024-02-23T00:51:34.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.568+0000 7f29027fc700 15 client.43711.objecter _session_command_op_remove 1 3 2024-02-23T00:51:34.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.584+0000 7f291359d700 10 client.43711.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:51:34.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.584+0000 7f291359d700 15 client.43711.objecter _session_command_op_assign -1 4 2024-02-23T00:51:34.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.584+0000 7f291359d700 20 client.43711.objecter _get_session s=0x7f290c059360 osd=1 3 2024-02-23T00:51:34.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.584+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c059360 osd=1 3 2024-02-23T00:51:34.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.584+0000 7f291359d700 20 client.43711.objecter _get_session s=0x7f290c059360 osd=1 3 2024-02-23T00:51:34.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.584+0000 7f291359d700 15 client.43711.objecter _session_command_op_remove -1 4 2024-02-23T00:51:34.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.584+0000 7f291359d700 20 client.43711.objecter get_session s=0x7f290c059360 osd=1 3 2024-02-23T00:51:34.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.584+0000 7f291359d700 15 client.43711.objecter _session_command_op_assign 1 4 2024-02-23T00:51:34.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.584+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c059360 osd=1 4 2024-02-23T00:51:34.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.584+0000 7f291359d700 10 client.43711.objecter _send_command 4 2024-02-23T00:51:34.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.584+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f290c0fe390 con 0x7f290c0594d0 2024-02-23T00:51:34.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.585+0000 7f29027fc700 1 -- 172.21.15.47:0/835569047 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f290c0fe390 con 0x7f290c0594d0 2024-02-23T00:51:34.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.585+0000 7f29027fc700 10 client.43711.objecter ms_dispatch 0x7f290c001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:51:34.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.585+0000 7f29027fc700 10 client.43711.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:51:34.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.585+0000 7f29027fc700 20 client.43711.objecter put_session s=0x7f290c059360 osd=1 4 2024-02-23T00:51:34.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.585+0000 7f29027fc700 15 client.43711.objecter _session_command_op_remove 1 4 2024-02-23T00:51:34.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.586+0000 7f291359d700 10 client.43711.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:34.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.586+0000 7f291359d700 15 client.43711.objecter _session_command_op_assign -1 5 2024-02-23T00:51:34.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.586+0000 7f291359d700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f290c061160 0x7f290c116010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:34.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.586+0000 7f291359d700 20 client.43711.objecter _get_session s=0x7f290c060ff0 osd=2 3 2024-02-23T00:51:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.586+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c060ff0 osd=2 3 2024-02-23T00:51:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.586+0000 7f291359d700 20 client.43711.objecter _get_session s=0x7f290c060ff0 osd=2 3 2024-02-23T00:51:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.586+0000 7f291359d700 15 client.43711.objecter _session_command_op_remove -1 5 2024-02-23T00:51:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.586+0000 7f291359d700 20 client.43711.objecter get_session s=0x7f290c060ff0 osd=2 3 2024-02-23T00:51:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.586+0000 7f291359d700 15 client.43711.objecter _session_command_op_assign 2 5 2024-02-23T00:51:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.586+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c060ff0 osd=2 4 2024-02-23T00:51:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.586+0000 7f291359d700 10 client.43711.objecter _send_command 5 2024-02-23T00:51:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.586+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f290c116590 con 0x7f290c061160 2024-02-23T00:51:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.586+0000 7f2910b38700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f290c061160 0x7f290c116010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.586+0000 7f29027fc700 10 client.43711.objecter ms_handle_connect 0x7f290c061160 2024-02-23T00:51:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.586+0000 7f2910b38700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f290c061160 0x7f290c116010 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.587+0000 7f29027fc700 1 -- 172.21.15.47:0/835569047 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f290c116590 con 0x7f290c061160 2024-02-23T00:51:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.587+0000 7f29027fc700 10 client.43711.objecter ms_dispatch 0x7f290c001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:51:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.587+0000 7f29027fc700 10 client.43711.objecter _finish_command 5 = system:0 2024-02-23T00:51:34.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.587+0000 7f29027fc700 20 client.43711.objecter put_session s=0x7f290c060ff0 osd=2 4 2024-02-23T00:51:34.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.587+0000 7f29027fc700 15 client.43711.objecter _session_command_op_remove 2 5 2024-02-23T00:51:34.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.601+0000 7f291359d700 10 client.43711.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:51:34.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.601+0000 7f291359d700 15 client.43711.objecter _session_command_op_assign -1 6 2024-02-23T00:51:34.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.601+0000 7f291359d700 20 client.43711.objecter _get_session s=0x7f290c060ff0 osd=2 3 2024-02-23T00:51:34.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.601+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c060ff0 osd=2 3 2024-02-23T00:51:34.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.601+0000 7f291359d700 20 client.43711.objecter _get_session s=0x7f290c060ff0 osd=2 3 2024-02-23T00:51:34.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.601+0000 7f291359d700 15 client.43711.objecter _session_command_op_remove -1 6 2024-02-23T00:51:34.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.601+0000 7f291359d700 20 client.43711.objecter get_session s=0x7f290c060ff0 osd=2 3 2024-02-23T00:51:34.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.601+0000 7f291359d700 15 client.43711.objecter _session_command_op_assign 2 6 2024-02-23T00:51:34.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.601+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c060ff0 osd=2 4 2024-02-23T00:51:34.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.601+0000 7f291359d700 10 client.43711.objecter _send_command 6 2024-02-23T00:51:34.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.601+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f290c0fe7a0 con 0x7f290c061160 2024-02-23T00:51:34.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.603+0000 7f29027fc700 1 -- 172.21.15.47:0/835569047 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f290c0fe7a0 con 0x7f290c061160 2024-02-23T00:51:34.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.603+0000 7f29027fc700 10 client.43711.objecter ms_dispatch 0x7f290c001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:51:34.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.603+0000 7f29027fc700 10 client.43711.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:51:34.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.603+0000 7f29027fc700 20 client.43711.objecter put_session s=0x7f290c060ff0 osd=2 4 2024-02-23T00:51:34.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.603+0000 7f29027fc700 15 client.43711.objecter _session_command_op_remove 2 6 2024-02-23T00:51:34.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.603+0000 7f291359d700 10 client.43711.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:34.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.603+0000 7f291359d700 15 client.43711.objecter _session_command_op_assign -1 7 2024-02-23T00:51:34.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.603+0000 7f291359d700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f290c11d820 0x7f290c11fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:34.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.603+0000 7f291359d700 20 client.43711.objecter _get_session s=0x7f290c116f00 osd=3 3 2024-02-23T00:51:34.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.603+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c116f00 osd=3 3 2024-02-23T00:51:34.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.603+0000 7f291359d700 20 client.43711.objecter _get_session s=0x7f290c116f00 osd=3 3 2024-02-23T00:51:34.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.603+0000 7f291359d700 15 client.43711.objecter _session_command_op_remove -1 7 2024-02-23T00:51:34.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.603+0000 7f291359d700 20 client.43711.objecter get_session s=0x7f290c116f00 osd=3 3 2024-02-23T00:51:34.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.603+0000 7f291359d700 15 client.43711.objecter _session_command_op_assign 3 7 2024-02-23T00:51:34.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.603+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c116f00 osd=3 4 2024-02-23T00:51:34.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.603+0000 7f291359d700 10 client.43711.objecter _send_command 7 2024-02-23T00:51:34.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.603+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f290c122910 con 0x7f290c11d820 2024-02-23T00:51:34.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.603+0000 7f2911b3a700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f290c11d820 0x7f290c11fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:34.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.603+0000 7f29027fc700 10 client.43711.objecter ms_handle_connect 0x7f290c11d820 2024-02-23T00:51:34.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.603+0000 7f2911b3a700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f290c11d820 0x7f290c11fc10 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:34.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.604+0000 7f29027fc700 1 -- 172.21.15.47:0/835569047 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f290c122910 con 0x7f290c11d820 2024-02-23T00:51:34.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.604+0000 7f29027fc700 10 client.43711.objecter ms_dispatch 0x7f290c001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:51:34.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.604+0000 7f29027fc700 10 client.43711.objecter _finish_command 7 = system:0 2024-02-23T00:51:34.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.604+0000 7f29027fc700 20 client.43711.objecter put_session s=0x7f290c116f00 osd=3 4 2024-02-23T00:51:34.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.604+0000 7f29027fc700 15 client.43711.objecter _session_command_op_remove 3 7 2024-02-23T00:51:34.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.619+0000 7f291359d700 10 client.43711.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:51:34.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.619+0000 7f291359d700 15 client.43711.objecter _session_command_op_assign -1 8 2024-02-23T00:51:34.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.619+0000 7f291359d700 20 client.43711.objecter _get_session s=0x7f290c116f00 osd=3 3 2024-02-23T00:51:34.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.619+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c116f00 osd=3 3 2024-02-23T00:51:34.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.619+0000 7f291359d700 20 client.43711.objecter _get_session s=0x7f290c116f00 osd=3 3 2024-02-23T00:51:34.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.619+0000 7f291359d700 15 client.43711.objecter _session_command_op_remove -1 8 2024-02-23T00:51:34.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.619+0000 7f291359d700 20 client.43711.objecter get_session s=0x7f290c116f00 osd=3 3 2024-02-23T00:51:34.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.619+0000 7f291359d700 15 client.43711.objecter _session_command_op_assign 3 8 2024-02-23T00:51:34.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.619+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c116f00 osd=3 4 2024-02-23T00:51:34.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.619+0000 7f291359d700 10 client.43711.objecter _send_command 8 2024-02-23T00:51:34.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.619+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f290c116970 con 0x7f290c11d820 2024-02-23T00:51:34.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.621+0000 7f29027fc700 1 -- 172.21.15.47:0/835569047 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f290c116970 con 0x7f290c11d820 2024-02-23T00:51:34.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.621+0000 7f29027fc700 10 client.43711.objecter ms_dispatch 0x7f290c001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:51:34.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.621+0000 7f29027fc700 10 client.43711.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:51:34.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.621+0000 7f29027fc700 20 client.43711.objecter put_session s=0x7f290c116f00 osd=3 4 2024-02-23T00:51:34.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.621+0000 7f29027fc700 15 client.43711.objecter _session_command_op_remove 3 8 2024-02-23T00:51:34.636 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:51:34.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.621+0000 7f291359d700 10 client.43711.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:34.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.621+0000 7f291359d700 15 client.43711.objecter _session_command_op_assign -1 9 2024-02-23T00:51:34.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.621+0000 7f291359d700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f290c0280a0 0x7f290c121280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:34.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.621+0000 7f291359d700 20 client.43711.objecter _get_session s=0x7f290c116730 osd=4 3 2024-02-23T00:51:34.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.621+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c116730 osd=4 3 2024-02-23T00:51:34.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.621+0000 7f291359d700 20 client.43711.objecter _get_session s=0x7f290c116730 osd=4 3 2024-02-23T00:51:34.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.621+0000 7f291359d700 15 client.43711.objecter _session_command_op_remove -1 9 2024-02-23T00:51:34.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.621+0000 7f291359d700 20 client.43711.objecter get_session s=0x7f290c116730 osd=4 3 2024-02-23T00:51:34.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.621+0000 7f291359d700 15 client.43711.objecter _session_command_op_assign 4 9 2024-02-23T00:51:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.621+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c116730 osd=4 4 2024-02-23T00:51:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.621+0000 7f291359d700 10 client.43711.objecter _send_command 9 2024-02-23T00:51:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.621+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f290c116930 con 0x7f290c0280a0 2024-02-23T00:51:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.621+0000 7f2911339700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f290c0280a0 0x7f290c121280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.621+0000 7f29027fc700 10 client.43711.objecter ms_handle_connect 0x7f290c0280a0 2024-02-23T00:51:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.621+0000 7f2911339700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f290c0280a0 0x7f290c121280 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.622+0000 7f29027fc700 1 -- 172.21.15.47:0/835569047 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f290c116930 con 0x7f290c0280a0 2024-02-23T00:51:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.622+0000 7f29027fc700 10 client.43711.objecter ms_dispatch 0x7f290c001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:51:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.622+0000 7f29027fc700 10 client.43711.objecter _finish_command 9 = system:0 2024-02-23T00:51:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.622+0000 7f29027fc700 20 client.43711.objecter put_session s=0x7f290c116730 osd=4 4 2024-02-23T00:51:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.622+0000 7f29027fc700 15 client.43711.objecter _session_command_op_remove 4 9 2024-02-23T00:51:34.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.637+0000 7f291359d700 10 client.43711.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:51:34.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.637+0000 7f291359d700 15 client.43711.objecter _session_command_op_assign -1 10 2024-02-23T00:51:34.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.637+0000 7f291359d700 20 client.43711.objecter _get_session s=0x7f290c116730 osd=4 3 2024-02-23T00:51:34.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.637+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c116730 osd=4 3 2024-02-23T00:51:34.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.637+0000 7f291359d700 20 client.43711.objecter _get_session s=0x7f290c116730 osd=4 3 2024-02-23T00:51:34.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.637+0000 7f291359d700 15 client.43711.objecter _session_command_op_remove -1 10 2024-02-23T00:51:34.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.637+0000 7f291359d700 20 client.43711.objecter get_session s=0x7f290c116730 osd=4 3 2024-02-23T00:51:34.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.637+0000 7f291359d700 15 client.43711.objecter _session_command_op_assign 4 10 2024-02-23T00:51:34.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.637+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c116730 osd=4 4 2024-02-23T00:51:34.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.637+0000 7f291359d700 10 client.43711.objecter _send_command 10 2024-02-23T00:51:34.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.637+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f290c116930 con 0x7f290c0280a0 2024-02-23T00:51:34.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.639+0000 7f29027fc700 1 -- 172.21.15.47:0/835569047 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f290c116930 con 0x7f290c0280a0 2024-02-23T00:51:34.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.639+0000 7f29027fc700 10 client.43711.objecter ms_dispatch 0x7f290c001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:51:34.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.639+0000 7f29027fc700 10 client.43711.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:51:34.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.639+0000 7f29027fc700 20 client.43711.objecter put_session s=0x7f290c116730 osd=4 4 2024-02-23T00:51:34.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.639+0000 7f29027fc700 15 client.43711.objecter _session_command_op_remove 4 10 2024-02-23T00:51:34.642 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:51:34.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.639+0000 7f291359d700 10 client.43711.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:34.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.639+0000 7f291359d700 15 client.43711.objecter _session_command_op_assign -1 11 2024-02-23T00:51:34.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.639+0000 7f291359d700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f290c121cd0 0x7f290c122130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:34.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.639+0000 7f291359d700 20 client.43711.objecter _get_session s=0x7f290c116930 osd=5 3 2024-02-23T00:51:34.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.639+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c116930 osd=5 3 2024-02-23T00:51:34.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.639+0000 7f291359d700 20 client.43711.objecter _get_session s=0x7f290c116930 osd=5 3 2024-02-23T00:51:34.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.639+0000 7f291359d700 15 client.43711.objecter _session_command_op_remove -1 11 2024-02-23T00:51:34.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.639+0000 7f291359d700 20 client.43711.objecter get_session s=0x7f290c116930 osd=5 3 2024-02-23T00:51:34.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.639+0000 7f291359d700 15 client.43711.objecter _session_command_op_assign 5 11 2024-02-23T00:51:34.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.639+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c116930 osd=5 4 2024-02-23T00:51:34.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.639+0000 7f291359d700 10 client.43711.objecter _send_command 11 2024-02-23T00:51:34.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.639+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f290c129230 con 0x7f290c121cd0 2024-02-23T00:51:34.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.639+0000 7f2910b38700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f290c121cd0 0x7f290c122130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:34.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.640+0000 7f29027fc700 10 client.43711.objecter ms_handle_connect 0x7f290c121cd0 2024-02-23T00:51:34.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.640+0000 7f2910b38700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f290c121cd0 0x7f290c122130 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:34.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.640+0000 7f29027fc700 1 -- 172.21.15.47:0/835569047 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f290c129230 con 0x7f290c121cd0 2024-02-23T00:51:34.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.640+0000 7f29027fc700 10 client.43711.objecter ms_dispatch 0x7f290c001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:51:34.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.640+0000 7f29027fc700 10 client.43711.objecter _finish_command 11 = system:0 2024-02-23T00:51:34.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.640+0000 7f29027fc700 20 client.43711.objecter put_session s=0x7f290c116930 osd=5 4 2024-02-23T00:51:34.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.640+0000 7f29027fc700 15 client.43711.objecter _session_command_op_remove 5 11 2024-02-23T00:51:34.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.655+0000 7f291359d700 10 client.43711.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:51:34.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.655+0000 7f291359d700 15 client.43711.objecter _session_command_op_assign -1 12 2024-02-23T00:51:34.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.655+0000 7f291359d700 20 client.43711.objecter _get_session s=0x7f290c116930 osd=5 3 2024-02-23T00:51:34.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.655+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c116930 osd=5 3 2024-02-23T00:51:34.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.655+0000 7f291359d700 20 client.43711.objecter _get_session s=0x7f290c116930 osd=5 3 2024-02-23T00:51:34.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.655+0000 7f291359d700 15 client.43711.objecter _session_command_op_remove -1 12 2024-02-23T00:51:34.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.655+0000 7f291359d700 20 client.43711.objecter get_session s=0x7f290c116930 osd=5 3 2024-02-23T00:51:34.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.655+0000 7f291359d700 15 client.43711.objecter _session_command_op_assign 5 12 2024-02-23T00:51:34.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.655+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c116930 osd=5 4 2024-02-23T00:51:34.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.655+0000 7f291359d700 10 client.43711.objecter _send_command 12 2024-02-23T00:51:34.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.655+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f290c1175f0 con 0x7f290c121cd0 2024-02-23T00:51:34.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.657+0000 7f29027fc700 1 -- 172.21.15.47:0/835569047 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f290c1175f0 con 0x7f290c121cd0 2024-02-23T00:51:34.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.657+0000 7f29027fc700 10 client.43711.objecter ms_dispatch 0x7f290c001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:51:34.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.657+0000 7f29027fc700 10 client.43711.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:51:34.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.657+0000 7f29027fc700 20 client.43711.objecter put_session s=0x7f290c116930 osd=5 4 2024-02-23T00:51:34.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.657+0000 7f29027fc700 15 client.43711.objecter _session_command_op_remove 5 12 2024-02-23T00:51:34.660 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:51:34.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.657+0000 7f291359d700 10 client.43711.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:34.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.657+0000 7f291359d700 15 client.43711.objecter _session_command_op_assign -1 13 2024-02-23T00:51:34.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.657+0000 7f291359d700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f290c117cd0 0x7f290c118110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:34.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.657+0000 7f291359d700 20 client.43711.objecter _get_session s=0x7f290c1175f0 osd=6 3 2024-02-23T00:51:34.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.657+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c1175f0 osd=6 3 2024-02-23T00:51:34.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.657+0000 7f291359d700 20 client.43711.objecter _get_session s=0x7f290c1175f0 osd=6 3 2024-02-23T00:51:34.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.657+0000 7f291359d700 15 client.43711.objecter _session_command_op_remove -1 13 2024-02-23T00:51:34.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.657+0000 7f291359d700 20 client.43711.objecter get_session s=0x7f290c1175f0 osd=6 3 2024-02-23T00:51:34.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.657+0000 7f291359d700 15 client.43711.objecter _session_command_op_assign 6 13 2024-02-23T00:51:34.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.657+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c1175f0 osd=6 4 2024-02-23T00:51:34.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.657+0000 7f291359d700 10 client.43711.objecter _send_command 13 2024-02-23T00:51:34.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.657+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f290c118760 con 0x7f290c117cd0 2024-02-23T00:51:34.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.657+0000 7f2911b3a700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f290c117cd0 0x7f290c118110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:34.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.658+0000 7f29027fc700 10 client.43711.objecter ms_handle_connect 0x7f290c117cd0 2024-02-23T00:51:34.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.658+0000 7f2911b3a700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f290c117cd0 0x7f290c118110 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:34.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.658+0000 7f29027fc700 1 -- 172.21.15.47:0/835569047 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f290c118760 con 0x7f290c117cd0 2024-02-23T00:51:34.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.658+0000 7f29027fc700 10 client.43711.objecter ms_dispatch 0x7f290c001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:51:34.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.658+0000 7f29027fc700 10 client.43711.objecter _finish_command 13 = system:0 2024-02-23T00:51:34.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.658+0000 7f29027fc700 20 client.43711.objecter put_session s=0x7f290c1175f0 osd=6 4 2024-02-23T00:51:34.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.658+0000 7f29027fc700 15 client.43711.objecter _session_command_op_remove 6 13 2024-02-23T00:51:34.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.673+0000 7f291359d700 10 client.43711.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:51:34.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.673+0000 7f291359d700 15 client.43711.objecter _session_command_op_assign -1 14 2024-02-23T00:51:34.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.673+0000 7f291359d700 20 client.43711.objecter _get_session s=0x7f290c1175f0 osd=6 3 2024-02-23T00:51:34.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.673+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c1175f0 osd=6 3 2024-02-23T00:51:34.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.673+0000 7f291359d700 20 client.43711.objecter _get_session s=0x7f290c1175f0 osd=6 3 2024-02-23T00:51:34.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.673+0000 7f291359d700 15 client.43711.objecter _session_command_op_remove -1 14 2024-02-23T00:51:34.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.673+0000 7f291359d700 20 client.43711.objecter get_session s=0x7f290c1175f0 osd=6 3 2024-02-23T00:51:34.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.673+0000 7f291359d700 15 client.43711.objecter _session_command_op_assign 6 14 2024-02-23T00:51:34.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.673+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c1175f0 osd=6 4 2024-02-23T00:51:34.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.673+0000 7f291359d700 10 client.43711.objecter _send_command 14 2024-02-23T00:51:34.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.673+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f290c118760 con 0x7f290c117cd0 2024-02-23T00:51:34.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.675+0000 7f29027fc700 1 -- 172.21.15.47:0/835569047 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f290c118760 con 0x7f290c117cd0 2024-02-23T00:51:34.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.675+0000 7f29027fc700 10 client.43711.objecter ms_dispatch 0x7f290c001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:51:34.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.675+0000 7f29027fc700 10 client.43711.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:51:34.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.675+0000 7f29027fc700 20 client.43711.objecter put_session s=0x7f290c1175f0 osd=6 4 2024-02-23T00:51:34.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.675+0000 7f29027fc700 15 client.43711.objecter _session_command_op_remove 6 14 2024-02-23T00:51:34.678 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:51:34.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.675+0000 7f291359d700 10 client.43711.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:34.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.675+0000 7f291359d700 15 client.43711.objecter _session_command_op_assign -1 15 2024-02-23T00:51:34.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.675+0000 7f291359d700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f290c118e40 0x7f290c1192a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:34.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.675+0000 7f291359d700 20 client.43711.objecter _get_session s=0x7f290c118760 osd=7 3 2024-02-23T00:51:34.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.675+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c118760 osd=7 3 2024-02-23T00:51:34.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.675+0000 7f291359d700 20 client.43711.objecter _get_session s=0x7f290c118760 osd=7 3 2024-02-23T00:51:34.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.675+0000 7f291359d700 15 client.43711.objecter _session_command_op_remove -1 15 2024-02-23T00:51:34.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.675+0000 7f291359d700 20 client.43711.objecter get_session s=0x7f290c118760 osd=7 3 2024-02-23T00:51:34.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.675+0000 7f291359d700 15 client.43711.objecter _session_command_op_assign 7 15 2024-02-23T00:51:34.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.675+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c118760 osd=7 4 2024-02-23T00:51:34.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.675+0000 7f291359d700 10 client.43711.objecter _send_command 15 2024-02-23T00:51:34.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.675+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f290c1317e0 con 0x7f290c118e40 2024-02-23T00:51:34.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.675+0000 7f2911339700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f290c118e40 0x7f290c1192a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:34.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.676+0000 7f29027fc700 10 client.43711.objecter ms_handle_connect 0x7f290c118e40 2024-02-23T00:51:34.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.676+0000 7f2911339700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f290c118e40 0x7f290c1192a0 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:34.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.676+0000 7f29027fc700 1 -- 172.21.15.47:0/835569047 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f290c1317e0 con 0x7f290c118e40 2024-02-23T00:51:34.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.676+0000 7f29027fc700 10 client.43711.objecter ms_dispatch 0x7f290c001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:51:34.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.676+0000 7f29027fc700 10 client.43711.objecter _finish_command 15 = system:0 2024-02-23T00:51:34.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.676+0000 7f29027fc700 20 client.43711.objecter put_session s=0x7f290c118760 osd=7 4 2024-02-23T00:51:34.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.676+0000 7f29027fc700 15 client.43711.objecter _session_command_op_remove 7 15 2024-02-23T00:51:34.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.691+0000 7f291359d700 10 client.43711.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:51:34.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.691+0000 7f291359d700 15 client.43711.objecter _session_command_op_assign -1 16 2024-02-23T00:51:34.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.691+0000 7f291359d700 20 client.43711.objecter _get_session s=0x7f290c118760 osd=7 3 2024-02-23T00:51:34.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.691+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c118760 osd=7 3 2024-02-23T00:51:34.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.691+0000 7f291359d700 20 client.43711.objecter _get_session s=0x7f290c118760 osd=7 3 2024-02-23T00:51:34.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.691+0000 7f291359d700 15 client.43711.objecter _session_command_op_remove -1 16 2024-02-23T00:51:34.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.691+0000 7f291359d700 20 client.43711.objecter get_session s=0x7f290c118760 osd=7 3 2024-02-23T00:51:34.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.691+0000 7f291359d700 15 client.43711.objecter _session_command_op_assign 7 16 2024-02-23T00:51:34.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.691+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c118760 osd=7 4 2024-02-23T00:51:34.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.691+0000 7f291359d700 10 client.43711.objecter _send_command 16 2024-02-23T00:51:34.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.691+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f290c1317e0 con 0x7f290c118e40 2024-02-23T00:51:34.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.692+0000 7f29027fc700 1 -- 172.21.15.47:0/835569047 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f290c1317e0 con 0x7f290c118e40 2024-02-23T00:51:34.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.692+0000 7f29027fc700 10 client.43711.objecter ms_dispatch 0x7f290c001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:51:34.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.692+0000 7f29027fc700 10 client.43711.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:51:34.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.692+0000 7f29027fc700 20 client.43711.objecter put_session s=0x7f290c118760 osd=7 4 2024-02-23T00:51:34.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.692+0000 7f29027fc700 15 client.43711.objecter _session_command_op_remove 7 16 2024-02-23T00:51:34.695 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:51:34.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 10 librados: watch_flush enter 2024-02-23T00:51:34.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 10 librados: watch_flush exit 2024-02-23T00:51:34.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 10 client.43711.objecter close_session for osd.0 2024-02-23T00:51:34.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f290c028540 msgr2=0x7f290c0fd9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:34.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f290c028540 0x7f290c0fd9b0 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:34.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c05ee80 osd=0 1 2024-02-23T00:51:34.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 10 client.43711.objecter close_session for osd.1 2024-02-23T00:51:34.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f290c0594d0 msgr2=0x7f290c0fdea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:34.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f290c0594d0 0x7f290c0fdea0 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:34.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c059360 osd=1 1 2024-02-23T00:51:34.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 10 client.43711.objecter close_session for osd.2 2024-02-23T00:51:34.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f290c061160 msgr2=0x7f290c116010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:34.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f290c061160 0x7f290c116010 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:34.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c060ff0 osd=2 1 2024-02-23T00:51:34.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 10 client.43711.objecter close_session for osd.3 2024-02-23T00:51:34.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f290c11d820 msgr2=0x7f290c11fc10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:34.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f2911b3a700 1 -- 172.21.15.47:0/835569047 reap_dead start 2024-02-23T00:51:34.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f290c11d820 0x7f290c11fc10 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:34.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c116f00 osd=3 1 2024-02-23T00:51:34.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 10 client.43711.objecter close_session for osd.4 2024-02-23T00:51:34.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f290c0280a0 msgr2=0x7f290c121280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:34.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f290c0280a0 0x7f290c121280 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:34.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c116730 osd=4 1 2024-02-23T00:51:34.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 10 client.43711.objecter close_session for osd.5 2024-02-23T00:51:34.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f290c121cd0 msgr2=0x7f290c122130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:34.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f290c121cd0 0x7f290c122130 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:34.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c116930 osd=5 1 2024-02-23T00:51:34.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 10 client.43711.objecter close_session for osd.6 2024-02-23T00:51:34.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f290c117cd0 msgr2=0x7f290c118110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:34.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f290c117cd0 0x7f290c118110 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:34.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c1175f0 osd=6 1 2024-02-23T00:51:34.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 10 client.43711.objecter close_session for osd.7 2024-02-23T00:51:34.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f290c118e40 msgr2=0x7f290c1192a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:34.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f290c118e40 0x7f290c1192a0 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:34.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 20 client.43711.objecter put_session s=0x7f290c118760 osd=7 1 2024-02-23T00:51:34.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 20 client.43711.objecter shutdown clearing up homeless session... 2024-02-23T00:51:34.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 10 client.43711.objecter successfully canceled tick 2024-02-23T00:51:34.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f2911b3a700 1 -- 172.21.15.47:0/835569047 reap_dead start 2024-02-23T00:51:34.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f28f8062cc0 msgr2=0x7f28f8065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:34.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f28f8062cc0 0x7f28f8065180 secure :-1 s=READY pgs=3125 cs=0 l=1 rev1=1 rx=0x7f28fc004ee0 tx=0x7f28fc00b040).stop 2024-02-23T00:51:34.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f290c0f7400 msgr2=0x7f290c10cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:34.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f290c0f7400 0x7f290c10cfa0 secure :-1 s=READY pgs=3978 cs=0 l=1 rev1=1 rx=0x7f290800e970 tx=0x7f290800b040).stop 2024-02-23T00:51:34.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 shutdown_connections 2024-02-23T00:51:34.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f28f8062cc0 0x7f28f8065180 unknown :-1 s=CLOSED pgs=3125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:34.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 1 --2- 172.21.15.47:0/835569047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f290c0f7400 0x7f290c10cfa0 unknown :-1 s=CLOSED pgs=3978 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:34.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 >> 172.21.15.47:0/835569047 conn(0x7f290c0f2820 msgr2=0x7f290c0f4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:34.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 shutdown_connections 2024-02-23T00:51:34.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 1 -- 172.21.15.47:0/835569047 wait complete. 2024-02-23T00:51:34.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:34.693+0000 7f291359d700 1 librados: shutdown 2024-02-23T00:51:34.700 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:51:34.700 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:51:34.701 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:51:34.701 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:51:34.701 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:51:34.701 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:51:34.701 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:51:34.701 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:51:34.746 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:51:34.748 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:34.748 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:51:34.849 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:51:34.879 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:34.967 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:32.758325+0000 mgr.y (mgr.14517) 511 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:51:34.968 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:33.282159+0000 osd.7 (osd.7) 273 : cluster [DBG] 18.2 scrub starts 2024-02-23T00:51:34.968 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:33.285045+0000 osd.7 (osd.7) 274 : cluster [DBG] 18.2 scrub ok 2024-02-23T00:51:34.968 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:33.343588+0000 mgr.y (mgr.14517) 512 : cluster [DBG] pgmap v850: 152 pgs: 8 unknown, 144 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:51:34.968 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:34.263804+0000 mon.a (mon.0) 3004 : cluster [DBG] osdmap e565: 8 total, 8 up, 7 in 2024-02-23T00:51:34.998 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:32.758325+0000 mgr.y (mgr.14517) 511 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:51:34.998 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:33.282159+0000 osd.7 (osd.7) 273 : cluster [DBG] 18.2 scrub starts 2024-02-23T00:51:34.998 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:33.285045+0000 osd.7 (osd.7) 274 : cluster [DBG] 18.2 scrub ok 2024-02-23T00:51:34.998 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:33.343588+0000 mgr.y (mgr.14517) 512 : cluster [DBG] pgmap v850: 152 pgs: 8 unknown, 144 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:51:34.998 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:34.263804+0000 mon.a (mon.0) 3004 : cluster [DBG] osdmap e565: 8 total, 8 up, 7 in 2024-02-23T00:51:34.998 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:32.758325+0000 mgr.y (mgr.14517) 511 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:51:34.998 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:33.282159+0000 osd.7 (osd.7) 273 : cluster [DBG] 18.2 scrub starts 2024-02-23T00:51:34.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:33.285045+0000 osd.7 (osd.7) 274 : cluster [DBG] 18.2 scrub ok 2024-02-23T00:51:34.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:33.343588+0000 mgr.y (mgr.14517) 512 : cluster [DBG] pgmap v850: 152 pgs: 8 unknown, 144 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:51:34.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:34.263804+0000 mon.a (mon.0) 3004 : cluster [DBG] osdmap e565: 8 total, 8 up, 7 in 2024-02-23T00:51:35.003 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:51:35.006 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:35.007 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:51:35.108 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:51:35.136 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:35.233 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:51:35.233 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:51:35.235 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:35.236 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:51:35.337 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:51:35.363 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:35.382 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:51:35.385 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:35.386 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:51:35.487 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:51:35.515 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:35.764 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:51:35.766 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:35.767 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:51:35.868 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:51:35.895 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:35.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:34.199418+0000 osd.6 (osd.6) 219 : cluster [DBG] 5.2 scrub starts 2024-02-23T00:51:35.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:34.200303+0000 osd.6 (osd.6) 220 : cluster [DBG] 5.2 scrub ok 2024-02-23T00:51:35.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:34.526150+0000 osd.5 (osd.5) 248 : cluster [DBG] 2.16 scrub starts 2024-02-23T00:51:35.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:34.529008+0000 osd.5 (osd.5) 249 : cluster [DBG] 2.16 scrub ok 2024-02-23T00:51:35.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:34.548806+0000 mon.b (mon.1) 378 : audit [DBG] from='client.? 172.21.15.47:0/835569047' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:51:35.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:34.573750+0000 osd.3 (osd.3) 266 : cluster [DBG] 5.0 scrub starts 2024-02-23T00:51:35.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:34.577218+0000 osd.3 (osd.3) 267 : cluster [DBG] 5.0 scrub ok 2024-02-23T00:51:35.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:35.217819+0000 osd.6 (osd.6) 221 : cluster [DBG] 2.17 scrub starts 2024-02-23T00:51:35.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:35.218579+0000 osd.6 (osd.6) 222 : cluster [DBG] 2.17 scrub ok 2024-02-23T00:51:35.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:35.246467+0000 mon.a (mon.0) 3005 : cluster [DBG] osdmap e566: 8 total, 8 up, 7 in 2024-02-23T00:51:35.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:35.253799+0000 mon.a (mon.0) 3006 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-97","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:51:35.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:35.344065+0000 mgr.y (mgr.14517) 513 : cluster [DBG] pgmap v853: 184 pgs: 40 unknown, 144 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:35.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:35.630842+0000 osd.0 (osd.0) 235 : cluster [DBG] 5.8 scrub starts 2024-02-23T00:51:35.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:35.631592+0000 osd.0 (osd.0) 236 : cluster [DBG] 5.8 scrub ok 2024-02-23T00:51:35.994 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:51:35.997 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:35.997 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:51:36.006 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:34.199418+0000 osd.6 (osd.6) 219 : cluster [DBG] 5.2 scrub starts 2024-02-23T00:51:36.007 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:34.200303+0000 osd.6 (osd.6) 220 : cluster [DBG] 5.2 scrub ok 2024-02-23T00:51:36.007 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:34.526150+0000 osd.5 (osd.5) 248 : cluster [DBG] 2.16 scrub starts 2024-02-23T00:51:36.007 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:34.529008+0000 osd.5 (osd.5) 249 : cluster [DBG] 2.16 scrub ok 2024-02-23T00:51:36.007 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:34.548806+0000 mon.b (mon.1) 378 : audit [DBG] from='client.? 172.21.15.47:0/835569047' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:51:36.007 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:34.573750+0000 osd.3 (osd.3) 266 : cluster [DBG] 5.0 scrub starts 2024-02-23T00:51:36.007 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:34.577218+0000 osd.3 (osd.3) 267 : cluster [DBG] 5.0 scrub ok 2024-02-23T00:51:36.007 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:35.217819+0000 osd.6 (osd.6) 221 : cluster [DBG] 2.17 scrub starts 2024-02-23T00:51:36.007 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:35.218579+0000 osd.6 (osd.6) 222 : cluster [DBG] 2.17 scrub ok 2024-02-23T00:51:36.007 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:35.246467+0000 mon.a (mon.0) 3005 : cluster [DBG] osdmap e566: 8 total, 8 up, 7 in 2024-02-23T00:51:36.007 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:35.253799+0000 mon.a (mon.0) 3006 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-97","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:51:36.007 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:35.344065+0000 mgr.y (mgr.14517) 513 : cluster [DBG] pgmap v853: 184 pgs: 40 unknown, 144 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:36.007 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:35.630842+0000 osd.0 (osd.0) 235 : cluster [DBG] 5.8 scrub starts 2024-02-23T00:51:36.007 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:35.631592+0000 osd.0 (osd.0) 236 : cluster [DBG] 5.8 scrub ok 2024-02-23T00:51:36.008 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:34.199418+0000 osd.6 (osd.6) 219 : cluster [DBG] 5.2 scrub starts 2024-02-23T00:51:36.008 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:34.200303+0000 osd.6 (osd.6) 220 : cluster [DBG] 5.2 scrub ok 2024-02-23T00:51:36.008 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:34.526150+0000 osd.5 (osd.5) 248 : cluster [DBG] 2.16 scrub starts 2024-02-23T00:51:36.008 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:34.529008+0000 osd.5 (osd.5) 249 : cluster [DBG] 2.16 scrub ok 2024-02-23T00:51:36.008 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:34.548806+0000 mon.b (mon.1) 378 : audit [DBG] from='client.? 172.21.15.47:0/835569047' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:51:36.008 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:34.573750+0000 osd.3 (osd.3) 266 : cluster [DBG] 5.0 scrub starts 2024-02-23T00:51:36.008 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:34.577218+0000 osd.3 (osd.3) 267 : cluster [DBG] 5.0 scrub ok 2024-02-23T00:51:36.008 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:35.217819+0000 osd.6 (osd.6) 221 : cluster [DBG] 2.17 scrub starts 2024-02-23T00:51:36.008 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:35.218579+0000 osd.6 (osd.6) 222 : cluster [DBG] 2.17 scrub ok 2024-02-23T00:51:36.008 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:35.246467+0000 mon.a (mon.0) 3005 : cluster [DBG] osdmap e566: 8 total, 8 up, 7 in 2024-02-23T00:51:36.008 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:35.253799+0000 mon.a (mon.0) 3006 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-97","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:51:36.008 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:35.344065+0000 mgr.y (mgr.14517) 513 : cluster [DBG] pgmap v853: 184 pgs: 40 unknown, 144 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:36.009 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:35.630842+0000 osd.0 (osd.0) 235 : cluster [DBG] 5.8 scrub starts 2024-02-23T00:51:36.009 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:35.631592+0000 osd.0 (osd.0) 236 : cluster [DBG] 5.8 scrub ok 2024-02-23T00:51:36.098 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:51:36.127 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:36.200 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:51:36.202 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:36.203 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:51:36.262 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:51:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-02-23T00:51:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-02-23T00:51:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-02-23T00:51:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.24769.0:11 8.d 8:b44cf66e:test-rados-api-smithi047-82428-5::foo:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected e111)", 2024-02-23T00:51:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:00.902153+0000", 2024-02-23T00:51:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 515.35380675700003, 2024-02-23T00:51:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.040007200999999999, 2024-02-23T00:51:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.24769", 2024-02-23T00:51:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/820656190", 2024-02-23T00:51:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 11 2024-02-23T00:51:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.902153+0000", 2024-02-23T00:51:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.902153+0000", 2024-02-23T00:51:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:51:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.902152+0000", 2024-02-23T00:51:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6229999999999999e-06 2024-02-23T00:51:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.902158+0000", 2024-02-23T00:51:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0861e-05 2024-02-23T00:51:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.902169+0000", 2024-02-23T00:51:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6389999999999997e-06 2024-02-23T00:51:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.902175+0000", 2024-02-23T00:51:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.002966186 2024-02-23T00:51:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.905141+0000", 2024-02-23T00:51:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000110677 2024-02-23T00:51:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.905252+0000", 2024-02-23T00:51:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00019720999999999999 2024-02-23T00:51:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 2,7", 2024-02-23T00:51:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.905449+0000", 2024-02-23T00:51:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0024371100000000001 2024-02-23T00:51:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:51:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.907886+0000", 2024-02-23T00:51:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0042338150000000001 2024-02-23T00:51:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:51:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.912120+0000", 2024-02-23T00:51:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029932031000000001 2024-02-23T00:51:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:51:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.942052+0000", 2024-02-23T00:51:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.2205000000000005e-05 2024-02-23T00:51:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.942114+0000", 2024-02-23T00:51:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6199999999999998e-05 2024-02-23T00:51:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:36.270 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:00.942160+0000", 2024-02-23T00:51:36.270 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.040007200999999999 2024-02-23T00:51:36.270 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.270 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:36.270 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.270 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.270 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.270 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.24775.0:2059 20.10 e112/109 20:0c23f028:test-rados-api-smithi047-82457-16::big:head v 112'2, mlcod=0'0)", 2024-02-23T00:51:36.270 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:01.336044+0000", 2024-02-23T00:51:36.270 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 514.91991631500002, 2024-02-23T00:51:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.070215984999999995, 2024-02-23T00:51:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.336044+0000", 2024-02-23T00:51:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.336044+0000", 2024-02-23T00:51:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:51:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.272 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:36.272 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.336043+0000", 2024-02-23T00:51:36.272 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.031567871999999997 2024-02-23T00:51:36.272 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.272 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.272 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:36.272 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.367610+0000", 2024-02-23T00:51:36.272 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2143000000000001e-05 2024-02-23T00:51:36.272 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.272 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.272 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:36.272 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.367633+0000", 2024-02-23T00:51:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9549999999999998e-06 2024-02-23T00:51:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.367641+0000", 2024-02-23T00:51:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.5167999999999998e-05 2024-02-23T00:51:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.367697+0000", 2024-02-23T00:51:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7206e-05 2024-02-23T00:51:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.367724+0000", 2024-02-23T00:51:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038475098999999999 2024-02-23T00:51:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.406199+0000", 2024-02-23T00:51:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0538999999999998e-05 2024-02-23T00:51:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.406260+0000", 2024-02-23T00:51:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.070215984999999995 2024-02-23T00:51:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(osd.6.0:245 20.12 e112/109 -22:4d27b8a4:::temp_20.12_0_24277_1:head v 0'0, mlcod=0'0)", 2024-02-23T00:51:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:01.452184+0000", 2024-02-23T00:51:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 514.80377590499995, 2024-02-23T00:51:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.016245961, 2024-02-23T00:51:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.452184+0000", 2024-02-23T00:51:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.452184+0000", 2024-02-23T00:51:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999976 2024-02-23T00:51:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.452182+0000", 2024-02-23T00:51:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.002515387 2024-02-23T00:51:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.454697+0000", 2024-02-23T00:51:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4111999999999999e-05 2024-02-23T00:51:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.454721+0000", 2024-02-23T00:51:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.5599999999999994e-06 2024-02-23T00:51:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.454730+0000", 2024-02-23T00:51:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1328000000000001e-05 2024-02-23T00:51:36.278 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.278 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.278 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:36.278 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.454781+0000", 2024-02-23T00:51:36.278 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9772999999999999e-05 2024-02-23T00:51:36.278 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.278 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.278 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:36.278 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.454801+0000", 2024-02-23T00:51:36.278 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013588922 2024-02-23T00:51:36.278 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.278 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.278 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.468390+0000", 2024-02-23T00:51:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0067000000000001e-05 2024-02-23T00:51:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.468430+0000", 2024-02-23T00:51:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.016245961 2024-02-23T00:51:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(osd.6.0:246 20.12 e112/109 -22:4d27b8a4:::temp_20.12_0_24277_1:head v 0'0, mlcod=0'0)", 2024-02-23T00:51:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:01.470926+0000", 2024-02-23T00:51:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 514.78503341700002, 2024-02-23T00:51:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0201228, 2024-02-23T00:51:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.470926+0000", 2024-02-23T00:51:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.470926+0000", 2024-02-23T00:51:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:51:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.470925+0000", 2024-02-23T00:51:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0031871600000000001 2024-02-23T00:51:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.474112+0000", 2024-02-23T00:51:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.6032000000000001e-05 2024-02-23T00:51:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.474138+0000", 2024-02-23T00:51:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.5290000000000008e-06 2024-02-23T00:51:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.474146+0000", 2024-02-23T00:51:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1718000000000002e-05 2024-02-23T00:51:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.474198+0000", 2024-02-23T00:51:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7005999999999998e-05 2024-02-23T00:51:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.474235+0000", 2024-02-23T00:51:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.016765140000000001 2024-02-23T00:51:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.491000+0000", 2024-02-23T00:51:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8930000000000001e-05 2024-02-23T00:51:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.491049+0000", 2024-02-23T00:51:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0201228 2024-02-23T00:51:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(osd.6.0:247 20.12 e112/109 -22:4d27b8a4:::temp_20.12_0_24277_1:head v 0'0, mlcod=0'0)", 2024-02-23T00:51:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:01.492074+0000", 2024-02-23T00:51:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 514.76388553799995, 2024-02-23T00:51:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017188332000000001, 2024-02-23T00:51:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.492074+0000", 2024-02-23T00:51:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.492074+0000", 2024-02-23T00:51:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:51:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.492073+0000", 2024-02-23T00:51:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00353272 2024-02-23T00:51:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.495606+0000", 2024-02-23T00:51:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2286999999999999e-05 2024-02-23T00:51:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.495628+0000", 2024-02-23T00:51:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0412e-05 2024-02-23T00:51:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.495638+0000", 2024-02-23T00:51:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.6077999999999998e-05 2024-02-23T00:51:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.495724+0000", 2024-02-23T00:51:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.8677999999999999e-05 2024-02-23T00:51:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.495753+0000", 2024-02-23T00:51:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013480469 2024-02-23T00:51:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.509234+0000", 2024-02-23T00:51:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.9009e-05 2024-02-23T00:51:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.509263+0000", 2024-02-23T00:51:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017188332000000001 2024-02-23T00:51:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.25117.0:5 29.d e112/109 29:b3c717c4:::4.obj:head v 112'24, mlcod=111'22)", 2024-02-23T00:51:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:01.872497+0000", 2024-02-23T00:51:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 514.38346308400003, 2024-02-23T00:51:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015898065999999999, 2024-02-23T00:51:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.872497+0000", 2024-02-23T00:51:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.872497+0000", 2024-02-23T00:51:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:51:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.872495+0000", 2024-02-23T00:51:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3344e-05 2024-02-23T00:51:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.872509+0000", 2024-02-23T00:51:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9233999999999999e-05 2024-02-23T00:51:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.872528+0000", 2024-02-23T00:51:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5700000000000004e-06 2024-02-23T00:51:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.872535+0000", 2024-02-23T00:51:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3579999999999999e-05 2024-02-23T00:51:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.872579+0000", 2024-02-23T00:51:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8539999999999999e-05 2024-02-23T00:51:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.872598+0000", 2024-02-23T00:51:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015771070000000002 2024-02-23T00:51:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.888369+0000", 2024-02-23T00:51:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.6197999999999999e-05 2024-02-23T00:51:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.888395+0000", 2024-02-23T00:51:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015898065999999999 2024-02-23T00:51:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.24775.0:2074 20.1a 20:5de0a470:test-rados-api-smithi047-82457-17::big3:head [writefull 0~25165828] snapc 0=[] ondisk+write+known_if_redirected e112)", 2024-02-23T00:51:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:01.911456+0000", 2024-02-23T00:51:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 514.34450401900006, 2024-02-23T00:51:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.135054068, 2024-02-23T00:51:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.24775", 2024-02-23T00:51:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1191937413", 2024-02-23T00:51:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 2074 2024-02-23T00:51:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.911456+0000", 2024-02-23T00:51:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.911456+0000", 2024-02-23T00:51:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:51:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.911454+0000", 2024-02-23T00:51:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.045409616999999999 2024-02-23T00:51:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.956864+0000", 2024-02-23T00:51:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8623000000000001e-05 2024-02-23T00:51:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.956882+0000", 2024-02-23T00:51:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6969999999999998e-06 2024-02-23T00:51:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.956888+0000", 2024-02-23T00:51:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.6355e-05 2024-02-23T00:51:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.956924+0000", 2024-02-23T00:51:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.2887999999999996e-05 2024-02-23T00:51:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.957017+0000", 2024-02-23T00:51:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7796999999999999e-05 2024-02-23T00:51:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0,7", 2024-02-23T00:51:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.957115+0000", 2024-02-23T00:51:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.060621657000000002 2024-02-23T00:51:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:51:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.017737+0000", 2024-02-23T00:51:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.468e-05 2024-02-23T00:51:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:51:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.017771+0000", 2024-02-23T00:51:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028684866 2024-02-23T00:51:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:51:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.046456+0000", 2024-02-23T00:51:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.1053000000000002e-05 2024-02-23T00:51:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.046487+0000", 2024-02-23T00:51:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2582000000000002e-05 2024-02-23T00:51:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.046510+0000", 2024-02-23T00:51:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.135054068 2024-02-23T00:51:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.24775.0:2076 20.1a 20:5de0a470:test-rados-api-smithi047-82457-17::big3:head [omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected e112)", 2024-02-23T00:51:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:02.049332+0000", 2024-02-23T00:51:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 514.20662825600004, 2024-02-23T00:51:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014523405, 2024-02-23T00:51:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.24775", 2024-02-23T00:51:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1191937413", 2024-02-23T00:51:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 2076 2024-02-23T00:51:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.049332+0000", 2024-02-23T00:51:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.049332+0000", 2024-02-23T00:51:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:51:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.049331+0000", 2024-02-23T00:51:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0026349609999999999 2024-02-23T00:51:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.051966+0000", 2024-02-23T00:51:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2293000000000001e-05 2024-02-23T00:51:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.051978+0000", 2024-02-23T00:51:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0080000000000002e-06 2024-02-23T00:51:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.051986+0000", 2024-02-23T00:51:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.5505e-05 2024-02-23T00:51:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.052021+0000", 2024-02-23T00:51:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3334000000000001e-05 2024-02-23T00:51:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.052075+0000", 2024-02-23T00:51:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0030188070000000001 2024-02-23T00:51:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0,7", 2024-02-23T00:51:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.055094+0000", 2024-02-23T00:51:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0080902790000000006 2024-02-23T00:51:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:51:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.063184+0000", 2024-02-23T00:51:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00051940000000000005 2024-02-23T00:51:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:51:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.063703+0000", 2024-02-23T00:51:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9100000000000002e-05 2024-02-23T00:51:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:51:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.063742+0000", 2024-02-23T00:51:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.8199999999999998e-05 2024-02-23T00:51:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.063801+0000", 2024-02-23T00:51:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4428999999999997e-05 2024-02-23T00:51:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.063855+0000", 2024-02-23T00:51:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014523405 2024-02-23T00:51:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.15099.0:2 34.6 34:602f83fe:::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e114)", 2024-02-23T00:51:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:03.223645+0000", 2024-02-23T00:51:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 513.03231482499996, 2024-02-23T00:51:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015291755000000001, 2024-02-23T00:51:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.15099", 2024-02-23T00:51:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/3592004948", 2024-02-23T00:51:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 2 2024-02-23T00:51:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.223645+0000", 2024-02-23T00:51:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.223645+0000", 2024-02-23T00:51:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:51:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.223643+0000", 2024-02-23T00:51:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5359999999999998e-06 2024-02-23T00:51:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.223653+0000", 2024-02-23T00:51:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9113000000000001e-05 2024-02-23T00:51:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.223672+0000", 2024-02-23T00:51:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.4780000000000008e-06 2024-02-23T00:51:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.223680+0000", 2024-02-23T00:51:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1610000000000001e-05 2024-02-23T00:51:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.223742+0000", 2024-02-23T00:51:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4283999999999994e-05 2024-02-23T00:51:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.223836+0000", 2024-02-23T00:51:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012437900000000001 2024-02-23T00:51:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1,2", 2024-02-23T00:51:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.223960+0000", 2024-02-23T00:51:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037544500000000002 2024-02-23T00:51:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:51:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.224336+0000", 2024-02-23T00:51:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013498499000000001 2024-02-23T00:51:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:51:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.237834+0000", 2024-02-23T00:51:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0010338649999999999 2024-02-23T00:51:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:51:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.238868+0000", 2024-02-23T00:51:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7762999999999997e-05 2024-02-23T00:51:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.238906+0000", 2024-02-23T00:51:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.0822999999999999e-05 2024-02-23T00:51:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.238937+0000", 2024-02-23T00:51:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015291755000000001 2024-02-23T00:51:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.24997.0:1 39.0s0 39:1e3e2c6e:test-rados-api-smithi047-82402-23::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e114)", 2024-02-23T00:51:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:03.229886+0000", 2024-02-23T00:51:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 513.02607410999997, 2024-02-23T00:51:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.126777119, 2024-02-23T00:51:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.24997", 2024-02-23T00:51:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1505701467", 2024-02-23T00:51:36.313 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1 2024-02-23T00:51:36.313 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.313 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:36.313 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.313 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:36.313 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.229886+0000", 2024-02-23T00:51:36.313 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:36.313 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.313 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.313 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:36.313 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.229886+0000", 2024-02-23T00:51:36.313 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:51:36.313 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.314 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.314 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:36.314 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.229884+0000", 2024-02-23T00:51:36.314 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4189999999999999e-06 2024-02-23T00:51:36.314 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.314 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.314 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:36.314 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.229891+0000", 2024-02-23T00:51:36.314 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.9709999999999999e-06 2024-02-23T00:51:36.314 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.314 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.314 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:36.314 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.229901+0000", 2024-02-23T00:51:36.314 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0509999999999997e-06 2024-02-23T00:51:36.315 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.315 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.315 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:36.315 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.229909+0000", 2024-02-23T00:51:36.315 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.2765e-05 2024-02-23T00:51:36.315 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.315 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.315 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:36.315 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.229961+0000", 2024-02-23T00:51:36.315 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.625e-06 2024-02-23T00:51:36.315 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.315 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.315 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:51:36.315 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.229965+0000", 2024-02-23T00:51:36.316 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1077668439999999 2024-02-23T00:51:36.316 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.316 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.316 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:36.316 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.337732+0000", 2024-02-23T00:51:36.316 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012659300000000001 2024-02-23T00:51:36.316 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.316 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.316 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:36.316 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.337859+0000", 2024-02-23T00:51:36.316 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0010284910000000001 2024-02-23T00:51:36.317 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.317 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.317 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:51:36.317 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.338887+0000", 2024-02-23T00:51:36.317 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00067548200000000004 2024-02-23T00:51:36.317 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.317 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.317 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:51:36.317 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.339563+0000", 2024-02-23T00:51:36.317 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017058336 2024-02-23T00:51:36.317 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.317 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.317 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:36.317 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.356621+0000", 2024-02-23T00:51:36.318 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1999999999999998e-05 2024-02-23T00:51:36.318 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.318 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.318 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:36.318 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.356663+0000", 2024-02-23T00:51:36.318 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.126777119 2024-02-23T00:51:36.318 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.318 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:36.318 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.318 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.318 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.318 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(osd.1.0:297 20.5 e140/109 -22:a70ec860:::temp_20.5_0_24170_1:head v 0'0, mlcod=112'1)", 2024-02-23T00:51:36.318 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:32.555598+0000", 2024-02-23T00:51:36.318 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 483.70036139400003, 2024-02-23T00:51:36.319 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.049529112, 2024-02-23T00:51:36.319 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:36.319 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:36.319 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:36.319 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.319 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:36.319 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.555598+0000", 2024-02-23T00:51:36.319 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:36.319 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.319 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.319 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:36.319 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.555598+0000", 2024-02-23T00:51:36.319 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:51:36.319 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.319 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.320 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:36.320 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.555597+0000", 2024-02-23T00:51:36.320 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.039391554000000002 2024-02-23T00:51:36.320 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.320 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.320 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:36.320 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.594989+0000", 2024-02-23T00:51:36.320 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9025000000000002e-05 2024-02-23T00:51:36.320 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.320 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.320 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:36.320 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.595008+0000", 2024-02-23T00:51:36.321 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7115000000000001e-05 2024-02-23T00:51:36.321 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.321 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.321 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:36.321 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.595025+0000", 2024-02-23T00:51:36.321 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.2793000000000001e-05 2024-02-23T00:51:36.323 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.323 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.323 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:36.323 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.595078+0000", 2024-02-23T00:51:36.323 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1537999999999999e-05 2024-02-23T00:51:36.323 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.323 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.323 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:36.323 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.595099+0000", 2024-02-23T00:51:36.324 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0099891800000000003 2024-02-23T00:51:36.324 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.324 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.324 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:36.324 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.605088+0000", 2024-02-23T00:51:36.324 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9180000000000001e-05 2024-02-23T00:51:36.324 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.324 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.324 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:36.324 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.605128+0000", 2024-02-23T00:51:36.324 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.049529112 2024-02-23T00:51:36.325 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.325 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:36.325 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.325 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.325 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.325 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(osd.1.0:298 20.5 e140/109 -22:a70ec860:::temp_20.5_0_24170_1:head v 0'0, mlcod=112'1)", 2024-02-23T00:51:36.325 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:32.595037+0000", 2024-02-23T00:51:36.325 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 483.66092326199998, 2024-02-23T00:51:36.325 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.052500485999999999, 2024-02-23T00:51:36.325 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:36.325 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:36.325 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:36.325 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.326 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:36.326 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.595037+0000", 2024-02-23T00:51:36.326 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:36.326 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.326 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.326 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:36.326 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.595037+0000", 2024-02-23T00:51:36.326 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:51:36.326 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.326 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.326 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:36.326 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.595036+0000", 2024-02-23T00:51:36.326 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.042091349 2024-02-23T00:51:36.326 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.327 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.327 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:36.327 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.637127+0000", 2024-02-23T00:51:36.327 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6313000000000001e-05 2024-02-23T00:51:36.327 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.327 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.327 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:36.327 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.637143+0000", 2024-02-23T00:51:36.327 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4218000000000001e-05 2024-02-23T00:51:36.327 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.327 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.327 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:36.327 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.637167+0000", 2024-02-23T00:51:36.327 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2020999999999994e-05 2024-02-23T00:51:36.327 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.328 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.328 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:36.328 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.637240+0000", 2024-02-23T00:51:36.328 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0267000000000001e-05 2024-02-23T00:51:36.328 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.328 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.328 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:36.328 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.637260+0000", 2024-02-23T00:51:36.328 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010234715 2024-02-23T00:51:36.328 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.328 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.328 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:36.328 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.647494+0000", 2024-02-23T00:51:36.329 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.2564000000000001e-05 2024-02-23T00:51:36.329 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.329 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.329 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:36.329 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.647537+0000", 2024-02-23T00:51:36.329 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.052500485999999999 2024-02-23T00:51:36.329 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.329 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:36.329 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.329 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.329 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.329 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(osd.1.0:299 20.5 e140/109 -22:a70ec860:::temp_20.5_0_24170_1:head v 0'0, mlcod=112'1)", 2024-02-23T00:51:36.329 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:32.637181+0000", 2024-02-23T00:51:36.329 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 483.618778819, 2024-02-23T00:51:36.329 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.039894883999999999, 2024-02-23T00:51:36.330 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:36.330 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:36.330 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:36.330 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.330 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:36.330 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.637181+0000", 2024-02-23T00:51:36.330 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:36.330 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.330 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.330 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:36.330 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.637181+0000", 2024-02-23T00:51:36.330 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:51:36.330 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.330 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.330 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:36.331 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.637180+0000", 2024-02-23T00:51:36.331 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029888125000000001 2024-02-23T00:51:36.331 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.331 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.331 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:36.331 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.667068+0000", 2024-02-23T00:51:36.331 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9562999999999999e-05 2024-02-23T00:51:36.331 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.331 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.331 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:36.331 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.667088+0000", 2024-02-23T00:51:36.331 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.0469999999999999e-06 2024-02-23T00:51:36.331 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.331 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.331 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:36.332 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.667097+0000", 2024-02-23T00:51:36.332 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2268000000000007e-05 2024-02-23T00:51:36.332 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.332 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.332 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:36.332 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.667169+0000", 2024-02-23T00:51:36.332 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7490999999999999e-05 2024-02-23T00:51:36.332 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.332 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.332 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:36.332 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.667187+0000", 2024-02-23T00:51:36.332 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0098505959999999997 2024-02-23T00:51:36.332 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.332 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.333 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:36.333 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.677037+0000", 2024-02-23T00:51:36.333 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.8726999999999997e-05 2024-02-23T00:51:36.333 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.333 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.333 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:36.333 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.677076+0000", 2024-02-23T00:51:36.333 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.039894883999999999 2024-02-23T00:51:36.333 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.333 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:36.333 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.333 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.333 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.333 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.24775.0:2080 20.5 e140/109 20:a70ec860:test-rados-api-smithi047-82457-17::big3.copy:head v 140'3, mlcod=112'1)", 2024-02-23T00:51:36.334 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:32.667110+0000", 2024-02-23T00:51:36.334 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 483.58884996699999, 2024-02-23T00:51:36.334 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.024788509, 2024-02-23T00:51:36.334 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:36.334 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:36.334 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:36.334 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.334 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:36.334 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.667110+0000", 2024-02-23T00:51:36.334 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:36.334 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.334 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.334 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:36.335 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.667110+0000", 2024-02-23T00:51:36.335 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:51:36.335 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.335 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.335 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:36.335 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.667109+0000", 2024-02-23T00:51:36.335 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.018720831 2024-02-23T00:51:36.335 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.335 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.335 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:36.335 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.685829+0000", 2024-02-23T00:51:36.335 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0949e-05 2024-02-23T00:51:36.335 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.335 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.336 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:36.336 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.685840+0000", 2024-02-23T00:51:36.336 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9130000000000003e-06 2024-02-23T00:51:36.336 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.336 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.336 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:36.336 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.685848+0000", 2024-02-23T00:51:36.336 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3827999999999999e-05 2024-02-23T00:51:36.336 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.336 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.336 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:36.337 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.685912+0000", 2024-02-23T00:51:36.337 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.12e-05 2024-02-23T00:51:36.337 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.337 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.337 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:36.337 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.685933+0000", 2024-02-23T00:51:36.337 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0059192359999999996 2024-02-23T00:51:36.337 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.337 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.337 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:36.337 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.691852+0000", 2024-02-23T00:51:36.337 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5884000000000003e-05 2024-02-23T00:51:36.337 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.337 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.337 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:36.338 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.691898+0000", 2024-02-23T00:51:36.338 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.024788509 2024-02-23T00:51:36.338 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.338 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:36.338 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.338 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.338 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.338 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.31603.0:7 79.6s0 79:6ddaff94:test-rados-api-smithi047-82487-12::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e145)", 2024-02-23T00:51:36.338 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:37.207111+0000", 2024-02-23T00:51:36.338 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 479.04884895700002, 2024-02-23T00:51:36.338 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.050316774000000002, 2024-02-23T00:51:36.338 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:36.339 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:36.339 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:36.339 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.31603", 2024-02-23T00:51:36.339 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1361361966", 2024-02-23T00:51:36.339 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 7 2024-02-23T00:51:36.339 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.339 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:36.339 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.339 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:36.339 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.207111+0000", 2024-02-23T00:51:36.339 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:36.339 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.339 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.340 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:36.340 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.207111+0000", 2024-02-23T00:51:36.340 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:51:36.340 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.340 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.340 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:36.340 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.207109+0000", 2024-02-23T00:51:36.340 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.571e-06 2024-02-23T00:51:36.340 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.340 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.340 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:36.340 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.207117+0000", 2024-02-23T00:51:36.341 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0654e-05 2024-02-23T00:51:36.341 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.341 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.341 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:36.341 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.207127+0000", 2024-02-23T00:51:36.341 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0709999999999997e-06 2024-02-23T00:51:36.341 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.341 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.341 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:36.341 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.207134+0000", 2024-02-23T00:51:36.341 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010019800000000001 2024-02-23T00:51:36.341 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.341 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.341 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:36.341 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.207235+0000", 2024-02-23T00:51:36.341 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013806100000000001 2024-02-23T00:51:36.342 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.342 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.342 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:36.342 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.207373+0000", 2024-02-23T00:51:36.342 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00043899 2024-02-23T00:51:36.342 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.342 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.342 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:51:36.342 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.207812+0000", 2024-02-23T00:51:36.342 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00089506700000000004 2024-02-23T00:51:36.342 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.342 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.342 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:51:36.343 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.208707+0000", 2024-02-23T00:51:36.343 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.048651597999999997 2024-02-23T00:51:36.343 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.343 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.343 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:36.343 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.257358+0000", 2024-02-23T00:51:36.343 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.9276000000000006e-05 2024-02-23T00:51:36.343 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.343 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.343 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:36.344 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.257428+0000", 2024-02-23T00:51:36.344 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.050316774000000002 2024-02-23T00:51:36.344 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.344 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:36.344 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.344 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.344 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.344 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.31603.0:9 79.6s0 79:6ddaff94:test-rados-api-smithi047-82487-12::foo:head [writefull 0~128] snapc 0=[] ondisk+write+known_if_redirected e146)", 2024-02-23T00:51:36.344 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:38.402504+0000", 2024-02-23T00:51:36.344 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 477.85345542200002, 2024-02-23T00:51:36.344 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036230714999999997, 2024-02-23T00:51:36.344 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:36.345 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:36.345 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:36.345 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.31603", 2024-02-23T00:51:36.345 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1361361966", 2024-02-23T00:51:36.345 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 9 2024-02-23T00:51:36.345 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.345 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:36.345 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.345 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:36.345 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.402504+0000", 2024-02-23T00:51:36.345 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:36.345 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.345 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.345 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:36.346 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.402504+0000", 2024-02-23T00:51:36.346 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:51:36.346 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.346 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.346 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:36.346 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.402503+0000", 2024-02-23T00:51:36.346 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.6512e-05 2024-02-23T00:51:36.346 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.346 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.346 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:36.346 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.402529+0000", 2024-02-23T00:51:36.346 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7428999999999999e-05 2024-02-23T00:51:36.346 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.346 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.346 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:36.347 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.402547+0000", 2024-02-23T00:51:36.347 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2077e-05 2024-02-23T00:51:36.347 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.347 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.347 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:36.347 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.402559+0000", 2024-02-23T00:51:36.347 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000108826 2024-02-23T00:51:36.347 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.347 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.347 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:36.347 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.402668+0000", 2024-02-23T00:51:36.347 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010388299999999999 2024-02-23T00:51:36.347 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.347 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.347 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:36.347 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.402771+0000", 2024-02-23T00:51:36.348 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00051973999999999998 2024-02-23T00:51:36.348 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.348 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.348 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:51:36.348 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.403291+0000", 2024-02-23T00:51:36.348 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00049441100000000005 2024-02-23T00:51:36.348 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.348 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.348 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:51:36.348 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.403786+0000", 2024-02-23T00:51:36.348 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034898881 2024-02-23T00:51:36.348 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.348 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.348 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:36.348 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.438685+0000", 2024-02-23T00:51:36.349 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0618000000000002e-05 2024-02-23T00:51:36.349 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.349 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.349 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:36.349 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.438735+0000", 2024-02-23T00:51:36.349 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036230714999999997 2024-02-23T00:51:36.349 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.349 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:36.349 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.349 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.349 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.349 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.24311.0:1282 4.4 4:22d26bf9:::data_loggenerations_metadata:head [watch ping cookie 94587070406528 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370)", 2024-02-23T00:51:36.349 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:47:27.941616+0000", 2024-02-23T00:51:36.349 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 248.31434375800001, 2024-02-23T00:51:36.349 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.30363403799999999, 2024-02-23T00:51:36.350 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:36.350 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:51:36.350 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:36.350 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.24311", 2024-02-23T00:51:36.350 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/3723202188", 2024-02-23T00:51:36.350 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1282 2024-02-23T00:51:36.350 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.350 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:36.350 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.350 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:36.350 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.941616+0000", 2024-02-23T00:51:36.350 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:36.350 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.350 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.350 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:36.350 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.941616+0000", 2024-02-23T00:51:36.350 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:51:36.351 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.351 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.351 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:36.351 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.941615+0000", 2024-02-23T00:51:36.351 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3249999999999998e-06 2024-02-23T00:51:36.351 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.351 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.351 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:36.351 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.941621+0000", 2024-02-23T00:51:36.351 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1557e-05 2024-02-23T00:51:36.351 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.351 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.351 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:36.351 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.941642+0000", 2024-02-23T00:51:36.351 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3829999999999997e-06 2024-02-23T00:51:36.352 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.352 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.352 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:36.352 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.941648+0000", 2024-02-23T00:51:36.352 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029746900000000002 2024-02-23T00:51:36.352 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.352 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.352 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:36.352 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.941946+0000", 2024-02-23T00:51:36.352 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3587000000000001e-05 2024-02-23T00:51:36.352 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.352 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.352 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:51:36.352 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.941960+0000", 2024-02-23T00:51:36.352 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.29858716899999999 2024-02-23T00:51:36.352 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.353 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.353 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:36.353 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:28.240547+0000", 2024-02-23T00:51:36.353 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001598099 2024-02-23T00:51:36.353 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.353 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.353 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for degraded object", 2024-02-23T00:51:36.353 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:28.242145+0000", 2024-02-23T00:51:36.353 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0029042780000000002 2024-02-23T00:51:36.353 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.353 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.353 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:36.353 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:28.245049+0000", 2024-02-23T00:51:36.353 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.7250999999999995e-05 2024-02-23T00:51:36.353 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.353 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.354 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:36.354 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:28.245136+0000", 2024-02-23T00:51:36.354 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000113793 2024-02-23T00:51:36.354 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.354 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.354 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:36.354 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:28.245250+0000", 2024-02-23T00:51:36.354 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.30363403799999999 2024-02-23T00:51:36.354 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.354 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:36.354 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.354 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.354 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.355 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.24311.0:1287 4.4 4:22d26bf9:::data_loggenerations_metadata:head [watch reconnect cookie 94587070406528 gen 2] snapc 0=[] ondisk+write+known_if_redirected e375)", 2024-02-23T00:51:36.355 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:47:27.942624+0000", 2024-02-23T00:51:36.355 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 248.31333535100001, 2024-02-23T00:51:36.355 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.30870664599999997, 2024-02-23T00:51:36.355 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:36.355 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:51:36.355 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:36.355 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.24311", 2024-02-23T00:51:36.355 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/3723202188", 2024-02-23T00:51:36.355 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1287 2024-02-23T00:51:36.355 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.355 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:36.355 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.355 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:36.355 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.942624+0000", 2024-02-23T00:51:36.355 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:36.356 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.356 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.356 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:36.356 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.942624+0000", 2024-02-23T00:51:36.356 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:51:36.356 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.356 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.356 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:36.356 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.942623+0000", 2024-02-23T00:51:36.356 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9109999999999993e-06 2024-02-23T00:51:36.356 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.356 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.356 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:36.357 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.942631+0000", 2024-02-23T00:51:36.357 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5931999999999999e-05 2024-02-23T00:51:36.357 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.357 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.357 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:36.357 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.942647+0000", 2024-02-23T00:51:36.357 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.4319999999999999e-06 2024-02-23T00:51:36.357 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.357 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.357 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:36.357 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.942655+0000", 2024-02-23T00:51:36.357 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6074e-05 2024-02-23T00:51:36.357 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.358 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.358 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:36.358 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.942702+0000", 2024-02-23T00:51:36.358 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3239999999999999e-06 2024-02-23T00:51:36.358 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.358 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.358 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:51:36.358 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.942708+0000", 2024-02-23T00:51:36.358 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.29946395100000001 2024-02-23T00:51:36.358 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.358 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.358 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:36.358 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:28.242172+0000", 2024-02-23T00:51:36.358 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.5534999999999998e-05 2024-02-23T00:51:36.358 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.358 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.359 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for degraded object", 2024-02-23T00:51:36.359 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:28.242207+0000", 2024-02-23T00:51:36.359 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0030798739999999998 2024-02-23T00:51:36.359 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.359 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.359 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:36.359 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:28.245287+0000", 2024-02-23T00:51:36.359 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5687999999999998e-05 2024-02-23T00:51:36.359 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.359 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.359 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:36.359 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:28.245363+0000", 2024-02-23T00:51:36.359 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0059682490000000001 2024-02-23T00:51:36.359 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.359 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.360 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:36.360 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:28.251331+0000", 2024-02-23T00:51:36.360 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.30870664599999997 2024-02-23T00:51:36.360 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.360 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:36.360 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.360 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.360 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.360 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(osd.6.47:62 62.7 e494/375 62:e796d8e9:test-rados-api-smithi047-82465-78::b74d4df232b43241f65ea39108ac0a45561a59cf:head v 494'70, mlcod=0'0)", 2024-02-23T00:51:36.360 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:50:23.248041+0000", 2024-02-23T00:51:36.360 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 73.007918914000001, 2024-02-23T00:51:36.360 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0031083170000000002, 2024-02-23T00:51:36.360 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:36.360 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:36.360 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:36.361 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.361 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:36.361 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:50:23.248041+0000", 2024-02-23T00:51:36.361 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:36.361 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.361 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.361 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:36.361 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:50:23.248041+0000", 2024-02-23T00:51:36.361 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999995 2024-02-23T00:51:36.361 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.361 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.361 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:36.361 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:50:23.248041+0000", 2024-02-23T00:51:36.361 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4213e-05 2024-02-23T00:51:36.362 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.362 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.362 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:36.362 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:50:23.248065+0000", 2024-02-23T00:51:36.362 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8329999999999996e-06 2024-02-23T00:51:36.362 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.362 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.362 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:36.362 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:50:23.248072+0000", 2024-02-23T00:51:36.362 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0069999999999997e-06 2024-02-23T00:51:36.362 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.362 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.362 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:36.362 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:50:23.248077+0000", 2024-02-23T00:51:36.362 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7543999999999998e-05 2024-02-23T00:51:36.362 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.363 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.363 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:36.363 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:50:23.248114+0000", 2024-02-23T00:51:36.363 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0871999999999999e-05 2024-02-23T00:51:36.363 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.363 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.363 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:36.363 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:50:23.248135+0000", 2024-02-23T00:51:36.363 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0029785459999999999 2024-02-23T00:51:36.363 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.363 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.363 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:36.364 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:50:23.251114+0000", 2024-02-23T00:51:36.364 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.5701000000000003e-05 2024-02-23T00:51:36.364 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.364 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.364 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:36.364 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:50:23.251149+0000", 2024-02-23T00:51:36.364 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0031083170000000002 2024-02-23T00:51:36.364 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.364 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:36.364 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.364 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.364 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.364 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.15270.0:7718 62.1d 62:bd8d0095:test-rados-api-smithi047-82465-89::propagate-base-tier-error:head [omap-cmp,omap-set-vals] snapc 0=[] ondisk+write+ignore_cache+ignore_overlay+known_if_redirected e539)", 2024-02-23T00:51:36.365 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:51:08.248873+0000", 2024-02-23T00:51:36.365 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 28.00708693, 2024-02-23T00:51:36.365 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0034122480000000001, 2024-02-23T00:51:36.365 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:36.365 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:51:36.365 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:36.365 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.365 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:36.365 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:51:08.248873+0000", 2024-02-23T00:51:36.365 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:36.365 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.365 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.365 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:36.366 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:51:08.248873+0000", 2024-02-23T00:51:36.366 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999976 2024-02-23T00:51:36.366 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.366 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.366 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:36.366 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:51:08.248870+0000", 2024-02-23T00:51:36.366 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4918e-05 2024-02-23T00:51:36.367 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.367 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.367 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:36.367 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:51:08.248885+0000", 2024-02-23T00:51:36.367 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0539000000000001e-05 2024-02-23T00:51:36.367 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.367 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.367 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:36.367 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:51:08.248906+0000", 2024-02-23T00:51:36.367 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1509999999999993e-06 2024-02-23T00:51:36.367 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.367 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.368 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:36.368 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:51:08.248915+0000", 2024-02-23T00:51:36.368 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9061000000000003e-05 2024-02-23T00:51:36.368 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.368 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.368 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:36.368 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:51:08.248974+0000", 2024-02-23T00:51:36.368 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1719999999999998e-06 2024-02-23T00:51:36.368 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.368 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.369 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op must wait for map", 2024-02-23T00:51:36.369 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:51:08.248979+0000", 2024-02-23T00:51:36.369 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.002122685 2024-02-23T00:51:36.369 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.369 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.369 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:36.369 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:51:08.251102+0000", 2024-02-23T00:51:36.369 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.2075000000000002e-05 2024-02-23T00:51:36.369 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.369 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.369 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:36.369 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:51:08.251194+0000", 2024-02-23T00:51:36.369 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001091269 2024-02-23T00:51:36.369 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:36.370 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:36.370 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:36.370 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:51:08.252285+0000", 2024-02-23T00:51:36.370 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0034122480000000001 2024-02-23T00:51:36.370 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.370 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:36.370 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.370 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:36.370 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:36.370 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:51:36.370 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:51:36.398 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:36.521 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:51:36.524 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:36.524 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:51:36.625 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:51:36.653 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:36.982 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.6 dump_ops_in_flight 2024-02-23T00:51:37.002 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:51:37.005 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:37.006 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:51:37.026 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-23T00:51:37.107 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:51:37.135 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:37.271 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:51:37.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:36.247696+0000 mon.a (mon.0) 3007 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-97","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:51:37.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:36.247803+0000 mon.a (mon.0) 3008 : cluster [DBG] osdmap e567: 8 total, 8 up, 7 in 2024-02-23T00:51:37.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:36.649445+0000 osd.0 (osd.0) 237 : cluster [DBG] 5.b scrub starts 2024-02-23T00:51:37.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:36.650300+0000 osd.0 (osd.0) 238 : cluster [DBG] 5.b scrub ok 2024-02-23T00:51:37.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:36.247696+0000 mon.a (mon.0) 3007 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-97","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:51:37.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:36.247803+0000 mon.a (mon.0) 3008 : cluster [DBG] osdmap e567: 8 total, 8 up, 7 in 2024-02-23T00:51:37.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:36.649445+0000 osd.0 (osd.0) 237 : cluster [DBG] 5.b scrub starts 2024-02-23T00:51:37.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:36.650300+0000 osd.0 (osd.0) 238 : cluster [DBG] 5.b scrub ok 2024-02-23T00:51:37.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:36.247696+0000 mon.a (mon.0) 3007 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-97","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:51:37.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:36.247803+0000 mon.a (mon.0) 3008 : cluster [DBG] osdmap e567: 8 total, 8 up, 7 in 2024-02-23T00:51:37.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:36.649445+0000 osd.0 (osd.0) 237 : cluster [DBG] 5.b scrub starts 2024-02-23T00:51:37.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:36.650300+0000 osd.0 (osd.0) 238 : cluster [DBG] 5.b scrub ok 2024-02-23T00:51:37.764 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:51:38.288 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:51:38.291 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:38.292 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:51:38.393 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:51:38.403 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:36.412409+0000 osd.1 (osd.1) 237 : cluster [DBG] 2.4 scrub starts 2024-02-23T00:51:38.404 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:36.415335+0000 osd.1 (osd.1) 238 : cluster [DBG] 2.4 scrub ok 2024-02-23T00:51:38.404 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:36.655699+0000 osd.3 (osd.3) 268 : cluster [DBG] 2.e scrub starts 2024-02-23T00:51:38.404 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:36.656568+0000 osd.3 (osd.3) 269 : cluster [DBG] 2.e scrub ok 2024-02-23T00:51:38.404 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:37.254073+0000 mon.a (mon.0) 3009 : cluster [DBG] osdmap e568: 8 total, 8 up, 7 in 2024-02-23T00:51:38.404 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:37.257805+0000 mon.a (mon.0) 3010 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-97-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:51:38.404 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:37.344588+0000 mgr.y (mgr.14517) 514 : cluster [DBG] pgmap v856: 216 pgs: 3 creating+peering, 29 unknown, 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:51:38.421 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:38.441 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:51:38.444 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:38.444 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:51:38.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.528+0000 7fcb05601700 1 -- 172.21.15.47:0/359237685 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcb000fcac0 msgr2=0x7fcb000fcea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:38.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.528+0000 7fcb05601700 1 --2- 172.21.15.47:0/359237685 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcb000fcac0 0x7fcb000fcea0 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 rx=0x7fcaf0004870 tx=0x7fcaf0009cb0).stop 2024-02-23T00:51:38.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.529+0000 7fcb05601700 1 -- 172.21.15.47:0/359237685 shutdown_connections 2024-02-23T00:51:38.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.529+0000 7fcb05601700 1 --2- 172.21.15.47:0/359237685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb000f7330 0x7fcb00103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:38.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.529+0000 7fcb05601700 1 --2- 172.21.15.47:0/359237685 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcb000fcac0 0x7fcb000fcea0 unknown :-1 s=CLOSED pgs=2737 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:38.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.529+0000 7fcb05601700 1 --2- 172.21.15.47:0/359237685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb000fd420 0x7fcb000f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:51:38.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.529+0000 7fcb05601700 1 -- 172.21.15.47:0/359237685 >> 172.21.15.47:0/359237685 conn(0x7fcb000f26f0 msgr2=0x7fcb000f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:38.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.529+0000 7fcb05601700 1 -- 172.21.15.47:0/359237685 shutdown_connections 2024-02-23T00:51:38.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.529+0000 7fcb05601700 1 -- 172.21.15.47:0/359237685 wait complete. 2024-02-23T00:51:38.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.529+0000 7fcb05601700 1 librados: starting msgr at 2024-02-23T00:51:38.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.529+0000 7fcb05601700 1 librados: starting objecter 2024-02-23T00:51:38.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.530+0000 7fcb05601700 1 Processor -- start 2024-02-23T00:51:38.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.530+0000 7fcb05601700 1 -- start start 2024-02-23T00:51:38.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.530+0000 7fcb05601700 1 librados: setting wanted keys 2024-02-23T00:51:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.530+0000 7fcb05601700 1 librados: calling monclient init 2024-02-23T00:51:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.530+0000 7fcb05601700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb000f7330 0x7fcb0010ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.530+0000 7fcb05601700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb000fcac0 0x7fcb0010d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.530+0000 7fcb05601700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcb000fd420 0x7fcb00107000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.530+0000 7fcb05601700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb001074f0 con 0x7fcb000fcac0 2024-02-23T00:51:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.530+0000 7fcb05601700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb00107660 con 0x7fcb000f7330 2024-02-23T00:51:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.530+0000 7fcb05601700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb00107820 con 0x7fcb000fd420 2024-02-23T00:51:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.530+0000 7fcafe7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb000fcac0 0x7fcb0010d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.530+0000 7fcafeffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb000f7330 0x7fcb0010ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.530+0000 7fcaff7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcb000fd420 0x7fcb00107000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:38.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.530+0000 7fcafe7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb000fcac0 0x7fcb0010d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40068/0 (socket says 172.21.15.47:40068) 2024-02-23T00:51:38.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.530+0000 7fcafe7fc700 1 -- 172.21.15.47:0/2331778496 learned_addr learned my addr 172.21.15.47:0/2331778496 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:51:38.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.531+0000 7fcafeffd700 1 -- 172.21.15.47:0/2331778496 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcb000fd420 msgr2=0x7fcb00107000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:38.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.531+0000 7fcafeffd700 1 --2- 172.21.15.47:0/2331778496 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcb000fd420 0x7fcb00107000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:38.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.531+0000 7fcafeffd700 1 -- 172.21.15.47:0/2331778496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb000fcac0 msgr2=0x7fcb0010d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:38.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.531+0000 7fcafeffd700 1 --2- 172.21.15.47:0/2331778496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb000fcac0 0x7fcb0010d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:38.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.531+0000 7fcafeffd700 1 -- 172.21.15.47:0/2331778496 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf80097c0 con 0x7fcb000f7330 2024-02-23T00:51:38.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.531+0000 7fcafe7fc700 1 --2- 172.21.15.47:0/2331778496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb000fcac0 0x7fcb0010d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:51:38.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.531+0000 7fcaff7fe700 1 --2- 172.21.15.47:0/2331778496 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcb000fd420 0x7fcb00107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:51:38.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.531+0000 7fcafeffd700 1 --2- 172.21.15.47:0/2331778496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb000f7330 0x7fcb0010ce60 secure :-1 s=READY pgs=3979 cs=0 l=1 rev1=1 rx=0x7fcaf0009da0 tx=0x7fcaf0009cb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:38.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.531+0000 7fcae7fff700 10 client.?.objecter ms_handle_connect 0x7fcb000f7330 2024-02-23T00:51:38.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.531+0000 7fcae7fff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:51:38.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.531+0000 7fcae7fff700 1 -- 172.21.15.47:0/2331778496 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf000e030 con 0x7fcb000f7330 2024-02-23T00:51:38.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.531+0000 7fcae7fff700 1 -- 172.21.15.47:0/2331778496 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fcaf0013d70 con 0x7fcb000f7330 2024-02-23T00:51:38.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.531+0000 7fcb05601700 1 -- 172.21.15.47:0/2331778496 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf0005040 con 0x7fcb000f7330 2024-02-23T00:51:38.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.531+0000 7fcae7fff700 1 -- 172.21.15.47:0/2331778496 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0018070 con 0x7fcb000f7330 2024-02-23T00:51:38.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.531+0000 7fcb05601700 10 client.43714.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:51:38.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.531+0000 7fcb05601700 1 -- 172.21.15.47:0/2331778496 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb00107cc0 con 0x7fcb000f7330 2024-02-23T00:51:38.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.531+0000 7fcb05601700 1 librados: init done 2024-02-23T00:51:38.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.532+0000 7fcb05601700 1 -- 172.21.15.47:0/2331778496 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb00107ee0 con 0x7fcb000f7330 2024-02-23T00:51:38.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.532+0000 7fcae7fff700 1 -- 172.21.15.47:0/2331778496 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fcaf0009240 con 0x7fcb000f7330 2024-02-23T00:51:38.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.533+0000 7fcae7fff700 1 --2- 172.21.15.47:0/2331778496 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcae8062cc0 0x7fcae8065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:38.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.533+0000 7fcae7fff700 1 -- 172.21.15.47:0/2331778496 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(569..569 src has 482..569) v4 ==== 9977+0+0 (secure 0 0 0) 0x7fcaf007ea90 con 0x7fcb000f7330 2024-02-23T00:51:38.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.533+0000 7fcae7fff700 10 client.43714.objecter ms_dispatch 0x7fcb00001550 osd_map(569..569 src has 482..569) v4 2024-02-23T00:51:38.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.533+0000 7fcae7fff700 3 client.43714.objecter handle_osd_map got epochs [569,569] > 0 2024-02-23T00:51:38.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.533+0000 7fcae7fff700 3 client.43714.objecter handle_osd_map decoding full epoch 569 2024-02-23T00:51:38.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.533+0000 7fcafe7fc700 1 --2- 172.21.15.47:0/2331778496 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcae8062cc0 0x7fcae8065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:38.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.533+0000 7fcae7fff700 20 client.43714.objecter dump_active .. 0 homeless 2024-02-23T00:51:38.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.533+0000 7fcafe7fc700 1 --2- 172.21.15.47:0/2331778496 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcae8062cc0 0x7fcae8065180 secure :-1 s=READY pgs=3126 cs=0 l=1 rev1=1 rx=0x7fcb001087a0 tx=0x7fcaf40090f0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:38.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.533+0000 7fcae7fff700 10 client.43714.objecter ms_handle_connect 0x7fcae8062cc0 2024-02-23T00:51:38.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.538+0000 7fcae7fff700 1 -- 172.21.15.47:0/2331778496 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fcaf004ddb0 con 0x7fcb000f7330 2024-02-23T00:51:38.545 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:51:38.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:36.412409+0000 osd.1 (osd.1) 237 : cluster [DBG] 2.4 scrub starts 2024-02-23T00:51:38.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:36.415335+0000 osd.1 (osd.1) 238 : cluster [DBG] 2.4 scrub ok 2024-02-23T00:51:38.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:36.655699+0000 osd.3 (osd.3) 268 : cluster [DBG] 2.e scrub starts 2024-02-23T00:51:38.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:36.656568+0000 osd.3 (osd.3) 269 : cluster [DBG] 2.e scrub ok 2024-02-23T00:51:38.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:37.254073+0000 mon.a (mon.0) 3009 : cluster [DBG] osdmap e568: 8 total, 8 up, 7 in 2024-02-23T00:51:38.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:37.257805+0000 mon.a (mon.0) 3010 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-97-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:51:38.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:37.344588+0000 mgr.y (mgr.14517) 514 : cluster [DBG] pgmap v856: 216 pgs: 3 creating+peering, 29 unknown, 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:51:38.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:36.412409+0000 osd.1 (osd.1) 237 : cluster [DBG] 2.4 scrub starts 2024-02-23T00:51:38.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:36.415335+0000 osd.1 (osd.1) 238 : cluster [DBG] 2.4 scrub ok 2024-02-23T00:51:38.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:36.655699+0000 osd.3 (osd.3) 268 : cluster [DBG] 2.e scrub starts 2024-02-23T00:51:38.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:36.656568+0000 osd.3 (osd.3) 269 : cluster [DBG] 2.e scrub ok 2024-02-23T00:51:38.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:37.254073+0000 mon.a (mon.0) 3009 : cluster [DBG] osdmap e568: 8 total, 8 up, 7 in 2024-02-23T00:51:38.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:37.257805+0000 mon.a (mon.0) 3010 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-97-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:51:38.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:37.344588+0000 mgr.y (mgr.14517) 514 : cluster [DBG] pgmap v856: 216 pgs: 3 creating+peering, 29 unknown, 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:51:38.582 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:38.590 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:51:38.612 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:51:38.614 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:38.615 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:51:38.615 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:51:38.716 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:51:38.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.725+0000 7fcb05601700 1 -- 172.21.15.47:0/2331778496 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcb001084e0 con 0x7fcae8062cc0 2024-02-23T00:51:38.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.740+0000 7fcae7fff700 1 -- 172.21.15.47:0/2331778496 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+547320 (secure 0 0 0) 0x7fcb001084e0 con 0x7fcae8062cc0 2024-02-23T00:51:38.744 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:51:38.745 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:38.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.744+0000 7fcb05601700 10 librados: watch_flush enter 2024-02-23T00:51:38.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.744+0000 7fcb05601700 10 librados: watch_flush exit 2024-02-23T00:51:38.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.744+0000 7fcb05601700 20 client.43714.objecter shutdown clearing up homeless session... 2024-02-23T00:51:38.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.744+0000 7fcb05601700 10 client.43714.objecter successfully canceled tick 2024-02-23T00:51:38.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.745+0000 7fcb05601700 1 -- 172.21.15.47:0/2331778496 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcae8062cc0 msgr2=0x7fcae8065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:38.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.745+0000 7fcb05601700 1 --2- 172.21.15.47:0/2331778496 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcae8062cc0 0x7fcae8065180 secure :-1 s=READY pgs=3126 cs=0 l=1 rev1=1 rx=0x7fcb001087a0 tx=0x7fcaf40090f0).stop 2024-02-23T00:51:38.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.745+0000 7fcb05601700 1 -- 172.21.15.47:0/2331778496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb000f7330 msgr2=0x7fcb0010ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:38.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.745+0000 7fcb05601700 1 --2- 172.21.15.47:0/2331778496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb000f7330 0x7fcb0010ce60 secure :-1 s=READY pgs=3979 cs=0 l=1 rev1=1 rx=0x7fcaf0009da0 tx=0x7fcaf0009cb0).stop 2024-02-23T00:51:38.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.745+0000 7fcb05601700 1 -- 172.21.15.47:0/2331778496 shutdown_connections 2024-02-23T00:51:38.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.745+0000 7fcb05601700 1 --2- 172.21.15.47:0/2331778496 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcae8062cc0 0x7fcae8065180 unknown :-1 s=CLOSED pgs=3126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:38.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.745+0000 7fcb05601700 1 --2- 172.21.15.47:0/2331778496 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcb000fd420 0x7fcb00107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:38.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.745+0000 7fcb05601700 1 --2- 172.21.15.47:0/2331778496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb000f7330 0x7fcb0010ce60 unknown :-1 s=CLOSED pgs=3979 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:38.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.745+0000 7fcb05601700 1 --2- 172.21.15.47:0/2331778496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb000fcac0 0x7fcb0010d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:38.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.745+0000 7fcb05601700 1 -- 172.21.15.47:0/2331778496 >> 172.21.15.47:0/2331778496 conn(0x7fcb000f26f0 msgr2=0x7fcb000faf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:38.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.745+0000 7fcb05601700 1 -- 172.21.15.47:0/2331778496 shutdown_connections 2024-02-23T00:51:38.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.745+0000 7fcb05601700 1 -- 172.21.15.47:0/2331778496 wait complete. 2024-02-23T00:51:38.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:38.745+0000 7fcb05601700 1 librados: shutdown 2024-02-23T00:51:38.757 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-02-23T00:51:38.916 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:51:38.916 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-02-23T00:51:38.916 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 0 2024-02-23T00:51:38.916 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:51:39.395 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:51:39.398 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:39.399 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:51:39.500 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:51:39.517 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:38.168841+0000 osd.7 (osd.7) 275 : cluster [DBG] 18.6 scrub starts 2024-02-23T00:51:39.517 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:38.169664+0000 osd.7 (osd.7) 276 : cluster [DBG] 18.6 scrub ok 2024-02-23T00:51:39.517 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:38.251103+0000 mon.a (mon.0) 3011 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:51:39.518 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:38.259720+0000 mon.a (mon.0) 3012 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-97-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:51:39.518 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:38.259836+0000 mon.a (mon.0) 3013 : cluster [DBG] osdmap e569: 8 total, 8 up, 7 in 2024-02-23T00:51:39.518 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:38.264218+0000 mon.a (mon.0) 3014 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-97", "tierpool": "test-rados-api-smithi047-82465-97-cache"}]: dispatch 2024-02-23T00:51:39.518 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:38.761564+0000 osd.4 (osd.4) 233 : cluster [DBG] 18.19 scrub starts 2024-02-23T00:51:39.518 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:38.762550+0000 osd.4 (osd.4) 234 : cluster [DBG] 18.19 scrub ok 2024-02-23T00:51:39.518 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:38.168841+0000 osd.7 (osd.7) 275 : cluster [DBG] 18.6 scrub starts 2024-02-23T00:51:39.518 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:38.169664+0000 osd.7 (osd.7) 276 : cluster [DBG] 18.6 scrub ok 2024-02-23T00:51:39.518 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:38.251103+0000 mon.a (mon.0) 3011 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:51:39.518 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:38.259720+0000 mon.a (mon.0) 3012 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-97-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:51:39.518 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:38.259836+0000 mon.a (mon.0) 3013 : cluster [DBG] osdmap e569: 8 total, 8 up, 7 in 2024-02-23T00:51:39.518 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:38.264218+0000 mon.a (mon.0) 3014 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-97", "tierpool": "test-rados-api-smithi047-82465-97-cache"}]: dispatch 2024-02-23T00:51:39.519 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:38.761564+0000 osd.4 (osd.4) 233 : cluster [DBG] 18.19 scrub starts 2024-02-23T00:51:39.519 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:38.762550+0000 osd.4 (osd.4) 234 : cluster [DBG] 18.19 scrub ok 2024-02-23T00:51:39.530 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:39.553 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:38.168841+0000 osd.7 (osd.7) 275 : cluster [DBG] 18.6 scrub starts 2024-02-23T00:51:39.553 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:38.169664+0000 osd.7 (osd.7) 276 : cluster [DBG] 18.6 scrub ok 2024-02-23T00:51:39.553 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:38.251103+0000 mon.a (mon.0) 3011 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:51:39.553 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:38.259720+0000 mon.a (mon.0) 3012 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-97-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:51:39.553 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:38.259836+0000 mon.a (mon.0) 3013 : cluster [DBG] osdmap e569: 8 total, 8 up, 7 in 2024-02-23T00:51:39.553 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:38.264218+0000 mon.a (mon.0) 3014 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-97", "tierpool": "test-rados-api-smithi047-82465-97-cache"}]: dispatch 2024-02-23T00:51:39.554 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:38.761564+0000 osd.4 (osd.4) 233 : cluster [DBG] 18.19 scrub starts 2024-02-23T00:51:39.554 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:38.762550+0000 osd.4 (osd.4) 234 : cluster [DBG] 18.19 scrub ok 2024-02-23T00:51:39.572 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.6 dump_blocked_ops 2024-02-23T00:51:39.838 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:51:40.270 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:38.726988+0000 mgr.y (mgr.14517) 515 : audit [DBG] from='client.43714 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:51:40.270 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:39.218720+0000 osd.7 (osd.7) 277 : cluster [DBG] 5.4 scrub starts 2024-02-23T00:51:40.270 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:39.221606+0000 osd.7 (osd.7) 278 : cluster [DBG] 5.4 scrub ok 2024-02-23T00:51:40.270 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:39.252643+0000 osd.6 (osd.6) 223 : cluster [DBG] 5.1f scrub starts 2024-02-23T00:51:40.270 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:39.253646+0000 osd.6 (osd.6) 224 : cluster [DBG] 5.1f scrub ok 2024-02-23T00:51:40.270 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:39.263015+0000 mon.a (mon.0) 3015 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-97", "tierpool": "test-rados-api-smithi047-82465-97-cache"}]': finished 2024-02-23T00:51:40.270 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:39.263091+0000 mon.a (mon.0) 3016 : cluster [DBG] osdmap e570: 8 total, 8 up, 7 in 2024-02-23T00:51:40.270 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:39.266311+0000 mon.a (mon.0) 3017 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-97", "overlaypool": "test-rados-api-smithi047-82465-97-cache"}]: dispatch 2024-02-23T00:51:40.270 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:39.345088+0000 mgr.y (mgr.14517) 516 : cluster [DBG] pgmap v859: 216 pgs: 3 creating+peering, 29 unknown, 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:51:40.271 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:38.726988+0000 mgr.y (mgr.14517) 515 : audit [DBG] from='client.43714 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:51:40.271 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:39.218720+0000 osd.7 (osd.7) 277 : cluster [DBG] 5.4 scrub starts 2024-02-23T00:51:40.271 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:39.221606+0000 osd.7 (osd.7) 278 : cluster [DBG] 5.4 scrub ok 2024-02-23T00:51:40.271 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:39.252643+0000 osd.6 (osd.6) 223 : cluster [DBG] 5.1f scrub starts 2024-02-23T00:51:40.271 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:39.253646+0000 osd.6 (osd.6) 224 : cluster [DBG] 5.1f scrub ok 2024-02-23T00:51:40.271 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:39.263015+0000 mon.a (mon.0) 3015 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-97", "tierpool": "test-rados-api-smithi047-82465-97-cache"}]': finished 2024-02-23T00:51:40.271 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:39.263091+0000 mon.a (mon.0) 3016 : cluster [DBG] osdmap e570: 8 total, 8 up, 7 in 2024-02-23T00:51:40.271 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:39.266311+0000 mon.a (mon.0) 3017 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-97", "overlaypool": "test-rados-api-smithi047-82465-97-cache"}]: dispatch 2024-02-23T00:51:40.271 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:39.345088+0000 mgr.y (mgr.14517) 516 : cluster [DBG] pgmap v859: 216 pgs: 3 creating+peering, 29 unknown, 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:51:40.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:38.726988+0000 mgr.y (mgr.14517) 515 : audit [DBG] from='client.43714 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:51:40.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:39.218720+0000 osd.7 (osd.7) 277 : cluster [DBG] 5.4 scrub starts 2024-02-23T00:51:40.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:39.221606+0000 osd.7 (osd.7) 278 : cluster [DBG] 5.4 scrub ok 2024-02-23T00:51:40.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:39.252643+0000 osd.6 (osd.6) 223 : cluster [DBG] 5.1f scrub starts 2024-02-23T00:51:40.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:39.253646+0000 osd.6 (osd.6) 224 : cluster [DBG] 5.1f scrub ok 2024-02-23T00:51:40.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:39.263015+0000 mon.a (mon.0) 3015 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-97", "tierpool": "test-rados-api-smithi047-82465-97-cache"}]': finished 2024-02-23T00:51:40.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:39.263091+0000 mon.a (mon.0) 3016 : cluster [DBG] osdmap e570: 8 total, 8 up, 7 in 2024-02-23T00:51:40.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:39.266311+0000 mon.a (mon.0) 3017 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-97", "overlaypool": "test-rados-api-smithi047-82465-97-cache"}]: dispatch 2024-02-23T00:51:40.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:39.345088+0000 mgr.y (mgr.14517) 516 : cluster [DBG] pgmap v859: 216 pgs: 3 creating+peering, 29 unknown, 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:51:40.557 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:51:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:51:40] "GET /metrics HTTP/1.1" 200 189781 "" "Prometheus/2.33.4" 2024-02-23T00:51:41.105 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":857,"stamp":"2024-02-23T00:51:38.255811+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5801,"num_objects":226,"num_object_clones":0,"num_object_copies":678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":225,"num_whiteouts":0,"num_read":2270,"num_read_kb":1988,"num_write":914,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":2626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1049,"ondisk_log_size":1049,"up":561,"acting":561,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":469,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":2170672,"kb_used_data":16376,"kb_used_omap":0,"kb_used_meta":2154240,"kb_avail":654045392,"statfs":{"total":671965249536,"available":669742481408,"internally_reserved":0,"allocated":16769024,"data_stored":6603634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2205941760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":5,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.994664"},"pg_stats":[{"pgid":"231.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:37.245210+0000","last_change":"2024-02-23T00:51:37.245210+0000","last_active":"2024-02-23T00:51:37.245210+0000","last_peered":"2024-02-23T00:51:37.245210+0000","last_clean":"2024-02-23T00:51:37.245210+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:37.245210+0000","last_undegraded":"2024-02-23T00:51:37.245210+0000","last_fullsized":"2024-02-23T00:51:37.245210+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_clean_scrub_stamp":"2024-02-23T00:51:37.245210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"230.1f","version":"0'0","reported_seq":9,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.266438+0000","last_change":"2024-02-23T00:51:36.266438+0000","last_active":"2024-02-23T00:51:36.266438+0000","last_peered":"2024-02-23T00:51:36.266438+0000","last_clean":"2024-02-23T00:51:36.266438+0000","last_became_active":"2024-02-23T00:51:36.266152+0000","last_became_peered":"2024-02-23T00:51:36.266152+0000","last_unstale":"2024-02-23T00:51:36.266438+0000","last_undegraded":"2024-02-23T00:51:36.266438+0000","last_fullsized":"2024-02-23T00:51:36.266438+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":566,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.241253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"231.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:37.245210+0000","last_change":"2024-02-23T00:51:37.245210+0000","last_active":"2024-02-23T00:51:37.245210+0000","last_peered":"2024-02-23T00:51:37.245210+0000","last_clean":"2024-02-23T00:51:37.245210+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:37.245210+0000","last_undegraded":"2024-02-23T00:51:37.245210+0000","last_fullsized":"2024-02-23T00:51:37.245210+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_clean_scrub_stamp":"2024-02-23T00:51:37.245210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"230.1e","version":"0'0","reported_seq":9,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.260727+0000","last_change":"2024-02-23T00:51:36.260727+0000","last_active":"2024-02-23T00:51:36.260727+0000","last_peered":"2024-02-23T00:51:36.260727+0000","last_clean":"2024-02-23T00:51:36.260727+0000","last_became_active":"2024-02-23T00:51:36.260443+0000","last_became_peered":"2024-02-23T00:51:36.260443+0000","last_unstale":"2024-02-23T00:51:36.260727+0000","last_undegraded":"2024-02-23T00:51:36.260727+0000","last_fullsized":"2024-02-23T00:51:36.260727+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":566,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.241253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"231.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:37.245210+0000","last_change":"2024-02-23T00:51:37.245210+0000","last_active":"2024-02-23T00:51:37.245210+0000","last_peered":"2024-02-23T00:51:37.245210+0000","last_clean":"2024-02-23T00:51:37.245210+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:37.245210+0000","last_undegraded":"2024-02-23T00:51:37.245210+0000","last_fullsized":"2024-02-23T00:51:37.245210+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_clean_scrub_stamp":"2024-02-23T00:51:37.245210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"230.1d","version":"0'0","reported_seq":10,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.266564+0000","last_change":"2024-02-23T00:51:36.264974+0000","last_active":"2024-02-23T00:51:36.266564+0000","last_peered":"2024-02-23T00:51:36.266564+0000","last_clean":"2024-02-23T00:51:36.266564+0000","last_became_active":"2024-02-23T00:51:36.260003+0000","last_became_peered":"2024-02-23T00:51:36.260003+0000","last_unstale":"2024-02-23T00:51:36.266564+0000","last_undegraded":"2024-02-23T00:51:36.266564+0000","last_fullsized":"2024-02-23T00:51:36.266564+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":566,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.241253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"231.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:37.245210+0000","last_change":"2024-02-23T00:51:37.245210+0000","last_active":"2024-02-23T00:51:37.245210+0000","last_peered":"2024-02-23T00:51:37.245210+0000","last_clean":"2024-02-23T00:51:37.245210+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:37.245210+0000","last_undegraded":"2024-02-23T00:51:37.245210+0000","last_fullsized":"2024-02-23T00:51:37.245210+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_clean_scrub_stamp":"2024-02-23T00:51:37.245210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"230.1c","version":"0'0","reported_seq":9,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.263792+0000","last_change":"2024-02-23T00:51:36.263792+0000","last_active":"2024-02-23T00:51:36.263792+0000","last_peered":"2024-02-23T00:51:36.263792+0000","last_clean":"2024-02-23T00:51:36.263792+0000","last_became_active":"2024-02-23T00:51:36.263520+0000","last_became_peered":"2024-02-23T00:51:36.263520+0000","last_unstale":"2024-02-23T00:51:36.263792+0000","last_undegraded":"2024-02-23T00:51:36.263792+0000","last_fullsized":"2024-02-23T00:51:36.263792+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":566,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.241253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"231.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:37.245210+0000","last_change":"2024-02-23T00:51:37.245210+0000","last_active":"2024-02-23T00:51:37.245210+0000","last_peered":"2024-02-23T00:51:37.245210+0000","last_clean":"2024-02-23T00:51:37.245210+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:37.245210+0000","last_undegraded":"2024-02-23T00:51:37.245210+0000","last_fullsized":"2024-02-23T00:51:37.245210+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_clean_scrub_stamp":"2024-02-23T00:51:37.245210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"230.1b","version":"0'0","reported_seq":10,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.265633+0000","last_change":"2024-02-23T00:51:36.260356+0000","last_active":"2024-02-23T00:51:36.265633+0000","last_peered":"2024-02-23T00:51:36.265633+0000","last_clean":"2024-02-23T00:51:36.265633+0000","last_became_active":"2024-02-23T00:51:36.260044+0000","last_became_peered":"2024-02-23T00:51:36.260044+0000","last_unstale":"2024-02-23T00:51:36.265633+0000","last_undegraded":"2024-02-23T00:51:36.265633+0000","last_fullsized":"2024-02-23T00:51:36.265633+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":566,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.241253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"231.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:37.245210+0000","last_change":"2024-02-23T00:51:37.245210+0000","last_active":"2024-02-23T00:51:37.245210+0000","last_peered":"2024-02-23T00:51:37.245210+0000","last_clean":"2024-02-23T00:51:37.245210+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:37.245210+0000","last_undegraded":"2024-02-23T00:51:37.245210+0000","last_fullsized":"2024-02-23T00:51:37.245210+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_clean_scrub_stamp":"2024-02-23T00:51:37.245210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"230.1a","version":"0'0","reported_seq":9,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.258567+0000","last_change":"2024-02-23T00:51:36.258567+0000","last_active":"2024-02-23T00:51:36.258567+0000","last_peered":"2024-02-23T00:51:36.258567+0000","last_clean":"2024-02-23T00:51:36.258567+0000","last_became_active":"2024-02-23T00:51:36.258192+0000","last_became_peered":"2024-02-23T00:51:36.258192+0000","last_unstale":"2024-02-23T00:51:36.258567+0000","last_undegraded":"2024-02-23T00:51:36.258567+0000","last_fullsized":"2024-02-23T00:51:36.258567+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":566,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.241253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"231.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:37.245210+0000","last_change":"2024-02-23T00:51:37.245210+0000","last_active":"2024-02-23T00:51:37.245210+0000","last_peered":"2024-02-23T00:51:37.245210+0000","last_clean":"2024-02-23T00:51:37.245210+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:37.245210+0000","last_undegraded":"2024-02-23T00:51:37.245210+0000","last_fullsized":"2024-02-23T00:51:37.245210+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_clean_scrub_stamp":"2024-02-23T00:51:37.245210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"230.19","version":"0'0","reported_seq":9,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.253229+0000","last_change":"2024-02-23T00:51:36.253229+0000","last_active":"2024-02-23T00:51:36.253229+0000","last_peered":"2024-02-23T00:51:36.253229+0000","last_clean":"2024-02-23T00:51:36.253229+0000","last_became_active":"2024-02-23T00:51:36.252925+0000","last_became_peered":"2024-02-23T00:51:36.252925+0000","last_unstale":"2024-02-23T00:51:36.253229+0000","last_undegraded":"2024-02-23T00:51:36.253229+0000","last_fullsized":"2024-02-23T00:51:36.253229+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":566,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.241253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"231.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:37.245210+0000","last_change":"2024-02-23T00:51:37.245210+0000","last_active":"2024-02-23T00:51:37.245210+0000","last_peered":"2024-02-23T00:51:37.245210+0000","last_clean":"2024-02-23T00:51:37.245210+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:37.245210+0000","last_undegraded":"2024-02-23T00:51:37.245210+0000","last_fullsized":"2024-02-23T00:51:37.245210+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_clean_scrub_stamp":"2024-02-23T00:51:37.245210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"230.18","version":"0'0","reported_seq":9,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.262708+0000","last_change":"2024-02-23T00:51:36.262708+0000","last_active":"2024-02-23T00:51:36.262708+0000","last_peered":"2024-02-23T00:51:36.262708+0000","last_clean":"2024-02-23T00:51:36.262708+0000","last_became_active":"2024-02-23T00:51:36.262351+0000","last_became_peered":"2024-02-23T00:51:36.262351+0000","last_unstale":"2024-02-23T00:51:36.262708+0000","last_undegraded":"2024-02-23T00:51:36.262708+0000","last_fullsized":"2024-02-23T00:51:36.262708+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":566,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.241253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"231.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:37.245210+0000","last_change":"2024-02-23T00:51:37.245210+0000","last_active":"2024-02-23T00:51:37.245210+0000","last_peered":"2024-02-23T00:51:37.245210+0000","last_clean":"2024-02-23T00:51:37.245210+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:37.245210+0000","last_undegraded":"2024-02-23T00:51:37.245210+0000","last_fullsized":"2024-02-23T00:51:37.245210+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_clean_scrub_stamp":"2024-02-23T00:51:37.245210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"230.17","version":"0'0","reported_seq":9,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.265779+0000","last_change":"2024-02-23T00:51:36.265779+0000","last_active":"2024-02-23T00:51:36.265779+0000","last_peered":"2024-02-23T00:51:36.265779+0000","last_clean":"2024-02-23T00:51:36.265779+0000","last_became_active":"2024-02-23T00:51:36.265483+0000","last_became_peered":"2024-02-23T00:51:36.265483+0000","last_unstale":"2024-02-23T00:51:36.265779+0000","last_undegraded":"2024-02-23T00:51:36.265779+0000","last_fullsized":"2024-02-23T00:51:36.265779+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":566,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.241253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"231.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:51:37.245210+0000","last_change":"2024-02-23T00:51:37.254445+0000","last_active":"2024-02-23T00:51:37.245210+0000","last_peered":"2024-02-23T00:51:37.245210+0000","last_clean":"2024-02-23T00:51:37.245210+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:37.245210+0000","last_undegraded":"2024-02-23T00:51:37.245210+0000","last_fullsized":"2024-02-23T00:51:37.245210+0000","mapping_epoch":568,"log_start":"0'0","ondisk_log_start":"0'0","created":568,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_clean_scrub_stamp":"2024-02-23T00:51:37.245210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"230.16","version":"0'0","reported_seq":10,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.259055+0000","last_change":"2024-02-23T00:51:36.258824+0000","last_active":"2024-02-23T00:51:36.259055+0000","last_peered":"2024-02-23T00:51:36.259055+0000","last_clean":"2024-02-23T00:51:36.259055+0000","last_became_active":"2024-02-23T00:51:36.258565+0000","last_became_peered":"2024-02-23T00:51:36.258565+0000","last_unstale":"2024-02-23T00:51:36.259055+0000","last_undegraded":"2024-02-23T00:51:36.259055+0000","last_fullsized":"2024-02-23T00:51:36.259055+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":566,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.241253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"231.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:37.245210+0000","last_change":"2024-02-23T00:51:37.245210+0000","last_active":"2024-02-23T00:51:37.245210+0000","last_peered":"2024-02-23T00:51:37.245210+0000","last_clean":"2024-02-23T00:51:37.245210+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:37.245210+0000","last_undegraded":"2024-02-23T00:51:37.245210+0000","last_fullsized":"2024-02-23T00:51:37.245210+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_clean_scrub_stamp":"2024-02-23T00:51:37.245210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"230.15","version":"0'0","reported_seq":9,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.261777+0000","last_change":"2024-02-23T00:51:36.261777+0000","last_active":"2024-02-23T00:51:36.261777+0000","last_peered":"2024-02-23T00:51:36.261777+0000","last_clean":"2024-02-23T00:51:36.261777+0000","last_became_active":"2024-02-23T00:51:36.261480+0000","last_became_peered":"2024-02-23T00:51:36.261480+0000","last_unstale":"2024-02-23T00:51:36.261777+0000","last_undegraded":"2024-02-23T00:51:36.261777+0000","last_fullsized":"2024-02-23T00:51:36.261777+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":566,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.241253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"231.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:37.245210+0000","last_change":"2024-02-23T00:51:37.245210+0000","last_active":"2024-02-23T00:51:37.245210+0000","last_peered":"2024-02-23T00:51:37.245210+0000","last_clean":"2024-02-23T00:51:37.245210+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:37.245210+0000","last_undegraded":"2024-02-23T00:51:37.245210+0000","last_fullsized":"2024-02-23T00:51:37.245210+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_clean_scrub_stamp":"2024-02-23T00:51:37.245210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"230.14","version":"0'0","reported_seq":9,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.261984+0000","last_change":"2024-02-23T00:51:36.261984+0000","last_active":"2024-02-23T00:51:36.261984+0000","last_peered":"2024-02-23T00:51:36.261984+0000","last_clean":"2024-02-23T00:51:36.261984+0000","last_became_active":"2024-02-23T00:51:36.261713+0000","last_became_peered":"2024-02-23T00:51:36.261713+0000","last_unstale":"2024-02-23T00:51:36.261984+0000","last_undegraded":"2024-02-23T00:51:36.261984+0000","last_fullsized":"2024-02-23T00:51:36.261984+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":566,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.241253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"231.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:37.245210+0000","last_change":"2024-02-23T00:51:37.245210+0000","last_active":"2024-02-23T00:51:37.245210+0000","last_peered":"2024-02-23T00:51:37.245210+0000","last_clean":"2024-02-23T00:51:37.245210+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:37.245210+0000","last_undegraded":"2024-02-23T00:51:37.245210+0000","last_fullsized":"2024-02-23T00:51:37.245210+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_clean_scrub_stamp":"2024-02-23T00:51:37.245210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"230.13","version":"0'0","reported_seq":9,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.267347+0000","last_change":"2024-02-23T00:51:36.267347+0000","last_active":"2024-02-23T00:51:36.267347+0000","last_peered":"2024-02-23T00:51:36.267347+0000","last_clean":"2024-02-23T00:51:36.267347+0000","last_became_active":"2024-02-23T00:51:36.267052+0000","last_became_peered":"2024-02-23T00:51:36.267052+0000","last_unstale":"2024-02-23T00:51:36.267347+0000","last_undegraded":"2024-02-23T00:51:36.267347+0000","last_fullsized":"2024-02-23T00:51:36.267347+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":566,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.241253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"231.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:37.245210+0000","last_change":"2024-02-23T00:51:37.245210+0000","last_active":"2024-02-23T00:51:37.245210+0000","last_peered":"2024-02-23T00:51:37.245210+0000","last_clean":"2024-02-23T00:51:37.245210+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:37.245210+0000","last_undegraded":"2024-02-23T00:51:37.245210+0000","last_fullsized":"2024-02-23T00:51:37.245210+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_clean_scrub_stamp":"2024-02-23T00:51:37.245210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"230.12","version":"0'0","reported_seq":10,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.259390+0000","last_change":"2024-02-23T00:51:36.257374+0000","last_active":"2024-02-23T00:51:36.259390+0000","last_peered":"2024-02-23T00:51:36.259390+0000","last_clean":"2024-02-23T00:51:36.259390+0000","last_became_active":"2024-02-23T00:51:36.256953+0000","last_became_peered":"2024-02-23T00:51:36.256953+0000","last_unstale":"2024-02-23T00:51:36.259390+0000","last_undegraded":"2024-02-23T00:51:36.259390+0000","last_fullsized":"2024-02-23T00:51:36.259390+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":566,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.241253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"231.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:37.245210+0000","last_change":"2024-02-23T00:51:37.245210+0000","last_active":"2024-02-23T00:51:37.245210+0000","last_peered":"2024-02-23T00:51:37.245210+0000","last_clean":"2024-02-23T00:51:37.245210+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:37.245210+0000","last_undegraded":"2024-02-23T00:51:37.245210+0000","last_fullsized":"2024-02-23T00:51:37.245210+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_clean_scrub_stamp":"2024-02-23T00:51:37.245210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"230.11","version":"0'0","reported_seq":9,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.259619+0000","last_change":"2024-02-23T00:51:36.259619+0000","last_active":"2024-02-23T00:51:36.259619+0000","last_peered":"2024-02-23T00:51:36.259619+0000","last_clean":"2024-02-23T00:51:36.259619+0000","last_became_active":"2024-02-23T00:51:36.259296+0000","last_became_peered":"2024-02-23T00:51:36.259296+0000","last_unstale":"2024-02-23T00:51:36.259619+0000","last_undegraded":"2024-02-23T00:51:36.259619+0000","last_fullsized":"2024-02-23T00:51:36.259619+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":566,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.241253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"231.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:37.245210+0000","last_change":"2024-02-23T00:51:37.245210+0000","last_active":"2024-02-23T00:51:37.245210+0000","last_peered":"2024-02-23T00:51:37.245210+0000","last_clean":"2024-02-23T00:51:37.245210+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:37.245210+0000","last_undegraded":"2024-02-23T00:51:37.245210+0000","last_fullsized":"2024-02-23T00:51:37.245210+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_clean_scrub_stamp":"2024-02-23T00:51:37.245210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"230.10","version":"0'0","reported_seq":9,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.261562+0000","last_change":"2024-02-23T00:51:36.261562+0000","last_active":"2024-02-23T00:51:36.261562+0000","last_peered":"2024-02-23T00:51:36.261562+0000","last_clean":"2024-02-23T00:51:36.261562+0000","last_became_active":"2024-02-23T00:51:36.260718+0000","last_became_peered":"2024-02-23T00:51:36.260718+0000","last_unstale":"2024-02-23T00:51:36.261562+0000","last_undegraded":"2024-02-23T00:51:36.261562+0000","last_fullsized":"2024-02-23T00:51:36.261562+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":566,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.241253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"231.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:51:37.245210+0000","last_change":"2024-02-23T00:51:37.254140+0000","last_active":"2024-02-23T00:51:37.245210+0000","last_peered":"2024-02-23T00:51:37.245210+0000","last_clean":"2024-02-23T00:51:37.245210+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:37.245210+0000","last_undegraded":"2024-02-23T00:51:37.245210+0000","last_fullsized":"2024-02-23T00:51:37.245210+0000","mapping_epoch":568,"log_start":"0'0","ondisk_log_start":"0'0","created":568,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_clean_scrub_stamp":"2024-02-23T00:51:37.245210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"230.f","version":"0'0","reported_seq":9,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.263010+0000","last_change":"2024-02-23T00:51:36.263010+0000","last_active":"2024-02-23T00:51:36.263010+0000","last_peered":"2024-02-23T00:51:36.263010+0000","last_clean":"2024-02-23T00:51:36.263010+0000","last_became_active":"2024-02-23T00:51:36.262728+0000","last_became_peered":"2024-02-23T00:51:36.262728+0000","last_unstale":"2024-02-23T00:51:36.263010+0000","last_undegraded":"2024-02-23T00:51:36.263010+0000","last_fullsized":"2024-02-23T00:51:36.263010+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":566,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.241253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"231.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:37.245210+0000","last_change":"2024-02-23T00:51:37.245210+0000","last_active":"2024-02-23T00:51:37.245210+0000","last_peered":"2024-02-23T00:51:37.245210+0000","last_clean":"2024-02-23T00:51:37.245210+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:37.245210+0000","last_undegraded":"2024-02-23T00:51:37.245210+0000","last_fullsized":"2024-02-23T00:51:37.245210+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_clean_scrub_stamp":"2024-02-23T00:51:37.245210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"230.e","version":"0'0","reported_seq":9,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.259771+0000","last_change":"2024-02-23T00:51:36.259771+0000","last_active":"2024-02-23T00:51:36.259771+0000","last_peered":"2024-02-23T00:51:36.259771+0000","last_clean":"2024-02-23T00:51:36.259771+0000","last_became_active":"2024-02-23T00:51:36.259492+0000","last_became_peered":"2024-02-23T00:51:36.259492+0000","last_unstale":"2024-02-23T00:51:36.259771+0000","last_undegraded":"2024-02-23T00:51:36.259771+0000","last_fullsized":"2024-02-23T00:51:36.259771+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":566,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.241253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"231.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:37.245210+0000","last_change":"2024-02-23T00:51:37.245210+0000","last_active":"2024-02-23T00:51:37.245210+0000","last_peered":"2024-02-23T00:51:37.245210+0000","last_clean":"2024-02-23T00:51:37.245210+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:37.245210+0000","last_undegraded":"2024-02-23T00:51:37.245210+0000","last_fullsized":"2024-02-23T00:51:37.245210+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_clean_scrub_stamp":"2024-02-23T00:51:37.245210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"230.d","version":"0'0","reported_seq":9,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.257914+0000","last_change":"2024-02-23T00:51:36.257914+0000","last_active":"2024-02-23T00:51:36.257914+0000","last_peered":"2024-02-23T00:51:36.257914+0000","last_clean":"2024-02-23T00:51:36.257914+0000","last_became_active":"2024-02-23T00:51:36.257632+0000","last_became_peered":"2024-02-23T00:51:36.257632+0000","last_unstale":"2024-02-23T00:51:36.257914+0000","last_undegraded":"2024-02-23T00:51:36.257914+0000","last_fullsized":"2024-02-23T00:51:36.257914+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":566,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.241253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"231.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:37.245210+0000","last_change":"2024-02-23T00:51:37.245210+0000","last_active":"2024-02-23T00:51:37.245210+0000","last_peered":"2024-02-23T00:51:37.245210+0000","last_clean":"2024-02-23T00:51:37.245210+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:37.245210+0000","last_undegraded":"2024-02-23T00:51:37.245210+0000","last_fullsized":"2024-02-23T00:51:37.245210+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_clean_scrub_stamp":"2024-02-23T00:51:37.245210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"230.c","version":"0'0","reported_seq":9,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.264000+0000","last_change":"2024-02-23T00:51:36.264000+0000","last_active":"2024-02-23T00:51:36.264000+0000","last_peered":"2024-02-23T00:51:36.264000+0000","last_clean":"2024-02-23T00:51:36.264000+0000","last_became_active":"2024-02-23T00:51:36.263733+0000","last_became_peered":"2024-02-23T00:51:36.263733+0000","last_unstale":"2024-02-23T00:51:36.264000+0000","last_undegraded":"2024-02-23T00:51:36.264000+0000","last_fullsized":"2024-02-23T00:51:36.264000+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":566,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.241253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"231.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-23T00:51:37.245210+0000","last_change":"2024-02-23T00:51:37.254952+0000","last_active":"2024-02-23T00:51:37.245210+0000","last_peered":"2024-02-23T00:51:37.245210+0000","last_clean":"2024-02-23T00:51:37.245210+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:37.245210+0000","last_undegraded":"2024-02-23T00:51:37.245210+0000","last_fullsized":"2024-02-23T00:51:37.245210+0000","mapping_epoch":568,"log_start":"0'0","ondisk_log_start":"0'0","created":568,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_clean_scrub_stamp":"2024-02-23T00:51:37.245210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"230.b","version":"0'0","reported_seq":9,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.256492+0000","last_change":"2024-02-23T00:51:36.256492+0000","last_active":"2024-02-23T00:51:36.256492+0000","last_peered":"2024-02-23T00:51:36.256492+0000","last_clean":"2024-02-23T00:51:36.256492+0000","last_became_active":"2024-02-23T00:51:36.256129+0000","last_became_peered":"2024-02-23T00:51:36.256129+0000","last_unstale":"2024-02-23T00:51:36.256492+0000","last_undegraded":"2024-02-23T00:51:36.256492+0000","last_fullsized":"2024-02-23T00:51:36.256492+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":566,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.241253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"231.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:37.245210+0000","last_change":"2024-02-23T00:51:37.245210+0000","last_active":"2024-02-23T00:51:37.245210+0000","last_peered":"2024-02-23T00:51:37.245210+0000","last_clean":"2024-02-23T00:51:37.245210+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:37.245210+0000","last_undegraded":"2024-02-23T00:51:37.245210+0000","last_fullsized":"2024-02-23T00:51:37.245210+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_clean_scrub_stamp":"2024-02-23T00:51:37.245210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"230.a","version":"0'0","reported_seq":9,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.260654+0000","last_change":"2024-02-23T00:51:36.260654+0000","last_active":"2024-02-23T00:51:36.260654+0000","last_peered":"2024-02-23T00:51:36.260654+0000","last_clean":"2024-02-23T00:51:36.260654+0000","last_became_active":"2024-02-23T00:51:36.260326+0000","last_became_peered":"2024-02-23T00:51:36.260326+0000","last_unstale":"2024-02-23T00:51:36.260654+0000","last_undegraded":"2024-02-23T00:51:36.260654+0000","last_fullsized":"2024-02-23T00:51:36.260654+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":566,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.241253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"231.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:37.245210+0000","last_change":"2024-02-23T00:51:37.245210+0000","last_active":"2024-02-23T00:51:37.245210+0000","last_peered":"2024-02-23T00:51:37.245210+0000","last_clean":"2024-02-23T00:51:37.245210+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:37.245210+0000","last_undegraded":"2024-02-23T00:51:37.245210+0000","last_fullsized":"2024-02-23T00:51:37.245210+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_clean_scrub_stamp":"2024-02-23T00:51:37.245210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"230.9","version":"0'0","reported_seq":9,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.263621+0000","last_change":"2024-02-23T00:51:36.263621+0000","last_active":"2024-02-23T00:51:36.263621+0000","last_peered":"2024-02-23T00:51:36.263621+0000","last_clean":"2024-02-23T00:51:36.263621+0000","last_became_active":"2024-02-23T00:51:36.263341+0000","last_became_peered":"2024-02-23T00:51:36.263341+0000","last_unstale":"2024-02-23T00:51:36.263621+0000","last_undegraded":"2024-02-23T00:51:36.263621+0000","last_fullsized":"2024-02-23T00:51:36.263621+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":566,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.241253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"231.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:37.245210+0000","last_change":"2024-02-23T00:51:37.245210+0000","last_active":"2024-02-23T00:51:37.245210+0000","last_peered":"2024-02-23T00:51:37.245210+0000","last_clean":"2024-02-23T00:51:37.245210+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:37.245210+0000","last_undegraded":"2024-02-23T00:51:37.245210+0000","last_fullsized":"2024-02-23T00:51:37.245210+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_clean_scrub_stamp":"2024-02-23T00:51:37.245210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"230.8","version":"0'0","reported_seq":9,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.265725+0000","last_change":"2024-02-23T00:51:36.265725+0000","last_active":"2024-02-23T00:51:36.265725+0000","last_peered":"2024-02-23T00:51:36.265725+0000","last_clean":"2024-02-23T00:51:36.265725+0000","last_became_active":"2024-02-23T00:51:36.265458+0000","last_became_peered":"2024-02-23T00:51:36.265458+0000","last_unstale":"2024-02-23T00:51:36.265725+0000","last_undegraded":"2024-02-23T00:51:36.265725+0000","last_fullsized":"2024-02-23T00:51:36.265725+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":566,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.241253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"231.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:37.245210+0000","last_change":"2024-02-23T00:51:37.245210+0000","last_active":"2024-02-23T00:51:37.245210+0000","last_peered":"2024-02-23T00:51:37.245210+0000","last_clean":"2024-02-23T00:51:37.245210+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:37.245210+0000","last_undegraded":"2024-02-23T00:51:37.245210+0000","last_fullsized":"2024-02-23T00:51:37.245210+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_clean_scrub_stamp":"2024-02-23T00:51:37.245210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"230.4","version":"0'0","reported_seq":9,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.265916+0000","last_change":"2024-02-23T00:51:36.265916+0000","last_active":"2024-02-23T00:51:36.265916+0000","last_peered":"2024-02-23T00:51:36.265916+0000","last_clean":"2024-02-23T00:51:36.265916+0000","last_became_active":"2024-02-23T00:51:36.265634+0000","last_became_peered":"2024-02-23T00:51:36.265634+0000","last_unstale":"2024-02-23T00:51:36.265916+0000","last_undegraded":"2024-02-23T00:51:36.265916+0000","last_fullsized":"2024-02-23T00:51:36.265916+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":566,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.241253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"229.7","version":"0'0","reported_seq":12,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.264295+0000","last_change":"2024-02-23T00:51:34.271578+0000","last_active":"2024-02-23T00:51:36.264295+0000","last_peered":"2024-02-23T00:51:36.264295+0000","last_clean":"2024-02-23T00:51:36.264295+0000","last_became_active":"2024-02-23T00:51:34.271103+0000","last_became_peered":"2024-02-23T00:51:34.271103+0000","last_unstale":"2024-02-23T00:51:36.264295+0000","last_undegraded":"2024-02-23T00:51:36.264295+0000","last_fullsized":"2024-02-23T00:51:36.264295+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":564,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_clean_scrub_stamp":"2024-02-23T00:51:33.255429+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"231.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:37.245210+0000","last_change":"2024-02-23T00:51:37.245210+0000","last_active":"2024-02-23T00:51:37.245210+0000","last_peered":"2024-02-23T00:51:37.245210+0000","last_clean":"2024-02-23T00:51:37.245210+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:37.245210+0000","last_undegraded":"2024-02-23T00:51:37.245210+0000","last_fullsized":"2024-02-23T00:51:37.245210+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_clean_scrub_stamp":"2024-02-23T00:51:37.245210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"230.5","version":"0'0","reported_seq":10,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.259239+0000","last_change":"2024-02-23T00:51:36.255369+0000","last_active":"2024-02-23T00:51:36.259239+0000","last_peered":"2024-02-23T00:51:36.259239+0000","last_clean":"2024-02-23T00:51:36.259239+0000","last_became_active":"2024-02-23T00:51:36.255013+0000","last_became_peered":"2024-02-23T00:51:36.255013+0000","last_unstale":"2024-02-23T00:51:36.259239+0000","last_undegraded":"2024-02-23T00:51:36.259239+0000","last_fullsized":"2024-02-23T00:51:36.259239+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":566,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.241253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"229.6","version":"0'0","reported_seq":11,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.250920+0000","last_change":"2024-02-23T00:51:34.269550+0000","last_active":"2024-02-23T00:51:35.250920+0000","last_peered":"2024-02-23T00:51:35.250920+0000","last_clean":"2024-02-23T00:51:35.250920+0000","last_became_active":"2024-02-23T00:51:34.269208+0000","last_became_peered":"2024-02-23T00:51:34.269208+0000","last_unstale":"2024-02-23T00:51:35.250920+0000","last_undegraded":"2024-02-23T00:51:35.250920+0000","last_fullsized":"2024-02-23T00:51:35.250920+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":564,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_clean_scrub_stamp":"2024-02-23T00:51:33.255429+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"231.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:37.245210+0000","last_change":"2024-02-23T00:51:37.245210+0000","last_active":"2024-02-23T00:51:37.245210+0000","last_peered":"2024-02-23T00:51:37.245210+0000","last_clean":"2024-02-23T00:51:37.245210+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:37.245210+0000","last_undegraded":"2024-02-23T00:51:37.245210+0000","last_fullsized":"2024-02-23T00:51:37.245210+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_clean_scrub_stamp":"2024-02-23T00:51:37.245210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"230.6","version":"0'0","reported_seq":10,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.266383+0000","last_change":"2024-02-23T00:51:36.260398+0000","last_active":"2024-02-23T00:51:36.266383+0000","last_peered":"2024-02-23T00:51:36.266383+0000","last_clean":"2024-02-23T00:51:36.266383+0000","last_became_active":"2024-02-23T00:51:36.259624+0000","last_became_peered":"2024-02-23T00:51:36.259624+0000","last_unstale":"2024-02-23T00:51:36.266383+0000","last_undegraded":"2024-02-23T00:51:36.266383+0000","last_fullsized":"2024-02-23T00:51:36.266383+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":566,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.241253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"229.5","version":"0'0","reported_seq":11,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.247879+0000","last_change":"2024-02-23T00:51:34.269112+0000","last_active":"2024-02-23T00:51:35.247879+0000","last_peered":"2024-02-23T00:51:35.247879+0000","last_clean":"2024-02-23T00:51:35.247879+0000","last_became_active":"2024-02-23T00:51:34.268826+0000","last_became_peered":"2024-02-23T00:51:34.268826+0000","last_unstale":"2024-02-23T00:51:35.247879+0000","last_undegraded":"2024-02-23T00:51:35.247879+0000","last_fullsized":"2024-02-23T00:51:35.247879+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":564,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_clean_scrub_stamp":"2024-02-23T00:51:33.255429+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"231.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:37.245210+0000","last_change":"2024-02-23T00:51:37.245210+0000","last_active":"2024-02-23T00:51:37.245210+0000","last_peered":"2024-02-23T00:51:37.245210+0000","last_clean":"2024-02-23T00:51:37.245210+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:37.245210+0000","last_undegraded":"2024-02-23T00:51:37.245210+0000","last_fullsized":"2024-02-23T00:51:37.245210+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_clean_scrub_stamp":"2024-02-23T00:51:37.245210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"230.7","version":"0'0","reported_seq":9,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.261935+0000","last_change":"2024-02-23T00:51:36.261935+0000","last_active":"2024-02-23T00:51:36.261935+0000","last_peered":"2024-02-23T00:51:36.261935+0000","last_clean":"2024-02-23T00:51:36.261935+0000","last_became_active":"2024-02-23T00:51:36.261641+0000","last_became_peered":"2024-02-23T00:51:36.261641+0000","last_unstale":"2024-02-23T00:51:36.261935+0000","last_undegraded":"2024-02-23T00:51:36.261935+0000","last_fullsized":"2024-02-23T00:51:36.261935+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":566,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.241253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"229.4","version":"0'0","reported_seq":12,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.259831+0000","last_change":"2024-02-23T00:51:34.271412+0000","last_active":"2024-02-23T00:51:36.259831+0000","last_peered":"2024-02-23T00:51:36.259831+0000","last_clean":"2024-02-23T00:51:36.259831+0000","last_became_active":"2024-02-23T00:51:34.270881+0000","last_became_peered":"2024-02-23T00:51:34.270881+0000","last_unstale":"2024-02-23T00:51:36.259831+0000","last_undegraded":"2024-02-23T00:51:36.259831+0000","last_fullsized":"2024-02-23T00:51:36.259831+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":564,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_clean_scrub_stamp":"2024-02-23T00:51:33.255429+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"231.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:37.245210+0000","last_change":"2024-02-23T00:51:37.245210+0000","last_active":"2024-02-23T00:51:37.245210+0000","last_peered":"2024-02-23T00:51:37.245210+0000","last_clean":"2024-02-23T00:51:37.245210+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:37.245210+0000","last_undegraded":"2024-02-23T00:51:37.245210+0000","last_fullsized":"2024-02-23T00:51:37.245210+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_clean_scrub_stamp":"2024-02-23T00:51:37.245210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"230.0","version":"0'0","reported_seq":9,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.267708+0000","last_change":"2024-02-23T00:51:36.267708+0000","last_active":"2024-02-23T00:51:36.267708+0000","last_peered":"2024-02-23T00:51:36.267708+0000","last_clean":"2024-02-23T00:51:36.267708+0000","last_became_active":"2024-02-23T00:51:36.267269+0000","last_became_peered":"2024-02-23T00:51:36.267269+0000","last_unstale":"2024-02-23T00:51:36.267708+0000","last_undegraded":"2024-02-23T00:51:36.267708+0000","last_fullsized":"2024-02-23T00:51:36.267708+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":566,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.241253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"229.3","version":"565'4","reported_seq":17,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.253328+0000","last_change":"2024-02-23T00:51:34.271564+0000","last_active":"2024-02-23T00:51:35.253328+0000","last_peered":"2024-02-23T00:51:35.253328+0000","last_clean":"2024-02-23T00:51:35.253328+0000","last_became_active":"2024-02-23T00:51:34.270817+0000","last_became_peered":"2024-02-23T00:51:34.270817+0000","last_unstale":"2024-02-23T00:51:35.253328+0000","last_undegraded":"2024-02-23T00:51:35.253328+0000","last_fullsized":"2024-02-23T00:51:35.253328+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":564,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_clean_scrub_stamp":"2024-02-23T00:51:33.255429+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"231.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:37.245210+0000","last_change":"2024-02-23T00:51:37.245210+0000","last_active":"2024-02-23T00:51:37.245210+0000","last_peered":"2024-02-23T00:51:37.245210+0000","last_clean":"2024-02-23T00:51:37.245210+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:37.245210+0000","last_undegraded":"2024-02-23T00:51:37.245210+0000","last_fullsized":"2024-02-23T00:51:37.245210+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_clean_scrub_stamp":"2024-02-23T00:51:37.245210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"230.1","version":"0'0","reported_seq":9,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.263184+0000","last_change":"2024-02-23T00:51:36.263184+0000","last_active":"2024-02-23T00:51:36.263184+0000","last_peered":"2024-02-23T00:51:36.263184+0000","last_clean":"2024-02-23T00:51:36.263184+0000","last_became_active":"2024-02-23T00:51:36.262877+0000","last_became_peered":"2024-02-23T00:51:36.262877+0000","last_unstale":"2024-02-23T00:51:36.263184+0000","last_undegraded":"2024-02-23T00:51:36.263184+0000","last_fullsized":"2024-02-23T00:51:36.263184+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":566,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.241253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"229.2","version":"0'0","reported_seq":11,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.248181+0000","last_change":"2024-02-23T00:51:34.271256+0000","last_active":"2024-02-23T00:51:35.248181+0000","last_peered":"2024-02-23T00:51:35.248181+0000","last_clean":"2024-02-23T00:51:35.248181+0000","last_became_active":"2024-02-23T00:51:34.270941+0000","last_became_peered":"2024-02-23T00:51:34.270941+0000","last_unstale":"2024-02-23T00:51:35.248181+0000","last_undegraded":"2024-02-23T00:51:35.248181+0000","last_fullsized":"2024-02-23T00:51:35.248181+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":564,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_clean_scrub_stamp":"2024-02-23T00:51:33.255429+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"231.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:37.245210+0000","last_change":"2024-02-23T00:51:37.245210+0000","last_active":"2024-02-23T00:51:37.245210+0000","last_peered":"2024-02-23T00:51:37.245210+0000","last_clean":"2024-02-23T00:51:37.245210+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:37.245210+0000","last_undegraded":"2024-02-23T00:51:37.245210+0000","last_fullsized":"2024-02-23T00:51:37.245210+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_clean_scrub_stamp":"2024-02-23T00:51:37.245210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"230.2","version":"0'0","reported_seq":9,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.264332+0000","last_change":"2024-02-23T00:51:36.264332+0000","last_active":"2024-02-23T00:51:36.264332+0000","last_peered":"2024-02-23T00:51:36.264332+0000","last_clean":"2024-02-23T00:51:36.264332+0000","last_became_active":"2024-02-23T00:51:36.263977+0000","last_became_peered":"2024-02-23T00:51:36.263977+0000","last_unstale":"2024-02-23T00:51:36.264332+0000","last_undegraded":"2024-02-23T00:51:36.264332+0000","last_fullsized":"2024-02-23T00:51:36.264332+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":566,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.241253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"229.1","version":"0'0","reported_seq":11,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.249283+0000","last_change":"2024-02-23T00:51:34.526061+0000","last_active":"2024-02-23T00:51:35.249283+0000","last_peered":"2024-02-23T00:51:35.249283+0000","last_clean":"2024-02-23T00:51:35.249283+0000","last_became_active":"2024-02-23T00:51:34.525742+0000","last_became_peered":"2024-02-23T00:51:34.525742+0000","last_unstale":"2024-02-23T00:51:35.249283+0000","last_undegraded":"2024-02-23T00:51:35.249283+0000","last_fullsized":"2024-02-23T00:51:35.249283+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":564,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_clean_scrub_stamp":"2024-02-23T00:51:33.255429+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"231.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-23T00:51:37.245210+0000","last_change":"2024-02-23T00:51:37.245210+0000","last_active":"2024-02-23T00:51:37.245210+0000","last_peered":"2024-02-23T00:51:37.245210+0000","last_clean":"2024-02-23T00:51:37.245210+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-23T00:51:37.245210+0000","last_undegraded":"2024-02-23T00:51:37.245210+0000","last_fullsized":"2024-02-23T00:51:37.245210+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:37.245210+0000","last_clean_scrub_stamp":"2024-02-23T00:51:37.245210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"230.3","version":"0'0","reported_seq":9,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.263096+0000","last_change":"2024-02-23T00:51:36.263096+0000","last_active":"2024-02-23T00:51:36.263096+0000","last_peered":"2024-02-23T00:51:36.263096+0000","last_clean":"2024-02-23T00:51:36.263096+0000","last_became_active":"2024-02-23T00:51:36.262828+0000","last_became_peered":"2024-02-23T00:51:36.262828+0000","last_unstale":"2024-02-23T00:51:36.263096+0000","last_undegraded":"2024-02-23T00:51:36.263096+0000","last_fullsized":"2024-02-23T00:51:36.263096+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":566,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:35.241253+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.241253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"229.0","version":"0'0","reported_seq":11,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.245938+0000","last_change":"2024-02-23T00:51:34.526282+0000","last_active":"2024-02-23T00:51:35.245938+0000","last_peered":"2024-02-23T00:51:35.245938+0000","last_clean":"2024-02-23T00:51:35.245938+0000","last_became_active":"2024-02-23T00:51:34.525872+0000","last_became_peered":"2024-02-23T00:51:34.525872+0000","last_unstale":"2024-02-23T00:51:35.245938+0000","last_undegraded":"2024-02-23T00:51:35.245938+0000","last_fullsized":"2024-02-23T00:51:35.245938+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":564,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_clean_scrub_stamp":"2024-02-23T00:51:33.255429+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"466'9","reported_seq":693,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.253945+0000","last_change":"2024-02-23T00:49:43.188066+0000","last_active":"2024-02-23T00:51:35.253945+0000","last_peered":"2024-02-23T00:51:35.253945+0000","last_clean":"2024-02-23T00:51:35.253945+0000","last_became_active":"2024-02-23T00:39:18.793414+0000","last_became_peered":"2024-02-23T00:39:18.793414+0000","last_unstale":"2024-02-23T00:51:35.253945+0000","last_undegraded":"2024-02-23T00:51:35.253945+0000","last_fullsized":"2024-02-23T00:51:35.253945+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'9","last_scrub_stamp":"2024-02-23T00:49:43.187989+0000","last_deep_scrub":"466'9","last_deep_scrub_stamp":"2024-02-23T00:49:43.187989+0000","last_clean_scrub_stamp":"2024-02-23T00:49:43.187989+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"470'42","reported_seq":612,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.247533+0000","last_change":"2024-02-23T00:49:47.653272+0000","last_active":"2024-02-23T00:51:35.247533+0000","last_peered":"2024-02-23T00:51:35.247533+0000","last_clean":"2024-02-23T00:51:35.247533+0000","last_became_active":"2024-02-23T00:39:16.859401+0000","last_became_peered":"2024-02-23T00:39:16.859401+0000","last_unstale":"2024-02-23T00:51:35.247533+0000","last_undegraded":"2024-02-23T00:51:35.247533+0000","last_fullsized":"2024-02-23T00:51:35.247533+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"470'42","last_scrub_stamp":"2024-02-23T00:49:47.653176+0000","last_deep_scrub":"470'42","last_deep_scrub_stamp":"2024-02-23T00:49:47.653176+0000","last_clean_scrub_stamp":"2024-02-23T00:49:47.653176+0000","log_size":42,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":213,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.656632+0000","last_change":"2024-02-23T00:51:36.656632+0000","last_active":"2024-02-23T00:51:36.656632+0000","last_peered":"2024-02-23T00:51:36.656632+0000","last_clean":"2024-02-23T00:51:36.656632+0000","last_became_active":"2024-02-23T00:47:28.220876+0000","last_became_peered":"2024-02-23T00:47:28.220876+0000","last_unstale":"2024-02-23T00:51:36.656632+0000","last_undegraded":"2024-02-23T00:51:36.656632+0000","last_fullsized":"2024-02-23T00:51:36.656632+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:36.656571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:04.330426+0000","last_clean_scrub_stamp":"2024-02-23T00:51:36.656571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":550,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.248277+0000","last_change":"2024-02-23T00:50:14.801990+0000","last_active":"2024-02-23T00:51:35.248277+0000","last_peered":"2024-02-23T00:51:35.248277+0000","last_clean":"2024-02-23T00:51:35.248277+0000","last_became_active":"2024-02-23T00:43:32.282545+0000","last_became_peered":"2024-02-23T00:43:32.282545+0000","last_unstale":"2024-02-23T00:51:35.248277+0000","last_undegraded":"2024-02-23T00:51:35.248277+0000","last_fullsized":"2024-02-23T00:51:35.248277+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:14.801915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:50:14.801915+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":479,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.248232+0000","last_change":"2024-02-23T00:50:23.727290+0000","last_active":"2024-02-23T00:51:35.248232+0000","last_peered":"2024-02-23T00:51:35.248232+0000","last_clean":"2024-02-23T00:51:35.248232+0000","last_became_active":"2024-02-23T00:42:58.828846+0000","last_became_peered":"2024-02-23T00:42:58.828846+0000","last_unstale":"2024-02-23T00:51:35.248232+0000","last_undegraded":"2024-02-23T00:51:35.248232+0000","last_fullsized":"2024-02-23T00:51:35.248232+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:23.727173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:23.727173+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"457'21","reported_seq":576,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.252646+0000","last_change":"2024-02-23T00:49:27.765115+0000","last_active":"2024-02-23T00:51:35.252646+0000","last_peered":"2024-02-23T00:51:35.252646+0000","last_clean":"2024-02-23T00:51:35.252646+0000","last_became_active":"2024-02-23T00:39:16.783701+0000","last_became_peered":"2024-02-23T00:39:16.783701+0000","last_unstale":"2024-02-23T00:51:35.252646+0000","last_undegraded":"2024-02-23T00:51:35.252646+0000","last_fullsized":"2024-02-23T00:51:35.252646+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-23T00:49:27.765048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:27.765048+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":529,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.260251+0000","last_change":"2024-02-23T00:49:04.528117+0000","last_active":"2024-02-23T00:51:36.260251+0000","last_peered":"2024-02-23T00:51:36.260251+0000","last_clean":"2024-02-23T00:51:36.260251+0000","last_became_active":"2024-02-23T00:39:14.772493+0000","last_became_peered":"2024-02-23T00:39:14.772493+0000","last_unstale":"2024-02-23T00:51:36.260251+0000","last_undegraded":"2024-02-23T00:51:36.260251+0000","last_fullsized":"2024-02-23T00:51:36.260251+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:04.528006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:04.528006+0000","last_clean_scrub_stamp":"2024-02-23T00:49:04.528006+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":204,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.246677+0000","last_change":"2024-02-23T00:49:30.499854+0000","last_active":"2024-02-23T00:51:35.246677+0000","last_peered":"2024-02-23T00:51:35.246677+0000","last_clean":"2024-02-23T00:51:35.246677+0000","last_became_active":"2024-02-23T00:47:28.231567+0000","last_became_peered":"2024-02-23T00:47:28.231567+0000","last_unstale":"2024-02-23T00:51:35.246677+0000","last_undegraded":"2024-02-23T00:51:35.246677+0000","last_fullsized":"2024-02-23T00:51:35.246677+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:30.499769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:08.335330+0000","last_clean_scrub_stamp":"2024-02-23T00:49:30.499769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":559,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.250090+0000","last_change":"2024-02-23T00:51:11.526671+0000","last_active":"2024-02-23T00:51:35.250090+0000","last_peered":"2024-02-23T00:51:35.250090+0000","last_clean":"2024-02-23T00:51:35.250090+0000","last_became_active":"2024-02-23T00:47:28.214157+0000","last_became_peered":"2024-02-23T00:47:28.214157+0000","last_unstale":"2024-02-23T00:51:35.250090+0000","last_undegraded":"2024-02-23T00:51:35.250090+0000","last_fullsized":"2024-02-23T00:51:35.250090+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:11.526584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:13.867716+0000","last_clean_scrub_stamp":"2024-02-23T00:51:11.526584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":479,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.251371+0000","last_change":"2024-02-23T00:50:32.622097+0000","last_active":"2024-02-23T00:51:35.251371+0000","last_peered":"2024-02-23T00:51:35.251371+0000","last_clean":"2024-02-23T00:51:35.251371+0000","last_became_active":"2024-02-23T00:42:58.867305+0000","last_became_peered":"2024-02-23T00:42:58.867305+0000","last_unstale":"2024-02-23T00:51:35.251371+0000","last_undegraded":"2024-02-23T00:51:35.251371+0000","last_fullsized":"2024-02-23T00:51:35.251371+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:32.622026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:32.622026+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"457'51","reported_seq":620,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.250795+0000","last_change":"2024-02-23T00:51:13.326646+0000","last_active":"2024-02-23T00:51:35.250795+0000","last_peered":"2024-02-23T00:51:35.250795+0000","last_clean":"2024-02-23T00:51:35.250795+0000","last_became_active":"2024-02-23T00:39:16.857413+0000","last_became_peered":"2024-02-23T00:39:16.857413+0000","last_unstale":"2024-02-23T00:51:35.250795+0000","last_undegraded":"2024-02-23T00:51:35.250795+0000","last_fullsized":"2024-02-23T00:51:35.250795+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'51","last_scrub_stamp":"2024-02-23T00:51:13.326535+0000","last_deep_scrub":"88'35","last_deep_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_clean_scrub_stamp":"2024-02-23T00:51:13.326535+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":68,"num_read_kb":56,"num_write":42,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":211,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.247884+0000","last_change":"2024-02-23T00:50:57.770551+0000","last_active":"2024-02-23T00:51:35.247884+0000","last_peered":"2024-02-23T00:51:35.247884+0000","last_clean":"2024-02-23T00:51:35.247884+0000","last_became_active":"2024-02-23T00:47:28.231891+0000","last_became_peered":"2024-02-23T00:47:28.231891+0000","last_unstale":"2024-02-23T00:51:35.247884+0000","last_undegraded":"2024-02-23T00:51:35.247884+0000","last_fullsized":"2024-02-23T00:51:35.247884+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:57.770450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_clean_scrub_stamp":"2024-02-23T00:50:57.770450+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":211,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.650367+0000","last_change":"2024-02-23T00:51:36.650367+0000","last_active":"2024-02-23T00:51:36.650367+0000","last_peered":"2024-02-23T00:51:36.650367+0000","last_clean":"2024-02-23T00:51:36.650367+0000","last_became_active":"2024-02-23T00:47:28.220578+0000","last_became_peered":"2024-02-23T00:47:28.220578+0000","last_unstale":"2024-02-23T00:51:36.650367+0000","last_undegraded":"2024-02-23T00:51:36.650367+0000","last_fullsized":"2024-02-23T00:51:36.650367+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:36.650303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:03.682745+0000","last_clean_scrub_stamp":"2024-02-23T00:51:36.650303+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":558,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.247125+0000","last_change":"2024-02-23T00:49:42.150972+0000","last_active":"2024-02-23T00:51:35.247125+0000","last_peered":"2024-02-23T00:51:35.247125+0000","last_clean":"2024-02-23T00:51:35.247125+0000","last_became_active":"2024-02-23T00:47:28.215636+0000","last_became_peered":"2024-02-23T00:47:28.215636+0000","last_unstale":"2024-02-23T00:51:35.247125+0000","last_undegraded":"2024-02-23T00:51:35.247125+0000","last_fullsized":"2024-02-23T00:51:35.247125+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:42.150875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:42.150875+0000","last_clean_scrub_stamp":"2024-02-23T00:49:42.150875+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":471,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.260049+0000","last_change":"2024-02-23T00:49:34.515593+0000","last_active":"2024-02-23T00:51:36.260049+0000","last_peered":"2024-02-23T00:51:36.260049+0000","last_clean":"2024-02-23T00:51:36.260049+0000","last_became_active":"2024-02-23T00:42:57.639013+0000","last_became_peered":"2024-02-23T00:42:57.639013+0000","last_unstale":"2024-02-23T00:51:36.260049+0000","last_undegraded":"2024-02-23T00:51:36.260049+0000","last_fullsized":"2024-02-23T00:51:36.260049+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:34.515496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:49:34.515496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"457'31","reported_seq":632,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.249546+0000","last_change":"2024-02-23T00:49:31.480372+0000","last_active":"2024-02-23T00:51:35.249546+0000","last_peered":"2024-02-23T00:51:35.249546+0000","last_clean":"2024-02-23T00:51:35.249546+0000","last_became_active":"2024-02-23T00:47:28.200653+0000","last_became_peered":"2024-02-23T00:47:28.200653+0000","last_unstale":"2024-02-23T00:51:35.249546+0000","last_undegraded":"2024-02-23T00:51:35.249546+0000","last_fullsized":"2024-02-23T00:51:35.249546+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:49:31.480253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:31.480253+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":210,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.253471+0000","last_change":"2024-02-23T00:51:10.213423+0000","last_active":"2024-02-23T00:51:35.253471+0000","last_peered":"2024-02-23T00:51:35.253471+0000","last_clean":"2024-02-23T00:51:35.253471+0000","last_became_active":"2024-02-23T00:47:28.201926+0000","last_became_peered":"2024-02-23T00:47:28.201926+0000","last_unstale":"2024-02-23T00:51:35.253471+0000","last_undegraded":"2024-02-23T00:51:35.253471+0000","last_fullsized":"2024-02-23T00:51:35.253471+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:10.213337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:10.213337+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":530,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.261302+0000","last_change":"2024-02-23T00:51:09.343951+0000","last_active":"2024-02-23T00:51:36.261302+0000","last_peered":"2024-02-23T00:51:36.261302+0000","last_clean":"2024-02-23T00:51:36.261302+0000","last_became_active":"2024-02-23T00:39:18.799875+0000","last_became_peered":"2024-02-23T00:39:18.799875+0000","last_unstale":"2024-02-23T00:51:36.261302+0000","last_undegraded":"2024-02-23T00:51:36.261302+0000","last_fullsized":"2024-02-23T00:51:36.261302+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:09.343884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:09.343884+0000","last_clean_scrub_stamp":"2024-02-23T00:51:09.343884+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":471,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.255898+0000","last_change":"2024-02-23T00:48:49.610457+0000","last_active":"2024-02-23T00:51:36.255898+0000","last_peered":"2024-02-23T00:51:36.255898+0000","last_clean":"2024-02-23T00:51:36.255898+0000","last_became_active":"2024-02-23T00:42:57.675306+0000","last_became_peered":"2024-02-23T00:42:57.675306+0000","last_unstale":"2024-02-23T00:51:36.255898+0000","last_undegraded":"2024-02-23T00:51:36.255898+0000","last_fullsized":"2024-02-23T00:51:36.255898+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:49.610391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:49.610391+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"457'20","reported_seq":564,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.246439+0000","last_change":"2024-02-23T00:51:01.488450+0000","last_active":"2024-02-23T00:51:35.246439+0000","last_peered":"2024-02-23T00:51:35.246439+0000","last_clean":"2024-02-23T00:51:35.246439+0000","last_became_active":"2024-02-23T00:39:16.777530+0000","last_became_peered":"2024-02-23T00:39:16.777530+0000","last_unstale":"2024-02-23T00:51:35.246439+0000","last_undegraded":"2024-02-23T00:51:35.246439+0000","last_fullsized":"2024-02-23T00:51:35.246439+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'20","last_scrub_stamp":"2024-02-23T00:51:01.488371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:01.488371+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":529,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.256095+0000","last_change":"2024-02-23T00:48:41.719867+0000","last_active":"2024-02-23T00:51:36.256095+0000","last_peered":"2024-02-23T00:51:36.256095+0000","last_clean":"2024-02-23T00:51:36.256095+0000","last_became_active":"2024-02-23T00:39:14.780379+0000","last_became_peered":"2024-02-23T00:39:14.780379+0000","last_unstale":"2024-02-23T00:51:36.256095+0000","last_undegraded":"2024-02-23T00:51:36.256095+0000","last_fullsized":"2024-02-23T00:51:36.256095+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:41.719772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.575242+0000","last_clean_scrub_stamp":"2024-02-23T00:48:41.719772+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"376'9","reported_seq":280,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.253213+0000","last_change":"2024-02-23T00:49:40.180695+0000","last_active":"2024-02-23T00:51:35.253213+0000","last_peered":"2024-02-23T00:51:35.253213+0000","last_clean":"2024-02-23T00:51:35.253213+0000","last_became_active":"2024-02-23T00:47:28.231104+0000","last_became_peered":"2024-02-23T00:47:28.231104+0000","last_unstale":"2024-02-23T00:51:35.253213+0000","last_undegraded":"2024-02-23T00:51:35.253213+0000","last_fullsized":"2024-02-23T00:51:35.253213+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"376'9","last_scrub_stamp":"2024-02-23T00:49:40.180616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:40.180616+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":482,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.246818+0000","last_change":"2024-02-23T00:51:12.335819+0000","last_active":"2024-02-23T00:51:35.246818+0000","last_peered":"2024-02-23T00:51:35.246818+0000","last_clean":"2024-02-23T00:51:35.246818+0000","last_became_active":"2024-02-23T00:42:58.765106+0000","last_became_peered":"2024-02-23T00:42:58.765106+0000","last_unstale":"2024-02-23T00:51:35.246818+0000","last_undegraded":"2024-02-23T00:51:35.246818+0000","last_fullsized":"2024-02-23T00:51:35.246818+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.335752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.335752+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.335752+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"457'27","reported_seq":570,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.264553+0000","last_change":"2024-02-23T00:48:00.698737+0000","last_active":"2024-02-23T00:51:36.264553+0000","last_peered":"2024-02-23T00:51:36.264553+0000","last_clean":"2024-02-23T00:51:36.264553+0000","last_became_active":"2024-02-23T00:39:16.789757+0000","last_became_peered":"2024-02-23T00:39:16.789757+0000","last_unstale":"2024-02-23T00:51:36.264553+0000","last_undegraded":"2024-02-23T00:51:36.264553+0000","last_fullsized":"2024-02-23T00:51:36.264553+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'15","last_scrub_stamp":"2024-02-23T00:48:00.698670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:48:00.698670+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":534,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.253907+0000","last_change":"2024-02-23T00:51:13.375106+0000","last_active":"2024-02-23T00:51:36.253907+0000","last_peered":"2024-02-23T00:51:36.253907+0000","last_clean":"2024-02-23T00:51:36.253907+0000","last_became_active":"2024-02-23T00:39:14.780526+0000","last_became_peered":"2024-02-23T00:39:14.780526+0000","last_unstale":"2024-02-23T00:51:36.253907+0000","last_undegraded":"2024-02-23T00:51:36.253907+0000","last_fullsized":"2024-02-23T00:51:36.253907+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:13.375039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:13.375039+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":688,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.249202+0000","last_change":"2024-02-23T00:49:40.384676+0000","last_active":"2024-02-23T00:51:35.249202+0000","last_peered":"2024-02-23T00:51:35.249202+0000","last_clean":"2024-02-23T00:51:35.249202+0000","last_became_active":"2024-02-23T00:39:18.800440+0000","last_became_peered":"2024-02-23T00:39:18.800440+0000","last_unstale":"2024-02-23T00:51:35.249202+0000","last_undegraded":"2024-02-23T00:51:35.249202+0000","last_fullsized":"2024-02-23T00:51:35.249202+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:49:40.384608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:40.384608+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":475,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.246772+0000","last_change":"2024-02-23T00:48:59.206327+0000","last_active":"2024-02-23T00:51:35.246772+0000","last_peered":"2024-02-23T00:51:35.246772+0000","last_clean":"2024-02-23T00:51:35.246772+0000","last_became_active":"2024-02-23T00:42:58.833924+0000","last_became_peered":"2024-02-23T00:42:58.833924+0000","last_unstale":"2024-02-23T00:51:35.246772+0000","last_undegraded":"2024-02-23T00:51:35.246772+0000","last_fullsized":"2024-02-23T00:51:35.246772+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:59.206249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:59.206249+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"457'31","reported_seq":627,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.249112+0000","last_change":"2024-02-23T00:49:04.235169+0000","last_active":"2024-02-23T00:51:35.249112+0000","last_peered":"2024-02-23T00:51:35.249112+0000","last_clean":"2024-02-23T00:51:35.249112+0000","last_became_active":"2024-02-23T00:47:28.201225+0000","last_became_peered":"2024-02-23T00:47:28.201225+0000","last_unstale":"2024-02-23T00:51:35.249112+0000","last_undegraded":"2024-02-23T00:51:35.249112+0000","last_fullsized":"2024-02-23T00:51:35.249112+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"336'23","last_scrub_stamp":"2024-02-23T00:49:04.235046+0000","last_deep_scrub":"336'23","last_deep_scrub_stamp":"2024-02-23T00:46:48.320085+0000","last_clean_scrub_stamp":"2024-02-23T00:49:04.235046+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":537,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.251224+0000","last_change":"2024-02-23T00:49:37.331334+0000","last_active":"2024-02-23T00:51:35.251224+0000","last_peered":"2024-02-23T00:51:35.251224+0000","last_clean":"2024-02-23T00:51:35.251224+0000","last_became_active":"2024-02-23T00:39:14.784121+0000","last_became_peered":"2024-02-23T00:39:14.784121+0000","last_unstale":"2024-02-23T00:51:35.251224+0000","last_undegraded":"2024-02-23T00:51:35.251224+0000","last_fullsized":"2024-02-23T00:51:35.251224+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:37.331270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:37.331270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":534,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.248533+0000","last_change":"2024-02-23T00:50:19.827889+0000","last_active":"2024-02-23T00:51:35.248533+0000","last_peered":"2024-02-23T00:51:35.248533+0000","last_clean":"2024-02-23T00:51:35.248533+0000","last_became_active":"2024-02-23T00:39:18.800777+0000","last_became_peered":"2024-02-23T00:39:18.800777+0000","last_unstale":"2024-02-23T00:51:35.248533+0000","last_undegraded":"2024-02-23T00:51:35.248533+0000","last_fullsized":"2024-02-23T00:51:35.248533+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:19.827814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_clean_scrub_stamp":"2024-02-23T00:50:19.827814+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":471,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.253445+0000","last_change":"2024-02-23T00:49:33.529224+0000","last_active":"2024-02-23T00:51:36.253445+0000","last_peered":"2024-02-23T00:51:36.253445+0000","last_clean":"2024-02-23T00:51:36.253445+0000","last_became_active":"2024-02-23T00:42:57.653279+0000","last_became_peered":"2024-02-23T00:42:57.653279+0000","last_unstale":"2024-02-23T00:51:36.253445+0000","last_undegraded":"2024-02-23T00:51:36.253445+0000","last_fullsized":"2024-02-23T00:51:36.253445+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:33.529135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:49:33.529135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":693,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.246399+0000","last_change":"2024-02-23T00:51:34.577307+0000","last_active":"2024-02-23T00:51:35.246399+0000","last_peered":"2024-02-23T00:51:35.246399+0000","last_clean":"2024-02-23T00:51:35.246399+0000","last_became_active":"2024-02-23T00:39:18.798536+0000","last_became_peered":"2024-02-23T00:39:18.798536+0000","last_unstale":"2024-02-23T00:51:35.246399+0000","last_undegraded":"2024-02-23T00:51:35.246399+0000","last_fullsized":"2024-02-23T00:51:35.246399+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:51:34.577222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:51:34.577222+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"457'42","reported_seq":599,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.246866+0000","last_change":"2024-02-23T00:50:24.532387+0000","last_active":"2024-02-23T00:51:35.246866+0000","last_peered":"2024-02-23T00:51:35.246866+0000","last_clean":"2024-02-23T00:51:35.246866+0000","last_became_active":"2024-02-23T00:39:16.780130+0000","last_became_peered":"2024-02-23T00:39:16.780130+0000","last_unstale":"2024-02-23T00:51:35.246866+0000","last_undegraded":"2024-02-23T00:51:35.246866+0000","last_fullsized":"2024-02-23T00:51:35.246866+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'42","last_scrub_stamp":"2024-02-23T00:50:24.532271+0000","last_deep_scrub":"423'28","last_deep_scrub_stamp":"2024-02-23T00:48:17.435906+0000","last_clean_scrub_stamp":"2024-02-23T00:50:24.532271+0000","log_size":42,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":46,"num_write":34,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":553,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.252554+0000","last_change":"2024-02-23T00:50:59.199966+0000","last_active":"2024-02-23T00:51:35.252554+0000","last_peered":"2024-02-23T00:51:35.252554+0000","last_clean":"2024-02-23T00:51:35.252554+0000","last_became_active":"2024-02-23T00:47:28.237083+0000","last_became_peered":"2024-02-23T00:47:28.237083+0000","last_unstale":"2024-02-23T00:51:35.252554+0000","last_undegraded":"2024-02-23T00:51:35.252554+0000","last_fullsized":"2024-02-23T00:51:35.252554+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:59.199899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:51.391939+0000","last_clean_scrub_stamp":"2024-02-23T00:50:59.199899+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":550,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.253839+0000","last_change":"2024-02-23T00:49:41.163033+0000","last_active":"2024-02-23T00:51:35.253839+0000","last_peered":"2024-02-23T00:51:35.253839+0000","last_clean":"2024-02-23T00:51:35.253839+0000","last_became_active":"2024-02-23T00:47:30.187386+0000","last_became_peered":"2024-02-23T00:47:30.187386+0000","last_unstale":"2024-02-23T00:51:35.253839+0000","last_undegraded":"2024-02-23T00:51:35.253839+0000","last_fullsized":"2024-02-23T00:51:35.253839+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:41.162895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:46.686764+0000","last_clean_scrub_stamp":"2024-02-23T00:49:41.162895+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":488,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.248638+0000","last_change":"2024-02-23T00:51:03.875340+0000","last_active":"2024-02-23T00:51:35.248638+0000","last_peered":"2024-02-23T00:51:35.248638+0000","last_clean":"2024-02-23T00:51:35.248638+0000","last_became_active":"2024-02-23T00:47:28.212307+0000","last_became_peered":"2024-02-23T00:47:28.212307+0000","last_unstale":"2024-02-23T00:51:35.248638+0000","last_undegraded":"2024-02-23T00:51:35.248638+0000","last_fullsized":"2024-02-23T00:51:35.248638+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:03.875273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:03.875273+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":534,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.247150+0000","last_change":"2024-02-23T00:50:25.575319+0000","last_active":"2024-02-23T00:51:35.247150+0000","last_peered":"2024-02-23T00:51:35.247150+0000","last_clean":"2024-02-23T00:51:35.247150+0000","last_became_active":"2024-02-23T00:39:18.798829+0000","last_became_peered":"2024-02-23T00:39:18.798829+0000","last_unstale":"2024-02-23T00:51:35.247150+0000","last_undegraded":"2024-02-23T00:51:35.247150+0000","last_fullsized":"2024-02-23T00:51:35.247150+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:25.575238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:10.493340+0000","last_clean_scrub_stamp":"2024-02-23T00:50:25.575238+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"457'52","reported_seq":629,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.246301+0000","last_change":"2024-02-23T00:51:05.221646+0000","last_active":"2024-02-23T00:51:35.246301+0000","last_peered":"2024-02-23T00:51:35.246301+0000","last_clean":"2024-02-23T00:51:35.246301+0000","last_became_active":"2024-02-23T00:39:16.859002+0000","last_became_peered":"2024-02-23T00:39:16.859002+0000","last_unstale":"2024-02-23T00:51:35.246301+0000","last_undegraded":"2024-02-23T00:51:35.246301+0000","last_fullsized":"2024-02-23T00:51:35.246301+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'52","last_scrub_stamp":"2024-02-23T00:51:05.221572+0000","last_deep_scrub":"251'40","last_deep_scrub_stamp":"2024-02-23T00:45:23.910546+0000","last_clean_scrub_stamp":"2024-02-23T00:51:05.221572+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":76,"num_read_kb":62,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":529,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.264454+0000","last_change":"2024-02-23T00:48:44.728592+0000","last_active":"2024-02-23T00:51:36.264454+0000","last_peered":"2024-02-23T00:51:36.264454+0000","last_clean":"2024-02-23T00:51:36.264454+0000","last_became_active":"2024-02-23T00:39:14.779193+0000","last_became_peered":"2024-02-23T00:39:14.779193+0000","last_unstale":"2024-02-23T00:51:36.264454+0000","last_undegraded":"2024-02-23T00:51:36.264454+0000","last_fullsized":"2024-02-23T00:51:36.264454+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:44.728520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:48:44.728520+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":557,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.246772+0000","last_change":"2024-02-23T00:50:17.599788+0000","last_active":"2024-02-23T00:51:35.246772+0000","last_peered":"2024-02-23T00:51:35.246772+0000","last_clean":"2024-02-23T00:51:35.246772+0000","last_became_active":"2024-02-23T00:47:33.257915+0000","last_became_peered":"2024-02-23T00:47:33.257915+0000","last_unstale":"2024-02-23T00:51:35.246772+0000","last_undegraded":"2024-02-23T00:51:35.246772+0000","last_fullsized":"2024-02-23T00:51:35.246772+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":381,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:17.599724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:50:17.599724+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":482,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.248207+0000","last_change":"2024-02-23T00:51:03.166243+0000","last_active":"2024-02-23T00:51:35.248207+0000","last_peered":"2024-02-23T00:51:35.248207+0000","last_clean":"2024-02-23T00:51:35.248207+0000","last_became_active":"2024-02-23T00:42:58.809283+0000","last_became_peered":"2024-02-23T00:42:58.809283+0000","last_unstale":"2024-02-23T00:51:35.248207+0000","last_undegraded":"2024-02-23T00:51:35.248207+0000","last_fullsized":"2024-02-23T00:51:35.248207+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:03.166140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:03.166140+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":538,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.250719+0000","last_change":"2024-02-23T00:51:34.200412+0000","last_active":"2024-02-23T00:51:35.250719+0000","last_peered":"2024-02-23T00:51:35.250719+0000","last_clean":"2024-02-23T00:51:35.250719+0000","last_became_active":"2024-02-23T00:39:18.796664+0000","last_became_peered":"2024-02-23T00:39:18.796664+0000","last_unstale":"2024-02-23T00:51:35.250719+0000","last_undegraded":"2024-02-23T00:51:35.250719+0000","last_fullsized":"2024-02-23T00:51:35.250719+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:34.200312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:51:34.200312+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"457'44","reported_seq":611,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.251973+0000","last_change":"2024-02-23T00:49:46.644982+0000","last_active":"2024-02-23T00:51:35.251973+0000","last_peered":"2024-02-23T00:51:35.251973+0000","last_clean":"2024-02-23T00:51:35.251973+0000","last_became_active":"2024-02-23T00:39:16.857278+0000","last_became_peered":"2024-02-23T00:39:16.857278+0000","last_unstale":"2024-02-23T00:51:35.251973+0000","last_undegraded":"2024-02-23T00:51:35.251973+0000","last_fullsized":"2024-02-23T00:51:35.251973+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'44","last_scrub_stamp":"2024-02-23T00:49:46.644911+0000","last_deep_scrub":"457'44","last_deep_scrub_stamp":"2024-02-23T00:49:46.644911+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.644911+0000","log_size":44,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":537,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.253356+0000","last_change":"2024-02-23T00:50:22.299591+0000","last_active":"2024-02-23T00:51:35.253356+0000","last_peered":"2024-02-23T00:51:35.253356+0000","last_clean":"2024-02-23T00:51:35.253356+0000","last_became_active":"2024-02-23T00:39:14.783662+0000","last_became_peered":"2024-02-23T00:39:14.783662+0000","last_unstale":"2024-02-23T00:51:35.253356+0000","last_undegraded":"2024-02-23T00:51:35.253356+0000","last_fullsized":"2024-02-23T00:51:35.253356+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:22.299534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:22.299534+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":550,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.253969+0000","last_change":"2024-02-23T00:48:13.821965+0000","last_active":"2024-02-23T00:51:36.253969+0000","last_peered":"2024-02-23T00:51:36.253969+0000","last_clean":"2024-02-23T00:51:36.253969+0000","last_became_active":"2024-02-23T00:47:37.184240+0000","last_became_peered":"2024-02-23T00:47:37.184240+0000","last_unstale":"2024-02-23T00:51:36.253969+0000","last_undegraded":"2024-02-23T00:51:36.253969+0000","last_fullsized":"2024-02-23T00:51:36.253969+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":385,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:13.821901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:48:13.821901+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":479,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.247994+0000","last_change":"2024-02-23T00:50:24.682130+0000","last_active":"2024-02-23T00:51:35.247994+0000","last_peered":"2024-02-23T00:51:35.247994+0000","last_clean":"2024-02-23T00:51:35.247994+0000","last_became_active":"2024-02-23T00:42:58.726652+0000","last_became_peered":"2024-02-23T00:42:58.726652+0000","last_unstale":"2024-02-23T00:51:35.247994+0000","last_undegraded":"2024-02-23T00:51:35.247994+0000","last_fullsized":"2024-02-23T00:51:35.247994+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:24.682039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:24.682039+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":534,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.248557+0000","last_change":"2024-02-23T00:49:34.709320+0000","last_active":"2024-02-23T00:51:35.248557+0000","last_peered":"2024-02-23T00:51:35.248557+0000","last_clean":"2024-02-23T00:51:35.248557+0000","last_became_active":"2024-02-23T00:39:18.800378+0000","last_became_peered":"2024-02-23T00:39:18.800378+0000","last_unstale":"2024-02-23T00:51:35.248557+0000","last_undegraded":"2024-02-23T00:51:35.248557+0000","last_fullsized":"2024-02-23T00:51:35.248557+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:34.709251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:59.694033+0000","last_clean_scrub_stamp":"2024-02-23T00:49:34.709251+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"457'30","reported_seq":601,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.251079+0000","last_change":"2024-02-23T00:49:42.730745+0000","last_active":"2024-02-23T00:51:35.251079+0000","last_peered":"2024-02-23T00:51:35.251079+0000","last_clean":"2024-02-23T00:51:35.251079+0000","last_became_active":"2024-02-23T00:47:28.215366+0000","last_became_peered":"2024-02-23T00:47:28.215366+0000","last_unstale":"2024-02-23T00:51:35.251079+0000","last_undegraded":"2024-02-23T00:51:35.251079+0000","last_fullsized":"2024-02-23T00:51:35.251079+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'30","last_scrub_stamp":"2024-02-23T00:49:42.730670+0000","last_deep_scrub":"95'18","last_deep_scrub_stamp":"2024-02-23T00:42:01.195545+0000","last_clean_scrub_stamp":"2024-02-23T00:49:42.730670+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":542,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.247275+0000","last_change":"2024-02-23T00:49:46.162522+0000","last_active":"2024-02-23T00:51:35.247275+0000","last_peered":"2024-02-23T00:51:35.247275+0000","last_clean":"2024-02-23T00:51:35.247275+0000","last_became_active":"2024-02-23T00:39:14.784578+0000","last_became_peered":"2024-02-23T00:39:14.784578+0000","last_unstale":"2024-02-23T00:51:35.247275+0000","last_undegraded":"2024-02-23T00:51:35.247275+0000","last_fullsized":"2024-02-23T00:51:35.247275+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:46.162444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.162444+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":538,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.249413+0000","last_change":"2024-02-23T00:49:31.768539+0000","last_active":"2024-02-23T00:51:35.249413+0000","last_peered":"2024-02-23T00:51:35.249413+0000","last_clean":"2024-02-23T00:51:35.249413+0000","last_became_active":"2024-02-23T00:39:14.775446+0000","last_became_peered":"2024-02-23T00:39:14.775446+0000","last_unstale":"2024-02-23T00:51:35.249413+0000","last_undegraded":"2024-02-23T00:51:35.249413+0000","last_fullsized":"2024-02-23T00:51:35.249413+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:31.768472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:31.768472+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"67'1","reported_seq":561,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.265364+0000","last_change":"2024-02-23T00:51:02.380789+0000","last_active":"2024-02-23T00:51:36.265364+0000","last_peered":"2024-02-23T00:51:36.265364+0000","last_clean":"2024-02-23T00:51:36.265364+0000","last_became_active":"2024-02-23T00:47:56.187940+0000","last_became_peered":"2024-02-23T00:47:56.187940+0000","last_unstale":"2024-02-23T00:51:36.265364+0000","last_undegraded":"2024-02-23T00:51:36.265364+0000","last_fullsized":"2024-02-23T00:51:36.265364+0000","mapping_epoch":65,"log_start":"67'1","ondisk_log_start":"67'1","created":65,"last_epoch_clean":404,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:51:02.380721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:51:02.380721+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":484,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.250305+0000","last_change":"2024-02-23T00:51:10.298816+0000","last_active":"2024-02-23T00:51:35.250305+0000","last_peered":"2024-02-23T00:51:35.250305+0000","last_clean":"2024-02-23T00:51:35.250305+0000","last_became_active":"2024-02-23T00:42:58.690858+0000","last_became_peered":"2024-02-23T00:42:58.690858+0000","last_unstale":"2024-02-23T00:51:35.250305+0000","last_undegraded":"2024-02-23T00:51:35.250305+0000","last_fullsized":"2024-02-23T00:51:35.250305+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:10.298740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:35.360054+0000","last_clean_scrub_stamp":"2024-02-23T00:51:10.298740+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":542,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.252964+0000","last_change":"2024-02-23T00:49:28.358727+0000","last_active":"2024-02-23T00:51:35.252964+0000","last_peered":"2024-02-23T00:51:35.252964+0000","last_clean":"2024-02-23T00:51:35.252964+0000","last_became_active":"2024-02-23T00:47:28.231152+0000","last_became_peered":"2024-02-23T00:47:28.231152+0000","last_unstale":"2024-02-23T00:51:35.252964+0000","last_undegraded":"2024-02-23T00:51:35.252964+0000","last_fullsized":"2024-02-23T00:51:35.252964+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:28.358647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:28.358647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"457'26","reported_seq":580,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.251993+0000","last_change":"2024-02-23T00:51:01.219903+0000","last_active":"2024-02-23T00:51:35.251993+0000","last_peered":"2024-02-23T00:51:35.251993+0000","last_clean":"2024-02-23T00:51:35.251993+0000","last_became_active":"2024-02-23T00:39:16.858880+0000","last_became_peered":"2024-02-23T00:39:16.858880+0000","last_unstale":"2024-02-23T00:51:35.251993+0000","last_undegraded":"2024-02-23T00:51:35.251993+0000","last_fullsized":"2024-02-23T00:51:35.251993+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-23T00:51:01.219824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:01.219824+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":547,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.251142+0000","last_change":"2024-02-23T00:50:24.297660+0000","last_active":"2024-02-23T00:51:35.251142+0000","last_peered":"2024-02-23T00:51:35.251142+0000","last_clean":"2024-02-23T00:51:35.251142+0000","last_became_active":"2024-02-23T00:39:08.248249+0000","last_became_peered":"2024-02-23T00:39:08.248249+0000","last_unstale":"2024-02-23T00:51:35.251142+0000","last_undegraded":"2024-02-23T00:51:35.251142+0000","last_fullsized":"2024-02-23T00:51:35.251142+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:24.297572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:49.977248+0000","last_clean_scrub_stamp":"2024-02-23T00:50:24.297572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":547,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.247881+0000","last_change":"2024-02-23T00:50:21.746072+0000","last_active":"2024-02-23T00:51:35.247881+0000","last_peered":"2024-02-23T00:51:35.247881+0000","last_clean":"2024-02-23T00:51:35.247881+0000","last_became_active":"2024-02-23T00:47:28.240482+0000","last_became_peered":"2024-02-23T00:47:28.240482+0000","last_unstale":"2024-02-23T00:51:35.247881+0000","last_undegraded":"2024-02-23T00:51:35.247881+0000","last_fullsized":"2024-02-23T00:51:35.247881+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:21.745961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:21.745961+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":541,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.246625+0000","last_change":"2024-02-23T00:50:34.055990+0000","last_active":"2024-02-23T00:51:35.246625+0000","last_peered":"2024-02-23T00:51:35.246625+0000","last_clean":"2024-02-23T00:51:35.246625+0000","last_became_active":"2024-02-23T00:39:14.783489+0000","last_became_peered":"2024-02-23T00:39:14.783489+0000","last_unstale":"2024-02-23T00:51:35.246625+0000","last_undegraded":"2024-02-23T00:51:35.246625+0000","last_fullsized":"2024-02-23T00:51:35.246625+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:34.055923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:34.055923+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":568,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.248584+0000","last_change":"2024-02-23T00:49:35.756168+0000","last_active":"2024-02-23T00:51:35.248584+0000","last_peered":"2024-02-23T00:51:35.248584+0000","last_clean":"2024-02-23T00:51:35.248584+0000","last_became_active":"2024-02-23T00:47:45.182550+0000","last_became_peered":"2024-02-23T00:47:45.182550+0000","last_unstale":"2024-02-23T00:51:35.248584+0000","last_undegraded":"2024-02-23T00:51:35.248584+0000","last_fullsized":"2024-02-23T00:51:35.248584+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:35.756073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:49:35.756073+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":491,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.253743+0000","last_change":"2024-02-23T00:50:16.257963+0000","last_active":"2024-02-23T00:51:35.253743+0000","last_peered":"2024-02-23T00:51:35.253743+0000","last_clean":"2024-02-23T00:51:35.253743+0000","last_became_active":"2024-02-23T00:47:28.233475+0000","last_became_peered":"2024-02-23T00:47:28.233475+0000","last_unstale":"2024-02-23T00:51:35.253743+0000","last_undegraded":"2024-02-23T00:51:35.253743+0000","last_fullsized":"2024-02-23T00:51:35.253743+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:16.257896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:16.257896+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":211,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.248314+0000","last_change":"2024-02-23T00:51:12.709695+0000","last_active":"2024-02-23T00:51:35.248314+0000","last_peered":"2024-02-23T00:51:35.248314+0000","last_clean":"2024-02-23T00:51:35.248314+0000","last_became_active":"2024-02-23T00:47:28.209125+0000","last_became_peered":"2024-02-23T00:47:28.209125+0000","last_unstale":"2024-02-23T00:51:35.248314+0000","last_undegraded":"2024-02-23T00:51:35.248314+0000","last_fullsized":"2024-02-23T00:51:35.248314+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.709617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.264240+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.709617+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"457'23","reported_seq":575,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.265315+0000","last_change":"2024-02-23T00:48:11.828616+0000","last_active":"2024-02-23T00:51:36.265315+0000","last_peered":"2024-02-23T00:51:36.265315+0000","last_clean":"2024-02-23T00:51:36.265315+0000","last_became_active":"2024-02-23T00:39:16.787484+0000","last_became_peered":"2024-02-23T00:39:16.787484+0000","last_unstale":"2024-02-23T00:51:36.265315+0000","last_undegraded":"2024-02-23T00:51:36.265315+0000","last_fullsized":"2024-02-23T00:51:36.265315+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'19","last_scrub_stamp":"2024-02-23T00:48:11.828547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:48:11.828547+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":48,"num_read_kb":36,"num_write":26,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":208,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.246472+0000","last_change":"2024-02-23T00:49:35.877867+0000","last_active":"2024-02-23T00:51:35.246472+0000","last_peered":"2024-02-23T00:51:35.246472+0000","last_clean":"2024-02-23T00:51:35.246472+0000","last_became_active":"2024-02-23T00:47:28.209575+0000","last_became_peered":"2024-02-23T00:47:28.209575+0000","last_unstale":"2024-02-23T00:51:35.246472+0000","last_undegraded":"2024-02-23T00:51:35.246472+0000","last_fullsized":"2024-02-23T00:51:35.246472+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:35.877800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:35.877800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"68'5","reported_seq":1278,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.643900+0000","last_change":"2024-02-23T00:50:57.456201+0000","last_active":"2024-02-23T00:51:36.643900+0000","last_peered":"2024-02-23T00:51:36.643900+0000","last_clean":"2024-02-23T00:51:36.643900+0000","last_became_active":"2024-02-23T00:47:28.227928+0000","last_became_peered":"2024-02-23T00:47:28.227928+0000","last_unstale":"2024-02-23T00:51:36.643900+0000","last_undegraded":"2024-02-23T00:51:36.643900+0000","last_fullsized":"2024-02-23T00:51:36.643900+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'5","last_scrub_stamp":"2024-02-23T00:50:57.456059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:57.456059+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":732,"num_read_kb":727,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":567,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.252994+0000","last_change":"2024-02-23T00:49:30.396935+0000","last_active":"2024-02-23T00:51:35.252994+0000","last_peered":"2024-02-23T00:51:35.252994+0000","last_clean":"2024-02-23T00:51:35.252994+0000","last_became_active":"2024-02-23T00:47:52.552244+0000","last_became_peered":"2024-02-23T00:47:52.552244+0000","last_unstale":"2024-02-23T00:51:35.252994+0000","last_undegraded":"2024-02-23T00:51:35.252994+0000","last_fullsized":"2024-02-23T00:51:35.252994+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:30.396865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:49:30.396865+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":480,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.253518+0000","last_change":"2024-02-23T00:49:40.533441+0000","last_active":"2024-02-23T00:51:36.253518+0000","last_peered":"2024-02-23T00:51:36.253518+0000","last_clean":"2024-02-23T00:51:36.253518+0000","last_became_active":"2024-02-23T00:47:28.212516+0000","last_became_peered":"2024-02-23T00:47:28.212516+0000","last_unstale":"2024-02-23T00:51:36.253518+0000","last_undegraded":"2024-02-23T00:51:36.253518+0000","last_fullsized":"2024-02-23T00:51:36.253518+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:40.533344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:49:40.533344+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":534,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.251279+0000","last_change":"2024-02-23T00:49:37.787041+0000","last_active":"2024-02-23T00:51:35.251279+0000","last_peered":"2024-02-23T00:51:35.251279+0000","last_clean":"2024-02-23T00:51:35.251279+0000","last_became_active":"2024-02-23T00:39:18.802134+0000","last_became_peered":"2024-02-23T00:39:18.802134+0000","last_unstale":"2024-02-23T00:51:35.251279+0000","last_undegraded":"2024-02-23T00:51:35.251279+0000","last_fullsized":"2024-02-23T00:51:35.251279+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:37.786978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:37.786978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"457'48","reported_seq":719,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.247716+0000","last_change":"2024-02-23T00:50:30.763802+0000","last_active":"2024-02-23T00:51:35.247716+0000","last_peered":"2024-02-23T00:51:35.247716+0000","last_clean":"2024-02-23T00:51:35.247716+0000","last_became_active":"2024-02-23T00:47:28.240257+0000","last_became_peered":"2024-02-23T00:47:28.240257+0000","last_unstale":"2024-02-23T00:51:35.247716+0000","last_undegraded":"2024-02-23T00:51:35.247716+0000","last_fullsized":"2024-02-23T00:51:35.247716+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'48","last_scrub_stamp":"2024-02-23T00:50:30.763686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:30.763686+0000","log_size":48,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":78,"num_read_kb":62,"num_write":44,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":538,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.248461+0000","last_change":"2024-02-23T00:49:41.639660+0000","last_active":"2024-02-23T00:51:35.248461+0000","last_peered":"2024-02-23T00:51:35.248461+0000","last_clean":"2024-02-23T00:51:35.248461+0000","last_became_active":"2024-02-23T00:39:14.784015+0000","last_became_peered":"2024-02-23T00:39:14.784015+0000","last_unstale":"2024-02-23T00:51:35.248461+0000","last_undegraded":"2024-02-23T00:51:35.248461+0000","last_fullsized":"2024-02-23T00:51:35.248461+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:41.639582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:41.639582+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":566,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.248053+0000","last_change":"2024-02-23T00:49:36.867589+0000","last_active":"2024-02-23T00:51:35.248053+0000","last_peered":"2024-02-23T00:51:35.248053+0000","last_clean":"2024-02-23T00:51:35.248053+0000","last_became_active":"2024-02-23T00:47:49.525421+0000","last_became_peered":"2024-02-23T00:47:49.525421+0000","last_unstale":"2024-02-23T00:51:35.248053+0000","last_undegraded":"2024-02-23T00:51:35.248053+0000","last_fullsized":"2024-02-23T00:51:35.248053+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":397,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:36.867488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:49:36.867488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":207,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.246925+0000","last_change":"2024-02-23T00:50:13.549604+0000","last_active":"2024-02-23T00:51:35.246925+0000","last_peered":"2024-02-23T00:51:35.246925+0000","last_clean":"2024-02-23T00:51:35.246925+0000","last_became_active":"2024-02-23T00:47:28.232194+0000","last_became_peered":"2024-02-23T00:47:28.232194+0000","last_unstale":"2024-02-23T00:51:35.246925+0000","last_undegraded":"2024-02-23T00:51:35.246925+0000","last_fullsized":"2024-02-23T00:51:35.246925+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:13.549529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:13.549529+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3","version":"91'9","reported_seq":691,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.252673+0000","last_change":"2024-02-23T00:49:41.776249+0000","last_active":"2024-02-23T00:51:35.252673+0000","last_peered":"2024-02-23T00:51:35.252673+0000","last_clean":"2024-02-23T00:51:35.252673+0000","last_became_active":"2024-02-23T00:39:18.801297+0000","last_became_peered":"2024-02-23T00:39:18.801297+0000","last_unstale":"2024-02-23T00:51:35.252673+0000","last_undegraded":"2024-02-23T00:51:35.252673+0000","last_fullsized":"2024-02-23T00:51:35.252673+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'9","last_scrub_stamp":"2024-02-23T00:49:41.776117+0000","last_deep_scrub":"91'9","last_deep_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_clean_scrub_stamp":"2024-02-23T00:49:41.776117+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"457'26","reported_seq":572,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.256552+0000","last_change":"2024-02-23T00:48:55.589320+0000","last_active":"2024-02-23T00:51:36.256552+0000","last_peered":"2024-02-23T00:51:36.256552+0000","last_clean":"2024-02-23T00:51:36.256552+0000","last_became_active":"2024-02-23T00:39:16.787468+0000","last_became_peered":"2024-02-23T00:39:16.787468+0000","last_unstale":"2024-02-23T00:51:36.256552+0000","last_undegraded":"2024-02-23T00:51:36.256552+0000","last_fullsized":"2024-02-23T00:51:36.256552+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:48:55.589209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:48:55.589209+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":36,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":534,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.415429+0000","last_change":"2024-02-23T00:51:36.415402+0000","last_active":"2024-02-23T00:51:36.415429+0000","last_peered":"2024-02-23T00:51:36.415429+0000","last_clean":"2024-02-23T00:51:36.415429+0000","last_became_active":"2024-02-23T00:39:14.779952+0000","last_became_peered":"2024-02-23T00:39:14.779952+0000","last_unstale":"2024-02-23T00:51:36.415429+0000","last_undegraded":"2024-02-23T00:51:36.415429+0000","last_fullsized":"2024-02-23T00:51:36.415429+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:36.415340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:36.415340+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"107'2","reported_seq":567,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.250996+0000","last_change":"2024-02-23T00:50:17.587665+0000","last_active":"2024-02-23T00:51:35.250996+0000","last_peered":"2024-02-23T00:51:35.250996+0000","last_clean":"2024-02-23T00:51:35.250996+0000","last_became_active":"2024-02-23T00:47:41.210074+0000","last_became_peered":"2024-02-23T00:47:41.210074+0000","last_unstale":"2024-02-23T00:51:35.250996+0000","last_undegraded":"2024-02-23T00:51:35.250996+0000","last_fullsized":"2024-02-23T00:51:35.250996+0000","mapping_epoch":375,"log_start":"107'2","ondisk_log_start":"107'2","created":65,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'2","last_scrub_stamp":"2024-02-23T00:50:17.587591+0000","last_deep_scrub":"107'2","last_deep_scrub_stamp":"2024-02-23T00:42:48.362739+0000","last_clean_scrub_stamp":"2024-02-23T00:50:17.587591+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":488,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.252113+0000","last_change":"2024-02-23T00:50:18.308436+0000","last_active":"2024-02-23T00:51:35.252113+0000","last_peered":"2024-02-23T00:51:35.252113+0000","last_clean":"2024-02-23T00:51:35.252113+0000","last_became_active":"2024-02-23T00:47:28.242255+0000","last_became_peered":"2024-02-23T00:47:28.242255+0000","last_unstale":"2024-02-23T00:51:35.252113+0000","last_undegraded":"2024-02-23T00:51:35.252113+0000","last_fullsized":"2024-02-23T00:51:35.252113+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:18.308350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:18.308350+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":211,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.631688+0000","last_change":"2024-02-23T00:51:35.631662+0000","last_active":"2024-02-23T00:51:35.631688+0000","last_peered":"2024-02-23T00:51:35.631688+0000","last_clean":"2024-02-23T00:51:35.631688+0000","last_became_active":"2024-02-23T00:47:28.234034+0000","last_became_peered":"2024-02-23T00:47:28.234034+0000","last_unstale":"2024-02-23T00:51:35.631688+0000","last_undegraded":"2024-02-23T00:51:35.631688+0000","last_fullsized":"2024-02-23T00:51:35.631688+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.631598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.631598+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"457'31","reported_seq":592,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.247074+0000","last_change":"2024-02-23T00:49:08.190437+0000","last_active":"2024-02-23T00:51:35.247074+0000","last_peered":"2024-02-23T00:51:35.247074+0000","last_clean":"2024-02-23T00:51:35.247074+0000","last_became_active":"2024-02-23T00:39:16.783926+0000","last_became_peered":"2024-02-23T00:39:16.783926+0000","last_unstale":"2024-02-23T00:51:35.247074+0000","last_undegraded":"2024-02-23T00:51:35.247074+0000","last_fullsized":"2024-02-23T00:51:35.247074+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'25","last_scrub_stamp":"2024-02-23T00:49:08.190363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:08.190363+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":58,"num_read_kb":47,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"95'4","reported_seq":551,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.251505+0000","last_change":"2024-02-23T00:49:33.465769+0000","last_active":"2024-02-23T00:51:35.251505+0000","last_peered":"2024-02-23T00:51:35.251505+0000","last_clean":"2024-02-23T00:51:35.251505+0000","last_became_active":"2024-02-23T00:39:14.779205+0000","last_became_peered":"2024-02-23T00:39:14.779205+0000","last_unstale":"2024-02-23T00:51:35.251505+0000","last_undegraded":"2024-02-23T00:51:35.251505+0000","last_fullsized":"2024-02-23T00:51:35.251505+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'4","last_scrub_stamp":"2024-02-23T00:49:33.465697+0000","last_deep_scrub":"95'4","last_deep_scrub_stamp":"2024-02-23T00:41:43.874397+0000","last_clean_scrub_stamp":"2024-02-23T00:49:33.465697+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":553,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.247152+0000","last_change":"2024-02-23T00:50:15.922829+0000","last_active":"2024-02-23T00:51:35.247152+0000","last_peered":"2024-02-23T00:51:35.247152+0000","last_clean":"2024-02-23T00:51:35.247152+0000","last_became_active":"2024-02-23T00:43:27.611660+0000","last_became_peered":"2024-02-23T00:43:27.611660+0000","last_unstale":"2024-02-23T00:51:35.247152+0000","last_undegraded":"2024-02-23T00:51:35.247152+0000","last_fullsized":"2024-02-23T00:51:35.247152+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:15.922745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:50:15.922745+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":489,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.253302+0000","last_change":"2024-02-23T00:49:44.143768+0000","last_active":"2024-02-23T00:51:35.253302+0000","last_peered":"2024-02-23T00:51:35.253302+0000","last_clean":"2024-02-23T00:51:35.253302+0000","last_became_active":"2024-02-23T00:47:28.231670+0000","last_became_peered":"2024-02-23T00:47:28.231670+0000","last_unstale":"2024-02-23T00:51:35.253302+0000","last_undegraded":"2024-02-23T00:51:35.253302+0000","last_fullsized":"2024-02-23T00:51:35.253302+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:44.143684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:49:44.143684+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"457'27","reported_seq":586,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.251188+0000","last_change":"2024-02-23T00:49:36.823322+0000","last_active":"2024-02-23T00:51:35.251188+0000","last_peered":"2024-02-23T00:51:35.251188+0000","last_clean":"2024-02-23T00:51:35.251188+0000","last_became_active":"2024-02-23T00:39:16.858760+0000","last_became_peered":"2024-02-23T00:39:16.858760+0000","last_unstale":"2024-02-23T00:51:35.251188+0000","last_undegraded":"2024-02-23T00:51:35.251188+0000","last_fullsized":"2024-02-23T00:51:35.251188+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-23T00:49:36.823214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:36.823214+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":204,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.253531+0000","last_change":"2024-02-23T00:49:02.601790+0000","last_active":"2024-02-23T00:51:36.253531+0000","last_peered":"2024-02-23T00:51:36.253531+0000","last_clean":"2024-02-23T00:51:36.253531+0000","last_became_active":"2024-02-23T00:47:28.219764+0000","last_became_peered":"2024-02-23T00:47:28.219764+0000","last_unstale":"2024-02-23T00:51:36.253531+0000","last_undegraded":"2024-02-23T00:51:36.253531+0000","last_fullsized":"2024-02-23T00:51:36.253531+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:02.601726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:02.601726+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":537,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.247712+0000","last_change":"2024-02-23T00:49:37.881251+0000","last_active":"2024-02-23T00:51:35.247712+0000","last_peered":"2024-02-23T00:51:35.247712+0000","last_clean":"2024-02-23T00:51:35.247712+0000","last_became_active":"2024-02-23T00:39:18.798624+0000","last_became_peered":"2024-02-23T00:39:18.798624+0000","last_unstale":"2024-02-23T00:51:35.247712+0000","last_undegraded":"2024-02-23T00:51:35.247712+0000","last_fullsized":"2024-02-23T00:51:35.247712+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:37.881183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:45.913526+0000","last_clean_scrub_stamp":"2024-02-23T00:49:37.881183+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":479,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.251946+0000","last_change":"2024-02-23T00:50:13.642424+0000","last_active":"2024-02-23T00:51:35.251946+0000","last_peered":"2024-02-23T00:51:35.251946+0000","last_clean":"2024-02-23T00:51:35.251946+0000","last_became_active":"2024-02-23T00:42:58.825015+0000","last_became_peered":"2024-02-23T00:42:58.825015+0000","last_unstale":"2024-02-23T00:51:35.251946+0000","last_undegraded":"2024-02-23T00:51:35.251946+0000","last_fullsized":"2024-02-23T00:51:35.251946+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:13.642359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:13.642359+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"457'22","reported_seq":576,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.247632+0000","last_change":"2024-02-23T00:51:01.201807+0000","last_active":"2024-02-23T00:51:35.247632+0000","last_peered":"2024-02-23T00:51:35.247632+0000","last_clean":"2024-02-23T00:51:35.247632+0000","last_became_active":"2024-02-23T00:39:16.858449+0000","last_became_peered":"2024-02-23T00:39:16.858449+0000","last_unstale":"2024-02-23T00:51:35.247632+0000","last_undegraded":"2024-02-23T00:51:35.247632+0000","last_fullsized":"2024-02-23T00:51:35.247632+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'22","last_scrub_stamp":"2024-02-23T00:51:01.201737+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-02-23T00:48:07.941101+0000","last_clean_scrub_stamp":"2024-02-23T00:51:01.201737+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":30,"num_read_kb":26,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":537,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.250668+0000","last_change":"2024-02-23T00:49:28.347354+0000","last_active":"2024-02-23T00:51:35.250668+0000","last_peered":"2024-02-23T00:51:35.250668+0000","last_clean":"2024-02-23T00:51:35.250668+0000","last_became_active":"2024-02-23T00:39:14.783164+0000","last_became_peered":"2024-02-23T00:39:14.783164+0000","last_unstale":"2024-02-23T00:51:35.250668+0000","last_undegraded":"2024-02-23T00:51:35.250668+0000","last_fullsized":"2024-02-23T00:51:35.250668+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:28.347281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:28.347281+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":534,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.248425+0000","last_change":"2024-02-23T00:49:37.721639+0000","last_active":"2024-02-23T00:51:35.248425+0000","last_peered":"2024-02-23T00:51:35.248425+0000","last_clean":"2024-02-23T00:51:35.248425+0000","last_became_active":"2024-02-23T00:39:18.800297+0000","last_became_peered":"2024-02-23T00:39:18.800297+0000","last_unstale":"2024-02-23T00:51:35.248425+0000","last_undegraded":"2024-02-23T00:51:35.248425+0000","last_fullsized":"2024-02-23T00:51:35.248425+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:37.721563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:37.721563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1","version":"110'7","reported_seq":495,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.246659+0000","last_change":"2024-02-23T00:51:13.352686+0000","last_active":"2024-02-23T00:51:35.246659+0000","last_peered":"2024-02-23T00:51:35.246659+0000","last_clean":"2024-02-23T00:51:35.246659+0000","last_became_active":"2024-02-23T00:42:58.777251+0000","last_became_peered":"2024-02-23T00:42:58.777251+0000","last_unstale":"2024-02-23T00:51:35.246659+0000","last_undegraded":"2024-02-23T00:51:35.246659+0000","last_fullsized":"2024-02-23T00:51:35.246659+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'7","last_scrub_stamp":"2024-02-23T00:51:13.352611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:13.352611+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"457'32","reported_seq":237,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.246199+0000","last_change":"2024-02-23T00:50:25.079859+0000","last_active":"2024-02-23T00:51:35.246199+0000","last_peered":"2024-02-23T00:51:35.246199+0000","last_clean":"2024-02-23T00:51:35.246199+0000","last_became_active":"2024-02-23T00:47:28.237898+0000","last_became_peered":"2024-02-23T00:47:28.237898+0000","last_unstale":"2024-02-23T00:51:35.246199+0000","last_undegraded":"2024-02-23T00:51:35.246199+0000","last_fullsized":"2024-02-23T00:51:35.246199+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-23T00:50:25.079789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:25.079789+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":52,"num_read_kb":45,"num_write":32,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":543,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.248383+0000","last_change":"2024-02-23T00:51:14.658413+0000","last_active":"2024-02-23T00:51:35.248383+0000","last_peered":"2024-02-23T00:51:35.248383+0000","last_clean":"2024-02-23T00:51:35.248383+0000","last_became_active":"2024-02-23T00:39:14.785590+0000","last_became_peered":"2024-02-23T00:39:14.785590+0000","last_unstale":"2024-02-23T00:51:35.248383+0000","last_undegraded":"2024-02-23T00:51:35.248383+0000","last_fullsized":"2024-02-23T00:51:35.248383+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:14.658297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_clean_scrub_stamp":"2024-02-23T00:51:14.658297+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"336'9","reported_seq":691,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.248186+0000","last_change":"2024-02-23T00:49:39.636503+0000","last_active":"2024-02-23T00:51:35.248186+0000","last_peered":"2024-02-23T00:51:35.248186+0000","last_clean":"2024-02-23T00:51:35.248186+0000","last_became_active":"2024-02-23T00:39:18.803566+0000","last_became_peered":"2024-02-23T00:39:18.803566+0000","last_unstale":"2024-02-23T00:51:35.248186+0000","last_undegraded":"2024-02-23T00:51:35.248186+0000","last_fullsized":"2024-02-23T00:51:35.248186+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"336'9","last_scrub_stamp":"2024-02-23T00:49:39.636409+0000","last_deep_scrub":"336'9","last_deep_scrub_stamp":"2024-02-23T00:46:49.160991+0000","last_clean_scrub_stamp":"2024-02-23T00:49:39.636409+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":480,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.252803+0000","last_change":"2024-02-23T00:51:33.285134+0000","last_active":"2024-02-23T00:51:35.252803+0000","last_peered":"2024-02-23T00:51:35.252803+0000","last_clean":"2024-02-23T00:51:35.252803+0000","last_became_active":"2024-02-23T00:42:58.757424+0000","last_became_peered":"2024-02-23T00:42:58.757424+0000","last_unstale":"2024-02-23T00:51:35.252803+0000","last_undegraded":"2024-02-23T00:51:35.252803+0000","last_fullsized":"2024-02-23T00:51:35.252803+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:33.285048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:33.285048+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"457'21","reported_seq":576,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.248083+0000","last_change":"2024-02-23T00:50:17.861556+0000","last_active":"2024-02-23T00:51:35.248083+0000","last_peered":"2024-02-23T00:51:35.248083+0000","last_clean":"2024-02-23T00:51:35.248083+0000","last_became_active":"2024-02-23T00:39:16.858953+0000","last_became_peered":"2024-02-23T00:39:16.858953+0000","last_unstale":"2024-02-23T00:51:35.248083+0000","last_undegraded":"2024-02-23T00:51:35.248083+0000","last_fullsized":"2024-02-23T00:51:35.248083+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-23T00:50:17.861490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:17.861490+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":543,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.252150+0000","last_change":"2024-02-23T00:51:33.662415+0000","last_active":"2024-02-23T00:51:35.252150+0000","last_peered":"2024-02-23T00:51:35.252150+0000","last_clean":"2024-02-23T00:51:35.252150+0000","last_became_active":"2024-02-23T00:39:14.773772+0000","last_became_peered":"2024-02-23T00:39:14.773772+0000","last_unstale":"2024-02-23T00:51:35.252150+0000","last_undegraded":"2024-02-23T00:51:35.252150+0000","last_fullsized":"2024-02-23T00:51:35.252150+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:33.662354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:33.662354+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"376'9","reported_seq":701,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.248355+0000","last_change":"2024-02-23T00:49:45.868359+0000","last_active":"2024-02-23T00:51:35.248355+0000","last_peered":"2024-02-23T00:51:35.248355+0000","last_clean":"2024-02-23T00:51:35.248355+0000","last_became_active":"2024-02-23T00:47:28.229316+0000","last_became_peered":"2024-02-23T00:47:28.229316+0000","last_unstale":"2024-02-23T00:51:35.248355+0000","last_undegraded":"2024-02-23T00:51:35.248355+0000","last_fullsized":"2024-02-23T00:51:35.248355+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"376'9","last_scrub_stamp":"2024-02-23T00:49:45.868289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:45.868289+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":471,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.255977+0000","last_change":"2024-02-23T00:50:15.714265+0000","last_active":"2024-02-23T00:51:36.255977+0000","last_peered":"2024-02-23T00:51:36.255977+0000","last_clean":"2024-02-23T00:51:36.255977+0000","last_became_active":"2024-02-23T00:42:57.652488+0000","last_became_peered":"2024-02-23T00:42:57.652488+0000","last_unstale":"2024-02-23T00:51:36.255977+0000","last_undegraded":"2024-02-23T00:51:36.255977+0000","last_fullsized":"2024-02-23T00:51:36.255977+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:15.714200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:51.254903+0000","last_clean_scrub_stamp":"2024-02-23T00:50:15.714200+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"457'15","reported_seq":571,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.256304+0000","last_change":"2024-02-23T00:49:39.573557+0000","last_active":"2024-02-23T00:51:36.256304+0000","last_peered":"2024-02-23T00:51:36.256304+0000","last_clean":"2024-02-23T00:51:36.256304+0000","last_became_active":"2024-02-23T00:47:28.225585+0000","last_became_peered":"2024-02-23T00:47:28.225585+0000","last_unstale":"2024-02-23T00:51:36.256304+0000","last_undegraded":"2024-02-23T00:51:36.256304+0000","last_fullsized":"2024-02-23T00:51:36.256304+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'15","last_scrub_stamp":"2024-02-23T00:49:39.573472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:39.573472+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":542,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.251985+0000","last_change":"2024-02-23T00:49:02.303864+0000","last_active":"2024-02-23T00:51:35.251985+0000","last_peered":"2024-02-23T00:51:35.251985+0000","last_clean":"2024-02-23T00:51:35.251985+0000","last_became_active":"2024-02-23T00:39:14.778455+0000","last_became_peered":"2024-02-23T00:39:14.778455+0000","last_unstale":"2024-02-23T00:51:35.251985+0000","last_undegraded":"2024-02-23T00:51:35.251985+0000","last_fullsized":"2024-02-23T00:51:35.251985+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:02.303789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:02.303789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":546,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.247068+0000","last_change":"2024-02-23T00:50:30.069404+0000","last_active":"2024-02-23T00:51:35.247068+0000","last_peered":"2024-02-23T00:51:35.247068+0000","last_clean":"2024-02-23T00:51:35.247068+0000","last_became_active":"2024-02-23T00:47:44.225379+0000","last_became_peered":"2024-02-23T00:47:44.225379+0000","last_unstale":"2024-02-23T00:51:35.247068+0000","last_undegraded":"2024-02-23T00:51:35.247068+0000","last_fullsized":"2024-02-23T00:51:35.247068+0000","mapping_epoch":391,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":392,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:30.069325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_clean_scrub_stamp":"2024-02-23T00:50:30.069325+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":479,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.253413+0000","last_change":"2024-02-23T00:49:40.770764+0000","last_active":"2024-02-23T00:51:35.253413+0000","last_peered":"2024-02-23T00:51:35.253413+0000","last_clean":"2024-02-23T00:51:35.253413+0000","last_became_active":"2024-02-23T00:42:58.835862+0000","last_became_peered":"2024-02-23T00:42:58.835862+0000","last_unstale":"2024-02-23T00:51:35.253413+0000","last_undegraded":"2024-02-23T00:51:35.253413+0000","last_fullsized":"2024-02-23T00:51:35.253413+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:40.770694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:49:40.770694+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"457'19","reported_seq":565,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.246528+0000","last_change":"2024-02-23T00:50:29.599563+0000","last_active":"2024-02-23T00:51:35.246528+0000","last_peered":"2024-02-23T00:51:35.246528+0000","last_clean":"2024-02-23T00:51:35.246528+0000","last_became_active":"2024-02-23T00:39:16.783896+0000","last_became_peered":"2024-02-23T00:39:16.783896+0000","last_unstale":"2024-02-23T00:51:35.246528+0000","last_undegraded":"2024-02-23T00:51:35.246528+0000","last_fullsized":"2024-02-23T00:51:35.246528+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'19","last_scrub_stamp":"2024-02-23T00:50:29.599483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:29.599483+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"202'2","reported_seq":537,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.260223+0000","last_change":"2024-02-23T00:48:04.832975+0000","last_active":"2024-02-23T00:51:36.260223+0000","last_peered":"2024-02-23T00:51:36.260223+0000","last_clean":"2024-02-23T00:51:36.260223+0000","last_became_active":"2024-02-23T00:39:14.775804+0000","last_became_peered":"2024-02-23T00:39:14.775804+0000","last_unstale":"2024-02-23T00:51:36.260223+0000","last_undegraded":"2024-02-23T00:51:36.260223+0000","last_fullsized":"2024-02-23T00:51:36.260223+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"202'2","last_scrub_stamp":"2024-02-23T00:48:04.832904+0000","last_deep_scrub":"202'2","last_deep_scrub_stamp":"2024-02-23T00:44:34.720640+0000","last_clean_scrub_stamp":"2024-02-23T00:48:04.832904+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":525,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.256434+0000","last_change":"2024-02-23T00:50:18.698805+0000","last_active":"2024-02-23T00:51:36.256434+0000","last_peered":"2024-02-23T00:51:36.256434+0000","last_clean":"2024-02-23T00:51:36.256434+0000","last_became_active":"2024-02-23T00:39:18.800382+0000","last_became_peered":"2024-02-23T00:39:18.800382+0000","last_unstale":"2024-02-23T00:51:36.256434+0000","last_undegraded":"2024-02-23T00:51:36.256434+0000","last_fullsized":"2024-02-23T00:51:36.256434+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:18.698687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:50:18.698687+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":480,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.260117+0000","last_change":"2024-02-23T00:49:46.462085+0000","last_active":"2024-02-23T00:51:36.260117+0000","last_peered":"2024-02-23T00:51:36.260117+0000","last_clean":"2024-02-23T00:51:36.260117+0000","last_became_active":"2024-02-23T00:47:28.224961+0000","last_became_peered":"2024-02-23T00:47:28.224961+0000","last_unstale":"2024-02-23T00:51:36.260117+0000","last_undegraded":"2024-02-23T00:51:36.260117+0000","last_fullsized":"2024-02-23T00:51:36.260117+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:46.462015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:46.462015+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.462015+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"457'8","reported_seq":551,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.246121+0000","last_change":"2024-02-23T00:50:35.045697+0000","last_active":"2024-02-23T00:51:35.246121+0000","last_peered":"2024-02-23T00:51:35.246121+0000","last_clean":"2024-02-23T00:51:35.246121+0000","last_became_active":"2024-02-23T00:39:16.785830+0000","last_became_peered":"2024-02-23T00:39:16.785830+0000","last_unstale":"2024-02-23T00:51:35.246121+0000","last_undegraded":"2024-02-23T00:51:35.246121+0000","last_fullsized":"2024-02-23T00:51:35.246121+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'8","last_scrub_stamp":"2024-02-23T00:50:35.045612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:35.045612+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":555,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.248305+0000","last_change":"2024-02-23T00:51:34.529078+0000","last_active":"2024-02-23T00:51:35.248305+0000","last_peered":"2024-02-23T00:51:35.248305+0000","last_clean":"2024-02-23T00:51:35.248305+0000","last_became_active":"2024-02-23T00:47:28.224477+0000","last_became_peered":"2024-02-23T00:47:28.224477+0000","last_unstale":"2024-02-23T00:51:35.248305+0000","last_undegraded":"2024-02-23T00:51:35.248305+0000","last_fullsized":"2024-02-23T00:51:35.248305+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:34.529012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:34.529012+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":547,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.250561+0000","last_change":"2024-02-23T00:49:33.461971+0000","last_active":"2024-02-23T00:51:35.250561+0000","last_peered":"2024-02-23T00:51:35.250561+0000","last_clean":"2024-02-23T00:51:35.250561+0000","last_became_active":"2024-02-23T00:47:44.230295+0000","last_became_peered":"2024-02-23T00:47:44.230295+0000","last_unstale":"2024-02-23T00:51:35.250561+0000","last_undegraded":"2024-02-23T00:51:35.250561+0000","last_fullsized":"2024-02-23T00:51:35.250561+0000","mapping_epoch":391,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":392,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:33.461907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:33.461907+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":475,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.253355+0000","last_change":"2024-02-23T00:48:51.717473+0000","last_active":"2024-02-23T00:51:35.253355+0000","last_peered":"2024-02-23T00:51:35.253355+0000","last_clean":"2024-02-23T00:51:35.253355+0000","last_became_active":"2024-02-23T00:42:58.741365+0000","last_became_peered":"2024-02-23T00:42:58.741365+0000","last_unstale":"2024-02-23T00:51:35.253355+0000","last_undegraded":"2024-02-23T00:51:35.253355+0000","last_fullsized":"2024-02-23T00:51:35.253355+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:51.717397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:51.717397+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"457'25","reported_seq":585,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.246852+0000","last_change":"2024-02-23T00:49:47.788110+0000","last_active":"2024-02-23T00:51:35.246852+0000","last_peered":"2024-02-23T00:51:35.246852+0000","last_clean":"2024-02-23T00:51:35.246852+0000","last_became_active":"2024-02-23T00:39:16.857232+0000","last_became_peered":"2024-02-23T00:39:16.857232+0000","last_unstale":"2024-02-23T00:51:35.246852+0000","last_undegraded":"2024-02-23T00:51:35.246852+0000","last_fullsized":"2024-02-23T00:51:35.246852+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-23T00:49:47.788044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:47.788044+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":551,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.252475+0000","last_change":"2024-02-23T00:51:35.218650+0000","last_active":"2024-02-23T00:51:35.252475+0000","last_peered":"2024-02-23T00:51:35.252475+0000","last_clean":"2024-02-23T00:51:35.252475+0000","last_became_active":"2024-02-23T00:47:28.240162+0000","last_became_peered":"2024-02-23T00:47:28.240162+0000","last_unstale":"2024-02-23T00:51:35.252475+0000","last_undegraded":"2024-02-23T00:51:35.252475+0000","last_fullsized":"2024-02-23T00:51:35.252475+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.218583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:44.285083+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.218583+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":190,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.246032+0000","last_change":"2024-02-23T00:49:42.872820+0000","last_active":"2024-02-23T00:51:35.246032+0000","last_peered":"2024-02-23T00:51:35.246032+0000","last_clean":"2024-02-23T00:51:35.246032+0000","last_became_active":"2024-02-23T00:47:44.226097+0000","last_became_peered":"2024-02-23T00:47:44.226097+0000","last_unstale":"2024-02-23T00:51:35.246032+0000","last_undegraded":"2024-02-23T00:51:35.246032+0000","last_fullsized":"2024-02-23T00:51:35.246032+0000","mapping_epoch":391,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":392,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:42.872748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:42.872748+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":212,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.248418+0000","last_change":"2024-02-23T00:50:28.678155+0000","last_active":"2024-02-23T00:51:35.248418+0000","last_peered":"2024-02-23T00:51:35.248418+0000","last_clean":"2024-02-23T00:51:35.248418+0000","last_became_active":"2024-02-23T00:47:28.200688+0000","last_became_peered":"2024-02-23T00:47:28.200688+0000","last_unstale":"2024-02-23T00:51:35.248418+0000","last_undegraded":"2024-02-23T00:51:35.248418+0000","last_fullsized":"2024-02-23T00:51:35.248418+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:28.678041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:28.678041+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"457'26","reported_seq":581,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.251497+0000","last_change":"2024-02-23T00:50:23.540361+0000","last_active":"2024-02-23T00:51:35.251497+0000","last_peered":"2024-02-23T00:51:35.251497+0000","last_clean":"2024-02-23T00:51:35.251497+0000","last_became_active":"2024-02-23T00:39:16.858945+0000","last_became_peered":"2024-02-23T00:39:16.858945+0000","last_unstale":"2024-02-23T00:51:35.251497+0000","last_undegraded":"2024-02-23T00:51:35.251497+0000","last_fullsized":"2024-02-23T00:51:35.251497+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-23T00:50:23.540232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:23.540232+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":36,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":550,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.247561+0000","last_change":"2024-02-23T00:51:09.827935+0000","last_active":"2024-02-23T00:51:35.247561+0000","last_peered":"2024-02-23T00:51:35.247561+0000","last_clean":"2024-02-23T00:51:35.247561+0000","last_became_active":"2024-02-23T00:39:14.785363+0000","last_became_peered":"2024-02-23T00:39:14.785363+0000","last_unstale":"2024-02-23T00:51:35.247561+0000","last_undegraded":"2024-02-23T00:51:35.247561+0000","last_fullsized":"2024-02-23T00:51:35.247561+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"80'2","last_scrub_stamp":"2024-02-23T00:51:09.827857+0000","last_deep_scrub":"80'2","last_deep_scrub_stamp":"2024-02-23T00:40:15.942725+0000","last_clean_scrub_stamp":"2024-02-23T00:51:09.827857+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"93'9","reported_seq":693,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.252640+0000","last_change":"2024-02-23T00:48:53.246622+0000","last_active":"2024-02-23T00:51:35.252640+0000","last_peered":"2024-02-23T00:51:35.252640+0000","last_clean":"2024-02-23T00:51:35.252640+0000","last_became_active":"2024-02-23T00:39:18.796680+0000","last_became_peered":"2024-02-23T00:39:18.796680+0000","last_unstale":"2024-02-23T00:51:35.252640+0000","last_undegraded":"2024-02-23T00:51:35.252640+0000","last_fullsized":"2024-02-23T00:51:35.252640+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-02-23T00:48:53.246535+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-02-23T00:43:50.393713+0000","last_clean_scrub_stamp":"2024-02-23T00:48:53.246535+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":488,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.251853+0000","last_change":"2024-02-23T00:51:12.493007+0000","last_active":"2024-02-23T00:51:35.251853+0000","last_peered":"2024-02-23T00:51:35.251853+0000","last_clean":"2024-02-23T00:51:35.251853+0000","last_became_active":"2024-02-23T00:47:28.234641+0000","last_became_peered":"2024-02-23T00:47:28.234641+0000","last_unstale":"2024-02-23T00:51:35.251853+0000","last_undegraded":"2024-02-23T00:51:35.251853+0000","last_fullsized":"2024-02-23T00:51:35.251853+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.492942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.492942+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"457'31","reported_seq":597,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.252727+0000","last_change":"2024-02-23T00:51:15.408847+0000","last_active":"2024-02-23T00:51:35.252727+0000","last_peered":"2024-02-23T00:51:35.252727+0000","last_clean":"2024-02-23T00:51:35.252727+0000","last_became_active":"2024-02-23T00:39:16.782660+0000","last_became_peered":"2024-02-23T00:39:16.782660+0000","last_unstale":"2024-02-23T00:51:35.252727+0000","last_undegraded":"2024-02-23T00:51:35.252727+0000","last_fullsized":"2024-02-23T00:51:35.252727+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:51:15.408782+0000","last_deep_scrub":"93'23","last_deep_scrub_stamp":"2024-02-23T00:41:35.954516+0000","last_clean_scrub_stamp":"2024-02-23T00:51:15.408782+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":541,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.246258+0000","last_change":"2024-02-23T00:50:29.029794+0000","last_active":"2024-02-23T00:51:35.246258+0000","last_peered":"2024-02-23T00:51:35.246258+0000","last_clean":"2024-02-23T00:51:35.246258+0000","last_became_active":"2024-02-23T00:39:14.781560+0000","last_became_peered":"2024-02-23T00:39:14.781560+0000","last_unstale":"2024-02-23T00:51:35.246258+0000","last_undegraded":"2024-02-23T00:51:35.246258+0000","last_fullsized":"2024-02-23T00:51:35.246258+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:29.029725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:29.029725+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":548,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.251582+0000","last_change":"2024-02-23T00:51:14.488316+0000","last_active":"2024-02-23T00:51:35.251582+0000","last_peered":"2024-02-23T00:51:35.251582+0000","last_clean":"2024-02-23T00:51:35.251582+0000","last_became_active":"2024-02-23T00:47:28.232797+0000","last_became_peered":"2024-02-23T00:47:28.232797+0000","last_unstale":"2024-02-23T00:51:35.251582+0000","last_undegraded":"2024-02-23T00:51:35.251582+0000","last_fullsized":"2024-02-23T00:51:35.251582+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:14.488205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:53.189881+0000","last_clean_scrub_stamp":"2024-02-23T00:51:14.488205+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":482,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.246335+0000","last_change":"2024-02-23T00:51:00.232343+0000","last_active":"2024-02-23T00:51:35.246335+0000","last_peered":"2024-02-23T00:51:35.246335+0000","last_clean":"2024-02-23T00:51:35.246335+0000","last_became_active":"2024-02-23T00:42:58.774720+0000","last_became_peered":"2024-02-23T00:42:58.774720+0000","last_unstale":"2024-02-23T00:51:35.246335+0000","last_undegraded":"2024-02-23T00:51:35.246335+0000","last_fullsized":"2024-02-23T00:51:35.246335+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.232261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.232261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"457'27","reported_seq":275,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.260296+0000","last_change":"2024-02-23T00:49:26.518048+0000","last_active":"2024-02-23T00:51:36.260296+0000","last_peered":"2024-02-23T00:51:36.260296+0000","last_clean":"2024-02-23T00:51:36.260296+0000","last_became_active":"2024-02-23T00:47:28.211342+0000","last_became_peered":"2024-02-23T00:47:28.211342+0000","last_unstale":"2024-02-23T00:51:36.260296+0000","last_undegraded":"2024-02-23T00:51:36.260296+0000","last_fullsized":"2024-02-23T00:51:36.260296+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-23T00:49:26.517965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:26.517965+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":542,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.250636+0000","last_change":"2024-02-23T00:51:08.235807+0000","last_active":"2024-02-23T00:51:35.250636+0000","last_peered":"2024-02-23T00:51:35.250636+0000","last_clean":"2024-02-23T00:51:35.250636+0000","last_became_active":"2024-02-23T00:39:14.782494+0000","last_became_peered":"2024-02-23T00:39:14.782494+0000","last_unstale":"2024-02-23T00:51:35.250636+0000","last_undegraded":"2024-02-23T00:51:35.250636+0000","last_fullsized":"2024-02-23T00:51:35.250636+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:08.235735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:08.235735+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":525,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.265171+0000","last_change":"2024-02-23T00:48:16.788875+0000","last_active":"2024-02-23T00:51:36.265171+0000","last_peered":"2024-02-23T00:51:36.265171+0000","last_clean":"2024-02-23T00:51:36.265171+0000","last_became_active":"2024-02-23T00:39:18.801039+0000","last_became_peered":"2024-02-23T00:39:18.801039+0000","last_unstale":"2024-02-23T00:51:36.265171+0000","last_undegraded":"2024-02-23T00:51:36.265171+0000","last_fullsized":"2024-02-23T00:51:36.265171+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:16.788806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.641689+0000","last_clean_scrub_stamp":"2024-02-23T00:48:16.788806+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":482,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.247138+0000","last_change":"2024-02-23T00:51:10.341831+0000","last_active":"2024-02-23T00:51:35.247138+0000","last_peered":"2024-02-23T00:51:35.247138+0000","last_clean":"2024-02-23T00:51:35.247138+0000","last_became_active":"2024-02-23T00:42:58.770759+0000","last_became_peered":"2024-02-23T00:42:58.770759+0000","last_unstale":"2024-02-23T00:51:35.247138+0000","last_undegraded":"2024-02-23T00:51:35.247138+0000","last_fullsized":"2024-02-23T00:51:35.247138+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:10.341768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:10.341768+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"457'24","reported_seq":586,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.248562+0000","last_change":"2024-02-23T00:50:13.861288+0000","last_active":"2024-02-23T00:51:35.248562+0000","last_peered":"2024-02-23T00:51:35.248562+0000","last_clean":"2024-02-23T00:51:35.248562+0000","last_became_active":"2024-02-23T00:39:16.782803+0000","last_became_peered":"2024-02-23T00:39:16.782803+0000","last_unstale":"2024-02-23T00:51:35.248562+0000","last_undegraded":"2024-02-23T00:51:35.248562+0000","last_fullsized":"2024-02-23T00:51:35.248562+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'24","last_scrub_stamp":"2024-02-23T00:50:13.861220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:13.861220+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":46,"num_read_kb":36,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":541,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.246987+0000","last_change":"2024-02-23T00:50:28.060186+0000","last_active":"2024-02-23T00:51:35.246987+0000","last_peered":"2024-02-23T00:51:35.246987+0000","last_clean":"2024-02-23T00:51:35.246987+0000","last_became_active":"2024-02-23T00:39:14.782163+0000","last_became_peered":"2024-02-23T00:39:14.782163+0000","last_unstale":"2024-02-23T00:51:35.246987+0000","last_undegraded":"2024-02-23T00:51:35.246987+0000","last_fullsized":"2024-02-23T00:51:35.246987+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:28.060109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:28.060109+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":541,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.247401+0000","last_change":"2024-02-23T00:50:18.629287+0000","last_active":"2024-02-23T00:51:35.247401+0000","last_peered":"2024-02-23T00:51:35.247401+0000","last_clean":"2024-02-23T00:51:35.247401+0000","last_became_active":"2024-02-23T00:47:28.215452+0000","last_became_peered":"2024-02-23T00:47:28.215452+0000","last_unstale":"2024-02-23T00:51:35.247401+0000","last_undegraded":"2024-02-23T00:51:35.247401+0000","last_fullsized":"2024-02-23T00:51:35.247401+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:18.629203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_clean_scrub_stamp":"2024-02-23T00:50:18.629203+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":488,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.248062+0000","last_change":"2024-02-23T00:50:56.774466+0000","last_active":"2024-02-23T00:51:35.248062+0000","last_peered":"2024-02-23T00:51:35.248062+0000","last_clean":"2024-02-23T00:51:35.248062+0000","last_became_active":"2024-02-23T00:47:28.203555+0000","last_became_peered":"2024-02-23T00:47:28.203555+0000","last_unstale":"2024-02-23T00:51:35.248062+0000","last_undegraded":"2024-02-23T00:51:35.248062+0000","last_fullsized":"2024-02-23T00:51:35.248062+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:56.774382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:56.774382+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"457'33","reported_seq":597,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.246380+0000","last_change":"2024-02-23T00:49:46.364498+0000","last_active":"2024-02-23T00:51:35.246380+0000","last_peered":"2024-02-23T00:51:35.246380+0000","last_clean":"2024-02-23T00:51:35.246380+0000","last_became_active":"2024-02-23T00:39:16.786791+0000","last_became_peered":"2024-02-23T00:39:16.786791+0000","last_unstale":"2024-02-23T00:51:35.246380+0000","last_undegraded":"2024-02-23T00:51:35.246380+0000","last_fullsized":"2024-02-23T00:51:35.246380+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'33","last_scrub_stamp":"2024-02-23T00:49:46.364418+0000","last_deep_scrub":"95'27","last_deep_scrub_stamp":"2024-02-23T00:41:55.919895+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.364418+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":546,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.252832+0000","last_change":"2024-02-23T00:49:27.403374+0000","last_active":"2024-02-23T00:51:35.252832+0000","last_peered":"2024-02-23T00:51:35.252832+0000","last_clean":"2024-02-23T00:51:35.252832+0000","last_became_active":"2024-02-23T00:47:28.208293+0000","last_became_peered":"2024-02-23T00:47:28.208293+0000","last_unstale":"2024-02-23T00:51:35.252832+0000","last_undegraded":"2024-02-23T00:51:35.252832+0000","last_fullsized":"2024-02-23T00:51:35.252832+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:27.403305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:27.403305+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":534,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.247941+0000","last_change":"2024-02-23T00:49:46.612538+0000","last_active":"2024-02-23T00:51:35.247941+0000","last_peered":"2024-02-23T00:51:35.247941+0000","last_clean":"2024-02-23T00:51:35.247941+0000","last_became_active":"2024-02-23T00:39:18.796431+0000","last_became_peered":"2024-02-23T00:39:18.796431+0000","last_unstale":"2024-02-23T00:51:35.247941+0000","last_undegraded":"2024-02-23T00:51:35.247941+0000","last_fullsized":"2024-02-23T00:51:35.247941+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:46.612461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.612461+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":479,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.248140+0000","last_change":"2024-02-23T00:50:26.630228+0000","last_active":"2024-02-23T00:51:35.248140+0000","last_peered":"2024-02-23T00:51:35.248140+0000","last_clean":"2024-02-23T00:51:35.248140+0000","last_became_active":"2024-02-23T00:42:58.832885+0000","last_became_peered":"2024-02-23T00:42:58.832885+0000","last_unstale":"2024-02-23T00:51:35.248140+0000","last_undegraded":"2024-02-23T00:51:35.248140+0000","last_fullsized":"2024-02-23T00:51:35.248140+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:26.630088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:26.630088+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"457'17","reported_seq":566,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.246657+0000","last_change":"2024-02-23T00:50:15.590233+0000","last_active":"2024-02-23T00:51:35.246657+0000","last_peered":"2024-02-23T00:51:35.246657+0000","last_clean":"2024-02-23T00:51:35.246657+0000","last_became_active":"2024-02-23T00:39:16.782762+0000","last_became_peered":"2024-02-23T00:39:16.782762+0000","last_unstale":"2024-02-23T00:51:35.246657+0000","last_undegraded":"2024-02-23T00:51:35.246657+0000","last_fullsized":"2024-02-23T00:51:35.246657+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'17","last_scrub_stamp":"2024-02-23T00:50:15.590096+0000","last_deep_scrub":"82'15","last_deep_scrub_stamp":"2024-02-23T00:41:50.975683+0000","last_clean_scrub_stamp":"2024-02-23T00:50:15.590096+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":21,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":542,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.252559+0000","last_change":"2024-02-23T00:51:08.199826+0000","last_active":"2024-02-23T00:51:35.252559+0000","last_peered":"2024-02-23T00:51:35.252559+0000","last_clean":"2024-02-23T00:51:35.252559+0000","last_became_active":"2024-02-23T00:39:14.783469+0000","last_became_peered":"2024-02-23T00:39:14.783469+0000","last_unstale":"2024-02-23T00:51:35.252559+0000","last_undegraded":"2024-02-23T00:51:35.252559+0000","last_fullsized":"2024-02-23T00:51:35.252559+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:08.199749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:08.199749+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":540,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.252597+0000","last_change":"2024-02-23T00:51:13.264970+0000","last_active":"2024-02-23T00:51:35.252597+0000","last_peered":"2024-02-23T00:51:35.252597+0000","last_clean":"2024-02-23T00:51:35.252597+0000","last_became_active":"2024-02-23T00:39:18.796952+0000","last_became_peered":"2024-02-23T00:39:18.796952+0000","last_unstale":"2024-02-23T00:51:35.252597+0000","last_undegraded":"2024-02-23T00:51:35.252597+0000","last_fullsized":"2024-02-23T00:51:35.252597+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:13.264868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:06.491793+0000","last_clean_scrub_stamp":"2024-02-23T00:51:13.264868+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":480,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.249230+0000","last_change":"2024-02-23T00:51:03.460376+0000","last_active":"2024-02-23T00:51:35.249230+0000","last_peered":"2024-02-23T00:51:35.249230+0000","last_clean":"2024-02-23T00:51:35.249230+0000","last_became_active":"2024-02-23T00:42:58.758147+0000","last_became_peered":"2024-02-23T00:42:58.758147+0000","last_unstale":"2024-02-23T00:51:35.249230+0000","last_undegraded":"2024-02-23T00:51:35.249230+0000","last_fullsized":"2024-02-23T00:51:35.249230+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:03.460302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:23.224663+0000","last_clean_scrub_stamp":"2024-02-23T00:51:03.460302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"457'21","reported_seq":574,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.246162+0000","last_change":"2024-02-23T00:49:32.434383+0000","last_active":"2024-02-23T00:51:35.246162+0000","last_peered":"2024-02-23T00:51:35.246162+0000","last_clean":"2024-02-23T00:51:35.246162+0000","last_became_active":"2024-02-23T00:39:16.784606+0000","last_became_peered":"2024-02-23T00:39:16.784606+0000","last_unstale":"2024-02-23T00:51:35.246162+0000","last_undegraded":"2024-02-23T00:51:35.246162+0000","last_fullsized":"2024-02-23T00:51:35.246162+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-23T00:49:32.434282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:32.434282+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":541,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.247968+0000","last_change":"2024-02-23T00:50:19.967076+0000","last_active":"2024-02-23T00:51:35.247968+0000","last_peered":"2024-02-23T00:51:35.247968+0000","last_clean":"2024-02-23T00:51:35.247968+0000","last_became_active":"2024-02-23T00:39:14.785362+0000","last_became_peered":"2024-02-23T00:39:14.785362+0000","last_unstale":"2024-02-23T00:51:35.247968+0000","last_undegraded":"2024-02-23T00:51:35.247968+0000","last_fullsized":"2024-02-23T00:51:35.247968+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:19.967009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:19.967009+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":525,"reported_epoch":567,"state":"active+clean","last_fresh":"2024-02-23T00:51:36.253805+0000","last_change":"2024-02-23T00:49:37.537279+0000","last_active":"2024-02-23T00:51:36.253805+0000","last_peered":"2024-02-23T00:51:36.253805+0000","last_clean":"2024-02-23T00:51:36.253805+0000","last_became_active":"2024-02-23T00:39:18.796312+0000","last_became_peered":"2024-02-23T00:39:18.796312+0000","last_unstale":"2024-02-23T00:51:36.253805+0000","last_undegraded":"2024-02-23T00:51:36.253805+0000","last_fullsized":"2024-02-23T00:51:36.253805+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:37.537211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:47.226331+0000","last_clean_scrub_stamp":"2024-02-23T00:49:37.537211+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":479,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.251333+0000","last_change":"2024-02-23T00:50:28.479578+0000","last_active":"2024-02-23T00:51:35.251333+0000","last_peered":"2024-02-23T00:51:35.251333+0000","last_clean":"2024-02-23T00:51:35.251333+0000","last_became_active":"2024-02-23T00:42:58.838257+0000","last_became_peered":"2024-02-23T00:42:58.838257+0000","last_unstale":"2024-02-23T00:51:35.251333+0000","last_undegraded":"2024-02-23T00:51:35.251333+0000","last_fullsized":"2024-02-23T00:51:35.251333+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:28.479512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:50:28.479512+0000","last_clean_scrub_stamp":"2024-02-23T00:50:28.479512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"457'62","reported_seq":686,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.246953+0000","last_change":"2024-02-23T00:50:58.151569+0000","last_active":"2024-02-23T00:51:35.246953+0000","last_peered":"2024-02-23T00:51:35.246953+0000","last_clean":"2024-02-23T00:51:35.246953+0000","last_became_active":"2024-02-23T00:47:28.205150+0000","last_became_peered":"2024-02-23T00:47:28.205150+0000","last_unstale":"2024-02-23T00:51:35.246953+0000","last_undegraded":"2024-02-23T00:51:35.246953+0000","last_fullsized":"2024-02-23T00:51:35.246953+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'62","last_scrub_stamp":"2024-02-23T00:50:58.151494+0000","last_deep_scrub":"88'50","last_deep_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_clean_scrub_stamp":"2024-02-23T00:50:58.151494+0000","log_size":62,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":92,"num_read_kb":79,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":538,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.251331+0000","last_change":"2024-02-23T00:50:34.600693+0000","last_active":"2024-02-23T00:51:35.251331+0000","last_peered":"2024-02-23T00:51:35.251331+0000","last_clean":"2024-02-23T00:51:35.251331+0000","last_became_active":"2024-02-23T00:39:14.782000+0000","last_became_peered":"2024-02-23T00:39:14.782000+0000","last_unstale":"2024-02-23T00:51:35.251331+0000","last_undegraded":"2024-02-23T00:51:35.251331+0000","last_fullsized":"2024-02-23T00:51:35.251331+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:34.600592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_clean_scrub_stamp":"2024-02-23T00:50:34.600592+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":527,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.246336+0000","last_change":"2024-02-23T00:49:27.459981+0000","last_active":"2024-02-23T00:51:35.246336+0000","last_peered":"2024-02-23T00:51:35.246336+0000","last_clean":"2024-02-23T00:51:35.246336+0000","last_became_active":"2024-02-23T00:39:18.799723+0000","last_became_peered":"2024-02-23T00:39:18.799723+0000","last_unstale":"2024-02-23T00:51:35.246336+0000","last_undegraded":"2024-02-23T00:51:35.246336+0000","last_fullsized":"2024-02-23T00:51:35.246336+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:27.459893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:27.459893+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":479,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-23T00:51:35.248339+0000","last_change":"2024-02-23T00:49:44.601162+0000","last_active":"2024-02-23T00:51:35.248339+0000","last_peered":"2024-02-23T00:51:35.248339+0000","last_clean":"2024-02-23T00:51:35.248339+0000","last_became_active":"2024-02-23T00:42:58.824573+0000","last_became_peered":"2024-02-23T00:42:58.824573+0000","last_unstale":"2024-02-23T00:51:35.248339+0000","last_undegraded":"2024-02-23T00:51:35.248339+0000","last_fullsized":"2024-02-23T00:51:35.248339+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:44.601086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:44.601086+0000","last_clean_scrub_stamp":"2024-02-23T00:49:44.601086+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0],"acting":[5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":231,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":9,"acting":9,"num_store_stats":1},{"poolid":230,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":7},{"poolid":229,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":24,"acting":24,"num_store_stats":6},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":12,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":36,"acting":36,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1508,"num_read_kb":1233,"num_write":895,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":1486,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":955,"ondisk_log_size":955,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":732,"num_read_kb":727,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234136,"num_pgs":76,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":315920,"kb_used_data":2312,"kb_used_omap":0,"kb_used_meta":313600,"kb_avail":93429232,"statfs":{"total":95995035648,"available":95671533568,"internally_reserved":0,"allocated":2367488,"data_stored":935327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321126400},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.50600000000000001,"15min":0.53200000000000003},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":4.9420000000000002},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51500000000000001,"15min":0.52000000000000002},"min":{"1min":0.34699999999999998,"5min":0.32000000000000001,"15min":0.24299999999999999},"max":{"1min":0.82599999999999996,"5min":1.151,"15min":4.9729999999999999},"last":0.55400000000000005}]},{"osd":1,"last update":"Fri Feb 23 00:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.50800000000000001,"15min":0.61199999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.626,"5min":1.25,"15min":10.786},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52400000000000002,"15min":0.61899999999999999},"min":{"1min":0.38600000000000001,"5min":0.33900000000000002,"15min":0.28100000000000003},"max":{"1min":0.79300000000000004,"5min":1.107,"15min":11.885999999999999},"last":0.52400000000000002}]},{"osd":2,"last update":"Fri Feb 23 00:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.51700000000000002,"15min":0.55000000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.28999999999999998},"max":{"1min":0.64900000000000002,"5min":0.80600000000000005,"15min":5.0789999999999997},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52800000000000002,"15min":0.54500000000000004},"min":{"1min":0.38900000000000001,"5min":0.34200000000000003,"15min":0.316},"max":{"1min":0.72299999999999998,"5min":1.1180000000000001,"15min":5.1280000000000001},"last":0.439}]},{"osd":3,"last update":"Fri Feb 23 00:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.52200000000000002,"15min":0.61799999999999999},"min":{"1min":0.33200000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.74399999999999999,"5min":1.095,"15min":15.712999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54700000000000004,"15min":0.61399999999999999},"min":{"1min":0.35799999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.879,"5min":0.879,"15min":9.4090000000000007},"last":0.39200000000000002}]},{"osd":4,"last update":"Fri Feb 23 00:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.439,"15min":0.439},"min":{"1min":0.30199999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.753,"5min":1.6819999999999999,"15min":2.0350000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.45500000000000002,"15min":0.48899999999999999},"min":{"1min":0.33600000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.68400000000000005,"5min":0.86699999999999999,"15min":11.236000000000001},"last":0.33600000000000002}]},{"osd":5,"last update":"Fri Feb 23 00:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47299999999999998,"15min":0.49199999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27000000000000002},"max":{"1min":0.625,"5min":0.73199999999999998,"15min":5.4000000000000004},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.50700000000000001,"15min":0.5},"min":{"1min":0.35499999999999998,"5min":0.29099999999999998,"15min":0.26700000000000002},"max":{"1min":0.67500000000000004,"5min":1.728,"15min":1.728},"last":0.44500000000000001}]},{"osd":6,"last update":"Fri Feb 23 00:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51200000000000001,"15min":0.51100000000000001},"min":{"1min":0.42099999999999999,"5min":0.32200000000000001,"15min":0.317},"max":{"1min":0.75700000000000001,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.50900000000000001,"15min":0.54900000000000004},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.626,"5min":0.70499999999999996,"15min":13.395},"last":0.36799999999999999}]}]},{"osd":6,"up_from":47,"seq":201863463071,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":307736,"kb_used_data":2320,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93437416,"statfs":{"total":95995035648,"available":95679913984,"internally_reserved":0,"allocated":2375680,"data_stored":935519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:51:07 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.505,"15min":0.68200000000000005},"min":{"1min":0.22500000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":30.641999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.53600000000000003,"15min":0.69699999999999995},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.577,"5min":3.7360000000000002,"15min":29.292000000000002},"last":0.41499999999999998}]},{"osd":1,"last update":"Fri Feb 23 00:51:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51500000000000001,"15min":0.68300000000000005},"min":{"1min":0.308,"5min":0.247,"15min":0.247},"max":{"1min":0.77100000000000002,"5min":0.77200000000000002,"15min":28.186},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.52900000000000003,"15min":0.86299999999999999},"min":{"1min":0.39400000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.752,"5min":1.083,"15min":33.104999999999997},"last":0.48599999999999999}]},{"osd":2,"last update":"Fri Feb 23 00:51:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52700000000000002,"15min":0.71399999999999997},"min":{"1min":0.36699999999999999,"5min":0.29399999999999998,"15min":0.27400000000000002},"max":{"1min":0.69599999999999995,"5min":1.1160000000000001,"15min":19.960000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53400000000000003,"15min":0.64600000000000002},"min":{"1min":0.40400000000000003,"5min":0.316,"15min":0.29899999999999999},"max":{"1min":0.79000000000000004,"5min":1.665,"15min":8.5860000000000003},"last":0.42499999999999999}]},{"osd":3,"last update":"Fri Feb 23 00:51:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54300000000000004,"15min":0.626},"min":{"1min":0.40999999999999998,"5min":0.29199999999999998,"15min":0.28599999999999998},"max":{"1min":0.78000000000000003,"5min":0.78400000000000003,"15min":19.562000000000001},"last":0.497},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56899999999999995,"15min":0.67000000000000004},"min":{"1min":0.42399999999999999,"5min":0.33500000000000002,"15min":0.33000000000000002},"max":{"1min":0.80700000000000005,"5min":2.6259999999999999,"15min":14.199},"last":0.38300000000000001}]},{"osd":4,"last update":"Fri Feb 23 00:51:07 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.44700000000000001,"15min":0.47999999999999998},"min":{"1min":0.317,"5min":0.245,"15min":0.218},"max":{"1min":0.61499999999999999,"5min":1.6220000000000001,"15min":11.109999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.45900000000000002,"15min":0.54800000000000004},"min":{"1min":0.32700000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.751,"5min":0.83099999999999996,"15min":16.734000000000002},"last":0.34200000000000003}]},{"osd":5,"last update":"Fri Feb 23 00:51:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48299999999999998,"15min":0.51300000000000001},"min":{"1min":0.34499999999999997,"5min":0.308,"15min":0.249},"max":{"1min":0.72399999999999998,"5min":1.784,"15min":14.154999999999999},"last":0.371},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.58499999999999996,"15min":0.55500000000000005},"min":{"1min":0.37,"5min":0.318,"15min":0.26400000000000001},"max":{"1min":0.68500000000000005,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.39500000000000002}]},{"osd":7,"last update":"Fri Feb 23 00:51:23 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.51000000000000001,"15min":0.53100000000000003},"min":{"1min":0.38600000000000001,"5min":0.29399999999999998,"15min":0.26800000000000002},"max":{"1min":0.63100000000000001,"5min":1.726,"15min":8.548},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53900000000000003,"15min":0.60699999999999998},"min":{"1min":0.40300000000000002,"5min":0.34200000000000003,"15min":0.309},"max":{"1min":0.71199999999999997,"5min":0.88500000000000001,"15min":15.618},"last":0.46600000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575033,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":307768,"kb_used_data":2352,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93437384,"statfs":{"total":95995035648,"available":95679881216,"internally_reserved":0,"allocated":2408448,"data_stored":947298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.39000000000000001,"15min":0.68400000000000005},"min":{"1min":0.23799999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.56999999999999995,"5min":6.202,"15min":78.361000000000004},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.379,"5min":0.36899999999999999,"15min":0.44400000000000001},"min":{"1min":0.26900000000000002,"5min":0.216,"15min":0.191},"max":{"1min":0.64300000000000002,"5min":0.69999999999999996,"15min":5.8849999999999998},"last":0.34100000000000003}]},{"osd":2,"last update":"Fri Feb 23 00:51:13 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.436,"15min":0.67100000000000004},"min":{"1min":0.24399999999999999,"5min":0.20499999999999999,"15min":0.188},"max":{"1min":2.3010000000000002,"5min":7.5789999999999997,"15min":48.606000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.40799999999999997,"15min":0.57599999999999996},"min":{"1min":0.29399999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.63500000000000001,"5min":1,"15min":23.911000000000001},"last":0.52000000000000002}]},{"osd":3,"last update":"Fri Feb 23 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.40500000000000003,"15min":0.59499999999999997},"min":{"1min":0.27800000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":35.244999999999997},"last":0.625},{"interface":"front","average":{"1min":0.437,"5min":0.47699999999999998,"15min":0.68700000000000006},"min":{"1min":0.28899999999999998,"5min":0.26900000000000002,"15min":0.248},"max":{"1min":0.73999999999999999,"5min":7.641,"15min":56.853999999999999},"last":0.29999999999999999}]},{"osd":4,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.63300000000000001,"15min":0.82299999999999995},"min":{"1min":0.40100000000000002,"5min":0.32700000000000001,"15min":0.29299999999999998},"max":{"1min":0.97999999999999998,"5min":7.6829999999999998,"15min":37.646999999999998},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.64700000000000002,"15min":0.70799999999999996},"min":{"1min":0.48099999999999998,"5min":0.35899999999999999,"15min":0.31900000000000001},"max":{"1min":0.91400000000000003,"5min":6.0519999999999996,"15min":9.6859999999999999},"last":0.45200000000000001}]},{"osd":5,"last update":"Fri Feb 23 00:51:35 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.65000000000000002,"15min":0.83299999999999996},"min":{"1min":0.35399999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.81999999999999995,"5min":7.742,"15min":48.792000000000002},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.625,"15min":0.71199999999999997},"min":{"1min":0.39100000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77900000000000003,"5min":0.92600000000000005,"15min":11.872999999999999},"last":0.71499999999999997}]},{"osd":6,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.64100000000000001,"15min":0.82199999999999995},"min":{"1min":0.40000000000000002,"5min":0.37,"15min":0.318},"max":{"1min":0.94199999999999995,"5min":7.7130000000000001,"15min":40.741},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60399999999999998,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.38400000000000001,"15min":0.34000000000000002},"max":{"1min":0.82199999999999995,"5min":0.85999999999999999,"15min":6.0069999999999997},"last":0.753}]},{"osd":7,"last update":"Fri Feb 23 00:51:35 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61899999999999999,"15min":0.71399999999999997},"min":{"1min":0.41799999999999998,"5min":0.40200000000000002,"15min":0.36699999999999999},"max":{"1min":0.83899999999999997,"5min":0.92900000000000005,"15min":20.225000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.66500000000000004,"15min":0.73899999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.95499999999999996,"5min":6.0949999999999998,"15min":20.254999999999999},"last":0.48399999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738559,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":311860,"kb_used_data":2348,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93433292,"statfs":{"total":95995035648,"available":95675691008,"internally_reserved":0,"allocated":2404352,"data_stored":946704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 23 00:50:52 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.35199999999999998,"15min":0.60299999999999998},"min":{"1min":0.224,"5min":0.20999999999999999,"15min":0.19500000000000001},"max":{"1min":0.68200000000000005,"5min":0.81399999999999995,"15min":28.856999999999999},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.373,"5min":0.39500000000000002,"15min":0.629},"min":{"1min":0.249,"5min":0.23300000000000001,"15min":0.20799999999999999},"max":{"1min":0.59099999999999997,"5min":2.3650000000000002,"15min":20.577000000000002},"last":0.38100000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:51:18 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.40400000000000003,"15min":0.60999999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.20499999999999999},"max":{"1min":0.61099999999999999,"5min":2.3889999999999998,"15min":13.863},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.35099999999999998,"15min":0.53800000000000003},"min":{"1min":0.252,"5min":0.223,"15min":0.19800000000000001},"max":{"1min":0.504,"5min":0.65000000000000002,"15min":10.455},"last":0.26500000000000001}]},{"osd":3,"last update":"Fri Feb 23 00:50:46 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.377,"15min":0.77900000000000003},"min":{"1min":0.246,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.64300000000000002,"5min":0.73099999999999998,"15min":35.566000000000003},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.438,"15min":0.76900000000000002},"min":{"1min":0.27700000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.66700000000000004,"5min":2.4049999999999998,"15min":20.702999999999999},"last":0.313}]},{"osd":4,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55900000000000005,"15min":0.752},"min":{"1min":0.40899999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.755,"5min":0.94699999999999995,"15min":10.584},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.61599999999999999,"15min":0.78100000000000003},"min":{"1min":0.42899999999999999,"5min":0.38100000000000001,"15min":0.35099999999999998},"max":{"1min":0.78900000000000003,"5min":2.5019999999999998,"15min":13.198},"last":0.49099999999999999}]},{"osd":5,"last update":"Fri Feb 23 00:51:37 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54500000000000004,"15min":0.71499999999999997},"min":{"1min":0.36699999999999999,"5min":0.33800000000000002,"15min":0.32200000000000001},"max":{"1min":0.72599999999999998,"5min":0.90800000000000003,"15min":10.397},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.60699999999999998,"15min":0.78100000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76800000000000002,"5min":0.92600000000000005,"15min":17.297000000000001},"last":0.51100000000000001}]},{"osd":6,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.58899999999999997,"15min":0.78000000000000003},"min":{"1min":0.39400000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.76400000000000001,"5min":0.94399999999999995,"15min":17.373999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.61299999999999999,"15min":0.76200000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.36799999999999999},"max":{"1min":0.72199999999999998,"5min":2.556,"15min":13.973000000000001},"last":0.57899999999999996}]},{"osd":7,"last update":"Fri Feb 23 00:51:27 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.59099999999999997,"15min":0.77800000000000002},"min":{"1min":0.379,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83399999999999996,"5min":0.96099999999999997,"15min":17.399999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.63100000000000001,"15min":0.78100000000000003},"min":{"1min":0.46000000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.752,"5min":2.5289999999999999,"15min":16.427},"last":0.53500000000000003}]}]},{"osd":2,"up_from":0,"seq":77309411508,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":26,"seq":111669149869,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":311868,"kb_used_data":2356,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93433284,"statfs":{"total":95995035648,"available":95675682816,"internally_reserved":0,"allocated":2412544,"data_stored":946082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:50:45 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.441,"15min":0.53800000000000003},"min":{"1min":0.246,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":8.3300000000000001,"5min":8.3300000000000001,"15min":12.212999999999999},"last":0.443},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.499,"15min":0.61099999999999999},"min":{"1min":0.23100000000000001,"5min":0.22800000000000001,"15min":0.21199999999999999},"max":{"1min":2.9740000000000002,"5min":5.9800000000000004,"15min":11.714},"last":0.46000000000000002}]},{"osd":1,"last update":"Fri Feb 23 00:50:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55700000000000005,"15min":0.63200000000000001},"min":{"1min":0.27200000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":3.056,"5min":6.851,"15min":11.742000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.58099999999999996,"15min":0.60199999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.19900000000000001},"max":{"1min":2.7709999999999999,"5min":8.2270000000000003,"15min":8.2270000000000003},"last":0.47399999999999998}]},{"osd":2,"last update":"Fri Feb 23 00:50:45 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.625,"15min":0.68999999999999995},"min":{"1min":0.29199999999999998,"5min":0.255,"15min":0.255},"max":{"1min":3.242,"5min":8.0269999999999992,"15min":13.359999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.52600000000000002,"15min":0.66600000000000004},"min":{"1min":0.27600000000000002,"5min":0.27500000000000002,"15min":0.26100000000000001},"max":{"1min":8.5820000000000007,"5min":8.5820000000000007,"15min":21.122},"last":0.50900000000000001}]},{"osd":4,"last update":"Fri Feb 23 00:51:10 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.58899999999999997,"15min":0.755},"min":{"1min":0.40300000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":3.1419999999999999,"5min":3.1419999999999999,"15min":11.644},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.74199999999999999,"15min":0.753},"min":{"1min":0.36899999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":2.9020000000000001,"5min":12.430999999999999,"15min":12.430999999999999},"last":0.70999999999999996}]},{"osd":5,"last update":"Fri Feb 23 00:50:35 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":0.69899999999999995,"15min":0.68899999999999995},"min":{"1min":0.375,"5min":0.37,"15min":0.35099999999999998},"max":{"1min":10.231999999999999,"5min":10.231999999999999,"15min":10.231999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.746,"5min":0.76200000000000001,"15min":0.80100000000000005},"min":{"1min":0.38300000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":3.1829999999999998,"5min":11.648,"15min":11.648},"last":0.80900000000000005}]},{"osd":6,"last update":"Fri Feb 23 00:50:53 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76300000000000001,"15min":0.75600000000000001},"min":{"1min":0.42399999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.9510000000000001,"5min":6.9420000000000002,"15min":6.9420000000000002},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.2050000000000001,"5min":0.751,"15min":0.72299999999999998},"min":{"1min":0.39200000000000002,"5min":0.376,"15min":0.376},"max":{"1min":10.965999999999999,"5min":10.965999999999999,"15min":10.965999999999999},"last":0.82299999999999995}]},{"osd":7,"last update":"Fri Feb 23 00:51:34 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.66000000000000003,"15min":0.83299999999999996},"min":{"1min":0.441,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.1030000000000002,"5min":3.1030000000000002,"15min":11.676},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.66600000000000004,"15min":0.81000000000000005},"min":{"1min":0.501,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":2.8580000000000001,"5min":2.8580000000000001,"15min":7.9800000000000004},"last":0.878}]}]},{"osd":4,"up_from":32,"seq":137438953641,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":307756,"kb_used_data":2340,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93437396,"statfs":{"total":95995035648,"available":95679893504,"internally_reserved":0,"allocated":2396160,"data_stored":945726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.504,"15min":0.52600000000000002},"min":{"1min":0.32200000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.65300000000000002,"5min":1.3819999999999999,"15min":3.5219999999999998},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.5,"5min":0.52400000000000002,"15min":0.58499999999999996},"min":{"1min":0.373,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.72799999999999998,"5min":1.8,"15min":8.6140000000000008},"last":0.46400000000000002}]},{"osd":1,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51600000000000001,"15min":0.57999999999999996},"min":{"1min":0.31900000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.70699999999999996,"5min":2.6600000000000001,"15min":9.6110000000000007},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.53500000000000003,"15min":0.58499999999999996},"min":{"1min":0.31900000000000001,"5min":0.26300000000000001,"15min":0.255},"max":{"1min":0.70099999999999996,"5min":2.6920000000000002,"15min":6.6950000000000003},"last":0.34799999999999998}]},{"osd":2,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.51900000000000002,"15min":0.56999999999999995},"min":{"1min":0.34699999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.65700000000000003,"5min":1.0960000000000001,"15min":3.3069999999999999},"last":0.374},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54000000000000004,"15min":0.57099999999999995},"min":{"1min":0.33700000000000002,"5min":0.33600000000000002,"15min":0.32500000000000001},"max":{"1min":0.68700000000000006,"5min":1.0189999999999999,"15min":5.8440000000000003},"last":0.49199999999999999}]},{"osd":3,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.67300000000000004,"15min":0.63400000000000001},"min":{"1min":0.38500000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.71999999999999997,"5min":9.1280000000000001,"15min":9.1280000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.58199999999999996,"15min":0.60399999999999998},"min":{"1min":0.39200000000000002,"5min":0.34599999999999997,"15min":0.29699999999999999},"max":{"1min":0.73299999999999998,"5min":4.0220000000000002,"15min":4.1909999999999998},"last":0.45500000000000002}]},{"osd":5,"last update":"Fri Feb 23 00:50:35 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.45600000000000002,"15min":0.41599999999999998},"min":{"1min":0.26700000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.51500000000000001,"15min":0.47199999999999998},"min":{"1min":0.35399999999999998,"5min":0.29499999999999998,"15min":0.28899999999999998},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.47599999999999998}]},{"osd":6,"last update":"Fri Feb 23 00:51:05 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.44900000000000001,"15min":0.435},"min":{"1min":0.34000000000000002,"5min":0.28399999999999997,"15min":0.253},"max":{"1min":0.67200000000000004,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47199999999999998,"15min":0.443},"min":{"1min":0.34699999999999998,"5min":0.28799999999999998,"15min":0.215},"max":{"1min":0.64100000000000001,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":0.33600000000000002}]},{"osd":7,"last update":"Fri Feb 23 00:51:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.49399999999999999,"15min":0.48699999999999999},"min":{"1min":0.33100000000000002,"5min":0.27600000000000002,"15min":0.254},"max":{"1min":0.70999999999999996,"5min":2.294,"15min":2.294},"last":0.434},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56899999999999995,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.752,"5min":2.282,"15min":2.282},"last":0.50800000000000001}]}]},{"osd":5,"up_from":39,"seq":167503724708,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":307764,"kb_used_data":2348,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93437388,"statfs":{"total":95995035648,"available":95679885312,"internally_reserved":0,"allocated":2404352,"data_stored":946978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:51:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.55300000000000005,"15min":0.61399999999999999},"min":{"1min":0.40400000000000003,"5min":0.253,"15min":0.246},"max":{"1min":0.67000000000000004,"5min":5.2220000000000004,"15min":23.881},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.57599999999999996,"15min":0.59399999999999997},"min":{"1min":0.38600000000000001,"5min":0.29099999999999998,"15min":0.27000000000000002},"max":{"1min":0.76500000000000001,"5min":3.9249999999999998,"15min":5.4649999999999999},"last":0.46999999999999997}]},{"osd":1,"last update":"Fri Feb 23 00:51:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54900000000000004,"15min":0.65600000000000003},"min":{"1min":0.35299999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.73899999999999999,"5min":3.9460000000000002,"15min":18.175999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.56899999999999995,"15min":0.628},"min":{"1min":0.40100000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.71599999999999997,"5min":3.976,"15min":9.0950000000000006},"last":0.42299999999999999}]},{"osd":2,"last update":"Fri Feb 23 00:51:32 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57899999999999996,"15min":0.71199999999999997},"min":{"1min":0.37,"5min":0.255,"15min":0.255},"max":{"1min":1.036,"5min":4.5899999999999999,"15min":22.282},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.58599999999999997,"15min":0.72599999999999998},"min":{"1min":0.34399999999999997,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.66900000000000004,"5min":3.8540000000000001,"15min":23.925000000000001},"last":0.48599999999999999}]},{"osd":3,"last update":"Fri Feb 23 00:51:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55800000000000005,"15min":0.621},"min":{"1min":0.35899999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.194,"5min":1.23,"15min":6.5730000000000004},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.60199999999999998,"15min":0.65500000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.33500000000000002},"max":{"1min":0.70499999999999996,"5min":3.9980000000000002,"15min":8.1760000000000002},"last":0.626}]},{"osd":4,"last update":"Fri Feb 23 00:51:32 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.48199999999999998,"15min":0.436},"min":{"1min":0.33000000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.60699999999999998,"5min":4.593,"15min":4.593},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.49199999999999999,"15min":0.47399999999999998},"min":{"1min":0.35799999999999998,"5min":0.27900000000000003,"15min":0.27300000000000002},"max":{"1min":1.169,"5min":1.5860000000000001,"15min":3.879},"last":0.61499999999999999}]},{"osd":6,"last update":"Fri Feb 23 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51300000000000001,"15min":0.47599999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.27900000000000003},"max":{"1min":0.71099999999999997,"5min":4.6740000000000004,"15min":4.6740000000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.49099999999999999,"15min":0.47699999999999998},"min":{"1min":0.35999999999999999,"5min":0.28000000000000003,"15min":0.27100000000000002},"max":{"1min":0.73199999999999998,"5min":1.159,"15min":1.159},"last":0.57599999999999996}]},{"osd":7,"last update":"Fri Feb 23 00:51:26 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.39000000000000001,"5min":0.32500000000000001,"15min":0.30399999999999999},"max":{"1min":0.74099999999999999,"5min":1.302,"15min":7.0300000000000002},"last":1.1870000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52600000000000002,"15min":0.53900000000000003},"min":{"1min":0.36099999999999999,"5min":0.314,"15min":0.29099999999999998},"max":{"1min":0.64600000000000002,"5min":2.016,"15min":9.0579999999999998},"last":0.56399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":229,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":229,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":229,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":229,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":229,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":229,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":230,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":230,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":230,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":230,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":230,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":230,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":230,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":231,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-23T00:51:41.114 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph status --format=json 2024-02-23T00:51:41.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.309+0000 7ffaed0ba700 1 -- 172.21.15.47:0/3039788591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffae80fdbb0 msgr2=0x7ffae80fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:41.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.309+0000 7ffaed0ba700 1 --2- 172.21.15.47:0/3039788591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffae80fdbb0 0x7ffae80fdf90 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 rx=0x7ffad80047b0 tx=0x7ffad8009cb0).stop 2024-02-23T00:51:41.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.309+0000 7ffaed0ba700 1 -- 172.21.15.47:0/3039788591 shutdown_connections 2024-02-23T00:51:41.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.309+0000 7ffaed0ba700 1 --2- 172.21.15.47:0/3039788591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffae80f8420 0x7ffae8104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:41.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.309+0000 7ffaed0ba700 1 --2- 172.21.15.47:0/3039788591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffae80fdbb0 0x7ffae80fdf90 unknown :-1 s=CLOSED pgs=2580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:41.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.309+0000 7ffaed0ba700 1 --2- 172.21.15.47:0/3039788591 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffae80fe510 0x7ffae80f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:41.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.309+0000 7ffaed0ba700 1 -- 172.21.15.47:0/3039788591 >> 172.21.15.47:0/3039788591 conn(0x7ffae80f3800 msgr2=0x7ffae80f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:41.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.309+0000 7ffaed0ba700 1 -- 172.21.15.47:0/3039788591 shutdown_connections 2024-02-23T00:51:41.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.309+0000 7ffaed0ba700 1 -- 172.21.15.47:0/3039788591 wait complete. 2024-02-23T00:51:41.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.309+0000 7ffaed0ba700 1 librados: starting msgr at 2024-02-23T00:51:41.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.309+0000 7ffaed0ba700 1 librados: starting objecter 2024-02-23T00:51:41.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.310+0000 7ffaed0ba700 1 Processor -- start 2024-02-23T00:51:41.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.310+0000 7ffaed0ba700 1 -- start start 2024-02-23T00:51:41.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.310+0000 7ffaed0ba700 1 librados: setting wanted keys 2024-02-23T00:51:41.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.310+0000 7ffaed0ba700 1 librados: calling monclient init 2024-02-23T00:51:41.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.310+0000 7ffaed0ba700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffae80f8420 0x7ffae810df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:41.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.310+0000 7ffaed0ba700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffae80fdbb0 0x7ffae810e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:41.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.310+0000 7ffaed0ba700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffae80fe510 0x7ffae8108120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:41.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.310+0000 7ffaed0ba700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae8108610 con 0x7ffae80f8420 2024-02-23T00:51:41.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.310+0000 7ffaed0ba700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae8108780 con 0x7ffae80fe510 2024-02-23T00:51:41.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.310+0000 7ffaed0ba700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae8108940 con 0x7ffae80fdbb0 2024-02-23T00:51:41.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.310+0000 7ffae659c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffae80fdbb0 0x7ffae810e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:41.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.310+0000 7ffae659c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffae80fdbb0 0x7ffae810e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:46780/0 (socket says 172.21.15.47:46780) 2024-02-23T00:51:41.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.310+0000 7ffae659c700 1 -- 172.21.15.47:0/373037557 learned_addr learned my addr 172.21.15.47:0/373037557 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:51:41.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.310+0000 7ffae759e700 1 --2- 172.21.15.47:0/373037557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffae80fe510 0x7ffae8108120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:41.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.311+0000 7ffae6d9d700 1 --2- 172.21.15.47:0/373037557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffae80f8420 0x7ffae810df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:41.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.311+0000 7ffae659c700 1 -- 172.21.15.47:0/373037557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffae80fe510 msgr2=0x7ffae8108120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:41.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.311+0000 7ffae659c700 1 --2- 172.21.15.47:0/373037557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffae80fe510 0x7ffae8108120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:41.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.311+0000 7ffae659c700 1 -- 172.21.15.47:0/373037557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffae80f8420 msgr2=0x7ffae810df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:41.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.311+0000 7ffae659c700 1 --2- 172.21.15.47:0/373037557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffae80f8420 0x7ffae810df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:41.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.311+0000 7ffae659c700 1 -- 172.21.15.47:0/373037557 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad8005040 con 0x7ffae80fdbb0 2024-02-23T00:51:41.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.311+0000 7ffae6d9d700 1 --2- 172.21.15.47:0/373037557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffae80f8420 0x7ffae810df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:51:41.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.311+0000 7ffae659c700 1 --2- 172.21.15.47:0/373037557 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffae80fdbb0 0x7ffae810e470 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 rx=0x7ffadc00e970 tx=0x7ffadc00b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:41.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.311+0000 7ffacffff700 10 client.?.objecter ms_handle_connect 0x7ffae80fdbb0 2024-02-23T00:51:41.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.311+0000 7ffacffff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:51:41.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.311+0000 7ffacffff700 1 -- 172.21.15.47:0/373037557 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffadc0046e0 con 0x7ffae80fdbb0 2024-02-23T00:51:41.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.311+0000 7ffacffff700 1 -- 172.21.15.47:0/373037557 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7ffadc00f040 con 0x7ffae80fdbb0 2024-02-23T00:51:41.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.311+0000 7ffaed0ba700 1 -- 172.21.15.47:0/373037557 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffae8108c20 con 0x7ffae80fdbb0 2024-02-23T00:51:41.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.311+0000 7ffacffff700 1 -- 172.21.15.47:0/373037557 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffadc00c5b0 con 0x7ffae80fdbb0 2024-02-23T00:51:41.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.311+0000 7ffaed0ba700 10 client.41705.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:51:41.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.311+0000 7ffaed0ba700 1 -- 172.21.15.47:0/373037557 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae8108fa0 con 0x7ffae80fdbb0 2024-02-23T00:51:41.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.311+0000 7ffaed0ba700 1 librados: init done 2024-02-23T00:51:41.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.312+0000 7ffaed0ba700 1 -- 172.21.15.47:0/373037557 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffae805ee80 con 0x7ffae80fdbb0 2024-02-23T00:51:41.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.313+0000 7ffacffff700 1 -- 172.21.15.47:0/373037557 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7ffadc00c9e0 con 0x7ffae80fdbb0 2024-02-23T00:51:41.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.313+0000 7ffacffff700 1 --2- 172.21.15.47:0/373037557 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ffad0062cc0 0x7ffad0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:41.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.314+0000 7ffacffff700 1 -- 172.21.15.47:0/373037557 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(572..572 src has 482..572) v4 ==== 9985+0+0 (secure 0 0 0) 0x7ffadc07d470 con 0x7ffae80fdbb0 2024-02-23T00:51:41.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.314+0000 7ffacffff700 10 client.41705.objecter ms_dispatch 0x7ffae8001550 osd_map(572..572 src has 482..572) v4 2024-02-23T00:51:41.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.314+0000 7ffacffff700 3 client.41705.objecter handle_osd_map got epochs [572,572] > 0 2024-02-23T00:51:41.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.314+0000 7ffacffff700 3 client.41705.objecter handle_osd_map decoding full epoch 572 2024-02-23T00:51:41.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.314+0000 7ffae6d9d700 1 --2- 172.21.15.47:0/373037557 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ffad0062cc0 0x7ffad0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:41.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.314+0000 7ffacffff700 20 client.41705.objecter dump_active .. 0 homeless 2024-02-23T00:51:41.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.314+0000 7ffacffff700 10 client.41705.objecter ms_handle_connect 0x7ffad0062cc0 2024-02-23T00:51:41.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.314+0000 7ffae6d9d700 1 --2- 172.21.15.47:0/373037557 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ffad0062cc0 0x7ffad0065180 secure :-1 s=READY pgs=3127 cs=0 l=1 rev1=1 rx=0x7ffad8004e20 tx=0x7ffad8009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:41.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.318+0000 7ffacffff700 1 -- 172.21.15.47:0/373037557 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ffadc04c8b0 con 0x7ffae80fdbb0 2024-02-23T00:51:41.359 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:51:41.361 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:40.269004+0000 mon.a (mon.0) 3018 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-97", "overlaypool": "test-rados-api-smithi047-82465-97-cache"}]': finished 2024-02-23T00:51:41.361 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:40.269091+0000 mon.a (mon.0) 3019 : cluster [DBG] osdmap e571: 8 total, 8 up, 7 in 2024-02-23T00:51:41.361 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:40.271232+0000 mon.a (mon.0) 3020 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-97-cache", "mode": "writeback"}]: dispatch 2024-02-23T00:51:41.361 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:40.697780+0000 osd.4 (osd.4) 235 : cluster [DBG] 4.e scrub starts 2024-02-23T00:51:41.362 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:40.699136+0000 osd.4 (osd.4) 236 : cluster [DBG] 4.e scrub ok 2024-02-23T00:51:41.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:40.269004+0000 mon.a (mon.0) 3018 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-97", "overlaypool": "test-rados-api-smithi047-82465-97-cache"}]': finished 2024-02-23T00:51:41.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:40.269091+0000 mon.a (mon.0) 3019 : cluster [DBG] osdmap e571: 8 total, 8 up, 7 in 2024-02-23T00:51:41.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:40.271232+0000 mon.a (mon.0) 3020 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-97-cache", "mode": "writeback"}]: dispatch 2024-02-23T00:51:41.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:40.697780+0000 osd.4 (osd.4) 235 : cluster [DBG] 4.e scrub starts 2024-02-23T00:51:41.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:40.699136+0000 osd.4 (osd.4) 236 : cluster [DBG] 4.e scrub ok 2024-02-23T00:51:41.363 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:41.363 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:51:41.372 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:40.269004+0000 mon.a (mon.0) 3018 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-97", "overlaypool": "test-rados-api-smithi047-82465-97-cache"}]': finished 2024-02-23T00:51:41.372 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:40.269091+0000 mon.a (mon.0) 3019 : cluster [DBG] osdmap e571: 8 total, 8 up, 7 in 2024-02-23T00:51:41.372 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:40.271232+0000 mon.a (mon.0) 3020 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-97-cache", "mode": "writeback"}]: dispatch 2024-02-23T00:51:41.372 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:40.697780+0000 osd.4 (osd.4) 235 : cluster [DBG] 4.e scrub starts 2024-02-23T00:51:41.372 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:40.699136+0000 osd.4 (osd.4) 236 : cluster [DBG] 4.e scrub ok 2024-02-23T00:51:41.464 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:51:41.493 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:41.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:41.505+0000 7ffaed0ba700 1 -- 172.21.15.47:0/373037557 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7ffae805fa60 con 0x7ffae80fdbb0 2024-02-23T00:51:41.592 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:51:41.596 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:41.596 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:51:41.620 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:51:41.621 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-02-23T00:51:41.621 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-02-23T00:51:41.621 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-02-23T00:51:41.621 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:51:41.697 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:51:41.725 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:41.824 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:51:41.826 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:41.827 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:51:41.928 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:51:41.954 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:41.975 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:51:41.978 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:41.979 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:51:42.080 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:51:42.115 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:42.135 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:51:42.138 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:42.138 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:51:42.239 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:51:42.271 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:42.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.284+0000 7ffacffff700 1 -- 172.21.15.47:0/373037557 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v573) v1 ==== 101+0+0 (secure 0 0 0) 0x7ffadc019020 con 0x7ffae80fdbb0 2024-02-23T00:51:42.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.287+0000 7ffaed0ba700 10 librados: watch_flush enter 2024-02-23T00:51:42.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.287+0000 7ffaed0ba700 10 librados: watch_flush exit 2024-02-23T00:51:42.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.287+0000 7ffaed0ba700 20 client.41705.objecter shutdown clearing up homeless session... 2024-02-23T00:51:42.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.287+0000 7ffaed0ba700 10 client.41705.objecter successfully canceled tick 2024-02-23T00:51:42.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.287+0000 7ffaed0ba700 1 -- 172.21.15.47:0/373037557 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ffad0062cc0 msgr2=0x7ffad0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:42.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.287+0000 7ffaed0ba700 1 --2- 172.21.15.47:0/373037557 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ffad0062cc0 0x7ffad0065180 secure :-1 s=READY pgs=3127 cs=0 l=1 rev1=1 rx=0x7ffad8004e20 tx=0x7ffad8009cb0).stop 2024-02-23T00:51:42.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.287+0000 7ffaed0ba700 1 -- 172.21.15.47:0/373037557 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffae80fdbb0 msgr2=0x7ffae810e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:42.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.287+0000 7ffaed0ba700 1 --2- 172.21.15.47:0/373037557 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffae80fdbb0 0x7ffae810e470 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 rx=0x7ffadc00e970 tx=0x7ffadc00b040).stop 2024-02-23T00:51:42.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.289+0000 7ffaed0ba700 1 -- 172.21.15.47:0/373037557 shutdown_connections 2024-02-23T00:51:42.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.289+0000 7ffaed0ba700 1 --2- 172.21.15.47:0/373037557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffae80fe510 0x7ffae8108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:42.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.289+0000 7ffaed0ba700 1 --2- 172.21.15.47:0/373037557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffae80f8420 0x7ffae810df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:42.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.289+0000 7ffaed0ba700 1 --2- 172.21.15.47:0/373037557 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ffad0062cc0 0x7ffad0065180 unknown :-1 s=CLOSED pgs=3127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:42.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.289+0000 7ffaed0ba700 1 --2- 172.21.15.47:0/373037557 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffae80fdbb0 0x7ffae810e470 unknown :-1 s=CLOSED pgs=2738 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:42.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.289+0000 7ffaed0ba700 1 -- 172.21.15.47:0/373037557 >> 172.21.15.47:0/373037557 conn(0x7ffae80f3800 msgr2=0x7ffae80fb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:42.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.289+0000 7ffaed0ba700 1 -- 172.21.15.47:0/373037557 shutdown_connections 2024-02-23T00:51:42.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.289+0000 7ffaed0ba700 1 -- 172.21.15.47:0/373037557 wait complete. 2024-02-23T00:51:42.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.289+0000 7ffaed0ba700 1 librados: shutdown 2024-02-23T00:51:42.292 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is unset 2024-02-23T00:51:42.369 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.6 dump_historic_ops 2024-02-23T00:51:42.388 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:40.290397+0000 osd.3 (osd.3) 270 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:51:42.388 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:40.290577+0000 osd.3 (osd.3) 271 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:51:42.388 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:40.470273+0000 osd.1 (osd.1) 239 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:51:42.388 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:40.470436+0000 osd.1 (osd.1) 240 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:51:42.388 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:40.509621+0000 osd.1 (osd.1) 241 : cluster [DBG] 2.15 scrub starts 2024-02-23T00:51:42.388 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:40.511255+0000 osd.1 (osd.1) 242 : cluster [DBG] 2.15 scrub ok 2024-02-23T00:51:42.388 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:40.868319+0000 osd.0 (osd.0) 239 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:51:42.388 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:40.868535+0000 osd.0 (osd.0) 240 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:51:42.389 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:41.265247+0000 mon.a (mon.0) 3021 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:51:42.389 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:41.269433+0000 mon.a (mon.0) 3022 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-97-cache", "mode": "writeback"}]': finished 2024-02-23T00:51:42.389 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:41.269479+0000 mon.a (mon.0) 3023 : cluster [DBG] osdmap e572: 8 total, 8 up, 7 in 2024-02-23T00:51:42.389 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:41.291272+0000 mon.a (mon.0) 3024 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-97"}]: dispatch 2024-02-23T00:51:42.389 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:41.307496+0000 mon.a (mon.0) 3025 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:51:42.389 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:41.345437+0000 mgr.y (mgr.14517) 517 : cluster [DBG] pgmap v862: 216 pgs: 3 creating+peering, 29 unknown, 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:51:42.389 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:41.506779+0000 mon.c (mon.2) 328 : audit [INF] from='client.? 172.21.15.47:0/373037557' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:51:42.389 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:41.507080+0000 mon.a (mon.0) 3026 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:51:42.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:40.290397+0000 osd.3 (osd.3) 270 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:51:42.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:40.290577+0000 osd.3 (osd.3) 271 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:51:42.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:40.470273+0000 osd.1 (osd.1) 239 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:51:42.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:40.470436+0000 osd.1 (osd.1) 240 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:51:42.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:40.509621+0000 osd.1 (osd.1) 241 : cluster [DBG] 2.15 scrub starts 2024-02-23T00:51:42.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:40.511255+0000 osd.1 (osd.1) 242 : cluster [DBG] 2.15 scrub ok 2024-02-23T00:51:42.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:40.868319+0000 osd.0 (osd.0) 239 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:51:42.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:40.868535+0000 osd.0 (osd.0) 240 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:51:42.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:41.265247+0000 mon.a (mon.0) 3021 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:51:42.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:41.269433+0000 mon.a (mon.0) 3022 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-97-cache", "mode": "writeback"}]': finished 2024-02-23T00:51:42.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:41.269479+0000 mon.a (mon.0) 3023 : cluster [DBG] osdmap e572: 8 total, 8 up, 7 in 2024-02-23T00:51:42.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:41.291272+0000 mon.a (mon.0) 3024 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-97"}]: dispatch 2024-02-23T00:51:42.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:41.307496+0000 mon.a (mon.0) 3025 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:51:42.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:41.345437+0000 mgr.y (mgr.14517) 517 : cluster [DBG] pgmap v862: 216 pgs: 3 creating+peering, 29 unknown, 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:51:42.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:41.506779+0000 mon.c (mon.2) 328 : audit [INF] from='client.? 172.21.15.47:0/373037557' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:51:42.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:41.507080+0000 mon.a (mon.0) 3026 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:51:42.636 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:51:42.694 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:51:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:51:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:51:42.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:40.290397+0000 osd.3 (osd.3) 270 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:51:42.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:40.290577+0000 osd.3 (osd.3) 271 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:51:42.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:40.470273+0000 osd.1 (osd.1) 239 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:51:42.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:40.470436+0000 osd.1 (osd.1) 240 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:51:42.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:40.509621+0000 osd.1 (osd.1) 241 : cluster [DBG] 2.15 scrub starts 2024-02-23T00:51:42.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:40.511255+0000 osd.1 (osd.1) 242 : cluster [DBG] 2.15 scrub ok 2024-02-23T00:51:42.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:40.868319+0000 osd.0 (osd.0) 239 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:51:42.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:40.868535+0000 osd.0 (osd.0) 240 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:51:42.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:41.265247+0000 mon.a (mon.0) 3021 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:51:42.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:41.269433+0000 mon.a (mon.0) 3022 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-97-cache", "mode": "writeback"}]': finished 2024-02-23T00:51:42.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:41.269479+0000 mon.a (mon.0) 3023 : cluster [DBG] osdmap e572: 8 total, 8 up, 7 in 2024-02-23T00:51:42.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:41.291272+0000 mon.a (mon.0) 3024 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-97"}]: dispatch 2024-02-23T00:51:42.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:41.307496+0000 mon.a (mon.0) 3025 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:51:42.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:41.345437+0000 mgr.y (mgr.14517) 517 : cluster [DBG] pgmap v862: 216 pgs: 3 creating+peering, 29 unknown, 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:51:42.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:41.506779+0000 mon.c (mon.2) 328 : audit [INF] from='client.? 172.21.15.47:0/373037557' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:51:42.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:41.507080+0000 mon.a (mon.0) 3026 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:51:42.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.802+0000 7fecbb682700 1 -- 172.21.15.47:0/1746709282 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fecb40f7400 msgr2=0x7fecb4103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:42.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.802+0000 7fecbb682700 1 --2- 172.21.15.47:0/1746709282 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fecb40f7400 0x7fecb4103a20 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 rx=0x7fecac009c30 tx=0x7fecac004c50).stop 2024-02-23T00:51:42.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.802+0000 7fecbb682700 1 -- 172.21.15.47:0/1746709282 shutdown_connections 2024-02-23T00:51:42.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.802+0000 7fecbb682700 1 --2- 172.21.15.47:0/1746709282 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fecb40f7400 0x7fecb4103a20 unknown :-1 s=CLOSED pgs=2739 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:42.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.802+0000 7fecbb682700 1 --2- 172.21.15.47:0/1746709282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecb40fcbb0 0x7fecb40fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:42.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.802+0000 7fecbb682700 1 --2- 172.21.15.47:0/1746709282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecb40fd510 0x7fecb40f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:42.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.802+0000 7fecbb682700 1 -- 172.21.15.47:0/1746709282 >> 172.21.15.47:0/1746709282 conn(0x7fecb40f2820 msgr2=0x7fecb40f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:42.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.803+0000 7fecbb682700 1 -- 172.21.15.47:0/1746709282 shutdown_connections 2024-02-23T00:51:42.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.803+0000 7fecbb682700 1 -- 172.21.15.47:0/1746709282 wait complete. 2024-02-23T00:51:42.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.803+0000 7fecbb682700 1 librados: starting msgr at 2024-02-23T00:51:42.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.803+0000 7fecbb682700 1 librados: starting objecter 2024-02-23T00:51:42.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.803+0000 7fecbb682700 1 Processor -- start 2024-02-23T00:51:42.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.803+0000 7fecbb682700 1 -- start start 2024-02-23T00:51:42.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.803+0000 7fecbb682700 1 librados: setting wanted keys 2024-02-23T00:51:42.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.803+0000 7fecbb682700 1 librados: calling monclient init 2024-02-23T00:51:42.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.803+0000 7fecbb682700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecb40f7400 0x7fecb40f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:42.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.803+0000 7fecbb682700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecb40fcbb0 0x7fecb40f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:42.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.803+0000 7fecbb682700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fecb40fd510 0x7fecb40f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:42.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.803+0000 7fecbb682700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb4116340 con 0x7fecb40f7400 2024-02-23T00:51:42.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.803+0000 7fecbb682700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb41164b0 con 0x7fecb40fcbb0 2024-02-23T00:51:42.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.803+0000 7fecbb682700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb4116670 con 0x7fecb40fd510 2024-02-23T00:51:42.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.804+0000 7fecb941e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecb40f7400 0x7fecb40f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:42.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.804+0000 7fecb9c1f700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fecb40fd510 0x7fecb40f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:42.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.804+0000 7fecb941e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecb40f7400 0x7fecb40f89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40128/0 (socket says 172.21.15.47:40128) 2024-02-23T00:51:42.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.804+0000 7fecb941e700 1 -- 172.21.15.47:0/109650152 learned_addr learned my addr 172.21.15.47:0/109650152 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:51:42.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.804+0000 7fecb8c1d700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecb40fcbb0 0x7fecb40f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:42.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.804+0000 7fecb941e700 1 -- 172.21.15.47:0/109650152 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fecb40fd510 msgr2=0x7fecb40f9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:42.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.804+0000 7fecb941e700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fecb40fd510 0x7fecb40f9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:42.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.804+0000 7fecb941e700 1 -- 172.21.15.47:0/109650152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecb40fcbb0 msgr2=0x7fecb40f8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:42.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.804+0000 7fecb941e700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecb40fcbb0 0x7fecb40f8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:42.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.804+0000 7fecb941e700 1 -- 172.21.15.47:0/109650152 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecac0097c0 con 0x7fecb40f7400 2024-02-23T00:51:42.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.804+0000 7fecb9c1f700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fecb40fd510 0x7fecb40f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:51:42.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.804+0000 7fecb941e700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecb40f7400 0x7fecb40f89b0 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 rx=0x7fecb000e970 tx=0x7fecb000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:42.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.804+0000 7feca67fc700 10 client.?.objecter ms_handle_connect 0x7fecb40f7400 2024-02-23T00:51:42.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.804+0000 7feca67fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:51:42.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.804+0000 7feca67fc700 1 -- 172.21.15.47:0/109650152 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb00046e0 con 0x7fecb40f7400 2024-02-23T00:51:42.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.804+0000 7feca67fc700 1 -- 172.21.15.47:0/109650152 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fecb000f040 con 0x7fecb40f7400 2024-02-23T00:51:42.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.804+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb4116950 con 0x7fecb40f7400 2024-02-23T00:51:42.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.804+0000 7feca67fc700 1 -- 172.21.15.47:0/109650152 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb00146f0 con 0x7fecb40f7400 2024-02-23T00:51:42.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.804+0000 7fecbb682700 10 client.31392.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:51:42.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.804+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb4116e90 con 0x7fecb40f7400 2024-02-23T00:51:42.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.804+0000 7fecbb682700 1 librados: init done 2024-02-23T00:51:42.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.805+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fecb405fa60 con 0x7fecb40f7400 2024-02-23T00:51:42.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.806+0000 7feca67fc700 1 -- 172.21.15.47:0/109650152 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fecb0007b40 con 0x7fecb40f7400 2024-02-23T00:51:42.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.806+0000 7feca67fc700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7feca0062cc0 0x7feca0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:42.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.806+0000 7feca67fc700 1 -- 172.21.15.47:0/109650152 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(573..573 src has 482..573) v4 ==== 9985+0+0 (secure 0 0 0) 0x7fecb007d340 con 0x7fecb40f7400 2024-02-23T00:51:42.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.806+0000 7feca67fc700 10 client.31392.objecter ms_dispatch 0x7fecb4001550 osd_map(573..573 src has 482..573) v4 2024-02-23T00:51:42.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.806+0000 7feca67fc700 3 client.31392.objecter handle_osd_map got epochs [573,573] > 0 2024-02-23T00:51:42.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.806+0000 7feca67fc700 3 client.31392.objecter handle_osd_map decoding full epoch 573 2024-02-23T00:51:42.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.806+0000 7fecb8c1d700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7feca0062cc0 0x7feca0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:42.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.806+0000 7feca67fc700 20 client.31392.objecter dump_active .. 0 homeless 2024-02-23T00:51:42.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.806+0000 7feca67fc700 1 -- 172.21.15.47:0/109650152 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v573) v1 ==== 54+0+16 (secure 0 0 0) 0x7fecb0084070 con 0x7fecb40f7400 2024-02-23T00:51:42.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.807+0000 7feca67fc700 10 client.31392.objecter ms_handle_connect 0x7feca0062cc0 2024-02-23T00:51:42.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.807+0000 7fecb8c1d700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7feca0062cc0 0x7feca0065180 secure :-1 s=READY pgs=3128 cs=0 l=1 rev1=1 rx=0x7fecb40fa150 tx=0x7feca800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:42.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.807+0000 7fecbb682700 10 client.31392.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:42.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.807+0000 7fecbb682700 15 client.31392.objecter _session_command_op_assign -1 1 2024-02-23T00:51:42.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.807+0000 7fecbb682700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fecb40598f0 0x7fecb4059cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:42.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.807+0000 7fecbb682700 20 client.31392.objecter _get_session s=0x7fecb4105f50 osd=0 3 2024-02-23T00:51:42.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.807+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb4105f50 osd=0 3 2024-02-23T00:51:42.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.807+0000 7fecbb682700 20 client.31392.objecter _get_session s=0x7fecb4105f50 osd=0 3 2024-02-23T00:51:42.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.807+0000 7fecbb682700 15 client.31392.objecter _session_command_op_remove -1 1 2024-02-23T00:51:42.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.807+0000 7fecbb682700 20 client.31392.objecter get_session s=0x7fecb4105f50 osd=0 3 2024-02-23T00:51:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.807+0000 7fecbb682700 15 client.31392.objecter _session_command_op_assign 0 1 2024-02-23T00:51:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.807+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb4105f50 osd=0 4 2024-02-23T00:51:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.807+0000 7fecbb682700 10 client.31392.objecter _send_command 1 2024-02-23T00:51:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.807+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fecb405ee80 con 0x7fecb40598f0 2024-02-23T00:51:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.807+0000 7fecb9c1f700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fecb40598f0 0x7fecb4059cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.807+0000 7feca67fc700 10 client.31392.objecter ms_handle_connect 0x7fecb40598f0 2024-02-23T00:51:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.807+0000 7fecb9c1f700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fecb40598f0 0x7fecb4059cf0 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.808+0000 7feca67fc700 1 -- 172.21.15.47:0/109650152 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fecb405ee80 con 0x7fecb40598f0 2024-02-23T00:51:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.808+0000 7feca67fc700 10 client.31392.objecter ms_dispatch 0x7fecb4001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:51:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.808+0000 7feca67fc700 10 client.31392.objecter _finish_command 1 = system:0 2024-02-23T00:51:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.808+0000 7feca67fc700 20 client.31392.objecter put_session s=0x7fecb4105f50 osd=0 4 2024-02-23T00:51:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.808+0000 7feca67fc700 15 client.31392.objecter _session_command_op_remove 0 1 2024-02-23T00:51:42.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.823+0000 7fecbb682700 10 client.31392.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:51:42.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.823+0000 7fecbb682700 15 client.31392.objecter _session_command_op_assign -1 2 2024-02-23T00:51:42.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.823+0000 7fecbb682700 20 client.31392.objecter _get_session s=0x7fecb4105f50 osd=0 3 2024-02-23T00:51:42.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.823+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb4105f50 osd=0 3 2024-02-23T00:51:42.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.823+0000 7fecbb682700 20 client.31392.objecter _get_session s=0x7fecb4105f50 osd=0 3 2024-02-23T00:51:42.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.823+0000 7fecbb682700 15 client.31392.objecter _session_command_op_remove -1 2 2024-02-23T00:51:42.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.823+0000 7fecbb682700 20 client.31392.objecter get_session s=0x7fecb4105f50 osd=0 3 2024-02-23T00:51:42.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.823+0000 7fecbb682700 15 client.31392.objecter _session_command_op_assign 0 2 2024-02-23T00:51:42.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.823+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb4105f50 osd=0 4 2024-02-23T00:51:42.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.823+0000 7fecbb682700 10 client.31392.objecter _send_command 2 2024-02-23T00:51:42.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.823+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fecb40273c0 con 0x7fecb40598f0 2024-02-23T00:51:42.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.825+0000 7feca67fc700 1 -- 172.21.15.47:0/109650152 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fecb40273c0 con 0x7fecb40598f0 2024-02-23T00:51:42.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.825+0000 7feca67fc700 10 client.31392.objecter ms_dispatch 0x7fecb4001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:51:42.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.825+0000 7feca67fc700 10 client.31392.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:51:42.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.825+0000 7feca67fc700 20 client.31392.objecter put_session s=0x7fecb4105f50 osd=0 4 2024-02-23T00:51:42.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.825+0000 7feca67fc700 15 client.31392.objecter _session_command_op_remove 0 2 2024-02-23T00:51:42.828 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:51:42.828 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:51:42.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.825+0000 7fecbb682700 10 client.31392.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:42.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.825+0000 7fecbb682700 15 client.31392.objecter _session_command_op_assign -1 3 2024-02-23T00:51:42.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.825+0000 7fecbb682700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fecb4100190 0x7fecb4100590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:42.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.825+0000 7fecbb682700 20 client.31392.objecter _get_session s=0x7fecb4028740 osd=1 3 2024-02-23T00:51:42.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.825+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb4028740 osd=1 3 2024-02-23T00:51:42.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.825+0000 7fecbb682700 20 client.31392.objecter _get_session s=0x7fecb4028740 osd=1 3 2024-02-23T00:51:42.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.825+0000 7fecbb682700 15 client.31392.objecter _session_command_op_remove -1 3 2024-02-23T00:51:42.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.825+0000 7fecbb682700 20 client.31392.objecter get_session s=0x7fecb4028740 osd=1 3 2024-02-23T00:51:42.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.825+0000 7fecbb682700 15 client.31392.objecter _session_command_op_assign 1 3 2024-02-23T00:51:42.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.825+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb4028740 osd=1 4 2024-02-23T00:51:42.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.825+0000 7fecbb682700 10 client.31392.objecter _send_command 3 2024-02-23T00:51:42.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.825+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fecb411d800 con 0x7fecb4100190 2024-02-23T00:51:42.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.825+0000 7fecb941e700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fecb4100190 0x7fecb4100590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:42.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.826+0000 7feca67fc700 10 client.31392.objecter ms_handle_connect 0x7fecb4100190 2024-02-23T00:51:42.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.826+0000 7fecb941e700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fecb4100190 0x7fecb4100590 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:42.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.826+0000 7feca67fc700 1 -- 172.21.15.47:0/109650152 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fecb411d800 con 0x7fecb4100190 2024-02-23T00:51:42.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.826+0000 7feca67fc700 10 client.31392.objecter ms_dispatch 0x7fecb4001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:51:42.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.826+0000 7feca67fc700 10 client.31392.objecter _finish_command 3 = system:0 2024-02-23T00:51:42.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.826+0000 7feca67fc700 20 client.31392.objecter put_session s=0x7fecb4028740 osd=1 4 2024-02-23T00:51:42.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.826+0000 7feca67fc700 15 client.31392.objecter _session_command_op_remove 1 3 2024-02-23T00:51:42.831 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:42.832 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:51:42.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.842+0000 7fecbb682700 10 client.31392.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:51:42.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.842+0000 7fecbb682700 15 client.31392.objecter _session_command_op_assign -1 4 2024-02-23T00:51:42.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.842+0000 7fecbb682700 20 client.31392.objecter _get_session s=0x7fecb4028740 osd=1 3 2024-02-23T00:51:42.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.842+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb4028740 osd=1 3 2024-02-23T00:51:42.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.842+0000 7fecbb682700 20 client.31392.objecter _get_session s=0x7fecb4028740 osd=1 3 2024-02-23T00:51:42.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.842+0000 7fecbb682700 15 client.31392.objecter _session_command_op_remove -1 4 2024-02-23T00:51:42.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.842+0000 7fecbb682700 20 client.31392.objecter get_session s=0x7fecb4028740 osd=1 3 2024-02-23T00:51:42.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.842+0000 7fecbb682700 15 client.31392.objecter _session_command_op_assign 1 4 2024-02-23T00:51:42.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.842+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb4028740 osd=1 4 2024-02-23T00:51:42.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.842+0000 7fecbb682700 10 client.31392.objecter _send_command 4 2024-02-23T00:51:42.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.842+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fecb40ffc00 con 0x7fecb4100190 2024-02-23T00:51:42.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.844+0000 7feca67fc700 1 -- 172.21.15.47:0/109650152 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fecb40ffc00 con 0x7fecb4100190 2024-02-23T00:51:42.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.844+0000 7feca67fc700 10 client.31392.objecter ms_dispatch 0x7fecb4001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:51:42.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.844+0000 7feca67fc700 10 client.31392.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:51:42.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.844+0000 7feca67fc700 20 client.31392.objecter put_session s=0x7fecb4028740 osd=1 4 2024-02-23T00:51:42.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.844+0000 7feca67fc700 15 client.31392.objecter _session_command_op_remove 1 4 2024-02-23T00:51:42.847 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:51:42.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.844+0000 7fecbb682700 10 client.31392.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:42.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.844+0000 7fecbb682700 15 client.31392.objecter _session_command_op_assign -1 5 2024-02-23T00:51:42.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.844+0000 7fecbb682700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fecb40282a0 0x7fecb411c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:42.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.844+0000 7fecbb682700 20 client.31392.objecter _get_session s=0x7fecb4100010 osd=2 3 2024-02-23T00:51:42.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.844+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb4100010 osd=2 3 2024-02-23T00:51:42.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.844+0000 7fecbb682700 20 client.31392.objecter _get_session s=0x7fecb4100010 osd=2 3 2024-02-23T00:51:42.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.844+0000 7fecbb682700 15 client.31392.objecter _session_command_op_remove -1 5 2024-02-23T00:51:42.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.844+0000 7fecbb682700 20 client.31392.objecter get_session s=0x7fecb4100010 osd=2 3 2024-02-23T00:51:42.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.844+0000 7fecbb682700 15 client.31392.objecter _session_command_op_assign 2 5 2024-02-23T00:51:42.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.844+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb4100010 osd=2 4 2024-02-23T00:51:42.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.844+0000 7fecbb682700 10 client.31392.objecter _send_command 5 2024-02-23T00:51:42.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.844+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fecb411c710 con 0x7fecb40282a0 2024-02-23T00:51:42.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.844+0000 7fecb8c1d700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fecb40282a0 0x7fecb411c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:42.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.844+0000 7feca67fc700 10 client.31392.objecter ms_handle_connect 0x7fecb40282a0 2024-02-23T00:51:42.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.844+0000 7fecb8c1d700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fecb40282a0 0x7fecb411c220 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:42.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.845+0000 7feca67fc700 1 -- 172.21.15.47:0/109650152 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fecb411c710 con 0x7fecb40282a0 2024-02-23T00:51:42.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.845+0000 7feca67fc700 10 client.31392.objecter ms_dispatch 0x7fecb4001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:51:42.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.845+0000 7feca67fc700 10 client.31392.objecter _finish_command 5 = system:0 2024-02-23T00:51:42.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.845+0000 7feca67fc700 20 client.31392.objecter put_session s=0x7fecb4100010 osd=2 4 2024-02-23T00:51:42.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.845+0000 7feca67fc700 15 client.31392.objecter _session_command_op_remove 2 5 2024-02-23T00:51:42.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.860+0000 7fecbb682700 10 client.31392.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:51:42.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.860+0000 7fecbb682700 15 client.31392.objecter _session_command_op_assign -1 6 2024-02-23T00:51:42.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.860+0000 7fecbb682700 20 client.31392.objecter _get_session s=0x7fecb4100010 osd=2 3 2024-02-23T00:51:42.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.860+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb4100010 osd=2 3 2024-02-23T00:51:42.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.860+0000 7fecbb682700 20 client.31392.objecter _get_session s=0x7fecb4100010 osd=2 3 2024-02-23T00:51:42.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.860+0000 7fecbb682700 15 client.31392.objecter _session_command_op_remove -1 6 2024-02-23T00:51:42.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.860+0000 7fecbb682700 20 client.31392.objecter get_session s=0x7fecb4100010 osd=2 3 2024-02-23T00:51:42.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.860+0000 7fecbb682700 15 client.31392.objecter _session_command_op_assign 2 6 2024-02-23T00:51:42.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.860+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb4100010 osd=2 4 2024-02-23T00:51:42.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.860+0000 7fecbb682700 10 client.31392.objecter _send_command 6 2024-02-23T00:51:42.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.860+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fecb4027800 con 0x7fecb40282a0 2024-02-23T00:51:42.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.862+0000 7feca67fc700 1 -- 172.21.15.47:0/109650152 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fecb4027800 con 0x7fecb40282a0 2024-02-23T00:51:42.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.862+0000 7feca67fc700 10 client.31392.objecter ms_dispatch 0x7fecb4001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:51:42.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.862+0000 7feca67fc700 10 client.31392.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:51:42.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.862+0000 7feca67fc700 20 client.31392.objecter put_session s=0x7fecb4100010 osd=2 4 2024-02-23T00:51:42.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.862+0000 7feca67fc700 15 client.31392.objecter _session_command_op_remove 2 6 2024-02-23T00:51:42.866 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:51:42.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.862+0000 7fecbb682700 10 client.31392.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:42.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.862+0000 7fecbb682700 15 client.31392.objecter _session_command_op_assign -1 7 2024-02-23T00:51:42.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.862+0000 7fecbb682700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fecb411f980 0x7fecb4121d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:42.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.862+0000 7fecbb682700 20 client.31392.objecter _get_session s=0x7fecb411d050 osd=3 3 2024-02-23T00:51:42.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.862+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb411d050 osd=3 3 2024-02-23T00:51:42.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.862+0000 7fecbb682700 20 client.31392.objecter _get_session s=0x7fecb411d050 osd=3 3 2024-02-23T00:51:42.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.862+0000 7fecbb682700 15 client.31392.objecter _session_command_op_remove -1 7 2024-02-23T00:51:42.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.862+0000 7fecbb682700 20 client.31392.objecter get_session s=0x7fecb411d050 osd=3 3 2024-02-23T00:51:42.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.862+0000 7fecbb682700 15 client.31392.objecter _session_command_op_assign 3 7 2024-02-23T00:51:42.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.862+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb411d050 osd=3 4 2024-02-23T00:51:42.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.862+0000 7fecbb682700 10 client.31392.objecter _send_command 7 2024-02-23T00:51:42.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.862+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fecb41249c0 con 0x7fecb411f980 2024-02-23T00:51:42.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.862+0000 7fecb9c1f700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fecb411f980 0x7fecb4121d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:42.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.863+0000 7feca67fc700 10 client.31392.objecter ms_handle_connect 0x7fecb411f980 2024-02-23T00:51:42.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.863+0000 7fecb9c1f700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fecb411f980 0x7fecb4121d70 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:42.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.863+0000 7feca67fc700 1 -- 172.21.15.47:0/109650152 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fecb41249c0 con 0x7fecb411f980 2024-02-23T00:51:42.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.863+0000 7feca67fc700 10 client.31392.objecter ms_dispatch 0x7fecb4001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:51:42.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.863+0000 7feca67fc700 10 client.31392.objecter _finish_command 7 = system:0 2024-02-23T00:51:42.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.863+0000 7feca67fc700 20 client.31392.objecter put_session s=0x7fecb411d050 osd=3 4 2024-02-23T00:51:42.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.863+0000 7feca67fc700 15 client.31392.objecter _session_command_op_remove 3 7 2024-02-23T00:51:42.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.878+0000 7fecbb682700 10 client.31392.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:51:42.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.878+0000 7fecbb682700 15 client.31392.objecter _session_command_op_assign -1 8 2024-02-23T00:51:42.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.878+0000 7fecbb682700 20 client.31392.objecter _get_session s=0x7fecb411d050 osd=3 3 2024-02-23T00:51:42.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.878+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb411d050 osd=3 3 2024-02-23T00:51:42.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.878+0000 7fecbb682700 20 client.31392.objecter _get_session s=0x7fecb411d050 osd=3 3 2024-02-23T00:51:42.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.878+0000 7fecbb682700 15 client.31392.objecter _session_command_op_remove -1 8 2024-02-23T00:51:42.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.878+0000 7fecbb682700 20 client.31392.objecter get_session s=0x7fecb411d050 osd=3 3 2024-02-23T00:51:42.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.878+0000 7fecbb682700 15 client.31392.objecter _session_command_op_assign 3 8 2024-02-23T00:51:42.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.878+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb411d050 osd=3 4 2024-02-23T00:51:42.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.878+0000 7fecbb682700 10 client.31392.objecter _send_command 8 2024-02-23T00:51:42.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.878+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fecb411cac0 con 0x7fecb411f980 2024-02-23T00:51:42.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.880+0000 7feca67fc700 1 -- 172.21.15.47:0/109650152 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fecb411cac0 con 0x7fecb411f980 2024-02-23T00:51:42.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.880+0000 7feca67fc700 10 client.31392.objecter ms_dispatch 0x7fecb4001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:51:42.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.880+0000 7feca67fc700 10 client.31392.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:51:42.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.880+0000 7feca67fc700 20 client.31392.objecter put_session s=0x7fecb411d050 osd=3 4 2024-02-23T00:51:42.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.880+0000 7feca67fc700 15 client.31392.objecter _session_command_op_remove 3 8 2024-02-23T00:51:42.883 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:51:42.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.880+0000 7fecbb682700 10 client.31392.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:42.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.880+0000 7fecbb682700 15 client.31392.objecter _session_command_op_assign -1 9 2024-02-23T00:51:42.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.880+0000 7fecbb682700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fecb411cc60 0x7fecb41233e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:42.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.880+0000 7fecbb682700 20 client.31392.objecter _get_session s=0x7fecb411cac0 osd=4 3 2024-02-23T00:51:42.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.880+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb411cac0 osd=4 3 2024-02-23T00:51:42.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.880+0000 7fecbb682700 20 client.31392.objecter _get_session s=0x7fecb411cac0 osd=4 3 2024-02-23T00:51:42.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.880+0000 7fecbb682700 15 client.31392.objecter _session_command_op_remove -1 9 2024-02-23T00:51:42.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.880+0000 7fecbb682700 20 client.31392.objecter get_session s=0x7fecb411cac0 osd=4 3 2024-02-23T00:51:42.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.880+0000 7fecbb682700 15 client.31392.objecter _session_command_op_assign 4 9 2024-02-23T00:51:42.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.880+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb411cac0 osd=4 4 2024-02-23T00:51:42.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.880+0000 7fecbb682700 10 client.31392.objecter _send_command 9 2024-02-23T00:51:42.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.880+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fecb411bf20 con 0x7fecb411cc60 2024-02-23T00:51:42.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.880+0000 7fecb941e700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fecb411cc60 0x7fecb41233e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:42.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.880+0000 7feca67fc700 10 client.31392.objecter ms_handle_connect 0x7fecb411cc60 2024-02-23T00:51:42.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.880+0000 7fecb941e700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fecb411cc60 0x7fecb41233e0 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:42.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.881+0000 7feca67fc700 1 -- 172.21.15.47:0/109650152 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fecb411bf20 con 0x7fecb411cc60 2024-02-23T00:51:42.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.881+0000 7feca67fc700 10 client.31392.objecter ms_dispatch 0x7fecb4001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:51:42.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.881+0000 7feca67fc700 10 client.31392.objecter _finish_command 9 = system:0 2024-02-23T00:51:42.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.881+0000 7feca67fc700 20 client.31392.objecter put_session s=0x7fecb411cac0 osd=4 4 2024-02-23T00:51:42.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.881+0000 7feca67fc700 15 client.31392.objecter _session_command_op_remove 4 9 2024-02-23T00:51:42.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.896+0000 7fecbb682700 10 client.31392.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:51:42.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.896+0000 7fecbb682700 15 client.31392.objecter _session_command_op_assign -1 10 2024-02-23T00:51:42.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.896+0000 7fecbb682700 20 client.31392.objecter _get_session s=0x7fecb411cac0 osd=4 3 2024-02-23T00:51:42.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.896+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb411cac0 osd=4 3 2024-02-23T00:51:42.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.896+0000 7fecbb682700 20 client.31392.objecter _get_session s=0x7fecb411cac0 osd=4 3 2024-02-23T00:51:42.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.896+0000 7fecbb682700 15 client.31392.objecter _session_command_op_remove -1 10 2024-02-23T00:51:42.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.896+0000 7fecbb682700 20 client.31392.objecter get_session s=0x7fecb411cac0 osd=4 3 2024-02-23T00:51:42.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.896+0000 7fecbb682700 15 client.31392.objecter _session_command_op_assign 4 10 2024-02-23T00:51:42.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.896+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb411cac0 osd=4 4 2024-02-23T00:51:42.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.896+0000 7fecbb682700 10 client.31392.objecter _send_command 10 2024-02-23T00:51:42.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.896+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fecb411bf20 con 0x7fecb411cc60 2024-02-23T00:51:42.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.898+0000 7feca67fc700 1 -- 172.21.15.47:0/109650152 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fecb411bf20 con 0x7fecb411cc60 2024-02-23T00:51:42.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.898+0000 7feca67fc700 10 client.31392.objecter ms_dispatch 0x7fecb4001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:51:42.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.898+0000 7feca67fc700 10 client.31392.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:51:42.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.898+0000 7feca67fc700 20 client.31392.objecter put_session s=0x7fecb411cac0 osd=4 4 2024-02-23T00:51:42.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.898+0000 7feca67fc700 15 client.31392.objecter _session_command_op_remove 4 10 2024-02-23T00:51:42.901 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:51:42.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.898+0000 7fecbb682700 10 client.31392.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:42.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.898+0000 7fecbb682700 15 client.31392.objecter _session_command_op_assign -1 11 2024-02-23T00:51:42.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.898+0000 7fecbb682700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fecb4123de0 0x7fecb4124260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:42.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.898+0000 7fecbb682700 20 client.31392.objecter _get_session s=0x7fecb411bf20 osd=5 3 2024-02-23T00:51:42.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.898+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb411bf20 osd=5 3 2024-02-23T00:51:42.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.898+0000 7fecbb682700 20 client.31392.objecter _get_session s=0x7fecb411bf20 osd=5 3 2024-02-23T00:51:42.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.898+0000 7fecbb682700 15 client.31392.objecter _session_command_op_remove -1 11 2024-02-23T00:51:42.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.898+0000 7fecbb682700 20 client.31392.objecter get_session s=0x7fecb411bf20 osd=5 3 2024-02-23T00:51:42.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.898+0000 7fecbb682700 15 client.31392.objecter _session_command_op_assign 5 11 2024-02-23T00:51:42.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.898+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb411bf20 osd=5 4 2024-02-23T00:51:42.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.898+0000 7fecbb682700 10 client.31392.objecter _send_command 11 2024-02-23T00:51:42.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.898+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fecb41247a0 con 0x7fecb4123de0 2024-02-23T00:51:42.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.898+0000 7fecb8c1d700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fecb4123de0 0x7fecb4124260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:42.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.898+0000 7feca67fc700 10 client.31392.objecter ms_handle_connect 0x7fecb4123de0 2024-02-23T00:51:42.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.898+0000 7fecb8c1d700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fecb4123de0 0x7fecb4124260 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:42.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.899+0000 7feca67fc700 1 -- 172.21.15.47:0/109650152 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fecb41247a0 con 0x7fecb4123de0 2024-02-23T00:51:42.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.899+0000 7feca67fc700 10 client.31392.objecter ms_dispatch 0x7fecb4001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:51:42.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.899+0000 7feca67fc700 10 client.31392.objecter _finish_command 11 = system:0 2024-02-23T00:51:42.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.899+0000 7feca67fc700 20 client.31392.objecter put_session s=0x7fecb411bf20 osd=5 4 2024-02-23T00:51:42.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.899+0000 7feca67fc700 15 client.31392.objecter _session_command_op_remove 5 11 2024-02-23T00:51:42.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.914+0000 7fecbb682700 10 client.31392.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:51:42.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.914+0000 7fecbb682700 15 client.31392.objecter _session_command_op_assign -1 12 2024-02-23T00:51:42.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.914+0000 7fecbb682700 20 client.31392.objecter _get_session s=0x7fecb411bf20 osd=5 3 2024-02-23T00:51:42.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.914+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb411bf20 osd=5 3 2024-02-23T00:51:42.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.914+0000 7fecbb682700 20 client.31392.objecter _get_session s=0x7fecb411bf20 osd=5 3 2024-02-23T00:51:42.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.914+0000 7fecbb682700 15 client.31392.objecter _session_command_op_remove -1 12 2024-02-23T00:51:42.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.914+0000 7fecbb682700 20 client.31392.objecter get_session s=0x7fecb411bf20 osd=5 3 2024-02-23T00:51:42.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.914+0000 7fecbb682700 15 client.31392.objecter _session_command_op_assign 5 12 2024-02-23T00:51:42.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.914+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb411bf20 osd=5 4 2024-02-23T00:51:42.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.914+0000 7fecbb682700 10 client.31392.objecter _send_command 12 2024-02-23T00:51:42.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.914+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fecb41247a0 con 0x7fecb4123de0 2024-02-23T00:51:42.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.916+0000 7feca67fc700 1 -- 172.21.15.47:0/109650152 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fecb41247a0 con 0x7fecb4123de0 2024-02-23T00:51:42.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.916+0000 7feca67fc700 10 client.31392.objecter ms_dispatch 0x7fecb4001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:51:42.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.916+0000 7feca67fc700 10 client.31392.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:51:42.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.916+0000 7feca67fc700 20 client.31392.objecter put_session s=0x7fecb411bf20 osd=5 4 2024-02-23T00:51:42.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.916+0000 7feca67fc700 15 client.31392.objecter _session_command_op_remove 5 12 2024-02-23T00:51:42.919 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:51:42.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.916+0000 7fecbb682700 10 client.31392.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:42.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.916+0000 7fecbb682700 15 client.31392.objecter _session_command_op_assign -1 13 2024-02-23T00:51:42.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.916+0000 7fecbb682700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fecb41290f0 0x7fecb412b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:42.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.916+0000 7fecbb682700 20 client.31392.objecter _get_session s=0x7fecb41247a0 osd=6 3 2024-02-23T00:51:42.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.916+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb41247a0 osd=6 3 2024-02-23T00:51:42.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.916+0000 7fecbb682700 20 client.31392.objecter _get_session s=0x7fecb41247a0 osd=6 3 2024-02-23T00:51:42.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.916+0000 7fecbb682700 15 client.31392.objecter _session_command_op_remove -1 13 2024-02-23T00:51:42.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.916+0000 7fecbb682700 20 client.31392.objecter get_session s=0x7fecb41247a0 osd=6 3 2024-02-23T00:51:42.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.916+0000 7fecbb682700 15 client.31392.objecter _session_command_op_assign 6 13 2024-02-23T00:51:42.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.916+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb41247a0 osd=6 4 2024-02-23T00:51:42.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.916+0000 7fecbb682700 10 client.31392.objecter _send_command 13 2024-02-23T00:51:42.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.916+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fecb412e280 con 0x7fecb41290f0 2024-02-23T00:51:42.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.917+0000 7fecb9c1f700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fecb41290f0 0x7fecb412b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:42.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.917+0000 7feca67fc700 10 client.31392.objecter ms_handle_connect 0x7fecb41290f0 2024-02-23T00:51:42.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.917+0000 7fecb9c1f700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fecb41290f0 0x7fecb412b4e0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:42.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.918+0000 7feca67fc700 1 -- 172.21.15.47:0/109650152 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fecb412e280 con 0x7fecb41290f0 2024-02-23T00:51:42.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.918+0000 7feca67fc700 10 client.31392.objecter ms_dispatch 0x7fecb4001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:51:42.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.918+0000 7feca67fc700 10 client.31392.objecter _finish_command 13 = system:0 2024-02-23T00:51:42.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.918+0000 7feca67fc700 20 client.31392.objecter put_session s=0x7fecb41247a0 osd=6 4 2024-02-23T00:51:42.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.918+0000 7feca67fc700 15 client.31392.objecter _session_command_op_remove 6 13 2024-02-23T00:51:42.933 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:51:42.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.932+0000 7fecbb682700 10 client.31392.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:51:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.932+0000 7fecbb682700 15 client.31392.objecter _session_command_op_assign -1 14 2024-02-23T00:51:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.932+0000 7fecbb682700 20 client.31392.objecter _get_session s=0x7fecb41247a0 osd=6 3 2024-02-23T00:51:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.932+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb41247a0 osd=6 3 2024-02-23T00:51:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.932+0000 7fecbb682700 20 client.31392.objecter _get_session s=0x7fecb41247a0 osd=6 3 2024-02-23T00:51:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.932+0000 7fecbb682700 15 client.31392.objecter _session_command_op_remove -1 14 2024-02-23T00:51:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.932+0000 7fecbb682700 20 client.31392.objecter get_session s=0x7fecb41247a0 osd=6 3 2024-02-23T00:51:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.932+0000 7fecbb682700 15 client.31392.objecter _session_command_op_assign 6 14 2024-02-23T00:51:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.932+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb41247a0 osd=6 4 2024-02-23T00:51:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.932+0000 7fecbb682700 10 client.31392.objecter _send_command 14 2024-02-23T00:51:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.932+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fecb412e280 con 0x7fecb41290f0 2024-02-23T00:51:42.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.934+0000 7feca67fc700 1 -- 172.21.15.47:0/109650152 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fecb412e280 con 0x7fecb41290f0 2024-02-23T00:51:42.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.934+0000 7feca67fc700 10 client.31392.objecter ms_dispatch 0x7fecb4001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:51:42.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.934+0000 7feca67fc700 10 client.31392.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:51:42.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.934+0000 7feca67fc700 20 client.31392.objecter put_session s=0x7fecb41247a0 osd=6 4 2024-02-23T00:51:42.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.934+0000 7feca67fc700 15 client.31392.objecter _session_command_op_remove 6 14 2024-02-23T00:51:42.937 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:51:42.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.934+0000 7fecbb682700 10 client.31392.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:42.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.934+0000 7fecbb682700 15 client.31392.objecter _session_command_op_assign -1 15 2024-02-23T00:51:42.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.934+0000 7fecbb682700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fecb412d5b0 0x7fecb412da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:42.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.934+0000 7fecbb682700 20 client.31392.objecter _get_session s=0x7fecb412e280 osd=7 3 2024-02-23T00:51:42.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.934+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb412e280 osd=7 3 2024-02-23T00:51:42.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.934+0000 7fecbb682700 20 client.31392.objecter _get_session s=0x7fecb412e280 osd=7 3 2024-02-23T00:51:42.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.934+0000 7fecbb682700 15 client.31392.objecter _session_command_op_remove -1 15 2024-02-23T00:51:42.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.934+0000 7fecbb682700 20 client.31392.objecter get_session s=0x7fecb412e280 osd=7 3 2024-02-23T00:51:42.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.934+0000 7fecbb682700 15 client.31392.objecter _session_command_op_assign 7 15 2024-02-23T00:51:42.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.934+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb412e280 osd=7 4 2024-02-23T00:51:42.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.934+0000 7fecbb682700 10 client.31392.objecter _send_command 15 2024-02-23T00:51:42.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.934+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fecb4131600 con 0x7fecb412d5b0 2024-02-23T00:51:42.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.935+0000 7fecb941e700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fecb412d5b0 0x7fecb412da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:42.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.935+0000 7feca67fc700 10 client.31392.objecter ms_handle_connect 0x7fecb412d5b0 2024-02-23T00:51:42.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.935+0000 7fecb941e700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fecb412d5b0 0x7fecb412da10 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:42.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.936+0000 7feca67fc700 1 -- 172.21.15.47:0/109650152 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fecb4131600 con 0x7fecb412d5b0 2024-02-23T00:51:42.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.936+0000 7feca67fc700 10 client.31392.objecter ms_dispatch 0x7fecb4001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:51:42.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.936+0000 7feca67fc700 10 client.31392.objecter _finish_command 15 = system:0 2024-02-23T00:51:42.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.936+0000 7feca67fc700 20 client.31392.objecter put_session s=0x7fecb412e280 osd=7 4 2024-02-23T00:51:42.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.936+0000 7feca67fc700 15 client.31392.objecter _session_command_op_remove 7 15 2024-02-23T00:51:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.952+0000 7fecbb682700 10 client.31392.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:51:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.952+0000 7fecbb682700 15 client.31392.objecter _session_command_op_assign -1 16 2024-02-23T00:51:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.952+0000 7fecbb682700 20 client.31392.objecter _get_session s=0x7fecb412e280 osd=7 3 2024-02-23T00:51:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.952+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb412e280 osd=7 3 2024-02-23T00:51:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.952+0000 7fecbb682700 20 client.31392.objecter _get_session s=0x7fecb412e280 osd=7 3 2024-02-23T00:51:42.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.952+0000 7fecbb682700 15 client.31392.objecter _session_command_op_remove -1 16 2024-02-23T00:51:42.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.952+0000 7fecbb682700 20 client.31392.objecter get_session s=0x7fecb412e280 osd=7 3 2024-02-23T00:51:42.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.952+0000 7fecbb682700 15 client.31392.objecter _session_command_op_assign 7 16 2024-02-23T00:51:42.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.952+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb412e280 osd=7 4 2024-02-23T00:51:42.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.952+0000 7fecbb682700 10 client.31392.objecter _send_command 16 2024-02-23T00:51:42.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.952+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fecb412ba40 con 0x7fecb412d5b0 2024-02-23T00:51:42.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.954+0000 7feca67fc700 1 -- 172.21.15.47:0/109650152 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fecb412ba40 con 0x7fecb412d5b0 2024-02-23T00:51:42.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.954+0000 7feca67fc700 10 client.31392.objecter ms_dispatch 0x7fecb4001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:51:42.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.954+0000 7feca67fc700 10 client.31392.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:51:42.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.954+0000 7feca67fc700 20 client.31392.objecter put_session s=0x7fecb412e280 osd=7 4 2024-02-23T00:51:42.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.954+0000 7feca67fc700 15 client.31392.objecter _session_command_op_remove 7 16 2024-02-23T00:51:42.958 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:51:42.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 10 librados: watch_flush enter 2024-02-23T00:51:42.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 10 librados: watch_flush exit 2024-02-23T00:51:42.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 10 client.31392.objecter close_session for osd.0 2024-02-23T00:51:42.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fecb40598f0 msgr2=0x7fecb4059cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:42.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fecb40598f0 0x7fecb4059cf0 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:42.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb4105f50 osd=0 1 2024-02-23T00:51:42.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 10 client.31392.objecter close_session for osd.1 2024-02-23T00:51:42.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fecb4100190 msgr2=0x7fecb4100590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:42.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fecb4100190 0x7fecb4100590 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:42.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb4028740 osd=1 1 2024-02-23T00:51:42.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 10 client.31392.objecter close_session for osd.2 2024-02-23T00:51:42.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fecb40282a0 msgr2=0x7fecb411c220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:42.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fecb40282a0 0x7fecb411c220 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:42.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb4100010 osd=2 1 2024-02-23T00:51:42.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 10 client.31392.objecter close_session for osd.3 2024-02-23T00:51:42.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fecb411f980 msgr2=0x7fecb4121d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:42.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fecb411f980 0x7fecb4121d70 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:42.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecb9c1f700 1 -- 172.21.15.47:0/109650152 reap_dead start 2024-02-23T00:51:42.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb411d050 osd=3 1 2024-02-23T00:51:42.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 10 client.31392.objecter close_session for osd.4 2024-02-23T00:51:42.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fecb411cc60 msgr2=0x7fecb41233e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:42.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fecb411cc60 0x7fecb41233e0 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:42.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb411cac0 osd=4 1 2024-02-23T00:51:42.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecb9c1f700 1 -- 172.21.15.47:0/109650152 reap_dead start 2024-02-23T00:51:42.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 10 client.31392.objecter close_session for osd.5 2024-02-23T00:51:42.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fecb4123de0 msgr2=0x7fecb4124260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:42.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fecb4123de0 0x7fecb4124260 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:42.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb411bf20 osd=5 1 2024-02-23T00:51:42.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 10 client.31392.objecter close_session for osd.6 2024-02-23T00:51:42.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fecb41290f0 msgr2=0x7fecb412b4e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:42.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fecb41290f0 0x7fecb412b4e0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:42.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb41247a0 osd=6 1 2024-02-23T00:51:42.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 10 client.31392.objecter close_session for osd.7 2024-02-23T00:51:42.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fecb412d5b0 msgr2=0x7fecb412da10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:42.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fecb412d5b0 0x7fecb412da10 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:42.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 20 client.31392.objecter put_session s=0x7fecb412e280 osd=7 1 2024-02-23T00:51:42.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 20 client.31392.objecter shutdown clearing up homeless session... 2024-02-23T00:51:42.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 10 client.31392.objecter successfully canceled tick 2024-02-23T00:51:42.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7feca0062cc0 msgr2=0x7feca0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7feca0062cc0 0x7feca0065180 secure :-1 s=READY pgs=3128 cs=0 l=1 rev1=1 rx=0x7fecb40fa150 tx=0x7feca800b040).stop 2024-02-23T00:51:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecb40f7400 msgr2=0x7fecb40f89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 1 --2- 172.21.15.47:0/109650152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecb40f7400 0x7fecb40f89b0 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 rx=0x7fecb000e970 tx=0x7fecb000b040).stop 2024-02-23T00:51:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecb9c1f700 1 -- 172.21.15.47:0/109650152 reap_dead start 2024-02-23T00:51:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 shutdown_connections 2024-02-23T00:51:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 >> 172.21.15.47:0/109650152 conn(0x7fecb40f2820 msgr2=0x7fecb4060d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 shutdown_connections 2024-02-23T00:51:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 1 -- 172.21.15.47:0/109650152 wait complete. 2024-02-23T00:51:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:42.955+0000 7fecbb682700 1 librados: shutdown 2024-02-23T00:51:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:51:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:51:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:51:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:51:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:51:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:51:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:51:42.963 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:51:42.963 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:43.133 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:51:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:51:43.313 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:51:43.317 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:43.317 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:51:43.418 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:51:43.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:42.246077+0000 osd.7 (osd.7) 279 : cluster [DBG] 2.1c scrub starts 2024-02-23T00:51:43.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:42.248978+0000 osd.7 (osd.7) 280 : cluster [DBG] 2.1c scrub ok 2024-02-23T00:51:43.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:42.275958+0000 mon.a (mon.0) 3027 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:51:43.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:42.285382+0000 mon.a (mon.0) 3028 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-97"}]': finished 2024-02-23T00:51:43.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:42.285461+0000 mon.a (mon.0) 3029 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:51:43.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:42.285535+0000 mon.a (mon.0) 3030 : cluster [DBG] osdmap e573: 8 total, 8 up, 7 in 2024-02-23T00:51:43.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:42.286108+0000 mon.a (mon.0) 3031 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-97", "tierpool": "test-rados-api-smithi047-82465-97-cache"}]: dispatch 2024-02-23T00:51:43.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:42.558454+0000 osd.1 (osd.1) 243 : cluster [DBG] 4.7 scrub starts 2024-02-23T00:51:43.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:42.560134+0000 osd.1 (osd.1) 244 : cluster [DBG] 4.7 scrub ok 2024-02-23T00:51:43.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:42.669679+0000 osd.0 (osd.0) 241 : cluster [DBG] 4.16 scrub starts 2024-02-23T00:51:43.446 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:42.670968+0000 osd.0 (osd.0) 242 : cluster [DBG] 4.16 scrub ok 2024-02-23T00:51:43.446 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:42.807520+0000 mon.a (mon.0) 3032 : audit [DBG] from='client.? 172.21.15.47:0/109650152' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:51:43.446 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:43.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:42.246077+0000 osd.7 (osd.7) 279 : cluster [DBG] 2.1c scrub starts 2024-02-23T00:51:43.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:42.248978+0000 osd.7 (osd.7) 280 : cluster [DBG] 2.1c scrub ok 2024-02-23T00:51:43.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:42.275958+0000 mon.a (mon.0) 3027 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:51:43.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:42.285382+0000 mon.a (mon.0) 3028 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-97"}]': finished 2024-02-23T00:51:43.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:42.285461+0000 mon.a (mon.0) 3029 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:51:43.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:42.285535+0000 mon.a (mon.0) 3030 : cluster [DBG] osdmap e573: 8 total, 8 up, 7 in 2024-02-23T00:51:43.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:42.286108+0000 mon.a (mon.0) 3031 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-97", "tierpool": "test-rados-api-smithi047-82465-97-cache"}]: dispatch 2024-02-23T00:51:43.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:42.558454+0000 osd.1 (osd.1) 243 : cluster [DBG] 4.7 scrub starts 2024-02-23T00:51:43.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:42.560134+0000 osd.1 (osd.1) 244 : cluster [DBG] 4.7 scrub ok 2024-02-23T00:51:43.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:42.669679+0000 osd.0 (osd.0) 241 : cluster [DBG] 4.16 scrub starts 2024-02-23T00:51:43.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:42.670968+0000 osd.0 (osd.0) 242 : cluster [DBG] 4.16 scrub ok 2024-02-23T00:51:43.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:42.807520+0000 mon.a (mon.0) 3032 : audit [DBG] from='client.? 172.21.15.47:0/109650152' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:51:43.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:42.246077+0000 osd.7 (osd.7) 279 : cluster [DBG] 2.1c scrub starts 2024-02-23T00:51:43.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:42.248978+0000 osd.7 (osd.7) 280 : cluster [DBG] 2.1c scrub ok 2024-02-23T00:51:43.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:42.275958+0000 mon.a (mon.0) 3027 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:51:43.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:42.285382+0000 mon.a (mon.0) 3028 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-97"}]': finished 2024-02-23T00:51:43.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:42.285461+0000 mon.a (mon.0) 3029 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:51:43.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:42.285535+0000 mon.a (mon.0) 3030 : cluster [DBG] osdmap e573: 8 total, 8 up, 7 in 2024-02-23T00:51:43.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:42.286108+0000 mon.a (mon.0) 3031 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-97", "tierpool": "test-rados-api-smithi047-82465-97-cache"}]: dispatch 2024-02-23T00:51:43.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:42.558454+0000 osd.1 (osd.1) 243 : cluster [DBG] 4.7 scrub starts 2024-02-23T00:51:43.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:42.560134+0000 osd.1 (osd.1) 244 : cluster [DBG] 4.7 scrub ok 2024-02-23T00:51:43.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:42.669679+0000 osd.0 (osd.0) 241 : cluster [DBG] 4.16 scrub starts 2024-02-23T00:51:43.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:42.670968+0000 osd.0 (osd.0) 242 : cluster [DBG] 4.16 scrub ok 2024-02-23T00:51:43.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:42.807520+0000 mon.a (mon.0) 3032 : audit [DBG] from='client.? 172.21.15.47:0/109650152' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:51:43.967 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:51:44.196 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:51:44.197 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:44.198 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:51:44.299 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:51:44.329 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:44.348 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:51:44.351 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:44.352 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:51:44.453 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:51:44.467 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:51:44.467 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:51:44.481 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:42.769050+0000 mgr.y (mgr.14517) 518 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:51:44.481 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:43.282466+0000 mon.a (mon.0) 3033 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:51:44.481 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:43.286190+0000 mon.a (mon.0) 3034 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-97", "tierpool": "test-rados-api-smithi047-82465-97-cache"}]': finished 2024-02-23T00:51:44.481 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:43.286248+0000 mon.a (mon.0) 3035 : cluster [DBG] osdmap e574: 8 total, 8 up, 7 in 2024-02-23T00:51:44.481 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:43.346119+0000 mgr.y (mgr.14517) 519 : cluster [DBG] pgmap v865: 216 pgs: 216 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-23T00:51:44.481 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:43.641706+0000 osd.5 (osd.5) 250 : cluster [DBG] 5.7 scrub starts 2024-02-23T00:51:44.481 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:43.644587+0000 osd.5 (osd.5) 251 : cluster [DBG] 5.7 scrub ok 2024-02-23T00:51:44.481 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:43.722889+0000 osd.4 (osd.4) 237 : cluster [DBG] 4.9 scrub starts 2024-02-23T00:51:44.481 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:43.724666+0000 osd.4 (osd.4) 238 : cluster [DBG] 4.9 scrub ok 2024-02-23T00:51:44.482 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:42.769050+0000 mgr.y (mgr.14517) 518 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:51:44.482 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:43.282466+0000 mon.a (mon.0) 3033 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:51:44.482 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:43.286190+0000 mon.a (mon.0) 3034 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-97", "tierpool": "test-rados-api-smithi047-82465-97-cache"}]': finished 2024-02-23T00:51:44.482 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:43.286248+0000 mon.a (mon.0) 3035 : cluster [DBG] osdmap e574: 8 total, 8 up, 7 in 2024-02-23T00:51:44.482 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:43.346119+0000 mgr.y (mgr.14517) 519 : cluster [DBG] pgmap v865: 216 pgs: 216 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-23T00:51:44.482 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:43.641706+0000 osd.5 (osd.5) 250 : cluster [DBG] 5.7 scrub starts 2024-02-23T00:51:44.482 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:43.644587+0000 osd.5 (osd.5) 251 : cluster [DBG] 5.7 scrub ok 2024-02-23T00:51:44.482 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:43.722889+0000 osd.4 (osd.4) 237 : cluster [DBG] 4.9 scrub starts 2024-02-23T00:51:44.482 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:43.724666+0000 osd.4 (osd.4) 238 : cluster [DBG] 4.9 scrub ok 2024-02-23T00:51:44.484 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:44.490 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:42.769050+0000 mgr.y (mgr.14517) 518 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:51:44.490 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:43.282466+0000 mon.a (mon.0) 3033 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:51:44.490 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:43.286190+0000 mon.a (mon.0) 3034 : audit [INF] from='client.? 172.21.15.47:0/567105858' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-97", "tierpool": "test-rados-api-smithi047-82465-97-cache"}]': finished 2024-02-23T00:51:44.490 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:43.286248+0000 mon.a (mon.0) 3035 : cluster [DBG] osdmap e574: 8 total, 8 up, 7 in 2024-02-23T00:51:44.490 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:43.346119+0000 mgr.y (mgr.14517) 519 : cluster [DBG] pgmap v865: 216 pgs: 216 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-23T00:51:44.490 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:43.641706+0000 osd.5 (osd.5) 250 : cluster [DBG] 5.7 scrub starts 2024-02-23T00:51:44.491 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:43.644587+0000 osd.5 (osd.5) 251 : cluster [DBG] 5.7 scrub ok 2024-02-23T00:51:44.491 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:43.722889+0000 osd.4 (osd.4) 237 : cluster [DBG] 4.9 scrub starts 2024-02-23T00:51:44.491 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:43.724666+0000 osd.4 (osd.4) 238 : cluster [DBG] 4.9 scrub ok 2024-02-23T00:51:44.836 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:51:44.838 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:44.839 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:51:44.940 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:51:44.970 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:45.128 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.7 dump_ops_in_flight 2024-02-23T00:51:45.329 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set noscrub 2024-02-23T00:51:45.418 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:51:45.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:44.269669+0000 osd.6 (osd.6) 225 : cluster [DBG] 2.14 scrub starts 2024-02-23T00:51:45.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:44.270460+0000 osd.6 (osd.6) 226 : cluster [DBG] 2.14 scrub ok 2024-02-23T00:51:45.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:44.289423+0000 mon.a (mon.0) 3036 : cluster [DBG] osdmap e575: 8 total, 8 up, 7 in 2024-02-23T00:51:45.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:44.637286+0000 osd.0 (osd.0) 243 : cluster [DBG] 18.4 scrub starts 2024-02-23T00:51:45.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:44.638125+0000 osd.0 (osd.0) 244 : cluster [DBG] 18.4 scrub ok 2024-02-23T00:51:45.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:44.658510+0000 osd.3 (osd.3) 272 : cluster [DBG] 2.1 deep-scrub starts 2024-02-23T00:51:45.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:44.659242+0000 osd.3 (osd.3) 273 : cluster [DBG] 2.1 deep-scrub ok 2024-02-23T00:51:45.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:44.697213+0000 osd.4 (osd.4) 239 : cluster [DBG] 5.a scrub starts 2024-02-23T00:51:45.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:44.698059+0000 osd.4 (osd.4) 240 : cluster [DBG] 5.a scrub ok 2024-02-23T00:51:45.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:45.248861+0000 mon.a (mon.0) 3037 : cluster [DBG] osdmap e576: 8 total, 8 up, 7 in 2024-02-23T00:51:45.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:45.280435+0000 mon.a (mon.0) 3038 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82465-99"}]: dispatch 2024-02-23T00:51:45.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:45.282059+0000 mon.a (mon.0) 3039 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82465-99"}]: dispatch 2024-02-23T00:51:45.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:45.282584+0000 mon.a (mon.0) 3040 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi047-82465-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:51:45.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:44.269669+0000 osd.6 (osd.6) 225 : cluster [DBG] 2.14 scrub starts 2024-02-23T00:51:45.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:44.270460+0000 osd.6 (osd.6) 226 : cluster [DBG] 2.14 scrub ok 2024-02-23T00:51:45.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:44.289423+0000 mon.a (mon.0) 3036 : cluster [DBG] osdmap e575: 8 total, 8 up, 7 in 2024-02-23T00:51:45.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:44.637286+0000 osd.0 (osd.0) 243 : cluster [DBG] 18.4 scrub starts 2024-02-23T00:51:45.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:44.638125+0000 osd.0 (osd.0) 244 : cluster [DBG] 18.4 scrub ok 2024-02-23T00:51:45.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:44.658510+0000 osd.3 (osd.3) 272 : cluster [DBG] 2.1 deep-scrub starts 2024-02-23T00:51:45.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:44.659242+0000 osd.3 (osd.3) 273 : cluster [DBG] 2.1 deep-scrub ok 2024-02-23T00:51:45.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:44.697213+0000 osd.4 (osd.4) 239 : cluster [DBG] 5.a scrub starts 2024-02-23T00:51:45.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:44.698059+0000 osd.4 (osd.4) 240 : cluster [DBG] 5.a scrub ok 2024-02-23T00:51:45.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:45.248861+0000 mon.a (mon.0) 3037 : cluster [DBG] osdmap e576: 8 total, 8 up, 7 in 2024-02-23T00:51:45.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:45.280435+0000 mon.a (mon.0) 3038 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82465-99"}]: dispatch 2024-02-23T00:51:45.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:45.282059+0000 mon.a (mon.0) 3039 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82465-99"}]: dispatch 2024-02-23T00:51:45.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:45.282584+0000 mon.a (mon.0) 3040 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi047-82465-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:51:45.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:44.269669+0000 osd.6 (osd.6) 225 : cluster [DBG] 2.14 scrub starts 2024-02-23T00:51:45.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:44.270460+0000 osd.6 (osd.6) 226 : cluster [DBG] 2.14 scrub ok 2024-02-23T00:51:45.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:44.289423+0000 mon.a (mon.0) 3036 : cluster [DBG] osdmap e575: 8 total, 8 up, 7 in 2024-02-23T00:51:45.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:44.637286+0000 osd.0 (osd.0) 243 : cluster [DBG] 18.4 scrub starts 2024-02-23T00:51:45.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:44.638125+0000 osd.0 (osd.0) 244 : cluster [DBG] 18.4 scrub ok 2024-02-23T00:51:45.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:44.658510+0000 osd.3 (osd.3) 272 : cluster [DBG] 2.1 deep-scrub starts 2024-02-23T00:51:45.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:44.659242+0000 osd.3 (osd.3) 273 : cluster [DBG] 2.1 deep-scrub ok 2024-02-23T00:51:45.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:44.697213+0000 osd.4 (osd.4) 239 : cluster [DBG] 5.a scrub starts 2024-02-23T00:51:45.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:44.698059+0000 osd.4 (osd.4) 240 : cluster [DBG] 5.a scrub ok 2024-02-23T00:51:45.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:45.248861+0000 mon.a (mon.0) 3037 : cluster [DBG] osdmap e576: 8 total, 8 up, 7 in 2024-02-23T00:51:45.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:45.280435+0000 mon.a (mon.0) 3038 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82465-99"}]: dispatch 2024-02-23T00:51:45.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:45.282059+0000 mon.a (mon.0) 3039 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82465-99"}]: dispatch 2024-02-23T00:51:45.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:45.282584+0000 mon.a (mon.0) 3040 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi047-82465-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:51:45.622 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:51:45.631 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:45.631 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:51:45.732 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:51:45.761 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:45.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.844+0000 7fad7e480700 1 -- 172.21.15.47:0/3543322684 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fad780fdbb0 msgr2=0x7fad780fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:45.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.844+0000 7fad7e480700 1 --2- 172.21.15.47:0/3543322684 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fad780fdbb0 0x7fad780fdf90 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 rx=0x7fad680047b0 tx=0x7fad68009cb0).stop 2024-02-23T00:51:45.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.844+0000 7fad7e480700 1 -- 172.21.15.47:0/3543322684 shutdown_connections 2024-02-23T00:51:45.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.844+0000 7fad7e480700 1 --2- 172.21.15.47:0/3543322684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad780f8420 0x7fad78104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:45.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.844+0000 7fad7e480700 1 --2- 172.21.15.47:0/3543322684 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fad780fdbb0 0x7fad780fdf90 unknown :-1 s=CLOSED pgs=2740 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:45.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.844+0000 7fad7e480700 1 --2- 172.21.15.47:0/3543322684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad780fe510 0x7fad780f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:45.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.844+0000 7fad7e480700 1 -- 172.21.15.47:0/3543322684 >> 172.21.15.47:0/3543322684 conn(0x7fad780f3800 msgr2=0x7fad780f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:45.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.844+0000 7fad7e480700 1 -- 172.21.15.47:0/3543322684 shutdown_connections 2024-02-23T00:51:45.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.844+0000 7fad7e480700 1 -- 172.21.15.47:0/3543322684 wait complete. 2024-02-23T00:51:45.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.845+0000 7fad7e480700 1 librados: starting msgr at 2024-02-23T00:51:45.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.845+0000 7fad7e480700 1 librados: starting objecter 2024-02-23T00:51:45.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.845+0000 7fad7e480700 1 Processor -- start 2024-02-23T00:51:45.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.845+0000 7fad7e480700 1 -- start start 2024-02-23T00:51:45.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.845+0000 7fad7e480700 1 librados: setting wanted keys 2024-02-23T00:51:45.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.845+0000 7fad7e480700 1 librados: calling monclient init 2024-02-23T00:51:45.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.845+0000 7fad7e480700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad780f8420 0x7fad7810dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:45.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.845+0000 7fad7e480700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad780fdbb0 0x7fad7810e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:45.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.845+0000 7fad7e480700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fad780fe510 0x7fad781081f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:45.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.845+0000 7fad7e480700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad781086e0 con 0x7fad780fdbb0 2024-02-23T00:51:45.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.845+0000 7fad7e480700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad78108850 con 0x7fad780f8420 2024-02-23T00:51:45.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.845+0000 7fad7e480700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad78108a10 con 0x7fad780fe510 2024-02-23T00:51:45.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.846+0000 7fad777fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad780fdbb0 0x7fad7810e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:45.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.846+0000 7fad7ca1d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fad780fe510 0x7fad781081f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:45.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.846+0000 7fad777fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad780fdbb0 0x7fad7810e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40164/0 (socket says 172.21.15.47:40164) 2024-02-23T00:51:45.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.846+0000 7fad777fe700 1 -- 172.21.15.47:0/1516800775 learned_addr learned my addr 172.21.15.47:0/1516800775 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:51:45.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.846+0000 7fad77fff700 1 --2- 172.21.15.47:0/1516800775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad780f8420 0x7fad7810dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:45.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.846+0000 7fad7ca1d700 1 -- 172.21.15.47:0/1516800775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad780f8420 msgr2=0x7fad7810dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:45.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.846+0000 7fad7ca1d700 1 --2- 172.21.15.47:0/1516800775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad780f8420 0x7fad7810dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:45.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.846+0000 7fad7ca1d700 1 -- 172.21.15.47:0/1516800775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad780fdbb0 msgr2=0x7fad7810e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:45.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.846+0000 7fad7ca1d700 1 --2- 172.21.15.47:0/1516800775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad780fdbb0 0x7fad7810e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:45.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.846+0000 7fad7ca1d700 1 -- 172.21.15.47:0/1516800775 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad700097c0 con 0x7fad780fe510 2024-02-23T00:51:45.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.846+0000 7fad777fe700 1 --2- 172.21.15.47:0/1516800775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad780fdbb0 0x7fad7810e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:51:45.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.846+0000 7fad7ca1d700 1 --2- 172.21.15.47:0/1516800775 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fad780fe510 0x7fad781081f0 secure :-1 s=READY pgs=2741 cs=0 l=1 rev1=1 rx=0x7fad70001df0 tx=0x7fad70004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:45.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.846+0000 7fad757fa700 10 client.?.objecter ms_handle_connect 0x7fad780fe510 2024-02-23T00:51:45.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.846+0000 7fad757fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:51:45.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.846+0000 7fad757fa700 1 -- 172.21.15.47:0/1516800775 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad700099f0 con 0x7fad780fe510 2024-02-23T00:51:45.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.846+0000 7fad757fa700 1 -- 172.21.15.47:0/1516800775 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fad700090d0 con 0x7fad780fe510 2024-02-23T00:51:45.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.846+0000 7fad7e480700 1 -- 172.21.15.47:0/1516800775 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad68005040 con 0x7fad780fe510 2024-02-23T00:51:45.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.846+0000 7fad757fa700 1 -- 172.21.15.47:0/1516800775 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad70007b10 con 0x7fad780fe510 2024-02-23T00:51:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.846+0000 7fad7e480700 10 client.41723.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:51:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.846+0000 7fad7e480700 1 -- 172.21.15.47:0/1516800775 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad78109050 con 0x7fad780fe510 2024-02-23T00:51:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.846+0000 7fad7e480700 1 librados: init done 2024-02-23T00:51:45.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.847+0000 7fad7e480700 1 -- 172.21.15.47:0/1516800775 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad7805ee80 con 0x7fad780fe510 2024-02-23T00:51:45.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.848+0000 7fad757fa700 1 -- 172.21.15.47:0/1516800775 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fad7000fad0 con 0x7fad780fe510 2024-02-23T00:51:45.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.848+0000 7fad757fa700 1 --2- 172.21.15.47:0/1516800775 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fad60062cc0 0x7fad60065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:45.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.848+0000 7fad757fa700 1 -- 172.21.15.47:0/1516800775 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(576..576 src has 482..576) v4 ==== 9201+0+0 (secure 0 0 0) 0x7fad7007de70 con 0x7fad780fe510 2024-02-23T00:51:45.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.848+0000 7fad757fa700 10 client.41723.objecter ms_dispatch 0x7fad78001550 osd_map(576..576 src has 482..576) v4 2024-02-23T00:51:45.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.848+0000 7fad757fa700 3 client.41723.objecter handle_osd_map got epochs [576,576] > 0 2024-02-23T00:51:45.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.848+0000 7fad757fa700 3 client.41723.objecter handle_osd_map decoding full epoch 576 2024-02-23T00:51:45.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.848+0000 7fad757fa700 20 client.41723.objecter dump_active .. 0 homeless 2024-02-23T00:51:45.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.851+0000 7fad77fff700 1 --2- 172.21.15.47:0/1516800775 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fad60062cc0 0x7fad60065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:45.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.851+0000 7fad757fa700 10 client.41723.objecter ms_handle_connect 0x7fad60062cc0 2024-02-23T00:51:45.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.851+0000 7fad77fff700 1 --2- 172.21.15.47:0/1516800775 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fad60062cc0 0x7fad60065180 secure :-1 s=READY pgs=3130 cs=0 l=1 rev1=1 rx=0x7fad68004e20 tx=0x7fad68009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:45.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:45.853+0000 7fad757fa700 1 -- 172.21.15.47:0/1516800775 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fad7004d490 con 0x7fad780fe510 2024-02-23T00:51:45.859 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:51:45.860 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:51:45.862 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:45.862 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:51:45.958 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:51:45.963 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:51:46.029 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:46.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:46.100+0000 7fad7e480700 1 -- 172.21.15.47:0/1516800775 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fad78109750 con 0x7fad780fe510 2024-02-23T00:51:46.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:46.101+0000 7fad757fa700 1 -- 172.21.15.47:0/1516800775 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+2397 (secure 0 0 0) 0x7fad78109750 con 0x7fad780fe510 2024-02-23T00:51:46.104 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:51:46.104 INFO:teuthology.orchestra.run.smithi047.stdout:{"fsid":"fce2617e-d1e2-11ee-95c0-87774f69a715","health":{"status":"HEALTH_WARN","checks":{"POOL_APP_NOT_ENABLED":{"severity":"HEALTH_WARN","summary":{"message":"2 pool(s) do not have an application enabled","count":2},"muted":false}},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":991,"monmap":{"epoch":3,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":576,"num_osds":8,"num_up_osds":8,"osd_up_since":1708648745,"num_in_osds":7,"osd_in_since":1708649247,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":216}],"num_pgs":216,"num_pools":10,"num_objects":228,"data_bytes":5817,"bytes_used":2223329280,"bytes_avail":669741920256,"bytes_total":671965249536,"read_bytes_sec":1279,"write_bytes_sec":1279,"read_op_per_sec":1,"write_op_per_sec":1,"flush_bytes_sec":511,"promote_op_per_sec":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"services":{"dashboard":"https://172.21.15.47:8443/","prometheus":"http://172.21.15.47:9283/"}},"servicemap":{"epoch":23,"modified":"2024-02-23T00:43:27.194339+0000","services":{"rgw":{"daemons":{"summary":"","14460":{"start_epoch":12,"start_stamp":"2024-02-23T00:39:21.905359+0000","gid":14460,"addr":"172.21.15.47:0/1573276039","metadata":{"arch":"x86_64","ceph_release":"pacific","ceph_version":"ceph version 16.2.14-550-geb66ed92 (eb66ed921e744301e6863be6353618d63967ea59) pacific (stable)","ceph_version_short":"16.2.14-550-geb66ed92","container_hostname":"smithi047","container_image":"quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:44277403522cf37edeee63b04643fd6c6e9416f13c3731a771e3cc8e53b7d010","cpu":"Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz","distro":"centos","distro_description":"CentOS Stream 8","distro_version":"8","frontend_config#0":"beast port=80","frontend_type#0":"beast","hostname":"smithi047","id":"foo.a","kernel_description":"#1 SMP Fri Feb 9 14:58:00 UTC 2024","kernel_version":"4.18.0-540.el8.x86_64","mem_swap_kb":"0","mem_total_kb":"32543848","num_handles":"1","os":"Linux","pid":"7","realm_id":"","realm_name":"","zone_id":"52970d38-d172-4ec0-ad23-3afdfeba31e4","zone_name":"default","zonegroup_id":"343d5407-1492-440c-b380-09e4750e9af7","zonegroup_name":"default"},"task_status":{}}}}}},"progress_events":{}} 2024-02-23T00:51:46.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:46.103+0000 7fad7e480700 10 librados: watch_flush enter 2024-02-23T00:51:46.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:46.103+0000 7fad7e480700 10 librados: watch_flush exit 2024-02-23T00:51:46.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:46.103+0000 7fad7e480700 20 client.41723.objecter shutdown clearing up homeless session... 2024-02-23T00:51:46.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:46.103+0000 7fad7e480700 10 client.41723.objecter successfully canceled tick 2024-02-23T00:51:46.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:46.103+0000 7fad7e480700 1 -- 172.21.15.47:0/1516800775 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fad60062cc0 msgr2=0x7fad60065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:46.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:46.103+0000 7fad7e480700 1 --2- 172.21.15.47:0/1516800775 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fad60062cc0 0x7fad60065180 secure :-1 s=READY pgs=3130 cs=0 l=1 rev1=1 rx=0x7fad68004e20 tx=0x7fad68009cb0).stop 2024-02-23T00:51:46.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:46.103+0000 7fad7e480700 1 -- 172.21.15.47:0/1516800775 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fad780fe510 msgr2=0x7fad781081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:46.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:46.103+0000 7fad7e480700 1 --2- 172.21.15.47:0/1516800775 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fad780fe510 0x7fad781081f0 secure :-1 s=READY pgs=2741 cs=0 l=1 rev1=1 rx=0x7fad70001df0 tx=0x7fad70004d10).stop 2024-02-23T00:51:46.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:46.103+0000 7fad7e480700 1 -- 172.21.15.47:0/1516800775 shutdown_connections 2024-02-23T00:51:46.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:46.103+0000 7fad7e480700 1 --2- 172.21.15.47:0/1516800775 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fad60062cc0 0x7fad60065180 unknown :-1 s=CLOSED pgs=3130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:46.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:46.103+0000 7fad7e480700 1 --2- 172.21.15.47:0/1516800775 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fad780fe510 0x7fad781081f0 unknown :-1 s=CLOSED pgs=2741 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:46.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:46.103+0000 7fad7e480700 1 --2- 172.21.15.47:0/1516800775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad780f8420 0x7fad7810dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:46.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:46.103+0000 7fad7e480700 1 --2- 172.21.15.47:0/1516800775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad780fdbb0 0x7fad7810e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:46.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:46.103+0000 7fad7e480700 1 -- 172.21.15.47:0/1516800775 >> 172.21.15.47:0/1516800775 conn(0x7fad780f3800 msgr2=0x7fad780fb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:46.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:46.103+0000 7fad7e480700 1 -- 172.21.15.47:0/1516800775 shutdown_connections 2024-02-23T00:51:46.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:46.103+0000 7fad7e480700 1 -- 172.21.15.47:0/1516800775 wait complete. 2024-02-23T00:51:46.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:46.103+0000 7fad7e480700 1 librados: shutdown 2024-02-23T00:51:46.154 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:51:46.158 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:46.158 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:51:46.259 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:51:46.287 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:46.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:45.346480+0000 mgr.y (mgr.14517) 520 : cluster [DBG] pgmap v868: 152 pgs: 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:46.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:45.694254+0000 osd.3 (osd.3) 274 : cluster [DBG] 5.17 scrub starts 2024-02-23T00:51:46.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:45.694969+0000 osd.3 (osd.3) 275 : cluster [DBG] 5.17 scrub ok 2024-02-23T00:51:46.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:46.102137+0000 mon.c (mon.2) 329 : audit [DBG] from='client.? 172.21.15.47:0/1516800775' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-23T00:51:46.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:46.256344+0000 mon.a (mon.0) 3041 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi047-82465-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:51:46.561 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:46.256440+0000 mon.a (mon.0) 3042 : cluster [DBG] osdmap e577: 8 total, 8 up, 7 in 2024-02-23T00:51:46.561 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:46.257839+0000 mon.a (mon.0) 3043 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi047-82465-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi047-82465-99"}]: dispatch 2024-02-23T00:51:46.561 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:45.346480+0000 mgr.y (mgr.14517) 520 : cluster [DBG] pgmap v868: 152 pgs: 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:46.561 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:45.694254+0000 osd.3 (osd.3) 274 : cluster [DBG] 5.17 scrub starts 2024-02-23T00:51:46.562 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:45.694969+0000 osd.3 (osd.3) 275 : cluster [DBG] 5.17 scrub ok 2024-02-23T00:51:46.562 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:46.102137+0000 mon.c (mon.2) 329 : audit [DBG] from='client.? 172.21.15.47:0/1516800775' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-23T00:51:46.562 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:46.256344+0000 mon.a (mon.0) 3041 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi047-82465-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:51:46.562 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:46.256440+0000 mon.a (mon.0) 3042 : cluster [DBG] osdmap e577: 8 total, 8 up, 7 in 2024-02-23T00:51:46.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:46.257839+0000 mon.a (mon.0) 3043 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi047-82465-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi047-82465-99"}]: dispatch 2024-02-23T00:51:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:45.346480+0000 mgr.y (mgr.14517) 520 : cluster [DBG] pgmap v868: 152 pgs: 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:45.694254+0000 osd.3 (osd.3) 274 : cluster [DBG] 5.17 scrub starts 2024-02-23T00:51:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:45.694969+0000 osd.3 (osd.3) 275 : cluster [DBG] 5.17 scrub ok 2024-02-23T00:51:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:46.102137+0000 mon.c (mon.2) 329 : audit [DBG] from='client.? 172.21.15.47:0/1516800775' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-23T00:51:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:46.256344+0000 mon.a (mon.0) 3041 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi047-82465-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:51:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:46.256440+0000 mon.a (mon.0) 3042 : cluster [DBG] osdmap e577: 8 total, 8 up, 7 in 2024-02-23T00:51:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:46.257839+0000 mon.a (mon.0) 3043 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi047-82465-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi047-82465-99"}]: dispatch 2024-02-23T00:51:47.287 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:51:47.288 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:51:47.853 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:46.334151+0000 osd.7 (osd.7) 281 : cluster [DBG] 18.1f scrub starts 2024-02-23T00:51:47.854 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:46.336714+0000 osd.7 (osd.7) 282 : cluster [DBG] 18.1f scrub ok 2024-02-23T00:51:47.854 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:46.660266+0000 osd.0 (osd.0) 245 : cluster [DBG] 4.b scrub starts 2024-02-23T00:51:47.854 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:46.661875+0000 osd.0 (osd.0) 246 : cluster [DBG] 4.b scrub ok 2024-02-23T00:51:47.854 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:47.689848+0000 mon.a (mon.0) 3044 : cluster [DBG] osdmap e578: 8 total, 8 up, 7 in 2024-02-23T00:51:47.873 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.7 dump_blocked_ops 2024-02-23T00:51:47.903 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:46.334151+0000 osd.7 (osd.7) 281 : cluster [DBG] 18.1f scrub starts 2024-02-23T00:51:47.903 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:46.336714+0000 osd.7 (osd.7) 282 : cluster [DBG] 18.1f scrub ok 2024-02-23T00:51:47.903 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:46.660266+0000 osd.0 (osd.0) 245 : cluster [DBG] 4.b scrub starts 2024-02-23T00:51:47.904 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:46.661875+0000 osd.0 (osd.0) 246 : cluster [DBG] 4.b scrub ok 2024-02-23T00:51:47.904 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:47.689848+0000 mon.a (mon.0) 3044 : cluster [DBG] osdmap e578: 8 total, 8 up, 7 in 2024-02-23T00:51:47.904 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:46.334151+0000 osd.7 (osd.7) 281 : cluster [DBG] 18.1f scrub starts 2024-02-23T00:51:47.904 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:46.336714+0000 osd.7 (osd.7) 282 : cluster [DBG] 18.1f scrub ok 2024-02-23T00:51:47.904 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:46.660266+0000 osd.0 (osd.0) 245 : cluster [DBG] 4.b scrub starts 2024-02-23T00:51:47.904 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:46.661875+0000 osd.0 (osd.0) 246 : cluster [DBG] 4.b scrub ok 2024-02-23T00:51:47.905 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:47.689848+0000 mon.a (mon.0) 3044 : cluster [DBG] osdmap e578: 8 total, 8 up, 7 in 2024-02-23T00:51:47.905 INFO:tasks.cephadm.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-02-23T00:51:47.906 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph pg dump --format=json 2024-02-23T00:51:47.937 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:51:47.940 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:47.941 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:51:48.041 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:51:48.070 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:48.091 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:51:48.094 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:48.094 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:51:48.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.116+0000 7f5a53ca3700 1 -- 172.21.15.47:0/2434093398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a4c0fd510 msgr2=0x7f5a4c0f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:48.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.116+0000 7f5a53ca3700 1 --2- 172.21.15.47:0/2434093398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a4c0fd510 0x7f5a4c0f6d40 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 rx=0x7f5a3c004870 tx=0x7f5a3c009cb0).stop 2024-02-23T00:51:48.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.117+0000 7f5a53ca3700 1 -- 172.21.15.47:0/2434093398 shutdown_connections 2024-02-23T00:51:48.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.117+0000 7f5a53ca3700 1 --2- 172.21.15.47:0/2434093398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a4c0f7400 0x7f5a4c103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:48.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.117+0000 7f5a53ca3700 1 --2- 172.21.15.47:0/2434093398 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a4c0fcbb0 0x7f5a4c0fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:48.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.117+0000 7f5a53ca3700 1 --2- 172.21.15.47:0/2434093398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a4c0fd510 0x7f5a4c0f6d40 unknown :-1 s=CLOSED pgs=2584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:48.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.117+0000 7f5a53ca3700 1 -- 172.21.15.47:0/2434093398 >> 172.21.15.47:0/2434093398 conn(0x7f5a4c0f2820 msgr2=0x7f5a4c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:48.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.117+0000 7f5a53ca3700 1 -- 172.21.15.47:0/2434093398 shutdown_connections 2024-02-23T00:51:48.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.117+0000 7f5a53ca3700 1 -- 172.21.15.47:0/2434093398 wait complete. 2024-02-23T00:51:48.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.117+0000 7f5a53ca3700 1 librados: starting msgr at 2024-02-23T00:51:48.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.117+0000 7f5a53ca3700 1 librados: starting objecter 2024-02-23T00:51:48.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.118+0000 7f5a53ca3700 1 Processor -- start 2024-02-23T00:51:48.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.118+0000 7f5a53ca3700 1 -- start start 2024-02-23T00:51:48.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.118+0000 7f5a53ca3700 1 librados: setting wanted keys 2024-02-23T00:51:48.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.118+0000 7f5a53ca3700 1 librados: calling monclient init 2024-02-23T00:51:48.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.118+0000 7f5a53ca3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a4c0f7400 0x7f5a4c105fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:48.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.118+0000 7f5a53ca3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a4c0fcbb0 0x7f5a4c1064b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:48.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.118+0000 7f5a53ca3700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a4c0fd510 0x7f5a4c1069a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:48.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.118+0000 7f5a53ca3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c10a090 con 0x7f5a4c0f7400 2024-02-23T00:51:48.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.118+0000 7f5a53ca3700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c11aad0 con 0x7f5a4c0fcbb0 2024-02-23T00:51:48.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.118+0000 7f5a53ca3700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c11ac90 con 0x7f5a4c0fd510 2024-02-23T00:51:48.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.118+0000 7f5a51a3f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a4c0f7400 0x7f5a4c105fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:48.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.118+0000 7f5a51a3f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a4c0f7400 0x7f5a4c105fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39962/0 (socket says 172.21.15.47:39962) 2024-02-23T00:51:48.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.118+0000 7f5a51a3f700 1 -- 172.21.15.47:0/1963902615 learned_addr learned my addr 172.21.15.47:0/1963902615 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:51:48.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.118+0000 7f5a5123e700 1 --2- 172.21.15.47:0/1963902615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a4c0fcbb0 0x7f5a4c1064b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:48.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.118+0000 7f5a52240700 1 --2- 172.21.15.47:0/1963902615 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a4c0fd510 0x7f5a4c1069a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:48.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.119+0000 7f5a51a3f700 1 -- 172.21.15.47:0/1963902615 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a4c0fd510 msgr2=0x7f5a4c1069a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:48.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.119+0000 7f5a51a3f700 1 --2- 172.21.15.47:0/1963902615 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a4c0fd510 0x7f5a4c1069a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:48.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.119+0000 7f5a51a3f700 1 -- 172.21.15.47:0/1963902615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a4c0fcbb0 msgr2=0x7f5a4c1064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:48.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.119+0000 7f5a51a3f700 1 --2- 172.21.15.47:0/1963902615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a4c0fcbb0 0x7f5a4c1064b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:48.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.119+0000 7f5a51a3f700 1 -- 172.21.15.47:0/1963902615 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3c005040 con 0x7f5a4c0f7400 2024-02-23T00:51:48.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.119+0000 7f5a5123e700 1 --2- 172.21.15.47:0/1963902615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a4c0fcbb0 0x7f5a4c1064b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:51:48.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.119+0000 7f5a52240700 1 --2- 172.21.15.47:0/1963902615 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a4c0fd510 0x7f5a4c1069a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:51:48.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.119+0000 7f5a51a3f700 1 --2- 172.21.15.47:0/1963902615 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a4c0f7400 0x7f5a4c105fc0 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 rx=0x7f5a4800e990 tx=0x7f5a4800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:48.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.119+0000 7f5a42ffd700 10 client.?.objecter ms_handle_connect 0x7f5a4c0f7400 2024-02-23T00:51:48.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.119+0000 7f5a42ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:51:48.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.119+0000 7f5a42ffd700 1 -- 172.21.15.47:0/1963902615 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a48004700 con 0x7f5a4c0f7400 2024-02-23T00:51:48.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.119+0000 7f5a42ffd700 1 -- 172.21.15.47:0/1963902615 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f5a4800f040 con 0x7f5a4c0f7400 2024-02-23T00:51:48.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.119+0000 7f5a53ca3700 1 -- 172.21.15.47:0/1963902615 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a4c11af10 con 0x7f5a4c0f7400 2024-02-23T00:51:48.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.119+0000 7f5a42ffd700 1 -- 172.21.15.47:0/1963902615 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a480146f0 con 0x7f5a4c0f7400 2024-02-23T00:51:48.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.119+0000 7f5a53ca3700 10 client.31410.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:51:48.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.119+0000 7f5a53ca3700 1 -- 172.21.15.47:0/1963902615 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a4c11b2c0 con 0x7f5a4c0f7400 2024-02-23T00:51:48.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.119+0000 7f5a53ca3700 1 librados: init done 2024-02-23T00:51:48.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.120+0000 7f5a53ca3700 1 -- 172.21.15.47:0/1963902615 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a4c05fa60 con 0x7f5a4c0f7400 2024-02-23T00:51:48.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.121+0000 7f5a42ffd700 1 -- 172.21.15.47:0/1963902615 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f5a48007b40 con 0x7f5a4c0f7400 2024-02-23T00:51:48.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.121+0000 7f5a42ffd700 1 --2- 172.21.15.47:0/1963902615 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5a38062cc0 0x7f5a38065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:48.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.121+0000 7f5a42ffd700 1 -- 172.21.15.47:0/1963902615 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(578..578 src has 482..578) v4 ==== 9600+0+0 (secure 0 0 0) 0x7f5a4807d2b0 con 0x7f5a4c0f7400 2024-02-23T00:51:48.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.121+0000 7f5a42ffd700 10 client.31410.objecter ms_dispatch 0x7f5a4c001550 osd_map(578..578 src has 482..578) v4 2024-02-23T00:51:48.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.121+0000 7f5a42ffd700 3 client.31410.objecter handle_osd_map got epochs [578,578] > 0 2024-02-23T00:51:48.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.121+0000 7f5a42ffd700 3 client.31410.objecter handle_osd_map decoding full epoch 578 2024-02-23T00:51:48.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.122+0000 7f5a5123e700 1 --2- 172.21.15.47:0/1963902615 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5a38062cc0 0x7f5a38065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:48.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.122+0000 7f5a42ffd700 20 client.31410.objecter dump_active .. 0 homeless 2024-02-23T00:51:48.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.122+0000 7f5a42ffd700 10 client.31410.objecter ms_handle_connect 0x7f5a38062cc0 2024-02-23T00:51:48.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.122+0000 7f5a5123e700 1 --2- 172.21.15.47:0/1963902615 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5a38062cc0 0x7f5a38065180 secure :-1 s=READY pgs=3131 cs=0 l=1 rev1=1 rx=0x7f5a3c004ee0 tx=0x7f5a3c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:48.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.127+0000 7f5a42ffd700 1 -- 172.21.15.47:0/1963902615 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f5a48049f60 con 0x7f5a4c0f7400 2024-02-23T00:51:48.155 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:51:48.188 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:51:48.196 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:51:48.229 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:48.353+0000 7f5a53ca3700 1 -- 172.21.15.47:0/1963902615 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7f5a4c05ee80 con 0x7f5a4c0f7400 2024-02-23T00:51:48.407 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:51:48.412 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:48.413 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:51:48.444 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:51:48.514 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:51:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:48.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:46.679103+0000 osd.3 (osd.3) 276 : cluster [DBG] 6.6 scrub starts 2024-02-23T00:51:48.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:46.679946+0000 osd.3 (osd.3) 277 : cluster [DBG] 6.6 scrub ok 2024-02-23T00:51:48.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:47.346895+0000 mgr.y (mgr.14517) 521 : cluster [DBG] pgmap v871: 152 pgs: 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:48.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:47.684813+0000 osd.4 (osd.4) 241 : cluster [DBG] 5.18 scrub starts 2024-02-23T00:51:48.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:47.685914+0000 osd.4 (osd.4) 242 : cluster [DBG] 5.18 scrub ok 2024-02-23T00:51:48.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:47.688291+0000 osd.5 (osd.5) 252 : cluster [DBG] 5.16 scrub starts 2024-02-23T00:51:48.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:47.691284+0000 osd.5 (osd.5) 253 : cluster [DBG] 5.16 scrub ok 2024-02-23T00:51:48.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:48.262285+0000 mon.a (mon.0) 3045 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi047-82465-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi047-82465-99"}]': finished 2024-02-23T00:51:48.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:48.262363+0000 mon.a (mon.0) 3046 : cluster [DBG] osdmap e579: 8 total, 8 up, 7 in 2024-02-23T00:51:48.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:46.679103+0000 osd.3 (osd.3) 276 : cluster [DBG] 6.6 scrub starts 2024-02-23T00:51:48.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:46.679946+0000 osd.3 (osd.3) 277 : cluster [DBG] 6.6 scrub ok 2024-02-23T00:51:48.811 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:47.346895+0000 mgr.y (mgr.14517) 521 : cluster [DBG] pgmap v871: 152 pgs: 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:48.811 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:47.684813+0000 osd.4 (osd.4) 241 : cluster [DBG] 5.18 scrub starts 2024-02-23T00:51:48.811 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:47.685914+0000 osd.4 (osd.4) 242 : cluster [DBG] 5.18 scrub ok 2024-02-23T00:51:48.811 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:47.688291+0000 osd.5 (osd.5) 252 : cluster [DBG] 5.16 scrub starts 2024-02-23T00:51:48.811 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:47.691284+0000 osd.5 (osd.5) 253 : cluster [DBG] 5.16 scrub ok 2024-02-23T00:51:48.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:48.262285+0000 mon.a (mon.0) 3045 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi047-82465-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi047-82465-99"}]': finished 2024-02-23T00:51:48.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:48.262363+0000 mon.a (mon.0) 3046 : cluster [DBG] osdmap e579: 8 total, 8 up, 7 in 2024-02-23T00:51:48.812 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:51:48.813 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:48.814 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:51:48.915 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:51:48.943 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:48.980 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:51:48.982 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:46.679103+0000 osd.3 (osd.3) 276 : cluster [DBG] 6.6 scrub starts 2024-02-23T00:51:48.982 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:46.679946+0000 osd.3 (osd.3) 277 : cluster [DBG] 6.6 scrub ok 2024-02-23T00:51:48.982 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:47.346895+0000 mgr.y (mgr.14517) 521 : cluster [DBG] pgmap v871: 152 pgs: 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:48.983 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:47.684813+0000 osd.4 (osd.4) 241 : cluster [DBG] 5.18 scrub starts 2024-02-23T00:51:48.983 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:47.685914+0000 osd.4 (osd.4) 242 : cluster [DBG] 5.18 scrub ok 2024-02-23T00:51:48.983 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:47.688291+0000 osd.5 (osd.5) 252 : cluster [DBG] 5.16 scrub starts 2024-02-23T00:51:48.983 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:47.691284+0000 osd.5 (osd.5) 253 : cluster [DBG] 5.16 scrub ok 2024-02-23T00:51:48.983 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:48.262285+0000 mon.a (mon.0) 3045 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi047-82465-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi047-82465-99"}]': finished 2024-02-23T00:51:48.983 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:48.262363+0000 mon.a (mon.0) 3046 : cluster [DBG] osdmap e579: 8 total, 8 up, 7 in 2024-02-23T00:51:48.984 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:48.984 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:51:49.084 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:51:49.112 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:49.266+0000 7f5a42ffd700 1 -- 172.21.15.47:0/1963902615 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v580) v1 ==== 87+0+0 (secure 0 0 0) 0x7f5a4804c750 con 0x7f5a4c0f7400 2024-02-23T00:51:49.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:49.270+0000 7f5a53ca3700 10 librados: watch_flush enter 2024-02-23T00:51:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:49.270+0000 7f5a53ca3700 10 librados: watch_flush exit 2024-02-23T00:51:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:49.270+0000 7f5a53ca3700 20 client.31410.objecter shutdown clearing up homeless session... 2024-02-23T00:51:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:49.270+0000 7f5a53ca3700 10 client.31410.objecter successfully canceled tick 2024-02-23T00:51:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:49.270+0000 7f5a53ca3700 1 -- 172.21.15.47:0/1963902615 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5a38062cc0 msgr2=0x7f5a38065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:49.270+0000 7f5a53ca3700 1 --2- 172.21.15.47:0/1963902615 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5a38062cc0 0x7f5a38065180 secure :-1 s=READY pgs=3131 cs=0 l=1 rev1=1 rx=0x7f5a3c004ee0 tx=0x7f5a3c00b040).stop 2024-02-23T00:51:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:49.270+0000 7f5a53ca3700 1 -- 172.21.15.47:0/1963902615 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a4c0f7400 msgr2=0x7f5a4c105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:49.270+0000 7f5a53ca3700 1 --2- 172.21.15.47:0/1963902615 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a4c0f7400 0x7f5a4c105fc0 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 rx=0x7f5a4800e990 tx=0x7f5a4800b040).stop 2024-02-23T00:51:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:49.270+0000 7f5a53ca3700 1 -- 172.21.15.47:0/1963902615 shutdown_connections 2024-02-23T00:51:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:49.270+0000 7f5a53ca3700 1 --2- 172.21.15.47:0/1963902615 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5a38062cc0 0x7f5a38065180 unknown :-1 s=CLOSED pgs=3131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:49.270+0000 7f5a53ca3700 1 --2- 172.21.15.47:0/1963902615 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a4c0fd510 0x7f5a4c1069a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:49.270+0000 7f5a53ca3700 1 --2- 172.21.15.47:0/1963902615 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a4c0f7400 0x7f5a4c105fc0 unknown :-1 s=CLOSED pgs=2585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:49.270+0000 7f5a53ca3700 1 --2- 172.21.15.47:0/1963902615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a4c0fcbb0 0x7f5a4c1064b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:49.271+0000 7f5a53ca3700 1 -- 172.21.15.47:0/1963902615 >> 172.21.15.47:0/1963902615 conn(0x7f5a4c0f2820 msgr2=0x7f5a4c0fa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:49.271+0000 7f5a53ca3700 1 -- 172.21.15.47:0/1963902615 shutdown_connections 2024-02-23T00:51:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:49.271+0000 7f5a53ca3700 1 -- 172.21.15.47:0/1963902615 wait complete. 2024-02-23T00:51:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:49.271+0000 7f5a53ca3700 1 librados: shutdown 2024-02-23T00:51:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is set 2024-02-23T00:51:49.461 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:51:49.464 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:49.464 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:51:49.565 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:51:49.595 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:49.649 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:51:49.651 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:49.652 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:51:49.753 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:51:49.783 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:49.881 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:51:49.883 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:49.884 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:51:49.985 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:51:50.004 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:51:50.005 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:51:50.015 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:48.536039+0000 osd.1 (osd.1) 245 : cluster [DBG] 4.f scrub starts 2024-02-23T00:51:50.016 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:48.537727+0000 osd.1 (osd.1) 246 : cluster [DBG] 4.f scrub ok 2024-02-23T00:51:50.016 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:48.724648+0000 mon.a (mon.0) 3047 : audit [INF] from='client.? 172.21.15.47:0/1963902615' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:51:50.016 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:49.260246+0000 mon.a (mon.0) 3048 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:51:50.016 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:49.265225+0000 mon.a (mon.0) 3049 : audit [INF] from='client.? 172.21.15.47:0/1963902615' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:51:50.016 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:49.265337+0000 mon.a (mon.0) 3050 : cluster [DBG] osdmap e580: 8 total, 8 up, 7 in 2024-02-23T00:51:50.016 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:49.268893+0000 mon.a (mon.0) 3051 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-99-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:51:50.016 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:49.347323+0000 mgr.y (mgr.14517) 522 : cluster [DBG] pgmap v874: 192 pgs: 40 unknown, 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:50.016 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:48.536039+0000 osd.1 (osd.1) 245 : cluster [DBG] 4.f scrub starts 2024-02-23T00:51:50.017 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:48.537727+0000 osd.1 (osd.1) 246 : cluster [DBG] 4.f scrub ok 2024-02-23T00:51:50.017 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:48.724648+0000 mon.a (mon.0) 3047 : audit [INF] from='client.? 172.21.15.47:0/1963902615' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:51:50.017 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:49.260246+0000 mon.a (mon.0) 3048 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:51:50.017 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:49.265225+0000 mon.a (mon.0) 3049 : audit [INF] from='client.? 172.21.15.47:0/1963902615' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:51:50.017 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:49.265337+0000 mon.a (mon.0) 3050 : cluster [DBG] osdmap e580: 8 total, 8 up, 7 in 2024-02-23T00:51:50.017 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:49.268893+0000 mon.a (mon.0) 3051 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-99-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:51:50.017 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:49.347323+0000 mgr.y (mgr.14517) 522 : cluster [DBG] pgmap v874: 192 pgs: 40 unknown, 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:50.017 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:50.025 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:48.536039+0000 osd.1 (osd.1) 245 : cluster [DBG] 4.f scrub starts 2024-02-23T00:51:50.025 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:48.537727+0000 osd.1 (osd.1) 246 : cluster [DBG] 4.f scrub ok 2024-02-23T00:51:50.025 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:48.724648+0000 mon.a (mon.0) 3047 : audit [INF] from='client.? 172.21.15.47:0/1963902615' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:51:50.025 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:49.260246+0000 mon.a (mon.0) 3048 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:51:50.026 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:49.265225+0000 mon.a (mon.0) 3049 : audit [INF] from='client.? 172.21.15.47:0/1963902615' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:51:50.026 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:49.265337+0000 mon.a (mon.0) 3050 : cluster [DBG] osdmap e580: 8 total, 8 up, 7 in 2024-02-23T00:51:50.026 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:49.268893+0000 mon.a (mon.0) 3051 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-99-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:51:50.026 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:49.347323+0000 mgr.y (mgr.14517) 522 : cluster [DBG] pgmap v874: 192 pgs: 40 unknown, 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:50.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.612+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/1402033246 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6d40f8420 msgr2=0x7fb6d4104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:50.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.612+0000 7fb6d8b9c700 1 --2- 172.21.15.47:0/1402033246 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6d40f8420 0x7fb6d4104a20 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 rx=0x7fb6cc009be0 tx=0x7fb6cc004d10).stop 2024-02-23T00:51:50.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.612+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/1402033246 shutdown_connections 2024-02-23T00:51:50.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.612+0000 7fb6d8b9c700 1 --2- 172.21.15.47:0/1402033246 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6d40f8420 0x7fb6d4104a20 unknown :-1 s=CLOSED pgs=2742 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:50.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.612+0000 7fb6d8b9c700 1 --2- 172.21.15.47:0/1402033246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6d40fdbb0 0x7fb6d40fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:50.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.612+0000 7fb6d8b9c700 1 --2- 172.21.15.47:0/1402033246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6d40fe510 0x7fb6d40f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:50.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.612+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/1402033246 >> 172.21.15.47:0/1402033246 conn(0x7fb6d40f3800 msgr2=0x7fb6d40f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:50.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.612+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/1402033246 shutdown_connections 2024-02-23T00:51:50.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.612+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/1402033246 wait complete. 2024-02-23T00:51:50.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.613+0000 7fb6d8b9c700 1 librados: starting msgr at 2024-02-23T00:51:50.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.613+0000 7fb6d8b9c700 1 librados: starting objecter 2024-02-23T00:51:50.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.613+0000 7fb6d8b9c700 1 Processor -- start 2024-02-23T00:51:50.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.613+0000 7fb6d8b9c700 1 -- start start 2024-02-23T00:51:50.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.613+0000 7fb6d8b9c700 1 librados: setting wanted keys 2024-02-23T00:51:50.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.613+0000 7fb6d8b9c700 1 librados: calling monclient init 2024-02-23T00:51:50.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.613+0000 7fb6d8b9c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6d40f8420 0x7fb6d406b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:50.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.613+0000 7fb6d8b9c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6d40fdbb0 0x7fb6d406b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:50.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.613+0000 7fb6d8b9c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6d40fe510 0x7fb6d406bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:50.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.613+0000 7fb6d8b9c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d4116120 con 0x7fb6d40fdbb0 2024-02-23T00:51:50.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.613+0000 7fb6d8b9c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d4116290 con 0x7fb6d40fe510 2024-02-23T00:51:50.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.613+0000 7fb6d8b9c700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d4116450 con 0x7fb6d40f8420 2024-02-23T00:51:50.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.613+0000 7fb6d259c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6d40f8420 0x7fb6d406b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:50.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.614+0000 7fb6d1d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6d40fdbb0 0x7fb6d406b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:50.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.614+0000 7fb6d259c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6d40f8420 0x7fb6d406b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:44370/0 (socket says 172.21.15.47:44370) 2024-02-23T00:51:50.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.614+0000 7fb6d259c700 1 -- 172.21.15.47:0/34198808 learned_addr learned my addr 172.21.15.47:0/34198808 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:51:50.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.614+0000 7fb6d2d9d700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6d40fe510 0x7fb6d406bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:50.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.614+0000 7fb6d259c700 1 -- 172.21.15.47:0/34198808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6d40fe510 msgr2=0x7fb6d406bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:50.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.614+0000 7fb6d259c700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6d40fe510 0x7fb6d406bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:50.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.614+0000 7fb6d259c700 1 -- 172.21.15.47:0/34198808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6d40fdbb0 msgr2=0x7fb6d406b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:50.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.614+0000 7fb6d259c700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6d40fdbb0 0x7fb6d406b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:50.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.614+0000 7fb6d259c700 1 -- 172.21.15.47:0/34198808 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6cc0097c0 con 0x7fb6d40f8420 2024-02-23T00:51:50.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.614+0000 7fb6d1d9b700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6d40fdbb0 0x7fb6d406b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:51:50.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.614+0000 7fb6d259c700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6d40f8420 0x7fb6d406b0d0 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 rx=0x7fb6c400e970 tx=0x7fb6c400b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:50.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.614+0000 7fb6cb7fe700 10 client.?.objecter ms_handle_connect 0x7fb6d40f8420 2024-02-23T00:51:50.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.614+0000 7fb6cb7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:51:50.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.614+0000 7fb6cb7fe700 1 -- 172.21.15.47:0/34198808 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c4004660 con 0x7fb6d40f8420 2024-02-23T00:51:50.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.614+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d4116730 con 0x7fb6d40f8420 2024-02-23T00:51:50.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.614+0000 7fb6cb7fe700 1 -- 172.21.15.47:0/34198808 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fb6c400f040 con 0x7fb6d40f8420 2024-02-23T00:51:50.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.614+0000 7fb6cb7fe700 1 -- 172.21.15.47:0/34198808 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c40146f0 con 0x7fb6d40f8420 2024-02-23T00:51:50.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.614+0000 7fb6d8b9c700 10 client.41735.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:51:50.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.614+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d4116cf0 con 0x7fb6d40f8420 2024-02-23T00:51:50.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.614+0000 7fb6d8b9c700 1 librados: init done 2024-02-23T00:51:50.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.615+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fb6d4106ed0 con 0x7fb6d40f8420 2024-02-23T00:51:50.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.615+0000 7fb6cb7fe700 1 -- 172.21.15.47:0/34198808 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fb6c40133c0 con 0x7fb6d40f8420 2024-02-23T00:51:50.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.616+0000 7fb6cb7fe700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb6c0062cc0 0x7fb6c0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:50.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.616+0000 7fb6cb7fe700 1 -- 172.21.15.47:0/34198808 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(581..581 src has 482..581) v4 ==== 10408+0+0 (secure 0 0 0) 0x7fb6c407d6a0 con 0x7fb6d40f8420 2024-02-23T00:51:50.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.616+0000 7fb6cb7fe700 10 client.41735.objecter ms_dispatch 0x7fb6d4001550 osd_map(581..581 src has 482..581) v4 2024-02-23T00:51:50.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.616+0000 7fb6cb7fe700 3 client.41735.objecter handle_osd_map got epochs [581,581] > 0 2024-02-23T00:51:50.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.616+0000 7fb6cb7fe700 3 client.41735.objecter handle_osd_map decoding full epoch 581 2024-02-23T00:51:50.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.616+0000 7fb6d1d9b700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb6c0062cc0 0x7fb6c0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:50.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.616+0000 7fb6cb7fe700 20 client.41735.objecter dump_active .. 0 homeless 2024-02-23T00:51:50.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.616+0000 7fb6cb7fe700 1 -- 172.21.15.47:0/34198808 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v581) v1 ==== 54+0+16 (secure 0 0 0) 0x7fb6c4084070 con 0x7fb6d40f8420 2024-02-23T00:51:50.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.616+0000 7fb6cb7fe700 10 client.41735.objecter ms_handle_connect 0x7fb6c0062cc0 2024-02-23T00:51:50.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.616+0000 7fb6d8b9c700 10 client.41735.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:50.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.616+0000 7fb6d1d9b700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb6c0062cc0 0x7fb6c0065180 secure :-1 s=READY pgs=3132 cs=0 l=1 rev1=1 rx=0x7fb6d406c870 tx=0x7fb6bc00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:50.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.616+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_assign -1 1 2024-02-23T00:51:50.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.616+0000 7fb6d8b9c700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fb6d40fee80 0x7fb6d40ff260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:50.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.616+0000 7fb6d8b9c700 20 client.41735.objecter _get_session s=0x7fb6d406c5b0 osd=0 3 2024-02-23T00:51:50.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.616+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d406c5b0 osd=0 3 2024-02-23T00:51:50.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.616+0000 7fb6d8b9c700 20 client.41735.objecter _get_session s=0x7fb6d406c5b0 osd=0 3 2024-02-23T00:51:50.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.616+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_remove -1 1 2024-02-23T00:51:50.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.616+0000 7fb6d8b9c700 20 client.41735.objecter get_session s=0x7fb6d406c5b0 osd=0 3 2024-02-23T00:51:50.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.616+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_assign 0 1 2024-02-23T00:51:50.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.616+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d406c5b0 osd=0 4 2024-02-23T00:51:50.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.616+0000 7fb6d8b9c700 10 client.41735.objecter _send_command 1 2024-02-23T00:51:50.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.616+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb6d40ff750 con 0x7fb6d40fee80 2024-02-23T00:51:50.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.617+0000 7fb6d2d9d700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fb6d40fee80 0x7fb6d40ff260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:50.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.617+0000 7fb6cb7fe700 10 client.41735.objecter ms_handle_connect 0x7fb6d40fee80 2024-02-23T00:51:50.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.617+0000 7fb6d2d9d700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fb6d40fee80 0x7fb6d40ff260 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:50.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.618+0000 7fb6cb7fe700 1 -- 172.21.15.47:0/34198808 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb6d40ff750 con 0x7fb6d40fee80 2024-02-23T00:51:50.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.618+0000 7fb6cb7fe700 10 client.41735.objecter ms_dispatch 0x7fb6d4001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:51:50.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.618+0000 7fb6cb7fe700 10 client.41735.objecter _finish_command 1 = system:0 2024-02-23T00:51:50.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.618+0000 7fb6cb7fe700 20 client.41735.objecter put_session s=0x7fb6d406c5b0 osd=0 4 2024-02-23T00:51:50.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.618+0000 7fb6cb7fe700 15 client.41735.objecter _session_command_op_remove 0 1 2024-02-23T00:51:50.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.632+0000 7fb6d8b9c700 10 client.41735.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:51:50.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.632+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_assign -1 2 2024-02-23T00:51:50.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.632+0000 7fb6d8b9c700 20 client.41735.objecter _get_session s=0x7fb6d406c5b0 osd=0 3 2024-02-23T00:51:50.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.632+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d406c5b0 osd=0 3 2024-02-23T00:51:50.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.632+0000 7fb6d8b9c700 20 client.41735.objecter _get_session s=0x7fb6d406c5b0 osd=0 3 2024-02-23T00:51:50.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.632+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_remove -1 2 2024-02-23T00:51:50.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.632+0000 7fb6d8b9c700 20 client.41735.objecter get_session s=0x7fb6d406c5b0 osd=0 3 2024-02-23T00:51:50.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.632+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_assign 0 2 2024-02-23T00:51:50.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.632+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d406c5b0 osd=0 4 2024-02-23T00:51:50.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.632+0000 7fb6d8b9c700 10 client.41735.objecter _send_command 2 2024-02-23T00:51:50.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.633+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fb6d40ff8c0 con 0x7fb6d40fee80 2024-02-23T00:51:50.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.634+0000 7fb6cb7fe700 1 -- 172.21.15.47:0/34198808 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fb6d40ff8c0 con 0x7fb6d40fee80 2024-02-23T00:51:50.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.634+0000 7fb6cb7fe700 10 client.41735.objecter ms_dispatch 0x7fb6d4001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:51:50.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.634+0000 7fb6cb7fe700 10 client.41735.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:51:50.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.634+0000 7fb6cb7fe700 20 client.41735.objecter put_session s=0x7fb6d406c5b0 osd=0 4 2024-02-23T00:51:50.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.634+0000 7fb6cb7fe700 15 client.41735.objecter _session_command_op_remove 0 2 2024-02-23T00:51:50.637 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:51:50.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.634+0000 7fb6d8b9c700 10 client.41735.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:50.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.634+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_assign -1 3 2024-02-23T00:51:50.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.634+0000 7fb6d8b9c700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fb6d4028540 0x7fb6d411af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:50.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.634+0000 7fb6d8b9c700 20 client.41735.objecter _get_session s=0x7fb6d40f5c80 osd=1 3 2024-02-23T00:51:50.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.634+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d40f5c80 osd=1 3 2024-02-23T00:51:50.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.634+0000 7fb6d8b9c700 20 client.41735.objecter _get_session s=0x7fb6d40f5c80 osd=1 3 2024-02-23T00:51:50.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.634+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_remove -1 3 2024-02-23T00:51:50.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.634+0000 7fb6d8b9c700 20 client.41735.objecter get_session s=0x7fb6d40f5c80 osd=1 3 2024-02-23T00:51:50.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.634+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_assign 1 3 2024-02-23T00:51:50.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.634+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d40f5c80 osd=1 4 2024-02-23T00:51:50.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.634+0000 7fb6d8b9c700 10 client.41735.objecter _send_command 3 2024-02-23T00:51:50.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.634+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb6d40f5df0 con 0x7fb6d4028540 2024-02-23T00:51:50.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.634+0000 7fb6d259c700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fb6d4028540 0x7fb6d411af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:50.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.635+0000 7fb6cb7fe700 10 client.41735.objecter ms_handle_connect 0x7fb6d4028540 2024-02-23T00:51:50.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.635+0000 7fb6d259c700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fb6d4028540 0x7fb6d411af90 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:50.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.635+0000 7fb6cb7fe700 1 -- 172.21.15.47:0/34198808 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb6d40f5df0 con 0x7fb6d4028540 2024-02-23T00:51:50.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.635+0000 7fb6cb7fe700 10 client.41735.objecter ms_dispatch 0x7fb6d4001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:51:50.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.635+0000 7fb6cb7fe700 10 client.41735.objecter _finish_command 3 = system:0 2024-02-23T00:51:50.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.635+0000 7fb6cb7fe700 20 client.41735.objecter put_session s=0x7fb6d40f5c80 osd=1 4 2024-02-23T00:51:50.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.635+0000 7fb6cb7fe700 15 client.41735.objecter _session_command_op_remove 1 3 2024-02-23T00:51:50.645 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.7 dump_historic_ops 2024-02-23T00:51:50.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.650+0000 7fb6d8b9c700 10 client.41735.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:51:50.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.650+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_assign -1 4 2024-02-23T00:51:50.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.650+0000 7fb6d8b9c700 20 client.41735.objecter _get_session s=0x7fb6d40f5c80 osd=1 3 2024-02-23T00:51:50.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.650+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d40f5c80 osd=1 3 2024-02-23T00:51:50.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.650+0000 7fb6d8b9c700 20 client.41735.objecter _get_session s=0x7fb6d40f5c80 osd=1 3 2024-02-23T00:51:50.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.650+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_remove -1 4 2024-02-23T00:51:50.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.650+0000 7fb6d8b9c700 20 client.41735.objecter get_session s=0x7fb6d40f5c80 osd=1 3 2024-02-23T00:51:50.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.650+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_assign 1 4 2024-02-23T00:51:50.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.650+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d40f5c80 osd=1 4 2024-02-23T00:51:50.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.650+0000 7fb6d8b9c700 10 client.41735.objecter _send_command 4 2024-02-23T00:51:50.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.650+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fb6d405a040 con 0x7fb6d4028540 2024-02-23T00:51:50.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.651+0000 7fb6cb7fe700 1 -- 172.21.15.47:0/34198808 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fb6d405a040 con 0x7fb6d4028540 2024-02-23T00:51:50.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.651+0000 7fb6cb7fe700 10 client.41735.objecter ms_dispatch 0x7fb6d4001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:51:50.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.651+0000 7fb6cb7fe700 10 client.41735.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:51:50.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.651+0000 7fb6cb7fe700 20 client.41735.objecter put_session s=0x7fb6d40f5c80 osd=1 4 2024-02-23T00:51:50.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.651+0000 7fb6cb7fe700 15 client.41735.objecter _session_command_op_remove 1 4 2024-02-23T00:51:50.654 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:51:50.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.651+0000 7fb6d8b9c700 10 client.41735.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:50.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.651+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_assign -1 5 2024-02-23T00:51:50.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.651+0000 7fb6d8b9c700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fb6d40594d0 0x7fb6d411e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:50.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.651+0000 7fb6d8b9c700 20 client.41735.objecter _get_session s=0x7fb6d4059360 osd=2 3 2024-02-23T00:51:50.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.651+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d4059360 osd=2 3 2024-02-23T00:51:50.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.651+0000 7fb6d8b9c700 20 client.41735.objecter _get_session s=0x7fb6d4059360 osd=2 3 2024-02-23T00:51:50.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.651+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_remove -1 5 2024-02-23T00:51:50.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.651+0000 7fb6d8b9c700 20 client.41735.objecter get_session s=0x7fb6d4059360 osd=2 3 2024-02-23T00:51:50.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.651+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_assign 2 5 2024-02-23T00:51:50.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.651+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d4059360 osd=2 4 2024-02-23T00:51:50.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.651+0000 7fb6d8b9c700 10 client.41735.objecter _send_command 5 2024-02-23T00:51:50.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.651+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb6d4028130 con 0x7fb6d40594d0 2024-02-23T00:51:50.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.652+0000 7fb6d1d9b700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fb6d40594d0 0x7fb6d411e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:50.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.652+0000 7fb6cb7fe700 10 client.41735.objecter ms_handle_connect 0x7fb6d40594d0 2024-02-23T00:51:50.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.652+0000 7fb6d1d9b700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fb6d40594d0 0x7fb6d411e610 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:50.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.652+0000 7fb6cb7fe700 1 -- 172.21.15.47:0/34198808 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb6d4028130 con 0x7fb6d40594d0 2024-02-23T00:51:50.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.652+0000 7fb6cb7fe700 10 client.41735.objecter ms_dispatch 0x7fb6d4001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:51:50.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.652+0000 7fb6cb7fe700 10 client.41735.objecter _finish_command 5 = system:0 2024-02-23T00:51:50.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.652+0000 7fb6cb7fe700 20 client.41735.objecter put_session s=0x7fb6d4059360 osd=2 4 2024-02-23T00:51:50.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.652+0000 7fb6cb7fe700 15 client.41735.objecter _session_command_op_remove 2 5 2024-02-23T00:51:50.665 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:51:50.668 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:50.668 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:51:50.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.667+0000 7fb6d8b9c700 10 client.41735.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:51:50.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.667+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_assign -1 6 2024-02-23T00:51:50.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.667+0000 7fb6d8b9c700 20 client.41735.objecter _get_session s=0x7fb6d4059360 osd=2 3 2024-02-23T00:51:50.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.667+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d4059360 osd=2 3 2024-02-23T00:51:50.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.667+0000 7fb6d8b9c700 20 client.41735.objecter _get_session s=0x7fb6d4059360 osd=2 3 2024-02-23T00:51:50.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.667+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_remove -1 6 2024-02-23T00:51:50.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.667+0000 7fb6d8b9c700 20 client.41735.objecter get_session s=0x7fb6d4059360 osd=2 3 2024-02-23T00:51:50.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.667+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_assign 2 6 2024-02-23T00:51:50.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.667+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d4059360 osd=2 4 2024-02-23T00:51:50.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.667+0000 7fb6d8b9c700 10 client.41735.objecter _send_command 6 2024-02-23T00:51:50.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.667+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fb6d411eb00 con 0x7fb6d40594d0 2024-02-23T00:51:50.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.669+0000 7fb6cb7fe700 1 -- 172.21.15.47:0/34198808 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fb6d411eb00 con 0x7fb6d40594d0 2024-02-23T00:51:50.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.669+0000 7fb6cb7fe700 10 client.41735.objecter ms_dispatch 0x7fb6d4001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:51:50.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.669+0000 7fb6cb7fe700 10 client.41735.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:51:50.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.669+0000 7fb6cb7fe700 20 client.41735.objecter put_session s=0x7fb6d4059360 osd=2 4 2024-02-23T00:51:50.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.669+0000 7fb6cb7fe700 15 client.41735.objecter _session_command_op_remove 2 6 2024-02-23T00:51:50.672 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:51:50.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.669+0000 7fb6d8b9c700 10 client.41735.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:50.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.669+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_assign -1 7 2024-02-23T00:51:50.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.669+0000 7fb6d8b9c700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fb6d411c160 0x7fb6d411fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:50.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.669+0000 7fb6d8b9c700 20 client.41735.objecter _get_session s=0x7fb6d4027600 osd=3 3 2024-02-23T00:51:50.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.669+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d4027600 osd=3 3 2024-02-23T00:51:50.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.669+0000 7fb6d8b9c700 20 client.41735.objecter _get_session s=0x7fb6d4027600 osd=3 3 2024-02-23T00:51:50.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.669+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_remove -1 7 2024-02-23T00:51:50.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.669+0000 7fb6d8b9c700 20 client.41735.objecter get_session s=0x7fb6d4027600 osd=3 3 2024-02-23T00:51:50.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.669+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_assign 3 7 2024-02-23T00:51:50.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.669+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d4027600 osd=3 4 2024-02-23T00:51:50.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.669+0000 7fb6d8b9c700 10 client.41735.objecter _send_command 7 2024-02-23T00:51:50.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.669+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb6d4027960 con 0x7fb6d411c160 2024-02-23T00:51:50.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.669+0000 7fb6d2d9d700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fb6d411c160 0x7fb6d411fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:50.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.669+0000 7fb6cb7fe700 10 client.41735.objecter ms_handle_connect 0x7fb6d411c160 2024-02-23T00:51:50.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.669+0000 7fb6d2d9d700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fb6d411c160 0x7fb6d411fe50 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:50.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.670+0000 7fb6cb7fe700 1 -- 172.21.15.47:0/34198808 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb6d4027960 con 0x7fb6d411c160 2024-02-23T00:51:50.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.670+0000 7fb6cb7fe700 10 client.41735.objecter ms_dispatch 0x7fb6d4001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:51:50.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.670+0000 7fb6cb7fe700 10 client.41735.objecter _finish_command 7 = system:0 2024-02-23T00:51:50.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.670+0000 7fb6cb7fe700 20 client.41735.objecter put_session s=0x7fb6d4027600 osd=3 4 2024-02-23T00:51:50.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.670+0000 7fb6cb7fe700 15 client.41735.objecter _session_command_op_remove 3 7 2024-02-23T00:51:50.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.688+0000 7fb6d8b9c700 10 client.41735.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:51:50.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.688+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_assign -1 8 2024-02-23T00:51:50.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.688+0000 7fb6d8b9c700 20 client.41735.objecter _get_session s=0x7fb6d4027600 osd=3 3 2024-02-23T00:51:50.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.688+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d4027600 osd=3 3 2024-02-23T00:51:50.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.688+0000 7fb6d8b9c700 20 client.41735.objecter _get_session s=0x7fb6d4027600 osd=3 3 2024-02-23T00:51:50.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.688+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_remove -1 8 2024-02-23T00:51:50.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.688+0000 7fb6d8b9c700 20 client.41735.objecter get_session s=0x7fb6d4027600 osd=3 3 2024-02-23T00:51:50.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.688+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_assign 3 8 2024-02-23T00:51:50.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.688+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d4027600 osd=3 4 2024-02-23T00:51:50.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.688+0000 7fb6d8b9c700 10 client.41735.objecter _send_command 8 2024-02-23T00:51:50.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.688+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fb6d411b6c0 con 0x7fb6d411c160 2024-02-23T00:51:50.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.689+0000 7fb6cb7fe700 1 -- 172.21.15.47:0/34198808 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fb6d411b6c0 con 0x7fb6d411c160 2024-02-23T00:51:50.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.689+0000 7fb6cb7fe700 10 client.41735.objecter ms_dispatch 0x7fb6d4001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:51:50.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.689+0000 7fb6cb7fe700 10 client.41735.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:51:50.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.689+0000 7fb6cb7fe700 20 client.41735.objecter put_session s=0x7fb6d4027600 osd=3 4 2024-02-23T00:51:50.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.689+0000 7fb6cb7fe700 15 client.41735.objecter _session_command_op_remove 3 8 2024-02-23T00:51:50.692 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:51:50.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.690+0000 7fb6d8b9c700 10 client.41735.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:50.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.690+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_assign -1 9 2024-02-23T00:51:50.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.690+0000 7fb6d8b9c700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fb6d4121020 0x7fb6d41254a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:50.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.690+0000 7fb6d8b9c700 20 client.41735.objecter _get_session s=0x7fb6d411b6c0 osd=4 3 2024-02-23T00:51:50.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.690+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d411b6c0 osd=4 3 2024-02-23T00:51:50.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.690+0000 7fb6d8b9c700 20 client.41735.objecter _get_session s=0x7fb6d411b6c0 osd=4 3 2024-02-23T00:51:50.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.690+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_remove -1 9 2024-02-23T00:51:50.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.690+0000 7fb6d8b9c700 20 client.41735.objecter get_session s=0x7fb6d411b6c0 osd=4 3 2024-02-23T00:51:50.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.690+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_assign 4 9 2024-02-23T00:51:50.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.690+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d411b6c0 osd=4 4 2024-02-23T00:51:50.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.690+0000 7fb6d8b9c700 10 client.41735.objecter _send_command 9 2024-02-23T00:51:50.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.690+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb6d411ba80 con 0x7fb6d4121020 2024-02-23T00:51:50.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.690+0000 7fb6d259c700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fb6d4121020 0x7fb6d41254a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:50.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.690+0000 7fb6cb7fe700 10 client.41735.objecter ms_handle_connect 0x7fb6d4121020 2024-02-23T00:51:50.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.690+0000 7fb6d259c700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fb6d4121020 0x7fb6d41254a0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:50.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.691+0000 7fb6cb7fe700 1 -- 172.21.15.47:0/34198808 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb6d411ba80 con 0x7fb6d4121020 2024-02-23T00:51:50.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.691+0000 7fb6cb7fe700 10 client.41735.objecter ms_dispatch 0x7fb6d4001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:51:50.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.691+0000 7fb6cb7fe700 10 client.41735.objecter _finish_command 9 = system:0 2024-02-23T00:51:50.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.691+0000 7fb6cb7fe700 20 client.41735.objecter put_session s=0x7fb6d411b6c0 osd=4 4 2024-02-23T00:51:50.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.691+0000 7fb6cb7fe700 15 client.41735.objecter _session_command_op_remove 4 9 2024-02-23T00:51:50.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.706+0000 7fb6d8b9c700 10 client.41735.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:51:50.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.706+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_assign -1 10 2024-02-23T00:51:50.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.706+0000 7fb6d8b9c700 20 client.41735.objecter _get_session s=0x7fb6d411b6c0 osd=4 3 2024-02-23T00:51:50.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.706+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d411b6c0 osd=4 3 2024-02-23T00:51:50.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.706+0000 7fb6d8b9c700 20 client.41735.objecter _get_session s=0x7fb6d411b6c0 osd=4 3 2024-02-23T00:51:50.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.706+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_remove -1 10 2024-02-23T00:51:50.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.706+0000 7fb6d8b9c700 20 client.41735.objecter get_session s=0x7fb6d411b6c0 osd=4 3 2024-02-23T00:51:50.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.706+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_assign 4 10 2024-02-23T00:51:50.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.706+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d411b6c0 osd=4 4 2024-02-23T00:51:50.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.706+0000 7fb6d8b9c700 10 client.41735.objecter _send_command 10 2024-02-23T00:51:50.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.706+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fb6d411ba80 con 0x7fb6d4121020 2024-02-23T00:51:50.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.708+0000 7fb6cb7fe700 1 -- 172.21.15.47:0/34198808 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fb6d411ba80 con 0x7fb6d4121020 2024-02-23T00:51:50.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.708+0000 7fb6cb7fe700 10 client.41735.objecter ms_dispatch 0x7fb6d4001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:51:50.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.708+0000 7fb6cb7fe700 10 client.41735.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:51:50.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.708+0000 7fb6cb7fe700 20 client.41735.objecter put_session s=0x7fb6d411b6c0 osd=4 4 2024-02-23T00:51:50.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.708+0000 7fb6cb7fe700 15 client.41735.objecter _session_command_op_remove 4 10 2024-02-23T00:51:50.711 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:51:50.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.708+0000 7fb6d8b9c700 10 client.41735.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:50.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.708+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_assign -1 11 2024-02-23T00:51:50.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.708+0000 7fb6d8b9c700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fb6d4125ef0 0x7fb6d4128340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:50.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.708+0000 7fb6d8b9c700 20 client.41735.objecter _get_session s=0x7fb6d411ba80 osd=5 3 2024-02-23T00:51:50.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.708+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d411ba80 osd=5 3 2024-02-23T00:51:50.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.708+0000 7fb6d8b9c700 20 client.41735.objecter _get_session s=0x7fb6d411ba80 osd=5 3 2024-02-23T00:51:50.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.708+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_remove -1 11 2024-02-23T00:51:50.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.708+0000 7fb6d8b9c700 20 client.41735.objecter get_session s=0x7fb6d411ba80 osd=5 3 2024-02-23T00:51:50.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.708+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_assign 5 11 2024-02-23T00:51:50.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.708+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d411ba80 osd=5 4 2024-02-23T00:51:50.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.708+0000 7fb6d8b9c700 10 client.41735.objecter _send_command 11 2024-02-23T00:51:50.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.708+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb6d412b1c0 con 0x7fb6d4125ef0 2024-02-23T00:51:50.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.708+0000 7fb6d1d9b700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fb6d4125ef0 0x7fb6d4128340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:50.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.709+0000 7fb6cb7fe700 10 client.41735.objecter ms_handle_connect 0x7fb6d4125ef0 2024-02-23T00:51:50.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.709+0000 7fb6d1d9b700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fb6d4125ef0 0x7fb6d4128340 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:50.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.709+0000 7fb6cb7fe700 1 -- 172.21.15.47:0/34198808 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb6d412b1c0 con 0x7fb6d4125ef0 2024-02-23T00:51:50.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.709+0000 7fb6cb7fe700 10 client.41735.objecter ms_dispatch 0x7fb6d4001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:51:50.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.709+0000 7fb6cb7fe700 10 client.41735.objecter _finish_command 11 = system:0 2024-02-23T00:51:50.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.709+0000 7fb6cb7fe700 20 client.41735.objecter put_session s=0x7fb6d411ba80 osd=5 4 2024-02-23T00:51:50.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.709+0000 7fb6cb7fe700 15 client.41735.objecter _session_command_op_remove 5 11 2024-02-23T00:51:50.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.727+0000 7fb6d8b9c700 10 client.41735.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:51:50.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.727+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_assign -1 12 2024-02-23T00:51:50.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.727+0000 7fb6d8b9c700 20 client.41735.objecter _get_session s=0x7fb6d411ba80 osd=5 3 2024-02-23T00:51:50.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.727+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d411ba80 osd=5 3 2024-02-23T00:51:50.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.727+0000 7fb6d8b9c700 20 client.41735.objecter _get_session s=0x7fb6d411ba80 osd=5 3 2024-02-23T00:51:50.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.727+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_remove -1 12 2024-02-23T00:51:50.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.727+0000 7fb6d8b9c700 20 client.41735.objecter get_session s=0x7fb6d411ba80 osd=5 3 2024-02-23T00:51:50.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.727+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_assign 5 12 2024-02-23T00:51:50.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.727+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d411ba80 osd=5 4 2024-02-23T00:51:50.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.727+0000 7fb6d8b9c700 10 client.41735.objecter _send_command 12 2024-02-23T00:51:50.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.727+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fb6d4129f50 con 0x7fb6d4125ef0 2024-02-23T00:51:50.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.728+0000 7fb6cb7fe700 1 -- 172.21.15.47:0/34198808 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fb6d4129f50 con 0x7fb6d4125ef0 2024-02-23T00:51:50.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.728+0000 7fb6cb7fe700 10 client.41735.objecter ms_dispatch 0x7fb6d4001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:51:50.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.728+0000 7fb6cb7fe700 10 client.41735.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:51:50.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.728+0000 7fb6cb7fe700 20 client.41735.objecter put_session s=0x7fb6d411ba80 osd=5 4 2024-02-23T00:51:50.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.728+0000 7fb6cb7fe700 15 client.41735.objecter _session_command_op_remove 5 12 2024-02-23T00:51:50.731 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:51:50.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.728+0000 7fb6d8b9c700 10 client.41735.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:50.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.729+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_assign -1 13 2024-02-23T00:51:50.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.729+0000 7fb6d8b9c700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fb6d412a690 0x7fb6d412aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:50.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.729+0000 7fb6d8b9c700 20 client.41735.objecter _get_session s=0x7fb6d4129f50 osd=6 3 2024-02-23T00:51:50.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.729+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d4129f50 osd=6 3 2024-02-23T00:51:50.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.729+0000 7fb6d8b9c700 20 client.41735.objecter _get_session s=0x7fb6d4129f50 osd=6 3 2024-02-23T00:51:50.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.729+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_remove -1 13 2024-02-23T00:51:50.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.729+0000 7fb6d8b9c700 20 client.41735.objecter get_session s=0x7fb6d4129f50 osd=6 3 2024-02-23T00:51:50.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.729+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_assign 6 13 2024-02-23T00:51:50.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.729+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d4129f50 osd=6 4 2024-02-23T00:51:50.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.729+0000 7fb6d8b9c700 10 client.41735.objecter _send_command 13 2024-02-23T00:51:50.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.729+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb6d412e430 con 0x7fb6d412a690 2024-02-23T00:51:50.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.729+0000 7fb6d2d9d700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fb6d412a690 0x7fb6d412aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:50.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.729+0000 7fb6cb7fe700 10 client.41735.objecter ms_handle_connect 0x7fb6d412a690 2024-02-23T00:51:50.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.729+0000 7fb6d2d9d700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fb6d412a690 0x7fb6d412aaf0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:50.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.730+0000 7fb6cb7fe700 1 -- 172.21.15.47:0/34198808 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb6d412e430 con 0x7fb6d412a690 2024-02-23T00:51:50.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.730+0000 7fb6cb7fe700 10 client.41735.objecter ms_dispatch 0x7fb6d4001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:51:50.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.730+0000 7fb6cb7fe700 10 client.41735.objecter _finish_command 13 = system:0 2024-02-23T00:51:50.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.730+0000 7fb6cb7fe700 20 client.41735.objecter put_session s=0x7fb6d4129f50 osd=6 4 2024-02-23T00:51:50.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.730+0000 7fb6cb7fe700 15 client.41735.objecter _session_command_op_remove 6 13 2024-02-23T00:51:50.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.744+0000 7fb6d8b9c700 10 client.41735.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:51:50.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.744+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_assign -1 14 2024-02-23T00:51:50.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.744+0000 7fb6d8b9c700 20 client.41735.objecter _get_session s=0x7fb6d4129f50 osd=6 3 2024-02-23T00:51:50.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.744+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d4129f50 osd=6 3 2024-02-23T00:51:50.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.744+0000 7fb6d8b9c700 20 client.41735.objecter _get_session s=0x7fb6d4129f50 osd=6 3 2024-02-23T00:51:50.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.744+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_remove -1 14 2024-02-23T00:51:50.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.744+0000 7fb6d8b9c700 20 client.41735.objecter get_session s=0x7fb6d4129f50 osd=6 3 2024-02-23T00:51:50.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.744+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_assign 6 14 2024-02-23T00:51:50.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.744+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d4129f50 osd=6 4 2024-02-23T00:51:50.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.744+0000 7fb6d8b9c700 10 client.41735.objecter _send_command 14 2024-02-23T00:51:50.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.744+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fb6d412e3f0 con 0x7fb6d412a690 2024-02-23T00:51:50.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.746+0000 7fb6cb7fe700 1 -- 172.21.15.47:0/34198808 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fb6d412e3f0 con 0x7fb6d412a690 2024-02-23T00:51:50.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.746+0000 7fb6cb7fe700 10 client.41735.objecter ms_dispatch 0x7fb6d4001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:51:50.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.746+0000 7fb6cb7fe700 10 client.41735.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:51:50.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.746+0000 7fb6cb7fe700 20 client.41735.objecter put_session s=0x7fb6d4129f50 osd=6 4 2024-02-23T00:51:50.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.746+0000 7fb6cb7fe700 15 client.41735.objecter _session_command_op_remove 6 14 2024-02-23T00:51:50.749 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:51:50.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.746+0000 7fb6d8b9c700 10 client.41735.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:51:50.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.746+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_assign -1 15 2024-02-23T00:51:50.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.746+0000 7fb6d8b9c700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fb6d412d6d0 0x7fb6d412db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:50.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.746+0000 7fb6d8b9c700 20 client.41735.objecter _get_session s=0x7fb6d412e3f0 osd=7 3 2024-02-23T00:51:50.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.746+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d412e3f0 osd=7 3 2024-02-23T00:51:50.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.746+0000 7fb6d8b9c700 20 client.41735.objecter _get_session s=0x7fb6d412e3f0 osd=7 3 2024-02-23T00:51:50.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.746+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_remove -1 15 2024-02-23T00:51:50.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.746+0000 7fb6d8b9c700 20 client.41735.objecter get_session s=0x7fb6d412e3f0 osd=7 3 2024-02-23T00:51:50.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.746+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_assign 7 15 2024-02-23T00:51:50.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.746+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d412e3f0 osd=7 4 2024-02-23T00:51:50.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.746+0000 7fb6d8b9c700 10 client.41735.objecter _send_command 15 2024-02-23T00:51:50.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.746+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb6d4131790 con 0x7fb6d412d6d0 2024-02-23T00:51:50.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.747+0000 7fb6d259c700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fb6d412d6d0 0x7fb6d412db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:50.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.747+0000 7fb6cb7fe700 10 client.41735.objecter ms_handle_connect 0x7fb6d412d6d0 2024-02-23T00:51:50.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.747+0000 7fb6d259c700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fb6d412d6d0 0x7fb6d412db50 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:50.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.748+0000 7fb6cb7fe700 1 -- 172.21.15.47:0/34198808 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb6d4131790 con 0x7fb6d412d6d0 2024-02-23T00:51:50.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.748+0000 7fb6cb7fe700 10 client.41735.objecter ms_dispatch 0x7fb6d4001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:51:50.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.748+0000 7fb6cb7fe700 10 client.41735.objecter _finish_command 15 = system:0 2024-02-23T00:51:50.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.748+0000 7fb6cb7fe700 20 client.41735.objecter put_session s=0x7fb6d412e3f0 osd=7 4 2024-02-23T00:51:50.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.748+0000 7fb6cb7fe700 15 client.41735.objecter _session_command_op_remove 7 15 2024-02-23T00:51:50.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.762+0000 7fb6d8b9c700 10 client.41735.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:51:50.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.762+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_assign -1 16 2024-02-23T00:51:50.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.762+0000 7fb6d8b9c700 20 client.41735.objecter _get_session s=0x7fb6d412e3f0 osd=7 3 2024-02-23T00:51:50.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.762+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d412e3f0 osd=7 3 2024-02-23T00:51:50.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.762+0000 7fb6d8b9c700 20 client.41735.objecter _get_session s=0x7fb6d412e3f0 osd=7 3 2024-02-23T00:51:50.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.762+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_remove -1 16 2024-02-23T00:51:50.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.762+0000 7fb6d8b9c700 20 client.41735.objecter get_session s=0x7fb6d412e3f0 osd=7 3 2024-02-23T00:51:50.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.762+0000 7fb6d8b9c700 15 client.41735.objecter _session_command_op_assign 7 16 2024-02-23T00:51:50.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.762+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d412e3f0 osd=7 4 2024-02-23T00:51:50.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.762+0000 7fb6d8b9c700 10 client.41735.objecter _send_command 16 2024-02-23T00:51:50.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.762+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fb6d4128940 con 0x7fb6d412d6d0 2024-02-23T00:51:50.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6cb7fe700 1 -- 172.21.15.47:0/34198808 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fb6d4128940 con 0x7fb6d412d6d0 2024-02-23T00:51:50.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6cb7fe700 10 client.41735.objecter ms_dispatch 0x7fb6d4001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:51:50.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6cb7fe700 10 client.41735.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:51:50.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6cb7fe700 20 client.41735.objecter put_session s=0x7fb6d412e3f0 osd=7 4 2024-02-23T00:51:50.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6cb7fe700 15 client.41735.objecter _session_command_op_remove 7 16 2024-02-23T00:51:50.767 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:51:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 10 librados: watch_flush enter 2024-02-23T00:51:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 10 librados: watch_flush exit 2024-02-23T00:51:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 10 client.41735.objecter close_session for osd.0 2024-02-23T00:51:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fb6d40fee80 msgr2=0x7fb6d40ff260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fb6d40fee80 0x7fb6d40ff260 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d406c5b0 osd=0 1 2024-02-23T00:51:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 10 client.41735.objecter close_session for osd.1 2024-02-23T00:51:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fb6d4028540 msgr2=0x7fb6d411af90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fb6d4028540 0x7fb6d411af90 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d40f5c80 osd=1 1 2024-02-23T00:51:50.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 10 client.41735.objecter close_session for osd.2 2024-02-23T00:51:50.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fb6d40594d0 msgr2=0x7fb6d411e610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:50.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fb6d40594d0 0x7fb6d411e610 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:50.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d4059360 osd=2 1 2024-02-23T00:51:50.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 10 client.41735.objecter close_session for osd.3 2024-02-23T00:51:50.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fb6d411c160 msgr2=0x7fb6d411fe50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:50.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fb6d411c160 0x7fb6d411fe50 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:50.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d2d9d700 1 -- 172.21.15.47:0/34198808 reap_dead start 2024-02-23T00:51:50.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d4027600 osd=3 1 2024-02-23T00:51:50.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 10 client.41735.objecter close_session for osd.4 2024-02-23T00:51:50.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fb6d4121020 msgr2=0x7fb6d41254a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:50.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fb6d4121020 0x7fb6d41254a0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:50.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d2d9d700 1 -- 172.21.15.47:0/34198808 reap_dead start 2024-02-23T00:51:50.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d411b6c0 osd=4 1 2024-02-23T00:51:50.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 10 client.41735.objecter close_session for osd.5 2024-02-23T00:51:50.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fb6d4125ef0 msgr2=0x7fb6d4128340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:50.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fb6d4125ef0 0x7fb6d4128340 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:50.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d411ba80 osd=5 1 2024-02-23T00:51:50.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 10 client.41735.objecter close_session for osd.6 2024-02-23T00:51:50.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fb6d412a690 msgr2=0x7fb6d412aaf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:50.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fb6d412a690 0x7fb6d412aaf0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:50.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d4129f50 osd=6 1 2024-02-23T00:51:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 10 client.41735.objecter close_session for osd.7 2024-02-23T00:51:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fb6d412d6d0 msgr2=0x7fb6d412db50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fb6d412d6d0 0x7fb6d412db50 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 20 client.41735.objecter put_session s=0x7fb6d412e3f0 osd=7 1 2024-02-23T00:51:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 20 client.41735.objecter shutdown clearing up homeless session... 2024-02-23T00:51:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 10 client.41735.objecter successfully canceled tick 2024-02-23T00:51:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb6c0062cc0 msgr2=0x7fb6c0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb6c0062cc0 0x7fb6c0065180 secure :-1 s=READY pgs=3132 cs=0 l=1 rev1=1 rx=0x7fb6d406c870 tx=0x7fb6bc00b040).stop 2024-02-23T00:51:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6d40f8420 msgr2=0x7fb6d406b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6d40f8420 0x7fb6d406b0d0 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 rx=0x7fb6c400e970 tx=0x7fb6c400b040).stop 2024-02-23T00:51:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d2d9d700 1 -- 172.21.15.47:0/34198808 reap_dead start 2024-02-23T00:51:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 shutdown_connections 2024-02-23T00:51:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.764+0000 7fb6d8b9c700 1 --2- 172.21.15.47:0/34198808 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6d40f8420 0x7fb6d406b0d0 unknown :-1 s=CLOSED pgs=2743 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.765+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 >> 172.21.15.47:0/34198808 conn(0x7fb6d40f3800 msgr2=0x7fb6d40f50f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.765+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 shutdown_connections 2024-02-23T00:51:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.765+0000 7fb6d8b9c700 1 -- 172.21.15.47:0/34198808 wait complete. 2024-02-23T00:51:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:50.765+0000 7fb6d8b9c700 1 librados: shutdown 2024-02-23T00:51:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:51:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:51:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:51:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:51:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:51:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:51:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:51:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:51:50.775 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:51:50.792 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:51:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:51:50] "GET /metrics HTTP/1.1" 200 192564 "" "Prometheus/2.33.4" 2024-02-23T00:51:50.806 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:50.913 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:51:51.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:50.268711+0000 mon.a (mon.0) 3052 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-99-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:51:51.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:50.269091+0000 mon.a (mon.0) 3053 : cluster [DBG] osdmap e581: 8 total, 8 up, 7 in 2024-02-23T00:51:51.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:50.277728+0000 mon.a (mon.0) 3054 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-99", "tierpool": "test-rados-api-smithi047-82465-99-cache"}]: dispatch 2024-02-23T00:51:51.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:50.617332+0000 mon.c (mon.2) 330 : audit [DBG] from='client.? 172.21.15.47:0/34198808' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:51:51.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:50.268711+0000 mon.a (mon.0) 3052 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-99-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:51:51.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:50.269091+0000 mon.a (mon.0) 3053 : cluster [DBG] osdmap e581: 8 total, 8 up, 7 in 2024-02-23T00:51:51.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:50.277728+0000 mon.a (mon.0) 3054 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-99", "tierpool": "test-rados-api-smithi047-82465-99-cache"}]: dispatch 2024-02-23T00:51:51.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:50.617332+0000 mon.c (mon.2) 330 : audit [DBG] from='client.? 172.21.15.47:0/34198808' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:51:51.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:50.268711+0000 mon.a (mon.0) 3052 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-99-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:51:51.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:50.269091+0000 mon.a (mon.0) 3053 : cluster [DBG] osdmap e581: 8 total, 8 up, 7 in 2024-02-23T00:51:51.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:50.277728+0000 mon.a (mon.0) 3054 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-99", "tierpool": "test-rados-api-smithi047-82465-99-cache"}]: dispatch 2024-02-23T00:51:51.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:50.617332+0000 mon.c (mon.2) 330 : audit [DBG] from='client.? 172.21.15.47:0/34198808' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:51:51.756 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:51:51.759 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:51.759 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:51:51.861 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:51:51.892 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:52.019 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:51:52.022 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:52.022 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:51:52.123 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:51:52.153 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:52.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.328+0000 7faef4a5a700 1 -- 172.21.15.47:0/1761334039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faef00fcbb0 msgr2=0x7faef00fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:52.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.328+0000 7faef4a5a700 1 --2- 172.21.15.47:0/1761334039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faef00fcbb0 0x7faef00fcf90 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 rx=0x7faee0004870 tx=0x7faee0009cb0).stop 2024-02-23T00:51:52.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.328+0000 7faef4a5a700 1 -- 172.21.15.47:0/1761334039 shutdown_connections 2024-02-23T00:51:52.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.328+0000 7faef4a5a700 1 --2- 172.21.15.47:0/1761334039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faef00f7400 0x7faef0103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:52.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.328+0000 7faef4a5a700 1 --2- 172.21.15.47:0/1761334039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faef00fcbb0 0x7faef00fcf90 unknown :-1 s=CLOSED pgs=2586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:52.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.328+0000 7faef4a5a700 1 --2- 172.21.15.47:0/1761334039 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faef00fd510 0x7faef00f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:52.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.328+0000 7faef4a5a700 1 -- 172.21.15.47:0/1761334039 >> 172.21.15.47:0/1761334039 conn(0x7faef00f2820 msgr2=0x7faef00f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:52.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.328+0000 7faef4a5a700 1 -- 172.21.15.47:0/1761334039 shutdown_connections 2024-02-23T00:51:52.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.328+0000 7faef4a5a700 1 -- 172.21.15.47:0/1761334039 wait complete. 2024-02-23T00:51:52.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.328+0000 7faef4a5a700 1 librados: starting msgr at 2024-02-23T00:51:52.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.328+0000 7faef4a5a700 1 librados: starting objecter 2024-02-23T00:51:52.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.328+0000 7faef4a5a700 1 Processor -- start 2024-02-23T00:51:52.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.328+0000 7faef4a5a700 1 -- start start 2024-02-23T00:51:52.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.328+0000 7faef4a5a700 1 librados: setting wanted keys 2024-02-23T00:51:52.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.328+0000 7faef4a5a700 1 librados: calling monclient init 2024-02-23T00:51:52.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.329+0000 7faef4a5a700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faef00f7400 0x7faef01101f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:52.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.329+0000 7faef4a5a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faef00fcbb0 0x7faef01106e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:52.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.329+0000 7faef4a5a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faef00fd510 0x7faef010a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:52.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.329+0000 7faef4a5a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef010a880 con 0x7faef00fcbb0 2024-02-23T00:51:52.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.329+0000 7faef4a5a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef010a9f0 con 0x7faef00fd510 2024-02-23T00:51:52.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.329+0000 7faef4a5a700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faef010abb0 con 0x7faef00f7400 2024-02-23T00:51:52.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.329+0000 7faeee59c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faef00f7400 0x7faef01101f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:52.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.329+0000 7faeedd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faef00fcbb0 0x7faef01106e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:52.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.329+0000 7faeee59c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faef00f7400 0x7faef01101f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:44386/0 (socket says 172.21.15.47:44386) 2024-02-23T00:51:52.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.329+0000 7faeee59c700 1 -- 172.21.15.47:0/1069537669 learned_addr learned my addr 172.21.15.47:0/1069537669 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:51:52.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.329+0000 7faeeed9d700 1 --2- 172.21.15.47:0/1069537669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faef00fd510 0x7faef010a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:52.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.329+0000 7faeee59c700 1 -- 172.21.15.47:0/1069537669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faef00fd510 msgr2=0x7faef010a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:52.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.329+0000 7faeee59c700 1 --2- 172.21.15.47:0/1069537669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faef00fd510 0x7faef010a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:52.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.329+0000 7faeee59c700 1 -- 172.21.15.47:0/1069537669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faef00fcbb0 msgr2=0x7faef01106e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:52.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.329+0000 7faeee59c700 1 --2- 172.21.15.47:0/1069537669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faef00fcbb0 0x7faef01106e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:52.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.329+0000 7faeee59c700 1 -- 172.21.15.47:0/1069537669 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee4005040 con 0x7faef00f7400 2024-02-23T00:51:52.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.329+0000 7faeedd9b700 1 --2- 172.21.15.47:0/1069537669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faef00fcbb0 0x7faef01106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:51:52.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.329+0000 7faeee59c700 1 --2- 172.21.15.47:0/1069537669 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faef00f7400 0x7faef01101f0 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 rx=0x7faee0009da0 tx=0x7faee0009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:52.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.329+0000 7faedf7fe700 10 client.?.objecter ms_handle_connect 0x7faef00f7400 2024-02-23T00:51:52.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.329+0000 7faedf7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:51:52.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.329+0000 7faedf7fe700 1 -- 172.21.15.47:0/1069537669 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee000e030 con 0x7faef00f7400 2024-02-23T00:51:52.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.329+0000 7faedf7fe700 1 -- 172.21.15.47:0/1069537669 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7faee0013d70 con 0x7faef00f7400 2024-02-23T00:51:52.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.329+0000 7faef4a5a700 1 -- 172.21.15.47:0/1069537669 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee0005040 con 0x7faef00f7400 2024-02-23T00:51:52.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.329+0000 7faedf7fe700 1 -- 172.21.15.47:0/1069537669 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee0018070 con 0x7faef00f7400 2024-02-23T00:51:52.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.329+0000 7faef4a5a700 10 client.41741.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:51:52.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.329+0000 7faef4a5a700 1 -- 172.21.15.47:0/1069537669 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef010b050 con 0x7faef00f7400 2024-02-23T00:51:52.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.329+0000 7faef4a5a700 1 librados: init done 2024-02-23T00:51:52.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.330+0000 7faef4a5a700 1 -- 172.21.15.47:0/1069537669 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faef010b240 con 0x7faef00f7400 2024-02-23T00:51:52.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.331+0000 7faedf7fe700 1 -- 172.21.15.47:0/1069537669 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7faee0009240 con 0x7faef00f7400 2024-02-23T00:51:52.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.331+0000 7faedf7fe700 1 --2- 172.21.15.47:0/1069537669 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7faed8062cc0 0x7faed8065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:52.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.331+0000 7faedf7fe700 1 -- 172.21.15.47:0/1069537669 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(583..583 src has 482..583) v4 ==== 10403+0+0 (secure 0 0 0) 0x7faee007ecb0 con 0x7faef00f7400 2024-02-23T00:51:52.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.331+0000 7faedf7fe700 10 client.41741.objecter ms_dispatch 0x7faef0001550 osd_map(583..583 src has 482..583) v4 2024-02-23T00:51:52.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.331+0000 7faedf7fe700 3 client.41741.objecter handle_osd_map got epochs [583,583] > 0 2024-02-23T00:51:52.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.331+0000 7faedf7fe700 3 client.41741.objecter handle_osd_map decoding full epoch 583 2024-02-23T00:51:52.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.331+0000 7faeedd9b700 1 --2- 172.21.15.47:0/1069537669 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7faed8062cc0 0x7faed8065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:52.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.331+0000 7faedf7fe700 20 client.41741.objecter dump_active .. 0 homeless 2024-02-23T00:51:52.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.332+0000 7faedf7fe700 10 client.41741.objecter ms_handle_connect 0x7faed8062cc0 2024-02-23T00:51:52.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.332+0000 7faeedd9b700 1 --2- 172.21.15.47:0/1069537669 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7faed8062cc0 0x7faed8065180 secure :-1 s=READY pgs=3133 cs=0 l=1 rev1=1 rx=0x7faef010bb30 tx=0x7faee400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:52.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.336+0000 7faedf7fe700 1 -- 172.21.15.47:0/1069537669 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7faee004de20 con 0x7faef00f7400 2024-02-23T00:51:52.357 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:51:52.360 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:52.360 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:51:52.383 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:51.270067+0000 mon.a (mon.0) 3055 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-99", "tierpool": "test-rados-api-smithi047-82465-99-cache"}]': finished 2024-02-23T00:51:52.383 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:51.270130+0000 mon.a (mon.0) 3056 : cluster [DBG] osdmap e582: 8 total, 8 up, 7 in 2024-02-23T00:51:52.383 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:51.272045+0000 mon.a (mon.0) 3057 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-99", "overlaypool": "test-rados-api-smithi047-82465-99-cache"}]: dispatch 2024-02-23T00:51:52.383 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:51.347994+0000 mgr.y (mgr.14517) 523 : cluster [DBG] pgmap v877: 192 pgs: 40 unknown, 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:51:52.383 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:52.271337+0000 mon.a (mon.0) 3058 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-99", "overlaypool": "test-rados-api-smithi047-82465-99-cache"}]': finished 2024-02-23T00:51:52.383 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:52.271400+0000 mon.a (mon.0) 3059 : cluster [DBG] osdmap e583: 8 total, 8 up, 7 in 2024-02-23T00:51:52.461 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:51:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:52.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.507+0000 7faef4a5a700 1 -- 172.21.15.47:0/1069537669 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faef0105f10 con 0x7faed8062cc0 2024-02-23T00:51:52.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.520+0000 7faedf7fe700 1 -- 172.21.15.47:0/1069537669 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+497295 (secure 0 0 0) 0x7faef0105f10 con 0x7faed8062cc0 2024-02-23T00:51:52.529 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:51:52.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.524+0000 7faef4a5a700 10 librados: watch_flush enter 2024-02-23T00:51:52.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.524+0000 7faef4a5a700 10 librados: watch_flush exit 2024-02-23T00:51:52.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.524+0000 7faef4a5a700 20 client.41741.objecter shutdown clearing up homeless session... 2024-02-23T00:51:52.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.524+0000 7faef4a5a700 10 client.41741.objecter successfully canceled tick 2024-02-23T00:51:52.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.524+0000 7faef4a5a700 1 -- 172.21.15.47:0/1069537669 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7faed8062cc0 msgr2=0x7faed8065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:52.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.524+0000 7faef4a5a700 1 --2- 172.21.15.47:0/1069537669 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7faed8062cc0 0x7faed8065180 secure :-1 s=READY pgs=3133 cs=0 l=1 rev1=1 rx=0x7faef010bb30 tx=0x7faee400b040).stop 2024-02-23T00:51:52.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.524+0000 7faef4a5a700 1 -- 172.21.15.47:0/1069537669 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faef00f7400 msgr2=0x7faef01101f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:52.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.524+0000 7faef4a5a700 1 --2- 172.21.15.47:0/1069537669 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faef00f7400 0x7faef01101f0 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 rx=0x7faee0009da0 tx=0x7faee0009cb0).stop 2024-02-23T00:51:52.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.524+0000 7faef4a5a700 1 -- 172.21.15.47:0/1069537669 shutdown_connections 2024-02-23T00:51:52.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.524+0000 7faef4a5a700 1 --2- 172.21.15.47:0/1069537669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faef00fd510 0x7faef010a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:52.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.524+0000 7faef4a5a700 1 --2- 172.21.15.47:0/1069537669 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7faed8062cc0 0x7faed8065180 unknown :-1 s=CLOSED pgs=3133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:52.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.524+0000 7faef4a5a700 1 --2- 172.21.15.47:0/1069537669 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faef00f7400 0x7faef01101f0 unknown :-1 s=CLOSED pgs=2744 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:52.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.524+0000 7faef4a5a700 1 --2- 172.21.15.47:0/1069537669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faef00fcbb0 0x7faef01106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:52.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.524+0000 7faef4a5a700 1 -- 172.21.15.47:0/1069537669 >> 172.21.15.47:0/1069537669 conn(0x7faef00f2820 msgr2=0x7faef00f42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:52.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.524+0000 7faef4a5a700 1 -- 172.21.15.47:0/1069537669 shutdown_connections 2024-02-23T00:51:52.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.524+0000 7faef4a5a700 1 -- 172.21.15.47:0/1069537669 wait complete. 2024-02-23T00:51:52.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:52.524+0000 7faef4a5a700 1 librados: shutdown 2024-02-23T00:51:52.535 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-02-23T00:51:52.547 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:51.270067+0000 mon.a (mon.0) 3055 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-99", "tierpool": "test-rados-api-smithi047-82465-99-cache"}]': finished 2024-02-23T00:51:52.547 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:51.270130+0000 mon.a (mon.0) 3056 : cluster [DBG] osdmap e582: 8 total, 8 up, 7 in 2024-02-23T00:51:52.547 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:51.272045+0000 mon.a (mon.0) 3057 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-99", "overlaypool": "test-rados-api-smithi047-82465-99-cache"}]: dispatch 2024-02-23T00:51:52.548 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:51.347994+0000 mgr.y (mgr.14517) 523 : cluster [DBG] pgmap v877: 192 pgs: 40 unknown, 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:51:52.548 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:52.271337+0000 mon.a (mon.0) 3058 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-99", "overlaypool": "test-rados-api-smithi047-82465-99-cache"}]': finished 2024-02-23T00:51:52.548 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:52.271400+0000 mon.a (mon.0) 3059 : cluster [DBG] osdmap e583: 8 total, 8 up, 7 in 2024-02-23T00:51:52.548 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:51.270067+0000 mon.a (mon.0) 3055 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-99", "tierpool": "test-rados-api-smithi047-82465-99-cache"}]': finished 2024-02-23T00:51:52.548 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:51.270130+0000 mon.a (mon.0) 3056 : cluster [DBG] osdmap e582: 8 total, 8 up, 7 in 2024-02-23T00:51:52.548 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:51.272045+0000 mon.a (mon.0) 3057 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-99", "overlaypool": "test-rados-api-smithi047-82465-99-cache"}]: dispatch 2024-02-23T00:51:52.548 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:51.347994+0000 mgr.y (mgr.14517) 523 : cluster [DBG] pgmap v877: 192 pgs: 40 unknown, 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:51:52.548 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:52.271337+0000 mon.a (mon.0) 3058 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-99", "overlaypool": "test-rados-api-smithi047-82465-99-cache"}]': finished 2024-02-23T00:51:52.548 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:52.271400+0000 mon.a (mon.0) 3059 : cluster [DBG] osdmap e583: 8 total, 8 up, 7 in 2024-02-23T00:51:52.636 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:51:52.636 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-02-23T00:51:52.636 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-02-23T00:51:52.636 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-02-23T00:51:52.636 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.636 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(osd.3.0:1295 20.1e e112/109 -22:7b6591d8:::temp_20.1e_0_14313_1:head v 0'0, mlcod=112'1002)", 2024-02-23T00:51:52.636 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:01.568631+0000", 2024-02-23T00:51:52.636 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 531.06305532199997, 2024-02-23T00:51:52.636 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034138363999999997, 2024-02-23T00:51:52.636 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:52.637 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:52.637 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:52.637 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.637 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:52.637 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.568631+0000", 2024-02-23T00:51:52.637 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:52.637 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.637 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.637 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:52.637 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.568631+0000", 2024-02-23T00:51:52.637 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999804 2024-02-23T00:51:52.637 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.637 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.638 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:52.638 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.568612+0000", 2024-02-23T00:51:52.638 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.020189048000000001 2024-02-23T00:51:52.638 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.638 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.638 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:52.638 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.588801+0000", 2024-02-23T00:51:52.638 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1197000000000001e-05 2024-02-23T00:51:52.638 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.638 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.638 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:52.638 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.588822+0000", 2024-02-23T00:51:52.638 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4791999999999999e-05 2024-02-23T00:51:52.638 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.639 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.639 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:52.639 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.588847+0000", 2024-02-23T00:51:52.639 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9911999999999998e-05 2024-02-23T00:51:52.639 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.639 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.639 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:52.639 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.588906+0000", 2024-02-23T00:51:52.639 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5961000000000001e-05 2024-02-23T00:51:52.639 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.639 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.639 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:52.639 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.588932+0000", 2024-02-23T00:51:52.640 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01379406 2024-02-23T00:51:52.640 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.640 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.640 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:52.640 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.602726+0000", 2024-02-23T00:51:52.640 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3027e-05 2024-02-23T00:51:52.640 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.640 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.640 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:52.640 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.602769+0000", 2024-02-23T00:51:52.640 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034138363999999997 2024-02-23T00:51:52.640 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.641 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:52.641 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.641 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.641 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.641 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.24775.0:2069 20.13 e112/109 20:cde74feb:test-rados-api-smithi047-82457-17::big:head v 112'1, mlcod=0'0)", 2024-02-23T00:51:52.641 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:01.712338+0000", 2024-02-23T00:51:52.641 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 530.91934823600002, 2024-02-23T00:51:52.641 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.057972005, 2024-02-23T00:51:52.641 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:52.641 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:52.641 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:52.641 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.641 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:52.642 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.712338+0000", 2024-02-23T00:51:52.642 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:52.642 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.642 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.642 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:52.642 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.712338+0000", 2024-02-23T00:51:52.642 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:51:52.642 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.642 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.642 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:52.642 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.712336+0000", 2024-02-23T00:51:52.642 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012853547 2024-02-23T00:51:52.642 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.643 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.643 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:52.643 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.725190+0000", 2024-02-23T00:51:52.643 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.3782999999999998e-05 2024-02-23T00:51:52.643 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.643 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.643 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:52.643 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.725224+0000", 2024-02-23T00:51:52.643 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3610000000000001e-06 2024-02-23T00:51:52.643 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.644 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.644 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:52.644 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.725232+0000", 2024-02-23T00:51:52.644 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5358000000000002e-05 2024-02-23T00:51:52.644 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.644 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.644 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:52.644 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.725277+0000", 2024-02-23T00:51:52.644 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.6744999999999999e-05 2024-02-23T00:51:52.644 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.644 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.644 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:52.644 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.725304+0000", 2024-02-23T00:51:52.645 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.04497164 2024-02-23T00:51:52.645 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.645 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.645 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:52.645 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.770276+0000", 2024-02-23T00:51:52.645 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4561000000000003e-05 2024-02-23T00:51:52.645 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.645 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.645 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:52.645 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.770310+0000", 2024-02-23T00:51:52.645 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.057972005 2024-02-23T00:51:52.645 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.645 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:52.645 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.646 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.646 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.646 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.24775.0:2070 20.6 e112/109 20:63faae7e:test-rados-api-smithi047-82457-17::big2:head v 112'4, mlcod=110'2)", 2024-02-23T00:51:52.646 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:01.807537+0000", 2024-02-23T00:51:52.646 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 530.82414987100003, 2024-02-23T00:51:52.646 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.085855267999999998, 2024-02-23T00:51:52.646 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:52.646 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:52.646 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:52.646 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.646 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:52.646 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.807537+0000", 2024-02-23T00:51:52.646 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:52.647 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.647 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.647 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:52.647 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.807537+0000", 2024-02-23T00:51:52.647 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999976 2024-02-23T00:51:52.647 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.647 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.647 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:52.647 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.807534+0000", 2024-02-23T00:51:52.647 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.057621555999999997 2024-02-23T00:51:52.647 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.647 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.647 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:52.648 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.865156+0000", 2024-02-23T00:51:52.648 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1656999999999999e-05 2024-02-23T00:51:52.648 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.648 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.648 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:52.648 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.865178+0000", 2024-02-23T00:51:52.648 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0491e-05 2024-02-23T00:51:52.648 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.648 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.648 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:52.648 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.865188+0000", 2024-02-23T00:51:52.648 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2298999999999993e-05 2024-02-23T00:51:52.648 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.649 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.649 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:52.649 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.865260+0000", 2024-02-23T00:51:52.649 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7316000000000001e-05 2024-02-23T00:51:52.649 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.649 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.649 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:52.649 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.865288+0000", 2024-02-23T00:51:52.649 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028062336 2024-02-23T00:51:52.649 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.649 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.649 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:52.649 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.893350+0000", 2024-02-23T00:51:52.649 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1810000000000001e-05 2024-02-23T00:51:52.650 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.650 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.650 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:52.650 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.893392+0000", 2024-02-23T00:51:52.650 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.085855267999999998 2024-02-23T00:51:52.650 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.650 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:52.650 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.650 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.650 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.650 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.24775.0:2074 20.1a e112/109 20:5de0a470:test-rados-api-smithi047-82457-17::big3:head v 112'3, mlcod=112'1)", 2024-02-23T00:51:52.650 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:01.957292+0000", 2024-02-23T00:51:52.650 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 530.67439492699998, 2024-02-23T00:51:52.651 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.060300126000000003, 2024-02-23T00:51:52.651 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:52.651 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:52.651 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:52.651 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.651 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:52.651 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.957292+0000", 2024-02-23T00:51:52.651 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:52.651 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.651 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.651 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:52.651 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.957292+0000", 2024-02-23T00:51:52.652 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:51:52.652 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.652 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.652 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:52.652 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.957291+0000", 2024-02-23T00:51:52.652 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0093502140000000008 2024-02-23T00:51:52.652 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.652 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.652 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:52.652 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.966641+0000", 2024-02-23T00:51:52.652 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0536999999999999e-05 2024-02-23T00:51:52.652 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.652 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.653 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:52.653 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.966652+0000", 2024-02-23T00:51:52.653 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9869999999999996e-06 2024-02-23T00:51:52.653 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.653 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.653 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:52.653 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.966658+0000", 2024-02-23T00:51:52.653 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1517999999999998e-05 2024-02-23T00:51:52.653 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.653 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.653 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:52.653 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.966699+0000", 2024-02-23T00:51:52.653 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.3855000000000001e-05 2024-02-23T00:51:52.653 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.654 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.654 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:52.654 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:01.966723+0000", 2024-02-23T00:51:52.654 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.050836219000000002 2024-02-23T00:51:52.654 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.654 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.654 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:52.654 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.017559+0000", 2024-02-23T00:51:52.654 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.2558999999999998e-05 2024-02-23T00:51:52.654 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.654 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.654 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:52.654 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:02.017592+0000", 2024-02-23T00:51:52.655 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.060300126000000003 2024-02-23T00:51:52.655 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.655 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:52.655 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.655 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.655 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.655 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.15144.0:2 38.1a 38:5c6776db:::0.obj:head [write 0~103] snapc 0=[] ondisk+write+known_if_redirected e113)", 2024-02-23T00:51:52.655 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:03.198372+0000", 2024-02-23T00:51:52.655 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 529.43331474700005, 2024-02-23T00:51:52.655 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1182399249999999, 2024-02-23T00:51:52.655 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:52.655 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:52.655 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:52.655 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.15144", 2024-02-23T00:51:52.656 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/2599587504", 2024-02-23T00:51:52.656 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 2 2024-02-23T00:51:52.656 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.656 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:52.656 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.656 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:52.656 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.198372+0000", 2024-02-23T00:51:52.656 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:52.656 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.656 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.656 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:52.656 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.198372+0000", 2024-02-23T00:51:52.657 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:51:52.657 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.657 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.657 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:52.657 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.198370+0000", 2024-02-23T00:51:52.657 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0789999999999997e-06 2024-02-23T00:51:52.657 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.657 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.657 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:52.657 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.198376+0000", 2024-02-23T00:51:52.657 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6660000000000003e-06 2024-02-23T00:51:52.657 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.657 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.657 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:52.658 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.198386+0000", 2024-02-23T00:51:52.658 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.5639999999999996e-06 2024-02-23T00:51:52.658 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.658 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.658 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:52.658 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.198392+0000", 2024-02-23T00:51:52.658 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8449999999999999e-05 2024-02-23T00:51:52.658 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.658 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.658 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:52.658 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.198440+0000", 2024-02-23T00:51:52.658 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1109999999999996e-06 2024-02-23T00:51:52.658 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.658 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.659 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:51:52.659 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:03.198446+0000", 2024-02-23T00:51:52.659 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1068513440000001 2024-02-23T00:51:52.659 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.659 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.659 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:52.659 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.305297+0000", 2024-02-23T00:51:52.659 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012047799999999999 2024-02-23T00:51:52.659 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.659 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.659 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:52.659 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.305418+0000", 2024-02-23T00:51:52.659 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000148796 2024-02-23T00:51:52.659 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.660 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.660 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 4,6", 2024-02-23T00:51:52.660 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.305567+0000", 2024-02-23T00:51:52.660 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0045039290000000003 2024-02-23T00:51:52.660 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.660 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.660 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:51:52.660 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.310071+0000", 2024-02-23T00:51:52.660 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0063369849999999998 2024-02-23T00:51:52.660 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.660 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.660 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:51:52.660 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.316408+0000", 2024-02-23T00:51:52.660 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000112998 2024-02-23T00:51:52.661 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.661 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.661 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:51:52.661 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.316521+0000", 2024-02-23T00:51:52.661 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9656e-05 2024-02-23T00:51:52.661 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.661 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.661 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:52.661 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.316570+0000", 2024-02-23T00:51:52.661 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1307999999999998e-05 2024-02-23T00:51:52.661 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.661 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.661 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:52.662 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:04.316612+0000", 2024-02-23T00:51:52.662 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1182399249999999 2024-02-23T00:51:52.662 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.662 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:52.662 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.662 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.662 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.662 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.25111.0:1 46.4s0 46:241ec7d8:test-rados-api-smithi047-82512-8::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e117)", 2024-02-23T00:51:52.662 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:06.589186+0000", 2024-02-23T00:51:52.662 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 526.04250048999995, 2024-02-23T00:51:52.662 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.40034441, 2024-02-23T00:51:52.662 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:52.662 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:52.662 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:52.663 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.25111", 2024-02-23T00:51:52.663 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/851664869", 2024-02-23T00:51:52.663 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1 2024-02-23T00:51:52.663 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.663 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:52.663 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.663 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:52.663 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:06.589186+0000", 2024-02-23T00:51:52.663 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:52.663 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.663 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.663 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:52.663 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:06.589186+0000", 2024-02-23T00:51:52.663 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:51:52.664 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.664 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.664 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:52.664 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:06.589185+0000", 2024-02-23T00:51:52.664 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0599999999999996e-06 2024-02-23T00:51:52.664 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.664 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.664 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:52.664 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:06.589191+0000", 2024-02-23T00:51:52.664 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000303713 2024-02-23T00:51:52.664 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.664 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.664 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:52.665 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:06.589494+0000", 2024-02-23T00:51:52.665 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0910000000000004e-06 2024-02-23T00:51:52.665 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.665 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.665 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:52.665 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:06.589501+0000", 2024-02-23T00:51:52.665 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6595999999999999e-05 2024-02-23T00:51:52.665 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.665 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.665 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:52.665 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:06.589548+0000", 2024-02-23T00:51:52.665 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.3579999999999999e-06 2024-02-23T00:51:52.665 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.666 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.666 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:51:52.666 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:06.589551+0000", 2024-02-23T00:51:52.666 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.392918256 2024-02-23T00:51:52.666 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.666 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.666 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:52.666 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.982470+0000", 2024-02-23T00:51:52.666 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013435300000000001 2024-02-23T00:51:52.666 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.666 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.666 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:52.666 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.982604+0000", 2024-02-23T00:51:52.667 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00045436599999999999 2024-02-23T00:51:52.667 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.667 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.667 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:51:52.667 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.983058+0000", 2024-02-23T00:51:52.667 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00046995799999999998 2024-02-23T00:51:52.667 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.667 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.667 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:51:52.667 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.983528+0000", 2024-02-23T00:51:52.667 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0059363580000000001 2024-02-23T00:51:52.667 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.667 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.667 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:52.668 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.989465+0000", 2024-02-23T00:51:52.668 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5662999999999998e-05 2024-02-23T00:51:52.668 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.668 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.668 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:52.668 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:07.989530+0000", 2024-02-23T00:51:52.668 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.40034441 2024-02-23T00:51:52.668 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.668 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:52.668 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.668 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.668 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.669 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(osd.3.0:1389 20.0 e139/109 -22:0000ebd9:::temp_20.0_0_14313_1:head v 0'0, mlcod=0'0)", 2024-02-23T00:51:52.669 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:32.123601+0000", 2024-02-23T00:51:52.669 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 500.50808513800001, 2024-02-23T00:51:52.669 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.024989610999999998, 2024-02-23T00:51:52.669 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:52.669 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:52.669 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:52.669 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.669 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:52.669 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.123601+0000", 2024-02-23T00:51:52.669 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:52.669 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.669 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.670 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:52.670 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.123601+0000", 2024-02-23T00:51:52.670 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:51:52.670 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.670 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.670 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:52.670 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.123599+0000", 2024-02-23T00:51:52.670 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013123701999999999 2024-02-23T00:51:52.670 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.670 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.670 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:52.670 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.136723+0000", 2024-02-23T00:51:52.670 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2133999999999998e-05 2024-02-23T00:51:52.671 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.671 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.671 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:52.671 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.136745+0000", 2024-02-23T00:51:52.671 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.4460000000000001e-06 2024-02-23T00:51:52.671 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.671 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.671 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:52.671 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.136754+0000", 2024-02-23T00:51:52.671 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8553000000000001e-05 2024-02-23T00:51:52.671 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.672 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.672 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:52.672 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.136802+0000", 2024-02-23T00:51:52.672 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9707999999999998e-05 2024-02-23T00:51:52.672 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.672 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.672 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:52.672 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.136822+0000", 2024-02-23T00:51:52.672 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011702494000000001 2024-02-23T00:51:52.672 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.672 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.672 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:52.673 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.148524+0000", 2024-02-23T00:51:52.673 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6593000000000004e-05 2024-02-23T00:51:52.673 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.673 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.673 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:52.673 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.148591+0000", 2024-02-23T00:51:52.673 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.024989610999999998 2024-02-23T00:51:52.673 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.673 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:52.673 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.673 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.673 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.673 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(osd.3.0:1390 20.0 e139/109 -22:0000ebd9:::temp_20.0_0_14313_1:head v 0'0, mlcod=0'0)", 2024-02-23T00:51:52.674 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:32.171787+0000", 2024-02-23T00:51:52.674 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 500.45989969099998, 2024-02-23T00:51:52.674 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.049768688999999998, 2024-02-23T00:51:52.674 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:52.674 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:52.674 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:52.674 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.674 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:52.674 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.171787+0000", 2024-02-23T00:51:52.674 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:52.674 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.674 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.675 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:52.675 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.171787+0000", 2024-02-23T00:51:52.675 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:51:52.675 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.675 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.675 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:52.675 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.171785+0000", 2024-02-23T00:51:52.675 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037314238 2024-02-23T00:51:52.675 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.675 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.675 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:52.675 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.209100+0000", 2024-02-23T00:51:52.676 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6101999999999999e-05 2024-02-23T00:51:52.676 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.676 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.676 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:52.676 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.209116+0000", 2024-02-23T00:51:52.676 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.3939999999999998e-06 2024-02-23T00:51:52.676 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.676 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.676 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:52.676 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.209123+0000", 2024-02-23T00:51:52.676 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0012550479999999999 2024-02-23T00:51:52.676 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.677 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.677 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:52.677 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.210378+0000", 2024-02-23T00:51:52.677 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7702000000000001e-05 2024-02-23T00:51:52.677 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.677 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.677 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:52.677 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.210396+0000", 2024-02-23T00:51:52.677 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011131066 2024-02-23T00:51:52.677 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.677 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.677 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:52.678 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.221527+0000", 2024-02-23T00:51:52.678 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.8481000000000001e-05 2024-02-23T00:51:52.678 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.678 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.678 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:52.678 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.221555+0000", 2024-02-23T00:51:52.678 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.049768688999999998 2024-02-23T00:51:52.678 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.678 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:52.678 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.678 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.678 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.679 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(osd.3.0:1391 20.0 e140/109 -22:0000ebd9:::temp_20.0_0_14313_1:head v 0'0, mlcod=0'0)", 2024-02-23T00:51:52.679 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:32.305524+0000", 2024-02-23T00:51:52.679 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 500.326162329, 2024-02-23T00:51:52.679 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.045602061999999999, 2024-02-23T00:51:52.679 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:52.679 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:52.679 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:52.679 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.679 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:52.679 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.305524+0000", 2024-02-23T00:51:52.679 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:52.679 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.679 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.679 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:52.680 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.305524+0000", 2024-02-23T00:51:52.680 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:51:52.680 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.680 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.680 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:52.680 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.305523+0000", 2024-02-23T00:51:52.680 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033906463999999997 2024-02-23T00:51:52.680 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.680 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.680 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:52.680 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.339430+0000", 2024-02-23T00:51:52.680 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8787000000000002e-05 2024-02-23T00:51:52.680 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.680 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.681 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:52.681 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.339449+0000", 2024-02-23T00:51:52.681 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5659999999999996e-06 2024-02-23T00:51:52.681 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.681 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.681 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:52.681 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.339458+0000", 2024-02-23T00:51:52.681 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5281999999999998e-05 2024-02-23T00:51:52.681 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.681 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.681 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:52.681 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.339503+0000", 2024-02-23T00:51:52.681 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9097e-05 2024-02-23T00:51:52.682 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.682 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.682 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:52.682 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.339523+0000", 2024-02-23T00:51:52.682 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01157158 2024-02-23T00:51:52.682 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.682 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.682 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:52.682 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.351094+0000", 2024-02-23T00:51:52.682 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.2014999999999998e-05 2024-02-23T00:51:52.682 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.682 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.682 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:52.682 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.351126+0000", 2024-02-23T00:51:52.682 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.045602061999999999 2024-02-23T00:51:52.682 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.683 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:52.683 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.683 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.683 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.683 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(osd.1.0:294 20.6 e140/109 -22:61acad7d:::temp_20.6_0_24170_1:head v 0'0, mlcod=112'6)", 2024-02-23T00:51:52.683 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:32.414966+0000", 2024-02-23T00:51:52.683 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 500.216720721, 2024-02-23T00:51:52.683 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.025904508, 2024-02-23T00:51:52.683 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:52.683 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:52.683 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:52.683 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.683 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:52.683 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.414966+0000", 2024-02-23T00:51:52.683 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:52.683 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.684 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.684 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:52.684 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.414966+0000", 2024-02-23T00:51:52.684 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:51:52.684 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.684 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.684 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:52.684 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.414965+0000", 2024-02-23T00:51:52.684 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750562999999999 2024-02-23T00:51:52.684 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.684 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.684 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:52.685 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.430715+0000", 2024-02-23T00:51:52.685 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0077000000000001e-05 2024-02-23T00:51:52.685 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.685 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.685 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:52.685 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.430735+0000", 2024-02-23T00:51:52.685 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1629999999999996e-06 2024-02-23T00:51:52.685 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.685 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.685 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:52.685 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.430743+0000", 2024-02-23T00:51:52.685 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.4369000000000002e-05 2024-02-23T00:51:52.686 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.686 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.686 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:52.686 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.430788+0000", 2024-02-23T00:51:52.686 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8938999999999999e-05 2024-02-23T00:51:52.686 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.686 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.686 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:52.686 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.430807+0000", 2024-02-23T00:51:52.686 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010016872 2024-02-23T00:51:52.686 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.686 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.686 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:52.687 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.440824+0000", 2024-02-23T00:51:52.687 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6726e-05 2024-02-23T00:51:52.687 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.687 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.687 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:52.687 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:32.440870+0000", 2024-02-23T00:51:52.687 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.025904508 2024-02-23T00:51:52.687 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.687 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:52.687 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.687 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.687 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.687 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.31603.0:4 79.5s0 79:a89d8add:test-rados-api-smithi047-82487-11::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e143)", 2024-02-23T00:51:52.688 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:34.937176+0000", 2024-02-23T00:51:52.688 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 497.69451012299999, 2024-02-23T00:51:52.688 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.101250728, 2024-02-23T00:51:52.688 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:52.688 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:52.688 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:52.688 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.31603", 2024-02-23T00:51:52.688 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1361361966", 2024-02-23T00:51:52.688 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 4 2024-02-23T00:51:52.688 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.688 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:52.688 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.688 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:52.689 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:34.937176+0000", 2024-02-23T00:51:52.689 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:52.689 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.689 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.689 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:52.689 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:34.937176+0000", 2024-02-23T00:51:52.689 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:51:52.689 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.689 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.689 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:52.689 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:34.937174+0000", 2024-02-23T00:51:52.689 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9299999999999992e-06 2024-02-23T00:51:52.689 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.690 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.690 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:52.690 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:34.937183+0000", 2024-02-23T00:51:52.690 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2823000000000001e-05 2024-02-23T00:51:52.690 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.690 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.690 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:52.690 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:34.937196+0000", 2024-02-23T00:51:52.690 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5089999999999999e-06 2024-02-23T00:51:52.690 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.690 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.690 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:52.690 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:34.937203+0000", 2024-02-23T00:51:52.691 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6110999999999999e-05 2024-02-23T00:51:52.691 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.691 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.691 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:52.691 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:34.937259+0000", 2024-02-23T00:51:52.691 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.3068999999999999e-05 2024-02-23T00:51:52.691 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.691 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.691 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:52.691 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:34.937352+0000", 2024-02-23T00:51:52.691 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028850399999999997 2024-02-23T00:51:52.691 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.691 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.692 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:51:52.692 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:34.937640+0000", 2024-02-23T00:51:52.692 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028285199999999999 2024-02-23T00:51:52.692 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.692 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.692 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:51:52.692 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:34.937923+0000", 2024-02-23T00:51:52.692 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.100441616 2024-02-23T00:51:52.692 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.692 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.692 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:52.692 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:35.038365+0000", 2024-02-23T00:51:52.692 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.2336999999999996e-05 2024-02-23T00:51:52.692 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.693 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.693 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:52.693 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:35.038427+0000", 2024-02-23T00:51:52.693 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.101250728 2024-02-23T00:51:52.693 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.693 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:52.693 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.693 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.693 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.693 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.15270.0:248 62.14 62:28b98b91:test-rados-api-smithi047-82465-12::bar:head [writefull 0~8] snapc 0=[] ondisk+write+known_if_redirected e145)", 2024-02-23T00:51:52.693 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:37.219883+0000", 2024-02-23T00:51:52.694 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 495.41180383199998, 2024-02-23T00:51:52.694 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037348011, 2024-02-23T00:51:52.694 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:52.694 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:52.694 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:52.694 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.15270", 2024-02-23T00:51:52.694 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/4133111990", 2024-02-23T00:51:52.694 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 248 2024-02-23T00:51:52.694 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.694 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:52.694 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.694 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:52.695 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.219883+0000", 2024-02-23T00:51:52.695 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:52.695 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.695 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.695 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:52.695 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.219883+0000", 2024-02-23T00:51:52.695 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:51:52.695 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.695 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.695 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:52.695 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.219881+0000", 2024-02-23T00:51:52.695 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4849999999999997e-06 2024-02-23T00:51:52.696 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.696 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.696 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:52.696 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.219888+0000", 2024-02-23T00:51:52.696 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0907000000000001e-05 2024-02-23T00:51:52.696 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.696 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.696 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:52.696 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.219899+0000", 2024-02-23T00:51:52.696 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1180000000000003e-06 2024-02-23T00:51:52.696 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.696 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.697 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:52.697 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.219906+0000", 2024-02-23T00:51:52.697 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1885999999999998e-05 2024-02-23T00:51:52.697 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.697 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.697 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:52.697 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.219958+0000", 2024-02-23T00:51:52.697 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000102293 2024-02-23T00:51:52.697 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.697 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.697 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:52.697 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.220060+0000", 2024-02-23T00:51:52.697 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4350000000000003e-05 2024-02-23T00:51:52.697 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.698 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.698 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 2,4", 2024-02-23T00:51:52.698 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.220154+0000", 2024-02-23T00:51:52.698 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00034315799999999998 2024-02-23T00:51:52.698 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.698 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.698 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:51:52.698 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.220497+0000", 2024-02-23T00:51:52.698 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00019797599999999999 2024-02-23T00:51:52.698 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.698 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.698 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:51:52.698 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.220695+0000", 2024-02-23T00:51:52.698 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036432616000000001 2024-02-23T00:51:52.699 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.699 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.699 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:51:52.699 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.257128+0000", 2024-02-23T00:51:52.699 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.5760000000000001e-05 2024-02-23T00:51:52.699 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.699 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.699 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:52.699 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.257184+0000", 2024-02-23T00:51:52.699 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6805999999999999e-05 2024-02-23T00:51:52.699 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.699 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.700 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:52.700 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.257231+0000", 2024-02-23T00:51:52.700 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037348011 2024-02-23T00:51:52.700 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.700 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:52.700 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.700 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.700 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.700 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.39416.0:13 95.2s0 95:442961eb:test-rados-api-smithi047-82487-16::foo:head [write 8192~8192] snapc 7=[7,6] ondisk+write+known_if_redirected e161)", 2024-02-23T00:51:52.700 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:53.113989+0000", 2024-02-23T00:51:52.700 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 479.517697376, 2024-02-23T00:51:52.701 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.024944128999999999, 2024-02-23T00:51:52.701 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:52.701 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:52.701 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:52.701 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.39416", 2024-02-23T00:51:52.701 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/2338570187", 2024-02-23T00:51:52.701 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 13 2024-02-23T00:51:52.701 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.701 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:52.701 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.701 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:52.701 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:53.113989+0000", 2024-02-23T00:51:52.701 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:52.701 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.702 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.702 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:52.702 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:53.113989+0000", 2024-02-23T00:51:52.702 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:51:52.702 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.702 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.702 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:52.702 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:53.113987+0000", 2024-02-23T00:51:52.702 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4941999999999999e-05 2024-02-23T00:51:52.702 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.702 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.702 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:52.702 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:53.114012+0000", 2024-02-23T00:51:52.703 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4773000000000001e-05 2024-02-23T00:51:52.703 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.703 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.703 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:52.703 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:53.114027+0000", 2024-02-23T00:51:52.703 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.8349999999999993e-06 2024-02-23T00:51:52.703 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.703 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.703 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:52.703 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:53.114036+0000", 2024-02-23T00:51:52.703 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015537468 2024-02-23T00:51:52.703 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.704 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.704 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:52.704 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:53.129573+0000", 2024-02-23T00:51:52.704 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103847 2024-02-23T00:51:52.704 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.704 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.704 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:52.704 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:53.129677+0000", 2024-02-23T00:51:52.704 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00069602900000000003 2024-02-23T00:51:52.704 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.704 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.704 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:51:52.704 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:53.130373+0000", 2024-02-23T00:51:52.705 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001455729 2024-02-23T00:51:52.705 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.705 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.705 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:51:52.705 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:53.131829+0000", 2024-02-23T00:51:52.705 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0070283079999999996 2024-02-23T00:51:52.705 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.705 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.705 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:52.705 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:53.138857+0000", 2024-02-23T00:51:52.705 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6155000000000005e-05 2024-02-23T00:51:52.705 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.705 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.705 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:52.706 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:53.138933+0000", 2024-02-23T00:51:52.706 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.024944128999999999 2024-02-23T00:51:52.706 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.706 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:52.706 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.706 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.706 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.706 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.40529.0:1 114.4s0 114:2a73ef27:test-rados-api-smithi047-82496-16::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e180)", 2024-02-23T00:51:52.706 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:44:12.479624+0000", 2024-02-23T00:51:52.706 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 460.15206268700001, 2024-02-23T00:51:52.706 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.035619211, 2024-02-23T00:51:52.706 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:52.706 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:52.706 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:52.706 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.40529", 2024-02-23T00:51:52.707 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/3414618047", 2024-02-23T00:51:52.707 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1 2024-02-23T00:51:52.707 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.707 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:52.707 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.707 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:52.707 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:12.479624+0000", 2024-02-23T00:51:52.707 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:52.707 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.707 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.707 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:52.707 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:12.479624+0000", 2024-02-23T00:51:52.707 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:51:52.708 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.708 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.708 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:52.708 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:12.479623+0000", 2024-02-23T00:51:52.708 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5939999999999996e-06 2024-02-23T00:51:52.712 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.712 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.712 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:52.712 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:12.479630+0000", 2024-02-23T00:51:52.712 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6150000000000002e-06 2024-02-23T00:51:52.712 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.712 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.712 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:52.712 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:12.479637+0000", 2024-02-23T00:51:52.712 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4399999999999996e-06 2024-02-23T00:51:52.712 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.713 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.713 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:52.713 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:12.479642+0000", 2024-02-23T00:51:52.713 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9409999999999997e-05 2024-02-23T00:51:52.713 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.713 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.713 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:52.713 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:12.479682+0000", 2024-02-23T00:51:52.713 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.311e-06 2024-02-23T00:51:52.713 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.713 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.713 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:51:52.714 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:12.479684+0000", 2024-02-23T00:51:52.714 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0225209479999999 2024-02-23T00:51:52.714 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.714 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.714 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:52.714 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:13.502205+0000", 2024-02-23T00:51:52.714 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000244198 2024-02-23T00:51:52.714 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.714 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.714 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:52.714 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:13.502449+0000", 2024-02-23T00:51:52.714 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00046503000000000001 2024-02-23T00:51:52.715 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.715 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.715 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:51:52.715 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:13.502914+0000", 2024-02-23T00:51:52.715 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0018608260000000001 2024-02-23T00:51:52.715 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.715 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.715 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:51:52.715 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:13.504775+0000", 2024-02-23T00:51:52.715 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010417872999999999 2024-02-23T00:51:52.715 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.715 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.715 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:52.716 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:13.515193+0000", 2024-02-23T00:51:52.716 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0229999999999998e-05 2024-02-23T00:51:52.716 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.716 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.716 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:52.716 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:13.515243+0000", 2024-02-23T00:51:52.716 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.035619211 2024-02-23T00:51:52.716 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.716 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:52.716 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.716 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.716 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.717 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.40529.0:4 114.4s0 114:34880124:test-rados-api-smithi047-82496-17::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e183)", 2024-02-23T00:51:52.717 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:44:15.132272+0000", 2024-02-23T00:51:52.717 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 457.49941478099998, 2024-02-23T00:51:52.717 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.026014500999999999, 2024-02-23T00:51:52.717 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:52.717 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:52.717 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:52.717 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.40529", 2024-02-23T00:51:52.717 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/3414618047", 2024-02-23T00:51:52.717 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 4 2024-02-23T00:51:52.717 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.717 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:52.717 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.718 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:52.718 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:15.132272+0000", 2024-02-23T00:51:52.718 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:52.718 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.718 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.718 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:52.718 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:15.132272+0000", 2024-02-23T00:51:52.718 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999976 2024-02-23T00:51:52.718 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.718 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.718 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:52.718 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:15.132269+0000", 2024-02-23T00:51:52.718 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9840000000000001e-06 2024-02-23T00:51:52.718 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.718 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.719 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:52.719 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:15.132277+0000", 2024-02-23T00:51:52.719 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1032999999999999e-05 2024-02-23T00:51:52.719 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.719 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.719 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:52.719 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:15.132288+0000", 2024-02-23T00:51:52.719 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.6419999999999996e-06 2024-02-23T00:51:52.719 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.719 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.719 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:52.719 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:15.132297+0000", 2024-02-23T00:51:52.719 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00036041099999999999 2024-02-23T00:51:52.719 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.719 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.720 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:52.720 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:15.132658+0000", 2024-02-23T00:51:52.720 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0790000000000001e-06 2024-02-23T00:51:52.720 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.720 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.720 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op must wait for map", 2024-02-23T00:51:52.720 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:15.132666+0000", 2024-02-23T00:51:52.720 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015467946 2024-02-23T00:51:52.720 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.720 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.720 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:52.720 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:15.148134+0000", 2024-02-23T00:51:52.720 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016813900000000001 2024-02-23T00:51:52.721 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.721 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.721 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:52.721 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:15.148302+0000", 2024-02-23T00:51:52.721 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00059727800000000002 2024-02-23T00:51:52.721 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.721 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.721 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:51:52.721 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:15.148899+0000", 2024-02-23T00:51:52.721 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0090316800000000003 2024-02-23T00:51:52.721 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.721 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.721 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:51:52.721 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:15.157931+0000", 2024-02-23T00:51:52.722 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2560999999999998e-05 2024-02-23T00:51:52.722 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.722 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.722 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:52.722 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:15.158013+0000", 2024-02-23T00:51:52.722 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00027295299999999999 2024-02-23T00:51:52.722 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.722 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.722 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:52.722 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:44:15.158286+0000", 2024-02-23T00:51:52.722 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.026014500999999999 2024-02-23T00:51:52.722 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.722 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:52.723 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.723 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.723 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.723 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.41009.0:1 179.6s0 179:602f83fe:::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e292)", 2024-02-23T00:51:52.723 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:46:04.517108+0000", 2024-02-23T00:51:52.723 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 348.114578591, 2024-02-23T00:51:52.723 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.66709289299999996, 2024-02-23T00:51:52.723 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:52.723 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:52.723 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:52.723 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.41009", 2024-02-23T00:51:52.723 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/2639020958", 2024-02-23T00:51:52.723 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1 2024-02-23T00:51:52.723 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.723 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:52.724 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.724 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:52.724 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:04.517108+0000", 2024-02-23T00:51:52.724 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:52.724 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.724 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.724 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:52.724 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:04.517108+0000", 2024-02-23T00:51:52.724 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:51:52.724 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.724 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.724 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:52.724 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:04.517107+0000", 2024-02-23T00:51:52.724 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0429999999999998e-06 2024-02-23T00:51:52.725 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.725 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.725 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:52.725 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:04.517112+0000", 2024-02-23T00:51:52.725 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5920000000000003e-06 2024-02-23T00:51:52.725 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.725 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.725 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:52.725 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:04.517119+0000", 2024-02-23T00:51:52.725 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1099999999999997e-06 2024-02-23T00:51:52.725 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.725 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.725 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:52.725 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:04.517126+0000", 2024-02-23T00:51:52.725 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.8010000000000001e-05 2024-02-23T00:51:52.726 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.726 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.726 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:52.726 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:04.517154+0000", 2024-02-23T00:51:52.726 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.593e-06 2024-02-23T00:51:52.726 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.726 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.726 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:51:52.726 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:04.517157+0000", 2024-02-23T00:51:52.726 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.66365052700000005 2024-02-23T00:51:52.726 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.726 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.726 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:52.727 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:05.180807+0000", 2024-02-23T00:51:52.727 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111835 2024-02-23T00:51:52.727 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.727 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.727 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:52.727 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:05.180919+0000", 2024-02-23T00:51:52.727 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040999 2024-02-23T00:51:52.727 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.727 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.727 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:51:52.727 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:05.181329+0000", 2024-02-23T00:51:52.727 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0011668410000000001 2024-02-23T00:51:52.727 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.727 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.727 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:51:52.728 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:05.182496+0000", 2024-02-23T00:51:52.728 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0016674039999999999 2024-02-23T00:51:52.728 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.728 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.728 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:52.728 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:05.184163+0000", 2024-02-23T00:51:52.728 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7271000000000003e-05 2024-02-23T00:51:52.728 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.728 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.728 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:52.728 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:05.184201+0000", 2024-02-23T00:51:52.728 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.66709289299999996 2024-02-23T00:51:52.729 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.729 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:52.729 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.729 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.729 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.729 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.30741.0:1 187.6s0 187:72c88bc2:::oid_IsCompletePP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e311)", 2024-02-23T00:51:52.729 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:46:23.195582+0000", 2024-02-23T00:51:52.729 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 329.436104747, 2024-02-23T00:51:52.729 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0060393860000001, 2024-02-23T00:51:52.729 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:52.729 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:52.729 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:52.729 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.30741", 2024-02-23T00:51:52.730 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1977736697", 2024-02-23T00:51:52.730 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1 2024-02-23T00:51:52.730 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.730 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:52.730 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.730 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:52.730 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:23.195582+0000", 2024-02-23T00:51:52.730 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:52.730 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.730 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.730 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:52.730 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:23.195582+0000", 2024-02-23T00:51:52.730 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:51:52.731 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.731 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.731 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:52.731 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:23.195580+0000", 2024-02-23T00:51:52.731 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.7450000000000001e-06 2024-02-23T00:51:52.731 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.731 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.731 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:52.731 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:23.195586+0000", 2024-02-23T00:51:52.731 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.6580000000000008e-06 2024-02-23T00:51:52.731 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.731 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.732 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:52.732 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:23.195595+0000", 2024-02-23T00:51:52.732 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1179999999999997e-06 2024-02-23T00:51:52.732 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.732 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.732 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:52.732 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:23.195601+0000", 2024-02-23T00:51:52.732 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.4499999999999997e-05 2024-02-23T00:51:52.732 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.732 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.732 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:52.732 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:23.195645+0000", 2024-02-23T00:51:52.733 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1960000000000001e-06 2024-02-23T00:51:52.733 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.733 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.733 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:51:52.733 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:23.195650+0000", 2024-02-23T00:51:52.733 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0044928049999999 2024-02-23T00:51:52.733 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.733 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.733 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:52.733 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:24.200142+0000", 2024-02-23T00:51:52.733 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000152762 2024-02-23T00:51:52.733 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.733 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.733 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:52.734 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:24.200295+0000", 2024-02-23T00:51:52.734 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00054695799999999995 2024-02-23T00:51:52.734 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.734 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.734 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:51:52.734 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:24.200842+0000", 2024-02-23T00:51:52.734 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00047161300000000001 2024-02-23T00:51:52.734 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.734 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.734 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:51:52.734 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:24.201314+0000", 2024-02-23T00:51:52.734 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026416699999999999 2024-02-23T00:51:52.734 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.734 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.735 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:52.735 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:24.201578+0000", 2024-02-23T00:51:52.735 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3269000000000002e-05 2024-02-23T00:51:52.735 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.735 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.735 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:52.735 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:24.201621+0000", 2024-02-23T00:51:52.735 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0060393860000001 2024-02-23T00:51:52.735 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.735 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:52.735 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.735 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.735 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.735 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.30861.0:1 199.2s0 199:482bd08b:::oid_SimpleStatPP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e356)", 2024-02-23T00:51:52.735 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:47:08.178993+0000", 2024-02-23T00:51:52.736 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 284.45269336899997, 2024-02-23T00:51:52.736 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1427090879999999, 2024-02-23T00:51:52.736 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:52.736 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:51:52.736 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:52.736 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.30861", 2024-02-23T00:51:52.736 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/3942253766", 2024-02-23T00:51:52.736 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1 2024-02-23T00:51:52.736 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.736 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:52.736 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.736 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:52.736 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:08.178993+0000", 2024-02-23T00:51:52.736 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:52.737 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.737 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.737 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:52.737 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:08.178993+0000", 2024-02-23T00:51:52.737 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:51:52.737 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.737 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.737 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:52.737 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:08.178992+0000", 2024-02-23T00:51:52.737 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.4229999999999998e-06 2024-02-23T00:51:52.737 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.737 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.737 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:52.738 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:08.178997+0000", 2024-02-23T00:51:52.738 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5320000000000001e-06 2024-02-23T00:51:52.738 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.738 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.738 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:52.738 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:08.179004+0000", 2024-02-23T00:51:52.738 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.815e-06 2024-02-23T00:51:52.738 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.738 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.738 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:52.738 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:08.179011+0000", 2024-02-23T00:51:52.738 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0550000000000003e-05 2024-02-23T00:51:52.738 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.738 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.738 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:52.739 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:08.179051+0000", 2024-02-23T00:51:52.739 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9099999999999999e-06 2024-02-23T00:51:52.739 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.739 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.739 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:51:52.739 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:08.179053+0000", 2024-02-23T00:51:52.739 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.141086343 2024-02-23T00:51:52.739 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.739 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.739 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:52.739 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:09.320140+0000", 2024-02-23T00:51:52.739 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156467 2024-02-23T00:51:52.739 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.740 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.740 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:52.740 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:09.320296+0000", 2024-02-23T00:51:52.740 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00042973700000000002 2024-02-23T00:51:52.740 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.740 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.740 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:51:52.740 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:09.320726+0000", 2024-02-23T00:51:52.740 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00049958999999999995 2024-02-23T00:51:52.740 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.740 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.740 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:51:52.740 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:09.321226+0000", 2024-02-23T00:51:52.741 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000398384 2024-02-23T00:51:52.741 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.741 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.741 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:51:52.741 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:09.321624+0000", 2024-02-23T00:51:52.741 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.8273000000000001e-05 2024-02-23T00:51:52.741 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.741 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.741 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:52.741 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:09.321702+0000", 2024-02-23T00:51:52.741 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1427090879999999 2024-02-23T00:51:52.741 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.741 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:52.741 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.742 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.742 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.742 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.15270.0:4850 196.18 196:18000000:%01:::head [pgnls start_epoch 358] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e358)", 2024-02-23T00:51:52.742 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:47:10.195882+0000", 2024-02-23T00:51:52.742 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 282.43580414799999, 2024-02-23T00:51:52.742 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.024687572000000001, 2024-02-23T00:51:52.742 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:52.742 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:51:52.742 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:52.742 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.15270", 2024-02-23T00:51:52.742 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/4133111990", 2024-02-23T00:51:52.742 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 4850 2024-02-23T00:51:52.742 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.742 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:52.743 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.743 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:52.743 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:10.195882+0000", 2024-02-23T00:51:52.743 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:52.743 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.743 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.743 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:52.743 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:10.195882+0000", 2024-02-23T00:51:52.743 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999976 2024-02-23T00:51:52.743 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.743 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.743 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:52.743 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:10.195880+0000", 2024-02-23T00:51:52.744 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3458999999999999e-05 2024-02-23T00:51:52.744 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.744 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.744 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:52.744 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:10.195893+0000", 2024-02-23T00:51:52.744 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0264000000000001e-05 2024-02-23T00:51:52.744 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.744 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.744 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:52.744 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:10.195913+0000", 2024-02-23T00:51:52.744 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2982e-05 2024-02-23T00:51:52.744 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.744 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.745 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:52.745 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:10.195926+0000", 2024-02-23T00:51:52.745 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6429999999999999e-05 2024-02-23T00:51:52.745 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.745 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.745 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:52.745 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:10.195993+0000", 2024-02-23T00:51:52.745 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1660000000000002e-06 2024-02-23T00:51:52.745 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.745 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.745 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op must wait for map", 2024-02-23T00:51:52.745 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:10.195998+0000", 2024-02-23T00:51:52.745 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.024425932000000001 2024-02-23T00:51:52.746 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.746 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.746 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:52.746 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:10.220424+0000", 2024-02-23T00:51:52.746 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7039000000000002e-05 2024-02-23T00:51:52.746 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.746 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.746 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:52.746 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:10.220461+0000", 2024-02-23T00:51:52.746 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000108857 2024-02-23T00:51:52.746 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.746 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.746 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:52.746 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:10.220570+0000", 2024-02-23T00:51:52.747 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.024687572000000001 2024-02-23T00:51:52.747 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.747 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:52.747 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.747 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.747 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.747 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.24311.0:1285 5.d 5:b4812045:::notify.4:head [watch reconnect cookie 94587070401920 gen 1] snapc 0=[] ondisk+write+known_if_redirected e375)", 2024-02-23T00:51:52.747 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:47:27.942599+0000", 2024-02-23T00:51:52.747 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 264.68908735600002, 2024-02-23T00:51:52.747 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.31385020000000002, 2024-02-23T00:51:52.747 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:51:52.747 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:51:52.747 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:51:52.748 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.24311", 2024-02-23T00:51:52.748 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/3723202188", 2024-02-23T00:51:52.748 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1285 2024-02-23T00:51:52.748 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.748 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:51:52.748 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.748 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:51:52.748 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.942599+0000", 2024-02-23T00:51:52.748 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:51:52.748 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.748 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.748 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:51:52.748 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.942599+0000", 2024-02-23T00:51:52.748 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:51:52.749 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.749 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.749 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:51:52.749 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.942598+0000", 2024-02-23T00:51:52.749 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7109999999999998e-06 2024-02-23T00:51:52.749 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.749 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.749 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:51:52.749 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.942605+0000", 2024-02-23T00:51:52.749 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2098e-05 2024-02-23T00:51:52.749 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.749 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.749 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:51:52.749 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.942617+0000", 2024-02-23T00:51:52.750 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.8320000000000002e-06 2024-02-23T00:51:52.750 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.750 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.750 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:51:52.750 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.942623+0000", 2024-02-23T00:51:52.750 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.5760000000000001e-05 2024-02-23T00:51:52.750 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.750 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.750 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:52.750 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.942678+0000", 2024-02-23T00:51:52.750 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.559e-06 2024-02-23T00:51:52.750 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.750 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.750 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:51:52.750 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.942686+0000", 2024-02-23T00:51:52.751 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.288747688 2024-02-23T00:51:52.751 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.751 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.751 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:52.751 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:28.231434+0000", 2024-02-23T00:51:52.751 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000296077 2024-02-23T00:51:52.751 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.751 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.751 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for degraded object", 2024-02-23T00:51:52.751 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:28.231730+0000", 2024-02-23T00:51:52.751 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.018676325000000001 2024-02-23T00:51:52.751 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.751 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.751 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:51:52.751 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:28.250406+0000", 2024-02-23T00:51:52.751 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2424999999999998e-05 2024-02-23T00:51:52.752 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.752 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.752 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:51:52.752 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:28.250488+0000", 2024-02-23T00:51:52.752 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0059609060000000002 2024-02-23T00:51:52.752 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:51:52.752 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:51:52.752 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:51:52.752 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:28.256449+0000", 2024-02-23T00:51:52.752 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.31385020000000002 2024-02-23T00:51:52.752 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.752 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:52.752 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.752 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:51:52.752 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:51:52.753 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:51:52.753 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:51:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:51:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:51:52.944 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:51:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:51:53.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:52.273818+0000 mon.a (mon.0) 3060 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-99-cache", "mode": "writeback"}]: dispatch 2024-02-23T00:51:53.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:52.508425+0000 mgr.y (mgr.14517) 524 : audit [DBG] from='client.41741 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:51:53.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:53.268660+0000 mon.a (mon.0) 3061 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:51:53.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:53.273708+0000 mon.a (mon.0) 3062 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-99-cache", "mode": "writeback"}]': finished 2024-02-23T00:51:53.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:53.273754+0000 mon.a (mon.0) 3063 : cluster [DBG] osdmap e584: 8 total, 8 up, 7 in 2024-02-23T00:51:53.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:53.274713+0000 mon.a (mon.0) 3064 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-99-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:51:53.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:52.273818+0000 mon.a (mon.0) 3060 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-99-cache", "mode": "writeback"}]: dispatch 2024-02-23T00:51:53.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:52.508425+0000 mgr.y (mgr.14517) 524 : audit [DBG] from='client.41741 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:51:53.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:53.268660+0000 mon.a (mon.0) 3061 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:51:53.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:53.273708+0000 mon.a (mon.0) 3062 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-99-cache", "mode": "writeback"}]': finished 2024-02-23T00:51:53.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:53.273754+0000 mon.a (mon.0) 3063 : cluster [DBG] osdmap e584: 8 total, 8 up, 7 in 2024-02-23T00:51:53.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:53.274713+0000 mon.a (mon.0) 3064 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-99-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:51:53.428 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.0 dump_ops_in_flight 2024-02-23T00:51:53.432 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":878,"stamp":"2024-02-23T00:51:52.268673+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5801,"num_objects":226,"num_object_clones":0,"num_object_copies":678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":225,"num_whiteouts":0,"num_read":2285,"num_read_kb":2003,"num_write":914,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":2626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1049,"ondisk_log_size":1049,"up":576,"acting":576,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":503,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":2172376,"kb_used_data":18080,"kb_used_omap":0,"kb_used_meta":2154240,"kb_avail":654043688,"statfs":{"total":671965249536,"available":669740736512,"internally_reserved":0,"allocated":18513920,"data_stored":7704017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2205941760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.008657"},"pg_stats":[{"pgid":"233.1f","version":"0'0","reported_seq":10,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.282277+0000","last_change":"2024-02-23T00:51:50.281288+0000","last_active":"2024-02-23T00:51:50.282277+0000","last_peered":"2024-02-23T00:51:50.282277+0000","last_clean":"2024-02-23T00:51:50.282277+0000","last_became_active":"2024-02-23T00:51:50.280978+0000","last_became_peered":"2024-02-23T00:51:50.280978+0000","last_unstale":"2024-02-23T00:51:50.282277+0000","last_undegraded":"2024-02-23T00:51:50.282277+0000","last_fullsized":"2024-02-23T00:51:50.282277+0000","mapping_epoch":580,"log_start":"0'0","ondisk_log_start":"0'0","created":580,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_clean_scrub_stamp":"2024-02-23T00:51:49.259744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"233.1e","version":"0'0","reported_seq":9,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.277482+0000","last_change":"2024-02-23T00:51:50.277482+0000","last_active":"2024-02-23T00:51:50.277482+0000","last_peered":"2024-02-23T00:51:50.277482+0000","last_clean":"2024-02-23T00:51:50.277482+0000","last_became_active":"2024-02-23T00:51:50.276862+0000","last_became_peered":"2024-02-23T00:51:50.276862+0000","last_unstale":"2024-02-23T00:51:50.277482+0000","last_undegraded":"2024-02-23T00:51:50.277482+0000","last_fullsized":"2024-02-23T00:51:50.277482+0000","mapping_epoch":580,"log_start":"0'0","ondisk_log_start":"0'0","created":580,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_clean_scrub_stamp":"2024-02-23T00:51:49.259744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"233.1d","version":"0'0","reported_seq":9,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.277163+0000","last_change":"2024-02-23T00:51:50.277163+0000","last_active":"2024-02-23T00:51:50.277163+0000","last_peered":"2024-02-23T00:51:50.277163+0000","last_clean":"2024-02-23T00:51:50.277163+0000","last_became_active":"2024-02-23T00:51:50.276879+0000","last_became_peered":"2024-02-23T00:51:50.276879+0000","last_unstale":"2024-02-23T00:51:50.277163+0000","last_undegraded":"2024-02-23T00:51:50.277163+0000","last_fullsized":"2024-02-23T00:51:50.277163+0000","mapping_epoch":580,"log_start":"0'0","ondisk_log_start":"0'0","created":580,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_clean_scrub_stamp":"2024-02-23T00:51:49.259744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"233.1c","version":"0'0","reported_seq":10,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.280969+0000","last_change":"2024-02-23T00:51:50.279200+0000","last_active":"2024-02-23T00:51:50.280969+0000","last_peered":"2024-02-23T00:51:50.280969+0000","last_clean":"2024-02-23T00:51:50.280969+0000","last_became_active":"2024-02-23T00:51:50.278931+0000","last_became_peered":"2024-02-23T00:51:50.278931+0000","last_unstale":"2024-02-23T00:51:50.280969+0000","last_undegraded":"2024-02-23T00:51:50.280969+0000","last_fullsized":"2024-02-23T00:51:50.280969+0000","mapping_epoch":580,"log_start":"0'0","ondisk_log_start":"0'0","created":580,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_clean_scrub_stamp":"2024-02-23T00:51:49.259744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"233.1b","version":"0'0","reported_seq":10,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.277264+0000","last_change":"2024-02-23T00:51:50.276646+0000","last_active":"2024-02-23T00:51:50.277264+0000","last_peered":"2024-02-23T00:51:50.277264+0000","last_clean":"2024-02-23T00:51:50.277264+0000","last_became_active":"2024-02-23T00:51:50.276323+0000","last_became_peered":"2024-02-23T00:51:50.276323+0000","last_unstale":"2024-02-23T00:51:50.277264+0000","last_undegraded":"2024-02-23T00:51:50.277264+0000","last_fullsized":"2024-02-23T00:51:50.277264+0000","mapping_epoch":580,"log_start":"0'0","ondisk_log_start":"0'0","created":580,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_clean_scrub_stamp":"2024-02-23T00:51:49.259744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"233.1a","version":"0'0","reported_seq":9,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.277163+0000","last_change":"2024-02-23T00:51:50.277163+0000","last_active":"2024-02-23T00:51:50.277163+0000","last_peered":"2024-02-23T00:51:50.277163+0000","last_clean":"2024-02-23T00:51:50.277163+0000","last_became_active":"2024-02-23T00:51:50.276858+0000","last_became_peered":"2024-02-23T00:51:50.276858+0000","last_unstale":"2024-02-23T00:51:50.277163+0000","last_undegraded":"2024-02-23T00:51:50.277163+0000","last_fullsized":"2024-02-23T00:51:50.277163+0000","mapping_epoch":580,"log_start":"0'0","ondisk_log_start":"0'0","created":580,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_clean_scrub_stamp":"2024-02-23T00:51:49.259744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"233.19","version":"0'0","reported_seq":9,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.284450+0000","last_change":"2024-02-23T00:51:50.284450+0000","last_active":"2024-02-23T00:51:50.284450+0000","last_peered":"2024-02-23T00:51:50.284450+0000","last_clean":"2024-02-23T00:51:50.284450+0000","last_became_active":"2024-02-23T00:51:50.284119+0000","last_became_peered":"2024-02-23T00:51:50.284119+0000","last_unstale":"2024-02-23T00:51:50.284450+0000","last_undegraded":"2024-02-23T00:51:50.284450+0000","last_fullsized":"2024-02-23T00:51:50.284450+0000","mapping_epoch":580,"log_start":"0'0","ondisk_log_start":"0'0","created":580,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_clean_scrub_stamp":"2024-02-23T00:51:49.259744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"233.18","version":"0'0","reported_seq":9,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.282983+0000","last_change":"2024-02-23T00:51:50.282983+0000","last_active":"2024-02-23T00:51:50.282983+0000","last_peered":"2024-02-23T00:51:50.282983+0000","last_clean":"2024-02-23T00:51:50.282983+0000","last_became_active":"2024-02-23T00:51:50.282674+0000","last_became_peered":"2024-02-23T00:51:50.282674+0000","last_unstale":"2024-02-23T00:51:50.282983+0000","last_undegraded":"2024-02-23T00:51:50.282983+0000","last_fullsized":"2024-02-23T00:51:50.282983+0000","mapping_epoch":580,"log_start":"0'0","ondisk_log_start":"0'0","created":580,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_clean_scrub_stamp":"2024-02-23T00:51:49.259744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"233.17","version":"0'0","reported_seq":9,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.282230+0000","last_change":"2024-02-23T00:51:50.282230+0000","last_active":"2024-02-23T00:51:50.282230+0000","last_peered":"2024-02-23T00:51:50.282230+0000","last_clean":"2024-02-23T00:51:50.282230+0000","last_became_active":"2024-02-23T00:51:50.281953+0000","last_became_peered":"2024-02-23T00:51:50.281953+0000","last_unstale":"2024-02-23T00:51:50.282230+0000","last_undegraded":"2024-02-23T00:51:50.282230+0000","last_fullsized":"2024-02-23T00:51:50.282230+0000","mapping_epoch":580,"log_start":"0'0","ondisk_log_start":"0'0","created":580,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_clean_scrub_stamp":"2024-02-23T00:51:49.259744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,3],"acting":[1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"233.16","version":"0'0","reported_seq":9,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.287403+0000","last_change":"2024-02-23T00:51:50.287403+0000","last_active":"2024-02-23T00:51:50.287403+0000","last_peered":"2024-02-23T00:51:50.287403+0000","last_clean":"2024-02-23T00:51:50.287403+0000","last_became_active":"2024-02-23T00:51:50.287101+0000","last_became_peered":"2024-02-23T00:51:50.287101+0000","last_unstale":"2024-02-23T00:51:50.287403+0000","last_undegraded":"2024-02-23T00:51:50.287403+0000","last_fullsized":"2024-02-23T00:51:50.287403+0000","mapping_epoch":580,"log_start":"0'0","ondisk_log_start":"0'0","created":580,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_clean_scrub_stamp":"2024-02-23T00:51:49.259744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"233.15","version":"0'0","reported_seq":10,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.283497+0000","last_change":"2024-02-23T00:51:50.283028+0000","last_active":"2024-02-23T00:51:50.283497+0000","last_peered":"2024-02-23T00:51:50.283497+0000","last_clean":"2024-02-23T00:51:50.283497+0000","last_became_active":"2024-02-23T00:51:50.282681+0000","last_became_peered":"2024-02-23T00:51:50.282681+0000","last_unstale":"2024-02-23T00:51:50.283497+0000","last_undegraded":"2024-02-23T00:51:50.283497+0000","last_fullsized":"2024-02-23T00:51:50.283497+0000","mapping_epoch":580,"log_start":"0'0","ondisk_log_start":"0'0","created":580,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_clean_scrub_stamp":"2024-02-23T00:51:49.259744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"233.14","version":"0'0","reported_seq":9,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.282863+0000","last_change":"2024-02-23T00:51:50.282863+0000","last_active":"2024-02-23T00:51:50.282863+0000","last_peered":"2024-02-23T00:51:50.282863+0000","last_clean":"2024-02-23T00:51:50.282863+0000","last_became_active":"2024-02-23T00:51:50.282169+0000","last_became_peered":"2024-02-23T00:51:50.282169+0000","last_unstale":"2024-02-23T00:51:50.282863+0000","last_undegraded":"2024-02-23T00:51:50.282863+0000","last_fullsized":"2024-02-23T00:51:50.282863+0000","mapping_epoch":580,"log_start":"0'0","ondisk_log_start":"0'0","created":580,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_clean_scrub_stamp":"2024-02-23T00:51:49.259744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"233.13","version":"0'0","reported_seq":10,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.286428+0000","last_change":"2024-02-23T00:51:50.285836+0000","last_active":"2024-02-23T00:51:50.286428+0000","last_peered":"2024-02-23T00:51:50.286428+0000","last_clean":"2024-02-23T00:51:50.286428+0000","last_became_active":"2024-02-23T00:51:50.285430+0000","last_became_peered":"2024-02-23T00:51:50.285430+0000","last_unstale":"2024-02-23T00:51:50.286428+0000","last_undegraded":"2024-02-23T00:51:50.286428+0000","last_fullsized":"2024-02-23T00:51:50.286428+0000","mapping_epoch":580,"log_start":"0'0","ondisk_log_start":"0'0","created":580,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_clean_scrub_stamp":"2024-02-23T00:51:49.259744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"233.12","version":"0'0","reported_seq":10,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.282329+0000","last_change":"2024-02-23T00:51:50.280887+0000","last_active":"2024-02-23T00:51:50.282329+0000","last_peered":"2024-02-23T00:51:50.282329+0000","last_clean":"2024-02-23T00:51:50.282329+0000","last_became_active":"2024-02-23T00:51:50.280638+0000","last_became_peered":"2024-02-23T00:51:50.280638+0000","last_unstale":"2024-02-23T00:51:50.282329+0000","last_undegraded":"2024-02-23T00:51:50.282329+0000","last_fullsized":"2024-02-23T00:51:50.282329+0000","mapping_epoch":580,"log_start":"0'0","ondisk_log_start":"0'0","created":580,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_clean_scrub_stamp":"2024-02-23T00:51:49.259744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"233.11","version":"0'0","reported_seq":9,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.281721+0000","last_change":"2024-02-23T00:51:50.281721+0000","last_active":"2024-02-23T00:51:50.281721+0000","last_peered":"2024-02-23T00:51:50.281721+0000","last_clean":"2024-02-23T00:51:50.281721+0000","last_became_active":"2024-02-23T00:51:50.281422+0000","last_became_peered":"2024-02-23T00:51:50.281422+0000","last_unstale":"2024-02-23T00:51:50.281721+0000","last_undegraded":"2024-02-23T00:51:50.281721+0000","last_fullsized":"2024-02-23T00:51:50.281721+0000","mapping_epoch":580,"log_start":"0'0","ondisk_log_start":"0'0","created":580,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_clean_scrub_stamp":"2024-02-23T00:51:49.259744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"233.10","version":"0'0","reported_seq":9,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.277483+0000","last_change":"2024-02-23T00:51:50.277483+0000","last_active":"2024-02-23T00:51:50.277483+0000","last_peered":"2024-02-23T00:51:50.277483+0000","last_clean":"2024-02-23T00:51:50.277483+0000","last_became_active":"2024-02-23T00:51:50.277174+0000","last_became_peered":"2024-02-23T00:51:50.277174+0000","last_unstale":"2024-02-23T00:51:50.277483+0000","last_undegraded":"2024-02-23T00:51:50.277483+0000","last_fullsized":"2024-02-23T00:51:50.277483+0000","mapping_epoch":580,"log_start":"0'0","ondisk_log_start":"0'0","created":580,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_clean_scrub_stamp":"2024-02-23T00:51:49.259744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"233.6","version":"0'0","reported_seq":9,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.285751+0000","last_change":"2024-02-23T00:51:50.285751+0000","last_active":"2024-02-23T00:51:50.285751+0000","last_peered":"2024-02-23T00:51:50.285751+0000","last_clean":"2024-02-23T00:51:50.285751+0000","last_became_active":"2024-02-23T00:51:50.285453+0000","last_became_peered":"2024-02-23T00:51:50.285453+0000","last_unstale":"2024-02-23T00:51:50.285751+0000","last_undegraded":"2024-02-23T00:51:50.285751+0000","last_fullsized":"2024-02-23T00:51:50.285751+0000","mapping_epoch":580,"log_start":"0'0","ondisk_log_start":"0'0","created":580,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_clean_scrub_stamp":"2024-02-23T00:51:49.259744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"232.7","version":"0'0","reported_seq":11,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.269476+0000","last_change":"2024-02-23T00:51:49.274035+0000","last_active":"2024-02-23T00:51:50.269476+0000","last_peered":"2024-02-23T00:51:50.269476+0000","last_clean":"2024-02-23T00:51:50.269476+0000","last_became_active":"2024-02-23T00:51:49.273661+0000","last_became_peered":"2024-02-23T00:51:49.273661+0000","last_unstale":"2024-02-23T00:51:50.269476+0000","last_undegraded":"2024-02-23T00:51:50.269476+0000","last_fullsized":"2024-02-23T00:51:50.269476+0000","mapping_epoch":579,"log_start":"0'0","ondisk_log_start":"0'0","created":579,"last_epoch_clean":580,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:48.255367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:48.255367+0000","last_clean_scrub_stamp":"2024-02-23T00:51:48.255367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"233.7","version":"0'0","reported_seq":9,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.285442+0000","last_change":"2024-02-23T00:51:50.285442+0000","last_active":"2024-02-23T00:51:50.285442+0000","last_peered":"2024-02-23T00:51:50.285442+0000","last_clean":"2024-02-23T00:51:50.285442+0000","last_became_active":"2024-02-23T00:51:50.285122+0000","last_became_peered":"2024-02-23T00:51:50.285122+0000","last_unstale":"2024-02-23T00:51:50.285442+0000","last_undegraded":"2024-02-23T00:51:50.285442+0000","last_fullsized":"2024-02-23T00:51:50.285442+0000","mapping_epoch":580,"log_start":"0'0","ondisk_log_start":"0'0","created":580,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_clean_scrub_stamp":"2024-02-23T00:51:49.259744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"232.6","version":"0'0","reported_seq":11,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.268076+0000","last_change":"2024-02-23T00:51:49.273499+0000","last_active":"2024-02-23T00:51:50.268076+0000","last_peered":"2024-02-23T00:51:50.268076+0000","last_clean":"2024-02-23T00:51:50.268076+0000","last_became_active":"2024-02-23T00:51:49.273156+0000","last_became_peered":"2024-02-23T00:51:49.273156+0000","last_unstale":"2024-02-23T00:51:50.268076+0000","last_undegraded":"2024-02-23T00:51:50.268076+0000","last_fullsized":"2024-02-23T00:51:50.268076+0000","mapping_epoch":579,"log_start":"0'0","ondisk_log_start":"0'0","created":579,"last_epoch_clean":580,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:48.255367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:48.255367+0000","last_clean_scrub_stamp":"2024-02-23T00:51:48.255367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"233.4","version":"0'0","reported_seq":9,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.282878+0000","last_change":"2024-02-23T00:51:50.282878+0000","last_active":"2024-02-23T00:51:50.282878+0000","last_peered":"2024-02-23T00:51:50.282878+0000","last_clean":"2024-02-23T00:51:50.282878+0000","last_became_active":"2024-02-23T00:51:50.282568+0000","last_became_peered":"2024-02-23T00:51:50.282568+0000","last_unstale":"2024-02-23T00:51:50.282878+0000","last_undegraded":"2024-02-23T00:51:50.282878+0000","last_fullsized":"2024-02-23T00:51:50.282878+0000","mapping_epoch":580,"log_start":"0'0","ondisk_log_start":"0'0","created":580,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_clean_scrub_stamp":"2024-02-23T00:51:49.259744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"232.5","version":"0'0","reported_seq":10,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.274744+0000","last_change":"2024-02-23T00:51:49.274188+0000","last_active":"2024-02-23T00:51:49.274744+0000","last_peered":"2024-02-23T00:51:49.274744+0000","last_clean":"2024-02-23T00:51:49.274744+0000","last_became_active":"2024-02-23T00:51:49.273783+0000","last_became_peered":"2024-02-23T00:51:49.273783+0000","last_unstale":"2024-02-23T00:51:49.274744+0000","last_undegraded":"2024-02-23T00:51:49.274744+0000","last_fullsized":"2024-02-23T00:51:49.274744+0000","mapping_epoch":579,"log_start":"0'0","ondisk_log_start":"0'0","created":579,"last_epoch_clean":580,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:48.255367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:48.255367+0000","last_clean_scrub_stamp":"2024-02-23T00:51:48.255367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"233.5","version":"0'0","reported_seq":9,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.287526+0000","last_change":"2024-02-23T00:51:50.287526+0000","last_active":"2024-02-23T00:51:50.287526+0000","last_peered":"2024-02-23T00:51:50.287526+0000","last_clean":"2024-02-23T00:51:50.287526+0000","last_became_active":"2024-02-23T00:51:50.287203+0000","last_became_peered":"2024-02-23T00:51:50.287203+0000","last_unstale":"2024-02-23T00:51:50.287526+0000","last_undegraded":"2024-02-23T00:51:50.287526+0000","last_fullsized":"2024-02-23T00:51:50.287526+0000","mapping_epoch":580,"log_start":"0'0","ondisk_log_start":"0'0","created":580,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_clean_scrub_stamp":"2024-02-23T00:51:49.259744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"232.4","version":"0'0","reported_seq":10,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.278251+0000","last_change":"2024-02-23T00:51:49.278028+0000","last_active":"2024-02-23T00:51:49.278251+0000","last_peered":"2024-02-23T00:51:49.278251+0000","last_clean":"2024-02-23T00:51:49.278251+0000","last_became_active":"2024-02-23T00:51:49.277723+0000","last_became_peered":"2024-02-23T00:51:49.277723+0000","last_unstale":"2024-02-23T00:51:49.278251+0000","last_undegraded":"2024-02-23T00:51:49.278251+0000","last_fullsized":"2024-02-23T00:51:49.278251+0000","mapping_epoch":579,"log_start":"0'0","ondisk_log_start":"0'0","created":579,"last_epoch_clean":580,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:48.255367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:48.255367+0000","last_clean_scrub_stamp":"2024-02-23T00:51:48.255367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"233.2","version":"0'0","reported_seq":9,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.285749+0000","last_change":"2024-02-23T00:51:50.285749+0000","last_active":"2024-02-23T00:51:50.285749+0000","last_peered":"2024-02-23T00:51:50.285749+0000","last_clean":"2024-02-23T00:51:50.285749+0000","last_became_active":"2024-02-23T00:51:50.285444+0000","last_became_peered":"2024-02-23T00:51:50.285444+0000","last_unstale":"2024-02-23T00:51:50.285749+0000","last_undegraded":"2024-02-23T00:51:50.285749+0000","last_fullsized":"2024-02-23T00:51:50.285749+0000","mapping_epoch":580,"log_start":"0'0","ondisk_log_start":"0'0","created":580,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_clean_scrub_stamp":"2024-02-23T00:51:49.259744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,4],"acting":[1,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"232.3","version":"0'0","reported_seq":10,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.275648+0000","last_change":"2024-02-23T00:51:49.274808+0000","last_active":"2024-02-23T00:51:49.275648+0000","last_peered":"2024-02-23T00:51:49.275648+0000","last_clean":"2024-02-23T00:51:49.275648+0000","last_became_active":"2024-02-23T00:51:49.274493+0000","last_became_peered":"2024-02-23T00:51:49.274493+0000","last_unstale":"2024-02-23T00:51:49.275648+0000","last_undegraded":"2024-02-23T00:51:49.275648+0000","last_fullsized":"2024-02-23T00:51:49.275648+0000","mapping_epoch":579,"log_start":"0'0","ondisk_log_start":"0'0","created":579,"last_epoch_clean":580,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:48.255367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:48.255367+0000","last_clean_scrub_stamp":"2024-02-23T00:51:48.255367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"233.3","version":"0'0","reported_seq":10,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.282415+0000","last_change":"2024-02-23T00:51:50.280652+0000","last_active":"2024-02-23T00:51:50.282415+0000","last_peered":"2024-02-23T00:51:50.282415+0000","last_clean":"2024-02-23T00:51:50.282415+0000","last_became_active":"2024-02-23T00:51:50.280354+0000","last_became_peered":"2024-02-23T00:51:50.280354+0000","last_unstale":"2024-02-23T00:51:50.282415+0000","last_undegraded":"2024-02-23T00:51:50.282415+0000","last_fullsized":"2024-02-23T00:51:50.282415+0000","mapping_epoch":580,"log_start":"0'0","ondisk_log_start":"0'0","created":580,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_clean_scrub_stamp":"2024-02-23T00:51:49.259744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"232.2","version":"0'0","reported_seq":10,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.277384+0000","last_change":"2024-02-23T00:51:49.277182+0000","last_active":"2024-02-23T00:51:49.277384+0000","last_peered":"2024-02-23T00:51:49.277384+0000","last_clean":"2024-02-23T00:51:49.277384+0000","last_became_active":"2024-02-23T00:51:49.276860+0000","last_became_peered":"2024-02-23T00:51:49.276860+0000","last_unstale":"2024-02-23T00:51:49.277384+0000","last_undegraded":"2024-02-23T00:51:49.277384+0000","last_fullsized":"2024-02-23T00:51:49.277384+0000","mapping_epoch":579,"log_start":"0'0","ondisk_log_start":"0'0","created":579,"last_epoch_clean":580,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:48.255367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:48.255367+0000","last_clean_scrub_stamp":"2024-02-23T00:51:48.255367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"233.0","version":"0'0","reported_seq":9,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.282125+0000","last_change":"2024-02-23T00:51:50.282125+0000","last_active":"2024-02-23T00:51:50.282125+0000","last_peered":"2024-02-23T00:51:50.282125+0000","last_clean":"2024-02-23T00:51:50.282125+0000","last_became_active":"2024-02-23T00:51:50.281839+0000","last_became_peered":"2024-02-23T00:51:50.281839+0000","last_unstale":"2024-02-23T00:51:50.282125+0000","last_undegraded":"2024-02-23T00:51:50.282125+0000","last_fullsized":"2024-02-23T00:51:50.282125+0000","mapping_epoch":580,"log_start":"0'0","ondisk_log_start":"0'0","created":580,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_clean_scrub_stamp":"2024-02-23T00:51:49.259744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"232.1","version":"0'0","reported_seq":11,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.271857+0000","last_change":"2024-02-23T00:51:49.269776+0000","last_active":"2024-02-23T00:51:50.271857+0000","last_peered":"2024-02-23T00:51:50.271857+0000","last_clean":"2024-02-23T00:51:50.271857+0000","last_became_active":"2024-02-23T00:51:49.269425+0000","last_became_peered":"2024-02-23T00:51:49.269425+0000","last_unstale":"2024-02-23T00:51:50.271857+0000","last_undegraded":"2024-02-23T00:51:50.271857+0000","last_fullsized":"2024-02-23T00:51:50.271857+0000","mapping_epoch":579,"log_start":"0'0","ondisk_log_start":"0'0","created":579,"last_epoch_clean":580,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:48.255367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:48.255367+0000","last_clean_scrub_stamp":"2024-02-23T00:51:48.255367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"233.1","version":"0'0","reported_seq":10,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.278903+0000","last_change":"2024-02-23T00:51:50.278263+0000","last_active":"2024-02-23T00:51:50.278903+0000","last_peered":"2024-02-23T00:51:50.278903+0000","last_clean":"2024-02-23T00:51:50.278903+0000","last_became_active":"2024-02-23T00:51:50.277935+0000","last_became_peered":"2024-02-23T00:51:50.277935+0000","last_unstale":"2024-02-23T00:51:50.278903+0000","last_undegraded":"2024-02-23T00:51:50.278903+0000","last_fullsized":"2024-02-23T00:51:50.278903+0000","mapping_epoch":580,"log_start":"0'0","ondisk_log_start":"0'0","created":580,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_clean_scrub_stamp":"2024-02-23T00:51:49.259744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3],"acting":[4,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"232.0","version":"0'0","reported_seq":10,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.277623+0000","last_change":"2024-02-23T00:51:49.276179+0000","last_active":"2024-02-23T00:51:49.277623+0000","last_peered":"2024-02-23T00:51:49.277623+0000","last_clean":"2024-02-23T00:51:49.277623+0000","last_became_active":"2024-02-23T00:51:49.275870+0000","last_became_peered":"2024-02-23T00:51:49.275870+0000","last_unstale":"2024-02-23T00:51:49.277623+0000","last_undegraded":"2024-02-23T00:51:49.277623+0000","last_fullsized":"2024-02-23T00:51:49.277623+0000","mapping_epoch":579,"log_start":"0'0","ondisk_log_start":"0'0","created":579,"last_epoch_clean":580,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:48.255367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:48.255367+0000","last_clean_scrub_stamp":"2024-02-23T00:51:48.255367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,5],"acting":[7,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"233.b","version":"0'0","reported_seq":9,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.276109+0000","last_change":"2024-02-23T00:51:50.276109+0000","last_active":"2024-02-23T00:51:50.276109+0000","last_peered":"2024-02-23T00:51:50.276109+0000","last_clean":"2024-02-23T00:51:50.276109+0000","last_became_active":"2024-02-23T00:51:50.275543+0000","last_became_peered":"2024-02-23T00:51:50.275543+0000","last_unstale":"2024-02-23T00:51:50.276109+0000","last_undegraded":"2024-02-23T00:51:50.276109+0000","last_fullsized":"2024-02-23T00:51:50.276109+0000","mapping_epoch":580,"log_start":"0'0","ondisk_log_start":"0'0","created":580,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_clean_scrub_stamp":"2024-02-23T00:51:49.259744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"229.7","version":"0'0","reported_seq":25,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.275536+0000","last_change":"2024-02-23T00:51:34.271578+0000","last_active":"2024-02-23T00:51:49.275536+0000","last_peered":"2024-02-23T00:51:49.275536+0000","last_clean":"2024-02-23T00:51:49.275536+0000","last_became_active":"2024-02-23T00:51:34.271103+0000","last_became_peered":"2024-02-23T00:51:34.271103+0000","last_unstale":"2024-02-23T00:51:49.275536+0000","last_undegraded":"2024-02-23T00:51:49.275536+0000","last_fullsized":"2024-02-23T00:51:49.275536+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":564,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_clean_scrub_stamp":"2024-02-23T00:51:33.255429+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"233.a","version":"0'0","reported_seq":10,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.282311+0000","last_change":"2024-02-23T00:51:50.280638+0000","last_active":"2024-02-23T00:51:50.282311+0000","last_peered":"2024-02-23T00:51:50.282311+0000","last_clean":"2024-02-23T00:51:50.282311+0000","last_became_active":"2024-02-23T00:51:50.280263+0000","last_became_peered":"2024-02-23T00:51:50.280263+0000","last_unstale":"2024-02-23T00:51:50.282311+0000","last_undegraded":"2024-02-23T00:51:50.282311+0000","last_fullsized":"2024-02-23T00:51:50.282311+0000","mapping_epoch":580,"log_start":"0'0","ondisk_log_start":"0'0","created":580,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_clean_scrub_stamp":"2024-02-23T00:51:49.259744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"229.6","version":"0'0","reported_seq":26,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.272763+0000","last_change":"2024-02-23T00:51:34.269550+0000","last_active":"2024-02-23T00:51:50.272763+0000","last_peered":"2024-02-23T00:51:50.272763+0000","last_clean":"2024-02-23T00:51:50.272763+0000","last_became_active":"2024-02-23T00:51:34.269208+0000","last_became_peered":"2024-02-23T00:51:34.269208+0000","last_unstale":"2024-02-23T00:51:50.272763+0000","last_undegraded":"2024-02-23T00:51:50.272763+0000","last_fullsized":"2024-02-23T00:51:50.272763+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":564,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_clean_scrub_stamp":"2024-02-23T00:51:33.255429+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"233.9","version":"0'0","reported_seq":9,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.284980+0000","last_change":"2024-02-23T00:51:50.284980+0000","last_active":"2024-02-23T00:51:50.284980+0000","last_peered":"2024-02-23T00:51:50.284980+0000","last_clean":"2024-02-23T00:51:50.284980+0000","last_became_active":"2024-02-23T00:51:50.284704+0000","last_became_peered":"2024-02-23T00:51:50.284704+0000","last_unstale":"2024-02-23T00:51:50.284980+0000","last_undegraded":"2024-02-23T00:51:50.284980+0000","last_fullsized":"2024-02-23T00:51:50.284980+0000","mapping_epoch":580,"log_start":"0'0","ondisk_log_start":"0'0","created":580,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_clean_scrub_stamp":"2024-02-23T00:51:49.259744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"229.5","version":"0'0","reported_seq":25,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.275409+0000","last_change":"2024-02-23T00:51:34.269112+0000","last_active":"2024-02-23T00:51:49.275409+0000","last_peered":"2024-02-23T00:51:49.275409+0000","last_clean":"2024-02-23T00:51:49.275409+0000","last_became_active":"2024-02-23T00:51:34.268826+0000","last_became_peered":"2024-02-23T00:51:34.268826+0000","last_unstale":"2024-02-23T00:51:49.275409+0000","last_undegraded":"2024-02-23T00:51:49.275409+0000","last_fullsized":"2024-02-23T00:51:49.275409+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":564,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_clean_scrub_stamp":"2024-02-23T00:51:33.255429+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"233.8","version":"0'0","reported_seq":10,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.286407+0000","last_change":"2024-02-23T00:51:50.285732+0000","last_active":"2024-02-23T00:51:50.286407+0000","last_peered":"2024-02-23T00:51:50.286407+0000","last_clean":"2024-02-23T00:51:50.286407+0000","last_became_active":"2024-02-23T00:51:50.285422+0000","last_became_peered":"2024-02-23T00:51:50.285422+0000","last_unstale":"2024-02-23T00:51:50.286407+0000","last_undegraded":"2024-02-23T00:51:50.286407+0000","last_fullsized":"2024-02-23T00:51:50.286407+0000","mapping_epoch":580,"log_start":"0'0","ondisk_log_start":"0'0","created":580,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_clean_scrub_stamp":"2024-02-23T00:51:49.259744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"229.4","version":"0'0","reported_seq":25,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.273017+0000","last_change":"2024-02-23T00:51:34.271412+0000","last_active":"2024-02-23T00:51:49.273017+0000","last_peered":"2024-02-23T00:51:49.273017+0000","last_clean":"2024-02-23T00:51:49.273017+0000","last_became_active":"2024-02-23T00:51:34.270881+0000","last_became_peered":"2024-02-23T00:51:34.270881+0000","last_unstale":"2024-02-23T00:51:49.273017+0000","last_undegraded":"2024-02-23T00:51:49.273017+0000","last_fullsized":"2024-02-23T00:51:49.273017+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":564,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_clean_scrub_stamp":"2024-02-23T00:51:33.255429+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"233.f","version":"0'0","reported_seq":10,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.283318+0000","last_change":"2024-02-23T00:51:50.278419+0000","last_active":"2024-02-23T00:51:50.283318+0000","last_peered":"2024-02-23T00:51:50.283318+0000","last_clean":"2024-02-23T00:51:50.283318+0000","last_became_active":"2024-02-23T00:51:50.278106+0000","last_became_peered":"2024-02-23T00:51:50.278106+0000","last_unstale":"2024-02-23T00:51:50.283318+0000","last_undegraded":"2024-02-23T00:51:50.283318+0000","last_fullsized":"2024-02-23T00:51:50.283318+0000","mapping_epoch":580,"log_start":"0'0","ondisk_log_start":"0'0","created":580,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_clean_scrub_stamp":"2024-02-23T00:51:49.259744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"229.3","version":"565'4","reported_seq":30,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.270997+0000","last_change":"2024-02-23T00:51:34.271564+0000","last_active":"2024-02-23T00:51:49.270997+0000","last_peered":"2024-02-23T00:51:49.270997+0000","last_clean":"2024-02-23T00:51:49.270997+0000","last_became_active":"2024-02-23T00:51:34.270817+0000","last_became_peered":"2024-02-23T00:51:34.270817+0000","last_unstale":"2024-02-23T00:51:49.270997+0000","last_undegraded":"2024-02-23T00:51:49.270997+0000","last_fullsized":"2024-02-23T00:51:49.270997+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":564,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_clean_scrub_stamp":"2024-02-23T00:51:33.255429+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"233.e","version":"0'0","reported_seq":10,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.282919+0000","last_change":"2024-02-23T00:51:50.281608+0000","last_active":"2024-02-23T00:51:50.282919+0000","last_peered":"2024-02-23T00:51:50.282919+0000","last_clean":"2024-02-23T00:51:50.282919+0000","last_became_active":"2024-02-23T00:51:50.281326+0000","last_became_peered":"2024-02-23T00:51:50.281326+0000","last_unstale":"2024-02-23T00:51:50.282919+0000","last_undegraded":"2024-02-23T00:51:50.282919+0000","last_fullsized":"2024-02-23T00:51:50.282919+0000","mapping_epoch":580,"log_start":"0'0","ondisk_log_start":"0'0","created":580,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_clean_scrub_stamp":"2024-02-23T00:51:49.259744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,5],"acting":[4,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"229.2","version":"0'0","reported_seq":25,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.274699+0000","last_change":"2024-02-23T00:51:34.271256+0000","last_active":"2024-02-23T00:51:49.274699+0000","last_peered":"2024-02-23T00:51:49.274699+0000","last_clean":"2024-02-23T00:51:49.274699+0000","last_became_active":"2024-02-23T00:51:34.270941+0000","last_became_peered":"2024-02-23T00:51:34.270941+0000","last_unstale":"2024-02-23T00:51:49.274699+0000","last_undegraded":"2024-02-23T00:51:49.274699+0000","last_fullsized":"2024-02-23T00:51:49.274699+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":564,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_clean_scrub_stamp":"2024-02-23T00:51:33.255429+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"233.d","version":"0'0","reported_seq":10,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.283356+0000","last_change":"2024-02-23T00:51:50.279259+0000","last_active":"2024-02-23T00:51:50.283356+0000","last_peered":"2024-02-23T00:51:50.283356+0000","last_clean":"2024-02-23T00:51:50.283356+0000","last_became_active":"2024-02-23T00:51:50.278971+0000","last_became_peered":"2024-02-23T00:51:50.278971+0000","last_unstale":"2024-02-23T00:51:50.283356+0000","last_undegraded":"2024-02-23T00:51:50.283356+0000","last_fullsized":"2024-02-23T00:51:50.283356+0000","mapping_epoch":580,"log_start":"0'0","ondisk_log_start":"0'0","created":580,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_clean_scrub_stamp":"2024-02-23T00:51:49.259744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"229.1","version":"0'0","reported_seq":26,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.271855+0000","last_change":"2024-02-23T00:51:34.526061+0000","last_active":"2024-02-23T00:51:50.271855+0000","last_peered":"2024-02-23T00:51:50.271855+0000","last_clean":"2024-02-23T00:51:50.271855+0000","last_became_active":"2024-02-23T00:51:34.525742+0000","last_became_peered":"2024-02-23T00:51:34.525742+0000","last_unstale":"2024-02-23T00:51:50.271855+0000","last_undegraded":"2024-02-23T00:51:50.271855+0000","last_fullsized":"2024-02-23T00:51:50.271855+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":564,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_clean_scrub_stamp":"2024-02-23T00:51:33.255429+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"233.c","version":"0'0","reported_seq":9,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.285504+0000","last_change":"2024-02-23T00:51:50.285504+0000","last_active":"2024-02-23T00:51:50.285504+0000","last_peered":"2024-02-23T00:51:50.285504+0000","last_clean":"2024-02-23T00:51:50.285504+0000","last_became_active":"2024-02-23T00:51:50.285194+0000","last_became_peered":"2024-02-23T00:51:50.285194+0000","last_unstale":"2024-02-23T00:51:50.285504+0000","last_undegraded":"2024-02-23T00:51:50.285504+0000","last_fullsized":"2024-02-23T00:51:50.285504+0000","mapping_epoch":580,"log_start":"0'0","ondisk_log_start":"0'0","created":580,"last_epoch_clean":581,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:49.259744+0000","last_clean_scrub_stamp":"2024-02-23T00:51:49.259744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"229.0","version":"0'0","reported_seq":26,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.267244+0000","last_change":"2024-02-23T00:51:34.526282+0000","last_active":"2024-02-23T00:51:50.267244+0000","last_peered":"2024-02-23T00:51:50.267244+0000","last_clean":"2024-02-23T00:51:50.267244+0000","last_became_active":"2024-02-23T00:51:34.525872+0000","last_became_peered":"2024-02-23T00:51:34.525872+0000","last_unstale":"2024-02-23T00:51:50.267244+0000","last_undegraded":"2024-02-23T00:51:50.267244+0000","last_fullsized":"2024-02-23T00:51:50.267244+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":564,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_clean_scrub_stamp":"2024-02-23T00:51:33.255429+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"466'9","reported_seq":709,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.271193+0000","last_change":"2024-02-23T00:49:43.188066+0000","last_active":"2024-02-23T00:51:49.271193+0000","last_peered":"2024-02-23T00:51:49.271193+0000","last_clean":"2024-02-23T00:51:49.271193+0000","last_became_active":"2024-02-23T00:39:18.793414+0000","last_became_peered":"2024-02-23T00:39:18.793414+0000","last_unstale":"2024-02-23T00:51:49.271193+0000","last_undegraded":"2024-02-23T00:51:49.271193+0000","last_fullsized":"2024-02-23T00:51:49.271193+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'9","last_scrub_stamp":"2024-02-23T00:49:43.187989+0000","last_deep_scrub":"466'9","last_deep_scrub_stamp":"2024-02-23T00:49:43.187989+0000","last_clean_scrub_stamp":"2024-02-23T00:49:43.187989+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"470'42","reported_seq":626,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.275301+0000","last_change":"2024-02-23T00:49:47.653272+0000","last_active":"2024-02-23T00:51:49.275301+0000","last_peered":"2024-02-23T00:51:49.275301+0000","last_clean":"2024-02-23T00:51:49.275301+0000","last_became_active":"2024-02-23T00:39:16.859401+0000","last_became_peered":"2024-02-23T00:39:16.859401+0000","last_unstale":"2024-02-23T00:51:49.275301+0000","last_undegraded":"2024-02-23T00:51:49.275301+0000","last_fullsized":"2024-02-23T00:51:49.275301+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"470'42","last_scrub_stamp":"2024-02-23T00:49:47.653176+0000","last_deep_scrub":"470'42","last_deep_scrub_stamp":"2024-02-23T00:49:47.653176+0000","last_clean_scrub_stamp":"2024-02-23T00:49:47.653176+0000","log_size":42,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":228,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.268158+0000","last_change":"2024-02-23T00:51:36.656632+0000","last_active":"2024-02-23T00:51:50.268158+0000","last_peered":"2024-02-23T00:51:50.268158+0000","last_clean":"2024-02-23T00:51:50.268158+0000","last_became_active":"2024-02-23T00:47:28.220876+0000","last_became_peered":"2024-02-23T00:47:28.220876+0000","last_unstale":"2024-02-23T00:51:50.268158+0000","last_undegraded":"2024-02-23T00:51:50.268158+0000","last_fullsized":"2024-02-23T00:51:50.268158+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:36.656571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:04.330426+0000","last_clean_scrub_stamp":"2024-02-23T00:51:36.656571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":564,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.274789+0000","last_change":"2024-02-23T00:50:14.801990+0000","last_active":"2024-02-23T00:51:49.274789+0000","last_peered":"2024-02-23T00:51:49.274789+0000","last_clean":"2024-02-23T00:51:49.274789+0000","last_became_active":"2024-02-23T00:43:32.282545+0000","last_became_peered":"2024-02-23T00:43:32.282545+0000","last_unstale":"2024-02-23T00:51:49.274789+0000","last_undegraded":"2024-02-23T00:51:49.274789+0000","last_fullsized":"2024-02-23T00:51:49.274789+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:14.801915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:50:14.801915+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":493,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.275705+0000","last_change":"2024-02-23T00:50:23.727290+0000","last_active":"2024-02-23T00:51:49.275705+0000","last_peered":"2024-02-23T00:51:49.275705+0000","last_clean":"2024-02-23T00:51:49.275705+0000","last_became_active":"2024-02-23T00:42:58.828846+0000","last_became_peered":"2024-02-23T00:42:58.828846+0000","last_unstale":"2024-02-23T00:51:49.275705+0000","last_undegraded":"2024-02-23T00:51:49.275705+0000","last_fullsized":"2024-02-23T00:51:49.275705+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:23.727173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:23.727173+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"457'21","reported_seq":596,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.272997+0000","last_change":"2024-02-23T00:51:46.661946+0000","last_active":"2024-02-23T00:51:50.272997+0000","last_peered":"2024-02-23T00:51:50.272997+0000","last_clean":"2024-02-23T00:51:50.272997+0000","last_became_active":"2024-02-23T00:39:16.783701+0000","last_became_peered":"2024-02-23T00:39:16.783701+0000","last_unstale":"2024-02-23T00:51:50.272997+0000","last_undegraded":"2024-02-23T00:51:50.272997+0000","last_fullsized":"2024-02-23T00:51:50.272997+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-23T00:51:46.661879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:46.661879+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":542,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.273451+0000","last_change":"2024-02-23T00:49:04.528117+0000","last_active":"2024-02-23T00:51:49.273451+0000","last_peered":"2024-02-23T00:51:49.273451+0000","last_clean":"2024-02-23T00:51:49.273451+0000","last_became_active":"2024-02-23T00:39:14.772493+0000","last_became_peered":"2024-02-23T00:39:14.772493+0000","last_unstale":"2024-02-23T00:51:49.273451+0000","last_undegraded":"2024-02-23T00:51:49.273451+0000","last_fullsized":"2024-02-23T00:51:49.273451+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:04.528006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:04.528006+0000","last_clean_scrub_stamp":"2024-02-23T00:49:04.528006+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":224,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.273086+0000","last_change":"2024-02-23T00:51:44.698156+0000","last_active":"2024-02-23T00:51:50.273086+0000","last_peered":"2024-02-23T00:51:50.273086+0000","last_clean":"2024-02-23T00:51:50.273086+0000","last_became_active":"2024-02-23T00:47:28.231567+0000","last_became_peered":"2024-02-23T00:47:28.231567+0000","last_unstale":"2024-02-23T00:51:50.273086+0000","last_undegraded":"2024-02-23T00:51:50.273086+0000","last_fullsized":"2024-02-23T00:51:50.273086+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:44.698067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:08.335330+0000","last_clean_scrub_stamp":"2024-02-23T00:51:44.698067+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":574,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.272115+0000","last_change":"2024-02-23T00:51:11.526671+0000","last_active":"2024-02-23T00:51:50.272115+0000","last_peered":"2024-02-23T00:51:50.272115+0000","last_clean":"2024-02-23T00:51:50.272115+0000","last_became_active":"2024-02-23T00:47:28.214157+0000","last_became_peered":"2024-02-23T00:47:28.214157+0000","last_unstale":"2024-02-23T00:51:50.272115+0000","last_undegraded":"2024-02-23T00:51:50.272115+0000","last_fullsized":"2024-02-23T00:51:50.272115+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:11.526584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:13.867716+0000","last_clean_scrub_stamp":"2024-02-23T00:51:11.526584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":494,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.275415+0000","last_change":"2024-02-23T00:50:32.622097+0000","last_active":"2024-02-23T00:51:50.275415+0000","last_peered":"2024-02-23T00:51:50.275415+0000","last_clean":"2024-02-23T00:51:50.275415+0000","last_became_active":"2024-02-23T00:42:58.867305+0000","last_became_peered":"2024-02-23T00:42:58.867305+0000","last_unstale":"2024-02-23T00:51:50.275415+0000","last_undegraded":"2024-02-23T00:51:50.275415+0000","last_fullsized":"2024-02-23T00:51:50.275415+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:32.622026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:32.622026+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"457'51","reported_seq":633,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.267179+0000","last_change":"2024-02-23T00:51:13.326646+0000","last_active":"2024-02-23T00:51:49.267179+0000","last_peered":"2024-02-23T00:51:49.267179+0000","last_clean":"2024-02-23T00:51:49.267179+0000","last_became_active":"2024-02-23T00:39:16.857413+0000","last_became_peered":"2024-02-23T00:39:16.857413+0000","last_unstale":"2024-02-23T00:51:49.267179+0000","last_undegraded":"2024-02-23T00:51:49.267179+0000","last_fullsized":"2024-02-23T00:51:49.267179+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'51","last_scrub_stamp":"2024-02-23T00:51:13.326535+0000","last_deep_scrub":"88'35","last_deep_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_clean_scrub_stamp":"2024-02-23T00:51:13.326535+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":68,"num_read_kb":56,"num_write":42,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":225,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.276421+0000","last_change":"2024-02-23T00:50:57.770551+0000","last_active":"2024-02-23T00:51:49.276421+0000","last_peered":"2024-02-23T00:51:49.276421+0000","last_clean":"2024-02-23T00:51:49.276421+0000","last_became_active":"2024-02-23T00:47:28.231891+0000","last_became_peered":"2024-02-23T00:47:28.231891+0000","last_unstale":"2024-02-23T00:51:49.276421+0000","last_undegraded":"2024-02-23T00:51:49.276421+0000","last_fullsized":"2024-02-23T00:51:49.276421+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:57.770450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_clean_scrub_stamp":"2024-02-23T00:50:57.770450+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":226,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.272903+0000","last_change":"2024-02-23T00:51:36.650367+0000","last_active":"2024-02-23T00:51:50.272903+0000","last_peered":"2024-02-23T00:51:50.272903+0000","last_clean":"2024-02-23T00:51:50.272903+0000","last_became_active":"2024-02-23T00:47:28.220578+0000","last_became_peered":"2024-02-23T00:47:28.220578+0000","last_unstale":"2024-02-23T00:51:50.272903+0000","last_undegraded":"2024-02-23T00:51:50.272903+0000","last_fullsized":"2024-02-23T00:51:50.272903+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:36.650303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:03.682745+0000","last_clean_scrub_stamp":"2024-02-23T00:51:36.650303+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":571,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.270094+0000","last_change":"2024-02-23T00:49:42.150972+0000","last_active":"2024-02-23T00:51:49.270094+0000","last_peered":"2024-02-23T00:51:49.270094+0000","last_clean":"2024-02-23T00:51:49.270094+0000","last_became_active":"2024-02-23T00:47:28.215636+0000","last_became_peered":"2024-02-23T00:47:28.215636+0000","last_unstale":"2024-02-23T00:51:49.270094+0000","last_undegraded":"2024-02-23T00:51:49.270094+0000","last_fullsized":"2024-02-23T00:51:49.270094+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:42.150875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:42.150875+0000","last_clean_scrub_stamp":"2024-02-23T00:49:42.150875+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":484,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.273259+0000","last_change":"2024-02-23T00:49:34.515593+0000","last_active":"2024-02-23T00:51:49.273259+0000","last_peered":"2024-02-23T00:51:49.273259+0000","last_clean":"2024-02-23T00:51:49.273259+0000","last_became_active":"2024-02-23T00:42:57.639013+0000","last_became_peered":"2024-02-23T00:42:57.639013+0000","last_unstale":"2024-02-23T00:51:49.273259+0000","last_undegraded":"2024-02-23T00:51:49.273259+0000","last_fullsized":"2024-02-23T00:51:49.273259+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:34.515496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:49:34.515496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"457'31","reported_seq":647,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.272256+0000","last_change":"2024-02-23T00:49:31.480372+0000","last_active":"2024-02-23T00:51:50.272256+0000","last_peered":"2024-02-23T00:51:50.272256+0000","last_clean":"2024-02-23T00:51:50.272256+0000","last_became_active":"2024-02-23T00:47:28.200653+0000","last_became_peered":"2024-02-23T00:47:28.200653+0000","last_unstale":"2024-02-23T00:51:50.272256+0000","last_undegraded":"2024-02-23T00:51:50.272256+0000","last_fullsized":"2024-02-23T00:51:50.272256+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:49:31.480253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:31.480253+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":223,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.271144+0000","last_change":"2024-02-23T00:51:10.213423+0000","last_active":"2024-02-23T00:51:49.271144+0000","last_peered":"2024-02-23T00:51:49.271144+0000","last_clean":"2024-02-23T00:51:49.271144+0000","last_became_active":"2024-02-23T00:47:28.201926+0000","last_became_peered":"2024-02-23T00:47:28.201926+0000","last_unstale":"2024-02-23T00:51:49.271144+0000","last_undegraded":"2024-02-23T00:51:49.271144+0000","last_fullsized":"2024-02-23T00:51:49.271144+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:10.213337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:10.213337+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":543,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.273658+0000","last_change":"2024-02-23T00:51:09.343951+0000","last_active":"2024-02-23T00:51:49.273658+0000","last_peered":"2024-02-23T00:51:49.273658+0000","last_clean":"2024-02-23T00:51:49.273658+0000","last_became_active":"2024-02-23T00:39:18.799875+0000","last_became_peered":"2024-02-23T00:39:18.799875+0000","last_unstale":"2024-02-23T00:51:49.273658+0000","last_undegraded":"2024-02-23T00:51:49.273658+0000","last_fullsized":"2024-02-23T00:51:49.273658+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:09.343884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:09.343884+0000","last_clean_scrub_stamp":"2024-02-23T00:51:09.343884+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":484,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.274523+0000","last_change":"2024-02-23T00:48:49.610457+0000","last_active":"2024-02-23T00:51:49.274523+0000","last_peered":"2024-02-23T00:51:49.274523+0000","last_clean":"2024-02-23T00:51:49.274523+0000","last_became_active":"2024-02-23T00:42:57.675306+0000","last_became_peered":"2024-02-23T00:42:57.675306+0000","last_unstale":"2024-02-23T00:51:49.274523+0000","last_undegraded":"2024-02-23T00:51:49.274523+0000","last_fullsized":"2024-02-23T00:51:49.274523+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:49.610391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:49.610391+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"457'20","reported_seq":579,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.272378+0000","last_change":"2024-02-23T00:51:01.488450+0000","last_active":"2024-02-23T00:51:50.272378+0000","last_peered":"2024-02-23T00:51:50.272378+0000","last_clean":"2024-02-23T00:51:50.272378+0000","last_became_active":"2024-02-23T00:39:16.777530+0000","last_became_peered":"2024-02-23T00:39:16.777530+0000","last_unstale":"2024-02-23T00:51:50.272378+0000","last_undegraded":"2024-02-23T00:51:50.272378+0000","last_fullsized":"2024-02-23T00:51:50.272378+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'20","last_scrub_stamp":"2024-02-23T00:51:01.488371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:01.488371+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":542,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.273315+0000","last_change":"2024-02-23T00:48:41.719867+0000","last_active":"2024-02-23T00:51:49.273315+0000","last_peered":"2024-02-23T00:51:49.273315+0000","last_clean":"2024-02-23T00:51:49.273315+0000","last_became_active":"2024-02-23T00:39:14.780379+0000","last_became_peered":"2024-02-23T00:39:14.780379+0000","last_unstale":"2024-02-23T00:51:49.273315+0000","last_undegraded":"2024-02-23T00:51:49.273315+0000","last_fullsized":"2024-02-23T00:51:49.273315+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:41.719772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.575242+0000","last_clean_scrub_stamp":"2024-02-23T00:48:41.719772+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"376'9","reported_seq":296,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.272661+0000","last_change":"2024-02-23T00:49:40.180695+0000","last_active":"2024-02-23T00:51:49.272661+0000","last_peered":"2024-02-23T00:51:49.272661+0000","last_clean":"2024-02-23T00:51:49.272661+0000","last_became_active":"2024-02-23T00:47:28.231104+0000","last_became_peered":"2024-02-23T00:47:28.231104+0000","last_unstale":"2024-02-23T00:51:49.272661+0000","last_undegraded":"2024-02-23T00:51:49.272661+0000","last_fullsized":"2024-02-23T00:51:49.272661+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"376'9","last_scrub_stamp":"2024-02-23T00:49:40.180616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:40.180616+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":497,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.267699+0000","last_change":"2024-02-23T00:51:12.335819+0000","last_active":"2024-02-23T00:51:50.267699+0000","last_peered":"2024-02-23T00:51:50.267699+0000","last_clean":"2024-02-23T00:51:50.267699+0000","last_became_active":"2024-02-23T00:42:58.765106+0000","last_became_peered":"2024-02-23T00:42:58.765106+0000","last_unstale":"2024-02-23T00:51:50.267699+0000","last_undegraded":"2024-02-23T00:51:50.267699+0000","last_fullsized":"2024-02-23T00:51:50.267699+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.335752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.335752+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.335752+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"457'27","reported_seq":588,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.275751+0000","last_change":"2024-02-23T00:51:48.537804+0000","last_active":"2024-02-23T00:51:49.275751+0000","last_peered":"2024-02-23T00:51:49.275751+0000","last_clean":"2024-02-23T00:51:49.275751+0000","last_became_active":"2024-02-23T00:39:16.789757+0000","last_became_peered":"2024-02-23T00:39:16.789757+0000","last_unstale":"2024-02-23T00:51:49.275751+0000","last_undegraded":"2024-02-23T00:51:49.275751+0000","last_fullsized":"2024-02-23T00:51:49.275751+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-23T00:51:48.537732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:48.537732+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":547,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.272836+0000","last_change":"2024-02-23T00:51:13.375106+0000","last_active":"2024-02-23T00:51:49.272836+0000","last_peered":"2024-02-23T00:51:49.272836+0000","last_clean":"2024-02-23T00:51:49.272836+0000","last_became_active":"2024-02-23T00:39:14.780526+0000","last_became_peered":"2024-02-23T00:39:14.780526+0000","last_unstale":"2024-02-23T00:51:49.272836+0000","last_undegraded":"2024-02-23T00:51:49.272836+0000","last_fullsized":"2024-02-23T00:51:49.272836+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:13.375039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:13.375039+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":706,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.271645+0000","last_change":"2024-02-23T00:49:40.384676+0000","last_active":"2024-02-23T00:51:50.271645+0000","last_peered":"2024-02-23T00:51:50.271645+0000","last_clean":"2024-02-23T00:51:50.271645+0000","last_became_active":"2024-02-23T00:39:18.800440+0000","last_became_peered":"2024-02-23T00:39:18.800440+0000","last_unstale":"2024-02-23T00:51:50.271645+0000","last_undegraded":"2024-02-23T00:51:50.271645+0000","last_fullsized":"2024-02-23T00:51:50.271645+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:49:40.384608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:40.384608+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":495,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.270563+0000","last_change":"2024-02-23T00:51:38.762620+0000","last_active":"2024-02-23T00:51:50.270563+0000","last_peered":"2024-02-23T00:51:50.270563+0000","last_clean":"2024-02-23T00:51:50.270563+0000","last_became_active":"2024-02-23T00:42:58.833924+0000","last_became_peered":"2024-02-23T00:42:58.833924+0000","last_unstale":"2024-02-23T00:51:50.270563+0000","last_undegraded":"2024-02-23T00:51:50.270563+0000","last_fullsized":"2024-02-23T00:51:50.270563+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:38.762554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:38.762554+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"457'31","reported_seq":647,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.271504+0000","last_change":"2024-02-23T00:51:40.699209+0000","last_active":"2024-02-23T00:51:50.271504+0000","last_peered":"2024-02-23T00:51:50.271504+0000","last_clean":"2024-02-23T00:51:50.271504+0000","last_became_active":"2024-02-23T00:47:28.201225+0000","last_became_peered":"2024-02-23T00:47:28.201225+0000","last_unstale":"2024-02-23T00:51:50.271504+0000","last_undegraded":"2024-02-23T00:51:50.271504+0000","last_fullsized":"2024-02-23T00:51:50.271504+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:51:40.699141+0000","last_deep_scrub":"336'23","last_deep_scrub_stamp":"2024-02-23T00:46:48.320085+0000","last_clean_scrub_stamp":"2024-02-23T00:51:40.699141+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":550,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.270268+0000","last_change":"2024-02-23T00:49:37.331334+0000","last_active":"2024-02-23T00:51:49.270268+0000","last_peered":"2024-02-23T00:51:49.270268+0000","last_clean":"2024-02-23T00:51:49.270268+0000","last_became_active":"2024-02-23T00:39:14.784121+0000","last_became_peered":"2024-02-23T00:39:14.784121+0000","last_unstale":"2024-02-23T00:51:49.270268+0000","last_undegraded":"2024-02-23T00:51:49.270268+0000","last_fullsized":"2024-02-23T00:51:49.270268+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:37.331270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:37.331270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":548,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.275258+0000","last_change":"2024-02-23T00:50:19.827889+0000","last_active":"2024-02-23T00:51:49.275258+0000","last_peered":"2024-02-23T00:51:49.275258+0000","last_clean":"2024-02-23T00:51:49.275258+0000","last_became_active":"2024-02-23T00:39:18.800777+0000","last_became_peered":"2024-02-23T00:39:18.800777+0000","last_unstale":"2024-02-23T00:51:49.275258+0000","last_undegraded":"2024-02-23T00:51:49.275258+0000","last_fullsized":"2024-02-23T00:51:49.275258+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:19.827814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_clean_scrub_stamp":"2024-02-23T00:50:19.827814+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":484,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.271767+0000","last_change":"2024-02-23T00:49:33.529224+0000","last_active":"2024-02-23T00:51:49.271767+0000","last_peered":"2024-02-23T00:51:49.271767+0000","last_clean":"2024-02-23T00:51:49.271767+0000","last_became_active":"2024-02-23T00:42:57.653279+0000","last_became_peered":"2024-02-23T00:42:57.653279+0000","last_unstale":"2024-02-23T00:51:49.271767+0000","last_undegraded":"2024-02-23T00:51:49.271767+0000","last_fullsized":"2024-02-23T00:51:49.271767+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:33.529135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:49:33.529135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":711,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.267458+0000","last_change":"2024-02-23T00:51:34.577307+0000","last_active":"2024-02-23T00:51:50.267458+0000","last_peered":"2024-02-23T00:51:50.267458+0000","last_clean":"2024-02-23T00:51:50.267458+0000","last_became_active":"2024-02-23T00:39:18.798536+0000","last_became_peered":"2024-02-23T00:39:18.798536+0000","last_unstale":"2024-02-23T00:51:50.267458+0000","last_undegraded":"2024-02-23T00:51:50.267458+0000","last_fullsized":"2024-02-23T00:51:50.267458+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:51:34.577222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:51:34.577222+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"457'42","reported_seq":614,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.270709+0000","last_change":"2024-02-23T00:50:24.532387+0000","last_active":"2024-02-23T00:51:50.270709+0000","last_peered":"2024-02-23T00:51:50.270709+0000","last_clean":"2024-02-23T00:51:50.270709+0000","last_became_active":"2024-02-23T00:39:16.780130+0000","last_became_peered":"2024-02-23T00:39:16.780130+0000","last_unstale":"2024-02-23T00:51:50.270709+0000","last_undegraded":"2024-02-23T00:51:50.270709+0000","last_fullsized":"2024-02-23T00:51:50.270709+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'42","last_scrub_stamp":"2024-02-23T00:50:24.532271+0000","last_deep_scrub":"423'28","last_deep_scrub_stamp":"2024-02-23T00:48:17.435906+0000","last_clean_scrub_stamp":"2024-02-23T00:50:24.532271+0000","log_size":42,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":46,"num_write":34,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":566,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.266956+0000","last_change":"2024-02-23T00:50:59.199966+0000","last_active":"2024-02-23T00:51:49.266956+0000","last_peered":"2024-02-23T00:51:49.266956+0000","last_clean":"2024-02-23T00:51:49.266956+0000","last_became_active":"2024-02-23T00:47:28.237083+0000","last_became_peered":"2024-02-23T00:47:28.237083+0000","last_unstale":"2024-02-23T00:51:49.266956+0000","last_undegraded":"2024-02-23T00:51:49.266956+0000","last_fullsized":"2024-02-23T00:51:49.266956+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:59.199899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:51.391939+0000","last_clean_scrub_stamp":"2024-02-23T00:50:59.199899+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":563,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.271510+0000","last_change":"2024-02-23T00:49:41.163033+0000","last_active":"2024-02-23T00:51:49.271510+0000","last_peered":"2024-02-23T00:51:49.271510+0000","last_clean":"2024-02-23T00:51:49.271510+0000","last_became_active":"2024-02-23T00:47:30.187386+0000","last_became_peered":"2024-02-23T00:47:30.187386+0000","last_unstale":"2024-02-23T00:51:49.271510+0000","last_undegraded":"2024-02-23T00:51:49.271510+0000","last_fullsized":"2024-02-23T00:51:49.271510+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:41.162895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:46.686764+0000","last_clean_scrub_stamp":"2024-02-23T00:49:41.162895+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":502,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.275413+0000","last_change":"2024-02-23T00:51:03.875340+0000","last_active":"2024-02-23T00:51:49.275413+0000","last_peered":"2024-02-23T00:51:49.275413+0000","last_clean":"2024-02-23T00:51:49.275413+0000","last_became_active":"2024-02-23T00:47:28.212307+0000","last_became_peered":"2024-02-23T00:47:28.212307+0000","last_unstale":"2024-02-23T00:51:49.275413+0000","last_undegraded":"2024-02-23T00:51:49.275413+0000","last_fullsized":"2024-02-23T00:51:49.275413+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:03.875273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:03.875273+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":549,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.271077+0000","last_change":"2024-02-23T00:50:25.575319+0000","last_active":"2024-02-23T00:51:50.271077+0000","last_peered":"2024-02-23T00:51:50.271077+0000","last_clean":"2024-02-23T00:51:50.271077+0000","last_became_active":"2024-02-23T00:39:18.798829+0000","last_became_peered":"2024-02-23T00:39:18.798829+0000","last_unstale":"2024-02-23T00:51:50.271077+0000","last_undegraded":"2024-02-23T00:51:50.271077+0000","last_fullsized":"2024-02-23T00:51:50.271077+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:25.575238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:10.493340+0000","last_clean_scrub_stamp":"2024-02-23T00:50:25.575238+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"457'52","reported_seq":644,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.267460+0000","last_change":"2024-02-23T00:51:05.221646+0000","last_active":"2024-02-23T00:51:50.267460+0000","last_peered":"2024-02-23T00:51:50.267460+0000","last_clean":"2024-02-23T00:51:50.267460+0000","last_became_active":"2024-02-23T00:39:16.859002+0000","last_became_peered":"2024-02-23T00:39:16.859002+0000","last_unstale":"2024-02-23T00:51:50.267460+0000","last_undegraded":"2024-02-23T00:51:50.267460+0000","last_fullsized":"2024-02-23T00:51:50.267460+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'52","last_scrub_stamp":"2024-02-23T00:51:05.221572+0000","last_deep_scrub":"251'40","last_deep_scrub_stamp":"2024-02-23T00:45:23.910546+0000","last_clean_scrub_stamp":"2024-02-23T00:51:05.221572+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":76,"num_read_kb":62,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":542,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.274059+0000","last_change":"2024-02-23T00:48:44.728592+0000","last_active":"2024-02-23T00:51:49.274059+0000","last_peered":"2024-02-23T00:51:49.274059+0000","last_clean":"2024-02-23T00:51:49.274059+0000","last_became_active":"2024-02-23T00:39:14.779193+0000","last_became_peered":"2024-02-23T00:39:14.779193+0000","last_unstale":"2024-02-23T00:51:49.274059+0000","last_undegraded":"2024-02-23T00:51:49.274059+0000","last_fullsized":"2024-02-23T00:51:49.274059+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:44.728520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:48:44.728520+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":572,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.273169+0000","last_change":"2024-02-23T00:50:17.599788+0000","last_active":"2024-02-23T00:51:50.273169+0000","last_peered":"2024-02-23T00:51:50.273169+0000","last_clean":"2024-02-23T00:51:50.273169+0000","last_became_active":"2024-02-23T00:47:33.257915+0000","last_became_peered":"2024-02-23T00:47:33.257915+0000","last_unstale":"2024-02-23T00:51:50.273169+0000","last_undegraded":"2024-02-23T00:51:50.273169+0000","last_fullsized":"2024-02-23T00:51:50.273169+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":381,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:17.599724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:50:17.599724+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":497,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.268388+0000","last_change":"2024-02-23T00:51:03.166243+0000","last_active":"2024-02-23T00:51:50.268388+0000","last_peered":"2024-02-23T00:51:50.268388+0000","last_clean":"2024-02-23T00:51:50.268388+0000","last_became_active":"2024-02-23T00:42:58.809283+0000","last_became_peered":"2024-02-23T00:42:58.809283+0000","last_unstale":"2024-02-23T00:51:50.268388+0000","last_undegraded":"2024-02-23T00:51:50.268388+0000","last_fullsized":"2024-02-23T00:51:50.268388+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:03.166140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:03.166140+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":551,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.267164+0000","last_change":"2024-02-23T00:51:34.200412+0000","last_active":"2024-02-23T00:51:49.267164+0000","last_peered":"2024-02-23T00:51:49.267164+0000","last_clean":"2024-02-23T00:51:49.267164+0000","last_became_active":"2024-02-23T00:39:18.796664+0000","last_became_peered":"2024-02-23T00:39:18.796664+0000","last_unstale":"2024-02-23T00:51:49.267164+0000","last_undegraded":"2024-02-23T00:51:49.267164+0000","last_fullsized":"2024-02-23T00:51:49.267164+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:34.200312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:51:34.200312+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"457'44","reported_seq":626,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.272819+0000","last_change":"2024-02-23T00:49:46.644982+0000","last_active":"2024-02-23T00:51:50.272819+0000","last_peered":"2024-02-23T00:51:50.272819+0000","last_clean":"2024-02-23T00:51:50.272819+0000","last_became_active":"2024-02-23T00:39:16.857278+0000","last_became_peered":"2024-02-23T00:39:16.857278+0000","last_unstale":"2024-02-23T00:51:50.272819+0000","last_undegraded":"2024-02-23T00:51:50.272819+0000","last_fullsized":"2024-02-23T00:51:50.272819+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'44","last_scrub_stamp":"2024-02-23T00:49:46.644911+0000","last_deep_scrub":"457'44","last_deep_scrub_stamp":"2024-02-23T00:49:46.644911+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.644911+0000","log_size":44,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":550,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.272507+0000","last_change":"2024-02-23T00:50:22.299591+0000","last_active":"2024-02-23T00:51:49.272507+0000","last_peered":"2024-02-23T00:51:49.272507+0000","last_clean":"2024-02-23T00:51:49.272507+0000","last_became_active":"2024-02-23T00:39:14.783662+0000","last_became_peered":"2024-02-23T00:39:14.783662+0000","last_unstale":"2024-02-23T00:51:49.272507+0000","last_undegraded":"2024-02-23T00:51:49.272507+0000","last_fullsized":"2024-02-23T00:51:49.272507+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:22.299534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:22.299534+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":563,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.272884+0000","last_change":"2024-02-23T00:48:13.821965+0000","last_active":"2024-02-23T00:51:49.272884+0000","last_peered":"2024-02-23T00:51:49.272884+0000","last_clean":"2024-02-23T00:51:49.272884+0000","last_became_active":"2024-02-23T00:47:37.184240+0000","last_became_peered":"2024-02-23T00:47:37.184240+0000","last_unstale":"2024-02-23T00:51:49.272884+0000","last_undegraded":"2024-02-23T00:51:49.272884+0000","last_fullsized":"2024-02-23T00:51:49.272884+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":385,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:13.821901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:48:13.821901+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":493,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.275494+0000","last_change":"2024-02-23T00:50:24.682130+0000","last_active":"2024-02-23T00:51:49.275494+0000","last_peered":"2024-02-23T00:51:49.275494+0000","last_clean":"2024-02-23T00:51:49.275494+0000","last_became_active":"2024-02-23T00:42:58.726652+0000","last_became_peered":"2024-02-23T00:42:58.726652+0000","last_unstale":"2024-02-23T00:51:49.275494+0000","last_undegraded":"2024-02-23T00:51:49.275494+0000","last_fullsized":"2024-02-23T00:51:49.275494+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:24.682039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:24.682039+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":553,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.275300+0000","last_change":"2024-02-23T00:51:43.644655+0000","last_active":"2024-02-23T00:51:49.275300+0000","last_peered":"2024-02-23T00:51:49.275300+0000","last_clean":"2024-02-23T00:51:49.275300+0000","last_became_active":"2024-02-23T00:39:18.800378+0000","last_became_peered":"2024-02-23T00:39:18.800378+0000","last_unstale":"2024-02-23T00:51:49.275300+0000","last_undegraded":"2024-02-23T00:51:49.275300+0000","last_fullsized":"2024-02-23T00:51:49.275300+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:43.644591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:59.694033+0000","last_clean_scrub_stamp":"2024-02-23T00:51:43.644591+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"457'30","reported_seq":616,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.273408+0000","last_change":"2024-02-23T00:49:42.730745+0000","last_active":"2024-02-23T00:51:50.273408+0000","last_peered":"2024-02-23T00:51:50.273408+0000","last_clean":"2024-02-23T00:51:50.273408+0000","last_became_active":"2024-02-23T00:47:28.215366+0000","last_became_peered":"2024-02-23T00:47:28.215366+0000","last_unstale":"2024-02-23T00:51:50.273408+0000","last_undegraded":"2024-02-23T00:51:50.273408+0000","last_fullsized":"2024-02-23T00:51:50.273408+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'30","last_scrub_stamp":"2024-02-23T00:49:42.730670+0000","last_deep_scrub":"95'18","last_deep_scrub_stamp":"2024-02-23T00:42:01.195545+0000","last_clean_scrub_stamp":"2024-02-23T00:49:42.730670+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":555,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.270171+0000","last_change":"2024-02-23T00:49:46.162522+0000","last_active":"2024-02-23T00:51:49.270171+0000","last_peered":"2024-02-23T00:51:49.270171+0000","last_clean":"2024-02-23T00:51:49.270171+0000","last_became_active":"2024-02-23T00:39:14.784578+0000","last_became_peered":"2024-02-23T00:39:14.784578+0000","last_unstale":"2024-02-23T00:51:49.270171+0000","last_undegraded":"2024-02-23T00:51:49.270171+0000","last_fullsized":"2024-02-23T00:51:49.270171+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:46.162444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.162444+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":553,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.271999+0000","last_change":"2024-02-23T00:49:31.768539+0000","last_active":"2024-02-23T00:51:50.271999+0000","last_peered":"2024-02-23T00:51:50.271999+0000","last_clean":"2024-02-23T00:51:50.271999+0000","last_became_active":"2024-02-23T00:39:14.775446+0000","last_became_peered":"2024-02-23T00:39:14.775446+0000","last_unstale":"2024-02-23T00:51:50.271999+0000","last_undegraded":"2024-02-23T00:51:50.271999+0000","last_fullsized":"2024-02-23T00:51:50.271999+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:31.768472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:31.768472+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"67'1","reported_seq":574,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.273775+0000","last_change":"2024-02-23T00:51:02.380789+0000","last_active":"2024-02-23T00:51:49.273775+0000","last_peered":"2024-02-23T00:51:49.273775+0000","last_clean":"2024-02-23T00:51:49.273775+0000","last_became_active":"2024-02-23T00:47:56.187940+0000","last_became_peered":"2024-02-23T00:47:56.187940+0000","last_unstale":"2024-02-23T00:51:49.273775+0000","last_undegraded":"2024-02-23T00:51:49.273775+0000","last_fullsized":"2024-02-23T00:51:49.273775+0000","mapping_epoch":65,"log_start":"67'1","ondisk_log_start":"67'1","created":65,"last_epoch_clean":404,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:51:02.380721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:51:02.380721+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":497,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.268529+0000","last_change":"2024-02-23T00:51:10.298816+0000","last_active":"2024-02-23T00:51:49.268529+0000","last_peered":"2024-02-23T00:51:49.268529+0000","last_clean":"2024-02-23T00:51:49.268529+0000","last_became_active":"2024-02-23T00:42:58.690858+0000","last_became_peered":"2024-02-23T00:42:58.690858+0000","last_unstale":"2024-02-23T00:51:49.268529+0000","last_undegraded":"2024-02-23T00:51:49.268529+0000","last_fullsized":"2024-02-23T00:51:49.268529+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:10.298740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:35.360054+0000","last_clean_scrub_stamp":"2024-02-23T00:51:10.298740+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":560,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.272124+0000","last_change":"2024-02-23T00:51:39.221685+0000","last_active":"2024-02-23T00:51:49.272124+0000","last_peered":"2024-02-23T00:51:49.272124+0000","last_clean":"2024-02-23T00:51:49.272124+0000","last_became_active":"2024-02-23T00:47:28.231152+0000","last_became_peered":"2024-02-23T00:47:28.231152+0000","last_unstale":"2024-02-23T00:51:49.272124+0000","last_undegraded":"2024-02-23T00:51:49.272124+0000","last_fullsized":"2024-02-23T00:51:49.272124+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:39.221609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:51:39.221609+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"457'26","reported_seq":593,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.267570+0000","last_change":"2024-02-23T00:51:01.219903+0000","last_active":"2024-02-23T00:51:49.267570+0000","last_peered":"2024-02-23T00:51:49.267570+0000","last_clean":"2024-02-23T00:51:49.267570+0000","last_became_active":"2024-02-23T00:39:16.858880+0000","last_became_peered":"2024-02-23T00:39:16.858880+0000","last_unstale":"2024-02-23T00:51:49.267570+0000","last_undegraded":"2024-02-23T00:51:49.267570+0000","last_fullsized":"2024-02-23T00:51:49.267570+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-23T00:51:01.219824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:01.219824+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":560,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.270186+0000","last_change":"2024-02-23T00:50:24.297660+0000","last_active":"2024-02-23T00:51:49.270186+0000","last_peered":"2024-02-23T00:51:49.270186+0000","last_clean":"2024-02-23T00:51:49.270186+0000","last_became_active":"2024-02-23T00:39:08.248249+0000","last_became_peered":"2024-02-23T00:39:08.248249+0000","last_unstale":"2024-02-23T00:51:49.270186+0000","last_undegraded":"2024-02-23T00:51:49.270186+0000","last_fullsized":"2024-02-23T00:51:49.270186+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:24.297572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:49.977248+0000","last_clean_scrub_stamp":"2024-02-23T00:50:24.297572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":561,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.275921+0000","last_change":"2024-02-23T00:50:21.746072+0000","last_active":"2024-02-23T00:51:49.275921+0000","last_peered":"2024-02-23T00:51:49.275921+0000","last_clean":"2024-02-23T00:51:49.275921+0000","last_became_active":"2024-02-23T00:47:28.240482+0000","last_became_peered":"2024-02-23T00:47:28.240482+0000","last_unstale":"2024-02-23T00:51:49.275921+0000","last_undegraded":"2024-02-23T00:51:49.275921+0000","last_fullsized":"2024-02-23T00:51:49.275921+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:21.745961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:21.745961+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":556,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.267579+0000","last_change":"2024-02-23T00:50:34.055990+0000","last_active":"2024-02-23T00:51:50.267579+0000","last_peered":"2024-02-23T00:51:50.267579+0000","last_clean":"2024-02-23T00:51:50.267579+0000","last_became_active":"2024-02-23T00:39:14.783489+0000","last_became_peered":"2024-02-23T00:39:14.783489+0000","last_unstale":"2024-02-23T00:51:50.267579+0000","last_undegraded":"2024-02-23T00:51:50.267579+0000","last_fullsized":"2024-02-23T00:51:50.267579+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:34.055923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:34.055923+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":582,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.275347+0000","last_change":"2024-02-23T00:49:35.756168+0000","last_active":"2024-02-23T00:51:49.275347+0000","last_peered":"2024-02-23T00:51:49.275347+0000","last_clean":"2024-02-23T00:51:49.275347+0000","last_became_active":"2024-02-23T00:47:45.182550+0000","last_became_peered":"2024-02-23T00:47:45.182550+0000","last_unstale":"2024-02-23T00:51:49.275347+0000","last_undegraded":"2024-02-23T00:51:49.275347+0000","last_fullsized":"2024-02-23T00:51:49.275347+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:35.756073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:49:35.756073+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":504,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.271426+0000","last_change":"2024-02-23T00:50:16.257963+0000","last_active":"2024-02-23T00:51:49.271426+0000","last_peered":"2024-02-23T00:51:49.271426+0000","last_clean":"2024-02-23T00:51:49.271426+0000","last_became_active":"2024-02-23T00:47:28.233475+0000","last_became_peered":"2024-02-23T00:47:28.233475+0000","last_unstale":"2024-02-23T00:51:49.271426+0000","last_undegraded":"2024-02-23T00:51:49.271426+0000","last_fullsized":"2024-02-23T00:51:49.271426+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:16.257896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:16.257896+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":225,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.275874+0000","last_change":"2024-02-23T00:51:12.709695+0000","last_active":"2024-02-23T00:51:49.275874+0000","last_peered":"2024-02-23T00:51:49.275874+0000","last_clean":"2024-02-23T00:51:49.275874+0000","last_became_active":"2024-02-23T00:47:28.209125+0000","last_became_peered":"2024-02-23T00:47:28.209125+0000","last_unstale":"2024-02-23T00:51:49.275874+0000","last_undegraded":"2024-02-23T00:51:49.275874+0000","last_fullsized":"2024-02-23T00:51:49.275874+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.709617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.264240+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.709617+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"457'23","reported_seq":593,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.275896+0000","last_change":"2024-02-23T00:51:42.560222+0000","last_active":"2024-02-23T00:51:49.275896+0000","last_peered":"2024-02-23T00:51:49.275896+0000","last_clean":"2024-02-23T00:51:49.275896+0000","last_became_active":"2024-02-23T00:39:16.787484+0000","last_became_peered":"2024-02-23T00:39:16.787484+0000","last_unstale":"2024-02-23T00:51:49.275896+0000","last_undegraded":"2024-02-23T00:51:49.275896+0000","last_fullsized":"2024-02-23T00:51:49.275896+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'23","last_scrub_stamp":"2024-02-23T00:51:42.560138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:42.560138+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":48,"num_read_kb":36,"num_write":26,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":228,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.267728+0000","last_change":"2024-02-23T00:51:44.659312+0000","last_active":"2024-02-23T00:51:50.267728+0000","last_peered":"2024-02-23T00:51:50.267728+0000","last_clean":"2024-02-23T00:51:50.267728+0000","last_became_active":"2024-02-23T00:47:28.209575+0000","last_became_peered":"2024-02-23T00:47:28.209575+0000","last_unstale":"2024-02-23T00:51:50.267728+0000","last_undegraded":"2024-02-23T00:51:50.267728+0000","last_fullsized":"2024-02-23T00:51:50.267728+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:44.659246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:44.659246+0000","last_clean_scrub_stamp":"2024-02-23T00:51:44.659246+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"68'5","reported_seq":1306,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.663742+0000","last_change":"2024-02-23T00:50:57.456201+0000","last_active":"2024-02-23T00:51:50.663742+0000","last_peered":"2024-02-23T00:51:50.663742+0000","last_clean":"2024-02-23T00:51:50.663742+0000","last_became_active":"2024-02-23T00:47:28.227928+0000","last_became_peered":"2024-02-23T00:47:28.227928+0000","last_unstale":"2024-02-23T00:51:50.663742+0000","last_undegraded":"2024-02-23T00:51:50.663742+0000","last_fullsized":"2024-02-23T00:51:50.663742+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'5","last_scrub_stamp":"2024-02-23T00:50:57.456059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:57.456059+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":747,"num_read_kb":742,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":580,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.272425+0000","last_change":"2024-02-23T00:49:30.396935+0000","last_active":"2024-02-23T00:51:49.272425+0000","last_peered":"2024-02-23T00:51:49.272425+0000","last_clean":"2024-02-23T00:51:49.272425+0000","last_became_active":"2024-02-23T00:47:52.552244+0000","last_became_peered":"2024-02-23T00:47:52.552244+0000","last_unstale":"2024-02-23T00:51:49.272425+0000","last_undegraded":"2024-02-23T00:51:49.272425+0000","last_fullsized":"2024-02-23T00:51:49.272425+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:30.396865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:49:30.396865+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":493,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.272672+0000","last_change":"2024-02-23T00:49:40.533441+0000","last_active":"2024-02-23T00:51:49.272672+0000","last_peered":"2024-02-23T00:51:49.272672+0000","last_clean":"2024-02-23T00:51:49.272672+0000","last_became_active":"2024-02-23T00:47:28.212516+0000","last_became_peered":"2024-02-23T00:47:28.212516+0000","last_unstale":"2024-02-23T00:51:49.272672+0000","last_undegraded":"2024-02-23T00:51:49.272672+0000","last_fullsized":"2024-02-23T00:51:49.272672+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:40.533344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:49:40.533344+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":549,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.275314+0000","last_change":"2024-02-23T00:49:37.787041+0000","last_active":"2024-02-23T00:51:50.275314+0000","last_peered":"2024-02-23T00:51:50.275314+0000","last_clean":"2024-02-23T00:51:50.275314+0000","last_became_active":"2024-02-23T00:39:18.802134+0000","last_became_peered":"2024-02-23T00:39:18.802134+0000","last_unstale":"2024-02-23T00:51:50.275314+0000","last_undegraded":"2024-02-23T00:51:50.275314+0000","last_fullsized":"2024-02-23T00:51:50.275314+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:37.786978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:37.786978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"457'48","reported_seq":736,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.276272+0000","last_change":"2024-02-23T00:50:30.763802+0000","last_active":"2024-02-23T00:51:49.276272+0000","last_peered":"2024-02-23T00:51:49.276272+0000","last_clean":"2024-02-23T00:51:49.276272+0000","last_became_active":"2024-02-23T00:47:28.240257+0000","last_became_peered":"2024-02-23T00:47:28.240257+0000","last_unstale":"2024-02-23T00:51:49.276272+0000","last_undegraded":"2024-02-23T00:51:49.276272+0000","last_fullsized":"2024-02-23T00:51:49.276272+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'48","last_scrub_stamp":"2024-02-23T00:50:30.763686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:30.763686+0000","log_size":48,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":78,"num_read_kb":62,"num_write":44,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":552,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.274950+0000","last_change":"2024-02-23T00:49:41.639660+0000","last_active":"2024-02-23T00:51:49.274950+0000","last_peered":"2024-02-23T00:51:49.274950+0000","last_clean":"2024-02-23T00:51:49.274950+0000","last_became_active":"2024-02-23T00:39:14.784015+0000","last_became_peered":"2024-02-23T00:39:14.784015+0000","last_unstale":"2024-02-23T00:51:49.274950+0000","last_undegraded":"2024-02-23T00:51:49.274950+0000","last_fullsized":"2024-02-23T00:51:49.274950+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:41.639582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:41.639582+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":586,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.268317+0000","last_change":"2024-02-23T00:51:46.680015+0000","last_active":"2024-02-23T00:51:50.268317+0000","last_peered":"2024-02-23T00:51:50.268317+0000","last_clean":"2024-02-23T00:51:50.268317+0000","last_became_active":"2024-02-23T00:47:49.525421+0000","last_became_peered":"2024-02-23T00:47:49.525421+0000","last_unstale":"2024-02-23T00:51:50.268317+0000","last_undegraded":"2024-02-23T00:51:50.268317+0000","last_fullsized":"2024-02-23T00:51:50.268317+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":397,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:46.679950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:51:46.679950+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":222,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.273277+0000","last_change":"2024-02-23T00:50:13.549604+0000","last_active":"2024-02-23T00:51:50.273277+0000","last_peered":"2024-02-23T00:51:50.273277+0000","last_clean":"2024-02-23T00:51:50.273277+0000","last_became_active":"2024-02-23T00:47:28.232194+0000","last_became_peered":"2024-02-23T00:47:28.232194+0000","last_unstale":"2024-02-23T00:51:50.273277+0000","last_undegraded":"2024-02-23T00:51:50.273277+0000","last_fullsized":"2024-02-23T00:51:50.273277+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:13.549529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:13.549529+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3","version":"91'9","reported_seq":709,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.273051+0000","last_change":"2024-02-23T00:49:41.776249+0000","last_active":"2024-02-23T00:51:50.273051+0000","last_peered":"2024-02-23T00:51:50.273051+0000","last_clean":"2024-02-23T00:51:50.273051+0000","last_became_active":"2024-02-23T00:39:18.801297+0000","last_became_peered":"2024-02-23T00:39:18.801297+0000","last_unstale":"2024-02-23T00:51:50.273051+0000","last_undegraded":"2024-02-23T00:51:50.273051+0000","last_fullsized":"2024-02-23T00:51:50.273051+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'9","last_scrub_stamp":"2024-02-23T00:49:41.776117+0000","last_deep_scrub":"91'9","last_deep_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_clean_scrub_stamp":"2024-02-23T00:49:41.776117+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"457'26","reported_seq":585,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.273781+0000","last_change":"2024-02-23T00:48:55.589320+0000","last_active":"2024-02-23T00:51:49.273781+0000","last_peered":"2024-02-23T00:51:49.273781+0000","last_clean":"2024-02-23T00:51:49.273781+0000","last_became_active":"2024-02-23T00:39:16.787468+0000","last_became_peered":"2024-02-23T00:39:16.787468+0000","last_unstale":"2024-02-23T00:51:49.273781+0000","last_undegraded":"2024-02-23T00:51:49.273781+0000","last_fullsized":"2024-02-23T00:51:49.273781+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:48:55.589209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:48:55.589209+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":36,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":547,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.273413+0000","last_change":"2024-02-23T00:51:36.415402+0000","last_active":"2024-02-23T00:51:49.273413+0000","last_peered":"2024-02-23T00:51:49.273413+0000","last_clean":"2024-02-23T00:51:49.273413+0000","last_became_active":"2024-02-23T00:39:14.779952+0000","last_became_peered":"2024-02-23T00:39:14.779952+0000","last_unstale":"2024-02-23T00:51:49.273413+0000","last_undegraded":"2024-02-23T00:51:49.273413+0000","last_fullsized":"2024-02-23T00:51:49.273413+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:36.415340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:36.415340+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"107'2","reported_seq":582,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.272017+0000","last_change":"2024-02-23T00:50:17.587665+0000","last_active":"2024-02-23T00:51:50.272017+0000","last_peered":"2024-02-23T00:51:50.272017+0000","last_clean":"2024-02-23T00:51:50.272017+0000","last_became_active":"2024-02-23T00:47:41.210074+0000","last_became_peered":"2024-02-23T00:47:41.210074+0000","last_unstale":"2024-02-23T00:51:50.272017+0000","last_undegraded":"2024-02-23T00:51:50.272017+0000","last_fullsized":"2024-02-23T00:51:50.272017+0000","mapping_epoch":375,"log_start":"107'2","ondisk_log_start":"107'2","created":65,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'2","last_scrub_stamp":"2024-02-23T00:50:17.587591+0000","last_deep_scrub":"107'2","last_deep_scrub_stamp":"2024-02-23T00:42:48.362739+0000","last_clean_scrub_stamp":"2024-02-23T00:50:17.587591+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":501,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.269926+0000","last_change":"2024-02-23T00:50:18.308436+0000","last_active":"2024-02-23T00:51:49.269926+0000","last_peered":"2024-02-23T00:51:49.269926+0000","last_clean":"2024-02-23T00:51:49.269926+0000","last_became_active":"2024-02-23T00:47:28.242255+0000","last_became_peered":"2024-02-23T00:47:28.242255+0000","last_unstale":"2024-02-23T00:51:49.269926+0000","last_undegraded":"2024-02-23T00:51:49.269926+0000","last_fullsized":"2024-02-23T00:51:49.269926+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:18.308350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:18.308350+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":226,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.272480+0000","last_change":"2024-02-23T00:51:35.631662+0000","last_active":"2024-02-23T00:51:50.272480+0000","last_peered":"2024-02-23T00:51:50.272480+0000","last_clean":"2024-02-23T00:51:50.272480+0000","last_became_active":"2024-02-23T00:47:28.234034+0000","last_became_peered":"2024-02-23T00:47:28.234034+0000","last_unstale":"2024-02-23T00:51:50.272480+0000","last_undegraded":"2024-02-23T00:51:50.272480+0000","last_fullsized":"2024-02-23T00:51:50.272480+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.631598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.631598+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"457'31","reported_seq":612,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.270996+0000","last_change":"2024-02-23T00:51:43.724733+0000","last_active":"2024-02-23T00:51:50.270996+0000","last_peered":"2024-02-23T00:51:50.270996+0000","last_clean":"2024-02-23T00:51:50.270996+0000","last_became_active":"2024-02-23T00:39:16.783926+0000","last_became_peered":"2024-02-23T00:39:16.783926+0000","last_unstale":"2024-02-23T00:51:50.270996+0000","last_undegraded":"2024-02-23T00:51:50.270996+0000","last_fullsized":"2024-02-23T00:51:50.270996+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:51:43.724670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:43.724670+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":58,"num_read_kb":47,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"95'4","reported_seq":566,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.272591+0000","last_change":"2024-02-23T00:49:33.465769+0000","last_active":"2024-02-23T00:51:50.272591+0000","last_peered":"2024-02-23T00:51:50.272591+0000","last_clean":"2024-02-23T00:51:50.272591+0000","last_became_active":"2024-02-23T00:39:14.779205+0000","last_became_peered":"2024-02-23T00:39:14.779205+0000","last_unstale":"2024-02-23T00:51:50.272591+0000","last_undegraded":"2024-02-23T00:51:50.272591+0000","last_fullsized":"2024-02-23T00:51:50.272591+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'4","last_scrub_stamp":"2024-02-23T00:49:33.465697+0000","last_deep_scrub":"95'4","last_deep_scrub_stamp":"2024-02-23T00:41:43.874397+0000","last_clean_scrub_stamp":"2024-02-23T00:49:33.465697+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":568,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.268984+0000","last_change":"2024-02-23T00:50:15.922829+0000","last_active":"2024-02-23T00:51:50.268984+0000","last_peered":"2024-02-23T00:51:50.268984+0000","last_clean":"2024-02-23T00:51:50.268984+0000","last_became_active":"2024-02-23T00:43:27.611660+0000","last_became_peered":"2024-02-23T00:43:27.611660+0000","last_unstale":"2024-02-23T00:51:50.268984+0000","last_undegraded":"2024-02-23T00:51:50.268984+0000","last_fullsized":"2024-02-23T00:51:50.268984+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:15.922745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:50:15.922745+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":507,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.273045+0000","last_change":"2024-02-23T00:51:46.336777+0000","last_active":"2024-02-23T00:51:49.273045+0000","last_peered":"2024-02-23T00:51:49.273045+0000","last_clean":"2024-02-23T00:51:49.273045+0000","last_became_active":"2024-02-23T00:47:28.231670+0000","last_became_peered":"2024-02-23T00:47:28.231670+0000","last_unstale":"2024-02-23T00:51:49.273045+0000","last_undegraded":"2024-02-23T00:51:49.273045+0000","last_fullsized":"2024-02-23T00:51:49.273045+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:46.336718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:46.336718+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"457'27","reported_seq":606,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.273500+0000","last_change":"2024-02-23T00:51:42.671042+0000","last_active":"2024-02-23T00:51:50.273500+0000","last_peered":"2024-02-23T00:51:50.273500+0000","last_clean":"2024-02-23T00:51:50.273500+0000","last_became_active":"2024-02-23T00:39:16.858760+0000","last_became_peered":"2024-02-23T00:39:16.858760+0000","last_unstale":"2024-02-23T00:51:50.273500+0000","last_undegraded":"2024-02-23T00:51:50.273500+0000","last_fullsized":"2024-02-23T00:51:50.273500+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-23T00:51:42.670972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:42.670972+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":217,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.271847+0000","last_change":"2024-02-23T00:49:02.601790+0000","last_active":"2024-02-23T00:51:49.271847+0000","last_peered":"2024-02-23T00:51:49.271847+0000","last_clean":"2024-02-23T00:51:49.271847+0000","last_became_active":"2024-02-23T00:47:28.219764+0000","last_became_peered":"2024-02-23T00:47:28.219764+0000","last_unstale":"2024-02-23T00:51:49.271847+0000","last_undegraded":"2024-02-23T00:51:49.271847+0000","last_fullsized":"2024-02-23T00:51:49.271847+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:02.601726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:02.601726+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":557,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.269760+0000","last_change":"2024-02-23T00:51:45.695038+0000","last_active":"2024-02-23T00:51:50.269760+0000","last_peered":"2024-02-23T00:51:50.269760+0000","last_clean":"2024-02-23T00:51:50.269760+0000","last_became_active":"2024-02-23T00:39:18.798624+0000","last_became_peered":"2024-02-23T00:39:18.798624+0000","last_unstale":"2024-02-23T00:51:50.269760+0000","last_undegraded":"2024-02-23T00:51:50.269760+0000","last_fullsized":"2024-02-23T00:51:50.269760+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:45.694974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:45.913526+0000","last_clean_scrub_stamp":"2024-02-23T00:51:45.694974+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":494,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.272228+0000","last_change":"2024-02-23T00:50:13.642424+0000","last_active":"2024-02-23T00:51:50.272228+0000","last_peered":"2024-02-23T00:51:50.272228+0000","last_clean":"2024-02-23T00:51:50.272228+0000","last_became_active":"2024-02-23T00:42:58.825015+0000","last_became_peered":"2024-02-23T00:42:58.825015+0000","last_unstale":"2024-02-23T00:51:50.272228+0000","last_undegraded":"2024-02-23T00:51:50.272228+0000","last_fullsized":"2024-02-23T00:51:50.272228+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:13.642359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:13.642359+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"457'22","reported_seq":591,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.269671+0000","last_change":"2024-02-23T00:51:01.201807+0000","last_active":"2024-02-23T00:51:50.269671+0000","last_peered":"2024-02-23T00:51:50.269671+0000","last_clean":"2024-02-23T00:51:50.269671+0000","last_became_active":"2024-02-23T00:39:16.858449+0000","last_became_peered":"2024-02-23T00:39:16.858449+0000","last_unstale":"2024-02-23T00:51:50.269671+0000","last_undegraded":"2024-02-23T00:51:50.269671+0000","last_fullsized":"2024-02-23T00:51:50.269671+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'22","last_scrub_stamp":"2024-02-23T00:51:01.201737+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-02-23T00:48:07.941101+0000","last_clean_scrub_stamp":"2024-02-23T00:51:01.201737+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":30,"num_read_kb":26,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":550,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.267829+0000","last_change":"2024-02-23T00:49:28.347354+0000","last_active":"2024-02-23T00:51:49.267829+0000","last_peered":"2024-02-23T00:51:49.267829+0000","last_clean":"2024-02-23T00:51:49.267829+0000","last_became_active":"2024-02-23T00:39:14.783164+0000","last_became_peered":"2024-02-23T00:39:14.783164+0000","last_unstale":"2024-02-23T00:51:49.267829+0000","last_undegraded":"2024-02-23T00:51:49.267829+0000","last_fullsized":"2024-02-23T00:51:49.267829+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:28.347281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:28.347281+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":553,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.275951+0000","last_change":"2024-02-23T00:51:47.691352+0000","last_active":"2024-02-23T00:51:49.275951+0000","last_peered":"2024-02-23T00:51:49.275951+0000","last_clean":"2024-02-23T00:51:49.275951+0000","last_became_active":"2024-02-23T00:39:18.800297+0000","last_became_peered":"2024-02-23T00:39:18.800297+0000","last_unstale":"2024-02-23T00:51:49.275951+0000","last_undegraded":"2024-02-23T00:51:49.275951+0000","last_fullsized":"2024-02-23T00:51:49.275951+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:47.691288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:51:47.691288+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1","version":"110'7","reported_seq":510,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.267938+0000","last_change":"2024-02-23T00:51:13.352686+0000","last_active":"2024-02-23T00:51:50.267938+0000","last_peered":"2024-02-23T00:51:50.267938+0000","last_clean":"2024-02-23T00:51:50.267938+0000","last_became_active":"2024-02-23T00:42:58.777251+0000","last_became_peered":"2024-02-23T00:42:58.777251+0000","last_unstale":"2024-02-23T00:51:50.267938+0000","last_undegraded":"2024-02-23T00:51:50.267938+0000","last_fullsized":"2024-02-23T00:51:50.267938+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'7","last_scrub_stamp":"2024-02-23T00:51:13.352611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:13.352611+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"457'32","reported_seq":252,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.269340+0000","last_change":"2024-02-23T00:50:25.079859+0000","last_active":"2024-02-23T00:51:50.269340+0000","last_peered":"2024-02-23T00:51:50.269340+0000","last_clean":"2024-02-23T00:51:50.269340+0000","last_became_active":"2024-02-23T00:47:28.237898+0000","last_became_peered":"2024-02-23T00:47:28.237898+0000","last_unstale":"2024-02-23T00:51:50.269340+0000","last_undegraded":"2024-02-23T00:51:50.269340+0000","last_fullsized":"2024-02-23T00:51:50.269340+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-23T00:50:25.079789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:25.079789+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":52,"num_read_kb":45,"num_write":32,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":557,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.274871+0000","last_change":"2024-02-23T00:51:14.658413+0000","last_active":"2024-02-23T00:51:49.274871+0000","last_peered":"2024-02-23T00:51:49.274871+0000","last_clean":"2024-02-23T00:51:49.274871+0000","last_became_active":"2024-02-23T00:39:14.785590+0000","last_became_peered":"2024-02-23T00:39:14.785590+0000","last_unstale":"2024-02-23T00:51:49.274871+0000","last_undegraded":"2024-02-23T00:51:49.274871+0000","last_fullsized":"2024-02-23T00:51:49.274871+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:14.658297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_clean_scrub_stamp":"2024-02-23T00:51:14.658297+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"336'9","reported_seq":708,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.275712+0000","last_change":"2024-02-23T00:49:39.636503+0000","last_active":"2024-02-23T00:51:49.275712+0000","last_peered":"2024-02-23T00:51:49.275712+0000","last_clean":"2024-02-23T00:51:49.275712+0000","last_became_active":"2024-02-23T00:39:18.803566+0000","last_became_peered":"2024-02-23T00:39:18.803566+0000","last_unstale":"2024-02-23T00:51:49.275712+0000","last_undegraded":"2024-02-23T00:51:49.275712+0000","last_fullsized":"2024-02-23T00:51:49.275712+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"336'9","last_scrub_stamp":"2024-02-23T00:49:39.636409+0000","last_deep_scrub":"336'9","last_deep_scrub_stamp":"2024-02-23T00:46:49.160991+0000","last_clean_scrub_stamp":"2024-02-23T00:49:39.636409+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":493,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.272561+0000","last_change":"2024-02-23T00:51:33.285134+0000","last_active":"2024-02-23T00:51:49.272561+0000","last_peered":"2024-02-23T00:51:49.272561+0000","last_clean":"2024-02-23T00:51:49.272561+0000","last_became_active":"2024-02-23T00:42:58.757424+0000","last_became_peered":"2024-02-23T00:42:58.757424+0000","last_unstale":"2024-02-23T00:51:49.272561+0000","last_undegraded":"2024-02-23T00:51:49.272561+0000","last_fullsized":"2024-02-23T00:51:49.272561+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:33.285048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:33.285048+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"457'21","reported_seq":590,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.275613+0000","last_change":"2024-02-23T00:50:17.861556+0000","last_active":"2024-02-23T00:51:49.275613+0000","last_peered":"2024-02-23T00:51:49.275613+0000","last_clean":"2024-02-23T00:51:49.275613+0000","last_became_active":"2024-02-23T00:39:16.858953+0000","last_became_peered":"2024-02-23T00:39:16.858953+0000","last_unstale":"2024-02-23T00:51:49.275613+0000","last_undegraded":"2024-02-23T00:51:49.275613+0000","last_fullsized":"2024-02-23T00:51:49.275613+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-23T00:50:17.861490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:17.861490+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":558,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.272954+0000","last_change":"2024-02-23T00:51:33.662415+0000","last_active":"2024-02-23T00:51:50.272954+0000","last_peered":"2024-02-23T00:51:50.272954+0000","last_clean":"2024-02-23T00:51:50.272954+0000","last_became_active":"2024-02-23T00:39:14.773772+0000","last_became_peered":"2024-02-23T00:39:14.773772+0000","last_unstale":"2024-02-23T00:51:50.272954+0000","last_undegraded":"2024-02-23T00:51:50.272954+0000","last_fullsized":"2024-02-23T00:51:50.272954+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:33.662354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:33.662354+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"376'9","reported_seq":719,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.270173+0000","last_change":"2024-02-23T00:49:45.868359+0000","last_active":"2024-02-23T00:51:50.270173+0000","last_peered":"2024-02-23T00:51:50.270173+0000","last_clean":"2024-02-23T00:51:50.270173+0000","last_became_active":"2024-02-23T00:47:28.229316+0000","last_became_peered":"2024-02-23T00:47:28.229316+0000","last_unstale":"2024-02-23T00:51:50.270173+0000","last_undegraded":"2024-02-23T00:51:50.270173+0000","last_fullsized":"2024-02-23T00:51:50.270173+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"376'9","last_scrub_stamp":"2024-02-23T00:49:45.868289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:45.868289+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":484,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.274626+0000","last_change":"2024-02-23T00:50:15.714265+0000","last_active":"2024-02-23T00:51:49.274626+0000","last_peered":"2024-02-23T00:51:49.274626+0000","last_clean":"2024-02-23T00:51:49.274626+0000","last_became_active":"2024-02-23T00:42:57.652488+0000","last_became_peered":"2024-02-23T00:42:57.652488+0000","last_unstale":"2024-02-23T00:51:49.274626+0000","last_undegraded":"2024-02-23T00:51:49.274626+0000","last_fullsized":"2024-02-23T00:51:49.274626+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:15.714200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:51.254903+0000","last_clean_scrub_stamp":"2024-02-23T00:50:15.714200+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"457'15","reported_seq":584,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.273482+0000","last_change":"2024-02-23T00:49:39.573557+0000","last_active":"2024-02-23T00:51:49.273482+0000","last_peered":"2024-02-23T00:51:49.273482+0000","last_clean":"2024-02-23T00:51:49.273482+0000","last_became_active":"2024-02-23T00:47:28.225585+0000","last_became_peered":"2024-02-23T00:47:28.225585+0000","last_unstale":"2024-02-23T00:51:49.273482+0000","last_undegraded":"2024-02-23T00:51:49.273482+0000","last_fullsized":"2024-02-23T00:51:49.273482+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'15","last_scrub_stamp":"2024-02-23T00:49:39.573472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:39.573472+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":560,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.269846+0000","last_change":"2024-02-23T00:51:44.270528+0000","last_active":"2024-02-23T00:51:49.269846+0000","last_peered":"2024-02-23T00:51:49.269846+0000","last_clean":"2024-02-23T00:51:49.269846+0000","last_became_active":"2024-02-23T00:39:14.778455+0000","last_became_peered":"2024-02-23T00:39:14.778455+0000","last_unstale":"2024-02-23T00:51:49.269846+0000","last_undegraded":"2024-02-23T00:51:49.269846+0000","last_fullsized":"2024-02-23T00:51:49.269846+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:44.270465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:44.270465+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":561,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.268906+0000","last_change":"2024-02-23T00:50:30.069404+0000","last_active":"2024-02-23T00:51:50.268906+0000","last_peered":"2024-02-23T00:51:50.268906+0000","last_clean":"2024-02-23T00:51:50.268906+0000","last_became_active":"2024-02-23T00:47:44.225379+0000","last_became_peered":"2024-02-23T00:47:44.225379+0000","last_unstale":"2024-02-23T00:51:50.268906+0000","last_undegraded":"2024-02-23T00:51:50.268906+0000","last_fullsized":"2024-02-23T00:51:50.268906+0000","mapping_epoch":391,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":392,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:30.069325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_clean_scrub_stamp":"2024-02-23T00:50:30.069325+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":499,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.273138+0000","last_change":"2024-02-23T00:51:44.638197+0000","last_active":"2024-02-23T00:51:50.273138+0000","last_peered":"2024-02-23T00:51:50.273138+0000","last_clean":"2024-02-23T00:51:50.273138+0000","last_became_active":"2024-02-23T00:42:58.835862+0000","last_became_peered":"2024-02-23T00:42:58.835862+0000","last_unstale":"2024-02-23T00:51:50.273138+0000","last_undegraded":"2024-02-23T00:51:50.273138+0000","last_fullsized":"2024-02-23T00:51:50.273138+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:44.638129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:44.638129+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"457'19","reported_seq":580,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.271707+0000","last_change":"2024-02-23T00:50:29.599563+0000","last_active":"2024-02-23T00:51:50.271707+0000","last_peered":"2024-02-23T00:51:50.271707+0000","last_clean":"2024-02-23T00:51:50.271707+0000","last_became_active":"2024-02-23T00:39:16.783896+0000","last_became_peered":"2024-02-23T00:39:16.783896+0000","last_unstale":"2024-02-23T00:51:50.271707+0000","last_undegraded":"2024-02-23T00:51:50.271707+0000","last_fullsized":"2024-02-23T00:51:50.271707+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'19","last_scrub_stamp":"2024-02-23T00:50:29.599483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:29.599483+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"202'2","reported_seq":555,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.273367+0000","last_change":"2024-02-23T00:51:40.511331+0000","last_active":"2024-02-23T00:51:49.273367+0000","last_peered":"2024-02-23T00:51:49.273367+0000","last_clean":"2024-02-23T00:51:49.273367+0000","last_became_active":"2024-02-23T00:39:14.775804+0000","last_became_peered":"2024-02-23T00:39:14.775804+0000","last_unstale":"2024-02-23T00:51:49.273367+0000","last_undegraded":"2024-02-23T00:51:49.273367+0000","last_fullsized":"2024-02-23T00:51:49.273367+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"202'2","last_scrub_stamp":"2024-02-23T00:51:40.511258+0000","last_deep_scrub":"202'2","last_deep_scrub_stamp":"2024-02-23T00:44:34.720640+0000","last_clean_scrub_stamp":"2024-02-23T00:51:40.511258+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":538,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.273571+0000","last_change":"2024-02-23T00:50:18.698805+0000","last_active":"2024-02-23T00:51:49.273571+0000","last_peered":"2024-02-23T00:51:49.273571+0000","last_clean":"2024-02-23T00:51:49.273571+0000","last_became_active":"2024-02-23T00:39:18.800382+0000","last_became_peered":"2024-02-23T00:39:18.800382+0000","last_unstale":"2024-02-23T00:51:49.273571+0000","last_undegraded":"2024-02-23T00:51:49.273571+0000","last_fullsized":"2024-02-23T00:51:49.273571+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:18.698687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:50:18.698687+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":493,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.273269+0000","last_change":"2024-02-23T00:49:46.462085+0000","last_active":"2024-02-23T00:51:49.273269+0000","last_peered":"2024-02-23T00:51:49.273269+0000","last_clean":"2024-02-23T00:51:49.273269+0000","last_became_active":"2024-02-23T00:47:28.224961+0000","last_became_peered":"2024-02-23T00:47:28.224961+0000","last_unstale":"2024-02-23T00:51:49.273269+0000","last_undegraded":"2024-02-23T00:51:49.273269+0000","last_fullsized":"2024-02-23T00:51:49.273269+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:46.462015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:46.462015+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.462015+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"457'8","reported_seq":566,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.267355+0000","last_change":"2024-02-23T00:50:35.045697+0000","last_active":"2024-02-23T00:51:50.267355+0000","last_peered":"2024-02-23T00:51:50.267355+0000","last_clean":"2024-02-23T00:51:50.267355+0000","last_became_active":"2024-02-23T00:39:16.785830+0000","last_became_peered":"2024-02-23T00:39:16.785830+0000","last_unstale":"2024-02-23T00:51:50.267355+0000","last_undegraded":"2024-02-23T00:51:50.267355+0000","last_fullsized":"2024-02-23T00:51:50.267355+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'8","last_scrub_stamp":"2024-02-23T00:50:35.045612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:35.045612+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":569,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.275793+0000","last_change":"2024-02-23T00:51:34.529078+0000","last_active":"2024-02-23T00:51:49.275793+0000","last_peered":"2024-02-23T00:51:49.275793+0000","last_clean":"2024-02-23T00:51:49.275793+0000","last_became_active":"2024-02-23T00:47:28.224477+0000","last_became_peered":"2024-02-23T00:47:28.224477+0000","last_unstale":"2024-02-23T00:51:49.275793+0000","last_undegraded":"2024-02-23T00:51:49.275793+0000","last_fullsized":"2024-02-23T00:51:49.275793+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:34.529012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:34.529012+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":560,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.267731+0000","last_change":"2024-02-23T00:49:33.461971+0000","last_active":"2024-02-23T00:51:49.267731+0000","last_peered":"2024-02-23T00:51:49.267731+0000","last_clean":"2024-02-23T00:51:49.267731+0000","last_became_active":"2024-02-23T00:47:44.230295+0000","last_became_peered":"2024-02-23T00:47:44.230295+0000","last_unstale":"2024-02-23T00:51:49.267731+0000","last_undegraded":"2024-02-23T00:51:49.267731+0000","last_fullsized":"2024-02-23T00:51:49.267731+0000","mapping_epoch":391,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":392,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:33.461907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:33.461907+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":493,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.272615+0000","last_change":"2024-02-23T00:51:38.169736+0000","last_active":"2024-02-23T00:51:49.272615+0000","last_peered":"2024-02-23T00:51:49.272615+0000","last_clean":"2024-02-23T00:51:49.272615+0000","last_became_active":"2024-02-23T00:42:58.741365+0000","last_became_peered":"2024-02-23T00:42:58.741365+0000","last_unstale":"2024-02-23T00:51:49.272615+0000","last_undegraded":"2024-02-23T00:51:49.272615+0000","last_fullsized":"2024-02-23T00:51:49.272615+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:38.169669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:38.169669+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"457'25","reported_seq":600,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.268053+0000","last_change":"2024-02-23T00:49:47.788110+0000","last_active":"2024-02-23T00:51:50.268053+0000","last_peered":"2024-02-23T00:51:50.268053+0000","last_clean":"2024-02-23T00:51:50.268053+0000","last_became_active":"2024-02-23T00:39:16.857232+0000","last_became_peered":"2024-02-23T00:39:16.857232+0000","last_unstale":"2024-02-23T00:51:50.268053+0000","last_undegraded":"2024-02-23T00:51:50.268053+0000","last_fullsized":"2024-02-23T00:51:50.268053+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-23T00:49:47.788044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:47.788044+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":564,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.266874+0000","last_change":"2024-02-23T00:51:35.218650+0000","last_active":"2024-02-23T00:51:49.266874+0000","last_peered":"2024-02-23T00:51:49.266874+0000","last_clean":"2024-02-23T00:51:49.266874+0000","last_became_active":"2024-02-23T00:47:28.240162+0000","last_became_peered":"2024-02-23T00:47:28.240162+0000","last_unstale":"2024-02-23T00:51:49.266874+0000","last_undegraded":"2024-02-23T00:51:49.266874+0000","last_fullsized":"2024-02-23T00:51:49.266874+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.218583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:44.285083+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.218583+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":205,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.267244+0000","last_change":"2024-02-23T00:49:42.872820+0000","last_active":"2024-02-23T00:51:50.267244+0000","last_peered":"2024-02-23T00:51:50.267244+0000","last_clean":"2024-02-23T00:51:50.267244+0000","last_became_active":"2024-02-23T00:47:44.226097+0000","last_became_peered":"2024-02-23T00:47:44.226097+0000","last_unstale":"2024-02-23T00:51:50.267244+0000","last_undegraded":"2024-02-23T00:51:50.267244+0000","last_fullsized":"2024-02-23T00:51:50.267244+0000","mapping_epoch":391,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":392,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:42.872748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:42.872748+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":226,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.275070+0000","last_change":"2024-02-23T00:50:28.678155+0000","last_active":"2024-02-23T00:51:49.275070+0000","last_peered":"2024-02-23T00:51:49.275070+0000","last_clean":"2024-02-23T00:51:49.275070+0000","last_became_active":"2024-02-23T00:47:28.200688+0000","last_became_peered":"2024-02-23T00:47:28.200688+0000","last_unstale":"2024-02-23T00:51:49.275070+0000","last_undegraded":"2024-02-23T00:51:49.275070+0000","last_fullsized":"2024-02-23T00:51:49.275070+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:28.678041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:28.678041+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"457'26","reported_seq":596,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.273810+0000","last_change":"2024-02-23T00:50:23.540361+0000","last_active":"2024-02-23T00:51:50.273810+0000","last_peered":"2024-02-23T00:51:50.273810+0000","last_clean":"2024-02-23T00:51:50.273810+0000","last_became_active":"2024-02-23T00:39:16.858945+0000","last_became_peered":"2024-02-23T00:39:16.858945+0000","last_unstale":"2024-02-23T00:51:50.273810+0000","last_undegraded":"2024-02-23T00:51:50.273810+0000","last_fullsized":"2024-02-23T00:51:50.273810+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-23T00:50:23.540232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:23.540232+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":36,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":564,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.275392+0000","last_change":"2024-02-23T00:51:09.827935+0000","last_active":"2024-02-23T00:51:49.275392+0000","last_peered":"2024-02-23T00:51:49.275392+0000","last_clean":"2024-02-23T00:51:49.275392+0000","last_became_active":"2024-02-23T00:39:14.785363+0000","last_became_peered":"2024-02-23T00:39:14.785363+0000","last_unstale":"2024-02-23T00:51:49.275392+0000","last_undegraded":"2024-02-23T00:51:49.275392+0000","last_fullsized":"2024-02-23T00:51:49.275392+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"80'2","last_scrub_stamp":"2024-02-23T00:51:09.827857+0000","last_deep_scrub":"80'2","last_deep_scrub_stamp":"2024-02-23T00:40:15.942725+0000","last_clean_scrub_stamp":"2024-02-23T00:51:09.827857+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"93'9","reported_seq":714,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.267047+0000","last_change":"2024-02-23T00:51:39.253717+0000","last_active":"2024-02-23T00:51:49.267047+0000","last_peered":"2024-02-23T00:51:49.267047+0000","last_clean":"2024-02-23T00:51:49.267047+0000","last_became_active":"2024-02-23T00:39:18.796680+0000","last_became_peered":"2024-02-23T00:39:18.796680+0000","last_unstale":"2024-02-23T00:51:49.267047+0000","last_undegraded":"2024-02-23T00:51:49.267047+0000","last_fullsized":"2024-02-23T00:51:49.267047+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-02-23T00:51:39.253651+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-02-23T00:43:50.393713+0000","last_clean_scrub_stamp":"2024-02-23T00:51:39.253651+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":503,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.272554+0000","last_change":"2024-02-23T00:51:12.493007+0000","last_active":"2024-02-23T00:51:50.272554+0000","last_peered":"2024-02-23T00:51:50.272554+0000","last_clean":"2024-02-23T00:51:50.272554+0000","last_became_active":"2024-02-23T00:47:28.234641+0000","last_became_peered":"2024-02-23T00:47:28.234641+0000","last_unstale":"2024-02-23T00:51:50.272554+0000","last_undegraded":"2024-02-23T00:51:50.272554+0000","last_fullsized":"2024-02-23T00:51:50.272554+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.492942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.492942+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"457'31","reported_seq":610,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.267134+0000","last_change":"2024-02-23T00:51:15.408847+0000","last_active":"2024-02-23T00:51:49.267134+0000","last_peered":"2024-02-23T00:51:49.267134+0000","last_clean":"2024-02-23T00:51:49.267134+0000","last_became_active":"2024-02-23T00:39:16.782660+0000","last_became_peered":"2024-02-23T00:39:16.782660+0000","last_unstale":"2024-02-23T00:51:49.267134+0000","last_undegraded":"2024-02-23T00:51:49.267134+0000","last_fullsized":"2024-02-23T00:51:49.267134+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:51:15.408782+0000","last_deep_scrub":"93'23","last_deep_scrub_stamp":"2024-02-23T00:41:35.954516+0000","last_clean_scrub_stamp":"2024-02-23T00:51:15.408782+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":556,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.267528+0000","last_change":"2024-02-23T00:50:29.029794+0000","last_active":"2024-02-23T00:51:50.267528+0000","last_peered":"2024-02-23T00:51:50.267528+0000","last_clean":"2024-02-23T00:51:50.267528+0000","last_became_active":"2024-02-23T00:39:14.781560+0000","last_became_peered":"2024-02-23T00:39:14.781560+0000","last_unstale":"2024-02-23T00:51:50.267528+0000","last_undegraded":"2024-02-23T00:51:50.267528+0000","last_fullsized":"2024-02-23T00:51:50.267528+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:29.029725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:29.029725+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":563,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.273887+0000","last_change":"2024-02-23T00:51:14.488316+0000","last_active":"2024-02-23T00:51:50.273887+0000","last_peered":"2024-02-23T00:51:50.273887+0000","last_clean":"2024-02-23T00:51:50.273887+0000","last_became_active":"2024-02-23T00:47:28.232797+0000","last_became_peered":"2024-02-23T00:47:28.232797+0000","last_unstale":"2024-02-23T00:51:50.273887+0000","last_undegraded":"2024-02-23T00:51:50.273887+0000","last_fullsized":"2024-02-23T00:51:50.273887+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:14.488205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:53.189881+0000","last_clean_scrub_stamp":"2024-02-23T00:51:14.488205+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":497,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.267578+0000","last_change":"2024-02-23T00:51:00.232343+0000","last_active":"2024-02-23T00:51:50.267578+0000","last_peered":"2024-02-23T00:51:50.267578+0000","last_clean":"2024-02-23T00:51:50.267578+0000","last_became_active":"2024-02-23T00:42:58.774720+0000","last_became_peered":"2024-02-23T00:42:58.774720+0000","last_unstale":"2024-02-23T00:51:50.267578+0000","last_undegraded":"2024-02-23T00:51:50.267578+0000","last_fullsized":"2024-02-23T00:51:50.267578+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.232261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.232261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"457'27","reported_seq":288,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.273507+0000","last_change":"2024-02-23T00:49:26.518048+0000","last_active":"2024-02-23T00:51:49.273507+0000","last_peered":"2024-02-23T00:51:49.273507+0000","last_clean":"2024-02-23T00:51:49.273507+0000","last_became_active":"2024-02-23T00:47:28.211342+0000","last_became_peered":"2024-02-23T00:47:28.211342+0000","last_unstale":"2024-02-23T00:51:49.273507+0000","last_undegraded":"2024-02-23T00:51:49.273507+0000","last_fullsized":"2024-02-23T00:51:49.273507+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-23T00:49:26.517965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:26.517965+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":555,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.267043+0000","last_change":"2024-02-23T00:51:08.235807+0000","last_active":"2024-02-23T00:51:49.267043+0000","last_peered":"2024-02-23T00:51:49.267043+0000","last_clean":"2024-02-23T00:51:49.267043+0000","last_became_active":"2024-02-23T00:39:14.782494+0000","last_became_peered":"2024-02-23T00:39:14.782494+0000","last_unstale":"2024-02-23T00:51:49.267043+0000","last_undegraded":"2024-02-23T00:51:49.267043+0000","last_fullsized":"2024-02-23T00:51:49.267043+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:08.235735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:08.235735+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":538,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.273605+0000","last_change":"2024-02-23T00:48:16.788875+0000","last_active":"2024-02-23T00:51:49.273605+0000","last_peered":"2024-02-23T00:51:49.273605+0000","last_clean":"2024-02-23T00:51:49.273605+0000","last_became_active":"2024-02-23T00:39:18.801039+0000","last_became_peered":"2024-02-23T00:39:18.801039+0000","last_unstale":"2024-02-23T00:51:49.273605+0000","last_undegraded":"2024-02-23T00:51:49.273605+0000","last_fullsized":"2024-02-23T00:51:49.273605+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:16.788806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.641689+0000","last_clean_scrub_stamp":"2024-02-23T00:48:16.788806+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":497,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.267929+0000","last_change":"2024-02-23T00:51:10.341831+0000","last_active":"2024-02-23T00:51:50.267929+0000","last_peered":"2024-02-23T00:51:50.267929+0000","last_clean":"2024-02-23T00:51:50.267929+0000","last_became_active":"2024-02-23T00:42:58.770759+0000","last_became_peered":"2024-02-23T00:42:58.770759+0000","last_unstale":"2024-02-23T00:51:50.267929+0000","last_undegraded":"2024-02-23T00:51:50.267929+0000","last_fullsized":"2024-02-23T00:51:50.267929+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:10.341768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:10.341768+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"457'24","reported_seq":601,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.267956+0000","last_change":"2024-02-23T00:50:13.861288+0000","last_active":"2024-02-23T00:51:50.267956+0000","last_peered":"2024-02-23T00:51:50.267956+0000","last_clean":"2024-02-23T00:51:50.267956+0000","last_became_active":"2024-02-23T00:39:16.782803+0000","last_became_peered":"2024-02-23T00:39:16.782803+0000","last_unstale":"2024-02-23T00:51:50.267956+0000","last_undegraded":"2024-02-23T00:51:50.267956+0000","last_fullsized":"2024-02-23T00:51:50.267956+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'24","last_scrub_stamp":"2024-02-23T00:50:13.861220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:13.861220+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":46,"num_read_kb":36,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":556,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.268878+0000","last_change":"2024-02-23T00:50:28.060186+0000","last_active":"2024-02-23T00:51:50.268878+0000","last_peered":"2024-02-23T00:51:50.268878+0000","last_clean":"2024-02-23T00:51:50.268878+0000","last_became_active":"2024-02-23T00:39:14.782163+0000","last_became_peered":"2024-02-23T00:39:14.782163+0000","last_unstale":"2024-02-23T00:51:50.268878+0000","last_undegraded":"2024-02-23T00:51:50.268878+0000","last_fullsized":"2024-02-23T00:51:50.268878+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:28.060109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:28.060109+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":556,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.272483+0000","last_change":"2024-02-23T00:50:18.629287+0000","last_active":"2024-02-23T00:51:50.272483+0000","last_peered":"2024-02-23T00:51:50.272483+0000","last_clean":"2024-02-23T00:51:50.272483+0000","last_became_active":"2024-02-23T00:47:28.215452+0000","last_became_peered":"2024-02-23T00:47:28.215452+0000","last_unstale":"2024-02-23T00:51:50.272483+0000","last_undegraded":"2024-02-23T00:51:50.272483+0000","last_fullsized":"2024-02-23T00:51:50.272483+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:18.629203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_clean_scrub_stamp":"2024-02-23T00:50:18.629203+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":502,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.276125+0000","last_change":"2024-02-23T00:50:56.774466+0000","last_active":"2024-02-23T00:51:49.276125+0000","last_peered":"2024-02-23T00:51:49.276125+0000","last_clean":"2024-02-23T00:51:49.276125+0000","last_became_active":"2024-02-23T00:47:28.203555+0000","last_became_peered":"2024-02-23T00:47:28.203555+0000","last_unstale":"2024-02-23T00:51:49.276125+0000","last_undegraded":"2024-02-23T00:51:49.276125+0000","last_fullsized":"2024-02-23T00:51:49.276125+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:56.774382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:56.774382+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"457'33","reported_seq":612,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.272839+0000","last_change":"2024-02-23T00:49:46.364498+0000","last_active":"2024-02-23T00:51:50.272839+0000","last_peered":"2024-02-23T00:51:50.272839+0000","last_clean":"2024-02-23T00:51:50.272839+0000","last_became_active":"2024-02-23T00:39:16.786791+0000","last_became_peered":"2024-02-23T00:39:16.786791+0000","last_unstale":"2024-02-23T00:51:50.272839+0000","last_undegraded":"2024-02-23T00:51:50.272839+0000","last_fullsized":"2024-02-23T00:51:50.272839+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'33","last_scrub_stamp":"2024-02-23T00:49:46.364418+0000","last_deep_scrub":"95'27","last_deep_scrub_stamp":"2024-02-23T00:41:55.919895+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.364418+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":564,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.272049+0000","last_change":"2024-02-23T00:51:42.249052+0000","last_active":"2024-02-23T00:51:49.272049+0000","last_peered":"2024-02-23T00:51:49.272049+0000","last_clean":"2024-02-23T00:51:49.272049+0000","last_became_active":"2024-02-23T00:47:28.208293+0000","last_became_peered":"2024-02-23T00:47:28.208293+0000","last_unstale":"2024-02-23T00:51:49.272049+0000","last_undegraded":"2024-02-23T00:51:49.272049+0000","last_fullsized":"2024-02-23T00:51:49.272049+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:42.248982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:42.248982+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":548,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.275997+0000","last_change":"2024-02-23T00:49:46.612538+0000","last_active":"2024-02-23T00:51:49.275997+0000","last_peered":"2024-02-23T00:51:49.275997+0000","last_clean":"2024-02-23T00:51:49.275997+0000","last_became_active":"2024-02-23T00:39:18.796431+0000","last_became_peered":"2024-02-23T00:39:18.796431+0000","last_unstale":"2024-02-23T00:51:49.275997+0000","last_undegraded":"2024-02-23T00:51:49.275997+0000","last_fullsized":"2024-02-23T00:51:49.275997+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:46.612461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.612461+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":493,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.276598+0000","last_change":"2024-02-23T00:50:26.630228+0000","last_active":"2024-02-23T00:51:49.276598+0000","last_peered":"2024-02-23T00:51:49.276598+0000","last_clean":"2024-02-23T00:51:49.276598+0000","last_became_active":"2024-02-23T00:42:58.832885+0000","last_became_peered":"2024-02-23T00:42:58.832885+0000","last_unstale":"2024-02-23T00:51:49.276598+0000","last_undegraded":"2024-02-23T00:51:49.276598+0000","last_fullsized":"2024-02-23T00:51:49.276598+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:26.630088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:26.630088+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"457'17","reported_seq":581,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.271862+0000","last_change":"2024-02-23T00:50:15.590233+0000","last_active":"2024-02-23T00:51:50.271862+0000","last_peered":"2024-02-23T00:51:50.271862+0000","last_clean":"2024-02-23T00:51:50.271862+0000","last_became_active":"2024-02-23T00:39:16.782762+0000","last_became_peered":"2024-02-23T00:39:16.782762+0000","last_unstale":"2024-02-23T00:51:50.271862+0000","last_undegraded":"2024-02-23T00:51:50.271862+0000","last_fullsized":"2024-02-23T00:51:50.271862+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'17","last_scrub_stamp":"2024-02-23T00:50:15.590096+0000","last_deep_scrub":"82'15","last_deep_scrub_stamp":"2024-02-23T00:41:50.975683+0000","last_clean_scrub_stamp":"2024-02-23T00:50:15.590096+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":21,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":555,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.272737+0000","last_change":"2024-02-23T00:51:08.199826+0000","last_active":"2024-02-23T00:51:49.272737+0000","last_peered":"2024-02-23T00:51:49.272737+0000","last_clean":"2024-02-23T00:51:49.272737+0000","last_became_active":"2024-02-23T00:39:14.783469+0000","last_became_peered":"2024-02-23T00:39:14.783469+0000","last_unstale":"2024-02-23T00:51:49.272737+0000","last_undegraded":"2024-02-23T00:51:49.272737+0000","last_fullsized":"2024-02-23T00:51:49.272737+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:08.199749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:08.199749+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":553,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.272379+0000","last_change":"2024-02-23T00:51:13.264970+0000","last_active":"2024-02-23T00:51:49.272379+0000","last_peered":"2024-02-23T00:51:49.272379+0000","last_clean":"2024-02-23T00:51:49.272379+0000","last_became_active":"2024-02-23T00:39:18.796952+0000","last_became_peered":"2024-02-23T00:39:18.796952+0000","last_unstale":"2024-02-23T00:51:49.272379+0000","last_undegraded":"2024-02-23T00:51:49.272379+0000","last_fullsized":"2024-02-23T00:51:49.272379+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:13.264868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:06.491793+0000","last_clean_scrub_stamp":"2024-02-23T00:51:13.264868+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":495,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.271848+0000","last_change":"2024-02-23T00:51:03.460376+0000","last_active":"2024-02-23T00:51:50.271848+0000","last_peered":"2024-02-23T00:51:50.271848+0000","last_clean":"2024-02-23T00:51:50.271848+0000","last_became_active":"2024-02-23T00:42:58.758147+0000","last_became_peered":"2024-02-23T00:42:58.758147+0000","last_unstale":"2024-02-23T00:51:50.271848+0000","last_undegraded":"2024-02-23T00:51:50.271848+0000","last_fullsized":"2024-02-23T00:51:50.271848+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:03.460302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:23.224663+0000","last_clean_scrub_stamp":"2024-02-23T00:51:03.460302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"457'21","reported_seq":589,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.270589+0000","last_change":"2024-02-23T00:49:32.434383+0000","last_active":"2024-02-23T00:51:50.270589+0000","last_peered":"2024-02-23T00:51:50.270589+0000","last_clean":"2024-02-23T00:51:50.270589+0000","last_became_active":"2024-02-23T00:39:16.784606+0000","last_became_peered":"2024-02-23T00:39:16.784606+0000","last_unstale":"2024-02-23T00:51:50.270589+0000","last_undegraded":"2024-02-23T00:51:50.270589+0000","last_fullsized":"2024-02-23T00:51:50.270589+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-23T00:49:32.434282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:32.434282+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":556,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.268275+0000","last_change":"2024-02-23T00:50:19.967076+0000","last_active":"2024-02-23T00:51:50.268275+0000","last_peered":"2024-02-23T00:51:50.268275+0000","last_clean":"2024-02-23T00:51:50.268275+0000","last_became_active":"2024-02-23T00:39:14.785362+0000","last_became_peered":"2024-02-23T00:39:14.785362+0000","last_unstale":"2024-02-23T00:51:50.268275+0000","last_undegraded":"2024-02-23T00:51:50.268275+0000","last_fullsized":"2024-02-23T00:51:50.268275+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:19.967009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:19.967009+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":538,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.272797+0000","last_change":"2024-02-23T00:49:37.537279+0000","last_active":"2024-02-23T00:51:49.272797+0000","last_peered":"2024-02-23T00:51:49.272797+0000","last_clean":"2024-02-23T00:51:49.272797+0000","last_became_active":"2024-02-23T00:39:18.796312+0000","last_became_peered":"2024-02-23T00:39:18.796312+0000","last_unstale":"2024-02-23T00:51:49.272797+0000","last_undegraded":"2024-02-23T00:51:49.272797+0000","last_fullsized":"2024-02-23T00:51:49.272797+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:37.537211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:47.226331+0000","last_clean_scrub_stamp":"2024-02-23T00:49:37.537211+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":494,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.273634+0000","last_change":"2024-02-23T00:50:28.479578+0000","last_active":"2024-02-23T00:51:50.273634+0000","last_peered":"2024-02-23T00:51:50.273634+0000","last_clean":"2024-02-23T00:51:50.273634+0000","last_became_active":"2024-02-23T00:42:58.838257+0000","last_became_peered":"2024-02-23T00:42:58.838257+0000","last_unstale":"2024-02-23T00:51:50.273634+0000","last_undegraded":"2024-02-23T00:51:50.273634+0000","last_fullsized":"2024-02-23T00:51:50.273634+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:28.479512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:50:28.479512+0000","last_clean_scrub_stamp":"2024-02-23T00:50:28.479512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"457'62","reported_seq":701,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.268137+0000","last_change":"2024-02-23T00:50:58.151569+0000","last_active":"2024-02-23T00:51:50.268137+0000","last_peered":"2024-02-23T00:51:50.268137+0000","last_clean":"2024-02-23T00:51:50.268137+0000","last_became_active":"2024-02-23T00:47:28.205150+0000","last_became_peered":"2024-02-23T00:47:28.205150+0000","last_unstale":"2024-02-23T00:51:50.268137+0000","last_undegraded":"2024-02-23T00:51:50.268137+0000","last_fullsized":"2024-02-23T00:51:50.268137+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'62","last_scrub_stamp":"2024-02-23T00:50:58.151494+0000","last_deep_scrub":"88'50","last_deep_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_clean_scrub_stamp":"2024-02-23T00:50:58.151494+0000","log_size":62,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":92,"num_read_kb":79,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":553,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.273401+0000","last_change":"2024-02-23T00:50:34.600693+0000","last_active":"2024-02-23T00:51:50.273401+0000","last_peered":"2024-02-23T00:51:50.273401+0000","last_clean":"2024-02-23T00:51:50.273401+0000","last_became_active":"2024-02-23T00:39:14.782000+0000","last_became_peered":"2024-02-23T00:39:14.782000+0000","last_unstale":"2024-02-23T00:51:50.273401+0000","last_undegraded":"2024-02-23T00:51:50.273401+0000","last_fullsized":"2024-02-23T00:51:50.273401+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:34.600592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_clean_scrub_stamp":"2024-02-23T00:50:34.600592+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":547,"reported_epoch":581,"state":"active+clean","last_fresh":"2024-02-23T00:51:50.270886+0000","last_change":"2024-02-23T00:51:47.686009+0000","last_active":"2024-02-23T00:51:50.270886+0000","last_peered":"2024-02-23T00:51:50.270886+0000","last_clean":"2024-02-23T00:51:50.270886+0000","last_became_active":"2024-02-23T00:39:18.799723+0000","last_became_peered":"2024-02-23T00:39:18.799723+0000","last_unstale":"2024-02-23T00:51:50.270886+0000","last_undegraded":"2024-02-23T00:51:50.270886+0000","last_fullsized":"2024-02-23T00:51:50.270886+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:47.685918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:51:47.685918+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":493,"reported_epoch":580,"state":"active+clean","last_fresh":"2024-02-23T00:51:49.274936+0000","last_change":"2024-02-23T00:49:44.601162+0000","last_active":"2024-02-23T00:51:49.274936+0000","last_peered":"2024-02-23T00:51:49.274936+0000","last_clean":"2024-02-23T00:51:49.274936+0000","last_became_active":"2024-02-23T00:42:58.824573+0000","last_became_peered":"2024-02-23T00:42:58.824573+0000","last_unstale":"2024-02-23T00:51:49.274936+0000","last_undegraded":"2024-02-23T00:51:49.274936+0000","last_fullsized":"2024-02-23T00:51:49.274936+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:44.601086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:44.601086+0000","last_clean_scrub_stamp":"2024-02-23T00:49:44.601086+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0],"acting":[5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":233,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":7},{"poolid":232,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":7},{"poolid":229,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":24,"acting":24,"num_store_stats":6},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":12,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":36,"acting":36,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1508,"num_read_kb":1233,"num_write":895,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":1486,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":955,"ondisk_log_size":955,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":747,"num_read_kb":742,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234139,"num_pgs":77,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":316168,"kb_used_data":2560,"kb_used_omap":0,"kb_used_meta":313600,"kb_avail":93428984,"statfs":{"total":95995035648,"available":95671279616,"internally_reserved":0,"allocated":2621440,"data_stored":1094948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321126400},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.50600000000000001,"15min":0.53200000000000003},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":4.9420000000000002},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51500000000000001,"15min":0.52000000000000002},"min":{"1min":0.34699999999999998,"5min":0.32000000000000001,"15min":0.24299999999999999},"max":{"1min":0.82599999999999996,"5min":1.151,"15min":4.9729999999999999},"last":0.42499999999999999}]},{"osd":1,"last update":"Fri Feb 23 00:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.50800000000000001,"15min":0.61199999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.626,"5min":1.25,"15min":10.786},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52400000000000002,"15min":0.61899999999999999},"min":{"1min":0.38600000000000001,"5min":0.33900000000000002,"15min":0.28100000000000003},"max":{"1min":0.79300000000000004,"5min":1.107,"15min":11.885999999999999},"last":0.38100000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.51700000000000002,"15min":0.55000000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.28999999999999998},"max":{"1min":0.64900000000000002,"5min":0.80600000000000005,"15min":5.0789999999999997},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52800000000000002,"15min":0.54500000000000004},"min":{"1min":0.38900000000000001,"5min":0.34200000000000003,"15min":0.316},"max":{"1min":0.72299999999999998,"5min":1.1180000000000001,"15min":5.1280000000000001},"last":0.46999999999999997}]},{"osd":3,"last update":"Fri Feb 23 00:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.52200000000000002,"15min":0.61799999999999999},"min":{"1min":0.33200000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.74399999999999999,"5min":1.095,"15min":15.712999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54700000000000004,"15min":0.61399999999999999},"min":{"1min":0.35799999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.879,"5min":0.879,"15min":9.4090000000000007},"last":0.51000000000000001}]},{"osd":4,"last update":"Fri Feb 23 00:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.439,"15min":0.439},"min":{"1min":0.30199999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.753,"5min":1.6819999999999999,"15min":2.0350000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.45500000000000002,"15min":0.48899999999999999},"min":{"1min":0.33600000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.68400000000000005,"5min":0.86699999999999999,"15min":11.236000000000001},"last":0.41399999999999998}]},{"osd":5,"last update":"Fri Feb 23 00:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47299999999999998,"15min":0.49199999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27000000000000002},"max":{"1min":0.625,"5min":0.73199999999999998,"15min":5.4000000000000004},"last":0.437},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.50700000000000001,"15min":0.5},"min":{"1min":0.35499999999999998,"5min":0.29099999999999998,"15min":0.26700000000000002},"max":{"1min":0.67500000000000004,"5min":1.728,"15min":1.728},"last":0.49399999999999999}]},{"osd":6,"last update":"Fri Feb 23 00:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51200000000000001,"15min":0.51100000000000001},"min":{"1min":0.42099999999999999,"5min":0.32200000000000001,"15min":0.317},"max":{"1min":0.75700000000000001,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.50900000000000001,"15min":0.54900000000000004},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.626,"5min":0.70499999999999996,"15min":13.395},"last":0.44800000000000001}]}]},{"osd":6,"up_from":47,"seq":201863463074,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":307984,"kb_used_data":2568,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93437168,"statfs":{"total":95995035648,"available":95679660032,"internally_reserved":0,"allocated":2629632,"data_stored":1095140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:51:07 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.505,"15min":0.68200000000000005},"min":{"1min":0.22500000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":30.641999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.53600000000000003,"15min":0.69699999999999995},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.577,"5min":3.7360000000000002,"15min":29.292000000000002},"last":0.315}]},{"osd":1,"last update":"Fri Feb 23 00:51:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51500000000000001,"15min":0.68300000000000005},"min":{"1min":0.308,"5min":0.247,"15min":0.247},"max":{"1min":0.77100000000000002,"5min":0.77200000000000002,"15min":28.186},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.52900000000000003,"15min":0.86299999999999999},"min":{"1min":0.39400000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.752,"5min":1.083,"15min":33.104999999999997},"last":0.39600000000000002}]},{"osd":2,"last update":"Fri Feb 23 00:51:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52700000000000002,"15min":0.71399999999999997},"min":{"1min":0.36699999999999999,"5min":0.29399999999999998,"15min":0.27400000000000002},"max":{"1min":0.69599999999999995,"5min":1.1160000000000001,"15min":19.960000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53400000000000003,"15min":0.64600000000000002},"min":{"1min":0.40400000000000003,"5min":0.316,"15min":0.29899999999999999},"max":{"1min":0.79000000000000004,"5min":1.665,"15min":8.5860000000000003},"last":0.32500000000000001}]},{"osd":3,"last update":"Fri Feb 23 00:51:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54300000000000004,"15min":0.626},"min":{"1min":0.40999999999999998,"5min":0.29199999999999998,"15min":0.28599999999999998},"max":{"1min":0.78000000000000003,"5min":0.78400000000000003,"15min":19.562000000000001},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56899999999999995,"15min":0.67000000000000004},"min":{"1min":0.42399999999999999,"5min":0.33500000000000002,"15min":0.33000000000000002},"max":{"1min":0.80700000000000005,"5min":2.6259999999999999,"15min":14.199},"last":0.41899999999999998}]},{"osd":4,"last update":"Fri Feb 23 00:51:07 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.44700000000000001,"15min":0.47999999999999998},"min":{"1min":0.317,"5min":0.245,"15min":0.218},"max":{"1min":0.61499999999999999,"5min":1.6220000000000001,"15min":11.109999999999999},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.45900000000000002,"15min":0.54800000000000004},"min":{"1min":0.32700000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.751,"5min":0.83099999999999996,"15min":16.734000000000002},"last":0.377}]},{"osd":5,"last update":"Fri Feb 23 00:51:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48299999999999998,"15min":0.51300000000000001},"min":{"1min":0.34499999999999997,"5min":0.308,"15min":0.249},"max":{"1min":0.72399999999999998,"5min":1.784,"15min":14.154999999999999},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.58499999999999996,"15min":0.55500000000000005},"min":{"1min":0.37,"5min":0.318,"15min":0.26400000000000001},"max":{"1min":0.68500000000000005,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.34000000000000002}]},{"osd":7,"last update":"Fri Feb 23 00:51:23 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.51000000000000001,"15min":0.53100000000000003},"min":{"1min":0.38600000000000001,"5min":0.29399999999999998,"15min":0.26800000000000002},"max":{"1min":0.63100000000000001,"5min":1.726,"15min":8.548},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53900000000000003,"15min":0.60699999999999998},"min":{"1min":0.40300000000000002,"5min":0.34200000000000003,"15min":0.309},"max":{"1min":0.71199999999999997,"5min":0.88500000000000001,"15min":15.618},"last":0.40200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575036,"num_pgs":76,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308000,"kb_used_data":2584,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93437152,"statfs":{"total":95995035648,"available":95679643648,"internally_reserved":0,"allocated":2646016,"data_stored":1096745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.39000000000000001,"15min":0.68400000000000005},"min":{"1min":0.23799999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.56999999999999995,"5min":6.202,"15min":78.361000000000004},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.379,"5min":0.36899999999999999,"15min":0.44400000000000001},"min":{"1min":0.26900000000000002,"5min":0.216,"15min":0.191},"max":{"1min":0.64300000000000002,"5min":0.69999999999999996,"15min":5.8849999999999998},"last":0.29699999999999999}]},{"osd":2,"last update":"Fri Feb 23 00:51:13 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.436,"15min":0.67100000000000004},"min":{"1min":0.24399999999999999,"5min":0.20499999999999999,"15min":0.188},"max":{"1min":2.3010000000000002,"5min":7.5789999999999997,"15min":48.606000000000002},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.40799999999999997,"15min":0.57599999999999996},"min":{"1min":0.29399999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.63500000000000001,"5min":1,"15min":23.911000000000001},"last":0.41899999999999998}]},{"osd":3,"last update":"Fri Feb 23 00:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.40500000000000003,"15min":0.59499999999999997},"min":{"1min":0.27800000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":35.244999999999997},"last":0.379},{"interface":"front","average":{"1min":0.437,"5min":0.47699999999999998,"15min":0.68700000000000006},"min":{"1min":0.28899999999999998,"5min":0.26900000000000002,"15min":0.248},"max":{"1min":0.73999999999999999,"5min":7.641,"15min":56.853999999999999},"last":0.36499999999999999}]},{"osd":4,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.63300000000000001,"15min":0.82299999999999995},"min":{"1min":0.40100000000000002,"5min":0.32700000000000001,"15min":0.29299999999999998},"max":{"1min":0.97999999999999998,"5min":7.6829999999999998,"15min":37.646999999999998},"last":0.624},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.64700000000000002,"15min":0.70799999999999996},"min":{"1min":0.48099999999999998,"5min":0.35899999999999999,"15min":0.31900000000000001},"max":{"1min":0.91400000000000003,"5min":6.0519999999999996,"15min":9.6859999999999999},"last":0.69699999999999995}]},{"osd":5,"last update":"Fri Feb 23 00:51:35 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.65000000000000002,"15min":0.83299999999999996},"min":{"1min":0.35399999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.81999999999999995,"5min":7.742,"15min":48.792000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.625,"15min":0.71199999999999997},"min":{"1min":0.39100000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77900000000000003,"5min":0.92600000000000005,"15min":11.872999999999999},"last":0.68000000000000005}]},{"osd":6,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.64100000000000001,"15min":0.82199999999999995},"min":{"1min":0.40000000000000002,"5min":0.37,"15min":0.318},"max":{"1min":0.94199999999999995,"5min":7.7130000000000001,"15min":40.741},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60399999999999998,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.38400000000000001,"15min":0.34000000000000002},"max":{"1min":0.82199999999999995,"5min":0.85999999999999999,"15min":6.0069999999999997},"last":0.55700000000000005}]},{"osd":7,"last update":"Fri Feb 23 00:51:35 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61899999999999999,"15min":0.71399999999999997},"min":{"1min":0.41799999999999998,"5min":0.40200000000000002,"15min":0.36699999999999999},"max":{"1min":0.83899999999999997,"5min":0.92900000000000005,"15min":20.225000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.66500000000000004,"15min":0.73899999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.95499999999999996,"5min":6.0949999999999998,"15min":20.254999999999999},"last":0.66800000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738562,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":312108,"kb_used_data":2596,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93433044,"statfs":{"total":95995035648,"available":95675437056,"internally_reserved":0,"allocated":2658304,"data_stored":1107453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 23 00:50:52 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.35199999999999998,"15min":0.60299999999999998},"min":{"1min":0.224,"5min":0.20999999999999999,"15min":0.19500000000000001},"max":{"1min":0.68200000000000005,"5min":0.81399999999999995,"15min":28.856999999999999},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.373,"5min":0.39500000000000002,"15min":0.629},"min":{"1min":0.249,"5min":0.23300000000000001,"15min":0.20799999999999999},"max":{"1min":0.59099999999999997,"5min":2.3650000000000002,"15min":20.577000000000002},"last":0.312}]},{"osd":2,"last update":"Fri Feb 23 00:51:18 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.40400000000000003,"15min":0.60999999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.20499999999999999},"max":{"1min":0.61099999999999999,"5min":2.3889999999999998,"15min":13.863},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.35099999999999998,"15min":0.53800000000000003},"min":{"1min":0.252,"5min":0.223,"15min":0.19800000000000001},"max":{"1min":0.504,"5min":0.65000000000000002,"15min":10.455},"last":0.36699999999999999}]},{"osd":3,"last update":"Fri Feb 23 00:51:48 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.378,"15min":0.77300000000000002},"min":{"1min":0.248,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.51600000000000001,"5min":0.73099999999999998,"15min":35.566000000000003},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.377,"5min":0.435,"15min":0.76000000000000001},"min":{"1min":0.27800000000000002,"5min":0.27400000000000002,"15min":0.26100000000000001},"max":{"1min":0.57499999999999996,"5min":2.4049999999999998,"15min":20.702999999999999},"last":0.35299999999999998}]},{"osd":4,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55900000000000005,"15min":0.752},"min":{"1min":0.40899999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.755,"5min":0.94699999999999995,"15min":10.584},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.61599999999999999,"15min":0.78100000000000003},"min":{"1min":0.42899999999999999,"5min":0.38100000000000001,"15min":0.35099999999999998},"max":{"1min":0.78900000000000003,"5min":2.5019999999999998,"15min":13.198},"last":0.71799999999999997}]},{"osd":5,"last update":"Fri Feb 23 00:51:37 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54500000000000004,"15min":0.71499999999999997},"min":{"1min":0.36699999999999999,"5min":0.33800000000000002,"15min":0.32200000000000001},"max":{"1min":0.72599999999999998,"5min":0.90800000000000003,"15min":10.397},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.60699999999999998,"15min":0.78100000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76800000000000002,"5min":0.92600000000000005,"15min":17.297000000000001},"last":0.67500000000000004}]},{"osd":6,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.58899999999999997,"15min":0.78000000000000003},"min":{"1min":0.39400000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.76400000000000001,"5min":0.94399999999999995,"15min":17.373999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.61299999999999999,"15min":0.76200000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.36799999999999999},"max":{"1min":0.72199999999999998,"5min":2.556,"15min":13.973000000000001},"last":0.65700000000000003}]},{"osd":7,"last update":"Fri Feb 23 00:51:27 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.59099999999999997,"15min":0.77800000000000002},"min":{"1min":0.379,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83399999999999996,"5min":0.96099999999999997,"15min":17.399999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.63100000000000001,"15min":0.78100000000000003},"min":{"1min":0.46000000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.752,"5min":2.5289999999999999,"15min":16.427},"last":0.69699999999999995}]}]},{"osd":2,"up_from":0,"seq":77309411511,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":26,"seq":111669149872,"num_pgs":86,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":312116,"kb_used_data":2604,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93433036,"statfs":{"total":95995035648,"available":95675428864,"internally_reserved":0,"allocated":2666496,"data_stored":1106831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.42999999999999999,"15min":0.53200000000000003},"min":{"1min":0.216,"5min":0.21199999999999999,"15min":0.20899999999999999},"max":{"1min":0.63600000000000001,"5min":8.3300000000000001,"15min":12.212999999999999},"last":0.251},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.42099999999999999,"15min":0.60299999999999998},"min":{"1min":0.25,"5min":0.22800000000000001,"15min":0.21199999999999999},"max":{"1min":0.48599999999999999,"5min":2.9740000000000002,"15min":11.714},"last":0.26500000000000001}]},{"osd":1,"last update":"Fri Feb 23 00:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.40999999999999998,"15min":0.624},"min":{"1min":0.25900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.52100000000000002,"5min":3.056,"15min":11.742000000000001},"last":0.31},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.433,"15min":0.59799999999999998},"min":{"1min":0.28199999999999997,"5min":0.24099999999999999,"15min":0.19900000000000001},"max":{"1min":0.71599999999999997,"5min":2.7709999999999999,"15min":8.2270000000000003},"last":0.28199999999999997}]},{"osd":2,"last update":"Fri Feb 23 00:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.47699999999999998,"15min":0.68300000000000005},"min":{"1min":0.30499999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.53600000000000003,"5min":3.242,"15min":13.359999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.49099999999999999,"15min":0.66100000000000003},"min":{"1min":0.28100000000000003,"5min":0.27500000000000002,"15min":0.26100000000000001},"max":{"1min":0.68700000000000006,"5min":8.5820000000000007,"15min":21.122},"last":0.29399999999999998}]},{"osd":4,"last update":"Fri Feb 23 00:51:10 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.58899999999999997,"15min":0.755},"min":{"1min":0.40300000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":3.1419999999999999,"5min":3.1419999999999999,"15min":11.644},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.74199999999999999,"15min":0.753},"min":{"1min":0.36899999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":2.9020000000000001,"5min":12.430999999999999,"15min":12.430999999999999},"last":0.40100000000000002}]},{"osd":5,"last update":"Fri Feb 23 00:51:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.67800000000000005,"15min":0.68899999999999995},"min":{"1min":0.42899999999999999,"5min":0.37,"15min":0.35099999999999998},"max":{"1min":1,"5min":10.231999999999999,"15min":10.231999999999999},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.624,"15min":0.80000000000000004},"min":{"1min":0.41699999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.94999999999999996,"5min":3.1829999999999998,"15min":11.648},"last":0.45300000000000001}]},{"osd":6,"last update":"Fri Feb 23 00:50:53 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76300000000000001,"15min":0.75600000000000001},"min":{"1min":0.42399999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.9510000000000001,"5min":6.9420000000000002,"15min":6.9420000000000002},"last":0.377},{"interface":"front","average":{"1min":1.2050000000000001,"5min":0.751,"15min":0.72299999999999998},"min":{"1min":0.39200000000000002,"5min":0.376,"15min":0.376},"max":{"1min":10.965999999999999,"5min":10.965999999999999,"15min":10.965999999999999},"last":0.442}]},{"osd":7,"last update":"Fri Feb 23 00:51:34 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.66000000000000003,"15min":0.83299999999999996},"min":{"1min":0.441,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.1030000000000002,"5min":3.1030000000000002,"15min":11.676},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.66600000000000004,"15min":0.81000000000000005},"min":{"1min":0.501,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":2.8580000000000001,"5min":2.8580000000000001,"15min":7.9800000000000004},"last":0.434}]}]},{"osd":4,"up_from":32,"seq":137438953644,"num_pgs":73,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308004,"kb_used_data":2588,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93437148,"statfs":{"total":95995035648,"available":95679639552,"internally_reserved":0,"allocated":2650112,"data_stored":1106475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.504,"15min":0.52600000000000002},"min":{"1min":0.32200000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.65300000000000002,"5min":1.3819999999999999,"15min":3.5219999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.52400000000000002,"15min":0.58499999999999996},"min":{"1min":0.373,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.72799999999999998,"5min":1.8,"15min":8.6140000000000008},"last":0.52400000000000002}]},{"osd":1,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51600000000000001,"15min":0.57999999999999996},"min":{"1min":0.31900000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.70699999999999996,"5min":2.6600000000000001,"15min":9.6110000000000007},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.53500000000000003,"15min":0.58499999999999996},"min":{"1min":0.31900000000000001,"5min":0.26300000000000001,"15min":0.255},"max":{"1min":0.70099999999999996,"5min":2.6920000000000002,"15min":6.6950000000000003},"last":0.496}]},{"osd":2,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.51900000000000002,"15min":0.56999999999999995},"min":{"1min":0.34699999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.65700000000000003,"5min":1.0960000000000001,"15min":3.3069999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54000000000000004,"15min":0.57099999999999995},"min":{"1min":0.33700000000000002,"5min":0.33600000000000002,"15min":0.32500000000000001},"max":{"1min":0.68700000000000006,"5min":1.0189999999999999,"15min":5.8440000000000003},"last":0.55700000000000005}]},{"osd":3,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.67300000000000004,"15min":0.63400000000000001},"min":{"1min":0.38500000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.71999999999999997,"5min":9.1280000000000001,"15min":9.1280000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.58199999999999996,"15min":0.60399999999999998},"min":{"1min":0.39200000000000002,"5min":0.34599999999999997,"15min":0.29699999999999999},"max":{"1min":0.73299999999999998,"5min":4.0220000000000002,"15min":4.1909999999999998},"last":0.47999999999999998}]},{"osd":5,"last update":"Fri Feb 23 00:51:38 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.45400000000000001,"15min":0.42099999999999999},"min":{"1min":0.31900000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.61899999999999999,"5min":1.694,"15min":1.694},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.51600000000000001,"15min":0.47999999999999998},"min":{"1min":0.34000000000000002,"5min":0.29499999999999998,"15min":0.28899999999999998},"max":{"1min":0.73899999999999999,"5min":1.663,"15min":1.663},"last":0.40400000000000003}]},{"osd":6,"last update":"Fri Feb 23 00:51:05 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.44900000000000001,"15min":0.435},"min":{"1min":0.34000000000000002,"5min":0.28399999999999997,"15min":0.253},"max":{"1min":0.67200000000000004,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47199999999999998,"15min":0.443},"min":{"1min":0.34699999999999998,"5min":0.28799999999999998,"15min":0.215},"max":{"1min":0.64100000000000001,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":0.34899999999999998}]},{"osd":7,"last update":"Fri Feb 23 00:51:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.49399999999999999,"15min":0.48699999999999999},"min":{"1min":0.33100000000000002,"5min":0.27600000000000002,"15min":0.254},"max":{"1min":0.70999999999999996,"5min":2.294,"15min":2.294},"last":0.374},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56899999999999995,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.752,"5min":2.282,"15min":2.282},"last":0.58799999999999997}]}]},{"osd":5,"up_from":39,"seq":167503724711,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":307996,"kb_used_data":2580,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93437156,"statfs":{"total":95995035648,"available":95679647744,"internally_reserved":0,"allocated":2641920,"data_stored":1096425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:51:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.55300000000000005,"15min":0.61399999999999999},"min":{"1min":0.40400000000000003,"5min":0.253,"15min":0.246},"max":{"1min":0.67000000000000004,"5min":5.2220000000000004,"15min":23.881},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.57599999999999996,"15min":0.59399999999999997},"min":{"1min":0.38600000000000001,"5min":0.29099999999999998,"15min":0.27000000000000002},"max":{"1min":0.76500000000000001,"5min":3.9249999999999998,"15min":5.4649999999999999},"last":0.46899999999999997}]},{"osd":1,"last update":"Fri Feb 23 00:51:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54900000000000004,"15min":0.65600000000000003},"min":{"1min":0.35299999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.73899999999999999,"5min":3.9460000000000002,"15min":18.175999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.56899999999999995,"15min":0.628},"min":{"1min":0.40100000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.71599999999999997,"5min":3.976,"15min":9.0950000000000006},"last":0.51800000000000002}]},{"osd":2,"last update":"Fri Feb 23 00:51:32 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57899999999999996,"15min":0.71199999999999997},"min":{"1min":0.37,"5min":0.255,"15min":0.255},"max":{"1min":1.036,"5min":4.5899999999999999,"15min":22.282},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.58599999999999997,"15min":0.72599999999999998},"min":{"1min":0.34399999999999997,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.66900000000000004,"5min":3.8540000000000001,"15min":23.925000000000001},"last":0.47699999999999998}]},{"osd":3,"last update":"Fri Feb 23 00:51:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55800000000000005,"15min":0.621},"min":{"1min":0.35899999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.194,"5min":1.23,"15min":6.5730000000000004},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.60199999999999998,"15min":0.65500000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.33500000000000002},"max":{"1min":0.70499999999999996,"5min":3.9980000000000002,"15min":8.1760000000000002},"last":0.56399999999999995}]},{"osd":4,"last update":"Fri Feb 23 00:51:32 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.48199999999999998,"15min":0.436},"min":{"1min":0.33000000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.60699999999999998,"5min":4.593,"15min":4.593},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.49199999999999999,"15min":0.47399999999999998},"min":{"1min":0.35799999999999998,"5min":0.27900000000000003,"15min":0.27300000000000002},"max":{"1min":1.169,"5min":1.5860000000000001,"15min":3.879},"last":0.40400000000000003}]},{"osd":6,"last update":"Fri Feb 23 00:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51300000000000001,"15min":0.47599999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.27900000000000003},"max":{"1min":0.71099999999999997,"5min":4.6740000000000004,"15min":4.6740000000000004},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.49099999999999999,"15min":0.47699999999999998},"min":{"1min":0.35999999999999999,"5min":0.28000000000000003,"15min":0.27100000000000002},"max":{"1min":0.73199999999999998,"5min":1.159,"15min":1.159},"last":0.34799999999999998}]},{"osd":7,"last update":"Fri Feb 23 00:51:26 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.39000000000000001,"5min":0.32500000000000001,"15min":0.30399999999999999},"max":{"1min":0.74099999999999999,"5min":1.302,"15min":7.0300000000000002},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52600000000000002,"15min":0.53900000000000003},"min":{"1min":0.36099999999999999,"5min":0.314,"15min":0.29099999999999998},"max":{"1min":0.64600000000000002,"5min":2.016,"15min":9.0579999999999998},"last":0.373}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":229,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":229,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":229,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":229,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":229,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":229,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":232,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":232,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":232,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":232,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":232,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":232,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":232,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":233,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":233,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":233,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":233,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":233,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":233,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":233,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-23T00:51:53.441 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:51:53.442 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:53.443 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:51:53.545 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:51:53.573 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:52.273818+0000 mon.a (mon.0) 3060 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-99-cache", "mode": "writeback"}]: dispatch 2024-02-23T00:51:53.573 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:52.508425+0000 mgr.y (mgr.14517) 524 : audit [DBG] from='client.41741 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:51:53.573 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:53.268660+0000 mon.a (mon.0) 3061 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:51:53.574 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:53.273708+0000 mon.a (mon.0) 3062 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-99-cache", "mode": "writeback"}]': finished 2024-02-23T00:51:53.574 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:53.273754+0000 mon.a (mon.0) 3063 : cluster [DBG] osdmap e584: 8 total, 8 up, 7 in 2024-02-23T00:51:53.574 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:53.274713+0000 mon.a (mon.0) 3064 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-99-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:51:53.576 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:53.596 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:51:53.599 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:53.600 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:51:53.700 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:51:53.710 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:51:53.727 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:53.746 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:51:53.747 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:53.748 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:51:53.772 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-23T00:51:53.848 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:51:53.876 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:53.950 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:51:53.953 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:53.953 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:51:54.054 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:51:54.082 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:54.498 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:51:54.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:52.780025+0000 mgr.y (mgr.14517) 525 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:51:54.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:53.348584+0000 mgr.y (mgr.14517) 526 : cluster [DBG] pgmap v880: 192 pgs: 192 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:54.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:54.274141+0000 mon.a (mon.0) 3065 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-99-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-23T00:51:54.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:54.274213+0000 mon.a (mon.0) 3066 : cluster [DBG] osdmap e585: 8 total, 8 up, 7 in 2024-02-23T00:51:54.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:54.275235+0000 mon.a (mon.0) 3067 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-99-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:51:54.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:52.780025+0000 mgr.y (mgr.14517) 525 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:51:54.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:53.348584+0000 mgr.y (mgr.14517) 526 : cluster [DBG] pgmap v880: 192 pgs: 192 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:54.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:54.274141+0000 mon.a (mon.0) 3065 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-99-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-23T00:51:54.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:54.274213+0000 mon.a (mon.0) 3066 : cluster [DBG] osdmap e585: 8 total, 8 up, 7 in 2024-02-23T00:51:54.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:54.275235+0000 mon.a (mon.0) 3067 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-99-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:51:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:52.780025+0000 mgr.y (mgr.14517) 525 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:51:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:53.348584+0000 mgr.y (mgr.14517) 526 : cluster [DBG] pgmap v880: 192 pgs: 192 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:54.274141+0000 mon.a (mon.0) 3065 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-99-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-23T00:51:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:54.274213+0000 mon.a (mon.0) 3066 : cluster [DBG] osdmap e585: 8 total, 8 up, 7 in 2024-02-23T00:51:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:54.275235+0000 mon.a (mon.0) 3067 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-99-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:51:54.762 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:51:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:51:54.965 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:54.966 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:51:55.067 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:51:55.099 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:55.120 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:51:55.123 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:55.124 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:51:55.224 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:51:55.263 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:55.712 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:51:55.715 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:55.715 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:51:55.816 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:51:55.849 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:55.870 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:51:55.873 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:55.873 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:51:55.974 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:51:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:56.202 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:51:56.205 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:56.205 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:51:56.306 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:51:56.332 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:56.441 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph pg dump --format=json 2024-02-23T00:51:56.455 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:51:56.457 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:56.458 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:51:56.544 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:55.251327+0000 mon.a (mon.0) 3068 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-99-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-23T00:51:56.544 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:55.251387+0000 mon.a (mon.0) 3069 : cluster [DBG] osdmap e586: 8 total, 8 up, 7 in 2024-02-23T00:51:56.545 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:55.257092+0000 mon.a (mon.0) 3070 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-99-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-23T00:51:56.545 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:55.349143+0000 mgr.y (mgr.14517) 527 : cluster [DBG] pgmap v883: 192 pgs: 192 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:56.545 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:55.930748+0000 mon.a (mon.0) 3071 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:51:56.545 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:55.251327+0000 mon.a (mon.0) 3068 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-99-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-23T00:51:56.545 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:55.251387+0000 mon.a (mon.0) 3069 : cluster [DBG] osdmap e586: 8 total, 8 up, 7 in 2024-02-23T00:51:56.545 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:55.257092+0000 mon.a (mon.0) 3070 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-99-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-23T00:51:56.545 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:55.349143+0000 mgr.y (mgr.14517) 527 : cluster [DBG] pgmap v883: 192 pgs: 192 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:56.545 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:55.930748+0000 mon.a (mon.0) 3071 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:51:56.559 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:51:56.587 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:55.251327+0000 mon.a (mon.0) 3068 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-99-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-23T00:51:56.588 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:55.251387+0000 mon.a (mon.0) 3069 : cluster [DBG] osdmap e586: 8 total, 8 up, 7 in 2024-02-23T00:51:56.588 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:55.257092+0000 mon.a (mon.0) 3070 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-99-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-23T00:51:56.588 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:55.349143+0000 mgr.y (mgr.14517) 527 : cluster [DBG] pgmap v883: 192 pgs: 192 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:56.588 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:55.930748+0000 mon.a (mon.0) 3071 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:51:56.588 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:56.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.736+0000 7f9d32a5c700 1 -- 172.21.15.47:0/1339387703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d2c0fe510 msgr2=0x7f9d2c0f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:56.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.736+0000 7f9d32a5c700 1 --2- 172.21.15.47:0/1339387703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d2c0fe510 0x7f9d2c0f7d40 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 rx=0x7f9d20004870 tx=0x7f9d20009cb0).stop 2024-02-23T00:51:56.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.737+0000 7f9d32a5c700 1 -- 172.21.15.47:0/1339387703 shutdown_connections 2024-02-23T00:51:56.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.737+0000 7f9d32a5c700 1 --2- 172.21.15.47:0/1339387703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d2c0f8420 0x7f9d2c104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:56.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.737+0000 7f9d32a5c700 1 --2- 172.21.15.47:0/1339387703 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9d2c0fdbb0 0x7f9d2c0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:56.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.737+0000 7f9d32a5c700 1 --2- 172.21.15.47:0/1339387703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d2c0fe510 0x7f9d2c0f7d40 unknown :-1 s=CLOSED pgs=2587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:56.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.737+0000 7f9d32a5c700 1 -- 172.21.15.47:0/1339387703 >> 172.21.15.47:0/1339387703 conn(0x7f9d2c0f3800 msgr2=0x7f9d2c0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:56.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.737+0000 7f9d32a5c700 1 -- 172.21.15.47:0/1339387703 shutdown_connections 2024-02-23T00:51:56.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.737+0000 7f9d32a5c700 1 -- 172.21.15.47:0/1339387703 wait complete. 2024-02-23T00:51:56.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.737+0000 7f9d32a5c700 1 librados: starting msgr at 2024-02-23T00:51:56.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.737+0000 7f9d32a5c700 1 librados: starting objecter 2024-02-23T00:51:56.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.737+0000 7f9d32a5c700 1 Processor -- start 2024-02-23T00:51:56.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.738+0000 7f9d32a5c700 1 -- start start 2024-02-23T00:51:56.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.738+0000 7f9d32a5c700 1 librados: setting wanted keys 2024-02-23T00:51:56.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.738+0000 7f9d32a5c700 1 librados: calling monclient init 2024-02-23T00:51:56.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.738+0000 7f9d32a5c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9d2c0f8420 0x7f9d2c10def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:56.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.738+0000 7f9d32a5c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d2c0fdbb0 0x7f9d2c10e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:56.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.738+0000 7f9d32a5c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d2c0fe510 0x7f9d2c108090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:56.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.738+0000 7f9d32a5c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c108580 con 0x7f9d2c0fe510 2024-02-23T00:51:56.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.738+0000 7f9d32a5c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c1086f0 con 0x7f9d2c0fdbb0 2024-02-23T00:51:56.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.738+0000 7f9d32a5c700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c1088b0 con 0x7f9d2c0f8420 2024-02-23T00:51:56.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.738+0000 7f9d30ff9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d2c0fe510 0x7f9d2c108090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:56.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.738+0000 7f9d30ff9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d2c0fe510 0x7f9d2c108090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36360/0 (socket says 172.21.15.47:36360) 2024-02-23T00:51:56.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.738+0000 7f9d2b7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d2c0fdbb0 0x7f9d2c10e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.738+0000 7f9d30ff9700 1 -- 172.21.15.47:0/3623459494 learned_addr learned my addr 172.21.15.47:0/3623459494 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:51:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.738+0000 7f9d2bfff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9d2c0f8420 0x7f9d2c10def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.738+0000 7f9d30ff9700 1 -- 172.21.15.47:0/3623459494 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9d2c0f8420 msgr2=0x7f9d2c10def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.738+0000 7f9d30ff9700 1 --2- 172.21.15.47:0/3623459494 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9d2c0f8420 0x7f9d2c10def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.738+0000 7f9d30ff9700 1 -- 172.21.15.47:0/3623459494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d2c0fdbb0 msgr2=0x7f9d2c10e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.738+0000 7f9d30ff9700 1 --2- 172.21.15.47:0/3623459494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d2c0fdbb0 0x7f9d2c10e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.738+0000 7f9d30ff9700 1 -- 172.21.15.47:0/3623459494 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1c005040 con 0x7f9d2c0fe510 2024-02-23T00:51:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.739+0000 7f9d2bfff700 1 --2- 172.21.15.47:0/3623459494 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9d2c0f8420 0x7f9d2c10def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:51:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.739+0000 7f9d30ff9700 1 --2- 172.21.15.47:0/3623459494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d2c0fe510 0x7f9d2c108090 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 rx=0x7f9d2400d970 tx=0x7f9d24004e60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.739+0000 7f9d297fa700 10 client.?.objecter ms_handle_connect 0x7f9d2c0fe510 2024-02-23T00:51:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.739+0000 7f9d297fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:51:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.739+0000 7f9d297fa700 1 -- 172.21.15.47:0/3623459494 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d24004020 con 0x7f9d2c0fe510 2024-02-23T00:51:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.739+0000 7f9d297fa700 1 -- 172.21.15.47:0/3623459494 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f9d2400f040 con 0x7f9d2c0fe510 2024-02-23T00:51:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.739+0000 7f9d32a5c700 1 -- 172.21.15.47:0/3623459494 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d20005040 con 0x7f9d2c0fe510 2024-02-23T00:51:56.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.739+0000 7f9d297fa700 1 -- 172.21.15.47:0/3623459494 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d24003430 con 0x7f9d2c0fe510 2024-02-23T00:51:56.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.739+0000 7f9d32a5c700 10 client.31428.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:51:56.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.739+0000 7f9d32a5c700 1 -- 172.21.15.47:0/3623459494 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d2c108e70 con 0x7f9d2c0fe510 2024-02-23T00:51:56.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.739+0000 7f9d32a5c700 1 librados: init done 2024-02-23T00:51:56.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.740+0000 7f9d32a5c700 1 -- 172.21.15.47:0/3623459494 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d2c060280 con 0x7f9d2c0fe510 2024-02-23T00:51:56.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.740+0000 7f9d297fa700 1 -- 172.21.15.47:0/3623459494 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f9d2400b510 con 0x7f9d2c0fe510 2024-02-23T00:51:56.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.741+0000 7f9d297fa700 1 --2- 172.21.15.47:0/3623459494 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9d14062cc0 0x7f9d14065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:51:56.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.741+0000 7f9d297fa700 1 -- 172.21.15.47:0/3623459494 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(587..587 src has 482..587) v4 ==== 10403+0+0 (secure 0 0 0) 0x7f9d24084350 con 0x7f9d2c0fe510 2024-02-23T00:51:56.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.741+0000 7f9d297fa700 10 client.31428.objecter ms_dispatch 0x7f9d2c001550 osd_map(587..587 src has 482..587) v4 2024-02-23T00:51:56.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.741+0000 7f9d297fa700 3 client.31428.objecter handle_osd_map got epochs [587,587] > 0 2024-02-23T00:51:56.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.741+0000 7f9d297fa700 3 client.31428.objecter handle_osd_map decoding full epoch 587 2024-02-23T00:51:56.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.741+0000 7f9d2bfff700 1 --2- 172.21.15.47:0/3623459494 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9d14062cc0 0x7f9d14065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:51:56.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.741+0000 7f9d297fa700 20 client.31428.objecter dump_active .. 0 homeless 2024-02-23T00:51:56.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.741+0000 7f9d297fa700 10 client.31428.objecter ms_handle_connect 0x7f9d14062cc0 2024-02-23T00:51:56.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.741+0000 7f9d2bfff700 1 --2- 172.21.15.47:0/3623459494 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9d14062cc0 0x7f9d14065180 secure :-1 s=READY pgs=3134 cs=0 l=1 rev1=1 rx=0x7f9d1c009760 tx=0x7f9d1c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:51:56.747 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:51:56.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.746+0000 7f9d297fa700 1 -- 172.21.15.47:0/3623459494 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f9d24081060 con 0x7f9d2c0fe510 2024-02-23T00:51:56.751 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:56.751 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:51:56.852 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:51:56.880 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:56.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:56.931+0000 7f9d32a5c700 1 -- 172.21.15.47:0/3623459494 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7f9d2c109570 con 0x7f9d2c0fe510 2024-02-23T00:51:56.934 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:51:56.936 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:56.937 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:51:56.970 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:51:57.016 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:51:57.018 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:51:57.018 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-02-23T00:51:57.019 INFO:teuthology.orchestra.run.smithi047.stdout: "num_ops": 0 2024-02-23T00:51:57.019 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:51:57.037 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:51:57.066 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:57.087 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:51:57.089 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:57.089 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:51:57.190 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:51:57.219 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:57.241 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:51:57.243 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:57.244 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:51:57.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:57.262+0000 7f9d297fa700 1 -- 172.21.15.47:0/3623459494 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v588) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9d2404c5b0 con 0x7f9d2c0fe510 2024-02-23T00:51:57.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:57.264+0000 7f9d32a5c700 10 librados: watch_flush enter 2024-02-23T00:51:57.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:57.264+0000 7f9d32a5c700 10 librados: watch_flush exit 2024-02-23T00:51:57.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:57.264+0000 7f9d32a5c700 20 client.31428.objecter shutdown clearing up homeless session... 2024-02-23T00:51:57.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:57.264+0000 7f9d32a5c700 10 client.31428.objecter successfully canceled tick 2024-02-23T00:51:57.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:57.264+0000 7f9d32a5c700 1 -- 172.21.15.47:0/3623459494 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9d14062cc0 msgr2=0x7f9d14065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:57.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:57.264+0000 7f9d32a5c700 1 --2- 172.21.15.47:0/3623459494 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9d14062cc0 0x7f9d14065180 secure :-1 s=READY pgs=3134 cs=0 l=1 rev1=1 rx=0x7f9d1c009760 tx=0x7f9d1c00b040).stop 2024-02-23T00:51:57.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:57.264+0000 7f9d32a5c700 1 -- 172.21.15.47:0/3623459494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d2c0fe510 msgr2=0x7f9d2c108090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:51:57.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:57.264+0000 7f9d32a5c700 1 --2- 172.21.15.47:0/3623459494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d2c0fe510 0x7f9d2c108090 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 rx=0x7f9d2400d970 tx=0x7f9d24004e60).stop 2024-02-23T00:51:57.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:57.266+0000 7f9d32a5c700 1 -- 172.21.15.47:0/3623459494 shutdown_connections 2024-02-23T00:51:57.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:57.266+0000 7f9d32a5c700 1 --2- 172.21.15.47:0/3623459494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d2c0fe510 0x7f9d2c108090 unknown :-1 s=CLOSED pgs=2588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:57.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:57.266+0000 7f9d32a5c700 1 --2- 172.21.15.47:0/3623459494 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9d14062cc0 0x7f9d14065180 unknown :-1 s=CLOSED pgs=3134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:57.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:57.266+0000 7f9d32a5c700 1 --2- 172.21.15.47:0/3623459494 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9d2c0f8420 0x7f9d2c10def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:57.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:57.266+0000 7f9d32a5c700 1 --2- 172.21.15.47:0/3623459494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d2c0fdbb0 0x7f9d2c10e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:51:57.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:57.266+0000 7f9d32a5c700 1 -- 172.21.15.47:0/3623459494 >> 172.21.15.47:0/3623459494 conn(0x7f9d2c0f3800 msgr2=0x7f9d2c0fb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:51:57.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:57.266+0000 7f9d32a5c700 1 -- 172.21.15.47:0/3623459494 shutdown_connections 2024-02-23T00:51:57.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:57.266+0000 7f9d32a5c700 1 -- 172.21.15.47:0/3623459494 wait complete. 2024-02-23T00:51:57.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:51:57.266+0000 7f9d32a5c700 1 librados: shutdown 2024-02-23T00:51:57.269 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is set 2024-02-23T00:51:57.295 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:56.253836+0000 mon.a (mon.0) 3072 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:51:57.295 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:56.259590+0000 mon.a (mon.0) 3073 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-99-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-23T00:51:57.295 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:56.259661+0000 mon.a (mon.0) 3074 : cluster [DBG] osdmap e587: 8 total, 8 up, 7 in 2024-02-23T00:51:57.295 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:56.260802+0000 mon.a (mon.0) 3075 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-99-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-23T00:51:57.295 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:56.307469+0000 mon.a (mon.0) 3076 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:51:57.295 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:56.932420+0000 mon.a (mon.0) 3077 : audit [INF] from='client.? 172.21.15.47:0/3623459494' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:51:57.295 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:56.962246+0000 osd.4 (osd.4) 243 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:51:57.295 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:56.962419+0000 osd.4 (osd.4) 244 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:51:57.296 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:56.253836+0000 mon.a (mon.0) 3072 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:51:57.296 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:56.259590+0000 mon.a (mon.0) 3073 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-99-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-23T00:51:57.296 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:56.259661+0000 mon.a (mon.0) 3074 : cluster [DBG] osdmap e587: 8 total, 8 up, 7 in 2024-02-23T00:51:57.296 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:56.260802+0000 mon.a (mon.0) 3075 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-99-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-23T00:51:57.296 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:56.307469+0000 mon.a (mon.0) 3076 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:51:57.296 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:56.932420+0000 mon.a (mon.0) 3077 : audit [INF] from='client.? 172.21.15.47:0/3623459494' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:51:57.296 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:56.962246+0000 osd.4 (osd.4) 243 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:51:57.296 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:56.962419+0000 osd.4 (osd.4) 244 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:51:57.345 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:51:57.377 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:57.526 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:51:57.529 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:57.529 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:51:57.563 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:56.253836+0000 mon.a (mon.0) 3072 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:51:57.563 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:56.259590+0000 mon.a (mon.0) 3073 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-99-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-23T00:51:57.563 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:56.259661+0000 mon.a (mon.0) 3074 : cluster [DBG] osdmap e587: 8 total, 8 up, 7 in 2024-02-23T00:51:57.563 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:56.260802+0000 mon.a (mon.0) 3075 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-99-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-23T00:51:57.564 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:56.307469+0000 mon.a (mon.0) 3076 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:51:57.564 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:56.932420+0000 mon.a (mon.0) 3077 : audit [INF] from='client.? 172.21.15.47:0/3623459494' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:51:57.564 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:56.962246+0000 osd.4 (osd.4) 243 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:51:57.564 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:56.962419+0000 osd.4 (osd.4) 244 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:51:57.630 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:51:57.659 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:58.213 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.0 dump_blocked_ops 2024-02-23T00:51:58.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:57.257592+0000 mon.a (mon.0) 3078 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:51:58.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:57.261775+0000 mon.a (mon.0) 3079 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-99-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-23T00:51:58.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:57.261864+0000 mon.a (mon.0) 3080 : audit [INF] from='client.? 172.21.15.47:0/3623459494' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:51:58.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:57.261918+0000 mon.a (mon.0) 3081 : cluster [DBG] osdmap e588: 8 total, 8 up, 7 in 2024-02-23T00:51:58.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:57.292967+0000 mon.a (mon.0) 3082 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-99"}]: dispatch 2024-02-23T00:51:58.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:57.349823+0000 mgr.y (mgr.14517) 528 : cluster [DBG] pgmap v886: 192 pgs: 192 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:58.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:57.257592+0000 mon.a (mon.0) 3078 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:51:58.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:57.261775+0000 mon.a (mon.0) 3079 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-99-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-23T00:51:58.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:57.261864+0000 mon.a (mon.0) 3080 : audit [INF] from='client.? 172.21.15.47:0/3623459494' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:51:58.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:57.261918+0000 mon.a (mon.0) 3081 : cluster [DBG] osdmap e588: 8 total, 8 up, 7 in 2024-02-23T00:51:58.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:57.292967+0000 mon.a (mon.0) 3082 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-99"}]: dispatch 2024-02-23T00:51:58.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:57.349823+0000 mgr.y (mgr.14517) 528 : cluster [DBG] pgmap v886: 192 pgs: 192 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:58.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:57.257592+0000 mon.a (mon.0) 3078 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:51:58.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:57.261775+0000 mon.a (mon.0) 3079 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-99-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-23T00:51:58.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:57.261864+0000 mon.a (mon.0) 3080 : audit [INF] from='client.? 172.21.15.47:0/3623459494' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:51:58.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:57.261918+0000 mon.a (mon.0) 3081 : cluster [DBG] osdmap e588: 8 total, 8 up, 7 in 2024-02-23T00:51:58.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:57.292967+0000 mon.a (mon.0) 3082 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-99"}]: dispatch 2024-02-23T00:51:58.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:57.349823+0000 mgr.y (mgr.14517) 528 : cluster [DBG] pgmap v886: 192 pgs: 192 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:51:59.051 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:51:59.054 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:59.055 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:51:59.156 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:51:59.184 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:59.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:58.266793+0000 mon.a (mon.0) 3083 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-99"}]': finished 2024-02-23T00:51:59.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:58.266869+0000 mon.a (mon.0) 3084 : cluster [DBG] osdmap e589: 8 total, 8 up, 7 in 2024-02-23T00:51:59.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:51:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:58.267819+0000 mon.a (mon.0) 3085 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-99", "tierpool": "test-rados-api-smithi047-82465-99-cache"}]: dispatch 2024-02-23T00:51:59.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:58.266793+0000 mon.a (mon.0) 3083 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-99"}]': finished 2024-02-23T00:51:59.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:58.266869+0000 mon.a (mon.0) 3084 : cluster [DBG] osdmap e589: 8 total, 8 up, 7 in 2024-02-23T00:51:59.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:51:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:58.267819+0000 mon.a (mon.0) 3085 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-99", "tierpool": "test-rados-api-smithi047-82465-99-cache"}]: dispatch 2024-02-23T00:51:59.308 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:51:59.312 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:59.313 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:51:59.413 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:51:59.442 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:59.516 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:51:59.520 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:59.521 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:51:59.621 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:51:59.649 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:58.266793+0000 mon.a (mon.0) 3083 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-99"}]': finished 2024-02-23T00:51:59.649 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:58.266869+0000 mon.a (mon.0) 3084 : cluster [DBG] osdmap e589: 8 total, 8 up, 7 in 2024-02-23T00:51:59.649 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:51:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:58.267819+0000 mon.a (mon.0) 3085 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-99", "tierpool": "test-rados-api-smithi047-82465-99-cache"}]: dispatch 2024-02-23T00:51:59.651 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:51:59.756 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:51:59.758 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:51:59.759 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:51:59.860 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:51:59.888 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:00.086 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:52:00.088 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:00.089 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:52:00.189 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:52:00.219 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:00.344 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:52:00.348 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:00.349 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:52:00.360 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:51:59.268124+0000 mon.a (mon.0) 3086 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-99", "tierpool": "test-rados-api-smithi047-82465-99-cache"}]': finished 2024-02-23T00:52:00.360 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:59.268205+0000 mon.a (mon.0) 3087 : cluster [DBG] osdmap e590: 8 total, 8 up, 7 in 2024-02-23T00:52:00.360 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:51:59.350271+0000 mgr.y (mgr.14517) 529 : cluster [DBG] pgmap v889: 192 pgs: 192 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:52:00.360 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:00.083111+0000 mon.a (mon.0) 3088 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:52:00.360 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:51:59.268124+0000 mon.a (mon.0) 3086 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-99", "tierpool": "test-rados-api-smithi047-82465-99-cache"}]': finished 2024-02-23T00:52:00.360 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:59.268205+0000 mon.a (mon.0) 3087 : cluster [DBG] osdmap e590: 8 total, 8 up, 7 in 2024-02-23T00:52:00.360 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:51:59.350271+0000 mgr.y (mgr.14517) 529 : cluster [DBG] pgmap v889: 192 pgs: 192 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:52:00.360 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:00.083111+0000 mon.a (mon.0) 3088 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:52:00.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:51:59.268124+0000 mon.a (mon.0) 3086 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-99", "tierpool": "test-rados-api-smithi047-82465-99-cache"}]': finished 2024-02-23T00:52:00.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:59.268205+0000 mon.a (mon.0) 3087 : cluster [DBG] osdmap e590: 8 total, 8 up, 7 in 2024-02-23T00:52:00.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:51:59.350271+0000 mgr.y (mgr.14517) 529 : cluster [DBG] pgmap v889: 192 pgs: 192 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:52:00.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:00.083111+0000 mon.a (mon.0) 3088 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:52:00.449 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:52:00.480 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:00.764 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:52:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:52:00] "GET /metrics HTTP/1.1" 200 192811 "" "Prometheus/2.33.4" 2024-02-23T00:52:01.083 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset noscrub 2024-02-23T00:52:01.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:00.291051+0000 mon.a (mon.0) 3089 : cluster [DBG] osdmap e591: 8 total, 8 up, 7 in 2024-02-23T00:52:01.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:00.291051+0000 mon.a (mon.0) 3089 : cluster [DBG] osdmap e591: 8 total, 8 up, 7 in 2024-02-23T00:52:01.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:00.291051+0000 mon.a (mon.0) 3089 : cluster [DBG] osdmap e591: 8 total, 8 up, 7 in 2024-02-23T00:52:01.904 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:52:01.904 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:52:01.905 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:52:01.905 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:52:01.905 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:52:01.905 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:52:01.905 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:52:01.905 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:52:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.647+0000 7fa33db1c700 1 -- 172.21.15.47:0/3098830413 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3380f8420 msgr2=0x7fa338104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.647+0000 7fa33db1c700 1 --2- 172.21.15.47:0/3098830413 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3380f8420 0x7fa338104a20 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 rx=0x7fa330009be0 tx=0x7fa330004d10).stop 2024-02-23T00:52:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.647+0000 7fa33db1c700 1 -- 172.21.15.47:0/3098830413 shutdown_connections 2024-02-23T00:52:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.647+0000 7fa33db1c700 1 --2- 172.21.15.47:0/3098830413 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3380f8420 0x7fa338104a20 unknown :-1 s=CLOSED pgs=2589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.647+0000 7fa33db1c700 1 --2- 172.21.15.47:0/3098830413 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa3380fdbb0 0x7fa3380fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.647+0000 7fa33db1c700 1 --2- 172.21.15.47:0/3098830413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3380fe510 0x7fa3380f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.647+0000 7fa33db1c700 1 -- 172.21.15.47:0/3098830413 >> 172.21.15.47:0/3098830413 conn(0x7fa3380f3800 msgr2=0x7fa3380f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:52:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.647+0000 7fa33db1c700 1 -- 172.21.15.47:0/3098830413 shutdown_connections 2024-02-23T00:52:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.647+0000 7fa33db1c700 1 -- 172.21.15.47:0/3098830413 wait complete. 2024-02-23T00:52:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.648+0000 7fa33db1c700 1 librados: starting msgr at 2024-02-23T00:52:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.648+0000 7fa33db1c700 1 librados: starting objecter 2024-02-23T00:52:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.648+0000 7fa33db1c700 1 Processor -- start 2024-02-23T00:52:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.648+0000 7fa33db1c700 1 -- start start 2024-02-23T00:52:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.648+0000 7fa33db1c700 1 librados: setting wanted keys 2024-02-23T00:52:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.648+0000 7fa33db1c700 1 librados: calling monclient init 2024-02-23T00:52:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.648+0000 7fa33db1c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3380f8420 0x7fa33810ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.648+0000 7fa33db1c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3380fdbb0 0x7fa338110470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:01.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.648+0000 7fa33db1c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa3380fe510 0x7fa33810a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:01.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.648+0000 7fa33db1c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33810a610 con 0x7fa3380f8420 2024-02-23T00:52:01.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.648+0000 7fa33db1c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33810a780 con 0x7fa3380fdbb0 2024-02-23T00:52:01.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.648+0000 7fa33db1c700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa33810a940 con 0x7fa3380fe510 2024-02-23T00:52:01.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.648+0000 7fa3377fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3380f8420 0x7fa33810ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:01.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.648+0000 7fa337fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa3380fe510 0x7fa33810a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:01.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.648+0000 7fa3377fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3380f8420 0x7fa33810ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36376/0 (socket says 172.21.15.47:36376) 2024-02-23T00:52:01.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.648+0000 7fa3377fe700 1 -- 172.21.15.47:0/2810096282 learned_addr learned my addr 172.21.15.47:0/2810096282 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:52:01.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.648+0000 7fa336ffd700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3380fdbb0 0x7fa338110470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:01.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.649+0000 7fa337fff700 1 -- 172.21.15.47:0/2810096282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3380fdbb0 msgr2=0x7fa338110470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:01.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.649+0000 7fa337fff700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3380fdbb0 0x7fa338110470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:01.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.649+0000 7fa337fff700 1 -- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3380f8420 msgr2=0x7fa33810ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:01.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.649+0000 7fa337fff700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3380f8420 0x7fa33810ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:01.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.649+0000 7fa337fff700 1 -- 172.21.15.47:0/2810096282 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3300097c0 con 0x7fa3380fe510 2024-02-23T00:52:01.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.649+0000 7fa3377fe700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3380f8420 0x7fa33810ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:52:01.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.649+0000 7fa337fff700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa3380fe510 0x7fa33810a120 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 rx=0x7fa330009790 tx=0x7fa330004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:01.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.649+0000 7fa334ff9700 10 client.?.objecter ms_handle_connect 0x7fa3380fe510 2024-02-23T00:52:01.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.649+0000 7fa334ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:52:01.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.649+0000 7fa334ff9700 1 -- 172.21.15.47:0/2810096282 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33000e040 con 0x7fa3380fe510 2024-02-23T00:52:01.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.649+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa33810abc0 con 0x7fa3380fe510 2024-02-23T00:52:01.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.649+0000 7fa334ff9700 1 -- 172.21.15.47:0/2810096282 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fa330018040 con 0x7fa3380fe510 2024-02-23T00:52:01.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.649+0000 7fa334ff9700 1 -- 172.21.15.47:0/2810096282 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa330013ce0 con 0x7fa3380fe510 2024-02-23T00:52:01.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.649+0000 7fa33db1c700 10 client.41753.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:52:01.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.649+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa338116db0 con 0x7fa3380fe510 2024-02-23T00:52:01.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.649+0000 7fa33db1c700 1 librados: init done 2024-02-23T00:52:01.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.650+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fa33805ee80 con 0x7fa3380fe510 2024-02-23T00:52:01.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.650+0000 7fa334ff9700 1 -- 172.21.15.47:0/2810096282 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fa3300134b0 con 0x7fa3380fe510 2024-02-23T00:52:01.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.651+0000 7fa334ff9700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa320062cc0 0x7fa320065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:01.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.651+0000 7fa334ff9700 1 -- 172.21.15.47:0/2810096282 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(592..592 src has 482..592) v4 ==== 9600+0+0 (secure 0 0 0) 0x7fa33007f2b0 con 0x7fa3380fe510 2024-02-23T00:52:01.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.651+0000 7fa334ff9700 10 client.41753.objecter ms_dispatch 0x7fa338001550 osd_map(592..592 src has 482..592) v4 2024-02-23T00:52:01.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.651+0000 7fa334ff9700 3 client.41753.objecter handle_osd_map got epochs [592,592] > 0 2024-02-23T00:52:01.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.651+0000 7fa334ff9700 3 client.41753.objecter handle_osd_map decoding full epoch 592 2024-02-23T00:52:01.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.651+0000 7fa3377fe700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa320062cc0 0x7fa320065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:01.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.651+0000 7fa334ff9700 20 client.41753.objecter dump_active .. 0 homeless 2024-02-23T00:52:01.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.651+0000 7fa334ff9700 1 -- 172.21.15.47:0/2810096282 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v592) v1 ==== 54+0+16 (secure 0 0 0) 0x7fa330086070 con 0x7fa3380fe510 2024-02-23T00:52:01.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.651+0000 7fa3377fe700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa320062cc0 0x7fa320065180 secure :-1 s=READY pgs=3135 cs=0 l=1 rev1=1 rx=0x7fa32c007d30 tx=0x7fa32c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:01.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.651+0000 7fa334ff9700 10 client.41753.objecter ms_handle_connect 0x7fa320062cc0 2024-02-23T00:52:01.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.651+0000 7fa33db1c700 10 client.41753.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:01.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.651+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_assign -1 1 2024-02-23T00:52:01.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.651+0000 7fa33db1c700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fa338028540 0x7fa3380f97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:01.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.651+0000 7fa33db1c700 20 client.41753.objecter _get_session s=0x7fa338106ed0 osd=0 3 2024-02-23T00:52:01.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.651+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa338106ed0 osd=0 3 2024-02-23T00:52:01.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.651+0000 7fa33db1c700 20 client.41753.objecter _get_session s=0x7fa338106ed0 osd=0 3 2024-02-23T00:52:01.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.651+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_remove -1 1 2024-02-23T00:52:01.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.651+0000 7fa33db1c700 20 client.41753.objecter get_session s=0x7fa338106ed0 osd=0 3 2024-02-23T00:52:01.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.651+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_assign 0 1 2024-02-23T00:52:01.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.651+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa338106ed0 osd=0 4 2024-02-23T00:52:01.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.651+0000 7fa33db1c700 10 client.41753.objecter _send_command 1 2024-02-23T00:52:01.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.651+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa33810b600 con 0x7fa338028540 2024-02-23T00:52:01.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.651+0000 7fa336ffd700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fa338028540 0x7fa3380f97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:01.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.652+0000 7fa334ff9700 10 client.41753.objecter ms_handle_connect 0x7fa338028540 2024-02-23T00:52:01.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.652+0000 7fa336ffd700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fa338028540 0x7fa3380f97f0 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:01.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.652+0000 7fa334ff9700 1 -- 172.21.15.47:0/2810096282 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa33810b600 con 0x7fa338028540 2024-02-23T00:52:01.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.652+0000 7fa334ff9700 10 client.41753.objecter ms_dispatch 0x7fa338001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:52:01.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.652+0000 7fa334ff9700 10 client.41753.objecter _finish_command 1 = system:0 2024-02-23T00:52:01.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.652+0000 7fa334ff9700 20 client.41753.objecter put_session s=0x7fa338106ed0 osd=0 4 2024-02-23T00:52:01.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.652+0000 7fa334ff9700 15 client.41753.objecter _session_command_op_remove 0 1 2024-02-23T00:52:01.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.667+0000 7fa33db1c700 10 client.41753.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:52:01.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.667+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_assign -1 2 2024-02-23T00:52:01.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.667+0000 7fa33db1c700 20 client.41753.objecter _get_session s=0x7fa338106ed0 osd=0 3 2024-02-23T00:52:01.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.667+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa338106ed0 osd=0 3 2024-02-23T00:52:01.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.667+0000 7fa33db1c700 20 client.41753.objecter _get_session s=0x7fa338106ed0 osd=0 3 2024-02-23T00:52:01.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.667+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_remove -1 2 2024-02-23T00:52:01.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.667+0000 7fa33db1c700 20 client.41753.objecter get_session s=0x7fa338106ed0 osd=0 3 2024-02-23T00:52:01.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.667+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_assign 0 2 2024-02-23T00:52:01.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.667+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa338106ed0 osd=0 4 2024-02-23T00:52:01.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.667+0000 7fa33db1c700 10 client.41753.objecter _send_command 2 2024-02-23T00:52:01.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.667+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fa33805a040 con 0x7fa338028540 2024-02-23T00:52:01.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.669+0000 7fa334ff9700 1 -- 172.21.15.47:0/2810096282 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fa33805a040 con 0x7fa338028540 2024-02-23T00:52:01.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.669+0000 7fa334ff9700 10 client.41753.objecter ms_dispatch 0x7fa338001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:52:01.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.669+0000 7fa334ff9700 10 client.41753.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:52:01.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.669+0000 7fa334ff9700 20 client.41753.objecter put_session s=0x7fa338106ed0 osd=0 4 2024-02-23T00:52:01.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.669+0000 7fa334ff9700 15 client.41753.objecter _session_command_op_remove 0 2 2024-02-23T00:52:01.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.669+0000 7fa33db1c700 10 client.41753.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:01.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.669+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_assign -1 3 2024-02-23T00:52:01.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.669+0000 7fa33db1c700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fa338027b90 0x7fa338027fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:01.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.669+0000 7fa33db1c700 20 client.41753.objecter _get_session s=0x7fa3380fae60 osd=1 3 2024-02-23T00:52:01.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.669+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa3380fae60 osd=1 3 2024-02-23T00:52:01.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.669+0000 7fa33db1c700 20 client.41753.objecter _get_session s=0x7fa3380fae60 osd=1 3 2024-02-23T00:52:01.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.669+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_remove -1 3 2024-02-23T00:52:01.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.669+0000 7fa33db1c700 20 client.41753.objecter get_session s=0x7fa3380fae60 osd=1 3 2024-02-23T00:52:01.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.669+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_assign 1 3 2024-02-23T00:52:01.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.669+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa3380fae60 osd=1 4 2024-02-23T00:52:01.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.669+0000 7fa33db1c700 10 client.41753.objecter _send_command 3 2024-02-23T00:52:01.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.669+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa3380fafd0 con 0x7fa338027b90 2024-02-23T00:52:01.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.669+0000 7fa3377fe700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fa338027b90 0x7fa338027fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:01.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.670+0000 7fa334ff9700 10 client.41753.objecter ms_handle_connect 0x7fa338027b90 2024-02-23T00:52:01.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.670+0000 7fa3377fe700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fa338027b90 0x7fa338027fd0 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:01.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.670+0000 7fa334ff9700 1 -- 172.21.15.47:0/2810096282 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa3380fafd0 con 0x7fa338027b90 2024-02-23T00:52:01.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.670+0000 7fa334ff9700 10 client.41753.objecter ms_dispatch 0x7fa338001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:52:01.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.670+0000 7fa334ff9700 10 client.41753.objecter _finish_command 3 = system:0 2024-02-23T00:52:01.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.670+0000 7fa334ff9700 20 client.41753.objecter put_session s=0x7fa3380fae60 osd=1 4 2024-02-23T00:52:01.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.670+0000 7fa334ff9700 15 client.41753.objecter _session_command_op_remove 1 3 2024-02-23T00:52:01.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.686+0000 7fa33db1c700 10 client.41753.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:52:01.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.686+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_assign -1 4 2024-02-23T00:52:01.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.686+0000 7fa33db1c700 20 client.41753.objecter _get_session s=0x7fa3380fae60 osd=1 3 2024-02-23T00:52:01.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.686+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa3380fae60 osd=1 3 2024-02-23T00:52:01.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.686+0000 7fa33db1c700 20 client.41753.objecter _get_session s=0x7fa3380fae60 osd=1 3 2024-02-23T00:52:01.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.686+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_remove -1 4 2024-02-23T00:52:01.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.686+0000 7fa33db1c700 20 client.41753.objecter get_session s=0x7fa3380fae60 osd=1 3 2024-02-23T00:52:01.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.686+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_assign 1 4 2024-02-23T00:52:01.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.686+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa3380fae60 osd=1 4 2024-02-23T00:52:01.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.686+0000 7fa33db1c700 10 client.41753.objecter _send_command 4 2024-02-23T00:52:01.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.686+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fa338027600 con 0x7fa338027b90 2024-02-23T00:52:01.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.688+0000 7fa334ff9700 1 -- 172.21.15.47:0/2810096282 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fa338027600 con 0x7fa338027b90 2024-02-23T00:52:01.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.688+0000 7fa334ff9700 10 client.41753.objecter ms_dispatch 0x7fa338001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:52:01.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.688+0000 7fa334ff9700 10 client.41753.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:52:01.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.688+0000 7fa334ff9700 20 client.41753.objecter put_session s=0x7fa3380fae60 osd=1 4 2024-02-23T00:52:01.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.688+0000 7fa334ff9700 15 client.41753.objecter _session_command_op_remove 1 4 2024-02-23T00:52:01.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.688+0000 7fa33db1c700 10 client.41753.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:01.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.688+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_assign -1 5 2024-02-23T00:52:01.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.688+0000 7fa33db1c700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa3380fa9c0 0x7fa33811e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:01.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.688+0000 7fa33db1c700 20 client.41753.objecter _get_session s=0x7fa338027a10 osd=2 3 2024-02-23T00:52:01.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.688+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa338027a10 osd=2 3 2024-02-23T00:52:01.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.688+0000 7fa33db1c700 20 client.41753.objecter _get_session s=0x7fa338027a10 osd=2 3 2024-02-23T00:52:01.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.688+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_remove -1 5 2024-02-23T00:52:01.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.688+0000 7fa33db1c700 20 client.41753.objecter get_session s=0x7fa338027a10 osd=2 3 2024-02-23T00:52:01.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.688+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_assign 2 5 2024-02-23T00:52:01.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.688+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa338027a10 osd=2 4 2024-02-23T00:52:01.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.688+0000 7fa33db1c700 10 client.41753.objecter _send_command 5 2024-02-23T00:52:01.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.688+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa338120eb0 con 0x7fa3380fa9c0 2024-02-23T00:52:01.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.688+0000 7fa336ffd700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa3380fa9c0 0x7fa33811e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:01.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.689+0000 7fa334ff9700 10 client.41753.objecter ms_handle_connect 0x7fa3380fa9c0 2024-02-23T00:52:01.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.689+0000 7fa336ffd700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa3380fa9c0 0x7fa33811e0f0 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:01.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.689+0000 7fa334ff9700 1 -- 172.21.15.47:0/2810096282 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa338120eb0 con 0x7fa3380fa9c0 2024-02-23T00:52:01.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.689+0000 7fa334ff9700 10 client.41753.objecter ms_dispatch 0x7fa338001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:52:01.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.689+0000 7fa334ff9700 10 client.41753.objecter _finish_command 5 = system:0 2024-02-23T00:52:01.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.689+0000 7fa334ff9700 20 client.41753.objecter put_session s=0x7fa338027a10 osd=2 4 2024-02-23T00:52:01.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.689+0000 7fa334ff9700 15 client.41753.objecter _session_command_op_remove 2 5 2024-02-23T00:52:01.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.704+0000 7fa33db1c700 10 client.41753.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:52:01.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.704+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_assign -1 6 2024-02-23T00:52:01.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.704+0000 7fa33db1c700 20 client.41753.objecter _get_session s=0x7fa338027a10 osd=2 3 2024-02-23T00:52:01.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.704+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa338027a10 osd=2 3 2024-02-23T00:52:01.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.704+0000 7fa33db1c700 20 client.41753.objecter _get_session s=0x7fa338027a10 osd=2 3 2024-02-23T00:52:01.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.704+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_remove -1 6 2024-02-23T00:52:01.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.704+0000 7fa33db1c700 20 client.41753.objecter get_session s=0x7fa338027a10 osd=2 3 2024-02-23T00:52:01.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.704+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_assign 2 6 2024-02-23T00:52:01.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.704+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa338027a10 osd=2 4 2024-02-23T00:52:01.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.704+0000 7fa33db1c700 10 client.41753.objecter _send_command 6 2024-02-23T00:52:01.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.704+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fa3380f9f80 con 0x7fa3380fa9c0 2024-02-23T00:52:01.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.706+0000 7fa334ff9700 1 -- 172.21.15.47:0/2810096282 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fa3380f9f80 con 0x7fa3380fa9c0 2024-02-23T00:52:01.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.706+0000 7fa334ff9700 10 client.41753.objecter ms_dispatch 0x7fa338001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:52:01.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.706+0000 7fa334ff9700 10 client.41753.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:52:01.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.706+0000 7fa334ff9700 20 client.41753.objecter put_session s=0x7fa338027a10 osd=2 4 2024-02-23T00:52:01.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.706+0000 7fa334ff9700 15 client.41753.objecter _session_command_op_remove 2 6 2024-02-23T00:52:01.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.706+0000 7fa33db1c700 10 client.41753.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:01.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.706+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_assign -1 7 2024-02-23T00:52:01.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.706+0000 7fa33db1c700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fa33811e5e0 0x7fa33811ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:01.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.706+0000 7fa33db1c700 20 client.41753.objecter _get_session s=0x7fa33811b730 osd=3 3 2024-02-23T00:52:01.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.706+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa33811b730 osd=3 3 2024-02-23T00:52:01.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.706+0000 7fa33db1c700 20 client.41753.objecter _get_session s=0x7fa33811b730 osd=3 3 2024-02-23T00:52:01.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.706+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_remove -1 7 2024-02-23T00:52:01.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.706+0000 7fa33db1c700 20 client.41753.objecter get_session s=0x7fa33811b730 osd=3 3 2024-02-23T00:52:01.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.706+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_assign 3 7 2024-02-23T00:52:01.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.706+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa33811b730 osd=3 4 2024-02-23T00:52:01.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.706+0000 7fa33db1c700 10 client.41753.objecter _send_command 7 2024-02-23T00:52:01.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.706+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa33811bf50 con 0x7fa33811e5e0 2024-02-23T00:52:01.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.706+0000 7fa337fff700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fa33811e5e0 0x7fa33811ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:01.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.706+0000 7fa334ff9700 10 client.41753.objecter ms_handle_connect 0x7fa33811e5e0 2024-02-23T00:52:01.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.706+0000 7fa337fff700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fa33811e5e0 0x7fa33811ea20 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:01.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.707+0000 7fa334ff9700 1 -- 172.21.15.47:0/2810096282 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa33811bf50 con 0x7fa33811e5e0 2024-02-23T00:52:01.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.707+0000 7fa334ff9700 10 client.41753.objecter ms_dispatch 0x7fa338001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:52:01.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.707+0000 7fa334ff9700 10 client.41753.objecter _finish_command 7 = system:0 2024-02-23T00:52:01.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.707+0000 7fa334ff9700 20 client.41753.objecter put_session s=0x7fa33811b730 osd=3 4 2024-02-23T00:52:01.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.707+0000 7fa334ff9700 15 client.41753.objecter _session_command_op_remove 3 7 2024-02-23T00:52:01.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.721+0000 7fa33db1c700 10 client.41753.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:52:01.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.721+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_assign -1 8 2024-02-23T00:52:01.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.721+0000 7fa33db1c700 20 client.41753.objecter _get_session s=0x7fa33811b730 osd=3 3 2024-02-23T00:52:01.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.721+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa33811b730 osd=3 3 2024-02-23T00:52:01.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.721+0000 7fa33db1c700 20 client.41753.objecter _get_session s=0x7fa33811b730 osd=3 3 2024-02-23T00:52:01.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.721+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_remove -1 8 2024-02-23T00:52:01.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.721+0000 7fa33db1c700 20 client.41753.objecter get_session s=0x7fa33811b730 osd=3 3 2024-02-23T00:52:01.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.721+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_assign 3 8 2024-02-23T00:52:01.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.721+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa33811b730 osd=3 4 2024-02-23T00:52:01.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.721+0000 7fa33db1c700 10 client.41753.objecter _send_command 8 2024-02-23T00:52:01.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.721+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fa3381208f0 con 0x7fa33811e5e0 2024-02-23T00:52:01.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.723+0000 7fa334ff9700 1 -- 172.21.15.47:0/2810096282 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fa3381208f0 con 0x7fa33811e5e0 2024-02-23T00:52:01.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.723+0000 7fa334ff9700 10 client.41753.objecter ms_dispatch 0x7fa338001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:52:01.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.723+0000 7fa334ff9700 10 client.41753.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:52:01.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.723+0000 7fa334ff9700 20 client.41753.objecter put_session s=0x7fa33811b730 osd=3 4 2024-02-23T00:52:01.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.723+0000 7fa334ff9700 15 client.41753.objecter _session_command_op_remove 3 8 2024-02-23T00:52:01.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.723+0000 7fa33db1c700 10 client.41753.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:01.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.723+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_assign -1 9 2024-02-23T00:52:01.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.723+0000 7fa33db1c700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fa3381230c0 0x7fa3381254b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:01.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.723+0000 7fa33db1c700 20 client.41753.objecter _get_session s=0x7fa338120120 osd=4 3 2024-02-23T00:52:01.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.723+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa338120120 osd=4 3 2024-02-23T00:52:01.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.723+0000 7fa33db1c700 20 client.41753.objecter _get_session s=0x7fa338120120 osd=4 3 2024-02-23T00:52:01.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.723+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_remove -1 9 2024-02-23T00:52:01.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.723+0000 7fa33db1c700 20 client.41753.objecter get_session s=0x7fa338120120 osd=4 3 2024-02-23T00:52:01.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.723+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_assign 4 9 2024-02-23T00:52:01.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.723+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa338120120 osd=4 4 2024-02-23T00:52:01.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.723+0000 7fa33db1c700 10 client.41753.objecter _send_command 9 2024-02-23T00:52:01.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.723+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa338128100 con 0x7fa3381230c0 2024-02-23T00:52:01.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.723+0000 7fa3377fe700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fa3381230c0 0x7fa3381254b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:01.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.724+0000 7fa3377fe700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fa3381230c0 0x7fa3381254b0 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:01.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.724+0000 7fa334ff9700 10 client.41753.objecter ms_handle_connect 0x7fa3381230c0 2024-02-23T00:52:01.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.724+0000 7fa334ff9700 1 -- 172.21.15.47:0/2810096282 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa338128100 con 0x7fa3381230c0 2024-02-23T00:52:01.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.724+0000 7fa334ff9700 10 client.41753.objecter ms_dispatch 0x7fa338001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:52:01.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.724+0000 7fa334ff9700 10 client.41753.objecter _finish_command 9 = system:0 2024-02-23T00:52:01.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.724+0000 7fa334ff9700 20 client.41753.objecter put_session s=0x7fa338120120 osd=4 4 2024-02-23T00:52:01.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.724+0000 7fa334ff9700 15 client.41753.objecter _session_command_op_remove 4 9 2024-02-23T00:52:01.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.739+0000 7fa33db1c700 10 client.41753.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:52:01.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.739+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_assign -1 10 2024-02-23T00:52:01.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.739+0000 7fa33db1c700 20 client.41753.objecter _get_session s=0x7fa338120120 osd=4 3 2024-02-23T00:52:01.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.739+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa338120120 osd=4 3 2024-02-23T00:52:01.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.739+0000 7fa33db1c700 20 client.41753.objecter _get_session s=0x7fa338120120 osd=4 3 2024-02-23T00:52:01.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.739+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_remove -1 10 2024-02-23T00:52:01.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.739+0000 7fa33db1c700 20 client.41753.objecter get_session s=0x7fa338120120 osd=4 3 2024-02-23T00:52:01.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.739+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_assign 4 10 2024-02-23T00:52:01.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.739+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa338120120 osd=4 4 2024-02-23T00:52:01.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.739+0000 7fa33db1c700 10 client.41753.objecter _send_command 10 2024-02-23T00:52:01.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.739+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fa3381259a0 con 0x7fa3381230c0 2024-02-23T00:52:01.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.741+0000 7fa334ff9700 1 -- 172.21.15.47:0/2810096282 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fa3381259a0 con 0x7fa3381230c0 2024-02-23T00:52:01.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.741+0000 7fa334ff9700 10 client.41753.objecter ms_dispatch 0x7fa338001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:52:01.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.741+0000 7fa334ff9700 10 client.41753.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:52:01.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.741+0000 7fa334ff9700 20 client.41753.objecter put_session s=0x7fa338120120 osd=4 4 2024-02-23T00:52:01.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.741+0000 7fa334ff9700 15 client.41753.objecter _session_command_op_remove 4 10 2024-02-23T00:52:01.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.742+0000 7fa33db1c700 10 client.41753.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:01.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.742+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_assign -1 11 2024-02-23T00:52:01.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.742+0000 7fa33db1c700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fa338126070 0x7fa3381284e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:01.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.742+0000 7fa33db1c700 20 client.41753.objecter _get_session s=0x7fa3381259a0 osd=5 3 2024-02-23T00:52:01.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.742+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa3381259a0 osd=5 3 2024-02-23T00:52:01.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.742+0000 7fa33db1c700 20 client.41753.objecter _get_session s=0x7fa3381259a0 osd=5 3 2024-02-23T00:52:01.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.742+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_remove -1 11 2024-02-23T00:52:01.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.742+0000 7fa33db1c700 20 client.41753.objecter get_session s=0x7fa3381259a0 osd=5 3 2024-02-23T00:52:01.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.742+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_assign 5 11 2024-02-23T00:52:01.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.742+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa3381259a0 osd=5 4 2024-02-23T00:52:01.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.742+0000 7fa33db1c700 10 client.41753.objecter _send_command 11 2024-02-23T00:52:01.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.742+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa33812b320 con 0x7fa338126070 2024-02-23T00:52:01.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.742+0000 7fa336ffd700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fa338126070 0x7fa3381284e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:01.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.742+0000 7fa334ff9700 10 client.41753.objecter ms_handle_connect 0x7fa338126070 2024-02-23T00:52:01.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.742+0000 7fa336ffd700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fa338126070 0x7fa3381284e0 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:01.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.743+0000 7fa334ff9700 1 -- 172.21.15.47:0/2810096282 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa33812b320 con 0x7fa338126070 2024-02-23T00:52:01.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.743+0000 7fa334ff9700 10 client.41753.objecter ms_dispatch 0x7fa338001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:52:01.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.743+0000 7fa334ff9700 10 client.41753.objecter _finish_command 11 = system:0 2024-02-23T00:52:01.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.743+0000 7fa334ff9700 20 client.41753.objecter put_session s=0x7fa3381259a0 osd=5 4 2024-02-23T00:52:01.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.743+0000 7fa334ff9700 15 client.41753.objecter _session_command_op_remove 5 11 2024-02-23T00:52:01.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.758+0000 7fa33db1c700 10 client.41753.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:52:01.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.758+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_assign -1 12 2024-02-23T00:52:01.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.758+0000 7fa33db1c700 20 client.41753.objecter _get_session s=0x7fa3381259a0 osd=5 3 2024-02-23T00:52:01.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.758+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa3381259a0 osd=5 3 2024-02-23T00:52:01.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.758+0000 7fa33db1c700 20 client.41753.objecter _get_session s=0x7fa3381259a0 osd=5 3 2024-02-23T00:52:01.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.758+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_remove -1 12 2024-02-23T00:52:01.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.758+0000 7fa33db1c700 20 client.41753.objecter get_session s=0x7fa3381259a0 osd=5 3 2024-02-23T00:52:01.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.758+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_assign 5 12 2024-02-23T00:52:01.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.758+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa3381259a0 osd=5 4 2024-02-23T00:52:01.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.758+0000 7fa33db1c700 10 client.41753.objecter _send_command 12 2024-02-23T00:52:01.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.758+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fa33812a0f0 con 0x7fa338126070 2024-02-23T00:52:01.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.760+0000 7fa334ff9700 1 -- 172.21.15.47:0/2810096282 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fa33812a0f0 con 0x7fa338126070 2024-02-23T00:52:01.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.760+0000 7fa334ff9700 10 client.41753.objecter ms_dispatch 0x7fa338001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:52:01.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.760+0000 7fa334ff9700 10 client.41753.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:52:01.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.760+0000 7fa334ff9700 20 client.41753.objecter put_session s=0x7fa3381259a0 osd=5 4 2024-02-23T00:52:01.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.760+0000 7fa334ff9700 15 client.41753.objecter _session_command_op_remove 5 12 2024-02-23T00:52:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.760+0000 7fa33db1c700 10 client.41753.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.760+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_assign -1 13 2024-02-23T00:52:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.760+0000 7fa33db1c700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fa33812a820 0x7fa33812aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.760+0000 7fa33db1c700 20 client.41753.objecter _get_session s=0x7fa33812a0f0 osd=6 3 2024-02-23T00:52:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.760+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa33812a0f0 osd=6 3 2024-02-23T00:52:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.760+0000 7fa33db1c700 20 client.41753.objecter _get_session s=0x7fa33812a0f0 osd=6 3 2024-02-23T00:52:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.760+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_remove -1 13 2024-02-23T00:52:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.760+0000 7fa33db1c700 20 client.41753.objecter get_session s=0x7fa33812a0f0 osd=6 3 2024-02-23T00:52:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.760+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_assign 6 13 2024-02-23T00:52:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.760+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa33812a0f0 osd=6 4 2024-02-23T00:52:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.760+0000 7fa33db1c700 10 client.41753.objecter _send_command 13 2024-02-23T00:52:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.760+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa338129d60 con 0x7fa33812a820 2024-02-23T00:52:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.760+0000 7fa337fff700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fa33812a820 0x7fa33812aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.760+0000 7fa334ff9700 10 client.41753.objecter ms_handle_connect 0x7fa33812a820 2024-02-23T00:52:01.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.760+0000 7fa337fff700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fa33812a820 0x7fa33812aca0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:01.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.761+0000 7fa334ff9700 1 -- 172.21.15.47:0/2810096282 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa338129d60 con 0x7fa33812a820 2024-02-23T00:52:01.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.761+0000 7fa334ff9700 10 client.41753.objecter ms_dispatch 0x7fa338001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:52:01.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.761+0000 7fa334ff9700 10 client.41753.objecter _finish_command 13 = system:0 2024-02-23T00:52:01.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.761+0000 7fa334ff9700 20 client.41753.objecter put_session s=0x7fa33812a0f0 osd=6 4 2024-02-23T00:52:01.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.761+0000 7fa334ff9700 15 client.41753.objecter _session_command_op_remove 6 13 2024-02-23T00:52:01.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.781+0000 7fa33db1c700 10 client.41753.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:52:01.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.781+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_assign -1 14 2024-02-23T00:52:01.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.781+0000 7fa33db1c700 20 client.41753.objecter _get_session s=0x7fa33812a0f0 osd=6 3 2024-02-23T00:52:01.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.781+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa33812a0f0 osd=6 3 2024-02-23T00:52:01.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.781+0000 7fa33db1c700 20 client.41753.objecter _get_session s=0x7fa33812a0f0 osd=6 3 2024-02-23T00:52:01.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.781+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_remove -1 14 2024-02-23T00:52:01.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.781+0000 7fa33db1c700 20 client.41753.objecter get_session s=0x7fa33812a0f0 osd=6 3 2024-02-23T00:52:01.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.781+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_assign 6 14 2024-02-23T00:52:01.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.781+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa33812a0f0 osd=6 4 2024-02-23T00:52:01.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.781+0000 7fa33db1c700 10 client.41753.objecter _send_command 14 2024-02-23T00:52:01.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.781+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fa338129d60 con 0x7fa33812a820 2024-02-23T00:52:01.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.783+0000 7fa334ff9700 1 -- 172.21.15.47:0/2810096282 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fa338129d60 con 0x7fa33812a820 2024-02-23T00:52:01.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.783+0000 7fa334ff9700 10 client.41753.objecter ms_dispatch 0x7fa338001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:52:01.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.783+0000 7fa334ff9700 10 client.41753.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:52:01.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.783+0000 7fa334ff9700 20 client.41753.objecter put_session s=0x7fa33812a0f0 osd=6 4 2024-02-23T00:52:01.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.783+0000 7fa334ff9700 15 client.41753.objecter _session_command_op_remove 6 14 2024-02-23T00:52:01.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.783+0000 7fa33db1c700 10 client.41753.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:01.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.783+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_assign -1 15 2024-02-23T00:52:01.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.783+0000 7fa33db1c700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fa338129040 0x7fa3381294a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:01.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.783+0000 7fa33db1c700 20 client.41753.objecter _get_session s=0x7fa338129d60 osd=7 3 2024-02-23T00:52:01.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.783+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa338129d60 osd=7 3 2024-02-23T00:52:01.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.783+0000 7fa33db1c700 20 client.41753.objecter _get_session s=0x7fa338129d60 osd=7 3 2024-02-23T00:52:01.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.783+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_remove -1 15 2024-02-23T00:52:01.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.783+0000 7fa33db1c700 20 client.41753.objecter get_session s=0x7fa338129d60 osd=7 3 2024-02-23T00:52:01.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.783+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_assign 7 15 2024-02-23T00:52:01.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.783+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa338129d60 osd=7 4 2024-02-23T00:52:01.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.783+0000 7fa33db1c700 10 client.41753.objecter _send_command 15 2024-02-23T00:52:01.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.783+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa338129990 con 0x7fa338129040 2024-02-23T00:52:01.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.783+0000 7fa3377fe700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fa338129040 0x7fa3381294a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:01.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.784+0000 7fa334ff9700 10 client.41753.objecter ms_handle_connect 0x7fa338129040 2024-02-23T00:52:01.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.784+0000 7fa3377fe700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fa338129040 0x7fa3381294a0 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:01.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.785+0000 7fa334ff9700 1 -- 172.21.15.47:0/2810096282 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa338129990 con 0x7fa338129040 2024-02-23T00:52:01.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.785+0000 7fa334ff9700 10 client.41753.objecter ms_dispatch 0x7fa338001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:52:01.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.785+0000 7fa334ff9700 10 client.41753.objecter _finish_command 15 = system:0 2024-02-23T00:52:01.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.785+0000 7fa334ff9700 20 client.41753.objecter put_session s=0x7fa338129d60 osd=7 4 2024-02-23T00:52:01.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.785+0000 7fa334ff9700 15 client.41753.objecter _session_command_op_remove 7 15 2024-02-23T00:52:01.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.800+0000 7fa33db1c700 10 client.41753.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:52:01.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.800+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_assign -1 16 2024-02-23T00:52:01.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.800+0000 7fa33db1c700 20 client.41753.objecter _get_session s=0x7fa338129d60 osd=7 3 2024-02-23T00:52:01.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.800+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa338129d60 osd=7 3 2024-02-23T00:52:01.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.800+0000 7fa33db1c700 20 client.41753.objecter _get_session s=0x7fa338129d60 osd=7 3 2024-02-23T00:52:01.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.800+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_remove -1 16 2024-02-23T00:52:01.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.800+0000 7fa33db1c700 20 client.41753.objecter get_session s=0x7fa338129d60 osd=7 3 2024-02-23T00:52:01.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.800+0000 7fa33db1c700 15 client.41753.objecter _session_command_op_assign 7 16 2024-02-23T00:52:01.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.800+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa338129d60 osd=7 4 2024-02-23T00:52:01.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.800+0000 7fa33db1c700 10 client.41753.objecter _send_command 16 2024-02-23T00:52:01.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.800+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fa338129b60 con 0x7fa338129040 2024-02-23T00:52:01.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.802+0000 7fa334ff9700 1 -- 172.21.15.47:0/2810096282 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fa338129b60 con 0x7fa338129040 2024-02-23T00:52:01.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.802+0000 7fa334ff9700 10 client.41753.objecter ms_dispatch 0x7fa338001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:52:01.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.802+0000 7fa334ff9700 10 client.41753.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:52:01.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.802+0000 7fa334ff9700 20 client.41753.objecter put_session s=0x7fa338129d60 osd=7 4 2024-02-23T00:52:01.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.802+0000 7fa334ff9700 15 client.41753.objecter _session_command_op_remove 7 16 2024-02-23T00:52:01.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 10 librados: watch_flush enter 2024-02-23T00:52:01.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 10 librados: watch_flush exit 2024-02-23T00:52:01.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 10 client.41753.objecter close_session for osd.0 2024-02-23T00:52:01.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fa338028540 msgr2=0x7fa3380f97f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:01.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fa338028540 0x7fa3380f97f0 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:01.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa338106ed0 osd=0 1 2024-02-23T00:52:01.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 10 client.41753.objecter close_session for osd.1 2024-02-23T00:52:01.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fa338027b90 msgr2=0x7fa338027fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:01.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fa338027b90 0x7fa338027fd0 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:01.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa3380fae60 osd=1 1 2024-02-23T00:52:01.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 10 client.41753.objecter close_session for osd.2 2024-02-23T00:52:01.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa3380fa9c0 msgr2=0x7fa33811e0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:01.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa3380fa9c0 0x7fa33811e0f0 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:01.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa338027a10 osd=2 1 2024-02-23T00:52:01.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 10 client.41753.objecter close_session for osd.3 2024-02-23T00:52:01.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fa33811e5e0 msgr2=0x7fa33811ea20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:01.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fa33811e5e0 0x7fa33811ea20 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:01.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa33811b730 osd=3 1 2024-02-23T00:52:01.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 10 client.41753.objecter close_session for osd.4 2024-02-23T00:52:01.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fa3381230c0 msgr2=0x7fa3381254b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:01.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fa3381230c0 0x7fa3381254b0 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:01.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa338120120 osd=4 1 2024-02-23T00:52:01.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 10 client.41753.objecter close_session for osd.5 2024-02-23T00:52:01.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fa338126070 msgr2=0x7fa3381284e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:01.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fa338126070 0x7fa3381284e0 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:01.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa3381259a0 osd=5 1 2024-02-23T00:52:01.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 10 client.41753.objecter close_session for osd.6 2024-02-23T00:52:01.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fa33812a820 msgr2=0x7fa33812aca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:01.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fa33812a820 0x7fa33812aca0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:01.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa33812a0f0 osd=6 1 2024-02-23T00:52:01.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 10 client.41753.objecter close_session for osd.7 2024-02-23T00:52:01.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fa338129040 msgr2=0x7fa3381294a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:01.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fa338129040 0x7fa3381294a0 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:01.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 20 client.41753.objecter put_session s=0x7fa338129d60 osd=7 1 2024-02-23T00:52:01.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 20 client.41753.objecter shutdown clearing up homeless session... 2024-02-23T00:52:01.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 10 client.41753.objecter successfully canceled tick 2024-02-23T00:52:01.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa337fff700 1 -- 172.21.15.47:0/2810096282 reap_dead start 2024-02-23T00:52:01.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa337fff700 1 -- 172.21.15.47:0/2810096282 reap_dead start 2024-02-23T00:52:01.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa337fff700 1 -- 172.21.15.47:0/2810096282 reap_dead start 2024-02-23T00:52:01.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa337fff700 1 -- 172.21.15.47:0/2810096282 reap_dead start 2024-02-23T00:52:01.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa337fff700 1 -- 172.21.15.47:0/2810096282 reap_dead start 2024-02-23T00:52:01.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa320062cc0 msgr2=0x7fa320065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:01.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa320062cc0 0x7fa320065180 secure :-1 s=READY pgs=3135 cs=0 l=1 rev1=1 rx=0x7fa32c007d30 tx=0x7fa32c00b040).stop 2024-02-23T00:52:01.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa3380fe510 msgr2=0x7fa33810a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:01.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.803+0000 7fa33db1c700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa3380fe510 0x7fa33810a120 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 rx=0x7fa330009790 tx=0x7fa330004d10).stop 2024-02-23T00:52:01.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.804+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 shutdown_connections 2024-02-23T00:52:01.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.804+0000 7fa33db1c700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa3380fe510 0x7fa33810a120 unknown :-1 s=CLOSED pgs=2745 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:01.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.804+0000 7fa33db1c700 1 --2- 172.21.15.47:0/2810096282 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa320062cc0 0x7fa320065180 unknown :-1 s=CLOSED pgs=3135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:01.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.804+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 >> 172.21.15.47:0/2810096282 conn(0x7fa3380f3800 msgr2=0x7fa338103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:52:01.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.804+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 shutdown_connections 2024-02-23T00:52:01.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.804+0000 7fa33db1c700 1 -- 172.21.15.47:0/2810096282 wait complete. 2024-02-23T00:52:01.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:01.804+0000 7fa33db1c700 1 librados: shutdown 2024-02-23T00:52:01.944 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:52:01.944 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:52:01.944 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:52:01.944 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:52:01.944 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:52:01.944 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:52:01.944 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:52:01.944 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:52:02.070 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:52:02.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.265+0000 7f4b7f0e3700 1 -- 172.21.15.47:0/160826960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b780f6a90 msgr2=0x7f4b78104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:02.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.265+0000 7f4b7f0e3700 1 --2- 172.21.15.47:0/160826960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b780f6a90 0x7f4b78104a20 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 rx=0x7f4b70009be0 tx=0x7f4b70004d10).stop 2024-02-23T00:52:02.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.265+0000 7f4b7f0e3700 1 -- 172.21.15.47:0/160826960 shutdown_connections 2024-02-23T00:52:02.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.265+0000 7f4b7f0e3700 1 --2- 172.21.15.47:0/160826960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b780f6a90 0x7f4b78104a20 unknown :-1 s=CLOSED pgs=2590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:02.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.265+0000 7f4b7f0e3700 1 --2- 172.21.15.47:0/160826960 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b780fdbd0 0x7f4b780fdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:02.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.265+0000 7f4b7f0e3700 1 --2- 172.21.15.47:0/160826960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b780fe530 0x7f4b780f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:02.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.265+0000 7f4b7f0e3700 1 -- 172.21.15.47:0/160826960 >> 172.21.15.47:0/160826960 conn(0x7f4b7806c370 msgr2=0x7f4b7806c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:52:02.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.265+0000 7f4b7f0e3700 1 -- 172.21.15.47:0/160826960 shutdown_connections 2024-02-23T00:52:02.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.265+0000 7f4b7f0e3700 1 -- 172.21.15.47:0/160826960 wait complete. 2024-02-23T00:52:02.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.266+0000 7f4b7f0e3700 1 librados: starting msgr at 2024-02-23T00:52:02.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.266+0000 7f4b7f0e3700 1 librados: starting objecter 2024-02-23T00:52:02.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.266+0000 7f4b7f0e3700 1 Processor -- start 2024-02-23T00:52:02.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.266+0000 7f4b7f0e3700 1 -- start start 2024-02-23T00:52:02.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.266+0000 7f4b7f0e3700 1 librados: setting wanted keys 2024-02-23T00:52:02.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.266+0000 7f4b7f0e3700 1 librados: calling monclient init 2024-02-23T00:52:02.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.266+0000 7f4b7f0e3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b780f6a90 0x7f4b7810de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:02.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.266+0000 7f4b7f0e3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b780fdbd0 0x7f4b7810e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:02.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.266+0000 7f4b7f0e3700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b780fe530 0x7f4b78107fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:02.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.266+0000 7f4b7f0e3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b781084c0 con 0x7f4b780fdbd0 2024-02-23T00:52:02.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.266+0000 7f4b7f0e3700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b78108630 con 0x7f4b780f6a90 2024-02-23T00:52:02.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.266+0000 7f4b7f0e3700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b781087f0 con 0x7f4b780fe530 2024-02-23T00:52:02.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.266+0000 7f4b77fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b780fdbd0 0x7f4b7810e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:02.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.266+0000 7f4b7d680700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b780fe530 0x7f4b78107fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:02.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.266+0000 7f4b77fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b780fdbd0 0x7f4b7810e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36394/0 (socket says 172.21.15.47:36394) 2024-02-23T00:52:02.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.266+0000 7f4b77fff700 1 -- 172.21.15.47:0/1550174213 learned_addr learned my addr 172.21.15.47:0/1550174213 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:52:02.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.267+0000 7f4b7ce7f700 1 --2- 172.21.15.47:0/1550174213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b780f6a90 0x7f4b7810de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:02.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.267+0000 7f4b7d680700 1 -- 172.21.15.47:0/1550174213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b780f6a90 msgr2=0x7f4b7810de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:02.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.267+0000 7f4b7d680700 1 --2- 172.21.15.47:0/1550174213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b780f6a90 0x7f4b7810de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:02.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.267+0000 7f4b7d680700 1 -- 172.21.15.47:0/1550174213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b780fdbd0 msgr2=0x7f4b7810e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:02.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.267+0000 7f4b7d680700 1 --2- 172.21.15.47:0/1550174213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b780fdbd0 0x7f4b7810e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:02.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.267+0000 7f4b7d680700 1 -- 172.21.15.47:0/1550174213 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b700097c0 con 0x7f4b780fe530 2024-02-23T00:52:02.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.267+0000 7f4b7d680700 1 --2- 172.21.15.47:0/1550174213 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b780fe530 0x7f4b78107fd0 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 rx=0x7f4b70009bb0 tx=0x7f4b70004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:02.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.267+0000 7f4b75ffb700 10 client.?.objecter ms_handle_connect 0x7f4b780fe530 2024-02-23T00:52:02.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.267+0000 7f4b75ffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:52:02.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.267+0000 7f4b75ffb700 1 -- 172.21.15.47:0/1550174213 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7000e040 con 0x7f4b780fe530 2024-02-23T00:52:02.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.267+0000 7f4b75ffb700 1 -- 172.21.15.47:0/1550174213 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f4b70018040 con 0x7f4b780fe530 2024-02-23T00:52:02.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.267+0000 7f4b7f0e3700 1 -- 172.21.15.47:0/1550174213 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b78108a70 con 0x7f4b780fe530 2024-02-23T00:52:02.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.267+0000 7f4b75ffb700 1 -- 172.21.15.47:0/1550174213 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7001c3f0 con 0x7f4b780fe530 2024-02-23T00:52:02.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.267+0000 7f4b7f0e3700 10 client.41759.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:52:02.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.267+0000 7f4b7f0e3700 1 -- 172.21.15.47:0/1550174213 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b78114c60 con 0x7f4b780fe530 2024-02-23T00:52:02.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.267+0000 7f4b7f0e3700 1 librados: init done 2024-02-23T00:52:02.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.268+0000 7f4b7f0e3700 1 -- 172.21.15.47:0/1550174213 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b7805f970 con 0x7f4b780fe530 2024-02-23T00:52:02.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.269+0000 7f4b75ffb700 1 -- 172.21.15.47:0/1550174213 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f4b70007d10 con 0x7f4b780fe530 2024-02-23T00:52:02.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.269+0000 7f4b75ffb700 1 --2- 172.21.15.47:0/1550174213 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4b60062cc0 0x7f4b60065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:02.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.269+0000 7f4b75ffb700 1 -- 172.21.15.47:0/1550174213 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(592..592 src has 482..592) v4 ==== 9600+0+0 (secure 0 0 0) 0x7f4b7007e460 con 0x7f4b780fe530 2024-02-23T00:52:02.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.269+0000 7f4b75ffb700 10 client.41759.objecter ms_dispatch 0x7f4b78001550 osd_map(592..592 src has 482..592) v4 2024-02-23T00:52:02.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.269+0000 7f4b75ffb700 3 client.41759.objecter handle_osd_map got epochs [592,592] > 0 2024-02-23T00:52:02.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.269+0000 7f4b75ffb700 3 client.41759.objecter handle_osd_map decoding full epoch 592 2024-02-23T00:52:02.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.269+0000 7f4b7ce7f700 1 --2- 172.21.15.47:0/1550174213 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4b60062cc0 0x7f4b60065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:02.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.269+0000 7f4b75ffb700 20 client.41759.objecter dump_active .. 0 homeless 2024-02-23T00:52:02.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.269+0000 7f4b7ce7f700 1 --2- 172.21.15.47:0/1550174213 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4b60062cc0 0x7f4b60065180 secure :-1 s=READY pgs=3136 cs=0 l=1 rev1=1 rx=0x7f4b68009f20 tx=0x7f4b6800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:02.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.269+0000 7f4b75ffb700 10 client.41759.objecter ms_handle_connect 0x7f4b60062cc0 2024-02-23T00:52:02.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.273+0000 7f4b75ffb700 1 -- 172.21.15.47:0/1550174213 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f4b7004d9b0 con 0x7f4b780fe530 2024-02-23T00:52:02.289 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:52:02.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.453+0000 7f4b7f0e3700 1 -- 172.21.15.47:0/1550174213 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4b78106e90 con 0x7f4b60062cc0 2024-02-23T00:52:02.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.465+0000 7f4b75ffb700 1 -- 172.21.15.47:0/1550174213 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+402622 (secure 0 0 0) 0x7f4b78106e90 con 0x7f4b60062cc0 2024-02-23T00:52:02.472 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:52:02.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.467+0000 7f4b7f0e3700 10 librados: watch_flush enter 2024-02-23T00:52:02.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.467+0000 7f4b7f0e3700 10 librados: watch_flush exit 2024-02-23T00:52:02.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.467+0000 7f4b7f0e3700 20 client.41759.objecter shutdown clearing up homeless session... 2024-02-23T00:52:02.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.467+0000 7f4b7f0e3700 10 client.41759.objecter successfully canceled tick 2024-02-23T00:52:02.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.468+0000 7f4b7f0e3700 1 -- 172.21.15.47:0/1550174213 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4b60062cc0 msgr2=0x7f4b60065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:02.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.468+0000 7f4b7f0e3700 1 --2- 172.21.15.47:0/1550174213 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4b60062cc0 0x7f4b60065180 secure :-1 s=READY pgs=3136 cs=0 l=1 rev1=1 rx=0x7f4b68009f20 tx=0x7f4b6800b040).stop 2024-02-23T00:52:02.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.468+0000 7f4b7f0e3700 1 -- 172.21.15.47:0/1550174213 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b780fe530 msgr2=0x7f4b78107fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.468+0000 7f4b7f0e3700 1 --2- 172.21.15.47:0/1550174213 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b780fe530 0x7f4b78107fd0 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 rx=0x7f4b70009bb0 tx=0x7f4b70004d10).stop 2024-02-23T00:52:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.468+0000 7f4b7f0e3700 1 -- 172.21.15.47:0/1550174213 shutdown_connections 2024-02-23T00:52:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.468+0000 7f4b7f0e3700 1 --2- 172.21.15.47:0/1550174213 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4b60062cc0 0x7f4b60065180 unknown :-1 s=CLOSED pgs=3136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.468+0000 7f4b7f0e3700 1 --2- 172.21.15.47:0/1550174213 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b780fe530 0x7f4b78107fd0 unknown :-1 s=CLOSED pgs=2746 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.468+0000 7f4b7f0e3700 1 --2- 172.21.15.47:0/1550174213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b780f6a90 0x7f4b7810de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.468+0000 7f4b7f0e3700 1 --2- 172.21.15.47:0/1550174213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b780fdbd0 0x7f4b7810e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.468+0000 7f4b7f0e3700 1 -- 172.21.15.47:0/1550174213 >> 172.21.15.47:0/1550174213 conn(0x7f4b7806c370 msgr2=0x7f4b780f5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:52:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.468+0000 7f4b7f0e3700 1 -- 172.21.15.47:0/1550174213 shutdown_connections 2024-02-23T00:52:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.468+0000 7f4b7f0e3700 1 -- 172.21.15.47:0/1550174213 wait complete. 2024-02-23T00:52:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:02.468+0000 7f4b7f0e3700 1 librados: shutdown 2024-02-23T00:52:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-02-23T00:52:02.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:01.288575+0000 mon.a (mon.0) 3090 : cluster [DBG] osdmap e592: 8 total, 8 up, 7 in 2024-02-23T00:52:02.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:01.292868+0000 mon.a (mon.0) 3091 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82465-99"}]: dispatch 2024-02-23T00:52:02.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:01.350784+0000 mgr.y (mgr.14517) 530 : cluster [DBG] pgmap v892: 152 pgs: 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:52:02.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:01.651030+0000 mon.c (mon.2) 331 : audit [DBG] from='client.? 172.21.15.47:0/2810096282' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:52:02.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:02.291511+0000 mon.a (mon.0) 3092 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82465-99"}]': finished 2024-02-23T00:52:02.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:02.291572+0000 mon.a (mon.0) 3093 : cluster [DBG] osdmap e593: 8 total, 8 up, 7 in 2024-02-23T00:52:02.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:01.288575+0000 mon.a (mon.0) 3090 : cluster [DBG] osdmap e592: 8 total, 8 up, 7 in 2024-02-23T00:52:02.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:01.292868+0000 mon.a (mon.0) 3091 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82465-99"}]: dispatch 2024-02-23T00:52:02.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:01.350784+0000 mgr.y (mgr.14517) 530 : cluster [DBG] pgmap v892: 152 pgs: 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:52:02.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:01.651030+0000 mon.c (mon.2) 331 : audit [DBG] from='client.? 172.21.15.47:0/2810096282' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:52:02.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:02.291511+0000 mon.a (mon.0) 3092 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82465-99"}]': finished 2024-02-23T00:52:02.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:02.291572+0000 mon.a (mon.0) 3093 : cluster [DBG] osdmap e593: 8 total, 8 up, 7 in 2024-02-23T00:52:02.694 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:52:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:52:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:52:02.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:01.288575+0000 mon.a (mon.0) 3090 : cluster [DBG] osdmap e592: 8 total, 8 up, 7 in 2024-02-23T00:52:02.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:01.292868+0000 mon.a (mon.0) 3091 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82465-99"}]: dispatch 2024-02-23T00:52:02.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:01.350784+0000 mgr.y (mgr.14517) 530 : cluster [DBG] pgmap v892: 152 pgs: 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:52:02.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:01.651030+0000 mon.c (mon.2) 331 : audit [DBG] from='client.? 172.21.15.47:0/2810096282' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:52:02.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:02.291511+0000 mon.a (mon.0) 3092 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82465-99"}]': finished 2024-02-23T00:52:02.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:02.291572+0000 mon.a (mon.0) 3093 : cluster [DBG] osdmap e593: 8 total, 8 up, 7 in 2024-02-23T00:52:03.194 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:52:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:52:03.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:02.300821+0000 mon.a (mon.0) 3094 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82465-99"}]: dispatch 2024-02-23T00:52:03.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:02.454209+0000 mgr.y (mgr.14517) 531 : audit [DBG] from='client.41759 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:52:03.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:03.302301+0000 mon.a (mon.0) 3095 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82465-99"}]': finished 2024-02-23T00:52:03.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:03.302369+0000 mon.a (mon.0) 3096 : cluster [DBG] osdmap e594: 8 total, 8 up, 7 in 2024-02-23T00:52:03.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:02.300821+0000 mon.a (mon.0) 3094 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82465-99"}]: dispatch 2024-02-23T00:52:03.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:02.454209+0000 mgr.y (mgr.14517) 531 : audit [DBG] from='client.41759 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:52:03.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:03.302301+0000 mon.a (mon.0) 3095 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82465-99"}]': finished 2024-02-23T00:52:03.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:03.302369+0000 mon.a (mon.0) 3096 : cluster [DBG] osdmap e594: 8 total, 8 up, 7 in 2024-02-23T00:52:03.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:02.300821+0000 mon.a (mon.0) 3094 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82465-99"}]: dispatch 2024-02-23T00:52:03.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:02.454209+0000 mgr.y (mgr.14517) 531 : audit [DBG] from='client.41759 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:52:03.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:03.302301+0000 mon.a (mon.0) 3095 : audit [INF] from='client.? 172.21.15.47:0/1824592256' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82465-99"}]': finished 2024-02-23T00:52:03.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:03.302369+0000 mon.a (mon.0) 3096 : cluster [DBG] osdmap e594: 8 total, 8 up, 7 in 2024-02-23T00:52:04.686 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:02.783201+0000 mgr.y (mgr.14517) 532 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:52:04.686 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:03.351141+0000 mgr.y (mgr.14517) 533 : cluster [DBG] pgmap v895: 152 pgs: 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:52:04.686 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:02.783201+0000 mgr.y (mgr.14517) 532 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:52:04.686 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:03.351141+0000 mgr.y (mgr.14517) 533 : cluster [DBG] pgmap v895: 152 pgs: 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:52:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:02.783201+0000 mgr.y (mgr.14517) 532 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:52:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:03.351141+0000 mgr.y (mgr.14517) 533 : cluster [DBG] pgmap v895: 152 pgs: 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:52:05.165 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:52:05.169 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:05.170 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:52:05.270 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:52:05.283 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:52:05.283 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:52:05.302 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:05.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:04.312814+0000 mon.a (mon.0) 3097 : cluster [DBG] osdmap e595: 8 total, 8 up, 7 in 2024-02-23T00:52:05.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:04.317138+0000 mon.a (mon.0) 3098 : audit [INF] from='client.? 172.21.15.47:0/956675132' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi047-82465-94"}]: dispatch 2024-02-23T00:52:05.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:05.253973+0000 mon.a (mon.0) 3099 : audit [INF] from='client.? 172.21.15.47:0/956675132' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi047-82465-94"}]': finished 2024-02-23T00:52:05.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:05.254019+0000 mon.a (mon.0) 3100 : cluster [DBG] osdmap e596: 8 total, 8 up, 7 in 2024-02-23T00:52:05.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:05.255986+0000 mon.a (mon.0) 3101 : audit [INF] from='client.? 172.21.15.47:0/956675132' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi047-82465-94"}]: dispatch 2024-02-23T00:52:05.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:04.312814+0000 mon.a (mon.0) 3097 : cluster [DBG] osdmap e595: 8 total, 8 up, 7 in 2024-02-23T00:52:05.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:04.317138+0000 mon.a (mon.0) 3098 : audit [INF] from='client.? 172.21.15.47:0/956675132' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi047-82465-94"}]: dispatch 2024-02-23T00:52:05.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:05.253973+0000 mon.a (mon.0) 3099 : audit [INF] from='client.? 172.21.15.47:0/956675132' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi047-82465-94"}]': finished 2024-02-23T00:52:05.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:05.254019+0000 mon.a (mon.0) 3100 : cluster [DBG] osdmap e596: 8 total, 8 up, 7 in 2024-02-23T00:52:05.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:05.255986+0000 mon.a (mon.0) 3101 : audit [INF] from='client.? 172.21.15.47:0/956675132' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi047-82465-94"}]: dispatch 2024-02-23T00:52:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:04.312814+0000 mon.a (mon.0) 3097 : cluster [DBG] osdmap e595: 8 total, 8 up, 7 in 2024-02-23T00:52:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:04.317138+0000 mon.a (mon.0) 3098 : audit [INF] from='client.? 172.21.15.47:0/956675132' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi047-82465-94"}]: dispatch 2024-02-23T00:52:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:05.253973+0000 mon.a (mon.0) 3099 : audit [INF] from='client.? 172.21.15.47:0/956675132' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi047-82465-94"}]': finished 2024-02-23T00:52:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:05.254019+0000 mon.a (mon.0) 3100 : cluster [DBG] osdmap e596: 8 total, 8 up, 7 in 2024-02-23T00:52:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:05.255986+0000 mon.a (mon.0) 3101 : audit [INF] from='client.? 172.21.15.47:0/956675132' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi047-82465-94"}]: dispatch 2024-02-23T00:52:05.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.873+0000 7fc52c1ef700 1 -- 172.21.15.47:0/2849949870 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc5240f7330 msgr2=0x7fc524103a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:05.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.873+0000 7fc52c1ef700 1 --2- 172.21.15.47:0/2849949870 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc5240f7330 0x7fc524103a00 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 rx=0x7fc514004870 tx=0x7fc514009cb0).stop 2024-02-23T00:52:05.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.873+0000 7fc52c1ef700 1 -- 172.21.15.47:0/2849949870 shutdown_connections 2024-02-23T00:52:05.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.873+0000 7fc52c1ef700 1 --2- 172.21.15.47:0/2849949870 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc5240f7330 0x7fc524103a00 unknown :-1 s=CLOSED pgs=2747 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:05.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.873+0000 7fc52c1ef700 1 --2- 172.21.15.47:0/2849949870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5240f89a0 0x7fc5240f8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:05.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.873+0000 7fc52c1ef700 1 --2- 172.21.15.47:0/2849949870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5240f9270 0x7fc5240f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:05.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.873+0000 7fc52c1ef700 1 -- 172.21.15.47:0/2849949870 >> 172.21.15.47:0/2849949870 conn(0x7fc5240f26f0 msgr2=0x7fc5240f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:52:05.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.873+0000 7fc52c1ef700 1 -- 172.21.15.47:0/2849949870 shutdown_connections 2024-02-23T00:52:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.873+0000 7fc52c1ef700 1 -- 172.21.15.47:0/2849949870 wait complete. 2024-02-23T00:52:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.874+0000 7fc52c1ef700 1 librados: starting msgr at 2024-02-23T00:52:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.874+0000 7fc52c1ef700 1 librados: starting objecter 2024-02-23T00:52:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.874+0000 7fc52c1ef700 1 Processor -- start 2024-02-23T00:52:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.874+0000 7fc52c1ef700 1 -- start start 2024-02-23T00:52:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.874+0000 7fc52c1ef700 1 librados: setting wanted keys 2024-02-23T00:52:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.874+0000 7fc52c1ef700 1 librados: calling monclient init 2024-02-23T00:52:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.874+0000 7fc52c1ef700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc5240f7330 0x7fc524108d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.874+0000 7fc52c1ef700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5240f89a0 0x7fc524109280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.874+0000 7fc52c1ef700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5240f9270 0x7fc524109890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.874+0000 7fc52c1ef700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc524113ff0 con 0x7fc5240f9270 2024-02-23T00:52:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.874+0000 7fc52c1ef700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc524114160 con 0x7fc5240f89a0 2024-02-23T00:52:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.874+0000 7fc52c1ef700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc524114320 con 0x7fc5240f7330 2024-02-23T00:52:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.874+0000 7fc529f8b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc5240f7330 0x7fc524108d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.874+0000 7fc52a78c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5240f9270 0x7fc524109890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.874+0000 7fc529f8b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc5240f7330 0x7fc524108d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:53206/0 (socket says 172.21.15.47:53206) 2024-02-23T00:52:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.874+0000 7fc52a78c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5240f9270 0x7fc524109890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36408/0 (socket says 172.21.15.47:36408) 2024-02-23T00:52:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.874+0000 7fc529f8b700 1 -- 172.21.15.47:0/1538028832 learned_addr learned my addr 172.21.15.47:0/1538028832 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:52:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.875+0000 7fc52a78c700 1 -- 172.21.15.47:0/1538028832 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc5240f7330 msgr2=0x7fc524108d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.875+0000 7fc52a78c700 1 --2- 172.21.15.47:0/1538028832 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc5240f7330 0x7fc524108d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.875+0000 7fc52a78c700 1 -- 172.21.15.47:0/1538028832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5240f89a0 msgr2=0x7fc524109280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.875+0000 7fc52a78c700 1 --2- 172.21.15.47:0/1538028832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5240f89a0 0x7fc524109280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:52:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.875+0000 7fc52a78c700 1 -- 172.21.15.47:0/1538028832 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc51c0097c0 con 0x7fc5240f9270 2024-02-23T00:52:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.875+0000 7fc529f8b700 1 --2- 172.21.15.47:0/1538028832 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc5240f7330 0x7fc524108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:52:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.875+0000 7fc52a78c700 1 --2- 172.21.15.47:0/1538028832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5240f9270 0x7fc524109890 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 rx=0x7fc514004710 tx=0x7fc514009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.875+0000 7fc522ffd700 10 client.?.objecter ms_handle_connect 0x7fc5240f9270 2024-02-23T00:52:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.875+0000 7fc522ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:52:05.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.875+0000 7fc522ffd700 1 -- 172.21.15.47:0/1538028832 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51400e030 con 0x7fc5240f9270 2024-02-23T00:52:05.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.875+0000 7fc522ffd700 1 -- 172.21.15.47:0/1538028832 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fc51401d420 con 0x7fc5240f9270 2024-02-23T00:52:05.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.875+0000 7fc522ffd700 1 -- 172.21.15.47:0/1538028832 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc514018070 con 0x7fc5240f9270 2024-02-23T00:52:05.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.875+0000 7fc52c1ef700 1 -- 172.21.15.47:0/1538028832 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc514005040 con 0x7fc5240f9270 2024-02-23T00:52:05.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.875+0000 7fc52c1ef700 10 client.31446.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:52:05.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.875+0000 7fc52c1ef700 1 -- 172.21.15.47:0/1538028832 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc524114820 con 0x7fc5240f9270 2024-02-23T00:52:05.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.875+0000 7fc52c1ef700 1 librados: init done 2024-02-23T00:52:05.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.876+0000 7fc52c1ef700 1 -- 172.21.15.47:0/1538028832 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc52405ee80 con 0x7fc5240f9270 2024-02-23T00:52:05.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.877+0000 7fc522ffd700 1 -- 172.21.15.47:0/1538028832 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fc51401d850 con 0x7fc5240f9270 2024-02-23T00:52:05.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.877+0000 7fc522ffd700 1 --2- 172.21.15.47:0/1538028832 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc50c062cc0 0x7fc50c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:05.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.877+0000 7fc522ffd700 1 -- 172.21.15.47:0/1538028832 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(596..596 src has 482..596) v4 ==== 8488+0+0 (secure 0 0 0) 0x7fc51407e1f0 con 0x7fc5240f9270 2024-02-23T00:52:05.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.877+0000 7fc522ffd700 10 client.31446.objecter ms_dispatch 0x7fc524001550 osd_map(596..596 src has 482..596) v4 2024-02-23T00:52:05.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.877+0000 7fc522ffd700 3 client.31446.objecter handle_osd_map got epochs [596,596] > 0 2024-02-23T00:52:05.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.877+0000 7fc522ffd700 3 client.31446.objecter handle_osd_map decoding full epoch 596 2024-02-23T00:52:05.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.877+0000 7fc529f8b700 1 --2- 172.21.15.47:0/1538028832 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc50c062cc0 0x7fc50c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:05.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.877+0000 7fc522ffd700 20 client.31446.objecter dump_active .. 0 homeless 2024-02-23T00:52:05.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.878+0000 7fc522ffd700 10 client.31446.objecter ms_handle_connect 0x7fc50c062cc0 2024-02-23T00:52:05.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.878+0000 7fc529f8b700 1 --2- 172.21.15.47:0/1538028832 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc50c062cc0 0x7fc50c065180 secure :-1 s=READY pgs=3137 cs=0 l=1 rev1=1 rx=0x7fc51c001940 tx=0x7fc51c004c50).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:05.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:05.882+0000 7fc522ffd700 1 -- 172.21.15.47:0/1538028832 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fc51404db60 con 0x7fc5240f9270 2024-02-23T00:52:06.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:06.065+0000 7fc52c1ef700 1 -- 172.21.15.47:0/1538028832 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7fc524061510 con 0x7fc5240f9270 2024-02-23T00:52:06.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:06.256+0000 7fc522ffd700 1 -- 172.21.15.47:0/1538028832 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v597) v1 ==== 91+0+0 (secure 0 0 0) 0x7fc514051180 con 0x7fc5240f9270 2024-02-23T00:52:06.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:06.258+0000 7fc52c1ef700 10 librados: watch_flush enter 2024-02-23T00:52:06.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:06.258+0000 7fc52c1ef700 10 librados: watch_flush exit 2024-02-23T00:52:06.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:06.258+0000 7fc52c1ef700 20 client.31446.objecter shutdown clearing up homeless session... 2024-02-23T00:52:06.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:06.258+0000 7fc52c1ef700 10 client.31446.objecter successfully canceled tick 2024-02-23T00:52:06.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:06.258+0000 7fc52c1ef700 1 -- 172.21.15.47:0/1538028832 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc50c062cc0 msgr2=0x7fc50c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:06.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:06.258+0000 7fc52c1ef700 1 --2- 172.21.15.47:0/1538028832 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc50c062cc0 0x7fc50c065180 secure :-1 s=READY pgs=3137 cs=0 l=1 rev1=1 rx=0x7fc51c001940 tx=0x7fc51c004c50).stop 2024-02-23T00:52:06.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:06.258+0000 7fc52c1ef700 1 -- 172.21.15.47:0/1538028832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5240f9270 msgr2=0x7fc524109890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:06.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:06.258+0000 7fc52c1ef700 1 --2- 172.21.15.47:0/1538028832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5240f9270 0x7fc524109890 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 rx=0x7fc514004710 tx=0x7fc514009cb0).stop 2024-02-23T00:52:06.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:06.259+0000 7fc52c1ef700 1 -- 172.21.15.47:0/1538028832 shutdown_connections 2024-02-23T00:52:06.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:06.259+0000 7fc52c1ef700 1 --2- 172.21.15.47:0/1538028832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5240f9270 0x7fc524109890 unknown :-1 s=CLOSED pgs=2591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:06.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:06.259+0000 7fc52c1ef700 1 --2- 172.21.15.47:0/1538028832 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc50c062cc0 0x7fc50c065180 unknown :-1 s=CLOSED pgs=3137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:06.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:06.259+0000 7fc52c1ef700 1 --2- 172.21.15.47:0/1538028832 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc5240f7330 0x7fc524108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:06.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:06.259+0000 7fc52c1ef700 1 --2- 172.21.15.47:0/1538028832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5240f89a0 0x7fc524109280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:52:06.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:06.259+0000 7fc52c1ef700 1 -- 172.21.15.47:0/1538028832 >> 172.21.15.47:0/1538028832 conn(0x7fc5240f26f0 msgr2=0x7fc5240f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:52:06.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:06.259+0000 7fc52c1ef700 1 -- 172.21.15.47:0/1538028832 shutdown_connections 2024-02-23T00:52:06.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:06.259+0000 7fc52c1ef700 1 -- 172.21.15.47:0/1538028832 wait complete. 2024-02-23T00:52:06.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:06.259+0000 7fc52c1ef700 1 librados: shutdown 2024-02-23T00:52:06.262 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is unset 2024-02-23T00:52:06.383 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":893,"stamp":"2024-02-23T00:52:02.288500+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5801,"num_objects":226,"num_object_clones":0,"num_object_copies":678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":225,"num_whiteouts":0,"num_read":2295,"num_read_kb":2013,"num_write":914,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":2626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1049,"ondisk_log_size":1049,"up":456,"acting":456,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":526,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":2157192,"kb_used_data":19280,"kb_used_omap":0,"kb_used_meta":2137856,"kb_avail":654058872,"statfs":{"total":671965249536,"available":669756284928,"internally_reserved":0,"allocated":19742720,"data_stored":8495366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2189164544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.024674"},"pg_stats":[{"pgid":"229.7","version":"0'0","reported_seq":34,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.505367+0000","last_change":"2024-02-23T00:51:34.271578+0000","last_active":"2024-02-23T00:52:00.505367+0000","last_peered":"2024-02-23T00:52:00.505367+0000","last_clean":"2024-02-23T00:52:00.505367+0000","last_became_active":"2024-02-23T00:51:34.271103+0000","last_became_peered":"2024-02-23T00:51:34.271103+0000","last_unstale":"2024-02-23T00:52:00.505367+0000","last_undegraded":"2024-02-23T00:52:00.505367+0000","last_fullsized":"2024-02-23T00:52:00.505367+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":564,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_clean_scrub_stamp":"2024-02-23T00:51:33.255429+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"229.6","version":"0'0","reported_seq":36,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.303525+0000","last_change":"2024-02-23T00:51:34.269550+0000","last_active":"2024-02-23T00:52:00.303525+0000","last_peered":"2024-02-23T00:52:00.303525+0000","last_clean":"2024-02-23T00:52:00.303525+0000","last_became_active":"2024-02-23T00:51:34.269208+0000","last_became_peered":"2024-02-23T00:51:34.269208+0000","last_unstale":"2024-02-23T00:52:00.303525+0000","last_undegraded":"2024-02-23T00:52:00.303525+0000","last_fullsized":"2024-02-23T00:52:00.303525+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":564,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_clean_scrub_stamp":"2024-02-23T00:51:33.255429+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"229.5","version":"0'0","reported_seq":35,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.439472+0000","last_change":"2024-02-23T00:51:34.269112+0000","last_active":"2024-02-23T00:52:00.439472+0000","last_peered":"2024-02-23T00:52:00.439472+0000","last_clean":"2024-02-23T00:52:00.439472+0000","last_became_active":"2024-02-23T00:51:34.268826+0000","last_became_peered":"2024-02-23T00:51:34.268826+0000","last_unstale":"2024-02-23T00:52:00.439472+0000","last_undegraded":"2024-02-23T00:52:00.439472+0000","last_fullsized":"2024-02-23T00:52:00.439472+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":564,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_clean_scrub_stamp":"2024-02-23T00:51:33.255429+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"229.4","version":"0'0","reported_seq":34,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.503376+0000","last_change":"2024-02-23T00:51:34.271412+0000","last_active":"2024-02-23T00:52:00.503376+0000","last_peered":"2024-02-23T00:52:00.503376+0000","last_clean":"2024-02-23T00:52:00.503376+0000","last_became_active":"2024-02-23T00:51:34.270881+0000","last_became_peered":"2024-02-23T00:51:34.270881+0000","last_unstale":"2024-02-23T00:52:00.503376+0000","last_undegraded":"2024-02-23T00:52:00.503376+0000","last_fullsized":"2024-02-23T00:52:00.503376+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":564,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_clean_scrub_stamp":"2024-02-23T00:51:33.255429+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"229.3","version":"565'4","reported_seq":39,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.398849+0000","last_change":"2024-02-23T00:51:34.271564+0000","last_active":"2024-02-23T00:52:00.398849+0000","last_peered":"2024-02-23T00:52:00.398849+0000","last_clean":"2024-02-23T00:52:00.398849+0000","last_became_active":"2024-02-23T00:51:34.270817+0000","last_became_peered":"2024-02-23T00:51:34.270817+0000","last_unstale":"2024-02-23T00:52:00.398849+0000","last_undegraded":"2024-02-23T00:52:00.398849+0000","last_fullsized":"2024-02-23T00:52:00.398849+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":564,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_clean_scrub_stamp":"2024-02-23T00:51:33.255429+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"229.2","version":"0'0","reported_seq":35,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.437662+0000","last_change":"2024-02-23T00:51:34.271256+0000","last_active":"2024-02-23T00:52:00.437662+0000","last_peered":"2024-02-23T00:52:00.437662+0000","last_clean":"2024-02-23T00:52:00.437662+0000","last_became_active":"2024-02-23T00:51:34.270941+0000","last_became_peered":"2024-02-23T00:51:34.270941+0000","last_unstale":"2024-02-23T00:52:00.437662+0000","last_undegraded":"2024-02-23T00:52:00.437662+0000","last_fullsized":"2024-02-23T00:52:00.437662+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":564,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_clean_scrub_stamp":"2024-02-23T00:51:33.255429+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"229.1","version":"0'0","reported_seq":36,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.302393+0000","last_change":"2024-02-23T00:51:34.526061+0000","last_active":"2024-02-23T00:52:00.302393+0000","last_peered":"2024-02-23T00:52:00.302393+0000","last_clean":"2024-02-23T00:52:00.302393+0000","last_became_active":"2024-02-23T00:51:34.525742+0000","last_became_peered":"2024-02-23T00:51:34.525742+0000","last_unstale":"2024-02-23T00:52:00.302393+0000","last_undegraded":"2024-02-23T00:52:00.302393+0000","last_fullsized":"2024-02-23T00:52:00.302393+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":564,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_clean_scrub_stamp":"2024-02-23T00:51:33.255429+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"229.0","version":"0'0","reported_seq":36,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.294385+0000","last_change":"2024-02-23T00:51:34.526282+0000","last_active":"2024-02-23T00:52:00.294385+0000","last_peered":"2024-02-23T00:52:00.294385+0000","last_clean":"2024-02-23T00:52:00.294385+0000","last_became_active":"2024-02-23T00:51:34.525872+0000","last_became_peered":"2024-02-23T00:51:34.525872+0000","last_unstale":"2024-02-23T00:52:00.294385+0000","last_undegraded":"2024-02-23T00:52:00.294385+0000","last_fullsized":"2024-02-23T00:52:00.294385+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":564,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:33.255429+0000","last_clean_scrub_stamp":"2024-02-23T00:51:33.255429+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"466'9","reported_seq":720,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.399354+0000","last_change":"2024-02-23T00:49:43.188066+0000","last_active":"2024-02-23T00:52:00.399354+0000","last_peered":"2024-02-23T00:52:00.399354+0000","last_clean":"2024-02-23T00:52:00.399354+0000","last_became_active":"2024-02-23T00:39:18.793414+0000","last_became_peered":"2024-02-23T00:39:18.793414+0000","last_unstale":"2024-02-23T00:52:00.399354+0000","last_undegraded":"2024-02-23T00:52:00.399354+0000","last_fullsized":"2024-02-23T00:52:00.399354+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'9","last_scrub_stamp":"2024-02-23T00:49:43.187989+0000","last_deep_scrub":"466'9","last_deep_scrub_stamp":"2024-02-23T00:49:43.187989+0000","last_clean_scrub_stamp":"2024-02-23T00:49:43.187989+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"470'42","reported_seq":636,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.438702+0000","last_change":"2024-02-23T00:49:47.653272+0000","last_active":"2024-02-23T00:52:00.438702+0000","last_peered":"2024-02-23T00:52:00.438702+0000","last_clean":"2024-02-23T00:52:00.438702+0000","last_became_active":"2024-02-23T00:39:16.859401+0000","last_became_peered":"2024-02-23T00:39:16.859401+0000","last_unstale":"2024-02-23T00:52:00.438702+0000","last_undegraded":"2024-02-23T00:52:00.438702+0000","last_fullsized":"2024-02-23T00:52:00.438702+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"470'42","last_scrub_stamp":"2024-02-23T00:49:47.653176+0000","last_deep_scrub":"470'42","last_deep_scrub_stamp":"2024-02-23T00:49:47.653176+0000","last_clean_scrub_stamp":"2024-02-23T00:49:47.653176+0000","log_size":42,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":238,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.296789+0000","last_change":"2024-02-23T00:51:36.656632+0000","last_active":"2024-02-23T00:52:00.296789+0000","last_peered":"2024-02-23T00:52:00.296789+0000","last_clean":"2024-02-23T00:52:00.296789+0000","last_became_active":"2024-02-23T00:47:28.220876+0000","last_became_peered":"2024-02-23T00:47:28.220876+0000","last_unstale":"2024-02-23T00:52:00.296789+0000","last_undegraded":"2024-02-23T00:52:00.296789+0000","last_fullsized":"2024-02-23T00:52:00.296789+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:36.656571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:04.330426+0000","last_clean_scrub_stamp":"2024-02-23T00:51:36.656571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":574,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.437674+0000","last_change":"2024-02-23T00:50:14.801990+0000","last_active":"2024-02-23T00:52:00.437674+0000","last_peered":"2024-02-23T00:52:00.437674+0000","last_clean":"2024-02-23T00:52:00.437674+0000","last_became_active":"2024-02-23T00:43:32.282545+0000","last_became_peered":"2024-02-23T00:43:32.282545+0000","last_unstale":"2024-02-23T00:52:00.437674+0000","last_undegraded":"2024-02-23T00:52:00.437674+0000","last_fullsized":"2024-02-23T00:52:00.437674+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:14.801915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:50:14.801915+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":503,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.439468+0000","last_change":"2024-02-23T00:50:23.727290+0000","last_active":"2024-02-23T00:52:00.439468+0000","last_peered":"2024-02-23T00:52:00.439468+0000","last_clean":"2024-02-23T00:52:00.439468+0000","last_became_active":"2024-02-23T00:42:58.828846+0000","last_became_peered":"2024-02-23T00:42:58.828846+0000","last_unstale":"2024-02-23T00:52:00.439468+0000","last_undegraded":"2024-02-23T00:52:00.439468+0000","last_fullsized":"2024-02-23T00:52:00.439468+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:23.727173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:23.727173+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"457'21","reported_seq":606,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.303886+0000","last_change":"2024-02-23T00:51:46.661946+0000","last_active":"2024-02-23T00:52:00.303886+0000","last_peered":"2024-02-23T00:52:00.303886+0000","last_clean":"2024-02-23T00:52:00.303886+0000","last_became_active":"2024-02-23T00:39:16.783701+0000","last_became_peered":"2024-02-23T00:39:16.783701+0000","last_unstale":"2024-02-23T00:52:00.303886+0000","last_undegraded":"2024-02-23T00:52:00.303886+0000","last_fullsized":"2024-02-23T00:52:00.303886+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-23T00:51:46.661879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:46.661879+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":551,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.505038+0000","last_change":"2024-02-23T00:49:04.528117+0000","last_active":"2024-02-23T00:52:00.505038+0000","last_peered":"2024-02-23T00:52:00.505038+0000","last_clean":"2024-02-23T00:52:00.505038+0000","last_became_active":"2024-02-23T00:39:14.772493+0000","last_became_peered":"2024-02-23T00:39:14.772493+0000","last_unstale":"2024-02-23T00:52:00.505038+0000","last_undegraded":"2024-02-23T00:52:00.505038+0000","last_fullsized":"2024-02-23T00:52:00.505038+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:04.528006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:04.528006+0000","last_clean_scrub_stamp":"2024-02-23T00:49:04.528006+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":234,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.306002+0000","last_change":"2024-02-23T00:51:44.698156+0000","last_active":"2024-02-23T00:52:00.306002+0000","last_peered":"2024-02-23T00:52:00.306002+0000","last_clean":"2024-02-23T00:52:00.306002+0000","last_became_active":"2024-02-23T00:47:28.231567+0000","last_became_peered":"2024-02-23T00:47:28.231567+0000","last_unstale":"2024-02-23T00:52:00.306002+0000","last_undegraded":"2024-02-23T00:52:00.306002+0000","last_fullsized":"2024-02-23T00:52:00.306002+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:44.698067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:08.335330+0000","last_clean_scrub_stamp":"2024-02-23T00:51:44.698067+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":584,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.302761+0000","last_change":"2024-02-23T00:51:11.526671+0000","last_active":"2024-02-23T00:52:00.302761+0000","last_peered":"2024-02-23T00:52:00.302761+0000","last_clean":"2024-02-23T00:52:00.302761+0000","last_became_active":"2024-02-23T00:47:28.214157+0000","last_became_peered":"2024-02-23T00:47:28.214157+0000","last_unstale":"2024-02-23T00:52:00.302761+0000","last_undegraded":"2024-02-23T00:52:00.302761+0000","last_fullsized":"2024-02-23T00:52:00.302761+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:11.526584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:13.867716+0000","last_clean_scrub_stamp":"2024-02-23T00:51:11.526584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":504,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.305821+0000","last_change":"2024-02-23T00:50:32.622097+0000","last_active":"2024-02-23T00:52:00.305821+0000","last_peered":"2024-02-23T00:52:00.305821+0000","last_clean":"2024-02-23T00:52:00.305821+0000","last_became_active":"2024-02-23T00:42:58.867305+0000","last_became_peered":"2024-02-23T00:42:58.867305+0000","last_unstale":"2024-02-23T00:52:00.305821+0000","last_undegraded":"2024-02-23T00:52:00.305821+0000","last_fullsized":"2024-02-23T00:52:00.305821+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:32.622026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:32.622026+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"457'51","reported_seq":644,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.555520+0000","last_change":"2024-02-23T00:51:13.326646+0000","last_active":"2024-02-23T00:52:00.555520+0000","last_peered":"2024-02-23T00:52:00.555520+0000","last_clean":"2024-02-23T00:52:00.555520+0000","last_became_active":"2024-02-23T00:39:16.857413+0000","last_became_peered":"2024-02-23T00:39:16.857413+0000","last_unstale":"2024-02-23T00:52:00.555520+0000","last_undegraded":"2024-02-23T00:52:00.555520+0000","last_fullsized":"2024-02-23T00:52:00.555520+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'51","last_scrub_stamp":"2024-02-23T00:51:13.326535+0000","last_deep_scrub":"88'35","last_deep_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_clean_scrub_stamp":"2024-02-23T00:51:13.326535+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":68,"num_read_kb":56,"num_write":42,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":235,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.438945+0000","last_change":"2024-02-23T00:50:57.770551+0000","last_active":"2024-02-23T00:52:00.438945+0000","last_peered":"2024-02-23T00:52:00.438945+0000","last_clean":"2024-02-23T00:52:00.438945+0000","last_became_active":"2024-02-23T00:47:28.231891+0000","last_became_peered":"2024-02-23T00:47:28.231891+0000","last_unstale":"2024-02-23T00:52:00.438945+0000","last_undegraded":"2024-02-23T00:52:00.438945+0000","last_fullsized":"2024-02-23T00:52:00.438945+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:57.770450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_clean_scrub_stamp":"2024-02-23T00:50:57.770450+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":236,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.303726+0000","last_change":"2024-02-23T00:51:36.650367+0000","last_active":"2024-02-23T00:52:00.303726+0000","last_peered":"2024-02-23T00:52:00.303726+0000","last_clean":"2024-02-23T00:52:00.303726+0000","last_became_active":"2024-02-23T00:47:28.220578+0000","last_became_peered":"2024-02-23T00:47:28.220578+0000","last_unstale":"2024-02-23T00:52:00.303726+0000","last_undegraded":"2024-02-23T00:52:00.303726+0000","last_fullsized":"2024-02-23T00:52:00.303726+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:36.650303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:03.682745+0000","last_clean_scrub_stamp":"2024-02-23T00:51:36.650303+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":580,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.397329+0000","last_change":"2024-02-23T00:49:42.150972+0000","last_active":"2024-02-23T00:52:00.397329+0000","last_peered":"2024-02-23T00:52:00.397329+0000","last_clean":"2024-02-23T00:52:00.397329+0000","last_became_active":"2024-02-23T00:47:28.215636+0000","last_became_peered":"2024-02-23T00:47:28.215636+0000","last_unstale":"2024-02-23T00:52:00.397329+0000","last_undegraded":"2024-02-23T00:52:00.397329+0000","last_fullsized":"2024-02-23T00:52:00.397329+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:42.150875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:42.150875+0000","last_clean_scrub_stamp":"2024-02-23T00:49:42.150875+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":493,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.502796+0000","last_change":"2024-02-23T00:49:34.515593+0000","last_active":"2024-02-23T00:52:00.502796+0000","last_peered":"2024-02-23T00:52:00.502796+0000","last_clean":"2024-02-23T00:52:00.502796+0000","last_became_active":"2024-02-23T00:42:57.639013+0000","last_became_peered":"2024-02-23T00:42:57.639013+0000","last_unstale":"2024-02-23T00:52:00.502796+0000","last_undegraded":"2024-02-23T00:52:00.502796+0000","last_fullsized":"2024-02-23T00:52:00.502796+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:34.515496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:49:34.515496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"457'31","reported_seq":657,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.307412+0000","last_change":"2024-02-23T00:49:31.480372+0000","last_active":"2024-02-23T00:52:00.307412+0000","last_peered":"2024-02-23T00:52:00.307412+0000","last_clean":"2024-02-23T00:52:00.307412+0000","last_became_active":"2024-02-23T00:47:28.200653+0000","last_became_peered":"2024-02-23T00:47:28.200653+0000","last_unstale":"2024-02-23T00:52:00.307412+0000","last_undegraded":"2024-02-23T00:52:00.307412+0000","last_fullsized":"2024-02-23T00:52:00.307412+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:49:31.480253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:31.480253+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":232,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.397984+0000","last_change":"2024-02-23T00:51:10.213423+0000","last_active":"2024-02-23T00:52:00.397984+0000","last_peered":"2024-02-23T00:52:00.397984+0000","last_clean":"2024-02-23T00:52:00.397984+0000","last_became_active":"2024-02-23T00:47:28.201926+0000","last_became_peered":"2024-02-23T00:47:28.201926+0000","last_unstale":"2024-02-23T00:52:00.397984+0000","last_undegraded":"2024-02-23T00:52:00.397984+0000","last_fullsized":"2024-02-23T00:52:00.397984+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:10.213337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:10.213337+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":552,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.503519+0000","last_change":"2024-02-23T00:51:09.343951+0000","last_active":"2024-02-23T00:52:00.503519+0000","last_peered":"2024-02-23T00:52:00.503519+0000","last_clean":"2024-02-23T00:52:00.503519+0000","last_became_active":"2024-02-23T00:39:18.799875+0000","last_became_peered":"2024-02-23T00:39:18.799875+0000","last_unstale":"2024-02-23T00:52:00.503519+0000","last_undegraded":"2024-02-23T00:52:00.503519+0000","last_fullsized":"2024-02-23T00:52:00.503519+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:09.343884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:09.343884+0000","last_clean_scrub_stamp":"2024-02-23T00:51:09.343884+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":493,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.506585+0000","last_change":"2024-02-23T00:48:49.610457+0000","last_active":"2024-02-23T00:52:00.506585+0000","last_peered":"2024-02-23T00:52:00.506585+0000","last_clean":"2024-02-23T00:52:00.506585+0000","last_became_active":"2024-02-23T00:42:57.675306+0000","last_became_peered":"2024-02-23T00:42:57.675306+0000","last_unstale":"2024-02-23T00:52:00.506585+0000","last_undegraded":"2024-02-23T00:52:00.506585+0000","last_fullsized":"2024-02-23T00:52:00.506585+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:49.610391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:49.610391+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"457'20","reported_seq":589,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.306789+0000","last_change":"2024-02-23T00:51:01.488450+0000","last_active":"2024-02-23T00:52:00.306789+0000","last_peered":"2024-02-23T00:52:00.306789+0000","last_clean":"2024-02-23T00:52:00.306789+0000","last_became_active":"2024-02-23T00:39:16.777530+0000","last_became_peered":"2024-02-23T00:39:16.777530+0000","last_unstale":"2024-02-23T00:52:00.306789+0000","last_undegraded":"2024-02-23T00:52:00.306789+0000","last_fullsized":"2024-02-23T00:52:00.306789+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'20","last_scrub_stamp":"2024-02-23T00:51:01.488371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:01.488371+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":551,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.505078+0000","last_change":"2024-02-23T00:48:41.719867+0000","last_active":"2024-02-23T00:52:00.505078+0000","last_peered":"2024-02-23T00:52:00.505078+0000","last_clean":"2024-02-23T00:52:00.505078+0000","last_became_active":"2024-02-23T00:39:14.780379+0000","last_became_peered":"2024-02-23T00:39:14.780379+0000","last_unstale":"2024-02-23T00:52:00.505078+0000","last_undegraded":"2024-02-23T00:52:00.505078+0000","last_fullsized":"2024-02-23T00:52:00.505078+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:41.719772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.575242+0000","last_clean_scrub_stamp":"2024-02-23T00:48:41.719772+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"376'9","reported_seq":307,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.397532+0000","last_change":"2024-02-23T00:49:40.180695+0000","last_active":"2024-02-23T00:52:00.397532+0000","last_peered":"2024-02-23T00:52:00.397532+0000","last_clean":"2024-02-23T00:52:00.397532+0000","last_became_active":"2024-02-23T00:47:28.231104+0000","last_became_peered":"2024-02-23T00:47:28.231104+0000","last_unstale":"2024-02-23T00:52:00.397532+0000","last_undegraded":"2024-02-23T00:52:00.397532+0000","last_fullsized":"2024-02-23T00:52:00.397532+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"376'9","last_scrub_stamp":"2024-02-23T00:49:40.180616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:40.180616+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":507,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.295977+0000","last_change":"2024-02-23T00:51:12.335819+0000","last_active":"2024-02-23T00:52:00.295977+0000","last_peered":"2024-02-23T00:52:00.295977+0000","last_clean":"2024-02-23T00:52:00.295977+0000","last_became_active":"2024-02-23T00:42:58.765106+0000","last_became_peered":"2024-02-23T00:42:58.765106+0000","last_unstale":"2024-02-23T00:52:00.295977+0000","last_undegraded":"2024-02-23T00:52:00.295977+0000","last_fullsized":"2024-02-23T00:52:00.295977+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.335752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.335752+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.335752+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"457'27","reported_seq":597,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.505594+0000","last_change":"2024-02-23T00:51:48.537804+0000","last_active":"2024-02-23T00:52:00.505594+0000","last_peered":"2024-02-23T00:52:00.505594+0000","last_clean":"2024-02-23T00:52:00.505594+0000","last_became_active":"2024-02-23T00:39:16.789757+0000","last_became_peered":"2024-02-23T00:39:16.789757+0000","last_unstale":"2024-02-23T00:52:00.505594+0000","last_undegraded":"2024-02-23T00:52:00.505594+0000","last_fullsized":"2024-02-23T00:52:00.505594+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-23T00:51:48.537732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:48.537732+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":556,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.503397+0000","last_change":"2024-02-23T00:51:13.375106+0000","last_active":"2024-02-23T00:52:00.503397+0000","last_peered":"2024-02-23T00:52:00.503397+0000","last_clean":"2024-02-23T00:52:00.503397+0000","last_became_active":"2024-02-23T00:39:14.780526+0000","last_became_peered":"2024-02-23T00:39:14.780526+0000","last_unstale":"2024-02-23T00:52:00.503397+0000","last_undegraded":"2024-02-23T00:52:00.503397+0000","last_fullsized":"2024-02-23T00:52:00.503397+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:13.375039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:13.375039+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":718,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.306930+0000","last_change":"2024-02-23T00:49:40.384676+0000","last_active":"2024-02-23T00:52:00.306930+0000","last_peered":"2024-02-23T00:52:00.306930+0000","last_clean":"2024-02-23T00:52:00.306930+0000","last_became_active":"2024-02-23T00:39:18.800440+0000","last_became_peered":"2024-02-23T00:39:18.800440+0000","last_unstale":"2024-02-23T00:52:00.306930+0000","last_undegraded":"2024-02-23T00:52:00.306930+0000","last_fullsized":"2024-02-23T00:52:00.306930+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:49:40.384608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:40.384608+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":505,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.306194+0000","last_change":"2024-02-23T00:51:38.762620+0000","last_active":"2024-02-23T00:52:00.306194+0000","last_peered":"2024-02-23T00:52:00.306194+0000","last_clean":"2024-02-23T00:52:00.306194+0000","last_became_active":"2024-02-23T00:42:58.833924+0000","last_became_peered":"2024-02-23T00:42:58.833924+0000","last_unstale":"2024-02-23T00:52:00.306194+0000","last_undegraded":"2024-02-23T00:52:00.306194+0000","last_fullsized":"2024-02-23T00:52:00.306194+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:38.762554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:38.762554+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"457'31","reported_seq":657,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.306783+0000","last_change":"2024-02-23T00:51:40.699209+0000","last_active":"2024-02-23T00:52:00.306783+0000","last_peered":"2024-02-23T00:52:00.306783+0000","last_clean":"2024-02-23T00:52:00.306783+0000","last_became_active":"2024-02-23T00:47:28.201225+0000","last_became_peered":"2024-02-23T00:47:28.201225+0000","last_unstale":"2024-02-23T00:52:00.306783+0000","last_undegraded":"2024-02-23T00:52:00.306783+0000","last_fullsized":"2024-02-23T00:52:00.306783+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:51:40.699141+0000","last_deep_scrub":"336'23","last_deep_scrub_stamp":"2024-02-23T00:46:48.320085+0000","last_clean_scrub_stamp":"2024-02-23T00:51:40.699141+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":559,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.399482+0000","last_change":"2024-02-23T00:49:37.331334+0000","last_active":"2024-02-23T00:52:00.399482+0000","last_peered":"2024-02-23T00:52:00.399482+0000","last_clean":"2024-02-23T00:52:00.399482+0000","last_became_active":"2024-02-23T00:39:14.784121+0000","last_became_peered":"2024-02-23T00:39:14.784121+0000","last_unstale":"2024-02-23T00:52:00.399482+0000","last_undegraded":"2024-02-23T00:52:00.399482+0000","last_fullsized":"2024-02-23T00:52:00.399482+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:37.331270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:37.331270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":558,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.439527+0000","last_change":"2024-02-23T00:50:19.827889+0000","last_active":"2024-02-23T00:52:00.439527+0000","last_peered":"2024-02-23T00:52:00.439527+0000","last_clean":"2024-02-23T00:52:00.439527+0000","last_became_active":"2024-02-23T00:39:18.800777+0000","last_became_peered":"2024-02-23T00:39:18.800777+0000","last_unstale":"2024-02-23T00:52:00.439527+0000","last_undegraded":"2024-02-23T00:52:00.439527+0000","last_fullsized":"2024-02-23T00:52:00.439527+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:19.827814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_clean_scrub_stamp":"2024-02-23T00:50:19.827814+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":493,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.502398+0000","last_change":"2024-02-23T00:49:33.529224+0000","last_active":"2024-02-23T00:52:00.502398+0000","last_peered":"2024-02-23T00:52:00.502398+0000","last_clean":"2024-02-23T00:52:00.502398+0000","last_became_active":"2024-02-23T00:42:57.653279+0000","last_became_peered":"2024-02-23T00:42:57.653279+0000","last_unstale":"2024-02-23T00:52:00.502398+0000","last_undegraded":"2024-02-23T00:52:00.502398+0000","last_fullsized":"2024-02-23T00:52:00.502398+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:33.529135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:49:33.529135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":723,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.294848+0000","last_change":"2024-02-23T00:51:34.577307+0000","last_active":"2024-02-23T00:52:00.294848+0000","last_peered":"2024-02-23T00:52:00.294848+0000","last_clean":"2024-02-23T00:52:00.294848+0000","last_became_active":"2024-02-23T00:39:18.798536+0000","last_became_peered":"2024-02-23T00:39:18.798536+0000","last_unstale":"2024-02-23T00:52:00.294848+0000","last_undegraded":"2024-02-23T00:52:00.294848+0000","last_fullsized":"2024-02-23T00:52:00.294848+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:51:34.577222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:51:34.577222+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"457'42","reported_seq":624,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.307308+0000","last_change":"2024-02-23T00:50:24.532387+0000","last_active":"2024-02-23T00:52:00.307308+0000","last_peered":"2024-02-23T00:52:00.307308+0000","last_clean":"2024-02-23T00:52:00.307308+0000","last_became_active":"2024-02-23T00:39:16.780130+0000","last_became_peered":"2024-02-23T00:39:16.780130+0000","last_unstale":"2024-02-23T00:52:00.307308+0000","last_undegraded":"2024-02-23T00:52:00.307308+0000","last_fullsized":"2024-02-23T00:52:00.307308+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'42","last_scrub_stamp":"2024-02-23T00:50:24.532271+0000","last_deep_scrub":"423'28","last_deep_scrub_stamp":"2024-02-23T00:48:17.435906+0000","last_clean_scrub_stamp":"2024-02-23T00:50:24.532271+0000","log_size":42,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":46,"num_write":34,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":577,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.554743+0000","last_change":"2024-02-23T00:50:59.199966+0000","last_active":"2024-02-23T00:52:00.554743+0000","last_peered":"2024-02-23T00:52:00.554743+0000","last_clean":"2024-02-23T00:52:00.554743+0000","last_became_active":"2024-02-23T00:47:28.237083+0000","last_became_peered":"2024-02-23T00:47:28.237083+0000","last_unstale":"2024-02-23T00:52:00.554743+0000","last_undegraded":"2024-02-23T00:52:00.554743+0000","last_fullsized":"2024-02-23T00:52:00.554743+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:59.199899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:51.391939+0000","last_clean_scrub_stamp":"2024-02-23T00:50:59.199899+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":572,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.399541+0000","last_change":"2024-02-23T00:49:41.163033+0000","last_active":"2024-02-23T00:52:00.399541+0000","last_peered":"2024-02-23T00:52:00.399541+0000","last_clean":"2024-02-23T00:52:00.399541+0000","last_became_active":"2024-02-23T00:47:30.187386+0000","last_became_peered":"2024-02-23T00:47:30.187386+0000","last_unstale":"2024-02-23T00:52:00.399541+0000","last_undegraded":"2024-02-23T00:52:00.399541+0000","last_fullsized":"2024-02-23T00:52:00.399541+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:41.162895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:46.686764+0000","last_clean_scrub_stamp":"2024-02-23T00:49:41.162895+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":512,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.439731+0000","last_change":"2024-02-23T00:51:03.875340+0000","last_active":"2024-02-23T00:52:00.439731+0000","last_peered":"2024-02-23T00:52:00.439731+0000","last_clean":"2024-02-23T00:52:00.439731+0000","last_became_active":"2024-02-23T00:47:28.212307+0000","last_became_peered":"2024-02-23T00:47:28.212307+0000","last_unstale":"2024-02-23T00:52:00.439731+0000","last_undegraded":"2024-02-23T00:52:00.439731+0000","last_fullsized":"2024-02-23T00:52:00.439731+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:03.875273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:03.875273+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":559,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.307509+0000","last_change":"2024-02-23T00:50:25.575319+0000","last_active":"2024-02-23T00:52:00.307509+0000","last_peered":"2024-02-23T00:52:00.307509+0000","last_clean":"2024-02-23T00:52:00.307509+0000","last_became_active":"2024-02-23T00:39:18.798829+0000","last_became_peered":"2024-02-23T00:39:18.798829+0000","last_unstale":"2024-02-23T00:52:00.307509+0000","last_undegraded":"2024-02-23T00:52:00.307509+0000","last_fullsized":"2024-02-23T00:52:00.307509+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:25.575238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:10.493340+0000","last_clean_scrub_stamp":"2024-02-23T00:50:25.575238+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"457'52","reported_seq":654,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.294760+0000","last_change":"2024-02-23T00:51:05.221646+0000","last_active":"2024-02-23T00:52:00.294760+0000","last_peered":"2024-02-23T00:52:00.294760+0000","last_clean":"2024-02-23T00:52:00.294760+0000","last_became_active":"2024-02-23T00:39:16.859002+0000","last_became_peered":"2024-02-23T00:39:16.859002+0000","last_unstale":"2024-02-23T00:52:00.294760+0000","last_undegraded":"2024-02-23T00:52:00.294760+0000","last_fullsized":"2024-02-23T00:52:00.294760+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'52","last_scrub_stamp":"2024-02-23T00:51:05.221572+0000","last_deep_scrub":"251'40","last_deep_scrub_stamp":"2024-02-23T00:45:23.910546+0000","last_clean_scrub_stamp":"2024-02-23T00:51:05.221572+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":76,"num_read_kb":62,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":551,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.503494+0000","last_change":"2024-02-23T00:48:44.728592+0000","last_active":"2024-02-23T00:52:00.503494+0000","last_peered":"2024-02-23T00:52:00.503494+0000","last_clean":"2024-02-23T00:52:00.503494+0000","last_became_active":"2024-02-23T00:39:14.779193+0000","last_became_peered":"2024-02-23T00:39:14.779193+0000","last_unstale":"2024-02-23T00:52:00.503494+0000","last_undegraded":"2024-02-23T00:52:00.503494+0000","last_fullsized":"2024-02-23T00:52:00.503494+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:44.728520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:48:44.728520+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":582,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.306095+0000","last_change":"2024-02-23T00:50:17.599788+0000","last_active":"2024-02-23T00:52:00.306095+0000","last_peered":"2024-02-23T00:52:00.306095+0000","last_clean":"2024-02-23T00:52:00.306095+0000","last_became_active":"2024-02-23T00:47:33.257915+0000","last_became_peered":"2024-02-23T00:47:33.257915+0000","last_unstale":"2024-02-23T00:52:00.306095+0000","last_undegraded":"2024-02-23T00:52:00.306095+0000","last_fullsized":"2024-02-23T00:52:00.306095+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":381,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:17.599724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:50:17.599724+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":507,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.297091+0000","last_change":"2024-02-23T00:51:03.166243+0000","last_active":"2024-02-23T00:52:00.297091+0000","last_peered":"2024-02-23T00:52:00.297091+0000","last_clean":"2024-02-23T00:52:00.297091+0000","last_became_active":"2024-02-23T00:42:58.809283+0000","last_became_peered":"2024-02-23T00:42:58.809283+0000","last_unstale":"2024-02-23T00:52:00.297091+0000","last_undegraded":"2024-02-23T00:52:00.297091+0000","last_fullsized":"2024-02-23T00:52:00.297091+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:03.166140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:03.166140+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":562,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.555317+0000","last_change":"2024-02-23T00:51:34.200412+0000","last_active":"2024-02-23T00:52:00.555317+0000","last_peered":"2024-02-23T00:52:00.555317+0000","last_clean":"2024-02-23T00:52:00.555317+0000","last_became_active":"2024-02-23T00:39:18.796664+0000","last_became_peered":"2024-02-23T00:39:18.796664+0000","last_unstale":"2024-02-23T00:52:00.555317+0000","last_undegraded":"2024-02-23T00:52:00.555317+0000","last_fullsized":"2024-02-23T00:52:00.555317+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:34.200312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:51:34.200312+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"457'44","reported_seq":636,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.303641+0000","last_change":"2024-02-23T00:49:46.644982+0000","last_active":"2024-02-23T00:52:00.303641+0000","last_peered":"2024-02-23T00:52:00.303641+0000","last_clean":"2024-02-23T00:52:00.303641+0000","last_became_active":"2024-02-23T00:39:16.857278+0000","last_became_peered":"2024-02-23T00:39:16.857278+0000","last_unstale":"2024-02-23T00:52:00.303641+0000","last_undegraded":"2024-02-23T00:52:00.303641+0000","last_fullsized":"2024-02-23T00:52:00.303641+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'44","last_scrub_stamp":"2024-02-23T00:49:46.644911+0000","last_deep_scrub":"457'44","last_deep_scrub_stamp":"2024-02-23T00:49:46.644911+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.644911+0000","log_size":44,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":559,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.400833+0000","last_change":"2024-02-23T00:50:22.299591+0000","last_active":"2024-02-23T00:52:00.400833+0000","last_peered":"2024-02-23T00:52:00.400833+0000","last_clean":"2024-02-23T00:52:00.400833+0000","last_became_active":"2024-02-23T00:39:14.783662+0000","last_became_peered":"2024-02-23T00:39:14.783662+0000","last_unstale":"2024-02-23T00:52:00.400833+0000","last_undegraded":"2024-02-23T00:52:00.400833+0000","last_fullsized":"2024-02-23T00:52:00.400833+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:22.299534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:22.299534+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":572,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.503572+0000","last_change":"2024-02-23T00:48:13.821965+0000","last_active":"2024-02-23T00:52:00.503572+0000","last_peered":"2024-02-23T00:52:00.503572+0000","last_clean":"2024-02-23T00:52:00.503572+0000","last_became_active":"2024-02-23T00:47:37.184240+0000","last_became_peered":"2024-02-23T00:47:37.184240+0000","last_unstale":"2024-02-23T00:52:00.503572+0000","last_undegraded":"2024-02-23T00:52:00.503572+0000","last_fullsized":"2024-02-23T00:52:00.503572+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":385,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:13.821901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:48:13.821901+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":503,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.439579+0000","last_change":"2024-02-23T00:50:24.682130+0000","last_active":"2024-02-23T00:52:00.439579+0000","last_peered":"2024-02-23T00:52:00.439579+0000","last_clean":"2024-02-23T00:52:00.439579+0000","last_became_active":"2024-02-23T00:42:58.726652+0000","last_became_peered":"2024-02-23T00:42:58.726652+0000","last_unstale":"2024-02-23T00:52:00.439579+0000","last_undegraded":"2024-02-23T00:52:00.439579+0000","last_fullsized":"2024-02-23T00:52:00.439579+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:24.682039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:24.682039+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":563,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.439565+0000","last_change":"2024-02-23T00:51:43.644655+0000","last_active":"2024-02-23T00:52:00.439565+0000","last_peered":"2024-02-23T00:52:00.439565+0000","last_clean":"2024-02-23T00:52:00.439565+0000","last_became_active":"2024-02-23T00:39:18.800378+0000","last_became_peered":"2024-02-23T00:39:18.800378+0000","last_unstale":"2024-02-23T00:52:00.439565+0000","last_undegraded":"2024-02-23T00:52:00.439565+0000","last_fullsized":"2024-02-23T00:52:00.439565+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:43.644591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:59.694033+0000","last_clean_scrub_stamp":"2024-02-23T00:51:43.644591+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"457'30","reported_seq":626,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.303684+0000","last_change":"2024-02-23T00:49:42.730745+0000","last_active":"2024-02-23T00:52:00.303684+0000","last_peered":"2024-02-23T00:52:00.303684+0000","last_clean":"2024-02-23T00:52:00.303684+0000","last_became_active":"2024-02-23T00:47:28.215366+0000","last_became_peered":"2024-02-23T00:47:28.215366+0000","last_unstale":"2024-02-23T00:52:00.303684+0000","last_undegraded":"2024-02-23T00:52:00.303684+0000","last_fullsized":"2024-02-23T00:52:00.303684+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'30","last_scrub_stamp":"2024-02-23T00:49:42.730670+0000","last_deep_scrub":"95'18","last_deep_scrub_stamp":"2024-02-23T00:42:01.195545+0000","last_clean_scrub_stamp":"2024-02-23T00:49:42.730670+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":564,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.398898+0000","last_change":"2024-02-23T00:49:46.162522+0000","last_active":"2024-02-23T00:52:00.398898+0000","last_peered":"2024-02-23T00:52:00.398898+0000","last_clean":"2024-02-23T00:52:00.398898+0000","last_became_active":"2024-02-23T00:39:14.784578+0000","last_became_peered":"2024-02-23T00:39:14.784578+0000","last_unstale":"2024-02-23T00:52:00.398898+0000","last_undegraded":"2024-02-23T00:52:00.398898+0000","last_fullsized":"2024-02-23T00:52:00.398898+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:46.162444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.162444+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":563,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.302535+0000","last_change":"2024-02-23T00:49:31.768539+0000","last_active":"2024-02-23T00:52:00.302535+0000","last_peered":"2024-02-23T00:52:00.302535+0000","last_clean":"2024-02-23T00:52:00.302535+0000","last_became_active":"2024-02-23T00:39:14.775446+0000","last_became_peered":"2024-02-23T00:39:14.775446+0000","last_unstale":"2024-02-23T00:52:00.302535+0000","last_undegraded":"2024-02-23T00:52:00.302535+0000","last_fullsized":"2024-02-23T00:52:00.302535+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:31.768472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:31.768472+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"67'1","reported_seq":583,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.503659+0000","last_change":"2024-02-23T00:51:02.380789+0000","last_active":"2024-02-23T00:52:00.503659+0000","last_peered":"2024-02-23T00:52:00.503659+0000","last_clean":"2024-02-23T00:52:00.503659+0000","last_became_active":"2024-02-23T00:47:56.187940+0000","last_became_peered":"2024-02-23T00:47:56.187940+0000","last_unstale":"2024-02-23T00:52:00.503659+0000","last_undegraded":"2024-02-23T00:52:00.503659+0000","last_fullsized":"2024-02-23T00:52:00.503659+0000","mapping_epoch":65,"log_start":"67'1","ondisk_log_start":"67'1","created":65,"last_epoch_clean":404,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:51:02.380721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:51:02.380721+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":508,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.554960+0000","last_change":"2024-02-23T00:51:10.298816+0000","last_active":"2024-02-23T00:52:00.554960+0000","last_peered":"2024-02-23T00:52:00.554960+0000","last_clean":"2024-02-23T00:52:00.554960+0000","last_became_active":"2024-02-23T00:42:58.690858+0000","last_became_peered":"2024-02-23T00:42:58.690858+0000","last_unstale":"2024-02-23T00:52:00.554960+0000","last_undegraded":"2024-02-23T00:52:00.554960+0000","last_fullsized":"2024-02-23T00:52:00.554960+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:10.298740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:35.360054+0000","last_clean_scrub_stamp":"2024-02-23T00:51:10.298740+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":569,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.397562+0000","last_change":"2024-02-23T00:51:39.221685+0000","last_active":"2024-02-23T00:52:00.397562+0000","last_peered":"2024-02-23T00:52:00.397562+0000","last_clean":"2024-02-23T00:52:00.397562+0000","last_became_active":"2024-02-23T00:47:28.231152+0000","last_became_peered":"2024-02-23T00:47:28.231152+0000","last_unstale":"2024-02-23T00:52:00.397562+0000","last_undegraded":"2024-02-23T00:52:00.397562+0000","last_fullsized":"2024-02-23T00:52:00.397562+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:39.221609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:51:39.221609+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"457'26","reported_seq":604,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.556499+0000","last_change":"2024-02-23T00:51:01.219903+0000","last_active":"2024-02-23T00:52:00.556499+0000","last_peered":"2024-02-23T00:52:00.556499+0000","last_clean":"2024-02-23T00:52:00.556499+0000","last_became_active":"2024-02-23T00:39:16.858880+0000","last_became_peered":"2024-02-23T00:39:16.858880+0000","last_unstale":"2024-02-23T00:52:00.556499+0000","last_undegraded":"2024-02-23T00:52:00.556499+0000","last_fullsized":"2024-02-23T00:52:00.556499+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-23T00:51:01.219824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:01.219824+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":569,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.399610+0000","last_change":"2024-02-23T00:50:24.297660+0000","last_active":"2024-02-23T00:52:00.399610+0000","last_peered":"2024-02-23T00:52:00.399610+0000","last_clean":"2024-02-23T00:52:00.399610+0000","last_became_active":"2024-02-23T00:39:08.248249+0000","last_became_peered":"2024-02-23T00:39:08.248249+0000","last_unstale":"2024-02-23T00:52:00.399610+0000","last_undegraded":"2024-02-23T00:52:00.399610+0000","last_fullsized":"2024-02-23T00:52:00.399610+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:24.297572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:49.977248+0000","last_clean_scrub_stamp":"2024-02-23T00:50:24.297572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":571,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.437437+0000","last_change":"2024-02-23T00:50:21.746072+0000","last_active":"2024-02-23T00:52:00.437437+0000","last_peered":"2024-02-23T00:52:00.437437+0000","last_clean":"2024-02-23T00:52:00.437437+0000","last_became_active":"2024-02-23T00:47:28.240482+0000","last_became_peered":"2024-02-23T00:47:28.240482+0000","last_unstale":"2024-02-23T00:52:00.437437+0000","last_undegraded":"2024-02-23T00:52:00.437437+0000","last_fullsized":"2024-02-23T00:52:00.437437+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:21.745961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:21.745961+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":566,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.295219+0000","last_change":"2024-02-23T00:50:34.055990+0000","last_active":"2024-02-23T00:52:00.295219+0000","last_peered":"2024-02-23T00:52:00.295219+0000","last_clean":"2024-02-23T00:52:00.295219+0000","last_became_active":"2024-02-23T00:39:14.783489+0000","last_became_peered":"2024-02-23T00:39:14.783489+0000","last_unstale":"2024-02-23T00:52:00.295219+0000","last_undegraded":"2024-02-23T00:52:00.295219+0000","last_fullsized":"2024-02-23T00:52:00.295219+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:34.055923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:34.055923+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":592,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.439637+0000","last_change":"2024-02-23T00:49:35.756168+0000","last_active":"2024-02-23T00:52:00.439637+0000","last_peered":"2024-02-23T00:52:00.439637+0000","last_clean":"2024-02-23T00:52:00.439637+0000","last_became_active":"2024-02-23T00:47:45.182550+0000","last_became_peered":"2024-02-23T00:47:45.182550+0000","last_unstale":"2024-02-23T00:52:00.439637+0000","last_undegraded":"2024-02-23T00:52:00.439637+0000","last_fullsized":"2024-02-23T00:52:00.439637+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:35.756073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:49:35.756073+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":513,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.397377+0000","last_change":"2024-02-23T00:50:16.257963+0000","last_active":"2024-02-23T00:52:00.397377+0000","last_peered":"2024-02-23T00:52:00.397377+0000","last_clean":"2024-02-23T00:52:00.397377+0000","last_became_active":"2024-02-23T00:47:28.233475+0000","last_became_peered":"2024-02-23T00:47:28.233475+0000","last_unstale":"2024-02-23T00:52:00.397377+0000","last_undegraded":"2024-02-23T00:52:00.397377+0000","last_fullsized":"2024-02-23T00:52:00.397377+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:16.257896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:16.257896+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":235,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.437434+0000","last_change":"2024-02-23T00:51:12.709695+0000","last_active":"2024-02-23T00:52:00.437434+0000","last_peered":"2024-02-23T00:52:00.437434+0000","last_clean":"2024-02-23T00:52:00.437434+0000","last_became_active":"2024-02-23T00:47:28.209125+0000","last_became_peered":"2024-02-23T00:47:28.209125+0000","last_unstale":"2024-02-23T00:52:00.437434+0000","last_undegraded":"2024-02-23T00:52:00.437434+0000","last_fullsized":"2024-02-23T00:52:00.437434+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.709617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.264240+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.709617+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"457'23","reported_seq":602,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.505683+0000","last_change":"2024-02-23T00:51:42.560222+0000","last_active":"2024-02-23T00:52:00.505683+0000","last_peered":"2024-02-23T00:52:00.505683+0000","last_clean":"2024-02-23T00:52:00.505683+0000","last_became_active":"2024-02-23T00:39:16.787484+0000","last_became_peered":"2024-02-23T00:39:16.787484+0000","last_unstale":"2024-02-23T00:52:00.505683+0000","last_undegraded":"2024-02-23T00:52:00.505683+0000","last_fullsized":"2024-02-23T00:52:00.505683+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'23","last_scrub_stamp":"2024-02-23T00:51:42.560138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:42.560138+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":48,"num_read_kb":36,"num_write":26,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":238,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.295046+0000","last_change":"2024-02-23T00:51:44.659312+0000","last_active":"2024-02-23T00:52:00.295046+0000","last_peered":"2024-02-23T00:52:00.295046+0000","last_clean":"2024-02-23T00:52:00.295046+0000","last_became_active":"2024-02-23T00:47:28.209575+0000","last_became_peered":"2024-02-23T00:47:28.209575+0000","last_unstale":"2024-02-23T00:52:00.295046+0000","last_undegraded":"2024-02-23T00:52:00.295046+0000","last_fullsized":"2024-02-23T00:52:00.295046+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:44.659246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:44.659246+0000","last_clean_scrub_stamp":"2024-02-23T00:51:44.659246+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"68'5","reported_seq":1325,"reported_epoch":592,"state":"active+clean","last_fresh":"2024-02-23T00:52:01.289985+0000","last_change":"2024-02-23T00:50:57.456201+0000","last_active":"2024-02-23T00:52:01.289985+0000","last_peered":"2024-02-23T00:52:01.289985+0000","last_clean":"2024-02-23T00:52:01.289985+0000","last_became_active":"2024-02-23T00:47:28.227928+0000","last_became_peered":"2024-02-23T00:47:28.227928+0000","last_unstale":"2024-02-23T00:52:01.289985+0000","last_undegraded":"2024-02-23T00:52:01.289985+0000","last_fullsized":"2024-02-23T00:52:01.289985+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'5","last_scrub_stamp":"2024-02-23T00:50:57.456059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:57.456059+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":757,"num_read_kb":752,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":589,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.399679+0000","last_change":"2024-02-23T00:49:30.396935+0000","last_active":"2024-02-23T00:52:00.399679+0000","last_peered":"2024-02-23T00:52:00.399679+0000","last_clean":"2024-02-23T00:52:00.399679+0000","last_became_active":"2024-02-23T00:47:52.552244+0000","last_became_peered":"2024-02-23T00:47:52.552244+0000","last_unstale":"2024-02-23T00:52:00.399679+0000","last_undegraded":"2024-02-23T00:52:00.399679+0000","last_fullsized":"2024-02-23T00:52:00.399679+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:30.396865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:49:30.396865+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":502,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.501384+0000","last_change":"2024-02-23T00:49:40.533441+0000","last_active":"2024-02-23T00:52:00.501384+0000","last_peered":"2024-02-23T00:52:00.501384+0000","last_clean":"2024-02-23T00:52:00.501384+0000","last_became_active":"2024-02-23T00:47:28.212516+0000","last_became_peered":"2024-02-23T00:47:28.212516+0000","last_unstale":"2024-02-23T00:52:00.501384+0000","last_undegraded":"2024-02-23T00:52:00.501384+0000","last_fullsized":"2024-02-23T00:52:00.501384+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:40.533344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:49:40.533344+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":559,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.305479+0000","last_change":"2024-02-23T00:49:37.787041+0000","last_active":"2024-02-23T00:52:00.305479+0000","last_peered":"2024-02-23T00:52:00.305479+0000","last_clean":"2024-02-23T00:52:00.305479+0000","last_became_active":"2024-02-23T00:39:18.802134+0000","last_became_peered":"2024-02-23T00:39:18.802134+0000","last_unstale":"2024-02-23T00:52:00.305479+0000","last_undegraded":"2024-02-23T00:52:00.305479+0000","last_fullsized":"2024-02-23T00:52:00.305479+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:37.786978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:37.786978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"457'48","reported_seq":748,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.438782+0000","last_change":"2024-02-23T00:50:30.763802+0000","last_active":"2024-02-23T00:52:00.438782+0000","last_peered":"2024-02-23T00:52:00.438782+0000","last_clean":"2024-02-23T00:52:00.438782+0000","last_became_active":"2024-02-23T00:47:28.240257+0000","last_became_peered":"2024-02-23T00:47:28.240257+0000","last_unstale":"2024-02-23T00:52:00.438782+0000","last_undegraded":"2024-02-23T00:52:00.438782+0000","last_fullsized":"2024-02-23T00:52:00.438782+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'48","last_scrub_stamp":"2024-02-23T00:50:30.763686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:30.763686+0000","log_size":48,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":78,"num_read_kb":62,"num_write":44,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":562,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.437769+0000","last_change":"2024-02-23T00:49:41.639660+0000","last_active":"2024-02-23T00:52:00.437769+0000","last_peered":"2024-02-23T00:52:00.437769+0000","last_clean":"2024-02-23T00:52:00.437769+0000","last_became_active":"2024-02-23T00:39:14.784015+0000","last_became_peered":"2024-02-23T00:39:14.784015+0000","last_unstale":"2024-02-23T00:52:00.437769+0000","last_undegraded":"2024-02-23T00:52:00.437769+0000","last_fullsized":"2024-02-23T00:52:00.437769+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:41.639582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:41.639582+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":596,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.297028+0000","last_change":"2024-02-23T00:51:46.680015+0000","last_active":"2024-02-23T00:52:00.297028+0000","last_peered":"2024-02-23T00:52:00.297028+0000","last_clean":"2024-02-23T00:52:00.297028+0000","last_became_active":"2024-02-23T00:47:49.525421+0000","last_became_peered":"2024-02-23T00:47:49.525421+0000","last_unstale":"2024-02-23T00:52:00.297028+0000","last_undegraded":"2024-02-23T00:52:00.297028+0000","last_fullsized":"2024-02-23T00:52:00.297028+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":397,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:46.679950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:51:46.679950+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":232,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.306188+0000","last_change":"2024-02-23T00:50:13.549604+0000","last_active":"2024-02-23T00:52:00.306188+0000","last_peered":"2024-02-23T00:52:00.306188+0000","last_clean":"2024-02-23T00:52:00.306188+0000","last_became_active":"2024-02-23T00:47:28.232194+0000","last_became_peered":"2024-02-23T00:47:28.232194+0000","last_unstale":"2024-02-23T00:52:00.306188+0000","last_undegraded":"2024-02-23T00:52:00.306188+0000","last_fullsized":"2024-02-23T00:52:00.306188+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:13.549529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:13.549529+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3","version":"91'9","reported_seq":721,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.303982+0000","last_change":"2024-02-23T00:49:41.776249+0000","last_active":"2024-02-23T00:52:00.303982+0000","last_peered":"2024-02-23T00:52:00.303982+0000","last_clean":"2024-02-23T00:52:00.303982+0000","last_became_active":"2024-02-23T00:39:18.801297+0000","last_became_peered":"2024-02-23T00:39:18.801297+0000","last_unstale":"2024-02-23T00:52:00.303982+0000","last_undegraded":"2024-02-23T00:52:00.303982+0000","last_fullsized":"2024-02-23T00:52:00.303982+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'9","last_scrub_stamp":"2024-02-23T00:49:41.776117+0000","last_deep_scrub":"91'9","last_deep_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_clean_scrub_stamp":"2024-02-23T00:49:41.776117+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"457'26","reported_seq":594,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.505242+0000","last_change":"2024-02-23T00:48:55.589320+0000","last_active":"2024-02-23T00:52:00.505242+0000","last_peered":"2024-02-23T00:52:00.505242+0000","last_clean":"2024-02-23T00:52:00.505242+0000","last_became_active":"2024-02-23T00:39:16.787468+0000","last_became_peered":"2024-02-23T00:39:16.787468+0000","last_unstale":"2024-02-23T00:52:00.505242+0000","last_undegraded":"2024-02-23T00:52:00.505242+0000","last_fullsized":"2024-02-23T00:52:00.505242+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:48:55.589209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:48:55.589209+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":36,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":556,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.502951+0000","last_change":"2024-02-23T00:51:36.415402+0000","last_active":"2024-02-23T00:52:00.502951+0000","last_peered":"2024-02-23T00:52:00.502951+0000","last_clean":"2024-02-23T00:52:00.502951+0000","last_became_active":"2024-02-23T00:39:14.779952+0000","last_became_peered":"2024-02-23T00:39:14.779952+0000","last_unstale":"2024-02-23T00:52:00.502951+0000","last_undegraded":"2024-02-23T00:52:00.502951+0000","last_fullsized":"2024-02-23T00:52:00.502951+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:36.415340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:36.415340+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"107'2","reported_seq":592,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.302284+0000","last_change":"2024-02-23T00:50:17.587665+0000","last_active":"2024-02-23T00:52:00.302284+0000","last_peered":"2024-02-23T00:52:00.302284+0000","last_clean":"2024-02-23T00:52:00.302284+0000","last_became_active":"2024-02-23T00:47:41.210074+0000","last_became_peered":"2024-02-23T00:47:41.210074+0000","last_unstale":"2024-02-23T00:52:00.302284+0000","last_undegraded":"2024-02-23T00:52:00.302284+0000","last_fullsized":"2024-02-23T00:52:00.302284+0000","mapping_epoch":375,"log_start":"107'2","ondisk_log_start":"107'2","created":65,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'2","last_scrub_stamp":"2024-02-23T00:50:17.587591+0000","last_deep_scrub":"107'2","last_deep_scrub_stamp":"2024-02-23T00:42:48.362739+0000","last_clean_scrub_stamp":"2024-02-23T00:50:17.587591+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":512,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.554707+0000","last_change":"2024-02-23T00:50:18.308436+0000","last_active":"2024-02-23T00:52:00.554707+0000","last_peered":"2024-02-23T00:52:00.554707+0000","last_clean":"2024-02-23T00:52:00.554707+0000","last_became_active":"2024-02-23T00:47:28.242255+0000","last_became_peered":"2024-02-23T00:47:28.242255+0000","last_unstale":"2024-02-23T00:52:00.554707+0000","last_undegraded":"2024-02-23T00:52:00.554707+0000","last_fullsized":"2024-02-23T00:52:00.554707+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:18.308350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:18.308350+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":236,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.302750+0000","last_change":"2024-02-23T00:51:35.631662+0000","last_active":"2024-02-23T00:52:00.302750+0000","last_peered":"2024-02-23T00:52:00.302750+0000","last_clean":"2024-02-23T00:52:00.302750+0000","last_became_active":"2024-02-23T00:47:28.234034+0000","last_became_peered":"2024-02-23T00:47:28.234034+0000","last_unstale":"2024-02-23T00:52:00.302750+0000","last_undegraded":"2024-02-23T00:52:00.302750+0000","last_fullsized":"2024-02-23T00:52:00.302750+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.631598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.631598+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"457'31","reported_seq":622,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.307423+0000","last_change":"2024-02-23T00:51:43.724733+0000","last_active":"2024-02-23T00:52:00.307423+0000","last_peered":"2024-02-23T00:52:00.307423+0000","last_clean":"2024-02-23T00:52:00.307423+0000","last_became_active":"2024-02-23T00:39:16.783926+0000","last_became_peered":"2024-02-23T00:39:16.783926+0000","last_unstale":"2024-02-23T00:52:00.307423+0000","last_undegraded":"2024-02-23T00:52:00.307423+0000","last_fullsized":"2024-02-23T00:52:00.307423+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:51:43.724670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:43.724670+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":58,"num_read_kb":47,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"95'4","reported_seq":576,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.307204+0000","last_change":"2024-02-23T00:49:33.465769+0000","last_active":"2024-02-23T00:52:00.307204+0000","last_peered":"2024-02-23T00:52:00.307204+0000","last_clean":"2024-02-23T00:52:00.307204+0000","last_became_active":"2024-02-23T00:39:14.779205+0000","last_became_peered":"2024-02-23T00:39:14.779205+0000","last_unstale":"2024-02-23T00:52:00.307204+0000","last_undegraded":"2024-02-23T00:52:00.307204+0000","last_fullsized":"2024-02-23T00:52:00.307204+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'4","last_scrub_stamp":"2024-02-23T00:49:33.465697+0000","last_deep_scrub":"95'4","last_deep_scrub_stamp":"2024-02-23T00:41:43.874397+0000","last_clean_scrub_stamp":"2024-02-23T00:49:33.465697+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":578,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.295567+0000","last_change":"2024-02-23T00:50:15.922829+0000","last_active":"2024-02-23T00:52:00.295567+0000","last_peered":"2024-02-23T00:52:00.295567+0000","last_clean":"2024-02-23T00:52:00.295567+0000","last_became_active":"2024-02-23T00:43:27.611660+0000","last_became_peered":"2024-02-23T00:43:27.611660+0000","last_unstale":"2024-02-23T00:52:00.295567+0000","last_undegraded":"2024-02-23T00:52:00.295567+0000","last_fullsized":"2024-02-23T00:52:00.295567+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:15.922745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:50:15.922745+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":516,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.397800+0000","last_change":"2024-02-23T00:51:46.336777+0000","last_active":"2024-02-23T00:52:00.397800+0000","last_peered":"2024-02-23T00:52:00.397800+0000","last_clean":"2024-02-23T00:52:00.397800+0000","last_became_active":"2024-02-23T00:47:28.231670+0000","last_became_peered":"2024-02-23T00:47:28.231670+0000","last_unstale":"2024-02-23T00:52:00.397800+0000","last_undegraded":"2024-02-23T00:52:00.397800+0000","last_fullsized":"2024-02-23T00:52:00.397800+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:46.336718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:46.336718+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"457'27","reported_seq":616,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.303774+0000","last_change":"2024-02-23T00:51:42.671042+0000","last_active":"2024-02-23T00:52:00.303774+0000","last_peered":"2024-02-23T00:52:00.303774+0000","last_clean":"2024-02-23T00:52:00.303774+0000","last_became_active":"2024-02-23T00:39:16.858760+0000","last_became_peered":"2024-02-23T00:39:16.858760+0000","last_unstale":"2024-02-23T00:52:00.303774+0000","last_undegraded":"2024-02-23T00:52:00.303774+0000","last_fullsized":"2024-02-23T00:52:00.303774+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-23T00:51:42.670972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:42.670972+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":226,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.501627+0000","last_change":"2024-02-23T00:49:02.601790+0000","last_active":"2024-02-23T00:52:00.501627+0000","last_peered":"2024-02-23T00:52:00.501627+0000","last_clean":"2024-02-23T00:52:00.501627+0000","last_became_active":"2024-02-23T00:47:28.219764+0000","last_became_peered":"2024-02-23T00:47:28.219764+0000","last_unstale":"2024-02-23T00:52:00.501627+0000","last_undegraded":"2024-02-23T00:52:00.501627+0000","last_fullsized":"2024-02-23T00:52:00.501627+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:02.601726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:02.601726+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":567,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.297912+0000","last_change":"2024-02-23T00:51:45.695038+0000","last_active":"2024-02-23T00:52:00.297912+0000","last_peered":"2024-02-23T00:52:00.297912+0000","last_clean":"2024-02-23T00:52:00.297912+0000","last_became_active":"2024-02-23T00:39:18.798624+0000","last_became_peered":"2024-02-23T00:39:18.798624+0000","last_unstale":"2024-02-23T00:52:00.297912+0000","last_undegraded":"2024-02-23T00:52:00.297912+0000","last_fullsized":"2024-02-23T00:52:00.297912+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:45.694974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:45.913526+0000","last_clean_scrub_stamp":"2024-02-23T00:51:45.694974+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":504,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.302509+0000","last_change":"2024-02-23T00:50:13.642424+0000","last_active":"2024-02-23T00:52:00.302509+0000","last_peered":"2024-02-23T00:52:00.302509+0000","last_clean":"2024-02-23T00:52:00.302509+0000","last_became_active":"2024-02-23T00:42:58.825015+0000","last_became_peered":"2024-02-23T00:42:58.825015+0000","last_unstale":"2024-02-23T00:52:00.302509+0000","last_undegraded":"2024-02-23T00:52:00.302509+0000","last_fullsized":"2024-02-23T00:52:00.302509+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:13.642359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:13.642359+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"457'22","reported_seq":601,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.297699+0000","last_change":"2024-02-23T00:51:01.201807+0000","last_active":"2024-02-23T00:52:00.297699+0000","last_peered":"2024-02-23T00:52:00.297699+0000","last_clean":"2024-02-23T00:52:00.297699+0000","last_became_active":"2024-02-23T00:39:16.858449+0000","last_became_peered":"2024-02-23T00:39:16.858449+0000","last_unstale":"2024-02-23T00:52:00.297699+0000","last_undegraded":"2024-02-23T00:52:00.297699+0000","last_fullsized":"2024-02-23T00:52:00.297699+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'22","last_scrub_stamp":"2024-02-23T00:51:01.201737+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-02-23T00:48:07.941101+0000","last_clean_scrub_stamp":"2024-02-23T00:51:01.201737+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":30,"num_read_kb":26,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":561,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.556179+0000","last_change":"2024-02-23T00:49:28.347354+0000","last_active":"2024-02-23T00:52:00.556179+0000","last_peered":"2024-02-23T00:52:00.556179+0000","last_clean":"2024-02-23T00:52:00.556179+0000","last_became_active":"2024-02-23T00:39:14.783164+0000","last_became_peered":"2024-02-23T00:39:14.783164+0000","last_unstale":"2024-02-23T00:52:00.556179+0000","last_undegraded":"2024-02-23T00:52:00.556179+0000","last_fullsized":"2024-02-23T00:52:00.556179+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:28.347281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:28.347281+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":563,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.439635+0000","last_change":"2024-02-23T00:51:47.691352+0000","last_active":"2024-02-23T00:52:00.439635+0000","last_peered":"2024-02-23T00:52:00.439635+0000","last_clean":"2024-02-23T00:52:00.439635+0000","last_became_active":"2024-02-23T00:39:18.800297+0000","last_became_peered":"2024-02-23T00:39:18.800297+0000","last_unstale":"2024-02-23T00:52:00.439635+0000","last_undegraded":"2024-02-23T00:52:00.439635+0000","last_fullsized":"2024-02-23T00:52:00.439635+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:47.691288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:51:47.691288+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1","version":"110'7","reported_seq":520,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.296731+0000","last_change":"2024-02-23T00:51:13.352686+0000","last_active":"2024-02-23T00:52:00.296731+0000","last_peered":"2024-02-23T00:52:00.296731+0000","last_clean":"2024-02-23T00:52:00.296731+0000","last_became_active":"2024-02-23T00:42:58.777251+0000","last_became_peered":"2024-02-23T00:42:58.777251+0000","last_unstale":"2024-02-23T00:52:00.296731+0000","last_undegraded":"2024-02-23T00:52:00.296731+0000","last_fullsized":"2024-02-23T00:52:00.296731+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'7","last_scrub_stamp":"2024-02-23T00:51:13.352611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:13.352611+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"457'32","reported_seq":262,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.296110+0000","last_change":"2024-02-23T00:50:25.079859+0000","last_active":"2024-02-23T00:52:00.296110+0000","last_peered":"2024-02-23T00:52:00.296110+0000","last_clean":"2024-02-23T00:52:00.296110+0000","last_became_active":"2024-02-23T00:47:28.237898+0000","last_became_peered":"2024-02-23T00:47:28.237898+0000","last_unstale":"2024-02-23T00:52:00.296110+0000","last_undegraded":"2024-02-23T00:52:00.296110+0000","last_fullsized":"2024-02-23T00:52:00.296110+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-23T00:50:25.079789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:25.079789+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":52,"num_read_kb":45,"num_write":32,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":567,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.437763+0000","last_change":"2024-02-23T00:51:14.658413+0000","last_active":"2024-02-23T00:52:00.437763+0000","last_peered":"2024-02-23T00:52:00.437763+0000","last_clean":"2024-02-23T00:52:00.437763+0000","last_became_active":"2024-02-23T00:39:14.785590+0000","last_became_peered":"2024-02-23T00:39:14.785590+0000","last_unstale":"2024-02-23T00:52:00.437763+0000","last_undegraded":"2024-02-23T00:52:00.437763+0000","last_fullsized":"2024-02-23T00:52:00.437763+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:14.658297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_clean_scrub_stamp":"2024-02-23T00:51:14.658297+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"336'9","reported_seq":720,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.439877+0000","last_change":"2024-02-23T00:49:39.636503+0000","last_active":"2024-02-23T00:52:00.439877+0000","last_peered":"2024-02-23T00:52:00.439877+0000","last_clean":"2024-02-23T00:52:00.439877+0000","last_became_active":"2024-02-23T00:39:18.803566+0000","last_became_peered":"2024-02-23T00:39:18.803566+0000","last_unstale":"2024-02-23T00:52:00.439877+0000","last_undegraded":"2024-02-23T00:52:00.439877+0000","last_fullsized":"2024-02-23T00:52:00.439877+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"336'9","last_scrub_stamp":"2024-02-23T00:49:39.636409+0000","last_deep_scrub":"336'9","last_deep_scrub_stamp":"2024-02-23T00:46:49.160991+0000","last_clean_scrub_stamp":"2024-02-23T00:49:39.636409+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":502,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.399665+0000","last_change":"2024-02-23T00:51:33.285134+0000","last_active":"2024-02-23T00:52:00.399665+0000","last_peered":"2024-02-23T00:52:00.399665+0000","last_clean":"2024-02-23T00:52:00.399665+0000","last_became_active":"2024-02-23T00:42:58.757424+0000","last_became_peered":"2024-02-23T00:42:58.757424+0000","last_unstale":"2024-02-23T00:52:00.399665+0000","last_undegraded":"2024-02-23T00:52:00.399665+0000","last_fullsized":"2024-02-23T00:52:00.399665+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:33.285048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:33.285048+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"457'21","reported_seq":600,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.439758+0000","last_change":"2024-02-23T00:50:17.861556+0000","last_active":"2024-02-23T00:52:00.439758+0000","last_peered":"2024-02-23T00:52:00.439758+0000","last_clean":"2024-02-23T00:52:00.439758+0000","last_became_active":"2024-02-23T00:39:16.858953+0000","last_became_peered":"2024-02-23T00:39:16.858953+0000","last_unstale":"2024-02-23T00:52:00.439758+0000","last_undegraded":"2024-02-23T00:52:00.439758+0000","last_fullsized":"2024-02-23T00:52:00.439758+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-23T00:50:17.861490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:17.861490+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":568,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.303803+0000","last_change":"2024-02-23T00:51:33.662415+0000","last_active":"2024-02-23T00:52:00.303803+0000","last_peered":"2024-02-23T00:52:00.303803+0000","last_clean":"2024-02-23T00:52:00.303803+0000","last_became_active":"2024-02-23T00:39:14.773772+0000","last_became_peered":"2024-02-23T00:39:14.773772+0000","last_unstale":"2024-02-23T00:52:00.303803+0000","last_undegraded":"2024-02-23T00:52:00.303803+0000","last_fullsized":"2024-02-23T00:52:00.303803+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:33.662354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:33.662354+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"376'9","reported_seq":731,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.296640+0000","last_change":"2024-02-23T00:49:45.868359+0000","last_active":"2024-02-23T00:52:00.296640+0000","last_peered":"2024-02-23T00:52:00.296640+0000","last_clean":"2024-02-23T00:52:00.296640+0000","last_became_active":"2024-02-23T00:47:28.229316+0000","last_became_peered":"2024-02-23T00:47:28.229316+0000","last_unstale":"2024-02-23T00:52:00.296640+0000","last_undegraded":"2024-02-23T00:52:00.296640+0000","last_fullsized":"2024-02-23T00:52:00.296640+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"376'9","last_scrub_stamp":"2024-02-23T00:49:45.868289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:45.868289+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":493,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.506573+0000","last_change":"2024-02-23T00:50:15.714265+0000","last_active":"2024-02-23T00:52:00.506573+0000","last_peered":"2024-02-23T00:52:00.506573+0000","last_clean":"2024-02-23T00:52:00.506573+0000","last_became_active":"2024-02-23T00:42:57.652488+0000","last_became_peered":"2024-02-23T00:42:57.652488+0000","last_unstale":"2024-02-23T00:52:00.506573+0000","last_undegraded":"2024-02-23T00:52:00.506573+0000","last_fullsized":"2024-02-23T00:52:00.506573+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:15.714200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:51.254903+0000","last_clean_scrub_stamp":"2024-02-23T00:50:15.714200+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"457'15","reported_seq":593,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.501399+0000","last_change":"2024-02-23T00:49:39.573557+0000","last_active":"2024-02-23T00:52:00.501399+0000","last_peered":"2024-02-23T00:52:00.501399+0000","last_clean":"2024-02-23T00:52:00.501399+0000","last_became_active":"2024-02-23T00:47:28.225585+0000","last_became_peered":"2024-02-23T00:47:28.225585+0000","last_unstale":"2024-02-23T00:52:00.501399+0000","last_undegraded":"2024-02-23T00:52:00.501399+0000","last_fullsized":"2024-02-23T00:52:00.501399+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'15","last_scrub_stamp":"2024-02-23T00:49:39.573472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:39.573472+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":571,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.555707+0000","last_change":"2024-02-23T00:51:44.270528+0000","last_active":"2024-02-23T00:52:00.555707+0000","last_peered":"2024-02-23T00:52:00.555707+0000","last_clean":"2024-02-23T00:52:00.555707+0000","last_became_active":"2024-02-23T00:39:14.778455+0000","last_became_peered":"2024-02-23T00:39:14.778455+0000","last_unstale":"2024-02-23T00:52:00.555707+0000","last_undegraded":"2024-02-23T00:52:00.555707+0000","last_fullsized":"2024-02-23T00:52:00.555707+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:44.270465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:44.270465+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":571,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.295480+0000","last_change":"2024-02-23T00:50:30.069404+0000","last_active":"2024-02-23T00:52:00.295480+0000","last_peered":"2024-02-23T00:52:00.295480+0000","last_clean":"2024-02-23T00:52:00.295480+0000","last_became_active":"2024-02-23T00:47:44.225379+0000","last_became_peered":"2024-02-23T00:47:44.225379+0000","last_unstale":"2024-02-23T00:52:00.295480+0000","last_undegraded":"2024-02-23T00:52:00.295480+0000","last_fullsized":"2024-02-23T00:52:00.295480+0000","mapping_epoch":391,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":392,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:30.069325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_clean_scrub_stamp":"2024-02-23T00:50:30.069325+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":509,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.306619+0000","last_change":"2024-02-23T00:51:44.638197+0000","last_active":"2024-02-23T00:52:00.306619+0000","last_peered":"2024-02-23T00:52:00.306619+0000","last_clean":"2024-02-23T00:52:00.306619+0000","last_became_active":"2024-02-23T00:42:58.835862+0000","last_became_peered":"2024-02-23T00:42:58.835862+0000","last_unstale":"2024-02-23T00:52:00.306619+0000","last_undegraded":"2024-02-23T00:52:00.306619+0000","last_fullsized":"2024-02-23T00:52:00.306619+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:44.638129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:44.638129+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"457'19","reported_seq":590,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.307856+0000","last_change":"2024-02-23T00:50:29.599563+0000","last_active":"2024-02-23T00:52:00.307856+0000","last_peered":"2024-02-23T00:52:00.307856+0000","last_clean":"2024-02-23T00:52:00.307856+0000","last_became_active":"2024-02-23T00:39:16.783896+0000","last_became_peered":"2024-02-23T00:39:16.783896+0000","last_unstale":"2024-02-23T00:52:00.307856+0000","last_undegraded":"2024-02-23T00:52:00.307856+0000","last_fullsized":"2024-02-23T00:52:00.307856+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'19","last_scrub_stamp":"2024-02-23T00:50:29.599483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:29.599483+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"202'2","reported_seq":564,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.504910+0000","last_change":"2024-02-23T00:51:40.511331+0000","last_active":"2024-02-23T00:52:00.504910+0000","last_peered":"2024-02-23T00:52:00.504910+0000","last_clean":"2024-02-23T00:52:00.504910+0000","last_became_active":"2024-02-23T00:39:14.775804+0000","last_became_peered":"2024-02-23T00:39:14.775804+0000","last_unstale":"2024-02-23T00:52:00.504910+0000","last_undegraded":"2024-02-23T00:52:00.504910+0000","last_fullsized":"2024-02-23T00:52:00.504910+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"202'2","last_scrub_stamp":"2024-02-23T00:51:40.511258+0000","last_deep_scrub":"202'2","last_deep_scrub_stamp":"2024-02-23T00:44:34.720640+0000","last_clean_scrub_stamp":"2024-02-23T00:51:40.511258+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":547,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.505233+0000","last_change":"2024-02-23T00:50:18.698805+0000","last_active":"2024-02-23T00:52:00.505233+0000","last_peered":"2024-02-23T00:52:00.505233+0000","last_clean":"2024-02-23T00:52:00.505233+0000","last_became_active":"2024-02-23T00:39:18.800382+0000","last_became_peered":"2024-02-23T00:39:18.800382+0000","last_unstale":"2024-02-23T00:52:00.505233+0000","last_undegraded":"2024-02-23T00:52:00.505233+0000","last_fullsized":"2024-02-23T00:52:00.505233+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:18.698687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:50:18.698687+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":502,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.501559+0000","last_change":"2024-02-23T00:49:46.462085+0000","last_active":"2024-02-23T00:52:00.501559+0000","last_peered":"2024-02-23T00:52:00.501559+0000","last_clean":"2024-02-23T00:52:00.501559+0000","last_became_active":"2024-02-23T00:47:28.224961+0000","last_became_peered":"2024-02-23T00:47:28.224961+0000","last_unstale":"2024-02-23T00:52:00.501559+0000","last_undegraded":"2024-02-23T00:52:00.501559+0000","last_fullsized":"2024-02-23T00:52:00.501559+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:46.462015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:46.462015+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.462015+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"457'8","reported_seq":576,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.294586+0000","last_change":"2024-02-23T00:50:35.045697+0000","last_active":"2024-02-23T00:52:00.294586+0000","last_peered":"2024-02-23T00:52:00.294586+0000","last_clean":"2024-02-23T00:52:00.294586+0000","last_became_active":"2024-02-23T00:39:16.785830+0000","last_became_peered":"2024-02-23T00:39:16.785830+0000","last_unstale":"2024-02-23T00:52:00.294586+0000","last_undegraded":"2024-02-23T00:52:00.294586+0000","last_fullsized":"2024-02-23T00:52:00.294586+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'8","last_scrub_stamp":"2024-02-23T00:50:35.045612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:35.045612+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":579,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.439544+0000","last_change":"2024-02-23T00:51:34.529078+0000","last_active":"2024-02-23T00:52:00.439544+0000","last_peered":"2024-02-23T00:52:00.439544+0000","last_clean":"2024-02-23T00:52:00.439544+0000","last_became_active":"2024-02-23T00:47:28.224477+0000","last_became_peered":"2024-02-23T00:47:28.224477+0000","last_unstale":"2024-02-23T00:52:00.439544+0000","last_undegraded":"2024-02-23T00:52:00.439544+0000","last_fullsized":"2024-02-23T00:52:00.439544+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:34.529012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:34.529012+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":571,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.554678+0000","last_change":"2024-02-23T00:49:33.461971+0000","last_active":"2024-02-23T00:52:00.554678+0000","last_peered":"2024-02-23T00:52:00.554678+0000","last_clean":"2024-02-23T00:52:00.554678+0000","last_became_active":"2024-02-23T00:47:44.230295+0000","last_became_peered":"2024-02-23T00:47:44.230295+0000","last_unstale":"2024-02-23T00:52:00.554678+0000","last_undegraded":"2024-02-23T00:52:00.554678+0000","last_fullsized":"2024-02-23T00:52:00.554678+0000","mapping_epoch":391,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":392,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:33.461907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:33.461907+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":502,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.401645+0000","last_change":"2024-02-23T00:51:38.169736+0000","last_active":"2024-02-23T00:52:00.401645+0000","last_peered":"2024-02-23T00:52:00.401645+0000","last_clean":"2024-02-23T00:52:00.401645+0000","last_became_active":"2024-02-23T00:42:58.741365+0000","last_became_peered":"2024-02-23T00:42:58.741365+0000","last_unstale":"2024-02-23T00:52:00.401645+0000","last_undegraded":"2024-02-23T00:52:00.401645+0000","last_fullsized":"2024-02-23T00:52:00.401645+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:38.169669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:38.169669+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"457'25","reported_seq":610,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.297097+0000","last_change":"2024-02-23T00:49:47.788110+0000","last_active":"2024-02-23T00:52:00.297097+0000","last_peered":"2024-02-23T00:52:00.297097+0000","last_clean":"2024-02-23T00:52:00.297097+0000","last_became_active":"2024-02-23T00:39:16.857232+0000","last_became_peered":"2024-02-23T00:39:16.857232+0000","last_unstale":"2024-02-23T00:52:00.297097+0000","last_undegraded":"2024-02-23T00:52:00.297097+0000","last_fullsized":"2024-02-23T00:52:00.297097+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-23T00:49:47.788044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:47.788044+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":575,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.554736+0000","last_change":"2024-02-23T00:51:35.218650+0000","last_active":"2024-02-23T00:52:00.554736+0000","last_peered":"2024-02-23T00:52:00.554736+0000","last_clean":"2024-02-23T00:52:00.554736+0000","last_became_active":"2024-02-23T00:47:28.240162+0000","last_became_peered":"2024-02-23T00:47:28.240162+0000","last_unstale":"2024-02-23T00:52:00.554736+0000","last_undegraded":"2024-02-23T00:52:00.554736+0000","last_fullsized":"2024-02-23T00:52:00.554736+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.218583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:44.285083+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.218583+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":215,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.294479+0000","last_change":"2024-02-23T00:49:42.872820+0000","last_active":"2024-02-23T00:52:00.294479+0000","last_peered":"2024-02-23T00:52:00.294479+0000","last_clean":"2024-02-23T00:52:00.294479+0000","last_became_active":"2024-02-23T00:47:44.226097+0000","last_became_peered":"2024-02-23T00:47:44.226097+0000","last_unstale":"2024-02-23T00:52:00.294479+0000","last_undegraded":"2024-02-23T00:52:00.294479+0000","last_fullsized":"2024-02-23T00:52:00.294479+0000","mapping_epoch":391,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":392,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:42.872748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:42.872748+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":236,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.437466+0000","last_change":"2024-02-23T00:50:28.678155+0000","last_active":"2024-02-23T00:52:00.437466+0000","last_peered":"2024-02-23T00:52:00.437466+0000","last_clean":"2024-02-23T00:52:00.437466+0000","last_became_active":"2024-02-23T00:47:28.200688+0000","last_became_peered":"2024-02-23T00:47:28.200688+0000","last_unstale":"2024-02-23T00:52:00.437466+0000","last_undegraded":"2024-02-23T00:52:00.437466+0000","last_fullsized":"2024-02-23T00:52:00.437466+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:28.678041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:28.678041+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"457'26","reported_seq":606,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.304025+0000","last_change":"2024-02-23T00:50:23.540361+0000","last_active":"2024-02-23T00:52:00.304025+0000","last_peered":"2024-02-23T00:52:00.304025+0000","last_clean":"2024-02-23T00:52:00.304025+0000","last_became_active":"2024-02-23T00:39:16.858945+0000","last_became_peered":"2024-02-23T00:39:16.858945+0000","last_unstale":"2024-02-23T00:52:00.304025+0000","last_undegraded":"2024-02-23T00:52:00.304025+0000","last_fullsized":"2024-02-23T00:52:00.304025+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-23T00:50:23.540232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:23.540232+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":36,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":574,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.438640+0000","last_change":"2024-02-23T00:51:09.827935+0000","last_active":"2024-02-23T00:52:00.438640+0000","last_peered":"2024-02-23T00:52:00.438640+0000","last_clean":"2024-02-23T00:52:00.438640+0000","last_became_active":"2024-02-23T00:39:14.785363+0000","last_became_peered":"2024-02-23T00:39:14.785363+0000","last_unstale":"2024-02-23T00:52:00.438640+0000","last_undegraded":"2024-02-23T00:52:00.438640+0000","last_fullsized":"2024-02-23T00:52:00.438640+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"80'2","last_scrub_stamp":"2024-02-23T00:51:09.827857+0000","last_deep_scrub":"80'2","last_deep_scrub_stamp":"2024-02-23T00:40:15.942725+0000","last_clean_scrub_stamp":"2024-02-23T00:51:09.827857+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"93'9","reported_seq":727,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.557638+0000","last_change":"2024-02-23T00:51:39.253717+0000","last_active":"2024-02-23T00:52:00.557638+0000","last_peered":"2024-02-23T00:52:00.557638+0000","last_clean":"2024-02-23T00:52:00.557638+0000","last_became_active":"2024-02-23T00:39:18.796680+0000","last_became_peered":"2024-02-23T00:39:18.796680+0000","last_unstale":"2024-02-23T00:52:00.557638+0000","last_undegraded":"2024-02-23T00:52:00.557638+0000","last_fullsized":"2024-02-23T00:52:00.557638+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-02-23T00:51:39.253651+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-02-23T00:43:50.393713+0000","last_clean_scrub_stamp":"2024-02-23T00:51:39.253651+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":513,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.302839+0000","last_change":"2024-02-23T00:51:12.493007+0000","last_active":"2024-02-23T00:52:00.302839+0000","last_peered":"2024-02-23T00:52:00.302839+0000","last_clean":"2024-02-23T00:52:00.302839+0000","last_became_active":"2024-02-23T00:47:28.234641+0000","last_became_peered":"2024-02-23T00:47:28.234641+0000","last_unstale":"2024-02-23T00:52:00.302839+0000","last_undegraded":"2024-02-23T00:52:00.302839+0000","last_fullsized":"2024-02-23T00:52:00.302839+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.492942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.492942+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"457'31","reported_seq":621,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.557691+0000","last_change":"2024-02-23T00:51:15.408847+0000","last_active":"2024-02-23T00:52:00.557691+0000","last_peered":"2024-02-23T00:52:00.557691+0000","last_clean":"2024-02-23T00:52:00.557691+0000","last_became_active":"2024-02-23T00:39:16.782660+0000","last_became_peered":"2024-02-23T00:39:16.782660+0000","last_unstale":"2024-02-23T00:52:00.557691+0000","last_undegraded":"2024-02-23T00:52:00.557691+0000","last_fullsized":"2024-02-23T00:52:00.557691+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:51:15.408782+0000","last_deep_scrub":"93'23","last_deep_scrub_stamp":"2024-02-23T00:41:35.954516+0000","last_clean_scrub_stamp":"2024-02-23T00:51:15.408782+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":566,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.294744+0000","last_change":"2024-02-23T00:50:29.029794+0000","last_active":"2024-02-23T00:52:00.294744+0000","last_peered":"2024-02-23T00:52:00.294744+0000","last_clean":"2024-02-23T00:52:00.294744+0000","last_became_active":"2024-02-23T00:39:14.781560+0000","last_became_peered":"2024-02-23T00:39:14.781560+0000","last_unstale":"2024-02-23T00:52:00.294744+0000","last_undegraded":"2024-02-23T00:52:00.294744+0000","last_fullsized":"2024-02-23T00:52:00.294744+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:29.029725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:29.029725+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":573,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.304026+0000","last_change":"2024-02-23T00:51:14.488316+0000","last_active":"2024-02-23T00:52:00.304026+0000","last_peered":"2024-02-23T00:52:00.304026+0000","last_clean":"2024-02-23T00:52:00.304026+0000","last_became_active":"2024-02-23T00:47:28.232797+0000","last_became_peered":"2024-02-23T00:47:28.232797+0000","last_unstale":"2024-02-23T00:52:00.304026+0000","last_undegraded":"2024-02-23T00:52:00.304026+0000","last_fullsized":"2024-02-23T00:52:00.304026+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:14.488205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:53.189881+0000","last_clean_scrub_stamp":"2024-02-23T00:51:14.488205+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":507,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.294833+0000","last_change":"2024-02-23T00:51:00.232343+0000","last_active":"2024-02-23T00:52:00.294833+0000","last_peered":"2024-02-23T00:52:00.294833+0000","last_clean":"2024-02-23T00:52:00.294833+0000","last_became_active":"2024-02-23T00:42:58.774720+0000","last_became_peered":"2024-02-23T00:42:58.774720+0000","last_unstale":"2024-02-23T00:52:00.294833+0000","last_undegraded":"2024-02-23T00:52:00.294833+0000","last_fullsized":"2024-02-23T00:52:00.294833+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.232261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.232261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"457'27","reported_seq":297,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.501640+0000","last_change":"2024-02-23T00:49:26.518048+0000","last_active":"2024-02-23T00:52:00.501640+0000","last_peered":"2024-02-23T00:52:00.501640+0000","last_clean":"2024-02-23T00:52:00.501640+0000","last_became_active":"2024-02-23T00:47:28.211342+0000","last_became_peered":"2024-02-23T00:47:28.211342+0000","last_unstale":"2024-02-23T00:52:00.501640+0000","last_undegraded":"2024-02-23T00:52:00.501640+0000","last_fullsized":"2024-02-23T00:52:00.501640+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-23T00:49:26.517965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:26.517965+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":566,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.555384+0000","last_change":"2024-02-23T00:51:08.235807+0000","last_active":"2024-02-23T00:52:00.555384+0000","last_peered":"2024-02-23T00:52:00.555384+0000","last_clean":"2024-02-23T00:52:00.555384+0000","last_became_active":"2024-02-23T00:39:14.782494+0000","last_became_peered":"2024-02-23T00:39:14.782494+0000","last_unstale":"2024-02-23T00:52:00.555384+0000","last_undegraded":"2024-02-23T00:52:00.555384+0000","last_fullsized":"2024-02-23T00:52:00.555384+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:08.235735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:08.235735+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":547,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.505257+0000","last_change":"2024-02-23T00:48:16.788875+0000","last_active":"2024-02-23T00:52:00.505257+0000","last_peered":"2024-02-23T00:52:00.505257+0000","last_clean":"2024-02-23T00:52:00.505257+0000","last_became_active":"2024-02-23T00:39:18.801039+0000","last_became_peered":"2024-02-23T00:39:18.801039+0000","last_unstale":"2024-02-23T00:52:00.505257+0000","last_undegraded":"2024-02-23T00:52:00.505257+0000","last_fullsized":"2024-02-23T00:52:00.505257+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:16.788806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.641689+0000","last_clean_scrub_stamp":"2024-02-23T00:48:16.788806+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":507,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.295572+0000","last_change":"2024-02-23T00:51:10.341831+0000","last_active":"2024-02-23T00:52:00.295572+0000","last_peered":"2024-02-23T00:52:00.295572+0000","last_clean":"2024-02-23T00:52:00.295572+0000","last_became_active":"2024-02-23T00:42:58.770759+0000","last_became_peered":"2024-02-23T00:42:58.770759+0000","last_unstale":"2024-02-23T00:52:00.295572+0000","last_undegraded":"2024-02-23T00:52:00.295572+0000","last_fullsized":"2024-02-23T00:52:00.295572+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:10.341768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:10.341768+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"457'24","reported_seq":611,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.295868+0000","last_change":"2024-02-23T00:50:13.861288+0000","last_active":"2024-02-23T00:52:00.295868+0000","last_peered":"2024-02-23T00:52:00.295868+0000","last_clean":"2024-02-23T00:52:00.295868+0000","last_became_active":"2024-02-23T00:39:16.782803+0000","last_became_peered":"2024-02-23T00:39:16.782803+0000","last_unstale":"2024-02-23T00:52:00.295868+0000","last_undegraded":"2024-02-23T00:52:00.295868+0000","last_fullsized":"2024-02-23T00:52:00.295868+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'24","last_scrub_stamp":"2024-02-23T00:50:13.861220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:13.861220+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":46,"num_read_kb":36,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":566,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.295392+0000","last_change":"2024-02-23T00:50:28.060186+0000","last_active":"2024-02-23T00:52:00.295392+0000","last_peered":"2024-02-23T00:52:00.295392+0000","last_clean":"2024-02-23T00:52:00.295392+0000","last_became_active":"2024-02-23T00:39:14.782163+0000","last_became_peered":"2024-02-23T00:39:14.782163+0000","last_unstale":"2024-02-23T00:52:00.295392+0000","last_undegraded":"2024-02-23T00:52:00.295392+0000","last_fullsized":"2024-02-23T00:52:00.295392+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:28.060109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:28.060109+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":566,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.306857+0000","last_change":"2024-02-23T00:50:18.629287+0000","last_active":"2024-02-23T00:52:00.306857+0000","last_peered":"2024-02-23T00:52:00.306857+0000","last_clean":"2024-02-23T00:52:00.306857+0000","last_became_active":"2024-02-23T00:47:28.215452+0000","last_became_peered":"2024-02-23T00:47:28.215452+0000","last_unstale":"2024-02-23T00:52:00.306857+0000","last_undegraded":"2024-02-23T00:52:00.306857+0000","last_fullsized":"2024-02-23T00:52:00.306857+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:18.629203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_clean_scrub_stamp":"2024-02-23T00:50:18.629203+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":512,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.437420+0000","last_change":"2024-02-23T00:50:56.774466+0000","last_active":"2024-02-23T00:52:00.437420+0000","last_peered":"2024-02-23T00:52:00.437420+0000","last_clean":"2024-02-23T00:52:00.437420+0000","last_became_active":"2024-02-23T00:47:28.203555+0000","last_became_peered":"2024-02-23T00:47:28.203555+0000","last_unstale":"2024-02-23T00:52:00.437420+0000","last_undegraded":"2024-02-23T00:52:00.437420+0000","last_fullsized":"2024-02-23T00:52:00.437420+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:56.774382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:56.774382+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"457'33","reported_seq":622,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.305687+0000","last_change":"2024-02-23T00:49:46.364498+0000","last_active":"2024-02-23T00:52:00.305687+0000","last_peered":"2024-02-23T00:52:00.305687+0000","last_clean":"2024-02-23T00:52:00.305687+0000","last_became_active":"2024-02-23T00:39:16.786791+0000","last_became_peered":"2024-02-23T00:39:16.786791+0000","last_unstale":"2024-02-23T00:52:00.305687+0000","last_undegraded":"2024-02-23T00:52:00.305687+0000","last_fullsized":"2024-02-23T00:52:00.305687+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'33","last_scrub_stamp":"2024-02-23T00:49:46.364418+0000","last_deep_scrub":"95'27","last_deep_scrub_stamp":"2024-02-23T00:41:55.919895+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.364418+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":573,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.397776+0000","last_change":"2024-02-23T00:51:42.249052+0000","last_active":"2024-02-23T00:52:00.397776+0000","last_peered":"2024-02-23T00:52:00.397776+0000","last_clean":"2024-02-23T00:52:00.397776+0000","last_became_active":"2024-02-23T00:47:28.208293+0000","last_became_peered":"2024-02-23T00:47:28.208293+0000","last_unstale":"2024-02-23T00:52:00.397776+0000","last_undegraded":"2024-02-23T00:52:00.397776+0000","last_fullsized":"2024-02-23T00:52:00.397776+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:42.248982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:42.248982+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":558,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.439072+0000","last_change":"2024-02-23T00:49:46.612538+0000","last_active":"2024-02-23T00:52:00.439072+0000","last_peered":"2024-02-23T00:52:00.439072+0000","last_clean":"2024-02-23T00:52:00.439072+0000","last_became_active":"2024-02-23T00:39:18.796431+0000","last_became_peered":"2024-02-23T00:39:18.796431+0000","last_unstale":"2024-02-23T00:52:00.439072+0000","last_undegraded":"2024-02-23T00:52:00.439072+0000","last_fullsized":"2024-02-23T00:52:00.439072+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:46.612461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.612461+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":503,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.439310+0000","last_change":"2024-02-23T00:50:26.630228+0000","last_active":"2024-02-23T00:52:00.439310+0000","last_peered":"2024-02-23T00:52:00.439310+0000","last_clean":"2024-02-23T00:52:00.439310+0000","last_became_active":"2024-02-23T00:42:58.832885+0000","last_became_peered":"2024-02-23T00:42:58.832885+0000","last_unstale":"2024-02-23T00:52:00.439310+0000","last_undegraded":"2024-02-23T00:52:00.439310+0000","last_fullsized":"2024-02-23T00:52:00.439310+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:26.630088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:26.630088+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"457'17","reported_seq":591,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.307931+0000","last_change":"2024-02-23T00:50:15.590233+0000","last_active":"2024-02-23T00:52:00.307931+0000","last_peered":"2024-02-23T00:52:00.307931+0000","last_clean":"2024-02-23T00:52:00.307931+0000","last_became_active":"2024-02-23T00:39:16.782762+0000","last_became_peered":"2024-02-23T00:39:16.782762+0000","last_unstale":"2024-02-23T00:52:00.307931+0000","last_undegraded":"2024-02-23T00:52:00.307931+0000","last_fullsized":"2024-02-23T00:52:00.307931+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'17","last_scrub_stamp":"2024-02-23T00:50:15.590096+0000","last_deep_scrub":"82'15","last_deep_scrub_stamp":"2024-02-23T00:41:50.975683+0000","last_clean_scrub_stamp":"2024-02-23T00:50:15.590096+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":21,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":564,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.399462+0000","last_change":"2024-02-23T00:51:08.199826+0000","last_active":"2024-02-23T00:52:00.399462+0000","last_peered":"2024-02-23T00:52:00.399462+0000","last_clean":"2024-02-23T00:52:00.399462+0000","last_became_active":"2024-02-23T00:39:14.783469+0000","last_became_peered":"2024-02-23T00:39:14.783469+0000","last_unstale":"2024-02-23T00:52:00.399462+0000","last_undegraded":"2024-02-23T00:52:00.399462+0000","last_fullsized":"2024-02-23T00:52:00.399462+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:08.199749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:08.199749+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":562,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.399476+0000","last_change":"2024-02-23T00:51:13.264970+0000","last_active":"2024-02-23T00:52:00.399476+0000","last_peered":"2024-02-23T00:52:00.399476+0000","last_clean":"2024-02-23T00:52:00.399476+0000","last_became_active":"2024-02-23T00:39:18.796952+0000","last_became_peered":"2024-02-23T00:39:18.796952+0000","last_unstale":"2024-02-23T00:52:00.399476+0000","last_undegraded":"2024-02-23T00:52:00.399476+0000","last_fullsized":"2024-02-23T00:52:00.399476+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:13.264868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:06.491793+0000","last_clean_scrub_stamp":"2024-02-23T00:51:13.264868+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":505,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.306228+0000","last_change":"2024-02-23T00:51:03.460376+0000","last_active":"2024-02-23T00:52:00.306228+0000","last_peered":"2024-02-23T00:52:00.306228+0000","last_clean":"2024-02-23T00:52:00.306228+0000","last_became_active":"2024-02-23T00:42:58.758147+0000","last_became_peered":"2024-02-23T00:42:58.758147+0000","last_unstale":"2024-02-23T00:52:00.306228+0000","last_undegraded":"2024-02-23T00:52:00.306228+0000","last_fullsized":"2024-02-23T00:52:00.306228+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:03.460302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:23.224663+0000","last_clean_scrub_stamp":"2024-02-23T00:51:03.460302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"457'21","reported_seq":599,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.304981+0000","last_change":"2024-02-23T00:49:32.434383+0000","last_active":"2024-02-23T00:52:00.304981+0000","last_peered":"2024-02-23T00:52:00.304981+0000","last_clean":"2024-02-23T00:52:00.304981+0000","last_became_active":"2024-02-23T00:39:16.784606+0000","last_became_peered":"2024-02-23T00:39:16.784606+0000","last_unstale":"2024-02-23T00:52:00.304981+0000","last_undegraded":"2024-02-23T00:52:00.304981+0000","last_fullsized":"2024-02-23T00:52:00.304981+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-23T00:49:32.434282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:32.434282+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":566,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.296973+0000","last_change":"2024-02-23T00:50:19.967076+0000","last_active":"2024-02-23T00:52:00.296973+0000","last_peered":"2024-02-23T00:52:00.296973+0000","last_clean":"2024-02-23T00:52:00.296973+0000","last_became_active":"2024-02-23T00:39:14.785362+0000","last_became_peered":"2024-02-23T00:39:14.785362+0000","last_unstale":"2024-02-23T00:52:00.296973+0000","last_undegraded":"2024-02-23T00:52:00.296973+0000","last_fullsized":"2024-02-23T00:52:00.296973+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:19.967009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:19.967009+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":547,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.502236+0000","last_change":"2024-02-23T00:49:37.537279+0000","last_active":"2024-02-23T00:52:00.502236+0000","last_peered":"2024-02-23T00:52:00.502236+0000","last_clean":"2024-02-23T00:52:00.502236+0000","last_became_active":"2024-02-23T00:39:18.796312+0000","last_became_peered":"2024-02-23T00:39:18.796312+0000","last_unstale":"2024-02-23T00:52:00.502236+0000","last_undegraded":"2024-02-23T00:52:00.502236+0000","last_fullsized":"2024-02-23T00:52:00.502236+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:37.537211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:47.226331+0000","last_clean_scrub_stamp":"2024-02-23T00:49:37.537211+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":504,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.303917+0000","last_change":"2024-02-23T00:50:28.479578+0000","last_active":"2024-02-23T00:52:00.303917+0000","last_peered":"2024-02-23T00:52:00.303917+0000","last_clean":"2024-02-23T00:52:00.303917+0000","last_became_active":"2024-02-23T00:42:58.838257+0000","last_became_peered":"2024-02-23T00:42:58.838257+0000","last_unstale":"2024-02-23T00:52:00.303917+0000","last_undegraded":"2024-02-23T00:52:00.303917+0000","last_fullsized":"2024-02-23T00:52:00.303917+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:28.479512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:50:28.479512+0000","last_clean_scrub_stamp":"2024-02-23T00:50:28.479512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"457'62","reported_seq":711,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.297149+0000","last_change":"2024-02-23T00:50:58.151569+0000","last_active":"2024-02-23T00:52:00.297149+0000","last_peered":"2024-02-23T00:52:00.297149+0000","last_clean":"2024-02-23T00:52:00.297149+0000","last_became_active":"2024-02-23T00:47:28.205150+0000","last_became_peered":"2024-02-23T00:47:28.205150+0000","last_unstale":"2024-02-23T00:52:00.297149+0000","last_undegraded":"2024-02-23T00:52:00.297149+0000","last_fullsized":"2024-02-23T00:52:00.297149+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'62","last_scrub_stamp":"2024-02-23T00:50:58.151494+0000","last_deep_scrub":"88'50","last_deep_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_clean_scrub_stamp":"2024-02-23T00:50:58.151494+0000","log_size":62,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":92,"num_read_kb":79,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":563,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.305505+0000","last_change":"2024-02-23T00:50:34.600693+0000","last_active":"2024-02-23T00:52:00.305505+0000","last_peered":"2024-02-23T00:52:00.305505+0000","last_clean":"2024-02-23T00:52:00.305505+0000","last_became_active":"2024-02-23T00:39:14.782000+0000","last_became_peered":"2024-02-23T00:39:14.782000+0000","last_unstale":"2024-02-23T00:52:00.305505+0000","last_undegraded":"2024-02-23T00:52:00.305505+0000","last_fullsized":"2024-02-23T00:52:00.305505+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:34.600592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_clean_scrub_stamp":"2024-02-23T00:50:34.600592+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":557,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.305094+0000","last_change":"2024-02-23T00:51:47.686009+0000","last_active":"2024-02-23T00:52:00.305094+0000","last_peered":"2024-02-23T00:52:00.305094+0000","last_clean":"2024-02-23T00:52:00.305094+0000","last_became_active":"2024-02-23T00:39:18.799723+0000","last_became_peered":"2024-02-23T00:39:18.799723+0000","last_unstale":"2024-02-23T00:52:00.305094+0000","last_undegraded":"2024-02-23T00:52:00.305094+0000","last_fullsized":"2024-02-23T00:52:00.305094+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:47.685918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:51:47.685918+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":503,"reported_epoch":591,"state":"active+clean","last_fresh":"2024-02-23T00:52:00.439293+0000","last_change":"2024-02-23T00:49:44.601162+0000","last_active":"2024-02-23T00:52:00.439293+0000","last_peered":"2024-02-23T00:52:00.439293+0000","last_clean":"2024-02-23T00:52:00.439293+0000","last_became_active":"2024-02-23T00:42:58.824573+0000","last_became_peered":"2024-02-23T00:42:58.824573+0000","last_unstale":"2024-02-23T00:52:00.439293+0000","last_undegraded":"2024-02-23T00:52:00.439293+0000","last_fullsized":"2024-02-23T00:52:00.439293+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:44.601086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:44.601086+0000","last_clean_scrub_stamp":"2024-02-23T00:49:44.601086+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0],"acting":[5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":229,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":24,"acting":24,"num_store_stats":6},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":12,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":36,"acting":36,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1508,"num_read_kb":1233,"num_write":895,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":1486,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":955,"ondisk_log_size":955,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":757,"num_read_kb":752,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234141,"num_pgs":86,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":299948,"kb_used_data":2724,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93445204,"statfs":{"total":95995035648,"available":95687888896,"internally_reserved":0,"allocated":2789376,"data_stored":1205383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.50600000000000001,"15min":0.53200000000000003},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":4.9420000000000002},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51500000000000001,"15min":0.52000000000000002},"min":{"1min":0.34699999999999998,"5min":0.32000000000000001,"15min":0.24299999999999999},"max":{"1min":0.82599999999999996,"5min":1.151,"15min":4.9729999999999999},"last":0.49299999999999999}]},{"osd":1,"last update":"Fri Feb 23 00:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.50800000000000001,"15min":0.61199999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.626,"5min":1.25,"15min":10.786},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52400000000000002,"15min":0.61899999999999999},"min":{"1min":0.38600000000000001,"5min":0.33900000000000002,"15min":0.28100000000000003},"max":{"1min":0.79300000000000004,"5min":1.107,"15min":11.885999999999999},"last":0.54000000000000004}]},{"osd":2,"last update":"Fri Feb 23 00:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.51700000000000002,"15min":0.55000000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.28999999999999998},"max":{"1min":0.64900000000000002,"5min":0.80600000000000005,"15min":5.0789999999999997},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52800000000000002,"15min":0.54500000000000004},"min":{"1min":0.38900000000000001,"5min":0.34200000000000003,"15min":0.316},"max":{"1min":0.72299999999999998,"5min":1.1180000000000001,"15min":5.1280000000000001},"last":0.47899999999999998}]},{"osd":3,"last update":"Fri Feb 23 00:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.52200000000000002,"15min":0.61799999999999999},"min":{"1min":0.33200000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.74399999999999999,"5min":1.095,"15min":15.712999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54700000000000004,"15min":0.61399999999999999},"min":{"1min":0.35799999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.879,"5min":0.879,"15min":9.4090000000000007},"last":0.53300000000000003}]},{"osd":4,"last update":"Fri Feb 23 00:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.439,"15min":0.439},"min":{"1min":0.30199999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.753,"5min":1.6819999999999999,"15min":2.0350000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.45500000000000002,"15min":0.48899999999999999},"min":{"1min":0.33600000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.68400000000000005,"5min":0.86699999999999999,"15min":11.236000000000001},"last":0.36299999999999999}]},{"osd":5,"last update":"Fri Feb 23 00:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47299999999999998,"15min":0.49199999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27000000000000002},"max":{"1min":0.625,"5min":0.73199999999999998,"15min":5.4000000000000004},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.50700000000000001,"15min":0.5},"min":{"1min":0.35499999999999998,"5min":0.29099999999999998,"15min":0.26700000000000002},"max":{"1min":0.67500000000000004,"5min":1.728,"15min":1.728},"last":0.38700000000000001}]},{"osd":6,"last update":"Fri Feb 23 00:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51200000000000001,"15min":0.51100000000000001},"min":{"1min":0.42099999999999999,"5min":0.32200000000000001,"15min":0.317},"max":{"1min":0.75700000000000001,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.50900000000000001,"15min":0.54900000000000004},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.626,"5min":0.70499999999999996,"15min":13.395},"last":0.39800000000000002}]}]},{"osd":6,"up_from":47,"seq":201863463076,"num_pgs":75,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308168,"kb_used_data":2752,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93436984,"statfs":{"total":95995035648,"available":95679471616,"internally_reserved":0,"allocated":2818048,"data_stored":1216017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:51:07 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.505,"15min":0.68200000000000005},"min":{"1min":0.22500000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":30.641999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.53600000000000003,"15min":0.69699999999999995},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.577,"5min":3.7360000000000002,"15min":29.292000000000002},"last":0.501}]},{"osd":1,"last update":"Fri Feb 23 00:51:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51500000000000001,"15min":0.68300000000000005},"min":{"1min":0.308,"5min":0.247,"15min":0.247},"max":{"1min":0.77100000000000002,"5min":0.77200000000000002,"15min":28.186},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.52900000000000003,"15min":0.86299999999999999},"min":{"1min":0.39400000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.752,"5min":1.083,"15min":33.104999999999997},"last":0.53800000000000003}]},{"osd":2,"last update":"Fri Feb 23 00:51:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52700000000000002,"15min":0.71399999999999997},"min":{"1min":0.36699999999999999,"5min":0.29399999999999998,"15min":0.27400000000000002},"max":{"1min":0.69599999999999995,"5min":1.1160000000000001,"15min":19.960000000000001},"last":0.439},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53400000000000003,"15min":0.64600000000000002},"min":{"1min":0.40400000000000003,"5min":0.316,"15min":0.29899999999999999},"max":{"1min":0.79000000000000004,"5min":1.665,"15min":8.5860000000000003},"last":0.49199999999999999}]},{"osd":3,"last update":"Fri Feb 23 00:51:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54300000000000004,"15min":0.626},"min":{"1min":0.40999999999999998,"5min":0.29199999999999998,"15min":0.28599999999999998},"max":{"1min":0.78000000000000003,"5min":0.78400000000000003,"15min":19.562000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56899999999999995,"15min":0.67000000000000004},"min":{"1min":0.42399999999999999,"5min":0.33500000000000002,"15min":0.33000000000000002},"max":{"1min":0.80700000000000005,"5min":2.6259999999999999,"15min":14.199},"last":0.41199999999999998}]},{"osd":4,"last update":"Fri Feb 23 00:51:07 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.44700000000000001,"15min":0.47999999999999998},"min":{"1min":0.317,"5min":0.245,"15min":0.218},"max":{"1min":0.61499999999999999,"5min":1.6220000000000001,"15min":11.109999999999999},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.45900000000000002,"15min":0.54800000000000004},"min":{"1min":0.32700000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.751,"5min":0.83099999999999996,"15min":16.734000000000002},"last":0.313}]},{"osd":5,"last update":"Fri Feb 23 00:51:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48299999999999998,"15min":0.51300000000000001},"min":{"1min":0.34499999999999997,"5min":0.308,"15min":0.249},"max":{"1min":0.72399999999999998,"5min":1.784,"15min":14.154999999999999},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.58499999999999996,"15min":0.55500000000000005},"min":{"1min":0.37,"5min":0.318,"15min":0.26400000000000001},"max":{"1min":0.68500000000000005,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.40300000000000002}]},{"osd":7,"last update":"Fri Feb 23 00:51:23 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.51000000000000001,"15min":0.53100000000000003},"min":{"1min":0.38600000000000001,"5min":0.29399999999999998,"15min":0.26800000000000002},"max":{"1min":0.63100000000000001,"5min":1.726,"15min":8.548},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53900000000000003,"15min":0.60699999999999998},"min":{"1min":0.40300000000000002,"5min":0.34200000000000003,"15min":0.309},"max":{"1min":0.71199999999999997,"5min":0.88500000000000001,"15min":15.618},"last":0.34599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575038,"num_pgs":76,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308184,"kb_used_data":2768,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93436968,"statfs":{"total":95995035648,"available":95679455232,"internally_reserved":0,"allocated":2834432,"data_stored":1217622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:51:52 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.39600000000000002,"15min":0.68400000000000005},"min":{"1min":0.27800000000000002,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.45900000000000002,"5min":6.202,"15min":78.361000000000004},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.371,"15min":0.44600000000000001},"min":{"1min":0.24199999999999999,"5min":0.216,"15min":0.191},"max":{"1min":0.625,"5min":0.69999999999999996,"15min":5.8849999999999998},"last":0.23499999999999999}]},{"osd":2,"last update":"Fri Feb 23 00:51:13 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.436,"15min":0.67100000000000004},"min":{"1min":0.24399999999999999,"5min":0.20499999999999999,"15min":0.188},"max":{"1min":2.3010000000000002,"5min":7.5789999999999997,"15min":48.606000000000002},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.40799999999999997,"15min":0.57599999999999996},"min":{"1min":0.29399999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.63500000000000001,"5min":1,"15min":23.911000000000001},"last":0.29899999999999999}]},{"osd":3,"last update":"Fri Feb 23 00:51:52 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40799999999999997,"15min":0.58899999999999997},"min":{"1min":0.25700000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.72499999999999998,"5min":0.82999999999999996,"15min":35.244999999999997},"last":0.373},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.47699999999999998,"15min":0.68300000000000005},"min":{"1min":0.29999999999999999,"5min":0.26900000000000002,"15min":0.248},"max":{"1min":0.71699999999999997,"5min":7.641,"15min":56.853999999999999},"last":0.40699999999999997}]},{"osd":4,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.63300000000000001,"15min":0.82299999999999995},"min":{"1min":0.40100000000000002,"5min":0.32700000000000001,"15min":0.29299999999999998},"max":{"1min":0.97999999999999998,"5min":7.6829999999999998,"15min":37.646999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.64700000000000002,"15min":0.70799999999999996},"min":{"1min":0.48099999999999998,"5min":0.35899999999999999,"15min":0.31900000000000001},"max":{"1min":0.91400000000000003,"5min":6.0519999999999996,"15min":9.6859999999999999},"last":0.60599999999999998}]},{"osd":5,"last update":"Fri Feb 23 00:51:35 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.65000000000000002,"15min":0.83299999999999996},"min":{"1min":0.35399999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.81999999999999995,"5min":7.742,"15min":48.792000000000002},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.625,"15min":0.71199999999999997},"min":{"1min":0.39100000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77900000000000003,"5min":0.92600000000000005,"15min":11.872999999999999},"last":0.621}]},{"osd":6,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.64100000000000001,"15min":0.82199999999999995},"min":{"1min":0.40000000000000002,"5min":0.37,"15min":0.318},"max":{"1min":0.94199999999999995,"5min":7.7130000000000001,"15min":40.741},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60399999999999998,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.38400000000000001,"15min":0.34000000000000002},"max":{"1min":0.82199999999999995,"5min":0.85999999999999999,"15min":6.0069999999999997},"last":0.56299999999999994}]},{"osd":7,"last update":"Fri Feb 23 00:51:35 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61899999999999999,"15min":0.71399999999999997},"min":{"1min":0.41799999999999998,"5min":0.40200000000000002,"15min":0.36699999999999999},"max":{"1min":0.83899999999999997,"5min":0.92900000000000005,"15min":20.225000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.66500000000000004,"15min":0.73899999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.95499999999999996,"5min":6.0949999999999998,"15min":20.254999999999999},"last":0.53500000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738564,"num_pgs":83,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":312276,"kb_used_data":2764,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93432876,"statfs":{"total":95995035648,"available":95675265024,"internally_reserved":0,"allocated":2830336,"data_stored":1217028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 23 00:51:53 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.35899999999999999,"15min":0.60499999999999998},"min":{"1min":0.246,"5min":0.224,"15min":0.19500000000000001},"max":{"1min":0.50900000000000001,"5min":0.81399999999999995,"15min":28.856999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.376,"5min":0.40400000000000003,"15min":0.63200000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.20799999999999999},"max":{"1min":0.53000000000000003,"5min":2.3650000000000002,"15min":20.577000000000002},"last":0.42899999999999999}]},{"osd":2,"last update":"Fri Feb 23 00:51:18 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.40400000000000003,"15min":0.60999999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.20499999999999999},"max":{"1min":0.61099999999999999,"5min":2.3889999999999998,"15min":13.863},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.35099999999999998,"15min":0.53800000000000003},"min":{"1min":0.252,"5min":0.223,"15min":0.19800000000000001},"max":{"1min":0.504,"5min":0.65000000000000002,"15min":10.455},"last":0.36399999999999999}]},{"osd":3,"last update":"Fri Feb 23 00:51:48 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.378,"15min":0.77300000000000002},"min":{"1min":0.248,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.51600000000000001,"5min":0.73099999999999998,"15min":35.566000000000003},"last":0.441},{"interface":"front","average":{"1min":0.377,"5min":0.435,"15min":0.76000000000000001},"min":{"1min":0.27800000000000002,"5min":0.27400000000000002,"15min":0.26100000000000001},"max":{"1min":0.57499999999999996,"5min":2.4049999999999998,"15min":20.702999999999999},"last":0.376}]},{"osd":4,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55900000000000005,"15min":0.752},"min":{"1min":0.40899999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.755,"5min":0.94699999999999995,"15min":10.584},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.61599999999999999,"15min":0.78100000000000003},"min":{"1min":0.42899999999999999,"5min":0.38100000000000001,"15min":0.35099999999999998},"max":{"1min":0.78900000000000003,"5min":2.5019999999999998,"15min":13.198},"last":0.61899999999999999}]},{"osd":5,"last update":"Fri Feb 23 00:51:37 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54500000000000004,"15min":0.71499999999999997},"min":{"1min":0.36699999999999999,"5min":0.33800000000000002,"15min":0.32200000000000001},"max":{"1min":0.72599999999999998,"5min":0.90800000000000003,"15min":10.397},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.60699999999999998,"15min":0.78100000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76800000000000002,"5min":0.92600000000000005,"15min":17.297000000000001},"last":0.56899999999999995}]},{"osd":6,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.58899999999999997,"15min":0.78000000000000003},"min":{"1min":0.39400000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.76400000000000001,"5min":0.94399999999999995,"15min":17.373999999999999},"last":0.628},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.61299999999999999,"15min":0.76200000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.36799999999999999},"max":{"1min":0.72199999999999998,"5min":2.556,"15min":13.973000000000001},"last":0.65700000000000003}]},{"osd":7,"last update":"Fri Feb 23 00:51:27 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.59099999999999997,"15min":0.77800000000000002},"min":{"1min":0.379,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83399999999999996,"5min":0.96099999999999997,"15min":17.399999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.63100000000000001,"15min":0.78100000000000003},"min":{"1min":0.46000000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.752,"5min":2.5289999999999999,"15min":16.427},"last":0.69099999999999995}]}]},{"osd":2,"up_from":0,"seq":77309411513,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":26,"seq":111669149874,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":312284,"kb_used_data":2772,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93432868,"statfs":{"total":95995035648,"available":95675256832,"internally_reserved":0,"allocated":2838528,"data_stored":1216406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.42999999999999999,"15min":0.53200000000000003},"min":{"1min":0.216,"5min":0.21199999999999999,"15min":0.20899999999999999},"max":{"1min":0.63600000000000001,"5min":8.3300000000000001,"15min":12.212999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.42099999999999999,"15min":0.60299999999999998},"min":{"1min":0.25,"5min":0.22800000000000001,"15min":0.21199999999999999},"max":{"1min":0.48599999999999999,"5min":2.9740000000000002,"15min":11.714},"last":0.48299999999999998}]},{"osd":1,"last update":"Fri Feb 23 00:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.40999999999999998,"15min":0.624},"min":{"1min":0.25900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.52100000000000002,"5min":3.056,"15min":11.742000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.433,"15min":0.59799999999999998},"min":{"1min":0.28199999999999997,"5min":0.24099999999999999,"15min":0.19900000000000001},"max":{"1min":0.71599999999999997,"5min":2.7709999999999999,"15min":8.2270000000000003},"last":1.601}]},{"osd":2,"last update":"Fri Feb 23 00:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.47699999999999998,"15min":0.68300000000000005},"min":{"1min":0.30499999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.53600000000000003,"5min":3.242,"15min":13.359999999999999},"last":1.5600000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.49099999999999999,"15min":0.66100000000000003},"min":{"1min":0.28100000000000003,"5min":0.27500000000000002,"15min":0.26100000000000001},"max":{"1min":0.68700000000000006,"5min":8.5820000000000007,"15min":21.122},"last":0.42899999999999999}]},{"osd":4,"last update":"Fri Feb 23 00:51:10 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.58899999999999997,"15min":0.755},"min":{"1min":0.40300000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":3.1419999999999999,"5min":3.1419999999999999,"15min":11.644},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.74199999999999999,"15min":0.753},"min":{"1min":0.36899999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":2.9020000000000001,"5min":12.430999999999999,"15min":12.430999999999999},"last":1.5329999999999999}]},{"osd":5,"last update":"Fri Feb 23 00:51:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.67800000000000005,"15min":0.68899999999999995},"min":{"1min":0.42899999999999999,"5min":0.37,"15min":0.35099999999999998},"max":{"1min":1,"5min":10.231999999999999,"15min":10.231999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.624,"15min":0.80000000000000004},"min":{"1min":0.41699999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.94999999999999996,"5min":3.1829999999999998,"15min":11.648},"last":0.83799999999999997}]},{"osd":6,"last update":"Fri Feb 23 00:51:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.629,"15min":0.755},"min":{"1min":0.377,"5min":0.377,"15min":0.35199999999999998},"max":{"1min":0.83799999999999997,"5min":2.9510000000000001,"15min":6.9420000000000002},"last":1.504},{"interface":"front","average":{"1min":0.629,"5min":0.73899999999999999,"15min":0.72299999999999998},"min":{"1min":0.442,"5min":0.376,"15min":0.376},"max":{"1min":0.82299999999999995,"5min":10.965999999999999,"15min":10.965999999999999},"last":0.72399999999999998}]},{"osd":7,"last update":"Fri Feb 23 00:51:34 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.66000000000000003,"15min":0.83299999999999996},"min":{"1min":0.441,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.1030000000000002,"5min":3.1030000000000002,"15min":11.676},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.66600000000000004,"15min":0.81000000000000005},"min":{"1min":0.501,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":2.8580000000000001,"5min":2.8580000000000001,"15min":7.9800000000000004},"last":1.4690000000000001}]}]},{"osd":4,"up_from":32,"seq":137438953646,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308172,"kb_used_data":2756,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93436980,"statfs":{"total":95995035648,"available":95679467520,"internally_reserved":0,"allocated":2822144,"data_stored":1216050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.504,"15min":0.52600000000000002},"min":{"1min":0.32200000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.65300000000000002,"5min":1.3819999999999999,"15min":3.5219999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.5,"5min":0.52400000000000002,"15min":0.58499999999999996},"min":{"1min":0.373,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.72799999999999998,"5min":1.8,"15min":8.6140000000000008},"last":0.38400000000000001}]},{"osd":1,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51600000000000001,"15min":0.57999999999999996},"min":{"1min":0.31900000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.70699999999999996,"5min":2.6600000000000001,"15min":9.6110000000000007},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.53500000000000003,"15min":0.58499999999999996},"min":{"1min":0.31900000000000001,"5min":0.26300000000000001,"15min":0.255},"max":{"1min":0.70099999999999996,"5min":2.6920000000000002,"15min":6.6950000000000003},"last":0.53700000000000003}]},{"osd":2,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.51900000000000002,"15min":0.56999999999999995},"min":{"1min":0.34699999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.65700000000000003,"5min":1.0960000000000001,"15min":3.3069999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54000000000000004,"15min":0.57099999999999995},"min":{"1min":0.33700000000000002,"5min":0.33600000000000002,"15min":0.32500000000000001},"max":{"1min":0.68700000000000006,"5min":1.0189999999999999,"15min":5.8440000000000003},"last":0.60999999999999999}]},{"osd":3,"last update":"Fri Feb 23 00:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.67300000000000004,"15min":0.63400000000000001},"min":{"1min":0.38500000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.71999999999999997,"5min":9.1280000000000001,"15min":9.1280000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.58199999999999996,"15min":0.60399999999999998},"min":{"1min":0.39200000000000002,"5min":0.34599999999999997,"15min":0.29699999999999999},"max":{"1min":0.73299999999999998,"5min":4.0220000000000002,"15min":4.1909999999999998},"last":0.40600000000000003}]},{"osd":5,"last update":"Fri Feb 23 00:51:38 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.45400000000000001,"15min":0.42099999999999999},"min":{"1min":0.31900000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.61899999999999999,"5min":1.694,"15min":1.694},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.51600000000000001,"15min":0.47999999999999998},"min":{"1min":0.34000000000000002,"5min":0.29499999999999998,"15min":0.28899999999999998},"max":{"1min":0.73899999999999999,"5min":1.663,"15min":1.663},"last":0.61699999999999999}]},{"osd":6,"last update":"Fri Feb 23 00:51:05 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.44900000000000001,"15min":0.435},"min":{"1min":0.34000000000000002,"5min":0.28399999999999997,"15min":0.253},"max":{"1min":0.67200000000000004,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47199999999999998,"15min":0.443},"min":{"1min":0.34699999999999998,"5min":0.28799999999999998,"15min":0.215},"max":{"1min":0.64100000000000001,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":0.52100000000000002}]},{"osd":7,"last update":"Fri Feb 23 00:51:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.49399999999999999,"15min":0.48699999999999999},"min":{"1min":0.33100000000000002,"5min":0.27600000000000002,"15min":0.254},"max":{"1min":0.70999999999999996,"5min":2.294,"15min":2.294},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56899999999999995,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.752,"5min":2.282,"15min":2.282},"last":0.54800000000000004}]}]},{"osd":5,"up_from":39,"seq":167503724713,"num_pgs":77,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308160,"kb_used_data":2744,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93436992,"statfs":{"total":95995035648,"available":95679479808,"internally_reserved":0,"allocated":2809856,"data_stored":1206860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:51:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.55300000000000005,"15min":0.61399999999999999},"min":{"1min":0.40400000000000003,"5min":0.253,"15min":0.246},"max":{"1min":0.67000000000000004,"5min":5.2220000000000004,"15min":23.881},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.57599999999999996,"15min":0.59399999999999997},"min":{"1min":0.38600000000000001,"5min":0.29099999999999998,"15min":0.27000000000000002},"max":{"1min":0.76500000000000001,"5min":3.9249999999999998,"15min":5.4649999999999999},"last":0.56699999999999995}]},{"osd":1,"last update":"Fri Feb 23 00:51:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54900000000000004,"15min":0.65600000000000003},"min":{"1min":0.35299999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.73899999999999999,"5min":3.9460000000000002,"15min":18.175999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.502,"5min":0.56899999999999995,"15min":0.628},"min":{"1min":0.40100000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.71599999999999997,"5min":3.976,"15min":9.0950000000000006},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Feb 23 00:51:32 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57899999999999996,"15min":0.71199999999999997},"min":{"1min":0.37,"5min":0.255,"15min":0.255},"max":{"1min":1.036,"5min":4.5899999999999999,"15min":22.282},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.58599999999999997,"15min":0.72599999999999998},"min":{"1min":0.34399999999999997,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.66900000000000004,"5min":3.8540000000000001,"15min":23.925000000000001},"last":0.68999999999999995}]},{"osd":3,"last update":"Fri Feb 23 00:51:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55800000000000005,"15min":0.621},"min":{"1min":0.35899999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.194,"5min":1.23,"15min":6.5730000000000004},"last":0.621},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.60199999999999998,"15min":0.65500000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.33500000000000002},"max":{"1min":0.70499999999999996,"5min":3.9980000000000002,"15min":8.1760000000000002},"last":0.65900000000000003}]},{"osd":4,"last update":"Fri Feb 23 00:51:32 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.48199999999999998,"15min":0.436},"min":{"1min":0.33000000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.60699999999999998,"5min":4.593,"15min":4.593},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.49199999999999999,"15min":0.47399999999999998},"min":{"1min":0.35799999999999998,"5min":0.27900000000000003,"15min":0.27300000000000002},"max":{"1min":1.169,"5min":1.5860000000000001,"15min":3.879},"last":0.44800000000000001}]},{"osd":6,"last update":"Fri Feb 23 00:51:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.51300000000000001,"15min":0.47999999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.27900000000000003},"max":{"1min":1.1299999999999999,"5min":4.6740000000000004,"15min":4.6740000000000004},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.48899999999999999,"15min":0.47999999999999998},"min":{"1min":0.33400000000000002,"5min":0.31,"15min":0.27100000000000002},"max":{"1min":0.71199999999999997,"5min":1.159,"15min":1.159},"last":0.63500000000000001}]},{"osd":7,"last update":"Fri Feb 23 00:51:26 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.39000000000000001,"5min":0.32500000000000001,"15min":0.30399999999999999},"max":{"1min":0.74099999999999999,"5min":1.302,"15min":7.0300000000000002},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52600000000000002,"15min":0.53900000000000003},"min":{"1min":0.36099999999999999,"5min":0.314,"15min":0.29099999999999998},"max":{"1min":0.64600000000000002,"5min":2.016,"15min":9.0579999999999998},"last":0.59599999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":229,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":229,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":229,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":229,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":229,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":229,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-23T00:52:06.391 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-02-23T00:52:06.391 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-23T00:52:06.392 INFO:tasks.thrashosds.thrasher:fixing pg num pool default.rgw.log 2024-02-23T00:52:06.392 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph pg dump --format=json 2024-02-23T00:52:06.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:05.351498+0000 mgr.y (mgr.14517) 534 : cluster [DBG] pgmap v898: 144 pgs: 144 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-23T00:52:06.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:06.065587+0000 mon.a (mon.0) 3102 : audit [INF] from='client.? 172.21.15.47:0/1538028832' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:52:06.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:06.251714+0000 mon.a (mon.0) 3103 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:52:06.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:06.256708+0000 mon.a (mon.0) 3104 : audit [INF] from='client.? 172.21.15.47:0/956675132' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi047-82465-94"}]': finished 2024-02-23T00:52:06.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:06.256762+0000 mon.a (mon.0) 3105 : audit [INF] from='client.? 172.21.15.47:0/1538028832' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:52:06.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:06.256810+0000 mon.a (mon.0) 3106 : cluster [DBG] osdmap e597: 8 total, 8 up, 7 in 2024-02-23T00:52:06.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:06.273901+0000 mon.c (mon.2) 332 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:06.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:06.274212+0000 mon.a (mon.0) 3107 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:06.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:06.274926+0000 mon.c (mon.2) 333 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:06.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:06.275180+0000 mon.a (mon.0) 3108 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:06.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:06.275614+0000 mon.c (mon.2) 334 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi047-82465-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:52:06.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:06.275868+0000 mon.a (mon.0) 3109 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi047-82465-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:52:06.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:05.351498+0000 mgr.y (mgr.14517) 534 : cluster [DBG] pgmap v898: 144 pgs: 144 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-23T00:52:06.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:06.065587+0000 mon.a (mon.0) 3102 : audit [INF] from='client.? 172.21.15.47:0/1538028832' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:52:06.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:06.251714+0000 mon.a (mon.0) 3103 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:52:06.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:06.256708+0000 mon.a (mon.0) 3104 : audit [INF] from='client.? 172.21.15.47:0/956675132' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi047-82465-94"}]': finished 2024-02-23T00:52:06.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:06.256762+0000 mon.a (mon.0) 3105 : audit [INF] from='client.? 172.21.15.47:0/1538028832' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:52:06.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:06.256810+0000 mon.a (mon.0) 3106 : cluster [DBG] osdmap e597: 8 total, 8 up, 7 in 2024-02-23T00:52:06.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:06.273901+0000 mon.c (mon.2) 332 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:06.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:06.274212+0000 mon.a (mon.0) 3107 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:06.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:06.274926+0000 mon.c (mon.2) 333 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:06.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:06.275180+0000 mon.a (mon.0) 3108 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:06.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:06.275614+0000 mon.c (mon.2) 334 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi047-82465-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:52:06.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:06.275868+0000 mon.a (mon.0) 3109 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi047-82465-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:52:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:05.351498+0000 mgr.y (mgr.14517) 534 : cluster [DBG] pgmap v898: 144 pgs: 144 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-23T00:52:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:06.065587+0000 mon.a (mon.0) 3102 : audit [INF] from='client.? 172.21.15.47:0/1538028832' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:52:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:06.251714+0000 mon.a (mon.0) 3103 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:52:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:06.256708+0000 mon.a (mon.0) 3104 : audit [INF] from='client.? 172.21.15.47:0/956675132' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi047-82465-94"}]': finished 2024-02-23T00:52:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:06.256762+0000 mon.a (mon.0) 3105 : audit [INF] from='client.? 172.21.15.47:0/1538028832' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:52:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:06.256810+0000 mon.a (mon.0) 3106 : cluster [DBG] osdmap e597: 8 total, 8 up, 7 in 2024-02-23T00:52:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:06.273901+0000 mon.c (mon.2) 332 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:06.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:06.274212+0000 mon.a (mon.0) 3107 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:06.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:06.274926+0000 mon.c (mon.2) 333 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:06.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:06.275180+0000 mon.a (mon.0) 3108 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:06.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:06.275614+0000 mon.c (mon.2) 334 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi047-82465-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:52:06.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:06.275868+0000 mon.a (mon.0) 3109 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi047-82465-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-23T00:52:07.355 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:52:07.357 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:07.358 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:52:07.459 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:52:07.492 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:07.513 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:52:07.515 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:07.516 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:52:07.584 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:06.310757+0000 mon.a (mon.0) 3110 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:52:07.584 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:06.370014+0000 osd.5 (osd.5) 254 : cluster [DBG] 5.15 scrub starts 2024-02-23T00:52:07.584 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:06.373456+0000 osd.5 (osd.5) 255 : cluster [DBG] 5.15 scrub ok 2024-02-23T00:52:07.584 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:06.733400+0000 osd.4 (osd.4) 245 : cluster [DBG] 2.f scrub starts 2024-02-23T00:52:07.584 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:06.734595+0000 osd.4 (osd.4) 246 : cluster [DBG] 2.f scrub ok 2024-02-23T00:52:07.584 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:07.257930+0000 mon.a (mon.0) 3111 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi047-82465-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:52:07.584 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:07.258011+0000 mon.a (mon.0) 3112 : cluster [DBG] osdmap e598: 8 total, 8 up, 7 in 2024-02-23T00:52:07.584 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:07.259816+0000 mon.c (mon.2) 335 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi047-82465-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:07.585 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:07.260096+0000 mon.a (mon.0) 3113 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi047-82465-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:07.585 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:06.310757+0000 mon.a (mon.0) 3110 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:52:07.585 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:06.370014+0000 osd.5 (osd.5) 254 : cluster [DBG] 5.15 scrub starts 2024-02-23T00:52:07.585 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:06.373456+0000 osd.5 (osd.5) 255 : cluster [DBG] 5.15 scrub ok 2024-02-23T00:52:07.585 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:06.733400+0000 osd.4 (osd.4) 245 : cluster [DBG] 2.f scrub starts 2024-02-23T00:52:07.585 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:06.734595+0000 osd.4 (osd.4) 246 : cluster [DBG] 2.f scrub ok 2024-02-23T00:52:07.585 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:07.257930+0000 mon.a (mon.0) 3111 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi047-82465-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:52:07.585 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:07.258011+0000 mon.a (mon.0) 3112 : cluster [DBG] osdmap e598: 8 total, 8 up, 7 in 2024-02-23T00:52:07.585 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:07.259816+0000 mon.c (mon.2) 335 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi047-82465-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:07.585 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:07.260096+0000 mon.a (mon.0) 3113 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi047-82465-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:07.617 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:52:07.643 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:06.310757+0000 mon.a (mon.0) 3110 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:52:07.644 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:06.370014+0000 osd.5 (osd.5) 254 : cluster [DBG] 5.15 scrub starts 2024-02-23T00:52:07.644 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:06.373456+0000 osd.5 (osd.5) 255 : cluster [DBG] 5.15 scrub ok 2024-02-23T00:52:07.644 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:06.733400+0000 osd.4 (osd.4) 245 : cluster [DBG] 2.f scrub starts 2024-02-23T00:52:07.644 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:06.734595+0000 osd.4 (osd.4) 246 : cluster [DBG] 2.f scrub ok 2024-02-23T00:52:07.644 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:07.257930+0000 mon.a (mon.0) 3111 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi047-82465-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-23T00:52:07.644 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:07.258011+0000 mon.a (mon.0) 3112 : cluster [DBG] osdmap e598: 8 total, 8 up, 7 in 2024-02-23T00:52:07.644 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:07.259816+0000 mon.c (mon.2) 335 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi047-82465-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:07.644 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:07.260096+0000 mon.a (mon.0) 3113 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi047-82465-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:07.645 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:07.665 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:52:07.667 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:07.668 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:52:07.769 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:52:07.796 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:07.816 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:52:07.818 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:07.818 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:52:07.900 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.0 dump_historic_ops 2024-02-23T00:52:07.909 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:52:07.919 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:52:07.946 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:07.965 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:52:07.967 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:07.968 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:52:08.068 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:52:08.095 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:08.170 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:52:08.173 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:08.173 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:52:08.274 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:52:08.304 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:08.504 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:52:08.507 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:08.507 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:52:08.608 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:52:08.639 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:08.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:07.208692+0000 osd.6 (osd.6) 227 : cluster [DBG] 5.11 scrub starts 2024-02-23T00:52:08.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:07.209622+0000 osd.6 (osd.6) 228 : cluster [DBG] 5.11 scrub ok 2024-02-23T00:52:08.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:07.352119+0000 mgr.y (mgr.14517) 535 : cluster [DBG] pgmap v901: 144 pgs: 144 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-23T00:52:08.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:07.421867+0000 osd.7 (osd.7) 283 : cluster [DBG] 6.8 scrub starts 2024-02-23T00:52:08.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:07.425304+0000 osd.7 (osd.7) 284 : cluster [DBG] 6.8 scrub ok 2024-02-23T00:52:08.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:07.721444+0000 osd.4 (osd.4) 247 : cluster [DBG] 5.e scrub starts 2024-02-23T00:52:08.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:07.722920+0000 osd.4 (osd.4) 248 : cluster [DBG] 5.e scrub ok 2024-02-23T00:52:08.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:07.208692+0000 osd.6 (osd.6) 227 : cluster [DBG] 5.11 scrub starts 2024-02-23T00:52:08.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:07.209622+0000 osd.6 (osd.6) 228 : cluster [DBG] 5.11 scrub ok 2024-02-23T00:52:08.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:07.352119+0000 mgr.y (mgr.14517) 535 : cluster [DBG] pgmap v901: 144 pgs: 144 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-23T00:52:08.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:07.421867+0000 osd.7 (osd.7) 283 : cluster [DBG] 6.8 scrub starts 2024-02-23T00:52:08.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:07.425304+0000 osd.7 (osd.7) 284 : cluster [DBG] 6.8 scrub ok 2024-02-23T00:52:08.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:07.721444+0000 osd.4 (osd.4) 247 : cluster [DBG] 5.e scrub starts 2024-02-23T00:52:08.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:07.722920+0000 osd.4 (osd.4) 248 : cluster [DBG] 5.e scrub ok 2024-02-23T00:52:08.996 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-23T00:52:09.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:07.208692+0000 osd.6 (osd.6) 227 : cluster [DBG] 5.11 scrub starts 2024-02-23T00:52:09.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:07.209622+0000 osd.6 (osd.6) 228 : cluster [DBG] 5.11 scrub ok 2024-02-23T00:52:09.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:07.352119+0000 mgr.y (mgr.14517) 535 : cluster [DBG] pgmap v901: 144 pgs: 144 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-23T00:52:09.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:07.421867+0000 osd.7 (osd.7) 283 : cluster [DBG] 6.8 scrub starts 2024-02-23T00:52:09.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:07.425304+0000 osd.7 (osd.7) 284 : cluster [DBG] 6.8 scrub ok 2024-02-23T00:52:09.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:07.721444+0000 osd.4 (osd.4) 247 : cluster [DBG] 5.e scrub starts 2024-02-23T00:52:09.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:07.722920+0000 osd.4 (osd.4) 248 : cluster [DBG] 5.e scrub ok 2024-02-23T00:52:09.375 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:52:09.989 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:52:09.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:07.533050+0000 osd.1 (osd.1) 247 : cluster [DBG] 6.1 scrub starts 2024-02-23T00:52:09.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:07.536284+0000 osd.1 (osd.1) 248 : cluster [DBG] 6.1 scrub ok 2024-02-23T00:52:09.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:07.715665+0000 osd.3 (osd.3) 278 : cluster [DBG] 5.14 scrub starts 2024-02-23T00:52:09.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:07.717181+0000 osd.3 (osd.3) 279 : cluster [DBG] 5.14 scrub ok 2024-02-23T00:52:09.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:08.246424+0000 osd.6 (osd.6) 229 : cluster [DBG] 2.11 scrub starts 2024-02-23T00:52:09.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:08.247253+0000 osd.6 (osd.6) 230 : cluster [DBG] 2.11 scrub ok 2024-02-23T00:52:09.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:08.662704+0000 osd.0 (osd.0) 247 : cluster [DBG] 5.3 scrub starts 2024-02-23T00:52:09.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:08.663859+0000 osd.0 (osd.0) 248 : cluster [DBG] 5.3 scrub ok 2024-02-23T00:52:09.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:08.675943+0000 osd.4 (osd.4) 249 : cluster [DBG] 4.d scrub starts 2024-02-23T00:52:09.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:08.677851+0000 osd.4 (osd.4) 250 : cluster [DBG] 4.d scrub ok 2024-02-23T00:52:09.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:08.700914+0000 mon.a (mon.0) 3114 : cluster [DBG] osdmap e599: 8 total, 8 up, 7 in 2024-02-23T00:52:09.992 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:08.713787+0000 osd.3 (osd.3) 280 : cluster [DBG] 4.11 scrub starts 2024-02-23T00:52:09.992 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:08.715545+0000 osd.3 (osd.3) 281 : cluster [DBG] 4.11 scrub ok 2024-02-23T00:52:09.992 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:09.261101+0000 mon.a (mon.0) 3115 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi047-82465-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:52:09.992 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:09.261182+0000 mon.a (mon.0) 3116 : cluster [DBG] osdmap e600: 8 total, 8 up, 7 in 2024-02-23T00:52:09.992 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:07.533050+0000 osd.1 (osd.1) 247 : cluster [DBG] 6.1 scrub starts 2024-02-23T00:52:09.992 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:07.536284+0000 osd.1 (osd.1) 248 : cluster [DBG] 6.1 scrub ok 2024-02-23T00:52:09.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:07.715665+0000 osd.3 (osd.3) 278 : cluster [DBG] 5.14 scrub starts 2024-02-23T00:52:09.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:07.717181+0000 osd.3 (osd.3) 279 : cluster [DBG] 5.14 scrub ok 2024-02-23T00:52:09.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:08.246424+0000 osd.6 (osd.6) 229 : cluster [DBG] 2.11 scrub starts 2024-02-23T00:52:09.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:08.247253+0000 osd.6 (osd.6) 230 : cluster [DBG] 2.11 scrub ok 2024-02-23T00:52:09.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:08.662704+0000 osd.0 (osd.0) 247 : cluster [DBG] 5.3 scrub starts 2024-02-23T00:52:09.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:08.663859+0000 osd.0 (osd.0) 248 : cluster [DBG] 5.3 scrub ok 2024-02-23T00:52:09.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:08.675943+0000 osd.4 (osd.4) 249 : cluster [DBG] 4.d scrub starts 2024-02-23T00:52:09.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:08.677851+0000 osd.4 (osd.4) 250 : cluster [DBG] 4.d scrub ok 2024-02-23T00:52:09.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:08.700914+0000 mon.a (mon.0) 3114 : cluster [DBG] osdmap e599: 8 total, 8 up, 7 in 2024-02-23T00:52:09.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:08.713787+0000 osd.3 (osd.3) 280 : cluster [DBG] 4.11 scrub starts 2024-02-23T00:52:09.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:08.715545+0000 osd.3 (osd.3) 281 : cluster [DBG] 4.11 scrub ok 2024-02-23T00:52:09.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:09.261101+0000 mon.a (mon.0) 3115 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi047-82465-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:52:09.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:09.261182+0000 mon.a (mon.0) 3116 : cluster [DBG] osdmap e600: 8 total, 8 up, 7 in 2024-02-23T00:52:09.994 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:09.994 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:52:10.095 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:52:10.124 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:10.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:07.533050+0000 osd.1 (osd.1) 247 : cluster [DBG] 6.1 scrub starts 2024-02-23T00:52:10.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:07.536284+0000 osd.1 (osd.1) 248 : cluster [DBG] 6.1 scrub ok 2024-02-23T00:52:10.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:07.715665+0000 osd.3 (osd.3) 278 : cluster [DBG] 5.14 scrub starts 2024-02-23T00:52:10.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:07.717181+0000 osd.3 (osd.3) 279 : cluster [DBG] 5.14 scrub ok 2024-02-23T00:52:10.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:08.246424+0000 osd.6 (osd.6) 229 : cluster [DBG] 2.11 scrub starts 2024-02-23T00:52:10.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:08.247253+0000 osd.6 (osd.6) 230 : cluster [DBG] 2.11 scrub ok 2024-02-23T00:52:10.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:08.662704+0000 osd.0 (osd.0) 247 : cluster [DBG] 5.3 scrub starts 2024-02-23T00:52:10.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:08.663859+0000 osd.0 (osd.0) 248 : cluster [DBG] 5.3 scrub ok 2024-02-23T00:52:10.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:08.675943+0000 osd.4 (osd.4) 249 : cluster [DBG] 4.d scrub starts 2024-02-23T00:52:10.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:08.677851+0000 osd.4 (osd.4) 250 : cluster [DBG] 4.d scrub ok 2024-02-23T00:52:10.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:08.700914+0000 mon.a (mon.0) 3114 : cluster [DBG] osdmap e599: 8 total, 8 up, 7 in 2024-02-23T00:52:10.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:08.713787+0000 osd.3 (osd.3) 280 : cluster [DBG] 4.11 scrub starts 2024-02-23T00:52:10.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:08.715545+0000 osd.3 (osd.3) 281 : cluster [DBG] 4.11 scrub ok 2024-02-23T00:52:10.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:09.261101+0000 mon.a (mon.0) 3115 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi047-82465-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:52:10.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:09.261182+0000 mon.a (mon.0) 3116 : cluster [DBG] osdmap e600: 8 total, 8 up, 7 in 2024-02-23T00:52:10.529 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:52:10.710 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:52:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:52:10] "GET /metrics HTTP/1.1" 200 192811 "" "Prometheus/2.33.4" 2024-02-23T00:52:11.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:09.352515+0000 mgr.y (mgr.14517) 536 : cluster [DBG] pgmap v904: 152 pgs: 8 unknown, 144 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:52:11.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:09.365340+0000 osd.5 (osd.5) 256 : cluster [DBG] 2.2 scrub starts 2024-02-23T00:52:11.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:09.366276+0000 osd.5 (osd.5) 257 : cluster [DBG] 2.2 scrub ok 2024-02-23T00:52:11.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:10.269487+0000 mon.a (mon.0) 3117 : cluster [DBG] osdmap e601: 8 total, 8 up, 7 in 2024-02-23T00:52:11.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:10.278524+0000 mon.c (mon.2) 336 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:52:11.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:10.279462+0000 mon.a (mon.0) 3118 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:52:11.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:09.352515+0000 mgr.y (mgr.14517) 536 : cluster [DBG] pgmap v904: 152 pgs: 8 unknown, 144 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:52:11.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:09.365340+0000 osd.5 (osd.5) 256 : cluster [DBG] 2.2 scrub starts 2024-02-23T00:52:11.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:09.366276+0000 osd.5 (osd.5) 257 : cluster [DBG] 2.2 scrub ok 2024-02-23T00:52:11.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:10.269487+0000 mon.a (mon.0) 3117 : cluster [DBG] osdmap e601: 8 total, 8 up, 7 in 2024-02-23T00:52:11.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:10.278524+0000 mon.c (mon.2) 336 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:52:11.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:10.279462+0000 mon.a (mon.0) 3118 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:52:11.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:09.352515+0000 mgr.y (mgr.14517) 536 : cluster [DBG] pgmap v904: 152 pgs: 8 unknown, 144 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:52:11.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:09.365340+0000 osd.5 (osd.5) 256 : cluster [DBG] 2.2 scrub starts 2024-02-23T00:52:11.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:09.366276+0000 osd.5 (osd.5) 257 : cluster [DBG] 2.2 scrub ok 2024-02-23T00:52:11.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:10.269487+0000 mon.a (mon.0) 3117 : cluster [DBG] osdmap e601: 8 total, 8 up, 7 in 2024-02-23T00:52:11.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:10.278524+0000 mon.c (mon.2) 336 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:52:11.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:10.279462+0000 mon.a (mon.0) 3118 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:52:12.005 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:10.286632+0000 osd.5 (osd.5) 258 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:52:12.006 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:10.286780+0000 osd.5 (osd.5) 259 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:52:12.006 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:10.426053+0000 osd.7 (osd.7) 285 : cluster [DBG] 6.5 scrub starts 2024-02-23T00:52:12.006 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:10.428941+0000 osd.7 (osd.7) 286 : cluster [DBG] 6.5 scrub ok 2024-02-23T00:52:12.006 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:10.561918+0000 osd.1 (osd.1) 249 : cluster [DBG] 5.1d scrub starts 2024-02-23T00:52:12.006 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:10.563129+0000 osd.1 (osd.1) 250 : cluster [DBG] 5.1d scrub ok 2024-02-23T00:52:12.006 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:11.270304+0000 mon.a (mon.0) 3119 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:52:12.006 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:11.270397+0000 mon.a (mon.0) 3120 : cluster [DBG] osdmap e602: 8 total, 8 up, 7 in 2024-02-23T00:52:12.006 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:11.293240+0000 mon.c (mon.2) 337 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-102", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:52:12.006 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:11.293524+0000 mon.a (mon.0) 3121 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-102", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:52:12.006 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:10.286632+0000 osd.5 (osd.5) 258 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:52:12.007 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:10.286780+0000 osd.5 (osd.5) 259 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:52:12.007 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:10.426053+0000 osd.7 (osd.7) 285 : cluster [DBG] 6.5 scrub starts 2024-02-23T00:52:12.007 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:10.428941+0000 osd.7 (osd.7) 286 : cluster [DBG] 6.5 scrub ok 2024-02-23T00:52:12.007 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:10.561918+0000 osd.1 (osd.1) 249 : cluster [DBG] 5.1d scrub starts 2024-02-23T00:52:12.007 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:10.563129+0000 osd.1 (osd.1) 250 : cluster [DBG] 5.1d scrub ok 2024-02-23T00:52:12.007 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:11.270304+0000 mon.a (mon.0) 3119 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:52:12.007 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:11.270397+0000 mon.a (mon.0) 3120 : cluster [DBG] osdmap e602: 8 total, 8 up, 7 in 2024-02-23T00:52:12.007 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:11.293240+0000 mon.c (mon.2) 337 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-102", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:52:12.008 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:11.293524+0000 mon.a (mon.0) 3121 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-102", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:52:12.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:10.286632+0000 osd.5 (osd.5) 258 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:52:12.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:10.286780+0000 osd.5 (osd.5) 259 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:52:12.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:10.426053+0000 osd.7 (osd.7) 285 : cluster [DBG] 6.5 scrub starts 2024-02-23T00:52:12.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:10.428941+0000 osd.7 (osd.7) 286 : cluster [DBG] 6.5 scrub ok 2024-02-23T00:52:12.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:10.561918+0000 osd.1 (osd.1) 249 : cluster [DBG] 5.1d scrub starts 2024-02-23T00:52:12.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:10.563129+0000 osd.1 (osd.1) 250 : cluster [DBG] 5.1d scrub ok 2024-02-23T00:52:12.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:11.270304+0000 mon.a (mon.0) 3119 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:52:12.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:11.270397+0000 mon.a (mon.0) 3120 : cluster [DBG] osdmap e602: 8 total, 8 up, 7 in 2024-02-23T00:52:12.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:11.293240+0000 mon.c (mon.2) 337 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-102", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:52:12.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:11.293524+0000 mon.a (mon.0) 3121 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-102", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:52:12.279 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:52:12.282 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:12.282 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:52:12.383 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:52:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:12.566 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:52:12.569 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:12.569 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:52:12.670 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:52:12.683 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:52:12.683 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:52:12.700 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:12.796 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:52:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:52:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:52:12.796 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:11.327286+0000 osd.5 (osd.5) 260 : cluster [DBG] 5.1b scrub starts 2024-02-23T00:52:12.796 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:11.328118+0000 osd.5 (osd.5) 261 : cluster [DBG] 5.1b scrub ok 2024-02-23T00:52:12.797 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:11.352850+0000 mgr.y (mgr.14517) 537 : cluster [DBG] pgmap v907: 184 pgs: 40 unknown, 144 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:52:12.797 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:11.693708+0000 osd.4 (osd.4) 251 : cluster [DBG] 4.1a scrub starts 2024-02-23T00:52:12.797 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:11.695330+0000 osd.4 (osd.4) 252 : cluster [DBG] 4.1a scrub ok 2024-02-23T00:52:12.797 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:12.275064+0000 mon.a (mon.0) 3122 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-102", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:52:12.797 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:12.275149+0000 mon.a (mon.0) 3123 : cluster [DBG] osdmap e603: 8 total, 8 up, 7 in 2024-02-23T00:52:12.797 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:12.278778+0000 mon.c (mon.2) 338 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-102"}]: dispatch 2024-02-23T00:52:12.797 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:12.279468+0000 mon.a (mon.0) 3124 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-102"}]: dispatch 2024-02-23T00:52:13.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:11.327286+0000 osd.5 (osd.5) 260 : cluster [DBG] 5.1b scrub starts 2024-02-23T00:52:13.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:11.328118+0000 osd.5 (osd.5) 261 : cluster [DBG] 5.1b scrub ok 2024-02-23T00:52:13.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:11.352850+0000 mgr.y (mgr.14517) 537 : cluster [DBG] pgmap v907: 184 pgs: 40 unknown, 144 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:52:13.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:11.693708+0000 osd.4 (osd.4) 251 : cluster [DBG] 4.1a scrub starts 2024-02-23T00:52:13.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:11.695330+0000 osd.4 (osd.4) 252 : cluster [DBG] 4.1a scrub ok 2024-02-23T00:52:13.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:12.275064+0000 mon.a (mon.0) 3122 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-102", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:52:13.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:12.275149+0000 mon.a (mon.0) 3123 : cluster [DBG] osdmap e603: 8 total, 8 up, 7 in 2024-02-23T00:52:13.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:12.278778+0000 mon.c (mon.2) 338 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-102"}]: dispatch 2024-02-23T00:52:13.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:12.279468+0000 mon.a (mon.0) 3124 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-102"}]: dispatch 2024-02-23T00:52:13.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:11.327286+0000 osd.5 (osd.5) 260 : cluster [DBG] 5.1b scrub starts 2024-02-23T00:52:13.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:11.328118+0000 osd.5 (osd.5) 261 : cluster [DBG] 5.1b scrub ok 2024-02-23T00:52:13.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:11.352850+0000 mgr.y (mgr.14517) 537 : cluster [DBG] pgmap v907: 184 pgs: 40 unknown, 144 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:52:13.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:11.693708+0000 osd.4 (osd.4) 251 : cluster [DBG] 4.1a scrub starts 2024-02-23T00:52:13.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:11.695330+0000 osd.4 (osd.4) 252 : cluster [DBG] 4.1a scrub ok 2024-02-23T00:52:13.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:12.275064+0000 mon.a (mon.0) 3122 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-102", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:52:13.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:12.275149+0000 mon.a (mon.0) 3123 : cluster [DBG] osdmap e603: 8 total, 8 up, 7 in 2024-02-23T00:52:13.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:12.278778+0000 mon.c (mon.2) 338 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-102"}]: dispatch 2024-02-23T00:52:13.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:12.279468+0000 mon.a (mon.0) 3124 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-102"}]: dispatch 2024-02-23T00:52:13.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.902+0000 7f2ac0112700 1 -- 172.21.15.47:0/8864708 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2ab80fd510 msgr2=0x7f2ab80f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:13.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.902+0000 7f2ac0112700 1 --2- 172.21.15.47:0/8864708 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2ab80fd510 0x7f2ab80f6d40 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 rx=0x7f2aac004870 tx=0x7f2aac009cb0).stop 2024-02-23T00:52:13.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.902+0000 7f2ac0112700 1 -- 172.21.15.47:0/8864708 shutdown_connections 2024-02-23T00:52:13.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.902+0000 7f2ac0112700 1 --2- 172.21.15.47:0/8864708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ab80f7400 0x7f2ab8103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:13.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.902+0000 7f2ac0112700 1 --2- 172.21.15.47:0/8864708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ab80fcbb0 0x7f2ab80fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:13.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.902+0000 7f2ac0112700 1 --2- 172.21.15.47:0/8864708 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2ab80fd510 0x7f2ab80f6d40 unknown :-1 s=CLOSED pgs=2750 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:13.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.902+0000 7f2ac0112700 1 -- 172.21.15.47:0/8864708 >> 172.21.15.47:0/8864708 conn(0x7f2ab80f2820 msgr2=0x7f2ab80f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:52:13.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.902+0000 7f2ac0112700 1 -- 172.21.15.47:0/8864708 shutdown_connections 2024-02-23T00:52:13.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.902+0000 7f2ac0112700 1 -- 172.21.15.47:0/8864708 wait complete. 2024-02-23T00:52:13.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.903+0000 7f2ac0112700 1 librados: starting msgr at 2024-02-23T00:52:13.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.903+0000 7f2ac0112700 1 librados: starting objecter 2024-02-23T00:52:13.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.903+0000 7f2ac0112700 1 Processor -- start 2024-02-23T00:52:13.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.903+0000 7f2ac0112700 1 -- start start 2024-02-23T00:52:13.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.903+0000 7f2ac0112700 1 librados: setting wanted keys 2024-02-23T00:52:13.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.903+0000 7f2ac0112700 1 librados: calling monclient init 2024-02-23T00:52:13.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.903+0000 7f2ac0112700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ab80f7400 0x7f2ab810e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:13.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.903+0000 7f2ac0112700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ab80fcbb0 0x7f2ab810e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:13.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.903+0000 7f2ac0112700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2ab80fd510 0x7f2ab81081f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:13.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.903+0000 7f2ac0112700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab81086e0 con 0x7f2ab80f7400 2024-02-23T00:52:13.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.903+0000 7f2ac0112700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab8108850 con 0x7f2ab80fcbb0 2024-02-23T00:52:13.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.903+0000 7f2ac0112700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab8108a10 con 0x7f2ab80fd510 2024-02-23T00:52:13.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.903+0000 7f2abdeae700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ab80f7400 0x7f2ab810e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:13.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.903+0000 7f2abe6af700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2ab80fd510 0x7f2ab81081f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:13.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.903+0000 7f2abdeae700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ab80f7400 0x7f2ab810e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44350/0 (socket says 172.21.15.47:44350) 2024-02-23T00:52:13.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.903+0000 7f2abdeae700 1 -- 172.21.15.47:0/2170833144 learned_addr learned my addr 172.21.15.47:0/2170833144 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:52:13.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.903+0000 7f2abe6af700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2ab80fd510 0x7f2ab81081f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:47906/0 (socket says 172.21.15.47:47906) 2024-02-23T00:52:13.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.904+0000 7f2abd6ad700 1 --2- 172.21.15.47:0/2170833144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ab80fcbb0 0x7f2ab810e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:13.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.904+0000 7f2abdeae700 1 -- 172.21.15.47:0/2170833144 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2ab80fd510 msgr2=0x7f2ab81081f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:13.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.904+0000 7f2abdeae700 1 --2- 172.21.15.47:0/2170833144 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2ab80fd510 0x7f2ab81081f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:13.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.904+0000 7f2abdeae700 1 -- 172.21.15.47:0/2170833144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ab80fcbb0 msgr2=0x7f2ab810e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:13.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.904+0000 7f2abdeae700 1 --2- 172.21.15.47:0/2170833144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ab80fcbb0 0x7f2ab810e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:13.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.904+0000 7f2abdeae700 1 -- 172.21.15.47:0/2170833144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aac005040 con 0x7f2ab80f7400 2024-02-23T00:52:13.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.904+0000 7f2abd6ad700 1 --2- 172.21.15.47:0/2170833144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ab80fcbb0 0x7f2ab810e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-23T00:52:13.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.904+0000 7f2abe6af700 1 --2- 172.21.15.47:0/2170833144 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2ab80fd510 0x7f2ab81081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:52:13.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.904+0000 7f2abdeae700 1 --2- 172.21.15.47:0/2170833144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ab80f7400 0x7f2ab810e050 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 rx=0x7f2ab400e970 tx=0x7f2ab400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:13.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.904+0000 7f2aaaffd700 10 client.?.objecter ms_handle_connect 0x7f2ab80f7400 2024-02-23T00:52:13.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.904+0000 7f2aaaffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:52:13.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.904+0000 7f2aaaffd700 1 -- 172.21.15.47:0/2170833144 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab400ee60 con 0x7f2ab80f7400 2024-02-23T00:52:13.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.904+0000 7f2ac0112700 1 -- 172.21.15.47:0/2170833144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab8108cf0 con 0x7f2ab80f7400 2024-02-23T00:52:13.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.904+0000 7f2aaaffd700 1 -- 172.21.15.47:0/2170833144 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f2ab400f040 con 0x7f2ab80f7400 2024-02-23T00:52:13.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.904+0000 7f2aaaffd700 1 -- 172.21.15.47:0/2170833144 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab400c5b0 con 0x7f2ab80f7400 2024-02-23T00:52:13.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.904+0000 7f2ac0112700 10 client.31458.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:52:13.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.904+0000 7f2ac0112700 1 -- 172.21.15.47:0/2170833144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab8109070 con 0x7f2ab80f7400 2024-02-23T00:52:13.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.904+0000 7f2ac0112700 1 librados: init done 2024-02-23T00:52:13.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.905+0000 7f2ac0112700 1 -- 172.21.15.47:0/2170833144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab805fa60 con 0x7f2ab80f7400 2024-02-23T00:52:13.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.906+0000 7f2aaaffd700 1 -- 172.21.15.47:0/2170833144 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f2ab4007b40 con 0x7f2ab80f7400 2024-02-23T00:52:13.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.906+0000 7f2aaaffd700 1 --2- 172.21.15.47:0/2170833144 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2aa4062cc0 0x7f2aa4065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:13.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.906+0000 7f2aaaffd700 1 -- 172.21.15.47:0/2170833144 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(603..603 src has 482..603) v4 ==== 9578+0+0 (secure 0 0 0) 0x7f2ab407d3f0 con 0x7f2ab80f7400 2024-02-23T00:52:13.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.906+0000 7f2aaaffd700 10 client.31458.objecter ms_dispatch 0x7f2ab8001550 osd_map(603..603 src has 482..603) v4 2024-02-23T00:52:13.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.906+0000 7f2aaaffd700 3 client.31458.objecter handle_osd_map got epochs [603,603] > 0 2024-02-23T00:52:13.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.906+0000 7f2aaaffd700 3 client.31458.objecter handle_osd_map decoding full epoch 603 2024-02-23T00:52:13.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.906+0000 7f2abd6ad700 1 --2- 172.21.15.47:0/2170833144 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2aa4062cc0 0x7f2aa4065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:13.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.906+0000 7f2aaaffd700 20 client.31458.objecter dump_active .. 0 homeless 2024-02-23T00:52:13.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.906+0000 7f2abd6ad700 1 --2- 172.21.15.47:0/2170833144 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2aa4062cc0 0x7f2aa4065180 secure :-1 s=READY pgs=3139 cs=0 l=1 rev1=1 rx=0x7f2aac009da0 tx=0x7f2aac00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:13.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.906+0000 7f2aaaffd700 10 client.31458.objecter ms_handle_connect 0x7f2aa4062cc0 2024-02-23T00:52:13.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:12.910+0000 7f2aaaffd700 1 -- 172.21.15.47:0/2170833144 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f2ab404af60 con 0x7f2ab80f7400 2024-02-23T00:52:13.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:13.080+0000 7f2ac0112700 1 -- 172.21.15.47:0/2170833144 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2ab81096d0 con 0x7f2aa4062cc0 2024-02-23T00:52:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:13.093+0000 7f2aaaffd700 1 -- 172.21.15.47:0/2170833144 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+477072 (secure 0 0 0) 0x7f2ab81096d0 con 0x7f2aa4062cc0 2024-02-23T00:52:13.103 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:52:13.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:13.097+0000 7f2ac0112700 10 librados: watch_flush enter 2024-02-23T00:52:13.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:13.097+0000 7f2ac0112700 10 librados: watch_flush exit 2024-02-23T00:52:13.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:13.097+0000 7f2ac0112700 20 client.31458.objecter shutdown clearing up homeless session... 2024-02-23T00:52:13.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:13.097+0000 7f2ac0112700 10 client.31458.objecter successfully canceled tick 2024-02-23T00:52:13.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:13.097+0000 7f2ac0112700 1 -- 172.21.15.47:0/2170833144 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2aa4062cc0 msgr2=0x7f2aa4065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:13.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:13.097+0000 7f2ac0112700 1 --2- 172.21.15.47:0/2170833144 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2aa4062cc0 0x7f2aa4065180 secure :-1 s=READY pgs=3139 cs=0 l=1 rev1=1 rx=0x7f2aac009da0 tx=0x7f2aac00b040).stop 2024-02-23T00:52:13.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:13.097+0000 7f2ac0112700 1 -- 172.21.15.47:0/2170833144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ab80f7400 msgr2=0x7f2ab810e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:13.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:13.097+0000 7f2ac0112700 1 --2- 172.21.15.47:0/2170833144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ab80f7400 0x7f2ab810e050 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 rx=0x7f2ab400e970 tx=0x7f2ab400b040).stop 2024-02-23T00:52:13.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:13.097+0000 7f2ac0112700 1 -- 172.21.15.47:0/2170833144 shutdown_connections 2024-02-23T00:52:13.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:13.097+0000 7f2ac0112700 1 --2- 172.21.15.47:0/2170833144 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2aa4062cc0 0x7f2aa4065180 unknown :-1 s=CLOSED pgs=3139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:13.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:13.097+0000 7f2ac0112700 1 --2- 172.21.15.47:0/2170833144 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2ab80fd510 0x7f2ab81081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:13.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:13.097+0000 7f2ac0112700 1 --2- 172.21.15.47:0/2170833144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ab80f7400 0x7f2ab810e050 unknown :-1 s=CLOSED pgs=2592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:13.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:13.097+0000 7f2ac0112700 1 --2- 172.21.15.47:0/2170833144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ab80fcbb0 0x7f2ab810e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:13.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:13.097+0000 7f2ac0112700 1 -- 172.21.15.47:0/2170833144 >> 172.21.15.47:0/2170833144 conn(0x7f2ab80f2820 msgr2=0x7f2ab8060e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:52:13.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:13.097+0000 7f2ac0112700 1 -- 172.21.15.47:0/2170833144 shutdown_connections 2024-02-23T00:52:13.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:13.097+0000 7f2ac0112700 1 -- 172.21.15.47:0/2170833144 wait complete. 2024-02-23T00:52:13.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:13.097+0000 7f2ac0112700 1 librados: shutdown 2024-02-23T00:52:13.109 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-02-23T00:52:13.194 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:52:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:52:13.820 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":908,"stamp":"2024-02-23T00:52:12.275720+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5810,"num_objects":227,"num_object_clones":0,"num_object_copies":681,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":227,"num_whiteouts":0,"num_read":2303,"num_read_kb":2023,"num_write":915,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":2626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1047,"ondisk_log_size":1047,"up":552,"acting":552,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":486,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":2162500,"kb_used_data":20492,"kb_used_omap":0,"kb_used_meta":2141952,"kb_avail":654053564,"statfs":{"total":671965249536,"available":669750849536,"internally_reserved":0,"allocated":20983808,"data_stored":9212362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2193358848},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.018514"},"pg_stats":[{"pgid":"235.1f","version":"0'0","reported_seq":9,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.287636+0000","last_change":"2024-02-23T00:52:11.287636+0000","last_active":"2024-02-23T00:52:11.287636+0000","last_peered":"2024-02-23T00:52:11.287636+0000","last_clean":"2024-02-23T00:52:11.287636+0000","last_became_active":"2024-02-23T00:52:11.287308+0000","last_became_peered":"2024-02-23T00:52:11.287308+0000","last_unstale":"2024-02-23T00:52:11.287636+0000","last_undegraded":"2024-02-23T00:52:11.287636+0000","last_fullsized":"2024-02-23T00:52:11.287636+0000","mapping_epoch":601,"log_start":"0'0","ondisk_log_start":"0'0","created":601,"last_epoch_clean":602,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.258798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"235.1e","version":"0'0","reported_seq":9,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.285229+0000","last_change":"2024-02-23T00:52:11.285229+0000","last_active":"2024-02-23T00:52:11.285229+0000","last_peered":"2024-02-23T00:52:11.285229+0000","last_clean":"2024-02-23T00:52:11.285229+0000","last_became_active":"2024-02-23T00:52:11.284856+0000","last_became_peered":"2024-02-23T00:52:11.284856+0000","last_unstale":"2024-02-23T00:52:11.285229+0000","last_undegraded":"2024-02-23T00:52:11.285229+0000","last_fullsized":"2024-02-23T00:52:11.285229+0000","mapping_epoch":601,"log_start":"0'0","ondisk_log_start":"0'0","created":601,"last_epoch_clean":602,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.258798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"235.1d","version":"602'1","reported_seq":10,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.291424+0000","last_change":"2024-02-23T00:52:11.291085+0000","last_active":"2024-02-23T00:52:11.291424+0000","last_peered":"2024-02-23T00:52:11.291424+0000","last_clean":"2024-02-23T00:52:11.291424+0000","last_became_active":"2024-02-23T00:52:11.290776+0000","last_became_peered":"2024-02-23T00:52:11.290776+0000","last_unstale":"2024-02-23T00:52:11.291424+0000","last_undegraded":"2024-02-23T00:52:11.291424+0000","last_fullsized":"2024-02-23T00:52:11.291424+0000","mapping_epoch":601,"log_start":"0'0","ondisk_log_start":"0'0","created":601,"last_epoch_clean":602,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.258798+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"235.1c","version":"0'0","reported_seq":9,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.282329+0000","last_change":"2024-02-23T00:52:11.282329+0000","last_active":"2024-02-23T00:52:11.282329+0000","last_peered":"2024-02-23T00:52:11.282329+0000","last_clean":"2024-02-23T00:52:11.282329+0000","last_became_active":"2024-02-23T00:52:11.281934+0000","last_became_peered":"2024-02-23T00:52:11.281934+0000","last_unstale":"2024-02-23T00:52:11.282329+0000","last_undegraded":"2024-02-23T00:52:11.282329+0000","last_fullsized":"2024-02-23T00:52:11.282329+0000","mapping_epoch":601,"log_start":"0'0","ondisk_log_start":"0'0","created":601,"last_epoch_clean":602,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.258798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"235.1b","version":"0'0","reported_seq":9,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.282417+0000","last_change":"2024-02-23T00:52:11.282417+0000","last_active":"2024-02-23T00:52:11.282417+0000","last_peered":"2024-02-23T00:52:11.282417+0000","last_clean":"2024-02-23T00:52:11.282417+0000","last_became_active":"2024-02-23T00:52:11.282112+0000","last_became_peered":"2024-02-23T00:52:11.282112+0000","last_unstale":"2024-02-23T00:52:11.282417+0000","last_undegraded":"2024-02-23T00:52:11.282417+0000","last_fullsized":"2024-02-23T00:52:11.282417+0000","mapping_epoch":601,"log_start":"0'0","ondisk_log_start":"0'0","created":601,"last_epoch_clean":602,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.258798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"235.1a","version":"0'0","reported_seq":9,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.287804+0000","last_change":"2024-02-23T00:52:11.287804+0000","last_active":"2024-02-23T00:52:11.287804+0000","last_peered":"2024-02-23T00:52:11.287804+0000","last_clean":"2024-02-23T00:52:11.287804+0000","last_became_active":"2024-02-23T00:52:11.287484+0000","last_became_peered":"2024-02-23T00:52:11.287484+0000","last_unstale":"2024-02-23T00:52:11.287804+0000","last_undegraded":"2024-02-23T00:52:11.287804+0000","last_fullsized":"2024-02-23T00:52:11.287804+0000","mapping_epoch":601,"log_start":"0'0","ondisk_log_start":"0'0","created":601,"last_epoch_clean":602,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.258798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"235.19","version":"0'0","reported_seq":9,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.287629+0000","last_change":"2024-02-23T00:52:11.287629+0000","last_active":"2024-02-23T00:52:11.287629+0000","last_peered":"2024-02-23T00:52:11.287629+0000","last_clean":"2024-02-23T00:52:11.287629+0000","last_became_active":"2024-02-23T00:52:11.287260+0000","last_became_peered":"2024-02-23T00:52:11.287260+0000","last_unstale":"2024-02-23T00:52:11.287629+0000","last_undegraded":"2024-02-23T00:52:11.287629+0000","last_fullsized":"2024-02-23T00:52:11.287629+0000","mapping_epoch":601,"log_start":"0'0","ondisk_log_start":"0'0","created":601,"last_epoch_clean":602,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.258798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"235.18","version":"0'0","reported_seq":9,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.283889+0000","last_change":"2024-02-23T00:52:11.283889+0000","last_active":"2024-02-23T00:52:11.283889+0000","last_peered":"2024-02-23T00:52:11.283889+0000","last_clean":"2024-02-23T00:52:11.283889+0000","last_became_active":"2024-02-23T00:52:11.283633+0000","last_became_peered":"2024-02-23T00:52:11.283633+0000","last_unstale":"2024-02-23T00:52:11.283889+0000","last_undegraded":"2024-02-23T00:52:11.283889+0000","last_fullsized":"2024-02-23T00:52:11.283889+0000","mapping_epoch":601,"log_start":"0'0","ondisk_log_start":"0'0","created":601,"last_epoch_clean":602,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.258798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"235.17","version":"0'0","reported_seq":9,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.291332+0000","last_change":"2024-02-23T00:52:11.291332+0000","last_active":"2024-02-23T00:52:11.291332+0000","last_peered":"2024-02-23T00:52:11.291332+0000","last_clean":"2024-02-23T00:52:11.291332+0000","last_became_active":"2024-02-23T00:52:11.290539+0000","last_became_peered":"2024-02-23T00:52:11.290539+0000","last_unstale":"2024-02-23T00:52:11.291332+0000","last_undegraded":"2024-02-23T00:52:11.291332+0000","last_fullsized":"2024-02-23T00:52:11.291332+0000","mapping_epoch":601,"log_start":"0'0","ondisk_log_start":"0'0","created":601,"last_epoch_clean":602,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.258798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"235.16","version":"0'0","reported_seq":9,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.288907+0000","last_change":"2024-02-23T00:52:11.288907+0000","last_active":"2024-02-23T00:52:11.288907+0000","last_peered":"2024-02-23T00:52:11.288907+0000","last_clean":"2024-02-23T00:52:11.288907+0000","last_became_active":"2024-02-23T00:52:11.288560+0000","last_became_peered":"2024-02-23T00:52:11.288560+0000","last_unstale":"2024-02-23T00:52:11.288907+0000","last_undegraded":"2024-02-23T00:52:11.288907+0000","last_fullsized":"2024-02-23T00:52:11.288907+0000","mapping_epoch":601,"log_start":"0'0","ondisk_log_start":"0'0","created":601,"last_epoch_clean":602,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.258798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"235.15","version":"0'0","reported_seq":9,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.285213+0000","last_change":"2024-02-23T00:52:11.285213+0000","last_active":"2024-02-23T00:52:11.285213+0000","last_peered":"2024-02-23T00:52:11.285213+0000","last_clean":"2024-02-23T00:52:11.285213+0000","last_became_active":"2024-02-23T00:52:11.284910+0000","last_became_peered":"2024-02-23T00:52:11.284910+0000","last_unstale":"2024-02-23T00:52:11.285213+0000","last_undegraded":"2024-02-23T00:52:11.285213+0000","last_fullsized":"2024-02-23T00:52:11.285213+0000","mapping_epoch":601,"log_start":"0'0","ondisk_log_start":"0'0","created":601,"last_epoch_clean":602,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.258798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"235.14","version":"0'0","reported_seq":9,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.287837+0000","last_change":"2024-02-23T00:52:11.287837+0000","last_active":"2024-02-23T00:52:11.287837+0000","last_peered":"2024-02-23T00:52:11.287837+0000","last_clean":"2024-02-23T00:52:11.287837+0000","last_became_active":"2024-02-23T00:52:11.287122+0000","last_became_peered":"2024-02-23T00:52:11.287122+0000","last_unstale":"2024-02-23T00:52:11.287837+0000","last_undegraded":"2024-02-23T00:52:11.287837+0000","last_fullsized":"2024-02-23T00:52:11.287837+0000","mapping_epoch":601,"log_start":"0'0","ondisk_log_start":"0'0","created":601,"last_epoch_clean":602,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.258798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"235.13","version":"0'0","reported_seq":9,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.286848+0000","last_change":"2024-02-23T00:52:11.286848+0000","last_active":"2024-02-23T00:52:11.286848+0000","last_peered":"2024-02-23T00:52:11.286848+0000","last_clean":"2024-02-23T00:52:11.286848+0000","last_became_active":"2024-02-23T00:52:11.286519+0000","last_became_peered":"2024-02-23T00:52:11.286519+0000","last_unstale":"2024-02-23T00:52:11.286848+0000","last_undegraded":"2024-02-23T00:52:11.286848+0000","last_fullsized":"2024-02-23T00:52:11.286848+0000","mapping_epoch":601,"log_start":"0'0","ondisk_log_start":"0'0","created":601,"last_epoch_clean":602,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.258798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"235.12","version":"0'0","reported_seq":9,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.282825+0000","last_change":"2024-02-23T00:52:11.282825+0000","last_active":"2024-02-23T00:52:11.282825+0000","last_peered":"2024-02-23T00:52:11.282825+0000","last_clean":"2024-02-23T00:52:11.282825+0000","last_became_active":"2024-02-23T00:52:11.282462+0000","last_became_peered":"2024-02-23T00:52:11.282462+0000","last_unstale":"2024-02-23T00:52:11.282825+0000","last_undegraded":"2024-02-23T00:52:11.282825+0000","last_fullsized":"2024-02-23T00:52:11.282825+0000","mapping_epoch":601,"log_start":"0'0","ondisk_log_start":"0'0","created":601,"last_epoch_clean":602,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.258798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"235.11","version":"0'0","reported_seq":9,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.279896+0000","last_change":"2024-02-23T00:52:11.279896+0000","last_active":"2024-02-23T00:52:11.279896+0000","last_peered":"2024-02-23T00:52:11.279896+0000","last_clean":"2024-02-23T00:52:11.279896+0000","last_became_active":"2024-02-23T00:52:11.279606+0000","last_became_peered":"2024-02-23T00:52:11.279606+0000","last_unstale":"2024-02-23T00:52:11.279896+0000","last_undegraded":"2024-02-23T00:52:11.279896+0000","last_fullsized":"2024-02-23T00:52:11.279896+0000","mapping_epoch":601,"log_start":"0'0","ondisk_log_start":"0'0","created":601,"last_epoch_clean":602,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.258798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"235.10","version":"0'0","reported_seq":9,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.284009+0000","last_change":"2024-02-23T00:52:11.284009+0000","last_active":"2024-02-23T00:52:11.284009+0000","last_peered":"2024-02-23T00:52:11.284009+0000","last_clean":"2024-02-23T00:52:11.284009+0000","last_became_active":"2024-02-23T00:52:11.283801+0000","last_became_peered":"2024-02-23T00:52:11.283801+0000","last_unstale":"2024-02-23T00:52:11.284009+0000","last_undegraded":"2024-02-23T00:52:11.284009+0000","last_fullsized":"2024-02-23T00:52:11.284009+0000","mapping_epoch":601,"log_start":"0'0","ondisk_log_start":"0'0","created":601,"last_epoch_clean":602,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.258798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"235.f","version":"0'0","reported_seq":9,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.291517+0000","last_change":"2024-02-23T00:52:11.291517+0000","last_active":"2024-02-23T00:52:11.291517+0000","last_peered":"2024-02-23T00:52:11.291517+0000","last_clean":"2024-02-23T00:52:11.291517+0000","last_became_active":"2024-02-23T00:52:11.291006+0000","last_became_peered":"2024-02-23T00:52:11.291006+0000","last_unstale":"2024-02-23T00:52:11.291517+0000","last_undegraded":"2024-02-23T00:52:11.291517+0000","last_fullsized":"2024-02-23T00:52:11.291517+0000","mapping_epoch":601,"log_start":"0'0","ondisk_log_start":"0'0","created":601,"last_epoch_clean":602,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.258798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"235.e","version":"0'0","reported_seq":9,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.286148+0000","last_change":"2024-02-23T00:52:11.286148+0000","last_active":"2024-02-23T00:52:11.286148+0000","last_peered":"2024-02-23T00:52:11.286148+0000","last_clean":"2024-02-23T00:52:11.286148+0000","last_became_active":"2024-02-23T00:52:11.285821+0000","last_became_peered":"2024-02-23T00:52:11.285821+0000","last_unstale":"2024-02-23T00:52:11.286148+0000","last_undegraded":"2024-02-23T00:52:11.286148+0000","last_fullsized":"2024-02-23T00:52:11.286148+0000","mapping_epoch":601,"log_start":"0'0","ondisk_log_start":"0'0","created":601,"last_epoch_clean":602,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.258798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"235.d","version":"0'0","reported_seq":9,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.279584+0000","last_change":"2024-02-23T00:52:11.279584+0000","last_active":"2024-02-23T00:52:11.279584+0000","last_peered":"2024-02-23T00:52:11.279584+0000","last_clean":"2024-02-23T00:52:11.279584+0000","last_became_active":"2024-02-23T00:52:11.279264+0000","last_became_peered":"2024-02-23T00:52:11.279264+0000","last_unstale":"2024-02-23T00:52:11.279584+0000","last_undegraded":"2024-02-23T00:52:11.279584+0000","last_fullsized":"2024-02-23T00:52:11.279584+0000","mapping_epoch":601,"log_start":"0'0","ondisk_log_start":"0'0","created":601,"last_epoch_clean":602,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.258798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"235.c","version":"0'0","reported_seq":9,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.279917+0000","last_change":"2024-02-23T00:52:11.279917+0000","last_active":"2024-02-23T00:52:11.279917+0000","last_peered":"2024-02-23T00:52:11.279917+0000","last_clean":"2024-02-23T00:52:11.279917+0000","last_became_active":"2024-02-23T00:52:11.279422+0000","last_became_peered":"2024-02-23T00:52:11.279422+0000","last_unstale":"2024-02-23T00:52:11.279917+0000","last_undegraded":"2024-02-23T00:52:11.279917+0000","last_fullsized":"2024-02-23T00:52:11.279917+0000","mapping_epoch":601,"log_start":"0'0","ondisk_log_start":"0'0","created":601,"last_epoch_clean":602,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.258798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"235.b","version":"0'0","reported_seq":9,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.287262+0000","last_change":"2024-02-23T00:52:11.287262+0000","last_active":"2024-02-23T00:52:11.287262+0000","last_peered":"2024-02-23T00:52:11.287262+0000","last_clean":"2024-02-23T00:52:11.287262+0000","last_became_active":"2024-02-23T00:52:11.286960+0000","last_became_peered":"2024-02-23T00:52:11.286960+0000","last_unstale":"2024-02-23T00:52:11.287262+0000","last_undegraded":"2024-02-23T00:52:11.287262+0000","last_fullsized":"2024-02-23T00:52:11.287262+0000","mapping_epoch":601,"log_start":"0'0","ondisk_log_start":"0'0","created":601,"last_epoch_clean":602,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.258798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"235.a","version":"0'0","reported_seq":9,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.281055+0000","last_change":"2024-02-23T00:52:11.281055+0000","last_active":"2024-02-23T00:52:11.281055+0000","last_peered":"2024-02-23T00:52:11.281055+0000","last_clean":"2024-02-23T00:52:11.281055+0000","last_became_active":"2024-02-23T00:52:11.280729+0000","last_became_peered":"2024-02-23T00:52:11.280729+0000","last_unstale":"2024-02-23T00:52:11.281055+0000","last_undegraded":"2024-02-23T00:52:11.281055+0000","last_fullsized":"2024-02-23T00:52:11.281055+0000","mapping_epoch":601,"log_start":"0'0","ondisk_log_start":"0'0","created":601,"last_epoch_clean":602,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.258798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"235.9","version":"0'0","reported_seq":9,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.288530+0000","last_change":"2024-02-23T00:52:11.288530+0000","last_active":"2024-02-23T00:52:11.288530+0000","last_peered":"2024-02-23T00:52:11.288530+0000","last_clean":"2024-02-23T00:52:11.288530+0000","last_became_active":"2024-02-23T00:52:11.288223+0000","last_became_peered":"2024-02-23T00:52:11.288223+0000","last_unstale":"2024-02-23T00:52:11.288530+0000","last_undegraded":"2024-02-23T00:52:11.288530+0000","last_fullsized":"2024-02-23T00:52:11.288530+0000","mapping_epoch":601,"log_start":"0'0","ondisk_log_start":"0'0","created":601,"last_epoch_clean":602,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.258798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,5],"acting":[4,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"235.8","version":"0'0","reported_seq":9,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.286323+0000","last_change":"2024-02-23T00:52:11.286323+0000","last_active":"2024-02-23T00:52:11.286323+0000","last_peered":"2024-02-23T00:52:11.286323+0000","last_clean":"2024-02-23T00:52:11.286323+0000","last_became_active":"2024-02-23T00:52:11.286047+0000","last_became_peered":"2024-02-23T00:52:11.286047+0000","last_unstale":"2024-02-23T00:52:11.286323+0000","last_undegraded":"2024-02-23T00:52:11.286323+0000","last_fullsized":"2024-02-23T00:52:11.286323+0000","mapping_epoch":601,"log_start":"0'0","ondisk_log_start":"0'0","created":601,"last_epoch_clean":602,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.258798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,3],"acting":[1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"235.6","version":"0'0","reported_seq":9,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.292064+0000","last_change":"2024-02-23T00:52:11.292064+0000","last_active":"2024-02-23T00:52:11.292064+0000","last_peered":"2024-02-23T00:52:11.292064+0000","last_clean":"2024-02-23T00:52:11.292064+0000","last_became_active":"2024-02-23T00:52:11.290793+0000","last_became_peered":"2024-02-23T00:52:11.290793+0000","last_unstale":"2024-02-23T00:52:11.292064+0000","last_undegraded":"2024-02-23T00:52:11.292064+0000","last_fullsized":"2024-02-23T00:52:11.292064+0000","mapping_epoch":601,"log_start":"0'0","ondisk_log_start":"0'0","created":601,"last_epoch_clean":602,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.258798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"234.7","version":"0'0","reported_seq":10,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.282313+0000","last_change":"2024-02-23T00:52:10.280669+0000","last_active":"2024-02-23T00:52:10.282313+0000","last_peered":"2024-02-23T00:52:10.282313+0000","last_clean":"2024-02-23T00:52:10.282313+0000","last_became_active":"2024-02-23T00:52:10.280357+0000","last_became_peered":"2024-02-23T00:52:10.280357+0000","last_unstale":"2024-02-23T00:52:10.282313+0000","last_undegraded":"2024-02-23T00:52:10.282313+0000","last_fullsized":"2024-02-23T00:52:10.282313+0000","mapping_epoch":600,"log_start":"0'0","ondisk_log_start":"0'0","created":600,"last_epoch_clean":601,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_clean_scrub_stamp":"2024-02-23T00:52:09.256902+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"235.7","version":"0'0","reported_seq":9,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.290086+0000","last_change":"2024-02-23T00:52:11.290086+0000","last_active":"2024-02-23T00:52:11.290086+0000","last_peered":"2024-02-23T00:52:11.290086+0000","last_clean":"2024-02-23T00:52:11.290086+0000","last_became_active":"2024-02-23T00:52:11.289772+0000","last_became_peered":"2024-02-23T00:52:11.289772+0000","last_unstale":"2024-02-23T00:52:11.290086+0000","last_undegraded":"2024-02-23T00:52:11.290086+0000","last_fullsized":"2024-02-23T00:52:11.290086+0000","mapping_epoch":601,"log_start":"0'0","ondisk_log_start":"0'0","created":601,"last_epoch_clean":602,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.258798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"234.6","version":"0'0","reported_seq":10,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.284172+0000","last_change":"2024-02-23T00:52:10.283817+0000","last_active":"2024-02-23T00:52:10.284172+0000","last_peered":"2024-02-23T00:52:10.284172+0000","last_clean":"2024-02-23T00:52:10.284172+0000","last_became_active":"2024-02-23T00:52:10.283500+0000","last_became_peered":"2024-02-23T00:52:10.283500+0000","last_unstale":"2024-02-23T00:52:10.284172+0000","last_undegraded":"2024-02-23T00:52:10.284172+0000","last_fullsized":"2024-02-23T00:52:10.284172+0000","mapping_epoch":600,"log_start":"0'0","ondisk_log_start":"0'0","created":600,"last_epoch_clean":601,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_clean_scrub_stamp":"2024-02-23T00:52:09.256902+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"235.4","version":"0'0","reported_seq":9,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.279700+0000","last_change":"2024-02-23T00:52:11.279700+0000","last_active":"2024-02-23T00:52:11.279700+0000","last_peered":"2024-02-23T00:52:11.279700+0000","last_clean":"2024-02-23T00:52:11.279700+0000","last_became_active":"2024-02-23T00:52:11.279067+0000","last_became_peered":"2024-02-23T00:52:11.279067+0000","last_unstale":"2024-02-23T00:52:11.279700+0000","last_undegraded":"2024-02-23T00:52:11.279700+0000","last_fullsized":"2024-02-23T00:52:11.279700+0000","mapping_epoch":601,"log_start":"0'0","ondisk_log_start":"0'0","created":601,"last_epoch_clean":602,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.258798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"234.5","version":"602'1","reported_seq":11,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.269771+0000","last_change":"2024-02-23T00:52:10.283130+0000","last_active":"2024-02-23T00:52:11.269771+0000","last_peered":"2024-02-23T00:52:11.269771+0000","last_clean":"2024-02-23T00:52:11.269771+0000","last_became_active":"2024-02-23T00:52:10.282830+0000","last_became_peered":"2024-02-23T00:52:10.282830+0000","last_unstale":"2024-02-23T00:52:11.269771+0000","last_undegraded":"2024-02-23T00:52:11.269771+0000","last_fullsized":"2024-02-23T00:52:11.269771+0000","mapping_epoch":600,"log_start":"0'0","ondisk_log_start":"0'0","created":600,"last_epoch_clean":601,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_clean_scrub_stamp":"2024-02-23T00:52:09.256902+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"235.5","version":"0'0","reported_seq":9,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.287934+0000","last_change":"2024-02-23T00:52:11.287934+0000","last_active":"2024-02-23T00:52:11.287934+0000","last_peered":"2024-02-23T00:52:11.287934+0000","last_clean":"2024-02-23T00:52:11.287934+0000","last_became_active":"2024-02-23T00:52:11.287604+0000","last_became_peered":"2024-02-23T00:52:11.287604+0000","last_unstale":"2024-02-23T00:52:11.287934+0000","last_undegraded":"2024-02-23T00:52:11.287934+0000","last_fullsized":"2024-02-23T00:52:11.287934+0000","mapping_epoch":601,"log_start":"0'0","ondisk_log_start":"0'0","created":601,"last_epoch_clean":602,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.258798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"234.4","version":"0'0","reported_seq":10,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.282929+0000","last_change":"2024-02-23T00:52:10.282043+0000","last_active":"2024-02-23T00:52:10.282929+0000","last_peered":"2024-02-23T00:52:10.282929+0000","last_clean":"2024-02-23T00:52:10.282929+0000","last_became_active":"2024-02-23T00:52:10.281742+0000","last_became_peered":"2024-02-23T00:52:10.281742+0000","last_unstale":"2024-02-23T00:52:10.282929+0000","last_undegraded":"2024-02-23T00:52:10.282929+0000","last_fullsized":"2024-02-23T00:52:10.282929+0000","mapping_epoch":600,"log_start":"0'0","ondisk_log_start":"0'0","created":600,"last_epoch_clean":601,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_clean_scrub_stamp":"2024-02-23T00:52:09.256902+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"235.2","version":"0'0","reported_seq":9,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.281221+0000","last_change":"2024-02-23T00:52:11.281221+0000","last_active":"2024-02-23T00:52:11.281221+0000","last_peered":"2024-02-23T00:52:11.281221+0000","last_clean":"2024-02-23T00:52:11.281221+0000","last_became_active":"2024-02-23T00:52:11.280885+0000","last_became_peered":"2024-02-23T00:52:11.280885+0000","last_unstale":"2024-02-23T00:52:11.281221+0000","last_undegraded":"2024-02-23T00:52:11.281221+0000","last_fullsized":"2024-02-23T00:52:11.281221+0000","mapping_epoch":601,"log_start":"0'0","ondisk_log_start":"0'0","created":601,"last_epoch_clean":602,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.258798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"234.3","version":"0'0","reported_seq":10,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.281350+0000","last_change":"2024-02-23T00:52:10.280647+0000","last_active":"2024-02-23T00:52:10.281350+0000","last_peered":"2024-02-23T00:52:10.281350+0000","last_clean":"2024-02-23T00:52:10.281350+0000","last_became_active":"2024-02-23T00:52:10.280342+0000","last_became_peered":"2024-02-23T00:52:10.280342+0000","last_unstale":"2024-02-23T00:52:10.281350+0000","last_undegraded":"2024-02-23T00:52:10.281350+0000","last_fullsized":"2024-02-23T00:52:10.281350+0000","mapping_epoch":600,"log_start":"0'0","ondisk_log_start":"0'0","created":600,"last_epoch_clean":601,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_clean_scrub_stamp":"2024-02-23T00:52:09.256902+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"235.3","version":"0'0","reported_seq":9,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.287697+0000","last_change":"2024-02-23T00:52:11.287697+0000","last_active":"2024-02-23T00:52:11.287697+0000","last_peered":"2024-02-23T00:52:11.287697+0000","last_clean":"2024-02-23T00:52:11.287697+0000","last_became_active":"2024-02-23T00:52:11.287381+0000","last_became_peered":"2024-02-23T00:52:11.287381+0000","last_unstale":"2024-02-23T00:52:11.287697+0000","last_undegraded":"2024-02-23T00:52:11.287697+0000","last_fullsized":"2024-02-23T00:52:11.287697+0000","mapping_epoch":601,"log_start":"0'0","ondisk_log_start":"0'0","created":601,"last_epoch_clean":602,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.258798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"234.2","version":"0'0","reported_seq":10,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.284122+0000","last_change":"2024-02-23T00:52:10.283002+0000","last_active":"2024-02-23T00:52:10.284122+0000","last_peered":"2024-02-23T00:52:10.284122+0000","last_clean":"2024-02-23T00:52:10.284122+0000","last_became_active":"2024-02-23T00:52:10.282695+0000","last_became_peered":"2024-02-23T00:52:10.282695+0000","last_unstale":"2024-02-23T00:52:10.284122+0000","last_undegraded":"2024-02-23T00:52:10.284122+0000","last_fullsized":"2024-02-23T00:52:10.284122+0000","mapping_epoch":600,"log_start":"0'0","ondisk_log_start":"0'0","created":600,"last_epoch_clean":601,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_clean_scrub_stamp":"2024-02-23T00:52:09.256902+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7],"acting":[6,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"235.0","version":"0'0","reported_seq":9,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.282802+0000","last_change":"2024-02-23T00:52:11.282802+0000","last_active":"2024-02-23T00:52:11.282802+0000","last_peered":"2024-02-23T00:52:11.282802+0000","last_clean":"2024-02-23T00:52:11.282802+0000","last_became_active":"2024-02-23T00:52:11.282449+0000","last_became_peered":"2024-02-23T00:52:11.282449+0000","last_unstale":"2024-02-23T00:52:11.282802+0000","last_undegraded":"2024-02-23T00:52:11.282802+0000","last_fullsized":"2024-02-23T00:52:11.282802+0000","mapping_epoch":601,"log_start":"0'0","ondisk_log_start":"0'0","created":601,"last_epoch_clean":602,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.258798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"234.1","version":"0'0","reported_seq":10,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.282950+0000","last_change":"2024-02-23T00:52:10.280911+0000","last_active":"2024-02-23T00:52:10.282950+0000","last_peered":"2024-02-23T00:52:10.282950+0000","last_clean":"2024-02-23T00:52:10.282950+0000","last_became_active":"2024-02-23T00:52:10.280583+0000","last_became_peered":"2024-02-23T00:52:10.280583+0000","last_unstale":"2024-02-23T00:52:10.282950+0000","last_undegraded":"2024-02-23T00:52:10.282950+0000","last_fullsized":"2024-02-23T00:52:10.282950+0000","mapping_epoch":600,"log_start":"0'0","ondisk_log_start":"0'0","created":600,"last_epoch_clean":601,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_clean_scrub_stamp":"2024-02-23T00:52:09.256902+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"235.1","version":"0'0","reported_seq":9,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.287746+0000","last_change":"2024-02-23T00:52:11.287746+0000","last_active":"2024-02-23T00:52:11.287746+0000","last_peered":"2024-02-23T00:52:11.287746+0000","last_clean":"2024-02-23T00:52:11.287746+0000","last_became_active":"2024-02-23T00:52:11.287426+0000","last_became_peered":"2024-02-23T00:52:11.287426+0000","last_unstale":"2024-02-23T00:52:11.287746+0000","last_undegraded":"2024-02-23T00:52:11.287746+0000","last_fullsized":"2024-02-23T00:52:11.287746+0000","mapping_epoch":601,"log_start":"0'0","ondisk_log_start":"0'0","created":601,"last_epoch_clean":602,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:10.258798+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.258798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"234.0","version":"0'0","reported_seq":10,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.280781+0000","last_change":"2024-02-23T00:52:10.279921+0000","last_active":"2024-02-23T00:52:10.280781+0000","last_peered":"2024-02-23T00:52:10.280781+0000","last_clean":"2024-02-23T00:52:10.280781+0000","last_became_active":"2024-02-23T00:52:10.279573+0000","last_became_peered":"2024-02-23T00:52:10.279573+0000","last_unstale":"2024-02-23T00:52:10.280781+0000","last_undegraded":"2024-02-23T00:52:10.280781+0000","last_fullsized":"2024-02-23T00:52:10.280781+0000","mapping_epoch":600,"log_start":"0'0","ondisk_log_start":"0'0","created":600,"last_epoch_clean":601,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_clean_scrub_stamp":"2024-02-23T00:52:09.256902+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"466'9","reported_seq":731,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.273554+0000","last_change":"2024-02-23T00:49:43.188066+0000","last_active":"2024-02-23T00:52:10.273554+0000","last_peered":"2024-02-23T00:52:10.273554+0000","last_clean":"2024-02-23T00:52:10.273554+0000","last_became_active":"2024-02-23T00:39:18.793414+0000","last_became_peered":"2024-02-23T00:39:18.793414+0000","last_unstale":"2024-02-23T00:52:10.273554+0000","last_undegraded":"2024-02-23T00:52:10.273554+0000","last_fullsized":"2024-02-23T00:52:10.273554+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'9","last_scrub_stamp":"2024-02-23T00:49:43.187989+0000","last_deep_scrub":"466'9","last_deep_scrub_stamp":"2024-02-23T00:49:43.187989+0000","last_clean_scrub_stamp":"2024-02-23T00:49:43.187989+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"470'42","reported_seq":644,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.270595+0000","last_change":"2024-02-23T00:49:47.653272+0000","last_active":"2024-02-23T00:52:10.270595+0000","last_peered":"2024-02-23T00:52:10.270595+0000","last_clean":"2024-02-23T00:52:10.270595+0000","last_became_active":"2024-02-23T00:39:16.859401+0000","last_became_peered":"2024-02-23T00:39:16.859401+0000","last_unstale":"2024-02-23T00:52:10.270595+0000","last_undegraded":"2024-02-23T00:52:10.270595+0000","last_fullsized":"2024-02-23T00:52:10.270595+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"470'42","last_scrub_stamp":"2024-02-23T00:49:47.653176+0000","last_deep_scrub":"470'42","last_deep_scrub_stamp":"2024-02-23T00:49:47.653176+0000","last_clean_scrub_stamp":"2024-02-23T00:49:47.653176+0000","log_size":42,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":248,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.269972+0000","last_change":"2024-02-23T00:51:36.656632+0000","last_active":"2024-02-23T00:52:10.269972+0000","last_peered":"2024-02-23T00:52:10.269972+0000","last_clean":"2024-02-23T00:52:10.269972+0000","last_became_active":"2024-02-23T00:47:28.220876+0000","last_became_peered":"2024-02-23T00:47:28.220876+0000","last_unstale":"2024-02-23T00:52:10.269972+0000","last_undegraded":"2024-02-23T00:52:10.269972+0000","last_fullsized":"2024-02-23T00:52:10.269972+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:36.656571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:04.330426+0000","last_clean_scrub_stamp":"2024-02-23T00:51:36.656571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":582,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.270934+0000","last_change":"2024-02-23T00:50:14.801990+0000","last_active":"2024-02-23T00:52:10.270934+0000","last_peered":"2024-02-23T00:52:10.270934+0000","last_clean":"2024-02-23T00:52:10.270934+0000","last_became_active":"2024-02-23T00:43:32.282545+0000","last_became_peered":"2024-02-23T00:43:32.282545+0000","last_unstale":"2024-02-23T00:52:10.270934+0000","last_undegraded":"2024-02-23T00:52:10.270934+0000","last_fullsized":"2024-02-23T00:52:10.270934+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:14.801915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:50:14.801915+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":511,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.271800+0000","last_change":"2024-02-23T00:50:23.727290+0000","last_active":"2024-02-23T00:52:10.271800+0000","last_peered":"2024-02-23T00:52:10.271800+0000","last_clean":"2024-02-23T00:52:10.271800+0000","last_became_active":"2024-02-23T00:42:58.828846+0000","last_became_peered":"2024-02-23T00:42:58.828846+0000","last_unstale":"2024-02-23T00:52:10.271800+0000","last_undegraded":"2024-02-23T00:52:10.271800+0000","last_fullsized":"2024-02-23T00:52:10.271800+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:23.727173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:23.727173+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"457'21","reported_seq":615,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.273932+0000","last_change":"2024-02-23T00:51:46.661946+0000","last_active":"2024-02-23T00:52:10.273932+0000","last_peered":"2024-02-23T00:52:10.273932+0000","last_clean":"2024-02-23T00:52:10.273932+0000","last_became_active":"2024-02-23T00:39:16.783701+0000","last_became_peered":"2024-02-23T00:39:16.783701+0000","last_unstale":"2024-02-23T00:52:10.273932+0000","last_undegraded":"2024-02-23T00:52:10.273932+0000","last_fullsized":"2024-02-23T00:52:10.273932+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-23T00:51:46.661879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:46.661879+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":561,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.279524+0000","last_change":"2024-02-23T00:49:04.528117+0000","last_active":"2024-02-23T00:52:10.279524+0000","last_peered":"2024-02-23T00:52:10.279524+0000","last_clean":"2024-02-23T00:52:10.279524+0000","last_became_active":"2024-02-23T00:39:14.772493+0000","last_became_peered":"2024-02-23T00:39:14.772493+0000","last_unstale":"2024-02-23T00:52:10.279524+0000","last_undegraded":"2024-02-23T00:52:10.279524+0000","last_fullsized":"2024-02-23T00:52:10.279524+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:04.528006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:04.528006+0000","last_clean_scrub_stamp":"2024-02-23T00:49:04.528006+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":244,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.269723+0000","last_change":"2024-02-23T00:51:44.698156+0000","last_active":"2024-02-23T00:52:10.269723+0000","last_peered":"2024-02-23T00:52:10.269723+0000","last_clean":"2024-02-23T00:52:10.269723+0000","last_became_active":"2024-02-23T00:47:28.231567+0000","last_became_peered":"2024-02-23T00:47:28.231567+0000","last_unstale":"2024-02-23T00:52:10.269723+0000","last_undegraded":"2024-02-23T00:52:10.269723+0000","last_fullsized":"2024-02-23T00:52:10.269723+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:44.698067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:08.335330+0000","last_clean_scrub_stamp":"2024-02-23T00:51:44.698067+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":593,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.268149+0000","last_change":"2024-02-23T00:51:11.526671+0000","last_active":"2024-02-23T00:52:10.268149+0000","last_peered":"2024-02-23T00:52:10.268149+0000","last_clean":"2024-02-23T00:52:10.268149+0000","last_became_active":"2024-02-23T00:47:28.214157+0000","last_became_peered":"2024-02-23T00:47:28.214157+0000","last_unstale":"2024-02-23T00:52:10.268149+0000","last_undegraded":"2024-02-23T00:52:10.268149+0000","last_fullsized":"2024-02-23T00:52:10.268149+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:11.526584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:13.867716+0000","last_clean_scrub_stamp":"2024-02-23T00:51:11.526584+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":513,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.270078+0000","last_change":"2024-02-23T00:50:32.622097+0000","last_active":"2024-02-23T00:52:10.270078+0000","last_peered":"2024-02-23T00:52:10.270078+0000","last_clean":"2024-02-23T00:52:10.270078+0000","last_became_active":"2024-02-23T00:42:58.867305+0000","last_became_peered":"2024-02-23T00:42:58.867305+0000","last_unstale":"2024-02-23T00:52:10.270078+0000","last_undegraded":"2024-02-23T00:52:10.270078+0000","last_fullsized":"2024-02-23T00:52:10.270078+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:32.622026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:32.622026+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"457'51","reported_seq":654,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.281795+0000","last_change":"2024-02-23T00:51:13.326646+0000","last_active":"2024-02-23T00:52:10.281795+0000","last_peered":"2024-02-23T00:52:10.281795+0000","last_clean":"2024-02-23T00:52:10.281795+0000","last_became_active":"2024-02-23T00:39:16.857413+0000","last_became_peered":"2024-02-23T00:39:16.857413+0000","last_unstale":"2024-02-23T00:52:10.281795+0000","last_undegraded":"2024-02-23T00:52:10.281795+0000","last_fullsized":"2024-02-23T00:52:10.281795+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'51","last_scrub_stamp":"2024-02-23T00:51:13.326535+0000","last_deep_scrub":"88'35","last_deep_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_clean_scrub_stamp":"2024-02-23T00:51:13.326535+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":68,"num_read_kb":56,"num_write":42,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":243,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.275247+0000","last_change":"2024-02-23T00:50:57.770551+0000","last_active":"2024-02-23T00:52:10.275247+0000","last_peered":"2024-02-23T00:52:10.275247+0000","last_clean":"2024-02-23T00:52:10.275247+0000","last_became_active":"2024-02-23T00:47:28.231891+0000","last_became_peered":"2024-02-23T00:47:28.231891+0000","last_unstale":"2024-02-23T00:52:10.275247+0000","last_undegraded":"2024-02-23T00:52:10.275247+0000","last_fullsized":"2024-02-23T00:52:10.275247+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:57.770450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_clean_scrub_stamp":"2024-02-23T00:50:57.770450+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":245,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.273797+0000","last_change":"2024-02-23T00:51:36.650367+0000","last_active":"2024-02-23T00:52:10.273797+0000","last_peered":"2024-02-23T00:52:10.273797+0000","last_clean":"2024-02-23T00:52:10.273797+0000","last_became_active":"2024-02-23T00:47:28.220578+0000","last_became_peered":"2024-02-23T00:47:28.220578+0000","last_unstale":"2024-02-23T00:52:10.273797+0000","last_undegraded":"2024-02-23T00:52:10.273797+0000","last_fullsized":"2024-02-23T00:52:10.273797+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:36.650303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:03.682745+0000","last_clean_scrub_stamp":"2024-02-23T00:51:36.650303+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":594,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.272956+0000","last_change":"2024-02-23T00:52:07.425432+0000","last_active":"2024-02-23T00:52:10.272956+0000","last_peered":"2024-02-23T00:52:10.272956+0000","last_clean":"2024-02-23T00:52:10.272956+0000","last_became_active":"2024-02-23T00:47:28.215636+0000","last_became_peered":"2024-02-23T00:47:28.215636+0000","last_unstale":"2024-02-23T00:52:10.272956+0000","last_undegraded":"2024-02-23T00:52:10.272956+0000","last_fullsized":"2024-02-23T00:52:10.272956+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:07.425311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:42.150875+0000","last_clean_scrub_stamp":"2024-02-23T00:52:07.425311+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":503,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.275872+0000","last_change":"2024-02-23T00:49:34.515593+0000","last_active":"2024-02-23T00:52:10.275872+0000","last_peered":"2024-02-23T00:52:10.275872+0000","last_clean":"2024-02-23T00:52:10.275872+0000","last_became_active":"2024-02-23T00:42:57.639013+0000","last_became_peered":"2024-02-23T00:42:57.639013+0000","last_unstale":"2024-02-23T00:52:10.275872+0000","last_undegraded":"2024-02-23T00:52:10.275872+0000","last_fullsized":"2024-02-23T00:52:10.275872+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:34.515496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:49:34.515496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"457'31","reported_seq":672,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.270412+0000","last_change":"2024-02-23T00:52:08.677928+0000","last_active":"2024-02-23T00:52:10.270412+0000","last_peered":"2024-02-23T00:52:10.270412+0000","last_clean":"2024-02-23T00:52:10.270412+0000","last_became_active":"2024-02-23T00:47:28.200653+0000","last_became_peered":"2024-02-23T00:47:28.200653+0000","last_unstale":"2024-02-23T00:52:10.270412+0000","last_undegraded":"2024-02-23T00:52:10.270412+0000","last_fullsized":"2024-02-23T00:52:10.270412+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:52:08.677856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:52:08.677856+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":241,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.274243+0000","last_change":"2024-02-23T00:51:10.213423+0000","last_active":"2024-02-23T00:52:10.274243+0000","last_peered":"2024-02-23T00:52:10.274243+0000","last_clean":"2024-02-23T00:52:10.274243+0000","last_became_active":"2024-02-23T00:47:28.201926+0000","last_became_peered":"2024-02-23T00:47:28.201926+0000","last_unstale":"2024-02-23T00:52:10.274243+0000","last_undegraded":"2024-02-23T00:52:10.274243+0000","last_fullsized":"2024-02-23T00:52:10.274243+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:10.213337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:10.213337+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":562,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.276249+0000","last_change":"2024-02-23T00:51:09.343951+0000","last_active":"2024-02-23T00:52:10.276249+0000","last_peered":"2024-02-23T00:52:10.276249+0000","last_clean":"2024-02-23T00:52:10.276249+0000","last_became_active":"2024-02-23T00:39:18.799875+0000","last_became_peered":"2024-02-23T00:39:18.799875+0000","last_unstale":"2024-02-23T00:52:10.276249+0000","last_undegraded":"2024-02-23T00:52:10.276249+0000","last_fullsized":"2024-02-23T00:52:10.276249+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:09.343884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:09.343884+0000","last_clean_scrub_stamp":"2024-02-23T00:51:09.343884+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":503,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.276500+0000","last_change":"2024-02-23T00:48:49.610457+0000","last_active":"2024-02-23T00:52:10.276500+0000","last_peered":"2024-02-23T00:52:10.276500+0000","last_clean":"2024-02-23T00:52:10.276500+0000","last_became_active":"2024-02-23T00:42:57.675306+0000","last_became_peered":"2024-02-23T00:42:57.675306+0000","last_unstale":"2024-02-23T00:52:10.276500+0000","last_undegraded":"2024-02-23T00:52:10.276500+0000","last_fullsized":"2024-02-23T00:52:10.276500+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:49.610391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:48:49.610391+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"457'20","reported_seq":599,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.269303+0000","last_change":"2024-02-23T00:51:01.488450+0000","last_active":"2024-02-23T00:52:10.269303+0000","last_peered":"2024-02-23T00:52:10.269303+0000","last_clean":"2024-02-23T00:52:10.269303+0000","last_became_active":"2024-02-23T00:39:16.777530+0000","last_became_peered":"2024-02-23T00:39:16.777530+0000","last_unstale":"2024-02-23T00:52:10.269303+0000","last_undegraded":"2024-02-23T00:52:10.269303+0000","last_fullsized":"2024-02-23T00:52:10.269303+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'20","last_scrub_stamp":"2024-02-23T00:51:01.488371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:01.488371+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":561,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.275804+0000","last_change":"2024-02-23T00:48:41.719867+0000","last_active":"2024-02-23T00:52:10.275804+0000","last_peered":"2024-02-23T00:52:10.275804+0000","last_clean":"2024-02-23T00:52:10.275804+0000","last_became_active":"2024-02-23T00:39:14.780379+0000","last_became_peered":"2024-02-23T00:39:14.780379+0000","last_unstale":"2024-02-23T00:52:10.275804+0000","last_undegraded":"2024-02-23T00:52:10.275804+0000","last_fullsized":"2024-02-23T00:52:10.275804+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:41.719772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.575242+0000","last_clean_scrub_stamp":"2024-02-23T00:48:41.719772+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"376'9","reported_seq":318,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.274673+0000","last_change":"2024-02-23T00:49:40.180695+0000","last_active":"2024-02-23T00:52:10.274673+0000","last_peered":"2024-02-23T00:52:10.274673+0000","last_clean":"2024-02-23T00:52:10.274673+0000","last_became_active":"2024-02-23T00:47:28.231104+0000","last_became_peered":"2024-02-23T00:47:28.231104+0000","last_unstale":"2024-02-23T00:52:10.274673+0000","last_undegraded":"2024-02-23T00:52:10.274673+0000","last_fullsized":"2024-02-23T00:52:10.274673+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"376'9","last_scrub_stamp":"2024-02-23T00:49:40.180616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:40.180616+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":517,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.269665+0000","last_change":"2024-02-23T00:51:12.335819+0000","last_active":"2024-02-23T00:52:10.269665+0000","last_peered":"2024-02-23T00:52:10.269665+0000","last_clean":"2024-02-23T00:52:10.269665+0000","last_became_active":"2024-02-23T00:42:58.765106+0000","last_became_peered":"2024-02-23T00:42:58.765106+0000","last_unstale":"2024-02-23T00:52:10.269665+0000","last_undegraded":"2024-02-23T00:52:10.269665+0000","last_fullsized":"2024-02-23T00:52:10.269665+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.335752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.335752+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.335752+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"457'27","reported_seq":607,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.279787+0000","last_change":"2024-02-23T00:51:48.537804+0000","last_active":"2024-02-23T00:52:10.279787+0000","last_peered":"2024-02-23T00:52:10.279787+0000","last_clean":"2024-02-23T00:52:10.279787+0000","last_became_active":"2024-02-23T00:39:16.789757+0000","last_became_peered":"2024-02-23T00:39:16.789757+0000","last_unstale":"2024-02-23T00:52:10.279787+0000","last_undegraded":"2024-02-23T00:52:10.279787+0000","last_fullsized":"2024-02-23T00:52:10.279787+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-23T00:51:48.537732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:48.537732+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":566,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.275243+0000","last_change":"2024-02-23T00:51:13.375106+0000","last_active":"2024-02-23T00:52:10.275243+0000","last_peered":"2024-02-23T00:52:10.275243+0000","last_clean":"2024-02-23T00:52:10.275243+0000","last_became_active":"2024-02-23T00:39:14.780526+0000","last_became_peered":"2024-02-23T00:39:14.780526+0000","last_unstale":"2024-02-23T00:52:10.275243+0000","last_undegraded":"2024-02-23T00:52:10.275243+0000","last_fullsized":"2024-02-23T00:52:10.275243+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:13.375039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:13.375039+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":735,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.270527+0000","last_change":"2024-02-23T00:52:07.723000+0000","last_active":"2024-02-23T00:52:10.270527+0000","last_peered":"2024-02-23T00:52:10.270527+0000","last_clean":"2024-02-23T00:52:10.270527+0000","last_became_active":"2024-02-23T00:39:18.800440+0000","last_became_peered":"2024-02-23T00:39:18.800440+0000","last_unstale":"2024-02-23T00:52:10.270527+0000","last_undegraded":"2024-02-23T00:52:10.270527+0000","last_fullsized":"2024-02-23T00:52:10.270527+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:52:07.722925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:52:07.722925+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":515,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.269243+0000","last_change":"2024-02-23T00:51:38.762620+0000","last_active":"2024-02-23T00:52:10.269243+0000","last_peered":"2024-02-23T00:52:10.269243+0000","last_clean":"2024-02-23T00:52:10.269243+0000","last_became_active":"2024-02-23T00:42:58.833924+0000","last_became_peered":"2024-02-23T00:42:58.833924+0000","last_unstale":"2024-02-23T00:52:10.269243+0000","last_undegraded":"2024-02-23T00:52:10.269243+0000","last_fullsized":"2024-02-23T00:52:10.269243+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:38.762554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:38.762554+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"457'31","reported_seq":667,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.270356+0000","last_change":"2024-02-23T00:51:40.699209+0000","last_active":"2024-02-23T00:52:10.270356+0000","last_peered":"2024-02-23T00:52:10.270356+0000","last_clean":"2024-02-23T00:52:10.270356+0000","last_became_active":"2024-02-23T00:47:28.201225+0000","last_became_peered":"2024-02-23T00:47:28.201225+0000","last_unstale":"2024-02-23T00:52:10.270356+0000","last_undegraded":"2024-02-23T00:52:10.270356+0000","last_fullsized":"2024-02-23T00:52:10.270356+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:51:40.699141+0000","last_deep_scrub":"336'23","last_deep_scrub_stamp":"2024-02-23T00:46:48.320085+0000","last_clean_scrub_stamp":"2024-02-23T00:51:40.699141+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":568,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.273148+0000","last_change":"2024-02-23T00:49:37.331334+0000","last_active":"2024-02-23T00:52:10.273148+0000","last_peered":"2024-02-23T00:52:10.273148+0000","last_clean":"2024-02-23T00:52:10.273148+0000","last_became_active":"2024-02-23T00:39:14.784121+0000","last_became_peered":"2024-02-23T00:39:14.784121+0000","last_unstale":"2024-02-23T00:52:10.273148+0000","last_undegraded":"2024-02-23T00:52:10.273148+0000","last_fullsized":"2024-02-23T00:52:10.273148+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:37.331270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:37.331270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":566,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.274249+0000","last_change":"2024-02-23T00:50:19.827889+0000","last_active":"2024-02-23T00:52:10.274249+0000","last_peered":"2024-02-23T00:52:10.274249+0000","last_clean":"2024-02-23T00:52:10.274249+0000","last_became_active":"2024-02-23T00:39:18.800777+0000","last_became_peered":"2024-02-23T00:39:18.800777+0000","last_unstale":"2024-02-23T00:52:10.274249+0000","last_undegraded":"2024-02-23T00:52:10.274249+0000","last_fullsized":"2024-02-23T00:52:10.274249+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:19.827814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_clean_scrub_stamp":"2024-02-23T00:50:19.827814+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":503,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.275170+0000","last_change":"2024-02-23T00:49:33.529224+0000","last_active":"2024-02-23T00:52:10.275170+0000","last_peered":"2024-02-23T00:52:10.275170+0000","last_clean":"2024-02-23T00:52:10.275170+0000","last_became_active":"2024-02-23T00:42:57.653279+0000","last_became_peered":"2024-02-23T00:42:57.653279+0000","last_unstale":"2024-02-23T00:52:10.275170+0000","last_undegraded":"2024-02-23T00:52:10.275170+0000","last_fullsized":"2024-02-23T00:52:10.275170+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:33.529135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:49:33.529135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":735,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.267706+0000","last_change":"2024-02-23T00:51:34.577307+0000","last_active":"2024-02-23T00:52:10.267706+0000","last_peered":"2024-02-23T00:52:10.267706+0000","last_clean":"2024-02-23T00:52:10.267706+0000","last_became_active":"2024-02-23T00:39:18.798536+0000","last_became_peered":"2024-02-23T00:39:18.798536+0000","last_unstale":"2024-02-23T00:52:10.267706+0000","last_undegraded":"2024-02-23T00:52:10.267706+0000","last_fullsized":"2024-02-23T00:52:10.267706+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:51:34.577222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:51:34.577222+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"457'42","reported_seq":634,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.269410+0000","last_change":"2024-02-23T00:50:24.532387+0000","last_active":"2024-02-23T00:52:10.269410+0000","last_peered":"2024-02-23T00:52:10.269410+0000","last_clean":"2024-02-23T00:52:10.269410+0000","last_became_active":"2024-02-23T00:39:16.780130+0000","last_became_peered":"2024-02-23T00:39:16.780130+0000","last_unstale":"2024-02-23T00:52:10.269410+0000","last_undegraded":"2024-02-23T00:52:10.269410+0000","last_fullsized":"2024-02-23T00:52:10.269410+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'42","last_scrub_stamp":"2024-02-23T00:50:24.532271+0000","last_deep_scrub":"423'28","last_deep_scrub_stamp":"2024-02-23T00:48:17.435906+0000","last_clean_scrub_stamp":"2024-02-23T00:50:24.532271+0000","log_size":42,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":46,"num_write":34,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":587,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.280967+0000","last_change":"2024-02-23T00:50:59.199966+0000","last_active":"2024-02-23T00:52:10.280967+0000","last_peered":"2024-02-23T00:52:10.280967+0000","last_clean":"2024-02-23T00:52:10.280967+0000","last_became_active":"2024-02-23T00:47:28.237083+0000","last_became_peered":"2024-02-23T00:47:28.237083+0000","last_unstale":"2024-02-23T00:52:10.280967+0000","last_undegraded":"2024-02-23T00:52:10.280967+0000","last_fullsized":"2024-02-23T00:52:10.280967+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:59.199899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:51.391939+0000","last_clean_scrub_stamp":"2024-02-23T00:50:59.199899+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":581,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.274907+0000","last_change":"2024-02-23T00:49:41.163033+0000","last_active":"2024-02-23T00:52:10.274907+0000","last_peered":"2024-02-23T00:52:10.274907+0000","last_clean":"2024-02-23T00:52:10.274907+0000","last_became_active":"2024-02-23T00:47:30.187386+0000","last_became_peered":"2024-02-23T00:47:30.187386+0000","last_unstale":"2024-02-23T00:52:10.274907+0000","last_undegraded":"2024-02-23T00:52:10.274907+0000","last_fullsized":"2024-02-23T00:52:10.274907+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:41.162895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:46.686764+0000","last_clean_scrub_stamp":"2024-02-23T00:49:41.162895+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":520,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.274452+0000","last_change":"2024-02-23T00:51:03.875340+0000","last_active":"2024-02-23T00:52:10.274452+0000","last_peered":"2024-02-23T00:52:10.274452+0000","last_clean":"2024-02-23T00:52:10.274452+0000","last_became_active":"2024-02-23T00:47:28.212307+0000","last_became_peered":"2024-02-23T00:47:28.212307+0000","last_unstale":"2024-02-23T00:52:10.274452+0000","last_undegraded":"2024-02-23T00:52:10.274452+0000","last_fullsized":"2024-02-23T00:52:10.274452+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:03.875273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:03.875273+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":569,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.270230+0000","last_change":"2024-02-23T00:50:25.575319+0000","last_active":"2024-02-23T00:52:10.270230+0000","last_peered":"2024-02-23T00:52:10.270230+0000","last_clean":"2024-02-23T00:52:10.270230+0000","last_became_active":"2024-02-23T00:39:18.798829+0000","last_became_peered":"2024-02-23T00:39:18.798829+0000","last_unstale":"2024-02-23T00:52:10.270230+0000","last_undegraded":"2024-02-23T00:52:10.270230+0000","last_fullsized":"2024-02-23T00:52:10.270230+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:25.575238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:10.493340+0000","last_clean_scrub_stamp":"2024-02-23T00:50:25.575238+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"457'52","reported_seq":664,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.267618+0000","last_change":"2024-02-23T00:51:05.221646+0000","last_active":"2024-02-23T00:52:10.267618+0000","last_peered":"2024-02-23T00:52:10.267618+0000","last_clean":"2024-02-23T00:52:10.267618+0000","last_became_active":"2024-02-23T00:39:16.859002+0000","last_became_peered":"2024-02-23T00:39:16.859002+0000","last_unstale":"2024-02-23T00:52:10.267618+0000","last_undegraded":"2024-02-23T00:52:10.267618+0000","last_fullsized":"2024-02-23T00:52:10.267618+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'52","last_scrub_stamp":"2024-02-23T00:51:05.221572+0000","last_deep_scrub":"251'40","last_deep_scrub_stamp":"2024-02-23T00:45:23.910546+0000","last_clean_scrub_stamp":"2024-02-23T00:51:05.221572+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":76,"num_read_kb":62,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":561,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.279876+0000","last_change":"2024-02-23T00:48:44.728592+0000","last_active":"2024-02-23T00:52:10.279876+0000","last_peered":"2024-02-23T00:52:10.279876+0000","last_clean":"2024-02-23T00:52:10.279876+0000","last_became_active":"2024-02-23T00:39:14.779193+0000","last_became_peered":"2024-02-23T00:39:14.779193+0000","last_unstale":"2024-02-23T00:52:10.279876+0000","last_undegraded":"2024-02-23T00:52:10.279876+0000","last_fullsized":"2024-02-23T00:52:10.279876+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:48:44.728520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:48:44.728520+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":592,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.269825+0000","last_change":"2024-02-23T00:50:17.599788+0000","last_active":"2024-02-23T00:52:10.269825+0000","last_peered":"2024-02-23T00:52:10.269825+0000","last_clean":"2024-02-23T00:52:10.269825+0000","last_became_active":"2024-02-23T00:47:33.257915+0000","last_became_peered":"2024-02-23T00:47:33.257915+0000","last_unstale":"2024-02-23T00:52:10.269825+0000","last_undegraded":"2024-02-23T00:52:10.269825+0000","last_fullsized":"2024-02-23T00:52:10.269825+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":381,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:17.599724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:50:17.599724+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":517,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.270365+0000","last_change":"2024-02-23T00:51:03.166243+0000","last_active":"2024-02-23T00:52:10.270365+0000","last_peered":"2024-02-23T00:52:10.270365+0000","last_clean":"2024-02-23T00:52:10.270365+0000","last_became_active":"2024-02-23T00:42:58.809283+0000","last_became_peered":"2024-02-23T00:42:58.809283+0000","last_unstale":"2024-02-23T00:52:10.270365+0000","last_undegraded":"2024-02-23T00:52:10.270365+0000","last_fullsized":"2024-02-23T00:52:10.270365+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:03.166140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:03.166140+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":572,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.282414+0000","last_change":"2024-02-23T00:51:34.200412+0000","last_active":"2024-02-23T00:52:10.282414+0000","last_peered":"2024-02-23T00:52:10.282414+0000","last_clean":"2024-02-23T00:52:10.282414+0000","last_became_active":"2024-02-23T00:39:18.796664+0000","last_became_peered":"2024-02-23T00:39:18.796664+0000","last_unstale":"2024-02-23T00:52:10.282414+0000","last_undegraded":"2024-02-23T00:52:10.282414+0000","last_fullsized":"2024-02-23T00:52:10.282414+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:34.200312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:51:34.200312+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"457'44","reported_seq":645,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.273782+0000","last_change":"2024-02-23T00:49:46.644982+0000","last_active":"2024-02-23T00:52:10.273782+0000","last_peered":"2024-02-23T00:52:10.273782+0000","last_clean":"2024-02-23T00:52:10.273782+0000","last_became_active":"2024-02-23T00:39:16.857278+0000","last_became_peered":"2024-02-23T00:39:16.857278+0000","last_unstale":"2024-02-23T00:52:10.273782+0000","last_undegraded":"2024-02-23T00:52:10.273782+0000","last_fullsized":"2024-02-23T00:52:10.273782+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'44","last_scrub_stamp":"2024-02-23T00:49:46.644911+0000","last_deep_scrub":"457'44","last_deep_scrub_stamp":"2024-02-23T00:49:46.644911+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.644911+0000","log_size":44,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":568,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.274545+0000","last_change":"2024-02-23T00:50:22.299591+0000","last_active":"2024-02-23T00:52:10.274545+0000","last_peered":"2024-02-23T00:52:10.274545+0000","last_clean":"2024-02-23T00:52:10.274545+0000","last_became_active":"2024-02-23T00:39:14.783662+0000","last_became_peered":"2024-02-23T00:39:14.783662+0000","last_unstale":"2024-02-23T00:52:10.274545+0000","last_undegraded":"2024-02-23T00:52:10.274545+0000","last_fullsized":"2024-02-23T00:52:10.274545+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:22.299534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:22.299534+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":587,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.275344+0000","last_change":"2024-02-23T00:52:07.536352+0000","last_active":"2024-02-23T00:52:10.275344+0000","last_peered":"2024-02-23T00:52:10.275344+0000","last_clean":"2024-02-23T00:52:10.275344+0000","last_became_active":"2024-02-23T00:47:37.184240+0000","last_became_peered":"2024-02-23T00:47:37.184240+0000","last_unstale":"2024-02-23T00:52:10.275344+0000","last_undegraded":"2024-02-23T00:52:10.275344+0000","last_fullsized":"2024-02-23T00:52:10.275344+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":385,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:07.536288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:52:07.536288+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":511,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.275535+0000","last_change":"2024-02-23T00:50:24.682130+0000","last_active":"2024-02-23T00:52:10.275535+0000","last_peered":"2024-02-23T00:52:10.275535+0000","last_clean":"2024-02-23T00:52:10.275535+0000","last_became_active":"2024-02-23T00:42:58.726652+0000","last_became_peered":"2024-02-23T00:42:58.726652+0000","last_unstale":"2024-02-23T00:52:10.275535+0000","last_undegraded":"2024-02-23T00:52:10.275535+0000","last_fullsized":"2024-02-23T00:52:10.275535+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:24.682039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:24.682039+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":571,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.274328+0000","last_change":"2024-02-23T00:51:43.644655+0000","last_active":"2024-02-23T00:52:10.274328+0000","last_peered":"2024-02-23T00:52:10.274328+0000","last_clean":"2024-02-23T00:52:10.274328+0000","last_became_active":"2024-02-23T00:39:18.800378+0000","last_became_peered":"2024-02-23T00:39:18.800378+0000","last_unstale":"2024-02-23T00:52:10.274328+0000","last_undegraded":"2024-02-23T00:52:10.274328+0000","last_fullsized":"2024-02-23T00:52:10.274328+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:43.644591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:59.694033+0000","last_clean_scrub_stamp":"2024-02-23T00:51:43.644591+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"457'30","reported_seq":635,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.273258+0000","last_change":"2024-02-23T00:49:42.730745+0000","last_active":"2024-02-23T00:52:10.273258+0000","last_peered":"2024-02-23T00:52:10.273258+0000","last_clean":"2024-02-23T00:52:10.273258+0000","last_became_active":"2024-02-23T00:47:28.215366+0000","last_became_peered":"2024-02-23T00:47:28.215366+0000","last_unstale":"2024-02-23T00:52:10.273258+0000","last_undegraded":"2024-02-23T00:52:10.273258+0000","last_fullsized":"2024-02-23T00:52:10.273258+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'30","last_scrub_stamp":"2024-02-23T00:49:42.730670+0000","last_deep_scrub":"95'18","last_deep_scrub_stamp":"2024-02-23T00:42:01.195545+0000","last_clean_scrub_stamp":"2024-02-23T00:49:42.730670+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":573,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.273053+0000","last_change":"2024-02-23T00:49:46.162522+0000","last_active":"2024-02-23T00:52:10.273053+0000","last_peered":"2024-02-23T00:52:10.273053+0000","last_clean":"2024-02-23T00:52:10.273053+0000","last_became_active":"2024-02-23T00:39:14.784578+0000","last_became_peered":"2024-02-23T00:39:14.784578+0000","last_unstale":"2024-02-23T00:52:10.273053+0000","last_undegraded":"2024-02-23T00:52:10.273053+0000","last_fullsized":"2024-02-23T00:52:10.273053+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:46.162444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.162444+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":572,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.267898+0000","last_change":"2024-02-23T00:49:31.768539+0000","last_active":"2024-02-23T00:52:10.267898+0000","last_peered":"2024-02-23T00:52:10.267898+0000","last_clean":"2024-02-23T00:52:10.267898+0000","last_became_active":"2024-02-23T00:39:14.775446+0000","last_became_peered":"2024-02-23T00:39:14.775446+0000","last_unstale":"2024-02-23T00:52:10.267898+0000","last_undegraded":"2024-02-23T00:52:10.267898+0000","last_fullsized":"2024-02-23T00:52:10.267898+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:31.768472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:31.768472+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"67'1","reported_seq":593,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.276285+0000","last_change":"2024-02-23T00:51:02.380789+0000","last_active":"2024-02-23T00:52:10.276285+0000","last_peered":"2024-02-23T00:52:10.276285+0000","last_clean":"2024-02-23T00:52:10.276285+0000","last_became_active":"2024-02-23T00:47:56.187940+0000","last_became_peered":"2024-02-23T00:47:56.187940+0000","last_unstale":"2024-02-23T00:52:10.276285+0000","last_undegraded":"2024-02-23T00:52:10.276285+0000","last_fullsized":"2024-02-23T00:52:10.276285+0000","mapping_epoch":65,"log_start":"67'1","ondisk_log_start":"67'1","created":65,"last_epoch_clean":404,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:51:02.380721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:51:02.380721+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":518,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.279620+0000","last_change":"2024-02-23T00:51:10.298816+0000","last_active":"2024-02-23T00:52:10.279620+0000","last_peered":"2024-02-23T00:52:10.279620+0000","last_clean":"2024-02-23T00:52:10.279620+0000","last_became_active":"2024-02-23T00:42:58.690858+0000","last_became_peered":"2024-02-23T00:42:58.690858+0000","last_unstale":"2024-02-23T00:52:10.279620+0000","last_undegraded":"2024-02-23T00:52:10.279620+0000","last_fullsized":"2024-02-23T00:52:10.279620+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:10.298740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:35.360054+0000","last_clean_scrub_stamp":"2024-02-23T00:51:10.298740+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":578,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.274322+0000","last_change":"2024-02-23T00:51:39.221685+0000","last_active":"2024-02-23T00:52:10.274322+0000","last_peered":"2024-02-23T00:52:10.274322+0000","last_clean":"2024-02-23T00:52:10.274322+0000","last_became_active":"2024-02-23T00:47:28.231152+0000","last_became_peered":"2024-02-23T00:47:28.231152+0000","last_unstale":"2024-02-23T00:52:10.274322+0000","last_undegraded":"2024-02-23T00:52:10.274322+0000","last_fullsized":"2024-02-23T00:52:10.274322+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:39.221609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:51:39.221609+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"457'26","reported_seq":614,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.280958+0000","last_change":"2024-02-23T00:51:01.219903+0000","last_active":"2024-02-23T00:52:10.280958+0000","last_peered":"2024-02-23T00:52:10.280958+0000","last_clean":"2024-02-23T00:52:10.280958+0000","last_became_active":"2024-02-23T00:39:16.858880+0000","last_became_peered":"2024-02-23T00:39:16.858880+0000","last_unstale":"2024-02-23T00:52:10.280958+0000","last_undegraded":"2024-02-23T00:52:10.280958+0000","last_fullsized":"2024-02-23T00:52:10.280958+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-23T00:51:01.219824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:01.219824+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":578,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.273067+0000","last_change":"2024-02-23T00:50:24.297660+0000","last_active":"2024-02-23T00:52:10.273067+0000","last_peered":"2024-02-23T00:52:10.273067+0000","last_clean":"2024-02-23T00:52:10.273067+0000","last_became_active":"2024-02-23T00:39:08.248249+0000","last_became_peered":"2024-02-23T00:39:08.248249+0000","last_unstale":"2024-02-23T00:52:10.273067+0000","last_undegraded":"2024-02-23T00:52:10.273067+0000","last_fullsized":"2024-02-23T00:52:10.273067+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:24.297572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:49.977248+0000","last_clean_scrub_stamp":"2024-02-23T00:50:24.297572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":579,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.272243+0000","last_change":"2024-02-23T00:50:21.746072+0000","last_active":"2024-02-23T00:52:10.272243+0000","last_peered":"2024-02-23T00:52:10.272243+0000","last_clean":"2024-02-23T00:52:10.272243+0000","last_became_active":"2024-02-23T00:47:28.240482+0000","last_became_peered":"2024-02-23T00:47:28.240482+0000","last_unstale":"2024-02-23T00:52:10.272243+0000","last_undegraded":"2024-02-23T00:52:10.272243+0000","last_fullsized":"2024-02-23T00:52:10.272243+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:21.745961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:21.745961+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":576,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.269582+0000","last_change":"2024-02-23T00:50:34.055990+0000","last_active":"2024-02-23T00:52:10.269582+0000","last_peered":"2024-02-23T00:52:10.269582+0000","last_clean":"2024-02-23T00:52:10.269582+0000","last_became_active":"2024-02-23T00:39:14.783489+0000","last_became_peered":"2024-02-23T00:39:14.783489+0000","last_unstale":"2024-02-23T00:52:10.269582+0000","last_undegraded":"2024-02-23T00:52:10.269582+0000","last_fullsized":"2024-02-23T00:52:10.269582+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:34.055923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:34.055923+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":600,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.274406+0000","last_change":"2024-02-23T00:49:35.756168+0000","last_active":"2024-02-23T00:52:10.274406+0000","last_peered":"2024-02-23T00:52:10.274406+0000","last_clean":"2024-02-23T00:52:10.274406+0000","last_became_active":"2024-02-23T00:47:45.182550+0000","last_became_peered":"2024-02-23T00:47:45.182550+0000","last_unstale":"2024-02-23T00:52:10.274406+0000","last_undegraded":"2024-02-23T00:52:10.274406+0000","last_fullsized":"2024-02-23T00:52:10.274406+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:35.756073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:49:35.756073+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":522,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.274692+0000","last_change":"2024-02-23T00:50:16.257963+0000","last_active":"2024-02-23T00:52:10.274692+0000","last_peered":"2024-02-23T00:52:10.274692+0000","last_clean":"2024-02-23T00:52:10.274692+0000","last_became_active":"2024-02-23T00:47:28.233475+0000","last_became_peered":"2024-02-23T00:47:28.233475+0000","last_unstale":"2024-02-23T00:52:10.274692+0000","last_undegraded":"2024-02-23T00:52:10.274692+0000","last_fullsized":"2024-02-23T00:52:10.274692+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:16.257896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:16.257896+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":243,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.271975+0000","last_change":"2024-02-23T00:51:12.709695+0000","last_active":"2024-02-23T00:52:10.271975+0000","last_peered":"2024-02-23T00:52:10.271975+0000","last_clean":"2024-02-23T00:52:10.271975+0000","last_became_active":"2024-02-23T00:47:28.209125+0000","last_became_peered":"2024-02-23T00:47:28.209125+0000","last_unstale":"2024-02-23T00:52:10.271975+0000","last_undegraded":"2024-02-23T00:52:10.271975+0000","last_fullsized":"2024-02-23T00:52:10.271975+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.709617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.264240+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.709617+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"457'23","reported_seq":612,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.279942+0000","last_change":"2024-02-23T00:51:42.560222+0000","last_active":"2024-02-23T00:52:10.279942+0000","last_peered":"2024-02-23T00:52:10.279942+0000","last_clean":"2024-02-23T00:52:10.279942+0000","last_became_active":"2024-02-23T00:39:16.787484+0000","last_became_peered":"2024-02-23T00:39:16.787484+0000","last_unstale":"2024-02-23T00:52:10.279942+0000","last_undegraded":"2024-02-23T00:52:10.279942+0000","last_fullsized":"2024-02-23T00:52:10.279942+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'23","last_scrub_stamp":"2024-02-23T00:51:42.560138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:42.560138+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":48,"num_read_kb":36,"num_write":26,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":248,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.271384+0000","last_change":"2024-02-23T00:51:44.659312+0000","last_active":"2024-02-23T00:52:10.271384+0000","last_peered":"2024-02-23T00:52:10.271384+0000","last_clean":"2024-02-23T00:52:10.271384+0000","last_became_active":"2024-02-23T00:47:28.209575+0000","last_became_peered":"2024-02-23T00:47:28.209575+0000","last_unstale":"2024-02-23T00:52:10.271384+0000","last_undegraded":"2024-02-23T00:52:10.271384+0000","last_fullsized":"2024-02-23T00:52:10.271384+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:44.659246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:44.659246+0000","last_clean_scrub_stamp":"2024-02-23T00:51:44.659246+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"68'5","reported_seq":1345,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.275371+0000","last_change":"2024-02-23T00:50:57.456201+0000","last_active":"2024-02-23T00:52:11.275371+0000","last_peered":"2024-02-23T00:52:11.275371+0000","last_clean":"2024-02-23T00:52:11.275371+0000","last_became_active":"2024-02-23T00:47:28.227928+0000","last_became_peered":"2024-02-23T00:47:28.227928+0000","last_unstale":"2024-02-23T00:52:11.275371+0000","last_undegraded":"2024-02-23T00:52:11.275371+0000","last_fullsized":"2024-02-23T00:52:11.275371+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'5","last_scrub_stamp":"2024-02-23T00:50:57.456059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:57.456059+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":767,"num_read_kb":762,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":603,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.429050+0000","last_change":"2024-02-23T00:52:10.429013+0000","last_active":"2024-02-23T00:52:10.429050+0000","last_peered":"2024-02-23T00:52:10.429050+0000","last_clean":"2024-02-23T00:52:10.429050+0000","last_became_active":"2024-02-23T00:47:52.552244+0000","last_became_peered":"2024-02-23T00:47:52.552244+0000","last_unstale":"2024-02-23T00:52:10.429050+0000","last_undegraded":"2024-02-23T00:52:10.429050+0000","last_fullsized":"2024-02-23T00:52:10.429050+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.428945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.428945+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":512,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.274920+0000","last_change":"2024-02-23T00:49:40.533441+0000","last_active":"2024-02-23T00:52:10.274920+0000","last_peered":"2024-02-23T00:52:10.274920+0000","last_clean":"2024-02-23T00:52:10.274920+0000","last_became_active":"2024-02-23T00:47:28.212516+0000","last_became_peered":"2024-02-23T00:47:28.212516+0000","last_unstale":"2024-02-23T00:52:10.274920+0000","last_undegraded":"2024-02-23T00:52:10.274920+0000","last_fullsized":"2024-02-23T00:52:10.274920+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:40.533344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:49:40.533344+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":568,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.269999+0000","last_change":"2024-02-23T00:49:37.787041+0000","last_active":"2024-02-23T00:52:10.269999+0000","last_peered":"2024-02-23T00:52:10.269999+0000","last_clean":"2024-02-23T00:52:10.269999+0000","last_became_active":"2024-02-23T00:39:18.802134+0000","last_became_peered":"2024-02-23T00:39:18.802134+0000","last_unstale":"2024-02-23T00:52:10.269999+0000","last_undegraded":"2024-02-23T00:52:10.269999+0000","last_fullsized":"2024-02-23T00:52:10.269999+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:37.786978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:37.786978+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"457'48","reported_seq":758,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.275119+0000","last_change":"2024-02-23T00:50:30.763802+0000","last_active":"2024-02-23T00:52:10.275119+0000","last_peered":"2024-02-23T00:52:10.275119+0000","last_clean":"2024-02-23T00:52:10.275119+0000","last_became_active":"2024-02-23T00:47:28.240257+0000","last_became_peered":"2024-02-23T00:47:28.240257+0000","last_unstale":"2024-02-23T00:52:10.275119+0000","last_undegraded":"2024-02-23T00:52:10.275119+0000","last_fullsized":"2024-02-23T00:52:10.275119+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'48","last_scrub_stamp":"2024-02-23T00:50:30.763686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:30.763686+0000","log_size":48,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":78,"num_read_kb":62,"num_write":44,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":575,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.271091+0000","last_change":"2024-02-23T00:52:09.366353+0000","last_active":"2024-02-23T00:52:10.271091+0000","last_peered":"2024-02-23T00:52:10.271091+0000","last_clean":"2024-02-23T00:52:10.271091+0000","last_became_active":"2024-02-23T00:39:14.784015+0000","last_became_peered":"2024-02-23T00:39:14.784015+0000","last_unstale":"2024-02-23T00:52:10.271091+0000","last_undegraded":"2024-02-23T00:52:10.271091+0000","last_fullsized":"2024-02-23T00:52:10.271091+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:09.366281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:09.366281+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":606,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.270228+0000","last_change":"2024-02-23T00:51:46.680015+0000","last_active":"2024-02-23T00:52:10.270228+0000","last_peered":"2024-02-23T00:52:10.270228+0000","last_clean":"2024-02-23T00:52:10.270228+0000","last_became_active":"2024-02-23T00:47:49.525421+0000","last_became_peered":"2024-02-23T00:47:49.525421+0000","last_unstale":"2024-02-23T00:52:10.270228+0000","last_undegraded":"2024-02-23T00:52:10.270228+0000","last_fullsized":"2024-02-23T00:52:10.270228+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":397,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:46.679950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:51:46.679950+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":242,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.269971+0000","last_change":"2024-02-23T00:50:13.549604+0000","last_active":"2024-02-23T00:52:10.269971+0000","last_peered":"2024-02-23T00:52:10.269971+0000","last_clean":"2024-02-23T00:52:10.269971+0000","last_became_active":"2024-02-23T00:47:28.232194+0000","last_became_peered":"2024-02-23T00:47:28.232194+0000","last_unstale":"2024-02-23T00:52:10.269971+0000","last_undegraded":"2024-02-23T00:52:10.269971+0000","last_fullsized":"2024-02-23T00:52:10.269971+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:13.549529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:13.549529+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3","version":"91'9","reported_seq":737,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.274032+0000","last_change":"2024-02-23T00:52:08.663976+0000","last_active":"2024-02-23T00:52:10.274032+0000","last_peered":"2024-02-23T00:52:10.274032+0000","last_clean":"2024-02-23T00:52:10.274032+0000","last_became_active":"2024-02-23T00:39:18.801297+0000","last_became_peered":"2024-02-23T00:39:18.801297+0000","last_unstale":"2024-02-23T00:52:10.274032+0000","last_undegraded":"2024-02-23T00:52:10.274032+0000","last_fullsized":"2024-02-23T00:52:10.274032+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'9","last_scrub_stamp":"2024-02-23T00:52:08.663869+0000","last_deep_scrub":"91'9","last_deep_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_clean_scrub_stamp":"2024-02-23T00:52:08.663869+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"457'26","reported_seq":604,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.276135+0000","last_change":"2024-02-23T00:48:55.589320+0000","last_active":"2024-02-23T00:52:10.276135+0000","last_peered":"2024-02-23T00:52:10.276135+0000","last_clean":"2024-02-23T00:52:10.276135+0000","last_became_active":"2024-02-23T00:39:16.787468+0000","last_became_peered":"2024-02-23T00:39:16.787468+0000","last_unstale":"2024-02-23T00:52:10.276135+0000","last_undegraded":"2024-02-23T00:52:10.276135+0000","last_fullsized":"2024-02-23T00:52:10.276135+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'16","last_scrub_stamp":"2024-02-23T00:48:55.589209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:48:55.589209+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":36,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":566,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.276006+0000","last_change":"2024-02-23T00:51:36.415402+0000","last_active":"2024-02-23T00:52:10.276006+0000","last_peered":"2024-02-23T00:52:10.276006+0000","last_clean":"2024-02-23T00:52:10.276006+0000","last_became_active":"2024-02-23T00:39:14.779952+0000","last_became_peered":"2024-02-23T00:39:14.779952+0000","last_unstale":"2024-02-23T00:52:10.276006+0000","last_undegraded":"2024-02-23T00:52:10.276006+0000","last_fullsized":"2024-02-23T00:52:10.276006+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:36.415340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:36.415340+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"107'2","reported_seq":601,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.270256+0000","last_change":"2024-02-23T00:50:17.587665+0000","last_active":"2024-02-23T00:52:10.270256+0000","last_peered":"2024-02-23T00:52:10.270256+0000","last_clean":"2024-02-23T00:52:10.270256+0000","last_became_active":"2024-02-23T00:47:41.210074+0000","last_became_peered":"2024-02-23T00:47:41.210074+0000","last_unstale":"2024-02-23T00:52:10.270256+0000","last_undegraded":"2024-02-23T00:52:10.270256+0000","last_fullsized":"2024-02-23T00:52:10.270256+0000","mapping_epoch":375,"log_start":"107'2","ondisk_log_start":"107'2","created":65,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'2","last_scrub_stamp":"2024-02-23T00:50:17.587591+0000","last_deep_scrub":"107'2","last_deep_scrub_stamp":"2024-02-23T00:42:48.362739+0000","last_clean_scrub_stamp":"2024-02-23T00:50:17.587591+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":522,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.280760+0000","last_change":"2024-02-23T00:50:18.308436+0000","last_active":"2024-02-23T00:52:10.280760+0000","last_peered":"2024-02-23T00:52:10.280760+0000","last_clean":"2024-02-23T00:52:10.280760+0000","last_became_active":"2024-02-23T00:47:28.242255+0000","last_became_peered":"2024-02-23T00:47:28.242255+0000","last_unstale":"2024-02-23T00:52:10.280760+0000","last_undegraded":"2024-02-23T00:52:10.280760+0000","last_fullsized":"2024-02-23T00:52:10.280760+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:18.308350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:18.308350+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":245,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.272198+0000","last_change":"2024-02-23T00:51:35.631662+0000","last_active":"2024-02-23T00:52:10.272198+0000","last_peered":"2024-02-23T00:52:10.272198+0000","last_clean":"2024-02-23T00:52:10.272198+0000","last_became_active":"2024-02-23T00:47:28.234034+0000","last_became_peered":"2024-02-23T00:47:28.234034+0000","last_unstale":"2024-02-23T00:52:10.272198+0000","last_undegraded":"2024-02-23T00:52:10.272198+0000","last_fullsized":"2024-02-23T00:52:10.272198+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.631598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.631598+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"457'31","reported_seq":632,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.270134+0000","last_change":"2024-02-23T00:51:43.724733+0000","last_active":"2024-02-23T00:52:10.270134+0000","last_peered":"2024-02-23T00:52:10.270134+0000","last_clean":"2024-02-23T00:52:10.270134+0000","last_became_active":"2024-02-23T00:39:16.783926+0000","last_became_peered":"2024-02-23T00:39:16.783926+0000","last_unstale":"2024-02-23T00:52:10.270134+0000","last_undegraded":"2024-02-23T00:52:10.270134+0000","last_fullsized":"2024-02-23T00:52:10.270134+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:51:43.724670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:43.724670+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":58,"num_read_kb":47,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"95'4","reported_seq":591,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.270713+0000","last_change":"2024-02-23T00:52:06.734682+0000","last_active":"2024-02-23T00:52:10.270713+0000","last_peered":"2024-02-23T00:52:10.270713+0000","last_clean":"2024-02-23T00:52:10.270713+0000","last_became_active":"2024-02-23T00:39:14.779205+0000","last_became_peered":"2024-02-23T00:39:14.779205+0000","last_unstale":"2024-02-23T00:52:10.270713+0000","last_undegraded":"2024-02-23T00:52:10.270713+0000","last_fullsized":"2024-02-23T00:52:10.270713+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'4","last_scrub_stamp":"2024-02-23T00:52:06.734599+0000","last_deep_scrub":"95'4","last_deep_scrub_stamp":"2024-02-23T00:41:43.874397+0000","last_clean_scrub_stamp":"2024-02-23T00:52:06.734599+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":588,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.268693+0000","last_change":"2024-02-23T00:50:15.922829+0000","last_active":"2024-02-23T00:52:10.268693+0000","last_peered":"2024-02-23T00:52:10.268693+0000","last_clean":"2024-02-23T00:52:10.268693+0000","last_became_active":"2024-02-23T00:43:27.611660+0000","last_became_peered":"2024-02-23T00:43:27.611660+0000","last_unstale":"2024-02-23T00:52:10.268693+0000","last_undegraded":"2024-02-23T00:52:10.268693+0000","last_fullsized":"2024-02-23T00:52:10.268693+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:15.922745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:50:15.922745+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":525,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.275091+0000","last_change":"2024-02-23T00:51:46.336777+0000","last_active":"2024-02-23T00:52:10.275091+0000","last_peered":"2024-02-23T00:52:10.275091+0000","last_clean":"2024-02-23T00:52:10.275091+0000","last_became_active":"2024-02-23T00:47:28.231670+0000","last_became_peered":"2024-02-23T00:47:28.231670+0000","last_unstale":"2024-02-23T00:52:10.275091+0000","last_undegraded":"2024-02-23T00:52:10.275091+0000","last_fullsized":"2024-02-23T00:52:10.275091+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:46.336718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:46.336718+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"457'27","reported_seq":625,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.273345+0000","last_change":"2024-02-23T00:51:42.671042+0000","last_active":"2024-02-23T00:52:10.273345+0000","last_peered":"2024-02-23T00:52:10.273345+0000","last_clean":"2024-02-23T00:52:10.273345+0000","last_became_active":"2024-02-23T00:39:16.858760+0000","last_became_peered":"2024-02-23T00:39:16.858760+0000","last_unstale":"2024-02-23T00:52:10.273345+0000","last_undegraded":"2024-02-23T00:52:10.273345+0000","last_fullsized":"2024-02-23T00:52:10.273345+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-23T00:51:42.670972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:42.670972+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":236,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.275250+0000","last_change":"2024-02-23T00:49:02.601790+0000","last_active":"2024-02-23T00:52:10.275250+0000","last_peered":"2024-02-23T00:52:10.275250+0000","last_clean":"2024-02-23T00:52:10.275250+0000","last_became_active":"2024-02-23T00:47:28.219764+0000","last_became_peered":"2024-02-23T00:47:28.219764+0000","last_unstale":"2024-02-23T00:52:10.275250+0000","last_undegraded":"2024-02-23T00:52:10.275250+0000","last_fullsized":"2024-02-23T00:52:10.275250+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:02.601726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:49:02.601726+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":577,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.267835+0000","last_change":"2024-02-23T00:51:45.695038+0000","last_active":"2024-02-23T00:52:10.267835+0000","last_peered":"2024-02-23T00:52:10.267835+0000","last_clean":"2024-02-23T00:52:10.267835+0000","last_became_active":"2024-02-23T00:39:18.798624+0000","last_became_peered":"2024-02-23T00:39:18.798624+0000","last_unstale":"2024-02-23T00:52:10.267835+0000","last_undegraded":"2024-02-23T00:52:10.267835+0000","last_fullsized":"2024-02-23T00:52:10.267835+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:45.694974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:45.913526+0000","last_clean_scrub_stamp":"2024-02-23T00:51:45.694974+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":513,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.270454+0000","last_change":"2024-02-23T00:50:13.642424+0000","last_active":"2024-02-23T00:52:10.270454+0000","last_peered":"2024-02-23T00:52:10.270454+0000","last_clean":"2024-02-23T00:52:10.270454+0000","last_became_active":"2024-02-23T00:42:58.825015+0000","last_became_peered":"2024-02-23T00:42:58.825015+0000","last_unstale":"2024-02-23T00:52:10.270454+0000","last_undegraded":"2024-02-23T00:52:10.270454+0000","last_fullsized":"2024-02-23T00:52:10.270454+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:13.642359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:13.642359+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"457'22","reported_seq":611,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.267760+0000","last_change":"2024-02-23T00:51:01.201807+0000","last_active":"2024-02-23T00:52:10.267760+0000","last_peered":"2024-02-23T00:52:10.267760+0000","last_clean":"2024-02-23T00:52:10.267760+0000","last_became_active":"2024-02-23T00:39:16.858449+0000","last_became_peered":"2024-02-23T00:39:16.858449+0000","last_unstale":"2024-02-23T00:52:10.267760+0000","last_undegraded":"2024-02-23T00:52:10.267760+0000","last_fullsized":"2024-02-23T00:52:10.267760+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'22","last_scrub_stamp":"2024-02-23T00:51:01.201737+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-02-23T00:48:07.941101+0000","last_clean_scrub_stamp":"2024-02-23T00:51:01.201737+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":30,"num_read_kb":26,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":576,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.279973+0000","last_change":"2024-02-23T00:52:08.247330+0000","last_active":"2024-02-23T00:52:10.279973+0000","last_peered":"2024-02-23T00:52:10.279973+0000","last_clean":"2024-02-23T00:52:10.279973+0000","last_became_active":"2024-02-23T00:39:14.783164+0000","last_became_peered":"2024-02-23T00:39:14.783164+0000","last_unstale":"2024-02-23T00:52:10.279973+0000","last_undegraded":"2024-02-23T00:52:10.279973+0000","last_fullsized":"2024-02-23T00:52:10.279973+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:08.247258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:08.247258+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":571,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.272050+0000","last_change":"2024-02-23T00:51:47.691352+0000","last_active":"2024-02-23T00:52:10.272050+0000","last_peered":"2024-02-23T00:52:10.272050+0000","last_clean":"2024-02-23T00:52:10.272050+0000","last_became_active":"2024-02-23T00:39:18.800297+0000","last_became_peered":"2024-02-23T00:39:18.800297+0000","last_unstale":"2024-02-23T00:52:10.272050+0000","last_undegraded":"2024-02-23T00:52:10.272050+0000","last_fullsized":"2024-02-23T00:52:10.272050+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:47.691288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:51:47.691288+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1","version":"110'7","reported_seq":530,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.271580+0000","last_change":"2024-02-23T00:51:13.352686+0000","last_active":"2024-02-23T00:52:10.271580+0000","last_peered":"2024-02-23T00:52:10.271580+0000","last_clean":"2024-02-23T00:52:10.271580+0000","last_became_active":"2024-02-23T00:42:58.777251+0000","last_became_peered":"2024-02-23T00:42:58.777251+0000","last_unstale":"2024-02-23T00:52:10.271580+0000","last_undegraded":"2024-02-23T00:52:10.271580+0000","last_fullsized":"2024-02-23T00:52:10.271580+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'7","last_scrub_stamp":"2024-02-23T00:51:13.352611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:13.352611+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"457'32","reported_seq":272,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.268965+0000","last_change":"2024-02-23T00:50:25.079859+0000","last_active":"2024-02-23T00:52:10.268965+0000","last_peered":"2024-02-23T00:52:10.268965+0000","last_clean":"2024-02-23T00:52:10.268965+0000","last_became_active":"2024-02-23T00:47:28.237898+0000","last_became_peered":"2024-02-23T00:47:28.237898+0000","last_unstale":"2024-02-23T00:52:10.268965+0000","last_undegraded":"2024-02-23T00:52:10.268965+0000","last_fullsized":"2024-02-23T00:52:10.268965+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-23T00:50:25.079789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:25.079789+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":52,"num_read_kb":45,"num_write":32,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":575,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.271014+0000","last_change":"2024-02-23T00:51:14.658413+0000","last_active":"2024-02-23T00:52:10.271014+0000","last_peered":"2024-02-23T00:52:10.271014+0000","last_clean":"2024-02-23T00:52:10.271014+0000","last_became_active":"2024-02-23T00:39:14.785590+0000","last_became_peered":"2024-02-23T00:39:14.785590+0000","last_unstale":"2024-02-23T00:52:10.271014+0000","last_undegraded":"2024-02-23T00:52:10.271014+0000","last_fullsized":"2024-02-23T00:52:10.271014+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:14.658297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_clean_scrub_stamp":"2024-02-23T00:51:14.658297+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"336'9","reported_seq":735,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.275642+0000","last_change":"2024-02-23T00:52:06.373546+0000","last_active":"2024-02-23T00:52:10.275642+0000","last_peered":"2024-02-23T00:52:10.275642+0000","last_clean":"2024-02-23T00:52:10.275642+0000","last_became_active":"2024-02-23T00:39:18.803566+0000","last_became_peered":"2024-02-23T00:39:18.803566+0000","last_unstale":"2024-02-23T00:52:10.275642+0000","last_undegraded":"2024-02-23T00:52:10.275642+0000","last_fullsized":"2024-02-23T00:52:10.275642+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"336'9","last_scrub_stamp":"2024-02-23T00:52:06.373463+0000","last_deep_scrub":"336'9","last_deep_scrub_stamp":"2024-02-23T00:46:49.160991+0000","last_clean_scrub_stamp":"2024-02-23T00:52:06.373463+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":511,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.273839+0000","last_change":"2024-02-23T00:51:33.285134+0000","last_active":"2024-02-23T00:52:10.273839+0000","last_peered":"2024-02-23T00:52:10.273839+0000","last_clean":"2024-02-23T00:52:10.273839+0000","last_became_active":"2024-02-23T00:42:58.757424+0000","last_became_peered":"2024-02-23T00:42:58.757424+0000","last_unstale":"2024-02-23T00:52:10.273839+0000","last_undegraded":"2024-02-23T00:52:10.273839+0000","last_fullsized":"2024-02-23T00:52:10.273839+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:33.285048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:33.285048+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"457'21","reported_seq":608,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.275440+0000","last_change":"2024-02-23T00:50:17.861556+0000","last_active":"2024-02-23T00:52:10.275440+0000","last_peered":"2024-02-23T00:52:10.275440+0000","last_clean":"2024-02-23T00:52:10.275440+0000","last_became_active":"2024-02-23T00:39:16.858953+0000","last_became_peered":"2024-02-23T00:39:16.858953+0000","last_unstale":"2024-02-23T00:52:10.275440+0000","last_undegraded":"2024-02-23T00:52:10.275440+0000","last_fullsized":"2024-02-23T00:52:10.275440+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-23T00:50:17.861490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:17.861490+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":577,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.274006+0000","last_change":"2024-02-23T00:51:33.662415+0000","last_active":"2024-02-23T00:52:10.274006+0000","last_peered":"2024-02-23T00:52:10.274006+0000","last_clean":"2024-02-23T00:52:10.274006+0000","last_became_active":"2024-02-23T00:39:14.773772+0000","last_became_peered":"2024-02-23T00:39:14.773772+0000","last_unstale":"2024-02-23T00:52:10.274006+0000","last_undegraded":"2024-02-23T00:52:10.274006+0000","last_fullsized":"2024-02-23T00:52:10.274006+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:33.662354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:33.662354+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"376'9","reported_seq":748,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.269355+0000","last_change":"2024-02-23T00:52:07.717254+0000","last_active":"2024-02-23T00:52:10.269355+0000","last_peered":"2024-02-23T00:52:10.269355+0000","last_clean":"2024-02-23T00:52:10.269355+0000","last_became_active":"2024-02-23T00:47:28.229316+0000","last_became_peered":"2024-02-23T00:47:28.229316+0000","last_unstale":"2024-02-23T00:52:10.269355+0000","last_undegraded":"2024-02-23T00:52:10.269355+0000","last_fullsized":"2024-02-23T00:52:10.269355+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"376'9","last_scrub_stamp":"2024-02-23T00:52:07.717186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:52:07.717186+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":503,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.276581+0000","last_change":"2024-02-23T00:50:15.714265+0000","last_active":"2024-02-23T00:52:10.276581+0000","last_peered":"2024-02-23T00:52:10.276581+0000","last_clean":"2024-02-23T00:52:10.276581+0000","last_became_active":"2024-02-23T00:42:57.652488+0000","last_became_peered":"2024-02-23T00:42:57.652488+0000","last_unstale":"2024-02-23T00:52:10.276581+0000","last_undegraded":"2024-02-23T00:52:10.276581+0000","last_fullsized":"2024-02-23T00:52:10.276581+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:15.714200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:51.254903+0000","last_clean_scrub_stamp":"2024-02-23T00:50:15.714200+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"457'15","reported_seq":603,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.275959+0000","last_change":"2024-02-23T00:49:39.573557+0000","last_active":"2024-02-23T00:52:10.275959+0000","last_peered":"2024-02-23T00:52:10.275959+0000","last_clean":"2024-02-23T00:52:10.275959+0000","last_became_active":"2024-02-23T00:47:28.225585+0000","last_became_peered":"2024-02-23T00:47:28.225585+0000","last_unstale":"2024-02-23T00:52:10.275959+0000","last_undegraded":"2024-02-23T00:52:10.275959+0000","last_fullsized":"2024-02-23T00:52:10.275959+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'15","last_scrub_stamp":"2024-02-23T00:49:39.573472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:39.573472+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":581,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.280760+0000","last_change":"2024-02-23T00:51:44.270528+0000","last_active":"2024-02-23T00:52:10.280760+0000","last_peered":"2024-02-23T00:52:10.280760+0000","last_clean":"2024-02-23T00:52:10.280760+0000","last_became_active":"2024-02-23T00:39:14.778455+0000","last_became_peered":"2024-02-23T00:39:14.778455+0000","last_unstale":"2024-02-23T00:52:10.280760+0000","last_undegraded":"2024-02-23T00:52:10.280760+0000","last_fullsized":"2024-02-23T00:52:10.280760+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:44.270465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:44.270465+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":581,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.268617+0000","last_change":"2024-02-23T00:50:30.069404+0000","last_active":"2024-02-23T00:52:10.268617+0000","last_peered":"2024-02-23T00:52:10.268617+0000","last_clean":"2024-02-23T00:52:10.268617+0000","last_became_active":"2024-02-23T00:47:44.225379+0000","last_became_peered":"2024-02-23T00:47:44.225379+0000","last_unstale":"2024-02-23T00:52:10.268617+0000","last_undegraded":"2024-02-23T00:52:10.268617+0000","last_fullsized":"2024-02-23T00:52:10.268617+0000","mapping_epoch":391,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":392,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:30.069325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_clean_scrub_stamp":"2024-02-23T00:50:30.069325+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":518,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.269181+0000","last_change":"2024-02-23T00:51:44.638197+0000","last_active":"2024-02-23T00:52:10.269181+0000","last_peered":"2024-02-23T00:52:10.269181+0000","last_clean":"2024-02-23T00:52:10.269181+0000","last_became_active":"2024-02-23T00:42:58.835862+0000","last_became_peered":"2024-02-23T00:42:58.835862+0000","last_unstale":"2024-02-23T00:52:10.269181+0000","last_undegraded":"2024-02-23T00:52:10.269181+0000","last_fullsized":"2024-02-23T00:52:10.269181+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:44.638129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:44.638129+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"457'19","reported_seq":600,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.269696+0000","last_change":"2024-02-23T00:50:29.599563+0000","last_active":"2024-02-23T00:52:10.269696+0000","last_peered":"2024-02-23T00:52:10.269696+0000","last_clean":"2024-02-23T00:52:10.269696+0000","last_became_active":"2024-02-23T00:39:16.783896+0000","last_became_peered":"2024-02-23T00:39:16.783896+0000","last_unstale":"2024-02-23T00:52:10.269696+0000","last_undegraded":"2024-02-23T00:52:10.269696+0000","last_fullsized":"2024-02-23T00:52:10.269696+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'19","last_scrub_stamp":"2024-02-23T00:50:29.599483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:29.599483+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"202'2","reported_seq":574,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.279486+0000","last_change":"2024-02-23T00:51:40.511331+0000","last_active":"2024-02-23T00:52:10.279486+0000","last_peered":"2024-02-23T00:52:10.279486+0000","last_clean":"2024-02-23T00:52:10.279486+0000","last_became_active":"2024-02-23T00:39:14.775804+0000","last_became_peered":"2024-02-23T00:39:14.775804+0000","last_unstale":"2024-02-23T00:52:10.279486+0000","last_undegraded":"2024-02-23T00:52:10.279486+0000","last_fullsized":"2024-02-23T00:52:10.279486+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"202'2","last_scrub_stamp":"2024-02-23T00:51:40.511258+0000","last_deep_scrub":"202'2","last_deep_scrub_stamp":"2024-02-23T00:44:34.720640+0000","last_clean_scrub_stamp":"2024-02-23T00:51:40.511258+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":557,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.276046+0000","last_change":"2024-02-23T00:50:18.698805+0000","last_active":"2024-02-23T00:52:10.276046+0000","last_peered":"2024-02-23T00:52:10.276046+0000","last_clean":"2024-02-23T00:52:10.276046+0000","last_became_active":"2024-02-23T00:39:18.800382+0000","last_became_peered":"2024-02-23T00:39:18.800382+0000","last_unstale":"2024-02-23T00:52:10.276046+0000","last_undegraded":"2024-02-23T00:52:10.276046+0000","last_fullsized":"2024-02-23T00:52:10.276046+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:18.698687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:50:18.698687+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":512,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.279441+0000","last_change":"2024-02-23T00:49:46.462085+0000","last_active":"2024-02-23T00:52:10.279441+0000","last_peered":"2024-02-23T00:52:10.279441+0000","last_clean":"2024-02-23T00:52:10.279441+0000","last_became_active":"2024-02-23T00:47:28.224961+0000","last_became_peered":"2024-02-23T00:47:28.224961+0000","last_unstale":"2024-02-23T00:52:10.279441+0000","last_undegraded":"2024-02-23T00:52:10.279441+0000","last_fullsized":"2024-02-23T00:52:10.279441+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:46.462015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:46.462015+0000","last_clean_scrub_stamp":"2024-02-23T00:49:46.462015+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"457'8","reported_seq":586,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.267443+0000","last_change":"2024-02-23T00:50:35.045697+0000","last_active":"2024-02-23T00:52:10.267443+0000","last_peered":"2024-02-23T00:52:10.267443+0000","last_clean":"2024-02-23T00:52:10.267443+0000","last_became_active":"2024-02-23T00:39:16.785830+0000","last_became_peered":"2024-02-23T00:39:16.785830+0000","last_unstale":"2024-02-23T00:52:10.267443+0000","last_undegraded":"2024-02-23T00:52:10.267443+0000","last_fullsized":"2024-02-23T00:52:10.267443+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'8","last_scrub_stamp":"2024-02-23T00:50:35.045612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:35.045612+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":587,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.271888+0000","last_change":"2024-02-23T00:51:34.529078+0000","last_active":"2024-02-23T00:52:10.271888+0000","last_peered":"2024-02-23T00:52:10.271888+0000","last_clean":"2024-02-23T00:52:10.271888+0000","last_became_active":"2024-02-23T00:47:28.224477+0000","last_became_peered":"2024-02-23T00:47:28.224477+0000","last_unstale":"2024-02-23T00:52:10.271888+0000","last_undegraded":"2024-02-23T00:52:10.271888+0000","last_fullsized":"2024-02-23T00:52:10.271888+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:34.529012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:34.529012+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":586,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.279805+0000","last_change":"2024-02-23T00:52:07.209710+0000","last_active":"2024-02-23T00:52:10.279805+0000","last_peered":"2024-02-23T00:52:10.279805+0000","last_clean":"2024-02-23T00:52:10.279805+0000","last_became_active":"2024-02-23T00:47:44.230295+0000","last_became_peered":"2024-02-23T00:47:44.230295+0000","last_unstale":"2024-02-23T00:52:10.279805+0000","last_undegraded":"2024-02-23T00:52:10.279805+0000","last_fullsized":"2024-02-23T00:52:10.279805+0000","mapping_epoch":391,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":392,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:07.209628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:52:07.209628+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":511,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.274903+0000","last_change":"2024-02-23T00:51:38.169736+0000","last_active":"2024-02-23T00:52:10.274903+0000","last_peered":"2024-02-23T00:52:10.274903+0000","last_clean":"2024-02-23T00:52:10.274903+0000","last_became_active":"2024-02-23T00:42:58.741365+0000","last_became_peered":"2024-02-23T00:42:58.741365+0000","last_unstale":"2024-02-23T00:52:10.274903+0000","last_undegraded":"2024-02-23T00:52:10.274903+0000","last_fullsized":"2024-02-23T00:52:10.274903+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:38.169669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:38.169669+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"457'25","reported_seq":625,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.271767+0000","last_change":"2024-02-23T00:52:08.715626+0000","last_active":"2024-02-23T00:52:10.271767+0000","last_peered":"2024-02-23T00:52:10.271767+0000","last_clean":"2024-02-23T00:52:10.271767+0000","last_became_active":"2024-02-23T00:39:16.857232+0000","last_became_peered":"2024-02-23T00:39:16.857232+0000","last_unstale":"2024-02-23T00:52:10.271767+0000","last_undegraded":"2024-02-23T00:52:10.271767+0000","last_fullsized":"2024-02-23T00:52:10.271767+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-23T00:52:08.715549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:52:08.715549+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":585,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.280879+0000","last_change":"2024-02-23T00:51:35.218650+0000","last_active":"2024-02-23T00:52:10.280879+0000","last_peered":"2024-02-23T00:52:10.280879+0000","last_clean":"2024-02-23T00:52:10.280879+0000","last_became_active":"2024-02-23T00:47:28.240162+0000","last_became_peered":"2024-02-23T00:47:28.240162+0000","last_unstale":"2024-02-23T00:52:10.280879+0000","last_undegraded":"2024-02-23T00:52:10.280879+0000","last_fullsized":"2024-02-23T00:52:10.280879+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:35.218583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:44.285083+0000","last_clean_scrub_stamp":"2024-02-23T00:51:35.218583+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":225,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.267330+0000","last_change":"2024-02-23T00:49:42.872820+0000","last_active":"2024-02-23T00:52:10.267330+0000","last_peered":"2024-02-23T00:52:10.267330+0000","last_clean":"2024-02-23T00:52:10.267330+0000","last_became_active":"2024-02-23T00:47:44.226097+0000","last_became_peered":"2024-02-23T00:47:44.226097+0000","last_unstale":"2024-02-23T00:52:10.267330+0000","last_undegraded":"2024-02-23T00:52:10.267330+0000","last_fullsized":"2024-02-23T00:52:10.267330+0000","mapping_epoch":391,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":392,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:42.872748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:49:42.872748+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":244,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.274035+0000","last_change":"2024-02-23T00:50:28.678155+0000","last_active":"2024-02-23T00:52:10.274035+0000","last_peered":"2024-02-23T00:52:10.274035+0000","last_clean":"2024-02-23T00:52:10.274035+0000","last_became_active":"2024-02-23T00:47:28.200688+0000","last_became_peered":"2024-02-23T00:47:28.200688+0000","last_unstale":"2024-02-23T00:52:10.274035+0000","last_undegraded":"2024-02-23T00:52:10.274035+0000","last_fullsized":"2024-02-23T00:52:10.274035+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:28.678041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:28.678041+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"457'26","reported_seq":615,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.273565+0000","last_change":"2024-02-23T00:50:23.540361+0000","last_active":"2024-02-23T00:52:10.273565+0000","last_peered":"2024-02-23T00:52:10.273565+0000","last_clean":"2024-02-23T00:52:10.273565+0000","last_became_active":"2024-02-23T00:39:16.858945+0000","last_became_peered":"2024-02-23T00:39:16.858945+0000","last_unstale":"2024-02-23T00:52:10.273565+0000","last_undegraded":"2024-02-23T00:52:10.273565+0000","last_fullsized":"2024-02-23T00:52:10.273565+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-23T00:50:23.540232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:23.540232+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":36,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":582,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.270683+0000","last_change":"2024-02-23T00:51:09.827935+0000","last_active":"2024-02-23T00:52:10.270683+0000","last_peered":"2024-02-23T00:52:10.270683+0000","last_clean":"2024-02-23T00:52:10.270683+0000","last_became_active":"2024-02-23T00:39:14.785363+0000","last_became_peered":"2024-02-23T00:39:14.785363+0000","last_unstale":"2024-02-23T00:52:10.270683+0000","last_undegraded":"2024-02-23T00:52:10.270683+0000","last_fullsized":"2024-02-23T00:52:10.270683+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"80'2","last_scrub_stamp":"2024-02-23T00:51:09.827857+0000","last_deep_scrub":"80'2","last_deep_scrub_stamp":"2024-02-23T00:40:15.942725+0000","last_clean_scrub_stamp":"2024-02-23T00:51:09.827857+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"93'9","reported_seq":739,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.281104+0000","last_change":"2024-02-23T00:51:39.253717+0000","last_active":"2024-02-23T00:52:10.281104+0000","last_peered":"2024-02-23T00:52:10.281104+0000","last_clean":"2024-02-23T00:52:10.281104+0000","last_became_active":"2024-02-23T00:39:18.796680+0000","last_became_peered":"2024-02-23T00:39:18.796680+0000","last_unstale":"2024-02-23T00:52:10.281104+0000","last_undegraded":"2024-02-23T00:52:10.281104+0000","last_fullsized":"2024-02-23T00:52:10.281104+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-02-23T00:51:39.253651+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-02-23T00:43:50.393713+0000","last_clean_scrub_stamp":"2024-02-23T00:51:39.253651+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":522,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.272091+0000","last_change":"2024-02-23T00:51:12.493007+0000","last_active":"2024-02-23T00:52:10.272091+0000","last_peered":"2024-02-23T00:52:10.272091+0000","last_clean":"2024-02-23T00:52:10.272091+0000","last_became_active":"2024-02-23T00:47:28.234641+0000","last_became_peered":"2024-02-23T00:47:28.234641+0000","last_unstale":"2024-02-23T00:52:10.272091+0000","last_undegraded":"2024-02-23T00:52:10.272091+0000","last_fullsized":"2024-02-23T00:52:10.272091+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.492942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.492942+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"457'31","reported_seq":631,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.281206+0000","last_change":"2024-02-23T00:51:15.408847+0000","last_active":"2024-02-23T00:52:10.281206+0000","last_peered":"2024-02-23T00:52:10.281206+0000","last_clean":"2024-02-23T00:52:10.281206+0000","last_became_active":"2024-02-23T00:39:16.782660+0000","last_became_peered":"2024-02-23T00:39:16.782660+0000","last_unstale":"2024-02-23T00:52:10.281206+0000","last_undegraded":"2024-02-23T00:52:10.281206+0000","last_fullsized":"2024-02-23T00:52:10.281206+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:51:15.408782+0000","last_deep_scrub":"93'23","last_deep_scrub_stamp":"2024-02-23T00:41:35.954516+0000","last_clean_scrub_stamp":"2024-02-23T00:51:15.408782+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":576,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.271140+0000","last_change":"2024-02-23T00:50:29.029794+0000","last_active":"2024-02-23T00:52:10.271140+0000","last_peered":"2024-02-23T00:52:10.271140+0000","last_clean":"2024-02-23T00:52:10.271140+0000","last_became_active":"2024-02-23T00:39:14.781560+0000","last_became_peered":"2024-02-23T00:39:14.781560+0000","last_unstale":"2024-02-23T00:52:10.271140+0000","last_undegraded":"2024-02-23T00:52:10.271140+0000","last_fullsized":"2024-02-23T00:52:10.271140+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:29.029725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:29.029725+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":582,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.273614+0000","last_change":"2024-02-23T00:51:14.488316+0000","last_active":"2024-02-23T00:52:10.273614+0000","last_peered":"2024-02-23T00:52:10.273614+0000","last_clean":"2024-02-23T00:52:10.273614+0000","last_became_active":"2024-02-23T00:47:28.232797+0000","last_became_peered":"2024-02-23T00:47:28.232797+0000","last_unstale":"2024-02-23T00:52:10.273614+0000","last_undegraded":"2024-02-23T00:52:10.273614+0000","last_fullsized":"2024-02-23T00:52:10.273614+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:14.488205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:53.189881+0000","last_clean_scrub_stamp":"2024-02-23T00:51:14.488205+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":517,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.271237+0000","last_change":"2024-02-23T00:51:00.232343+0000","last_active":"2024-02-23T00:52:10.271237+0000","last_peered":"2024-02-23T00:52:10.271237+0000","last_clean":"2024-02-23T00:52:10.271237+0000","last_became_active":"2024-02-23T00:42:58.774720+0000","last_became_peered":"2024-02-23T00:42:58.774720+0000","last_unstale":"2024-02-23T00:52:10.271237+0000","last_undegraded":"2024-02-23T00:52:10.271237+0000","last_fullsized":"2024-02-23T00:52:10.271237+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:00.232261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:00.232261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"457'27","reported_seq":307,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.276093+0000","last_change":"2024-02-23T00:49:26.518048+0000","last_active":"2024-02-23T00:52:10.276093+0000","last_peered":"2024-02-23T00:52:10.276093+0000","last_clean":"2024-02-23T00:52:10.276093+0000","last_became_active":"2024-02-23T00:47:28.211342+0000","last_became_peered":"2024-02-23T00:47:28.211342+0000","last_unstale":"2024-02-23T00:52:10.276093+0000","last_undegraded":"2024-02-23T00:52:10.276093+0000","last_fullsized":"2024-02-23T00:52:10.276093+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-23T00:49:26.517965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:26.517965+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":576,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.280677+0000","last_change":"2024-02-23T00:51:08.235807+0000","last_active":"2024-02-23T00:52:10.280677+0000","last_peered":"2024-02-23T00:52:10.280677+0000","last_clean":"2024-02-23T00:52:10.280677+0000","last_became_active":"2024-02-23T00:39:14.782494+0000","last_became_peered":"2024-02-23T00:39:14.782494+0000","last_unstale":"2024-02-23T00:52:10.280677+0000","last_undegraded":"2024-02-23T00:52:10.280677+0000","last_fullsized":"2024-02-23T00:52:10.280677+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:08.235735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:08.235735+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":562,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.563318+0000","last_change":"2024-02-23T00:52:10.563274+0000","last_active":"2024-02-23T00:52:10.563318+0000","last_peered":"2024-02-23T00:52:10.563318+0000","last_clean":"2024-02-23T00:52:10.563318+0000","last_became_active":"2024-02-23T00:39:18.801039+0000","last_became_peered":"2024-02-23T00:39:18.801039+0000","last_unstale":"2024-02-23T00:52:10.563318+0000","last_undegraded":"2024-02-23T00:52:10.563318+0000","last_fullsized":"2024-02-23T00:52:10.563318+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.563137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.641689+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.563137+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":517,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.269873+0000","last_change":"2024-02-23T00:51:10.341831+0000","last_active":"2024-02-23T00:52:10.269873+0000","last_peered":"2024-02-23T00:52:10.269873+0000","last_clean":"2024-02-23T00:52:10.269873+0000","last_became_active":"2024-02-23T00:42:58.770759+0000","last_became_peered":"2024-02-23T00:42:58.770759+0000","last_unstale":"2024-02-23T00:52:10.269873+0000","last_undegraded":"2024-02-23T00:52:10.269873+0000","last_fullsized":"2024-02-23T00:52:10.269873+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:10.341768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:10.341768+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"457'24","reported_seq":621,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.268132+0000","last_change":"2024-02-23T00:50:13.861288+0000","last_active":"2024-02-23T00:52:10.268132+0000","last_peered":"2024-02-23T00:52:10.268132+0000","last_clean":"2024-02-23T00:52:10.268132+0000","last_became_active":"2024-02-23T00:39:16.782803+0000","last_became_peered":"2024-02-23T00:39:16.782803+0000","last_unstale":"2024-02-23T00:52:10.268132+0000","last_undegraded":"2024-02-23T00:52:10.268132+0000","last_fullsized":"2024-02-23T00:52:10.268132+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'24","last_scrub_stamp":"2024-02-23T00:50:13.861220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:50:13.861220+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":46,"num_read_kb":36,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":576,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.268547+0000","last_change":"2024-02-23T00:50:28.060186+0000","last_active":"2024-02-23T00:52:10.268547+0000","last_peered":"2024-02-23T00:52:10.268547+0000","last_clean":"2024-02-23T00:52:10.268547+0000","last_became_active":"2024-02-23T00:39:14.782163+0000","last_became_peered":"2024-02-23T00:39:14.782163+0000","last_unstale":"2024-02-23T00:52:10.268547+0000","last_undegraded":"2024-02-23T00:52:10.268547+0000","last_fullsized":"2024-02-23T00:52:10.268547+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:28.060109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:28.060109+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":576,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.269389+0000","last_change":"2024-02-23T00:50:18.629287+0000","last_active":"2024-02-23T00:52:10.269389+0000","last_peered":"2024-02-23T00:52:10.269389+0000","last_clean":"2024-02-23T00:52:10.269389+0000","last_became_active":"2024-02-23T00:47:28.215452+0000","last_became_peered":"2024-02-23T00:47:28.215452+0000","last_unstale":"2024-02-23T00:52:10.269389+0000","last_undegraded":"2024-02-23T00:52:10.269389+0000","last_fullsized":"2024-02-23T00:52:10.269389+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:18.629203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_clean_scrub_stamp":"2024-02-23T00:50:18.629203+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":520,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.271946+0000","last_change":"2024-02-23T00:50:56.774466+0000","last_active":"2024-02-23T00:52:10.271946+0000","last_peered":"2024-02-23T00:52:10.271946+0000","last_clean":"2024-02-23T00:52:10.271946+0000","last_became_active":"2024-02-23T00:47:28.203555+0000","last_became_peered":"2024-02-23T00:47:28.203555+0000","last_unstale":"2024-02-23T00:52:10.271946+0000","last_undegraded":"2024-02-23T00:52:10.271946+0000","last_fullsized":"2024-02-23T00:52:10.271946+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:56.774382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:56.774382+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"457'33","reported_seq":637,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.695414+0000","last_change":"2024-02-23T00:52:11.695414+0000","last_active":"2024-02-23T00:52:11.695414+0000","last_peered":"2024-02-23T00:52:11.695414+0000","last_clean":"2024-02-23T00:52:11.695414+0000","last_became_active":"2024-02-23T00:39:16.786791+0000","last_became_peered":"2024-02-23T00:39:16.786791+0000","last_unstale":"2024-02-23T00:52:11.695414+0000","last_undegraded":"2024-02-23T00:52:11.695414+0000","last_fullsized":"2024-02-23T00:52:11.695414+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'33","last_scrub_stamp":"2024-02-23T00:52:11.695335+0000","last_deep_scrub":"95'27","last_deep_scrub_stamp":"2024-02-23T00:41:55.919895+0000","last_clean_scrub_stamp":"2024-02-23T00:52:11.695335+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":582,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.274235+0000","last_change":"2024-02-23T00:51:42.249052+0000","last_active":"2024-02-23T00:52:10.274235+0000","last_peered":"2024-02-23T00:52:10.274235+0000","last_clean":"2024-02-23T00:52:10.274235+0000","last_became_active":"2024-02-23T00:47:28.208293+0000","last_became_peered":"2024-02-23T00:47:28.208293+0000","last_unstale":"2024-02-23T00:52:10.274235+0000","last_undegraded":"2024-02-23T00:52:10.274235+0000","last_fullsized":"2024-02-23T00:52:10.274235+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:42.248982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:42.248982+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":571,"reported_epoch":602,"state":"active+clean","last_fresh":"2024-02-23T00:52:11.328198+0000","last_change":"2024-02-23T00:52:11.328198+0000","last_active":"2024-02-23T00:52:11.328198+0000","last_peered":"2024-02-23T00:52:11.328198+0000","last_clean":"2024-02-23T00:52:11.328198+0000","last_became_active":"2024-02-23T00:39:18.796431+0000","last_became_peered":"2024-02-23T00:39:18.796431+0000","last_unstale":"2024-02-23T00:52:11.328198+0000","last_undegraded":"2024-02-23T00:52:11.328198+0000","last_fullsized":"2024-02-23T00:52:11.328198+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:11.328122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:52:11.328122+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":511,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.275418+0000","last_change":"2024-02-23T00:50:26.630228+0000","last_active":"2024-02-23T00:52:10.275418+0000","last_peered":"2024-02-23T00:52:10.275418+0000","last_clean":"2024-02-23T00:52:10.275418+0000","last_became_active":"2024-02-23T00:42:58.832885+0000","last_became_peered":"2024-02-23T00:42:58.832885+0000","last_unstale":"2024-02-23T00:52:10.275418+0000","last_undegraded":"2024-02-23T00:52:10.275418+0000","last_fullsized":"2024-02-23T00:52:10.275418+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:26.630088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:50:26.630088+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"457'17","reported_seq":601,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.269866+0000","last_change":"2024-02-23T00:50:15.590233+0000","last_active":"2024-02-23T00:52:10.269866+0000","last_peered":"2024-02-23T00:52:10.269866+0000","last_clean":"2024-02-23T00:52:10.269866+0000","last_became_active":"2024-02-23T00:39:16.782762+0000","last_became_peered":"2024-02-23T00:39:16.782762+0000","last_unstale":"2024-02-23T00:52:10.269866+0000","last_undegraded":"2024-02-23T00:52:10.269866+0000","last_fullsized":"2024-02-23T00:52:10.269866+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'17","last_scrub_stamp":"2024-02-23T00:50:15.590096+0000","last_deep_scrub":"82'15","last_deep_scrub_stamp":"2024-02-23T00:41:50.975683+0000","last_clean_scrub_stamp":"2024-02-23T00:50:15.590096+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":21,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":573,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.274152+0000","last_change":"2024-02-23T00:51:08.199826+0000","last_active":"2024-02-23T00:52:10.274152+0000","last_peered":"2024-02-23T00:52:10.274152+0000","last_clean":"2024-02-23T00:52:10.274152+0000","last_became_active":"2024-02-23T00:39:14.783469+0000","last_became_peered":"2024-02-23T00:39:14.783469+0000","last_unstale":"2024-02-23T00:52:10.274152+0000","last_undegraded":"2024-02-23T00:52:10.274152+0000","last_fullsized":"2024-02-23T00:52:10.274152+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:08.199749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:08.199749+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":571,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.273697+0000","last_change":"2024-02-23T00:51:13.264970+0000","last_active":"2024-02-23T00:52:10.273697+0000","last_peered":"2024-02-23T00:52:10.273697+0000","last_clean":"2024-02-23T00:52:10.273697+0000","last_became_active":"2024-02-23T00:39:18.796952+0000","last_became_peered":"2024-02-23T00:39:18.796952+0000","last_unstale":"2024-02-23T00:52:10.273697+0000","last_undegraded":"2024-02-23T00:52:10.273697+0000","last_fullsized":"2024-02-23T00:52:10.273697+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:13.264868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:06.491793+0000","last_clean_scrub_stamp":"2024-02-23T00:51:13.264868+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":515,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.269241+0000","last_change":"2024-02-23T00:51:03.460376+0000","last_active":"2024-02-23T00:52:10.269241+0000","last_peered":"2024-02-23T00:52:10.269241+0000","last_clean":"2024-02-23T00:52:10.269241+0000","last_became_active":"2024-02-23T00:42:58.758147+0000","last_became_peered":"2024-02-23T00:42:58.758147+0000","last_unstale":"2024-02-23T00:52:10.269241+0000","last_undegraded":"2024-02-23T00:52:10.269241+0000","last_fullsized":"2024-02-23T00:52:10.269241+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:03.460302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:23.224663+0000","last_clean_scrub_stamp":"2024-02-23T00:51:03.460302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"457'21","reported_seq":609,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.268730+0000","last_change":"2024-02-23T00:49:32.434383+0000","last_active":"2024-02-23T00:52:10.268730+0000","last_peered":"2024-02-23T00:52:10.268730+0000","last_clean":"2024-02-23T00:52:10.268730+0000","last_became_active":"2024-02-23T00:39:16.784606+0000","last_became_peered":"2024-02-23T00:39:16.784606+0000","last_unstale":"2024-02-23T00:52:10.268730+0000","last_undegraded":"2024-02-23T00:52:10.268730+0000","last_fullsized":"2024-02-23T00:52:10.268730+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-23T00:49:32.434282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:49:32.434282+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":576,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.270140+0000","last_change":"2024-02-23T00:50:19.967076+0000","last_active":"2024-02-23T00:52:10.270140+0000","last_peered":"2024-02-23T00:52:10.270140+0000","last_clean":"2024-02-23T00:52:10.270140+0000","last_became_active":"2024-02-23T00:39:14.785362+0000","last_became_peered":"2024-02-23T00:39:14.785362+0000","last_unstale":"2024-02-23T00:52:10.270140+0000","last_undegraded":"2024-02-23T00:52:10.270140+0000","last_fullsized":"2024-02-23T00:52:10.270140+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:19.967009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:50:19.967009+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":557,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.275137+0000","last_change":"2024-02-23T00:49:37.537279+0000","last_active":"2024-02-23T00:52:10.275137+0000","last_peered":"2024-02-23T00:52:10.275137+0000","last_clean":"2024-02-23T00:52:10.275137+0000","last_became_active":"2024-02-23T00:39:18.796312+0000","last_became_peered":"2024-02-23T00:39:18.796312+0000","last_unstale":"2024-02-23T00:52:10.275137+0000","last_undegraded":"2024-02-23T00:52:10.275137+0000","last_fullsized":"2024-02-23T00:52:10.275137+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:37.537211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:47.226331+0000","last_clean_scrub_stamp":"2024-02-23T00:49:37.537211+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":513,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.273465+0000","last_change":"2024-02-23T00:50:28.479578+0000","last_active":"2024-02-23T00:52:10.273465+0000","last_peered":"2024-02-23T00:52:10.273465+0000","last_clean":"2024-02-23T00:52:10.273465+0000","last_became_active":"2024-02-23T00:42:58.838257+0000","last_became_peered":"2024-02-23T00:42:58.838257+0000","last_unstale":"2024-02-23T00:52:10.273465+0000","last_undegraded":"2024-02-23T00:52:10.273465+0000","last_fullsized":"2024-02-23T00:52:10.273465+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:28.479512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:50:28.479512+0000","last_clean_scrub_stamp":"2024-02-23T00:50:28.479512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"457'62","reported_seq":721,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.271784+0000","last_change":"2024-02-23T00:50:58.151569+0000","last_active":"2024-02-23T00:52:10.271784+0000","last_peered":"2024-02-23T00:52:10.271784+0000","last_clean":"2024-02-23T00:52:10.271784+0000","last_became_active":"2024-02-23T00:47:28.205150+0000","last_became_peered":"2024-02-23T00:47:28.205150+0000","last_unstale":"2024-02-23T00:52:10.271784+0000","last_undegraded":"2024-02-23T00:52:10.271784+0000","last_fullsized":"2024-02-23T00:52:10.271784+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'62","last_scrub_stamp":"2024-02-23T00:50:58.151494+0000","last_deep_scrub":"88'50","last_deep_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_clean_scrub_stamp":"2024-02-23T00:50:58.151494+0000","log_size":62,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":92,"num_read_kb":79,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":572,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.268772+0000","last_change":"2024-02-23T00:50:34.600693+0000","last_active":"2024-02-23T00:52:10.268772+0000","last_peered":"2024-02-23T00:52:10.268772+0000","last_clean":"2024-02-23T00:52:10.268772+0000","last_became_active":"2024-02-23T00:39:14.782000+0000","last_became_peered":"2024-02-23T00:39:14.782000+0000","last_unstale":"2024-02-23T00:52:10.268772+0000","last_undegraded":"2024-02-23T00:52:10.268772+0000","last_fullsized":"2024-02-23T00:52:10.268772+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:50:34.600592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_clean_scrub_stamp":"2024-02-23T00:50:34.600592+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":567,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.268959+0000","last_change":"2024-02-23T00:51:47.686009+0000","last_active":"2024-02-23T00:52:10.268959+0000","last_peered":"2024-02-23T00:52:10.268959+0000","last_clean":"2024-02-23T00:52:10.268959+0000","last_became_active":"2024-02-23T00:39:18.799723+0000","last_became_peered":"2024-02-23T00:39:18.799723+0000","last_unstale":"2024-02-23T00:52:10.268959+0000","last_undegraded":"2024-02-23T00:52:10.268959+0000","last_fullsized":"2024-02-23T00:52:10.268959+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:47.685918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:51:47.685918+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":511,"reported_epoch":601,"state":"active+clean","last_fresh":"2024-02-23T00:52:10.273848+0000","last_change":"2024-02-23T00:49:44.601162+0000","last_active":"2024-02-23T00:52:10.273848+0000","last_peered":"2024-02-23T00:52:10.273848+0000","last_clean":"2024-02-23T00:52:10.273848+0000","last_became_active":"2024-02-23T00:42:58.824573+0000","last_became_peered":"2024-02-23T00:42:58.824573+0000","last_unstale":"2024-02-23T00:52:10.273848+0000","last_undegraded":"2024-02-23T00:52:10.273848+0000","last_fullsized":"2024-02-23T00:52:10.273848+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:49:44.601086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:44.601086+0000","last_clean_scrub_stamp":"2024-02-23T00:49:44.601086+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0],"acting":[5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":235,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":96,"acting":96,"num_store_stats":7},{"poolid":234,"num_pg":8,"stat_sum":{"num_bytes":4,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":24,"acting":24,"num_store_stats":7},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":12,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":36,"acting":36,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1508,"num_read_kb":1233,"num_write":895,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":1486,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":955,"ondisk_log_size":955,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":767,"num_read_kb":762,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234143,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":300132,"kb_used_data":2908,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93445020,"statfs":{"total":95995035648,"available":95687700480,"internally_reserved":0,"allocated":2977792,"data_stored":1313512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.50600000000000001,"15min":0.53200000000000003},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":4.9420000000000002},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51500000000000001,"15min":0.52000000000000002},"min":{"1min":0.34699999999999998,"5min":0.32000000000000001,"15min":0.24299999999999999},"max":{"1min":0.82599999999999996,"5min":1.151,"15min":4.9729999999999999},"last":0.39500000000000002}]},{"osd":1,"last update":"Fri Feb 23 00:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.50800000000000001,"15min":0.61199999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.626,"5min":1.25,"15min":10.786},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52400000000000002,"15min":0.61899999999999999},"min":{"1min":0.38600000000000001,"5min":0.33900000000000002,"15min":0.28100000000000003},"max":{"1min":0.79300000000000004,"5min":1.107,"15min":11.885999999999999},"last":0.51600000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.51700000000000002,"15min":0.55000000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.28999999999999998},"max":{"1min":0.64900000000000002,"5min":0.80600000000000005,"15min":5.0789999999999997},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52800000000000002,"15min":0.54500000000000004},"min":{"1min":0.38900000000000001,"5min":0.34200000000000003,"15min":0.316},"max":{"1min":0.72299999999999998,"5min":1.1180000000000001,"15min":5.1280000000000001},"last":0.502}]},{"osd":3,"last update":"Fri Feb 23 00:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.52200000000000002,"15min":0.61799999999999999},"min":{"1min":0.33200000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.74399999999999999,"5min":1.095,"15min":15.712999999999999},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54700000000000004,"15min":0.61399999999999999},"min":{"1min":0.35799999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.879,"5min":0.879,"15min":9.4090000000000007},"last":0.52500000000000002}]},{"osd":4,"last update":"Fri Feb 23 00:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.439,"15min":0.439},"min":{"1min":0.30199999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.753,"5min":1.6819999999999999,"15min":2.0350000000000001},"last":0.307},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.45500000000000002,"15min":0.48899999999999999},"min":{"1min":0.33600000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.68400000000000005,"5min":0.86699999999999999,"15min":11.236000000000001},"last":0.375}]},{"osd":5,"last update":"Fri Feb 23 00:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47299999999999998,"15min":0.49199999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27000000000000002},"max":{"1min":0.625,"5min":0.73199999999999998,"15min":5.4000000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.50700000000000001,"15min":0.5},"min":{"1min":0.35499999999999998,"5min":0.29099999999999998,"15min":0.26700000000000002},"max":{"1min":0.67500000000000004,"5min":1.728,"15min":1.728},"last":0.36199999999999999}]},{"osd":6,"last update":"Fri Feb 23 00:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51200000000000001,"15min":0.51100000000000001},"min":{"1min":0.42099999999999999,"5min":0.32200000000000001,"15min":0.317},"max":{"1min":0.75700000000000001,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.50900000000000001,"15min":0.54900000000000004},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.626,"5min":0.70499999999999996,"15min":13.395},"last":0.48999999999999999}]}]},{"osd":6,"up_from":47,"seq":201863463078,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308336,"kb_used_data":2920,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93436816,"statfs":{"total":95995035648,"available":95679299584,"internally_reserved":0,"allocated":2990080,"data_stored":1317800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:52:08 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51100000000000001,"15min":0.67800000000000005},"min":{"1min":0.28499999999999998,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.82299999999999995,"5min":0.93799999999999994,"15min":30.641999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.59799999999999998,"15min":0.70899999999999996},"min":{"1min":0.315,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":4.3609999999999998,"5min":4.3609999999999998,"15min":29.292000000000002},"last":0.57699999999999996}]},{"osd":1,"last update":"Fri Feb 23 00:52:08 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.56000000000000005,"15min":0.69599999999999995},"min":{"1min":0.27100000000000002,"5min":0.25900000000000001,"15min":0.247},"max":{"1min":4.1980000000000004,"5min":4.1980000000000004,"15min":28.186},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.61699999999999999,"15min":0.88500000000000001},"min":{"1min":0.39600000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":6.4370000000000003,"5min":6.4370000000000003,"15min":33.104999999999997},"last":0.53600000000000003}]},{"osd":2,"last update":"Fri Feb 23 00:52:08 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.57599999999999996,"15min":0.72599999999999998},"min":{"1min":0.36699999999999999,"5min":0.29399999999999998,"15min":0.27400000000000002},"max":{"1min":4.0919999999999996,"5min":4.0919999999999996,"15min":19.960000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54200000000000004,"15min":0.64400000000000002},"min":{"1min":0.32500000000000001,"5min":0.316,"15min":0.29899999999999999},"max":{"1min":0.67100000000000004,"5min":1.665,"15min":8.5860000000000003},"last":0.51500000000000001}]},{"osd":3,"last update":"Fri Feb 23 00:52:08 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.59099999999999997,"15min":0.63900000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.28599999999999998},"max":{"1min":4.1340000000000003,"5min":4.1340000000000003,"15min":19.562000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.61099999999999999,"15min":0.67900000000000005},"min":{"1min":0.38300000000000001,"5min":0.34999999999999998,"15min":0.33000000000000002},"max":{"1min":3.7839999999999998,"5min":3.7839999999999998,"15min":14.199},"last":0.60199999999999998}]},{"osd":4,"last update":"Fri Feb 23 00:52:08 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.44400000000000001,"15min":0.48099999999999998},"min":{"1min":0.25600000000000001,"5min":0.249,"15min":0.218},"max":{"1min":0.59099999999999997,"5min":1.6220000000000001,"15min":11.109999999999999},"last":0.379},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.44400000000000001,"15min":0.54600000000000004},"min":{"1min":0.313,"5min":0.27300000000000002,"15min":0.26900000000000002},"max":{"1min":0.54000000000000004,"5min":0.83099999999999996,"15min":16.734000000000002},"last":0.36099999999999999}]},{"osd":5,"last update":"Fri Feb 23 00:52:08 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.47599999999999998,"15min":0.51500000000000001},"min":{"1min":0.32500000000000001,"5min":0.308,"15min":0.249},"max":{"1min":0.57499999999999996,"5min":1.784,"15min":14.154999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.57399999999999995,"15min":0.55600000000000005},"min":{"1min":0.34000000000000002,"5min":0.318,"15min":0.26400000000000001},"max":{"1min":0.57499999999999996,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.40899999999999997}]},{"osd":7,"last update":"Fri Feb 23 00:51:23 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.51000000000000001,"15min":0.53100000000000003},"min":{"1min":0.38600000000000001,"5min":0.29399999999999998,"15min":0.26800000000000002},"max":{"1min":0.63100000000000001,"5min":1.726,"15min":8.548},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53900000000000003,"15min":0.60699999999999998},"min":{"1min":0.40300000000000002,"5min":0.34200000000000003,"15min":0.309},"max":{"1min":0.71199999999999997,"5min":0.88500000000000001,"15min":15.618},"last":0.438}]}]},{"osd":1,"up_from":13,"seq":55834575040,"num_pgs":86,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308352,"kb_used_data":2936,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93436800,"statfs":{"total":95995035648,"available":95679283200,"internally_reserved":0,"allocated":3006464,"data_stored":1315314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:51:52 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.39600000000000002,"15min":0.68400000000000005},"min":{"1min":0.27800000000000002,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.45900000000000002,"5min":6.202,"15min":78.361000000000004},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.371,"15min":0.44600000000000001},"min":{"1min":0.24199999999999999,"5min":0.216,"15min":0.191},"max":{"1min":0.625,"5min":0.69999999999999996,"15min":5.8849999999999998},"last":0.33500000000000002}]},{"osd":2,"last update":"Fri Feb 23 00:51:13 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.436,"15min":0.67100000000000004},"min":{"1min":0.24399999999999999,"5min":0.20499999999999999,"15min":0.188},"max":{"1min":2.3010000000000002,"5min":7.5789999999999997,"15min":48.606000000000002},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.40799999999999997,"15min":0.57599999999999996},"min":{"1min":0.29399999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.63500000000000001,"5min":1,"15min":23.911000000000001},"last":0.374}]},{"osd":3,"last update":"Fri Feb 23 00:51:52 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40799999999999997,"15min":0.58899999999999997},"min":{"1min":0.25700000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.72499999999999998,"5min":0.82999999999999996,"15min":35.244999999999997},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.47699999999999998,"15min":0.68300000000000005},"min":{"1min":0.29999999999999999,"5min":0.26900000000000002,"15min":0.248},"max":{"1min":0.71699999999999997,"5min":7.641,"15min":56.853999999999999},"last":0.34799999999999998}]},{"osd":4,"last update":"Fri Feb 23 00:52:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65200000000000002,"15min":0.82599999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.29299999999999998},"max":{"1min":1.575,"5min":7.6829999999999998,"15min":37.646999999999998},"last":0.749},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.72099999999999997,"15min":0.73099999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":6.9160000000000004,"5min":6.9160000000000004,"15min":9.6859999999999999},"last":0.75800000000000001}]},{"osd":5,"last update":"Fri Feb 23 00:51:35 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.65000000000000002,"15min":0.83299999999999996},"min":{"1min":0.35399999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.81999999999999995,"5min":7.742,"15min":48.792000000000002},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.625,"15min":0.71199999999999997},"min":{"1min":0.39100000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77900000000000003,"5min":0.92600000000000005,"15min":11.872999999999999},"last":0.70499999999999996}]},{"osd":6,"last update":"Fri Feb 23 00:52:12 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66200000000000003,"15min":0.82199999999999995},"min":{"1min":0.40100000000000002,"5min":0.37,"15min":0.318},"max":{"1min":1.5249999999999999,"5min":7.7130000000000001,"15min":40.741},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60399999999999998,"15min":0.66800000000000004},"min":{"1min":0.44800000000000001,"5min":0.38400000000000001,"15min":0.34000000000000002},"max":{"1min":0.78200000000000003,"5min":0.85999999999999999,"15min":6.0069999999999997},"last":0.58199999999999996}]},{"osd":7,"last update":"Fri Feb 23 00:51:35 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61899999999999999,"15min":0.71399999999999997},"min":{"1min":0.41799999999999998,"5min":0.40200000000000002,"15min":0.36699999999999999},"max":{"1min":0.83899999999999997,"5min":0.92900000000000005,"15min":20.225000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.66500000000000004,"15min":0.73899999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.95499999999999996,"5min":6.0949999999999998,"15min":20.254999999999999},"last":0.72299999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738566,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":312444,"kb_used_data":2932,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93432708,"statfs":{"total":95995035648,"available":95675092992,"internally_reserved":0,"allocated":3002368,"data_stored":1318811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 23 00:51:53 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.35899999999999999,"15min":0.60499999999999998},"min":{"1min":0.246,"5min":0.224,"15min":0.19500000000000001},"max":{"1min":0.50900000000000001,"5min":0.81399999999999995,"15min":28.856999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.40400000000000003,"15min":0.63200000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.20799999999999999},"max":{"1min":0.53000000000000003,"5min":2.3650000000000002,"15min":20.577000000000002},"last":0.36399999999999999}]},{"osd":2,"last update":"Fri Feb 23 00:51:18 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.40400000000000003,"15min":0.60999999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.20499999999999999},"max":{"1min":0.61099999999999999,"5min":2.3889999999999998,"15min":13.863},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.35099999999999998,"15min":0.53800000000000003},"min":{"1min":0.252,"5min":0.223,"15min":0.19800000000000001},"max":{"1min":0.504,"5min":0.65000000000000002,"15min":10.455},"last":0.42099999999999999}]},{"osd":3,"last update":"Fri Feb 23 00:51:48 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.378,"15min":0.77300000000000002},"min":{"1min":0.248,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.51600000000000001,"5min":0.73099999999999998,"15min":35.566000000000003},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.377,"5min":0.435,"15min":0.76000000000000001},"min":{"1min":0.27800000000000002,"5min":0.27400000000000002,"15min":0.26100000000000001},"max":{"1min":0.57499999999999996,"5min":2.4049999999999998,"15min":20.702999999999999},"last":0.441}]},{"osd":4,"last update":"Fri Feb 23 00:52:12 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55900000000000005,"15min":0.74199999999999999},"min":{"1min":0.32800000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80400000000000005,"5min":0.94699999999999995,"15min":10.584},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61499999999999999,"15min":0.76800000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35099999999999998},"max":{"1min":0.85099999999999998,"5min":2.5019999999999998,"15min":13.198},"last":0.51800000000000002}]},{"osd":5,"last update":"Fri Feb 23 00:51:37 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54500000000000004,"15min":0.71499999999999997},"min":{"1min":0.36699999999999999,"5min":0.33800000000000002,"15min":0.32200000000000001},"max":{"1min":0.72599999999999998,"5min":0.90800000000000003,"15min":10.397},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.60699999999999998,"15min":0.78100000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76800000000000002,"5min":0.92600000000000005,"15min":17.297000000000001},"last":0.55400000000000005}]},{"osd":6,"last update":"Fri Feb 23 00:52:12 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59399999999999997,"15min":0.77700000000000002},"min":{"1min":0.41099999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.748,"5min":0.94399999999999995,"15min":17.373999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.61599999999999999,"15min":0.75800000000000001},"min":{"1min":0.436,"5min":0.38,"15min":0.36799999999999999},"max":{"1min":0.85199999999999998,"5min":2.556,"15min":13.973000000000001},"last":0.47599999999999998}]},{"osd":7,"last update":"Fri Feb 23 00:51:27 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.59099999999999997,"15min":0.77800000000000002},"min":{"1min":0.379,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83399999999999996,"5min":0.96099999999999997,"15min":17.399999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.63100000000000001,"15min":0.78100000000000003},"min":{"1min":0.46000000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.752,"5min":2.5289999999999999,"15min":16.427},"last":0.49299999999999999}]}]},{"osd":2,"up_from":0,"seq":77309411515,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":26,"seq":111669149876,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":312452,"kb_used_data":2940,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93432700,"statfs":{"total":95995035648,"available":95675084800,"internally_reserved":0,"allocated":3010560,"data_stored":1314098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.42999999999999999,"15min":0.53200000000000003},"min":{"1min":0.216,"5min":0.21199999999999999,"15min":0.20899999999999999},"max":{"1min":0.63600000000000001,"5min":8.3300000000000001,"15min":12.212999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.42099999999999999,"15min":0.60299999999999998},"min":{"1min":0.25,"5min":0.22800000000000001,"15min":0.21199999999999999},"max":{"1min":0.48599999999999999,"5min":2.9740000000000002,"15min":11.714},"last":0.36599999999999999}]},{"osd":1,"last update":"Fri Feb 23 00:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.40999999999999998,"15min":0.624},"min":{"1min":0.25900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.52100000000000002,"5min":3.056,"15min":11.742000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.433,"15min":0.59799999999999998},"min":{"1min":0.28199999999999997,"5min":0.24099999999999999,"15min":0.19900000000000001},"max":{"1min":0.71599999999999997,"5min":2.7709999999999999,"15min":8.2270000000000003},"last":0.28199999999999997}]},{"osd":2,"last update":"Fri Feb 23 00:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.47699999999999998,"15min":0.68300000000000005},"min":{"1min":0.30499999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.53600000000000003,"5min":3.242,"15min":13.359999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.49099999999999999,"15min":0.66100000000000003},"min":{"1min":0.28100000000000003,"5min":0.27500000000000002,"15min":0.26100000000000001},"max":{"1min":0.68700000000000006,"5min":8.5820000000000007,"15min":21.122},"last":0.67000000000000004}]},{"osd":4,"last update":"Fri Feb 23 00:51:10 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.58899999999999997,"15min":0.755},"min":{"1min":0.40300000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":3.1419999999999999,"5min":3.1419999999999999,"15min":11.644},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.74199999999999999,"15min":0.753},"min":{"1min":0.36899999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":2.9020000000000001,"5min":12.430999999999999,"15min":12.430999999999999},"last":0.45900000000000002}]},{"osd":5,"last update":"Fri Feb 23 00:51:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.67800000000000005,"15min":0.68899999999999995},"min":{"1min":0.42899999999999999,"5min":0.37,"15min":0.35099999999999998},"max":{"1min":1,"5min":10.231999999999999,"15min":10.231999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.624,"15min":0.80000000000000004},"min":{"1min":0.41699999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.94999999999999996,"5min":3.1829999999999998,"15min":11.648},"last":0.73899999999999999}]},{"osd":6,"last update":"Fri Feb 23 00:51:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.629,"15min":0.755},"min":{"1min":0.377,"5min":0.377,"15min":0.35199999999999998},"max":{"1min":0.83799999999999997,"5min":2.9510000000000001,"15min":6.9420000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.73899999999999999,"15min":0.72299999999999998},"min":{"1min":0.442,"5min":0.376,"15min":0.376},"max":{"1min":0.82299999999999995,"5min":10.965999999999999,"15min":10.965999999999999},"last":0.60199999999999998}]},{"osd":7,"last update":"Fri Feb 23 00:51:34 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.66000000000000003,"15min":0.83299999999999996},"min":{"1min":0.441,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.1030000000000002,"5min":3.1030000000000002,"15min":11.676},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.66600000000000004,"15min":0.81000000000000005},"min":{"1min":0.501,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":2.8580000000000001,"5min":2.8580000000000001,"15min":7.9800000000000004},"last":0.62}]}]},{"osd":4,"up_from":32,"seq":137438953648,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":312436,"kb_used_data":2924,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93432716,"statfs":{"total":95995035648,"available":95675101184,"internally_reserved":0,"allocated":2994176,"data_stored":1317833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:52:11 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51100000000000001,"15min":0.52200000000000002},"min":{"1min":0.30399999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.70499999999999996,"5min":1.3819999999999999,"15min":3.5219999999999998},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53900000000000003,"15min":0.58299999999999996},"min":{"1min":0.29799999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.048,"5min":1.8,"15min":8.6140000000000008},"last":0.71399999999999997}]},{"osd":1,"last update":"Fri Feb 23 00:52:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52900000000000003,"15min":0.57599999999999996},"min":{"1min":0.36699999999999999,"5min":0.29699999999999999,"15min":0.27300000000000002},"max":{"1min":0.70599999999999996,"5min":2.6600000000000001,"15min":9.6110000000000007},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54500000000000004,"15min":0.58299999999999996},"min":{"1min":0.33100000000000002,"5min":0.26300000000000001,"15min":0.255},"max":{"1min":0.80700000000000005,"5min":2.6920000000000002,"15min":6.6950000000000003},"last":0.80700000000000005}]},{"osd":2,"last update":"Fri Feb 23 00:52:11 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.55400000000000005,"15min":0.57699999999999996},"min":{"1min":0.373,"5min":0.307,"15min":0.26500000000000001},"max":{"1min":2.9209999999999998,"5min":2.9209999999999998,"15min":3.3069999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55500000000000005,"15min":0.56799999999999995},"min":{"1min":0.34300000000000003,"5min":0.33600000000000002,"15min":0.32500000000000001},"max":{"1min":0.98199999999999998,"5min":1.0189999999999999,"15min":5.8440000000000003},"last":0.752}]},{"osd":3,"last update":"Fri Feb 23 00:52:11 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.69999999999999996,"15min":0.63800000000000001},"min":{"1min":0.35699999999999998,"5min":0.316,"15min":0.316},"max":{"1min":2.0710000000000002,"5min":9.1280000000000001,"15min":9.1280000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.58699999999999997,"15min":0.60099999999999998},"min":{"1min":0.36099999999999999,"5min":0.34599999999999997,"15min":0.29699999999999999},"max":{"1min":0.73799999999999999,"5min":4.0220000000000002,"15min":4.1909999999999998},"last":0.73799999999999999}]},{"osd":5,"last update":"Fri Feb 23 00:51:38 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.45400000000000001,"15min":0.42099999999999999},"min":{"1min":0.31900000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.61899999999999999,"5min":1.694,"15min":1.694},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.51600000000000001,"15min":0.47999999999999998},"min":{"1min":0.34000000000000002,"5min":0.29499999999999998,"15min":0.28899999999999998},"max":{"1min":0.73899999999999999,"5min":1.663,"15min":1.663},"last":0.51600000000000001}]},{"osd":6,"last update":"Fri Feb 23 00:52:06 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.44600000000000001,"15min":0.436},"min":{"1min":0.29099999999999998,"5min":0.28399999999999997,"15min":0.253},"max":{"1min":0.59299999999999997,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.47599999999999998,"15min":0.44900000000000001},"min":{"1min":0.31900000000000001,"5min":0.28799999999999998,"15min":0.215},"max":{"1min":1.3100000000000001,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":0.44400000000000001}]},{"osd":7,"last update":"Fri Feb 23 00:51:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.49399999999999999,"15min":0.48699999999999999},"min":{"1min":0.33100000000000002,"5min":0.27600000000000002,"15min":0.254},"max":{"1min":0.70999999999999996,"5min":2.294,"15min":2.294},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56899999999999995,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.752,"5min":2.282,"15min":2.282},"last":0.54700000000000004}]}]},{"osd":5,"up_from":39,"seq":167503724715,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308348,"kb_used_data":2932,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93436804,"statfs":{"total":95995035648,"available":95679287296,"internally_reserved":0,"allocated":3002368,"data_stored":1314994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:51:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.55300000000000005,"15min":0.61399999999999999},"min":{"1min":0.40400000000000003,"5min":0.253,"15min":0.246},"max":{"1min":0.67000000000000004,"5min":5.2220000000000004,"15min":23.881},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.57599999999999996,"15min":0.59399999999999997},"min":{"1min":0.38600000000000001,"5min":0.29099999999999998,"15min":0.27000000000000002},"max":{"1min":0.76500000000000001,"5min":3.9249999999999998,"15min":5.4649999999999999},"last":0.55800000000000005}]},{"osd":1,"last update":"Fri Feb 23 00:51:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54900000000000004,"15min":0.65600000000000003},"min":{"1min":0.35299999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.73899999999999999,"5min":3.9460000000000002,"15min":18.175999999999998},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.502,"5min":0.56899999999999995,"15min":0.628},"min":{"1min":0.40100000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.71599999999999997,"5min":3.976,"15min":9.0950000000000006},"last":0.63600000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:51:32 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57899999999999996,"15min":0.71199999999999997},"min":{"1min":0.37,"5min":0.255,"15min":0.255},"max":{"1min":1.036,"5min":4.5899999999999999,"15min":22.282},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.58599999999999997,"15min":0.72599999999999998},"min":{"1min":0.34399999999999997,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.66900000000000004,"5min":3.8540000000000001,"15min":23.925000000000001},"last":0.621}]},{"osd":3,"last update":"Fri Feb 23 00:51:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55800000000000005,"15min":0.621},"min":{"1min":0.35899999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.194,"5min":1.23,"15min":6.5730000000000004},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.60199999999999998,"15min":0.65500000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.33500000000000002},"max":{"1min":0.70499999999999996,"5min":3.9980000000000002,"15min":8.1760000000000002},"last":0.59899999999999998}]},{"osd":4,"last update":"Fri Feb 23 00:51:32 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.48199999999999998,"15min":0.436},"min":{"1min":0.33000000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.60699999999999998,"5min":4.593,"15min":4.593},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.49199999999999999,"15min":0.47399999999999998},"min":{"1min":0.35799999999999998,"5min":0.27900000000000003,"15min":0.27300000000000002},"max":{"1min":1.169,"5min":1.5860000000000001,"15min":3.879},"last":0.41299999999999998}]},{"osd":6,"last update":"Fri Feb 23 00:51:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.51300000000000001,"15min":0.47999999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.27900000000000003},"max":{"1min":1.1299999999999999,"5min":4.6740000000000004,"15min":4.6740000000000004},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.48899999999999999,"15min":0.47999999999999998},"min":{"1min":0.33400000000000002,"5min":0.31,"15min":0.27100000000000002},"max":{"1min":0.71199999999999997,"5min":1.159,"15min":1.159},"last":0.434}]},{"osd":7,"last update":"Fri Feb 23 00:51:26 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.39000000000000001,"5min":0.32500000000000001,"15min":0.30399999999999999},"max":{"1min":0.74099999999999999,"5min":1.302,"15min":7.0300000000000002},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52600000000000002,"15min":0.53900000000000003},"min":{"1min":0.36099999999999999,"5min":0.314,"15min":0.29099999999999998},"max":{"1min":0.64600000000000002,"5min":2.016,"15min":9.0579999999999998},"last":0.42499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":235,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":235,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":235,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":235,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":235,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":235,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":235,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-23T00:52:13.828 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd pool set default.rgw.log pgp_num 32 2024-02-23T00:52:14.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:12.299057+0000 osd.5 (osd.5) 262 : cluster [DBG] 6.7 scrub starts 2024-02-23T00:52:14.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:12.302828+0000 osd.5 (osd.5) 263 : cluster [DBG] 6.7 scrub ok 2024-02-23T00:52:14.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:12.657240+0000 osd.3 (osd.3) 282 : cluster [DBG] 5.10 scrub starts 2024-02-23T00:52:14.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:12.660174+0000 osd.3 (osd.3) 283 : cluster [DBG] 5.10 scrub ok 2024-02-23T00:52:14.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:12.663974+0000 osd.4 (osd.4) 253 : cluster [DBG] 4.18 scrub starts 2024-02-23T00:52:14.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:12.665409+0000 osd.4 (osd.4) 254 : cluster [DBG] 4.18 scrub ok 2024-02-23T00:52:14.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:13.276089+0000 mon.a (mon.0) 3125 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-102"}]': finished 2024-02-23T00:52:14.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:13.276184+0000 mon.a (mon.0) 3126 : cluster [DBG] osdmap e604: 8 total, 8 up, 7 in 2024-02-23T00:52:14.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:13.302973+0000 mon.c (mon.2) 339 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:14.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:13.303306+0000 mon.a (mon.0) 3127 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:14.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:12.299057+0000 osd.5 (osd.5) 262 : cluster [DBG] 6.7 scrub starts 2024-02-23T00:52:14.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:12.302828+0000 osd.5 (osd.5) 263 : cluster [DBG] 6.7 scrub ok 2024-02-23T00:52:14.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:12.657240+0000 osd.3 (osd.3) 282 : cluster [DBG] 5.10 scrub starts 2024-02-23T00:52:14.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:12.660174+0000 osd.3 (osd.3) 283 : cluster [DBG] 5.10 scrub ok 2024-02-23T00:52:14.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:12.663974+0000 osd.4 (osd.4) 253 : cluster [DBG] 4.18 scrub starts 2024-02-23T00:52:14.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:12.665409+0000 osd.4 (osd.4) 254 : cluster [DBG] 4.18 scrub ok 2024-02-23T00:52:14.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:13.276089+0000 mon.a (mon.0) 3125 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-102"}]': finished 2024-02-23T00:52:14.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:13.276184+0000 mon.a (mon.0) 3126 : cluster [DBG] osdmap e604: 8 total, 8 up, 7 in 2024-02-23T00:52:14.060 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:13.302973+0000 mon.c (mon.2) 339 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:14.061 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:13.303306+0000 mon.a (mon.0) 3127 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:14.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:12.299057+0000 osd.5 (osd.5) 262 : cluster [DBG] 6.7 scrub starts 2024-02-23T00:52:14.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:12.302828+0000 osd.5 (osd.5) 263 : cluster [DBG] 6.7 scrub ok 2024-02-23T00:52:14.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:12.657240+0000 osd.3 (osd.3) 282 : cluster [DBG] 5.10 scrub starts 2024-02-23T00:52:14.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:12.660174+0000 osd.3 (osd.3) 283 : cluster [DBG] 5.10 scrub ok 2024-02-23T00:52:14.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:12.663974+0000 osd.4 (osd.4) 253 : cluster [DBG] 4.18 scrub starts 2024-02-23T00:52:14.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:12.665409+0000 osd.4 (osd.4) 254 : cluster [DBG] 4.18 scrub ok 2024-02-23T00:52:14.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:13.276089+0000 mon.a (mon.0) 3125 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-102"}]': finished 2024-02-23T00:52:14.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:13.276184+0000 mon.a (mon.0) 3126 : cluster [DBG] osdmap e604: 8 total, 8 up, 7 in 2024-02-23T00:52:14.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:13.302973+0000 mon.c (mon.2) 339 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:14.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:13.303306+0000 mon.a (mon.0) 3127 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:14.413 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:52:14.555 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:52:14.557 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:14.558 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:52:14.595 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:52:14.619 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.1 dump_ops_in_flight 2024-02-23T00:52:14.659 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:52:14.688 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:14.714 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:52:14.718 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:14.719 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:52:14.819 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:52:14.856 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:12.794173+0000 mgr.y (mgr.14517) 538 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:52:14.857 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:13.081872+0000 mgr.y (mgr.14517) 539 : audit [DBG] from='client.31458 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:52:14.857 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:13.353345+0000 mgr.y (mgr.14517) 540 : cluster [DBG] pgmap v910: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-23T00:52:14.857 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:13.439432+0000 osd.7 (osd.7) 287 : cluster [DBG] 5.d scrub starts 2024-02-23T00:52:14.857 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:13.442889+0000 osd.7 (osd.7) 288 : cluster [DBG] 5.d scrub ok 2024-02-23T00:52:14.857 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:13.527513+0000 osd.1 (osd.1) 251 : cluster [DBG] 2.a scrub starts 2024-02-23T00:52:14.857 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:13.530622+0000 osd.1 (osd.1) 252 : cluster [DBG] 2.a scrub ok 2024-02-23T00:52:14.857 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:13.667584+0000 osd.4 (osd.4) 255 : cluster [DBG] 5.1c scrub starts 2024-02-23T00:52:14.857 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:13.668462+0000 osd.4 (osd.4) 256 : cluster [DBG] 5.1c scrub ok 2024-02-23T00:52:14.857 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:14.273238+0000 mon.a (mon.0) 3128 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:52:14.857 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:14.280737+0000 mon.a (mon.0) 3129 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:52:14.857 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:14.280813+0000 mon.a (mon.0) 3130 : cluster [DBG] osdmap e605: 8 total, 8 up, 7 in 2024-02-23T00:52:14.858 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:14.282056+0000 mon.c (mon.2) 340 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-102"}]: dispatch 2024-02-23T00:52:14.858 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:14.282639+0000 mon.a (mon.0) 3131 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-102"}]: dispatch 2024-02-23T00:52:14.858 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:14.444818+0000 mon.a (mon.0) 3132 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:52:14.858 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:12.794173+0000 mgr.y (mgr.14517) 538 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:52:14.858 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:13.081872+0000 mgr.y (mgr.14517) 539 : audit [DBG] from='client.31458 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:52:14.858 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:13.353345+0000 mgr.y (mgr.14517) 540 : cluster [DBG] pgmap v910: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-23T00:52:14.858 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:13.439432+0000 osd.7 (osd.7) 287 : cluster [DBG] 5.d scrub starts 2024-02-23T00:52:14.858 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:13.442889+0000 osd.7 (osd.7) 288 : cluster [DBG] 5.d scrub ok 2024-02-23T00:52:14.858 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:13.527513+0000 osd.1 (osd.1) 251 : cluster [DBG] 2.a scrub starts 2024-02-23T00:52:14.858 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:13.530622+0000 osd.1 (osd.1) 252 : cluster [DBG] 2.a scrub ok 2024-02-23T00:52:14.858 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:13.667584+0000 osd.4 (osd.4) 255 : cluster [DBG] 5.1c scrub starts 2024-02-23T00:52:14.858 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:13.668462+0000 osd.4 (osd.4) 256 : cluster [DBG] 5.1c scrub ok 2024-02-23T00:52:14.859 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:14.273238+0000 mon.a (mon.0) 3128 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:52:14.859 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:14.280737+0000 mon.a (mon.0) 3129 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:52:14.859 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:14.280813+0000 mon.a (mon.0) 3130 : cluster [DBG] osdmap e605: 8 total, 8 up, 7 in 2024-02-23T00:52:14.859 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:14.282056+0000 mon.c (mon.2) 340 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-102"}]: dispatch 2024-02-23T00:52:14.859 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:14.282639+0000 mon.a (mon.0) 3131 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-102"}]: dispatch 2024-02-23T00:52:14.859 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:14.444818+0000 mon.a (mon.0) 3132 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:52:14.859 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:14.960 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:52:14.963 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:14.964 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:52:15.064 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:52:15.096 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:12.794173+0000 mgr.y (mgr.14517) 538 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:52:15.096 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:13.081872+0000 mgr.y (mgr.14517) 539 : audit [DBG] from='client.31458 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:52:15.096 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:13.353345+0000 mgr.y (mgr.14517) 540 : cluster [DBG] pgmap v910: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-23T00:52:15.096 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:13.439432+0000 osd.7 (osd.7) 287 : cluster [DBG] 5.d scrub starts 2024-02-23T00:52:15.096 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:13.442889+0000 osd.7 (osd.7) 288 : cluster [DBG] 5.d scrub ok 2024-02-23T00:52:15.096 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:13.527513+0000 osd.1 (osd.1) 251 : cluster [DBG] 2.a scrub starts 2024-02-23T00:52:15.096 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:13.530622+0000 osd.1 (osd.1) 252 : cluster [DBG] 2.a scrub ok 2024-02-23T00:52:15.096 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:13.667584+0000 osd.4 (osd.4) 255 : cluster [DBG] 5.1c scrub starts 2024-02-23T00:52:15.096 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:13.668462+0000 osd.4 (osd.4) 256 : cluster [DBG] 5.1c scrub ok 2024-02-23T00:52:15.096 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:14.273238+0000 mon.a (mon.0) 3128 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:52:15.096 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:14.280737+0000 mon.a (mon.0) 3129 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:52:15.097 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:14.280813+0000 mon.a (mon.0) 3130 : cluster [DBG] osdmap e605: 8 total, 8 up, 7 in 2024-02-23T00:52:15.097 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:14.282056+0000 mon.c (mon.2) 340 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-102"}]: dispatch 2024-02-23T00:52:15.097 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:14.282639+0000 mon.a (mon.0) 3131 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-102"}]: dispatch 2024-02-23T00:52:15.097 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:14.444818+0000 mon.a (mon.0) 3132 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:52:15.098 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:15.119 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:52:15.122 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:15.122 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:52:15.223 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:52:15.256 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:15.280 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:52:15.283 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:15.283 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:52:15.384 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:52:15.414 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:15.612 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:52:15.615 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:15.616 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:52:15.717 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:52:15.749 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:16.003 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:14.515065+0000 osd.1 (osd.1) 253 : cluster [DBG] 4.2 scrub starts 2024-02-23T00:52:16.003 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:14.519045+0000 osd.1 (osd.1) 254 : cluster [DBG] 4.2 scrub ok 2024-02-23T00:52:16.003 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:14.835315+0000 mon.a (mon.0) 3133 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:52:16.004 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:14.836384+0000 mon.a (mon.0) 3134 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:52:16.004 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:14.845700+0000 mon.a (mon.0) 3135 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:52:16.004 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:15.256534+0000 mon.a (mon.0) 3136 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-102"}]': finished 2024-02-23T00:52:16.004 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:15.256609+0000 mon.a (mon.0) 3137 : cluster [DBG] osdmap e606: 8 total, 8 up, 7 in 2024-02-23T00:52:16.004 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:15.353827+0000 mgr.y (mgr.14517) 541 : cluster [DBG] pgmap v913: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-23T00:52:16.004 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:14.515065+0000 osd.1 (osd.1) 253 : cluster [DBG] 4.2 scrub starts 2024-02-23T00:52:16.004 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:14.519045+0000 osd.1 (osd.1) 254 : cluster [DBG] 4.2 scrub ok 2024-02-23T00:52:16.004 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:14.835315+0000 mon.a (mon.0) 3133 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:52:16.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:14.836384+0000 mon.a (mon.0) 3134 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:52:16.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:14.845700+0000 mon.a (mon.0) 3135 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:52:16.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:15.256534+0000 mon.a (mon.0) 3136 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-102"}]': finished 2024-02-23T00:52:16.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:15.256609+0000 mon.a (mon.0) 3137 : cluster [DBG] osdmap e606: 8 total, 8 up, 7 in 2024-02-23T00:52:16.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:15.353827+0000 mgr.y (mgr.14517) 541 : cluster [DBG] pgmap v913: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-23T00:52:16.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:14.515065+0000 osd.1 (osd.1) 253 : cluster [DBG] 4.2 scrub starts 2024-02-23T00:52:16.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:14.519045+0000 osd.1 (osd.1) 254 : cluster [DBG] 4.2 scrub ok 2024-02-23T00:52:16.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:14.835315+0000 mon.a (mon.0) 3133 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:52:16.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:14.836384+0000 mon.a (mon.0) 3134 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:52:16.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:14.845700+0000 mon.a (mon.0) 3135 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:52:16.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:15.256534+0000 mon.a (mon.0) 3136 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-102"}]': finished 2024-02-23T00:52:16.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:15.256609+0000 mon.a (mon.0) 3137 : cluster [DBG] osdmap e606: 8 total, 8 up, 7 in 2024-02-23T00:52:16.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:15.353827+0000 mgr.y (mgr.14517) 541 : cluster [DBG] pgmap v913: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-23T00:52:16.786 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:52:16.997 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:15.447858+0000 osd.7 (osd.7) 289 : cluster [DBG] 5.9 scrub starts 2024-02-23T00:52:16.997 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:15.451328+0000 osd.7 (osd.7) 290 : cluster [DBG] 5.9 scrub ok 2024-02-23T00:52:16.997 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:15.679440+0000 osd.4 (osd.4) 257 : cluster [DBG] 6.2 scrub starts 2024-02-23T00:52:16.997 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:15.680329+0000 osd.4 (osd.4) 258 : cluster [DBG] 6.2 scrub ok 2024-02-23T00:52:16.997 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:16.266698+0000 mon.a (mon.0) 3138 : cluster [DBG] osdmap e607: 8 total, 8 up, 7 in 2024-02-23T00:52:16.997 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:16.686356+0000 osd.0 (osd.0) 249 : cluster [DBG] 4.6 scrub starts 2024-02-23T00:52:16.997 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:16.690855+0000 osd.0 (osd.0) 250 : cluster [DBG] 4.6 scrub ok 2024-02-23T00:52:16.998 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:15.447858+0000 osd.7 (osd.7) 289 : cluster [DBG] 5.9 scrub starts 2024-02-23T00:52:16.998 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:15.451328+0000 osd.7 (osd.7) 290 : cluster [DBG] 5.9 scrub ok 2024-02-23T00:52:16.998 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:15.679440+0000 osd.4 (osd.4) 257 : cluster [DBG] 6.2 scrub starts 2024-02-23T00:52:16.998 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:15.680329+0000 osd.4 (osd.4) 258 : cluster [DBG] 6.2 scrub ok 2024-02-23T00:52:16.998 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:16.266698+0000 mon.a (mon.0) 3138 : cluster [DBG] osdmap e607: 8 total, 8 up, 7 in 2024-02-23T00:52:16.998 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:16.686356+0000 osd.0 (osd.0) 249 : cluster [DBG] 4.6 scrub starts 2024-02-23T00:52:16.998 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:16.690855+0000 osd.0 (osd.0) 250 : cluster [DBG] 4.6 scrub ok 2024-02-23T00:52:16.998 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:52:17.001 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:17.001 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:52:17.103 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:52:17.133 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:17.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:15.447858+0000 osd.7 (osd.7) 289 : cluster [DBG] 5.9 scrub starts 2024-02-23T00:52:17.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:15.451328+0000 osd.7 (osd.7) 290 : cluster [DBG] 5.9 scrub ok 2024-02-23T00:52:17.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:15.679440+0000 osd.4 (osd.4) 257 : cluster [DBG] 6.2 scrub starts 2024-02-23T00:52:17.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:15.680329+0000 osd.4 (osd.4) 258 : cluster [DBG] 6.2 scrub ok 2024-02-23T00:52:17.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:16.266698+0000 mon.a (mon.0) 3138 : cluster [DBG] osdmap e607: 8 total, 8 up, 7 in 2024-02-23T00:52:17.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:16.686356+0000 osd.0 (osd.0) 249 : cluster [DBG] 4.6 scrub starts 2024-02-23T00:52:17.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:16.690855+0000 osd.0 (osd.0) 250 : cluster [DBG] 4.6 scrub ok 2024-02-23T00:52:17.231 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:52:17.234 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:17.234 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:52:17.335 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:52:17.367 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:17.388 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:52:17.390 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:17.391 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:52:17.491 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:52:17.521 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:17.541 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:52:17.544 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:17.545 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:52:17.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.495+0000 7fa72f2bd700 1 -- 172.21.15.47:0/1262956738 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa7280f8420 msgr2=0x7fa728104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:17.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.495+0000 7fa72f2bd700 1 --2- 172.21.15.47:0/1262956738 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa7280f8420 0x7fa728104a20 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 rx=0x7fa720009c30 tx=0x7fa720004c50).stop 2024-02-23T00:52:17.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.495+0000 7fa72f2bd700 1 -- 172.21.15.47:0/1262956738 shutdown_connections 2024-02-23T00:52:17.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.495+0000 7fa72f2bd700 1 --2- 172.21.15.47:0/1262956738 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa7280f8420 0x7fa728104a20 unknown :-1 s=CLOSED pgs=2751 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:17.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.495+0000 7fa72f2bd700 1 --2- 172.21.15.47:0/1262956738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7280fdbb0 0x7fa7280fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:17.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.495+0000 7fa72f2bd700 1 --2- 172.21.15.47:0/1262956738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7280fe510 0x7fa7280f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:17.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.495+0000 7fa72f2bd700 1 -- 172.21.15.47:0/1262956738 >> 172.21.15.47:0/1262956738 conn(0x7fa7280f3800 msgr2=0x7fa7280f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:52:17.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.495+0000 7fa72f2bd700 1 -- 172.21.15.47:0/1262956738 shutdown_connections 2024-02-23T00:52:17.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.495+0000 7fa72f2bd700 1 -- 172.21.15.47:0/1262956738 wait complete. 2024-02-23T00:52:17.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.496+0000 7fa72f2bd700 1 librados: starting msgr at 2024-02-23T00:52:17.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.496+0000 7fa72f2bd700 1 librados: starting objecter 2024-02-23T00:52:17.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.496+0000 7fa72f2bd700 1 Processor -- start 2024-02-23T00:52:17.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.496+0000 7fa72f2bd700 1 -- start start 2024-02-23T00:52:17.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.496+0000 7fa72f2bd700 1 librados: setting wanted keys 2024-02-23T00:52:17.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.496+0000 7fa72f2bd700 1 librados: calling monclient init 2024-02-23T00:52:17.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.497+0000 7fa72f2bd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7280f8420 0x7fa72810dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:17.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.497+0000 7fa72f2bd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa7280fdbb0 0x7fa72810e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:17.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.497+0000 7fa72f2bd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7280fe510 0x7fa728107f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:17.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.497+0000 7fa72f2bd700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa728108430 con 0x7fa7280fe510 2024-02-23T00:52:17.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.497+0000 7fa72f2bd700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7281085a0 con 0x7fa7280f8420 2024-02-23T00:52:17.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.497+0000 7fa72f2bd700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa728108760 con 0x7fa7280fdbb0 2024-02-23T00:52:17.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.497+0000 7fa72c858700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa7280fdbb0 0x7fa72810e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:17.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.497+0000 7fa72d85a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7280fe510 0x7fa728107f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:17.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.497+0000 7fa72c858700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa7280fdbb0 0x7fa72810e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:42928/0 (socket says 172.21.15.47:42928) 2024-02-23T00:52:17.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.497+0000 7fa72d85a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7280fe510 0x7fa728107f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39894/0 (socket says 172.21.15.47:39894) 2024-02-23T00:52:17.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.497+0000 7fa72c858700 1 -- 172.21.15.47:0/2912854677 learned_addr learned my addr 172.21.15.47:0/2912854677 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:52:17.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.497+0000 7fa72d059700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7280f8420 0x7fa72810dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:17.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.497+0000 7fa72c858700 1 -- 172.21.15.47:0/2912854677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7280f8420 msgr2=0x7fa72810dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:17.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.497+0000 7fa72c858700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7280f8420 0x7fa72810dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:17.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.497+0000 7fa72c858700 1 -- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7280fe510 msgr2=0x7fa728107f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:17.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.497+0000 7fa72c858700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7280fe510 0x7fa728107f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:17.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.497+0000 7fa72c858700 1 -- 172.21.15.47:0/2912854677 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa724005040 con 0x7fa7280fdbb0 2024-02-23T00:52:17.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.497+0000 7fa72d85a700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7280fe510 0x7fa728107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:52:17.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.497+0000 7fa72c858700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa7280fdbb0 0x7fa72810e290 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 rx=0x7fa71800e990 tx=0x7fa71800b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:17.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.497+0000 7fa71e777700 10 client.?.objecter ms_handle_connect 0x7fa7280fdbb0 2024-02-23T00:52:17.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.497+0000 7fa71e777700 10 client.?.objecter resend_mon_ops 2024-02-23T00:52:17.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.497+0000 7fa71e777700 1 -- 172.21.15.47:0/2912854677 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa718004680 con 0x7fa7280fdbb0 2024-02-23T00:52:17.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.497+0000 7fa71e777700 1 -- 172.21.15.47:0/2912854677 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fa71800f040 con 0x7fa7280fdbb0 2024-02-23T00:52:17.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.497+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7200097c0 con 0x7fa7280fdbb0 2024-02-23T00:52:17.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.498+0000 7fa71e777700 1 -- 172.21.15.47:0/2912854677 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7180146f0 con 0x7fa7280fdbb0 2024-02-23T00:52:17.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.498+0000 7fa72f2bd700 10 client.41783.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:52:17.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.498+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa728108d10 con 0x7fa7280fdbb0 2024-02-23T00:52:17.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.498+0000 7fa72f2bd700 1 librados: init done 2024-02-23T00:52:17.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.498+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fa72805ee80 con 0x7fa7280fdbb0 2024-02-23T00:52:17.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.499+0000 7fa71e777700 1 -- 172.21.15.47:0/2912854677 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fa718007b40 con 0x7fa7280fdbb0 2024-02-23T00:52:17.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.499+0000 7fa71e777700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa714062cc0 0x7fa714065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:17.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.500+0000 7fa71e777700 1 -- 172.21.15.47:0/2912854677 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(608..608 src has 482..608) v4 ==== 9570+0+0 (secure 0 0 0) 0x7fa71807d670 con 0x7fa7280fdbb0 2024-02-23T00:52:17.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.500+0000 7fa71e777700 10 client.41783.objecter ms_dispatch 0x7fa728001550 osd_map(608..608 src has 482..608) v4 2024-02-23T00:52:17.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.500+0000 7fa71e777700 3 client.41783.objecter handle_osd_map got epochs [608,608] > 0 2024-02-23T00:52:17.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.500+0000 7fa71e777700 3 client.41783.objecter handle_osd_map decoding full epoch 608 2024-02-23T00:52:17.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.500+0000 7fa72d059700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa714062cc0 0x7fa714065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:17.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.500+0000 7fa71e777700 20 client.41783.objecter dump_active .. 0 homeless 2024-02-23T00:52:17.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.500+0000 7fa71e777700 1 -- 172.21.15.47:0/2912854677 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v608) v1 ==== 54+0+16 (secure 0 0 0) 0x7fa71807db10 con 0x7fa7280fdbb0 2024-02-23T00:52:17.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.500+0000 7fa72f2bd700 10 client.41783.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:17.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.500+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_assign -1 1 2024-02-23T00:52:17.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.500+0000 7fa72f2bd700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fa7280fee80 0x7fa7280ff260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:17.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.500+0000 7fa72f2bd700 20 client.41783.objecter _get_session s=0x7fa728106ed0 osd=0 3 2024-02-23T00:52:17.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.500+0000 7fa71e777700 10 client.41783.objecter ms_handle_connect 0x7fa714062cc0 2024-02-23T00:52:17.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.500+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa728106ed0 osd=0 3 2024-02-23T00:52:17.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.500+0000 7fa72f2bd700 20 client.41783.objecter _get_session s=0x7fa728106ed0 osd=0 3 2024-02-23T00:52:17.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.500+0000 7fa72d059700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa714062cc0 0x7fa714065180 secure :-1 s=READY pgs=3140 cs=0 l=1 rev1=1 rx=0x7fa724004870 tx=0x7fa72400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:17.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.500+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_remove -1 1 2024-02-23T00:52:17.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.500+0000 7fa72f2bd700 20 client.41783.objecter get_session s=0x7fa728106ed0 osd=0 3 2024-02-23T00:52:17.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.500+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_assign 0 1 2024-02-23T00:52:17.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.500+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa728106ed0 osd=0 4 2024-02-23T00:52:17.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.500+0000 7fa72f2bd700 10 client.41783.objecter _send_command 1 2024-02-23T00:52:17.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.500+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa72806cf20 con 0x7fa7280fee80 2024-02-23T00:52:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.500+0000 7fa72d85a700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fa7280fee80 0x7fa7280ff260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.500+0000 7fa72d85a700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fa7280fee80 0x7fa7280ff260 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.500+0000 7fa71e777700 10 client.41783.objecter ms_handle_connect 0x7fa7280fee80 2024-02-23T00:52:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.501+0000 7fa71e777700 1 -- 172.21.15.47:0/2912854677 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa72806cf20 con 0x7fa7280fee80 2024-02-23T00:52:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.501+0000 7fa71e777700 10 client.41783.objecter ms_dispatch 0x7fa728001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:52:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.501+0000 7fa71e777700 10 client.41783.objecter _finish_command 1 = system:0 2024-02-23T00:52:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.501+0000 7fa71e777700 20 client.41783.objecter put_session s=0x7fa728106ed0 osd=0 4 2024-02-23T00:52:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.501+0000 7fa71e777700 15 client.41783.objecter _session_command_op_remove 0 1 2024-02-23T00:52:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.516+0000 7fa72f2bd700 10 client.41783.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:52:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.516+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_assign -1 2 2024-02-23T00:52:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.516+0000 7fa72f2bd700 20 client.41783.objecter _get_session s=0x7fa728106ed0 osd=0 3 2024-02-23T00:52:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.516+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa728106ed0 osd=0 3 2024-02-23T00:52:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.516+0000 7fa72f2bd700 20 client.41783.objecter _get_session s=0x7fa728106ed0 osd=0 3 2024-02-23T00:52:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.516+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_remove -1 2 2024-02-23T00:52:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.516+0000 7fa72f2bd700 20 client.41783.objecter get_session s=0x7fa728106ed0 osd=0 3 2024-02-23T00:52:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.516+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_assign 0 2 2024-02-23T00:52:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.516+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa728106ed0 osd=0 4 2024-02-23T00:52:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.516+0000 7fa72f2bd700 10 client.41783.objecter _send_command 2 2024-02-23T00:52:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.516+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa7280ff750 con 0x7fa7280fee80 2024-02-23T00:52:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.518+0000 7fa71e777700 1 -- 172.21.15.47:0/2912854677 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa7280ff750 con 0x7fa7280fee80 2024-02-23T00:52:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.518+0000 7fa71e777700 10 client.41783.objecter ms_dispatch 0x7fa728001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:52:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.518+0000 7fa71e777700 10 client.41783.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:52:17.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.518+0000 7fa71e777700 20 client.41783.objecter put_session s=0x7fa728106ed0 osd=0 4 2024-02-23T00:52:17.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.518+0000 7fa71e777700 15 client.41783.objecter _session_command_op_remove 0 2 2024-02-23T00:52:17.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.518+0000 7fa72f2bd700 10 client.41783.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:17.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.518+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_assign -1 3 2024-02-23T00:52:17.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.518+0000 7fa72f2bd700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fa728028540 0x7fa7280598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:17.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.518+0000 7fa72f2bd700 20 client.41783.objecter _get_session s=0x7fa7280ff920 osd=1 3 2024-02-23T00:52:17.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.518+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa7280ff920 osd=1 3 2024-02-23T00:52:17.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.518+0000 7fa72f2bd700 20 client.41783.objecter _get_session s=0x7fa7280ff920 osd=1 3 2024-02-23T00:52:17.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.518+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_remove -1 3 2024-02-23T00:52:17.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.518+0000 7fa72f2bd700 20 client.41783.objecter get_session s=0x7fa7280ff920 osd=1 3 2024-02-23T00:52:17.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.518+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_assign 1 3 2024-02-23T00:52:17.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.518+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa7280ff920 osd=1 4 2024-02-23T00:52:17.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.518+0000 7fa72f2bd700 10 client.41783.objecter _send_command 3 2024-02-23T00:52:17.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.518+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa728059de0 con 0x7fa728028540 2024-02-23T00:52:17.591 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:52:17.591 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:52:17.592 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:52:17.592 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:52:17.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.518+0000 7fa72d059700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fa728028540 0x7fa7280598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:17.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.518+0000 7fa71e777700 10 client.41783.objecter ms_handle_connect 0x7fa728028540 2024-02-23T00:52:17.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.518+0000 7fa72d059700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fa728028540 0x7fa7280598f0 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:17.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.519+0000 7fa71e777700 1 -- 172.21.15.47:0/2912854677 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa728059de0 con 0x7fa728028540 2024-02-23T00:52:17.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.519+0000 7fa71e777700 10 client.41783.objecter ms_dispatch 0x7fa728001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:52:17.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.519+0000 7fa71e777700 10 client.41783.objecter _finish_command 3 = system:0 2024-02-23T00:52:17.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.519+0000 7fa71e777700 20 client.41783.objecter put_session s=0x7fa7280ff920 osd=1 4 2024-02-23T00:52:17.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.519+0000 7fa71e777700 15 client.41783.objecter _session_command_op_remove 1 3 2024-02-23T00:52:17.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.534+0000 7fa72f2bd700 10 client.41783.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:52:17.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.534+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_assign -1 4 2024-02-23T00:52:17.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.534+0000 7fa72f2bd700 20 client.41783.objecter _get_session s=0x7fa7280ff920 osd=1 3 2024-02-23T00:52:17.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.534+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa7280ff920 osd=1 3 2024-02-23T00:52:17.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.534+0000 7fa72f2bd700 20 client.41783.objecter _get_session s=0x7fa7280ff920 osd=1 3 2024-02-23T00:52:17.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.534+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_remove -1 4 2024-02-23T00:52:17.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.534+0000 7fa72f2bd700 20 client.41783.objecter get_session s=0x7fa7280ff920 osd=1 3 2024-02-23T00:52:17.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.534+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_assign 1 4 2024-02-23T00:52:17.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.534+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa7280ff920 osd=1 4 2024-02-23T00:52:17.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.534+0000 7fa72f2bd700 10 client.41783.objecter _send_command 4 2024-02-23T00:52:17.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.534+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa728116be0 con 0x7fa728028540 2024-02-23T00:52:17.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.535+0000 7fa71e777700 1 -- 172.21.15.47:0/2912854677 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa728116be0 con 0x7fa728028540 2024-02-23T00:52:17.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.535+0000 7fa71e777700 10 client.41783.objecter ms_dispatch 0x7fa728001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:52:17.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.535+0000 7fa71e777700 10 client.41783.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:52:17.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.535+0000 7fa71e777700 20 client.41783.objecter put_session s=0x7fa7280ff920 osd=1 4 2024-02-23T00:52:17.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.535+0000 7fa71e777700 15 client.41783.objecter _session_command_op_remove 1 4 2024-02-23T00:52:17.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.536+0000 7fa72f2bd700 10 client.41783.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:17.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.536+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_assign -1 5 2024-02-23T00:52:17.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.536+0000 7fa72f2bd700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa7280594d0 0x7fa728119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:17.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.536+0000 7fa72f2bd700 20 client.41783.objecter _get_session s=0x7fa728059360 osd=2 3 2024-02-23T00:52:17.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.536+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa728059360 osd=2 3 2024-02-23T00:52:17.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.536+0000 7fa72f2bd700 20 client.41783.objecter _get_session s=0x7fa728059360 osd=2 3 2024-02-23T00:52:17.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.536+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_remove -1 5 2024-02-23T00:52:17.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.536+0000 7fa72f2bd700 20 client.41783.objecter get_session s=0x7fa728059360 osd=2 3 2024-02-23T00:52:17.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.536+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_assign 2 5 2024-02-23T00:52:17.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.536+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa728059360 osd=2 4 2024-02-23T00:52:17.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.536+0000 7fa72f2bd700 10 client.41783.objecter _send_command 5 2024-02-23T00:52:17.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.536+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa7280282b0 con 0x7fa7280594d0 2024-02-23T00:52:17.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.536+0000 7fa72c858700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa7280594d0 0x7fa728119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:17.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.536+0000 7fa71e777700 10 client.41783.objecter ms_handle_connect 0x7fa7280594d0 2024-02-23T00:52:17.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.536+0000 7fa72c858700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa7280594d0 0x7fa728119f40 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:17.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.537+0000 7fa71e777700 1 -- 172.21.15.47:0/2912854677 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa7280282b0 con 0x7fa7280594d0 2024-02-23T00:52:17.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.537+0000 7fa71e777700 10 client.41783.objecter ms_dispatch 0x7fa728001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:52:17.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.537+0000 7fa71e777700 10 client.41783.objecter _finish_command 5 = system:0 2024-02-23T00:52:17.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.537+0000 7fa71e777700 20 client.41783.objecter put_session s=0x7fa728059360 osd=2 4 2024-02-23T00:52:17.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.537+0000 7fa71e777700 15 client.41783.objecter _session_command_op_remove 2 5 2024-02-23T00:52:17.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.552+0000 7fa72f2bd700 10 client.41783.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:52:17.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.552+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_assign -1 6 2024-02-23T00:52:17.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.552+0000 7fa72f2bd700 20 client.41783.objecter _get_session s=0x7fa728059360 osd=2 3 2024-02-23T00:52:17.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.552+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa728059360 osd=2 3 2024-02-23T00:52:17.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.552+0000 7fa72f2bd700 20 client.41783.objecter _get_session s=0x7fa728059360 osd=2 3 2024-02-23T00:52:17.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.552+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_remove -1 6 2024-02-23T00:52:17.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.552+0000 7fa72f2bd700 20 client.41783.objecter get_session s=0x7fa728059360 osd=2 3 2024-02-23T00:52:17.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.552+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_assign 2 6 2024-02-23T00:52:17.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.552+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa728059360 osd=2 4 2024-02-23T00:52:17.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.552+0000 7fa72f2bd700 10 client.41783.objecter _send_command 6 2024-02-23T00:52:17.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.552+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa728027600 con 0x7fa7280594d0 2024-02-23T00:52:17.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.554+0000 7fa71e777700 1 -- 172.21.15.47:0/2912854677 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa728027600 con 0x7fa7280594d0 2024-02-23T00:52:17.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.554+0000 7fa71e777700 10 client.41783.objecter ms_dispatch 0x7fa728001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:52:17.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.554+0000 7fa71e777700 10 client.41783.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:52:17.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.554+0000 7fa71e777700 20 client.41783.objecter put_session s=0x7fa728059360 osd=2 4 2024-02-23T00:52:17.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.554+0000 7fa71e777700 15 client.41783.objecter _session_command_op_remove 2 6 2024-02-23T00:52:17.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.554+0000 7fa72f2bd700 10 client.41783.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:17.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.554+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_assign -1 7 2024-02-23T00:52:17.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.554+0000 7fa72f2bd700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fa728117a30 0x7fa72811d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:17.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.554+0000 7fa72f2bd700 20 client.41783.objecter _get_session s=0x7fa728027a00 osd=3 3 2024-02-23T00:52:17.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.554+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa728027a00 osd=3 3 2024-02-23T00:52:17.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.554+0000 7fa72f2bd700 20 client.41783.objecter _get_session s=0x7fa728027a00 osd=3 3 2024-02-23T00:52:17.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.554+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_remove -1 7 2024-02-23T00:52:17.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.554+0000 7fa72f2bd700 20 client.41783.objecter get_session s=0x7fa728027a00 osd=3 3 2024-02-23T00:52:17.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.554+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_assign 3 7 2024-02-23T00:52:17.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.554+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa728027a00 osd=3 4 2024-02-23T00:52:17.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.554+0000 7fa72f2bd700 10 client.41783.objecter _send_command 7 2024-02-23T00:52:17.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.554+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa728120320 con 0x7fa728117a30 2024-02-23T00:52:17.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.554+0000 7fa72d85a700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fa728117a30 0x7fa72811d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:17.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.554+0000 7fa71e777700 10 client.41783.objecter ms_handle_connect 0x7fa728117a30 2024-02-23T00:52:17.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.554+0000 7fa72d85a700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fa728117a30 0x7fa72811d5c0 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:17.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.555+0000 7fa71e777700 1 -- 172.21.15.47:0/2912854677 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa728120320 con 0x7fa728117a30 2024-02-23T00:52:17.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.555+0000 7fa71e777700 10 client.41783.objecter ms_dispatch 0x7fa728001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:52:17.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.555+0000 7fa71e777700 10 client.41783.objecter _finish_command 7 = system:0 2024-02-23T00:52:17.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.555+0000 7fa71e777700 20 client.41783.objecter put_session s=0x7fa728027a00 osd=3 4 2024-02-23T00:52:17.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.555+0000 7fa71e777700 15 client.41783.objecter _session_command_op_remove 3 7 2024-02-23T00:52:17.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.569+0000 7fa72f2bd700 10 client.41783.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:52:17.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.569+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_assign -1 8 2024-02-23T00:52:17.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.569+0000 7fa72f2bd700 20 client.41783.objecter _get_session s=0x7fa728027a00 osd=3 3 2024-02-23T00:52:17.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.569+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa728027a00 osd=3 3 2024-02-23T00:52:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.569+0000 7fa72f2bd700 20 client.41783.objecter _get_session s=0x7fa728027a00 osd=3 3 2024-02-23T00:52:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.569+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_remove -1 8 2024-02-23T00:52:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.569+0000 7fa72f2bd700 20 client.41783.objecter get_session s=0x7fa728027a00 osd=3 3 2024-02-23T00:52:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.569+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_assign 3 8 2024-02-23T00:52:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.569+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa728027a00 osd=3 4 2024-02-23T00:52:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.569+0000 7fa72f2bd700 10 client.41783.objecter _send_command 8 2024-02-23T00:52:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.569+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa728116f90 con 0x7fa728117a30 2024-02-23T00:52:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.571+0000 7fa71e777700 1 -- 172.21.15.47:0/2912854677 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa728116f90 con 0x7fa728117a30 2024-02-23T00:52:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.571+0000 7fa71e777700 10 client.41783.objecter ms_dispatch 0x7fa728001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:52:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.571+0000 7fa71e777700 10 client.41783.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:52:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.571+0000 7fa71e777700 20 client.41783.objecter put_session s=0x7fa728027a00 osd=3 4 2024-02-23T00:52:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.571+0000 7fa71e777700 15 client.41783.objecter _session_command_op_remove 3 8 2024-02-23T00:52:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.571+0000 7fa72f2bd700 10 client.41783.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.571+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_assign -1 9 2024-02-23T00:52:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.571+0000 7fa72f2bd700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fa728117130 0x7fa72811db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.571+0000 7fa72f2bd700 20 client.41783.objecter _get_session s=0x7fa728116f90 osd=4 3 2024-02-23T00:52:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.571+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa728116f90 osd=4 3 2024-02-23T00:52:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.571+0000 7fa72f2bd700 20 client.41783.objecter _get_session s=0x7fa728116f90 osd=4 3 2024-02-23T00:52:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.571+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_remove -1 9 2024-02-23T00:52:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.571+0000 7fa72f2bd700 20 client.41783.objecter get_session s=0x7fa728116f90 osd=4 3 2024-02-23T00:52:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.571+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_assign 4 9 2024-02-23T00:52:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.571+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa728116f90 osd=4 4 2024-02-23T00:52:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.571+0000 7fa72f2bd700 10 client.41783.objecter _send_command 9 2024-02-23T00:52:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.571+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa72811b360 con 0x7fa728117130 2024-02-23T00:52:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.571+0000 7fa72d059700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fa728117130 0x7fa72811db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.572+0000 7fa71e777700 10 client.41783.objecter ms_handle_connect 0x7fa728117130 2024-02-23T00:52:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.572+0000 7fa72d059700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fa728117130 0x7fa72811db00 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.572+0000 7fa71e777700 1 -- 172.21.15.47:0/2912854677 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa72811b360 con 0x7fa728117130 2024-02-23T00:52:17.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.572+0000 7fa71e777700 10 client.41783.objecter ms_dispatch 0x7fa728001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:52:17.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.572+0000 7fa71e777700 10 client.41783.objecter _finish_command 9 = system:0 2024-02-23T00:52:17.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.572+0000 7fa71e777700 20 client.41783.objecter put_session s=0x7fa728116f90 osd=4 4 2024-02-23T00:52:17.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.572+0000 7fa71e777700 15 client.41783.objecter _session_command_op_remove 4 9 2024-02-23T00:52:17.601 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:52:17.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.589+0000 7fa72f2bd700 10 client.41783.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:52:17.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.589+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_assign -1 10 2024-02-23T00:52:17.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.589+0000 7fa72f2bd700 20 client.41783.objecter _get_session s=0x7fa728116f90 osd=4 3 2024-02-23T00:52:17.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.589+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa728116f90 osd=4 3 2024-02-23T00:52:17.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.589+0000 7fa72f2bd700 20 client.41783.objecter _get_session s=0x7fa728116f90 osd=4 3 2024-02-23T00:52:17.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.589+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_remove -1 10 2024-02-23T00:52:17.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.589+0000 7fa72f2bd700 20 client.41783.objecter get_session s=0x7fa728116f90 osd=4 3 2024-02-23T00:52:17.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.589+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_assign 4 10 2024-02-23T00:52:17.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.589+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa728116f90 osd=4 4 2024-02-23T00:52:17.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.589+0000 7fa72f2bd700 10 client.41783.objecter _send_command 10 2024-02-23T00:52:17.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.589+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa72811b360 con 0x7fa728117130 2024-02-23T00:52:17.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.590+0000 7fa71e777700 1 -- 172.21.15.47:0/2912854677 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa72811b360 con 0x7fa728117130 2024-02-23T00:52:17.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.590+0000 7fa71e777700 10 client.41783.objecter ms_dispatch 0x7fa728001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:52:17.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.590+0000 7fa71e777700 10 client.41783.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:52:17.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.590+0000 7fa71e777700 20 client.41783.objecter put_session s=0x7fa728116f90 osd=4 4 2024-02-23T00:52:17.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.590+0000 7fa71e777700 15 client.41783.objecter _session_command_op_remove 4 10 2024-02-23T00:52:17.603 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:52:17.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.591+0000 7fa72f2bd700 10 client.41783.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:17.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.591+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_assign -1 11 2024-02-23T00:52:17.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.591+0000 7fa72f2bd700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fa72811e500 0x7fa72811e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:17.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.591+0000 7fa72f2bd700 20 client.41783.objecter _get_session s=0x7fa72811b360 osd=5 3 2024-02-23T00:52:17.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.591+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa72811b360 osd=5 3 2024-02-23T00:52:17.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.591+0000 7fa72f2bd700 20 client.41783.objecter _get_session s=0x7fa72811b360 osd=5 3 2024-02-23T00:52:17.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.591+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_remove -1 11 2024-02-23T00:52:17.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.591+0000 7fa72f2bd700 20 client.41783.objecter get_session s=0x7fa72811b360 osd=5 3 2024-02-23T00:52:17.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.591+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_assign 5 11 2024-02-23T00:52:17.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.591+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa72811b360 osd=5 4 2024-02-23T00:52:17.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.591+0000 7fa72f2bd700 10 client.41783.objecter _send_command 11 2024-02-23T00:52:17.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.591+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa728126c10 con 0x7fa72811e500 2024-02-23T00:52:17.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.591+0000 7fa72c858700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fa72811e500 0x7fa72811e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:17.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.591+0000 7fa71e777700 10 client.41783.objecter ms_handle_connect 0x7fa72811e500 2024-02-23T00:52:17.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.591+0000 7fa72c858700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fa72811e500 0x7fa72811e960 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:17.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.592+0000 7fa71e777700 1 -- 172.21.15.47:0/2912854677 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa728126c10 con 0x7fa72811e500 2024-02-23T00:52:17.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.592+0000 7fa71e777700 10 client.41783.objecter ms_dispatch 0x7fa728001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:52:17.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.592+0000 7fa71e777700 10 client.41783.objecter _finish_command 11 = system:0 2024-02-23T00:52:17.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.592+0000 7fa71e777700 20 client.41783.objecter put_session s=0x7fa72811b360 osd=5 4 2024-02-23T00:52:17.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.592+0000 7fa71e777700 15 client.41783.objecter _session_command_op_remove 5 11 2024-02-23T00:52:17.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.608+0000 7fa72f2bd700 10 client.41783.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:52:17.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.608+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_assign -1 12 2024-02-23T00:52:17.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.608+0000 7fa72f2bd700 20 client.41783.objecter _get_session s=0x7fa72811b360 osd=5 3 2024-02-23T00:52:17.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.608+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa72811b360 osd=5 3 2024-02-23T00:52:17.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.608+0000 7fa72f2bd700 20 client.41783.objecter _get_session s=0x7fa72811b360 osd=5 3 2024-02-23T00:52:17.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.608+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_remove -1 12 2024-02-23T00:52:17.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.608+0000 7fa72f2bd700 20 client.41783.objecter get_session s=0x7fa72811b360 osd=5 3 2024-02-23T00:52:17.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.608+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_assign 5 12 2024-02-23T00:52:17.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.608+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa72811b360 osd=5 4 2024-02-23T00:52:17.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.608+0000 7fa72f2bd700 10 client.41783.objecter _send_command 12 2024-02-23T00:52:17.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.608+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa7281244b0 con 0x7fa72811e500 2024-02-23T00:52:17.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.610+0000 7fa71e777700 1 -- 172.21.15.47:0/2912854677 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa7281244b0 con 0x7fa72811e500 2024-02-23T00:52:17.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.610+0000 7fa71e777700 10 client.41783.objecter ms_dispatch 0x7fa728001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:52:17.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.610+0000 7fa71e777700 10 client.41783.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:52:17.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.610+0000 7fa71e777700 20 client.41783.objecter put_session s=0x7fa72811b360 osd=5 4 2024-02-23T00:52:17.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.610+0000 7fa71e777700 15 client.41783.objecter _session_command_op_remove 5 12 2024-02-23T00:52:17.613 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:52:17.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.610+0000 7fa72f2bd700 10 client.41783.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:17.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.610+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_assign -1 13 2024-02-23T00:52:17.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.610+0000 7fa72f2bd700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fa728124b30 0x7fa728126fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:17.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.610+0000 7fa72f2bd700 20 client.41783.objecter _get_session s=0x7fa7281244b0 osd=6 3 2024-02-23T00:52:17.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.610+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa7281244b0 osd=6 3 2024-02-23T00:52:17.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.610+0000 7fa72f2bd700 20 client.41783.objecter _get_session s=0x7fa7281244b0 osd=6 3 2024-02-23T00:52:17.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.610+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_remove -1 13 2024-02-23T00:52:17.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.610+0000 7fa72f2bd700 20 client.41783.objecter get_session s=0x7fa7281244b0 osd=6 3 2024-02-23T00:52:17.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.610+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_assign 6 13 2024-02-23T00:52:17.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.610+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa7281244b0 osd=6 4 2024-02-23T00:52:17.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.610+0000 7fa72f2bd700 10 client.41783.objecter _send_command 13 2024-02-23T00:52:17.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.610+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa728129de0 con 0x7fa728124b30 2024-02-23T00:52:17.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.610+0000 7fa72d85a700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fa728124b30 0x7fa728126fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:17.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.610+0000 7fa71e777700 10 client.41783.objecter ms_handle_connect 0x7fa728124b30 2024-02-23T00:52:17.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.610+0000 7fa72d85a700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fa728124b30 0x7fa728126fa0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:17.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.611+0000 7fa71e777700 1 -- 172.21.15.47:0/2912854677 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa728129de0 con 0x7fa728124b30 2024-02-23T00:52:17.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.611+0000 7fa71e777700 10 client.41783.objecter ms_dispatch 0x7fa728001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:52:17.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.611+0000 7fa71e777700 10 client.41783.objecter _finish_command 13 = system:0 2024-02-23T00:52:17.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.611+0000 7fa71e777700 20 client.41783.objecter put_session s=0x7fa7281244b0 osd=6 4 2024-02-23T00:52:17.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.611+0000 7fa71e777700 15 client.41783.objecter _session_command_op_remove 6 13 2024-02-23T00:52:17.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.626+0000 7fa72f2bd700 10 client.41783.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:52:17.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.626+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_assign -1 14 2024-02-23T00:52:17.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.626+0000 7fa72f2bd700 20 client.41783.objecter _get_session s=0x7fa7281244b0 osd=6 3 2024-02-23T00:52:17.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.626+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa7281244b0 osd=6 3 2024-02-23T00:52:17.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.626+0000 7fa72f2bd700 20 client.41783.objecter _get_session s=0x7fa7281244b0 osd=6 3 2024-02-23T00:52:17.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.626+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_remove -1 14 2024-02-23T00:52:17.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.626+0000 7fa72f2bd700 20 client.41783.objecter get_session s=0x7fa7281244b0 osd=6 3 2024-02-23T00:52:17.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.626+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_assign 6 14 2024-02-23T00:52:17.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.626+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa7281244b0 osd=6 4 2024-02-23T00:52:17.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.626+0000 7fa72f2bd700 10 client.41783.objecter _send_command 14 2024-02-23T00:52:17.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.626+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa728129de0 con 0x7fa728124b30 2024-02-23T00:52:17.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.628+0000 7fa71e777700 1 -- 172.21.15.47:0/2912854677 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa728129de0 con 0x7fa728124b30 2024-02-23T00:52:17.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.628+0000 7fa71e777700 10 client.41783.objecter ms_dispatch 0x7fa728001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:52:17.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.628+0000 7fa71e777700 10 client.41783.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:52:17.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.628+0000 7fa71e777700 20 client.41783.objecter put_session s=0x7fa7281244b0 osd=6 4 2024-02-23T00:52:17.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.628+0000 7fa71e777700 15 client.41783.objecter _session_command_op_remove 6 14 2024-02-23T00:52:17.631 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:52:17.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.628+0000 7fa72f2bd700 10 client.41783.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:17.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.628+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_assign -1 15 2024-02-23T00:52:17.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.628+0000 7fa72f2bd700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fa7281290c0 0x7fa728129540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:17.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.628+0000 7fa72f2bd700 20 client.41783.objecter _get_session s=0x7fa728129de0 osd=7 3 2024-02-23T00:52:17.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.628+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa728129de0 osd=7 3 2024-02-23T00:52:17.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.628+0000 7fa72f2bd700 20 client.41783.objecter _get_session s=0x7fa728129de0 osd=7 3 2024-02-23T00:52:17.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.628+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_remove -1 15 2024-02-23T00:52:17.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.628+0000 7fa72f2bd700 20 client.41783.objecter get_session s=0x7fa728129de0 osd=7 3 2024-02-23T00:52:17.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.628+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_assign 7 15 2024-02-23T00:52:17.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.628+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa728129de0 osd=7 4 2024-02-23T00:52:17.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.628+0000 7fa72f2bd700 10 client.41783.objecter _send_command 15 2024-02-23T00:52:17.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.628+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa72812d160 con 0x7fa7281290c0 2024-02-23T00:52:17.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.628+0000 7fa72d059700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fa7281290c0 0x7fa728129540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:17.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.628+0000 7fa71e777700 10 client.41783.objecter ms_handle_connect 0x7fa7281290c0 2024-02-23T00:52:17.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.628+0000 7fa72d059700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fa7281290c0 0x7fa728129540 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:17.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.629+0000 7fa71e777700 1 -- 172.21.15.47:0/2912854677 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa72812d160 con 0x7fa7281290c0 2024-02-23T00:52:17.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.629+0000 7fa71e777700 10 client.41783.objecter ms_dispatch 0x7fa728001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:52:17.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.629+0000 7fa71e777700 10 client.41783.objecter _finish_command 15 = system:0 2024-02-23T00:52:17.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.629+0000 7fa71e777700 20 client.41783.objecter put_session s=0x7fa728129de0 osd=7 4 2024-02-23T00:52:17.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.629+0000 7fa71e777700 15 client.41783.objecter _session_command_op_remove 7 15 2024-02-23T00:52:17.645 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:52:17.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.644+0000 7fa72f2bd700 10 client.41783.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:52:17.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.644+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_assign -1 16 2024-02-23T00:52:17.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.644+0000 7fa72f2bd700 20 client.41783.objecter _get_session s=0x7fa728129de0 osd=7 3 2024-02-23T00:52:17.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.644+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa728129de0 osd=7 3 2024-02-23T00:52:17.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.644+0000 7fa72f2bd700 20 client.41783.objecter _get_session s=0x7fa728129de0 osd=7 3 2024-02-23T00:52:17.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.644+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_remove -1 16 2024-02-23T00:52:17.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.644+0000 7fa72f2bd700 20 client.41783.objecter get_session s=0x7fa728129de0 osd=7 3 2024-02-23T00:52:17.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.644+0000 7fa72f2bd700 15 client.41783.objecter _session_command_op_assign 7 16 2024-02-23T00:52:17.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.644+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa728129de0 osd=7 4 2024-02-23T00:52:17.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.644+0000 7fa72f2bd700 10 client.41783.objecter _send_command 16 2024-02-23T00:52:17.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.644+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa7281275a0 con 0x7fa7281290c0 2024-02-23T00:52:17.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa71e777700 1 -- 172.21.15.47:0/2912854677 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa7281275a0 con 0x7fa7281290c0 2024-02-23T00:52:17.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa71e777700 10 client.41783.objecter ms_dispatch 0x7fa728001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:52:17.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa71e777700 10 client.41783.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:52:17.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa71e777700 20 client.41783.objecter put_session s=0x7fa728129de0 osd=7 4 2024-02-23T00:52:17.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa71e777700 15 client.41783.objecter _session_command_op_remove 7 16 2024-02-23T00:52:17.649 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:52:17.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 10 librados: watch_flush enter 2024-02-23T00:52:17.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 10 librados: watch_flush exit 2024-02-23T00:52:17.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 10 client.41783.objecter close_session for osd.0 2024-02-23T00:52:17.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fa7280fee80 msgr2=0x7fa7280ff260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:17.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fa7280fee80 0x7fa7280ff260 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:17.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa728106ed0 osd=0 1 2024-02-23T00:52:17.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 10 client.41783.objecter close_session for osd.1 2024-02-23T00:52:17.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fa728028540 msgr2=0x7fa7280598f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:17.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fa728028540 0x7fa7280598f0 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:17.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa7280ff920 osd=1 1 2024-02-23T00:52:17.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 10 client.41783.objecter close_session for osd.2 2024-02-23T00:52:17.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa7280594d0 msgr2=0x7fa728119f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:17.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa7280594d0 0x7fa728119f40 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:17.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa728059360 osd=2 1 2024-02-23T00:52:17.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 10 client.41783.objecter close_session for osd.3 2024-02-23T00:52:17.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fa728117a30 msgr2=0x7fa72811d5c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:17.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72d85a700 1 -- 172.21.15.47:0/2912854677 reap_dead start 2024-02-23T00:52:17.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fa728117a30 0x7fa72811d5c0 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:17.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa728027a00 osd=3 1 2024-02-23T00:52:17.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 10 client.41783.objecter close_session for osd.4 2024-02-23T00:52:17.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fa728117130 msgr2=0x7fa72811db00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:17.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fa728117130 0x7fa72811db00 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:17.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa728116f90 osd=4 1 2024-02-23T00:52:17.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 10 client.41783.objecter close_session for osd.5 2024-02-23T00:52:17.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fa72811e500 msgr2=0x7fa72811e960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:17.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fa72811e500 0x7fa72811e960 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:17.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa72811b360 osd=5 1 2024-02-23T00:52:17.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 10 client.41783.objecter close_session for osd.6 2024-02-23T00:52:17.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fa728124b30 msgr2=0x7fa728126fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:17.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fa728124b30 0x7fa728126fa0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:17.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa7281244b0 osd=6 1 2024-02-23T00:52:17.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 10 client.41783.objecter close_session for osd.7 2024-02-23T00:52:17.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fa7281290c0 msgr2=0x7fa728129540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:17.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fa7281290c0 0x7fa728129540 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:17.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 20 client.41783.objecter put_session s=0x7fa728129de0 osd=7 1 2024-02-23T00:52:17.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 20 client.41783.objecter shutdown clearing up homeless session... 2024-02-23T00:52:17.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 10 client.41783.objecter successfully canceled tick 2024-02-23T00:52:17.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72d85a700 1 -- 172.21.15.47:0/2912854677 reap_dead start 2024-02-23T00:52:17.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa714062cc0 msgr2=0x7fa714065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:17.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa714062cc0 0x7fa714065180 secure :-1 s=READY pgs=3140 cs=0 l=1 rev1=1 rx=0x7fa724004870 tx=0x7fa72400b040).stop 2024-02-23T00:52:17.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa7280fdbb0 msgr2=0x7fa72810e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:17.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa7280fdbb0 0x7fa72810e290 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 rx=0x7fa71800e990 tx=0x7fa71800b040).stop 2024-02-23T00:52:17.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 shutdown_connections 2024-02-23T00:52:17.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa7280fdbb0 0x7fa72810e290 unknown :-1 s=CLOSED pgs=2752 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:17.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 1 --2- 172.21.15.47:0/2912854677 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa714062cc0 0x7fa714065180 unknown :-1 s=CLOSED pgs=3140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:17.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 >> 172.21.15.47:0/2912854677 conn(0x7fa7280f3800 msgr2=0x7fa7280f5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:52:17.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.646+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 shutdown_connections 2024-02-23T00:52:17.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.647+0000 7fa72f2bd700 1 -- 172.21.15.47:0/2912854677 wait complete. 2024-02-23T00:52:17.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:17.647+0000 7fa72f2bd700 1 librados: shutdown 2024-02-23T00:52:17.653 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:52:17.654 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:52:17.654 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:52:17.654 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:52:17.654 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:52:17.654 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:52:17.654 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:52:17.654 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:52:17.674 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:17.694 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:52:17.696 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:17.697 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:52:17.798 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:52:17.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:16.474327+0000 osd.1 (osd.1) 255 : cluster [DBG] 18.1b scrub starts 2024-02-23T00:52:17.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:16.478216+0000 osd.1 (osd.1) 256 : cluster [DBG] 18.1b scrub ok 2024-02-23T00:52:17.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:16.712668+0000 osd.4 (osd.4) 259 : cluster [DBG] 4.1b scrub starts 2024-02-23T00:52:17.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:16.714565+0000 osd.4 (osd.4) 260 : cluster [DBG] 4.1b scrub ok 2024-02-23T00:52:17.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:17.268739+0000 mon.a (mon.0) 3139 : cluster [DBG] osdmap e608: 8 total, 8 up, 7 in 2024-02-23T00:52:17.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:17.272400+0000 mon.c (mon.2) 341 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-104","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:52:17.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:17.273360+0000 mon.a (mon.0) 3140 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-104","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:52:17.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:17.354275+0000 mgr.y (mgr.14517) 542 : cluster [DBG] pgmap v916: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-23T00:52:17.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:17.500343+0000 mon.c (mon.2) 342 : audit [DBG] from='client.? 172.21.15.47:0/2912854677' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:52:17.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:16.474327+0000 osd.1 (osd.1) 255 : cluster [DBG] 18.1b scrub starts 2024-02-23T00:52:17.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:16.478216+0000 osd.1 (osd.1) 256 : cluster [DBG] 18.1b scrub ok 2024-02-23T00:52:17.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:16.712668+0000 osd.4 (osd.4) 259 : cluster [DBG] 4.1b scrub starts 2024-02-23T00:52:17.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:16.714565+0000 osd.4 (osd.4) 260 : cluster [DBG] 4.1b scrub ok 2024-02-23T00:52:17.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:17.268739+0000 mon.a (mon.0) 3139 : cluster [DBG] osdmap e608: 8 total, 8 up, 7 in 2024-02-23T00:52:17.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:17.272400+0000 mon.c (mon.2) 341 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-104","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:52:17.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:17.273360+0000 mon.a (mon.0) 3140 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-104","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:52:17.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:17.354275+0000 mgr.y (mgr.14517) 542 : cluster [DBG] pgmap v916: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-23T00:52:17.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:17.500343+0000 mon.c (mon.2) 342 : audit [DBG] from='client.? 172.21.15.47:0/2912854677' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:52:17.825 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:17.845 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:52:17.848 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:17.848 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:52:17.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:16.474327+0000 osd.1 (osd.1) 255 : cluster [DBG] 18.1b scrub starts 2024-02-23T00:52:17.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:16.478216+0000 osd.1 (osd.1) 256 : cluster [DBG] 18.1b scrub ok 2024-02-23T00:52:17.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:16.712668+0000 osd.4 (osd.4) 259 : cluster [DBG] 4.1b scrub starts 2024-02-23T00:52:17.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:16.714565+0000 osd.4 (osd.4) 260 : cluster [DBG] 4.1b scrub ok 2024-02-23T00:52:17.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:17.268739+0000 mon.a (mon.0) 3139 : cluster [DBG] osdmap e608: 8 total, 8 up, 7 in 2024-02-23T00:52:17.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:17.272400+0000 mon.c (mon.2) 341 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-104","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:52:17.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:17.273360+0000 mon.a (mon.0) 3140 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-104","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:52:17.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:17.354275+0000 mgr.y (mgr.14517) 542 : cluster [DBG] pgmap v916: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-23T00:52:17.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:17.500343+0000 mon.c (mon.2) 342 : audit [DBG] from='client.? 172.21.15.47:0/2912854677' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:52:17.949 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:52:17.975 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:17.993 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:52:17.996 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:17.997 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:52:18.097 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:52:18.129 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:18.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:17.391132+0000 osd.5 (osd.5) 264 : cluster [DBG] 4.8 scrub starts 2024-02-23T00:52:18.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:17.393080+0000 osd.5 (osd.5) 265 : cluster [DBG] 4.8 scrub ok 2024-02-23T00:52:18.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:17.702997+0000 osd.0 (osd.0) 251 : cluster [DBG] 5.5 scrub starts 2024-02-23T00:52:18.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:17.704157+0000 osd.0 (osd.0) 252 : cluster [DBG] 5.5 scrub ok 2024-02-23T00:52:18.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:17.777476+0000 osd.3 (osd.3) 284 : cluster [DBG] 2.1e scrub starts 2024-02-23T00:52:18.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:17.778212+0000 osd.3 (osd.3) 285 : cluster [DBG] 2.1e scrub ok 2024-02-23T00:52:18.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:18.272488+0000 mon.a (mon.0) 3141 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-104","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:52:18.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:18.272620+0000 mon.a (mon.0) 3142 : cluster [DBG] osdmap e609: 8 total, 8 up, 7 in 2024-02-23T00:52:18.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:18.290997+0000 mon.c (mon.2) 343 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-104", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:52:18.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:18.291421+0000 mon.a (mon.0) 3143 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-104", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:52:18.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:18.425340+0000 mon.c (mon.2) 344 : audit [INF] from='client.? 172.21.15.47:0/4095809625' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:52:18.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:18.425676+0000 mon.a (mon.0) 3144 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:52:18.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:17.391132+0000 osd.5 (osd.5) 264 : cluster [DBG] 4.8 scrub starts 2024-02-23T00:52:18.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:17.393080+0000 osd.5 (osd.5) 265 : cluster [DBG] 4.8 scrub ok 2024-02-23T00:52:18.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:17.702997+0000 osd.0 (osd.0) 251 : cluster [DBG] 5.5 scrub starts 2024-02-23T00:52:18.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:17.704157+0000 osd.0 (osd.0) 252 : cluster [DBG] 5.5 scrub ok 2024-02-23T00:52:18.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:17.777476+0000 osd.3 (osd.3) 284 : cluster [DBG] 2.1e scrub starts 2024-02-23T00:52:18.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:17.778212+0000 osd.3 (osd.3) 285 : cluster [DBG] 2.1e scrub ok 2024-02-23T00:52:18.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:18.272488+0000 mon.a (mon.0) 3141 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-104","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:52:18.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:18.272620+0000 mon.a (mon.0) 3142 : cluster [DBG] osdmap e609: 8 total, 8 up, 7 in 2024-02-23T00:52:18.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:18.290997+0000 mon.c (mon.2) 343 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-104", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:52:18.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:18.291421+0000 mon.a (mon.0) 3143 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-104", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:52:18.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:18.425340+0000 mon.c (mon.2) 344 : audit [INF] from='client.? 172.21.15.47:0/4095809625' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:52:18.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:18.425676+0000 mon.a (mon.0) 3144 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:52:19.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.218+0000 7f25d2e65700 1 -- 172.21.15.47:0/694967943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25cc0f8d10 msgr2=0x7f25cc0f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:19.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.218+0000 7f25d2e65700 1 --2- 172.21.15.47:0/694967943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25cc0f8d10 0x7f25cc0f90f0 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 rx=0x7f25b4004870 tx=0x7f25b4009cb0).stop 2024-02-23T00:52:19.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.218+0000 7f25d2e65700 1 -- 172.21.15.47:0/694967943 shutdown_connections 2024-02-23T00:52:19.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.218+0000 7f25d2e65700 1 --2- 172.21.15.47:0/694967943 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f25cc102270 0x7f25cc104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:19.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.218+0000 7f25d2e65700 1 --2- 172.21.15.47:0/694967943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25cc0f8d10 0x7f25cc0f90f0 unknown :-1 s=CLOSED pgs=2593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:19.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.218+0000 7f25d2e65700 1 --2- 172.21.15.47:0/694967943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25cc0f9670 0x7f25cc101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.218+0000 7f25d2e65700 1 -- 172.21.15.47:0/694967943 >> 172.21.15.47:0/694967943 conn(0x7f25cc0f2820 msgr2=0x7f25cc0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:52:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.218+0000 7f25d2e65700 1 -- 172.21.15.47:0/694967943 shutdown_connections 2024-02-23T00:52:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.218+0000 7f25d2e65700 1 -- 172.21.15.47:0/694967943 wait complete. 2024-02-23T00:52:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.219+0000 7f25d2e65700 1 librados: starting msgr at 2024-02-23T00:52:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.219+0000 7f25d2e65700 1 librados: starting objecter 2024-02-23T00:52:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.219+0000 7f25d2e65700 1 Processor -- start 2024-02-23T00:52:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.219+0000 7f25d2e65700 1 -- start start 2024-02-23T00:52:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.219+0000 7f25d2e65700 1 librados: setting wanted keys 2024-02-23T00:52:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.219+0000 7f25d2e65700 1 librados: calling monclient init 2024-02-23T00:52:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.219+0000 7f25d2e65700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25cc0f8d10 0x7f25cc10e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.219+0000 7f25d2e65700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25cc0f9670 0x7f25cc10e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.219+0000 7f25d2e65700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f25cc102270 0x7f25cc108230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.219+0000 7f25d2e65700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc108720 con 0x7f25cc0f8d10 2024-02-23T00:52:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.219+0000 7f25d2e65700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc108890 con 0x7f25cc0f9670 2024-02-23T00:52:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.219+0000 7f25d2e65700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc108a50 con 0x7f25cc102270 2024-02-23T00:52:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.219+0000 7f25d1402700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f25cc102270 0x7f25cc108230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.219+0000 7f25d0c01700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25cc0f8d10 0x7f25cc10e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.219+0000 7f25d1402700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f25cc102270 0x7f25cc108230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:42942/0 (socket says 172.21.15.47:42942) 2024-02-23T00:52:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.219+0000 7f25d1402700 1 -- 172.21.15.47:0/4095809625 learned_addr learned my addr 172.21.15.47:0/4095809625 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:52:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.219+0000 7f25cbfff700 1 --2- 172.21.15.47:0/4095809625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25cc0f9670 0x7f25cc10e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.219+0000 7f25d1402700 1 -- 172.21.15.47:0/4095809625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25cc0f9670 msgr2=0x7f25cc10e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.219+0000 7f25d1402700 1 --2- 172.21.15.47:0/4095809625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25cc0f9670 0x7f25cc10e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.219+0000 7f25d1402700 1 -- 172.21.15.47:0/4095809625 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25cc0f8d10 msgr2=0x7f25cc10e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.219+0000 7f25d1402700 1 --2- 172.21.15.47:0/4095809625 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25cc0f8d10 0x7f25cc10e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:19.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.220+0000 7f25d1402700 1 -- 172.21.15.47:0/4095809625 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b4005040 con 0x7f25cc102270 2024-02-23T00:52:19.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.220+0000 7f25d0c01700 1 --2- 172.21.15.47:0/4095809625 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25cc0f8d10 0x7f25cc10e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:52:19.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.220+0000 7f25d1402700 1 --2- 172.21.15.47:0/4095809625 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f25cc102270 0x7f25cc108230 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 rx=0x7f25c400eb70 tx=0x7f25c400b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:19.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.220+0000 7f25c9ffb700 10 client.?.objecter ms_handle_connect 0x7f25cc102270 2024-02-23T00:52:19.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.220+0000 7f25c9ffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:52:19.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.220+0000 7f25c9ffb700 1 -- 172.21.15.47:0/4095809625 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c4009910 con 0x7f25cc102270 2024-02-23T00:52:19.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.220+0000 7f25c9ffb700 1 -- 172.21.15.47:0/4095809625 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f25c400f040 con 0x7f25cc102270 2024-02-23T00:52:19.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.220+0000 7f25d2e65700 1 -- 172.21.15.47:0/4095809625 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25cc108cd0 con 0x7f25cc102270 2024-02-23T00:52:19.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.220+0000 7f25c9ffb700 1 -- 172.21.15.47:0/4095809625 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c4007d50 con 0x7f25cc102270 2024-02-23T00:52:19.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.220+0000 7f25d2e65700 10 client.41789.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:52:19.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.220+0000 7f25d2e65700 1 -- 172.21.15.47:0/4095809625 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25cc114ec0 con 0x7f25cc102270 2024-02-23T00:52:19.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.220+0000 7f25d2e65700 1 librados: init done 2024-02-23T00:52:19.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.221+0000 7f25d2e65700 1 -- 172.21.15.47:0/4095809625 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25cc05fa60 con 0x7f25cc102270 2024-02-23T00:52:19.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.221+0000 7f25c9ffb700 1 -- 172.21.15.47:0/4095809625 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f25c4004020 con 0x7f25cc102270 2024-02-23T00:52:19.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.222+0000 7f25c9ffb700 1 --2- 172.21.15.47:0/4095809625 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f25b8062cc0 0x7f25b8065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:19.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.222+0000 7f25d0c01700 1 --2- 172.21.15.47:0/4095809625 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f25b8062cc0 0x7f25b8065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:19.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.222+0000 7f25c9ffb700 1 -- 172.21.15.47:0/4095809625 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(608..608 src has 482..608) v4 ==== 9570+0+0 (secure 0 0 0) 0x7f25c407d410 con 0x7f25cc102270 2024-02-23T00:52:19.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.222+0000 7f25c9ffb700 10 client.41789.objecter ms_dispatch 0x7f25cc001550 osd_map(608..608 src has 482..608) v4 2024-02-23T00:52:19.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.222+0000 7f25c9ffb700 3 client.41789.objecter handle_osd_map got epochs [608,608] > 0 2024-02-23T00:52:19.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.222+0000 7f25c9ffb700 3 client.41789.objecter handle_osd_map decoding full epoch 608 2024-02-23T00:52:19.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.222+0000 7f25c9ffb700 20 client.41789.objecter dump_active .. 0 homeless 2024-02-23T00:52:19.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.222+0000 7f25c9ffb700 10 client.41789.objecter ms_handle_connect 0x7f25b8062cc0 2024-02-23T00:52:19.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.222+0000 7f25d0c01700 1 --2- 172.21.15.47:0/4095809625 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f25b8062cc0 0x7f25b8065180 secure :-1 s=READY pgs=3141 cs=0 l=1 rev1=1 rx=0x7f25b4004ee0 tx=0x7f25b400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:19.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.226+0000 7f25c9ffb700 1 -- 172.21.15.47:0/4095809625 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f25c4049f60 con 0x7f25cc102270 2024-02-23T00:52:19.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:18.423+0000 7f25d2e65700 1 -- 172.21.15.47:0/4095809625 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7f25cc107090 con 0x7f25cc102270 2024-02-23T00:52:19.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:17.391132+0000 osd.5 (osd.5) 264 : cluster [DBG] 4.8 scrub starts 2024-02-23T00:52:19.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:17.393080+0000 osd.5 (osd.5) 265 : cluster [DBG] 4.8 scrub ok 2024-02-23T00:52:19.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:17.702997+0000 osd.0 (osd.0) 251 : cluster [DBG] 5.5 scrub starts 2024-02-23T00:52:19.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:17.704157+0000 osd.0 (osd.0) 252 : cluster [DBG] 5.5 scrub ok 2024-02-23T00:52:19.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:17.777476+0000 osd.3 (osd.3) 284 : cluster [DBG] 2.1e scrub starts 2024-02-23T00:52:19.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:17.778212+0000 osd.3 (osd.3) 285 : cluster [DBG] 2.1e scrub ok 2024-02-23T00:52:19.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:18.272488+0000 mon.a (mon.0) 3141 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-104","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:52:19.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:18.272620+0000 mon.a (mon.0) 3142 : cluster [DBG] osdmap e609: 8 total, 8 up, 7 in 2024-02-23T00:52:19.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:18.290997+0000 mon.c (mon.2) 343 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-104", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:52:19.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:18.291421+0000 mon.a (mon.0) 3143 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-104", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:52:19.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:18.425340+0000 mon.c (mon.2) 344 : audit [INF] from='client.? 172.21.15.47:0/4095809625' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:52:19.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:18.425676+0000 mon.a (mon.0) 3144 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:52:19.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.273+0000 7f25c9ffb700 1 -- 172.21.15.47:0/4095809625 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v610) v1 ==== 101+0+0 (secure 0 0 0) 0x7f25c404c9b0 con 0x7f25cc102270 2024-02-23T00:52:19.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.276+0000 7f25d2e65700 10 librados: watch_flush enter 2024-02-23T00:52:19.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.276+0000 7f25d2e65700 10 librados: watch_flush exit 2024-02-23T00:52:19.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.276+0000 7f25d2e65700 20 client.41789.objecter shutdown clearing up homeless session... 2024-02-23T00:52:19.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.276+0000 7f25d2e65700 10 client.41789.objecter successfully canceled tick 2024-02-23T00:52:19.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.276+0000 7f25d2e65700 1 -- 172.21.15.47:0/4095809625 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f25b8062cc0 msgr2=0x7f25b8065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:19.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.276+0000 7f25d2e65700 1 --2- 172.21.15.47:0/4095809625 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f25b8062cc0 0x7f25b8065180 secure :-1 s=READY pgs=3141 cs=0 l=1 rev1=1 rx=0x7f25b4004ee0 tx=0x7f25b400b040).stop 2024-02-23T00:52:19.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.276+0000 7f25d2e65700 1 -- 172.21.15.47:0/4095809625 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f25cc102270 msgr2=0x7f25cc108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:19.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.276+0000 7f25d2e65700 1 --2- 172.21.15.47:0/4095809625 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f25cc102270 0x7f25cc108230 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 rx=0x7f25c400eb70 tx=0x7f25c400b040).stop 2024-02-23T00:52:19.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.277+0000 7f25d2e65700 1 -- 172.21.15.47:0/4095809625 shutdown_connections 2024-02-23T00:52:19.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.277+0000 7f25d2e65700 1 --2- 172.21.15.47:0/4095809625 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f25b8062cc0 0x7f25b8065180 unknown :-1 s=CLOSED pgs=3141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:19.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.277+0000 7f25d2e65700 1 --2- 172.21.15.47:0/4095809625 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f25cc102270 0x7f25cc108230 unknown :-1 s=CLOSED pgs=2753 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:19.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.277+0000 7f25d2e65700 1 --2- 172.21.15.47:0/4095809625 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25cc0f8d10 0x7f25cc10e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:19.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.277+0000 7f25d2e65700 1 --2- 172.21.15.47:0/4095809625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25cc0f9670 0x7f25cc10e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:19.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.277+0000 7f25d2e65700 1 -- 172.21.15.47:0/4095809625 >> 172.21.15.47:0/4095809625 conn(0x7f25cc0f2820 msgr2=0x7f25cc0f3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:52:19.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.277+0000 7f25d2e65700 1 -- 172.21.15.47:0/4095809625 shutdown_connections 2024-02-23T00:52:19.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.277+0000 7f25d2e65700 1 -- 172.21.15.47:0/4095809625 wait complete. 2024-02-23T00:52:19.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.277+0000 7f25d2e65700 1 librados: shutdown 2024-02-23T00:52:19.281 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is unset 2024-02-23T00:52:19.686 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:52:19.691 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:19.691 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:52:19.792 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:52:19.823 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:19.843 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:52:19.846 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:19.846 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:52:19.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.942+0000 7f4ba6dbb700 1 -- 172.21.15.47:0/1019384352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ba00f9670 msgr2=0x7f4ba0101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:19.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.942+0000 7f4ba6dbb700 1 --2- 172.21.15.47:0/1019384352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ba00f9670 0x7f4ba0101d80 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 rx=0x7f4b8c0047b0 tx=0x7f4b8c009cb0).stop 2024-02-23T00:52:19.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.942+0000 7f4ba6dbb700 1 -- 172.21.15.47:0/1019384352 shutdown_connections 2024-02-23T00:52:19.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.942+0000 7f4ba6dbb700 1 --2- 172.21.15.47:0/1019384352 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4ba0102270 0x7f4ba0104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:19.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.942+0000 7f4ba6dbb700 1 --2- 172.21.15.47:0/1019384352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ba00f8d10 0x7f4ba00f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:19.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.942+0000 7f4ba6dbb700 1 --2- 172.21.15.47:0/1019384352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ba00f9670 0x7f4ba0101d80 unknown :-1 s=CLOSED pgs=2594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:19.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.942+0000 7f4ba6dbb700 1 -- 172.21.15.47:0/1019384352 >> 172.21.15.47:0/1019384352 conn(0x7f4ba00f2820 msgr2=0x7f4ba00f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:52:19.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.942+0000 7f4ba6dbb700 1 -- 172.21.15.47:0/1019384352 shutdown_connections 2024-02-23T00:52:19.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.942+0000 7f4ba6dbb700 1 -- 172.21.15.47:0/1019384352 wait complete. 2024-02-23T00:52:19.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.943+0000 7f4ba6dbb700 1 librados: starting msgr at 2024-02-23T00:52:19.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.943+0000 7f4ba6dbb700 1 librados: starting objecter 2024-02-23T00:52:19.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.943+0000 7f4ba6dbb700 1 Processor -- start 2024-02-23T00:52:19.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.943+0000 7f4ba6dbb700 1 -- start start 2024-02-23T00:52:19.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.943+0000 7f4ba6dbb700 1 librados: setting wanted keys 2024-02-23T00:52:19.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.943+0000 7f4ba6dbb700 1 librados: calling monclient init 2024-02-23T00:52:19.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.943+0000 7f4ba6dbb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ba00f8d10 0x7f4ba010dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:19.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.943+0000 7f4ba6dbb700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4ba00f9670 0x7f4ba010e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:19.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.943+0000 7f4ba6dbb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ba0102270 0x7f4ba0108180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:19.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.943+0000 7f4ba6dbb700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba0108670 con 0x7f4ba00f8d10 2024-02-23T00:52:19.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.943+0000 7f4ba6dbb700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba01087e0 con 0x7f4ba0102270 2024-02-23T00:52:19.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.943+0000 7f4ba6dbb700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba01089a0 con 0x7f4ba00f9670 2024-02-23T00:52:19.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.943+0000 7f4ba4b57700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ba00f8d10 0x7f4ba010dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:19.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.943+0000 7f4b9ffff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4ba00f9670 0x7f4ba010e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:19.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.943+0000 7f4ba4b57700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ba00f8d10 0x7f4ba010dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39924/0 (socket says 172.21.15.47:39924) 2024-02-23T00:52:19.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.943+0000 7f4ba4b57700 1 -- 172.21.15.47:0/2480473871 learned_addr learned my addr 172.21.15.47:0/2480473871 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:52:19.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.944+0000 7f4ba4b57700 1 -- 172.21.15.47:0/2480473871 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4ba00f9670 msgr2=0x7f4ba010e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:19.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.944+0000 7f4ba4b57700 1 --2- 172.21.15.47:0/2480473871 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4ba00f9670 0x7f4ba010e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:19.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.944+0000 7f4ba4b57700 1 -- 172.21.15.47:0/2480473871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ba0102270 msgr2=0x7f4ba0108180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:19.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.944+0000 7f4ba5358700 1 --2- 172.21.15.47:0/2480473871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ba0102270 0x7f4ba0108180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:19.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.944+0000 7f4ba4b57700 1 --2- 172.21.15.47:0/2480473871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ba0102270 0x7f4ba0108180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:19.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.944+0000 7f4ba4b57700 1 -- 172.21.15.47:0/2480473871 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b980097c0 con 0x7f4ba00f8d10 2024-02-23T00:52:19.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.944+0000 7f4b9ffff700 1 --2- 172.21.15.47:0/2480473871 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4ba00f9670 0x7f4ba010e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:52:19.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.944+0000 7f4ba5358700 1 --2- 172.21.15.47:0/2480473871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ba0102270 0x7f4ba0108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-23T00:52:19.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.944+0000 7f4ba4b57700 1 --2- 172.21.15.47:0/2480473871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ba00f8d10 0x7f4ba010dfe0 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 rx=0x7f4b9400c810 tx=0x7f4b94004330).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:19.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.944+0000 7f4b9dffb700 10 client.?.objecter ms_handle_connect 0x7f4ba00f8d10 2024-02-23T00:52:19.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.944+0000 7f4b9dffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:52:19.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.944+0000 7f4b9dffb700 1 -- 172.21.15.47:0/2480473871 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b940043a0 con 0x7f4ba00f8d10 2024-02-23T00:52:19.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.944+0000 7f4b9dffb700 1 -- 172.21.15.47:0/2480473871 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f4b940109a0 con 0x7f4ba00f8d10 2024-02-23T00:52:19.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.944+0000 7f4ba6dbb700 1 -- 172.21.15.47:0/2480473871 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b8c005040 con 0x7f4ba00f8d10 2024-02-23T00:52:19.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.944+0000 7f4b9dffb700 1 -- 172.21.15.47:0/2480473871 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b94012940 con 0x7f4ba00f8d10 2024-02-23T00:52:19.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.944+0000 7f4ba6dbb700 10 client.31476.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:52:19.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.944+0000 7f4ba6dbb700 1 -- 172.21.15.47:0/2480473871 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba0108fe0 con 0x7f4ba00f8d10 2024-02-23T00:52:19.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.944+0000 7f4ba6dbb700 1 librados: init done 2024-02-23T00:52:19.950 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:52:19.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.945+0000 7f4ba6dbb700 1 -- 172.21.15.47:0/2480473871 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba005fa60 con 0x7f4ba00f8d10 2024-02-23T00:52:19.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.946+0000 7f4b9dffb700 1 -- 172.21.15.47:0/2480473871 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f4b94010510 con 0x7f4ba00f8d10 2024-02-23T00:52:19.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.946+0000 7f4b9dffb700 1 --2- 172.21.15.47:0/2480473871 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4b90062cc0 0x7f4b90065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:19.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.946+0000 7f4b9dffb700 1 -- 172.21.15.47:0/2480473871 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(610..610 src has 482..610) v4 ==== 9578+0+0 (secure 0 0 0) 0x7f4b9407d710 con 0x7f4ba00f8d10 2024-02-23T00:52:19.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.946+0000 7f4b9dffb700 10 client.31476.objecter ms_dispatch 0x7f4ba0001550 osd_map(610..610 src has 482..610) v4 2024-02-23T00:52:19.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.946+0000 7f4b9dffb700 3 client.31476.objecter handle_osd_map got epochs [610,610] > 0 2024-02-23T00:52:19.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.946+0000 7f4b9dffb700 3 client.31476.objecter handle_osd_map decoding full epoch 610 2024-02-23T00:52:19.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.946+0000 7f4b9ffff700 1 --2- 172.21.15.47:0/2480473871 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4b90062cc0 0x7f4b90065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:19.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.946+0000 7f4b9dffb700 20 client.31476.objecter dump_active .. 0 homeless 2024-02-23T00:52:19.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.946+0000 7f4b9dffb700 10 client.31476.objecter ms_handle_connect 0x7f4b90062cc0 2024-02-23T00:52:19.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.946+0000 7f4b9ffff700 1 --2- 172.21.15.47:0/2480473871 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4b90062cc0 0x7f4b90065180 secure :-1 s=READY pgs=3142 cs=0 l=1 rev1=1 rx=0x7f4ba0109920 tx=0x7f4b8c009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:19.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:19.950+0000 7f4b9dffb700 1 -- 172.21.15.47:0/2480473871 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f4b9404cbc0 con 0x7f4ba00f8d10 2024-02-23T00:52:19.978 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:18.474605+0000 osd.1 (osd.1) 257 : cluster [DBG] 2.6 scrub starts 2024-02-23T00:52:19.978 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:18.475797+0000 osd.1 (osd.1) 258 : cluster [DBG] 2.6 scrub ok 2024-02-23T00:52:19.978 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:19.270194+0000 mon.a (mon.0) 3145 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:52:19.979 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:19.274582+0000 mon.a (mon.0) 3146 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-104", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:52:19.979 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:19.274685+0000 mon.a (mon.0) 3147 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:52:19.979 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:19.274748+0000 mon.a (mon.0) 3148 : cluster [DBG] osdmap e610: 8 total, 8 up, 7 in 2024-02-23T00:52:19.979 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:19.275796+0000 mon.c (mon.2) 345 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-104"}]: dispatch 2024-02-23T00:52:19.979 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:19.276260+0000 mon.a (mon.0) 3149 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-104"}]: dispatch 2024-02-23T00:52:19.979 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:19.354652+0000 mgr.y (mgr.14517) 543 : cluster [DBG] pgmap v919: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-23T00:52:19.979 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:18.474605+0000 osd.1 (osd.1) 257 : cluster [DBG] 2.6 scrub starts 2024-02-23T00:52:19.979 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:18.475797+0000 osd.1 (osd.1) 258 : cluster [DBG] 2.6 scrub ok 2024-02-23T00:52:19.979 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:19.270194+0000 mon.a (mon.0) 3145 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:52:19.979 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:19.274582+0000 mon.a (mon.0) 3146 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-104", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:52:19.980 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:19.274685+0000 mon.a (mon.0) 3147 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:52:19.980 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:19.274748+0000 mon.a (mon.0) 3148 : cluster [DBG] osdmap e610: 8 total, 8 up, 7 in 2024-02-23T00:52:19.980 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:19.275796+0000 mon.c (mon.2) 345 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-104"}]: dispatch 2024-02-23T00:52:19.980 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:19.276260+0000 mon.a (mon.0) 3149 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-104"}]: dispatch 2024-02-23T00:52:19.980 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:19.354652+0000 mgr.y (mgr.14517) 543 : cluster [DBG] pgmap v919: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-23T00:52:19.980 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:20.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:20.125+0000 7f4ba6dbb700 1 -- 172.21.15.47:0/2480473871 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pgp_num", "val": "32"} v 0) v1 -- 0x7f4ba0109660 con 0x7f4ba00f8d10 2024-02-23T00:52:20.179 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:52:20.184 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:20.185 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:52:20.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:18.474605+0000 osd.1 (osd.1) 257 : cluster [DBG] 2.6 scrub starts 2024-02-23T00:52:20.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:18.475797+0000 osd.1 (osd.1) 258 : cluster [DBG] 2.6 scrub ok 2024-02-23T00:52:20.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:19.270194+0000 mon.a (mon.0) 3145 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:52:20.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:19.274582+0000 mon.a (mon.0) 3146 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-104", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:52:20.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:19.274685+0000 mon.a (mon.0) 3147 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:52:20.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:19.274748+0000 mon.a (mon.0) 3148 : cluster [DBG] osdmap e610: 8 total, 8 up, 7 in 2024-02-23T00:52:20.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:19.275796+0000 mon.c (mon.2) 345 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-104"}]: dispatch 2024-02-23T00:52:20.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:19.276260+0000 mon.a (mon.0) 3149 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-104"}]: dispatch 2024-02-23T00:52:20.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:19.354652+0000 mgr.y (mgr.14517) 543 : cluster [DBG] pgmap v919: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-23T00:52:20.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:20.277+0000 7f4b9dffb700 1 -- 172.21.15.47:0/2480473871 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pgp_num", "val": "32"}]=0 set pool 4 pgp_num to 32 v611) v1 ==== 142+0+0 (secure 0 0 0) 0x7f4b940501e0 con 0x7f4ba00f8d10 2024-02-23T00:52:20.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:20.280+0000 7f4ba6dbb700 10 librados: watch_flush enter 2024-02-23T00:52:20.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:20.280+0000 7f4ba6dbb700 10 librados: watch_flush exit 2024-02-23T00:52:20.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:20.280+0000 7f4ba6dbb700 20 client.31476.objecter shutdown clearing up homeless session... 2024-02-23T00:52:20.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:20.280+0000 7f4ba6dbb700 10 client.31476.objecter successfully canceled tick 2024-02-23T00:52:20.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:20.281+0000 7f4ba6dbb700 1 -- 172.21.15.47:0/2480473871 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4b90062cc0 msgr2=0x7f4b90065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:20.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:20.281+0000 7f4ba6dbb700 1 --2- 172.21.15.47:0/2480473871 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4b90062cc0 0x7f4b90065180 secure :-1 s=READY pgs=3142 cs=0 l=1 rev1=1 rx=0x7f4ba0109920 tx=0x7f4b8c009cb0).stop 2024-02-23T00:52:20.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:20.281+0000 7f4ba6dbb700 1 -- 172.21.15.47:0/2480473871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ba00f8d10 msgr2=0x7f4ba010dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:20.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:20.281+0000 7f4ba6dbb700 1 --2- 172.21.15.47:0/2480473871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ba00f8d10 0x7f4ba010dfe0 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 rx=0x7f4b9400c810 tx=0x7f4b94004330).stop 2024-02-23T00:52:20.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:20.281+0000 7f4ba6dbb700 1 -- 172.21.15.47:0/2480473871 shutdown_connections 2024-02-23T00:52:20.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:20.281+0000 7f4ba6dbb700 1 --2- 172.21.15.47:0/2480473871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ba0102270 0x7f4ba0108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:20.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:20.281+0000 7f4ba6dbb700 1 --2- 172.21.15.47:0/2480473871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ba00f8d10 0x7f4ba010dfe0 unknown :-1 s=CLOSED pgs=2595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:20.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:20.281+0000 7f4ba6dbb700 1 --2- 172.21.15.47:0/2480473871 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4b90062cc0 0x7f4b90065180 unknown :-1 s=CLOSED pgs=3142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:20.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:20.281+0000 7f4ba6dbb700 1 --2- 172.21.15.47:0/2480473871 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4ba00f9670 0x7f4ba010e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:20.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:20.281+0000 7f4ba6dbb700 1 -- 172.21.15.47:0/2480473871 >> 172.21.15.47:0/2480473871 conn(0x7f4ba00f2820 msgr2=0x7f4ba00f3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:52:20.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:20.281+0000 7f4ba6dbb700 1 -- 172.21.15.47:0/2480473871 shutdown_connections 2024-02-23T00:52:20.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:20.281+0000 7f4ba6dbb700 1 -- 172.21.15.47:0/2480473871 wait complete. 2024-02-23T00:52:20.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:20.281+0000 7f4ba6dbb700 1 librados: shutdown 2024-02-23T00:52:20.285 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:52:20.286 INFO:teuthology.orchestra.run.smithi047.stderr:set pool 4 pgp_num to 32 2024-02-23T00:52:20.316 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:20.557 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:52:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:52:20] "GET /metrics HTTP/1.1" 200 190634 "" "Prometheus/2.33.4" 2024-02-23T00:52:20.665 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:52:20.669 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:20.670 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:52:20.771 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:52:20.799 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:20.872 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:52:20.875 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:20.876 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:52:20.977 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:52:20.996 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:19.343086+0000 osd.6 (osd.6) 231 : cluster [DBG] 18.14 scrub starts 2024-02-23T00:52:20.997 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:19.344008+0000 osd.6 (osd.6) 232 : cluster [DBG] 18.14 scrub ok 2024-02-23T00:52:20.997 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:19.721122+0000 osd.4 (osd.4) 261 : cluster [DBG] 18.12 deep-scrub starts 2024-02-23T00:52:20.997 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:19.722200+0000 osd.4 (osd.4) 262 : cluster [DBG] 18.12 deep-scrub ok 2024-02-23T00:52:20.997 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:19.763921+0000 osd.0 (osd.0) 253 : cluster [DBG] 3.1 scrub starts 2024-02-23T00:52:20.997 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:19.764822+0000 osd.0 (osd.0) 254 : cluster [DBG] 3.1 scrub ok 2024-02-23T00:52:20.997 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:20.127300+0000 mon.a (mon.0) 3150 : audit [INF] from='client.? 172.21.15.47:0/2480473871' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pgp_num", "val": "32"}]: dispatch 2024-02-23T00:52:20.997 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:20.278565+0000 mon.a (mon.0) 3151 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-104"}]': finished 2024-02-23T00:52:20.997 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:20.278642+0000 mon.a (mon.0) 3152 : audit [INF] from='client.? 172.21.15.47:0/2480473871' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pgp_num", "val": "32"}]': finished 2024-02-23T00:52:20.997 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:20.278701+0000 mon.a (mon.0) 3153 : cluster [DBG] osdmap e611: 8 total, 8 up, 7 in 2024-02-23T00:52:20.997 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:20.282567+0000 mon.c (mon.2) 346 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-104", "mode": "writeback"}]: dispatch 2024-02-23T00:52:20.997 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:20.282944+0000 mon.a (mon.0) 3154 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-104", "mode": "writeback"}]: dispatch 2024-02-23T00:52:20.998 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:19.343086+0000 osd.6 (osd.6) 231 : cluster [DBG] 18.14 scrub starts 2024-02-23T00:52:20.998 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:19.344008+0000 osd.6 (osd.6) 232 : cluster [DBG] 18.14 scrub ok 2024-02-23T00:52:20.998 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:19.721122+0000 osd.4 (osd.4) 261 : cluster [DBG] 18.12 deep-scrub starts 2024-02-23T00:52:20.998 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:19.722200+0000 osd.4 (osd.4) 262 : cluster [DBG] 18.12 deep-scrub ok 2024-02-23T00:52:20.998 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:19.763921+0000 osd.0 (osd.0) 253 : cluster [DBG] 3.1 scrub starts 2024-02-23T00:52:20.998 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:19.764822+0000 osd.0 (osd.0) 254 : cluster [DBG] 3.1 scrub ok 2024-02-23T00:52:20.998 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:20.127300+0000 mon.a (mon.0) 3150 : audit [INF] from='client.? 172.21.15.47:0/2480473871' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pgp_num", "val": "32"}]: dispatch 2024-02-23T00:52:20.998 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:20.278565+0000 mon.a (mon.0) 3151 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-104"}]': finished 2024-02-23T00:52:20.998 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:20.278642+0000 mon.a (mon.0) 3152 : audit [INF] from='client.? 172.21.15.47:0/2480473871' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pgp_num", "val": "32"}]': finished 2024-02-23T00:52:20.998 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:20.278701+0000 mon.a (mon.0) 3153 : cluster [DBG] osdmap e611: 8 total, 8 up, 7 in 2024-02-23T00:52:20.998 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:20.282567+0000 mon.c (mon.2) 346 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-104", "mode": "writeback"}]: dispatch 2024-02-23T00:52:20.998 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:20.282944+0000 mon.a (mon.0) 3154 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-104", "mode": "writeback"}]: dispatch 2024-02-23T00:52:21.008 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:19.343086+0000 osd.6 (osd.6) 231 : cluster [DBG] 18.14 scrub starts 2024-02-23T00:52:21.008 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:19.344008+0000 osd.6 (osd.6) 232 : cluster [DBG] 18.14 scrub ok 2024-02-23T00:52:21.008 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:19.721122+0000 osd.4 (osd.4) 261 : cluster [DBG] 18.12 deep-scrub starts 2024-02-23T00:52:21.008 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:19.722200+0000 osd.4 (osd.4) 262 : cluster [DBG] 18.12 deep-scrub ok 2024-02-23T00:52:21.009 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:19.763921+0000 osd.0 (osd.0) 253 : cluster [DBG] 3.1 scrub starts 2024-02-23T00:52:21.009 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:19.764822+0000 osd.0 (osd.0) 254 : cluster [DBG] 3.1 scrub ok 2024-02-23T00:52:21.009 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:20.127300+0000 mon.a (mon.0) 3150 : audit [INF] from='client.? 172.21.15.47:0/2480473871' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pgp_num", "val": "32"}]: dispatch 2024-02-23T00:52:21.009 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:20.278565+0000 mon.a (mon.0) 3151 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-104"}]': finished 2024-02-23T00:52:21.009 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:20.278642+0000 mon.a (mon.0) 3152 : audit [INF] from='client.? 172.21.15.47:0/2480473871' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pgp_num", "val": "32"}]': finished 2024-02-23T00:52:21.009 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:20.278701+0000 mon.a (mon.0) 3153 : cluster [DBG] osdmap e611: 8 total, 8 up, 7 in 2024-02-23T00:52:21.009 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:20.282567+0000 mon.c (mon.2) 346 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-104", "mode": "writeback"}]: dispatch 2024-02-23T00:52:21.009 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:20.282944+0000 mon.a (mon.0) 3154 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-104", "mode": "writeback"}]: dispatch 2024-02-23T00:52:21.011 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:21.031 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:52:21.034 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:21.035 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:52:21.136 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:52:21.166 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:21.186 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:52:21.189 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:21.190 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:52:21.290 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:52:21.320 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:21.339 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:52:21.341 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:21.342 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:52:21.443 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:52:21.470 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:21.489 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:52:21.492 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:21.492 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:52:21.520 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:52:21.593 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:52:21.625 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:21.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:20.400152+0000 osd.7 (osd.7) 291 : cluster [DBG] 6.3 scrub starts 2024-02-23T00:52:21.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:20.403027+0000 osd.7 (osd.7) 292 : cluster [DBG] 6.3 scrub ok 2024-02-23T00:52:21.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:20.414630+0000 osd.5 (osd.5) 266 : cluster [DBG] 18.f scrub starts 2024-02-23T00:52:21.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:20.418236+0000 osd.5 (osd.5) 267 : cluster [DBG] 18.f scrub ok 2024-02-23T00:52:21.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:20.773564+0000 osd.3 (osd.3) 286 : cluster [DBG] 2.1b scrub starts 2024-02-23T00:52:21.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:20.774547+0000 osd.3 (osd.3) 287 : cluster [DBG] 2.1b scrub ok 2024-02-23T00:52:21.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:21.275874+0000 mon.a (mon.0) 3155 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:52:21.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:21.280250+0000 mon.a (mon.0) 3156 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-104", "mode": "writeback"}]': finished 2024-02-23T00:52:21.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:21.280326+0000 mon.a (mon.0) 3157 : cluster [DBG] osdmap e612: 8 total, 8 up, 7 in 2024-02-23T00:52:21.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:21.318078+0000 mon.c (mon.2) 347 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:21.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:21.318366+0000 mon.a (mon.0) 3158 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:21.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:21.355273+0000 mgr.y (mgr.14517) 544 : cluster [DBG] pgmap v922: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:52:21.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:20.400152+0000 osd.7 (osd.7) 291 : cluster [DBG] 6.3 scrub starts 2024-02-23T00:52:21.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:20.403027+0000 osd.7 (osd.7) 292 : cluster [DBG] 6.3 scrub ok 2024-02-23T00:52:21.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:20.414630+0000 osd.5 (osd.5) 266 : cluster [DBG] 18.f scrub starts 2024-02-23T00:52:21.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:20.418236+0000 osd.5 (osd.5) 267 : cluster [DBG] 18.f scrub ok 2024-02-23T00:52:21.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:20.773564+0000 osd.3 (osd.3) 286 : cluster [DBG] 2.1b scrub starts 2024-02-23T00:52:21.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:20.774547+0000 osd.3 (osd.3) 287 : cluster [DBG] 2.1b scrub ok 2024-02-23T00:52:21.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:21.275874+0000 mon.a (mon.0) 3155 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:52:21.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:21.280250+0000 mon.a (mon.0) 3156 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-104", "mode": "writeback"}]': finished 2024-02-23T00:52:21.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:21.280326+0000 mon.a (mon.0) 3157 : cluster [DBG] osdmap e612: 8 total, 8 up, 7 in 2024-02-23T00:52:21.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:21.318078+0000 mon.c (mon.2) 347 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:21.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:21.318366+0000 mon.a (mon.0) 3158 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:21.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:21.355273+0000 mgr.y (mgr.14517) 544 : cluster [DBG] pgmap v922: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:52:22.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:20.400152+0000 osd.7 (osd.7) 291 : cluster [DBG] 6.3 scrub starts 2024-02-23T00:52:22.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:20.403027+0000 osd.7 (osd.7) 292 : cluster [DBG] 6.3 scrub ok 2024-02-23T00:52:22.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:20.414630+0000 osd.5 (osd.5) 266 : cluster [DBG] 18.f scrub starts 2024-02-23T00:52:22.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:20.418236+0000 osd.5 (osd.5) 267 : cluster [DBG] 18.f scrub ok 2024-02-23T00:52:22.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:20.773564+0000 osd.3 (osd.3) 286 : cluster [DBG] 2.1b scrub starts 2024-02-23T00:52:22.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:20.774547+0000 osd.3 (osd.3) 287 : cluster [DBG] 2.1b scrub ok 2024-02-23T00:52:22.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:21.275874+0000 mon.a (mon.0) 3155 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:52:22.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:21.280250+0000 mon.a (mon.0) 3156 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-104", "mode": "writeback"}]': finished 2024-02-23T00:52:22.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:21.280326+0000 mon.a (mon.0) 3157 : cluster [DBG] osdmap e612: 8 total, 8 up, 7 in 2024-02-23T00:52:22.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:21.318078+0000 mon.c (mon.2) 347 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:22.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:21.318366+0000 mon.a (mon.0) 3158 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:22.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:21.355273+0000 mgr.y (mgr.14517) 544 : cluster [DBG] pgmap v922: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:52:22.687 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set noscrub 2024-02-23T00:52:22.798 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:52:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:52:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:52:23.035 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:52:23.035 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-02-23T00:52:23.035 INFO:teuthology.orchestra.run.smithi047.stdout: "num_ops": 0 2024-02-23T00:52:23.035 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:52:23.075 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:52:23.078 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:23.079 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:52:23.080 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:52:23.081 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:52:23.180 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:52:23.194 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:52:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:52:23.210 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:23.285 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:52:23.292 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:23.292 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:52:23.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:21.534118+0000 osd.1 (osd.1) 259 : cluster [DBG] 2.10 scrub starts 2024-02-23T00:52:23.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:21.536415+0000 osd.1 (osd.1) 260 : cluster [DBG] 2.10 scrub ok 2024-02-23T00:52:23.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:22.034495+0000 osd.7 (osd.7) 293 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:52:23.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:22.034843+0000 osd.7 (osd.7) 294 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:52:23.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:22.284425+0000 mon.a (mon.0) 3159 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:52:23.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:22.284496+0000 mon.a (mon.0) 3160 : cluster [DBG] osdmap e613: 8 total, 8 up, 7 in 2024-02-23T00:52:23.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:22.287747+0000 mon.c (mon.2) 348 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-104"}]: dispatch 2024-02-23T00:52:23.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:22.288031+0000 mon.a (mon.0) 3161 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-104"}]: dispatch 2024-02-23T00:52:23.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:22.387406+0000 osd.7 (osd.7) 295 : cluster [DBG] 2.0 scrub starts 2024-02-23T00:52:23.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:22.390391+0000 osd.7 (osd.7) 296 : cluster [DBG] 2.0 scrub ok 2024-02-23T00:52:23.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:21.534118+0000 osd.1 (osd.1) 259 : cluster [DBG] 2.10 scrub starts 2024-02-23T00:52:23.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:21.536415+0000 osd.1 (osd.1) 260 : cluster [DBG] 2.10 scrub ok 2024-02-23T00:52:23.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:22.034495+0000 osd.7 (osd.7) 293 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:52:23.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:22.034843+0000 osd.7 (osd.7) 294 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:52:23.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:22.284425+0000 mon.a (mon.0) 3159 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:52:23.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:22.284496+0000 mon.a (mon.0) 3160 : cluster [DBG] osdmap e613: 8 total, 8 up, 7 in 2024-02-23T00:52:23.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:22.287747+0000 mon.c (mon.2) 348 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-104"}]: dispatch 2024-02-23T00:52:23.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:22.288031+0000 mon.a (mon.0) 3161 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-104"}]: dispatch 2024-02-23T00:52:23.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:22.387406+0000 osd.7 (osd.7) 295 : cluster [DBG] 2.0 scrub starts 2024-02-23T00:52:23.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:22.390391+0000 osd.7 (osd.7) 296 : cluster [DBG] 2.0 scrub ok 2024-02-23T00:52:23.393 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:52:23.426 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:23.447 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:52:23.449 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:23.450 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:52:23.550 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:52:23.578 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:21.534118+0000 osd.1 (osd.1) 259 : cluster [DBG] 2.10 scrub starts 2024-02-23T00:52:23.578 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:21.536415+0000 osd.1 (osd.1) 260 : cluster [DBG] 2.10 scrub ok 2024-02-23T00:52:23.578 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:22.034495+0000 osd.7 (osd.7) 293 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:52:23.578 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:22.034843+0000 osd.7 (osd.7) 294 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:52:23.578 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:22.284425+0000 mon.a (mon.0) 3159 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:52:23.578 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:22.284496+0000 mon.a (mon.0) 3160 : cluster [DBG] osdmap e613: 8 total, 8 up, 7 in 2024-02-23T00:52:23.578 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:22.287747+0000 mon.c (mon.2) 348 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-104"}]: dispatch 2024-02-23T00:52:23.578 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:22.288031+0000 mon.a (mon.0) 3161 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-104"}]: dispatch 2024-02-23T00:52:23.578 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:22.387406+0000 osd.7 (osd.7) 295 : cluster [DBG] 2.0 scrub starts 2024-02-23T00:52:23.579 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:22.390391+0000 osd.7 (osd.7) 296 : cluster [DBG] 2.0 scrub ok 2024-02-23T00:52:23.580 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:23.600 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:52:23.602 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:23.603 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:52:23.703 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:52:23.733 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:24.225 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.1 dump_blocked_ops 2024-02-23T00:52:24.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:22.748956+0000 osd.3 (osd.3) 288 : cluster [DBG] 4.14 scrub starts 2024-02-23T00:52:24.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:22.751143+0000 osd.3 (osd.3) 289 : cluster [DBG] 4.14 scrub ok 2024-02-23T00:52:24.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:22.796656+0000 mgr.y (mgr.14517) 545 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:52:24.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:23.281924+0000 mon.a (mon.0) 3162 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:52:24.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:23.286516+0000 mon.a (mon.0) 3163 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-104"}]': finished 2024-02-23T00:52:24.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:23.286588+0000 mon.a (mon.0) 3164 : cluster [DBG] osdmap e614: 8 total, 8 up, 7 in 2024-02-23T00:52:24.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:23.355848+0000 mgr.y (mgr.14517) 546 : cluster [DBG] pgmap v925: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-23T00:52:24.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:23.775694+0000 osd.3 (osd.3) 290 : cluster [DBG] 6.b deep-scrub starts 2024-02-23T00:52:24.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:23.776768+0000 osd.3 (osd.3) 291 : cluster [DBG] 6.b deep-scrub ok 2024-02-23T00:52:24.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:22.748956+0000 osd.3 (osd.3) 288 : cluster [DBG] 4.14 scrub starts 2024-02-23T00:52:24.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:22.751143+0000 osd.3 (osd.3) 289 : cluster [DBG] 4.14 scrub ok 2024-02-23T00:52:24.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:22.796656+0000 mgr.y (mgr.14517) 545 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:52:24.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:23.281924+0000 mon.a (mon.0) 3162 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:52:24.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:23.286516+0000 mon.a (mon.0) 3163 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-104"}]': finished 2024-02-23T00:52:24.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:23.286588+0000 mon.a (mon.0) 3164 : cluster [DBG] osdmap e614: 8 total, 8 up, 7 in 2024-02-23T00:52:24.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:23.355848+0000 mgr.y (mgr.14517) 546 : cluster [DBG] pgmap v925: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-23T00:52:24.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:23.775694+0000 osd.3 (osd.3) 290 : cluster [DBG] 6.b deep-scrub starts 2024-02-23T00:52:24.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:23.776768+0000 osd.3 (osd.3) 291 : cluster [DBG] 6.b deep-scrub ok 2024-02-23T00:52:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:22.748956+0000 osd.3 (osd.3) 288 : cluster [DBG] 4.14 scrub starts 2024-02-23T00:52:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:22.751143+0000 osd.3 (osd.3) 289 : cluster [DBG] 4.14 scrub ok 2024-02-23T00:52:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:22.796656+0000 mgr.y (mgr.14517) 545 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:52:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:23.281924+0000 mon.a (mon.0) 3162 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:52:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:23.286516+0000 mon.a (mon.0) 3163 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-104"}]': finished 2024-02-23T00:52:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:23.286588+0000 mon.a (mon.0) 3164 : cluster [DBG] osdmap e614: 8 total, 8 up, 7 in 2024-02-23T00:52:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:23.355848+0000 mgr.y (mgr.14517) 546 : cluster [DBG] pgmap v925: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-23T00:52:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:23.775694+0000 osd.3 (osd.3) 290 : cluster [DBG] 6.b deep-scrub starts 2024-02-23T00:52:24.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:23.776768+0000 osd.3 (osd.3) 291 : cluster [DBG] 6.b deep-scrub ok 2024-02-23T00:52:25.429 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:23.705755+0000 osd.0 (osd.0) 255 : cluster [DBG] 4.3 scrub starts 2024-02-23T00:52:25.429 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:23.707985+0000 osd.0 (osd.0) 256 : cluster [DBG] 4.3 scrub ok 2024-02-23T00:52:25.429 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:24.298243+0000 mon.a (mon.0) 3165 : cluster [DBG] osdmap e615: 8 total, 8 up, 7 in 2024-02-23T00:52:25.429 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:24.676648+0000 osd.0 (osd.0) 257 : cluster [DBG] 18.0 scrub starts 2024-02-23T00:52:25.429 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:24.677782+0000 osd.0 (osd.0) 258 : cluster [DBG] 18.0 scrub ok 2024-02-23T00:52:25.429 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:25.261200+0000 mon.a (mon.0) 3166 : cluster [DBG] osdmap e616: 8 total, 8 up, 7 in 2024-02-23T00:52:25.429 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:25.263834+0000 mon.c (mon.2) 349 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-106","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:52:25.430 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:25.266197+0000 mon.a (mon.0) 3167 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-106","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:52:25.430 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:23.705755+0000 osd.0 (osd.0) 255 : cluster [DBG] 4.3 scrub starts 2024-02-23T00:52:25.430 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:23.707985+0000 osd.0 (osd.0) 256 : cluster [DBG] 4.3 scrub ok 2024-02-23T00:52:25.430 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:24.298243+0000 mon.a (mon.0) 3165 : cluster [DBG] osdmap e615: 8 total, 8 up, 7 in 2024-02-23T00:52:25.430 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:24.676648+0000 osd.0 (osd.0) 257 : cluster [DBG] 18.0 scrub starts 2024-02-23T00:52:25.430 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:24.677782+0000 osd.0 (osd.0) 258 : cluster [DBG] 18.0 scrub ok 2024-02-23T00:52:25.430 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:25.261200+0000 mon.a (mon.0) 3166 : cluster [DBG] osdmap e616: 8 total, 8 up, 7 in 2024-02-23T00:52:25.430 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:25.263834+0000 mon.c (mon.2) 349 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-106","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:52:25.430 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:25.266197+0000 mon.a (mon.0) 3167 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-106","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:52:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:52:25.690 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:25.690 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:52:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:23.705755+0000 osd.0 (osd.0) 255 : cluster [DBG] 4.3 scrub starts 2024-02-23T00:52:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:23.707985+0000 osd.0 (osd.0) 256 : cluster [DBG] 4.3 scrub ok 2024-02-23T00:52:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:24.298243+0000 mon.a (mon.0) 3165 : cluster [DBG] osdmap e615: 8 total, 8 up, 7 in 2024-02-23T00:52:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:24.676648+0000 osd.0 (osd.0) 257 : cluster [DBG] 18.0 scrub starts 2024-02-23T00:52:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:24.677782+0000 osd.0 (osd.0) 258 : cluster [DBG] 18.0 scrub ok 2024-02-23T00:52:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:25.261200+0000 mon.a (mon.0) 3166 : cluster [DBG] osdmap e616: 8 total, 8 up, 7 in 2024-02-23T00:52:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:25.263834+0000 mon.c (mon.2) 349 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-106","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:52:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:25.266197+0000 mon.a (mon.0) 3167 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-106","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:52:25.791 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:52:25.823 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:25.845 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:52:25.847 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:25.847 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:52:25.948 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:52:25.979 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:26.017 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:52:26.019 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:26.020 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:52:26.121 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:52:26.153 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:26.176 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:52:26.179 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:26.179 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:52:26.280 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:52:26.294 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-23T00:52:26.294 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-23T00:52:26.294 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-02-23T00:52:26.295 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-02-23T00:52:26.295 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-02-23T00:52:26.295 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.0 config set filestore_inject_stall 3 2024-02-23T00:52:26.304 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:25.356283+0000 mgr.y (mgr.14517) 547 : cluster [DBG] pgmap v928: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:52:26.304 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:25.402798+0000 osd.7 (osd.7) 297 : cluster [DBG] 2.8 scrub starts 2024-02-23T00:52:26.304 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:25.405676+0000 osd.7 (osd.7) 298 : cluster [DBG] 2.8 scrub ok 2024-02-23T00:52:26.304 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:25.489990+0000 osd.5 (osd.5) 268 : cluster [DBG] 6.a scrub starts 2024-02-23T00:52:26.304 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:25.491454+0000 osd.5 (osd.5) 269 : cluster [DBG] 6.a scrub ok 2024-02-23T00:52:26.304 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:25.553241+0000 osd.1 (osd.1) 261 : cluster [DBG] 2.d scrub starts 2024-02-23T00:52:26.304 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:25.554387+0000 osd.1 (osd.1) 262 : cluster [DBG] 2.d scrub ok 2024-02-23T00:52:26.304 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:25.760351+0000 osd.3 (osd.3) 292 : cluster [DBG] 4.1d deep-scrub starts 2024-02-23T00:52:26.304 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:25.764384+0000 osd.3 (osd.3) 293 : cluster [DBG] 4.1d deep-scrub ok 2024-02-23T00:52:26.305 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:26.124904+0000 mon.a (mon.0) 3168 : audit [DBG] from='client.? 172.21.15.47:0/3816125392' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:52:26.305 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:26.261504+0000 mon.a (mon.0) 3169 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-106","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:52:26.305 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:26.261623+0000 mon.a (mon.0) 3170 : cluster [DBG] osdmap e617: 8 total, 8 up, 7 in 2024-02-23T00:52:26.305 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:25.356283+0000 mgr.y (mgr.14517) 547 : cluster [DBG] pgmap v928: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:52:26.305 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:25.402798+0000 osd.7 (osd.7) 297 : cluster [DBG] 2.8 scrub starts 2024-02-23T00:52:26.305 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:25.405676+0000 osd.7 (osd.7) 298 : cluster [DBG] 2.8 scrub ok 2024-02-23T00:52:26.305 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:25.489990+0000 osd.5 (osd.5) 268 : cluster [DBG] 6.a scrub starts 2024-02-23T00:52:26.305 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:25.491454+0000 osd.5 (osd.5) 269 : cluster [DBG] 6.a scrub ok 2024-02-23T00:52:26.305 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:25.553241+0000 osd.1 (osd.1) 261 : cluster [DBG] 2.d scrub starts 2024-02-23T00:52:26.305 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:25.554387+0000 osd.1 (osd.1) 262 : cluster [DBG] 2.d scrub ok 2024-02-23T00:52:26.305 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:25.760351+0000 osd.3 (osd.3) 292 : cluster [DBG] 4.1d deep-scrub starts 2024-02-23T00:52:26.305 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:25.764384+0000 osd.3 (osd.3) 293 : cluster [DBG] 4.1d deep-scrub ok 2024-02-23T00:52:26.306 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:26.124904+0000 mon.a (mon.0) 3168 : audit [DBG] from='client.? 172.21.15.47:0/3816125392' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:52:26.306 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:26.261504+0000 mon.a (mon.0) 3169 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-106","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:52:26.306 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:26.261623+0000 mon.a (mon.0) 3170 : cluster [DBG] osdmap e617: 8 total, 8 up, 7 in 2024-02-23T00:52:26.310 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:26.330 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:52:26.332 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:26.332 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:52:26.433 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:52:26.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:25.356283+0000 mgr.y (mgr.14517) 547 : cluster [DBG] pgmap v928: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:52:26.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:25.402798+0000 osd.7 (osd.7) 297 : cluster [DBG] 2.8 scrub starts 2024-02-23T00:52:26.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:25.405676+0000 osd.7 (osd.7) 298 : cluster [DBG] 2.8 scrub ok 2024-02-23T00:52:26.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:25.489990+0000 osd.5 (osd.5) 268 : cluster [DBG] 6.a scrub starts 2024-02-23T00:52:26.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:25.491454+0000 osd.5 (osd.5) 269 : cluster [DBG] 6.a scrub ok 2024-02-23T00:52:26.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:25.553241+0000 osd.1 (osd.1) 261 : cluster [DBG] 2.d scrub starts 2024-02-23T00:52:26.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:25.554387+0000 osd.1 (osd.1) 262 : cluster [DBG] 2.d scrub ok 2024-02-23T00:52:26.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:25.760351+0000 osd.3 (osd.3) 292 : cluster [DBG] 4.1d deep-scrub starts 2024-02-23T00:52:26.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:25.764384+0000 osd.3 (osd.3) 293 : cluster [DBG] 4.1d deep-scrub ok 2024-02-23T00:52:26.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:26.124904+0000 mon.a (mon.0) 3168 : audit [DBG] from='client.? 172.21.15.47:0/3816125392' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:52:26.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:26.261504+0000 mon.a (mon.0) 3169 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-106","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:52:26.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:26.261623+0000 mon.a (mon.0) 3170 : cluster [DBG] osdmap e617: 8 total, 8 up, 7 in 2024-02-23T00:52:26.460 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:26.479 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:52:26.481 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:26.482 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:52:26.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.120+0000 7f1b5df74700 1 -- 172.21.15.47:0/3416463131 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1b58102210 msgr2=0x7f1b58104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:26.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.120+0000 7f1b5df74700 1 --2- 172.21.15.47:0/3416463131 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1b58102210 0x7f1b58104600 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 rx=0x7f1b50009be0 tx=0x7f1b50004d10).stop 2024-02-23T00:52:26.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.120+0000 7f1b5df74700 1 -- 172.21.15.47:0/3416463131 shutdown_connections 2024-02-23T00:52:26.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.120+0000 7f1b5df74700 1 --2- 172.21.15.47:0/3416463131 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1b58102210 0x7f1b58104600 unknown :-1 s=CLOSED pgs=2754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:26.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.120+0000 7f1b5df74700 1 --2- 172.21.15.47:0/3416463131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b580f8cb0 0x7f1b580f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:26.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.120+0000 7f1b5df74700 1 --2- 172.21.15.47:0/3416463131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b580f9610 0x7f1b58101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:26.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.120+0000 7f1b5df74700 1 -- 172.21.15.47:0/3416463131 >> 172.21.15.47:0/3416463131 conn(0x7f1b580f2820 msgr2=0x7f1b580f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:52:26.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.120+0000 7f1b5df74700 1 -- 172.21.15.47:0/3416463131 shutdown_connections 2024-02-23T00:52:26.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.120+0000 7f1b5df74700 1 -- 172.21.15.47:0/3416463131 wait complete. 2024-02-23T00:52:26.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.121+0000 7f1b5df74700 1 librados: starting msgr at 2024-02-23T00:52:26.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.121+0000 7f1b5df74700 1 librados: starting objecter 2024-02-23T00:52:26.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.121+0000 7f1b5df74700 1 Processor -- start 2024-02-23T00:52:26.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.121+0000 7f1b5df74700 1 -- start start 2024-02-23T00:52:26.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.121+0000 7f1b5df74700 1 librados: setting wanted keys 2024-02-23T00:52:26.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.121+0000 7f1b5df74700 1 librados: calling monclient init 2024-02-23T00:52:26.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.121+0000 7f1b5df74700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1b580f8cb0 0x7f1b580f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:26.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.121+0000 7f1b5df74700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b580f9610 0x7f1b580f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:26.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.121+0000 7f1b5df74700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b58102210 0x7f1b580f7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:26.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.121+0000 7f1b5df74700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b581163e0 con 0x7f1b580f9610 2024-02-23T00:52:26.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.121+0000 7f1b5df74700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b58116550 con 0x7f1b58102210 2024-02-23T00:52:26.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.121+0000 7f1b5df74700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b58116710 con 0x7f1b580f8cb0 2024-02-23T00:52:26.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.121+0000 7f1b56ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b580f9610 0x7f1b580f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:26.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.121+0000 7f1b577fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1b580f8cb0 0x7f1b580f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:26.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.121+0000 7f1b56ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b580f9610 0x7f1b580f7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39932/0 (socket says 172.21.15.47:39932) 2024-02-23T00:52:26.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.121+0000 7f1b56ffd700 1 -- 172.21.15.47:0/3816125392 learned_addr learned my addr 172.21.15.47:0/3816125392 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:52:26.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.121+0000 7f1b57fff700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b58102210 0x7f1b580f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:26.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.122+0000 7f1b56ffd700 1 -- 172.21.15.47:0/3816125392 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1b580f8cb0 msgr2=0x7f1b580f6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:26.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.122+0000 7f1b56ffd700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1b580f8cb0 0x7f1b580f6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:26.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.122+0000 7f1b56ffd700 1 -- 172.21.15.47:0/3816125392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b58102210 msgr2=0x7f1b580f7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:26.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.122+0000 7f1b56ffd700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b58102210 0x7f1b580f7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:26.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.122+0000 7f1b56ffd700 1 -- 172.21.15.47:0/3816125392 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b40005040 con 0x7f1b580f9610 2024-02-23T00:52:26.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.122+0000 7f1b577fe700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1b580f8cb0 0x7f1b580f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:52:26.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.122+0000 7f1b57fff700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b58102210 0x7f1b580f7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:52:26.535 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:52:26.535 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:52:26.536 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:52:26.536 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:52:26.536 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:52:26.536 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:52:26.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.122+0000 7f1b56ffd700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b580f9610 0x7f1b580f7150 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 rx=0x7f1b400048f0 tx=0x7f1b40009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:26.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.122+0000 7f1b54ff9700 10 client.?.objecter ms_handle_connect 0x7f1b580f9610 2024-02-23T00:52:26.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.122+0000 7f1b54ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:52:26.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.122+0000 7f1b54ff9700 1 -- 172.21.15.47:0/3816125392 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b40007b00 con 0x7f1b580f9610 2024-02-23T00:52:26.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.122+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b500097c0 con 0x7f1b580f9610 2024-02-23T00:52:26.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.122+0000 7f1b54ff9700 1 -- 172.21.15.47:0/3816125392 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f1b4000f040 con 0x7f1b580f9610 2024-02-23T00:52:26.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.122+0000 7f1b54ff9700 1 -- 172.21.15.47:0/3816125392 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b40014890 con 0x7f1b580f9610 2024-02-23T00:52:26.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.122+0000 7f1b5df74700 10 client.31482.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:52:26.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.122+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b58116d70 con 0x7f1b580f9610 2024-02-23T00:52:26.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.122+0000 7f1b5df74700 1 librados: init done 2024-02-23T00:52:26.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.123+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f1b5805fa60 con 0x7f1b580f9610 2024-02-23T00:52:26.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.124+0000 7f1b54ff9700 1 -- 172.21.15.47:0/3816125392 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f1b40014cc0 con 0x7f1b580f9610 2024-02-23T00:52:26.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.124+0000 7f1b54ff9700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1b44062cc0 0x7f1b44065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:26.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.124+0000 7f1b54ff9700 1 -- 172.21.15.47:0/3816125392 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(616..616 src has 482..616) v4 ==== 9570+0+0 (secure 0 0 0) 0x7f1b4007d670 con 0x7f1b580f9610 2024-02-23T00:52:26.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.124+0000 7f1b54ff9700 10 client.31482.objecter ms_dispatch 0x7f1b58001550 osd_map(616..616 src has 482..616) v4 2024-02-23T00:52:26.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.124+0000 7f1b54ff9700 3 client.31482.objecter handle_osd_map got epochs [616,616] > 0 2024-02-23T00:52:26.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.124+0000 7f1b54ff9700 3 client.31482.objecter handle_osd_map decoding full epoch 616 2024-02-23T00:52:26.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.124+0000 7f1b577fe700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1b44062cc0 0x7f1b44065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:26.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.124+0000 7f1b54ff9700 20 client.31482.objecter dump_active .. 0 homeless 2024-02-23T00:52:26.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.124+0000 7f1b54ff9700 1 -- 172.21.15.47:0/3816125392 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v616) v1 ==== 54+0+16 (secure 0 0 0) 0x7f1b4007db50 con 0x7f1b580f9610 2024-02-23T00:52:26.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.124+0000 7f1b54ff9700 10 client.31482.objecter ms_handle_connect 0x7f1b44062cc0 2024-02-23T00:52:26.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.124+0000 7f1b577fe700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1b44062cc0 0x7f1b44065180 secure :-1 s=READY pgs=3143 cs=0 l=1 rev1=1 rx=0x7f1b480103c0 tx=0x7f1b4800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:26.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.124+0000 7f1b5df74700 10 client.31482.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:26.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.125+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_assign -1 1 2024-02-23T00:52:26.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.125+0000 7f1b5df74700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f1b580fc1a0 0x7f1b580fc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:26.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.125+0000 7f1b5df74700 20 client.31482.objecter _get_session s=0x7f1b5805ee80 osd=0 3 2024-02-23T00:52:26.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.125+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b5805ee80 osd=0 3 2024-02-23T00:52:26.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.125+0000 7f1b5df74700 20 client.31482.objecter _get_session s=0x7f1b5805ee80 osd=0 3 2024-02-23T00:52:26.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.125+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_remove -1 1 2024-02-23T00:52:26.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.125+0000 7f1b5df74700 20 client.31482.objecter get_session s=0x7f1b5805ee80 osd=0 3 2024-02-23T00:52:26.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.125+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_assign 0 1 2024-02-23T00:52:26.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.125+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b5805ee80 osd=0 4 2024-02-23T00:52:26.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.125+0000 7f1b5df74700 10 client.31482.objecter _send_command 1 2024-02-23T00:52:26.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.125+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b58107050 con 0x7f1b580fc1a0 2024-02-23T00:52:26.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.125+0000 7f1b57fff700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f1b580fc1a0 0x7f1b580fc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:26.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.125+0000 7f1b54ff9700 10 client.31482.objecter ms_handle_connect 0x7f1b580fc1a0 2024-02-23T00:52:26.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.125+0000 7f1b57fff700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f1b580fc1a0 0x7f1b580fc580 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:26.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.126+0000 7f1b54ff9700 1 -- 172.21.15.47:0/3816125392 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1b58107050 con 0x7f1b580fc1a0 2024-02-23T00:52:26.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.126+0000 7f1b54ff9700 10 client.31482.objecter ms_dispatch 0x7f1b58001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:52:26.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.126+0000 7f1b54ff9700 10 client.31482.objecter _finish_command 1 = system:0 2024-02-23T00:52:26.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.126+0000 7f1b54ff9700 20 client.31482.objecter put_session s=0x7f1b5805ee80 osd=0 4 2024-02-23T00:52:26.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.126+0000 7f1b54ff9700 15 client.31482.objecter _session_command_op_remove 0 1 2024-02-23T00:52:26.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.140+0000 7f1b5df74700 10 client.31482.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:52:26.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.140+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_assign -1 2 2024-02-23T00:52:26.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.140+0000 7f1b5df74700 20 client.31482.objecter _get_session s=0x7f1b5805ee80 osd=0 3 2024-02-23T00:52:26.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.140+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b5805ee80 osd=0 3 2024-02-23T00:52:26.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.140+0000 7f1b5df74700 20 client.31482.objecter _get_session s=0x7f1b5805ee80 osd=0 3 2024-02-23T00:52:26.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.140+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_remove -1 2 2024-02-23T00:52:26.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.140+0000 7f1b5df74700 20 client.31482.objecter get_session s=0x7f1b5805ee80 osd=0 3 2024-02-23T00:52:26.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.140+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_assign 0 2 2024-02-23T00:52:26.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.140+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b5805ee80 osd=0 4 2024-02-23T00:52:26.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.140+0000 7f1b5df74700 10 client.31482.objecter _send_command 2 2024-02-23T00:52:26.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.140+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1b580273c0 con 0x7f1b580fc1a0 2024-02-23T00:52:26.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.142+0000 7f1b54ff9700 1 -- 172.21.15.47:0/3816125392 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1b580273c0 con 0x7f1b580fc1a0 2024-02-23T00:52:26.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.142+0000 7f1b54ff9700 10 client.31482.objecter ms_dispatch 0x7f1b58001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:52:26.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.142+0000 7f1b54ff9700 10 client.31482.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:52:26.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.142+0000 7f1b54ff9700 20 client.31482.objecter put_session s=0x7f1b5805ee80 osd=0 4 2024-02-23T00:52:26.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.142+0000 7f1b54ff9700 15 client.31482.objecter _session_command_op_remove 0 2 2024-02-23T00:52:26.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.142+0000 7f1b5df74700 10 client.31482.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:26.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.142+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_assign -1 3 2024-02-23T00:52:26.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.142+0000 7f1b5df74700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f1b580598f0 0x7f1b58059cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:26.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.142+0000 7f1b5df74700 20 client.31482.objecter _get_session s=0x7f1b58028710 osd=1 3 2024-02-23T00:52:26.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.142+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b58028710 osd=1 3 2024-02-23T00:52:26.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.142+0000 7f1b5df74700 20 client.31482.objecter _get_session s=0x7f1b58028710 osd=1 3 2024-02-23T00:52:26.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.142+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_remove -1 3 2024-02-23T00:52:26.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.142+0000 7f1b5df74700 20 client.31482.objecter get_session s=0x7f1b58028710 osd=1 3 2024-02-23T00:52:26.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.142+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_assign 1 3 2024-02-23T00:52:26.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.142+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b58028710 osd=1 4 2024-02-23T00:52:26.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.142+0000 7f1b5df74700 10 client.31482.objecter _send_command 3 2024-02-23T00:52:26.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.142+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b5811d770 con 0x7f1b580598f0 2024-02-23T00:52:26.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.142+0000 7f1b577fe700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f1b580598f0 0x7f1b58059cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:26.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.142+0000 7f1b54ff9700 10 client.31482.objecter ms_handle_connect 0x7f1b580598f0 2024-02-23T00:52:26.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.142+0000 7f1b577fe700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f1b580598f0 0x7f1b58059cf0 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:26.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.143+0000 7f1b54ff9700 1 -- 172.21.15.47:0/3816125392 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1b5811d770 con 0x7f1b580598f0 2024-02-23T00:52:26.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.143+0000 7f1b54ff9700 10 client.31482.objecter ms_dispatch 0x7f1b58001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:52:26.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.143+0000 7f1b54ff9700 10 client.31482.objecter _finish_command 3 = system:0 2024-02-23T00:52:26.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.143+0000 7f1b54ff9700 20 client.31482.objecter put_session s=0x7f1b58028710 osd=1 4 2024-02-23T00:52:26.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.143+0000 7f1b54ff9700 15 client.31482.objecter _session_command_op_remove 1 3 2024-02-23T00:52:26.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.158+0000 7f1b5df74700 10 client.31482.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:52:26.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.158+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_assign -1 4 2024-02-23T00:52:26.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.158+0000 7f1b5df74700 20 client.31482.objecter _get_session s=0x7f1b58028710 osd=1 3 2024-02-23T00:52:26.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.158+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b58028710 osd=1 3 2024-02-23T00:52:26.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.158+0000 7f1b5df74700 20 client.31482.objecter _get_session s=0x7f1b58028710 osd=1 3 2024-02-23T00:52:26.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.158+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_remove -1 4 2024-02-23T00:52:26.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.158+0000 7f1b5df74700 20 client.31482.objecter get_session s=0x7f1b58028710 osd=1 3 2024-02-23T00:52:26.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.158+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_assign 1 4 2024-02-23T00:52:26.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.158+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b58028710 osd=1 4 2024-02-23T00:52:26.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.158+0000 7f1b5df74700 10 client.31482.objecter _send_command 4 2024-02-23T00:52:26.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.158+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1b58059360 con 0x7f1b580598f0 2024-02-23T00:52:26.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.159+0000 7f1b54ff9700 1 -- 172.21.15.47:0/3816125392 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1b58059360 con 0x7f1b580598f0 2024-02-23T00:52:26.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.159+0000 7f1b54ff9700 10 client.31482.objecter ms_dispatch 0x7f1b58001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:52:26.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.159+0000 7f1b54ff9700 10 client.31482.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:52:26.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.159+0000 7f1b54ff9700 20 client.31482.objecter put_session s=0x7f1b58028710 osd=1 4 2024-02-23T00:52:26.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.159+0000 7f1b54ff9700 15 client.31482.objecter _session_command_op_remove 1 4 2024-02-23T00:52:26.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.159+0000 7f1b5df74700 10 client.31482.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:26.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.159+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_assign -1 5 2024-02-23T00:52:26.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.159+0000 7f1b5df74700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f1b58028270 0x7f1b5811c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:26.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.159+0000 7f1b5df74700 20 client.31482.objecter _get_session s=0x7f1b58059770 osd=2 3 2024-02-23T00:52:26.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.159+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b58059770 osd=2 3 2024-02-23T00:52:26.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.159+0000 7f1b5df74700 20 client.31482.objecter _get_session s=0x7f1b58059770 osd=2 3 2024-02-23T00:52:26.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.159+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_remove -1 5 2024-02-23T00:52:26.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.159+0000 7f1b5df74700 20 client.31482.objecter get_session s=0x7f1b58059770 osd=2 3 2024-02-23T00:52:26.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.159+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_assign 2 5 2024-02-23T00:52:26.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.159+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b58059770 osd=2 4 2024-02-23T00:52:26.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.159+0000 7f1b5df74700 10 client.31482.objecter _send_command 5 2024-02-23T00:52:26.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.159+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b5811c680 con 0x7f1b58028270 2024-02-23T00:52:26.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.160+0000 7f1b56ffd700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f1b58028270 0x7f1b5811c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:26.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.160+0000 7f1b54ff9700 10 client.31482.objecter ms_handle_connect 0x7f1b58028270 2024-02-23T00:52:26.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.160+0000 7f1b56ffd700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f1b58028270 0x7f1b5811c190 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:26.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.160+0000 7f1b54ff9700 1 -- 172.21.15.47:0/3816125392 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1b5811c680 con 0x7f1b58028270 2024-02-23T00:52:26.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.160+0000 7f1b54ff9700 10 client.31482.objecter ms_dispatch 0x7f1b58001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:52:26.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.160+0000 7f1b54ff9700 10 client.31482.objecter _finish_command 5 = system:0 2024-02-23T00:52:26.546 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:52:26.546 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:52:26.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.160+0000 7f1b54ff9700 20 client.31482.objecter put_session s=0x7f1b58059770 osd=2 4 2024-02-23T00:52:26.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.160+0000 7f1b54ff9700 15 client.31482.objecter _session_command_op_remove 2 5 2024-02-23T00:52:26.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.175+0000 7f1b5df74700 10 client.31482.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:52:26.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.175+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_assign -1 6 2024-02-23T00:52:26.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.175+0000 7f1b5df74700 20 client.31482.objecter _get_session s=0x7f1b58059770 osd=2 3 2024-02-23T00:52:26.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.175+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b58059770 osd=2 3 2024-02-23T00:52:26.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.175+0000 7f1b5df74700 20 client.31482.objecter _get_session s=0x7f1b58059770 osd=2 3 2024-02-23T00:52:26.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.175+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_remove -1 6 2024-02-23T00:52:26.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.175+0000 7f1b5df74700 20 client.31482.objecter get_session s=0x7f1b58059770 osd=2 3 2024-02-23T00:52:26.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.175+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_assign 2 6 2024-02-23T00:52:26.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.175+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b58059770 osd=2 4 2024-02-23T00:52:26.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.175+0000 7f1b5df74700 10 client.31482.objecter _send_command 6 2024-02-23T00:52:26.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.175+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1b580277d0 con 0x7f1b58028270 2024-02-23T00:52:26.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.176+0000 7f1b54ff9700 1 -- 172.21.15.47:0/3816125392 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1b580277d0 con 0x7f1b58028270 2024-02-23T00:52:26.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.176+0000 7f1b54ff9700 10 client.31482.objecter ms_dispatch 0x7f1b58001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:52:26.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.176+0000 7f1b54ff9700 10 client.31482.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:52:26.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.176+0000 7f1b54ff9700 20 client.31482.objecter put_session s=0x7f1b58059770 osd=2 4 2024-02-23T00:52:26.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.176+0000 7f1b54ff9700 15 client.31482.objecter _session_command_op_remove 2 6 2024-02-23T00:52:26.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.176+0000 7f1b5df74700 10 client.31482.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:26.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.177+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_assign -1 7 2024-02-23T00:52:26.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.177+0000 7f1b5df74700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f1b5811f8f0 0x7f1b58121ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:26.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.177+0000 7f1b5df74700 20 client.31482.objecter _get_session s=0x7f1b58027be0 osd=3 3 2024-02-23T00:52:26.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.177+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b58027be0 osd=3 3 2024-02-23T00:52:26.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.177+0000 7f1b5df74700 20 client.31482.objecter _get_session s=0x7f1b58027be0 osd=3 3 2024-02-23T00:52:26.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.177+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_remove -1 7 2024-02-23T00:52:26.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.177+0000 7f1b5df74700 20 client.31482.objecter get_session s=0x7f1b58027be0 osd=3 3 2024-02-23T00:52:26.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.177+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_assign 3 7 2024-02-23T00:52:26.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.177+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b58027be0 osd=3 4 2024-02-23T00:52:26.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.177+0000 7f1b5df74700 10 client.31482.objecter _send_command 7 2024-02-23T00:52:26.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.177+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b58124930 con 0x7f1b5811f8f0 2024-02-23T00:52:26.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.177+0000 7f1b57fff700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f1b5811f8f0 0x7f1b58121ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:26.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.177+0000 7f1b54ff9700 10 client.31482.objecter ms_handle_connect 0x7f1b5811f8f0 2024-02-23T00:52:26.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.177+0000 7f1b57fff700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f1b5811f8f0 0x7f1b58121ce0 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:26.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.177+0000 7f1b54ff9700 1 -- 172.21.15.47:0/3816125392 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1b58124930 con 0x7f1b5811f8f0 2024-02-23T00:52:26.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.177+0000 7f1b54ff9700 10 client.31482.objecter ms_dispatch 0x7f1b58001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:52:26.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.177+0000 7f1b54ff9700 10 client.31482.objecter _finish_command 7 = system:0 2024-02-23T00:52:26.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.177+0000 7f1b54ff9700 20 client.31482.objecter put_session s=0x7f1b58027be0 osd=3 4 2024-02-23T00:52:26.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.177+0000 7f1b54ff9700 15 client.31482.objecter _session_command_op_remove 3 7 2024-02-23T00:52:26.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.192+0000 7f1b5df74700 10 client.31482.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:52:26.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.192+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_assign -1 8 2024-02-23T00:52:26.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.192+0000 7f1b5df74700 20 client.31482.objecter _get_session s=0x7f1b58027be0 osd=3 3 2024-02-23T00:52:26.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.192+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b58027be0 osd=3 3 2024-02-23T00:52:26.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.192+0000 7f1b5df74700 20 client.31482.objecter _get_session s=0x7f1b58027be0 osd=3 3 2024-02-23T00:52:26.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.192+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_remove -1 8 2024-02-23T00:52:26.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.192+0000 7f1b5df74700 20 client.31482.objecter get_session s=0x7f1b58027be0 osd=3 3 2024-02-23T00:52:26.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.192+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_assign 3 8 2024-02-23T00:52:26.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.192+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b58027be0 osd=3 4 2024-02-23T00:52:26.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.192+0000 7f1b5df74700 10 client.31482.objecter _send_command 8 2024-02-23T00:52:26.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.192+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1b5811ca90 con 0x7f1b5811f8f0 2024-02-23T00:52:26.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.193+0000 7f1b54ff9700 1 -- 172.21.15.47:0/3816125392 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1b5811ca90 con 0x7f1b5811f8f0 2024-02-23T00:52:26.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.193+0000 7f1b54ff9700 10 client.31482.objecter ms_dispatch 0x7f1b58001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:52:26.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.193+0000 7f1b54ff9700 10 client.31482.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:52:26.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.193+0000 7f1b54ff9700 20 client.31482.objecter put_session s=0x7f1b58027be0 osd=3 4 2024-02-23T00:52:26.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.193+0000 7f1b54ff9700 15 client.31482.objecter _session_command_op_remove 3 8 2024-02-23T00:52:26.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.193+0000 7f1b5df74700 10 client.31482.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:26.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.193+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_assign -1 9 2024-02-23T00:52:26.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.194+0000 7f1b5df74700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f1b5811cc00 0x7f1b58123350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:26.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.194+0000 7f1b5df74700 20 client.31482.objecter _get_session s=0x7f1b5811ca90 osd=4 3 2024-02-23T00:52:26.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.194+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b5811ca90 osd=4 3 2024-02-23T00:52:26.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.194+0000 7f1b5df74700 20 client.31482.objecter _get_session s=0x7f1b5811ca90 osd=4 3 2024-02-23T00:52:26.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.194+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_remove -1 9 2024-02-23T00:52:26.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.194+0000 7f1b5df74700 20 client.31482.objecter get_session s=0x7f1b5811ca90 osd=4 3 2024-02-23T00:52:26.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.194+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_assign 4 9 2024-02-23T00:52:26.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.194+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b5811ca90 osd=4 4 2024-02-23T00:52:26.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.194+0000 7f1b5df74700 10 client.31482.objecter _send_command 9 2024-02-23T00:52:26.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.194+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b5811be00 con 0x7f1b5811cc00 2024-02-23T00:52:26.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.194+0000 7f1b577fe700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f1b5811cc00 0x7f1b58123350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:26.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.194+0000 7f1b577fe700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f1b5811cc00 0x7f1b58123350 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:26.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.194+0000 7f1b54ff9700 10 client.31482.objecter ms_handle_connect 0x7f1b5811cc00 2024-02-23T00:52:26.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.195+0000 7f1b54ff9700 1 -- 172.21.15.47:0/3816125392 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1b5811be00 con 0x7f1b5811cc00 2024-02-23T00:52:26.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.195+0000 7f1b54ff9700 10 client.31482.objecter ms_dispatch 0x7f1b58001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:52:26.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.195+0000 7f1b54ff9700 10 client.31482.objecter _finish_command 9 = system:0 2024-02-23T00:52:26.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.195+0000 7f1b54ff9700 20 client.31482.objecter put_session s=0x7f1b5811ca90 osd=4 4 2024-02-23T00:52:26.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.195+0000 7f1b54ff9700 15 client.31482.objecter _session_command_op_remove 4 9 2024-02-23T00:52:26.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.210+0000 7f1b5df74700 10 client.31482.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:52:26.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.210+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_assign -1 10 2024-02-23T00:52:26.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.210+0000 7f1b5df74700 20 client.31482.objecter _get_session s=0x7f1b5811ca90 osd=4 3 2024-02-23T00:52:26.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.210+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b5811ca90 osd=4 3 2024-02-23T00:52:26.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.210+0000 7f1b5df74700 20 client.31482.objecter _get_session s=0x7f1b5811ca90 osd=4 3 2024-02-23T00:52:26.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.210+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_remove -1 10 2024-02-23T00:52:26.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.210+0000 7f1b5df74700 20 client.31482.objecter get_session s=0x7f1b5811ca90 osd=4 3 2024-02-23T00:52:26.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.210+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_assign 4 10 2024-02-23T00:52:26.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.210+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b5811ca90 osd=4 4 2024-02-23T00:52:26.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.210+0000 7f1b5df74700 10 client.31482.objecter _send_command 10 2024-02-23T00:52:26.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.210+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1b5811be00 con 0x7f1b5811cc00 2024-02-23T00:52:26.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.212+0000 7f1b54ff9700 1 -- 172.21.15.47:0/3816125392 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1b5811be00 con 0x7f1b5811cc00 2024-02-23T00:52:26.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.212+0000 7f1b54ff9700 10 client.31482.objecter ms_dispatch 0x7f1b58001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:52:26.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.212+0000 7f1b54ff9700 10 client.31482.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:52:26.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.212+0000 7f1b54ff9700 20 client.31482.objecter put_session s=0x7f1b5811ca90 osd=4 4 2024-02-23T00:52:26.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.212+0000 7f1b54ff9700 15 client.31482.objecter _session_command_op_remove 4 10 2024-02-23T00:52:26.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.212+0000 7f1b5df74700 10 client.31482.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:26.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.212+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_assign -1 11 2024-02-23T00:52:26.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.212+0000 7f1b5df74700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f1b58123d50 0x7f1b58124190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:26.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.212+0000 7f1b5df74700 20 client.31482.objecter _get_session s=0x7f1b5811be00 osd=5 3 2024-02-23T00:52:26.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.212+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b5811be00 osd=5 3 2024-02-23T00:52:26.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.212+0000 7f1b5df74700 20 client.31482.objecter _get_session s=0x7f1b5811be00 osd=5 3 2024-02-23T00:52:26.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.212+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_remove -1 11 2024-02-23T00:52:26.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.212+0000 7f1b5df74700 20 client.31482.objecter get_session s=0x7f1b5811be00 osd=5 3 2024-02-23T00:52:26.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.212+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_assign 5 11 2024-02-23T00:52:26.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.212+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b5811be00 osd=5 4 2024-02-23T00:52:26.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.212+0000 7f1b5df74700 10 client.31482.objecter _send_command 11 2024-02-23T00:52:26.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.212+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b58124730 con 0x7f1b58123d50 2024-02-23T00:52:26.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.212+0000 7f1b56ffd700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f1b58123d50 0x7f1b58124190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:26.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.213+0000 7f1b54ff9700 10 client.31482.objecter ms_handle_connect 0x7f1b58123d50 2024-02-23T00:52:26.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.213+0000 7f1b56ffd700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f1b58123d50 0x7f1b58124190 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:26.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.213+0000 7f1b54ff9700 1 -- 172.21.15.47:0/3816125392 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1b58124730 con 0x7f1b58123d50 2024-02-23T00:52:26.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.213+0000 7f1b54ff9700 10 client.31482.objecter ms_dispatch 0x7f1b58001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:52:26.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.213+0000 7f1b54ff9700 10 client.31482.objecter _finish_command 11 = system:0 2024-02-23T00:52:26.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.213+0000 7f1b54ff9700 20 client.31482.objecter put_session s=0x7f1b5811be00 osd=5 4 2024-02-23T00:52:26.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.214+0000 7f1b54ff9700 15 client.31482.objecter _session_command_op_remove 5 11 2024-02-23T00:52:26.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.228+0000 7f1b5df74700 10 client.31482.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:52:26.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.228+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_assign -1 12 2024-02-23T00:52:26.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.228+0000 7f1b5df74700 20 client.31482.objecter _get_session s=0x7f1b5811be00 osd=5 3 2024-02-23T00:52:26.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.228+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b5811be00 osd=5 3 2024-02-23T00:52:26.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.228+0000 7f1b5df74700 20 client.31482.objecter _get_session s=0x7f1b5811be00 osd=5 3 2024-02-23T00:52:26.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.228+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_remove -1 12 2024-02-23T00:52:26.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.228+0000 7f1b5df74700 20 client.31482.objecter get_session s=0x7f1b5811be00 osd=5 3 2024-02-23T00:52:26.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.228+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_assign 5 12 2024-02-23T00:52:26.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.228+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b5811be00 osd=5 4 2024-02-23T00:52:26.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.228+0000 7f1b5df74700 10 client.31482.objecter _send_command 12 2024-02-23T00:52:26.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.228+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1b58124730 con 0x7f1b58123d50 2024-02-23T00:52:26.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.230+0000 7f1b54ff9700 1 -- 172.21.15.47:0/3816125392 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1b58124730 con 0x7f1b58123d50 2024-02-23T00:52:26.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.230+0000 7f1b54ff9700 10 client.31482.objecter ms_dispatch 0x7f1b58001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:52:26.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.230+0000 7f1b54ff9700 10 client.31482.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:52:26.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.230+0000 7f1b54ff9700 20 client.31482.objecter put_session s=0x7f1b5811be00 osd=5 4 2024-02-23T00:52:26.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.230+0000 7f1b54ff9700 15 client.31482.objecter _session_command_op_remove 5 12 2024-02-23T00:52:26.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.230+0000 7f1b5df74700 10 client.31482.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:26.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.230+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_assign -1 13 2024-02-23T00:52:26.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.230+0000 7f1b5df74700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f1b58128fd0 0x7f1b5812b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:26.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.230+0000 7f1b5df74700 20 client.31482.objecter _get_session s=0x7f1b58124730 osd=6 3 2024-02-23T00:52:26.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.230+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b58124730 osd=6 3 2024-02-23T00:52:26.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.230+0000 7f1b5df74700 20 client.31482.objecter _get_session s=0x7f1b58124730 osd=6 3 2024-02-23T00:52:26.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.230+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_remove -1 13 2024-02-23T00:52:26.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.230+0000 7f1b5df74700 20 client.31482.objecter get_session s=0x7f1b58124730 osd=6 3 2024-02-23T00:52:26.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.230+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_assign 6 13 2024-02-23T00:52:26.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.230+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b58124730 osd=6 4 2024-02-23T00:52:26.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.230+0000 7f1b5df74700 10 client.31482.objecter _send_command 13 2024-02-23T00:52:26.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.230+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b5812e280 con 0x7f1b58128fd0 2024-02-23T00:52:26.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.230+0000 7f1b57fff700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f1b58128fd0 0x7f1b5812b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:26.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.231+0000 7f1b54ff9700 10 client.31482.objecter ms_handle_connect 0x7f1b58128fd0 2024-02-23T00:52:26.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.231+0000 7f1b57fff700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f1b58128fd0 0x7f1b5812b440 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:26.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.231+0000 7f1b54ff9700 1 -- 172.21.15.47:0/3816125392 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1b5812e280 con 0x7f1b58128fd0 2024-02-23T00:52:26.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.231+0000 7f1b54ff9700 10 client.31482.objecter ms_dispatch 0x7f1b58001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:52:26.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.231+0000 7f1b54ff9700 10 client.31482.objecter _finish_command 13 = system:0 2024-02-23T00:52:26.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.231+0000 7f1b54ff9700 20 client.31482.objecter put_session s=0x7f1b58124730 osd=6 4 2024-02-23T00:52:26.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.231+0000 7f1b54ff9700 15 client.31482.objecter _session_command_op_remove 6 13 2024-02-23T00:52:26.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.246+0000 7f1b5df74700 10 client.31482.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:52:26.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.246+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_assign -1 14 2024-02-23T00:52:26.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.246+0000 7f1b5df74700 20 client.31482.objecter _get_session s=0x7f1b58124730 osd=6 3 2024-02-23T00:52:26.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.246+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b58124730 osd=6 3 2024-02-23T00:52:26.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.246+0000 7f1b5df74700 20 client.31482.objecter _get_session s=0x7f1b58124730 osd=6 3 2024-02-23T00:52:26.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.246+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_remove -1 14 2024-02-23T00:52:26.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.246+0000 7f1b5df74700 20 client.31482.objecter get_session s=0x7f1b58124730 osd=6 3 2024-02-23T00:52:26.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.246+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_assign 6 14 2024-02-23T00:52:26.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.246+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b58124730 osd=6 4 2024-02-23T00:52:26.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.246+0000 7f1b5df74700 10 client.31482.objecter _send_command 14 2024-02-23T00:52:26.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.246+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1b5812e280 con 0x7f1b58128fd0 2024-02-23T00:52:26.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.248+0000 7f1b54ff9700 1 -- 172.21.15.47:0/3816125392 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1b5812e280 con 0x7f1b58128fd0 2024-02-23T00:52:26.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.248+0000 7f1b54ff9700 10 client.31482.objecter ms_dispatch 0x7f1b58001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:52:26.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.248+0000 7f1b54ff9700 10 client.31482.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:52:26.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.248+0000 7f1b54ff9700 20 client.31482.objecter put_session s=0x7f1b58124730 osd=6 4 2024-02-23T00:52:26.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.248+0000 7f1b54ff9700 15 client.31482.objecter _session_command_op_remove 6 14 2024-02-23T00:52:26.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.248+0000 7f1b5df74700 10 client.31482.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:26.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.248+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_assign -1 15 2024-02-23T00:52:26.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.248+0000 7f1b5df74700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f1b5812d5b0 0x7f1b5812da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:26.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.248+0000 7f1b5df74700 20 client.31482.objecter _get_session s=0x7f1b5812e280 osd=7 3 2024-02-23T00:52:26.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.248+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b5812e280 osd=7 3 2024-02-23T00:52:26.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.248+0000 7f1b5df74700 20 client.31482.objecter _get_session s=0x7f1b5812e280 osd=7 3 2024-02-23T00:52:26.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.248+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_remove -1 15 2024-02-23T00:52:26.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.248+0000 7f1b5df74700 20 client.31482.objecter get_session s=0x7f1b5812e280 osd=7 3 2024-02-23T00:52:26.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.248+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_assign 7 15 2024-02-23T00:52:26.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.248+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b5812e280 osd=7 4 2024-02-23T00:52:26.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.248+0000 7f1b5df74700 10 client.31482.objecter _send_command 15 2024-02-23T00:52:26.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.248+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b581315b0 con 0x7f1b5812d5b0 2024-02-23T00:52:26.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.248+0000 7f1b577fe700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f1b5812d5b0 0x7f1b5812da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:26.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.249+0000 7f1b577fe700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f1b5812d5b0 0x7f1b5812da30 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:26.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.249+0000 7f1b54ff9700 10 client.31482.objecter ms_handle_connect 0x7f1b5812d5b0 2024-02-23T00:52:26.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.249+0000 7f1b54ff9700 1 -- 172.21.15.47:0/3816125392 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1b581315b0 con 0x7f1b5812d5b0 2024-02-23T00:52:26.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.249+0000 7f1b54ff9700 10 client.31482.objecter ms_dispatch 0x7f1b58001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:52:26.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.249+0000 7f1b54ff9700 10 client.31482.objecter _finish_command 15 = system:0 2024-02-23T00:52:26.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.249+0000 7f1b54ff9700 20 client.31482.objecter put_session s=0x7f1b5812e280 osd=7 4 2024-02-23T00:52:26.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.249+0000 7f1b54ff9700 15 client.31482.objecter _session_command_op_remove 7 15 2024-02-23T00:52:26.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.273+0000 7f1b5df74700 10 client.31482.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:52:26.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.273+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_assign -1 16 2024-02-23T00:52:26.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.273+0000 7f1b5df74700 20 client.31482.objecter _get_session s=0x7f1b5812e280 osd=7 3 2024-02-23T00:52:26.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.273+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b5812e280 osd=7 3 2024-02-23T00:52:26.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.273+0000 7f1b5df74700 20 client.31482.objecter _get_session s=0x7f1b5812e280 osd=7 3 2024-02-23T00:52:26.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.273+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_remove -1 16 2024-02-23T00:52:26.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.273+0000 7f1b5df74700 20 client.31482.objecter get_session s=0x7f1b5812e280 osd=7 3 2024-02-23T00:52:26.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.273+0000 7f1b5df74700 15 client.31482.objecter _session_command_op_assign 7 16 2024-02-23T00:52:26.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.273+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b5812e280 osd=7 4 2024-02-23T00:52:26.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.273+0000 7f1b5df74700 10 client.31482.objecter _send_command 16 2024-02-23T00:52:26.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.273+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1b5812ba40 con 0x7f1b5812d5b0 2024-02-23T00:52:26.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b54ff9700 1 -- 172.21.15.47:0/3816125392 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1b5812ba40 con 0x7f1b5812d5b0 2024-02-23T00:52:26.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b54ff9700 10 client.31482.objecter ms_dispatch 0x7f1b58001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:52:26.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b54ff9700 10 client.31482.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:52:26.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b54ff9700 20 client.31482.objecter put_session s=0x7f1b5812e280 osd=7 4 2024-02-23T00:52:26.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b54ff9700 15 client.31482.objecter _session_command_op_remove 7 16 2024-02-23T00:52:26.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b5df74700 10 librados: watch_flush enter 2024-02-23T00:52:26.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b5df74700 10 librados: watch_flush exit 2024-02-23T00:52:26.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b5df74700 10 client.31482.objecter close_session for osd.0 2024-02-23T00:52:26.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f1b580fc1a0 msgr2=0x7f1b580fc580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:26.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b5df74700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f1b580fc1a0 0x7f1b580fc580 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:26.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b5805ee80 osd=0 1 2024-02-23T00:52:26.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b5df74700 10 client.31482.objecter close_session for osd.1 2024-02-23T00:52:26.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f1b580598f0 msgr2=0x7f1b58059cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:26.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b5df74700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f1b580598f0 0x7f1b58059cf0 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:26.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b58028710 osd=1 1 2024-02-23T00:52:26.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b5df74700 10 client.31482.objecter close_session for osd.2 2024-02-23T00:52:26.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f1b58028270 msgr2=0x7f1b5811c190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:26.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b5df74700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f1b58028270 0x7f1b5811c190 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:26.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b58059770 osd=2 1 2024-02-23T00:52:26.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b5df74700 10 client.31482.objecter close_session for osd.3 2024-02-23T00:52:26.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f1b5811f8f0 msgr2=0x7f1b58121ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:26.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b5df74700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f1b5811f8f0 0x7f1b58121ce0 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:26.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b58027be0 osd=3 1 2024-02-23T00:52:26.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b5df74700 10 client.31482.objecter close_session for osd.4 2024-02-23T00:52:26.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f1b5811cc00 msgr2=0x7f1b58123350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:26.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b5df74700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f1b5811cc00 0x7f1b58123350 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:26.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b5811ca90 osd=4 1 2024-02-23T00:52:26.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b57fff700 1 -- 172.21.15.47:0/3816125392 reap_dead start 2024-02-23T00:52:26.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b5df74700 10 client.31482.objecter close_session for osd.5 2024-02-23T00:52:26.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f1b58123d50 msgr2=0x7f1b58124190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:26.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b5df74700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f1b58123d50 0x7f1b58124190 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:26.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b57fff700 1 -- 172.21.15.47:0/3816125392 reap_dead start 2024-02-23T00:52:26.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b57fff700 1 -- 172.21.15.47:0/3816125392 reap_dead start 2024-02-23T00:52:26.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b5811be00 osd=5 1 2024-02-23T00:52:26.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b5df74700 10 client.31482.objecter close_session for osd.6 2024-02-23T00:52:26.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.276+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f1b58128fd0 msgr2=0x7f1b5812b440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:26.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.277+0000 7f1b5df74700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f1b58128fd0 0x7f1b5812b440 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:26.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.277+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b58124730 osd=6 1 2024-02-23T00:52:26.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.277+0000 7f1b5df74700 10 client.31482.objecter close_session for osd.7 2024-02-23T00:52:26.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.277+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f1b5812d5b0 msgr2=0x7f1b5812da30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:26.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.277+0000 7f1b5df74700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f1b5812d5b0 0x7f1b5812da30 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:26.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.277+0000 7f1b5df74700 20 client.31482.objecter put_session s=0x7f1b5812e280 osd=7 1 2024-02-23T00:52:26.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.277+0000 7f1b5df74700 20 client.31482.objecter shutdown clearing up homeless session... 2024-02-23T00:52:26.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.277+0000 7f1b5df74700 10 client.31482.objecter successfully canceled tick 2024-02-23T00:52:26.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.277+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1b44062cc0 msgr2=0x7f1b44065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:26.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.277+0000 7f1b5df74700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1b44062cc0 0x7f1b44065180 secure :-1 s=READY pgs=3143 cs=0 l=1 rev1=1 rx=0x7f1b480103c0 tx=0x7f1b4800b040).stop 2024-02-23T00:52:26.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.277+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b580f9610 msgr2=0x7f1b580f7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:26.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.277+0000 7f1b5df74700 1 --2- 172.21.15.47:0/3816125392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b580f9610 0x7f1b580f7150 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 rx=0x7f1b400048f0 tx=0x7f1b40009cb0).stop 2024-02-23T00:52:26.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.277+0000 7f1b57fff700 1 -- 172.21.15.47:0/3816125392 reap_dead start 2024-02-23T00:52:26.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.277+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 shutdown_connections 2024-02-23T00:52:26.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.277+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 >> 172.21.15.47:0/3816125392 conn(0x7f1b580f2820 msgr2=0x7f1b580f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:52:26.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.277+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 shutdown_connections 2024-02-23T00:52:26.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.277+0000 7f1b5df74700 1 -- 172.21.15.47:0/3816125392 wait complete. 2024-02-23T00:52:26.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:26.277+0000 7f1b5df74700 1 librados: shutdown 2024-02-23T00:52:26.568 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:52:26.568 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:52:26.568 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:52:26.568 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:52:26.569 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:52:26.569 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:52:26.569 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:52:26.569 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:52:26.582 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:52:26.611 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:26.699 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:52:27.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.237+0000 7ff5c5df8700 1 -- 172.21.15.47:0/961917077 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff5c00f8d10 msgr2=0x7ff5c00f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:27.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.237+0000 7ff5c5df8700 1 --2- 172.21.15.47:0/961917077 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff5c00f8d10 0x7ff5c00f90f0 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 rx=0x7ff5b0004870 tx=0x7ff5b0009cb0).stop 2024-02-23T00:52:27.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.237+0000 7ff5c5df8700 1 -- 172.21.15.47:0/961917077 shutdown_connections 2024-02-23T00:52:27.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.237+0000 7ff5c5df8700 1 --2- 172.21.15.47:0/961917077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5c0102270 0x7ff5c0104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:27.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.237+0000 7ff5c5df8700 1 --2- 172.21.15.47:0/961917077 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff5c00f8d10 0x7ff5c00f90f0 unknown :-1 s=CLOSED pgs=2755 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:27.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.237+0000 7ff5c5df8700 1 --2- 172.21.15.47:0/961917077 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5c00f9670 0x7ff5c0101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:27.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.237+0000 7ff5c5df8700 1 -- 172.21.15.47:0/961917077 >> 172.21.15.47:0/961917077 conn(0x7ff5c00f2820 msgr2=0x7ff5c00f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:52:27.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.237+0000 7ff5c5df8700 1 -- 172.21.15.47:0/961917077 shutdown_connections 2024-02-23T00:52:27.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.237+0000 7ff5c5df8700 1 -- 172.21.15.47:0/961917077 wait complete. 2024-02-23T00:52:27.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.238+0000 7ff5c5df8700 1 librados: starting msgr at 2024-02-23T00:52:27.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.238+0000 7ff5c5df8700 1 librados: starting objecter 2024-02-23T00:52:27.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.238+0000 7ff5c5df8700 1 Processor -- start 2024-02-23T00:52:27.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.238+0000 7ff5c5df8700 1 -- start start 2024-02-23T00:52:27.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.238+0000 7ff5c5df8700 1 librados: setting wanted keys 2024-02-23T00:52:27.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.238+0000 7ff5c5df8700 1 librados: calling monclient init 2024-02-23T00:52:27.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.238+0000 7ff5c5df8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5c00f8d10 0x7ff5c01102a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:27.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.238+0000 7ff5c5df8700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff5c00f9670 0x7ff5c0110790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:27.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.238+0000 7ff5c5df8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5c0102270 0x7ff5c010a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:27.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.238+0000 7ff5c5df8700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c010a970 con 0x7ff5c00f8d10 2024-02-23T00:52:27.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.238+0000 7ff5c5df8700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c010aae0 con 0x7ff5c0102270 2024-02-23T00:52:27.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.238+0000 7ff5c5df8700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c010ac50 con 0x7ff5c00f9670 2024-02-23T00:52:27.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.238+0000 7ff5bf7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5c00f8d10 0x7ff5c01102a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:27.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.238+0000 7ff5b6ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff5c00f9670 0x7ff5c0110790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:27.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.238+0000 7ff5bf7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5c00f8d10 0x7ff5c01102a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36738/0 (socket says 172.21.15.47:36738) 2024-02-23T00:52:27.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.238+0000 7ff5bf7fe700 1 -- 172.21.15.47:0/1245177694 learned_addr learned my addr 172.21.15.47:0/1245177694 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:52:27.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.239+0000 7ff5bffff700 1 --2- 172.21.15.47:0/1245177694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5c0102270 0x7ff5c010a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:27.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.239+0000 7ff5b6ffd700 1 -- 172.21.15.47:0/1245177694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5c0102270 msgr2=0x7ff5c010a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:27.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.239+0000 7ff5b6ffd700 1 --2- 172.21.15.47:0/1245177694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5c0102270 0x7ff5c010a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:27.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.239+0000 7ff5b6ffd700 1 -- 172.21.15.47:0/1245177694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5c00f8d10 msgr2=0x7ff5c01102a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:27.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.239+0000 7ff5b6ffd700 1 --2- 172.21.15.47:0/1245177694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5c00f8d10 0x7ff5c01102a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:27.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.239+0000 7ff5b6ffd700 1 -- 172.21.15.47:0/1245177694 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a8005040 con 0x7ff5c00f9670 2024-02-23T00:52:27.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.239+0000 7ff5bf7fe700 1 --2- 172.21.15.47:0/1245177694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5c00f8d10 0x7ff5c01102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:52:27.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.239+0000 7ff5b6ffd700 1 --2- 172.21.15.47:0/1245177694 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff5c00f9670 0x7ff5c0110790 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 rx=0x7ff5a8004920 tx=0x7ff5a8009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:27.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.239+0000 7ff5bd7fa700 10 client.?.objecter ms_handle_connect 0x7ff5c00f9670 2024-02-23T00:52:27.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.239+0000 7ff5bd7fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:52:27.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.239+0000 7ff5bd7fa700 1 -- 172.21.15.47:0/1245177694 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8007a40 con 0x7ff5c00f9670 2024-02-23T00:52:27.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.239+0000 7ff5bd7fa700 1 -- 172.21.15.47:0/1245177694 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7ff5a800f040 con 0x7ff5c00f9670 2024-02-23T00:52:27.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.239+0000 7ff5c5df8700 1 -- 172.21.15.47:0/1245177694 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5b0005040 con 0x7ff5c00f9670 2024-02-23T00:52:27.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.239+0000 7ff5bd7fa700 1 -- 172.21.15.47:0/1245177694 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8014900 con 0x7ff5c00f9670 2024-02-23T00:52:27.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.239+0000 7ff5c5df8700 10 client.41807.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:52:27.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.239+0000 7ff5c5df8700 1 -- 172.21.15.47:0/1245177694 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c010b290 con 0x7ff5c00f9670 2024-02-23T00:52:27.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.239+0000 7ff5c5df8700 1 librados: init done 2024-02-23T00:52:27.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.240+0000 7ff5c5df8700 1 -- 172.21.15.47:0/1245177694 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c005fa60 con 0x7ff5c00f9670 2024-02-23T00:52:27.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.240+0000 7ff5bd7fa700 1 -- 172.21.15.47:0/1245177694 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7ff5a8014d30 con 0x7ff5c00f9670 2024-02-23T00:52:27.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.241+0000 7ff5bd7fa700 1 --2- 172.21.15.47:0/1245177694 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff5ac062cc0 0x7ff5ac065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:27.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.241+0000 7ff5bd7fa700 1 -- 172.21.15.47:0/1245177694 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(617..617 src has 482..617) v4 ==== 9583+0+0 (secure 0 0 0) 0x7ff5a807d340 con 0x7ff5c00f9670 2024-02-23T00:52:27.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.241+0000 7ff5bd7fa700 10 client.41807.objecter ms_dispatch 0x7ff5c0001550 osd_map(617..617 src has 482..617) v4 2024-02-23T00:52:27.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.241+0000 7ff5bd7fa700 3 client.41807.objecter handle_osd_map got epochs [617,617] > 0 2024-02-23T00:52:27.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.241+0000 7ff5bd7fa700 3 client.41807.objecter handle_osd_map decoding full epoch 617 2024-02-23T00:52:27.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.241+0000 7ff5bf7fe700 1 --2- 172.21.15.47:0/1245177694 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff5ac062cc0 0x7ff5ac065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:27.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.241+0000 7ff5bd7fa700 20 client.41807.objecter dump_active .. 0 homeless 2024-02-23T00:52:27.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.241+0000 7ff5bd7fa700 10 client.41807.objecter ms_handle_connect 0x7ff5ac062cc0 2024-02-23T00:52:27.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.241+0000 7ff5bf7fe700 1 --2- 172.21.15.47:0/1245177694 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff5ac062cc0 0x7ff5ac065180 secure :-1 s=READY pgs=3144 cs=0 l=1 rev1=1 rx=0x7ff5b0007fd0 tx=0x7ff5b000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:27.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.245+0000 7ff5bd7fa700 1 -- 172.21.15.47:0/1245177694 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ff5a8049f60 con 0x7ff5c00f9670 2024-02-23T00:52:27.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:27.440+0000 7ff5c5df8700 1 -- 172.21.15.47:0/1245177694 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7ff5c005ee80 con 0x7ff5c00f9670 2024-02-23T00:52:27.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:25.396694+0000 osd.6 (osd.6) 233 : cluster [DBG] 2.7 scrub starts 2024-02-23T00:52:27.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:25.397562+0000 osd.6 (osd.6) 234 : cluster [DBG] 2.7 scrub ok 2024-02-23T00:52:27.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:26.307332+0000 mon.a (mon.0) 3171 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:52:27.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:26.395938+0000 osd.7 (osd.7) 299 : cluster [DBG] 1.0 scrub starts 2024-02-23T00:52:27.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:26.398829+0000 osd.7 (osd.7) 300 : cluster [DBG] 1.0 scrub ok 2024-02-23T00:52:27.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:26.752633+0000 osd.3 (osd.3) 294 : cluster [DBG] 2.19 scrub starts 2024-02-23T00:52:27.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:26.753435+0000 osd.3 (osd.3) 295 : cluster [DBG] 2.19 scrub ok 2024-02-23T00:52:27.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:26.762560+0000 osd.4 (osd.4) 263 : cluster [DBG] 4.1 scrub starts 2024-02-23T00:52:27.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:26.764026+0000 osd.4 (osd.4) 264 : cluster [DBG] 4.1 scrub ok 2024-02-23T00:52:27.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:27.263183+0000 mon.a (mon.0) 3172 : cluster [DBG] osdmap e618: 8 total, 8 up, 7 in 2024-02-23T00:52:27.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:27.276989+0000 mon.c (mon.2) 350 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-106", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:52:27.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:27.277497+0000 mon.a (mon.0) 3173 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-106", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:52:27.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:25.396694+0000 osd.6 (osd.6) 233 : cluster [DBG] 2.7 scrub starts 2024-02-23T00:52:27.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:25.397562+0000 osd.6 (osd.6) 234 : cluster [DBG] 2.7 scrub ok 2024-02-23T00:52:27.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:26.307332+0000 mon.a (mon.0) 3171 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:52:27.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:26.395938+0000 osd.7 (osd.7) 299 : cluster [DBG] 1.0 scrub starts 2024-02-23T00:52:27.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:26.398829+0000 osd.7 (osd.7) 300 : cluster [DBG] 1.0 scrub ok 2024-02-23T00:52:27.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:26.752633+0000 osd.3 (osd.3) 294 : cluster [DBG] 2.19 scrub starts 2024-02-23T00:52:27.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:26.753435+0000 osd.3 (osd.3) 295 : cluster [DBG] 2.19 scrub ok 2024-02-23T00:52:27.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:26.762560+0000 osd.4 (osd.4) 263 : cluster [DBG] 4.1 scrub starts 2024-02-23T00:52:27.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:26.764026+0000 osd.4 (osd.4) 264 : cluster [DBG] 4.1 scrub ok 2024-02-23T00:52:27.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:27.263183+0000 mon.a (mon.0) 3172 : cluster [DBG] osdmap e618: 8 total, 8 up, 7 in 2024-02-23T00:52:27.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:27.276989+0000 mon.c (mon.2) 350 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-106", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:52:27.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:27.277497+0000 mon.a (mon.0) 3173 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-106", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:52:27.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:25.396694+0000 osd.6 (osd.6) 233 : cluster [DBG] 2.7 scrub starts 2024-02-23T00:52:27.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:25.397562+0000 osd.6 (osd.6) 234 : cluster [DBG] 2.7 scrub ok 2024-02-23T00:52:27.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:26.307332+0000 mon.a (mon.0) 3171 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:52:27.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:26.395938+0000 osd.7 (osd.7) 299 : cluster [DBG] 1.0 scrub starts 2024-02-23T00:52:27.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:26.398829+0000 osd.7 (osd.7) 300 : cluster [DBG] 1.0 scrub ok 2024-02-23T00:52:27.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:26.752633+0000 osd.3 (osd.3) 294 : cluster [DBG] 2.19 scrub starts 2024-02-23T00:52:27.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:26.753435+0000 osd.3 (osd.3) 295 : cluster [DBG] 2.19 scrub ok 2024-02-23T00:52:27.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:26.762560+0000 osd.4 (osd.4) 263 : cluster [DBG] 4.1 scrub starts 2024-02-23T00:52:27.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:26.764026+0000 osd.4 (osd.4) 264 : cluster [DBG] 4.1 scrub ok 2024-02-23T00:52:27.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:27.263183+0000 mon.a (mon.0) 3172 : cluster [DBG] osdmap e618: 8 total, 8 up, 7 in 2024-02-23T00:52:27.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:27.276989+0000 mon.c (mon.2) 350 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-106", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:52:27.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:27.277497+0000 mon.a (mon.0) 3173 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-106", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:52:27.863 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:52:27.866 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:27.866 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:52:27.967 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:52:27.997 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:28.035 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:52:28.038 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:28.038 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:52:28.139 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:52:28.171 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:28.192 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:52:28.195 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:28.195 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:52:28.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:28.265+0000 7ff5bd7fa700 1 -- 172.21.15.47:0/1245177694 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v619) v1 ==== 87+0+0 (secure 0 0 0) 0x7ff5a804c790 con 0x7ff5c00f9670 2024-02-23T00:52:28.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:28.267+0000 7ff5c5df8700 10 librados: watch_flush enter 2024-02-23T00:52:28.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:28.267+0000 7ff5c5df8700 10 librados: watch_flush exit 2024-02-23T00:52:28.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:28.267+0000 7ff5c5df8700 20 client.41807.objecter shutdown clearing up homeless session... 2024-02-23T00:52:28.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:28.267+0000 7ff5c5df8700 10 client.41807.objecter successfully canceled tick 2024-02-23T00:52:28.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:28.267+0000 7ff5c5df8700 1 -- 172.21.15.47:0/1245177694 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff5ac062cc0 msgr2=0x7ff5ac065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:28.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:28.267+0000 7ff5c5df8700 1 --2- 172.21.15.47:0/1245177694 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff5ac062cc0 0x7ff5ac065180 secure :-1 s=READY pgs=3144 cs=0 l=1 rev1=1 rx=0x7ff5b0007fd0 tx=0x7ff5b000b040).stop 2024-02-23T00:52:28.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:28.267+0000 7ff5c5df8700 1 -- 172.21.15.47:0/1245177694 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff5c00f9670 msgr2=0x7ff5c0110790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:28.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:28.267+0000 7ff5c5df8700 1 --2- 172.21.15.47:0/1245177694 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff5c00f9670 0x7ff5c0110790 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 rx=0x7ff5a8004920 tx=0x7ff5a8009cb0).stop 2024-02-23T00:52:28.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:28.267+0000 7ff5c5df8700 1 -- 172.21.15.47:0/1245177694 shutdown_connections 2024-02-23T00:52:28.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:28.267+0000 7ff5c5df8700 1 --2- 172.21.15.47:0/1245177694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5c0102270 0x7ff5c010a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:28.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:28.267+0000 7ff5c5df8700 1 --2- 172.21.15.47:0/1245177694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5c00f8d10 0x7ff5c01102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:28.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:28.267+0000 7ff5c5df8700 1 --2- 172.21.15.47:0/1245177694 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff5ac062cc0 0x7ff5ac065180 unknown :-1 s=CLOSED pgs=3144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:28.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:28.267+0000 7ff5c5df8700 1 --2- 172.21.15.47:0/1245177694 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff5c00f9670 0x7ff5c0110790 unknown :-1 s=CLOSED pgs=2756 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:28.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:28.267+0000 7ff5c5df8700 1 -- 172.21.15.47:0/1245177694 >> 172.21.15.47:0/1245177694 conn(0x7ff5c00f2820 msgr2=0x7ff5c00f4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:52:28.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:28.267+0000 7ff5c5df8700 1 -- 172.21.15.47:0/1245177694 shutdown_connections 2024-02-23T00:52:28.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:28.267+0000 7ff5c5df8700 1 -- 172.21.15.47:0/1245177694 wait complete. 2024-02-23T00:52:28.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:28.267+0000 7ff5c5df8700 1 librados: shutdown 2024-02-23T00:52:28.271 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is set 2024-02-23T00:52:28.296 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:52:28.319 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:52:28.326 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:28.348 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:52:28.350 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:28.351 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:52:28.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:26.371991+0000 osd.6 (osd.6) 235 : cluster [DBG] 4.5 scrub starts 2024-02-23T00:52:28.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:26.373814+0000 osd.6 (osd.6) 236 : cluster [DBG] 4.5 scrub ok 2024-02-23T00:52:28.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:26.700150+0000 osd.0 (osd.0) 259 : cluster [DBG] 6.0 scrub starts 2024-02-23T00:52:28.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:26.704491+0000 osd.0 (osd.0) 260 : cluster [DBG] 6.0 scrub ok 2024-02-23T00:52:28.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:27.356841+0000 mgr.y (mgr.14517) 548 : cluster [DBG] pgmap v931: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1023 B/s rd, 1023 B/s wr, 1 op/s 2024-02-23T00:52:28.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:27.442470+0000 mon.c (mon.2) 351 : audit [INF] from='client.? 172.21.15.47:0/1245177694' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:52:28.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:27.442796+0000 mon.a (mon.0) 3174 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:52:28.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:27.453314+0000 osd.5 (osd.5) 270 : cluster [DBG] 4.15 scrub starts 2024-02-23T00:52:28.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:27.456783+0000 osd.5 (osd.5) 271 : cluster [DBG] 4.15 scrub ok 2024-02-23T00:52:28.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:27.761011+0000 osd.3 (osd.3) 296 : cluster [DBG] 5.13 scrub starts 2024-02-23T00:52:28.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:27.761916+0000 osd.3 (osd.3) 297 : cluster [DBG] 5.13 scrub ok 2024-02-23T00:52:28.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:28.260403+0000 mon.a (mon.0) 3175 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:52:28.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:28.264839+0000 mon.a (mon.0) 3176 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-106", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:52:28.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:28.264924+0000 mon.a (mon.0) 3177 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:52:28.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:28.264982+0000 mon.a (mon.0) 3178 : cluster [DBG] osdmap e619: 8 total, 8 up, 7 in 2024-02-23T00:52:28.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:28.266640+0000 mon.c (mon.2) 352 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-106"}]: dispatch 2024-02-23T00:52:28.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:28.266984+0000 mon.a (mon.0) 3179 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-106"}]: dispatch 2024-02-23T00:52:28.451 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:52:28.479 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:28.498 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:52:28.501 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:28.502 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:52:28.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:26.371991+0000 osd.6 (osd.6) 235 : cluster [DBG] 4.5 scrub starts 2024-02-23T00:52:28.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:26.373814+0000 osd.6 (osd.6) 236 : cluster [DBG] 4.5 scrub ok 2024-02-23T00:52:28.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:26.700150+0000 osd.0 (osd.0) 259 : cluster [DBG] 6.0 scrub starts 2024-02-23T00:52:28.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:26.704491+0000 osd.0 (osd.0) 260 : cluster [DBG] 6.0 scrub ok 2024-02-23T00:52:28.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:27.356841+0000 mgr.y (mgr.14517) 548 : cluster [DBG] pgmap v931: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1023 B/s rd, 1023 B/s wr, 1 op/s 2024-02-23T00:52:28.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:27.442470+0000 mon.c (mon.2) 351 : audit [INF] from='client.? 172.21.15.47:0/1245177694' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:52:28.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:27.442796+0000 mon.a (mon.0) 3174 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:52:28.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:27.453314+0000 osd.5 (osd.5) 270 : cluster [DBG] 4.15 scrub starts 2024-02-23T00:52:28.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:27.456783+0000 osd.5 (osd.5) 271 : cluster [DBG] 4.15 scrub ok 2024-02-23T00:52:28.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:27.761011+0000 osd.3 (osd.3) 296 : cluster [DBG] 5.13 scrub starts 2024-02-23T00:52:28.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:27.761916+0000 osd.3 (osd.3) 297 : cluster [DBG] 5.13 scrub ok 2024-02-23T00:52:28.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:28.260403+0000 mon.a (mon.0) 3175 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:52:28.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:28.264839+0000 mon.a (mon.0) 3176 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-106", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:52:28.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:28.264924+0000 mon.a (mon.0) 3177 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:52:28.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:28.264982+0000 mon.a (mon.0) 3178 : cluster [DBG] osdmap e619: 8 total, 8 up, 7 in 2024-02-23T00:52:28.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:28.266640+0000 mon.c (mon.2) 352 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-106"}]: dispatch 2024-02-23T00:52:28.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:28.266984+0000 mon.a (mon.0) 3179 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-106"}]: dispatch 2024-02-23T00:52:28.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:26.371991+0000 osd.6 (osd.6) 235 : cluster [DBG] 4.5 scrub starts 2024-02-23T00:52:28.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:26.373814+0000 osd.6 (osd.6) 236 : cluster [DBG] 4.5 scrub ok 2024-02-23T00:52:28.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:26.700150+0000 osd.0 (osd.0) 259 : cluster [DBG] 6.0 scrub starts 2024-02-23T00:52:28.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:26.704491+0000 osd.0 (osd.0) 260 : cluster [DBG] 6.0 scrub ok 2024-02-23T00:52:28.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:27.356841+0000 mgr.y (mgr.14517) 548 : cluster [DBG] pgmap v931: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1023 B/s rd, 1023 B/s wr, 1 op/s 2024-02-23T00:52:28.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:27.442470+0000 mon.c (mon.2) 351 : audit [INF] from='client.? 172.21.15.47:0/1245177694' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:52:28.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:27.442796+0000 mon.a (mon.0) 3174 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:52:28.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:27.453314+0000 osd.5 (osd.5) 270 : cluster [DBG] 4.15 scrub starts 2024-02-23T00:52:28.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:27.456783+0000 osd.5 (osd.5) 271 : cluster [DBG] 4.15 scrub ok 2024-02-23T00:52:28.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:27.761011+0000 osd.3 (osd.3) 296 : cluster [DBG] 5.13 scrub starts 2024-02-23T00:52:28.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:27.761916+0000 osd.3 (osd.3) 297 : cluster [DBG] 5.13 scrub ok 2024-02-23T00:52:28.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:28.260403+0000 mon.a (mon.0) 3175 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:52:28.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:28.264839+0000 mon.a (mon.0) 3176 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-106", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:52:28.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:28.264924+0000 mon.a (mon.0) 3177 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:52:28.560 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:28.264982+0000 mon.a (mon.0) 3178 : cluster [DBG] osdmap e619: 8 total, 8 up, 7 in 2024-02-23T00:52:28.560 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:28.266640+0000 mon.c (mon.2) 352 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-106"}]: dispatch 2024-02-23T00:52:28.560 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:28.266984+0000 mon.a (mon.0) 3179 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-106"}]: dispatch 2024-02-23T00:52:28.602 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:52:28.630 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:28.648 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:52:28.651 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:28.651 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:52:28.752 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:52:28.779 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:28.797 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:52:28.799 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:28.799 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:52:28.900 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:52:28.930 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:29.280 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:52:29.282 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:29.283 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:52:29.384 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:52:29.411 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:29.430 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:52:29.432 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:29.432 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:52:29.533 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:52:29.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:27.355338+0000 osd.6 (osd.6) 237 : cluster [DBG] 4.1f deep-scrub starts 2024-02-23T00:52:29.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:27.357364+0000 osd.6 (osd.6) 238 : cluster [DBG] 4.1f deep-scrub ok 2024-02-23T00:52:29.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:29.266390+0000 mon.a (mon.0) 3180 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-106"}]': finished 2024-02-23T00:52:29.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:29.266472+0000 mon.a (mon.0) 3181 : cluster [DBG] osdmap e620: 8 total, 8 up, 7 in 2024-02-23T00:52:29.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:29.267817+0000 mon.c (mon.2) 353 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-106", "mode": "writeback"}]: dispatch 2024-02-23T00:52:29.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:29.268134+0000 mon.a (mon.0) 3182 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-106", "mode": "writeback"}]: dispatch 2024-02-23T00:52:29.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:27.355338+0000 osd.6 (osd.6) 237 : cluster [DBG] 4.1f deep-scrub starts 2024-02-23T00:52:29.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:27.357364+0000 osd.6 (osd.6) 238 : cluster [DBG] 4.1f deep-scrub ok 2024-02-23T00:52:29.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:29.266390+0000 mon.a (mon.0) 3180 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-106"}]': finished 2024-02-23T00:52:29.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:29.266472+0000 mon.a (mon.0) 3181 : cluster [DBG] osdmap e620: 8 total, 8 up, 7 in 2024-02-23T00:52:29.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:29.267817+0000 mon.c (mon.2) 353 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-106", "mode": "writeback"}]: dispatch 2024-02-23T00:52:29.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:29.268134+0000 mon.a (mon.0) 3182 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-106", "mode": "writeback"}]: dispatch 2024-02-23T00:52:29.560 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:29.580 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:52:29.582 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:27.355338+0000 osd.6 (osd.6) 237 : cluster [DBG] 4.1f deep-scrub starts 2024-02-23T00:52:29.582 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:27.357364+0000 osd.6 (osd.6) 238 : cluster [DBG] 4.1f deep-scrub ok 2024-02-23T00:52:29.582 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:29.266390+0000 mon.a (mon.0) 3180 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-106"}]': finished 2024-02-23T00:52:29.582 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:29.266472+0000 mon.a (mon.0) 3181 : cluster [DBG] osdmap e620: 8 total, 8 up, 7 in 2024-02-23T00:52:29.582 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:29.267817+0000 mon.c (mon.2) 353 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-106", "mode": "writeback"}]: dispatch 2024-02-23T00:52:29.582 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:29.268134+0000 mon.a (mon.0) 3182 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-106", "mode": "writeback"}]: dispatch 2024-02-23T00:52:29.583 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:29.584 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:52:29.684 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:52:29.710 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:29.728 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:52:29.731 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:29.732 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:52:29.833 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:52:29.855 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:52:29.867 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:29.994 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:52:29.997 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:29.997 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:52:30.098 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:52:30.125 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:30.223 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:52:30.228 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:30.228 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:52:30.329 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:52:30.358 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:30.377 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:52:30.379 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:30.379 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:52:30.480 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:52:30.497 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:52:30.497 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:52:30.508 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:30.519 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:52:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:52:30] "GET /metrics HTTP/1.1" 200 190784 "" "Prometheus/2.33.4" 2024-02-23T00:52:30.519 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:29.357464+0000 mgr.y (mgr.14517) 549 : cluster [DBG] pgmap v934: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-23T00:52:30.520 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:30.263658+0000 mon.a (mon.0) 3183 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:52:30.520 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:30.268054+0000 mon.a (mon.0) 3184 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-106", "mode": "writeback"}]': finished 2024-02-23T00:52:30.520 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:30.268103+0000 mon.a (mon.0) 3185 : cluster [DBG] osdmap e621: 8 total, 8 up, 7 in 2024-02-23T00:52:30.520 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:30.286678+0000 mon.c (mon.2) 354 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "237.2"}]: dispatch 2024-02-23T00:52:30.520 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:29.357464+0000 mgr.y (mgr.14517) 549 : cluster [DBG] pgmap v934: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-23T00:52:30.520 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:30.263658+0000 mon.a (mon.0) 3183 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:52:30.520 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:30.268054+0000 mon.a (mon.0) 3184 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-106", "mode": "writeback"}]': finished 2024-02-23T00:52:30.520 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:30.268103+0000 mon.a (mon.0) 3185 : cluster [DBG] osdmap e621: 8 total, 8 up, 7 in 2024-02-23T00:52:30.520 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:30.286678+0000 mon.c (mon.2) 354 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "237.2"}]: dispatch 2024-02-23T00:52:30.527 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:52:30.529 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:30.530 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:52:30.630 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:52:30.655 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:29.357464+0000 mgr.y (mgr.14517) 549 : cluster [DBG] pgmap v934: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-23T00:52:30.655 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:30.263658+0000 mon.a (mon.0) 3183 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:52:30.655 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:30.268054+0000 mon.a (mon.0) 3184 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-106", "mode": "writeback"}]': finished 2024-02-23T00:52:30.655 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:30.268103+0000 mon.a (mon.0) 3185 : cluster [DBG] osdmap e621: 8 total, 8 up, 7 in 2024-02-23T00:52:30.655 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:30.286678+0000 mon.c (mon.2) 354 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "237.2"}]: dispatch 2024-02-23T00:52:30.657 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:30.675 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:52:30.678 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:30.678 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:52:30.779 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:52:30.809 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:30.828 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:52:30.831 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:30.832 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:52:30.933 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:52:30.960 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:30.979 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:52:30.981 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:30.982 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:52:31.082 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:52:31.111 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:31.133 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:52:31.136 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:31.136 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:52:31.237 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:52:31.266 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:31.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:30.286845+0000 mgr.y (mgr.14517) 550 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "237.2"}]: dispatch 2024-02-23T00:52:31.365 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:30.286845+0000 mgr.y (mgr.14517) 550 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "237.2"}]: dispatch 2024-02-23T00:52:31.365 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:52:31.368 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:31.368 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:52:31.469 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:52:31.501 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:31.523 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:52:31.525 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:31.526 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:52:31.626 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:52:31.657 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:31.674 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-23T00:52:31.686 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:52:31.689 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:31.690 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:52:31.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:30.286845+0000 mgr.y (mgr.14517) 550 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "237.2"}]: dispatch 2024-02-23T00:52:31.703 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:52:31.703 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "filestore_inject_stall = '3' (not observed, change may require restart) " 2024-02-23T00:52:31.703 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:52:31.791 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:52:31.824 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:31.845 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:52:31.848 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:31.849 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:52:31.949 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:52:31.977 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:32.079 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:52:32.084 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:32.084 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:52:32.104 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.1 dump_historic_ops 2024-02-23T00:52:32.185 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:52:32.216 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:32.237 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:52:32.239 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:32.240 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:52:32.296 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:52:32.341 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:52:32.371 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:32.424 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:52:32.433 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:32.434 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:52:32.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:31.357832+0000 mgr.y (mgr.14517) 551 : cluster [DBG] pgmap v936: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:52:32.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:31.870318+0000 osd.3 (osd.3) 298 : cluster [DBG] 237.2 scrub starts 2024-02-23T00:52:32.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:31.871295+0000 osd.3 (osd.3) 299 : cluster [DBG] 237.2 scrub ok 2024-02-23T00:52:32.534 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:52:32.562 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:32.799 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:31.357832+0000 mgr.y (mgr.14517) 551 : cluster [DBG] pgmap v936: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:52:32.799 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:31.870318+0000 osd.3 (osd.3) 298 : cluster [DBG] 237.2 scrub starts 2024-02-23T00:52:32.800 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:31.871295+0000 osd.3 (osd.3) 299 : cluster [DBG] 237.2 scrub ok 2024-02-23T00:52:32.800 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:31.357832+0000 mgr.y (mgr.14517) 551 : cluster [DBG] pgmap v936: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:52:32.800 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:31.870318+0000 osd.3 (osd.3) 298 : cluster [DBG] 237.2 scrub starts 2024-02-23T00:52:32.800 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:31.871295+0000 osd.3 (osd.3) 299 : cluster [DBG] 237.2 scrub ok 2024-02-23T00:52:32.809 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:52:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:52:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:52:32.814 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:52:32.816 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:32.817 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:52:32.918 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:52:32.950 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:32.971 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:52:32.974 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:32.974 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:52:33.075 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:52:33.105 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:52:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:52:33.107 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:33.127 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:52:33.131 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:33.131 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:52:33.232 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:52:33.261 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:33.281 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:52:33.283 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:33.284 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:52:33.385 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:52:33.416 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:33.572 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:52:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:52:33.615 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:52:33.617 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:33.618 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:52:33.719 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:52:33.751 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:33.773 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:52:33.775 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:33.776 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:52:33.876 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:52:33.905 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:33.957 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:52:33.963 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:33.964 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:52:34.065 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:52:34.093 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:32.807772+0000 mgr.y (mgr.14517) 552 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:52:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:33.358783+0000 mgr.y (mgr.14517) 553 : cluster [DBG] pgmap v937: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-23T00:52:34.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:32.807772+0000 mgr.y (mgr.14517) 552 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:52:34.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:33.358783+0000 mgr.y (mgr.14517) 553 : cluster [DBG] pgmap v937: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-23T00:52:34.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:32.807772+0000 mgr.y (mgr.14517) 552 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:52:34.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:33.358783+0000 mgr.y (mgr.14517) 553 : cluster [DBG] pgmap v937: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-23T00:52:35.678 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:33.576462+0000 osd.2 (osd.2) 167 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:52:35.678 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:33.576681+0000 osd.2 (osd.2) 168 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:52:35.678 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:34.419672+0000 mon.a (mon.0) 3186 : cluster [DBG] osdmap e622: 8 total, 8 up, 7 in 2024-02-23T00:52:35.678 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:33.576462+0000 osd.2 (osd.2) 167 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:52:35.679 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:33.576681+0000 osd.2 (osd.2) 168 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:52:35.679 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:34.419672+0000 mon.a (mon.0) 3186 : cluster [DBG] osdmap e622: 8 total, 8 up, 7 in 2024-02-23T00:52:35.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:33.576462+0000 osd.2 (osd.2) 167 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:52:35.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:33.576681+0000 osd.2 (osd.2) 168 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:52:35.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:34.419672+0000 mon.a (mon.0) 3186 : cluster [DBG] osdmap e622: 8 total, 8 up, 7 in 2024-02-23T00:52:35.879 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:52:35.883 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:35.884 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:52:35.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.973+0000 7f211f79e700 1 -- 172.21.15.47:0/3059440311 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21180f8420 msgr2=0x7f2118104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:35.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.973+0000 7f211f79e700 1 --2- 172.21.15.47:0/3059440311 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21180f8420 0x7f2118104a20 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 rx=0x7f2110009be0 tx=0x7f2110004d10).stop 2024-02-23T00:52:35.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.973+0000 7f211f79e700 1 -- 172.21.15.47:0/3059440311 shutdown_connections 2024-02-23T00:52:35.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.973+0000 7f211f79e700 1 --2- 172.21.15.47:0/3059440311 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21180f8420 0x7f2118104a20 unknown :-1 s=CLOSED pgs=2757 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:35.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.973+0000 7f211f79e700 1 --2- 172.21.15.47:0/3059440311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21180fdbb0 0x7f21180fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:35.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.973+0000 7f211f79e700 1 --2- 172.21.15.47:0/3059440311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21180fe510 0x7f21180f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:35.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.973+0000 7f211f79e700 1 -- 172.21.15.47:0/3059440311 >> 172.21.15.47:0/3059440311 conn(0x7f21180f3800 msgr2=0x7f21180f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:52:35.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.973+0000 7f211f79e700 1 -- 172.21.15.47:0/3059440311 shutdown_connections 2024-02-23T00:52:35.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.973+0000 7f211f79e700 1 -- 172.21.15.47:0/3059440311 wait complete. 2024-02-23T00:52:35.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.974+0000 7f211f79e700 1 librados: starting msgr at 2024-02-23T00:52:35.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.974+0000 7f211f79e700 1 librados: starting objecter 2024-02-23T00:52:35.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.974+0000 7f211f79e700 1 Processor -- start 2024-02-23T00:52:35.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.974+0000 7f211f79e700 1 -- start start 2024-02-23T00:52:35.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.974+0000 7f211f79e700 1 librados: setting wanted keys 2024-02-23T00:52:35.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.974+0000 7f211f79e700 1 librados: calling monclient init 2024-02-23T00:52:35.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.974+0000 7f211f79e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21180f8420 0x7f211810dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:35.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.974+0000 7f211f79e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21180fdbb0 0x7f211810e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:35.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.974+0000 7f211f79e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21180fe510 0x7f2118107d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:35.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.974+0000 7f211f79e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2118108280 con 0x7f21180f8420 2024-02-23T00:52:35.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.974+0000 7f211f79e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21181083f0 con 0x7f21180fdbb0 2024-02-23T00:52:35.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.974+0000 7f211f79e700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21181085b0 con 0x7f21180fe510 2024-02-23T00:52:35.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.974+0000 7f211dd3b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21180fe510 0x7f2118107d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:35.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.974+0000 7f211d53a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21180f8420 0x7f211810dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:35.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.974+0000 7f211dd3b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21180fe510 0x7f2118107d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:43876/0 (socket says 172.21.15.47:43876) 2024-02-23T00:52:35.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.974+0000 7f211dd3b700 1 -- 172.21.15.47:0/1423377278 learned_addr learned my addr 172.21.15.47:0/1423377278 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:52:35.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.975+0000 7f211cd39700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21180fdbb0 0x7f211810e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:35.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.975+0000 7f211dd3b700 1 -- 172.21.15.47:0/1423377278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21180fdbb0 msgr2=0x7f211810e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:35.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.975+0000 7f211dd3b700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21180fdbb0 0x7f211810e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:35.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.975+0000 7f211dd3b700 1 -- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21180f8420 msgr2=0x7f211810dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:35.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.975+0000 7f211dd3b700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21180f8420 0x7f211810dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:35.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.975+0000 7f211dd3b700 1 -- 172.21.15.47:0/1423377278 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2114005040 con 0x7f21180fe510 2024-02-23T00:52:35.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.975+0000 7f211d53a700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21180f8420 0x7f211810dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:52:35.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.975+0000 7f211cd39700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21180fdbb0 0x7f211810e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:52:35.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.975+0000 7f211dd3b700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21180fe510 0x7f2118107d90 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 rx=0x7f211000bfc0 tx=0x7f2110004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:35.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.975+0000 7f210a7fc700 10 client.?.objecter ms_handle_connect 0x7f21180fe510 2024-02-23T00:52:35.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.975+0000 7f210a7fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:52:35.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.975+0000 7f210a7fc700 1 -- 172.21.15.47:0/1423377278 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211000e040 con 0x7f21180fe510 2024-02-23T00:52:35.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.975+0000 7f210a7fc700 1 -- 172.21.15.47:0/1423377278 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f2110018040 con 0x7f21180fe510 2024-02-23T00:52:35.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.975+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21100097c0 con 0x7f21180fe510 2024-02-23T00:52:35.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.975+0000 7f210a7fc700 1 -- 172.21.15.47:0/1423377278 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211001c3f0 con 0x7f21180fe510 2024-02-23T00:52:35.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.975+0000 7f211f79e700 10 client.41813.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:52:35.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.975+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2118108ad0 con 0x7f21180fe510 2024-02-23T00:52:35.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.975+0000 7f211f79e700 1 librados: init done 2024-02-23T00:52:35.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.976+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f211805ee80 con 0x7f21180fe510 2024-02-23T00:52:35.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.977+0000 7f210a7fc700 1 -- 172.21.15.47:0/1423377278 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f2110013420 con 0x7f21180fe510 2024-02-23T00:52:35.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.977+0000 7f210a7fc700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2104062cc0 0x7f2104065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:35.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.977+0000 7f210a7fc700 1 -- 172.21.15.47:0/1423377278 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(622..622 src has 482..622) v4 ==== 9578+0+0 (secure 0 0 0) 0x7f211007e9b0 con 0x7f21180fe510 2024-02-23T00:52:35.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.977+0000 7f210a7fc700 10 client.41813.objecter ms_dispatch 0x7f2118001550 osd_map(622..622 src has 482..622) v4 2024-02-23T00:52:35.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.977+0000 7f210a7fc700 3 client.41813.objecter handle_osd_map got epochs [622,622] > 0 2024-02-23T00:52:35.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.977+0000 7f210a7fc700 3 client.41813.objecter handle_osd_map decoding full epoch 622 2024-02-23T00:52:35.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.977+0000 7f211d53a700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2104062cc0 0x7f2104065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:35.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.977+0000 7f210a7fc700 20 client.41813.objecter dump_active .. 0 homeless 2024-02-23T00:52:35.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.977+0000 7f210a7fc700 1 -- 172.21.15.47:0/1423377278 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v622) v1 ==== 54+0+16 (secure 0 0 0) 0x7f2110083220 con 0x7f21180fe510 2024-02-23T00:52:35.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.977+0000 7f211f79e700 10 client.41813.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:35.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.977+0000 7f211f79e700 15 client.41813.objecter _session_command_op_assign -1 1 2024-02-23T00:52:35.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.977+0000 7f211f79e700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f21180598f0 0x7f2118059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:35.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.978+0000 7f211f79e700 20 client.41813.objecter _get_session s=0x7f211805fb20 osd=0 3 2024-02-23T00:52:35.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.978+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f211805fb20 osd=0 3 2024-02-23T00:52:35.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.978+0000 7f211f79e700 20 client.41813.objecter _get_session s=0x7f211805fb20 osd=0 3 2024-02-23T00:52:35.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.978+0000 7f211f79e700 15 client.41813.objecter _session_command_op_remove -1 1 2024-02-23T00:52:35.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.978+0000 7f211f79e700 20 client.41813.objecter get_session s=0x7f211805fb20 osd=0 3 2024-02-23T00:52:35.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.978+0000 7f211f79e700 15 client.41813.objecter _session_command_op_assign 0 1 2024-02-23T00:52:35.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.978+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f211805fb20 osd=0 4 2024-02-23T00:52:35.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.978+0000 7f211f79e700 10 client.41813.objecter _send_command 1 2024-02-23T00:52:35.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.978+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2118106ed0 con 0x7f21180598f0 2024-02-23T00:52:35.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.978+0000 7f210a7fc700 10 client.41813.objecter ms_handle_connect 0x7f2104062cc0 2024-02-23T00:52:35.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.978+0000 7f211d53a700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2104062cc0 0x7f2104065180 secure :-1 s=READY pgs=3145 cs=0 l=1 rev1=1 rx=0x7f210c00f2f0 tx=0x7f210c009040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:35.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.978+0000 7f211cd39700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f21180598f0 0x7f2118059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:35.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.978+0000 7f210a7fc700 10 client.41813.objecter ms_handle_connect 0x7f21180598f0 2024-02-23T00:52:35.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.978+0000 7f211cd39700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f21180598f0 0x7f2118059cd0 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:35.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.978+0000 7f210a7fc700 1 -- 172.21.15.47:0/1423377278 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2118106ed0 con 0x7f21180598f0 2024-02-23T00:52:35.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.978+0000 7f210a7fc700 10 client.41813.objecter ms_dispatch 0x7f2118001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:52:35.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.978+0000 7f210a7fc700 10 client.41813.objecter _finish_command 1 = system:0 2024-02-23T00:52:35.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.978+0000 7f210a7fc700 20 client.41813.objecter put_session s=0x7f211805fb20 osd=0 4 2024-02-23T00:52:35.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.978+0000 7f210a7fc700 15 client.41813.objecter _session_command_op_remove 0 1 2024-02-23T00:52:35.985 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:52:35.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.994+0000 7f211f79e700 10 client.41813.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:52:35.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.994+0000 7f211f79e700 15 client.41813.objecter _session_command_op_assign -1 2 2024-02-23T00:52:35.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.994+0000 7f211f79e700 20 client.41813.objecter _get_session s=0x7f211805fb20 osd=0 3 2024-02-23T00:52:35.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.994+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f211805fb20 osd=0 3 2024-02-23T00:52:35.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.994+0000 7f211f79e700 20 client.41813.objecter _get_session s=0x7f211805fb20 osd=0 3 2024-02-23T00:52:35.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.994+0000 7f211f79e700 15 client.41813.objecter _session_command_op_remove -1 2 2024-02-23T00:52:35.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.994+0000 7f211f79e700 20 client.41813.objecter get_session s=0x7f211805fb20 osd=0 3 2024-02-23T00:52:35.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.994+0000 7f211f79e700 15 client.41813.objecter _session_command_op_assign 0 2 2024-02-23T00:52:35.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.994+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f211805fb20 osd=0 4 2024-02-23T00:52:35.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.994+0000 7f211f79e700 10 client.41813.objecter _send_command 2 2024-02-23T00:52:35.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.994+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f21180273c0 con 0x7f21180598f0 2024-02-23T00:52:35.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.995+0000 7f210a7fc700 1 -- 172.21.15.47:0/1423377278 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f21180273c0 con 0x7f21180598f0 2024-02-23T00:52:35.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.995+0000 7f210a7fc700 10 client.41813.objecter ms_dispatch 0x7f2118001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:52:35.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.995+0000 7f210a7fc700 10 client.41813.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:52:35.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.995+0000 7f210a7fc700 20 client.41813.objecter put_session s=0x7f211805fb20 osd=0 4 2024-02-23T00:52:35.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.995+0000 7f210a7fc700 15 client.41813.objecter _session_command_op_remove 0 2 2024-02-23T00:52:35.998 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:52:35.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.995+0000 7f211f79e700 10 client.41813.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:35.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.995+0000 7f211f79e700 15 client.41813.objecter _session_command_op_assign -1 3 2024-02-23T00:52:35.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.995+0000 7f211f79e700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f21180ff140 0x7f21180ff580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:35.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.995+0000 7f211f79e700 20 client.41813.objecter _get_session s=0x7f2118028710 osd=1 3 2024-02-23T00:52:35.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.995+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f2118028710 osd=1 3 2024-02-23T00:52:35.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.995+0000 7f211f79e700 20 client.41813.objecter _get_session s=0x7f2118028710 osd=1 3 2024-02-23T00:52:36.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.995+0000 7f211f79e700 15 client.41813.objecter _session_command_op_remove -1 3 2024-02-23T00:52:36.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.995+0000 7f211f79e700 20 client.41813.objecter get_session s=0x7f2118028710 osd=1 3 2024-02-23T00:52:36.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.995+0000 7f211f79e700 15 client.41813.objecter _session_command_op_assign 1 3 2024-02-23T00:52:36.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.995+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f2118028710 osd=1 4 2024-02-23T00:52:36.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.995+0000 7f211f79e700 10 client.41813.objecter _send_command 3 2024-02-23T00:52:36.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.995+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f211811b1a0 con 0x7f21180ff140 2024-02-23T00:52:36.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.996+0000 7f211d53a700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f21180ff140 0x7f21180ff580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:36.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.996+0000 7f211d53a700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f21180ff140 0x7f21180ff580 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:36.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.996+0000 7f210a7fc700 10 client.41813.objecter ms_handle_connect 0x7f21180ff140 2024-02-23T00:52:36.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.996+0000 7f210a7fc700 1 -- 172.21.15.47:0/1423377278 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f211811b1a0 con 0x7f21180ff140 2024-02-23T00:52:36.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.996+0000 7f210a7fc700 10 client.41813.objecter ms_dispatch 0x7f2118001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:52:36.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.996+0000 7f210a7fc700 10 client.41813.objecter _finish_command 3 = system:0 2024-02-23T00:52:36.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.996+0000 7f210a7fc700 20 client.41813.objecter put_session s=0x7f2118028710 osd=1 4 2024-02-23T00:52:36.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:35.996+0000 7f210a7fc700 15 client.41813.objecter _session_command_op_remove 1 3 2024-02-23T00:52:36.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.012+0000 7f211f79e700 10 client.41813.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:52:36.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.012+0000 7f211f79e700 15 client.41813.objecter _session_command_op_assign -1 4 2024-02-23T00:52:36.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.012+0000 7f211f79e700 20 client.41813.objecter _get_session s=0x7f2118028710 osd=1 3 2024-02-23T00:52:36.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.012+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f2118028710 osd=1 3 2024-02-23T00:52:36.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.012+0000 7f211f79e700 20 client.41813.objecter _get_session s=0x7f2118028710 osd=1 3 2024-02-23T00:52:36.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.012+0000 7f211f79e700 15 client.41813.objecter _session_command_op_remove -1 4 2024-02-23T00:52:36.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.012+0000 7f211f79e700 20 client.41813.objecter get_session s=0x7f2118028710 osd=1 3 2024-02-23T00:52:36.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.012+0000 7f211f79e700 15 client.41813.objecter _session_command_op_assign 1 4 2024-02-23T00:52:36.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.012+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f2118028710 osd=1 4 2024-02-23T00:52:36.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.012+0000 7f211f79e700 10 client.41813.objecter _send_command 4 2024-02-23T00:52:36.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.012+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f21180febb0 con 0x7f21180ff140 2024-02-23T00:52:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.013+0000 7f210a7fc700 1 -- 172.21.15.47:0/1423377278 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f21180febb0 con 0x7f21180ff140 2024-02-23T00:52:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.013+0000 7f210a7fc700 10 client.41813.objecter ms_dispatch 0x7f2118001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:52:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.013+0000 7f210a7fc700 10 client.41813.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:52:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.013+0000 7f210a7fc700 20 client.41813.objecter put_session s=0x7f2118028710 osd=1 4 2024-02-23T00:52:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.013+0000 7f210a7fc700 15 client.41813.objecter _session_command_op_remove 1 4 2024-02-23T00:52:36.016 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:52:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.013+0000 7f211f79e700 10 client.41813.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.013+0000 7f211f79e700 15 client.41813.objecter _session_command_op_assign -1 5 2024-02-23T00:52:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.013+0000 7f211f79e700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f2118028270 0x7f2118119bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.013+0000 7f211f79e700 20 client.41813.objecter _get_session s=0x7f21180fefc0 osd=2 3 2024-02-23T00:52:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.013+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f21180fefc0 osd=2 3 2024-02-23T00:52:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.013+0000 7f211f79e700 20 client.41813.objecter _get_session s=0x7f21180fefc0 osd=2 3 2024-02-23T00:52:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.013+0000 7f211f79e700 15 client.41813.objecter _session_command_op_remove -1 5 2024-02-23T00:52:36.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.013+0000 7f211f79e700 20 client.41813.objecter get_session s=0x7f21180fefc0 osd=2 3 2024-02-23T00:52:36.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.013+0000 7f211f79e700 15 client.41813.objecter _session_command_op_assign 2 5 2024-02-23T00:52:36.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.013+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f21180fefc0 osd=2 4 2024-02-23T00:52:36.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.013+0000 7f211f79e700 10 client.41813.objecter _send_command 5 2024-02-23T00:52:36.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.013+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f211811a140 con 0x7f2118028270 2024-02-23T00:52:36.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.013+0000 7f211cd39700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f2118028270 0x7f2118119bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:36.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.014+0000 7f210a7fc700 10 client.41813.objecter ms_handle_connect 0x7f2118028270 2024-02-23T00:52:36.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.014+0000 7f211cd39700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f2118028270 0x7f2118119bc0 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:36.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.014+0000 7f210a7fc700 1 -- 172.21.15.47:0/1423377278 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f211811a140 con 0x7f2118028270 2024-02-23T00:52:36.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.014+0000 7f210a7fc700 10 client.41813.objecter ms_dispatch 0x7f2118001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:52:36.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.014+0000 7f210a7fc700 10 client.41813.objecter _finish_command 5 = system:0 2024-02-23T00:52:36.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.014+0000 7f210a7fc700 20 client.41813.objecter put_session s=0x7f21180fefc0 osd=2 4 2024-02-23T00:52:36.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.014+0000 7f210a7fc700 15 client.41813.objecter _session_command_op_remove 2 5 2024-02-23T00:52:36.019 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:36.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.029+0000 7f211f79e700 10 client.41813.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:52:36.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.029+0000 7f211f79e700 15 client.41813.objecter _session_command_op_assign -1 6 2024-02-23T00:52:36.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.029+0000 7f211f79e700 20 client.41813.objecter _get_session s=0x7f21180fefc0 osd=2 3 2024-02-23T00:52:36.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.029+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f21180fefc0 osd=2 3 2024-02-23T00:52:36.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.029+0000 7f211f79e700 20 client.41813.objecter _get_session s=0x7f21180fefc0 osd=2 3 2024-02-23T00:52:36.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.029+0000 7f211f79e700 15 client.41813.objecter _session_command_op_remove -1 6 2024-02-23T00:52:36.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.029+0000 7f211f79e700 20 client.41813.objecter get_session s=0x7f21180fefc0 osd=2 3 2024-02-23T00:52:36.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.029+0000 7f211f79e700 15 client.41813.objecter _session_command_op_assign 2 6 2024-02-23T00:52:36.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.029+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f21180fefc0 osd=2 4 2024-02-23T00:52:36.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.029+0000 7f211f79e700 10 client.41813.objecter _send_command 6 2024-02-23T00:52:36.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.029+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f21180277d0 con 0x7f2118028270 2024-02-23T00:52:36.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.030+0000 7f210a7fc700 1 -- 172.21.15.47:0/1423377278 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f21180277d0 con 0x7f2118028270 2024-02-23T00:52:36.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.030+0000 7f210a7fc700 10 client.41813.objecter ms_dispatch 0x7f2118001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:52:36.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.030+0000 7f210a7fc700 10 client.41813.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:52:36.034 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:52:36.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.030+0000 7f210a7fc700 20 client.41813.objecter put_session s=0x7f21180fefc0 osd=2 4 2024-02-23T00:52:36.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.030+0000 7f210a7fc700 15 client.41813.objecter _session_command_op_remove 2 6 2024-02-23T00:52:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.031+0000 7f211f79e700 10 client.41813.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.031+0000 7f211f79e700 15 client.41813.objecter _session_command_op_assign -1 7 2024-02-23T00:52:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.031+0000 7f211f79e700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f211811d320 0x7f211811f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.031+0000 7f211f79e700 20 client.41813.objecter _get_session s=0x7f211811ab10 osd=3 3 2024-02-23T00:52:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.031+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f211811ab10 osd=3 3 2024-02-23T00:52:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.031+0000 7f211f79e700 20 client.41813.objecter _get_session s=0x7f211811ab10 osd=3 3 2024-02-23T00:52:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.031+0000 7f211f79e700 15 client.41813.objecter _session_command_op_remove -1 7 2024-02-23T00:52:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.031+0000 7f211f79e700 20 client.41813.objecter get_session s=0x7f211811ab10 osd=3 3 2024-02-23T00:52:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.031+0000 7f211f79e700 15 client.41813.objecter _session_command_op_assign 3 7 2024-02-23T00:52:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.031+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f211811ab10 osd=3 4 2024-02-23T00:52:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.031+0000 7f211f79e700 10 client.41813.objecter _send_command 7 2024-02-23T00:52:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.031+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f21181224a0 con 0x7f211811d320 2024-02-23T00:52:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.031+0000 7f211dd3b700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f211811d320 0x7f211811f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:36.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.031+0000 7f210a7fc700 10 client.41813.objecter ms_handle_connect 0x7f211811d320 2024-02-23T00:52:36.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.031+0000 7f211dd3b700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f211811d320 0x7f211811f710 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:36.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.031+0000 7f210a7fc700 1 -- 172.21.15.47:0/1423377278 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f21181224a0 con 0x7f211811d320 2024-02-23T00:52:36.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.031+0000 7f210a7fc700 10 client.41813.objecter ms_dispatch 0x7f2118001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:52:36.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.031+0000 7f210a7fc700 10 client.41813.objecter _finish_command 7 = system:0 2024-02-23T00:52:36.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.031+0000 7f210a7fc700 20 client.41813.objecter put_session s=0x7f211811ab10 osd=3 4 2024-02-23T00:52:36.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.031+0000 7f210a7fc700 15 client.41813.objecter _session_command_op_remove 3 7 2024-02-23T00:52:36.039 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:52:36.042 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:36.042 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:52:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.046+0000 7f211f79e700 10 client.41813.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:52:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.046+0000 7f211f79e700 15 client.41813.objecter _session_command_op_assign -1 8 2024-02-23T00:52:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.046+0000 7f211f79e700 20 client.41813.objecter _get_session s=0x7f211811ab10 osd=3 3 2024-02-23T00:52:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.046+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f211811ab10 osd=3 3 2024-02-23T00:52:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.046+0000 7f211f79e700 20 client.41813.objecter _get_session s=0x7f211811ab10 osd=3 3 2024-02-23T00:52:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.046+0000 7f211f79e700 15 client.41813.objecter _session_command_op_remove -1 8 2024-02-23T00:52:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.046+0000 7f211f79e700 20 client.41813.objecter get_session s=0x7f211811ab10 osd=3 3 2024-02-23T00:52:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.046+0000 7f211f79e700 15 client.41813.objecter _session_command_op_assign 3 8 2024-02-23T00:52:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.046+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f211811ab10 osd=3 4 2024-02-23T00:52:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.046+0000 7f211f79e700 10 client.41813.objecter _send_command 8 2024-02-23T00:52:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.046+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f211811a580 con 0x7f211811d320 2024-02-23T00:52:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.047+0000 7f210a7fc700 1 -- 172.21.15.47:0/1423377278 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f211811a580 con 0x7f211811d320 2024-02-23T00:52:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.047+0000 7f210a7fc700 10 client.41813.objecter ms_dispatch 0x7f2118001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:52:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.047+0000 7f210a7fc700 10 client.41813.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:52:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.047+0000 7f210a7fc700 20 client.41813.objecter put_session s=0x7f211811ab10 osd=3 4 2024-02-23T00:52:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.047+0000 7f210a7fc700 15 client.41813.objecter _session_command_op_remove 3 8 2024-02-23T00:52:36.051 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:52:36.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.048+0000 7f211f79e700 10 client.41813.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:36.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.048+0000 7f211f79e700 15 client.41813.objecter _session_command_op_assign -1 9 2024-02-23T00:52:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.048+0000 7f211f79e700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2118119720 0x7f211811fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.048+0000 7f211f79e700 20 client.41813.objecter _get_session s=0x7f211811a580 osd=4 3 2024-02-23T00:52:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.048+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f211811a580 osd=4 3 2024-02-23T00:52:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.048+0000 7f211f79e700 20 client.41813.objecter _get_session s=0x7f211811a580 osd=4 3 2024-02-23T00:52:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.048+0000 7f211f79e700 15 client.41813.objecter _session_command_op_remove -1 9 2024-02-23T00:52:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.048+0000 7f211f79e700 20 client.41813.objecter get_session s=0x7f211811a580 osd=4 3 2024-02-23T00:52:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.048+0000 7f211f79e700 15 client.41813.objecter _session_command_op_assign 4 9 2024-02-23T00:52:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.048+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f211811a580 osd=4 4 2024-02-23T00:52:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.048+0000 7f211f79e700 10 client.41813.objecter _send_command 9 2024-02-23T00:52:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.048+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f211811a8f0 con 0x7f2118119720 2024-02-23T00:52:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.048+0000 7f211d53a700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2118119720 0x7f211811fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.048+0000 7f210a7fc700 10 client.41813.objecter ms_handle_connect 0x7f2118119720 2024-02-23T00:52:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.048+0000 7f211d53a700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2118119720 0x7f211811fca0 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:36.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.049+0000 7f210a7fc700 1 -- 172.21.15.47:0/1423377278 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f211811a8f0 con 0x7f2118119720 2024-02-23T00:52:36.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.049+0000 7f210a7fc700 10 client.41813.objecter ms_dispatch 0x7f2118001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:52:36.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.049+0000 7f210a7fc700 10 client.41813.objecter _finish_command 9 = system:0 2024-02-23T00:52:36.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.049+0000 7f210a7fc700 20 client.41813.objecter put_session s=0x7f211811a580 osd=4 4 2024-02-23T00:52:36.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.049+0000 7f210a7fc700 15 client.41813.objecter _session_command_op_remove 4 9 2024-02-23T00:52:36.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.064+0000 7f211f79e700 10 client.41813.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:52:36.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.064+0000 7f211f79e700 15 client.41813.objecter _session_command_op_assign -1 10 2024-02-23T00:52:36.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.064+0000 7f211f79e700 20 client.41813.objecter _get_session s=0x7f211811a580 osd=4 3 2024-02-23T00:52:36.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.064+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f211811a580 osd=4 3 2024-02-23T00:52:36.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.064+0000 7f211f79e700 20 client.41813.objecter _get_session s=0x7f211811a580 osd=4 3 2024-02-23T00:52:36.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.064+0000 7f211f79e700 15 client.41813.objecter _session_command_op_remove -1 10 2024-02-23T00:52:36.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.064+0000 7f211f79e700 20 client.41813.objecter get_session s=0x7f211811a580 osd=4 3 2024-02-23T00:52:36.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.064+0000 7f211f79e700 15 client.41813.objecter _session_command_op_assign 4 10 2024-02-23T00:52:36.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.064+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f211811a580 osd=4 4 2024-02-23T00:52:36.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.064+0000 7f211f79e700 10 client.41813.objecter _send_command 10 2024-02-23T00:52:36.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.064+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f211811a8f0 con 0x7f2118119720 2024-02-23T00:52:36.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.066+0000 7f210a7fc700 1 -- 172.21.15.47:0/1423377278 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f211811a8f0 con 0x7f2118119720 2024-02-23T00:52:36.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.066+0000 7f210a7fc700 10 client.41813.objecter ms_dispatch 0x7f2118001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:52:36.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.066+0000 7f210a7fc700 10 client.41813.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:52:36.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.066+0000 7f210a7fc700 20 client.41813.objecter put_session s=0x7f211811a580 osd=4 4 2024-02-23T00:52:36.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.066+0000 7f210a7fc700 15 client.41813.objecter _session_command_op_remove 4 10 2024-02-23T00:52:36.069 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:52:36.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.066+0000 7f211f79e700 10 client.41813.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:36.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.066+0000 7f211f79e700 15 client.41813.objecter _session_command_op_assign -1 11 2024-02-23T00:52:36.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.066+0000 7f211f79e700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f21181206a0 0x7f2118120b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:36.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.066+0000 7f211f79e700 20 client.41813.objecter _get_session s=0x7f211811a8f0 osd=5 3 2024-02-23T00:52:36.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.066+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f211811a8f0 osd=5 3 2024-02-23T00:52:36.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.066+0000 7f211f79e700 20 client.41813.objecter _get_session s=0x7f211811a8f0 osd=5 3 2024-02-23T00:52:36.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.066+0000 7f211f79e700 15 client.41813.objecter _session_command_op_remove -1 11 2024-02-23T00:52:36.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.066+0000 7f211f79e700 20 client.41813.objecter get_session s=0x7f211811a8f0 osd=5 3 2024-02-23T00:52:36.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.066+0000 7f211f79e700 15 client.41813.objecter _session_command_op_assign 5 11 2024-02-23T00:52:36.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.066+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f211811a8f0 osd=5 4 2024-02-23T00:52:36.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.066+0000 7f211f79e700 10 client.41813.objecter _send_command 11 2024-02-23T00:52:36.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.066+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2118128df0 con 0x7f21181206a0 2024-02-23T00:52:36.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.067+0000 7f211cd39700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f21181206a0 0x7f2118120b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:36.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.067+0000 7f210a7fc700 10 client.41813.objecter ms_handle_connect 0x7f21181206a0 2024-02-23T00:52:36.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.067+0000 7f211cd39700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f21181206a0 0x7f2118120b00 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:36.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.068+0000 7f210a7fc700 1 -- 172.21.15.47:0/1423377278 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2118128df0 con 0x7f21181206a0 2024-02-23T00:52:36.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.068+0000 7f210a7fc700 10 client.41813.objecter ms_dispatch 0x7f2118001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:52:36.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.068+0000 7f210a7fc700 10 client.41813.objecter _finish_command 11 = system:0 2024-02-23T00:52:36.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.068+0000 7f210a7fc700 20 client.41813.objecter put_session s=0x7f211811a8f0 osd=5 4 2024-02-23T00:52:36.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.068+0000 7f210a7fc700 15 client.41813.objecter _session_command_op_remove 5 11 2024-02-23T00:52:36.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.082+0000 7f211f79e700 10 client.41813.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:52:36.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.083+0000 7f211f79e700 15 client.41813.objecter _session_command_op_assign -1 12 2024-02-23T00:52:36.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.083+0000 7f211f79e700 20 client.41813.objecter _get_session s=0x7f211811a8f0 osd=5 3 2024-02-23T00:52:36.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.083+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f211811a8f0 osd=5 3 2024-02-23T00:52:36.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.083+0000 7f211f79e700 20 client.41813.objecter _get_session s=0x7f211811a8f0 osd=5 3 2024-02-23T00:52:36.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.083+0000 7f211f79e700 15 client.41813.objecter _session_command_op_remove -1 12 2024-02-23T00:52:36.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.083+0000 7f211f79e700 20 client.41813.objecter get_session s=0x7f211811a8f0 osd=5 3 2024-02-23T00:52:36.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.083+0000 7f211f79e700 15 client.41813.objecter _session_command_op_assign 5 12 2024-02-23T00:52:36.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.083+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f211811a8f0 osd=5 4 2024-02-23T00:52:36.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.083+0000 7f211f79e700 10 client.41813.objecter _send_command 12 2024-02-23T00:52:36.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.083+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2118126690 con 0x7f21181206a0 2024-02-23T00:52:36.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.084+0000 7f210a7fc700 1 -- 172.21.15.47:0/1423377278 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2118126690 con 0x7f21181206a0 2024-02-23T00:52:36.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.084+0000 7f210a7fc700 10 client.41813.objecter ms_dispatch 0x7f2118001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:52:36.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.084+0000 7f210a7fc700 10 client.41813.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:52:36.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.084+0000 7f210a7fc700 20 client.41813.objecter put_session s=0x7f211811a8f0 osd=5 4 2024-02-23T00:52:36.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.084+0000 7f210a7fc700 15 client.41813.objecter _session_command_op_remove 5 12 2024-02-23T00:52:36.087 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:52:36.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.084+0000 7f211f79e700 10 client.41813.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:36.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.084+0000 7f211f79e700 15 client.41813.objecter _session_command_op_assign -1 13 2024-02-23T00:52:36.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.084+0000 7f211f79e700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f211806b420 0x7f2118128870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:36.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.084+0000 7f211f79e700 20 client.41813.objecter _get_session s=0x7f2118126690 osd=6 3 2024-02-23T00:52:36.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.084+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f2118126690 osd=6 3 2024-02-23T00:52:36.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.084+0000 7f211f79e700 20 client.41813.objecter _get_session s=0x7f2118126690 osd=6 3 2024-02-23T00:52:36.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.084+0000 7f211f79e700 15 client.41813.objecter _session_command_op_remove -1 13 2024-02-23T00:52:36.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.084+0000 7f211f79e700 20 client.41813.objecter get_session s=0x7f2118126690 osd=6 3 2024-02-23T00:52:36.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.084+0000 7f211f79e700 15 client.41813.objecter _session_command_op_assign 6 13 2024-02-23T00:52:36.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.084+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f2118126690 osd=6 4 2024-02-23T00:52:36.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.084+0000 7f211f79e700 10 client.41813.objecter _send_command 13 2024-02-23T00:52:36.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.084+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f211806ce40 con 0x7f211806b420 2024-02-23T00:52:36.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.085+0000 7f211dd3b700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f211806b420 0x7f2118128870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:36.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.085+0000 7f210a7fc700 10 client.41813.objecter ms_handle_connect 0x7f211806b420 2024-02-23T00:52:36.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.085+0000 7f211dd3b700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f211806b420 0x7f2118128870 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:36.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.086+0000 7f210a7fc700 1 -- 172.21.15.47:0/1423377278 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f211806ce40 con 0x7f211806b420 2024-02-23T00:52:36.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.086+0000 7f210a7fc700 10 client.41813.objecter ms_dispatch 0x7f2118001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:52:36.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.086+0000 7f210a7fc700 10 client.41813.objecter _finish_command 13 = system:0 2024-02-23T00:52:36.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.086+0000 7f210a7fc700 20 client.41813.objecter put_session s=0x7f2118126690 osd=6 4 2024-02-23T00:52:36.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.086+0000 7f210a7fc700 15 client.41813.objecter _session_command_op_remove 6 13 2024-02-23T00:52:36.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.103+0000 7f211f79e700 10 client.41813.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:52:36.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.103+0000 7f211f79e700 15 client.41813.objecter _session_command_op_assign -1 14 2024-02-23T00:52:36.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.103+0000 7f211f79e700 20 client.41813.objecter _get_session s=0x7f2118126690 osd=6 3 2024-02-23T00:52:36.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.103+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f2118126690 osd=6 3 2024-02-23T00:52:36.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.103+0000 7f211f79e700 20 client.41813.objecter _get_session s=0x7f2118126690 osd=6 3 2024-02-23T00:52:36.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.103+0000 7f211f79e700 15 client.41813.objecter _session_command_op_remove -1 14 2024-02-23T00:52:36.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.103+0000 7f211f79e700 20 client.41813.objecter get_session s=0x7f2118126690 osd=6 3 2024-02-23T00:52:36.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.103+0000 7f211f79e700 15 client.41813.objecter _session_command_op_assign 6 14 2024-02-23T00:52:36.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.103+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f2118126690 osd=6 4 2024-02-23T00:52:36.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.103+0000 7f211f79e700 10 client.41813.objecter _send_command 14 2024-02-23T00:52:36.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.103+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f211806b860 con 0x7f211806b420 2024-02-23T00:52:36.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.105+0000 7f210a7fc700 1 -- 172.21.15.47:0/1423377278 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f211806b860 con 0x7f211806b420 2024-02-23T00:52:36.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.105+0000 7f210a7fc700 10 client.41813.objecter ms_dispatch 0x7f2118001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:52:36.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.105+0000 7f210a7fc700 10 client.41813.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:52:36.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.105+0000 7f210a7fc700 20 client.41813.objecter put_session s=0x7f2118126690 osd=6 4 2024-02-23T00:52:36.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.105+0000 7f210a7fc700 15 client.41813.objecter _session_command_op_remove 6 14 2024-02-23T00:52:36.108 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:52:36.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.105+0000 7f211f79e700 10 client.41813.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:36.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.105+0000 7f211f79e700 15 client.41813.objecter _session_command_op_assign -1 15 2024-02-23T00:52:36.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.105+0000 7f211f79e700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f211806bee0 0x7f211806c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:36.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.105+0000 7f211f79e700 20 client.41813.objecter _get_session s=0x7f211806b860 osd=7 3 2024-02-23T00:52:36.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.105+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f211806b860 osd=7 3 2024-02-23T00:52:36.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.105+0000 7f211f79e700 20 client.41813.objecter _get_session s=0x7f211806b860 osd=7 3 2024-02-23T00:52:36.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.105+0000 7f211f79e700 15 client.41813.objecter _session_command_op_remove -1 15 2024-02-23T00:52:36.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.105+0000 7f211f79e700 20 client.41813.objecter get_session s=0x7f211806b860 osd=7 3 2024-02-23T00:52:36.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.105+0000 7f211f79e700 15 client.41813.objecter _session_command_op_assign 7 15 2024-02-23T00:52:36.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.105+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f211806b860 osd=7 4 2024-02-23T00:52:36.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.105+0000 7f211f79e700 10 client.41813.objecter _send_command 15 2024-02-23T00:52:36.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.105+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f211806c8c0 con 0x7f211806bee0 2024-02-23T00:52:36.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.105+0000 7f211d53a700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f211806bee0 0x7f211806c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:36.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.106+0000 7f210a7fc700 10 client.41813.objecter ms_handle_connect 0x7f211806bee0 2024-02-23T00:52:36.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.106+0000 7f211d53a700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f211806bee0 0x7f211806c320 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:36.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.107+0000 7f210a7fc700 1 -- 172.21.15.47:0/1423377278 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f211806c8c0 con 0x7f211806bee0 2024-02-23T00:52:36.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.107+0000 7f210a7fc700 10 client.41813.objecter ms_dispatch 0x7f2118001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:52:36.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.107+0000 7f210a7fc700 10 client.41813.objecter _finish_command 15 = system:0 2024-02-23T00:52:36.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.107+0000 7f210a7fc700 20 client.41813.objecter put_session s=0x7f211806b860 osd=7 4 2024-02-23T00:52:36.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.107+0000 7f210a7fc700 15 client.41813.objecter _session_command_op_remove 7 15 2024-02-23T00:52:36.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.121+0000 7f211f79e700 10 client.41813.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:52:36.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.121+0000 7f211f79e700 15 client.41813.objecter _session_command_op_assign -1 16 2024-02-23T00:52:36.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.121+0000 7f211f79e700 20 client.41813.objecter _get_session s=0x7f211806b860 osd=7 3 2024-02-23T00:52:36.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.121+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f211806b860 osd=7 3 2024-02-23T00:52:36.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.121+0000 7f211f79e700 20 client.41813.objecter _get_session s=0x7f211806b860 osd=7 3 2024-02-23T00:52:36.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.121+0000 7f211f79e700 15 client.41813.objecter _session_command_op_remove -1 16 2024-02-23T00:52:36.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.121+0000 7f211f79e700 20 client.41813.objecter get_session s=0x7f211806b860 osd=7 3 2024-02-23T00:52:36.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.121+0000 7f211f79e700 15 client.41813.objecter _session_command_op_assign 7 16 2024-02-23T00:52:36.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.121+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f211806b860 osd=7 4 2024-02-23T00:52:36.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.121+0000 7f211f79e700 10 client.41813.objecter _send_command 16 2024-02-23T00:52:36.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.121+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f211806c8c0 con 0x7f211806bee0 2024-02-23T00:52:36.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.123+0000 7f210a7fc700 1 -- 172.21.15.47:0/1423377278 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f211806c8c0 con 0x7f211806bee0 2024-02-23T00:52:36.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.123+0000 7f210a7fc700 10 client.41813.objecter ms_dispatch 0x7f2118001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:52:36.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.123+0000 7f210a7fc700 10 client.41813.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:52:36.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.123+0000 7f210a7fc700 20 client.41813.objecter put_session s=0x7f211806b860 osd=7 4 2024-02-23T00:52:36.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.123+0000 7f210a7fc700 15 client.41813.objecter _session_command_op_remove 7 16 2024-02-23T00:52:36.127 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:52:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 10 librados: watch_flush enter 2024-02-23T00:52:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 10 librados: watch_flush exit 2024-02-23T00:52:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 10 client.41813.objecter close_session for osd.0 2024-02-23T00:52:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f21180598f0 msgr2=0x7f2118059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f21180598f0 0x7f2118059cd0 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f211805fb20 osd=0 1 2024-02-23T00:52:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 10 client.41813.objecter close_session for osd.1 2024-02-23T00:52:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f21180ff140 msgr2=0x7f21180ff580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f21180ff140 0x7f21180ff580 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f2118028710 osd=1 1 2024-02-23T00:52:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 10 client.41813.objecter close_session for osd.2 2024-02-23T00:52:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f2118028270 msgr2=0x7f2118119bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f2118028270 0x7f2118119bc0 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f21180fefc0 osd=2 1 2024-02-23T00:52:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 10 client.41813.objecter close_session for osd.3 2024-02-23T00:52:36.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f211811d320 msgr2=0x7f211811f710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:36.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f211811d320 0x7f211811f710 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:36.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f211811ab10 osd=3 1 2024-02-23T00:52:36.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 10 client.41813.objecter close_session for osd.4 2024-02-23T00:52:36.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2118119720 msgr2=0x7f211811fca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:36.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2118119720 0x7f211811fca0 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:36.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f211811a580 osd=4 1 2024-02-23T00:52:36.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 10 client.41813.objecter close_session for osd.5 2024-02-23T00:52:36.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f21181206a0 msgr2=0x7f2118120b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:36.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f21181206a0 0x7f2118120b00 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:36.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f211811a8f0 osd=5 1 2024-02-23T00:52:36.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 10 client.41813.objecter close_session for osd.6 2024-02-23T00:52:36.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f211806b420 msgr2=0x7f2118128870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:36.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f211806b420 0x7f2118128870 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:36.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f2118126690 osd=6 1 2024-02-23T00:52:36.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 10 client.41813.objecter close_session for osd.7 2024-02-23T00:52:36.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211dd3b700 1 -- 172.21.15.47:0/1423377278 reap_dead start 2024-02-23T00:52:36.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f211806bee0 msgr2=0x7f211806c320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:36.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f211806bee0 0x7f211806c320 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:36.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211dd3b700 1 -- 172.21.15.47:0/1423377278 reap_dead start 2024-02-23T00:52:36.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 20 client.41813.objecter put_session s=0x7f211806b860 osd=7 1 2024-02-23T00:52:36.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 20 client.41813.objecter shutdown clearing up homeless session... 2024-02-23T00:52:36.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 10 client.41813.objecter successfully canceled tick 2024-02-23T00:52:36.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211dd3b700 1 -- 172.21.15.47:0/1423377278 reap_dead start 2024-02-23T00:52:36.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211dd3b700 1 -- 172.21.15.47:0/1423377278 reap_dead start 2024-02-23T00:52:36.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2104062cc0 msgr2=0x7f2104065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:36.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2104062cc0 0x7f2104065180 secure :-1 s=READY pgs=3145 cs=0 l=1 rev1=1 rx=0x7f210c00f2f0 tx=0x7f210c009040).stop 2024-02-23T00:52:36.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21180fe510 msgr2=0x7f2118107d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:36.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21180fe510 0x7f2118107d90 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 rx=0x7f211000bfc0 tx=0x7f2110004d10).stop 2024-02-23T00:52:36.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 shutdown_connections 2024-02-23T00:52:36.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21180fe510 0x7f2118107d90 unknown :-1 s=CLOSED pgs=2758 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:36.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 1 --2- 172.21.15.47:0/1423377278 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2104062cc0 0x7f2104065180 unknown :-1 s=CLOSED pgs=3145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:36.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 >> 172.21.15.47:0/1423377278 conn(0x7f21180f3800 msgr2=0x7f21180fbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:52:36.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.124+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 shutdown_connections 2024-02-23T00:52:36.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.125+0000 7f211f79e700 1 -- 172.21.15.47:0/1423377278 wait complete. 2024-02-23T00:52:36.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:36.125+0000 7f211f79e700 1 librados: shutdown 2024-02-23T00:52:36.133 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:52:36.133 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:52:36.133 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:52:36.133 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:52:36.133 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:52:36.133 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:52:36.133 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:52:36.133 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:52:36.143 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:52:36.175 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:36.425 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:52:36.425 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:35.359170+0000 mgr.y (mgr.14517) 554 : cluster [DBG] pgmap v939: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.1 KiB/s rd, 504 B/s wr, 2 op/s 2024-02-23T00:52:36.425 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:35.978391+0000 mon.c (mon.2) 355 : audit [DBG] from='client.? 172.21.15.47:0/1423377278' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:52:36.428 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:36.428 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:52:36.529 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:52:36.559 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:36.589 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:52:36.591 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:36.592 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:52:36.688 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:35.359170+0000 mgr.y (mgr.14517) 554 : cluster [DBG] pgmap v939: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.1 KiB/s rd, 504 B/s wr, 2 op/s 2024-02-23T00:52:36.688 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:35.978391+0000 mon.c (mon.2) 355 : audit [DBG] from='client.? 172.21.15.47:0/1423377278' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:52:36.695 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:52:36.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:35.359170+0000 mgr.y (mgr.14517) 554 : cluster [DBG] pgmap v939: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.1 KiB/s rd, 504 B/s wr, 2 op/s 2024-02-23T00:52:36.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:35.978391+0000 mon.c (mon.2) 355 : audit [DBG] from='client.? 172.21.15.47:0/1423377278' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:52:36.727 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:36.749 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:52:36.752 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:36.753 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:52:36.853 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:52:36.883 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:36.904 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:52:36.907 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:36.907 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:52:37.008 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:52:37.036 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:37.055 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:52:37.057 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:37.058 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:52:37.158 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:52:37.185 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:37.204 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:52:37.206 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:37.207 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:52:37.308 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:52:37.334 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:37.353 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:52:37.356 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:37.356 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:52:37.457 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:52:37.483 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:37.502 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:52:37.504 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:37.505 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:52:37.605 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:52:37.632 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:37.650 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:52:37.652 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:37.653 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:52:37.753 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:52:37.779 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:37.798 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:52:37.800 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:37.800 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:52:37.819 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-23T00:52:37.819 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-23T00:52:37.819 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-02-23T00:52:37.819 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-02-23T00:52:37.819 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-02-23T00:52:37.820 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.3 config set filestore_inject_stall 3 2024-02-23T00:52:37.902 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:52:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:37.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.672+0000 7f38d9bca700 1 -- 172.21.15.47:0/621705356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38d40f8420 msgr2=0x7f38d4104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:37.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.672+0000 7f38d9bca700 1 --2- 172.21.15.47:0/621705356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38d40f8420 0x7f38d4104a20 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 rx=0x7f38cc009be0 tx=0x7f38cc004d10).stop 2024-02-23T00:52:37.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.672+0000 7f38d9bca700 1 -- 172.21.15.47:0/621705356 shutdown_connections 2024-02-23T00:52:37.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.672+0000 7f38d9bca700 1 --2- 172.21.15.47:0/621705356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38d40f8420 0x7f38d4104a20 unknown :-1 s=CLOSED pgs=2597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:37.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.672+0000 7f38d9bca700 1 --2- 172.21.15.47:0/621705356 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f38d40fdbb0 0x7f38d40fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:37.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.672+0000 7f38d9bca700 1 --2- 172.21.15.47:0/621705356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38d40fe510 0x7f38d40f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:37.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.672+0000 7f38d9bca700 1 -- 172.21.15.47:0/621705356 >> 172.21.15.47:0/621705356 conn(0x7f38d40f3800 msgr2=0x7f38d40f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:52:37.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.672+0000 7f38d9bca700 1 -- 172.21.15.47:0/621705356 shutdown_connections 2024-02-23T00:52:37.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.672+0000 7f38d9bca700 1 -- 172.21.15.47:0/621705356 wait complete. 2024-02-23T00:52:37.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.673+0000 7f38d9bca700 1 librados: starting msgr at 2024-02-23T00:52:37.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.673+0000 7f38d9bca700 1 librados: starting objecter 2024-02-23T00:52:37.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.673+0000 7f38d9bca700 1 Processor -- start 2024-02-23T00:52:37.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.673+0000 7f38d9bca700 1 -- start start 2024-02-23T00:52:37.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.673+0000 7f38d9bca700 1 librados: setting wanted keys 2024-02-23T00:52:37.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.673+0000 7f38d9bca700 1 librados: calling monclient init 2024-02-23T00:52:37.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.673+0000 7f38d9bca700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38d40f8420 0x7f38d406b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:37.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.673+0000 7f38d9bca700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f38d40fdbb0 0x7f38d406b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:37.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.673+0000 7f38d9bca700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38d40fe510 0x7f38d406bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:37.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.674+0000 7f38d9bca700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d4116200 con 0x7f38d40fe510 2024-02-23T00:52:37.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.674+0000 7f38d9bca700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d4116370 con 0x7f38d40f8420 2024-02-23T00:52:37.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.674+0000 7f38d9bca700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d4116530 con 0x7f38d40fdbb0 2024-02-23T00:52:37.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.674+0000 7f38d2ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f38d40fdbb0 0x7f38d406b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:37.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.674+0000 7f38d2ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f38d40fdbb0 0x7f38d406b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:55876/0 (socket says 172.21.15.47:55876) 2024-02-23T00:52:37.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.674+0000 7f38d3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38d40fe510 0x7f38d406bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:37.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.674+0000 7f38d2ffd700 1 -- 172.21.15.47:0/2164424395 learned_addr learned my addr 172.21.15.47:0/2164424395 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:52:37.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.674+0000 7f38d37fe700 1 --2- 172.21.15.47:0/2164424395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38d40f8420 0x7f38d406b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:37.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.674+0000 7f38d2ffd700 1 -- 172.21.15.47:0/2164424395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38d40f8420 msgr2=0x7f38d406b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:37.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.674+0000 7f38d2ffd700 1 --2- 172.21.15.47:0/2164424395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38d40f8420 0x7f38d406b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:37.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.674+0000 7f38d2ffd700 1 -- 172.21.15.47:0/2164424395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38d40fe510 msgr2=0x7f38d406bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:37.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.674+0000 7f38d2ffd700 1 --2- 172.21.15.47:0/2164424395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38d40fe510 0x7f38d406bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:37.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.674+0000 7f38d2ffd700 1 -- 172.21.15.47:0/2164424395 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38cc0097c0 con 0x7f38d40fdbb0 2024-02-23T00:52:37.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.674+0000 7f38d3fff700 1 --2- 172.21.15.47:0/2164424395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38d40fe510 0x7f38d406bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:52:37.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.674+0000 7f38d2ffd700 1 --2- 172.21.15.47:0/2164424395 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f38d40fdbb0 0x7f38d406b5c0 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 rx=0x7f38c80041d0 tx=0x7f38c8009f60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:37.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.674+0000 7f38d0ff9700 10 client.?.objecter ms_handle_connect 0x7f38d40fdbb0 2024-02-23T00:52:37.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.674+0000 7f38d0ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:52:37.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.674+0000 7f38d0ff9700 1 -- 172.21.15.47:0/2164424395 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8007b40 con 0x7f38d40fdbb0 2024-02-23T00:52:37.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.674+0000 7f38d0ff9700 1 -- 172.21.15.47:0/2164424395 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f38c800f040 con 0x7f38d40fdbb0 2024-02-23T00:52:37.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.674+0000 7f38d9bca700 1 -- 172.21.15.47:0/2164424395 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d4116810 con 0x7f38d40fdbb0 2024-02-23T00:52:37.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.674+0000 7f38d0ff9700 1 -- 172.21.15.47:0/2164424395 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c80146e0 con 0x7f38d40fdbb0 2024-02-23T00:52:37.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.674+0000 7f38d9bca700 10 client.41819.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:52:37.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.674+0000 7f38d9bca700 1 -- 172.21.15.47:0/2164424395 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d4116d80 con 0x7f38d40fdbb0 2024-02-23T00:52:37.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.674+0000 7f38d9bca700 1 librados: init done 2024-02-23T00:52:37.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.675+0000 7f38d9bca700 1 -- 172.21.15.47:0/2164424395 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d405ee80 con 0x7f38d40fdbb0 2024-02-23T00:52:37.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.676+0000 7f38d0ff9700 1 -- 172.21.15.47:0/2164424395 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f38c8014ad0 con 0x7f38d40fdbb0 2024-02-23T00:52:37.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.676+0000 7f38d0ff9700 1 --2- 172.21.15.47:0/2164424395 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f38bc062cc0 0x7f38bc065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:37.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.676+0000 7f38d0ff9700 1 -- 172.21.15.47:0/2164424395 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(622..622 src has 482..622) v4 ==== 9578+0+0 (secure 0 0 0) 0x7f38c807d3c0 con 0x7f38d40fdbb0 2024-02-23T00:52:37.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.676+0000 7f38d0ff9700 10 client.41819.objecter ms_dispatch 0x7f38d4001550 osd_map(622..622 src has 482..622) v4 2024-02-23T00:52:37.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.676+0000 7f38d0ff9700 3 client.41819.objecter handle_osd_map got epochs [622,622] > 0 2024-02-23T00:52:37.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.676+0000 7f38d0ff9700 3 client.41819.objecter handle_osd_map decoding full epoch 622 2024-02-23T00:52:37.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.676+0000 7f38d37fe700 1 --2- 172.21.15.47:0/2164424395 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f38bc062cc0 0x7f38bc065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:37.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.676+0000 7f38d0ff9700 20 client.41819.objecter dump_active .. 0 homeless 2024-02-23T00:52:37.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.676+0000 7f38d0ff9700 10 client.41819.objecter ms_handle_connect 0x7f38bc062cc0 2024-02-23T00:52:37.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.676+0000 7f38d37fe700 1 --2- 172.21.15.47:0/2164424395 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f38bc062cc0 0x7f38bc065180 secure :-1 s=READY pgs=3146 cs=0 l=1 rev1=1 rx=0x7f38c40092f0 tx=0x7f38c400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:37.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.680+0000 7f38d0ff9700 1 -- 172.21.15.47:0/2164424395 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f38c804c8f0 con 0x7f38d40fdbb0 2024-02-23T00:52:37.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:37.864+0000 7f38d9bca700 1 -- 172.21.15.47:0/2164424395 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7f38d405fa60 con 0x7f38d40fdbb0 2024-02-23T00:52:38.003 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:52:38.006 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:38.007 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:52:38.107 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:52:38.135 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:38.154 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:52:38.156 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:38.157 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:52:38.258 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:52:38.285 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:38.303 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:52:38.305 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:38.306 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:52:38.407 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:52:38.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:38.426+0000 7f38d0ff9700 1 -- 172.21.15.47:0/2164424395 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v623) v1 ==== 97+0+0 (secure 0 0 0) 0x7f38c804ff10 con 0x7f38d40fdbb0 2024-02-23T00:52:38.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:38.429+0000 7f38d9bca700 10 librados: watch_flush enter 2024-02-23T00:52:38.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:38.429+0000 7f38d9bca700 10 librados: watch_flush exit 2024-02-23T00:52:38.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:38.429+0000 7f38d9bca700 20 client.41819.objecter shutdown clearing up homeless session... 2024-02-23T00:52:38.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:38.429+0000 7f38d9bca700 10 client.41819.objecter successfully canceled tick 2024-02-23T00:52:38.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:38.429+0000 7f38d9bca700 1 -- 172.21.15.47:0/2164424395 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f38bc062cc0 msgr2=0x7f38bc065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:38.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:38.429+0000 7f38d9bca700 1 --2- 172.21.15.47:0/2164424395 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f38bc062cc0 0x7f38bc065180 secure :-1 s=READY pgs=3146 cs=0 l=1 rev1=1 rx=0x7f38c40092f0 tx=0x7f38c400b040).stop 2024-02-23T00:52:38.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:38.429+0000 7f38d9bca700 1 -- 172.21.15.47:0/2164424395 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f38d40fdbb0 msgr2=0x7f38d406b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:38.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:38.429+0000 7f38d9bca700 1 --2- 172.21.15.47:0/2164424395 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f38d40fdbb0 0x7f38d406b5c0 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 rx=0x7f38c80041d0 tx=0x7f38c8009f60).stop 2024-02-23T00:52:38.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:38.431+0000 7f38d9bca700 1 -- 172.21.15.47:0/2164424395 shutdown_connections 2024-02-23T00:52:38.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:38.431+0000 7f38d9bca700 1 --2- 172.21.15.47:0/2164424395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38d40fe510 0x7f38d406bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:38.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:38.431+0000 7f38d9bca700 1 --2- 172.21.15.47:0/2164424395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38d40f8420 0x7f38d406b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:38.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:38.431+0000 7f38d9bca700 1 --2- 172.21.15.47:0/2164424395 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f38bc062cc0 0x7f38bc065180 unknown :-1 s=CLOSED pgs=3146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:38.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:38.431+0000 7f38d9bca700 1 --2- 172.21.15.47:0/2164424395 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f38d40fdbb0 0x7f38d406b5c0 unknown :-1 s=CLOSED pgs=2759 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:38.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:38.431+0000 7f38d9bca700 1 -- 172.21.15.47:0/2164424395 >> 172.21.15.47:0/2164424395 conn(0x7f38d40f3800 msgr2=0x7f38d40fbb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:52:38.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:38.431+0000 7f38d9bca700 1 -- 172.21.15.47:0/2164424395 shutdown_connections 2024-02-23T00:52:38.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:38.431+0000 7f38d9bca700 1 -- 172.21.15.47:0/2164424395 wait complete. 2024-02-23T00:52:38.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:38.432+0000 7f38d9bca700 1 librados: shutdown 2024-02-23T00:52:38.435 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:38.435 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is set 2024-02-23T00:52:38.453 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:52:38.455 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:38.455 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:52:38.477 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:52:38.477 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 20, 2024-02-23T00:52:38.477 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 600, 2024-02-23T00:52:38.477 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [ 2024-02-23T00:52:38.477 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.477 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24811.0:10 19.4 19:2094d267:test-rados-api-smithi047-82438-5::foo:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected e111)", 2024-02-23T00:52:38.478 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:00.849994+0000", 2024-02-23T00:52:38.478 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 577.62249535800004, 2024-02-23T00:52:38.478 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.13391424800000001, 2024-02-23T00:52:38.478 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:52:38.478 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:52:38.478 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:52:38.478 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24811", 2024-02-23T00:52:38.478 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/3614542636", 2024-02-23T00:52:38.478 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 10 2024-02-23T00:52:38.478 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.478 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:52:38.478 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.478 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:52:38.478 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.849994+0000", 2024-02-23T00:52:38.478 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:52:38.478 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.479 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.479 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:52:38.479 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.849994+0000", 2024-02-23T00:52:38.479 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:52:38.479 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.479 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.479 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:52:38.479 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.849992+0000", 2024-02-23T00:52:38.479 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1450999999999999e-05 2024-02-23T00:52:38.479 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.479 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.479 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:52:38.479 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.850004+0000", 2024-02-23T00:52:38.479 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3368e-05 2024-02-23T00:52:38.480 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.480 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.480 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:52:38.480 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.850017+0000", 2024-02-23T00:52:38.480 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0490000000000004e-06 2024-02-23T00:52:38.480 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.480 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.480 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:52:38.480 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.850025+0000", 2024-02-23T00:52:38.480 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.7912999999999999e-05 2024-02-23T00:52:38.480 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.480 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.480 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:52:38.480 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.850083+0000", 2024-02-23T00:52:38.481 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000110516 2024-02-23T00:52:38.481 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.481 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.481 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:52:38.481 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.850193+0000", 2024-02-23T00:52:38.481 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.133715062 2024-02-23T00:52:38.481 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.481 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.481 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:52:38.481 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.983908+0000", 2024-02-23T00:52:38.481 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.13391424800000001 2024-02-23T00:52:38.481 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.481 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:52:38.481 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.481 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.481 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.482 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(osd.6.0:245 20.12 e112/109 -22:4d27b8a4:::temp_20.12_0_24277_1:head v 0'0, mlcod=0'0)", 2024-02-23T00:52:38.482 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:01.452136+0000", 2024-02-23T00:52:38.482 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 577.02035333699996, 2024-02-23T00:52:38.482 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.033887135999999998, 2024-02-23T00:52:38.482 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:52:38.482 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:52:38.482 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:52:38.482 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.482 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:52:38.482 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.452136+0000", 2024-02-23T00:52:38.482 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:52:38.482 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.482 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.482 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:52:38.482 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.452136+0000", 2024-02-23T00:52:38.482 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:52:38.483 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.483 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.483 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:52:38.483 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.452134+0000", 2024-02-23T00:52:38.483 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018909622000000001 2024-02-23T00:52:38.483 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.483 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.483 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:52:38.483 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.471044+0000", 2024-02-23T00:52:38.483 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.2262000000000001e-05 2024-02-23T00:52:38.483 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.483 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.483 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:52:38.483 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.471066+0000", 2024-02-23T00:52:38.484 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0569999999999993e-06 2024-02-23T00:52:38.484 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.484 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.484 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:52:38.484 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.471074+0000", 2024-02-23T00:52:38.484 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.4327999999999999e-05 2024-02-23T00:52:38.484 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.484 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.484 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:52:38.484 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.471128+0000", 2024-02-23T00:52:38.484 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.9809e-05 2024-02-23T00:52:38.484 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.485 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.485 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:52:38.485 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.471148+0000", 2024-02-23T00:52:38.485 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.014816957 2024-02-23T00:52:38.485 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.485 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.485 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:52:38.485 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.485965+0000", 2024-02-23T00:52:38.485 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.8130000000000001e-05 2024-02-23T00:52:38.485 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.485 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.485 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:52:38.485 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.486023+0000", 2024-02-23T00:52:38.485 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.033887135999999998 2024-02-23T00:52:38.485 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.486 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:52:38.486 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.486 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.486 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.486 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24775.0:2070 20.6 20:63faae7e:test-rados-api-smithi047-82457-17::big2:head [writefull 0~25165828] snapc 0=[] ondisk+write+known_if_redirected e112)", 2024-02-23T00:52:38.486 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:01.792251+0000", 2024-02-23T00:52:38.486 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 576.68023843100002, 2024-02-23T00:52:38.486 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.10134280900000001, 2024-02-23T00:52:38.486 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:52:38.486 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:52:38.486 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:52:38.486 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24775", 2024-02-23T00:52:38.486 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1191937413", 2024-02-23T00:52:38.486 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2070 2024-02-23T00:52:38.486 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.487 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:52:38.487 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.487 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:52:38.487 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.792251+0000", 2024-02-23T00:52:38.487 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:52:38.487 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.487 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.487 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:52:38.487 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.792251+0000", 2024-02-23T00:52:38.487 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999962 2024-02-23T00:52:38.487 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.487 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.487 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:52:38.487 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.792247+0000", 2024-02-23T00:52:38.488 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.014557680999999999 2024-02-23T00:52:38.488 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.488 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.488 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:52:38.488 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.806805+0000", 2024-02-23T00:52:38.488 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.7353999999999999e-05 2024-02-23T00:52:38.488 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.488 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.488 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:52:38.488 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.806832+0000", 2024-02-23T00:52:38.488 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1474000000000001e-05 2024-02-23T00:52:38.489 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.489 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.489 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:52:38.489 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.806844+0000", 2024-02-23T00:52:38.489 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.9030000000000002e-05 2024-02-23T00:52:38.489 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.489 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.489 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:52:38.489 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.806903+0000", 2024-02-23T00:52:38.489 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014977100000000001 2024-02-23T00:52:38.489 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.489 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.489 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:52:38.489 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.807053+0000", 2024-02-23T00:52:38.489 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00017362000000000001 2024-02-23T00:52:38.489 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.490 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.490 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 3,7", 2024-02-23T00:52:38.490 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.807226+0000", 2024-02-23T00:52:38.490 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.052890589000000002 2024-02-23T00:52:38.490 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.490 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.490 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:52:38.490 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.860117+0000", 2024-02-23T00:52:38.490 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0059036360000000003 2024-02-23T00:52:38.490 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.490 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.490 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:52:38.490 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.866021+0000", 2024-02-23T00:52:38.491 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.027494049 2024-02-23T00:52:38.491 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.491 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.491 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:52:38.491 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.893515+0000", 2024-02-23T00:52:38.491 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.4363999999999998e-05 2024-02-23T00:52:38.491 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.491 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.491 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:52:38.491 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.893549+0000", 2024-02-23T00:52:38.491 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.4903e-05 2024-02-23T00:52:38.491 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.491 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.491 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:52:38.491 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.893594+0000", 2024-02-23T00:52:38.492 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.10134280900000001 2024-02-23T00:52:38.492 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.492 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:52:38.492 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.492 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.492 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.492 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24403.0:216 3.0 3:353ec8f9:::gateway.conf:head [getxattr epoch (1) in=5b] snapc 0=[] ondisk+read+known_if_redirected e113)", 2024-02-23T00:52:38.492 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:04.226432+0000", 2024-02-23T00:52:38.492 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 574.24605798000005, 2024-02-23T00:52:38.492 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.027281850999999999, 2024-02-23T00:52:38.492 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:52:38.492 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:52:38.493 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:52:38.493 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24403", 2024-02-23T00:52:38.493 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.196:0/519523863", 2024-02-23T00:52:38.493 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 216 2024-02-23T00:52:38.493 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.493 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:52:38.493 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.493 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:52:38.493 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.226432+0000", 2024-02-23T00:52:38.494 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:52:38.494 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.494 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.494 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:52:38.494 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.226432+0000", 2024-02-23T00:52:38.494 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999976 2024-02-23T00:52:38.494 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.494 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.494 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:52:38.494 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.226429+0000", 2024-02-23T00:52:38.494 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3016e-05 2024-02-23T00:52:38.494 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.494 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.495 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:52:38.495 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.226442+0000", 2024-02-23T00:52:38.495 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0083690269999999994 2024-02-23T00:52:38.495 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.495 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.495 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:52:38.495 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.234811+0000", 2024-02-23T00:52:38.495 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.7221999999999999e-05 2024-02-23T00:52:38.495 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.495 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.496 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:52:38.496 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.234828+0000", 2024-02-23T00:52:38.496 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018723701999999998 2024-02-23T00:52:38.496 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.496 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.496 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:52:38.496 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.253552+0000", 2024-02-23T00:52:38.496 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.2403000000000004e-05 2024-02-23T00:52:38.496 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.496 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.497 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:52:38.497 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.253634+0000", 2024-02-23T00:52:38.497 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8904000000000002e-05 2024-02-23T00:52:38.497 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.497 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.497 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:52:38.497 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.253713+0000", 2024-02-23T00:52:38.497 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.027281850999999999 2024-02-23T00:52:38.497 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.497 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:52:38.497 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.497 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.497 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.497 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.15144.0:3 38.f 38:f4b50b61:::1.obj:head [write 0~198] snapc 0=[] ondisk+write+known_if_redirected e113)", 2024-02-23T00:52:38.498 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:04.331121+0000", 2024-02-23T00:52:38.498 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 574.14136898799995, 2024-02-23T00:52:38.498 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.024663518999999998, 2024-02-23T00:52:38.498 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:52:38.498 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:52:38.498 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:52:38.498 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.15144", 2024-02-23T00:52:38.498 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/2599587504", 2024-02-23T00:52:38.498 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 3 2024-02-23T00:52:38.498 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.498 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:52:38.499 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.499 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:52:38.499 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.331121+0000", 2024-02-23T00:52:38.499 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:52:38.499 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.499 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.499 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:52:38.499 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.331121+0000", 2024-02-23T00:52:38.499 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:52:38.499 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.499 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.499 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:52:38.499 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.331119+0000", 2024-02-23T00:52:38.500 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.1100000000000003e-06 2024-02-23T00:52:38.500 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.500 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.500 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:52:38.500 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.331127+0000", 2024-02-23T00:52:38.500 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6103000000000002e-05 2024-02-23T00:52:38.500 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.500 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.500 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:52:38.500 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.331143+0000", 2024-02-23T00:52:38.500 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0609999999999996e-06 2024-02-23T00:52:38.500 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.500 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.501 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:52:38.501 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.331151+0000", 2024-02-23T00:52:38.501 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.4344000000000004e-05 2024-02-23T00:52:38.501 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.501 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.501 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:52:38.501 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.331235+0000", 2024-02-23T00:52:38.501 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000134793 2024-02-23T00:52:38.501 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.501 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.501 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:52:38.501 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.331370+0000", 2024-02-23T00:52:38.501 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00015517100000000001 2024-02-23T00:52:38.501 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.501 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.502 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 0,5", 2024-02-23T00:52:38.502 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.331525+0000", 2024-02-23T00:52:38.502 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00081146600000000005 2024-02-23T00:52:38.502 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.502 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.502 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:52:38.502 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.332337+0000", 2024-02-23T00:52:38.502 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0019837330000000001 2024-02-23T00:52:38.502 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.502 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.502 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:52:38.502 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.334320+0000", 2024-02-23T00:52:38.502 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021412520000000001 2024-02-23T00:52:38.502 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.503 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.503 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:52:38.503 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.355733+0000", 2024-02-23T00:52:38.503 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0108999999999999e-05 2024-02-23T00:52:38.503 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.503 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.503 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:52:38.503 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.355763+0000", 2024-02-23T00:52:38.503 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1069000000000001e-05 2024-02-23T00:52:38.503 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.503 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.503 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:52:38.503 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.355784+0000", 2024-02-23T00:52:38.503 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.024663518999999998 2024-02-23T00:52:38.503 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.504 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:52:38.504 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.504 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.504 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.504 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.25529.0:16 63.0 63:0725c07e:test-rados-api-smithi047-82496-9::foo:head [write 128~128] snapc 7=[7,6] ondisk+write+known_if_redirected e136)", 2024-02-23T00:52:38.504 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:27.581440+0000", 2024-02-23T00:52:38.504 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 550.89104991800002, 2024-02-23T00:52:38.504 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.031491658999999998, 2024-02-23T00:52:38.504 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:52:38.504 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:52:38.504 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:52:38.504 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.25529", 2024-02-23T00:52:38.505 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/2064770043", 2024-02-23T00:52:38.505 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 16 2024-02-23T00:52:38.505 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.505 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:52:38.505 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.505 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:52:38.505 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:27.581440+0000", 2024-02-23T00:52:38.505 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:52:38.505 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.505 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.505 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:52:38.505 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:27.581440+0000", 2024-02-23T00:52:38.505 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999962 2024-02-23T00:52:38.505 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.506 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.506 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:52:38.506 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:27.581436+0000", 2024-02-23T00:52:38.506 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.329e-06 2024-02-23T00:52:38.506 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.506 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.506 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:52:38.506 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:27.581445+0000", 2024-02-23T00:52:38.506 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6963999999999999e-05 2024-02-23T00:52:38.506 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.506 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.506 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:52:38.506 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:27.581462+0000", 2024-02-23T00:52:38.506 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.9680000000000006e-06 2024-02-23T00:52:38.507 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.507 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.507 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:52:38.507 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:27.581470+0000", 2024-02-23T00:52:38.507 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010562578 2024-02-23T00:52:38.507 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.507 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.507 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:52:38.507 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:27.592033+0000", 2024-02-23T00:52:38.507 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.9858000000000005e-05 2024-02-23T00:52:38.507 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.507 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.508 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:52:38.508 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:27.592133+0000", 2024-02-23T00:52:38.508 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00022296100000000001 2024-02-23T00:52:38.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.508 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.508 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 0,6", 2024-02-23T00:52:38.508 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:27.592356+0000", 2024-02-23T00:52:38.508 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.005479556 2024-02-23T00:52:38.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.508 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.508 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:52:38.508 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:27.597835+0000", 2024-02-23T00:52:38.508 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012266022 2024-02-23T00:52:38.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.509 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.509 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:52:38.509 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:27.610101+0000", 2024-02-23T00:52:38.509 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00274479 2024-02-23T00:52:38.509 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.509 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.509 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:52:38.509 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:27.612846+0000", 2024-02-23T00:52:38.509 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.5865999999999998e-05 2024-02-23T00:52:38.509 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.509 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.509 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:52:38.509 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:27.612892+0000", 2024-02-23T00:52:38.509 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.9390000000000001e-05 2024-02-23T00:52:38.509 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.510 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.510 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:52:38.510 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:27.612931+0000", 2024-02-23T00:52:38.510 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.031491658999999998 2024-02-23T00:52:38.510 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.510 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:52:38.511 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.511 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.511 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.511 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.25529.0:28 63.e 63:7037fd60:test-rados-api-smithi047-82496-10::foo:head [write 0~128] snapc a=[a] ondisk+write+known_if_redirected e139)", 2024-02-23T00:52:38.511 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:31.196152+0000", 2024-02-23T00:52:38.511 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 547.276337506, 2024-02-23T00:52:38.511 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.035570335000000002, 2024-02-23T00:52:38.511 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:52:38.511 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:52:38.511 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:52:38.511 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.25529", 2024-02-23T00:52:38.512 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/2064770043", 2024-02-23T00:52:38.512 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 28 2024-02-23T00:52:38.512 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.512 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:52:38.512 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.512 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:52:38.512 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.196152+0000", 2024-02-23T00:52:38.512 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:52:38.512 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.512 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.512 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:52:38.512 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.196152+0000", 2024-02-23T00:52:38.513 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:52:38.513 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.513 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.513 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:52:38.513 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.196150+0000", 2024-02-23T00:52:38.513 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.6059999999999999e-06 2024-02-23T00:52:38.513 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.513 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.513 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:52:38.513 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.196157+0000", 2024-02-23T00:52:38.513 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4922e-05 2024-02-23T00:52:38.513 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.513 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.513 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:52:38.513 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.196172+0000", 2024-02-23T00:52:38.514 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.1940000000000001e-06 2024-02-23T00:52:38.514 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.514 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.514 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:52:38.514 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.196179+0000", 2024-02-23T00:52:38.514 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0038786519999999998 2024-02-23T00:52:38.514 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.514 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.514 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:52:38.514 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.200058+0000", 2024-02-23T00:52:38.514 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00015678799999999999 2024-02-23T00:52:38.514 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.514 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.514 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:52:38.515 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.200214+0000", 2024-02-23T00:52:38.515 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00017207999999999999 2024-02-23T00:52:38.515 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.515 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.515 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 2,3", 2024-02-23T00:52:38.515 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.200386+0000", 2024-02-23T00:52:38.515 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0032031099999999999 2024-02-23T00:52:38.515 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.515 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.515 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:52:38.515 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.203590+0000", 2024-02-23T00:52:38.515 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.011119851 2024-02-23T00:52:38.515 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.515 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.516 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:52:38.516 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.214709+0000", 2024-02-23T00:52:38.516 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.01692623 2024-02-23T00:52:38.516 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.516 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.516 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:52:38.516 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.231636+0000", 2024-02-23T00:52:38.516 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.6619000000000002e-05 2024-02-23T00:52:38.516 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.516 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.516 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:52:38.516 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.231682+0000", 2024-02-23T00:52:38.516 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.0085e-05 2024-02-23T00:52:38.516 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.517 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.517 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:52:38.517 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:31.231722+0000", 2024-02-23T00:52:38.517 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.035570335000000002 2024-02-23T00:52:38.517 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.517 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:52:38.517 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.517 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.517 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.517 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.25529.0:35 63.e 63:7037fd60:test-rados-api-smithi047-82496-10::foo:head [write 128~128] snapc b=[b,a] ondisk+write+known_if_redirected e140)", 2024-02-23T00:52:38.517 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:32.199273+0000", 2024-02-23T00:52:38.517 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 546.27321675200005, 2024-02-23T00:52:38.517 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.106436984, 2024-02-23T00:52:38.517 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:52:38.518 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:52:38.518 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:52:38.518 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.25529", 2024-02-23T00:52:38.518 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/2064770043", 2024-02-23T00:52:38.518 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 35 2024-02-23T00:52:38.518 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.518 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:52:38.518 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.518 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:52:38.519 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.199273+0000", 2024-02-23T00:52:38.519 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:52:38.519 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.519 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.519 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:52:38.519 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.199273+0000", 2024-02-23T00:52:38.520 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999967 2024-02-23T00:52:38.520 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.520 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.520 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:52:38.520 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.199270+0000", 2024-02-23T00:52:38.520 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4188999999999999e-05 2024-02-23T00:52:38.521 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.521 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.521 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:52:38.521 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.199284+0000", 2024-02-23T00:52:38.521 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.9290999999999999e-05 2024-02-23T00:52:38.521 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.521 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.521 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:52:38.521 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.199303+0000", 2024-02-23T00:52:38.521 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.6670000000000008e-06 2024-02-23T00:52:38.521 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.522 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.522 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:52:38.522 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.199313+0000", 2024-02-23T00:52:38.522 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0039358450000000003 2024-02-23T00:52:38.522 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.522 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.522 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:52:38.522 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.203249+0000", 2024-02-23T00:52:38.522 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.1759999999999999e-06 2024-02-23T00:52:38.522 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.523 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.523 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-02-23T00:52:38.523 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.203258+0000", 2024-02-23T00:52:38.523 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.057894644000000002 2024-02-23T00:52:38.523 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.523 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.523 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:52:38.523 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.261152+0000", 2024-02-23T00:52:38.524 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00015342099999999999 2024-02-23T00:52:38.524 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.524 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.524 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:52:38.524 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.261306+0000", 2024-02-23T00:52:38.524 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00030064399999999998 2024-02-23T00:52:38.524 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.524 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.524 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 2,3", 2024-02-23T00:52:38.524 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.261606+0000", 2024-02-23T00:52:38.524 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.031497770000000001 2024-02-23T00:52:38.524 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.524 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.525 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:52:38.525 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.293104+0000", 2024-02-23T00:52:38.525 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00070708999999999995 2024-02-23T00:52:38.525 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.525 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.525 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:52:38.525 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.293811+0000", 2024-02-23T00:52:38.525 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.011786562 2024-02-23T00:52:38.525 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.525 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.525 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:52:38.525 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.305598+0000", 2024-02-23T00:52:38.525 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.5371999999999998e-05 2024-02-23T00:52:38.525 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.525 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.526 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:52:38.526 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.305653+0000", 2024-02-23T00:52:38.526 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.6471999999999997e-05 2024-02-23T00:52:38.526 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.526 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.526 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:52:38.526 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.305710+0000", 2024-02-23T00:52:38.526 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.106436984 2024-02-23T00:52:38.526 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.526 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:52:38.526 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.526 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.526 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.526 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWrite(79.2s1 140/139 ECSubWrite(tid=341, reqid=client.31603.0:1, at_version=140'1, trim_to=0'0, roll_forward_to=0'0))", 2024-02-23T00:52:38.527 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:32.291530+0000", 2024-02-23T00:52:38.527 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 546.18095976899997, 2024-02-23T00:52:38.527 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.022260156, 2024-02-23T00:52:38.527 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:52:38.527 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-02-23T00:52:38.527 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:52:38.527 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.527 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:52:38.527 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.291530+0000", 2024-02-23T00:52:38.527 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:52:38.527 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.527 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.527 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:52:38.527 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.291530+0000", 2024-02-23T00:52:38.527 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:52:38.528 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.528 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.528 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:52:38.528 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.291529+0000", 2024-02-23T00:52:38.528 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.4465999999999998e-05 2024-02-23T00:52:38.528 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.528 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.528 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:52:38.528 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.291564+0000", 2024-02-23T00:52:38.528 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.7619999999999998e-06 2024-02-23T00:52:38.528 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.528 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.528 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:52:38.529 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.291571+0000", 2024-02-23T00:52:38.529 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.585e-06 2024-02-23T00:52:38.529 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.529 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.529 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:52:38.529 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.291579+0000", 2024-02-23T00:52:38.529 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00090052800000000005 2024-02-23T00:52:38.529 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.529 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.529 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:52:38.529 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.292480+0000", 2024-02-23T00:52:38.529 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2649999999999999e-05 2024-02-23T00:52:38.530 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.530 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.530 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-02-23T00:52:38.530 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.292492+0000", 2024-02-23T00:52:38.530 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021268694000000001 2024-02-23T00:52:38.530 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.530 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.531 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-02-23T00:52:38.531 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.313761+0000", 2024-02-23T00:52:38.531 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8813999999999999e-05 2024-02-23T00:52:38.531 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.531 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.532 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:52:38.532 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.313790+0000", 2024-02-23T00:52:38.532 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.022260156 2024-02-23T00:52:38.532 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.532 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:52:38.532 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.533 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.533 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.533 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.25529.0:38 63.e 63:7037fd60:test-rados-api-smithi047-82496-10::foo:head [write 896~128] snapc b=[b,a] ondisk+write+known_if_redirected e140)", 2024-02-23T00:52:38.533 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:32.320224+0000", 2024-02-23T00:52:38.533 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 546.15226602400003, 2024-02-23T00:52:38.534 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029558615999999999, 2024-02-23T00:52:38.534 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:52:38.534 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:52:38.534 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:52:38.534 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.25529", 2024-02-23T00:52:38.534 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/2064770043", 2024-02-23T00:52:38.535 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 38 2024-02-23T00:52:38.535 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.535 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:52:38.535 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.535 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:52:38.536 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.320224+0000", 2024-02-23T00:52:38.536 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:52:38.536 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.536 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.536 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:52:38.536 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.320224+0000", 2024-02-23T00:52:38.537 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999986 2024-02-23T00:52:38.537 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.537 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.537 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:52:38.537 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.320222+0000", 2024-02-23T00:52:38.537 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0709999999999995e-06 2024-02-23T00:52:38.538 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.538 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.538 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:52:38.538 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.320230+0000", 2024-02-23T00:52:38.538 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1715999999999999e-05 2024-02-23T00:52:38.538 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.538 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.539 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:52:38.539 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.320242+0000", 2024-02-23T00:52:38.539 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.1199999999999999e-06 2024-02-23T00:52:38.539 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.539 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.540 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:52:38.540 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.320248+0000", 2024-02-23T00:52:38.540 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.4313999999999997e-05 2024-02-23T00:52:38.540 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.540 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.541 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:52:38.541 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.320282+0000", 2024-02-23T00:52:38.541 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.2701999999999997e-05 2024-02-23T00:52:38.541 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.541 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.541 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:52:38.542 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.320345+0000", 2024-02-23T00:52:38.542 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012842100000000001 2024-02-23T00:52:38.542 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.542 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.542 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 2,3", 2024-02-23T00:52:38.542 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.320474+0000", 2024-02-23T00:52:38.543 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.003900073 2024-02-23T00:52:38.543 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.543 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.543 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:52:38.543 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.324374+0000", 2024-02-23T00:52:38.544 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.011614757999999999 2024-02-23T00:52:38.544 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.544 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.544 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:52:38.544 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.335988+0000", 2024-02-23T00:52:38.544 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.013680285 2024-02-23T00:52:38.545 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.545 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.545 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:52:38.545 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.349669+0000", 2024-02-23T00:52:38.545 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.4411000000000004e-05 2024-02-23T00:52:38.546 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.546 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.546 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:52:38.546 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.349733+0000", 2024-02-23T00:52:38.546 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.9004e-05 2024-02-23T00:52:38.547 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.547 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.547 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:52:38.547 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.349782+0000", 2024-02-23T00:52:38.547 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029558615999999999 2024-02-23T00:52:38.547 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.548 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:52:38.548 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.548 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.548 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.548 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24775.0:2079 20.6 20:61acad7d:test-rados-api-smithi047-82457-17::big2.copy:head [copy-from ver 0] snapc 0=[] ondisk+write+known_if_redirected e139)", 2024-02-23T00:52:38.549 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:32.367186+0000", 2024-02-23T00:52:38.549 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 546.10530351, 2024-02-23T00:52:38.549 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.13614552299999999, 2024-02-23T00:52:38.549 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:52:38.549 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:52:38.549 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:52:38.550 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24775", 2024-02-23T00:52:38.550 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1191937413", 2024-02-23T00:52:38.550 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2079 2024-02-23T00:52:38.550 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.550 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:52:38.550 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.551 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:52:38.551 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.367186+0000", 2024-02-23T00:52:38.551 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:52:38.551 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.551 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.551 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:52:38.552 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.367186+0000", 2024-02-23T00:52:38.552 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999986 2024-02-23T00:52:38.552 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.552 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.552 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:52:38.553 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.367185+0000", 2024-02-23T00:52:38.553 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.5270000000000004e-06 2024-02-23T00:52:38.553 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.553 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.553 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:52:38.553 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.367194+0000", 2024-02-23T00:52:38.554 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.277e-05 2024-02-23T00:52:38.554 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.554 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.554 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:52:38.554 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.367207+0000", 2024-02-23T00:52:38.554 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.579e-06 2024-02-23T00:52:38.555 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.555 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.555 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:52:38.555 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.367213+0000", 2024-02-23T00:52:38.555 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000202981 2024-02-23T00:52:38.555 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.556 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.556 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:52:38.556 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.367416+0000", 2024-02-23T00:52:38.556 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012661399999999999 2024-02-23T00:52:38.556 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.557 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.557 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:52:38.557 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.367543+0000", 2024-02-23T00:52:38.557 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.11779671899999999 2024-02-23T00:52:38.557 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.557 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.558 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 3,7", 2024-02-23T00:52:38.558 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.485340+0000", 2024-02-23T00:52:38.558 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012007673999999999 2024-02-23T00:52:38.558 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.558 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.558 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:52:38.559 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.497347+0000", 2024-02-23T00:52:38.559 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00144239 2024-02-23T00:52:38.559 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.559 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.559 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:52:38.559 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.498790+0000", 2024-02-23T00:52:38.560 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0044293090000000002 2024-02-23T00:52:38.560 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.560 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.560 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:52:38.560 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.503219+0000", 2024-02-23T00:52:38.561 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.0754000000000001e-05 2024-02-23T00:52:38.561 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.561 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.561 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:52:38.561 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.503280+0000", 2024-02-23T00:52:38.561 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.1668000000000001e-05 2024-02-23T00:52:38.562 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.562 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.562 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:52:38.562 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.503332+0000", 2024-02-23T00:52:38.562 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.13614552299999999 2024-02-23T00:52:38.562 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.562 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:52:38.563 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.563 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.563 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.563 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24775.0:2080 20.5 20:a70ec860:test-rados-api-smithi047-82457-17::big3.copy:head [copy-from ver 0] snapc 0=[] ondisk+write+known_if_redirected e140)", 2024-02-23T00:52:38.563 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:32.508611+0000", 2024-02-23T00:52:38.564 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 545.96387840199998, 2024-02-23T00:52:38.564 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.18482968499999999, 2024-02-23T00:52:38.564 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:52:38.564 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:52:38.564 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:52:38.564 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24775", 2024-02-23T00:52:38.565 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1191937413", 2024-02-23T00:52:38.565 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2080 2024-02-23T00:52:38.565 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.565 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:52:38.565 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.565 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:52:38.566 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.508611+0000", 2024-02-23T00:52:38.566 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:52:38.566 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.566 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.566 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:52:38.566 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.508611+0000", 2024-02-23T00:52:38.566 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999986 2024-02-23T00:52:38.567 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.567 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.567 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:52:38.567 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.508610+0000", 2024-02-23T00:52:38.567 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.0729999999999994e-06 2024-02-23T00:52:38.567 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.568 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.568 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:52:38.568 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.508619+0000", 2024-02-23T00:52:38.568 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6867e-05 2024-02-23T00:52:38.568 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.568 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.568 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:52:38.569 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.508636+0000", 2024-02-23T00:52:38.569 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.9800000000000001e-06 2024-02-23T00:52:38.569 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.569 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.569 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:52:38.569 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.508642+0000", 2024-02-23T00:52:38.569 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.8140000000000003e-05 2024-02-23T00:52:38.570 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.570 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.570 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:52:38.570 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.508691+0000", 2024-02-23T00:52:38.570 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014609599999999999 2024-02-23T00:52:38.570 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.571 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.571 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:52:38.571 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.508837+0000", 2024-02-23T00:52:38.571 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.13144261099999999 2024-02-23T00:52:38.571 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.571 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.571 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 2,5", 2024-02-23T00:52:38.571 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.640279+0000", 2024-02-23T00:52:38.571 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0094432939999999996 2024-02-23T00:52:38.571 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.571 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.571 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:52:38.571 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.649723+0000", 2024-02-23T00:52:38.572 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.027088807999999999 2024-02-23T00:52:38.572 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.572 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.572 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:52:38.572 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.676811+0000", 2024-02-23T00:52:38.572 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.016523389999999999 2024-02-23T00:52:38.572 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.572 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.572 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:52:38.572 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.693335+0000", 2024-02-23T00:52:38.572 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.1526999999999996e-05 2024-02-23T00:52:38.572 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.572 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.572 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:52:38.572 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.693396+0000", 2024-02-23T00:52:38.573 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.4455e-05 2024-02-23T00:52:38.573 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.573 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.573 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:52:38.573 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.693441+0000", 2024-02-23T00:52:38.573 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.18482968499999999 2024-02-23T00:52:38.573 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.573 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:52:38.573 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.573 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.573 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.573 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.25529.0:42 63.e 63:7037fd60:test-rados-api-smithi047-82496-10::foo:head [write 128~128] snapc c=[c,b,a] ondisk+write+known_if_redirected e142)", 2024-02-23T00:52:38.573 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:34.496096+0000", 2024-02-23T00:52:38.573 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 543.976393336, 2024-02-23T00:52:38.574 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.061437618999999999, 2024-02-23T00:52:38.574 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:52:38.574 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:52:38.574 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:52:38.574 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.25529", 2024-02-23T00:52:38.574 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/2064770043", 2024-02-23T00:52:38.574 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 42 2024-02-23T00:52:38.574 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.574 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:52:38.574 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.574 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:52:38.574 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.496096+0000", 2024-02-23T00:52:38.574 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:52:38.574 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.575 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.575 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:52:38.575 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.496096+0000", 2024-02-23T00:52:38.575 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999986 2024-02-23T00:52:38.575 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.575 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.575 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:52:38.575 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.496095+0000", 2024-02-23T00:52:38.575 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.0389999999999999e-06 2024-02-23T00:52:38.575 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.575 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.575 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:52:38.575 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.496102+0000", 2024-02-23T00:52:38.575 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2e-05 2024-02-23T00:52:38.575 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.576 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.576 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:52:38.576 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.496114+0000", 2024-02-23T00:52:38.576 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0590000000000003e-06 2024-02-23T00:52:38.576 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.576 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.576 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:52:38.576 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.496122+0000", 2024-02-23T00:52:38.576 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0080011270000000002 2024-02-23T00:52:38.576 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.576 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.576 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:52:38.576 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.504123+0000", 2024-02-23T00:52:38.577 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0022672809999999999 2024-02-23T00:52:38.577 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.577 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.577 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:52:38.577 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.506390+0000", 2024-02-23T00:52:38.577 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00021441099999999999 2024-02-23T00:52:38.577 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.577 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.577 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 2,3", 2024-02-23T00:52:38.577 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.506605+0000", 2024-02-23T00:52:38.577 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0067776329999999999 2024-02-23T00:52:38.577 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.577 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.577 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:52:38.577 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.513382+0000", 2024-02-23T00:52:38.578 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.036179537999999997 2024-02-23T00:52:38.578 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.578 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.578 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:52:38.578 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.549562+0000", 2024-02-23T00:52:38.578 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0078643380000000002 2024-02-23T00:52:38.578 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.578 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.578 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:52:38.578 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.557426+0000", 2024-02-23T00:52:38.578 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.9908999999999998e-05 2024-02-23T00:52:38.578 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.578 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.578 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:52:38.579 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.557486+0000", 2024-02-23T00:52:38.579 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.7828999999999999e-05 2024-02-23T00:52:38.579 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.579 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.579 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:52:38.579 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.557534+0000", 2024-02-23T00:52:38.579 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.061437618999999999 2024-02-23T00:52:38.579 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.579 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:52:38.579 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.579 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.579 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.579 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(client.34321.0:2 83.6 e143/142 83:602f83fe:::foo:head v 143'1, mlcod=0'0)", 2024-02-23T00:52:38.579 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:35.131473+0000", 2024-02-23T00:52:38.579 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 543.34101631500005, 2024-02-23T00:52:38.580 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.025342482, 2024-02-23T00:52:38.580 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:52:38.580 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:52:38.580 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:52:38.580 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.580 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:52:38.580 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:35.131473+0000", 2024-02-23T00:52:38.580 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:52:38.580 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.580 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.580 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:52:38.580 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:35.131473+0000", 2024-02-23T00:52:38.580 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:52:38.580 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.581 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.581 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:52:38.581 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:35.131473+0000", 2024-02-23T00:52:38.581 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.3780000000000003e-06 2024-02-23T00:52:38.581 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.581 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.581 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:52:38.581 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:35.131477+0000", 2024-02-23T00:52:38.581 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.25e-06 2024-02-23T00:52:38.581 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.581 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.581 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:52:38.581 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:35.131481+0000", 2024-02-23T00:52:38.581 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.6960000000000002e-06 2024-02-23T00:52:38.581 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.582 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.582 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:52:38.582 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:35.131487+0000", 2024-02-23T00:52:38.582 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012547017000000001 2024-02-23T00:52:38.582 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.582 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.582 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:52:38.582 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:35.144034+0000", 2024-02-23T00:52:38.582 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0068000000000002e-05 2024-02-23T00:52:38.582 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.582 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.582 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:52:38.582 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:35.144054+0000", 2024-02-23T00:52:38.582 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012724839999999999 2024-02-23T00:52:38.582 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.582 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.583 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:52:38.583 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:35.156779+0000", 2024-02-23T00:52:38.583 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.6603000000000002e-05 2024-02-23T00:52:38.583 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.583 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.583 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:52:38.583 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:35.156816+0000", 2024-02-23T00:52:38.583 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.025342482 2024-02-23T00:52:38.583 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.583 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:52:38.583 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.583 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.583 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.583 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.25529.0:51 63.10 63:0b20e63d:test-rados-api-smithi047-82496-11::foo:head [write 0~1048576] snapc 10=[10] ondisk+write+known_if_redirected e146)", 2024-02-23T00:52:38.584 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:38.411127+0000", 2024-02-23T00:52:38.584 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 540.06136230100003, 2024-02-23T00:52:38.584 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.031541215999999997, 2024-02-23T00:52:38.584 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:52:38.584 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:52:38.584 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:52:38.584 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.25529", 2024-02-23T00:52:38.584 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/2064770043", 2024-02-23T00:52:38.584 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 51 2024-02-23T00:52:38.584 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.584 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:52:38.584 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.584 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:52:38.584 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.411127+0000", 2024-02-23T00:52:38.585 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:52:38.585 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.585 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.585 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:52:38.585 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.411127+0000", 2024-02-23T00:52:38.585 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:52:38.585 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.585 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.585 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:52:38.585 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.411125+0000", 2024-02-23T00:52:38.585 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00060817599999999999 2024-02-23T00:52:38.585 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.585 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.585 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:52:38.585 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.411734+0000", 2024-02-23T00:52:38.586 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0353999999999999e-05 2024-02-23T00:52:38.586 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.586 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.586 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:52:38.586 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.411754+0000", 2024-02-23T00:52:38.586 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.5280000000000003e-06 2024-02-23T00:52:38.586 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.586 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.586 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:52:38.586 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.411763+0000", 2024-02-23T00:52:38.586 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0017375159999999999 2024-02-23T00:52:38.586 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.586 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.586 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:52:38.586 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.413500+0000", 2024-02-23T00:52:38.587 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012374400000000001 2024-02-23T00:52:38.587 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.587 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.587 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:52:38.587 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.413624+0000", 2024-02-23T00:52:38.587 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000156458 2024-02-23T00:52:38.587 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.587 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.587 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 0,7", 2024-02-23T00:52:38.587 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.413780+0000", 2024-02-23T00:52:38.587 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0035642120000000002 2024-02-23T00:52:38.587 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.587 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.587 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:52:38.587 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.417344+0000", 2024-02-23T00:52:38.587 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0014813980000000001 2024-02-23T00:52:38.588 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.588 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.588 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:52:38.588 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.418826+0000", 2024-02-23T00:52:38.588 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021229049 2024-02-23T00:52:38.588 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.588 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.588 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:52:38.588 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.440055+0000", 2024-02-23T00:52:38.588 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0025668330000000001 2024-02-23T00:52:38.588 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.588 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.589 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:52:38.589 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.442622+0000", 2024-02-23T00:52:38.589 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.6690999999999997e-05 2024-02-23T00:52:38.589 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.589 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.589 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:52:38.589 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.442668+0000", 2024-02-23T00:52:38.589 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.031541215999999997 2024-02-23T00:52:38.589 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.589 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:52:38.589 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.589 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.589 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.589 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWrite(131.5s1 199/198 ECSubWrite(tid=1455, reqid=client.40652.0:1, at_version=199'1, trim_to=0'0, roll_forward_to=0'0))", 2024-02-23T00:52:38.589 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:31.177786+0000", 2024-02-23T00:52:38.590 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 487.294703927, 2024-02-23T00:52:38.590 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.022289363, 2024-02-23T00:52:38.590 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:52:38.590 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-02-23T00:52:38.590 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:52:38.590 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.590 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:52:38.590 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.177786+0000", 2024-02-23T00:52:38.590 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:52:38.590 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.590 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.590 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:52:38.590 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.177786+0000", 2024-02-23T00:52:38.590 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967296 2024-02-23T00:52:38.590 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.590 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.590 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:52:38.591 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.177785+0000", 2024-02-23T00:52:38.591 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0548e-05 2024-02-23T00:52:38.591 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.591 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.591 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:52:38.591 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.177816+0000", 2024-02-23T00:52:38.591 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.0389999999999993e-06 2024-02-23T00:52:38.591 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.591 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.591 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:52:38.591 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.177825+0000", 2024-02-23T00:52:38.591 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3852999999999999e-05 2024-02-23T00:52:38.591 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.591 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.592 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:52:38.592 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.177839+0000", 2024-02-23T00:52:38.592 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0065961850000000001 2024-02-23T00:52:38.592 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.592 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.592 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:52:38.592 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.184435+0000", 2024-02-23T00:52:38.592 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2191000000000001e-05 2024-02-23T00:52:38.592 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.592 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.592 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-02-23T00:52:38.592 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.184447+0000", 2024-02-23T00:52:38.592 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.01559761 2024-02-23T00:52:38.593 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.593 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.593 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-02-23T00:52:38.593 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.200045+0000", 2024-02-23T00:52:38.593 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0137e-05 2024-02-23T00:52:38.593 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.593 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.593 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:52:38.593 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.200075+0000", 2024-02-23T00:52:38.593 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.022289363 2024-02-23T00:52:38.593 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.593 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:52:38.593 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.593 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.593 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.594 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(osd.6.0:446 62.1a e198/126 62:5b3390c0:test-rados-api-smithi047-82465-14::foo25:4 v 198'22, mlcod=159'16)", 2024-02-23T00:52:38.594 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:31.179336+0000", 2024-02-23T00:52:38.594 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 487.29315348, 2024-02-23T00:52:38.594 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.024145261000000001, 2024-02-23T00:52:38.594 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:52:38.594 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:52:38.594 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:52:38.594 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.594 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:52:38.594 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.179336+0000", 2024-02-23T00:52:38.594 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:52:38.594 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.594 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.595 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:52:38.595 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.179336+0000", 2024-02-23T00:52:38.595 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:52:38.595 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.595 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.595 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:52:38.595 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.179336+0000", 2024-02-23T00:52:38.595 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0668e-05 2024-02-23T00:52:38.595 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.595 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.595 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:52:38.595 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.179346+0000", 2024-02-23T00:52:38.595 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.3579999999999999e-06 2024-02-23T00:52:38.595 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.596 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.596 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:52:38.596 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.179353+0000", 2024-02-23T00:52:38.596 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.2529999999999998e-06 2024-02-23T00:52:38.596 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.596 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.596 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:52:38.596 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.179357+0000", 2024-02-23T00:52:38.596 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.011838892 2024-02-23T00:52:38.596 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.596 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.596 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:52:38.596 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.191196+0000", 2024-02-23T00:52:38.596 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.2711999999999999e-05 2024-02-23T00:52:38.597 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.597 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.597 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:52:38.597 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.191219+0000", 2024-02-23T00:52:38.597 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012223862 2024-02-23T00:52:38.597 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.597 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.597 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:52:38.597 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.203442+0000", 2024-02-23T00:52:38.597 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.8853999999999998e-05 2024-02-23T00:52:38.597 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.597 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.597 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:52:38.597 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.203481+0000", 2024-02-23T00:52:38.597 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.024145261000000001 2024-02-23T00:52:38.598 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.598 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:52:38.598 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.598 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.598 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.598 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.41147.0:1 193.1s0 193:90ca3134:::oid_FlushAsyncPP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e339)", 2024-02-23T00:52:38.598 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:46:51.185479+0000", 2024-02-23T00:52:38.598 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 347.28701037899998, 2024-02-23T00:52:38.598 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.013269247, 2024-02-23T00:52:38.598 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:52:38.598 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:52:38.598 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:52:38.598 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.41147", 2024-02-23T00:52:38.598 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1633196565", 2024-02-23T00:52:38.598 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 1 2024-02-23T00:52:38.599 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.599 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:52:38.599 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.599 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:52:38.599 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:51.185479+0000", 2024-02-23T00:52:38.599 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:52:38.599 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.599 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.599 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:52:38.599 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:51.185479+0000", 2024-02-23T00:52:38.599 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:52:38.599 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.599 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.599 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:52:38.600 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:51.185477+0000", 2024-02-23T00:52:38.600 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.5139999999999998e-06 2024-02-23T00:52:38.600 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.600 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.600 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:52:38.600 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:51.185484+0000", 2024-02-23T00:52:38.600 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.9850000000000006e-06 2024-02-23T00:52:38.600 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.600 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.600 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:52:38.600 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:51.185492+0000", 2024-02-23T00:52:38.600 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.4729999999999999e-06 2024-02-23T00:52:38.600 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.600 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.601 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:52:38.601 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:51.185497+0000", 2024-02-23T00:52:38.601 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.3026999999999999e-05 2024-02-23T00:52:38.601 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.601 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.601 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:52:38.601 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:51.185550+0000", 2024-02-23T00:52:38.601 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8739999999999999e-06 2024-02-23T00:52:38.601 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.601 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.601 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:52:38.601 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:51.185553+0000", 2024-02-23T00:52:38.601 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.009254662 2024-02-23T00:52:38.601 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.602 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.602 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:52:38.602 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:52.194808+0000", 2024-02-23T00:52:38.602 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00015087 2024-02-23T00:52:38.602 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.602 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.602 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:52:38.602 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:52.194959+0000", 2024-02-23T00:52:38.602 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00046873999999999999 2024-02-23T00:52:38.602 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.602 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.602 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-02-23T00:52:38.602 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:52.195427+0000", 2024-02-23T00:52:38.602 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000537624 2024-02-23T00:52:38.602 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.603 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.603 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-02-23T00:52:38.603 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:52.195965+0000", 2024-02-23T00:52:38.603 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0027150640000000001 2024-02-23T00:52:38.603 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.603 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.603 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:52:38.603 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:52.198680+0000", 2024-02-23T00:52:38.603 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.8254000000000001e-05 2024-02-23T00:52:38.603 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.603 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.603 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:52:38.603 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:52.198748+0000", 2024-02-23T00:52:38.603 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.013269247 2024-02-23T00:52:38.603 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.603 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:52:38.604 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.604 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.604 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.604 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24403.0:479 3.0 3:353ec8f9:::gateway.conf:head [getxattr epoch (1) in=5b] snapc 0=[] ondisk+read+known_if_redirected e374)", 2024-02-23T00:52:38.604 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:47:27.842443+0000", 2024-02-23T00:52:38.604 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 310.63004670200002, 2024-02-23T00:52:38.604 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.38625488699999999, 2024-02-23T00:52:38.604 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:52:38.604 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:52:38.604 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:52:38.604 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24403", 2024-02-23T00:52:38.604 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.196:0/519523863", 2024-02-23T00:52:38.604 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 479 2024-02-23T00:52:38.604 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.605 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:52:38.605 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.605 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:52:38.605 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:27.842443+0000", 2024-02-23T00:52:38.605 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:52:38.605 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.605 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.605 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:52:38.605 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:27.842443+0000", 2024-02-23T00:52:38.605 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999971 2024-02-23T00:52:38.605 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.605 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.605 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:52:38.605 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:27.842440+0000", 2024-02-23T00:52:38.605 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6144e-05 2024-02-23T00:52:38.605 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.606 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.606 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:52:38.606 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:27.842456+0000", 2024-02-23T00:52:38.606 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.4341e-05 2024-02-23T00:52:38.606 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.606 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.606 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:52:38.606 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:27.842480+0000", 2024-02-23T00:52:38.606 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.0359999999999995e-06 2024-02-23T00:52:38.606 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.606 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.606 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:52:38.606 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:27.842490+0000", 2024-02-23T00:52:38.606 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000117791 2024-02-23T00:52:38.606 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.606 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.606 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:52:38.607 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:27.842607+0000", 2024-02-23T00:52:38.607 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1094000000000001e-05 2024-02-23T00:52:38.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.607 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.607 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:52:38.607 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:27.842618+0000", 2024-02-23T00:52:38.607 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.38582140700000001 2024-02-23T00:52:38.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.607 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.607 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:52:38.607 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:28.228440+0000", 2024-02-23T00:52:38.607 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014037300000000001 2024-02-23T00:52:38.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.607 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.607 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:52:38.607 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:28.228580+0000", 2024-02-23T00:52:38.607 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000117533 2024-02-23T00:52:38.608 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.608 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.608 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:52:38.608 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:28.228698+0000", 2024-02-23T00:52:38.608 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.38625488699999999 2024-02-23T00:52:38.608 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.608 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:52:38.608 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.608 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.608 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.608 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDScrubReserve(2.8 REQUEST e465)", 2024-02-23T00:52:38.608 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:49:37.286792+0000", 2024-02-23T00:52:38.608 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 181.185697374, 2024-02-23T00:52:38.608 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.041402224000000001, 2024-02-23T00:52:38.609 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:52:38.609 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:52:38.609 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:52:38.609 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.609 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:52:38.609 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:49:37.286792+0000", 2024-02-23T00:52:38.609 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:52:38.609 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.609 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.609 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:52:38.609 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:49:37.286792+0000", 2024-02-23T00:52:38.609 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.999999 2024-02-23T00:52:38.609 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.609 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.609 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:52:38.610 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:49:37.286791+0000", 2024-02-23T00:52:38.610 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.7779999999999998e-06 2024-02-23T00:52:38.610 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.610 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.610 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:52:38.610 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:49:37.286796+0000", 2024-02-23T00:52:38.610 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.040995433999999997 2024-02-23T00:52:38.610 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.610 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.610 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:52:38.610 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:49:37.327792+0000", 2024-02-23T00:52:38.610 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.3815000000000002e-05 2024-02-23T00:52:38.610 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.610 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.610 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:52:38.611 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:49:37.327815+0000", 2024-02-23T00:52:38.611 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00032572299999999998 2024-02-23T00:52:38.611 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.611 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.611 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:52:38.611 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:49:37.328141+0000", 2024-02-23T00:52:38.611 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3543999999999999e-05 2024-02-23T00:52:38.611 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.611 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.611 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:52:38.611 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:49:37.328155+0000", 2024-02-23T00:52:38.611 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.9666000000000003e-05 2024-02-23T00:52:38.611 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:52:38.611 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:52:38.611 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:52:38.611 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:49:37.328194+0000", 2024-02-23T00:52:38.611 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.041402224000000001 2024-02-23T00:52:38.612 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.612 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:52:38.612 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.612 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:52:38.612 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:52:38.612 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:52:38.612 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:52:38.613 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:37.360141+0000 mgr.y (mgr.14517) 555 : cluster [DBG] pgmap v940: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 383 B/s wr, 2 op/s 2024-02-23T00:52:38.613 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:37.866006+0000 mon.c (mon.2) 356 : audit [INF] from='client.? 172.21.15.47:0/2164424395' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:52:38.613 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:37.866344+0000 mon.a (mon.0) 3187 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:52:38.614 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:37.360141+0000 mgr.y (mgr.14517) 555 : cluster [DBG] pgmap v940: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 383 B/s wr, 2 op/s 2024-02-23T00:52:38.614 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:37.866006+0000 mon.c (mon.2) 356 : audit [INF] from='client.? 172.21.15.47:0/2164424395' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:52:38.614 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:37.866344+0000 mon.a (mon.0) 3187 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:52:38.643 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:38.682 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:52:38.684 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:38.685 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:52:38.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:37.360141+0000 mgr.y (mgr.14517) 555 : cluster [DBG] pgmap v940: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 383 B/s wr, 2 op/s 2024-02-23T00:52:38.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:37.866006+0000 mon.c (mon.2) 356 : audit [INF] from='client.? 172.21.15.47:0/2164424395' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:52:38.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:37.866344+0000 mon.a (mon.0) 3187 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:52:38.785 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:52:38.815 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:38.914 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:52:38.916 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:38.917 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:52:39.018 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:52:39.047 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:39.066 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:52:39.069 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:39.070 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:52:39.103 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:52:39.170 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:52:39.200 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:39.447 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:52:39.450 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:39.450 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:52:39.481 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:52:39.552 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:52:39.580 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:39.599 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:52:39.614 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:39.615 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:52:39.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:38.423379+0000 mon.a (mon.0) 3188 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:52:39.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:38.427471+0000 mon.a (mon.0) 3189 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:52:39.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:38.427546+0000 mon.a (mon.0) 3190 : cluster [DBG] osdmap e623: 8 total, 8 up, 7 in 2024-02-23T00:52:39.715 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:52:39.740 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:38.423379+0000 mon.a (mon.0) 3188 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:52:39.740 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:38.427471+0000 mon.a (mon.0) 3189 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:52:39.740 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:38.427546+0000 mon.a (mon.0) 3190 : cluster [DBG] osdmap e623: 8 total, 8 up, 7 in 2024-02-23T00:52:39.740 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:38.423379+0000 mon.a (mon.0) 3188 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:52:39.741 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:38.427471+0000 mon.a (mon.0) 3189 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:52:39.741 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:38.427546+0000 mon.a (mon.0) 3190 : cluster [DBG] osdmap e623: 8 total, 8 up, 7 in 2024-02-23T00:52:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:39.941 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:52:39.945 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:39.945 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:52:39.955 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:52:39.966 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.2 dump_ops_in_flight 2024-02-23T00:52:40.046 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:52:40.077 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:40.176 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:52:40.182 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:40.183 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:52:40.283 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:52:40.313 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:40.437 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:52:40.439 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:40.440 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:52:40.540 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:52:40.566 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:52:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:52:40] "GET /metrics HTTP/1.1" 200 190784 "" "Prometheus/2.33.4" 2024-02-23T00:52:40.566 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:39.360678+0000 mgr.y (mgr.14517) 556 : cluster [DBG] pgmap v942: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 383 B/s wr, 2 op/s 2024-02-23T00:52:40.567 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:40.263545+0000 mon.a (mon.0) 3191 : cluster [DBG] osdmap e624: 8 total, 8 up, 7 in 2024-02-23T00:52:40.567 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:39.360678+0000 mgr.y (mgr.14517) 556 : cluster [DBG] pgmap v942: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 383 B/s wr, 2 op/s 2024-02-23T00:52:40.567 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:40.263545+0000 mon.a (mon.0) 3191 : cluster [DBG] osdmap e624: 8 total, 8 up, 7 in 2024-02-23T00:52:40.568 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:39.360678+0000 mgr.y (mgr.14517) 556 : cluster [DBG] pgmap v942: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 383 B/s wr, 2 op/s 2024-02-23T00:52:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:40.263545+0000 mon.a (mon.0) 3191 : cluster [DBG] osdmap e624: 8 total, 8 up, 7 in 2024-02-23T00:52:41.444 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset noscrub 2024-02-23T00:52:41.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:41.076933+0000 osd.3 (osd.3) 300 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:52:41.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:41.077118+0000 osd.3 (osd.3) 301 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:52:41.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:41.308087+0000 mon.a (mon.0) 3192 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:52:41.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:41.076933+0000 osd.3 (osd.3) 300 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:52:41.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:41.077118+0000 osd.3 (osd.3) 301 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:52:41.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:41.308087+0000 mon.a (mon.0) 3192 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:52:41.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:41.076933+0000 osd.3 (osd.3) 300 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:52:41.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:41.077118+0000 osd.3 (osd.3) 301 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:52:41.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:41.308087+0000 mon.a (mon.0) 3192 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:52:42.602 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:41.361060+0000 mgr.y (mgr.14517) 557 : cluster [DBG] pgmap v944: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 737 B/s rd, 0 op/s 2024-02-23T00:52:42.602 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:41.440443+0000 mon.a (mon.0) 3193 : cluster [DBG] osdmap e625: 8 total, 8 up, 7 in 2024-02-23T00:52:42.602 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:41.361060+0000 mgr.y (mgr.14517) 557 : cluster [DBG] pgmap v944: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 737 B/s rd, 0 op/s 2024-02-23T00:52:42.602 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:41.440443+0000 mon.a (mon.0) 3193 : cluster [DBG] osdmap e625: 8 total, 8 up, 7 in 2024-02-23T00:52:42.802 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:52:42.802 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "" 2024-02-23T00:52:42.802 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:52:42.819 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:52:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:52:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:52:42.820 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:41.361060+0000 mgr.y (mgr.14517) 557 : cluster [DBG] pgmap v944: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 737 B/s rd, 0 op/s 2024-02-23T00:52:42.820 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:41.440443+0000 mon.a (mon.0) 3193 : cluster [DBG] osdmap e625: 8 total, 8 up, 7 in 2024-02-23T00:52:43.194 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:52:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:52:43.702 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:43.322314+0000 mon.a (mon.0) 3194 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.9", "id": [7, 3]}]: dispatch 2024-02-23T00:52:43.703 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:43.322314+0000 mon.a (mon.0) 3194 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.9", "id": [7, 3]}]: dispatch 2024-02-23T00:52:43.721 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:52:43.726 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:43.726 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:52:43.827 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:52:43.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:43.322314+0000 mon.a (mon.0) 3194 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.9", "id": [7, 3]}]: dispatch 2024-02-23T00:52:43.859 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:43.880 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:52:43.883 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:43.883 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:52:43.984 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:52:44.016 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:44.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.029+0000 7f1aa91c3700 1 -- 172.21.15.47:0/2624628534 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1aa40f6a90 msgr2=0x7f1aa4104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:44.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.029+0000 7f1aa91c3700 1 --2- 172.21.15.47:0/2624628534 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1aa40f6a90 0x7f1aa4104a20 secure :-1 s=READY pgs=2760 cs=0 l=1 rev1=1 rx=0x7f1a9c009be0 tx=0x7f1a9c004d10).stop 2024-02-23T00:52:44.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.029+0000 7f1aa91c3700 1 -- 172.21.15.47:0/2624628534 shutdown_connections 2024-02-23T00:52:44.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.029+0000 7f1aa91c3700 1 --2- 172.21.15.47:0/2624628534 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1aa40f6a90 0x7f1aa4104a20 unknown :-1 s=CLOSED pgs=2760 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:44.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.029+0000 7f1aa91c3700 1 --2- 172.21.15.47:0/2624628534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1aa40fdbd0 0x7f1aa40fdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:44.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.029+0000 7f1aa91c3700 1 --2- 172.21.15.47:0/2624628534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1aa40fe530 0x7f1aa40f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:44.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.029+0000 7f1aa91c3700 1 -- 172.21.15.47:0/2624628534 >> 172.21.15.47:0/2624628534 conn(0x7f1aa406c370 msgr2=0x7f1aa406c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:52:44.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.029+0000 7f1aa91c3700 1 -- 172.21.15.47:0/2624628534 shutdown_connections 2024-02-23T00:52:44.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.029+0000 7f1aa91c3700 1 -- 172.21.15.47:0/2624628534 wait complete. 2024-02-23T00:52:44.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.030+0000 7f1aa91c3700 1 librados: starting msgr at 2024-02-23T00:52:44.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.030+0000 7f1aa91c3700 1 librados: starting objecter 2024-02-23T00:52:44.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.030+0000 7f1aa91c3700 1 Processor -- start 2024-02-23T00:52:44.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.030+0000 7f1aa91c3700 1 -- start start 2024-02-23T00:52:44.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.030+0000 7f1aa91c3700 1 librados: setting wanted keys 2024-02-23T00:52:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.031+0000 7f1aa91c3700 1 librados: calling monclient init 2024-02-23T00:52:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.031+0000 7f1aa91c3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1aa40f6a90 0x7f1aa4114460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.031+0000 7f1aa91c3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1aa40fdbd0 0x7f1aa4114950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.031+0000 7f1aa91c3700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1aa40fe530 0x7f1aa4101670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.031+0000 7f1aa91c3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa4101b60 con 0x7f1aa40fdbd0 2024-02-23T00:52:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.031+0000 7f1aa91c3700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa4101ca0 con 0x7f1aa40f6a90 2024-02-23T00:52:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.031+0000 7f1aa91c3700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa4101e60 con 0x7f1aa40fe530 2024-02-23T00:52:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.031+0000 7f1aa259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1aa40fdbd0 0x7f1aa4114950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.031+0000 7f1aa259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1aa40fdbd0 0x7f1aa4114950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36372/0 (socket says 172.21.15.47:36372) 2024-02-23T00:52:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.031+0000 7f1aa359e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1aa40fe530 0x7f1aa4101670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.031+0000 7f1aa259c700 1 -- 172.21.15.47:0/1115176460 learned_addr learned my addr 172.21.15.47:0/1115176460 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:52:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.031+0000 7f1aa2d9d700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1aa40f6a90 0x7f1aa4114460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.031+0000 7f1aa259c700 1 -- 172.21.15.47:0/1115176460 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1aa40fe530 msgr2=0x7f1aa4101670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.031+0000 7f1aa259c700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1aa40fe530 0x7f1aa4101670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.031+0000 7f1aa259c700 1 -- 172.21.15.47:0/1115176460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1aa40f6a90 msgr2=0x7f1aa4114460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.031+0000 7f1aa259c700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1aa40f6a90 0x7f1aa4114460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.031+0000 7f1aa259c700 1 -- 172.21.15.47:0/1115176460 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a98005040 con 0x7f1aa40fdbd0 2024-02-23T00:52:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.031+0000 7f1aa359e700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1aa40fe530 0x7f1aa4101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:52:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.032+0000 7f1aa259c700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1aa40fdbd0 0x7f1aa4114950 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 rx=0x7f1a980048f0 tx=0x7f1a98009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.032+0000 7f1a8bfff700 10 client.?.objecter ms_handle_connect 0x7f1aa40fdbd0 2024-02-23T00:52:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.032+0000 7f1a8bfff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:52:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.032+0000 7f1a8bfff700 1 -- 172.21.15.47:0/1115176460 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a98007b00 con 0x7f1aa40fdbd0 2024-02-23T00:52:44.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.032+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a9c0097c0 con 0x7f1aa40fdbd0 2024-02-23T00:52:44.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.032+0000 7f1a8bfff700 1 -- 172.21.15.47:0/1115176460 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f1a9800f040 con 0x7f1aa40fdbd0 2024-02-23T00:52:44.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.032+0000 7f1a8bfff700 1 -- 172.21.15.47:0/1115176460 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a98014820 con 0x7f1aa40fdbd0 2024-02-23T00:52:44.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.032+0000 7f1aa91c3700 10 client.31506.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:52:44.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.032+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa41024f0 con 0x7f1aa40fdbd0 2024-02-23T00:52:44.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.032+0000 7f1aa91c3700 1 librados: init done 2024-02-23T00:52:44.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.033+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f1aa405f970 con 0x7f1aa40fdbd0 2024-02-23T00:52:44.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.034+0000 7f1a8bfff700 1 -- 172.21.15.47:0/1115176460 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f1a98014c50 con 0x7f1aa40fdbd0 2024-02-23T00:52:44.039 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:52:44.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.034+0000 7f1a8bfff700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1a8c062cc0 0x7f1a8c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:44.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.034+0000 7f1a8bfff700 1 -- 172.21.15.47:0/1115176460 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(626..626 src has 482..626) v4 ==== 9607+0+0 (secure 0 0 0) 0x7f1a9807d590 con 0x7f1aa40fdbd0 2024-02-23T00:52:44.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.034+0000 7f1a8bfff700 10 client.31506.objecter ms_dispatch 0x7f1aa4001550 osd_map(626..626 src has 482..626) v4 2024-02-23T00:52:44.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.034+0000 7f1a8bfff700 3 client.31506.objecter handle_osd_map got epochs [626,626] > 0 2024-02-23T00:52:44.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.034+0000 7f1a8bfff700 3 client.31506.objecter handle_osd_map decoding full epoch 626 2024-02-23T00:52:44.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.034+0000 7f1aa2d9d700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1a8c062cc0 0x7f1a8c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:44.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.034+0000 7f1a8bfff700 20 client.31506.objecter dump_active .. 0 homeless 2024-02-23T00:52:44.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.034+0000 7f1a8bfff700 1 -- 172.21.15.47:0/1115176460 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v626) v1 ==== 54+0+16 (secure 0 0 0) 0x7f1a9807dac0 con 0x7f1aa40fdbd0 2024-02-23T00:52:44.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.034+0000 7f1a8bfff700 10 client.31506.objecter ms_handle_connect 0x7f1a8c062cc0 2024-02-23T00:52:44.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.034+0000 7f1aa2d9d700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1a8c062cc0 0x7f1a8c065180 secure :-1 s=READY pgs=3147 cs=0 l=1 rev1=1 rx=0x7f1a94004ee0 tx=0x7f1a9400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:44.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.035+0000 7f1aa91c3700 10 client.31506.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:44.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.035+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_assign -1 1 2024-02-23T00:52:44.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.035+0000 7f1aa91c3700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f1aa40f9d30 0x7f1aa40fa110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:44.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.035+0000 7f1aa91c3700 20 client.31506.objecter _get_session s=0x7f1aa405ee80 osd=0 3 2024-02-23T00:52:44.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.035+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa405ee80 osd=0 3 2024-02-23T00:52:44.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.035+0000 7f1aa91c3700 20 client.31506.objecter _get_session s=0x7f1aa405ee80 osd=0 3 2024-02-23T00:52:44.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.035+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_remove -1 1 2024-02-23T00:52:44.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.035+0000 7f1aa91c3700 20 client.31506.objecter get_session s=0x7f1aa405ee80 osd=0 3 2024-02-23T00:52:44.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.035+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_assign 0 1 2024-02-23T00:52:44.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.035+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa405ee80 osd=0 4 2024-02-23T00:52:44.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.035+0000 7f1aa91c3700 10 client.31506.objecter _send_command 1 2024-02-23T00:52:44.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.035+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1aa4106ed0 con 0x7f1aa40f9d30 2024-02-23T00:52:44.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.035+0000 7f1aa359e700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f1aa40f9d30 0x7f1aa40fa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:44.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.035+0000 7f1a8bfff700 10 client.31506.objecter ms_handle_connect 0x7f1aa40f9d30 2024-02-23T00:52:44.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.035+0000 7f1aa359e700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f1aa40f9d30 0x7f1aa40fa110 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:44.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.036+0000 7f1a8bfff700 1 -- 172.21.15.47:0/1115176460 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1aa4106ed0 con 0x7f1aa40f9d30 2024-02-23T00:52:44.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.036+0000 7f1a8bfff700 10 client.31506.objecter ms_dispatch 0x7f1aa4001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:52:44.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.036+0000 7f1a8bfff700 10 client.31506.objecter _finish_command 1 = system:0 2024-02-23T00:52:44.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.036+0000 7f1a8bfff700 20 client.31506.objecter put_session s=0x7f1aa405ee80 osd=0 4 2024-02-23T00:52:44.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.036+0000 7f1a8bfff700 15 client.31506.objecter _session_command_op_remove 0 1 2024-02-23T00:52:44.042 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:44.043 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:52:44.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.051+0000 7f1aa91c3700 10 client.31506.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:52:44.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.051+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_assign -1 2 2024-02-23T00:52:44.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.051+0000 7f1aa91c3700 20 client.31506.objecter _get_session s=0x7f1aa405ee80 osd=0 3 2024-02-23T00:52:44.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.051+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa405ee80 osd=0 3 2024-02-23T00:52:44.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.051+0000 7f1aa91c3700 20 client.31506.objecter _get_session s=0x7f1aa405ee80 osd=0 3 2024-02-23T00:52:44.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.051+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_remove -1 2 2024-02-23T00:52:44.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.051+0000 7f1aa91c3700 20 client.31506.objecter get_session s=0x7f1aa405ee80 osd=0 3 2024-02-23T00:52:44.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.051+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_assign 0 2 2024-02-23T00:52:44.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.051+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa405ee80 osd=0 4 2024-02-23T00:52:44.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.051+0000 7f1aa91c3700 10 client.31506.objecter _send_command 2 2024-02-23T00:52:44.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.051+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1aa40273c0 con 0x7f1aa40f9d30 2024-02-23T00:52:44.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.053+0000 7f1a8bfff700 1 -- 172.21.15.47:0/1115176460 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1aa40273c0 con 0x7f1aa40f9d30 2024-02-23T00:52:44.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.053+0000 7f1a8bfff700 10 client.31506.objecter ms_dispatch 0x7f1aa4001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:52:44.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.053+0000 7f1a8bfff700 10 client.31506.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:52:44.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.053+0000 7f1a8bfff700 20 client.31506.objecter put_session s=0x7f1aa405ee80 osd=0 4 2024-02-23T00:52:44.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.053+0000 7f1a8bfff700 15 client.31506.objecter _session_command_op_remove 0 2 2024-02-23T00:52:44.056 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:52:44.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.053+0000 7f1aa91c3700 10 client.31506.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:44.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.053+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_assign -1 3 2024-02-23T00:52:44.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.053+0000 7f1aa91c3700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f1aa40598f0 0x7f1aa4059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:44.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.053+0000 7f1aa91c3700 20 client.31506.objecter _get_session s=0x7f1aa4028710 osd=1 3 2024-02-23T00:52:44.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.053+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa4028710 osd=1 3 2024-02-23T00:52:44.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.053+0000 7f1aa91c3700 20 client.31506.objecter _get_session s=0x7f1aa4028710 osd=1 3 2024-02-23T00:52:44.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.053+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_remove -1 3 2024-02-23T00:52:44.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.053+0000 7f1aa91c3700 20 client.31506.objecter get_session s=0x7f1aa4028710 osd=1 3 2024-02-23T00:52:44.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.053+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_assign 1 3 2024-02-23T00:52:44.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.053+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa4028710 osd=1 4 2024-02-23T00:52:44.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.053+0000 7f1aa91c3700 10 client.31506.objecter _send_command 3 2024-02-23T00:52:44.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.053+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1aa4121ad0 con 0x7f1aa40598f0 2024-02-23T00:52:44.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.053+0000 7f1aa2d9d700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f1aa40598f0 0x7f1aa4059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:44.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.053+0000 7f1a8bfff700 10 client.31506.objecter ms_handle_connect 0x7f1aa40598f0 2024-02-23T00:52:44.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.053+0000 7f1aa2d9d700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f1aa40598f0 0x7f1aa4059cd0 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:44.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.054+0000 7f1a8bfff700 1 -- 172.21.15.47:0/1115176460 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1aa4121ad0 con 0x7f1aa40598f0 2024-02-23T00:52:44.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.054+0000 7f1a8bfff700 10 client.31506.objecter ms_dispatch 0x7f1aa4001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:52:44.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.054+0000 7f1a8bfff700 10 client.31506.objecter _finish_command 3 = system:0 2024-02-23T00:52:44.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.054+0000 7f1a8bfff700 20 client.31506.objecter put_session s=0x7f1aa4028710 osd=1 4 2024-02-23T00:52:44.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.054+0000 7f1a8bfff700 15 client.31506.objecter _session_command_op_remove 1 3 2024-02-23T00:52:44.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.069+0000 7f1aa91c3700 10 client.31506.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:52:44.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.069+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_assign -1 4 2024-02-23T00:52:44.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.069+0000 7f1aa91c3700 20 client.31506.objecter _get_session s=0x7f1aa4028710 osd=1 3 2024-02-23T00:52:44.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.069+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa4028710 osd=1 3 2024-02-23T00:52:44.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.069+0000 7f1aa91c3700 20 client.31506.objecter _get_session s=0x7f1aa4028710 osd=1 3 2024-02-23T00:52:44.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.069+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_remove -1 4 2024-02-23T00:52:44.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.069+0000 7f1aa91c3700 20 client.31506.objecter get_session s=0x7f1aa4028710 osd=1 3 2024-02-23T00:52:44.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.069+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_assign 1 4 2024-02-23T00:52:44.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.069+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa4028710 osd=1 4 2024-02-23T00:52:44.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.069+0000 7f1aa91c3700 10 client.31506.objecter _send_command 4 2024-02-23T00:52:44.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.069+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1aa4059360 con 0x7f1aa40598f0 2024-02-23T00:52:44.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.071+0000 7f1a8bfff700 1 -- 172.21.15.47:0/1115176460 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1aa4059360 con 0x7f1aa40598f0 2024-02-23T00:52:44.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.071+0000 7f1a8bfff700 10 client.31506.objecter ms_dispatch 0x7f1aa4001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:52:44.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.071+0000 7f1a8bfff700 10 client.31506.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:52:44.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.071+0000 7f1a8bfff700 20 client.31506.objecter put_session s=0x7f1aa4028710 osd=1 4 2024-02-23T00:52:44.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.071+0000 7f1a8bfff700 15 client.31506.objecter _session_command_op_remove 1 4 2024-02-23T00:52:44.074 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:52:44.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.071+0000 7f1aa91c3700 10 client.31506.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:44.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.071+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_assign -1 5 2024-02-23T00:52:44.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.071+0000 7f1aa91c3700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f1aa4028270 0x7f1aa41204f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:44.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.071+0000 7f1aa91c3700 20 client.31506.objecter _get_session s=0x7f1aa4059770 osd=2 3 2024-02-23T00:52:44.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.071+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa4059770 osd=2 3 2024-02-23T00:52:44.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.071+0000 7f1aa91c3700 20 client.31506.objecter _get_session s=0x7f1aa4059770 osd=2 3 2024-02-23T00:52:44.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.071+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_remove -1 5 2024-02-23T00:52:44.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.071+0000 7f1aa91c3700 20 client.31506.objecter get_session s=0x7f1aa4059770 osd=2 3 2024-02-23T00:52:44.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.071+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_assign 2 5 2024-02-23T00:52:44.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.071+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa4059770 osd=2 4 2024-02-23T00:52:44.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.071+0000 7f1aa91c3700 10 client.31506.objecter _send_command 5 2024-02-23T00:52:44.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.071+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1aa41209e0 con 0x7f1aa4028270 2024-02-23T00:52:44.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.071+0000 7f1aa259c700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f1aa4028270 0x7f1aa41204f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:44.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.071+0000 7f1a8bfff700 10 client.31506.objecter ms_handle_connect 0x7f1aa4028270 2024-02-23T00:52:44.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.071+0000 7f1aa259c700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f1aa4028270 0x7f1aa41204f0 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:44.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.072+0000 7f1a8bfff700 1 -- 172.21.15.47:0/1115176460 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1aa41209e0 con 0x7f1aa4028270 2024-02-23T00:52:44.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.072+0000 7f1a8bfff700 10 client.31506.objecter ms_dispatch 0x7f1aa4001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:52:44.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.072+0000 7f1a8bfff700 10 client.31506.objecter _finish_command 5 = system:0 2024-02-23T00:52:44.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.072+0000 7f1a8bfff700 20 client.31506.objecter put_session s=0x7f1aa4059770 osd=2 4 2024-02-23T00:52:44.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.072+0000 7f1a8bfff700 15 client.31506.objecter _session_command_op_remove 2 5 2024-02-23T00:52:44.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.086+0000 7f1aa91c3700 10 client.31506.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:52:44.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.086+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_assign -1 6 2024-02-23T00:52:44.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.086+0000 7f1aa91c3700 20 client.31506.objecter _get_session s=0x7f1aa4059770 osd=2 3 2024-02-23T00:52:44.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.086+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa4059770 osd=2 3 2024-02-23T00:52:44.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.086+0000 7f1aa91c3700 20 client.31506.objecter _get_session s=0x7f1aa4059770 osd=2 3 2024-02-23T00:52:44.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.086+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_remove -1 6 2024-02-23T00:52:44.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.086+0000 7f1aa91c3700 20 client.31506.objecter get_session s=0x7f1aa4059770 osd=2 3 2024-02-23T00:52:44.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.086+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_assign 2 6 2024-02-23T00:52:44.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.086+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa4059770 osd=2 4 2024-02-23T00:52:44.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.086+0000 7f1aa91c3700 10 client.31506.objecter _send_command 6 2024-02-23T00:52:44.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.086+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1aa40277d0 con 0x7f1aa4028270 2024-02-23T00:52:44.090 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:52:44.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.088+0000 7f1a8bfff700 1 -- 172.21.15.47:0/1115176460 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1aa40277d0 con 0x7f1aa4028270 2024-02-23T00:52:44.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.088+0000 7f1a8bfff700 10 client.31506.objecter ms_dispatch 0x7f1aa4001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:52:44.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.088+0000 7f1a8bfff700 10 client.31506.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:52:44.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.088+0000 7f1a8bfff700 20 client.31506.objecter put_session s=0x7f1aa4059770 osd=2 4 2024-02-23T00:52:44.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.088+0000 7f1a8bfff700 15 client.31506.objecter _session_command_op_remove 2 6 2024-02-23T00:52:44.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.088+0000 7f1aa91c3700 10 client.31506.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:44.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.088+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_assign -1 7 2024-02-23T00:52:44.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.088+0000 7f1aa91c3700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f1aa4123c50 0x7f1aa4126040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:44.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.088+0000 7f1aa91c3700 20 client.31506.objecter _get_session s=0x7f1aa4027bb0 osd=3 3 2024-02-23T00:52:44.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.088+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa4027bb0 osd=3 3 2024-02-23T00:52:44.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.088+0000 7f1aa91c3700 20 client.31506.objecter _get_session s=0x7f1aa4027bb0 osd=3 3 2024-02-23T00:52:44.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.088+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_remove -1 7 2024-02-23T00:52:44.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.088+0000 7f1aa91c3700 20 client.31506.objecter get_session s=0x7f1aa4027bb0 osd=3 3 2024-02-23T00:52:44.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.088+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_assign 3 7 2024-02-23T00:52:44.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.088+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa4027bb0 osd=3 4 2024-02-23T00:52:44.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.088+0000 7f1aa91c3700 10 client.31506.objecter _send_command 7 2024-02-23T00:52:44.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.088+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1aa4121950 con 0x7f1aa4123c50 2024-02-23T00:52:44.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.088+0000 7f1aa359e700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f1aa4123c50 0x7f1aa4126040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:44.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.088+0000 7f1a8bfff700 10 client.31506.objecter ms_handle_connect 0x7f1aa4123c50 2024-02-23T00:52:44.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.088+0000 7f1aa359e700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f1aa4123c50 0x7f1aa4126040 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:44.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.089+0000 7f1a8bfff700 1 -- 172.21.15.47:0/1115176460 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1a9c018040 con 0x7f1aa4123c50 2024-02-23T00:52:44.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.089+0000 7f1a8bfff700 10 client.31506.objecter ms_dispatch 0x7f1aa4001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:52:44.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.089+0000 7f1a8bfff700 10 client.31506.objecter _finish_command 7 = system:0 2024-02-23T00:52:44.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.089+0000 7f1a8bfff700 20 client.31506.objecter put_session s=0x7f1aa4027bb0 osd=3 4 2024-02-23T00:52:44.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.089+0000 7f1a8bfff700 15 client.31506.objecter _session_command_op_remove 3 7 2024-02-23T00:52:44.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.104+0000 7f1aa91c3700 10 client.31506.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:52:44.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.104+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_assign -1 8 2024-02-23T00:52:44.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.104+0000 7f1aa91c3700 20 client.31506.objecter _get_session s=0x7f1aa4027bb0 osd=3 3 2024-02-23T00:52:44.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.104+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa4027bb0 osd=3 3 2024-02-23T00:52:44.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.104+0000 7f1aa91c3700 20 client.31506.objecter _get_session s=0x7f1aa4027bb0 osd=3 3 2024-02-23T00:52:44.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.104+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_remove -1 8 2024-02-23T00:52:44.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.104+0000 7f1aa91c3700 20 client.31506.objecter get_session s=0x7f1aa4027bb0 osd=3 3 2024-02-23T00:52:44.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.104+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_assign 3 8 2024-02-23T00:52:44.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.104+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa4027bb0 osd=3 4 2024-02-23T00:52:44.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.104+0000 7f1aa91c3700 10 client.31506.objecter _send_command 8 2024-02-23T00:52:44.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.104+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1aa4120e20 con 0x7f1aa4123c50 2024-02-23T00:52:44.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.105+0000 7f1a8bfff700 1 -- 172.21.15.47:0/1115176460 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1aa4120e20 con 0x7f1aa4123c50 2024-02-23T00:52:44.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.105+0000 7f1a8bfff700 10 client.31506.objecter ms_dispatch 0x7f1aa4001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:52:44.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.105+0000 7f1a8bfff700 10 client.31506.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:52:44.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.105+0000 7f1a8bfff700 20 client.31506.objecter put_session s=0x7f1aa4027bb0 osd=3 4 2024-02-23T00:52:44.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.105+0000 7f1a8bfff700 15 client.31506.objecter _session_command_op_remove 3 8 2024-02-23T00:52:44.108 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:52:44.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.105+0000 7f1aa91c3700 10 client.31506.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:44.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.105+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_assign -1 9 2024-02-23T00:52:44.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.105+0000 7f1aa91c3700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f1aa4120f90 0x7f1aa41276b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:44.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.105+0000 7f1aa91c3700 20 client.31506.objecter _get_session s=0x7f1aa4120e20 osd=4 3 2024-02-23T00:52:44.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.105+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa4120e20 osd=4 3 2024-02-23T00:52:44.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.105+0000 7f1aa91c3700 20 client.31506.objecter _get_session s=0x7f1aa4120e20 osd=4 3 2024-02-23T00:52:44.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.105+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_remove -1 9 2024-02-23T00:52:44.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.105+0000 7f1aa91c3700 20 client.31506.objecter get_session s=0x7f1aa4120e20 osd=4 3 2024-02-23T00:52:44.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.105+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_assign 4 9 2024-02-23T00:52:44.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.105+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa4120e20 osd=4 4 2024-02-23T00:52:44.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.105+0000 7f1aa91c3700 10 client.31506.objecter _send_command 9 2024-02-23T00:52:44.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.105+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1aa4120240 con 0x7f1aa4120f90 2024-02-23T00:52:44.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.106+0000 7f1aa2d9d700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f1aa4120f90 0x7f1aa41276b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:44.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.106+0000 7f1aa2d9d700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f1aa4120f90 0x7f1aa41276b0 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:44.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.106+0000 7f1a8bfff700 10 client.31506.objecter ms_handle_connect 0x7f1aa4120f90 2024-02-23T00:52:44.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.107+0000 7f1a8bfff700 1 -- 172.21.15.47:0/1115176460 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1aa4120240 con 0x7f1aa4120f90 2024-02-23T00:52:44.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.107+0000 7f1a8bfff700 10 client.31506.objecter ms_dispatch 0x7f1aa4001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:52:44.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.107+0000 7f1a8bfff700 10 client.31506.objecter _finish_command 9 = system:0 2024-02-23T00:52:44.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.107+0000 7f1a8bfff700 20 client.31506.objecter put_session s=0x7f1aa4120e20 osd=4 4 2024-02-23T00:52:44.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.107+0000 7f1a8bfff700 15 client.31506.objecter _session_command_op_remove 4 9 2024-02-23T00:52:44.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.122+0000 7f1aa91c3700 10 client.31506.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:52:44.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.122+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_assign -1 10 2024-02-23T00:52:44.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.122+0000 7f1aa91c3700 20 client.31506.objecter _get_session s=0x7f1aa4120e20 osd=4 3 2024-02-23T00:52:44.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.122+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa4120e20 osd=4 3 2024-02-23T00:52:44.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.122+0000 7f1aa91c3700 20 client.31506.objecter _get_session s=0x7f1aa4120e20 osd=4 3 2024-02-23T00:52:44.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.122+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_remove -1 10 2024-02-23T00:52:44.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.122+0000 7f1aa91c3700 20 client.31506.objecter get_session s=0x7f1aa4120e20 osd=4 3 2024-02-23T00:52:44.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.122+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_assign 4 10 2024-02-23T00:52:44.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.122+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa4120e20 osd=4 4 2024-02-23T00:52:44.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.122+0000 7f1aa91c3700 10 client.31506.objecter _send_command 10 2024-02-23T00:52:44.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.122+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1aa4120240 con 0x7f1aa4120f90 2024-02-23T00:52:44.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.125+0000 7f1a8bfff700 1 -- 172.21.15.47:0/1115176460 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1aa4120240 con 0x7f1aa4120f90 2024-02-23T00:52:44.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.125+0000 7f1a8bfff700 10 client.31506.objecter ms_dispatch 0x7f1aa4001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:52:44.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.125+0000 7f1a8bfff700 10 client.31506.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:52:44.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.125+0000 7f1a8bfff700 20 client.31506.objecter put_session s=0x7f1aa4120e20 osd=4 4 2024-02-23T00:52:44.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.125+0000 7f1a8bfff700 15 client.31506.objecter _session_command_op_remove 4 10 2024-02-23T00:52:44.127 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:52:44.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.125+0000 7f1aa91c3700 10 client.31506.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:44.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.125+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_assign -1 11 2024-02-23T00:52:44.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.125+0000 7f1aa91c3700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f1aa41280b0 0x7f1aa41284f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:44.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.125+0000 7f1aa91c3700 20 client.31506.objecter _get_session s=0x7f1aa4120240 osd=5 3 2024-02-23T00:52:44.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.125+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa4120240 osd=5 3 2024-02-23T00:52:44.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.125+0000 7f1aa91c3700 20 client.31506.objecter _get_session s=0x7f1aa4120240 osd=5 3 2024-02-23T00:52:44.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.125+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_remove -1 11 2024-02-23T00:52:44.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.125+0000 7f1aa91c3700 20 client.31506.objecter get_session s=0x7f1aa4120240 osd=5 3 2024-02-23T00:52:44.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.125+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_assign 5 11 2024-02-23T00:52:44.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.125+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa4120240 osd=5 4 2024-02-23T00:52:44.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.125+0000 7f1aa91c3700 10 client.31506.objecter _send_command 11 2024-02-23T00:52:44.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.125+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1aa412f470 con 0x7f1aa41280b0 2024-02-23T00:52:44.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.125+0000 7f1aa259c700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f1aa41280b0 0x7f1aa41284f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:44.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.125+0000 7f1aa259c700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f1aa41280b0 0x7f1aa41284f0 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:44.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.125+0000 7f1a8bfff700 10 client.31506.objecter ms_handle_connect 0x7f1aa41280b0 2024-02-23T00:52:44.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.126+0000 7f1a8bfff700 1 -- 172.21.15.47:0/1115176460 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1aa412f470 con 0x7f1aa41280b0 2024-02-23T00:52:44.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.126+0000 7f1a8bfff700 10 client.31506.objecter ms_dispatch 0x7f1aa4001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:52:44.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.126+0000 7f1a8bfff700 10 client.31506.objecter _finish_command 11 = system:0 2024-02-23T00:52:44.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.126+0000 7f1a8bfff700 20 client.31506.objecter put_session s=0x7f1aa4120240 osd=5 4 2024-02-23T00:52:44.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.126+0000 7f1a8bfff700 15 client.31506.objecter _session_command_op_remove 5 11 2024-02-23T00:52:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.140+0000 7f1aa91c3700 10 client.31506.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:52:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.140+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_assign -1 12 2024-02-23T00:52:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.140+0000 7f1aa91c3700 20 client.31506.objecter _get_session s=0x7f1aa4120240 osd=5 3 2024-02-23T00:52:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.140+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa4120240 osd=5 3 2024-02-23T00:52:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.140+0000 7f1aa91c3700 20 client.31506.objecter _get_session s=0x7f1aa4120240 osd=5 3 2024-02-23T00:52:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.140+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_remove -1 12 2024-02-23T00:52:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.140+0000 7f1aa91c3700 20 client.31506.objecter get_session s=0x7f1aa4120240 osd=5 3 2024-02-23T00:52:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.140+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_assign 5 12 2024-02-23T00:52:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.140+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa4120240 osd=5 4 2024-02-23T00:52:44.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.140+0000 7f1aa91c3700 10 client.31506.objecter _send_command 12 2024-02-23T00:52:44.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.140+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1aa412cd10 con 0x7f1aa41280b0 2024-02-23T00:52:44.144 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:52:44.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.142+0000 7f1a8bfff700 1 -- 172.21.15.47:0/1115176460 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1aa412cd10 con 0x7f1aa41280b0 2024-02-23T00:52:44.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.142+0000 7f1a8bfff700 10 client.31506.objecter ms_dispatch 0x7f1aa4001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:52:44.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.142+0000 7f1a8bfff700 10 client.31506.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:52:44.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.142+0000 7f1a8bfff700 20 client.31506.objecter put_session s=0x7f1aa4120240 osd=5 4 2024-02-23T00:52:44.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.142+0000 7f1a8bfff700 15 client.31506.objecter _session_command_op_remove 5 12 2024-02-23T00:52:44.146 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:52:44.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.143+0000 7f1aa91c3700 10 client.31506.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:44.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.143+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_assign -1 13 2024-02-23T00:52:44.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.143+0000 7f1aa91c3700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f1aa412d390 0x7f1aa412f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:44.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.143+0000 7f1aa91c3700 20 client.31506.objecter _get_session s=0x7f1aa412cd10 osd=6 3 2024-02-23T00:52:44.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.143+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa412cd10 osd=6 3 2024-02-23T00:52:44.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.143+0000 7f1aa91c3700 20 client.31506.objecter _get_session s=0x7f1aa412cd10 osd=6 3 2024-02-23T00:52:44.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.143+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_remove -1 13 2024-02-23T00:52:44.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.143+0000 7f1aa91c3700 20 client.31506.objecter get_session s=0x7f1aa412cd10 osd=6 3 2024-02-23T00:52:44.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.143+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_assign 6 13 2024-02-23T00:52:44.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.143+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa412cd10 osd=6 4 2024-02-23T00:52:44.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.143+0000 7f1aa91c3700 10 client.31506.objecter _send_command 13 2024-02-23T00:52:44.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.143+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1aa4132570 con 0x7f1aa412d390 2024-02-23T00:52:44.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.143+0000 7f1aa359e700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f1aa412d390 0x7f1aa412f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:44.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.143+0000 7f1a8bfff700 10 client.31506.objecter ms_handle_connect 0x7f1aa412d390 2024-02-23T00:52:44.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.143+0000 7f1aa359e700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f1aa412d390 0x7f1aa412f780 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:44.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.144+0000 7f1a8bfff700 1 -- 172.21.15.47:0/1115176460 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1aa4132570 con 0x7f1aa412d390 2024-02-23T00:52:44.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.144+0000 7f1a8bfff700 10 client.31506.objecter ms_dispatch 0x7f1aa4001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:52:44.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.144+0000 7f1a8bfff700 10 client.31506.objecter _finish_command 13 = system:0 2024-02-23T00:52:44.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.144+0000 7f1a8bfff700 20 client.31506.objecter put_session s=0x7f1aa412cd10 osd=6 4 2024-02-23T00:52:44.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.144+0000 7f1a8bfff700 15 client.31506.objecter _session_command_op_remove 6 13 2024-02-23T00:52:44.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.158+0000 7f1aa91c3700 10 client.31506.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:52:44.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.158+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_assign -1 14 2024-02-23T00:52:44.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.158+0000 7f1aa91c3700 20 client.31506.objecter _get_session s=0x7f1aa412cd10 osd=6 3 2024-02-23T00:52:44.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.158+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa412cd10 osd=6 3 2024-02-23T00:52:44.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.158+0000 7f1aa91c3700 20 client.31506.objecter _get_session s=0x7f1aa412cd10 osd=6 3 2024-02-23T00:52:44.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.158+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_remove -1 14 2024-02-23T00:52:44.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.158+0000 7f1aa91c3700 20 client.31506.objecter get_session s=0x7f1aa412cd10 osd=6 3 2024-02-23T00:52:44.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.158+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_assign 6 14 2024-02-23T00:52:44.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.158+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa412cd10 osd=6 4 2024-02-23T00:52:44.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.158+0000 7f1aa91c3700 10 client.31506.objecter _send_command 14 2024-02-23T00:52:44.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.158+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1aa4132570 con 0x7f1aa412d390 2024-02-23T00:52:44.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.161+0000 7f1a8bfff700 1 -- 172.21.15.47:0/1115176460 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1aa4132570 con 0x7f1aa412d390 2024-02-23T00:52:44.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.161+0000 7f1a8bfff700 10 client.31506.objecter ms_dispatch 0x7f1aa4001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:52:44.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.161+0000 7f1a8bfff700 10 client.31506.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:52:44.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.161+0000 7f1a8bfff700 20 client.31506.objecter put_session s=0x7f1aa412cd10 osd=6 4 2024-02-23T00:52:44.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.161+0000 7f1a8bfff700 15 client.31506.objecter _session_command_op_remove 6 14 2024-02-23T00:52:44.164 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:52:44.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.161+0000 7f1aa91c3700 10 client.31506.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:44.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.161+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_assign -1 15 2024-02-23T00:52:44.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.161+0000 7f1aa91c3700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f1aa4130270 0x7f1aa41306d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:44.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.161+0000 7f1aa91c3700 20 client.31506.objecter _get_session s=0x7f1aa4132570 osd=7 3 2024-02-23T00:52:44.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.161+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa4132570 osd=7 3 2024-02-23T00:52:44.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.161+0000 7f1aa91c3700 20 client.31506.objecter _get_session s=0x7f1aa4132570 osd=7 3 2024-02-23T00:52:44.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.161+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_remove -1 15 2024-02-23T00:52:44.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.161+0000 7f1aa91c3700 20 client.31506.objecter get_session s=0x7f1aa4132570 osd=7 3 2024-02-23T00:52:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.161+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_assign 7 15 2024-02-23T00:52:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.161+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa4132570 osd=7 4 2024-02-23T00:52:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.161+0000 7f1aa91c3700 10 client.31506.objecter _send_command 15 2024-02-23T00:52:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.161+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1aa41358f0 con 0x7f1aa4130270 2024-02-23T00:52:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.161+0000 7f1aa2d9d700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f1aa4130270 0x7f1aa41306d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.161+0000 7f1a8bfff700 10 client.31506.objecter ms_handle_connect 0x7f1aa4130270 2024-02-23T00:52:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.161+0000 7f1aa2d9d700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f1aa4130270 0x7f1aa41306d0 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.162+0000 7f1a8bfff700 1 -- 172.21.15.47:0/1115176460 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1aa41358f0 con 0x7f1aa4130270 2024-02-23T00:52:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.162+0000 7f1a8bfff700 10 client.31506.objecter ms_dispatch 0x7f1aa4001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:52:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.162+0000 7f1a8bfff700 10 client.31506.objecter _finish_command 15 = system:0 2024-02-23T00:52:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.162+0000 7f1a8bfff700 20 client.31506.objecter put_session s=0x7f1aa4132570 osd=7 4 2024-02-23T00:52:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.162+0000 7f1a8bfff700 15 client.31506.objecter _session_command_op_remove 7 15 2024-02-23T00:52:44.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.176+0000 7f1aa91c3700 10 client.31506.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:52:44.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.176+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_assign -1 16 2024-02-23T00:52:44.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.176+0000 7f1aa91c3700 20 client.31506.objecter _get_session s=0x7f1aa4132570 osd=7 3 2024-02-23T00:52:44.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.176+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa4132570 osd=7 3 2024-02-23T00:52:44.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.176+0000 7f1aa91c3700 20 client.31506.objecter _get_session s=0x7f1aa4132570 osd=7 3 2024-02-23T00:52:44.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.176+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_remove -1 16 2024-02-23T00:52:44.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.176+0000 7f1aa91c3700 20 client.31506.objecter get_session s=0x7f1aa4132570 osd=7 3 2024-02-23T00:52:44.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.176+0000 7f1aa91c3700 15 client.31506.objecter _session_command_op_assign 7 16 2024-02-23T00:52:44.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.176+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa4132570 osd=7 4 2024-02-23T00:52:44.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.176+0000 7f1aa91c3700 10 client.31506.objecter _send_command 16 2024-02-23T00:52:44.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.176+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1aa4130cd0 con 0x7f1aa4130270 2024-02-23T00:52:44.180 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:44.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.178+0000 7f1a8bfff700 1 -- 172.21.15.47:0/1115176460 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1aa4130cd0 con 0x7f1aa4130270 2024-02-23T00:52:44.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.178+0000 7f1a8bfff700 10 client.31506.objecter ms_dispatch 0x7f1aa4001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:52:44.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.178+0000 7f1a8bfff700 10 client.31506.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:52:44.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.178+0000 7f1a8bfff700 20 client.31506.objecter put_session s=0x7f1aa4132570 osd=7 4 2024-02-23T00:52:44.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.178+0000 7f1a8bfff700 15 client.31506.objecter _session_command_op_remove 7 16 2024-02-23T00:52:44.182 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:52:44.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.178+0000 7f1aa91c3700 10 librados: watch_flush enter 2024-02-23T00:52:44.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.178+0000 7f1aa91c3700 10 librados: watch_flush exit 2024-02-23T00:52:44.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 10 client.31506.objecter close_session for osd.0 2024-02-23T00:52:44.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f1aa40f9d30 msgr2=0x7f1aa40fa110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:44.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f1aa40f9d30 0x7f1aa40fa110 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:44.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa405ee80 osd=0 1 2024-02-23T00:52:44.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 10 client.31506.objecter close_session for osd.1 2024-02-23T00:52:44.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f1aa40598f0 msgr2=0x7f1aa4059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:44.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f1aa40598f0 0x7f1aa4059cd0 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:44.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa4028710 osd=1 1 2024-02-23T00:52:44.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 10 client.31506.objecter close_session for osd.2 2024-02-23T00:52:44.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f1aa4028270 msgr2=0x7f1aa41204f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:44.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f1aa4028270 0x7f1aa41204f0 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:44.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa4059770 osd=2 1 2024-02-23T00:52:44.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 10 client.31506.objecter close_session for osd.3 2024-02-23T00:52:44.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f1aa4123c50 msgr2=0x7f1aa4126040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:44.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f1aa4123c50 0x7f1aa4126040 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:44.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa359e700 1 -- 172.21.15.47:0/1115176460 reap_dead start 2024-02-23T00:52:44.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa4027bb0 osd=3 1 2024-02-23T00:52:44.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 10 client.31506.objecter close_session for osd.4 2024-02-23T00:52:44.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f1aa4120f90 msgr2=0x7f1aa41276b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:44.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f1aa4120f90 0x7f1aa41276b0 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:44.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa359e700 1 -- 172.21.15.47:0/1115176460 reap_dead start 2024-02-23T00:52:44.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa4120e20 osd=4 1 2024-02-23T00:52:44.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 10 client.31506.objecter close_session for osd.5 2024-02-23T00:52:44.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f1aa41280b0 msgr2=0x7f1aa41284f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:44.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f1aa41280b0 0x7f1aa41284f0 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:44.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa4120240 osd=5 1 2024-02-23T00:52:44.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 10 client.31506.objecter close_session for osd.6 2024-02-23T00:52:44.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f1aa412d390 msgr2=0x7f1aa412f780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:44.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f1aa412d390 0x7f1aa412f780 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:44.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa412cd10 osd=6 1 2024-02-23T00:52:44.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 10 client.31506.objecter close_session for osd.7 2024-02-23T00:52:44.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f1aa4130270 msgr2=0x7f1aa41306d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:44.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f1aa4130270 0x7f1aa41306d0 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:44.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 20 client.31506.objecter put_session s=0x7f1aa4132570 osd=7 1 2024-02-23T00:52:44.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 20 client.31506.objecter shutdown clearing up homeless session... 2024-02-23T00:52:44.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 10 client.31506.objecter successfully canceled tick 2024-02-23T00:52:44.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1a8c062cc0 msgr2=0x7f1a8c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:44.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1a8c062cc0 0x7f1a8c065180 secure :-1 s=READY pgs=3147 cs=0 l=1 rev1=1 rx=0x7f1a94004ee0 tx=0x7f1a9400b040).stop 2024-02-23T00:52:44.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1aa40fdbd0 msgr2=0x7f1aa4114950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:44.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 1 --2- 172.21.15.47:0/1115176460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1aa40fdbd0 0x7f1aa4114950 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 rx=0x7f1a980048f0 tx=0x7f1a98009cb0).stop 2024-02-23T00:52:44.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa359e700 1 -- 172.21.15.47:0/1115176460 reap_dead start 2024-02-23T00:52:44.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 shutdown_connections 2024-02-23T00:52:44.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 >> 172.21.15.47:0/1115176460 conn(0x7f1aa406c370 msgr2=0x7f1aa40f56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:52:44.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 shutdown_connections 2024-02-23T00:52:44.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 1 -- 172.21.15.47:0/1115176460 wait complete. 2024-02-23T00:52:44.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:44.179+0000 7f1aa91c3700 1 librados: shutdown 2024-02-23T00:52:44.186 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:52:44.186 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:52:44.186 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:52:44.187 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:52:44.187 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:52:44.187 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:52:44.187 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:52:44.187 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:52:44.201 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:52:44.203 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:44.204 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:52:44.304 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:52:44.333 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:44.538 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:52:44.539 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:52:44.541 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:44.542 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:52:44.571 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:52:44.642 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:52:44.675 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:44.696 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:52:44.698 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:44.699 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:52:44.741 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:42.817668+0000 mgr.y (mgr.14517) 558 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:52:44.741 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:43.362096+0000 mgr.y (mgr.14517) 559 : cluster [DBG] pgmap v946: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:52:44.741 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:43.453513+0000 mon.a (mon.0) 3195 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.9", "id": [7, 3]}]': finished 2024-02-23T00:52:44.741 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:43.453574+0000 mon.a (mon.0) 3196 : cluster [DBG] osdmap e626: 8 total, 8 up, 7 in 2024-02-23T00:52:44.741 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:43.589971+0000 osd.0 (osd.0) 261 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:52:44.741 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:43.590069+0000 osd.0 (osd.0) 262 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:52:44.741 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:44.034764+0000 mon.a (mon.0) 3197 : audit [DBG] from='client.? 172.21.15.47:0/1115176460' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:52:44.741 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:42.817668+0000 mgr.y (mgr.14517) 558 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:52:44.742 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:43.362096+0000 mgr.y (mgr.14517) 559 : cluster [DBG] pgmap v946: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:52:44.742 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:43.453513+0000 mon.a (mon.0) 3195 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.9", "id": [7, 3]}]': finished 2024-02-23T00:52:44.742 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:43.453574+0000 mon.a (mon.0) 3196 : cluster [DBG] osdmap e626: 8 total, 8 up, 7 in 2024-02-23T00:52:44.742 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:43.589971+0000 osd.0 (osd.0) 261 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:52:44.742 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:43.590069+0000 osd.0 (osd.0) 262 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:52:44.742 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:44.034764+0000 mon.a (mon.0) 3197 : audit [DBG] from='client.? 172.21.15.47:0/1115176460' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:52:44.799 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:52:44.827 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:42.817668+0000 mgr.y (mgr.14517) 558 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:52:44.828 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:43.362096+0000 mgr.y (mgr.14517) 559 : cluster [DBG] pgmap v946: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:52:44.828 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:43.453513+0000 mon.a (mon.0) 3195 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.9", "id": [7, 3]}]': finished 2024-02-23T00:52:44.828 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:43.453574+0000 mon.a (mon.0) 3196 : cluster [DBG] osdmap e626: 8 total, 8 up, 7 in 2024-02-23T00:52:44.828 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:43.589971+0000 osd.0 (osd.0) 261 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:52:44.828 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:43.590069+0000 osd.0 (osd.0) 262 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:52:44.828 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:44.034764+0000 mon.a (mon.0) 3197 : audit [DBG] from='client.? 172.21.15.47:0/1115176460' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:52:44.829 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:44.850 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:52:44.853 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:44.853 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:52:44.954 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:52:44.985 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:45.012 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:52:45.015 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:45.016 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:52:45.116 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:52:45.149 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:45.170 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:52:45.172 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:45.173 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:52:45.274 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:52:45.306 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:45.327 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:52:45.329 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:45.329 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:52:45.430 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:52:45.459 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:45.479 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:52:45.482 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:45.482 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:52:45.583 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:52:45.613 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:45.803 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:44.458151+0000 mon.a (mon.0) 3198 : cluster [DBG] osdmap e627: 8 total, 8 up, 7 in 2024-02-23T00:52:45.803 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:45.264339+0000 mon.a (mon.0) 3199 : cluster [DBG] osdmap e628: 8 total, 8 up, 7 in 2024-02-23T00:52:45.803 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:44.458151+0000 mon.a (mon.0) 3198 : cluster [DBG] osdmap e627: 8 total, 8 up, 7 in 2024-02-23T00:52:45.803 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:45.264339+0000 mon.a (mon.0) 3199 : cluster [DBG] osdmap e628: 8 total, 8 up, 7 in 2024-02-23T00:52:45.811 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:52:45.815 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:45.816 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:52:45.917 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:52:45.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:44.458151+0000 mon.a (mon.0) 3198 : cluster [DBG] osdmap e627: 8 total, 8 up, 7 in 2024-02-23T00:52:45.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:45.264339+0000 mon.a (mon.0) 3199 : cluster [DBG] osdmap e628: 8 total, 8 up, 7 in 2024-02-23T00:52:45.944 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:46.019 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:52:46.022 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:46.022 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:52:46.123 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:52:46.152 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:46.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:45.362660+0000 mgr.y (mgr.14517) 560 : cluster [DBG] pgmap v950: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:52:46.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:45.384562+0000 osd.6 (osd.6) 239 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:52:46.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:45.384671+0000 osd.6 (osd.6) 240 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:52:46.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:46.268881+0000 mon.a (mon.0) 3200 : cluster [DBG] osdmap e629: 8 total, 8 up, 7 in 2024-02-23T00:52:46.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:46.299635+0000 mon.c (mon.2) 357 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:46.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:46.299964+0000 mon.a (mon.0) 3201 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:46.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:45.362660+0000 mgr.y (mgr.14517) 560 : cluster [DBG] pgmap v950: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:52:46.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:45.384562+0000 osd.6 (osd.6) 239 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:52:46.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:45.384671+0000 osd.6 (osd.6) 240 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:52:46.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:46.268881+0000 mon.a (mon.0) 3200 : cluster [DBG] osdmap e629: 8 total, 8 up, 7 in 2024-02-23T00:52:46.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:46.299635+0000 mon.c (mon.2) 357 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:46.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:46.299964+0000 mon.a (mon.0) 3201 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:46.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:45.362660+0000 mgr.y (mgr.14517) 560 : cluster [DBG] pgmap v950: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:52:46.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:45.384562+0000 osd.6 (osd.6) 239 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:52:46.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:45.384671+0000 osd.6 (osd.6) 240 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:52:46.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:46.268881+0000 mon.a (mon.0) 3200 : cluster [DBG] osdmap e629: 8 total, 8 up, 7 in 2024-02-23T00:52:46.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:46.299635+0000 mon.c (mon.2) 357 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:46.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:46.299964+0000 mon.a (mon.0) 3201 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:47.003 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:52:47.006 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:47.007 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:52:47.107 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:52:47.138 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:47.158 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:52:47.168 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:47.169 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:52:47.269 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:52:47.302 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:47.371 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:47.372 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:52:47.372 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:52:47.473 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:52:47.505 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:47.525 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:52:47.528 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:47.529 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:52:47.630 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:52:47.660 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:47.681 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:52:47.684 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:47.684 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:52:47.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.192+0000 7f948cdda700 1 -- 172.21.15.47:0/1234357448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94880fd510 msgr2=0x7f94880f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:47.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.192+0000 7f948cdda700 1 --2- 172.21.15.47:0/1234357448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94880fd510 0x7f94880f6d40 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 rx=0x7f947c0047b0 tx=0x7f947c009cb0).stop 2024-02-23T00:52:47.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.192+0000 7f948cdda700 1 -- 172.21.15.47:0/1234357448 shutdown_connections 2024-02-23T00:52:47.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.192+0000 7f948cdda700 1 --2- 172.21.15.47:0/1234357448 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f94880f7400 0x7f9488103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:47.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.192+0000 7f948cdda700 1 --2- 172.21.15.47:0/1234357448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94880fcbb0 0x7f94880fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:47.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.192+0000 7f948cdda700 1 --2- 172.21.15.47:0/1234357448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94880fd510 0x7f94880f6d40 unknown :-1 s=CLOSED pgs=2599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:47.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.192+0000 7f948cdda700 1 -- 172.21.15.47:0/1234357448 >> 172.21.15.47:0/1234357448 conn(0x7f94880f2820 msgr2=0x7f94880f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:52:47.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.192+0000 7f948cdda700 1 -- 172.21.15.47:0/1234357448 shutdown_connections 2024-02-23T00:52:47.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.192+0000 7f948cdda700 1 -- 172.21.15.47:0/1234357448 wait complete. 2024-02-23T00:52:47.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.193+0000 7f948cdda700 1 librados: starting msgr at 2024-02-23T00:52:47.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.193+0000 7f948cdda700 1 librados: starting objecter 2024-02-23T00:52:47.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.193+0000 7f948cdda700 1 Processor -- start 2024-02-23T00:52:47.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.193+0000 7f948cdda700 1 -- start start 2024-02-23T00:52:47.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.193+0000 7f948cdda700 1 librados: setting wanted keys 2024-02-23T00:52:47.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.193+0000 7f948cdda700 1 librados: calling monclient init 2024-02-23T00:52:47.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.193+0000 7f948cdda700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f94880f7400 0x7f94880f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:47.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.193+0000 7f948cdda700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94880fcbb0 0x7f94880f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:47.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.193+0000 7f948cdda700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94880fd510 0x7f94880f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:47.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.193+0000 7f948cdda700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9488116460 con 0x7f94880fd510 2024-02-23T00:52:47.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.193+0000 7f948cdda700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94881165d0 con 0x7f94880fcbb0 2024-02-23T00:52:47.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.193+0000 7f948cdda700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9488116790 con 0x7f94880f7400 2024-02-23T00:52:47.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.193+0000 7f948659c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f94880f7400 0x7f94880f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:47.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.193+0000 7f9486d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94880fd510 0x7f94880f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:47.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.193+0000 7f948659c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f94880f7400 0x7f94880f89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:46174/0 (socket says 172.21.15.47:46174) 2024-02-23T00:52:47.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.193+0000 7f948659c700 1 -- 172.21.15.47:0/3578238271 learned_addr learned my addr 172.21.15.47:0/3578238271 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:52:47.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.193+0000 7f9485d9b700 1 --2- 172.21.15.47:0/3578238271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94880fcbb0 0x7f94880f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:47.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.194+0000 7f948659c700 1 -- 172.21.15.47:0/3578238271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94880fcbb0 msgr2=0x7f94880f8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:47.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.194+0000 7f948659c700 1 --2- 172.21.15.47:0/3578238271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94880fcbb0 0x7f94880f8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:47.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.194+0000 7f948659c700 1 -- 172.21.15.47:0/3578238271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94880fd510 msgr2=0x7f94880f9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:47.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.194+0000 7f948659c700 1 --2- 172.21.15.47:0/3578238271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94880fd510 0x7f94880f9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:47.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.194+0000 7f948659c700 1 -- 172.21.15.47:0/3578238271 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94800097c0 con 0x7f94880f7400 2024-02-23T00:52:47.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.194+0000 7f9486d9d700 1 --2- 172.21.15.47:0/3578238271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94880fd510 0x7f94880f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:52:47.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.194+0000 7f948659c700 1 --2- 172.21.15.47:0/3578238271 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f94880f7400 0x7f94880f89b0 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 rx=0x7f94780041d0 tx=0x7f9478009f60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:47.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.194+0000 7f94777fe700 10 client.?.objecter ms_handle_connect 0x7f94880f7400 2024-02-23T00:52:47.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.194+0000 7f94777fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:52:47.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.194+0000 7f94777fe700 1 -- 172.21.15.47:0/3578238271 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9478007b40 con 0x7f94880f7400 2024-02-23T00:52:47.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.194+0000 7f94777fe700 1 -- 172.21.15.47:0/3578238271 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f947800f040 con 0x7f94880f7400 2024-02-23T00:52:47.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.194+0000 7f948cdda700 1 -- 172.21.15.47:0/3578238271 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f947c005040 con 0x7f94880f7400 2024-02-23T00:52:47.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.194+0000 7f94777fe700 1 -- 172.21.15.47:0/3578238271 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9478014730 con 0x7f94880f7400 2024-02-23T00:52:47.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.194+0000 7f948cdda700 10 client.41831.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:52:47.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.194+0000 7f948cdda700 1 -- 172.21.15.47:0/3578238271 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9488116dd0 con 0x7f94880f7400 2024-02-23T00:52:47.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.194+0000 7f948cdda700 1 librados: init done 2024-02-23T00:52:47.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.195+0000 7f948cdda700 1 -- 172.21.15.47:0/3578238271 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f948805fa60 con 0x7f94880f7400 2024-02-23T00:52:47.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.195+0000 7f94777fe700 1 -- 172.21.15.47:0/3578238271 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f9478009110 con 0x7f94880f7400 2024-02-23T00:52:47.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.196+0000 7f94777fe700 1 --2- 172.21.15.47:0/3578238271 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9470062cc0 0x7f9470065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:47.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.196+0000 7f94777fe700 1 -- 172.21.15.47:0/3578238271 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(629..629 src has 482..629) v4 ==== 9719+0+0 (secure 0 0 0) 0x7f947807d870 con 0x7f94880f7400 2024-02-23T00:52:47.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.196+0000 7f94777fe700 10 client.41831.objecter ms_dispatch 0x7f9488001550 osd_map(629..629 src has 482..629) v4 2024-02-23T00:52:47.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.196+0000 7f94777fe700 3 client.41831.objecter handle_osd_map got epochs [629,629] > 0 2024-02-23T00:52:47.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.196+0000 7f94777fe700 3 client.41831.objecter handle_osd_map decoding full epoch 629 2024-02-23T00:52:47.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.196+0000 7f9485d9b700 1 --2- 172.21.15.47:0/3578238271 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9470062cc0 0x7f9470065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:47.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.196+0000 7f94777fe700 20 client.41831.objecter dump_active .. 0 homeless 2024-02-23T00:52:47.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.196+0000 7f94777fe700 10 client.41831.objecter ms_handle_connect 0x7f9470062cc0 2024-02-23T00:52:47.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.196+0000 7f9485d9b700 1 --2- 172.21.15.47:0/3578238271 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9470062cc0 0x7f9470065180 secure :-1 s=READY pgs=3148 cs=0 l=1 rev1=1 rx=0x7f94880fa150 tx=0x7f947c009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:47.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.200+0000 7f94777fe700 1 -- 172.21.15.47:0/3578238271 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f947804cc90 con 0x7f94880f7400 2024-02-23T00:52:47.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:47.386+0000 7f948cdda700 1 -- 172.21.15.47:0/3578238271 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7f94880f9e90 con 0x7f94880f7400 2024-02-23T00:52:47.784 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:52:47.813 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:47.832 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:52:47.835 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:47.835 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:52:47.936 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:52:47.968 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:47.991 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:52:47.994 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:47.994 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:52:48.095 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:52:48.125 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:48.145 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:52:48.147 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:48.148 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:52:48.242 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:52:48.242 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:52:48.249 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:52:48.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:48.279+0000 7f94777fe700 1 -- 172.21.15.47:0/3578238271 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v631) v1 ==== 91+0+0 (secure 0 0 0) 0x7f94780502b0 con 0x7f94880f7400 2024-02-23T00:52:48.283 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:48.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:48.281+0000 7f948cdda700 10 librados: watch_flush enter 2024-02-23T00:52:48.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:48.281+0000 7f948cdda700 10 librados: watch_flush exit 2024-02-23T00:52:48.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:48.281+0000 7f948cdda700 20 client.41831.objecter shutdown clearing up homeless session... 2024-02-23T00:52:48.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:48.281+0000 7f948cdda700 10 client.41831.objecter successfully canceled tick 2024-02-23T00:52:48.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:48.281+0000 7f948cdda700 1 -- 172.21.15.47:0/3578238271 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9470062cc0 msgr2=0x7f9470065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:48.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:48.281+0000 7f948cdda700 1 --2- 172.21.15.47:0/3578238271 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9470062cc0 0x7f9470065180 secure :-1 s=READY pgs=3148 cs=0 l=1 rev1=1 rx=0x7f94880fa150 tx=0x7f947c009cb0).stop 2024-02-23T00:52:48.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:48.281+0000 7f948cdda700 1 -- 172.21.15.47:0/3578238271 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f94880f7400 msgr2=0x7f94880f89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:48.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:48.281+0000 7f948cdda700 1 --2- 172.21.15.47:0/3578238271 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f94880f7400 0x7f94880f89b0 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 rx=0x7f94780041d0 tx=0x7f9478009f60).stop 2024-02-23T00:52:48.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:48.282+0000 7f948cdda700 1 -- 172.21.15.47:0/3578238271 shutdown_connections 2024-02-23T00:52:48.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:48.282+0000 7f948cdda700 1 --2- 172.21.15.47:0/3578238271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94880fd510 0x7f94880f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:48.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:48.282+0000 7f948cdda700 1 --2- 172.21.15.47:0/3578238271 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9470062cc0 0x7f9470065180 unknown :-1 s=CLOSED pgs=3148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:48.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:48.282+0000 7f948cdda700 1 --2- 172.21.15.47:0/3578238271 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f94880f7400 0x7f94880f89b0 unknown :-1 s=CLOSED pgs=2761 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:48.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:48.282+0000 7f948cdda700 1 --2- 172.21.15.47:0/3578238271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94880fcbb0 0x7f94880f8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:48.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:48.282+0000 7f948cdda700 1 -- 172.21.15.47:0/3578238271 >> 172.21.15.47:0/3578238271 conn(0x7f94880f2820 msgr2=0x7f9488060e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:52:48.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:48.282+0000 7f948cdda700 1 -- 172.21.15.47:0/3578238271 shutdown_connections 2024-02-23T00:52:48.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:48.282+0000 7f948cdda700 1 -- 172.21.15.47:0/3578238271 wait complete. 2024-02-23T00:52:48.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:48.282+0000 7f948cdda700 1 librados: shutdown 2024-02-23T00:52:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is unset 2024-02-23T00:52:48.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:47.270962+0000 mon.a (mon.0) 3202 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:52:48.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:47.271037+0000 mon.a (mon.0) 3203 : cluster [DBG] osdmap e630: 8 total, 8 up, 7 in 2024-02-23T00:52:48.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:47.273207+0000 mon.c (mon.2) 358 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-106"}]: dispatch 2024-02-23T00:52:48.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:47.274044+0000 mon.a (mon.0) 3204 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-106"}]: dispatch 2024-02-23T00:52:48.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:47.363101+0000 mgr.y (mgr.14517) 561 : cluster [DBG] pgmap v953: 184 pgs: 1 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 178 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.6 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-23T00:52:48.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:47.387316+0000 mon.c (mon.2) 359 : audit [INF] from='client.? 172.21.15.47:0/3578238271' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:52:48.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:47.387598+0000 mon.a (mon.0) 3205 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:52:48.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:47.270962+0000 mon.a (mon.0) 3202 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:52:48.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:47.271037+0000 mon.a (mon.0) 3203 : cluster [DBG] osdmap e630: 8 total, 8 up, 7 in 2024-02-23T00:52:48.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:47.273207+0000 mon.c (mon.2) 358 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-106"}]: dispatch 2024-02-23T00:52:48.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:47.274044+0000 mon.a (mon.0) 3204 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-106"}]: dispatch 2024-02-23T00:52:48.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:47.363101+0000 mgr.y (mgr.14517) 561 : cluster [DBG] pgmap v953: 184 pgs: 1 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 178 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.6 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-23T00:52:48.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:47.387316+0000 mon.c (mon.2) 359 : audit [INF] from='client.? 172.21.15.47:0/3578238271' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:52:48.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:47.387598+0000 mon.a (mon.0) 3205 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:52:48.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:47.270962+0000 mon.a (mon.0) 3202 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:52:48.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:47.271037+0000 mon.a (mon.0) 3203 : cluster [DBG] osdmap e630: 8 total, 8 up, 7 in 2024-02-23T00:52:48.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:47.273207+0000 mon.c (mon.2) 358 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-106"}]: dispatch 2024-02-23T00:52:48.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:47.274044+0000 mon.a (mon.0) 3204 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-106"}]: dispatch 2024-02-23T00:52:48.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:47.363101+0000 mgr.y (mgr.14517) 561 : cluster [DBG] pgmap v953: 184 pgs: 1 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 178 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.6 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-23T00:52:48.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:47.387316+0000 mon.c (mon.2) 359 : audit [INF] from='client.? 172.21.15.47:0/3578238271' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:52:48.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:47.387598+0000 mon.a (mon.0) 3205 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:52:49.182 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.2 dump_blocked_ops 2024-02-23T00:52:49.236 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:52:49.239 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:49.240 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:52:49.340 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:52:49.368 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-23T00:52:49.368 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-23T00:52:49.369 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-02-23T00:52:49.369 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-02-23T00:52:49.369 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-02-23T00:52:49.369 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.3 config set filestore_inject_stall 3 2024-02-23T00:52:49.370 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:49.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:48.268758+0000 mon.a (mon.0) 3206 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:52:49.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:48.268783+0000 mon.a (mon.0) 3207 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:52:49.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:48.278256+0000 mon.a (mon.0) 3208 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-106"}]': finished 2024-02-23T00:52:49.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:48.278330+0000 mon.a (mon.0) 3209 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:52:49.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:48.278379+0000 mon.a (mon.0) 3210 : cluster [DBG] osdmap e631: 8 total, 8 up, 7 in 2024-02-23T00:52:49.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:48.445903+0000 osd.5 (osd.5) 272 : cluster [DBG] 18.7 scrub starts 2024-02-23T00:52:49.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:48.449275+0000 osd.5 (osd.5) 273 : cluster [DBG] 18.7 scrub ok 2024-02-23T00:52:49.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:48.268758+0000 mon.a (mon.0) 3206 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:52:49.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:48.268783+0000 mon.a (mon.0) 3207 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:52:49.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:48.278256+0000 mon.a (mon.0) 3208 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-106"}]': finished 2024-02-23T00:52:49.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:48.278330+0000 mon.a (mon.0) 3209 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:52:49.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:48.278379+0000 mon.a (mon.0) 3210 : cluster [DBG] osdmap e631: 8 total, 8 up, 7 in 2024-02-23T00:52:49.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:48.445903+0000 osd.5 (osd.5) 272 : cluster [DBG] 18.7 scrub starts 2024-02-23T00:52:49.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:48.449275+0000 osd.5 (osd.5) 273 : cluster [DBG] 18.7 scrub ok 2024-02-23T00:52:49.582 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:52:49.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:48.268758+0000 mon.a (mon.0) 3206 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:52:49.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:48.268783+0000 mon.a (mon.0) 3207 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:52:49.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:48.278256+0000 mon.a (mon.0) 3208 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-106"}]': finished 2024-02-23T00:52:49.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:48.278330+0000 mon.a (mon.0) 3209 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:52:49.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:48.278379+0000 mon.a (mon.0) 3210 : cluster [DBG] osdmap e631: 8 total, 8 up, 7 in 2024-02-23T00:52:49.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:48.445903+0000 osd.5 (osd.5) 272 : cluster [DBG] 18.7 scrub starts 2024-02-23T00:52:49.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:48.449275+0000 osd.5 (osd.5) 273 : cluster [DBG] 18.7 scrub ok 2024-02-23T00:52:49.719 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:52:49.723 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:49.723 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:52:49.751 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:52:49.774 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:52:49.824 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:52:49.851 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:49.999 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:52:50.003 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:50.004 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:52:50.105 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:52:50.134 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:50.258 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:52:50.260 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:50.261 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:52:50.361 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:52:50.390 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:50.410 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:52:50.412 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:50.413 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:52:50.514 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:52:50.542 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:50.557 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:52:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:52:50] "GET /metrics HTTP/1.1" 200 190826 "" "Prometheus/2.33.4" 2024-02-23T00:52:50.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:48.748851+0000 osd.0 (osd.0) 263 : cluster [DBG] 18.e scrub starts 2024-02-23T00:52:50.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:48.749897+0000 osd.0 (osd.0) 264 : cluster [DBG] 18.e scrub ok 2024-02-23T00:52:50.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:49.277792+0000 mon.a (mon.0) 3211 : cluster [DBG] osdmap e632: 8 total, 8 up, 7 in 2024-02-23T00:52:50.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:49.363477+0000 mgr.y (mgr.14517) 562 : cluster [DBG] pgmap v956: 152 pgs: 3 active+clean+snaptrim, 149 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 767 B/s rd, 1 op/s 2024-02-23T00:52:50.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:49.464159+0000 osd.5 (osd.5) 274 : cluster [DBG] 2.3 scrub starts 2024-02-23T00:52:50.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:49.465041+0000 osd.5 (osd.5) 275 : cluster [DBG] 2.3 scrub ok 2024-02-23T00:52:50.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:48.748851+0000 osd.0 (osd.0) 263 : cluster [DBG] 18.e scrub starts 2024-02-23T00:52:50.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:48.749897+0000 osd.0 (osd.0) 264 : cluster [DBG] 18.e scrub ok 2024-02-23T00:52:50.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:49.277792+0000 mon.a (mon.0) 3211 : cluster [DBG] osdmap e632: 8 total, 8 up, 7 in 2024-02-23T00:52:50.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:49.363477+0000 mgr.y (mgr.14517) 562 : cluster [DBG] pgmap v956: 152 pgs: 3 active+clean+snaptrim, 149 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 767 B/s rd, 1 op/s 2024-02-23T00:52:50.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:49.464159+0000 osd.5 (osd.5) 274 : cluster [DBG] 2.3 scrub starts 2024-02-23T00:52:50.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:49.465041+0000 osd.5 (osd.5) 275 : cluster [DBG] 2.3 scrub ok 2024-02-23T00:52:50.560 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:52:50.562 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:48.748851+0000 osd.0 (osd.0) 263 : cluster [DBG] 18.e scrub starts 2024-02-23T00:52:50.562 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:48.749897+0000 osd.0 (osd.0) 264 : cluster [DBG] 18.e scrub ok 2024-02-23T00:52:50.563 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:49.277792+0000 mon.a (mon.0) 3211 : cluster [DBG] osdmap e632: 8 total, 8 up, 7 in 2024-02-23T00:52:50.563 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:49.363477+0000 mgr.y (mgr.14517) 562 : cluster [DBG] pgmap v956: 152 pgs: 3 active+clean+snaptrim, 149 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 767 B/s rd, 1 op/s 2024-02-23T00:52:50.563 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:49.464159+0000 osd.5 (osd.5) 274 : cluster [DBG] 2.3 scrub starts 2024-02-23T00:52:50.563 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:49.465041+0000 osd.5 (osd.5) 275 : cluster [DBG] 2.3 scrub ok 2024-02-23T00:52:50.563 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:50.564 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:52:50.664 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:52:50.693 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:50.746 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:52:50.749 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:50.749 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:52:50.850 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:52:50.877 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:51.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:49.495744+0000 osd.6 (osd.6) 241 : cluster [DBG] 2.1a scrub starts 2024-02-23T00:52:51.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:49.496732+0000 osd.6 (osd.6) 242 : cluster [DBG] 2.1a scrub ok 2024-02-23T00:52:51.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:49.886780+0000 osd.3 (osd.3) 302 : cluster [DBG] 4.10 scrub starts 2024-02-23T00:52:51.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:49.890249+0000 osd.3 (osd.3) 303 : cluster [DBG] 4.10 scrub ok 2024-02-23T00:52:51.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:50.290768+0000 mon.a (mon.0) 3212 : cluster [DBG] osdmap e633: 8 total, 8 up, 7 in 2024-02-23T00:52:51.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:50.292747+0000 mon.c (mon.2) 360 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-108","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:52:51.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:50.293247+0000 mon.a (mon.0) 3213 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-108","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:52:51.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:50.379193+0000 osd.7 (osd.7) 301 : cluster [DBG] 18.13 scrub starts 2024-02-23T00:52:51.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:50.382422+0000 osd.7 (osd.7) 302 : cluster [DBG] 18.13 scrub ok 2024-02-23T00:52:51.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:50.466972+0000 osd.5 (osd.5) 276 : cluster [DBG] 18.c scrub starts 2024-02-23T00:52:51.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:50.469852+0000 osd.5 (osd.5) 277 : cluster [DBG] 18.c scrub ok 2024-02-23T00:52:51.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:49.495744+0000 osd.6 (osd.6) 241 : cluster [DBG] 2.1a scrub starts 2024-02-23T00:52:51.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:49.496732+0000 osd.6 (osd.6) 242 : cluster [DBG] 2.1a scrub ok 2024-02-23T00:52:51.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:49.886780+0000 osd.3 (osd.3) 302 : cluster [DBG] 4.10 scrub starts 2024-02-23T00:52:51.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:49.890249+0000 osd.3 (osd.3) 303 : cluster [DBG] 4.10 scrub ok 2024-02-23T00:52:51.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:50.290768+0000 mon.a (mon.0) 3212 : cluster [DBG] osdmap e633: 8 total, 8 up, 7 in 2024-02-23T00:52:51.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:50.292747+0000 mon.c (mon.2) 360 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-108","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:52:51.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:50.293247+0000 mon.a (mon.0) 3213 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-108","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:52:51.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:50.379193+0000 osd.7 (osd.7) 301 : cluster [DBG] 18.13 scrub starts 2024-02-23T00:52:51.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:50.382422+0000 osd.7 (osd.7) 302 : cluster [DBG] 18.13 scrub ok 2024-02-23T00:52:51.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:50.466972+0000 osd.5 (osd.5) 276 : cluster [DBG] 18.c scrub starts 2024-02-23T00:52:51.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:50.469852+0000 osd.5 (osd.5) 277 : cluster [DBG] 18.c scrub ok 2024-02-23T00:52:51.671 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-23T00:52:51.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:49.495744+0000 osd.6 (osd.6) 241 : cluster [DBG] 2.1a scrub starts 2024-02-23T00:52:51.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:49.496732+0000 osd.6 (osd.6) 242 : cluster [DBG] 2.1a scrub ok 2024-02-23T00:52:51.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:49.886780+0000 osd.3 (osd.3) 302 : cluster [DBG] 4.10 scrub starts 2024-02-23T00:52:51.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:49.890249+0000 osd.3 (osd.3) 303 : cluster [DBG] 4.10 scrub ok 2024-02-23T00:52:51.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:50.290768+0000 mon.a (mon.0) 3212 : cluster [DBG] osdmap e633: 8 total, 8 up, 7 in 2024-02-23T00:52:51.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:50.292747+0000 mon.c (mon.2) 360 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-108","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:52:51.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:50.293247+0000 mon.a (mon.0) 3213 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-108","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:52:51.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:50.379193+0000 osd.7 (osd.7) 301 : cluster [DBG] 18.13 scrub starts 2024-02-23T00:52:51.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:50.382422+0000 osd.7 (osd.7) 302 : cluster [DBG] 18.13 scrub ok 2024-02-23T00:52:51.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:50.466972+0000 osd.5 (osd.5) 276 : cluster [DBG] 18.c scrub starts 2024-02-23T00:52:51.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:50.469852+0000 osd.5 (osd.5) 277 : cluster [DBG] 18.c scrub ok 2024-02-23T00:52:51.736 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:52:51.744 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:52:51.748 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:51.748 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:52:51.849 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:52:51.876 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:51.932 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:52:51.938 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:51.939 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:52:52.040 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:52:52.076 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:52.099 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:52:52.102 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:52.102 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:52:52.203 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:52:52.234 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:52.254 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:52:52.257 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:52.257 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:52:52.358 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:52:52.366 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:52:52.366 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "" 2024-02-23T00:52:52.366 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:52:52.384 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:50.532543+0000 osd.6 (osd.6) 243 : cluster [DBG] 4.a scrub starts 2024-02-23T00:52:52.384 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:50.534763+0000 osd.6 (osd.6) 244 : cluster [DBG] 4.a scrub ok 2024-02-23T00:52:52.384 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:51.295770+0000 mon.a (mon.0) 3214 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-108","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:52:52.384 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:51.295937+0000 mon.a (mon.0) 3215 : cluster [DBG] osdmap e634: 8 total, 8 up, 7 in 2024-02-23T00:52:52.384 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:51.364077+0000 mgr.y (mgr.14517) 563 : cluster [DBG] pgmap v959: 184 pgs: 32 unknown, 3 active+clean+snaptrim, 149 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:52:52.384 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:51.377868+0000 osd.7 (osd.7) 303 : cluster [DBG] 2.5 scrub starts 2024-02-23T00:52:52.384 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:51.380771+0000 osd.7 (osd.7) 304 : cluster [DBG] 2.5 scrub ok 2024-02-23T00:52:52.384 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:51.436944+0000 osd.5 (osd.5) 278 : cluster [DBG] 18.1e scrub starts 2024-02-23T00:52:52.384 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:51.439814+0000 osd.5 (osd.5) 279 : cluster [DBG] 18.1e scrub ok 2024-02-23T00:52:52.385 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:51.721406+0000 osd.4 (osd.4) 265 : cluster [DBG] 5.1 scrub starts 2024-02-23T00:52:52.385 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:51.722285+0000 osd.4 (osd.4) 266 : cluster [DBG] 5.1 scrub ok 2024-02-23T00:52:52.386 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:52.405 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:52:52.407 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:52.408 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:52:52.493 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:50.532543+0000 osd.6 (osd.6) 243 : cluster [DBG] 4.a scrub starts 2024-02-23T00:52:52.493 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:50.534763+0000 osd.6 (osd.6) 244 : cluster [DBG] 4.a scrub ok 2024-02-23T00:52:52.493 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:51.295770+0000 mon.a (mon.0) 3214 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-108","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:52:52.493 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:51.295937+0000 mon.a (mon.0) 3215 : cluster [DBG] osdmap e634: 8 total, 8 up, 7 in 2024-02-23T00:52:52.493 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:51.364077+0000 mgr.y (mgr.14517) 563 : cluster [DBG] pgmap v959: 184 pgs: 32 unknown, 3 active+clean+snaptrim, 149 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:52:52.493 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:51.377868+0000 osd.7 (osd.7) 303 : cluster [DBG] 2.5 scrub starts 2024-02-23T00:52:52.493 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:51.380771+0000 osd.7 (osd.7) 304 : cluster [DBG] 2.5 scrub ok 2024-02-23T00:52:52.494 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:51.436944+0000 osd.5 (osd.5) 278 : cluster [DBG] 18.1e scrub starts 2024-02-23T00:52:52.494 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:51.439814+0000 osd.5 (osd.5) 279 : cluster [DBG] 18.1e scrub ok 2024-02-23T00:52:52.494 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:51.721406+0000 osd.4 (osd.4) 265 : cluster [DBG] 5.1 scrub starts 2024-02-23T00:52:52.494 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:51.722285+0000 osd.4 (osd.4) 266 : cluster [DBG] 5.1 scrub ok 2024-02-23T00:52:52.494 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:50.532543+0000 osd.6 (osd.6) 243 : cluster [DBG] 4.a scrub starts 2024-02-23T00:52:52.494 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:50.534763+0000 osd.6 (osd.6) 244 : cluster [DBG] 4.a scrub ok 2024-02-23T00:52:52.494 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:51.295770+0000 mon.a (mon.0) 3214 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-108","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:52:52.494 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:51.295937+0000 mon.a (mon.0) 3215 : cluster [DBG] osdmap e634: 8 total, 8 up, 7 in 2024-02-23T00:52:52.494 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:51.364077+0000 mgr.y (mgr.14517) 563 : cluster [DBG] pgmap v959: 184 pgs: 32 unknown, 3 active+clean+snaptrim, 149 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:52:52.494 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:51.377868+0000 osd.7 (osd.7) 303 : cluster [DBG] 2.5 scrub starts 2024-02-23T00:52:52.494 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:51.380771+0000 osd.7 (osd.7) 304 : cluster [DBG] 2.5 scrub ok 2024-02-23T00:52:52.494 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:51.436944+0000 osd.5 (osd.5) 278 : cluster [DBG] 18.1e scrub starts 2024-02-23T00:52:52.494 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:51.439814+0000 osd.5 (osd.5) 279 : cluster [DBG] 18.1e scrub ok 2024-02-23T00:52:52.495 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:51.721406+0000 osd.4 (osd.4) 265 : cluster [DBG] 5.1 scrub starts 2024-02-23T00:52:52.495 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:51.722285+0000 osd.4 (osd.4) 266 : cluster [DBG] 5.1 scrub ok 2024-02-23T00:52:52.509 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:52:52.535 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:52.554 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:52:52.557 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:52.557 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:52:52.658 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:52:52.690 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:52.694 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:52:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:52:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:52:52.789 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:52:52.791 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:52.792 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:52:52.893 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:52:52.920 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:52.939 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:52:52.942 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:52.943 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:52:53.043 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:52:53.070 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:53.088 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:52:53.090 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:52:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:52:53.091 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:53.091 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:52:53.192 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:52:53.219 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:53.237 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:52:53.239 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:53.240 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:52:53.341 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:52:53.365 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:51.509911+0000 osd.6 (osd.6) 245 : cluster [DBG] 18.10 scrub starts 2024-02-23T00:52:53.365 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:51.510964+0000 osd.6 (osd.6) 246 : cluster [DBG] 18.10 scrub ok 2024-02-23T00:52:53.365 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:52.301673+0000 mon.a (mon.0) 3216 : cluster [DBG] osdmap e635: 8 total, 8 up, 7 in 2024-02-23T00:52:53.365 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:52.308672+0000 mon.c (mon.2) 361 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-108", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:52:53.365 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:52.309025+0000 mon.a (mon.0) 3217 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-108", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:52:53.365 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:52.332498+0000 osd.7 (osd.7) 305 : cluster [DBG] 2.1d scrub starts 2024-02-23T00:52:53.365 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:52.335387+0000 osd.7 (osd.7) 306 : cluster [DBG] 2.1d scrub ok 2024-02-23T00:52:53.365 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:52.758426+0000 osd.4 (osd.4) 267 : cluster [DBG] 4.13 scrub starts 2024-02-23T00:52:53.366 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:52.760042+0000 osd.4 (osd.4) 268 : cluster [DBG] 4.13 scrub ok 2024-02-23T00:52:53.367 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:53.388 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:52:53.390 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:53.391 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:52:53.492 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:52:53.517 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:53.536 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:52:53.539 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:53.540 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:52:53.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:51.509911+0000 osd.6 (osd.6) 245 : cluster [DBG] 18.10 scrub starts 2024-02-23T00:52:53.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:51.510964+0000 osd.6 (osd.6) 246 : cluster [DBG] 18.10 scrub ok 2024-02-23T00:52:53.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:52.301673+0000 mon.a (mon.0) 3216 : cluster [DBG] osdmap e635: 8 total, 8 up, 7 in 2024-02-23T00:52:53.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:52.308672+0000 mon.c (mon.2) 361 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-108", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:52:53.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:52.309025+0000 mon.a (mon.0) 3217 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-108", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:52:53.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:52.332498+0000 osd.7 (osd.7) 305 : cluster [DBG] 2.1d scrub starts 2024-02-23T00:52:53.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:52.335387+0000 osd.7 (osd.7) 306 : cluster [DBG] 2.1d scrub ok 2024-02-23T00:52:53.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:52.758426+0000 osd.4 (osd.4) 267 : cluster [DBG] 4.13 scrub starts 2024-02-23T00:52:53.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:52.760042+0000 osd.4 (osd.4) 268 : cluster [DBG] 4.13 scrub ok 2024-02-23T00:52:53.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:51.509911+0000 osd.6 (osd.6) 245 : cluster [DBG] 18.10 scrub starts 2024-02-23T00:52:53.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:51.510964+0000 osd.6 (osd.6) 246 : cluster [DBG] 18.10 scrub ok 2024-02-23T00:52:53.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:52.301673+0000 mon.a (mon.0) 3216 : cluster [DBG] osdmap e635: 8 total, 8 up, 7 in 2024-02-23T00:52:53.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:52.308672+0000 mon.c (mon.2) 361 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-108", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:52:53.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:52.309025+0000 mon.a (mon.0) 3217 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-108", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:52:53.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:52.332498+0000 osd.7 (osd.7) 305 : cluster [DBG] 2.1d scrub starts 2024-02-23T00:52:53.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:52.335387+0000 osd.7 (osd.7) 306 : cluster [DBG] 2.1d scrub ok 2024-02-23T00:52:53.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:52.758426+0000 osd.4 (osd.4) 267 : cluster [DBG] 4.13 scrub starts 2024-02-23T00:52:53.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:52.760042+0000 osd.4 (osd.4) 268 : cluster [DBG] 4.13 scrub ok 2024-02-23T00:52:53.640 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:52:53.667 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:53.685 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:52:53.688 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:53.688 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:52:53.789 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:52:53.821 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:53.970 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:52:53.973 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:53.973 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:52:54.074 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:52:54.103 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:54.122 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:52:54.124 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:54.125 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:52:54.226 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:52:54.253 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:54.272 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:52:54.274 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:54.275 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:52:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:52:54.375 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:52:54.401 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:52.828724+0000 mgr.y (mgr.14517) 564 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:52:54.401 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:53.305537+0000 mon.a (mon.0) 3218 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-108", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:52:54.401 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:53.305621+0000 mon.a (mon.0) 3219 : cluster [DBG] osdmap e636: 8 total, 8 up, 7 in 2024-02-23T00:52:54.401 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:53.306374+0000 mon.c (mon.2) 362 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-108"}]: dispatch 2024-02-23T00:52:54.401 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:53.306685+0000 mon.a (mon.0) 3220 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-108"}]: dispatch 2024-02-23T00:52:54.401 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:53.364444+0000 mgr.y (mgr.14517) 565 : cluster [DBG] pgmap v962: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:52:54.402 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:53.805955+0000 osd.4 (osd.4) 269 : cluster [DBG] 4.c scrub starts 2024-02-23T00:52:54.402 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:53.807330+0000 osd.4 (osd.4) 270 : cluster [DBG] 4.c scrub ok 2024-02-23T00:52:54.402 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:53.879929+0000 osd.3 (osd.3) 304 : cluster [DBG] 3.2 scrub starts 2024-02-23T00:52:54.402 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:53.881149+0000 osd.3 (osd.3) 305 : cluster [DBG] 3.2 scrub ok 2024-02-23T00:52:54.403 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:54.422 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:52:54.425 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:54.425 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:52:54.526 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:52:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:54.562 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:52.828724+0000 mgr.y (mgr.14517) 564 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:52:54.562 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:53.305537+0000 mon.a (mon.0) 3218 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-108", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:52:54.563 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:53.305621+0000 mon.a (mon.0) 3219 : cluster [DBG] osdmap e636: 8 total, 8 up, 7 in 2024-02-23T00:52:54.563 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:53.306374+0000 mon.c (mon.2) 362 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-108"}]: dispatch 2024-02-23T00:52:54.563 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:53.306685+0000 mon.a (mon.0) 3220 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-108"}]: dispatch 2024-02-23T00:52:54.563 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:53.364444+0000 mgr.y (mgr.14517) 565 : cluster [DBG] pgmap v962: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:52:54.563 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:53.805955+0000 osd.4 (osd.4) 269 : cluster [DBG] 4.c scrub starts 2024-02-23T00:52:54.563 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:53.807330+0000 osd.4 (osd.4) 270 : cluster [DBG] 4.c scrub ok 2024-02-23T00:52:54.563 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:53.879929+0000 osd.3 (osd.3) 304 : cluster [DBG] 3.2 scrub starts 2024-02-23T00:52:54.563 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:53.881149+0000 osd.3 (osd.3) 305 : cluster [DBG] 3.2 scrub ok 2024-02-23T00:52:54.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:52.828724+0000 mgr.y (mgr.14517) 564 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:52:54.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:53.305537+0000 mon.a (mon.0) 3218 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-108", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:52:54.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:53.305621+0000 mon.a (mon.0) 3219 : cluster [DBG] osdmap e636: 8 total, 8 up, 7 in 2024-02-23T00:52:54.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:53.306374+0000 mon.c (mon.2) 362 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-108"}]: dispatch 2024-02-23T00:52:54.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:53.306685+0000 mon.a (mon.0) 3220 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-108"}]: dispatch 2024-02-23T00:52:54.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:53.364444+0000 mgr.y (mgr.14517) 565 : cluster [DBG] pgmap v962: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:52:54.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:53.805955+0000 osd.4 (osd.4) 269 : cluster [DBG] 4.c scrub starts 2024-02-23T00:52:54.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:53.807330+0000 osd.4 (osd.4) 270 : cluster [DBG] 4.c scrub ok 2024-02-23T00:52:54.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:53.879929+0000 osd.3 (osd.3) 304 : cluster [DBG] 3.2 scrub starts 2024-02-23T00:52:54.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:53.881149+0000 osd.3 (osd.3) 305 : cluster [DBG] 3.2 scrub ok 2024-02-23T00:52:54.813 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:52:54.815 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:54.816 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:52:54.916 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:52:54.944 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:54.963 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:52:54.966 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:54.966 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:52:55.067 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:52:55.094 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:55.112 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:52:55.115 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:55.115 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:52:55.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.145+0000 7fbfc11aa700 1 -- 172.21.15.47:0/763696332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfbc0f8d30 msgr2=0x7fbfbc0f9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:55.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.145+0000 7fbfc11aa700 1 --2- 172.21.15.47:0/763696332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfbc0f8d30 0x7fbfbc0f9110 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 rx=0x7fbfac0047b0 tx=0x7fbfac009cb0).stop 2024-02-23T00:52:55.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.145+0000 7fbfc11aa700 1 -- 172.21.15.47:0/763696332 shutdown_connections 2024-02-23T00:52:55.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.145+0000 7fbfc11aa700 1 --2- 172.21.15.47:0/763696332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfbc102290 0x7fbfbc104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:55.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.145+0000 7fbfc11aa700 1 --2- 172.21.15.47:0/763696332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfbc0f8d30 0x7fbfbc0f9110 unknown :-1 s=CLOSED pgs=2600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:55.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.145+0000 7fbfc11aa700 1 --2- 172.21.15.47:0/763696332 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbfbc0f9600 0x7fbfbc101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:55.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.145+0000 7fbfc11aa700 1 -- 172.21.15.47:0/763696332 >> 172.21.15.47:0/763696332 conn(0x7fbfbc0f2820 msgr2=0x7fbfbc0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:52:55.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.145+0000 7fbfc11aa700 1 -- 172.21.15.47:0/763696332 shutdown_connections 2024-02-23T00:52:55.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.145+0000 7fbfc11aa700 1 -- 172.21.15.47:0/763696332 wait complete. 2024-02-23T00:52:55.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.145+0000 7fbfc11aa700 1 librados: starting msgr at 2024-02-23T00:52:55.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.145+0000 7fbfc11aa700 1 librados: starting objecter 2024-02-23T00:52:55.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.146+0000 7fbfc11aa700 1 Processor -- start 2024-02-23T00:52:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.146+0000 7fbfc11aa700 1 -- start start 2024-02-23T00:52:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.146+0000 7fbfc11aa700 1 librados: setting wanted keys 2024-02-23T00:52:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.146+0000 7fbfc11aa700 1 librados: calling monclient init 2024-02-23T00:52:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.146+0000 7fbfc11aa700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfbc0f8d30 0x7fbfbc110260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.146+0000 7fbfc11aa700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfbc0f9600 0x7fbfbc110750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.146+0000 7fbfc11aa700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbfbc102290 0x7fbfbc10a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.146+0000 7fbfc11aa700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc10a8f0 con 0x7fbfbc0f9600 2024-02-23T00:52:55.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.146+0000 7fbfc11aa700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc10aa60 con 0x7fbfbc0f8d30 2024-02-23T00:52:55.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.146+0000 7fbfc11aa700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc10ac20 con 0x7fbfbc102290 2024-02-23T00:52:55.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.146+0000 7fbfba59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfbc0f9600 0x7fbfbc110750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:55.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.146+0000 7fbfba59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfbc0f9600 0x7fbfbc110750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46056/0 (socket says 172.21.15.47:46056) 2024-02-23T00:52:55.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.146+0000 7fbfba59c700 1 -- 172.21.15.47:0/3577590144 learned_addr learned my addr 172.21.15.47:0/3577590144 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:52:55.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.146+0000 7fbfbb59e700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbfbc102290 0x7fbfbc10a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:55.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.147+0000 7fbfbad9d700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfbc0f8d30 0x7fbfbc110260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:55.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.147+0000 7fbfba59c700 1 -- 172.21.15.47:0/3577590144 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbfbc102290 msgr2=0x7fbfbc10a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:55.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.147+0000 7fbfba59c700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbfbc102290 0x7fbfbc10a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:55.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.147+0000 7fbfba59c700 1 -- 172.21.15.47:0/3577590144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfbc0f8d30 msgr2=0x7fbfbc110260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:55.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.147+0000 7fbfba59c700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfbc0f8d30 0x7fbfbc110260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:55.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.147+0000 7fbfba59c700 1 -- 172.21.15.47:0/3577590144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa4005040 con 0x7fbfbc0f9600 2024-02-23T00:52:55.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.147+0000 7fbfbb59e700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbfbc102290 0x7fbfbc10a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:52:55.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.147+0000 7fbfba59c700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfbc0f9600 0x7fbfbc110750 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 rx=0x7fbfa4009400 tx=0x7fbfa4009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:55.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.147+0000 7fbfb3fff700 10 client.?.objecter ms_handle_connect 0x7fbfbc0f9600 2024-02-23T00:52:55.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.147+0000 7fbfb3fff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:52:55.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.147+0000 7fbfb3fff700 1 -- 172.21.15.47:0/3577590144 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4007a40 con 0x7fbfbc0f9600 2024-02-23T00:52:55.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.147+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfac005040 con 0x7fbfbc0f9600 2024-02-23T00:52:55.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.147+0000 7fbfb3fff700 1 -- 172.21.15.47:0/3577590144 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fbfa400f040 con 0x7fbfbc0f9600 2024-02-23T00:52:55.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.147+0000 7fbfc11aa700 10 client.31518.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:52:55.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.147+0000 7fbfb3fff700 1 -- 172.21.15.47:0/3577590144 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4014990 con 0x7fbfbc0f9600 2024-02-23T00:52:55.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.147+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfbc117090 con 0x7fbfbc0f9600 2024-02-23T00:52:55.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.147+0000 7fbfc11aa700 1 librados: init done 2024-02-23T00:52:55.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.148+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fbfbc05fa60 con 0x7fbfbc0f9600 2024-02-23T00:52:55.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.149+0000 7fbfb3fff700 1 -- 172.21.15.47:0/3577590144 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fbfa400e630 con 0x7fbfbc0f9600 2024-02-23T00:52:55.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.149+0000 7fbfb3fff700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbfa8062cc0 0x7fbfa8065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:55.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.149+0000 7fbfb3fff700 1 -- 172.21.15.47:0/3577590144 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(637..637 src has 482..637) v4 ==== 9634+0+0 (secure 0 0 0) 0x7fbfa407d9d0 con 0x7fbfbc0f9600 2024-02-23T00:52:55.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.149+0000 7fbfb3fff700 10 client.31518.objecter ms_dispatch 0x7fbfbc001550 osd_map(637..637 src has 482..637) v4 2024-02-23T00:52:55.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.149+0000 7fbfb3fff700 3 client.31518.objecter handle_osd_map got epochs [637,637] > 0 2024-02-23T00:52:55.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.149+0000 7fbfb3fff700 3 client.31518.objecter handle_osd_map decoding full epoch 637 2024-02-23T00:52:55.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.149+0000 7fbfbad9d700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbfa8062cc0 0x7fbfa8065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:55.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.149+0000 7fbfb3fff700 20 client.31518.objecter dump_active .. 0 homeless 2024-02-23T00:52:55.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.149+0000 7fbfb3fff700 1 -- 172.21.15.47:0/3577590144 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v637) v1 ==== 54+0+16 (secure 0 0 0) 0x7fbfa407e0a0 con 0x7fbfbc0f9600 2024-02-23T00:52:55.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.149+0000 7fbfb3fff700 10 client.31518.objecter ms_handle_connect 0x7fbfa8062cc0 2024-02-23T00:52:55.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.149+0000 7fbfbad9d700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbfa8062cc0 0x7fbfa8065180 secure :-1 s=READY pgs=3149 cs=0 l=1 rev1=1 rx=0x7fbfac004e20 tx=0x7fbfac009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:55.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.149+0000 7fbfc11aa700 10 client.31518.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:55.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.149+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_assign -1 1 2024-02-23T00:52:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.149+0000 7fbfc11aa700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fbfbc0fe5f0 0x7fbfbc0fe9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.149+0000 7fbfc11aa700 20 client.31518.objecter _get_session s=0x7fbfbc05ee80 osd=0 3 2024-02-23T00:52:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.149+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc05ee80 osd=0 3 2024-02-23T00:52:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.149+0000 7fbfc11aa700 20 client.31518.objecter _get_session s=0x7fbfbc05ee80 osd=0 3 2024-02-23T00:52:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.149+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_remove -1 1 2024-02-23T00:52:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.149+0000 7fbfc11aa700 20 client.31518.objecter get_session s=0x7fbfbc05ee80 osd=0 3 2024-02-23T00:52:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.149+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_assign 0 1 2024-02-23T00:52:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.149+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc05ee80 osd=0 4 2024-02-23T00:52:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.149+0000 7fbfc11aa700 10 client.31518.objecter _send_command 1 2024-02-23T00:52:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.149+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbfbc107040 con 0x7fbfbc0fe5f0 2024-02-23T00:52:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.150+0000 7fbfbb59e700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fbfbc0fe5f0 0x7fbfbc0fe9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.150+0000 7fbfb3fff700 10 client.31518.objecter ms_handle_connect 0x7fbfbc0fe5f0 2024-02-23T00:52:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.150+0000 7fbfbb59e700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fbfbc0fe5f0 0x7fbfbc0fe9d0 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.150+0000 7fbfb3fff700 1 -- 172.21.15.47:0/3577590144 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fbfbc107040 con 0x7fbfbc0fe5f0 2024-02-23T00:52:55.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.150+0000 7fbfb3fff700 10 client.31518.objecter ms_dispatch 0x7fbfbc001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:52:55.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.150+0000 7fbfb3fff700 10 client.31518.objecter _finish_command 1 = system:0 2024-02-23T00:52:55.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.150+0000 7fbfb3fff700 20 client.31518.objecter put_session s=0x7fbfbc05ee80 osd=0 4 2024-02-23T00:52:55.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.150+0000 7fbfb3fff700 15 client.31518.objecter _session_command_op_remove 0 1 2024-02-23T00:52:55.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.166+0000 7fbfc11aa700 10 client.31518.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:52:55.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.166+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_assign -1 2 2024-02-23T00:52:55.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.166+0000 7fbfc11aa700 20 client.31518.objecter _get_session s=0x7fbfbc05ee80 osd=0 3 2024-02-23T00:52:55.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.166+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc05ee80 osd=0 3 2024-02-23T00:52:55.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.166+0000 7fbfc11aa700 20 client.31518.objecter _get_session s=0x7fbfbc05ee80 osd=0 3 2024-02-23T00:52:55.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.166+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_remove -1 2 2024-02-23T00:52:55.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.166+0000 7fbfc11aa700 20 client.31518.objecter get_session s=0x7fbfbc05ee80 osd=0 3 2024-02-23T00:52:55.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.166+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_assign 0 2 2024-02-23T00:52:55.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.166+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc05ee80 osd=0 4 2024-02-23T00:52:55.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.166+0000 7fbfc11aa700 10 client.31518.objecter _send_command 2 2024-02-23T00:52:55.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.166+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fbfbc0feec0 con 0x7fbfbc0fe5f0 2024-02-23T00:52:55.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.168+0000 7fbfb3fff700 1 -- 172.21.15.47:0/3577590144 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fbfbc0feec0 con 0x7fbfbc0fe5f0 2024-02-23T00:52:55.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.168+0000 7fbfb3fff700 10 client.31518.objecter ms_dispatch 0x7fbfbc001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:52:55.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.168+0000 7fbfb3fff700 10 client.31518.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:52:55.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.168+0000 7fbfb3fff700 20 client.31518.objecter put_session s=0x7fbfbc05ee80 osd=0 4 2024-02-23T00:52:55.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.168+0000 7fbfb3fff700 15 client.31518.objecter _session_command_op_remove 0 2 2024-02-23T00:52:55.171 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:52:55.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.168+0000 7fbfc11aa700 10 client.31518.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:55.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.168+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_assign -1 3 2024-02-23T00:52:55.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.168+0000 7fbfc11aa700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fbfbc028540 0x7fbfbc0598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.168+0000 7fbfc11aa700 20 client.31518.objecter _get_session s=0x7fbfbc0ff090 osd=1 3 2024-02-23T00:52:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.168+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc0ff090 osd=1 3 2024-02-23T00:52:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.168+0000 7fbfc11aa700 20 client.31518.objecter _get_session s=0x7fbfbc0ff090 osd=1 3 2024-02-23T00:52:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.168+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_remove -1 3 2024-02-23T00:52:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.168+0000 7fbfc11aa700 20 client.31518.objecter get_session s=0x7fbfbc0ff090 osd=1 3 2024-02-23T00:52:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.168+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_assign 1 3 2024-02-23T00:52:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.168+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc0ff090 osd=1 4 2024-02-23T00:52:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.168+0000 7fbfc11aa700 10 client.31518.objecter _send_command 3 2024-02-23T00:52:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.168+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbfbc059de0 con 0x7fbfbc028540 2024-02-23T00:52:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.168+0000 7fbfbad9d700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fbfbc028540 0x7fbfbc0598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.168+0000 7fbfb3fff700 10 client.31518.objecter ms_handle_connect 0x7fbfbc028540 2024-02-23T00:52:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.168+0000 7fbfbad9d700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fbfbc028540 0x7fbfbc0598f0 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.169+0000 7fbfb3fff700 1 -- 172.21.15.47:0/3577590144 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fbfbc059de0 con 0x7fbfbc028540 2024-02-23T00:52:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.169+0000 7fbfb3fff700 10 client.31518.objecter ms_dispatch 0x7fbfbc001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:52:55.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.169+0000 7fbfb3fff700 10 client.31518.objecter _finish_command 3 = system:0 2024-02-23T00:52:55.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.169+0000 7fbfb3fff700 20 client.31518.objecter put_session s=0x7fbfbc0ff090 osd=1 4 2024-02-23T00:52:55.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.169+0000 7fbfb3fff700 15 client.31518.objecter _session_command_op_remove 1 3 2024-02-23T00:52:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.184+0000 7fbfc11aa700 10 client.31518.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:52:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.184+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_assign -1 4 2024-02-23T00:52:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.184+0000 7fbfc11aa700 20 client.31518.objecter _get_session s=0x7fbfbc0ff090 osd=1 3 2024-02-23T00:52:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.184+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc0ff090 osd=1 3 2024-02-23T00:52:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.184+0000 7fbfc11aa700 20 client.31518.objecter _get_session s=0x7fbfbc0ff090 osd=1 3 2024-02-23T00:52:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.184+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_remove -1 4 2024-02-23T00:52:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.184+0000 7fbfc11aa700 20 client.31518.objecter get_session s=0x7fbfbc0ff090 osd=1 3 2024-02-23T00:52:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.184+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_assign 1 4 2024-02-23T00:52:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.184+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc0ff090 osd=1 4 2024-02-23T00:52:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.184+0000 7fbfc11aa700 10 client.31518.objecter _send_command 4 2024-02-23T00:52:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.184+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fbfbc11b240 con 0x7fbfbc028540 2024-02-23T00:52:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.185+0000 7fbfb3fff700 1 -- 172.21.15.47:0/3577590144 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fbfbc11b240 con 0x7fbfbc028540 2024-02-23T00:52:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.185+0000 7fbfb3fff700 10 client.31518.objecter ms_dispatch 0x7fbfbc001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:52:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.185+0000 7fbfb3fff700 10 client.31518.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:52:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.185+0000 7fbfb3fff700 20 client.31518.objecter put_session s=0x7fbfbc0ff090 osd=1 4 2024-02-23T00:52:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.185+0000 7fbfb3fff700 15 client.31518.objecter _session_command_op_remove 1 4 2024-02-23T00:52:55.189 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:52:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.185+0000 7fbfc11aa700 10 client.31518.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.185+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_assign -1 5 2024-02-23T00:52:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.185+0000 7fbfc11aa700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fbfbc0594d0 0x7fbfbc11e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.185+0000 7fbfc11aa700 20 client.31518.objecter _get_session s=0x7fbfbc059360 osd=2 3 2024-02-23T00:52:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.185+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc059360 osd=2 3 2024-02-23T00:52:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.185+0000 7fbfc11aa700 20 client.31518.objecter _get_session s=0x7fbfbc059360 osd=2 3 2024-02-23T00:52:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.185+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_remove -1 5 2024-02-23T00:52:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.185+0000 7fbfc11aa700 20 client.31518.objecter get_session s=0x7fbfbc059360 osd=2 3 2024-02-23T00:52:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.185+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_assign 2 5 2024-02-23T00:52:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.185+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc059360 osd=2 4 2024-02-23T00:52:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.185+0000 7fbfc11aa700 10 client.31518.objecter _send_command 5 2024-02-23T00:52:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.185+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbfbc028240 con 0x7fbfbc0594d0 2024-02-23T00:52:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.185+0000 7fbfba59c700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fbfbc0594d0 0x7fbfbc11e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.186+0000 7fbfb3fff700 10 client.31518.objecter ms_handle_connect 0x7fbfbc0594d0 2024-02-23T00:52:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.186+0000 7fbfba59c700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fbfbc0594d0 0x7fbfbc11e5a0 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.186+0000 7fbfb3fff700 1 -- 172.21.15.47:0/3577590144 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fbfbc028240 con 0x7fbfbc0594d0 2024-02-23T00:52:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.186+0000 7fbfb3fff700 10 client.31518.objecter ms_dispatch 0x7fbfbc001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:52:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.186+0000 7fbfb3fff700 10 client.31518.objecter _finish_command 5 = system:0 2024-02-23T00:52:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.186+0000 7fbfb3fff700 20 client.31518.objecter put_session s=0x7fbfbc059360 osd=2 4 2024-02-23T00:52:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.186+0000 7fbfb3fff700 15 client.31518.objecter _session_command_op_remove 2 5 2024-02-23T00:52:55.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.201+0000 7fbfc11aa700 10 client.31518.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:52:55.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.201+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_assign -1 6 2024-02-23T00:52:55.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.201+0000 7fbfc11aa700 20 client.31518.objecter _get_session s=0x7fbfbc059360 osd=2 3 2024-02-23T00:52:55.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.201+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc059360 osd=2 3 2024-02-23T00:52:55.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.201+0000 7fbfc11aa700 20 client.31518.objecter _get_session s=0x7fbfbc059360 osd=2 3 2024-02-23T00:52:55.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.201+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_remove -1 6 2024-02-23T00:52:55.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.201+0000 7fbfc11aa700 20 client.31518.objecter get_session s=0x7fbfbc059360 osd=2 3 2024-02-23T00:52:55.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.201+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_assign 2 6 2024-02-23T00:52:55.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.201+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc059360 osd=2 4 2024-02-23T00:52:55.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.201+0000 7fbfc11aa700 10 client.31518.objecter _send_command 6 2024-02-23T00:52:55.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.201+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fbfbc027600 con 0x7fbfbc0594d0 2024-02-23T00:52:55.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.202+0000 7fbfb3fff700 1 -- 172.21.15.47:0/3577590144 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fbfbc027600 con 0x7fbfbc0594d0 2024-02-23T00:52:55.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.202+0000 7fbfb3fff700 10 client.31518.objecter ms_dispatch 0x7fbfbc001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:52:55.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.202+0000 7fbfb3fff700 10 client.31518.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:52:55.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.202+0000 7fbfb3fff700 20 client.31518.objecter put_session s=0x7fbfbc059360 osd=2 4 2024-02-23T00:52:55.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.202+0000 7fbfb3fff700 15 client.31518.objecter _session_command_op_remove 2 6 2024-02-23T00:52:55.205 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:52:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.202+0000 7fbfc11aa700 10 client.31518.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.202+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_assign -1 7 2024-02-23T00:52:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.202+0000 7fbfc11aa700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fbfbc11c090 0x7fbfbc121c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.202+0000 7fbfc11aa700 20 client.31518.objecter _get_session s=0x7fbfbc0279b0 osd=3 3 2024-02-23T00:52:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.202+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc0279b0 osd=3 3 2024-02-23T00:52:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.202+0000 7fbfc11aa700 20 client.31518.objecter _get_session s=0x7fbfbc0279b0 osd=3 3 2024-02-23T00:52:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.202+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_remove -1 7 2024-02-23T00:52:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.202+0000 7fbfc11aa700 20 client.31518.objecter get_session s=0x7fbfbc0279b0 osd=3 3 2024-02-23T00:52:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.202+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_assign 3 7 2024-02-23T00:52:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.202+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc0279b0 osd=3 4 2024-02-23T00:52:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.202+0000 7fbfc11aa700 10 client.31518.objecter _send_command 7 2024-02-23T00:52:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.202+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbfbc124870 con 0x7fbfbc11c090 2024-02-23T00:52:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.203+0000 7fbfbb59e700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fbfbc11c090 0x7fbfbc121c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.203+0000 7fbfb3fff700 10 client.31518.objecter ms_handle_connect 0x7fbfbc11c090 2024-02-23T00:52:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.203+0000 7fbfbb59e700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fbfbc11c090 0x7fbfbc121c20 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.203+0000 7fbfb3fff700 1 -- 172.21.15.47:0/3577590144 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fbfbc124870 con 0x7fbfbc11c090 2024-02-23T00:52:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.203+0000 7fbfb3fff700 10 client.31518.objecter ms_dispatch 0x7fbfbc001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:52:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.203+0000 7fbfb3fff700 10 client.31518.objecter _finish_command 7 = system:0 2024-02-23T00:52:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.203+0000 7fbfb3fff700 20 client.31518.objecter put_session s=0x7fbfbc0279b0 osd=3 4 2024-02-23T00:52:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.203+0000 7fbfb3fff700 15 client.31518.objecter _session_command_op_remove 3 7 2024-02-23T00:52:55.216 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:52:55.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.218+0000 7fbfc11aa700 10 client.31518.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:52:55.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.218+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_assign -1 8 2024-02-23T00:52:55.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.218+0000 7fbfc11aa700 20 client.31518.objecter _get_session s=0x7fbfbc0279b0 osd=3 3 2024-02-23T00:52:55.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.218+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc0279b0 osd=3 3 2024-02-23T00:52:55.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.218+0000 7fbfc11aa700 20 client.31518.objecter _get_session s=0x7fbfbc0279b0 osd=3 3 2024-02-23T00:52:55.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.218+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_remove -1 8 2024-02-23T00:52:55.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.218+0000 7fbfc11aa700 20 client.31518.objecter get_session s=0x7fbfbc0279b0 osd=3 3 2024-02-23T00:52:55.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.218+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_assign 3 8 2024-02-23T00:52:55.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.218+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc0279b0 osd=3 4 2024-02-23T00:52:55.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.218+0000 7fbfc11aa700 10 client.31518.objecter _send_command 8 2024-02-23T00:52:55.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.218+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fbfbc11b5f0 con 0x7fbfbc11c090 2024-02-23T00:52:55.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.219+0000 7fbfb3fff700 1 -- 172.21.15.47:0/3577590144 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fbfbc11b5f0 con 0x7fbfbc11c090 2024-02-23T00:52:55.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.219+0000 7fbfb3fff700 10 client.31518.objecter ms_dispatch 0x7fbfbc001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:52:55.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.219+0000 7fbfb3fff700 10 client.31518.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:52:55.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.219+0000 7fbfb3fff700 20 client.31518.objecter put_session s=0x7fbfbc0279b0 osd=3 4 2024-02-23T00:52:55.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.219+0000 7fbfb3fff700 15 client.31518.objecter _session_command_op_remove 3 8 2024-02-23T00:52:55.223 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:52:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.220+0000 7fbfc11aa700 10 client.31518.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.220+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_assign -1 9 2024-02-23T00:52:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.220+0000 7fbfc11aa700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fbfbc11f770 0x7fbfbc123290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.220+0000 7fbfc11aa700 20 client.31518.objecter _get_session s=0x7fbfbc11b5f0 osd=4 3 2024-02-23T00:52:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.220+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc11b5f0 osd=4 3 2024-02-23T00:52:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.220+0000 7fbfc11aa700 20 client.31518.objecter _get_session s=0x7fbfbc11b5f0 osd=4 3 2024-02-23T00:52:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.220+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_remove -1 9 2024-02-23T00:52:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.220+0000 7fbfc11aa700 20 client.31518.objecter get_session s=0x7fbfbc11b5f0 osd=4 3 2024-02-23T00:52:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.220+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_assign 4 9 2024-02-23T00:52:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.220+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc11b5f0 osd=4 4 2024-02-23T00:52:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.220+0000 7fbfc11aa700 10 client.31518.objecter _send_command 9 2024-02-23T00:52:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.220+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbfbc11b9a0 con 0x7fbfbc11f770 2024-02-23T00:52:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.220+0000 7fbfbad9d700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fbfbc11f770 0x7fbfbc123290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.220+0000 7fbfbad9d700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fbfbc11f770 0x7fbfbc123290 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.220+0000 7fbfb3fff700 10 client.31518.objecter ms_handle_connect 0x7fbfbc11f770 2024-02-23T00:52:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.221+0000 7fbfb3fff700 1 -- 172.21.15.47:0/3577590144 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fbfbc11b9a0 con 0x7fbfbc11f770 2024-02-23T00:52:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.221+0000 7fbfb3fff700 10 client.31518.objecter ms_dispatch 0x7fbfbc001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:52:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.221+0000 7fbfb3fff700 10 client.31518.objecter _finish_command 9 = system:0 2024-02-23T00:52:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.221+0000 7fbfb3fff700 20 client.31518.objecter put_session s=0x7fbfbc11b5f0 osd=4 4 2024-02-23T00:52:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.221+0000 7fbfb3fff700 15 client.31518.objecter _session_command_op_remove 4 9 2024-02-23T00:52:55.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.236+0000 7fbfc11aa700 10 client.31518.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:52:55.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.236+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_assign -1 10 2024-02-23T00:52:55.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.236+0000 7fbfc11aa700 20 client.31518.objecter _get_session s=0x7fbfbc11b5f0 osd=4 3 2024-02-23T00:52:55.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.236+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc11b5f0 osd=4 3 2024-02-23T00:52:55.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.236+0000 7fbfc11aa700 20 client.31518.objecter _get_session s=0x7fbfbc11b5f0 osd=4 3 2024-02-23T00:52:55.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.236+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_remove -1 10 2024-02-23T00:52:55.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.236+0000 7fbfc11aa700 20 client.31518.objecter get_session s=0x7fbfbc11b5f0 osd=4 3 2024-02-23T00:52:55.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.236+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_assign 4 10 2024-02-23T00:52:55.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.236+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc11b5f0 osd=4 4 2024-02-23T00:52:55.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.236+0000 7fbfc11aa700 10 client.31518.objecter _send_command 10 2024-02-23T00:52:55.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.236+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fbfbc11b9a0 con 0x7fbfbc11f770 2024-02-23T00:52:55.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.237+0000 7fbfb3fff700 1 -- 172.21.15.47:0/3577590144 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fbfbc11b9a0 con 0x7fbfbc11f770 2024-02-23T00:52:55.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.238+0000 7fbfb3fff700 10 client.31518.objecter ms_dispatch 0x7fbfbc001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:52:55.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.238+0000 7fbfb3fff700 10 client.31518.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:52:55.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.238+0000 7fbfb3fff700 20 client.31518.objecter put_session s=0x7fbfbc11b5f0 osd=4 4 2024-02-23T00:52:55.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.238+0000 7fbfb3fff700 15 client.31518.objecter _session_command_op_remove 4 10 2024-02-23T00:52:55.241 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:52:55.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.238+0000 7fbfc11aa700 10 client.31518.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:55.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.238+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_assign -1 11 2024-02-23T00:52:55.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.238+0000 7fbfc11aa700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fbfbc123c90 0x7fbfbc124110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:55.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.238+0000 7fbfc11aa700 20 client.31518.objecter _get_session s=0x7fbfbc11b9a0 osd=5 3 2024-02-23T00:52:55.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.238+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc11b9a0 osd=5 3 2024-02-23T00:52:55.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.238+0000 7fbfc11aa700 20 client.31518.objecter _get_session s=0x7fbfbc11b9a0 osd=5 3 2024-02-23T00:52:55.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.238+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_remove -1 11 2024-02-23T00:52:55.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.238+0000 7fbfc11aa700 20 client.31518.objecter get_session s=0x7fbfbc11b9a0 osd=5 3 2024-02-23T00:52:55.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.238+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_assign 5 11 2024-02-23T00:52:55.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.238+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc11b9a0 osd=5 4 2024-02-23T00:52:55.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.238+0000 7fbfc11aa700 10 client.31518.objecter _send_command 11 2024-02-23T00:52:55.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.238+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbfbc12b160 con 0x7fbfbc123c90 2024-02-23T00:52:55.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.238+0000 7fbfba59c700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fbfbc123c90 0x7fbfbc124110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:55.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.238+0000 7fbfb3fff700 10 client.31518.objecter ms_handle_connect 0x7fbfbc123c90 2024-02-23T00:52:55.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.238+0000 7fbfba59c700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fbfbc123c90 0x7fbfbc124110 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:55.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.239+0000 7fbfb3fff700 1 -- 172.21.15.47:0/3577590144 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fbfbc12b160 con 0x7fbfbc123c90 2024-02-23T00:52:55.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.239+0000 7fbfb3fff700 10 client.31518.objecter ms_dispatch 0x7fbfbc001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:52:55.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.239+0000 7fbfb3fff700 10 client.31518.objecter _finish_command 11 = system:0 2024-02-23T00:52:55.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.239+0000 7fbfb3fff700 20 client.31518.objecter put_session s=0x7fbfbc11b9a0 osd=5 4 2024-02-23T00:52:55.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.239+0000 7fbfb3fff700 15 client.31518.objecter _session_command_op_remove 5 11 2024-02-23T00:52:55.244 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:55.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.254+0000 7fbfc11aa700 10 client.31518.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:52:55.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.254+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_assign -1 12 2024-02-23T00:52:55.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.254+0000 7fbfc11aa700 20 client.31518.objecter _get_session s=0x7fbfbc11b9a0 osd=5 3 2024-02-23T00:52:55.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.254+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc11b9a0 osd=5 3 2024-02-23T00:52:55.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.254+0000 7fbfc11aa700 20 client.31518.objecter _get_session s=0x7fbfbc11b9a0 osd=5 3 2024-02-23T00:52:55.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.254+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_remove -1 12 2024-02-23T00:52:55.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.254+0000 7fbfc11aa700 20 client.31518.objecter get_session s=0x7fbfbc11b9a0 osd=5 3 2024-02-23T00:52:55.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.254+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_assign 5 12 2024-02-23T00:52:55.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.254+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc11b9a0 osd=5 4 2024-02-23T00:52:55.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.254+0000 7fbfc11aa700 10 client.31518.objecter _send_command 12 2024-02-23T00:52:55.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.254+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fbfbc128a00 con 0x7fbfbc123c90 2024-02-23T00:52:55.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.256+0000 7fbfb3fff700 1 -- 172.21.15.47:0/3577590144 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fbfbc128a00 con 0x7fbfbc123c90 2024-02-23T00:52:55.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.256+0000 7fbfb3fff700 10 client.31518.objecter ms_dispatch 0x7fbfbc001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:52:55.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.256+0000 7fbfb3fff700 10 client.31518.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:52:55.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.256+0000 7fbfb3fff700 20 client.31518.objecter put_session s=0x7fbfbc11b9a0 osd=5 4 2024-02-23T00:52:55.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.256+0000 7fbfb3fff700 15 client.31518.objecter _session_command_op_remove 5 12 2024-02-23T00:52:55.259 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:52:55.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.256+0000 7fbfc11aa700 10 client.31518.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:55.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.256+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_assign -1 13 2024-02-23T00:52:55.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.256+0000 7fbfc11aa700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fbfbc1290e0 0x7fbfbc12b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:55.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.256+0000 7fbfc11aa700 20 client.31518.objecter _get_session s=0x7fbfbc128a00 osd=6 3 2024-02-23T00:52:55.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.256+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc128a00 osd=6 3 2024-02-23T00:52:55.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.256+0000 7fbfc11aa700 20 client.31518.objecter _get_session s=0x7fbfbc128a00 osd=6 3 2024-02-23T00:52:55.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.256+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_remove -1 13 2024-02-23T00:52:55.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.256+0000 7fbfc11aa700 20 client.31518.objecter get_session s=0x7fbfbc128a00 osd=6 3 2024-02-23T00:52:55.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.256+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_assign 6 13 2024-02-23T00:52:55.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.256+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc128a00 osd=6 4 2024-02-23T00:52:55.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.256+0000 7fbfc11aa700 10 client.31518.objecter _send_command 13 2024-02-23T00:52:55.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.256+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbfbc12e2c0 con 0x7fbfbc1290e0 2024-02-23T00:52:55.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.256+0000 7fbfbb59e700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fbfbc1290e0 0x7fbfbc12b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:55.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.256+0000 7fbfb3fff700 10 client.31518.objecter ms_handle_connect 0x7fbfbc1290e0 2024-02-23T00:52:55.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.256+0000 7fbfbb59e700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fbfbc1290e0 0x7fbfbc12b4d0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:55.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.257+0000 7fbfb3fff700 1 -- 172.21.15.47:0/3577590144 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fbfbc12e2c0 con 0x7fbfbc1290e0 2024-02-23T00:52:55.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.257+0000 7fbfb3fff700 10 client.31518.objecter ms_dispatch 0x7fbfbc001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:52:55.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.257+0000 7fbfb3fff700 10 client.31518.objecter _finish_command 13 = system:0 2024-02-23T00:52:55.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.257+0000 7fbfb3fff700 20 client.31518.objecter put_session s=0x7fbfbc128a00 osd=6 4 2024-02-23T00:52:55.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.257+0000 7fbfb3fff700 15 client.31518.objecter _session_command_op_remove 6 13 2024-02-23T00:52:55.261 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:52:55.264 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:55.264 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:52:55.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.278+0000 7fbfc11aa700 10 client.31518.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:52:55.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.278+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_assign -1 14 2024-02-23T00:52:55.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.278+0000 7fbfc11aa700 20 client.31518.objecter _get_session s=0x7fbfbc128a00 osd=6 3 2024-02-23T00:52:55.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.278+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc128a00 osd=6 3 2024-02-23T00:52:55.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.278+0000 7fbfc11aa700 20 client.31518.objecter _get_session s=0x7fbfbc128a00 osd=6 3 2024-02-23T00:52:55.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.278+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_remove -1 14 2024-02-23T00:52:55.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.278+0000 7fbfc11aa700 20 client.31518.objecter get_session s=0x7fbfbc128a00 osd=6 3 2024-02-23T00:52:55.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.278+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_assign 6 14 2024-02-23T00:52:55.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.278+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc128a00 osd=6 4 2024-02-23T00:52:55.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.278+0000 7fbfc11aa700 10 client.31518.objecter _send_command 14 2024-02-23T00:52:55.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.278+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fbfbc12bab0 con 0x7fbfbc1290e0 2024-02-23T00:52:55.283 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:52:55.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.280+0000 7fbfb3fff700 1 -- 172.21.15.47:0/3577590144 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fbfbc12bab0 con 0x7fbfbc1290e0 2024-02-23T00:52:55.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.280+0000 7fbfb3fff700 10 client.31518.objecter ms_dispatch 0x7fbfbc001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:52:55.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.280+0000 7fbfb3fff700 10 client.31518.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:52:55.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.280+0000 7fbfb3fff700 20 client.31518.objecter put_session s=0x7fbfbc128a00 osd=6 4 2024-02-23T00:52:55.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.280+0000 7fbfb3fff700 15 client.31518.objecter _session_command_op_remove 6 14 2024-02-23T00:52:55.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.281+0000 7fbfc11aa700 10 client.31518.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:52:55.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.281+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_assign -1 15 2024-02-23T00:52:55.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.281+0000 7fbfc11aa700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fbfbc12d960 0x7fbfbc12bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:55.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.281+0000 7fbfc11aa700 20 client.31518.objecter _get_session s=0x7fbfbc12d250 osd=7 3 2024-02-23T00:52:55.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.281+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc12d250 osd=7 3 2024-02-23T00:52:55.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.281+0000 7fbfc11aa700 20 client.31518.objecter _get_session s=0x7fbfbc12d250 osd=7 3 2024-02-23T00:52:55.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.281+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_remove -1 15 2024-02-23T00:52:55.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.281+0000 7fbfc11aa700 20 client.31518.objecter get_session s=0x7fbfbc12d250 osd=7 3 2024-02-23T00:52:55.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.281+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_assign 7 15 2024-02-23T00:52:55.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.281+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc12d250 osd=7 4 2024-02-23T00:52:55.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.281+0000 7fbfc11aa700 10 client.31518.objecter _send_command 15 2024-02-23T00:52:55.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.281+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbfbc12df40 con 0x7fbfbc12d960 2024-02-23T00:52:55.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.281+0000 7fbfbad9d700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fbfbc12d960 0x7fbfbc12bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:55.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.281+0000 7fbfb3fff700 10 client.31518.objecter ms_handle_connect 0x7fbfbc12d960 2024-02-23T00:52:55.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.281+0000 7fbfbad9d700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fbfbc12d960 0x7fbfbc12bab0 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:55.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.282+0000 7fbfb3fff700 1 -- 172.21.15.47:0/3577590144 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fbfbc12df40 con 0x7fbfbc12d960 2024-02-23T00:52:55.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.282+0000 7fbfb3fff700 10 client.31518.objecter ms_dispatch 0x7fbfbc001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:52:55.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.282+0000 7fbfb3fff700 10 client.31518.objecter _finish_command 15 = system:0 2024-02-23T00:52:55.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.282+0000 7fbfb3fff700 20 client.31518.objecter put_session s=0x7fbfbc12d250 osd=7 4 2024-02-23T00:52:55.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.282+0000 7fbfb3fff700 15 client.31518.objecter _session_command_op_remove 7 15 2024-02-23T00:52:55.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.298+0000 7fbfc11aa700 10 client.31518.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:52:55.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.298+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_assign -1 16 2024-02-23T00:52:55.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.298+0000 7fbfc11aa700 20 client.31518.objecter _get_session s=0x7fbfbc12d250 osd=7 3 2024-02-23T00:52:55.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.298+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc12d250 osd=7 3 2024-02-23T00:52:55.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.298+0000 7fbfc11aa700 20 client.31518.objecter _get_session s=0x7fbfbc12d250 osd=7 3 2024-02-23T00:52:55.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.298+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_remove -1 16 2024-02-23T00:52:55.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.298+0000 7fbfc11aa700 20 client.31518.objecter get_session s=0x7fbfbc12d250 osd=7 3 2024-02-23T00:52:55.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.298+0000 7fbfc11aa700 15 client.31518.objecter _session_command_op_assign 7 16 2024-02-23T00:52:55.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.298+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc12d250 osd=7 4 2024-02-23T00:52:55.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.298+0000 7fbfc11aa700 10 client.31518.objecter _send_command 16 2024-02-23T00:52:55.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.298+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fbfbc12df00 con 0x7fbfbc12d960 2024-02-23T00:52:55.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.299+0000 7fbfb3fff700 1 -- 172.21.15.47:0/3577590144 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fbfbc12df00 con 0x7fbfbc12d960 2024-02-23T00:52:55.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.299+0000 7fbfb3fff700 10 client.31518.objecter ms_dispatch 0x7fbfbc001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:52:55.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.299+0000 7fbfb3fff700 10 client.31518.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:52:55.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.299+0000 7fbfb3fff700 20 client.31518.objecter put_session s=0x7fbfbc12d250 osd=7 4 2024-02-23T00:52:55.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.299+0000 7fbfb3fff700 15 client.31518.objecter _session_command_op_remove 7 16 2024-02-23T00:52:55.303 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:52:55.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 10 librados: watch_flush enter 2024-02-23T00:52:55.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 10 librados: watch_flush exit 2024-02-23T00:52:55.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 10 client.31518.objecter close_session for osd.0 2024-02-23T00:52:55.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fbfbc0fe5f0 msgr2=0x7fbfbc0fe9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:55.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fbfbc0fe5f0 0x7fbfbc0fe9d0 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:55.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc05ee80 osd=0 1 2024-02-23T00:52:55.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 10 client.31518.objecter close_session for osd.1 2024-02-23T00:52:55.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fbfbc028540 msgr2=0x7fbfbc0598f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:55.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fbfbc028540 0x7fbfbc0598f0 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:55.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc0ff090 osd=1 1 2024-02-23T00:52:55.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 10 client.31518.objecter close_session for osd.2 2024-02-23T00:52:55.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fbfbc0594d0 msgr2=0x7fbfbc11e5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:55.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fbfbc0594d0 0x7fbfbc11e5a0 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:55.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc059360 osd=2 1 2024-02-23T00:52:55.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 10 client.31518.objecter close_session for osd.3 2024-02-23T00:52:55.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fbfbc11c090 msgr2=0x7fbfbc121c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:55.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fbfbc11c090 0x7fbfbc121c20 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:55.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc0279b0 osd=3 1 2024-02-23T00:52:55.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 10 client.31518.objecter close_session for osd.4 2024-02-23T00:52:55.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fbfbc11f770 msgr2=0x7fbfbc123290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:55.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfbb59e700 1 -- 172.21.15.47:0/3577590144 reap_dead start 2024-02-23T00:52:55.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fbfbc11f770 0x7fbfbc123290 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:55.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfbb59e700 1 -- 172.21.15.47:0/3577590144 reap_dead start 2024-02-23T00:52:55.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc11b5f0 osd=4 1 2024-02-23T00:52:55.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 10 client.31518.objecter close_session for osd.5 2024-02-23T00:52:55.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fbfbc123c90 msgr2=0x7fbfbc124110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:55.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fbfbc123c90 0x7fbfbc124110 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:55.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc11b9a0 osd=5 1 2024-02-23T00:52:55.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 10 client.31518.objecter close_session for osd.6 2024-02-23T00:52:55.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fbfbc1290e0 msgr2=0x7fbfbc12b4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:55.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fbfbc1290e0 0x7fbfbc12b4d0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:55.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc128a00 osd=6 1 2024-02-23T00:52:55.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 10 client.31518.objecter close_session for osd.7 2024-02-23T00:52:55.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fbfbc12d960 msgr2=0x7fbfbc12bab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:55.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fbfbc12d960 0x7fbfbc12bab0 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:55.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 20 client.31518.objecter put_session s=0x7fbfbc12d250 osd=7 1 2024-02-23T00:52:55.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 20 client.31518.objecter shutdown clearing up homeless session... 2024-02-23T00:52:55.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 10 client.31518.objecter successfully canceled tick 2024-02-23T00:52:55.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbfa8062cc0 msgr2=0x7fbfa8065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:55.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbfa8062cc0 0x7fbfa8065180 secure :-1 s=READY pgs=3149 cs=0 l=1 rev1=1 rx=0x7fbfac004e20 tx=0x7fbfac009cb0).stop 2024-02-23T00:52:55.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfbc0f9600 msgr2=0x7fbfbc110750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:55.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.300+0000 7fbfc11aa700 1 --2- 172.21.15.47:0/3577590144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfbc0f9600 0x7fbfbc110750 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 rx=0x7fbfa4009400 tx=0x7fbfa4009cb0).stop 2024-02-23T00:52:55.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.301+0000 7fbfbb59e700 1 -- 172.21.15.47:0/3577590144 reap_dead start 2024-02-23T00:52:55.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.301+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 shutdown_connections 2024-02-23T00:52:55.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.301+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 >> 172.21.15.47:0/3577590144 conn(0x7fbfbc0f2820 msgr2=0x7fbfbc0f4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:52:55.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.301+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 shutdown_connections 2024-02-23T00:52:55.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.301+0000 7fbfc11aa700 1 -- 172.21.15.47:0/3577590144 wait complete. 2024-02-23T00:52:55.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:55.301+0000 7fbfc11aa700 1 librados: shutdown 2024-02-23T00:52:55.307 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:52:55.308 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:52:55.308 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:52:55.308 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:52:55.308 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:52:55.308 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:52:55.308 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:52:55.308 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:52:55.365 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:52:55.393 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:55.412 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:52:55.414 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:55.414 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:52:55.515 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:52:55.544 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:53.474609+0000 osd.6 (osd.6) 247 : cluster [DBG] 2.17 scrub starts 2024-02-23T00:52:55.545 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:53.475440+0000 osd.6 (osd.6) 248 : cluster [DBG] 2.17 scrub ok 2024-02-23T00:52:55.545 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:53.809199+0000 osd.0 (osd.0) 265 : cluster [DBG] 4.1e scrub starts 2024-02-23T00:52:55.545 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:53.810587+0000 osd.0 (osd.0) 266 : cluster [DBG] 4.1e scrub ok 2024-02-23T00:52:55.545 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:54.307477+0000 mon.a (mon.0) 3221 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-108"}]': finished 2024-02-23T00:52:55.545 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:54.307567+0000 mon.a (mon.0) 3222 : cluster [DBG] osdmap e637: 8 total, 8 up, 7 in 2024-02-23T00:52:55.545 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:54.308590+0000 mon.c (mon.2) 363 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-108", "mode": "writeback"}]: dispatch 2024-02-23T00:52:55.545 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:54.314355+0000 mon.a (mon.0) 3223 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-108", "mode": "writeback"}]: dispatch 2024-02-23T00:52:55.545 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:54.386015+0000 osd.5 (osd.5) 280 : cluster [DBG] 5.f scrub starts 2024-02-23T00:52:55.545 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:54.388941+0000 osd.5 (osd.5) 281 : cluster [DBG] 5.f scrub ok 2024-02-23T00:52:55.545 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:54.627502+0000 osd.1 (osd.1) 263 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:52:55.545 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:54.627791+0000 osd.1 (osd.1) 264 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:52:55.545 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:55.149800+0000 mon.a (mon.0) 3224 : audit [DBG] from='client.? 172.21.15.47:0/3577590144' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:52:55.546 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:55.264120+0000 mon.a (mon.0) 3225 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:52:55.546 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:55.268554+0000 mon.a (mon.0) 3226 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-108", "mode": "writeback"}]': finished 2024-02-23T00:52:55.546 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:55.268631+0000 mon.a (mon.0) 3227 : cluster [DBG] osdmap e638: 8 total, 8 up, 7 in 2024-02-23T00:52:55.546 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:53.474609+0000 osd.6 (osd.6) 247 : cluster [DBG] 2.17 scrub starts 2024-02-23T00:52:55.546 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:53.475440+0000 osd.6 (osd.6) 248 : cluster [DBG] 2.17 scrub ok 2024-02-23T00:52:55.546 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:53.809199+0000 osd.0 (osd.0) 265 : cluster [DBG] 4.1e scrub starts 2024-02-23T00:52:55.546 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:53.810587+0000 osd.0 (osd.0) 266 : cluster [DBG] 4.1e scrub ok 2024-02-23T00:52:55.546 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:54.307477+0000 mon.a (mon.0) 3221 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-108"}]': finished 2024-02-23T00:52:55.546 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:54.307567+0000 mon.a (mon.0) 3222 : cluster [DBG] osdmap e637: 8 total, 8 up, 7 in 2024-02-23T00:52:55.546 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:54.308590+0000 mon.c (mon.2) 363 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-108", "mode": "writeback"}]: dispatch 2024-02-23T00:52:55.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:54.314355+0000 mon.a (mon.0) 3223 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-108", "mode": "writeback"}]: dispatch 2024-02-23T00:52:55.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:54.386015+0000 osd.5 (osd.5) 280 : cluster [DBG] 5.f scrub starts 2024-02-23T00:52:55.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:54.388941+0000 osd.5 (osd.5) 281 : cluster [DBG] 5.f scrub ok 2024-02-23T00:52:55.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:54.627502+0000 osd.1 (osd.1) 263 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:52:55.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:54.627791+0000 osd.1 (osd.1) 264 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:52:55.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:55.149800+0000 mon.a (mon.0) 3224 : audit [DBG] from='client.? 172.21.15.47:0/3577590144' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:52:55.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:55.264120+0000 mon.a (mon.0) 3225 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:52:55.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:55.268554+0000 mon.a (mon.0) 3226 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-108", "mode": "writeback"}]': finished 2024-02-23T00:52:55.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:55.268631+0000 mon.a (mon.0) 3227 : cluster [DBG] osdmap e638: 8 total, 8 up, 7 in 2024-02-23T00:52:55.548 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:55.570 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:52:55.573 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:55.573 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:52:55.665 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:52:55.666 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-02-23T00:52:55.666 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-02-23T00:52:55.666 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-02-23T00:52:55.666 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:52:55.674 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:52:55.693 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:53.474609+0000 osd.6 (osd.6) 247 : cluster [DBG] 2.17 scrub starts 2024-02-23T00:52:55.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:53.475440+0000 osd.6 (osd.6) 248 : cluster [DBG] 2.17 scrub ok 2024-02-23T00:52:55.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:53.809199+0000 osd.0 (osd.0) 265 : cluster [DBG] 4.1e scrub starts 2024-02-23T00:52:55.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:53.810587+0000 osd.0 (osd.0) 266 : cluster [DBG] 4.1e scrub ok 2024-02-23T00:52:55.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:54.307477+0000 mon.a (mon.0) 3221 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-108"}]': finished 2024-02-23T00:52:55.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:54.307567+0000 mon.a (mon.0) 3222 : cluster [DBG] osdmap e637: 8 total, 8 up, 7 in 2024-02-23T00:52:55.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:54.308590+0000 mon.c (mon.2) 363 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-108", "mode": "writeback"}]: dispatch 2024-02-23T00:52:55.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:54.314355+0000 mon.a (mon.0) 3223 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-108", "mode": "writeback"}]: dispatch 2024-02-23T00:52:55.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:54.386015+0000 osd.5 (osd.5) 280 : cluster [DBG] 5.f scrub starts 2024-02-23T00:52:55.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:54.388941+0000 osd.5 (osd.5) 281 : cluster [DBG] 5.f scrub ok 2024-02-23T00:52:55.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:54.627502+0000 osd.1 (osd.1) 263 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:52:55.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:54.627791+0000 osd.1 (osd.1) 264 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:52:55.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:55.149800+0000 mon.a (mon.0) 3224 : audit [DBG] from='client.? 172.21.15.47:0/3577590144' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:52:55.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:55.264120+0000 mon.a (mon.0) 3225 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:52:55.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:55.268554+0000 mon.a (mon.0) 3226 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-108", "mode": "writeback"}]': finished 2024-02-23T00:52:55.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:55.268631+0000 mon.a (mon.0) 3227 : cluster [DBG] osdmap e638: 8 total, 8 up, 7 in 2024-02-23T00:52:55.702 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:55.902 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:52:55.904 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:55.905 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:52:56.006 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:52:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:56.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:55.364795+0000 mgr.y (mgr.14517) 566 : cluster [DBG] pgmap v965: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:52:56.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:55.379336+0000 osd.7 (osd.7) 307 : cluster [DBG] 2.b scrub starts 2024-02-23T00:52:56.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:55.382206+0000 osd.7 (osd.7) 308 : cluster [DBG] 2.b scrub ok 2024-02-23T00:52:56.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:55.397590+0000 osd.5 (osd.5) 282 : cluster [DBG] 18.15 scrub starts 2024-02-23T00:52:56.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:55.400938+0000 osd.5 (osd.5) 283 : cluster [DBG] 18.15 scrub ok 2024-02-23T00:52:56.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:55.477320+0000 osd.1 (osd.1) 265 : cluster [DBG] 4.1c scrub starts 2024-02-23T00:52:56.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:55.480766+0000 osd.1 (osd.1) 266 : cluster [DBG] 4.1c scrub ok 2024-02-23T00:52:56.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:55.786870+0000 osd.4 (osd.4) 271 : cluster [DBG] 18.d scrub starts 2024-02-23T00:52:56.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:55.788440+0000 osd.4 (osd.4) 272 : cluster [DBG] 18.d scrub ok 2024-02-23T00:52:56.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:56.275924+0000 mon.a (mon.0) 3228 : cluster [DBG] osdmap e639: 8 total, 8 up, 7 in 2024-02-23T00:52:56.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:56.305459+0000 mon.c (mon.2) 364 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:56.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:56.305777+0000 mon.a (mon.0) 3229 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:56.783 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:52:56.785 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:55.364795+0000 mgr.y (mgr.14517) 566 : cluster [DBG] pgmap v965: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:52:56.785 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:55.379336+0000 osd.7 (osd.7) 307 : cluster [DBG] 2.b scrub starts 2024-02-23T00:52:56.785 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:55.382206+0000 osd.7 (osd.7) 308 : cluster [DBG] 2.b scrub ok 2024-02-23T00:52:56.785 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:55.397590+0000 osd.5 (osd.5) 282 : cluster [DBG] 18.15 scrub starts 2024-02-23T00:52:56.785 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:55.400938+0000 osd.5 (osd.5) 283 : cluster [DBG] 18.15 scrub ok 2024-02-23T00:52:56.785 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:55.477320+0000 osd.1 (osd.1) 265 : cluster [DBG] 4.1c scrub starts 2024-02-23T00:52:56.786 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:55.480766+0000 osd.1 (osd.1) 266 : cluster [DBG] 4.1c scrub ok 2024-02-23T00:52:56.786 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:55.786870+0000 osd.4 (osd.4) 271 : cluster [DBG] 18.d scrub starts 2024-02-23T00:52:56.786 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:55.788440+0000 osd.4 (osd.4) 272 : cluster [DBG] 18.d scrub ok 2024-02-23T00:52:56.786 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:56.275924+0000 mon.a (mon.0) 3228 : cluster [DBG] osdmap e639: 8 total, 8 up, 7 in 2024-02-23T00:52:56.786 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:56.305459+0000 mon.c (mon.2) 364 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:56.786 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:56.305777+0000 mon.a (mon.0) 3229 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:56.786 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:55.364795+0000 mgr.y (mgr.14517) 566 : cluster [DBG] pgmap v965: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:52:56.786 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:55.379336+0000 osd.7 (osd.7) 307 : cluster [DBG] 2.b scrub starts 2024-02-23T00:52:56.786 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:55.382206+0000 osd.7 (osd.7) 308 : cluster [DBG] 2.b scrub ok 2024-02-23T00:52:56.786 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:55.397590+0000 osd.5 (osd.5) 282 : cluster [DBG] 18.15 scrub starts 2024-02-23T00:52:56.786 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:55.400938+0000 osd.5 (osd.5) 283 : cluster [DBG] 18.15 scrub ok 2024-02-23T00:52:56.787 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:55.477320+0000 osd.1 (osd.1) 265 : cluster [DBG] 4.1c scrub starts 2024-02-23T00:52:56.787 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:55.480766+0000 osd.1 (osd.1) 266 : cluster [DBG] 4.1c scrub ok 2024-02-23T00:52:56.787 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:55.786870+0000 osd.4 (osd.4) 271 : cluster [DBG] 18.d scrub starts 2024-02-23T00:52:56.787 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:55.788440+0000 osd.4 (osd.4) 272 : cluster [DBG] 18.d scrub ok 2024-02-23T00:52:56.787 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:56.275924+0000 mon.a (mon.0) 3228 : cluster [DBG] osdmap e639: 8 total, 8 up, 7 in 2024-02-23T00:52:56.787 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:56.305459+0000 mon.c (mon.2) 364 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:56.787 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:56.305777+0000 mon.a (mon.0) 3229 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:52:56.787 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:56.788 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:52:56.889 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:52:56.920 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:56.942 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:52:56.944 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:56.944 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:52:57.045 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:52:57.075 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:57.354 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.2 dump_historic_ops 2024-02-23T00:52:57.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:56.394764+0000 osd.7 (osd.7) 309 : cluster [DBG] 5.1a scrub starts 2024-02-23T00:52:57.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:56.397843+0000 osd.7 (osd.7) 310 : cluster [DBG] 5.1a scrub ok 2024-02-23T00:52:57.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:57.277668+0000 mon.a (mon.0) 3230 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:52:57.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:57.277740+0000 mon.a (mon.0) 3231 : cluster [DBG] osdmap e640: 8 total, 8 up, 7 in 2024-02-23T00:52:57.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:57.279150+0000 mon.c (mon.2) 365 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-108"}]: dispatch 2024-02-23T00:52:57.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:57.279531+0000 mon.a (mon.0) 3232 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-108"}]: dispatch 2024-02-23T00:52:57.800 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:56.394764+0000 osd.7 (osd.7) 309 : cluster [DBG] 5.1a scrub starts 2024-02-23T00:52:57.800 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:56.397843+0000 osd.7 (osd.7) 310 : cluster [DBG] 5.1a scrub ok 2024-02-23T00:52:57.800 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:57.277668+0000 mon.a (mon.0) 3230 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:52:57.800 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:57.277740+0000 mon.a (mon.0) 3231 : cluster [DBG] osdmap e640: 8 total, 8 up, 7 in 2024-02-23T00:52:57.800 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:57.279150+0000 mon.c (mon.2) 365 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-108"}]: dispatch 2024-02-23T00:52:57.800 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:57.279531+0000 mon.a (mon.0) 3232 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-108"}]: dispatch 2024-02-23T00:52:57.801 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:56.394764+0000 osd.7 (osd.7) 309 : cluster [DBG] 5.1a scrub starts 2024-02-23T00:52:57.801 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:56.397843+0000 osd.7 (osd.7) 310 : cluster [DBG] 5.1a scrub ok 2024-02-23T00:52:57.801 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:57.277668+0000 mon.a (mon.0) 3230 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:52:57.801 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:57.277740+0000 mon.a (mon.0) 3231 : cluster [DBG] osdmap e640: 8 total, 8 up, 7 in 2024-02-23T00:52:57.801 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:57.279150+0000 mon.c (mon.2) 365 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-108"}]: dispatch 2024-02-23T00:52:57.801 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:57.279531+0000 mon.a (mon.0) 3232 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-108"}]: dispatch 2024-02-23T00:52:57.826 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:52:57.829 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:57.829 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:52:57.930 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:52:57.958 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:58.082 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:52:58.085 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:58.085 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:52:58.186 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:52:58.218 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:58.239 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:52:58.242 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:58.243 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:52:58.343 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:52:58.345 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:56.512791+0000 osd.6 (osd.6) 249 : cluster [DBG] 5.1f scrub starts 2024-02-23T00:52:58.345 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:56.514063+0000 osd.6 (osd.6) 250 : cluster [DBG] 5.1f scrub ok 2024-02-23T00:52:58.345 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:56.843314+0000 osd.0 (osd.0) 267 : cluster [DBG] 18.1d scrub starts 2024-02-23T00:52:58.345 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:56.845546+0000 osd.0 (osd.0) 268 : cluster [DBG] 18.1d scrub ok 2024-02-23T00:52:58.345 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:57.365208+0000 mgr.y (mgr.14517) 567 : cluster [DBG] pgmap v968: 184 pgs: 1 active+clean+snaptrim, 183 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:52:58.345 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:57.916067+0000 osd.3 (osd.3) 306 : cluster [DBG] 18.9 scrub starts 2024-02-23T00:52:58.345 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:57.917363+0000 osd.3 (osd.3) 307 : cluster [DBG] 18.9 scrub ok 2024-02-23T00:52:58.345 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:58.274866+0000 mon.a (mon.0) 3233 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:52:58.345 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:58.279597+0000 mon.a (mon.0) 3234 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-108"}]': finished 2024-02-23T00:52:58.345 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:58.279717+0000 mon.a (mon.0) 3235 : cluster [DBG] osdmap e641: 8 total, 8 up, 7 in 2024-02-23T00:52:58.346 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:56.512791+0000 osd.6 (osd.6) 249 : cluster [DBG] 5.1f scrub starts 2024-02-23T00:52:58.346 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:56.514063+0000 osd.6 (osd.6) 250 : cluster [DBG] 5.1f scrub ok 2024-02-23T00:52:58.346 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:56.843314+0000 osd.0 (osd.0) 267 : cluster [DBG] 18.1d scrub starts 2024-02-23T00:52:58.346 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:56.845546+0000 osd.0 (osd.0) 268 : cluster [DBG] 18.1d scrub ok 2024-02-23T00:52:58.346 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:57.365208+0000 mgr.y (mgr.14517) 567 : cluster [DBG] pgmap v968: 184 pgs: 1 active+clean+snaptrim, 183 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:52:58.346 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:57.916067+0000 osd.3 (osd.3) 306 : cluster [DBG] 18.9 scrub starts 2024-02-23T00:52:58.346 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:57.917363+0000 osd.3 (osd.3) 307 : cluster [DBG] 18.9 scrub ok 2024-02-23T00:52:58.346 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:58.274866+0000 mon.a (mon.0) 3233 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:52:58.346 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:58.279597+0000 mon.a (mon.0) 3234 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-108"}]': finished 2024-02-23T00:52:58.346 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:58.279717+0000 mon.a (mon.0) 3235 : cluster [DBG] osdmap e641: 8 total, 8 up, 7 in 2024-02-23T00:52:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.354+0000 7f9e6cdb6700 1 -- 172.21.15.47:0/4164393538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e680f96e0 msgr2=0x7f9e68101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.354+0000 7f9e6cdb6700 1 --2- 172.21.15.47:0/4164393538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e680f96e0 0x7f9e68101c30 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 rx=0x7f9e5c004870 tx=0x7f9e5c009cb0).stop 2024-02-23T00:52:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.355+0000 7f9e6cdb6700 1 -- 172.21.15.47:0/4164393538 shutdown_connections 2024-02-23T00:52:58.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.355+0000 7f9e6cdb6700 1 --2- 172.21.15.47:0/4164393538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e68102250 0x7f9e68104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:58.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.355+0000 7f9e6cdb6700 1 --2- 172.21.15.47:0/4164393538 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9e680f8d80 0x7f9e680f9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:58.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.355+0000 7f9e6cdb6700 1 --2- 172.21.15.47:0/4164393538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e680f96e0 0x7f9e68101c30 unknown :-1 s=CLOSED pgs=2602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:58.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.355+0000 7f9e6cdb6700 1 -- 172.21.15.47:0/4164393538 >> 172.21.15.47:0/4164393538 conn(0x7f9e680f26f0 msgr2=0x7f9e680f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:52:58.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.355+0000 7f9e6cdb6700 1 -- 172.21.15.47:0/4164393538 shutdown_connections 2024-02-23T00:52:58.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.355+0000 7f9e6cdb6700 1 -- 172.21.15.47:0/4164393538 wait complete. 2024-02-23T00:52:58.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.355+0000 7f9e6cdb6700 1 librados: starting msgr at 2024-02-23T00:52:58.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.355+0000 7f9e6cdb6700 1 librados: starting objecter 2024-02-23T00:52:58.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.355+0000 7f9e6cdb6700 1 Processor -- start 2024-02-23T00:52:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.356+0000 7f9e6cdb6700 1 -- start start 2024-02-23T00:52:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.356+0000 7f9e6cdb6700 1 librados: setting wanted keys 2024-02-23T00:52:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.356+0000 7f9e6cdb6700 1 librados: calling monclient init 2024-02-23T00:52:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.356+0000 7f9e6cdb6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e680f8d80 0x7f9e6810e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.356+0000 7f9e6cdb6700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9e680f96e0 0x7f9e6810e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.356+0000 7f9e6cdb6700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e68102250 0x7f9e681082e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.356+0000 7f9e6cdb6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e681087d0 con 0x7f9e680f8d80 2024-02-23T00:52:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.356+0000 7f9e6cdb6700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e68108940 con 0x7f9e68102250 2024-02-23T00:52:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.356+0000 7f9e6cdb6700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e68108b00 con 0x7f9e680f96e0 2024-02-23T00:52:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.356+0000 7f9e65d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9e680f96e0 0x7f9e6810e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.356+0000 7f9e6659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e680f8d80 0x7f9e6810e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.356+0000 7f9e65d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9e680f96e0 0x7f9e6810e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:48716/0 (socket says 172.21.15.47:48716) 2024-02-23T00:52:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.356+0000 7f9e65d9b700 1 -- 172.21.15.47:0/1537052409 learned_addr learned my addr 172.21.15.47:0/1537052409 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:52:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.356+0000 7f9e66d9d700 1 --2- 172.21.15.47:0/1537052409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e68102250 0x7f9e681082e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.356+0000 7f9e6659c700 1 -- 172.21.15.47:0/1537052409 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9e680f96e0 msgr2=0x7f9e6810e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.356+0000 7f9e6659c700 1 --2- 172.21.15.47:0/1537052409 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9e680f96e0 0x7f9e6810e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.356+0000 7f9e6659c700 1 -- 172.21.15.47:0/1537052409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e68102250 msgr2=0x7f9e681082e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.356+0000 7f9e6659c700 1 --2- 172.21.15.47:0/1537052409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e68102250 0x7f9e681082e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.356+0000 7f9e6659c700 1 -- 172.21.15.47:0/1537052409 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5c005040 con 0x7f9e680f8d80 2024-02-23T00:52:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.356+0000 7f9e6659c700 1 --2- 172.21.15.47:0/1537052409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e680f8d80 0x7f9e6810e0f0 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 rx=0x7f9e5800e8f0 tx=0x7f9e5800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.356+0000 7f9e577fe700 10 client.?.objecter ms_handle_connect 0x7f9e680f8d80 2024-02-23T00:52:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.356+0000 7f9e577fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:52:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.357+0000 7f9e577fe700 1 -- 172.21.15.47:0/1537052409 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e58007b40 con 0x7f9e680f8d80 2024-02-23T00:52:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.357+0000 7f9e577fe700 1 -- 172.21.15.47:0/1537052409 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f9e5800f040 con 0x7f9e680f8d80 2024-02-23T00:52:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.357+0000 7f9e6cdb6700 1 -- 172.21.15.47:0/1537052409 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e68108de0 con 0x7f9e680f8d80 2024-02-23T00:52:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.357+0000 7f9e577fe700 1 -- 172.21.15.47:0/1537052409 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e580146f0 con 0x7f9e680f8d80 2024-02-23T00:52:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.357+0000 7f9e6cdb6700 10 client.31524.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:52:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.357+0000 7f9e6cdb6700 1 -- 172.21.15.47:0/1537052409 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e68114f20 con 0x7f9e680f8d80 2024-02-23T00:52:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.357+0000 7f9e6cdb6700 1 librados: init done 2024-02-23T00:52:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.358+0000 7f9e577fe700 1 -- 172.21.15.47:0/1537052409 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f9e580090d0 con 0x7f9e680f8d80 2024-02-23T00:52:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.359+0000 7f9e577fe700 1 --2- 172.21.15.47:0/1537052409 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9e50062cc0 0x7f9e50065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:52:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.359+0000 7f9e65d9b700 1 --2- 172.21.15.47:0/1537052409 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9e50062cc0 0x7f9e50065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:52:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.359+0000 7f9e577fe700 1 -- 172.21.15.47:0/1537052409 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(641..641 src has 482..641) v4 ==== 9654+0+0 (secure 0 0 0) 0x7f9e58007d30 con 0x7f9e680f8d80 2024-02-23T00:52:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.359+0000 7f9e577fe700 10 client.31524.objecter ms_dispatch 0x7f9e68001550 osd_map(641..641 src has 482..641) v4 2024-02-23T00:52:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.359+0000 7f9e577fe700 3 client.31524.objecter handle_osd_map got epochs [641,641] > 0 2024-02-23T00:52:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.359+0000 7f9e577fe700 3 client.31524.objecter handle_osd_map decoding full epoch 641 2024-02-23T00:52:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.359+0000 7f9e6cdb6700 1 -- 172.21.15.47:0/1537052409 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e6805ee80 con 0x7f9e680f8d80 2024-02-23T00:52:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.359+0000 7f9e577fe700 20 client.31524.objecter dump_active .. 0 homeless 2024-02-23T00:52:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.360+0000 7f9e65d9b700 1 --2- 172.21.15.47:0/1537052409 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9e50062cc0 0x7f9e50065180 secure :-1 s=READY pgs=3150 cs=0 l=1 rev1=1 rx=0x7f9e5c009da0 tx=0x7f9e5c009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:52:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.360+0000 7f9e577fe700 10 client.31524.objecter ms_handle_connect 0x7f9e50062cc0 2024-02-23T00:52:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:52:58.368 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:52:58.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.366+0000 7f9e577fe700 1 -- 172.21.15.47:0/1537052409 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f9e5804c6c0 con 0x7f9e680f8d80 2024-02-23T00:52:58.371 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:58.390 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:52:58.393 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:58.393 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:52:58.494 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:52:58.520 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:56.512791+0000 osd.6 (osd.6) 249 : cluster [DBG] 5.1f scrub starts 2024-02-23T00:52:58.520 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:56.514063+0000 osd.6 (osd.6) 250 : cluster [DBG] 5.1f scrub ok 2024-02-23T00:52:58.520 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:56.843314+0000 osd.0 (osd.0) 267 : cluster [DBG] 18.1d scrub starts 2024-02-23T00:52:58.520 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:56.845546+0000 osd.0 (osd.0) 268 : cluster [DBG] 18.1d scrub ok 2024-02-23T00:52:58.520 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:57.365208+0000 mgr.y (mgr.14517) 567 : cluster [DBG] pgmap v968: 184 pgs: 1 active+clean+snaptrim, 183 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:52:58.520 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:57.916067+0000 osd.3 (osd.3) 306 : cluster [DBG] 18.9 scrub starts 2024-02-23T00:52:58.520 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:57.917363+0000 osd.3 (osd.3) 307 : cluster [DBG] 18.9 scrub ok 2024-02-23T00:52:58.521 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:58.274866+0000 mon.a (mon.0) 3233 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:52:58.521 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:58.279597+0000 mon.a (mon.0) 3234 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-108"}]': finished 2024-02-23T00:52:58.521 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:58.279717+0000 mon.a (mon.0) 3235 : cluster [DBG] osdmap e641: 8 total, 8 up, 7 in 2024-02-23T00:52:58.521 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:58.540 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:52:58.543 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:58.543 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:52:58.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:58.562+0000 7f9e6cdb6700 1 -- 172.21.15.47:0/1537052409 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7f9e68107060 con 0x7f9e680f8d80 2024-02-23T00:52:58.644 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:52:58.671 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:58.689 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:52:58.692 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:58.692 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:52:58.717 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-23T00:52:58.718 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-23T00:52:58.718 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-02-23T00:52:58.718 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-02-23T00:52:58.718 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-02-23T00:52:58.718 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.5 config set filestore_inject_stall 3 2024-02-23T00:52:58.793 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:52:58.822 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:58.844 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:52:58.847 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:58.847 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:52:58.948 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:52:58.975 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:58.995 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:52:58.997 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:58.998 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:52:59.011 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:52:59.099 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:52:59.126 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:59.275 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:52:59.278 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:52:59.278 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:52:59.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:59.279+0000 7f9e577fe700 1 -- 172.21.15.47:0/1537052409 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v642) v1 ==== 101+0+0 (secure 0 0 0) 0x7f9e58018e80 con 0x7f9e680f8d80 2024-02-23T00:52:59.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:59.285+0000 7f9e6cdb6700 10 librados: watch_flush enter 2024-02-23T00:52:59.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:59.286+0000 7f9e6cdb6700 10 librados: watch_flush exit 2024-02-23T00:52:59.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:59.286+0000 7f9e6cdb6700 20 client.31524.objecter shutdown clearing up homeless session... 2024-02-23T00:52:59.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:59.286+0000 7f9e6cdb6700 10 client.31524.objecter successfully canceled tick 2024-02-23T00:52:59.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:59.286+0000 7f9e6cdb6700 1 -- 172.21.15.47:0/1537052409 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9e50062cc0 msgr2=0x7f9e50065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:59.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:59.286+0000 7f9e6cdb6700 1 --2- 172.21.15.47:0/1537052409 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9e50062cc0 0x7f9e50065180 secure :-1 s=READY pgs=3150 cs=0 l=1 rev1=1 rx=0x7f9e5c009da0 tx=0x7f9e5c009cb0).stop 2024-02-23T00:52:59.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:59.286+0000 7f9e6cdb6700 1 -- 172.21.15.47:0/1537052409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e680f8d80 msgr2=0x7f9e6810e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:52:59.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:59.286+0000 7f9e6cdb6700 1 --2- 172.21.15.47:0/1537052409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e680f8d80 0x7f9e6810e0f0 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 rx=0x7f9e5800e8f0 tx=0x7f9e5800b040).stop 2024-02-23T00:52:59.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:59.287+0000 7f9e6cdb6700 1 -- 172.21.15.47:0/1537052409 shutdown_connections 2024-02-23T00:52:59.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:59.287+0000 7f9e6cdb6700 1 --2- 172.21.15.47:0/1537052409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e68102250 0x7f9e681082e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:59.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:59.287+0000 7f9e6cdb6700 1 --2- 172.21.15.47:0/1537052409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e680f8d80 0x7f9e6810e0f0 unknown :-1 s=CLOSED pgs=2603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:59.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:59.287+0000 7f9e6cdb6700 1 --2- 172.21.15.47:0/1537052409 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9e50062cc0 0x7f9e50065180 unknown :-1 s=CLOSED pgs=3150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:59.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:59.287+0000 7f9e6cdb6700 1 --2- 172.21.15.47:0/1537052409 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9e680f96e0 0x7f9e6810e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:52:59.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:59.287+0000 7f9e6cdb6700 1 -- 172.21.15.47:0/1537052409 >> 172.21.15.47:0/1537052409 conn(0x7f9e680f26f0 msgr2=0x7f9e680f3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:52:59.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:59.288+0000 7f9e6cdb6700 1 -- 172.21.15.47:0/1537052409 shutdown_connections 2024-02-23T00:52:59.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:59.288+0000 7f9e6cdb6700 1 -- 172.21.15.47:0/1537052409 wait complete. 2024-02-23T00:52:59.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:52:59.290+0000 7f9e6cdb6700 1 librados: shutdown 2024-02-23T00:52:59.293 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is unset 2024-02-23T00:52:59.379 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:52:59.407 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:52:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:53:00.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:57.489201+0000 osd.6 (osd.6) 251 : cluster [DBG] 5.2 scrub starts 2024-02-23T00:53:00.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:57.490990+0000 osd.6 (osd.6) 252 : cluster [DBG] 5.2 scrub ok 2024-02-23T00:53:00.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:58.563972+0000 mon.a (mon.0) 3236 : audit [INF] from='client.? 172.21.15.47:0/1537052409' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:53:00.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:58.825801+0000 osd.4 (osd.4) 273 : cluster [DBG] 4.9 scrub starts 2024-02-23T00:53:00.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:58.827557+0000 osd.4 (osd.4) 274 : cluster [DBG] 4.9 scrub ok 2024-02-23T00:53:00.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:58.852730+0000 osd.4 (osd.4) 275 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:53:00.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:58.852886+0000 osd.4 (osd.4) 276 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:53:00.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:58.947477+0000 osd.3 (osd.3) 308 : cluster [DBG] 4.19 scrub starts 2024-02-23T00:53:00.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:58.949114+0000 osd.3 (osd.3) 309 : cluster [DBG] 4.19 scrub ok 2024-02-23T00:53:00.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:59.276537+0000 mon.a (mon.0) 3237 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:53:00.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:52:59.280565+0000 mon.a (mon.0) 3238 : audit [INF] from='client.? 172.21.15.47:0/1537052409' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:53:00.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:52:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:59.280629+0000 mon.a (mon.0) 3239 : cluster [DBG] osdmap e642: 8 total, 8 up, 7 in 2024-02-23T00:53:00.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:57.489201+0000 osd.6 (osd.6) 251 : cluster [DBG] 5.2 scrub starts 2024-02-23T00:53:00.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:57.490990+0000 osd.6 (osd.6) 252 : cluster [DBG] 5.2 scrub ok 2024-02-23T00:53:00.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:58.563972+0000 mon.a (mon.0) 3236 : audit [INF] from='client.? 172.21.15.47:0/1537052409' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:53:00.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:58.825801+0000 osd.4 (osd.4) 273 : cluster [DBG] 4.9 scrub starts 2024-02-23T00:53:00.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:58.827557+0000 osd.4 (osd.4) 274 : cluster [DBG] 4.9 scrub ok 2024-02-23T00:53:00.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:58.852730+0000 osd.4 (osd.4) 275 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:53:00.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:58.852886+0000 osd.4 (osd.4) 276 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:53:00.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:58.947477+0000 osd.3 (osd.3) 308 : cluster [DBG] 4.19 scrub starts 2024-02-23T00:53:00.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:58.949114+0000 osd.3 (osd.3) 309 : cluster [DBG] 4.19 scrub ok 2024-02-23T00:53:00.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:59.276537+0000 mon.a (mon.0) 3237 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:53:00.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:52:59.280565+0000 mon.a (mon.0) 3238 : audit [INF] from='client.? 172.21.15.47:0/1537052409' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:53:00.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:52:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:59.280629+0000 mon.a (mon.0) 3239 : cluster [DBG] osdmap e642: 8 total, 8 up, 7 in 2024-02-23T00:53:00.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:57.489201+0000 osd.6 (osd.6) 251 : cluster [DBG] 5.2 scrub starts 2024-02-23T00:53:00.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:57.490990+0000 osd.6 (osd.6) 252 : cluster [DBG] 5.2 scrub ok 2024-02-23T00:53:00.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:58.563972+0000 mon.a (mon.0) 3236 : audit [INF] from='client.? 172.21.15.47:0/1537052409' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:53:00.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:58.825801+0000 osd.4 (osd.4) 273 : cluster [DBG] 4.9 scrub starts 2024-02-23T00:53:00.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:58.827557+0000 osd.4 (osd.4) 274 : cluster [DBG] 4.9 scrub ok 2024-02-23T00:53:00.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:58.852730+0000 osd.4 (osd.4) 275 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:53:00.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:58.852886+0000 osd.4 (osd.4) 276 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:53:00.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:58.947477+0000 osd.3 (osd.3) 308 : cluster [DBG] 4.19 scrub starts 2024-02-23T00:53:00.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:58.949114+0000 osd.3 (osd.3) 309 : cluster [DBG] 4.19 scrub ok 2024-02-23T00:53:00.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:59.276537+0000 mon.a (mon.0) 3237 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:53:00.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:52:59.280565+0000 mon.a (mon.0) 3238 : audit [INF] from='client.? 172.21.15.47:0/1537052409' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:53:00.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:52:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:59.280629+0000 mon.a (mon.0) 3239 : cluster [DBG] osdmap e642: 8 total, 8 up, 7 in 2024-02-23T00:53:00.257 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:53:00.260 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:00.260 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:53:00.361 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:53:00.392 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:00.594 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:53:00.596 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:00.597 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:53:00.682 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:53:00.683 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 20, 2024-02-23T00:53:00.683 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 600, 2024-02-23T00:53:00.683 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [ 2024-02-23T00:53:00.683 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.683 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24832.0:15 21.15 21:ac8d7e44:test-rados-api-smithi047-82517-4::foo:head [watch unwatch cookie 94456142960576] snapc 0=[] ondisk+write+known_if_redirected e111)", 2024-02-23T00:53:00.683 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:00.909423+0000", 2024-02-23T00:53:00.683 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 599.58167955299996, 2024-02-23T00:53:00.683 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.048904532000000001, 2024-02-23T00:53:00.683 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:00.683 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:53:00.683 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:53:00.684 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24832", 2024-02-23T00:53:00.684 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/2893985577", 2024-02-23T00:53:00.684 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 15 2024-02-23T00:53:00.684 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.684 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:00.684 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.684 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:00.684 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.909423+0000", 2024-02-23T00:53:00.684 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:00.684 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.684 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.684 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:00.684 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.909423+0000", 2024-02-23T00:53:00.684 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999976 2024-02-23T00:53:00.685 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.685 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.685 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:00.685 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.909421+0000", 2024-02-23T00:53:00.685 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.5889999999999992e-06 2024-02-23T00:53:00.685 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.685 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.685 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:00.685 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.909430+0000", 2024-02-23T00:53:00.685 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.025760463000000001 2024-02-23T00:53:00.685 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.685 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.685 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:00.685 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.935191+0000", 2024-02-23T00:53:00.685 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002311156 2024-02-23T00:53:00.686 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.686 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.686 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:00.686 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.937502+0000", 2024-02-23T00:53:00.686 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00039955999999999998 2024-02-23T00:53:00.686 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.686 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.686 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:00.686 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.937902+0000", 2024-02-23T00:53:00.686 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.8650999999999998e-05 2024-02-23T00:53:00.686 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.686 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.686 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:00.686 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.937960+0000", 2024-02-23T00:53:00.686 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.020367645 2024-02-23T00:53:00.687 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.687 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.687 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:00.687 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:00.958328+0000", 2024-02-23T00:53:00.687 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.048904532000000001 2024-02-23T00:53:00.687 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.687 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:00.687 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.687 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.687 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.687 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24775.0:2059 20.10 20:0c23f028:test-rados-api-smithi047-82457-16::big:head [writefull 0~25165836] snapc 0=[] ondisk+write+known_if_redirected e112)", 2024-02-23T00:53:00.687 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:01.316014+0000", 2024-02-23T00:53:00.687 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 599.17508934199998, 2024-02-23T00:53:00.687 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.116455373, 2024-02-23T00:53:00.687 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:00.688 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:53:00.688 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:53:00.688 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24775", 2024-02-23T00:53:00.688 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1191937413", 2024-02-23T00:53:00.688 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2059 2024-02-23T00:53:00.688 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.688 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:00.688 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.688 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:00.688 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.316014+0000", 2024-02-23T00:53:00.688 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:00.688 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.689 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.689 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:00.689 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.316014+0000", 2024-02-23T00:53:00.689 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999948 2024-02-23T00:53:00.689 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.689 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.689 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:00.689 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.316008+0000", 2024-02-23T00:53:00.689 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.019319273000000001 2024-02-23T00:53:00.689 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.689 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.689 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:00.690 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.335328+0000", 2024-02-23T00:53:00.690 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.7900000000000001e-05 2024-02-23T00:53:00.690 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.690 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.690 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:00.690 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.335356+0000", 2024-02-23T00:53:00.690 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.3740000000000004e-06 2024-02-23T00:53:00.690 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.690 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.690 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:00.690 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.335365+0000", 2024-02-23T00:53:00.690 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.4500999999999998e-05 2024-02-23T00:53:00.690 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.690 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.691 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:00.691 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.335429+0000", 2024-02-23T00:53:00.691 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000157138 2024-02-23T00:53:00.691 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.691 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.691 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:00.691 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.335587+0000", 2024-02-23T00:53:00.691 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000189459 2024-02-23T00:53:00.691 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.691 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.691 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 5,6", 2024-02-23T00:53:00.691 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.335776+0000", 2024-02-23T00:53:00.691 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.028177522 2024-02-23T00:53:00.691 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.691 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.692 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:53:00.692 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.363954+0000", 2024-02-23T00:53:00.692 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.042488566999999998 2024-02-23T00:53:00.692 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.692 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.692 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:53:00.692 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.406442+0000", 2024-02-23T00:53:00.692 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.025953894000000002 2024-02-23T00:53:00.692 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.692 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.692 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:53:00.692 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.432396+0000", 2024-02-23T00:53:00.692 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.9501e-05 2024-02-23T00:53:00.692 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.693 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.693 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:53:00.693 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.432435+0000", 2024-02-23T00:53:00.693 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.3473999999999998e-05 2024-02-23T00:53:00.693 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.693 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.693 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:00.693 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:01.432469+0000", 2024-02-23T00:53:00.693 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.116455373 2024-02-23T00:53:00.693 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.693 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:00.693 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.693 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.693 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.693 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.14883.0:8 25.9 25:916e78a7:test-rados-api-smithi047-82496-4::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e115)", 2024-02-23T00:53:00.693 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:04.314816+0000", 2024-02-23T00:53:00.694 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 596.17628648000004, 2024-02-23T00:53:00.694 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029852838999999999, 2024-02-23T00:53:00.694 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:00.694 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:53:00.694 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:53:00.694 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.14883", 2024-02-23T00:53:00.694 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1494681915", 2024-02-23T00:53:00.694 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 8 2024-02-23T00:53:00.694 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.694 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:00.694 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.694 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:00.694 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.314816+0000", 2024-02-23T00:53:00.694 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:00.694 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.695 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.695 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:00.695 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.314816+0000", 2024-02-23T00:53:00.695 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:53:00.695 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.695 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.695 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:00.695 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.314815+0000", 2024-02-23T00:53:00.695 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00058973700000000001 2024-02-23T00:53:00.695 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.695 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.695 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:00.695 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.315404+0000", 2024-02-23T00:53:00.695 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0613e-05 2024-02-23T00:53:00.696 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.696 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.696 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:00.696 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.315415+0000", 2024-02-23T00:53:00.696 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.267e-06 2024-02-23T00:53:00.696 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.696 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.696 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:00.696 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.315422+0000", 2024-02-23T00:53:00.696 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014541700000000001 2024-02-23T00:53:00.696 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.696 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.696 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:00.696 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.315568+0000", 2024-02-23T00:53:00.697 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012594799999999999 2024-02-23T00:53:00.697 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.697 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.697 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:00.697 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.315694+0000", 2024-02-23T00:53:00.697 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00016006899999999999 2024-02-23T00:53:00.697 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.697 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.697 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 0,1", 2024-02-23T00:53:00.697 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.315854+0000", 2024-02-23T00:53:00.697 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0030685589999999998 2024-02-23T00:53:00.697 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.697 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.697 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:53:00.697 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.318922+0000", 2024-02-23T00:53:00.698 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0044993280000000004 2024-02-23T00:53:00.698 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.698 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.698 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:53:00.698 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.323422+0000", 2024-02-23T00:53:00.698 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021145840999999999 2024-02-23T00:53:00.698 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.698 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.698 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:53:00.698 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.344568+0000", 2024-02-23T00:53:00.698 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.4187999999999997e-05 2024-02-23T00:53:00.698 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.698 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.698 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:53:00.698 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.344622+0000", 2024-02-23T00:53:00.699 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.7571000000000002e-05 2024-02-23T00:53:00.699 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.699 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.699 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:00.699 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:04.344669+0000", 2024-02-23T00:53:00.699 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029852838999999999 2024-02-23T00:53:00.699 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.699 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:00.699 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.699 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.699 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.700 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(client.25120.0:2 41.1a e116/115 41:5c6776db:::0.obj:head v 116'1, mlcod=0'0)", 2024-02-23T00:53:00.700 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:05.279600+0000", 2024-02-23T00:53:00.700 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 595.211502526, 2024-02-23T00:53:00.700 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.019639007999999999, 2024-02-23T00:53:00.700 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:00.700 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:53:00.700 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:00.700 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.700 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:00.700 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:05.279600+0000", 2024-02-23T00:53:00.700 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:00.700 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.700 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.700 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:00.701 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:05.279600+0000", 2024-02-23T00:53:00.701 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:53:00.701 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.701 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.701 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:00.701 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:05.279600+0000", 2024-02-23T00:53:00.701 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.7436999999999999e-05 2024-02-23T00:53:00.701 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.701 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.701 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:00.701 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:05.279617+0000", 2024-02-23T00:53:00.701 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8709999999999997e-06 2024-02-23T00:53:00.701 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.701 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.702 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:00.702 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:05.279625+0000", 2024-02-23T00:53:00.702 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.5129999999999999e-06 2024-02-23T00:53:00.702 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.702 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.702 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:00.702 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:05.279633+0000", 2024-02-23T00:53:00.702 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.015584344999999999 2024-02-23T00:53:00.702 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.702 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.703 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:00.703 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:05.295217+0000", 2024-02-23T00:53:00.703 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.9414000000000001e-05 2024-02-23T00:53:00.703 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.703 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.703 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:00.703 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:05.295237+0000", 2024-02-23T00:53:00.703 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0039728480000000002 2024-02-23T00:53:00.703 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.703 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.704 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:53:00.704 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:05.299209+0000", 2024-02-23T00:53:00.704 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0018e-05 2024-02-23T00:53:00.704 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.704 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.704 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:00.704 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:05.299239+0000", 2024-02-23T00:53:00.704 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.019639007999999999 2024-02-23T00:53:00.704 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.704 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:00.704 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.704 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.704 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.705 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(osd.1.0:298 20.5 e140/109 -22:a70ec860:::temp_20.5_0_24170_1:head v 0'0, mlcod=112'1)", 2024-02-23T00:53:00.705 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:32.581619+0000", 2024-02-23T00:53:00.705 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 567.90948390899996, 2024-02-23T00:53:00.705 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.027222695000000002, 2024-02-23T00:53:00.705 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:00.705 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:53:00.705 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:00.705 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.705 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:00.705 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.581619+0000", 2024-02-23T00:53:00.705 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:00.705 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.705 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.706 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:00.706 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.581619+0000", 2024-02-23T00:53:00.706 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999976 2024-02-23T00:53:00.706 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.706 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.706 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:00.706 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.581616+0000", 2024-02-23T00:53:00.706 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0060049730000000003 2024-02-23T00:53:00.706 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.706 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.706 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:00.706 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.587621+0000", 2024-02-23T00:53:00.706 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.9785e-05 2024-02-23T00:53:00.706 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.706 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.707 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:00.707 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.587651+0000", 2024-02-23T00:53:00.707 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2072999999999999e-05 2024-02-23T00:53:00.707 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.707 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.707 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:00.707 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.587663+0000", 2024-02-23T00:53:00.707 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000194685 2024-02-23T00:53:00.707 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.707 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.707 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:00.707 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.587858+0000", 2024-02-23T00:53:00.707 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8759e-05 2024-02-23T00:53:00.707 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.707 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.708 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:00.708 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.587887+0000", 2024-02-23T00:53:00.708 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.020889690999999998 2024-02-23T00:53:00.708 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.708 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.708 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:53:00.708 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.608776+0000", 2024-02-23T00:53:00.708 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.5248999999999998e-05 2024-02-23T00:53:00.708 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.708 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.708 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:00.708 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.608842+0000", 2024-02-23T00:53:00.708 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.027222695000000002 2024-02-23T00:53:00.708 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.709 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:00.709 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.709 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.709 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.709 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(osd.1.0:299 20.5 e140/109 -22:a70ec860:::temp_20.5_0_24170_1:head v 0'0, mlcod=112'1)", 2024-02-23T00:53:00.709 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:32.615170+0000", 2024-02-23T00:53:00.709 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 567.87593257200001, 2024-02-23T00:53:00.709 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021461912, 2024-02-23T00:53:00.709 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:00.709 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:53:00.709 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:00.709 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.709 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:00.709 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.615170+0000", 2024-02-23T00:53:00.709 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:00.710 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.710 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.710 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:00.710 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.615170+0000", 2024-02-23T00:53:00.710 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999976 2024-02-23T00:53:00.710 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.710 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.710 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:00.710 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.615168+0000", 2024-02-23T00:53:00.710 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0063487930000000001 2024-02-23T00:53:00.710 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.710 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.710 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:00.710 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.621517+0000", 2024-02-23T00:53:00.710 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.3031000000000003e-05 2024-02-23T00:53:00.711 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.711 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.711 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:00.711 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.621550+0000", 2024-02-23T00:53:00.711 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2002e-05 2024-02-23T00:53:00.711 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.711 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.711 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:00.711 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.621562+0000", 2024-02-23T00:53:00.711 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.0396000000000007e-05 2024-02-23T00:53:00.711 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.711 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.711 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:00.711 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.621652+0000", 2024-02-23T00:53:00.711 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.4893e-05 2024-02-23T00:53:00.712 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.712 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.712 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:00.712 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.621677+0000", 2024-02-23T00:53:00.712 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.014898613999999999 2024-02-23T00:53:00.712 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.712 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.712 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:53:00.712 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.636576+0000", 2024-02-23T00:53:00.712 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.6641999999999997e-05 2024-02-23T00:53:00.712 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.712 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.712 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:00.712 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.636632+0000", 2024-02-23T00:53:00.713 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021461912 2024-02-23T00:53:00.713 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.713 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:00.713 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.713 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.713 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.713 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(client.25529.0:42 63.e e142/127 63:7037fd60:test-rados-api-smithi047-82496-10::foo:head v 142'13, mlcod=140'10)", 2024-02-23T00:53:00.713 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:34.508189+0000", 2024-02-23T00:53:00.713 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 565.982913801, 2024-02-23T00:53:00.713 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.046943154000000001, 2024-02-23T00:53:00.713 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:00.713 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:53:00.713 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:00.714 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.714 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:00.714 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.508189+0000", 2024-02-23T00:53:00.714 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:00.714 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.714 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.714 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:00.714 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.508189+0000", 2024-02-23T00:53:00.714 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.999999 2024-02-23T00:53:00.714 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.714 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.714 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:00.714 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.508188+0000", 2024-02-23T00:53:00.714 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0925e-05 2024-02-23T00:53:00.715 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.715 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.715 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:00.715 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.508199+0000", 2024-02-23T00:53:00.715 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.5159999999999994e-06 2024-02-23T00:53:00.715 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.715 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.715 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:00.715 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.508208+0000", 2024-02-23T00:53:00.715 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.1750000000000002e-06 2024-02-23T00:53:00.715 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.715 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.715 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:00.715 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.508214+0000", 2024-02-23T00:53:00.715 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00015588599999999999 2024-02-23T00:53:00.715 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.716 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.716 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:00.716 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.508370+0000", 2024-02-23T00:53:00.716 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1435000000000001e-05 2024-02-23T00:53:00.716 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.716 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.716 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:00.716 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.508391+0000", 2024-02-23T00:53:00.716 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.046710464 2024-02-23T00:53:00.716 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.716 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.716 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:53:00.716 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.555102+0000", 2024-02-23T00:53:00.716 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0552000000000002e-05 2024-02-23T00:53:00.717 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.717 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.717 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:00.717 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:34.555132+0000", 2024-02-23T00:53:00.717 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.046943154000000001 2024-02-23T00:53:00.717 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.717 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:00.717 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.717 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.717 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.717 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDScrubReserve(6.e RELEASE e144)", 2024-02-23T00:53:00.717 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:36.207155+0000", 2024-02-23T00:53:00.717 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 564.28394764999996, 2024-02-23T00:53:00.717 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.024392191000000001, 2024-02-23T00:53:00.717 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:00.718 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:53:00.718 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:00.718 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.718 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:00.718 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.207155+0000", 2024-02-23T00:53:00.718 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:00.718 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.718 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.718 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:00.718 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.207155+0000", 2024-02-23T00:53:00.718 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999986 2024-02-23T00:53:00.718 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.718 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.718 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:00.719 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.207154+0000", 2024-02-23T00:53:00.719 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.7485999999999996e-05 2024-02-23T00:53:00.719 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.719 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.719 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:00.719 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.207241+0000", 2024-02-23T00:53:00.719 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.394e-05 2024-02-23T00:53:00.719 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.719 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.719 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:00.719 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.207255+0000", 2024-02-23T00:53:00.719 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.6729999999999992e-06 2024-02-23T00:53:00.719 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.719 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.720 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:00.720 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.207263+0000", 2024-02-23T00:53:00.720 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.016180798 2024-02-23T00:53:00.720 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.720 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.720 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:00.720 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.223444+0000", 2024-02-23T00:53:00.720 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.7590000000000003e-06 2024-02-23T00:53:00.720 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.720 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.720 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-02-23T00:53:00.720 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.223452+0000", 2024-02-23T00:53:00.720 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.008038847 2024-02-23T00:53:00.721 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.721 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.721 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:00.721 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.231491+0000", 2024-02-23T00:53:00.721 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1171000000000001e-05 2024-02-23T00:53:00.721 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.721 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.721 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:00.721 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.231512+0000", 2024-02-23T00:53:00.721 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.4987000000000002e-05 2024-02-23T00:53:00.721 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.721 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.721 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:00.721 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:36.231547+0000", 2024-02-23T00:53:00.721 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.024392191000000001 2024-02-23T00:53:00.722 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.722 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:00.722 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.722 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.722 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.722 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWrite(79.6s2 145/139 ECSubWrite(tid=229, reqid=client.31603.0:7, at_version=145'1, trim_to=0'0, roll_forward_to=0'0))", 2024-02-23T00:53:00.722 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:37.220998+0000", 2024-02-23T00:53:00.722 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 563.27010534399994, 2024-02-23T00:53:00.722 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029900639999999999, 2024-02-23T00:53:00.722 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:00.722 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-02-23T00:53:00.722 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:00.722 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.723 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:00.723 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.220998+0000", 2024-02-23T00:53:00.723 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:00.723 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.723 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.723 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:00.723 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.220998+0000", 2024-02-23T00:53:00.723 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:53:00.723 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.723 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.723 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:00.723 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.220997+0000", 2024-02-23T00:53:00.723 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.4640000000000001e-05 2024-02-23T00:53:00.723 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.723 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.724 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:00.724 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.221022+0000", 2024-02-23T00:53:00.724 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.4220000000000006e-06 2024-02-23T00:53:00.724 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.724 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.724 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:00.724 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.221031+0000", 2024-02-23T00:53:00.724 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.4699999999999996e-06 2024-02-23T00:53:00.724 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.724 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.724 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:00.724 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.221039+0000", 2024-02-23T00:53:00.724 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0063503040000000002 2024-02-23T00:53:00.724 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.725 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.725 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:00.725 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.227389+0000", 2024-02-23T00:53:00.725 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.9499999999999996e-06 2024-02-23T00:53:00.725 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.725 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.725 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-02-23T00:53:00.725 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.227399+0000", 2024-02-23T00:53:00.725 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023482154000000002 2024-02-23T00:53:00.725 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.725 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.725 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-02-23T00:53:00.725 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.250881+0000", 2024-02-23T00:53:00.725 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.7016000000000001e-05 2024-02-23T00:53:00.726 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.726 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.726 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:00.726 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.250898+0000", 2024-02-23T00:53:00.726 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029900639999999999 2024-02-23T00:53:00.726 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.726 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:00.726 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.726 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.726 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.726 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(client.15270.0:248 62.14 e145/126 62:28b98b91:test-rados-api-smithi047-82465-12::bar:head v 145'1, mlcod=0'0)", 2024-02-23T00:53:00.726 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:37.222926+0000", 2024-02-23T00:53:00.726 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 563.26817740399997, 2024-02-23T00:53:00.726 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029711336000000001, 2024-02-23T00:53:00.727 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:00.727 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:53:00.727 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:00.727 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.727 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:00.727 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.222926+0000", 2024-02-23T00:53:00.727 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:00.727 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.727 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.727 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:00.727 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.222926+0000", 2024-02-23T00:53:00.727 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:53:00.727 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.727 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.728 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:00.728 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.222925+0000", 2024-02-23T00:53:00.728 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.8849999999999994e-06 2024-02-23T00:53:00.728 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.728 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.728 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:00.728 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.222935+0000", 2024-02-23T00:53:00.728 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.3560000000000004e-06 2024-02-23T00:53:00.728 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.728 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.728 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:00.728 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.222942+0000", 2024-02-23T00:53:00.728 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.0180000000000004e-06 2024-02-23T00:53:00.729 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.729 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.729 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:00.729 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.222949+0000", 2024-02-23T00:53:00.729 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001121143 2024-02-23T00:53:00.729 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.729 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.729 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:00.729 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.224070+0000", 2024-02-23T00:53:00.729 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5017e-05 2024-02-23T00:53:00.729 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.729 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.729 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:00.730 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.224085+0000", 2024-02-23T00:53:00.730 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.028523943999999999 2024-02-23T00:53:00.730 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.730 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.730 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:53:00.730 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.252609+0000", 2024-02-23T00:53:00.730 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.7416e-05 2024-02-23T00:53:00.730 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.730 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.730 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:00.730 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:37.252637+0000", 2024-02-23T00:53:00.730 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029711336000000001 2024-02-23T00:53:00.731 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.731 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:00.731 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.731 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.731 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.731 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(client.15270.0:252 62.18 e146/126 62:1be66eec:test-rados-api-smithi047-82465-12::foo:head v 146'3, mlcod=0'0)", 2024-02-23T00:53:00.731 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:38.406443+0000", 2024-02-23T00:53:00.731 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 562.08466029900001, 2024-02-23T00:53:00.731 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021146257000000002, 2024-02-23T00:53:00.731 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:00.731 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:53:00.731 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:00.731 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.732 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:00.732 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.406443+0000", 2024-02-23T00:53:00.732 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:00.732 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.732 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.732 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:00.732 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.406443+0000", 2024-02-23T00:53:00.732 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:53:00.732 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.732 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.732 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:00.732 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.406442+0000", 2024-02-23T00:53:00.732 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0117e-05 2024-02-23T00:53:00.732 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.733 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.733 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:00.733 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.406452+0000", 2024-02-23T00:53:00.733 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.5140000000000004e-06 2024-02-23T00:53:00.733 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.733 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.733 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:00.733 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.406460+0000", 2024-02-23T00:53:00.733 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8930000000000005e-06 2024-02-23T00:53:00.733 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.733 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.733 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:00.733 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.406468+0000", 2024-02-23T00:53:00.733 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010643227 2024-02-23T00:53:00.733 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.733 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.734 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:00.734 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.417111+0000", 2024-02-23T00:53:00.734 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1742999999999998e-05 2024-02-23T00:53:00.734 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.734 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.734 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:00.734 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.417133+0000", 2024-02-23T00:53:00.734 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010419154999999999 2024-02-23T00:53:00.734 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.734 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.734 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:53:00.734 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.427552+0000", 2024-02-23T00:53:00.734 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.6937000000000002e-05 2024-02-23T00:53:00.734 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.735 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.735 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:00.735 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:38.427589+0000", 2024-02-23T00:53:00.735 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021146257000000002 2024-02-23T00:53:00.735 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.735 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:00.735 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.735 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.735 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.735 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.40541.0:2 116.6 116:602f83fe:::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e181)", 2024-02-23T00:53:00.735 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:13.469852+0000", 2024-02-23T00:53:00.735 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 527.02125128700004, 2024-02-23T00:53:00.735 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.042655076, 2024-02-23T00:53:00.735 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:00.736 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:53:00.736 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:53:00.736 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.40541", 2024-02-23T00:53:00.736 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/865084253", 2024-02-23T00:53:00.736 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2 2024-02-23T00:53:00.736 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.736 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:00.736 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.736 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:00.736 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.469852+0000", 2024-02-23T00:53:00.736 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:00.736 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.736 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.737 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:00.737 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.469852+0000", 2024-02-23T00:53:00.737 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:53:00.737 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.737 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.737 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:00.737 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.469850+0000", 2024-02-23T00:53:00.737 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.166e-06 2024-02-23T00:53:00.737 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.737 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.737 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:00.737 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.469859+0000", 2024-02-23T00:53:00.737 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2982999999999999e-05 2024-02-23T00:53:00.737 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.738 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.738 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:00.738 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.469872+0000", 2024-02-23T00:53:00.738 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.6010000000000001e-06 2024-02-23T00:53:00.738 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.738 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.738 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:00.738 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.469881+0000", 2024-02-23T00:53:00.738 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.8393999999999996e-05 2024-02-23T00:53:00.738 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.738 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.738 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:00.738 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.469950+0000", 2024-02-23T00:53:00.738 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.5129999999999999e-06 2024-02-23T00:53:00.738 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.739 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.739 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-02-23T00:53:00.739 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.469957+0000", 2024-02-23T00:53:00.739 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.013799713 2024-02-23T00:53:00.739 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.739 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.739 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:00.739 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.483757+0000", 2024-02-23T00:53:00.739 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8869999999999992e-06 2024-02-23T00:53:00.739 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.739 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.739 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:53:00.739 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.483765+0000", 2024-02-23T00:53:00.739 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.015659230999999999 2024-02-23T00:53:00.740 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.740 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.740 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:00.740 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.499424+0000", 2024-02-23T00:53:00.740 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00013537900000000001 2024-02-23T00:53:00.740 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.740 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.740 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:00.740 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.499559+0000", 2024-02-23T00:53:00.740 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000169433 2024-02-23T00:53:00.740 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.740 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.740 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 5,6", 2024-02-23T00:53:00.740 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.499729+0000", 2024-02-23T00:53:00.740 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012107788 2024-02-23T00:53:00.741 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.741 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.741 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:53:00.741 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.511837+0000", 2024-02-23T00:53:00.741 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00053866299999999999 2024-02-23T00:53:00.741 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.741 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.741 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:53:00.741 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.512375+0000", 2024-02-23T00:53:00.741 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.4773000000000001e-05 2024-02-23T00:53:00.741 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.741 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.741 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:53:00.741 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.512410+0000", 2024-02-23T00:53:00.741 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.5303000000000002e-05 2024-02-23T00:53:00.742 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.742 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.742 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:53:00.742 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.512465+0000", 2024-02-23T00:53:00.742 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.1287e-05 2024-02-23T00:53:00.742 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.742 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.742 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:00.742 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:13.512507+0000", 2024-02-23T00:53:00.742 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.042655076 2024-02-23T00:53:00.742 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.742 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:00.742 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.742 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.742 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.743 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.40517.0:2 115.1b 115:d8841f39:test-rados-api-smithi047-82457-26::foo:head [writefull 0~8388608] snapc 0=[] ondisk+write+known_if_redirected e182)", 2024-02-23T00:53:00.743 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:14.489687+0000", 2024-02-23T00:53:00.743 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 526.00141589099997, 2024-02-23T00:53:00.743 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.02886706, 2024-02-23T00:53:00.743 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:00.743 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:53:00.743 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:53:00.743 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.40517", 2024-02-23T00:53:00.743 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/3108721855", 2024-02-23T00:53:00.743 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2 2024-02-23T00:53:00.743 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.743 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:00.743 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.743 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:00.743 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.489687+0000", 2024-02-23T00:53:00.744 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:00.744 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.744 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.744 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:00.744 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.489687+0000", 2024-02-23T00:53:00.744 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999976 2024-02-23T00:53:00.744 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.744 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.744 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:00.744 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.489684+0000", 2024-02-23T00:53:00.744 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0052556039999999997 2024-02-23T00:53:00.744 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.744 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.745 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:00.745 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.494940+0000", 2024-02-23T00:53:00.745 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.5494e-05 2024-02-23T00:53:00.745 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.745 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.745 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:00.745 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.494966+0000", 2024-02-23T00:53:00.745 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.4069999999999999e-06 2024-02-23T00:53:00.745 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.745 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.745 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:00.745 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.494975+0000", 2024-02-23T00:53:00.745 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.6199999999999996e-05 2024-02-23T00:53:00.746 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.746 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.746 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:00.746 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.495041+0000", 2024-02-23T00:53:00.746 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000167143 2024-02-23T00:53:00.746 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.746 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.746 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:00.746 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.495208+0000", 2024-02-23T00:53:00.746 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000190027 2024-02-23T00:53:00.746 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.746 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.746 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 0,1", 2024-02-23T00:53:00.746 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.495398+0000", 2024-02-23T00:53:00.746 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.013352564000000001 2024-02-23T00:53:00.747 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.747 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.747 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:53:00.747 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.508751+0000", 2024-02-23T00:53:00.747 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0094962550000000003 2024-02-23T00:53:00.747 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.747 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.747 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:53:00.747 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.518247+0000", 2024-02-23T00:53:00.747 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000230545 2024-02-23T00:53:00.747 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.747 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.747 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:53:00.747 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.518478+0000", 2024-02-23T00:53:00.747 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.8698000000000001e-05 2024-02-23T00:53:00.748 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.748 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.748 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:53:00.748 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.518516+0000", 2024-02-23T00:53:00.748 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7685999999999998e-05 2024-02-23T00:53:00.748 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.748 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.748 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:00.748 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:14.518554+0000", 2024-02-23T00:53:00.748 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.02886706 2024-02-23T00:53:00.748 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.748 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:00.748 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.748 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.749 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.749 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(osd.4.0:477 62.f e199/126 62:f64c5448:test-rados-api-smithi047-82465-14::foo96:4 v 199'80, mlcod=197'74)", 2024-02-23T00:53:00.749 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:31.164866+0000", 2024-02-23T00:53:00.749 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 509.3262373, 2024-02-23T00:53:00.749 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018575001000000001, 2024-02-23T00:53:00.749 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:00.749 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:53:00.749 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:00.749 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.749 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:00.749 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164866+0000", 2024-02-23T00:53:00.749 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:00.749 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.749 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.749 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:00.750 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164866+0000", 2024-02-23T00:53:00.750 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:53:00.750 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.750 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.750 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:00.750 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164865+0000", 2024-02-23T00:53:00.750 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1764e-05 2024-02-23T00:53:00.750 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.750 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.750 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:00.750 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164877+0000", 2024-02-23T00:53:00.750 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.5299999999999996e-06 2024-02-23T00:53:00.750 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.750 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.750 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:00.751 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164885+0000", 2024-02-23T00:53:00.751 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.5619999999999997e-06 2024-02-23T00:53:00.751 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.751 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.751 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:00.751 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.164890+0000", 2024-02-23T00:53:00.751 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0022547829999999998 2024-02-23T00:53:00.751 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.751 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.751 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:00.751 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.167145+0000", 2024-02-23T00:53:00.751 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.8792000000000003e-05 2024-02-23T00:53:00.751 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.751 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.751 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:00.752 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.167193+0000", 2024-02-23T00:53:00.752 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.016211952000000002 2024-02-23T00:53:00.752 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.752 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.752 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:53:00.752 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.183405+0000", 2024-02-23T00:53:00.752 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.5181000000000001e-05 2024-02-23T00:53:00.752 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.752 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.752 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:00.752 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.183441+0000", 2024-02-23T00:53:00.752 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018575001000000001 2024-02-23T00:53:00.752 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.753 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:00.753 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.753 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.753 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.753 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.40754.0:2 145.6 145:602f83fe:::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e222)", 2024-02-23T00:53:00.753 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:54.146645+0000", 2024-02-23T00:53:00.753 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 486.34445817400001, 2024-02-23T00:53:00.753 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021068969, 2024-02-23T00:53:00.753 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:00.753 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:53:00.753 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:53:00.753 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.40754", 2024-02-23T00:53:00.753 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/2235208660", 2024-02-23T00:53:00.753 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2 2024-02-23T00:53:00.753 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.754 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:00.754 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.754 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:00.754 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.146645+0000", 2024-02-23T00:53:00.754 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:00.754 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.754 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.754 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:00.754 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.146645+0000", 2024-02-23T00:53:00.754 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999976 2024-02-23T00:53:00.754 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.754 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.754 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:00.754 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.146643+0000", 2024-02-23T00:53:00.754 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.918e-06 2024-02-23T00:53:00.755 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.755 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.755 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:00.755 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.146651+0000", 2024-02-23T00:53:00.755 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5065000000000001e-05 2024-02-23T00:53:00.755 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.755 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.755 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:00.755 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.146667+0000", 2024-02-23T00:53:00.755 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1534e-05 2024-02-23T00:53:00.755 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.755 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.755 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:00.755 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.146678+0000", 2024-02-23T00:53:00.755 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.5680000000000002e-05 2024-02-23T00:53:00.755 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.756 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.756 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:00.756 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.146734+0000", 2024-02-23T00:53:00.756 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.5449999999999998e-06 2024-02-23T00:53:00.756 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.756 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.756 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-02-23T00:53:00.756 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.146741+0000", 2024-02-23T00:53:00.756 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0075490319999999998 2024-02-23T00:53:00.756 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.756 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.756 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:00.756 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.154290+0000", 2024-02-23T00:53:00.756 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.8569999999999997e-06 2024-02-23T00:53:00.757 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.757 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.757 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:53:00.757 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.154297+0000", 2024-02-23T00:53:00.757 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0090633199999999997 2024-02-23T00:53:00.757 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.757 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.757 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:00.757 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.163361+0000", 2024-02-23T00:53:00.757 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000107849 2024-02-23T00:53:00.757 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.757 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.757 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:00.758 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.163468+0000", 2024-02-23T00:53:00.758 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00015853699999999999 2024-02-23T00:53:00.758 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.758 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.758 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 1,4", 2024-02-23T00:53:00.758 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.163627+0000", 2024-02-23T00:53:00.758 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00066408199999999998 2024-02-23T00:53:00.758 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.758 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.758 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:53:00.758 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.164291+0000", 2024-02-23T00:53:00.758 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000633827 2024-02-23T00:53:00.758 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.758 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.759 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:53:00.759 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.164925+0000", 2024-02-23T00:53:00.759 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002695514 2024-02-23T00:53:00.759 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.759 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.759 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:53:00.759 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.167620+0000", 2024-02-23T00:53:00.759 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.2404999999999997e-05 2024-02-23T00:53:00.759 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.759 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.759 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:53:00.759 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.167673+0000", 2024-02-23T00:53:00.759 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.0967000000000003e-05 2024-02-23T00:53:00.759 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.759 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.760 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:00.760 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.167714+0000", 2024-02-23T00:53:00.760 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021068969 2024-02-23T00:53:00.760 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.760 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:00.760 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.760 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.760 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.760 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.15270.0:2078 62.3 62:c3aa4723:test-rados-api-smithi047-82465-26::foo:head [writefull 0~8] snapc 0=[] ondisk+write+known_if_redirected e245)", 2024-02-23T00:53:00.760 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:45:17.141076+0000", 2024-02-23T00:53:00.760 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 463.35002674899999, 2024-02-23T00:53:00.760 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029775917999999998, 2024-02-23T00:53:00.760 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:00.760 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:53:00.760 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:53:00.761 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.15270", 2024-02-23T00:53:00.761 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/4133111990", 2024-02-23T00:53:00.761 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2078 2024-02-23T00:53:00.761 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.761 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:00.761 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.761 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:00.761 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.141076+0000", 2024-02-23T00:53:00.761 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:00.761 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.761 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.761 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:00.761 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.141076+0000", 2024-02-23T00:53:00.761 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999976 2024-02-23T00:53:00.761 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.762 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.762 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:00.762 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.141074+0000", 2024-02-23T00:53:00.762 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.3340000000000008e-06 2024-02-23T00:53:00.762 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.762 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.762 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:00.762 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.141083+0000", 2024-02-23T00:53:00.762 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3730999999999999e-05 2024-02-23T00:53:00.762 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.762 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.762 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:00.762 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.141097+0000", 2024-02-23T00:53:00.762 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.3159999999999997e-06 2024-02-23T00:53:00.763 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.763 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.763 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:00.763 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.141105+0000", 2024-02-23T00:53:00.763 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.1898999999999999e-05 2024-02-23T00:53:00.763 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.763 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.763 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:00.763 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.141157+0000", 2024-02-23T00:53:00.763 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00016548999999999999 2024-02-23T00:53:00.763 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.763 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.763 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:00.763 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.141323+0000", 2024-02-23T00:53:00.763 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00020201299999999999 2024-02-23T00:53:00.764 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.764 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.764 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 1,4", 2024-02-23T00:53:00.764 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.141525+0000", 2024-02-23T00:53:00.764 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0095168300000000004 2024-02-23T00:53:00.764 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.764 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.764 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:53:00.764 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.151042+0000", 2024-02-23T00:53:00.764 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018502253 2024-02-23T00:53:00.764 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.764 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.764 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:53:00.764 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.169544+0000", 2024-02-23T00:53:00.765 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001218156 2024-02-23T00:53:00.765 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.765 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.765 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:53:00.765 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.170762+0000", 2024-02-23T00:53:00.765 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.3652000000000002e-05 2024-02-23T00:53:00.765 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.765 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.765 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:53:00.765 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.170806+0000", 2024-02-23T00:53:00.765 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.6415000000000002e-05 2024-02-23T00:53:00.765 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.765 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.765 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:00.766 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.170852+0000", 2024-02-23T00:53:00.766 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029775917999999998 2024-02-23T00:53:00.766 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.766 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:00.766 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.766 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.766 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.766 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.30483.0:9 155.c 155:3197012c:::foo_rmxattr:head [rmxattr attr2 in=5b] snapc 0=[] ondisk+write+known_if_redirected e243)", 2024-02-23T00:53:00.766 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:45:17.146980+0000", 2024-02-23T00:53:00.766 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 463.34412272700001, 2024-02-23T00:53:00.766 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.022584933000000001, 2024-02-23T00:53:00.766 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:00.766 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:53:00.766 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:53:00.767 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.30483", 2024-02-23T00:53:00.767 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/3044143263", 2024-02-23T00:53:00.767 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 9 2024-02-23T00:53:00.767 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.767 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:00.767 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.767 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:00.767 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.146980+0000", 2024-02-23T00:53:00.767 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:00.767 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.767 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.767 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:00.767 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.146980+0000", 2024-02-23T00:53:00.768 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:53:00.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.768 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.768 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:00.768 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.146978+0000", 2024-02-23T00:53:00.768 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0020000000000006e-06 2024-02-23T00:53:00.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.768 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.768 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:00.768 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.146986+0000", 2024-02-23T00:53:00.768 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4829e-05 2024-02-23T00:53:00.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.768 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.768 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:00.769 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.147001+0000", 2024-02-23T00:53:00.769 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.8660000000000001e-06 2024-02-23T00:53:00.769 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.769 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.769 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:00.769 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.147011+0000", 2024-02-23T00:53:00.769 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0099999999999995e-05 2024-02-23T00:53:00.769 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.769 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.769 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:00.769 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.147091+0000", 2024-02-23T00:53:00.769 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00010793000000000001 2024-02-23T00:53:00.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.770 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.770 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:00.770 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.147199+0000", 2024-02-23T00:53:00.770 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.022366104000000001 2024-02-23T00:53:00.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.770 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.770 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:00.770 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:17.169565+0000", 2024-02-23T00:53:00.770 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.022584933000000001 2024-02-23T00:53:00.770 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.770 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:00.770 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.771 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.771 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24820.0:1124 15.0 15:00000000::::head [pgnls start_epoch 307] snapc 0=[] RETRY=1 ondisk+retry+read+ignore_overlay+known_if_redirected e337)", 2024-02-23T00:53:00.771 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:46:50.181661+0000", 2024-02-23T00:53:00.771 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 370.30944199200002, 2024-02-23T00:53:00.771 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.005913203, 2024-02-23T00:53:00.771 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:00.771 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:53:00.771 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:53:00.771 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24820", 2024-02-23T00:53:00.771 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/2979574659", 2024-02-23T00:53:00.771 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 1124 2024-02-23T00:53:00.771 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.771 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:00.772 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.772 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:00.772 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:50.181661+0000", 2024-02-23T00:53:00.772 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:00.772 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.772 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.772 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:00.772 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:50.181661+0000", 2024-02-23T00:53:00.772 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:53:00.772 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.772 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.772 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:00.772 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:50.181659+0000", 2024-02-23T00:53:00.773 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.6400000000000003e-06 2024-02-23T00:53:00.773 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.773 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.773 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:00.773 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:50.181668+0000", 2024-02-23T00:53:00.773 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1449e-05 2024-02-23T00:53:00.773 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.773 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.773 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:00.773 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:50.181679+0000", 2024-02-23T00:53:00.773 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.6820000000000003e-06 2024-02-23T00:53:00.773 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.773 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.773 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:00.773 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:50.181687+0000", 2024-02-23T00:53:00.774 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00045155199999999998 2024-02-23T00:53:00.774 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.774 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.774 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:00.774 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:50.182138+0000", 2024-02-23T00:53:00.774 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1856e-05 2024-02-23T00:53:00.774 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.774 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.774 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:53:00.774 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:50.182150+0000", 2024-02-23T00:53:00.774 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0051927949999999 2024-02-23T00:53:00.774 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.774 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.774 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:00.774 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:51.187343+0000", 2024-02-23T00:53:00.775 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.9544e-05 2024-02-23T00:53:00.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.775 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.775 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:00.775 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:51.187383+0000", 2024-02-23T00:53:00.775 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000191547 2024-02-23T00:53:00.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.775 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.775 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:00.775 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:51.187574+0000", 2024-02-23T00:53:00.775 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.005913203 2024-02-23T00:53:00.775 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.775 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:00.775 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.776 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.776 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24820.0:1124 15.0 15:00000000::::head [pgnls start_epoch 307] snapc 0=[] RETRY=2 ondisk+retry+read+ignore_overlay+known_if_redirected e338)", 2024-02-23T00:53:00.776 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:46:50.182461+0000", 2024-02-23T00:53:00.776 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 370.308641975, 2024-02-23T00:53:00.776 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.005281654, 2024-02-23T00:53:00.776 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:00.776 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:53:00.776 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:53:00.776 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24820", 2024-02-23T00:53:00.776 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/2979574659", 2024-02-23T00:53:00.776 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 1124 2024-02-23T00:53:00.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.776 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:00.777 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.777 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:00.777 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:50.182461+0000", 2024-02-23T00:53:00.777 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:00.777 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.777 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.777 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:00.777 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:50.182461+0000", 2024-02-23T00:53:00.777 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.999999 2024-02-23T00:53:00.777 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.777 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.777 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:00.777 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:50.182460+0000", 2024-02-23T00:53:00.777 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.8390000000000003e-06 2024-02-23T00:53:00.778 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.778 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.778 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:00.778 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:50.182466+0000", 2024-02-23T00:53:00.778 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0042e-05 2024-02-23T00:53:00.778 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.778 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.778 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:00.778 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:50.182476+0000", 2024-02-23T00:53:00.778 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.3230000000000003e-06 2024-02-23T00:53:00.778 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.778 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.778 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:00.778 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:50.182482+0000", 2024-02-23T00:53:00.778 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.3739000000000006e-05 2024-02-23T00:53:00.778 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.779 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.779 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:00.779 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:50.182566+0000", 2024-02-23T00:53:00.779 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.098e-06 2024-02-23T00:53:00.779 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.779 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.779 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:53:00.779 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:50.182575+0000", 2024-02-23T00:53:00.779 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.005027262 2024-02-23T00:53:00.779 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.779 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.779 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:00.779 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:51.187602+0000", 2024-02-23T00:53:00.779 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7106e-05 2024-02-23T00:53:00.779 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.780 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.780 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:00.780 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:51.187639+0000", 2024-02-23T00:53:00.780 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00010318 2024-02-23T00:53:00.780 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:00.780 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:00.780 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:00.780 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:51.187743+0000", 2024-02-23T00:53:00.780 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.005281654 2024-02-23T00:53:00.780 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.780 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:00.780 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.781 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:00.781 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:00.781 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:53:00.781 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:53:00.782 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:53:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:53:00] "GET /metrics HTTP/1.1" 200 190922 "" "Prometheus/2.33.4" 2024-02-23T00:53:00.812 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:00.859 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:53:00.859 INFO:teuthology.orchestra.run.smithi196.stdout: "success": "filestore_inject_stall = '3' (not observed, change may require restart) " 2024-02-23T00:53:00.859 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:53:00.881 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:58.514562+0000 osd.1 (osd.1) 267 : cluster [DBG] 18.5 scrub starts 2024-02-23T00:53:00.881 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:58.516224+0000 osd.1 (osd.1) 268 : cluster [DBG] 18.5 scrub ok 2024-02-23T00:53:00.881 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:59.365672+0000 mgr.y (mgr.14517) 568 : cluster [DBG] pgmap v971: 152 pgs: 1 active+clean+snaptrim, 151 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:53:00.881 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:59.796601+0000 osd.4 (osd.4) 277 : cluster [DBG] 18.19 scrub starts 2024-02-23T00:53:00.881 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:59.798980+0000 osd.4 (osd.4) 278 : cluster [DBG] 18.19 scrub ok 2024-02-23T00:53:00.881 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:59.830606+0000 osd.0 (osd.0) 269 : cluster [DBG] 2.1f scrub starts 2024-02-23T00:53:00.881 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:52:59.831406+0000 osd.0 (osd.0) 270 : cluster [DBG] 2.1f scrub ok 2024-02-23T00:53:00.881 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:00.286307+0000 mon.a (mon.0) 3240 : cluster [DBG] osdmap e643: 8 total, 8 up, 7 in 2024-02-23T00:53:00.881 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:00.294271+0000 mon.c (mon.2) 366 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-110","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:00.881 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:00.294974+0000 mon.a (mon.0) 3241 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-110","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:01.031 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:58.514562+0000 osd.1 (osd.1) 267 : cluster [DBG] 18.5 scrub starts 2024-02-23T00:53:01.031 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:58.516224+0000 osd.1 (osd.1) 268 : cluster [DBG] 18.5 scrub ok 2024-02-23T00:53:01.031 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:59.365672+0000 mgr.y (mgr.14517) 568 : cluster [DBG] pgmap v971: 152 pgs: 1 active+clean+snaptrim, 151 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:53:01.031 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:59.796601+0000 osd.4 (osd.4) 277 : cluster [DBG] 18.19 scrub starts 2024-02-23T00:53:01.031 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:59.798980+0000 osd.4 (osd.4) 278 : cluster [DBG] 18.19 scrub ok 2024-02-23T00:53:01.031 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:59.830606+0000 osd.0 (osd.0) 269 : cluster [DBG] 2.1f scrub starts 2024-02-23T00:53:01.031 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:52:59.831406+0000 osd.0 (osd.0) 270 : cluster [DBG] 2.1f scrub ok 2024-02-23T00:53:01.031 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:00.286307+0000 mon.a (mon.0) 3240 : cluster [DBG] osdmap e643: 8 total, 8 up, 7 in 2024-02-23T00:53:01.031 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:00.294271+0000 mon.c (mon.2) 366 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-110","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:01.032 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:00.294974+0000 mon.a (mon.0) 3241 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-110","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:01.032 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:58.514562+0000 osd.1 (osd.1) 267 : cluster [DBG] 18.5 scrub starts 2024-02-23T00:53:01.032 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:58.516224+0000 osd.1 (osd.1) 268 : cluster [DBG] 18.5 scrub ok 2024-02-23T00:53:01.032 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:59.365672+0000 mgr.y (mgr.14517) 568 : cluster [DBG] pgmap v971: 152 pgs: 1 active+clean+snaptrim, 151 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:53:01.032 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:59.796601+0000 osd.4 (osd.4) 277 : cluster [DBG] 18.19 scrub starts 2024-02-23T00:53:01.032 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:59.798980+0000 osd.4 (osd.4) 278 : cluster [DBG] 18.19 scrub ok 2024-02-23T00:53:01.032 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:59.830606+0000 osd.0 (osd.0) 269 : cluster [DBG] 2.1f scrub starts 2024-02-23T00:53:01.032 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:52:59.831406+0000 osd.0 (osd.0) 270 : cluster [DBG] 2.1f scrub ok 2024-02-23T00:53:01.032 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:00.286307+0000 mon.a (mon.0) 3240 : cluster [DBG] osdmap e643: 8 total, 8 up, 7 in 2024-02-23T00:53:01.033 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:00.294271+0000 mon.c (mon.2) 366 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-110","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:01.033 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:00.294974+0000 mon.a (mon.0) 3241 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-110","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:01.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:00.407316+0000 osd.7 (osd.7) 311 : cluster [DBG] 18.2 deep-scrub starts 2024-02-23T00:53:01.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:00.410221+0000 osd.7 (osd.7) 312 : cluster [DBG] 18.2 deep-scrub ok 2024-02-23T00:53:01.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:00.844328+0000 osd.0 (osd.0) 271 : cluster [DBG] 5.1e scrub starts 2024-02-23T00:53:01.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:00.845158+0000 osd.0 (osd.0) 272 : cluster [DBG] 5.1e scrub ok 2024-02-23T00:53:01.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:01.287874+0000 mon.a (mon.0) 3242 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-110","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:53:01.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:01.288018+0000 mon.a (mon.0) 3243 : cluster [DBG] osdmap e644: 8 total, 8 up, 7 in 2024-02-23T00:53:01.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:01.309807+0000 mon.c (mon.2) 367 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-110", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:01.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:01.310314+0000 mon.a (mon.0) 3244 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-110", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:01.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:01.366199+0000 mgr.y (mgr.14517) 569 : cluster [DBG] pgmap v974: 184 pgs: 32 unknown, 1 active+clean+snaptrim, 151 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:53:02.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:00.407316+0000 osd.7 (osd.7) 311 : cluster [DBG] 18.2 deep-scrub starts 2024-02-23T00:53:02.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:00.410221+0000 osd.7 (osd.7) 312 : cluster [DBG] 18.2 deep-scrub ok 2024-02-23T00:53:02.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:00.844328+0000 osd.0 (osd.0) 271 : cluster [DBG] 5.1e scrub starts 2024-02-23T00:53:02.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:00.845158+0000 osd.0 (osd.0) 272 : cluster [DBG] 5.1e scrub ok 2024-02-23T00:53:02.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:01.287874+0000 mon.a (mon.0) 3242 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-110","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:53:02.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:01.288018+0000 mon.a (mon.0) 3243 : cluster [DBG] osdmap e644: 8 total, 8 up, 7 in 2024-02-23T00:53:02.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:01.309807+0000 mon.c (mon.2) 367 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-110", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:02.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:01.310314+0000 mon.a (mon.0) 3244 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-110", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:02.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:01.366199+0000 mgr.y (mgr.14517) 569 : cluster [DBG] pgmap v974: 184 pgs: 32 unknown, 1 active+clean+snaptrim, 151 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:53:02.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:00.407316+0000 osd.7 (osd.7) 311 : cluster [DBG] 18.2 deep-scrub starts 2024-02-23T00:53:02.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:00.410221+0000 osd.7 (osd.7) 312 : cluster [DBG] 18.2 deep-scrub ok 2024-02-23T00:53:02.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:00.844328+0000 osd.0 (osd.0) 271 : cluster [DBG] 5.1e scrub starts 2024-02-23T00:53:02.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:00.845158+0000 osd.0 (osd.0) 272 : cluster [DBG] 5.1e scrub ok 2024-02-23T00:53:02.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:01.287874+0000 mon.a (mon.0) 3242 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-110","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:53:02.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:01.288018+0000 mon.a (mon.0) 3243 : cluster [DBG] osdmap e644: 8 total, 8 up, 7 in 2024-02-23T00:53:02.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:01.309807+0000 mon.c (mon.2) 367 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-110", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:02.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:01.310314+0000 mon.a (mon.0) 3244 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-110", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:02.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:01.366199+0000 mgr.y (mgr.14517) 569 : cluster [DBG] pgmap v974: 184 pgs: 32 unknown, 1 active+clean+snaptrim, 151 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:53:02.263 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:53:02.266 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:02.267 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:53:02.368 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:53:02.402 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:02.425 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:53:02.427 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:02.428 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:53:02.529 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:53:02.561 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:02.661 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.3 dump_ops_in_flight 2024-02-23T00:53:02.661 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:53:02.662 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:02.663 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:53:02.694 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:53:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:53:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:53:02.764 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:53:02.793 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:02.811 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:01.387331+0000 osd.7 (osd.7) 313 : cluster [DBG] 18.6 scrub starts 2024-02-23T00:53:02.811 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:01.388168+0000 osd.7 (osd.7) 314 : cluster [DBG] 18.6 scrub ok 2024-02-23T00:53:02.811 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:01.801483+0000 osd.0 (osd.0) 273 : cluster [DBG] 18.8 scrub starts 2024-02-23T00:53:02.811 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:01.802400+0000 osd.0 (osd.0) 274 : cluster [DBG] 18.8 scrub ok 2024-02-23T00:53:02.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:02.292007+0000 mon.a (mon.0) 3245 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-110", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:53:02.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:02.292096+0000 mon.a (mon.0) 3246 : cluster [DBG] osdmap e645: 8 total, 8 up, 7 in 2024-02-23T00:53:02.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:02.293676+0000 mon.c (mon.2) 368 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-110"}]: dispatch 2024-02-23T00:53:02.812 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:02.294049+0000 mon.a (mon.0) 3247 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-110"}]: dispatch 2024-02-23T00:53:02.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:01.387331+0000 osd.7 (osd.7) 313 : cluster [DBG] 18.6 scrub starts 2024-02-23T00:53:02.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:01.388168+0000 osd.7 (osd.7) 314 : cluster [DBG] 18.6 scrub ok 2024-02-23T00:53:02.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:01.801483+0000 osd.0 (osd.0) 273 : cluster [DBG] 18.8 scrub starts 2024-02-23T00:53:02.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:01.802400+0000 osd.0 (osd.0) 274 : cluster [DBG] 18.8 scrub ok 2024-02-23T00:53:02.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:02.292007+0000 mon.a (mon.0) 3245 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-110", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:53:02.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:02.292096+0000 mon.a (mon.0) 3246 : cluster [DBG] osdmap e645: 8 total, 8 up, 7 in 2024-02-23T00:53:02.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:02.293676+0000 mon.c (mon.2) 368 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-110"}]: dispatch 2024-02-23T00:53:02.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:02.294049+0000 mon.a (mon.0) 3247 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-110"}]: dispatch 2024-02-23T00:53:02.818 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:53:02.821 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:02.822 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:53:02.922 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:53:02.951 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:03.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.064+0000 7f6808367700 1 -- 172.21.15.47:0/46119003 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6800102210 msgr2=0x7f6800104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:03.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.064+0000 7f6808367700 1 --2- 172.21.15.47:0/46119003 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6800102210 0x7f6800104600 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 rx=0x7f67f8009be0 tx=0x7f67f8004d10).stop 2024-02-23T00:53:03.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.064+0000 7f6808367700 1 -- 172.21.15.47:0/46119003 shutdown_connections 2024-02-23T00:53:03.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.064+0000 7f6808367700 1 --2- 172.21.15.47:0/46119003 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6800102210 0x7f6800104600 unknown :-1 s=CLOSED pgs=2762 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:03.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.064+0000 7f6808367700 1 --2- 172.21.15.47:0/46119003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68000f8cb0 0x7f68000f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:03.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.064+0000 7f6808367700 1 --2- 172.21.15.47:0/46119003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68000f9610 0x7f6800101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:03.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.064+0000 7f6808367700 1 -- 172.21.15.47:0/46119003 >> 172.21.15.47:0/46119003 conn(0x7f68000f2820 msgr2=0x7f68000f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:03.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.064+0000 7f6808367700 1 -- 172.21.15.47:0/46119003 shutdown_connections 2024-02-23T00:53:03.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.064+0000 7f6808367700 1 -- 172.21.15.47:0/46119003 wait complete. 2024-02-23T00:53:03.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.064+0000 7f6808367700 1 librados: starting msgr at 2024-02-23T00:53:03.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.064+0000 7f6808367700 1 librados: starting objecter 2024-02-23T00:53:03.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.064+0000 7f6808367700 1 Processor -- start 2024-02-23T00:53:03.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.064+0000 7f6808367700 1 -- start start 2024-02-23T00:53:03.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.064+0000 7f6808367700 1 librados: setting wanted keys 2024-02-23T00:53:03.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.064+0000 7f6808367700 1 librados: calling monclient init 2024-02-23T00:53:03.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.065+0000 7f6808367700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68000f8cb0 0x7f68000f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:03.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.065+0000 7f6808367700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68000f9610 0x7f68000f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:03.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.065+0000 7f6808367700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6800102210 0x7f68000f7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:03.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.065+0000 7f6808367700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6800116350 con 0x7f68000f8cb0 2024-02-23T00:53:03.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.065+0000 7f6808367700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68001164c0 con 0x7f68000f9610 2024-02-23T00:53:03.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.065+0000 7f6808367700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6800116680 con 0x7f6800102210 2024-02-23T00:53:03.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.065+0000 7f6806103700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68000f8cb0 0x7f68000f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:03.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.065+0000 7f6806103700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68000f8cb0 0x7f68000f6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46588/0 (socket says 172.21.15.47:46588) 2024-02-23T00:53:03.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.065+0000 7f6806904700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6800102210 0x7f68000f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:03.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.065+0000 7f6806103700 1 -- 172.21.15.47:0/2787991081 learned_addr learned my addr 172.21.15.47:0/2787991081 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:53:03.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.065+0000 7f6805902700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68000f9610 0x7f68000f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:03.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.065+0000 7f6806103700 1 -- 172.21.15.47:0/2787991081 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6800102210 msgr2=0x7f68000f7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:03.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.065+0000 7f6806103700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6800102210 0x7f68000f7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:03.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.065+0000 7f6806103700 1 -- 172.21.15.47:0/2787991081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68000f9610 msgr2=0x7f68000f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:03.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.065+0000 7f6806103700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68000f9610 0x7f68000f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:03.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.065+0000 7f6806103700 1 -- 172.21.15.47:0/2787991081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67fc005040 con 0x7f68000f8cb0 2024-02-23T00:53:03.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.065+0000 7f6806103700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68000f8cb0 0x7f68000f6c60 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 rx=0x7f67fc0048f0 tx=0x7f67fc009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:03.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.065+0000 7f67f77fe700 10 client.?.objecter ms_handle_connect 0x7f68000f8cb0 2024-02-23T00:53:03.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.065+0000 7f67f77fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:53:03.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.065+0000 7f67f77fe700 1 -- 172.21.15.47:0/2787991081 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc007b00 con 0x7f68000f8cb0 2024-02-23T00:53:03.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.065+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67f80097c0 con 0x7f68000f8cb0 2024-02-23T00:53:03.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.065+0000 7f67f77fe700 1 -- 172.21.15.47:0/2787991081 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f67fc00f040 con 0x7f68000f8cb0 2024-02-23T00:53:03.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.065+0000 7f67f77fe700 1 -- 172.21.15.47:0/2787991081 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc014820 con 0x7f68000f8cb0 2024-02-23T00:53:03.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.065+0000 7f6808367700 10 client.31530.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:53:03.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.065+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6800116ce0 con 0x7f68000f8cb0 2024-02-23T00:53:03.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.065+0000 7f6808367700 1 librados: init done 2024-02-23T00:53:03.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.066+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f680005fa60 con 0x7f68000f8cb0 2024-02-23T00:53:03.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.067+0000 7f67f77fe700 1 -- 172.21.15.47:0/2787991081 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f67fc014c50 con 0x7f68000f8cb0 2024-02-23T00:53:03.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.067+0000 7f67f77fe700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f67ec062cc0 0x7f67ec065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:03.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.067+0000 7f67f77fe700 1 -- 172.21.15.47:0/2787991081 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(645..645 src has 482..645) v4 ==== 9662+0+0 (secure 0 0 0) 0x7f67fc07d680 con 0x7f68000f8cb0 2024-02-23T00:53:03.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.067+0000 7f67f77fe700 10 client.31530.objecter ms_dispatch 0x7f6800001550 osd_map(645..645 src has 482..645) v4 2024-02-23T00:53:03.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.067+0000 7f67f77fe700 3 client.31530.objecter handle_osd_map got epochs [645,645] > 0 2024-02-23T00:53:03.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.067+0000 7f67f77fe700 3 client.31530.objecter handle_osd_map decoding full epoch 645 2024-02-23T00:53:03.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.067+0000 7f6805902700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f67ec062cc0 0x7f67ec065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:03.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.067+0000 7f67f77fe700 20 client.31530.objecter dump_active .. 0 homeless 2024-02-23T00:53:03.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.067+0000 7f67f77fe700 1 -- 172.21.15.47:0/2787991081 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v645) v1 ==== 54+0+16 (secure 0 0 0) 0x7f67fc07db40 con 0x7f68000f8cb0 2024-02-23T00:53:03.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.068+0000 7f6808367700 10 client.31530.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:03.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.068+0000 7f6808367700 15 client.31530.objecter _session_command_op_assign -1 1 2024-02-23T00:53:03.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.068+0000 7f6808367700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f68000fc1a0 0x7f68000fc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:03.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.068+0000 7f6808367700 20 client.31530.objecter _get_session s=0x7f680005ee80 osd=0 3 2024-02-23T00:53:03.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.068+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f680005ee80 osd=0 3 2024-02-23T00:53:03.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.068+0000 7f6808367700 20 client.31530.objecter _get_session s=0x7f680005ee80 osd=0 3 2024-02-23T00:53:03.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.068+0000 7f6808367700 15 client.31530.objecter _session_command_op_remove -1 1 2024-02-23T00:53:03.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.068+0000 7f6808367700 20 client.31530.objecter get_session s=0x7f680005ee80 osd=0 3 2024-02-23T00:53:03.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.068+0000 7f6808367700 15 client.31530.objecter _session_command_op_assign 0 1 2024-02-23T00:53:03.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.068+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f680005ee80 osd=0 4 2024-02-23T00:53:03.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.068+0000 7f6808367700 10 client.31530.objecter _send_command 1 2024-02-23T00:53:03.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.068+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6800107010 con 0x7f68000fc1a0 2024-02-23T00:53:03.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.068+0000 7f6805902700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f67ec062cc0 0x7f67ec065180 secure :-1 s=READY pgs=3151 cs=0 l=1 rev1=1 rx=0x7f68000f8400 tx=0x7f67f000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:03.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.068+0000 7f67f77fe700 10 client.31530.objecter ms_handle_connect 0x7f67ec062cc0 2024-02-23T00:53:03.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.068+0000 7f6806904700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f68000fc1a0 0x7f68000fc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:03.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.068+0000 7f67f77fe700 10 client.31530.objecter ms_handle_connect 0x7f68000fc1a0 2024-02-23T00:53:03.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.068+0000 7f6806904700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f68000fc1a0 0x7f68000fc580 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:03.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.069+0000 7f67f77fe700 1 -- 172.21.15.47:0/2787991081 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f6800107010 con 0x7f68000fc1a0 2024-02-23T00:53:03.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.069+0000 7f67f77fe700 10 client.31530.objecter ms_dispatch 0x7f6800001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:53:03.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.069+0000 7f67f77fe700 10 client.31530.objecter _finish_command 1 = system:0 2024-02-23T00:53:03.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.069+0000 7f67f77fe700 20 client.31530.objecter put_session s=0x7f680005ee80 osd=0 4 2024-02-23T00:53:03.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.069+0000 7f67f77fe700 15 client.31530.objecter _session_command_op_remove 0 1 2024-02-23T00:53:03.076 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:53:03.080 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:03.080 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:53:03.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.084+0000 7f6808367700 10 client.31530.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:53:03.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.084+0000 7f6808367700 15 client.31530.objecter _session_command_op_assign -1 2 2024-02-23T00:53:03.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.084+0000 7f6808367700 20 client.31530.objecter _get_session s=0x7f680005ee80 osd=0 3 2024-02-23T00:53:03.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.084+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f680005ee80 osd=0 3 2024-02-23T00:53:03.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.084+0000 7f6808367700 20 client.31530.objecter _get_session s=0x7f680005ee80 osd=0 3 2024-02-23T00:53:03.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.084+0000 7f6808367700 15 client.31530.objecter _session_command_op_remove -1 2 2024-02-23T00:53:03.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.084+0000 7f6808367700 20 client.31530.objecter get_session s=0x7f680005ee80 osd=0 3 2024-02-23T00:53:03.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.084+0000 7f6808367700 15 client.31530.objecter _session_command_op_assign 0 2 2024-02-23T00:53:03.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.084+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f680005ee80 osd=0 4 2024-02-23T00:53:03.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.084+0000 7f6808367700 10 client.31530.objecter _send_command 2 2024-02-23T00:53:03.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.084+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f68000273c0 con 0x7f68000fc1a0 2024-02-23T00:53:03.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.086+0000 7f67f77fe700 1 -- 172.21.15.47:0/2787991081 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f68000273c0 con 0x7f68000fc1a0 2024-02-23T00:53:03.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.086+0000 7f67f77fe700 10 client.31530.objecter ms_dispatch 0x7f6800001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:53:03.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.086+0000 7f67f77fe700 10 client.31530.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:53:03.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.086+0000 7f67f77fe700 20 client.31530.objecter put_session s=0x7f680005ee80 osd=0 4 2024-02-23T00:53:03.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.086+0000 7f67f77fe700 15 client.31530.objecter _session_command_op_remove 0 2 2024-02-23T00:53:03.089 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:53:03.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.086+0000 7f6808367700 10 client.31530.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:03.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.086+0000 7f6808367700 15 client.31530.objecter _session_command_op_assign -1 3 2024-02-23T00:53:03.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.086+0000 7f6808367700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f68000598f0 0x7f6800059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:03.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.086+0000 7f6808367700 20 client.31530.objecter _get_session s=0x7f6800028710 osd=1 3 2024-02-23T00:53:03.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.086+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f6800028710 osd=1 3 2024-02-23T00:53:03.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.086+0000 7f6808367700 20 client.31530.objecter _get_session s=0x7f6800028710 osd=1 3 2024-02-23T00:53:03.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.086+0000 7f6808367700 15 client.31530.objecter _session_command_op_remove -1 3 2024-02-23T00:53:03.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.086+0000 7f6808367700 20 client.31530.objecter get_session s=0x7f6800028710 osd=1 3 2024-02-23T00:53:03.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.086+0000 7f6808367700 15 client.31530.objecter _session_command_op_assign 1 3 2024-02-23T00:53:03.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.086+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f6800028710 osd=1 4 2024-02-23T00:53:03.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.086+0000 7f6808367700 10 client.31530.objecter _send_command 3 2024-02-23T00:53:03.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.086+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f680011d6e0 con 0x7f68000598f0 2024-02-23T00:53:03.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.086+0000 7f6806103700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f68000598f0 0x7f6800059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:03.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.086+0000 7f67f77fe700 10 client.31530.objecter ms_handle_connect 0x7f68000598f0 2024-02-23T00:53:03.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.086+0000 7f6806103700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f68000598f0 0x7f6800059cd0 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:03.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.087+0000 7f67f77fe700 1 -- 172.21.15.47:0/2787991081 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f680011d6e0 con 0x7f68000598f0 2024-02-23T00:53:03.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.087+0000 7f67f77fe700 10 client.31530.objecter ms_dispatch 0x7f6800001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:53:03.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.087+0000 7f67f77fe700 10 client.31530.objecter _finish_command 3 = system:0 2024-02-23T00:53:03.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.087+0000 7f67f77fe700 20 client.31530.objecter put_session s=0x7f6800028710 osd=1 4 2024-02-23T00:53:03.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.087+0000 7f67f77fe700 15 client.31530.objecter _session_command_op_remove 1 3 2024-02-23T00:53:03.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.102+0000 7f6808367700 10 client.31530.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:53:03.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.102+0000 7f6808367700 15 client.31530.objecter _session_command_op_assign -1 4 2024-02-23T00:53:03.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.102+0000 7f6808367700 20 client.31530.objecter _get_session s=0x7f6800028710 osd=1 3 2024-02-23T00:53:03.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.102+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f6800028710 osd=1 3 2024-02-23T00:53:03.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.102+0000 7f6808367700 20 client.31530.objecter _get_session s=0x7f6800028710 osd=1 3 2024-02-23T00:53:03.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.102+0000 7f6808367700 15 client.31530.objecter _session_command_op_remove -1 4 2024-02-23T00:53:03.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.102+0000 7f6808367700 20 client.31530.objecter get_session s=0x7f6800028710 osd=1 3 2024-02-23T00:53:03.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.102+0000 7f6808367700 15 client.31530.objecter _session_command_op_assign 1 4 2024-02-23T00:53:03.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.102+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f6800028710 osd=1 4 2024-02-23T00:53:03.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.102+0000 7f6808367700 10 client.31530.objecter _send_command 4 2024-02-23T00:53:03.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.102+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f6800059360 con 0x7f68000598f0 2024-02-23T00:53:03.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.103+0000 7f67f77fe700 1 -- 172.21.15.47:0/2787991081 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f6800059360 con 0x7f68000598f0 2024-02-23T00:53:03.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.103+0000 7f67f77fe700 10 client.31530.objecter ms_dispatch 0x7f6800001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:53:03.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.103+0000 7f67f77fe700 10 client.31530.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:53:03.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.103+0000 7f67f77fe700 20 client.31530.objecter put_session s=0x7f6800028710 osd=1 4 2024-02-23T00:53:03.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.103+0000 7f67f77fe700 15 client.31530.objecter _session_command_op_remove 1 4 2024-02-23T00:53:03.106 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:53:03.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.103+0000 7f6808367700 10 client.31530.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:03.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.103+0000 7f6808367700 15 client.31530.objecter _session_command_op_assign -1 5 2024-02-23T00:53:03.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.103+0000 7f6808367700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f6800028270 0x7f680011c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:03.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.103+0000 7f6808367700 20 client.31530.objecter _get_session s=0x7f6800059770 osd=2 3 2024-02-23T00:53:03.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.103+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f6800059770 osd=2 3 2024-02-23T00:53:03.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.103+0000 7f6808367700 20 client.31530.objecter _get_session s=0x7f6800059770 osd=2 3 2024-02-23T00:53:03.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.103+0000 7f6808367700 15 client.31530.objecter _session_command_op_remove -1 5 2024-02-23T00:53:03.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.103+0000 7f6808367700 20 client.31530.objecter get_session s=0x7f6800059770 osd=2 3 2024-02-23T00:53:03.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.103+0000 7f6808367700 15 client.31530.objecter _session_command_op_assign 2 5 2024-02-23T00:53:03.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.103+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f6800059770 osd=2 4 2024-02-23T00:53:03.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.103+0000 7f6808367700 10 client.31530.objecter _send_command 5 2024-02-23T00:53:03.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.103+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f680011c5f0 con 0x7f6800028270 2024-02-23T00:53:03.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.104+0000 7f6805902700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f6800028270 0x7f680011c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:03.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.104+0000 7f67f77fe700 10 client.31530.objecter ms_handle_connect 0x7f6800028270 2024-02-23T00:53:03.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.104+0000 7f6805902700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f6800028270 0x7f680011c100 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:03.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.104+0000 7f67f77fe700 1 -- 172.21.15.47:0/2787991081 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f680011c5f0 con 0x7f6800028270 2024-02-23T00:53:03.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.104+0000 7f67f77fe700 10 client.31530.objecter ms_dispatch 0x7f6800001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:53:03.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.104+0000 7f67f77fe700 10 client.31530.objecter _finish_command 5 = system:0 2024-02-23T00:53:03.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.104+0000 7f67f77fe700 20 client.31530.objecter put_session s=0x7f6800059770 osd=2 4 2024-02-23T00:53:03.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.104+0000 7f67f77fe700 15 client.31530.objecter _session_command_op_remove 2 5 2024-02-23T00:53:03.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.120+0000 7f6808367700 10 client.31530.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:53:03.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.120+0000 7f6808367700 15 client.31530.objecter _session_command_op_assign -1 6 2024-02-23T00:53:03.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.120+0000 7f6808367700 20 client.31530.objecter _get_session s=0x7f6800059770 osd=2 3 2024-02-23T00:53:03.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.120+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f6800059770 osd=2 3 2024-02-23T00:53:03.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.120+0000 7f6808367700 20 client.31530.objecter _get_session s=0x7f6800059770 osd=2 3 2024-02-23T00:53:03.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.120+0000 7f6808367700 15 client.31530.objecter _session_command_op_remove -1 6 2024-02-23T00:53:03.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.120+0000 7f6808367700 20 client.31530.objecter get_session s=0x7f6800059770 osd=2 3 2024-02-23T00:53:03.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.120+0000 7f6808367700 15 client.31530.objecter _session_command_op_assign 2 6 2024-02-23T00:53:03.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.120+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f6800059770 osd=2 4 2024-02-23T00:53:03.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.120+0000 7f6808367700 10 client.31530.objecter _send_command 6 2024-02-23T00:53:03.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.120+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f68000277d0 con 0x7f6800028270 2024-02-23T00:53:03.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.121+0000 7f67f77fe700 1 -- 172.21.15.47:0/2787991081 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f68000277d0 con 0x7f6800028270 2024-02-23T00:53:03.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.121+0000 7f67f77fe700 10 client.31530.objecter ms_dispatch 0x7f6800001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:53:03.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.121+0000 7f67f77fe700 10 client.31530.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:53:03.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.121+0000 7f67f77fe700 20 client.31530.objecter put_session s=0x7f6800059770 osd=2 4 2024-02-23T00:53:03.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.121+0000 7f67f77fe700 15 client.31530.objecter _session_command_op_remove 2 6 2024-02-23T00:53:03.124 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:53:03.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.122+0000 7f6808367700 10 client.31530.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:03.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.122+0000 7f6808367700 15 client.31530.objecter _session_command_op_assign -1 7 2024-02-23T00:53:03.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.122+0000 7f6808367700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f680011f860 0x7f6800121c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:03.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.122+0000 7f6808367700 20 client.31530.objecter _get_session s=0x7f6800027b80 osd=3 3 2024-02-23T00:53:03.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.122+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f6800027b80 osd=3 3 2024-02-23T00:53:03.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.122+0000 7f6808367700 20 client.31530.objecter _get_session s=0x7f6800027b80 osd=3 3 2024-02-23T00:53:03.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.122+0000 7f6808367700 15 client.31530.objecter _session_command_op_remove -1 7 2024-02-23T00:53:03.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.122+0000 7f6808367700 20 client.31530.objecter get_session s=0x7f6800027b80 osd=3 3 2024-02-23T00:53:03.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.122+0000 7f6808367700 15 client.31530.objecter _session_command_op_assign 3 7 2024-02-23T00:53:03.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.122+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f6800027b80 osd=3 4 2024-02-23T00:53:03.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.122+0000 7f6808367700 10 client.31530.objecter _send_command 7 2024-02-23T00:53:03.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.122+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f680011d4e0 con 0x7f680011f860 2024-02-23T00:53:03.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.122+0000 7f6806904700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f680011f860 0x7f6800121c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:03.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.122+0000 7f67f77fe700 10 client.31530.objecter ms_handle_connect 0x7f680011f860 2024-02-23T00:53:03.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.122+0000 7f6806904700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f680011f860 0x7f6800121c50 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:03.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.123+0000 7f67f77fe700 1 -- 172.21.15.47:0/2787991081 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f680011d4e0 con 0x7f680011f860 2024-02-23T00:53:03.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.123+0000 7f67f77fe700 10 client.31530.objecter ms_dispatch 0x7f6800001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:53:03.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.123+0000 7f67f77fe700 10 client.31530.objecter _finish_command 7 = system:0 2024-02-23T00:53:03.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.123+0000 7f67f77fe700 20 client.31530.objecter put_session s=0x7f6800027b80 osd=3 4 2024-02-23T00:53:03.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.123+0000 7f67f77fe700 15 client.31530.objecter _session_command_op_remove 3 7 2024-02-23T00:53:03.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.142+0000 7f6808367700 10 client.31530.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:53:03.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.142+0000 7f6808367700 15 client.31530.objecter _session_command_op_assign -1 8 2024-02-23T00:53:03.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.142+0000 7f6808367700 20 client.31530.objecter _get_session s=0x7f6800027b80 osd=3 3 2024-02-23T00:53:03.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.142+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f6800027b80 osd=3 3 2024-02-23T00:53:03.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.142+0000 7f6808367700 20 client.31530.objecter _get_session s=0x7f6800027b80 osd=3 3 2024-02-23T00:53:03.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.142+0000 7f6808367700 15 client.31530.objecter _session_command_op_remove -1 8 2024-02-23T00:53:03.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.142+0000 7f6808367700 20 client.31530.objecter get_session s=0x7f6800027b80 osd=3 3 2024-02-23T00:53:03.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.142+0000 7f6808367700 15 client.31530.objecter _session_command_op_assign 3 8 2024-02-23T00:53:03.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.142+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f6800027b80 osd=3 4 2024-02-23T00:53:03.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.142+0000 7f6808367700 10 client.31530.objecter _send_command 8 2024-02-23T00:53:03.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.142+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f680011c9a0 con 0x7f680011f860 2024-02-23T00:53:03.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.144+0000 7f67f77fe700 1 -- 172.21.15.47:0/2787991081 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f680011c9a0 con 0x7f680011f860 2024-02-23T00:53:03.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.144+0000 7f67f77fe700 10 client.31530.objecter ms_dispatch 0x7f6800001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:53:03.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.144+0000 7f67f77fe700 10 client.31530.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:53:03.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.144+0000 7f67f77fe700 20 client.31530.objecter put_session s=0x7f6800027b80 osd=3 4 2024-02-23T00:53:03.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.144+0000 7f67f77fe700 15 client.31530.objecter _session_command_op_remove 3 8 2024-02-23T00:53:03.147 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:53:03.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.144+0000 7f6808367700 10 client.31530.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:03.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.144+0000 7f6808367700 15 client.31530.objecter _session_command_op_assign -1 9 2024-02-23T00:53:03.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.144+0000 7f6808367700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f680011cb10 0x7f68001252d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:03.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.144+0000 7f6808367700 20 client.31530.objecter _get_session s=0x7f680011c9a0 osd=4 3 2024-02-23T00:53:03.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.144+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f680011c9a0 osd=4 3 2024-02-23T00:53:03.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.144+0000 7f6808367700 20 client.31530.objecter _get_session s=0x7f680011c9a0 osd=4 3 2024-02-23T00:53:03.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.144+0000 7f6808367700 15 client.31530.objecter _session_command_op_remove -1 9 2024-02-23T00:53:03.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.144+0000 7f6808367700 20 client.31530.objecter get_session s=0x7f680011c9a0 osd=4 3 2024-02-23T00:53:03.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.144+0000 7f6808367700 15 client.31530.objecter _session_command_op_assign 4 9 2024-02-23T00:53:03.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.144+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f680011c9a0 osd=4 4 2024-02-23T00:53:03.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.144+0000 7f6808367700 10 client.31530.objecter _send_command 9 2024-02-23T00:53:03.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.144+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f680011be70 con 0x7f680011cb10 2024-02-23T00:53:03.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.144+0000 7f6806103700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f680011cb10 0x7f68001252d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:03.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.145+0000 7f67f77fe700 10 client.31530.objecter ms_handle_connect 0x7f680011cb10 2024-02-23T00:53:03.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.145+0000 7f6806103700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f680011cb10 0x7f68001252d0 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:03.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.145+0000 7f67f77fe700 1 -- 172.21.15.47:0/2787991081 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f680011be70 con 0x7f680011cb10 2024-02-23T00:53:03.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.145+0000 7f67f77fe700 10 client.31530.objecter ms_dispatch 0x7f6800001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:53:03.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.145+0000 7f67f77fe700 10 client.31530.objecter _finish_command 9 = system:0 2024-02-23T00:53:03.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.145+0000 7f67f77fe700 20 client.31530.objecter put_session s=0x7f680011c9a0 osd=4 4 2024-02-23T00:53:03.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.145+0000 7f67f77fe700 15 client.31530.objecter _session_command_op_remove 4 9 2024-02-23T00:53:03.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.162+0000 7f6808367700 10 client.31530.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:53:03.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.162+0000 7f6808367700 15 client.31530.objecter _session_command_op_assign -1 10 2024-02-23T00:53:03.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.162+0000 7f6808367700 20 client.31530.objecter _get_session s=0x7f680011c9a0 osd=4 3 2024-02-23T00:53:03.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.162+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f680011c9a0 osd=4 3 2024-02-23T00:53:03.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.162+0000 7f6808367700 20 client.31530.objecter _get_session s=0x7f680011c9a0 osd=4 3 2024-02-23T00:53:03.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.162+0000 7f6808367700 15 client.31530.objecter _session_command_op_remove -1 10 2024-02-23T00:53:03.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.162+0000 7f6808367700 20 client.31530.objecter get_session s=0x7f680011c9a0 osd=4 3 2024-02-23T00:53:03.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.162+0000 7f6808367700 15 client.31530.objecter _session_command_op_assign 4 10 2024-02-23T00:53:03.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.162+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f680011c9a0 osd=4 4 2024-02-23T00:53:03.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.162+0000 7f6808367700 10 client.31530.objecter _send_command 10 2024-02-23T00:53:03.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.162+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f68000008d0 con 0x7f680011cb10 2024-02-23T00:53:03.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.164+0000 7f67f77fe700 1 -- 172.21.15.47:0/2787991081 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f68000008d0 con 0x7f680011cb10 2024-02-23T00:53:03.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.164+0000 7f67f77fe700 10 client.31530.objecter ms_dispatch 0x7f6800001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:53:03.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.164+0000 7f67f77fe700 10 client.31530.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:53:03.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.164+0000 7f67f77fe700 20 client.31530.objecter put_session s=0x7f680011c9a0 osd=4 4 2024-02-23T00:53:03.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.164+0000 7f67f77fe700 15 client.31530.objecter _session_command_op_remove 4 10 2024-02-23T00:53:03.167 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:53:03.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.164+0000 7f6808367700 10 client.31530.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:03.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.164+0000 7f6808367700 15 client.31530.objecter _session_command_op_assign -1 11 2024-02-23T00:53:03.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.164+0000 7f6808367700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f6800125cd0 0x7f68001280c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:03.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.164+0000 7f6808367700 20 client.31530.objecter _get_session s=0x7f680011c760 osd=5 3 2024-02-23T00:53:03.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.164+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f680011c760 osd=5 3 2024-02-23T00:53:03.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.164+0000 7f6808367700 20 client.31530.objecter _get_session s=0x7f680011c760 osd=5 3 2024-02-23T00:53:03.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.164+0000 7f6808367700 15 client.31530.objecter _session_command_op_remove -1 11 2024-02-23T00:53:03.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.164+0000 7f6808367700 20 client.31530.objecter get_session s=0x7f680011c760 osd=5 3 2024-02-23T00:53:03.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.164+0000 7f6808367700 15 client.31530.objecter _session_command_op_assign 5 11 2024-02-23T00:53:03.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.164+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f680011c760 osd=5 4 2024-02-23T00:53:03.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.164+0000 7f6808367700 10 client.31530.objecter _send_command 11 2024-02-23T00:53:03.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.164+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f680012aeb0 con 0x7f6800125cd0 2024-02-23T00:53:03.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.164+0000 7f6805902700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f6800125cd0 0x7f68001280c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:03.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.165+0000 7f67f77fe700 10 client.31530.objecter ms_handle_connect 0x7f6800125cd0 2024-02-23T00:53:03.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.165+0000 7f6805902700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f6800125cd0 0x7f68001280c0 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:03.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.165+0000 7f67f77fe700 1 -- 172.21.15.47:0/2787991081 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f680012aeb0 con 0x7f6800125cd0 2024-02-23T00:53:03.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.165+0000 7f67f77fe700 10 client.31530.objecter ms_dispatch 0x7f6800001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:53:03.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.165+0000 7f67f77fe700 10 client.31530.objecter _finish_command 11 = system:0 2024-02-23T00:53:03.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.165+0000 7f67f77fe700 20 client.31530.objecter put_session s=0x7f680011c760 osd=5 4 2024-02-23T00:53:03.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.165+0000 7f67f77fe700 15 client.31530.objecter _session_command_op_remove 5 11 2024-02-23T00:53:03.178 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:53:03.181 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:53:03.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.181+0000 7f6808367700 10 client.31530.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:53:03.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.181+0000 7f6808367700 15 client.31530.objecter _session_command_op_assign -1 12 2024-02-23T00:53:03.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.181+0000 7f6808367700 20 client.31530.objecter _get_session s=0x7f680011c760 osd=5 3 2024-02-23T00:53:03.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.181+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f680011c760 osd=5 3 2024-02-23T00:53:03.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.181+0000 7f6808367700 20 client.31530.objecter _get_session s=0x7f680011c760 osd=5 3 2024-02-23T00:53:03.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.181+0000 7f6808367700 15 client.31530.objecter _session_command_op_remove -1 12 2024-02-23T00:53:03.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.181+0000 7f6808367700 20 client.31530.objecter get_session s=0x7f680011c760 osd=5 3 2024-02-23T00:53:03.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.181+0000 7f6808367700 15 client.31530.objecter _session_command_op_assign 5 12 2024-02-23T00:53:03.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.181+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f680011c760 osd=5 4 2024-02-23T00:53:03.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.181+0000 7f6808367700 10 client.31530.objecter _send_command 12 2024-02-23T00:53:03.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.181+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f68001286a0 con 0x7f6800125cd0 2024-02-23T00:53:03.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.183+0000 7f67f77fe700 1 -- 172.21.15.47:0/2787991081 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f68001286a0 con 0x7f6800125cd0 2024-02-23T00:53:03.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.183+0000 7f67f77fe700 10 client.31530.objecter ms_dispatch 0x7f6800001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:53:03.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.183+0000 7f67f77fe700 10 client.31530.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:53:03.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.183+0000 7f67f77fe700 20 client.31530.objecter put_session s=0x7f680011c760 osd=5 4 2024-02-23T00:53:03.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.183+0000 7f67f77fe700 15 client.31530.objecter _session_command_op_remove 5 12 2024-02-23T00:53:03.186 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:53:03.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.183+0000 7f6808367700 10 client.31530.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:03.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.183+0000 7f6808367700 15 client.31530.objecter _session_command_op_assign -1 13 2024-02-23T00:53:03.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.183+0000 7f6808367700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f6800128e10 0x7f6800129250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:03.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.183+0000 7f6808367700 20 client.31530.objecter _get_session s=0x7f68001286a0 osd=6 3 2024-02-23T00:53:03.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.183+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f68001286a0 osd=6 3 2024-02-23T00:53:03.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.183+0000 7f6808367700 20 client.31530.objecter _get_session s=0x7f68001286a0 osd=6 3 2024-02-23T00:53:03.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.183+0000 7f6808367700 15 client.31530.objecter _session_command_op_remove -1 13 2024-02-23T00:53:03.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.183+0000 7f6808367700 20 client.31530.objecter get_session s=0x7f68001286a0 osd=6 3 2024-02-23T00:53:03.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.183+0000 7f6808367700 15 client.31530.objecter _session_command_op_assign 6 13 2024-02-23T00:53:03.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.183+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f68001286a0 osd=6 4 2024-02-23T00:53:03.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.183+0000 7f6808367700 10 client.31530.objecter _send_command 13 2024-02-23T00:53:03.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.183+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f680012aab0 con 0x7f6800128e10 2024-02-23T00:53:03.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.183+0000 7f6806904700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f6800128e10 0x7f6800129250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:03.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.184+0000 7f67f77fe700 10 client.31530.objecter ms_handle_connect 0x7f6800128e10 2024-02-23T00:53:03.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.184+0000 7f6806904700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f6800128e10 0x7f6800129250 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:03.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.185+0000 7f67f77fe700 1 -- 172.21.15.47:0/2787991081 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f680012aab0 con 0x7f6800128e10 2024-02-23T00:53:03.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.185+0000 7f67f77fe700 10 client.31530.objecter ms_dispatch 0x7f6800001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:53:03.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.185+0000 7f67f77fe700 10 client.31530.objecter _finish_command 13 = system:0 2024-02-23T00:53:03.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.185+0000 7f67f77fe700 20 client.31530.objecter put_session s=0x7f68001286a0 osd=6 4 2024-02-23T00:53:03.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.185+0000 7f67f77fe700 15 client.31530.objecter _session_command_op_remove 6 13 2024-02-23T00:53:03.194 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:53:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:53:03.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:01.387331+0000 osd.7 (osd.7) 313 : cluster [DBG] 18.6 scrub starts 2024-02-23T00:53:03.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:01.388168+0000 osd.7 (osd.7) 314 : cluster [DBG] 18.6 scrub ok 2024-02-23T00:53:03.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:01.801483+0000 osd.0 (osd.0) 273 : cluster [DBG] 18.8 scrub starts 2024-02-23T00:53:03.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:01.802400+0000 osd.0 (osd.0) 274 : cluster [DBG] 18.8 scrub ok 2024-02-23T00:53:03.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:02.292007+0000 mon.a (mon.0) 3245 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-110", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:53:03.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:02.292096+0000 mon.a (mon.0) 3246 : cluster [DBG] osdmap e645: 8 total, 8 up, 7 in 2024-02-23T00:53:03.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:02.293676+0000 mon.c (mon.2) 368 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-110"}]: dispatch 2024-02-23T00:53:03.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:02.294049+0000 mon.a (mon.0) 3247 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-110"}]: dispatch 2024-02-23T00:53:03.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.201+0000 7f6808367700 10 client.31530.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:53:03.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.201+0000 7f6808367700 15 client.31530.objecter _session_command_op_assign -1 14 2024-02-23T00:53:03.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.201+0000 7f6808367700 20 client.31530.objecter _get_session s=0x7f68001286a0 osd=6 3 2024-02-23T00:53:03.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.201+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f68001286a0 osd=6 3 2024-02-23T00:53:03.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.201+0000 7f6808367700 20 client.31530.objecter _get_session s=0x7f68001286a0 osd=6 3 2024-02-23T00:53:03.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.201+0000 7f6808367700 15 client.31530.objecter _session_command_op_remove -1 14 2024-02-23T00:53:03.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.201+0000 7f6808367700 20 client.31530.objecter get_session s=0x7f68001286a0 osd=6 3 2024-02-23T00:53:03.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.201+0000 7f6808367700 15 client.31530.objecter _session_command_op_assign 6 14 2024-02-23T00:53:03.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.201+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f68001286a0 osd=6 4 2024-02-23T00:53:03.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.201+0000 7f6808367700 10 client.31530.objecter _send_command 14 2024-02-23T00:53:03.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.201+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f680012aab0 con 0x7f6800128e10 2024-02-23T00:53:03.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.203+0000 7f67f77fe700 1 -- 172.21.15.47:0/2787991081 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f680012aab0 con 0x7f6800128e10 2024-02-23T00:53:03.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.203+0000 7f67f77fe700 10 client.31530.objecter ms_dispatch 0x7f6800001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:53:03.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.203+0000 7f67f77fe700 10 client.31530.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:53:03.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.203+0000 7f67f77fe700 20 client.31530.objecter put_session s=0x7f68001286a0 osd=6 4 2024-02-23T00:53:03.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.203+0000 7f67f77fe700 15 client.31530.objecter _session_command_op_remove 6 14 2024-02-23T00:53:03.206 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:53:03.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.203+0000 7f6808367700 10 client.31530.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:03.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.203+0000 7f6808367700 15 client.31530.objecter _session_command_op_assign -1 15 2024-02-23T00:53:03.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.203+0000 7f6808367700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f6800129d90 0x7f680012a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:03.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.203+0000 7f6808367700 20 client.31530.objecter _get_session s=0x7f680012aab0 osd=7 3 2024-02-23T00:53:03.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.203+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f680012aab0 osd=7 3 2024-02-23T00:53:03.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.203+0000 7f6808367700 20 client.31530.objecter _get_session s=0x7f680012aab0 osd=7 3 2024-02-23T00:53:03.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.203+0000 7f6808367700 15 client.31530.objecter _session_command_op_remove -1 15 2024-02-23T00:53:03.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.203+0000 7f6808367700 20 client.31530.objecter get_session s=0x7f680012aab0 osd=7 3 2024-02-23T00:53:03.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.203+0000 7f6808367700 15 client.31530.objecter _session_command_op_assign 7 15 2024-02-23T00:53:03.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.203+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f680012aab0 osd=7 4 2024-02-23T00:53:03.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.203+0000 7f6808367700 10 client.31530.objecter _send_command 15 2024-02-23T00:53:03.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.203+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f680012a7f0 con 0x7f6800129d90 2024-02-23T00:53:03.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.203+0000 7f6806103700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f6800129d90 0x7f680012a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:03.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.203+0000 7f67f77fe700 10 client.31530.objecter ms_handle_connect 0x7f6800129d90 2024-02-23T00:53:03.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.203+0000 7f6806103700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f6800129d90 0x7f680012a210 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:03.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.204+0000 7f67f77fe700 1 -- 172.21.15.47:0/2787991081 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f680012a7f0 con 0x7f6800129d90 2024-02-23T00:53:03.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.204+0000 7f67f77fe700 10 client.31530.objecter ms_dispatch 0x7f6800001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:53:03.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.204+0000 7f67f77fe700 10 client.31530.objecter _finish_command 15 = system:0 2024-02-23T00:53:03.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.204+0000 7f67f77fe700 20 client.31530.objecter put_session s=0x7f680012aab0 osd=7 4 2024-02-23T00:53:03.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.204+0000 7f67f77fe700 15 client.31530.objecter _session_command_op_remove 7 15 2024-02-23T00:53:03.215 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.219+0000 7f6808367700 10 client.31530.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:53:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.219+0000 7f6808367700 15 client.31530.objecter _session_command_op_assign -1 16 2024-02-23T00:53:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.219+0000 7f6808367700 20 client.31530.objecter _get_session s=0x7f680012aab0 osd=7 3 2024-02-23T00:53:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.219+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f680012aab0 osd=7 3 2024-02-23T00:53:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.219+0000 7f6808367700 20 client.31530.objecter _get_session s=0x7f680012aab0 osd=7 3 2024-02-23T00:53:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.219+0000 7f6808367700 15 client.31530.objecter _session_command_op_remove -1 16 2024-02-23T00:53:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.219+0000 7f6808367700 20 client.31530.objecter get_session s=0x7f680012aab0 osd=7 3 2024-02-23T00:53:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.219+0000 7f6808367700 15 client.31530.objecter _session_command_op_assign 7 16 2024-02-23T00:53:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.219+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f680012aab0 osd=7 4 2024-02-23T00:53:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.219+0000 7f6808367700 10 client.31530.objecter _send_command 16 2024-02-23T00:53:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.219+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f680012a7b0 con 0x7f6800129d90 2024-02-23T00:53:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.221+0000 7f67f77fe700 1 -- 172.21.15.47:0/2787991081 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f680012a7b0 con 0x7f6800129d90 2024-02-23T00:53:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.221+0000 7f67f77fe700 10 client.31530.objecter ms_dispatch 0x7f6800001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:53:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.221+0000 7f67f77fe700 10 client.31530.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:53:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.221+0000 7f67f77fe700 20 client.31530.objecter put_session s=0x7f680012aab0 osd=7 4 2024-02-23T00:53:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.221+0000 7f67f77fe700 15 client.31530.objecter _session_command_op_remove 7 16 2024-02-23T00:53:03.224 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:53:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.221+0000 7f6808367700 10 librados: watch_flush enter 2024-02-23T00:53:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.221+0000 7f6808367700 10 librados: watch_flush exit 2024-02-23T00:53:03.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.221+0000 7f6808367700 10 client.31530.objecter close_session for osd.0 2024-02-23T00:53:03.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.221+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f68000fc1a0 msgr2=0x7f68000fc580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:03.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.221+0000 7f6808367700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f68000fc1a0 0x7f68000fc580 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:03.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.221+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f680005ee80 osd=0 1 2024-02-23T00:53:03.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.221+0000 7f6808367700 10 client.31530.objecter close_session for osd.1 2024-02-23T00:53:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.221+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f68000598f0 msgr2=0x7f6800059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.221+0000 7f6808367700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f68000598f0 0x7f6800059cd0 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.221+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f6800028710 osd=1 1 2024-02-23T00:53:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.221+0000 7f6808367700 10 client.31530.objecter close_session for osd.2 2024-02-23T00:53:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.221+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f6800028270 msgr2=0x7f680011c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.221+0000 7f6808367700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f6800028270 0x7f680011c100 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.221+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f6800059770 osd=2 1 2024-02-23T00:53:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.221+0000 7f6808367700 10 client.31530.objecter close_session for osd.3 2024-02-23T00:53:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.221+0000 7f6806904700 1 -- 172.21.15.47:0/2787991081 reap_dead start 2024-02-23T00:53:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.221+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f680011f860 msgr2=0x7f6800121c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.221+0000 7f6808367700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f680011f860 0x7f6800121c50 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.221+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f6800027b80 osd=3 1 2024-02-23T00:53:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.221+0000 7f6808367700 10 client.31530.objecter close_session for osd.4 2024-02-23T00:53:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.221+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f680011cb10 msgr2=0x7f68001252d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.221+0000 7f6808367700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f680011cb10 0x7f68001252d0 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.221+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f680011c9a0 osd=4 1 2024-02-23T00:53:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.221+0000 7f6808367700 10 client.31530.objecter close_session for osd.5 2024-02-23T00:53:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.221+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f6800125cd0 msgr2=0x7f68001280c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.222+0000 7f6808367700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f6800125cd0 0x7f68001280c0 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.222+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f680011c760 osd=5 1 2024-02-23T00:53:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.222+0000 7f6808367700 10 client.31530.objecter close_session for osd.6 2024-02-23T00:53:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.222+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f6800128e10 msgr2=0x7f6800129250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.222+0000 7f6808367700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f6800128e10 0x7f6800129250 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.222+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f68001286a0 osd=6 1 2024-02-23T00:53:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.222+0000 7f6808367700 10 client.31530.objecter close_session for osd.7 2024-02-23T00:53:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.222+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f6800129d90 msgr2=0x7f680012a210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.222+0000 7f6808367700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f6800129d90 0x7f680012a210 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.222+0000 7f6808367700 20 client.31530.objecter put_session s=0x7f680012aab0 osd=7 1 2024-02-23T00:53:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.222+0000 7f6808367700 20 client.31530.objecter shutdown clearing up homeless session... 2024-02-23T00:53:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.222+0000 7f6808367700 10 client.31530.objecter successfully canceled tick 2024-02-23T00:53:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.222+0000 7f6806904700 1 -- 172.21.15.47:0/2787991081 reap_dead start 2024-02-23T00:53:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.222+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f67ec062cc0 msgr2=0x7f67ec065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.222+0000 7f6808367700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f67ec062cc0 0x7f67ec065180 secure :-1 s=READY pgs=3151 cs=0 l=1 rev1=1 rx=0x7f68000f8400 tx=0x7f67f000b040).stop 2024-02-23T00:53:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.222+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68000f8cb0 msgr2=0x7f68000f6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.222+0000 7f6808367700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68000f8cb0 0x7f68000f6c60 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 rx=0x7f67fc0048f0 tx=0x7f67fc009cb0).stop 2024-02-23T00:53:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.222+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 shutdown_connections 2024-02-23T00:53:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.222+0000 7f6808367700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68000f8cb0 0x7f68000f6c60 unknown :-1 s=CLOSED pgs=2604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.222+0000 7f6808367700 1 --2- 172.21.15.47:0/2787991081 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f67ec062cc0 0x7f67ec065180 unknown :-1 s=CLOSED pgs=3151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.222+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 >> 172.21.15.47:0/2787991081 conn(0x7f68000f2820 msgr2=0x7f68000f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.222+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 shutdown_connections 2024-02-23T00:53:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.222+0000 7f6808367700 1 -- 172.21.15.47:0/2787991081 wait complete. 2024-02-23T00:53:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:03.222+0000 7f6808367700 1 librados: shutdown 2024-02-23T00:53:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:53:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:53:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:53:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:53:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:53:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:53:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:53:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:53:03.238 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:53:03.241 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:03.241 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:53:03.342 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:53:03.372 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:03.445 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set noscrub 2024-02-23T00:53:03.446 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:53:03.448 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:03.449 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:53:03.550 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:53:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:03.653 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:53:03.655 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:03.656 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:53:03.757 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:53:03.785 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:04.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:02.351866+0000 osd.7 (osd.7) 315 : cluster [DBG] 18.1f scrub starts 2024-02-23T00:53:04.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:02.354541+0000 osd.7 (osd.7) 316 : cluster [DBG] 18.1f scrub ok 2024-02-23T00:53:04.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:02.412678+0000 osd.5 (osd.5) 284 : cluster [DBG] 4.4 scrub starts 2024-02-23T00:53:04.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:02.417512+0000 osd.5 (osd.5) 285 : cluster [DBG] 4.4 scrub ok 2024-02-23T00:53:04.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:02.563299+0000 osd.1 (osd.1) 269 : cluster [DBG] 5.19 scrub starts 2024-02-23T00:53:04.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:02.566640+0000 osd.1 (osd.1) 270 : cluster [DBG] 5.19 scrub ok 2024-02-23T00:53:04.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:02.818915+0000 osd.0 (osd.0) 275 : cluster [DBG] 6.9 scrub starts 2024-02-23T00:53:04.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:02.821528+0000 osd.0 (osd.0) 276 : cluster [DBG] 6.9 scrub ok 2024-02-23T00:53:04.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:02.838824+0000 mgr.y (mgr.14517) 570 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:53:04.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:02.961999+0000 osd.3 (osd.3) 310 : cluster [DBG] 4.0 deep-scrub starts 2024-02-23T00:53:04.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:02.966135+0000 osd.3 (osd.3) 311 : cluster [DBG] 4.0 deep-scrub ok 2024-02-23T00:53:04.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:03.068049+0000 mon.a (mon.0) 3248 : audit [DBG] from='client.? 172.21.15.47:0/2787991081' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:53:04.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:03.293686+0000 mon.a (mon.0) 3249 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-110"}]': finished 2024-02-23T00:53:04.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:03.293765+0000 mon.a (mon.0) 3250 : cluster [DBG] osdmap e646: 8 total, 8 up, 7 in 2024-02-23T00:53:04.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:03.294977+0000 mon.c (mon.2) 369 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-110", "mode": "writeback"}]: dispatch 2024-02-23T00:53:04.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:03.295300+0000 mon.a (mon.0) 3251 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-110", "mode": "writeback"}]: dispatch 2024-02-23T00:53:04.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:03.366596+0000 mgr.y (mgr.14517) 571 : cluster [DBG] pgmap v977: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:53:04.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:02.351866+0000 osd.7 (osd.7) 315 : cluster [DBG] 18.1f scrub starts 2024-02-23T00:53:04.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:02.354541+0000 osd.7 (osd.7) 316 : cluster [DBG] 18.1f scrub ok 2024-02-23T00:53:04.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:02.412678+0000 osd.5 (osd.5) 284 : cluster [DBG] 4.4 scrub starts 2024-02-23T00:53:04.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:02.417512+0000 osd.5 (osd.5) 285 : cluster [DBG] 4.4 scrub ok 2024-02-23T00:53:04.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:02.563299+0000 osd.1 (osd.1) 269 : cluster [DBG] 5.19 scrub starts 2024-02-23T00:53:04.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:02.566640+0000 osd.1 (osd.1) 270 : cluster [DBG] 5.19 scrub ok 2024-02-23T00:53:04.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:02.818915+0000 osd.0 (osd.0) 275 : cluster [DBG] 6.9 scrub starts 2024-02-23T00:53:04.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:02.821528+0000 osd.0 (osd.0) 276 : cluster [DBG] 6.9 scrub ok 2024-02-23T00:53:04.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:02.838824+0000 mgr.y (mgr.14517) 570 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:53:04.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:02.961999+0000 osd.3 (osd.3) 310 : cluster [DBG] 4.0 deep-scrub starts 2024-02-23T00:53:04.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:02.966135+0000 osd.3 (osd.3) 311 : cluster [DBG] 4.0 deep-scrub ok 2024-02-23T00:53:04.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:03.068049+0000 mon.a (mon.0) 3248 : audit [DBG] from='client.? 172.21.15.47:0/2787991081' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:53:04.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:03.293686+0000 mon.a (mon.0) 3249 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-110"}]': finished 2024-02-23T00:53:04.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:03.293765+0000 mon.a (mon.0) 3250 : cluster [DBG] osdmap e646: 8 total, 8 up, 7 in 2024-02-23T00:53:04.061 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:03.294977+0000 mon.c (mon.2) 369 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-110", "mode": "writeback"}]: dispatch 2024-02-23T00:53:04.061 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:03.295300+0000 mon.a (mon.0) 3251 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-110", "mode": "writeback"}]: dispatch 2024-02-23T00:53:04.061 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:03.366596+0000 mgr.y (mgr.14517) 571 : cluster [DBG] pgmap v977: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:53:04.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:02.351866+0000 osd.7 (osd.7) 315 : cluster [DBG] 18.1f scrub starts 2024-02-23T00:53:04.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:02.354541+0000 osd.7 (osd.7) 316 : cluster [DBG] 18.1f scrub ok 2024-02-23T00:53:04.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:02.412678+0000 osd.5 (osd.5) 284 : cluster [DBG] 4.4 scrub starts 2024-02-23T00:53:04.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:02.417512+0000 osd.5 (osd.5) 285 : cluster [DBG] 4.4 scrub ok 2024-02-23T00:53:04.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:02.563299+0000 osd.1 (osd.1) 269 : cluster [DBG] 5.19 scrub starts 2024-02-23T00:53:04.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:02.566640+0000 osd.1 (osd.1) 270 : cluster [DBG] 5.19 scrub ok 2024-02-23T00:53:04.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:02.818915+0000 osd.0 (osd.0) 275 : cluster [DBG] 6.9 scrub starts 2024-02-23T00:53:04.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:02.821528+0000 osd.0 (osd.0) 276 : cluster [DBG] 6.9 scrub ok 2024-02-23T00:53:04.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:02.838824+0000 mgr.y (mgr.14517) 570 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:53:04.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:02.961999+0000 osd.3 (osd.3) 310 : cluster [DBG] 4.0 deep-scrub starts 2024-02-23T00:53:04.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:02.966135+0000 osd.3 (osd.3) 311 : cluster [DBG] 4.0 deep-scrub ok 2024-02-23T00:53:04.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:03.068049+0000 mon.a (mon.0) 3248 : audit [DBG] from='client.? 172.21.15.47:0/2787991081' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:53:04.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:03.293686+0000 mon.a (mon.0) 3249 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-110"}]': finished 2024-02-23T00:53:04.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:03.293765+0000 mon.a (mon.0) 3250 : cluster [DBG] osdmap e646: 8 total, 8 up, 7 in 2024-02-23T00:53:04.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:03.294977+0000 mon.c (mon.2) 369 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-110", "mode": "writeback"}]: dispatch 2024-02-23T00:53:04.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:03.295300+0000 mon.a (mon.0) 3251 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-110", "mode": "writeback"}]: dispatch 2024-02-23T00:53:04.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:03.366596+0000 mgr.y (mgr.14517) 571 : cluster [DBG] pgmap v977: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:53:04.735 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:53:04.740 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:04.741 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:53:04.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:03.381688+0000 osd.6 (osd.6) 253 : cluster [DBG] 2.14 scrub starts 2024-02-23T00:53:04.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:03.382638+0000 osd.6 (osd.6) 254 : cluster [DBG] 2.14 scrub ok 2024-02-23T00:53:04.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:04.291678+0000 mon.a (mon.0) 3252 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:53:04.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:04.298413+0000 mon.a (mon.0) 3253 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-110", "mode": "writeback"}]': finished 2024-02-23T00:53:04.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:04.298469+0000 mon.a (mon.0) 3254 : cluster [DBG] osdmap e647: 8 total, 8 up, 7 in 2024-02-23T00:53:04.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:04.364074+0000 mon.c (mon.2) 370 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:04.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:04.364353+0000 mon.a (mon.0) 3255 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:04.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:03.381688+0000 osd.6 (osd.6) 253 : cluster [DBG] 2.14 scrub starts 2024-02-23T00:53:04.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:03.382638+0000 osd.6 (osd.6) 254 : cluster [DBG] 2.14 scrub ok 2024-02-23T00:53:04.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:04.291678+0000 mon.a (mon.0) 3252 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:53:04.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:04.298413+0000 mon.a (mon.0) 3253 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-110", "mode": "writeback"}]': finished 2024-02-23T00:53:04.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:04.298469+0000 mon.a (mon.0) 3254 : cluster [DBG] osdmap e647: 8 total, 8 up, 7 in 2024-02-23T00:53:04.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:04.364074+0000 mon.c (mon.2) 370 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:04.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:04.364353+0000 mon.a (mon.0) 3255 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:04.842 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:53:04.875 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:04.897 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:53:04.899 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:04.900 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:53:05.001 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:53:05.033 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:05.071 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:53:05.075 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:05.075 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:53:05.077 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:53:05.131 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:53:05.132 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:53:05.176 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:53:05.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:03.381688+0000 osd.6 (osd.6) 253 : cluster [DBG] 2.14 scrub starts 2024-02-23T00:53:05.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:03.382638+0000 osd.6 (osd.6) 254 : cluster [DBG] 2.14 scrub ok 2024-02-23T00:53:05.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:04.291678+0000 mon.a (mon.0) 3252 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:53:05.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:04.298413+0000 mon.a (mon.0) 3253 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-110", "mode": "writeback"}]': finished 2024-02-23T00:53:05.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:04.298469+0000 mon.a (mon.0) 3254 : cluster [DBG] osdmap e647: 8 total, 8 up, 7 in 2024-02-23T00:53:05.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:04.364074+0000 mon.c (mon.2) 370 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:05.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:04.364353+0000 mon.a (mon.0) 3255 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:05.206 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:05.956 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:53:05.959 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:05.960 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:53:06.061 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:53:06.093 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:06.114 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:53:06.116 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:06.117 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:53:06.218 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:53:06.248 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:06.267 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:53:06.270 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:06.270 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:53:06.371 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:53:06.397 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:04.956594+0000 osd.3 (osd.3) 312 : cluster [DBG] 18.1 scrub starts 2024-02-23T00:53:06.397 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:04.958277+0000 osd.3 (osd.3) 313 : cluster [DBG] 18.1 scrub ok 2024-02-23T00:53:06.397 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:05.272087+0000 mon.a (mon.0) 3256 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:53:06.397 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:05.272165+0000 mon.a (mon.0) 3257 : cluster [DBG] osdmap e648: 8 total, 8 up, 7 in 2024-02-23T00:53:06.397 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:05.274114+0000 mon.c (mon.2) 371 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-110"}]: dispatch 2024-02-23T00:53:06.397 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:05.274555+0000 mon.a (mon.0) 3258 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-110"}]: dispatch 2024-02-23T00:53:06.397 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:05.305267+0000 osd.7 (osd.7) 317 : cluster [DBG] 2.1c scrub starts 2024-02-23T00:53:06.398 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:05.309729+0000 osd.7 (osd.7) 318 : cluster [DBG] 2.1c scrub ok 2024-02-23T00:53:06.398 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:05.366974+0000 mgr.y (mgr.14517) 572 : cluster [DBG] pgmap v980: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:53:06.399 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:06.418 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:53:06.420 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:06.421 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:53:06.516 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-23T00:53:06.517 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-23T00:53:06.517 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [0, 1, 3, 4, 5, 6, 7] 2024-02-23T00:53:06.517 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd out 0 2024-02-23T00:53:06.521 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:53:06.544 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:04.956594+0000 osd.3 (osd.3) 312 : cluster [DBG] 18.1 scrub starts 2024-02-23T00:53:06.545 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:04.958277+0000 osd.3 (osd.3) 313 : cluster [DBG] 18.1 scrub ok 2024-02-23T00:53:06.545 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:05.272087+0000 mon.a (mon.0) 3256 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:53:06.545 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:05.272165+0000 mon.a (mon.0) 3257 : cluster [DBG] osdmap e648: 8 total, 8 up, 7 in 2024-02-23T00:53:06.545 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:05.274114+0000 mon.c (mon.2) 371 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-110"}]: dispatch 2024-02-23T00:53:06.545 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:05.274555+0000 mon.a (mon.0) 3258 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-110"}]: dispatch 2024-02-23T00:53:06.545 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:05.305267+0000 osd.7 (osd.7) 317 : cluster [DBG] 2.1c scrub starts 2024-02-23T00:53:06.545 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:05.309729+0000 osd.7 (osd.7) 318 : cluster [DBG] 2.1c scrub ok 2024-02-23T00:53:06.545 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:05.366974+0000 mgr.y (mgr.14517) 572 : cluster [DBG] pgmap v980: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:53:06.545 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:04.956594+0000 osd.3 (osd.3) 312 : cluster [DBG] 18.1 scrub starts 2024-02-23T00:53:06.546 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:04.958277+0000 osd.3 (osd.3) 313 : cluster [DBG] 18.1 scrub ok 2024-02-23T00:53:06.546 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:05.272087+0000 mon.a (mon.0) 3256 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:53:06.546 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:05.272165+0000 mon.a (mon.0) 3257 : cluster [DBG] osdmap e648: 8 total, 8 up, 7 in 2024-02-23T00:53:06.546 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:05.274114+0000 mon.c (mon.2) 371 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-110"}]: dispatch 2024-02-23T00:53:06.546 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:05.274555+0000 mon.a (mon.0) 3258 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-110"}]: dispatch 2024-02-23T00:53:06.546 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:05.305267+0000 osd.7 (osd.7) 317 : cluster [DBG] 2.1c scrub starts 2024-02-23T00:53:06.546 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:05.309729+0000 osd.7 (osd.7) 318 : cluster [DBG] 2.1c scrub ok 2024-02-23T00:53:06.546 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:05.366974+0000 mgr.y (mgr.14517) 572 : cluster [DBG] pgmap v980: 184 pgs: 184 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:53:06.547 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:06.568 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:53:06.572 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:06.573 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:53:06.674 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:53:06.702 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:07.330 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:53:07.333 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:07.333 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:53:07.355 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.3 dump_blocked_ops 2024-02-23T00:53:07.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.354+0000 7f975cd5d700 1 -- 172.21.15.47:0/270679998 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f97580fe510 msgr2=0x7f97580f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:07.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.354+0000 7f975cd5d700 1 --2- 172.21.15.47:0/270679998 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f97580fe510 0x7f97580f7d40 secure :-1 s=READY pgs=2763 cs=0 l=1 rev1=1 rx=0x7f974c004870 tx=0x7f974c009cb0).stop 2024-02-23T00:53:07.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.354+0000 7f975cd5d700 1 -- 172.21.15.47:0/270679998 shutdown_connections 2024-02-23T00:53:07.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.354+0000 7f975cd5d700 1 --2- 172.21.15.47:0/270679998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97580f8420 0x7f9758104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:07.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.354+0000 7f975cd5d700 1 --2- 172.21.15.47:0/270679998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97580fdbb0 0x7f97580fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:07.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.354+0000 7f975cd5d700 1 --2- 172.21.15.47:0/270679998 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f97580fe510 0x7f97580f7d40 unknown :-1 s=CLOSED pgs=2763 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:07.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.354+0000 7f975cd5d700 1 -- 172.21.15.47:0/270679998 >> 172.21.15.47:0/270679998 conn(0x7f97580f3800 msgr2=0x7f97580f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:07.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.354+0000 7f975cd5d700 1 -- 172.21.15.47:0/270679998 shutdown_connections 2024-02-23T00:53:07.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.354+0000 7f975cd5d700 1 -- 172.21.15.47:0/270679998 wait complete. 2024-02-23T00:53:07.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.355+0000 7f975cd5d700 1 librados: starting msgr at 2024-02-23T00:53:07.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.355+0000 7f975cd5d700 1 librados: starting objecter 2024-02-23T00:53:07.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.355+0000 7f975cd5d700 1 Processor -- start 2024-02-23T00:53:07.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.355+0000 7f975cd5d700 1 -- start start 2024-02-23T00:53:07.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.355+0000 7f975cd5d700 1 librados: setting wanted keys 2024-02-23T00:53:07.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.355+0000 7f975cd5d700 1 librados: calling monclient init 2024-02-23T00:53:07.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.355+0000 7f975cd5d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97580f8420 0x7f975810de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:07.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.355+0000 7f975cd5d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f97580fdbb0 0x7f975810e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:07.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.355+0000 7f975cd5d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97580fe510 0x7f9758108090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:07.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.355+0000 7f975cd5d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9758108580 con 0x7f97580f8420 2024-02-23T00:53:07.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.355+0000 7f975cd5d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97581086f0 con 0x7f97580fe510 2024-02-23T00:53:07.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.355+0000 7f975cd5d700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97581088b0 con 0x7f97580fdbb0 2024-02-23T00:53:07.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.356+0000 7f975659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97580f8420 0x7f975810de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:07.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.356+0000 7f975659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97580f8420 0x7f975810de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37248/0 (socket says 172.21.15.47:37248) 2024-02-23T00:53:07.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.356+0000 7f9756d9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97580fe510 0x7f9758108090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:07.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.356+0000 7f975659c700 1 -- 172.21.15.47:0/2066494222 learned_addr learned my addr 172.21.15.47:0/2066494222 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:53:07.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.356+0000 7f9755d9b700 1 --2- 172.21.15.47:0/2066494222 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f97580fdbb0 0x7f975810e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:07.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.356+0000 7f9756d9d700 1 -- 172.21.15.47:0/2066494222 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f97580fdbb0 msgr2=0x7f975810e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:07.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.356+0000 7f9756d9d700 1 --2- 172.21.15.47:0/2066494222 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f97580fdbb0 0x7f975810e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:07.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.356+0000 7f9756d9d700 1 -- 172.21.15.47:0/2066494222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97580f8420 msgr2=0x7f975810de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:07.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.356+0000 7f9756d9d700 1 --2- 172.21.15.47:0/2066494222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97580f8420 0x7f975810de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:07.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.356+0000 7f9756d9d700 1 -- 172.21.15.47:0/2066494222 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f974c005040 con 0x7f97580fe510 2024-02-23T00:53:07.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.356+0000 7f9756d9d700 1 --2- 172.21.15.47:0/2066494222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97580fe510 0x7f9758108090 secure :-1 s=READY pgs=3980 cs=0 l=1 rev1=1 rx=0x7f975000fc60 tx=0x7f9750004db0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:07.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.356+0000 7f97477fe700 10 client.?.objecter ms_handle_connect 0x7f97580fe510 2024-02-23T00:53:07.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.356+0000 7f97477fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:53:07.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.356+0000 7f97477fe700 1 -- 172.21.15.47:0/2066494222 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9750014070 con 0x7f97580fe510 2024-02-23T00:53:07.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.356+0000 7f97477fe700 1 -- 172.21.15.47:0/2066494222 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f9750011cb0 con 0x7f97580fe510 2024-02-23T00:53:07.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.356+0000 7f97477fe700 1 -- 172.21.15.47:0/2066494222 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9750003880 con 0x7f97580fe510 2024-02-23T00:53:07.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.356+0000 7f975cd5d700 1 -- 172.21.15.47:0/2066494222 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9758108b30 con 0x7f97580fe510 2024-02-23T00:53:07.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.356+0000 7f975cd5d700 10 client.43846.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:53:07.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.356+0000 7f975cd5d700 1 -- 172.21.15.47:0/2066494222 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f975806af10 con 0x7f97580fe510 2024-02-23T00:53:07.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.356+0000 7f975cd5d700 1 librados: init done 2024-02-23T00:53:07.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.357+0000 7f975cd5d700 1 -- 172.21.15.47:0/2066494222 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f975805ee80 con 0x7f97580fe510 2024-02-23T00:53:07.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.358+0000 7f97477fe700 1 -- 172.21.15.47:0/2066494222 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f9750004020 con 0x7f97580fe510 2024-02-23T00:53:07.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.358+0000 7f97477fe700 1 --2- 172.21.15.47:0/2066494222 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9740062cc0 0x7f9740065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:07.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.359+0000 7f97477fe700 1 -- 172.21.15.47:0/2066494222 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(650..650 src has 482..650) v4 ==== 9253+0+0 (secure 0 0 0) 0x7f9750081800 con 0x7f97580fe510 2024-02-23T00:53:07.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.359+0000 7f97477fe700 10 client.43846.objecter ms_dispatch 0x7f9758001550 osd_map(650..650 src has 482..650) v4 2024-02-23T00:53:07.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.359+0000 7f97477fe700 3 client.43846.objecter handle_osd_map got epochs [650,650] > 0 2024-02-23T00:53:07.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.359+0000 7f97477fe700 3 client.43846.objecter handle_osd_map decoding full epoch 650 2024-02-23T00:53:07.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.359+0000 7f975659c700 1 --2- 172.21.15.47:0/2066494222 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9740062cc0 0x7f9740065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:07.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.359+0000 7f97477fe700 20 client.43846.objecter dump_active .. 0 homeless 2024-02-23T00:53:07.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.359+0000 7f97477fe700 10 client.43846.objecter ms_handle_connect 0x7f9740062cc0 2024-02-23T00:53:07.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.359+0000 7f975659c700 1 --2- 172.21.15.47:0/2066494222 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9740062cc0 0x7f9740065180 secure :-1 s=READY pgs=3152 cs=0 l=1 rev1=1 rx=0x7f9748009fd0 tx=0x7f974800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:07.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.363+0000 7f97477fe700 1 -- 172.21.15.47:0/2066494222 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f9750050d70 con 0x7f97580fe510 2024-02-23T00:53:07.434 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:53:07.461 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:07.480 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:53:07.482 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:07.483 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:53:07.546 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:05.858192+0000 osd.0 (osd.0) 277 : cluster [DBG] 5.8 scrub starts 2024-02-23T00:53:07.546 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:05.859120+0000 osd.0 (osd.0) 278 : cluster [DBG] 5.8 scrub ok 2024-02-23T00:53:07.546 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:06.269886+0000 mon.a (mon.0) 3259 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:53:07.547 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:06.280704+0000 mon.a (mon.0) 3260 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-110"}]': finished 2024-02-23T00:53:07.547 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:06.280758+0000 mon.a (mon.0) 3261 : cluster [DBG] osdmap e649: 8 total, 8 up, 7 in 2024-02-23T00:53:07.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:05.858192+0000 osd.0 (osd.0) 277 : cluster [DBG] 5.8 scrub starts 2024-02-23T00:53:07.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:05.859120+0000 osd.0 (osd.0) 278 : cluster [DBG] 5.8 scrub ok 2024-02-23T00:53:07.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:06.269886+0000 mon.a (mon.0) 3259 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:53:07.548 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:06.280704+0000 mon.a (mon.0) 3260 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-110"}]': finished 2024-02-23T00:53:07.548 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:06.280758+0000 mon.a (mon.0) 3261 : cluster [DBG] osdmap e649: 8 total, 8 up, 7 in 2024-02-23T00:53:07.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:07.572+0000 7f975cd5d700 1 -- 172.21.15.47:0/2066494222 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7f97581095d0 con 0x7f97580fe510 2024-02-23T00:53:07.583 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:53:07.611 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:07.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:05.858192+0000 osd.0 (osd.0) 277 : cluster [DBG] 5.8 scrub starts 2024-02-23T00:53:07.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:05.859120+0000 osd.0 (osd.0) 278 : cluster [DBG] 5.8 scrub ok 2024-02-23T00:53:07.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:06.269886+0000 mon.a (mon.0) 3259 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:53:07.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:06.280704+0000 mon.a (mon.0) 3260 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-110"}]': finished 2024-02-23T00:53:07.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:06.280758+0000 mon.a (mon.0) 3261 : cluster [DBG] osdmap e649: 8 total, 8 up, 7 in 2024-02-23T00:53:07.961 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:53:07.964 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:07.964 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:53:08.001 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:53:08.065 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:53:08.093 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:08.112 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:53:08.115 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:08.115 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:53:08.216 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:53:08.244 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:08.263 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:53:08.266 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:08.266 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:53:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:08.290+0000 7f97477fe700 1 -- 172.21.15.47:0/2066494222 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v651) v1 ==== 87+0+0 (secure 0 0 0) 0x7f9750054390 con 0x7f97580fe510 2024-02-23T00:53:08.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:08.296+0000 7f975cd5d700 10 librados: watch_flush enter 2024-02-23T00:53:08.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:08.296+0000 7f975cd5d700 10 librados: watch_flush exit 2024-02-23T00:53:08.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:08.296+0000 7f975cd5d700 20 client.43846.objecter shutdown clearing up homeless session... 2024-02-23T00:53:08.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:08.296+0000 7f975cd5d700 10 client.43846.objecter successfully canceled tick 2024-02-23T00:53:08.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:08.296+0000 7f975cd5d700 1 -- 172.21.15.47:0/2066494222 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9740062cc0 msgr2=0x7f9740065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:08.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:08.296+0000 7f975cd5d700 1 --2- 172.21.15.47:0/2066494222 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9740062cc0 0x7f9740065180 secure :-1 s=READY pgs=3152 cs=0 l=1 rev1=1 rx=0x7f9748009fd0 tx=0x7f974800b040).stop 2024-02-23T00:53:08.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:08.296+0000 7f975cd5d700 1 -- 172.21.15.47:0/2066494222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97580fe510 msgr2=0x7f9758108090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:08.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:08.296+0000 7f975cd5d700 1 --2- 172.21.15.47:0/2066494222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97580fe510 0x7f9758108090 secure :-1 s=READY pgs=3980 cs=0 l=1 rev1=1 rx=0x7f975000fc60 tx=0x7f9750004db0).stop 2024-02-23T00:53:08.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:08.296+0000 7f975cd5d700 1 -- 172.21.15.47:0/2066494222 shutdown_connections 2024-02-23T00:53:08.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:08.296+0000 7f975cd5d700 1 --2- 172.21.15.47:0/2066494222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97580fe510 0x7f9758108090 unknown :-1 s=CLOSED pgs=3980 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:08.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:08.296+0000 7f975cd5d700 1 --2- 172.21.15.47:0/2066494222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97580f8420 0x7f975810de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:08.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:08.296+0000 7f975cd5d700 1 --2- 172.21.15.47:0/2066494222 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9740062cc0 0x7f9740065180 unknown :-1 s=CLOSED pgs=3152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:08.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:08.296+0000 7f975cd5d700 1 --2- 172.21.15.47:0/2066494222 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f97580fdbb0 0x7f975810e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:08.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:08.296+0000 7f975cd5d700 1 -- 172.21.15.47:0/2066494222 >> 172.21.15.47:0/2066494222 conn(0x7f97580f3800 msgr2=0x7f97580fb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:08.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:08.296+0000 7f975cd5d700 1 -- 172.21.15.47:0/2066494222 shutdown_connections 2024-02-23T00:53:08.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:08.296+0000 7f975cd5d700 1 -- 172.21.15.47:0/2066494222 wait complete. 2024-02-23T00:53:08.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:08.296+0000 7f975cd5d700 1 librados: shutdown 2024-02-23T00:53:08.301 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is set 2024-02-23T00:53:08.367 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:53:08.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:06.462961+0000 osd.1 (osd.1) 271 : cluster [DBG] 4.12 scrub starts 2024-02-23T00:53:08.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:06.466477+0000 osd.1 (osd.1) 272 : cluster [DBG] 4.12 scrub ok 2024-02-23T00:53:08.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:06.982381+0000 osd.3 (osd.3) 314 : cluster [DBG] 18.16 scrub starts 2024-02-23T00:53:08.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:06.983201+0000 osd.3 (osd.3) 315 : cluster [DBG] 18.16 scrub ok 2024-02-23T00:53:08.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:07.282394+0000 mon.a (mon.0) 3262 : cluster [DBG] osdmap e650: 8 total, 8 up, 7 in 2024-02-23T00:53:08.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:07.367404+0000 mgr.y (mgr.14517) 573 : cluster [DBG] pgmap v983: 152 pgs: 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:53:08.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:07.573535+0000 mon.b (mon.1) 379 : audit [INF] from='client.? 172.21.15.47:0/2066494222' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:53:08.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:07.737848+0000 mon.a (mon.0) 3263 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:53:08.394 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:08.412 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:53:08.415 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:08.415 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:53:08.516 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:53:08.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:06.462961+0000 osd.1 (osd.1) 271 : cluster [DBG] 4.12 scrub starts 2024-02-23T00:53:08.542 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:06.466477+0000 osd.1 (osd.1) 272 : cluster [DBG] 4.12 scrub ok 2024-02-23T00:53:08.542 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:06.982381+0000 osd.3 (osd.3) 314 : cluster [DBG] 18.16 scrub starts 2024-02-23T00:53:08.542 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:06.983201+0000 osd.3 (osd.3) 315 : cluster [DBG] 18.16 scrub ok 2024-02-23T00:53:08.542 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:07.282394+0000 mon.a (mon.0) 3262 : cluster [DBG] osdmap e650: 8 total, 8 up, 7 in 2024-02-23T00:53:08.542 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:07.367404+0000 mgr.y (mgr.14517) 573 : cluster [DBG] pgmap v983: 152 pgs: 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:53:08.542 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:07.573535+0000 mon.b (mon.1) 379 : audit [INF] from='client.? 172.21.15.47:0/2066494222' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:53:08.542 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:07.737848+0000 mon.a (mon.0) 3263 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:53:08.542 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:06.462961+0000 osd.1 (osd.1) 271 : cluster [DBG] 4.12 scrub starts 2024-02-23T00:53:08.542 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:06.466477+0000 osd.1 (osd.1) 272 : cluster [DBG] 4.12 scrub ok 2024-02-23T00:53:08.543 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:06.982381+0000 osd.3 (osd.3) 314 : cluster [DBG] 18.16 scrub starts 2024-02-23T00:53:08.543 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:06.983201+0000 osd.3 (osd.3) 315 : cluster [DBG] 18.16 scrub ok 2024-02-23T00:53:08.543 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:07.282394+0000 mon.a (mon.0) 3262 : cluster [DBG] osdmap e650: 8 total, 8 up, 7 in 2024-02-23T00:53:08.543 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:07.367404+0000 mgr.y (mgr.14517) 573 : cluster [DBG] pgmap v983: 152 pgs: 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:53:08.543 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:07.573535+0000 mon.b (mon.1) 379 : audit [INF] from='client.? 172.21.15.47:0/2066494222' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:53:08.543 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:07.737848+0000 mon.a (mon.0) 3263 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:53:08.544 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:08.617 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:53:08.621 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:08.622 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:53:08.722 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:53:08.751 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:08.770 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:53:08.772 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:08.773 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:53:08.873 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:53:08.901 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:08.919 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:53:08.922 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:08.922 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:53:09.023 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:53:09.049 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:09.068 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:53:09.070 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:09.071 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:53:09.171 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:53:09.200 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:09.386 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:53:09.399 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:53:09.402 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:09.402 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:53:09.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:07.479923+0000 osd.1 (osd.1) 273 : cluster [DBG] 18.1c deep-scrub starts 2024-02-23T00:53:09.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:07.482977+0000 osd.1 (osd.1) 274 : cluster [DBG] 18.1c deep-scrub ok 2024-02-23T00:53:09.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:08.280481+0000 mon.a (mon.0) 3264 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:53:09.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:08.288796+0000 mon.a (mon.0) 3265 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:53:09.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:08.288846+0000 mon.a (mon.0) 3266 : cluster [DBG] osdmap e651: 8 total, 8 up, 7 in 2024-02-23T00:53:09.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:08.290623+0000 mon.c (mon.2) 372 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:09.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:08.292692+0000 mon.a (mon.0) 3267 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:09.503 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:53:09.529 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:07.479923+0000 osd.1 (osd.1) 273 : cluster [DBG] 18.1c deep-scrub starts 2024-02-23T00:53:09.529 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:07.482977+0000 osd.1 (osd.1) 274 : cluster [DBG] 18.1c deep-scrub ok 2024-02-23T00:53:09.529 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:08.280481+0000 mon.a (mon.0) 3264 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:53:09.529 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:08.288796+0000 mon.a (mon.0) 3265 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:53:09.529 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:08.288846+0000 mon.a (mon.0) 3266 : cluster [DBG] osdmap e651: 8 total, 8 up, 7 in 2024-02-23T00:53:09.529 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:08.290623+0000 mon.c (mon.2) 372 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:09.529 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:08.292692+0000 mon.a (mon.0) 3267 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:09.529 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:07.479923+0000 osd.1 (osd.1) 273 : cluster [DBG] 18.1c deep-scrub starts 2024-02-23T00:53:09.529 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:07.482977+0000 osd.1 (osd.1) 274 : cluster [DBG] 18.1c deep-scrub ok 2024-02-23T00:53:09.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:08.280481+0000 mon.a (mon.0) 3264 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:53:09.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:08.288796+0000 mon.a (mon.0) 3265 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:53:09.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:08.288846+0000 mon.a (mon.0) 3266 : cluster [DBG] osdmap e651: 8 total, 8 up, 7 in 2024-02-23T00:53:09.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:08.290623+0000 mon.c (mon.2) 372 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:09.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:08.292692+0000 mon.a (mon.0) 3267 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:09.530 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:09.629 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:53:09.631 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:09.632 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:53:09.733 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:53:09.761 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:09.780 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:53:09.782 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:09.782 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:53:09.883 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:53:09.910 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:09.929 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:53:09.931 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:09.931 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:53:09.953 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:53:10.032 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:53:10.058 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:10.076 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:53:10.079 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:10.079 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:53:10.180 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:53:10.207 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:10.304 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:53:10.307 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:10.307 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:53:10.408 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:53:10.436 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:10.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:09.292852+0000 mon.a (mon.0) 3268 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:53:10.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:09.292992+0000 mon.a (mon.0) 3269 : cluster [DBG] osdmap e652: 8 total, 8 up, 7 in 2024-02-23T00:53:10.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:09.302004+0000 mon.c (mon.2) 373 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:10.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:09.304177+0000 mon.a (mon.0) 3270 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:10.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:09.367768+0000 mgr.y (mgr.14517) 574 : cluster [DBG] pgmap v986: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:53:10.455 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:09.292852+0000 mon.a (mon.0) 3268 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:53:10.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:09.292992+0000 mon.a (mon.0) 3269 : cluster [DBG] osdmap e652: 8 total, 8 up, 7 in 2024-02-23T00:53:10.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:09.302004+0000 mon.c (mon.2) 373 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:10.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:09.304177+0000 mon.a (mon.0) 3270 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:10.456 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:09.367768+0000 mgr.y (mgr.14517) 574 : cluster [DBG] pgmap v986: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:53:10.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:09.292852+0000 mon.a (mon.0) 3268 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:53:10.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:09.292992+0000 mon.a (mon.0) 3269 : cluster [DBG] osdmap e652: 8 total, 8 up, 7 in 2024-02-23T00:53:10.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:09.302004+0000 mon.c (mon.2) 373 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:10.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:09.304177+0000 mon.a (mon.0) 3270 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:10.456 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:09.367768+0000 mgr.y (mgr.14517) 574 : cluster [DBG] pgmap v986: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:53:10.635 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:53:10.637 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:10.638 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:53:10.734 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:53:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:53:10] "GET /metrics HTTP/1.1" 200 190922 "" "Prometheus/2.33.4" 2024-02-23T00:53:10.739 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:53:10.767 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:10.787 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:53:10.789 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:10.790 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:53:10.890 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:53:10.917 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:10.918 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-23T00:53:10.937 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:53:10.939 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:10.940 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:53:11.041 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:53:11.069 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:11.273 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:53:11.277 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:11.278 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:53:11.304 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:10.299221+0000 mon.a (mon.0) 3271 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:53:11.304 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:10.299287+0000 mon.a (mon.0) 3272 : cluster [DBG] osdmap e653: 8 total, 8 up, 7 in 2024-02-23T00:53:11.304 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:10.303004+0000 mon.c (mon.2) 374 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-112"}]: dispatch 2024-02-23T00:53:11.305 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:10.303379+0000 mon.a (mon.0) 3273 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-112"}]: dispatch 2024-02-23T00:53:11.378 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:53:11.408 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:11.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:10.299221+0000 mon.a (mon.0) 3271 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:53:11.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:10.299287+0000 mon.a (mon.0) 3272 : cluster [DBG] osdmap e653: 8 total, 8 up, 7 in 2024-02-23T00:53:11.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:10.303004+0000 mon.c (mon.2) 374 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-112"}]: dispatch 2024-02-23T00:53:11.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:10.303379+0000 mon.a (mon.0) 3273 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-112"}]: dispatch 2024-02-23T00:53:11.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:10.299221+0000 mon.a (mon.0) 3271 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:53:11.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:10.299287+0000 mon.a (mon.0) 3272 : cluster [DBG] osdmap e653: 8 total, 8 up, 7 in 2024-02-23T00:53:11.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:10.303004+0000 mon.c (mon.2) 374 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-112"}]: dispatch 2024-02-23T00:53:11.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:10.303379+0000 mon.a (mon.0) 3273 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-112"}]: dispatch 2024-02-23T00:53:11.822 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:53:11.822 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:53:12.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:53:11.907+0000 7f3c89cf1700 -1 mon.a@0(leader).osd e654 definitely_dead 0 2024-02-23T00:53:12.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.709+0000 7f0a67f42700 1 -- 172.21.15.47:0/4197699258 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a600f8420 msgr2=0x7f0a60104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:12.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.709+0000 7f0a67f42700 1 --2- 172.21.15.47:0/4197699258 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a600f8420 0x7f0a60104a20 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 rx=0x7f0a58009be0 tx=0x7f0a58004d10).stop 2024-02-23T00:53:12.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.709+0000 7f0a67f42700 1 -- 172.21.15.47:0/4197699258 shutdown_connections 2024-02-23T00:53:12.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.709+0000 7f0a67f42700 1 --2- 172.21.15.47:0/4197699258 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a600f8420 0x7f0a60104a20 unknown :-1 s=CLOSED pgs=2764 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:12.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.709+0000 7f0a67f42700 1 --2- 172.21.15.47:0/4197699258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a600fdbb0 0x7f0a600fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:12.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.709+0000 7f0a67f42700 1 --2- 172.21.15.47:0/4197699258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a600fe510 0x7f0a600f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:12.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.709+0000 7f0a67f42700 1 -- 172.21.15.47:0/4197699258 >> 172.21.15.47:0/4197699258 conn(0x7f0a600f3800 msgr2=0x7f0a600f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:12.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.709+0000 7f0a67f42700 1 -- 172.21.15.47:0/4197699258 shutdown_connections 2024-02-23T00:53:12.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.709+0000 7f0a67f42700 1 -- 172.21.15.47:0/4197699258 wait complete. 2024-02-23T00:53:12.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.709+0000 7f0a67f42700 1 librados: starting msgr at 2024-02-23T00:53:12.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.709+0000 7f0a67f42700 1 librados: starting objecter 2024-02-23T00:53:12.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.709+0000 7f0a67f42700 1 Processor -- start 2024-02-23T00:53:12.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.710+0000 7f0a67f42700 1 -- start start 2024-02-23T00:53:12.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.710+0000 7f0a67f42700 1 librados: setting wanted keys 2024-02-23T00:53:12.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.710+0000 7f0a67f42700 1 librados: calling monclient init 2024-02-23T00:53:12.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.710+0000 7f0a67f42700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a600f8420 0x7f0a6010ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:12.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.710+0000 7f0a67f42700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a600fdbb0 0x7f0a601104d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:12.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.710+0000 7f0a67f42700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a600fe510 0x7f0a6010a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:12.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.710+0000 7f0a67f42700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6010a6c0 con 0x7f0a600fe510 2024-02-23T00:53:12.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.710+0000 7f0a67f42700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6010a830 con 0x7f0a600fdbb0 2024-02-23T00:53:12.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.710+0000 7f0a67f42700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6010a9f0 con 0x7f0a600f8420 2024-02-23T00:53:12.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.710+0000 7f0a65cde700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a600f8420 0x7f0a6010ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:12.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.710+0000 7f0a65cde700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a600f8420 0x7f0a6010ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:33404/0 (socket says 172.21.15.47:33404) 2024-02-23T00:53:12.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.710+0000 7f0a65cde700 1 -- 172.21.15.47:0/2639808780 learned_addr learned my addr 172.21.15.47:0/2639808780 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:53:12.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.710+0000 7f0a654dd700 1 --2- 172.21.15.47:0/2639808780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a600fdbb0 0x7f0a601104d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:12.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.710+0000 7f0a664df700 1 --2- 172.21.15.47:0/2639808780 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a600fe510 0x7f0a6010a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:12.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.710+0000 7f0a65cde700 1 -- 172.21.15.47:0/2639808780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a600fdbb0 msgr2=0x7f0a601104d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:12.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.710+0000 7f0a65cde700 1 --2- 172.21.15.47:0/2639808780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a600fdbb0 0x7f0a601104d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:12.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.710+0000 7f0a65cde700 1 -- 172.21.15.47:0/2639808780 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a600fe510 msgr2=0x7f0a6010a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:12.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.710+0000 7f0a65cde700 1 --2- 172.21.15.47:0/2639808780 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a600fe510 0x7f0a6010a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:12.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.710+0000 7f0a65cde700 1 -- 172.21.15.47:0/2639808780 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a50005040 con 0x7f0a600f8420 2024-02-23T00:53:12.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.710+0000 7f0a654dd700 1 --2- 172.21.15.47:0/2639808780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a600fdbb0 0x7f0a601104d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:53:12.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.710+0000 7f0a65cde700 1 --2- 172.21.15.47:0/2639808780 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a600f8420 0x7f0a6010ffe0 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 rx=0x7f0a5c00e990 tx=0x7f0a5c00b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:12.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.710+0000 7f0a56ffd700 10 client.?.objecter ms_handle_connect 0x7f0a600f8420 2024-02-23T00:53:12.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.710+0000 7f0a56ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:53:12.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.710+0000 7f0a56ffd700 1 -- 172.21.15.47:0/2639808780 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c00ee80 con 0x7f0a600f8420 2024-02-23T00:53:12.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.711+0000 7f0a67f42700 1 -- 172.21.15.47:0/2639808780 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a580097c0 con 0x7f0a600f8420 2024-02-23T00:53:12.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.711+0000 7f0a56ffd700 1 -- 172.21.15.47:0/2639808780 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f0a5c00f040 con 0x7f0a600f8420 2024-02-23T00:53:12.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.711+0000 7f0a56ffd700 1 -- 172.21.15.47:0/2639808780 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c00c840 con 0x7f0a600f8420 2024-02-23T00:53:12.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.711+0000 7f0a67f42700 10 client.41861.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:53:12.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.711+0000 7f0a67f42700 1 -- 172.21.15.47:0/2639808780 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a6010afd0 con 0x7f0a600f8420 2024-02-23T00:53:12.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.711+0000 7f0a67f42700 1 librados: init done 2024-02-23T00:53:12.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.712+0000 7f0a67f42700 1 -- 172.21.15.47:0/2639808780 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a60106ed0 con 0x7f0a600f8420 2024-02-23T00:53:12.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.712+0000 7f0a56ffd700 1 -- 172.21.15.47:0/2639808780 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f0a5c00cc70 con 0x7f0a600f8420 2024-02-23T00:53:12.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.712+0000 7f0a56ffd700 1 --2- 172.21.15.47:0/2639808780 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0a4c062cc0 0x7f0a4c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:12.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.712+0000 7f0a56ffd700 1 -- 172.21.15.47:0/2639808780 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(654..654 src has 482..654) v4 ==== 9634+0+0 (secure 0 0 0) 0x7f0a5c07d3c0 con 0x7f0a600f8420 2024-02-23T00:53:12.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.712+0000 7f0a56ffd700 10 client.41861.objecter ms_dispatch 0x7f0a60001550 osd_map(654..654 src has 482..654) v4 2024-02-23T00:53:12.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.712+0000 7f0a56ffd700 3 client.41861.objecter handle_osd_map got epochs [654,654] > 0 2024-02-23T00:53:12.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.712+0000 7f0a56ffd700 3 client.41861.objecter handle_osd_map decoding full epoch 654 2024-02-23T00:53:12.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.712+0000 7f0a654dd700 1 --2- 172.21.15.47:0/2639808780 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0a4c062cc0 0x7f0a4c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:12.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.713+0000 7f0a56ffd700 20 client.41861.objecter dump_active .. 0 homeless 2024-02-23T00:53:12.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.713+0000 7f0a56ffd700 10 client.41861.objecter ms_handle_connect 0x7f0a4c062cc0 2024-02-23T00:53:12.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.713+0000 7f0a654dd700 1 --2- 172.21.15.47:0/2639808780 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0a4c062cc0 0x7f0a4c065180 secure :-1 s=READY pgs=3153 cs=0 l=1 rev1=1 rx=0x7f0a6010b920 tx=0x7f0a50009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:12.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.717+0000 7f0a56ffd700 1 -- 172.21.15.47:0/2639808780 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f0a5c049f60 con 0x7f0a600f8420 2024-02-23T00:53:12.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:11.907+0000 7f0a67f42700 1 -- 172.21.15.47:0/2639808780 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd out", "ids": ["0"]} v 0) v1 -- 0x7f0a6010b660 con 0x7f0a600f8420 2024-02-23T00:53:12.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:12.324+0000 7f0a56ffd700 1 -- 172.21.15.47:0/2639808780 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd out", "ids": ["0"]}]=0 marked out osd.0. v655) v1 ==== 87+0+0 (secure 0 0 0) 0x7f0a5c04c920 con 0x7f0a600f8420 2024-02-23T00:53:12.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:12.331+0000 7f0a67f42700 10 librados: watch_flush enter 2024-02-23T00:53:12.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:12.331+0000 7f0a67f42700 10 librados: watch_flush exit 2024-02-23T00:53:12.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:12.331+0000 7f0a67f42700 20 client.41861.objecter shutdown clearing up homeless session... 2024-02-23T00:53:12.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:12.331+0000 7f0a67f42700 10 client.41861.objecter successfully canceled tick 2024-02-23T00:53:12.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:12.331+0000 7f0a67f42700 1 -- 172.21.15.47:0/2639808780 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0a4c062cc0 msgr2=0x7f0a4c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:12.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:12.331+0000 7f0a67f42700 1 --2- 172.21.15.47:0/2639808780 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0a4c062cc0 0x7f0a4c065180 secure :-1 s=READY pgs=3153 cs=0 l=1 rev1=1 rx=0x7f0a6010b920 tx=0x7f0a50009cb0).stop 2024-02-23T00:53:12.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:12.332+0000 7f0a67f42700 1 -- 172.21.15.47:0/2639808780 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a600f8420 msgr2=0x7f0a6010ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:12.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:12.332+0000 7f0a67f42700 1 --2- 172.21.15.47:0/2639808780 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a600f8420 0x7f0a6010ffe0 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 rx=0x7f0a5c00e990 tx=0x7f0a5c00b040).stop 2024-02-23T00:53:12.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:12.335+0000 7f0a67f42700 1 -- 172.21.15.47:0/2639808780 shutdown_connections 2024-02-23T00:53:12.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:12.335+0000 7f0a67f42700 1 --2- 172.21.15.47:0/2639808780 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a600fe510 0x7f0a6010a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:12.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:12.335+0000 7f0a67f42700 1 --2- 172.21.15.47:0/2639808780 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0a4c062cc0 0x7f0a4c065180 unknown :-1 s=CLOSED pgs=3153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:12.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:12.335+0000 7f0a67f42700 1 --2- 172.21.15.47:0/2639808780 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a600f8420 0x7f0a6010ffe0 unknown :-1 s=CLOSED pgs=2765 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:12.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:12.335+0000 7f0a67f42700 1 --2- 172.21.15.47:0/2639808780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a600fdbb0 0x7f0a601104d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:12.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:12.335+0000 7f0a67f42700 1 -- 172.21.15.47:0/2639808780 >> 172.21.15.47:0/2639808780 conn(0x7f0a600f3800 msgr2=0x7f0a600f51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:12.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:12.336+0000 7f0a67f42700 1 -- 172.21.15.47:0/2639808780 shutdown_connections 2024-02-23T00:53:12.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:12.340+0000 7f0a67f42700 1 -- 172.21.15.47:0/2639808780 wait complete. 2024-02-23T00:53:12.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:12.341+0000 7f0a67f42700 1 librados: shutdown 2024-02-23T00:53:12.679 INFO:teuthology.orchestra.run.smithi047.stderr:marked out osd.0. 2024-02-23T00:53:12.694 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:53:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:53:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:53:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:11.308797+0000 mon.a (mon.0) 3274 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-112"}]': finished 2024-02-23T00:53:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:11.308874+0000 mon.a (mon.0) 3275 : cluster [DBG] osdmap e654: 8 total, 8 up, 7 in 2024-02-23T00:53:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:11.315369+0000 mon.c (mon.2) 375 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-112", "mode": "writeback"}]: dispatch 2024-02-23T00:53:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:11.315593+0000 mon.a (mon.0) 3276 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:53:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:11.316183+0000 mon.a (mon.0) 3277 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-112", "mode": "writeback"}]: dispatch 2024-02-23T00:53:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:11.368256+0000 mgr.y (mgr.14517) 575 : cluster [DBG] pgmap v989: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:53:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:11.908643+0000 mon.c (mon.2) 376 : audit [INF] from='client.? 172.21.15.47:0/2639808780' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["0"]}]: dispatch 2024-02-23T00:53:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:11.908958+0000 mon.a (mon.0) 3278 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["0"]}]: dispatch 2024-02-23T00:53:12.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:11.909034+0000 mon.a (mon.0) 3279 : cluster [INF] Client client.admin marked osd.0 out, while it was still marked up 2024-02-23T00:53:12.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:11.308797+0000 mon.a (mon.0) 3274 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-112"}]': finished 2024-02-23T00:53:12.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:11.308874+0000 mon.a (mon.0) 3275 : cluster [DBG] osdmap e654: 8 total, 8 up, 7 in 2024-02-23T00:53:12.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:11.315369+0000 mon.c (mon.2) 375 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-112", "mode": "writeback"}]: dispatch 2024-02-23T00:53:12.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:11.315593+0000 mon.a (mon.0) 3276 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:53:12.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:11.316183+0000 mon.a (mon.0) 3277 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-112", "mode": "writeback"}]: dispatch 2024-02-23T00:53:12.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:11.368256+0000 mgr.y (mgr.14517) 575 : cluster [DBG] pgmap v989: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:53:12.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:11.908643+0000 mon.c (mon.2) 376 : audit [INF] from='client.? 172.21.15.47:0/2639808780' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["0"]}]: dispatch 2024-02-23T00:53:12.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:11.908958+0000 mon.a (mon.0) 3278 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["0"]}]: dispatch 2024-02-23T00:53:12.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:11.909034+0000 mon.a (mon.0) 3279 : cluster [INF] Client client.admin marked osd.0 out, while it was still marked up 2024-02-23T00:53:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:11.308797+0000 mon.a (mon.0) 3274 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-112"}]': finished 2024-02-23T00:53:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:11.308874+0000 mon.a (mon.0) 3275 : cluster [DBG] osdmap e654: 8 total, 8 up, 7 in 2024-02-23T00:53:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:11.315369+0000 mon.c (mon.2) 375 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-112", "mode": "writeback"}]: dispatch 2024-02-23T00:53:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:11.315593+0000 mon.a (mon.0) 3276 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:53:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:11.316183+0000 mon.a (mon.0) 3277 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-112", "mode": "writeback"}]: dispatch 2024-02-23T00:53:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:11.368256+0000 mgr.y (mgr.14517) 575 : cluster [DBG] pgmap v989: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-23T00:53:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:11.908643+0000 mon.c (mon.2) 376 : audit [INF] from='client.? 172.21.15.47:0/2639808780' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["0"]}]: dispatch 2024-02-23T00:53:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:11.908958+0000 mon.a (mon.0) 3278 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["0"]}]: dispatch 2024-02-23T00:53:12.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:11.909034+0000 mon.a (mon.0) 3279 : cluster [INF] Client client.admin marked osd.0 out, while it was still marked up 2024-02-23T00:53:12.859 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:53:12.862 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:12.863 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:53:12.964 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:53:12.998 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:13.194 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:53:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:53:13.352 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:53:13.360 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:13.361 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:53:13.461 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:53:13.488 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:13.506 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:53:13.509 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:13.509 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:53:13.610 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:53:13.634 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:12.313627+0000 mon.a (mon.0) 3280 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:53:13.635 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:12.319565+0000 mon.a (mon.0) 3281 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-112", "mode": "writeback"}]': finished 2024-02-23T00:53:13.635 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:12.319653+0000 mon.a (mon.0) 3282 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["0"]}]': finished 2024-02-23T00:53:13.635 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:12.319711+0000 mon.a (mon.0) 3283 : cluster [DBG] osdmap e655: 8 total, 8 up, 6 in 2024-02-23T00:53:13.636 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:13.655 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:53:13.657 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:13.658 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:53:13.668 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:12.313627+0000 mon.a (mon.0) 3280 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:53:13.668 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:12.319565+0000 mon.a (mon.0) 3281 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-112", "mode": "writeback"}]': finished 2024-02-23T00:53:13.668 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:12.319653+0000 mon.a (mon.0) 3282 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["0"]}]': finished 2024-02-23T00:53:13.668 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:12.319711+0000 mon.a (mon.0) 3283 : cluster [DBG] osdmap e655: 8 total, 8 up, 6 in 2024-02-23T00:53:13.669 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:12.313627+0000 mon.a (mon.0) 3280 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:53:13.669 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:12.319565+0000 mon.a (mon.0) 3281 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-112", "mode": "writeback"}]': finished 2024-02-23T00:53:13.669 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:12.319653+0000 mon.a (mon.0) 3282 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["0"]}]': finished 2024-02-23T00:53:13.669 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:12.319711+0000 mon.a (mon.0) 3283 : cluster [DBG] osdmap e655: 8 total, 8 up, 6 in 2024-02-23T00:53:13.758 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:53:13.784 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:13.803 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:53:13.806 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:13.806 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:53:13.907 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:53:13.933 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:13.951 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:53:13.954 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:13.955 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:53:13.985 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:53:14.007 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.3 dump_historic_ops 2024-02-23T00:53:14.055 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:53:14.084 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:14.285 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:53:14.287 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:14.288 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:53:14.389 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:53:14.418 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:14.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:12.846356+0000 mgr.y (mgr.14517) 576 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:53:14.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:13.321911+0000 mon.a (mon.0) 3284 : cluster [DBG] osdmap e656: 8 total, 8 up, 6 in 2024-02-23T00:53:14.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:13.368869+0000 mgr.y (mgr.14517) 577 : cluster [DBG] pgmap v992: 184 pgs: 184 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:53:14.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:13.415970+0000 mon.c (mon.2) 377 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:14.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:13.416374+0000 mon.a (mon.0) 3285 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:14.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:12.846356+0000 mgr.y (mgr.14517) 576 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:53:14.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:13.321911+0000 mon.a (mon.0) 3284 : cluster [DBG] osdmap e656: 8 total, 8 up, 6 in 2024-02-23T00:53:14.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:13.368869+0000 mgr.y (mgr.14517) 577 : cluster [DBG] pgmap v992: 184 pgs: 184 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:53:14.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:13.415970+0000 mon.c (mon.2) 377 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:14.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:13.416374+0000 mon.a (mon.0) 3285 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:14.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:12.846356+0000 mgr.y (mgr.14517) 576 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:53:14.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:13.321911+0000 mon.a (mon.0) 3284 : cluster [DBG] osdmap e656: 8 total, 8 up, 6 in 2024-02-23T00:53:14.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:13.368869+0000 mgr.y (mgr.14517) 577 : cluster [DBG] pgmap v992: 184 pgs: 184 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:53:14.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:13.415970+0000 mon.c (mon.2) 377 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:14.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:13.416374+0000 mon.a (mon.0) 3285 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:14.732 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:53:14.768 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:53:14.772 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:14.773 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:53:14.773 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:53:14.874 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:53:14.904 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:15.657 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:14.321518+0000 osd.3 (osd.3) 316 : cluster [DBG] 6.0 starting backfill to osd.1 from (0'0,0'0] MAX to 107'2 2024-02-23T00:53:15.657 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:14.322570+0000 mon.a (mon.0) 3286 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:53:15.657 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:14.322660+0000 mon.a (mon.0) 3287 : cluster [DBG] osdmap e657: 8 total, 8 up, 6 in 2024-02-23T00:53:15.657 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:14.323716+0000 mon.c (mon.2) 378 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-112"}]: dispatch 2024-02-23T00:53:15.657 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:14.327926+0000 mon.a (mon.0) 3288 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-112"}]: dispatch 2024-02-23T00:53:15.657 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:14.852277+0000 mon.a (mon.0) 3289 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:53:15.657 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:15.179846+0000 mon.a (mon.0) 3290 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:53:15.657 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:15.180579+0000 mon.a (mon.0) 3291 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:53:15.657 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:15.185799+0000 mon.a (mon.0) 3292 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:53:15.658 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:15.269943+0000 mon.a (mon.0) 3293 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:53:15.658 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:15.275334+0000 mon.a (mon.0) 3294 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-112"}]': finished 2024-02-23T00:53:15.658 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:15.275463+0000 mon.a (mon.0) 3295 : cluster [DBG] osdmap e658: 8 total, 8 up, 6 in 2024-02-23T00:53:15.658 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:14.321518+0000 osd.3 (osd.3) 316 : cluster [DBG] 6.0 starting backfill to osd.1 from (0'0,0'0] MAX to 107'2 2024-02-23T00:53:15.658 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:14.322570+0000 mon.a (mon.0) 3286 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:53:15.658 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:14.322660+0000 mon.a (mon.0) 3287 : cluster [DBG] osdmap e657: 8 total, 8 up, 6 in 2024-02-23T00:53:15.658 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:14.323716+0000 mon.c (mon.2) 378 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-112"}]: dispatch 2024-02-23T00:53:15.658 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:14.327926+0000 mon.a (mon.0) 3288 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-112"}]: dispatch 2024-02-23T00:53:15.658 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:14.852277+0000 mon.a (mon.0) 3289 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:53:15.658 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:15.179846+0000 mon.a (mon.0) 3290 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:53:15.659 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:15.180579+0000 mon.a (mon.0) 3291 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:53:15.659 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:15.185799+0000 mon.a (mon.0) 3292 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:53:15.659 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:15.269943+0000 mon.a (mon.0) 3293 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:53:15.659 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:15.275334+0000 mon.a (mon.0) 3294 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-112"}]': finished 2024-02-23T00:53:15.659 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:15.275463+0000 mon.a (mon.0) 3295 : cluster [DBG] osdmap e658: 8 total, 8 up, 6 in 2024-02-23T00:53:15.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:14.321518+0000 osd.3 (osd.3) 316 : cluster [DBG] 6.0 starting backfill to osd.1 from (0'0,0'0] MAX to 107'2 2024-02-23T00:53:15.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:14.322570+0000 mon.a (mon.0) 3286 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:53:15.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:14.322660+0000 mon.a (mon.0) 3287 : cluster [DBG] osdmap e657: 8 total, 8 up, 6 in 2024-02-23T00:53:15.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:14.323716+0000 mon.c (mon.2) 378 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-112"}]: dispatch 2024-02-23T00:53:15.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:14.327926+0000 mon.a (mon.0) 3288 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-112"}]: dispatch 2024-02-23T00:53:15.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:14.852277+0000 mon.a (mon.0) 3289 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:53:15.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:15.179846+0000 mon.a (mon.0) 3290 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:53:15.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:15.180579+0000 mon.a (mon.0) 3291 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:53:15.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:15.185799+0000 mon.a (mon.0) 3292 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:53:15.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:15.269943+0000 mon.a (mon.0) 3293 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:53:15.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:15.275334+0000 mon.a (mon.0) 3294 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-112"}]': finished 2024-02-23T00:53:15.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:15.275463+0000 mon.a (mon.0) 3295 : cluster [DBG] osdmap e658: 8 total, 8 up, 6 in 2024-02-23T00:53:16.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.289+0000 7f3a0de74700 1 -- 172.21.15.47:0/202946379 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3a080fd510 msgr2=0x7f3a080f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:16.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.289+0000 7f3a0de74700 1 --2- 172.21.15.47:0/202946379 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3a080fd510 0x7f3a080f6d40 secure :-1 s=READY pgs=2766 cs=0 l=1 rev1=1 rx=0x7f39fc004870 tx=0x7f39fc009cb0).stop 2024-02-23T00:53:16.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.289+0000 7f3a0de74700 1 -- 172.21.15.47:0/202946379 shutdown_connections 2024-02-23T00:53:16.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.289+0000 7f3a0de74700 1 --2- 172.21.15.47:0/202946379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a080f7400 0x7f3a08103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:16.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.289+0000 7f3a0de74700 1 --2- 172.21.15.47:0/202946379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a080fcbb0 0x7f3a080fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:16.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.289+0000 7f3a0de74700 1 --2- 172.21.15.47:0/202946379 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3a080fd510 0x7f3a080f6d40 unknown :-1 s=CLOSED pgs=2766 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:16.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.289+0000 7f3a0de74700 1 -- 172.21.15.47:0/202946379 >> 172.21.15.47:0/202946379 conn(0x7f3a080f2820 msgr2=0x7f3a080f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:16.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.289+0000 7f3a0de74700 1 -- 172.21.15.47:0/202946379 shutdown_connections 2024-02-23T00:53:16.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.289+0000 7f3a0de74700 1 -- 172.21.15.47:0/202946379 wait complete. 2024-02-23T00:53:16.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.290+0000 7f3a0de74700 1 librados: starting msgr at 2024-02-23T00:53:16.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.290+0000 7f3a0de74700 1 librados: starting objecter 2024-02-23T00:53:16.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.290+0000 7f3a0de74700 1 Processor -- start 2024-02-23T00:53:16.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.290+0000 7f3a0de74700 1 -- start start 2024-02-23T00:53:16.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.290+0000 7f3a0de74700 1 librados: setting wanted keys 2024-02-23T00:53:16.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.290+0000 7f3a0de74700 1 librados: calling monclient init 2024-02-23T00:53:16.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.291+0000 7f3a0de74700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a080f7400 0x7f3a08109080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:16.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.291+0000 7f3a0de74700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3a080fcbb0 0x7f3a08109570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:16.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.291+0000 7f3a0de74700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a080fd510 0x7f3a08109b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:16.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.291+0000 7f3a0de74700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a08114270 con 0x7f3a080f7400 2024-02-23T00:53:16.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.291+0000 7f3a0de74700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a081143e0 con 0x7f3a080fd510 2024-02-23T00:53:16.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.291+0000 7f3a0de74700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a081145a0 con 0x7f3a080fcbb0 2024-02-23T00:53:16.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.291+0000 7f3a077fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a080f7400 0x7f3a08109080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:16.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.291+0000 7f3a06ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3a080fcbb0 0x7f3a08109570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:16.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.291+0000 7f3a077fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a080f7400 0x7f3a08109080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37282/0 (socket says 172.21.15.47:37282) 2024-02-23T00:53:16.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.291+0000 7f3a077fe700 1 -- 172.21.15.47:0/3299731258 learned_addr learned my addr 172.21.15.47:0/3299731258 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:53:16.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.291+0000 7f3a07fff700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a080fd510 0x7f3a08109b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:16.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.291+0000 7f3a07fff700 1 -- 172.21.15.47:0/3299731258 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3a080fcbb0 msgr2=0x7f3a08109570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:16.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.291+0000 7f3a07fff700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3a080fcbb0 0x7f3a08109570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:16.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.291+0000 7f3a07fff700 1 -- 172.21.15.47:0/3299731258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a080f7400 msgr2=0x7f3a08109080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:16.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.291+0000 7f3a07fff700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a080f7400 0x7f3a08109080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:16.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.291+0000 7f3a07fff700 1 -- 172.21.15.47:0/3299731258 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39fc005040 con 0x7f3a080fd510 2024-02-23T00:53:16.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.292+0000 7f3a07fff700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a080fd510 0x7f3a08109b30 secure :-1 s=READY pgs=3981 cs=0 l=1 rev1=1 rx=0x7f3a0000eb70 tx=0x7f3a0000b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:16.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.292+0000 7f3a04ff9700 10 client.?.objecter ms_handle_connect 0x7f3a080fd510 2024-02-23T00:53:16.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.292+0000 7f3a04ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:53:16.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.292+0000 7f3a04ff9700 1 -- 172.21.15.47:0/3299731258 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a00009870 con 0x7f3a080fd510 2024-02-23T00:53:16.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.292+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a08114820 con 0x7f3a080fd510 2024-02-23T00:53:16.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.292+0000 7f3a04ff9700 1 -- 172.21.15.47:0/3299731258 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f3a0000f040 con 0x7f3a080fd510 2024-02-23T00:53:16.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.292+0000 7f3a0de74700 10 client.43858.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:53:16.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.292+0000 7f3a04ff9700 1 -- 172.21.15.47:0/3299731258 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a00007c70 con 0x7f3a080fd510 2024-02-23T00:53:16.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.292+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a08114d30 con 0x7f3a080fd510 2024-02-23T00:53:16.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.292+0000 7f3a0de74700 1 librados: init done 2024-02-23T00:53:16.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.293+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f39f4004f30 con 0x7f3a080fd510 2024-02-23T00:53:16.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.293+0000 7f3a04ff9700 1 -- 172.21.15.47:0/3299731258 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f3a000040b0 con 0x7f3a080fd510 2024-02-23T00:53:16.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.294+0000 7f3a04ff9700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f39f0062cc0 0x7f39f0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:16.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.294+0000 7f3a077fe700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f39f0062cc0 0x7f39f0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:16.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.294+0000 7f3a04ff9700 1 -- 172.21.15.47:0/3299731258 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(659..659 src has 482..659) v4 ==== 9224+0+0 (secure 0 0 0) 0x7f3a0007d9e0 con 0x7f3a080fd510 2024-02-23T00:53:16.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.294+0000 7f3a04ff9700 10 client.43858.objecter ms_dispatch 0x7f3a08001550 osd_map(659..659 src has 482..659) v4 2024-02-23T00:53:16.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.294+0000 7f3a04ff9700 3 client.43858.objecter handle_osd_map got epochs [659,659] > 0 2024-02-23T00:53:16.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.294+0000 7f3a04ff9700 3 client.43858.objecter handle_osd_map decoding full epoch 659 2024-02-23T00:53:16.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.295+0000 7f3a077fe700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f39f0062cc0 0x7f39f0065180 secure :-1 s=READY pgs=3154 cs=0 l=1 rev1=1 rx=0x7f39f8009fd0 tx=0x7f39f800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:16.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.295+0000 7f3a04ff9700 20 client.43858.objecter dump_active .. 0 homeless 2024-02-23T00:53:16.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.295+0000 7f3a04ff9700 10 client.43858.objecter ms_handle_connect 0x7f39f0062cc0 2024-02-23T00:53:16.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.295+0000 7f3a04ff9700 1 -- 172.21.15.47:0/3299731258 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v659) v1 ==== 54+0+16 (secure 0 0 0) 0x7f3a0007e0a0 con 0x7f3a080fd510 2024-02-23T00:53:16.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.295+0000 7f3a0de74700 10 client.43858.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:16.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.295+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_assign -1 1 2024-02-23T00:53:16.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.295+0000 7f3a0de74700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f39f4004560 0x7f39f4004980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:16.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.295+0000 7f3a0de74700 20 client.43858.objecter _get_session s=0x7f39f40051d0 osd=0 3 2024-02-23T00:53:16.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.295+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f40051d0 osd=0 3 2024-02-23T00:53:16.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.295+0000 7f3a0de74700 20 client.43858.objecter _get_session s=0x7f39f40051d0 osd=0 3 2024-02-23T00:53:16.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.295+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_remove -1 1 2024-02-23T00:53:16.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.295+0000 7f3a0de74700 20 client.43858.objecter get_session s=0x7f39f40051d0 osd=0 3 2024-02-23T00:53:16.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.295+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_assign 0 1 2024-02-23T00:53:16.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.295+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f40051d0 osd=0 4 2024-02-23T00:53:16.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.295+0000 7f3a0de74700 10 client.43858.objecter _send_command 1 2024-02-23T00:53:16.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.295+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39f4008960 con 0x7f39f4004560 2024-02-23T00:53:16.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.295+0000 7f3a06ffd700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f39f4004560 0x7f39f4004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:16.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.295+0000 7f3a06ffd700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f39f4004560 0x7f39f4004980 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:16.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.295+0000 7f3a04ff9700 10 client.43858.objecter ms_handle_connect 0x7f39f4004560 2024-02-23T00:53:16.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.297+0000 7f3a04ff9700 1 -- 172.21.15.47:0/3299731258 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f39f4008960 con 0x7f39f4004560 2024-02-23T00:53:16.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.297+0000 7f3a04ff9700 10 client.43858.objecter ms_dispatch 0x7f3a08001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:53:16.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.297+0000 7f3a04ff9700 10 client.43858.objecter _finish_command 1 = system:0 2024-02-23T00:53:16.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.297+0000 7f3a04ff9700 20 client.43858.objecter put_session s=0x7f39f40051d0 osd=0 4 2024-02-23T00:53:16.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.297+0000 7f3a04ff9700 15 client.43858.objecter _session_command_op_remove 0 1 2024-02-23T00:53:16.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.312+0000 7f3a0de74700 10 client.43858.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:53:16.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.312+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_assign -1 2 2024-02-23T00:53:16.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.312+0000 7f3a0de74700 20 client.43858.objecter _get_session s=0x7f39f40051d0 osd=0 3 2024-02-23T00:53:16.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.312+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f40051d0 osd=0 3 2024-02-23T00:53:16.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.312+0000 7f3a0de74700 20 client.43858.objecter _get_session s=0x7f39f40051d0 osd=0 3 2024-02-23T00:53:16.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.312+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_remove -1 2 2024-02-23T00:53:16.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.312+0000 7f3a0de74700 20 client.43858.objecter get_session s=0x7f39f40051d0 osd=0 3 2024-02-23T00:53:16.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.312+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_assign 0 2 2024-02-23T00:53:16.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.312+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f40051d0 osd=0 4 2024-02-23T00:53:16.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.312+0000 7f3a0de74700 10 client.43858.objecter _send_command 2 2024-02-23T00:53:16.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.312+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f39f40054c0 con 0x7f39f4004560 2024-02-23T00:53:16.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.315+0000 7f3a04ff9700 1 -- 172.21.15.47:0/3299731258 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f39f40054c0 con 0x7f39f4004560 2024-02-23T00:53:16.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.315+0000 7f3a04ff9700 10 client.43858.objecter ms_dispatch 0x7f3a08001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:53:16.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.315+0000 7f3a04ff9700 10 client.43858.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:53:16.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.315+0000 7f3a04ff9700 20 client.43858.objecter put_session s=0x7f39f40051d0 osd=0 4 2024-02-23T00:53:16.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.315+0000 7f3a04ff9700 15 client.43858.objecter _session_command_op_remove 0 2 2024-02-23T00:53:16.318 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:53:16.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.315+0000 7f3a0de74700 10 client.43858.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:16.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.315+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_assign -1 3 2024-02-23T00:53:16.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.315+0000 7f3a0de74700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f39f40084c0 0x7f39f4002740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:16.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.315+0000 7f3a0de74700 20 client.43858.objecter _get_session s=0x7f39f40042c0 osd=1 3 2024-02-23T00:53:16.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.315+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f40042c0 osd=1 3 2024-02-23T00:53:16.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.315+0000 7f3a0de74700 20 client.43858.objecter _get_session s=0x7f39f40042c0 osd=1 3 2024-02-23T00:53:16.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.315+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_remove -1 3 2024-02-23T00:53:16.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.315+0000 7f3a0de74700 20 client.43858.objecter get_session s=0x7f39f40042c0 osd=1 3 2024-02-23T00:53:16.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.315+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_assign 1 3 2024-02-23T00:53:16.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.315+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f40042c0 osd=1 4 2024-02-23T00:53:16.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.315+0000 7f3a0de74700 10 client.43858.objecter _send_command 3 2024-02-23T00:53:16.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.315+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39f4002d60 con 0x7f39f40084c0 2024-02-23T00:53:16.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.315+0000 7f3a077fe700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f39f40084c0 0x7f39f4002740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:16.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.316+0000 7f3a04ff9700 10 client.43858.objecter ms_handle_connect 0x7f39f40084c0 2024-02-23T00:53:16.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.316+0000 7f3a077fe700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f39f40084c0 0x7f39f4002740 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:16.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.316+0000 7f3a04ff9700 1 -- 172.21.15.47:0/3299731258 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f39f4002d60 con 0x7f39f40084c0 2024-02-23T00:53:16.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.316+0000 7f3a04ff9700 10 client.43858.objecter ms_dispatch 0x7f3a08001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:53:16.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.316+0000 7f3a04ff9700 10 client.43858.objecter _finish_command 3 = system:0 2024-02-23T00:53:16.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.316+0000 7f3a04ff9700 20 client.43858.objecter put_session s=0x7f39f40042c0 osd=1 4 2024-02-23T00:53:16.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.316+0000 7f3a04ff9700 15 client.43858.objecter _session_command_op_remove 1 3 2024-02-23T00:53:16.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.332+0000 7f3a0de74700 10 client.43858.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:53:16.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.332+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_assign -1 4 2024-02-23T00:53:16.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.332+0000 7f3a0de74700 20 client.43858.objecter _get_session s=0x7f39f40042c0 osd=1 3 2024-02-23T00:53:16.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.332+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f40042c0 osd=1 3 2024-02-23T00:53:16.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.332+0000 7f3a0de74700 20 client.43858.objecter _get_session s=0x7f39f40042c0 osd=1 3 2024-02-23T00:53:16.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.332+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_remove -1 4 2024-02-23T00:53:16.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.332+0000 7f3a0de74700 20 client.43858.objecter get_session s=0x7f39f40042c0 osd=1 3 2024-02-23T00:53:16.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.332+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_assign 1 4 2024-02-23T00:53:16.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.332+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f40042c0 osd=1 4 2024-02-23T00:53:16.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.332+0000 7f3a0de74700 10 client.43858.objecter _send_command 4 2024-02-23T00:53:16.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.332+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f39f4007ae0 con 0x7f39f40084c0 2024-02-23T00:53:16.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.333+0000 7f3a04ff9700 1 -- 172.21.15.47:0/3299731258 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f39f4007ae0 con 0x7f39f40084c0 2024-02-23T00:53:16.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.333+0000 7f3a04ff9700 10 client.43858.objecter ms_dispatch 0x7f3a08001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:53:16.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.333+0000 7f3a04ff9700 10 client.43858.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:53:16.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.333+0000 7f3a04ff9700 20 client.43858.objecter put_session s=0x7f39f40042c0 osd=1 4 2024-02-23T00:53:16.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.333+0000 7f3a04ff9700 15 client.43858.objecter _session_command_op_remove 1 4 2024-02-23T00:53:16.336 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:53:16.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.333+0000 7f3a0de74700 10 client.43858.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:16.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.333+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_assign -1 5 2024-02-23T00:53:16.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.333+0000 7f3a0de74700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f39f40038a0 0x7f39f400d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:16.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.333+0000 7f3a0de74700 20 client.43858.objecter _get_session s=0x7f39f4003730 osd=2 3 2024-02-23T00:53:16.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.333+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f4003730 osd=2 3 2024-02-23T00:53:16.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.333+0000 7f3a0de74700 20 client.43858.objecter _get_session s=0x7f39f4003730 osd=2 3 2024-02-23T00:53:16.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.333+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_remove -1 5 2024-02-23T00:53:16.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.333+0000 7f3a0de74700 20 client.43858.objecter get_session s=0x7f39f4003730 osd=2 3 2024-02-23T00:53:16.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.333+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_assign 2 5 2024-02-23T00:53:16.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.333+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f4003730 osd=2 4 2024-02-23T00:53:16.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.333+0000 7f3a0de74700 10 client.43858.objecter _send_command 5 2024-02-23T00:53:16.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.333+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39f400ff00 con 0x7f39f40038a0 2024-02-23T00:53:16.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.333+0000 7f3a06ffd700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f39f40038a0 0x7f39f400d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:16.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.334+0000 7f3a06ffd700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f39f40038a0 0x7f39f400d000 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:16.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.334+0000 7f3a04ff9700 10 client.43858.objecter ms_handle_connect 0x7f39f40038a0 2024-02-23T00:53:16.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.334+0000 7f3a04ff9700 1 -- 172.21.15.47:0/3299731258 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f39f400ff00 con 0x7f39f40038a0 2024-02-23T00:53:16.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.334+0000 7f3a04ff9700 10 client.43858.objecter ms_dispatch 0x7f3a08001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:53:16.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.334+0000 7f3a04ff9700 10 client.43858.objecter _finish_command 5 = system:0 2024-02-23T00:53:16.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.334+0000 7f3a04ff9700 20 client.43858.objecter put_session s=0x7f39f4003730 osd=2 4 2024-02-23T00:53:16.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.334+0000 7f3a04ff9700 15 client.43858.objecter _session_command_op_remove 2 5 2024-02-23T00:53:16.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.349+0000 7f3a0de74700 10 client.43858.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:53:16.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.349+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_assign -1 6 2024-02-23T00:53:16.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.349+0000 7f3a0de74700 20 client.43858.objecter _get_session s=0x7f39f4003730 osd=2 3 2024-02-23T00:53:16.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.349+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f4003730 osd=2 3 2024-02-23T00:53:16.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.349+0000 7f3a0de74700 20 client.43858.objecter _get_session s=0x7f39f4003730 osd=2 3 2024-02-23T00:53:16.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.349+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_remove -1 6 2024-02-23T00:53:16.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.349+0000 7f3a0de74700 20 client.43858.objecter get_session s=0x7f39f4003730 osd=2 3 2024-02-23T00:53:16.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.349+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_assign 2 6 2024-02-23T00:53:16.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.349+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f4003730 osd=2 4 2024-02-23T00:53:16.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.349+0000 7f3a0de74700 10 client.43858.objecter _send_command 6 2024-02-23T00:53:16.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.349+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f39f40031a0 con 0x7f39f40038a0 2024-02-23T00:53:16.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.351+0000 7f3a04ff9700 1 -- 172.21.15.47:0/3299731258 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f39f40031a0 con 0x7f39f40038a0 2024-02-23T00:53:16.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.351+0000 7f3a04ff9700 10 client.43858.objecter ms_dispatch 0x7f3a08001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:53:16.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.351+0000 7f3a04ff9700 10 client.43858.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:53:16.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.351+0000 7f3a04ff9700 20 client.43858.objecter put_session s=0x7f39f4003730 osd=2 4 2024-02-23T00:53:16.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.351+0000 7f3a04ff9700 15 client.43858.objecter _session_command_op_remove 2 6 2024-02-23T00:53:16.354 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:53:16.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.351+0000 7f3a0de74700 10 client.43858.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:16.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.351+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_assign -1 7 2024-02-23T00:53:16.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.351+0000 7f3a0de74700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f39f400e7b0 0x7f39f400ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:16.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.351+0000 7f3a0de74700 20 client.43858.objecter _get_session s=0x7f39f4001b70 osd=3 3 2024-02-23T00:53:16.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.351+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f4001b70 osd=3 3 2024-02-23T00:53:16.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.351+0000 7f3a0de74700 20 client.43858.objecter _get_session s=0x7f39f4001b70 osd=3 3 2024-02-23T00:53:16.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.351+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_remove -1 7 2024-02-23T00:53:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.351+0000 7f3a0de74700 20 client.43858.objecter get_session s=0x7f39f4001b70 osd=3 3 2024-02-23T00:53:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.351+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_assign 3 7 2024-02-23T00:53:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.351+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f4001b70 osd=3 4 2024-02-23T00:53:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.351+0000 7f3a0de74700 10 client.43858.objecter _send_command 7 2024-02-23T00:53:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.351+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39f4002330 con 0x7f39f400e7b0 2024-02-23T00:53:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.351+0000 7f3a07fff700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f39f400e7b0 0x7f39f400ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.352+0000 7f3a04ff9700 10 client.43858.objecter ms_handle_connect 0x7f39f400e7b0 2024-02-23T00:53:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.352+0000 7f3a07fff700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f39f400e7b0 0x7f39f400ec60 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.352+0000 7f3a04ff9700 1 -- 172.21.15.47:0/3299731258 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f39f4002330 con 0x7f39f400e7b0 2024-02-23T00:53:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.352+0000 7f3a04ff9700 10 client.43858.objecter ms_dispatch 0x7f3a08001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:53:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.352+0000 7f3a04ff9700 10 client.43858.objecter _finish_command 7 = system:0 2024-02-23T00:53:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.352+0000 7f3a04ff9700 20 client.43858.objecter put_session s=0x7f39f4001b70 osd=3 4 2024-02-23T00:53:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.352+0000 7f3a04ff9700 15 client.43858.objecter _session_command_op_remove 3 7 2024-02-23T00:53:16.357 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:53:16.359 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:16.360 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:53:16.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.368+0000 7f3a0de74700 10 client.43858.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:53:16.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.368+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_assign -1 8 2024-02-23T00:53:16.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.368+0000 7f3a0de74700 20 client.43858.objecter _get_session s=0x7f39f4001b70 osd=3 3 2024-02-23T00:53:16.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.368+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f4001b70 osd=3 3 2024-02-23T00:53:16.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.368+0000 7f3a0de74700 20 client.43858.objecter _get_session s=0x7f39f4001b70 osd=3 3 2024-02-23T00:53:16.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.368+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_remove -1 8 2024-02-23T00:53:16.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.368+0000 7f3a0de74700 20 client.43858.objecter get_session s=0x7f39f4001b70 osd=3 3 2024-02-23T00:53:16.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.368+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_assign 3 8 2024-02-23T00:53:16.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.368+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f4001b70 osd=3 4 2024-02-23T00:53:16.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.368+0000 7f3a0de74700 10 client.43858.objecter _send_command 8 2024-02-23T00:53:16.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.368+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f39f400f950 con 0x7f39f400e7b0 2024-02-23T00:53:16.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.370+0000 7f3a04ff9700 1 -- 172.21.15.47:0/3299731258 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f39f400f950 con 0x7f39f400e7b0 2024-02-23T00:53:16.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.370+0000 7f3a04ff9700 10 client.43858.objecter ms_dispatch 0x7f3a08001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:53:16.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.370+0000 7f3a04ff9700 10 client.43858.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:53:16.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.370+0000 7f3a04ff9700 20 client.43858.objecter put_session s=0x7f39f4001b70 osd=3 4 2024-02-23T00:53:16.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.370+0000 7f3a04ff9700 15 client.43858.objecter _session_command_op_remove 3 8 2024-02-23T00:53:16.373 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:53:16.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.370+0000 7f3a0de74700 10 client.43858.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:16.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.370+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_assign -1 9 2024-02-23T00:53:16.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.370+0000 7f3a0de74700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f39f4012110 0x7f39f4014500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:16.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.370+0000 7f3a0de74700 20 client.43858.objecter _get_session s=0x7f39f400f950 osd=4 3 2024-02-23T00:53:16.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.370+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f400f950 osd=4 3 2024-02-23T00:53:16.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.370+0000 7f3a0de74700 20 client.43858.objecter _get_session s=0x7f39f400f950 osd=4 3 2024-02-23T00:53:16.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.370+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_remove -1 9 2024-02-23T00:53:16.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.370+0000 7f3a0de74700 20 client.43858.objecter get_session s=0x7f39f400f950 osd=4 3 2024-02-23T00:53:16.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.370+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_assign 4 9 2024-02-23T00:53:16.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.370+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f400f950 osd=4 4 2024-02-23T00:53:16.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.370+0000 7f3a0de74700 10 client.43858.objecter _send_command 9 2024-02-23T00:53:16.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.370+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39f4017150 con 0x7f39f4012110 2024-02-23T00:53:16.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.371+0000 7f3a077fe700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f39f4012110 0x7f39f4014500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:16.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.371+0000 7f3a04ff9700 10 client.43858.objecter ms_handle_connect 0x7f39f4012110 2024-02-23T00:53:16.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.371+0000 7f3a077fe700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f39f4012110 0x7f39f4014500 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:16.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.372+0000 7f3a04ff9700 1 -- 172.21.15.47:0/3299731258 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f39f4017150 con 0x7f39f4012110 2024-02-23T00:53:16.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.372+0000 7f3a04ff9700 10 client.43858.objecter ms_dispatch 0x7f3a08001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:53:16.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.372+0000 7f3a04ff9700 10 client.43858.objecter _finish_command 9 = system:0 2024-02-23T00:53:16.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.372+0000 7f3a04ff9700 20 client.43858.objecter put_session s=0x7f39f400f950 osd=4 4 2024-02-23T00:53:16.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.372+0000 7f3a04ff9700 15 client.43858.objecter _session_command_op_remove 4 9 2024-02-23T00:53:16.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.387+0000 7f3a0de74700 10 client.43858.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:53:16.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.387+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_assign -1 10 2024-02-23T00:53:16.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.387+0000 7f3a0de74700 20 client.43858.objecter _get_session s=0x7f39f400f950 osd=4 3 2024-02-23T00:53:16.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.387+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f400f950 osd=4 3 2024-02-23T00:53:16.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.387+0000 7f3a0de74700 20 client.43858.objecter _get_session s=0x7f39f400f950 osd=4 3 2024-02-23T00:53:16.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.387+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_remove -1 10 2024-02-23T00:53:16.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.387+0000 7f3a0de74700 20 client.43858.objecter get_session s=0x7f39f400f950 osd=4 3 2024-02-23T00:53:16.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.387+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_assign 4 10 2024-02-23T00:53:16.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.387+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f400f950 osd=4 4 2024-02-23T00:53:16.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.387+0000 7f3a0de74700 10 client.43858.objecter _send_command 10 2024-02-23T00:53:16.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.387+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f39f40149f0 con 0x7f39f4012110 2024-02-23T00:53:16.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.389+0000 7f3a04ff9700 1 -- 172.21.15.47:0/3299731258 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f39f40149f0 con 0x7f39f4012110 2024-02-23T00:53:16.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.389+0000 7f3a04ff9700 10 client.43858.objecter ms_dispatch 0x7f3a08001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:53:16.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.389+0000 7f3a04ff9700 10 client.43858.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:53:16.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.389+0000 7f3a04ff9700 20 client.43858.objecter put_session s=0x7f39f400f950 osd=4 4 2024-02-23T00:53:16.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.389+0000 7f3a04ff9700 15 client.43858.objecter _session_command_op_remove 4 10 2024-02-23T00:53:16.392 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:53:16.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.389+0000 7f3a0de74700 10 client.43858.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:16.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.389+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_assign -1 11 2024-02-23T00:53:16.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.389+0000 7f3a0de74700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f39f4015070 0x7f39f4017480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:16.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.389+0000 7f3a0de74700 20 client.43858.objecter _get_session s=0x7f39f40149f0 osd=5 3 2024-02-23T00:53:16.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.389+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f40149f0 osd=5 3 2024-02-23T00:53:16.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.389+0000 7f3a0de74700 20 client.43858.objecter _get_session s=0x7f39f40149f0 osd=5 3 2024-02-23T00:53:16.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.389+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_remove -1 11 2024-02-23T00:53:16.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.389+0000 7f3a0de74700 20 client.43858.objecter get_session s=0x7f39f40149f0 osd=5 3 2024-02-23T00:53:16.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.389+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_assign 5 11 2024-02-23T00:53:16.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.389+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f40149f0 osd=5 4 2024-02-23T00:53:16.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.389+0000 7f3a0de74700 10 client.43858.objecter _send_command 11 2024-02-23T00:53:16.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.389+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39f401e6d0 con 0x7f39f4015070 2024-02-23T00:53:16.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.390+0000 7f3a06ffd700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f39f4015070 0x7f39f4017480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:16.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.390+0000 7f3a06ffd700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f39f4015070 0x7f39f4017480 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:16.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.390+0000 7f3a04ff9700 10 client.43858.objecter ms_handle_connect 0x7f39f4015070 2024-02-23T00:53:16.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.391+0000 7f3a04ff9700 1 -- 172.21.15.47:0/3299731258 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f39f401e6d0 con 0x7f39f4015070 2024-02-23T00:53:16.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.391+0000 7f3a04ff9700 10 client.43858.objecter ms_dispatch 0x7f3a08001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:53:16.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.391+0000 7f3a04ff9700 10 client.43858.objecter _finish_command 11 = system:0 2024-02-23T00:53:16.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.391+0000 7f3a04ff9700 20 client.43858.objecter put_session s=0x7f39f40149f0 osd=5 4 2024-02-23T00:53:16.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.391+0000 7f3a04ff9700 15 client.43858.objecter _session_command_op_remove 5 11 2024-02-23T00:53:16.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.405+0000 7f3a0de74700 10 client.43858.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:53:16.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.405+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_assign -1 12 2024-02-23T00:53:16.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.405+0000 7f3a0de74700 20 client.43858.objecter _get_session s=0x7f39f40149f0 osd=5 3 2024-02-23T00:53:16.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.405+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f40149f0 osd=5 3 2024-02-23T00:53:16.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.405+0000 7f3a0de74700 20 client.43858.objecter _get_session s=0x7f39f40149f0 osd=5 3 2024-02-23T00:53:16.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.405+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_remove -1 12 2024-02-23T00:53:16.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.405+0000 7f3a0de74700 20 client.43858.objecter get_session s=0x7f39f40149f0 osd=5 3 2024-02-23T00:53:16.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.405+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_assign 5 12 2024-02-23T00:53:16.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.405+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f40149f0 osd=5 4 2024-02-23T00:53:16.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.405+0000 7f3a0de74700 10 client.43858.objecter _send_command 12 2024-02-23T00:53:16.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.405+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f39f401e6d0 con 0x7f39f4015070 2024-02-23T00:53:16.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.407+0000 7f3a04ff9700 1 -- 172.21.15.47:0/3299731258 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f39f401e6d0 con 0x7f39f4015070 2024-02-23T00:53:16.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.407+0000 7f3a04ff9700 10 client.43858.objecter ms_dispatch 0x7f3a08001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:53:16.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.407+0000 7f3a04ff9700 10 client.43858.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:53:16.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.407+0000 7f3a04ff9700 20 client.43858.objecter put_session s=0x7f39f40149f0 osd=5 4 2024-02-23T00:53:16.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.407+0000 7f3a04ff9700 15 client.43858.objecter _session_command_op_remove 5 12 2024-02-23T00:53:16.410 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:53:16.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.407+0000 7f3a0de74700 10 client.43858.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:16.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.407+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_assign -1 13 2024-02-23T00:53:16.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.407+0000 7f3a0de74700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f39f40195f0 0x7f39f4019a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:16.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.407+0000 7f3a0de74700 20 client.43858.objecter _get_session s=0x7f39f401e6d0 osd=6 3 2024-02-23T00:53:16.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.407+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f401e6d0 osd=6 3 2024-02-23T00:53:16.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.407+0000 7f3a0de74700 20 client.43858.objecter _get_session s=0x7f39f401e6d0 osd=6 3 2024-02-23T00:53:16.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.407+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_remove -1 13 2024-02-23T00:53:16.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.407+0000 7f3a0de74700 20 client.43858.objecter get_session s=0x7f39f401e6d0 osd=6 3 2024-02-23T00:53:16.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.407+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_assign 6 13 2024-02-23T00:53:16.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.407+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f401e6d0 osd=6 4 2024-02-23T00:53:16.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.407+0000 7f3a0de74700 10 client.43858.objecter _send_command 13 2024-02-23T00:53:16.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.407+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39f4021a00 con 0x7f39f40195f0 2024-02-23T00:53:16.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.407+0000 7f3a07fff700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f39f40195f0 0x7f39f4019a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:16.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.408+0000 7f3a04ff9700 10 client.43858.objecter ms_handle_connect 0x7f39f40195f0 2024-02-23T00:53:16.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.408+0000 7f3a07fff700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f39f40195f0 0x7f39f4019a70 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:16.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.408+0000 7f3a04ff9700 1 -- 172.21.15.47:0/3299731258 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f39f4021a00 con 0x7f39f40195f0 2024-02-23T00:53:16.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.408+0000 7f3a04ff9700 10 client.43858.objecter ms_dispatch 0x7f3a08001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:53:16.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.408+0000 7f3a04ff9700 10 client.43858.objecter _finish_command 13 = system:0 2024-02-23T00:53:16.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.408+0000 7f3a04ff9700 20 client.43858.objecter put_session s=0x7f39f401e6d0 osd=6 4 2024-02-23T00:53:16.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.408+0000 7f3a04ff9700 15 client.43858.objecter _session_command_op_remove 6 13 2024-02-23T00:53:16.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.423+0000 7f3a0de74700 10 client.43858.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:53:16.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.423+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_assign -1 14 2024-02-23T00:53:16.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.423+0000 7f3a0de74700 20 client.43858.objecter _get_session s=0x7f39f401e6d0 osd=6 3 2024-02-23T00:53:16.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.423+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f401e6d0 osd=6 3 2024-02-23T00:53:16.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.423+0000 7f3a0de74700 20 client.43858.objecter _get_session s=0x7f39f401e6d0 osd=6 3 2024-02-23T00:53:16.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.423+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_remove -1 14 2024-02-23T00:53:16.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.423+0000 7f3a0de74700 20 client.43858.objecter get_session s=0x7f39f401e6d0 osd=6 3 2024-02-23T00:53:16.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.423+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_assign 6 14 2024-02-23T00:53:16.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.423+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f401e6d0 osd=6 4 2024-02-23T00:53:16.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.423+0000 7f3a0de74700 10 client.43858.objecter _send_command 14 2024-02-23T00:53:16.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.423+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f39f4017a80 con 0x7f39f40195f0 2024-02-23T00:53:16.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.425+0000 7f3a04ff9700 1 -- 172.21.15.47:0/3299731258 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f39f4017a80 con 0x7f39f40195f0 2024-02-23T00:53:16.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.425+0000 7f3a04ff9700 10 client.43858.objecter ms_dispatch 0x7f3a08001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:53:16.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.425+0000 7f3a04ff9700 10 client.43858.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:53:16.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.425+0000 7f3a04ff9700 20 client.43858.objecter put_session s=0x7f39f401e6d0 osd=6 4 2024-02-23T00:53:16.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.425+0000 7f3a04ff9700 15 client.43858.objecter _session_command_op_remove 6 14 2024-02-23T00:53:16.428 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:53:16.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.425+0000 7f3a0de74700 10 client.43858.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:16.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.425+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_assign -1 15 2024-02-23T00:53:16.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.425+0000 7f3a0de74700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f39f4018100 0x7f39f4018560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:16.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.425+0000 7f3a0de74700 20 client.43858.objecter _get_session s=0x7f39f4017a80 osd=7 3 2024-02-23T00:53:16.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.425+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f4017a80 osd=7 3 2024-02-23T00:53:16.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.425+0000 7f3a0de74700 20 client.43858.objecter _get_session s=0x7f39f4017a80 osd=7 3 2024-02-23T00:53:16.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.425+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_remove -1 15 2024-02-23T00:53:16.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.425+0000 7f3a0de74700 20 client.43858.objecter get_session s=0x7f39f4017a80 osd=7 3 2024-02-23T00:53:16.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.425+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_assign 7 15 2024-02-23T00:53:16.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.425+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f4017a80 osd=7 4 2024-02-23T00:53:16.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.425+0000 7f3a0de74700 10 client.43858.objecter _send_command 15 2024-02-23T00:53:16.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.425+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39f4018c40 con 0x7f39f4018100 2024-02-23T00:53:16.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.426+0000 7f3a077fe700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f39f4018100 0x7f39f4018560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:16.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.426+0000 7f3a04ff9700 10 client.43858.objecter ms_handle_connect 0x7f39f4018100 2024-02-23T00:53:16.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.426+0000 7f3a077fe700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f39f4018100 0x7f39f4018560 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:16.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.427+0000 7f3a04ff9700 1 -- 172.21.15.47:0/3299731258 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f39f4018c40 con 0x7f39f4018100 2024-02-23T00:53:16.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.427+0000 7f3a04ff9700 10 client.43858.objecter ms_dispatch 0x7f3a08001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:53:16.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.427+0000 7f3a04ff9700 10 client.43858.objecter _finish_command 15 = system:0 2024-02-23T00:53:16.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.427+0000 7f3a04ff9700 20 client.43858.objecter put_session s=0x7f39f4017a80 osd=7 4 2024-02-23T00:53:16.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.427+0000 7f3a04ff9700 15 client.43858.objecter _session_command_op_remove 7 15 2024-02-23T00:53:16.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.441+0000 7f3a0de74700 10 client.43858.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:53:16.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.441+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_assign -1 16 2024-02-23T00:53:16.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.441+0000 7f3a0de74700 20 client.43858.objecter _get_session s=0x7f39f4017a80 osd=7 3 2024-02-23T00:53:16.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.441+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f4017a80 osd=7 3 2024-02-23T00:53:16.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.441+0000 7f3a0de74700 20 client.43858.objecter _get_session s=0x7f39f4017a80 osd=7 3 2024-02-23T00:53:16.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.441+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_remove -1 16 2024-02-23T00:53:16.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.441+0000 7f3a0de74700 20 client.43858.objecter get_session s=0x7f39f4017a80 osd=7 3 2024-02-23T00:53:16.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.441+0000 7f3a0de74700 15 client.43858.objecter _session_command_op_assign 7 16 2024-02-23T00:53:16.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.441+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f4017a80 osd=7 4 2024-02-23T00:53:16.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.441+0000 7f3a0de74700 10 client.43858.objecter _send_command 16 2024-02-23T00:53:16.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.441+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f39f4018c40 con 0x7f39f4018100 2024-02-23T00:53:16.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.443+0000 7f3a04ff9700 1 -- 172.21.15.47:0/3299731258 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f39f4018c40 con 0x7f39f4018100 2024-02-23T00:53:16.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.443+0000 7f3a04ff9700 10 client.43858.objecter ms_dispatch 0x7f3a08001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:53:16.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.443+0000 7f3a04ff9700 10 client.43858.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:53:16.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.443+0000 7f3a04ff9700 20 client.43858.objecter put_session s=0x7f39f4017a80 osd=7 4 2024-02-23T00:53:16.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.443+0000 7f3a04ff9700 15 client.43858.objecter _session_command_op_remove 7 16 2024-02-23T00:53:16.446 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:53:16.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 10 librados: watch_flush enter 2024-02-23T00:53:16.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 10 librados: watch_flush exit 2024-02-23T00:53:16.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 10 client.43858.objecter close_session for osd.0 2024-02-23T00:53:16.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f39f4004560 msgr2=0x7f39f4004980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:16.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f39f4004560 0x7f39f4004980 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:16.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f40051d0 osd=0 1 2024-02-23T00:53:16.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 10 client.43858.objecter close_session for osd.1 2024-02-23T00:53:16.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f39f40084c0 msgr2=0x7f39f4002740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:16.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f39f40084c0 0x7f39f4002740 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:16.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f40042c0 osd=1 1 2024-02-23T00:53:16.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 10 client.43858.objecter close_session for osd.2 2024-02-23T00:53:16.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f39f40038a0 msgr2=0x7f39f400d000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:16.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f39f40038a0 0x7f39f400d000 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:16.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f4003730 osd=2 1 2024-02-23T00:53:16.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 10 client.43858.objecter close_session for osd.3 2024-02-23T00:53:16.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f39f400e7b0 msgr2=0x7f39f400ec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:16.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f39f400e7b0 0x7f39f400ec60 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:16.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a07fff700 1 -- 172.21.15.47:0/3299731258 reap_dead start 2024-02-23T00:53:16.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f4001b70 osd=3 1 2024-02-23T00:53:16.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 10 client.43858.objecter close_session for osd.4 2024-02-23T00:53:16.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f39f4012110 msgr2=0x7f39f4014500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:16.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f39f4012110 0x7f39f4014500 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:16.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f400f950 osd=4 1 2024-02-23T00:53:16.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 10 client.43858.objecter close_session for osd.5 2024-02-23T00:53:16.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f39f4015070 msgr2=0x7f39f4017480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:16.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f39f4015070 0x7f39f4017480 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:16.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f40149f0 osd=5 1 2024-02-23T00:53:16.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 10 client.43858.objecter close_session for osd.6 2024-02-23T00:53:16.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f39f40195f0 msgr2=0x7f39f4019a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:16.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f39f40195f0 0x7f39f4019a70 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:16.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f401e6d0 osd=6 1 2024-02-23T00:53:16.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 10 client.43858.objecter close_session for osd.7 2024-02-23T00:53:16.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f39f4018100 msgr2=0x7f39f4018560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:16.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f39f4018100 0x7f39f4018560 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:16.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 20 client.43858.objecter put_session s=0x7f39f4017a80 osd=7 1 2024-02-23T00:53:16.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 20 client.43858.objecter shutdown clearing up homeless session... 2024-02-23T00:53:16.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 10 client.43858.objecter successfully canceled tick 2024-02-23T00:53:16.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f39f0062cc0 msgr2=0x7f39f0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:16.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f39f0062cc0 0x7f39f0065180 secure :-1 s=READY pgs=3154 cs=0 l=1 rev1=1 rx=0x7f39f8009fd0 tx=0x7f39f800b040).stop 2024-02-23T00:53:16.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a07fff700 1 -- 172.21.15.47:0/3299731258 reap_dead start 2024-02-23T00:53:16.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a080fd510 msgr2=0x7f3a08109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:16.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a080fd510 0x7f3a08109b30 secure :-1 s=READY pgs=3981 cs=0 l=1 rev1=1 rx=0x7f3a0000eb70 tx=0x7f3a0000b040).stop 2024-02-23T00:53:16.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 shutdown_connections 2024-02-23T00:53:16.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 1 --2- 172.21.15.47:0/3299731258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a080fd510 0x7f3a08109b30 unknown :-1 s=CLOSED pgs=3981 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:16.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 >> 172.21.15.47:0/3299731258 conn(0x7f3a080f2820 msgr2=0x7f3a080fa940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:16.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 shutdown_connections 2024-02-23T00:53:16.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 1 -- 172.21.15.47:0/3299731258 wait complete. 2024-02-23T00:53:16.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:16.444+0000 7f3a0de74700 1 librados: shutdown 2024-02-23T00:53:16.451 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:53:16.451 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:53:16.451 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:53:16.451 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:53:16.451 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:53:16.451 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:53:16.451 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:53:16.451 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:53:16.461 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:53:16.492 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:16.513 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:53:16.516 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:16.517 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:53:16.618 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:53:16.654 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:16.895 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:15.369296+0000 mgr.y (mgr.14517) 578 : cluster [DBG] pgmap v995: 184 pgs: 184 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:53:16.896 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:16.276726+0000 mon.a (mon.0) 3296 : cluster [DBG] osdmap e659: 8 total, 8 up, 6 in 2024-02-23T00:53:16.896 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:15.369296+0000 mgr.y (mgr.14517) 578 : cluster [DBG] pgmap v995: 184 pgs: 184 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:53:16.896 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:16.276726+0000 mon.a (mon.0) 3296 : cluster [DBG] osdmap e659: 8 total, 8 up, 6 in 2024-02-23T00:53:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:15.369296+0000 mgr.y (mgr.14517) 578 : cluster [DBG] pgmap v995: 184 pgs: 184 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-23T00:53:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:16.276726+0000 mon.a (mon.0) 3296 : cluster [DBG] osdmap e659: 8 total, 8 up, 6 in 2024-02-23T00:53:18.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:16.294420+0000 mon.b (mon.1) 380 : audit [DBG] from='client.? 172.21.15.47:0/3299731258' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:53:18.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:17.278505+0000 mon.a (mon.0) 3297 : cluster [DBG] osdmap e660: 8 total, 8 up, 6 in 2024-02-23T00:53:18.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:17.280501+0000 mon.c (mon.2) 379 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:18.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:17.280973+0000 mon.a (mon.0) 3298 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:18.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:17.369875+0000 mgr.y (mgr.14517) 579 : cluster [DBG] pgmap v998: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s; 730 B/s, 23 objects/s recovering 2024-02-23T00:53:18.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:16.294420+0000 mon.b (mon.1) 380 : audit [DBG] from='client.? 172.21.15.47:0/3299731258' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:53:18.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:17.278505+0000 mon.a (mon.0) 3297 : cluster [DBG] osdmap e660: 8 total, 8 up, 6 in 2024-02-23T00:53:18.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:17.280501+0000 mon.c (mon.2) 379 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:18.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:17.280973+0000 mon.a (mon.0) 3298 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:18.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:17.369875+0000 mgr.y (mgr.14517) 579 : cluster [DBG] pgmap v998: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s; 730 B/s, 23 objects/s recovering 2024-02-23T00:53:18.134 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:53:18.137 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:18.137 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:53:18.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:16.294420+0000 mon.b (mon.1) 380 : audit [DBG] from='client.? 172.21.15.47:0/3299731258' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:53:18.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:17.278505+0000 mon.a (mon.0) 3297 : cluster [DBG] osdmap e660: 8 total, 8 up, 6 in 2024-02-23T00:53:18.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:17.280501+0000 mon.c (mon.2) 379 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:18.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:17.280973+0000 mon.a (mon.0) 3298 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:18.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:17.369875+0000 mgr.y (mgr.14517) 579 : cluster [DBG] pgmap v998: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s; 730 B/s, 23 objects/s recovering 2024-02-23T00:53:18.238 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:53:18.270 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:18.301 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:53:18.308 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:18.308 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:53:18.351 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6, 7] out_osds: [2, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-23T00:53:18.351 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-23T00:53:18.351 INFO:tasks.thrashosds.thrasher:Reweighting osd 3 to 0.3921672767487391 2024-02-23T00:53:18.352 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight 3 0.3921672767487391 2024-02-23T00:53:18.410 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:53:18.438 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:18.640 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:53:18.642 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:18.642 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:53:18.743 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:53:18.771 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:18.870 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:53:18.873 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:18.873 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:53:18.974 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:53:19.006 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:19.027 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:53:19.029 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:19.030 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:53:19.130 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:53:19.161 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:19.310 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:53:19.314 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:19.314 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:53:19.415 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:53:19.428 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:53:19.428 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 20, 2024-02-23T00:53:19.428 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 600, 2024-02-23T00:53:19.428 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [ 2024-02-23T00:53:19.428 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.429 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(client.15270.0:109 62.17 e137/126 62:eabbf5ab:test-rados-api-smithi047-82465-10::foo:head v 137'1, mlcod=0'0)", 2024-02-23T00:53:19.429 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:28.563970+0000", 2024-02-23T00:53:19.429 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 590.86034108199999, 2024-02-23T00:53:19.429 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.080085445000000005, 2024-02-23T00:53:19.429 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:19.429 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:53:19.429 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:19.429 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.429 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:19.429 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:28.563970+0000", 2024-02-23T00:53:19.429 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:19.429 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.429 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.430 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:19.430 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:28.563970+0000", 2024-02-23T00:53:19.430 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:53:19.430 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.430 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.430 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:19.430 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:28.563970+0000", 2024-02-23T00:53:19.430 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.9320000000000008e-06 2024-02-23T00:53:19.430 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.430 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.430 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:19.430 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:28.563980+0000", 2024-02-23T00:53:19.430 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.4470000000000006e-06 2024-02-23T00:53:19.430 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.431 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.431 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:19.431 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:28.563988+0000", 2024-02-23T00:53:19.431 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.7030000000000006e-06 2024-02-23T00:53:19.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.431 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.431 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:19.431 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:28.563996+0000", 2024-02-23T00:53:19.431 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0013059230000000001 2024-02-23T00:53:19.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.431 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.431 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:19.431 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:28.565302+0000", 2024-02-23T00:53:19.431 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0299999999999999e-05 2024-02-23T00:53:19.432 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.432 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.432 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:19.432 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:28.565322+0000", 2024-02-23T00:53:19.432 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.078701344000000006 2024-02-23T00:53:19.432 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.432 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.432 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:53:19.432 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:28.644024+0000", 2024-02-23T00:53:19.432 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.2206999999999998e-05 2024-02-23T00:53:19.432 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.432 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.432 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:19.433 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:28.644056+0000", 2024-02-23T00:53:19.433 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.080085445000000005 2024-02-23T00:53:19.433 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.433 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:19.433 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.433 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.433 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.433 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24775.0:2078 20.0 20:0000ebd9:test-rados-api-smithi047-82457-17::big.copy:head [copy-from ver 0] snapc 0=[] ondisk+write+known_if_redirected e139)", 2024-02-23T00:53:19.433 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:32.082505+0000", 2024-02-23T00:53:19.433 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 587.34180618300002, 2024-02-23T00:53:19.433 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.27429739199999997, 2024-02-23T00:53:19.433 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:19.433 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:53:19.433 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:53:19.434 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24775", 2024-02-23T00:53:19.434 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1191937413", 2024-02-23T00:53:19.434 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2078 2024-02-23T00:53:19.434 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.434 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:19.434 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.434 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:19.434 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.082505+0000", 2024-02-23T00:53:19.434 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:19.434 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.434 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.434 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:19.434 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.082505+0000", 2024-02-23T00:53:19.435 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999976 2024-02-23T00:53:19.435 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.435 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.435 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:19.435 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.082503+0000", 2024-02-23T00:53:19.435 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3360000000000001e-05 2024-02-23T00:53:19.435 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.435 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.435 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:19.435 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.082516+0000", 2024-02-23T00:53:19.435 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1208000000000001e-05 2024-02-23T00:53:19.435 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.435 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.435 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:19.436 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.082537+0000", 2024-02-23T00:53:19.436 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1620000000000001e-05 2024-02-23T00:53:19.436 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.436 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.436 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:19.436 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.082549+0000", 2024-02-23T00:53:19.436 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.1419000000000003e-05 2024-02-23T00:53:19.436 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.436 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.436 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:19.436 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.082610+0000", 2024-02-23T00:53:19.436 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.3010000000000006e-05 2024-02-23T00:53:19.436 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.436 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.437 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:19.437 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.082703+0000", 2024-02-23T00:53:19.437 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.22105509400000001 2024-02-23T00:53:19.437 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.437 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.437 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 0,7", 2024-02-23T00:53:19.437 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.303759+0000", 2024-02-23T00:53:19.437 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.005574959 2024-02-23T00:53:19.437 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.437 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.437 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:53:19.437 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.309333+0000", 2024-02-23T00:53:19.437 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0011910919999999999 2024-02-23T00:53:19.437 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.437 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.438 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-02-23T00:53:19.438 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.310525+0000", 2024-02-23T00:53:19.438 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.046185191 2024-02-23T00:53:19.438 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.438 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.438 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:53:19.438 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.356710+0000", 2024-02-23T00:53:19.438 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.4830999999999997e-05 2024-02-23T00:53:19.438 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.438 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.438 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:53:19.438 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.356765+0000", 2024-02-23T00:53:19.438 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.8046999999999997e-05 2024-02-23T00:53:19.438 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.439 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.439 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:19.439 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.356803+0000", 2024-02-23T00:53:19.439 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.27429739199999997 2024-02-23T00:53:19.439 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.439 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:19.439 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.439 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.439 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.439 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.15270.0:110 75.17 75:eabbf5ab:test-rados-api-smithi047-82465-10::foo:head [read 0~1] snapc 0=[] ondisk+read+known_if_redirected e140)", 2024-02-23T00:53:19.439 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:32.203593+0000", 2024-02-23T00:53:19.439 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 587.22071825900002, 2024-02-23T00:53:19.439 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.11083016499999999, 2024-02-23T00:53:19.439 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:19.440 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:53:19.440 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:53:19.440 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.15270", 2024-02-23T00:53:19.440 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/4133111990", 2024-02-23T00:53:19.440 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 110 2024-02-23T00:53:19.440 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.440 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:19.440 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.440 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:19.440 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.203593+0000", 2024-02-23T00:53:19.440 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:19.440 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.440 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.440 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:19.441 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.203593+0000", 2024-02-23T00:53:19.441 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999971 2024-02-23T00:53:19.441 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.441 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.441 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:19.441 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.203590+0000", 2024-02-23T00:53:19.441 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3745e-05 2024-02-23T00:53:19.441 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.441 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.441 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:19.441 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.203604+0000", 2024-02-23T00:53:19.441 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6659000000000001e-05 2024-02-23T00:53:19.441 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.441 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.442 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:19.442 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.203621+0000", 2024-02-23T00:53:19.442 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.7660000000000004e-06 2024-02-23T00:53:19.442 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.442 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.442 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:19.442 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.203630+0000", 2024-02-23T00:53:19.442 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.2704000000000002e-05 2024-02-23T00:53:19.442 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.442 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.442 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:19.442 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.203682+0000", 2024-02-23T00:53:19.442 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.9869999999999996e-06 2024-02-23T00:53:19.442 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.442 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.443 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-02-23T00:53:19.443 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.203688+0000", 2024-02-23T00:53:19.443 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.027028343 2024-02-23T00:53:19.443 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.443 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.443 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:19.443 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.230717+0000", 2024-02-23T00:53:19.443 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00018909299999999999 2024-02-23T00:53:19.443 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.443 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.443 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for blocked object", 2024-02-23T00:53:19.443 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.230906+0000", 2024-02-23T00:53:19.443 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.068127912999999998 2024-02-23T00:53:19.443 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.443 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.443 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:19.443 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.299034+0000", 2024-02-23T00:53:19.444 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000158457 2024-02-23T00:53:19.444 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.444 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.444 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for rw locks", 2024-02-23T00:53:19.444 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.299192+0000", 2024-02-23T00:53:19.444 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.014964832000000001 2024-02-23T00:53:19.444 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.444 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.444 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:19.444 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.314157+0000", 2024-02-23T00:53:19.444 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000107994 2024-02-23T00:53:19.444 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.444 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.445 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:19.445 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.314265+0000", 2024-02-23T00:53:19.445 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00015843500000000001 2024-02-23T00:53:19.445 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.445 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.445 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:19.445 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.314423+0000", 2024-02-23T00:53:19.445 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.11083016499999999 2024-02-23T00:53:19.445 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.445 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:19.445 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.445 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.445 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.445 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_repop(osd.1.0:294 20.6 e140/109 -22:61acad7d:::temp_20.6_0_24170_1:head v 0'0, mlcod=112'6)", 2024-02-23T00:53:19.445 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:43:32.415893+0000", 2024-02-23T00:53:19.446 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 587.008418678, 2024-02-23T00:53:19.446 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.041150669000000001, 2024-02-23T00:53:19.446 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:19.446 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:53:19.446 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:19.446 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.446 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:19.446 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.415893+0000", 2024-02-23T00:53:19.446 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:19.446 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.446 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.446 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:19.446 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.415893+0000", 2024-02-23T00:53:19.446 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999976 2024-02-23T00:53:19.446 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.446 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.447 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:19.447 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.415890+0000", 2024-02-23T00:53:19.447 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0044047399999999999 2024-02-23T00:53:19.447 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.447 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.447 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:19.447 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.420295+0000", 2024-02-23T00:53:19.447 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.7456000000000001e-05 2024-02-23T00:53:19.447 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.447 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.447 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:19.447 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.420343+0000", 2024-02-23T00:53:19.447 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5447e-05 2024-02-23T00:53:19.447 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.447 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.447 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:19.447 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.420358+0000", 2024-02-23T00:53:19.448 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000149753 2024-02-23T00:53:19.448 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.448 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.448 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:19.448 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.420508+0000", 2024-02-23T00:53:19.448 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.9539000000000002e-05 2024-02-23T00:53:19.448 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.448 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.448 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:19.448 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.420547+0000", 2024-02-23T00:53:19.448 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.036411805999999998 2024-02-23T00:53:19.448 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.448 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.448 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:53:19.449 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.456959+0000", 2024-02-23T00:53:19.449 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.4338000000000005e-05 2024-02-23T00:53:19.449 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.449 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.449 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:19.449 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:43:32.457043+0000", 2024-02-23T00:53:19.449 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.041150669000000001 2024-02-23T00:53:19.449 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.449 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:19.449 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.449 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.449 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.449 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.40652.0:1 131.5s0 131:b7c88859:test-rados-api-smithi047-82457-37::foo:head [write 0~4] snapc 0=[] ondisk+write+known_if_redirected e198)", 2024-02-23T00:53:19.449 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:30.140263+0000", 2024-02-23T00:53:19.449 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 529.28404824300003, 2024-02-23T00:53:19.450 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.060159552, 2024-02-23T00:53:19.450 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:19.450 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:53:19.450 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:53:19.450 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.40652", 2024-02-23T00:53:19.450 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/2935214693", 2024-02-23T00:53:19.450 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 1 2024-02-23T00:53:19.450 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.450 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:19.450 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.450 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:19.450 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:30.140263+0000", 2024-02-23T00:53:19.450 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:19.450 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.450 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.451 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:19.451 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:30.140263+0000", 2024-02-23T00:53:19.451 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:53:19.451 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.451 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.451 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:19.451 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:30.140261+0000", 2024-02-23T00:53:19.451 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.7969999999999997e-06 2024-02-23T00:53:19.451 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.451 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.451 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:19.451 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:30.140267+0000", 2024-02-23T00:53:19.451 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.1540000000000008e-06 2024-02-23T00:53:19.451 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.452 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.452 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:19.452 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:30.140276+0000", 2024-02-23T00:53:19.452 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.166e-06 2024-02-23T00:53:19.452 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.452 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.452 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:19.452 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:30.140285+0000", 2024-02-23T00:53:19.452 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000120987 2024-02-23T00:53:19.452 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.452 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.452 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:19.453 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:30.140406+0000", 2024-02-23T00:53:19.453 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.523e-06 2024-02-23T00:53:19.453 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.453 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.453 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:53:19.453 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:30.140410+0000", 2024-02-23T00:53:19.453 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.026032182 2024-02-23T00:53:19.453 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.453 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.453 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:19.453 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.166442+0000", 2024-02-23T00:53:19.453 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00011500199999999999 2024-02-23T00:53:19.453 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.453 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.453 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:19.454 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.166557+0000", 2024-02-23T00:53:19.454 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00040118199999999998 2024-02-23T00:53:19.454 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.454 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.454 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-02-23T00:53:19.454 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.166958+0000", 2024-02-23T00:53:19.454 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001243271 2024-02-23T00:53:19.454 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.454 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.454 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-02-23T00:53:19.454 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.168202+0000", 2024-02-23T00:53:19.454 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.032140427999999999 2024-02-23T00:53:19.454 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.455 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.455 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:53:19.455 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.200342+0000", 2024-02-23T00:53:19.455 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0808000000000003e-05 2024-02-23T00:53:19.455 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.455 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.455 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:19.455 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:31.200423+0000", 2024-02-23T00:53:19.455 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.060159552 2024-02-23T00:53:19.455 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.455 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:19.455 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.455 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.455 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.456 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.40748.0:1 146.6s0 146:602f83fe:::foo:head [append 0~8192] snapc 0=[] ondisk+write+known_if_redirected e222)", 2024-02-23T00:53:19.456 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:44:54.153497+0000", 2024-02-23T00:53:19.456 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 505.270814433, 2024-02-23T00:53:19.456 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.99506503800000001, 2024-02-23T00:53:19.456 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:19.456 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:53:19.456 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:53:19.456 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.40748", 2024-02-23T00:53:19.456 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/2606886447", 2024-02-23T00:53:19.456 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 1 2024-02-23T00:53:19.456 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.456 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:19.457 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.457 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:19.457 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.153497+0000", 2024-02-23T00:53:19.457 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:19.457 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.457 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.457 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:19.457 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.153497+0000", 2024-02-23T00:53:19.457 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:53:19.457 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.457 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.457 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:19.457 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.153495+0000", 2024-02-23T00:53:19.457 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5336e-05 2024-02-23T00:53:19.458 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.458 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.458 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:19.458 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.153511+0000", 2024-02-23T00:53:19.458 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2704000000000001e-05 2024-02-23T00:53:19.458 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.458 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.458 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:19.458 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.153523+0000", 2024-02-23T00:53:19.458 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0260000000000007e-06 2024-02-23T00:53:19.458 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.458 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.458 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:19.458 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.153531+0000", 2024-02-23T00:53:19.459 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.4209000000000002e-05 2024-02-23T00:53:19.459 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.459 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.459 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:19.459 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.153585+0000", 2024-02-23T00:53:19.459 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.168e-06 2024-02-23T00:53:19.459 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.459 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.459 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:53:19.459 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:54.153589+0000", 2024-02-23T00:53:19.459 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.98702179099999998 2024-02-23T00:53:19.459 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.459 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.459 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:19.459 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:55.140610+0000", 2024-02-23T00:53:19.460 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00013427 2024-02-23T00:53:19.460 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.460 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.460 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:19.460 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:55.140745+0000", 2024-02-23T00:53:19.460 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00049049300000000005 2024-02-23T00:53:19.460 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.460 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.460 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-02-23T00:53:19.460 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:55.141235+0000", 2024-02-23T00:53:19.460 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00080861699999999997 2024-02-23T00:53:19.460 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.460 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.461 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-02-23T00:53:19.461 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:55.142044+0000", 2024-02-23T00:53:19.461 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0064437599999999998 2024-02-23T00:53:19.461 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.461 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.461 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:53:19.461 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:55.148488+0000", 2024-02-23T00:53:19.461 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.4482999999999998e-05 2024-02-23T00:53:19.461 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.461 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.461 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:19.461 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:44:55.148562+0000", 2024-02-23T00:53:19.461 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.99506503800000001 2024-02-23T00:53:19.461 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.462 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:19.462 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.462 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.462 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.462 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.40841.0:1 159.5s0 159:ab97ba35:::nonexistent:head [read 0~128] snapc 0=[] ondisk+read+known_if_redirected e243)", 2024-02-23T00:53:19.462 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:45:15.601155+0000", 2024-02-23T00:53:19.462 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 483.82315599499998, 2024-02-23T00:53:19.462 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.54852251600000002, 2024-02-23T00:53:19.462 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:19.462 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-02-23T00:53:19.462 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:53:19.462 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.40841", 2024-02-23T00:53:19.462 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1302226360", 2024-02-23T00:53:19.462 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 1 2024-02-23T00:53:19.463 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.463 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:19.463 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.463 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:19.463 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:15.601155+0000", 2024-02-23T00:53:19.463 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:19.463 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.463 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.463 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:19.463 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:15.601155+0000", 2024-02-23T00:53:19.463 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999971 2024-02-23T00:53:19.463 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.463 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.463 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:19.464 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:15.601153+0000", 2024-02-23T00:53:19.464 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.3654000000000001e-05 2024-02-23T00:53:19.464 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.464 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.464 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:19.464 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:15.601176+0000", 2024-02-23T00:53:19.464 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4418e-05 2024-02-23T00:53:19.464 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.464 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.464 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:19.464 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:15.601191+0000", 2024-02-23T00:53:19.464 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.3100000000000006e-06 2024-02-23T00:53:19.465 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.465 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.465 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:19.465 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:15.601200+0000", 2024-02-23T00:53:19.465 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.0349000000000001e-05 2024-02-23T00:53:19.465 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.465 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.465 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:19.465 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:15.601260+0000", 2024-02-23T00:53:19.465 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.45e-06 2024-02-23T00:53:19.465 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.465 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.465 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:53:19.465 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:15.601264+0000", 2024-02-23T00:53:19.466 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.54829864299999997 2024-02-23T00:53:19.466 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.466 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.466 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:19.466 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:16.149562+0000", 2024-02-23T00:53:19.466 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000115525 2024-02-23T00:53:19.466 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.466 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.466 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:19.466 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:16.149678+0000", 2024-02-23T00:53:19.466 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.54852251600000002 2024-02-23T00:53:19.466 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.466 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:19.466 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.466 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.467 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.467 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.30609.0:1 171.6s0 171:602f83fe:::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e271)", 2024-02-23T00:53:19.467 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:45:43.152924+0000", 2024-02-23T00:53:19.467 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 456.27138709799999, 2024-02-23T00:53:19.467 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.024487457, 2024-02-23T00:53:19.467 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:19.467 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:53:19.467 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:53:19.467 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.30609", 2024-02-23T00:53:19.467 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/1534291680", 2024-02-23T00:53:19.467 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 1 2024-02-23T00:53:19.467 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.467 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:19.467 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.467 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:19.468 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:43.152924+0000", 2024-02-23T00:53:19.468 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:19.468 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.468 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.468 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:19.468 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:43.152924+0000", 2024-02-23T00:53:19.468 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:53:19.468 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.468 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.468 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:19.468 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:43.152923+0000", 2024-02-23T00:53:19.468 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.7919999999999993e-06 2024-02-23T00:53:19.468 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.468 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.469 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:19.469 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:43.152930+0000", 2024-02-23T00:53:19.469 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.096e-05 2024-02-23T00:53:19.469 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.469 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.469 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:19.469 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:43.152941+0000", 2024-02-23T00:53:19.469 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8970000000000008e-06 2024-02-23T00:53:19.469 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.469 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.469 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:19.469 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:43.152949+0000", 2024-02-23T00:53:19.469 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.095e-05 2024-02-23T00:53:19.469 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.470 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.470 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:19.470 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:43.152980+0000", 2024-02-23T00:53:19.470 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7560000000000001e-06 2024-02-23T00:53:19.470 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.470 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.470 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:53:19.470 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:43.152984+0000", 2024-02-23T00:53:19.470 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0192820869999999 2024-02-23T00:53:19.470 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.470 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.470 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:19.470 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:44.172266+0000", 2024-02-23T00:53:19.470 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012433600000000001 2024-02-23T00:53:19.471 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.471 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.471 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:19.471 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:44.172390+0000", 2024-02-23T00:53:19.471 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00040993399999999998 2024-02-23T00:53:19.471 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.471 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.471 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-02-23T00:53:19.471 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:44.172800+0000", 2024-02-23T00:53:19.471 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000399152 2024-02-23T00:53:19.471 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.471 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.471 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-02-23T00:53:19.471 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:44.173199+0000", 2024-02-23T00:53:19.472 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.004146327 2024-02-23T00:53:19.472 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.472 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.472 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:53:19.472 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:44.177346+0000", 2024-02-23T00:53:19.472 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.6025000000000006e-05 2024-02-23T00:53:19.472 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.472 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.472 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:19.472 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:45:44.177412+0000", 2024-02-23T00:53:19.472 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.024487457 2024-02-23T00:53:19.472 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.472 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:19.473 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.473 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.473 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.473 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.41033.0:1 182.0s0 182:1e0071ee:::oid_RoundTripSparseReadPP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e297)", 2024-02-23T00:53:19.473 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:46:09.162867+0000", 2024-02-23T00:53:19.473 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 430.26144422700003, 2024-02-23T00:53:19.473 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.99942831600000004, 2024-02-23T00:53:19.473 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:19.473 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:53:19.473 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:53:19.473 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.41033", 2024-02-23T00:53:19.473 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/2123205582", 2024-02-23T00:53:19.473 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 1 2024-02-23T00:53:19.473 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.473 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:19.474 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.474 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:19.474 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:09.162867+0000", 2024-02-23T00:53:19.474 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:19.474 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.474 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.474 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:19.474 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:09.162867+0000", 2024-02-23T00:53:19.474 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:53:19.474 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.474 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.474 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:19.474 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:09.162865+0000", 2024-02-23T00:53:19.474 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.8800000000000002e-06 2024-02-23T00:53:19.475 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.475 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.475 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:19.475 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:09.162872+0000", 2024-02-23T00:53:19.475 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3331e-05 2024-02-23T00:53:19.475 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.475 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.475 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:19.475 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:09.162886+0000", 2024-02-23T00:53:19.475 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.225e-06 2024-02-23T00:53:19.475 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.475 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.475 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:19.475 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:09.162894+0000", 2024-02-23T00:53:19.476 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.4615000000000003e-05 2024-02-23T00:53:19.476 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.476 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.476 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:19.476 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:09.162969+0000", 2024-02-23T00:53:19.476 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.9339999999999999e-06 2024-02-23T00:53:19.476 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.476 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.476 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:53:19.476 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:09.162972+0000", 2024-02-23T00:53:19.476 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.99772507799999999 2024-02-23T00:53:19.476 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.476 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.477 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:19.477 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:10.160698+0000", 2024-02-23T00:53:19.477 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014021500000000001 2024-02-23T00:53:19.477 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.477 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.477 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:19.477 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:10.160838+0000", 2024-02-23T00:53:19.477 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00041179999999999998 2024-02-23T00:53:19.477 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.477 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.477 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-02-23T00:53:19.477 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:10.161250+0000", 2024-02-23T00:53:19.477 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00043617599999999998 2024-02-23T00:53:19.477 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.477 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.478 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-02-23T00:53:19.478 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:10.161686+0000", 2024-02-23T00:53:19.478 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00056253300000000004 2024-02-23T00:53:19.478 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.478 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.478 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-02-23T00:53:19.478 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:10.162248+0000", 2024-02-23T00:53:19.478 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.7240000000000002e-05 2024-02-23T00:53:19.478 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.478 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.478 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:19.478 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:46:10.162296+0000", 2024-02-23T00:53:19.478 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.99942831600000004 2024-02-23T00:53:19.478 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.478 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:19.479 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.479 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.479 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.479 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24311.0:1279 5.14 5:2b04a3e9:::notify.6:head [watch ping cookie 94587070404224] snapc 0=[] ondisk+write+known_if_redirected e370)", 2024-02-23T00:53:19.479 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:47:27.941496+0000", 2024-02-23T00:53:19.479 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 351.48281593199999, 2024-02-23T00:53:19.479 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.30738969700000002, 2024-02-23T00:53:19.479 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:19.479 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:53:19.479 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:53:19.479 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24311", 2024-02-23T00:53:19.479 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/3723202188", 2024-02-23T00:53:19.479 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 1279 2024-02-23T00:53:19.479 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.480 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:19.480 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.480 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:19.480 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:27.941496+0000", 2024-02-23T00:53:19.480 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:19.480 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.480 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.480 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:19.480 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:27.941496+0000", 2024-02-23T00:53:19.480 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:53:19.480 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.480 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.480 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:19.480 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:27.941495+0000", 2024-02-23T00:53:19.481 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7960000000000002e-06 2024-02-23T00:53:19.481 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.481 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.481 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:19.481 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:27.941499+0000", 2024-02-23T00:53:19.481 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.5870000000000004e-06 2024-02-23T00:53:19.481 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.481 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.481 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:19.481 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:27.941506+0000", 2024-02-23T00:53:19.481 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.9089999999999999e-06 2024-02-23T00:53:19.481 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.481 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.481 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:19.482 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:27.941511+0000", 2024-02-23T00:53:19.482 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000105887 2024-02-23T00:53:19.482 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.482 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.482 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:19.482 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:27.941617+0000", 2024-02-23T00:53:19.482 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5998000000000002e-05 2024-02-23T00:53:19.482 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.482 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.482 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:53:19.482 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:27.941633+0000", 2024-02-23T00:53:19.482 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.28945720200000002 2024-02-23T00:53:19.482 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.482 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.483 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:19.483 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:28.231090+0000", 2024-02-23T00:53:19.483 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00048983700000000002 2024-02-23T00:53:19.483 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.483 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.483 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for degraded object", 2024-02-23T00:53:19.483 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:28.231580+0000", 2024-02-23T00:53:19.483 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.017133735000000001 2024-02-23T00:53:19.483 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.483 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.483 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:19.483 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:28.248714+0000", 2024-02-23T00:53:19.483 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.1651999999999997e-05 2024-02-23T00:53:19.483 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.484 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.484 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:19.484 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:28.248795+0000", 2024-02-23T00:53:19.484 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.9703000000000005e-05 2024-02-23T00:53:19.484 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.484 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.484 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:19.484 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:28.248885+0000", 2024-02-23T00:53:19.484 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.30738969700000002 2024-02-23T00:53:19.484 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.484 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:19.484 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.484 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.485 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.485 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24311.0:1286 5.14 5:2b04a3e9:::notify.6:head [watch reconnect cookie 94587070404224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e375)", 2024-02-23T00:53:19.485 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:47:27.942597+0000", 2024-02-23T00:53:19.485 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 351.48171481000003, 2024-02-23T00:53:19.485 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.31510674500000002, 2024-02-23T00:53:19.485 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:19.485 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:53:19.485 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:53:19.485 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24311", 2024-02-23T00:53:19.485 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/3723202188", 2024-02-23T00:53:19.485 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 1286 2024-02-23T00:53:19.485 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.485 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:19.485 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.486 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:19.486 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:27.942597+0000", 2024-02-23T00:53:19.486 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:19.486 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.486 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.486 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:19.486 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:27.942597+0000", 2024-02-23T00:53:19.486 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.999999 2024-02-23T00:53:19.486 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.486 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.486 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:19.486 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:27.942596+0000", 2024-02-23T00:53:19.486 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.203e-06 2024-02-23T00:53:19.486 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.487 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.487 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:19.487 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:27.942601+0000", 2024-02-23T00:53:19.487 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.065e-05 2024-02-23T00:53:19.487 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.487 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.487 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:19.487 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:27.942612+0000", 2024-02-23T00:53:19.487 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.8669999999999999e-06 2024-02-23T00:53:19.487 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.487 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.487 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:19.487 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:27.942617+0000", 2024-02-23T00:53:19.487 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.6706999999999999e-05 2024-02-23T00:53:19.487 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.488 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.488 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:19.488 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:27.942654+0000", 2024-02-23T00:53:19.488 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.6640000000000003e-06 2024-02-23T00:53:19.488 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.488 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.488 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-02-23T00:53:19.488 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:27.942660+0000", 2024-02-23T00:53:19.488 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.29652368600000001 2024-02-23T00:53:19.488 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.488 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.488 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:19.488 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:28.239184+0000", 2024-02-23T00:53:19.488 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000101123 2024-02-23T00:53:19.488 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.489 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.489 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for degraded object", 2024-02-23T00:53:19.489 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:28.239285+0000", 2024-02-23T00:53:19.489 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0096292229999999993 2024-02-23T00:53:19.489 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.489 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.489 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:19.489 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:28.248914+0000", 2024-02-23T00:53:19.489 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.7242000000000004e-05 2024-02-23T00:53:19.489 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.489 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.489 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:19.489 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:28.248981+0000", 2024-02-23T00:53:19.489 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0087222510000000003 2024-02-23T00:53:19.489 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.490 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.490 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:19.490 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:47:28.257703+0000", 2024-02-23T00:53:19.490 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.31510674500000002 2024-02-23T00:53:19.490 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.490 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:19.490 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.490 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.490 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.490 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDScrubReserve(4.4 REQUEST e644)", 2024-02-23T00:53:19.490 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:53:02.412289+0000", 2024-02-23T00:53:19.490 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 17.012022755, 2024-02-23T00:53:19.490 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00039617800000000001, 2024-02-23T00:53:19.490 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:19.490 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:53:19.490 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:19.491 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.491 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:19.491 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:02.412289+0000", 2024-02-23T00:53:19.491 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:19.491 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.491 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.491 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:19.491 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:02.412289+0000", 2024-02-23T00:53:19.491 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:53:19.491 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.491 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.491 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:19.491 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:02.412287+0000", 2024-02-23T00:53:19.492 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3736e-05 2024-02-23T00:53:19.492 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.492 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.492 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:19.492 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:02.412300+0000", 2024-02-23T00:53:19.492 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.7577e-05 2024-02-23T00:53:19.492 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.492 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.492 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:19.492 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:02.412328+0000", 2024-02-23T00:53:19.492 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2699e-05 2024-02-23T00:53:19.492 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.492 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.492 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:19.493 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:02.412341+0000", 2024-02-23T00:53:19.493 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000247562 2024-02-23T00:53:19.493 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.493 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.493 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:19.493 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:02.412588+0000", 2024-02-23T00:53:19.493 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.8087e-05 2024-02-23T00:53:19.493 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.493 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.493 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:19.493 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:02.412656+0000", 2024-02-23T00:53:19.493 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8574000000000001e-05 2024-02-23T00:53:19.493 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.494 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.494 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:19.494 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:02.412685+0000", 2024-02-23T00:53:19.494 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00039617800000000001 2024-02-23T00:53:19.494 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.494 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:19.494 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.494 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.494 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.494 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24311.0:2010 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 94587070398464] snapc 0=[] ondisk+write+known_if_redirected e640)", 2024-02-23T00:53:19.494 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:53:03.001326+0000", 2024-02-23T00:53:19.494 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 16.422985220000001, 2024-02-23T00:53:19.494 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00034448200000000001, 2024-02-23T00:53:19.494 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:19.495 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:53:19.495 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:53:19.495 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24311", 2024-02-23T00:53:19.495 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/3723202188", 2024-02-23T00:53:19.495 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2010 2024-02-23T00:53:19.495 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.495 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:19.495 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.495 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:19.495 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:03.001326+0000", 2024-02-23T00:53:19.495 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:19.495 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.495 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.496 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:19.496 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:03.001326+0000", 2024-02-23T00:53:19.496 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999981 2024-02-23T00:53:19.496 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.496 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.496 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:19.496 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:03.001324+0000", 2024-02-23T00:53:19.496 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2109e-05 2024-02-23T00:53:19.496 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.496 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.496 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:19.496 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:03.001336+0000", 2024-02-23T00:53:19.496 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.5857e-05 2024-02-23T00:53:19.496 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.497 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.497 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:19.497 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:03.001362+0000", 2024-02-23T00:53:19.497 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.1099999999999992e-06 2024-02-23T00:53:19.497 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.497 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.497 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:19.497 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:03.001371+0000", 2024-02-23T00:53:19.497 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.3342000000000002e-05 2024-02-23T00:53:19.497 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.497 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.497 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:19.497 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:03.001465+0000", 2024-02-23T00:53:19.498 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.9586000000000005e-05 2024-02-23T00:53:19.498 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.498 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.498 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:19.498 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:03.001564+0000", 2024-02-23T00:53:19.498 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000106607 2024-02-23T00:53:19.498 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.498 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.498 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:19.498 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:03.001671+0000", 2024-02-23T00:53:19.498 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00034448200000000001 2024-02-23T00:53:19.498 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.498 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:19.498 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.498 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.498 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.499 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24311.0:2011 5.14 5:2b04a3e9:::notify.6:head [watch ping cookie 94587070404224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e640)", 2024-02-23T00:53:19.499 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:53:03.001394+0000", 2024-02-23T00:53:19.499 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 16.422917024, 2024-02-23T00:53:19.499 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000274555, 2024-02-23T00:53:19.499 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:19.499 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:53:19.499 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:53:19.499 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24311", 2024-02-23T00:53:19.499 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/3723202188", 2024-02-23T00:53:19.499 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2011 2024-02-23T00:53:19.499 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.499 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:19.499 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.499 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:19.500 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:03.001394+0000", 2024-02-23T00:53:19.500 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:19.500 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.500 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.500 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:19.500 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:03.001394+0000", 2024-02-23T00:53:19.500 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999995 2024-02-23T00:53:19.500 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.500 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.500 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:19.500 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:03.001394+0000", 2024-02-23T00:53:19.500 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7890000000000001e-06 2024-02-23T00:53:19.500 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.501 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.501 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:19.501 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:03.001398+0000", 2024-02-23T00:53:19.501 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.473e-06 2024-02-23T00:53:19.501 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.501 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.501 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:19.501 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:03.001406+0000", 2024-02-23T00:53:19.501 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.8300000000000003e-06 2024-02-23T00:53:19.501 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.501 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.501 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:19.501 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:03.001411+0000", 2024-02-23T00:53:19.501 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.3613999999999999e-05 2024-02-23T00:53:19.501 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.502 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.502 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:19.502 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:03.001464+0000", 2024-02-23T00:53:19.502 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.7380000000000006e-05 2024-02-23T00:53:19.502 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.502 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.502 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:19.502 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:03.001562+0000", 2024-02-23T00:53:19.502 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000107086 2024-02-23T00:53:19.502 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.502 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.502 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:19.502 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:03.001669+0000", 2024-02-23T00:53:19.503 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000274555 2024-02-23T00:53:19.503 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.503 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:19.503 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.503 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.503 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.503 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24311.0:2040 5.0 5:05bf5b68:::notify.1:head [watch ping cookie 94587070398464] snapc 0=[] ondisk+write+known_if_redirected e656)", 2024-02-23T00:53:19.503 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:53:18.004435+0000", 2024-02-23T00:53:19.503 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 1.41987643, 2024-02-23T00:53:19.503 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00035349300000000003, 2024-02-23T00:53:19.503 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:19.503 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:53:19.503 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:53:19.503 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24311", 2024-02-23T00:53:19.504 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/3723202188", 2024-02-23T00:53:19.504 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2040 2024-02-23T00:53:19.504 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.504 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:19.504 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.504 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:19.504 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.004435+0000", 2024-02-23T00:53:19.504 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:19.504 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.504 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.504 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:19.504 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.004435+0000", 2024-02-23T00:53:19.504 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.9999957 2024-02-23T00:53:19.505 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.505 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.505 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:19.505 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.004431+0000", 2024-02-23T00:53:19.505 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6523000000000001e-05 2024-02-23T00:53:19.505 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.505 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.505 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:19.505 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.004447+0000", 2024-02-23T00:53:19.505 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8713000000000001e-05 2024-02-23T00:53:19.505 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.505 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.505 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:19.505 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.004476+0000", 2024-02-23T00:53:19.505 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.1009999999999998e-06 2024-02-23T00:53:19.506 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.506 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.506 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:19.506 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.004485+0000", 2024-02-23T00:53:19.506 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000109747 2024-02-23T00:53:19.506 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.506 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.506 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:19.506 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.004595+0000", 2024-02-23T00:53:19.506 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.6954999999999998e-05 2024-02-23T00:53:19.506 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.506 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.506 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:19.507 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.004672+0000", 2024-02-23T00:53:19.507 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000116845 2024-02-23T00:53:19.507 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.507 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.507 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:19.507 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.004788+0000", 2024-02-23T00:53:19.507 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00035349300000000003 2024-02-23T00:53:19.507 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.507 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:19.507 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.507 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.507 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.507 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.24311.0:2041 5.14 5:2b04a3e9:::notify.6:head [watch ping cookie 94587070404224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e656)", 2024-02-23T00:53:19.507 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:53:18.004514+0000", 2024-02-23T00:53:19.508 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 1.4197977399999999, 2024-02-23T00:53:19.508 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000262453, 2024-02-23T00:53:19.508 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:19.508 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-02-23T00:53:19.508 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-02-23T00:53:19.508 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.24311", 2024-02-23T00:53:19.508 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.47:0/3723202188", 2024-02-23T00:53:19.508 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 2041 2024-02-23T00:53:19.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.508 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:19.508 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.508 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:19.508 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.004514+0000", 2024-02-23T00:53:19.508 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:19.509 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.509 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.509 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:19.509 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.004514+0000", 2024-02-23T00:53:19.509 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.999999 2024-02-23T00:53:19.509 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.509 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.509 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:19.509 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.004513+0000", 2024-02-23T00:53:19.509 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.5499999999999996e-06 2024-02-23T00:53:19.509 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.509 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.509 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:19.509 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.004518+0000", 2024-02-23T00:53:19.510 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2616000000000001e-05 2024-02-23T00:53:19.510 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.510 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.510 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:19.510 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.004530+0000", 2024-02-23T00:53:19.510 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.9580000000000004e-06 2024-02-23T00:53:19.510 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.510 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.510 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:19.510 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.004536+0000", 2024-02-23T00:53:19.510 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.8640000000000001e-05 2024-02-23T00:53:19.510 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.510 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.510 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:19.510 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.004595+0000", 2024-02-23T00:53:19.511 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.7263999999999998e-05 2024-02-23T00:53:19.511 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.511 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.511 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-02-23T00:53:19.511 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.004662+0000", 2024-02-23T00:53:19.511 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00011416 2024-02-23T00:53:19.511 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.511 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.511 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:19.511 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.004776+0000", 2024-02-23T00:53:19.511 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000262453 2024-02-23T00:53:19.511 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.511 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:19.511 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.512 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.512 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.512 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWrite(234.3s1 661/600 ECSubWrite(tid=519, reqid=client.41771.0:791, at_version=661'1, trim_to=0'0, roll_forward_to=0'0))", 2024-02-23T00:53:19.512 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:53:18.290946+0000", 2024-02-23T00:53:19.512 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 1.1333658630000001, 2024-02-23T00:53:19.512 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000435358, 2024-02-23T00:53:19.512 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:19.512 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-02-23T00:53:19.512 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:19.512 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.512 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:19.512 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.290946+0000", 2024-02-23T00:53:19.512 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:19.512 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.513 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.513 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:19.513 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.290946+0000", 2024-02-23T00:53:19.513 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967296 2024-02-23T00:53:19.513 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.513 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.513 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:19.513 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.290945+0000", 2024-02-23T00:53:19.513 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0497999999999999e-05 2024-02-23T00:53:19.513 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.513 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.513 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:19.513 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.290966+0000", 2024-02-23T00:53:19.514 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2632e-05 2024-02-23T00:53:19.514 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.514 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.514 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:19.514 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.290978+0000", 2024-02-23T00:53:19.514 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0970000000000006e-06 2024-02-23T00:53:19.514 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.514 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.514 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:19.514 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.290987+0000", 2024-02-23T00:53:19.514 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.5513000000000003e-05 2024-02-23T00:53:19.514 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.514 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.514 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:19.515 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.291032+0000", 2024-02-23T00:53:19.515 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.1260000000000004e-06 2024-02-23T00:53:19.515 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.515 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.515 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-02-23T00:53:19.515 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.291041+0000", 2024-02-23T00:53:19.515 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00031616599999999999 2024-02-23T00:53:19.515 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.515 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.515 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-02-23T00:53:19.515 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.291357+0000", 2024-02-23T00:53:19.515 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.3533e-05 2024-02-23T00:53:19.515 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.515 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.516 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:19.516 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.291381+0000", 2024-02-23T00:53:19.516 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000435358 2024-02-23T00:53:19.516 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.516 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:19.516 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.516 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.516 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.516 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWrite(234.3s1 661/600 ECSubWrite(tid=520, reqid=client.41771.0:791, at_version=0'0, trim_to=0'0, roll_forward_to=661'1))", 2024-02-23T00:53:19.516 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:53:18.305457+0000", 2024-02-23T00:53:19.516 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 1.1188548540000001, 2024-02-23T00:53:19.516 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00061581799999999996, 2024-02-23T00:53:19.516 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:19.516 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-02-23T00:53:19.517 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:19.517 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.517 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:19.517 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.305457+0000", 2024-02-23T00:53:19.517 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:19.517 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.517 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.517 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:19.517 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.305457+0000", 2024-02-23T00:53:19.517 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.999999 2024-02-23T00:53:19.517 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.517 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.517 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:19.518 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.305456+0000", 2024-02-23T00:53:19.518 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.342e-05 2024-02-23T00:53:19.518 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.518 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.518 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:19.518 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.305469+0000", 2024-02-23T00:53:19.518 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2683e-05 2024-02-23T00:53:19.518 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.518 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.518 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:19.518 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.305482+0000", 2024-02-23T00:53:19.518 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8140000000000001e-06 2024-02-23T00:53:19.518 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.519 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.519 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:19.519 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.305490+0000", 2024-02-23T00:53:19.519 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00011525499999999999 2024-02-23T00:53:19.519 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.519 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.519 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:19.519 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.305605+0000", 2024-02-23T00:53:19.519 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.112e-05 2024-02-23T00:53:19.519 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.519 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.519 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-02-23T00:53:19.519 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.305616+0000", 2024-02-23T00:53:19.519 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00044046899999999997 2024-02-23T00:53:19.520 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.520 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.520 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-02-23T00:53:19.520 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.306057+0000", 2024-02-23T00:53:19.520 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5835999999999999e-05 2024-02-23T00:53:19.520 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.520 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.520 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:19.520 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.306072+0000", 2024-02-23T00:53:19.520 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00061581799999999996 2024-02-23T00:53:19.520 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.520 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:19.520 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.521 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.521 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.521 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWrite(234.0s2 661/600 ECSubWrite(tid=841, reqid=client.41771.0:794, at_version=661'1, trim_to=0'0, roll_forward_to=0'0))", 2024-02-23T00:53:19.521 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:53:18.313282+0000", 2024-02-23T00:53:19.521 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 1.1110293979999999, 2024-02-23T00:53:19.521 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000421786, 2024-02-23T00:53:19.521 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:19.521 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-02-23T00:53:19.521 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:19.521 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.521 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:19.521 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.313282+0000", 2024-02-23T00:53:19.521 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:19.521 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.522 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.522 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:19.522 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.313282+0000", 2024-02-23T00:53:19.522 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967296 2024-02-23T00:53:19.522 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.522 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.522 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:19.522 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.313282+0000", 2024-02-23T00:53:19.522 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.1130999999999997e-05 2024-02-23T00:53:19.522 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.522 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.522 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:19.522 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.313313+0000", 2024-02-23T00:53:19.522 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.1220000000000001e-06 2024-02-23T00:53:19.522 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.523 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.523 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:19.523 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.313322+0000", 2024-02-23T00:53:19.523 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.9849999999999997e-06 2024-02-23T00:53:19.523 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.523 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.523 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:19.523 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.313327+0000", 2024-02-23T00:53:19.523 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.1118000000000002e-05 2024-02-23T00:53:19.523 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.523 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.523 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:19.523 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.313368+0000", 2024-02-23T00:53:19.523 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.2650000000000002e-06 2024-02-23T00:53:19.524 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.524 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.524 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-02-23T00:53:19.524 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.313374+0000", 2024-02-23T00:53:19.524 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00031532000000000001 2024-02-23T00:53:19.524 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.524 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.524 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-02-23T00:53:19.524 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.313690+0000", 2024-02-23T00:53:19.524 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4045e-05 2024-02-23T00:53:19.524 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.524 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.524 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:19.524 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.313704+0000", 2024-02-23T00:53:19.525 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000421786 2024-02-23T00:53:19.525 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.525 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:19.525 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.525 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.525 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.525 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWrite(234.0s2 661/600 ECSubWrite(tid=842, reqid=client.41771.0:794, at_version=0'0, trim_to=0'0, roll_forward_to=661'1))", 2024-02-23T00:53:19.525 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-02-23T00:53:18.313988+0000", 2024-02-23T00:53:19.525 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 1.1103232329999999, 2024-02-23T00:53:19.525 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00035470800000000002, 2024-02-23T00:53:19.525 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-02-23T00:53:19.525 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-02-23T00:53:19.525 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-02-23T00:53:19.525 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.526 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-02-23T00:53:19.526 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.313988+0000", 2024-02-23T00:53:19.526 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-02-23T00:53:19.526 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.526 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.526 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-02-23T00:53:19.526 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.313988+0000", 2024-02-23T00:53:19.526 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4294967295.999999 2024-02-23T00:53:19.526 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.526 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.526 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-02-23T00:53:19.526 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.313987+0000", 2024-02-23T00:53:19.526 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4297000000000001e-05 2024-02-23T00:53:19.526 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.527 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.527 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-02-23T00:53:19.527 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.314002+0000", 2024-02-23T00:53:19.527 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3295000000000001e-05 2024-02-23T00:53:19.527 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.527 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.527 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-02-23T00:53:19.527 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.314015+0000", 2024-02-23T00:53:19.527 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.4590000000000003e-06 2024-02-23T00:53:19.527 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.527 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.527 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-02-23T00:53:19.527 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.314022+0000", 2024-02-23T00:53:19.527 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.7902999999999999e-05 2024-02-23T00:53:19.528 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.528 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.528 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-02-23T00:53:19.528 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.314069+0000", 2024-02-23T00:53:19.528 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1005000000000001e-05 2024-02-23T00:53:19.528 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.528 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.528 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-02-23T00:53:19.528 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.314080+0000", 2024-02-23T00:53:19.528 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000247204 2024-02-23T00:53:19.528 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.528 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.528 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-02-23T00:53:19.528 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.314328+0000", 2024-02-23T00:53:19.529 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5271999999999999e-05 2024-02-23T00:53:19.529 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-02-23T00:53:19.529 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-02-23T00:53:19.529 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-02-23T00:53:19.529 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-02-23T00:53:18.314343+0000", 2024-02-23T00:53:19.529 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00035470800000000002 2024-02-23T00:53:19.529 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.529 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:19.529 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.529 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-02-23T00:53:19.529 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-02-23T00:53:19.529 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:53:19.530 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:18.283005+0000 mon.a (mon.0) 3299 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:53:19.530 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:18.283209+0000 mon.a (mon.0) 3300 : cluster [DBG] osdmap e661: 8 total, 8 up, 6 in 2024-02-23T00:53:19.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:18.283005+0000 mon.a (mon.0) 3299 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:53:19.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:18.283209+0000 mon.a (mon.0) 3300 : cluster [DBG] osdmap e661: 8 total, 8 up, 6 in 2024-02-23T00:53:19.531 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:19.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.334+0000 7f7a8c84b700 1 -- 172.21.15.47:0/887265335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a8806b0f0 msgr2=0x7f7a8806b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:19.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.334+0000 7f7a8c84b700 1 --2- 172.21.15.47:0/887265335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a8806b0f0 0x7f7a8806b4d0 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 rx=0x7f7a78004870 tx=0x7f7a78009cb0).stop 2024-02-23T00:53:19.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.334+0000 7f7a8c84b700 1 -- 172.21.15.47:0/887265335 shutdown_connections 2024-02-23T00:53:19.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.334+0000 7f7a8c84b700 1 --2- 172.21.15.47:0/887265335 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a881044c0 0x7f7a881068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:19.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.334+0000 7f7a8c84b700 1 --2- 172.21.15.47:0/887265335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a8806b0f0 0x7f7a8806b4d0 unknown :-1 s=CLOSED pgs=2605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:19.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.334+0000 7f7a8c84b700 1 --2- 172.21.15.47:0/887265335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a8806b9c0 0x7f7a88103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:19.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.334+0000 7f7a8c84b700 1 -- 172.21.15.47:0/887265335 >> 172.21.15.47:0/887265335 conn(0x7f7a880f3800 msgr2=0x7f7a880f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:19.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.334+0000 7f7a8c84b700 1 -- 172.21.15.47:0/887265335 shutdown_connections 2024-02-23T00:53:19.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.334+0000 7f7a8c84b700 1 -- 172.21.15.47:0/887265335 wait complete. 2024-02-23T00:53:19.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.334+0000 7f7a8c84b700 1 librados: starting msgr at 2024-02-23T00:53:19.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.334+0000 7f7a8c84b700 1 librados: starting objecter 2024-02-23T00:53:19.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.334+0000 7f7a8c84b700 1 Processor -- start 2024-02-23T00:53:19.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.335+0000 7f7a8c84b700 1 -- start start 2024-02-23T00:53:19.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.335+0000 7f7a8c84b700 1 librados: setting wanted keys 2024-02-23T00:53:19.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.335+0000 7f7a8c84b700 1 librados: calling monclient init 2024-02-23T00:53:19.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.335+0000 7f7a8c84b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a8806b0f0 0x7f7a881101a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:19.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.335+0000 7f7a8c84b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a8806b9c0 0x7f7a88110690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:19.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.335+0000 7f7a8c84b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a881044c0 0x7f7a8810a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:19.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.335+0000 7f7a8c84b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8810a830 con 0x7f7a881044c0 2024-02-23T00:53:19.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.335+0000 7f7a8c84b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8810a9a0 con 0x7f7a8806b0f0 2024-02-23T00:53:19.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.335+0000 7f7a8c84b700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8810ab60 con 0x7f7a8806b9c0 2024-02-23T00:53:19.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.335+0000 7f7a85d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a8806b9c0 0x7f7a88110690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:19.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.335+0000 7f7a86d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a881044c0 0x7f7a8810a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:19.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.335+0000 7f7a85d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a8806b9c0 0x7f7a88110690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:60090/0 (socket says 172.21.15.47:60090) 2024-02-23T00:53:19.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.335+0000 7f7a86d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a881044c0 0x7f7a8810a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49872/0 (socket says 172.21.15.47:49872) 2024-02-23T00:53:19.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.335+0000 7f7a85d9b700 1 -- 172.21.15.47:0/4275810160 learned_addr learned my addr 172.21.15.47:0/4275810160 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:53:19.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.335+0000 7f7a8659c700 1 --2- 172.21.15.47:0/4275810160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a8806b0f0 0x7f7a881101a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:19.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.335+0000 7f7a86d9d700 1 -- 172.21.15.47:0/4275810160 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a8806b9c0 msgr2=0x7f7a88110690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:19.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.335+0000 7f7a86d9d700 1 --2- 172.21.15.47:0/4275810160 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a8806b9c0 0x7f7a88110690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:19.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.335+0000 7f7a86d9d700 1 -- 172.21.15.47:0/4275810160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a8806b0f0 msgr2=0x7f7a881101a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:19.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.335+0000 7f7a86d9d700 1 --2- 172.21.15.47:0/4275810160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a8806b0f0 0x7f7a881101a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:19.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.335+0000 7f7a86d9d700 1 -- 172.21.15.47:0/4275810160 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a800097c0 con 0x7f7a881044c0 2024-02-23T00:53:19.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.335+0000 7f7a86d9d700 1 --2- 172.21.15.47:0/4275810160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a881044c0 0x7f7a8810a340 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 rx=0x7f7a8805fed0 tx=0x7f7a80004c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:19.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.335+0000 7f7a777fe700 10 client.?.objecter ms_handle_connect 0x7f7a881044c0 2024-02-23T00:53:19.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.335+0000 7f7a777fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:53:19.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.335+0000 7f7a777fe700 1 -- 172.21.15.47:0/4275810160 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a800079b0 con 0x7f7a881044c0 2024-02-23T00:53:19.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.336+0000 7f7a8c84b700 1 -- 172.21.15.47:0/4275810160 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a78005040 con 0x7f7a881044c0 2024-02-23T00:53:19.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.336+0000 7f7a777fe700 1 -- 172.21.15.47:0/4275810160 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f7a800090d0 con 0x7f7a881044c0 2024-02-23T00:53:19.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.336+0000 7f7a777fe700 1 -- 172.21.15.47:0/4275810160 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8000fc30 con 0x7f7a881044c0 2024-02-23T00:53:19.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.336+0000 7f7a8c84b700 10 client.31554.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:53:19.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.336+0000 7f7a8c84b700 1 -- 172.21.15.47:0/4275810160 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a8810b1a0 con 0x7f7a881044c0 2024-02-23T00:53:19.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.336+0000 7f7a8c84b700 1 librados: init done 2024-02-23T00:53:19.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.337+0000 7f7a8c84b700 1 -- 172.21.15.47:0/4275810160 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a8805ee80 con 0x7f7a881044c0 2024-02-23T00:53:19.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.337+0000 7f7a777fe700 1 -- 172.21.15.47:0/4275810160 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f7a8000fd90 con 0x7f7a881044c0 2024-02-23T00:53:19.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.338+0000 7f7a777fe700 1 --2- 172.21.15.47:0/4275810160 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7a70062cc0 0x7f7a70065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:19.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.338+0000 7f7a777fe700 1 -- 172.21.15.47:0/4275810160 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(662..662 src has 482..662) v4 ==== 9610+0+0 (secure 0 0 0) 0x7f7a8007e3c0 con 0x7f7a881044c0 2024-02-23T00:53:19.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.338+0000 7f7a777fe700 10 client.31554.objecter ms_dispatch 0x7f7a88001550 osd_map(662..662 src has 482..662) v4 2024-02-23T00:53:19.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.338+0000 7f7a777fe700 3 client.31554.objecter handle_osd_map got epochs [662,662] > 0 2024-02-23T00:53:19.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.338+0000 7f7a777fe700 3 client.31554.objecter handle_osd_map decoding full epoch 662 2024-02-23T00:53:19.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.338+0000 7f7a8659c700 1 --2- 172.21.15.47:0/4275810160 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7a70062cc0 0x7f7a70065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:19.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.338+0000 7f7a777fe700 20 client.31554.objecter dump_active .. 0 homeless 2024-02-23T00:53:19.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.338+0000 7f7a777fe700 10 client.31554.objecter ms_handle_connect 0x7f7a70062cc0 2024-02-23T00:53:19.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.338+0000 7f7a8659c700 1 --2- 172.21.15.47:0/4275810160 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7a70062cc0 0x7f7a70065180 secure :-1 s=READY pgs=3155 cs=0 l=1 rev1=1 rx=0x7f7a78000e50 tx=0x7f7a7800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:19.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.342+0000 7f7a777fe700 1 -- 172.21.15.47:0/4275810160 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f7a8004d8d0 con 0x7f7a881044c0 2024-02-23T00:53:19.536 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:53:19.536 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:19.537 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:53:19.537 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:53:19.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:19.532+0000 7f7a8c84b700 1 -- 172.21.15.47:0/4275810160 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7f7a881090d0 con 0x7f7a881044c0 2024-02-23T00:53:19.637 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:53:19.665 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:18.283005+0000 mon.a (mon.0) 3299 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:53:19.666 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:18.283209+0000 mon.a (mon.0) 3300 : cluster [DBG] osdmap e661: 8 total, 8 up, 6 in 2024-02-23T00:53:19.668 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:19.688 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:53:19.691 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:19.691 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:53:19.792 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:53:19.823 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:20.294 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:19.288519+0000 mon.a (mon.0) 3301 : cluster [DBG] osdmap e662: 8 total, 8 up, 6 in 2024-02-23T00:53:20.294 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:19.305992+0000 mon.c (mon.2) 380 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:20.294 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:19.306363+0000 mon.a (mon.0) 3302 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:20.294 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:19.370370+0000 mgr.y (mgr.14517) 580 : cluster [DBG] pgmap v1001: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s; 730 B/s, 23 objects/s recovering 2024-02-23T00:53:20.294 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:19.534089+0000 mon.a (mon.0) 3303 : audit [INF] from='client.? 172.21.15.47:0/4275810160' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:53:20.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:20.293+0000 7f7a777fe700 1 -- 172.21.15.47:0/4275810160 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v663) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7a80050ef0 con 0x7f7a881044c0 2024-02-23T00:53:20.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:20.295+0000 7f7a8c84b700 10 librados: watch_flush enter 2024-02-23T00:53:20.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:20.296+0000 7f7a8c84b700 10 librados: watch_flush exit 2024-02-23T00:53:20.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:20.296+0000 7f7a8c84b700 20 client.31554.objecter shutdown clearing up homeless session... 2024-02-23T00:53:20.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:20.296+0000 7f7a8c84b700 10 client.31554.objecter successfully canceled tick 2024-02-23T00:53:20.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:20.296+0000 7f7a8c84b700 1 -- 172.21.15.47:0/4275810160 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7a70062cc0 msgr2=0x7f7a70065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:20.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:20.296+0000 7f7a8c84b700 1 --2- 172.21.15.47:0/4275810160 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7a70062cc0 0x7f7a70065180 secure :-1 s=READY pgs=3155 cs=0 l=1 rev1=1 rx=0x7f7a78000e50 tx=0x7f7a7800b040).stop 2024-02-23T00:53:20.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:20.296+0000 7f7a8c84b700 1 -- 172.21.15.47:0/4275810160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a881044c0 msgr2=0x7f7a8810a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:20.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:20.296+0000 7f7a8c84b700 1 --2- 172.21.15.47:0/4275810160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a881044c0 0x7f7a8810a340 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 rx=0x7f7a8805fed0 tx=0x7f7a80004c50).stop 2024-02-23T00:53:20.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:20.296+0000 7f7a8c84b700 1 -- 172.21.15.47:0/4275810160 shutdown_connections 2024-02-23T00:53:20.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:20.296+0000 7f7a8c84b700 1 --2- 172.21.15.47:0/4275810160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a881044c0 0x7f7a8810a340 unknown :-1 s=CLOSED pgs=2606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:20.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:20.296+0000 7f7a8c84b700 1 --2- 172.21.15.47:0/4275810160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a8806b0f0 0x7f7a881101a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:20.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:20.296+0000 7f7a8c84b700 1 --2- 172.21.15.47:0/4275810160 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7a70062cc0 0x7f7a70065180 unknown :-1 s=CLOSED pgs=3155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:20.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:20.296+0000 7f7a8c84b700 1 --2- 172.21.15.47:0/4275810160 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a8806b9c0 0x7f7a88110690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:20.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:20.296+0000 7f7a8c84b700 1 -- 172.21.15.47:0/4275810160 >> 172.21.15.47:0/4275810160 conn(0x7f7a880f3800 msgr2=0x7f7a880f4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:20.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:20.296+0000 7f7a8c84b700 1 -- 172.21.15.47:0/4275810160 shutdown_connections 2024-02-23T00:53:20.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:20.296+0000 7f7a8c84b700 1 -- 172.21.15.47:0/4275810160 wait complete. 2024-02-23T00:53:20.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:20.296+0000 7f7a8c84b700 1 librados: shutdown 2024-02-23T00:53:20.300 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is set 2024-02-23T00:53:20.480 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.4 dump_ops_in_flight 2024-02-23T00:53:20.557 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:53:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:53:20] "GET /metrics HTTP/1.1" 200 191040 "" "Prometheus/2.33.4" 2024-02-23T00:53:20.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:19.288519+0000 mon.a (mon.0) 3301 : cluster [DBG] osdmap e662: 8 total, 8 up, 6 in 2024-02-23T00:53:20.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:19.305992+0000 mon.c (mon.2) 380 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:20.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:19.306363+0000 mon.a (mon.0) 3302 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:20.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:19.370370+0000 mgr.y (mgr.14517) 580 : cluster [DBG] pgmap v1001: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s; 730 B/s, 23 objects/s recovering 2024-02-23T00:53:20.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:19.534089+0000 mon.a (mon.0) 3303 : audit [INF] from='client.? 172.21.15.47:0/4275810160' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:53:20.576 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:53:20.578 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:20.579 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:53:20.663 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:53:20.680 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:53:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:19.288519+0000 mon.a (mon.0) 3301 : cluster [DBG] osdmap e662: 8 total, 8 up, 6 in 2024-02-23T00:53:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:19.305992+0000 mon.c (mon.2) 380 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:19.306363+0000 mon.a (mon.0) 3302 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:19.370370+0000 mgr.y (mgr.14517) 580 : cluster [DBG] pgmap v1001: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s; 730 B/s, 23 objects/s recovering 2024-02-23T00:53:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:19.534089+0000 mon.a (mon.0) 3303 : audit [INF] from='client.? 172.21.15.47:0/4275810160' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:53:20.707 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:20.726 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:53:20.732 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:20.733 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:53:20.755 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:53:20.834 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:53:20.860 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:21.059 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:53:21.060 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:21.061 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:53:21.162 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:53:21.190 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:21.302 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:20.286338+0000 mon.a (mon.0) 3304 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:53:21.302 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:20.294053+0000 mon.a (mon.0) 3305 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:53:21.302 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:20.294149+0000 mon.a (mon.0) 3306 : audit [INF] from='client.? 172.21.15.47:0/4275810160' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:53:21.302 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:20.294218+0000 mon.a (mon.0) 3307 : cluster [DBG] osdmap e663: 8 total, 8 up, 6 in 2024-02-23T00:53:21.302 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:20.295509+0000 mon.c (mon.2) 381 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-114"}]: dispatch 2024-02-23T00:53:21.302 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:20.295893+0000 mon.a (mon.0) 3308 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-114"}]: dispatch 2024-02-23T00:53:21.302 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:21.157060+0000 osd.5 (osd.5) 286 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:53:21.302 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:21.157139+0000 osd.5 (osd.5) 287 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:53:21.539 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:53:21.541 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:21.542 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:53:21.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:20.286338+0000 mon.a (mon.0) 3304 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:53:21.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:20.294053+0000 mon.a (mon.0) 3305 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:53:21.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:20.294149+0000 mon.a (mon.0) 3306 : audit [INF] from='client.? 172.21.15.47:0/4275810160' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:53:21.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:20.294218+0000 mon.a (mon.0) 3307 : cluster [DBG] osdmap e663: 8 total, 8 up, 6 in 2024-02-23T00:53:21.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:20.295509+0000 mon.c (mon.2) 381 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-114"}]: dispatch 2024-02-23T00:53:21.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:20.295893+0000 mon.a (mon.0) 3308 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-114"}]: dispatch 2024-02-23T00:53:21.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:21.157060+0000 osd.5 (osd.5) 286 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:53:21.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:21.157139+0000 osd.5 (osd.5) 287 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:53:21.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:20.286338+0000 mon.a (mon.0) 3304 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:53:21.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:20.294053+0000 mon.a (mon.0) 3305 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:53:21.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:20.294149+0000 mon.a (mon.0) 3306 : audit [INF] from='client.? 172.21.15.47:0/4275810160' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:53:21.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:20.294218+0000 mon.a (mon.0) 3307 : cluster [DBG] osdmap e663: 8 total, 8 up, 6 in 2024-02-23T00:53:21.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:20.295509+0000 mon.c (mon.2) 381 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-114"}]: dispatch 2024-02-23T00:53:21.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:20.295893+0000 mon.a (mon.0) 3308 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-114"}]: dispatch 2024-02-23T00:53:21.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:21.157060+0000 osd.5 (osd.5) 286 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:53:21.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:21.157139+0000 osd.5 (osd.5) 287 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:53:21.642 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:53:21.670 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:22.220 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:53:22.222 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:22.223 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:53:22.323 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:53:22.353 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:22.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.417+0000 7feb85d74700 1 -- 172.21.15.47:0/3057486422 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb800f7400 msgr2=0x7feb80103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:22.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.417+0000 7feb85d74700 1 --2- 172.21.15.47:0/3057486422 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb800f7400 0x7feb80103a20 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 rx=0x7feb78009be0 tx=0x7feb78004d10).stop 2024-02-23T00:53:22.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.417+0000 7feb85d74700 1 -- 172.21.15.47:0/3057486422 shutdown_connections 2024-02-23T00:53:22.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.417+0000 7feb85d74700 1 --2- 172.21.15.47:0/3057486422 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb800f7400 0x7feb80103a20 unknown :-1 s=CLOSED pgs=2767 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:22.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.417+0000 7feb85d74700 1 --2- 172.21.15.47:0/3057486422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb800fcbb0 0x7feb800fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:22.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.417+0000 7feb85d74700 1 --2- 172.21.15.47:0/3057486422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb800fd510 0x7feb800f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:22.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.417+0000 7feb85d74700 1 -- 172.21.15.47:0/3057486422 >> 172.21.15.47:0/3057486422 conn(0x7feb800f2820 msgr2=0x7feb800f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:22.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.417+0000 7feb85d74700 1 -- 172.21.15.47:0/3057486422 shutdown_connections 2024-02-23T00:53:22.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.417+0000 7feb85d74700 1 -- 172.21.15.47:0/3057486422 wait complete. 2024-02-23T00:53:22.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.418+0000 7feb85d74700 1 librados: starting msgr at 2024-02-23T00:53:22.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.418+0000 7feb85d74700 1 librados: starting objecter 2024-02-23T00:53:22.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.418+0000 7feb85d74700 1 Processor -- start 2024-02-23T00:53:22.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.418+0000 7feb85d74700 1 -- start start 2024-02-23T00:53:22.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.418+0000 7feb85d74700 1 librados: setting wanted keys 2024-02-23T00:53:22.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.418+0000 7feb85d74700 1 librados: calling monclient init 2024-02-23T00:53:22.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.418+0000 7feb85d74700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb800f7400 0x7feb8010ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:22.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.418+0000 7feb85d74700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb800fcbb0 0x7feb8010d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:22.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.418+0000 7feb85d74700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb800fd510 0x7feb801070d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:22.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.418+0000 7feb85d74700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb801075c0 con 0x7feb800fd510 2024-02-23T00:53:22.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.418+0000 7feb85d74700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb80107730 con 0x7feb800fcbb0 2024-02-23T00:53:22.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.418+0000 7feb85d74700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb801078a0 con 0x7feb800f7400 2024-02-23T00:53:22.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.418+0000 7feb7f7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb800f7400 0x7feb8010ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:22.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.419+0000 7feb7effd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb800fcbb0 0x7feb8010d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:22.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.419+0000 7feb7f7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb800f7400 0x7feb8010ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:60116/0 (socket says 172.21.15.47:60116) 2024-02-23T00:53:22.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.419+0000 7feb7ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb800fd510 0x7feb801070d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:22.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.419+0000 7feb7effd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb800fcbb0 0x7feb8010d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.47:45804/0 (socket says 172.21.15.47:45804) 2024-02-23T00:53:22.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.419+0000 7feb7f7fe700 1 -- 172.21.15.47:0/1483032426 learned_addr learned my addr 172.21.15.47:0/1483032426 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:53:22.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.419+0000 7feb7f7fe700 1 -- 172.21.15.47:0/1483032426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb800fcbb0 msgr2=0x7feb8010d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:22.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.419+0000 7feb7f7fe700 1 --2- 172.21.15.47:0/1483032426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb800fcbb0 0x7feb8010d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:22.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.419+0000 7feb7f7fe700 1 -- 172.21.15.47:0/1483032426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb800fd510 msgr2=0x7feb801070d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:22.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.419+0000 7feb7f7fe700 1 --2- 172.21.15.47:0/1483032426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb800fd510 0x7feb801070d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:22.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.419+0000 7feb7f7fe700 1 -- 172.21.15.47:0/1483032426 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb70005040 con 0x7feb800f7400 2024-02-23T00:53:22.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.419+0000 7feb7ffff700 1 --2- 172.21.15.47:0/1483032426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb800fd510 0x7feb801070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:53:22.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.419+0000 7feb7f7fe700 1 --2- 172.21.15.47:0/1483032426 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb800f7400 0x7feb8010ce50 secure :-1 s=READY pgs=2768 cs=0 l=1 rev1=1 rx=0x7feb6800e970 tx=0x7feb6800b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:22.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.419+0000 7feb7cff9700 10 client.?.objecter ms_handle_connect 0x7feb800f7400 2024-02-23T00:53:22.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.419+0000 7feb7cff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:53:22.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.419+0000 7feb7cff9700 1 -- 172.21.15.47:0/1483032426 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb68004660 con 0x7feb800f7400 2024-02-23T00:53:22.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.419+0000 7feb7cff9700 1 -- 172.21.15.47:0/1483032426 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7feb6800f040 con 0x7feb800f7400 2024-02-23T00:53:22.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.419+0000 7feb85d74700 1 -- 172.21.15.47:0/1483032426 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb780097c0 con 0x7feb800f7400 2024-02-23T00:53:22.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.419+0000 7feb7cff9700 1 -- 172.21.15.47:0/1483032426 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb680146f0 con 0x7feb800f7400 2024-02-23T00:53:22.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.419+0000 7feb85d74700 10 client.41879.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:53:22.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.419+0000 7feb85d74700 1 -- 172.21.15.47:0/1483032426 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb80107f30 con 0x7feb800f7400 2024-02-23T00:53:22.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.419+0000 7feb85d74700 1 librados: init done 2024-02-23T00:53:22.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.420+0000 7feb85d74700 1 -- 172.21.15.47:0/1483032426 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb8005fa60 con 0x7feb800f7400 2024-02-23T00:53:22.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.420+0000 7feb7cff9700 1 -- 172.21.15.47:0/1483032426 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7feb680133c0 con 0x7feb800f7400 2024-02-23T00:53:22.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.421+0000 7feb7cff9700 1 --2- 172.21.15.47:0/1483032426 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7feb6c062cc0 0x7feb6c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:22.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.421+0000 7feb7cff9700 1 -- 172.21.15.47:0/1483032426 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(665..665 src has 482..665) v4 ==== 9605+0+0 (secure 0 0 0) 0x7feb6807d3a0 con 0x7feb800f7400 2024-02-23T00:53:22.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.421+0000 7feb7cff9700 10 client.41879.objecter ms_dispatch 0x7feb80001550 osd_map(665..665 src has 482..665) v4 2024-02-23T00:53:22.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.421+0000 7feb7cff9700 3 client.41879.objecter handle_osd_map got epochs [665,665] > 0 2024-02-23T00:53:22.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.421+0000 7feb7cff9700 3 client.41879.objecter handle_osd_map decoding full epoch 665 2024-02-23T00:53:22.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.421+0000 7feb7effd700 1 --2- 172.21.15.47:0/1483032426 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7feb6c062cc0 0x7feb6c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:22.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.421+0000 7feb7cff9700 20 client.41879.objecter dump_active .. 0 homeless 2024-02-23T00:53:22.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.421+0000 7feb7cff9700 10 client.41879.objecter ms_handle_connect 0x7feb6c062cc0 2024-02-23T00:53:22.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.421+0000 7feb7effd700 1 --2- 172.21.15.47:0/1483032426 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7feb6c062cc0 0x7feb6c065180 secure :-1 s=READY pgs=3156 cs=0 l=1 rev1=1 rx=0x7feb801087d0 tx=0x7feb7000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:22.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.425+0000 7feb7cff9700 1 -- 172.21.15.47:0/1483032426 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7feb6804af60 con 0x7feb800f7400 2024-02-23T00:53:22.451 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:53:22.454 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:22.455 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:53:22.472 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:53:22.472 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-02-23T00:53:22.472 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 0 2024-02-23T00:53:22.472 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:53:22.483 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:53:22.498 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:21.298831+0000 mon.a (mon.0) 3309 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-114"}]': finished 2024-02-23T00:53:22.498 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:21.298921+0000 mon.a (mon.0) 3310 : cluster [DBG] osdmap e664: 8 total, 8 up, 6 in 2024-02-23T00:53:22.498 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:21.302021+0000 mon.c (mon.2) 382 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-114", "mode": "writeback"}]: dispatch 2024-02-23T00:53:22.498 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:21.302316+0000 mon.a (mon.0) 3311 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-114", "mode": "writeback"}]: dispatch 2024-02-23T00:53:22.499 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:21.370731+0000 mgr.y (mgr.14517) 581 : cluster [DBG] pgmap v1004: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:53:22.556 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:53:22.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:21.298831+0000 mon.a (mon.0) 3309 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-114"}]': finished 2024-02-23T00:53:22.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:21.298921+0000 mon.a (mon.0) 3310 : cluster [DBG] osdmap e664: 8 total, 8 up, 6 in 2024-02-23T00:53:22.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:21.302021+0000 mon.c (mon.2) 382 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-114", "mode": "writeback"}]: dispatch 2024-02-23T00:53:22.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:21.302316+0000 mon.a (mon.0) 3311 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-114", "mode": "writeback"}]: dispatch 2024-02-23T00:53:22.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:21.370731+0000 mgr.y (mgr.14517) 581 : cluster [DBG] pgmap v1004: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:53:22.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:21.298831+0000 mon.a (mon.0) 3309 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-114"}]': finished 2024-02-23T00:53:22.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:21.298921+0000 mon.a (mon.0) 3310 : cluster [DBG] osdmap e664: 8 total, 8 up, 6 in 2024-02-23T00:53:22.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:21.302021+0000 mon.c (mon.2) 382 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-114", "mode": "writeback"}]: dispatch 2024-02-23T00:53:22.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:21.302316+0000 mon.a (mon.0) 3311 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-114", "mode": "writeback"}]: dispatch 2024-02-23T00:53:22.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:21.370731+0000 mgr.y (mgr.14517) 581 : cluster [DBG] pgmap v1004: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:53:22.588 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:22.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:22.640+0000 7feb85d74700 1 -- 172.21.15.47:0/1483032426 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd reweight", "id": 3, "weight": 0.3921672767487391} v 0) v1 -- 0x7feb80108510 con 0x7feb800f7400 2024-02-23T00:53:22.737 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:53:22.740 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:22.740 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:53:22.841 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:53:22.845 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:53:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:53:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:53:22.870 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:22.943 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:53:22.946 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:22.946 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:53:23.047 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:53:23.076 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:23.114 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:53:23.116 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:53:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:53:23.117 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:23.117 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:53:23.137 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.4 dump_blocked_ops 2024-02-23T00:53:23.219 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:53:23.247 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:23.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:23.315+0000 7feb7cff9700 1 -- 172.21.15.47:0/1483032426 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 3, "weight": 0.3921672767487391}]=0 reweighted osd.3 to 0.392167 (6465) v666) v1 ==== 134+0+0 (secure 0 0 0) 0x7feb68019070 con 0x7feb800f7400 2024-02-23T00:53:23.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:23.318+0000 7feb85d74700 10 librados: watch_flush enter 2024-02-23T00:53:23.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:23.318+0000 7feb85d74700 10 librados: watch_flush exit 2024-02-23T00:53:23.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:23.318+0000 7feb85d74700 20 client.41879.objecter shutdown clearing up homeless session... 2024-02-23T00:53:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:23.318+0000 7feb85d74700 10 client.41879.objecter successfully canceled tick 2024-02-23T00:53:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:23.318+0000 7feb85d74700 1 -- 172.21.15.47:0/1483032426 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7feb6c062cc0 msgr2=0x7feb6c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:23.318+0000 7feb85d74700 1 --2- 172.21.15.47:0/1483032426 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7feb6c062cc0 0x7feb6c065180 secure :-1 s=READY pgs=3156 cs=0 l=1 rev1=1 rx=0x7feb801087d0 tx=0x7feb7000b040).stop 2024-02-23T00:53:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:23.318+0000 7feb85d74700 1 -- 172.21.15.47:0/1483032426 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb800f7400 msgr2=0x7feb8010ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:23.318+0000 7feb85d74700 1 --2- 172.21.15.47:0/1483032426 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb800f7400 0x7feb8010ce50 secure :-1 s=READY pgs=2768 cs=0 l=1 rev1=1 rx=0x7feb6800e970 tx=0x7feb6800b040).stop 2024-02-23T00:53:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:23.319+0000 7feb85d74700 1 -- 172.21.15.47:0/1483032426 shutdown_connections 2024-02-23T00:53:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:23.319+0000 7feb85d74700 1 --2- 172.21.15.47:0/1483032426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb800fd510 0x7feb801070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:23.319+0000 7feb85d74700 1 --2- 172.21.15.47:0/1483032426 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7feb6c062cc0 0x7feb6c065180 unknown :-1 s=CLOSED pgs=3156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:23.319+0000 7feb85d74700 1 --2- 172.21.15.47:0/1483032426 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb800f7400 0x7feb8010ce50 unknown :-1 s=CLOSED pgs=2768 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:23.319+0000 7feb85d74700 1 --2- 172.21.15.47:0/1483032426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb800fcbb0 0x7feb8010d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:23.319+0000 7feb85d74700 1 -- 172.21.15.47:0/1483032426 >> 172.21.15.47:0/1483032426 conn(0x7feb800f2820 msgr2=0x7feb800f41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:23.319+0000 7feb85d74700 1 -- 172.21.15.47:0/1483032426 shutdown_connections 2024-02-23T00:53:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:23.319+0000 7feb85d74700 1 -- 172.21.15.47:0/1483032426 wait complete. 2024-02-23T00:53:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:23.319+0000 7feb85d74700 1 librados: shutdown 2024-02-23T00:53:23.323 INFO:teuthology.orchestra.run.smithi047.stderr:reweighted osd.3 to 0.392167 (6465) 2024-02-23T00:53:23.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:22.296869+0000 mon.a (mon.0) 3312 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:53:23.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:22.304614+0000 mon.a (mon.0) 3313 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-114", "mode": "writeback"}]': finished 2024-02-23T00:53:23.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:22.304675+0000 mon.a (mon.0) 3314 : cluster [DBG] osdmap e665: 8 total, 8 up, 6 in 2024-02-23T00:53:23.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:22.641802+0000 mon.c (mon.2) 383 : audit [INF] from='client.? 172.21.15.47:0/1483032426' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 0.3921672767487391}]: dispatch 2024-02-23T00:53:23.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:22.642111+0000 mon.a (mon.0) 3315 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 0.3921672767487391}]: dispatch 2024-02-23T00:53:23.454 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:53:23.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:22.296869+0000 mon.a (mon.0) 3312 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:53:23.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:22.304614+0000 mon.a (mon.0) 3313 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-114", "mode": "writeback"}]': finished 2024-02-23T00:53:23.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:22.304675+0000 mon.a (mon.0) 3314 : cluster [DBG] osdmap e665: 8 total, 8 up, 6 in 2024-02-23T00:53:23.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:22.641802+0000 mon.c (mon.2) 383 : audit [INF] from='client.? 172.21.15.47:0/1483032426' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 0.3921672767487391}]: dispatch 2024-02-23T00:53:23.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:22.642111+0000 mon.a (mon.0) 3315 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 0.3921672767487391}]: dispatch 2024-02-23T00:53:23.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:22.296869+0000 mon.a (mon.0) 3312 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:53:23.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:22.304614+0000 mon.a (mon.0) 3313 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-114", "mode": "writeback"}]': finished 2024-02-23T00:53:23.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:22.304675+0000 mon.a (mon.0) 3314 : cluster [DBG] osdmap e665: 8 total, 8 up, 6 in 2024-02-23T00:53:23.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:22.641802+0000 mon.c (mon.2) 383 : audit [INF] from='client.? 172.21.15.47:0/1483032426' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 0.3921672767487391}]: dispatch 2024-02-23T00:53:23.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:22.642111+0000 mon.a (mon.0) 3315 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 0.3921672767487391}]: dispatch 2024-02-23T00:53:23.665 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset noscrub 2024-02-23T00:53:24.127 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:53:24.129 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:24.129 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:53:24.230 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:53:24.260 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:24.280 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:53:24.282 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:24.283 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:53:24.384 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:53:24.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.391+0000 7f109f559700 1 -- 172.21.15.47:0/1128439945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1098102210 msgr2=0x7f1098104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:24.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.391+0000 7f109f559700 1 --2- 172.21.15.47:0/1128439945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1098102210 0x7f1098104600 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 rx=0x7f1090009c30 tx=0x7f1090004c50).stop 2024-02-23T00:53:24.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.391+0000 7f109f559700 1 -- 172.21.15.47:0/1128439945 shutdown_connections 2024-02-23T00:53:24.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.391+0000 7f109f559700 1 --2- 172.21.15.47:0/1128439945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1098102210 0x7f1098104600 unknown :-1 s=CLOSED pgs=2607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:24.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.391+0000 7f109f559700 1 --2- 172.21.15.47:0/1128439945 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f10980f8cb0 0x7f10980f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:24.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.391+0000 7f109f559700 1 --2- 172.21.15.47:0/1128439945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10980f9610 0x7f1098101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:24.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.391+0000 7f109f559700 1 -- 172.21.15.47:0/1128439945 >> 172.21.15.47:0/1128439945 conn(0x7f10980f2820 msgr2=0x7f10980f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:24.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.391+0000 7f109f559700 1 -- 172.21.15.47:0/1128439945 shutdown_connections 2024-02-23T00:53:24.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.391+0000 7f109f559700 1 -- 172.21.15.47:0/1128439945 wait complete. 2024-02-23T00:53:24.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.392+0000 7f109f559700 1 librados: starting msgr at 2024-02-23T00:53:24.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.392+0000 7f109f559700 1 librados: starting objecter 2024-02-23T00:53:24.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.392+0000 7f109f559700 1 Processor -- start 2024-02-23T00:53:24.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.392+0000 7f109f559700 1 -- start start 2024-02-23T00:53:24.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.392+0000 7f109f559700 1 librados: setting wanted keys 2024-02-23T00:53:24.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.392+0000 7f109f559700 1 librados: calling monclient init 2024-02-23T00:53:24.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.392+0000 7f109f559700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f10980f8cb0 0x7f10981100d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:24.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.392+0000 7f109f559700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10980f9610 0x7f10981105c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:24.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.392+0000 7f109f559700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1098102210 0x7f109810a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:24.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.392+0000 7f109f559700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109810a760 con 0x7f1098102210 2024-02-23T00:53:24.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.392+0000 7f109f559700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109810a8d0 con 0x7f10980f9610 2024-02-23T00:53:24.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.392+0000 7f109f559700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109810aa90 con 0x7f10980f8cb0 2024-02-23T00:53:24.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.392+0000 7f109daf6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1098102210 0x7f109810a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:24.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.392+0000 7f109daf6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1098102210 0x7f109810a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49912/0 (socket says 172.21.15.47:49912) 2024-02-23T00:53:24.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.392+0000 7f109caf4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10980f9610 0x7f10981105c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:24.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.392+0000 7f109daf6700 1 -- 172.21.15.47:0/1424677917 learned_addr learned my addr 172.21.15.47:0/1424677917 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:53:24.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.393+0000 7f109d2f5700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f10980f8cb0 0x7f10981100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:24.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.393+0000 7f109daf6700 1 -- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f10980f8cb0 msgr2=0x7f10981100d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:24.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.393+0000 7f109daf6700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f10980f8cb0 0x7f10981100d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:24.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.393+0000 7f109daf6700 1 -- 172.21.15.47:0/1424677917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10980f9610 msgr2=0x7f10981105c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:24.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.393+0000 7f109daf6700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10980f9610 0x7f10981105c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:24.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.393+0000 7f109daf6700 1 -- 172.21.15.47:0/1424677917 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10900097c0 con 0x7f1098102210 2024-02-23T00:53:24.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.393+0000 7f109caf4700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10980f9610 0x7f10981105c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:53:24.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.393+0000 7f109daf6700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1098102210 0x7f109810a270 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 rx=0x7f1090009610 tx=0x7f1090004c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:24.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.393+0000 7f108e7fc700 10 client.?.objecter ms_handle_connect 0x7f1098102210 2024-02-23T00:53:24.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.393+0000 7f108e7fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:53:24.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.393+0000 7f108e7fc700 1 -- 172.21.15.47:0/1424677917 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109000e040 con 0x7f1098102210 2024-02-23T00:53:24.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.393+0000 7f108e7fc700 1 -- 172.21.15.47:0/1424677917 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f1090018040 con 0x7f1098102210 2024-02-23T00:53:24.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.393+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f109810ad10 con 0x7f1098102210 2024-02-23T00:53:24.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.393+0000 7f108e7fc700 1 -- 172.21.15.47:0/1424677917 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1090003970 con 0x7f1098102210 2024-02-23T00:53:24.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.393+0000 7f109f559700 10 client.31566.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:53:24.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.393+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1098116f00 con 0x7f1098102210 2024-02-23T00:53:24.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.393+0000 7f109f559700 1 librados: init done 2024-02-23T00:53:24.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.394+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f109810b140 con 0x7f1098102210 2024-02-23T00:53:24.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.395+0000 7f108e7fc700 1 -- 172.21.15.47:0/1424677917 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f1090004020 con 0x7f1098102210 2024-02-23T00:53:24.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.395+0000 7f108e7fc700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1084062cc0 0x7f1084065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:24.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.396+0000 7f108e7fc700 1 -- 172.21.15.47:0/1424677917 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(667..667 src has 482..667) v4 ==== 9727+0+0 (secure 0 0 0) 0x7f109007e790 con 0x7f1098102210 2024-02-23T00:53:24.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.396+0000 7f108e7fc700 10 client.31566.objecter ms_dispatch 0x7f1098001550 osd_map(667..667 src has 482..667) v4 2024-02-23T00:53:24.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.396+0000 7f108e7fc700 3 client.31566.objecter handle_osd_map got epochs [667,667] > 0 2024-02-23T00:53:24.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.396+0000 7f108e7fc700 3 client.31566.objecter handle_osd_map decoding full epoch 667 2024-02-23T00:53:24.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.396+0000 7f109d2f5700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1084062cc0 0x7f1084065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:24.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.396+0000 7f108e7fc700 20 client.31566.objecter dump_active .. 0 homeless 2024-02-23T00:53:24.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.396+0000 7f108e7fc700 1 -- 172.21.15.47:0/1424677917 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v667) v1 ==== 54+0+16 (secure 0 0 0) 0x7f109810b140 con 0x7f1098102210 2024-02-23T00:53:24.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.396+0000 7f108e7fc700 10 client.31566.objecter ms_handle_connect 0x7f1084062cc0 2024-02-23T00:53:24.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.396+0000 7f109d2f5700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1084062cc0 0x7f1084065180 secure :-1 s=READY pgs=3157 cs=0 l=1 rev1=1 rx=0x7f10940043d0 tx=0x7f109400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:24.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.396+0000 7f109f559700 10 client.31566.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:24.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.396+0000 7f109f559700 15 client.31566.objecter _session_command_op_assign -1 1 2024-02-23T00:53:24.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.396+0000 7f109f559700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f10980fe5a0 0x7f10980fe980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:24.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.396+0000 7f109f559700 20 client.31566.objecter _get_session s=0x7f109810b140 osd=0 3 2024-02-23T00:53:24.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.396+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f109810b140 osd=0 3 2024-02-23T00:53:24.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.396+0000 7f109f559700 20 client.31566.objecter _get_session s=0x7f109810b140 osd=0 3 2024-02-23T00:53:24.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.396+0000 7f109f559700 15 client.31566.objecter _session_command_op_remove -1 1 2024-02-23T00:53:24.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.396+0000 7f109f559700 20 client.31566.objecter get_session s=0x7f109810b140 osd=0 3 2024-02-23T00:53:24.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.396+0000 7f109f559700 15 client.31566.objecter _session_command_op_assign 0 1 2024-02-23T00:53:24.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.396+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f109810b140 osd=0 4 2024-02-23T00:53:24.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.396+0000 7f109f559700 10 client.31566.objecter _send_command 1 2024-02-23T00:53:24.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.396+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f109805fa60 con 0x7f10980fe5a0 2024-02-23T00:53:24.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.396+0000 7f109caf4700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f10980fe5a0 0x7f10980fe980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:24.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.396+0000 7f108e7fc700 10 client.31566.objecter ms_handle_connect 0x7f10980fe5a0 2024-02-23T00:53:24.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.396+0000 7f109caf4700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f10980fe5a0 0x7f10980fe980 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:24.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.397+0000 7f108e7fc700 1 -- 172.21.15.47:0/1424677917 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f109805fa60 con 0x7f10980fe5a0 2024-02-23T00:53:24.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.397+0000 7f108e7fc700 10 client.31566.objecter ms_dispatch 0x7f1098001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:53:24.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.397+0000 7f108e7fc700 10 client.31566.objecter _finish_command 1 = system:0 2024-02-23T00:53:24.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.397+0000 7f108e7fc700 20 client.31566.objecter put_session s=0x7f109810b140 osd=0 4 2024-02-23T00:53:24.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.397+0000 7f108e7fc700 15 client.31566.objecter _session_command_op_remove 0 1 2024-02-23T00:53:24.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.412+0000 7f109f559700 10 client.31566.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:53:24.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.412+0000 7f109f559700 15 client.31566.objecter _session_command_op_assign -1 2 2024-02-23T00:53:24.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.412+0000 7f109f559700 20 client.31566.objecter _get_session s=0x7f109810b140 osd=0 3 2024-02-23T00:53:24.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.412+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f109810b140 osd=0 3 2024-02-23T00:53:24.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.412+0000 7f109f559700 20 client.31566.objecter _get_session s=0x7f109810b140 osd=0 3 2024-02-23T00:53:24.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.412+0000 7f109f559700 15 client.31566.objecter _session_command_op_remove -1 2 2024-02-23T00:53:24.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.412+0000 7f109f559700 20 client.31566.objecter get_session s=0x7f109810b140 osd=0 3 2024-02-23T00:53:24.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.412+0000 7f109f559700 15 client.31566.objecter _session_command_op_assign 0 2 2024-02-23T00:53:24.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.412+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f109810b140 osd=0 4 2024-02-23T00:53:24.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.412+0000 7f109f559700 10 client.31566.objecter _send_command 2 2024-02-23T00:53:24.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.412+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f109805ee80 con 0x7f10980fe5a0 2024-02-23T00:53:24.416 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.414+0000 7f108e7fc700 1 -- 172.21.15.47:0/1424677917 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f109805ee80 con 0x7f10980fe5a0 2024-02-23T00:53:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.414+0000 7f108e7fc700 10 client.31566.objecter ms_dispatch 0x7f1098001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:53:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.414+0000 7f108e7fc700 10 client.31566.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:53:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.414+0000 7f108e7fc700 20 client.31566.objecter put_session s=0x7f109810b140 osd=0 4 2024-02-23T00:53:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.414+0000 7f108e7fc700 15 client.31566.objecter _session_command_op_remove 0 2 2024-02-23T00:53:24.417 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:53:24.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.414+0000 7f109f559700 10 client.31566.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:24.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.414+0000 7f109f559700 15 client.31566.objecter _session_command_op_assign -1 3 2024-02-23T00:53:24.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.414+0000 7f109f559700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f1098028540 0x7f10980598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:24.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.414+0000 7f109f559700 20 client.31566.objecter _get_session s=0x7f1098107050 osd=1 3 2024-02-23T00:53:24.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.414+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f1098107050 osd=1 3 2024-02-23T00:53:24.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.414+0000 7f109f559700 20 client.31566.objecter _get_session s=0x7f1098107050 osd=1 3 2024-02-23T00:53:24.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.414+0000 7f109f559700 15 client.31566.objecter _session_command_op_remove -1 3 2024-02-23T00:53:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.414+0000 7f109f559700 20 client.31566.objecter get_session s=0x7f1098107050 osd=1 3 2024-02-23T00:53:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.414+0000 7f109f559700 15 client.31566.objecter _session_command_op_assign 1 3 2024-02-23T00:53:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.414+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f1098107050 osd=1 4 2024-02-23T00:53:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.414+0000 7f109f559700 10 client.31566.objecter _send_command 3 2024-02-23T00:53:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.414+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1098059de0 con 0x7f1098028540 2024-02-23T00:53:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.414+0000 7f109d2f5700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f1098028540 0x7f10980598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.414+0000 7f108e7fc700 10 client.31566.objecter ms_handle_connect 0x7f1098028540 2024-02-23T00:53:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.414+0000 7f109d2f5700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f1098028540 0x7f10980598f0 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.415+0000 7f108e7fc700 1 -- 172.21.15.47:0/1424677917 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1098059de0 con 0x7f1098028540 2024-02-23T00:53:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.415+0000 7f108e7fc700 10 client.31566.objecter ms_dispatch 0x7f1098001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:53:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.415+0000 7f108e7fc700 10 client.31566.objecter _finish_command 3 = system:0 2024-02-23T00:53:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.415+0000 7f108e7fc700 20 client.31566.objecter put_session s=0x7f1098107050 osd=1 4 2024-02-23T00:53:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.415+0000 7f108e7fc700 15 client.31566.objecter _session_command_op_remove 1 3 2024-02-23T00:53:24.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.442+0000 7f109f559700 10 client.31566.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:53:24.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.442+0000 7f109f559700 15 client.31566.objecter _session_command_op_assign -1 4 2024-02-23T00:53:24.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.442+0000 7f109f559700 20 client.31566.objecter _get_session s=0x7f1098107050 osd=1 3 2024-02-23T00:53:24.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.442+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f1098107050 osd=1 3 2024-02-23T00:53:24.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.442+0000 7f109f559700 20 client.31566.objecter _get_session s=0x7f1098107050 osd=1 3 2024-02-23T00:53:24.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.442+0000 7f109f559700 15 client.31566.objecter _session_command_op_remove -1 4 2024-02-23T00:53:24.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.442+0000 7f109f559700 20 client.31566.objecter get_session s=0x7f1098107050 osd=1 3 2024-02-23T00:53:24.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.442+0000 7f109f559700 15 client.31566.objecter _session_command_op_assign 1 4 2024-02-23T00:53:24.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.442+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f1098107050 osd=1 4 2024-02-23T00:53:24.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.442+0000 7f109f559700 10 client.31566.objecter _send_command 4 2024-02-23T00:53:24.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.442+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f109811b0b0 con 0x7f1098028540 2024-02-23T00:53:24.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.444+0000 7f108e7fc700 1 -- 172.21.15.47:0/1424677917 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f109811b0b0 con 0x7f1098028540 2024-02-23T00:53:24.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.444+0000 7f108e7fc700 10 client.31566.objecter ms_dispatch 0x7f1098001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:53:24.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.444+0000 7f108e7fc700 10 client.31566.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:53:24.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.444+0000 7f108e7fc700 20 client.31566.objecter put_session s=0x7f1098107050 osd=1 4 2024-02-23T00:53:24.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.444+0000 7f108e7fc700 15 client.31566.objecter _session_command_op_remove 1 4 2024-02-23T00:53:24.448 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:53:24.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.445+0000 7f109f559700 10 client.31566.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:24.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.445+0000 7f109f559700 15 client.31566.objecter _session_command_op_assign -1 5 2024-02-23T00:53:24.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.445+0000 7f109f559700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f10980594d0 0x7f109811e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:24.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.445+0000 7f109f559700 20 client.31566.objecter _get_session s=0x7f1098059360 osd=2 3 2024-02-23T00:53:24.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.445+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f1098059360 osd=2 3 2024-02-23T00:53:24.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.445+0000 7f109f559700 20 client.31566.objecter _get_session s=0x7f1098059360 osd=2 3 2024-02-23T00:53:24.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.445+0000 7f109f559700 15 client.31566.objecter _session_command_op_remove -1 5 2024-02-23T00:53:24.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.445+0000 7f109f559700 20 client.31566.objecter get_session s=0x7f1098059360 osd=2 3 2024-02-23T00:53:24.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.445+0000 7f109f559700 15 client.31566.objecter _session_command_op_assign 2 5 2024-02-23T00:53:24.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.445+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f1098059360 osd=2 4 2024-02-23T00:53:24.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.445+0000 7f109f559700 10 client.31566.objecter _send_command 5 2024-02-23T00:53:24.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.445+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1098028290 con 0x7f10980594d0 2024-02-23T00:53:24.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.445+0000 7f109caf4700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f10980594d0 0x7f109811e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:24.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.445+0000 7f108e7fc700 10 client.31566.objecter ms_handle_connect 0x7f10980594d0 2024-02-23T00:53:24.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.445+0000 7f109caf4700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f10980594d0 0x7f109811e440 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:24.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.446+0000 7f108e7fc700 1 -- 172.21.15.47:0/1424677917 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1098028290 con 0x7f10980594d0 2024-02-23T00:53:24.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.446+0000 7f108e7fc700 10 client.31566.objecter ms_dispatch 0x7f1098001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:53:24.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.446+0000 7f108e7fc700 10 client.31566.objecter _finish_command 5 = system:0 2024-02-23T00:53:24.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.446+0000 7f108e7fc700 20 client.31566.objecter put_session s=0x7f1098059360 osd=2 4 2024-02-23T00:53:24.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.446+0000 7f108e7fc700 15 client.31566.objecter _session_command_op_remove 2 5 2024-02-23T00:53:24.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.460+0000 7f109f559700 10 client.31566.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:53:24.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.460+0000 7f109f559700 15 client.31566.objecter _session_command_op_assign -1 6 2024-02-23T00:53:24.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.460+0000 7f109f559700 20 client.31566.objecter _get_session s=0x7f1098059360 osd=2 3 2024-02-23T00:53:24.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.460+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f1098059360 osd=2 3 2024-02-23T00:53:24.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.460+0000 7f109f559700 20 client.31566.objecter _get_session s=0x7f1098059360 osd=2 3 2024-02-23T00:53:24.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.460+0000 7f109f559700 15 client.31566.objecter _session_command_op_remove -1 6 2024-02-23T00:53:24.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.460+0000 7f109f559700 20 client.31566.objecter get_session s=0x7f1098059360 osd=2 3 2024-02-23T00:53:24.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.460+0000 7f109f559700 15 client.31566.objecter _session_command_op_assign 2 6 2024-02-23T00:53:24.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.460+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f1098059360 osd=2 4 2024-02-23T00:53:24.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.460+0000 7f109f559700 10 client.31566.objecter _send_command 6 2024-02-23T00:53:24.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.460+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1098027600 con 0x7f10980594d0 2024-02-23T00:53:24.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.462+0000 7f108e7fc700 1 -- 172.21.15.47:0/1424677917 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1098027600 con 0x7f10980594d0 2024-02-23T00:53:24.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.462+0000 7f108e7fc700 10 client.31566.objecter ms_dispatch 0x7f1098001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:53:24.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.462+0000 7f108e7fc700 10 client.31566.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:53:24.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.462+0000 7f108e7fc700 20 client.31566.objecter put_session s=0x7f1098059360 osd=2 4 2024-02-23T00:53:24.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.462+0000 7f108e7fc700 15 client.31566.objecter _session_command_op_remove 2 6 2024-02-23T00:53:24.465 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:53:24.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.462+0000 7f109f559700 10 client.31566.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:24.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.462+0000 7f109f559700 15 client.31566.objecter _session_command_op_assign -1 7 2024-02-23T00:53:24.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.462+0000 7f109f559700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f109811fab0 0x7f1098121ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:24.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.462+0000 7f109f559700 20 client.31566.objecter _get_session s=0x7f109811ba20 osd=3 3 2024-02-23T00:53:24.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.462+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f109811ba20 osd=3 3 2024-02-23T00:53:24.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.462+0000 7f109f559700 20 client.31566.objecter _get_session s=0x7f109811ba20 osd=3 3 2024-02-23T00:53:24.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.462+0000 7f109f559700 15 client.31566.objecter _session_command_op_remove -1 7 2024-02-23T00:53:24.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.462+0000 7f109f559700 20 client.31566.objecter get_session s=0x7f109811ba20 osd=3 3 2024-02-23T00:53:24.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.462+0000 7f109f559700 15 client.31566.objecter _session_command_op_assign 3 7 2024-02-23T00:53:24.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.462+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f109811ba20 osd=3 4 2024-02-23T00:53:24.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.462+0000 7f109f559700 10 client.31566.objecter _send_command 7 2024-02-23T00:53:24.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.462+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f109811c1a0 con 0x7f109811fab0 2024-02-23T00:53:24.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.462+0000 7f109daf6700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f109811fab0 0x7f1098121ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:24.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.462+0000 7f108e7fc700 10 client.31566.objecter ms_handle_connect 0x7f109811fab0 2024-02-23T00:53:24.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.462+0000 7f109daf6700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f109811fab0 0x7f1098121ea0 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:24.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.463+0000 7f108e7fc700 1 -- 172.21.15.47:0/1424677917 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f109811c1a0 con 0x7f109811fab0 2024-02-23T00:53:24.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.463+0000 7f108e7fc700 10 client.31566.objecter ms_dispatch 0x7f1098001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:53:24.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.463+0000 7f108e7fc700 10 client.31566.objecter _finish_command 7 = system:0 2024-02-23T00:53:24.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.463+0000 7f108e7fc700 20 client.31566.objecter put_session s=0x7f109811ba20 osd=3 4 2024-02-23T00:53:24.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.463+0000 7f108e7fc700 15 client.31566.objecter _session_command_op_remove 3 7 2024-02-23T00:53:24.477 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:22.856739+0000 mgr.y (mgr.14517) 582 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:53:24.477 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:23.308045+0000 mon.a (mon.0) 3316 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 0.3921672767487391}]': finished 2024-02-23T00:53:24.477 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:23.308117+0000 mon.a (mon.0) 3317 : cluster [DBG] osdmap e666: 8 total, 8 up, 6 in 2024-02-23T00:53:24.477 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:23.371115+0000 mgr.y (mgr.14517) 583 : cluster [DBG] pgmap v1007: 184 pgs: 184 active+clean; 5.8 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-23T00:53:24.477 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:22.856739+0000 mgr.y (mgr.14517) 582 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:53:24.477 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:23.308045+0000 mon.a (mon.0) 3316 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 0.3921672767487391}]': finished 2024-02-23T00:53:24.477 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:23.308117+0000 mon.a (mon.0) 3317 : cluster [DBG] osdmap e666: 8 total, 8 up, 6 in 2024-02-23T00:53:24.478 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:23.371115+0000 mgr.y (mgr.14517) 583 : cluster [DBG] pgmap v1007: 184 pgs: 184 active+clean; 5.8 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-23T00:53:24.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.490+0000 7f109f559700 10 client.31566.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:53:24.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.490+0000 7f109f559700 15 client.31566.objecter _session_command_op_assign -1 8 2024-02-23T00:53:24.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.490+0000 7f109f559700 20 client.31566.objecter _get_session s=0x7f109811ba20 osd=3 3 2024-02-23T00:53:24.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.490+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f109811ba20 osd=3 3 2024-02-23T00:53:24.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.490+0000 7f109f559700 20 client.31566.objecter _get_session s=0x7f109811ba20 osd=3 3 2024-02-23T00:53:24.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.490+0000 7f109f559700 15 client.31566.objecter _session_command_op_remove -1 8 2024-02-23T00:53:24.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.490+0000 7f109f559700 20 client.31566.objecter get_session s=0x7f109811ba20 osd=3 3 2024-02-23T00:53:24.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.490+0000 7f109f559700 15 client.31566.objecter _session_command_op_assign 3 8 2024-02-23T00:53:24.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.490+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f109811ba20 osd=3 4 2024-02-23T00:53:24.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.490+0000 7f109f559700 10 client.31566.objecter _send_command 8 2024-02-23T00:53:24.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.490+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f109811b490 con 0x7f109811fab0 2024-02-23T00:53:24.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.492+0000 7f108e7fc700 1 -- 172.21.15.47:0/1424677917 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f109811b490 con 0x7f109811fab0 2024-02-23T00:53:24.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.492+0000 7f108e7fc700 10 client.31566.objecter ms_dispatch 0x7f1098001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:53:24.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.492+0000 7f108e7fc700 10 client.31566.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:53:24.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.492+0000 7f108e7fc700 20 client.31566.objecter put_session s=0x7f109811ba20 osd=3 4 2024-02-23T00:53:24.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.492+0000 7f108e7fc700 15 client.31566.objecter _session_command_op_remove 3 8 2024-02-23T00:53:24.495 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:53:24.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.492+0000 7f109f559700 10 client.31566.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:24.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.492+0000 7f109f559700 15 client.31566.objecter _session_command_op_assign -1 9 2024-02-23T00:53:24.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.492+0000 7f109f559700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f109811f610 0x7f1098125520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:24.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.492+0000 7f109f559700 20 client.31566.objecter _get_session s=0x7f109811b490 osd=4 3 2024-02-23T00:53:24.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.492+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f109811b490 osd=4 3 2024-02-23T00:53:24.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.492+0000 7f109f559700 20 client.31566.objecter _get_session s=0x7f109811b490 osd=4 3 2024-02-23T00:53:24.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.492+0000 7f109f559700 15 client.31566.objecter _session_command_op_remove -1 9 2024-02-23T00:53:24.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.492+0000 7f109f559700 20 client.31566.objecter get_session s=0x7f109811b490 osd=4 3 2024-02-23T00:53:24.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.492+0000 7f109f559700 15 client.31566.objecter _session_command_op_assign 4 9 2024-02-23T00:53:24.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.492+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f109811b490 osd=4 4 2024-02-23T00:53:24.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.492+0000 7f109f559700 10 client.31566.objecter _send_command 9 2024-02-23T00:53:24.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.492+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f109811b740 con 0x7f109811f610 2024-02-23T00:53:24.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.493+0000 7f109d2f5700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f109811f610 0x7f1098125520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:24.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.493+0000 7f108e7fc700 10 client.31566.objecter ms_handle_connect 0x7f109811f610 2024-02-23T00:53:24.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.493+0000 7f109d2f5700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f109811f610 0x7f1098125520 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:24.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.494+0000 7f108e7fc700 1 -- 172.21.15.47:0/1424677917 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f109811b740 con 0x7f109811f610 2024-02-23T00:53:24.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.494+0000 7f108e7fc700 10 client.31566.objecter ms_dispatch 0x7f1098001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:53:24.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.494+0000 7f108e7fc700 10 client.31566.objecter _finish_command 9 = system:0 2024-02-23T00:53:24.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.494+0000 7f108e7fc700 20 client.31566.objecter put_session s=0x7f109811b490 osd=4 4 2024-02-23T00:53:24.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.494+0000 7f108e7fc700 15 client.31566.objecter _session_command_op_remove 4 9 2024-02-23T00:53:24.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.521+0000 7f109f559700 10 client.31566.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:53:24.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.521+0000 7f109f559700 15 client.31566.objecter _session_command_op_assign -1 10 2024-02-23T00:53:24.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.521+0000 7f109f559700 20 client.31566.objecter _get_session s=0x7f109811b490 osd=4 3 2024-02-23T00:53:24.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.521+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f109811b490 osd=4 3 2024-02-23T00:53:24.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.521+0000 7f109f559700 20 client.31566.objecter _get_session s=0x7f109811b490 osd=4 3 2024-02-23T00:53:24.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.521+0000 7f109f559700 15 client.31566.objecter _session_command_op_remove -1 10 2024-02-23T00:53:24.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.521+0000 7f109f559700 20 client.31566.objecter get_session s=0x7f109811b490 osd=4 3 2024-02-23T00:53:24.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.521+0000 7f109f559700 15 client.31566.objecter _session_command_op_assign 4 10 2024-02-23T00:53:24.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.521+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f109811b490 osd=4 4 2024-02-23T00:53:24.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.521+0000 7f109f559700 10 client.31566.objecter _send_command 10 2024-02-23T00:53:24.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.521+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f109811b740 con 0x7f109811f610 2024-02-23T00:53:24.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.522+0000 7f108e7fc700 1 -- 172.21.15.47:0/1424677917 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f109811b740 con 0x7f109811f610 2024-02-23T00:53:24.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.522+0000 7f108e7fc700 10 client.31566.objecter ms_dispatch 0x7f1098001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:53:24.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.522+0000 7f108e7fc700 10 client.31566.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:53:24.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.522+0000 7f108e7fc700 20 client.31566.objecter put_session s=0x7f109811b490 osd=4 4 2024-02-23T00:53:24.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.522+0000 7f108e7fc700 15 client.31566.objecter _session_command_op_remove 4 10 2024-02-23T00:53:24.526 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:53:24.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.523+0000 7f109f559700 10 client.31566.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:24.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.523+0000 7f109f559700 15 client.31566.objecter _session_command_op_assign -1 11 2024-02-23T00:53:24.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.523+0000 7f109f559700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f1098125f20 0x7f1098128310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:24.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.523+0000 7f109f559700 20 client.31566.objecter _get_session s=0x7f109811b740 osd=5 3 2024-02-23T00:53:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.523+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f109811b740 osd=5 3 2024-02-23T00:53:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.523+0000 7f109f559700 20 client.31566.objecter _get_session s=0x7f109811b740 osd=5 3 2024-02-23T00:53:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.523+0000 7f109f559700 15 client.31566.objecter _session_command_op_remove -1 11 2024-02-23T00:53:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.523+0000 7f109f559700 20 client.31566.objecter get_session s=0x7f109811b740 osd=5 3 2024-02-23T00:53:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.523+0000 7f109f559700 15 client.31566.objecter _session_command_op_assign 5 11 2024-02-23T00:53:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.523+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f109811b740 osd=5 4 2024-02-23T00:53:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.523+0000 7f109f559700 10 client.31566.objecter _send_command 11 2024-02-23T00:53:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.523+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f109812b110 con 0x7f1098125f20 2024-02-23T00:53:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.523+0000 7f109caf4700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f1098125f20 0x7f1098128310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.523+0000 7f108e7fc700 10 client.31566.objecter ms_handle_connect 0x7f1098125f20 2024-02-23T00:53:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.523+0000 7f109caf4700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f1098125f20 0x7f1098128310 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:24.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.524+0000 7f108e7fc700 1 -- 172.21.15.47:0/1424677917 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f109812b110 con 0x7f1098125f20 2024-02-23T00:53:24.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.524+0000 7f108e7fc700 10 client.31566.objecter ms_dispatch 0x7f1098001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:53:24.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.524+0000 7f108e7fc700 10 client.31566.objecter _finish_command 11 = system:0 2024-02-23T00:53:24.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.524+0000 7f108e7fc700 20 client.31566.objecter put_session s=0x7f109811b740 osd=5 4 2024-02-23T00:53:24.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.524+0000 7f108e7fc700 15 client.31566.objecter _session_command_op_remove 5 11 2024-02-23T00:53:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.539+0000 7f109f559700 10 client.31566.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:53:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.539+0000 7f109f559700 15 client.31566.objecter _session_command_op_assign -1 12 2024-02-23T00:53:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.539+0000 7f109f559700 20 client.31566.objecter _get_session s=0x7f109811b740 osd=5 3 2024-02-23T00:53:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.539+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f109811b740 osd=5 3 2024-02-23T00:53:24.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.539+0000 7f109f559700 20 client.31566.objecter _get_session s=0x7f109811b740 osd=5 3 2024-02-23T00:53:24.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.539+0000 7f109f559700 15 client.31566.objecter _session_command_op_remove -1 12 2024-02-23T00:53:24.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.539+0000 7f109f559700 20 client.31566.objecter get_session s=0x7f109811b740 osd=5 3 2024-02-23T00:53:24.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.539+0000 7f109f559700 15 client.31566.objecter _session_command_op_assign 5 12 2024-02-23T00:53:24.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.539+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f109811b740 osd=5 4 2024-02-23T00:53:24.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.539+0000 7f109f559700 10 client.31566.objecter _send_command 12 2024-02-23T00:53:24.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.539+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f109812b110 con 0x7f1098125f20 2024-02-23T00:53:24.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.541+0000 7f108e7fc700 1 -- 172.21.15.47:0/1424677917 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f109812b110 con 0x7f1098125f20 2024-02-23T00:53:24.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.541+0000 7f108e7fc700 10 client.31566.objecter ms_dispatch 0x7f1098001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:53:24.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.541+0000 7f108e7fc700 10 client.31566.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:53:24.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.541+0000 7f108e7fc700 20 client.31566.objecter put_session s=0x7f109811b740 osd=5 4 2024-02-23T00:53:24.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.541+0000 7f108e7fc700 15 client.31566.objecter _session_command_op_remove 5 12 2024-02-23T00:53:24.544 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:53:24.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.541+0000 7f109f559700 10 client.31566.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:24.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.541+0000 7f109f559700 15 client.31566.objecter _session_command_op_assign -1 13 2024-02-23T00:53:24.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.541+0000 7f109f559700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f109812a460 0x7f109812a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:24.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.541+0000 7f109f559700 20 client.31566.objecter _get_session s=0x7f109812b110 osd=6 3 2024-02-23T00:53:24.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.541+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f109812b110 osd=6 3 2024-02-23T00:53:24.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.541+0000 7f109f559700 20 client.31566.objecter _get_session s=0x7f109812b110 osd=6 3 2024-02-23T00:53:24.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.541+0000 7f109f559700 15 client.31566.objecter _session_command_op_remove -1 13 2024-02-23T00:53:24.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.541+0000 7f109f559700 20 client.31566.objecter get_session s=0x7f109812b110 osd=6 3 2024-02-23T00:53:24.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.541+0000 7f109f559700 15 client.31566.objecter _session_command_op_assign 6 13 2024-02-23T00:53:24.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.541+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f109812b110 osd=6 4 2024-02-23T00:53:24.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.541+0000 7f109f559700 10 client.31566.objecter _send_command 13 2024-02-23T00:53:24.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.541+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f109812e440 con 0x7f109812a460 2024-02-23T00:53:24.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.541+0000 7f109daf6700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f109812a460 0x7f109812a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:24.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.542+0000 7f108e7fc700 10 client.31566.objecter ms_handle_connect 0x7f109812a460 2024-02-23T00:53:24.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.542+0000 7f109daf6700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f109812a460 0x7f109812a8c0 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:24.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.543+0000 7f108e7fc700 1 -- 172.21.15.47:0/1424677917 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f109812e440 con 0x7f109812a460 2024-02-23T00:53:24.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.543+0000 7f108e7fc700 10 client.31566.objecter ms_dispatch 0x7f1098001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:53:24.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.543+0000 7f108e7fc700 10 client.31566.objecter _finish_command 13 = system:0 2024-02-23T00:53:24.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.543+0000 7f108e7fc700 20 client.31566.objecter put_session s=0x7f109812b110 osd=6 4 2024-02-23T00:53:24.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.543+0000 7f108e7fc700 15 client.31566.objecter _session_command_op_remove 6 13 2024-02-23T00:53:24.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.570+0000 7f109f559700 10 client.31566.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:53:24.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.570+0000 7f109f559700 15 client.31566.objecter _session_command_op_assign -1 14 2024-02-23T00:53:24.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.570+0000 7f109f559700 20 client.31566.objecter _get_session s=0x7f109812b110 osd=6 3 2024-02-23T00:53:24.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.570+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f109812b110 osd=6 3 2024-02-23T00:53:24.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.570+0000 7f109f559700 20 client.31566.objecter _get_session s=0x7f109812b110 osd=6 3 2024-02-23T00:53:24.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.570+0000 7f109f559700 15 client.31566.objecter _session_command_op_remove -1 14 2024-02-23T00:53:24.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.570+0000 7f109f559700 20 client.31566.objecter get_session s=0x7f109812b110 osd=6 3 2024-02-23T00:53:24.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.570+0000 7f109f559700 15 client.31566.objecter _session_command_op_assign 6 14 2024-02-23T00:53:24.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.570+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f109812b110 osd=6 4 2024-02-23T00:53:24.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.570+0000 7f109f559700 10 client.31566.objecter _send_command 14 2024-02-23T00:53:24.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.570+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f10981288f0 con 0x7f109812a460 2024-02-23T00:53:24.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.572+0000 7f108e7fc700 1 -- 172.21.15.47:0/1424677917 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f10981288f0 con 0x7f109812a460 2024-02-23T00:53:24.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.572+0000 7f108e7fc700 10 client.31566.objecter ms_dispatch 0x7f1098001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:53:24.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.572+0000 7f108e7fc700 10 client.31566.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:53:24.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.572+0000 7f108e7fc700 20 client.31566.objecter put_session s=0x7f109812b110 osd=6 4 2024-02-23T00:53:24.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.572+0000 7f108e7fc700 15 client.31566.objecter _session_command_op_remove 6 14 2024-02-23T00:53:24.575 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:53:24.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.572+0000 7f109f559700 10 client.31566.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:24.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.572+0000 7f109f559700 15 client.31566.objecter _session_command_op_assign -1 15 2024-02-23T00:53:24.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.572+0000 7f109f559700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f1098128f70 0x7f10981293d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:24.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.572+0000 7f109f559700 20 client.31566.objecter _get_session s=0x7f10981288f0 osd=7 3 2024-02-23T00:53:24.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.572+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f10981288f0 osd=7 3 2024-02-23T00:53:24.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.572+0000 7f109f559700 20 client.31566.objecter _get_session s=0x7f10981288f0 osd=7 3 2024-02-23T00:53:24.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.572+0000 7f109f559700 15 client.31566.objecter _session_command_op_remove -1 15 2024-02-23T00:53:24.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.572+0000 7f109f559700 20 client.31566.objecter get_session s=0x7f10981288f0 osd=7 3 2024-02-23T00:53:24.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.572+0000 7f109f559700 15 client.31566.objecter _session_command_op_assign 7 15 2024-02-23T00:53:24.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.572+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f10981288f0 osd=7 4 2024-02-23T00:53:24.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.572+0000 7f109f559700 10 client.31566.objecter _send_command 15 2024-02-23T00:53:24.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.572+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1098129a60 con 0x7f1098128f70 2024-02-23T00:53:24.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.572+0000 7f109d2f5700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f1098128f70 0x7f10981293d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:24.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.573+0000 7f108e7fc700 10 client.31566.objecter ms_handle_connect 0x7f1098128f70 2024-02-23T00:53:24.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.573+0000 7f109d2f5700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f1098128f70 0x7f10981293d0 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:24.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.574+0000 7f108e7fc700 1 -- 172.21.15.47:0/1424677917 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1098129a60 con 0x7f1098128f70 2024-02-23T00:53:24.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.574+0000 7f108e7fc700 10 client.31566.objecter ms_dispatch 0x7f1098001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:53:24.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.574+0000 7f108e7fc700 10 client.31566.objecter _finish_command 15 = system:0 2024-02-23T00:53:24.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.574+0000 7f108e7fc700 20 client.31566.objecter put_session s=0x7f10981288f0 osd=7 4 2024-02-23T00:53:24.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.574+0000 7f108e7fc700 15 client.31566.objecter _session_command_op_remove 7 15 2024-02-23T00:53:24.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.588+0000 7f109f559700 10 client.31566.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:53:24.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.588+0000 7f109f559700 15 client.31566.objecter _session_command_op_assign -1 16 2024-02-23T00:53:24.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.588+0000 7f109f559700 20 client.31566.objecter _get_session s=0x7f10981288f0 osd=7 3 2024-02-23T00:53:24.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.588+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f10981288f0 osd=7 3 2024-02-23T00:53:24.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.588+0000 7f109f559700 20 client.31566.objecter _get_session s=0x7f10981288f0 osd=7 3 2024-02-23T00:53:24.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.588+0000 7f109f559700 15 client.31566.objecter _session_command_op_remove -1 16 2024-02-23T00:53:24.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.588+0000 7f109f559700 20 client.31566.objecter get_session s=0x7f10981288f0 osd=7 3 2024-02-23T00:53:24.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.588+0000 7f109f559700 15 client.31566.objecter _session_command_op_assign 7 16 2024-02-23T00:53:24.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.588+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f10981288f0 osd=7 4 2024-02-23T00:53:24.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.588+0000 7f109f559700 10 client.31566.objecter _send_command 16 2024-02-23T00:53:24.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.588+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1098129a60 con 0x7f1098128f70 2024-02-23T00:53:24.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f108e7fc700 1 -- 172.21.15.47:0/1424677917 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1098129a60 con 0x7f1098128f70 2024-02-23T00:53:24.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f108e7fc700 10 client.31566.objecter ms_dispatch 0x7f1098001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:53:24.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f108e7fc700 10 client.31566.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:53:24.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f108e7fc700 20 client.31566.objecter put_session s=0x7f10981288f0 osd=7 4 2024-02-23T00:53:24.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f108e7fc700 15 client.31566.objecter _session_command_op_remove 7 16 2024-02-23T00:53:24.594 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:53:24.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 10 librados: watch_flush enter 2024-02-23T00:53:24.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 10 librados: watch_flush exit 2024-02-23T00:53:24.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 10 client.31566.objecter close_session for osd.0 2024-02-23T00:53:24.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f10980fe5a0 msgr2=0x7f10980fe980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:24.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f10980fe5a0 0x7f10980fe980 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:24.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f109810b140 osd=0 1 2024-02-23T00:53:24.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 10 client.31566.objecter close_session for osd.1 2024-02-23T00:53:24.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f1098028540 msgr2=0x7f10980598f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:24.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f1098028540 0x7f10980598f0 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:24.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f1098107050 osd=1 1 2024-02-23T00:53:24.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 10 client.31566.objecter close_session for osd.2 2024-02-23T00:53:24.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f10980594d0 msgr2=0x7f109811e440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:24.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f10980594d0 0x7f109811e440 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:24.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f1098059360 osd=2 1 2024-02-23T00:53:24.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 10 client.31566.objecter close_session for osd.3 2024-02-23T00:53:24.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f109811fab0 msgr2=0x7f1098121ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:24.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f109811fab0 0x7f1098121ea0 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:24.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f109811ba20 osd=3 1 2024-02-23T00:53:24.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 10 client.31566.objecter close_session for osd.4 2024-02-23T00:53:24.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f109811f610 msgr2=0x7f1098125520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:24.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f109811f610 0x7f1098125520 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:24.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f109811b490 osd=4 1 2024-02-23T00:53:24.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 10 client.31566.objecter close_session for osd.5 2024-02-23T00:53:24.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f1098125f20 msgr2=0x7f1098128310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:24.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f1098125f20 0x7f1098128310 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:24.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109daf6700 1 -- 172.21.15.47:0/1424677917 reap_dead start 2024-02-23T00:53:24.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f109811b740 osd=5 1 2024-02-23T00:53:24.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 10 client.31566.objecter close_session for osd.6 2024-02-23T00:53:24.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f109812a460 msgr2=0x7f109812a8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:24.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f109812a460 0x7f109812a8c0 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:24.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f109812b110 osd=6 1 2024-02-23T00:53:24.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 10 client.31566.objecter close_session for osd.7 2024-02-23T00:53:24.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f1098128f70 msgr2=0x7f10981293d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:24.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f1098128f70 0x7f10981293d0 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:24.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109daf6700 1 -- 172.21.15.47:0/1424677917 reap_dead start 2024-02-23T00:53:24.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 20 client.31566.objecter put_session s=0x7f10981288f0 osd=7 1 2024-02-23T00:53:24.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 20 client.31566.objecter shutdown clearing up homeless session... 2024-02-23T00:53:24.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 10 client.31566.objecter successfully canceled tick 2024-02-23T00:53:24.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109daf6700 1 -- 172.21.15.47:0/1424677917 reap_dead start 2024-02-23T00:53:24.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109daf6700 1 -- 172.21.15.47:0/1424677917 reap_dead start 2024-02-23T00:53:24.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109daf6700 1 -- 172.21.15.47:0/1424677917 reap_dead start 2024-02-23T00:53:24.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1084062cc0 msgr2=0x7f1084065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:24.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1084062cc0 0x7f1084065180 secure :-1 s=READY pgs=3157 cs=0 l=1 rev1=1 rx=0x7f10940043d0 tx=0x7f109400b040).stop 2024-02-23T00:53:24.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1098102210 msgr2=0x7f109810a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:24.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.591+0000 7f109f559700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1098102210 0x7f109810a270 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 rx=0x7f1090009610 tx=0x7f1090004c50).stop 2024-02-23T00:53:24.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.592+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 shutdown_connections 2024-02-23T00:53:24.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.592+0000 7f109f559700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1084062cc0 0x7f1084065180 unknown :-1 s=CLOSED pgs=3157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:24.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.592+0000 7f109f559700 1 --2- 172.21.15.47:0/1424677917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1098102210 0x7f109810a270 unknown :-1 s=CLOSED pgs=2608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:24.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.592+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 >> 172.21.15.47:0/1424677917 conn(0x7f10980f2820 msgr2=0x7f10980f4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:24.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.592+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 shutdown_connections 2024-02-23T00:53:24.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.592+0000 7f109f559700 1 -- 172.21.15.47:0/1424677917 wait complete. 2024-02-23T00:53:24.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:24.592+0000 7f109f559700 1 librados: shutdown 2024-02-23T00:53:24.601 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:53:24.602 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:53:24.602 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:53:24.602 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:53:24.602 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:53:24.602 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:53:24.602 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:53:24.602 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:53:24.673 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:22.856739+0000 mgr.y (mgr.14517) 582 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:53:24.674 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:23.308045+0000 mon.a (mon.0) 3316 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 0.3921672767487391}]': finished 2024-02-23T00:53:24.674 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:23.308117+0000 mon.a (mon.0) 3317 : cluster [DBG] osdmap e666: 8 total, 8 up, 6 in 2024-02-23T00:53:24.674 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:23.371115+0000 mgr.y (mgr.14517) 583 : cluster [DBG] pgmap v1007: 184 pgs: 184 active+clean; 5.8 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-23T00:53:24.966 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:53:24.972 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:24.972 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:53:25.073 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:53:25.101 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:25.142 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:53:25.144 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:25.145 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:53:25.169 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:53:25.246 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:53:25.262 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:53:25.262 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:53:25.275 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:25.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:24.313844+0000 mon.a (mon.0) 3318 : cluster [DBG] osdmap e667: 8 total, 8 up, 6 in 2024-02-23T00:53:25.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:24.363832+0000 mon.c (mon.2) 384 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:25.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:24.364332+0000 mon.a (mon.0) 3319 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:25.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:24.396839+0000 mon.a (mon.0) 3320 : audit [DBG] from='client.? 172.21.15.47:0/1424677917' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:53:25.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:25.277325+0000 mon.a (mon.0) 3321 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:53:25.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:25.277401+0000 mon.a (mon.0) 3322 : cluster [DBG] osdmap e668: 8 total, 8 up, 6 in 2024-02-23T00:53:25.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:25.278400+0000 mon.c (mon.2) 385 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-114"}]: dispatch 2024-02-23T00:53:25.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:25.278820+0000 mon.a (mon.0) 3323 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-114"}]: dispatch 2024-02-23T00:53:25.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:25.279224+0000 osd.1 (osd.1) 275 : cluster [DBG] 6.4 starting backfill to osd.6 from (0'0,0'0] MAX to 67'1 2024-02-23T00:53:25.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:24.313844+0000 mon.a (mon.0) 3318 : cluster [DBG] osdmap e667: 8 total, 8 up, 6 in 2024-02-23T00:53:25.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:24.363832+0000 mon.c (mon.2) 384 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:25.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:24.364332+0000 mon.a (mon.0) 3319 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:25.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:24.396839+0000 mon.a (mon.0) 3320 : audit [DBG] from='client.? 172.21.15.47:0/1424677917' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:53:25.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:25.277325+0000 mon.a (mon.0) 3321 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:53:25.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:25.277401+0000 mon.a (mon.0) 3322 : cluster [DBG] osdmap e668: 8 total, 8 up, 6 in 2024-02-23T00:53:25.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:25.278400+0000 mon.c (mon.2) 385 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-114"}]: dispatch 2024-02-23T00:53:25.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:25.278820+0000 mon.a (mon.0) 3323 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-114"}]: dispatch 2024-02-23T00:53:25.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:25.279224+0000 osd.1 (osd.1) 275 : cluster [DBG] 6.4 starting backfill to osd.6 from (0'0,0'0] MAX to 67'1 2024-02-23T00:53:25.627 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:53:25.630 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:25.631 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:53:25.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:24.313844+0000 mon.a (mon.0) 3318 : cluster [DBG] osdmap e667: 8 total, 8 up, 6 in 2024-02-23T00:53:25.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:24.363832+0000 mon.c (mon.2) 384 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:25.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:24.364332+0000 mon.a (mon.0) 3319 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:25.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:24.396839+0000 mon.a (mon.0) 3320 : audit [DBG] from='client.? 172.21.15.47:0/1424677917' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:53:25.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:25.277325+0000 mon.a (mon.0) 3321 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:53:25.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:25.277401+0000 mon.a (mon.0) 3322 : cluster [DBG] osdmap e668: 8 total, 8 up, 6 in 2024-02-23T00:53:25.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:25.278400+0000 mon.c (mon.2) 385 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-114"}]: dispatch 2024-02-23T00:53:25.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:25.278820+0000 mon.a (mon.0) 3323 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-114"}]: dispatch 2024-02-23T00:53:25.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:25.279224+0000 osd.1 (osd.1) 275 : cluster [DBG] 6.4 starting backfill to osd.6 from (0'0,0'0] MAX to 67'1 2024-02-23T00:53:25.732 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:53:25.760 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:25.958 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:53:25.961 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:25.961 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:53:25.975 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.4 dump_historic_ops 2024-02-23T00:53:26.063 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:53:26.089 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:26.108 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:53:26.110 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:26.110 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:53:26.211 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:53:26.241 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:26.290 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:53:26.494 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:53:26.496 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:26.496 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:25.281419+0000 osd.5 (osd.5) 288 : cluster [DBG] 6.0 starting backfill to osd.4 from (0'0,0'0] MAX to 107'2 2024-02-23T00:53:26.497 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:25.371697+0000 mgr.y (mgr.14517) 584 : cluster [DBG] pgmap v1010: 184 pgs: 184 active+clean; 5.8 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-23T00:53:26.497 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:26.274648+0000 mon.a (mon.0) 3324 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:53:26.497 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:26.278696+0000 mon.a (mon.0) 3325 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-114"}]': finished 2024-02-23T00:53:26.497 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:26.278800+0000 mon.a (mon.0) 3326 : cluster [DBG] osdmap e669: 8 total, 8 up, 6 in 2024-02-23T00:53:26.497 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:25.281419+0000 osd.5 (osd.5) 288 : cluster [DBG] 6.0 starting backfill to osd.4 from (0'0,0'0] MAX to 107'2 2024-02-23T00:53:26.497 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:25.371697+0000 mgr.y (mgr.14517) 584 : cluster [DBG] pgmap v1010: 184 pgs: 184 active+clean; 5.8 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-23T00:53:26.497 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:26.274648+0000 mon.a (mon.0) 3324 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:53:26.497 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:26.278696+0000 mon.a (mon.0) 3325 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-114"}]': finished 2024-02-23T00:53:26.497 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:26.278800+0000 mon.a (mon.0) 3326 : cluster [DBG] osdmap e669: 8 total, 8 up, 6 in 2024-02-23T00:53:26.498 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:53:26.568 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:25.281419+0000 osd.5 (osd.5) 288 : cluster [DBG] 6.0 starting backfill to osd.4 from (0'0,0'0] MAX to 107'2 2024-02-23T00:53:26.568 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:25.371697+0000 mgr.y (mgr.14517) 584 : cluster [DBG] pgmap v1010: 184 pgs: 184 active+clean; 5.8 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-23T00:53:26.569 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:26.274648+0000 mon.a (mon.0) 3324 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:53:26.569 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:26.278696+0000 mon.a (mon.0) 3325 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-114"}]': finished 2024-02-23T00:53:26.569 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:26.278800+0000 mon.a (mon.0) 3326 : cluster [DBG] osdmap e669: 8 total, 8 up, 6 in 2024-02-23T00:53:26.599 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:53:26.627 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:26.878 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:53:26.881 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:26.881 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:53:26.982 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:53:27.013 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:27.112 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:53:27.114 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:27.114 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:53:27.215 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:53:27.245 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:27.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.399+0000 7fa4cd81a700 1 -- 172.21.15.47:0/1507068980 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4c80f8d50 msgr2=0x7fa4c80f9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:27.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.399+0000 7fa4cd81a700 1 --2- 172.21.15.47:0/1507068980 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4c80f8d50 0x7fa4c80f9130 secure :-1 s=READY pgs=2769 cs=0 l=1 rev1=1 rx=0x7fa4b8004870 tx=0x7fa4b8009cb0).stop 2024-02-23T00:53:27.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.399+0000 7fa4cd81a700 1 -- 172.21.15.47:0/1507068980 shutdown_connections 2024-02-23T00:53:27.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.399+0000 7fa4cd81a700 1 --2- 172.21.15.47:0/1507068980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c81022b0 0x7fa4c81046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:27.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.399+0000 7fa4cd81a700 1 --2- 172.21.15.47:0/1507068980 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4c80f8d50 0x7fa4c80f9130 unknown :-1 s=CLOSED pgs=2769 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:27.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.399+0000 7fa4cd81a700 1 --2- 172.21.15.47:0/1507068980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4c80f9620 0x7fa4c8101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:27.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.399+0000 7fa4cd81a700 1 -- 172.21.15.47:0/1507068980 >> 172.21.15.47:0/1507068980 conn(0x7fa4c80f2820 msgr2=0x7fa4c80f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:27.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.399+0000 7fa4cd81a700 1 -- 172.21.15.47:0/1507068980 shutdown_connections 2024-02-23T00:53:27.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.399+0000 7fa4cd81a700 1 -- 172.21.15.47:0/1507068980 wait complete. 2024-02-23T00:53:27.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.400+0000 7fa4cd81a700 1 librados: starting msgr at 2024-02-23T00:53:27.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.400+0000 7fa4cd81a700 1 librados: starting objecter 2024-02-23T00:53:27.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.400+0000 7fa4cd81a700 1 Processor -- start 2024-02-23T00:53:27.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.400+0000 7fa4cd81a700 1 -- start start 2024-02-23T00:53:27.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.400+0000 7fa4cd81a700 1 librados: setting wanted keys 2024-02-23T00:53:27.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.400+0000 7fa4cd81a700 1 librados: calling monclient init 2024-02-23T00:53:27.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.400+0000 7fa4cd81a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4c80f8d50 0x7fa4c810dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:27.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.400+0000 7fa4cd81a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c80f9620 0x7fa4c810e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:27.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.400+0000 7fa4cd81a700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4c81022b0 0x7fa4c8108180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:27.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.400+0000 7fa4cd81a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c8108670 con 0x7fa4c80f8d50 2024-02-23T00:53:27.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.400+0000 7fa4cd81a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c81087e0 con 0x7fa4c80f9620 2024-02-23T00:53:27.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.400+0000 7fa4cd81a700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c81089a0 con 0x7fa4c81022b0 2024-02-23T00:53:27.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.401+0000 7fa4c77fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4c81022b0 0x7fa4c8108180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:27.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.401+0000 7fa4c67fc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c80f9620 0x7fa4c810e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:27.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.401+0000 7fa4c77fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4c81022b0 0x7fa4c8108180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:43226/0 (socket says 172.21.15.47:43226) 2024-02-23T00:53:27.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.401+0000 7fa4c77fe700 1 -- 172.21.15.47:0/1457642832 learned_addr learned my addr 172.21.15.47:0/1457642832 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:53:27.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.401+0000 7fa4c6ffd700 1 --2- 172.21.15.47:0/1457642832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4c80f8d50 0x7fa4c810dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:27.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.401+0000 7fa4c67fc700 1 -- 172.21.15.47:0/1457642832 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4c81022b0 msgr2=0x7fa4c8108180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:27.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.401+0000 7fa4c67fc700 1 --2- 172.21.15.47:0/1457642832 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4c81022b0 0x7fa4c8108180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:27.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.401+0000 7fa4c67fc700 1 -- 172.21.15.47:0/1457642832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4c80f8d50 msgr2=0x7fa4c810dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:27.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.401+0000 7fa4c67fc700 1 --2- 172.21.15.47:0/1457642832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4c80f8d50 0x7fa4c810dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:27.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.401+0000 7fa4c67fc700 1 -- 172.21.15.47:0/1457642832 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b8005040 con 0x7fa4c80f9620 2024-02-23T00:53:27.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.401+0000 7fa4c77fe700 1 --2- 172.21.15.47:0/1457642832 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4c81022b0 0x7fa4c8108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:53:27.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.401+0000 7fa4c67fc700 1 --2- 172.21.15.47:0/1457642832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c80f9620 0x7fa4c810e4d0 secure :-1 s=READY pgs=3982 cs=0 l=1 rev1=1 rx=0x7fa4bc002780 tx=0x7fa4bc009e40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:27.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.401+0000 7fa4affff700 10 client.?.objecter ms_handle_connect 0x7fa4c80f9620 2024-02-23T00:53:27.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.401+0000 7fa4affff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:53:27.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.401+0000 7fa4affff700 1 -- 172.21.15.47:0/1457642832 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc007b40 con 0x7fa4c80f9620 2024-02-23T00:53:27.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.401+0000 7fa4cd81a700 1 -- 172.21.15.47:0/1457642832 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4c8108c20 con 0x7fa4c80f9620 2024-02-23T00:53:27.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.401+0000 7fa4affff700 1 -- 172.21.15.47:0/1457642832 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fa4bc00f040 con 0x7fa4c80f9620 2024-02-23T00:53:27.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.401+0000 7fa4affff700 1 -- 172.21.15.47:0/1457642832 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc014650 con 0x7fa4c80f9620 2024-02-23T00:53:27.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.401+0000 7fa4cd81a700 10 client.43882.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:53:27.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.401+0000 7fa4cd81a700 1 -- 172.21.15.47:0/1457642832 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c80f6b40 con 0x7fa4c80f9620 2024-02-23T00:53:27.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.401+0000 7fa4cd81a700 1 librados: init done 2024-02-23T00:53:27.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.402+0000 7fa4cd81a700 1 -- 172.21.15.47:0/1457642832 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4c805fa60 con 0x7fa4c80f9620 2024-02-23T00:53:27.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.403+0000 7fa4affff700 1 -- 172.21.15.47:0/1457642832 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fa4bc007ca0 con 0x7fa4c80f9620 2024-02-23T00:53:27.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.403+0000 7fa4affff700 1 --2- 172.21.15.47:0/1457642832 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa4b0062cc0 0x7fa4b0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:27.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.403+0000 7fa4affff700 1 -- 172.21.15.47:0/1457642832 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(670..670 src has 482..670) v4 ==== 9280+0+0 (secure 0 0 0) 0x7fa4bc07d700 con 0x7fa4c80f9620 2024-02-23T00:53:27.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.403+0000 7fa4affff700 10 client.43882.objecter ms_dispatch 0x7fa4c8001550 osd_map(670..670 src has 482..670) v4 2024-02-23T00:53:27.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.403+0000 7fa4affff700 3 client.43882.objecter handle_osd_map got epochs [670,670] > 0 2024-02-23T00:53:27.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.403+0000 7fa4affff700 3 client.43882.objecter handle_osd_map decoding full epoch 670 2024-02-23T00:53:27.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.403+0000 7fa4c6ffd700 1 --2- 172.21.15.47:0/1457642832 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa4b0062cc0 0x7fa4b0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:27.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.403+0000 7fa4affff700 20 client.43882.objecter dump_active .. 0 homeless 2024-02-23T00:53:27.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.404+0000 7fa4affff700 10 client.43882.objecter ms_handle_connect 0x7fa4b0062cc0 2024-02-23T00:53:27.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.404+0000 7fa4c6ffd700 1 --2- 172.21.15.47:0/1457642832 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa4b0062cc0 0x7fa4b0065180 secure :-1 s=READY pgs=3158 cs=0 l=1 rev1=1 rx=0x7fa4b8004710 tx=0x7fa4b8009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:27.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.408+0000 7fa4affff700 1 -- 172.21.15.47:0/1457642832 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fa4bc04cce0 con 0x7fa4c80f9620 2024-02-23T00:53:27.494 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:53:27.496 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:27.496 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:53:27.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:27.590+0000 7fa4cd81a700 1 -- 172.21.15.47:0/1457642832 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7fa4c81070a0 con 0x7fa4c80f9620 2024-02-23T00:53:27.597 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:53:27.626 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:27.646 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:53:27.649 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:53:27.651 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:27.652 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:53:27.753 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:53:27.787 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:27.924 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:53:27.988 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:53:27.992 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:27.993 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:53:27.994 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:53:27.994 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:53:28.093 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:53:28.122 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:28.174 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:53:28.177 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:28.177 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:53:28.278 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:53:28.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:28.289+0000 7fa4affff700 1 -- 172.21.15.47:0/1457642832 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v671) v1 ==== 91+0+0 (secure 0 0 0) 0x7fa4bc0181b0 con 0x7fa4c80f9620 2024-02-23T00:53:28.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:28.292+0000 7fa4cd81a700 10 librados: watch_flush enter 2024-02-23T00:53:28.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:28.292+0000 7fa4cd81a700 10 librados: watch_flush exit 2024-02-23T00:53:28.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:28.292+0000 7fa4cd81a700 20 client.43882.objecter shutdown clearing up homeless session... 2024-02-23T00:53:28.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:28.292+0000 7fa4cd81a700 10 client.43882.objecter successfully canceled tick 2024-02-23T00:53:28.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:28.292+0000 7fa4cd81a700 1 -- 172.21.15.47:0/1457642832 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa4b0062cc0 msgr2=0x7fa4b0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:28.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:28.292+0000 7fa4cd81a700 1 --2- 172.21.15.47:0/1457642832 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa4b0062cc0 0x7fa4b0065180 secure :-1 s=READY pgs=3158 cs=0 l=1 rev1=1 rx=0x7fa4b8004710 tx=0x7fa4b8009cb0).stop 2024-02-23T00:53:28.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:28.292+0000 7fa4cd81a700 1 -- 172.21.15.47:0/1457642832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c80f9620 msgr2=0x7fa4c810e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:28.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:28.292+0000 7fa4cd81a700 1 --2- 172.21.15.47:0/1457642832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c80f9620 0x7fa4c810e4d0 secure :-1 s=READY pgs=3982 cs=0 l=1 rev1=1 rx=0x7fa4bc002780 tx=0x7fa4bc009e40).stop 2024-02-23T00:53:28.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:28.292+0000 7fa4cd81a700 1 -- 172.21.15.47:0/1457642832 shutdown_connections 2024-02-23T00:53:28.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:28.292+0000 7fa4cd81a700 1 --2- 172.21.15.47:0/1457642832 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa4b0062cc0 0x7fa4b0065180 unknown :-1 s=CLOSED pgs=3158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:28.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:28.292+0000 7fa4cd81a700 1 --2- 172.21.15.47:0/1457642832 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4c81022b0 0x7fa4c8108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:28.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:28.292+0000 7fa4cd81a700 1 --2- 172.21.15.47:0/1457642832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4c80f8d50 0x7fa4c810dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:28.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:28.292+0000 7fa4cd81a700 1 --2- 172.21.15.47:0/1457642832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c80f9620 0x7fa4c810e4d0 unknown :-1 s=CLOSED pgs=3982 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:28.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:28.292+0000 7fa4cd81a700 1 -- 172.21.15.47:0/1457642832 >> 172.21.15.47:0/1457642832 conn(0x7fa4c80f2820 msgr2=0x7fa4c80f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:28.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:28.292+0000 7fa4cd81a700 1 -- 172.21.15.47:0/1457642832 shutdown_connections 2024-02-23T00:53:28.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:28.292+0000 7fa4cd81a700 1 -- 172.21.15.47:0/1457642832 wait complete. 2024-02-23T00:53:28.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:28.293+0000 7fa4cd81a700 1 librados: shutdown 2024-02-23T00:53:28.297 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is unset 2024-02-23T00:53:28.314 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:28.366 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:53:28.373 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:28.373 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:53:28.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:27.280618+0000 mon.a (mon.0) 3327 : cluster [DBG] osdmap e670: 8 total, 8 up, 6 in 2024-02-23T00:53:28.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:27.372060+0000 mgr.y (mgr.14517) 585 : cluster [DBG] pgmap v1013: 152 pgs: 2 peering, 150 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 255 B/s rd, 0 op/s; 349 B/s, 13 objects/s recovering 2024-02-23T00:53:28.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:27.591434+0000 mon.b (mon.1) 381 : audit [INF] from='client.? 172.21.15.47:0/1457642832' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:53:28.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:27.731511+0000 mon.a (mon.0) 3328 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:53:28.473 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:27.280618+0000 mon.a (mon.0) 3327 : cluster [DBG] osdmap e670: 8 total, 8 up, 6 in 2024-02-23T00:53:28.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:27.372060+0000 mgr.y (mgr.14517) 585 : cluster [DBG] pgmap v1013: 152 pgs: 2 peering, 150 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 255 B/s rd, 0 op/s; 349 B/s, 13 objects/s recovering 2024-02-23T00:53:28.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:27.591434+0000 mon.b (mon.1) 381 : audit [INF] from='client.? 172.21.15.47:0/1457642832' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:53:28.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:27.731511+0000 mon.a (mon.0) 3328 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:53:28.474 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:53:28.501 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:28.678 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:27.280618+0000 mon.a (mon.0) 3327 : cluster [DBG] osdmap e670: 8 total, 8 up, 6 in 2024-02-23T00:53:28.678 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:27.372060+0000 mgr.y (mgr.14517) 585 : cluster [DBG] pgmap v1013: 152 pgs: 2 peering, 150 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 255 B/s rd, 0 op/s; 349 B/s, 13 objects/s recovering 2024-02-23T00:53:28.678 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:27.591434+0000 mon.b (mon.1) 381 : audit [INF] from='client.? 172.21.15.47:0/1457642832' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:53:28.678 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:27.731511+0000 mon.a (mon.0) 3328 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:53:28.695 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.5 dump_ops_in_flight 2024-02-23T00:53:28.699 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:53:28.701 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:28.701 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:53:28.802 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:53:28.832 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:28.853 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:53:28.868 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:28.871 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:53:28.960 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:53:28.972 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:53:28.998 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:29.547 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:53:29.550 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:29.550 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:53:29.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:28.278609+0000 mon.a (mon.0) 3329 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:53:29.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:28.286523+0000 mon.a (mon.0) 3330 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:53:29.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:28.286600+0000 mon.a (mon.0) 3331 : cluster [DBG] osdmap e671: 8 total, 8 up, 6 in 2024-02-23T00:53:29.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:28.287252+0000 mon.c (mon.2) 386 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:29.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:28.287629+0000 mon.a (mon.0) 3332 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:29.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:28.311759+0000 osd.5 (osd.5) 289 : cluster [DBG] 2.c scrub starts 2024-02-23T00:53:29.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:28.314655+0000 osd.5 (osd.5) 290 : cluster [DBG] 2.c scrub ok 2024-02-23T00:53:29.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:28.278609+0000 mon.a (mon.0) 3329 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:53:29.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:28.286523+0000 mon.a (mon.0) 3330 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:53:29.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:28.286600+0000 mon.a (mon.0) 3331 : cluster [DBG] osdmap e671: 8 total, 8 up, 6 in 2024-02-23T00:53:29.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:28.287252+0000 mon.c (mon.2) 386 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:29.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:28.287629+0000 mon.a (mon.0) 3332 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:29.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:28.311759+0000 osd.5 (osd.5) 289 : cluster [DBG] 2.c scrub starts 2024-02-23T00:53:29.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:28.314655+0000 osd.5 (osd.5) 290 : cluster [DBG] 2.c scrub ok 2024-02-23T00:53:29.651 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:53:29.680 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:29.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:28.278609+0000 mon.a (mon.0) 3329 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:53:29.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:28.286523+0000 mon.a (mon.0) 3330 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:53:29.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:28.286600+0000 mon.a (mon.0) 3331 : cluster [DBG] osdmap e671: 8 total, 8 up, 6 in 2024-02-23T00:53:29.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:28.287252+0000 mon.c (mon.2) 386 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:29.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:28.287629+0000 mon.a (mon.0) 3332 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:29.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:28.311759+0000 osd.5 (osd.5) 289 : cluster [DBG] 2.c scrub starts 2024-02-23T00:53:29.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:28.314655+0000 osd.5 (osd.5) 290 : cluster [DBG] 2.c scrub ok 2024-02-23T00:53:29.924 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6, 7] out_osds: [2, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-23T00:53:29.924 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-23T00:53:29.924 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-02-23T00:53:29.924 INFO:tasks.thrashosds.thrasher:Setting osd 3 primary_affinity to 0.000000 2024-02-23T00:53:29.925 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd primary-affinity 3 0 2024-02-23T00:53:30.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.509+0000 7f167980d700 1 -- 172.21.15.47:0/1181831682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16740fe530 msgr2=0x7f16740f63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:30.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.509+0000 7f167980d700 1 --2- 172.21.15.47:0/1181831682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16740fe530 0x7f16740f63b0 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 rx=0x7f165c004870 tx=0x7f165c009cb0).stop 2024-02-23T00:53:30.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.509+0000 7f167980d700 1 -- 172.21.15.47:0/1181831682 shutdown_connections 2024-02-23T00:53:30.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.509+0000 7f167980d700 1 --2- 172.21.15.47:0/1181831682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16740f6a90 0x7f1674104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:30.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.509+0000 7f167980d700 1 --2- 172.21.15.47:0/1181831682 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f16740fdbd0 0x7f16740fdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:30.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.509+0000 7f167980d700 1 --2- 172.21.15.47:0/1181831682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16740fe530 0x7f16740f63b0 unknown :-1 s=CLOSED pgs=2609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:30.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.509+0000 7f167980d700 1 -- 172.21.15.47:0/1181831682 >> 172.21.15.47:0/1181831682 conn(0x7f167406c370 msgr2=0x7f167406c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:30.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.509+0000 7f167980d700 1 -- 172.21.15.47:0/1181831682 shutdown_connections 2024-02-23T00:53:30.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.509+0000 7f167980d700 1 -- 172.21.15.47:0/1181831682 wait complete. 2024-02-23T00:53:30.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.509+0000 7f167980d700 1 librados: starting msgr at 2024-02-23T00:53:30.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.509+0000 7f167980d700 1 librados: starting objecter 2024-02-23T00:53:30.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.510+0000 7f167980d700 1 Processor -- start 2024-02-23T00:53:30.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.510+0000 7f167980d700 1 -- start start 2024-02-23T00:53:30.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.510+0000 7f167980d700 1 librados: setting wanted keys 2024-02-23T00:53:30.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.510+0000 7f167980d700 1 librados: calling monclient init 2024-02-23T00:53:30.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.510+0000 7f167980d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16740f6a90 0x7f16740f7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:30.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.510+0000 7f167980d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16740fdbd0 0x7f16740f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:30.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.510+0000 7f167980d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f16740fe530 0x7f16740f8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:30.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.510+0000 7f167980d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1674116490 con 0x7f16740f6a90 2024-02-23T00:53:30.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.510+0000 7f167980d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1674116600 con 0x7f16740fdbd0 2024-02-23T00:53:30.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.510+0000 7f167980d700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16741167c0 con 0x7f16740fe530 2024-02-23T00:53:30.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.510+0000 7f1672ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16740f6a90 0x7f16740f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:30.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.510+0000 7f1672ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16740f6a90 0x7f16740f7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52310/0 (socket says 172.21.15.47:52310) 2024-02-23T00:53:30.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.510+0000 7f1672ffd700 1 -- 172.21.15.47:0/123394742 learned_addr learned my addr 172.21.15.47:0/123394742 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:53:30.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.510+0000 7f16737fe700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f16740fe530 0x7f16740f8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:30.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.510+0000 7f16727fc700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16740fdbd0 0x7f16740f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:30.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.510+0000 7f1672ffd700 1 -- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f16740fe530 msgr2=0x7f16740f8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:30.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.510+0000 7f1672ffd700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f16740fe530 0x7f16740f8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:30.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.510+0000 7f1672ffd700 1 -- 172.21.15.47:0/123394742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16740fdbd0 msgr2=0x7f16740f7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:30.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.510+0000 7f1672ffd700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16740fdbd0 0x7f16740f7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:30.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.511+0000 7f1672ffd700 1 -- 172.21.15.47:0/123394742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1664005040 con 0x7f16740f6a90 2024-02-23T00:53:30.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.511+0000 7f16727fc700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16740fdbd0 0x7f16740f7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:53:30.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.511+0000 7f16737fe700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f16740fe530 0x7f16740f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:53:30.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.511+0000 7f1672ffd700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16740f6a90 0x7f16740f7680 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 rx=0x7f16640048f0 tx=0x7f1664009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:30.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.511+0000 7f166bfff700 10 client.?.objecter ms_handle_connect 0x7f16740f6a90 2024-02-23T00:53:30.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.511+0000 7f166bfff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:53:30.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.511+0000 7f166bfff700 1 -- 172.21.15.47:0/123394742 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1664007b00 con 0x7f16740f6a90 2024-02-23T00:53:30.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.511+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f165c005040 con 0x7f16740f6a90 2024-02-23T00:53:30.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.511+0000 7f166bfff700 1 -- 172.21.15.47:0/123394742 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f166400f040 con 0x7f16740f6a90 2024-02-23T00:53:30.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.511+0000 7f166bfff700 1 -- 172.21.15.47:0/123394742 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1664014890 con 0x7f16740f6a90 2024-02-23T00:53:30.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.511+0000 7f167980d700 10 client.31578.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:53:30.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.511+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1674116e00 con 0x7f16740f6a90 2024-02-23T00:53:30.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.511+0000 7f167980d700 1 librados: init done 2024-02-23T00:53:30.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.512+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f167405f930 con 0x7f16740f6a90 2024-02-23T00:53:30.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.513+0000 7f166bfff700 1 -- 172.21.15.47:0/123394742 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f1664014cc0 con 0x7f16740f6a90 2024-02-23T00:53:30.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.513+0000 7f166bfff700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1660062cc0 0x7f1660065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:30.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.513+0000 7f166bfff700 1 -- 172.21.15.47:0/123394742 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(673..673 src has 482..673) v4 ==== 9661+0+0 (secure 0 0 0) 0x7f166407d790 con 0x7f16740f6a90 2024-02-23T00:53:30.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.513+0000 7f166bfff700 10 client.31578.objecter ms_dispatch 0x7f1674001550 osd_map(673..673 src has 482..673) v4 2024-02-23T00:53:30.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.513+0000 7f166bfff700 3 client.31578.objecter handle_osd_map got epochs [673,673] > 0 2024-02-23T00:53:30.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.513+0000 7f166bfff700 3 client.31578.objecter handle_osd_map decoding full epoch 673 2024-02-23T00:53:30.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.513+0000 7f16727fc700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1660062cc0 0x7f1660065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:30.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.513+0000 7f166bfff700 20 client.31578.objecter dump_active .. 0 homeless 2024-02-23T00:53:30.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.513+0000 7f166bfff700 1 -- 172.21.15.47:0/123394742 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v673) v1 ==== 54+0+16 (secure 0 0 0) 0x7f166407dc30 con 0x7f16740f6a90 2024-02-23T00:53:30.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.513+0000 7f167980d700 10 client.31578.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:30.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.513+0000 7f167980d700 15 client.31578.objecter _session_command_op_assign -1 1 2024-02-23T00:53:30.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.513+0000 7f167980d700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f16740feee0 0x7f16740ff300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:30.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.513+0000 7f167980d700 20 client.31578.objecter _get_session s=0x7f167405ee80 osd=0 3 2024-02-23T00:53:30.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.513+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f167405ee80 osd=0 3 2024-02-23T00:53:30.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.513+0000 7f167980d700 20 client.31578.objecter _get_session s=0x7f167405ee80 osd=0 3 2024-02-23T00:53:30.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.513+0000 7f167980d700 15 client.31578.objecter _session_command_op_remove -1 1 2024-02-23T00:53:30.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.513+0000 7f167980d700 20 client.31578.objecter get_session s=0x7f167405ee80 osd=0 3 2024-02-23T00:53:30.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.513+0000 7f167980d700 15 client.31578.objecter _session_command_op_assign 0 1 2024-02-23T00:53:30.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.513+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f167405ee80 osd=0 4 2024-02-23T00:53:30.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.513+0000 7f167980d700 10 client.31578.objecter _send_command 1 2024-02-23T00:53:30.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.513+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1674106e90 con 0x7f16740feee0 2024-02-23T00:53:30.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.514+0000 7f166bfff700 10 client.31578.objecter ms_handle_connect 0x7f1660062cc0 2024-02-23T00:53:30.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.514+0000 7f16727fc700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1660062cc0 0x7f1660065180 secure :-1 s=READY pgs=3159 cs=0 l=1 rev1=1 rx=0x7f165c004ee0 tx=0x7f165c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:30.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.514+0000 7f16737fe700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f16740feee0 0x7f16740ff300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:30.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.514+0000 7f16737fe700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f16740feee0 0x7f16740ff300 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:30.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.514+0000 7f166bfff700 10 client.31578.objecter ms_handle_connect 0x7f16740feee0 2024-02-23T00:53:30.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.515+0000 7f166bfff700 1 -- 172.21.15.47:0/123394742 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1674106e90 con 0x7f16740feee0 2024-02-23T00:53:30.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.515+0000 7f166bfff700 10 client.31578.objecter ms_dispatch 0x7f1674001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:53:30.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.515+0000 7f166bfff700 10 client.31578.objecter _finish_command 1 = system:0 2024-02-23T00:53:30.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.515+0000 7f166bfff700 20 client.31578.objecter put_session s=0x7f167405ee80 osd=0 4 2024-02-23T00:53:30.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.515+0000 7f166bfff700 15 client.31578.objecter _session_command_op_remove 0 1 2024-02-23T00:53:30.531 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:53:30.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.531+0000 7f167980d700 10 client.31578.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:53:30.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.531+0000 7f167980d700 15 client.31578.objecter _session_command_op_assign -1 2 2024-02-23T00:53:30.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.531+0000 7f167980d700 20 client.31578.objecter _get_session s=0x7f167405ee80 osd=0 3 2024-02-23T00:53:30.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.531+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f167405ee80 osd=0 3 2024-02-23T00:53:30.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.531+0000 7f167980d700 20 client.31578.objecter _get_session s=0x7f167405ee80 osd=0 3 2024-02-23T00:53:30.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.531+0000 7f167980d700 15 client.31578.objecter _session_command_op_remove -1 2 2024-02-23T00:53:30.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.531+0000 7f167980d700 20 client.31578.objecter get_session s=0x7f167405ee80 osd=0 3 2024-02-23T00:53:30.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.531+0000 7f167980d700 15 client.31578.objecter _session_command_op_assign 0 2 2024-02-23T00:53:30.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.531+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f167405ee80 osd=0 4 2024-02-23T00:53:30.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.531+0000 7f167980d700 10 client.31578.objecter _send_command 2 2024-02-23T00:53:30.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.531+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f16740273c0 con 0x7f16740feee0 2024-02-23T00:53:30.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.532+0000 7f166bfff700 1 -- 172.21.15.47:0/123394742 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f16740273c0 con 0x7f16740feee0 2024-02-23T00:53:30.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.532+0000 7f166bfff700 10 client.31578.objecter ms_dispatch 0x7f1674001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:53:30.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.532+0000 7f166bfff700 10 client.31578.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:53:30.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.532+0000 7f166bfff700 20 client.31578.objecter put_session s=0x7f167405ee80 osd=0 4 2024-02-23T00:53:30.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.532+0000 7f166bfff700 15 client.31578.objecter _session_command_op_remove 0 2 2024-02-23T00:53:30.536 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:53:30.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.533+0000 7f167980d700 10 client.31578.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:30.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.533+0000 7f167980d700 15 client.31578.objecter _session_command_op_assign -1 3 2024-02-23T00:53:30.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.533+0000 7f167980d700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f16740598f0 0x7f1674059d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:30.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.533+0000 7f167980d700 20 client.31578.objecter _get_session s=0x7f1674028710 osd=1 3 2024-02-23T00:53:30.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.533+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f1674028710 osd=1 3 2024-02-23T00:53:30.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.533+0000 7f167980d700 20 client.31578.objecter _get_session s=0x7f1674028710 osd=1 3 2024-02-23T00:53:30.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.533+0000 7f167980d700 15 client.31578.objecter _session_command_op_remove -1 3 2024-02-23T00:53:30.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.533+0000 7f167980d700 20 client.31578.objecter get_session s=0x7f1674028710 osd=1 3 2024-02-23T00:53:30.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.533+0000 7f167980d700 15 client.31578.objecter _session_command_op_assign 1 3 2024-02-23T00:53:30.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.533+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f1674028710 osd=1 4 2024-02-23T00:53:30.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.533+0000 7f167980d700 10 client.31578.objecter _send_command 3 2024-02-23T00:53:30.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.533+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f167411d7a0 con 0x7f16740598f0 2024-02-23T00:53:30.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.533+0000 7f1672ffd700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f16740598f0 0x7f1674059d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:30.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.534+0000 7f166bfff700 10 client.31578.objecter ms_handle_connect 0x7f16740598f0 2024-02-23T00:53:30.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.534+0000 7f1672ffd700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f16740598f0 0x7f1674059d10 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:30.538 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:53:30] "GET /metrics HTTP/1.1" 200 191182 "" "Prometheus/2.33.4" 2024-02-23T00:53:30.538 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:28.442311+0000 osd.6 (osd.6) 255 : cluster [DBG] 18.a scrub starts 2024-02-23T00:53:30.538 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:28.443172+0000 osd.6 (osd.6) 256 : cluster [DBG] 18.a scrub ok 2024-02-23T00:53:30.538 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:29.190365+0000 osd.3 (osd.3) 317 : cluster [DBG] 4.11 scrub starts 2024-02-23T00:53:30.538 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:29.192154+0000 osd.3 (osd.3) 318 : cluster [DBG] 4.11 scrub ok 2024-02-23T00:53:30.538 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:29.288175+0000 mon.a (mon.0) 3333 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:53:30.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:29.288386+0000 mon.a (mon.0) 3334 : cluster [DBG] osdmap e672: 8 total, 8 up, 6 in 2024-02-23T00:53:30.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:29.294865+0000 mon.c (mon.2) 387 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:30.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:29.302390+0000 mon.a (mon.0) 3335 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:30.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:29.372566+0000 mgr.y (mgr.14517) 586 : cluster [DBG] pgmap v1016: 184 pgs: 32 unknown, 2 peering, 150 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 255 B/s rd, 0 op/s; 349 B/s, 13 objects/s recovering 2024-02-23T00:53:30.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:29.499306+0000 osd.4 (osd.4) 279 : cluster [DBG] 2.13 scrub starts 2024-02-23T00:53:30.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:29.500105+0000 osd.4 (osd.4) 280 : cluster [DBG] 2.13 scrub ok 2024-02-23T00:53:30.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:30.198029+0000 osd.3 (osd.3) 319 : cluster [DBG] 4.14 scrub starts 2024-02-23T00:53:30.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:30.199994+0000 osd.3 (osd.3) 320 : cluster [DBG] 4.14 scrub ok 2024-02-23T00:53:30.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:30.279370+0000 mon.a (mon.0) 3336 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:53:30.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:30.279447+0000 mon.a (mon.0) 3337 : cluster [DBG] osdmap e673: 8 total, 8 up, 6 in 2024-02-23T00:53:30.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:30.282280+0000 mon.c (mon.2) 388 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-116"}]: dispatch 2024-02-23T00:53:30.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:30.287901+0000 mon.a (mon.0) 3338 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-116"}]: dispatch 2024-02-23T00:53:30.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:28.442311+0000 osd.6 (osd.6) 255 : cluster [DBG] 18.a scrub starts 2024-02-23T00:53:30.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:28.443172+0000 osd.6 (osd.6) 256 : cluster [DBG] 18.a scrub ok 2024-02-23T00:53:30.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:29.190365+0000 osd.3 (osd.3) 317 : cluster [DBG] 4.11 scrub starts 2024-02-23T00:53:30.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:29.192154+0000 osd.3 (osd.3) 318 : cluster [DBG] 4.11 scrub ok 2024-02-23T00:53:30.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:29.288175+0000 mon.a (mon.0) 3333 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:53:30.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:29.288386+0000 mon.a (mon.0) 3334 : cluster [DBG] osdmap e672: 8 total, 8 up, 6 in 2024-02-23T00:53:30.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:29.294865+0000 mon.c (mon.2) 387 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:30.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:29.302390+0000 mon.a (mon.0) 3335 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:30.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:29.372566+0000 mgr.y (mgr.14517) 586 : cluster [DBG] pgmap v1016: 184 pgs: 32 unknown, 2 peering, 150 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 255 B/s rd, 0 op/s; 349 B/s, 13 objects/s recovering 2024-02-23T00:53:30.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:29.499306+0000 osd.4 (osd.4) 279 : cluster [DBG] 2.13 scrub starts 2024-02-23T00:53:30.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:29.500105+0000 osd.4 (osd.4) 280 : cluster [DBG] 2.13 scrub ok 2024-02-23T00:53:30.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:30.198029+0000 osd.3 (osd.3) 319 : cluster [DBG] 4.14 scrub starts 2024-02-23T00:53:30.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:30.199994+0000 osd.3 (osd.3) 320 : cluster [DBG] 4.14 scrub ok 2024-02-23T00:53:30.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:30.279370+0000 mon.a (mon.0) 3336 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:53:30.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:30.279447+0000 mon.a (mon.0) 3337 : cluster [DBG] osdmap e673: 8 total, 8 up, 6 in 2024-02-23T00:53:30.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:30.282280+0000 mon.c (mon.2) 388 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-116"}]: dispatch 2024-02-23T00:53:30.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:30.287901+0000 mon.a (mon.0) 3338 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-116"}]: dispatch 2024-02-23T00:53:30.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.534+0000 7f166bfff700 1 -- 172.21.15.47:0/123394742 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f167411d7a0 con 0x7f16740598f0 2024-02-23T00:53:30.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.534+0000 7f166bfff700 10 client.31578.objecter ms_dispatch 0x7f1674001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:53:30.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.534+0000 7f166bfff700 10 client.31578.objecter _finish_command 3 = system:0 2024-02-23T00:53:30.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.534+0000 7f166bfff700 20 client.31578.objecter put_session s=0x7f1674028710 osd=1 4 2024-02-23T00:53:30.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.534+0000 7f166bfff700 15 client.31578.objecter _session_command_op_remove 1 3 2024-02-23T00:53:30.542 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:30.542 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:53:30.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.552+0000 7f167980d700 10 client.31578.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:53:30.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.552+0000 7f167980d700 15 client.31578.objecter _session_command_op_assign -1 4 2024-02-23T00:53:30.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.552+0000 7f167980d700 20 client.31578.objecter _get_session s=0x7f1674028710 osd=1 3 2024-02-23T00:53:30.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.552+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f1674028710 osd=1 3 2024-02-23T00:53:30.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.552+0000 7f167980d700 20 client.31578.objecter _get_session s=0x7f1674028710 osd=1 3 2024-02-23T00:53:30.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.552+0000 7f167980d700 15 client.31578.objecter _session_command_op_remove -1 4 2024-02-23T00:53:30.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.552+0000 7f167980d700 20 client.31578.objecter get_session s=0x7f1674028710 osd=1 3 2024-02-23T00:53:30.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.552+0000 7f167980d700 15 client.31578.objecter _session_command_op_assign 1 4 2024-02-23T00:53:30.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.552+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f1674028710 osd=1 4 2024-02-23T00:53:30.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.552+0000 7f167980d700 10 client.31578.objecter _send_command 4 2024-02-23T00:53:30.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.552+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f1674059360 con 0x7f16740598f0 2024-02-23T00:53:30.555 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:53:30.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.553+0000 7f166bfff700 1 -- 172.21.15.47:0/123394742 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f1674059360 con 0x7f16740598f0 2024-02-23T00:53:30.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.553+0000 7f166bfff700 10 client.31578.objecter ms_dispatch 0x7f1674001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:53:30.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.553+0000 7f166bfff700 10 client.31578.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:53:30.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.553+0000 7f166bfff700 20 client.31578.objecter put_session s=0x7f1674028710 osd=1 4 2024-02-23T00:53:30.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.553+0000 7f166bfff700 15 client.31578.objecter _session_command_op_remove 1 4 2024-02-23T00:53:30.556 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:53:30.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.553+0000 7f167980d700 10 client.31578.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:30.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.553+0000 7f167980d700 15 client.31578.objecter _session_command_op_assign -1 5 2024-02-23T00:53:30.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.553+0000 7f167980d700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f167411c1c0 0x7f167411c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:30.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.553+0000 7f167980d700 20 client.31578.objecter _get_session s=0x7f1674027d60 osd=2 3 2024-02-23T00:53:30.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.553+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f1674027d60 osd=2 3 2024-02-23T00:53:30.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.553+0000 7f167980d700 20 client.31578.objecter _get_session s=0x7f1674027d60 osd=2 3 2024-02-23T00:53:30.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.553+0000 7f167980d700 15 client.31578.objecter _session_command_op_remove -1 5 2024-02-23T00:53:30.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.553+0000 7f167980d700 20 client.31578.objecter get_session s=0x7f1674027d60 osd=2 3 2024-02-23T00:53:30.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.553+0000 7f167980d700 15 client.31578.objecter _session_command_op_assign 2 5 2024-02-23T00:53:30.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.553+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f1674027d60 osd=2 4 2024-02-23T00:53:30.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.553+0000 7f167980d700 10 client.31578.objecter _send_command 5 2024-02-23T00:53:30.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.553+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f16740283e0 con 0x7f167411c1c0 2024-02-23T00:53:30.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.553+0000 7f16727fc700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f167411c1c0 0x7f167411c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:30.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.553+0000 7f166bfff700 10 client.31578.objecter ms_handle_connect 0x7f167411c1c0 2024-02-23T00:53:30.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.553+0000 7f16727fc700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f167411c1c0 0x7f167411c620 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:30.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.554+0000 7f166bfff700 1 -- 172.21.15.47:0/123394742 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f16740283e0 con 0x7f167411c1c0 2024-02-23T00:53:30.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.554+0000 7f166bfff700 10 client.31578.objecter ms_dispatch 0x7f1674001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:53:30.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.554+0000 7f166bfff700 10 client.31578.objecter _finish_command 5 = system:0 2024-02-23T00:53:30.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.554+0000 7f166bfff700 20 client.31578.objecter put_session s=0x7f1674027d60 osd=2 4 2024-02-23T00:53:30.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.554+0000 7f166bfff700 15 client.31578.objecter _session_command_op_remove 2 5 2024-02-23T00:53:30.559 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:28.442311+0000 osd.6 (osd.6) 255 : cluster [DBG] 18.a scrub starts 2024-02-23T00:53:30.559 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:28.443172+0000 osd.6 (osd.6) 256 : cluster [DBG] 18.a scrub ok 2024-02-23T00:53:30.559 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:29.190365+0000 osd.3 (osd.3) 317 : cluster [DBG] 4.11 scrub starts 2024-02-23T00:53:30.559 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:29.192154+0000 osd.3 (osd.3) 318 : cluster [DBG] 4.11 scrub ok 2024-02-23T00:53:30.559 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:29.288175+0000 mon.a (mon.0) 3333 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:53:30.559 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:29.288386+0000 mon.a (mon.0) 3334 : cluster [DBG] osdmap e672: 8 total, 8 up, 6 in 2024-02-23T00:53:30.559 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:29.294865+0000 mon.c (mon.2) 387 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:30.559 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:29.302390+0000 mon.a (mon.0) 3335 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:30.559 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:29.372566+0000 mgr.y (mgr.14517) 586 : cluster [DBG] pgmap v1016: 184 pgs: 32 unknown, 2 peering, 150 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 255 B/s rd, 0 op/s; 349 B/s, 13 objects/s recovering 2024-02-23T00:53:30.559 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:29.499306+0000 osd.4 (osd.4) 279 : cluster [DBG] 2.13 scrub starts 2024-02-23T00:53:30.559 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:29.500105+0000 osd.4 (osd.4) 280 : cluster [DBG] 2.13 scrub ok 2024-02-23T00:53:30.560 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:30.198029+0000 osd.3 (osd.3) 319 : cluster [DBG] 4.14 scrub starts 2024-02-23T00:53:30.560 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:30.199994+0000 osd.3 (osd.3) 320 : cluster [DBG] 4.14 scrub ok 2024-02-23T00:53:30.560 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:30.279370+0000 mon.a (mon.0) 3336 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:53:30.560 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:30.279447+0000 mon.a (mon.0) 3337 : cluster [DBG] osdmap e673: 8 total, 8 up, 6 in 2024-02-23T00:53:30.560 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:30.282280+0000 mon.c (mon.2) 388 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-116"}]: dispatch 2024-02-23T00:53:30.560 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:30.287901+0000 mon.a (mon.0) 3338 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-116"}]: dispatch 2024-02-23T00:53:30.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.579+0000 7f167980d700 10 client.31578.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:53:30.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.579+0000 7f167980d700 15 client.31578.objecter _session_command_op_assign -1 6 2024-02-23T00:53:30.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.579+0000 7f167980d700 20 client.31578.objecter _get_session s=0x7f1674027d60 osd=2 3 2024-02-23T00:53:30.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.579+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f1674027d60 osd=2 3 2024-02-23T00:53:30.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.579+0000 7f167980d700 20 client.31578.objecter _get_session s=0x7f1674027d60 osd=2 3 2024-02-23T00:53:30.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.579+0000 7f167980d700 15 client.31578.objecter _session_command_op_remove -1 6 2024-02-23T00:53:30.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.579+0000 7f167980d700 20 client.31578.objecter get_session s=0x7f1674027d60 osd=2 3 2024-02-23T00:53:30.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.579+0000 7f167980d700 15 client.31578.objecter _session_command_op_assign 2 6 2024-02-23T00:53:30.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.579+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f1674027d60 osd=2 4 2024-02-23T00:53:30.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.579+0000 7f167980d700 10 client.31578.objecter _send_command 6 2024-02-23T00:53:30.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.579+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f1674028580 con 0x7f167411c1c0 2024-02-23T00:53:30.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.581+0000 7f166bfff700 1 -- 172.21.15.47:0/123394742 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f1674028580 con 0x7f167411c1c0 2024-02-23T00:53:30.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.581+0000 7f166bfff700 10 client.31578.objecter ms_dispatch 0x7f1674001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:53:30.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.581+0000 7f166bfff700 10 client.31578.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:53:30.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.581+0000 7f166bfff700 20 client.31578.objecter put_session s=0x7f1674027d60 osd=2 4 2024-02-23T00:53:30.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.581+0000 7f166bfff700 15 client.31578.objecter _session_command_op_remove 2 6 2024-02-23T00:53:30.584 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:53:30.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.581+0000 7f167980d700 10 client.31578.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:30.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.581+0000 7f167980d700 15 client.31578.objecter _session_command_op_assign -1 7 2024-02-23T00:53:30.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.581+0000 7f167980d700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f167411d2e0 0x7f1674121e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:30.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.581+0000 7f167980d700 20 client.31578.objecter _get_session s=0x7f1674027a10 osd=3 3 2024-02-23T00:53:30.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.581+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f1674027a10 osd=3 3 2024-02-23T00:53:30.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.581+0000 7f167980d700 20 client.31578.objecter _get_session s=0x7f1674027a10 osd=3 3 2024-02-23T00:53:30.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.581+0000 7f167980d700 15 client.31578.objecter _session_command_op_remove -1 7 2024-02-23T00:53:30.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.581+0000 7f167980d700 20 client.31578.objecter get_session s=0x7f1674027a10 osd=3 3 2024-02-23T00:53:30.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.581+0000 7f167980d700 15 client.31578.objecter _session_command_op_assign 3 7 2024-02-23T00:53:30.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.581+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f1674027a10 osd=3 4 2024-02-23T00:53:30.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.581+0000 7f167980d700 10 client.31578.objecter _send_command 7 2024-02-23T00:53:30.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.581+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1674027b80 con 0x7f167411d2e0 2024-02-23T00:53:30.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.581+0000 7f16737fe700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f167411d2e0 0x7f1674121e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:30.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.582+0000 7f166bfff700 10 client.31578.objecter ms_handle_connect 0x7f167411d2e0 2024-02-23T00:53:30.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.582+0000 7f16737fe700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f167411d2e0 0x7f1674121e40 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:30.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.582+0000 7f166bfff700 1 -- 172.21.15.47:0/123394742 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1674027b80 con 0x7f167411d2e0 2024-02-23T00:53:30.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.582+0000 7f166bfff700 10 client.31578.objecter ms_dispatch 0x7f1674001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:53:30.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.582+0000 7f166bfff700 10 client.31578.objecter _finish_command 7 = system:0 2024-02-23T00:53:30.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.582+0000 7f166bfff700 20 client.31578.objecter put_session s=0x7f1674027a10 osd=3 4 2024-02-23T00:53:30.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.582+0000 7f166bfff700 15 client.31578.objecter _session_command_op_remove 3 7 2024-02-23T00:53:30.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.599+0000 7f167980d700 10 client.31578.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:53:30.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.599+0000 7f167980d700 15 client.31578.objecter _session_command_op_assign -1 8 2024-02-23T00:53:30.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.599+0000 7f167980d700 20 client.31578.objecter _get_session s=0x7f1674027a10 osd=3 3 2024-02-23T00:53:30.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.599+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f1674027a10 osd=3 3 2024-02-23T00:53:30.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.599+0000 7f167980d700 20 client.31578.objecter _get_session s=0x7f1674027a10 osd=3 3 2024-02-23T00:53:30.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.599+0000 7f167980d700 15 client.31578.objecter _session_command_op_remove -1 8 2024-02-23T00:53:30.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.599+0000 7f167980d700 20 client.31578.objecter get_session s=0x7f1674027a10 osd=3 3 2024-02-23T00:53:30.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.599+0000 7f167980d700 15 client.31578.objecter _session_command_op_assign 3 8 2024-02-23T00:53:30.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.599+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f1674027a10 osd=3 4 2024-02-23T00:53:30.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.599+0000 7f167980d700 10 client.31578.objecter _send_command 8 2024-02-23T00:53:30.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.599+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f167411cd50 con 0x7f167411d2e0 2024-02-23T00:53:30.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.600+0000 7f166bfff700 1 -- 172.21.15.47:0/123394742 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f167411cd50 con 0x7f167411d2e0 2024-02-23T00:53:30.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.600+0000 7f166bfff700 10 client.31578.objecter ms_dispatch 0x7f1674001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:53:30.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.600+0000 7f166bfff700 10 client.31578.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:53:30.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.600+0000 7f166bfff700 20 client.31578.objecter put_session s=0x7f1674027a10 osd=3 4 2024-02-23T00:53:30.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.600+0000 7f166bfff700 15 client.31578.objecter _session_command_op_remove 3 8 2024-02-23T00:53:30.604 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:53:30.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.600+0000 7f167980d700 10 client.31578.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:30.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.600+0000 7f167980d700 15 client.31578.objecter _session_command_op_assign -1 9 2024-02-23T00:53:30.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.600+0000 7f167980d700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f167411cec0 0x7f16741254c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:30.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.600+0000 7f167980d700 20 client.31578.objecter _get_session s=0x7f167411cd50 osd=4 3 2024-02-23T00:53:30.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.600+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f167411cd50 osd=4 3 2024-02-23T00:53:30.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.600+0000 7f167980d700 20 client.31578.objecter _get_session s=0x7f167411cd50 osd=4 3 2024-02-23T00:53:30.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.600+0000 7f167980d700 15 client.31578.objecter _session_command_op_remove -1 9 2024-02-23T00:53:30.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.600+0000 7f167980d700 20 client.31578.objecter get_session s=0x7f167411cd50 osd=4 3 2024-02-23T00:53:30.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.600+0000 7f167980d700 15 client.31578.objecter _session_command_op_assign 4 9 2024-02-23T00:53:30.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.600+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f167411cd50 osd=4 4 2024-02-23T00:53:30.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.600+0000 7f167980d700 10 client.31578.objecter _send_command 9 2024-02-23T00:53:30.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.600+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f167411bf10 con 0x7f167411cec0 2024-02-23T00:53:30.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.600+0000 7f1672ffd700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f167411cec0 0x7f16741254c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:30.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.600+0000 7f166bfff700 10 client.31578.objecter ms_handle_connect 0x7f167411cec0 2024-02-23T00:53:30.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.600+0000 7f1672ffd700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f167411cec0 0x7f16741254c0 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:30.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.602+0000 7f166bfff700 1 -- 172.21.15.47:0/123394742 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f167411bf10 con 0x7f167411cec0 2024-02-23T00:53:30.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.602+0000 7f166bfff700 10 client.31578.objecter ms_dispatch 0x7f1674001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:53:30.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.602+0000 7f166bfff700 10 client.31578.objecter _finish_command 9 = system:0 2024-02-23T00:53:30.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.602+0000 7f166bfff700 20 client.31578.objecter put_session s=0x7f167411cd50 osd=4 4 2024-02-23T00:53:30.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.602+0000 7f166bfff700 15 client.31578.objecter _session_command_op_remove 4 9 2024-02-23T00:53:30.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.616+0000 7f167980d700 10 client.31578.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:53:30.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.616+0000 7f167980d700 15 client.31578.objecter _session_command_op_assign -1 10 2024-02-23T00:53:30.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.616+0000 7f167980d700 20 client.31578.objecter _get_session s=0x7f167411cd50 osd=4 3 2024-02-23T00:53:30.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.616+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f167411cd50 osd=4 3 2024-02-23T00:53:30.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.616+0000 7f167980d700 20 client.31578.objecter _get_session s=0x7f167411cd50 osd=4 3 2024-02-23T00:53:30.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.616+0000 7f167980d700 15 client.31578.objecter _session_command_op_remove -1 10 2024-02-23T00:53:30.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.616+0000 7f167980d700 20 client.31578.objecter get_session s=0x7f167411cd50 osd=4 3 2024-02-23T00:53:30.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.616+0000 7f167980d700 15 client.31578.objecter _session_command_op_assign 4 10 2024-02-23T00:53:30.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.616+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f167411cd50 osd=4 4 2024-02-23T00:53:30.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.616+0000 7f167980d700 10 client.31578.objecter _send_command 10 2024-02-23T00:53:30.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.616+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f167411bf10 con 0x7f167411cec0 2024-02-23T00:53:30.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.619+0000 7f166bfff700 1 -- 172.21.15.47:0/123394742 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f167411bf10 con 0x7f167411cec0 2024-02-23T00:53:30.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.619+0000 7f166bfff700 10 client.31578.objecter ms_dispatch 0x7f1674001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:53:30.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.619+0000 7f166bfff700 10 client.31578.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:53:30.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.619+0000 7f166bfff700 20 client.31578.objecter put_session s=0x7f167411cd50 osd=4 4 2024-02-23T00:53:30.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.619+0000 7f166bfff700 15 client.31578.objecter _session_command_op_remove 4 10 2024-02-23T00:53:30.623 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:53:30.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.619+0000 7f167980d700 10 client.31578.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:30.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.619+0000 7f167980d700 15 client.31578.objecter _session_command_op_assign -1 11 2024-02-23T00:53:30.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.619+0000 7f167980d700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f1674125ec0 0x7f16741282b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:30.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.619+0000 7f167980d700 20 client.31578.objecter _get_session s=0x7f167411bf10 osd=5 3 2024-02-23T00:53:30.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.619+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f167411bf10 osd=5 3 2024-02-23T00:53:30.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.619+0000 7f167980d700 20 client.31578.objecter _get_session s=0x7f167411bf10 osd=5 3 2024-02-23T00:53:30.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.619+0000 7f167980d700 15 client.31578.objecter _session_command_op_remove -1 11 2024-02-23T00:53:30.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.619+0000 7f167980d700 20 client.31578.objecter get_session s=0x7f167411bf10 osd=5 3 2024-02-23T00:53:30.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.619+0000 7f167980d700 15 client.31578.objecter _session_command_op_assign 5 11 2024-02-23T00:53:30.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.619+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f167411bf10 osd=5 4 2024-02-23T00:53:30.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.619+0000 7f167980d700 10 client.31578.objecter _send_command 11 2024-02-23T00:53:30.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.619+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f167412b060 con 0x7f1674125ec0 2024-02-23T00:53:30.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.619+0000 7f16727fc700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f1674125ec0 0x7f16741282b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:30.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.619+0000 7f166bfff700 10 client.31578.objecter ms_handle_connect 0x7f1674125ec0 2024-02-23T00:53:30.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.619+0000 7f16727fc700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f1674125ec0 0x7f16741282b0 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:30.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.619+0000 7f166bfff700 1 -- 172.21.15.47:0/123394742 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f167412b060 con 0x7f1674125ec0 2024-02-23T00:53:30.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.619+0000 7f166bfff700 10 client.31578.objecter ms_dispatch 0x7f1674001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:53:30.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.619+0000 7f166bfff700 10 client.31578.objecter _finish_command 11 = system:0 2024-02-23T00:53:30.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.619+0000 7f166bfff700 20 client.31578.objecter put_session s=0x7f167411bf10 osd=5 4 2024-02-23T00:53:30.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.619+0000 7f166bfff700 15 client.31578.objecter _session_command_op_remove 5 11 2024-02-23T00:53:30.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.635+0000 7f167980d700 10 client.31578.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:53:30.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.635+0000 7f167980d700 15 client.31578.objecter _session_command_op_assign -1 12 2024-02-23T00:53:30.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.635+0000 7f167980d700 20 client.31578.objecter _get_session s=0x7f167411bf10 osd=5 3 2024-02-23T00:53:30.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.635+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f167411bf10 osd=5 3 2024-02-23T00:53:30.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.635+0000 7f167980d700 20 client.31578.objecter _get_session s=0x7f167411bf10 osd=5 3 2024-02-23T00:53:30.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.635+0000 7f167980d700 15 client.31578.objecter _session_command_op_remove -1 12 2024-02-23T00:53:30.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.635+0000 7f167980d700 20 client.31578.objecter get_session s=0x7f167411bf10 osd=5 3 2024-02-23T00:53:30.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.635+0000 7f167980d700 15 client.31578.objecter _session_command_op_assign 5 12 2024-02-23T00:53:30.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.635+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f167411bf10 osd=5 4 2024-02-23T00:53:30.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.635+0000 7f167980d700 10 client.31578.objecter _send_command 12 2024-02-23T00:53:30.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.635+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f167412b060 con 0x7f1674125ec0 2024-02-23T00:53:30.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.637+0000 7f166bfff700 1 -- 172.21.15.47:0/123394742 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f167412b060 con 0x7f1674125ec0 2024-02-23T00:53:30.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.637+0000 7f166bfff700 10 client.31578.objecter ms_dispatch 0x7f1674001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:53:30.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.637+0000 7f166bfff700 10 client.31578.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:53:30.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.637+0000 7f166bfff700 20 client.31578.objecter put_session s=0x7f167411bf10 osd=5 4 2024-02-23T00:53:30.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.637+0000 7f166bfff700 15 client.31578.objecter _session_command_op_remove 5 12 2024-02-23T00:53:30.639 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:53:30.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.637+0000 7f167980d700 10 client.31578.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:30.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.637+0000 7f167980d700 15 client.31578.objecter _session_command_op_assign -1 13 2024-02-23T00:53:30.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.637+0000 7f167980d700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f1674128da0 0x7f1674129200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:30.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.637+0000 7f167980d700 20 client.31578.objecter _get_session s=0x7f167412b060 osd=6 3 2024-02-23T00:53:30.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.637+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f167412b060 osd=6 3 2024-02-23T00:53:30.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.637+0000 7f167980d700 20 client.31578.objecter _get_session s=0x7f167412b060 osd=6 3 2024-02-23T00:53:30.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.637+0000 7f167980d700 15 client.31578.objecter _session_command_op_remove -1 13 2024-02-23T00:53:30.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.637+0000 7f167980d700 20 client.31578.objecter get_session s=0x7f167412b060 osd=6 3 2024-02-23T00:53:30.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.637+0000 7f167980d700 15 client.31578.objecter _session_command_op_assign 6 13 2024-02-23T00:53:30.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.637+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f167412b060 osd=6 4 2024-02-23T00:53:30.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.637+0000 7f167980d700 10 client.31578.objecter _send_command 13 2024-02-23T00:53:30.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.637+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f167412e3e0 con 0x7f1674128da0 2024-02-23T00:53:30.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.638+0000 7f16737fe700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f1674128da0 0x7f1674129200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:30.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.638+0000 7f16737fe700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f1674128da0 0x7f1674129200 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:30.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.638+0000 7f166bfff700 10 client.31578.objecter ms_handle_connect 0x7f1674128da0 2024-02-23T00:53:30.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.639+0000 7f166bfff700 1 -- 172.21.15.47:0/123394742 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f167412e3e0 con 0x7f1674128da0 2024-02-23T00:53:30.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.639+0000 7f166bfff700 10 client.31578.objecter ms_dispatch 0x7f1674001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:53:30.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.639+0000 7f166bfff700 10 client.31578.objecter _finish_command 13 = system:0 2024-02-23T00:53:30.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.639+0000 7f166bfff700 20 client.31578.objecter put_session s=0x7f167412b060 osd=6 4 2024-02-23T00:53:30.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.639+0000 7f166bfff700 15 client.31578.objecter _session_command_op_remove 6 13 2024-02-23T00:53:30.643 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:53:30.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.653+0000 7f167980d700 10 client.31578.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:53:30.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.653+0000 7f167980d700 15 client.31578.objecter _session_command_op_assign -1 14 2024-02-23T00:53:30.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.653+0000 7f167980d700 20 client.31578.objecter _get_session s=0x7f167412b060 osd=6 3 2024-02-23T00:53:30.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.653+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f167412b060 osd=6 3 2024-02-23T00:53:30.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.653+0000 7f167980d700 20 client.31578.objecter _get_session s=0x7f167412b060 osd=6 3 2024-02-23T00:53:30.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.653+0000 7f167980d700 15 client.31578.objecter _session_command_op_remove -1 14 2024-02-23T00:53:30.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.653+0000 7f167980d700 20 client.31578.objecter get_session s=0x7f167412b060 osd=6 3 2024-02-23T00:53:30.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.653+0000 7f167980d700 15 client.31578.objecter _session_command_op_assign 6 14 2024-02-23T00:53:30.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.653+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f167412b060 osd=6 4 2024-02-23T00:53:30.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.653+0000 7f167980d700 10 client.31578.objecter _send_command 14 2024-02-23T00:53:30.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.653+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f1674129800 con 0x7f1674128da0 2024-02-23T00:53:30.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.655+0000 7f166bfff700 1 -- 172.21.15.47:0/123394742 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f1674129800 con 0x7f1674128da0 2024-02-23T00:53:30.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.655+0000 7f166bfff700 10 client.31578.objecter ms_dispatch 0x7f1674001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:53:30.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.655+0000 7f166bfff700 10 client.31578.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:53:30.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.655+0000 7f166bfff700 20 client.31578.objecter put_session s=0x7f167412b060 osd=6 4 2024-02-23T00:53:30.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.655+0000 7f166bfff700 15 client.31578.objecter _session_command_op_remove 6 14 2024-02-23T00:53:30.660 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:53:30.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.655+0000 7f167980d700 10 client.31578.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:30.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.655+0000 7f167980d700 15 client.31578.objecter _session_command_op_assign -1 15 2024-02-23T00:53:30.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.655+0000 7f167980d700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f1674129e80 0x7f167412a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:30.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.655+0000 7f167980d700 20 client.31578.objecter _get_session s=0x7f1674129800 osd=7 3 2024-02-23T00:53:30.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.655+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f1674129800 osd=7 3 2024-02-23T00:53:30.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.655+0000 7f167980d700 20 client.31578.objecter _get_session s=0x7f1674129800 osd=7 3 2024-02-23T00:53:30.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.655+0000 7f167980d700 15 client.31578.objecter _session_command_op_remove -1 15 2024-02-23T00:53:30.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.655+0000 7f167980d700 20 client.31578.objecter get_session s=0x7f1674129800 osd=7 3 2024-02-23T00:53:30.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.655+0000 7f167980d700 15 client.31578.objecter _session_command_op_assign 7 15 2024-02-23T00:53:30.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.655+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f1674129800 osd=7 4 2024-02-23T00:53:30.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.655+0000 7f167980d700 10 client.31578.objecter _send_command 15 2024-02-23T00:53:30.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.655+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f167412a920 con 0x7f1674129e80 2024-02-23T00:53:30.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.656+0000 7f1672ffd700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f1674129e80 0x7f167412a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:30.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.656+0000 7f166bfff700 10 client.31578.objecter ms_handle_connect 0x7f1674129e80 2024-02-23T00:53:30.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.656+0000 7f1672ffd700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f1674129e80 0x7f167412a2e0 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:30.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.657+0000 7f166bfff700 1 -- 172.21.15.47:0/123394742 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f167412a920 con 0x7f1674129e80 2024-02-23T00:53:30.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.657+0000 7f166bfff700 10 client.31578.objecter ms_dispatch 0x7f1674001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:53:30.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.657+0000 7f166bfff700 10 client.31578.objecter _finish_command 15 = system:0 2024-02-23T00:53:30.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.657+0000 7f166bfff700 20 client.31578.objecter put_session s=0x7f1674129800 osd=7 4 2024-02-23T00:53:30.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.657+0000 7f166bfff700 15 client.31578.objecter _session_command_op_remove 7 15 2024-02-23T00:53:30.674 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:30.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.672+0000 7f167980d700 10 client.31578.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:53:30.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.672+0000 7f167980d700 15 client.31578.objecter _session_command_op_assign -1 16 2024-02-23T00:53:30.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.672+0000 7f167980d700 20 client.31578.objecter _get_session s=0x7f1674129800 osd=7 3 2024-02-23T00:53:30.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.672+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f1674129800 osd=7 3 2024-02-23T00:53:30.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.672+0000 7f167980d700 20 client.31578.objecter _get_session s=0x7f1674129800 osd=7 3 2024-02-23T00:53:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.672+0000 7f167980d700 15 client.31578.objecter _session_command_op_remove -1 16 2024-02-23T00:53:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.672+0000 7f167980d700 20 client.31578.objecter get_session s=0x7f1674129800 osd=7 3 2024-02-23T00:53:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.672+0000 7f167980d700 15 client.31578.objecter _session_command_op_assign 7 16 2024-02-23T00:53:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.672+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f1674129800 osd=7 4 2024-02-23T00:53:30.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.672+0000 7f167980d700 10 client.31578.objecter _send_command 16 2024-02-23T00:53:30.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.672+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f167412a920 con 0x7f1674129e80 2024-02-23T00:53:30.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.674+0000 7f166bfff700 1 -- 172.21.15.47:0/123394742 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f167412a920 con 0x7f1674129e80 2024-02-23T00:53:30.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.674+0000 7f166bfff700 10 client.31578.objecter ms_dispatch 0x7f1674001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:53:30.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.674+0000 7f166bfff700 10 client.31578.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:53:30.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.674+0000 7f166bfff700 20 client.31578.objecter put_session s=0x7f1674129800 osd=7 4 2024-02-23T00:53:30.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.674+0000 7f166bfff700 15 client.31578.objecter _session_command_op_remove 7 16 2024-02-23T00:53:30.679 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:53:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 10 librados: watch_flush enter 2024-02-23T00:53:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 10 librados: watch_flush exit 2024-02-23T00:53:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 10 client.31578.objecter close_session for osd.0 2024-02-23T00:53:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f16740feee0 msgr2=0x7f16740ff300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f16740feee0 0x7f16740ff300 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f167405ee80 osd=0 1 2024-02-23T00:53:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 10 client.31578.objecter close_session for osd.1 2024-02-23T00:53:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f16740598f0 msgr2=0x7f1674059d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f16740598f0 0x7f1674059d10 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f1674028710 osd=1 1 2024-02-23T00:53:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 10 client.31578.objecter close_session for osd.2 2024-02-23T00:53:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f167411c1c0 msgr2=0x7f167411c620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f167411c1c0 0x7f167411c620 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f1674027d60 osd=2 1 2024-02-23T00:53:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 10 client.31578.objecter close_session for osd.3 2024-02-23T00:53:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f167411d2e0 msgr2=0x7f1674121e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f16737fe700 1 -- 172.21.15.47:0/123394742 reap_dead start 2024-02-23T00:53:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f167411d2e0 0x7f1674121e40 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f1674027a10 osd=3 1 2024-02-23T00:53:30.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 10 client.31578.objecter close_session for osd.4 2024-02-23T00:53:30.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f167411cec0 msgr2=0x7f16741254c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:30.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f167411cec0 0x7f16741254c0 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:30.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f167411cd50 osd=4 1 2024-02-23T00:53:30.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 10 client.31578.objecter close_session for osd.5 2024-02-23T00:53:30.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f1674125ec0 msgr2=0x7f16741282b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:30.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f1674125ec0 0x7f16741282b0 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:30.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f167411bf10 osd=5 1 2024-02-23T00:53:30.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 10 client.31578.objecter close_session for osd.6 2024-02-23T00:53:30.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f1674128da0 msgr2=0x7f1674129200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:30.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f1674128da0 0x7f1674129200 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:30.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f167412b060 osd=6 1 2024-02-23T00:53:30.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 10 client.31578.objecter close_session for osd.7 2024-02-23T00:53:30.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f1674129e80 msgr2=0x7f167412a2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:30.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f1674129e80 0x7f167412a2e0 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:30.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 20 client.31578.objecter put_session s=0x7f1674129800 osd=7 1 2024-02-23T00:53:30.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 20 client.31578.objecter shutdown clearing up homeless session... 2024-02-23T00:53:30.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 10 client.31578.objecter successfully canceled tick 2024-02-23T00:53:30.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f16737fe700 1 -- 172.21.15.47:0/123394742 reap_dead start 2024-02-23T00:53:30.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1660062cc0 msgr2=0x7f1660065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:30.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1660062cc0 0x7f1660065180 secure :-1 s=READY pgs=3159 cs=0 l=1 rev1=1 rx=0x7f165c004ee0 tx=0x7f165c00b040).stop 2024-02-23T00:53:30.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16740f6a90 msgr2=0x7f16740f7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:30.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16740f6a90 0x7f16740f7680 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 rx=0x7f16640048f0 tx=0x7f1664009cb0).stop 2024-02-23T00:53:30.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 shutdown_connections 2024-02-23T00:53:30.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16740f6a90 0x7f16740f7680 unknown :-1 s=CLOSED pgs=2610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:30.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 1 --2- 172.21.15.47:0/123394742 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1660062cc0 0x7f1660065180 unknown :-1 s=CLOSED pgs=3159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:30.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 >> 172.21.15.47:0/123394742 conn(0x7f167406c370 msgr2=0x7f16740fb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:30.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 shutdown_connections 2024-02-23T00:53:30.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 1 -- 172.21.15.47:0/123394742 wait complete. 2024-02-23T00:53:30.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:30.675+0000 7f167980d700 1 librados: shutdown 2024-02-23T00:53:30.692 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:53:30.692 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:53:30.692 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:53:30.692 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:53:30.692 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:53:30.693 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:53:30.693 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:53:30.693 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:53:30.824 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:53:30.827 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:30.827 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:53:30.835 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:53:30.835 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-02-23T00:53:30.836 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 0 2024-02-23T00:53:30.836 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:53:30.928 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:53:30.958 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:31.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:30.269995+0000 osd.5 (osd.5) 291 : cluster [DBG] 18.b scrub starts 2024-02-23T00:53:31.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:30.271405+0000 osd.5 (osd.5) 292 : cluster [DBG] 18.b scrub ok 2024-02-23T00:53:31.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:30.465680+0000 osd.4 (osd.4) 281 : cluster [DBG] 5.a scrub starts 2024-02-23T00:53:31.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:30.466462+0000 osd.4 (osd.4) 282 : cluster [DBG] 5.a scrub ok 2024-02-23T00:53:31.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:30.513839+0000 mon.a (mon.0) 3339 : audit [DBG] from='client.? 172.21.15.47:0/123394742' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:53:31.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:30.602197+0000 osd.1 (osd.1) 276 : cluster [DBG] 18.18 scrub starts 2024-02-23T00:53:31.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:30.603085+0000 osd.1 (osd.1) 277 : cluster [DBG] 18.18 scrub ok 2024-02-23T00:53:31.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:31.287188+0000 mon.a (mon.0) 3340 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-116"}]': finished 2024-02-23T00:53:31.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:31.287263+0000 mon.a (mon.0) 3341 : cluster [DBG] osdmap e674: 8 total, 8 up, 6 in 2024-02-23T00:53:31.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:31.287770+0000 mon.c (mon.2) 389 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-116", "mode": "writeback"}]: dispatch 2024-02-23T00:53:31.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:31.288071+0000 mon.a (mon.0) 3342 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-116", "mode": "writeback"}]: dispatch 2024-02-23T00:53:31.527 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.5 dump_blocked_ops 2024-02-23T00:53:31.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:30.269995+0000 osd.5 (osd.5) 291 : cluster [DBG] 18.b scrub starts 2024-02-23T00:53:31.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:30.271405+0000 osd.5 (osd.5) 292 : cluster [DBG] 18.b scrub ok 2024-02-23T00:53:31.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:30.465680+0000 osd.4 (osd.4) 281 : cluster [DBG] 5.a scrub starts 2024-02-23T00:53:31.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:30.466462+0000 osd.4 (osd.4) 282 : cluster [DBG] 5.a scrub ok 2024-02-23T00:53:31.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:30.513839+0000 mon.a (mon.0) 3339 : audit [DBG] from='client.? 172.21.15.47:0/123394742' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:53:31.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:30.602197+0000 osd.1 (osd.1) 276 : cluster [DBG] 18.18 scrub starts 2024-02-23T00:53:31.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:30.603085+0000 osd.1 (osd.1) 277 : cluster [DBG] 18.18 scrub ok 2024-02-23T00:53:31.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:31.287188+0000 mon.a (mon.0) 3340 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-116"}]': finished 2024-02-23T00:53:31.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:31.287263+0000 mon.a (mon.0) 3341 : cluster [DBG] osdmap e674: 8 total, 8 up, 6 in 2024-02-23T00:53:31.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:31.287770+0000 mon.c (mon.2) 389 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-116", "mode": "writeback"}]: dispatch 2024-02-23T00:53:31.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:31.288071+0000 mon.a (mon.0) 3342 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-116", "mode": "writeback"}]: dispatch 2024-02-23T00:53:31.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:30.269995+0000 osd.5 (osd.5) 291 : cluster [DBG] 18.b scrub starts 2024-02-23T00:53:31.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:30.271405+0000 osd.5 (osd.5) 292 : cluster [DBG] 18.b scrub ok 2024-02-23T00:53:31.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:30.465680+0000 osd.4 (osd.4) 281 : cluster [DBG] 5.a scrub starts 2024-02-23T00:53:31.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:30.466462+0000 osd.4 (osd.4) 282 : cluster [DBG] 5.a scrub ok 2024-02-23T00:53:31.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:30.513839+0000 mon.a (mon.0) 3339 : audit [DBG] from='client.? 172.21.15.47:0/123394742' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:53:31.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:30.602197+0000 osd.1 (osd.1) 276 : cluster [DBG] 18.18 scrub starts 2024-02-23T00:53:31.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:30.603085+0000 osd.1 (osd.1) 277 : cluster [DBG] 18.18 scrub ok 2024-02-23T00:53:31.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:31.287188+0000 mon.a (mon.0) 3340 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-116"}]': finished 2024-02-23T00:53:31.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:31.287263+0000 mon.a (mon.0) 3341 : cluster [DBG] osdmap e674: 8 total, 8 up, 6 in 2024-02-23T00:53:31.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:31.287770+0000 mon.c (mon.2) 389 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-116", "mode": "writeback"}]: dispatch 2024-02-23T00:53:31.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:31.288071+0000 mon.a (mon.0) 3342 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-116", "mode": "writeback"}]: dispatch 2024-02-23T00:53:31.795 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:53:32.008 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:53:32.010 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:32.011 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:53:32.070 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-23T00:53:32.112 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:53:32.144 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:32.348 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:53:32.351 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:32.352 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:53:32.453 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:53:32.482 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:32.555 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:53:32.557 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:32.557 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:53:32.658 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:53:32.684 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:32.792 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:53:32.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.810+0000 7f62101d2700 1 -- 172.21.15.47:0/1817069651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62081052f0 msgr2=0x7f62081056d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:32.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.810+0000 7f62101d2700 1 --2- 172.21.15.47:0/1817069651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62081052f0 0x7f62081056d0 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 rx=0x7f6204004870 tx=0x7f6204009cb0).stop 2024-02-23T00:53:32.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.810+0000 7f62101d2700 1 -- 172.21.15.47:0/1817069651 shutdown_connections 2024-02-23T00:53:32.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.810+0000 7f62101d2700 1 --2- 172.21.15.47:0/1817069651 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6208069c30 0x7f620810c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:32.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.810+0000 7f62101d2700 1 --2- 172.21.15.47:0/1817069651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62081052f0 0x7f62081056d0 unknown :-1 s=CLOSED pgs=2611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:32.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.810+0000 7f62101d2700 1 --2- 172.21.15.47:0/1817069651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6208105c50 0x7f6208069570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:32.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.810+0000 7f62101d2700 1 -- 172.21.15.47:0/1817069651 >> 172.21.15.47:0/1817069651 conn(0x7f6208063db0 msgr2=0x7f6208067560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:32.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.810+0000 7f62101d2700 1 -- 172.21.15.47:0/1817069651 shutdown_connections 2024-02-23T00:53:32.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.810+0000 7f62101d2700 1 -- 172.21.15.47:0/1817069651 wait complete. 2024-02-23T00:53:32.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.810+0000 7f62101d2700 1 librados: starting msgr at 2024-02-23T00:53:32.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.810+0000 7f62101d2700 1 librados: starting objecter 2024-02-23T00:53:32.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.811+0000 7f62101d2700 1 Processor -- start 2024-02-23T00:53:32.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.811+0000 7f62101d2700 1 -- start start 2024-02-23T00:53:32.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.811+0000 7f62101d2700 1 librados: setting wanted keys 2024-02-23T00:53:32.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.811+0000 7f62101d2700 1 librados: calling monclient init 2024-02-23T00:53:32.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.811+0000 7f62101d2700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6208069c30 0x7f62081118b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:32.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.811+0000 7f62101d2700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62081052f0 0x7f6208111da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:32.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.811+0000 7f62101d2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6208105c50 0x7f6208112380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:32.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.811+0000 7f62101d2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620811ca90 con 0x7f6208105c50 2024-02-23T00:53:32.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.811+0000 7f62101d2700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620811cc00 con 0x7f62081052f0 2024-02-23T00:53:32.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.811+0000 7f62101d2700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f620811cdc0 con 0x7f6208069c30 2024-02-23T00:53:32.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.811+0000 7f620e76f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6208105c50 0x7f6208112380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:32.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.811+0000 7f620e76f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6208105c50 0x7f6208112380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52322/0 (socket says 172.21.15.47:52322) 2024-02-23T00:53:32.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.811+0000 7f620e76f700 1 -- 172.21.15.47:0/1252972828 learned_addr learned my addr 172.21.15.47:0/1252972828 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:53:32.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.811+0000 7f620df6e700 1 --2- 172.21.15.47:0/1252972828 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6208069c30 0x7f62081118b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:32.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.811+0000 7f620d76d700 1 --2- 172.21.15.47:0/1252972828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62081052f0 0x7f6208111da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:32.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.811+0000 7f620e76f700 1 -- 172.21.15.47:0/1252972828 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6208069c30 msgr2=0x7f62081118b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:32.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.811+0000 7f620e76f700 1 --2- 172.21.15.47:0/1252972828 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6208069c30 0x7f62081118b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:32.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.811+0000 7f620e76f700 1 -- 172.21.15.47:0/1252972828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62081052f0 msgr2=0x7f6208111da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:32.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.811+0000 7f620e76f700 1 --2- 172.21.15.47:0/1252972828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62081052f0 0x7f6208111da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:32.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.811+0000 7f620e76f700 1 -- 172.21.15.47:0/1252972828 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62000097c0 con 0x7f6208105c50 2024-02-23T00:53:32.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.811+0000 7f620df6e700 1 --2- 172.21.15.47:0/1252972828 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6208069c30 0x7f62081118b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:53:32.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.812+0000 7f620e76f700 1 --2- 172.21.15.47:0/1252972828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6208105c50 0x7f6208112380 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 rx=0x7f6200001df0 tx=0x7f6200004c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:32.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.812+0000 7f61feffd700 10 client.?.objecter ms_handle_connect 0x7f6208105c50 2024-02-23T00:53:32.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.812+0000 7f61feffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:53:32.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.812+0000 7f61feffd700 1 -- 172.21.15.47:0/1252972828 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6200009a10 con 0x7f6208105c50 2024-02-23T00:53:32.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.812+0000 7f62101d2700 1 -- 172.21.15.47:0/1252972828 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6204005040 con 0x7f6208105c50 2024-02-23T00:53:32.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.812+0000 7f61feffd700 1 -- 172.21.15.47:0/1252972828 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f62000090d0 con 0x7f6208105c50 2024-02-23T00:53:32.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.812+0000 7f61feffd700 1 -- 172.21.15.47:0/1252972828 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6200007be0 con 0x7f6208105c50 2024-02-23T00:53:32.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.812+0000 7f62101d2700 10 client.31584.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:53:32.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.812+0000 7f62101d2700 1 -- 172.21.15.47:0/1252972828 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f620811d400 con 0x7f6208105c50 2024-02-23T00:53:32.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.812+0000 7f62101d2700 1 librados: init done 2024-02-23T00:53:32.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.813+0000 7f62101d2700 1 -- 172.21.15.47:0/1252972828 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6208060280 con 0x7f6208105c50 2024-02-23T00:53:32.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.813+0000 7f61feffd700 1 -- 172.21.15.47:0/1252972828 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f6200009240 con 0x7f6208105c50 2024-02-23T00:53:32.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.814+0000 7f61feffd700 1 --2- 172.21.15.47:0/1252972828 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f61f4062cc0 0x7f61f4065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:32.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.814+0000 7f61feffd700 1 -- 172.21.15.47:0/1252972828 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(675..675 src has 482..675) v4 ==== 9633+0+0 (secure 0 0 0) 0x7f620007e150 con 0x7f6208105c50 2024-02-23T00:53:32.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.814+0000 7f61feffd700 10 client.31584.objecter ms_dispatch 0x7f6208001550 osd_map(675..675 src has 482..675) v4 2024-02-23T00:53:32.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.814+0000 7f61feffd700 3 client.31584.objecter handle_osd_map got epochs [675,675] > 0 2024-02-23T00:53:32.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.814+0000 7f61feffd700 3 client.31584.objecter handle_osd_map decoding full epoch 675 2024-02-23T00:53:32.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.814+0000 7f620df6e700 1 --2- 172.21.15.47:0/1252972828 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f61f4062cc0 0x7f61f4065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:32.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.814+0000 7f61feffd700 20 client.31584.objecter dump_active .. 0 homeless 2024-02-23T00:53:32.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.814+0000 7f61feffd700 10 client.31584.objecter ms_handle_connect 0x7f61f4062cc0 2024-02-23T00:53:32.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.814+0000 7f620df6e700 1 --2- 172.21.15.47:0/1252972828 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f61f4062cc0 0x7f61f4065180 secure :-1 s=READY pgs=3160 cs=0 l=1 rev1=1 rx=0x7f6204004ee0 tx=0x7f620400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:32.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:32.819+0000 7f61feffd700 1 -- 172.21.15.47:0/1252972828 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f620004d5c0 con 0x7f6208105c50 2024-02-23T00:53:32.866 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:53:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:53:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:53:32.866 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:31.372982+0000 mgr.y (mgr.14517) 587 : cluster [DBG] pgmap v1019: 184 pgs: 32 unknown, 2 peering, 150 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:53:32.866 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:31.649763+0000 mon.a (mon.0) 3343 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:53:32.866 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:32.255305+0000 osd.5 (osd.5) 293 : cluster [DBG] 18.17 scrub starts 2024-02-23T00:53:32.866 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:32.258816+0000 osd.5 (osd.5) 294 : cluster [DBG] 18.17 scrub ok 2024-02-23T00:53:32.867 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:32.284481+0000 mon.a (mon.0) 3344 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:53:32.867 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:32.288824+0000 mon.a (mon.0) 3345 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-116", "mode": "writeback"}]': finished 2024-02-23T00:53:32.867 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:32.288873+0000 mon.a (mon.0) 3346 : cluster [DBG] osdmap e675: 8 total, 8 up, 6 in 2024-02-23T00:53:32.867 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:32.345195+0000 mon.c (mon.2) 390 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:32.867 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:32.346012+0000 mon.a (mon.0) 3347 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:33.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:33.015+0000 7f62101d2700 1 -- 172.21.15.47:0/1252972828 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 3, "weight": 0.0} v 0) v1 -- 0x7f620805ee80 con 0x7f6208105c50 2024-02-23T00:53:33.037 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:31.372982+0000 mgr.y (mgr.14517) 587 : cluster [DBG] pgmap v1019: 184 pgs: 32 unknown, 2 peering, 150 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:53:33.037 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:31.649763+0000 mon.a (mon.0) 3343 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:53:33.037 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:32.255305+0000 osd.5 (osd.5) 293 : cluster [DBG] 18.17 scrub starts 2024-02-23T00:53:33.037 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:32.258816+0000 osd.5 (osd.5) 294 : cluster [DBG] 18.17 scrub ok 2024-02-23T00:53:33.037 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:32.284481+0000 mon.a (mon.0) 3344 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:53:33.037 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:32.288824+0000 mon.a (mon.0) 3345 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-116", "mode": "writeback"}]': finished 2024-02-23T00:53:33.037 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:32.288873+0000 mon.a (mon.0) 3346 : cluster [DBG] osdmap e675: 8 total, 8 up, 6 in 2024-02-23T00:53:33.038 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:32.345195+0000 mon.c (mon.2) 390 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:33.038 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:32.346012+0000 mon.a (mon.0) 3347 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:33.038 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:31.372982+0000 mgr.y (mgr.14517) 587 : cluster [DBG] pgmap v1019: 184 pgs: 32 unknown, 2 peering, 150 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:53:33.038 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:31.649763+0000 mon.a (mon.0) 3343 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:53:33.038 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:32.255305+0000 osd.5 (osd.5) 293 : cluster [DBG] 18.17 scrub starts 2024-02-23T00:53:33.038 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:32.258816+0000 osd.5 (osd.5) 294 : cluster [DBG] 18.17 scrub ok 2024-02-23T00:53:33.038 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:32.284481+0000 mon.a (mon.0) 3344 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:53:33.038 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:32.288824+0000 mon.a (mon.0) 3345 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-116", "mode": "writeback"}]': finished 2024-02-23T00:53:33.038 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:32.288873+0000 mon.a (mon.0) 3346 : cluster [DBG] osdmap e675: 8 total, 8 up, 6 in 2024-02-23T00:53:33.038 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:32.345195+0000 mon.c (mon.2) 390 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:33.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:32.346012+0000 mon.a (mon.0) 3347 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:33.194 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:53:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:53:33.233 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:53:33.235 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:33.235 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:53:33.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:33.310+0000 7f61feffd700 1 -- 172.21.15.47:0/1252972828 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 3, "weight": 0.0}]=0 set osd.3 primary-affinity to 0 (802) v676) v1 ==== 129+0+0 (secure 0 0 0) 0x7f62000191b0 con 0x7f6208105c50 2024-02-23T00:53:33.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:33.315+0000 7f62101d2700 10 librados: watch_flush enter 2024-02-23T00:53:33.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:33.315+0000 7f62101d2700 10 librados: watch_flush exit 2024-02-23T00:53:33.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:33.315+0000 7f62101d2700 20 client.31584.objecter shutdown clearing up homeless session... 2024-02-23T00:53:33.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:33.315+0000 7f62101d2700 10 client.31584.objecter successfully canceled tick 2024-02-23T00:53:33.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:33.315+0000 7f62101d2700 1 -- 172.21.15.47:0/1252972828 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f61f4062cc0 msgr2=0x7f61f4065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:33.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:33.315+0000 7f62101d2700 1 --2- 172.21.15.47:0/1252972828 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f61f4062cc0 0x7f61f4065180 secure :-1 s=READY pgs=3160 cs=0 l=1 rev1=1 rx=0x7f6204004ee0 tx=0x7f620400b040).stop 2024-02-23T00:53:33.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:33.315+0000 7f62101d2700 1 -- 172.21.15.47:0/1252972828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6208105c50 msgr2=0x7f6208112380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:33.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:33.315+0000 7f62101d2700 1 --2- 172.21.15.47:0/1252972828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6208105c50 0x7f6208112380 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 rx=0x7f6200001df0 tx=0x7f6200004c50).stop 2024-02-23T00:53:33.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:33.315+0000 7f62101d2700 1 -- 172.21.15.47:0/1252972828 shutdown_connections 2024-02-23T00:53:33.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:33.315+0000 7f62101d2700 1 --2- 172.21.15.47:0/1252972828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6208105c50 0x7f6208112380 unknown :-1 s=CLOSED pgs=2612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:33.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:33.315+0000 7f62101d2700 1 --2- 172.21.15.47:0/1252972828 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f61f4062cc0 0x7f61f4065180 unknown :-1 s=CLOSED pgs=3160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:33.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:33.315+0000 7f62101d2700 1 --2- 172.21.15.47:0/1252972828 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6208069c30 0x7f62081118b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:33.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:33.315+0000 7f62101d2700 1 --2- 172.21.15.47:0/1252972828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62081052f0 0x7f6208111da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:33.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:33.315+0000 7f62101d2700 1 -- 172.21.15.47:0/1252972828 >> 172.21.15.47:0/1252972828 conn(0x7f6208063db0 msgr2=0x7f6208103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:33.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:33.316+0000 7f62101d2700 1 -- 172.21.15.47:0/1252972828 shutdown_connections 2024-02-23T00:53:33.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:33.316+0000 7f62101d2700 1 -- 172.21.15.47:0/1252972828 wait complete. 2024-02-23T00:53:33.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:33.316+0000 7f62101d2700 1 librados: shutdown 2024-02-23T00:53:33.320 INFO:teuthology.orchestra.run.smithi047.stderr:set osd.3 primary-affinity to 0 (802) 2024-02-23T00:53:33.336 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:53:33.371 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:33.478 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:53:33.480 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-02-23T00:53:33.481 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-02-23T00:53:33.481 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-02-23T00:53:33.481 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:53:33.521 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:53:33.525 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:33.525 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:53:33.626 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:53:33.656 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:33.677 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:53:33.841 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:32.316063+0000 osd.7 (osd.7) 319 : cluster [DBG] 4.17 scrub starts 2024-02-23T00:53:33.841 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:32.317734+0000 osd.7 (osd.7) 320 : cluster [DBG] 4.17 scrub ok 2024-02-23T00:53:33.841 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:32.449660+0000 osd.6 (osd.6) 257 : cluster [DBG] 234.5 scrub starts 2024-02-23T00:53:33.841 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:32.451121+0000 osd.6 (osd.6) 258 : cluster [DBG] 234.5 scrub ok 2024-02-23T00:53:33.841 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:33.015465+0000 mon.a (mon.0) 3348 : audit [INF] from='client.? 172.21.15.47:0/1252972828' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 0.0}]: dispatch 2024-02-23T00:53:33.841 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:33.309349+0000 mon.a (mon.0) 3349 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:53:33.841 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:33.309437+0000 mon.a (mon.0) 3350 : audit [INF] from='client.? 172.21.15.47:0/1252972828' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 0.0}]': finished 2024-02-23T00:53:33.841 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:33.309499+0000 mon.a (mon.0) 3351 : cluster [DBG] osdmap e676: 8 total, 8 up, 6 in 2024-02-23T00:53:33.841 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:33.311072+0000 mon.c (mon.2) 391 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-116"}]: dispatch 2024-02-23T00:53:33.842 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:33.311588+0000 mon.a (mon.0) 3352 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-116"}]: dispatch 2024-02-23T00:53:33.956 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:32.316063+0000 osd.7 (osd.7) 319 : cluster [DBG] 4.17 scrub starts 2024-02-23T00:53:33.957 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:32.317734+0000 osd.7 (osd.7) 320 : cluster [DBG] 4.17 scrub ok 2024-02-23T00:53:33.957 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:32.449660+0000 osd.6 (osd.6) 257 : cluster [DBG] 234.5 scrub starts 2024-02-23T00:53:33.957 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:32.451121+0000 osd.6 (osd.6) 258 : cluster [DBG] 234.5 scrub ok 2024-02-23T00:53:33.957 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:33.015465+0000 mon.a (mon.0) 3348 : audit [INF] from='client.? 172.21.15.47:0/1252972828' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 0.0}]: dispatch 2024-02-23T00:53:33.957 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:33.309349+0000 mon.a (mon.0) 3349 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:53:33.957 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:33.309437+0000 mon.a (mon.0) 3350 : audit [INF] from='client.? 172.21.15.47:0/1252972828' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 0.0}]': finished 2024-02-23T00:53:33.957 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:33.309499+0000 mon.a (mon.0) 3351 : cluster [DBG] osdmap e676: 8 total, 8 up, 6 in 2024-02-23T00:53:33.957 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:33.311072+0000 mon.c (mon.2) 391 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-116"}]: dispatch 2024-02-23T00:53:33.957 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:33.311588+0000 mon.a (mon.0) 3352 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-116"}]: dispatch 2024-02-23T00:53:33.957 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:32.316063+0000 osd.7 (osd.7) 319 : cluster [DBG] 4.17 scrub starts 2024-02-23T00:53:33.958 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:32.317734+0000 osd.7 (osd.7) 320 : cluster [DBG] 4.17 scrub ok 2024-02-23T00:53:33.958 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:32.449660+0000 osd.6 (osd.6) 257 : cluster [DBG] 234.5 scrub starts 2024-02-23T00:53:33.958 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:32.451121+0000 osd.6 (osd.6) 258 : cluster [DBG] 234.5 scrub ok 2024-02-23T00:53:33.958 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:33.015465+0000 mon.a (mon.0) 3348 : audit [INF] from='client.? 172.21.15.47:0/1252972828' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 0.0}]: dispatch 2024-02-23T00:53:33.958 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:33.309349+0000 mon.a (mon.0) 3349 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:53:33.958 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:33.309437+0000 mon.a (mon.0) 3350 : audit [INF] from='client.? 172.21.15.47:0/1252972828' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 0.0}]': finished 2024-02-23T00:53:33.958 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:33.309499+0000 mon.a (mon.0) 3351 : cluster [DBG] osdmap e676: 8 total, 8 up, 6 in 2024-02-23T00:53:33.958 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:33.311072+0000 mon.c (mon.2) 391 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-116"}]: dispatch 2024-02-23T00:53:33.958 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:33.311588+0000 mon.a (mon.0) 3352 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-116"}]: dispatch 2024-02-23T00:53:34.173 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.5 dump_historic_ops 2024-02-23T00:53:34.451 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:53:34.807 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:53:34.810 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:34.810 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:53:34.911 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:53:34.938 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:32.864026+0000 mgr.y (mgr.14517) 588 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:53:34.938 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:33.373502+0000 mgr.y (mgr.14517) 589 : cluster [DBG] pgmap v1022: 184 pgs: 184 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:53:34.938 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:33.478059+0000 osd.4 (osd.4) 283 : cluster [DBG] 6.6 scrub starts 2024-02-23T00:53:34.938 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:33.479047+0000 osd.4 (osd.4) 284 : cluster [DBG] 6.6 scrub ok 2024-02-23T00:53:34.938 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:34.206483+0000 osd.5 (osd.5) 295 : cluster [DBG] 2.18 scrub starts 2024-02-23T00:53:34.938 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:34.208026+0000 osd.5 (osd.5) 296 : cluster [DBG] 2.18 scrub ok 2024-02-23T00:53:34.938 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:34.306458+0000 mon.a (mon.0) 3353 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:53:34.938 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:34.310512+0000 mon.a (mon.0) 3354 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-116"}]': finished 2024-02-23T00:53:34.939 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:34.310678+0000 mon.a (mon.0) 3355 : cluster [DBG] osdmap e677: 8 total, 8 up, 6 in 2024-02-23T00:53:34.940 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:35.024 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:53:35.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.029+0000 7f1f9dec8700 1 -- 172.21.15.47:0/4264254013 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1f980f6a90 msgr2=0x7f1f98104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:35.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.029+0000 7f1f9dec8700 1 --2- 172.21.15.47:0/4264254013 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1f980f6a90 0x7f1f98104a20 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 rx=0x7f1f90009be0 tx=0x7f1f90004d10).stop 2024-02-23T00:53:35.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.029+0000 7f1f9dec8700 1 -- 172.21.15.47:0/4264254013 shutdown_connections 2024-02-23T00:53:35.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.029+0000 7f1f9dec8700 1 --2- 172.21.15.47:0/4264254013 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1f980f6a90 0x7f1f98104a20 unknown :-1 s=CLOSED pgs=2770 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:35.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.029+0000 7f1f9dec8700 1 --2- 172.21.15.47:0/4264254013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f980fdbd0 0x7f1f980fdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:35.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.029+0000 7f1f9dec8700 1 --2- 172.21.15.47:0/4264254013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f980fe530 0x7f1f980f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:35.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.029+0000 7f1f9dec8700 1 -- 172.21.15.47:0/4264254013 >> 172.21.15.47:0/4264254013 conn(0x7f1f9806c370 msgr2=0x7f1f9806c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:35.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.029+0000 7f1f9dec8700 1 -- 172.21.15.47:0/4264254013 shutdown_connections 2024-02-23T00:53:35.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.030+0000 7f1f9dec8700 1 -- 172.21.15.47:0/4264254013 wait complete. 2024-02-23T00:53:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.030+0000 7f1f9dec8700 1 librados: starting msgr at 2024-02-23T00:53:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.030+0000 7f1f9dec8700 1 librados: starting objecter 2024-02-23T00:53:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.030+0000 7f1f9dec8700 1 Processor -- start 2024-02-23T00:53:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.030+0000 7f1f9dec8700 1 -- start start 2024-02-23T00:53:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.030+0000 7f1f9dec8700 1 librados: setting wanted keys 2024-02-23T00:53:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.030+0000 7f1f9dec8700 1 librados: calling monclient init 2024-02-23T00:53:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.031+0000 7f1f9dec8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f980f6a90 0x7f1f980f7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.031+0000 7f1f9dec8700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1f980fdbd0 0x7f1f980f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.031+0000 7f1f9dec8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f980fe530 0x7f1f980f8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.031+0000 7f1f9dec8700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f98116260 con 0x7f1f980f6a90 2024-02-23T00:53:35.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.031+0000 7f1f9dec8700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f981163d0 con 0x7f1f980fe530 2024-02-23T00:53:35.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.031+0000 7f1f9dec8700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f98116590 con 0x7f1f980fdbd0 2024-02-23T00:53:35.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.031+0000 7f1f977fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f980f6a90 0x7f1f980f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:35.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.031+0000 7f1f96ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1f980fdbd0 0x7f1f980f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:35.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.031+0000 7f1f977fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f980f6a90 0x7f1f980f7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52340/0 (socket says 172.21.15.47:52340) 2024-02-23T00:53:35.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.031+0000 7f1f977fe700 1 -- 172.21.15.47:0/2375783506 learned_addr learned my addr 172.21.15.47:0/2375783506 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:53:35.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.031+0000 7f1f97fff700 1 --2- 172.21.15.47:0/2375783506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f980fe530 0x7f1f980f8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:35.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.031+0000 7f1f96ffd700 1 -- 172.21.15.47:0/2375783506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f980fe530 msgr2=0x7f1f980f8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:35.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.031+0000 7f1f96ffd700 1 --2- 172.21.15.47:0/2375783506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f980fe530 0x7f1f980f8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:35.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.031+0000 7f1f96ffd700 1 -- 172.21.15.47:0/2375783506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f980f6a90 msgr2=0x7f1f980f7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:35.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.031+0000 7f1f96ffd700 1 --2- 172.21.15.47:0/2375783506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f980f6a90 0x7f1f980f7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:35.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.031+0000 7f1f96ffd700 1 -- 172.21.15.47:0/2375783506 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f900097c0 con 0x7f1f980fdbd0 2024-02-23T00:53:35.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.031+0000 7f1f977fe700 1 --2- 172.21.15.47:0/2375783506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f980f6a90 0x7f1f980f7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:53:35.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.031+0000 7f1f96ffd700 1 --2- 172.21.15.47:0/2375783506 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1f980fdbd0 0x7f1f980f7b70 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 rx=0x7f1f80002760 tx=0x7f1f80009e80).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:35.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.031+0000 7f1f94ff9700 10 client.?.objecter ms_handle_connect 0x7f1f980fdbd0 2024-02-23T00:53:35.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.031+0000 7f1f94ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:53:35.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.032+0000 7f1f94ff9700 1 -- 172.21.15.47:0/2375783506 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f80007b40 con 0x7f1f980fdbd0 2024-02-23T00:53:35.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.032+0000 7f1f94ff9700 1 -- 172.21.15.47:0/2375783506 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f1f8000f040 con 0x7f1f980fdbd0 2024-02-23T00:53:35.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.032+0000 7f1f9dec8700 1 -- 172.21.15.47:0/2375783506 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f98116870 con 0x7f1f980fdbd0 2024-02-23T00:53:35.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.032+0000 7f1f94ff9700 1 -- 172.21.15.47:0/2375783506 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f800146d0 con 0x7f1f980fdbd0 2024-02-23T00:53:35.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.032+0000 7f1f9dec8700 10 client.41909.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:53:35.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.032+0000 7f1f9dec8700 1 -- 172.21.15.47:0/2375783506 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f98116d60 con 0x7f1f980fdbd0 2024-02-23T00:53:35.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.032+0000 7f1f9dec8700 1 librados: init done 2024-02-23T00:53:35.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.033+0000 7f1f9dec8700 1 -- 172.21.15.47:0/2375783506 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f78004f50 con 0x7f1f980fdbd0 2024-02-23T00:53:35.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.033+0000 7f1f94ff9700 1 -- 172.21.15.47:0/2375783506 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f1f80014b00 con 0x7f1f980fdbd0 2024-02-23T00:53:35.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.033+0000 7f1f94ff9700 1 --2- 172.21.15.47:0/2375783506 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1f84062cc0 0x7f1f84065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:35.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.034+0000 7f1f94ff9700 1 -- 172.21.15.47:0/2375783506 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(677..677 src has 482..677) v4 ==== 9625+0+0 (secure 0 0 0) 0x7f1f8007d2d0 con 0x7f1f980fdbd0 2024-02-23T00:53:35.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.034+0000 7f1f94ff9700 10 client.41909.objecter ms_dispatch 0x7f1f98001550 osd_map(677..677 src has 482..677) v4 2024-02-23T00:53:35.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.034+0000 7f1f94ff9700 3 client.41909.objecter handle_osd_map got epochs [677,677] > 0 2024-02-23T00:53:35.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.034+0000 7f1f94ff9700 3 client.41909.objecter handle_osd_map decoding full epoch 677 2024-02-23T00:53:35.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.034+0000 7f1f94ff9700 20 client.41909.objecter dump_active .. 0 homeless 2024-02-23T00:53:35.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.037+0000 7f1f977fe700 1 --2- 172.21.15.47:0/2375783506 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1f84062cc0 0x7f1f84065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:35.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.037+0000 7f1f977fe700 1 --2- 172.21.15.47:0/2375783506 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1f84062cc0 0x7f1f84065180 secure :-1 s=READY pgs=3161 cs=0 l=1 rev1=1 rx=0x7f1f88007d30 tx=0x7f1f8800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:35.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.037+0000 7f1f94ff9700 10 client.41909.objecter ms_handle_connect 0x7f1f84062cc0 2024-02-23T00:53:35.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.038+0000 7f1f94ff9700 1 -- 172.21.15.47:0/2375783506 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f1f8004c920 con 0x7f1f980fdbd0 2024-02-23T00:53:35.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:32.864026+0000 mgr.y (mgr.14517) 588 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:53:35.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:33.373502+0000 mgr.y (mgr.14517) 589 : cluster [DBG] pgmap v1022: 184 pgs: 184 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:53:35.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:33.478059+0000 osd.4 (osd.4) 283 : cluster [DBG] 6.6 scrub starts 2024-02-23T00:53:35.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:33.479047+0000 osd.4 (osd.4) 284 : cluster [DBG] 6.6 scrub ok 2024-02-23T00:53:35.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:34.206483+0000 osd.5 (osd.5) 295 : cluster [DBG] 2.18 scrub starts 2024-02-23T00:53:35.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:34.208026+0000 osd.5 (osd.5) 296 : cluster [DBG] 2.18 scrub ok 2024-02-23T00:53:35.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:34.306458+0000 mon.a (mon.0) 3353 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:53:35.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:34.310512+0000 mon.a (mon.0) 3354 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-116"}]': finished 2024-02-23T00:53:35.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:34.310678+0000 mon.a (mon.0) 3355 : cluster [DBG] osdmap e677: 8 total, 8 up, 6 in 2024-02-23T00:53:35.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:32.864026+0000 mgr.y (mgr.14517) 588 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:53:35.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:33.373502+0000 mgr.y (mgr.14517) 589 : cluster [DBG] pgmap v1022: 184 pgs: 184 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:53:35.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:33.478059+0000 osd.4 (osd.4) 283 : cluster [DBG] 6.6 scrub starts 2024-02-23T00:53:35.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:33.479047+0000 osd.4 (osd.4) 284 : cluster [DBG] 6.6 scrub ok 2024-02-23T00:53:35.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:34.206483+0000 osd.5 (osd.5) 295 : cluster [DBG] 2.18 scrub starts 2024-02-23T00:53:35.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:34.208026+0000 osd.5 (osd.5) 296 : cluster [DBG] 2.18 scrub ok 2024-02-23T00:53:35.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:34.306458+0000 mon.a (mon.0) 3353 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:53:35.061 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:34.310512+0000 mon.a (mon.0) 3354 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-116"}]': finished 2024-02-23T00:53:35.061 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:34.310678+0000 mon.a (mon.0) 3355 : cluster [DBG] osdmap e677: 8 total, 8 up, 6 in 2024-02-23T00:53:35.217 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:53:35.220 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:35.221 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:53:35.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.227+0000 7f1f9dec8700 1 -- 172.21.15.47:0/2375783506 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7f1f78005200 con 0x7f1f980fdbd0 2024-02-23T00:53:35.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.281+0000 7f1f94ff9700 1 -- 172.21.15.47:0/2375783506 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v678) v1 ==== 101+0+0 (secure 0 0 0) 0x7f1f8004ff40 con 0x7f1f980fdbd0 2024-02-23T00:53:35.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.284+0000 7f1f9dec8700 10 librados: watch_flush enter 2024-02-23T00:53:35.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.285+0000 7f1f9dec8700 10 librados: watch_flush exit 2024-02-23T00:53:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.285+0000 7f1f9dec8700 20 client.41909.objecter shutdown clearing up homeless session... 2024-02-23T00:53:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.285+0000 7f1f9dec8700 10 client.41909.objecter successfully canceled tick 2024-02-23T00:53:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.285+0000 7f1f9dec8700 1 -- 172.21.15.47:0/2375783506 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1f84062cc0 msgr2=0x7f1f84065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.285+0000 7f1f9dec8700 1 --2- 172.21.15.47:0/2375783506 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1f84062cc0 0x7f1f84065180 secure :-1 s=READY pgs=3161 cs=0 l=1 rev1=1 rx=0x7f1f88007d30 tx=0x7f1f8800b040).stop 2024-02-23T00:53:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.285+0000 7f1f9dec8700 1 -- 172.21.15.47:0/2375783506 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1f980fdbd0 msgr2=0x7f1f980f7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.285+0000 7f1f9dec8700 1 --2- 172.21.15.47:0/2375783506 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1f980fdbd0 0x7f1f980f7b70 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 rx=0x7f1f80002760 tx=0x7f1f80009e80).stop 2024-02-23T00:53:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.285+0000 7f1f9dec8700 1 -- 172.21.15.47:0/2375783506 shutdown_connections 2024-02-23T00:53:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.285+0000 7f1f9dec8700 1 --2- 172.21.15.47:0/2375783506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f980fe530 0x7f1f980f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.285+0000 7f1f9dec8700 1 --2- 172.21.15.47:0/2375783506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f980f6a90 0x7f1f980f7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.285+0000 7f1f9dec8700 1 --2- 172.21.15.47:0/2375783506 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1f84062cc0 0x7f1f84065180 unknown :-1 s=CLOSED pgs=3161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.285+0000 7f1f9dec8700 1 --2- 172.21.15.47:0/2375783506 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1f980fdbd0 0x7f1f980f7b70 unknown :-1 s=CLOSED pgs=2771 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.285+0000 7f1f9dec8700 1 -- 172.21.15.47:0/2375783506 >> 172.21.15.47:0/2375783506 conn(0x7f1f9806c370 msgr2=0x7f1f980f56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:35.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.285+0000 7f1f9dec8700 1 -- 172.21.15.47:0/2375783506 shutdown_connections 2024-02-23T00:53:35.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.285+0000 7f1f9dec8700 1 -- 172.21.15.47:0/2375783506 wait complete. 2024-02-23T00:53:35.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:35.285+0000 7f1f9dec8700 1 librados: shutdown 2024-02-23T00:53:35.288 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is unset 2024-02-23T00:53:35.322 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:53:35.350 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:35.422 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:53:35.424 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:35.425 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:53:35.525 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:53:35.553 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:35.752 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:53:35.754 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:35.754 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:53:35.855 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:53:35.884 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:34.439083+0000 osd.4 (osd.4) 285 : cluster [DBG] 4.e scrub starts 2024-02-23T00:53:35.885 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:34.440435+0000 osd.4 (osd.4) 286 : cluster [DBG] 4.e scrub ok 2024-02-23T00:53:35.885 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:35.189500+0000 osd.5 (osd.5) 297 : cluster [DBG] 2.12 scrub starts 2024-02-23T00:53:35.885 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:35.190283+0000 osd.5 (osd.5) 298 : cluster [DBG] 2.12 scrub ok 2024-02-23T00:53:35.885 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:35.227716+0000 mon.c (mon.2) 392 : audit [INF] from='client.? 172.21.15.47:0/2375783506' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:53:35.885 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:35.228076+0000 mon.a (mon.0) 3356 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:53:35.885 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:35.276865+0000 mon.a (mon.0) 3357 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:53:35.885 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:35.281374+0000 mon.a (mon.0) 3358 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:53:35.885 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:35.281435+0000 mon.a (mon.0) 3359 : cluster [DBG] osdmap e678: 8 total, 8 up, 6 in 2024-02-23T00:53:35.885 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:34.439083+0000 osd.4 (osd.4) 285 : cluster [DBG] 4.e scrub starts 2024-02-23T00:53:35.885 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:34.440435+0000 osd.4 (osd.4) 286 : cluster [DBG] 4.e scrub ok 2024-02-23T00:53:35.885 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:35.189500+0000 osd.5 (osd.5) 297 : cluster [DBG] 2.12 scrub starts 2024-02-23T00:53:35.886 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:35.190283+0000 osd.5 (osd.5) 298 : cluster [DBG] 2.12 scrub ok 2024-02-23T00:53:35.886 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:35.227716+0000 mon.c (mon.2) 392 : audit [INF] from='client.? 172.21.15.47:0/2375783506' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:53:35.886 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:35.228076+0000 mon.a (mon.0) 3356 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:53:35.886 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:35.276865+0000 mon.a (mon.0) 3357 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:53:35.886 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:35.281374+0000 mon.a (mon.0) 3358 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:53:35.886 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:35.281435+0000 mon.a (mon.0) 3359 : cluster [DBG] osdmap e678: 8 total, 8 up, 6 in 2024-02-23T00:53:35.887 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:35.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:34.439083+0000 osd.4 (osd.4) 285 : cluster [DBG] 4.e scrub starts 2024-02-23T00:53:35.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:34.440435+0000 osd.4 (osd.4) 286 : cluster [DBG] 4.e scrub ok 2024-02-23T00:53:35.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:35.189500+0000 osd.5 (osd.5) 297 : cluster [DBG] 2.12 scrub starts 2024-02-23T00:53:35.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:35.190283+0000 osd.5 (osd.5) 298 : cluster [DBG] 2.12 scrub ok 2024-02-23T00:53:35.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:35.227716+0000 mon.c (mon.2) 392 : audit [INF] from='client.? 172.21.15.47:0/2375783506' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:53:35.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:35.228076+0000 mon.a (mon.0) 3356 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:53:35.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:35.276865+0000 mon.a (mon.0) 3357 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:53:35.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:35.281374+0000 mon.a (mon.0) 3358 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:53:35.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:35.281435+0000 mon.a (mon.0) 3359 : cluster [DBG] osdmap e678: 8 total, 8 up, 6 in 2024-02-23T00:53:36.539 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:53:36.542 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:36.543 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:53:36.643 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:53:36.676 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:36.802 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:53:36.802 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-02-23T00:53:36.802 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-02-23T00:53:36.802 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-02-23T00:53:36.802 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.802 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.31603.0:7 79.6s0 79:6ddaff94:test-rados-api-smithi047-82487-12::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e145)", 2024-02-23T00:53:36.802 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:37.207111+0000", 2024-02-23T00:53:36.803 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 599.59078558700003, 2024-02-23T00:53:36.803 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.050316774000000002, 2024-02-23T00:53:36.803 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:53:36.803 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:53:36.803 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:53:36.803 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.31603", 2024-02-23T00:53:36.803 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1361361966", 2024-02-23T00:53:36.803 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 7 2024-02-23T00:53:36.803 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.803 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:53:36.804 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.804 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:53:36.804 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.207111+0000", 2024-02-23T00:53:36.804 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:53:36.804 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.804 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.804 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:53:36.804 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.207111+0000", 2024-02-23T00:53:36.804 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:53:36.804 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.804 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.804 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:53:36.804 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.207109+0000", 2024-02-23T00:53:36.805 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.571e-06 2024-02-23T00:53:36.805 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.805 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.805 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:53:36.805 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.207117+0000", 2024-02-23T00:53:36.805 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0654e-05 2024-02-23T00:53:36.805 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.805 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.805 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:53:36.805 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.207127+0000", 2024-02-23T00:53:36.805 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0709999999999997e-06 2024-02-23T00:53:36.805 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.805 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.805 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:53:36.806 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.207134+0000", 2024-02-23T00:53:36.806 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010019800000000001 2024-02-23T00:53:36.806 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.806 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.806 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.806 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.207235+0000", 2024-02-23T00:53:36.806 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013806100000000001 2024-02-23T00:53:36.806 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.806 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.806 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:53:36.806 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.207373+0000", 2024-02-23T00:53:36.806 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00043899 2024-02-23T00:53:36.806 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.806 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.806 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:53:36.807 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.207812+0000", 2024-02-23T00:53:36.807 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00089506700000000004 2024-02-23T00:53:36.807 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.807 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.807 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:53:36.807 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.208707+0000", 2024-02-23T00:53:36.807 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.048651597999999997 2024-02-23T00:53:36.807 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.807 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.807 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:53:36.807 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.257358+0000", 2024-02-23T00:53:36.807 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.9276000000000006e-05 2024-02-23T00:53:36.808 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.808 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.808 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:53:36.808 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:37.257428+0000", 2024-02-23T00:53:36.808 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.050316774000000002 2024-02-23T00:53:36.808 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.808 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:53:36.808 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.808 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.808 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.808 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.31603.0:9 79.6s0 79:6ddaff94:test-rados-api-smithi047-82487-12::foo:head [writefull 0~128] snapc 0=[] ondisk+write+known_if_redirected e146)", 2024-02-23T00:53:36.808 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:43:38.402504+0000", 2024-02-23T00:53:36.808 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 598.39539205200003, 2024-02-23T00:53:36.808 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036230714999999997, 2024-02-23T00:53:36.809 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:53:36.809 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:53:36.809 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:53:36.809 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.31603", 2024-02-23T00:53:36.809 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1361361966", 2024-02-23T00:53:36.809 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 9 2024-02-23T00:53:36.809 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.809 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:53:36.809 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.809 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:53:36.809 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.402504+0000", 2024-02-23T00:53:36.809 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:53:36.809 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.809 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.810 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:53:36.810 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.402504+0000", 2024-02-23T00:53:36.810 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:53:36.810 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.810 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.810 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:53:36.810 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.402503+0000", 2024-02-23T00:53:36.810 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.6512e-05 2024-02-23T00:53:36.810 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.810 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.810 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:53:36.810 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.402529+0000", 2024-02-23T00:53:36.810 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7428999999999999e-05 2024-02-23T00:53:36.810 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.811 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.811 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:53:36.811 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.402547+0000", 2024-02-23T00:53:36.811 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2077e-05 2024-02-23T00:53:36.811 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.811 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.811 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:53:36.811 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.402559+0000", 2024-02-23T00:53:36.811 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000108826 2024-02-23T00:53:36.811 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.811 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.811 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.811 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.402668+0000", 2024-02-23T00:53:36.812 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010388299999999999 2024-02-23T00:53:36.812 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.812 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.812 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:53:36.812 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.402771+0000", 2024-02-23T00:53:36.812 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00051973999999999998 2024-02-23T00:53:36.812 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.812 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.812 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:53:36.812 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.403291+0000", 2024-02-23T00:53:36.813 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00049441100000000005 2024-02-23T00:53:36.813 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.813 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.813 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:53:36.813 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.403786+0000", 2024-02-23T00:53:36.813 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034898881 2024-02-23T00:53:36.813 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.813 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.813 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:53:36.813 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.438685+0000", 2024-02-23T00:53:36.814 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0618000000000002e-05 2024-02-23T00:53:36.814 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.814 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.814 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:53:36.814 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:43:38.438735+0000", 2024-02-23T00:53:36.814 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036230714999999997 2024-02-23T00:53:36.814 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.814 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:53:36.814 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.814 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.814 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.814 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.24311.0:1282 4.4 4:22d26bf9:::data_loggenerations_metadata:head [watch ping cookie 94587070406528 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370)", 2024-02-23T00:53:36.814 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:47:27.941616+0000", 2024-02-23T00:53:36.814 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 368.85628038800002, 2024-02-23T00:53:36.815 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.30363403799999999, 2024-02-23T00:53:36.815 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:53:36.815 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:53:36.815 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:53:36.815 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.24311", 2024-02-23T00:53:36.815 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/3723202188", 2024-02-23T00:53:36.815 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1282 2024-02-23T00:53:36.815 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.815 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:53:36.815 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.815 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:53:36.815 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.941616+0000", 2024-02-23T00:53:36.816 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:53:36.816 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.816 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.816 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:53:36.816 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.941616+0000", 2024-02-23T00:53:36.816 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:53:36.816 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.816 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.816 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:53:36.816 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.941615+0000", 2024-02-23T00:53:36.817 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3249999999999998e-06 2024-02-23T00:53:36.817 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.817 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.817 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:53:36.817 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.941621+0000", 2024-02-23T00:53:36.817 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1557e-05 2024-02-23T00:53:36.817 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.818 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.818 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:53:36.818 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.941642+0000", 2024-02-23T00:53:36.818 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3829999999999997e-06 2024-02-23T00:53:36.818 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.818 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.819 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:53:36.819 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.941648+0000", 2024-02-23T00:53:36.819 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029746900000000002 2024-02-23T00:53:36.819 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.819 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.819 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.820 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.941946+0000", 2024-02-23T00:53:36.820 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3587000000000001e-05 2024-02-23T00:53:36.820 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.820 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.820 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:53:36.820 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.941960+0000", 2024-02-23T00:53:36.821 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.29858716899999999 2024-02-23T00:53:36.821 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.821 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.821 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.821 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:28.240547+0000", 2024-02-23T00:53:36.821 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001598099 2024-02-23T00:53:36.822 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.822 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.822 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for degraded object", 2024-02-23T00:53:36.822 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:28.242145+0000", 2024-02-23T00:53:36.822 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0029042780000000002 2024-02-23T00:53:36.822 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.822 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.822 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.823 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:28.245049+0000", 2024-02-23T00:53:36.823 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.7250999999999995e-05 2024-02-23T00:53:36.823 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.823 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.823 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:53:36.823 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:28.245136+0000", 2024-02-23T00:53:36.823 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000113793 2024-02-23T00:53:36.824 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.824 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.824 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:53:36.824 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:28.245250+0000", 2024-02-23T00:53:36.824 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.30363403799999999 2024-02-23T00:53:36.824 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.825 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:53:36.825 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.825 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.825 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.825 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.24311.0:1287 4.4 4:22d26bf9:::data_loggenerations_metadata:head [watch reconnect cookie 94587070406528 gen 2] snapc 0=[] ondisk+write+known_if_redirected e375)", 2024-02-23T00:53:36.825 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:47:27.942624+0000", 2024-02-23T00:53:36.826 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 368.85527198099999, 2024-02-23T00:53:36.826 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.30870664599999997, 2024-02-23T00:53:36.826 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:53:36.826 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:53:36.826 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:53:36.826 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.24311", 2024-02-23T00:53:36.827 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/3723202188", 2024-02-23T00:53:36.827 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1287 2024-02-23T00:53:36.827 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.827 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:53:36.827 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.827 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:53:36.827 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.942624+0000", 2024-02-23T00:53:36.827 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:53:36.828 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.828 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.828 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:53:36.828 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.942624+0000", 2024-02-23T00:53:36.828 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:53:36.828 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.828 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.828 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:53:36.828 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.942623+0000", 2024-02-23T00:53:36.828 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9109999999999993e-06 2024-02-23T00:53:36.828 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.829 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.829 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:53:36.829 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.942631+0000", 2024-02-23T00:53:36.829 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5931999999999999e-05 2024-02-23T00:53:36.829 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.829 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.829 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:53:36.829 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.942647+0000", 2024-02-23T00:53:36.829 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.4319999999999999e-06 2024-02-23T00:53:36.829 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.829 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.829 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:53:36.829 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.942655+0000", 2024-02-23T00:53:36.829 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6074e-05 2024-02-23T00:53:36.830 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.830 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.830 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.830 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.942702+0000", 2024-02-23T00:53:36.830 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3239999999999999e-06 2024-02-23T00:53:36.830 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.830 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.830 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:53:36.830 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:27.942708+0000", 2024-02-23T00:53:36.830 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.29946395100000001 2024-02-23T00:53:36.830 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.830 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.830 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.830 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:28.242172+0000", 2024-02-23T00:53:36.830 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.5534999999999998e-05 2024-02-23T00:53:36.831 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.831 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.831 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for degraded object", 2024-02-23T00:53:36.831 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:28.242207+0000", 2024-02-23T00:53:36.831 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0030798739999999998 2024-02-23T00:53:36.831 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.831 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.831 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.831 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:28.245287+0000", 2024-02-23T00:53:36.831 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5687999999999998e-05 2024-02-23T00:53:36.831 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.831 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.831 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:53:36.831 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:28.245363+0000", 2024-02-23T00:53:36.831 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0059682490000000001 2024-02-23T00:53:36.832 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.832 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.832 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:53:36.832 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:28.251331+0000", 2024-02-23T00:53:36.832 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.30870664599999997 2024-02-23T00:53:36.832 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.832 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:53:36.832 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.832 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.832 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.832 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.15270.0:7718 62.1d 62:bd8d0095:test-rados-api-smithi047-82465-89::propagate-base-tier-error:head [omap-cmp,omap-set-vals] snapc 0=[] ondisk+write+ignore_cache+ignore_overlay+known_if_redirected e539)", 2024-02-23T00:53:36.832 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:51:08.248873+0000", 2024-02-23T00:53:36.832 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 148.54902355999999, 2024-02-23T00:53:36.833 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0034122480000000001, 2024-02-23T00:53:36.833 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:53:36.833 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:53:36.833 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:53:36.833 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.833 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:53:36.833 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:51:08.248873+0000", 2024-02-23T00:53:36.833 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:53:36.833 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.833 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.833 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:53:36.833 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:51:08.248873+0000", 2024-02-23T00:53:36.833 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999976 2024-02-23T00:53:36.833 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.834 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.834 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:53:36.834 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:51:08.248870+0000", 2024-02-23T00:53:36.834 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4918e-05 2024-02-23T00:53:36.834 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.834 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.834 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:53:36.834 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:51:08.248885+0000", 2024-02-23T00:53:36.834 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0539000000000001e-05 2024-02-23T00:53:36.834 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.834 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.834 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:53:36.834 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:51:08.248906+0000", 2024-02-23T00:53:36.834 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1509999999999993e-06 2024-02-23T00:53:36.835 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.835 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.835 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:53:36.835 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:51:08.248915+0000", 2024-02-23T00:53:36.835 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9061000000000003e-05 2024-02-23T00:53:36.835 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.835 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.835 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.835 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:51:08.248974+0000", 2024-02-23T00:53:36.835 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1719999999999998e-06 2024-02-23T00:53:36.835 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.835 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.835 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op must wait for map", 2024-02-23T00:53:36.835 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:51:08.248979+0000", 2024-02-23T00:53:36.836 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.002122685 2024-02-23T00:53:36.836 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.836 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.836 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.836 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:51:08.251102+0000", 2024-02-23T00:53:36.836 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.2075000000000002e-05 2024-02-23T00:53:36.836 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.836 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.836 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:53:36.836 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:51:08.251194+0000", 2024-02-23T00:53:36.836 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001091269 2024-02-23T00:53:36.837 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.837 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.837 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:53:36.837 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:51:08.252285+0000", 2024-02-23T00:53:36.837 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0034122480000000001 2024-02-23T00:53:36.837 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.837 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:53:36.837 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.837 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.837 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.837 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "MOSDScrubReserve(5.19 REQUEST e642)", 2024-02-23T00:53:36.837 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:52:59.524331+0000", 2024-02-23T00:53:36.837 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 37.273565757, 2024-02-23T00:53:36.838 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0071273270000000001, 2024-02-23T00:53:36.838 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:53:36.838 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:53:36.838 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:53:36.838 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.838 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:53:36.838 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:52:59.524331+0000", 2024-02-23T00:53:36.838 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:53:36.838 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.839 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.839 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:53:36.839 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:52:59.524331+0000", 2024-02-23T00:53:36.839 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967296 2024-02-23T00:53:36.839 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.839 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.839 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:53:36.839 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:52:59.524331+0000", 2024-02-23T00:53:36.839 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.287e-06 2024-02-23T00:53:36.839 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.840 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.840 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:53:36.840 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:52:59.524334+0000", 2024-02-23T00:53:36.840 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.3139999999999999e-05 2024-02-23T00:53:36.840 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.841 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.841 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:53:36.841 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:52:59.524357+0000", 2024-02-23T00:53:36.841 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.3510000000000008e-06 2024-02-23T00:53:36.841 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.841 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.841 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:53:36.841 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:52:59.524366+0000", 2024-02-23T00:53:36.841 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111863 2024-02-23T00:53:36.841 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.841 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.842 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.842 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:52:59.524478+0000", 2024-02-23T00:53:36.842 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.7440000000000007e-06 2024-02-23T00:53:36.842 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.842 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.842 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op must wait for map", 2024-02-23T00:53:36.842 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:52:59.524486+0000", 2024-02-23T00:53:36.842 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0069414389999999998 2024-02-23T00:53:36.843 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.843 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.843 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.843 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:52:59.531427+0000", 2024-02-23T00:53:36.843 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3403e-05 2024-02-23T00:53:36.843 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.843 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.843 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:53:36.843 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:52:59.531441+0000", 2024-02-23T00:53:36.843 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7280000000000001e-05 2024-02-23T00:53:36.843 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.843 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.844 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:53:36.844 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:52:59.531458+0000", 2024-02-23T00:53:36.844 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0071273270000000001 2024-02-23T00:53:36.844 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.844 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:53:36.844 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.844 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.844 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.845 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.41771.0:791 234.3s0 234:d2733045:test-rados-api-smithi047-82465-115::foo:head [writefull 0~8] snapc 0=[] ondisk+write+known_if_redirected e661)", 2024-02-23T00:53:36.845 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:53:18.284258+0000", 2024-02-23T00:53:36.845 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 18.513638118999999, 2024-02-23T00:53:36.845 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.020429075000000001, 2024-02-23T00:53:36.845 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:53:36.845 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:53:36.845 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:53:36.845 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.41771", 2024-02-23T00:53:36.845 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1032768195", 2024-02-23T00:53:36.845 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 791 2024-02-23T00:53:36.846 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.846 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:53:36.846 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.846 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:53:36.846 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:18.284258+0000", 2024-02-23T00:53:36.846 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:53:36.846 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.846 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.846 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:53:36.846 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:18.284258+0000", 2024-02-23T00:53:36.846 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999976 2024-02-23T00:53:36.846 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.847 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.847 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:53:36.847 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:18.284256+0000", 2024-02-23T00:53:36.847 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2877999999999999e-05 2024-02-23T00:53:36.847 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.847 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.847 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:53:36.847 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:18.284269+0000", 2024-02-23T00:53:36.847 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9595000000000001e-05 2024-02-23T00:53:36.847 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.847 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.847 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:53:36.847 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:18.284289+0000", 2024-02-23T00:53:36.847 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4560000000000006e-06 2024-02-23T00:53:36.848 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.848 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.848 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:53:36.848 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:18.284298+0000", 2024-02-23T00:53:36.848 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3637000000000001e-05 2024-02-23T00:53:36.848 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.848 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.848 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.848 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:18.284362+0000", 2024-02-23T00:53:36.848 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7690000000000008e-06 2024-02-23T00:53:36.848 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.848 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.849 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op must wait for map", 2024-02-23T00:53:36.849 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:18.284371+0000", 2024-02-23T00:53:36.849 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0051526200000000001 2024-02-23T00:53:36.849 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.849 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.849 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.849 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:18.289524+0000", 2024-02-23T00:53:36.849 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013932899999999999 2024-02-23T00:53:36.849 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.851 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.851 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:53:36.851 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:18.289663+0000", 2024-02-23T00:53:36.851 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00048397099999999998 2024-02-23T00:53:36.851 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.851 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.851 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:53:36.851 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:18.290147+0000", 2024-02-23T00:53:36.851 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00065817499999999999 2024-02-23T00:53:36.851 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.852 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.852 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:53:36.852 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:18.290806+0000", 2024-02-23T00:53:36.852 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013801860000000001 2024-02-23T00:53:36.852 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.852 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.852 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:53:36.852 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:18.304607+0000", 2024-02-23T00:53:36.852 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0037000000000006e-05 2024-02-23T00:53:36.852 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.852 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.852 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:53:36.852 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:18.304687+0000", 2024-02-23T00:53:36.853 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.020429075000000001 2024-02-23T00:53:36.853 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.853 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:53:36.853 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.853 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.853 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.853 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.41771.0:796 234.3s0 234:d2733045:test-rados-api-smithi047-82465-115::foo:head [writefull 0~5] snapc 6=[6] ondisk+write+known_if_redirected e662)", 2024-02-23T00:53:36.853 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:53:19.291673+0000", 2024-02-23T00:53:36.853 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 17.506223103, 2024-02-23T00:53:36.853 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0040996970000000002, 2024-02-23T00:53:36.853 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:53:36.853 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:53:36.853 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:53:36.853 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.41771", 2024-02-23T00:53:36.853 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1032768195", 2024-02-23T00:53:36.854 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 796 2024-02-23T00:53:36.854 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.854 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:53:36.854 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.854 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:53:36.854 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:19.291673+0000", 2024-02-23T00:53:36.854 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:53:36.854 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.854 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.854 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:53:36.854 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:19.291673+0000", 2024-02-23T00:53:36.854 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:53:36.854 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.854 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.855 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:53:36.855 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:19.291671+0000", 2024-02-23T00:53:36.855 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3298e-05 2024-02-23T00:53:36.855 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.855 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.855 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:53:36.855 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:19.291685+0000", 2024-02-23T00:53:36.855 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9194e-05 2024-02-23T00:53:36.856 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.856 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.856 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:53:36.856 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:19.291704+0000", 2024-02-23T00:53:36.856 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.3969999999999997e-06 2024-02-23T00:53:36.856 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.857 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.857 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:53:36.857 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:19.291711+0000", 2024-02-23T00:53:36.857 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8281000000000003e-05 2024-02-23T00:53:36.857 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.857 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.857 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.857 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:19.291809+0000", 2024-02-23T00:53:36.857 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2169999999999997e-06 2024-02-23T00:53:36.857 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.857 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.857 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op must wait for map", 2024-02-23T00:53:36.857 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:19.291817+0000", 2024-02-23T00:53:36.857 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0022196199999999998 2024-02-23T00:53:36.857 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.857 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.858 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.858 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:19.294036+0000", 2024-02-23T00:53:36.858 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010186199999999999 2024-02-23T00:53:36.858 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.858 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.858 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:53:36.858 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:19.294138+0000", 2024-02-23T00:53:36.858 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00066619200000000002 2024-02-23T00:53:36.858 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.858 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.858 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:53:36.858 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:19.294804+0000", 2024-02-23T00:53:36.858 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00056617799999999997 2024-02-23T00:53:36.858 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.858 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.858 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:53:36.859 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:19.295370+0000", 2024-02-23T00:53:36.859 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00035144000000000001 2024-02-23T00:53:36.859 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.859 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.859 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:53:36.859 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:19.295722+0000", 2024-02-23T00:53:36.859 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1128999999999998e-05 2024-02-23T00:53:36.859 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.859 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.859 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:53:36.859 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:19.295773+0000", 2024-02-23T00:53:36.859 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0040996970000000002 2024-02-23T00:53:36.859 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.859 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:53:36.860 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.860 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.860 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.860 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(osd.4.32:69 234.3s0 234:d2733045:test-rados-api-smithi047-82465-115::foo:snapdir [list-snaps] snapc 0=[] ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected e665)", 2024-02-23T00:53:36.860 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:53:22.308856+0000", 2024-02-23T00:53:36.860 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 14.489040572, 2024-02-23T00:53:36.860 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0027807660000000001, 2024-02-23T00:53:36.860 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:53:36.860 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:53:36.860 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:53:36.860 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.860 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:53:36.860 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.308856+0000", 2024-02-23T00:53:36.861 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:53:36.861 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.861 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.861 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:53:36.861 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.308856+0000", 2024-02-23T00:53:36.861 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:53:36.861 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.861 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.861 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:53:36.861 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.308854+0000", 2024-02-23T00:53:36.861 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.5639999999999997e-06 2024-02-23T00:53:36.861 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.862 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.862 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:53:36.862 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.308863+0000", 2024-02-23T00:53:36.862 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6946e-05 2024-02-23T00:53:36.862 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.862 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.862 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:53:36.862 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.308880+0000", 2024-02-23T00:53:36.862 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8789999999999997e-06 2024-02-23T00:53:36.863 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.863 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.863 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:53:36.863 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.308887+0000", 2024-02-23T00:53:36.863 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4698999999999998e-05 2024-02-23T00:53:36.863 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.863 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.863 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.863 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.308951+0000", 2024-02-23T00:53:36.863 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4020000000000008e-06 2024-02-23T00:53:36.863 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.863 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.864 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op must wait for map", 2024-02-23T00:53:36.864 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.308961+0000", 2024-02-23T00:53:36.864 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0025073589999999998 2024-02-23T00:53:36.864 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.864 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.864 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.864 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.311468+0000", 2024-02-23T00:53:36.864 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3487999999999998e-05 2024-02-23T00:53:36.864 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.864 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.864 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:53:36.864 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.311552+0000", 2024-02-23T00:53:36.864 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.5138999999999999e-05 2024-02-23T00:53:36.864 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.865 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.865 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:53:36.865 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.311637+0000", 2024-02-23T00:53:36.865 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0027807660000000001 2024-02-23T00:53:36.865 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.865 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:53:36.865 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.865 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.865 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.865 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(osd.4.32:70 234.3s0 234:d2733045:test-rados-api-smithi047-82465-115::foo:head [copy-get max 8388608] snapc 0=[] ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected e665)", 2024-02-23T00:53:36.865 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:53:22.308895+0000", 2024-02-23T00:53:36.866 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 14.489001051000001, 2024-02-23T00:53:36.866 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0035952760000000001, 2024-02-23T00:53:36.866 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:53:36.866 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:53:36.866 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:53:36.866 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.866 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:53:36.866 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.308895+0000", 2024-02-23T00:53:36.866 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:53:36.866 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.866 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.866 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:53:36.867 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.308895+0000", 2024-02-23T00:53:36.867 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999995 2024-02-23T00:53:36.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.867 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.867 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:53:36.867 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.308895+0000", 2024-02-23T00:53:36.867 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.5999999999999998e-06 2024-02-23T00:53:36.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.867 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.867 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:53:36.867 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.308899+0000", 2024-02-23T00:53:36.867 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.8240000000000001e-06 2024-02-23T00:53:36.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.867 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.868 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:53:36.868 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.308902+0000", 2024-02-23T00:53:36.868 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4729999999999999e-06 2024-02-23T00:53:36.868 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.868 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.868 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:53:36.868 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.308905+0000", 2024-02-23T00:53:36.868 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000109005 2024-02-23T00:53:36.868 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.868 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.868 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.868 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.309014+0000", 2024-02-23T00:53:36.868 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6359999999999997e-06 2024-02-23T00:53:36.868 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.869 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.869 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting_for_map not empty", 2024-02-23T00:53:36.869 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.309021+0000", 2024-02-23T00:53:36.869 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.002648933 2024-02-23T00:53:36.869 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.869 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.869 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.869 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.311669+0000", 2024-02-23T00:53:36.869 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7318000000000003e-05 2024-02-23T00:53:36.869 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.869 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.869 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:53:36.869 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.311717+0000", 2024-02-23T00:53:36.870 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00077393000000000004 2024-02-23T00:53:36.870 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.870 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.870 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:53:36.870 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.312491+0000", 2024-02-23T00:53:36.870 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0035952760000000001 2024-02-23T00:53:36.870 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.870 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:53:36.870 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.870 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.870 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.870 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.41771.0:808 241.7 241:e4db8f9e:test-rados-api-smithi047-82465-115::bar:head [read 0~1] snapc 0=[] ondisk+read+known_if_redirected e665)", 2024-02-23T00:53:36.871 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:53:22.325830+0000", 2024-02-23T00:53:36.871 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 14.472066714, 2024-02-23T00:53:36.871 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0049289379999999999, 2024-02-23T00:53:36.871 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:53:36.871 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:53:36.871 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:53:36.871 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.41771", 2024-02-23T00:53:36.871 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1032768195", 2024-02-23T00:53:36.871 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 808 2024-02-23T00:53:36.871 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.871 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:53:36.871 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.871 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:53:36.871 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.325830+0000", 2024-02-23T00:53:36.871 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:53:36.872 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.872 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.872 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:53:36.872 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.325830+0000", 2024-02-23T00:53:36.872 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:53:36.872 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.872 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.872 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:53:36.872 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.325829+0000", 2024-02-23T00:53:36.872 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.951e-06 2024-02-23T00:53:36.872 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.872 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.872 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:53:36.873 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.325835+0000", 2024-02-23T00:53:36.873 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0811e-05 2024-02-23T00:53:36.873 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.873 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.873 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:53:36.873 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.325846+0000", 2024-02-23T00:53:36.873 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6110000000000004e-06 2024-02-23T00:53:36.873 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.873 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.873 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:53:36.873 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.325851+0000", 2024-02-23T00:53:36.873 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6496999999999999e-05 2024-02-23T00:53:36.873 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.873 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.873 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.874 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.325897+0000", 2024-02-23T00:53:36.874 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012361300000000001 2024-02-23T00:53:36.874 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.874 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.874 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for blocked object", 2024-02-23T00:53:36.874 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.326021+0000", 2024-02-23T00:53:36.874 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0018023749999999999 2024-02-23T00:53:36.874 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.874 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.874 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.874 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.327823+0000", 2024-02-23T00:53:36.874 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.7247e-05 2024-02-23T00:53:36.874 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.874 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.875 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for rw locks", 2024-02-23T00:53:36.875 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.327911+0000", 2024-02-23T00:53:36.875 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00027270499999999999 2024-02-23T00:53:36.875 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.875 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.875 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.875 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.328183+0000", 2024-02-23T00:53:36.875 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.8801000000000002e-05 2024-02-23T00:53:36.875 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.875 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.875 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:53:36.875 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.328242+0000", 2024-02-23T00:53:36.875 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0025165500000000002 2024-02-23T00:53:36.875 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.875 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.876 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:53:36.876 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.330759+0000", 2024-02-23T00:53:36.876 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0049289379999999999 2024-02-23T00:53:36.876 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.876 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:53:36.876 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.876 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.876 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.876 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.41771.0:809 241.7 241:e4db8f9e:test-rados-api-smithi047-82465-115::bar:6 [read 0~1] snapc 0=[] ondisk+read+known_if_redirected e665)", 2024-02-23T00:53:36.876 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:53:22.331025+0000", 2024-02-23T00:53:36.876 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 14.466871132, 2024-02-23T00:53:36.876 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0044728839999999999, 2024-02-23T00:53:36.876 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:53:36.877 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:53:36.877 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:53:36.877 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.41771", 2024-02-23T00:53:36.877 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1032768195", 2024-02-23T00:53:36.877 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 809 2024-02-23T00:53:36.877 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.877 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:53:36.877 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.877 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:53:36.877 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.331025+0000", 2024-02-23T00:53:36.878 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:53:36.878 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.878 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.878 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:53:36.878 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.331025+0000", 2024-02-23T00:53:36.878 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:53:36.878 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.878 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.878 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:53:36.878 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.331024+0000", 2024-02-23T00:53:36.878 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9200000000000001e-06 2024-02-23T00:53:36.878 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.878 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.878 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:53:36.878 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.331030+0000", 2024-02-23T00:53:36.879 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.219e-05 2024-02-23T00:53:36.879 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.879 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.879 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:53:36.879 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.331042+0000", 2024-02-23T00:53:36.879 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4300000000000003e-06 2024-02-23T00:53:36.879 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.879 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.879 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:53:36.879 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.331049+0000", 2024-02-23T00:53:36.879 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6910999999999999e-05 2024-02-23T00:53:36.879 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.879 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.879 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.879 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.331096+0000", 2024-02-23T00:53:36.879 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015493000000000001 2024-02-23T00:53:36.880 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.880 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.880 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for blocked object", 2024-02-23T00:53:36.880 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.331251+0000", 2024-02-23T00:53:36.880 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0013701869999999999 2024-02-23T00:53:36.880 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.880 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.880 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.880 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.332621+0000", 2024-02-23T00:53:36.880 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000105726 2024-02-23T00:53:36.880 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.880 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.881 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for rw locks", 2024-02-23T00:53:36.881 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.332727+0000", 2024-02-23T00:53:36.881 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024718199999999998 2024-02-23T00:53:36.881 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.881 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.881 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.881 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.332974+0000", 2024-02-23T00:53:36.881 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0055e-05 2024-02-23T00:53:36.881 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.881 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.881 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:53:36.881 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.333034+0000", 2024-02-23T00:53:36.881 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0024643719999999998 2024-02-23T00:53:36.881 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.882 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.882 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:53:36.882 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.335498+0000", 2024-02-23T00:53:36.882 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0044728839999999999 2024-02-23T00:53:36.882 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.882 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:53:36.882 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.882 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.882 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.882 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "MOSDECSubOpWrite(234.7s1 666/600 ECSubWrite(tid=831, reqid=osd.7.0:831, at_version=666'13, trim_to=0'0, roll_forward_to=662'11))", 2024-02-23T00:53:36.882 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:53:23.327400+0000", 2024-02-23T00:53:36.882 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 13.470496627999999, 2024-02-23T00:53:36.882 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0044059090000000004, 2024-02-23T00:53:36.883 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:53:36.883 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "reached pg", 2024-02-23T00:53:36.883 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:53:36.883 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.883 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:53:36.883 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.327400+0000", 2024-02-23T00:53:36.883 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:53:36.883 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.883 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.883 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:53:36.883 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.327400+0000", 2024-02-23T00:53:36.883 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967296 2024-02-23T00:53:36.883 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.883 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.884 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:53:36.884 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.327400+0000", 2024-02-23T00:53:36.884 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9933e-05 2024-02-23T00:53:36.884 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.884 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.884 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:53:36.884 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.327420+0000", 2024-02-23T00:53:36.884 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9649999999999997e-06 2024-02-23T00:53:36.884 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.884 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.884 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:53:36.884 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.327429+0000", 2024-02-23T00:53:36.884 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6229999999999999e-06 2024-02-23T00:53:36.885 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.885 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.885 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:53:36.885 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.327435+0000", 2024-02-23T00:53:36.885 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0024721539999999998 2024-02-23T00:53:36.885 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.885 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.885 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.885 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.329907+0000", 2024-02-23T00:53:36.885 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.8999999999999995e-06 2024-02-23T00:53:36.885 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.885 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.886 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:53:36.886 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.329916+0000", 2024-02-23T00:53:36.886 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0018708819999999999 2024-02-23T00:53:36.886 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.886 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.886 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:53:36.886 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.331787+0000", 2024-02-23T00:53:36.886 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8651000000000002e-05 2024-02-23T00:53:36.886 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.886 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.886 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:53:36.886 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.331806+0000", 2024-02-23T00:53:36.887 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0044059090000000004 2024-02-23T00:53:36.887 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.887 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:53:36.887 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.887 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.887 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.887 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "MOSDECSubOpWrite(234.5s2 666/655 ECSubWrite(tid=798, reqid=osd.6.0:798, at_version=666'6, trim_to=0'0, roll_forward_to=662'4))", 2024-02-23T00:53:36.887 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:53:23.329781+0000", 2024-02-23T00:53:36.887 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 13.468115835000001, 2024-02-23T00:53:36.887 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0021713729999999999, 2024-02-23T00:53:36.887 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:53:36.887 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "reached pg", 2024-02-23T00:53:36.887 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:53:36.887 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.887 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:53:36.888 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.329781+0000", 2024-02-23T00:53:36.888 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:53:36.888 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.888 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.888 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:53:36.888 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.329781+0000", 2024-02-23T00:53:36.888 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999995 2024-02-23T00:53:36.888 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.888 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.888 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:53:36.888 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.329780+0000", 2024-02-23T00:53:36.888 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4167000000000001e-05 2024-02-23T00:53:36.888 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.888 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.889 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:53:36.889 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.329804+0000", 2024-02-23T00:53:36.889 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0940000000000008e-06 2024-02-23T00:53:36.889 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.889 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.889 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:53:36.889 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.329813+0000", 2024-02-23T00:53:36.889 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4049999999999997e-06 2024-02-23T00:53:36.889 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.889 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.889 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:53:36.889 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.329819+0000", 2024-02-23T00:53:36.889 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032476500000000003 2024-02-23T00:53:36.889 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.889 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.889 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.890 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.330144+0000", 2024-02-23T00:53:36.890 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.6519999999999995e-06 2024-02-23T00:53:36.890 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.890 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.890 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:53:36.890 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.330152+0000", 2024-02-23T00:53:36.890 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0017798 2024-02-23T00:53:36.890 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.890 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.890 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:53:36.890 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.331932+0000", 2024-02-23T00:53:36.890 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9851e-05 2024-02-23T00:53:36.890 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.890 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.890 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:53:36.890 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.331952+0000", 2024-02-23T00:53:36.890 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0021713729999999999 2024-02-23T00:53:36.891 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.891 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:53:36.891 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.891 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.891 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.891 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "MOSDECSubOpWrite(234.7s1 666/600 ECSubWrite(tid=832, reqid=osd.7.0:831, at_version=0'0, trim_to=0'0, roll_forward_to=666'13))", 2024-02-23T00:53:36.891 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:53:23.334506+0000", 2024-02-23T00:53:36.891 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 13.463390421, 2024-02-23T00:53:36.891 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001096238, 2024-02-23T00:53:36.891 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:53:36.891 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "reached pg", 2024-02-23T00:53:36.891 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:53:36.891 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.891 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:53:36.891 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.334506+0000", 2024-02-23T00:53:36.891 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:53:36.892 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.892 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.892 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:53:36.892 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.334506+0000", 2024-02-23T00:53:36.892 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:53:36.892 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.892 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.892 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:53:36.892 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.334505+0000", 2024-02-23T00:53:36.892 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1640999999999999e-05 2024-02-23T00:53:36.892 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.892 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.892 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:53:36.893 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.334517+0000", 2024-02-23T00:53:36.893 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2064e-05 2024-02-23T00:53:36.893 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.893 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.893 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:53:36.893 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.334529+0000", 2024-02-23T00:53:36.893 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9269000000000001e-05 2024-02-23T00:53:36.893 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.893 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.893 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:53:36.893 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.334548+0000", 2024-02-23T00:53:36.893 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3065999999999996e-05 2024-02-23T00:53:36.893 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.893 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.893 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.893 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.334611+0000", 2024-02-23T00:53:36.894 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.8389999999999996e-06 2024-02-23T00:53:36.894 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.894 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.894 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:53:36.894 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.334620+0000", 2024-02-23T00:53:36.894 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00096371300000000005 2024-02-23T00:53:36.894 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.894 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.894 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:53:36.894 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.335584+0000", 2024-02-23T00:53:36.894 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8417e-05 2024-02-23T00:53:36.894 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.895 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.895 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:53:36.895 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.335602+0000", 2024-02-23T00:53:36.895 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001096238 2024-02-23T00:53:36.895 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.895 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:53:36.895 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.895 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.895 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.895 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.41771.0:918 242.8 242:10000000:test-rados-api-smithi047-82465-117:::head [pgnls start_epoch 675] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e675)", 2024-02-23T00:53:36.895 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:53:32.296038+0000", 2024-02-23T00:53:36.895 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 4.5018589159999998, 2024-02-23T00:53:36.895 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001895398, 2024-02-23T00:53:36.895 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:53:36.895 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:53:36.896 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:53:36.896 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.41771", 2024-02-23T00:53:36.896 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1032768195", 2024-02-23T00:53:36.896 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 918 2024-02-23T00:53:36.896 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.896 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:53:36.896 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.896 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:53:36.896 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.296038+0000", 2024-02-23T00:53:36.896 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:53:36.896 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.896 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.896 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:53:36.897 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.296038+0000", 2024-02-23T00:53:36.897 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:53:36.897 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.897 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.897 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:53:36.897 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.296035+0000", 2024-02-23T00:53:36.897 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1960999999999999e-05 2024-02-23T00:53:36.897 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.897 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.897 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:53:36.897 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.296047+0000", 2024-02-23T00:53:36.897 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7430000000000001e-05 2024-02-23T00:53:36.897 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.897 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.898 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:53:36.898 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.296065+0000", 2024-02-23T00:53:36.898 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6390000000000003e-06 2024-02-23T00:53:36.898 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.898 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.898 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:53:36.898 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.296071+0000", 2024-02-23T00:53:36.898 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3231e-05 2024-02-23T00:53:36.898 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.898 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.898 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.898 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.296115+0000", 2024-02-23T00:53:36.898 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3170000000000003e-06 2024-02-23T00:53:36.898 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.898 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.898 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op must wait for map", 2024-02-23T00:53:36.899 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.296119+0000", 2024-02-23T00:53:36.899 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0016671430000000001 2024-02-23T00:53:36.899 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.899 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.899 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.899 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.297786+0000", 2024-02-23T00:53:36.899 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4178999999999999e-05 2024-02-23T00:53:36.899 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.899 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.899 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:53:36.899 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.297820+0000", 2024-02-23T00:53:36.899 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000112621 2024-02-23T00:53:36.899 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.899 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.899 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:53:36.900 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.297933+0000", 2024-02-23T00:53:36.900 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001895398 2024-02-23T00:53:36.900 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.900 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:53:36.900 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.900 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.900 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.900 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.24311.0:2073 5.15 5:a93a5511:::notify.2:head [watch ping cookie 94587070399616 gen 1] snapc 0=[] ondisk+write+known_if_redirected e670)", 2024-02-23T00:53:36.900 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:53:33.006789+0000", 2024-02-23T00:53:36.900 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 3.7911076349999999, 2024-02-23T00:53:36.900 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00061704399999999997, 2024-02-23T00:53:36.900 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:53:36.900 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:53:36.900 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:53:36.900 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.24311", 2024-02-23T00:53:36.901 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/3723202188", 2024-02-23T00:53:36.901 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 2073 2024-02-23T00:53:36.901 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.901 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:53:36.901 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.901 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:53:36.901 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:33.006789+0000", 2024-02-23T00:53:36.901 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:53:36.901 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.901 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.901 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:53:36.901 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:33.006789+0000", 2024-02-23T00:53:36.901 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999971 2024-02-23T00:53:36.901 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.901 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.901 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:53:36.902 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:33.006786+0000", 2024-02-23T00:53:36.902 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.677e-05 2024-02-23T00:53:36.902 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.902 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.902 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:53:36.902 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:33.006803+0000", 2024-02-23T00:53:36.902 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.9770000000000001e-05 2024-02-23T00:53:36.902 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.902 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.902 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:53:36.902 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:33.006832+0000", 2024-02-23T00:53:36.902 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0213e-05 2024-02-23T00:53:36.902 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.902 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.902 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:53:36.903 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:33.006843+0000", 2024-02-23T00:53:36.903 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038476299999999999 2024-02-23T00:53:36.903 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.903 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.903 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.903 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:33.007227+0000", 2024-02-23T00:53:36.903 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3867000000000002e-05 2024-02-23T00:53:36.903 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.903 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.903 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:53:36.903 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:33.007311+0000", 2024-02-23T00:53:36.903 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4532999999999998e-05 2024-02-23T00:53:36.904 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.904 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.904 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:53:36.904 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:33.007406+0000", 2024-02-23T00:53:36.904 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00061704399999999997 2024-02-23T00:53:36.904 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.904 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:53:36.904 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.904 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.904 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.904 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.24311.0:2074 4.4 4:22d26bf9:::data_loggenerations_metadata:head [watch ping cookie 94587070406528 gen 3] snapc 0=[] ondisk+write+known_if_redirected e670)", 2024-02-23T00:53:36.905 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:53:33.006865+0000", 2024-02-23T00:53:36.905 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 3.7910318460000001, 2024-02-23T00:53:36.905 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00067854799999999996, 2024-02-23T00:53:36.905 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:53:36.905 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:53:36.905 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:53:36.905 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.24311", 2024-02-23T00:53:36.905 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/3723202188", 2024-02-23T00:53:36.905 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 2074 2024-02-23T00:53:36.906 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.906 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:53:36.906 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.906 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:53:36.906 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:33.006865+0000", 2024-02-23T00:53:36.906 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:53:36.906 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.906 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.906 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:53:36.906 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:33.006865+0000", 2024-02-23T00:53:36.906 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999995 2024-02-23T00:53:36.906 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.906 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.907 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:53:36.907 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:33.006864+0000", 2024-02-23T00:53:36.907 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.3809999999999999e-06 2024-02-23T00:53:36.907 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.907 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.907 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:53:36.907 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:33.006867+0000", 2024-02-23T00:53:36.907 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7329999999999997e-06 2024-02-23T00:53:36.907 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.907 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.907 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:53:36.907 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:33.006874+0000", 2024-02-23T00:53:36.908 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4079999999999997e-06 2024-02-23T00:53:36.908 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.908 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.908 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:53:36.908 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:33.006880+0000", 2024-02-23T00:53:36.908 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00050598799999999997 2024-02-23T00:53:36.908 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.908 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.908 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.908 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:33.007386+0000", 2024-02-23T00:53:36.908 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.9499000000000007e-05 2024-02-23T00:53:36.908 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.908 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.909 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:53:36.909 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:33.007455+0000", 2024-02-23T00:53:36.909 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.8040000000000004e-05 2024-02-23T00:53:36.909 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.909 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.909 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:53:36.909 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:33.007543+0000", 2024-02-23T00:53:36.909 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00067854799999999996 2024-02-23T00:53:36.909 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.909 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:53:36.909 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.909 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.909 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.909 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.41771.0:1037 242.8 242:10000000::::head [pgnls start_epoch 677] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e677)", 2024-02-23T00:53:36.909 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:53:34.316007+0000", 2024-02-23T00:53:36.909 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 2.4818895240000001, 2024-02-23T00:53:36.910 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0021270379999999999, 2024-02-23T00:53:36.910 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:53:36.910 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:53:36.910 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:53:36.910 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.41771", 2024-02-23T00:53:36.910 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1032768195", 2024-02-23T00:53:36.910 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1037 2024-02-23T00:53:36.910 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.910 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:53:36.910 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.910 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:53:36.910 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:34.316007+0000", 2024-02-23T00:53:36.910 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:53:36.910 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.910 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.910 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:53:36.911 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:34.316007+0000", 2024-02-23T00:53:36.911 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:53:36.911 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.911 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.911 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:53:36.911 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:34.316005+0000", 2024-02-23T00:53:36.911 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.2210000000000003e-06 2024-02-23T00:53:36.911 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.911 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.911 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:53:36.911 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:34.316015+0000", 2024-02-23T00:53:36.911 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2371000000000001e-05 2024-02-23T00:53:36.911 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.911 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.911 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:53:36.911 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:34.316027+0000", 2024-02-23T00:53:36.912 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.3889999999999999e-06 2024-02-23T00:53:36.912 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.912 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.912 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:53:36.912 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:34.316034+0000", 2024-02-23T00:53:36.912 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0019545679999999998 2024-02-23T00:53:36.912 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.912 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.912 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.912 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:34.317989+0000", 2024-02-23T00:53:36.912 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9019e-05 2024-02-23T00:53:36.912 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.912 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.912 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:53:36.912 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:34.318028+0000", 2024-02-23T00:53:36.913 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010612599999999999 2024-02-23T00:53:36.913 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.913 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.913 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:53:36.913 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:34.318134+0000", 2024-02-23T00:53:36.913 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0021270379999999999 2024-02-23T00:53:36.913 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.913 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:53:36.913 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.913 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.913 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.913 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "MOSDScrubReserve(234.2s1 RELEASE e677)", 2024-02-23T00:53:36.913 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:53:34.474294+0000", 2024-02-23T00:53:36.913 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 2.323602771, 2024-02-23T00:53:36.913 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00027506399999999999, 2024-02-23T00:53:36.913 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:53:36.914 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:53:36.914 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:53:36.914 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.914 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:53:36.914 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:34.474294+0000", 2024-02-23T00:53:36.914 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:53:36.914 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.914 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.914 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:53:36.914 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:34.474294+0000", 2024-02-23T00:53:36.914 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:53:36.914 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.914 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.914 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:53:36.915 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:34.474293+0000", 2024-02-23T00:53:36.915 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4879999999999998e-06 2024-02-23T00:53:36.915 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.915 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.915 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:53:36.915 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:34.474298+0000", 2024-02-23T00:53:36.915 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5614e-05 2024-02-23T00:53:36.915 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.915 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.915 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:53:36.915 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:34.474314+0000", 2024-02-23T00:53:36.915 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.7760000000000003e-06 2024-02-23T00:53:36.915 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.916 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.916 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:53:36.916 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:34.474320+0000", 2024-02-23T00:53:36.916 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00020935100000000001 2024-02-23T00:53:36.916 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.916 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.916 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:53:36.916 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:34.474529+0000", 2024-02-23T00:53:36.916 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5549e-05 2024-02-23T00:53:36.916 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.916 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.916 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:53:36.916 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:34.474545+0000", 2024-02-23T00:53:36.917 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4039000000000001e-05 2024-02-23T00:53:36.917 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:53:36.917 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:53:36.917 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:53:36.917 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:34.474569+0000", 2024-02-23T00:53:36.917 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00027506399999999999 2024-02-23T00:53:36.917 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.917 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:53:36.917 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.917 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:53:36.917 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:53:36.918 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:53:36.918 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:34.472509+0000 osd.6 (osd.6) 259 : cluster [DBG] 234.2 scrub starts 2024-02-23T00:53:36.918 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:34.474117+0000 osd.6 (osd.6) 260 : cluster [DBG] 234.2 scrub ok 2024-02-23T00:53:36.919 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:34.684133+0000 osd.1 (osd.1) 278 : cluster [DBG] 18.11 scrub starts 2024-02-23T00:53:36.919 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:34.687342+0000 osd.1 (osd.1) 279 : cluster [DBG] 18.11 scrub ok 2024-02-23T00:53:36.919 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:35.374370+0000 mgr.y (mgr.14517) 590 : cluster [DBG] pgmap v1025: 152 pgs: 152 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:53:36.919 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:35.472016+0000 osd.4 (osd.4) 287 : cluster [DBG] 5.18 scrub starts 2024-02-23T00:53:36.919 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:35.476614+0000 osd.4 (osd.4) 288 : cluster [DBG] 5.18 scrub ok 2024-02-23T00:53:36.919 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:36.292620+0000 mon.a (mon.0) 3360 : cluster [DBG] osdmap e679: 8 total, 8 up, 6 in 2024-02-23T00:53:36.919 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:36.295240+0000 mon.c (mon.2) 393 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:36.919 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:36.295658+0000 mon.a (mon.0) 3361 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:36.920 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:34.472509+0000 osd.6 (osd.6) 259 : cluster [DBG] 234.2 scrub starts 2024-02-23T00:53:36.920 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:34.474117+0000 osd.6 (osd.6) 260 : cluster [DBG] 234.2 scrub ok 2024-02-23T00:53:36.920 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:34.684133+0000 osd.1 (osd.1) 278 : cluster [DBG] 18.11 scrub starts 2024-02-23T00:53:36.920 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:34.687342+0000 osd.1 (osd.1) 279 : cluster [DBG] 18.11 scrub ok 2024-02-23T00:53:36.920 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:35.374370+0000 mgr.y (mgr.14517) 590 : cluster [DBG] pgmap v1025: 152 pgs: 152 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:53:36.920 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:35.472016+0000 osd.4 (osd.4) 287 : cluster [DBG] 5.18 scrub starts 2024-02-23T00:53:36.920 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:35.476614+0000 osd.4 (osd.4) 288 : cluster [DBG] 5.18 scrub ok 2024-02-23T00:53:36.920 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:36.292620+0000 mon.a (mon.0) 3360 : cluster [DBG] osdmap e679: 8 total, 8 up, 6 in 2024-02-23T00:53:36.920 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:36.295240+0000 mon.c (mon.2) 393 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:36.920 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:36.295658+0000 mon.a (mon.0) 3361 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:36.920 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:34.472509+0000 osd.6 (osd.6) 259 : cluster [DBG] 234.2 scrub starts 2024-02-23T00:53:36.920 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:34.474117+0000 osd.6 (osd.6) 260 : cluster [DBG] 234.2 scrub ok 2024-02-23T00:53:36.921 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:34.684133+0000 osd.1 (osd.1) 278 : cluster [DBG] 18.11 scrub starts 2024-02-23T00:53:36.921 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:34.687342+0000 osd.1 (osd.1) 279 : cluster [DBG] 18.11 scrub ok 2024-02-23T00:53:36.921 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:35.374370+0000 mgr.y (mgr.14517) 590 : cluster [DBG] pgmap v1025: 152 pgs: 152 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:53:36.921 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:35.472016+0000 osd.4 (osd.4) 287 : cluster [DBG] 5.18 scrub starts 2024-02-23T00:53:36.921 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:35.476614+0000 osd.4 (osd.4) 288 : cluster [DBG] 5.18 scrub ok 2024-02-23T00:53:36.921 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:36.292620+0000 mon.a (mon.0) 3360 : cluster [DBG] osdmap e679: 8 total, 8 up, 6 in 2024-02-23T00:53:36.921 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:36.295240+0000 mon.c (mon.2) 393 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:36.921 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:36.295658+0000 mon.a (mon.0) 3361 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.863+0000 7f2c41450700 1 -- 172.21.15.47:0/3758536662 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2c3c0fdbd0 msgr2=0x7f2c3c0fdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.863+0000 7f2c41450700 1 --2- 172.21.15.47:0/3758536662 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2c3c0fdbd0 0x7f2c3c0fdfb0 secure :-1 s=READY pgs=2772 cs=0 l=1 rev1=1 rx=0x7f2c2c004870 tx=0x7f2c2c009cb0).stop 2024-02-23T00:53:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.863+0000 7f2c41450700 1 -- 172.21.15.47:0/3758536662 shutdown_connections 2024-02-23T00:53:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.863+0000 7f2c41450700 1 --2- 172.21.15.47:0/3758536662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c3c0f6a90 0x7f2c3c104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.863+0000 7f2c41450700 1 --2- 172.21.15.47:0/3758536662 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2c3c0fdbd0 0x7f2c3c0fdfb0 unknown :-1 s=CLOSED pgs=2772 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.863+0000 7f2c41450700 1 --2- 172.21.15.47:0/3758536662 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c3c0fe530 0x7f2c3c0f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.863+0000 7f2c41450700 1 -- 172.21.15.47:0/3758536662 >> 172.21.15.47:0/3758536662 conn(0x7f2c3c06c370 msgr2=0x7f2c3c06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.863+0000 7f2c41450700 1 -- 172.21.15.47:0/3758536662 shutdown_connections 2024-02-23T00:53:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.863+0000 7f2c41450700 1 -- 172.21.15.47:0/3758536662 wait complete. 2024-02-23T00:53:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.864+0000 7f2c41450700 1 librados: starting msgr at 2024-02-23T00:53:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.864+0000 7f2c41450700 1 librados: starting objecter 2024-02-23T00:53:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.864+0000 7f2c41450700 1 Processor -- start 2024-02-23T00:53:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.864+0000 7f2c41450700 1 -- start start 2024-02-23T00:53:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.864+0000 7f2c41450700 1 librados: setting wanted keys 2024-02-23T00:53:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.864+0000 7f2c41450700 1 librados: calling monclient init 2024-02-23T00:53:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.864+0000 7f2c41450700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c3c0f6a90 0x7f2c3c0f7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.864+0000 7f2c41450700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c3c0fdbd0 0x7f2c3c0f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.864+0000 7f2c41450700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2c3c0fe530 0x7f2c3c0f8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.864+0000 7f2c41450700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c116460 con 0x7f2c3c0fdbd0 2024-02-23T00:53:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.864+0000 7f2c41450700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c1165d0 con 0x7f2c3c0f6a90 2024-02-23T00:53:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.864+0000 7f2c41450700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c116790 con 0x7f2c3c0fe530 2024-02-23T00:53:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.865+0000 7f2c3a7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c3c0fdbd0 0x7f2c3c0f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.865+0000 7f2c3affd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c3c0f6a90 0x7f2c3c0f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.865+0000 7f2c3b7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2c3c0fe530 0x7f2c3c0f8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.865+0000 7f2c3affd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c3c0f6a90 0x7f2c3c0f7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.47:57134/0 (socket says 172.21.15.47:57134) 2024-02-23T00:53:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.865+0000 7f2c3b7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2c3c0fe530 0x7f2c3c0f8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:50332/0 (socket says 172.21.15.47:50332) 2024-02-23T00:53:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.865+0000 7f2c3affd700 1 -- 172.21.15.47:0/3017409601 learned_addr learned my addr 172.21.15.47:0/3017409601 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:53:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.865+0000 7f2c3a7fc700 1 -- 172.21.15.47:0/3017409601 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2c3c0fe530 msgr2=0x7f2c3c0f8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:36.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.865+0000 7f2c3a7fc700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2c3c0fe530 0x7f2c3c0f8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:36.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.865+0000 7f2c3a7fc700 1 -- 172.21.15.47:0/3017409601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c3c0f6a90 msgr2=0x7f2c3c0f7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:36.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.865+0000 7f2c3a7fc700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c3c0f6a90 0x7f2c3c0f7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:36.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.865+0000 7f2c3a7fc700 1 -- 172.21.15.47:0/3017409601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c24005040 con 0x7f2c3c0fdbd0 2024-02-23T00:53:36.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.865+0000 7f2c3b7fe700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2c3c0fe530 0x7f2c3c0f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:53:36.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.865+0000 7f2c3a7fc700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c3c0fdbd0 0x7f2c3c0f7b70 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 rx=0x7f2c24004920 tx=0x7f2c24009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:36.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.865+0000 7f2c33fff700 10 client.?.objecter ms_handle_connect 0x7f2c3c0fdbd0 2024-02-23T00:53:36.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.865+0000 7f2c33fff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:53:36.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.865+0000 7f2c33fff700 1 -- 172.21.15.47:0/3017409601 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2400e030 con 0x7f2c3c0fdbd0 2024-02-23T00:53:36.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.865+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c2c005040 con 0x7f2c3c0fdbd0 2024-02-23T00:53:36.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.865+0000 7f2c33fff700 1 -- 172.21.15.47:0/3017409601 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f2c2400f040 con 0x7f2c3c0fdbd0 2024-02-23T00:53:36.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.865+0000 7f2c33fff700 1 -- 172.21.15.47:0/3017409601 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c24014720 con 0x7f2c3c0fdbd0 2024-02-23T00:53:36.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.865+0000 7f2c41450700 10 client.31596.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:53:36.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.865+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c3c116dd0 con 0x7f2c3c0fdbd0 2024-02-23T00:53:36.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.865+0000 7f2c41450700 1 librados: init done 2024-02-23T00:53:36.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.866+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f2c3c05f950 con 0x7f2c3c0fdbd0 2024-02-23T00:53:36.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.867+0000 7f2c33fff700 1 -- 172.21.15.47:0/3017409601 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f2c24010500 con 0x7f2c3c0fdbd0 2024-02-23T00:53:36.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.867+0000 7f2c33fff700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2c28062cc0 0x7f2c28065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:36.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.867+0000 7f2c33fff700 1 -- 172.21.15.47:0/3017409601 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(679..679 src has 482..679) v4 ==== 9625+0+0 (secure 0 0 0) 0x7f2c2407ded0 con 0x7f2c3c0fdbd0 2024-02-23T00:53:36.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.867+0000 7f2c33fff700 10 client.31596.objecter ms_dispatch 0x7f2c3c001550 osd_map(679..679 src has 482..679) v4 2024-02-23T00:53:36.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.867+0000 7f2c33fff700 3 client.31596.objecter handle_osd_map got epochs [679,679] > 0 2024-02-23T00:53:36.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.867+0000 7f2c33fff700 3 client.31596.objecter handle_osd_map decoding full epoch 679 2024-02-23T00:53:36.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.867+0000 7f2c3affd700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2c28062cc0 0x7f2c28065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:36.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.867+0000 7f2c33fff700 20 client.31596.objecter dump_active .. 0 homeless 2024-02-23T00:53:36.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.867+0000 7f2c33fff700 1 -- 172.21.15.47:0/3017409601 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v679) v1 ==== 54+0+16 (secure 0 0 0) 0x7f2c2407e370 con 0x7f2c3c0fdbd0 2024-02-23T00:53:36.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.868+0000 7f2c33fff700 10 client.31596.objecter ms_handle_connect 0x7f2c28062cc0 2024-02-23T00:53:36.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.868+0000 7f2c3affd700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2c28062cc0 0x7f2c28065180 secure :-1 s=READY pgs=3162 cs=0 l=1 rev1=1 rx=0x7f2c2c005010 tx=0x7f2c2c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:36.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.868+0000 7f2c41450700 10 client.31596.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:36.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.868+0000 7f2c41450700 15 client.31596.objecter _session_command_op_assign -1 1 2024-02-23T00:53:36.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.868+0000 7f2c41450700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f2c3c0598f0 0x7f2c3c059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.868+0000 7f2c41450700 20 client.31596.objecter _get_session s=0x7f2c3c05ee80 osd=0 3 2024-02-23T00:53:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.868+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c05ee80 osd=0 3 2024-02-23T00:53:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.868+0000 7f2c41450700 20 client.31596.objecter _get_session s=0x7f2c3c05ee80 osd=0 3 2024-02-23T00:53:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.868+0000 7f2c41450700 15 client.31596.objecter _session_command_op_remove -1 1 2024-02-23T00:53:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.868+0000 7f2c41450700 20 client.31596.objecter get_session s=0x7f2c3c05ee80 osd=0 3 2024-02-23T00:53:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.868+0000 7f2c41450700 15 client.31596.objecter _session_command_op_assign 0 1 2024-02-23T00:53:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.868+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c05ee80 osd=0 4 2024-02-23T00:53:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.868+0000 7f2c41450700 10 client.31596.objecter _send_command 1 2024-02-23T00:53:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.868+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c3c106e90 con 0x7f2c3c0598f0 2024-02-23T00:53:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.868+0000 7f2c3b7fe700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f2c3c0598f0 0x7f2c3c059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.868+0000 7f2c33fff700 10 client.31596.objecter ms_handle_connect 0x7f2c3c0598f0 2024-02-23T00:53:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.868+0000 7f2c3b7fe700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f2c3c0598f0 0x7f2c3c059cd0 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.869+0000 7f2c33fff700 1 -- 172.21.15.47:0/3017409601 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2c3c106e90 con 0x7f2c3c0598f0 2024-02-23T00:53:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.869+0000 7f2c33fff700 10 client.31596.objecter ms_dispatch 0x7f2c3c001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:53:36.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.869+0000 7f2c33fff700 10 client.31596.objecter _finish_command 1 = system:0 2024-02-23T00:53:36.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.869+0000 7f2c33fff700 20 client.31596.objecter put_session s=0x7f2c3c05ee80 osd=0 4 2024-02-23T00:53:36.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.869+0000 7f2c33fff700 15 client.31596.objecter _session_command_op_remove 0 1 2024-02-23T00:53:36.929 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:53:36.930 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:53:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.884+0000 7f2c41450700 10 client.31596.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:53:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.884+0000 7f2c41450700 15 client.31596.objecter _session_command_op_assign -1 2 2024-02-23T00:53:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.884+0000 7f2c41450700 20 client.31596.objecter _get_session s=0x7f2c3c05ee80 osd=0 3 2024-02-23T00:53:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.884+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c05ee80 osd=0 3 2024-02-23T00:53:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.884+0000 7f2c41450700 20 client.31596.objecter _get_session s=0x7f2c3c05ee80 osd=0 3 2024-02-23T00:53:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.884+0000 7f2c41450700 15 client.31596.objecter _session_command_op_remove -1 2 2024-02-23T00:53:36.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.884+0000 7f2c41450700 20 client.31596.objecter get_session s=0x7f2c3c05ee80 osd=0 3 2024-02-23T00:53:36.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.884+0000 7f2c41450700 15 client.31596.objecter _session_command_op_assign 0 2 2024-02-23T00:53:36.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.884+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c05ee80 osd=0 4 2024-02-23T00:53:36.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.884+0000 7f2c41450700 10 client.31596.objecter _send_command 2 2024-02-23T00:53:36.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.884+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f2c3c0273c0 con 0x7f2c3c0598f0 2024-02-23T00:53:36.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.885+0000 7f2c33fff700 1 -- 172.21.15.47:0/3017409601 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f2c3c0273c0 con 0x7f2c3c0598f0 2024-02-23T00:53:36.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.885+0000 7f2c33fff700 10 client.31596.objecter ms_dispatch 0x7f2c3c001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:53:36.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.885+0000 7f2c33fff700 10 client.31596.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:53:36.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.885+0000 7f2c33fff700 20 client.31596.objecter put_session s=0x7f2c3c05ee80 osd=0 4 2024-02-23T00:53:36.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.885+0000 7f2c33fff700 15 client.31596.objecter _session_command_op_remove 0 2 2024-02-23T00:53:36.931 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:53:36.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.886+0000 7f2c41450700 10 client.31596.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:36.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.886+0000 7f2c41450700 15 client.31596.objecter _session_command_op_assign -1 3 2024-02-23T00:53:36.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.886+0000 7f2c41450700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f2c3c0ff180 0x7f2c3c0ff560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:36.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.886+0000 7f2c41450700 20 client.31596.objecter _get_session s=0x7f2c3c028710 osd=1 3 2024-02-23T00:53:36.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.886+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c028710 osd=1 3 2024-02-23T00:53:36.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.886+0000 7f2c41450700 20 client.31596.objecter _get_session s=0x7f2c3c028710 osd=1 3 2024-02-23T00:53:36.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.886+0000 7f2c41450700 15 client.31596.objecter _session_command_op_remove -1 3 2024-02-23T00:53:36.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.886+0000 7f2c41450700 20 client.31596.objecter get_session s=0x7f2c3c028710 osd=1 3 2024-02-23T00:53:36.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.886+0000 7f2c41450700 15 client.31596.objecter _session_command_op_assign 1 3 2024-02-23T00:53:36.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.886+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c028710 osd=1 4 2024-02-23T00:53:36.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.886+0000 7f2c41450700 10 client.31596.objecter _send_command 3 2024-02-23T00:53:36.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.886+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c3c11d770 con 0x7f2c3c0ff180 2024-02-23T00:53:36.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.886+0000 7f2c3affd700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f2c3c0ff180 0x7f2c3c0ff560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:36.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.886+0000 7f2c33fff700 10 client.31596.objecter ms_handle_connect 0x7f2c3c0ff180 2024-02-23T00:53:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.886+0000 7f2c3affd700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f2c3c0ff180 0x7f2c3c0ff560 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.887+0000 7f2c33fff700 1 -- 172.21.15.47:0/3017409601 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2c3c11d770 con 0x7f2c3c0ff180 2024-02-23T00:53:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.887+0000 7f2c33fff700 10 client.31596.objecter ms_dispatch 0x7f2c3c001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:53:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.887+0000 7f2c33fff700 10 client.31596.objecter _finish_command 3 = system:0 2024-02-23T00:53:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.887+0000 7f2c33fff700 20 client.31596.objecter put_session s=0x7f2c3c028710 osd=1 4 2024-02-23T00:53:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.887+0000 7f2c33fff700 15 client.31596.objecter _session_command_op_remove 1 3 2024-02-23T00:53:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.902+0000 7f2c41450700 10 client.31596.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:53:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.902+0000 7f2c41450700 15 client.31596.objecter _session_command_op_assign -1 4 2024-02-23T00:53:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.902+0000 7f2c41450700 20 client.31596.objecter _get_session s=0x7f2c3c028710 osd=1 3 2024-02-23T00:53:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.902+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c028710 osd=1 3 2024-02-23T00:53:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.902+0000 7f2c41450700 20 client.31596.objecter _get_session s=0x7f2c3c028710 osd=1 3 2024-02-23T00:53:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.902+0000 7f2c41450700 15 client.31596.objecter _session_command_op_remove -1 4 2024-02-23T00:53:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.902+0000 7f2c41450700 20 client.31596.objecter get_session s=0x7f2c3c028710 osd=1 3 2024-02-23T00:53:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.902+0000 7f2c41450700 15 client.31596.objecter _session_command_op_assign 1 4 2024-02-23T00:53:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.902+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c028710 osd=1 4 2024-02-23T00:53:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.902+0000 7f2c41450700 10 client.31596.objecter _send_command 4 2024-02-23T00:53:36.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.902+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f2c3c0febf0 con 0x7f2c3c0ff180 2024-02-23T00:53:36.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.903+0000 7f2c33fff700 1 -- 172.21.15.47:0/3017409601 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f2c3c0febf0 con 0x7f2c3c0ff180 2024-02-23T00:53:36.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.903+0000 7f2c33fff700 10 client.31596.objecter ms_dispatch 0x7f2c3c001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:53:36.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.903+0000 7f2c33fff700 10 client.31596.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:53:36.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.903+0000 7f2c33fff700 20 client.31596.objecter put_session s=0x7f2c3c028710 osd=1 4 2024-02-23T00:53:36.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.903+0000 7f2c33fff700 15 client.31596.objecter _session_command_op_remove 1 4 2024-02-23T00:53:36.935 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:53:36.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.904+0000 7f2c41450700 10 client.31596.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:36.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.904+0000 7f2c41450700 15 client.31596.objecter _session_command_op_assign -1 5 2024-02-23T00:53:36.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.904+0000 7f2c41450700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f2c3c11c190 0x7f2c3c11c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:36.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.904+0000 7f2c41450700 20 client.31596.objecter _get_session s=0x7f2c3c027d60 osd=2 3 2024-02-23T00:53:36.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.904+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c027d60 osd=2 3 2024-02-23T00:53:36.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.904+0000 7f2c41450700 20 client.31596.objecter _get_session s=0x7f2c3c027d60 osd=2 3 2024-02-23T00:53:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.904+0000 7f2c41450700 15 client.31596.objecter _session_command_op_remove -1 5 2024-02-23T00:53:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.904+0000 7f2c41450700 20 client.31596.objecter get_session s=0x7f2c3c027d60 osd=2 3 2024-02-23T00:53:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.904+0000 7f2c41450700 15 client.31596.objecter _session_command_op_assign 2 5 2024-02-23T00:53:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.904+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c027d60 osd=2 4 2024-02-23T00:53:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.904+0000 7f2c41450700 10 client.31596.objecter _send_command 5 2024-02-23T00:53:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.904+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c3c0283e0 con 0x7f2c3c11c190 2024-02-23T00:53:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.904+0000 7f2c3a7fc700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f2c3c11c190 0x7f2c3c11c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.904+0000 7f2c33fff700 10 client.31596.objecter ms_handle_connect 0x7f2c3c11c190 2024-02-23T00:53:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.904+0000 7f2c3a7fc700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f2c3c11c190 0x7f2c3c11c5d0 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.904+0000 7f2c33fff700 1 -- 172.21.15.47:0/3017409601 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2c3c0283e0 con 0x7f2c3c11c190 2024-02-23T00:53:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.904+0000 7f2c33fff700 10 client.31596.objecter ms_dispatch 0x7f2c3c001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:53:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.904+0000 7f2c33fff700 10 client.31596.objecter _finish_command 5 = system:0 2024-02-23T00:53:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.904+0000 7f2c33fff700 20 client.31596.objecter put_session s=0x7f2c3c027d60 osd=2 4 2024-02-23T00:53:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.904+0000 7f2c33fff700 15 client.31596.objecter _session_command_op_remove 2 5 2024-02-23T00:53:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.919+0000 7f2c41450700 10 client.31596.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:53:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.919+0000 7f2c41450700 15 client.31596.objecter _session_command_op_assign -1 6 2024-02-23T00:53:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.919+0000 7f2c41450700 20 client.31596.objecter _get_session s=0x7f2c3c027d60 osd=2 3 2024-02-23T00:53:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.919+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c027d60 osd=2 3 2024-02-23T00:53:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.919+0000 7f2c41450700 20 client.31596.objecter _get_session s=0x7f2c3c027d60 osd=2 3 2024-02-23T00:53:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.919+0000 7f2c41450700 15 client.31596.objecter _session_command_op_remove -1 6 2024-02-23T00:53:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.919+0000 7f2c41450700 20 client.31596.objecter get_session s=0x7f2c3c027d60 osd=2 3 2024-02-23T00:53:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.919+0000 7f2c41450700 15 client.31596.objecter _session_command_op_assign 2 6 2024-02-23T00:53:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.919+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c027d60 osd=2 4 2024-02-23T00:53:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.919+0000 7f2c41450700 10 client.31596.objecter _send_command 6 2024-02-23T00:53:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.919+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f2c3c11d290 con 0x7f2c3c11c190 2024-02-23T00:53:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.920+0000 7f2c33fff700 1 -- 172.21.15.47:0/3017409601 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f2c3c11d290 con 0x7f2c3c11c190 2024-02-23T00:53:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.920+0000 7f2c33fff700 10 client.31596.objecter ms_dispatch 0x7f2c3c001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:53:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.920+0000 7f2c33fff700 10 client.31596.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:53:36.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.920+0000 7f2c33fff700 20 client.31596.objecter put_session s=0x7f2c3c027d60 osd=2 4 2024-02-23T00:53:36.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.920+0000 7f2c33fff700 15 client.31596.objecter _session_command_op_remove 2 6 2024-02-23T00:53:36.939 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:53:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.920+0000 7f2c41450700 10 client.31596.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.920+0000 7f2c41450700 15 client.31596.objecter _session_command_op_assign -1 7 2024-02-23T00:53:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.921+0000 7f2c41450700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f2c3c027940 0x7f2c3c121e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.921+0000 7f2c41450700 20 client.31596.objecter _get_session s=0x7f2c3c0277d0 osd=3 3 2024-02-23T00:53:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.921+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c0277d0 osd=3 3 2024-02-23T00:53:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.921+0000 7f2c41450700 20 client.31596.objecter _get_session s=0x7f2c3c0277d0 osd=3 3 2024-02-23T00:53:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.921+0000 7f2c41450700 15 client.31596.objecter _session_command_op_remove -1 7 2024-02-23T00:53:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.921+0000 7f2c41450700 20 client.31596.objecter get_session s=0x7f2c3c0277d0 osd=3 3 2024-02-23T00:53:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.921+0000 7f2c41450700 15 client.31596.objecter _session_command_op_assign 3 7 2024-02-23T00:53:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.921+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c0277d0 osd=3 4 2024-02-23T00:53:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.921+0000 7f2c41450700 10 client.31596.objecter _send_command 7 2024-02-23T00:53:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.921+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c3c124a60 con 0x7f2c3c027940 2024-02-23T00:53:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.921+0000 7f2c3b7fe700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f2c3c027940 0x7f2c3c121e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.921+0000 7f2c33fff700 10 client.31596.objecter ms_handle_connect 0x7f2c3c027940 2024-02-23T00:53:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.921+0000 7f2c3b7fe700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f2c3c027940 0x7f2c3c121e10 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.921+0000 7f2c33fff700 1 -- 172.21.15.47:0/3017409601 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2c3c124a60 con 0x7f2c3c027940 2024-02-23T00:53:36.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.921+0000 7f2c33fff700 10 client.31596.objecter ms_dispatch 0x7f2c3c001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:53:36.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.921+0000 7f2c33fff700 10 client.31596.objecter _finish_command 7 = system:0 2024-02-23T00:53:36.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.921+0000 7f2c33fff700 20 client.31596.objecter put_session s=0x7f2c3c0277d0 osd=3 4 2024-02-23T00:53:36.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.921+0000 7f2c33fff700 15 client.31596.objecter _session_command_op_remove 3 7 2024-02-23T00:53:36.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.936+0000 7f2c41450700 10 client.31596.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:53:36.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.936+0000 7f2c41450700 15 client.31596.objecter _session_command_op_assign -1 8 2024-02-23T00:53:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.936+0000 7f2c41450700 20 client.31596.objecter _get_session s=0x7f2c3c0277d0 osd=3 3 2024-02-23T00:53:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.936+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c0277d0 osd=3 3 2024-02-23T00:53:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.936+0000 7f2c41450700 20 client.31596.objecter _get_session s=0x7f2c3c0277d0 osd=3 3 2024-02-23T00:53:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.936+0000 7f2c41450700 15 client.31596.objecter _session_command_op_remove -1 8 2024-02-23T00:53:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.936+0000 7f2c41450700 20 client.31596.objecter get_session s=0x7f2c3c0277d0 osd=3 3 2024-02-23T00:53:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.936+0000 7f2c41450700 15 client.31596.objecter _session_command_op_assign 3 8 2024-02-23T00:53:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.936+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c0277d0 osd=3 4 2024-02-23T00:53:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.936+0000 7f2c41450700 10 client.31596.objecter _send_command 8 2024-02-23T00:53:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.936+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f2c3c11cd00 con 0x7f2c3c027940 2024-02-23T00:53:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.937+0000 7f2c33fff700 1 -- 172.21.15.47:0/3017409601 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f2c3c11cd00 con 0x7f2c3c027940 2024-02-23T00:53:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.937+0000 7f2c33fff700 10 client.31596.objecter ms_dispatch 0x7f2c3c001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:53:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.937+0000 7f2c33fff700 10 client.31596.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:53:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.937+0000 7f2c33fff700 20 client.31596.objecter put_session s=0x7f2c3c0277d0 osd=3 4 2024-02-23T00:53:36.944 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:53:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.937+0000 7f2c33fff700 15 client.31596.objecter _session_command_op_remove 3 8 2024-02-23T00:53:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.938+0000 7f2c41450700 10 client.31596.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.938+0000 7f2c41450700 15 client.31596.objecter _session_command_op_assign -1 9 2024-02-23T00:53:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.938+0000 7f2c41450700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2c3c11bcf0 0x7f2c3c123480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.938+0000 7f2c41450700 20 client.31596.objecter _get_session s=0x7f2c3c11cac0 osd=4 3 2024-02-23T00:53:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.938+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c11cac0 osd=4 3 2024-02-23T00:53:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.938+0000 7f2c41450700 20 client.31596.objecter _get_session s=0x7f2c3c11cac0 osd=4 3 2024-02-23T00:53:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.938+0000 7f2c41450700 15 client.31596.objecter _session_command_op_remove -1 9 2024-02-23T00:53:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.938+0000 7f2c41450700 20 client.31596.objecter get_session s=0x7f2c3c11cac0 osd=4 3 2024-02-23T00:53:36.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.938+0000 7f2c41450700 15 client.31596.objecter _session_command_op_assign 4 9 2024-02-23T00:53:36.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.938+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c11cac0 osd=4 4 2024-02-23T00:53:36.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.938+0000 7f2c41450700 10 client.31596.objecter _send_command 9 2024-02-23T00:53:36.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.938+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c3c11cc30 con 0x7f2c3c11bcf0 2024-02-23T00:53:36.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.938+0000 7f2c3affd700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2c3c11bcf0 0x7f2c3c123480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:36.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.938+0000 7f2c33fff700 10 client.31596.objecter ms_handle_connect 0x7f2c3c11bcf0 2024-02-23T00:53:36.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.938+0000 7f2c3affd700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2c3c11bcf0 0x7f2c3c123480 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:36.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.939+0000 7f2c33fff700 1 -- 172.21.15.47:0/3017409601 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2c3c11cc30 con 0x7f2c3c11bcf0 2024-02-23T00:53:36.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.939+0000 7f2c33fff700 10 client.31596.objecter ms_dispatch 0x7f2c3c001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:53:36.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.939+0000 7f2c33fff700 10 client.31596.objecter _finish_command 9 = system:0 2024-02-23T00:53:36.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.939+0000 7f2c33fff700 20 client.31596.objecter put_session s=0x7f2c3c11cac0 osd=4 4 2024-02-23T00:53:36.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.939+0000 7f2c33fff700 15 client.31596.objecter _session_command_op_remove 4 9 2024-02-23T00:53:36.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.954+0000 7f2c41450700 10 client.31596.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:53:36.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.954+0000 7f2c41450700 15 client.31596.objecter _session_command_op_assign -1 10 2024-02-23T00:53:36.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.954+0000 7f2c41450700 20 client.31596.objecter _get_session s=0x7f2c3c11cac0 osd=4 3 2024-02-23T00:53:36.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.954+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c11cac0 osd=4 3 2024-02-23T00:53:36.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.954+0000 7f2c41450700 20 client.31596.objecter _get_session s=0x7f2c3c11cac0 osd=4 3 2024-02-23T00:53:36.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.954+0000 7f2c41450700 15 client.31596.objecter _session_command_op_remove -1 10 2024-02-23T00:53:36.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.954+0000 7f2c41450700 20 client.31596.objecter get_session s=0x7f2c3c11cac0 osd=4 3 2024-02-23T00:53:36.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.954+0000 7f2c41450700 15 client.31596.objecter _session_command_op_assign 4 10 2024-02-23T00:53:36.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.954+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c11cac0 osd=4 4 2024-02-23T00:53:36.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.954+0000 7f2c41450700 10 client.31596.objecter _send_command 10 2024-02-23T00:53:36.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.954+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f2c3c11cc30 con 0x7f2c3c11bcf0 2024-02-23T00:53:36.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.956+0000 7f2c33fff700 1 -- 172.21.15.47:0/3017409601 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f2c3c11cc30 con 0x7f2c3c11bcf0 2024-02-23T00:53:36.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.956+0000 7f2c33fff700 10 client.31596.objecter ms_dispatch 0x7f2c3c001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:53:36.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.956+0000 7f2c33fff700 10 client.31596.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:53:36.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.956+0000 7f2c33fff700 20 client.31596.objecter put_session s=0x7f2c3c11cac0 osd=4 4 2024-02-23T00:53:36.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.956+0000 7f2c33fff700 15 client.31596.objecter _session_command_op_remove 4 10 2024-02-23T00:53:36.958 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:53:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.956+0000 7f2c41450700 10 client.31596.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.956+0000 7f2c41450700 15 client.31596.objecter _session_command_op_assign -1 11 2024-02-23T00:53:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.956+0000 7f2c41450700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f2c3c123e80 0x7f2c3c1242c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.956+0000 7f2c41450700 20 client.31596.objecter _get_session s=0x7f2c3c11cc30 osd=5 3 2024-02-23T00:53:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.956+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c11cc30 osd=5 3 2024-02-23T00:53:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.956+0000 7f2c41450700 20 client.31596.objecter _get_session s=0x7f2c3c11cc30 osd=5 3 2024-02-23T00:53:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.956+0000 7f2c41450700 15 client.31596.objecter _session_command_op_remove -1 11 2024-02-23T00:53:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.956+0000 7f2c41450700 20 client.31596.objecter get_session s=0x7f2c3c11cc30 osd=5 3 2024-02-23T00:53:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.956+0000 7f2c41450700 15 client.31596.objecter _session_command_op_assign 5 11 2024-02-23T00:53:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.956+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c11cc30 osd=5 4 2024-02-23T00:53:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.956+0000 7f2c41450700 10 client.31596.objecter _send_command 11 2024-02-23T00:53:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.956+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c3c1248c0 con 0x7f2c3c123e80 2024-02-23T00:53:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.956+0000 7f2c3a7fc700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f2c3c123e80 0x7f2c3c1242c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.956+0000 7f2c33fff700 10 client.31596.objecter ms_handle_connect 0x7f2c3c123e80 2024-02-23T00:53:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.956+0000 7f2c3a7fc700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f2c3c123e80 0x7f2c3c1242c0 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.957+0000 7f2c33fff700 1 -- 172.21.15.47:0/3017409601 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2c3c1248c0 con 0x7f2c3c123e80 2024-02-23T00:53:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.957+0000 7f2c33fff700 10 client.31596.objecter ms_dispatch 0x7f2c3c001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:53:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.957+0000 7f2c33fff700 10 client.31596.objecter _finish_command 11 = system:0 2024-02-23T00:53:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.957+0000 7f2c33fff700 20 client.31596.objecter put_session s=0x7f2c3c11cc30 osd=5 4 2024-02-23T00:53:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.957+0000 7f2c33fff700 15 client.31596.objecter _session_command_op_remove 5 11 2024-02-23T00:53:36.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.971+0000 7f2c41450700 10 client.31596.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:53:36.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.971+0000 7f2c41450700 15 client.31596.objecter _session_command_op_assign -1 12 2024-02-23T00:53:36.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.971+0000 7f2c41450700 20 client.31596.objecter _get_session s=0x7f2c3c11cc30 osd=5 3 2024-02-23T00:53:36.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.971+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c11cc30 osd=5 3 2024-02-23T00:53:36.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.971+0000 7f2c41450700 20 client.31596.objecter _get_session s=0x7f2c3c11cc30 osd=5 3 2024-02-23T00:53:36.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.971+0000 7f2c41450700 15 client.31596.objecter _session_command_op_remove -1 12 2024-02-23T00:53:36.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.971+0000 7f2c41450700 20 client.31596.objecter get_session s=0x7f2c3c11cc30 osd=5 3 2024-02-23T00:53:36.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.971+0000 7f2c41450700 15 client.31596.objecter _session_command_op_assign 5 12 2024-02-23T00:53:36.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.971+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c11cc30 osd=5 4 2024-02-23T00:53:36.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.971+0000 7f2c41450700 10 client.31596.objecter _send_command 12 2024-02-23T00:53:36.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.971+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f2c3c1248c0 con 0x7f2c3c123e80 2024-02-23T00:53:36.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.973+0000 7f2c33fff700 1 -- 172.21.15.47:0/3017409601 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f2c3c1248c0 con 0x7f2c3c123e80 2024-02-23T00:53:36.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.973+0000 7f2c33fff700 10 client.31596.objecter ms_dispatch 0x7f2c3c001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:53:36.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.973+0000 7f2c33fff700 10 client.31596.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:53:36.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.973+0000 7f2c33fff700 20 client.31596.objecter put_session s=0x7f2c3c11cc30 osd=5 4 2024-02-23T00:53:36.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.973+0000 7f2c33fff700 15 client.31596.objecter _session_command_op_remove 5 12 2024-02-23T00:53:36.975 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:53:36.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.973+0000 7f2c41450700 10 client.31596.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:36.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.973+0000 7f2c41450700 15 client.31596.objecter _session_command_op_assign -1 13 2024-02-23T00:53:36.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.973+0000 7f2c41450700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f2c3c129190 0x7f2c3c1295f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:36.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.973+0000 7f2c41450700 20 client.31596.objecter _get_session s=0x7f2c3c1248c0 osd=6 3 2024-02-23T00:53:36.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.973+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c1248c0 osd=6 3 2024-02-23T00:53:36.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.973+0000 7f2c41450700 20 client.31596.objecter _get_session s=0x7f2c3c1248c0 osd=6 3 2024-02-23T00:53:36.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.973+0000 7f2c41450700 15 client.31596.objecter _session_command_op_remove -1 13 2024-02-23T00:53:36.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.973+0000 7f2c41450700 20 client.31596.objecter get_session s=0x7f2c3c1248c0 osd=6 3 2024-02-23T00:53:36.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.973+0000 7f2c41450700 15 client.31596.objecter _session_command_op_assign 6 13 2024-02-23T00:53:36.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.973+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c1248c0 osd=6 4 2024-02-23T00:53:36.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.973+0000 7f2c41450700 10 client.31596.objecter _send_command 13 2024-02-23T00:53:36.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.973+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c3c12e510 con 0x7f2c3c129190 2024-02-23T00:53:36.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.974+0000 7f2c3b7fe700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f2c3c129190 0x7f2c3c1295f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:36.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.974+0000 7f2c33fff700 10 client.31596.objecter ms_handle_connect 0x7f2c3c129190 2024-02-23T00:53:36.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.974+0000 7f2c3b7fe700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f2c3c129190 0x7f2c3c1295f0 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:36.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.975+0000 7f2c33fff700 1 -- 172.21.15.47:0/3017409601 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2c3c12e510 con 0x7f2c3c129190 2024-02-23T00:53:36.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.975+0000 7f2c33fff700 10 client.31596.objecter ms_dispatch 0x7f2c3c001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:53:36.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.975+0000 7f2c33fff700 10 client.31596.objecter _finish_command 13 = system:0 2024-02-23T00:53:36.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.975+0000 7f2c33fff700 20 client.31596.objecter put_session s=0x7f2c3c1248c0 osd=6 4 2024-02-23T00:53:36.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.975+0000 7f2c33fff700 15 client.31596.objecter _session_command_op_remove 6 13 2024-02-23T00:53:36.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.989+0000 7f2c41450700 10 client.31596.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:53:36.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.989+0000 7f2c41450700 15 client.31596.objecter _session_command_op_assign -1 14 2024-02-23T00:53:36.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.989+0000 7f2c41450700 20 client.31596.objecter _get_session s=0x7f2c3c1248c0 osd=6 3 2024-02-23T00:53:36.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.989+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c1248c0 osd=6 3 2024-02-23T00:53:36.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.989+0000 7f2c41450700 20 client.31596.objecter _get_session s=0x7f2c3c1248c0 osd=6 3 2024-02-23T00:53:36.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.989+0000 7f2c41450700 15 client.31596.objecter _session_command_op_remove -1 14 2024-02-23T00:53:36.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.989+0000 7f2c41450700 20 client.31596.objecter get_session s=0x7f2c3c1248c0 osd=6 3 2024-02-23T00:53:36.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.989+0000 7f2c41450700 15 client.31596.objecter _session_command_op_assign 6 14 2024-02-23T00:53:36.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.989+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c1248c0 osd=6 4 2024-02-23T00:53:36.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.989+0000 7f2c41450700 10 client.31596.objecter _send_command 14 2024-02-23T00:53:36.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.989+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f2c3c129bf0 con 0x7f2c3c129190 2024-02-23T00:53:36.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.991+0000 7f2c33fff700 1 -- 172.21.15.47:0/3017409601 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f2c3c129bf0 con 0x7f2c3c129190 2024-02-23T00:53:36.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.991+0000 7f2c33fff700 10 client.31596.objecter ms_dispatch 0x7f2c3c001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:53:36.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.991+0000 7f2c33fff700 10 client.31596.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:53:36.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.991+0000 7f2c33fff700 20 client.31596.objecter put_session s=0x7f2c3c1248c0 osd=6 4 2024-02-23T00:53:36.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.991+0000 7f2c33fff700 15 client.31596.objecter _session_command_op_remove 6 14 2024-02-23T00:53:36.993 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:53:36.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.991+0000 7f2c41450700 10 client.31596.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:36.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.991+0000 7f2c41450700 15 client.31596.objecter _session_command_op_assign -1 15 2024-02-23T00:53:36.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.991+0000 7f2c41450700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f2c3c12a270 0x7f2c3c12a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:36.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.991+0000 7f2c41450700 20 client.31596.objecter _get_session s=0x7f2c3c129bf0 osd=7 3 2024-02-23T00:53:36.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.991+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c129bf0 osd=7 3 2024-02-23T00:53:36.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.991+0000 7f2c41450700 20 client.31596.objecter _get_session s=0x7f2c3c129bf0 osd=7 3 2024-02-23T00:53:36.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.991+0000 7f2c41450700 15 client.31596.objecter _session_command_op_remove -1 15 2024-02-23T00:53:36.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.991+0000 7f2c41450700 20 client.31596.objecter get_session s=0x7f2c3c129bf0 osd=7 3 2024-02-23T00:53:36.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.991+0000 7f2c41450700 15 client.31596.objecter _session_command_op_assign 7 15 2024-02-23T00:53:36.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.991+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c129bf0 osd=7 4 2024-02-23T00:53:36.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.991+0000 7f2c41450700 10 client.31596.objecter _send_command 15 2024-02-23T00:53:36.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.991+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c3c12ad90 con 0x7f2c3c12a270 2024-02-23T00:53:36.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.991+0000 7f2c3affd700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f2c3c12a270 0x7f2c3c12a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:36.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.992+0000 7f2c33fff700 10 client.31596.objecter ms_handle_connect 0x7f2c3c12a270 2024-02-23T00:53:36.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.992+0000 7f2c3affd700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f2c3c12a270 0x7f2c3c12a6b0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:36.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.993+0000 7f2c33fff700 1 -- 172.21.15.47:0/3017409601 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2c3c12ad90 con 0x7f2c3c12a270 2024-02-23T00:53:36.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.993+0000 7f2c33fff700 10 client.31596.objecter ms_dispatch 0x7f2c3c001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:53:36.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.993+0000 7f2c33fff700 10 client.31596.objecter _finish_command 15 = system:0 2024-02-23T00:53:36.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.993+0000 7f2c33fff700 20 client.31596.objecter put_session s=0x7f2c3c129bf0 osd=7 4 2024-02-23T00:53:36.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:36.993+0000 7f2c33fff700 15 client.31596.objecter _session_command_op_remove 7 15 2024-02-23T00:53:37.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.007+0000 7f2c41450700 10 client.31596.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:53:37.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.007+0000 7f2c41450700 15 client.31596.objecter _session_command_op_assign -1 16 2024-02-23T00:53:37.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.007+0000 7f2c41450700 20 client.31596.objecter _get_session s=0x7f2c3c129bf0 osd=7 3 2024-02-23T00:53:37.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.007+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c129bf0 osd=7 3 2024-02-23T00:53:37.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.007+0000 7f2c41450700 20 client.31596.objecter _get_session s=0x7f2c3c129bf0 osd=7 3 2024-02-23T00:53:37.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.007+0000 7f2c41450700 15 client.31596.objecter _session_command_op_remove -1 16 2024-02-23T00:53:37.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.007+0000 7f2c41450700 20 client.31596.objecter get_session s=0x7f2c3c129bf0 osd=7 3 2024-02-23T00:53:37.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.007+0000 7f2c41450700 15 client.31596.objecter _session_command_op_assign 7 16 2024-02-23T00:53:37.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.007+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c129bf0 osd=7 4 2024-02-23T00:53:37.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.007+0000 7f2c41450700 10 client.31596.objecter _send_command 16 2024-02-23T00:53:37.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.007+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f2c3c12ad90 con 0x7f2c3c12a270 2024-02-23T00:53:37.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c33fff700 1 -- 172.21.15.47:0/3017409601 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f2c3c12ad90 con 0x7f2c3c12a270 2024-02-23T00:53:37.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c33fff700 10 client.31596.objecter ms_dispatch 0x7f2c3c001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:53:37.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c33fff700 10 client.31596.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:53:37.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c33fff700 20 client.31596.objecter put_session s=0x7f2c3c129bf0 osd=7 4 2024-02-23T00:53:37.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c33fff700 15 client.31596.objecter _session_command_op_remove 7 16 2024-02-23T00:53:37.011 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:53:37.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 10 librados: watch_flush enter 2024-02-23T00:53:37.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 10 librados: watch_flush exit 2024-02-23T00:53:37.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 10 client.31596.objecter close_session for osd.0 2024-02-23T00:53:37.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f2c3c0598f0 msgr2=0x7f2c3c059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:37.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f2c3c0598f0 0x7f2c3c059cd0 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:37.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c05ee80 osd=0 1 2024-02-23T00:53:37.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 10 client.31596.objecter close_session for osd.1 2024-02-23T00:53:37.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f2c3c0ff180 msgr2=0x7f2c3c0ff560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:37.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f2c3c0ff180 0x7f2c3c0ff560 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:37.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c028710 osd=1 1 2024-02-23T00:53:37.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 10 client.31596.objecter close_session for osd.2 2024-02-23T00:53:37.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f2c3c11c190 msgr2=0x7f2c3c11c5d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:37.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f2c3c11c190 0x7f2c3c11c5d0 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:37.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c027d60 osd=2 1 2024-02-23T00:53:37.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 10 client.31596.objecter close_session for osd.3 2024-02-23T00:53:37.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f2c3c027940 msgr2=0x7f2c3c121e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:37.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f2c3c027940 0x7f2c3c121e10 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:37.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c0277d0 osd=3 1 2024-02-23T00:53:37.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 10 client.31596.objecter close_session for osd.4 2024-02-23T00:53:37.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c3b7fe700 1 -- 172.21.15.47:0/3017409601 reap_dead start 2024-02-23T00:53:37.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2c3c11bcf0 msgr2=0x7f2c3c123480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:37.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2c3c11bcf0 0x7f2c3c123480 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:37.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c3b7fe700 1 -- 172.21.15.47:0/3017409601 reap_dead start 2024-02-23T00:53:37.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c11cac0 osd=4 1 2024-02-23T00:53:37.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 10 client.31596.objecter close_session for osd.5 2024-02-23T00:53:37.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f2c3c123e80 msgr2=0x7f2c3c1242c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:37.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f2c3c123e80 0x7f2c3c1242c0 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:37.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c11cc30 osd=5 1 2024-02-23T00:53:37.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 10 client.31596.objecter close_session for osd.6 2024-02-23T00:53:37.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f2c3c129190 msgr2=0x7f2c3c1295f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:37.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f2c3c129190 0x7f2c3c1295f0 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:37.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c1248c0 osd=6 1 2024-02-23T00:53:37.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 10 client.31596.objecter close_session for osd.7 2024-02-23T00:53:37.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f2c3c12a270 msgr2=0x7f2c3c12a6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:37.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f2c3c12a270 0x7f2c3c12a6b0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:37.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 20 client.31596.objecter put_session s=0x7f2c3c129bf0 osd=7 1 2024-02-23T00:53:37.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 20 client.31596.objecter shutdown clearing up homeless session... 2024-02-23T00:53:37.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 10 client.31596.objecter successfully canceled tick 2024-02-23T00:53:37.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2c28062cc0 msgr2=0x7f2c28065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:37.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2c28062cc0 0x7f2c28065180 secure :-1 s=READY pgs=3162 cs=0 l=1 rev1=1 rx=0x7f2c2c005010 tx=0x7f2c2c00b040).stop 2024-02-23T00:53:37.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c3c0fdbd0 msgr2=0x7f2c3c0f7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:37.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 1 --2- 172.21.15.47:0/3017409601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c3c0fdbd0 0x7f2c3c0f7b70 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 rx=0x7f2c24004920 tx=0x7f2c24009cb0).stop 2024-02-23T00:53:37.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c3b7fe700 1 -- 172.21.15.47:0/3017409601 reap_dead start 2024-02-23T00:53:37.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.009+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 shutdown_connections 2024-02-23T00:53:37.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.010+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 >> 172.21.15.47:0/3017409601 conn(0x7f2c3c06c370 msgr2=0x7f2c3c0fb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:37.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.010+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 shutdown_connections 2024-02-23T00:53:37.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.010+0000 7f2c41450700 1 -- 172.21.15.47:0/3017409601 wait complete. 2024-02-23T00:53:37.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:37.010+0000 7f2c41450700 1 librados: shutdown 2024-02-23T00:53:37.017 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:53:37.017 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:53:37.017 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:53:37.017 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:53:37.017 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:53:37.017 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:53:37.017 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:53:37.017 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:53:37.031 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:53:37.059 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:37.361 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:53:37.365 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:37.365 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:53:37.466 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:53:37.497 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:37.525 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:53:37.528 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:37.528 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:53:37.545 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.6 dump_ops_in_flight 2024-02-23T00:53:37.629 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:53:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:37.849 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:53:37.856 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:53:37.859 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:37.860 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:53:37.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:36.492962+0000 osd.6 (osd.6) 261 : cluster [DBG] 234.6 scrub starts 2024-02-23T00:53:37.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:36.494398+0000 osd.6 (osd.6) 262 : cluster [DBG] 234.6 scrub ok 2024-02-23T00:53:37.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:36.518175+0000 osd.4 (osd.4) 289 : cluster [DBG] 4.d scrub starts 2024-02-23T00:53:37.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:36.519980+0000 osd.4 (osd.4) 290 : cluster [DBG] 4.d scrub ok 2024-02-23T00:53:37.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:36.866891+0000 mon.a (mon.0) 3362 : audit [DBG] from='client.? 172.21.15.47:0/3017409601' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:53:37.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:37.293923+0000 mon.a (mon.0) 3363 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:53:37.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:37.294039+0000 mon.a (mon.0) 3364 : cluster [DBG] osdmap e680: 8 total, 8 up, 6 in 2024-02-23T00:53:37.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:37.297612+0000 mon.c (mon.2) 394 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:37.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:37.297925+0000 mon.a (mon.0) 3365 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:37.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:37.374756+0000 mgr.y (mgr.14517) 591 : cluster [DBG] pgmap v1028: 184 pgs: 32 creating+peering, 152 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 255 B/s wr, 0 op/s 2024-02-23T00:53:37.960 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:53:37.988 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:38.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:36.492962+0000 osd.6 (osd.6) 261 : cluster [DBG] 234.6 scrub starts 2024-02-23T00:53:38.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:36.494398+0000 osd.6 (osd.6) 262 : cluster [DBG] 234.6 scrub ok 2024-02-23T00:53:38.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:36.518175+0000 osd.4 (osd.4) 289 : cluster [DBG] 4.d scrub starts 2024-02-23T00:53:38.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:36.519980+0000 osd.4 (osd.4) 290 : cluster [DBG] 4.d scrub ok 2024-02-23T00:53:38.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:36.866891+0000 mon.a (mon.0) 3362 : audit [DBG] from='client.? 172.21.15.47:0/3017409601' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:53:38.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:37.293923+0000 mon.a (mon.0) 3363 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:53:38.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:37.294039+0000 mon.a (mon.0) 3364 : cluster [DBG] osdmap e680: 8 total, 8 up, 6 in 2024-02-23T00:53:38.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:37.297612+0000 mon.c (mon.2) 394 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:38.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:37.297925+0000 mon.a (mon.0) 3365 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:38.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:37.374756+0000 mgr.y (mgr.14517) 591 : cluster [DBG] pgmap v1028: 184 pgs: 32 creating+peering, 152 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 255 B/s wr, 0 op/s 2024-02-23T00:53:38.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:36.492962+0000 osd.6 (osd.6) 261 : cluster [DBG] 234.6 scrub starts 2024-02-23T00:53:38.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:36.494398+0000 osd.6 (osd.6) 262 : cluster [DBG] 234.6 scrub ok 2024-02-23T00:53:38.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:36.518175+0000 osd.4 (osd.4) 289 : cluster [DBG] 4.d scrub starts 2024-02-23T00:53:38.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:36.519980+0000 osd.4 (osd.4) 290 : cluster [DBG] 4.d scrub ok 2024-02-23T00:53:38.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:36.866891+0000 mon.a (mon.0) 3362 : audit [DBG] from='client.? 172.21.15.47:0/3017409601' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:53:38.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:37.293923+0000 mon.a (mon.0) 3363 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:53:38.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:37.294039+0000 mon.a (mon.0) 3364 : cluster [DBG] osdmap e680: 8 total, 8 up, 6 in 2024-02-23T00:53:38.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:37.297612+0000 mon.c (mon.2) 394 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:38.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:37.297925+0000 mon.a (mon.0) 3365 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:38.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:37.374756+0000 mgr.y (mgr.14517) 591 : cluster [DBG] pgmap v1028: 184 pgs: 32 creating+peering, 152 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 255 B/s wr, 0 op/s 2024-02-23T00:53:38.086 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:53:38.088 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:38.089 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:53:38.190 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:53:38.219 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:38.239 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:53:38.241 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:38.242 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:53:38.343 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:53:38.371 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:38.390 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:53:38.392 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:38.393 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:53:38.494 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:53:38.523 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:38.541 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:53:38.544 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:38.545 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:53:38.646 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:53:38.673 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:38.692 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:53:38.694 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:38.695 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:53:38.796 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:53:38.823 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:39.073 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:53:39.075 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:39.076 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:53:39.177 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:53:39.192 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set noscrub 2024-02-23T00:53:39.203 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:39.352 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:53:39.353 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:38.297815+0000 mon.a (mon.0) 3366 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:53:39.353 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:38.297902+0000 mon.a (mon.0) 3367 : cluster [DBG] osdmap e681: 8 total, 8 up, 6 in 2024-02-23T00:53:39.353 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:38.298460+0000 mon.c (mon.2) 395 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-118"}]: dispatch 2024-02-23T00:53:39.354 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:38.298748+0000 mon.a (mon.0) 3368 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-118"}]: dispatch 2024-02-23T00:53:39.354 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:38.348036+0000 osd.7 (osd.7) 321 : cluster [DBG] 5.4 scrub starts 2024-02-23T00:53:39.354 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:38.350913+0000 osd.7 (osd.7) 322 : cluster [DBG] 5.4 scrub ok 2024-02-23T00:53:39.354 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:38.579362+0000 osd.4 (osd.4) 291 : cluster [DBG] 2.f scrub starts 2024-02-23T00:53:39.354 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:38.580853+0000 osd.4 (osd.4) 292 : cluster [DBG] 2.f scrub ok 2024-02-23T00:53:39.354 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:39.121342+0000 osd.5 (osd.5) 299 : cluster [DBG] 5.6 scrub starts 2024-02-23T00:53:39.354 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:39.124248+0000 osd.5 (osd.5) 300 : cluster [DBG] 5.6 scrub ok 2024-02-23T00:53:39.354 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:39.355 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:53:39.456 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:53:39.479 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:53:39.482 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:38.297815+0000 mon.a (mon.0) 3366 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:53:39.482 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:38.297902+0000 mon.a (mon.0) 3367 : cluster [DBG] osdmap e681: 8 total, 8 up, 6 in 2024-02-23T00:53:39.483 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:38.298460+0000 mon.c (mon.2) 395 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-118"}]: dispatch 2024-02-23T00:53:39.483 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:38.298748+0000 mon.a (mon.0) 3368 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-118"}]: dispatch 2024-02-23T00:53:39.483 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:38.348036+0000 osd.7 (osd.7) 321 : cluster [DBG] 5.4 scrub starts 2024-02-23T00:53:39.483 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:38.350913+0000 osd.7 (osd.7) 322 : cluster [DBG] 5.4 scrub ok 2024-02-23T00:53:39.483 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:38.579362+0000 osd.4 (osd.4) 291 : cluster [DBG] 2.f scrub starts 2024-02-23T00:53:39.483 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:38.580853+0000 osd.4 (osd.4) 292 : cluster [DBG] 2.f scrub ok 2024-02-23T00:53:39.483 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:39.121342+0000 osd.5 (osd.5) 299 : cluster [DBG] 5.6 scrub starts 2024-02-23T00:53:39.483 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:39.124248+0000 osd.5 (osd.5) 300 : cluster [DBG] 5.6 scrub ok 2024-02-23T00:53:39.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:38.297815+0000 mon.a (mon.0) 3366 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:53:39.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:38.297902+0000 mon.a (mon.0) 3367 : cluster [DBG] osdmap e681: 8 total, 8 up, 6 in 2024-02-23T00:53:39.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:38.298460+0000 mon.c (mon.2) 395 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-118"}]: dispatch 2024-02-23T00:53:39.484 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:38.298748+0000 mon.a (mon.0) 3368 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-118"}]: dispatch 2024-02-23T00:53:39.484 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:38.348036+0000 osd.7 (osd.7) 321 : cluster [DBG] 5.4 scrub starts 2024-02-23T00:53:39.484 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:38.350913+0000 osd.7 (osd.7) 322 : cluster [DBG] 5.4 scrub ok 2024-02-23T00:53:39.484 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:38.579362+0000 osd.4 (osd.4) 291 : cluster [DBG] 2.f scrub starts 2024-02-23T00:53:39.484 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:38.580853+0000 osd.4 (osd.4) 292 : cluster [DBG] 2.f scrub ok 2024-02-23T00:53:39.484 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:39.121342+0000 osd.5 (osd.5) 299 : cluster [DBG] 5.6 scrub starts 2024-02-23T00:53:39.484 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:39.124248+0000 osd.5 (osd.5) 300 : cluster [DBG] 5.6 scrub ok 2024-02-23T00:53:39.485 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:39.508 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:53:39.511 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:39.512 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:53:39.598 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:53:39.598 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:53:39.612 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:53:39.641 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:39.701 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6, 7] out_osds: [2, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-23T00:53:39.701 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight 1 1 2024-02-23T00:53:39.842 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:53:39.845 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:39.845 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:53:39.880 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:53:39.946 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:53:39.975 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:40.200 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.6 dump_blocked_ops 2024-02-23T00:53:40.223 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:53:40.226 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:40.226 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:53:40.327 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:53:40.355 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:40.498 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:53:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:39.299827+0000 mon.a (mon.0) 3369 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-118"}]': finished 2024-02-23T00:53:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:39.299918+0000 mon.a (mon.0) 3370 : cluster [DBG] osdmap e682: 8 total, 8 up, 6 in 2024-02-23T00:53:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:39.300515+0000 mon.c (mon.2) 396 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-118", "mode": "writeback"}]: dispatch 2024-02-23T00:53:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:39.300812+0000 mon.a (mon.0) 3371 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-118", "mode": "writeback"}]: dispatch 2024-02-23T00:53:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:39.375132+0000 mgr.y (mgr.14517) 592 : cluster [DBG] pgmap v1031: 184 pgs: 32 creating+peering, 152 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 255 B/s wr, 0 op/s 2024-02-23T00:53:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:40.163091+0000 osd.5 (osd.5) 301 : cluster [DBG] 2.16 scrub starts 2024-02-23T00:53:40.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:40.166165+0000 osd.5 (osd.5) 302 : cluster [DBG] 2.16 scrub ok 2024-02-23T00:53:40.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:40.297013+0000 mon.a (mon.0) 3372 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:53:40.802 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:53:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:53:40] "GET /metrics HTTP/1.1" 200 191182 "" "Prometheus/2.33.4" 2024-02-23T00:53:40.803 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:39.299827+0000 mon.a (mon.0) 3369 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-118"}]': finished 2024-02-23T00:53:40.803 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:39.299918+0000 mon.a (mon.0) 3370 : cluster [DBG] osdmap e682: 8 total, 8 up, 6 in 2024-02-23T00:53:40.803 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:39.300515+0000 mon.c (mon.2) 396 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-118", "mode": "writeback"}]: dispatch 2024-02-23T00:53:40.803 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:39.300812+0000 mon.a (mon.0) 3371 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-118", "mode": "writeback"}]: dispatch 2024-02-23T00:53:40.803 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:39.375132+0000 mgr.y (mgr.14517) 592 : cluster [DBG] pgmap v1031: 184 pgs: 32 creating+peering, 152 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 255 B/s wr, 0 op/s 2024-02-23T00:53:40.803 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:40.163091+0000 osd.5 (osd.5) 301 : cluster [DBG] 2.16 scrub starts 2024-02-23T00:53:40.803 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:40.166165+0000 osd.5 (osd.5) 302 : cluster [DBG] 2.16 scrub ok 2024-02-23T00:53:40.803 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:40.297013+0000 mon.a (mon.0) 3372 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:53:40.803 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:39.299827+0000 mon.a (mon.0) 3369 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-118"}]': finished 2024-02-23T00:53:40.804 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:39.299918+0000 mon.a (mon.0) 3370 : cluster [DBG] osdmap e682: 8 total, 8 up, 6 in 2024-02-23T00:53:40.804 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:39.300515+0000 mon.c (mon.2) 396 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-118", "mode": "writeback"}]: dispatch 2024-02-23T00:53:40.804 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:39.300812+0000 mon.a (mon.0) 3371 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-118", "mode": "writeback"}]: dispatch 2024-02-23T00:53:40.804 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:39.375132+0000 mgr.y (mgr.14517) 592 : cluster [DBG] pgmap v1031: 184 pgs: 32 creating+peering, 152 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 255 B/s wr, 0 op/s 2024-02-23T00:53:40.804 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:40.163091+0000 osd.5 (osd.5) 301 : cluster [DBG] 2.16 scrub starts 2024-02-23T00:53:40.804 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:40.166165+0000 osd.5 (osd.5) 302 : cluster [DBG] 2.16 scrub ok 2024-02-23T00:53:40.804 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:40.297013+0000 mon.a (mon.0) 3372 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:53:40.805 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:53:40.808 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:40.808 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:53:40.909 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:53:40.938 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:40.991 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:53:40.995 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:40.995 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:53:41.096 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:53:41.124 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:41.274 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:53:41.276 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:41.276 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:53:41.324 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:53:41.325 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:53:41.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.344+0000 7f45d3b7a700 1 -- 172.21.15.47:0/2395637597 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f45cc0f63b0 msgr2=0x7f45cc0f6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:41.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.344+0000 7f45d3b7a700 1 --2- 172.21.15.47:0/2395637597 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f45cc0f63b0 0x7f45cc0f6790 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 rx=0x7f45c80047b0 tx=0x7f45c8009cb0).stop 2024-02-23T00:53:41.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.344+0000 7f45d3b7a700 1 -- 172.21.15.47:0/2395637597 shutdown_connections 2024-02-23T00:53:41.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.344+0000 7f45d3b7a700 1 --2- 172.21.15.47:0/2395637597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45cc1087f0 0x7f45cc10abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:41.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.344+0000 7f45d3b7a700 1 --2- 172.21.15.47:0/2395637597 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f45cc0f63b0 0x7f45cc0f6790 unknown :-1 s=CLOSED pgs=2773 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:41.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.344+0000 7f45d3b7a700 1 --2- 172.21.15.47:0/2395637597 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45cc0f6c80 0x7f45cc108300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:41.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.344+0000 7f45d3b7a700 1 -- 172.21.15.47:0/2395637597 >> 172.21.15.47:0/2395637597 conn(0x7f45cc06c370 msgr2=0x7f45cc06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:41.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.345+0000 7f45d3b7a700 1 -- 172.21.15.47:0/2395637597 shutdown_connections 2024-02-23T00:53:41.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.345+0000 7f45d3b7a700 1 -- 172.21.15.47:0/2395637597 wait complete. 2024-02-23T00:53:41.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.345+0000 7f45d3b7a700 1 librados: starting msgr at 2024-02-23T00:53:41.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.345+0000 7f45d3b7a700 1 librados: starting objecter 2024-02-23T00:53:41.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.345+0000 7f45d3b7a700 1 Processor -- start 2024-02-23T00:53:41.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.345+0000 7f45d3b7a700 1 -- start start 2024-02-23T00:53:41.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.345+0000 7f45d3b7a700 1 librados: setting wanted keys 2024-02-23T00:53:41.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.345+0000 7f45d3b7a700 1 librados: calling monclient init 2024-02-23T00:53:41.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.346+0000 7f45d3b7a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45cc0f63b0 0x7f45cc114310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:41.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.346+0000 7f45d3b7a700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f45cc0f6c80 0x7f45cc114800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:41.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.346+0000 7f45d3b7a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45cc1087f0 0x7f45cc10e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:41.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.346+0000 7f45d3b7a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc10ebb0 con 0x7f45cc1087f0 2024-02-23T00:53:41.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.346+0000 7f45d3b7a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc10ed20 con 0x7f45cc0f63b0 2024-02-23T00:53:41.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.346+0000 7f45d3b7a700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc10eee0 con 0x7f45cc0f6c80 2024-02-23T00:53:41.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.346+0000 7f45d1916700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45cc0f63b0 0x7f45cc114310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:41.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.346+0000 7f45d1115700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f45cc0f6c80 0x7f45cc114800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:41.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.346+0000 7f45d1916700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45cc0f63b0 0x7f45cc114310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.47:57150/0 (socket says 172.21.15.47:57150) 2024-02-23T00:53:41.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.346+0000 7f45d1115700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f45cc0f6c80 0x7f45cc114800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:50344/0 (socket says 172.21.15.47:50344) 2024-02-23T00:53:41.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.346+0000 7f45d1916700 1 -- 172.21.15.47:0/2216335458 learned_addr learned my addr 172.21.15.47:0/2216335458 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:53:41.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.346+0000 7f45d1916700 1 -- 172.21.15.47:0/2216335458 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f45cc0f6c80 msgr2=0x7f45cc114800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:41.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.346+0000 7f45d1916700 1 --2- 172.21.15.47:0/2216335458 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f45cc0f6c80 0x7f45cc114800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:41.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.346+0000 7f45d1916700 1 -- 172.21.15.47:0/2216335458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45cc1087f0 msgr2=0x7f45cc10e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:41.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.346+0000 7f45d1916700 1 --2- 172.21.15.47:0/2216335458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45cc1087f0 0x7f45cc10e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:53:41.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.346+0000 7f45d1916700 1 -- 172.21.15.47:0/2216335458 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c8005040 con 0x7f45cc0f63b0 2024-02-23T00:53:41.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.346+0000 7f45d1916700 1 --2- 172.21.15.47:0/2216335458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45cc0f63b0 0x7f45cc114310 secure :-1 s=READY pgs=3983 cs=0 l=1 rev1=1 rx=0x7f45c8013d80 tx=0x7f45c8009cb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:41.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.347+0000 7f45c2ffd700 10 client.?.objecter ms_handle_connect 0x7f45cc0f63b0 2024-02-23T00:53:41.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.347+0000 7f45c2ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:53:41.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.347+0000 7f45c2ffd700 1 -- 172.21.15.47:0/2216335458 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c8018070 con 0x7f45cc0f63b0 2024-02-23T00:53:41.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.347+0000 7f45c2ffd700 1 -- 172.21.15.47:0/2216335458 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f45c8009520 con 0x7f45cc0f63b0 2024-02-23T00:53:41.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.347+0000 7f45c2ffd700 1 -- 172.21.15.47:0/2216335458 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c801c3c0 con 0x7f45cc0f63b0 2024-02-23T00:53:41.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.348+0000 7f45d3b7a700 1 -- 172.21.15.47:0/2216335458 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45cc10f160 con 0x7f45cc0f63b0 2024-02-23T00:53:41.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.348+0000 7f45d3b7a700 10 client.43912.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:53:41.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.348+0000 7f45d3b7a700 1 -- 172.21.15.47:0/2216335458 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45cc11b140 con 0x7f45cc0f63b0 2024-02-23T00:53:41.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.348+0000 7f45d3b7a700 1 librados: init done 2024-02-23T00:53:41.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.349+0000 7f45c0ff9700 1 -- 172.21.15.47:0/2216335458 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45cc10f5a0 con 0x7f45cc0f63b0 2024-02-23T00:53:41.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.349+0000 7f45c2ffd700 1 -- 172.21.15.47:0/2216335458 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f45c801d420 con 0x7f45cc0f63b0 2024-02-23T00:53:41.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.349+0000 7f45c2ffd700 1 --2- 172.21.15.47:0/2216335458 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f45b8062d10 0x7f45b80651d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:41.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.350+0000 7f45c2ffd700 1 -- 172.21.15.47:0/2216335458 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(683..683 src has 482..683) v4 ==== 9633+0+0 (secure 0 0 0) 0x7f45c807ee20 con 0x7f45cc0f63b0 2024-02-23T00:53:41.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.350+0000 7f45c2ffd700 10 client.43912.objecter ms_dispatch 0x7f45cc001550 osd_map(683..683 src has 482..683) v4 2024-02-23T00:53:41.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.350+0000 7f45c2ffd700 3 client.43912.objecter handle_osd_map got epochs [683,683] > 0 2024-02-23T00:53:41.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.350+0000 7f45c2ffd700 3 client.43912.objecter handle_osd_map decoding full epoch 683 2024-02-23T00:53:41.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.350+0000 7f45d1115700 1 --2- 172.21.15.47:0/2216335458 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f45b8062d10 0x7f45b80651d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:41.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.350+0000 7f45c2ffd700 20 client.43912.objecter dump_active .. 0 homeless 2024-02-23T00:53:41.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.350+0000 7f45c2ffd700 10 client.43912.objecter ms_handle_connect 0x7f45b8062d10 2024-02-23T00:53:41.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.350+0000 7f45d1115700 1 --2- 172.21.15.47:0/2216335458 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f45b8062d10 0x7f45b80651d0 secure :-1 s=READY pgs=3163 cs=0 l=1 rev1=1 rx=0x7f45cc10fc50 tx=0x7f45bc00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:41.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.354+0000 7f45c2ffd700 1 -- 172.21.15.47:0/2216335458 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f45c804e270 con 0x7f45cc0f63b0 2024-02-23T00:53:41.377 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:53:41.405 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:40.300980+0000 mon.a (mon.0) 3373 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-118", "mode": "writeback"}]': finished 2024-02-23T00:53:41.405 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:40.301072+0000 mon.a (mon.0) 3374 : cluster [DBG] osdmap e683: 8 total, 8 up, 6 in 2024-02-23T00:53:41.405 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:40.349612+0000 osd.7 (osd.7) 323 : cluster [DBG] 2.e scrub starts 2024-02-23T00:53:41.405 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:40.350364+0000 osd.7 (osd.7) 324 : cluster [DBG] 2.e scrub ok 2024-02-23T00:53:41.405 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:41.131926+0000 osd.5 (osd.5) 303 : cluster [DBG] 5.b scrub starts 2024-02-23T00:53:41.405 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:41.132683+0000 osd.5 (osd.5) 304 : cluster [DBG] 5.b scrub ok 2024-02-23T00:53:41.405 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:40.300980+0000 mon.a (mon.0) 3373 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-118", "mode": "writeback"}]': finished 2024-02-23T00:53:41.406 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:40.301072+0000 mon.a (mon.0) 3374 : cluster [DBG] osdmap e683: 8 total, 8 up, 6 in 2024-02-23T00:53:41.406 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:40.349612+0000 osd.7 (osd.7) 323 : cluster [DBG] 2.e scrub starts 2024-02-23T00:53:41.406 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:40.350364+0000 osd.7 (osd.7) 324 : cluster [DBG] 2.e scrub ok 2024-02-23T00:53:41.406 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:41.131926+0000 osd.5 (osd.5) 303 : cluster [DBG] 5.b scrub starts 2024-02-23T00:53:41.406 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:41.132683+0000 osd.5 (osd.5) 304 : cluster [DBG] 5.b scrub ok 2024-02-23T00:53:41.408 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:41.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:40.300980+0000 mon.a (mon.0) 3373 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-118", "mode": "writeback"}]': finished 2024-02-23T00:53:41.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:40.301072+0000 mon.a (mon.0) 3374 : cluster [DBG] osdmap e683: 8 total, 8 up, 6 in 2024-02-23T00:53:41.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:40.349612+0000 osd.7 (osd.7) 323 : cluster [DBG] 2.e scrub starts 2024-02-23T00:53:41.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:40.350364+0000 osd.7 (osd.7) 324 : cluster [DBG] 2.e scrub ok 2024-02-23T00:53:41.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:41.131926+0000 osd.5 (osd.5) 303 : cluster [DBG] 5.b scrub starts 2024-02-23T00:53:41.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:41.132683+0000 osd.5 (osd.5) 304 : cluster [DBG] 5.b scrub ok 2024-02-23T00:53:41.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:41.545+0000 7f45c0ff9700 1 -- 172.21.15.47:0/2216335458 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7f45cc05f930 con 0x7f45cc0f63b0 2024-02-23T00:53:42.286 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:53:42.286 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:53:42.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:42.317+0000 7f45c2ffd700 1 -- 172.21.15.47:0/2216335458 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v684) v1 ==== 87+0+0 (secure 0 0 0) 0x7f45c80043d0 con 0x7f45cc0f63b0 2024-02-23T00:53:42.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:42.319+0000 7f45c0ff9700 10 librados: watch_flush enter 2024-02-23T00:53:42.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:42.319+0000 7f45c0ff9700 10 librados: watch_flush exit 2024-02-23T00:53:42.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:42.319+0000 7f45c0ff9700 20 client.43912.objecter shutdown clearing up homeless session... 2024-02-23T00:53:42.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:42.319+0000 7f45c0ff9700 10 client.43912.objecter successfully canceled tick 2024-02-23T00:53:42.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:42.319+0000 7f45c0ff9700 1 -- 172.21.15.47:0/2216335458 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f45b8062d10 msgr2=0x7f45b80651d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:42.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:42.319+0000 7f45c0ff9700 1 --2- 172.21.15.47:0/2216335458 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f45b8062d10 0x7f45b80651d0 secure :-1 s=READY pgs=3163 cs=0 l=1 rev1=1 rx=0x7f45cc10fc50 tx=0x7f45bc00b040).stop 2024-02-23T00:53:42.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:42.319+0000 7f45c0ff9700 1 -- 172.21.15.47:0/2216335458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45cc0f63b0 msgr2=0x7f45cc114310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:42.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:42.319+0000 7f45c0ff9700 1 --2- 172.21.15.47:0/2216335458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45cc0f63b0 0x7f45cc114310 secure :-1 s=READY pgs=3983 cs=0 l=1 rev1=1 rx=0x7f45c8013d80 tx=0x7f45c8009cb0).stop 2024-02-23T00:53:42.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:42.319+0000 7f45c0ff9700 1 -- 172.21.15.47:0/2216335458 shutdown_connections 2024-02-23T00:53:42.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:42.319+0000 7f45c0ff9700 1 --2- 172.21.15.47:0/2216335458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45cc1087f0 0x7f45cc10e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:53:42.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:42.319+0000 7f45c0ff9700 1 --2- 172.21.15.47:0/2216335458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45cc0f63b0 0x7f45cc114310 unknown :-1 s=CLOSED pgs=3983 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:42.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:42.319+0000 7f45c0ff9700 1 --2- 172.21.15.47:0/2216335458 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f45b8062d10 0x7f45b80651d0 unknown :-1 s=CLOSED pgs=3163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:42.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:42.319+0000 7f45c0ff9700 1 --2- 172.21.15.47:0/2216335458 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f45cc0f6c80 0x7f45cc114800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:42.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:42.319+0000 7f45c0ff9700 1 -- 172.21.15.47:0/2216335458 >> 172.21.15.47:0/2216335458 conn(0x7f45cc06c370 msgr2=0x7f45cc106b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:42.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:42.319+0000 7f45c0ff9700 1 -- 172.21.15.47:0/2216335458 shutdown_connections 2024-02-23T00:53:42.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:42.320+0000 7f45c0ff9700 1 -- 172.21.15.47:0/2216335458 wait complete. 2024-02-23T00:53:42.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:42.320+0000 7f45c0ff9700 1 librados: shutdown 2024-02-23T00:53:42.324 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is set 2024-02-23T00:53:42.632 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:53:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:53:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:53:42.632 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:41.308326+0000 mon.a (mon.0) 3375 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:53:42.632 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:41.375580+0000 mgr.y (mgr.14517) 593 : cluster [DBG] pgmap v1033: 184 pgs: 32 creating+peering, 152 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:53:42.632 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:41.469192+0000 osd.1 (osd.1) 280 : cluster [DBG] 5.12 scrub starts 2024-02-23T00:53:42.632 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:41.472341+0000 osd.1 (osd.1) 281 : cluster [DBG] 5.12 scrub ok 2024-02-23T00:53:42.632 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:41.547314+0000 mon.b (mon.1) 382 : audit [INF] from='client.? 172.21.15.47:0/2216335458' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:53:42.633 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:41.547859+0000 mon.a (mon.0) 3376 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:53:42.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:41.308326+0000 mon.a (mon.0) 3375 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:53:42.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:41.375580+0000 mgr.y (mgr.14517) 593 : cluster [DBG] pgmap v1033: 184 pgs: 32 creating+peering, 152 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:53:42.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:41.469192+0000 osd.1 (osd.1) 280 : cluster [DBG] 5.12 scrub starts 2024-02-23T00:53:42.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:41.472341+0000 osd.1 (osd.1) 281 : cluster [DBG] 5.12 scrub ok 2024-02-23T00:53:42.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:41.547314+0000 mon.b (mon.1) 382 : audit [INF] from='client.? 172.21.15.47:0/2216335458' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:53:42.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:41.547859+0000 mon.a (mon.0) 3376 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:53:42.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:41.308326+0000 mon.a (mon.0) 3375 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:53:42.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:41.375580+0000 mgr.y (mgr.14517) 593 : cluster [DBG] pgmap v1033: 184 pgs: 32 creating+peering, 152 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:53:42.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:41.469192+0000 osd.1 (osd.1) 280 : cluster [DBG] 5.12 scrub starts 2024-02-23T00:53:42.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:41.472341+0000 osd.1 (osd.1) 281 : cluster [DBG] 5.12 scrub ok 2024-02-23T00:53:42.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:41.547314+0000 mon.b (mon.1) 382 : audit [INF] from='client.? 172.21.15.47:0/2216335458' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:53:42.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:41.547859+0000 mon.a (mon.0) 3376 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:53:42.920 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:53:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:53:42.940 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.6 dump_historic_ops 2024-02-23T00:53:42.959 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:53:42.963 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:42.963 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:53:43.064 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:53:43.092 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:43.111 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:53:43.114 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:43.114 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:53:43.213 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:53:43.216 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:53:43.248 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:43.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:42.310410+0000 mon.a (mon.0) 3377 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:53:43.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:42.317454+0000 mon.a (mon.0) 3378 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:53:43.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:42.317531+0000 mon.a (mon.0) 3379 : cluster [DBG] osdmap e684: 8 total, 8 up, 6 in 2024-02-23T00:53:43.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:42.310410+0000 mon.a (mon.0) 3377 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:53:43.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:42.317454+0000 mon.a (mon.0) 3378 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:53:43.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:42.317531+0000 mon.a (mon.0) 3379 : cluster [DBG] osdmap e684: 8 total, 8 up, 6 in 2024-02-23T00:53:43.685 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:42.310410+0000 mon.a (mon.0) 3377 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:53:43.685 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:42.317454+0000 mon.a (mon.0) 3378 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:53:43.685 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:42.317531+0000 mon.a (mon.0) 3379 : cluster [DBG] osdmap e684: 8 total, 8 up, 6 in 2024-02-23T00:53:44.301 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:53:44.304 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:44.305 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:53:44.406 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:53:44.439 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:44.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:42.875082+0000 mgr.y (mgr.14517) 594 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:53:44.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:43.376045+0000 mgr.y (mgr.14517) 595 : cluster [DBG] pgmap v1035: 184 pgs: 184 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 403 B/s wr, 1 op/s 2024-02-23T00:53:44.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:43.458195+0000 mon.a (mon.0) 3380 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [3, 4]}]: dispatch 2024-02-23T00:53:44.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:43.458406+0000 mon.a (mon.0) 3381 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.6", "id": [5, 6]}]: dispatch 2024-02-23T00:53:44.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:43.458591+0000 mon.a (mon.0) 3382 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1c", "id": [4, 6]}]: dispatch 2024-02-23T00:53:44.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:43.458751+0000 mon.a (mon.0) 3383 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "5.10"}]: dispatch 2024-02-23T00:53:44.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:42.875082+0000 mgr.y (mgr.14517) 594 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:53:44.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:43.376045+0000 mgr.y (mgr.14517) 595 : cluster [DBG] pgmap v1035: 184 pgs: 184 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 403 B/s wr, 1 op/s 2024-02-23T00:53:44.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:43.458195+0000 mon.a (mon.0) 3380 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [3, 4]}]: dispatch 2024-02-23T00:53:44.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:43.458406+0000 mon.a (mon.0) 3381 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.6", "id": [5, 6]}]: dispatch 2024-02-23T00:53:44.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:43.458591+0000 mon.a (mon.0) 3382 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1c", "id": [4, 6]}]: dispatch 2024-02-23T00:53:44.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:43.458751+0000 mon.a (mon.0) 3383 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "5.10"}]: dispatch 2024-02-23T00:53:44.617 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:42.875082+0000 mgr.y (mgr.14517) 594 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:53:44.617 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:43.376045+0000 mgr.y (mgr.14517) 595 : cluster [DBG] pgmap v1035: 184 pgs: 184 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 403 B/s wr, 1 op/s 2024-02-23T00:53:44.618 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:43.458195+0000 mon.a (mon.0) 3380 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [3, 4]}]: dispatch 2024-02-23T00:53:44.618 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:43.458406+0000 mon.a (mon.0) 3381 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.6", "id": [5, 6]}]: dispatch 2024-02-23T00:53:44.618 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:43.458591+0000 mon.a (mon.0) 3382 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1c", "id": [4, 6]}]: dispatch 2024-02-23T00:53:44.618 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:43.458751+0000 mon.a (mon.0) 3383 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "5.10"}]: dispatch 2024-02-23T00:53:44.689 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:53:44.693 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:44.693 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:53:44.794 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:53:44.824 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:44.884 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:53:44.884 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:53:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:53:44.976 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:44.977 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:53:45.078 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:53:45.108 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:45.129 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:53:45.132 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:45.132 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:53:45.233 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:53:45.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.230+0000 7f51fbb56700 1 -- 172.21.15.47:0/3079706136 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f51f40fcab0 msgr2=0x7f51f40fce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:45.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.230+0000 7f51fbb56700 1 --2- 172.21.15.47:0/3079706136 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f51f40fcab0 0x7f51f40fce90 secure :-1 s=READY pgs=2774 cs=0 l=1 rev1=1 rx=0x7f51e4004870 tx=0x7f51e4009cb0).stop 2024-02-23T00:53:45.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.230+0000 7f51fbb56700 1 -- 172.21.15.47:0/3079706136 shutdown_connections 2024-02-23T00:53:45.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.230+0000 7f51fbb56700 1 --2- 172.21.15.47:0/3079706136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51f40f7320 0x7f51f4103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:45.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.230+0000 7f51fbb56700 1 --2- 172.21.15.47:0/3079706136 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f51f40fcab0 0x7f51f40fce90 unknown :-1 s=CLOSED pgs=2774 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:45.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.230+0000 7f51fbb56700 1 --2- 172.21.15.47:0/3079706136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51f40fd410 0x7f51f40f6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:45.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.230+0000 7f51fbb56700 1 -- 172.21.15.47:0/3079706136 >> 172.21.15.47:0/3079706136 conn(0x7f51f40f26e0 msgr2=0x7f51f40f4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:45.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.230+0000 7f51fbb56700 1 -- 172.21.15.47:0/3079706136 shutdown_connections 2024-02-23T00:53:45.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.230+0000 7f51fbb56700 1 -- 172.21.15.47:0/3079706136 wait complete. 2024-02-23T00:53:45.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.231+0000 7f51fbb56700 1 librados: starting msgr at 2024-02-23T00:53:45.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.231+0000 7f51fbb56700 1 librados: starting objecter 2024-02-23T00:53:45.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.231+0000 7f51fbb56700 1 Processor -- start 2024-02-23T00:53:45.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.231+0000 7f51fbb56700 1 -- start start 2024-02-23T00:53:45.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.231+0000 7f51fbb56700 1 librados: setting wanted keys 2024-02-23T00:53:45.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.231+0000 7f51fbb56700 1 librados: calling monclient init 2024-02-23T00:53:45.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.231+0000 7f51fbb56700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51f40f7320 0x7f51f410df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:45.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.231+0000 7f51fbb56700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51f40fcab0 0x7f51f410e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:45.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.231+0000 7f51fbb56700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f51f40fd410 0x7f51f41080a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:45.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.231+0000 7f51fbb56700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f4108590 con 0x7f51f40f7320 2024-02-23T00:53:45.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.231+0000 7f51fbb56700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f4108700 con 0x7f51f40fcab0 2024-02-23T00:53:45.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.231+0000 7f51fbb56700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f41088c0 con 0x7f51f40fd410 2024-02-23T00:53:45.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.232+0000 7f51f98f2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51f40f7320 0x7f51f410df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:45.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.232+0000 7f51f90f1700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51f40fcab0 0x7f51f410e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:45.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.232+0000 7f51f98f2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51f40f7320 0x7f51f410df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54122/0 (socket says 172.21.15.47:54122) 2024-02-23T00:53:45.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.232+0000 7f51f98f2700 1 -- 172.21.15.47:0/1116100076 learned_addr learned my addr 172.21.15.47:0/1116100076 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:53:45.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.232+0000 7f51fa0f3700 1 --2- 172.21.15.47:0/1116100076 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f51f40fd410 0x7f51f41080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:45.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.232+0000 7f51f98f2700 1 -- 172.21.15.47:0/1116100076 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f51f40fd410 msgr2=0x7f51f41080a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:45.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.232+0000 7f51f98f2700 1 --2- 172.21.15.47:0/1116100076 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f51f40fd410 0x7f51f41080a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:45.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.232+0000 7f51f98f2700 1 -- 172.21.15.47:0/1116100076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51f40fcab0 msgr2=0x7f51f410e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:45.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.232+0000 7f51f98f2700 1 --2- 172.21.15.47:0/1116100076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51f40fcab0 0x7f51f410e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:45.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.232+0000 7f51f98f2700 1 -- 172.21.15.47:0/1116100076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f0005040 con 0x7f51f40f7320 2024-02-23T00:53:45.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.232+0000 7f51f98f2700 1 --2- 172.21.15.47:0/1116100076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51f40f7320 0x7f51f410df00 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 rx=0x7f51e4009da0 tx=0x7f51e4009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:45.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.232+0000 7f51eaffd700 10 client.?.objecter ms_handle_connect 0x7f51f40f7320 2024-02-23T00:53:45.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.232+0000 7f51eaffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:53:45.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.232+0000 7f51eaffd700 1 -- 172.21.15.47:0/1116100076 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e400e030 con 0x7f51f40f7320 2024-02-23T00:53:45.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.232+0000 7f51fbb56700 1 -- 172.21.15.47:0/1116100076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e4005040 con 0x7f51f40f7320 2024-02-23T00:53:45.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.232+0000 7f51eaffd700 1 -- 172.21.15.47:0/1116100076 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f51e4013d70 con 0x7f51f40f7320 2024-02-23T00:53:45.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.232+0000 7f51eaffd700 1 -- 172.21.15.47:0/1116100076 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e4018070 con 0x7f51f40f7320 2024-02-23T00:53:45.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.232+0000 7f51fbb56700 10 client.31605.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:53:45.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.232+0000 7f51fbb56700 1 -- 172.21.15.47:0/1116100076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f4108d60 con 0x7f51f40f7320 2024-02-23T00:53:45.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.232+0000 7f51fbb56700 1 librados: init done 2024-02-23T00:53:45.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.233+0000 7f51fbb56700 1 -- 172.21.15.47:0/1116100076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51f4108f80 con 0x7f51f40f7320 2024-02-23T00:53:45.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.233+0000 7f51eaffd700 1 -- 172.21.15.47:0/1116100076 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f51e4009240 con 0x7f51f40f7320 2024-02-23T00:53:45.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.234+0000 7f51eaffd700 1 --2- 172.21.15.47:0/1116100076 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f51e0062cc0 0x7f51e0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:45.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.234+0000 7f51eaffd700 1 -- 172.21.15.47:0/1116100076 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(685..685 src has 482..685) v4 ==== 9691+0+0 (secure 0 0 0) 0x7f51e407e9e0 con 0x7f51f40f7320 2024-02-23T00:53:45.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.234+0000 7f51eaffd700 10 client.31605.objecter ms_dispatch 0x7f51f4001550 osd_map(685..685 src has 482..685) v4 2024-02-23T00:53:45.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.234+0000 7f51eaffd700 3 client.31605.objecter handle_osd_map got epochs [685,685] > 0 2024-02-23T00:53:45.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.234+0000 7f51eaffd700 3 client.31605.objecter handle_osd_map decoding full epoch 685 2024-02-23T00:53:45.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.234+0000 7f51f90f1700 1 --2- 172.21.15.47:0/1116100076 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f51e0062cc0 0x7f51e0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:45.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.234+0000 7f51eaffd700 20 client.31605.objecter dump_active .. 0 homeless 2024-02-23T00:53:45.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.234+0000 7f51eaffd700 10 client.31605.objecter ms_handle_connect 0x7f51e0062cc0 2024-02-23T00:53:45.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.234+0000 7f51f90f1700 1 --2- 172.21.15.47:0/1116100076 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f51e0062cc0 0x7f51e0065180 secure :-1 s=READY pgs=3164 cs=0 l=1 rev1=1 rx=0x7f51f4109840 tx=0x7f51f000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:45.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.238+0000 7f51eaffd700 1 -- 172.21.15.47:0/1116100076 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f51e404de20 con 0x7f51f40f7320 2024-02-23T00:53:45.267 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:45.442 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-23T00:53:45.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.440+0000 7f51fbb56700 1 -- 172.21.15.47:0/1116100076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd reweight", "id": 1, "weight": 1.0} v 0) v1 -- 0x7f51f4105d20 con 0x7f51f40f7320 2024-02-23T00:53:45.584 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:44.325365+0000 mon.a (mon.0) 3384 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [3, 4]}]': finished 2024-02-23T00:53:45.584 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:44.325476+0000 mon.a (mon.0) 3385 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.6", "id": [5, 6]}]': finished 2024-02-23T00:53:45.584 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:44.325566+0000 mon.a (mon.0) 3386 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1c", "id": [4, 6]}]': finished 2024-02-23T00:53:45.584 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:44.325654+0000 mon.a (mon.0) 3387 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "5.10"}]': finished 2024-02-23T00:53:45.584 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:44.325711+0000 mon.a (mon.0) 3388 : cluster [DBG] osdmap e685: 8 total, 8 up, 6 in 2024-02-23T00:53:45.584 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:45.283657+0000 mon.a (mon.0) 3389 : cluster [DBG] osdmap e686: 8 total, 8 up, 6 in 2024-02-23T00:53:45.602 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.7 dump_ops_in_flight 2024-02-23T00:53:45.619 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:53:45.621 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:45.621 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:53:45.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.716+0000 7f2ee21bf700 1 -- 172.21.15.47:0/2363124062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2edc0f68a0 msgr2=0x7f2edc104a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:45.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.716+0000 7f2ee21bf700 1 --2- 172.21.15.47:0/2363124062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2edc0f68a0 0x7f2edc104a40 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 rx=0x7f2ed4009be0 tx=0x7f2ed4004d10).stop 2024-02-23T00:53:45.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.717+0000 7f2ee21bf700 1 -- 172.21.15.47:0/2363124062 shutdown_connections 2024-02-23T00:53:45.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.717+0000 7f2ee21bf700 1 --2- 172.21.15.47:0/2363124062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2edc0f68a0 0x7f2edc104a40 unknown :-1 s=CLOSED pgs=2615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:45.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.717+0000 7f2ee21bf700 1 --2- 172.21.15.47:0/2363124062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2edc0fdc40 0x7f2edc0fe020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:45.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.717+0000 7f2ee21bf700 1 --2- 172.21.15.47:0/2363124062 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2edc0fe5a0 0x7f2edc0f61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:45.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.717+0000 7f2ee21bf700 1 -- 172.21.15.47:0/2363124062 >> 172.21.15.47:0/2363124062 conn(0x7f2edc06ffc0 msgr2=0x7f2edc0f43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:45.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.717+0000 7f2ee21bf700 1 -- 172.21.15.47:0/2363124062 shutdown_connections 2024-02-23T00:53:45.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.717+0000 7f2ee21bf700 1 -- 172.21.15.47:0/2363124062 wait complete. 2024-02-23T00:53:45.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.717+0000 7f2ee21bf700 1 librados: starting msgr at 2024-02-23T00:53:45.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.717+0000 7f2ee21bf700 1 librados: starting objecter 2024-02-23T00:53:45.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.717+0000 7f2ee21bf700 1 Processor -- start 2024-02-23T00:53:45.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.717+0000 7f2ee21bf700 1 -- start start 2024-02-23T00:53:45.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.717+0000 7f2ee21bf700 1 librados: setting wanted keys 2024-02-23T00:53:45.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.717+0000 7f2ee21bf700 1 librados: calling monclient init 2024-02-23T00:53:45.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.718+0000 7f2ee21bf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2edc0f68a0 0x7f2edc10dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:45.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.718+0000 7f2ee21bf700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2edc0fdc40 0x7f2edc10e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:45.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.718+0000 7f2ee21bf700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2edc0fe5a0 0x7f2edc108040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:45.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.718+0000 7f2ee21bf700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc108530 con 0x7f2edc0f68a0 2024-02-23T00:53:45.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.718+0000 7f2ee21bf700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc1086a0 con 0x7f2edc0fe5a0 2024-02-23T00:53:45.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.718+0000 7f2ee21bf700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc108860 con 0x7f2edc0fdc40 2024-02-23T00:53:45.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.718+0000 7f2edb7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2edc0f68a0 0x7f2edc10dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:45.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.718+0000 7f2edaffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2edc0fdc40 0x7f2edc10e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:45.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.718+0000 7f2edb7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2edc0f68a0 0x7f2edc10dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54152/0 (socket says 172.21.15.47:54152) 2024-02-23T00:53:45.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.718+0000 7f2edaffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2edc0fdc40 0x7f2edc10e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:50378/0 (socket says 172.21.15.47:50378) 2024-02-23T00:53:45.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.718+0000 7f2edb7fe700 1 -- 172.21.15.47:0/4268187552 learned_addr learned my addr 172.21.15.47:0/4268187552 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:53:45.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.718+0000 7f2edbfff700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2edc0fe5a0 0x7f2edc108040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:45.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.718+0000 7f2edb7fe700 1 -- 172.21.15.47:0/4268187552 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2edc0fdc40 msgr2=0x7f2edc10e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:45.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.718+0000 7f2edb7fe700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2edc0fdc40 0x7f2edc10e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:45.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.718+0000 7f2edb7fe700 1 -- 172.21.15.47:0/4268187552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2edc0fe5a0 msgr2=0x7f2edc108040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:45.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.718+0000 7f2edb7fe700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2edc0fe5a0 0x7f2edc108040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:45.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.718+0000 7f2edb7fe700 1 -- 172.21.15.47:0/4268187552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed0005040 con 0x7f2edc0f68a0 2024-02-23T00:53:45.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.718+0000 7f2edaffd700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2edc0fdc40 0x7f2edc10e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:53:45.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.718+0000 7f2edb7fe700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2edc0f68a0 0x7f2edc10dea0 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 rx=0x7f2ecc00e940 tx=0x7f2ecc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:45.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.718+0000 7f2ed8ff9700 10 client.?.objecter ms_handle_connect 0x7f2edc0f68a0 2024-02-23T00:53:45.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.718+0000 7f2ed8ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:53:45.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.718+0000 7f2ed8ff9700 1 -- 172.21.15.47:0/4268187552 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc004660 con 0x7f2edc0f68a0 2024-02-23T00:53:45.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.718+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ed40097c0 con 0x7f2edc0f68a0 2024-02-23T00:53:45.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.718+0000 7f2ed8ff9700 1 -- 172.21.15.47:0/4268187552 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f2ecc00f040 con 0x7f2edc0f68a0 2024-02-23T00:53:45.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.718+0000 7f2ed8ff9700 1 -- 172.21.15.47:0/4268187552 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc0146f0 con 0x7f2edc0f68a0 2024-02-23T00:53:45.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.718+0000 7f2ee21bf700 10 client.31611.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:53:45.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.718+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2edc114cd0 con 0x7f2edc0f68a0 2024-02-23T00:53:45.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.718+0000 7f2ee21bf700 1 librados: init done 2024-02-23T00:53:45.724 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:53:45.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.719+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f2edc05f9b0 con 0x7f2edc0f68a0 2024-02-23T00:53:45.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.720+0000 7f2ed8ff9700 1 -- 172.21.15.47:0/4268187552 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f2ecc009240 con 0x7f2edc0f68a0 2024-02-23T00:53:45.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.720+0000 7f2ed8ff9700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2ec4062cc0 0x7f2ec4065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:45.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.720+0000 7f2ed8ff9700 1 -- 172.21.15.47:0/4268187552 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(686..686 src has 482..686) v4 ==== 9691+0+0 (secure 0 0 0) 0x7f2ecc07d300 con 0x7f2edc0f68a0 2024-02-23T00:53:45.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.720+0000 7f2ed8ff9700 10 client.31611.objecter ms_dispatch 0x7f2edc001550 osd_map(686..686 src has 482..686) v4 2024-02-23T00:53:45.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.720+0000 7f2ed8ff9700 3 client.31611.objecter handle_osd_map got epochs [686,686] > 0 2024-02-23T00:53:45.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.720+0000 7f2edaffd700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2ec4062cc0 0x7f2ec4065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:45.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.720+0000 7f2ed8ff9700 3 client.31611.objecter handle_osd_map decoding full epoch 686 2024-02-23T00:53:45.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.720+0000 7f2ed8ff9700 20 client.31611.objecter dump_active .. 0 homeless 2024-02-23T00:53:45.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.720+0000 7f2ed8ff9700 1 -- 172.21.15.47:0/4268187552 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v686) v1 ==== 54+0+16 (secure 0 0 0) 0x7f2ecc084070 con 0x7f2edc0f68a0 2024-02-23T00:53:45.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.721+0000 7f2ed8ff9700 10 client.31611.objecter ms_handle_connect 0x7f2ec4062cc0 2024-02-23T00:53:45.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.721+0000 7f2edaffd700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2ec4062cc0 0x7f2ec4065180 secure :-1 s=READY pgs=3165 cs=0 l=1 rev1=1 rx=0x7f2ed0009fd0 tx=0x7f2ed000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:45.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.721+0000 7f2ee21bf700 10 client.31611.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:45.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.721+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_assign -1 1 2024-02-23T00:53:45.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.721+0000 7f2ee21bf700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f2edc0feee0 0x7f2edc0ff2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:45.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.721+0000 7f2ee21bf700 20 client.31611.objecter _get_session s=0x7f2edc05ee80 osd=0 3 2024-02-23T00:53:45.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.721+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc05ee80 osd=0 3 2024-02-23T00:53:45.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.721+0000 7f2ee21bf700 20 client.31611.objecter _get_session s=0x7f2edc05ee80 osd=0 3 2024-02-23T00:53:45.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.721+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_remove -1 1 2024-02-23T00:53:45.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.721+0000 7f2ee21bf700 20 client.31611.objecter get_session s=0x7f2edc05ee80 osd=0 3 2024-02-23T00:53:45.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.721+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_assign 0 1 2024-02-23T00:53:45.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.721+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc05ee80 osd=0 4 2024-02-23T00:53:45.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.721+0000 7f2ee21bf700 10 client.31611.objecter _send_command 1 2024-02-23T00:53:45.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.721+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2edc106ef0 con 0x7f2edc0feee0 2024-02-23T00:53:45.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.721+0000 7f2edbfff700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f2edc0feee0 0x7f2edc0ff2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:45.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.721+0000 7f2ed8ff9700 10 client.31611.objecter ms_handle_connect 0x7f2edc0feee0 2024-02-23T00:53:45.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.721+0000 7f2edbfff700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f2edc0feee0 0x7f2edc0ff2c0 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:45.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.722+0000 7f2ed8ff9700 1 -- 172.21.15.47:0/4268187552 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2edc106ef0 con 0x7f2edc0feee0 2024-02-23T00:53:45.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.722+0000 7f2ed8ff9700 10 client.31611.objecter ms_dispatch 0x7f2edc001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:53:45.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.722+0000 7f2ed8ff9700 10 client.31611.objecter _finish_command 1 = system:0 2024-02-23T00:53:45.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.722+0000 7f2ed8ff9700 20 client.31611.objecter put_session s=0x7f2edc05ee80 osd=0 4 2024-02-23T00:53:45.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.722+0000 7f2ed8ff9700 15 client.31611.objecter _session_command_op_remove 0 1 2024-02-23T00:53:45.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.737+0000 7f2ee21bf700 10 client.31611.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:53:45.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.737+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_assign -1 2 2024-02-23T00:53:45.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.737+0000 7f2ee21bf700 20 client.31611.objecter _get_session s=0x7f2edc05ee80 osd=0 3 2024-02-23T00:53:45.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.737+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc05ee80 osd=0 3 2024-02-23T00:53:45.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.737+0000 7f2ee21bf700 20 client.31611.objecter _get_session s=0x7f2edc05ee80 osd=0 3 2024-02-23T00:53:45.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.737+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_remove -1 2 2024-02-23T00:53:45.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.737+0000 7f2ee21bf700 20 client.31611.objecter get_session s=0x7f2edc05ee80 osd=0 3 2024-02-23T00:53:45.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.737+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_assign 0 2 2024-02-23T00:53:45.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.737+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc05ee80 osd=0 4 2024-02-23T00:53:45.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.737+0000 7f2ee21bf700 10 client.31611.objecter _send_command 2 2024-02-23T00:53:45.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.737+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2edc0273c0 con 0x7f2edc0feee0 2024-02-23T00:53:45.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.739+0000 7f2ed8ff9700 1 -- 172.21.15.47:0/4268187552 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2edc0273c0 con 0x7f2edc0feee0 2024-02-23T00:53:45.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.739+0000 7f2ed8ff9700 10 client.31611.objecter ms_dispatch 0x7f2edc001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:53:45.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.739+0000 7f2ed8ff9700 10 client.31611.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:53:45.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.739+0000 7f2ed8ff9700 20 client.31611.objecter put_session s=0x7f2edc05ee80 osd=0 4 2024-02-23T00:53:45.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.739+0000 7f2ed8ff9700 15 client.31611.objecter _session_command_op_remove 0 2 2024-02-23T00:53:45.742 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:53:45.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.739+0000 7f2ee21bf700 10 client.31611.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:45.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.739+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_assign -1 3 2024-02-23T00:53:45.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.739+0000 7f2ee21bf700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f2edc0598f0 0x7f2edc059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:45.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.739+0000 7f2ee21bf700 20 client.31611.objecter _get_session s=0x7f2edc028710 osd=1 3 2024-02-23T00:53:45.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.739+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc028710 osd=1 3 2024-02-23T00:53:45.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.739+0000 7f2ee21bf700 20 client.31611.objecter _get_session s=0x7f2edc028710 osd=1 3 2024-02-23T00:53:45.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.739+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_remove -1 3 2024-02-23T00:53:45.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.739+0000 7f2ee21bf700 20 client.31611.objecter get_session s=0x7f2edc028710 osd=1 3 2024-02-23T00:53:45.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.739+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_assign 1 3 2024-02-23T00:53:45.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.739+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc028710 osd=1 4 2024-02-23T00:53:45.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.739+0000 7f2ee21bf700 10 client.31611.objecter _send_command 3 2024-02-23T00:53:45.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.739+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2edc11b5e0 con 0x7f2edc0598f0 2024-02-23T00:53:45.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.740+0000 7f2edb7fe700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f2edc0598f0 0x7f2edc059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:45.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.740+0000 7f2ed8ff9700 10 client.31611.objecter ms_handle_connect 0x7f2edc0598f0 2024-02-23T00:53:45.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.740+0000 7f2edb7fe700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f2edc0598f0 0x7f2edc059cd0 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:45.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.740+0000 7f2ed8ff9700 1 -- 172.21.15.47:0/4268187552 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2edc11b5e0 con 0x7f2edc0598f0 2024-02-23T00:53:45.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.740+0000 7f2ed8ff9700 10 client.31611.objecter ms_dispatch 0x7f2edc001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:53:45.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.740+0000 7f2ed8ff9700 10 client.31611.objecter _finish_command 3 = system:0 2024-02-23T00:53:45.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.740+0000 7f2ed8ff9700 20 client.31611.objecter put_session s=0x7f2edc028710 osd=1 4 2024-02-23T00:53:45.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.740+0000 7f2ed8ff9700 15 client.31611.objecter _session_command_op_remove 1 3 2024-02-23T00:53:45.751 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:45.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.766+0000 7f2ee21bf700 10 client.31611.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:53:45.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.766+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_assign -1 4 2024-02-23T00:53:45.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.766+0000 7f2ee21bf700 20 client.31611.objecter _get_session s=0x7f2edc028710 osd=1 3 2024-02-23T00:53:45.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.766+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc028710 osd=1 3 2024-02-23T00:53:45.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.766+0000 7f2ee21bf700 20 client.31611.objecter _get_session s=0x7f2edc028710 osd=1 3 2024-02-23T00:53:45.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.766+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_remove -1 4 2024-02-23T00:53:45.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.766+0000 7f2ee21bf700 20 client.31611.objecter get_session s=0x7f2edc028710 osd=1 3 2024-02-23T00:53:45.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.766+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_assign 1 4 2024-02-23T00:53:45.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.766+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc028710 osd=1 4 2024-02-23T00:53:45.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.766+0000 7f2ee21bf700 10 client.31611.objecter _send_command 4 2024-02-23T00:53:45.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.766+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2edc059360 con 0x7f2edc0598f0 2024-02-23T00:53:45.770 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:53:45.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.768+0000 7f2ed8ff9700 1 -- 172.21.15.47:0/4268187552 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2edc059360 con 0x7f2edc0598f0 2024-02-23T00:53:45.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.768+0000 7f2ed8ff9700 10 client.31611.objecter ms_dispatch 0x7f2edc001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:53:45.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.768+0000 7f2ed8ff9700 10 client.31611.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:53:45.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.768+0000 7f2ed8ff9700 20 client.31611.objecter put_session s=0x7f2edc028710 osd=1 4 2024-02-23T00:53:45.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.768+0000 7f2ed8ff9700 15 client.31611.objecter _session_command_op_remove 1 4 2024-02-23T00:53:45.771 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:53:45.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.768+0000 7f2ee21bf700 10 client.31611.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:45.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.768+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_assign -1 5 2024-02-23T00:53:45.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.768+0000 7f2ee21bf700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f2edc028270 0x7f2edc11a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:45.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.768+0000 7f2ee21bf700 20 client.31611.objecter _get_session s=0x7f2edc059770 osd=2 3 2024-02-23T00:53:45.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.768+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc059770 osd=2 3 2024-02-23T00:53:45.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.768+0000 7f2ee21bf700 20 client.31611.objecter _get_session s=0x7f2edc059770 osd=2 3 2024-02-23T00:53:45.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.768+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_remove -1 5 2024-02-23T00:53:45.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.768+0000 7f2ee21bf700 20 client.31611.objecter get_session s=0x7f2edc059770 osd=2 3 2024-02-23T00:53:45.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.768+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_assign 2 5 2024-02-23T00:53:45.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.768+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc059770 osd=2 4 2024-02-23T00:53:45.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.768+0000 7f2ee21bf700 10 client.31611.objecter _send_command 5 2024-02-23T00:53:45.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.768+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2edc11a4f0 con 0x7f2edc028270 2024-02-23T00:53:45.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.768+0000 7f2edaffd700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f2edc028270 0x7f2edc11a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:45.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.768+0000 7f2ed8ff9700 10 client.31611.objecter ms_handle_connect 0x7f2edc028270 2024-02-23T00:53:45.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.768+0000 7f2edaffd700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f2edc028270 0x7f2edc11a000 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:45.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.769+0000 7f2ed8ff9700 1 -- 172.21.15.47:0/4268187552 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2edc11a4f0 con 0x7f2edc028270 2024-02-23T00:53:45.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.769+0000 7f2ed8ff9700 10 client.31611.objecter ms_dispatch 0x7f2edc001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:53:45.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.769+0000 7f2ed8ff9700 10 client.31611.objecter _finish_command 5 = system:0 2024-02-23T00:53:45.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.769+0000 7f2ed8ff9700 20 client.31611.objecter put_session s=0x7f2edc059770 osd=2 4 2024-02-23T00:53:45.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.769+0000 7f2ed8ff9700 15 client.31611.objecter _session_command_op_remove 2 5 2024-02-23T00:53:45.774 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:45.775 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:53:45.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.784+0000 7f2ee21bf700 10 client.31611.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:53:45.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.784+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_assign -1 6 2024-02-23T00:53:45.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.784+0000 7f2ee21bf700 20 client.31611.objecter _get_session s=0x7f2edc059770 osd=2 3 2024-02-23T00:53:45.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.784+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc059770 osd=2 3 2024-02-23T00:53:45.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.784+0000 7f2ee21bf700 20 client.31611.objecter _get_session s=0x7f2edc059770 osd=2 3 2024-02-23T00:53:45.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.784+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_remove -1 6 2024-02-23T00:53:45.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.784+0000 7f2ee21bf700 20 client.31611.objecter get_session s=0x7f2edc059770 osd=2 3 2024-02-23T00:53:45.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.784+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_assign 2 6 2024-02-23T00:53:45.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.784+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc059770 osd=2 4 2024-02-23T00:53:45.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.784+0000 7f2ee21bf700 10 client.31611.objecter _send_command 6 2024-02-23T00:53:45.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.784+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2edc0277d0 con 0x7f2edc028270 2024-02-23T00:53:45.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.787+0000 7f2ed8ff9700 1 -- 172.21.15.47:0/4268187552 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2edc0277d0 con 0x7f2edc028270 2024-02-23T00:53:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.787+0000 7f2ed8ff9700 10 client.31611.objecter ms_dispatch 0x7f2edc001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:53:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.787+0000 7f2ed8ff9700 10 client.31611.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:53:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.787+0000 7f2ed8ff9700 20 client.31611.objecter put_session s=0x7f2edc059770 osd=2 4 2024-02-23T00:53:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.787+0000 7f2ed8ff9700 15 client.31611.objecter _session_command_op_remove 2 6 2024-02-23T00:53:45.790 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:53:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.787+0000 7f2ee21bf700 10 client.31611.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.787+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_assign -1 7 2024-02-23T00:53:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.787+0000 7f2ee21bf700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f2edc0f72d0 0x7f2edc0f7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.787+0000 7f2ee21bf700 20 client.31611.objecter _get_session s=0x7f2edc027b80 osd=3 3 2024-02-23T00:53:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.787+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc027b80 osd=3 3 2024-02-23T00:53:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.787+0000 7f2ee21bf700 20 client.31611.objecter _get_session s=0x7f2edc027b80 osd=3 3 2024-02-23T00:53:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.787+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_remove -1 7 2024-02-23T00:53:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.787+0000 7f2ee21bf700 20 client.31611.objecter get_session s=0x7f2edc027b80 osd=3 3 2024-02-23T00:53:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.787+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_assign 3 7 2024-02-23T00:53:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.787+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc027b80 osd=3 4 2024-02-23T00:53:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.787+0000 7f2ee21bf700 10 client.31611.objecter _send_command 7 2024-02-23T00:53:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.787+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2edc0f91e0 con 0x7f2edc0f72d0 2024-02-23T00:53:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.787+0000 7f2edbfff700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f2edc0f72d0 0x7f2edc0f7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.788+0000 7f2ed8ff9700 10 client.31611.objecter ms_handle_connect 0x7f2edc0f72d0 2024-02-23T00:53:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.788+0000 7f2edbfff700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f2edc0f72d0 0x7f2edc0f7710 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.788+0000 7f2ed8ff9700 1 -- 172.21.15.47:0/4268187552 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2edc0f91e0 con 0x7f2edc0f72d0 2024-02-23T00:53:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.788+0000 7f2ed8ff9700 10 client.31611.objecter ms_dispatch 0x7f2edc001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:53:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.788+0000 7f2ed8ff9700 10 client.31611.objecter _finish_command 7 = system:0 2024-02-23T00:53:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.788+0000 7f2ed8ff9700 20 client.31611.objecter put_session s=0x7f2edc027b80 osd=3 4 2024-02-23T00:53:45.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.788+0000 7f2ed8ff9700 15 client.31611.objecter _session_command_op_remove 3 7 2024-02-23T00:53:45.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:44.325365+0000 mon.a (mon.0) 3384 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [3, 4]}]': finished 2024-02-23T00:53:45.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:44.325476+0000 mon.a (mon.0) 3385 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.6", "id": [5, 6]}]': finished 2024-02-23T00:53:45.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:44.325566+0000 mon.a (mon.0) 3386 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1c", "id": [4, 6]}]': finished 2024-02-23T00:53:45.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:44.325654+0000 mon.a (mon.0) 3387 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "5.10"}]': finished 2024-02-23T00:53:45.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:44.325711+0000 mon.a (mon.0) 3388 : cluster [DBG] osdmap e685: 8 total, 8 up, 6 in 2024-02-23T00:53:45.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:45.283657+0000 mon.a (mon.0) 3389 : cluster [DBG] osdmap e686: 8 total, 8 up, 6 in 2024-02-23T00:53:45.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:44.325365+0000 mon.a (mon.0) 3384 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [3, 4]}]': finished 2024-02-23T00:53:45.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:44.325476+0000 mon.a (mon.0) 3385 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.6", "id": [5, 6]}]': finished 2024-02-23T00:53:45.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:44.325566+0000 mon.a (mon.0) 3386 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1c", "id": [4, 6]}]': finished 2024-02-23T00:53:45.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:44.325654+0000 mon.a (mon.0) 3387 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "5.10"}]': finished 2024-02-23T00:53:45.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:44.325711+0000 mon.a (mon.0) 3388 : cluster [DBG] osdmap e685: 8 total, 8 up, 6 in 2024-02-23T00:53:45.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:45.283657+0000 mon.a (mon.0) 3389 : cluster [DBG] osdmap e686: 8 total, 8 up, 6 in 2024-02-23T00:53:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.809+0000 7f2ee21bf700 10 client.31611.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:53:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.809+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_assign -1 8 2024-02-23T00:53:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.809+0000 7f2ee21bf700 20 client.31611.objecter _get_session s=0x7f2edc027b80 osd=3 3 2024-02-23T00:53:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.809+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc027b80 osd=3 3 2024-02-23T00:53:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.809+0000 7f2ee21bf700 20 client.31611.objecter _get_session s=0x7f2edc027b80 osd=3 3 2024-02-23T00:53:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.809+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_remove -1 8 2024-02-23T00:53:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.809+0000 7f2ee21bf700 20 client.31611.objecter get_session s=0x7f2edc027b80 osd=3 3 2024-02-23T00:53:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.809+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_assign 3 8 2024-02-23T00:53:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.809+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc027b80 osd=3 4 2024-02-23T00:53:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.809+0000 7f2ee21bf700 10 client.31611.objecter _send_command 8 2024-02-23T00:53:45.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.809+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2edc0f8d80 con 0x7f2edc0f72d0 2024-02-23T00:53:45.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.811+0000 7f2ed8ff9700 1 -- 172.21.15.47:0/4268187552 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2edc0f8d80 con 0x7f2edc0f72d0 2024-02-23T00:53:45.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.811+0000 7f2ed8ff9700 10 client.31611.objecter ms_dispatch 0x7f2edc001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:53:45.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.811+0000 7f2ed8ff9700 10 client.31611.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:53:45.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.811+0000 7f2ed8ff9700 20 client.31611.objecter put_session s=0x7f2edc027b80 osd=3 4 2024-02-23T00:53:45.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.811+0000 7f2ed8ff9700 15 client.31611.objecter _session_command_op_remove 3 8 2024-02-23T00:53:45.814 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:53:45.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.811+0000 7f2ee21bf700 10 client.31611.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:45.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.811+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_assign -1 9 2024-02-23T00:53:45.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.811+0000 7f2ee21bf700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2edc0f8280 0x7f2edc0f8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:45.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.811+0000 7f2ee21bf700 20 client.31611.objecter _get_session s=0x7f2edc0f7c00 osd=4 3 2024-02-23T00:53:45.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.811+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc0f7c00 osd=4 3 2024-02-23T00:53:45.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.811+0000 7f2ee21bf700 20 client.31611.objecter _get_session s=0x7f2edc0f7c00 osd=4 3 2024-02-23T00:53:45.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.811+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_remove -1 9 2024-02-23T00:53:45.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.811+0000 7f2ee21bf700 20 client.31611.objecter get_session s=0x7f2edc0f7c00 osd=4 3 2024-02-23T00:53:45.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.811+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_assign 4 9 2024-02-23T00:53:45.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.811+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc0f7c00 osd=4 4 2024-02-23T00:53:45.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.811+0000 7f2ee21bf700 10 client.31611.objecter _send_command 9 2024-02-23T00:53:45.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.811+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2edc0f8bf0 con 0x7f2edc0f8280 2024-02-23T00:53:45.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.812+0000 7f2edb7fe700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2edc0f8280 0x7f2edc0f8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:45.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.812+0000 7f2ed8ff9700 10 client.31611.objecter ms_handle_connect 0x7f2edc0f8280 2024-02-23T00:53:45.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.812+0000 7f2edb7fe700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2edc0f8280 0x7f2edc0f8700 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:45.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.813+0000 7f2ed8ff9700 1 -- 172.21.15.47:0/4268187552 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2edc0f8bf0 con 0x7f2edc0f8280 2024-02-23T00:53:45.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.813+0000 7f2ed8ff9700 10 client.31611.objecter ms_dispatch 0x7f2edc001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:53:45.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.813+0000 7f2ed8ff9700 10 client.31611.objecter _finish_command 9 = system:0 2024-02-23T00:53:45.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.813+0000 7f2ed8ff9700 20 client.31611.objecter put_session s=0x7f2edc0f7c00 osd=4 4 2024-02-23T00:53:45.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.813+0000 7f2ed8ff9700 15 client.31611.objecter _session_command_op_remove 4 9 2024-02-23T00:53:45.821 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:53:45.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.829+0000 7f2ee21bf700 10 client.31611.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:53:45.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.829+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_assign -1 10 2024-02-23T00:53:45.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.829+0000 7f2ee21bf700 20 client.31611.objecter _get_session s=0x7f2edc0f7c00 osd=4 3 2024-02-23T00:53:45.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.829+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc0f7c00 osd=4 3 2024-02-23T00:53:45.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.829+0000 7f2ee21bf700 20 client.31611.objecter _get_session s=0x7f2edc0f7c00 osd=4 3 2024-02-23T00:53:45.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.829+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_remove -1 10 2024-02-23T00:53:45.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.829+0000 7f2ee21bf700 20 client.31611.objecter get_session s=0x7f2edc0f7c00 osd=4 3 2024-02-23T00:53:45.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.829+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_assign 4 10 2024-02-23T00:53:45.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.829+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc0f7c00 osd=4 4 2024-02-23T00:53:45.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.829+0000 7f2ee21bf700 10 client.31611.objecter _send_command 10 2024-02-23T00:53:45.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.829+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2edc0f8bf0 con 0x7f2edc0f8280 2024-02-23T00:53:45.834 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:53:45.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.831+0000 7f2ed8ff9700 1 -- 172.21.15.47:0/4268187552 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2edc0f8bf0 con 0x7f2edc0f8280 2024-02-23T00:53:45.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.831+0000 7f2ed8ff9700 10 client.31611.objecter ms_dispatch 0x7f2edc001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:53:45.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.831+0000 7f2ed8ff9700 10 client.31611.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:53:45.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.831+0000 7f2ed8ff9700 20 client.31611.objecter put_session s=0x7f2edc0f7c00 osd=4 4 2024-02-23T00:53:45.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.831+0000 7f2ed8ff9700 15 client.31611.objecter _session_command_op_remove 4 10 2024-02-23T00:53:45.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.832+0000 7f2ee21bf700 10 client.31611.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:45.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.832+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_assign -1 11 2024-02-23T00:53:45.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.832+0000 7f2ee21bf700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f2edc1260a0 0x7f2edc128490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:45.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.832+0000 7f2ee21bf700 20 client.31611.objecter _get_session s=0x7f2edc0f8bf0 osd=5 3 2024-02-23T00:53:45.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.832+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc0f8bf0 osd=5 3 2024-02-23T00:53:45.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.832+0000 7f2ee21bf700 20 client.31611.objecter _get_session s=0x7f2edc0f8bf0 osd=5 3 2024-02-23T00:53:45.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.832+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_remove -1 11 2024-02-23T00:53:45.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.832+0000 7f2ee21bf700 20 client.31611.objecter get_session s=0x7f2edc0f8bf0 osd=5 3 2024-02-23T00:53:45.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.832+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_assign 5 11 2024-02-23T00:53:45.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.832+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc0f8bf0 osd=5 4 2024-02-23T00:53:45.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.832+0000 7f2ee21bf700 10 client.31611.objecter _send_command 11 2024-02-23T00:53:45.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.832+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2edc12b0e0 con 0x7f2edc1260a0 2024-02-23T00:53:45.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.832+0000 7f2edaffd700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f2edc1260a0 0x7f2edc128490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:45.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.832+0000 7f2ed8ff9700 10 client.31611.objecter ms_handle_connect 0x7f2edc1260a0 2024-02-23T00:53:45.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.832+0000 7f2edaffd700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f2edc1260a0 0x7f2edc128490 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:45.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.833+0000 7f2ed8ff9700 1 -- 172.21.15.47:0/4268187552 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2edc12b0e0 con 0x7f2edc1260a0 2024-02-23T00:53:45.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.833+0000 7f2ed8ff9700 10 client.31611.objecter ms_dispatch 0x7f2edc001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:53:45.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.833+0000 7f2ed8ff9700 10 client.31611.objecter _finish_command 11 = system:0 2024-02-23T00:53:45.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.833+0000 7f2ed8ff9700 20 client.31611.objecter put_session s=0x7f2edc0f8bf0 osd=5 4 2024-02-23T00:53:45.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.833+0000 7f2ed8ff9700 15 client.31611.objecter _session_command_op_remove 5 11 2024-02-23T00:53:45.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.848+0000 7f2ee21bf700 10 client.31611.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:53:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.849+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_assign -1 12 2024-02-23T00:53:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.849+0000 7f2ee21bf700 20 client.31611.objecter _get_session s=0x7f2edc0f8bf0 osd=5 3 2024-02-23T00:53:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.849+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc0f8bf0 osd=5 3 2024-02-23T00:53:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.849+0000 7f2ee21bf700 20 client.31611.objecter _get_session s=0x7f2edc0f8bf0 osd=5 3 2024-02-23T00:53:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.849+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_remove -1 12 2024-02-23T00:53:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.849+0000 7f2ee21bf700 20 client.31611.objecter get_session s=0x7f2edc0f8bf0 osd=5 3 2024-02-23T00:53:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.849+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_assign 5 12 2024-02-23T00:53:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.849+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc0f8bf0 osd=5 4 2024-02-23T00:53:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.849+0000 7f2ee21bf700 10 client.31611.objecter _send_command 12 2024-02-23T00:53:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.849+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2edc128980 con 0x7f2edc1260a0 2024-02-23T00:53:45.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.851+0000 7f2ed8ff9700 1 -- 172.21.15.47:0/4268187552 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2edc128980 con 0x7f2edc1260a0 2024-02-23T00:53:45.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.851+0000 7f2ed8ff9700 10 client.31611.objecter ms_dispatch 0x7f2edc001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:53:45.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.851+0000 7f2ed8ff9700 10 client.31611.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:53:45.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.851+0000 7f2ed8ff9700 20 client.31611.objecter put_session s=0x7f2edc0f8bf0 osd=5 4 2024-02-23T00:53:45.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.851+0000 7f2ed8ff9700 15 client.31611.objecter _session_command_op_remove 5 12 2024-02-23T00:53:45.854 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:53:45.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.851+0000 7f2ee21bf700 10 client.31611.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:45.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.851+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_assign -1 13 2024-02-23T00:53:45.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.851+0000 7f2ee21bf700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f2edc129000 0x7f2edc129440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:45.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.851+0000 7f2ee21bf700 20 client.31611.objecter _get_session s=0x7f2edc128980 osd=6 3 2024-02-23T00:53:45.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.851+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc128980 osd=6 3 2024-02-23T00:53:45.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.851+0000 7f2ee21bf700 20 client.31611.objecter _get_session s=0x7f2edc128980 osd=6 3 2024-02-23T00:53:45.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.851+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_remove -1 13 2024-02-23T00:53:45.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.851+0000 7f2ee21bf700 20 client.31611.objecter get_session s=0x7f2edc128980 osd=6 3 2024-02-23T00:53:45.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.851+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_assign 6 13 2024-02-23T00:53:45.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.851+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc128980 osd=6 4 2024-02-23T00:53:45.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.851+0000 7f2ee21bf700 10 client.31611.objecter _send_command 13 2024-02-23T00:53:45.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.851+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2edc12e460 con 0x7f2edc129000 2024-02-23T00:53:45.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.851+0000 7f2edbfff700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f2edc129000 0x7f2edc129440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:45.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.852+0000 7f2ed8ff9700 10 client.31611.objecter ms_handle_connect 0x7f2edc129000 2024-02-23T00:53:45.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.852+0000 7f2edbfff700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f2edc129000 0x7f2edc129440 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:45.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.852+0000 7f2ed8ff9700 1 -- 172.21.15.47:0/4268187552 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2edc12e460 con 0x7f2edc129000 2024-02-23T00:53:45.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.852+0000 7f2ed8ff9700 10 client.31611.objecter ms_dispatch 0x7f2edc001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:53:45.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.852+0000 7f2ed8ff9700 10 client.31611.objecter _finish_command 13 = system:0 2024-02-23T00:53:45.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.852+0000 7f2ed8ff9700 20 client.31611.objecter put_session s=0x7f2edc128980 osd=6 4 2024-02-23T00:53:45.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.852+0000 7f2ed8ff9700 15 client.31611.objecter _session_command_op_remove 6 13 2024-02-23T00:53:45.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.867+0000 7f2ee21bf700 10 client.31611.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:53:45.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.867+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_assign -1 14 2024-02-23T00:53:45.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.867+0000 7f2ee21bf700 20 client.31611.objecter _get_session s=0x7f2edc128980 osd=6 3 2024-02-23T00:53:45.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.867+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc128980 osd=6 3 2024-02-23T00:53:45.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.867+0000 7f2ee21bf700 20 client.31611.objecter _get_session s=0x7f2edc128980 osd=6 3 2024-02-23T00:53:45.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.867+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_remove -1 14 2024-02-23T00:53:45.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.867+0000 7f2ee21bf700 20 client.31611.objecter get_session s=0x7f2edc128980 osd=6 3 2024-02-23T00:53:45.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.867+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_assign 6 14 2024-02-23T00:53:45.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.867+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc128980 osd=6 4 2024-02-23T00:53:45.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.867+0000 7f2ee21bf700 10 client.31611.objecter _send_command 14 2024-02-23T00:53:45.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.867+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2edc129a40 con 0x7f2edc129000 2024-02-23T00:53:45.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.869+0000 7f2ed8ff9700 1 -- 172.21.15.47:0/4268187552 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2edc129a40 con 0x7f2edc129000 2024-02-23T00:53:45.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.869+0000 7f2ed8ff9700 10 client.31611.objecter ms_dispatch 0x7f2edc001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:53:45.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.869+0000 7f2ed8ff9700 10 client.31611.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:53:45.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.869+0000 7f2ed8ff9700 20 client.31611.objecter put_session s=0x7f2edc128980 osd=6 4 2024-02-23T00:53:45.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.869+0000 7f2ed8ff9700 15 client.31611.objecter _session_command_op_remove 6 14 2024-02-23T00:53:45.872 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:53:45.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.869+0000 7f2ee21bf700 10 client.31611.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:45.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.869+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_assign -1 15 2024-02-23T00:53:45.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.869+0000 7f2ee21bf700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f2edc12a0c0 0x7f2edc12a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:45.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.869+0000 7f2ee21bf700 20 client.31611.objecter _get_session s=0x7f2edc129a40 osd=7 3 2024-02-23T00:53:45.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.869+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc129a40 osd=7 3 2024-02-23T00:53:45.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.869+0000 7f2ee21bf700 20 client.31611.objecter _get_session s=0x7f2edc129a40 osd=7 3 2024-02-23T00:53:45.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.869+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_remove -1 15 2024-02-23T00:53:45.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.869+0000 7f2ee21bf700 20 client.31611.objecter get_session s=0x7f2edc129a40 osd=7 3 2024-02-23T00:53:45.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.869+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_assign 7 15 2024-02-23T00:53:45.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.869+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc129a40 osd=7 4 2024-02-23T00:53:45.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.869+0000 7f2ee21bf700 10 client.31611.objecter _send_command 15 2024-02-23T00:53:45.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.869+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2edc12ab60 con 0x7f2edc12a0c0 2024-02-23T00:53:45.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.870+0000 7f2edb7fe700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f2edc12a0c0 0x7f2edc12a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:45.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.870+0000 7f2ed8ff9700 10 client.31611.objecter ms_handle_connect 0x7f2edc12a0c0 2024-02-23T00:53:45.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.870+0000 7f2edb7fe700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f2edc12a0c0 0x7f2edc12a520 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:45.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.871+0000 7f2ed8ff9700 1 -- 172.21.15.47:0/4268187552 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2edc12ab60 con 0x7f2edc12a0c0 2024-02-23T00:53:45.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.871+0000 7f2ed8ff9700 10 client.31611.objecter ms_dispatch 0x7f2edc001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:53:45.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.871+0000 7f2ed8ff9700 10 client.31611.objecter _finish_command 15 = system:0 2024-02-23T00:53:45.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.871+0000 7f2ed8ff9700 20 client.31611.objecter put_session s=0x7f2edc129a40 osd=7 4 2024-02-23T00:53:45.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.871+0000 7f2ed8ff9700 15 client.31611.objecter _session_command_op_remove 7 15 2024-02-23T00:53:45.875 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:53:45.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.885+0000 7f2ee21bf700 10 client.31611.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:53:45.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.885+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_assign -1 16 2024-02-23T00:53:45.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.885+0000 7f2ee21bf700 20 client.31611.objecter _get_session s=0x7f2edc129a40 osd=7 3 2024-02-23T00:53:45.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.885+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc129a40 osd=7 3 2024-02-23T00:53:45.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.885+0000 7f2ee21bf700 20 client.31611.objecter _get_session s=0x7f2edc129a40 osd=7 3 2024-02-23T00:53:45.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.885+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_remove -1 16 2024-02-23T00:53:45.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.885+0000 7f2ee21bf700 20 client.31611.objecter get_session s=0x7f2edc129a40 osd=7 3 2024-02-23T00:53:45.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.885+0000 7f2ee21bf700 15 client.31611.objecter _session_command_op_assign 7 16 2024-02-23T00:53:45.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.885+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc129a40 osd=7 4 2024-02-23T00:53:45.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.885+0000 7f2ee21bf700 10 client.31611.objecter _send_command 16 2024-02-23T00:53:45.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.885+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2edc12ab60 con 0x7f2edc12a0c0 2024-02-23T00:53:45.889 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:53:45.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ed8ff9700 1 -- 172.21.15.47:0/4268187552 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2edc12ab60 con 0x7f2edc12a0c0 2024-02-23T00:53:45.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ed8ff9700 10 client.31611.objecter ms_dispatch 0x7f2edc001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:53:45.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ed8ff9700 10 client.31611.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:53:45.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ed8ff9700 20 client.31611.objecter put_session s=0x7f2edc129a40 osd=7 4 2024-02-23T00:53:45.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ed8ff9700 15 client.31611.objecter _session_command_op_remove 7 16 2024-02-23T00:53:45.890 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:53:45.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 10 librados: watch_flush enter 2024-02-23T00:53:45.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 10 librados: watch_flush exit 2024-02-23T00:53:45.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 10 client.31611.objecter close_session for osd.0 2024-02-23T00:53:45.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f2edc0feee0 msgr2=0x7f2edc0ff2c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:45.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f2edc0feee0 0x7f2edc0ff2c0 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:45.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc05ee80 osd=0 1 2024-02-23T00:53:45.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 10 client.31611.objecter close_session for osd.1 2024-02-23T00:53:45.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f2edc0598f0 msgr2=0x7f2edc059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:45.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f2edc0598f0 0x7f2edc059cd0 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:45.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc028710 osd=1 1 2024-02-23T00:53:45.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 10 client.31611.objecter close_session for osd.2 2024-02-23T00:53:45.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f2edc028270 msgr2=0x7f2edc11a000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:45.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f2edc028270 0x7f2edc11a000 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc059770 osd=2 1 2024-02-23T00:53:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 10 client.31611.objecter close_session for osd.3 2024-02-23T00:53:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f2edc0f72d0 msgr2=0x7f2edc0f7710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f2edc0f72d0 0x7f2edc0f7710 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2edbfff700 1 -- 172.21.15.47:0/4268187552 reap_dead start 2024-02-23T00:53:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc027b80 osd=3 1 2024-02-23T00:53:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 10 client.31611.objecter close_session for osd.4 2024-02-23T00:53:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2edc0f8280 msgr2=0x7f2edc0f8700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2edc0f8280 0x7f2edc0f8700 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2edbfff700 1 -- 172.21.15.47:0/4268187552 reap_dead start 2024-02-23T00:53:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc0f7c00 osd=4 1 2024-02-23T00:53:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 10 client.31611.objecter close_session for osd.5 2024-02-23T00:53:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f2edc1260a0 msgr2=0x7f2edc128490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f2edc1260a0 0x7f2edc128490 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:45.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc0f8bf0 osd=5 1 2024-02-23T00:53:45.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 10 client.31611.objecter close_session for osd.6 2024-02-23T00:53:45.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f2edc129000 msgr2=0x7f2edc129440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:45.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f2edc129000 0x7f2edc129440 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:45.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc128980 osd=6 1 2024-02-23T00:53:45.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 10 client.31611.objecter close_session for osd.7 2024-02-23T00:53:45.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f2edc12a0c0 msgr2=0x7f2edc12a520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:45.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f2edc12a0c0 0x7f2edc12a520 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:45.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.887+0000 7f2ee21bf700 20 client.31611.objecter put_session s=0x7f2edc129a40 osd=7 1 2024-02-23T00:53:45.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.888+0000 7f2ee21bf700 20 client.31611.objecter shutdown clearing up homeless session... 2024-02-23T00:53:45.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.888+0000 7f2ee21bf700 10 client.31611.objecter successfully canceled tick 2024-02-23T00:53:45.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.888+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2ec4062cc0 msgr2=0x7f2ec4065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:45.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.888+0000 7f2ee21bf700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2ec4062cc0 0x7f2ec4065180 secure :-1 s=READY pgs=3165 cs=0 l=1 rev1=1 rx=0x7f2ed0009fd0 tx=0x7f2ed000b040).stop 2024-02-23T00:53:45.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.888+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2edc0f68a0 msgr2=0x7f2edc10dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:45.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.888+0000 7f2ee21bf700 1 --2- 172.21.15.47:0/4268187552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2edc0f68a0 0x7f2edc10dea0 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 rx=0x7f2ecc00e940 tx=0x7f2ecc00b040).stop 2024-02-23T00:53:45.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.888+0000 7f2edbfff700 1 -- 172.21.15.47:0/4268187552 reap_dead start 2024-02-23T00:53:45.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.888+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 shutdown_connections 2024-02-23T00:53:45.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.888+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 >> 172.21.15.47:0/4268187552 conn(0x7f2edc06ffc0 msgr2=0x7f2edc06d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:45.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.888+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 shutdown_connections 2024-02-23T00:53:45.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.888+0000 7f2ee21bf700 1 -- 172.21.15.47:0/4268187552 wait complete. 2024-02-23T00:53:45.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:45.888+0000 7f2ee21bf700 1 librados: shutdown 2024-02-23T00:53:45.895 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:53:45.895 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:53:45.895 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:53:45.895 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:53:45.895 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:53:45.895 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:53:45.895 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:53:45.895 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:53:45.903 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:45.925 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:53:45.928 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:45.928 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:53:46.029 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:53:46.059 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:46.158 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:53:46.161 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:46.161 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:53:46.262 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:53:46.291 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:46.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:46.355+0000 7f51eaffd700 1 -- 172.21.15.47:0/1116100076 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 1, "weight": 1.0}]=0 reweighted osd.1 to 1 (10000) v687) v1 ==== 113+0+0 (secure 0 0 0) 0x7f51e4051440 con 0x7f51f40f7320 2024-02-23T00:53:46.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:46.356+0000 7f51fbb56700 10 librados: watch_flush enter 2024-02-23T00:53:46.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:46.356+0000 7f51fbb56700 10 librados: watch_flush exit 2024-02-23T00:53:46.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:46.356+0000 7f51fbb56700 20 client.31605.objecter shutdown clearing up homeless session... 2024-02-23T00:53:46.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:46.356+0000 7f51fbb56700 10 client.31605.objecter successfully canceled tick 2024-02-23T00:53:46.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:46.356+0000 7f51fbb56700 1 -- 172.21.15.47:0/1116100076 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f51e0062cc0 msgr2=0x7f51e0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:46.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:46.356+0000 7f51fbb56700 1 --2- 172.21.15.47:0/1116100076 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f51e0062cc0 0x7f51e0065180 secure :-1 s=READY pgs=3164 cs=0 l=1 rev1=1 rx=0x7f51f4109840 tx=0x7f51f000b040).stop 2024-02-23T00:53:46.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:46.357+0000 7f51fbb56700 1 -- 172.21.15.47:0/1116100076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51f40f7320 msgr2=0x7f51f410df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:46.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:46.357+0000 7f51fbb56700 1 --2- 172.21.15.47:0/1116100076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51f40f7320 0x7f51f410df00 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 rx=0x7f51e4009da0 tx=0x7f51e4009cb0).stop 2024-02-23T00:53:46.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:46.357+0000 7f51fbb56700 1 -- 172.21.15.47:0/1116100076 shutdown_connections 2024-02-23T00:53:46.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:46.357+0000 7f51fbb56700 1 --2- 172.21.15.47:0/1116100076 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f51e0062cc0 0x7f51e0065180 unknown :-1 s=CLOSED pgs=3164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:46.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:46.357+0000 7f51fbb56700 1 --2- 172.21.15.47:0/1116100076 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f51f40fd410 0x7f51f41080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:46.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:46.357+0000 7f51fbb56700 1 --2- 172.21.15.47:0/1116100076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51f40f7320 0x7f51f410df00 unknown :-1 s=CLOSED pgs=2614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:46.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:46.357+0000 7f51fbb56700 1 --2- 172.21.15.47:0/1116100076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51f40fcab0 0x7f51f410e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:46.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:46.357+0000 7f51fbb56700 1 -- 172.21.15.47:0/1116100076 >> 172.21.15.47:0/1116100076 conn(0x7f51f40f26e0 msgr2=0x7f51f40f41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:46.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:46.357+0000 7f51fbb56700 1 -- 172.21.15.47:0/1116100076 shutdown_connections 2024-02-23T00:53:46.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:46.357+0000 7f51fbb56700 1 -- 172.21.15.47:0/1116100076 wait complete. 2024-02-23T00:53:46.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:46.357+0000 7f51fbb56700 1 librados: shutdown 2024-02-23T00:53:46.361 INFO:teuthology.orchestra.run.smithi047.stderr:reweighted osd.1 to 1 (10000) 2024-02-23T00:53:46.388 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:53:46.390 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:46.391 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:53:46.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:45.366004+0000 mon.c (mon.2) 397 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:46.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:45.366315+0000 mon.a (mon.0) 3390 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:46.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:45.376393+0000 mgr.y (mgr.14517) 596 : cluster [DBG] pgmap v1038: 184 pgs: 184 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 403 B/s wr, 1 op/s 2024-02-23T00:53:46.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:45.441602+0000 mon.a (mon.0) 3391 : audit [INF] from='client.? 172.21.15.47:0/1116100076' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-23T00:53:46.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:45.721542+0000 mon.a (mon.0) 3392 : audit [DBG] from='client.? 172.21.15.47:0/4268187552' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:53:46.492 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:53:46.518 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:45.366004+0000 mon.c (mon.2) 397 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:46.518 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:45.366315+0000 mon.a (mon.0) 3390 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:46.518 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:45.376393+0000 mgr.y (mgr.14517) 596 : cluster [DBG] pgmap v1038: 184 pgs: 184 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 403 B/s wr, 1 op/s 2024-02-23T00:53:46.519 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:45.441602+0000 mon.a (mon.0) 3391 : audit [INF] from='client.? 172.21.15.47:0/1116100076' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-23T00:53:46.519 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:45.721542+0000 mon.a (mon.0) 3392 : audit [DBG] from='client.? 172.21.15.47:0/4268187552' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:53:46.519 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:45.366004+0000 mon.c (mon.2) 397 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:46.519 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:45.366315+0000 mon.a (mon.0) 3390 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:46.519 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:45.376393+0000 mgr.y (mgr.14517) 596 : cluster [DBG] pgmap v1038: 184 pgs: 184 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 403 B/s wr, 1 op/s 2024-02-23T00:53:46.519 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:45.441602+0000 mon.a (mon.0) 3391 : audit [INF] from='client.? 172.21.15.47:0/1116100076' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-23T00:53:46.519 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:45.721542+0000 mon.a (mon.0) 3392 : audit [DBG] from='client.? 172.21.15.47:0/4268187552' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:53:46.520 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:47.511 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:53:47.511 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-02-23T00:53:47.512 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 0 2024-02-23T00:53:47.512 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:53:47.538 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:46.283059+0000 osd.7 (osd.7) 325 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:53:47.538 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:46.283433+0000 osd.7 (osd.7) 326 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:53:47.538 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:46.351455+0000 mon.a (mon.0) 3393 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:53:47.538 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:46.351903+0000 mon.a (mon.0) 3394 : audit [INF] from='client.? 172.21.15.47:0/1116100076' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-23T00:53:47.538 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:46.351958+0000 mon.a (mon.0) 3395 : cluster [DBG] osdmap e687: 8 total, 8 up, 6 in 2024-02-23T00:53:47.538 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:46.352966+0000 mon.c (mon.2) 398 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-118"}]: dispatch 2024-02-23T00:53:47.538 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:46.353279+0000 mon.a (mon.0) 3396 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-118"}]: dispatch 2024-02-23T00:53:47.538 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:46.621539+0000 osd.0 (osd.0) 279 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:53:47.538 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:46.621717+0000 osd.0 (osd.0) 280 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:53:47.756 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:46.283059+0000 osd.7 (osd.7) 325 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:53:47.756 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:46.283433+0000 osd.7 (osd.7) 326 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:53:47.756 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:46.351455+0000 mon.a (mon.0) 3393 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:53:47.756 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:46.351903+0000 mon.a (mon.0) 3394 : audit [INF] from='client.? 172.21.15.47:0/1116100076' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-23T00:53:47.756 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:46.351958+0000 mon.a (mon.0) 3395 : cluster [DBG] osdmap e687: 8 total, 8 up, 6 in 2024-02-23T00:53:47.756 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:46.352966+0000 mon.c (mon.2) 398 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-118"}]: dispatch 2024-02-23T00:53:47.756 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:46.353279+0000 mon.a (mon.0) 3396 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-118"}]: dispatch 2024-02-23T00:53:47.756 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:46.621539+0000 osd.0 (osd.0) 279 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:53:47.756 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:46.621717+0000 osd.0 (osd.0) 280 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:53:47.757 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:46.283059+0000 osd.7 (osd.7) 325 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:53:47.757 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:46.283433+0000 osd.7 (osd.7) 326 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:53:47.757 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:46.351455+0000 mon.a (mon.0) 3393 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:53:47.757 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:46.351903+0000 mon.a (mon.0) 3394 : audit [INF] from='client.? 172.21.15.47:0/1116100076' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-23T00:53:47.757 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:46.351958+0000 mon.a (mon.0) 3395 : cluster [DBG] osdmap e687: 8 total, 8 up, 6 in 2024-02-23T00:53:47.757 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:46.352966+0000 mon.c (mon.2) 398 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-118"}]: dispatch 2024-02-23T00:53:47.757 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:46.353279+0000 mon.a (mon.0) 3396 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-118"}]: dispatch 2024-02-23T00:53:47.757 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:46.621539+0000 osd.0 (osd.0) 279 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:53:47.757 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:46.621717+0000 osd.0 (osd.0) 280 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:53:47.770 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:53:47.773 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:47.774 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:53:47.777 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight 3 1 2024-02-23T00:53:47.875 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:53:47.904 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:47.943 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:53:47.952 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:47.952 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:53:48.053 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:53:48.083 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:48.207 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:53:48.209 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:48.210 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:53:48.224 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.7 dump_blocked_ops 2024-02-23T00:53:48.311 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:53:48.340 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:48.360 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:53:48.366 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:48.367 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:53:48.468 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:53:48.505 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:48.530 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:53:48.532 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:47.349025+0000 mon.a (mon.0) 3397 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:53:48.532 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:47.353952+0000 mon.a (mon.0) 3398 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-118"}]': finished 2024-02-23T00:53:48.532 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:47.354036+0000 mon.a (mon.0) 3399 : cluster [DBG] osdmap e688: 8 total, 8 up, 6 in 2024-02-23T00:53:48.532 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:47.376754+0000 mgr.y (mgr.14517) 597 : cluster [DBG] pgmap v1041: 184 pgs: 184 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:53:48.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:47.349025+0000 mon.a (mon.0) 3397 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:53:48.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:47.353952+0000 mon.a (mon.0) 3398 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-118"}]': finished 2024-02-23T00:53:48.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:47.354036+0000 mon.a (mon.0) 3399 : cluster [DBG] osdmap e688: 8 total, 8 up, 6 in 2024-02-23T00:53:48.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:47.376754+0000 mgr.y (mgr.14517) 597 : cluster [DBG] pgmap v1041: 184 pgs: 184 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:53:48.533 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:48.533 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:53:48.540 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:53:48.634 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:53:48.663 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:48.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:47.349025+0000 mon.a (mon.0) 3397 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:53:48.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:47.353952+0000 mon.a (mon.0) 3398 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-118"}]': finished 2024-02-23T00:53:48.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:47.354036+0000 mon.a (mon.0) 3399 : cluster [DBG] osdmap e688: 8 total, 8 up, 6 in 2024-02-23T00:53:48.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:47.376754+0000 mgr.y (mgr.14517) 597 : cluster [DBG] pgmap v1041: 184 pgs: 184 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:53:48.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.726+0000 7f251c50f700 1 -- 172.21.15.47:0/811470738 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f25140f8d10 msgr2=0x7f25140f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:48.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.726+0000 7f251c50f700 1 --2- 172.21.15.47:0/811470738 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f25140f8d10 0x7f25140f90f0 secure :-1 s=READY pgs=2775 cs=0 l=1 rev1=1 rx=0x7f2510004870 tx=0x7f2510009cb0).stop 2024-02-23T00:53:48.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.726+0000 7f251c50f700 1 -- 172.21.15.47:0/811470738 shutdown_connections 2024-02-23T00:53:48.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.726+0000 7f251c50f700 1 --2- 172.21.15.47:0/811470738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2514102270 0x7f2514104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:48.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.726+0000 7f251c50f700 1 --2- 172.21.15.47:0/811470738 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f25140f8d10 0x7f25140f90f0 unknown :-1 s=CLOSED pgs=2775 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:48.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.726+0000 7f251c50f700 1 --2- 172.21.15.47:0/811470738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25140f9670 0x7f2514101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:48.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.726+0000 7f251c50f700 1 -- 172.21.15.47:0/811470738 >> 172.21.15.47:0/811470738 conn(0x7f25140f2820 msgr2=0x7f25140f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:48.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.726+0000 7f251c50f700 1 -- 172.21.15.47:0/811470738 shutdown_connections 2024-02-23T00:53:48.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.727+0000 7f251c50f700 1 -- 172.21.15.47:0/811470738 wait complete. 2024-02-23T00:53:48.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.727+0000 7f251c50f700 1 librados: starting msgr at 2024-02-23T00:53:48.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.727+0000 7f251c50f700 1 librados: starting objecter 2024-02-23T00:53:48.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.727+0000 7f251c50f700 1 Processor -- start 2024-02-23T00:53:48.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.727+0000 7f251c50f700 1 -- start start 2024-02-23T00:53:48.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.727+0000 7f251c50f700 1 librados: setting wanted keys 2024-02-23T00:53:48.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.727+0000 7f251c50f700 1 librados: calling monclient init 2024-02-23T00:53:48.731 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:53:48.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.728+0000 7f251c50f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25140f8d10 0x7f25141102c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:48.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.728+0000 7f251c50f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25140f9670 0x7f25141107b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:48.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.728+0000 7f251c50f700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2514102270 0x7f251410a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:48.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.728+0000 7f251c50f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251410a950 con 0x7f25140f9670 2024-02-23T00:53:48.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.728+0000 7f251c50f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251410aac0 con 0x7f25140f8d10 2024-02-23T00:53:48.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.728+0000 7f251c50f700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251410ac80 con 0x7f2514102270 2024-02-23T00:53:48.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.728+0000 7f2519aaa700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25140f9670 0x7f25141107b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:48.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.728+0000 7f251aaac700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2514102270 0x7f251410a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:48.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.728+0000 7f2519aaa700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25140f9670 0x7f25141107b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49172/0 (socket says 172.21.15.47:49172) 2024-02-23T00:53:48.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.728+0000 7f251aaac700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2514102270 0x7f251410a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:53284/0 (socket says 172.21.15.47:53284) 2024-02-23T00:53:48.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.728+0000 7f2519aaa700 1 -- 172.21.15.47:0/3567481411 learned_addr learned my addr 172.21.15.47:0/3567481411 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:53:48.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.728+0000 7f251a2ab700 1 --2- 172.21.15.47:0/3567481411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25140f8d10 0x7f25141102c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:48.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.728+0000 7f251aaac700 1 -- 172.21.15.47:0/3567481411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25140f8d10 msgr2=0x7f25141102c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:48.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.728+0000 7f251aaac700 1 --2- 172.21.15.47:0/3567481411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25140f8d10 0x7f25141102c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:48.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.728+0000 7f251aaac700 1 -- 172.21.15.47:0/3567481411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25140f9670 msgr2=0x7f25141107b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:48.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.728+0000 7f251aaac700 1 --2- 172.21.15.47:0/3567481411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25140f9670 0x7f25141107b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:48.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.728+0000 7f251aaac700 1 -- 172.21.15.47:0/3567481411 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2510005040 con 0x7f2514102270 2024-02-23T00:53:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.728+0000 7f251aaac700 1 --2- 172.21.15.47:0/3567481411 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2514102270 0x7f251410a460 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 rx=0x7f2514061e90 tx=0x7f250c004e30).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.728+0000 7f250b7fe700 10 client.?.objecter ms_handle_connect 0x7f2514102270 2024-02-23T00:53:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.728+0000 7f250b7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:53:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.728+0000 7f250b7fe700 1 -- 172.21.15.47:0/3567481411 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c00b4f0 con 0x7f2514102270 2024-02-23T00:53:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.729+0000 7f251c50f700 1 -- 172.21.15.47:0/3567481411 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f251410af60 con 0x7f2514102270 2024-02-23T00:53:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.729+0000 7f250b7fe700 1 -- 172.21.15.47:0/3567481411 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f250c00f040 con 0x7f2514102270 2024-02-23T00:53:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.729+0000 7f250b7fe700 1 -- 172.21.15.47:0/3567481411 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c013890 con 0x7f2514102270 2024-02-23T00:53:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.729+0000 7f251c50f700 10 client.41939.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:53:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.729+0000 7f251c50f700 1 -- 172.21.15.47:0/3567481411 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f251410b2e0 con 0x7f2514102270 2024-02-23T00:53:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.729+0000 7f251c50f700 1 librados: init done 2024-02-23T00:53:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.730+0000 7f251c50f700 1 -- 172.21.15.47:0/3567481411 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f251405fa60 con 0x7f2514102270 2024-02-23T00:53:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.730+0000 7f250b7fe700 1 -- 172.21.15.47:0/3567481411 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f250c013cc0 con 0x7f2514102270 2024-02-23T00:53:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.730+0000 7f250b7fe700 1 --2- 172.21.15.47:0/3567481411 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2500062cc0 0x7f2500065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.730+0000 7f250b7fe700 1 -- 172.21.15.47:0/3567481411 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(689..689 src has 482..689) v4 ==== 9310+0+0 (secure 0 0 0) 0x7f250c07d190 con 0x7f2514102270 2024-02-23T00:53:48.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.730+0000 7f250b7fe700 10 client.41939.objecter ms_dispatch 0x7f2514001550 osd_map(689..689 src has 482..689) v4 2024-02-23T00:53:48.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.730+0000 7f250b7fe700 3 client.41939.objecter handle_osd_map got epochs [689,689] > 0 2024-02-23T00:53:48.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.730+0000 7f250b7fe700 3 client.41939.objecter handle_osd_map decoding full epoch 689 2024-02-23T00:53:48.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.730+0000 7f251a2ab700 1 --2- 172.21.15.47:0/3567481411 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2500062cc0 0x7f2500065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:48.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.731+0000 7f250b7fe700 20 client.41939.objecter dump_active .. 0 homeless 2024-02-23T00:53:48.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.731+0000 7f250b7fe700 10 client.41939.objecter ms_handle_connect 0x7f2500062cc0 2024-02-23T00:53:48.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.731+0000 7f251a2ab700 1 --2- 172.21.15.47:0/3567481411 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2500062cc0 0x7f2500065180 secure :-1 s=READY pgs=3166 cs=0 l=1 rev1=1 rx=0x7f2510004ee0 tx=0x7f251000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:48.737 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:53:48.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.736+0000 7f250b7fe700 1 -- 172.21.15.47:0/3567481411 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f250c04c800 con 0x7f2514102270 2024-02-23T00:53:48.745 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:48.746 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:53:48.847 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:53:48.875 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:48.929 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:53:48.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:48.937+0000 7f251c50f700 1 -- 172.21.15.47:0/3567481411 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7f2514107060 con 0x7f2514102270 2024-02-23T00:53:48.972 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:53:48.975 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:48.975 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:53:49.076 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:53:49.103 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:49.351 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:53:49.354 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:49.354 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:53:49.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:49.371+0000 7f250b7fe700 1 -- 172.21.15.47:0/3567481411 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v690) v1 ==== 97+0+0 (secure 0 0 0) 0x7f250c04fe20 con 0x7f2514102270 2024-02-23T00:53:49.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:49.374+0000 7f251c50f700 10 librados: watch_flush enter 2024-02-23T00:53:49.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:49.374+0000 7f251c50f700 10 librados: watch_flush exit 2024-02-23T00:53:49.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:49.374+0000 7f251c50f700 20 client.41939.objecter shutdown clearing up homeless session... 2024-02-23T00:53:49.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:49.374+0000 7f251c50f700 10 client.41939.objecter successfully canceled tick 2024-02-23T00:53:49.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:49.374+0000 7f251c50f700 1 -- 172.21.15.47:0/3567481411 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2500062cc0 msgr2=0x7f2500065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:49.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:49.374+0000 7f251c50f700 1 --2- 172.21.15.47:0/3567481411 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2500062cc0 0x7f2500065180 secure :-1 s=READY pgs=3166 cs=0 l=1 rev1=1 rx=0x7f2510004ee0 tx=0x7f251000b040).stop 2024-02-23T00:53:49.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:49.374+0000 7f251c50f700 1 -- 172.21.15.47:0/3567481411 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2514102270 msgr2=0x7f251410a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:49.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:49.374+0000 7f251c50f700 1 --2- 172.21.15.47:0/3567481411 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2514102270 0x7f251410a460 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 rx=0x7f2514061e90 tx=0x7f250c004e30).stop 2024-02-23T00:53:49.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:49.374+0000 7f251c50f700 1 -- 172.21.15.47:0/3567481411 shutdown_connections 2024-02-23T00:53:49.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:49.374+0000 7f251c50f700 1 --2- 172.21.15.47:0/3567481411 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2500062cc0 0x7f2500065180 unknown :-1 s=CLOSED pgs=3166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:49.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:49.374+0000 7f251c50f700 1 --2- 172.21.15.47:0/3567481411 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2514102270 0x7f251410a460 unknown :-1 s=CLOSED pgs=2776 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:49.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:49.374+0000 7f251c50f700 1 --2- 172.21.15.47:0/3567481411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25140f8d10 0x7f25141102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:49.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:49.374+0000 7f251c50f700 1 --2- 172.21.15.47:0/3567481411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25140f9670 0x7f25141107b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:49.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:49.374+0000 7f251c50f700 1 -- 172.21.15.47:0/3567481411 >> 172.21.15.47:0/3567481411 conn(0x7f25140f2820 msgr2=0x7f25140f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:49.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:49.374+0000 7f251c50f700 1 -- 172.21.15.47:0/3567481411 shutdown_connections 2024-02-23T00:53:49.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:49.374+0000 7f251c50f700 1 -- 172.21.15.47:0/3567481411 wait complete. 2024-02-23T00:53:49.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:49.374+0000 7f251c50f700 1 librados: shutdown 2024-02-23T00:53:49.378 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is set 2024-02-23T00:53:49.455 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:53:49.480 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:49.518 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:53:49.527 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:49.527 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:53:49.628 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:53:49.652 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:48.360112+0000 mon.a (mon.0) 3400 : cluster [DBG] osdmap e689: 8 total, 8 up, 6 in 2024-02-23T00:53:49.652 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:48.939343+0000 mon.c (mon.2) 399 : audit [INF] from='client.? 172.21.15.47:0/3567481411' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:53:49.653 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:48.939771+0000 mon.a (mon.0) 3401 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:53:49.654 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:49.672 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:48.360112+0000 mon.a (mon.0) 3400 : cluster [DBG] osdmap e689: 8 total, 8 up, 6 in 2024-02-23T00:53:49.672 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:48.939343+0000 mon.c (mon.2) 399 : audit [INF] from='client.? 172.21.15.47:0/3567481411' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:53:49.672 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:48.939771+0000 mon.a (mon.0) 3401 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:53:49.672 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:48.360112+0000 mon.a (mon.0) 3400 : cluster [DBG] osdmap e689: 8 total, 8 up, 6 in 2024-02-23T00:53:49.672 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:48.939343+0000 mon.c (mon.2) 399 : audit [INF] from='client.? 172.21.15.47:0/3567481411' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:53:49.672 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:48.939771+0000 mon.a (mon.0) 3401 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:53:49.852 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:53:49.854 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:49.855 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:53:49.956 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:53:49.988 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:50.012 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:53:50.015 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:50.015 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:53:50.059 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:53:50.116 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:53:50.144 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:50.360 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:53:50.360 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-02-23T00:53:50.361 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-02-23T00:53:50.361 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-02-23T00:53:50.361 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:53:50.593 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:53:50.595 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:50.595 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:53:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:49.363953+0000 mon.a (mon.0) 3402 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:53:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:49.371893+0000 mon.a (mon.0) 3403 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:53:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:49.371974+0000 mon.a (mon.0) 3404 : cluster [DBG] osdmap e690: 8 total, 8 up, 6 in 2024-02-23T00:53:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:49.374006+0000 mon.c (mon.2) 400 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:50.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:49.375286+0000 mon.a (mon.0) 3405 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:50.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:49.377137+0000 mgr.y (mgr.14517) 598 : cluster [DBG] pgmap v1044: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:53:50.696 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:53:50.726 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:50.807 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:53:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:53:50] "GET /metrics HTTP/1.1" 200 191308 "" "Prometheus/2.33.4" 2024-02-23T00:53:50.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:49.363953+0000 mon.a (mon.0) 3402 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:53:50.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:49.371893+0000 mon.a (mon.0) 3403 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:53:50.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:49.371974+0000 mon.a (mon.0) 3404 : cluster [DBG] osdmap e690: 8 total, 8 up, 6 in 2024-02-23T00:53:50.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:49.374006+0000 mon.c (mon.2) 400 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:50.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:49.375286+0000 mon.a (mon.0) 3405 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:50.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:49.377137+0000 mgr.y (mgr.14517) 598 : cluster [DBG] pgmap v1044: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:53:50.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:49.363953+0000 mon.a (mon.0) 3402 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:53:50.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:49.371893+0000 mon.a (mon.0) 3403 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:53:50.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:49.371974+0000 mon.a (mon.0) 3404 : cluster [DBG] osdmap e690: 8 total, 8 up, 6 in 2024-02-23T00:53:50.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:49.374006+0000 mon.c (mon.2) 400 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:50.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:49.375286+0000 mon.a (mon.0) 3405 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:50.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:49.377137+0000 mgr.y (mgr.14517) 598 : cluster [DBG] pgmap v1044: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:53:51.028 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.7 dump_historic_ops 2024-02-23T00:53:51.075 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:53:51.078 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:51.078 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:53:51.179 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:53:51.206 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:51.224 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:53:51.227 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:51.227 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:53:51.328 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:53:51.338 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:53:51.355 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:51.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:50.374905+0000 mon.a (mon.0) 3406 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:53:51.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:50.375081+0000 mon.a (mon.0) 3407 : cluster [DBG] osdmap e691: 8 total, 8 up, 6 in 2024-02-23T00:53:51.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:50.376318+0000 mon.c (mon.2) 401 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:51.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:50.379684+0000 mon.a (mon.0) 3408 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:51.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:50.374905+0000 mon.a (mon.0) 3406 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:53:51.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:50.375081+0000 mon.a (mon.0) 3407 : cluster [DBG] osdmap e691: 8 total, 8 up, 6 in 2024-02-23T00:53:51.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:50.376318+0000 mon.c (mon.2) 401 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:51.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:50.379684+0000 mon.a (mon.0) 3408 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:51.605 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:53:51.607 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:51.608 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:53:51.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:50.374905+0000 mon.a (mon.0) 3406 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:53:51.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:50.375081+0000 mon.a (mon.0) 3407 : cluster [DBG] osdmap e691: 8 total, 8 up, 6 in 2024-02-23T00:53:51.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:50.376318+0000 mon.c (mon.2) 401 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:51.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:50.379684+0000 mon.a (mon.0) 3408 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:51.709 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:53:51.736 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:52.023 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset noscrub 2024-02-23T00:53:52.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.278+0000 7f48cbc89700 1 -- 172.21.15.47:0/2139652824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48c40fe510 msgr2=0x7f48c40f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:52.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.278+0000 7f48cbc89700 1 --2- 172.21.15.47:0/2139652824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48c40fe510 0x7f48c40f7d40 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 rx=0x7f48b4004870 tx=0x7f48b4009cb0).stop 2024-02-23T00:53:52.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.278+0000 7f48cbc89700 1 -- 172.21.15.47:0/2139652824 shutdown_connections 2024-02-23T00:53:52.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.278+0000 7f48cbc89700 1 --2- 172.21.15.47:0/2139652824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48c40f8420 0x7f48c4104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:52.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.278+0000 7f48cbc89700 1 --2- 172.21.15.47:0/2139652824 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f48c40fdbb0 0x7f48c40fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:52.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.278+0000 7f48cbc89700 1 --2- 172.21.15.47:0/2139652824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48c40fe510 0x7f48c40f7d40 unknown :-1 s=CLOSED pgs=2617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:52.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.278+0000 7f48cbc89700 1 -- 172.21.15.47:0/2139652824 >> 172.21.15.47:0/2139652824 conn(0x7f48c40f3800 msgr2=0x7f48c40f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:52.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.278+0000 7f48cbc89700 1 -- 172.21.15.47:0/2139652824 shutdown_connections 2024-02-23T00:53:52.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.278+0000 7f48cbc89700 1 -- 172.21.15.47:0/2139652824 wait complete. 2024-02-23T00:53:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.278+0000 7f48cbc89700 1 librados: starting msgr at 2024-02-23T00:53:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.278+0000 7f48cbc89700 1 librados: starting objecter 2024-02-23T00:53:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.279+0000 7f48cbc89700 1 Processor -- start 2024-02-23T00:53:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.279+0000 7f48cbc89700 1 -- start start 2024-02-23T00:53:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.279+0000 7f48cbc89700 1 librados: setting wanted keys 2024-02-23T00:53:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.279+0000 7f48cbc89700 1 librados: calling monclient init 2024-02-23T00:53:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.279+0000 7f48cbc89700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48c40f8420 0x7f48c410dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.279+0000 7f48cbc89700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48c40fdbb0 0x7f48c410e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.279+0000 7f48cbc89700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f48c40fe510 0x7f48c4108080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.279+0000 7f48cbc89700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c4108570 con 0x7f48c40f8420 2024-02-23T00:53:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.279+0000 7f48cbc89700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c41086e0 con 0x7f48c40fdbb0 2024-02-23T00:53:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.279+0000 7f48cbc89700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c41088a0 con 0x7f48c40fe510 2024-02-23T00:53:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.279+0000 7f48ca226700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f48c40fe510 0x7f48c4108080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.279+0000 7f48ca226700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f48c40fe510 0x7f48c4108080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:53302/0 (socket says 172.21.15.47:53302) 2024-02-23T00:53:52.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.279+0000 7f48c9224700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48c40fdbb0 0x7f48c410e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:52.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.279+0000 7f48ca226700 1 -- 172.21.15.47:0/2302355375 learned_addr learned my addr 172.21.15.47:0/2302355375 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:53:52.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.279+0000 7f48c9a25700 1 --2- 172.21.15.47:0/2302355375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48c40f8420 0x7f48c410dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:52.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.279+0000 7f48ca226700 1 -- 172.21.15.47:0/2302355375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48c40fdbb0 msgr2=0x7f48c410e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:52.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.279+0000 7f48ca226700 1 --2- 172.21.15.47:0/2302355375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48c40fdbb0 0x7f48c410e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:52.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.279+0000 7f48ca226700 1 -- 172.21.15.47:0/2302355375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48c40f8420 msgr2=0x7f48c410dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:52.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.279+0000 7f48ca226700 1 --2- 172.21.15.47:0/2302355375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48c40f8420 0x7f48c410dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:52.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.279+0000 7f48ca226700 1 -- 172.21.15.47:0/2302355375 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c0005040 con 0x7f48c40fe510 2024-02-23T00:53:52.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.279+0000 7f48c9a25700 1 --2- 172.21.15.47:0/2302355375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48c40f8420 0x7f48c410dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:53:52.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.279+0000 7f48ca226700 1 --2- 172.21.15.47:0/2302355375 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f48c40fe510 0x7f48c4108080 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 rx=0x7f48bc00db50 tx=0x7f48bc004f00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:52.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.279+0000 7f48baffd700 10 client.?.objecter ms_handle_connect 0x7f48c40fe510 2024-02-23T00:53:52.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.279+0000 7f48baffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:53:52.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.280+0000 7f48baffd700 1 -- 172.21.15.47:0/2302355375 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc00b750 con 0x7f48c40fe510 2024-02-23T00:53:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.280+0000 7f48cbc89700 1 -- 172.21.15.47:0/2302355375 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b4005040 con 0x7f48c40fe510 2024-02-23T00:53:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.280+0000 7f48cbc89700 10 client.41945.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:53:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.280+0000 7f48cbc89700 1 -- 172.21.15.47:0/2302355375 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c4108eb0 con 0x7f48c40fe510 2024-02-23T00:53:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.280+0000 7f48cbc89700 1 librados: init done 2024-02-23T00:53:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.280+0000 7f48baffd700 1 -- 172.21.15.47:0/2302355375 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f48bc00f040 con 0x7f48c40fe510 2024-02-23T00:53:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.280+0000 7f48baffd700 1 -- 172.21.15.47:0/2302355375 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc013ab0 con 0x7f48c40fe510 2024-02-23T00:53:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.281+0000 7f48cbc89700 1 -- 172.21.15.47:0/2302355375 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48c405ee80 con 0x7f48c40fe510 2024-02-23T00:53:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.281+0000 7f48baffd700 1 -- 172.21.15.47:0/2302355375 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f48bc00b8b0 con 0x7f48c40fe510 2024-02-23T00:53:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.281+0000 7f48baffd700 1 --2- 172.21.15.47:0/2302355375 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f48b0062cc0 0x7f48b0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.282+0000 7f48baffd700 1 -- 172.21.15.47:0/2302355375 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(691..691 src has 482..691) v4 ==== 9696+0+0 (secure 0 0 0) 0x7f48bc07d490 con 0x7f48c40fe510 2024-02-23T00:53:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.282+0000 7f48baffd700 10 client.41945.objecter ms_dispatch 0x7f48c4001550 osd_map(691..691 src has 482..691) v4 2024-02-23T00:53:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.282+0000 7f48baffd700 3 client.41945.objecter handle_osd_map got epochs [691,691] > 0 2024-02-23T00:53:52.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.282+0000 7f48baffd700 3 client.41945.objecter handle_osd_map decoding full epoch 691 2024-02-23T00:53:52.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.282+0000 7f48baffd700 20 client.41945.objecter dump_active .. 0 homeless 2024-02-23T00:53:52.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.284+0000 7f48c9a25700 1 --2- 172.21.15.47:0/2302355375 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f48b0062cc0 0x7f48b0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:52.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.284+0000 7f48baffd700 10 client.41945.objecter ms_handle_connect 0x7f48b0062cc0 2024-02-23T00:53:52.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.284+0000 7f48c9a25700 1 --2- 172.21.15.47:0/2302355375 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f48b0062cc0 0x7f48b0065180 secure :-1 s=READY pgs=3167 cs=0 l=1 rev1=1 rx=0x7f48c0007fd0 tx=0x7f48c0009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:52.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.286+0000 7f48baffd700 1 -- 172.21.15.47:0/2302355375 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f48bc04c950 con 0x7f48c40fe510 2024-02-23T00:53:52.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:51.477+0000 7f48cbc89700 1 -- 172.21.15.47:0/2302355375 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd reweight", "id": 3, "weight": 1.0} v 0) v1 -- 0x7f48c41095f0 con 0x7f48c40fe510 2024-02-23T00:53:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.382+0000 7f48baffd700 1 -- 172.21.15.47:0/2302355375 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 3, "weight": 1.0}]=0 reweighted osd.3 to 1 (10000) v693) v1 ==== 113+0+0 (secure 0 0 0) 0x7f48bc018020 con 0x7f48c40fe510 2024-02-23T00:53:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.385+0000 7f48cbc89700 10 librados: watch_flush enter 2024-02-23T00:53:52.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.387+0000 7f48cbc89700 10 librados: watch_flush exit 2024-02-23T00:53:52.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.387+0000 7f48cbc89700 20 client.41945.objecter shutdown clearing up homeless session... 2024-02-23T00:53:52.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.387+0000 7f48cbc89700 10 client.41945.objecter successfully canceled tick 2024-02-23T00:53:52.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.387+0000 7f48cbc89700 1 -- 172.21.15.47:0/2302355375 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f48b0062cc0 msgr2=0x7f48b0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:52.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.387+0000 7f48cbc89700 1 --2- 172.21.15.47:0/2302355375 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f48b0062cc0 0x7f48b0065180 secure :-1 s=READY pgs=3167 cs=0 l=1 rev1=1 rx=0x7f48c0007fd0 tx=0x7f48c0009cb0).stop 2024-02-23T00:53:52.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.387+0000 7f48cbc89700 1 -- 172.21.15.47:0/2302355375 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f48c40fe510 msgr2=0x7f48c4108080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:52.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.387+0000 7f48cbc89700 1 --2- 172.21.15.47:0/2302355375 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f48c40fe510 0x7f48c4108080 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 rx=0x7f48bc00db50 tx=0x7f48bc004f00).stop 2024-02-23T00:53:52.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.387+0000 7f48cbc89700 1 -- 172.21.15.47:0/2302355375 shutdown_connections 2024-02-23T00:53:52.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.387+0000 7f48cbc89700 1 --2- 172.21.15.47:0/2302355375 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f48b0062cc0 0x7f48b0065180 unknown :-1 s=CLOSED pgs=3167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:52.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.387+0000 7f48cbc89700 1 --2- 172.21.15.47:0/2302355375 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f48c40fe510 0x7f48c4108080 unknown :-1 s=CLOSED pgs=2777 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:52.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.387+0000 7f48cbc89700 1 --2- 172.21.15.47:0/2302355375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48c40f8420 0x7f48c410dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:52.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.387+0000 7f48cbc89700 1 --2- 172.21.15.47:0/2302355375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48c40fdbb0 0x7f48c410e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:52.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.387+0000 7f48cbc89700 1 -- 172.21.15.47:0/2302355375 >> 172.21.15.47:0/2302355375 conn(0x7f48c40f3800 msgr2=0x7f48c40fb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:52.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.388+0000 7f48cbc89700 1 -- 172.21.15.47:0/2302355375 shutdown_connections 2024-02-23T00:53:52.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.388+0000 7f48cbc89700 1 -- 172.21.15.47:0/2302355375 wait complete. 2024-02-23T00:53:52.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.388+0000 7f48cbc89700 1 librados: shutdown 2024-02-23T00:53:52.393 INFO:teuthology.orchestra.run.smithi047.stderr:reweighted osd.3 to 1 (10000) 2024-02-23T00:53:52.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:51.376419+0000 mon.a (mon.0) 3409 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:53:52.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:51.376491+0000 mon.a (mon.0) 3410 : cluster [DBG] osdmap e692: 8 total, 8 up, 6 in 2024-02-23T00:53:52.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:51.377405+0000 mon.c (mon.2) 402 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-120"}]: dispatch 2024-02-23T00:53:52.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:51.377510+0000 mgr.y (mgr.14517) 599 : cluster [DBG] pgmap v1047: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:53:52.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:51.381261+0000 mon.a (mon.0) 3411 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-120"}]: dispatch 2024-02-23T00:53:52.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:51.479039+0000 mon.c (mon.2) 403 : audit [INF] from='client.? 172.21.15.47:0/2302355375' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-23T00:53:52.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:51.479343+0000 mon.a (mon.0) 3412 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-23T00:53:52.586 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:53:52.588 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:52.589 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:53:52.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.598+0000 7fcba26cd700 1 -- 172.21.15.47:0/4071274474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb9c0f8cb0 msgr2=0x7fcb9c0f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:52.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.598+0000 7fcba26cd700 1 --2- 172.21.15.47:0/4071274474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb9c0f8cb0 0x7fcb9c0f9090 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 rx=0x7fcb8c0047b0 tx=0x7fcb8c009cb0).stop 2024-02-23T00:53:52.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.598+0000 7fcba26cd700 1 -- 172.21.15.47:0/4071274474 shutdown_connections 2024-02-23T00:53:52.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.598+0000 7fcba26cd700 1 --2- 172.21.15.47:0/4071274474 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcb9c102210 0x7fcb9c104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:52.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.598+0000 7fcba26cd700 1 --2- 172.21.15.47:0/4071274474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb9c0f8cb0 0x7fcb9c0f9090 unknown :-1 s=CLOSED pgs=2618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:52.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.598+0000 7fcba26cd700 1 --2- 172.21.15.47:0/4071274474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb9c0f9610 0x7fcb9c101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:52.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.598+0000 7fcba26cd700 1 -- 172.21.15.47:0/4071274474 >> 172.21.15.47:0/4071274474 conn(0x7fcb9c0f2820 msgr2=0x7fcb9c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:52.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.598+0000 7fcba26cd700 1 -- 172.21.15.47:0/4071274474 shutdown_connections 2024-02-23T00:53:52.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.598+0000 7fcba26cd700 1 -- 172.21.15.47:0/4071274474 wait complete. 2024-02-23T00:53:52.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.598+0000 7fcba26cd700 1 librados: starting msgr at 2024-02-23T00:53:52.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.598+0000 7fcba26cd700 1 librados: starting objecter 2024-02-23T00:53:52.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.598+0000 7fcba26cd700 1 Processor -- start 2024-02-23T00:53:52.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.599+0000 7fcba26cd700 1 -- start start 2024-02-23T00:53:52.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.599+0000 7fcba26cd700 1 librados: setting wanted keys 2024-02-23T00:53:52.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.599+0000 7fcba26cd700 1 librados: calling monclient init 2024-02-23T00:53:52.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.599+0000 7fcba26cd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb9c0f8cb0 0x7fcb9c110220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:52.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.599+0000 7fcba26cd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcb9c0f9610 0x7fcb9c110710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:52.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.599+0000 7fcba26cd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb9c102210 0x7fcb9c10a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:52.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.599+0000 7fcba26cd700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c10a8b0 con 0x7fcb9c102210 2024-02-23T00:53:52.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.599+0000 7fcba26cd700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c10aa20 con 0x7fcb9c0f8cb0 2024-02-23T00:53:52.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.599+0000 7fcba26cd700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c10abe0 con 0x7fcb9c0f9610 2024-02-23T00:53:52.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.599+0000 7fcba0c6a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb9c102210 0x7fcb9c10a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:52.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.599+0000 7fcb9bfff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb9c0f8cb0 0x7fcb9c110220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:52.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.599+0000 7fcb9b7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcb9c0f9610 0x7fcb9c110710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:52.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.599+0000 7fcba0c6a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb9c102210 0x7fcb9c10a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49208/0 (socket says 172.21.15.47:49208) 2024-02-23T00:53:52.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.599+0000 7fcb9b7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcb9c0f9610 0x7fcb9c110710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:53316/0 (socket says 172.21.15.47:53316) 2024-02-23T00:53:52.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.599+0000 7fcba0c6a700 1 -- 172.21.15.47:0/3015858306 learned_addr learned my addr 172.21.15.47:0/3015858306 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:53:52.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.599+0000 7fcba0c6a700 1 -- 172.21.15.47:0/3015858306 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcb9c0f9610 msgr2=0x7fcb9c110710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:52.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.599+0000 7fcba0c6a700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcb9c0f9610 0x7fcb9c110710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:52.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.599+0000 7fcba0c6a700 1 -- 172.21.15.47:0/3015858306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb9c0f8cb0 msgr2=0x7fcb9c110220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:52.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.599+0000 7fcba0c6a700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb9c0f8cb0 0x7fcb9c110220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:52.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.599+0000 7fcba0c6a700 1 -- 172.21.15.47:0/3015858306 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb940097c0 con 0x7fcb9c102210 2024-02-23T00:53:52.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.599+0000 7fcb9bfff700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb9c0f8cb0 0x7fcb9c110220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:53:52.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.600+0000 7fcba0c6a700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb9c102210 0x7fcb9c10a3c0 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 rx=0x7fcb94001e80 tx=0x7fcb94004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:52.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.600+0000 7fcb997fa700 10 client.?.objecter ms_handle_connect 0x7fcb9c102210 2024-02-23T00:53:52.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.600+0000 7fcb997fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:53:52.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.600+0000 7fcb997fa700 1 -- 172.21.15.47:0/3015858306 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9400fb30 con 0x7fcb9c102210 2024-02-23T00:53:52.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.600+0000 7fcb997fa700 1 -- 172.21.15.47:0/3015858306 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fcb94009560 con 0x7fcb9c102210 2024-02-23T00:53:52.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.600+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb8c005040 con 0x7fcb9c102210 2024-02-23T00:53:52.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.600+0000 7fcb997fa700 1 -- 172.21.15.47:0/3015858306 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb94007ce0 con 0x7fcb9c102210 2024-02-23T00:53:52.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.600+0000 7fcba26cd700 10 client.31629.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:53:52.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.600+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb9c10b1d0 con 0x7fcb9c102210 2024-02-23T00:53:52.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.600+0000 7fcba26cd700 1 librados: init done 2024-02-23T00:53:52.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.601+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fcb9c05fa60 con 0x7fcb9c102210 2024-02-23T00:53:52.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.601+0000 7fcb997fa700 1 -- 172.21.15.47:0/3015858306 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fcb940090d0 con 0x7fcb9c102210 2024-02-23T00:53:52.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.602+0000 7fcb997fa700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcb88062cc0 0x7fcb88065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:52.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.602+0000 7fcb997fa700 1 -- 172.21.15.47:0/3015858306 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(693..693 src has 482..693) v4 ==== 9691+0+0 (secure 0 0 0) 0x7fcb94007e40 con 0x7fcb9c102210 2024-02-23T00:53:52.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.602+0000 7fcb997fa700 10 client.31629.objecter ms_dispatch 0x7fcb9c001550 osd_map(693..693 src has 482..693) v4 2024-02-23T00:53:52.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.602+0000 7fcb997fa700 3 client.31629.objecter handle_osd_map got epochs [693,693] > 0 2024-02-23T00:53:52.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.602+0000 7fcb997fa700 3 client.31629.objecter handle_osd_map decoding full epoch 693 2024-02-23T00:53:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.602+0000 7fcb9bfff700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcb88062cc0 0x7fcb88065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.602+0000 7fcb997fa700 20 client.31629.objecter dump_active .. 0 homeless 2024-02-23T00:53:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.602+0000 7fcb997fa700 1 -- 172.21.15.47:0/3015858306 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v693) v1 ==== 54+0+16 (secure 0 0 0) 0x7fcb9407e940 con 0x7fcb9c102210 2024-02-23T00:53:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.602+0000 7fcba26cd700 10 client.31629.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.602+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_assign -1 1 2024-02-23T00:53:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.602+0000 7fcb997fa700 10 client.31629.objecter ms_handle_connect 0x7fcb88062cc0 2024-02-23T00:53:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.602+0000 7fcb9bfff700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcb88062cc0 0x7fcb88065180 secure :-1 s=READY pgs=3168 cs=0 l=1 rev1=1 rx=0x7fcb8c004e20 tx=0x7fcb8c009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.602+0000 7fcba26cd700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fcb9c0fe5e0 0x7fcb9c0fe9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.602+0000 7fcba26cd700 20 client.31629.objecter _get_session s=0x7fcb9c05ee80 osd=0 3 2024-02-23T00:53:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.602+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c05ee80 osd=0 3 2024-02-23T00:53:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.602+0000 7fcba26cd700 20 client.31629.objecter _get_session s=0x7fcb9c05ee80 osd=0 3 2024-02-23T00:53:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.602+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_remove -1 1 2024-02-23T00:53:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.602+0000 7fcba26cd700 20 client.31629.objecter get_session s=0x7fcb9c05ee80 osd=0 3 2024-02-23T00:53:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.602+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_assign 0 1 2024-02-23T00:53:52.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.602+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c05ee80 osd=0 4 2024-02-23T00:53:52.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.602+0000 7fcba26cd700 10 client.31629.objecter _send_command 1 2024-02-23T00:53:52.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.602+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb9c107050 con 0x7fcb9c0fe5e0 2024-02-23T00:53:52.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.602+0000 7fcb9b7fe700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fcb9c0fe5e0 0x7fcb9c0fe9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:52.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.602+0000 7fcb997fa700 10 client.31629.objecter ms_handle_connect 0x7fcb9c0fe5e0 2024-02-23T00:53:52.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.602+0000 7fcb9b7fe700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fcb9c0fe5e0 0x7fcb9c0fe9c0 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:52.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.603+0000 7fcb997fa700 1 -- 172.21.15.47:0/3015858306 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcb9c107050 con 0x7fcb9c0fe5e0 2024-02-23T00:53:52.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.603+0000 7fcb997fa700 10 client.31629.objecter ms_dispatch 0x7fcb9c001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:53:52.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.603+0000 7fcb997fa700 10 client.31629.objecter _finish_command 1 = system:0 2024-02-23T00:53:52.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.603+0000 7fcb997fa700 20 client.31629.objecter put_session s=0x7fcb9c05ee80 osd=0 4 2024-02-23T00:53:52.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.603+0000 7fcb997fa700 15 client.31629.objecter _session_command_op_remove 0 1 2024-02-23T00:53:52.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.618+0000 7fcba26cd700 10 client.31629.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:53:52.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.618+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_assign -1 2 2024-02-23T00:53:52.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.618+0000 7fcba26cd700 20 client.31629.objecter _get_session s=0x7fcb9c05ee80 osd=0 3 2024-02-23T00:53:52.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.618+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c05ee80 osd=0 3 2024-02-23T00:53:52.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.618+0000 7fcba26cd700 20 client.31629.objecter _get_session s=0x7fcb9c05ee80 osd=0 3 2024-02-23T00:53:52.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.618+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_remove -1 2 2024-02-23T00:53:52.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.618+0000 7fcba26cd700 20 client.31629.objecter get_session s=0x7fcb9c05ee80 osd=0 3 2024-02-23T00:53:52.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.618+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_assign 0 2 2024-02-23T00:53:52.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.618+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c05ee80 osd=0 4 2024-02-23T00:53:52.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.618+0000 7fcba26cd700 10 client.31629.objecter _send_command 2 2024-02-23T00:53:52.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.618+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fcb9c0273c0 con 0x7fcb9c0fe5e0 2024-02-23T00:53:52.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.619+0000 7fcb997fa700 1 -- 172.21.15.47:0/3015858306 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fcb9c0273c0 con 0x7fcb9c0fe5e0 2024-02-23T00:53:52.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.619+0000 7fcb997fa700 10 client.31629.objecter ms_dispatch 0x7fcb9c001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:53:52.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.619+0000 7fcb997fa700 10 client.31629.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:53:52.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.619+0000 7fcb997fa700 20 client.31629.objecter put_session s=0x7fcb9c05ee80 osd=0 4 2024-02-23T00:53:52.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.619+0000 7fcb997fa700 15 client.31629.objecter _session_command_op_remove 0 2 2024-02-23T00:53:52.625 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:53:52.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.620+0000 7fcba26cd700 10 client.31629.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:52.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.620+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_assign -1 3 2024-02-23T00:53:52.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.620+0000 7fcba26cd700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fcb9c0598f0 0x7fcb9c059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:52.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.620+0000 7fcba26cd700 20 client.31629.objecter _get_session s=0x7fcb9c028710 osd=1 3 2024-02-23T00:53:52.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.620+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c028710 osd=1 3 2024-02-23T00:53:52.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.620+0000 7fcba26cd700 20 client.31629.objecter _get_session s=0x7fcb9c028710 osd=1 3 2024-02-23T00:53:52.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.620+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_remove -1 3 2024-02-23T00:53:52.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.620+0000 7fcba26cd700 20 client.31629.objecter get_session s=0x7fcb9c028710 osd=1 3 2024-02-23T00:53:52.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.620+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_assign 1 3 2024-02-23T00:53:52.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.620+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c028710 osd=1 4 2024-02-23T00:53:52.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.620+0000 7fcba26cd700 10 client.31629.objecter _send_command 3 2024-02-23T00:53:52.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.620+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb9c11d830 con 0x7fcb9c0598f0 2024-02-23T00:53:52.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.620+0000 7fcb9bfff700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fcb9c0598f0 0x7fcb9c059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:52.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.620+0000 7fcb997fa700 10 client.31629.objecter ms_handle_connect 0x7fcb9c0598f0 2024-02-23T00:53:52.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.620+0000 7fcb9bfff700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fcb9c0598f0 0x7fcb9c059cd0 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:52.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.621+0000 7fcb997fa700 1 -- 172.21.15.47:0/3015858306 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcb9c11d830 con 0x7fcb9c0598f0 2024-02-23T00:53:52.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.621+0000 7fcb997fa700 10 client.31629.objecter ms_dispatch 0x7fcb9c001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:53:52.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.621+0000 7fcb997fa700 10 client.31629.objecter _finish_command 3 = system:0 2024-02-23T00:53:52.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.621+0000 7fcb997fa700 20 client.31629.objecter put_session s=0x7fcb9c028710 osd=1 4 2024-02-23T00:53:52.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.621+0000 7fcb997fa700 15 client.31629.objecter _session_command_op_remove 1 3 2024-02-23T00:53:52.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.635+0000 7fcba26cd700 10 client.31629.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:53:52.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.635+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_assign -1 4 2024-02-23T00:53:52.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.635+0000 7fcba26cd700 20 client.31629.objecter _get_session s=0x7fcb9c028710 osd=1 3 2024-02-23T00:53:52.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.635+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c028710 osd=1 3 2024-02-23T00:53:52.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.635+0000 7fcba26cd700 20 client.31629.objecter _get_session s=0x7fcb9c028710 osd=1 3 2024-02-23T00:53:52.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.635+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_remove -1 4 2024-02-23T00:53:52.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.635+0000 7fcba26cd700 20 client.31629.objecter get_session s=0x7fcb9c028710 osd=1 3 2024-02-23T00:53:52.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.635+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_assign 1 4 2024-02-23T00:53:52.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.635+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c028710 osd=1 4 2024-02-23T00:53:52.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.635+0000 7fcba26cd700 10 client.31629.objecter _send_command 4 2024-02-23T00:53:52.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.635+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fcb9c059360 con 0x7fcb9c0598f0 2024-02-23T00:53:52.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.637+0000 7fcb997fa700 1 -- 172.21.15.47:0/3015858306 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fcb9c059360 con 0x7fcb9c0598f0 2024-02-23T00:53:52.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.637+0000 7fcb997fa700 10 client.31629.objecter ms_dispatch 0x7fcb9c001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:53:52.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.637+0000 7fcb997fa700 10 client.31629.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:53:52.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.637+0000 7fcb997fa700 20 client.31629.objecter put_session s=0x7fcb9c028710 osd=1 4 2024-02-23T00:53:52.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.637+0000 7fcb997fa700 15 client.31629.objecter _session_command_op_remove 1 4 2024-02-23T00:53:52.640 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:53:52.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.637+0000 7fcba26cd700 10 client.31629.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:52.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.637+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_assign -1 5 2024-02-23T00:53:52.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.637+0000 7fcba26cd700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fcb9c028270 0x7fcb9c11c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:52.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.637+0000 7fcba26cd700 20 client.31629.objecter _get_session s=0x7fcb9c059770 osd=2 3 2024-02-23T00:53:52.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.637+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c059770 osd=2 3 2024-02-23T00:53:52.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.637+0000 7fcba26cd700 20 client.31629.objecter _get_session s=0x7fcb9c059770 osd=2 3 2024-02-23T00:53:52.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.637+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_remove -1 5 2024-02-23T00:53:52.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.637+0000 7fcba26cd700 20 client.31629.objecter get_session s=0x7fcb9c059770 osd=2 3 2024-02-23T00:53:52.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.637+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_assign 2 5 2024-02-23T00:53:52.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.637+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c059770 osd=2 4 2024-02-23T00:53:52.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.637+0000 7fcba26cd700 10 client.31629.objecter _send_command 5 2024-02-23T00:53:52.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.637+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb9c11c740 con 0x7fcb9c028270 2024-02-23T00:53:52.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.637+0000 7fcb9b7fe700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fcb9c028270 0x7fcb9c11c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:52.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.637+0000 7fcb997fa700 10 client.31629.objecter ms_handle_connect 0x7fcb9c028270 2024-02-23T00:53:52.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.637+0000 7fcb9b7fe700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fcb9c028270 0x7fcb9c11c250 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:52.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.638+0000 7fcb997fa700 1 -- 172.21.15.47:0/3015858306 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcb9c11c740 con 0x7fcb9c028270 2024-02-23T00:53:52.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.638+0000 7fcb997fa700 10 client.31629.objecter ms_dispatch 0x7fcb9c001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:53:52.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.638+0000 7fcb997fa700 10 client.31629.objecter _finish_command 5 = system:0 2024-02-23T00:53:52.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.638+0000 7fcb997fa700 20 client.31629.objecter put_session s=0x7fcb9c059770 osd=2 4 2024-02-23T00:53:52.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.638+0000 7fcb997fa700 15 client.31629.objecter _session_command_op_remove 2 5 2024-02-23T00:53:52.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.652+0000 7fcba26cd700 10 client.31629.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:53:52.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.652+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_assign -1 6 2024-02-23T00:53:52.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.652+0000 7fcba26cd700 20 client.31629.objecter _get_session s=0x7fcb9c059770 osd=2 3 2024-02-23T00:53:52.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.652+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c059770 osd=2 3 2024-02-23T00:53:52.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.652+0000 7fcba26cd700 20 client.31629.objecter _get_session s=0x7fcb9c059770 osd=2 3 2024-02-23T00:53:52.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.652+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_remove -1 6 2024-02-23T00:53:52.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.652+0000 7fcba26cd700 20 client.31629.objecter get_session s=0x7fcb9c059770 osd=2 3 2024-02-23T00:53:52.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.652+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_assign 2 6 2024-02-23T00:53:52.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.652+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c059770 osd=2 4 2024-02-23T00:53:52.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.652+0000 7fcba26cd700 10 client.31629.objecter _send_command 6 2024-02-23T00:53:52.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.652+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fcb9c0277d0 con 0x7fcb9c028270 2024-02-23T00:53:52.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.654+0000 7fcb997fa700 1 -- 172.21.15.47:0/3015858306 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fcb9c0277d0 con 0x7fcb9c028270 2024-02-23T00:53:52.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.654+0000 7fcb997fa700 10 client.31629.objecter ms_dispatch 0x7fcb9c001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:53:52.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.654+0000 7fcb997fa700 10 client.31629.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:53:52.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.654+0000 7fcb997fa700 20 client.31629.objecter put_session s=0x7fcb9c059770 osd=2 4 2024-02-23T00:53:52.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.654+0000 7fcb997fa700 15 client.31629.objecter _session_command_op_remove 2 6 2024-02-23T00:53:52.657 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:53:52.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.654+0000 7fcba26cd700 10 client.31629.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:52.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.654+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_assign -1 7 2024-02-23T00:53:52.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.654+0000 7fcba26cd700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fcb9c11f9b0 0x7fcb9c121da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:52.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.654+0000 7fcba26cd700 20 client.31629.objecter _get_session s=0x7fcb9c027b80 osd=3 3 2024-02-23T00:53:52.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.654+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c027b80 osd=3 3 2024-02-23T00:53:52.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.654+0000 7fcba26cd700 20 client.31629.objecter _get_session s=0x7fcb9c027b80 osd=3 3 2024-02-23T00:53:52.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.654+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_remove -1 7 2024-02-23T00:53:52.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.654+0000 7fcba26cd700 20 client.31629.objecter get_session s=0x7fcb9c027b80 osd=3 3 2024-02-23T00:53:52.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.654+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_assign 3 7 2024-02-23T00:53:52.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.654+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c027b80 osd=3 4 2024-02-23T00:53:52.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.654+0000 7fcba26cd700 10 client.31629.objecter _send_command 7 2024-02-23T00:53:52.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.654+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb9c11d6a0 con 0x7fcb9c11f9b0 2024-02-23T00:53:52.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.654+0000 7fcba0c6a700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fcb9c11f9b0 0x7fcb9c121da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:52.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.654+0000 7fcb997fa700 10 client.31629.objecter ms_handle_connect 0x7fcb9c11f9b0 2024-02-23T00:53:52.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.654+0000 7fcba0c6a700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fcb9c11f9b0 0x7fcb9c121da0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:52.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.655+0000 7fcb997fa700 1 -- 172.21.15.47:0/3015858306 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcb9c11d6a0 con 0x7fcb9c11f9b0 2024-02-23T00:53:52.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.655+0000 7fcb997fa700 10 client.31629.objecter ms_dispatch 0x7fcb9c001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:53:52.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.655+0000 7fcb997fa700 10 client.31629.objecter _finish_command 7 = system:0 2024-02-23T00:53:52.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.655+0000 7fcb997fa700 20 client.31629.objecter put_session s=0x7fcb9c027b80 osd=3 4 2024-02-23T00:53:52.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.655+0000 7fcb997fa700 15 client.31629.objecter _session_command_op_remove 3 7 2024-02-23T00:53:52.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.669+0000 7fcba26cd700 10 client.31629.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:53:52.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.669+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_assign -1 8 2024-02-23T00:53:52.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.669+0000 7fcba26cd700 20 client.31629.objecter _get_session s=0x7fcb9c027b80 osd=3 3 2024-02-23T00:53:52.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.669+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c027b80 osd=3 3 2024-02-23T00:53:52.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.669+0000 7fcba26cd700 20 client.31629.objecter _get_session s=0x7fcb9c027b80 osd=3 3 2024-02-23T00:53:52.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.669+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_remove -1 8 2024-02-23T00:53:52.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.669+0000 7fcba26cd700 20 client.31629.objecter get_session s=0x7fcb9c027b80 osd=3 3 2024-02-23T00:53:52.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.669+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_assign 3 8 2024-02-23T00:53:52.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.669+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c027b80 osd=3 4 2024-02-23T00:53:52.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.669+0000 7fcba26cd700 10 client.31629.objecter _send_command 8 2024-02-23T00:53:52.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.669+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fcb9c11cb20 con 0x7fcb9c11f9b0 2024-02-23T00:53:52.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.671+0000 7fcb997fa700 1 -- 172.21.15.47:0/3015858306 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fcb9c11cb20 con 0x7fcb9c11f9b0 2024-02-23T00:53:52.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.671+0000 7fcb997fa700 10 client.31629.objecter ms_dispatch 0x7fcb9c001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:53:52.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.671+0000 7fcb997fa700 10 client.31629.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:53:52.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.671+0000 7fcb997fa700 20 client.31629.objecter put_session s=0x7fcb9c027b80 osd=3 4 2024-02-23T00:53:52.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.671+0000 7fcb997fa700 15 client.31629.objecter _session_command_op_remove 3 8 2024-02-23T00:53:52.674 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:53:52.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.671+0000 7fcba26cd700 10 client.31629.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:52.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.671+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_assign -1 9 2024-02-23T00:53:52.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.671+0000 7fcba26cd700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fcb9c11bdb0 0x7fcb9c123410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:52.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.671+0000 7fcba26cd700 20 client.31629.objecter _get_session s=0x7fcb9c11cb20 osd=4 3 2024-02-23T00:53:52.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.671+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c11cb20 osd=4 3 2024-02-23T00:53:52.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.671+0000 7fcba26cd700 20 client.31629.objecter _get_session s=0x7fcb9c11cb20 osd=4 3 2024-02-23T00:53:52.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.671+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_remove -1 9 2024-02-23T00:53:52.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.671+0000 7fcba26cd700 20 client.31629.objecter get_session s=0x7fcb9c11cb20 osd=4 3 2024-02-23T00:53:52.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.671+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_assign 4 9 2024-02-23T00:53:52.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.671+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c11cb20 osd=4 4 2024-02-23T00:53:52.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.671+0000 7fcba26cd700 10 client.31629.objecter _send_command 9 2024-02-23T00:53:52.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.671+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb9c11ce80 con 0x7fcb9c11bdb0 2024-02-23T00:53:52.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.671+0000 7fcb9bfff700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fcb9c11bdb0 0x7fcb9c123410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:52.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.672+0000 7fcb997fa700 10 client.31629.objecter ms_handle_connect 0x7fcb9c11bdb0 2024-02-23T00:53:52.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.672+0000 7fcb9bfff700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fcb9c11bdb0 0x7fcb9c123410 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:52.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.672+0000 7fcb997fa700 1 -- 172.21.15.47:0/3015858306 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcb9c11ce80 con 0x7fcb9c11bdb0 2024-02-23T00:53:52.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.672+0000 7fcb997fa700 10 client.31629.objecter ms_dispatch 0x7fcb9c001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:53:52.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.672+0000 7fcb997fa700 10 client.31629.objecter _finish_command 9 = system:0 2024-02-23T00:53:52.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.672+0000 7fcb997fa700 20 client.31629.objecter put_session s=0x7fcb9c11cb20 osd=4 4 2024-02-23T00:53:52.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.672+0000 7fcb997fa700 15 client.31629.objecter _session_command_op_remove 4 9 2024-02-23T00:53:52.691 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:53:52.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.687+0000 7fcba26cd700 10 client.31629.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:53:52.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.687+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_assign -1 10 2024-02-23T00:53:52.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.687+0000 7fcba26cd700 20 client.31629.objecter _get_session s=0x7fcb9c11cb20 osd=4 3 2024-02-23T00:53:52.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.687+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c11cb20 osd=4 3 2024-02-23T00:53:52.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.687+0000 7fcba26cd700 20 client.31629.objecter _get_session s=0x7fcb9c11cb20 osd=4 3 2024-02-23T00:53:52.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.687+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_remove -1 10 2024-02-23T00:53:52.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.687+0000 7fcba26cd700 20 client.31629.objecter get_session s=0x7fcb9c11cb20 osd=4 3 2024-02-23T00:53:52.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.687+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_assign 4 10 2024-02-23T00:53:52.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.687+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c11cb20 osd=4 4 2024-02-23T00:53:52.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.687+0000 7fcba26cd700 10 client.31629.objecter _send_command 10 2024-02-23T00:53:52.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.687+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fcb9c11ce80 con 0x7fcb9c11bdb0 2024-02-23T00:53:52.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.689+0000 7fcb997fa700 1 -- 172.21.15.47:0/3015858306 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fcb9c11ce80 con 0x7fcb9c11bdb0 2024-02-23T00:53:52.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.689+0000 7fcb997fa700 10 client.31629.objecter ms_dispatch 0x7fcb9c001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:53:52.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.689+0000 7fcb997fa700 10 client.31629.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:53:52.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.689+0000 7fcb997fa700 20 client.31629.objecter put_session s=0x7fcb9c11cb20 osd=4 4 2024-02-23T00:53:52.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.689+0000 7fcb997fa700 15 client.31629.objecter _session_command_op_remove 4 10 2024-02-23T00:53:52.694 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:53:52.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.689+0000 7fcba26cd700 10 client.31629.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:52.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.689+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_assign -1 11 2024-02-23T00:53:52.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.689+0000 7fcba26cd700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fcb9c123e10 0x7fcb9c124270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:52.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.689+0000 7fcba26cd700 20 client.31629.objecter _get_session s=0x7fcb9c11ce80 osd=5 3 2024-02-23T00:53:52.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.689+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c11ce80 osd=5 3 2024-02-23T00:53:52.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.689+0000 7fcba26cd700 20 client.31629.objecter _get_session s=0x7fcb9c11ce80 osd=5 3 2024-02-23T00:53:52.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.689+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_remove -1 11 2024-02-23T00:53:52.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.689+0000 7fcba26cd700 20 client.31629.objecter get_session s=0x7fcb9c11ce80 osd=5 3 2024-02-23T00:53:52.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.689+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_assign 5 11 2024-02-23T00:53:52.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.689+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c11ce80 osd=5 4 2024-02-23T00:53:52.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.689+0000 7fcba26cd700 10 client.31629.objecter _send_command 11 2024-02-23T00:53:52.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.689+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb9c12b1a0 con 0x7fcb9c123e10 2024-02-23T00:53:52.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.689+0000 7fcb9b7fe700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fcb9c123e10 0x7fcb9c124270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:52.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.690+0000 7fcb997fa700 10 client.31629.objecter ms_handle_connect 0x7fcb9c123e10 2024-02-23T00:53:52.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.690+0000 7fcb9b7fe700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fcb9c123e10 0x7fcb9c124270 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:52.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.690+0000 7fcb997fa700 1 -- 172.21.15.47:0/3015858306 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcb9c12b1a0 con 0x7fcb9c123e10 2024-02-23T00:53:52.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.690+0000 7fcb997fa700 10 client.31629.objecter ms_dispatch 0x7fcb9c001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:53:52.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.690+0000 7fcb997fa700 10 client.31629.objecter _finish_command 11 = system:0 2024-02-23T00:53:52.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.690+0000 7fcb997fa700 20 client.31629.objecter put_session s=0x7fcb9c11ce80 osd=5 4 2024-02-23T00:53:52.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.690+0000 7fcb997fa700 15 client.31629.objecter _session_command_op_remove 5 11 2024-02-23T00:53:52.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.705+0000 7fcba26cd700 10 client.31629.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:53:52.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.705+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_assign -1 12 2024-02-23T00:53:52.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.705+0000 7fcba26cd700 20 client.31629.objecter _get_session s=0x7fcb9c11ce80 osd=5 3 2024-02-23T00:53:52.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.705+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c11ce80 osd=5 3 2024-02-23T00:53:52.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.705+0000 7fcba26cd700 20 client.31629.objecter _get_session s=0x7fcb9c11ce80 osd=5 3 2024-02-23T00:53:52.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.705+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_remove -1 12 2024-02-23T00:53:52.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.705+0000 7fcba26cd700 20 client.31629.objecter get_session s=0x7fcb9c11ce80 osd=5 3 2024-02-23T00:53:52.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.705+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_assign 5 12 2024-02-23T00:53:52.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.705+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c11ce80 osd=5 4 2024-02-23T00:53:52.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.705+0000 7fcba26cd700 10 client.31629.objecter _send_command 12 2024-02-23T00:53:52.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.705+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fcb9c128a40 con 0x7fcb9c123e10 2024-02-23T00:53:52.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.707+0000 7fcb997fa700 1 -- 172.21.15.47:0/3015858306 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fcb9c128a40 con 0x7fcb9c123e10 2024-02-23T00:53:52.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.707+0000 7fcb997fa700 10 client.31629.objecter ms_dispatch 0x7fcb9c001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:53:52.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.707+0000 7fcb997fa700 10 client.31629.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:53:52.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.707+0000 7fcb997fa700 20 client.31629.objecter put_session s=0x7fcb9c11ce80 osd=5 4 2024-02-23T00:53:52.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.707+0000 7fcb997fa700 15 client.31629.objecter _session_command_op_remove 5 12 2024-02-23T00:53:52.710 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:53:52.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.707+0000 7fcba26cd700 10 client.31629.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:52.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.707+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_assign -1 13 2024-02-23T00:53:52.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.707+0000 7fcba26cd700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fcb9c1290c0 0x7fcb9c12b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:52.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.707+0000 7fcba26cd700 20 client.31629.objecter _get_session s=0x7fcb9c128a40 osd=6 3 2024-02-23T00:53:52.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.707+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c128a40 osd=6 3 2024-02-23T00:53:52.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.707+0000 7fcba26cd700 20 client.31629.objecter _get_session s=0x7fcb9c128a40 osd=6 3 2024-02-23T00:53:52.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.707+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_remove -1 13 2024-02-23T00:53:52.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.707+0000 7fcba26cd700 20 client.31629.objecter get_session s=0x7fcb9c128a40 osd=6 3 2024-02-23T00:53:52.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.707+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_assign 6 13 2024-02-23T00:53:52.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.707+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c128a40 osd=6 4 2024-02-23T00:53:52.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.707+0000 7fcba26cd700 10 client.31629.objecter _send_command 13 2024-02-23T00:53:52.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.707+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb9c12e310 con 0x7fcb9c1290c0 2024-02-23T00:53:52.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.707+0000 7fcba0c6a700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fcb9c1290c0 0x7fcb9c12b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:52.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.708+0000 7fcb997fa700 10 client.31629.objecter ms_handle_connect 0x7fcb9c1290c0 2024-02-23T00:53:52.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.708+0000 7fcba0c6a700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fcb9c1290c0 0x7fcb9c12b4d0 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:52.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.708+0000 7fcb997fa700 1 -- 172.21.15.47:0/3015858306 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcb9c12e310 con 0x7fcb9c1290c0 2024-02-23T00:53:52.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.708+0000 7fcb997fa700 10 client.31629.objecter ms_dispatch 0x7fcb9c001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:53:52.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.708+0000 7fcb997fa700 10 client.31629.objecter _finish_command 13 = system:0 2024-02-23T00:53:52.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.708+0000 7fcb997fa700 20 client.31629.objecter put_session s=0x7fcb9c128a40 osd=6 4 2024-02-23T00:53:52.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.708+0000 7fcb997fa700 15 client.31629.objecter _session_command_op_remove 6 13 2024-02-23T00:53:52.719 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:52.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.723+0000 7fcba26cd700 10 client.31629.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:53:52.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.723+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_assign -1 14 2024-02-23T00:53:52.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.723+0000 7fcba26cd700 20 client.31629.objecter _get_session s=0x7fcb9c128a40 osd=6 3 2024-02-23T00:53:52.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.723+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c128a40 osd=6 3 2024-02-23T00:53:52.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.723+0000 7fcba26cd700 20 client.31629.objecter _get_session s=0x7fcb9c128a40 osd=6 3 2024-02-23T00:53:52.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.723+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_remove -1 14 2024-02-23T00:53:52.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.723+0000 7fcba26cd700 20 client.31629.objecter get_session s=0x7fcb9c128a40 osd=6 3 2024-02-23T00:53:52.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.723+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_assign 6 14 2024-02-23T00:53:52.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.723+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c128a40 osd=6 4 2024-02-23T00:53:52.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.723+0000 7fcba26cd700 10 client.31629.objecter _send_command 14 2024-02-23T00:53:52.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.723+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fcb9c12e310 con 0x7fcb9c1290c0 2024-02-23T00:53:52.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.724+0000 7fcb997fa700 1 -- 172.21.15.47:0/3015858306 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fcb9c12e310 con 0x7fcb9c1290c0 2024-02-23T00:53:52.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.724+0000 7fcb997fa700 10 client.31629.objecter ms_dispatch 0x7fcb9c001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:53:52.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.724+0000 7fcb997fa700 10 client.31629.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:53:52.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.724+0000 7fcb997fa700 20 client.31629.objecter put_session s=0x7fcb9c128a40 osd=6 4 2024-02-23T00:53:52.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.724+0000 7fcb997fa700 15 client.31629.objecter _session_command_op_remove 6 14 2024-02-23T00:53:52.728 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:53:52.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.725+0000 7fcba26cd700 10 client.31629.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:53:52.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.725+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_assign -1 15 2024-02-23T00:53:52.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.725+0000 7fcba26cd700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fcb9c12d680 0x7fcb9c12db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:52.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.725+0000 7fcba26cd700 20 client.31629.objecter _get_session s=0x7fcb9c12e310 osd=7 3 2024-02-23T00:53:52.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.725+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c12e310 osd=7 3 2024-02-23T00:53:52.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.725+0000 7fcba26cd700 20 client.31629.objecter _get_session s=0x7fcb9c12e310 osd=7 3 2024-02-23T00:53:52.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.725+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_remove -1 15 2024-02-23T00:53:52.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.725+0000 7fcba26cd700 20 client.31629.objecter get_session s=0x7fcb9c12e310 osd=7 3 2024-02-23T00:53:52.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.725+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_assign 7 15 2024-02-23T00:53:52.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.725+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c12e310 osd=7 4 2024-02-23T00:53:52.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.725+0000 7fcba26cd700 10 client.31629.objecter _send_command 15 2024-02-23T00:53:52.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.725+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb9c131640 con 0x7fcb9c12d680 2024-02-23T00:53:52.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.725+0000 7fcb9bfff700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fcb9c12d680 0x7fcb9c12db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:52.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.725+0000 7fcb997fa700 10 client.31629.objecter ms_handle_connect 0x7fcb9c12d680 2024-02-23T00:53:52.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.725+0000 7fcb9bfff700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fcb9c12d680 0x7fcb9c12db00 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.726+0000 7fcb997fa700 1 -- 172.21.15.47:0/3015858306 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcb9c131640 con 0x7fcb9c12d680 2024-02-23T00:53:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.726+0000 7fcb997fa700 10 client.31629.objecter ms_dispatch 0x7fcb9c001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:53:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.726+0000 7fcb997fa700 10 client.31629.objecter _finish_command 15 = system:0 2024-02-23T00:53:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.726+0000 7fcb997fa700 20 client.31629.objecter put_session s=0x7fcb9c12e310 osd=7 4 2024-02-23T00:53:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.726+0000 7fcb997fa700 15 client.31629.objecter _session_command_op_remove 7 15 2024-02-23T00:53:52.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.740+0000 7fcba26cd700 10 client.31629.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:53:52.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.740+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_assign -1 16 2024-02-23T00:53:52.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.740+0000 7fcba26cd700 20 client.31629.objecter _get_session s=0x7fcb9c12e310 osd=7 3 2024-02-23T00:53:52.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.740+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c12e310 osd=7 3 2024-02-23T00:53:52.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.740+0000 7fcba26cd700 20 client.31629.objecter _get_session s=0x7fcb9c12e310 osd=7 3 2024-02-23T00:53:52.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.740+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_remove -1 16 2024-02-23T00:53:52.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.740+0000 7fcba26cd700 20 client.31629.objecter get_session s=0x7fcb9c12e310 osd=7 3 2024-02-23T00:53:52.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.740+0000 7fcba26cd700 15 client.31629.objecter _session_command_op_assign 7 16 2024-02-23T00:53:52.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.740+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c12e310 osd=7 4 2024-02-23T00:53:52.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.740+0000 7fcba26cd700 10 client.31629.objecter _send_command 16 2024-02-23T00:53:52.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.740+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fcb9c12bad0 con 0x7fcb9c12d680 2024-02-23T00:53:52.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.742+0000 7fcb997fa700 1 -- 172.21.15.47:0/3015858306 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fcb9c12bad0 con 0x7fcb9c12d680 2024-02-23T00:53:52.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.742+0000 7fcb997fa700 10 client.31629.objecter ms_dispatch 0x7fcb9c001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:53:52.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.742+0000 7fcb997fa700 10 client.31629.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:53:52.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.742+0000 7fcb997fa700 20 client.31629.objecter put_session s=0x7fcb9c12e310 osd=7 4 2024-02-23T00:53:52.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.742+0000 7fcb997fa700 15 client.31629.objecter _session_command_op_remove 7 16 2024-02-23T00:53:52.746 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:53:52.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 10 librados: watch_flush enter 2024-02-23T00:53:52.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 10 librados: watch_flush exit 2024-02-23T00:53:52.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 10 client.31629.objecter close_session for osd.0 2024-02-23T00:53:52.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fcb9c0fe5e0 msgr2=0x7fcb9c0fe9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:52.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fcb9c0fe5e0 0x7fcb9c0fe9c0 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:52.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c05ee80 osd=0 1 2024-02-23T00:53:52.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 10 client.31629.objecter close_session for osd.1 2024-02-23T00:53:52.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fcb9c0598f0 msgr2=0x7fcb9c059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:52.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fcb9c0598f0 0x7fcb9c059cd0 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:52.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c028710 osd=1 1 2024-02-23T00:53:52.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 10 client.31629.objecter close_session for osd.2 2024-02-23T00:53:52.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fcb9c028270 msgr2=0x7fcb9c11c250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:52.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fcb9c028270 0x7fcb9c11c250 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:52.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c059770 osd=2 1 2024-02-23T00:53:52.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba0c6a700 1 -- 172.21.15.47:0/3015858306 reap_dead start 2024-02-23T00:53:52.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 10 client.31629.objecter close_session for osd.3 2024-02-23T00:53:52.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fcb9c11f9b0 msgr2=0x7fcb9c121da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:52.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fcb9c11f9b0 0x7fcb9c121da0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:52.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c027b80 osd=3 1 2024-02-23T00:53:52.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 10 client.31629.objecter close_session for osd.4 2024-02-23T00:53:52.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fcb9c11bdb0 msgr2=0x7fcb9c123410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:52.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fcb9c11bdb0 0x7fcb9c123410 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:52.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c11cb20 osd=4 1 2024-02-23T00:53:52.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 10 client.31629.objecter close_session for osd.5 2024-02-23T00:53:52.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fcb9c123e10 msgr2=0x7fcb9c124270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:52.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fcb9c123e10 0x7fcb9c124270 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:52.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c11ce80 osd=5 1 2024-02-23T00:53:52.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 10 client.31629.objecter close_session for osd.6 2024-02-23T00:53:52.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fcb9c1290c0 msgr2=0x7fcb9c12b4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:52.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fcb9c1290c0 0x7fcb9c12b4d0 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:52.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c128a40 osd=6 1 2024-02-23T00:53:52.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 10 client.31629.objecter close_session for osd.7 2024-02-23T00:53:52.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fcb9c12d680 msgr2=0x7fcb9c12db00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:52.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fcb9c12d680 0x7fcb9c12db00 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:52.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 20 client.31629.objecter put_session s=0x7fcb9c12e310 osd=7 1 2024-02-23T00:53:52.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 20 client.31629.objecter shutdown clearing up homeless session... 2024-02-23T00:53:52.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 10 client.31629.objecter successfully canceled tick 2024-02-23T00:53:52.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba0c6a700 1 -- 172.21.15.47:0/3015858306 reap_dead start 2024-02-23T00:53:52.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcb88062cc0 msgr2=0x7fcb88065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:52.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcb88062cc0 0x7fcb88065180 secure :-1 s=READY pgs=3168 cs=0 l=1 rev1=1 rx=0x7fcb8c004e20 tx=0x7fcb8c009cb0).stop 2024-02-23T00:53:52.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb9c102210 msgr2=0x7fcb9c10a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:52.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb9c102210 0x7fcb9c10a3c0 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 rx=0x7fcb94001e80 tx=0x7fcb94004d10).stop 2024-02-23T00:53:52.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 shutdown_connections 2024-02-23T00:53:52.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcb88062cc0 0x7fcb88065180 unknown :-1 s=CLOSED pgs=3168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:52.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 1 --2- 172.21.15.47:0/3015858306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb9c102210 0x7fcb9c10a3c0 unknown :-1 s=CLOSED pgs=2619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:52.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 >> 172.21.15.47:0/3015858306 conn(0x7fcb9c0f2820 msgr2=0x7fcb9c0f4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:52.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 shutdown_connections 2024-02-23T00:53:52.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 1 -- 172.21.15.47:0/3015858306 wait complete. 2024-02-23T00:53:52.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:52.743+0000 7fcba26cd700 1 librados: shutdown 2024-02-23T00:53:52.750 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:53:52.750 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:53:52.750 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:53:52.751 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:53:52.751 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:53:52.751 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:53:52.751 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:53:52.751 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:53:52.765 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:51.376419+0000 mon.a (mon.0) 3409 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:53:52.765 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:51.376491+0000 mon.a (mon.0) 3410 : cluster [DBG] osdmap e692: 8 total, 8 up, 6 in 2024-02-23T00:53:52.765 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:51.377405+0000 mon.c (mon.2) 402 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-120"}]: dispatch 2024-02-23T00:53:52.765 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:51.377510+0000 mgr.y (mgr.14517) 599 : cluster [DBG] pgmap v1047: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:53:52.765 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:51.381261+0000 mon.a (mon.0) 3411 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-120"}]: dispatch 2024-02-23T00:53:52.766 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:51.479039+0000 mon.c (mon.2) 403 : audit [INF] from='client.? 172.21.15.47:0/2302355375' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-23T00:53:52.766 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:51.479343+0000 mon.a (mon.0) 3412 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-23T00:53:52.766 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:51.376419+0000 mon.a (mon.0) 3409 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:53:52.766 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:51.376491+0000 mon.a (mon.0) 3410 : cluster [DBG] osdmap e692: 8 total, 8 up, 6 in 2024-02-23T00:53:52.766 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:51.377405+0000 mon.c (mon.2) 402 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-120"}]: dispatch 2024-02-23T00:53:52.766 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:51.377510+0000 mgr.y (mgr.14517) 599 : cluster [DBG] pgmap v1047: 184 pgs: 32 unknown, 152 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:53:52.766 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:51.381261+0000 mon.a (mon.0) 3411 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-120"}]: dispatch 2024-02-23T00:53:52.766 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:51.479039+0000 mon.c (mon.2) 403 : audit [INF] from='client.? 172.21.15.47:0/2302355375' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-23T00:53:52.766 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:51.479343+0000 mon.a (mon.0) 3412 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-23T00:53:52.817 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:53:52.818 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:53:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:53:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:53:52.820 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:52.820 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:53:52.921 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:53:52.953 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:52.972 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:53:52.975 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:52.975 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:53:53.057 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:53:53.057 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:53:53.076 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:53:53.079 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:53:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:53:53.106 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:53.305 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:53:53.308 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:53.308 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:53:53.399 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:52.382707+0000 mon.a (mon.0) 3413 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-120"}]': finished 2024-02-23T00:53:53.399 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:52.382798+0000 mon.a (mon.0) 3414 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-23T00:53:53.399 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:52.382863+0000 mon.a (mon.0) 3415 : cluster [DBG] osdmap e693: 8 total, 8 up, 6 in 2024-02-23T00:53:53.399 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:52.384309+0000 mon.c (mon.2) 404 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-120", "mode": "writeback"}]: dispatch 2024-02-23T00:53:53.399 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:52.393404+0000 mon.a (mon.0) 3416 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-120", "mode": "writeback"}]: dispatch 2024-02-23T00:53:53.399 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:52.602440+0000 mon.a (mon.0) 3417 : audit [DBG] from='client.? 172.21.15.47:0/3015858306' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:53:53.400 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:52.382707+0000 mon.a (mon.0) 3413 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-120"}]': finished 2024-02-23T00:53:53.400 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:52.382798+0000 mon.a (mon.0) 3414 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-23T00:53:53.400 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:52.382863+0000 mon.a (mon.0) 3415 : cluster [DBG] osdmap e693: 8 total, 8 up, 6 in 2024-02-23T00:53:53.400 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:52.384309+0000 mon.c (mon.2) 404 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-120", "mode": "writeback"}]: dispatch 2024-02-23T00:53:53.400 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:52.393404+0000 mon.a (mon.0) 3416 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-120", "mode": "writeback"}]: dispatch 2024-02-23T00:53:53.400 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:52.602440+0000 mon.a (mon.0) 3417 : audit [DBG] from='client.? 172.21.15.47:0/3015858306' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:53:53.409 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:53:53.453 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:53.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:52.382707+0000 mon.a (mon.0) 3413 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-120"}]': finished 2024-02-23T00:53:53.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:52.382798+0000 mon.a (mon.0) 3414 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-23T00:53:53.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:52.382863+0000 mon.a (mon.0) 3415 : cluster [DBG] osdmap e693: 8 total, 8 up, 6 in 2024-02-23T00:53:53.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:52.384309+0000 mon.c (mon.2) 404 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-120", "mode": "writeback"}]: dispatch 2024-02-23T00:53:53.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:52.393404+0000 mon.a (mon.0) 3416 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-120", "mode": "writeback"}]: dispatch 2024-02-23T00:53:53.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:52.602440+0000 mon.a (mon.0) 3417 : audit [DBG] from='client.? 172.21.15.47:0/3015858306' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:53:53.767 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.0 dump_ops_in_flight 2024-02-23T00:53:53.806 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:53:53.809 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:53.809 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:53:53.826 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight 4 1 2024-02-23T00:53:53.866 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:53:53.942 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:53:53.971 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:54.186 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:53:54.190 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:54.190 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:53:54.291 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:53:54.320 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:54.345 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:53:54.347 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:54.348 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:53:54.449 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:53:54.476 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:54.674 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:53:54.675 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:53:54.675 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:53:54.677 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:52.885936+0000 mgr.y (mgr.14517) 600 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:53:54.677 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:53.378092+0000 mgr.y (mgr.14517) 601 : cluster [DBG] pgmap v1049: 184 pgs: 184 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:53:54.677 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:53.389494+0000 mon.a (mon.0) 3418 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:53:54.677 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:53.396936+0000 mon.a (mon.0) 3419 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-120", "mode": "writeback"}]': finished 2024-02-23T00:53:54.677 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:53.397107+0000 mon.a (mon.0) 3420 : cluster [DBG] osdmap e694: 8 total, 8 up, 6 in 2024-02-23T00:53:54.677 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:53.495844+0000 mon.c (mon.2) 405 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:54.678 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:53.496200+0000 mon.a (mon.0) 3421 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:54.678 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:52.885936+0000 mgr.y (mgr.14517) 600 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:53:54.678 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:53.378092+0000 mgr.y (mgr.14517) 601 : cluster [DBG] pgmap v1049: 184 pgs: 184 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:53:54.678 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:53.389494+0000 mon.a (mon.0) 3418 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:53:54.678 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:53.396936+0000 mon.a (mon.0) 3419 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-120", "mode": "writeback"}]': finished 2024-02-23T00:53:54.678 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:53.397107+0000 mon.a (mon.0) 3420 : cluster [DBG] osdmap e694: 8 total, 8 up, 6 in 2024-02-23T00:53:54.678 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:53.495844+0000 mon.c (mon.2) 405 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:54.678 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:53.496200+0000 mon.a (mon.0) 3421 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:54.679 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:54.679 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:53:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:52.885936+0000 mgr.y (mgr.14517) 600 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:53:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:53.378092+0000 mgr.y (mgr.14517) 601 : cluster [DBG] pgmap v1049: 184 pgs: 184 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:53:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:53.389494+0000 mon.a (mon.0) 3418 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:53:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:53.396936+0000 mon.a (mon.0) 3419 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-120", "mode": "writeback"}]': finished 2024-02-23T00:53:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:53.397107+0000 mon.a (mon.0) 3420 : cluster [DBG] osdmap e694: 8 total, 8 up, 6 in 2024-02-23T00:53:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:53.495844+0000 mon.c (mon.2) 405 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:54.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:53.496200+0000 mon.a (mon.0) 3421 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:53:54.780 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:53:54.808 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:54.826 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:53:54.828 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:54.829 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:53:54.930 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:53:54.958 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:54.986 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:53:54.988 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:54.989 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:53:55.090 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:53:55.116 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:55.135 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:53:55.137 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:55.138 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:53:55.239 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:53:55.268 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:55.368 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:53:55.371 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:55.371 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:53:55.469 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:53:55.472 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:53:55.498 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:55.516 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:53:55.518 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:55.518 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:53:55.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:54.397395+0000 mon.a (mon.0) 3422 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:53:55.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:54.397467+0000 mon.a (mon.0) 3423 : cluster [DBG] osdmap e695: 8 total, 8 up, 6 in 2024-02-23T00:53:55.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:54.397894+0000 osd.5 (osd.5) 305 : cluster [DBG] 6.0 starting backfill to osd.3 from (0'0,0'0] MAX to 107'2 2024-02-23T00:53:55.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:54.398787+0000 osd.1 (osd.1) 282 : cluster [DBG] 6.4 starting backfill to osd.3 from (0'0,0'0] MAX to 67'1 2024-02-23T00:53:55.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:54.399703+0000 mon.c (mon.2) 406 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-120"}]: dispatch 2024-02-23T00:53:55.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:54.401426+0000 mon.a (mon.0) 3424 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-120"}]: dispatch 2024-02-23T00:53:55.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:55.281876+0000 mon.a (mon.0) 3425 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:53:55.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:55.287054+0000 mon.a (mon.0) 3426 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-120"}]': finished 2024-02-23T00:53:55.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:55.287166+0000 mon.a (mon.0) 3427 : cluster [DBG] osdmap e696: 8 total, 8 up, 6 in 2024-02-23T00:53:55.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:54.397395+0000 mon.a (mon.0) 3422 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:53:55.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:54.397467+0000 mon.a (mon.0) 3423 : cluster [DBG] osdmap e695: 8 total, 8 up, 6 in 2024-02-23T00:53:55.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:54.397894+0000 osd.5 (osd.5) 305 : cluster [DBG] 6.0 starting backfill to osd.3 from (0'0,0'0] MAX to 107'2 2024-02-23T00:53:55.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:54.398787+0000 osd.1 (osd.1) 282 : cluster [DBG] 6.4 starting backfill to osd.3 from (0'0,0'0] MAX to 67'1 2024-02-23T00:53:55.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:54.399703+0000 mon.c (mon.2) 406 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-120"}]: dispatch 2024-02-23T00:53:55.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:54.401426+0000 mon.a (mon.0) 3424 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-120"}]: dispatch 2024-02-23T00:53:55.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:55.281876+0000 mon.a (mon.0) 3425 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:53:55.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:55.287054+0000 mon.a (mon.0) 3426 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-120"}]': finished 2024-02-23T00:53:55.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:55.287166+0000 mon.a (mon.0) 3427 : cluster [DBG] osdmap e696: 8 total, 8 up, 6 in 2024-02-23T00:53:55.619 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:53:55.651 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:55.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:54.397395+0000 mon.a (mon.0) 3422 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:53:55.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:54.397467+0000 mon.a (mon.0) 3423 : cluster [DBG] osdmap e695: 8 total, 8 up, 6 in 2024-02-23T00:53:55.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:54.397894+0000 osd.5 (osd.5) 305 : cluster [DBG] 6.0 starting backfill to osd.3 from (0'0,0'0] MAX to 107'2 2024-02-23T00:53:55.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:54.398787+0000 osd.1 (osd.1) 282 : cluster [DBG] 6.4 starting backfill to osd.3 from (0'0,0'0] MAX to 67'1 2024-02-23T00:53:55.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:54.399703+0000 mon.c (mon.2) 406 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-120"}]: dispatch 2024-02-23T00:53:55.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:54.401426+0000 mon.a (mon.0) 3424 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-120"}]: dispatch 2024-02-23T00:53:55.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:55.281876+0000 mon.a (mon.0) 3425 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:53:55.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:55.287054+0000 mon.a (mon.0) 3426 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-120"}]': finished 2024-02-23T00:53:55.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:55.287166+0000 mon.a (mon.0) 3427 : cluster [DBG] osdmap e696: 8 total, 8 up, 6 in 2024-02-23T00:53:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:53:56.608 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:56.608 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:53:56.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.871+0000 7f8d513dd700 1 -- 172.21.15.47:0/702054694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4c0f8bb0 msgr2=0x7f8d4c0f8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:56.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.871+0000 7f8d513dd700 1 --2- 172.21.15.47:0/702054694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4c0f8bb0 0x7f8d4c0f8f90 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 rx=0x7f8d3c004870 tx=0x7f8d3c009cb0).stop 2024-02-23T00:53:56.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.871+0000 7f8d513dd700 1 -- 172.21.15.47:0/702054694 shutdown_connections 2024-02-23T00:53:56.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.871+0000 7f8d513dd700 1 --2- 172.21.15.47:0/702054694 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8d4c102110 0x7f8d4c104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.871+0000 7f8d513dd700 1 --2- 172.21.15.47:0/702054694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4c0f8bb0 0x7f8d4c0f8f90 unknown :-1 s=CLOSED pgs=2620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.871+0000 7f8d513dd700 1 --2- 172.21.15.47:0/702054694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d4c0f9510 0x7f8d4c101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.871+0000 7f8d513dd700 1 -- 172.21.15.47:0/702054694 >> 172.21.15.47:0/702054694 conn(0x7f8d4c0f26c0 msgr2=0x7f8d4c0f4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.871+0000 7f8d513dd700 1 -- 172.21.15.47:0/702054694 shutdown_connections 2024-02-23T00:53:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.872+0000 7f8d513dd700 1 -- 172.21.15.47:0/702054694 wait complete. 2024-02-23T00:53:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.872+0000 7f8d513dd700 1 librados: starting msgr at 2024-02-23T00:53:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.872+0000 7f8d513dd700 1 librados: starting objecter 2024-02-23T00:53:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.872+0000 7f8d513dd700 1 Processor -- start 2024-02-23T00:53:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.873+0000 7f8d513dd700 1 -- start start 2024-02-23T00:53:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.873+0000 7f8d513dd700 1 librados: setting wanted keys 2024-02-23T00:53:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.873+0000 7f8d513dd700 1 librados: calling monclient init 2024-02-23T00:53:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.873+0000 7f8d513dd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4c0f8bb0 0x7f8d4c10e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.873+0000 7f8d513dd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d4c0f9510 0x7f8d4c10e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.873+0000 7f8d513dd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8d4c102110 0x7f8d4c1081e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.873+0000 7f8d513dd700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c1086d0 con 0x7f8d4c0f8bb0 2024-02-23T00:53:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.873+0000 7f8d513dd700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c108840 con 0x7f8d4c0f9510 2024-02-23T00:53:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.873+0000 7f8d513dd700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c108a00 con 0x7f8d4c102110 2024-02-23T00:53:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.873+0000 7f8d4affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4c0f8bb0 0x7f8d4c10e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.873+0000 7f8d4affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4c0f8bb0 0x7f8d4c10e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49232/0 (socket says 172.21.15.47:49232) 2024-02-23T00:53:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.873+0000 7f8d4affd700 1 -- 172.21.15.47:0/935307023 learned_addr learned my addr 172.21.15.47:0/935307023 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:53:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.873+0000 7f8d4b7fe700 1 --2- 172.21.15.47:0/935307023 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8d4c102110 0x7f8d4c1081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.873+0000 7f8d4a7fc700 1 --2- 172.21.15.47:0/935307023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d4c0f9510 0x7f8d4c10e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.873+0000 7f8d4affd700 1 -- 172.21.15.47:0/935307023 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8d4c102110 msgr2=0x7f8d4c1081e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.873+0000 7f8d4affd700 1 --2- 172.21.15.47:0/935307023 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8d4c102110 0x7f8d4c1081e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.873+0000 7f8d4affd700 1 -- 172.21.15.47:0/935307023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d4c0f9510 msgr2=0x7f8d4c10e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.873+0000 7f8d4affd700 1 --2- 172.21.15.47:0/935307023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d4c0f9510 0x7f8d4c10e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.873+0000 7f8d4affd700 1 -- 172.21.15.47:0/935307023 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d440097c0 con 0x7f8d4c0f8bb0 2024-02-23T00:53:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.873+0000 7f8d4affd700 1 --2- 172.21.15.47:0/935307023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4c0f8bb0 0x7f8d4c10e040 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 rx=0x7f8d3c000e50 tx=0x7f8d3c009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.873+0000 7f8d33fff700 10 client.?.objecter ms_handle_connect 0x7f8d4c0f8bb0 2024-02-23T00:53:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.873+0000 7f8d33fff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:53:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.874+0000 7f8d33fff700 1 -- 172.21.15.47:0/935307023 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c00e030 con 0x7f8d4c0f8bb0 2024-02-23T00:53:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.874+0000 7f8d33fff700 1 -- 172.21.15.47:0/935307023 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f8d3c013d70 con 0x7f8d4c0f8bb0 2024-02-23T00:53:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.874+0000 7f8d33fff700 1 -- 172.21.15.47:0/935307023 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c018070 con 0x7f8d4c0f8bb0 2024-02-23T00:53:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.874+0000 7f8d513dd700 1 -- 172.21.15.47:0/935307023 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d3c005040 con 0x7f8d4c0f8bb0 2024-02-23T00:53:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.874+0000 7f8d513dd700 10 client.31635.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:53:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.874+0000 7f8d513dd700 1 -- 172.21.15.47:0/935307023 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d4c108ea0 con 0x7f8d4c0f8bb0 2024-02-23T00:53:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.874+0000 7f8d513dd700 1 librados: init done 2024-02-23T00:53:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.875+0000 7f8d513dd700 1 -- 172.21.15.47:0/935307023 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d4c109090 con 0x7f8d4c0f8bb0 2024-02-23T00:53:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.875+0000 7f8d33fff700 1 -- 172.21.15.47:0/935307023 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f8d3c009240 con 0x7f8d4c0f8bb0 2024-02-23T00:53:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.875+0000 7f8d33fff700 1 --2- 172.21.15.47:0/935307023 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8d34062cc0 0x7f8d34065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.875+0000 7f8d33fff700 1 -- 172.21.15.47:0/935307023 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(696..696 src has 482..696) v4 ==== 9716+0+0 (secure 0 0 0) 0x7f8d3c07e980 con 0x7f8d4c0f8bb0 2024-02-23T00:53:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.875+0000 7f8d33fff700 10 client.31635.objecter ms_dispatch 0x7f8d4c001550 osd_map(696..696 src has 482..696) v4 2024-02-23T00:53:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.875+0000 7f8d33fff700 3 client.31635.objecter handle_osd_map got epochs [696,696] > 0 2024-02-23T00:53:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.875+0000 7f8d33fff700 3 client.31635.objecter handle_osd_map decoding full epoch 696 2024-02-23T00:53:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.876+0000 7f8d4a7fc700 1 --2- 172.21.15.47:0/935307023 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8d34062cc0 0x7f8d34065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.876+0000 7f8d33fff700 20 client.31635.objecter dump_active .. 0 homeless 2024-02-23T00:53:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.876+0000 7f8d33fff700 10 client.31635.objecter ms_handle_connect 0x7f8d34062cc0 2024-02-23T00:53:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.876+0000 7f8d4a7fc700 1 --2- 172.21.15.47:0/935307023 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8d34062cc0 0x7f8d34065180 secure :-1 s=READY pgs=3169 cs=0 l=1 rev1=1 rx=0x7f8d4c109980 tx=0x7f8d4000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:55.880+0000 7f8d33fff700 1 -- 172.21.15.47:0/935307023 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f8d3c04de20 con 0x7f8d4c0f8bb0 2024-02-23T00:53:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:56.064+0000 7f8d513dd700 1 -- 172.21.15.47:0/935307023 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7f8d4c106f00 con 0x7f8d4c0f8bb0 2024-02-23T00:53:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:56.294+0000 7f8d33fff700 1 -- 172.21.15.47:0/935307023 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v697) v1 ==== 91+0+0 (secure 0 0 0) 0x7f8d3c051440 con 0x7f8d4c0f8bb0 2024-02-23T00:53:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:56.297+0000 7f8d513dd700 10 librados: watch_flush enter 2024-02-23T00:53:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:56.297+0000 7f8d513dd700 10 librados: watch_flush exit 2024-02-23T00:53:56.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:56.297+0000 7f8d513dd700 20 client.31635.objecter shutdown clearing up homeless session... 2024-02-23T00:53:56.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:56.297+0000 7f8d513dd700 10 client.31635.objecter successfully canceled tick 2024-02-23T00:53:56.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:56.297+0000 7f8d513dd700 1 -- 172.21.15.47:0/935307023 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8d34062cc0 msgr2=0x7f8d34065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:56.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:56.297+0000 7f8d513dd700 1 --2- 172.21.15.47:0/935307023 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8d34062cc0 0x7f8d34065180 secure :-1 s=READY pgs=3169 cs=0 l=1 rev1=1 rx=0x7f8d4c109980 tx=0x7f8d4000b040).stop 2024-02-23T00:53:56.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:56.297+0000 7f8d513dd700 1 -- 172.21.15.47:0/935307023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4c0f8bb0 msgr2=0x7f8d4c10e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:56.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:56.297+0000 7f8d513dd700 1 --2- 172.21.15.47:0/935307023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4c0f8bb0 0x7f8d4c10e040 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 rx=0x7f8d3c000e50 tx=0x7f8d3c009cb0).stop 2024-02-23T00:53:56.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:56.298+0000 7f8d513dd700 1 -- 172.21.15.47:0/935307023 shutdown_connections 2024-02-23T00:53:56.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:56.298+0000 7f8d513dd700 1 --2- 172.21.15.47:0/935307023 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8d34062cc0 0x7f8d34065180 unknown :-1 s=CLOSED pgs=3169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:56.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:56.298+0000 7f8d513dd700 1 --2- 172.21.15.47:0/935307023 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8d4c102110 0x7f8d4c1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:56.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:56.298+0000 7f8d513dd700 1 --2- 172.21.15.47:0/935307023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4c0f8bb0 0x7f8d4c10e040 unknown :-1 s=CLOSED pgs=2621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:56.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:56.298+0000 7f8d513dd700 1 --2- 172.21.15.47:0/935307023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d4c0f9510 0x7f8d4c10e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:56.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:56.298+0000 7f8d513dd700 1 -- 172.21.15.47:0/935307023 >> 172.21.15.47:0/935307023 conn(0x7f8d4c0f26c0 msgr2=0x7f8d4c0f3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:56.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:56.298+0000 7f8d513dd700 1 -- 172.21.15.47:0/935307023 shutdown_connections 2024-02-23T00:53:56.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:56.298+0000 7f8d513dd700 1 -- 172.21.15.47:0/935307023 wait complete. 2024-02-23T00:53:56.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:56.298+0000 7f8d513dd700 1 librados: shutdown 2024-02-23T00:53:56.703 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is unset 2024-02-23T00:53:56.709 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:53:56.742 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:56.764 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:53:56.766 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:56.767 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:53:56.868 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:53:56.897 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:56.917 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:53:56.920 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:56.920 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:53:57.021 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:53:57.048 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:55.378415+0000 mgr.y (mgr.14517) 602 : cluster [DBG] pgmap v1053: 184 pgs: 184 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:53:57.048 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:56.066105+0000 mon.a (mon.0) 3428 : audit [INF] from='client.? 172.21.15.47:0/935307023' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:53:57.049 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:56.289220+0000 mon.a (mon.0) 3429 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:53:57.049 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:56.293550+0000 mon.a (mon.0) 3430 : audit [INF] from='client.? 172.21.15.47:0/935307023' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:53:57.049 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:56.293616+0000 mon.a (mon.0) 3431 : cluster [DBG] osdmap e697: 8 total, 8 up, 6 in 2024-02-23T00:53:57.049 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:55.378415+0000 mgr.y (mgr.14517) 602 : cluster [DBG] pgmap v1053: 184 pgs: 184 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:53:57.049 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:56.066105+0000 mon.a (mon.0) 3428 : audit [INF] from='client.? 172.21.15.47:0/935307023' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:53:57.049 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:56.289220+0000 mon.a (mon.0) 3429 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:53:57.049 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:56.293550+0000 mon.a (mon.0) 3430 : audit [INF] from='client.? 172.21.15.47:0/935307023' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:53:57.049 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:56.293616+0000 mon.a (mon.0) 3431 : cluster [DBG] osdmap e697: 8 total, 8 up, 6 in 2024-02-23T00:53:57.051 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:57.070 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:53:57.073 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:57.073 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:53:57.174 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:53:57.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:55.378415+0000 mgr.y (mgr.14517) 602 : cluster [DBG] pgmap v1053: 184 pgs: 184 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:53:57.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:56.066105+0000 mon.a (mon.0) 3428 : audit [INF] from='client.? 172.21.15.47:0/935307023' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:53:57.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:56.289220+0000 mon.a (mon.0) 3429 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:53:57.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:56.293550+0000 mon.a (mon.0) 3430 : audit [INF] from='client.? 172.21.15.47:0/935307023' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:53:57.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:56.293616+0000 mon.a (mon.0) 3431 : cluster [DBG] osdmap e697: 8 total, 8 up, 6 in 2024-02-23T00:53:57.202 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:57.302 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:53:57.326 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:57.326 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:53:57.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.325+0000 7f870d90c700 1 -- 172.21.15.47:0/579857268 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87080f8cb0 msgr2=0x7f87080f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:57.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.325+0000 7f870d90c700 1 --2- 172.21.15.47:0/579857268 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87080f8cb0 0x7f87080f9090 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 rx=0x7f86f80047b0 tx=0x7f86f8009cb0).stop 2024-02-23T00:53:57.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.325+0000 7f870d90c700 1 -- 172.21.15.47:0/579857268 shutdown_connections 2024-02-23T00:53:57.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.325+0000 7f870d90c700 1 --2- 172.21.15.47:0/579857268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8708102210 0x7f8708104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:57.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.325+0000 7f870d90c700 1 --2- 172.21.15.47:0/579857268 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87080f8cb0 0x7f87080f9090 unknown :-1 s=CLOSED pgs=2778 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:57.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.325+0000 7f870d90c700 1 --2- 172.21.15.47:0/579857268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87080f9610 0x7f8708101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:57.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.325+0000 7f870d90c700 1 -- 172.21.15.47:0/579857268 >> 172.21.15.47:0/579857268 conn(0x7f87080f2820 msgr2=0x7f87080f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:57.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.325+0000 7f870d90c700 1 -- 172.21.15.47:0/579857268 shutdown_connections 2024-02-23T00:53:57.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.325+0000 7f870d90c700 1 -- 172.21.15.47:0/579857268 wait complete. 2024-02-23T00:53:57.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.326+0000 7f870d90c700 1 librados: starting msgr at 2024-02-23T00:53:57.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.326+0000 7f870d90c700 1 librados: starting objecter 2024-02-23T00:53:57.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.326+0000 7f870d90c700 1 Processor -- start 2024-02-23T00:53:57.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.326+0000 7f870d90c700 1 -- start start 2024-02-23T00:53:57.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.326+0000 7f870d90c700 1 librados: setting wanted keys 2024-02-23T00:53:57.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.326+0000 7f870d90c700 1 librados: calling monclient init 2024-02-23T00:53:57.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.326+0000 7f870d90c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87080f8cb0 0x7f870810e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:57.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.326+0000 7f870d90c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87080f9610 0x7f870810e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:57.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.326+0000 7f870d90c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8708102210 0x7f8708108230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:57.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.326+0000 7f870d90c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8708108720 con 0x7f8708102210 2024-02-23T00:53:57.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.326+0000 7f870d90c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8708108890 con 0x7f87080f9610 2024-02-23T00:53:57.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.326+0000 7f870d90c700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8708108a50 con 0x7f87080f8cb0 2024-02-23T00:53:57.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.326+0000 7f8706ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87080f8cb0 0x7f870810e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:57.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.326+0000 7f87077fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8708102210 0x7f8708108230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:57.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.327+0000 7f8706ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87080f8cb0 0x7f870810e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:51330/0 (socket says 172.21.15.47:51330) 2024-02-23T00:53:57.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.327+0000 7f8706ffd700 1 -- 172.21.15.47:0/602669385 learned_addr learned my addr 172.21.15.47:0/602669385 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:53:57.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.327+0000 7f87067fc700 1 --2- 172.21.15.47:0/602669385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87080f9610 0x7f870810e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:57.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.327+0000 7f8706ffd700 1 -- 172.21.15.47:0/602669385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87080f9610 msgr2=0x7f870810e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:57.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.327+0000 7f8706ffd700 1 --2- 172.21.15.47:0/602669385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87080f9610 0x7f870810e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:57.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.327+0000 7f8706ffd700 1 -- 172.21.15.47:0/602669385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8708102210 msgr2=0x7f8708108230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:57.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.327+0000 7f8706ffd700 1 --2- 172.21.15.47:0/602669385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8708102210 0x7f8708108230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:57.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.327+0000 7f8706ffd700 1 -- 172.21.15.47:0/602669385 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86fc005040 con 0x7f87080f8cb0 2024-02-23T00:53:57.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.327+0000 7f87077fe700 1 --2- 172.21.15.47:0/602669385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8708102210 0x7f8708108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:53:57.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.327+0000 7f8706ffd700 1 --2- 172.21.15.47:0/602669385 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87080f8cb0 0x7f870810e090 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 rx=0x7f86f8009f20 tx=0x7f86f8009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:57.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.327+0000 7f86effff700 10 client.?.objecter ms_handle_connect 0x7f87080f8cb0 2024-02-23T00:53:57.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.327+0000 7f86effff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:53:57.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.327+0000 7f86effff700 1 -- 172.21.15.47:0/602669385 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f8018070 con 0x7f87080f8cb0 2024-02-23T00:53:57.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.327+0000 7f870d90c700 1 -- 172.21.15.47:0/602669385 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f8005040 con 0x7f87080f8cb0 2024-02-23T00:53:57.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.327+0000 7f86effff700 1 -- 172.21.15.47:0/602669385 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f86f8013dd0 con 0x7f87080f8cb0 2024-02-23T00:53:57.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.327+0000 7f86effff700 1 -- 172.21.15.47:0/602669385 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f801da10 con 0x7f87080f8cb0 2024-02-23T00:53:57.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.327+0000 7f870d90c700 10 client.41963.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:53:57.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.327+0000 7f870d90c700 1 -- 172.21.15.47:0/602669385 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8708108ef0 con 0x7f87080f8cb0 2024-02-23T00:53:57.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.327+0000 7f870d90c700 1 librados: init done 2024-02-23T00:53:57.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.328+0000 7f870d90c700 1 -- 172.21.15.47:0/602669385 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f870805fa60 con 0x7f87080f8cb0 2024-02-23T00:53:57.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.328+0000 7f86effff700 1 -- 172.21.15.47:0/602669385 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f86f8009240 con 0x7f87080f8cb0 2024-02-23T00:53:57.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.329+0000 7f86effff700 1 --2- 172.21.15.47:0/602669385 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f86f0062cc0 0x7f86f0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:53:57.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.329+0000 7f86effff700 1 -- 172.21.15.47:0/602669385 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(698..698 src has 482..698) v4 ==== 9716+0+0 (secure 0 0 0) 0x7f86f807eb30 con 0x7f87080f8cb0 2024-02-23T00:53:57.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.329+0000 7f86effff700 10 client.41963.objecter ms_dispatch 0x7f8708001550 osd_map(698..698 src has 482..698) v4 2024-02-23T00:53:57.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.329+0000 7f86effff700 3 client.41963.objecter handle_osd_map got epochs [698,698] > 0 2024-02-23T00:53:57.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.329+0000 7f86effff700 3 client.41963.objecter handle_osd_map decoding full epoch 698 2024-02-23T00:53:57.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.329+0000 7f87067fc700 1 --2- 172.21.15.47:0/602669385 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f86f0062cc0 0x7f86f0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:53:57.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.329+0000 7f86effff700 20 client.41963.objecter dump_active .. 0 homeless 2024-02-23T00:53:57.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.329+0000 7f87067fc700 1 --2- 172.21.15.47:0/602669385 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f86f0062cc0 0x7f86f0065180 secure :-1 s=READY pgs=3170 cs=0 l=1 rev1=1 rx=0x7f87081099d0 tx=0x7f86fc00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:53:57.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.329+0000 7f86effff700 10 client.41963.objecter ms_handle_connect 0x7f86f0062cc0 2024-02-23T00:53:57.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.333+0000 7f86effff700 1 -- 172.21.15.47:0/602669385 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f86f804df50 con 0x7f87080f8cb0 2024-02-23T00:53:57.427 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:53:57.455 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:57.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:57.523+0000 7f870d90c700 1 -- 172.21.15.47:0/602669385 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd reweight", "id": 4, "weight": 1.0} v 0) v1 -- 0x7f87081070e0 con 0x7f87080f8cb0 2024-02-23T00:53:57.654 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:53:57.657 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:57.657 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:53:57.759 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:53:57.788 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:57.863 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:53:57.865 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:57.866 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:53:57.920 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:55.755674+0000 osd.2 (osd.2) 169 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:53:57.920 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:55.755952+0000 osd.2 (osd.2) 170 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:53:57.920 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:56.753420+0000 mon.a (mon.0) 3432 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:53:57.920 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:57.305555+0000 mon.a (mon.0) 3433 : cluster [DBG] osdmap e698: 8 total, 8 up, 6 in 2024-02-23T00:53:57.920 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:57.307853+0000 mon.c (mon.2) 407 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:57.920 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:57.308365+0000 mon.a (mon.0) 3434 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:57.920 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:57.378803+0000 mgr.y (mgr.14517) 603 : cluster [DBG] pgmap v1056: 184 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 151 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 256 B/s wr, 2 op/s; 1/714 objects misplaced (0.140%); 370 B/s, 20 objects/s recovering 2024-02-23T00:53:57.920 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:57.525243+0000 mon.c (mon.2) 408 : audit [INF] from='client.? 172.21.15.47:0/602669385' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-23T00:53:57.920 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:57.525559+0000 mon.a (mon.0) 3435 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-23T00:53:57.921 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:55.755674+0000 osd.2 (osd.2) 169 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:53:57.921 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:55.755952+0000 osd.2 (osd.2) 170 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:53:57.921 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:56.753420+0000 mon.a (mon.0) 3432 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:53:57.921 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:57.305555+0000 mon.a (mon.0) 3433 : cluster [DBG] osdmap e698: 8 total, 8 up, 6 in 2024-02-23T00:53:57.921 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:57.307853+0000 mon.c (mon.2) 407 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:57.921 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:57.308365+0000 mon.a (mon.0) 3434 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:57.921 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:57.378803+0000 mgr.y (mgr.14517) 603 : cluster [DBG] pgmap v1056: 184 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 151 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 256 B/s wr, 2 op/s; 1/714 objects misplaced (0.140%); 370 B/s, 20 objects/s recovering 2024-02-23T00:53:57.921 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:57.525243+0000 mon.c (mon.2) 408 : audit [INF] from='client.? 172.21.15.47:0/602669385' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-23T00:53:57.921 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:57.525559+0000 mon.a (mon.0) 3435 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-23T00:53:57.967 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:53:57.996 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:58.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:55.755674+0000 osd.2 (osd.2) 169 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:53:58.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:55.755952+0000 osd.2 (osd.2) 170 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:53:58.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:56.753420+0000 mon.a (mon.0) 3432 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:53:58.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:57.305555+0000 mon.a (mon.0) 3433 : cluster [DBG] osdmap e698: 8 total, 8 up, 6 in 2024-02-23T00:53:58.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:57.307853+0000 mon.c (mon.2) 407 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:58.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:57.308365+0000 mon.a (mon.0) 3434 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:53:58.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:57.378803+0000 mgr.y (mgr.14517) 603 : cluster [DBG] pgmap v1056: 184 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 151 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 256 B/s wr, 2 op/s; 1/714 objects misplaced (0.140%); 370 B/s, 20 objects/s recovering 2024-02-23T00:53:58.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:57.525243+0000 mon.c (mon.2) 408 : audit [INF] from='client.? 172.21.15.47:0/602669385' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-23T00:53:58.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:57.525559+0000 mon.a (mon.0) 3435 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-23T00:53:58.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:58.307+0000 7f86effff700 1 -- 172.21.15.47:0/602669385 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 4, "weight": 1.0}]=0 reweighted osd.4 to 1 (10000) v699) v1 ==== 113+0+0 (secure 0 0 0) 0x7f86f80043d0 con 0x7f87080f8cb0 2024-02-23T00:53:58.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:58.310+0000 7f870d90c700 10 librados: watch_flush enter 2024-02-23T00:53:58.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:58.312+0000 7f870d90c700 10 librados: watch_flush exit 2024-02-23T00:53:58.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:58.312+0000 7f870d90c700 20 client.41963.objecter shutdown clearing up homeless session... 2024-02-23T00:53:58.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:58.312+0000 7f870d90c700 10 client.41963.objecter successfully canceled tick 2024-02-23T00:53:58.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:58.312+0000 7f870d90c700 1 -- 172.21.15.47:0/602669385 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f86f0062cc0 msgr2=0x7f86f0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:58.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:58.312+0000 7f870d90c700 1 --2- 172.21.15.47:0/602669385 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f86f0062cc0 0x7f86f0065180 secure :-1 s=READY pgs=3170 cs=0 l=1 rev1=1 rx=0x7f87081099d0 tx=0x7f86fc00b040).stop 2024-02-23T00:53:58.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:58.312+0000 7f870d90c700 1 -- 172.21.15.47:0/602669385 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87080f8cb0 msgr2=0x7f870810e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:53:58.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:58.312+0000 7f870d90c700 1 --2- 172.21.15.47:0/602669385 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87080f8cb0 0x7f870810e090 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 rx=0x7f86f8009f20 tx=0x7f86f8009cb0).stop 2024-02-23T00:53:58.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:58.313+0000 7f870d90c700 1 -- 172.21.15.47:0/602669385 shutdown_connections 2024-02-23T00:53:58.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:58.313+0000 7f870d90c700 1 --2- 172.21.15.47:0/602669385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8708102210 0x7f8708108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:58.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:58.313+0000 7f870d90c700 1 --2- 172.21.15.47:0/602669385 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f86f0062cc0 0x7f86f0065180 unknown :-1 s=CLOSED pgs=3170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:58.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:58.313+0000 7f870d90c700 1 --2- 172.21.15.47:0/602669385 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87080f8cb0 0x7f870810e090 unknown :-1 s=CLOSED pgs=2779 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:58.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:58.313+0000 7f870d90c700 1 --2- 172.21.15.47:0/602669385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87080f9610 0x7f870810e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:53:58.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:58.313+0000 7f870d90c700 1 -- 172.21.15.47:0/602669385 >> 172.21.15.47:0/602669385 conn(0x7f87080f2820 msgr2=0x7f87080f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:53:58.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:58.313+0000 7f870d90c700 1 -- 172.21.15.47:0/602669385 shutdown_connections 2024-02-23T00:53:58.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:58.314+0000 7f870d90c700 1 -- 172.21.15.47:0/602669385 wait complete. 2024-02-23T00:53:58.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:53:58.314+0000 7f870d90c700 1 librados: shutdown 2024-02-23T00:53:58.318 INFO:teuthology.orchestra.run.smithi047.stderr:reweighted osd.4 to 1 (10000) 2024-02-23T00:53:58.341 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:53:58.345 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:53:58.349 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:58.350 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:53:58.450 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:53:58.483 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:58.504 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:53:58.507 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:58.507 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:53:58.608 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:53:58.638 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:58.676 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:53:58.680 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:58.681 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:53:58.782 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:53:58.814 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:58.836 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:53:58.839 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:58.839 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:53:58.940 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:53:58.967 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:57.427255+0000 osd.4 (osd.4) 293 : cluster [DBG] 4.18 scrub starts 2024-02-23T00:53:58.967 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:57.428621+0000 osd.4 (osd.4) 294 : cluster [DBG] 4.18 scrub ok 2024-02-23T00:53:58.968 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:58.303513+0000 mon.a (mon.0) 3436 : cluster [WRN] Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-02-23T00:53:58.968 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:58.307575+0000 mon.a (mon.0) 3437 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:53:58.968 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:58.307751+0000 mon.a (mon.0) 3438 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-02-23T00:53:58.968 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:58.307807+0000 mon.a (mon.0) 3439 : cluster [DBG] osdmap e699: 8 total, 8 up, 6 in 2024-02-23T00:53:58.968 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:58.316424+0000 mon.c (mon.2) 409 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:58.968 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:53:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:58.317027+0000 mon.a (mon.0) 3440 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:58.968 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:57.427255+0000 osd.4 (osd.4) 293 : cluster [DBG] 4.18 scrub starts 2024-02-23T00:53:58.968 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:57.428621+0000 osd.4 (osd.4) 294 : cluster [DBG] 4.18 scrub ok 2024-02-23T00:53:58.968 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:58.303513+0000 mon.a (mon.0) 3436 : cluster [WRN] Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-02-23T00:53:58.969 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:58.307575+0000 mon.a (mon.0) 3437 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:53:58.969 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:58.307751+0000 mon.a (mon.0) 3438 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-02-23T00:53:58.969 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:58.307807+0000 mon.a (mon.0) 3439 : cluster [DBG] osdmap e699: 8 total, 8 up, 6 in 2024-02-23T00:53:58.969 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:58.316424+0000 mon.c (mon.2) 409 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:58.969 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:53:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:58.317027+0000 mon.a (mon.0) 3440 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:58.969 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:59.042 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:53:59.044 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:59.045 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:53:59.146 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:53:59.173 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:59.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:57.427255+0000 osd.4 (osd.4) 293 : cluster [DBG] 4.18 scrub starts 2024-02-23T00:53:59.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:57.428621+0000 osd.4 (osd.4) 294 : cluster [DBG] 4.18 scrub ok 2024-02-23T00:53:59.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:58.303513+0000 mon.a (mon.0) 3436 : cluster [WRN] Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-02-23T00:53:59.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:58.307575+0000 mon.a (mon.0) 3437 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:53:59.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:58.307751+0000 mon.a (mon.0) 3438 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-02-23T00:53:59.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:58.307807+0000 mon.a (mon.0) 3439 : cluster [DBG] osdmap e699: 8 total, 8 up, 6 in 2024-02-23T00:53:59.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:58.316424+0000 mon.c (mon.2) 409 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:59.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:53:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:58.317027+0000 mon.a (mon.0) 3440 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:53:59.273 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:53:59.276 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:59.276 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:53:59.377 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:53:59.409 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:59.430 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:53:59.433 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:53:59.434 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:53:59.534 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:53:59.564 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:53:59.677 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-23T00:54:00.105 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight 5 1 2024-02-23T00:54:00.112 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:54:00.115 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:00.116 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:54:00.216 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:54:00.245 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:00.352 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:54:00.354 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:58.482840+0000 osd.6 (osd.6) 263 : cluster [DBG] 5.1c scrub starts 2024-02-23T00:54:00.354 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:58.483851+0000 osd.6 (osd.6) 264 : cluster [DBG] 5.1c scrub ok 2024-02-23T00:54:00.354 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:59.316512+0000 mon.a (mon.0) 3441 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:54:00.355 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:59.316593+0000 mon.a (mon.0) 3442 : cluster [DBG] osdmap e700: 8 total, 8 up, 6 in 2024-02-23T00:54:00.355 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:59.318076+0000 mon.c (mon.2) 410 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-122"}]: dispatch 2024-02-23T00:54:00.355 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:53:59.318372+0000 mon.a (mon.0) 3443 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-122"}]: dispatch 2024-02-23T00:54:00.355 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:59.379380+0000 mgr.y (mgr.14517) 604 : cluster [DBG] pgmap v1059: 184 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 151 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s; 1/714 objects misplaced (0.140%); 368 B/s, 20 objects/s recovering 2024-02-23T00:54:00.356 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:58.482840+0000 osd.6 (osd.6) 263 : cluster [DBG] 5.1c scrub starts 2024-02-23T00:54:00.356 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:58.483851+0000 osd.6 (osd.6) 264 : cluster [DBG] 5.1c scrub ok 2024-02-23T00:54:00.356 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:59.316512+0000 mon.a (mon.0) 3441 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:54:00.356 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:59.316593+0000 mon.a (mon.0) 3442 : cluster [DBG] osdmap e700: 8 total, 8 up, 6 in 2024-02-23T00:54:00.356 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:59.318076+0000 mon.c (mon.2) 410 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-122"}]: dispatch 2024-02-23T00:54:00.356 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:53:59.318372+0000 mon.a (mon.0) 3443 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-122"}]: dispatch 2024-02-23T00:54:00.356 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:59.379380+0000 mgr.y (mgr.14517) 604 : cluster [DBG] pgmap v1059: 184 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 151 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s; 1/714 objects misplaced (0.140%); 368 B/s, 20 objects/s recovering 2024-02-23T00:54:00.357 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:00.357 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:54:00.383 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:54:00.383 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:54:00.458 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:54:00.487 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:00.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:58.482840+0000 osd.6 (osd.6) 263 : cluster [DBG] 5.1c scrub starts 2024-02-23T00:54:00.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:58.483851+0000 osd.6 (osd.6) 264 : cluster [DBG] 5.1c scrub ok 2024-02-23T00:54:00.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:59.316512+0000 mon.a (mon.0) 3441 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:54:00.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:59.316593+0000 mon.a (mon.0) 3442 : cluster [DBG] osdmap e700: 8 total, 8 up, 6 in 2024-02-23T00:54:00.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:59.318076+0000 mon.c (mon.2) 410 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-122"}]: dispatch 2024-02-23T00:54:00.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:53:59.318372+0000 mon.a (mon.0) 3443 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-122"}]: dispatch 2024-02-23T00:54:00.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:59.379380+0000 mgr.y (mgr.14517) 604 : cluster [DBG] pgmap v1059: 184 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 151 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s; 1/714 objects misplaced (0.140%); 368 B/s, 20 objects/s recovering 2024-02-23T00:54:00.745 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:54:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:54:00] "GET /metrics HTTP/1.1" 200 188863 "" "Prometheus/2.33.4" 2024-02-23T00:54:01.036 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:54:01.039 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:01.039 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:54:01.140 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:54:01.170 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:01.244 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:54:01.246 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:01.247 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:54:01.348 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:54:01.378 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:01.452 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:54:01.456 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:01.456 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:54:01.557 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:54:01.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:59.498602+0000 osd.6 (osd.6) 265 : cluster [DBG] 5.11 scrub starts 2024-02-23T00:54:01.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:53:59.499655+0000 osd.6 (osd.6) 266 : cluster [DBG] 5.11 scrub ok 2024-02-23T00:54:01.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:00.285248+0000 osd.5 (osd.5) 306 : cluster [DBG] 5.16 scrub starts 2024-02-23T00:54:01.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:00.288128+0000 osd.5 (osd.5) 307 : cluster [DBG] 5.16 scrub ok 2024-02-23T00:54:01.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:00.317872+0000 mon.a (mon.0) 3444 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-122"}]': finished 2024-02-23T00:54:01.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:00.317949+0000 mon.a (mon.0) 3445 : cluster [DBG] osdmap e701: 8 total, 8 up, 6 in 2024-02-23T00:54:01.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:00.318877+0000 mon.c (mon.2) 411 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-122", "mode": "writeback"}]: dispatch 2024-02-23T00:54:01.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:00.326375+0000 mon.a (mon.0) 3446 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-122", "mode": "writeback"}]: dispatch 2024-02-23T00:54:01.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:59.498602+0000 osd.6 (osd.6) 265 : cluster [DBG] 5.11 scrub starts 2024-02-23T00:54:01.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:53:59.499655+0000 osd.6 (osd.6) 266 : cluster [DBG] 5.11 scrub ok 2024-02-23T00:54:01.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:00.285248+0000 osd.5 (osd.5) 306 : cluster [DBG] 5.16 scrub starts 2024-02-23T00:54:01.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:00.288128+0000 osd.5 (osd.5) 307 : cluster [DBG] 5.16 scrub ok 2024-02-23T00:54:01.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:00.317872+0000 mon.a (mon.0) 3444 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-122"}]': finished 2024-02-23T00:54:01.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:00.317949+0000 mon.a (mon.0) 3445 : cluster [DBG] osdmap e701: 8 total, 8 up, 6 in 2024-02-23T00:54:01.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:00.318877+0000 mon.c (mon.2) 411 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-122", "mode": "writeback"}]: dispatch 2024-02-23T00:54:01.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:00.326375+0000 mon.a (mon.0) 3446 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-122", "mode": "writeback"}]: dispatch 2024-02-23T00:54:01.587 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:59.498602+0000 osd.6 (osd.6) 265 : cluster [DBG] 5.11 scrub starts 2024-02-23T00:54:01.588 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:53:59.499655+0000 osd.6 (osd.6) 266 : cluster [DBG] 5.11 scrub ok 2024-02-23T00:54:01.588 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:00.285248+0000 osd.5 (osd.5) 306 : cluster [DBG] 5.16 scrub starts 2024-02-23T00:54:01.588 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:00.288128+0000 osd.5 (osd.5) 307 : cluster [DBG] 5.16 scrub ok 2024-02-23T00:54:01.588 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:00.317872+0000 mon.a (mon.0) 3444 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-122"}]': finished 2024-02-23T00:54:01.588 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:00.317949+0000 mon.a (mon.0) 3445 : cluster [DBG] osdmap e701: 8 total, 8 up, 6 in 2024-02-23T00:54:01.588 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:00.318877+0000 mon.c (mon.2) 411 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-122", "mode": "writeback"}]: dispatch 2024-02-23T00:54:01.588 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:00.326375+0000 mon.a (mon.0) 3446 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-122", "mode": "writeback"}]: dispatch 2024-02-23T00:54:01.589 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:01.610 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:54:01.612 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:01.613 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:54:01.713 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:54:01.741 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:01.762 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:54:01.764 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:01.765 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:54:01.865 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:54:01.893 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:01.912 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:54:01.915 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:01.915 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:54:02.014 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.0 dump_blocked_ops 2024-02-23T00:54:02.016 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:54:02.045 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:02.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.233+0000 7f7c38dc3700 1 -- 172.21.15.47:0/166812552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c340fcac0 msgr2=0x7f7c340fcea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:02.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.233+0000 7f7c38dc3700 1 --2- 172.21.15.47:0/166812552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c340fcac0 0x7f7c340fcea0 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 rx=0x7f7c24004870 tx=0x7f7c24009cb0).stop 2024-02-23T00:54:02.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.233+0000 7f7c38dc3700 1 -- 172.21.15.47:0/166812552 shutdown_connections 2024-02-23T00:54:02.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.233+0000 7f7c38dc3700 1 --2- 172.21.15.47:0/166812552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c340f7330 0x7f7c34103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:02.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.233+0000 7f7c38dc3700 1 --2- 172.21.15.47:0/166812552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c340fcac0 0x7f7c340fcea0 unknown :-1 s=CLOSED pgs=2622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:02.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.233+0000 7f7c38dc3700 1 --2- 172.21.15.47:0/166812552 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7c340fd420 0x7f7c340f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:54:02.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.233+0000 7f7c38dc3700 1 -- 172.21.15.47:0/166812552 >> 172.21.15.47:0/166812552 conn(0x7f7c340f26f0 msgr2=0x7f7c340f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:02.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.233+0000 7f7c38dc3700 1 -- 172.21.15.47:0/166812552 shutdown_connections 2024-02-23T00:54:02.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.233+0000 7f7c38dc3700 1 -- 172.21.15.47:0/166812552 wait complete. 2024-02-23T00:54:02.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.234+0000 7f7c38dc3700 1 librados: starting msgr at 2024-02-23T00:54:02.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.234+0000 7f7c38dc3700 1 librados: starting objecter 2024-02-23T00:54:02.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.234+0000 7f7c38dc3700 1 Processor -- start 2024-02-23T00:54:02.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.234+0000 7f7c38dc3700 1 -- start start 2024-02-23T00:54:02.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.234+0000 7f7c38dc3700 1 librados: setting wanted keys 2024-02-23T00:54:02.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.234+0000 7f7c38dc3700 1 librados: calling monclient init 2024-02-23T00:54:02.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.235+0000 7f7c38dc3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c340f7330 0x7f7c34110340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:02.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.235+0000 7f7c38dc3700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7c340fd420 0x7f7c34110830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:02.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.235+0000 7f7c38dc3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c3410aca0 0x7f7c3410b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:02.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.235+0000 7f7c38dc3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3410b610 con 0x7f7c3410aca0 2024-02-23T00:54:02.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.235+0000 7f7c38dc3700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3410b780 con 0x7f7c340f7330 2024-02-23T00:54:02.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.235+0000 7f7c38dc3700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3410b940 con 0x7f7c340fd420 2024-02-23T00:54:02.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.235+0000 7f7c32d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c3410aca0 0x7f7c3410b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:02.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.235+0000 7f7c31d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7c340fd420 0x7f7c34110830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:02.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.235+0000 7f7c32d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c3410aca0 0x7f7c3410b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53278/0 (socket says 172.21.15.47:53278) 2024-02-23T00:54:02.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.235+0000 7f7c32d9d700 1 -- 172.21.15.47:0/691724080 learned_addr learned my addr 172.21.15.47:0/691724080 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:54:02.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.235+0000 7f7c3259c700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c340f7330 0x7f7c34110340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:02.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.235+0000 7f7c32d9d700 1 -- 172.21.15.47:0/691724080 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7c340fd420 msgr2=0x7f7c34110830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:02.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.235+0000 7f7c32d9d700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7c340fd420 0x7f7c34110830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:02.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.235+0000 7f7c32d9d700 1 -- 172.21.15.47:0/691724080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c340f7330 msgr2=0x7f7c34110340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:02.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.235+0000 7f7c32d9d700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c340f7330 0x7f7c34110340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:02.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.235+0000 7f7c32d9d700 1 -- 172.21.15.47:0/691724080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c24005040 con 0x7f7c3410aca0 2024-02-23T00:54:02.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.235+0000 7f7c32d9d700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c3410aca0 0x7f7c3410b120 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 rx=0x7f7c2c00ec20 tx=0x7f7c2c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:02.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.235+0000 7f7c237fe700 10 client.?.objecter ms_handle_connect 0x7f7c3410aca0 2024-02-23T00:54:02.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.235+0000 7f7c237fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:54:02.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.236+0000 7f7c237fe700 1 -- 172.21.15.47:0/691724080 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c009950 con 0x7f7c3410aca0 2024-02-23T00:54:02.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.236+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c34117170 con 0x7f7c3410aca0 2024-02-23T00:54:02.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.236+0000 7f7c38dc3700 10 client.31647.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:54:02.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.236+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c341174c0 con 0x7f7c3410aca0 2024-02-23T00:54:02.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.236+0000 7f7c38dc3700 1 librados: init done 2024-02-23T00:54:02.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.236+0000 7f7c237fe700 1 -- 172.21.15.47:0/691724080 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f7c2c00f040 con 0x7f7c3410aca0 2024-02-23T00:54:02.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.236+0000 7f7c237fe700 1 -- 172.21.15.47:0/691724080 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c007d10 con 0x7f7c3410aca0 2024-02-23T00:54:02.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.237+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f7c14004f30 con 0x7f7c3410aca0 2024-02-23T00:54:02.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.238+0000 7f7c237fe700 1 -- 172.21.15.47:0/691724080 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f7c2c0092e0 con 0x7f7c3410aca0 2024-02-23T00:54:02.245 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:54:02.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.238+0000 7f7c237fe700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7c1c062cc0 0x7f7c1c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:02.246 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:54:02.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.238+0000 7f7c237fe700 1 -- 172.21.15.47:0/691724080 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(702..702 src has 482..702) v4 ==== 9724+0+0 (secure 0 0 0) 0x7f7c2c07d850 con 0x7f7c3410aca0 2024-02-23T00:54:02.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.238+0000 7f7c237fe700 10 client.31647.objecter ms_dispatch 0x7f7c34001550 osd_map(702..702 src has 482..702) v4 2024-02-23T00:54:02.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.238+0000 7f7c237fe700 3 client.31647.objecter handle_osd_map got epochs [702,702] > 0 2024-02-23T00:54:02.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.238+0000 7f7c237fe700 3 client.31647.objecter handle_osd_map decoding full epoch 702 2024-02-23T00:54:02.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.238+0000 7f7c3259c700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7c1c062cc0 0x7f7c1c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.239+0000 7f7c237fe700 20 client.31647.objecter dump_active .. 0 homeless 2024-02-23T00:54:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.239+0000 7f7c237fe700 1 -- 172.21.15.47:0/691724080 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v702) v1 ==== 54+0+16 (secure 0 0 0) 0x7f7c2c07e0a0 con 0x7f7c3410aca0 2024-02-23T00:54:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.239+0000 7f7c237fe700 10 client.31647.objecter ms_handle_connect 0x7f7c1c062cc0 2024-02-23T00:54:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.239+0000 7f7c3259c700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7c1c062cc0 0x7f7c1c065180 secure :-1 s=READY pgs=3171 cs=0 l=1 rev1=1 rx=0x7f7c24004710 tx=0x7f7c2400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.239+0000 7f7c38dc3700 10 client.31647.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.239+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_assign -1 1 2024-02-23T00:54:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.239+0000 7f7c38dc3700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f7c14004560 0x7f7c14004980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.239+0000 7f7c38dc3700 20 client.31647.objecter _get_session s=0x7f7c140051d0 osd=0 3 2024-02-23T00:54:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.239+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c140051d0 osd=0 3 2024-02-23T00:54:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.239+0000 7f7c38dc3700 20 client.31647.objecter _get_session s=0x7f7c140051d0 osd=0 3 2024-02-23T00:54:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.239+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_remove -1 1 2024-02-23T00:54:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.239+0000 7f7c38dc3700 20 client.31647.objecter get_session s=0x7f7c140051d0 osd=0 3 2024-02-23T00:54:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.239+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_assign 0 1 2024-02-23T00:54:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.239+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c140051d0 osd=0 4 2024-02-23T00:54:02.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.239+0000 7f7c38dc3700 10 client.31647.objecter _send_command 1 2024-02-23T00:54:02.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.239+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7c14008960 con 0x7f7c14004560 2024-02-23T00:54:02.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.239+0000 7f7c31d9b700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f7c14004560 0x7f7c14004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:02.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.240+0000 7f7c237fe700 10 client.31647.objecter ms_handle_connect 0x7f7c14004560 2024-02-23T00:54:02.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.240+0000 7f7c31d9b700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f7c14004560 0x7f7c14004980 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:02.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.240+0000 7f7c237fe700 1 -- 172.21.15.47:0/691724080 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7c14008960 con 0x7f7c14004560 2024-02-23T00:54:02.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.240+0000 7f7c237fe700 10 client.31647.objecter ms_dispatch 0x7f7c34001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:54:02.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.240+0000 7f7c237fe700 10 client.31647.objecter _finish_command 1 = system:0 2024-02-23T00:54:02.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.240+0000 7f7c237fe700 20 client.31647.objecter put_session s=0x7f7c140051d0 osd=0 4 2024-02-23T00:54:02.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.240+0000 7f7c237fe700 15 client.31647.objecter _session_command_op_remove 0 1 2024-02-23T00:54:02.249 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:54:02.252 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:02.252 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:54:02.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.267+0000 7f7c38dc3700 10 client.31647.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:54:02.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.267+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_assign -1 2 2024-02-23T00:54:02.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.267+0000 7f7c38dc3700 20 client.31647.objecter _get_session s=0x7f7c140051d0 osd=0 3 2024-02-23T00:54:02.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.267+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c140051d0 osd=0 3 2024-02-23T00:54:02.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.267+0000 7f7c38dc3700 20 client.31647.objecter _get_session s=0x7f7c140051d0 osd=0 3 2024-02-23T00:54:02.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.267+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_remove -1 2 2024-02-23T00:54:02.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.267+0000 7f7c38dc3700 20 client.31647.objecter get_session s=0x7f7c140051d0 osd=0 3 2024-02-23T00:54:02.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.267+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_assign 0 2 2024-02-23T00:54:02.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.267+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c140051d0 osd=0 4 2024-02-23T00:54:02.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.267+0000 7f7c38dc3700 10 client.31647.objecter _send_command 2 2024-02-23T00:54:02.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.267+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f7c14003fd0 con 0x7f7c14004560 2024-02-23T00:54:02.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.270+0000 7f7c237fe700 1 -- 172.21.15.47:0/691724080 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f7c14003fd0 con 0x7f7c14004560 2024-02-23T00:54:02.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.270+0000 7f7c237fe700 10 client.31647.objecter ms_dispatch 0x7f7c34001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:54:02.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.270+0000 7f7c237fe700 10 client.31647.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:54:02.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.270+0000 7f7c237fe700 20 client.31647.objecter put_session s=0x7f7c140051d0 osd=0 4 2024-02-23T00:54:02.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.270+0000 7f7c237fe700 15 client.31647.objecter _session_command_op_remove 0 2 2024-02-23T00:54:02.274 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:54:02.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.271+0000 7f7c38dc3700 10 client.31647.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:02.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.271+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_assign -1 3 2024-02-23T00:54:02.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.271+0000 7f7c38dc3700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f7c14008510 0x7f7c14002780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:02.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.271+0000 7f7c38dc3700 20 client.31647.objecter _get_session s=0x7f7c140043e0 osd=1 3 2024-02-23T00:54:02.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.271+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c140043e0 osd=1 3 2024-02-23T00:54:02.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.271+0000 7f7c38dc3700 20 client.31647.objecter _get_session s=0x7f7c140043e0 osd=1 3 2024-02-23T00:54:02.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.271+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_remove -1 3 2024-02-23T00:54:02.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.271+0000 7f7c38dc3700 20 client.31647.objecter get_session s=0x7f7c140043e0 osd=1 3 2024-02-23T00:54:02.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.271+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_assign 1 3 2024-02-23T00:54:02.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.271+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c140043e0 osd=1 4 2024-02-23T00:54:02.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.271+0000 7f7c38dc3700 10 client.31647.objecter _send_command 3 2024-02-23T00:54:02.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.271+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7c14002ed0 con 0x7f7c14008510 2024-02-23T00:54:02.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.271+0000 7f7c3259c700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f7c14008510 0x7f7c14002780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:02.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.272+0000 7f7c3259c700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f7c14008510 0x7f7c14002780 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:02.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.272+0000 7f7c237fe700 10 client.31647.objecter ms_handle_connect 0x7f7c14008510 2024-02-23T00:54:02.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.273+0000 7f7c237fe700 1 -- 172.21.15.47:0/691724080 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7c14002ed0 con 0x7f7c14008510 2024-02-23T00:54:02.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.273+0000 7f7c237fe700 10 client.31647.objecter ms_dispatch 0x7f7c34001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:54:02.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.273+0000 7f7c237fe700 10 client.31647.objecter _finish_command 3 = system:0 2024-02-23T00:54:02.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.273+0000 7f7c237fe700 20 client.31647.objecter put_session s=0x7f7c140043e0 osd=1 4 2024-02-23T00:54:02.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.273+0000 7f7c237fe700 15 client.31647.objecter _session_command_op_remove 1 3 2024-02-23T00:54:02.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.289+0000 7f7c38dc3700 10 client.31647.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:54:02.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.289+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_assign -1 4 2024-02-23T00:54:02.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.289+0000 7f7c38dc3700 20 client.31647.objecter _get_session s=0x7f7c140043e0 osd=1 3 2024-02-23T00:54:02.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.289+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c140043e0 osd=1 3 2024-02-23T00:54:02.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.289+0000 7f7c38dc3700 20 client.31647.objecter _get_session s=0x7f7c140043e0 osd=1 3 2024-02-23T00:54:02.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.289+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_remove -1 4 2024-02-23T00:54:02.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.289+0000 7f7c38dc3700 20 client.31647.objecter get_session s=0x7f7c140043e0 osd=1 3 2024-02-23T00:54:02.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.289+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_assign 1 4 2024-02-23T00:54:02.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.289+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c140043e0 osd=1 4 2024-02-23T00:54:02.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.289+0000 7f7c38dc3700 10 client.31647.objecter _send_command 4 2024-02-23T00:54:02.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.289+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f7c140038a0 con 0x7f7c14008510 2024-02-23T00:54:02.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.290+0000 7f7c237fe700 1 -- 172.21.15.47:0/691724080 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f7c140038a0 con 0x7f7c14008510 2024-02-23T00:54:02.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.290+0000 7f7c237fe700 10 client.31647.objecter ms_dispatch 0x7f7c34001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:54:02.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.290+0000 7f7c237fe700 10 client.31647.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:54:02.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.290+0000 7f7c237fe700 20 client.31647.objecter put_session s=0x7f7c140043e0 osd=1 4 2024-02-23T00:54:02.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.290+0000 7f7c237fe700 15 client.31647.objecter _session_command_op_remove 1 4 2024-02-23T00:54:02.294 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:54:02.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.290+0000 7f7c38dc3700 10 client.31647.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:02.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.290+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_assign -1 5 2024-02-23T00:54:02.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.290+0000 7f7c38dc3700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f7c1400aff0 0x7f7c1400d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:02.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.291+0000 7f7c38dc3700 20 client.31647.objecter _get_session s=0x7f7c14007a80 osd=2 3 2024-02-23T00:54:02.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.291+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c14007a80 osd=2 3 2024-02-23T00:54:02.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.291+0000 7f7c38dc3700 20 client.31647.objecter _get_session s=0x7f7c14007a80 osd=2 3 2024-02-23T00:54:02.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.291+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_remove -1 5 2024-02-23T00:54:02.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.291+0000 7f7c38dc3700 20 client.31647.objecter get_session s=0x7f7c14007a80 osd=2 3 2024-02-23T00:54:02.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.291+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_assign 2 5 2024-02-23T00:54:02.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.291+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c14007a80 osd=2 4 2024-02-23T00:54:02.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.291+0000 7f7c38dc3700 10 client.31647.objecter _send_command 5 2024-02-23T00:54:02.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.291+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7c14010030 con 0x7f7c1400aff0 2024-02-23T00:54:02.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.291+0000 7f7c31d9b700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f7c1400aff0 0x7f7c1400d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:02.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.291+0000 7f7c237fe700 10 client.31647.objecter ms_handle_connect 0x7f7c1400aff0 2024-02-23T00:54:02.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.291+0000 7f7c31d9b700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f7c1400aff0 0x7f7c1400d3e0 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:02.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.292+0000 7f7c237fe700 1 -- 172.21.15.47:0/691724080 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7c14010030 con 0x7f7c1400aff0 2024-02-23T00:54:02.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.292+0000 7f7c237fe700 10 client.31647.objecter ms_dispatch 0x7f7c34001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:54:02.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.292+0000 7f7c237fe700 10 client.31647.objecter _finish_command 5 = system:0 2024-02-23T00:54:02.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.292+0000 7f7c237fe700 20 client.31647.objecter put_session s=0x7f7c14007a80 osd=2 4 2024-02-23T00:54:02.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.292+0000 7f7c237fe700 15 client.31647.objecter _session_command_op_remove 2 5 2024-02-23T00:54:02.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.306+0000 7f7c38dc3700 10 client.31647.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:54:02.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.306+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_assign -1 6 2024-02-23T00:54:02.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.306+0000 7f7c38dc3700 20 client.31647.objecter _get_session s=0x7f7c14007a80 osd=2 3 2024-02-23T00:54:02.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.306+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c14007a80 osd=2 3 2024-02-23T00:54:02.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.306+0000 7f7c38dc3700 20 client.31647.objecter _get_session s=0x7f7c14007a80 osd=2 3 2024-02-23T00:54:02.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.306+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_remove -1 6 2024-02-23T00:54:02.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.306+0000 7f7c38dc3700 20 client.31647.objecter get_session s=0x7f7c14007a80 osd=2 3 2024-02-23T00:54:02.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.306+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_assign 2 6 2024-02-23T00:54:02.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.306+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c14007a80 osd=2 4 2024-02-23T00:54:02.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.306+0000 7f7c38dc3700 10 client.31647.objecter _send_command 6 2024-02-23T00:54:02.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.306+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f7c14003280 con 0x7f7c1400aff0 2024-02-23T00:54:02.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.308+0000 7f7c237fe700 1 -- 172.21.15.47:0/691724080 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f7c14003280 con 0x7f7c1400aff0 2024-02-23T00:54:02.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.308+0000 7f7c237fe700 10 client.31647.objecter ms_dispatch 0x7f7c34001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:54:02.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.308+0000 7f7c237fe700 10 client.31647.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:54:02.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.308+0000 7f7c237fe700 20 client.31647.objecter put_session s=0x7f7c14007a80 osd=2 4 2024-02-23T00:54:02.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.308+0000 7f7c237fe700 15 client.31647.objecter _session_command_op_remove 2 6 2024-02-23T00:54:02.311 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:54:02.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.308+0000 7f7c38dc3700 10 client.31647.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:02.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.308+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_assign -1 7 2024-02-23T00:54:02.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.308+0000 7f7c38dc3700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f7c14002080 0x7f7c1400ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:02.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.308+0000 7f7c38dc3700 20 client.31647.objecter _get_session s=0x7f7c14003630 osd=3 3 2024-02-23T00:54:02.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.308+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c14003630 osd=3 3 2024-02-23T00:54:02.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.308+0000 7f7c38dc3700 20 client.31647.objecter _get_session s=0x7f7c14003630 osd=3 3 2024-02-23T00:54:02.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.308+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_remove -1 7 2024-02-23T00:54:02.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.308+0000 7f7c38dc3700 20 client.31647.objecter get_session s=0x7f7c14003630 osd=3 3 2024-02-23T00:54:02.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.308+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_assign 3 7 2024-02-23T00:54:02.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.308+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c14003630 osd=3 4 2024-02-23T00:54:02.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.308+0000 7f7c38dc3700 10 client.31647.objecter _send_command 7 2024-02-23T00:54:02.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.308+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7c1400f0e0 con 0x7f7c14002080 2024-02-23T00:54:02.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.308+0000 7f7c32d9d700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f7c14002080 0x7f7c1400ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:02.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.308+0000 7f7c237fe700 10 client.31647.objecter ms_handle_connect 0x7f7c14002080 2024-02-23T00:54:02.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.308+0000 7f7c32d9d700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f7c14002080 0x7f7c1400ea50 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:02.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.309+0000 7f7c237fe700 1 -- 172.21.15.47:0/691724080 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7c1400f0e0 con 0x7f7c14002080 2024-02-23T00:54:02.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.309+0000 7f7c237fe700 10 client.31647.objecter ms_dispatch 0x7f7c34001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:54:02.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.309+0000 7f7c237fe700 10 client.31647.objecter _finish_command 7 = system:0 2024-02-23T00:54:02.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.309+0000 7f7c237fe700 20 client.31647.objecter put_session s=0x7f7c14003630 osd=3 4 2024-02-23T00:54:02.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.309+0000 7f7c237fe700 15 client.31647.objecter _session_command_op_remove 3 7 2024-02-23T00:54:02.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.325+0000 7f7c38dc3700 10 client.31647.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:54:02.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.325+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_assign -1 8 2024-02-23T00:54:02.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.325+0000 7f7c38dc3700 20 client.31647.objecter _get_session s=0x7f7c14003630 osd=3 3 2024-02-23T00:54:02.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.325+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c14003630 osd=3 3 2024-02-23T00:54:02.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.325+0000 7f7c38dc3700 20 client.31647.objecter _get_session s=0x7f7c14003630 osd=3 3 2024-02-23T00:54:02.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.325+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_remove -1 8 2024-02-23T00:54:02.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.325+0000 7f7c38dc3700 20 client.31647.objecter get_session s=0x7f7c14003630 osd=3 3 2024-02-23T00:54:02.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.325+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_assign 3 8 2024-02-23T00:54:02.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.325+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c14003630 osd=3 4 2024-02-23T00:54:02.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.325+0000 7f7c38dc3700 10 client.31647.objecter _send_command 8 2024-02-23T00:54:02.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.325+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f7c14001640 con 0x7f7c14002080 2024-02-23T00:54:02.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.326+0000 7f7c237fe700 1 -- 172.21.15.47:0/691724080 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f7c14001640 con 0x7f7c14002080 2024-02-23T00:54:02.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.326+0000 7f7c237fe700 10 client.31647.objecter ms_dispatch 0x7f7c34001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:54:02.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.326+0000 7f7c237fe700 10 client.31647.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:54:02.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.326+0000 7f7c237fe700 20 client.31647.objecter put_session s=0x7f7c14003630 osd=3 4 2024-02-23T00:54:02.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.326+0000 7f7c237fe700 15 client.31647.objecter _session_command_op_remove 3 8 2024-02-23T00:54:02.330 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:54:02.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.327+0000 7f7c38dc3700 10 client.31647.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:02.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.327+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_assign -1 9 2024-02-23T00:54:02.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.327+0000 7f7c38dc3700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f7c140121b0 0x7f7c140145a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:02.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.327+0000 7f7c38dc3700 20 client.31647.objecter _get_session s=0x7f7c14001400 osd=4 3 2024-02-23T00:54:02.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.327+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c14001400 osd=4 3 2024-02-23T00:54:02.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.327+0000 7f7c38dc3700 20 client.31647.objecter _get_session s=0x7f7c14001400 osd=4 3 2024-02-23T00:54:02.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.327+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_remove -1 9 2024-02-23T00:54:02.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.327+0000 7f7c38dc3700 20 client.31647.objecter get_session s=0x7f7c14001400 osd=4 3 2024-02-23T00:54:02.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.327+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_assign 4 9 2024-02-23T00:54:02.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.327+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c14001400 osd=4 4 2024-02-23T00:54:02.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.327+0000 7f7c38dc3700 10 client.31647.objecter _send_command 9 2024-02-23T00:54:02.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.327+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7c140171f0 con 0x7f7c140121b0 2024-02-23T00:54:02.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.327+0000 7f7c3259c700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f7c140121b0 0x7f7c140145a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:02.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.327+0000 7f7c237fe700 10 client.31647.objecter ms_handle_connect 0x7f7c140121b0 2024-02-23T00:54:02.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.327+0000 7f7c3259c700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f7c140121b0 0x7f7c140145a0 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:02.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.328+0000 7f7c237fe700 1 -- 172.21.15.47:0/691724080 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7c140171f0 con 0x7f7c140121b0 2024-02-23T00:54:02.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.328+0000 7f7c237fe700 10 client.31647.objecter ms_dispatch 0x7f7c34001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:54:02.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.328+0000 7f7c237fe700 10 client.31647.objecter _finish_command 9 = system:0 2024-02-23T00:54:02.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.328+0000 7f7c237fe700 20 client.31647.objecter put_session s=0x7f7c14001400 osd=4 4 2024-02-23T00:54:02.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.328+0000 7f7c237fe700 15 client.31647.objecter _session_command_op_remove 4 9 2024-02-23T00:54:02.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.345+0000 7f7c38dc3700 10 client.31647.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:54:02.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.345+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_assign -1 10 2024-02-23T00:54:02.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.345+0000 7f7c38dc3700 20 client.31647.objecter _get_session s=0x7f7c14001400 osd=4 3 2024-02-23T00:54:02.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.345+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c14001400 osd=4 3 2024-02-23T00:54:02.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.345+0000 7f7c38dc3700 20 client.31647.objecter _get_session s=0x7f7c14001400 osd=4 3 2024-02-23T00:54:02.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.345+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_remove -1 10 2024-02-23T00:54:02.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.345+0000 7f7c38dc3700 20 client.31647.objecter get_session s=0x7f7c14001400 osd=4 3 2024-02-23T00:54:02.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.345+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_assign 4 10 2024-02-23T00:54:02.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.345+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c14001400 osd=4 4 2024-02-23T00:54:02.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.345+0000 7f7c38dc3700 10 client.31647.objecter _send_command 10 2024-02-23T00:54:02.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.345+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f7c14014a90 con 0x7f7c140121b0 2024-02-23T00:54:02.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.347+0000 7f7c237fe700 1 -- 172.21.15.47:0/691724080 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f7c14014a90 con 0x7f7c140121b0 2024-02-23T00:54:02.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.347+0000 7f7c237fe700 10 client.31647.objecter ms_dispatch 0x7f7c34001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:54:02.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.347+0000 7f7c237fe700 10 client.31647.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:54:02.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.347+0000 7f7c237fe700 20 client.31647.objecter put_session s=0x7f7c14001400 osd=4 4 2024-02-23T00:54:02.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.347+0000 7f7c237fe700 15 client.31647.objecter _session_command_op_remove 4 10 2024-02-23T00:54:02.350 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:54:02.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.347+0000 7f7c38dc3700 10 client.31647.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:02.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.347+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_assign -1 11 2024-02-23T00:54:02.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.347+0000 7f7c38dc3700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f7c14015110 0x7f7c14017520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:02.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.347+0000 7f7c38dc3700 20 client.31647.objecter _get_session s=0x7f7c14014a90 osd=5 3 2024-02-23T00:54:02.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.347+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c14014a90 osd=5 3 2024-02-23T00:54:02.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.347+0000 7f7c38dc3700 20 client.31647.objecter _get_session s=0x7f7c14014a90 osd=5 3 2024-02-23T00:54:02.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.347+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_remove -1 11 2024-02-23T00:54:02.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.347+0000 7f7c38dc3700 20 client.31647.objecter get_session s=0x7f7c14014a90 osd=5 3 2024-02-23T00:54:02.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.347+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_assign 5 11 2024-02-23T00:54:02.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.347+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c14014a90 osd=5 4 2024-02-23T00:54:02.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.347+0000 7f7c38dc3700 10 client.31647.objecter _send_command 11 2024-02-23T00:54:02.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.347+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7c1401a360 con 0x7f7c14015110 2024-02-23T00:54:02.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.347+0000 7f7c31d9b700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f7c14015110 0x7f7c14017520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:02.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.348+0000 7f7c237fe700 10 client.31647.objecter ms_handle_connect 0x7f7c14015110 2024-02-23T00:54:02.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.348+0000 7f7c31d9b700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f7c14015110 0x7f7c14017520 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:02.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.348+0000 7f7c237fe700 1 -- 172.21.15.47:0/691724080 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7c1401a360 con 0x7f7c14015110 2024-02-23T00:54:02.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.348+0000 7f7c237fe700 10 client.31647.objecter ms_dispatch 0x7f7c34001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:54:02.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.348+0000 7f7c237fe700 10 client.31647.objecter _finish_command 11 = system:0 2024-02-23T00:54:02.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.348+0000 7f7c237fe700 20 client.31647.objecter put_session s=0x7f7c14014a90 osd=5 4 2024-02-23T00:54:02.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.348+0000 7f7c237fe700 15 client.31647.objecter _session_command_op_remove 5 11 2024-02-23T00:54:02.353 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:54:02.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.363+0000 7f7c38dc3700 10 client.31647.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:54:02.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.363+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_assign -1 12 2024-02-23T00:54:02.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.363+0000 7f7c38dc3700 20 client.31647.objecter _get_session s=0x7f7c14014a90 osd=5 3 2024-02-23T00:54:02.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.363+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c14014a90 osd=5 3 2024-02-23T00:54:02.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.363+0000 7f7c38dc3700 20 client.31647.objecter _get_session s=0x7f7c14014a90 osd=5 3 2024-02-23T00:54:02.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.363+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_remove -1 12 2024-02-23T00:54:02.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.363+0000 7f7c38dc3700 20 client.31647.objecter get_session s=0x7f7c14014a90 osd=5 3 2024-02-23T00:54:02.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.363+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_assign 5 12 2024-02-23T00:54:02.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.363+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c14014a90 osd=5 4 2024-02-23T00:54:02.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.363+0000 7f7c38dc3700 10 client.31647.objecter _send_command 12 2024-02-23T00:54:02.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.363+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f7c1401a360 con 0x7f7c14015110 2024-02-23T00:54:02.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.365+0000 7f7c237fe700 1 -- 172.21.15.47:0/691724080 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f7c1401a360 con 0x7f7c14015110 2024-02-23T00:54:02.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.365+0000 7f7c237fe700 10 client.31647.objecter ms_dispatch 0x7f7c34001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:54:02.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.365+0000 7f7c237fe700 10 client.31647.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:54:02.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.365+0000 7f7c237fe700 20 client.31647.objecter put_session s=0x7f7c14014a90 osd=5 4 2024-02-23T00:54:02.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.365+0000 7f7c237fe700 15 client.31647.objecter _session_command_op_remove 5 12 2024-02-23T00:54:02.368 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:54:02.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.365+0000 7f7c38dc3700 10 client.31647.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:02.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.365+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_assign -1 13 2024-02-23T00:54:02.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.365+0000 7f7c38dc3700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f7c14019690 0x7f7c14019b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:02.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.365+0000 7f7c38dc3700 20 client.31647.objecter _get_session s=0x7f7c1401a360 osd=6 3 2024-02-23T00:54:02.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.365+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c1401a360 osd=6 3 2024-02-23T00:54:02.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.365+0000 7f7c38dc3700 20 client.31647.objecter _get_session s=0x7f7c1401a360 osd=6 3 2024-02-23T00:54:02.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.365+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_remove -1 13 2024-02-23T00:54:02.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.365+0000 7f7c38dc3700 20 client.31647.objecter get_session s=0x7f7c1401a360 osd=6 3 2024-02-23T00:54:02.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.365+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_assign 6 13 2024-02-23T00:54:02.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.365+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c1401a360 osd=6 4 2024-02-23T00:54:02.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.365+0000 7f7c38dc3700 10 client.31647.objecter _send_command 13 2024-02-23T00:54:02.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.365+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7c1401d6c0 con 0x7f7c14019690 2024-02-23T00:54:02.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.366+0000 7f7c32d9d700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f7c14019690 0x7f7c14019b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:02.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.366+0000 7f7c237fe700 10 client.31647.objecter ms_handle_connect 0x7f7c14019690 2024-02-23T00:54:02.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.366+0000 7f7c32d9d700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f7c14019690 0x7f7c14019b10 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:02.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.367+0000 7f7c237fe700 1 -- 172.21.15.47:0/691724080 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7c1401d6c0 con 0x7f7c14019690 2024-02-23T00:54:02.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.367+0000 7f7c237fe700 10 client.31647.objecter ms_dispatch 0x7f7c34001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:54:02.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.367+0000 7f7c237fe700 10 client.31647.objecter _finish_command 13 = system:0 2024-02-23T00:54:02.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.367+0000 7f7c237fe700 20 client.31647.objecter put_session s=0x7f7c1401a360 osd=6 4 2024-02-23T00:54:02.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.367+0000 7f7c237fe700 15 client.31647.objecter _session_command_op_remove 6 13 2024-02-23T00:54:02.382 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:02.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.382+0000 7f7c38dc3700 10 client.31647.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:54:02.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.382+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_assign -1 14 2024-02-23T00:54:02.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.382+0000 7f7c38dc3700 20 client.31647.objecter _get_session s=0x7f7c1401a360 osd=6 3 2024-02-23T00:54:02.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.382+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c1401a360 osd=6 3 2024-02-23T00:54:02.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.382+0000 7f7c38dc3700 20 client.31647.objecter _get_session s=0x7f7c1401a360 osd=6 3 2024-02-23T00:54:02.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.382+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_remove -1 14 2024-02-23T00:54:02.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.382+0000 7f7c38dc3700 20 client.31647.objecter get_session s=0x7f7c1401a360 osd=6 3 2024-02-23T00:54:02.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.382+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_assign 6 14 2024-02-23T00:54:02.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.382+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c1401a360 osd=6 4 2024-02-23T00:54:02.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.382+0000 7f7c38dc3700 10 client.31647.objecter _send_command 14 2024-02-23T00:54:02.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.382+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f7c14017b20 con 0x7f7c14019690 2024-02-23T00:54:02.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.384+0000 7f7c237fe700 1 -- 172.21.15.47:0/691724080 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f7c14017b20 con 0x7f7c14019690 2024-02-23T00:54:02.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.384+0000 7f7c237fe700 10 client.31647.objecter ms_dispatch 0x7f7c34001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:54:02.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.384+0000 7f7c237fe700 10 client.31647.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:54:02.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.384+0000 7f7c237fe700 20 client.31647.objecter put_session s=0x7f7c1401a360 osd=6 4 2024-02-23T00:54:02.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.384+0000 7f7c237fe700 15 client.31647.objecter _session_command_op_remove 6 14 2024-02-23T00:54:02.387 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:54:02.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.384+0000 7f7c38dc3700 10 client.31647.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:02.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.384+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_assign -1 15 2024-02-23T00:54:02.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.384+0000 7f7c38dc3700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f7c140181a0 0x7f7c140185e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:02.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.384+0000 7f7c38dc3700 20 client.31647.objecter _get_session s=0x7f7c14017b20 osd=7 3 2024-02-23T00:54:02.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.384+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c14017b20 osd=7 3 2024-02-23T00:54:02.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.384+0000 7f7c38dc3700 20 client.31647.objecter _get_session s=0x7f7c14017b20 osd=7 3 2024-02-23T00:54:02.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.384+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_remove -1 15 2024-02-23T00:54:02.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.384+0000 7f7c38dc3700 20 client.31647.objecter get_session s=0x7f7c14017b20 osd=7 3 2024-02-23T00:54:02.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.384+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_assign 7 15 2024-02-23T00:54:02.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.384+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c14017b20 osd=7 4 2024-02-23T00:54:02.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.384+0000 7f7c38dc3700 10 client.31647.objecter _send_command 15 2024-02-23T00:54:02.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.384+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7c14018cc0 con 0x7f7c140181a0 2024-02-23T00:54:02.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.384+0000 7f7c3259c700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f7c140181a0 0x7f7c140185e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:02.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.385+0000 7f7c237fe700 10 client.31647.objecter ms_handle_connect 0x7f7c140181a0 2024-02-23T00:54:02.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.385+0000 7f7c3259c700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f7c140181a0 0x7f7c140185e0 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:02.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.385+0000 7f7c237fe700 1 -- 172.21.15.47:0/691724080 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7c14018cc0 con 0x7f7c140181a0 2024-02-23T00:54:02.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.385+0000 7f7c237fe700 10 client.31647.objecter ms_dispatch 0x7f7c34001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:54:02.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.385+0000 7f7c237fe700 10 client.31647.objecter _finish_command 15 = system:0 2024-02-23T00:54:02.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.385+0000 7f7c237fe700 20 client.31647.objecter put_session s=0x7f7c14017b20 osd=7 4 2024-02-23T00:54:02.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.385+0000 7f7c237fe700 15 client.31647.objecter _session_command_op_remove 7 15 2024-02-23T00:54:02.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.401+0000 7f7c38dc3700 10 client.31647.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:54:02.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.401+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_assign -1 16 2024-02-23T00:54:02.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.401+0000 7f7c38dc3700 20 client.31647.objecter _get_session s=0x7f7c14017b20 osd=7 3 2024-02-23T00:54:02.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.401+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c14017b20 osd=7 3 2024-02-23T00:54:02.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.401+0000 7f7c38dc3700 20 client.31647.objecter _get_session s=0x7f7c14017b20 osd=7 3 2024-02-23T00:54:02.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.401+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_remove -1 16 2024-02-23T00:54:02.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.401+0000 7f7c38dc3700 20 client.31647.objecter get_session s=0x7f7c14017b20 osd=7 3 2024-02-23T00:54:02.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.401+0000 7f7c38dc3700 15 client.31647.objecter _session_command_op_assign 7 16 2024-02-23T00:54:02.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.401+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c14017b20 osd=7 4 2024-02-23T00:54:02.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.401+0000 7f7c38dc3700 10 client.31647.objecter _send_command 16 2024-02-23T00:54:02.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.401+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f7c14018cc0 con 0x7f7c140181a0 2024-02-23T00:54:02.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.403+0000 7f7c237fe700 1 -- 172.21.15.47:0/691724080 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f7c14018cc0 con 0x7f7c140181a0 2024-02-23T00:54:02.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.403+0000 7f7c237fe700 10 client.31647.objecter ms_dispatch 0x7f7c34001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:54:02.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.403+0000 7f7c237fe700 10 client.31647.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:54:02.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.403+0000 7f7c237fe700 20 client.31647.objecter put_session s=0x7f7c14017b20 osd=7 4 2024-02-23T00:54:02.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.403+0000 7f7c237fe700 15 client.31647.objecter _session_command_op_remove 7 16 2024-02-23T00:54:02.406 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:54:02.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 10 librados: watch_flush enter 2024-02-23T00:54:02.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 10 librados: watch_flush exit 2024-02-23T00:54:02.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 10 client.31647.objecter close_session for osd.0 2024-02-23T00:54:02.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f7c14004560 msgr2=0x7f7c14004980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:02.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f7c14004560 0x7f7c14004980 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:02.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c140051d0 osd=0 1 2024-02-23T00:54:02.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 10 client.31647.objecter close_session for osd.1 2024-02-23T00:54:02.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f7c14008510 msgr2=0x7f7c14002780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:02.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f7c14008510 0x7f7c14002780 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:02.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c140043e0 osd=1 1 2024-02-23T00:54:02.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 10 client.31647.objecter close_session for osd.2 2024-02-23T00:54:02.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f7c1400aff0 msgr2=0x7f7c1400d3e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:02.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f7c1400aff0 0x7f7c1400d3e0 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:02.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c14007a80 osd=2 1 2024-02-23T00:54:02.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 10 client.31647.objecter close_session for osd.3 2024-02-23T00:54:02.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f7c14002080 msgr2=0x7f7c1400ea50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:02.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c32d9d700 1 -- 172.21.15.47:0/691724080 reap_dead start 2024-02-23T00:54:02.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f7c14002080 0x7f7c1400ea50 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:02.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c14003630 osd=3 1 2024-02-23T00:54:02.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 10 client.31647.objecter close_session for osd.4 2024-02-23T00:54:02.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f7c140121b0 msgr2=0x7f7c140145a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:02.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f7c140121b0 0x7f7c140145a0 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:02.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c14001400 osd=4 1 2024-02-23T00:54:02.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 10 client.31647.objecter close_session for osd.5 2024-02-23T00:54:02.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f7c14015110 msgr2=0x7f7c14017520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:02.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f7c14015110 0x7f7c14017520 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:02.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c14014a90 osd=5 1 2024-02-23T00:54:02.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 10 client.31647.objecter close_session for osd.6 2024-02-23T00:54:02.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f7c14019690 msgr2=0x7f7c14019b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:02.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f7c14019690 0x7f7c14019b10 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:02.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c1401a360 osd=6 1 2024-02-23T00:54:02.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 10 client.31647.objecter close_session for osd.7 2024-02-23T00:54:02.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f7c140181a0 msgr2=0x7f7c140185e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:02.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f7c140181a0 0x7f7c140185e0 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:02.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 20 client.31647.objecter put_session s=0x7f7c14017b20 osd=7 1 2024-02-23T00:54:02.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 20 client.31647.objecter shutdown clearing up homeless session... 2024-02-23T00:54:02.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 10 client.31647.objecter successfully canceled tick 2024-02-23T00:54:02.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c32d9d700 1 -- 172.21.15.47:0/691724080 reap_dead start 2024-02-23T00:54:02.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7c1c062cc0 msgr2=0x7f7c1c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:02.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7c1c062cc0 0x7f7c1c065180 secure :-1 s=READY pgs=3171 cs=0 l=1 rev1=1 rx=0x7f7c24004710 tx=0x7f7c2400b040).stop 2024-02-23T00:54:02.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c3410aca0 msgr2=0x7f7c3410b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:02.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c3410aca0 0x7f7c3410b120 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 rx=0x7f7c2c00ec20 tx=0x7f7c2c00b040).stop 2024-02-23T00:54:02.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 shutdown_connections 2024-02-23T00:54:02.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7c1c062cc0 0x7f7c1c065180 secure :-1 s=CLOSED pgs=3171 cs=0 l=1 rev1=1 rx=0x7f7c24004710 tx=0x7f7c2400b040).stop 2024-02-23T00:54:02.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 1 --2- 172.21.15.47:0/691724080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c3410aca0 0x7f7c3410b120 unknown :-1 s=CLOSED pgs=2623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:02.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 >> 172.21.15.47:0/691724080 conn(0x7f7c340f26f0 msgr2=0x7f7c340f3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:02.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.404+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 shutdown_connections 2024-02-23T00:54:02.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.407+0000 7f7c38dc3700 1 -- 172.21.15.47:0/691724080 wait complete. 2024-02-23T00:54:02.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:02.407+0000 7f7c38dc3700 1 librados: shutdown 2024-02-23T00:54:02.411 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:54:02.411 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:54:02.411 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:54:02.411 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:54:02.411 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:54:02.412 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:54:02.412 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:54:02.412 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:54:02.516 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:00.541718+0000 osd.6 (osd.6) 267 : cluster [DBG] 18.14 scrub starts 2024-02-23T00:54:02.516 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:00.542661+0000 osd.6 (osd.6) 268 : cluster [DBG] 18.14 scrub ok 2024-02-23T00:54:02.516 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:01.320984+0000 mon.a (mon.0) 3447 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:54:02.516 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:01.329481+0000 mon.a (mon.0) 3448 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-122", "mode": "writeback"}]': finished 2024-02-23T00:54:02.516 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:01.329561+0000 mon.a (mon.0) 3449 : cluster [DBG] osdmap e702: 8 total, 8 up, 6 in 2024-02-23T00:54:02.516 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:01.379821+0000 mgr.y (mgr.14517) 605 : cluster [DBG] pgmap v1062: 184 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 151 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1/714 objects misplaced (0.140%) 2024-02-23T00:54:02.516 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:01.480975+0000 osd.4 (osd.4) 295 : cluster [DBG] 5.e scrub starts 2024-02-23T00:54:02.516 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:01.484383+0000 osd.4 (osd.4) 296 : cluster [DBG] 5.e scrub ok 2024-02-23T00:54:02.516 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:02.238686+0000 mon.a (mon.0) 3450 : audit [DBG] from='client.? 172.21.15.47:0/691724080' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:54:02.516 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:02.264005+0000 osd.5 (osd.5) 308 : cluster [DBG] 5.15 scrub starts 2024-02-23T00:54:02.517 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:02.269189+0000 osd.5 (osd.5) 309 : cluster [DBG] 5.15 scrub ok 2024-02-23T00:54:02.517 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:00.541718+0000 osd.6 (osd.6) 267 : cluster [DBG] 18.14 scrub starts 2024-02-23T00:54:02.517 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:00.542661+0000 osd.6 (osd.6) 268 : cluster [DBG] 18.14 scrub ok 2024-02-23T00:54:02.517 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:01.320984+0000 mon.a (mon.0) 3447 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:54:02.517 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:01.329481+0000 mon.a (mon.0) 3448 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-122", "mode": "writeback"}]': finished 2024-02-23T00:54:02.517 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:01.329561+0000 mon.a (mon.0) 3449 : cluster [DBG] osdmap e702: 8 total, 8 up, 6 in 2024-02-23T00:54:02.517 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:01.379821+0000 mgr.y (mgr.14517) 605 : cluster [DBG] pgmap v1062: 184 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 151 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1/714 objects misplaced (0.140%) 2024-02-23T00:54:02.517 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:01.480975+0000 osd.4 (osd.4) 295 : cluster [DBG] 5.e scrub starts 2024-02-23T00:54:02.518 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:01.484383+0000 osd.4 (osd.4) 296 : cluster [DBG] 5.e scrub ok 2024-02-23T00:54:02.518 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:02.238686+0000 mon.a (mon.0) 3450 : audit [DBG] from='client.? 172.21.15.47:0/691724080' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:54:02.518 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:02.264005+0000 osd.5 (osd.5) 308 : cluster [DBG] 5.15 scrub starts 2024-02-23T00:54:02.518 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:02.269189+0000 osd.5 (osd.5) 309 : cluster [DBG] 5.15 scrub ok 2024-02-23T00:54:02.694 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:54:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:54:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:54:02.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:00.541718+0000 osd.6 (osd.6) 267 : cluster [DBG] 18.14 scrub starts 2024-02-23T00:54:02.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:00.542661+0000 osd.6 (osd.6) 268 : cluster [DBG] 18.14 scrub ok 2024-02-23T00:54:02.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:01.320984+0000 mon.a (mon.0) 3447 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:54:02.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:01.329481+0000 mon.a (mon.0) 3448 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-122", "mode": "writeback"}]': finished 2024-02-23T00:54:02.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:01.329561+0000 mon.a (mon.0) 3449 : cluster [DBG] osdmap e702: 8 total, 8 up, 6 in 2024-02-23T00:54:02.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:01.379821+0000 mgr.y (mgr.14517) 605 : cluster [DBG] pgmap v1062: 184 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 151 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1/714 objects misplaced (0.140%) 2024-02-23T00:54:02.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:01.480975+0000 osd.4 (osd.4) 295 : cluster [DBG] 5.e scrub starts 2024-02-23T00:54:02.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:01.484383+0000 osd.4 (osd.4) 296 : cluster [DBG] 5.e scrub ok 2024-02-23T00:54:02.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:02.238686+0000 mon.a (mon.0) 3450 : audit [DBG] from='client.? 172.21.15.47:0/691724080' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:54:02.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:02.264005+0000 osd.5 (osd.5) 308 : cluster [DBG] 5.15 scrub starts 2024-02-23T00:54:02.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:02.269189+0000 osd.5 (osd.5) 309 : cluster [DBG] 5.15 scrub ok 2024-02-23T00:54:03.194 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:54:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:54:03.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:01.499161+0000 osd.6 (osd.6) 269 : cluster [DBG] 4.5 scrub starts 2024-02-23T00:54:03.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:01.500977+0000 osd.6 (osd.6) 270 : cluster [DBG] 4.5 scrub ok 2024-02-23T00:54:03.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:02.334604+0000 mon.a (mon.0) 3451 : cluster [DBG] osdmap e703: 8 total, 8 up, 6 in 2024-02-23T00:54:03.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:03.253951+0000 osd.5 (osd.5) 310 : cluster [DBG] 5.7 scrub starts 2024-02-23T00:54:03.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:03.256912+0000 osd.5 (osd.5) 311 : cluster [DBG] 5.7 scrub ok 2024-02-23T00:54:03.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:01.499161+0000 osd.6 (osd.6) 269 : cluster [DBG] 4.5 scrub starts 2024-02-23T00:54:03.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:01.500977+0000 osd.6 (osd.6) 270 : cluster [DBG] 4.5 scrub ok 2024-02-23T00:54:03.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:02.334604+0000 mon.a (mon.0) 3451 : cluster [DBG] osdmap e703: 8 total, 8 up, 6 in 2024-02-23T00:54:03.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:03.253951+0000 osd.5 (osd.5) 310 : cluster [DBG] 5.7 scrub starts 2024-02-23T00:54:03.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:03.256912+0000 osd.5 (osd.5) 311 : cluster [DBG] 5.7 scrub ok 2024-02-23T00:54:03.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:01.499161+0000 osd.6 (osd.6) 269 : cluster [DBG] 4.5 scrub starts 2024-02-23T00:54:03.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:01.500977+0000 osd.6 (osd.6) 270 : cluster [DBG] 4.5 scrub ok 2024-02-23T00:54:03.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:02.334604+0000 mon.a (mon.0) 3451 : cluster [DBG] osdmap e703: 8 total, 8 up, 6 in 2024-02-23T00:54:03.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:03.253951+0000 osd.5 (osd.5) 310 : cluster [DBG] 5.7 scrub starts 2024-02-23T00:54:03.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:03.256912+0000 osd.5 (osd.5) 311 : cluster [DBG] 5.7 scrub ok 2024-02-23T00:54:03.832 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:54:03.842 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:03.843 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:54:03.944 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:54:03.975 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:04.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:02.896705+0000 mgr.y (mgr.14517) 606 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:54:04.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:03.161501+0000 osd.7 (osd.7) 327 : cluster [DBG] 4.16 scrub starts 2024-02-23T00:54:04.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:03.163046+0000 osd.7 (osd.7) 328 : cluster [DBG] 4.16 scrub ok 2024-02-23T00:54:04.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:03.336148+0000 mon.a (mon.0) 3452 : cluster [DBG] osdmap e704: 8 total, 8 up, 6 in 2024-02-23T00:54:04.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:03.378988+0000 mon.c (mon.2) 412 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:04.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:03.379355+0000 mon.a (mon.0) 3453 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:04.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:03.380229+0000 mgr.y (mgr.14517) 607 : cluster [DBG] pgmap v1065: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s; 1/717 objects misplaced (0.139%) 2024-02-23T00:54:04.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:03.409753+0000 osd.1 (osd.1) 283 : cluster [DBG] 18.3 scrub starts 2024-02-23T00:54:04.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:03.413604+0000 osd.1 (osd.1) 284 : cluster [DBG] 18.3 scrub ok 2024-02-23T00:54:04.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:03.713018+0000 osd.4 (osd.4) 297 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:54:04.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:03.713333+0000 osd.4 (osd.4) 298 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:54:04.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:02.896705+0000 mgr.y (mgr.14517) 606 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:54:04.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:03.161501+0000 osd.7 (osd.7) 327 : cluster [DBG] 4.16 scrub starts 2024-02-23T00:54:04.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:03.163046+0000 osd.7 (osd.7) 328 : cluster [DBG] 4.16 scrub ok 2024-02-23T00:54:04.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:03.336148+0000 mon.a (mon.0) 3452 : cluster [DBG] osdmap e704: 8 total, 8 up, 6 in 2024-02-23T00:54:04.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:03.378988+0000 mon.c (mon.2) 412 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:04.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:03.379355+0000 mon.a (mon.0) 3453 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:04.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:03.380229+0000 mgr.y (mgr.14517) 607 : cluster [DBG] pgmap v1065: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s; 1/717 objects misplaced (0.139%) 2024-02-23T00:54:04.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:03.409753+0000 osd.1 (osd.1) 283 : cluster [DBG] 18.3 scrub starts 2024-02-23T00:54:04.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:03.413604+0000 osd.1 (osd.1) 284 : cluster [DBG] 18.3 scrub ok 2024-02-23T00:54:04.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:03.713018+0000 osd.4 (osd.4) 297 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:54:04.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:03.713333+0000 osd.4 (osd.4) 298 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:54:04.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:02.896705+0000 mgr.y (mgr.14517) 606 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:54:04.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:03.161501+0000 osd.7 (osd.7) 327 : cluster [DBG] 4.16 scrub starts 2024-02-23T00:54:04.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:03.163046+0000 osd.7 (osd.7) 328 : cluster [DBG] 4.16 scrub ok 2024-02-23T00:54:04.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:03.336148+0000 mon.a (mon.0) 3452 : cluster [DBG] osdmap e704: 8 total, 8 up, 6 in 2024-02-23T00:54:04.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:03.378988+0000 mon.c (mon.2) 412 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:04.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:03.379355+0000 mon.a (mon.0) 3453 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:04.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:03.380229+0000 mgr.y (mgr.14517) 607 : cluster [DBG] pgmap v1065: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s; 1/717 objects misplaced (0.139%) 2024-02-23T00:54:04.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:03.409753+0000 osd.1 (osd.1) 283 : cluster [DBG] 18.3 scrub starts 2024-02-23T00:54:04.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:03.413604+0000 osd.1 (osd.1) 284 : cluster [DBG] 18.3 scrub ok 2024-02-23T00:54:04.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:03.713018+0000 osd.4 (osd.4) 297 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:54:04.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:03.713333+0000 osd.4 (osd.4) 298 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:54:04.890 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:54:04.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.910+0000 7fbcd61f8700 1 -- 172.21.15.47:0/264725063 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbcd00f8bb0 msgr2=0x7fbcd00f8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:04.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.910+0000 7fbcd61f8700 1 --2- 172.21.15.47:0/264725063 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbcd00f8bb0 0x7fbcd00f8f90 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 rx=0x7fbcc0004870 tx=0x7fbcc0009cb0).stop 2024-02-23T00:54:04.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.911+0000 7fbcd61f8700 1 -- 172.21.15.47:0/264725063 shutdown_connections 2024-02-23T00:54:04.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.911+0000 7fbcd61f8700 1 --2- 172.21.15.47:0/264725063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcd0102110 0x7fbcd0104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:04.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.911+0000 7fbcd61f8700 1 --2- 172.21.15.47:0/264725063 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbcd00f8bb0 0x7fbcd00f8f90 unknown :-1 s=CLOSED pgs=2780 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:04.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.911+0000 7fbcd61f8700 1 --2- 172.21.15.47:0/264725063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd00f9510 0x7fbcd0101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:04.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.911+0000 7fbcd61f8700 1 -- 172.21.15.47:0/264725063 >> 172.21.15.47:0/264725063 conn(0x7fbcd00f26c0 msgr2=0x7fbcd00f4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:04.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.911+0000 7fbcd61f8700 1 -- 172.21.15.47:0/264725063 shutdown_connections 2024-02-23T00:54:04.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.911+0000 7fbcd61f8700 1 -- 172.21.15.47:0/264725063 wait complete. 2024-02-23T00:54:04.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.911+0000 7fbcd61f8700 1 librados: starting msgr at 2024-02-23T00:54:04.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.911+0000 7fbcd61f8700 1 librados: starting objecter 2024-02-23T00:54:04.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.911+0000 7fbcd61f8700 1 Processor -- start 2024-02-23T00:54:04.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.912+0000 7fbcd61f8700 1 -- start start 2024-02-23T00:54:04.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.912+0000 7fbcd61f8700 1 librados: setting wanted keys 2024-02-23T00:54:04.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.912+0000 7fbcd61f8700 1 librados: calling monclient init 2024-02-23T00:54:04.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.912+0000 7fbcd61f8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd00f8bb0 0x7fbcd010ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:04.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.912+0000 7fbcd61f8700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbcd00f9510 0x7fbcd010d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:04.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.912+0000 7fbcd61f8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcd0102110 0x7fbcd0105f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:04.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.912+0000 7fbcd61f8700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd01064b0 con 0x7fbcd00f8bb0 2024-02-23T00:54:04.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.912+0000 7fbcd61f8700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd0106620 con 0x7fbcd0102110 2024-02-23T00:54:04.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.912+0000 7fbcd61f8700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd01067e0 con 0x7fbcd00f9510 2024-02-23T00:54:04.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.912+0000 7fbcceffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbcd00f9510 0x7fbcd010d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:04.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.912+0000 7fbccf7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd00f8bb0 0x7fbcd010ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:04.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.912+0000 7fbcceffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbcd00f9510 0x7fbcd010d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:51376/0 (socket says 172.21.15.47:51376) 2024-02-23T00:54:04.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.912+0000 7fbcceffd700 1 -- 172.21.15.47:0/2120819500 learned_addr learned my addr 172.21.15.47:0/2120819500 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:54:04.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.912+0000 7fbccffff700 1 --2- 172.21.15.47:0/2120819500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcd0102110 0x7fbcd0105f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:04.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.912+0000 7fbcceffd700 1 -- 172.21.15.47:0/2120819500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcd0102110 msgr2=0x7fbcd0105f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:04.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.912+0000 7fbcceffd700 1 --2- 172.21.15.47:0/2120819500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcd0102110 0x7fbcd0105f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:04.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.912+0000 7fbcceffd700 1 -- 172.21.15.47:0/2120819500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd00f8bb0 msgr2=0x7fbcd010ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:04.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.912+0000 7fbcceffd700 1 --2- 172.21.15.47:0/2120819500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd00f8bb0 0x7fbcd010ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:04.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.912+0000 7fbcceffd700 1 -- 172.21.15.47:0/2120819500 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc4005040 con 0x7fbcd00f9510 2024-02-23T00:54:04.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.913+0000 7fbcceffd700 1 --2- 172.21.15.47:0/2120819500 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbcd00f9510 0x7fbcd010d340 secure :-1 s=READY pgs=2781 cs=0 l=1 rev1=1 rx=0x7fbcc40048f0 tx=0x7fbcc4009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:04.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.913+0000 7fbcccff9700 10 client.?.objecter ms_handle_connect 0x7fbcd00f9510 2024-02-23T00:54:04.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.913+0000 7fbcccff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:54:04.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.913+0000 7fbcccff9700 1 -- 172.21.15.47:0/2120819500 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4007b00 con 0x7fbcd00f9510 2024-02-23T00:54:04.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.913+0000 7fbcd61f8700 1 -- 172.21.15.47:0/2120819500 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc0005040 con 0x7fbcd00f9510 2024-02-23T00:54:04.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.913+0000 7fbcccff9700 1 -- 172.21.15.47:0/2120819500 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fbcc400f040 con 0x7fbcd00f9510 2024-02-23T00:54:04.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.913+0000 7fbcccff9700 1 -- 172.21.15.47:0/2120819500 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4014820 con 0x7fbcd00f9510 2024-02-23T00:54:04.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.913+0000 7fbcd61f8700 10 client.41972.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:54:04.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.913+0000 7fbcd61f8700 1 -- 172.21.15.47:0/2120819500 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd010aa00 con 0x7fbcd00f9510 2024-02-23T00:54:04.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.913+0000 7fbcd61f8700 1 librados: init done 2024-02-23T00:54:04.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.914+0000 7fbcd61f8700 1 -- 172.21.15.47:0/2120819500 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcd0106e20 con 0x7fbcd00f9510 2024-02-23T00:54:04.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.914+0000 7fbcccff9700 1 -- 172.21.15.47:0/2120819500 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fbcc4014c50 con 0x7fbcd00f9510 2024-02-23T00:54:04.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.915+0000 7fbcccff9700 1 --2- 172.21.15.47:0/2120819500 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbcb8062ce0 0x7fbcb80651a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:04.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.915+0000 7fbcccff9700 1 -- 172.21.15.47:0/2120819500 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(705..705 src has 482..705) v4 ==== 9724+0+0 (secure 0 0 0) 0x7fbcc407d3d0 con 0x7fbcd00f9510 2024-02-23T00:54:04.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.915+0000 7fbcccff9700 10 client.41972.objecter ms_dispatch 0x7fbcd0001550 osd_map(705..705 src has 482..705) v4 2024-02-23T00:54:04.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.915+0000 7fbcccff9700 3 client.41972.objecter handle_osd_map got epochs [705,705] > 0 2024-02-23T00:54:04.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.915+0000 7fbcccff9700 3 client.41972.objecter handle_osd_map decoding full epoch 705 2024-02-23T00:54:04.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.915+0000 7fbccf7fe700 1 --2- 172.21.15.47:0/2120819500 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbcb8062ce0 0x7fbcb80651a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:04.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.915+0000 7fbcccff9700 20 client.41972.objecter dump_active .. 0 homeless 2024-02-23T00:54:04.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.916+0000 7fbcccff9700 10 client.41972.objecter ms_handle_connect 0x7fbcb8062ce0 2024-02-23T00:54:04.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.916+0000 7fbccf7fe700 1 --2- 172.21.15.47:0/2120819500 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbcb8062ce0 0x7fbcb80651a0 secure :-1 s=READY pgs=3172 cs=0 l=1 rev1=1 rx=0x7fbcc0004ee0 tx=0x7fbcc000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:04.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:04.919+0000 7fbcccff9700 1 -- 172.21.15.47:0/2120819500 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fbcc4049f60 con 0x7fbcd00f9510 2024-02-23T00:54:04.924 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:54:04.930 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:04.931 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:54:05.032 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:54:05.065 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:05.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:05.112+0000 7fbcd61f8700 1 -- 172.21.15.47:0/2120819500 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7fbcd00f84b0 con 0x7fbcd00f9510 2024-02-23T00:54:05.264 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:54:05.268 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:05.268 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:54:05.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:05.288+0000 7fbcccff9700 1 -- 172.21.15.47:0/2120819500 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v706) v1 ==== 101+0+0 (secure 0 0 0) 0x7fbcc404c7f0 con 0x7fbcd00f9510 2024-02-23T00:54:05.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:05.290+0000 7fbcd61f8700 10 librados: watch_flush enter 2024-02-23T00:54:05.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:05.290+0000 7fbcd61f8700 10 librados: watch_flush exit 2024-02-23T00:54:05.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:05.290+0000 7fbcd61f8700 20 client.41972.objecter shutdown clearing up homeless session... 2024-02-23T00:54:05.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:05.290+0000 7fbcd61f8700 10 client.41972.objecter successfully canceled tick 2024-02-23T00:54:05.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:05.290+0000 7fbcd61f8700 1 -- 172.21.15.47:0/2120819500 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbcb8062ce0 msgr2=0x7fbcb80651a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:05.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:05.290+0000 7fbcd61f8700 1 --2- 172.21.15.47:0/2120819500 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbcb8062ce0 0x7fbcb80651a0 secure :-1 s=READY pgs=3172 cs=0 l=1 rev1=1 rx=0x7fbcc0004ee0 tx=0x7fbcc000b040).stop 2024-02-23T00:54:05.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:05.290+0000 7fbcd61f8700 1 -- 172.21.15.47:0/2120819500 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbcd00f9510 msgr2=0x7fbcd010d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:05.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:05.290+0000 7fbcd61f8700 1 --2- 172.21.15.47:0/2120819500 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbcd00f9510 0x7fbcd010d340 secure :-1 s=READY pgs=2781 cs=0 l=1 rev1=1 rx=0x7fbcc40048f0 tx=0x7fbcc4009cb0).stop 2024-02-23T00:54:05.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:05.290+0000 7fbcd61f8700 1 -- 172.21.15.47:0/2120819500 shutdown_connections 2024-02-23T00:54:05.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:05.290+0000 7fbcd61f8700 1 --2- 172.21.15.47:0/2120819500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcd0102110 0x7fbcd0105f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:05.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:05.290+0000 7fbcd61f8700 1 --2- 172.21.15.47:0/2120819500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd00f8bb0 0x7fbcd010ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:05.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:05.290+0000 7fbcd61f8700 1 --2- 172.21.15.47:0/2120819500 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbcb8062ce0 0x7fbcb80651a0 unknown :-1 s=CLOSED pgs=3172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:05.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:05.290+0000 7fbcd61f8700 1 --2- 172.21.15.47:0/2120819500 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbcd00f9510 0x7fbcd010d340 unknown :-1 s=CLOSED pgs=2781 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:05.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:05.290+0000 7fbcd61f8700 1 -- 172.21.15.47:0/2120819500 >> 172.21.15.47:0/2120819500 conn(0x7fbcd00f26c0 msgr2=0x7fbcd00f3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:05.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:05.290+0000 7fbcd61f8700 1 -- 172.21.15.47:0/2120819500 shutdown_connections 2024-02-23T00:54:05.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:05.290+0000 7fbcd61f8700 1 -- 172.21.15.47:0/2120819500 wait complete. 2024-02-23T00:54:05.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:05.290+0000 7fbcd61f8700 1 librados: shutdown 2024-02-23T00:54:05.295 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is unset 2024-02-23T00:54:05.369 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:54:05.401 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:05.422 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:54:05.425 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:05.425 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:54:05.526 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:54:05.552 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:03.456032+0000 osd.6 (osd.6) 271 : cluster [DBG] 5.3 scrub starts 2024-02-23T00:54:05.553 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:03.457061+0000 osd.6 (osd.6) 272 : cluster [DBG] 5.3 scrub ok 2024-02-23T00:54:05.553 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:04.346662+0000 mon.a (mon.0) 3454 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:54:05.553 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:04.346742+0000 mon.a (mon.0) 3455 : cluster [DBG] osdmap e705: 8 total, 8 up, 6 in 2024-02-23T00:54:05.553 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:04.352954+0000 mon.c (mon.2) 413 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-122"}]: dispatch 2024-02-23T00:54:05.553 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:04.353239+0000 mon.a (mon.0) 3456 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-122"}]: dispatch 2024-02-23T00:54:05.553 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:05.113842+0000 mon.c (mon.2) 414 : audit [INF] from='client.? 172.21.15.47:0/2120819500' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:54:05.553 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:05.114176+0000 mon.a (mon.0) 3457 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:54:05.553 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:05.284815+0000 mon.a (mon.0) 3458 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:54:05.553 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:05.289111+0000 mon.a (mon.0) 3459 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-122"}]': finished 2024-02-23T00:54:05.553 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:05.289195+0000 mon.a (mon.0) 3460 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:54:05.553 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:05.289252+0000 mon.a (mon.0) 3461 : cluster [DBG] osdmap e706: 8 total, 8 up, 6 in 2024-02-23T00:54:05.554 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:03.456032+0000 osd.6 (osd.6) 271 : cluster [DBG] 5.3 scrub starts 2024-02-23T00:54:05.554 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:03.457061+0000 osd.6 (osd.6) 272 : cluster [DBG] 5.3 scrub ok 2024-02-23T00:54:05.554 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:04.346662+0000 mon.a (mon.0) 3454 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:54:05.554 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:04.346742+0000 mon.a (mon.0) 3455 : cluster [DBG] osdmap e705: 8 total, 8 up, 6 in 2024-02-23T00:54:05.554 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:04.352954+0000 mon.c (mon.2) 413 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-122"}]: dispatch 2024-02-23T00:54:05.554 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:04.353239+0000 mon.a (mon.0) 3456 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-122"}]: dispatch 2024-02-23T00:54:05.554 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:05.113842+0000 mon.c (mon.2) 414 : audit [INF] from='client.? 172.21.15.47:0/2120819500' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:54:05.554 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:05.114176+0000 mon.a (mon.0) 3457 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:54:05.555 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:05.284815+0000 mon.a (mon.0) 3458 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:54:05.555 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:05.289111+0000 mon.a (mon.0) 3459 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-122"}]': finished 2024-02-23T00:54:05.555 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:05.289195+0000 mon.a (mon.0) 3460 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:54:05.555 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:05.289252+0000 mon.a (mon.0) 3461 : cluster [DBG] osdmap e706: 8 total, 8 up, 6 in 2024-02-23T00:54:05.555 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:05.654 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:54:05.656 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:05.656 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:54:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:03.456032+0000 osd.6 (osd.6) 271 : cluster [DBG] 5.3 scrub starts 2024-02-23T00:54:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:03.457061+0000 osd.6 (osd.6) 272 : cluster [DBG] 5.3 scrub ok 2024-02-23T00:54:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:04.346662+0000 mon.a (mon.0) 3454 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:54:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:04.346742+0000 mon.a (mon.0) 3455 : cluster [DBG] osdmap e705: 8 total, 8 up, 6 in 2024-02-23T00:54:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:04.352954+0000 mon.c (mon.2) 413 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-122"}]: dispatch 2024-02-23T00:54:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:04.353239+0000 mon.a (mon.0) 3456 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-122"}]: dispatch 2024-02-23T00:54:05.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:05.113842+0000 mon.c (mon.2) 414 : audit [INF] from='client.? 172.21.15.47:0/2120819500' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:54:05.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:05.114176+0000 mon.a (mon.0) 3457 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:54:05.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:05.284815+0000 mon.a (mon.0) 3458 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:54:05.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:05.289111+0000 mon.a (mon.0) 3459 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-122"}]': finished 2024-02-23T00:54:05.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:05.289195+0000 mon.a (mon.0) 3460 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:54:05.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:05.289252+0000 mon.a (mon.0) 3461 : cluster [DBG] osdmap e706: 8 total, 8 up, 6 in 2024-02-23T00:54:05.757 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:54:05.785 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:05.908 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:54:05.911 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:05.912 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:54:06.013 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:54:06.044 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:06.066 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:54:06.069 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:06.069 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:54:06.151 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:54:06.169 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:54:06.204 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:06.225 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:54:06.227 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:06.228 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:54:06.328 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:54:06.351 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:05.380808+0000 mgr.y (mgr.14517) 608 : cluster [DBG] pgmap v1068: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s; 1/717 objects misplaced (0.139%) 2024-02-23T00:54:06.351 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:05.802277+0000 osd.3 (osd.3) 321 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:54:06.351 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:05.802366+0000 osd.3 (osd.3) 322 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:54:06.352 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:06.290851+0000 mon.a (mon.0) 3462 : cluster [DBG] osdmap e707: 8 total, 8 up, 6 in 2024-02-23T00:54:06.352 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:06.336243+0000 mon.c (mon.2) 415 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:06.352 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:06.336601+0000 mon.a (mon.0) 3463 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:06.358 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:06.377 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:54:06.380 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:06.381 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:54:06.481 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:54:06.491 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:05.380808+0000 mgr.y (mgr.14517) 608 : cluster [DBG] pgmap v1068: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s; 1/717 objects misplaced (0.139%) 2024-02-23T00:54:06.491 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:05.802277+0000 osd.3 (osd.3) 321 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:54:06.491 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:05.802366+0000 osd.3 (osd.3) 322 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:54:06.491 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:06.290851+0000 mon.a (mon.0) 3462 : cluster [DBG] osdmap e707: 8 total, 8 up, 6 in 2024-02-23T00:54:06.491 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:06.336243+0000 mon.c (mon.2) 415 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:06.491 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:06.336601+0000 mon.a (mon.0) 3463 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:06.492 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:05.380808+0000 mgr.y (mgr.14517) 608 : cluster [DBG] pgmap v1068: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s; 1/717 objects misplaced (0.139%) 2024-02-23T00:54:06.492 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:05.802277+0000 osd.3 (osd.3) 321 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:54:06.492 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:05.802366+0000 osd.3 (osd.3) 322 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:54:06.492 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:06.290851+0000 mon.a (mon.0) 3462 : cluster [DBG] osdmap e707: 8 total, 8 up, 6 in 2024-02-23T00:54:06.492 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:06.336243+0000 mon.c (mon.2) 415 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:06.492 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:06.336601+0000 mon.a (mon.0) 3463 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:06.509 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:06.529 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:54:06.531 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:06.531 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:54:06.632 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:54:06.660 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:06.680 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:54:06.682 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:06.682 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:54:06.783 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:54:06.810 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:06.829 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:54:06.832 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:06.832 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:54:06.933 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:54:06.962 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:07.112 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:54:07.114 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:07.115 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:54:07.216 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:54:07.244 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:07.264 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:54:07.266 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:07.266 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:54:07.367 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:54:07.396 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:05.453317+0000 osd.6 (osd.6) 273 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:54:07.397 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:05.453520+0000 osd.6 (osd.6) 274 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:54:07.397 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:06.316085+0000 osd.1 (osd.1) 285 : cluster [DBG] 3.0 deep-scrub starts 2024-02-23T00:54:07.397 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:06.324301+0000 osd.1 (osd.1) 286 : cluster [DBG] 3.0 deep-scrub ok 2024-02-23T00:54:07.397 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:07.273590+0000 osd.1 (osd.1) 287 : cluster [DBG] 5.c scrub starts 2024-02-23T00:54:07.397 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:07.276545+0000 osd.1 (osd.1) 288 : cluster [DBG] 5.c scrub ok 2024-02-23T00:54:07.397 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:07.292829+0000 mon.a (mon.0) 3464 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:54:07.397 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:07.292914+0000 mon.a (mon.0) 3465 : cluster [DBG] osdmap e708: 8 total, 8 up, 6 in 2024-02-23T00:54:07.397 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:07.293854+0000 mon.c (mon.2) 416 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-122"}]: dispatch 2024-02-23T00:54:07.397 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:07.294227+0000 mon.a (mon.0) 3466 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-122"}]: dispatch 2024-02-23T00:54:07.397 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:05.453317+0000 osd.6 (osd.6) 273 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:54:07.397 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:05.453520+0000 osd.6 (osd.6) 274 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:54:07.398 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:06.316085+0000 osd.1 (osd.1) 285 : cluster [DBG] 3.0 deep-scrub starts 2024-02-23T00:54:07.398 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:06.324301+0000 osd.1 (osd.1) 286 : cluster [DBG] 3.0 deep-scrub ok 2024-02-23T00:54:07.398 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:07.273590+0000 osd.1 (osd.1) 287 : cluster [DBG] 5.c scrub starts 2024-02-23T00:54:07.398 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:07.276545+0000 osd.1 (osd.1) 288 : cluster [DBG] 5.c scrub ok 2024-02-23T00:54:07.398 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:07.292829+0000 mon.a (mon.0) 3464 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:54:07.398 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:07.292914+0000 mon.a (mon.0) 3465 : cluster [DBG] osdmap e708: 8 total, 8 up, 6 in 2024-02-23T00:54:07.398 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:07.293854+0000 mon.c (mon.2) 416 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-122"}]: dispatch 2024-02-23T00:54:07.398 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:07.294227+0000 mon.a (mon.0) 3466 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-122"}]: dispatch 2024-02-23T00:54:07.399 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:07.549 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:54:07.552 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:07.552 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:54:07.653 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:54:07.684 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:07.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:05.453317+0000 osd.6 (osd.6) 273 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:54:07.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:05.453520+0000 osd.6 (osd.6) 274 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:54:07.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:06.316085+0000 osd.1 (osd.1) 285 : cluster [DBG] 3.0 deep-scrub starts 2024-02-23T00:54:07.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:06.324301+0000 osd.1 (osd.1) 286 : cluster [DBG] 3.0 deep-scrub ok 2024-02-23T00:54:07.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:07.273590+0000 osd.1 (osd.1) 287 : cluster [DBG] 5.c scrub starts 2024-02-23T00:54:07.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:07.276545+0000 osd.1 (osd.1) 288 : cluster [DBG] 5.c scrub ok 2024-02-23T00:54:07.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:07.292829+0000 mon.a (mon.0) 3464 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:54:07.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:07.292914+0000 mon.a (mon.0) 3465 : cluster [DBG] osdmap e708: 8 total, 8 up, 6 in 2024-02-23T00:54:07.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:07.293854+0000 mon.c (mon.2) 416 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-122"}]: dispatch 2024-02-23T00:54:07.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:07.294227+0000 mon.a (mon.0) 3466 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-122"}]: dispatch 2024-02-23T00:54:07.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.759+0000 7fb8a9379700 1 -- 172.21.15.47:0/1404020518 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb8a40fcbb0 msgr2=0x7fb8a40fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:07.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.759+0000 7fb8a9379700 1 --2- 172.21.15.47:0/1404020518 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb8a40fcbb0 0x7fb8a40fcf90 secure :-1 s=READY pgs=2782 cs=0 l=1 rev1=1 rx=0x7fb89c004870 tx=0x7fb89c009cb0).stop 2024-02-23T00:54:07.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.759+0000 7fb8a9379700 1 -- 172.21.15.47:0/1404020518 shutdown_connections 2024-02-23T00:54:07.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.759+0000 7fb8a9379700 1 --2- 172.21.15.47:0/1404020518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8a40f7400 0x7fb8a4103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:07.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.759+0000 7fb8a9379700 1 --2- 172.21.15.47:0/1404020518 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb8a40fcbb0 0x7fb8a40fcf90 unknown :-1 s=CLOSED pgs=2782 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:07.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.759+0000 7fb8a9379700 1 --2- 172.21.15.47:0/1404020518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8a40fd510 0x7fb8a40f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:07.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.759+0000 7fb8a9379700 1 -- 172.21.15.47:0/1404020518 >> 172.21.15.47:0/1404020518 conn(0x7fb8a40f2820 msgr2=0x7fb8a40f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:07.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.759+0000 7fb8a9379700 1 -- 172.21.15.47:0/1404020518 shutdown_connections 2024-02-23T00:54:07.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.759+0000 7fb8a9379700 1 -- 172.21.15.47:0/1404020518 wait complete. 2024-02-23T00:54:07.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.760+0000 7fb8a9379700 1 librados: starting msgr at 2024-02-23T00:54:07.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.760+0000 7fb8a9379700 1 librados: starting objecter 2024-02-23T00:54:07.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.760+0000 7fb8a9379700 1 Processor -- start 2024-02-23T00:54:07.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.760+0000 7fb8a9379700 1 -- start start 2024-02-23T00:54:07.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.760+0000 7fb8a9379700 1 librados: setting wanted keys 2024-02-23T00:54:07.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.760+0000 7fb8a9379700 1 librados: calling monclient init 2024-02-23T00:54:07.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.760+0000 7fb8a9379700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb8a40f7400 0x7fb8a40f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:07.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.760+0000 7fb8a9379700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8a40fcbb0 0x7fb8a40f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:07.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.760+0000 7fb8a9379700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8a40fd510 0x7fb8a40f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:07.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.760+0000 7fb8a9379700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a4116470 con 0x7fb8a40fd510 2024-02-23T00:54:07.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.760+0000 7fb8a9379700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a41165e0 con 0x7fb8a40fcbb0 2024-02-23T00:54:07.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.760+0000 7fb8a9379700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a41167a0 con 0x7fb8a40f7400 2024-02-23T00:54:07.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.760+0000 7fb8a2ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb8a40f7400 0x7fb8a40f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:07.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.760+0000 7fb8a2ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb8a40f7400 0x7fb8a40f89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:39924/0 (socket says 172.21.15.47:39924) 2024-02-23T00:54:07.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.760+0000 7fb8a2ffd700 1 -- 172.21.15.47:0/3302287872 learned_addr learned my addr 172.21.15.47:0/3302287872 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:54:07.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.760+0000 7fb8a37fe700 1 --2- 172.21.15.47:0/3302287872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8a40fd510 0x7fb8a40f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:07.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.760+0000 7fb8a27fc700 1 --2- 172.21.15.47:0/3302287872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8a40fcbb0 0x7fb8a40f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:07.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.761+0000 7fb8a2ffd700 1 -- 172.21.15.47:0/3302287872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8a40fcbb0 msgr2=0x7fb8a40f8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:07.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.761+0000 7fb8a2ffd700 1 --2- 172.21.15.47:0/3302287872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8a40fcbb0 0x7fb8a40f8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:07.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.761+0000 7fb8a2ffd700 1 -- 172.21.15.47:0/3302287872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8a40fd510 msgr2=0x7fb8a40f9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:07.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.761+0000 7fb8a2ffd700 1 --2- 172.21.15.47:0/3302287872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8a40fd510 0x7fb8a40f9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:07.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.761+0000 7fb8a2ffd700 1 -- 172.21.15.47:0/3302287872 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8940097c0 con 0x7fb8a40f7400 2024-02-23T00:54:07.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.761+0000 7fb8a2ffd700 1 --2- 172.21.15.47:0/3302287872 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb8a40f7400 0x7fb8a40f89b0 secure :-1 s=READY pgs=2783 cs=0 l=1 rev1=1 rx=0x7fb89c009da0 tx=0x7fb89c009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:07.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.761+0000 7fb89bfff700 10 client.?.objecter ms_handle_connect 0x7fb8a40f7400 2024-02-23T00:54:07.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.761+0000 7fb89bfff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:54:07.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.761+0000 7fb89bfff700 1 -- 172.21.15.47:0/3302287872 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89c00e030 con 0x7fb8a40f7400 2024-02-23T00:54:07.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.761+0000 7fb89bfff700 1 -- 172.21.15.47:0/3302287872 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fb89c013d70 con 0x7fb8a40f7400 2024-02-23T00:54:07.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.761+0000 7fb89bfff700 1 -- 172.21.15.47:0/3302287872 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89c018070 con 0x7fb8a40f7400 2024-02-23T00:54:07.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.761+0000 7fb8a9379700 1 -- 172.21.15.47:0/3302287872 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb89c005040 con 0x7fb8a40f7400 2024-02-23T00:54:07.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.761+0000 7fb8a9379700 10 client.41978.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:54:07.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.761+0000 7fb8a9379700 1 -- 172.21.15.47:0/3302287872 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a4116c40 con 0x7fb8a40f7400 2024-02-23T00:54:07.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.761+0000 7fb8a9379700 1 librados: init done 2024-02-23T00:54:07.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.762+0000 7fb8a9379700 1 -- 172.21.15.47:0/3302287872 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8a405fa60 con 0x7fb8a40f7400 2024-02-23T00:54:07.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.762+0000 7fb89bfff700 1 -- 172.21.15.47:0/3302287872 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fb89c009240 con 0x7fb8a40f7400 2024-02-23T00:54:07.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.763+0000 7fb89bfff700 1 --2- 172.21.15.47:0/3302287872 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb890062cc0 0x7fb890065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:07.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.763+0000 7fb89bfff700 1 -- 172.21.15.47:0/3302287872 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(708..708 src has 482..708) v4 ==== 9780+0+0 (secure 0 0 0) 0x7fb89c07eaf0 con 0x7fb8a40f7400 2024-02-23T00:54:07.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.763+0000 7fb89bfff700 10 client.41978.objecter ms_dispatch 0x7fb8a4001550 osd_map(708..708 src has 482..708) v4 2024-02-23T00:54:07.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.763+0000 7fb89bfff700 3 client.41978.objecter handle_osd_map got epochs [708,708] > 0 2024-02-23T00:54:07.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.763+0000 7fb89bfff700 3 client.41978.objecter handle_osd_map decoding full epoch 708 2024-02-23T00:54:07.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.763+0000 7fb8a27fc700 1 --2- 172.21.15.47:0/3302287872 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb890062cc0 0x7fb890065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:07.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.763+0000 7fb89bfff700 20 client.41978.objecter dump_active .. 0 homeless 2024-02-23T00:54:07.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.763+0000 7fb89bfff700 10 client.41978.objecter ms_handle_connect 0x7fb890062cc0 2024-02-23T00:54:07.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.763+0000 7fb8a27fc700 1 --2- 172.21.15.47:0/3302287872 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb890062cc0 0x7fb890065180 secure :-1 s=READY pgs=3173 cs=0 l=1 rev1=1 rx=0x7fb8a40fa150 tx=0x7fb88c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:07.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.767+0000 7fb89bfff700 1 -- 172.21.15.47:0/3302287872 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb89c04ded0 con 0x7fb8a40f7400 2024-02-23T00:54:07.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:07.953+0000 7fb8a9379700 1 -- 172.21.15.47:0/3302287872 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd reweight", "id": 5, "weight": 1.0} v 0) v1 -- 0x7fb8a40f9e90 con 0x7fb8a40f7400 2024-02-23T00:54:07.962 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:54:07.993 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:54:07.993 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-02-23T00:54:07.993 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-02-23T00:54:07.993 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-02-23T00:54:07.993 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:54:08.234 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:54:08.237 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:08.237 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:54:08.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:08.293+0000 7fb89bfff700 1 -- 172.21.15.47:0/3302287872 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 5, "weight": 1.0}]=0 reweighted osd.5 to 1 (10000) v709) v1 ==== 113+0+0 (secure 0 0 0) 0x7fb89c0514f0 con 0x7fb8a40f7400 2024-02-23T00:54:08.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:08.295+0000 7fb8a9379700 10 librados: watch_flush enter 2024-02-23T00:54:08.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:08.295+0000 7fb8a9379700 10 librados: watch_flush exit 2024-02-23T00:54:08.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:08.295+0000 7fb8a9379700 20 client.41978.objecter shutdown clearing up homeless session... 2024-02-23T00:54:08.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:08.295+0000 7fb8a9379700 10 client.41978.objecter successfully canceled tick 2024-02-23T00:54:08.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:08.295+0000 7fb8a9379700 1 -- 172.21.15.47:0/3302287872 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb890062cc0 msgr2=0x7fb890065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:08.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:08.296+0000 7fb8a9379700 1 --2- 172.21.15.47:0/3302287872 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb890062cc0 0x7fb890065180 secure :-1 s=READY pgs=3173 cs=0 l=1 rev1=1 rx=0x7fb8a40fa150 tx=0x7fb88c00b040).stop 2024-02-23T00:54:08.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:08.296+0000 7fb8a9379700 1 -- 172.21.15.47:0/3302287872 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb8a40f7400 msgr2=0x7fb8a40f89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:08.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:08.296+0000 7fb8a9379700 1 --2- 172.21.15.47:0/3302287872 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb8a40f7400 0x7fb8a40f89b0 secure :-1 s=READY pgs=2783 cs=0 l=1 rev1=1 rx=0x7fb89c009da0 tx=0x7fb89c009cb0).stop 2024-02-23T00:54:08.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:08.296+0000 7fb8a9379700 1 -- 172.21.15.47:0/3302287872 shutdown_connections 2024-02-23T00:54:08.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:08.296+0000 7fb8a9379700 1 --2- 172.21.15.47:0/3302287872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8a40fd510 0x7fb8a40f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:08.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:08.296+0000 7fb8a9379700 1 --2- 172.21.15.47:0/3302287872 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb890062cc0 0x7fb890065180 unknown :-1 s=CLOSED pgs=3173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:08.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:08.296+0000 7fb8a9379700 1 --2- 172.21.15.47:0/3302287872 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb8a40f7400 0x7fb8a40f89b0 unknown :-1 s=CLOSED pgs=2783 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:08.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:08.296+0000 7fb8a9379700 1 --2- 172.21.15.47:0/3302287872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8a40fcbb0 0x7fb8a40f8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:08.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:08.296+0000 7fb8a9379700 1 -- 172.21.15.47:0/3302287872 >> 172.21.15.47:0/3302287872 conn(0x7fb8a40f2820 msgr2=0x7fb8a4060eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:08.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:08.296+0000 7fb8a9379700 1 -- 172.21.15.47:0/3302287872 shutdown_connections 2024-02-23T00:54:08.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:08.296+0000 7fb8a9379700 1 -- 172.21.15.47:0/3302287872 wait complete. 2024-02-23T00:54:08.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:08.296+0000 7fb8a9379700 1 librados: shutdown 2024-02-23T00:54:08.300 INFO:teuthology.orchestra.run.smithi047.stderr:reweighted osd.5 to 1 (10000) 2024-02-23T00:54:08.338 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:54:08.369 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:08.508 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set noscrub 2024-02-23T00:54:08.570 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:07.345995+0000 osd.5 (osd.5) 312 : cluster [DBG] 234.3 scrub starts 2024-02-23T00:54:08.570 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:07.348017+0000 osd.5 (osd.5) 313 : cluster [DBG] 234.3 scrub ok 2024-02-23T00:54:08.570 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:07.381198+0000 mgr.y (mgr.14517) 609 : cluster [DBG] pgmap v1071: 184 pgs: 2 active+clean+snaptrim, 1 active+remapped+backfill_toofull, 1 active+clean+scrubbing, 180 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s; 1/726 objects misplaced (0.138%) 2024-02-23T00:54:08.571 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:07.954661+0000 mon.c (mon.2) 417 : audit [INF] from='client.? 172.21.15.47:0/3302287872' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-23T00:54:08.571 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:07.955048+0000 mon.a (mon.0) 3467 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-23T00:54:08.571 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:08.290048+0000 mon.a (mon.0) 3468 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:54:08.571 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:08.294295+0000 mon.a (mon.0) 3469 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-122"}]': finished 2024-02-23T00:54:08.571 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:08.294366+0000 mon.a (mon.0) 3470 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-23T00:54:08.571 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:08.294429+0000 mon.a (mon.0) 3471 : cluster [DBG] osdmap e709: 8 total, 8 up, 6 in 2024-02-23T00:54:08.571 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:07.345995+0000 osd.5 (osd.5) 312 : cluster [DBG] 234.3 scrub starts 2024-02-23T00:54:08.571 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:07.348017+0000 osd.5 (osd.5) 313 : cluster [DBG] 234.3 scrub ok 2024-02-23T00:54:08.571 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:07.381198+0000 mgr.y (mgr.14517) 609 : cluster [DBG] pgmap v1071: 184 pgs: 2 active+clean+snaptrim, 1 active+remapped+backfill_toofull, 1 active+clean+scrubbing, 180 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s; 1/726 objects misplaced (0.138%) 2024-02-23T00:54:08.571 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:07.954661+0000 mon.c (mon.2) 417 : audit [INF] from='client.? 172.21.15.47:0/3302287872' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-23T00:54:08.571 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:07.955048+0000 mon.a (mon.0) 3467 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-23T00:54:08.572 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:08.290048+0000 mon.a (mon.0) 3468 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:54:08.572 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:08.294295+0000 mon.a (mon.0) 3469 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-122"}]': finished 2024-02-23T00:54:08.572 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:08.294366+0000 mon.a (mon.0) 3470 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-23T00:54:08.572 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:08.294429+0000 mon.a (mon.0) 3471 : cluster [DBG] osdmap e709: 8 total, 8 up, 6 in 2024-02-23T00:54:08.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:07.345995+0000 osd.5 (osd.5) 312 : cluster [DBG] 234.3 scrub starts 2024-02-23T00:54:08.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:07.348017+0000 osd.5 (osd.5) 313 : cluster [DBG] 234.3 scrub ok 2024-02-23T00:54:08.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:07.381198+0000 mgr.y (mgr.14517) 609 : cluster [DBG] pgmap v1071: 184 pgs: 2 active+clean+snaptrim, 1 active+remapped+backfill_toofull, 1 active+clean+scrubbing, 180 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s; 1/726 objects misplaced (0.138%) 2024-02-23T00:54:08.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:07.954661+0000 mon.c (mon.2) 417 : audit [INF] from='client.? 172.21.15.47:0/3302287872' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-23T00:54:08.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:07.955048+0000 mon.a (mon.0) 3467 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-23T00:54:08.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:08.290048+0000 mon.a (mon.0) 3468 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:54:08.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:08.294295+0000 mon.a (mon.0) 3469 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-122"}]': finished 2024-02-23T00:54:08.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:08.294366+0000 mon.a (mon.0) 3470 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-23T00:54:08.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:08.294429+0000 mon.a (mon.0) 3471 : cluster [DBG] osdmap e709: 8 total, 8 up, 6 in 2024-02-23T00:54:09.421 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:54:09.425 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:09.425 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:54:09.526 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:54:09.558 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:09.942 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight 6 1 2024-02-23T00:54:10.009 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:08.393839+0000 osd.5 (osd.5) 314 : cluster [DBG] 6.7 scrub starts 2024-02-23T00:54:10.009 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:08.396851+0000 osd.5 (osd.5) 315 : cluster [DBG] 6.7 scrub ok 2024-02-23T00:54:10.009 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:09.295878+0000 mon.a (mon.0) 3472 : cluster [DBG] osdmap e710: 8 total, 8 up, 6 in 2024-02-23T00:54:10.009 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:08.393839+0000 osd.5 (osd.5) 314 : cluster [DBG] 6.7 scrub starts 2024-02-23T00:54:10.009 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:08.396851+0000 osd.5 (osd.5) 315 : cluster [DBG] 6.7 scrub ok 2024-02-23T00:54:10.009 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:09.295878+0000 mon.a (mon.0) 3472 : cluster [DBG] osdmap e710: 8 total, 8 up, 6 in 2024-02-23T00:54:10.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:08.393839+0000 osd.5 (osd.5) 314 : cluster [DBG] 6.7 scrub starts 2024-02-23T00:54:10.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:08.396851+0000 osd.5 (osd.5) 315 : cluster [DBG] 6.7 scrub ok 2024-02-23T00:54:10.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:09.295878+0000 mon.a (mon.0) 3472 : cluster [DBG] osdmap e710: 8 total, 8 up, 6 in 2024-02-23T00:54:10.557 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:54:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:54:10] "GET /metrics HTTP/1.1" 200 188863 "" "Prometheus/2.33.4" 2024-02-23T00:54:10.708 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:54:10.711 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:10.711 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:54:10.714 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.0 dump_historic_ops 2024-02-23T00:54:10.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.471+0000 7f350eb9f700 1 -- 172.21.15.47:0/3047852806 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35080fdbb0 msgr2=0x7f35080fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:10.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.471+0000 7f350eb9f700 1 --2- 172.21.15.47:0/3047852806 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35080fdbb0 0x7f35080fdf90 secure :-1 s=READY pgs=2784 cs=0 l=1 rev1=1 rx=0x7f34fc004870 tx=0x7f34fc009cb0).stop 2024-02-23T00:54:10.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.471+0000 7f350eb9f700 1 -- 172.21.15.47:0/3047852806 shutdown_connections 2024-02-23T00:54:10.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.471+0000 7f350eb9f700 1 --2- 172.21.15.47:0/3047852806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35080f8420 0x7f3508104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:10.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.471+0000 7f350eb9f700 1 --2- 172.21.15.47:0/3047852806 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35080fdbb0 0x7f35080fdf90 unknown :-1 s=CLOSED pgs=2784 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:10.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.471+0000 7f350eb9f700 1 --2- 172.21.15.47:0/3047852806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35080fe510 0x7f35080f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:10.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.471+0000 7f350eb9f700 1 -- 172.21.15.47:0/3047852806 >> 172.21.15.47:0/3047852806 conn(0x7f35080f3800 msgr2=0x7f35080f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:10.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.471+0000 7f350eb9f700 1 -- 172.21.15.47:0/3047852806 shutdown_connections 2024-02-23T00:54:10.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.471+0000 7f350eb9f700 1 -- 172.21.15.47:0/3047852806 wait complete. 2024-02-23T00:54:10.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.471+0000 7f350eb9f700 1 librados: starting msgr at 2024-02-23T00:54:10.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.471+0000 7f350eb9f700 1 librados: starting objecter 2024-02-23T00:54:10.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.471+0000 7f350eb9f700 1 Processor -- start 2024-02-23T00:54:10.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.471+0000 7f350eb9f700 1 -- start start 2024-02-23T00:54:10.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.471+0000 7f350eb9f700 1 librados: setting wanted keys 2024-02-23T00:54:10.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.471+0000 7f350eb9f700 1 librados: calling monclient init 2024-02-23T00:54:10.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.472+0000 7f350eb9f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35080f8420 0x7f350810df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:10.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.472+0000 7f350eb9f700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35080fdbb0 0x7f350810e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:10.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.472+0000 7f350eb9f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35080fe510 0x7f3508108120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:10.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.472+0000 7f350eb9f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3508108610 con 0x7f35080fe510 2024-02-23T00:54:10.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.472+0000 7f350eb9f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3508108780 con 0x7f35080f8420 2024-02-23T00:54:10.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.472+0000 7f350eb9f700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3508108940 con 0x7f35080fdbb0 2024-02-23T00:54:10.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.472+0000 7f3507fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35080fdbb0 0x7f350810e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:10.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.472+0000 7f3507fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35080fdbb0 0x7f350810e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:39940/0 (socket says 172.21.15.47:39940) 2024-02-23T00:54:10.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.472+0000 7f3507fff700 1 -- 172.21.15.47:0/1985591043 learned_addr learned my addr 172.21.15.47:0/1985591043 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:54:10.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.472+0000 7f350d13c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35080fe510 0x7f3508108120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:10.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.472+0000 7f350c93b700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35080f8420 0x7f350810df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:10.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.472+0000 7f3507fff700 1 -- 172.21.15.47:0/1985591043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35080f8420 msgr2=0x7f350810df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:10.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.472+0000 7f3507fff700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35080f8420 0x7f350810df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:10.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.472+0000 7f3507fff700 1 -- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35080fe510 msgr2=0x7f3508108120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:10.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.472+0000 7f3507fff700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35080fe510 0x7f3508108120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:10.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.472+0000 7f3507fff700 1 -- 172.21.15.47:0/1985591043 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f4005040 con 0x7f35080fdbb0 2024-02-23T00:54:10.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.472+0000 7f350d13c700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35080fe510 0x7f3508108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:54:10.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.472+0000 7f3507fff700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35080fdbb0 0x7f350810e420 secure :-1 s=READY pgs=2785 cs=0 l=1 rev1=1 rx=0x7f34f40048f0 tx=0x7f34f4009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:10.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.472+0000 7f3505ffb700 10 client.?.objecter ms_handle_connect 0x7f35080fdbb0 2024-02-23T00:54:10.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.472+0000 7f3505ffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:54:10.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.472+0000 7f3505ffb700 1 -- 172.21.15.47:0/1985591043 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f4007b00 con 0x7f35080fdbb0 2024-02-23T00:54:10.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.472+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34fc005040 con 0x7f35080fdbb0 2024-02-23T00:54:10.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.472+0000 7f3505ffb700 1 -- 172.21.15.47:0/1985591043 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f34f400f040 con 0x7f35080fdbb0 2024-02-23T00:54:10.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.472+0000 7f3505ffb700 1 -- 172.21.15.47:0/1985591043 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f4014820 con 0x7f35080fdbb0 2024-02-23T00:54:10.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.473+0000 7f350eb9f700 10 client.41984.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:54:10.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.473+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f350806af10 con 0x7f35080fdbb0 2024-02-23T00:54:10.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.473+0000 7f350eb9f700 1 librados: init done 2024-02-23T00:54:10.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.473+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f3508108ff0 con 0x7f35080fdbb0 2024-02-23T00:54:10.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.474+0000 7f3505ffb700 1 -- 172.21.15.47:0/1985591043 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f34f4014c50 con 0x7f35080fdbb0 2024-02-23T00:54:10.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.474+0000 7f3505ffb700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f34f8062cc0 0x7f34f8065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:10.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.474+0000 7f3505ffb700 1 -- 172.21.15.47:0/1985591043 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(711..711 src has 621..711) v4 ==== 9772+0+0 (secure 0 0 0) 0x7f34f407d3e0 con 0x7f35080fdbb0 2024-02-23T00:54:10.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.474+0000 7f3505ffb700 10 client.41984.objecter ms_dispatch 0x7f3508001550 osd_map(711..711 src has 621..711) v4 2024-02-23T00:54:10.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.474+0000 7f3505ffb700 3 client.41984.objecter handle_osd_map got epochs [711,711] > 0 2024-02-23T00:54:10.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.474+0000 7f3505ffb700 3 client.41984.objecter handle_osd_map decoding full epoch 711 2024-02-23T00:54:10.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.474+0000 7f350c93b700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f34f8062cc0 0x7f34f8065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:10.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.474+0000 7f3505ffb700 20 client.41984.objecter dump_active .. 0 homeless 2024-02-23T00:54:10.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.474+0000 7f3505ffb700 1 -- 172.21.15.47:0/1985591043 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v711) v1 ==== 54+0+16 (secure 0 0 0) 0x7f3508108ff0 con 0x7f35080fdbb0 2024-02-23T00:54:10.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.475+0000 7f350eb9f700 10 client.41984.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:10.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.475+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_assign -1 1 2024-02-23T00:54:10.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.475+0000 7f3505ffb700 10 client.41984.objecter ms_handle_connect 0x7f34f8062cc0 2024-02-23T00:54:10.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.475+0000 7f350c93b700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f34f8062cc0 0x7f34f8065180 secure :-1 s=READY pgs=3174 cs=0 l=1 rev1=1 rx=0x7f34fc004ee0 tx=0x7f34fc00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:10.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.475+0000 7f350eb9f700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f3508028540 0x7f35080fe950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:10.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.475+0000 7f350eb9f700 20 client.41984.objecter _get_session s=0x7f3508108ff0 osd=0 3 2024-02-23T00:54:10.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.475+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f3508108ff0 osd=0 3 2024-02-23T00:54:10.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.475+0000 7f350eb9f700 20 client.41984.objecter _get_session s=0x7f3508108ff0 osd=0 3 2024-02-23T00:54:10.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.475+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_remove -1 1 2024-02-23T00:54:10.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.475+0000 7f350eb9f700 20 client.41984.objecter get_session s=0x7f3508108ff0 osd=0 3 2024-02-23T00:54:10.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.475+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_assign 0 1 2024-02-23T00:54:10.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.475+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f3508108ff0 osd=0 4 2024-02-23T00:54:10.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.475+0000 7f350eb9f700 10 client.41984.objecter _send_command 1 2024-02-23T00:54:10.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.475+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f350805ee80 con 0x7f3508028540 2024-02-23T00:54:10.805 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:54:10.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.475+0000 7f350d13c700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f3508028540 0x7f35080fe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:10.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.475+0000 7f3505ffb700 10 client.41984.objecter ms_handle_connect 0x7f3508028540 2024-02-23T00:54:10.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.475+0000 7f350d13c700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f3508028540 0x7f35080fe950 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:10.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.476+0000 7f3505ffb700 1 -- 172.21.15.47:0/1985591043 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f350805ee80 con 0x7f3508028540 2024-02-23T00:54:10.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.476+0000 7f3505ffb700 10 client.41984.objecter ms_dispatch 0x7f3508001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:54:10.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.476+0000 7f3505ffb700 10 client.41984.objecter _finish_command 1 = system:0 2024-02-23T00:54:10.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.476+0000 7f3505ffb700 20 client.41984.objecter put_session s=0x7f3508108ff0 osd=0 4 2024-02-23T00:54:10.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.476+0000 7f3505ffb700 15 client.41984.objecter _session_command_op_remove 0 1 2024-02-23T00:54:10.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.491+0000 7f350eb9f700 10 client.41984.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:54:10.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.491+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_assign -1 2 2024-02-23T00:54:10.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.491+0000 7f350eb9f700 20 client.41984.objecter _get_session s=0x7f3508108ff0 osd=0 3 2024-02-23T00:54:10.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.491+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f3508108ff0 osd=0 3 2024-02-23T00:54:10.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.491+0000 7f350eb9f700 20 client.41984.objecter _get_session s=0x7f3508108ff0 osd=0 3 2024-02-23T00:54:10.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.491+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_remove -1 2 2024-02-23T00:54:10.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.491+0000 7f350eb9f700 20 client.41984.objecter get_session s=0x7f3508108ff0 osd=0 3 2024-02-23T00:54:10.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.491+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_assign 0 2 2024-02-23T00:54:10.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.491+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f3508108ff0 osd=0 4 2024-02-23T00:54:10.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.491+0000 7f350eb9f700 10 client.41984.objecter _send_command 2 2024-02-23T00:54:10.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.491+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f3508106e60 con 0x7f3508028540 2024-02-23T00:54:10.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.492+0000 7f3505ffb700 1 -- 172.21.15.47:0/1985591043 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f3508106e60 con 0x7f3508028540 2024-02-23T00:54:10.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.492+0000 7f3505ffb700 10 client.41984.objecter ms_dispatch 0x7f3508001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:54:10.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.492+0000 7f3505ffb700 10 client.41984.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:54:10.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.492+0000 7f3505ffb700 20 client.41984.objecter put_session s=0x7f3508108ff0 osd=0 4 2024-02-23T00:54:10.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.492+0000 7f3505ffb700 15 client.41984.objecter _session_command_op_remove 0 2 2024-02-23T00:54:10.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.492+0000 7f350eb9f700 10 client.41984.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:10.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.492+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_assign -1 3 2024-02-23T00:54:10.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.492+0000 7f350eb9f700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f35080ff350 0x7f35080ff730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:10.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.492+0000 7f350eb9f700 20 client.41984.objecter _get_session s=0x7f350805a040 osd=1 3 2024-02-23T00:54:10.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.492+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f350805a040 osd=1 3 2024-02-23T00:54:10.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.492+0000 7f350eb9f700 20 client.41984.objecter _get_session s=0x7f350805a040 osd=1 3 2024-02-23T00:54:10.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.492+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_remove -1 3 2024-02-23T00:54:10.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.492+0000 7f350eb9f700 20 client.41984.objecter get_session s=0x7f350805a040 osd=1 3 2024-02-23T00:54:10.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.492+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_assign 1 3 2024-02-23T00:54:10.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.492+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f350805a040 osd=1 4 2024-02-23T00:54:10.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.492+0000 7f350eb9f700 10 client.41984.objecter _send_command 3 2024-02-23T00:54:10.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.492+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35080ffc20 con 0x7f35080ff350 2024-02-23T00:54:10.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.492+0000 7f350c93b700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f35080ff350 0x7f35080ff730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:10.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.493+0000 7f3505ffb700 10 client.41984.objecter ms_handle_connect 0x7f35080ff350 2024-02-23T00:54:10.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.493+0000 7f350c93b700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f35080ff350 0x7f35080ff730 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:10.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.493+0000 7f3505ffb700 1 -- 172.21.15.47:0/1985591043 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f35080ffc20 con 0x7f35080ff350 2024-02-23T00:54:10.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.493+0000 7f3505ffb700 10 client.41984.objecter ms_dispatch 0x7f3508001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:54:10.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.493+0000 7f3505ffb700 10 client.41984.objecter _finish_command 3 = system:0 2024-02-23T00:54:10.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.493+0000 7f3505ffb700 20 client.41984.objecter put_session s=0x7f350805a040 osd=1 4 2024-02-23T00:54:10.809 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:54:10.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.493+0000 7f3505ffb700 15 client.41984.objecter _session_command_op_remove 1 3 2024-02-23T00:54:10.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.508+0000 7f350eb9f700 10 client.41984.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:54:10.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.508+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_assign -1 4 2024-02-23T00:54:10.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.508+0000 7f350eb9f700 20 client.41984.objecter _get_session s=0x7f350805a040 osd=1 3 2024-02-23T00:54:10.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.508+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f350805a040 osd=1 3 2024-02-23T00:54:10.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.508+0000 7f350eb9f700 20 client.41984.objecter _get_session s=0x7f350805a040 osd=1 3 2024-02-23T00:54:10.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.508+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_remove -1 4 2024-02-23T00:54:10.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.508+0000 7f350eb9f700 20 client.41984.objecter get_session s=0x7f350805a040 osd=1 3 2024-02-23T00:54:10.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.508+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_assign 1 4 2024-02-23T00:54:10.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.508+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f350805a040 osd=1 4 2024-02-23T00:54:10.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.508+0000 7f350eb9f700 10 client.41984.objecter _send_command 4 2024-02-23T00:54:10.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.508+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f35081000c0 con 0x7f35080ff350 2024-02-23T00:54:10.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.509+0000 7f3505ffb700 1 -- 172.21.15.47:0/1985591043 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f35081000c0 con 0x7f35080ff350 2024-02-23T00:54:10.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.509+0000 7f3505ffb700 10 client.41984.objecter ms_dispatch 0x7f3508001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:54:10.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.509+0000 7f3505ffb700 10 client.41984.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:54:10.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.509+0000 7f3505ffb700 20 client.41984.objecter put_session s=0x7f350805a040 osd=1 4 2024-02-23T00:54:10.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.509+0000 7f3505ffb700 15 client.41984.objecter _session_command_op_remove 1 4 2024-02-23T00:54:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.509+0000 7f350eb9f700 10 client.41984.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.509+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_assign -1 5 2024-02-23T00:54:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.509+0000 7f350eb9f700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f350806b190 0x7f350806b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.509+0000 7f350eb9f700 20 client.41984.objecter _get_session s=0x7f350806cf90 osd=2 3 2024-02-23T00:54:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.509+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f350806cf90 osd=2 3 2024-02-23T00:54:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.509+0000 7f350eb9f700 20 client.41984.objecter _get_session s=0x7f350806cf90 osd=2 3 2024-02-23T00:54:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.509+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_remove -1 5 2024-02-23T00:54:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.509+0000 7f350eb9f700 20 client.41984.objecter get_session s=0x7f350806cf90 osd=2 3 2024-02-23T00:54:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.509+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_assign 2 5 2024-02-23T00:54:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.509+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f350806cf90 osd=2 4 2024-02-23T00:54:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.509+0000 7f350eb9f700 10 client.41984.objecter _send_command 5 2024-02-23T00:54:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.509+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f350806bcb0 con 0x7f350806b190 2024-02-23T00:54:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.510+0000 7f3507fff700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f350806b190 0x7f350806b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.510+0000 7f3505ffb700 10 client.41984.objecter ms_handle_connect 0x7f350806b190 2024-02-23T00:54:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.510+0000 7f3507fff700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f350806b190 0x7f350806b590 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:10.812 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:54:10.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.510+0000 7f3505ffb700 1 -- 172.21.15.47:0/1985591043 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f350806bcb0 con 0x7f350806b190 2024-02-23T00:54:10.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.510+0000 7f3505ffb700 10 client.41984.objecter ms_dispatch 0x7f3508001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:54:10.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.510+0000 7f3505ffb700 10 client.41984.objecter _finish_command 5 = system:0 2024-02-23T00:54:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.510+0000 7f3505ffb700 20 client.41984.objecter put_session s=0x7f350806cf90 osd=2 4 2024-02-23T00:54:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.510+0000 7f3505ffb700 15 client.41984.objecter _session_command_op_remove 2 5 2024-02-23T00:54:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.525+0000 7f350eb9f700 10 client.41984.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:54:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.525+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_assign -1 6 2024-02-23T00:54:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.525+0000 7f350eb9f700 20 client.41984.objecter _get_session s=0x7f350806cf90 osd=2 3 2024-02-23T00:54:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.525+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f350806cf90 osd=2 3 2024-02-23T00:54:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.525+0000 7f350eb9f700 20 client.41984.objecter _get_session s=0x7f350806cf90 osd=2 3 2024-02-23T00:54:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.525+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_remove -1 6 2024-02-23T00:54:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.525+0000 7f350eb9f700 20 client.41984.objecter get_session s=0x7f350806cf90 osd=2 3 2024-02-23T00:54:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.525+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_assign 2 6 2024-02-23T00:54:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.525+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f350806cf90 osd=2 4 2024-02-23T00:54:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.525+0000 7f350eb9f700 10 client.41984.objecter _send_command 6 2024-02-23T00:54:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.525+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f350806c5f0 con 0x7f350806b190 2024-02-23T00:54:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.526+0000 7f3505ffb700 1 -- 172.21.15.47:0/1985591043 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f34f404ffd0 con 0x7f350806b190 2024-02-23T00:54:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.526+0000 7f3505ffb700 10 client.41984.objecter ms_dispatch 0x7f3508001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:54:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.526+0000 7f3505ffb700 10 client.41984.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:54:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.526+0000 7f3505ffb700 20 client.41984.objecter put_session s=0x7f350806cf90 osd=2 4 2024-02-23T00:54:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.526+0000 7f3505ffb700 15 client.41984.objecter _session_command_op_remove 2 6 2024-02-23T00:54:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.526+0000 7f350eb9f700 10 client.41984.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.527+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_assign -1 7 2024-02-23T00:54:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.527+0000 7f350eb9f700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f350811b2a0 0x7f350811d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.527+0000 7f350eb9f700 20 client.41984.objecter _get_session s=0x7f350806cd00 osd=3 3 2024-02-23T00:54:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.527+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f350806cd00 osd=3 3 2024-02-23T00:54:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.527+0000 7f350eb9f700 20 client.41984.objecter _get_session s=0x7f350806cd00 osd=3 3 2024-02-23T00:54:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.527+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_remove -1 7 2024-02-23T00:54:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.527+0000 7f350eb9f700 20 client.41984.objecter get_session s=0x7f350806cd00 osd=3 3 2024-02-23T00:54:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.527+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_assign 3 7 2024-02-23T00:54:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.527+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f350806cd00 osd=3 4 2024-02-23T00:54:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.527+0000 7f350eb9f700 10 client.41984.objecter _send_command 7 2024-02-23T00:54:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.527+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3508120460 con 0x7f350811b2a0 2024-02-23T00:54:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.527+0000 7f350d13c700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f350811b2a0 0x7f350811d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.527+0000 7f3505ffb700 10 client.41984.objecter ms_handle_connect 0x7f350811b2a0 2024-02-23T00:54:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.527+0000 7f350d13c700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f350811b2a0 0x7f350811d690 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.527+0000 7f3505ffb700 1 -- 172.21.15.47:0/1985591043 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f3508120460 con 0x7f350811b2a0 2024-02-23T00:54:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.527+0000 7f3505ffb700 10 client.41984.objecter ms_dispatch 0x7f3508001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:54:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.527+0000 7f3505ffb700 10 client.41984.objecter _finish_command 7 = system:0 2024-02-23T00:54:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.527+0000 7f3505ffb700 20 client.41984.objecter put_session s=0x7f350806cd00 osd=3 4 2024-02-23T00:54:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.527+0000 7f3505ffb700 15 client.41984.objecter _session_command_op_remove 3 7 2024-02-23T00:54:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.542+0000 7f350eb9f700 10 client.41984.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:54:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.542+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_assign -1 8 2024-02-23T00:54:10.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.542+0000 7f350eb9f700 20 client.41984.objecter _get_session s=0x7f350806cd00 osd=3 3 2024-02-23T00:54:10.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.542+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f350806cd00 osd=3 3 2024-02-23T00:54:10.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.542+0000 7f350eb9f700 20 client.41984.objecter _get_session s=0x7f350806cd00 osd=3 3 2024-02-23T00:54:10.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.542+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_remove -1 8 2024-02-23T00:54:10.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.542+0000 7f350eb9f700 20 client.41984.objecter get_session s=0x7f350806cd00 osd=3 3 2024-02-23T00:54:10.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.542+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_assign 3 8 2024-02-23T00:54:10.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.542+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f350806cd00 osd=3 4 2024-02-23T00:54:10.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.542+0000 7f350eb9f700 10 client.41984.objecter _send_command 8 2024-02-23T00:54:10.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.542+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f350806c060 con 0x7f350811b2a0 2024-02-23T00:54:10.817 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:54:10.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.543+0000 7f3505ffb700 1 -- 172.21.15.47:0/1985591043 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f350806c060 con 0x7f350811b2a0 2024-02-23T00:54:10.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.543+0000 7f3505ffb700 10 client.41984.objecter ms_dispatch 0x7f3508001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:54:10.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.543+0000 7f3505ffb700 10 client.41984.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:54:10.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.543+0000 7f3505ffb700 20 client.41984.objecter put_session s=0x7f350806cd00 osd=3 4 2024-02-23T00:54:10.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.543+0000 7f3505ffb700 15 client.41984.objecter _session_command_op_remove 3 8 2024-02-23T00:54:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.544+0000 7f350eb9f700 10 client.41984.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.544+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_assign -1 9 2024-02-23T00:54:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.544+0000 7f350eb9f700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f35080280a0 0x7f350811dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.544+0000 7f350eb9f700 20 client.41984.objecter _get_session s=0x7f350806c060 osd=4 3 2024-02-23T00:54:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.544+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f350806c060 osd=4 3 2024-02-23T00:54:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.544+0000 7f350eb9f700 20 client.41984.objecter _get_session s=0x7f350806c060 osd=4 3 2024-02-23T00:54:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.544+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_remove -1 9 2024-02-23T00:54:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.544+0000 7f350eb9f700 20 client.41984.objecter get_session s=0x7f350806c060 osd=4 3 2024-02-23T00:54:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.544+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_assign 4 9 2024-02-23T00:54:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.544+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f350806c060 osd=4 4 2024-02-23T00:54:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.544+0000 7f350eb9f700 10 client.41984.objecter _send_command 9 2024-02-23T00:54:10.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.544+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f350811e110 con 0x7f35080280a0 2024-02-23T00:54:10.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.544+0000 7f350c93b700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f35080280a0 0x7f350811dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:10.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.544+0000 7f350c93b700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f35080280a0 0x7f350811dc20 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:10.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.544+0000 7f3505ffb700 10 client.41984.objecter ms_handle_connect 0x7f35080280a0 2024-02-23T00:54:10.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.545+0000 7f3505ffb700 1 -- 172.21.15.47:0/1985591043 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f350811e110 con 0x7f35080280a0 2024-02-23T00:54:10.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.545+0000 7f3505ffb700 10 client.41984.objecter ms_dispatch 0x7f3508001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:54:10.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.545+0000 7f3505ffb700 10 client.41984.objecter _finish_command 9 = system:0 2024-02-23T00:54:10.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.545+0000 7f3505ffb700 20 client.41984.objecter put_session s=0x7f350806c060 osd=4 4 2024-02-23T00:54:10.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.545+0000 7f3505ffb700 15 client.41984.objecter _session_command_op_remove 4 9 2024-02-23T00:54:10.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.561+0000 7f350eb9f700 10 client.41984.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:54:10.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.561+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_assign -1 10 2024-02-23T00:54:10.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.561+0000 7f350eb9f700 20 client.41984.objecter _get_session s=0x7f350806c060 osd=4 3 2024-02-23T00:54:10.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.561+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f350806c060 osd=4 3 2024-02-23T00:54:10.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.561+0000 7f350eb9f700 20 client.41984.objecter _get_session s=0x7f350806c060 osd=4 3 2024-02-23T00:54:10.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.561+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_remove -1 10 2024-02-23T00:54:10.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.561+0000 7f350eb9f700 20 client.41984.objecter get_session s=0x7f350806c060 osd=4 3 2024-02-23T00:54:10.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.561+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_assign 4 10 2024-02-23T00:54:10.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.561+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f350806c060 osd=4 4 2024-02-23T00:54:10.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.561+0000 7f350eb9f700 10 client.41984.objecter _send_command 10 2024-02-23T00:54:10.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.561+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f350806c450 con 0x7f35080280a0 2024-02-23T00:54:10.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.563+0000 7f3505ffb700 1 -- 172.21.15.47:0/1985591043 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f350806c450 con 0x7f35080280a0 2024-02-23T00:54:10.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.563+0000 7f3505ffb700 10 client.41984.objecter ms_dispatch 0x7f3508001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:54:10.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.563+0000 7f3505ffb700 10 client.41984.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:54:10.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.563+0000 7f3505ffb700 20 client.41984.objecter put_session s=0x7f350806c060 osd=4 4 2024-02-23T00:54:10.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.563+0000 7f3505ffb700 15 client.41984.objecter _session_command_op_remove 4 10 2024-02-23T00:54:10.821 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:54:10.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.563+0000 7f350eb9f700 10 client.41984.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:10.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.563+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_assign -1 11 2024-02-23T00:54:10.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.563+0000 7f350eb9f700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f350811f7f0 0x7f350811fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:10.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.563+0000 7f350eb9f700 20 client.41984.objecter _get_session s=0x7f350806c450 osd=5 3 2024-02-23T00:54:10.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.563+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f350806c450 osd=5 3 2024-02-23T00:54:10.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.563+0000 7f350eb9f700 20 client.41984.objecter _get_session s=0x7f350806c450 osd=5 3 2024-02-23T00:54:10.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.563+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_remove -1 11 2024-02-23T00:54:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.563+0000 7f350eb9f700 20 client.41984.objecter get_session s=0x7f350806c450 osd=5 3 2024-02-23T00:54:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.563+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_assign 5 11 2024-02-23T00:54:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.563+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f350806c450 osd=5 4 2024-02-23T00:54:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.563+0000 7f350eb9f700 10 client.41984.objecter _send_command 11 2024-02-23T00:54:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.563+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3508126d50 con 0x7f350811f7f0 2024-02-23T00:54:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.563+0000 7f3507fff700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f350811f7f0 0x7f350811fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.563+0000 7f3507fff700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f350811f7f0 0x7f350811fc70 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.563+0000 7f3505ffb700 10 client.41984.objecter ms_handle_connect 0x7f350811f7f0 2024-02-23T00:54:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.564+0000 7f3505ffb700 1 -- 172.21.15.47:0/1985591043 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f3508126d50 con 0x7f350811f7f0 2024-02-23T00:54:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.564+0000 7f3505ffb700 10 client.41984.objecter ms_dispatch 0x7f3508001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:54:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.564+0000 7f3505ffb700 10 client.41984.objecter _finish_command 11 = system:0 2024-02-23T00:54:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.564+0000 7f3505ffb700 20 client.41984.objecter put_session s=0x7f350806c450 osd=5 4 2024-02-23T00:54:10.823 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:54:10.823 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:54:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.564+0000 7f3505ffb700 15 client.41984.objecter _session_command_op_remove 5 11 2024-02-23T00:54:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.579+0000 7f350eb9f700 10 client.41984.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:54:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.579+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_assign -1 12 2024-02-23T00:54:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.579+0000 7f350eb9f700 20 client.41984.objecter _get_session s=0x7f350806c450 osd=5 3 2024-02-23T00:54:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.579+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f350806c450 osd=5 3 2024-02-23T00:54:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.579+0000 7f350eb9f700 20 client.41984.objecter _get_session s=0x7f350806c450 osd=5 3 2024-02-23T00:54:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.579+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_remove -1 12 2024-02-23T00:54:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.579+0000 7f350eb9f700 20 client.41984.objecter get_session s=0x7f350806c450 osd=5 3 2024-02-23T00:54:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.579+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_assign 5 12 2024-02-23T00:54:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.579+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f350806c450 osd=5 4 2024-02-23T00:54:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.579+0000 7f350eb9f700 10 client.41984.objecter _send_command 12 2024-02-23T00:54:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.579+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f35081245f0 con 0x7f350811f7f0 2024-02-23T00:54:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.581+0000 7f3505ffb700 1 -- 172.21.15.47:0/1985591043 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f35081245f0 con 0x7f350811f7f0 2024-02-23T00:54:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.581+0000 7f3505ffb700 10 client.41984.objecter ms_dispatch 0x7f3508001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:54:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.581+0000 7f3505ffb700 10 client.41984.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:54:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.581+0000 7f3505ffb700 20 client.41984.objecter put_session s=0x7f350806c450 osd=5 4 2024-02-23T00:54:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.581+0000 7f3505ffb700 15 client.41984.objecter _session_command_op_remove 5 12 2024-02-23T00:54:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.581+0000 7f350eb9f700 10 client.41984.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.581+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_assign -1 13 2024-02-23T00:54:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.581+0000 7f350eb9f700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f3508124cc0 0x7f35081270f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.581+0000 7f350eb9f700 20 client.41984.objecter _get_session s=0x7f35081245f0 osd=6 3 2024-02-23T00:54:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.581+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f35081245f0 osd=6 3 2024-02-23T00:54:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.581+0000 7f350eb9f700 20 client.41984.objecter _get_session s=0x7f35081245f0 osd=6 3 2024-02-23T00:54:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.581+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_remove -1 13 2024-02-23T00:54:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.581+0000 7f350eb9f700 20 client.41984.objecter get_session s=0x7f35081245f0 osd=6 3 2024-02-23T00:54:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.581+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_assign 6 13 2024-02-23T00:54:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.581+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f35081245f0 osd=6 4 2024-02-23T00:54:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.581+0000 7f350eb9f700 10 client.41984.objecter _send_command 13 2024-02-23T00:54:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.581+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3508129f30 con 0x7f3508124cc0 2024-02-23T00:54:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.581+0000 7f350d13c700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f3508124cc0 0x7f35081270f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.582+0000 7f350d13c700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f3508124cc0 0x7f35081270f0 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.582+0000 7f3505ffb700 10 client.41984.objecter ms_handle_connect 0x7f3508124cc0 2024-02-23T00:54:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.582+0000 7f3505ffb700 1 -- 172.21.15.47:0/1985591043 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f3508129f30 con 0x7f3508124cc0 2024-02-23T00:54:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.582+0000 7f3505ffb700 10 client.41984.objecter ms_dispatch 0x7f3508001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:54:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.582+0000 7f3505ffb700 10 client.41984.objecter _finish_command 13 = system:0 2024-02-23T00:54:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.582+0000 7f3505ffb700 20 client.41984.objecter put_session s=0x7f35081245f0 osd=6 4 2024-02-23T00:54:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.582+0000 7f3505ffb700 15 client.41984.objecter _session_command_op_remove 6 13 2024-02-23T00:54:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.597+0000 7f350eb9f700 10 client.41984.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:54:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.597+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_assign -1 14 2024-02-23T00:54:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.597+0000 7f350eb9f700 20 client.41984.objecter _get_session s=0x7f35081245f0 osd=6 3 2024-02-23T00:54:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.597+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f35081245f0 osd=6 3 2024-02-23T00:54:10.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.597+0000 7f350eb9f700 20 client.41984.objecter _get_session s=0x7f35081245f0 osd=6 3 2024-02-23T00:54:10.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.597+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_remove -1 14 2024-02-23T00:54:10.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.597+0000 7f350eb9f700 20 client.41984.objecter get_session s=0x7f35081245f0 osd=6 3 2024-02-23T00:54:10.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.597+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_assign 6 14 2024-02-23T00:54:10.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.597+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f35081245f0 osd=6 4 2024-02-23T00:54:10.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.597+0000 7f350eb9f700 10 client.41984.objecter _send_command 14 2024-02-23T00:54:10.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.597+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f3508128d00 con 0x7f3508124cc0 2024-02-23T00:54:10.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.598+0000 7f3505ffb700 1 -- 172.21.15.47:0/1985591043 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f3500023020 con 0x7f3508124cc0 2024-02-23T00:54:10.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.598+0000 7f3505ffb700 10 client.41984.objecter ms_dispatch 0x7f3508001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:54:10.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.598+0000 7f3505ffb700 10 client.41984.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:54:10.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.598+0000 7f3505ffb700 20 client.41984.objecter put_session s=0x7f35081245f0 osd=6 4 2024-02-23T00:54:10.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.598+0000 7f3505ffb700 15 client.41984.objecter _session_command_op_remove 6 14 2024-02-23T00:54:10.828 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:54:10.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.599+0000 7f350eb9f700 10 client.41984.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:10.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.599+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_assign -1 15 2024-02-23T00:54:10.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.599+0000 7f350eb9f700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f3508129540 0x7f35081276f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:10.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.599+0000 7f350eb9f700 20 client.41984.objecter _get_session s=0x7f3508128e70 osd=7 3 2024-02-23T00:54:10.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.599+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f3508128e70 osd=7 3 2024-02-23T00:54:10.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.599+0000 7f350eb9f700 20 client.41984.objecter _get_session s=0x7f3508128e70 osd=7 3 2024-02-23T00:54:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.599+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_remove -1 15 2024-02-23T00:54:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.599+0000 7f350eb9f700 20 client.41984.objecter get_session s=0x7f3508128e70 osd=7 3 2024-02-23T00:54:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.599+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_assign 7 15 2024-02-23T00:54:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.599+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f3508128e70 osd=7 4 2024-02-23T00:54:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.599+0000 7f350eb9f700 10 client.41984.objecter _send_command 15 2024-02-23T00:54:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.599+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3508129bb0 con 0x7f3508129540 2024-02-23T00:54:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.599+0000 7f350c93b700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f3508129540 0x7f35081276f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.599+0000 7f3505ffb700 10 client.41984.objecter ms_handle_connect 0x7f3508129540 2024-02-23T00:54:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.599+0000 7f350c93b700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f3508129540 0x7f35081276f0 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.600+0000 7f3505ffb700 1 -- 172.21.15.47:0/1985591043 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f3508129bb0 con 0x7f3508129540 2024-02-23T00:54:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.600+0000 7f3505ffb700 10 client.41984.objecter ms_dispatch 0x7f3508001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:54:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.600+0000 7f3505ffb700 10 client.41984.objecter _finish_command 15 = system:0 2024-02-23T00:54:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.600+0000 7f3505ffb700 20 client.41984.objecter put_session s=0x7f3508128e70 osd=7 4 2024-02-23T00:54:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.600+0000 7f3505ffb700 15 client.41984.objecter _session_command_op_remove 7 15 2024-02-23T00:54:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.614+0000 7f350eb9f700 10 client.41984.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:54:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.614+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_assign -1 16 2024-02-23T00:54:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.614+0000 7f350eb9f700 20 client.41984.objecter _get_session s=0x7f3508128e70 osd=7 3 2024-02-23T00:54:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.614+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f3508128e70 osd=7 3 2024-02-23T00:54:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.614+0000 7f350eb9f700 20 client.41984.objecter _get_session s=0x7f3508128e70 osd=7 3 2024-02-23T00:54:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.614+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_remove -1 16 2024-02-23T00:54:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.614+0000 7f350eb9f700 20 client.41984.objecter get_session s=0x7f3508128e70 osd=7 3 2024-02-23T00:54:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.614+0000 7f350eb9f700 15 client.41984.objecter _session_command_op_assign 7 16 2024-02-23T00:54:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.614+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f3508128e70 osd=7 4 2024-02-23T00:54:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.614+0000 7f350eb9f700 10 client.41984.objecter _send_command 16 2024-02-23T00:54:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.614+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f3508129bb0 con 0x7f3508129540 2024-02-23T00:54:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f3505ffb700 1 -- 172.21.15.47:0/1985591043 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f3508129bb0 con 0x7f3508129540 2024-02-23T00:54:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f3505ffb700 10 client.41984.objecter ms_dispatch 0x7f3508001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:54:10.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f3505ffb700 10 client.41984.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:54:10.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f3505ffb700 20 client.41984.objecter put_session s=0x7f3508128e70 osd=7 4 2024-02-23T00:54:10.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f3505ffb700 15 client.41984.objecter _session_command_op_remove 7 16 2024-02-23T00:54:10.831 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:54:10.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 10 librados: watch_flush enter 2024-02-23T00:54:10.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 10 librados: watch_flush exit 2024-02-23T00:54:10.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 10 client.41984.objecter close_session for osd.0 2024-02-23T00:54:10.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f3508028540 msgr2=0x7f35080fe950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:10.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f3508028540 0x7f35080fe950 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:10.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f3508108ff0 osd=0 1 2024-02-23T00:54:10.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 10 client.41984.objecter close_session for osd.1 2024-02-23T00:54:10.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f35080ff350 msgr2=0x7f35080ff730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:10.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f35080ff350 0x7f35080ff730 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:10.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f350805a040 osd=1 1 2024-02-23T00:54:10.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 10 client.41984.objecter close_session for osd.2 2024-02-23T00:54:10.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f350806b190 msgr2=0x7f350806b590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:10.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f350806b190 0x7f350806b590 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:10.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f350806cf90 osd=2 1 2024-02-23T00:54:10.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 10 client.41984.objecter close_session for osd.3 2024-02-23T00:54:10.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f350811b2a0 msgr2=0x7f350811d690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:10.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350d13c700 1 -- 172.21.15.47:0/1985591043 reap_dead start 2024-02-23T00:54:10.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f350811b2a0 0x7f350811d690 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:10.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f350806cd00 osd=3 1 2024-02-23T00:54:10.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 10 client.41984.objecter close_session for osd.4 2024-02-23T00:54:10.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f35080280a0 msgr2=0x7f350811dc20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f35080280a0 0x7f350811dc20 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f350806c060 osd=4 1 2024-02-23T00:54:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 10 client.41984.objecter close_session for osd.5 2024-02-23T00:54:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f350811f7f0 msgr2=0x7f350811fc70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f350811f7f0 0x7f350811fc70 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f350806c450 osd=5 1 2024-02-23T00:54:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 10 client.41984.objecter close_session for osd.6 2024-02-23T00:54:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f3508124cc0 msgr2=0x7f35081270f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f3508124cc0 0x7f35081270f0 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f35081245f0 osd=6 1 2024-02-23T00:54:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 10 client.41984.objecter close_session for osd.7 2024-02-23T00:54:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f3508129540 msgr2=0x7f35081276f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f3508129540 0x7f35081276f0 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:10.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 20 client.41984.objecter put_session s=0x7f3508128e70 osd=7 1 2024-02-23T00:54:10.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 20 client.41984.objecter shutdown clearing up homeless session... 2024-02-23T00:54:10.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350eb9f700 10 client.41984.objecter successfully canceled tick 2024-02-23T00:54:10.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.616+0000 7f350d13c700 1 -- 172.21.15.47:0/1985591043 reap_dead start 2024-02-23T00:54:10.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.617+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f34f8062cc0 msgr2=0x7f34f8065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:10.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.617+0000 7f350eb9f700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f34f8062cc0 0x7f34f8065180 secure :-1 s=READY pgs=3174 cs=0 l=1 rev1=1 rx=0x7f34fc004ee0 tx=0x7f34fc00b040).stop 2024-02-23T00:54:10.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.617+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35080fdbb0 msgr2=0x7f350810e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:10.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.617+0000 7f350eb9f700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35080fdbb0 0x7f350810e420 secure :-1 s=READY pgs=2785 cs=0 l=1 rev1=1 rx=0x7f34f40048f0 tx=0x7f34f4009cb0).stop 2024-02-23T00:54:10.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.617+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 shutdown_connections 2024-02-23T00:54:10.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.617+0000 7f350eb9f700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35080fdbb0 0x7f350810e420 unknown :-1 s=CLOSED pgs=2785 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:10.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.617+0000 7f350eb9f700 1 --2- 172.21.15.47:0/1985591043 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f34f8062cc0 0x7f34f8065180 unknown :-1 s=CLOSED pgs=3174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:10.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.617+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 >> 172.21.15.47:0/1985591043 conn(0x7f35080f3800 msgr2=0x7f35080fb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:10.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.617+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 shutdown_connections 2024-02-23T00:54:10.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.617+0000 7f350eb9f700 1 -- 172.21.15.47:0/1985591043 wait complete. 2024-02-23T00:54:10.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:10.617+0000 7f350eb9f700 1 librados: shutdown 2024-02-23T00:54:10.836 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:54:10.836 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:54:10.836 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:54:10.836 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:54:10.836 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:54:10.836 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:54:10.836 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:54:10.836 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:54:10.865 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:10.988 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:54:10.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:09.381569+0000 mgr.y (mgr.14517) 610 : cluster [DBG] pgmap v1074: 152 pgs: 1 active+clean+snaptrim, 1 active+remapped+backfill_toofull, 1 active+clean+scrubbing, 149 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s; 1/720 objects misplaced (0.139%) 2024-02-23T00:54:10.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:09.496507+0000 osd.6 (osd.6) 275 : cluster [DBG] 2.11 scrub starts 2024-02-23T00:54:10.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:09.497693+0000 osd.6 (osd.6) 276 : cluster [DBG] 2.11 scrub ok 2024-02-23T00:54:10.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:10.303110+0000 mon.a (mon.0) 3473 : cluster [DBG] osdmap e711: 8 total, 8 up, 6 in 2024-02-23T00:54:10.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:10.304521+0000 mon.c (mon.2) 418 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:54:10.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:10.305953+0000 mon.a (mon.0) 3474 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:54:10.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:10.475534+0000 mon.c (mon.2) 419 : audit [DBG] from='client.? 172.21.15.47:0/1985591043' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:54:10.992 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:09.381569+0000 mgr.y (mgr.14517) 610 : cluster [DBG] pgmap v1074: 152 pgs: 1 active+clean+snaptrim, 1 active+remapped+backfill_toofull, 1 active+clean+scrubbing, 149 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s; 1/720 objects misplaced (0.139%) 2024-02-23T00:54:10.992 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:09.496507+0000 osd.6 (osd.6) 275 : cluster [DBG] 2.11 scrub starts 2024-02-23T00:54:10.992 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:09.497693+0000 osd.6 (osd.6) 276 : cluster [DBG] 2.11 scrub ok 2024-02-23T00:54:10.992 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:10.303110+0000 mon.a (mon.0) 3473 : cluster [DBG] osdmap e711: 8 total, 8 up, 6 in 2024-02-23T00:54:10.992 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:10.304521+0000 mon.c (mon.2) 418 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:54:10.992 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:10.305953+0000 mon.a (mon.0) 3474 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:54:10.992 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:10.475534+0000 mon.c (mon.2) 419 : audit [DBG] from='client.? 172.21.15.47:0/1985591043' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:54:10.992 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:10.993 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:54:10.997 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:54:11.094 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:54:11.122 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:11.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:09.381569+0000 mgr.y (mgr.14517) 610 : cluster [DBG] pgmap v1074: 152 pgs: 1 active+clean+snaptrim, 1 active+remapped+backfill_toofull, 1 active+clean+scrubbing, 149 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s; 1/720 objects misplaced (0.139%) 2024-02-23T00:54:11.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:09.496507+0000 osd.6 (osd.6) 275 : cluster [DBG] 2.11 scrub starts 2024-02-23T00:54:11.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:09.497693+0000 osd.6 (osd.6) 276 : cluster [DBG] 2.11 scrub ok 2024-02-23T00:54:11.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:10.303110+0000 mon.a (mon.0) 3473 : cluster [DBG] osdmap e711: 8 total, 8 up, 6 in 2024-02-23T00:54:11.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:10.304521+0000 mon.c (mon.2) 418 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:54:11.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:10.305953+0000 mon.a (mon.0) 3474 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:54:11.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:10.475534+0000 mon.c (mon.2) 419 : audit [DBG] from='client.? 172.21.15.47:0/1985591043' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:54:11.539 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:54:11.777 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:10.265985+0000 osd.7 (osd.7) 329 : cluster [DBG] 234.7 scrub starts 2024-02-23T00:54:11.777 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:10.267936+0000 osd.7 (osd.7) 330 : cluster [DBG] 234.7 scrub ok 2024-02-23T00:54:11.777 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:10.843933+0000 osd.1 (osd.1) 289 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:54:11.777 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:10.844088+0000 osd.1 (osd.1) 290 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:54:11.777 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:11.303730+0000 mon.a (mon.0) 3475 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:54:11.778 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:11.303844+0000 mon.a (mon.0) 3476 : cluster [DBG] osdmap e712: 8 total, 8 up, 6 in 2024-02-23T00:54:11.778 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:11.306328+0000 mon.c (mon.2) 420 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:54:11.778 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:11.306759+0000 mon.a (mon.0) 3477 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:54:11.778 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:11.313717+0000 mon.a (mon.0) 3478 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:54:11.778 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:11.381889+0000 mgr.y (mgr.14517) 611 : cluster [DBG] pgmap v1077: 184 pgs: 32 unknown, 1 active+clean+snaptrim, 1 active+remapped+backfill_toofull, 1 active+clean+scrubbing, 149 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1/720 objects misplaced (0.139%) 2024-02-23T00:54:11.778 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:10.265985+0000 osd.7 (osd.7) 329 : cluster [DBG] 234.7 scrub starts 2024-02-23T00:54:11.778 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:10.267936+0000 osd.7 (osd.7) 330 : cluster [DBG] 234.7 scrub ok 2024-02-23T00:54:11.778 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:10.843933+0000 osd.1 (osd.1) 289 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:54:11.778 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:10.844088+0000 osd.1 (osd.1) 290 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:54:11.778 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:11.303730+0000 mon.a (mon.0) 3475 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:54:11.779 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:11.303844+0000 mon.a (mon.0) 3476 : cluster [DBG] osdmap e712: 8 total, 8 up, 6 in 2024-02-23T00:54:11.779 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:11.306328+0000 mon.c (mon.2) 420 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:54:11.779 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:11.306759+0000 mon.a (mon.0) 3477 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:54:11.779 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:11.313717+0000 mon.a (mon.0) 3478 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:54:11.779 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:11.381889+0000 mgr.y (mgr.14517) 611 : cluster [DBG] pgmap v1077: 184 pgs: 32 unknown, 1 active+clean+snaptrim, 1 active+remapped+backfill_toofull, 1 active+clean+scrubbing, 149 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1/720 objects misplaced (0.139%) 2024-02-23T00:54:12.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:10.265985+0000 osd.7 (osd.7) 329 : cluster [DBG] 234.7 scrub starts 2024-02-23T00:54:12.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:10.267936+0000 osd.7 (osd.7) 330 : cluster [DBG] 234.7 scrub ok 2024-02-23T00:54:12.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:10.843933+0000 osd.1 (osd.1) 289 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:54:12.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:10.844088+0000 osd.1 (osd.1) 290 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:54:12.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:11.303730+0000 mon.a (mon.0) 3475 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:54:12.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:11.303844+0000 mon.a (mon.0) 3476 : cluster [DBG] osdmap e712: 8 total, 8 up, 6 in 2024-02-23T00:54:12.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:11.306328+0000 mon.c (mon.2) 420 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:54:12.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:11.306759+0000 mon.a (mon.0) 3477 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:54:12.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:11.313717+0000 mon.a (mon.0) 3478 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:54:12.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:11.381889+0000 mgr.y (mgr.14517) 611 : cluster [DBG] pgmap v1077: 184 pgs: 32 unknown, 1 active+clean+snaptrim, 1 active+remapped+backfill_toofull, 1 active+clean+scrubbing, 149 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1/720 objects misplaced (0.139%) 2024-02-23T00:54:12.795 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:54:12.909 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:54:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:54:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:54:12.910 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:11.257021+0000 osd.7 (osd.7) 331 : cluster [DBG] 5.9 scrub starts 2024-02-23T00:54:12.910 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:11.260483+0000 osd.7 (osd.7) 332 : cluster [DBG] 5.9 scrub ok 2024-02-23T00:54:12.910 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:11.402416+0000 osd.4 (osd.4) 299 : cluster [DBG] 4.1b scrub starts 2024-02-23T00:54:12.910 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:11.404589+0000 osd.4 (osd.4) 300 : cluster [DBG] 4.1b scrub ok 2024-02-23T00:54:12.910 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:11.500620+0000 osd.6 (osd.6) 277 : cluster [DBG] 4.1f scrub starts 2024-02-23T00:54:12.910 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:11.502508+0000 osd.6 (osd.6) 278 : cluster [DBG] 4.1f scrub ok 2024-02-23T00:54:12.910 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:12.304259+0000 mon.a (mon.0) 3479 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:54:12.910 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:12.304350+0000 mon.a (mon.0) 3480 : cluster [DBG] osdmap e713: 8 total, 8 up, 6 in 2024-02-23T00:54:12.910 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:12.306630+0000 mon.c (mon.2) 421 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-124"}]: dispatch 2024-02-23T00:54:12.910 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:12.307025+0000 mon.a (mon.0) 3481 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-124"}]: dispatch 2024-02-23T00:54:13.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:11.257021+0000 osd.7 (osd.7) 331 : cluster [DBG] 5.9 scrub starts 2024-02-23T00:54:13.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:11.260483+0000 osd.7 (osd.7) 332 : cluster [DBG] 5.9 scrub ok 2024-02-23T00:54:13.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:11.402416+0000 osd.4 (osd.4) 299 : cluster [DBG] 4.1b scrub starts 2024-02-23T00:54:13.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:11.404589+0000 osd.4 (osd.4) 300 : cluster [DBG] 4.1b scrub ok 2024-02-23T00:54:13.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:11.500620+0000 osd.6 (osd.6) 277 : cluster [DBG] 4.1f scrub starts 2024-02-23T00:54:13.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:11.502508+0000 osd.6 (osd.6) 278 : cluster [DBG] 4.1f scrub ok 2024-02-23T00:54:13.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:12.304259+0000 mon.a (mon.0) 3479 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:54:13.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:12.304350+0000 mon.a (mon.0) 3480 : cluster [DBG] osdmap e713: 8 total, 8 up, 6 in 2024-02-23T00:54:13.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:12.306630+0000 mon.c (mon.2) 421 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-124"}]: dispatch 2024-02-23T00:54:13.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:12.307025+0000 mon.a (mon.0) 3481 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-124"}]: dispatch 2024-02-23T00:54:13.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:11.257021+0000 osd.7 (osd.7) 331 : cluster [DBG] 5.9 scrub starts 2024-02-23T00:54:13.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:11.260483+0000 osd.7 (osd.7) 332 : cluster [DBG] 5.9 scrub ok 2024-02-23T00:54:13.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:11.402416+0000 osd.4 (osd.4) 299 : cluster [DBG] 4.1b scrub starts 2024-02-23T00:54:13.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:11.404589+0000 osd.4 (osd.4) 300 : cluster [DBG] 4.1b scrub ok 2024-02-23T00:54:13.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:11.500620+0000 osd.6 (osd.6) 277 : cluster [DBG] 4.1f scrub starts 2024-02-23T00:54:13.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:11.502508+0000 osd.6 (osd.6) 278 : cluster [DBG] 4.1f scrub ok 2024-02-23T00:54:13.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:12.304259+0000 mon.a (mon.0) 3479 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:54:13.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:12.304350+0000 mon.a (mon.0) 3480 : cluster [DBG] osdmap e713: 8 total, 8 up, 6 in 2024-02-23T00:54:13.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:12.306630+0000 mon.c (mon.2) 421 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-124"}]: dispatch 2024-02-23T00:54:13.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:12.307025+0000 mon.a (mon.0) 3481 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-124"}]: dispatch 2024-02-23T00:54:13.194 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:54:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:54:13.538 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:54:13.573 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:54:13.576 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:13.577 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:54:13.677 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:54:13.706 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:13.929 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:12.294923+0000 osd.7 (osd.7) 333 : cluster [DBG] 5.10 scrub starts 2024-02-23T00:54:13.929 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:12.297861+0000 osd.7 (osd.7) 334 : cluster [DBG] 5.10 scrub ok 2024-02-23T00:54:13.929 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:12.459266+0000 osd.6 (osd.6) 279 : cluster [DBG] 2.19 scrub starts 2024-02-23T00:54:13.929 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:12.460333+0000 osd.6 (osd.6) 280 : cluster [DBG] 2.19 scrub ok 2024-02-23T00:54:13.929 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:12.907651+0000 mgr.y (mgr.14517) 612 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:54:13.929 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:13.306245+0000 mon.a (mon.0) 3482 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-124"}]': finished 2024-02-23T00:54:13.929 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:13.306328+0000 mon.a (mon.0) 3483 : cluster [DBG] osdmap e714: 8 total, 8 up, 6 in 2024-02-23T00:54:13.929 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:13.307186+0000 mon.c (mon.2) 422 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-124", "mode": "writeback"}]: dispatch 2024-02-23T00:54:13.930 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:13.307558+0000 mon.a (mon.0) 3484 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-124", "mode": "writeback"}]: dispatch 2024-02-23T00:54:13.930 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:13.382311+0000 mgr.y (mgr.14517) 613 : cluster [DBG] pgmap v1080: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/720 objects misplaced (0.139%) 2024-02-23T00:54:13.930 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:12.294923+0000 osd.7 (osd.7) 333 : cluster [DBG] 5.10 scrub starts 2024-02-23T00:54:13.930 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:12.297861+0000 osd.7 (osd.7) 334 : cluster [DBG] 5.10 scrub ok 2024-02-23T00:54:13.930 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:12.459266+0000 osd.6 (osd.6) 279 : cluster [DBG] 2.19 scrub starts 2024-02-23T00:54:13.930 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:12.460333+0000 osd.6 (osd.6) 280 : cluster [DBG] 2.19 scrub ok 2024-02-23T00:54:13.930 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:12.907651+0000 mgr.y (mgr.14517) 612 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:54:13.930 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:13.306245+0000 mon.a (mon.0) 3482 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-124"}]': finished 2024-02-23T00:54:13.930 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:13.306328+0000 mon.a (mon.0) 3483 : cluster [DBG] osdmap e714: 8 total, 8 up, 6 in 2024-02-23T00:54:13.930 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:13.307186+0000 mon.c (mon.2) 422 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-124", "mode": "writeback"}]: dispatch 2024-02-23T00:54:13.930 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:13.307558+0000 mon.a (mon.0) 3484 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-124", "mode": "writeback"}]: dispatch 2024-02-23T00:54:13.931 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:13.382311+0000 mgr.y (mgr.14517) 613 : cluster [DBG] pgmap v1080: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/720 objects misplaced (0.139%) 2024-02-23T00:54:14.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:12.294923+0000 osd.7 (osd.7) 333 : cluster [DBG] 5.10 scrub starts 2024-02-23T00:54:14.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:12.297861+0000 osd.7 (osd.7) 334 : cluster [DBG] 5.10 scrub ok 2024-02-23T00:54:14.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:12.459266+0000 osd.6 (osd.6) 279 : cluster [DBG] 2.19 scrub starts 2024-02-23T00:54:14.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:12.460333+0000 osd.6 (osd.6) 280 : cluster [DBG] 2.19 scrub ok 2024-02-23T00:54:14.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:12.907651+0000 mgr.y (mgr.14517) 612 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:54:14.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:13.306245+0000 mon.a (mon.0) 3482 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-124"}]': finished 2024-02-23T00:54:14.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:13.306328+0000 mon.a (mon.0) 3483 : cluster [DBG] osdmap e714: 8 total, 8 up, 6 in 2024-02-23T00:54:14.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:13.307186+0000 mon.c (mon.2) 422 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-124", "mode": "writeback"}]: dispatch 2024-02-23T00:54:14.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:13.307558+0000 mon.a (mon.0) 3484 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-124", "mode": "writeback"}]: dispatch 2024-02-23T00:54:14.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:13.382311+0000 mgr.y (mgr.14517) 613 : cluster [DBG] pgmap v1080: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/720 objects misplaced (0.139%) 2024-02-23T00:54:14.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.681+0000 7f6d2f76d700 1 -- 172.21.15.47:0/4099625520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d280f8420 msgr2=0x7f6d28104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:14.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.681+0000 7f6d2f76d700 1 --2- 172.21.15.47:0/4099625520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d280f8420 0x7f6d28104a20 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 rx=0x7f6d20009be0 tx=0x7f6d20004d10).stop 2024-02-23T00:54:14.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.681+0000 7f6d2f76d700 1 -- 172.21.15.47:0/4099625520 shutdown_connections 2024-02-23T00:54:14.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.681+0000 7f6d2f76d700 1 --2- 172.21.15.47:0/4099625520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d280f8420 0x7f6d28104a20 unknown :-1 s=CLOSED pgs=2624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:14.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.681+0000 7f6d2f76d700 1 --2- 172.21.15.47:0/4099625520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d280fdbb0 0x7f6d280fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:14.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.681+0000 7f6d2f76d700 1 --2- 172.21.15.47:0/4099625520 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6d280fe510 0x7f6d280f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:14.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.681+0000 7f6d2f76d700 1 -- 172.21.15.47:0/4099625520 >> 172.21.15.47:0/4099625520 conn(0x7f6d280f3800 msgr2=0x7f6d280f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:14.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.681+0000 7f6d2f76d700 1 -- 172.21.15.47:0/4099625520 shutdown_connections 2024-02-23T00:54:14.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.681+0000 7f6d2f76d700 1 -- 172.21.15.47:0/4099625520 wait complete. 2024-02-23T00:54:14.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.681+0000 7f6d2f76d700 1 librados: starting msgr at 2024-02-23T00:54:14.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.681+0000 7f6d2f76d700 1 librados: starting objecter 2024-02-23T00:54:14.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.682+0000 7f6d2f76d700 1 Processor -- start 2024-02-23T00:54:14.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.682+0000 7f6d2f76d700 1 -- start start 2024-02-23T00:54:14.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.682+0000 7f6d2f76d700 1 librados: setting wanted keys 2024-02-23T00:54:14.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.682+0000 7f6d2f76d700 1 librados: calling monclient init 2024-02-23T00:54:14.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.682+0000 7f6d2f76d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d280f8420 0x7f6d2810dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:14.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.682+0000 7f6d2f76d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6d280fdbb0 0x7f6d2810e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:14.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.682+0000 7f6d2f76d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d280fe510 0x7f6d28107e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:14.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.682+0000 7f6d2f76d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d28108340 con 0x7f6d280fe510 2024-02-23T00:54:14.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.682+0000 7f6d2f76d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d281084b0 con 0x7f6d280f8420 2024-02-23T00:54:14.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.682+0000 7f6d2f76d700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d28108670 con 0x7f6d280fdbb0 2024-02-23T00:54:14.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.682+0000 7f6d2dd0a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d280fe510 0x7f6d28107e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:14.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.682+0000 7f6d2dd0a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d280fe510 0x7f6d28107e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43792/0 (socket says 172.21.15.47:43792) 2024-02-23T00:54:14.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.682+0000 7f6d2cd08700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6d280fdbb0 0x7f6d2810e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:14.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.682+0000 7f6d2dd0a700 1 -- 172.21.15.47:0/1303355281 learned_addr learned my addr 172.21.15.47:0/1303355281 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:54:14.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.682+0000 7f6d2d509700 1 --2- 172.21.15.47:0/1303355281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d280f8420 0x7f6d2810dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:14.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.682+0000 7f6d2dd0a700 1 -- 172.21.15.47:0/1303355281 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6d280fdbb0 msgr2=0x7f6d2810e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:14.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.682+0000 7f6d2dd0a700 1 --2- 172.21.15.47:0/1303355281 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6d280fdbb0 0x7f6d2810e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:14.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.682+0000 7f6d2dd0a700 1 -- 172.21.15.47:0/1303355281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d280f8420 msgr2=0x7f6d2810dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:14.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.682+0000 7f6d2dd0a700 1 --2- 172.21.15.47:0/1303355281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d280f8420 0x7f6d2810dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:14.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.682+0000 7f6d2dd0a700 1 -- 172.21.15.47:0/1303355281 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d18005040 con 0x7f6d280fe510 2024-02-23T00:54:14.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.683+0000 7f6d2dd0a700 1 --2- 172.21.15.47:0/1303355281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d280fe510 0x7f6d28107e50 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 rx=0x7f6d2000bfc0 tx=0x7f6d20004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:14.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.683+0000 7f6d1e7fc700 10 client.?.objecter ms_handle_connect 0x7f6d280fe510 2024-02-23T00:54:14.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.683+0000 7f6d1e7fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:54:14.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.683+0000 7f6d1e7fc700 1 -- 172.21.15.47:0/1303355281 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2000e040 con 0x7f6d280fe510 2024-02-23T00:54:14.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.683+0000 7f6d1e7fc700 1 -- 172.21.15.47:0/1303355281 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f6d20018040 con 0x7f6d280fe510 2024-02-23T00:54:14.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.683+0000 7f6d2f76d700 1 -- 172.21.15.47:0/1303355281 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d200097c0 con 0x7f6d280fe510 2024-02-23T00:54:14.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.683+0000 7f6d1e7fc700 1 -- 172.21.15.47:0/1303355281 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d20013ea0 con 0x7f6d280fe510 2024-02-23T00:54:14.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.683+0000 7f6d2f76d700 10 client.31671.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:54:14.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.683+0000 7f6d2f76d700 1 -- 172.21.15.47:0/1303355281 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d28108b90 con 0x7f6d280fe510 2024-02-23T00:54:14.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.683+0000 7f6d2f76d700 1 librados: init done 2024-02-23T00:54:14.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.684+0000 7f6d2f76d700 1 -- 172.21.15.47:0/1303355281 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d2805ee80 con 0x7f6d280fe510 2024-02-23T00:54:14.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.684+0000 7f6d1e7fc700 1 -- 172.21.15.47:0/1303355281 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f6d20013420 con 0x7f6d280fe510 2024-02-23T00:54:14.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.684+0000 7f6d1e7fc700 1 --2- 172.21.15.47:0/1303355281 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6d14062cc0 0x7f6d14065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:14.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.685+0000 7f6d1e7fc700 1 -- 172.21.15.47:0/1303355281 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(715..715 src has 621..715) v4 ==== 9780+0+0 (secure 0 0 0) 0x7f6d2007e9e0 con 0x7f6d280fe510 2024-02-23T00:54:14.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.685+0000 7f6d1e7fc700 10 client.31671.objecter ms_dispatch 0x7f6d28001550 osd_map(715..715 src has 621..715) v4 2024-02-23T00:54:14.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.685+0000 7f6d1e7fc700 3 client.31671.objecter handle_osd_map got epochs [715,715] > 0 2024-02-23T00:54:14.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.685+0000 7f6d1e7fc700 3 client.31671.objecter handle_osd_map decoding full epoch 715 2024-02-23T00:54:14.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.685+0000 7f6d2d509700 1 --2- 172.21.15.47:0/1303355281 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6d14062cc0 0x7f6d14065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:14.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.685+0000 7f6d1e7fc700 20 client.31671.objecter dump_active .. 0 homeless 2024-02-23T00:54:14.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.685+0000 7f6d1e7fc700 10 client.31671.objecter ms_handle_connect 0x7f6d14062cc0 2024-02-23T00:54:14.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.685+0000 7f6d2d509700 1 --2- 172.21.15.47:0/1303355281 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6d14062cc0 0x7f6d14065180 secure :-1 s=READY pgs=3175 cs=0 l=1 rev1=1 rx=0x7f6d24004dc0 tx=0x7f6d2400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:14.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.689+0000 7f6d1e7fc700 1 -- 172.21.15.47:0/1303355281 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f6d2004de70 con 0x7f6d280fe510 2024-02-23T00:54:14.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:14.877+0000 7f6d2f76d700 1 -- 172.21.15.47:0/1303355281 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd reweight", "id": 6, "weight": 1.0} v 0) v1 -- 0x7f6d28109380 con 0x7f6d280fe510 2024-02-23T00:54:15.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:13.437965+0000 osd.6 (osd.6) 281 : cluster [DBG] 2.7 scrub starts 2024-02-23T00:54:15.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:13.439148+0000 osd.6 (osd.6) 282 : cluster [DBG] 2.7 scrub ok 2024-02-23T00:54:15.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:14.304018+0000 mon.a (mon.0) 3485 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:54:15.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:14.310669+0000 mon.a (mon.0) 3486 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-124", "mode": "writeback"}]': finished 2024-02-23T00:54:15.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:14.310740+0000 mon.a (mon.0) 3487 : cluster [DBG] osdmap e715: 8 total, 8 up, 6 in 2024-02-23T00:54:15.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:14.354719+0000 mon.c (mon.2) 423 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:15.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:14.355156+0000 mon.a (mon.0) 3488 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:15.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:13.437965+0000 osd.6 (osd.6) 281 : cluster [DBG] 2.7 scrub starts 2024-02-23T00:54:15.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:13.439148+0000 osd.6 (osd.6) 282 : cluster [DBG] 2.7 scrub ok 2024-02-23T00:54:15.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:14.304018+0000 mon.a (mon.0) 3485 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:54:15.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:14.310669+0000 mon.a (mon.0) 3486 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-124", "mode": "writeback"}]': finished 2024-02-23T00:54:15.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:14.310740+0000 mon.a (mon.0) 3487 : cluster [DBG] osdmap e715: 8 total, 8 up, 6 in 2024-02-23T00:54:15.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:14.354719+0000 mon.c (mon.2) 423 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:15.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:14.355156+0000 mon.a (mon.0) 3488 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:15.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:13.437965+0000 osd.6 (osd.6) 281 : cluster [DBG] 2.7 scrub starts 2024-02-23T00:54:15.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:13.439148+0000 osd.6 (osd.6) 282 : cluster [DBG] 2.7 scrub ok 2024-02-23T00:54:15.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:14.304018+0000 mon.a (mon.0) 3485 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:54:15.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:14.310669+0000 mon.a (mon.0) 3486 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-124", "mode": "writeback"}]': finished 2024-02-23T00:54:15.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:14.310740+0000 mon.a (mon.0) 3487 : cluster [DBG] osdmap e715: 8 total, 8 up, 6 in 2024-02-23T00:54:15.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:14.354719+0000 mon.c (mon.2) 423 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:15.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:14.355156+0000 mon.a (mon.0) 3488 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:15.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.293+0000 7f6d1e7fc700 1 -- 172.21.15.47:0/1303355281 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 6, "weight": 1.0}]=0 reweighted osd.6 to 1 (10000) v716) v1 ==== 113+0+0 (secure 0 0 0) 0x7f6d20021030 con 0x7f6d280fe510 2024-02-23T00:54:15.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.296+0000 7f6d2f76d700 10 librados: watch_flush enter 2024-02-23T00:54:15.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.296+0000 7f6d2f76d700 10 librados: watch_flush exit 2024-02-23T00:54:15.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.296+0000 7f6d2f76d700 20 client.31671.objecter shutdown clearing up homeless session... 2024-02-23T00:54:15.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.296+0000 7f6d2f76d700 10 client.31671.objecter successfully canceled tick 2024-02-23T00:54:15.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.297+0000 7f6d2f76d700 1 -- 172.21.15.47:0/1303355281 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6d14062cc0 msgr2=0x7f6d14065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:15.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.297+0000 7f6d2f76d700 1 --2- 172.21.15.47:0/1303355281 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6d14062cc0 0x7f6d14065180 secure :-1 s=READY pgs=3175 cs=0 l=1 rev1=1 rx=0x7f6d24004dc0 tx=0x7f6d2400b040).stop 2024-02-23T00:54:15.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.297+0000 7f6d2f76d700 1 -- 172.21.15.47:0/1303355281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d280fe510 msgr2=0x7f6d28107e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:15.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.297+0000 7f6d2f76d700 1 --2- 172.21.15.47:0/1303355281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d280fe510 0x7f6d28107e50 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 rx=0x7f6d2000bfc0 tx=0x7f6d20004d10).stop 2024-02-23T00:54:15.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.300+0000 7f6d2f76d700 1 -- 172.21.15.47:0/1303355281 shutdown_connections 2024-02-23T00:54:15.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.300+0000 7f6d2f76d700 1 --2- 172.21.15.47:0/1303355281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d280fe510 0x7f6d28107e50 unknown :-1 s=CLOSED pgs=2625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:15.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.300+0000 7f6d2f76d700 1 --2- 172.21.15.47:0/1303355281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d280f8420 0x7f6d2810dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:15.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.300+0000 7f6d2f76d700 1 --2- 172.21.15.47:0/1303355281 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6d14062cc0 0x7f6d14065180 unknown :-1 s=CLOSED pgs=3175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:15.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.300+0000 7f6d2f76d700 1 --2- 172.21.15.47:0/1303355281 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6d280fdbb0 0x7f6d2810e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:15.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.300+0000 7f6d2f76d700 1 -- 172.21.15.47:0/1303355281 >> 172.21.15.47:0/1303355281 conn(0x7f6d280f3800 msgr2=0x7f6d280f50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:15.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.300+0000 7f6d2f76d700 1 -- 172.21.15.47:0/1303355281 shutdown_connections 2024-02-23T00:54:15.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.300+0000 7f6d2f76d700 1 -- 172.21.15.47:0/1303355281 wait complete. 2024-02-23T00:54:15.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.300+0000 7f6d2f76d700 1 librados: shutdown 2024-02-23T00:54:15.304 INFO:teuthology.orchestra.run.smithi047.stderr:reweighted osd.6 to 1 (10000) 2024-02-23T00:54:15.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.845+0000 7f88eb115700 1 -- 172.21.15.47:0/1709370847 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f88e40f7400 msgr2=0x7f88e4103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:15.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.845+0000 7f88eb115700 1 --2- 172.21.15.47:0/1709370847 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f88e40f7400 0x7f88e4103a20 secure :-1 s=READY pgs=2786 cs=0 l=1 rev1=1 rx=0x7f88dc004870 tx=0x7f88dc009cb0).stop 2024-02-23T00:54:15.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.845+0000 7f88eb115700 1 -- 172.21.15.47:0/1709370847 shutdown_connections 2024-02-23T00:54:15.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.845+0000 7f88eb115700 1 --2- 172.21.15.47:0/1709370847 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f88e40f7400 0x7f88e4103a20 unknown :-1 s=CLOSED pgs=2786 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:15.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.845+0000 7f88eb115700 1 --2- 172.21.15.47:0/1709370847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88e40fcbb0 0x7f88e40fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:15.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.845+0000 7f88eb115700 1 --2- 172.21.15.47:0/1709370847 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88e40fd510 0x7f88e40f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:15.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.845+0000 7f88eb115700 1 -- 172.21.15.47:0/1709370847 >> 172.21.15.47:0/1709370847 conn(0x7f88e40f2820 msgr2=0x7f88e40f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:15.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.845+0000 7f88eb115700 1 -- 172.21.15.47:0/1709370847 shutdown_connections 2024-02-23T00:54:15.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.845+0000 7f88eb115700 1 -- 172.21.15.47:0/1709370847 wait complete. 2024-02-23T00:54:15.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.846+0000 7f88eb115700 1 librados: starting msgr at 2024-02-23T00:54:15.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.846+0000 7f88eb115700 1 librados: starting objecter 2024-02-23T00:54:15.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.846+0000 7f88eb115700 1 Processor -- start 2024-02-23T00:54:15.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.846+0000 7f88eb115700 1 -- start start 2024-02-23T00:54:15.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.846+0000 7f88eb115700 1 librados: setting wanted keys 2024-02-23T00:54:15.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.846+0000 7f88eb115700 1 librados: calling monclient init 2024-02-23T00:54:15.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.846+0000 7f88eb115700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f88e40f7400 0x7f88e410cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:15.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.846+0000 7f88eb115700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88e40fcbb0 0x7f88e410d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:15.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.846+0000 7f88eb115700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88e40fd510 0x7f88e4106f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:15.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.846+0000 7f88eb115700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e4107430 con 0x7f88e40fd510 2024-02-23T00:54:15.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.846+0000 7f88eb115700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e41075a0 con 0x7f88e40fcbb0 2024-02-23T00:54:15.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.846+0000 7f88eb115700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e4107760 con 0x7f88e40f7400 2024-02-23T00:54:15.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.846+0000 7f88e8eb1700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f88e40f7400 0x7f88e410cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:15.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.846+0000 7f88e8eb1700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f88e40f7400 0x7f88e410cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:39978/0 (socket says 172.21.15.47:39978) 2024-02-23T00:54:15.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.846+0000 7f88e8eb1700 1 -- 172.21.15.47:0/2811044482 learned_addr learned my addr 172.21.15.47:0/2811044482 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:54:15.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.846+0000 7f88e96b2700 1 --2- 172.21.15.47:0/2811044482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88e40fd510 0x7f88e4106f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:15.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.847+0000 7f88e3fff700 1 --2- 172.21.15.47:0/2811044482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88e40fcbb0 0x7f88e410d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:15.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.847+0000 7f88e8eb1700 1 -- 172.21.15.47:0/2811044482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88e40fcbb0 msgr2=0x7f88e410d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:15.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.847+0000 7f88e8eb1700 1 --2- 172.21.15.47:0/2811044482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88e40fcbb0 0x7f88e410d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:15.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.847+0000 7f88e8eb1700 1 -- 172.21.15.47:0/2811044482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88e40fd510 msgr2=0x7f88e4106f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:15.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.847+0000 7f88e8eb1700 1 --2- 172.21.15.47:0/2811044482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88e40fd510 0x7f88e4106f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:15.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.847+0000 7f88e8eb1700 1 -- 172.21.15.47:0/2811044482 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88dc005040 con 0x7f88e40f7400 2024-02-23T00:54:15.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.847+0000 7f88e8eb1700 1 --2- 172.21.15.47:0/2811044482 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f88e40f7400 0x7f88e410cd50 secure :-1 s=READY pgs=2787 cs=0 l=1 rev1=1 rx=0x7f88d400db80 tx=0x7f88d4004e60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:15.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.847+0000 7f88e1ffb700 10 client.?.objecter ms_handle_connect 0x7f88e40f7400 2024-02-23T00:54:15.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.847+0000 7f88e1ffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:54:15.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.847+0000 7f88e1ffb700 1 -- 172.21.15.47:0/2811044482 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d4007e70 con 0x7f88e40f7400 2024-02-23T00:54:15.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.847+0000 7f88e1ffb700 1 -- 172.21.15.47:0/2811044482 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f88d400f040 con 0x7f88e40f7400 2024-02-23T00:54:15.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.847+0000 7f88e1ffb700 1 -- 172.21.15.47:0/2811044482 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d4003430 con 0x7f88e40f7400 2024-02-23T00:54:15.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.847+0000 7f88eb115700 1 -- 172.21.15.47:0/2811044482 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88e4107a40 con 0x7f88e40f7400 2024-02-23T00:54:15.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.847+0000 7f88eb115700 10 client.41996.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:54:15.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.847+0000 7f88eb115700 1 -- 172.21.15.47:0/2811044482 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88e4113b80 con 0x7f88e40f7400 2024-02-23T00:54:15.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.847+0000 7f88eb115700 1 librados: init done 2024-02-23T00:54:15.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.848+0000 7f88eb115700 1 -- 172.21.15.47:0/2811044482 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88e405fa60 con 0x7f88e40f7400 2024-02-23T00:54:15.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.848+0000 7f88e1ffb700 1 -- 172.21.15.47:0/2811044482 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f88d400b7d0 con 0x7f88e40f7400 2024-02-23T00:54:15.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.849+0000 7f88e1ffb700 1 --2- 172.21.15.47:0/2811044482 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f88cc062cc0 0x7f88cc065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:15.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.849+0000 7f88e1ffb700 1 -- 172.21.15.47:0/2811044482 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(716..716 src has 621..716) v4 ==== 9780+0+0 (secure 0 0 0) 0x7f88d407d330 con 0x7f88e40f7400 2024-02-23T00:54:15.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.849+0000 7f88e1ffb700 10 client.41996.objecter ms_dispatch 0x7f88e4001550 osd_map(716..716 src has 621..716) v4 2024-02-23T00:54:15.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.849+0000 7f88e1ffb700 3 client.41996.objecter handle_osd_map got epochs [716,716] > 0 2024-02-23T00:54:15.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.849+0000 7f88e1ffb700 3 client.41996.objecter handle_osd_map decoding full epoch 716 2024-02-23T00:54:15.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.849+0000 7f88e3fff700 1 --2- 172.21.15.47:0/2811044482 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f88cc062cc0 0x7f88cc065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:15.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.849+0000 7f88e1ffb700 20 client.41996.objecter dump_active .. 0 homeless 2024-02-23T00:54:15.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.849+0000 7f88e1ffb700 10 client.41996.objecter ms_handle_connect 0x7f88cc062cc0 2024-02-23T00:54:15.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.849+0000 7f88e3fff700 1 --2- 172.21.15.47:0/2811044482 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f88cc062cc0 0x7f88cc065180 secure :-1 s=READY pgs=3176 cs=0 l=1 rev1=1 rx=0x7f88e4108690 tx=0x7f88d800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:15.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:15.854+0000 7f88e1ffb700 1 -- 172.21.15.47:0/2811044482 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f88d404c710 con 0x7f88e40f7400 2024-02-23T00:54:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:16.036+0000 7f88eb115700 1 -- 172.21.15.47:0/2811044482 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7f88e41083d0 con 0x7f88e40f7400 2024-02-23T00:54:16.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:14.879003+0000 mon.a (mon.0) 3489 : audit [INF] from='client.? 172.21.15.47:0/1303355281' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-23T00:54:16.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:15.190364+0000 mon.a (mon.0) 3490 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:54:16.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:15.294299+0000 mon.a (mon.0) 3491 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:54:16.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:15.294402+0000 mon.a (mon.0) 3492 : audit [INF] from='client.? 172.21.15.47:0/1303355281' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-02-23T00:54:16.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:15.294456+0000 mon.a (mon.0) 3493 : cluster [DBG] osdmap e716: 8 total, 8 up, 6 in 2024-02-23T00:54:16.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:15.299065+0000 mon.c (mon.2) 424 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-124"}]: dispatch 2024-02-23T00:54:16.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:15.299474+0000 mon.a (mon.0) 3494 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-124"}]: dispatch 2024-02-23T00:54:16.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:15.382705+0000 mgr.y (mgr.14517) 614 : cluster [DBG] pgmap v1083: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/720 objects misplaced (0.139%) 2024-02-23T00:54:16.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:14.879003+0000 mon.a (mon.0) 3489 : audit [INF] from='client.? 172.21.15.47:0/1303355281' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-23T00:54:16.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:15.190364+0000 mon.a (mon.0) 3490 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:54:16.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:15.294299+0000 mon.a (mon.0) 3491 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:54:16.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:15.294402+0000 mon.a (mon.0) 3492 : audit [INF] from='client.? 172.21.15.47:0/1303355281' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-02-23T00:54:16.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:15.294456+0000 mon.a (mon.0) 3493 : cluster [DBG] osdmap e716: 8 total, 8 up, 6 in 2024-02-23T00:54:16.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:15.299065+0000 mon.c (mon.2) 424 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-124"}]: dispatch 2024-02-23T00:54:16.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:15.299474+0000 mon.a (mon.0) 3494 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-124"}]: dispatch 2024-02-23T00:54:16.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:15.382705+0000 mgr.y (mgr.14517) 614 : cluster [DBG] pgmap v1083: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/720 objects misplaced (0.139%) 2024-02-23T00:54:16.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:14.879003+0000 mon.a (mon.0) 3489 : audit [INF] from='client.? 172.21.15.47:0/1303355281' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-23T00:54:16.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:15.190364+0000 mon.a (mon.0) 3490 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:54:16.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:15.294299+0000 mon.a (mon.0) 3491 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:54:16.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:15.294402+0000 mon.a (mon.0) 3492 : audit [INF] from='client.? 172.21.15.47:0/1303355281' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-02-23T00:54:16.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:15.294456+0000 mon.a (mon.0) 3493 : cluster [DBG] osdmap e716: 8 total, 8 up, 6 in 2024-02-23T00:54:16.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:15.299065+0000 mon.c (mon.2) 424 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-124"}]: dispatch 2024-02-23T00:54:16.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:15.299474+0000 mon.a (mon.0) 3494 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-124"}]: dispatch 2024-02-23T00:54:16.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:15.382705+0000 mgr.y (mgr.14517) 614 : cluster [DBG] pgmap v1083: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/720 objects misplaced (0.139%) 2024-02-23T00:54:16.259 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:54:16.262 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:16.263 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:54:16.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:16.303+0000 7f88e1ffb700 1 -- 172.21.15.47:0/2811044482 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v717) v1 ==== 87+0+0 (secure 0 0 0) 0x7f88d404fd30 con 0x7f88e40f7400 2024-02-23T00:54:16.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:16.307+0000 7f88eb115700 10 librados: watch_flush enter 2024-02-23T00:54:16.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:16.307+0000 7f88eb115700 10 librados: watch_flush exit 2024-02-23T00:54:16.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:16.307+0000 7f88eb115700 20 client.41996.objecter shutdown clearing up homeless session... 2024-02-23T00:54:16.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:16.307+0000 7f88eb115700 10 client.41996.objecter successfully canceled tick 2024-02-23T00:54:16.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:16.307+0000 7f88eb115700 1 -- 172.21.15.47:0/2811044482 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f88cc062cc0 msgr2=0x7f88cc065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:16.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:16.307+0000 7f88eb115700 1 --2- 172.21.15.47:0/2811044482 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f88cc062cc0 0x7f88cc065180 secure :-1 s=READY pgs=3176 cs=0 l=1 rev1=1 rx=0x7f88e4108690 tx=0x7f88d800b040).stop 2024-02-23T00:54:16.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:16.307+0000 7f88eb115700 1 -- 172.21.15.47:0/2811044482 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f88e40f7400 msgr2=0x7f88e410cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:16.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:16.307+0000 7f88eb115700 1 --2- 172.21.15.47:0/2811044482 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f88e40f7400 0x7f88e410cd50 secure :-1 s=READY pgs=2787 cs=0 l=1 rev1=1 rx=0x7f88d400db80 tx=0x7f88d4004e60).stop 2024-02-23T00:54:16.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:16.307+0000 7f88eb115700 1 -- 172.21.15.47:0/2811044482 shutdown_connections 2024-02-23T00:54:16.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:16.307+0000 7f88eb115700 1 --2- 172.21.15.47:0/2811044482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88e40fd510 0x7f88e4106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:16.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:16.307+0000 7f88eb115700 1 --2- 172.21.15.47:0/2811044482 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f88cc062cc0 0x7f88cc065180 unknown :-1 s=CLOSED pgs=3176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:16.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:16.307+0000 7f88eb115700 1 --2- 172.21.15.47:0/2811044482 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f88e40f7400 0x7f88e410cd50 unknown :-1 s=CLOSED pgs=2787 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:16.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:16.307+0000 7f88eb115700 1 --2- 172.21.15.47:0/2811044482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88e40fcbb0 0x7f88e410d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:16.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:16.307+0000 7f88eb115700 1 -- 172.21.15.47:0/2811044482 >> 172.21.15.47:0/2811044482 conn(0x7f88e40f2820 msgr2=0x7f88e40f4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:16.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:16.308+0000 7f88eb115700 1 -- 172.21.15.47:0/2811044482 shutdown_connections 2024-02-23T00:54:16.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:16.308+0000 7f88eb115700 1 -- 172.21.15.47:0/2811044482 wait complete. 2024-02-23T00:54:16.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:16.308+0000 7f88eb115700 1 librados: shutdown 2024-02-23T00:54:16.311 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is set 2024-02-23T00:54:16.364 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:54:16.396 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:16.417 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:54:16.420 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:16.420 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:54:16.521 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:54:16.550 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:16.572 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:54:16.575 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:16.576 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:54:16.676 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:54:16.704 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:16.724 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:54:16.726 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:16.727 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:54:16.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:15.368455+0000 osd.4 (osd.4) 301 : cluster [DBG] 234.0 scrub starts 2024-02-23T00:54:16.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:15.372229+0000 osd.4 (osd.4) 302 : cluster [DBG] 234.0 scrub ok 2024-02-23T00:54:16.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:16.037715+0000 mon.c (mon.2) 425 : audit [INF] from='client.? 172.21.15.47:0/2811044482' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:54:16.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:16.038068+0000 mon.a (mon.0) 3495 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:54:16.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:16.296240+0000 mon.a (mon.0) 3496 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:54:16.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:16.296275+0000 mon.a (mon.0) 3497 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:54:16.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:16.302873+0000 mon.a (mon.0) 3498 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-124"}]': finished 2024-02-23T00:54:16.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:16.302956+0000 mon.a (mon.0) 3499 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:54:16.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:16.303003+0000 mon.a (mon.0) 3500 : cluster [DBG] osdmap e717: 8 total, 8 up, 6 in 2024-02-23T00:54:16.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:15.368455+0000 osd.4 (osd.4) 301 : cluster [DBG] 234.0 scrub starts 2024-02-23T00:54:16.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:15.372229+0000 osd.4 (osd.4) 302 : cluster [DBG] 234.0 scrub ok 2024-02-23T00:54:16.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:16.037715+0000 mon.c (mon.2) 425 : audit [INF] from='client.? 172.21.15.47:0/2811044482' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:54:16.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:16.038068+0000 mon.a (mon.0) 3495 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:54:16.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:16.296240+0000 mon.a (mon.0) 3496 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:54:16.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:16.296275+0000 mon.a (mon.0) 3497 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:54:16.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:16.302873+0000 mon.a (mon.0) 3498 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-124"}]': finished 2024-02-23T00:54:16.810 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:16.302956+0000 mon.a (mon.0) 3499 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:54:16.811 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:16.303003+0000 mon.a (mon.0) 3500 : cluster [DBG] osdmap e717: 8 total, 8 up, 6 in 2024-02-23T00:54:16.828 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:54:16.861 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:16.884 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:54:16.886 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:16.886 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:54:16.987 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:54:17.016 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:17.016 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:54:17.016 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:54:17.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:15.368455+0000 osd.4 (osd.4) 301 : cluster [DBG] 234.0 scrub starts 2024-02-23T00:54:17.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:15.372229+0000 osd.4 (osd.4) 302 : cluster [DBG] 234.0 scrub ok 2024-02-23T00:54:17.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:16.037715+0000 mon.c (mon.2) 425 : audit [INF] from='client.? 172.21.15.47:0/2811044482' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:54:17.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:16.038068+0000 mon.a (mon.0) 3495 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:54:17.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:16.296240+0000 mon.a (mon.0) 3496 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:54:17.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:16.296275+0000 mon.a (mon.0) 3497 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:54:17.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:16.302873+0000 mon.a (mon.0) 3498 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-124"}]': finished 2024-02-23T00:54:17.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:16.302956+0000 mon.a (mon.0) 3499 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:54:17.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:16.303003+0000 mon.a (mon.0) 3500 : cluster [DBG] osdmap e717: 8 total, 8 up, 6 in 2024-02-23T00:54:17.701 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight 7 1 2024-02-23T00:54:18.070 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:54:18.072 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:18.073 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:54:18.174 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:54:18.205 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:18.226 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:54:18.229 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:18.229 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:54:18.291 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:54:18.330 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:54:18.355 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:17.304580+0000 mon.a (mon.0) 3501 : cluster [DBG] osdmap e718: 8 total, 8 up, 6 in 2024-02-23T00:54:18.356 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:17.383111+0000 mgr.y (mgr.14517) 615 : cluster [DBG] pgmap v1086: 152 pgs: 1 active+remapped+backfill_toofull, 151 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s; 1/723 objects misplaced (0.138%) 2024-02-23T00:54:18.356 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:17.304580+0000 mon.a (mon.0) 3501 : cluster [DBG] osdmap e718: 8 total, 8 up, 6 in 2024-02-23T00:54:18.356 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:17.383111+0000 mgr.y (mgr.14517) 615 : cluster [DBG] pgmap v1086: 152 pgs: 1 active+remapped+backfill_toofull, 151 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s; 1/723 objects misplaced (0.138%) 2024-02-23T00:54:18.358 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:18.411 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:54:18.416 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:18.417 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:54:18.518 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:54:18.546 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:18.644 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:54:18.646 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.1 dump_ops_in_flight 2024-02-23T00:54:18.647 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:18.647 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:54:18.670 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:54:18.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:17.304580+0000 mon.a (mon.0) 3501 : cluster [DBG] osdmap e718: 8 total, 8 up, 6 in 2024-02-23T00:54:18.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:17.383111+0000 mgr.y (mgr.14517) 615 : cluster [DBG] pgmap v1086: 152 pgs: 1 active+remapped+backfill_toofull, 151 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s; 1/723 objects misplaced (0.138%) 2024-02-23T00:54:18.748 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:54:18.780 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:18.801 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:54:18.804 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:18.805 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:54:18.905 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:54:18.936 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:18.956 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:54:18.959 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:18.960 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:54:19.061 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:54:19.089 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:19.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:18.312707+0000 mon.a (mon.0) 3502 : cluster [DBG] osdmap e719: 8 total, 8 up, 6 in 2024-02-23T00:54:19.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:18.317373+0000 mon.c (mon.2) 426 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:54:19.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:18.317734+0000 mon.a (mon.0) 3503 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:54:19.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:18.690610+0000 mon.a (mon.0) 3504 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:54:19.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:19.007689+0000 mon.a (mon.0) 3505 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:54:19.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:19.008363+0000 mon.a (mon.0) 3506 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:54:19.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:19.012869+0000 mon.a (mon.0) 3507 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:54:19.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:18.312707+0000 mon.a (mon.0) 3502 : cluster [DBG] osdmap e719: 8 total, 8 up, 6 in 2024-02-23T00:54:19.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:18.317373+0000 mon.c (mon.2) 426 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:54:19.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:18.317734+0000 mon.a (mon.0) 3503 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:54:19.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:18.690610+0000 mon.a (mon.0) 3504 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:54:19.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:19.007689+0000 mon.a (mon.0) 3505 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:54:19.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:19.008363+0000 mon.a (mon.0) 3506 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:54:19.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:19.012869+0000 mon.a (mon.0) 3507 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:54:19.592 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:54:19.595 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:19.596 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:54:19.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:18.312707+0000 mon.a (mon.0) 3502 : cluster [DBG] osdmap e719: 8 total, 8 up, 6 in 2024-02-23T00:54:19.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:18.317373+0000 mon.c (mon.2) 426 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:54:19.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:18.317734+0000 mon.a (mon.0) 3503 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:54:19.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:18.690610+0000 mon.a (mon.0) 3504 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:54:19.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:19.007689+0000 mon.a (mon.0) 3505 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:54:19.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:19.008363+0000 mon.a (mon.0) 3506 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:54:19.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:19.012869+0000 mon.a (mon.0) 3507 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:54:19.697 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:54:19.726 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:20.088 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-23T00:54:20.635 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:19.313618+0000 mon.a (mon.0) 3508 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:54:20.636 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:19.313754+0000 mon.a (mon.0) 3509 : cluster [DBG] osdmap e720: 8 total, 8 up, 6 in 2024-02-23T00:54:20.636 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:19.314979+0000 mon.c (mon.2) 427 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:54:20.636 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:19.324479+0000 mon.a (mon.0) 3510 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:54:20.636 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:19.383501+0000 mgr.y (mgr.14517) 616 : cluster [DBG] pgmap v1089: 184 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 151 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s; 1/723 objects misplaced (0.138%) 2024-02-23T00:54:20.636 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:19.313618+0000 mon.a (mon.0) 3508 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:54:20.636 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:19.313754+0000 mon.a (mon.0) 3509 : cluster [DBG] osdmap e720: 8 total, 8 up, 6 in 2024-02-23T00:54:20.636 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:19.314979+0000 mon.c (mon.2) 427 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:54:20.637 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:19.324479+0000 mon.a (mon.0) 3510 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:54:20.637 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:19.383501+0000 mgr.y (mgr.14517) 616 : cluster [DBG] pgmap v1089: 184 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 151 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s; 1/723 objects misplaced (0.138%) 2024-02-23T00:54:20.637 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:54:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:54:20] "GET /metrics HTTP/1.1" 200 191407 "" "Prometheus/2.33.4" 2024-02-23T00:54:20.677 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:54:20.681 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:20.681 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:54:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:19.313618+0000 mon.a (mon.0) 3508 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:54:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:19.313754+0000 mon.a (mon.0) 3509 : cluster [DBG] osdmap e720: 8 total, 8 up, 6 in 2024-02-23T00:54:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:19.314979+0000 mon.c (mon.2) 427 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:54:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:19.324479+0000 mon.a (mon.0) 3510 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:54:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:19.383501+0000 mgr.y (mgr.14517) 616 : cluster [DBG] pgmap v1089: 184 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 151 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s; 1/723 objects misplaced (0.138%) 2024-02-23T00:54:20.699 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:54:20.782 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:54:20.816 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:20.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.906+0000 7f4038cc9700 1 -- 172.21.15.47:0/2207955513 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40340fdbd0 msgr2=0x7f40340fdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:20.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.906+0000 7f4038cc9700 1 --2- 172.21.15.47:0/2207955513 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40340fdbd0 0x7f40340fdfb0 secure :-1 s=READY pgs=2788 cs=0 l=1 rev1=1 rx=0x7f40240047b0 tx=0x7f4024009cb0).stop 2024-02-23T00:54:20.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.906+0000 7f4038cc9700 1 -- 172.21.15.47:0/2207955513 shutdown_connections 2024-02-23T00:54:20.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.906+0000 7f4038cc9700 1 --2- 172.21.15.47:0/2207955513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40340f6a90 0x7f4034104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:20.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.906+0000 7f4038cc9700 1 --2- 172.21.15.47:0/2207955513 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40340fdbd0 0x7f40340fdfb0 unknown :-1 s=CLOSED pgs=2788 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:20.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.906+0000 7f4038cc9700 1 --2- 172.21.15.47:0/2207955513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40340fe530 0x7f40340f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:20.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.906+0000 7f4038cc9700 1 -- 172.21.15.47:0/2207955513 >> 172.21.15.47:0/2207955513 conn(0x7f403406c370 msgr2=0x7f403406c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:20.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.907+0000 7f4038cc9700 1 -- 172.21.15.47:0/2207955513 shutdown_connections 2024-02-23T00:54:20.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.907+0000 7f4038cc9700 1 -- 172.21.15.47:0/2207955513 wait complete. 2024-02-23T00:54:20.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.907+0000 7f4038cc9700 1 librados: starting msgr at 2024-02-23T00:54:20.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.907+0000 7f4038cc9700 1 librados: starting objecter 2024-02-23T00:54:20.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.907+0000 7f4038cc9700 1 Processor -- start 2024-02-23T00:54:20.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.907+0000 7f4038cc9700 1 -- start start 2024-02-23T00:54:20.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.907+0000 7f4038cc9700 1 librados: setting wanted keys 2024-02-23T00:54:20.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.907+0000 7f4038cc9700 1 librados: calling monclient init 2024-02-23T00:54:20.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.908+0000 7f4038cc9700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40340f6a90 0x7f40340f7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:20.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.908+0000 7f4038cc9700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40340fdbd0 0x7f40340f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:20.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.908+0000 7f4038cc9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40340fe530 0x7f40340f8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:20.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.908+0000 7f4038cc9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4034116490 con 0x7f40340fe530 2024-02-23T00:54:20.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.908+0000 7f4038cc9700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4034116600 con 0x7f40340f6a90 2024-02-23T00:54:20.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.908+0000 7f4038cc9700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40341167c0 con 0x7f40340fdbd0 2024-02-23T00:54:20.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.908+0000 7f4031d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40340fdbd0 0x7f40340f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:20.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.908+0000 7f4032d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40340fe530 0x7f40340f8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:20.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.908+0000 7f4031d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40340fdbd0 0x7f40340f7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:54600/0 (socket says 172.21.15.47:54600) 2024-02-23T00:54:20.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.908+0000 7f4031d9b700 1 -- 172.21.15.47:0/3851244483 learned_addr learned my addr 172.21.15.47:0/3851244483 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:54:20.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.908+0000 7f403259c700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40340f6a90 0x7f40340f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:20.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.908+0000 7f4031d9b700 1 -- 172.21.15.47:0/3851244483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40340f6a90 msgr2=0x7f40340f7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:20.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.908+0000 7f4031d9b700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40340f6a90 0x7f40340f7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:20.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.908+0000 7f4031d9b700 1 -- 172.21.15.47:0/3851244483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40340fe530 msgr2=0x7f40340f8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:20.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.908+0000 7f4031d9b700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40340fe530 0x7f40340f8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:20.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.908+0000 7f4031d9b700 1 -- 172.21.15.47:0/3851244483 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f402c0097c0 con 0x7f40340fdbd0 2024-02-23T00:54:20.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.908+0000 7f4032d9d700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40340fe530 0x7f40340f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:54:20.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.908+0000 7f4031d9b700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40340fdbd0 0x7f40340f7b70 secure :-1 s=READY pgs=2789 cs=0 l=1 rev1=1 rx=0x7f40280041d0 tx=0x7f4028009f60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:20.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.908+0000 7f40237fe700 10 client.?.objecter ms_handle_connect 0x7f40340fdbd0 2024-02-23T00:54:20.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.908+0000 7f40237fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:54:20.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.908+0000 7f40237fe700 1 -- 172.21.15.47:0/3851244483 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4028007b40 con 0x7f40340fdbd0 2024-02-23T00:54:20.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.908+0000 7f40237fe700 1 -- 172.21.15.47:0/3851244483 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f402800f040 con 0x7f40340fdbd0 2024-02-23T00:54:20.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.908+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4024005040 con 0x7f40340fdbd0 2024-02-23T00:54:20.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.908+0000 7f40237fe700 1 -- 172.21.15.47:0/3851244483 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4028014730 con 0x7f40340fdbd0 2024-02-23T00:54:20.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.908+0000 7f4038cc9700 10 client.42002.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:54:20.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.908+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4034116e50 con 0x7f40340fdbd0 2024-02-23T00:54:20.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.908+0000 7f4038cc9700 1 librados: init done 2024-02-23T00:54:20.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.909+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f403405f950 con 0x7f40340fdbd0 2024-02-23T00:54:20.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.910+0000 7f40237fe700 1 -- 172.21.15.47:0/3851244483 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f4028014ae0 con 0x7f40340fdbd0 2024-02-23T00:54:20.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.910+0000 7f40237fe700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f401c062cc0 0x7f401c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:20.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.910+0000 7f40237fe700 1 -- 172.21.15.47:0/3851244483 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(721..721 src has 621..721) v4 ==== 9752+0+0 (secure 0 0 0) 0x7f402807db90 con 0x7f40340fdbd0 2024-02-23T00:54:20.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.910+0000 7f40237fe700 10 client.42002.objecter ms_dispatch 0x7f4034001550 osd_map(721..721 src has 621..721) v4 2024-02-23T00:54:20.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.910+0000 7f40237fe700 3 client.42002.objecter handle_osd_map got epochs [721,721] > 0 2024-02-23T00:54:20.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.910+0000 7f40237fe700 3 client.42002.objecter handle_osd_map decoding full epoch 721 2024-02-23T00:54:20.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.910+0000 7f403259c700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f401c062cc0 0x7f401c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:20.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.911+0000 7f40237fe700 20 client.42002.objecter dump_active .. 0 homeless 2024-02-23T00:54:20.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.911+0000 7f40237fe700 1 -- 172.21.15.47:0/3851244483 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v721) v1 ==== 54+0+16 (secure 0 0 0) 0x7f402807e070 con 0x7f40340fdbd0 2024-02-23T00:54:20.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.911+0000 7f40237fe700 10 client.42002.objecter ms_handle_connect 0x7f401c062cc0 2024-02-23T00:54:20.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.911+0000 7f403259c700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f401c062cc0 0x7f401c065180 secure :-1 s=READY pgs=3177 cs=0 l=1 rev1=1 rx=0x7f4024004e20 tx=0x7f4024009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:20.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.911+0000 7f4038cc9700 10 client.42002.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:20.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.911+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_assign -1 1 2024-02-23T00:54:20.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.911+0000 7f4038cc9700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f4034028540 0x7f40340fe9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:20.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.911+0000 7f4038cc9700 20 client.42002.objecter _get_session s=0x7f403405ee80 osd=0 3 2024-02-23T00:54:20.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.911+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f403405ee80 osd=0 3 2024-02-23T00:54:20.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.911+0000 7f4038cc9700 20 client.42002.objecter _get_session s=0x7f403405ee80 osd=0 3 2024-02-23T00:54:20.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.911+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_remove -1 1 2024-02-23T00:54:20.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.911+0000 7f4038cc9700 20 client.42002.objecter get_session s=0x7f403405ee80 osd=0 3 2024-02-23T00:54:20.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.911+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_assign 0 1 2024-02-23T00:54:20.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.911+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f403405ee80 osd=0 4 2024-02-23T00:54:20.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.911+0000 7f4038cc9700 10 client.42002.objecter _send_command 1 2024-02-23T00:54:20.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.911+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4034106e90 con 0x7f4034028540 2024-02-23T00:54:20.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.911+0000 7f4032d9d700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f4034028540 0x7f40340fe9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:20.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.911+0000 7f40237fe700 10 client.42002.objecter ms_handle_connect 0x7f4034028540 2024-02-23T00:54:20.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.911+0000 7f4032d9d700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f4034028540 0x7f40340fe9b0 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:20.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.912+0000 7f40237fe700 1 -- 172.21.15.47:0/3851244483 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4034106e90 con 0x7f4034028540 2024-02-23T00:54:20.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.912+0000 7f40237fe700 10 client.42002.objecter ms_dispatch 0x7f4034001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:54:20.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.912+0000 7f40237fe700 10 client.42002.objecter _finish_command 1 = system:0 2024-02-23T00:54:20.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.912+0000 7f40237fe700 20 client.42002.objecter put_session s=0x7f403405ee80 osd=0 4 2024-02-23T00:54:20.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.912+0000 7f40237fe700 15 client.42002.objecter _session_command_op_remove 0 1 2024-02-23T00:54:20.919 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:20.919 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:54:20.919 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:54:20.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.927+0000 7f4038cc9700 10 client.42002.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:54:20.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.927+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_assign -1 2 2024-02-23T00:54:20.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.927+0000 7f4038cc9700 20 client.42002.objecter _get_session s=0x7f403405ee80 osd=0 3 2024-02-23T00:54:20.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.927+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f403405ee80 osd=0 3 2024-02-23T00:54:20.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.927+0000 7f4038cc9700 20 client.42002.objecter _get_session s=0x7f403405ee80 osd=0 3 2024-02-23T00:54:20.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.927+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_remove -1 2 2024-02-23T00:54:20.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.927+0000 7f4038cc9700 20 client.42002.objecter get_session s=0x7f403405ee80 osd=0 3 2024-02-23T00:54:20.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.927+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_assign 0 2 2024-02-23T00:54:20.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.927+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f403405ee80 osd=0 4 2024-02-23T00:54:20.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.927+0000 7f4038cc9700 10 client.42002.objecter _send_command 2 2024-02-23T00:54:20.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.927+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f40340f8b60 con 0x7f4034028540 2024-02-23T00:54:20.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.929+0000 7f40237fe700 1 -- 172.21.15.47:0/3851244483 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f40340f8b60 con 0x7f4034028540 2024-02-23T00:54:20.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.929+0000 7f40237fe700 10 client.42002.objecter ms_dispatch 0x7f4034001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:54:20.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.929+0000 7f40237fe700 10 client.42002.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:54:20.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.929+0000 7f40237fe700 20 client.42002.objecter put_session s=0x7f403405ee80 osd=0 4 2024-02-23T00:54:20.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.929+0000 7f40237fe700 15 client.42002.objecter _session_command_op_remove 0 2 2024-02-23T00:54:20.931 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:54:20.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.929+0000 7f4038cc9700 10 client.42002.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:20.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.929+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_assign -1 3 2024-02-23T00:54:20.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.929+0000 7f4038cc9700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f4034027b90 0x7f4034027f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:20.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.929+0000 7f4038cc9700 20 client.42002.objecter _get_session s=0x7f4034100020 osd=1 3 2024-02-23T00:54:20.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.929+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f4034100020 osd=1 3 2024-02-23T00:54:20.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.929+0000 7f4038cc9700 20 client.42002.objecter _get_session s=0x7f4034100020 osd=1 3 2024-02-23T00:54:20.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.929+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_remove -1 3 2024-02-23T00:54:20.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.929+0000 7f4038cc9700 20 client.42002.objecter get_session s=0x7f4034100020 osd=1 3 2024-02-23T00:54:20.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.929+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_assign 1 3 2024-02-23T00:54:20.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.929+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f4034100020 osd=1 4 2024-02-23T00:54:20.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.929+0000 7f4038cc9700 10 client.42002.objecter _send_command 3 2024-02-23T00:54:20.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.929+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4034100190 con 0x7f4034027b90 2024-02-23T00:54:20.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.929+0000 7f403259c700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f4034027b90 0x7f4034027f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:20.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.930+0000 7f40237fe700 10 client.42002.objecter ms_handle_connect 0x7f4034027b90 2024-02-23T00:54:20.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.930+0000 7f403259c700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f4034027b90 0x7f4034027f90 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:20.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.930+0000 7f40237fe700 1 -- 172.21.15.47:0/3851244483 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4034100190 con 0x7f4034027b90 2024-02-23T00:54:20.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.930+0000 7f40237fe700 10 client.42002.objecter ms_dispatch 0x7f4034001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:54:20.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.930+0000 7f40237fe700 10 client.42002.objecter _finish_command 3 = system:0 2024-02-23T00:54:20.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.930+0000 7f40237fe700 20 client.42002.objecter put_session s=0x7f4034100020 osd=1 4 2024-02-23T00:54:20.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.930+0000 7f40237fe700 15 client.42002.objecter _session_command_op_remove 1 3 2024-02-23T00:54:20.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.945+0000 7f4038cc9700 10 client.42002.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:54:20.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.945+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_assign -1 4 2024-02-23T00:54:20.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.945+0000 7f4038cc9700 20 client.42002.objecter _get_session s=0x7f4034100020 osd=1 3 2024-02-23T00:54:20.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.945+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f4034100020 osd=1 3 2024-02-23T00:54:20.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.945+0000 7f4038cc9700 20 client.42002.objecter _get_session s=0x7f4034100020 osd=1 3 2024-02-23T00:54:20.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.945+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_remove -1 4 2024-02-23T00:54:20.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.945+0000 7f4038cc9700 20 client.42002.objecter get_session s=0x7f4034100020 osd=1 3 2024-02-23T00:54:20.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.945+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_assign 1 4 2024-02-23T00:54:20.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.945+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f4034100020 osd=1 4 2024-02-23T00:54:20.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.945+0000 7f4038cc9700 10 client.42002.objecter _send_command 4 2024-02-23T00:54:20.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.945+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4034027600 con 0x7f4034027b90 2024-02-23T00:54:20.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.947+0000 7f40237fe700 1 -- 172.21.15.47:0/3851244483 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4034027600 con 0x7f4034027b90 2024-02-23T00:54:20.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.947+0000 7f40237fe700 10 client.42002.objecter ms_dispatch 0x7f4034001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:54:20.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.947+0000 7f40237fe700 10 client.42002.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:54:20.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.947+0000 7f40237fe700 20 client.42002.objecter put_session s=0x7f4034100020 osd=1 4 2024-02-23T00:54:20.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.947+0000 7f40237fe700 15 client.42002.objecter _session_command_op_remove 1 4 2024-02-23T00:54:20.949 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:54:20.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.947+0000 7f4038cc9700 10 client.42002.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:20.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.947+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_assign -1 5 2024-02-23T00:54:20.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.947+0000 7f4038cc9700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f40340ffb80 0x7f403411e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:20.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.947+0000 7f4038cc9700 20 client.42002.objecter _get_session s=0x7f4034027a10 osd=2 3 2024-02-23T00:54:20.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.947+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f4034027a10 osd=2 3 2024-02-23T00:54:20.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.947+0000 7f4038cc9700 20 client.42002.objecter _get_session s=0x7f4034027a10 osd=2 3 2024-02-23T00:54:20.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.947+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_remove -1 5 2024-02-23T00:54:20.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.947+0000 7f4038cc9700 20 client.42002.objecter get_session s=0x7f4034027a10 osd=2 3 2024-02-23T00:54:20.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.947+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_assign 2 5 2024-02-23T00:54:20.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.947+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f4034027a10 osd=2 4 2024-02-23T00:54:20.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.947+0000 7f4038cc9700 10 client.42002.objecter _send_command 5 2024-02-23T00:54:20.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.947+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4034120f20 con 0x7f40340ffb80 2024-02-23T00:54:20.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.947+0000 7f4031d9b700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f40340ffb80 0x7f403411e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:20.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.947+0000 7f40237fe700 10 client.42002.objecter ms_handle_connect 0x7f40340ffb80 2024-02-23T00:54:20.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.947+0000 7f4031d9b700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f40340ffb80 0x7f403411e1f0 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:20.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.948+0000 7f40237fe700 1 -- 172.21.15.47:0/3851244483 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4034120f20 con 0x7f40340ffb80 2024-02-23T00:54:20.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.948+0000 7f40237fe700 10 client.42002.objecter ms_dispatch 0x7f4034001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:54:20.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.948+0000 7f40237fe700 10 client.42002.objecter _finish_command 5 = system:0 2024-02-23T00:54:20.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.948+0000 7f40237fe700 20 client.42002.objecter put_session s=0x7f4034027a10 osd=2 4 2024-02-23T00:54:20.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.948+0000 7f40237fe700 15 client.42002.objecter _session_command_op_remove 2 5 2024-02-23T00:54:20.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.962+0000 7f4038cc9700 10 client.42002.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:54:20.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.962+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_assign -1 6 2024-02-23T00:54:20.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.962+0000 7f4038cc9700 20 client.42002.objecter _get_session s=0x7f4034027a10 osd=2 3 2024-02-23T00:54:20.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.962+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f4034027a10 osd=2 3 2024-02-23T00:54:20.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.962+0000 7f4038cc9700 20 client.42002.objecter _get_session s=0x7f4034027a10 osd=2 3 2024-02-23T00:54:20.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.962+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_remove -1 6 2024-02-23T00:54:20.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.962+0000 7f4038cc9700 20 client.42002.objecter get_session s=0x7f4034027a10 osd=2 3 2024-02-23T00:54:20.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.962+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_assign 2 6 2024-02-23T00:54:20.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.962+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f4034027a10 osd=2 4 2024-02-23T00:54:20.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.962+0000 7f4038cc9700 10 client.42002.objecter _send_command 6 2024-02-23T00:54:20.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.962+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f40340ff0e0 con 0x7f40340ffb80 2024-02-23T00:54:20.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.964+0000 7f40237fe700 1 -- 172.21.15.47:0/3851244483 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f40340ff0e0 con 0x7f40340ffb80 2024-02-23T00:54:20.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.964+0000 7f40237fe700 10 client.42002.objecter ms_dispatch 0x7f4034001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:54:20.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.964+0000 7f40237fe700 10 client.42002.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:54:20.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.964+0000 7f40237fe700 20 client.42002.objecter put_session s=0x7f4034027a10 osd=2 4 2024-02-23T00:54:20.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.964+0000 7f40237fe700 15 client.42002.objecter _session_command_op_remove 2 6 2024-02-23T00:54:20.966 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:54:20.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.964+0000 7f4038cc9700 10 client.42002.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:20.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.964+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_assign -1 7 2024-02-23T00:54:20.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.964+0000 7f4038cc9700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f403411bd40 0x7f403411f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:20.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.964+0000 7f4038cc9700 20 client.42002.objecter _get_session s=0x7f40340ff490 osd=3 3 2024-02-23T00:54:20.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.964+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f40340ff490 osd=3 3 2024-02-23T00:54:20.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.964+0000 7f4038cc9700 20 client.42002.objecter _get_session s=0x7f40340ff490 osd=3 3 2024-02-23T00:54:20.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.964+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_remove -1 7 2024-02-23T00:54:20.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.964+0000 7f4038cc9700 20 client.42002.objecter get_session s=0x7f40340ff490 osd=3 3 2024-02-23T00:54:20.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.964+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_assign 3 7 2024-02-23T00:54:20.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.964+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f40340ff490 osd=3 4 2024-02-23T00:54:20.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.964+0000 7f4038cc9700 10 client.42002.objecter _send_command 7 2024-02-23T00:54:20.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.964+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f403411fef0 con 0x7f403411bd40 2024-02-23T00:54:20.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.964+0000 7f4032d9d700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f403411bd40 0x7f403411f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:20.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.964+0000 7f40237fe700 10 client.42002.objecter ms_handle_connect 0x7f403411bd40 2024-02-23T00:54:20.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.964+0000 7f4032d9d700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f403411bd40 0x7f403411f860 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:20.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.965+0000 7f40237fe700 1 -- 172.21.15.47:0/3851244483 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f403411fef0 con 0x7f403411bd40 2024-02-23T00:54:20.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.965+0000 7f40237fe700 10 client.42002.objecter ms_dispatch 0x7f4034001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:54:20.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.965+0000 7f40237fe700 10 client.42002.objecter _finish_command 7 = system:0 2024-02-23T00:54:20.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.965+0000 7f40237fe700 20 client.42002.objecter put_session s=0x7f40340ff490 osd=3 4 2024-02-23T00:54:20.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.965+0000 7f40237fe700 15 client.42002.objecter _session_command_op_remove 3 7 2024-02-23T00:54:20.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.979+0000 7f4038cc9700 10 client.42002.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:54:20.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.979+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_assign -1 8 2024-02-23T00:54:20.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.979+0000 7f4038cc9700 20 client.42002.objecter _get_session s=0x7f40340ff490 osd=3 3 2024-02-23T00:54:20.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.979+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f40340ff490 osd=3 3 2024-02-23T00:54:20.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.979+0000 7f4038cc9700 20 client.42002.objecter _get_session s=0x7f40340ff490 osd=3 3 2024-02-23T00:54:20.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.979+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_remove -1 8 2024-02-23T00:54:20.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.979+0000 7f4038cc9700 20 client.42002.objecter get_session s=0x7f40340ff490 osd=3 3 2024-02-23T00:54:20.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.979+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_assign 3 8 2024-02-23T00:54:20.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.979+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f40340ff490 osd=3 4 2024-02-23T00:54:20.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.979+0000 7f4038cc9700 10 client.42002.objecter _send_command 8 2024-02-23T00:54:20.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.979+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f403411b2a0 con 0x7f403411bd40 2024-02-23T00:54:20.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.981+0000 7f40237fe700 1 -- 172.21.15.47:0/3851244483 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f403411b2a0 con 0x7f403411bd40 2024-02-23T00:54:20.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.981+0000 7f40237fe700 10 client.42002.objecter ms_dispatch 0x7f4034001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:54:20.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.981+0000 7f40237fe700 10 client.42002.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:54:20.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.981+0000 7f40237fe700 20 client.42002.objecter put_session s=0x7f40340ff490 osd=3 4 2024-02-23T00:54:20.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.981+0000 7f40237fe700 15 client.42002.objecter _session_command_op_remove 3 8 2024-02-23T00:54:20.983 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:54:20.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.981+0000 7f4038cc9700 10 client.42002.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:20.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.981+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_assign -1 9 2024-02-23T00:54:20.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.981+0000 7f4038cc9700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f40341230a0 0x7f4034125490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:20.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.981+0000 7f4038cc9700 20 client.42002.objecter _get_session s=0x7f4034120a90 osd=4 3 2024-02-23T00:54:20.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.981+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f4034120a90 osd=4 3 2024-02-23T00:54:20.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.981+0000 7f4038cc9700 20 client.42002.objecter _get_session s=0x7f4034120a90 osd=4 3 2024-02-23T00:54:20.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.981+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_remove -1 9 2024-02-23T00:54:20.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.981+0000 7f4038cc9700 20 client.42002.objecter get_session s=0x7f4034120a90 osd=4 3 2024-02-23T00:54:20.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.981+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_assign 4 9 2024-02-23T00:54:20.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.981+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f4034120a90 osd=4 4 2024-02-23T00:54:20.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.981+0000 7f4038cc9700 10 client.42002.objecter _send_command 9 2024-02-23T00:54:20.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.981+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4034120670 con 0x7f40341230a0 2024-02-23T00:54:20.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.981+0000 7f403259c700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f40341230a0 0x7f4034125490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:20.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.982+0000 7f40237fe700 10 client.42002.objecter ms_handle_connect 0x7f40341230a0 2024-02-23T00:54:20.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.982+0000 7f403259c700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f40341230a0 0x7f4034125490 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:20.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.982+0000 7f40237fe700 1 -- 172.21.15.47:0/3851244483 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4034120670 con 0x7f40341230a0 2024-02-23T00:54:20.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.982+0000 7f40237fe700 10 client.42002.objecter ms_dispatch 0x7f4034001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:54:20.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.982+0000 7f40237fe700 10 client.42002.objecter _finish_command 9 = system:0 2024-02-23T00:54:20.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.982+0000 7f40237fe700 20 client.42002.objecter put_session s=0x7f4034120a90 osd=4 4 2024-02-23T00:54:20.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.982+0000 7f40237fe700 15 client.42002.objecter _session_command_op_remove 4 9 2024-02-23T00:54:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.997+0000 7f4038cc9700 10 client.42002.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:54:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.997+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_assign -1 10 2024-02-23T00:54:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.997+0000 7f4038cc9700 20 client.42002.objecter _get_session s=0x7f4034120a90 osd=4 3 2024-02-23T00:54:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.997+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f4034120a90 osd=4 3 2024-02-23T00:54:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.997+0000 7f4038cc9700 20 client.42002.objecter _get_session s=0x7f4034120a90 osd=4 3 2024-02-23T00:54:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.997+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_remove -1 10 2024-02-23T00:54:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.997+0000 7f4038cc9700 20 client.42002.objecter get_session s=0x7f4034120a90 osd=4 3 2024-02-23T00:54:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.997+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_assign 4 10 2024-02-23T00:54:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.997+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f4034120a90 osd=4 4 2024-02-23T00:54:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.997+0000 7f4038cc9700 10 client.42002.objecter _send_command 10 2024-02-23T00:54:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.997+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4034120630 con 0x7f40341230a0 2024-02-23T00:54:21.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.999+0000 7f40237fe700 1 -- 172.21.15.47:0/3851244483 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4034120630 con 0x7f40341230a0 2024-02-23T00:54:21.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.999+0000 7f40237fe700 10 client.42002.objecter ms_dispatch 0x7f4034001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:54:21.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.999+0000 7f40237fe700 10 client.42002.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:54:21.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.999+0000 7f40237fe700 20 client.42002.objecter put_session s=0x7f4034120a90 osd=4 4 2024-02-23T00:54:21.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.999+0000 7f40237fe700 15 client.42002.objecter _session_command_op_remove 4 10 2024-02-23T00:54:21.001 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:54:21.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.999+0000 7f4038cc9700 10 client.42002.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:21.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.999+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_assign -1 11 2024-02-23T00:54:21.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.999+0000 7f4038cc9700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f4034125e90 0x7f4034128300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:21.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.999+0000 7f4038cc9700 20 client.42002.objecter _get_session s=0x7f4034120630 osd=5 3 2024-02-23T00:54:21.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.999+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f4034120630 osd=5 3 2024-02-23T00:54:21.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.999+0000 7f4038cc9700 20 client.42002.objecter _get_session s=0x7f4034120630 osd=5 3 2024-02-23T00:54:21.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.999+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_remove -1 11 2024-02-23T00:54:21.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.999+0000 7f4038cc9700 20 client.42002.objecter get_session s=0x7f4034120630 osd=5 3 2024-02-23T00:54:21.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.999+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_assign 5 11 2024-02-23T00:54:21.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.999+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f4034120630 osd=5 4 2024-02-23T00:54:21.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.999+0000 7f4038cc9700 10 client.42002.objecter _send_command 11 2024-02-23T00:54:21.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:20.999+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f403412b0f0 con 0x7f4034125e90 2024-02-23T00:54:21.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.000+0000 7f4031d9b700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f4034125e90 0x7f4034128300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:21.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.000+0000 7f4031d9b700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f4034125e90 0x7f4034128300 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:21.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.000+0000 7f40237fe700 10 client.42002.objecter ms_handle_connect 0x7f4034125e90 2024-02-23T00:54:21.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.001+0000 7f40237fe700 1 -- 172.21.15.47:0/3851244483 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f403412b0f0 con 0x7f4034125e90 2024-02-23T00:54:21.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.001+0000 7f40237fe700 10 client.42002.objecter ms_dispatch 0x7f4034001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:54:21.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.001+0000 7f40237fe700 10 client.42002.objecter _finish_command 11 = system:0 2024-02-23T00:54:21.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.001+0000 7f40237fe700 20 client.42002.objecter put_session s=0x7f4034120630 osd=5 4 2024-02-23T00:54:21.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.001+0000 7f40237fe700 15 client.42002.objecter _session_command_op_remove 5 11 2024-02-23T00:54:21.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.015+0000 7f4038cc9700 10 client.42002.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:54:21.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.015+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_assign -1 12 2024-02-23T00:54:21.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.015+0000 7f4038cc9700 20 client.42002.objecter _get_session s=0x7f4034120630 osd=5 3 2024-02-23T00:54:21.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.015+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f4034120630 osd=5 3 2024-02-23T00:54:21.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.015+0000 7f4038cc9700 20 client.42002.objecter _get_session s=0x7f4034120630 osd=5 3 2024-02-23T00:54:21.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.015+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_remove -1 12 2024-02-23T00:54:21.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.015+0000 7f4038cc9700 20 client.42002.objecter get_session s=0x7f4034120630 osd=5 3 2024-02-23T00:54:21.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.015+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_assign 5 12 2024-02-23T00:54:21.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.015+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f4034120630 osd=5 4 2024-02-23T00:54:21.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.015+0000 7f4038cc9700 10 client.42002.objecter _send_command 12 2024-02-23T00:54:21.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.015+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f403412b0f0 con 0x7f4034125e90 2024-02-23T00:54:21.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.017+0000 7f40237fe700 1 -- 172.21.15.47:0/3851244483 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f403412b0f0 con 0x7f4034125e90 2024-02-23T00:54:21.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.017+0000 7f40237fe700 10 client.42002.objecter ms_dispatch 0x7f4034001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:54:21.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.017+0000 7f40237fe700 10 client.42002.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:54:21.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.017+0000 7f40237fe700 20 client.42002.objecter put_session s=0x7f4034120630 osd=5 4 2024-02-23T00:54:21.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.017+0000 7f40237fe700 15 client.42002.objecter _session_command_op_remove 5 12 2024-02-23T00:54:21.019 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:54:21.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.017+0000 7f4038cc9700 10 client.42002.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:21.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.017+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_assign -1 13 2024-02-23T00:54:21.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.017+0000 7f4038cc9700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f403412a420 0x7f403412a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:21.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.017+0000 7f4038cc9700 20 client.42002.objecter _get_session s=0x7f403412b0f0 osd=6 3 2024-02-23T00:54:21.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.017+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f403412b0f0 osd=6 3 2024-02-23T00:54:21.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.017+0000 7f4038cc9700 20 client.42002.objecter _get_session s=0x7f403412b0f0 osd=6 3 2024-02-23T00:54:21.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.017+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_remove -1 13 2024-02-23T00:54:21.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.017+0000 7f4038cc9700 20 client.42002.objecter get_session s=0x7f403412b0f0 osd=6 3 2024-02-23T00:54:21.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.017+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_assign 6 13 2024-02-23T00:54:21.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.017+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f403412b0f0 osd=6 4 2024-02-23T00:54:21.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.017+0000 7f4038cc9700 10 client.42002.objecter _send_command 13 2024-02-23T00:54:21.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.017+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f403412e450 con 0x7f403412a420 2024-02-23T00:54:21.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.017+0000 7f4032d9d700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f403412a420 0x7f403412a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:21.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.018+0000 7f40237fe700 10 client.42002.objecter ms_handle_connect 0x7f403412a420 2024-02-23T00:54:21.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.018+0000 7f4032d9d700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f403412a420 0x7f403412a8a0 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:21.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.019+0000 7f40237fe700 1 -- 172.21.15.47:0/3851244483 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f403412e450 con 0x7f403412a420 2024-02-23T00:54:21.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.019+0000 7f40237fe700 10 client.42002.objecter ms_dispatch 0x7f4034001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:54:21.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.019+0000 7f40237fe700 10 client.42002.objecter _finish_command 13 = system:0 2024-02-23T00:54:21.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.019+0000 7f40237fe700 20 client.42002.objecter put_session s=0x7f403412b0f0 osd=6 4 2024-02-23T00:54:21.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.019+0000 7f40237fe700 15 client.42002.objecter _session_command_op_remove 6 13 2024-02-23T00:54:21.022 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:54:21.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.034+0000 7f4038cc9700 10 client.42002.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:54:21.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.034+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_assign -1 14 2024-02-23T00:54:21.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.034+0000 7f4038cc9700 20 client.42002.objecter _get_session s=0x7f403412b0f0 osd=6 3 2024-02-23T00:54:21.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.034+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f403412b0f0 osd=6 3 2024-02-23T00:54:21.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.034+0000 7f4038cc9700 20 client.42002.objecter _get_session s=0x7f403412b0f0 osd=6 3 2024-02-23T00:54:21.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.034+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_remove -1 14 2024-02-23T00:54:21.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.034+0000 7f4038cc9700 20 client.42002.objecter get_session s=0x7f403412b0f0 osd=6 3 2024-02-23T00:54:21.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.034+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_assign 6 14 2024-02-23T00:54:21.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.034+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f403412b0f0 osd=6 4 2024-02-23T00:54:21.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.034+0000 7f4038cc9700 10 client.42002.objecter _send_command 14 2024-02-23T00:54:21.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.034+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f40341288b0 con 0x7f403412a420 2024-02-23T00:54:21.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.035+0000 7f40237fe700 1 -- 172.21.15.47:0/3851244483 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f40341288b0 con 0x7f403412a420 2024-02-23T00:54:21.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.035+0000 7f40237fe700 10 client.42002.objecter ms_dispatch 0x7f4034001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:54:21.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.036+0000 7f40237fe700 10 client.42002.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:54:21.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.036+0000 7f40237fe700 20 client.42002.objecter put_session s=0x7f403412b0f0 osd=6 4 2024-02-23T00:54:21.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.036+0000 7f40237fe700 15 client.42002.objecter _session_command_op_remove 6 14 2024-02-23T00:54:21.038 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:54:21.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.036+0000 7f4038cc9700 10 client.42002.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:21.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.036+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_assign -1 15 2024-02-23T00:54:21.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.036+0000 7f4038cc9700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f4034128f30 0x7f4034129390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:21.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.036+0000 7f4038cc9700 20 client.42002.objecter _get_session s=0x7f40341288b0 osd=7 3 2024-02-23T00:54:21.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.036+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f40341288b0 osd=7 3 2024-02-23T00:54:21.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.036+0000 7f4038cc9700 20 client.42002.objecter _get_session s=0x7f40341288b0 osd=7 3 2024-02-23T00:54:21.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.036+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_remove -1 15 2024-02-23T00:54:21.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.036+0000 7f4038cc9700 20 client.42002.objecter get_session s=0x7f40341288b0 osd=7 3 2024-02-23T00:54:21.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.036+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_assign 7 15 2024-02-23T00:54:21.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.036+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f40341288b0 osd=7 4 2024-02-23T00:54:21.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.036+0000 7f4038cc9700 10 client.42002.objecter _send_command 15 2024-02-23T00:54:21.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.036+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4034129a20 con 0x7f4034128f30 2024-02-23T00:54:21.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.036+0000 7f403259c700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f4034128f30 0x7f4034129390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:21.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.036+0000 7f40237fe700 10 client.42002.objecter ms_handle_connect 0x7f4034128f30 2024-02-23T00:54:21.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.036+0000 7f403259c700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f4034128f30 0x7f4034129390 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:21.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.037+0000 7f40237fe700 1 -- 172.21.15.47:0/3851244483 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4034129a20 con 0x7f4034128f30 2024-02-23T00:54:21.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.037+0000 7f40237fe700 10 client.42002.objecter ms_dispatch 0x7f4034001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:54:21.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.037+0000 7f40237fe700 10 client.42002.objecter _finish_command 15 = system:0 2024-02-23T00:54:21.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.037+0000 7f40237fe700 20 client.42002.objecter put_session s=0x7f40341288b0 osd=7 4 2024-02-23T00:54:21.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.037+0000 7f40237fe700 15 client.42002.objecter _session_command_op_remove 7 15 2024-02-23T00:54:21.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.052+0000 7f4038cc9700 10 client.42002.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:54:21.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.052+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_assign -1 16 2024-02-23T00:54:21.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.052+0000 7f4038cc9700 20 client.42002.objecter _get_session s=0x7f40341288b0 osd=7 3 2024-02-23T00:54:21.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.052+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f40341288b0 osd=7 3 2024-02-23T00:54:21.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.052+0000 7f4038cc9700 20 client.42002.objecter _get_session s=0x7f40341288b0 osd=7 3 2024-02-23T00:54:21.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.052+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_remove -1 16 2024-02-23T00:54:21.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.052+0000 7f4038cc9700 20 client.42002.objecter get_session s=0x7f40341288b0 osd=7 3 2024-02-23T00:54:21.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.052+0000 7f4038cc9700 15 client.42002.objecter _session_command_op_assign 7 16 2024-02-23T00:54:21.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.052+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f40341288b0 osd=7 4 2024-02-23T00:54:21.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.052+0000 7f4038cc9700 10 client.42002.objecter _send_command 16 2024-02-23T00:54:21.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.052+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4034129a20 con 0x7f4034128f30 2024-02-23T00:54:21.055 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:21.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f40237fe700 1 -- 172.21.15.47:0/3851244483 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4034129a20 con 0x7f4034128f30 2024-02-23T00:54:21.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f40237fe700 10 client.42002.objecter ms_dispatch 0x7f4034001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:54:21.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f40237fe700 10 client.42002.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:54:21.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f40237fe700 20 client.42002.objecter put_session s=0x7f40341288b0 osd=7 4 2024-02-23T00:54:21.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f40237fe700 15 client.42002.objecter _session_command_op_remove 7 16 2024-02-23T00:54:21.056 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:54:21.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 10 librados: watch_flush enter 2024-02-23T00:54:21.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 10 librados: watch_flush exit 2024-02-23T00:54:21.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 10 client.42002.objecter close_session for osd.0 2024-02-23T00:54:21.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f4034028540 msgr2=0x7f40340fe9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:21.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f4034028540 0x7f40340fe9b0 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:21.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f403405ee80 osd=0 1 2024-02-23T00:54:21.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 10 client.42002.objecter close_session for osd.1 2024-02-23T00:54:21.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f4034027b90 msgr2=0x7f4034027f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:21.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f4034027b90 0x7f4034027f90 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:21.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f4034100020 osd=1 1 2024-02-23T00:54:21.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 10 client.42002.objecter close_session for osd.2 2024-02-23T00:54:21.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f40340ffb80 msgr2=0x7f403411e1f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:21.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f40340ffb80 0x7f403411e1f0 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:21.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f4034027a10 osd=2 1 2024-02-23T00:54:21.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 10 client.42002.objecter close_session for osd.3 2024-02-23T00:54:21.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f403411bd40 msgr2=0x7f403411f860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:21.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f403411bd40 0x7f403411f860 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:21.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f40340ff490 osd=3 1 2024-02-23T00:54:21.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 10 client.42002.objecter close_session for osd.4 2024-02-23T00:54:21.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f40341230a0 msgr2=0x7f4034125490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:21.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4032d9d700 1 -- 172.21.15.47:0/3851244483 reap_dead start 2024-02-23T00:54:21.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f40341230a0 0x7f4034125490 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:21.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4032d9d700 1 -- 172.21.15.47:0/3851244483 reap_dead start 2024-02-23T00:54:21.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f4034120a90 osd=4 1 2024-02-23T00:54:21.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 10 client.42002.objecter close_session for osd.5 2024-02-23T00:54:21.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f4034125e90 msgr2=0x7f4034128300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:21.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f4034125e90 0x7f4034128300 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:21.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f4034120630 osd=5 1 2024-02-23T00:54:21.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 10 client.42002.objecter close_session for osd.6 2024-02-23T00:54:21.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f403412a420 msgr2=0x7f403412a8a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:21.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f403412a420 0x7f403412a8a0 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:21.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f403412b0f0 osd=6 1 2024-02-23T00:54:21.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 10 client.42002.objecter close_session for osd.7 2024-02-23T00:54:21.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f4034128f30 msgr2=0x7f4034129390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:21.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f4034128f30 0x7f4034129390 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:21.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 20 client.42002.objecter put_session s=0x7f40341288b0 osd=7 1 2024-02-23T00:54:21.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 20 client.42002.objecter shutdown clearing up homeless session... 2024-02-23T00:54:21.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 10 client.42002.objecter successfully canceled tick 2024-02-23T00:54:21.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f401c062cc0 msgr2=0x7f401c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:21.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f401c062cc0 0x7f401c065180 secure :-1 s=READY pgs=3177 cs=0 l=1 rev1=1 rx=0x7f4024004e20 tx=0x7f4024009cb0).stop 2024-02-23T00:54:21.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40340fdbd0 msgr2=0x7f40340f7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:21.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 1 --2- 172.21.15.47:0/3851244483 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40340fdbd0 0x7f40340f7b70 secure :-1 s=READY pgs=2789 cs=0 l=1 rev1=1 rx=0x7f40280041d0 tx=0x7f4028009f60).stop 2024-02-23T00:54:21.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4032d9d700 1 -- 172.21.15.47:0/3851244483 reap_dead start 2024-02-23T00:54:21.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 shutdown_connections 2024-02-23T00:54:21.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 >> 172.21.15.47:0/3851244483 conn(0x7f403406c370 msgr2=0x7f40340fba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:21.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 shutdown_connections 2024-02-23T00:54:21.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 1 -- 172.21.15.47:0/3851244483 wait complete. 2024-02-23T00:54:21.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:21.054+0000 7f4038cc9700 1 librados: shutdown 2024-02-23T00:54:21.063 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:54:21.063 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:54:21.063 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:54:21.063 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:54:21.063 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:54:21.063 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:54:21.063 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:54:21.063 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:54:21.075 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:54:21.078 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:21.079 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:54:21.180 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:54:21.210 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:21.231 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:54:21.233 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:21.234 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:54:21.335 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:54:21.365 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:21.385 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:54:21.388 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:21.388 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:54:21.489 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:54:21.520 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:21.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:20.357242+0000 mon.a (mon.0) 3511 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:54:21.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:20.357327+0000 mon.a (mon.0) 3512 : cluster [DBG] osdmap e721: 8 total, 8 up, 6 in 2024-02-23T00:54:21.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:20.358110+0000 mon.c (mon.2) 428 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-126"}]: dispatch 2024-02-23T00:54:21.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:20.358441+0000 mon.a (mon.0) 3513 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-126"}]: dispatch 2024-02-23T00:54:21.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:20.910221+0000 mon.c (mon.2) 429 : audit [DBG] from='client.? 172.21.15.47:0/3851244483' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:54:21.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:20.357242+0000 mon.a (mon.0) 3511 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:54:21.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:20.357327+0000 mon.a (mon.0) 3512 : cluster [DBG] osdmap e721: 8 total, 8 up, 6 in 2024-02-23T00:54:21.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:20.358110+0000 mon.c (mon.2) 428 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-126"}]: dispatch 2024-02-23T00:54:21.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:20.358441+0000 mon.a (mon.0) 3513 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-126"}]: dispatch 2024-02-23T00:54:21.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:20.910221+0000 mon.c (mon.2) 429 : audit [DBG] from='client.? 172.21.15.47:0/3851244483' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:54:21.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:20.357242+0000 mon.a (mon.0) 3511 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:54:21.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:20.357327+0000 mon.a (mon.0) 3512 : cluster [DBG] osdmap e721: 8 total, 8 up, 6 in 2024-02-23T00:54:21.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:20.358110+0000 mon.c (mon.2) 428 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-126"}]: dispatch 2024-02-23T00:54:21.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:20.358441+0000 mon.a (mon.0) 3513 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-126"}]: dispatch 2024-02-23T00:54:21.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:20.910221+0000 mon.c (mon.2) 429 : audit [DBG] from='client.? 172.21.15.47:0/3851244483' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:54:22.173 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:54:22.176 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:22.176 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:54:22.277 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:54:22.306 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:22.381 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:54:22.384 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:22.385 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:54:22.486 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:54:22.513 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:22.533 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:54:22.535 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:22.536 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:54:22.637 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:54:22.651 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:54:22.651 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-02-23T00:54:22.651 INFO:teuthology.orchestra.run.smithi047.stdout: "num_ops": 0 2024-02-23T00:54:22.651 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:54:22.664 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:21.358197+0000 mon.a (mon.0) 3514 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-126"}]': finished 2024-02-23T00:54:22.664 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:21.358258+0000 mon.a (mon.0) 3515 : cluster [DBG] osdmap e722: 8 total, 8 up, 6 in 2024-02-23T00:54:22.664 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:21.363653+0000 mon.c (mon.2) 430 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-126", "mode": "writeback"}]: dispatch 2024-02-23T00:54:22.664 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:21.364087+0000 mon.a (mon.0) 3516 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-126", "mode": "writeback"}]: dispatch 2024-02-23T00:54:22.664 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:21.383866+0000 mgr.y (mgr.14517) 617 : cluster [DBG] pgmap v1092: 184 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 151 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1/723 objects misplaced (0.138%) 2024-02-23T00:54:22.664 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:21.358197+0000 mon.a (mon.0) 3514 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-126"}]': finished 2024-02-23T00:54:22.665 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:21.358258+0000 mon.a (mon.0) 3515 : cluster [DBG] osdmap e722: 8 total, 8 up, 6 in 2024-02-23T00:54:22.665 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:21.363653+0000 mon.c (mon.2) 430 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-126", "mode": "writeback"}]: dispatch 2024-02-23T00:54:22.665 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:21.364087+0000 mon.a (mon.0) 3516 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-126", "mode": "writeback"}]: dispatch 2024-02-23T00:54:22.665 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:21.383866+0000 mgr.y (mgr.14517) 617 : cluster [DBG] pgmap v1092: 184 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 151 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1/723 objects misplaced (0.138%) 2024-02-23T00:54:22.666 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:22.694 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:54:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:54:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:54:22.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:21.358197+0000 mon.a (mon.0) 3514 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-126"}]': finished 2024-02-23T00:54:22.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:21.358258+0000 mon.a (mon.0) 3515 : cluster [DBG] osdmap e722: 8 total, 8 up, 6 in 2024-02-23T00:54:22.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:21.363653+0000 mon.c (mon.2) 430 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-126", "mode": "writeback"}]: dispatch 2024-02-23T00:54:22.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:21.364087+0000 mon.a (mon.0) 3516 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-126", "mode": "writeback"}]: dispatch 2024-02-23T00:54:22.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:21.383866+0000 mgr.y (mgr.14517) 617 : cluster [DBG] pgmap v1092: 184 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 151 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1/723 objects misplaced (0.138%) 2024-02-23T00:54:23.194 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:54:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:54:23.371 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:54:23.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:22.361371+0000 mon.a (mon.0) 3517 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:54:23.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:22.369130+0000 mon.a (mon.0) 3518 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-126", "mode": "writeback"}]': finished 2024-02-23T00:54:23.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:22.369198+0000 mon.a (mon.0) 3519 : cluster [DBG] osdmap e723: 8 total, 8 up, 6 in 2024-02-23T00:54:23.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:22.434742+0000 mon.c (mon.2) 431 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:23.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:22.435092+0000 mon.a (mon.0) 3520 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:23.701 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:22.361371+0000 mon.a (mon.0) 3517 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:54:23.701 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:22.369130+0000 mon.a (mon.0) 3518 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-126", "mode": "writeback"}]': finished 2024-02-23T00:54:23.701 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:22.369198+0000 mon.a (mon.0) 3519 : cluster [DBG] osdmap e723: 8 total, 8 up, 6 in 2024-02-23T00:54:23.701 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:22.434742+0000 mon.c (mon.2) 431 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:23.701 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:22.435092+0000 mon.a (mon.0) 3520 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:23.701 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:22.361371+0000 mon.a (mon.0) 3517 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:54:23.701 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:22.369130+0000 mon.a (mon.0) 3518 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-126", "mode": "writeback"}]': finished 2024-02-23T00:54:23.701 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:22.369198+0000 mon.a (mon.0) 3519 : cluster [DBG] osdmap e723: 8 total, 8 up, 6 in 2024-02-23T00:54:23.701 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:22.434742+0000 mon.c (mon.2) 431 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:23.701 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:22.435092+0000 mon.a (mon.0) 3520 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:23.717 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:54:23.721 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:23.721 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:54:23.822 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:54:23.854 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:23.875 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:54:23.878 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:23.878 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:54:23.979 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:54:24.015 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:24.171 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:54:24.173 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:24.173 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:54:24.179 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.1 dump_blocked_ops 2024-02-23T00:54:24.274 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:54:24.305 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:24.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:22.918642+0000 mgr.y (mgr.14517) 618 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:54:24.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:23.370913+0000 mon.a (mon.0) 3521 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:54:24.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:23.371006+0000 mon.a (mon.0) 3522 : cluster [DBG] osdmap e724: 8 total, 8 up, 6 in 2024-02-23T00:54:24.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:23.372975+0000 mon.c (mon.2) 432 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-126"}]: dispatch 2024-02-23T00:54:24.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:23.380340+0000 mon.a (mon.0) 3523 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-126"}]: dispatch 2024-02-23T00:54:24.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:23.384290+0000 mgr.y (mgr.14517) 619 : cluster [DBG] pgmap v1095: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/723 objects misplaced (0.138%) 2024-02-23T00:54:24.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:22.918642+0000 mgr.y (mgr.14517) 618 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:54:24.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:23.370913+0000 mon.a (mon.0) 3521 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:54:24.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:23.371006+0000 mon.a (mon.0) 3522 : cluster [DBG] osdmap e724: 8 total, 8 up, 6 in 2024-02-23T00:54:24.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:23.372975+0000 mon.c (mon.2) 432 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-126"}]: dispatch 2024-02-23T00:54:24.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:23.380340+0000 mon.a (mon.0) 3523 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-126"}]: dispatch 2024-02-23T00:54:24.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:23.384290+0000 mgr.y (mgr.14517) 619 : cluster [DBG] pgmap v1095: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/723 objects misplaced (0.138%) 2024-02-23T00:54:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:22.918642+0000 mgr.y (mgr.14517) 618 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:54:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:23.370913+0000 mon.a (mon.0) 3521 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:54:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:23.371006+0000 mon.a (mon.0) 3522 : cluster [DBG] osdmap e724: 8 total, 8 up, 6 in 2024-02-23T00:54:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:23.372975+0000 mon.c (mon.2) 432 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-126"}]: dispatch 2024-02-23T00:54:24.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:23.380340+0000 mon.a (mon.0) 3523 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-126"}]: dispatch 2024-02-23T00:54:24.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:23.384290+0000 mgr.y (mgr.14517) 619 : cluster [DBG] pgmap v1095: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/723 objects misplaced (0.138%) 2024-02-23T00:54:25.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.240+0000 7fb5417af700 1 -- 172.21.15.47:0/2876221403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb53c0f8420 msgr2=0x7fb53c104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:25.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.240+0000 7fb5417af700 1 --2- 172.21.15.47:0/2876221403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb53c0f8420 0x7fb53c104a20 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 rx=0x7fb534009be0 tx=0x7fb534004d10).stop 2024-02-23T00:54:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.240+0000 7fb5417af700 1 -- 172.21.15.47:0/2876221403 shutdown_connections 2024-02-23T00:54:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.240+0000 7fb5417af700 1 --2- 172.21.15.47:0/2876221403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb53c0f8420 0x7fb53c104a20 unknown :-1 s=CLOSED pgs=2626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.240+0000 7fb5417af700 1 --2- 172.21.15.47:0/2876221403 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb53c0fdbb0 0x7fb53c0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.240+0000 7fb5417af700 1 --2- 172.21.15.47:0/2876221403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb53c0fe510 0x7fb53c0f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.240+0000 7fb5417af700 1 -- 172.21.15.47:0/2876221403 >> 172.21.15.47:0/2876221403 conn(0x7fb53c0f3800 msgr2=0x7fb53c0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.240+0000 7fb5417af700 1 -- 172.21.15.47:0/2876221403 shutdown_connections 2024-02-23T00:54:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.240+0000 7fb5417af700 1 -- 172.21.15.47:0/2876221403 wait complete. 2024-02-23T00:54:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.241+0000 7fb5417af700 1 librados: starting msgr at 2024-02-23T00:54:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.241+0000 7fb5417af700 1 librados: starting objecter 2024-02-23T00:54:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.241+0000 7fb5417af700 1 Processor -- start 2024-02-23T00:54:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.241+0000 7fb5417af700 1 -- start start 2024-02-23T00:54:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.241+0000 7fb5417af700 1 librados: setting wanted keys 2024-02-23T00:54:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.241+0000 7fb5417af700 1 librados: calling monclient init 2024-02-23T00:54:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.241+0000 7fb5417af700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb53c0f8420 0x7fb53c10dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.242+0000 7fb5417af700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb53c0fdbb0 0x7fb53c10e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.242+0000 7fb5417af700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb53c0fe510 0x7fb53c107df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.242+0000 7fb5417af700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c1082e0 con 0x7fb53c0f8420 2024-02-23T00:54:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.242+0000 7fb5417af700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c108450 con 0x7fb53c0fe510 2024-02-23T00:54:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.242+0000 7fb5417af700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c108610 con 0x7fb53c0fdbb0 2024-02-23T00:54:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.242+0000 7fb53affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb53c0f8420 0x7fb53c10dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.242+0000 7fb53a7fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb53c0fdbb0 0x7fb53c10e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.242+0000 7fb53affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb53c0f8420 0x7fb53c10dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51736/0 (socket says 172.21.15.47:51736) 2024-02-23T00:54:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.242+0000 7fb53affd700 1 -- 172.21.15.47:0/825564822 learned_addr learned my addr 172.21.15.47:0/825564822 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:54:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.242+0000 7fb53b7fe700 1 --2- 172.21.15.47:0/825564822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb53c0fe510 0x7fb53c107df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.242+0000 7fb53a7fc700 1 -- 172.21.15.47:0/825564822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb53c0fe510 msgr2=0x7fb53c107df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.242+0000 7fb53a7fc700 1 --2- 172.21.15.47:0/825564822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb53c0fe510 0x7fb53c107df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.242+0000 7fb53a7fc700 1 -- 172.21.15.47:0/825564822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb53c0f8420 msgr2=0x7fb53c10dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.242+0000 7fb53a7fc700 1 --2- 172.21.15.47:0/825564822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb53c0f8420 0x7fb53c10dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.242+0000 7fb53a7fc700 1 -- 172.21.15.47:0/825564822 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52c005040 con 0x7fb53c0fdbb0 2024-02-23T00:54:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.242+0000 7fb53affd700 1 --2- 172.21.15.47:0/825564822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb53c0f8420 0x7fb53c10dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:54:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.242+0000 7fb53a7fc700 1 --2- 172.21.15.47:0/825564822 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb53c0fdbb0 0x7fb53c10e140 secure :-1 s=READY pgs=2790 cs=0 l=1 rev1=1 rx=0x7fb530002780 tx=0x7fb530009e40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.242+0000 7fb523fff700 10 client.?.objecter ms_handle_connect 0x7fb53c0fdbb0 2024-02-23T00:54:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.242+0000 7fb523fff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:54:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.242+0000 7fb523fff700 1 -- 172.21.15.47:0/825564822 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb530007b40 con 0x7fb53c0fdbb0 2024-02-23T00:54:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.242+0000 7fb5417af700 1 -- 172.21.15.47:0/825564822 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5340097c0 con 0x7fb53c0fdbb0 2024-02-23T00:54:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.242+0000 7fb523fff700 1 -- 172.21.15.47:0/825564822 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fb53000f040 con 0x7fb53c0fdbb0 2024-02-23T00:54:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.242+0000 7fb523fff700 1 -- 172.21.15.47:0/825564822 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb530014690 con 0x7fb53c0fdbb0 2024-02-23T00:54:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.242+0000 7fb5417af700 10 client.42008.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:54:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.242+0000 7fb5417af700 1 -- 172.21.15.47:0/825564822 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb53c108c50 con 0x7fb53c0fdbb0 2024-02-23T00:54:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.242+0000 7fb5417af700 1 librados: init done 2024-02-23T00:54:25.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.243+0000 7fb5417af700 1 -- 172.21.15.47:0/825564822 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb53c05ee80 con 0x7fb53c0fdbb0 2024-02-23T00:54:25.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.244+0000 7fb523fff700 1 -- 172.21.15.47:0/825564822 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fb530007ca0 con 0x7fb53c0fdbb0 2024-02-23T00:54:25.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.244+0000 7fb523fff700 1 --2- 172.21.15.47:0/825564822 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb524062cc0 0x7fb524065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:25.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.244+0000 7fb523fff700 1 -- 172.21.15.47:0/825564822 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(725..725 src has 621..725) v4 ==== 9744+0+0 (secure 0 0 0) 0x7fb53007d880 con 0x7fb53c0fdbb0 2024-02-23T00:54:25.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.244+0000 7fb523fff700 10 client.42008.objecter ms_dispatch 0x7fb53c001550 osd_map(725..725 src has 621..725) v4 2024-02-23T00:54:25.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.244+0000 7fb523fff700 3 client.42008.objecter handle_osd_map got epochs [725,725] > 0 2024-02-23T00:54:25.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.244+0000 7fb523fff700 3 client.42008.objecter handle_osd_map decoding full epoch 725 2024-02-23T00:54:25.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.244+0000 7fb53affd700 1 --2- 172.21.15.47:0/825564822 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb524062cc0 0x7fb524065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:25.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.244+0000 7fb523fff700 20 client.42008.objecter dump_active .. 0 homeless 2024-02-23T00:54:25.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.244+0000 7fb523fff700 10 client.42008.objecter ms_handle_connect 0x7fb524062cc0 2024-02-23T00:54:25.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.244+0000 7fb53affd700 1 --2- 172.21.15.47:0/825564822 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb524062cc0 0x7fb524065180 secure :-1 s=READY pgs=3178 cs=0 l=1 rev1=1 rx=0x7fb52c009810 tx=0x7fb52c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:25.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.248+0000 7fb523fff700 1 -- 172.21.15.47:0/825564822 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb53004cc90 con 0x7fb53c0fdbb0 2024-02-23T00:54:25.355 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:54:25.359 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:25.360 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:54:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:25.448+0000 7fb5417af700 1 -- 172.21.15.47:0/825564822 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd reweight", "id": 7, "weight": 1.0} v 0) v1 -- 0x7fb53c05fa60 con 0x7fb53c0fdbb0 2024-02-23T00:54:25.460 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:54:25.489 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:25.724 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:54:25.738 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:54:25.741 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:25.741 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:54:25.842 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:54:25.873 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:25.979 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:54:25.983 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:25.983 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:54:26.013 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:24.374156+0000 mon.a (mon.0) 3524 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:54:26.013 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:24.382019+0000 mon.a (mon.0) 3525 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-126"}]': finished 2024-02-23T00:54:26.013 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:24.382100+0000 mon.a (mon.0) 3526 : cluster [DBG] osdmap e725: 8 total, 8 up, 6 in 2024-02-23T00:54:26.013 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:25.296507+0000 mon.a (mon.0) 3527 : cluster [DBG] osdmap e726: 8 total, 8 up, 6 in 2024-02-23T00:54:26.014 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:24.374156+0000 mon.a (mon.0) 3524 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:54:26.014 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:24.382019+0000 mon.a (mon.0) 3525 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-126"}]': finished 2024-02-23T00:54:26.014 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:24.382100+0000 mon.a (mon.0) 3526 : cluster [DBG] osdmap e725: 8 total, 8 up, 6 in 2024-02-23T00:54:26.014 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:25.296507+0000 mon.a (mon.0) 3527 : cluster [DBG] osdmap e726: 8 total, 8 up, 6 in 2024-02-23T00:54:26.084 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:54:26.114 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:24.374156+0000 mon.a (mon.0) 3524 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:54:26.115 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:24.382019+0000 mon.a (mon.0) 3525 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-126"}]': finished 2024-02-23T00:54:26.115 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:24.382100+0000 mon.a (mon.0) 3526 : cluster [DBG] osdmap e725: 8 total, 8 up, 6 in 2024-02-23T00:54:26.115 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:25.296507+0000 mon.a (mon.0) 3527 : cluster [DBG] osdmap e726: 8 total, 8 up, 6 in 2024-02-23T00:54:26.117 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:26.138 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:54:26.141 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:26.142 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:54:26.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.204+0000 7f4ee7cf1700 1 -- 172.21.15.47:0/1704920319 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4ee00fdbb0 msgr2=0x7f4ee00fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:26.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.204+0000 7f4ee7cf1700 1 --2- 172.21.15.47:0/1704920319 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4ee00fdbb0 0x7f4ee00fdf90 secure :-1 s=READY pgs=2791 cs=0 l=1 rev1=1 rx=0x7f4edc004870 tx=0x7f4edc009cb0).stop 2024-02-23T00:54:26.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.204+0000 7f4ee7cf1700 1 -- 172.21.15.47:0/1704920319 shutdown_connections 2024-02-23T00:54:26.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.204+0000 7f4ee7cf1700 1 --2- 172.21.15.47:0/1704920319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ee00f8420 0x7f4ee0104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:26.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.204+0000 7f4ee7cf1700 1 --2- 172.21.15.47:0/1704920319 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4ee00fdbb0 0x7f4ee00fdf90 unknown :-1 s=CLOSED pgs=2791 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:26.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.204+0000 7f4ee7cf1700 1 --2- 172.21.15.47:0/1704920319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ee00fe510 0x7f4ee00f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:26.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.204+0000 7f4ee7cf1700 1 -- 172.21.15.47:0/1704920319 >> 172.21.15.47:0/1704920319 conn(0x7f4ee00f3800 msgr2=0x7f4ee00f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:26.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.204+0000 7f4ee7cf1700 1 -- 172.21.15.47:0/1704920319 shutdown_connections 2024-02-23T00:54:26.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.204+0000 7f4ee7cf1700 1 -- 172.21.15.47:0/1704920319 wait complete. 2024-02-23T00:54:26.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.205+0000 7f4ee7cf1700 1 librados: starting msgr at 2024-02-23T00:54:26.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.205+0000 7f4ee7cf1700 1 librados: starting objecter 2024-02-23T00:54:26.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.205+0000 7f4ee7cf1700 1 Processor -- start 2024-02-23T00:54:26.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.205+0000 7f4ee7cf1700 1 -- start start 2024-02-23T00:54:26.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.205+0000 7f4ee7cf1700 1 librados: setting wanted keys 2024-02-23T00:54:26.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.205+0000 7f4ee7cf1700 1 librados: calling monclient init 2024-02-23T00:54:26.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.205+0000 7f4ee7cf1700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4ee00f8420 0x7f4ee010deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:26.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.205+0000 7f4ee7cf1700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ee00fdbb0 0x7f4ee010e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:26.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.205+0000 7f4ee7cf1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ee00fe510 0x7f4ee0108050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:26.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.205+0000 7f4ee7cf1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee0108540 con 0x7f4ee00fe510 2024-02-23T00:54:26.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.205+0000 7f4ee7cf1700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee01086b0 con 0x7f4ee00fdbb0 2024-02-23T00:54:26.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.205+0000 7f4ee7cf1700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee0108870 con 0x7f4ee00f8420 2024-02-23T00:54:26.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.205+0000 7f4ee5a8d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4ee00f8420 0x7f4ee010deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:26.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.205+0000 7f4ee5a8d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4ee00f8420 0x7f4ee010deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:54640/0 (socket says 172.21.15.47:54640) 2024-02-23T00:54:26.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.206+0000 7f4ee5a8d700 1 -- 172.21.15.47:0/1381251367 learned_addr learned my addr 172.21.15.47:0/1381251367 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:54:26.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.206+0000 7f4ee628e700 1 --2- 172.21.15.47:0/1381251367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ee00fe510 0x7f4ee0108050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:26.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.206+0000 7f4ee528c700 1 --2- 172.21.15.47:0/1381251367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ee00fdbb0 0x7f4ee010e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:26.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.206+0000 7f4ee5a8d700 1 -- 172.21.15.47:0/1381251367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ee00fdbb0 msgr2=0x7f4ee010e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:26.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.206+0000 7f4ee5a8d700 1 --2- 172.21.15.47:0/1381251367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ee00fdbb0 0x7f4ee010e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:26.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.206+0000 7f4ee5a8d700 1 -- 172.21.15.47:0/1381251367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ee00fe510 msgr2=0x7f4ee0108050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:26.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.206+0000 7f4ee5a8d700 1 --2- 172.21.15.47:0/1381251367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ee00fe510 0x7f4ee0108050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:26.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.206+0000 7f4ee5a8d700 1 -- 172.21.15.47:0/1381251367 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4edc005040 con 0x7f4ee00f8420 2024-02-23T00:54:26.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.206+0000 7f4ee528c700 1 --2- 172.21.15.47:0/1381251367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ee00fdbb0 0x7f4ee010e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:54:26.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.206+0000 7f4ee628e700 1 --2- 172.21.15.47:0/1381251367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ee00fe510 0x7f4ee0108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:54:26.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.206+0000 7f4ee5a8d700 1 --2- 172.21.15.47:0/1381251367 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4ee00f8420 0x7f4ee010deb0 secure :-1 s=READY pgs=2792 cs=0 l=1 rev1=1 rx=0x7f4edc000e50 tx=0x7f4edc009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:26.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.206+0000 7f4ed6ffd700 10 client.?.objecter ms_handle_connect 0x7f4ee00f8420 2024-02-23T00:54:26.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.206+0000 7f4ed6ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:54:26.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.206+0000 7f4ed6ffd700 1 -- 172.21.15.47:0/1381251367 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc00e030 con 0x7f4ee00f8420 2024-02-23T00:54:26.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.206+0000 7f4ed6ffd700 1 -- 172.21.15.47:0/1381251367 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f4edc013df0 con 0x7f4ee00f8420 2024-02-23T00:54:26.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.206+0000 7f4ed6ffd700 1 -- 172.21.15.47:0/1381251367 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc018070 con 0x7f4ee00f8420 2024-02-23T00:54:26.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.206+0000 7f4ee7cf1700 1 -- 172.21.15.47:0/1381251367 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee0108af0 con 0x7f4ee00f8420 2024-02-23T00:54:26.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.206+0000 7f4ee7cf1700 10 client.42014.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:54:26.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.206+0000 7f4ee7cf1700 1 -- 172.21.15.47:0/1381251367 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee0114ce0 con 0x7f4ee00f8420 2024-02-23T00:54:26.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.206+0000 7f4ee7cf1700 1 librados: init done 2024-02-23T00:54:26.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.207+0000 7f4ee7cf1700 1 -- 172.21.15.47:0/1381251367 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ee005ee80 con 0x7f4ee00f8420 2024-02-23T00:54:26.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.208+0000 7f4ed6ffd700 1 -- 172.21.15.47:0/1381251367 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f4edc009240 con 0x7f4ee00f8420 2024-02-23T00:54:26.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.209+0000 7f4ed6ffd700 1 --2- 172.21.15.47:0/1381251367 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4ecc062cc0 0x7f4ecc065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:26.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.209+0000 7f4ed6ffd700 1 -- 172.21.15.47:0/1381251367 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(726..726 src has 621..726) v4 ==== 9371+0+0 (secure 0 0 0) 0x7f4edc07e820 con 0x7f4ee00f8420 2024-02-23T00:54:26.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.209+0000 7f4ed6ffd700 10 client.42014.objecter ms_dispatch 0x7f4ee0001550 osd_map(726..726 src has 621..726) v4 2024-02-23T00:54:26.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.209+0000 7f4ed6ffd700 3 client.42014.objecter handle_osd_map got epochs [726,726] > 0 2024-02-23T00:54:26.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.209+0000 7f4ed6ffd700 3 client.42014.objecter handle_osd_map decoding full epoch 726 2024-02-23T00:54:26.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.209+0000 7f4ed6ffd700 20 client.42014.objecter dump_active .. 0 homeless 2024-02-23T00:54:26.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.209+0000 7f4ee528c700 1 --2- 172.21.15.47:0/1381251367 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4ecc062cc0 0x7f4ecc065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:26.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.210+0000 7f4ed6ffd700 10 client.42014.objecter ms_handle_connect 0x7f4ecc062cc0 2024-02-23T00:54:26.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.210+0000 7f4ee528c700 1 --2- 172.21.15.47:0/1381251367 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4ecc062cc0 0x7f4ecc065180 secure :-1 s=READY pgs=3179 cs=0 l=1 rev1=1 rx=0x7f4ed0007fd0 tx=0x7f4ed000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:26.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.215+0000 7f4ed6ffd700 1 -- 172.21.15.47:0/1381251367 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f4edc04ded0 con 0x7f4ee00f8420 2024-02-23T00:54:26.242 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:54:26.249 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:54:26.255 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:54:26.271 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:26.292 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:54:26.294 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:26.295 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:54:26.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.308+0000 7fb523fff700 1 -- 172.21.15.47:0/825564822 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 7, "weight": 1.0}]=0 reweighted osd.7 to 1 (10000) v727) v1 ==== 113+0+0 (secure 0 0 0) 0x7fb5300502b0 con 0x7fb53c0fdbb0 2024-02-23T00:54:26.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.311+0000 7fb5417af700 10 librados: watch_flush enter 2024-02-23T00:54:26.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.311+0000 7fb5417af700 10 librados: watch_flush exit 2024-02-23T00:54:26.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.311+0000 7fb5417af700 20 client.42008.objecter shutdown clearing up homeless session... 2024-02-23T00:54:26.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.311+0000 7fb5417af700 10 client.42008.objecter successfully canceled tick 2024-02-23T00:54:26.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.312+0000 7fb5417af700 1 -- 172.21.15.47:0/825564822 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb524062cc0 msgr2=0x7fb524065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:26.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.312+0000 7fb5417af700 1 --2- 172.21.15.47:0/825564822 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb524062cc0 0x7fb524065180 secure :-1 s=READY pgs=3178 cs=0 l=1 rev1=1 rx=0x7fb52c009810 tx=0x7fb52c00b040).stop 2024-02-23T00:54:26.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.312+0000 7fb5417af700 1 -- 172.21.15.47:0/825564822 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb53c0fdbb0 msgr2=0x7fb53c10e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:26.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.312+0000 7fb5417af700 1 --2- 172.21.15.47:0/825564822 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb53c0fdbb0 0x7fb53c10e140 secure :-1 s=READY pgs=2790 cs=0 l=1 rev1=1 rx=0x7fb530002780 tx=0x7fb530009e40).stop 2024-02-23T00:54:26.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.314+0000 7fb5417af700 1 -- 172.21.15.47:0/825564822 shutdown_connections 2024-02-23T00:54:26.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.314+0000 7fb5417af700 1 --2- 172.21.15.47:0/825564822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb53c0fe510 0x7fb53c107df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:26.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.314+0000 7fb5417af700 1 --2- 172.21.15.47:0/825564822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb53c0f8420 0x7fb53c10dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:26.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.314+0000 7fb5417af700 1 --2- 172.21.15.47:0/825564822 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb524062cc0 0x7fb524065180 unknown :-1 s=CLOSED pgs=3178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:26.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.314+0000 7fb5417af700 1 --2- 172.21.15.47:0/825564822 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb53c0fdbb0 0x7fb53c10e140 unknown :-1 s=CLOSED pgs=2790 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:26.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.314+0000 7fb5417af700 1 -- 172.21.15.47:0/825564822 >> 172.21.15.47:0/825564822 conn(0x7fb53c0f3800 msgr2=0x7fb53c0f50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:26.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.314+0000 7fb5417af700 1 -- 172.21.15.47:0/825564822 shutdown_connections 2024-02-23T00:54:26.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.314+0000 7fb5417af700 1 -- 172.21.15.47:0/825564822 wait complete. 2024-02-23T00:54:26.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.314+0000 7fb5417af700 1 librados: shutdown 2024-02-23T00:54:26.318 INFO:teuthology.orchestra.run.smithi047.stderr:reweighted osd.7 to 1 (10000) 2024-02-23T00:54:26.396 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:54:26.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:26.422+0000 7f4ee7cf1700 1 -- 172.21.15.47:0/1381251367 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7f4ee0109650 con 0x7f4ee00f8420 2024-02-23T00:54:26.425 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:27.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:25.384911+0000 mgr.y (mgr.14517) 620 : cluster [DBG] pgmap v1098: 152 pgs: 1 active+remapped+backfill_toofull, 151 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/723 objects misplaced (0.138%) 2024-02-23T00:54:27.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:25.449911+0000 mon.c (mon.2) 433 : audit [INF] from='client.? 172.21.15.47:0/825564822' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-23T00:54:27.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:25.752870+0000 mon.a (mon.0) 3528 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-23T00:54:27.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:26.308728+0000 mon.a (mon.0) 3529 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-02-23T00:54:27.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:26.308811+0000 mon.a (mon.0) 3530 : cluster [DBG] osdmap e727: 8 total, 8 up, 6 in 2024-02-23T00:54:27.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:26.310809+0000 mon.c (mon.2) 434 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:54:27.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:26.311217+0000 mon.a (mon.0) 3531 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:54:27.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:26.316780+0000 mon.a (mon.0) 3532 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:54:27.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:26.423651+0000 mon.c (mon.2) 435 : audit [INF] from='client.? 172.21.15.47:0/1381251367' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:54:27.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:26.424004+0000 mon.a (mon.0) 3533 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:54:27.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:25.384911+0000 mgr.y (mgr.14517) 620 : cluster [DBG] pgmap v1098: 152 pgs: 1 active+remapped+backfill_toofull, 151 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/723 objects misplaced (0.138%) 2024-02-23T00:54:27.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:25.449911+0000 mon.c (mon.2) 433 : audit [INF] from='client.? 172.21.15.47:0/825564822' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-23T00:54:27.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:25.752870+0000 mon.a (mon.0) 3528 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-23T00:54:27.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:26.308728+0000 mon.a (mon.0) 3529 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-02-23T00:54:27.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:26.308811+0000 mon.a (mon.0) 3530 : cluster [DBG] osdmap e727: 8 total, 8 up, 6 in 2024-02-23T00:54:27.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:26.310809+0000 mon.c (mon.2) 434 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:54:27.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:26.311217+0000 mon.a (mon.0) 3531 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:54:27.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:26.316780+0000 mon.a (mon.0) 3532 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:54:27.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:26.423651+0000 mon.c (mon.2) 435 : audit [INF] from='client.? 172.21.15.47:0/1381251367' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:54:27.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:26.424004+0000 mon.a (mon.0) 3533 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:54:27.075 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:54:27.078 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:27.078 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:54:27.179 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:54:27.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:25.384911+0000 mgr.y (mgr.14517) 620 : cluster [DBG] pgmap v1098: 152 pgs: 1 active+remapped+backfill_toofull, 151 active+clean; 5.7 KiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/723 objects misplaced (0.138%) 2024-02-23T00:54:27.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:25.449911+0000 mon.c (mon.2) 433 : audit [INF] from='client.? 172.21.15.47:0/825564822' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-23T00:54:27.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:25.752870+0000 mon.a (mon.0) 3528 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-23T00:54:27.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:26.308728+0000 mon.a (mon.0) 3529 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-02-23T00:54:27.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:26.308811+0000 mon.a (mon.0) 3530 : cluster [DBG] osdmap e727: 8 total, 8 up, 6 in 2024-02-23T00:54:27.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:26.310809+0000 mon.c (mon.2) 434 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:54:27.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:26.311217+0000 mon.a (mon.0) 3531 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:54:27.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:26.316780+0000 mon.a (mon.0) 3532 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:54:27.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:26.423651+0000 mon.c (mon.2) 435 : audit [INF] from='client.? 172.21.15.47:0/1381251367' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:54:27.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:26.424004+0000 mon.a (mon.0) 3533 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:54:27.211 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:27.233 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:54:27.235 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:27.236 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:54:27.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:27.310+0000 7f4ed6ffd700 1 -- 172.21.15.47:0/1381251367 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v728) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4ee0109650 con 0x7f4ee00f8420 2024-02-23T00:54:27.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:27.314+0000 7f4ee7cf1700 10 librados: watch_flush enter 2024-02-23T00:54:27.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:27.316+0000 7f4ee7cf1700 10 librados: watch_flush exit 2024-02-23T00:54:27.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:27.316+0000 7f4ee7cf1700 20 client.42014.objecter shutdown clearing up homeless session... 2024-02-23T00:54:27.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:27.316+0000 7f4ee7cf1700 10 client.42014.objecter successfully canceled tick 2024-02-23T00:54:27.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:27.316+0000 7f4ee7cf1700 1 -- 172.21.15.47:0/1381251367 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4ecc062cc0 msgr2=0x7f4ecc065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:27.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:27.316+0000 7f4ee7cf1700 1 --2- 172.21.15.47:0/1381251367 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4ecc062cc0 0x7f4ecc065180 secure :-1 s=READY pgs=3179 cs=0 l=1 rev1=1 rx=0x7f4ed0007fd0 tx=0x7f4ed000b040).stop 2024-02-23T00:54:27.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:27.316+0000 7f4ee7cf1700 1 -- 172.21.15.47:0/1381251367 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4ee00f8420 msgr2=0x7f4ee010deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:27.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:27.316+0000 7f4ee7cf1700 1 --2- 172.21.15.47:0/1381251367 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4ee00f8420 0x7f4ee010deb0 secure :-1 s=READY pgs=2792 cs=0 l=1 rev1=1 rx=0x7f4edc000e50 tx=0x7f4edc009cb0).stop 2024-02-23T00:54:27.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:27.318+0000 7f4ee7cf1700 1 -- 172.21.15.47:0/1381251367 shutdown_connections 2024-02-23T00:54:27.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:27.318+0000 7f4ee7cf1700 1 --2- 172.21.15.47:0/1381251367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ee00fe510 0x7f4ee0108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:27.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:27.318+0000 7f4ee7cf1700 1 --2- 172.21.15.47:0/1381251367 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4ecc062cc0 0x7f4ecc065180 unknown :-1 s=CLOSED pgs=3179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:27.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:27.318+0000 7f4ee7cf1700 1 --2- 172.21.15.47:0/1381251367 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4ee00f8420 0x7f4ee010deb0 unknown :-1 s=CLOSED pgs=2792 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:27.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:27.318+0000 7f4ee7cf1700 1 --2- 172.21.15.47:0/1381251367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ee00fdbb0 0x7f4ee010e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:27.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:27.318+0000 7f4ee7cf1700 1 -- 172.21.15.47:0/1381251367 >> 172.21.15.47:0/1381251367 conn(0x7f4ee00f3800 msgr2=0x7f4ee00fb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:27.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:27.318+0000 7f4ee7cf1700 1 -- 172.21.15.47:0/1381251367 shutdown_connections 2024-02-23T00:54:27.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:27.318+0000 7f4ee7cf1700 1 -- 172.21.15.47:0/1381251367 wait complete. 2024-02-23T00:54:27.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:27.318+0000 7f4ee7cf1700 1 librados: shutdown 2024-02-23T00:54:27.322 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is set 2024-02-23T00:54:27.337 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:54:27.366 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:27.615 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:54:27.618 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:27.618 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:54:27.719 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:54:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:27.820 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:54:27.822 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:27.823 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:54:27.924 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:54:27.950 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:27.306557+0000 mon.a (mon.0) 3534 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:54:27.950 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:27.311041+0000 mon.a (mon.0) 3535 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:54:27.950 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:27.311244+0000 mon.a (mon.0) 3536 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:54:27.950 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:27.311331+0000 mon.a (mon.0) 3537 : cluster [DBG] osdmap e728: 8 total, 8 up, 6 in 2024-02-23T00:54:27.950 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:27.312780+0000 mon.c (mon.2) 436 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:54:27.950 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:27.313961+0000 mon.a (mon.0) 3538 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:54:27.950 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:27.385328+0000 mgr.y (mgr.14517) 621 : cluster [DBG] pgmap v1101: 184 pgs: 1 active+remapped+backfill_toofull, 32 creating+peering, 151 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s; 1/726 objects misplaced (0.138%) 2024-02-23T00:54:27.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:27.306557+0000 mon.a (mon.0) 3534 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:54:27.951 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:27.311041+0000 mon.a (mon.0) 3535 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:54:27.951 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:27.311244+0000 mon.a (mon.0) 3536 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:54:27.951 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:27.311331+0000 mon.a (mon.0) 3537 : cluster [DBG] osdmap e728: 8 total, 8 up, 6 in 2024-02-23T00:54:27.951 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:27.312780+0000 mon.c (mon.2) 436 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:54:27.951 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:27.313961+0000 mon.a (mon.0) 3538 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:54:27.951 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:27.385328+0000 mgr.y (mgr.14517) 621 : cluster [DBG] pgmap v1101: 184 pgs: 1 active+remapped+backfill_toofull, 32 creating+peering, 151 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s; 1/726 objects misplaced (0.138%) 2024-02-23T00:54:27.952 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:28.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:27.306557+0000 mon.a (mon.0) 3534 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:54:28.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:27.311041+0000 mon.a (mon.0) 3535 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:54:28.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:27.311244+0000 mon.a (mon.0) 3536 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:54:28.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:27.311331+0000 mon.a (mon.0) 3537 : cluster [DBG] osdmap e728: 8 total, 8 up, 6 in 2024-02-23T00:54:28.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:27.312780+0000 mon.c (mon.2) 436 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:54:28.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:27.313961+0000 mon.a (mon.0) 3538 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:54:28.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:27.385328+0000 mgr.y (mgr.14517) 621 : cluster [DBG] pgmap v1101: 184 pgs: 1 active+remapped+backfill_toofull, 32 creating+peering, 151 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s; 1/726 objects misplaced (0.138%) 2024-02-23T00:54:28.401 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:54:28.405 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:28.405 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:54:28.506 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:54:28.538 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:28.559 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:54:28.561 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:28.562 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:54:28.662 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:54:28.693 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:28.875 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:54:28.875 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-02-23T00:54:28.875 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-02-23T00:54:28.875 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-02-23T00:54:28.875 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:54:28.892 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:54:28.895 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:28.895 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:54:28.996 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:54:29.024 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:29.098 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:54:29.100 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:29.101 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:54:29.202 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:54:29.230 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:29.576 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:28.315292+0000 mon.a (mon.0) 3539 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:54:29.576 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:28.315377+0000 mon.a (mon.0) 3540 : cluster [DBG] osdmap e729: 8 total, 8 up, 6 in 2024-02-23T00:54:29.576 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:28.316582+0000 mon.c (mon.2) 437 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-128","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:54:29.576 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:28.316936+0000 mon.a (mon.0) 3541 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-128","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:54:29.577 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:28.315292+0000 mon.a (mon.0) 3539 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:54:29.577 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:28.315377+0000 mon.a (mon.0) 3540 : cluster [DBG] osdmap e729: 8 total, 8 up, 6 in 2024-02-23T00:54:29.577 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:28.316582+0000 mon.c (mon.2) 437 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-128","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:54:29.577 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:28.316936+0000 mon.a (mon.0) 3541 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-128","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:54:29.579 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:54:29.582 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:29.582 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:54:29.683 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:54:29.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:28.315292+0000 mon.a (mon.0) 3539 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:54:29.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:28.315377+0000 mon.a (mon.0) 3540 : cluster [DBG] osdmap e729: 8 total, 8 up, 6 in 2024-02-23T00:54:29.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:28.316582+0000 mon.c (mon.2) 437 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-128","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:54:29.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:28.316936+0000 mon.a (mon.0) 3541 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-128","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:54:29.715 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:29.737 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:54:29.739 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:29.740 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:54:29.841 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:54:29.871 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:29.892 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:54:29.894 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:29.895 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:54:29.996 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:54:30.026 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:30.080 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:54:30.091 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:30.091 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:54:30.192 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:54:30.224 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:30.245 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:54:30.248 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:30.248 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:54:30.349 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:54:30.379 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:30.399 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:54:30.402 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:30.403 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:54:30.504 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:54:30.534 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:30.573 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:54:30.588 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:54:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:54:30] "GET /metrics HTTP/1.1" 200 191529 "" "Prometheus/2.33.4" 2024-02-23T00:54:30.588 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:29.320235+0000 mon.a (mon.0) 3542 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-128","var": "hit_set_count","val": "2"}]': finished 2024-02-23T00:54:30.588 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:29.320308+0000 mon.a (mon.0) 3543 : cluster [DBG] osdmap e730: 8 total, 8 up, 6 in 2024-02-23T00:54:30.589 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:29.320845+0000 mon.c (mon.2) 438 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-128","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:54:30.589 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:29.321530+0000 mon.a (mon.0) 3544 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-128","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:54:30.589 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:29.385842+0000 mgr.y (mgr.14517) 622 : cluster [DBG] pgmap v1104: 184 pgs: 1 active+remapped+backfill_toofull, 32 creating+peering, 151 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s; 1/726 objects misplaced (0.138%) 2024-02-23T00:54:30.589 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:29.868646+0000 osd.5 (osd.5) 316 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:54:30.589 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:29.868819+0000 osd.5 (osd.5) 317 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:54:30.589 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:29.320235+0000 mon.a (mon.0) 3542 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-128","var": "hit_set_count","val": "2"}]': finished 2024-02-23T00:54:30.589 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:29.320308+0000 mon.a (mon.0) 3543 : cluster [DBG] osdmap e730: 8 total, 8 up, 6 in 2024-02-23T00:54:30.589 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:29.320845+0000 mon.c (mon.2) 438 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-128","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:54:30.589 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:29.321530+0000 mon.a (mon.0) 3544 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-128","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:54:30.589 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:29.385842+0000 mgr.y (mgr.14517) 622 : cluster [DBG] pgmap v1104: 184 pgs: 1 active+remapped+backfill_toofull, 32 creating+peering, 151 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s; 1/726 objects misplaced (0.138%) 2024-02-23T00:54:30.589 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:29.868646+0000 osd.5 (osd.5) 316 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:54:30.590 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:29.868819+0000 osd.5 (osd.5) 317 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:54:30.590 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:30.590 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:54:30.691 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:54:30.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:29.320235+0000 mon.a (mon.0) 3542 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-128","var": "hit_set_count","val": "2"}]': finished 2024-02-23T00:54:30.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:29.320308+0000 mon.a (mon.0) 3543 : cluster [DBG] osdmap e730: 8 total, 8 up, 6 in 2024-02-23T00:54:30.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:29.320845+0000 mon.c (mon.2) 438 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-128","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:54:30.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:29.321530+0000 mon.a (mon.0) 3544 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-128","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:54:30.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:29.385842+0000 mgr.y (mgr.14517) 622 : cluster [DBG] pgmap v1104: 184 pgs: 1 active+remapped+backfill_toofull, 32 creating+peering, 151 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s; 1/726 objects misplaced (0.138%) 2024-02-23T00:54:30.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:29.868646+0000 osd.5 (osd.5) 316 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:54:30.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:29.868819+0000 osd.5 (osd.5) 317 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:54:30.723 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:30.745 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:54:30.748 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:30.748 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:54:30.849 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:54:30.878 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:30.897 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.1 dump_historic_ops 2024-02-23T00:54:30.898 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:54:30.900 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:30.900 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:54:31.001 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:54:31.030 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:31.086 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:54:31.090 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:31.090 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:54:31.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.088+0000 7fd274e72700 1 -- 172.21.15.47:0/1950158195 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2700fcba0 msgr2=0x7fd2700f6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:31.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.088+0000 7fd274e72700 1 --2- 172.21.15.47:0/1950158195 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2700fcba0 0x7fd2700f6450 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 rx=0x7fd264004970 tx=0x7fd26400adc0).stop 2024-02-23T00:54:31.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.089+0000 7fd274e72700 1 -- 172.21.15.47:0/1950158195 shutdown_connections 2024-02-23T00:54:31.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.089+0000 7fd274e72700 1 --2- 172.21.15.47:0/1950158195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2700f6b10 0x7fd2701030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:31.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.089+0000 7fd274e72700 1 --2- 172.21.15.47:0/1950158195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2700fc240 0x7fd2700fc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:31.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.089+0000 7fd274e72700 1 --2- 172.21.15.47:0/1950158195 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2700fcba0 0x7fd2700f6450 unknown :-1 s=CLOSED pgs=2793 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:31.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.089+0000 7fd274e72700 1 -- 172.21.15.47:0/1950158195 >> 172.21.15.47:0/1950158195 conn(0x7fd27006c370 msgr2=0x7fd27006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:31.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.089+0000 7fd274e72700 1 -- 172.21.15.47:0/1950158195 shutdown_connections 2024-02-23T00:54:31.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.089+0000 7fd274e72700 1 -- 172.21.15.47:0/1950158195 wait complete. 2024-02-23T00:54:31.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.089+0000 7fd274e72700 1 librados: starting msgr at 2024-02-23T00:54:31.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.089+0000 7fd274e72700 1 librados: starting objecter 2024-02-23T00:54:31.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.089+0000 7fd274e72700 1 Processor -- start 2024-02-23T00:54:31.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.090+0000 7fd274e72700 1 -- start start 2024-02-23T00:54:31.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.090+0000 7fd274e72700 1 librados: setting wanted keys 2024-02-23T00:54:31.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.090+0000 7fd274e72700 1 librados: calling monclient init 2024-02-23T00:54:31.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.090+0000 7fd274e72700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2700f6b10 0x7fd2700f8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:31.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.090+0000 7fd274e72700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2700fc240 0x7fd2700f8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:31.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.090+0000 7fd274e72700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2700fcba0 0x7fd2700f8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:31.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.090+0000 7fd274e72700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd270116480 con 0x7fd2700fcba0 2024-02-23T00:54:31.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.090+0000 7fd274e72700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2701165f0 con 0x7fd2700f6b10 2024-02-23T00:54:31.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.090+0000 7fd274e72700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2701167b0 con 0x7fd2700fc240 2024-02-23T00:54:31.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.090+0000 7fd26dd9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2700fc240 0x7fd2700f8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:31.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.090+0000 7fd26ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2700fcba0 0x7fd2700f8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:31.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.090+0000 7fd26dd9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2700fc240 0x7fd2700f8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:55254/0 (socket says 172.21.15.47:55254) 2024-02-23T00:54:31.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.090+0000 7fd26dd9b700 1 -- 172.21.15.47:0/3764643624 learned_addr learned my addr 172.21.15.47:0/3764643624 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:54:31.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.090+0000 7fd26e59c700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2700f6b10 0x7fd2700f8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:31.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.090+0000 7fd26dd9b700 1 -- 172.21.15.47:0/3764643624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2700f6b10 msgr2=0x7fd2700f8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:31.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.090+0000 7fd26dd9b700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2700f6b10 0x7fd2700f8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:31.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.090+0000 7fd26dd9b700 1 -- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2700fcba0 msgr2=0x7fd2700f8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:31.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.090+0000 7fd26dd9b700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2700fcba0 0x7fd2700f8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:31.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.090+0000 7fd26dd9b700 1 -- 172.21.15.47:0/3764643624 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2680097c0 con 0x7fd2700fc240 2024-02-23T00:54:31.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.090+0000 7fd26e59c700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2700f6b10 0x7fd2700f8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:54:31.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.090+0000 7fd26ed9d700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2700fcba0 0x7fd2700f8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:54:31.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.090+0000 7fd26dd9b700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2700fc240 0x7fd2700f8530 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 rx=0x7fd264004fe0 tx=0x7fd26400adc0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:31.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.090+0000 7fd25f7fe700 10 client.?.objecter ms_handle_connect 0x7fd2700fc240 2024-02-23T00:54:31.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.090+0000 7fd25f7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:54:31.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.091+0000 7fd25f7fe700 1 -- 172.21.15.47:0/3764643624 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26400f040 con 0x7fd2700fc240 2024-02-23T00:54:31.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.091+0000 7fd25f7fe700 1 -- 172.21.15.47:0/3764643624 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fd264014680 con 0x7fd2700fc240 2024-02-23T00:54:31.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.091+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd264005040 con 0x7fd2700fc240 2024-02-23T00:54:31.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.091+0000 7fd25f7fe700 1 -- 172.21.15.47:0/3764643624 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26401dbb0 con 0x7fd2700fc240 2024-02-23T00:54:31.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.091+0000 7fd274e72700 10 client.42020.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:54:31.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.091+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd270116c40 con 0x7fd2700fc240 2024-02-23T00:54:31.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.091+0000 7fd274e72700 1 librados: init done 2024-02-23T00:54:31.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.091+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fd27005f970 con 0x7fd2700fc240 2024-02-23T00:54:31.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.093+0000 7fd25f7fe700 1 -- 172.21.15.47:0/3764643624 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fd26400a2e0 con 0x7fd2700fc240 2024-02-23T00:54:31.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.093+0000 7fd25f7fe700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd258062cc0 0x7fd258065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:31.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.093+0000 7fd25f7fe700 1 -- 172.21.15.47:0/3764643624 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(731..731 src has 621..731) v4 ==== 9752+0+0 (secure 0 0 0) 0x7fd26407ee00 con 0x7fd2700fc240 2024-02-23T00:54:31.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.093+0000 7fd25f7fe700 10 client.42020.objecter ms_dispatch 0x7fd270001550 osd_map(731..731 src has 621..731) v4 2024-02-23T00:54:31.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.093+0000 7fd25f7fe700 3 client.42020.objecter handle_osd_map got epochs [731,731] > 0 2024-02-23T00:54:31.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.093+0000 7fd25f7fe700 3 client.42020.objecter handle_osd_map decoding full epoch 731 2024-02-23T00:54:31.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.093+0000 7fd26e59c700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd258062cc0 0x7fd258065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:31.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.094+0000 7fd25f7fe700 20 client.42020.objecter dump_active .. 0 homeless 2024-02-23T00:54:31.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.094+0000 7fd25f7fe700 1 -- 172.21.15.47:0/3764643624 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v731) v1 ==== 54+0+16 (secure 0 0 0) 0x7fd26407f2e0 con 0x7fd2700fc240 2024-02-23T00:54:31.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.094+0000 7fd274e72700 10 client.42020.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:31.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.094+0000 7fd274e72700 15 client.42020.objecter _session_command_op_assign -1 1 2024-02-23T00:54:31.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.094+0000 7fd274e72700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fd270028540 0x7fd2700ff050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:31.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.094+0000 7fd25f7fe700 10 client.42020.objecter ms_handle_connect 0x7fd258062cc0 2024-02-23T00:54:31.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.094+0000 7fd274e72700 20 client.42020.objecter _get_session s=0x7fd27005ee80 osd=0 3 2024-02-23T00:54:31.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.094+0000 7fd26e59c700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd258062cc0 0x7fd258065180 secure :-1 s=READY pgs=3180 cs=0 l=1 rev1=1 rx=0x7fd260009290 tx=0x7fd26000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:31.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.094+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd27005ee80 osd=0 3 2024-02-23T00:54:31.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.094+0000 7fd274e72700 20 client.42020.objecter _get_session s=0x7fd27005ee80 osd=0 3 2024-02-23T00:54:31.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.094+0000 7fd274e72700 15 client.42020.objecter _session_command_op_remove -1 1 2024-02-23T00:54:31.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.094+0000 7fd274e72700 20 client.42020.objecter get_session s=0x7fd27005ee80 osd=0 3 2024-02-23T00:54:31.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.094+0000 7fd274e72700 15 client.42020.objecter _session_command_op_assign 0 1 2024-02-23T00:54:31.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.094+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd27005ee80 osd=0 4 2024-02-23T00:54:31.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.094+0000 7fd274e72700 10 client.42020.objecter _send_command 1 2024-02-23T00:54:31.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.094+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd270104ce0 con 0x7fd270028540 2024-02-23T00:54:31.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.094+0000 7fd26ed9d700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fd270028540 0x7fd2700ff050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:31.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.094+0000 7fd25f7fe700 10 client.42020.objecter ms_handle_connect 0x7fd270028540 2024-02-23T00:54:31.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.094+0000 7fd26ed9d700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fd270028540 0x7fd2700ff050 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:31.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.095+0000 7fd25f7fe700 1 -- 172.21.15.47:0/3764643624 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd270104ce0 con 0x7fd270028540 2024-02-23T00:54:31.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.095+0000 7fd25f7fe700 10 client.42020.objecter ms_dispatch 0x7fd270001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:54:31.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.095+0000 7fd25f7fe700 10 client.42020.objecter _finish_command 1 = system:0 2024-02-23T00:54:31.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.095+0000 7fd25f7fe700 20 client.42020.objecter put_session s=0x7fd27005ee80 osd=0 4 2024-02-23T00:54:31.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.095+0000 7fd25f7fe700 15 client.42020.objecter _session_command_op_remove 0 1 2024-02-23T00:54:31.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.115+0000 7fd274e72700 10 client.42020.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:54:31.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.115+0000 7fd274e72700 15 client.42020.objecter _session_command_op_assign -1 2 2024-02-23T00:54:31.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.115+0000 7fd274e72700 20 client.42020.objecter _get_session s=0x7fd27005ee80 osd=0 3 2024-02-23T00:54:31.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.115+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd27005ee80 osd=0 3 2024-02-23T00:54:31.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.115+0000 7fd274e72700 20 client.42020.objecter _get_session s=0x7fd27005ee80 osd=0 3 2024-02-23T00:54:31.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.115+0000 7fd274e72700 15 client.42020.objecter _session_command_op_remove -1 2 2024-02-23T00:54:31.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.115+0000 7fd274e72700 20 client.42020.objecter get_session s=0x7fd27005ee80 osd=0 3 2024-02-23T00:54:31.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.115+0000 7fd274e72700 15 client.42020.objecter _session_command_op_assign 0 2 2024-02-23T00:54:31.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.115+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd27005ee80 osd=0 4 2024-02-23T00:54:31.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.115+0000 7fd274e72700 10 client.42020.objecter _send_command 2 2024-02-23T00:54:31.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.115+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fd2700f94d0 con 0x7fd270028540 2024-02-23T00:54:31.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.117+0000 7fd25f7fe700 1 -- 172.21.15.47:0/3764643624 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fd2700f94d0 con 0x7fd270028540 2024-02-23T00:54:31.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.117+0000 7fd25f7fe700 10 client.42020.objecter ms_dispatch 0x7fd270001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:54:31.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.117+0000 7fd25f7fe700 10 client.42020.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:54:31.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.117+0000 7fd25f7fe700 20 client.42020.objecter put_session s=0x7fd27005ee80 osd=0 4 2024-02-23T00:54:31.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.117+0000 7fd25f7fe700 15 client.42020.objecter _session_command_op_remove 0 2 2024-02-23T00:54:31.120 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:54:31.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.117+0000 7fd274e72700 10 client.42020.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:31.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.117+0000 7fd274e72700 15 client.42020.objecter _session_command_op_assign -1 3 2024-02-23T00:54:31.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.117+0000 7fd274e72700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fd270027b90 0x7fd270027f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:31.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.117+0000 7fd274e72700 20 client.42020.objecter _get_session s=0x7fd2701006c0 osd=1 3 2024-02-23T00:54:31.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.117+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd2701006c0 osd=1 3 2024-02-23T00:54:31.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.117+0000 7fd274e72700 20 client.42020.objecter _get_session s=0x7fd2701006c0 osd=1 3 2024-02-23T00:54:31.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.117+0000 7fd274e72700 15 client.42020.objecter _session_command_op_remove -1 3 2024-02-23T00:54:31.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.117+0000 7fd274e72700 20 client.42020.objecter get_session s=0x7fd2701006c0 osd=1 3 2024-02-23T00:54:31.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.117+0000 7fd274e72700 15 client.42020.objecter _session_command_op_assign 1 3 2024-02-23T00:54:31.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.117+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd2701006c0 osd=1 4 2024-02-23T00:54:31.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.117+0000 7fd274e72700 10 client.42020.objecter _send_command 3 2024-02-23T00:54:31.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.117+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd270100830 con 0x7fd270027b90 2024-02-23T00:54:31.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.117+0000 7fd26e59c700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fd270027b90 0x7fd270027f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:31.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.117+0000 7fd25f7fe700 10 client.42020.objecter ms_handle_connect 0x7fd270027b90 2024-02-23T00:54:31.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.117+0000 7fd26e59c700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fd270027b90 0x7fd270027f70 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:31.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.118+0000 7fd25f7fe700 1 -- 172.21.15.47:0/3764643624 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd270100830 con 0x7fd270027b90 2024-02-23T00:54:31.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.118+0000 7fd25f7fe700 10 client.42020.objecter ms_dispatch 0x7fd270001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:54:31.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.118+0000 7fd25f7fe700 10 client.42020.objecter _finish_command 3 = system:0 2024-02-23T00:54:31.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.118+0000 7fd25f7fe700 20 client.42020.objecter put_session s=0x7fd2701006c0 osd=1 4 2024-02-23T00:54:31.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.118+0000 7fd25f7fe700 15 client.42020.objecter _session_command_op_remove 1 3 2024-02-23T00:54:31.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.136+0000 7fd274e72700 10 client.42020.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:54:31.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.136+0000 7fd274e72700 15 client.42020.objecter _session_command_op_assign -1 4 2024-02-23T00:54:31.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.136+0000 7fd274e72700 20 client.42020.objecter _get_session s=0x7fd2701006c0 osd=1 3 2024-02-23T00:54:31.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.136+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd2701006c0 osd=1 3 2024-02-23T00:54:31.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.136+0000 7fd274e72700 20 client.42020.objecter _get_session s=0x7fd2701006c0 osd=1 3 2024-02-23T00:54:31.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.136+0000 7fd274e72700 15 client.42020.objecter _session_command_op_remove -1 4 2024-02-23T00:54:31.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.136+0000 7fd274e72700 20 client.42020.objecter get_session s=0x7fd2701006c0 osd=1 3 2024-02-23T00:54:31.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.136+0000 7fd274e72700 15 client.42020.objecter _session_command_op_assign 1 4 2024-02-23T00:54:31.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.136+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd2701006c0 osd=1 4 2024-02-23T00:54:31.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.136+0000 7fd274e72700 10 client.42020.objecter _send_command 4 2024-02-23T00:54:31.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.136+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fd270027600 con 0x7fd270027b90 2024-02-23T00:54:31.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.138+0000 7fd25f7fe700 1 -- 172.21.15.47:0/3764643624 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fd270027600 con 0x7fd270027b90 2024-02-23T00:54:31.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.138+0000 7fd25f7fe700 10 client.42020.objecter ms_dispatch 0x7fd270001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:54:31.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.138+0000 7fd25f7fe700 10 client.42020.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:54:31.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.138+0000 7fd25f7fe700 20 client.42020.objecter put_session s=0x7fd2701006c0 osd=1 4 2024-02-23T00:54:31.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.138+0000 7fd25f7fe700 15 client.42020.objecter _session_command_op_remove 1 4 2024-02-23T00:54:31.141 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:54:31.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.138+0000 7fd274e72700 10 client.42020.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:31.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.138+0000 7fd274e72700 15 client.42020.objecter _session_command_op_assign -1 5 2024-02-23T00:54:31.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.138+0000 7fd274e72700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fd27011c060 0x7fd27011e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:31.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.138+0000 7fd274e72700 20 client.42020.objecter _get_session s=0x7fd2700ffd10 osd=2 3 2024-02-23T00:54:31.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.138+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd2700ffd10 osd=2 3 2024-02-23T00:54:31.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.138+0000 7fd274e72700 20 client.42020.objecter _get_session s=0x7fd2700ffd10 osd=2 3 2024-02-23T00:54:31.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.138+0000 7fd274e72700 15 client.42020.objecter _session_command_op_remove -1 5 2024-02-23T00:54:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.138+0000 7fd274e72700 20 client.42020.objecter get_session s=0x7fd2700ffd10 osd=2 3 2024-02-23T00:54:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.138+0000 7fd274e72700 15 client.42020.objecter _session_command_op_assign 2 5 2024-02-23T00:54:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.138+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd2700ffd10 osd=2 4 2024-02-23T00:54:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.138+0000 7fd274e72700 10 client.42020.objecter _send_command 5 2024-02-23T00:54:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.138+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd270100420 con 0x7fd27011c060 2024-02-23T00:54:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.138+0000 7fd26dd9b700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fd27011c060 0x7fd27011e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.139+0000 7fd25f7fe700 10 client.42020.objecter ms_handle_connect 0x7fd27011c060 2024-02-23T00:54:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.139+0000 7fd26dd9b700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fd27011c060 0x7fd27011e450 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.139+0000 7fd25f7fe700 1 -- 172.21.15.47:0/3764643624 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd270100420 con 0x7fd27011c060 2024-02-23T00:54:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.139+0000 7fd25f7fe700 10 client.42020.objecter ms_dispatch 0x7fd270001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:54:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.139+0000 7fd25f7fe700 10 client.42020.objecter _finish_command 5 = system:0 2024-02-23T00:54:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.139+0000 7fd25f7fe700 20 client.42020.objecter put_session s=0x7fd2700ffd10 osd=2 4 2024-02-23T00:54:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.139+0000 7fd25f7fe700 15 client.42020.objecter _session_command_op_remove 2 5 2024-02-23T00:54:31.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.154+0000 7fd274e72700 10 client.42020.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:54:31.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.154+0000 7fd274e72700 15 client.42020.objecter _session_command_op_assign -1 6 2024-02-23T00:54:31.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.154+0000 7fd274e72700 20 client.42020.objecter _get_session s=0x7fd2700ffd10 osd=2 3 2024-02-23T00:54:31.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.154+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd2700ffd10 osd=2 3 2024-02-23T00:54:31.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.154+0000 7fd274e72700 20 client.42020.objecter _get_session s=0x7fd2700ffd10 osd=2 3 2024-02-23T00:54:31.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.154+0000 7fd274e72700 15 client.42020.objecter _session_command_op_remove -1 6 2024-02-23T00:54:31.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.154+0000 7fd274e72700 20 client.42020.objecter get_session s=0x7fd2700ffd10 osd=2 3 2024-02-23T00:54:31.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.154+0000 7fd274e72700 15 client.42020.objecter _session_command_op_assign 2 6 2024-02-23T00:54:31.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.154+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd2700ffd10 osd=2 4 2024-02-23T00:54:31.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.154+0000 7fd274e72700 10 client.42020.objecter _send_command 6 2024-02-23T00:54:31.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.154+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fd2700ff780 con 0x7fd27011c060 2024-02-23T00:54:31.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.155+0000 7fd25f7fe700 1 -- 172.21.15.47:0/3764643624 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fd2700ff780 con 0x7fd27011c060 2024-02-23T00:54:31.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.155+0000 7fd25f7fe700 10 client.42020.objecter ms_dispatch 0x7fd270001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:54:31.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.155+0000 7fd25f7fe700 10 client.42020.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:54:31.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.155+0000 7fd25f7fe700 20 client.42020.objecter put_session s=0x7fd2700ffd10 osd=2 4 2024-02-23T00:54:31.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.155+0000 7fd25f7fe700 15 client.42020.objecter _session_command_op_remove 2 6 2024-02-23T00:54:31.159 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:54:31.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.155+0000 7fd274e72700 10 client.42020.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:31.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.155+0000 7fd274e72700 15 client.42020.objecter _session_command_op_assign -1 7 2024-02-23T00:54:31.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.155+0000 7fd274e72700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fd27011fac0 0x7fd270121eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:31.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.155+0000 7fd274e72700 20 client.42020.objecter _get_session s=0x7fd27011b6b0 osd=3 3 2024-02-23T00:54:31.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.155+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd27011b6b0 osd=3 3 2024-02-23T00:54:31.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.155+0000 7fd274e72700 20 client.42020.objecter _get_session s=0x7fd27011b6b0 osd=3 3 2024-02-23T00:54:31.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.155+0000 7fd274e72700 15 client.42020.objecter _session_command_op_remove -1 7 2024-02-23T00:54:31.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.155+0000 7fd274e72700 20 client.42020.objecter get_session s=0x7fd27011b6b0 osd=3 3 2024-02-23T00:54:31.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.155+0000 7fd274e72700 15 client.42020.objecter _session_command_op_assign 3 7 2024-02-23T00:54:31.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.155+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd27011b6b0 osd=3 4 2024-02-23T00:54:31.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.155+0000 7fd274e72700 10 client.42020.objecter _send_command 7 2024-02-23T00:54:31.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.155+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd27011bdb0 con 0x7fd27011fac0 2024-02-23T00:54:31.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.155+0000 7fd26ed9d700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fd27011fac0 0x7fd270121eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:31.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.156+0000 7fd25f7fe700 10 client.42020.objecter ms_handle_connect 0x7fd27011fac0 2024-02-23T00:54:31.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.156+0000 7fd26ed9d700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fd27011fac0 0x7fd270121eb0 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:31.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.156+0000 7fd25f7fe700 1 -- 172.21.15.47:0/3764643624 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd27011bdb0 con 0x7fd27011fac0 2024-02-23T00:54:31.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.156+0000 7fd25f7fe700 10 client.42020.objecter ms_dispatch 0x7fd270001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:54:31.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.156+0000 7fd25f7fe700 10 client.42020.objecter _finish_command 7 = system:0 2024-02-23T00:54:31.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.156+0000 7fd25f7fe700 20 client.42020.objecter put_session s=0x7fd27011b6b0 osd=3 4 2024-02-23T00:54:31.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.156+0000 7fd25f7fe700 15 client.42020.objecter _session_command_op_remove 3 7 2024-02-23T00:54:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.171+0000 7fd274e72700 10 client.42020.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:54:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.171+0000 7fd274e72700 15 client.42020.objecter _session_command_op_assign -1 8 2024-02-23T00:54:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.171+0000 7fd274e72700 20 client.42020.objecter _get_session s=0x7fd27011b6b0 osd=3 3 2024-02-23T00:54:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.171+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd27011b6b0 osd=3 3 2024-02-23T00:54:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.171+0000 7fd274e72700 20 client.42020.objecter _get_session s=0x7fd27011b6b0 osd=3 3 2024-02-23T00:54:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.171+0000 7fd274e72700 15 client.42020.objecter _session_command_op_remove -1 8 2024-02-23T00:54:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.171+0000 7fd274e72700 20 client.42020.objecter get_session s=0x7fd27011b6b0 osd=3 3 2024-02-23T00:54:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.171+0000 7fd274e72700 15 client.42020.objecter _session_command_op_assign 3 8 2024-02-23T00:54:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.171+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd27011b6b0 osd=3 4 2024-02-23T00:54:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.171+0000 7fd274e72700 10 client.42020.objecter _send_command 8 2024-02-23T00:54:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.171+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fd27011b120 con 0x7fd27011fac0 2024-02-23T00:54:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.172+0000 7fd25f7fe700 1 -- 172.21.15.47:0/3764643624 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fd27011b120 con 0x7fd27011fac0 2024-02-23T00:54:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.172+0000 7fd25f7fe700 10 client.42020.objecter ms_dispatch 0x7fd270001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:54:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.172+0000 7fd25f7fe700 10 client.42020.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:54:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.172+0000 7fd25f7fe700 20 client.42020.objecter put_session s=0x7fd27011b6b0 osd=3 4 2024-02-23T00:54:31.175 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:54:31.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.172+0000 7fd25f7fe700 15 client.42020.objecter _session_command_op_remove 3 8 2024-02-23T00:54:31.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.172+0000 7fd274e72700 10 client.42020.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:31.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.172+0000 7fd274e72700 15 client.42020.objecter _session_command_op_assign -1 9 2024-02-23T00:54:31.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.172+0000 7fd274e72700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fd27011b2c0 0x7fd270125530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:31.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.173+0000 7fd274e72700 20 client.42020.objecter _get_session s=0x7fd27011b120 osd=4 3 2024-02-23T00:54:31.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.173+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd27011b120 osd=4 3 2024-02-23T00:54:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.173+0000 7fd274e72700 20 client.42020.objecter _get_session s=0x7fd27011b120 osd=4 3 2024-02-23T00:54:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.173+0000 7fd274e72700 15 client.42020.objecter _session_command_op_remove -1 9 2024-02-23T00:54:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.173+0000 7fd274e72700 20 client.42020.objecter get_session s=0x7fd27011b120 osd=4 3 2024-02-23T00:54:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.173+0000 7fd274e72700 15 client.42020.objecter _session_command_op_assign 4 9 2024-02-23T00:54:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.173+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd27011b120 osd=4 4 2024-02-23T00:54:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.173+0000 7fd274e72700 10 client.42020.objecter _send_command 9 2024-02-23T00:54:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.173+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd27011f7e0 con 0x7fd27011b2c0 2024-02-23T00:54:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.173+0000 7fd26e59c700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fd27011b2c0 0x7fd270125530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.173+0000 7fd25f7fe700 10 client.42020.objecter ms_handle_connect 0x7fd27011b2c0 2024-02-23T00:54:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.173+0000 7fd26e59c700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fd27011b2c0 0x7fd270125530 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.174+0000 7fd25f7fe700 1 -- 172.21.15.47:0/3764643624 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd27011f7e0 con 0x7fd27011b2c0 2024-02-23T00:54:31.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.174+0000 7fd25f7fe700 10 client.42020.objecter ms_dispatch 0x7fd270001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:54:31.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.174+0000 7fd25f7fe700 10 client.42020.objecter _finish_command 9 = system:0 2024-02-23T00:54:31.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.174+0000 7fd25f7fe700 20 client.42020.objecter put_session s=0x7fd27011b120 osd=4 4 2024-02-23T00:54:31.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.174+0000 7fd25f7fe700 15 client.42020.objecter _session_command_op_remove 4 9 2024-02-23T00:54:31.191 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:54:31.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.189+0000 7fd274e72700 10 client.42020.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:54:31.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.189+0000 7fd274e72700 15 client.42020.objecter _session_command_op_assign -1 10 2024-02-23T00:54:31.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.189+0000 7fd274e72700 20 client.42020.objecter _get_session s=0x7fd27011b120 osd=4 3 2024-02-23T00:54:31.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.189+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd27011b120 osd=4 3 2024-02-23T00:54:31.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.189+0000 7fd274e72700 20 client.42020.objecter _get_session s=0x7fd27011b120 osd=4 3 2024-02-23T00:54:31.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.189+0000 7fd274e72700 15 client.42020.objecter _session_command_op_remove -1 10 2024-02-23T00:54:31.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.189+0000 7fd274e72700 20 client.42020.objecter get_session s=0x7fd27011b120 osd=4 3 2024-02-23T00:54:31.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.189+0000 7fd274e72700 15 client.42020.objecter _session_command_op_assign 4 10 2024-02-23T00:54:31.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.189+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd27011b120 osd=4 4 2024-02-23T00:54:31.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.189+0000 7fd274e72700 10 client.42020.objecter _send_command 10 2024-02-23T00:54:31.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.189+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fd27011f7e0 con 0x7fd27011b2c0 2024-02-23T00:54:31.195 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:54:31.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.191+0000 7fd25f7fe700 1 -- 172.21.15.47:0/3764643624 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fd27011f7e0 con 0x7fd27011b2c0 2024-02-23T00:54:31.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.191+0000 7fd25f7fe700 10 client.42020.objecter ms_dispatch 0x7fd270001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:54:31.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.191+0000 7fd25f7fe700 10 client.42020.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:54:31.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.191+0000 7fd25f7fe700 20 client.42020.objecter put_session s=0x7fd27011b120 osd=4 4 2024-02-23T00:54:31.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.191+0000 7fd25f7fe700 15 client.42020.objecter _session_command_op_remove 4 10 2024-02-23T00:54:31.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.192+0000 7fd274e72700 10 client.42020.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:31.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.192+0000 7fd274e72700 15 client.42020.objecter _session_command_op_assign -1 11 2024-02-23T00:54:31.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.192+0000 7fd274e72700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fd270125f30 0x7fd270128320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:31.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.192+0000 7fd274e72700 20 client.42020.objecter _get_session s=0x7fd27011f7e0 osd=5 3 2024-02-23T00:54:31.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.192+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd27011f7e0 osd=5 3 2024-02-23T00:54:31.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.192+0000 7fd274e72700 20 client.42020.objecter _get_session s=0x7fd27011f7e0 osd=5 3 2024-02-23T00:54:31.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.192+0000 7fd274e72700 15 client.42020.objecter _session_command_op_remove -1 11 2024-02-23T00:54:31.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.192+0000 7fd274e72700 20 client.42020.objecter get_session s=0x7fd27011f7e0 osd=5 3 2024-02-23T00:54:31.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.192+0000 7fd274e72700 15 client.42020.objecter _session_command_op_assign 5 11 2024-02-23T00:54:31.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.192+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd27011f7e0 osd=5 4 2024-02-23T00:54:31.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.192+0000 7fd274e72700 10 client.42020.objecter _send_command 11 2024-02-23T00:54:31.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.192+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd27012b110 con 0x7fd270125f30 2024-02-23T00:54:31.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.192+0000 7fd26dd9b700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fd270125f30 0x7fd270128320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:31.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.192+0000 7fd25f7fe700 10 client.42020.objecter ms_handle_connect 0x7fd270125f30 2024-02-23T00:54:31.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.192+0000 7fd26dd9b700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fd270125f30 0x7fd270128320 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:31.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.193+0000 7fd25f7fe700 1 -- 172.21.15.47:0/3764643624 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd27012b110 con 0x7fd270125f30 2024-02-23T00:54:31.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.193+0000 7fd25f7fe700 10 client.42020.objecter ms_dispatch 0x7fd270001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:54:31.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.193+0000 7fd25f7fe700 10 client.42020.objecter _finish_command 11 = system:0 2024-02-23T00:54:31.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.193+0000 7fd25f7fe700 20 client.42020.objecter put_session s=0x7fd27011f7e0 osd=5 4 2024-02-23T00:54:31.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.193+0000 7fd25f7fe700 15 client.42020.objecter _session_command_op_remove 5 11 2024-02-23T00:54:31.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.211+0000 7fd274e72700 10 client.42020.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:54:31.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.211+0000 7fd274e72700 15 client.42020.objecter _session_command_op_assign -1 12 2024-02-23T00:54:31.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.211+0000 7fd274e72700 20 client.42020.objecter _get_session s=0x7fd27011f7e0 osd=5 3 2024-02-23T00:54:31.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.211+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd27011f7e0 osd=5 3 2024-02-23T00:54:31.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.211+0000 7fd274e72700 20 client.42020.objecter _get_session s=0x7fd27011f7e0 osd=5 3 2024-02-23T00:54:31.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.211+0000 7fd274e72700 15 client.42020.objecter _session_command_op_remove -1 12 2024-02-23T00:54:31.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.211+0000 7fd274e72700 20 client.42020.objecter get_session s=0x7fd27011f7e0 osd=5 3 2024-02-23T00:54:31.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.211+0000 7fd274e72700 15 client.42020.objecter _session_command_op_assign 5 12 2024-02-23T00:54:31.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.211+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd27011f7e0 osd=5 4 2024-02-23T00:54:31.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.211+0000 7fd274e72700 10 client.42020.objecter _send_command 12 2024-02-23T00:54:31.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.211+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fd27012b110 con 0x7fd270125f30 2024-02-23T00:54:31.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.213+0000 7fd25f7fe700 1 -- 172.21.15.47:0/3764643624 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fd27012b110 con 0x7fd270125f30 2024-02-23T00:54:31.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.213+0000 7fd25f7fe700 10 client.42020.objecter ms_dispatch 0x7fd270001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:54:31.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.213+0000 7fd25f7fe700 10 client.42020.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:54:31.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.213+0000 7fd25f7fe700 20 client.42020.objecter put_session s=0x7fd27011f7e0 osd=5 4 2024-02-23T00:54:31.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.213+0000 7fd25f7fe700 15 client.42020.objecter _session_command_op_remove 5 12 2024-02-23T00:54:31.216 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:54:31.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.213+0000 7fd274e72700 10 client.42020.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:31.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.213+0000 7fd274e72700 15 client.42020.objecter _session_command_op_assign -1 13 2024-02-23T00:54:31.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.213+0000 7fd274e72700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fd270128e60 0x7fd2701292c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:31.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.213+0000 7fd274e72700 20 client.42020.objecter _get_session s=0x7fd27012b110 osd=6 3 2024-02-23T00:54:31.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.213+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd27012b110 osd=6 3 2024-02-23T00:54:31.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.213+0000 7fd274e72700 20 client.42020.objecter _get_session s=0x7fd27012b110 osd=6 3 2024-02-23T00:54:31.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.213+0000 7fd274e72700 15 client.42020.objecter _session_command_op_remove -1 13 2024-02-23T00:54:31.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.213+0000 7fd274e72700 20 client.42020.objecter get_session s=0x7fd27012b110 osd=6 3 2024-02-23T00:54:31.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.213+0000 7fd274e72700 15 client.42020.objecter _session_command_op_assign 6 13 2024-02-23T00:54:31.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.213+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd27012b110 osd=6 4 2024-02-23T00:54:31.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.213+0000 7fd274e72700 10 client.42020.objecter _send_command 13 2024-02-23T00:54:31.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.213+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd27012e4d0 con 0x7fd270128e60 2024-02-23T00:54:31.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.214+0000 7fd26ed9d700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fd270128e60 0x7fd2701292c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:31.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.214+0000 7fd26ed9d700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fd270128e60 0x7fd2701292c0 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:31.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.214+0000 7fd25f7fe700 10 client.42020.objecter ms_handle_connect 0x7fd270128e60 2024-02-23T00:54:31.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.215+0000 7fd25f7fe700 1 -- 172.21.15.47:0/3764643624 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd27012e4d0 con 0x7fd270128e60 2024-02-23T00:54:31.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.215+0000 7fd25f7fe700 10 client.42020.objecter ms_dispatch 0x7fd270001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:54:31.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.215+0000 7fd25f7fe700 10 client.42020.objecter _finish_command 13 = system:0 2024-02-23T00:54:31.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.215+0000 7fd25f7fe700 20 client.42020.objecter put_session s=0x7fd27012b110 osd=6 4 2024-02-23T00:54:31.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.215+0000 7fd25f7fe700 15 client.42020.objecter _session_command_op_remove 6 13 2024-02-23T00:54:31.223 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:31.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.230+0000 7fd274e72700 10 client.42020.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:54:31.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.230+0000 7fd274e72700 15 client.42020.objecter _session_command_op_assign -1 14 2024-02-23T00:54:31.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.230+0000 7fd274e72700 20 client.42020.objecter _get_session s=0x7fd27012b110 osd=6 3 2024-02-23T00:54:31.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.230+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd27012b110 osd=6 3 2024-02-23T00:54:31.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.230+0000 7fd274e72700 20 client.42020.objecter _get_session s=0x7fd27012b110 osd=6 3 2024-02-23T00:54:31.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.230+0000 7fd274e72700 15 client.42020.objecter _session_command_op_remove -1 14 2024-02-23T00:54:31.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.230+0000 7fd274e72700 20 client.42020.objecter get_session s=0x7fd27012b110 osd=6 3 2024-02-23T00:54:31.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.230+0000 7fd274e72700 15 client.42020.objecter _session_command_op_assign 6 14 2024-02-23T00:54:31.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.230+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd27012b110 osd=6 4 2024-02-23T00:54:31.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.230+0000 7fd274e72700 10 client.42020.objecter _send_command 14 2024-02-23T00:54:31.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.230+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fd2701298c0 con 0x7fd270128e60 2024-02-23T00:54:31.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.232+0000 7fd25f7fe700 1 -- 172.21.15.47:0/3764643624 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fd2701298c0 con 0x7fd270128e60 2024-02-23T00:54:31.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.232+0000 7fd25f7fe700 10 client.42020.objecter ms_dispatch 0x7fd270001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:54:31.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.232+0000 7fd25f7fe700 10 client.42020.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:54:31.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.232+0000 7fd25f7fe700 20 client.42020.objecter put_session s=0x7fd27012b110 osd=6 4 2024-02-23T00:54:31.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.232+0000 7fd25f7fe700 15 client.42020.objecter _session_command_op_remove 6 14 2024-02-23T00:54:31.236 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:54:31.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.233+0000 7fd274e72700 10 client.42020.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:31.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.233+0000 7fd274e72700 15 client.42020.objecter _session_command_op_assign -1 15 2024-02-23T00:54:31.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.233+0000 7fd274e72700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fd270129f40 0x7fd27012a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:31.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.233+0000 7fd274e72700 20 client.42020.objecter _get_session s=0x7fd2701298c0 osd=7 3 2024-02-23T00:54:31.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.233+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd2701298c0 osd=7 3 2024-02-23T00:54:31.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.233+0000 7fd274e72700 20 client.42020.objecter _get_session s=0x7fd2701298c0 osd=7 3 2024-02-23T00:54:31.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.233+0000 7fd274e72700 15 client.42020.objecter _session_command_op_remove -1 15 2024-02-23T00:54:31.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.233+0000 7fd274e72700 20 client.42020.objecter get_session s=0x7fd2701298c0 osd=7 3 2024-02-23T00:54:31.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.233+0000 7fd274e72700 15 client.42020.objecter _session_command_op_assign 7 15 2024-02-23T00:54:31.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.233+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd2701298c0 osd=7 4 2024-02-23T00:54:31.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.233+0000 7fd274e72700 10 client.42020.objecter _send_command 15 2024-02-23T00:54:31.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.233+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd27012aa60 con 0x7fd270129f40 2024-02-23T00:54:31.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.233+0000 7fd26e59c700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fd270129f40 0x7fd27012a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:31.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.233+0000 7fd25f7fe700 10 client.42020.objecter ms_handle_connect 0x7fd270129f40 2024-02-23T00:54:31.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.233+0000 7fd26e59c700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fd270129f40 0x7fd27012a380 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:31.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.234+0000 7fd25f7fe700 1 -- 172.21.15.47:0/3764643624 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd27012aa60 con 0x7fd270129f40 2024-02-23T00:54:31.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.234+0000 7fd25f7fe700 10 client.42020.objecter ms_dispatch 0x7fd270001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:54:31.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.234+0000 7fd25f7fe700 10 client.42020.objecter _finish_command 15 = system:0 2024-02-23T00:54:31.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.234+0000 7fd25f7fe700 20 client.42020.objecter put_session s=0x7fd2701298c0 osd=7 4 2024-02-23T00:54:31.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.234+0000 7fd25f7fe700 15 client.42020.objecter _session_command_op_remove 7 15 2024-02-23T00:54:31.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.250+0000 7fd274e72700 10 client.42020.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:54:31.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.250+0000 7fd274e72700 15 client.42020.objecter _session_command_op_assign -1 16 2024-02-23T00:54:31.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.250+0000 7fd274e72700 20 client.42020.objecter _get_session s=0x7fd2701298c0 osd=7 3 2024-02-23T00:54:31.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.250+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd2701298c0 osd=7 3 2024-02-23T00:54:31.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.250+0000 7fd274e72700 20 client.42020.objecter _get_session s=0x7fd2701298c0 osd=7 3 2024-02-23T00:54:31.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.250+0000 7fd274e72700 15 client.42020.objecter _session_command_op_remove -1 16 2024-02-23T00:54:31.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.250+0000 7fd274e72700 20 client.42020.objecter get_session s=0x7fd2701298c0 osd=7 3 2024-02-23T00:54:31.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.250+0000 7fd274e72700 15 client.42020.objecter _session_command_op_assign 7 16 2024-02-23T00:54:31.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.250+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd2701298c0 osd=7 4 2024-02-23T00:54:31.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.250+0000 7fd274e72700 10 client.42020.objecter _send_command 16 2024-02-23T00:54:31.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.250+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fd27012aa60 con 0x7fd270129f40 2024-02-23T00:54:31.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd25f7fe700 1 -- 172.21.15.47:0/3764643624 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fd27012aa60 con 0x7fd270129f40 2024-02-23T00:54:31.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd25f7fe700 10 client.42020.objecter ms_dispatch 0x7fd270001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:54:31.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd25f7fe700 10 client.42020.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:54:31.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd25f7fe700 20 client.42020.objecter put_session s=0x7fd2701298c0 osd=7 4 2024-02-23T00:54:31.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd25f7fe700 15 client.42020.objecter _session_command_op_remove 7 16 2024-02-23T00:54:31.255 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:54:31.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 10 librados: watch_flush enter 2024-02-23T00:54:31.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 10 librados: watch_flush exit 2024-02-23T00:54:31.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 10 client.42020.objecter close_session for osd.0 2024-02-23T00:54:31.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fd270028540 msgr2=0x7fd2700ff050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:31.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fd270028540 0x7fd2700ff050 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:31.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd27005ee80 osd=0 1 2024-02-23T00:54:31.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 10 client.42020.objecter close_session for osd.1 2024-02-23T00:54:31.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fd270027b90 msgr2=0x7fd270027f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:31.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fd270027b90 0x7fd270027f70 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:31.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd2701006c0 osd=1 1 2024-02-23T00:54:31.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 10 client.42020.objecter close_session for osd.2 2024-02-23T00:54:31.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fd27011c060 msgr2=0x7fd27011e450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:31.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fd27011c060 0x7fd27011e450 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:31.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd2700ffd10 osd=2 1 2024-02-23T00:54:31.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 10 client.42020.objecter close_session for osd.3 2024-02-23T00:54:31.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fd27011fac0 msgr2=0x7fd270121eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:31.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd26ed9d700 1 -- 172.21.15.47:0/3764643624 reap_dead start 2024-02-23T00:54:31.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fd27011fac0 0x7fd270121eb0 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:31.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd27011b6b0 osd=3 1 2024-02-23T00:54:31.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 10 client.42020.objecter close_session for osd.4 2024-02-23T00:54:31.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fd27011b2c0 msgr2=0x7fd270125530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:31.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fd27011b2c0 0x7fd270125530 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:31.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd27011b120 osd=4 1 2024-02-23T00:54:31.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 10 client.42020.objecter close_session for osd.5 2024-02-23T00:54:31.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fd270125f30 msgr2=0x7fd270128320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:31.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fd270125f30 0x7fd270128320 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:31.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd27011f7e0 osd=5 1 2024-02-23T00:54:31.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 10 client.42020.objecter close_session for osd.6 2024-02-23T00:54:31.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fd270128e60 msgr2=0x7fd2701292c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:31.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fd270128e60 0x7fd2701292c0 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:31.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd27012b110 osd=6 1 2024-02-23T00:54:31.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 10 client.42020.objecter close_session for osd.7 2024-02-23T00:54:31.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fd270129f40 msgr2=0x7fd27012a380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:31.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fd270129f40 0x7fd27012a380 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:31.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 20 client.42020.objecter put_session s=0x7fd2701298c0 osd=7 1 2024-02-23T00:54:31.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 20 client.42020.objecter shutdown clearing up homeless session... 2024-02-23T00:54:31.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 10 client.42020.objecter successfully canceled tick 2024-02-23T00:54:31.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd26ed9d700 1 -- 172.21.15.47:0/3764643624 reap_dead start 2024-02-23T00:54:31.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd258062cc0 msgr2=0x7fd258065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:31.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd258062cc0 0x7fd258065180 secure :-1 s=READY pgs=3180 cs=0 l=1 rev1=1 rx=0x7fd260009290 tx=0x7fd26000b040).stop 2024-02-23T00:54:31.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2700fc240 msgr2=0x7fd2700f8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:31.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.252+0000 7fd274e72700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2700fc240 0x7fd2700f8530 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 rx=0x7fd264004fe0 tx=0x7fd26400adc0).stop 2024-02-23T00:54:31.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.253+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 shutdown_connections 2024-02-23T00:54:31.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.253+0000 7fd274e72700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2700fc240 0x7fd2700f8530 unknown :-1 s=CLOSED pgs=2794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:31.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.253+0000 7fd274e72700 1 --2- 172.21.15.47:0/3764643624 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd258062cc0 0x7fd258065180 unknown :-1 s=CLOSED pgs=3180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:31.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.253+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 >> 172.21.15.47:0/3764643624 conn(0x7fd27006c370 msgr2=0x7fd2700fa270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:31.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.253+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 shutdown_connections 2024-02-23T00:54:31.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.253+0000 7fd274e72700 1 -- 172.21.15.47:0/3764643624 wait complete. 2024-02-23T00:54:31.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:31.253+0000 7fd274e72700 1 librados: shutdown 2024-02-23T00:54:31.261 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:54:31.261 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:54:31.261 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:54:31.261 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:54:31.261 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:54:31.261 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:54:31.261 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:54:31.261 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:54:31.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:30.321881+0000 mon.a (mon.0) 3545 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-128","var": "hit_set_period","val": "600"}]': finished 2024-02-23T00:54:31.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:30.321975+0000 mon.a (mon.0) 3546 : cluster [DBG] osdmap e731: 8 total, 8 up, 6 in 2024-02-23T00:54:31.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:30.323124+0000 mon.c (mon.2) 439 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-128","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-23T00:54:31.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:30.329049+0000 mon.a (mon.0) 3547 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-128","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-23T00:54:31.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:31.093380+0000 mon.c (mon.2) 440 : audit [DBG] from='client.? 172.21.15.47:0/3764643624' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:54:31.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:30.321881+0000 mon.a (mon.0) 3545 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-128","var": "hit_set_period","val": "600"}]': finished 2024-02-23T00:54:31.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:30.321975+0000 mon.a (mon.0) 3546 : cluster [DBG] osdmap e731: 8 total, 8 up, 6 in 2024-02-23T00:54:31.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:30.323124+0000 mon.c (mon.2) 439 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-128","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-23T00:54:31.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:30.329049+0000 mon.a (mon.0) 3547 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-128","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-23T00:54:31.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:31.093380+0000 mon.c (mon.2) 440 : audit [DBG] from='client.? 172.21.15.47:0/3764643624' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:54:31.659 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-02-23T00:54:31.660 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph pg dump --format=json 2024-02-23T00:54:31.672 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:54:31.675 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:31.675 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:54:31.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:30.321881+0000 mon.a (mon.0) 3545 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-128","var": "hit_set_period","val": "600"}]': finished 2024-02-23T00:54:31.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:30.321975+0000 mon.a (mon.0) 3546 : cluster [DBG] osdmap e731: 8 total, 8 up, 6 in 2024-02-23T00:54:31.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:30.323124+0000 mon.c (mon.2) 439 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-128","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-23T00:54:31.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:30.329049+0000 mon.a (mon.0) 3547 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-128","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-23T00:54:31.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:31.093380+0000 mon.c (mon.2) 440 : audit [DBG] from='client.? 172.21.15.47:0/3764643624' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:54:31.776 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:54:31.808 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:31.829 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:54:31.832 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:31.832 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:54:31.933 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:54:31.964 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:31.984 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:54:31.986 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:31.987 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:54:32.088 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:54:32.118 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:32.140 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:54:32.141 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:54:32.146 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:32.147 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:54:32.168 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:54:32.248 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:54:32.276 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:32.296 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:54:32.298 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:32.299 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:54:32.338 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:31.329233+0000 mon.a (mon.0) 3548 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-128","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-23T00:54:32.338 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:31.329320+0000 mon.a (mon.0) 3549 : cluster [DBG] osdmap e732: 8 total, 8 up, 6 in 2024-02-23T00:54:32.338 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:31.386336+0000 mgr.y (mgr.14517) 623 : cluster [DBG] pgmap v1107: 184 pgs: 1 active+remapped+backfill_toofull, 32 creating+peering, 151 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1/726 objects misplaced (0.138%) 2024-02-23T00:54:32.400 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:54:32.428 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:32.447 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:54:32.449 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:32.450 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:54:32.471 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset noscrub 2024-02-23T00:54:32.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:31.329233+0000 mon.a (mon.0) 3548 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-128","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-23T00:54:32.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:31.329320+0000 mon.a (mon.0) 3549 : cluster [DBG] osdmap e732: 8 total, 8 up, 6 in 2024-02-23T00:54:32.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:31.386336+0000 mgr.y (mgr.14517) 623 : cluster [DBG] pgmap v1107: 184 pgs: 1 active+remapped+backfill_toofull, 32 creating+peering, 151 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1/726 objects misplaced (0.138%) 2024-02-23T00:54:32.551 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:54:32.579 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:32.652 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:31.329233+0000 mon.a (mon.0) 3548 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-128","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-23T00:54:32.652 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:31.329320+0000 mon.a (mon.0) 3549 : cluster [DBG] osdmap e732: 8 total, 8 up, 6 in 2024-02-23T00:54:32.652 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:31.386336+0000 mgr.y (mgr.14517) 623 : cluster [DBG] pgmap v1107: 184 pgs: 1 active+remapped+backfill_toofull, 32 creating+peering, 151 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1/726 objects misplaced (0.138%) 2024-02-23T00:54:32.932 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:54:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:54:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:54:33.194 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:54:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:54:33.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:32.346784+0000 mon.c (mon.2) 441 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:33.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:32.347125+0000 mon.a (mon.0) 3550 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:33.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:32.347563+0000 mon.c (mon.2) 442 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-128"}]: dispatch 2024-02-23T00:54:33.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:32.347809+0000 mon.a (mon.0) 3551 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-128"}]: dispatch 2024-02-23T00:54:33.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:32.346784+0000 mon.c (mon.2) 441 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:33.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:32.347125+0000 mon.a (mon.0) 3550 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:33.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:32.347563+0000 mon.c (mon.2) 442 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-128"}]: dispatch 2024-02-23T00:54:33.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:32.347809+0000 mon.a (mon.0) 3551 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-128"}]: dispatch 2024-02-23T00:54:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:32.346784+0000 mon.c (mon.2) 441 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:32.347125+0000 mon.a (mon.0) 3550 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:32.347563+0000 mon.c (mon.2) 442 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-128"}]: dispatch 2024-02-23T00:54:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:32.347809+0000 mon.a (mon.0) 3551 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-128"}]: dispatch 2024-02-23T00:54:34.029 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:54:34.033 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:34.033 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:54:34.086 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:54:34.134 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:54:34.166 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:34.187 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:54:34.190 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:34.190 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:54:34.291 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:54:34.320 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:34.339 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:54:34.342 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:34.342 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:54:34.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-02-23T00:54:34.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (11930 ms) 2024-02-23T00:54:34.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-02-23T00:54:34.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-02-23T00:54:34.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: 00000010 69 73 |is| 2024-02-23T00:54:34.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: 00000012 2024-02-23T00:54:34.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (12004 ms) 2024-02-23T00:54:34.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-02-23T00:54:34.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: require_osd_release = pacific 2024-02-23T00:54:34.356 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (5966 ms) 2024-02-23T00:54:34.357 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [----------] 43 tests from LibRadosTwoPoolsPP (491483 ms total) 2024-02-23T00:54:34.357 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: 2024-02-23T00:54:34.357 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2024-02-23T00:54:34.357 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2024-02-23T00:54:34.357 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1014 ms) 2024-02-23T00:54:34.357 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-02-23T00:54:34.357 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (10975 ms) 2024-02-23T00:54:34.357 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2024-02-23T00:54:34.357 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (18053 ms) 2024-02-23T00:54:34.357 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2024-02-23T00:54:34.357 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (2 ms) 2024-02-23T00:54:34.357 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (30045 ms total) 2024-02-23T00:54:34.357 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: 2024-02-23T00:54:34.357 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2024-02-23T00:54:34.357 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-02-23T00:54:34.358 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (7005 ms) 2024-02-23T00:54:34.358 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2024-02-23T00:54:34.358 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (8036 ms) 2024-02-23T00:54:34.358 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-02-23T00:54:34.358 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: waiting for scrub... 2024-02-23T00:54:34.358 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: done waiting 2024-02-23T00:54:34.358 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (24983 ms) 2024-02-23T00:54:34.358 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-02-23T00:54:34.358 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (10001 ms) 2024-02-23T00:54:34.358 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-02-23T00:54:34.358 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (8001 ms) 2024-02-23T00:54:34.358 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2024-02-23T00:54:34.358 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (8992 ms) 2024-02-23T00:54:34.358 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-02-23T00:54:34.358 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (11003 ms) 2024-02-23T00:54:34.359 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-02-23T00:54:34.359 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlush (7999 ms) 2024-02-23T00:54:34.359 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-02-23T00:54:34.359 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (13079 ms) 2024-02-23T00:54:34.359 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2024-02-23T00:54:34.359 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (7937 ms) 2024-02-23T00:54:34.359 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-02-23T00:54:34.359 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (13002 ms) 2024-02-23T00:54:34.359 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-02-23T00:54:34.359 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (8006 ms) 2024-02-23T00:54:34.359 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-02-23T00:54:34.359 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (7993 ms) 2024-02-23T00:54:34.359 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-02-23T00:54:34.359 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: hmm, no HitSet yet 2024-02-23T00:54:34.360 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: ok, hit_set contains 248:602f83fe:::foo:head 2024-02-23T00:54:34.443 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:54:34.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:32.929995+0000 mgr.y (mgr.14517) 624 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:54:34.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:33.340257+0000 mon.a (mon.0) 3552 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-128"}]': finished 2024-02-23T00:54:34.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:33.340335+0000 mon.a (mon.0) 3553 : cluster [DBG] osdmap e733: 8 total, 8 up, 6 in 2024-02-23T00:54:34.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:33.386747+0000 mgr.y (mgr.14517) 625 : cluster [DBG] pgmap v1109: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/726 objects misplaced (0.138%) 2024-02-23T00:54:34.473 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:34.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:32.929995+0000 mgr.y (mgr.14517) 624 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:54:34.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:33.340257+0000 mon.a (mon.0) 3552 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-128"}]': finished 2024-02-23T00:54:34.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:33.340335+0000 mon.a (mon.0) 3553 : cluster [DBG] osdmap e733: 8 total, 8 up, 6 in 2024-02-23T00:54:34.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:33.386747+0000 mgr.y (mgr.14517) 625 : cluster [DBG] pgmap v1109: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/726 objects misplaced (0.138%) 2024-02-23T00:54:34.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:32.929995+0000 mgr.y (mgr.14517) 624 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:54:34.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:33.340257+0000 mon.a (mon.0) 3552 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-128"}]': finished 2024-02-23T00:54:34.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:33.340335+0000 mon.a (mon.0) 3553 : cluster [DBG] osdmap e733: 8 total, 8 up, 6 in 2024-02-23T00:54:34.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:33.386747+0000 mgr.y (mgr.14517) 625 : cluster [DBG] pgmap v1109: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/726 objects misplaced (0.138%) 2024-02-23T00:54:34.924 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:54:34.927 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:34.927 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:54:35.014 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:54:35.028 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:54:35.060 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:54:35.060 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:54:35.062 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:35.691 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:34.350295+0000 mon.a (mon.0) 3554 : cluster [DBG] osdmap e734: 8 total, 8 up, 6 in 2024-02-23T00:54:35.692 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:34.350295+0000 mon.a (mon.0) 3554 : cluster [DBG] osdmap e734: 8 total, 8 up, 6 in 2024-02-23T00:54:35.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:34.350295+0000 mon.a (mon.0) 3554 : cluster [DBG] osdmap e734: 8 total, 8 up, 6 in 2024-02-23T00:54:36.379 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:35.371842+0000 mon.a (mon.0) 3555 : cluster [DBG] osdmap e735: 8 total, 8 up, 6 in 2024-02-23T00:54:36.380 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:35.375897+0000 mon.c (mon.2) 443 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:54:36.380 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:35.376232+0000 mon.a (mon.0) 3556 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:54:36.380 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:35.387126+0000 mgr.y (mgr.14517) 626 : cluster [DBG] pgmap v1112: 184 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 151 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/726 objects misplaced (0.138%) 2024-02-23T00:54:36.380 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:35.371842+0000 mon.a (mon.0) 3555 : cluster [DBG] osdmap e735: 8 total, 8 up, 6 in 2024-02-23T00:54:36.380 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:35.375897+0000 mon.c (mon.2) 443 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:54:36.380 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:35.376232+0000 mon.a (mon.0) 3556 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:54:36.380 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:35.387126+0000 mgr.y (mgr.14517) 626 : cluster [DBG] pgmap v1112: 184 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 151 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/726 objects misplaced (0.138%) 2024-02-23T00:54:36.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:35.371842+0000 mon.a (mon.0) 3555 : cluster [DBG] osdmap e735: 8 total, 8 up, 6 in 2024-02-23T00:54:36.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:35.375897+0000 mon.c (mon.2) 443 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:54:36.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:35.376232+0000 mon.a (mon.0) 3556 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:54:36.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:35.387126+0000 mgr.y (mgr.14517) 626 : cluster [DBG] pgmap v1112: 184 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 151 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/726 objects misplaced (0.138%) 2024-02-23T00:54:36.742 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:54:36.746 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:36.747 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:54:36.747 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:54:36.773 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.2 dump_ops_in_flight 2024-02-23T00:54:36.847 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:54:36.885 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:36.988 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:54:36.991 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:36.991 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:54:37.092 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:54:37.125 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:37.147 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:54:37.149 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:37.150 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:54:37.251 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:54:37.280 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:37.300 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:54:37.302 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:37.303 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:54:37.404 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:54:37.437 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:37.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.427+0000 7fedbdf8a700 1 -- 172.21.15.47:0/4289327283 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fedb80fd510 msgr2=0x7fedb80f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:37.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.427+0000 7fedbdf8a700 1 --2- 172.21.15.47:0/4289327283 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fedb80fd510 0x7fedb80f6d40 secure :-1 s=READY pgs=2795 cs=0 l=1 rev1=1 rx=0x7fedac004870 tx=0x7fedac009cb0).stop 2024-02-23T00:54:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.427+0000 7fedbdf8a700 1 -- 172.21.15.47:0/4289327283 shutdown_connections 2024-02-23T00:54:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.427+0000 7fedbdf8a700 1 --2- 172.21.15.47:0/4289327283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedb80f7400 0x7fedb8103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.427+0000 7fedbdf8a700 1 --2- 172.21.15.47:0/4289327283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedb80fcbb0 0x7fedb80fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.427+0000 7fedbdf8a700 1 --2- 172.21.15.47:0/4289327283 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fedb80fd510 0x7fedb80f6d40 unknown :-1 s=CLOSED pgs=2795 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.427+0000 7fedbdf8a700 1 -- 172.21.15.47:0/4289327283 >> 172.21.15.47:0/4289327283 conn(0x7fedb80f2820 msgr2=0x7fedb80f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.428+0000 7fedbdf8a700 1 -- 172.21.15.47:0/4289327283 shutdown_connections 2024-02-23T00:54:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.428+0000 7fedbdf8a700 1 -- 172.21.15.47:0/4289327283 wait complete. 2024-02-23T00:54:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.428+0000 7fedbdf8a700 1 librados: starting msgr at 2024-02-23T00:54:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.428+0000 7fedbdf8a700 1 librados: starting objecter 2024-02-23T00:54:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.429+0000 7fedbdf8a700 1 Processor -- start 2024-02-23T00:54:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.429+0000 7fedbdf8a700 1 -- start start 2024-02-23T00:54:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.429+0000 7fedbdf8a700 1 librados: setting wanted keys 2024-02-23T00:54:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.429+0000 7fedbdf8a700 1 librados: calling monclient init 2024-02-23T00:54:37.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.429+0000 7fedbdf8a700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fedb80f7400 0x7fedb8109160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:37.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.429+0000 7fedbdf8a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedb80fcbb0 0x7fedb8109650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:37.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.429+0000 7fedbdf8a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedb80fd510 0x7fedb8109c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:37.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.429+0000 7fedbdf8a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb8114390 con 0x7fedb80fd510 2024-02-23T00:54:37.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.429+0000 7fedbdf8a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb8114500 con 0x7fedb80fcbb0 2024-02-23T00:54:37.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.429+0000 7fedbdf8a700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb81146c0 con 0x7fedb80f7400 2024-02-23T00:54:37.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.429+0000 7fedb77fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fedb80f7400 0x7fedb8109160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:37.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.429+0000 7fedb77fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fedb80f7400 0x7fedb8109160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:50094/0 (socket says 172.21.15.47:50094) 2024-02-23T00:54:37.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.429+0000 7fedb77fe700 1 -- 172.21.15.47:0/3015809014 learned_addr learned my addr 172.21.15.47:0/3015809014 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:54:37.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.429+0000 7fedb7fff700 1 --2- 172.21.15.47:0/3015809014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedb80fd510 0x7fedb8109c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:37.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.429+0000 7fedb6ffd700 1 --2- 172.21.15.47:0/3015809014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedb80fcbb0 0x7fedb8109650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:37.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.430+0000 7fedb77fe700 1 -- 172.21.15.47:0/3015809014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedb80fcbb0 msgr2=0x7fedb8109650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:37.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.430+0000 7fedb77fe700 1 --2- 172.21.15.47:0/3015809014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedb80fcbb0 0x7fedb8109650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:37.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.430+0000 7fedb77fe700 1 -- 172.21.15.47:0/3015809014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedb80fd510 msgr2=0x7fedb8109c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:37.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.430+0000 7fedb77fe700 1 --2- 172.21.15.47:0/3015809014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedb80fd510 0x7fedb8109c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:37.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.430+0000 7fedb77fe700 1 -- 172.21.15.47:0/3015809014 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda8005040 con 0x7fedb80f7400 2024-02-23T00:54:37.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.430+0000 7fedb77fe700 1 --2- 172.21.15.47:0/3015809014 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fedb80f7400 0x7fedb8109160 secure :-1 s=READY pgs=2796 cs=0 l=1 rev1=1 rx=0x7feda80048f0 tx=0x7feda8009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.430+0000 7fedb4ff9700 10 client.?.objecter ms_handle_connect 0x7fedb80f7400 2024-02-23T00:54:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.430+0000 7fedb4ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:54:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.430+0000 7fedb4ff9700 1 -- 172.21.15.47:0/3015809014 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda8007b00 con 0x7fedb80f7400 2024-02-23T00:54:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.430+0000 7fedbdf8a700 1 -- 172.21.15.47:0/3015809014 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedac005040 con 0x7fedb80f7400 2024-02-23T00:54:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.430+0000 7fedb4ff9700 1 -- 172.21.15.47:0/3015809014 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7feda800f040 con 0x7fedb80f7400 2024-02-23T00:54:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.430+0000 7fedb4ff9700 1 -- 172.21.15.47:0/3015809014 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda8014820 con 0x7fedb80f7400 2024-02-23T00:54:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.430+0000 7fedbdf8a700 10 client.42026.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:54:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.431+0000 7fedbdf8a700 1 -- 172.21.15.47:0/3015809014 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb8114d50 con 0x7fedb80f7400 2024-02-23T00:54:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.431+0000 7fedbdf8a700 1 librados: init done 2024-02-23T00:54:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.432+0000 7fedb4ff9700 1 -- 172.21.15.47:0/3015809014 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7feda8014c50 con 0x7fedb80f7400 2024-02-23T00:54:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.432+0000 7fedbdf8a700 1 -- 172.21.15.47:0/3015809014 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedb805fa60 con 0x7fedb80f7400 2024-02-23T00:54:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.432+0000 7fedb4ff9700 1 --2- 172.21.15.47:0/3015809014 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7feda0062cc0 0x7feda0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.432+0000 7fedb4ff9700 1 -- 172.21.15.47:0/3015809014 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(737..737 src has 621..737) v4 ==== 9752+0+0 (secure 0 0 0) 0x7feda807d3d0 con 0x7fedb80f7400 2024-02-23T00:54:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.432+0000 7fedb4ff9700 10 client.42026.objecter ms_dispatch 0x7fedb8001550 osd_map(737..737 src has 621..737) v4 2024-02-23T00:54:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.432+0000 7fedb4ff9700 3 client.42026.objecter handle_osd_map got epochs [737,737] > 0 2024-02-23T00:54:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.432+0000 7fedb4ff9700 3 client.42026.objecter handle_osd_map decoding full epoch 737 2024-02-23T00:54:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.433+0000 7fedb6ffd700 1 --2- 172.21.15.47:0/3015809014 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7feda0062cc0 0x7feda0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.433+0000 7fedb4ff9700 20 client.42026.objecter dump_active .. 0 homeless 2024-02-23T00:54:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.433+0000 7fedb4ff9700 10 client.42026.objecter ms_handle_connect 0x7feda0062cc0 2024-02-23T00:54:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.433+0000 7fedb6ffd700 1 --2- 172.21.15.47:0/3015809014 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7feda0062cc0 0x7feda0065180 secure :-1 s=READY pgs=3181 cs=0 l=1 rev1=1 rx=0x7fedb8061760 tx=0x7fedac009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.437+0000 7fedb4ff9700 1 -- 172.21.15.47:0/3015809014 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7feda8049f60 con 0x7fedb80f7400 2024-02-23T00:54:37.495 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:37.496 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:54:37.496 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:54:37.597 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:54:37.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:37.621+0000 7fedbdf8a700 1 -- 172.21.15.47:0/3015809014 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7fedb805ee80 con 0x7fedb80f7400 2024-02-23T00:54:37.627 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:37.647 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:54:37.649 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:36.377364+0000 mon.a (mon.0) 3557 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:54:37.649 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:36.377487+0000 mon.a (mon.0) 3558 : cluster [DBG] osdmap e736: 8 total, 8 up, 6 in 2024-02-23T00:54:37.649 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:36.379221+0000 mon.c (mon.2) 444 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:54:37.649 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:36.379723+0000 mon.a (mon.0) 3559 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:54:37.650 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:37.650 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:54:37.751 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:54:37.780 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:37.799 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:54:37.801 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:37.802 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:54:37.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:36.377364+0000 mon.a (mon.0) 3557 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:54:37.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:36.377487+0000 mon.a (mon.0) 3558 : cluster [DBG] osdmap e736: 8 total, 8 up, 6 in 2024-02-23T00:54:37.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:36.379221+0000 mon.c (mon.2) 444 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:54:37.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:36.379723+0000 mon.a (mon.0) 3559 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:54:37.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:36.377364+0000 mon.a (mon.0) 3557 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:54:37.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:36.377487+0000 mon.a (mon.0) 3558 : cluster [DBG] osdmap e736: 8 total, 8 up, 6 in 2024-02-23T00:54:37.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:36.379221+0000 mon.c (mon.2) 444 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:54:37.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:36.379723+0000 mon.a (mon.0) 3559 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:54:37.902 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:54:37.931 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:37.950 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:54:37.953 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:37.953 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:54:38.054 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:54:38.081 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:38.104 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:38.105 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:54:38.105 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:54:38.206 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:54:38.235 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:38.334 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:54:38.336 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:38.337 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:54:38.389 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:37.378217+0000 mon.a (mon.0) 3560 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:54:38.390 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:37.378217+0000 mon.a (mon.0) 3560 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:54:38.390 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:37.378301+0000 mon.a (mon.0) 3561 : cluster [DBG] osdmap e737: 8 total, 8 up, 6 in 2024-02-23T00:54:38.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.389+0000 7fedb4ff9700 1 -- 172.21.15.47:0/3015809014 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v738) v1 ==== 91+0+0 (secure 0 0 0) 0x7feda804c7d0 con 0x7fedb80f7400 2024-02-23T00:54:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.392+0000 7fedbdf8a700 10 librados: watch_flush enter 2024-02-23T00:54:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.392+0000 7fedbdf8a700 10 librados: watch_flush exit 2024-02-23T00:54:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.392+0000 7fedbdf8a700 20 client.42026.objecter shutdown clearing up homeless session... 2024-02-23T00:54:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.392+0000 7fedbdf8a700 10 client.42026.objecter successfully canceled tick 2024-02-23T00:54:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.392+0000 7fedbdf8a700 1 -- 172.21.15.47:0/3015809014 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7feda0062cc0 msgr2=0x7feda0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.392+0000 7fedbdf8a700 1 --2- 172.21.15.47:0/3015809014 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7feda0062cc0 0x7feda0065180 secure :-1 s=READY pgs=3181 cs=0 l=1 rev1=1 rx=0x7fedb8061760 tx=0x7fedac009cb0).stop 2024-02-23T00:54:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.392+0000 7fedbdf8a700 1 -- 172.21.15.47:0/3015809014 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fedb80f7400 msgr2=0x7fedb8109160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.392+0000 7fedbdf8a700 1 --2- 172.21.15.47:0/3015809014 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fedb80f7400 0x7fedb8109160 secure :-1 s=READY pgs=2796 cs=0 l=1 rev1=1 rx=0x7feda80048f0 tx=0x7feda8009cb0).stop 2024-02-23T00:54:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.392+0000 7fedbdf8a700 1 -- 172.21.15.47:0/3015809014 shutdown_connections 2024-02-23T00:54:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.392+0000 7fedbdf8a700 1 --2- 172.21.15.47:0/3015809014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedb80fd510 0x7fedb8109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.392+0000 7fedbdf8a700 1 --2- 172.21.15.47:0/3015809014 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7feda0062cc0 0x7feda0065180 unknown :-1 s=CLOSED pgs=3181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.392+0000 7fedbdf8a700 1 --2- 172.21.15.47:0/3015809014 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fedb80f7400 0x7fedb8109160 unknown :-1 s=CLOSED pgs=2796 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.392+0000 7fedbdf8a700 1 --2- 172.21.15.47:0/3015809014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedb80fcbb0 0x7fedb8109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.392+0000 7fedbdf8a700 1 -- 172.21.15.47:0/3015809014 >> 172.21.15.47:0/3015809014 conn(0x7fedb80f2820 msgr2=0x7fedb80fb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.392+0000 7fedbdf8a700 1 -- 172.21.15.47:0/3015809014 shutdown_connections 2024-02-23T00:54:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.392+0000 7fedbdf8a700 1 -- 172.21.15.47:0/3015809014 wait complete. 2024-02-23T00:54:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.392+0000 7fedbdf8a700 1 librados: shutdown 2024-02-23T00:54:38.397 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is unset 2024-02-23T00:54:38.438 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:54:38.467 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:38.487 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:54:38.490 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:38.490 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:54:38.591 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:54:38.622 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:38.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:37.378217+0000 mon.a (mon.0) 3560 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:54:38.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:37.378301+0000 mon.a (mon.0) 3561 : cluster [DBG] osdmap e737: 8 total, 8 up, 6 in 2024-02-23T00:54:38.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:37.381193+0000 mon.c (mon.2) 445 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_count","val": "3"}]: dispatch 2024-02-23T00:54:38.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:37.384265+0000 mon.a (mon.0) 3562 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_count","val": "3"}]: dispatch 2024-02-23T00:54:38.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:37.387492+0000 mgr.y (mgr.14517) 627 : cluster [DBG] pgmap v1115: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/726 objects misplaced (0.138%) 2024-02-23T00:54:38.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:37.622692+0000 mon.c (mon.2) 446 : audit [INF] from='client.? 172.21.15.47:0/3015809014' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:54:38.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:37.623022+0000 mon.a (mon.0) 3563 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:54:38.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.796+0000 7f0b35b45700 1 -- 172.21.15.47:0/4117375600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b300fcbb0 msgr2=0x7f0b300fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:38.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.796+0000 7f0b35b45700 1 --2- 172.21.15.47:0/4117375600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b300fcbb0 0x7f0b300fcf90 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 rx=0x7f0b200047b0 tx=0x7f0b20009cb0).stop 2024-02-23T00:54:38.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.796+0000 7f0b35b45700 1 -- 172.21.15.47:0/4117375600 shutdown_connections 2024-02-23T00:54:38.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.796+0000 7f0b35b45700 1 --2- 172.21.15.47:0/4117375600 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0b300f7400 0x7f0b30103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:38.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.796+0000 7f0b35b45700 1 --2- 172.21.15.47:0/4117375600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b300fcbb0 0x7f0b300fcf90 unknown :-1 s=CLOSED pgs=2627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:38.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.796+0000 7f0b35b45700 1 --2- 172.21.15.47:0/4117375600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b300fd510 0x7f0b300f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:38.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.796+0000 7f0b35b45700 1 -- 172.21.15.47:0/4117375600 >> 172.21.15.47:0/4117375600 conn(0x7f0b300f2820 msgr2=0x7f0b300f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:38.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.796+0000 7f0b35b45700 1 -- 172.21.15.47:0/4117375600 shutdown_connections 2024-02-23T00:54:38.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.796+0000 7f0b35b45700 1 -- 172.21.15.47:0/4117375600 wait complete. 2024-02-23T00:54:38.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.797+0000 7f0b35b45700 1 librados: starting msgr at 2024-02-23T00:54:38.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.797+0000 7f0b35b45700 1 librados: starting objecter 2024-02-23T00:54:38.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.797+0000 7f0b35b45700 1 Processor -- start 2024-02-23T00:54:38.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.797+0000 7f0b35b45700 1 -- start start 2024-02-23T00:54:38.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.797+0000 7f0b35b45700 1 librados: setting wanted keys 2024-02-23T00:54:38.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.797+0000 7f0b35b45700 1 librados: calling monclient init 2024-02-23T00:54:38.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.797+0000 7f0b35b45700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b300f7400 0x7f0b3010dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:38.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.797+0000 7f0b35b45700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b300fcbb0 0x7f0b3010e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:38.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.797+0000 7f0b35b45700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0b300fd510 0x7f0b30108160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:38.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.797+0000 7f0b35b45700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b30108650 con 0x7f0b300fcbb0 2024-02-23T00:54:38.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.797+0000 7f0b35b45700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b301087c0 con 0x7f0b300f7400 2024-02-23T00:54:38.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.797+0000 7f0b35b45700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b30108980 con 0x7f0b300fd510 2024-02-23T00:54:38.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.797+0000 7f0b2ffff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0b300fd510 0x7f0b30108160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:38.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.797+0000 7f0b2ffff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0b300fd510 0x7f0b30108160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:50098/0 (socket says 172.21.15.47:50098) 2024-02-23T00:54:38.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.797+0000 7f0b2ffff700 1 -- 172.21.15.47:0/152260955 learned_addr learned my addr 172.21.15.47:0/152260955 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:54:38.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.797+0000 7f0b2f7fe700 1 --2- 172.21.15.47:0/152260955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b300f7400 0x7f0b3010dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:38.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.797+0000 7f0b2effd700 1 --2- 172.21.15.47:0/152260955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b300fcbb0 0x7f0b3010e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:38.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.798+0000 7f0b2ffff700 1 -- 172.21.15.47:0/152260955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b300f7400 msgr2=0x7f0b3010dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:38.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.798+0000 7f0b2ffff700 1 --2- 172.21.15.47:0/152260955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b300f7400 0x7f0b3010dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:38.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.798+0000 7f0b2ffff700 1 -- 172.21.15.47:0/152260955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b300fcbb0 msgr2=0x7f0b3010e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:38.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.798+0000 7f0b2ffff700 1 --2- 172.21.15.47:0/152260955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b300fcbb0 0x7f0b3010e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:38.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.798+0000 7f0b2ffff700 1 -- 172.21.15.47:0/152260955 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b280097c0 con 0x7f0b300fd510 2024-02-23T00:54:38.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.798+0000 7f0b2ffff700 1 --2- 172.21.15.47:0/152260955 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0b300fd510 0x7f0b30108160 secure :-1 s=READY pgs=2797 cs=0 l=1 rev1=1 rx=0x7f0b28001df0 tx=0x7f0b28004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:38.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.798+0000 7f0b2cff9700 10 client.?.objecter ms_handle_connect 0x7f0b300fd510 2024-02-23T00:54:38.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.798+0000 7f0b2cff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:54:38.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.798+0000 7f0b2cff9700 1 -- 172.21.15.47:0/152260955 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b28009a10 con 0x7f0b300fd510 2024-02-23T00:54:38.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.798+0000 7f0b2cff9700 1 -- 172.21.15.47:0/152260955 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f0b280090d0 con 0x7f0b300fd510 2024-02-23T00:54:38.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.798+0000 7f0b35b45700 1 -- 172.21.15.47:0/152260955 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b20005040 con 0x7f0b300fd510 2024-02-23T00:54:38.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.798+0000 7f0b2cff9700 1 -- 172.21.15.47:0/152260955 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b28007be0 con 0x7f0b300fd510 2024-02-23T00:54:38.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.798+0000 7f0b35b45700 10 client.42032.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:54:38.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.798+0000 7f0b35b45700 1 -- 172.21.15.47:0/152260955 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b30108f70 con 0x7f0b300fd510 2024-02-23T00:54:38.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.798+0000 7f0b35b45700 1 librados: init done 2024-02-23T00:54:38.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.799+0000 7f0b35b45700 1 -- 172.21.15.47:0/152260955 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b3005fa60 con 0x7f0b300fd510 2024-02-23T00:54:38.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.799+0000 7f0b2cff9700 1 -- 172.21.15.47:0/152260955 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f0b2800fb30 con 0x7f0b300fd510 2024-02-23T00:54:38.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.800+0000 7f0b2cff9700 1 --2- 172.21.15.47:0/152260955 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0b1c062cc0 0x7f0b1c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:38.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.800+0000 7f0b2cff9700 1 -- 172.21.15.47:0/152260955 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(738..738 src has 621..738) v4 ==== 9752+0+0 (secure 0 0 0) 0x7f0b2807e1a0 con 0x7f0b300fd510 2024-02-23T00:54:38.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.800+0000 7f0b2cff9700 10 client.42032.objecter ms_dispatch 0x7f0b30001550 osd_map(738..738 src has 621..738) v4 2024-02-23T00:54:38.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.800+0000 7f0b2cff9700 3 client.42032.objecter handle_osd_map got epochs [738,738] > 0 2024-02-23T00:54:38.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.800+0000 7f0b2cff9700 3 client.42032.objecter handle_osd_map decoding full epoch 738 2024-02-23T00:54:38.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.800+0000 7f0b2f7fe700 1 --2- 172.21.15.47:0/152260955 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0b1c062cc0 0x7f0b1c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:38.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.800+0000 7f0b2cff9700 20 client.42032.objecter dump_active .. 0 homeless 2024-02-23T00:54:38.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.800+0000 7f0b2cff9700 10 client.42032.objecter ms_handle_connect 0x7f0b1c062cc0 2024-02-23T00:54:38.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.800+0000 7f0b2f7fe700 1 --2- 172.21.15.47:0/152260955 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0b1c062cc0 0x7f0b1c065180 secure :-1 s=READY pgs=3182 cs=0 l=1 rev1=1 rx=0x7f0b20004e20 tx=0x7f0b20009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:38.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:37.381193+0000 mon.c (mon.2) 445 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_count","val": "3"}]: dispatch 2024-02-23T00:54:38.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:37.384265+0000 mon.a (mon.0) 3562 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_count","val": "3"}]: dispatch 2024-02-23T00:54:38.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:37.387492+0000 mgr.y (mgr.14517) 627 : cluster [DBG] pgmap v1115: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/726 objects misplaced (0.138%) 2024-02-23T00:54:38.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:37.622692+0000 mon.c (mon.2) 446 : audit [INF] from='client.? 172.21.15.47:0/3015809014' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:54:38.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:37.623022+0000 mon.a (mon.0) 3563 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:54:38.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:37.378301+0000 mon.a (mon.0) 3561 : cluster [DBG] osdmap e737: 8 total, 8 up, 6 in 2024-02-23T00:54:38.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:37.381193+0000 mon.c (mon.2) 445 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_count","val": "3"}]: dispatch 2024-02-23T00:54:38.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:37.384265+0000 mon.a (mon.0) 3562 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_count","val": "3"}]: dispatch 2024-02-23T00:54:38.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:37.387492+0000 mgr.y (mgr.14517) 627 : cluster [DBG] pgmap v1115: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/726 objects misplaced (0.138%) 2024-02-23T00:54:38.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:37.622692+0000 mon.c (mon.2) 446 : audit [INF] from='client.? 172.21.15.47:0/3015809014' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:54:38.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:37.623022+0000 mon.a (mon.0) 3563 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:54:38.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.804+0000 7f0b2cff9700 1 -- 172.21.15.47:0/152260955 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f0b2804d5a0 con 0x7f0b300fd510 2024-02-23T00:54:38.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.977+0000 7f0b35b45700 1 -- 172.21.15.47:0/152260955 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0b30109640 con 0x7f0b1c062cc0 2024-02-23T00:54:38.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.990+0000 7f0b2cff9700 1 -- 172.21.15.47:0/152260955 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+472923 (secure 0 0 0) 0x7f0b30109640 con 0x7f0b1c062cc0 2024-02-23T00:54:38.999 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:54:39.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.993+0000 7f0b35b45700 10 librados: watch_flush enter 2024-02-23T00:54:39.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.993+0000 7f0b35b45700 10 librados: watch_flush exit 2024-02-23T00:54:39.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.993+0000 7f0b35b45700 20 client.42032.objecter shutdown clearing up homeless session... 2024-02-23T00:54:39.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.993+0000 7f0b35b45700 10 client.42032.objecter successfully canceled tick 2024-02-23T00:54:39.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.993+0000 7f0b35b45700 1 -- 172.21.15.47:0/152260955 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0b1c062cc0 msgr2=0x7f0b1c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:39.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.993+0000 7f0b35b45700 1 --2- 172.21.15.47:0/152260955 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0b1c062cc0 0x7f0b1c065180 secure :-1 s=READY pgs=3182 cs=0 l=1 rev1=1 rx=0x7f0b20004e20 tx=0x7f0b20009cb0).stop 2024-02-23T00:54:39.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.993+0000 7f0b35b45700 1 -- 172.21.15.47:0/152260955 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0b300fd510 msgr2=0x7f0b30108160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:39.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.993+0000 7f0b35b45700 1 --2- 172.21.15.47:0/152260955 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0b300fd510 0x7f0b30108160 secure :-1 s=READY pgs=2797 cs=0 l=1 rev1=1 rx=0x7f0b28001df0 tx=0x7f0b28004d10).stop 2024-02-23T00:54:39.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.993+0000 7f0b35b45700 1 -- 172.21.15.47:0/152260955 shutdown_connections 2024-02-23T00:54:39.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.993+0000 7f0b35b45700 1 --2- 172.21.15.47:0/152260955 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0b1c062cc0 0x7f0b1c065180 unknown :-1 s=CLOSED pgs=3182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:39.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.993+0000 7f0b35b45700 1 --2- 172.21.15.47:0/152260955 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0b300fd510 0x7f0b30108160 unknown :-1 s=CLOSED pgs=2797 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:39.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.993+0000 7f0b35b45700 1 --2- 172.21.15.47:0/152260955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b300f7400 0x7f0b3010dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:39.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.993+0000 7f0b35b45700 1 --2- 172.21.15.47:0/152260955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b300fcbb0 0x7f0b3010e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:39.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.993+0000 7f0b35b45700 1 -- 172.21.15.47:0/152260955 >> 172.21.15.47:0/152260955 conn(0x7f0b300f2820 msgr2=0x7f0b30060e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:39.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.994+0000 7f0b35b45700 1 -- 172.21.15.47:0/152260955 shutdown_connections 2024-02-23T00:54:39.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.994+0000 7f0b35b45700 1 -- 172.21.15.47:0/152260955 wait complete. 2024-02-23T00:54:39.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:38.994+0000 7f0b35b45700 1 librados: shutdown 2024-02-23T00:54:39.005 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-02-23T00:54:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.235+0000 7f5a2c992700 1 -- 172.21.15.47:0/2208839976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a280f8bc0 msgr2=0x7f5a280f8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.235+0000 7f5a2c992700 1 --2- 172.21.15.47:0/2208839976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a280f8bc0 0x7f5a280f8fa0 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 rx=0x7f5a1c0047b0 tx=0x7f5a1c009cb0).stop 2024-02-23T00:54:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.235+0000 7f5a2c992700 1 -- 172.21.15.47:0/2208839976 shutdown_connections 2024-02-23T00:54:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.235+0000 7f5a2c992700 1 --2- 172.21.15.47:0/2208839976 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a28102120 0x7f5a28104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.235+0000 7f5a2c992700 1 --2- 172.21.15.47:0/2208839976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a280f8bc0 0x7f5a280f8fa0 unknown :-1 s=CLOSED pgs=2628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.235+0000 7f5a2c992700 1 --2- 172.21.15.47:0/2208839976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a280f9520 0x7f5a28101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.235+0000 7f5a2c992700 1 -- 172.21.15.47:0/2208839976 >> 172.21.15.47:0/2208839976 conn(0x7f5a280f26f0 msgr2=0x7f5a280f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.235+0000 7f5a2c992700 1 -- 172.21.15.47:0/2208839976 shutdown_connections 2024-02-23T00:54:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.235+0000 7f5a2c992700 1 -- 172.21.15.47:0/2208839976 wait complete. 2024-02-23T00:54:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.235+0000 7f5a2c992700 1 librados: starting msgr at 2024-02-23T00:54:39.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.235+0000 7f5a2c992700 1 librados: starting objecter 2024-02-23T00:54:39.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.236+0000 7f5a2c992700 1 Processor -- start 2024-02-23T00:54:39.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.236+0000 7f5a2c992700 1 -- start start 2024-02-23T00:54:39.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.236+0000 7f5a2c992700 1 librados: setting wanted keys 2024-02-23T00:54:39.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.236+0000 7f5a2c992700 1 librados: calling monclient init 2024-02-23T00:54:39.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.236+0000 7f5a2c992700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a280f8bc0 0x7f5a28110140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:39.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.236+0000 7f5a2c992700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a280f9520 0x7f5a28110630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:39.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.236+0000 7f5a2c992700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a28102120 0x7f5a2810a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:39.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.236+0000 7f5a2c992700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2810a7d0 con 0x7f5a280f8bc0 2024-02-23T00:54:39.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.236+0000 7f5a2c992700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2810a940 con 0x7f5a28102120 2024-02-23T00:54:39.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.236+0000 7f5a2c992700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2810ab00 con 0x7f5a280f9520 2024-02-23T00:54:39.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.236+0000 7f5a2659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a280f8bc0 0x7f5a28110140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:39.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.236+0000 7f5a2659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a280f8bc0 0x7f5a28110140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33466/0 (socket says 172.21.15.47:33466) 2024-02-23T00:54:39.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.236+0000 7f5a2659c700 1 -- 172.21.15.47:0/2596560130 learned_addr learned my addr 172.21.15.47:0/2596560130 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:54:39.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.236+0000 7f5a25d9b700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a280f9520 0x7f5a28110630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:39.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.236+0000 7f5a26d9d700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a28102120 0x7f5a2810a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:39.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.236+0000 7f5a2659c700 1 -- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a280f9520 msgr2=0x7f5a28110630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:39.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.236+0000 7f5a2659c700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a280f9520 0x7f5a28110630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:39.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.236+0000 7f5a2659c700 1 -- 172.21.15.47:0/2596560130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a28102120 msgr2=0x7f5a2810a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:39.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.236+0000 7f5a2659c700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a28102120 0x7f5a2810a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:39.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.236+0000 7f5a2659c700 1 -- 172.21.15.47:0/2596560130 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a200097c0 con 0x7f5a280f8bc0 2024-02-23T00:54:39.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.236+0000 7f5a25d9b700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a280f9520 0x7f5a28110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:54:39.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.236+0000 7f5a2659c700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a280f8bc0 0x7f5a28110140 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 rx=0x7f5a1c009f20 tx=0x7f5a1c009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:39.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.236+0000 7f5a177fe700 10 client.?.objecter ms_handle_connect 0x7f5a280f8bc0 2024-02-23T00:54:39.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.236+0000 7f5a177fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:54:39.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.237+0000 7f5a177fe700 1 -- 172.21.15.47:0/2596560130 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c018070 con 0x7f5a280f8bc0 2024-02-23T00:54:39.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.237+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a1c005040 con 0x7f5a280f8bc0 2024-02-23T00:54:39.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.237+0000 7f5a177fe700 1 -- 172.21.15.47:0/2596560130 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f5a1c0096d0 con 0x7f5a280f8bc0 2024-02-23T00:54:39.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.237+0000 7f5a177fe700 1 -- 172.21.15.47:0/2596560130 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c01dba0 con 0x7f5a280f8bc0 2024-02-23T00:54:39.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.237+0000 7f5a2c992700 10 client.31719.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:54:39.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.237+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a2810afa0 con 0x7f5a280f8bc0 2024-02-23T00:54:39.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.237+0000 7f5a2c992700 1 librados: init done 2024-02-23T00:54:39.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.238+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f5a280606d0 con 0x7f5a280f8bc0 2024-02-23T00:54:39.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.238+0000 7f5a177fe700 1 -- 172.21.15.47:0/2596560130 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f5a1c009240 con 0x7f5a280f8bc0 2024-02-23T00:54:39.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.238+0000 7f5a177fe700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5a10062cc0 0x7f5a10065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:39.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.238+0000 7f5a177fe700 1 -- 172.21.15.47:0/2596560130 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(738..738 src has 621..738) v4 ==== 9752+0+0 (secure 0 0 0) 0x7f5a1c07ecd0 con 0x7f5a280f8bc0 2024-02-23T00:54:39.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.238+0000 7f5a177fe700 10 client.31719.objecter ms_dispatch 0x7f5a28001550 osd_map(738..738 src has 621..738) v4 2024-02-23T00:54:39.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.238+0000 7f5a177fe700 3 client.31719.objecter handle_osd_map got epochs [738,738] > 0 2024-02-23T00:54:39.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.238+0000 7f5a177fe700 3 client.31719.objecter handle_osd_map decoding full epoch 738 2024-02-23T00:54:39.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.239+0000 7f5a25d9b700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5a10062cc0 0x7f5a10065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:39.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.239+0000 7f5a177fe700 20 client.31719.objecter dump_active .. 0 homeless 2024-02-23T00:54:39.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.239+0000 7f5a177fe700 1 -- 172.21.15.47:0/2596560130 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v738) v1 ==== 54+0+16 (secure 0 0 0) 0x7f5a1c081360 con 0x7f5a280f8bc0 2024-02-23T00:54:39.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.239+0000 7f5a25d9b700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5a10062cc0 0x7f5a10065180 secure :-1 s=READY pgs=3183 cs=0 l=1 rev1=1 rx=0x7f5a2810ba80 tx=0x7f5a18009040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:39.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.239+0000 7f5a177fe700 10 client.31719.objecter ms_handle_connect 0x7f5a10062cc0 2024-02-23T00:54:39.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.239+0000 7f5a2c992700 10 client.31719.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.239+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_assign -1 1 2024-02-23T00:54:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.239+0000 7f5a2c992700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f5a280fe4f0 0x7f5a280fe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.239+0000 7f5a2c992700 20 client.31719.objecter _get_session s=0x7f5a2805ee80 osd=0 3 2024-02-23T00:54:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.239+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a2805ee80 osd=0 3 2024-02-23T00:54:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.239+0000 7f5a2c992700 20 client.31719.objecter _get_session s=0x7f5a2805ee80 osd=0 3 2024-02-23T00:54:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.239+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_remove -1 1 2024-02-23T00:54:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.239+0000 7f5a2c992700 20 client.31719.objecter get_session s=0x7f5a2805ee80 osd=0 3 2024-02-23T00:54:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.239+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_assign 0 1 2024-02-23T00:54:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.239+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a2805ee80 osd=0 4 2024-02-23T00:54:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.239+0000 7f5a2c992700 10 client.31719.objecter _send_command 1 2024-02-23T00:54:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.239+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5a28106f10 con 0x7f5a280fe4f0 2024-02-23T00:54:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.239+0000 7f5a26d9d700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f5a280fe4f0 0x7f5a280fe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.239+0000 7f5a177fe700 10 client.31719.objecter ms_handle_connect 0x7f5a280fe4f0 2024-02-23T00:54:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.239+0000 7f5a26d9d700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f5a280fe4f0 0x7f5a280fe8f0 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:39.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.240+0000 7f5a177fe700 1 -- 172.21.15.47:0/2596560130 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5a28106f10 con 0x7f5a280fe4f0 2024-02-23T00:54:39.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.240+0000 7f5a177fe700 10 client.31719.objecter ms_dispatch 0x7f5a28001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:54:39.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.240+0000 7f5a177fe700 10 client.31719.objecter _finish_command 1 = system:0 2024-02-23T00:54:39.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.240+0000 7f5a177fe700 20 client.31719.objecter put_session s=0x7f5a2805ee80 osd=0 4 2024-02-23T00:54:39.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.240+0000 7f5a177fe700 15 client.31719.objecter _session_command_op_remove 0 1 2024-02-23T00:54:39.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.255+0000 7f5a2c992700 10 client.31719.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:54:39.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.255+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_assign -1 2 2024-02-23T00:54:39.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.255+0000 7f5a2c992700 20 client.31719.objecter _get_session s=0x7f5a2805ee80 osd=0 3 2024-02-23T00:54:39.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.255+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a2805ee80 osd=0 3 2024-02-23T00:54:39.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.255+0000 7f5a2c992700 20 client.31719.objecter _get_session s=0x7f5a2805ee80 osd=0 3 2024-02-23T00:54:39.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.255+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_remove -1 2 2024-02-23T00:54:39.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.255+0000 7f5a2c992700 20 client.31719.objecter get_session s=0x7f5a2805ee80 osd=0 3 2024-02-23T00:54:39.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.255+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_assign 0 2 2024-02-23T00:54:39.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.255+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a2805ee80 osd=0 4 2024-02-23T00:54:39.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.255+0000 7f5a2c992700 10 client.31719.objecter _send_command 2 2024-02-23T00:54:39.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.255+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f5a280273c0 con 0x7f5a280fe4f0 2024-02-23T00:54:39.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.257+0000 7f5a177fe700 1 -- 172.21.15.47:0/2596560130 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f5a280273c0 con 0x7f5a280fe4f0 2024-02-23T00:54:39.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.257+0000 7f5a177fe700 10 client.31719.objecter ms_dispatch 0x7f5a28001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:54:39.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.257+0000 7f5a177fe700 10 client.31719.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:54:39.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.257+0000 7f5a177fe700 20 client.31719.objecter put_session s=0x7f5a2805ee80 osd=0 4 2024-02-23T00:54:39.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.257+0000 7f5a177fe700 15 client.31719.objecter _session_command_op_remove 0 2 2024-02-23T00:54:39.260 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:54:39.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.257+0000 7f5a2c992700 10 client.31719.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:39.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.257+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_assign -1 3 2024-02-23T00:54:39.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.257+0000 7f5a2c992700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f5a280598f0 0x7f5a28059d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:39.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.257+0000 7f5a2c992700 20 client.31719.objecter _get_session s=0x7f5a28028740 osd=1 3 2024-02-23T00:54:39.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.257+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a28028740 osd=1 3 2024-02-23T00:54:39.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.257+0000 7f5a2c992700 20 client.31719.objecter _get_session s=0x7f5a28028740 osd=1 3 2024-02-23T00:54:39.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.257+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_remove -1 3 2024-02-23T00:54:39.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.257+0000 7f5a2c992700 20 client.31719.objecter get_session s=0x7f5a28028740 osd=1 3 2024-02-23T00:54:39.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.257+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_assign 1 3 2024-02-23T00:54:39.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.257+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a28028740 osd=1 4 2024-02-23T00:54:39.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.257+0000 7f5a2c992700 10 client.31719.objecter _send_command 3 2024-02-23T00:54:39.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.257+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5a2811d6f0 con 0x7f5a280598f0 2024-02-23T00:54:39.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.257+0000 7f5a2659c700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f5a280598f0 0x7f5a28059d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:39.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.258+0000 7f5a177fe700 10 client.31719.objecter ms_handle_connect 0x7f5a280598f0 2024-02-23T00:54:39.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.258+0000 7f5a2659c700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f5a280598f0 0x7f5a28059d10 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:39.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.258+0000 7f5a177fe700 1 -- 172.21.15.47:0/2596560130 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5a2811d6f0 con 0x7f5a280598f0 2024-02-23T00:54:39.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.258+0000 7f5a177fe700 10 client.31719.objecter ms_dispatch 0x7f5a28001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:54:39.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.258+0000 7f5a177fe700 10 client.31719.objecter _finish_command 3 = system:0 2024-02-23T00:54:39.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.258+0000 7f5a177fe700 20 client.31719.objecter put_session s=0x7f5a28028740 osd=1 4 2024-02-23T00:54:39.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.258+0000 7f5a177fe700 15 client.31719.objecter _session_command_op_remove 1 3 2024-02-23T00:54:39.272 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:54:39.276 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:39.276 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:54:39.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.276+0000 7f5a2c992700 10 client.31719.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:54:39.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.276+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_assign -1 4 2024-02-23T00:54:39.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.276+0000 7f5a2c992700 20 client.31719.objecter _get_session s=0x7f5a28028740 osd=1 3 2024-02-23T00:54:39.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.276+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a28028740 osd=1 3 2024-02-23T00:54:39.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.276+0000 7f5a2c992700 20 client.31719.objecter _get_session s=0x7f5a28028740 osd=1 3 2024-02-23T00:54:39.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.276+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_remove -1 4 2024-02-23T00:54:39.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.276+0000 7f5a2c992700 20 client.31719.objecter get_session s=0x7f5a28028740 osd=1 3 2024-02-23T00:54:39.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.276+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_assign 1 4 2024-02-23T00:54:39.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.276+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a28028740 osd=1 4 2024-02-23T00:54:39.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.276+0000 7f5a2c992700 10 client.31719.objecter _send_command 4 2024-02-23T00:54:39.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.276+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f5a28059360 con 0x7f5a280598f0 2024-02-23T00:54:39.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.278+0000 7f5a177fe700 1 -- 172.21.15.47:0/2596560130 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f5a28059360 con 0x7f5a280598f0 2024-02-23T00:54:39.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.278+0000 7f5a177fe700 10 client.31719.objecter ms_dispatch 0x7f5a28001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:54:39.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.278+0000 7f5a177fe700 10 client.31719.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:54:39.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.278+0000 7f5a177fe700 20 client.31719.objecter put_session s=0x7f5a28028740 osd=1 4 2024-02-23T00:54:39.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.278+0000 7f5a177fe700 15 client.31719.objecter _session_command_op_remove 1 4 2024-02-23T00:54:39.281 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:54:39.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.278+0000 7f5a2c992700 10 client.31719.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:39.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.278+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_assign -1 5 2024-02-23T00:54:39.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.278+0000 7f5a2c992700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f5a280282a0 0x7f5a2811c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:39.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.278+0000 7f5a2c992700 20 client.31719.objecter _get_session s=0x7f5a28059710 osd=2 3 2024-02-23T00:54:39.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.278+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a28059710 osd=2 3 2024-02-23T00:54:39.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.278+0000 7f5a2c992700 20 client.31719.objecter _get_session s=0x7f5a28059710 osd=2 3 2024-02-23T00:54:39.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.278+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_remove -1 5 2024-02-23T00:54:39.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.278+0000 7f5a2c992700 20 client.31719.objecter get_session s=0x7f5a28059710 osd=2 3 2024-02-23T00:54:39.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.278+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_assign 2 5 2024-02-23T00:54:39.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.278+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a28059710 osd=2 4 2024-02-23T00:54:39.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.278+0000 7f5a2c992700 10 client.31719.objecter _send_command 5 2024-02-23T00:54:39.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.278+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5a2811c600 con 0x7f5a280282a0 2024-02-23T00:54:39.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.278+0000 7f5a25d9b700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f5a280282a0 0x7f5a2811c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:39.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.278+0000 7f5a177fe700 10 client.31719.objecter ms_handle_connect 0x7f5a280282a0 2024-02-23T00:54:39.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.278+0000 7f5a25d9b700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f5a280282a0 0x7f5a2811c110 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:39.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.279+0000 7f5a177fe700 1 -- 172.21.15.47:0/2596560130 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5a2811c600 con 0x7f5a280282a0 2024-02-23T00:54:39.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.279+0000 7f5a177fe700 10 client.31719.objecter ms_dispatch 0x7f5a28001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:54:39.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.279+0000 7f5a177fe700 10 client.31719.objecter _finish_command 5 = system:0 2024-02-23T00:54:39.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.279+0000 7f5a177fe700 20 client.31719.objecter put_session s=0x7f5a28059710 osd=2 4 2024-02-23T00:54:39.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.279+0000 7f5a177fe700 15 client.31719.objecter _session_command_op_remove 2 5 2024-02-23T00:54:39.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.294+0000 7f5a2c992700 10 client.31719.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:54:39.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.294+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_assign -1 6 2024-02-23T00:54:39.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.294+0000 7f5a2c992700 20 client.31719.objecter _get_session s=0x7f5a28059710 osd=2 3 2024-02-23T00:54:39.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.294+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a28059710 osd=2 3 2024-02-23T00:54:39.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.294+0000 7f5a2c992700 20 client.31719.objecter _get_session s=0x7f5a28059710 osd=2 3 2024-02-23T00:54:39.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.294+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_remove -1 6 2024-02-23T00:54:39.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.294+0000 7f5a2c992700 20 client.31719.objecter get_session s=0x7f5a28059710 osd=2 3 2024-02-23T00:54:39.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.294+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_assign 2 6 2024-02-23T00:54:39.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.294+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a28059710 osd=2 4 2024-02-23T00:54:39.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.294+0000 7f5a2c992700 10 client.31719.objecter _send_command 6 2024-02-23T00:54:39.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.294+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f5a28027800 con 0x7f5a280282a0 2024-02-23T00:54:39.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.296+0000 7f5a177fe700 1 -- 172.21.15.47:0/2596560130 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f5a28027800 con 0x7f5a280282a0 2024-02-23T00:54:39.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.296+0000 7f5a177fe700 10 client.31719.objecter ms_dispatch 0x7f5a28001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:54:39.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.296+0000 7f5a177fe700 10 client.31719.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:54:39.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.296+0000 7f5a177fe700 20 client.31719.objecter put_session s=0x7f5a28059710 osd=2 4 2024-02-23T00:54:39.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.296+0000 7f5a177fe700 15 client.31719.objecter _session_command_op_remove 2 6 2024-02-23T00:54:39.299 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:54:39.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.296+0000 7f5a2c992700 10 client.31719.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:39.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.296+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_assign -1 7 2024-02-23T00:54:39.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.296+0000 7f5a2c992700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f5a2811f870 0x7f5a28121c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:39.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.296+0000 7f5a2c992700 20 client.31719.objecter _get_session s=0x7f5a2811cf70 osd=3 3 2024-02-23T00:54:39.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.296+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a2811cf70 osd=3 3 2024-02-23T00:54:39.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.296+0000 7f5a2c992700 20 client.31719.objecter _get_session s=0x7f5a2811cf70 osd=3 3 2024-02-23T00:54:39.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.296+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_remove -1 7 2024-02-23T00:54:39.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.296+0000 7f5a2c992700 20 client.31719.objecter get_session s=0x7f5a2811cf70 osd=3 3 2024-02-23T00:54:39.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.296+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_assign 3 7 2024-02-23T00:54:39.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.296+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a2811cf70 osd=3 4 2024-02-23T00:54:39.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.296+0000 7f5a2c992700 10 client.31719.objecter _send_command 7 2024-02-23T00:54:39.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.296+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5a281248b0 con 0x7f5a2811f870 2024-02-23T00:54:39.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.296+0000 7f5a26d9d700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f5a2811f870 0x7f5a28121c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:39.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.297+0000 7f5a177fe700 10 client.31719.objecter ms_handle_connect 0x7f5a2811f870 2024-02-23T00:54:39.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.297+0000 7f5a26d9d700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f5a2811f870 0x7f5a28121c60 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:39.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.297+0000 7f5a177fe700 1 -- 172.21.15.47:0/2596560130 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5a281248b0 con 0x7f5a2811f870 2024-02-23T00:54:39.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.297+0000 7f5a177fe700 10 client.31719.objecter ms_dispatch 0x7f5a28001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:54:39.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.297+0000 7f5a177fe700 10 client.31719.objecter _finish_command 7 = system:0 2024-02-23T00:54:39.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.297+0000 7f5a177fe700 20 client.31719.objecter put_session s=0x7f5a2811cf70 osd=3 4 2024-02-23T00:54:39.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.297+0000 7f5a177fe700 15 client.31719.objecter _session_command_op_remove 3 7 2024-02-23T00:54:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.312+0000 7f5a2c992700 10 client.31719.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:54:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.312+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_assign -1 8 2024-02-23T00:54:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.312+0000 7f5a2c992700 20 client.31719.objecter _get_session s=0x7f5a2811cf70 osd=3 3 2024-02-23T00:54:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.312+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a2811cf70 osd=3 3 2024-02-23T00:54:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.312+0000 7f5a2c992700 20 client.31719.objecter _get_session s=0x7f5a2811cf70 osd=3 3 2024-02-23T00:54:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.312+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_remove -1 8 2024-02-23T00:54:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.312+0000 7f5a2c992700 20 client.31719.objecter get_session s=0x7f5a2811cf70 osd=3 3 2024-02-23T00:54:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.312+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_assign 3 8 2024-02-23T00:54:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.312+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a2811cf70 osd=3 4 2024-02-23T00:54:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.312+0000 7f5a2c992700 10 client.31719.objecter _send_command 8 2024-02-23T00:54:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.312+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f5a2811c9e0 con 0x7f5a2811f870 2024-02-23T00:54:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.313+0000 7f5a177fe700 1 -- 172.21.15.47:0/2596560130 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f5a2811c9e0 con 0x7f5a2811f870 2024-02-23T00:54:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.313+0000 7f5a177fe700 10 client.31719.objecter ms_dispatch 0x7f5a28001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:54:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.313+0000 7f5a177fe700 10 client.31719.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:54:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.313+0000 7f5a177fe700 20 client.31719.objecter put_session s=0x7f5a2811cf70 osd=3 4 2024-02-23T00:54:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.313+0000 7f5a177fe700 15 client.31719.objecter _session_command_op_remove 3 8 2024-02-23T00:54:39.317 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:54:39.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.314+0000 7f5a2c992700 10 client.31719.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:39.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.314+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_assign -1 9 2024-02-23T00:54:39.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.314+0000 7f5a2c992700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f5a2811bc70 0x7f5a281232d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:39.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.314+0000 7f5a2c992700 20 client.31719.objecter _get_session s=0x7f5a2811c9e0 osd=4 3 2024-02-23T00:54:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.314+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a2811c9e0 osd=4 3 2024-02-23T00:54:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.314+0000 7f5a2c992700 20 client.31719.objecter _get_session s=0x7f5a2811c9e0 osd=4 3 2024-02-23T00:54:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.314+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_remove -1 9 2024-02-23T00:54:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.314+0000 7f5a2c992700 20 client.31719.objecter get_session s=0x7f5a2811c9e0 osd=4 3 2024-02-23T00:54:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.314+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_assign 4 9 2024-02-23T00:54:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.314+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a2811c9e0 osd=4 4 2024-02-23T00:54:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.314+0000 7f5a2c992700 10 client.31719.objecter _send_command 9 2024-02-23T00:54:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.314+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5a2811ccc0 con 0x7f5a2811bc70 2024-02-23T00:54:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.314+0000 7f5a2659c700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f5a2811bc70 0x7f5a281232d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.314+0000 7f5a177fe700 10 client.31719.objecter ms_handle_connect 0x7f5a2811bc70 2024-02-23T00:54:39.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.314+0000 7f5a2659c700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f5a2811bc70 0x7f5a281232d0 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:39.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.315+0000 7f5a177fe700 1 -- 172.21.15.47:0/2596560130 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5a2811ccc0 con 0x7f5a2811bc70 2024-02-23T00:54:39.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.315+0000 7f5a177fe700 10 client.31719.objecter ms_dispatch 0x7f5a28001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:54:39.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.315+0000 7f5a177fe700 10 client.31719.objecter _finish_command 9 = system:0 2024-02-23T00:54:39.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.315+0000 7f5a177fe700 20 client.31719.objecter put_session s=0x7f5a2811c9e0 osd=4 4 2024-02-23T00:54:39.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.315+0000 7f5a177fe700 15 client.31719.objecter _session_command_op_remove 4 9 2024-02-23T00:54:39.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.330+0000 7f5a2c992700 10 client.31719.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:54:39.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.330+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_assign -1 10 2024-02-23T00:54:39.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.330+0000 7f5a2c992700 20 client.31719.objecter _get_session s=0x7f5a2811c9e0 osd=4 3 2024-02-23T00:54:39.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.330+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a2811c9e0 osd=4 3 2024-02-23T00:54:39.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.330+0000 7f5a2c992700 20 client.31719.objecter _get_session s=0x7f5a2811c9e0 osd=4 3 2024-02-23T00:54:39.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.330+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_remove -1 10 2024-02-23T00:54:39.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.330+0000 7f5a2c992700 20 client.31719.objecter get_session s=0x7f5a2811c9e0 osd=4 3 2024-02-23T00:54:39.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.330+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_assign 4 10 2024-02-23T00:54:39.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.330+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a2811c9e0 osd=4 4 2024-02-23T00:54:39.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.330+0000 7f5a2c992700 10 client.31719.objecter _send_command 10 2024-02-23T00:54:39.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.330+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f5a2811ccc0 con 0x7f5a2811bc70 2024-02-23T00:54:39.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.333+0000 7f5a177fe700 1 -- 172.21.15.47:0/2596560130 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f5a2811ccc0 con 0x7f5a2811bc70 2024-02-23T00:54:39.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.333+0000 7f5a177fe700 10 client.31719.objecter ms_dispatch 0x7f5a28001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:54:39.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.333+0000 7f5a177fe700 10 client.31719.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:54:39.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.333+0000 7f5a177fe700 20 client.31719.objecter put_session s=0x7f5a2811c9e0 osd=4 4 2024-02-23T00:54:39.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.333+0000 7f5a177fe700 15 client.31719.objecter _session_command_op_remove 4 10 2024-02-23T00:54:39.336 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:54:39.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.333+0000 7f5a2c992700 10 client.31719.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:39.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.333+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_assign -1 11 2024-02-23T00:54:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.333+0000 7f5a2c992700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f5a28123cd0 0x7f5a28124130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.333+0000 7f5a2c992700 20 client.31719.objecter _get_session s=0x7f5a2811ccc0 osd=5 3 2024-02-23T00:54:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.333+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a2811ccc0 osd=5 3 2024-02-23T00:54:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.333+0000 7f5a2c992700 20 client.31719.objecter _get_session s=0x7f5a2811ccc0 osd=5 3 2024-02-23T00:54:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.333+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_remove -1 11 2024-02-23T00:54:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.333+0000 7f5a2c992700 20 client.31719.objecter get_session s=0x7f5a2811ccc0 osd=5 3 2024-02-23T00:54:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.333+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_assign 5 11 2024-02-23T00:54:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.333+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a2811ccc0 osd=5 4 2024-02-23T00:54:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.333+0000 7f5a2c992700 10 client.31719.objecter _send_command 11 2024-02-23T00:54:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.333+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5a2812b1d0 con 0x7f5a28123cd0 2024-02-23T00:54:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.333+0000 7f5a25d9b700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f5a28123cd0 0x7f5a28124130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.333+0000 7f5a177fe700 10 client.31719.objecter ms_handle_connect 0x7f5a28123cd0 2024-02-23T00:54:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.333+0000 7f5a25d9b700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f5a28123cd0 0x7f5a28124130 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.334+0000 7f5a177fe700 1 -- 172.21.15.47:0/2596560130 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5a2812b1d0 con 0x7f5a28123cd0 2024-02-23T00:54:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.334+0000 7f5a177fe700 10 client.31719.objecter ms_dispatch 0x7f5a28001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:54:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.334+0000 7f5a177fe700 10 client.31719.objecter _finish_command 11 = system:0 2024-02-23T00:54:39.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.334+0000 7f5a177fe700 20 client.31719.objecter put_session s=0x7f5a2811ccc0 osd=5 4 2024-02-23T00:54:39.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.334+0000 7f5a177fe700 15 client.31719.objecter _session_command_op_remove 5 11 2024-02-23T00:54:39.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.348+0000 7f5a2c992700 10 client.31719.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:54:39.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.349+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_assign -1 12 2024-02-23T00:54:39.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.349+0000 7f5a2c992700 20 client.31719.objecter _get_session s=0x7f5a2811ccc0 osd=5 3 2024-02-23T00:54:39.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.349+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a2811ccc0 osd=5 3 2024-02-23T00:54:39.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.349+0000 7f5a2c992700 20 client.31719.objecter _get_session s=0x7f5a2811ccc0 osd=5 3 2024-02-23T00:54:39.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.349+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_remove -1 12 2024-02-23T00:54:39.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.349+0000 7f5a2c992700 20 client.31719.objecter get_session s=0x7f5a2811ccc0 osd=5 3 2024-02-23T00:54:39.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.349+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_assign 5 12 2024-02-23T00:54:39.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.349+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a2811ccc0 osd=5 4 2024-02-23T00:54:39.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.349+0000 7f5a2c992700 10 client.31719.objecter _send_command 12 2024-02-23T00:54:39.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.349+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f5a28128a70 con 0x7f5a28123cd0 2024-02-23T00:54:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.351+0000 7f5a177fe700 1 -- 172.21.15.47:0/2596560130 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f5a28128a70 con 0x7f5a28123cd0 2024-02-23T00:54:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.351+0000 7f5a177fe700 10 client.31719.objecter ms_dispatch 0x7f5a28001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:54:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.351+0000 7f5a177fe700 10 client.31719.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:54:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.351+0000 7f5a177fe700 20 client.31719.objecter put_session s=0x7f5a2811ccc0 osd=5 4 2024-02-23T00:54:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.351+0000 7f5a177fe700 15 client.31719.objecter _session_command_op_remove 5 12 2024-02-23T00:54:39.354 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:54:39.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.351+0000 7f5a2c992700 10 client.31719.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:39.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.351+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_assign -1 13 2024-02-23T00:54:39.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.351+0000 7f5a2c992700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f5a281290f0 0x7f5a2812b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:39.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.351+0000 7f5a2c992700 20 client.31719.objecter _get_session s=0x7f5a28128a70 osd=6 3 2024-02-23T00:54:39.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.351+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a28128a70 osd=6 3 2024-02-23T00:54:39.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.351+0000 7f5a2c992700 20 client.31719.objecter _get_session s=0x7f5a28128a70 osd=6 3 2024-02-23T00:54:39.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.351+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_remove -1 13 2024-02-23T00:54:39.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.351+0000 7f5a2c992700 20 client.31719.objecter get_session s=0x7f5a28128a70 osd=6 3 2024-02-23T00:54:39.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.351+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_assign 6 13 2024-02-23T00:54:39.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.351+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a28128a70 osd=6 4 2024-02-23T00:54:39.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.351+0000 7f5a2c992700 10 client.31719.objecter _send_command 13 2024-02-23T00:54:39.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.351+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5a2812e380 con 0x7f5a281290f0 2024-02-23T00:54:39.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.351+0000 7f5a26d9d700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f5a281290f0 0x7f5a2812b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:39.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.351+0000 7f5a177fe700 10 client.31719.objecter ms_handle_connect 0x7f5a281290f0 2024-02-23T00:54:39.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.351+0000 7f5a26d9d700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f5a281290f0 0x7f5a2812b500 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:39.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.352+0000 7f5a177fe700 1 -- 172.21.15.47:0/2596560130 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5a2812e380 con 0x7f5a281290f0 2024-02-23T00:54:39.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.352+0000 7f5a177fe700 10 client.31719.objecter ms_dispatch 0x7f5a28001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:54:39.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.352+0000 7f5a177fe700 10 client.31719.objecter _finish_command 13 = system:0 2024-02-23T00:54:39.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.352+0000 7f5a177fe700 20 client.31719.objecter put_session s=0x7f5a28128a70 osd=6 4 2024-02-23T00:54:39.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.352+0000 7f5a177fe700 15 client.31719.objecter _session_command_op_remove 6 13 2024-02-23T00:54:39.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.367+0000 7f5a2c992700 10 client.31719.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:54:39.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.367+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_assign -1 14 2024-02-23T00:54:39.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.367+0000 7f5a2c992700 20 client.31719.objecter _get_session s=0x7f5a28128a70 osd=6 3 2024-02-23T00:54:39.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.367+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a28128a70 osd=6 3 2024-02-23T00:54:39.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.367+0000 7f5a2c992700 20 client.31719.objecter _get_session s=0x7f5a28128a70 osd=6 3 2024-02-23T00:54:39.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.367+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_remove -1 14 2024-02-23T00:54:39.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.367+0000 7f5a2c992700 20 client.31719.objecter get_session s=0x7f5a28128a70 osd=6 3 2024-02-23T00:54:39.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.367+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_assign 6 14 2024-02-23T00:54:39.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.367+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a28128a70 osd=6 4 2024-02-23T00:54:39.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.367+0000 7f5a2c992700 10 client.31719.objecter _send_command 14 2024-02-23T00:54:39.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.367+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f5a2812e340 con 0x7f5a281290f0 2024-02-23T00:54:39.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.369+0000 7f5a177fe700 1 -- 172.21.15.47:0/2596560130 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f5a2812e340 con 0x7f5a281290f0 2024-02-23T00:54:39.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.369+0000 7f5a177fe700 10 client.31719.objecter ms_dispatch 0x7f5a28001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:54:39.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.369+0000 7f5a177fe700 10 client.31719.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:54:39.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.369+0000 7f5a177fe700 20 client.31719.objecter put_session s=0x7f5a28128a70 osd=6 4 2024-02-23T00:54:39.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.369+0000 7f5a177fe700 15 client.31719.objecter _session_command_op_remove 6 14 2024-02-23T00:54:39.372 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:54:39.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.369+0000 7f5a2c992700 10 client.31719.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:39.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.369+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_assign -1 15 2024-02-23T00:54:39.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.369+0000 7f5a2c992700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f5a2812d620 0x7f5a2812daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:39.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.369+0000 7f5a2c992700 20 client.31719.objecter _get_session s=0x7f5a2812e340 osd=7 3 2024-02-23T00:54:39.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.369+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a2812e340 osd=7 3 2024-02-23T00:54:39.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.369+0000 7f5a2c992700 20 client.31719.objecter _get_session s=0x7f5a2812e340 osd=7 3 2024-02-23T00:54:39.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.369+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_remove -1 15 2024-02-23T00:54:39.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.369+0000 7f5a2c992700 20 client.31719.objecter get_session s=0x7f5a2812e340 osd=7 3 2024-02-23T00:54:39.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.369+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_assign 7 15 2024-02-23T00:54:39.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.369+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a2812e340 osd=7 4 2024-02-23T00:54:39.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.369+0000 7f5a2c992700 10 client.31719.objecter _send_command 15 2024-02-23T00:54:39.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.369+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5a281316b0 con 0x7f5a2812d620 2024-02-23T00:54:39.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.369+0000 7f5a2659c700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f5a2812d620 0x7f5a2812daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:39.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.370+0000 7f5a177fe700 10 client.31719.objecter ms_handle_connect 0x7f5a2812d620 2024-02-23T00:54:39.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.370+0000 7f5a2659c700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f5a2812d620 0x7f5a2812daa0 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:39.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.371+0000 7f5a177fe700 1 -- 172.21.15.47:0/2596560130 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5a281316b0 con 0x7f5a2812d620 2024-02-23T00:54:39.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.371+0000 7f5a177fe700 10 client.31719.objecter ms_dispatch 0x7f5a28001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:54:39.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.371+0000 7f5a177fe700 10 client.31719.objecter _finish_command 15 = system:0 2024-02-23T00:54:39.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.371+0000 7f5a177fe700 20 client.31719.objecter put_session s=0x7f5a2812e340 osd=7 4 2024-02-23T00:54:39.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.371+0000 7f5a177fe700 15 client.31719.objecter _session_command_op_remove 7 15 2024-02-23T00:54:39.377 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:54:39.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.385+0000 7f5a2c992700 10 client.31719.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:54:39.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.385+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_assign -1 16 2024-02-23T00:54:39.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.385+0000 7f5a2c992700 20 client.31719.objecter _get_session s=0x7f5a2812e340 osd=7 3 2024-02-23T00:54:39.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.385+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a2812e340 osd=7 3 2024-02-23T00:54:39.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.385+0000 7f5a2c992700 20 client.31719.objecter _get_session s=0x7f5a2812e340 osd=7 3 2024-02-23T00:54:39.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.385+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_remove -1 16 2024-02-23T00:54:39.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.385+0000 7f5a2c992700 20 client.31719.objecter get_session s=0x7f5a2812e340 osd=7 3 2024-02-23T00:54:39.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.385+0000 7f5a2c992700 15 client.31719.objecter _session_command_op_assign 7 16 2024-02-23T00:54:39.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.385+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a2812e340 osd=7 4 2024-02-23T00:54:39.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.385+0000 7f5a2c992700 10 client.31719.objecter _send_command 16 2024-02-23T00:54:39.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.385+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f5a2812bb00 con 0x7f5a2812d620 2024-02-23T00:54:39.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.387+0000 7f5a177fe700 1 -- 172.21.15.47:0/2596560130 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f5a2812bb00 con 0x7f5a2812d620 2024-02-23T00:54:39.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.387+0000 7f5a177fe700 10 client.31719.objecter ms_dispatch 0x7f5a28001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:54:39.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.387+0000 7f5a177fe700 10 client.31719.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:54:39.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.387+0000 7f5a177fe700 20 client.31719.objecter put_session s=0x7f5a2812e340 osd=7 4 2024-02-23T00:54:39.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.387+0000 7f5a177fe700 15 client.31719.objecter _session_command_op_remove 7 16 2024-02-23T00:54:39.391 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:54:39.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.389+0000 7f5a2c992700 10 librados: watch_flush enter 2024-02-23T00:54:39.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 10 librados: watch_flush exit 2024-02-23T00:54:39.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 10 client.31719.objecter close_session for osd.0 2024-02-23T00:54:39.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f5a280fe4f0 msgr2=0x7f5a280fe8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:39.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f5a280fe4f0 0x7f5a280fe8f0 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:39.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a2805ee80 osd=0 1 2024-02-23T00:54:39.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 10 client.31719.objecter close_session for osd.1 2024-02-23T00:54:39.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f5a280598f0 msgr2=0x7f5a28059d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:39.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f5a280598f0 0x7f5a28059d10 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:39.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a28028740 osd=1 1 2024-02-23T00:54:39.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 10 client.31719.objecter close_session for osd.2 2024-02-23T00:54:39.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f5a280282a0 msgr2=0x7f5a2811c110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:39.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f5a280282a0 0x7f5a2811c110 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:39.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a28059710 osd=2 1 2024-02-23T00:54:39.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 10 client.31719.objecter close_session for osd.3 2024-02-23T00:54:39.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f5a2811f870 msgr2=0x7f5a28121c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:39.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f5a2811f870 0x7f5a28121c60 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:39.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a2811cf70 osd=3 1 2024-02-23T00:54:39.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 10 client.31719.objecter close_session for osd.4 2024-02-23T00:54:39.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f5a2811bc70 msgr2=0x7f5a281232d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:39.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f5a2811bc70 0x7f5a281232d0 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:39.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a2811c9e0 osd=4 1 2024-02-23T00:54:39.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 10 client.31719.objecter close_session for osd.5 2024-02-23T00:54:39.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f5a28123cd0 msgr2=0x7f5a28124130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:39.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f5a28123cd0 0x7f5a28124130 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:39.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a2811ccc0 osd=5 1 2024-02-23T00:54:39.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 10 client.31719.objecter close_session for osd.6 2024-02-23T00:54:39.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f5a281290f0 msgr2=0x7f5a2812b500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:39.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f5a281290f0 0x7f5a2812b500 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:39.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a28128a70 osd=6 1 2024-02-23T00:54:39.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 10 client.31719.objecter close_session for osd.7 2024-02-23T00:54:39.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f5a2812d620 msgr2=0x7f5a2812daa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:39.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f5a2812d620 0x7f5a2812daa0 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:39.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 20 client.31719.objecter put_session s=0x7f5a2812e340 osd=7 1 2024-02-23T00:54:39.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 20 client.31719.objecter shutdown clearing up homeless session... 2024-02-23T00:54:39.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 10 client.31719.objecter successfully canceled tick 2024-02-23T00:54:39.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a26d9d700 1 -- 172.21.15.47:0/2596560130 reap_dead start 2024-02-23T00:54:39.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a26d9d700 1 -- 172.21.15.47:0/2596560130 reap_dead start 2024-02-23T00:54:39.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a26d9d700 1 -- 172.21.15.47:0/2596560130 reap_dead start 2024-02-23T00:54:39.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a26d9d700 1 -- 172.21.15.47:0/2596560130 reap_dead start 2024-02-23T00:54:39.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a26d9d700 1 -- 172.21.15.47:0/2596560130 reap_dead start 2024-02-23T00:54:39.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5a10062cc0 msgr2=0x7f5a10065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:39.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5a10062cc0 0x7f5a10065180 secure :-1 s=READY pgs=3183 cs=0 l=1 rev1=1 rx=0x7f5a2810ba80 tx=0x7f5a18009040).stop 2024-02-23T00:54:39.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a280f8bc0 msgr2=0x7f5a28110140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:39.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.390+0000 7f5a2c992700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a280f8bc0 0x7f5a28110140 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 rx=0x7f5a1c009f20 tx=0x7f5a1c009cb0).stop 2024-02-23T00:54:39.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.391+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 shutdown_connections 2024-02-23T00:54:39.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.391+0000 7f5a2c992700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5a10062cc0 0x7f5a10065180 unknown :-1 s=CLOSED pgs=3183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:39.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.391+0000 7f5a2c992700 1 --2- 172.21.15.47:0/2596560130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a280f8bc0 0x7f5a28110140 unknown :-1 s=CLOSED pgs=2629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:39.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.391+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 >> 172.21.15.47:0/2596560130 conn(0x7f5a280f26f0 msgr2=0x7f5a280f4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:39.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.391+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 shutdown_connections 2024-02-23T00:54:39.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.391+0000 7f5a2c992700 1 -- 172.21.15.47:0/2596560130 wait complete. 2024-02-23T00:54:39.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:39.391+0000 7f5a2c992700 1 librados: shutdown 2024-02-23T00:54:39.398 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:54:39.398 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:54:39.399 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:54:39.399 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:54:39.399 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:54:39.399 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:54:39.399 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:54:39.399 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:54:39.411 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:39.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:38.381839+0000 mon.a (mon.0) 3564 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:54:39.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:38.389991+0000 mon.a (mon.0) 3565 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_count","val": "3"}]': finished 2024-02-23T00:54:39.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:38.390078+0000 mon.a (mon.0) 3566 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:54:39.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:38.390132+0000 mon.a (mon.0) 3567 : cluster [DBG] osdmap e738: 8 total, 8 up, 6 in 2024-02-23T00:54:39.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:38.391370+0000 mon.c (mon.2) 447 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_period","val": "3"}]: dispatch 2024-02-23T00:54:39.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:38.391845+0000 mon.a (mon.0) 3568 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_period","val": "3"}]: dispatch 2024-02-23T00:54:39.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:39.239844+0000 mon.a (mon.0) 3569 : audit [DBG] from='client.? 172.21.15.47:0/2596560130' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:54:39.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:38.381839+0000 mon.a (mon.0) 3564 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:54:39.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:38.389991+0000 mon.a (mon.0) 3565 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_count","val": "3"}]': finished 2024-02-23T00:54:39.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:38.390078+0000 mon.a (mon.0) 3566 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:54:39.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:38.390132+0000 mon.a (mon.0) 3567 : cluster [DBG] osdmap e738: 8 total, 8 up, 6 in 2024-02-23T00:54:39.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:38.391370+0000 mon.c (mon.2) 447 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_period","val": "3"}]: dispatch 2024-02-23T00:54:39.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:38.391845+0000 mon.a (mon.0) 3568 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_period","val": "3"}]: dispatch 2024-02-23T00:54:39.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:39.239844+0000 mon.a (mon.0) 3569 : audit [DBG] from='client.? 172.21.15.47:0/2596560130' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:54:39.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:38.381839+0000 mon.a (mon.0) 3564 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:54:39.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:38.389991+0000 mon.a (mon.0) 3565 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_count","val": "3"}]': finished 2024-02-23T00:54:39.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:38.390078+0000 mon.a (mon.0) 3566 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:54:39.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:38.390132+0000 mon.a (mon.0) 3567 : cluster [DBG] osdmap e738: 8 total, 8 up, 6 in 2024-02-23T00:54:39.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:38.391370+0000 mon.c (mon.2) 447 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_period","val": "3"}]: dispatch 2024-02-23T00:54:39.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:38.391845+0000 mon.a (mon.0) 3568 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_period","val": "3"}]: dispatch 2024-02-23T00:54:39.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:39.239844+0000 mon.a (mon.0) 3569 : audit [DBG] from='client.? 172.21.15.47:0/2596560130' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:54:39.862 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:54:39.864 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:39.865 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:54:39.966 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:54:39.998 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:40.019 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:54:40.021 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:40.022 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:54:40.123 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:54:40.155 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:40.172 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:54:40.174 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:40.175 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:54:40.275 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:54:40.312 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:40.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:38.978534+0000 mgr.y (mgr.14517) 628 : audit [DBG] from='client.42032 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:54:40.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:39.387913+0000 mgr.y (mgr.14517) 629 : cluster [DBG] pgmap v1117: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/726 objects misplaced (0.138%) 2024-02-23T00:54:40.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:39.390570+0000 mon.a (mon.0) 3570 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_period","val": "3"}]': finished 2024-02-23T00:54:40.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:39.390622+0000 mon.a (mon.0) 3571 : cluster [DBG] osdmap e739: 8 total, 8 up, 6 in 2024-02-23T00:54:40.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:39.391691+0000 mon.c (mon.2) 448 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-23T00:54:40.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:39.396974+0000 mon.a (mon.0) 3572 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-23T00:54:40.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:39.423651+0000 osd.4 (osd.4) 303 : cluster [DBG] 18.0 scrub starts 2024-02-23T00:54:40.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:39.424884+0000 osd.4 (osd.4) 304 : cluster [DBG] 18.0 scrub ok 2024-02-23T00:54:40.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:39.498068+0000 osd.6 (osd.6) 283 : cluster [DBG] 18.e scrub starts 2024-02-23T00:54:40.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:39.498995+0000 osd.6 (osd.6) 284 : cluster [DBG] 18.e scrub ok 2024-02-23T00:54:40.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:40.301234+0000 mon.a (mon.0) 3573 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_type","val": "bloom"}]': finished 2024-02-23T00:54:40.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:40.301321+0000 mon.a (mon.0) 3574 : cluster [DBG] osdmap e740: 8 total, 8 up, 6 in 2024-02-23T00:54:40.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:40.303122+0000 mon.c (mon.2) 449 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-23T00:54:40.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:40.303596+0000 mon.a (mon.0) 3575 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-23T00:54:40.458 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:38.978534+0000 mgr.y (mgr.14517) 628 : audit [DBG] from='client.42032 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:54:40.458 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:39.387913+0000 mgr.y (mgr.14517) 629 : cluster [DBG] pgmap v1117: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/726 objects misplaced (0.138%) 2024-02-23T00:54:40.458 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:39.390570+0000 mon.a (mon.0) 3570 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_period","val": "3"}]': finished 2024-02-23T00:54:40.459 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:39.390622+0000 mon.a (mon.0) 3571 : cluster [DBG] osdmap e739: 8 total, 8 up, 6 in 2024-02-23T00:54:40.459 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:39.391691+0000 mon.c (mon.2) 448 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-23T00:54:40.459 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:39.396974+0000 mon.a (mon.0) 3572 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-23T00:54:40.459 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:39.423651+0000 osd.4 (osd.4) 303 : cluster [DBG] 18.0 scrub starts 2024-02-23T00:54:40.459 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:39.424884+0000 osd.4 (osd.4) 304 : cluster [DBG] 18.0 scrub ok 2024-02-23T00:54:40.459 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:39.498068+0000 osd.6 (osd.6) 283 : cluster [DBG] 18.e scrub starts 2024-02-23T00:54:40.459 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:39.498995+0000 osd.6 (osd.6) 284 : cluster [DBG] 18.e scrub ok 2024-02-23T00:54:40.459 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:40.301234+0000 mon.a (mon.0) 3573 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_type","val": "bloom"}]': finished 2024-02-23T00:54:40.459 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:40.301321+0000 mon.a (mon.0) 3574 : cluster [DBG] osdmap e740: 8 total, 8 up, 6 in 2024-02-23T00:54:40.459 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:40.303122+0000 mon.c (mon.2) 449 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-23T00:54:40.459 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:40.303596+0000 mon.a (mon.0) 3575 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-23T00:54:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:38.978534+0000 mgr.y (mgr.14517) 628 : audit [DBG] from='client.42032 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:54:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:39.387913+0000 mgr.y (mgr.14517) 629 : cluster [DBG] pgmap v1117: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/726 objects misplaced (0.138%) 2024-02-23T00:54:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:39.390570+0000 mon.a (mon.0) 3570 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_period","val": "3"}]': finished 2024-02-23T00:54:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:39.390622+0000 mon.a (mon.0) 3571 : cluster [DBG] osdmap e739: 8 total, 8 up, 6 in 2024-02-23T00:54:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:39.391691+0000 mon.c (mon.2) 448 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-23T00:54:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:39.396974+0000 mon.a (mon.0) 3572 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-23T00:54:40.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:39.423651+0000 osd.4 (osd.4) 303 : cluster [DBG] 18.0 scrub starts 2024-02-23T00:54:40.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:39.424884+0000 osd.4 (osd.4) 304 : cluster [DBG] 18.0 scrub ok 2024-02-23T00:54:40.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:39.498068+0000 osd.6 (osd.6) 283 : cluster [DBG] 18.e scrub starts 2024-02-23T00:54:40.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:39.498995+0000 osd.6 (osd.6) 284 : cluster [DBG] 18.e scrub ok 2024-02-23T00:54:40.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:40.301234+0000 mon.a (mon.0) 3573 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_type","val": "bloom"}]': finished 2024-02-23T00:54:40.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:40.301321+0000 mon.a (mon.0) 3574 : cluster [DBG] osdmap e740: 8 total, 8 up, 6 in 2024-02-23T00:54:40.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:40.303122+0000 mon.c (mon.2) 449 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-23T00:54:40.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:40.303596+0000 mon.a (mon.0) 3575 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-23T00:54:40.807 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:54:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:54:40] "GET /metrics HTTP/1.1" 200 191529 "" "Prometheus/2.33.4" 2024-02-23T00:54:41.030 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":1116,"stamp":"2024-02-23T00:54:38.386504+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4005890,"num_objects":242,"num_object_clones":1,"num_object_copies":726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":242,"num_whiteouts":0,"num_read":2451,"num_read_kb":2160,"num_write":950,"num_write_kb":3987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":8426,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1134,"ondisk_log_size":1134,"up":552,"acting":552,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":527,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":562470912,"kb_used":1843680,"kb_used_data":23472,"kb_used_omap":0,"kb_used_meta":1820160,"kb_avail":560627232,"statfs":{"total":575970213888,"available":574082285568,"internally_reserved":0,"allocated":24035328,"data_stored":14039238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1863843840},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.039230"},"pg_stats":[{"pgid":"249.1f","version":"0'0","reported_seq":9,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.386385+0000","last_change":"2024-02-23T00:54:36.386385+0000","last_active":"2024-02-23T00:54:36.386385+0000","last_peered":"2024-02-23T00:54:36.386385+0000","last_clean":"2024-02-23T00:54:36.386385+0000","last_became_active":"2024-02-23T00:54:36.386049+0000","last_became_peered":"2024-02-23T00:54:36.386049+0000","last_unstale":"2024-02-23T00:54:36.386385+0000","last_undegraded":"2024-02-23T00:54:36.386385+0000","last_fullsized":"2024-02-23T00:54:36.386385+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"249.1e","version":"0'0","reported_seq":9,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.396193+0000","last_change":"2024-02-23T00:54:36.396193+0000","last_active":"2024-02-23T00:54:36.396193+0000","last_peered":"2024-02-23T00:54:36.396193+0000","last_clean":"2024-02-23T00:54:36.396193+0000","last_became_active":"2024-02-23T00:54:36.395865+0000","last_became_peered":"2024-02-23T00:54:36.395865+0000","last_unstale":"2024-02-23T00:54:36.396193+0000","last_undegraded":"2024-02-23T00:54:36.396193+0000","last_fullsized":"2024-02-23T00:54:36.396193+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"249.1d","version":"0'0","reported_seq":9,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.390358+0000","last_change":"2024-02-23T00:54:36.390358+0000","last_active":"2024-02-23T00:54:36.390358+0000","last_peered":"2024-02-23T00:54:36.390358+0000","last_clean":"2024-02-23T00:54:36.390358+0000","last_became_active":"2024-02-23T00:54:36.390032+0000","last_became_peered":"2024-02-23T00:54:36.390032+0000","last_unstale":"2024-02-23T00:54:36.390358+0000","last_undegraded":"2024-02-23T00:54:36.390358+0000","last_fullsized":"2024-02-23T00:54:36.390358+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"249.1c","version":"0'0","reported_seq":9,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.385506+0000","last_change":"2024-02-23T00:54:36.385506+0000","last_active":"2024-02-23T00:54:36.385506+0000","last_peered":"2024-02-23T00:54:36.385506+0000","last_clean":"2024-02-23T00:54:36.385506+0000","last_became_active":"2024-02-23T00:54:36.385132+0000","last_became_peered":"2024-02-23T00:54:36.385132+0000","last_unstale":"2024-02-23T00:54:36.385506+0000","last_undegraded":"2024-02-23T00:54:36.385506+0000","last_fullsized":"2024-02-23T00:54:36.385506+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"249.1b","version":"0'0","reported_seq":9,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.393642+0000","last_change":"2024-02-23T00:54:36.393642+0000","last_active":"2024-02-23T00:54:36.393642+0000","last_peered":"2024-02-23T00:54:36.393642+0000","last_clean":"2024-02-23T00:54:36.393642+0000","last_became_active":"2024-02-23T00:54:36.393213+0000","last_became_peered":"2024-02-23T00:54:36.393213+0000","last_unstale":"2024-02-23T00:54:36.393642+0000","last_undegraded":"2024-02-23T00:54:36.393642+0000","last_fullsized":"2024-02-23T00:54:36.393642+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"249.1a","version":"0'0","reported_seq":9,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.398155+0000","last_change":"2024-02-23T00:54:36.398155+0000","last_active":"2024-02-23T00:54:36.398155+0000","last_peered":"2024-02-23T00:54:36.398155+0000","last_clean":"2024-02-23T00:54:36.398155+0000","last_became_active":"2024-02-23T00:54:36.397542+0000","last_became_peered":"2024-02-23T00:54:36.397542+0000","last_unstale":"2024-02-23T00:54:36.398155+0000","last_undegraded":"2024-02-23T00:54:36.398155+0000","last_fullsized":"2024-02-23T00:54:36.398155+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"249.19","version":"0'0","reported_seq":9,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.394690+0000","last_change":"2024-02-23T00:54:36.394690+0000","last_active":"2024-02-23T00:54:36.394690+0000","last_peered":"2024-02-23T00:54:36.394690+0000","last_clean":"2024-02-23T00:54:36.394690+0000","last_became_active":"2024-02-23T00:54:36.394382+0000","last_became_peered":"2024-02-23T00:54:36.394382+0000","last_unstale":"2024-02-23T00:54:36.394690+0000","last_undegraded":"2024-02-23T00:54:36.394690+0000","last_fullsized":"2024-02-23T00:54:36.394690+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"249.18","version":"0'0","reported_seq":9,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.394629+0000","last_change":"2024-02-23T00:54:36.394629+0000","last_active":"2024-02-23T00:54:36.394629+0000","last_peered":"2024-02-23T00:54:36.394629+0000","last_clean":"2024-02-23T00:54:36.394629+0000","last_became_active":"2024-02-23T00:54:36.394305+0000","last_became_peered":"2024-02-23T00:54:36.394305+0000","last_unstale":"2024-02-23T00:54:36.394629+0000","last_undegraded":"2024-02-23T00:54:36.394629+0000","last_fullsized":"2024-02-23T00:54:36.394629+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,5],"acting":[4,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"249.f","version":"0'0","reported_seq":9,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.400186+0000","last_change":"2024-02-23T00:54:36.400186+0000","last_active":"2024-02-23T00:54:36.400186+0000","last_peered":"2024-02-23T00:54:36.400186+0000","last_clean":"2024-02-23T00:54:36.400186+0000","last_became_active":"2024-02-23T00:54:36.399866+0000","last_became_peered":"2024-02-23T00:54:36.399866+0000","last_unstale":"2024-02-23T00:54:36.400186+0000","last_undegraded":"2024-02-23T00:54:36.400186+0000","last_fullsized":"2024-02-23T00:54:36.400186+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"249.e","version":"0'0","reported_seq":9,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.402347+0000","last_change":"2024-02-23T00:54:36.402347+0000","last_active":"2024-02-23T00:54:36.402347+0000","last_peered":"2024-02-23T00:54:36.402347+0000","last_clean":"2024-02-23T00:54:36.402347+0000","last_became_active":"2024-02-23T00:54:36.401994+0000","last_became_peered":"2024-02-23T00:54:36.401994+0000","last_unstale":"2024-02-23T00:54:36.402347+0000","last_undegraded":"2024-02-23T00:54:36.402347+0000","last_fullsized":"2024-02-23T00:54:36.402347+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"249.d","version":"0'0","reported_seq":9,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.395715+0000","last_change":"2024-02-23T00:54:36.395715+0000","last_active":"2024-02-23T00:54:36.395715+0000","last_peered":"2024-02-23T00:54:36.395715+0000","last_clean":"2024-02-23T00:54:36.395715+0000","last_became_active":"2024-02-23T00:54:36.395415+0000","last_became_peered":"2024-02-23T00:54:36.395415+0000","last_unstale":"2024-02-23T00:54:36.395715+0000","last_undegraded":"2024-02-23T00:54:36.395715+0000","last_fullsized":"2024-02-23T00:54:36.395715+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"249.c","version":"0'0","reported_seq":9,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.402659+0000","last_change":"2024-02-23T00:54:36.402659+0000","last_active":"2024-02-23T00:54:36.402659+0000","last_peered":"2024-02-23T00:54:36.402659+0000","last_clean":"2024-02-23T00:54:36.402659+0000","last_became_active":"2024-02-23T00:54:36.402162+0000","last_became_peered":"2024-02-23T00:54:36.402162+0000","last_unstale":"2024-02-23T00:54:36.402659+0000","last_undegraded":"2024-02-23T00:54:36.402659+0000","last_fullsized":"2024-02-23T00:54:36.402659+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"249.b","version":"0'0","reported_seq":9,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.392526+0000","last_change":"2024-02-23T00:54:36.392526+0000","last_active":"2024-02-23T00:54:36.392526+0000","last_peered":"2024-02-23T00:54:36.392526+0000","last_clean":"2024-02-23T00:54:36.392526+0000","last_became_active":"2024-02-23T00:54:36.392141+0000","last_became_peered":"2024-02-23T00:54:36.392141+0000","last_unstale":"2024-02-23T00:54:36.392526+0000","last_undegraded":"2024-02-23T00:54:36.392526+0000","last_fullsized":"2024-02-23T00:54:36.392526+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"249.a","version":"0'0","reported_seq":9,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.398004+0000","last_change":"2024-02-23T00:54:36.398004+0000","last_active":"2024-02-23T00:54:36.398004+0000","last_peered":"2024-02-23T00:54:36.398004+0000","last_clean":"2024-02-23T00:54:36.398004+0000","last_became_active":"2024-02-23T00:54:36.397176+0000","last_became_peered":"2024-02-23T00:54:36.397176+0000","last_unstale":"2024-02-23T00:54:36.398004+0000","last_undegraded":"2024-02-23T00:54:36.398004+0000","last_fullsized":"2024-02-23T00:54:36.398004+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"249.9","version":"0'0","reported_seq":9,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.401271+0000","last_change":"2024-02-23T00:54:36.401271+0000","last_active":"2024-02-23T00:54:36.401271+0000","last_peered":"2024-02-23T00:54:36.401271+0000","last_clean":"2024-02-23T00:54:36.401271+0000","last_became_active":"2024-02-23T00:54:36.400942+0000","last_became_peered":"2024-02-23T00:54:36.400942+0000","last_unstale":"2024-02-23T00:54:36.401271+0000","last_undegraded":"2024-02-23T00:54:36.401271+0000","last_fullsized":"2024-02-23T00:54:36.401271+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"249.8","version":"0'0","reported_seq":9,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.387804+0000","last_change":"2024-02-23T00:54:36.387804+0000","last_active":"2024-02-23T00:54:36.387804+0000","last_peered":"2024-02-23T00:54:36.387804+0000","last_clean":"2024-02-23T00:54:36.387804+0000","last_became_active":"2024-02-23T00:54:36.387475+0000","last_became_peered":"2024-02-23T00:54:36.387475+0000","last_unstale":"2024-02-23T00:54:36.387804+0000","last_undegraded":"2024-02-23T00:54:36.387804+0000","last_fullsized":"2024-02-23T00:54:36.387804+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"249.7","version":"0'0","reported_seq":9,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.394608+0000","last_change":"2024-02-23T00:54:36.394608+0000","last_active":"2024-02-23T00:54:36.394608+0000","last_peered":"2024-02-23T00:54:36.394608+0000","last_clean":"2024-02-23T00:54:36.394608+0000","last_became_active":"2024-02-23T00:54:36.393720+0000","last_became_peered":"2024-02-23T00:54:36.393720+0000","last_unstale":"2024-02-23T00:54:36.394608+0000","last_undegraded":"2024-02-23T00:54:36.394608+0000","last_fullsized":"2024-02-23T00:54:36.394608+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"249.6","version":"0'0","reported_seq":9,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.384583+0000","last_change":"2024-02-23T00:54:36.384583+0000","last_active":"2024-02-23T00:54:36.384583+0000","last_peered":"2024-02-23T00:54:36.384583+0000","last_clean":"2024-02-23T00:54:36.384583+0000","last_became_active":"2024-02-23T00:54:36.384289+0000","last_became_peered":"2024-02-23T00:54:36.384289+0000","last_unstale":"2024-02-23T00:54:36.384583+0000","last_undegraded":"2024-02-23T00:54:36.384583+0000","last_fullsized":"2024-02-23T00:54:36.384583+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"249.5","version":"0'0","reported_seq":9,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.400343+0000","last_change":"2024-02-23T00:54:36.400343+0000","last_active":"2024-02-23T00:54:36.400343+0000","last_peered":"2024-02-23T00:54:36.400343+0000","last_clean":"2024-02-23T00:54:36.400343+0000","last_became_active":"2024-02-23T00:54:36.400029+0000","last_became_peered":"2024-02-23T00:54:36.400029+0000","last_unstale":"2024-02-23T00:54:36.400343+0000","last_undegraded":"2024-02-23T00:54:36.400343+0000","last_fullsized":"2024-02-23T00:54:36.400343+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"249.4","version":"0'0","reported_seq":9,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.397013+0000","last_change":"2024-02-23T00:54:36.397013+0000","last_active":"2024-02-23T00:54:36.397013+0000","last_peered":"2024-02-23T00:54:36.397013+0000","last_clean":"2024-02-23T00:54:36.397013+0000","last_became_active":"2024-02-23T00:54:36.396654+0000","last_became_peered":"2024-02-23T00:54:36.396654+0000","last_unstale":"2024-02-23T00:54:36.397013+0000","last_undegraded":"2024-02-23T00:54:36.397013+0000","last_fullsized":"2024-02-23T00:54:36.397013+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"249.3","version":"0'0","reported_seq":9,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.399205+0000","last_change":"2024-02-23T00:54:36.399205+0000","last_active":"2024-02-23T00:54:36.399205+0000","last_peered":"2024-02-23T00:54:36.399205+0000","last_clean":"2024-02-23T00:54:36.399205+0000","last_became_active":"2024-02-23T00:54:36.398901+0000","last_became_peered":"2024-02-23T00:54:36.398901+0000","last_unstale":"2024-02-23T00:54:36.399205+0000","last_undegraded":"2024-02-23T00:54:36.399205+0000","last_fullsized":"2024-02-23T00:54:36.399205+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"249.2","version":"0'0","reported_seq":9,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.397799+0000","last_change":"2024-02-23T00:54:36.397799+0000","last_active":"2024-02-23T00:54:36.397799+0000","last_peered":"2024-02-23T00:54:36.397799+0000","last_clean":"2024-02-23T00:54:36.397799+0000","last_became_active":"2024-02-23T00:54:36.396828+0000","last_became_peered":"2024-02-23T00:54:36.396828+0000","last_unstale":"2024-02-23T00:54:36.397799+0000","last_undegraded":"2024-02-23T00:54:36.397799+0000","last_fullsized":"2024-02-23T00:54:36.397799+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"249.1","version":"0'0","reported_seq":9,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.395999+0000","last_change":"2024-02-23T00:54:36.395999+0000","last_active":"2024-02-23T00:54:36.395999+0000","last_peered":"2024-02-23T00:54:36.395999+0000","last_clean":"2024-02-23T00:54:36.395999+0000","last_became_active":"2024-02-23T00:54:36.395660+0000","last_became_peered":"2024-02-23T00:54:36.395660+0000","last_unstale":"2024-02-23T00:54:36.395999+0000","last_undegraded":"2024-02-23T00:54:36.395999+0000","last_fullsized":"2024-02-23T00:54:36.395999+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"249.0","version":"0'0","reported_seq":9,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.402398+0000","last_change":"2024-02-23T00:54:36.402398+0000","last_active":"2024-02-23T00:54:36.402398+0000","last_peered":"2024-02-23T00:54:36.402398+0000","last_clean":"2024-02-23T00:54:36.402398+0000","last_became_active":"2024-02-23T00:54:36.401890+0000","last_became_peered":"2024-02-23T00:54:36.401890+0000","last_unstale":"2024-02-23T00:54:36.402398+0000","last_undegraded":"2024-02-23T00:54:36.402398+0000","last_fullsized":"2024-02-23T00:54:36.402398+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"249.14","version":"0'0","reported_seq":9,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.402474+0000","last_change":"2024-02-23T00:54:36.402474+0000","last_active":"2024-02-23T00:54:36.402474+0000","last_peered":"2024-02-23T00:54:36.402474+0000","last_clean":"2024-02-23T00:54:36.402474+0000","last_became_active":"2024-02-23T00:54:36.401791+0000","last_became_peered":"2024-02-23T00:54:36.401791+0000","last_unstale":"2024-02-23T00:54:36.402474+0000","last_undegraded":"2024-02-23T00:54:36.402474+0000","last_fullsized":"2024-02-23T00:54:36.402474+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"234.7","version":"666'13","reported_seq":186,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.382150+0000","last_change":"2024-02-23T00:54:10.268034+0000","last_active":"2024-02-23T00:54:35.382150+0000","last_peered":"2024-02-23T00:54:35.382150+0000","last_clean":"2024-02-23T00:54:35.382150+0000","last_became_active":"2024-02-23T00:52:10.280357+0000","last_became_peered":"2024-02-23T00:52:10.280357+0000","last_unstale":"2024-02-23T00:54:35.382150+0000","last_undegraded":"2024-02-23T00:54:35.382150+0000","last_fullsized":"2024-02-23T00:54:35.382150+0000","mapping_epoch":600,"log_start":"0'0","ondisk_log_start":"0'0","created":600,"last_epoch_clean":601,"parent":"0.0","parent_split_bits":0,"last_scrub":"666'13","last_scrub_stamp":"2024-02-23T00:54:10.267942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_clean_scrub_stamp":"2024-02-23T00:54:10.267942+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":18,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":7,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"249.15","version":"0'0","reported_seq":9,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.394683+0000","last_change":"2024-02-23T00:54:36.394683+0000","last_active":"2024-02-23T00:54:36.394683+0000","last_peered":"2024-02-23T00:54:36.394683+0000","last_clean":"2024-02-23T00:54:36.394683+0000","last_became_active":"2024-02-23T00:54:36.394396+0000","last_became_peered":"2024-02-23T00:54:36.394396+0000","last_unstale":"2024-02-23T00:54:36.394683+0000","last_undegraded":"2024-02-23T00:54:36.394683+0000","last_fullsized":"2024-02-23T00:54:36.394683+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5],"acting":[6,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"234.6","version":"629'6","reported_seq":191,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.380206+0000","last_change":"2024-02-23T00:54:06.318420+0000","last_active":"2024-02-23T00:54:35.380206+0000","last_peered":"2024-02-23T00:54:35.380206+0000","last_clean":"2024-02-23T00:54:35.380206+0000","last_became_active":"2024-02-23T00:53:13.348600+0000","last_became_peered":"2024-02-23T00:53:13.348600+0000","last_unstale":"2024-02-23T00:54:35.380206+0000","last_undegraded":"2024-02-23T00:54:35.380206+0000","last_fullsized":"2024-02-23T00:54:35.380206+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":600,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"629'6","last_scrub_stamp":"2024-02-23T00:53:36.494408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_clean_scrub_stamp":"2024-02-23T00:53:36.494408+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":8,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"249.16","version":"0'0","reported_seq":9,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.394966+0000","last_change":"2024-02-23T00:54:36.394966+0000","last_active":"2024-02-23T00:54:36.394966+0000","last_peered":"2024-02-23T00:54:36.394966+0000","last_clean":"2024-02-23T00:54:36.394966+0000","last_became_active":"2024-02-23T00:54:36.394664+0000","last_became_peered":"2024-02-23T00:54:36.394664+0000","last_unstale":"2024-02-23T00:54:36.394966+0000","last_undegraded":"2024-02-23T00:54:36.394966+0000","last_fullsized":"2024-02-23T00:54:36.394966+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"234.5","version":"715'11","reported_seq":189,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.382022+0000","last_change":"2024-02-23T00:54:06.319329+0000","last_active":"2024-02-23T00:54:35.382022+0000","last_peered":"2024-02-23T00:54:35.382022+0000","last_clean":"2024-02-23T00:54:35.382022+0000","last_became_active":"2024-02-23T00:53:13.350717+0000","last_became_peered":"2024-02-23T00:53:13.350717+0000","last_unstale":"2024-02-23T00:54:35.382022+0000","last_undegraded":"2024-02-23T00:54:35.382022+0000","last_fullsized":"2024-02-23T00:54:35.382022+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":600,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"666'6","last_scrub_stamp":"2024-02-23T00:53:32.451126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_clean_scrub_stamp":"2024-02-23T00:53:32.451126+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"249.17","version":"0'0","reported_seq":9,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.391603+0000","last_change":"2024-02-23T00:54:36.391603+0000","last_active":"2024-02-23T00:54:36.391603+0000","last_peered":"2024-02-23T00:54:36.391603+0000","last_clean":"2024-02-23T00:54:36.391603+0000","last_became_active":"2024-02-23T00:54:36.391155+0000","last_became_peered":"2024-02-23T00:54:36.391155+0000","last_unstale":"2024-02-23T00:54:36.391603+0000","last_undegraded":"2024-02-23T00:54:36.391603+0000","last_fullsized":"2024-02-23T00:54:36.391603+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"234.4","version":"707'13","reported_seq":127,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.374081+0000","last_change":"2024-02-23T00:54:08.325933+0000","last_active":"2024-02-23T00:54:35.374081+0000","last_peered":"2024-02-23T00:54:35.374081+0000","last_clean":"2024-02-23T00:54:35.374081+0000","last_became_active":"2024-02-23T00:53:34.320586+0000","last_became_peered":"2024-02-23T00:53:34.320586+0000","last_unstale":"2024-02-23T00:54:35.374081+0000","last_undegraded":"2024-02-23T00:54:35.374081+0000","last_fullsized":"2024-02-23T00:54:35.374081+0000","mapping_epoch":676,"log_start":"0'0","ondisk_log_start":"0'0","created":600,"last_epoch_clean":677,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_clean_scrub_stamp":"2024-02-23T00:52:09.256902+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"249.10","version":"0'0","reported_seq":9,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.401993+0000","last_change":"2024-02-23T00:54:36.401993+0000","last_active":"2024-02-23T00:54:36.401993+0000","last_peered":"2024-02-23T00:54:36.401993+0000","last_clean":"2024-02-23T00:54:36.401993+0000","last_became_active":"2024-02-23T00:54:36.401399+0000","last_became_peered":"2024-02-23T00:54:36.401399+0000","last_unstale":"2024-02-23T00:54:36.401993+0000","last_undegraded":"2024-02-23T00:54:36.401993+0000","last_fullsized":"2024-02-23T00:54:36.401993+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"234.3","version":"667'5","reported_seq":205,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.377225+0000","last_change":"2024-02-23T00:54:07.348103+0000","last_active":"2024-02-23T00:54:35.377225+0000","last_peered":"2024-02-23T00:54:35.377225+0000","last_clean":"2024-02-23T00:54:35.377225+0000","last_became_active":"2024-02-23T00:53:53.411778+0000","last_became_peered":"2024-02-23T00:53:53.411778+0000","last_unstale":"2024-02-23T00:54:35.377225+0000","last_undegraded":"2024-02-23T00:54:35.377225+0000","last_fullsized":"2024-02-23T00:54:35.377225+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":600,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"667'5","last_scrub_stamp":"2024-02-23T00:54:07.348022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_clean_scrub_stamp":"2024-02-23T00:54:07.348022+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":18,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"249.11","version":"0'0","reported_seq":9,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.396058+0000","last_change":"2024-02-23T00:54:36.396058+0000","last_active":"2024-02-23T00:54:36.396058+0000","last_peered":"2024-02-23T00:54:36.396058+0000","last_clean":"2024-02-23T00:54:36.396058+0000","last_became_active":"2024-02-23T00:54:36.395126+0000","last_became_peered":"2024-02-23T00:54:36.395126+0000","last_unstale":"2024-02-23T00:54:36.396058+0000","last_undegraded":"2024-02-23T00:54:36.396058+0000","last_fullsized":"2024-02-23T00:54:36.396058+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"234.2","version":"629'10","reported_seq":197,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.387508+0000","last_change":"2024-02-23T00:54:06.318884+0000","last_active":"2024-02-23T00:54:35.387508+0000","last_peered":"2024-02-23T00:54:35.387508+0000","last_clean":"2024-02-23T00:54:35.387508+0000","last_became_active":"2024-02-23T00:53:13.351731+0000","last_became_peered":"2024-02-23T00:53:13.351731+0000","last_unstale":"2024-02-23T00:54:35.387508+0000","last_undegraded":"2024-02-23T00:54:35.387508+0000","last_fullsized":"2024-02-23T00:54:35.387508+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":600,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"629'10","last_scrub_stamp":"2024-02-23T00:53:34.474122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_clean_scrub_stamp":"2024-02-23T00:53:34.474122+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":10,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":10,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,7],"acting":[6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"249.12","version":"0'0","reported_seq":9,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.402194+0000","last_change":"2024-02-23T00:54:36.402194+0000","last_active":"2024-02-23T00:54:36.402194+0000","last_peered":"2024-02-23T00:54:36.402194+0000","last_clean":"2024-02-23T00:54:36.402194+0000","last_became_active":"2024-02-23T00:54:36.401838+0000","last_became_peered":"2024-02-23T00:54:36.401838+0000","last_unstale":"2024-02-23T00:54:36.402194+0000","last_undegraded":"2024-02-23T00:54:36.402194+0000","last_fullsized":"2024-02-23T00:54:36.402194+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"234.1","version":"723'5","reported_seq":192,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.378399+0000","last_change":"2024-02-23T00:54:06.318698+0000","last_active":"2024-02-23T00:54:35.378399+0000","last_peered":"2024-02-23T00:54:35.378399+0000","last_clean":"2024-02-23T00:54:35.378399+0000","last_became_active":"2024-02-23T00:53:34.318140+0000","last_became_peered":"2024-02-23T00:53:34.318140+0000","last_unstale":"2024-02-23T00:54:35.378399+0000","last_undegraded":"2024-02-23T00:54:35.378399+0000","last_fullsized":"2024-02-23T00:54:35.378399+0000","mapping_epoch":676,"log_start":"0'0","ondisk_log_start":"0'0","created":600,"last_epoch_clean":677,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_clean_scrub_stamp":"2024-02-23T00:52:09.256902+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4000024,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4,"num_write_kb":3910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":8,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"249.13","version":"0'0","reported_seq":9,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.395966+0000","last_change":"2024-02-23T00:54:36.395966+0000","last_active":"2024-02-23T00:54:36.395966+0000","last_peered":"2024-02-23T00:54:36.395966+0000","last_clean":"2024-02-23T00:54:36.395966+0000","last_became_active":"2024-02-23T00:54:36.394040+0000","last_became_peered":"2024-02-23T00:54:36.394040+0000","last_unstale":"2024-02-23T00:54:36.395966+0000","last_undegraded":"2024-02-23T00:54:36.395966+0000","last_fullsized":"2024-02-23T00:54:36.395966+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,5],"acting":[4,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"234.0","version":"694'7","reported_seq":178,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.371990+0000","last_change":"2024-02-23T00:54:15.372319+0000","last_active":"2024-02-23T00:54:35.371990+0000","last_peered":"2024-02-23T00:54:35.371990+0000","last_clean":"2024-02-23T00:54:35.371990+0000","last_became_active":"2024-02-23T00:52:10.279573+0000","last_became_peered":"2024-02-23T00:52:10.279573+0000","last_unstale":"2024-02-23T00:54:35.371990+0000","last_undegraded":"2024-02-23T00:54:35.371990+0000","last_fullsized":"2024-02-23T00:54:35.371990+0000","mapping_epoch":600,"log_start":"0'0","ondisk_log_start":"0'0","created":600,"last_epoch_clean":601,"parent":"0.0","parent_split_bits":0,"last_scrub":"694'7","last_scrub_stamp":"2024-02-23T00:54:15.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_clean_scrub_stamp":"2024-02-23T00:54:15.372237+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"466'9","reported_seq":903,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.379383+0000","last_change":"2024-02-23T00:54:11.260611+0000","last_active":"2024-02-23T00:54:35.379383+0000","last_peered":"2024-02-23T00:54:35.379383+0000","last_clean":"2024-02-23T00:54:35.379383+0000","last_became_active":"2024-02-23T00:39:18.793414+0000","last_became_peered":"2024-02-23T00:39:18.793414+0000","last_unstale":"2024-02-23T00:54:35.379383+0000","last_undegraded":"2024-02-23T00:54:35.379383+0000","last_fullsized":"2024-02-23T00:54:35.379383+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'9","last_scrub_stamp":"2024-02-23T00:54:11.260489+0000","last_deep_scrub":"466'9","last_deep_scrub_stamp":"2024-02-23T00:49:43.187989+0000","last_clean_scrub_stamp":"2024-02-23T00:54:11.260489+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"470'42","reported_seq":782,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.376430+0000","last_change":"2024-02-23T00:52:17.393167+0000","last_active":"2024-02-23T00:54:35.376430+0000","last_peered":"2024-02-23T00:54:35.376430+0000","last_clean":"2024-02-23T00:54:35.376430+0000","last_became_active":"2024-02-23T00:39:16.859401+0000","last_became_peered":"2024-02-23T00:39:16.859401+0000","last_unstale":"2024-02-23T00:54:35.376430+0000","last_undegraded":"2024-02-23T00:54:35.376430+0000","last_fullsized":"2024-02-23T00:54:35.376430+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"470'42","last_scrub_stamp":"2024-02-23T00:52:17.393084+0000","last_deep_scrub":"470'42","last_deep_scrub_stamp":"2024-02-23T00:49:47.653176+0000","last_clean_scrub_stamp":"2024-02-23T00:52:17.393084+0000","log_size":42,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":97,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.380255+0000","last_change":"2024-02-23T00:53:53.424388+0000","last_active":"2024-02-23T00:54:35.380255+0000","last_peered":"2024-02-23T00:54:35.380255+0000","last_clean":"2024-02-23T00:54:35.380255+0000","last_became_active":"2024-02-23T00:53:53.424032+0000","last_became_peered":"2024-02-23T00:53:53.424032+0000","last_unstale":"2024-02-23T00:54:35.380255+0000","last_undegraded":"2024-02-23T00:54:35.380255+0000","last_fullsized":"2024-02-23T00:54:35.380255+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:40.350368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:04.330426+0000","last_clean_scrub_stamp":"2024-02-23T00:53:40.350368+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":729,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.376789+0000","last_change":"2024-02-23T00:53:13.349648+0000","last_active":"2024-02-23T00:54:35.376789+0000","last_peered":"2024-02-23T00:54:35.376789+0000","last_clean":"2024-02-23T00:54:35.376789+0000","last_became_active":"2024-02-23T00:53:13.349304+0000","last_became_peered":"2024-02-23T00:53:13.349304+0000","last_unstale":"2024-02-23T00:54:35.376789+0000","last_undegraded":"2024-02-23T00:54:35.376789+0000","last_fullsized":"2024-02-23T00:54:35.376789+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:25.491458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:52:25.491458+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":658,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.378003+0000","last_change":"2024-02-23T00:53:13.350523+0000","last_active":"2024-02-23T00:54:35.378003+0000","last_peered":"2024-02-23T00:54:35.378003+0000","last_clean":"2024-02-23T00:54:35.378003+0000","last_became_active":"2024-02-23T00:53:13.350150+0000","last_became_peered":"2024-02-23T00:53:13.350150+0000","last_unstale":"2024-02-23T00:54:35.378003+0000","last_undegraded":"2024-02-23T00:54:35.378003+0000","last_fullsized":"2024-02-23T00:54:35.378003+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:51.439818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:52:51.439818+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"457'21","reported_seq":605,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.374571+0000","last_change":"2024-02-23T00:53:53.849527+0000","last_active":"2024-02-23T00:54:35.374571+0000","last_peered":"2024-02-23T00:54:35.374571+0000","last_clean":"2024-02-23T00:54:35.374571+0000","last_became_active":"2024-02-23T00:53:53.410969+0000","last_became_peered":"2024-02-23T00:53:53.410969+0000","last_unstale":"2024-02-23T00:54:35.374571+0000","last_undegraded":"2024-02-23T00:54:35.374571+0000","last_fullsized":"2024-02-23T00:54:35.374571+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-23T00:51:46.661879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:46.661879+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":37,"num_read_kb":30,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":697,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.375323+0000","last_change":"2024-02-23T00:52:25.554460+0000","last_active":"2024-02-23T00:54:35.375323+0000","last_peered":"2024-02-23T00:54:35.375323+0000","last_clean":"2024-02-23T00:54:35.375323+0000","last_became_active":"2024-02-23T00:39:14.772493+0000","last_became_peered":"2024-02-23T00:39:14.772493+0000","last_unstale":"2024-02-23T00:54:35.375323+0000","last_undegraded":"2024-02-23T00:54:35.375323+0000","last_fullsized":"2024-02-23T00:54:35.375323+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:25.554392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:04.528006+0000","last_clean_scrub_stamp":"2024-02-23T00:52:25.554392+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":409,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.372595+0000","last_change":"2024-02-23T00:53:53.411602+0000","last_active":"2024-02-23T00:54:35.372595+0000","last_peered":"2024-02-23T00:54:35.372595+0000","last_clean":"2024-02-23T00:54:35.372595+0000","last_became_active":"2024-02-23T00:53:53.411237+0000","last_became_peered":"2024-02-23T00:53:53.411237+0000","last_unstale":"2024-02-23T00:54:35.372595+0000","last_undegraded":"2024-02-23T00:54:35.372595+0000","last_fullsized":"2024-02-23T00:54:35.372595+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:30.466466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:08.335330+0000","last_clean_scrub_stamp":"2024-02-23T00:53:30.466466+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":442,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.378989+0000","last_change":"2024-02-23T00:53:13.348694+0000","last_active":"2024-02-23T00:54:35.378989+0000","last_peered":"2024-02-23T00:54:35.378989+0000","last_clean":"2024-02-23T00:54:35.378989+0000","last_became_active":"2024-02-23T00:53:13.348208+0000","last_became_peered":"2024-02-23T00:53:13.348208+0000","last_unstale":"2024-02-23T00:54:35.378989+0000","last_undegraded":"2024-02-23T00:54:35.378989+0000","last_fullsized":"2024-02-23T00:54:35.378989+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:02.821535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:13.867716+0000","last_clean_scrub_stamp":"2024-02-23T00:53:02.821535+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":93,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.379387+0000","last_change":"2024-02-23T00:53:53.424522+0000","last_active":"2024-02-23T00:54:35.379387+0000","last_peered":"2024-02-23T00:54:35.379387+0000","last_clean":"2024-02-23T00:54:35.379387+0000","last_became_active":"2024-02-23T00:53:53.424204+0000","last_became_peered":"2024-02-23T00:53:53.424204+0000","last_unstale":"2024-02-23T00:54:35.379387+0000","last_undegraded":"2024-02-23T00:54:35.379387+0000","last_fullsized":"2024-02-23T00:54:35.379387+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:56.845554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:52:56.845554+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"457'51","reported_seq":789,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.388034+0000","last_change":"2024-02-23T00:52:50.534889+0000","last_active":"2024-02-23T00:54:35.388034+0000","last_peered":"2024-02-23T00:54:35.388034+0000","last_clean":"2024-02-23T00:54:35.388034+0000","last_became_active":"2024-02-23T00:39:16.857413+0000","last_became_peered":"2024-02-23T00:39:16.857413+0000","last_unstale":"2024-02-23T00:54:35.388034+0000","last_undegraded":"2024-02-23T00:54:35.388034+0000","last_fullsized":"2024-02-23T00:54:35.388034+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'51","last_scrub_stamp":"2024-02-23T00:52:50.534772+0000","last_deep_scrub":"88'35","last_deep_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_clean_scrub_stamp":"2024-02-23T00:52:50.534772+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":68,"num_read_kb":56,"num_write":42,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":390,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.380194+0000","last_change":"2024-02-23T00:53:28.314751+0000","last_active":"2024-02-23T00:54:35.380194+0000","last_peered":"2024-02-23T00:54:35.380194+0000","last_clean":"2024-02-23T00:54:35.380194+0000","last_became_active":"2024-02-23T00:53:13.345499+0000","last_became_peered":"2024-02-23T00:53:13.345499+0000","last_unstale":"2024-02-23T00:54:35.380194+0000","last_undegraded":"2024-02-23T00:54:35.380194+0000","last_fullsized":"2024-02-23T00:54:35.380194+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:28.314660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_clean_scrub_stamp":"2024-02-23T00:53:28.314660+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":99,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.377309+0000","last_change":"2024-02-23T00:53:41.132759+0000","last_active":"2024-02-23T00:54:35.377309+0000","last_peered":"2024-02-23T00:54:35.377309+0000","last_clean":"2024-02-23T00:54:35.377309+0000","last_became_active":"2024-02-23T00:53:13.357189+0000","last_became_peered":"2024-02-23T00:53:13.357189+0000","last_unstale":"2024-02-23T00:54:35.377309+0000","last_undegraded":"2024-02-23T00:54:35.377309+0000","last_fullsized":"2024-02-23T00:54:35.377309+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:41.132688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:03.682745+0000","last_clean_scrub_stamp":"2024-02-23T00:53:41.132688+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":727,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.374939+0000","last_change":"2024-02-23T00:52:07.425432+0000","last_active":"2024-02-23T00:54:35.374939+0000","last_peered":"2024-02-23T00:54:35.374939+0000","last_clean":"2024-02-23T00:54:35.374939+0000","last_became_active":"2024-02-23T00:47:28.215636+0000","last_became_peered":"2024-02-23T00:47:28.215636+0000","last_unstale":"2024-02-23T00:54:35.374939+0000","last_undegraded":"2024-02-23T00:54:35.374939+0000","last_fullsized":"2024-02-23T00:54:35.374939+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:07.425311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:42.150875+0000","last_clean_scrub_stamp":"2024-02-23T00:52:07.425311+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":666,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.374357+0000","last_change":"2024-02-23T00:53:53.412741+0000","last_active":"2024-02-23T00:54:35.374357+0000","last_peered":"2024-02-23T00:54:35.374357+0000","last_clean":"2024-02-23T00:54:35.374357+0000","last_became_active":"2024-02-23T00:53:53.412355+0000","last_became_peered":"2024-02-23T00:53:53.412355+0000","last_unstale":"2024-02-23T00:54:35.374357+0000","last_undegraded":"2024-02-23T00:54:35.374357+0000","last_fullsized":"2024-02-23T00:54:35.374357+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:07.482987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:53:07.482987+0000","last_clean_scrub_stamp":"2024-02-23T00:53:07.482987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"457'31","reported_seq":810,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.373337+0000","last_change":"2024-02-23T00:53:36.520058+0000","last_active":"2024-02-23T00:54:35.373337+0000","last_peered":"2024-02-23T00:54:35.373337+0000","last_clean":"2024-02-23T00:54:35.373337+0000","last_became_active":"2024-02-23T00:47:28.200653+0000","last_became_peered":"2024-02-23T00:47:28.200653+0000","last_unstale":"2024-02-23T00:54:35.373337+0000","last_undegraded":"2024-02-23T00:54:35.373337+0000","last_fullsized":"2024-02-23T00:54:35.373337+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:53:36.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:53:36.519986+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":379,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.381162+0000","last_change":"2024-02-23T00:52:55.382296+0000","last_active":"2024-02-23T00:54:35.381162+0000","last_peered":"2024-02-23T00:54:35.381162+0000","last_clean":"2024-02-23T00:54:35.381162+0000","last_became_active":"2024-02-23T00:47:28.201926+0000","last_became_peered":"2024-02-23T00:47:28.201926+0000","last_unstale":"2024-02-23T00:54:35.381162+0000","last_undegraded":"2024-02-23T00:54:35.381162+0000","last_fullsized":"2024-02-23T00:54:35.381162+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:55.382210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:55.382210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":707,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.375234+0000","last_change":"2024-02-23T00:54:07.276618+0000","last_active":"2024-02-23T00:54:35.375234+0000","last_peered":"2024-02-23T00:54:35.375234+0000","last_clean":"2024-02-23T00:54:35.375234+0000","last_became_active":"2024-02-23T00:53:13.350335+0000","last_became_peered":"2024-02-23T00:53:13.350335+0000","last_unstale":"2024-02-23T00:54:35.375234+0000","last_undegraded":"2024-02-23T00:54:35.375234+0000","last_fullsized":"2024-02-23T00:54:35.375234+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:07.276549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:09.343884+0000","last_clean_scrub_stamp":"2024-02-23T00:54:07.276549+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":639,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.374340+0000","last_change":"2024-02-23T00:52:16.478287+0000","last_active":"2024-02-23T00:54:35.374340+0000","last_peered":"2024-02-23T00:54:35.374340+0000","last_clean":"2024-02-23T00:54:35.374340+0000","last_became_active":"2024-02-23T00:42:57.675306+0000","last_became_peered":"2024-02-23T00:42:57.675306+0000","last_unstale":"2024-02-23T00:54:35.374340+0000","last_undegraded":"2024-02-23T00:54:35.374340+0000","last_fullsized":"2024-02-23T00:54:35.374340+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:16.478219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:52:16.478219+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"457'20","reported_seq":737,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.375894+0000","last_change":"2024-02-23T00:52:53.807404+0000","last_active":"2024-02-23T00:54:35.375894+0000","last_peered":"2024-02-23T00:54:35.375894+0000","last_clean":"2024-02-23T00:54:35.375894+0000","last_became_active":"2024-02-23T00:39:16.777530+0000","last_became_peered":"2024-02-23T00:39:16.777530+0000","last_unstale":"2024-02-23T00:54:35.375894+0000","last_undegraded":"2024-02-23T00:54:35.375894+0000","last_fullsized":"2024-02-23T00:54:35.375894+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'20","last_scrub_stamp":"2024-02-23T00:52:53.807334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:52:53.807334+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":697,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.374332+0000","last_change":"2024-02-23T00:52:13.530732+0000","last_active":"2024-02-23T00:54:35.374332+0000","last_peered":"2024-02-23T00:54:35.374332+0000","last_clean":"2024-02-23T00:54:35.374332+0000","last_became_active":"2024-02-23T00:39:14.780379+0000","last_became_peered":"2024-02-23T00:39:14.780379+0000","last_unstale":"2024-02-23T00:54:35.374332+0000","last_undegraded":"2024-02-23T00:54:35.374332+0000","last_fullsized":"2024-02-23T00:54:35.374332+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:13.530630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.575242+0000","last_clean_scrub_stamp":"2024-02-23T00:52:13.530630+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"656'10","reported_seq":498,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.379891+0000","last_change":"2024-02-23T00:53:13.448730+0000","last_active":"2024-02-23T00:54:35.379891+0000","last_peered":"2024-02-23T00:54:35.379891+0000","last_clean":"2024-02-23T00:54:35.379891+0000","last_became_active":"2024-02-23T00:53:13.359465+0000","last_became_peered":"2024-02-23T00:53:13.359465+0000","last_unstale":"2024-02-23T00:54:35.379891+0000","last_undegraded":"2024-02-23T00:54:35.379891+0000","last_fullsized":"2024-02-23T00:54:35.379891+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"376'9","last_scrub_stamp":"2024-02-23T00:52:13.442894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:52:13.442894+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":89,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.374417+0000","last_change":"2024-02-23T00:53:53.411805+0000","last_active":"2024-02-23T00:54:35.374417+0000","last_peered":"2024-02-23T00:54:35.374417+0000","last_clean":"2024-02-23T00:54:35.374417+0000","last_became_active":"2024-02-23T00:53:53.411396+0000","last_became_peered":"2024-02-23T00:53:53.411396+0000","last_unstale":"2024-02-23T00:54:35.374417+0000","last_undegraded":"2024-02-23T00:54:35.374417+0000","last_fullsized":"2024-02-23T00:54:35.374417+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.335752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.335752+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.335752+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"457'27","reported_seq":738,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.375561+0000","last_change":"2024-02-23T00:51:48.537804+0000","last_active":"2024-02-23T00:54:35.375561+0000","last_peered":"2024-02-23T00:54:35.375561+0000","last_clean":"2024-02-23T00:54:35.375561+0000","last_became_active":"2024-02-23T00:39:16.789757+0000","last_became_peered":"2024-02-23T00:39:16.789757+0000","last_unstale":"2024-02-23T00:54:35.375561+0000","last_undegraded":"2024-02-23T00:54:35.375561+0000","last_fullsized":"2024-02-23T00:54:35.375561+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-23T00:51:48.537732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:48.537732+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":697,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.374020+0000","last_change":"2024-02-23T00:51:13.375106+0000","last_active":"2024-02-23T00:54:35.374020+0000","last_peered":"2024-02-23T00:54:35.374020+0000","last_clean":"2024-02-23T00:54:35.374020+0000","last_became_active":"2024-02-23T00:39:14.780526+0000","last_became_peered":"2024-02-23T00:39:14.780526+0000","last_unstale":"2024-02-23T00:54:35.374020+0000","last_undegraded":"2024-02-23T00:54:35.374020+0000","last_fullsized":"2024-02-23T00:54:35.374020+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:13.375039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:13.375039+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"694'11","reported_seq":941,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.376533+0000","last_change":"2024-02-23T00:54:01.484460+0000","last_active":"2024-02-23T00:54:35.376533+0000","last_peered":"2024-02-23T00:54:35.376533+0000","last_clean":"2024-02-23T00:54:35.376533+0000","last_became_active":"2024-02-23T00:53:53.417721+0000","last_became_peered":"2024-02-23T00:53:53.417721+0000","last_unstale":"2024-02-23T00:54:35.376533+0000","last_undegraded":"2024-02-23T00:54:35.376533+0000","last_fullsized":"2024-02-23T00:54:35.376533+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"694'11","last_scrub_stamp":"2024-02-23T00:54:01.484388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:54:01.484388+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":671,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.372983+0000","last_change":"2024-02-23T00:53:53.414508+0000","last_active":"2024-02-23T00:54:35.372983+0000","last_peered":"2024-02-23T00:54:35.372983+0000","last_clean":"2024-02-23T00:54:35.372983+0000","last_became_active":"2024-02-23T00:53:53.413978+0000","last_became_peered":"2024-02-23T00:53:53.413978+0000","last_unstale":"2024-02-23T00:54:35.372983+0000","last_undegraded":"2024-02-23T00:54:35.372983+0000","last_fullsized":"2024-02-23T00:54:35.372983+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:59.798987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:52:59.798987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"457'31","reported_seq":805,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.376369+0000","last_change":"2024-02-23T00:53:34.440512+0000","last_active":"2024-02-23T00:54:35.376369+0000","last_peered":"2024-02-23T00:54:35.376369+0000","last_clean":"2024-02-23T00:54:35.376369+0000","last_became_active":"2024-02-23T00:47:28.201225+0000","last_became_peered":"2024-02-23T00:47:28.201225+0000","last_unstale":"2024-02-23T00:54:35.376369+0000","last_undegraded":"2024-02-23T00:54:35.376369+0000","last_fullsized":"2024-02-23T00:54:35.376369+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:53:34.440440+0000","last_deep_scrub":"336'23","last_deep_scrub_stamp":"2024-02-23T00:46:48.320085+0000","last_clean_scrub_stamp":"2024-02-23T00:53:34.440440+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":715,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.375145+0000","last_change":"2024-02-23T00:53:13.358586+0000","last_active":"2024-02-23T00:54:35.375145+0000","last_peered":"2024-02-23T00:54:35.375145+0000","last_clean":"2024-02-23T00:54:35.375145+0000","last_became_active":"2024-02-23T00:53:13.358230+0000","last_became_peered":"2024-02-23T00:53:13.358230+0000","last_unstale":"2024-02-23T00:54:35.375145+0000","last_undegraded":"2024-02-23T00:54:35.375145+0000","last_fullsized":"2024-02-23T00:54:35.375145+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:25.405680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:25.405680+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":704,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.379251+0000","last_change":"2024-02-23T00:52:54.389055+0000","last_active":"2024-02-23T00:54:35.379251+0000","last_peered":"2024-02-23T00:54:35.379251+0000","last_clean":"2024-02-23T00:54:35.379251+0000","last_became_active":"2024-02-23T00:39:18.800777+0000","last_became_peered":"2024-02-23T00:39:18.800777+0000","last_unstale":"2024-02-23T00:54:35.379251+0000","last_undegraded":"2024-02-23T00:54:35.379251+0000","last_fullsized":"2024-02-23T00:54:35.379251+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:54.388949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_clean_scrub_stamp":"2024-02-23T00:52:54.388949+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":639,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.374190+0000","last_change":"2024-02-23T00:53:30.603155+0000","last_active":"2024-02-23T00:54:35.374190+0000","last_peered":"2024-02-23T00:54:35.374190+0000","last_clean":"2024-02-23T00:54:35.374190+0000","last_became_active":"2024-02-23T00:42:57.653279+0000","last_became_peered":"2024-02-23T00:42:57.653279+0000","last_unstale":"2024-02-23T00:54:35.374190+0000","last_undegraded":"2024-02-23T00:54:35.374190+0000","last_fullsized":"2024-02-23T00:54:35.374190+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:30.603090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:53:30.603090+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"694'10","reported_seq":129,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.373900+0000","last_change":"2024-02-23T00:53:53.415736+0000","last_active":"2024-02-23T00:54:35.373900+0000","last_peered":"2024-02-23T00:54:35.373900+0000","last_clean":"2024-02-23T00:54:35.373900+0000","last_became_active":"2024-02-23T00:53:53.407501+0000","last_became_peered":"2024-02-23T00:53:53.407501+0000","last_unstale":"2024-02-23T00:54:35.373900+0000","last_undegraded":"2024-02-23T00:54:35.373900+0000","last_fullsized":"2024-02-23T00:54:35.373900+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:51:34.577222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:51:34.577222+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"457'42","reported_seq":772,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.373120+0000","last_change":"2024-02-23T00:52:26.764098+0000","last_active":"2024-02-23T00:54:35.373120+0000","last_peered":"2024-02-23T00:54:35.373120+0000","last_clean":"2024-02-23T00:54:35.373120+0000","last_became_active":"2024-02-23T00:39:16.780130+0000","last_became_peered":"2024-02-23T00:39:16.780130+0000","last_unstale":"2024-02-23T00:54:35.373120+0000","last_undegraded":"2024-02-23T00:54:35.373120+0000","last_fullsized":"2024-02-23T00:54:35.373120+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'42","last_scrub_stamp":"2024-02-23T00:52:26.764031+0000","last_deep_scrub":"423'28","last_deep_scrub_stamp":"2024-02-23T00:48:17.435906+0000","last_clean_scrub_stamp":"2024-02-23T00:52:26.764031+0000","log_size":42,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":46,"num_write":34,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":727,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.378658+0000","last_change":"2024-02-23T00:54:13.439244+0000","last_active":"2024-02-23T00:54:35.378658+0000","last_peered":"2024-02-23T00:54:35.378658+0000","last_clean":"2024-02-23T00:54:35.378658+0000","last_became_active":"2024-02-23T00:47:28.237083+0000","last_became_peered":"2024-02-23T00:47:28.237083+0000","last_unstale":"2024-02-23T00:54:35.378658+0000","last_undegraded":"2024-02-23T00:54:35.378658+0000","last_fullsized":"2024-02-23T00:54:35.378658+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:13.439154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:51.391939+0000","last_clean_scrub_stamp":"2024-02-23T00:54:13.439154+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":719,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.381753+0000","last_change":"2024-02-23T00:52:20.403117+0000","last_active":"2024-02-23T00:54:35.381753+0000","last_peered":"2024-02-23T00:54:35.381753+0000","last_clean":"2024-02-23T00:54:35.381753+0000","last_became_active":"2024-02-23T00:47:30.187386+0000","last_became_peered":"2024-02-23T00:47:30.187386+0000","last_unstale":"2024-02-23T00:54:35.381753+0000","last_undegraded":"2024-02-23T00:54:35.381753+0000","last_fullsized":"2024-02-23T00:54:35.381753+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:20.403031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:46.686764+0000","last_clean_scrub_stamp":"2024-02-23T00:52:20.403031+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":658,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.379494+0000","last_change":"2024-02-23T00:53:32.258905+0000","last_active":"2024-02-23T00:54:35.379494+0000","last_peered":"2024-02-23T00:54:35.379494+0000","last_clean":"2024-02-23T00:54:35.379494+0000","last_became_active":"2024-02-23T00:47:28.212307+0000","last_became_peered":"2024-02-23T00:47:28.212307+0000","last_unstale":"2024-02-23T00:54:35.379494+0000","last_undegraded":"2024-02-23T00:54:35.379494+0000","last_fullsized":"2024-02-23T00:54:35.379494+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:32.258822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:53:32.258822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":725,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.373579+0000","last_change":"2024-02-23T00:53:53.414314+0000","last_active":"2024-02-23T00:54:35.373579+0000","last_peered":"2024-02-23T00:54:35.373579+0000","last_clean":"2024-02-23T00:54:35.373579+0000","last_became_active":"2024-02-23T00:53:53.413534+0000","last_became_peered":"2024-02-23T00:53:53.413534+0000","last_unstale":"2024-02-23T00:54:35.373579+0000","last_undegraded":"2024-02-23T00:54:35.373579+0000","last_fullsized":"2024-02-23T00:54:35.373579+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:51.722289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:10.493340+0000","last_clean_scrub_stamp":"2024-02-23T00:52:51.722289+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"457'52","reported_seq":645,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.375434+0000","last_change":"2024-02-23T00:53:54.301015+0000","last_active":"2024-02-23T00:54:35.375434+0000","last_peered":"2024-02-23T00:54:35.375434+0000","last_clean":"2024-02-23T00:54:35.375434+0000","last_became_active":"2024-02-23T00:53:53.416131+0000","last_became_peered":"2024-02-23T00:53:53.416131+0000","last_unstale":"2024-02-23T00:54:35.375434+0000","last_undegraded":"2024-02-23T00:54:35.375434+0000","last_fullsized":"2024-02-23T00:54:35.375434+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'52","last_scrub_stamp":"2024-02-23T00:53:02.966145+0000","last_deep_scrub":"457'52","last_deep_scrub_stamp":"2024-02-23T00:53:02.966145+0000","last_clean_scrub_stamp":"2024-02-23T00:53:02.966145+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":75,"num_read_kb":61,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":697,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.375490+0000","last_change":"2024-02-23T00:52:18.475876+0000","last_active":"2024-02-23T00:54:35.375490+0000","last_peered":"2024-02-23T00:54:35.375490+0000","last_clean":"2024-02-23T00:54:35.375490+0000","last_became_active":"2024-02-23T00:39:14.779193+0000","last_became_peered":"2024-02-23T00:39:14.779193+0000","last_unstale":"2024-02-23T00:54:35.375490+0000","last_undegraded":"2024-02-23T00:54:35.375490+0000","last_fullsized":"2024-02-23T00:54:35.375490+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:18.475801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:18.475801+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":748,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.372596+0000","last_change":"2024-02-23T00:53:53.408511+0000","last_active":"2024-02-23T00:54:35.372596+0000","last_peered":"2024-02-23T00:54:35.372596+0000","last_clean":"2024-02-23T00:54:35.372596+0000","last_became_active":"2024-02-23T00:53:53.408081+0000","last_became_peered":"2024-02-23T00:53:53.408081+0000","last_unstale":"2024-02-23T00:54:35.372596+0000","last_undegraded":"2024-02-23T00:54:35.372596+0000","last_fullsized":"2024-02-23T00:54:35.372596+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:15.680333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:52:15.680333+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":662,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.380673+0000","last_change":"2024-02-23T00:53:53.421417+0000","last_active":"2024-02-23T00:54:35.380673+0000","last_peered":"2024-02-23T00:54:35.380673+0000","last_clean":"2024-02-23T00:54:35.380673+0000","last_became_active":"2024-02-23T00:53:53.421080+0000","last_became_peered":"2024-02-23T00:53:53.421080+0000","last_unstale":"2024-02-23T00:54:35.380673+0000","last_undegraded":"2024-02-23T00:54:35.380673+0000","last_fullsized":"2024-02-23T00:54:35.380673+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:06.983205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:53:06.983205+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":716,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.387967+0000","last_change":"2024-02-23T00:53:13.356926+0000","last_active":"2024-02-23T00:54:35.387967+0000","last_peered":"2024-02-23T00:54:35.387967+0000","last_clean":"2024-02-23T00:54:35.387967+0000","last_became_active":"2024-02-23T00:53:13.352382+0000","last_became_peered":"2024-02-23T00:53:13.352382+0000","last_unstale":"2024-02-23T00:54:35.387967+0000","last_undegraded":"2024-02-23T00:54:35.387967+0000","last_fullsized":"2024-02-23T00:54:35.387967+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:57.490995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:52:57.490995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,7],"acting":[6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"457'44","reported_seq":609,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.377361+0000","last_change":"2024-02-23T00:54:09.415844+0000","last_active":"2024-02-23T00:54:35.377361+0000","last_peered":"2024-02-23T00:54:35.377361+0000","last_clean":"2024-02-23T00:54:35.377361+0000","last_became_active":"2024-02-23T00:53:13.357708+0000","last_became_peered":"2024-02-23T00:53:13.357708+0000","last_unstale":"2024-02-23T00:54:35.377361+0000","last_undegraded":"2024-02-23T00:54:35.377361+0000","last_fullsized":"2024-02-23T00:54:35.377361+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'44","last_scrub_stamp":"2024-02-23T00:52:23.707992+0000","last_deep_scrub":"457'44","last_deep_scrub_stamp":"2024-02-23T00:49:46.644911+0000","last_clean_scrub_stamp":"2024-02-23T00:52:23.707992+0000","log_size":44,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":715,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.379734+0000","last_change":"2024-02-23T00:53:13.352639+0000","last_active":"2024-02-23T00:54:35.379734+0000","last_peered":"2024-02-23T00:54:35.379734+0000","last_clean":"2024-02-23T00:54:35.379734+0000","last_became_active":"2024-02-23T00:53:13.352119+0000","last_became_peered":"2024-02-23T00:53:13.352119+0000","last_unstale":"2024-02-23T00:54:35.379734+0000","last_undegraded":"2024-02-23T00:54:35.379734+0000","last_fullsized":"2024-02-23T00:54:35.379734+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:51.380778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:51.380778+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":718,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.374048+0000","last_change":"2024-02-23T00:52:07.536352+0000","last_active":"2024-02-23T00:54:35.374048+0000","last_peered":"2024-02-23T00:54:35.374048+0000","last_clean":"2024-02-23T00:54:35.374048+0000","last_became_active":"2024-02-23T00:47:37.184240+0000","last_became_peered":"2024-02-23T00:47:37.184240+0000","last_unstale":"2024-02-23T00:54:35.374048+0000","last_undegraded":"2024-02-23T00:54:35.374048+0000","last_fullsized":"2024-02-23T00:54:35.374048+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":385,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:07.536288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:52:07.536288+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":649,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.377748+0000","last_change":"2024-02-23T00:52:55.401053+0000","last_active":"2024-02-23T00:54:35.377748+0000","last_peered":"2024-02-23T00:54:35.377748+0000","last_clean":"2024-02-23T00:54:35.377748+0000","last_became_active":"2024-02-23T00:42:58.726652+0000","last_became_peered":"2024-02-23T00:42:58.726652+0000","last_unstale":"2024-02-23T00:54:35.377748+0000","last_undegraded":"2024-02-23T00:54:35.377748+0000","last_fullsized":"2024-02-23T00:54:35.377748+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:55.400946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:52:55.400946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":718,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.379332+0000","last_change":"2024-02-23T00:54:03.257007+0000","last_active":"2024-02-23T00:54:35.379332+0000","last_peered":"2024-02-23T00:54:35.379332+0000","last_clean":"2024-02-23T00:54:35.379332+0000","last_became_active":"2024-02-23T00:53:13.346347+0000","last_became_peered":"2024-02-23T00:53:13.346347+0000","last_unstale":"2024-02-23T00:54:35.379332+0000","last_undegraded":"2024-02-23T00:54:35.379332+0000","last_fullsized":"2024-02-23T00:54:35.379332+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:03.256917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:59.694033+0000","last_clean_scrub_stamp":"2024-02-23T00:54:03.256917+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"457'30","reported_seq":590,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.375576+0000","last_change":"2024-02-23T00:53:13.377058+0000","last_active":"2024-02-23T00:54:35.375576+0000","last_peered":"2024-02-23T00:54:35.375576+0000","last_clean":"2024-02-23T00:54:35.375576+0000","last_became_active":"2024-02-23T00:53:13.346657+0000","last_became_peered":"2024-02-23T00:53:13.346657+0000","last_unstale":"2024-02-23T00:54:35.375576+0000","last_undegraded":"2024-02-23T00:54:35.375576+0000","last_fullsized":"2024-02-23T00:54:35.375576+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'30","last_scrub_stamp":"2024-02-23T00:52:16.690860+0000","last_deep_scrub":"95'18","last_deep_scrub_stamp":"2024-02-23T00:42:01.195545+0000","last_clean_scrub_stamp":"2024-02-23T00:52:16.690860+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":720,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.375054+0000","last_change":"2024-02-23T00:53:13.357250+0000","last_active":"2024-02-23T00:54:35.375054+0000","last_peered":"2024-02-23T00:54:35.375054+0000","last_clean":"2024-02-23T00:54:35.375054+0000","last_became_active":"2024-02-23T00:53:13.356475+0000","last_became_peered":"2024-02-23T00:53:13.356475+0000","last_unstale":"2024-02-23T00:54:35.375054+0000","last_undegraded":"2024-02-23T00:54:35.375054+0000","last_fullsized":"2024-02-23T00:54:35.375054+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:22.390399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:22.390399+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":110,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.373341+0000","last_change":"2024-02-23T00:53:53.410002+0000","last_active":"2024-02-23T00:54:35.373341+0000","last_peered":"2024-02-23T00:54:35.373341+0000","last_clean":"2024-02-23T00:54:35.373341+0000","last_became_active":"2024-02-23T00:53:53.409639+0000","last_became_peered":"2024-02-23T00:53:53.409639+0000","last_unstale":"2024-02-23T00:54:35.373341+0000","last_undegraded":"2024-02-23T00:54:35.373341+0000","last_fullsized":"2024-02-23T00:54:35.373341+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:19.764826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:19.764826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.4","version":"67'1","reported_seq":765,"reported_epoch":735,"state":"active+remapped+backfill_toofull","last_fresh":"2024-02-23T00:54:35.375327+0000","last_change":"2024-02-23T00:54:24.406489+0000","last_active":"2024-02-23T00:54:35.375327+0000","last_peered":"2024-02-23T00:54:35.375327+0000","last_clean":"2024-02-23T00:53:51.591230+0000","last_became_active":"2024-02-23T00:53:54.403839+0000","last_became_peered":"2024-02-23T00:53:54.403839+0000","last_unstale":"2024-02-23T00:54:35.375327+0000","last_undegraded":"2024-02-23T00:54:35.375327+0000","last_fullsized":"2024-02-23T00:54:35.375327+0000","mapping_epoch":694,"log_start":"67'1","ondisk_log_start":"67'1","created":65,"last_epoch_clean":670,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:51:02.380721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:51:02.380721+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,6],"avail_no_missing":["1","5","6"],"object_location_counts":[{"shards":"1,5,6","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":671,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.383115+0000","last_change":"2024-02-23T00:53:53.420917+0000","last_active":"2024-02-23T00:54:35.383115+0000","last_peered":"2024-02-23T00:54:35.383115+0000","last_clean":"2024-02-23T00:54:35.383115+0000","last_became_active":"2024-02-23T00:53:53.420593+0000","last_became_peered":"2024-02-23T00:53:53.420593+0000","last_unstale":"2024-02-23T00:54:35.383115+0000","last_undegraded":"2024-02-23T00:54:35.383115+0000","last_fullsized":"2024-02-23T00:54:35.383115+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:51.510968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:35.360054+0000","last_clean_scrub_stamp":"2024-02-23T00:52:51.510968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":725,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.378995+0000","last_change":"2024-02-23T00:53:38.350986+0000","last_active":"2024-02-23T00:54:35.378995+0000","last_peered":"2024-02-23T00:54:35.378995+0000","last_clean":"2024-02-23T00:54:35.378995+0000","last_became_active":"2024-02-23T00:53:13.358164+0000","last_became_peered":"2024-02-23T00:53:13.358164+0000","last_unstale":"2024-02-23T00:54:35.378995+0000","last_undegraded":"2024-02-23T00:54:35.378995+0000","last_fullsized":"2024-02-23T00:54:35.378995+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:38.350917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:53:38.350917+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"457'26","reported_seq":754,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.384130+0000","last_change":"2024-02-23T00:54:01.501056+0000","last_active":"2024-02-23T00:54:35.384130+0000","last_peered":"2024-02-23T00:54:35.384130+0000","last_clean":"2024-02-23T00:54:35.384130+0000","last_became_active":"2024-02-23T00:39:16.858880+0000","last_became_peered":"2024-02-23T00:39:16.858880+0000","last_unstale":"2024-02-23T00:54:35.384130+0000","last_undegraded":"2024-02-23T00:54:35.384130+0000","last_fullsized":"2024-02-23T00:54:35.384130+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-23T00:54:01.500983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:54:01.500983+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":725,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.375101+0000","last_change":"2024-02-23T00:53:13.357436+0000","last_active":"2024-02-23T00:54:35.375101+0000","last_peered":"2024-02-23T00:54:35.375101+0000","last_clean":"2024-02-23T00:54:35.375101+0000","last_became_active":"2024-02-23T00:53:13.356903+0000","last_became_peered":"2024-02-23T00:53:13.356903+0000","last_unstale":"2024-02-23T00:54:35.375101+0000","last_undegraded":"2024-02-23T00:54:35.375101+0000","last_fullsized":"2024-02-23T00:54:35.375101+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:26.398833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:49.977248+0000","last_clean_scrub_stamp":"2024-02-23T00:52:26.398833+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":717,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.377559+0000","last_change":"2024-02-23T00:52:49.465113+0000","last_active":"2024-02-23T00:54:35.377559+0000","last_peered":"2024-02-23T00:54:35.377559+0000","last_clean":"2024-02-23T00:54:35.377559+0000","last_became_active":"2024-02-23T00:47:28.240482+0000","last_became_peered":"2024-02-23T00:47:28.240482+0000","last_unstale":"2024-02-23T00:54:35.377559+0000","last_undegraded":"2024-02-23T00:54:35.377559+0000","last_fullsized":"2024-02-23T00:54:35.377559+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:49.465046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:49.465046+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":89,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.377543+0000","last_change":"2024-02-23T00:53:53.411112+0000","last_active":"2024-02-23T00:54:35.377543+0000","last_peered":"2024-02-23T00:54:35.377543+0000","last_clean":"2024-02-23T00:54:35.377543+0000","last_became_active":"2024-02-23T00:53:53.410801+0000","last_became_peered":"2024-02-23T00:53:53.410801+0000","last_unstale":"2024-02-23T00:54:35.377543+0000","last_undegraded":"2024-02-23T00:54:35.377543+0000","last_fullsized":"2024-02-23T00:54:35.377543+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:53.881152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:53.881152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":743,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.379416+0000","last_change":"2024-02-23T00:54:08.396938+0000","last_active":"2024-02-23T00:54:35.379416+0000","last_peered":"2024-02-23T00:54:35.379416+0000","last_clean":"2024-02-23T00:54:35.379416+0000","last_became_active":"2024-02-23T00:47:45.182550+0000","last_became_peered":"2024-02-23T00:47:45.182550+0000","last_unstale":"2024-02-23T00:54:35.379416+0000","last_undegraded":"2024-02-23T00:54:35.379416+0000","last_fullsized":"2024-02-23T00:54:35.379416+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:08.396857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:54:08.396857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":662,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.381561+0000","last_change":"2024-02-23T00:52:50.382543+0000","last_active":"2024-02-23T00:54:35.381561+0000","last_peered":"2024-02-23T00:54:35.381561+0000","last_clean":"2024-02-23T00:54:35.381561+0000","last_became_active":"2024-02-23T00:47:28.233475+0000","last_became_peered":"2024-02-23T00:47:28.233475+0000","last_unstale":"2024-02-23T00:54:35.381561+0000","last_undegraded":"2024-02-23T00:54:35.381561+0000","last_fullsized":"2024-02-23T00:54:35.381561+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:50.382430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:52:50.382430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":56,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.379964+0000","last_change":"2024-02-23T00:53:45.294558+0000","last_active":"2024-02-23T00:54:35.379964+0000","last_peered":"2024-02-23T00:54:35.379964+0000","last_clean":"2024-02-23T00:54:35.379964+0000","last_became_active":"2024-02-23T00:53:45.294157+0000","last_became_peered":"2024-02-23T00:53:45.294157+0000","last_unstale":"2024-02-23T00:54:35.379964+0000","last_undegraded":"2024-02-23T00:54:35.379964+0000","last_fullsized":"2024-02-23T00:54:35.379964+0000","mapping_epoch":685,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":686,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:39.124251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.264240+0000","last_clean_scrub_stamp":"2024-02-23T00:53:39.124251+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"457'23","reported_seq":766,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.375652+0000","last_change":"2024-02-23T00:53:13.375860+0000","last_active":"2024-02-23T00:54:35.375652+0000","last_peered":"2024-02-23T00:54:35.375652+0000","last_clean":"2024-02-23T00:54:35.375652+0000","last_became_active":"2024-02-23T00:53:13.344389+0000","last_became_peered":"2024-02-23T00:53:13.344389+0000","last_unstale":"2024-02-23T00:54:35.375652+0000","last_undegraded":"2024-02-23T00:54:35.375652+0000","last_fullsized":"2024-02-23T00:54:35.375652+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'23","last_scrub_stamp":"2024-02-23T00:51:42.560138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:42.560138+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":48,"num_read_kb":36,"num_write":26,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":390,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.373742+0000","last_change":"2024-02-23T00:53:53.415443+0000","last_active":"2024-02-23T00:54:35.373742+0000","last_peered":"2024-02-23T00:54:35.373742+0000","last_clean":"2024-02-23T00:54:35.373742+0000","last_became_active":"2024-02-23T00:53:53.415051+0000","last_became_peered":"2024-02-23T00:53:53.415051+0000","last_unstale":"2024-02-23T00:54:35.373742+0000","last_undegraded":"2024-02-23T00:54:35.373742+0000","last_fullsized":"2024-02-23T00:54:35.373742+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:44.659246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:44.659246+0000","last_clean_scrub_stamp":"2024-02-23T00:51:44.659246+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"707'6","reported_seq":1639,"reported_epoch":736,"state":"active+clean","last_fresh":"2024-02-23T00:54:36.381528+0000","last_change":"2024-02-23T00:54:06.324408+0000","last_active":"2024-02-23T00:54:36.381528+0000","last_peered":"2024-02-23T00:54:36.381528+0000","last_clean":"2024-02-23T00:54:36.381528+0000","last_became_active":"2024-02-23T00:53:13.351958+0000","last_became_peered":"2024-02-23T00:53:13.351958+0000","last_unstale":"2024-02-23T00:54:36.381528+0000","last_undegraded":"2024-02-23T00:54:36.381528+0000","last_fullsized":"2024-02-23T00:54:36.381528+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"707'6","last_scrub_stamp":"2024-02-23T00:54:06.324310+0000","last_deep_scrub":"707'6","last_deep_scrub_stamp":"2024-02-23T00:54:06.324310+0000","last_clean_scrub_stamp":"2024-02-23T00:54:06.324310+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":911,"num_read_kb":906,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":736,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.379653+0000","last_change":"2024-02-23T00:52:10.429013+0000","last_active":"2024-02-23T00:54:35.379653+0000","last_peered":"2024-02-23T00:54:35.379653+0000","last_clean":"2024-02-23T00:54:35.379653+0000","last_became_active":"2024-02-23T00:47:52.552244+0000","last_became_peered":"2024-02-23T00:47:52.552244+0000","last_unstale":"2024-02-23T00:54:35.379653+0000","last_undegraded":"2024-02-23T00:54:35.379653+0000","last_fullsized":"2024-02-23T00:54:35.379653+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.428945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.428945+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":648,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.373828+0000","last_change":"2024-02-23T00:53:34.687443+0000","last_active":"2024-02-23T00:54:35.373828+0000","last_peered":"2024-02-23T00:54:35.373828+0000","last_clean":"2024-02-23T00:54:35.373828+0000","last_became_active":"2024-02-23T00:47:28.212516+0000","last_became_peered":"2024-02-23T00:47:28.212516+0000","last_unstale":"2024-02-23T00:54:35.373828+0000","last_undegraded":"2024-02-23T00:54:35.373828+0000","last_fullsized":"2024-02-23T00:54:35.373828+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:34.687347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:53:34.687347+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":90,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.375110+0000","last_change":"2024-02-23T00:53:13.347149+0000","last_active":"2024-02-23T00:54:35.375110+0000","last_peered":"2024-02-23T00:54:35.375110+0000","last_clean":"2024-02-23T00:54:35.375110+0000","last_became_active":"2024-02-23T00:53:13.346793+0000","last_became_peered":"2024-02-23T00:53:13.346793+0000","last_unstale":"2024-02-23T00:54:35.375110+0000","last_undegraded":"2024-02-23T00:54:35.375110+0000","last_fullsized":"2024-02-23T00:54:35.375110+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:17.704169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:52:17.704169+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"694'50","reported_seq":991,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.378004+0000","last_change":"2024-02-23T00:53:54.363459+0000","last_active":"2024-02-23T00:54:35.378004+0000","last_peered":"2024-02-23T00:54:35.378004+0000","last_clean":"2024-02-23T00:54:35.378004+0000","last_became_active":"2024-02-23T00:53:53.412119+0000","last_became_peered":"2024-02-23T00:53:53.412119+0000","last_unstale":"2024-02-23T00:54:35.378004+0000","last_undegraded":"2024-02-23T00:54:35.378004+0000","last_fullsized":"2024-02-23T00:54:35.378004+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'48","last_scrub_stamp":"2024-02-23T00:53:02.417518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:53:02.417518+0000","log_size":50,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":78,"num_read_kb":62,"num_write":44,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":1432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":708,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.376960+0000","last_change":"2024-02-23T00:52:09.366353+0000","last_active":"2024-02-23T00:54:35.376960+0000","last_peered":"2024-02-23T00:54:35.376960+0000","last_clean":"2024-02-23T00:54:35.376960+0000","last_became_active":"2024-02-23T00:39:14.784015+0000","last_became_peered":"2024-02-23T00:39:14.784015+0000","last_unstale":"2024-02-23T00:54:35.376960+0000","last_undegraded":"2024-02-23T00:54:35.376960+0000","last_fullsized":"2024-02-23T00:54:35.376960+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:09.366281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:09.366281+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":477,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.376629+0000","last_change":"2024-02-23T00:53:53.417644+0000","last_active":"2024-02-23T00:54:35.376629+0000","last_peered":"2024-02-23T00:54:35.376629+0000","last_clean":"2024-02-23T00:54:35.376629+0000","last_became_active":"2024-02-23T00:53:53.417293+0000","last_became_peered":"2024-02-23T00:53:53.417293+0000","last_unstale":"2024-02-23T00:54:35.376629+0000","last_undegraded":"2024-02-23T00:54:35.376629+0000","last_fullsized":"2024-02-23T00:54:35.376629+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:33.479056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:53:33.479056+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":380,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.372845+0000","last_change":"2024-02-23T00:52:19.722324+0000","last_active":"2024-02-23T00:54:35.372845+0000","last_peered":"2024-02-23T00:54:35.372845+0000","last_clean":"2024-02-23T00:54:35.372845+0000","last_became_active":"2024-02-23T00:47:28.232194+0000","last_became_peered":"2024-02-23T00:47:28.232194+0000","last_unstale":"2024-02-23T00:54:35.372845+0000","last_undegraded":"2024-02-23T00:54:35.372845+0000","last_fullsized":"2024-02-23T00:54:35.372845+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:19.722210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:19.722210+0000","last_clean_scrub_stamp":"2024-02-23T00:52:19.722210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3","version":"656'10","reported_seq":182,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.378283+0000","last_change":"2024-02-23T00:54:03.457138+0000","last_active":"2024-02-23T00:54:35.378283+0000","last_peered":"2024-02-23T00:54:35.378283+0000","last_clean":"2024-02-23T00:54:35.378283+0000","last_became_active":"2024-02-23T00:53:13.352049+0000","last_became_peered":"2024-02-23T00:53:13.352049+0000","last_unstale":"2024-02-23T00:54:35.378283+0000","last_undegraded":"2024-02-23T00:54:35.378283+0000","last_fullsized":"2024-02-23T00:54:35.378283+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"656'10","last_scrub_stamp":"2024-02-23T00:54:03.457066+0000","last_deep_scrub":"91'9","last_deep_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_clean_scrub_stamp":"2024-02-23T00:54:03.457066+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"457'26","reported_seq":740,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.374753+0000","last_change":"2024-02-23T00:52:14.519112+0000","last_active":"2024-02-23T00:54:35.374753+0000","last_peered":"2024-02-23T00:54:35.374753+0000","last_clean":"2024-02-23T00:54:35.374753+0000","last_became_active":"2024-02-23T00:39:16.787468+0000","last_became_peered":"2024-02-23T00:39:16.787468+0000","last_unstale":"2024-02-23T00:54:35.374753+0000","last_undegraded":"2024-02-23T00:54:35.374753+0000","last_fullsized":"2024-02-23T00:54:35.374753+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-23T00:52:14.519049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:52:14.519049+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":36,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":724,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.374946+0000","last_change":"2024-02-23T00:53:53.412122+0000","last_active":"2024-02-23T00:54:35.374946+0000","last_peered":"2024-02-23T00:54:35.374946+0000","last_clean":"2024-02-23T00:54:35.374946+0000","last_became_active":"2024-02-23T00:53:53.411773+0000","last_became_peered":"2024-02-23T00:53:53.411773+0000","last_unstale":"2024-02-23T00:54:35.374946+0000","last_undegraded":"2024-02-23T00:54:35.374946+0000","last_fullsized":"2024-02-23T00:54:35.374946+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:36.415340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:36.415340+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"107'2","reported_seq":456,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.376323+0000","last_change":"2024-02-23T00:53:56.306419+0000","last_active":"2024-02-23T00:54:35.376323+0000","last_peered":"2024-02-23T00:54:35.376323+0000","last_clean":"2024-02-23T00:54:35.376323+0000","last_became_active":"2024-02-23T00:53:56.306072+0000","last_became_peered":"2024-02-23T00:53:56.306072+0000","last_unstale":"2024-02-23T00:54:35.376323+0000","last_undegraded":"2024-02-23T00:54:35.376323+0000","last_fullsized":"2024-02-23T00:54:35.376323+0000","mapping_epoch":696,"log_start":"107'2","ondisk_log_start":"107'2","created":65,"last_epoch_clean":697,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'2","last_scrub_stamp":"2024-02-23T00:52:26.704495+0000","last_deep_scrub":"107'2","last_deep_scrub_stamp":"2024-02-23T00:42:48.362739+0000","last_clean_scrub_stamp":"2024-02-23T00:52:26.704495+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":671,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.381982+0000","last_change":"2024-02-23T00:54:00.542746+0000","last_active":"2024-02-23T00:54:35.381982+0000","last_peered":"2024-02-23T00:54:35.381982+0000","last_clean":"2024-02-23T00:54:35.381982+0000","last_became_active":"2024-02-23T00:53:13.355381+0000","last_became_peered":"2024-02-23T00:53:13.355381+0000","last_unstale":"2024-02-23T00:54:35.381982+0000","last_undegraded":"2024-02-23T00:54:35.381982+0000","last_fullsized":"2024-02-23T00:54:35.381982+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:00.542667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:54:00.542667+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":93,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.374016+0000","last_change":"2024-02-23T00:53:13.346513+0000","last_active":"2024-02-23T00:54:35.374016+0000","last_peered":"2024-02-23T00:54:35.374016+0000","last_clean":"2024-02-23T00:54:35.374016+0000","last_became_active":"2024-02-23T00:53:13.346122+0000","last_became_peered":"2024-02-23T00:53:13.346122+0000","last_unstale":"2024-02-23T00:54:35.374016+0000","last_undegraded":"2024-02-23T00:54:35.374016+0000","last_fullsized":"2024-02-23T00:54:35.374016+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:05.859124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_clean_scrub_stamp":"2024-02-23T00:53:05.859124+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"457'31","reported_seq":820,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.373468+0000","last_change":"2024-02-23T00:53:54.307638+0000","last_active":"2024-02-23T00:54:35.373468+0000","last_peered":"2024-02-23T00:54:35.373468+0000","last_clean":"2024-02-23T00:54:35.373468+0000","last_became_active":"2024-02-23T00:53:53.410425+0000","last_became_peered":"2024-02-23T00:53:53.410425+0000","last_unstale":"2024-02-23T00:54:35.373468+0000","last_undegraded":"2024-02-23T00:54:35.373468+0000","last_fullsized":"2024-02-23T00:54:35.373468+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:52:58.827562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:52:58.827562+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":58,"num_read_kb":47,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"95'4","reported_seq":744,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.372555+0000","last_change":"2024-02-23T00:53:38.580929+0000","last_active":"2024-02-23T00:54:35.372555+0000","last_peered":"2024-02-23T00:54:35.372555+0000","last_clean":"2024-02-23T00:54:35.372555+0000","last_became_active":"2024-02-23T00:53:13.336812+0000","last_became_peered":"2024-02-23T00:53:13.336812+0000","last_unstale":"2024-02-23T00:54:35.372555+0000","last_undegraded":"2024-02-23T00:54:35.372555+0000","last_fullsized":"2024-02-23T00:54:35.372555+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'4","last_scrub_stamp":"2024-02-23T00:53:38.580857+0000","last_deep_scrub":"95'4","last_deep_scrub_stamp":"2024-02-23T00:41:43.874397+0000","last_clean_scrub_stamp":"2024-02-23T00:53:38.580857+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,5],"acting":[4,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":168,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.381478+0000","last_change":"2024-02-23T00:53:34.318174+0000","last_active":"2024-02-23T00:54:35.381478+0000","last_peered":"2024-02-23T00:54:35.381478+0000","last_clean":"2024-02-23T00:54:35.381478+0000","last_became_active":"2024-02-23T00:53:34.317883+0000","last_became_peered":"2024-02-23T00:53:34.317883+0000","last_unstale":"2024-02-23T00:54:35.381478+0000","last_undegraded":"2024-02-23T00:54:35.381478+0000","last_fullsized":"2024-02-23T00:54:35.381478+0000","mapping_epoch":676,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":677,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:23.776775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:23.776775+0000","last_clean_scrub_stamp":"2024-02-23T00:52:23.776775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":672,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.382904+0000","last_change":"2024-02-23T00:53:13.341671+0000","last_active":"2024-02-23T00:54:35.382904+0000","last_peered":"2024-02-23T00:54:35.382904+0000","last_clean":"2024-02-23T00:54:35.382904+0000","last_became_active":"2024-02-23T00:53:13.341337+0000","last_became_peered":"2024-02-23T00:53:13.341337+0000","last_unstale":"2024-02-23T00:54:35.382904+0000","last_undegraded":"2024-02-23T00:54:35.382904+0000","last_fullsized":"2024-02-23T00:54:35.382904+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:02.354546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:53:02.354546+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"457'27","reported_seq":593,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.380162+0000","last_change":"2024-02-23T00:54:03.163137+0000","last_active":"2024-02-23T00:54:35.380162+0000","last_peered":"2024-02-23T00:54:35.380162+0000","last_clean":"2024-02-23T00:54:35.380162+0000","last_became_active":"2024-02-23T00:53:53.424060+0000","last_became_peered":"2024-02-23T00:53:53.424060+0000","last_unstale":"2024-02-23T00:54:35.380162+0000","last_undegraded":"2024-02-23T00:54:35.380162+0000","last_fullsized":"2024-02-23T00:54:35.380162+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-23T00:54:03.163053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:54:03.163053+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":49,"num_read_kb":41,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":381,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.374208+0000","last_change":"2024-02-23T00:53:13.352149+0000","last_active":"2024-02-23T00:54:35.374208+0000","last_peered":"2024-02-23T00:54:35.374208+0000","last_clean":"2024-02-23T00:54:35.374208+0000","last_became_active":"2024-02-23T00:53:13.351563+0000","last_became_peered":"2024-02-23T00:53:13.351563+0000","last_unstale":"2024-02-23T00:54:35.374208+0000","last_undegraded":"2024-02-23T00:54:35.374208+0000","last_fullsized":"2024-02-23T00:54:35.374208+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:21.536419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:21.536419+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":88,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.375562+0000","last_change":"2024-02-23T00:53:53.411932+0000","last_active":"2024-02-23T00:54:35.375562+0000","last_peered":"2024-02-23T00:54:35.375562+0000","last_clean":"2024-02-23T00:54:35.375562+0000","last_became_active":"2024-02-23T00:53:53.411604+0000","last_became_peered":"2024-02-23T00:53:53.411604+0000","last_unstale":"2024-02-23T00:54:35.375562+0000","last_undegraded":"2024-02-23T00:54:35.375562+0000","last_fullsized":"2024-02-23T00:54:35.375562+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:45.694974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:45.913526+0000","last_clean_scrub_stamp":"2024-02-23T00:51:45.694974+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":93,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.372136+0000","last_change":"2024-02-23T00:53:13.338468+0000","last_active":"2024-02-23T00:54:35.372136+0000","last_peered":"2024-02-23T00:54:35.372136+0000","last_clean":"2024-02-23T00:54:35.372136+0000","last_became_active":"2024-02-23T00:53:13.338084+0000","last_became_peered":"2024-02-23T00:53:13.338084+0000","last_unstale":"2024-02-23T00:54:35.372136+0000","last_undegraded":"2024-02-23T00:54:35.372136+0000","last_fullsized":"2024-02-23T00:54:35.372136+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:24.677786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:52:24.677786+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"457'22","reported_seq":573,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.383071+0000","last_change":"2024-02-23T00:53:54.319331+0000","last_active":"2024-02-23T00:54:35.383071+0000","last_peered":"2024-02-23T00:54:35.383071+0000","last_clean":"2024-02-23T00:54:35.383071+0000","last_became_active":"2024-02-23T00:53:53.416008+0000","last_became_peered":"2024-02-23T00:53:53.416008+0000","last_unstale":"2024-02-23T00:54:35.383071+0000","last_undegraded":"2024-02-23T00:54:35.383071+0000","last_fullsized":"2024-02-23T00:54:35.383071+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'22","last_scrub_stamp":"2024-02-23T00:53:32.317744+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-02-23T00:48:07.941101+0000","last_clean_scrub_stamp":"2024-02-23T00:53:32.317744+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":711,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.382692+0000","last_change":"2024-02-23T00:54:09.497814+0000","last_active":"2024-02-23T00:54:35.382692+0000","last_peered":"2024-02-23T00:54:35.382692+0000","last_clean":"2024-02-23T00:54:35.382692+0000","last_became_active":"2024-02-23T00:39:14.783164+0000","last_became_peered":"2024-02-23T00:39:14.783164+0000","last_unstale":"2024-02-23T00:54:35.382692+0000","last_undegraded":"2024-02-23T00:54:35.382692+0000","last_fullsized":"2024-02-23T00:54:35.382692+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:09.497701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:54:09.497701+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":709,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.378126+0000","last_change":"2024-02-23T00:54:00.288206+0000","last_active":"2024-02-23T00:54:35.378126+0000","last_peered":"2024-02-23T00:54:35.378126+0000","last_clean":"2024-02-23T00:54:35.378126+0000","last_became_active":"2024-02-23T00:39:18.800297+0000","last_became_peered":"2024-02-23T00:39:18.800297+0000","last_unstale":"2024-02-23T00:54:35.378126+0000","last_undegraded":"2024-02-23T00:54:35.378126+0000","last_fullsized":"2024-02-23T00:54:35.378126+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:00.288132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:54:00.288132+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1","version":"110'7","reported_seq":88,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.380986+0000","last_change":"2024-02-23T00:53:34.322710+0000","last_active":"2024-02-23T00:54:35.380986+0000","last_peered":"2024-02-23T00:54:35.380986+0000","last_clean":"2024-02-23T00:54:35.380986+0000","last_became_active":"2024-02-23T00:53:34.322370+0000","last_became_peered":"2024-02-23T00:53:34.322370+0000","last_unstale":"2024-02-23T00:54:35.380986+0000","last_undegraded":"2024-02-23T00:54:35.380986+0000","last_fullsized":"2024-02-23T00:54:35.380986+0000","mapping_epoch":676,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":677,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'7","last_scrub_stamp":"2024-02-23T00:53:04.958282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:53:04.958282+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"457'32","reported_seq":189,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.375140+0000","last_change":"2024-02-23T00:53:34.316882+0000","last_active":"2024-02-23T00:54:35.375140+0000","last_peered":"2024-02-23T00:54:35.375140+0000","last_clean":"2024-02-23T00:54:35.375140+0000","last_became_active":"2024-02-23T00:53:34.316552+0000","last_became_peered":"2024-02-23T00:53:34.316552+0000","last_unstale":"2024-02-23T00:54:35.375140+0000","last_undegraded":"2024-02-23T00:54:35.375140+0000","last_fullsized":"2024-02-23T00:54:35.375140+0000","mapping_epoch":676,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":677,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-23T00:53:30.199998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:53:30.199998+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":32,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":713,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.376877+0000","last_change":"2024-02-23T00:53:35.190352+0000","last_active":"2024-02-23T00:54:35.376877+0000","last_peered":"2024-02-23T00:54:35.376877+0000","last_clean":"2024-02-23T00:54:35.376877+0000","last_became_active":"2024-02-23T00:39:14.785590+0000","last_became_peered":"2024-02-23T00:39:14.785590+0000","last_unstale":"2024-02-23T00:54:35.376877+0000","last_undegraded":"2024-02-23T00:54:35.376877+0000","last_fullsized":"2024-02-23T00:54:35.376877+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:35.190288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_clean_scrub_stamp":"2024-02-23T00:53:35.190288+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"656'10","reported_seq":915,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.378088+0000","last_change":"2024-02-23T00:54:02.269290+0000","last_active":"2024-02-23T00:54:35.378088+0000","last_peered":"2024-02-23T00:54:35.378088+0000","last_clean":"2024-02-23T00:54:35.378088+0000","last_became_active":"2024-02-23T00:53:13.343175+0000","last_became_peered":"2024-02-23T00:53:13.343175+0000","last_unstale":"2024-02-23T00:54:35.378088+0000","last_undegraded":"2024-02-23T00:54:35.378088+0000","last_fullsized":"2024-02-23T00:54:35.378088+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"656'10","last_scrub_stamp":"2024-02-23T00:54:02.269193+0000","last_deep_scrub":"336'9","last_deep_scrub_stamp":"2024-02-23T00:46:49.160991+0000","last_clean_scrub_stamp":"2024-02-23T00:54:02.269193+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":658,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.378602+0000","last_change":"2024-02-23T00:53:13.349504+0000","last_active":"2024-02-23T00:54:35.378602+0000","last_peered":"2024-02-23T00:54:35.378602+0000","last_clean":"2024-02-23T00:54:35.378602+0000","last_became_active":"2024-02-23T00:53:13.349155+0000","last_became_peered":"2024-02-23T00:53:13.349155+0000","last_unstale":"2024-02-23T00:54:35.378602+0000","last_undegraded":"2024-02-23T00:54:35.378602+0000","last_fullsized":"2024-02-23T00:54:35.378602+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:00.410226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:53:00.410226+0000","last_clean_scrub_stamp":"2024-02-23T00:53:00.410226+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"457'21","reported_seq":746,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.377935+0000","last_change":"2024-02-23T00:52:27.456856+0000","last_active":"2024-02-23T00:54:35.377935+0000","last_peered":"2024-02-23T00:54:35.377935+0000","last_clean":"2024-02-23T00:54:35.377935+0000","last_became_active":"2024-02-23T00:39:16.858953+0000","last_became_peered":"2024-02-23T00:39:16.858953+0000","last_unstale":"2024-02-23T00:54:35.377935+0000","last_undegraded":"2024-02-23T00:54:35.377935+0000","last_fullsized":"2024-02-23T00:54:35.377935+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-23T00:52:27.456787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:52:27.456787+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":97,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.372800+0000","last_change":"2024-02-23T00:53:29.500174+0000","last_active":"2024-02-23T00:54:35.372800+0000","last_peered":"2024-02-23T00:54:35.372800+0000","last_clean":"2024-02-23T00:54:35.372800+0000","last_became_active":"2024-02-23T00:53:13.340985+0000","last_became_peered":"2024-02-23T00:53:13.340985+0000","last_unstale":"2024-02-23T00:54:35.372800+0000","last_undegraded":"2024-02-23T00:54:35.372800+0000","last_fullsized":"2024-02-23T00:54:35.372800+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:29.500109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:53:29.500109+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"694'11","reported_seq":130,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.379077+0000","last_change":"2024-02-23T00:53:54.308918+0000","last_active":"2024-02-23T00:54:35.379077+0000","last_peered":"2024-02-23T00:54:35.379077+0000","last_clean":"2024-02-23T00:54:35.379077+0000","last_became_active":"2024-02-23T00:53:53.415965+0000","last_became_peered":"2024-02-23T00:53:53.415965+0000","last_unstale":"2024-02-23T00:54:35.379077+0000","last_undegraded":"2024-02-23T00:54:35.379077+0000","last_fullsized":"2024-02-23T00:54:35.379077+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"376'9","last_scrub_stamp":"2024-02-23T00:52:07.717186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:52:07.717186+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":639,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.374424+0000","last_change":"2024-02-23T00:54:03.413676+0000","last_active":"2024-02-23T00:54:35.374424+0000","last_peered":"2024-02-23T00:54:35.374424+0000","last_clean":"2024-02-23T00:54:35.374424+0000","last_became_active":"2024-02-23T00:42:57.652488+0000","last_became_peered":"2024-02-23T00:42:57.652488+0000","last_unstale":"2024-02-23T00:54:35.374424+0000","last_undegraded":"2024-02-23T00:54:35.374424+0000","last_fullsized":"2024-02-23T00:54:35.374424+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:03.413609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:51.254903+0000","last_clean_scrub_stamp":"2024-02-23T00:54:03.413609+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"457'15","reported_seq":758,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.374546+0000","last_change":"2024-02-23T00:53:13.722705+0000","last_active":"2024-02-23T00:54:35.374546+0000","last_peered":"2024-02-23T00:54:35.374546+0000","last_clean":"2024-02-23T00:54:35.374546+0000","last_became_active":"2024-02-23T00:53:13.350056+0000","last_became_peered":"2024-02-23T00:53:13.350056+0000","last_unstale":"2024-02-23T00:54:35.374546+0000","last_undegraded":"2024-02-23T00:54:35.374546+0000","last_fullsized":"2024-02-23T00:54:35.374546+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'15","last_scrub_stamp":"2024-02-23T00:53:06.466487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:53:06.466487+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":725,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.381849+0000","last_change":"2024-02-23T00:53:45.293671+0000","last_active":"2024-02-23T00:54:35.381849+0000","last_peered":"2024-02-23T00:54:35.381849+0000","last_clean":"2024-02-23T00:54:35.381849+0000","last_became_active":"2024-02-23T00:53:45.293336+0000","last_became_peered":"2024-02-23T00:53:45.293336+0000","last_unstale":"2024-02-23T00:54:35.381849+0000","last_undegraded":"2024-02-23T00:54:35.381849+0000","last_fullsized":"2024-02-23T00:54:35.381849+0000","mapping_epoch":685,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":686,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:03.382646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:53:03.382646+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":709,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.374671+0000","last_change":"2024-02-23T00:53:34.322872+0000","last_active":"2024-02-23T00:54:35.374671+0000","last_peered":"2024-02-23T00:54:35.374671+0000","last_clean":"2024-02-23T00:54:35.374671+0000","last_became_active":"2024-02-23T00:53:34.322592+0000","last_became_peered":"2024-02-23T00:53:34.322592+0000","last_unstale":"2024-02-23T00:54:35.374671+0000","last_undegraded":"2024-02-23T00:54:35.374671+0000","last_fullsized":"2024-02-23T00:54:35.374671+0000","mapping_epoch":676,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":677,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:27.761922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_clean_scrub_stamp":"2024-02-23T00:52:27.761922+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":91,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.375481+0000","last_change":"2024-02-23T00:53:13.351294+0000","last_active":"2024-02-23T00:54:35.375481+0000","last_peered":"2024-02-23T00:54:35.375481+0000","last_clean":"2024-02-23T00:54:35.375481+0000","last_became_active":"2024-02-23T00:53:13.350769+0000","last_became_peered":"2024-02-23T00:53:13.350769+0000","last_unstale":"2024-02-23T00:54:35.375481+0000","last_undegraded":"2024-02-23T00:54:35.375481+0000","last_fullsized":"2024-02-23T00:54:35.375481+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:44.638129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:44.638129+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"457'19","reported_seq":738,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.372938+0000","last_change":"2024-02-23T00:52:52.760115+0000","last_active":"2024-02-23T00:54:35.372938+0000","last_peered":"2024-02-23T00:54:35.372938+0000","last_clean":"2024-02-23T00:54:35.372938+0000","last_became_active":"2024-02-23T00:39:16.783896+0000","last_became_peered":"2024-02-23T00:39:16.783896+0000","last_unstale":"2024-02-23T00:54:35.372938+0000","last_undegraded":"2024-02-23T00:54:35.372938+0000","last_fullsized":"2024-02-23T00:54:35.372938+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'19","last_scrub_stamp":"2024-02-23T00:52:52.760046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:52:52.760046+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"202'2","reported_seq":718,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.375227+0000","last_change":"2024-02-23T00:53:13.385758+0000","last_active":"2024-02-23T00:54:35.375227+0000","last_peered":"2024-02-23T00:54:35.375227+0000","last_clean":"2024-02-23T00:54:35.375227+0000","last_became_active":"2024-02-23T00:53:13.351686+0000","last_became_peered":"2024-02-23T00:53:13.351686+0000","last_unstale":"2024-02-23T00:54:35.375227+0000","last_undegraded":"2024-02-23T00:54:35.375227+0000","last_fullsized":"2024-02-23T00:54:35.375227+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"202'2","last_scrub_stamp":"2024-02-23T00:51:40.511258+0000","last_deep_scrub":"202'2","last_deep_scrub_stamp":"2024-02-23T00:44:34.720640+0000","last_clean_scrub_stamp":"2024-02-23T00:51:40.511258+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":873,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":711,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.374637+0000","last_change":"2024-02-23T00:53:53.417039+0000","last_active":"2024-02-23T00:54:35.374637+0000","last_peered":"2024-02-23T00:54:35.374637+0000","last_clean":"2024-02-23T00:54:35.374637+0000","last_became_active":"2024-02-23T00:53:53.416685+0000","last_became_peered":"2024-02-23T00:53:53.416685+0000","last_unstale":"2024-02-23T00:54:35.374637+0000","last_undegraded":"2024-02-23T00:54:35.374637+0000","last_fullsized":"2024-02-23T00:54:35.374637+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:41.472347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:53:41.472347+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":657,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.375299+0000","last_change":"2024-02-23T00:53:13.360915+0000","last_active":"2024-02-23T00:54:35.375299+0000","last_peered":"2024-02-23T00:54:35.375299+0000","last_clean":"2024-02-23T00:54:35.375299+0000","last_became_active":"2024-02-23T00:53:13.360410+0000","last_became_peered":"2024-02-23T00:53:13.360410+0000","last_unstale":"2024-02-23T00:54:35.375299+0000","last_undegraded":"2024-02-23T00:54:35.375299+0000","last_fullsized":"2024-02-23T00:54:35.375299+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:58.516233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:46.462015+0000","last_clean_scrub_stamp":"2024-02-23T00:52:58.516233+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"457'8","reported_seq":534,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.374376+0000","last_change":"2024-02-23T00:53:53.418225+0000","last_active":"2024-02-23T00:54:35.374376+0000","last_peered":"2024-02-23T00:54:35.374376+0000","last_clean":"2024-02-23T00:54:35.374376+0000","last_became_active":"2024-02-23T00:53:53.403080+0000","last_became_peered":"2024-02-23T00:53:53.403080+0000","last_unstale":"2024-02-23T00:54:35.374376+0000","last_undegraded":"2024-02-23T00:54:35.374376+0000","last_fullsized":"2024-02-23T00:54:35.374376+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'8","last_scrub_stamp":"2024-02-23T00:52:49.890254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:52:49.890254+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":725,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.378099+0000","last_change":"2024-02-23T00:53:40.166252+0000","last_active":"2024-02-23T00:54:35.378099+0000","last_peered":"2024-02-23T00:54:35.378099+0000","last_clean":"2024-02-23T00:54:35.378099+0000","last_became_active":"2024-02-23T00:47:28.224477+0000","last_became_peered":"2024-02-23T00:47:28.224477+0000","last_unstale":"2024-02-23T00:54:35.378099+0000","last_undegraded":"2024-02-23T00:54:35.378099+0000","last_fullsized":"2024-02-23T00:54:35.378099+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:40.166169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:53:40.166169+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":721,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.380852+0000","last_change":"2024-02-23T00:53:59.499809+0000","last_active":"2024-02-23T00:54:35.380852+0000","last_peered":"2024-02-23T00:54:35.380852+0000","last_clean":"2024-02-23T00:54:35.380852+0000","last_became_active":"2024-02-23T00:47:44.230295+0000","last_became_peered":"2024-02-23T00:47:44.230295+0000","last_unstale":"2024-02-23T00:54:35.380852+0000","last_undegraded":"2024-02-23T00:54:35.380852+0000","last_fullsized":"2024-02-23T00:54:35.380852+0000","mapping_epoch":391,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":392,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:59.499666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:53:59.499666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":649,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.380491+0000","last_change":"2024-02-23T00:53:01.388275+0000","last_active":"2024-02-23T00:54:35.380491+0000","last_peered":"2024-02-23T00:54:35.380491+0000","last_clean":"2024-02-23T00:54:35.380491+0000","last_became_active":"2024-02-23T00:42:58.741365+0000","last_became_peered":"2024-02-23T00:42:58.741365+0000","last_unstale":"2024-02-23T00:54:35.380491+0000","last_undegraded":"2024-02-23T00:54:35.380491+0000","last_fullsized":"2024-02-23T00:54:35.380491+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:01.388176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:53:01.388176+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"457'25","reported_seq":538,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.379289+0000","last_change":"2024-02-23T00:53:34.322025+0000","last_active":"2024-02-23T00:54:35.379289+0000","last_peered":"2024-02-23T00:54:35.379289+0000","last_clean":"2024-02-23T00:54:35.379289+0000","last_became_active":"2024-02-23T00:53:34.321651+0000","last_became_peered":"2024-02-23T00:53:34.321651+0000","last_unstale":"2024-02-23T00:54:35.379289+0000","last_undegraded":"2024-02-23T00:54:35.379289+0000","last_fullsized":"2024-02-23T00:54:35.379289+0000","mapping_epoch":676,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":677,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-23T00:53:29.192169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:53:29.192169+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":43,"num_read_kb":35,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":720,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.378559+0000","last_change":"2024-02-23T00:52:53.475514+0000","last_active":"2024-02-23T00:54:35.378559+0000","last_peered":"2024-02-23T00:54:35.378559+0000","last_clean":"2024-02-23T00:54:35.378559+0000","last_became_active":"2024-02-23T00:47:28.240162+0000","last_became_peered":"2024-02-23T00:47:28.240162+0000","last_unstale":"2024-02-23T00:54:35.378559+0000","last_undegraded":"2024-02-23T00:54:35.378559+0000","last_fullsized":"2024-02-23T00:54:35.378559+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:53.475444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:44.285083+0000","last_clean_scrub_stamp":"2024-02-23T00:52:53.475444+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":64,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.374868+0000","last_change":"2024-02-23T00:54:12.297949+0000","last_active":"2024-02-23T00:54:35.374868+0000","last_peered":"2024-02-23T00:54:35.374868+0000","last_clean":"2024-02-23T00:54:35.374868+0000","last_became_active":"2024-02-23T00:53:45.290784+0000","last_became_peered":"2024-02-23T00:53:45.290784+0000","last_unstale":"2024-02-23T00:54:35.374868+0000","last_undegraded":"2024-02-23T00:54:35.374868+0000","last_fullsized":"2024-02-23T00:54:35.374868+0000","mapping_epoch":685,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":686,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:12.297867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:54:12.297867+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":400,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.378759+0000","last_change":"2024-02-23T00:53:53.412837+0000","last_active":"2024-02-23T00:54:35.378759+0000","last_peered":"2024-02-23T00:54:35.378759+0000","last_clean":"2024-02-23T00:54:35.378759+0000","last_became_active":"2024-02-23T00:53:53.412497+0000","last_became_peered":"2024-02-23T00:53:53.412497+0000","last_unstale":"2024-02-23T00:54:35.378759+0000","last_undegraded":"2024-02-23T00:54:35.378759+0000","last_fullsized":"2024-02-23T00:54:35.378759+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:48.449279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:52:48.449279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"457'26","reported_seq":617,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.380382+0000","last_change":"2024-02-23T00:53:54.315328+0000","last_active":"2024-02-23T00:54:35.380382+0000","last_peered":"2024-02-23T00:54:35.380382+0000","last_clean":"2024-02-23T00:54:35.380382+0000","last_became_active":"2024-02-23T00:53:53.416434+0000","last_became_peered":"2024-02-23T00:53:53.416434+0000","last_unstale":"2024-02-23T00:54:35.380382+0000","last_undegraded":"2024-02-23T00:54:35.380382+0000","last_fullsized":"2024-02-23T00:54:35.380382+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-23T00:52:53.810591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:52:53.810591+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":35,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":720,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.376529+0000","last_change":"2024-02-23T00:53:34.208099+0000","last_active":"2024-02-23T00:54:35.376529+0000","last_peered":"2024-02-23T00:54:35.376529+0000","last_clean":"2024-02-23T00:54:35.376529+0000","last_became_active":"2024-02-23T00:39:14.785363+0000","last_became_peered":"2024-02-23T00:39:14.785363+0000","last_unstale":"2024-02-23T00:54:35.376529+0000","last_undegraded":"2024-02-23T00:54:35.376529+0000","last_fullsized":"2024-02-23T00:54:35.376529+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"80'2","last_scrub_stamp":"2024-02-23T00:53:34.208030+0000","last_deep_scrub":"80'2","last_deep_scrub_stamp":"2024-02-23T00:40:15.942725+0000","last_clean_scrub_stamp":"2024-02-23T00:53:34.208030+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"93'9","reported_seq":903,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.378691+0000","last_change":"2024-02-23T00:52:56.514149+0000","last_active":"2024-02-23T00:54:35.378691+0000","last_peered":"2024-02-23T00:54:35.378691+0000","last_clean":"2024-02-23T00:54:35.378691+0000","last_became_active":"2024-02-23T00:39:18.796680+0000","last_became_peered":"2024-02-23T00:39:18.796680+0000","last_unstale":"2024-02-23T00:54:35.378691+0000","last_undegraded":"2024-02-23T00:54:35.378691+0000","last_fullsized":"2024-02-23T00:54:35.378691+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-02-23T00:52:56.514068+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-02-23T00:43:50.393713+0000","last_clean_scrub_stamp":"2024-02-23T00:52:56.514068+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":368,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.383827+0000","last_change":"2024-02-23T00:53:13.353721+0000","last_active":"2024-02-23T00:54:35.383827+0000","last_peered":"2024-02-23T00:54:35.383827+0000","last_clean":"2024-02-23T00:54:35.383827+0000","last_became_active":"2024-02-23T00:53:13.352348+0000","last_became_peered":"2024-02-23T00:53:13.352348+0000","last_unstale":"2024-02-23T00:54:35.383827+0000","last_undegraded":"2024-02-23T00:54:35.383827+0000","last_fullsized":"2024-02-23T00:54:35.383827+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:01.802405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:53:01.802405+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"457'31","reported_seq":771,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.378785+0000","last_change":"2024-02-23T00:54:11.502592+0000","last_active":"2024-02-23T00:54:35.378785+0000","last_peered":"2024-02-23T00:54:35.378785+0000","last_clean":"2024-02-23T00:54:35.378785+0000","last_became_active":"2024-02-23T00:39:16.782660+0000","last_became_peered":"2024-02-23T00:39:16.782660+0000","last_unstale":"2024-02-23T00:54:35.378785+0000","last_undegraded":"2024-02-23T00:54:35.378785+0000","last_fullsized":"2024-02-23T00:54:35.378785+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:54:11.502514+0000","last_deep_scrub":"457'31","last_deep_scrub_stamp":"2024-02-23T00:52:27.357369+0000","last_clean_scrub_stamp":"2024-02-23T00:54:11.502514+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":726,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.380729+0000","last_change":"2024-02-23T00:54:12.460423+0000","last_active":"2024-02-23T00:54:35.380729+0000","last_peered":"2024-02-23T00:54:35.380729+0000","last_clean":"2024-02-23T00:54:35.380729+0000","last_became_active":"2024-02-23T00:53:53.419607+0000","last_became_peered":"2024-02-23T00:53:53.419607+0000","last_unstale":"2024-02-23T00:54:35.380729+0000","last_undegraded":"2024-02-23T00:54:35.380729+0000","last_fullsized":"2024-02-23T00:54:35.380729+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:12.460337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:54:12.460337+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":431,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.380629+0000","last_change":"2024-02-23T00:53:13.346669+0000","last_active":"2024-02-23T00:54:35.380629+0000","last_peered":"2024-02-23T00:54:35.380629+0000","last_clean":"2024-02-23T00:54:35.380629+0000","last_became_active":"2024-02-23T00:53:13.346312+0000","last_became_peered":"2024-02-23T00:53:13.346312+0000","last_unstale":"2024-02-23T00:54:35.380629+0000","last_undegraded":"2024-02-23T00:54:35.380629+0000","last_fullsized":"2024-02-23T00:54:35.380629+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:00.845179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:53.189881+0000","last_clean_scrub_stamp":"2024-02-23T00:53:00.845179+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":71,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.376847+0000","last_change":"2024-02-23T00:53:34.321705+0000","last_active":"2024-02-23T00:54:35.376847+0000","last_peered":"2024-02-23T00:54:35.376847+0000","last_clean":"2024-02-23T00:54:35.376847+0000","last_became_active":"2024-02-23T00:53:34.320314+0000","last_became_peered":"2024-02-23T00:53:34.320314+0000","last_unstale":"2024-02-23T00:54:35.376847+0000","last_undegraded":"2024-02-23T00:54:35.376847+0000","last_fullsized":"2024-02-23T00:54:35.376847+0000","mapping_epoch":676,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":677,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:57.917367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:52:57.917367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"457'27","reported_seq":518,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.375061+0000","last_change":"2024-02-23T00:53:54.456607+0000","last_active":"2024-02-23T00:54:35.375061+0000","last_peered":"2024-02-23T00:54:35.375061+0000","last_clean":"2024-02-23T00:54:35.375061+0000","last_became_active":"2024-02-23T00:53:53.420397+0000","last_became_peered":"2024-02-23T00:53:53.420397+0000","last_unstale":"2024-02-23T00:54:35.375061+0000","last_undegraded":"2024-02-23T00:54:35.375061+0000","last_fullsized":"2024-02-23T00:54:35.375061+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":695,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-23T00:52:55.480770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:52:55.480770+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":711,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.385651+0000","last_change":"2024-02-23T00:52:49.496869+0000","last_active":"2024-02-23T00:54:35.385651+0000","last_peered":"2024-02-23T00:54:35.385651+0000","last_clean":"2024-02-23T00:54:35.385651+0000","last_became_active":"2024-02-23T00:39:14.782494+0000","last_became_peered":"2024-02-23T00:39:14.782494+0000","last_unstale":"2024-02-23T00:54:35.385651+0000","last_undegraded":"2024-02-23T00:54:35.385651+0000","last_fullsized":"2024-02-23T00:54:35.385651+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:49.496739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:49.496739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":702,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.375401+0000","last_change":"2024-02-23T00:53:13.350017+0000","last_active":"2024-02-23T00:54:35.375401+0000","last_peered":"2024-02-23T00:54:35.375401+0000","last_clean":"2024-02-23T00:54:35.375401+0000","last_became_active":"2024-02-23T00:53:13.349684+0000","last_became_peered":"2024-02-23T00:53:13.349684+0000","last_unstale":"2024-02-23T00:54:35.375401+0000","last_undegraded":"2024-02-23T00:54:35.375401+0000","last_fullsized":"2024-02-23T00:54:35.375401+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.563137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.641689+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.563137+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":92,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.387878+0000","last_change":"2024-02-23T00:53:53.424895+0000","last_active":"2024-02-23T00:54:35.387878+0000","last_peered":"2024-02-23T00:54:35.387878+0000","last_clean":"2024-02-23T00:54:35.387878+0000","last_became_active":"2024-02-23T00:53:53.419653+0000","last_became_peered":"2024-02-23T00:53:53.419653+0000","last_unstale":"2024-02-23T00:54:35.387878+0000","last_undegraded":"2024-02-23T00:54:35.387878+0000","last_fullsized":"2024-02-23T00:54:35.387878+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:28.443176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:53:28.443176+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"616'30","reported_seq":754,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.375415+0000","last_change":"2024-02-23T00:53:53.426870+0000","last_active":"2024-02-23T00:54:35.375415+0000","last_peered":"2024-02-23T00:54:35.375415+0000","last_clean":"2024-02-23T00:54:35.375415+0000","last_became_active":"2024-02-23T00:53:53.405043+0000","last_became_peered":"2024-02-23T00:53:53.405043+0000","last_unstale":"2024-02-23T00:54:35.375415+0000","last_undegraded":"2024-02-23T00:54:35.375415+0000","last_fullsized":"2024-02-23T00:54:35.375415+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"616'30","last_scrub_stamp":"2024-02-23T00:52:25.764389+0000","last_deep_scrub":"616'30","last_deep_scrub_stamp":"2024-02-23T00:52:25.764389+0000","last_clean_scrub_stamp":"2024-02-23T00:52:25.764389+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":46,"num_read_kb":36,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":89,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.381306+0000","last_change":"2024-02-23T00:53:53.424319+0000","last_active":"2024-02-23T00:54:35.381306+0000","last_peered":"2024-02-23T00:54:35.381306+0000","last_clean":"2024-02-23T00:54:35.381306+0000","last_became_active":"2024-02-23T00:53:53.423915+0000","last_became_peered":"2024-02-23T00:53:53.423915+0000","last_unstale":"2024-02-23T00:54:35.381306+0000","last_undegraded":"2024-02-23T00:54:35.381306+0000","last_fullsized":"2024-02-23T00:54:35.381306+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:20.774550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:20.774550+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":61,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.381523+0000","last_change":"2024-02-23T00:53:58.483987+0000","last_active":"2024-02-23T00:54:35.381523+0000","last_peered":"2024-02-23T00:54:35.381523+0000","last_clean":"2024-02-23T00:54:35.381523+0000","last_became_active":"2024-02-23T00:53:45.291586+0000","last_became_peered":"2024-02-23T00:53:45.291586+0000","last_unstale":"2024-02-23T00:54:35.381523+0000","last_undegraded":"2024-02-23T00:54:35.381523+0000","last_fullsized":"2024-02-23T00:54:35.381523+0000","mapping_epoch":685,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":686,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:58.483860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_clean_scrub_stamp":"2024-02-23T00:53:58.483860+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":658,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.377650+0000","last_change":"2024-02-23T00:53:30.271476+0000","last_active":"2024-02-23T00:54:35.377650+0000","last_peered":"2024-02-23T00:54:35.377650+0000","last_clean":"2024-02-23T00:54:35.377650+0000","last_became_active":"2024-02-23T00:47:28.203555+0000","last_became_peered":"2024-02-23T00:47:28.203555+0000","last_unstale":"2024-02-23T00:54:35.377650+0000","last_undegraded":"2024-02-23T00:54:35.377650+0000","last_fullsized":"2024-02-23T00:54:35.377650+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:30.271410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:53:30.271410+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"457'33","reported_seq":793,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.371850+0000","last_change":"2024-02-23T00:53:13.369812+0000","last_active":"2024-02-23T00:54:35.371850+0000","last_peered":"2024-02-23T00:54:35.371850+0000","last_clean":"2024-02-23T00:54:35.371850+0000","last_became_active":"2024-02-23T00:53:13.347848+0000","last_became_peered":"2024-02-23T00:53:13.347848+0000","last_unstale":"2024-02-23T00:54:35.371850+0000","last_undegraded":"2024-02-23T00:54:35.371850+0000","last_fullsized":"2024-02-23T00:54:35.371850+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'33","last_scrub_stamp":"2024-02-23T00:52:11.695335+0000","last_deep_scrub":"95'27","last_deep_scrub_stamp":"2024-02-23T00:41:55.919895+0000","last_clean_scrub_stamp":"2024-02-23T00:52:11.695335+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":720,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.378847+0000","last_change":"2024-02-23T00:53:05.309846+0000","last_active":"2024-02-23T00:54:35.378847+0000","last_peered":"2024-02-23T00:54:35.378847+0000","last_clean":"2024-02-23T00:54:35.378847+0000","last_became_active":"2024-02-23T00:47:28.208293+0000","last_became_peered":"2024-02-23T00:47:28.208293+0000","last_unstale":"2024-02-23T00:54:35.378847+0000","last_undegraded":"2024-02-23T00:54:35.378847+0000","last_fullsized":"2024-02-23T00:54:35.378847+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:05.309734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:53:05.309734+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":713,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.377466+0000","last_change":"2024-02-23T00:53:13.358900+0000","last_active":"2024-02-23T00:54:35.377466+0000","last_peered":"2024-02-23T00:54:35.377466+0000","last_clean":"2024-02-23T00:54:35.377466+0000","last_became_active":"2024-02-23T00:53:13.358230+0000","last_became_peered":"2024-02-23T00:53:13.358230+0000","last_unstale":"2024-02-23T00:54:35.377466+0000","last_undegraded":"2024-02-23T00:54:35.377466+0000","last_fullsized":"2024-02-23T00:54:35.377466+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:11.328122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:52:11.328122+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":658,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.380383+0000","last_change":"2024-02-23T00:53:13.351154+0000","last_active":"2024-02-23T00:54:35.380383+0000","last_peered":"2024-02-23T00:54:35.380383+0000","last_clean":"2024-02-23T00:54:35.380383+0000","last_became_active":"2024-02-23T00:53:13.350854+0000","last_became_peered":"2024-02-23T00:53:13.350854+0000","last_unstale":"2024-02-23T00:54:35.380383+0000","last_undegraded":"2024-02-23T00:54:35.380383+0000","last_fullsized":"2024-02-23T00:54:35.380383+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:50.469857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:52:50.469857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"457'17","reported_seq":746,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.372915+0000","last_change":"2024-02-23T00:54:11.404685+0000","last_active":"2024-02-23T00:54:35.372915+0000","last_peered":"2024-02-23T00:54:35.372915+0000","last_clean":"2024-02-23T00:54:35.372915+0000","last_became_active":"2024-02-23T00:39:16.782762+0000","last_became_peered":"2024-02-23T00:39:16.782762+0000","last_unstale":"2024-02-23T00:54:35.372915+0000","last_undegraded":"2024-02-23T00:54:35.372915+0000","last_fullsized":"2024-02-23T00:54:35.372915+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'17","last_scrub_stamp":"2024-02-23T00:54:11.404593+0000","last_deep_scrub":"82'15","last_deep_scrub_stamp":"2024-02-23T00:41:50.975683+0000","last_clean_scrub_stamp":"2024-02-23T00:54:11.404593+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":21,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":720,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.379504+0000","last_change":"2024-02-23T00:53:13.352457+0000","last_active":"2024-02-23T00:54:35.379504+0000","last_peered":"2024-02-23T00:54:35.379504+0000","last_clean":"2024-02-23T00:54:35.379504+0000","last_became_active":"2024-02-23T00:53:13.351527+0000","last_became_peered":"2024-02-23T00:53:13.351527+0000","last_unstale":"2024-02-23T00:54:35.379504+0000","last_undegraded":"2024-02-23T00:54:35.379504+0000","last_fullsized":"2024-02-23T00:54:35.379504+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:52.335390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:52.335390+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":709,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.378497+0000","last_change":"2024-02-23T00:52:56.397935+0000","last_active":"2024-02-23T00:54:35.378497+0000","last_peered":"2024-02-23T00:54:35.378497+0000","last_clean":"2024-02-23T00:54:35.378497+0000","last_became_active":"2024-02-23T00:39:18.796952+0000","last_became_peered":"2024-02-23T00:39:18.796952+0000","last_unstale":"2024-02-23T00:54:35.378497+0000","last_undegraded":"2024-02-23T00:54:35.378497+0000","last_fullsized":"2024-02-23T00:54:35.378497+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:56.397846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:06.491793+0000","last_clean_scrub_stamp":"2024-02-23T00:52:56.397846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":671,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.372610+0000","last_change":"2024-02-23T00:53:53.417926+0000","last_active":"2024-02-23T00:54:35.372610+0000","last_peered":"2024-02-23T00:54:35.372610+0000","last_clean":"2024-02-23T00:54:35.372610+0000","last_became_active":"2024-02-23T00:53:53.417585+0000","last_became_peered":"2024-02-23T00:53:53.417585+0000","last_unstale":"2024-02-23T00:54:35.372610+0000","last_undegraded":"2024-02-23T00:54:35.372610+0000","last_fullsized":"2024-02-23T00:54:35.372610+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:55.788444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:23.224663+0000","last_clean_scrub_stamp":"2024-02-23T00:52:55.788444+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"457'21","reported_seq":752,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.372233+0000","last_change":"2024-02-23T00:53:57.428696+0000","last_active":"2024-02-23T00:54:35.372233+0000","last_peered":"2024-02-23T00:54:35.372233+0000","last_clean":"2024-02-23T00:54:35.372233+0000","last_became_active":"2024-02-23T00:39:16.784606+0000","last_became_peered":"2024-02-23T00:39:16.784606+0000","last_unstale":"2024-02-23T00:54:35.372233+0000","last_undegraded":"2024-02-23T00:54:35.372233+0000","last_fullsized":"2024-02-23T00:54:35.372233+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-23T00:53:57.428626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:53:57.428626+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":707,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.378228+0000","last_change":"2024-02-23T00:53:34.320659+0000","last_active":"2024-02-23T00:54:35.378228+0000","last_peered":"2024-02-23T00:54:35.378228+0000","last_clean":"2024-02-23T00:54:35.378228+0000","last_became_active":"2024-02-23T00:53:34.320264+0000","last_became_peered":"2024-02-23T00:53:34.320264+0000","last_unstale":"2024-02-23T00:54:35.378228+0000","last_undegraded":"2024-02-23T00:54:35.378228+0000","last_fullsized":"2024-02-23T00:54:35.378228+0000","mapping_epoch":676,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":677,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:17.778217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:17.778217+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":695,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.373929+0000","last_change":"2024-02-23T00:53:02.566710+0000","last_active":"2024-02-23T00:54:35.373929+0000","last_peered":"2024-02-23T00:54:35.373929+0000","last_clean":"2024-02-23T00:54:35.373929+0000","last_became_active":"2024-02-23T00:39:18.796312+0000","last_became_peered":"2024-02-23T00:39:18.796312+0000","last_unstale":"2024-02-23T00:54:35.373929+0000","last_undegraded":"2024-02-23T00:54:35.373929+0000","last_fullsized":"2024-02-23T00:54:35.373929+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:02.566644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:47.226331+0000","last_clean_scrub_stamp":"2024-02-23T00:53:02.566644+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":89,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.380462+0000","last_change":"2024-02-23T00:53:13.349831+0000","last_active":"2024-02-23T00:54:35.380462+0000","last_peered":"2024-02-23T00:54:35.380462+0000","last_clean":"2024-02-23T00:54:35.380462+0000","last_became_active":"2024-02-23T00:53:13.349167+0000","last_became_peered":"2024-02-23T00:53:13.349167+0000","last_unstale":"2024-02-23T00:54:35.380462+0000","last_undegraded":"2024-02-23T00:54:35.380462+0000","last_fullsized":"2024-02-23T00:54:35.380462+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:48.749901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:50:28.479512+0000","last_clean_scrub_stamp":"2024-02-23T00:52:48.749901+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"457'62","reported_seq":706,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.376754+0000","last_change":"2024-02-23T00:53:53.441103+0000","last_active":"2024-02-23T00:54:35.376754+0000","last_peered":"2024-02-23T00:54:35.376754+0000","last_clean":"2024-02-23T00:54:35.376754+0000","last_became_active":"2024-02-23T00:53:53.411502+0000","last_became_peered":"2024-02-23T00:53:53.411502+0000","last_unstale":"2024-02-23T00:54:35.376754+0000","last_undegraded":"2024-02-23T00:54:35.376754+0000","last_fullsized":"2024-02-23T00:54:35.376754+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'62","last_scrub_stamp":"2024-02-23T00:52:58.949119+0000","last_deep_scrub":"88'50","last_deep_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_clean_scrub_stamp":"2024-02-23T00:52:58.949119+0000","log_size":62,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":91,"num_read_kb":78,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":91,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.382774+0000","last_change":"2024-02-23T00:53:13.351352+0000","last_active":"2024-02-23T00:54:35.382774+0000","last_peered":"2024-02-23T00:54:35.382774+0000","last_clean":"2024-02-23T00:54:35.382774+0000","last_became_active":"2024-02-23T00:53:13.350934+0000","last_became_peered":"2024-02-23T00:53:13.350934+0000","last_unstale":"2024-02-23T00:54:35.382774+0000","last_undegraded":"2024-02-23T00:54:35.382774+0000","last_fullsized":"2024-02-23T00:54:35.382774+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:59.831412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_clean_scrub_stamp":"2024-02-23T00:52:59.831412+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":705,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.372890+0000","last_change":"2024-02-23T00:53:35.476706+0000","last_active":"2024-02-23T00:54:35.372890+0000","last_peered":"2024-02-23T00:54:35.372890+0000","last_clean":"2024-02-23T00:54:35.372890+0000","last_became_active":"2024-02-23T00:39:18.799723+0000","last_became_peered":"2024-02-23T00:39:18.799723+0000","last_unstale":"2024-02-23T00:54:35.372890+0000","last_undegraded":"2024-02-23T00:54:35.372890+0000","last_fullsized":"2024-02-23T00:54:35.372890+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:35.476621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:53:35.476621+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":658,"reported_epoch":735,"state":"active+clean","last_fresh":"2024-02-23T00:54:35.378622+0000","last_change":"2024-02-23T00:53:13.358051+0000","last_active":"2024-02-23T00:54:35.378622+0000","last_peered":"2024-02-23T00:54:35.378622+0000","last_clean":"2024-02-23T00:54:35.378622+0000","last_became_active":"2024-02-23T00:53:13.357060+0000","last_became_peered":"2024-02-23T00:53:13.357060+0000","last_unstale":"2024-02-23T00:54:35.378622+0000","last_undegraded":"2024-02-23T00:54:35.378622+0000","last_fullsized":"2024-02-23T00:54:35.378622+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:20.418240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:44.601086+0000","last_clean_scrub_stamp":"2024-02-23T00:52:20.418240+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":249,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":6},{"poolid":234,"num_pg":8,"stat_sum":{"num_bytes":4000089,"num_objects":17,"num_object_clones":1,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":15,"num_read_kb":4,"num_write":37,"num_write_kb":3939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":48,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6213632,"data_stored":6213632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":24,"acting":24,"num_store_stats":7},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":12,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1133,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":36,"acting":36,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80,"ondisk_log_size":80,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1497,"num_read_kb":1222,"num_write":895,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":5502,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":512000,"data_stored":13554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":963,"ondisk_log_size":963,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":911,"num_read_kb":906,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":965,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234172,"num_pgs":92,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":306200,"kb_used_data":4880,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93438952,"statfs":{"total":95995035648,"available":95681486848,"internally_reserved":0,"allocated":4997120,"data_stored":3343595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:53:39 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.5,"15min":0.52600000000000002},"min":{"1min":0.375,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.82999999999999996,"5min":0.83899999999999997,"15min":4.9420000000000002},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.51800000000000002,"15min":0.52100000000000002},"min":{"1min":0.38300000000000001,"5min":0.309,"15min":0.24299999999999999},"max":{"1min":0.71199999999999997,"5min":0.82599999999999996,"15min":4.9729999999999999},"last":0.64800000000000002}]},{"osd":1,"last update":"Fri Feb 23 00:53:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.50600000000000001,"15min":0.60699999999999998},"min":{"1min":0.39600000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.78100000000000003,"5min":0.79700000000000004,"15min":10.786},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54300000000000004,"15min":0.622},"min":{"1min":0.41199999999999998,"5min":0.32500000000000001,"15min":0.28100000000000003},"max":{"1min":0.67500000000000004,"5min":2.4199999999999999,"15min":11.885999999999999},"last":0.58899999999999997}]},{"osd":2,"last update":"Fri Feb 23 00:53:39 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51200000000000001,"15min":0.54500000000000004},"min":{"1min":0.33000000000000002,"5min":0.29999999999999999,"15min":0.28999999999999998},"max":{"1min":0.79100000000000004,"5min":0.80600000000000005,"15min":5.0789999999999997},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.52500000000000002,"15min":0.54800000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.316},"max":{"1min":0.72599999999999998,"5min":1.786,"15min":5.1280000000000001},"last":0.71599999999999997}]},{"osd":3,"last update":"Fri Feb 23 00:53:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53700000000000003,"15min":0.623},"min":{"1min":0.34899999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.77100000000000002,"5min":2.536,"15min":15.712999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57299999999999995,"15min":0.62},"min":{"1min":0.35599999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.81999999999999995,"5min":3.7000000000000002,"15min":9.4090000000000007},"last":0.66400000000000003}]},{"osd":4,"last update":"Fri Feb 23 00:53:39 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42199999999999999,"15min":0.438},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.23499999999999999},"max":{"1min":0.74399999999999999,"5min":0.753,"15min":2.0350000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.48899999999999999},"min":{"1min":0.26700000000000002,"5min":0.251,"15min":0.248},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":11.236000000000001},"last":0.41199999999999998}]},{"osd":5,"last update":"Fri Feb 23 00:53:39 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.46200000000000002,"15min":0.48699999999999999},"min":{"1min":0.28999999999999998,"5min":0.27500000000000002,"15min":0.27000000000000002},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":5.4000000000000004},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.49399999999999999,"15min":0.503},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.26700000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":1.728},"last":0.54500000000000004}]},{"osd":6,"last update":"Fri Feb 23 00:53:39 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.502,"15min":0.51100000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.752,"5min":0.75700000000000001,"15min":1.6339999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.502,"15min":0.54800000000000004},"min":{"1min":0.33800000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":13.395},"last":0.56499999999999995}]}]},{"osd":6,"up_from":47,"seq":201863463107,"num_pgs":91,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":316556,"kb_used_data":2948,"kb_used_omap":0,"kb_used_meta":313600,"kb_avail":93428596,"statfs":{"total":95995035648,"available":95670882304,"internally_reserved":0,"allocated":3018752,"data_stored":1354263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321126400},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:54:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52300000000000002,"15min":0.67400000000000004},"min":{"1min":0.36199999999999999,"5min":0.22500000000000001,"15min":0.21099999999999999},"max":{"1min":0.81799999999999995,"5min":0.93799999999999994,"15min":30.641999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57399999999999995,"15min":0.70099999999999996},"min":{"1min":0.29899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.268,"5min":4.3609999999999998,"15min":29.292000000000002},"last":0.46999999999999997}]},{"osd":1,"last update":"Fri Feb 23 00:54:09 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.55900000000000005,"15min":0.68999999999999995},"min":{"1min":0.308,"5min":0.27100000000000002,"15min":0.247},"max":{"1min":0.79100000000000004,"5min":4.1980000000000004,"15min":28.186},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.61799999999999999,"15min":0.877},"min":{"1min":0.41299999999999998,"5min":0.34499999999999997,"15min":0.26900000000000002},"max":{"1min":1.2529999999999999,"5min":6.4370000000000003,"15min":33.104999999999997},"last":0.57399999999999995}]},{"osd":2,"last update":"Fri Feb 23 00:54:09 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.60099999999999998,"15min":0.72599999999999998},"min":{"1min":0.376,"5min":0.35299999999999998,"15min":0.27400000000000002},"max":{"1min":1.893,"5min":4.0919999999999996,"15min":19.960000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52400000000000002,"15min":0.63800000000000001},"min":{"1min":0.35499999999999998,"5min":0.32500000000000001,"15min":0.29899999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":8.5860000000000003},"last":0.45600000000000002}]},{"osd":3,"last update":"Fri Feb 23 00:54:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.63300000000000001},"min":{"1min":0.39400000000000002,"5min":0.30399999999999999,"15min":0.28599999999999998},"max":{"1min":1.236,"5min":4.1340000000000003,"15min":19.562000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.57899999999999996,"15min":0.66900000000000004},"min":{"1min":0.375,"5min":0.35699999999999998,"15min":0.33000000000000002},"max":{"1min":0.75600000000000001,"5min":3.7839999999999998,"15min":14.199},"last":0.51100000000000001}]},{"osd":4,"last update":"Fri Feb 23 00:54:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.42399999999999999,"15min":0.48299999999999998},"min":{"1min":0.313,"5min":0.23200000000000001,"15min":0.218},"max":{"1min":0.54600000000000004,"5min":0.63800000000000001,"15min":11.109999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.436,"15min":0.54700000000000004},"min":{"1min":0.29799999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.56699999999999995,"5min":0.751,"15min":16.734000000000002},"last":0.63400000000000001}]},{"osd":5,"last update":"Fri Feb 23 00:54:09 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.45000000000000001,"15min":0.51900000000000002},"min":{"1min":0.311,"5min":0.308,"15min":0.249},"max":{"1min":0.54800000000000004,"5min":0.72399999999999998,"15min":14.154999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.46899999999999997,"15min":0.55800000000000005},"min":{"1min":0.32400000000000001,"5min":0.29699999999999999,"15min":0.26400000000000001},"max":{"1min":0.61099999999999999,"5min":0.73899999999999999,"15min":8.1400000000000006},"last":0.68899999999999995}]},{"osd":7,"last update":"Fri Feb 23 00:54:32 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47999999999999998,"15min":0.52600000000000002},"min":{"1min":0.34399999999999997,"5min":0.27400000000000002,"15min":0.26800000000000002},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":8.548},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.51700000000000002,"15min":0.60399999999999998},"min":{"1min":0.33600000000000002,"5min":0.32000000000000001,"15min":0.309},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":15.618},"last":0.66800000000000004}]}]},{"osd":1,"up_from":13,"seq":55834575069,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":300160,"kb_used_data":2936,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93444992,"statfs":{"total":95995035648,"available":95687671808,"internally_reserved":0,"allocated":3006464,"data_stored":1320174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:53:59 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.48299999999999998,"15min":0.70899999999999996},"min":{"1min":0.24399999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.624,"5min":7.0540000000000003,"15min":78.361000000000004},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.44,"15min":0.46500000000000002},"min":{"1min":0.26300000000000001,"5min":0.222,"15min":0.191},"max":{"1min":0.80600000000000005,"5min":5.3109999999999999,"15min":5.8849999999999998},"last":0.30199999999999999}]},{"osd":2,"last update":"Fri Feb 23 00:54:20 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.52300000000000002,"15min":0.68600000000000005},"min":{"1min":0.25800000000000001,"5min":0.22700000000000001,"15min":0.188},"max":{"1min":0.60899999999999999,"5min":7.5789999999999997,"15min":48.606000000000002},"last":0.312},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.47499999999999998,"15min":0.58899999999999997},"min":{"1min":0.29399999999999998,"5min":0.28000000000000003,"15min":0.23699999999999999},"max":{"1min":0.95199999999999996,"5min":5.2279999999999998,"15min":23.911000000000001},"last":0.36799999999999999}]},{"osd":3,"last update":"Fri Feb 23 00:53:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48199999999999998,"15min":0.60099999999999998},"min":{"1min":0.34300000000000003,"5min":0.25700000000000001,"15min":0.21199999999999999},"max":{"1min":1.01,"5min":5.2800000000000002,"15min":35.244999999999997},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55600000000000005,"15min":0.70099999999999996},"min":{"1min":0.27700000000000002,"5min":0.26900000000000002,"15min":0.248},"max":{"1min":2.2349999999999999,"5min":7.641,"15min":56.853999999999999},"last":0.378}]},{"osd":4,"last update":"Fri Feb 23 00:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.66300000000000003,"15min":0.82099999999999995},"min":{"1min":0.35899999999999999,"5min":0.32400000000000001,"15min":0.29299999999999998},"max":{"1min":0.84099999999999997,"5min":7.6829999999999998,"15min":37.646999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.71799999999999997,"15min":0.72799999999999998},"min":{"1min":0.39300000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.90500000000000003,"5min":6.9160000000000004,"15min":9.6859999999999999},"last":0.49299999999999999}]},{"osd":5,"last update":"Fri Feb 23 00:53:40 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.68100000000000005,"15min":0.82899999999999996},"min":{"1min":0.379,"5min":0.309,"15min":0.309},"max":{"1min":0.94799999999999995,"5min":7.742,"15min":48.792000000000002},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63500000000000001,"15min":0.69999999999999996},"min":{"1min":0.48499999999999999,"5min":0.39100000000000001,"15min":0.34100000000000003},"max":{"1min":0.88700000000000001,"5min":0.92600000000000005,"15min":11.872999999999999},"last":0.54500000000000004}]},{"osd":6,"last update":"Fri Feb 23 00:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.67400000000000004,"15min":0.81599999999999995},"min":{"1min":0.40500000000000003,"5min":0.37,"15min":0.318},"max":{"1min":0.87,"5min":7.7130000000000001,"15min":40.741},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61599999999999999,"15min":0.66400000000000003},"min":{"1min":0.46500000000000002,"5min":0.40999999999999998,"15min":0.34000000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":6.0069999999999997},"last":0.46700000000000003}]},{"osd":7,"last update":"Fri Feb 23 00:53:40 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.622,"15min":0.71299999999999997},"min":{"1min":0.46400000000000002,"5min":0.40699999999999997,"15min":0.36699999999999999},"max":{"1min":0.84599999999999997,"5min":0.92900000000000005,"15min":20.225000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.745,"15min":0.76400000000000001},"min":{"1min":0.52700000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85599999999999998,"5min":6.8220000000000001,"15min":20.254999999999999},"last":0.55800000000000005}]}]},{"osd":0,"up_from":0,"seq":34359738595,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":0,"seq":77309411544,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":26,"seq":111669149905,"num_pgs":98,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":306200,"kb_used_data":4880,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93438952,"statfs":{"total":95995035648,"available":95681486848,"internally_reserved":0,"allocated":4997120,"data_stored":3333258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.434,"15min":0.52100000000000002},"min":{"1min":0.25800000000000001,"5min":0.216,"15min":0.20899999999999999},"max":{"1min":0.63,"5min":8.3300000000000001,"15min":12.212999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.42999999999999999,"15min":0.59799999999999998},"min":{"1min":0.28699999999999998,"5min":0.22800000000000001,"15min":0.21199999999999999},"max":{"1min":1.3089999999999999,"5min":2.9740000000000002,"15min":11.714},"last":0.41199999999999998}]},{"osd":1,"last update":"Fri Feb 23 00:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41599999999999998,"15min":0.61299999999999999},"min":{"1min":0.28499999999999998,"5min":0.25600000000000001,"15min":0.19900000000000001},"max":{"1min":0.54900000000000004,"5min":3.056,"15min":11.742000000000001},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.44500000000000001,"15min":0.59799999999999998},"min":{"1min":0.218,"5min":0.218,"15min":0.19900000000000001},"max":{"1min":1.7470000000000001,"5min":2.7709999999999999,"15min":8.2270000000000003},"last":0.28399999999999997}]},{"osd":2,"last update":"Fri Feb 23 00:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.48599999999999999,"15min":0.67700000000000005},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.255},"max":{"1min":1.2430000000000001,"5min":3.242,"15min":13.359999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.49299999999999999,"15min":0.65300000000000002},"min":{"1min":0.27600000000000002,"5min":0.27500000000000002,"15min":0.26100000000000001},"max":{"1min":0.70399999999999996,"5min":8.5820000000000007,"15min":21.122},"last":0.439}]},{"osd":4,"last update":"Fri Feb 23 00:54:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59799999999999998,"15min":0.751},"min":{"1min":0.39800000000000002,"5min":0.36099999999999999,"15min":0.29899999999999999},"max":{"1min":0.97599999999999998,"5min":3.1419999999999999,"15min":11.644},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61599999999999999,"15min":0.75600000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.32700000000000001},"max":{"1min":1.2969999999999999,"5min":2.9020000000000001,"15min":12.430999999999999},"last":0.72699999999999998}]},{"osd":5,"last update":"Fri Feb 23 00:53:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.68100000000000005,"15min":0.68600000000000005},"min":{"1min":0.39300000000000002,"5min":0.36799999999999999,"15min":0.35099999999999998},"max":{"1min":0.84899999999999998,"5min":10.231999999999999,"15min":10.231999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63200000000000001,"15min":0.79700000000000004},"min":{"1min":0.40300000000000002,"5min":0.376,"15min":0.35499999999999998},"max":{"1min":0.92800000000000005,"5min":3.1829999999999998,"15min":11.648},"last":0.60499999999999998}]},{"osd":6,"last update":"Fri Feb 23 00:54:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64100000000000001,"15min":0.75800000000000001},"min":{"1min":0.39300000000000002,"5min":0.377,"15min":0.35199999999999998},"max":{"1min":1.3400000000000001,"5min":2.9510000000000001,"15min":6.9420000000000002},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.73499999999999999,"15min":0.72299999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.89500000000000002,"5min":10.965999999999999,"15min":10.965999999999999},"last":0.59799999999999998}]},{"osd":7,"last update":"Fri Feb 23 00:53:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.66200000000000003,"15min":0.83199999999999996},"min":{"1min":0.435,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.022,"5min":3.1030000000000002,"15min":11.676},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.66700000000000004,"15min":0.80900000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41299999999999998},"max":{"1min":0.876,"5min":2.8580000000000001,"15min":7.9800000000000004},"last":0.745}]}]},{"osd":4,"up_from":32,"seq":137438953677,"num_pgs":77,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":302088,"kb_used_data":4864,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93443064,"statfs":{"total":95995035648,"available":95685697536,"internally_reserved":0,"allocated":4980736,"data_stored":3329175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.51600000000000001,"15min":0.51000000000000001},"min":{"1min":0.33100000000000002,"5min":0.28999999999999998,"15min":0.223},"max":{"1min":0.75800000000000001,"5min":1.3819999999999999,"15min":3.5219999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.56299999999999994,"15min":0.58199999999999996},"min":{"1min":0.32900000000000001,"5min":0.29799999999999999,"15min":0.23999999999999999},"max":{"1min":2.5859999999999999,"5min":2.5859999999999999,"15min":8.6140000000000008},"last":0.58199999999999996}]},{"osd":1,"last update":"Fri Feb 23 00:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53200000000000003,"15min":0.57199999999999995},"min":{"1min":0.36699999999999999,"5min":0.31900000000000001,"15min":0.27300000000000002},"max":{"1min":1.23,"5min":1.23,"15min":9.6110000000000007},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.55100000000000005,"15min":0.57399999999999995},"min":{"1min":0.34200000000000003,"5min":0.31900000000000001,"15min":0.255},"max":{"1min":0.71299999999999997,"5min":2.6920000000000002,"15min":6.6950000000000003},"last":0.55300000000000005}]},{"osd":2,"last update":"Fri Feb 23 00:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.54700000000000004,"15min":0.56699999999999995},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.26500000000000001},"max":{"1min":0.78600000000000003,"5min":2.9209999999999998,"15min":3.3069999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.373,"5min":0.33700000000000002,"15min":0.32500000000000001},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":5.8440000000000003},"last":0.621}]},{"osd":3,"last update":"Fri Feb 23 00:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67800000000000005,"15min":0.64000000000000001},"min":{"1min":0.38100000000000001,"5min":0.35699999999999998,"15min":0.316},"max":{"1min":2.6150000000000002,"5min":9.1280000000000001,"15min":9.1280000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56899999999999995,"15min":0.60099999999999998},"min":{"1min":0.39700000000000002,"5min":0.36099999999999999,"15min":0.29699999999999999},"max":{"1min":1.288,"5min":2.1699999999999999,"15min":4.1909999999999998},"last":0.56499999999999995}]},{"osd":5,"last update":"Fri Feb 23 00:53:44 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.46600000000000003,"15min":0.433},"min":{"1min":0.29199999999999998,"5min":0.26700000000000002,"15min":0.24399999999999999},"max":{"1min":0.66000000000000003,"5min":1.694,"15min":1.694},"last":0.498},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.50700000000000001,"15min":0.48799999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.63200000000000001,"5min":1.663,"15min":1.663},"last":0.48899999999999999}]},{"osd":6,"last update":"Fri Feb 23 00:54:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.46100000000000002,"15min":0.44800000000000001},"min":{"1min":0.309,"5min":0.29099999999999998,"15min":0.253},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":0.439},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.49199999999999999,"15min":0.46200000000000002},"min":{"1min":0.30399999999999999,"5min":0.28599999999999998,"15min":0.215},"max":{"1min":1.264,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":0.42399999999999999}]},{"osd":7,"last update":"Fri Feb 23 00:54:36 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.51300000000000001,"15min":0.48799999999999999},"min":{"1min":0.32400000000000001,"5min":0.30599999999999999,"15min":0.254},"max":{"1min":1.363,"5min":2.294,"15min":2.294},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.58699999999999997,"15min":0.55200000000000005},"min":{"1min":0.39200000000000002,"5min":0.37,"15min":0.32600000000000001},"max":{"1min":1.726,"5min":2.282,"15min":2.282},"last":0.54000000000000004}]}]},{"osd":5,"up_from":39,"seq":167503724744,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":312476,"kb_used_data":2964,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93432676,"statfs":{"total":95995035648,"available":95675060224,"internally_reserved":0,"allocated":3035136,"data_stored":1358773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:54:33 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.56399999999999995,"15min":0.61699999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.246},"max":{"1min":0.74399999999999999,"5min":5.2220000000000004,"15min":23.881},"last":0.625},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.53900000000000003,"15min":0.58599999999999997},"min":{"1min":0.38400000000000001,"5min":0.29099999999999998,"15min":0.27000000000000002},"max":{"1min":0.63200000000000001,"5min":3.9249999999999998,"15min":5.4649999999999999},"last":0.50600000000000001}]},{"osd":1,"last update":"Fri Feb 23 00:54:33 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.55300000000000005,"15min":0.65600000000000003},"min":{"1min":0.379,"5min":0.35299999999999998,"15min":0.254},"max":{"1min":0.68700000000000006,"5min":3.9460000000000002,"15min":18.175999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.54100000000000004,"15min":0.621},"min":{"1min":0.41899999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.63600000000000001,"5min":3.976,"15min":9.0950000000000006},"last":0.53400000000000003}]},{"osd":2,"last update":"Fri Feb 23 00:54:33 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.60299999999999998,"15min":0.71799999999999997},"min":{"1min":0.40300000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.72599999999999998,"5min":4.5899999999999999,"15min":22.282},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.60599999999999998,"15min":0.73499999999999999},"min":{"1min":0.375,"5min":0.34399999999999997,"15min":0.29199999999999998},"max":{"1min":0.65400000000000003,"5min":3.8540000000000001,"15min":23.925000000000001},"last":0.55900000000000005}]},{"osd":3,"last update":"Fri Feb 23 00:54:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.54600000000000004,"15min":0.622},"min":{"1min":0.40200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.71499999999999997,"5min":1.194,"15min":6.5730000000000004},"last":0.495},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.61799999999999999,"15min":0.66300000000000003},"min":{"1min":0.34999999999999998,"5min":0.34699999999999998,"15min":0.33500000000000002},"max":{"1min":0.72099999999999997,"5min":3.9980000000000002,"15min":8.1760000000000002},"last":0.59299999999999997}]},{"osd":4,"last update":"Fri Feb 23 00:54:33 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.435,"15min":0.44700000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.23499999999999999},"max":{"1min":0.57099999999999995,"5min":0.86099999999999999,"15min":4.593},"last":0.373},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.47899999999999998,"15min":0.49099999999999999},"min":{"1min":0.33000000000000002,"5min":0.27900000000000003,"15min":0.27500000000000002},"max":{"1min":0.63700000000000001,"5min":1.169,"15min":3.879},"last":0.44700000000000001}]},{"osd":6,"last update":"Fri Feb 23 00:54:02 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.47899999999999998,"15min":0.48699999999999999},"min":{"1min":0.34300000000000003,"5min":0.30199999999999999,"15min":0.27900000000000003},"max":{"1min":0.67100000000000004,"5min":1.1299999999999999,"15min":4.6740000000000004},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.27100000000000002},"max":{"1min":0.74299999999999999,"5min":0.79900000000000004,"15min":1.159},"last":0.40699999999999997}]},{"osd":7,"last update":"Fri Feb 23 00:54:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54500000000000004,"15min":0.55200000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.30399999999999999},"max":{"1min":0.70099999999999996,"5min":1.1870000000000001,"15min":7.0300000000000002},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.51400000000000001,"15min":0.54000000000000004},"min":{"1min":0.35999999999999999,"5min":0.35699999999999998,"15min":0.29099999999999998},"max":{"1min":0.61899999999999999,"5min":0.73599999999999999,"15min":9.0579999999999998},"last":0.46400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":2530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":20480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":16384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2031616,"data_stored":2031616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":2027520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":45056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":40960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2031616,"data_stored":2031616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":249,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":249,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":249,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":249,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":249,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":249,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-23T00:54:41.040 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph pg dump --format=json 2024-02-23T00:54:41.073 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:54:41.076 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:41.077 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:54:41.106 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:54:41.180 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:54:41.207 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:41.280 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:54:41.287 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:41.288 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:54:41.388 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:54:41.422 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:41.442 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:54:41.444 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:41.444 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:54:41.545 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:54:41.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:41.252738+0000 osd.5 (osd.5) 318 : cluster [DBG] 4.3 scrub starts 2024-02-23T00:54:41.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:41.254923+0000 osd.5 (osd.5) 319 : cluster [DBG] 4.3 scrub ok 2024-02-23T00:54:41.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:41.302708+0000 mon.a (mon.0) 3576 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-23T00:54:41.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:41.302811+0000 mon.a (mon.0) 3577 : cluster [DBG] osdmap e741: 8 total, 8 up, 6 in 2024-02-23T00:54:41.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:41.252738+0000 osd.5 (osd.5) 318 : cluster [DBG] 4.3 scrub starts 2024-02-23T00:54:41.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:41.254923+0000 osd.5 (osd.5) 319 : cluster [DBG] 4.3 scrub ok 2024-02-23T00:54:41.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:41.302708+0000 mon.a (mon.0) 3576 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-23T00:54:41.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:41.302811+0000 mon.a (mon.0) 3577 : cluster [DBG] osdmap e741: 8 total, 8 up, 6 in 2024-02-23T00:54:41.577 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:41.598 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:54:41.601 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:41.601 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:54:41.694 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-23T00:54:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:41.252738+0000 osd.5 (osd.5) 318 : cluster [DBG] 4.3 scrub starts 2024-02-23T00:54:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:41.254923+0000 osd.5 (osd.5) 319 : cluster [DBG] 4.3 scrub ok 2024-02-23T00:54:41.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:41.302708+0000 mon.a (mon.0) 3576 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-130","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-23T00:54:41.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:41.302811+0000 mon.a (mon.0) 3577 : cluster [DBG] osdmap e741: 8 total, 8 up, 6 in 2024-02-23T00:54:41.702 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:54:41.767 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:41.891 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:54:41.894 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:41.894 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:54:41.995 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:54:42.027 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:42.048 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:54:42.051 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:42.052 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:54:42.152 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:54:42.181 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:42.201 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:54:42.204 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:42.204 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:54:42.305 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:54:42.332 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:42.351 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:54:42.353 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:42.354 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:54:42.411 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:54:42.454 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:54:42.457 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:54:42.482 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:42.499 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:54:42.502 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:42.502 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:54:42.603 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:54:42.633 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:42.653 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:54:42.656 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:41.388379+0000 mgr.y (mgr.14517) 630 : cluster [DBG] pgmap v1121: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1/726 objects misplaced (0.138%) 2024-02-23T00:54:42.656 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:42.270107+0000 osd.5 (osd.5) 320 : cluster [DBG] 6.0 scrub starts 2024-02-23T00:54:42.656 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:42.271662+0000 osd.5 (osd.5) 321 : cluster [DBG] 6.0 scrub ok 2024-02-23T00:54:42.656 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:41.388379+0000 mgr.y (mgr.14517) 630 : cluster [DBG] pgmap v1121: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1/726 objects misplaced (0.138%) 2024-02-23T00:54:42.656 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:42.270107+0000 osd.5 (osd.5) 320 : cluster [DBG] 6.0 scrub starts 2024-02-23T00:54:42.657 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:42.271662+0000 osd.5 (osd.5) 321 : cluster [DBG] 6.0 scrub ok 2024-02-23T00:54:42.657 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:42.657 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:54:42.694 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:54:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:54:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:54:42.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:41.388379+0000 mgr.y (mgr.14517) 630 : cluster [DBG] pgmap v1121: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1/726 objects misplaced (0.138%) 2024-02-23T00:54:42.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:42.270107+0000 osd.5 (osd.5) 320 : cluster [DBG] 6.0 scrub starts 2024-02-23T00:54:42.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:42.271662+0000 osd.5 (osd.5) 321 : cluster [DBG] 6.0 scrub ok 2024-02-23T00:54:42.702 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:54:42.758 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:54:42.786 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:43.188 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:54:43.188 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-02-23T00:54:43.188 INFO:teuthology.orchestra.run.smithi047.stdout: "num_ops": 0 2024-02-23T00:54:43.188 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:54:43.194 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:54:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:54:43.436 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:54:43.439 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:43.439 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:54:43.540 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:54:43.569 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:43.642 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:54:43.645 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:43.645 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:54:43.746 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:54:43.775 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:44.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:42.941003+0000 mgr.y (mgr.14517) 631 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:54:44.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:43.086423+0000 osd.7 (osd.7) 335 : cluster [DBG] 234.1 scrub starts 2024-02-23T00:54:44.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:43.089892+0000 osd.7 (osd.7) 336 : cluster [DBG] 234.1 scrub ok 2024-02-23T00:54:44.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:43.389131+0000 mgr.y (mgr.14517) 632 : cluster [DBG] pgmap v1122: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1.6 KiB/s wr, 1 op/s; 1/729 objects misplaced (0.137%) 2024-02-23T00:54:44.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:44.329657+0000 mon.a (mon.0) 3578 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 7]}]: dispatch 2024-02-23T00:54:44.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:42.941003+0000 mgr.y (mgr.14517) 631 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:54:44.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:43.086423+0000 osd.7 (osd.7) 335 : cluster [DBG] 234.1 scrub starts 2024-02-23T00:54:44.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:43.089892+0000 osd.7 (osd.7) 336 : cluster [DBG] 234.1 scrub ok 2024-02-23T00:54:44.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:43.389131+0000 mgr.y (mgr.14517) 632 : cluster [DBG] pgmap v1122: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1.6 KiB/s wr, 1 op/s; 1/729 objects misplaced (0.137%) 2024-02-23T00:54:44.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:44.329657+0000 mon.a (mon.0) 3578 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 7]}]: dispatch 2024-02-23T00:54:44.677 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:54:44.680 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:44.681 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:54:44.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:42.941003+0000 mgr.y (mgr.14517) 631 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:54:44.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:43.086423+0000 osd.7 (osd.7) 335 : cluster [DBG] 234.1 scrub starts 2024-02-23T00:54:44.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:43.089892+0000 osd.7 (osd.7) 336 : cluster [DBG] 234.1 scrub ok 2024-02-23T00:54:44.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:43.389131+0000 mgr.y (mgr.14517) 632 : cluster [DBG] pgmap v1122: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1.6 KiB/s wr, 1 op/s; 1/729 objects misplaced (0.137%) 2024-02-23T00:54:44.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:44.329657+0000 mon.a (mon.0) 3578 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 7]}]: dispatch 2024-02-23T00:54:44.782 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:54:44.814 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:44.835 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:54:44.838 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:44.838 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:54:44.939 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:54:44.968 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:44.988 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:54:44.990 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:44.991 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:54:45.091 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:54:45.119 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:45.139 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:54:45.141 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:45.142 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:54:45.243 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:54:45.280 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:45.408 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:44.401998+0000 mon.a (mon.0) 3579 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 7]}]': finished 2024-02-23T00:54:45.408 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:44.401998+0000 mon.a (mon.0) 3579 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 7]}]': finished 2024-02-23T00:54:45.408 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:44.402099+0000 mon.a (mon.0) 3580 : cluster [DBG] osdmap e742: 8 total, 8 up, 6 in 2024-02-23T00:54:45.480 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:54:45.484 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:45.485 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:54:45.585 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:54:45.613 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:45.632 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:54:45.634 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:45.635 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:54:45.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:44.401998+0000 mon.a (mon.0) 3579 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 7]}]': finished 2024-02-23T00:54:45.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:44.402099+0000 mon.a (mon.0) 3580 : cluster [DBG] osdmap e742: 8 total, 8 up, 6 in 2024-02-23T00:54:45.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:44.495779+0000 osd.6 (osd.6) 285 : cluster [DBG] 4.a scrub starts 2024-02-23T00:54:45.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:44.497957+0000 osd.6 (osd.6) 286 : cluster [DBG] 4.a scrub ok 2024-02-23T00:54:45.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:44.503761+0000 osd.4 (osd.4) 305 : cluster [DBG] 18.12 scrub starts 2024-02-23T00:54:45.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:44.504622+0000 osd.4 (osd.4) 306 : cluster [DBG] 18.12 scrub ok 2024-02-23T00:54:45.736 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:54:45.765 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:44.495779+0000 osd.6 (osd.6) 285 : cluster [DBG] 4.a scrub starts 2024-02-23T00:54:45.765 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:44.497957+0000 osd.6 (osd.6) 286 : cluster [DBG] 4.a scrub ok 2024-02-23T00:54:45.765 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:44.503761+0000 osd.4 (osd.4) 305 : cluster [DBG] 18.12 scrub starts 2024-02-23T00:54:45.765 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:44.504622+0000 osd.4 (osd.4) 306 : cluster [DBG] 18.12 scrub ok 2024-02-23T00:54:45.766 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:44.402099+0000 mon.a (mon.0) 3580 : cluster [DBG] osdmap e742: 8 total, 8 up, 6 in 2024-02-23T00:54:45.766 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:44.495779+0000 osd.6 (osd.6) 285 : cluster [DBG] 4.a scrub starts 2024-02-23T00:54:45.766 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:44.497957+0000 osd.6 (osd.6) 286 : cluster [DBG] 4.a scrub ok 2024-02-23T00:54:45.766 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:44.503761+0000 osd.4 (osd.4) 305 : cluster [DBG] 18.12 scrub starts 2024-02-23T00:54:45.766 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:44.504622+0000 osd.4 (osd.4) 306 : cluster [DBG] 18.12 scrub ok 2024-02-23T00:54:45.768 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:45.947 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.2 dump_blocked_ops 2024-02-23T00:54:46.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:45.389555+0000 mgr.y (mgr.14517) 633 : cluster [DBG] pgmap v1124: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1.3 KiB/s wr, 0 op/s; 1/729 objects misplaced (0.137%) 2024-02-23T00:54:46.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:45.412916+0000 mon.a (mon.0) 3581 : cluster [DBG] osdmap e743: 8 total, 8 up, 6 in 2024-02-23T00:54:46.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:45.473453+0000 osd.4 (osd.4) 307 : cluster [DBG] 6.2 scrub starts 2024-02-23T00:54:46.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:45.474834+0000 osd.4 (osd.4) 308 : cluster [DBG] 6.2 scrub ok 2024-02-23T00:54:46.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:45.389555+0000 mgr.y (mgr.14517) 633 : cluster [DBG] pgmap v1124: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1.3 KiB/s wr, 0 op/s; 1/729 objects misplaced (0.137%) 2024-02-23T00:54:46.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:45.412916+0000 mon.a (mon.0) 3581 : cluster [DBG] osdmap e743: 8 total, 8 up, 6 in 2024-02-23T00:54:46.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:45.473453+0000 osd.4 (osd.4) 307 : cluster [DBG] 6.2 scrub starts 2024-02-23T00:54:46.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:45.474834+0000 osd.4 (osd.4) 308 : cluster [DBG] 6.2 scrub ok 2024-02-23T00:54:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:45.389555+0000 mgr.y (mgr.14517) 633 : cluster [DBG] pgmap v1124: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1.3 KiB/s wr, 0 op/s; 1/729 objects misplaced (0.137%) 2024-02-23T00:54:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:45.412916+0000 mon.a (mon.0) 3581 : cluster [DBG] osdmap e743: 8 total, 8 up, 6 in 2024-02-23T00:54:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:45.473453+0000 osd.4 (osd.4) 307 : cluster [DBG] 6.2 scrub starts 2024-02-23T00:54:46.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:45.474834+0000 osd.4 (osd.4) 308 : cluster [DBG] 6.2 scrub ok 2024-02-23T00:54:46.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.526+0000 7f70f8158700 1 -- 172.21.15.47:0/1513833718 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f70f00f8420 msgr2=0x7f70f0104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:46.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.526+0000 7f70f8158700 1 --2- 172.21.15.47:0/1513833718 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f70f00f8420 0x7f70f0104a20 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 rx=0x7f70e8009be0 tx=0x7f70e8004d10).stop 2024-02-23T00:54:46.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.526+0000 7f70f8158700 1 -- 172.21.15.47:0/1513833718 shutdown_connections 2024-02-23T00:54:46.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.526+0000 7f70f8158700 1 --2- 172.21.15.47:0/1513833718 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f70f00f8420 0x7f70f0104a20 unknown :-1 s=CLOSED pgs=2798 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:46.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.526+0000 7f70f8158700 1 --2- 172.21.15.47:0/1513833718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70f00fdbb0 0x7f70f00fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:46.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.526+0000 7f70f8158700 1 --2- 172.21.15.47:0/1513833718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70f00fe510 0x7f70f00f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:46.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.526+0000 7f70f8158700 1 -- 172.21.15.47:0/1513833718 >> 172.21.15.47:0/1513833718 conn(0x7f70f00f3800 msgr2=0x7f70f00f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:46.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.527+0000 7f70f8158700 1 -- 172.21.15.47:0/1513833718 shutdown_connections 2024-02-23T00:54:46.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.527+0000 7f70f8158700 1 -- 172.21.15.47:0/1513833718 wait complete. 2024-02-23T00:54:46.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.527+0000 7f70f8158700 1 librados: starting msgr at 2024-02-23T00:54:46.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.527+0000 7f70f8158700 1 librados: starting objecter 2024-02-23T00:54:46.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.527+0000 7f70f8158700 1 Processor -- start 2024-02-23T00:54:46.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.527+0000 7f70f8158700 1 -- start start 2024-02-23T00:54:46.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.527+0000 7f70f8158700 1 librados: setting wanted keys 2024-02-23T00:54:46.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.527+0000 7f70f8158700 1 librados: calling monclient init 2024-02-23T00:54:46.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.527+0000 7f70f8158700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70f00f8420 0x7f70f010dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:46.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.527+0000 7f70f8158700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70f00fdbb0 0x7f70f010e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:46.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.527+0000 7f70f8158700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f70f00fe510 0x7f70f0107ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:46.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.527+0000 7f70f8158700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f01083e0 con 0x7f70f00f8420 2024-02-23T00:54:47.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.527+0000 7f70f8158700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f0108550 con 0x7f70f00fdbb0 2024-02-23T00:54:47.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.527+0000 7f70f8158700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f0108710 con 0x7f70f00fe510 2024-02-23T00:54:47.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.528+0000 7f70f66f5700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f70f00fe510 0x7f70f0107ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:47.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.528+0000 7f70f66f5700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f70f00fe510 0x7f70f0107ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:42760/0 (socket says 172.21.15.47:42760) 2024-02-23T00:54:47.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.528+0000 7f70f66f5700 1 -- 172.21.15.47:0/1466821215 learned_addr learned my addr 172.21.15.47:0/1466821215 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:54:47.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.528+0000 7f70f5ef4700 1 --2- 172.21.15.47:0/1466821215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70f00f8420 0x7f70f010dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:47.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.528+0000 7f70f56f3700 1 --2- 172.21.15.47:0/1466821215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70f00fdbb0 0x7f70f010e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:47.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.528+0000 7f70f66f5700 1 -- 172.21.15.47:0/1466821215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70f00fdbb0 msgr2=0x7f70f010e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:47.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.528+0000 7f70f66f5700 1 --2- 172.21.15.47:0/1466821215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70f00fdbb0 0x7f70f010e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:47.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.528+0000 7f70f66f5700 1 -- 172.21.15.47:0/1466821215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70f00f8420 msgr2=0x7f70f010dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:47.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.528+0000 7f70f66f5700 1 --2- 172.21.15.47:0/1466821215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70f00f8420 0x7f70f010dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:47.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.528+0000 7f70f66f5700 1 -- 172.21.15.47:0/1466821215 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ec005040 con 0x7f70f00fe510 2024-02-23T00:54:47.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.528+0000 7f70f56f3700 1 --2- 172.21.15.47:0/1466821215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70f00fdbb0 0x7f70f010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-23T00:54:47.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.528+0000 7f70f5ef4700 1 --2- 172.21.15.47:0/1466821215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70f00f8420 0x7f70f010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:54:47.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.528+0000 7f70f66f5700 1 --2- 172.21.15.47:0/1466821215 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f70f00fe510 0x7f70f0107ef0 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 rx=0x7f70e8005010 tx=0x7f70e8004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:47.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.528+0000 7f70e2ffd700 10 client.?.objecter ms_handle_connect 0x7f70f00fe510 2024-02-23T00:54:47.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.528+0000 7f70e2ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:54:47.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.528+0000 7f70e2ffd700 1 -- 172.21.15.47:0/1466821215 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e800e030 con 0x7f70f00fe510 2024-02-23T00:54:47.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.528+0000 7f70e2ffd700 1 -- 172.21.15.47:0/1466821215 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f70e8007bf0 con 0x7f70f00fe510 2024-02-23T00:54:47.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.528+0000 7f70f8158700 1 -- 172.21.15.47:0/1466821215 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e80097c0 con 0x7f70f00fe510 2024-02-23T00:54:47.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.528+0000 7f70e2ffd700 1 -- 172.21.15.47:0/1466821215 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e8018070 con 0x7f70f00fe510 2024-02-23T00:54:47.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.528+0000 7f70f8158700 10 client.42044.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:54:47.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.528+0000 7f70f8158700 1 -- 172.21.15.47:0/1466821215 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f0108c30 con 0x7f70f00fe510 2024-02-23T00:54:47.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.528+0000 7f70f8158700 1 librados: init done 2024-02-23T00:54:47.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.529+0000 7f70f8158700 1 -- 172.21.15.47:0/1466821215 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70f005ee80 con 0x7f70f00fe510 2024-02-23T00:54:47.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.530+0000 7f70e2ffd700 1 -- 172.21.15.47:0/1466821215 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f70e801d420 con 0x7f70f00fe510 2024-02-23T00:54:47.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.530+0000 7f70e2ffd700 1 --2- 172.21.15.47:0/1466821215 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f70dc062cc0 0x7f70dc065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:47.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.530+0000 7f70e2ffd700 1 -- 172.21.15.47:0/1466821215 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(743..743 src has 621..743) v4 ==== 9807+0+0 (secure 0 0 0) 0x7f70e807f870 con 0x7f70f00fe510 2024-02-23T00:54:47.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.530+0000 7f70e2ffd700 10 client.42044.objecter ms_dispatch 0x7f70f0001550 osd_map(743..743 src has 621..743) v4 2024-02-23T00:54:47.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.530+0000 7f70e2ffd700 3 client.42044.objecter handle_osd_map got epochs [743,743] > 0 2024-02-23T00:54:47.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.530+0000 7f70e2ffd700 3 client.42044.objecter handle_osd_map decoding full epoch 743 2024-02-23T00:54:47.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.530+0000 7f70f5ef4700 1 --2- 172.21.15.47:0/1466821215 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f70dc062cc0 0x7f70dc065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:47.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.531+0000 7f70e2ffd700 20 client.42044.objecter dump_active .. 0 homeless 2024-02-23T00:54:47.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.531+0000 7f70e2ffd700 10 client.42044.objecter ms_handle_connect 0x7f70dc062cc0 2024-02-23T00:54:47.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.531+0000 7f70f5ef4700 1 --2- 172.21.15.47:0/1466821215 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f70dc062cc0 0x7f70dc065180 secure :-1 s=READY pgs=3191 cs=0 l=1 rev1=1 rx=0x7f70ec009760 tx=0x7f70ec00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:47.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.535+0000 7f70e2ffd700 1 -- 172.21.15.47:0/1466821215 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f70e804ec40 con 0x7f70f00fe510 2024-02-23T00:54:47.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.704+0000 7f70f8158700 1 -- 172.21.15.47:0/1466821215 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f70f005fa60 con 0x7f70dc062cc0 2024-02-23T00:54:47.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.717+0000 7f70e2ffd700 1 -- 172.21.15.47:0/1466821215 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+473009 (secure 0 0 0) 0x7f70f005fa60 con 0x7f70dc062cc0 2024-02-23T00:54:47.008 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:54:47.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.997+0000 7f70f8158700 10 librados: watch_flush enter 2024-02-23T00:54:47.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.997+0000 7f70f8158700 10 librados: watch_flush exit 2024-02-23T00:54:47.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.997+0000 7f70f8158700 20 client.42044.objecter shutdown clearing up homeless session... 2024-02-23T00:54:47.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.997+0000 7f70f8158700 10 client.42044.objecter successfully canceled tick 2024-02-23T00:54:47.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.997+0000 7f70f8158700 1 -- 172.21.15.47:0/1466821215 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f70dc062cc0 msgr2=0x7f70dc065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:47.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.997+0000 7f70f8158700 1 --2- 172.21.15.47:0/1466821215 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f70dc062cc0 0x7f70dc065180 secure :-1 s=READY pgs=3191 cs=0 l=1 rev1=1 rx=0x7f70ec009760 tx=0x7f70ec00b040).stop 2024-02-23T00:54:47.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.997+0000 7f70f8158700 1 -- 172.21.15.47:0/1466821215 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f70f00fe510 msgr2=0x7f70f0107ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:47.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.997+0000 7f70f8158700 1 --2- 172.21.15.47:0/1466821215 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f70f00fe510 0x7f70f0107ef0 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 rx=0x7f70e8005010 tx=0x7f70e8004d10).stop 2024-02-23T00:54:47.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.997+0000 7f70f8158700 1 -- 172.21.15.47:0/1466821215 shutdown_connections 2024-02-23T00:54:47.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.997+0000 7f70f8158700 1 --2- 172.21.15.47:0/1466821215 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f70dc062cc0 0x7f70dc065180 unknown :-1 s=CLOSED pgs=3191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:47.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.997+0000 7f70f8158700 1 --2- 172.21.15.47:0/1466821215 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f70f00fe510 0x7f70f0107ef0 unknown :-1 s=CLOSED pgs=2799 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:47.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.997+0000 7f70f8158700 1 --2- 172.21.15.47:0/1466821215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70f00f8420 0x7f70f010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:47.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.997+0000 7f70f8158700 1 --2- 172.21.15.47:0/1466821215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70f00fdbb0 0x7f70f010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:47.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.997+0000 7f70f8158700 1 -- 172.21.15.47:0/1466821215 >> 172.21.15.47:0/1466821215 conn(0x7f70f00f3800 msgr2=0x7f70f00fbb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:47.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.997+0000 7f70f8158700 1 -- 172.21.15.47:0/1466821215 shutdown_connections 2024-02-23T00:54:47.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.997+0000 7f70f8158700 1 -- 172.21.15.47:0/1466821215 wait complete. 2024-02-23T00:54:47.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.997+0000 7f70f8158700 1 librados: shutdown 2024-02-23T00:54:47.014 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-02-23T00:54:47.018 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:54:47.021 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:47.022 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:54:47.123 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:54:47.155 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:47.177 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:54:47.180 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:47.180 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:54:47.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.781+0000 7f3c988cc700 1 -- 172.21.15.47:0/4180381408 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3c940fdbb0 msgr2=0x7f3c940fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:47.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.781+0000 7f3c988cc700 1 --2- 172.21.15.47:0/4180381408 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3c940fdbb0 0x7f3c940fdf90 secure :-1 s=READY pgs=2800 cs=0 l=1 rev1=1 rx=0x7f3c84009be0 tx=0x7f3c84004d10).stop 2024-02-23T00:54:47.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.781+0000 7f3c988cc700 1 -- 172.21.15.47:0/4180381408 shutdown_connections 2024-02-23T00:54:47.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.781+0000 7f3c988cc700 1 --2- 172.21.15.47:0/4180381408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c940f8420 0x7f3c94104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:47.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.781+0000 7f3c988cc700 1 --2- 172.21.15.47:0/4180381408 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3c940fdbb0 0x7f3c940fdf90 unknown :-1 s=CLOSED pgs=2800 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:47.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.781+0000 7f3c988cc700 1 --2- 172.21.15.47:0/4180381408 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c940fe510 0x7f3c940f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:47.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.781+0000 7f3c988cc700 1 -- 172.21.15.47:0/4180381408 >> 172.21.15.47:0/4180381408 conn(0x7f3c940f3800 msgr2=0x7f3c940f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:47.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.781+0000 7f3c988cc700 1 -- 172.21.15.47:0/4180381408 shutdown_connections 2024-02-23T00:54:47.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.781+0000 7f3c988cc700 1 -- 172.21.15.47:0/4180381408 wait complete. 2024-02-23T00:54:47.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.781+0000 7f3c988cc700 1 librados: starting msgr at 2024-02-23T00:54:47.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.781+0000 7f3c988cc700 1 librados: starting objecter 2024-02-23T00:54:47.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.782+0000 7f3c988cc700 1 Processor -- start 2024-02-23T00:54:47.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.782+0000 7f3c988cc700 1 -- start start 2024-02-23T00:54:47.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.782+0000 7f3c988cc700 1 librados: setting wanted keys 2024-02-23T00:54:47.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.782+0000 7f3c988cc700 1 librados: calling monclient init 2024-02-23T00:54:47.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.782+0000 7f3c988cc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c940f8420 0x7f3c9406b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:47.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.782+0000 7f3c988cc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3c940fdbb0 0x7f3c9406b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:47.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.782+0000 7f3c988cc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c940fe510 0x7f3c9406bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:47.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.782+0000 7f3c988cc700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c94116380 con 0x7f3c940f8420 2024-02-23T00:54:47.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.782+0000 7f3c988cc700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c941164f0 con 0x7f3c940fe510 2024-02-23T00:54:47.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.782+0000 7f3c988cc700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c941166b0 con 0x7f3c940fdbb0 2024-02-23T00:54:47.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.782+0000 7f3c91d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3c940fdbb0 0x7f3c9406b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:47.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.782+0000 7f3c9259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c940f8420 0x7f3c9406b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:47.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.782+0000 7f3c91d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3c940fdbb0 0x7f3c9406b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:42766/0 (socket says 172.21.15.47:42766) 2024-02-23T00:54:47.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.782+0000 7f3c91d9b700 1 -- 172.21.15.47:0/860559830 learned_addr learned my addr 172.21.15.47:0/860559830 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:54:47.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.782+0000 7f3c92d9d700 1 --2- 172.21.15.47:0/860559830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c940fe510 0x7f3c9406bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:47.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.782+0000 7f3c91d9b700 1 -- 172.21.15.47:0/860559830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c940fe510 msgr2=0x7f3c9406bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:47.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.782+0000 7f3c91d9b700 1 --2- 172.21.15.47:0/860559830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c940fe510 0x7f3c9406bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:47.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.782+0000 7f3c91d9b700 1 -- 172.21.15.47:0/860559830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c940f8420 msgr2=0x7f3c9406b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:47.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.782+0000 7f3c91d9b700 1 --2- 172.21.15.47:0/860559830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c940f8420 0x7f3c9406b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:47.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.782+0000 7f3c91d9b700 1 -- 172.21.15.47:0/860559830 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c8c005040 con 0x7f3c940fdbb0 2024-02-23T00:54:47.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.783+0000 7f3c9259c700 1 --2- 172.21.15.47:0/860559830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c940f8420 0x7f3c9406b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:54:47.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.783+0000 7f3c91d9b700 1 --2- 172.21.15.47:0/860559830 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3c940fdbb0 0x7f3c9406b5c0 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 rx=0x7f3c7c00c7c0 tx=0x7f3c7c007ef0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:47.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.783+0000 7f3c8b7fe700 10 client.?.objecter ms_handle_connect 0x7f3c940fdbb0 2024-02-23T00:54:47.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.783+0000 7f3c8b7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:54:47.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.783+0000 7f3c8b7fe700 1 -- 172.21.15.47:0/860559830 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c004430 con 0x7f3c940fdbb0 2024-02-23T00:54:47.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.783+0000 7f3c8b7fe700 1 -- 172.21.15.47:0/860559830 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f3c7c00e930 con 0x7f3c940fdbb0 2024-02-23T00:54:47.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.783+0000 7f3c988cc700 1 -- 172.21.15.47:0/860559830 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c840097c0 con 0x7f3c940fdbb0 2024-02-23T00:54:47.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.783+0000 7f3c8b7fe700 1 -- 172.21.15.47:0/860559830 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c0139a0 con 0x7f3c940fdbb0 2024-02-23T00:54:47.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.783+0000 7f3c988cc700 10 client.42050.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:54:47.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.783+0000 7f3c988cc700 1 -- 172.21.15.47:0/860559830 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c94116dc0 con 0x7f3c940fdbb0 2024-02-23T00:54:47.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.783+0000 7f3c988cc700 1 librados: init done 2024-02-23T00:54:47.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.784+0000 7f3c988cc700 1 -- 172.21.15.47:0/860559830 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c9405ee80 con 0x7f3c940fdbb0 2024-02-23T00:54:47.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.784+0000 7f3c8b7fe700 1 -- 172.21.15.47:0/860559830 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f3c7c00e4a0 con 0x7f3c940fdbb0 2024-02-23T00:54:47.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.784+0000 7f3c8b7fe700 1 --2- 172.21.15.47:0/860559830 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3c80062cc0 0x7f3c80065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:47.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.785+0000 7f3c8b7fe700 1 -- 172.21.15.47:0/860559830 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(743..743 src has 621..743) v4 ==== 9807+0+0 (secure 0 0 0) 0x7f3c7c07d550 con 0x7f3c940fdbb0 2024-02-23T00:54:47.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.785+0000 7f3c8b7fe700 10 client.42050.objecter ms_dispatch 0x7f3c94001550 osd_map(743..743 src has 621..743) v4 2024-02-23T00:54:47.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.785+0000 7f3c8b7fe700 3 client.42050.objecter handle_osd_map got epochs [743,743] > 0 2024-02-23T00:54:47.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.785+0000 7f3c8b7fe700 3 client.42050.objecter handle_osd_map decoding full epoch 743 2024-02-23T00:54:47.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.785+0000 7f3c9259c700 1 --2- 172.21.15.47:0/860559830 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3c80062cc0 0x7f3c80065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:47.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.785+0000 7f3c8b7fe700 20 client.42050.objecter dump_active .. 0 homeless 2024-02-23T00:54:47.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.785+0000 7f3c8b7fe700 10 client.42050.objecter ms_handle_connect 0x7f3c80062cc0 2024-02-23T00:54:47.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.785+0000 7f3c9259c700 1 --2- 172.21.15.47:0/860559830 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3c80062cc0 0x7f3c80065180 secure :-1 s=READY pgs=3192 cs=0 l=1 rev1=1 rx=0x7f3c84004e00 tx=0x7f3c84004d10).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:47.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.789+0000 7f3c8b7fe700 1 -- 172.21.15.47:0/860559830 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f3c7c04c920 con 0x7f3c940fdbb0 2024-02-23T00:54:47.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:46.970+0000 7f3c988cc700 1 -- 172.21.15.47:0/860559830 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7f3c9406c650 con 0x7f3c940fdbb0 2024-02-23T00:54:47.281 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:54:47.310 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:47.331 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:54:47.334 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:47.334 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:54:47.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:47.418+0000 7f3c8b7fe700 1 -- 172.21.15.47:0/860559830 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v744) v1 ==== 101+0+0 (secure 0 0 0) 0x7f3c7c04ff40 con 0x7f3c940fdbb0 2024-02-23T00:54:47.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:47.420+0000 7f3c988cc700 10 librados: watch_flush enter 2024-02-23T00:54:47.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:47.420+0000 7f3c988cc700 10 librados: watch_flush exit 2024-02-23T00:54:47.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:47.420+0000 7f3c988cc700 20 client.42050.objecter shutdown clearing up homeless session... 2024-02-23T00:54:47.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:47.420+0000 7f3c988cc700 10 client.42050.objecter successfully canceled tick 2024-02-23T00:54:47.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:47.420+0000 7f3c988cc700 1 -- 172.21.15.47:0/860559830 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3c80062cc0 msgr2=0x7f3c80065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:47.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:47.420+0000 7f3c988cc700 1 --2- 172.21.15.47:0/860559830 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3c80062cc0 0x7f3c80065180 secure :-1 s=READY pgs=3192 cs=0 l=1 rev1=1 rx=0x7f3c84004e00 tx=0x7f3c84004d10).stop 2024-02-23T00:54:47.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:47.420+0000 7f3c988cc700 1 -- 172.21.15.47:0/860559830 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3c940fdbb0 msgr2=0x7f3c9406b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:47.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:47.420+0000 7f3c988cc700 1 --2- 172.21.15.47:0/860559830 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3c940fdbb0 0x7f3c9406b5c0 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 rx=0x7f3c7c00c7c0 tx=0x7f3c7c007ef0).stop 2024-02-23T00:54:47.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:47.420+0000 7f3c988cc700 1 -- 172.21.15.47:0/860559830 shutdown_connections 2024-02-23T00:54:47.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:47.420+0000 7f3c988cc700 1 --2- 172.21.15.47:0/860559830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c940fe510 0x7f3c9406bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:47.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:47.420+0000 7f3c988cc700 1 --2- 172.21.15.47:0/860559830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c940f8420 0x7f3c9406b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:47.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:47.420+0000 7f3c988cc700 1 --2- 172.21.15.47:0/860559830 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3c80062cc0 0x7f3c80065180 unknown :-1 s=CLOSED pgs=3192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:47.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:47.420+0000 7f3c988cc700 1 --2- 172.21.15.47:0/860559830 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3c940fdbb0 0x7f3c9406b5c0 unknown :-1 s=CLOSED pgs=2801 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:47.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:47.420+0000 7f3c988cc700 1 -- 172.21.15.47:0/860559830 >> 172.21.15.47:0/860559830 conn(0x7f3c940f3800 msgr2=0x7f3c940f5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:47.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:47.420+0000 7f3c988cc700 1 -- 172.21.15.47:0/860559830 shutdown_connections 2024-02-23T00:54:47.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:47.420+0000 7f3c988cc700 1 -- 172.21.15.47:0/860559830 wait complete. 2024-02-23T00:54:47.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:47.420+0000 7f3c988cc700 1 librados: shutdown 2024-02-23T00:54:47.425 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is unset 2024-02-23T00:54:47.435 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:54:47.465 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:47.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:46.329448+0000 osd.1 (osd.1) 291 : cluster [DBG] 18.1a scrub starts 2024-02-23T00:54:47.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:46.332511+0000 osd.1 (osd.1) 292 : cluster [DBG] 18.1a scrub ok 2024-02-23T00:54:47.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:46.509454+0000 osd.6 (osd.6) 287 : cluster [DBG] 18.10 scrub starts 2024-02-23T00:54:47.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:46.510468+0000 osd.6 (osd.6) 288 : cluster [DBG] 18.10 scrub ok 2024-02-23T00:54:47.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:46.706115+0000 mgr.y (mgr.14517) 634 : audit [DBG] from='client.42044 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:54:47.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:46.972140+0000 mon.c (mon.2) 450 : audit [INF] from='client.? 172.21.15.47:0/860559830' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:54:47.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:46.972477+0000 mon.a (mon.0) 3582 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:54:47.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:46.329448+0000 osd.1 (osd.1) 291 : cluster [DBG] 18.1a scrub starts 2024-02-23T00:54:47.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:46.332511+0000 osd.1 (osd.1) 292 : cluster [DBG] 18.1a scrub ok 2024-02-23T00:54:47.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:46.509454+0000 osd.6 (osd.6) 287 : cluster [DBG] 18.10 scrub starts 2024-02-23T00:54:47.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:46.510468+0000 osd.6 (osd.6) 288 : cluster [DBG] 18.10 scrub ok 2024-02-23T00:54:47.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:46.706115+0000 mgr.y (mgr.14517) 634 : audit [DBG] from='client.42044 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:54:47.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:46.972140+0000 mon.c (mon.2) 450 : audit [INF] from='client.? 172.21.15.47:0/860559830' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:54:47.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:46.972477+0000 mon.a (mon.0) 3582 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:54:47.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:46.329448+0000 osd.1 (osd.1) 291 : cluster [DBG] 18.1a scrub starts 2024-02-23T00:54:47.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:46.332511+0000 osd.1 (osd.1) 292 : cluster [DBG] 18.1a scrub ok 2024-02-23T00:54:47.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:46.509454+0000 osd.6 (osd.6) 287 : cluster [DBG] 18.10 scrub starts 2024-02-23T00:54:47.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:46.510468+0000 osd.6 (osd.6) 288 : cluster [DBG] 18.10 scrub ok 2024-02-23T00:54:47.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:46.706115+0000 mgr.y (mgr.14517) 634 : audit [DBG] from='client.42044 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:54:47.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:46.972140+0000 mon.c (mon.2) 450 : audit [INF] from='client.? 172.21.15.47:0/860559830' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:54:47.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:46.972477+0000 mon.a (mon.0) 3582 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:54:47.940 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":1125,"stamp":"2024-02-23T00:54:45.410101+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4014082,"num_objects":243,"num_object_clones":1,"num_object_copies":729,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":0,"num_read":2456,"num_read_kb":2165,"num_write":951,"num_write_kb":3995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":8426,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1135,"ondisk_log_size":1135,"up":552,"acting":552,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":553,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":562470912,"kb_used":1844200,"kb_used_data":23992,"kb_used_omap":0,"kb_used_meta":1820160,"kb_avail":560626712,"statfs":{"total":575970213888,"available":574081753088,"internally_reserved":0,"allocated":24567808,"data_stored":14385195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1863843840},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8192,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.111479"},"pg_stats":[{"pgid":"249.1f","version":"0'0","reported_seq":14,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.306254+0000","last_change":"2024-02-23T00:54:36.386385+0000","last_active":"2024-02-23T00:54:40.306254+0000","last_peered":"2024-02-23T00:54:40.306254+0000","last_clean":"2024-02-23T00:54:40.306254+0000","last_became_active":"2024-02-23T00:54:36.386049+0000","last_became_peered":"2024-02-23T00:54:36.386049+0000","last_unstale":"2024-02-23T00:54:40.306254+0000","last_undegraded":"2024-02-23T00:54:40.306254+0000","last_fullsized":"2024-02-23T00:54:40.306254+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"249.1e","version":"0'0","reported_seq":13,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.783807+0000","last_change":"2024-02-23T00:54:36.396193+0000","last_active":"2024-02-23T00:54:40.783807+0000","last_peered":"2024-02-23T00:54:40.783807+0000","last_clean":"2024-02-23T00:54:40.783807+0000","last_became_active":"2024-02-23T00:54:36.395865+0000","last_became_peered":"2024-02-23T00:54:36.395865+0000","last_unstale":"2024-02-23T00:54:40.783807+0000","last_undegraded":"2024-02-23T00:54:40.783807+0000","last_fullsized":"2024-02-23T00:54:40.783807+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"249.1d","version":"0'0","reported_seq":13,"reported_epoch":739,"state":"active+clean","last_fresh":"2024-02-23T00:54:39.390348+0000","last_change":"2024-02-23T00:54:36.390358+0000","last_active":"2024-02-23T00:54:39.390348+0000","last_peered":"2024-02-23T00:54:39.390348+0000","last_clean":"2024-02-23T00:54:39.390348+0000","last_became_active":"2024-02-23T00:54:36.390032+0000","last_became_peered":"2024-02-23T00:54:36.390032+0000","last_unstale":"2024-02-23T00:54:39.390348+0000","last_undegraded":"2024-02-23T00:54:39.390348+0000","last_fullsized":"2024-02-23T00:54:39.390348+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"249.1c","version":"0'0","reported_seq":13,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.783642+0000","last_change":"2024-02-23T00:54:36.385506+0000","last_active":"2024-02-23T00:54:40.783642+0000","last_peered":"2024-02-23T00:54:40.783642+0000","last_clean":"2024-02-23T00:54:40.783642+0000","last_became_active":"2024-02-23T00:54:36.385132+0000","last_became_peered":"2024-02-23T00:54:36.385132+0000","last_unstale":"2024-02-23T00:54:40.783642+0000","last_undegraded":"2024-02-23T00:54:40.783642+0000","last_fullsized":"2024-02-23T00:54:40.783642+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"249.1b","version":"0'0","reported_seq":13,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.302767+0000","last_change":"2024-02-23T00:54:36.393642+0000","last_active":"2024-02-23T00:54:40.302767+0000","last_peered":"2024-02-23T00:54:40.302767+0000","last_clean":"2024-02-23T00:54:40.302767+0000","last_became_active":"2024-02-23T00:54:36.393213+0000","last_became_peered":"2024-02-23T00:54:36.393213+0000","last_unstale":"2024-02-23T00:54:40.302767+0000","last_undegraded":"2024-02-23T00:54:40.302767+0000","last_fullsized":"2024-02-23T00:54:40.302767+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"249.1a","version":"0'0","reported_seq":13,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.780980+0000","last_change":"2024-02-23T00:54:36.398155+0000","last_active":"2024-02-23T00:54:40.780980+0000","last_peered":"2024-02-23T00:54:40.780980+0000","last_clean":"2024-02-23T00:54:40.780980+0000","last_became_active":"2024-02-23T00:54:36.397542+0000","last_became_peered":"2024-02-23T00:54:36.397542+0000","last_unstale":"2024-02-23T00:54:40.780980+0000","last_undegraded":"2024-02-23T00:54:40.780980+0000","last_fullsized":"2024-02-23T00:54:40.780980+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"249.19","version":"0'0","reported_seq":13,"reported_epoch":739,"state":"active+clean","last_fresh":"2024-02-23T00:54:39.389553+0000","last_change":"2024-02-23T00:54:36.394690+0000","last_active":"2024-02-23T00:54:39.389553+0000","last_peered":"2024-02-23T00:54:39.389553+0000","last_clean":"2024-02-23T00:54:39.389553+0000","last_became_active":"2024-02-23T00:54:36.394382+0000","last_became_peered":"2024-02-23T00:54:36.394382+0000","last_unstale":"2024-02-23T00:54:39.389553+0000","last_undegraded":"2024-02-23T00:54:39.389553+0000","last_fullsized":"2024-02-23T00:54:39.389553+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"249.18","version":"0'0","reported_seq":13,"reported_epoch":739,"state":"active+clean","last_fresh":"2024-02-23T00:54:39.389480+0000","last_change":"2024-02-23T00:54:36.394629+0000","last_active":"2024-02-23T00:54:39.389480+0000","last_peered":"2024-02-23T00:54:39.389480+0000","last_clean":"2024-02-23T00:54:39.389480+0000","last_became_active":"2024-02-23T00:54:36.394305+0000","last_became_peered":"2024-02-23T00:54:36.394305+0000","last_unstale":"2024-02-23T00:54:39.389480+0000","last_undegraded":"2024-02-23T00:54:39.389480+0000","last_fullsized":"2024-02-23T00:54:39.389480+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,5],"acting":[4,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"249.f","version":"0'0","reported_seq":13,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.304134+0000","last_change":"2024-02-23T00:54:36.400186+0000","last_active":"2024-02-23T00:54:40.304134+0000","last_peered":"2024-02-23T00:54:40.304134+0000","last_clean":"2024-02-23T00:54:40.304134+0000","last_became_active":"2024-02-23T00:54:36.399866+0000","last_became_peered":"2024-02-23T00:54:36.399866+0000","last_unstale":"2024-02-23T00:54:40.304134+0000","last_undegraded":"2024-02-23T00:54:40.304134+0000","last_fullsized":"2024-02-23T00:54:40.304134+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"249.e","version":"0'0","reported_seq":13,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.303091+0000","last_change":"2024-02-23T00:54:36.402347+0000","last_active":"2024-02-23T00:54:40.303091+0000","last_peered":"2024-02-23T00:54:40.303091+0000","last_clean":"2024-02-23T00:54:40.303091+0000","last_became_active":"2024-02-23T00:54:36.401994+0000","last_became_peered":"2024-02-23T00:54:36.401994+0000","last_unstale":"2024-02-23T00:54:40.303091+0000","last_undegraded":"2024-02-23T00:54:40.303091+0000","last_fullsized":"2024-02-23T00:54:40.303091+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"249.d","version":"0'0","reported_seq":13,"reported_epoch":739,"state":"active+clean","last_fresh":"2024-02-23T00:54:39.390664+0000","last_change":"2024-02-23T00:54:36.395715+0000","last_active":"2024-02-23T00:54:39.390664+0000","last_peered":"2024-02-23T00:54:39.390664+0000","last_clean":"2024-02-23T00:54:39.390664+0000","last_became_active":"2024-02-23T00:54:36.395415+0000","last_became_peered":"2024-02-23T00:54:36.395415+0000","last_unstale":"2024-02-23T00:54:39.390664+0000","last_undegraded":"2024-02-23T00:54:39.390664+0000","last_fullsized":"2024-02-23T00:54:39.390664+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"249.c","version":"0'0","reported_seq":13,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.303050+0000","last_change":"2024-02-23T00:54:36.402659+0000","last_active":"2024-02-23T00:54:40.303050+0000","last_peered":"2024-02-23T00:54:40.303050+0000","last_clean":"2024-02-23T00:54:40.303050+0000","last_became_active":"2024-02-23T00:54:36.402162+0000","last_became_peered":"2024-02-23T00:54:36.402162+0000","last_unstale":"2024-02-23T00:54:40.303050+0000","last_undegraded":"2024-02-23T00:54:40.303050+0000","last_fullsized":"2024-02-23T00:54:40.303050+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"249.b","version":"0'0","reported_seq":13,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.781894+0000","last_change":"2024-02-23T00:54:36.392526+0000","last_active":"2024-02-23T00:54:40.781894+0000","last_peered":"2024-02-23T00:54:40.781894+0000","last_clean":"2024-02-23T00:54:40.781894+0000","last_became_active":"2024-02-23T00:54:36.392141+0000","last_became_peered":"2024-02-23T00:54:36.392141+0000","last_unstale":"2024-02-23T00:54:40.781894+0000","last_undegraded":"2024-02-23T00:54:40.781894+0000","last_fullsized":"2024-02-23T00:54:40.781894+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"249.a","version":"0'0","reported_seq":13,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.781247+0000","last_change":"2024-02-23T00:54:36.398004+0000","last_active":"2024-02-23T00:54:40.781247+0000","last_peered":"2024-02-23T00:54:40.781247+0000","last_clean":"2024-02-23T00:54:40.781247+0000","last_became_active":"2024-02-23T00:54:36.397176+0000","last_became_peered":"2024-02-23T00:54:36.397176+0000","last_unstale":"2024-02-23T00:54:40.781247+0000","last_undegraded":"2024-02-23T00:54:40.781247+0000","last_fullsized":"2024-02-23T00:54:40.781247+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"249.9","version":"0'0","reported_seq":13,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.303484+0000","last_change":"2024-02-23T00:54:36.401271+0000","last_active":"2024-02-23T00:54:40.303484+0000","last_peered":"2024-02-23T00:54:40.303484+0000","last_clean":"2024-02-23T00:54:40.303484+0000","last_became_active":"2024-02-23T00:54:36.400942+0000","last_became_peered":"2024-02-23T00:54:36.400942+0000","last_unstale":"2024-02-23T00:54:40.303484+0000","last_undegraded":"2024-02-23T00:54:40.303484+0000","last_fullsized":"2024-02-23T00:54:40.303484+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"249.8","version":"0'0","reported_seq":14,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.303863+0000","last_change":"2024-02-23T00:54:36.387804+0000","last_active":"2024-02-23T00:54:40.303863+0000","last_peered":"2024-02-23T00:54:40.303863+0000","last_clean":"2024-02-23T00:54:40.303863+0000","last_became_active":"2024-02-23T00:54:36.387475+0000","last_became_peered":"2024-02-23T00:54:36.387475+0000","last_unstale":"2024-02-23T00:54:40.303863+0000","last_undegraded":"2024-02-23T00:54:40.303863+0000","last_fullsized":"2024-02-23T00:54:40.303863+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"249.7","version":"0'0","reported_seq":13,"reported_epoch":739,"state":"active+clean","last_fresh":"2024-02-23T00:54:39.391176+0000","last_change":"2024-02-23T00:54:36.394608+0000","last_active":"2024-02-23T00:54:39.391176+0000","last_peered":"2024-02-23T00:54:39.391176+0000","last_clean":"2024-02-23T00:54:39.391176+0000","last_became_active":"2024-02-23T00:54:36.393720+0000","last_became_peered":"2024-02-23T00:54:36.393720+0000","last_unstale":"2024-02-23T00:54:39.391176+0000","last_undegraded":"2024-02-23T00:54:39.391176+0000","last_fullsized":"2024-02-23T00:54:39.391176+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"249.6","version":"741'1","reported_seq":14,"reported_epoch":741,"state":"active+clean","last_fresh":"2024-02-23T00:54:41.307587+0000","last_change":"2024-02-23T00:54:36.384583+0000","last_active":"2024-02-23T00:54:41.307587+0000","last_peered":"2024-02-23T00:54:41.307587+0000","last_clean":"2024-02-23T00:54:41.307587+0000","last_became_active":"2024-02-23T00:54:36.384289+0000","last_became_peered":"2024-02-23T00:54:36.384289+0000","last_unstale":"2024-02-23T00:54:41.307587+0000","last_undegraded":"2024-02-23T00:54:41.307587+0000","last_fullsized":"2024-02-23T00:54:41.307587+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8192,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"249.5","version":"0'0","reported_seq":13,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.304061+0000","last_change":"2024-02-23T00:54:36.400343+0000","last_active":"2024-02-23T00:54:40.304061+0000","last_peered":"2024-02-23T00:54:40.304061+0000","last_clean":"2024-02-23T00:54:40.304061+0000","last_became_active":"2024-02-23T00:54:36.400029+0000","last_became_peered":"2024-02-23T00:54:36.400029+0000","last_unstale":"2024-02-23T00:54:40.304061+0000","last_undegraded":"2024-02-23T00:54:40.304061+0000","last_fullsized":"2024-02-23T00:54:40.304061+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"249.4","version":"0'0","reported_seq":13,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.303521+0000","last_change":"2024-02-23T00:54:36.397013+0000","last_active":"2024-02-23T00:54:40.303521+0000","last_peered":"2024-02-23T00:54:40.303521+0000","last_clean":"2024-02-23T00:54:40.303521+0000","last_became_active":"2024-02-23T00:54:36.396654+0000","last_became_peered":"2024-02-23T00:54:36.396654+0000","last_unstale":"2024-02-23T00:54:40.303521+0000","last_undegraded":"2024-02-23T00:54:40.303521+0000","last_fullsized":"2024-02-23T00:54:40.303521+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"249.3","version":"0'0","reported_seq":13,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.781365+0000","last_change":"2024-02-23T00:54:36.399205+0000","last_active":"2024-02-23T00:54:40.781365+0000","last_peered":"2024-02-23T00:54:40.781365+0000","last_clean":"2024-02-23T00:54:40.781365+0000","last_became_active":"2024-02-23T00:54:36.398901+0000","last_became_peered":"2024-02-23T00:54:36.398901+0000","last_unstale":"2024-02-23T00:54:40.781365+0000","last_undegraded":"2024-02-23T00:54:40.781365+0000","last_fullsized":"2024-02-23T00:54:40.781365+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"249.2","version":"0'0","reported_seq":13,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.781906+0000","last_change":"2024-02-23T00:54:36.397799+0000","last_active":"2024-02-23T00:54:40.781906+0000","last_peered":"2024-02-23T00:54:40.781906+0000","last_clean":"2024-02-23T00:54:40.781906+0000","last_became_active":"2024-02-23T00:54:36.396828+0000","last_became_peered":"2024-02-23T00:54:36.396828+0000","last_unstale":"2024-02-23T00:54:40.781906+0000","last_undegraded":"2024-02-23T00:54:40.781906+0000","last_fullsized":"2024-02-23T00:54:40.781906+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"249.1","version":"0'0","reported_seq":13,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.783242+0000","last_change":"2024-02-23T00:54:36.395999+0000","last_active":"2024-02-23T00:54:40.783242+0000","last_peered":"2024-02-23T00:54:40.783242+0000","last_clean":"2024-02-23T00:54:40.783242+0000","last_became_active":"2024-02-23T00:54:36.395660+0000","last_became_peered":"2024-02-23T00:54:36.395660+0000","last_unstale":"2024-02-23T00:54:40.783242+0000","last_undegraded":"2024-02-23T00:54:40.783242+0000","last_fullsized":"2024-02-23T00:54:40.783242+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"249.0","version":"0'0","reported_seq":13,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.303263+0000","last_change":"2024-02-23T00:54:36.402398+0000","last_active":"2024-02-23T00:54:40.303263+0000","last_peered":"2024-02-23T00:54:40.303263+0000","last_clean":"2024-02-23T00:54:40.303263+0000","last_became_active":"2024-02-23T00:54:36.401890+0000","last_became_peered":"2024-02-23T00:54:36.401890+0000","last_unstale":"2024-02-23T00:54:40.303263+0000","last_undegraded":"2024-02-23T00:54:40.303263+0000","last_fullsized":"2024-02-23T00:54:40.303263+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"249.14","version":"0'0","reported_seq":13,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.303095+0000","last_change":"2024-02-23T00:54:36.402474+0000","last_active":"2024-02-23T00:54:40.303095+0000","last_peered":"2024-02-23T00:54:40.303095+0000","last_clean":"2024-02-23T00:54:40.303095+0000","last_became_active":"2024-02-23T00:54:36.401791+0000","last_became_peered":"2024-02-23T00:54:36.401791+0000","last_unstale":"2024-02-23T00:54:40.303095+0000","last_undegraded":"2024-02-23T00:54:40.303095+0000","last_fullsized":"2024-02-23T00:54:40.303095+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"234.7","version":"666'13","reported_seq":190,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.303940+0000","last_change":"2024-02-23T00:54:10.268034+0000","last_active":"2024-02-23T00:54:40.303940+0000","last_peered":"2024-02-23T00:54:40.303940+0000","last_clean":"2024-02-23T00:54:40.303940+0000","last_became_active":"2024-02-23T00:52:10.280357+0000","last_became_peered":"2024-02-23T00:52:10.280357+0000","last_unstale":"2024-02-23T00:54:40.303940+0000","last_undegraded":"2024-02-23T00:54:40.303940+0000","last_fullsized":"2024-02-23T00:54:40.303940+0000","mapping_epoch":600,"log_start":"0'0","ondisk_log_start":"0'0","created":600,"last_epoch_clean":601,"parent":"0.0","parent_split_bits":0,"last_scrub":"666'13","last_scrub_stamp":"2024-02-23T00:54:10.267942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_clean_scrub_stamp":"2024-02-23T00:54:10.267942+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":18,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":7,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"249.15","version":"0'0","reported_seq":13,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.781923+0000","last_change":"2024-02-23T00:54:36.394683+0000","last_active":"2024-02-23T00:54:40.781923+0000","last_peered":"2024-02-23T00:54:40.781923+0000","last_clean":"2024-02-23T00:54:40.781923+0000","last_became_active":"2024-02-23T00:54:36.394396+0000","last_became_peered":"2024-02-23T00:54:36.394396+0000","last_unstale":"2024-02-23T00:54:40.781923+0000","last_undegraded":"2024-02-23T00:54:40.781923+0000","last_fullsized":"2024-02-23T00:54:40.781923+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5],"acting":[6,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"234.6","version":"629'6","reported_seq":195,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.782294+0000","last_change":"2024-02-23T00:54:06.318420+0000","last_active":"2024-02-23T00:54:40.782294+0000","last_peered":"2024-02-23T00:54:40.782294+0000","last_clean":"2024-02-23T00:54:40.782294+0000","last_became_active":"2024-02-23T00:53:13.348600+0000","last_became_peered":"2024-02-23T00:53:13.348600+0000","last_unstale":"2024-02-23T00:54:40.782294+0000","last_undegraded":"2024-02-23T00:54:40.782294+0000","last_fullsized":"2024-02-23T00:54:40.782294+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":600,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"629'6","last_scrub_stamp":"2024-02-23T00:53:36.494408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_clean_scrub_stamp":"2024-02-23T00:53:36.494408+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":8,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"249.16","version":"0'0","reported_seq":13,"reported_epoch":739,"state":"active+clean","last_fresh":"2024-02-23T00:54:39.390621+0000","last_change":"2024-02-23T00:54:36.394966+0000","last_active":"2024-02-23T00:54:39.390621+0000","last_peered":"2024-02-23T00:54:39.390621+0000","last_clean":"2024-02-23T00:54:39.390621+0000","last_became_active":"2024-02-23T00:54:36.394664+0000","last_became_peered":"2024-02-23T00:54:36.394664+0000","last_unstale":"2024-02-23T00:54:39.390621+0000","last_undegraded":"2024-02-23T00:54:39.390621+0000","last_fullsized":"2024-02-23T00:54:39.390621+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"234.5","version":"715'11","reported_seq":193,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.782138+0000","last_change":"2024-02-23T00:54:06.319329+0000","last_active":"2024-02-23T00:54:40.782138+0000","last_peered":"2024-02-23T00:54:40.782138+0000","last_clean":"2024-02-23T00:54:40.782138+0000","last_became_active":"2024-02-23T00:53:13.350717+0000","last_became_peered":"2024-02-23T00:53:13.350717+0000","last_unstale":"2024-02-23T00:54:40.782138+0000","last_undegraded":"2024-02-23T00:54:40.782138+0000","last_fullsized":"2024-02-23T00:54:40.782138+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":600,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"666'6","last_scrub_stamp":"2024-02-23T00:53:32.451126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_clean_scrub_stamp":"2024-02-23T00:53:32.451126+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"249.17","version":"0'0","reported_seq":13,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.784582+0000","last_change":"2024-02-23T00:54:36.391603+0000","last_active":"2024-02-23T00:54:40.784582+0000","last_peered":"2024-02-23T00:54:40.784582+0000","last_clean":"2024-02-23T00:54:40.784582+0000","last_became_active":"2024-02-23T00:54:36.391155+0000","last_became_peered":"2024-02-23T00:54:36.391155+0000","last_unstale":"2024-02-23T00:54:40.784582+0000","last_undegraded":"2024-02-23T00:54:40.784582+0000","last_fullsized":"2024-02-23T00:54:40.784582+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"234.4","version":"707'13","reported_seq":132,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.304997+0000","last_change":"2024-02-23T00:54:08.325933+0000","last_active":"2024-02-23T00:54:40.304997+0000","last_peered":"2024-02-23T00:54:40.304997+0000","last_clean":"2024-02-23T00:54:40.304997+0000","last_became_active":"2024-02-23T00:53:34.320586+0000","last_became_peered":"2024-02-23T00:53:34.320586+0000","last_unstale":"2024-02-23T00:54:40.304997+0000","last_undegraded":"2024-02-23T00:54:40.304997+0000","last_fullsized":"2024-02-23T00:54:40.304997+0000","mapping_epoch":676,"log_start":"0'0","ondisk_log_start":"0'0","created":600,"last_epoch_clean":677,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_clean_scrub_stamp":"2024-02-23T00:52:09.256902+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"249.10","version":"0'0","reported_seq":13,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.302439+0000","last_change":"2024-02-23T00:54:36.401993+0000","last_active":"2024-02-23T00:54:40.302439+0000","last_peered":"2024-02-23T00:54:40.302439+0000","last_clean":"2024-02-23T00:54:40.302439+0000","last_became_active":"2024-02-23T00:54:36.401399+0000","last_became_peered":"2024-02-23T00:54:36.401399+0000","last_unstale":"2024-02-23T00:54:40.302439+0000","last_undegraded":"2024-02-23T00:54:40.302439+0000","last_fullsized":"2024-02-23T00:54:40.302439+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"234.3","version":"667'5","reported_seq":209,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.783751+0000","last_change":"2024-02-23T00:54:07.348103+0000","last_active":"2024-02-23T00:54:40.783751+0000","last_peered":"2024-02-23T00:54:40.783751+0000","last_clean":"2024-02-23T00:54:40.783751+0000","last_became_active":"2024-02-23T00:53:53.411778+0000","last_became_peered":"2024-02-23T00:53:53.411778+0000","last_unstale":"2024-02-23T00:54:40.783751+0000","last_undegraded":"2024-02-23T00:54:40.783751+0000","last_fullsized":"2024-02-23T00:54:40.783751+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":600,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"667'5","last_scrub_stamp":"2024-02-23T00:54:07.348022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_clean_scrub_stamp":"2024-02-23T00:54:07.348022+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":18,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"249.11","version":"0'0","reported_seq":13,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.780708+0000","last_change":"2024-02-23T00:54:36.396058+0000","last_active":"2024-02-23T00:54:40.780708+0000","last_peered":"2024-02-23T00:54:40.780708+0000","last_clean":"2024-02-23T00:54:40.780708+0000","last_became_active":"2024-02-23T00:54:36.395126+0000","last_became_peered":"2024-02-23T00:54:36.395126+0000","last_unstale":"2024-02-23T00:54:40.780708+0000","last_undegraded":"2024-02-23T00:54:40.780708+0000","last_fullsized":"2024-02-23T00:54:40.780708+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"234.2","version":"629'10","reported_seq":201,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.781380+0000","last_change":"2024-02-23T00:54:06.318884+0000","last_active":"2024-02-23T00:54:40.781380+0000","last_peered":"2024-02-23T00:54:40.781380+0000","last_clean":"2024-02-23T00:54:40.781380+0000","last_became_active":"2024-02-23T00:53:13.351731+0000","last_became_peered":"2024-02-23T00:53:13.351731+0000","last_unstale":"2024-02-23T00:54:40.781380+0000","last_undegraded":"2024-02-23T00:54:40.781380+0000","last_fullsized":"2024-02-23T00:54:40.781380+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":600,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"629'10","last_scrub_stamp":"2024-02-23T00:53:34.474122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_clean_scrub_stamp":"2024-02-23T00:53:34.474122+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":10,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":10,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,7],"acting":[6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"249.12","version":"0'0","reported_seq":13,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.302689+0000","last_change":"2024-02-23T00:54:36.402194+0000","last_active":"2024-02-23T00:54:40.302689+0000","last_peered":"2024-02-23T00:54:40.302689+0000","last_clean":"2024-02-23T00:54:40.302689+0000","last_became_active":"2024-02-23T00:54:36.401838+0000","last_became_peered":"2024-02-23T00:54:36.401838+0000","last_unstale":"2024-02-23T00:54:40.302689+0000","last_undegraded":"2024-02-23T00:54:40.302689+0000","last_fullsized":"2024-02-23T00:54:40.302689+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"234.1","version":"723'5","reported_seq":196,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.302452+0000","last_change":"2024-02-23T00:54:06.318698+0000","last_active":"2024-02-23T00:54:40.302452+0000","last_peered":"2024-02-23T00:54:40.302452+0000","last_clean":"2024-02-23T00:54:40.302452+0000","last_became_active":"2024-02-23T00:53:34.318140+0000","last_became_peered":"2024-02-23T00:53:34.318140+0000","last_unstale":"2024-02-23T00:54:40.302452+0000","last_undegraded":"2024-02-23T00:54:40.302452+0000","last_fullsized":"2024-02-23T00:54:40.302452+0000","mapping_epoch":676,"log_start":"0'0","ondisk_log_start":"0'0","created":600,"last_epoch_clean":677,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_clean_scrub_stamp":"2024-02-23T00:52:09.256902+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4000024,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4,"num_write_kb":3910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":8,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"249.13","version":"0'0","reported_seq":13,"reported_epoch":739,"state":"active+clean","last_fresh":"2024-02-23T00:54:39.390113+0000","last_change":"2024-02-23T00:54:36.395966+0000","last_active":"2024-02-23T00:54:39.390113+0000","last_peered":"2024-02-23T00:54:39.390113+0000","last_clean":"2024-02-23T00:54:39.390113+0000","last_became_active":"2024-02-23T00:54:36.394040+0000","last_became_peered":"2024-02-23T00:54:36.394040+0000","last_unstale":"2024-02-23T00:54:39.390113+0000","last_undegraded":"2024-02-23T00:54:39.390113+0000","last_fullsized":"2024-02-23T00:54:39.390113+0000","mapping_epoch":735,"log_start":"0'0","ondisk_log_start":"0'0","created":735,"last_epoch_clean":736,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:35.350825+0000","last_clean_scrub_stamp":"2024-02-23T00:54:35.350825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,5],"acting":[4,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"234.0","version":"694'7","reported_seq":182,"reported_epoch":739,"state":"active+clean","last_fresh":"2024-02-23T00:54:39.389686+0000","last_change":"2024-02-23T00:54:15.372319+0000","last_active":"2024-02-23T00:54:39.389686+0000","last_peered":"2024-02-23T00:54:39.389686+0000","last_clean":"2024-02-23T00:54:39.389686+0000","last_became_active":"2024-02-23T00:52:10.279573+0000","last_became_peered":"2024-02-23T00:52:10.279573+0000","last_unstale":"2024-02-23T00:54:39.389686+0000","last_undegraded":"2024-02-23T00:54:39.389686+0000","last_fullsized":"2024-02-23T00:54:39.389686+0000","mapping_epoch":600,"log_start":"0'0","ondisk_log_start":"0'0","created":600,"last_epoch_clean":601,"parent":"0.0","parent_split_bits":0,"last_scrub":"694'7","last_scrub_stamp":"2024-02-23T00:54:15.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_clean_scrub_stamp":"2024-02-23T00:54:15.372237+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"466'9","reported_seq":908,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.303291+0000","last_change":"2024-02-23T00:54:11.260611+0000","last_active":"2024-02-23T00:54:40.303291+0000","last_peered":"2024-02-23T00:54:40.303291+0000","last_clean":"2024-02-23T00:54:40.303291+0000","last_became_active":"2024-02-23T00:39:18.793414+0000","last_became_peered":"2024-02-23T00:39:18.793414+0000","last_unstale":"2024-02-23T00:54:40.303291+0000","last_undegraded":"2024-02-23T00:54:40.303291+0000","last_fullsized":"2024-02-23T00:54:40.303291+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'9","last_scrub_stamp":"2024-02-23T00:54:11.260489+0000","last_deep_scrub":"466'9","last_deep_scrub_stamp":"2024-02-23T00:49:43.187989+0000","last_clean_scrub_stamp":"2024-02-23T00:54:11.260489+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"470'42","reported_seq":786,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.783074+0000","last_change":"2024-02-23T00:52:17.393167+0000","last_active":"2024-02-23T00:54:40.783074+0000","last_peered":"2024-02-23T00:54:40.783074+0000","last_clean":"2024-02-23T00:54:40.783074+0000","last_became_active":"2024-02-23T00:39:16.859401+0000","last_became_peered":"2024-02-23T00:39:16.859401+0000","last_unstale":"2024-02-23T00:54:40.783074+0000","last_undegraded":"2024-02-23T00:54:40.783074+0000","last_fullsized":"2024-02-23T00:54:40.783074+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"470'42","last_scrub_stamp":"2024-02-23T00:52:17.393084+0000","last_deep_scrub":"470'42","last_deep_scrub_stamp":"2024-02-23T00:49:47.653176+0000","last_clean_scrub_stamp":"2024-02-23T00:52:17.393084+0000","log_size":42,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":101,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.303541+0000","last_change":"2024-02-23T00:53:53.424388+0000","last_active":"2024-02-23T00:54:40.303541+0000","last_peered":"2024-02-23T00:54:40.303541+0000","last_clean":"2024-02-23T00:54:40.303541+0000","last_became_active":"2024-02-23T00:53:53.424032+0000","last_became_peered":"2024-02-23T00:53:53.424032+0000","last_unstale":"2024-02-23T00:54:40.303541+0000","last_undegraded":"2024-02-23T00:54:40.303541+0000","last_fullsized":"2024-02-23T00:54:40.303541+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:40.350368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:04.330426+0000","last_clean_scrub_stamp":"2024-02-23T00:53:40.350368+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":733,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.783227+0000","last_change":"2024-02-23T00:53:13.349648+0000","last_active":"2024-02-23T00:54:40.783227+0000","last_peered":"2024-02-23T00:54:40.783227+0000","last_clean":"2024-02-23T00:54:40.783227+0000","last_became_active":"2024-02-23T00:53:13.349304+0000","last_became_peered":"2024-02-23T00:53:13.349304+0000","last_unstale":"2024-02-23T00:54:40.783227+0000","last_undegraded":"2024-02-23T00:54:40.783227+0000","last_fullsized":"2024-02-23T00:54:40.783227+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:25.491458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:52:25.491458+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":662,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.783929+0000","last_change":"2024-02-23T00:53:13.350523+0000","last_active":"2024-02-23T00:54:40.783929+0000","last_peered":"2024-02-23T00:54:40.783929+0000","last_clean":"2024-02-23T00:54:40.783929+0000","last_became_active":"2024-02-23T00:53:13.350150+0000","last_became_peered":"2024-02-23T00:53:13.350150+0000","last_unstale":"2024-02-23T00:54:40.783929+0000","last_undegraded":"2024-02-23T00:54:40.783929+0000","last_fullsized":"2024-02-23T00:54:40.783929+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:51.439818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:52:51.439818+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"457'21","reported_seq":610,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.305024+0000","last_change":"2024-02-23T00:53:53.849527+0000","last_active":"2024-02-23T00:54:40.305024+0000","last_peered":"2024-02-23T00:54:40.305024+0000","last_clean":"2024-02-23T00:54:40.305024+0000","last_became_active":"2024-02-23T00:53:53.410969+0000","last_became_peered":"2024-02-23T00:53:53.410969+0000","last_unstale":"2024-02-23T00:54:40.305024+0000","last_undegraded":"2024-02-23T00:54:40.305024+0000","last_fullsized":"2024-02-23T00:54:40.305024+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-23T00:51:46.661879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:46.661879+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":37,"num_read_kb":30,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":702,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.305635+0000","last_change":"2024-02-23T00:52:25.554460+0000","last_active":"2024-02-23T00:54:40.305635+0000","last_peered":"2024-02-23T00:54:40.305635+0000","last_clean":"2024-02-23T00:54:40.305635+0000","last_became_active":"2024-02-23T00:39:14.772493+0000","last_became_peered":"2024-02-23T00:39:14.772493+0000","last_unstale":"2024-02-23T00:54:40.305635+0000","last_undegraded":"2024-02-23T00:54:40.305635+0000","last_fullsized":"2024-02-23T00:54:40.305635+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:25.554392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:04.528006+0000","last_clean_scrub_stamp":"2024-02-23T00:52:25.554392+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":413,"reported_epoch":739,"state":"active+clean","last_fresh":"2024-02-23T00:54:39.390291+0000","last_change":"2024-02-23T00:53:53.411602+0000","last_active":"2024-02-23T00:54:39.390291+0000","last_peered":"2024-02-23T00:54:39.390291+0000","last_clean":"2024-02-23T00:54:39.390291+0000","last_became_active":"2024-02-23T00:53:53.411237+0000","last_became_peered":"2024-02-23T00:53:53.411237+0000","last_unstale":"2024-02-23T00:54:39.390291+0000","last_undegraded":"2024-02-23T00:54:39.390291+0000","last_fullsized":"2024-02-23T00:54:39.390291+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:30.466466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:08.335330+0000","last_clean_scrub_stamp":"2024-02-23T00:53:30.466466+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":446,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.302562+0000","last_change":"2024-02-23T00:53:13.348694+0000","last_active":"2024-02-23T00:54:40.302562+0000","last_peered":"2024-02-23T00:54:40.302562+0000","last_clean":"2024-02-23T00:54:40.302562+0000","last_became_active":"2024-02-23T00:53:13.348208+0000","last_became_peered":"2024-02-23T00:53:13.348208+0000","last_unstale":"2024-02-23T00:54:40.302562+0000","last_undegraded":"2024-02-23T00:54:40.302562+0000","last_fullsized":"2024-02-23T00:54:40.302562+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:02.821535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:13.867716+0000","last_clean_scrub_stamp":"2024-02-23T00:53:02.821535+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":97,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.303523+0000","last_change":"2024-02-23T00:53:53.424522+0000","last_active":"2024-02-23T00:54:40.303523+0000","last_peered":"2024-02-23T00:54:40.303523+0000","last_clean":"2024-02-23T00:54:40.303523+0000","last_became_active":"2024-02-23T00:53:53.424204+0000","last_became_peered":"2024-02-23T00:53:53.424204+0000","last_unstale":"2024-02-23T00:54:40.303523+0000","last_undegraded":"2024-02-23T00:54:40.303523+0000","last_fullsized":"2024-02-23T00:54:40.303523+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:56.845554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:52:56.845554+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"457'51","reported_seq":793,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.781866+0000","last_change":"2024-02-23T00:52:50.534889+0000","last_active":"2024-02-23T00:54:40.781866+0000","last_peered":"2024-02-23T00:54:40.781866+0000","last_clean":"2024-02-23T00:54:40.781866+0000","last_became_active":"2024-02-23T00:39:16.857413+0000","last_became_peered":"2024-02-23T00:39:16.857413+0000","last_unstale":"2024-02-23T00:54:40.781866+0000","last_undegraded":"2024-02-23T00:54:40.781866+0000","last_fullsized":"2024-02-23T00:54:40.781866+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'51","last_scrub_stamp":"2024-02-23T00:52:50.534772+0000","last_deep_scrub":"88'35","last_deep_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_clean_scrub_stamp":"2024-02-23T00:52:50.534772+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":68,"num_read_kb":56,"num_write":42,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":394,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.783985+0000","last_change":"2024-02-23T00:53:28.314751+0000","last_active":"2024-02-23T00:54:40.783985+0000","last_peered":"2024-02-23T00:54:40.783985+0000","last_clean":"2024-02-23T00:54:40.783985+0000","last_became_active":"2024-02-23T00:53:13.345499+0000","last_became_peered":"2024-02-23T00:53:13.345499+0000","last_unstale":"2024-02-23T00:54:40.783985+0000","last_undegraded":"2024-02-23T00:54:40.783985+0000","last_fullsized":"2024-02-23T00:54:40.783985+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:28.314660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_clean_scrub_stamp":"2024-02-23T00:53:28.314660+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":103,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.784040+0000","last_change":"2024-02-23T00:53:41.132759+0000","last_active":"2024-02-23T00:54:40.784040+0000","last_peered":"2024-02-23T00:54:40.784040+0000","last_clean":"2024-02-23T00:54:40.784040+0000","last_became_active":"2024-02-23T00:53:13.357189+0000","last_became_peered":"2024-02-23T00:53:13.357189+0000","last_unstale":"2024-02-23T00:54:40.784040+0000","last_undegraded":"2024-02-23T00:54:40.784040+0000","last_fullsized":"2024-02-23T00:54:40.784040+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:41.132688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:03.682745+0000","last_clean_scrub_stamp":"2024-02-23T00:53:41.132688+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":731,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.302463+0000","last_change":"2024-02-23T00:52:07.425432+0000","last_active":"2024-02-23T00:54:40.302463+0000","last_peered":"2024-02-23T00:54:40.302463+0000","last_clean":"2024-02-23T00:54:40.302463+0000","last_became_active":"2024-02-23T00:47:28.215636+0000","last_became_peered":"2024-02-23T00:47:28.215636+0000","last_unstale":"2024-02-23T00:54:40.302463+0000","last_undegraded":"2024-02-23T00:54:40.302463+0000","last_fullsized":"2024-02-23T00:54:40.302463+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:07.425311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:42.150875+0000","last_clean_scrub_stamp":"2024-02-23T00:52:07.425311+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":671,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.305227+0000","last_change":"2024-02-23T00:53:53.412741+0000","last_active":"2024-02-23T00:54:40.305227+0000","last_peered":"2024-02-23T00:54:40.305227+0000","last_clean":"2024-02-23T00:54:40.305227+0000","last_became_active":"2024-02-23T00:53:53.412355+0000","last_became_peered":"2024-02-23T00:53:53.412355+0000","last_unstale":"2024-02-23T00:54:40.305227+0000","last_undegraded":"2024-02-23T00:54:40.305227+0000","last_fullsized":"2024-02-23T00:54:40.305227+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:07.482987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:53:07.482987+0000","last_clean_scrub_stamp":"2024-02-23T00:53:07.482987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"457'31","reported_seq":814,"reported_epoch":739,"state":"active+clean","last_fresh":"2024-02-23T00:54:39.391040+0000","last_change":"2024-02-23T00:53:36.520058+0000","last_active":"2024-02-23T00:54:39.391040+0000","last_peered":"2024-02-23T00:54:39.391040+0000","last_clean":"2024-02-23T00:54:39.391040+0000","last_became_active":"2024-02-23T00:47:28.200653+0000","last_became_peered":"2024-02-23T00:47:28.200653+0000","last_unstale":"2024-02-23T00:54:39.391040+0000","last_undegraded":"2024-02-23T00:54:39.391040+0000","last_fullsized":"2024-02-23T00:54:39.391040+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:53:36.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:53:36.519986+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":383,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.302972+0000","last_change":"2024-02-23T00:52:55.382296+0000","last_active":"2024-02-23T00:54:40.302972+0000","last_peered":"2024-02-23T00:54:40.302972+0000","last_clean":"2024-02-23T00:54:40.302972+0000","last_became_active":"2024-02-23T00:47:28.201926+0000","last_became_peered":"2024-02-23T00:47:28.201926+0000","last_unstale":"2024-02-23T00:54:40.302972+0000","last_undegraded":"2024-02-23T00:54:40.302972+0000","last_fullsized":"2024-02-23T00:54:40.302972+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:55.382210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:55.382210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":712,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.305573+0000","last_change":"2024-02-23T00:54:07.276618+0000","last_active":"2024-02-23T00:54:40.305573+0000","last_peered":"2024-02-23T00:54:40.305573+0000","last_clean":"2024-02-23T00:54:40.305573+0000","last_became_active":"2024-02-23T00:53:13.350335+0000","last_became_peered":"2024-02-23T00:53:13.350335+0000","last_unstale":"2024-02-23T00:54:40.305573+0000","last_undegraded":"2024-02-23T00:54:40.305573+0000","last_fullsized":"2024-02-23T00:54:40.305573+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:07.276549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:09.343884+0000","last_clean_scrub_stamp":"2024-02-23T00:54:07.276549+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":644,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.304691+0000","last_change":"2024-02-23T00:52:16.478287+0000","last_active":"2024-02-23T00:54:40.304691+0000","last_peered":"2024-02-23T00:54:40.304691+0000","last_clean":"2024-02-23T00:54:40.304691+0000","last_became_active":"2024-02-23T00:42:57.675306+0000","last_became_peered":"2024-02-23T00:42:57.675306+0000","last_unstale":"2024-02-23T00:54:40.304691+0000","last_undegraded":"2024-02-23T00:54:40.304691+0000","last_fullsized":"2024-02-23T00:54:40.304691+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:16.478219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:52:16.478219+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"457'20","reported_seq":741,"reported_epoch":739,"state":"active+clean","last_fresh":"2024-02-23T00:54:39.390460+0000","last_change":"2024-02-23T00:52:53.807404+0000","last_active":"2024-02-23T00:54:39.390460+0000","last_peered":"2024-02-23T00:54:39.390460+0000","last_clean":"2024-02-23T00:54:39.390460+0000","last_became_active":"2024-02-23T00:39:16.777530+0000","last_became_peered":"2024-02-23T00:39:16.777530+0000","last_unstale":"2024-02-23T00:54:39.390460+0000","last_undegraded":"2024-02-23T00:54:39.390460+0000","last_fullsized":"2024-02-23T00:54:39.390460+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'20","last_scrub_stamp":"2024-02-23T00:52:53.807334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:52:53.807334+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":702,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.304431+0000","last_change":"2024-02-23T00:52:13.530732+0000","last_active":"2024-02-23T00:54:40.304431+0000","last_peered":"2024-02-23T00:54:40.304431+0000","last_clean":"2024-02-23T00:54:40.304431+0000","last_became_active":"2024-02-23T00:39:14.780379+0000","last_became_peered":"2024-02-23T00:39:14.780379+0000","last_unstale":"2024-02-23T00:54:40.304431+0000","last_undegraded":"2024-02-23T00:54:40.304431+0000","last_fullsized":"2024-02-23T00:54:40.304431+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:13.530630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.575242+0000","last_clean_scrub_stamp":"2024-02-23T00:52:13.530630+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"656'10","reported_seq":503,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.304063+0000","last_change":"2024-02-23T00:53:13.448730+0000","last_active":"2024-02-23T00:54:40.304063+0000","last_peered":"2024-02-23T00:54:40.304063+0000","last_clean":"2024-02-23T00:54:40.304063+0000","last_became_active":"2024-02-23T00:53:13.359465+0000","last_became_peered":"2024-02-23T00:53:13.359465+0000","last_unstale":"2024-02-23T00:54:40.304063+0000","last_undegraded":"2024-02-23T00:54:40.304063+0000","last_fullsized":"2024-02-23T00:54:40.304063+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"376'9","last_scrub_stamp":"2024-02-23T00:52:13.442894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:52:13.442894+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":94,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.304523+0000","last_change":"2024-02-23T00:53:53.411805+0000","last_active":"2024-02-23T00:54:40.304523+0000","last_peered":"2024-02-23T00:54:40.304523+0000","last_clean":"2024-02-23T00:54:40.304523+0000","last_became_active":"2024-02-23T00:53:53.411396+0000","last_became_peered":"2024-02-23T00:53:53.411396+0000","last_unstale":"2024-02-23T00:54:40.304523+0000","last_undegraded":"2024-02-23T00:54:40.304523+0000","last_fullsized":"2024-02-23T00:54:40.304523+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:12.335752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.335752+0000","last_clean_scrub_stamp":"2024-02-23T00:51:12.335752+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"457'27","reported_seq":743,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.306407+0000","last_change":"2024-02-23T00:51:48.537804+0000","last_active":"2024-02-23T00:54:40.306407+0000","last_peered":"2024-02-23T00:54:40.306407+0000","last_clean":"2024-02-23T00:54:40.306407+0000","last_became_active":"2024-02-23T00:39:16.789757+0000","last_became_peered":"2024-02-23T00:39:16.789757+0000","last_unstale":"2024-02-23T00:54:40.306407+0000","last_undegraded":"2024-02-23T00:54:40.306407+0000","last_fullsized":"2024-02-23T00:54:40.306407+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-23T00:51:48.537732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:48.537732+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":702,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.304450+0000","last_change":"2024-02-23T00:51:13.375106+0000","last_active":"2024-02-23T00:54:40.304450+0000","last_peered":"2024-02-23T00:54:40.304450+0000","last_clean":"2024-02-23T00:54:40.304450+0000","last_became_active":"2024-02-23T00:39:14.780526+0000","last_became_peered":"2024-02-23T00:39:14.780526+0000","last_unstale":"2024-02-23T00:54:40.304450+0000","last_undegraded":"2024-02-23T00:54:40.304450+0000","last_fullsized":"2024-02-23T00:54:40.304450+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:13.375039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:13.375039+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"694'11","reported_seq":946,"reported_epoch":739,"state":"active+clean","last_fresh":"2024-02-23T00:54:39.391268+0000","last_change":"2024-02-23T00:54:01.484460+0000","last_active":"2024-02-23T00:54:39.391268+0000","last_peered":"2024-02-23T00:54:39.391268+0000","last_clean":"2024-02-23T00:54:39.391268+0000","last_became_active":"2024-02-23T00:53:53.417721+0000","last_became_peered":"2024-02-23T00:53:53.417721+0000","last_unstale":"2024-02-23T00:54:39.391268+0000","last_undegraded":"2024-02-23T00:54:39.391268+0000","last_fullsized":"2024-02-23T00:54:39.391268+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"694'11","last_scrub_stamp":"2024-02-23T00:54:01.484388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:54:01.484388+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":675,"reported_epoch":739,"state":"active+clean","last_fresh":"2024-02-23T00:54:39.389706+0000","last_change":"2024-02-23T00:53:53.414508+0000","last_active":"2024-02-23T00:54:39.389706+0000","last_peered":"2024-02-23T00:54:39.389706+0000","last_clean":"2024-02-23T00:54:39.389706+0000","last_became_active":"2024-02-23T00:53:53.413978+0000","last_became_peered":"2024-02-23T00:53:53.413978+0000","last_unstale":"2024-02-23T00:54:39.389706+0000","last_undegraded":"2024-02-23T00:54:39.389706+0000","last_fullsized":"2024-02-23T00:54:39.389706+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:59.798987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:52:59.798987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"457'31","reported_seq":809,"reported_epoch":739,"state":"active+clean","last_fresh":"2024-02-23T00:54:39.391032+0000","last_change":"2024-02-23T00:53:34.440512+0000","last_active":"2024-02-23T00:54:39.391032+0000","last_peered":"2024-02-23T00:54:39.391032+0000","last_clean":"2024-02-23T00:54:39.391032+0000","last_became_active":"2024-02-23T00:47:28.201225+0000","last_became_peered":"2024-02-23T00:47:28.201225+0000","last_unstale":"2024-02-23T00:54:39.391032+0000","last_undegraded":"2024-02-23T00:54:39.391032+0000","last_fullsized":"2024-02-23T00:54:39.391032+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:53:34.440440+0000","last_deep_scrub":"336'23","last_deep_scrub_stamp":"2024-02-23T00:46:48.320085+0000","last_clean_scrub_stamp":"2024-02-23T00:53:34.440440+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":719,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.302860+0000","last_change":"2024-02-23T00:53:13.358586+0000","last_active":"2024-02-23T00:54:40.302860+0000","last_peered":"2024-02-23T00:54:40.302860+0000","last_clean":"2024-02-23T00:54:40.302860+0000","last_became_active":"2024-02-23T00:53:13.358230+0000","last_became_peered":"2024-02-23T00:53:13.358230+0000","last_unstale":"2024-02-23T00:54:40.302860+0000","last_undegraded":"2024-02-23T00:54:40.302860+0000","last_fullsized":"2024-02-23T00:54:40.302860+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:25.405680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:25.405680+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":708,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.784375+0000","last_change":"2024-02-23T00:52:54.389055+0000","last_active":"2024-02-23T00:54:40.784375+0000","last_peered":"2024-02-23T00:54:40.784375+0000","last_clean":"2024-02-23T00:54:40.784375+0000","last_became_active":"2024-02-23T00:39:18.800777+0000","last_became_peered":"2024-02-23T00:39:18.800777+0000","last_unstale":"2024-02-23T00:54:40.784375+0000","last_undegraded":"2024-02-23T00:54:40.784375+0000","last_fullsized":"2024-02-23T00:54:40.784375+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:54.388949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_clean_scrub_stamp":"2024-02-23T00:52:54.388949+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":644,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.304153+0000","last_change":"2024-02-23T00:53:30.603155+0000","last_active":"2024-02-23T00:54:40.304153+0000","last_peered":"2024-02-23T00:54:40.304153+0000","last_clean":"2024-02-23T00:54:40.304153+0000","last_became_active":"2024-02-23T00:42:57.653279+0000","last_became_peered":"2024-02-23T00:42:57.653279+0000","last_unstale":"2024-02-23T00:54:40.304153+0000","last_undegraded":"2024-02-23T00:54:40.304153+0000","last_fullsized":"2024-02-23T00:54:40.304153+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:30.603090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:53:30.603090+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"694'10","reported_seq":135,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.304002+0000","last_change":"2024-02-23T00:53:53.415736+0000","last_active":"2024-02-23T00:54:40.304002+0000","last_peered":"2024-02-23T00:54:40.304002+0000","last_clean":"2024-02-23T00:54:40.304002+0000","last_became_active":"2024-02-23T00:53:53.407501+0000","last_became_peered":"2024-02-23T00:53:53.407501+0000","last_unstale":"2024-02-23T00:54:40.304002+0000","last_undegraded":"2024-02-23T00:54:40.304002+0000","last_fullsized":"2024-02-23T00:54:40.304002+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'8","last_scrub_stamp":"2024-02-23T00:51:34.577222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:51:34.577222+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"457'42","reported_seq":776,"reported_epoch":739,"state":"active+clean","last_fresh":"2024-02-23T00:54:39.389833+0000","last_change":"2024-02-23T00:52:26.764098+0000","last_active":"2024-02-23T00:54:39.389833+0000","last_peered":"2024-02-23T00:54:39.389833+0000","last_clean":"2024-02-23T00:54:39.389833+0000","last_became_active":"2024-02-23T00:39:16.780130+0000","last_became_peered":"2024-02-23T00:39:16.780130+0000","last_unstale":"2024-02-23T00:54:39.389833+0000","last_undegraded":"2024-02-23T00:54:39.389833+0000","last_fullsized":"2024-02-23T00:54:39.389833+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'42","last_scrub_stamp":"2024-02-23T00:52:26.764031+0000","last_deep_scrub":"423'28","last_deep_scrub_stamp":"2024-02-23T00:48:17.435906+0000","last_clean_scrub_stamp":"2024-02-23T00:52:26.764031+0000","log_size":42,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":46,"num_write":34,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":731,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.782273+0000","last_change":"2024-02-23T00:54:13.439244+0000","last_active":"2024-02-23T00:54:40.782273+0000","last_peered":"2024-02-23T00:54:40.782273+0000","last_clean":"2024-02-23T00:54:40.782273+0000","last_became_active":"2024-02-23T00:47:28.237083+0000","last_became_peered":"2024-02-23T00:47:28.237083+0000","last_unstale":"2024-02-23T00:54:40.782273+0000","last_undegraded":"2024-02-23T00:54:40.782273+0000","last_fullsized":"2024-02-23T00:54:40.782273+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:13.439154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:51.391939+0000","last_clean_scrub_stamp":"2024-02-23T00:54:13.439154+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":723,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.303750+0000","last_change":"2024-02-23T00:52:20.403117+0000","last_active":"2024-02-23T00:54:40.303750+0000","last_peered":"2024-02-23T00:54:40.303750+0000","last_clean":"2024-02-23T00:54:40.303750+0000","last_became_active":"2024-02-23T00:47:30.187386+0000","last_became_peered":"2024-02-23T00:47:30.187386+0000","last_unstale":"2024-02-23T00:54:40.303750+0000","last_undegraded":"2024-02-23T00:54:40.303750+0000","last_fullsized":"2024-02-23T00:54:40.303750+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:20.403031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:46.686764+0000","last_clean_scrub_stamp":"2024-02-23T00:52:20.403031+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":662,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.784584+0000","last_change":"2024-02-23T00:53:32.258905+0000","last_active":"2024-02-23T00:54:40.784584+0000","last_peered":"2024-02-23T00:54:40.784584+0000","last_clean":"2024-02-23T00:54:40.784584+0000","last_became_active":"2024-02-23T00:47:28.212307+0000","last_became_peered":"2024-02-23T00:47:28.212307+0000","last_unstale":"2024-02-23T00:54:40.784584+0000","last_undegraded":"2024-02-23T00:54:40.784584+0000","last_fullsized":"2024-02-23T00:54:40.784584+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:32.258822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:53:32.258822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":729,"reported_epoch":739,"state":"active+clean","last_fresh":"2024-02-23T00:54:39.390056+0000","last_change":"2024-02-23T00:53:53.414314+0000","last_active":"2024-02-23T00:54:39.390056+0000","last_peered":"2024-02-23T00:54:39.390056+0000","last_clean":"2024-02-23T00:54:39.390056+0000","last_became_active":"2024-02-23T00:53:53.413534+0000","last_became_peered":"2024-02-23T00:53:53.413534+0000","last_unstale":"2024-02-23T00:54:39.390056+0000","last_undegraded":"2024-02-23T00:54:39.390056+0000","last_fullsized":"2024-02-23T00:54:39.390056+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:51.722289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:10.493340+0000","last_clean_scrub_stamp":"2024-02-23T00:52:51.722289+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"457'52","reported_seq":649,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.303336+0000","last_change":"2024-02-23T00:53:54.301015+0000","last_active":"2024-02-23T00:54:40.303336+0000","last_peered":"2024-02-23T00:54:40.303336+0000","last_clean":"2024-02-23T00:54:40.303336+0000","last_became_active":"2024-02-23T00:53:53.416131+0000","last_became_peered":"2024-02-23T00:53:53.416131+0000","last_unstale":"2024-02-23T00:54:40.303336+0000","last_undegraded":"2024-02-23T00:54:40.303336+0000","last_fullsized":"2024-02-23T00:54:40.303336+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'52","last_scrub_stamp":"2024-02-23T00:53:02.966145+0000","last_deep_scrub":"457'52","last_deep_scrub_stamp":"2024-02-23T00:53:02.966145+0000","last_clean_scrub_stamp":"2024-02-23T00:53:02.966145+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":75,"num_read_kb":61,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":702,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.305944+0000","last_change":"2024-02-23T00:52:18.475876+0000","last_active":"2024-02-23T00:54:40.305944+0000","last_peered":"2024-02-23T00:54:40.305944+0000","last_clean":"2024-02-23T00:54:40.305944+0000","last_became_active":"2024-02-23T00:39:14.779193+0000","last_became_peered":"2024-02-23T00:39:14.779193+0000","last_unstale":"2024-02-23T00:54:40.305944+0000","last_undegraded":"2024-02-23T00:54:40.305944+0000","last_fullsized":"2024-02-23T00:54:40.305944+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:18.475801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:18.475801+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":752,"reported_epoch":739,"state":"active+clean","last_fresh":"2024-02-23T00:54:39.390557+0000","last_change":"2024-02-23T00:53:53.408511+0000","last_active":"2024-02-23T00:54:39.390557+0000","last_peered":"2024-02-23T00:54:39.390557+0000","last_clean":"2024-02-23T00:54:39.390557+0000","last_became_active":"2024-02-23T00:53:53.408081+0000","last_became_peered":"2024-02-23T00:53:53.408081+0000","last_unstale":"2024-02-23T00:54:39.390557+0000","last_undegraded":"2024-02-23T00:54:39.390557+0000","last_fullsized":"2024-02-23T00:54:39.390557+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:15.680333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:52:15.680333+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":666,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.782538+0000","last_change":"2024-02-23T00:53:53.421417+0000","last_active":"2024-02-23T00:54:40.782538+0000","last_peered":"2024-02-23T00:54:40.782538+0000","last_clean":"2024-02-23T00:54:40.782538+0000","last_became_active":"2024-02-23T00:53:53.421080+0000","last_became_peered":"2024-02-23T00:53:53.421080+0000","last_unstale":"2024-02-23T00:54:40.782538+0000","last_undegraded":"2024-02-23T00:54:40.782538+0000","last_fullsized":"2024-02-23T00:54:40.782538+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:06.983205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:53:06.983205+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":720,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.781758+0000","last_change":"2024-02-23T00:53:13.356926+0000","last_active":"2024-02-23T00:54:40.781758+0000","last_peered":"2024-02-23T00:54:40.781758+0000","last_clean":"2024-02-23T00:54:40.781758+0000","last_became_active":"2024-02-23T00:53:13.352382+0000","last_became_peered":"2024-02-23T00:53:13.352382+0000","last_unstale":"2024-02-23T00:54:40.781758+0000","last_undegraded":"2024-02-23T00:54:40.781758+0000","last_fullsized":"2024-02-23T00:54:40.781758+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:57.490995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:52:57.490995+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,7],"acting":[6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"457'44","reported_seq":618,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:41.255072+0000","last_change":"2024-02-23T00:54:41.255021+0000","last_active":"2024-02-23T00:54:41.255072+0000","last_peered":"2024-02-23T00:54:41.255072+0000","last_clean":"2024-02-23T00:54:41.255072+0000","last_became_active":"2024-02-23T00:53:13.357708+0000","last_became_peered":"2024-02-23T00:53:13.357708+0000","last_unstale":"2024-02-23T00:54:41.255072+0000","last_undegraded":"2024-02-23T00:54:41.255072+0000","last_fullsized":"2024-02-23T00:54:41.255072+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'44","last_scrub_stamp":"2024-02-23T00:54:41.254929+0000","last_deep_scrub":"457'44","last_deep_scrub_stamp":"2024-02-23T00:49:46.644911+0000","last_clean_scrub_stamp":"2024-02-23T00:54:41.254929+0000","log_size":44,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":719,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.303856+0000","last_change":"2024-02-23T00:53:13.352639+0000","last_active":"2024-02-23T00:54:40.303856+0000","last_peered":"2024-02-23T00:54:40.303856+0000","last_clean":"2024-02-23T00:54:40.303856+0000","last_became_active":"2024-02-23T00:53:13.352119+0000","last_became_peered":"2024-02-23T00:53:13.352119+0000","last_unstale":"2024-02-23T00:54:40.303856+0000","last_undegraded":"2024-02-23T00:54:40.303856+0000","last_fullsized":"2024-02-23T00:54:40.303856+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:51.380778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:51.380778+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":723,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.304479+0000","last_change":"2024-02-23T00:52:07.536352+0000","last_active":"2024-02-23T00:54:40.304479+0000","last_peered":"2024-02-23T00:54:40.304479+0000","last_clean":"2024-02-23T00:54:40.304479+0000","last_became_active":"2024-02-23T00:47:37.184240+0000","last_became_peered":"2024-02-23T00:47:37.184240+0000","last_unstale":"2024-02-23T00:54:40.304479+0000","last_undegraded":"2024-02-23T00:54:40.304479+0000","last_fullsized":"2024-02-23T00:54:40.304479+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":385,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:07.536288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:52:07.536288+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":653,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.783833+0000","last_change":"2024-02-23T00:52:55.401053+0000","last_active":"2024-02-23T00:54:40.783833+0000","last_peered":"2024-02-23T00:54:40.783833+0000","last_clean":"2024-02-23T00:54:40.783833+0000","last_became_active":"2024-02-23T00:42:58.726652+0000","last_became_peered":"2024-02-23T00:42:58.726652+0000","last_unstale":"2024-02-23T00:54:40.783833+0000","last_undegraded":"2024-02-23T00:54:40.783833+0000","last_fullsized":"2024-02-23T00:54:40.783833+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:55.400946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:52:55.400946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":722,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.784407+0000","last_change":"2024-02-23T00:54:03.257007+0000","last_active":"2024-02-23T00:54:40.784407+0000","last_peered":"2024-02-23T00:54:40.784407+0000","last_clean":"2024-02-23T00:54:40.784407+0000","last_became_active":"2024-02-23T00:53:13.346347+0000","last_became_peered":"2024-02-23T00:53:13.346347+0000","last_unstale":"2024-02-23T00:54:40.784407+0000","last_undegraded":"2024-02-23T00:54:40.784407+0000","last_fullsized":"2024-02-23T00:54:40.784407+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:03.256917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:59.694033+0000","last_clean_scrub_stamp":"2024-02-23T00:54:03.256917+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"457'30","reported_seq":595,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.306058+0000","last_change":"2024-02-23T00:53:13.377058+0000","last_active":"2024-02-23T00:54:40.306058+0000","last_peered":"2024-02-23T00:54:40.306058+0000","last_clean":"2024-02-23T00:54:40.306058+0000","last_became_active":"2024-02-23T00:53:13.346657+0000","last_became_peered":"2024-02-23T00:53:13.346657+0000","last_unstale":"2024-02-23T00:54:40.306058+0000","last_undegraded":"2024-02-23T00:54:40.306058+0000","last_fullsized":"2024-02-23T00:54:40.306058+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'30","last_scrub_stamp":"2024-02-23T00:52:16.690860+0000","last_deep_scrub":"95'18","last_deep_scrub_stamp":"2024-02-23T00:42:01.195545+0000","last_clean_scrub_stamp":"2024-02-23T00:52:16.690860+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":724,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.302667+0000","last_change":"2024-02-23T00:53:13.357250+0000","last_active":"2024-02-23T00:54:40.302667+0000","last_peered":"2024-02-23T00:54:40.302667+0000","last_clean":"2024-02-23T00:54:40.302667+0000","last_became_active":"2024-02-23T00:53:13.356475+0000","last_became_peered":"2024-02-23T00:53:13.356475+0000","last_unstale":"2024-02-23T00:54:40.302667+0000","last_undegraded":"2024-02-23T00:54:40.302667+0000","last_fullsized":"2024-02-23T00:54:40.302667+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:22.390399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:22.390399+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":114,"reported_epoch":739,"state":"active+clean","last_fresh":"2024-02-23T00:54:39.389965+0000","last_change":"2024-02-23T00:53:53.410002+0000","last_active":"2024-02-23T00:54:39.389965+0000","last_peered":"2024-02-23T00:54:39.389965+0000","last_clean":"2024-02-23T00:54:39.389965+0000","last_became_active":"2024-02-23T00:53:53.409639+0000","last_became_peered":"2024-02-23T00:53:53.409639+0000","last_unstale":"2024-02-23T00:54:39.389965+0000","last_undegraded":"2024-02-23T00:54:39.389965+0000","last_fullsized":"2024-02-23T00:54:39.389965+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:19.764826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:19.764826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.4","version":"67'1","reported_seq":770,"reported_epoch":740,"state":"active+remapped+backfill_toofull","last_fresh":"2024-02-23T00:54:40.305668+0000","last_change":"2024-02-23T00:54:24.406489+0000","last_active":"2024-02-23T00:54:40.305668+0000","last_peered":"2024-02-23T00:54:40.305668+0000","last_clean":"2024-02-23T00:53:51.591230+0000","last_became_active":"2024-02-23T00:53:54.403839+0000","last_became_peered":"2024-02-23T00:53:54.403839+0000","last_unstale":"2024-02-23T00:54:40.305668+0000","last_undegraded":"2024-02-23T00:54:40.305668+0000","last_fullsized":"2024-02-23T00:54:40.305668+0000","mapping_epoch":694,"log_start":"67'1","ondisk_log_start":"67'1","created":65,"last_epoch_clean":670,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:51:02.380721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:51:02.380721+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,6],"avail_no_missing":["1","5","6"],"object_location_counts":[{"shards":"1,5,6","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":675,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.780663+0000","last_change":"2024-02-23T00:53:53.420917+0000","last_active":"2024-02-23T00:54:40.780663+0000","last_peered":"2024-02-23T00:54:40.780663+0000","last_clean":"2024-02-23T00:54:40.780663+0000","last_became_active":"2024-02-23T00:53:53.420593+0000","last_became_peered":"2024-02-23T00:53:53.420593+0000","last_unstale":"2024-02-23T00:54:40.780663+0000","last_undegraded":"2024-02-23T00:54:40.780663+0000","last_fullsized":"2024-02-23T00:54:40.780663+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:51.510968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:35.360054+0000","last_clean_scrub_stamp":"2024-02-23T00:52:51.510968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":729,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.303995+0000","last_change":"2024-02-23T00:53:38.350986+0000","last_active":"2024-02-23T00:54:40.303995+0000","last_peered":"2024-02-23T00:54:40.303995+0000","last_clean":"2024-02-23T00:54:40.303995+0000","last_became_active":"2024-02-23T00:53:13.358164+0000","last_became_peered":"2024-02-23T00:53:13.358164+0000","last_unstale":"2024-02-23T00:54:40.303995+0000","last_undegraded":"2024-02-23T00:54:40.303995+0000","last_fullsized":"2024-02-23T00:54:40.303995+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:38.350917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:53:38.350917+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"457'26","reported_seq":758,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.782316+0000","last_change":"2024-02-23T00:54:01.501056+0000","last_active":"2024-02-23T00:54:40.782316+0000","last_peered":"2024-02-23T00:54:40.782316+0000","last_clean":"2024-02-23T00:54:40.782316+0000","last_became_active":"2024-02-23T00:39:16.858880+0000","last_became_peered":"2024-02-23T00:39:16.858880+0000","last_unstale":"2024-02-23T00:54:40.782316+0000","last_undegraded":"2024-02-23T00:54:40.782316+0000","last_fullsized":"2024-02-23T00:54:40.782316+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-23T00:54:01.500983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:54:01.500983+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":729,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.302634+0000","last_change":"2024-02-23T00:53:13.357436+0000","last_active":"2024-02-23T00:54:40.302634+0000","last_peered":"2024-02-23T00:54:40.302634+0000","last_clean":"2024-02-23T00:54:40.302634+0000","last_became_active":"2024-02-23T00:53:13.356903+0000","last_became_peered":"2024-02-23T00:53:13.356903+0000","last_unstale":"2024-02-23T00:54:40.302634+0000","last_undegraded":"2024-02-23T00:54:40.302634+0000","last_fullsized":"2024-02-23T00:54:40.302634+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:26.398833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:49.977248+0000","last_clean_scrub_stamp":"2024-02-23T00:52:26.398833+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":721,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.784186+0000","last_change":"2024-02-23T00:52:49.465113+0000","last_active":"2024-02-23T00:54:40.784186+0000","last_peered":"2024-02-23T00:54:40.784186+0000","last_clean":"2024-02-23T00:54:40.784186+0000","last_became_active":"2024-02-23T00:47:28.240482+0000","last_became_peered":"2024-02-23T00:47:28.240482+0000","last_unstale":"2024-02-23T00:54:40.784186+0000","last_undegraded":"2024-02-23T00:54:40.784186+0000","last_fullsized":"2024-02-23T00:54:40.784186+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:49.465046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:49.465046+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":93,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.783412+0000","last_change":"2024-02-23T00:53:53.411112+0000","last_active":"2024-02-23T00:54:40.783412+0000","last_peered":"2024-02-23T00:54:40.783412+0000","last_clean":"2024-02-23T00:54:40.783412+0000","last_became_active":"2024-02-23T00:53:53.410801+0000","last_became_peered":"2024-02-23T00:53:53.410801+0000","last_unstale":"2024-02-23T00:54:40.783412+0000","last_undegraded":"2024-02-23T00:54:40.783412+0000","last_fullsized":"2024-02-23T00:54:40.783412+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:53.881152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:53.881152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":747,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.784505+0000","last_change":"2024-02-23T00:54:08.396938+0000","last_active":"2024-02-23T00:54:40.784505+0000","last_peered":"2024-02-23T00:54:40.784505+0000","last_clean":"2024-02-23T00:54:40.784505+0000","last_became_active":"2024-02-23T00:47:45.182550+0000","last_became_peered":"2024-02-23T00:47:45.182550+0000","last_unstale":"2024-02-23T00:54:40.784505+0000","last_undegraded":"2024-02-23T00:54:40.784505+0000","last_fullsized":"2024-02-23T00:54:40.784505+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:08.396857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:54:08.396857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":666,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.303520+0000","last_change":"2024-02-23T00:52:50.382543+0000","last_active":"2024-02-23T00:54:40.303520+0000","last_peered":"2024-02-23T00:54:40.303520+0000","last_clean":"2024-02-23T00:54:40.303520+0000","last_became_active":"2024-02-23T00:47:28.233475+0000","last_became_peered":"2024-02-23T00:47:28.233475+0000","last_unstale":"2024-02-23T00:54:40.303520+0000","last_undegraded":"2024-02-23T00:54:40.303520+0000","last_fullsized":"2024-02-23T00:54:40.303520+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:50.382430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:52:50.382430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":60,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.782071+0000","last_change":"2024-02-23T00:53:45.294558+0000","last_active":"2024-02-23T00:54:40.782071+0000","last_peered":"2024-02-23T00:54:40.782071+0000","last_clean":"2024-02-23T00:54:40.782071+0000","last_became_active":"2024-02-23T00:53:45.294157+0000","last_became_peered":"2024-02-23T00:53:45.294157+0000","last_unstale":"2024-02-23T00:54:40.782071+0000","last_undegraded":"2024-02-23T00:54:40.782071+0000","last_fullsized":"2024-02-23T00:54:40.782071+0000","mapping_epoch":685,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":686,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:39.124251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.264240+0000","last_clean_scrub_stamp":"2024-02-23T00:53:39.124251+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"457'23","reported_seq":771,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.306497+0000","last_change":"2024-02-23T00:53:13.375860+0000","last_active":"2024-02-23T00:54:40.306497+0000","last_peered":"2024-02-23T00:54:40.306497+0000","last_clean":"2024-02-23T00:54:40.306497+0000","last_became_active":"2024-02-23T00:53:13.344389+0000","last_became_peered":"2024-02-23T00:53:13.344389+0000","last_unstale":"2024-02-23T00:54:40.306497+0000","last_undegraded":"2024-02-23T00:54:40.306497+0000","last_fullsized":"2024-02-23T00:54:40.306497+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'23","last_scrub_stamp":"2024-02-23T00:51:42.560138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:42.560138+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":48,"num_read_kb":36,"num_write":26,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":395,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.304361+0000","last_change":"2024-02-23T00:53:53.415443+0000","last_active":"2024-02-23T00:54:40.304361+0000","last_peered":"2024-02-23T00:54:40.304361+0000","last_clean":"2024-02-23T00:54:40.304361+0000","last_became_active":"2024-02-23T00:53:53.415051+0000","last_became_peered":"2024-02-23T00:53:53.415051+0000","last_unstale":"2024-02-23T00:54:40.304361+0000","last_undegraded":"2024-02-23T00:54:40.304361+0000","last_fullsized":"2024-02-23T00:54:40.304361+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:44.659246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:44.659246+0000","last_clean_scrub_stamp":"2024-02-23T00:51:44.659246+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"707'6","reported_seq":1649,"reported_epoch":741,"state":"active+clean","last_fresh":"2024-02-23T00:54:41.310511+0000","last_change":"2024-02-23T00:54:06.324408+0000","last_active":"2024-02-23T00:54:41.310511+0000","last_peered":"2024-02-23T00:54:41.310511+0000","last_clean":"2024-02-23T00:54:41.310511+0000","last_became_active":"2024-02-23T00:53:13.351958+0000","last_became_peered":"2024-02-23T00:53:13.351958+0000","last_unstale":"2024-02-23T00:54:41.310511+0000","last_undegraded":"2024-02-23T00:54:41.310511+0000","last_fullsized":"2024-02-23T00:54:41.310511+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"707'6","last_scrub_stamp":"2024-02-23T00:54:06.324310+0000","last_deep_scrub":"707'6","last_deep_scrub_stamp":"2024-02-23T00:54:06.324310+0000","last_clean_scrub_stamp":"2024-02-23T00:54:06.324310+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":916,"num_read_kb":911,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":740,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.303770+0000","last_change":"2024-02-23T00:52:10.429013+0000","last_active":"2024-02-23T00:54:40.303770+0000","last_peered":"2024-02-23T00:54:40.303770+0000","last_clean":"2024-02-23T00:54:40.303770+0000","last_became_active":"2024-02-23T00:47:52.552244+0000","last_became_peered":"2024-02-23T00:47:52.552244+0000","last_unstale":"2024-02-23T00:54:40.303770+0000","last_undegraded":"2024-02-23T00:54:40.303770+0000","last_fullsized":"2024-02-23T00:54:40.303770+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.428945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.428945+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":653,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.304099+0000","last_change":"2024-02-23T00:53:34.687443+0000","last_active":"2024-02-23T00:54:40.304099+0000","last_peered":"2024-02-23T00:54:40.304099+0000","last_clean":"2024-02-23T00:54:40.304099+0000","last_became_active":"2024-02-23T00:47:28.212516+0000","last_became_peered":"2024-02-23T00:47:28.212516+0000","last_unstale":"2024-02-23T00:54:40.304099+0000","last_undegraded":"2024-02-23T00:54:40.304099+0000","last_fullsized":"2024-02-23T00:54:40.304099+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:34.687347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:53:34.687347+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":95,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.305329+0000","last_change":"2024-02-23T00:53:13.347149+0000","last_active":"2024-02-23T00:54:40.305329+0000","last_peered":"2024-02-23T00:54:40.305329+0000","last_clean":"2024-02-23T00:54:40.305329+0000","last_became_active":"2024-02-23T00:53:13.346793+0000","last_became_peered":"2024-02-23T00:53:13.346793+0000","last_unstale":"2024-02-23T00:54:40.305329+0000","last_undegraded":"2024-02-23T00:54:40.305329+0000","last_fullsized":"2024-02-23T00:54:40.305329+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:17.704169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:52:17.704169+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"694'50","reported_seq":996,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.783755+0000","last_change":"2024-02-23T00:53:54.363459+0000","last_active":"2024-02-23T00:54:40.783755+0000","last_peered":"2024-02-23T00:54:40.783755+0000","last_clean":"2024-02-23T00:54:40.783755+0000","last_became_active":"2024-02-23T00:53:53.412119+0000","last_became_peered":"2024-02-23T00:53:53.412119+0000","last_unstale":"2024-02-23T00:54:40.783755+0000","last_undegraded":"2024-02-23T00:54:40.783755+0000","last_fullsized":"2024-02-23T00:54:40.783755+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'48","last_scrub_stamp":"2024-02-23T00:53:02.417518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:53:02.417518+0000","log_size":50,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":78,"num_read_kb":62,"num_write":44,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":1432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":712,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.783353+0000","last_change":"2024-02-23T00:52:09.366353+0000","last_active":"2024-02-23T00:54:40.783353+0000","last_peered":"2024-02-23T00:54:40.783353+0000","last_clean":"2024-02-23T00:54:40.783353+0000","last_became_active":"2024-02-23T00:39:14.784015+0000","last_became_peered":"2024-02-23T00:39:14.784015+0000","last_unstale":"2024-02-23T00:54:40.783353+0000","last_undegraded":"2024-02-23T00:54:40.783353+0000","last_fullsized":"2024-02-23T00:54:40.783353+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:09.366281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:09.366281+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":481,"reported_epoch":739,"state":"active+clean","last_fresh":"2024-02-23T00:54:39.391286+0000","last_change":"2024-02-23T00:53:53.417644+0000","last_active":"2024-02-23T00:54:39.391286+0000","last_peered":"2024-02-23T00:54:39.391286+0000","last_clean":"2024-02-23T00:54:39.391286+0000","last_became_active":"2024-02-23T00:53:53.417293+0000","last_became_peered":"2024-02-23T00:53:53.417293+0000","last_unstale":"2024-02-23T00:54:39.391286+0000","last_undegraded":"2024-02-23T00:54:39.391286+0000","last_fullsized":"2024-02-23T00:54:39.391286+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:33.479056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:53:33.479056+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":384,"reported_epoch":739,"state":"active+clean","last_fresh":"2024-02-23T00:54:39.390783+0000","last_change":"2024-02-23T00:52:19.722324+0000","last_active":"2024-02-23T00:54:39.390783+0000","last_peered":"2024-02-23T00:54:39.390783+0000","last_clean":"2024-02-23T00:54:39.390783+0000","last_became_active":"2024-02-23T00:47:28.232194+0000","last_became_peered":"2024-02-23T00:47:28.232194+0000","last_unstale":"2024-02-23T00:54:39.390783+0000","last_undegraded":"2024-02-23T00:54:39.390783+0000","last_fullsized":"2024-02-23T00:54:39.390783+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:19.722210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:19.722210+0000","last_clean_scrub_stamp":"2024-02-23T00:52:19.722210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3","version":"656'10","reported_seq":187,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.781670+0000","last_change":"2024-02-23T00:54:03.457138+0000","last_active":"2024-02-23T00:54:40.781670+0000","last_peered":"2024-02-23T00:54:40.781670+0000","last_clean":"2024-02-23T00:54:40.781670+0000","last_became_active":"2024-02-23T00:53:13.352049+0000","last_became_peered":"2024-02-23T00:53:13.352049+0000","last_unstale":"2024-02-23T00:54:40.781670+0000","last_undegraded":"2024-02-23T00:54:40.781670+0000","last_fullsized":"2024-02-23T00:54:40.781670+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"656'10","last_scrub_stamp":"2024-02-23T00:54:03.457066+0000","last_deep_scrub":"91'9","last_deep_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_clean_scrub_stamp":"2024-02-23T00:54:03.457066+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"457'26","reported_seq":745,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.304799+0000","last_change":"2024-02-23T00:52:14.519112+0000","last_active":"2024-02-23T00:54:40.304799+0000","last_peered":"2024-02-23T00:54:40.304799+0000","last_clean":"2024-02-23T00:54:40.304799+0000","last_became_active":"2024-02-23T00:39:16.787468+0000","last_became_peered":"2024-02-23T00:39:16.787468+0000","last_unstale":"2024-02-23T00:54:40.304799+0000","last_undegraded":"2024-02-23T00:54:40.304799+0000","last_fullsized":"2024-02-23T00:54:40.304799+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-23T00:52:14.519049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:52:14.519049+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":36,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":729,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.305403+0000","last_change":"2024-02-23T00:53:53.412122+0000","last_active":"2024-02-23T00:54:40.305403+0000","last_peered":"2024-02-23T00:54:40.305403+0000","last_clean":"2024-02-23T00:54:40.305403+0000","last_became_active":"2024-02-23T00:53:53.411773+0000","last_became_peered":"2024-02-23T00:53:53.411773+0000","last_unstale":"2024-02-23T00:54:40.305403+0000","last_undegraded":"2024-02-23T00:54:40.305403+0000","last_fullsized":"2024-02-23T00:54:40.305403+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:36.415340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:51:36.415340+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"107'2","reported_seq":460,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.783016+0000","last_change":"2024-02-23T00:53:56.306419+0000","last_active":"2024-02-23T00:54:40.783016+0000","last_peered":"2024-02-23T00:54:40.783016+0000","last_clean":"2024-02-23T00:54:40.783016+0000","last_became_active":"2024-02-23T00:53:56.306072+0000","last_became_peered":"2024-02-23T00:53:56.306072+0000","last_unstale":"2024-02-23T00:54:40.783016+0000","last_undegraded":"2024-02-23T00:54:40.783016+0000","last_fullsized":"2024-02-23T00:54:40.783016+0000","mapping_epoch":696,"log_start":"107'2","ondisk_log_start":"107'2","created":65,"last_epoch_clean":697,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'2","last_scrub_stamp":"2024-02-23T00:52:26.704495+0000","last_deep_scrub":"107'2","last_deep_scrub_stamp":"2024-02-23T00:42:48.362739+0000","last_clean_scrub_stamp":"2024-02-23T00:52:26.704495+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":675,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.782208+0000","last_change":"2024-02-23T00:54:00.542746+0000","last_active":"2024-02-23T00:54:40.782208+0000","last_peered":"2024-02-23T00:54:40.782208+0000","last_clean":"2024-02-23T00:54:40.782208+0000","last_became_active":"2024-02-23T00:53:13.355381+0000","last_became_peered":"2024-02-23T00:53:13.355381+0000","last_unstale":"2024-02-23T00:54:40.782208+0000","last_undegraded":"2024-02-23T00:54:40.782208+0000","last_fullsized":"2024-02-23T00:54:40.782208+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:00.542667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:54:00.542667+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":98,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.304055+0000","last_change":"2024-02-23T00:53:13.346513+0000","last_active":"2024-02-23T00:54:40.304055+0000","last_peered":"2024-02-23T00:54:40.304055+0000","last_clean":"2024-02-23T00:54:40.304055+0000","last_became_active":"2024-02-23T00:53:13.346122+0000","last_became_peered":"2024-02-23T00:53:13.346122+0000","last_unstale":"2024-02-23T00:54:40.304055+0000","last_undegraded":"2024-02-23T00:54:40.304055+0000","last_fullsized":"2024-02-23T00:54:40.304055+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:05.859124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_clean_scrub_stamp":"2024-02-23T00:53:05.859124+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"457'31","reported_seq":824,"reported_epoch":739,"state":"active+clean","last_fresh":"2024-02-23T00:54:39.390049+0000","last_change":"2024-02-23T00:53:54.307638+0000","last_active":"2024-02-23T00:54:39.390049+0000","last_peered":"2024-02-23T00:54:39.390049+0000","last_clean":"2024-02-23T00:54:39.390049+0000","last_became_active":"2024-02-23T00:53:53.410425+0000","last_became_peered":"2024-02-23T00:53:53.410425+0000","last_unstale":"2024-02-23T00:54:39.390049+0000","last_undegraded":"2024-02-23T00:54:39.390049+0000","last_fullsized":"2024-02-23T00:54:39.390049+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:52:58.827562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:52:58.827562+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":58,"num_read_kb":47,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"95'4","reported_seq":748,"reported_epoch":739,"state":"active+clean","last_fresh":"2024-02-23T00:54:39.391452+0000","last_change":"2024-02-23T00:53:38.580929+0000","last_active":"2024-02-23T00:54:39.391452+0000","last_peered":"2024-02-23T00:54:39.391452+0000","last_clean":"2024-02-23T00:54:39.391452+0000","last_became_active":"2024-02-23T00:53:13.336812+0000","last_became_peered":"2024-02-23T00:53:13.336812+0000","last_unstale":"2024-02-23T00:54:39.391452+0000","last_undegraded":"2024-02-23T00:54:39.391452+0000","last_fullsized":"2024-02-23T00:54:39.391452+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'4","last_scrub_stamp":"2024-02-23T00:53:38.580857+0000","last_deep_scrub":"95'4","last_deep_scrub_stamp":"2024-02-23T00:41:43.874397+0000","last_clean_scrub_stamp":"2024-02-23T00:53:38.580857+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,5],"acting":[4,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":172,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.303403+0000","last_change":"2024-02-23T00:53:34.318174+0000","last_active":"2024-02-23T00:54:40.303403+0000","last_peered":"2024-02-23T00:54:40.303403+0000","last_clean":"2024-02-23T00:54:40.303403+0000","last_became_active":"2024-02-23T00:53:34.317883+0000","last_became_peered":"2024-02-23T00:53:34.317883+0000","last_unstale":"2024-02-23T00:54:40.303403+0000","last_undegraded":"2024-02-23T00:54:40.303403+0000","last_fullsized":"2024-02-23T00:54:40.303403+0000","mapping_epoch":676,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":677,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:23.776775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:23.776775+0000","last_clean_scrub_stamp":"2024-02-23T00:52:23.776775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":676,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.304212+0000","last_change":"2024-02-23T00:53:13.341671+0000","last_active":"2024-02-23T00:54:40.304212+0000","last_peered":"2024-02-23T00:54:40.304212+0000","last_clean":"2024-02-23T00:54:40.304212+0000","last_became_active":"2024-02-23T00:53:13.341337+0000","last_became_peered":"2024-02-23T00:53:13.341337+0000","last_unstale":"2024-02-23T00:54:40.304212+0000","last_undegraded":"2024-02-23T00:54:40.304212+0000","last_fullsized":"2024-02-23T00:54:40.304212+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:02.354546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:53:02.354546+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"457'27","reported_seq":597,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.303454+0000","last_change":"2024-02-23T00:54:03.163137+0000","last_active":"2024-02-23T00:54:40.303454+0000","last_peered":"2024-02-23T00:54:40.303454+0000","last_clean":"2024-02-23T00:54:40.303454+0000","last_became_active":"2024-02-23T00:53:53.424060+0000","last_became_peered":"2024-02-23T00:53:53.424060+0000","last_unstale":"2024-02-23T00:54:40.303454+0000","last_undegraded":"2024-02-23T00:54:40.303454+0000","last_fullsized":"2024-02-23T00:54:40.303454+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-23T00:54:03.163053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:54:03.163053+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":49,"num_read_kb":41,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":386,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.304202+0000","last_change":"2024-02-23T00:53:13.352149+0000","last_active":"2024-02-23T00:54:40.304202+0000","last_peered":"2024-02-23T00:54:40.304202+0000","last_clean":"2024-02-23T00:54:40.304202+0000","last_became_active":"2024-02-23T00:53:13.351563+0000","last_became_peered":"2024-02-23T00:53:13.351563+0000","last_unstale":"2024-02-23T00:54:40.304202+0000","last_undegraded":"2024-02-23T00:54:40.304202+0000","last_fullsized":"2024-02-23T00:54:40.304202+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:21.536419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:21.536419+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":93,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.306450+0000","last_change":"2024-02-23T00:53:53.411932+0000","last_active":"2024-02-23T00:54:40.306450+0000","last_peered":"2024-02-23T00:54:40.306450+0000","last_clean":"2024-02-23T00:54:40.306450+0000","last_became_active":"2024-02-23T00:53:53.411604+0000","last_became_peered":"2024-02-23T00:53:53.411604+0000","last_unstale":"2024-02-23T00:54:40.306450+0000","last_undegraded":"2024-02-23T00:54:40.306450+0000","last_fullsized":"2024-02-23T00:54:40.306450+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:45.694974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:45.913526+0000","last_clean_scrub_stamp":"2024-02-23T00:51:45.694974+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":102,"reported_epoch":739,"state":"active+clean","last_fresh":"2024-02-23T00:54:39.425012+0000","last_change":"2024-02-23T00:54:39.424971+0000","last_active":"2024-02-23T00:54:39.425012+0000","last_peered":"2024-02-23T00:54:39.425012+0000","last_clean":"2024-02-23T00:54:39.425012+0000","last_became_active":"2024-02-23T00:53:13.338084+0000","last_became_peered":"2024-02-23T00:53:13.338084+0000","last_unstale":"2024-02-23T00:54:39.425012+0000","last_undegraded":"2024-02-23T00:54:39.425012+0000","last_fullsized":"2024-02-23T00:54:39.425012+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:39.424888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:54:39.424888+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"457'22","reported_seq":577,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.304330+0000","last_change":"2024-02-23T00:53:54.319331+0000","last_active":"2024-02-23T00:54:40.304330+0000","last_peered":"2024-02-23T00:54:40.304330+0000","last_clean":"2024-02-23T00:54:40.304330+0000","last_became_active":"2024-02-23T00:53:53.416008+0000","last_became_peered":"2024-02-23T00:53:53.416008+0000","last_unstale":"2024-02-23T00:54:40.304330+0000","last_undegraded":"2024-02-23T00:54:40.304330+0000","last_fullsized":"2024-02-23T00:54:40.304330+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'22","last_scrub_stamp":"2024-02-23T00:53:32.317744+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-02-23T00:48:07.941101+0000","last_clean_scrub_stamp":"2024-02-23T00:53:32.317744+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":715,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.780982+0000","last_change":"2024-02-23T00:54:09.497814+0000","last_active":"2024-02-23T00:54:40.780982+0000","last_peered":"2024-02-23T00:54:40.780982+0000","last_clean":"2024-02-23T00:54:40.780982+0000","last_became_active":"2024-02-23T00:39:14.783164+0000","last_became_peered":"2024-02-23T00:39:14.783164+0000","last_unstale":"2024-02-23T00:54:40.780982+0000","last_undegraded":"2024-02-23T00:54:40.780982+0000","last_fullsized":"2024-02-23T00:54:40.780982+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:09.497701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:54:09.497701+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":713,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.784027+0000","last_change":"2024-02-23T00:54:00.288206+0000","last_active":"2024-02-23T00:54:40.784027+0000","last_peered":"2024-02-23T00:54:40.784027+0000","last_clean":"2024-02-23T00:54:40.784027+0000","last_became_active":"2024-02-23T00:39:18.800297+0000","last_became_peered":"2024-02-23T00:39:18.800297+0000","last_unstale":"2024-02-23T00:54:40.784027+0000","last_undegraded":"2024-02-23T00:54:40.784027+0000","last_fullsized":"2024-02-23T00:54:40.784027+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:00.288132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:54:00.288132+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1","version":"110'7","reported_seq":92,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.780974+0000","last_change":"2024-02-23T00:53:34.322710+0000","last_active":"2024-02-23T00:54:40.780974+0000","last_peered":"2024-02-23T00:54:40.780974+0000","last_clean":"2024-02-23T00:54:40.780974+0000","last_became_active":"2024-02-23T00:53:34.322370+0000","last_became_peered":"2024-02-23T00:53:34.322370+0000","last_unstale":"2024-02-23T00:54:40.780974+0000","last_undegraded":"2024-02-23T00:54:40.780974+0000","last_fullsized":"2024-02-23T00:54:40.780974+0000","mapping_epoch":676,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":677,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'7","last_scrub_stamp":"2024-02-23T00:53:04.958282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:53:04.958282+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"457'32","reported_seq":194,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.305504+0000","last_change":"2024-02-23T00:53:34.316882+0000","last_active":"2024-02-23T00:54:40.305504+0000","last_peered":"2024-02-23T00:54:40.305504+0000","last_clean":"2024-02-23T00:54:40.305504+0000","last_became_active":"2024-02-23T00:53:34.316552+0000","last_became_peered":"2024-02-23T00:53:34.316552+0000","last_unstale":"2024-02-23T00:54:40.305504+0000","last_undegraded":"2024-02-23T00:54:40.305504+0000","last_fullsized":"2024-02-23T00:54:40.305504+0000","mapping_epoch":676,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":677,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-23T00:53:30.199998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:53:30.199998+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":32,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":717,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.783257+0000","last_change":"2024-02-23T00:53:35.190352+0000","last_active":"2024-02-23T00:54:40.783257+0000","last_peered":"2024-02-23T00:54:40.783257+0000","last_clean":"2024-02-23T00:54:40.783257+0000","last_became_active":"2024-02-23T00:39:14.785590+0000","last_became_peered":"2024-02-23T00:39:14.785590+0000","last_unstale":"2024-02-23T00:54:40.783257+0000","last_undegraded":"2024-02-23T00:54:40.783257+0000","last_fullsized":"2024-02-23T00:54:40.783257+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:35.190288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_clean_scrub_stamp":"2024-02-23T00:53:35.190288+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"656'10","reported_seq":920,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.784428+0000","last_change":"2024-02-23T00:54:02.269290+0000","last_active":"2024-02-23T00:54:40.784428+0000","last_peered":"2024-02-23T00:54:40.784428+0000","last_clean":"2024-02-23T00:54:40.784428+0000","last_became_active":"2024-02-23T00:53:13.343175+0000","last_became_peered":"2024-02-23T00:53:13.343175+0000","last_unstale":"2024-02-23T00:54:40.784428+0000","last_undegraded":"2024-02-23T00:54:40.784428+0000","last_fullsized":"2024-02-23T00:54:40.784428+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"656'10","last_scrub_stamp":"2024-02-23T00:54:02.269193+0000","last_deep_scrub":"336'9","last_deep_scrub_stamp":"2024-02-23T00:46:49.160991+0000","last_clean_scrub_stamp":"2024-02-23T00:54:02.269193+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":662,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.303775+0000","last_change":"2024-02-23T00:53:13.349504+0000","last_active":"2024-02-23T00:54:40.303775+0000","last_peered":"2024-02-23T00:54:40.303775+0000","last_clean":"2024-02-23T00:54:40.303775+0000","last_became_active":"2024-02-23T00:53:13.349155+0000","last_became_peered":"2024-02-23T00:53:13.349155+0000","last_unstale":"2024-02-23T00:54:40.303775+0000","last_undegraded":"2024-02-23T00:54:40.303775+0000","last_fullsized":"2024-02-23T00:54:40.303775+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:00.410226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:53:00.410226+0000","last_clean_scrub_stamp":"2024-02-23T00:53:00.410226+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"457'21","reported_seq":750,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.784322+0000","last_change":"2024-02-23T00:52:27.456856+0000","last_active":"2024-02-23T00:54:40.784322+0000","last_peered":"2024-02-23T00:54:40.784322+0000","last_clean":"2024-02-23T00:54:40.784322+0000","last_became_active":"2024-02-23T00:39:16.858953+0000","last_became_peered":"2024-02-23T00:39:16.858953+0000","last_unstale":"2024-02-23T00:54:40.784322+0000","last_undegraded":"2024-02-23T00:54:40.784322+0000","last_fullsized":"2024-02-23T00:54:40.784322+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-23T00:52:27.456787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:52:27.456787+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":101,"reported_epoch":739,"state":"active+clean","last_fresh":"2024-02-23T00:54:39.390756+0000","last_change":"2024-02-23T00:53:29.500174+0000","last_active":"2024-02-23T00:54:39.390756+0000","last_peered":"2024-02-23T00:54:39.390756+0000","last_clean":"2024-02-23T00:54:39.390756+0000","last_became_active":"2024-02-23T00:53:13.340985+0000","last_became_peered":"2024-02-23T00:53:13.340985+0000","last_unstale":"2024-02-23T00:54:39.390756+0000","last_undegraded":"2024-02-23T00:54:39.390756+0000","last_fullsized":"2024-02-23T00:54:39.390756+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:29.500109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:53:29.500109+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"694'11","reported_seq":135,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.303741+0000","last_change":"2024-02-23T00:53:54.308918+0000","last_active":"2024-02-23T00:54:40.303741+0000","last_peered":"2024-02-23T00:54:40.303741+0000","last_clean":"2024-02-23T00:54:40.303741+0000","last_became_active":"2024-02-23T00:53:53.415965+0000","last_became_peered":"2024-02-23T00:53:53.415965+0000","last_unstale":"2024-02-23T00:54:40.303741+0000","last_undegraded":"2024-02-23T00:54:40.303741+0000","last_fullsized":"2024-02-23T00:54:40.303741+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"376'9","last_scrub_stamp":"2024-02-23T00:52:07.717186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:52:07.717186+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":644,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.304773+0000","last_change":"2024-02-23T00:54:03.413676+0000","last_active":"2024-02-23T00:54:40.304773+0000","last_peered":"2024-02-23T00:54:40.304773+0000","last_clean":"2024-02-23T00:54:40.304773+0000","last_became_active":"2024-02-23T00:42:57.652488+0000","last_became_peered":"2024-02-23T00:42:57.652488+0000","last_unstale":"2024-02-23T00:54:40.304773+0000","last_undegraded":"2024-02-23T00:54:40.304773+0000","last_fullsized":"2024-02-23T00:54:40.304773+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:03.413609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:51.254903+0000","last_clean_scrub_stamp":"2024-02-23T00:54:03.413609+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"457'15","reported_seq":763,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.304624+0000","last_change":"2024-02-23T00:53:13.722705+0000","last_active":"2024-02-23T00:54:40.304624+0000","last_peered":"2024-02-23T00:54:40.304624+0000","last_clean":"2024-02-23T00:54:40.304624+0000","last_became_active":"2024-02-23T00:53:13.350056+0000","last_became_peered":"2024-02-23T00:53:13.350056+0000","last_unstale":"2024-02-23T00:54:40.304624+0000","last_undegraded":"2024-02-23T00:54:40.304624+0000","last_fullsized":"2024-02-23T00:54:40.304624+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'15","last_scrub_stamp":"2024-02-23T00:53:06.466487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:53:06.466487+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":729,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.782115+0000","last_change":"2024-02-23T00:53:45.293671+0000","last_active":"2024-02-23T00:54:40.782115+0000","last_peered":"2024-02-23T00:54:40.782115+0000","last_clean":"2024-02-23T00:54:40.782115+0000","last_became_active":"2024-02-23T00:53:45.293336+0000","last_became_peered":"2024-02-23T00:53:45.293336+0000","last_unstale":"2024-02-23T00:54:40.782115+0000","last_undegraded":"2024-02-23T00:54:40.782115+0000","last_fullsized":"2024-02-23T00:54:40.782115+0000","mapping_epoch":685,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":686,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:03.382646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:53:03.382646+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":714,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.305113+0000","last_change":"2024-02-23T00:53:34.322872+0000","last_active":"2024-02-23T00:54:40.305113+0000","last_peered":"2024-02-23T00:54:40.305113+0000","last_clean":"2024-02-23T00:54:40.305113+0000","last_became_active":"2024-02-23T00:53:34.322592+0000","last_became_peered":"2024-02-23T00:53:34.322592+0000","last_unstale":"2024-02-23T00:54:40.305113+0000","last_undegraded":"2024-02-23T00:54:40.305113+0000","last_fullsized":"2024-02-23T00:54:40.305113+0000","mapping_epoch":676,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":677,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:27.761922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_clean_scrub_stamp":"2024-02-23T00:52:27.761922+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":96,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.305786+0000","last_change":"2024-02-23T00:53:13.351294+0000","last_active":"2024-02-23T00:54:40.305786+0000","last_peered":"2024-02-23T00:54:40.305786+0000","last_clean":"2024-02-23T00:54:40.305786+0000","last_became_active":"2024-02-23T00:53:13.350769+0000","last_became_peered":"2024-02-23T00:53:13.350769+0000","last_unstale":"2024-02-23T00:54:40.305786+0000","last_undegraded":"2024-02-23T00:54:40.305786+0000","last_fullsized":"2024-02-23T00:54:40.305786+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:44.638129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:51:44.638129+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"457'19","reported_seq":742,"reported_epoch":739,"state":"active+clean","last_fresh":"2024-02-23T00:54:39.390718+0000","last_change":"2024-02-23T00:52:52.760115+0000","last_active":"2024-02-23T00:54:39.390718+0000","last_peered":"2024-02-23T00:54:39.390718+0000","last_clean":"2024-02-23T00:54:39.390718+0000","last_became_active":"2024-02-23T00:39:16.783896+0000","last_became_peered":"2024-02-23T00:39:16.783896+0000","last_unstale":"2024-02-23T00:54:39.390718+0000","last_undegraded":"2024-02-23T00:54:39.390718+0000","last_fullsized":"2024-02-23T00:54:39.390718+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'19","last_scrub_stamp":"2024-02-23T00:52:52.760046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:52:52.760046+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"202'2","reported_seq":723,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.305498+0000","last_change":"2024-02-23T00:53:13.385758+0000","last_active":"2024-02-23T00:54:40.305498+0000","last_peered":"2024-02-23T00:54:40.305498+0000","last_clean":"2024-02-23T00:54:40.305498+0000","last_became_active":"2024-02-23T00:53:13.351686+0000","last_became_peered":"2024-02-23T00:53:13.351686+0000","last_unstale":"2024-02-23T00:54:40.305498+0000","last_undegraded":"2024-02-23T00:54:40.305498+0000","last_fullsized":"2024-02-23T00:54:40.305498+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"202'2","last_scrub_stamp":"2024-02-23T00:51:40.511258+0000","last_deep_scrub":"202'2","last_deep_scrub_stamp":"2024-02-23T00:44:34.720640+0000","last_clean_scrub_stamp":"2024-02-23T00:51:40.511258+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":873,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":716,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.304710+0000","last_change":"2024-02-23T00:53:53.417039+0000","last_active":"2024-02-23T00:54:40.304710+0000","last_peered":"2024-02-23T00:54:40.304710+0000","last_clean":"2024-02-23T00:54:40.304710+0000","last_became_active":"2024-02-23T00:53:53.416685+0000","last_became_peered":"2024-02-23T00:53:53.416685+0000","last_unstale":"2024-02-23T00:54:40.304710+0000","last_undegraded":"2024-02-23T00:54:40.304710+0000","last_fullsized":"2024-02-23T00:54:40.304710+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:41.472347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:53:41.472347+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":662,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.305407+0000","last_change":"2024-02-23T00:53:13.360915+0000","last_active":"2024-02-23T00:54:40.305407+0000","last_peered":"2024-02-23T00:54:40.305407+0000","last_clean":"2024-02-23T00:54:40.305407+0000","last_became_active":"2024-02-23T00:53:13.360410+0000","last_became_peered":"2024-02-23T00:53:13.360410+0000","last_unstale":"2024-02-23T00:54:40.305407+0000","last_undegraded":"2024-02-23T00:54:40.305407+0000","last_fullsized":"2024-02-23T00:54:40.305407+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:58.516233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:46.462015+0000","last_clean_scrub_stamp":"2024-02-23T00:52:58.516233+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"457'8","reported_seq":539,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.304346+0000","last_change":"2024-02-23T00:53:53.418225+0000","last_active":"2024-02-23T00:54:40.304346+0000","last_peered":"2024-02-23T00:54:40.304346+0000","last_clean":"2024-02-23T00:54:40.304346+0000","last_became_active":"2024-02-23T00:53:53.403080+0000","last_became_peered":"2024-02-23T00:53:53.403080+0000","last_unstale":"2024-02-23T00:54:40.304346+0000","last_undegraded":"2024-02-23T00:54:40.304346+0000","last_fullsized":"2024-02-23T00:54:40.304346+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'8","last_scrub_stamp":"2024-02-23T00:52:49.890254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:52:49.890254+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":729,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.784022+0000","last_change":"2024-02-23T00:53:40.166252+0000","last_active":"2024-02-23T00:54:40.784022+0000","last_peered":"2024-02-23T00:54:40.784022+0000","last_clean":"2024-02-23T00:54:40.784022+0000","last_became_active":"2024-02-23T00:47:28.224477+0000","last_became_peered":"2024-02-23T00:47:28.224477+0000","last_unstale":"2024-02-23T00:54:40.784022+0000","last_undegraded":"2024-02-23T00:54:40.784022+0000","last_fullsized":"2024-02-23T00:54:40.784022+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:40.166169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:53:40.166169+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":725,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.780866+0000","last_change":"2024-02-23T00:53:59.499809+0000","last_active":"2024-02-23T00:54:40.780866+0000","last_peered":"2024-02-23T00:54:40.780866+0000","last_clean":"2024-02-23T00:54:40.780866+0000","last_became_active":"2024-02-23T00:47:44.230295+0000","last_became_peered":"2024-02-23T00:47:44.230295+0000","last_unstale":"2024-02-23T00:54:40.780866+0000","last_undegraded":"2024-02-23T00:54:40.780866+0000","last_fullsized":"2024-02-23T00:54:40.780866+0000","mapping_epoch":391,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":392,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:59.499666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:53:59.499666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":653,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.303732+0000","last_change":"2024-02-23T00:53:01.388275+0000","last_active":"2024-02-23T00:54:40.303732+0000","last_peered":"2024-02-23T00:54:40.303732+0000","last_clean":"2024-02-23T00:54:40.303732+0000","last_became_active":"2024-02-23T00:42:58.741365+0000","last_became_peered":"2024-02-23T00:42:58.741365+0000","last_unstale":"2024-02-23T00:54:40.303732+0000","last_undegraded":"2024-02-23T00:54:40.303732+0000","last_fullsized":"2024-02-23T00:54:40.303732+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:01.388176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:53:01.388176+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"457'25","reported_seq":542,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.302782+0000","last_change":"2024-02-23T00:53:34.322025+0000","last_active":"2024-02-23T00:54:40.302782+0000","last_peered":"2024-02-23T00:54:40.302782+0000","last_clean":"2024-02-23T00:54:40.302782+0000","last_became_active":"2024-02-23T00:53:34.321651+0000","last_became_peered":"2024-02-23T00:53:34.321651+0000","last_unstale":"2024-02-23T00:54:40.302782+0000","last_undegraded":"2024-02-23T00:54:40.302782+0000","last_fullsized":"2024-02-23T00:54:40.302782+0000","mapping_epoch":676,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":677,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-23T00:53:29.192169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:53:29.192169+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":43,"num_read_kb":35,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":724,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.782177+0000","last_change":"2024-02-23T00:52:53.475514+0000","last_active":"2024-02-23T00:54:40.782177+0000","last_peered":"2024-02-23T00:54:40.782177+0000","last_clean":"2024-02-23T00:54:40.782177+0000","last_became_active":"2024-02-23T00:47:28.240162+0000","last_became_peered":"2024-02-23T00:47:28.240162+0000","last_unstale":"2024-02-23T00:54:40.782177+0000","last_undegraded":"2024-02-23T00:54:40.782177+0000","last_fullsized":"2024-02-23T00:54:40.782177+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:53.475444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:44.285083+0000","last_clean_scrub_stamp":"2024-02-23T00:52:53.475444+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":68,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.302369+0000","last_change":"2024-02-23T00:54:12.297949+0000","last_active":"2024-02-23T00:54:40.302369+0000","last_peered":"2024-02-23T00:54:40.302369+0000","last_clean":"2024-02-23T00:54:40.302369+0000","last_became_active":"2024-02-23T00:53:45.290784+0000","last_became_peered":"2024-02-23T00:53:45.290784+0000","last_unstale":"2024-02-23T00:54:40.302369+0000","last_undegraded":"2024-02-23T00:54:40.302369+0000","last_fullsized":"2024-02-23T00:54:40.302369+0000","mapping_epoch":685,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":686,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:12.297867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:54:12.297867+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":404,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.784230+0000","last_change":"2024-02-23T00:53:53.412837+0000","last_active":"2024-02-23T00:54:40.784230+0000","last_peered":"2024-02-23T00:54:40.784230+0000","last_clean":"2024-02-23T00:54:40.784230+0000","last_became_active":"2024-02-23T00:53:53.412497+0000","last_became_peered":"2024-02-23T00:53:53.412497+0000","last_unstale":"2024-02-23T00:54:40.784230+0000","last_undegraded":"2024-02-23T00:54:40.784230+0000","last_fullsized":"2024-02-23T00:54:40.784230+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:48.449279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:52:48.449279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"457'26","reported_seq":621,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.303627+0000","last_change":"2024-02-23T00:53:54.315328+0000","last_active":"2024-02-23T00:54:40.303627+0000","last_peered":"2024-02-23T00:54:40.303627+0000","last_clean":"2024-02-23T00:54:40.303627+0000","last_became_active":"2024-02-23T00:53:53.416434+0000","last_became_peered":"2024-02-23T00:53:53.416434+0000","last_unstale":"2024-02-23T00:54:40.303627+0000","last_undegraded":"2024-02-23T00:54:40.303627+0000","last_fullsized":"2024-02-23T00:54:40.303627+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-23T00:52:53.810591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:52:53.810591+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":35,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":724,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.783110+0000","last_change":"2024-02-23T00:53:34.208099+0000","last_active":"2024-02-23T00:54:40.783110+0000","last_peered":"2024-02-23T00:54:40.783110+0000","last_clean":"2024-02-23T00:54:40.783110+0000","last_became_active":"2024-02-23T00:39:14.785363+0000","last_became_peered":"2024-02-23T00:39:14.785363+0000","last_unstale":"2024-02-23T00:54:40.783110+0000","last_undegraded":"2024-02-23T00:54:40.783110+0000","last_fullsized":"2024-02-23T00:54:40.783110+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"80'2","last_scrub_stamp":"2024-02-23T00:53:34.208030+0000","last_deep_scrub":"80'2","last_deep_scrub_stamp":"2024-02-23T00:40:15.942725+0000","last_clean_scrub_stamp":"2024-02-23T00:53:34.208030+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"93'9","reported_seq":908,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.782371+0000","last_change":"2024-02-23T00:52:56.514149+0000","last_active":"2024-02-23T00:54:40.782371+0000","last_peered":"2024-02-23T00:54:40.782371+0000","last_clean":"2024-02-23T00:54:40.782371+0000","last_became_active":"2024-02-23T00:39:18.796680+0000","last_became_peered":"2024-02-23T00:39:18.796680+0000","last_unstale":"2024-02-23T00:54:40.782371+0000","last_undegraded":"2024-02-23T00:54:40.782371+0000","last_fullsized":"2024-02-23T00:54:40.782371+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-02-23T00:52:56.514068+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-02-23T00:43:50.393713+0000","last_clean_scrub_stamp":"2024-02-23T00:52:56.514068+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":372,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.781123+0000","last_change":"2024-02-23T00:53:13.353721+0000","last_active":"2024-02-23T00:54:40.781123+0000","last_peered":"2024-02-23T00:54:40.781123+0000","last_clean":"2024-02-23T00:54:40.781123+0000","last_became_active":"2024-02-23T00:53:13.352348+0000","last_became_peered":"2024-02-23T00:53:13.352348+0000","last_unstale":"2024-02-23T00:54:40.781123+0000","last_undegraded":"2024-02-23T00:54:40.781123+0000","last_fullsized":"2024-02-23T00:54:40.781123+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:01.802405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:53:01.802405+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"457'31","reported_seq":775,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.782455+0000","last_change":"2024-02-23T00:54:11.502592+0000","last_active":"2024-02-23T00:54:40.782455+0000","last_peered":"2024-02-23T00:54:40.782455+0000","last_clean":"2024-02-23T00:54:40.782455+0000","last_became_active":"2024-02-23T00:39:16.782660+0000","last_became_peered":"2024-02-23T00:39:16.782660+0000","last_unstale":"2024-02-23T00:54:40.782455+0000","last_undegraded":"2024-02-23T00:54:40.782455+0000","last_fullsized":"2024-02-23T00:54:40.782455+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:54:11.502514+0000","last_deep_scrub":"457'31","last_deep_scrub_stamp":"2024-02-23T00:52:27.357369+0000","last_clean_scrub_stamp":"2024-02-23T00:54:11.502514+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":730,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.780865+0000","last_change":"2024-02-23T00:54:12.460423+0000","last_active":"2024-02-23T00:54:40.780865+0000","last_peered":"2024-02-23T00:54:40.780865+0000","last_clean":"2024-02-23T00:54:40.780865+0000","last_became_active":"2024-02-23T00:53:53.419607+0000","last_became_peered":"2024-02-23T00:53:53.419607+0000","last_unstale":"2024-02-23T00:54:40.780865+0000","last_undegraded":"2024-02-23T00:54:40.780865+0000","last_fullsized":"2024-02-23T00:54:40.780865+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:12.460337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:54:12.460337+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":435,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.303903+0000","last_change":"2024-02-23T00:53:13.346669+0000","last_active":"2024-02-23T00:54:40.303903+0000","last_peered":"2024-02-23T00:54:40.303903+0000","last_clean":"2024-02-23T00:54:40.303903+0000","last_became_active":"2024-02-23T00:53:13.346312+0000","last_became_peered":"2024-02-23T00:53:13.346312+0000","last_unstale":"2024-02-23T00:54:40.303903+0000","last_undegraded":"2024-02-23T00:54:40.303903+0000","last_fullsized":"2024-02-23T00:54:40.303903+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:00.845179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:53.189881+0000","last_clean_scrub_stamp":"2024-02-23T00:53:00.845179+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":75,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.783322+0000","last_change":"2024-02-23T00:53:34.321705+0000","last_active":"2024-02-23T00:54:40.783322+0000","last_peered":"2024-02-23T00:54:40.783322+0000","last_clean":"2024-02-23T00:54:40.783322+0000","last_became_active":"2024-02-23T00:53:34.320314+0000","last_became_peered":"2024-02-23T00:53:34.320314+0000","last_unstale":"2024-02-23T00:54:40.783322+0000","last_undegraded":"2024-02-23T00:54:40.783322+0000","last_fullsized":"2024-02-23T00:54:40.783322+0000","mapping_epoch":676,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":677,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:57.917367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:52:57.917367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"457'27","reported_seq":523,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.305453+0000","last_change":"2024-02-23T00:53:54.456607+0000","last_active":"2024-02-23T00:54:40.305453+0000","last_peered":"2024-02-23T00:54:40.305453+0000","last_clean":"2024-02-23T00:54:40.305453+0000","last_became_active":"2024-02-23T00:53:53.420397+0000","last_became_peered":"2024-02-23T00:53:53.420397+0000","last_unstale":"2024-02-23T00:54:40.305453+0000","last_undegraded":"2024-02-23T00:54:40.305453+0000","last_fullsized":"2024-02-23T00:54:40.305453+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":695,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-23T00:52:55.480770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:52:55.480770+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":715,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.781651+0000","last_change":"2024-02-23T00:52:49.496869+0000","last_active":"2024-02-23T00:54:40.781651+0000","last_peered":"2024-02-23T00:54:40.781651+0000","last_clean":"2024-02-23T00:54:40.781651+0000","last_became_active":"2024-02-23T00:39:14.782494+0000","last_became_peered":"2024-02-23T00:39:14.782494+0000","last_unstale":"2024-02-23T00:54:40.781651+0000","last_undegraded":"2024-02-23T00:54:40.781651+0000","last_fullsized":"2024-02-23T00:54:40.781651+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:49.496739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:49.496739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":707,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.305738+0000","last_change":"2024-02-23T00:53:13.350017+0000","last_active":"2024-02-23T00:54:40.305738+0000","last_peered":"2024-02-23T00:54:40.305738+0000","last_clean":"2024-02-23T00:54:40.305738+0000","last_became_active":"2024-02-23T00:53:13.349684+0000","last_became_peered":"2024-02-23T00:53:13.349684+0000","last_unstale":"2024-02-23T00:54:40.305738+0000","last_undegraded":"2024-02-23T00:54:40.305738+0000","last_fullsized":"2024-02-23T00:54:40.305738+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.563137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.641689+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.563137+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":96,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.781670+0000","last_change":"2024-02-23T00:53:53.424895+0000","last_active":"2024-02-23T00:54:40.781670+0000","last_peered":"2024-02-23T00:54:40.781670+0000","last_clean":"2024-02-23T00:54:40.781670+0000","last_became_active":"2024-02-23T00:53:53.419653+0000","last_became_peered":"2024-02-23T00:53:53.419653+0000","last_unstale":"2024-02-23T00:54:40.781670+0000","last_undegraded":"2024-02-23T00:54:40.781670+0000","last_fullsized":"2024-02-23T00:54:40.781670+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:28.443176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:53:28.443176+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"616'30","reported_seq":759,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.305689+0000","last_change":"2024-02-23T00:53:53.426870+0000","last_active":"2024-02-23T00:54:40.305689+0000","last_peered":"2024-02-23T00:54:40.305689+0000","last_clean":"2024-02-23T00:54:40.305689+0000","last_became_active":"2024-02-23T00:53:53.405043+0000","last_became_peered":"2024-02-23T00:53:53.405043+0000","last_unstale":"2024-02-23T00:54:40.305689+0000","last_undegraded":"2024-02-23T00:54:40.305689+0000","last_fullsized":"2024-02-23T00:54:40.305689+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"616'30","last_scrub_stamp":"2024-02-23T00:52:25.764389+0000","last_deep_scrub":"616'30","last_deep_scrub_stamp":"2024-02-23T00:52:25.764389+0000","last_clean_scrub_stamp":"2024-02-23T00:52:25.764389+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":46,"num_read_kb":36,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":93,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.303239+0000","last_change":"2024-02-23T00:53:53.424319+0000","last_active":"2024-02-23T00:54:40.303239+0000","last_peered":"2024-02-23T00:54:40.303239+0000","last_clean":"2024-02-23T00:54:40.303239+0000","last_became_active":"2024-02-23T00:53:53.423915+0000","last_became_peered":"2024-02-23T00:53:53.423915+0000","last_unstale":"2024-02-23T00:54:40.303239+0000","last_undegraded":"2024-02-23T00:54:40.303239+0000","last_fullsized":"2024-02-23T00:54:40.303239+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:20.774550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:20.774550+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":65,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.781360+0000","last_change":"2024-02-23T00:53:58.483987+0000","last_active":"2024-02-23T00:54:40.781360+0000","last_peered":"2024-02-23T00:54:40.781360+0000","last_clean":"2024-02-23T00:54:40.781360+0000","last_became_active":"2024-02-23T00:53:45.291586+0000","last_became_peered":"2024-02-23T00:53:45.291586+0000","last_unstale":"2024-02-23T00:54:40.781360+0000","last_undegraded":"2024-02-23T00:54:40.781360+0000","last_fullsized":"2024-02-23T00:54:40.781360+0000","mapping_epoch":685,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":686,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:58.483860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_clean_scrub_stamp":"2024-02-23T00:53:58.483860+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":662,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.784400+0000","last_change":"2024-02-23T00:53:30.271476+0000","last_active":"2024-02-23T00:54:40.784400+0000","last_peered":"2024-02-23T00:54:40.784400+0000","last_clean":"2024-02-23T00:54:40.784400+0000","last_became_active":"2024-02-23T00:47:28.203555+0000","last_became_peered":"2024-02-23T00:47:28.203555+0000","last_unstale":"2024-02-23T00:54:40.784400+0000","last_undegraded":"2024-02-23T00:54:40.784400+0000","last_fullsized":"2024-02-23T00:54:40.784400+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:30.271410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:53:30.271410+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"457'33","reported_seq":797,"reported_epoch":739,"state":"active+clean","last_fresh":"2024-02-23T00:54:39.389804+0000","last_change":"2024-02-23T00:53:13.369812+0000","last_active":"2024-02-23T00:54:39.389804+0000","last_peered":"2024-02-23T00:54:39.389804+0000","last_clean":"2024-02-23T00:54:39.389804+0000","last_became_active":"2024-02-23T00:53:13.347848+0000","last_became_peered":"2024-02-23T00:53:13.347848+0000","last_unstale":"2024-02-23T00:54:39.389804+0000","last_undegraded":"2024-02-23T00:54:39.389804+0000","last_fullsized":"2024-02-23T00:54:39.389804+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'33","last_scrub_stamp":"2024-02-23T00:52:11.695335+0000","last_deep_scrub":"95'27","last_deep_scrub_stamp":"2024-02-23T00:41:55.919895+0000","last_clean_scrub_stamp":"2024-02-23T00:52:11.695335+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":724,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.303330+0000","last_change":"2024-02-23T00:53:05.309846+0000","last_active":"2024-02-23T00:54:40.303330+0000","last_peered":"2024-02-23T00:54:40.303330+0000","last_clean":"2024-02-23T00:54:40.303330+0000","last_became_active":"2024-02-23T00:47:28.208293+0000","last_became_peered":"2024-02-23T00:47:28.208293+0000","last_unstale":"2024-02-23T00:54:40.303330+0000","last_undegraded":"2024-02-23T00:54:40.303330+0000","last_fullsized":"2024-02-23T00:54:40.303330+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:05.309734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:53:05.309734+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":717,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.784310+0000","last_change":"2024-02-23T00:53:13.358900+0000","last_active":"2024-02-23T00:54:40.784310+0000","last_peered":"2024-02-23T00:54:40.784310+0000","last_clean":"2024-02-23T00:54:40.784310+0000","last_became_active":"2024-02-23T00:53:13.358230+0000","last_became_peered":"2024-02-23T00:53:13.358230+0000","last_unstale":"2024-02-23T00:54:40.784310+0000","last_undegraded":"2024-02-23T00:54:40.784310+0000","last_fullsized":"2024-02-23T00:54:40.784310+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:11.328122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:52:11.328122+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":662,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.784308+0000","last_change":"2024-02-23T00:53:13.351154+0000","last_active":"2024-02-23T00:54:40.784308+0000","last_peered":"2024-02-23T00:54:40.784308+0000","last_clean":"2024-02-23T00:54:40.784308+0000","last_became_active":"2024-02-23T00:53:13.350854+0000","last_became_peered":"2024-02-23T00:53:13.350854+0000","last_unstale":"2024-02-23T00:54:40.784308+0000","last_undegraded":"2024-02-23T00:54:40.784308+0000","last_fullsized":"2024-02-23T00:54:40.784308+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:50.469857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:52:50.469857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"457'17","reported_seq":750,"reported_epoch":739,"state":"active+clean","last_fresh":"2024-02-23T00:54:39.390933+0000","last_change":"2024-02-23T00:54:11.404685+0000","last_active":"2024-02-23T00:54:39.390933+0000","last_peered":"2024-02-23T00:54:39.390933+0000","last_clean":"2024-02-23T00:54:39.390933+0000","last_became_active":"2024-02-23T00:39:16.782762+0000","last_became_peered":"2024-02-23T00:39:16.782762+0000","last_unstale":"2024-02-23T00:54:39.390933+0000","last_undegraded":"2024-02-23T00:54:39.390933+0000","last_fullsized":"2024-02-23T00:54:39.390933+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'17","last_scrub_stamp":"2024-02-23T00:54:11.404593+0000","last_deep_scrub":"82'15","last_deep_scrub_stamp":"2024-02-23T00:41:50.975683+0000","last_clean_scrub_stamp":"2024-02-23T00:54:11.404593+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":21,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":724,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.303588+0000","last_change":"2024-02-23T00:53:13.352457+0000","last_active":"2024-02-23T00:54:40.303588+0000","last_peered":"2024-02-23T00:54:40.303588+0000","last_clean":"2024-02-23T00:54:40.303588+0000","last_became_active":"2024-02-23T00:53:13.351527+0000","last_became_peered":"2024-02-23T00:53:13.351527+0000","last_unstale":"2024-02-23T00:54:40.303588+0000","last_undegraded":"2024-02-23T00:54:40.303588+0000","last_fullsized":"2024-02-23T00:54:40.303588+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:52.335390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:52.335390+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":713,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.302983+0000","last_change":"2024-02-23T00:52:56.397935+0000","last_active":"2024-02-23T00:54:40.302983+0000","last_peered":"2024-02-23T00:54:40.302983+0000","last_clean":"2024-02-23T00:54:40.302983+0000","last_became_active":"2024-02-23T00:39:18.796952+0000","last_became_peered":"2024-02-23T00:39:18.796952+0000","last_unstale":"2024-02-23T00:54:40.302983+0000","last_undegraded":"2024-02-23T00:54:40.302983+0000","last_fullsized":"2024-02-23T00:54:40.302983+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:56.397846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:06.491793+0000","last_clean_scrub_stamp":"2024-02-23T00:52:56.397846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":675,"reported_epoch":739,"state":"active+clean","last_fresh":"2024-02-23T00:54:39.390453+0000","last_change":"2024-02-23T00:53:53.417926+0000","last_active":"2024-02-23T00:54:39.390453+0000","last_peered":"2024-02-23T00:54:39.390453+0000","last_clean":"2024-02-23T00:54:39.390453+0000","last_became_active":"2024-02-23T00:53:53.417585+0000","last_became_peered":"2024-02-23T00:53:53.417585+0000","last_unstale":"2024-02-23T00:54:39.390453+0000","last_undegraded":"2024-02-23T00:54:39.390453+0000","last_fullsized":"2024-02-23T00:54:39.390453+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:55.788444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:23.224663+0000","last_clean_scrub_stamp":"2024-02-23T00:52:55.788444+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"457'21","reported_seq":756,"reported_epoch":739,"state":"active+clean","last_fresh":"2024-02-23T00:54:39.389826+0000","last_change":"2024-02-23T00:53:57.428696+0000","last_active":"2024-02-23T00:54:39.389826+0000","last_peered":"2024-02-23T00:54:39.389826+0000","last_clean":"2024-02-23T00:54:39.389826+0000","last_became_active":"2024-02-23T00:39:16.784606+0000","last_became_peered":"2024-02-23T00:39:16.784606+0000","last_unstale":"2024-02-23T00:54:39.389826+0000","last_undegraded":"2024-02-23T00:54:39.389826+0000","last_fullsized":"2024-02-23T00:54:39.389826+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-23T00:53:57.428626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:53:57.428626+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":711,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.784163+0000","last_change":"2024-02-23T00:53:34.320659+0000","last_active":"2024-02-23T00:54:40.784163+0000","last_peered":"2024-02-23T00:54:40.784163+0000","last_clean":"2024-02-23T00:54:40.784163+0000","last_became_active":"2024-02-23T00:53:34.320264+0000","last_became_peered":"2024-02-23T00:53:34.320264+0000","last_unstale":"2024-02-23T00:54:40.784163+0000","last_undegraded":"2024-02-23T00:54:40.784163+0000","last_fullsized":"2024-02-23T00:54:40.784163+0000","mapping_epoch":676,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":677,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:17.778217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:17.778217+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":700,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.304343+0000","last_change":"2024-02-23T00:53:02.566710+0000","last_active":"2024-02-23T00:54:40.304343+0000","last_peered":"2024-02-23T00:54:40.304343+0000","last_clean":"2024-02-23T00:54:40.304343+0000","last_became_active":"2024-02-23T00:39:18.796312+0000","last_became_peered":"2024-02-23T00:39:18.796312+0000","last_unstale":"2024-02-23T00:54:40.304343+0000","last_undegraded":"2024-02-23T00:54:40.304343+0000","last_fullsized":"2024-02-23T00:54:40.304343+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:02.566644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:47.226331+0000","last_clean_scrub_stamp":"2024-02-23T00:53:02.566644+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":98,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.782435+0000","last_change":"2024-02-23T00:54:39.499071+0000","last_active":"2024-02-23T00:54:40.782435+0000","last_peered":"2024-02-23T00:54:40.782435+0000","last_clean":"2024-02-23T00:54:40.782435+0000","last_became_active":"2024-02-23T00:53:13.349167+0000","last_became_peered":"2024-02-23T00:53:13.349167+0000","last_unstale":"2024-02-23T00:54:40.782435+0000","last_undegraded":"2024-02-23T00:54:40.782435+0000","last_fullsized":"2024-02-23T00:54:40.782435+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:39.498999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:50:28.479512+0000","last_clean_scrub_stamp":"2024-02-23T00:54:39.498999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"457'62","reported_seq":710,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.783334+0000","last_change":"2024-02-23T00:53:53.441103+0000","last_active":"2024-02-23T00:54:40.783334+0000","last_peered":"2024-02-23T00:54:40.783334+0000","last_clean":"2024-02-23T00:54:40.783334+0000","last_became_active":"2024-02-23T00:53:53.411502+0000","last_became_peered":"2024-02-23T00:53:53.411502+0000","last_unstale":"2024-02-23T00:54:40.783334+0000","last_undegraded":"2024-02-23T00:54:40.783334+0000","last_fullsized":"2024-02-23T00:54:40.783334+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'62","last_scrub_stamp":"2024-02-23T00:52:58.949119+0000","last_deep_scrub":"88'50","last_deep_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_clean_scrub_stamp":"2024-02-23T00:52:58.949119+0000","log_size":62,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":91,"num_read_kb":78,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":95,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.304119+0000","last_change":"2024-02-23T00:53:13.351352+0000","last_active":"2024-02-23T00:54:40.304119+0000","last_peered":"2024-02-23T00:54:40.304119+0000","last_clean":"2024-02-23T00:54:40.304119+0000","last_became_active":"2024-02-23T00:53:13.350934+0000","last_became_peered":"2024-02-23T00:53:13.350934+0000","last_unstale":"2024-02-23T00:54:40.304119+0000","last_undegraded":"2024-02-23T00:54:40.304119+0000","last_fullsized":"2024-02-23T00:54:40.304119+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:59.831412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_clean_scrub_stamp":"2024-02-23T00:52:59.831412+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":709,"reported_epoch":739,"state":"active+clean","last_fresh":"2024-02-23T00:54:39.390381+0000","last_change":"2024-02-23T00:53:35.476706+0000","last_active":"2024-02-23T00:54:39.390381+0000","last_peered":"2024-02-23T00:54:39.390381+0000","last_clean":"2024-02-23T00:54:39.390381+0000","last_became_active":"2024-02-23T00:39:18.799723+0000","last_became_peered":"2024-02-23T00:39:18.799723+0000","last_unstale":"2024-02-23T00:54:39.390381+0000","last_undegraded":"2024-02-23T00:54:39.390381+0000","last_fullsized":"2024-02-23T00:54:39.390381+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:35.476621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:53:35.476621+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":662,"reported_epoch":740,"state":"active+clean","last_fresh":"2024-02-23T00:54:40.784119+0000","last_change":"2024-02-23T00:53:13.358051+0000","last_active":"2024-02-23T00:54:40.784119+0000","last_peered":"2024-02-23T00:54:40.784119+0000","last_clean":"2024-02-23T00:54:40.784119+0000","last_became_active":"2024-02-23T00:53:13.357060+0000","last_became_peered":"2024-02-23T00:53:13.357060+0000","last_unstale":"2024-02-23T00:54:40.784119+0000","last_undegraded":"2024-02-23T00:54:40.784119+0000","last_fullsized":"2024-02-23T00:54:40.784119+0000","mapping_epoch":655,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":656,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:20.418240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:44.601086+0000","last_clean_scrub_stamp":"2024-02-23T00:52:20.418240+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":249,"num_pg":32,"stat_sum":{"num_bytes":8192,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":24576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":96,"acting":96,"num_store_stats":6},{"poolid":234,"num_pg":8,"stat_sum":{"num_bytes":4000089,"num_objects":17,"num_object_clones":1,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":15,"num_read_kb":4,"num_write":37,"num_write_kb":3939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":48,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6213632,"data_stored":6213632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":24,"acting":24,"num_store_stats":7},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":12,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1133,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":36,"acting":36,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80,"ondisk_log_size":80,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1497,"num_read_kb":1222,"num_write":895,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":5502,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":512000,"data_stored":13554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":963,"ondisk_log_size":963,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":916,"num_read_kb":911,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":965,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234173,"num_pgs":98,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":306280,"kb_used_data":4960,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93438872,"statfs":{"total":95995035648,"available":95681404928,"internally_reserved":0,"allocated":5079040,"data_stored":3395435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.50600000000000001,"15min":0.52500000000000002},"min":{"1min":0.38400000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.70399999999999996,"5min":0.83899999999999997,"15min":4.9420000000000002},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.52900000000000003,"15min":0.52400000000000002},"min":{"1min":0.42999999999999999,"5min":0.309,"15min":0.24299999999999999},"max":{"1min":0.70199999999999996,"5min":0.82599999999999996,"15min":4.9729999999999999},"last":0.53200000000000003}]},{"osd":1,"last update":"Fri Feb 23 00:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.50900000000000001,"15min":0.60599999999999998},"min":{"1min":0.38400000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.69199999999999995,"5min":0.79700000000000004,"15min":10.786},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55300000000000005,"15min":0.621},"min":{"1min":0.33400000000000002,"5min":0.32500000000000001,"15min":0.28100000000000003},"max":{"1min":0.68300000000000005,"5min":2.4199999999999999,"15min":11.885999999999999},"last":0.61199999999999999}]},{"osd":2,"last update":"Fri Feb 23 00:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.51700000000000002,"15min":0.54600000000000004},"min":{"1min":0.42799999999999999,"5min":0.29999999999999999,"15min":0.28999999999999998},"max":{"1min":0.68400000000000005,"5min":0.80600000000000005,"15min":5.0789999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53400000000000003,"15min":0.54800000000000004},"min":{"1min":0.40300000000000002,"5min":0.33200000000000002,"15min":0.316},"max":{"1min":0.71599999999999997,"5min":1.786,"15min":5.1280000000000001},"last":0.51000000000000001}]},{"osd":3,"last update":"Fri Feb 23 00:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54400000000000004,"15min":0.625},"min":{"1min":0.41399999999999998,"5min":0.33100000000000002,"15min":0.29399999999999998},"max":{"1min":0.71299999999999997,"5min":2.536,"15min":15.712999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58999999999999997,"15min":0.622},"min":{"1min":0.443,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.3700000000000001,"5min":3.7000000000000002,"15min":9.4090000000000007},"last":0.63}]},{"osd":4,"last update":"Fri Feb 23 00:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42499999999999999,"15min":0.437},"min":{"1min":0.29499999999999998,"5min":0.25800000000000001,"15min":0.23499999999999999},"max":{"1min":0.67400000000000004,"5min":0.753,"15min":2.0350000000000001},"last":0.371},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45200000000000001,"15min":0.48899999999999999},"min":{"1min":0.34999999999999998,"5min":0.26700000000000002,"15min":0.248},"max":{"1min":0.73499999999999999,"5min":0.85299999999999998,"15min":11.236000000000001},"last":0.47199999999999998}]},{"osd":5,"last update":"Fri Feb 23 00:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.46100000000000002,"15min":0.48699999999999999},"min":{"1min":0.28799999999999998,"5min":0.27500000000000002,"15min":0.27000000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":5.4000000000000004},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.498,"15min":0.503},"min":{"1min":0.32900000000000001,"5min":0.29199999999999998,"15min":0.26700000000000002},"max":{"1min":0.60399999999999998,"5min":0.79000000000000004,"15min":1.728},"last":0.45700000000000002}]},{"osd":6,"last update":"Fri Feb 23 00:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.503,"15min":0.51100000000000001},"min":{"1min":0.36399999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.67200000000000004,"5min":0.75700000000000001,"15min":1.6339999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.50600000000000001,"15min":0.54800000000000004},"min":{"1min":0.316,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73999999999999999,"5min":0.82499999999999996,"15min":13.395},"last":0.54800000000000004}]}]},{"osd":6,"up_from":47,"seq":201863463108,"num_pgs":94,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":316620,"kb_used_data":3012,"kb_used_omap":0,"kb_used_meta":313600,"kb_avail":93428532,"statfs":{"total":95995035648,"available":95670816768,"internally_reserved":0,"allocated":3084288,"data_stored":1395780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321126400},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:54:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52300000000000002,"15min":0.67400000000000004},"min":{"1min":0.36199999999999999,"5min":0.22500000000000001,"15min":0.21099999999999999},"max":{"1min":0.81799999999999995,"5min":0.93799999999999994,"15min":30.641999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57399999999999995,"15min":0.70099999999999996},"min":{"1min":0.29899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.268,"5min":4.3609999999999998,"15min":29.292000000000002},"last":0.56799999999999995}]},{"osd":1,"last update":"Fri Feb 23 00:54:09 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.55900000000000005,"15min":0.68999999999999995},"min":{"1min":0.308,"5min":0.27100000000000002,"15min":0.247},"max":{"1min":0.79100000000000004,"5min":4.1980000000000004,"15min":28.186},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.61799999999999999,"15min":0.877},"min":{"1min":0.41299999999999998,"5min":0.34499999999999997,"15min":0.26900000000000002},"max":{"1min":1.2529999999999999,"5min":6.4370000000000003,"15min":33.104999999999997},"last":0.59699999999999998}]},{"osd":2,"last update":"Fri Feb 23 00:54:09 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.60099999999999998,"15min":0.72599999999999998},"min":{"1min":0.376,"5min":0.35299999999999998,"15min":0.27400000000000002},"max":{"1min":1.893,"5min":4.0919999999999996,"15min":19.960000000000001},"last":0.501},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52400000000000002,"15min":0.63800000000000001},"min":{"1min":0.35499999999999998,"5min":0.32500000000000001,"15min":0.29899999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":8.5860000000000003},"last":0.55700000000000005}]},{"osd":3,"last update":"Fri Feb 23 00:54:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.63300000000000001},"min":{"1min":0.39400000000000002,"5min":0.30399999999999999,"15min":0.28599999999999998},"max":{"1min":1.236,"5min":4.1340000000000003,"15min":19.562000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.57899999999999996,"15min":0.66900000000000004},"min":{"1min":0.375,"5min":0.35699999999999998,"15min":0.33000000000000002},"max":{"1min":0.75600000000000001,"5min":3.7839999999999998,"15min":14.199},"last":0.64800000000000002}]},{"osd":4,"last update":"Fri Feb 23 00:54:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.42399999999999999,"15min":0.48299999999999998},"min":{"1min":0.313,"5min":0.23200000000000001,"15min":0.218},"max":{"1min":0.54600000000000004,"5min":0.63800000000000001,"15min":11.109999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.436,"15min":0.54700000000000004},"min":{"1min":0.29799999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.56699999999999995,"5min":0.751,"15min":16.734000000000002},"last":0.35799999999999998}]},{"osd":5,"last update":"Fri Feb 23 00:54:09 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.45000000000000001,"15min":0.51900000000000002},"min":{"1min":0.311,"5min":0.308,"15min":0.249},"max":{"1min":0.54800000000000004,"5min":0.72399999999999998,"15min":14.154999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.46899999999999997,"15min":0.55800000000000005},"min":{"1min":0.32400000000000001,"5min":0.29699999999999999,"15min":0.26400000000000001},"max":{"1min":0.61099999999999999,"5min":0.73899999999999999,"15min":8.1400000000000006},"last":0.40300000000000002}]},{"osd":7,"last update":"Fri Feb 23 00:54:32 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47999999999999998,"15min":0.52600000000000002},"min":{"1min":0.34399999999999997,"5min":0.27400000000000002,"15min":0.26800000000000002},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":8.548},"last":0.438},{"interface":"front","average":{"1min":0.498,"5min":0.51700000000000002,"15min":0.60399999999999998},"min":{"1min":0.33600000000000002,"5min":0.32000000000000001,"15min":0.309},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":15.618},"last":0.51900000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575070,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":300264,"kb_used_data":3040,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93444888,"statfs":{"total":95995035648,"available":95687565312,"internally_reserved":0,"allocated":3112960,"data_stored":1390535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:53:59 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.48299999999999998,"15min":0.70899999999999996},"min":{"1min":0.24399999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.624,"5min":7.0540000000000003,"15min":78.361000000000004},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.44,"15min":0.46500000000000002},"min":{"1min":0.26300000000000001,"5min":0.222,"15min":0.191},"max":{"1min":0.80600000000000005,"5min":5.3109999999999999,"15min":5.8849999999999998},"last":0.25600000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:54:20 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.52300000000000002,"15min":0.68600000000000005},"min":{"1min":0.25800000000000001,"5min":0.22700000000000001,"15min":0.188},"max":{"1min":0.60899999999999999,"5min":7.5789999999999997,"15min":48.606000000000002},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.47499999999999998,"15min":0.58899999999999997},"min":{"1min":0.29399999999999998,"5min":0.28000000000000003,"15min":0.23699999999999999},"max":{"1min":0.95199999999999996,"5min":5.2279999999999998,"15min":23.911000000000001},"last":0.32500000000000001}]},{"osd":3,"last update":"Fri Feb 23 00:53:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48199999999999998,"15min":0.60099999999999998},"min":{"1min":0.34300000000000003,"5min":0.25700000000000001,"15min":0.21199999999999999},"max":{"1min":1.01,"5min":5.2800000000000002,"15min":35.244999999999997},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55600000000000005,"15min":0.70099999999999996},"min":{"1min":0.27700000000000002,"5min":0.26900000000000002,"15min":0.248},"max":{"1min":2.2349999999999999,"5min":7.641,"15min":56.853999999999999},"last":0.372}]},{"osd":4,"last update":"Fri Feb 23 00:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.66300000000000003,"15min":0.82099999999999995},"min":{"1min":0.35899999999999999,"5min":0.32400000000000001,"15min":0.29299999999999998},"max":{"1min":0.84099999999999997,"5min":7.6829999999999998,"15min":37.646999999999998},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.71799999999999997,"15min":0.72799999999999998},"min":{"1min":0.39300000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.90500000000000003,"5min":6.9160000000000004,"15min":9.6859999999999999},"last":0.45000000000000001}]},{"osd":5,"last update":"Fri Feb 23 00:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.68000000000000005,"15min":0.82299999999999995},"min":{"1min":0.38700000000000001,"5min":0.35399999999999998,"15min":0.309},"max":{"1min":0.78400000000000003,"5min":7.742,"15min":48.792000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64100000000000001,"15min":0.69499999999999995},"min":{"1min":0.45600000000000002,"5min":0.39100000000000001,"15min":0.34100000000000003},"max":{"1min":0.998,"5min":0.998,"15min":11.872999999999999},"last":0.55400000000000005}]},{"osd":6,"last update":"Fri Feb 23 00:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.67400000000000004,"15min":0.81599999999999995},"min":{"1min":0.40500000000000003,"5min":0.37,"15min":0.318},"max":{"1min":0.87,"5min":7.7130000000000001,"15min":40.741},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61599999999999999,"15min":0.66400000000000003},"min":{"1min":0.46500000000000002,"5min":0.40999999999999998,"15min":0.34000000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":6.0069999999999997},"last":0.495}]},{"osd":7,"last update":"Fri Feb 23 00:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63100000000000001,"15min":0.71399999999999997},"min":{"1min":0.496,"5min":0.41799999999999998,"15min":0.36699999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":20.225000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.749,"15min":0.76500000000000001},"min":{"1min":0.48999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.88300000000000001,"5min":6.8220000000000001,"15min":20.254999999999999},"last":0.503}]}]},{"osd":0,"up_from":0,"seq":34359738596,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":0,"seq":77309411545,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":26,"seq":111669149906,"num_pgs":92,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":306304,"kb_used_data":4984,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93438848,"statfs":{"total":95995035648,"available":95681380352,"internally_reserved":0,"allocated":5103616,"data_stored":3403619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.434,"15min":0.52100000000000002},"min":{"1min":0.25800000000000001,"5min":0.216,"15min":0.20899999999999999},"max":{"1min":0.63,"5min":8.3300000000000001,"15min":12.212999999999999},"last":0.38},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.42999999999999999,"15min":0.59799999999999998},"min":{"1min":0.28699999999999998,"5min":0.22800000000000001,"15min":0.21199999999999999},"max":{"1min":1.3089999999999999,"5min":2.9740000000000002,"15min":11.714},"last":0.35999999999999999}]},{"osd":1,"last update":"Fri Feb 23 00:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41599999999999998,"15min":0.61299999999999999},"min":{"1min":0.28499999999999998,"5min":0.25600000000000001,"15min":0.19900000000000001},"max":{"1min":0.54900000000000004,"5min":3.056,"15min":11.742000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.44500000000000001,"15min":0.59799999999999998},"min":{"1min":0.218,"5min":0.218,"15min":0.19900000000000001},"max":{"1min":1.7470000000000001,"5min":2.7709999999999999,"15min":8.2270000000000003},"last":0.29399999999999998}]},{"osd":2,"last update":"Fri Feb 23 00:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.48599999999999999,"15min":0.67700000000000005},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.255},"max":{"1min":1.2430000000000001,"5min":3.242,"15min":13.359999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.49299999999999999,"15min":0.65300000000000002},"min":{"1min":0.27600000000000002,"5min":0.27500000000000002,"15min":0.26100000000000001},"max":{"1min":0.70399999999999996,"5min":8.5820000000000007,"15min":21.122},"last":0.53800000000000003}]},{"osd":4,"last update":"Fri Feb 23 00:54:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59799999999999998,"15min":0.751},"min":{"1min":0.39800000000000002,"5min":0.36099999999999999,"15min":0.29899999999999999},"max":{"1min":0.97599999999999998,"5min":3.1419999999999999,"15min":11.644},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61599999999999999,"15min":0.75600000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.32700000000000001},"max":{"1min":1.2969999999999999,"5min":2.9020000000000001,"15min":12.430999999999999},"last":0.73299999999999998}]},{"osd":5,"last update":"Fri Feb 23 00:53:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.68100000000000005,"15min":0.68600000000000005},"min":{"1min":0.39300000000000002,"5min":0.36799999999999999,"15min":0.35099999999999998},"max":{"1min":0.84899999999999998,"5min":10.231999999999999,"15min":10.231999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63200000000000001,"15min":0.79700000000000004},"min":{"1min":0.40300000000000002,"5min":0.376,"15min":0.35499999999999998},"max":{"1min":0.92800000000000005,"5min":3.1829999999999998,"15min":11.648},"last":0.66600000000000004}]},{"osd":6,"last update":"Fri Feb 23 00:54:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64100000000000001,"15min":0.75800000000000001},"min":{"1min":0.39300000000000002,"5min":0.377,"15min":0.35199999999999998},"max":{"1min":1.3400000000000001,"5min":2.9510000000000001,"15min":6.9420000000000002},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.73499999999999999,"15min":0.72299999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.89500000000000002,"5min":10.965999999999999,"15min":10.965999999999999},"last":0.747}]},{"osd":7,"last update":"Fri Feb 23 00:54:38 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.66300000000000003,"15min":0.83199999999999996},"min":{"1min":0.432,"5min":0.40999999999999998,"15min":0.38800000000000001},"max":{"1min":0.90600000000000003,"5min":3.1030000000000002,"15min":11.676},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67600000000000005,"15min":0.81000000000000005},"min":{"1min":0.42299999999999999,"5min":0.41499999999999998,"15min":0.41299999999999998},"max":{"1min":1.375,"5min":2.8580000000000001,"15min":7.9800000000000004},"last":0.76500000000000001}]}]},{"osd":4,"up_from":32,"seq":137438953678,"num_pgs":88,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":302192,"kb_used_data":4968,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93442960,"statfs":{"total":95995035648,"available":95685591040,"internally_reserved":0,"allocated":5087232,"data_stored":3399536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.51600000000000001,"15min":0.51000000000000001},"min":{"1min":0.33100000000000002,"5min":0.28999999999999998,"15min":0.223},"max":{"1min":0.75800000000000001,"5min":1.3819999999999999,"15min":3.5219999999999998},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.56299999999999994,"15min":0.58199999999999996},"min":{"1min":0.32900000000000001,"5min":0.29799999999999999,"15min":0.23999999999999999},"max":{"1min":2.5859999999999999,"5min":2.5859999999999999,"15min":8.6140000000000008},"last":0.57999999999999996}]},{"osd":1,"last update":"Fri Feb 23 00:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53200000000000003,"15min":0.57199999999999995},"min":{"1min":0.36699999999999999,"5min":0.31900000000000001,"15min":0.27300000000000002},"max":{"1min":1.23,"5min":1.23,"15min":9.6110000000000007},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.504,"5min":0.55100000000000005,"15min":0.57399999999999995},"min":{"1min":0.34200000000000003,"5min":0.31900000000000001,"15min":0.255},"max":{"1min":0.71299999999999997,"5min":2.6920000000000002,"15min":6.6950000000000003},"last":0.441}]},{"osd":2,"last update":"Fri Feb 23 00:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.54700000000000004,"15min":0.56699999999999995},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.26500000000000001},"max":{"1min":0.78600000000000003,"5min":2.9209999999999998,"15min":3.3069999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.373,"5min":0.33700000000000002,"15min":0.32500000000000001},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":5.8440000000000003},"last":0.51500000000000001}]},{"osd":3,"last update":"Fri Feb 23 00:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67800000000000005,"15min":0.64000000000000001},"min":{"1min":0.38100000000000001,"5min":0.35699999999999998,"15min":0.316},"max":{"1min":2.6150000000000002,"5min":9.1280000000000001,"15min":9.1280000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56899999999999995,"15min":0.60099999999999998},"min":{"1min":0.39700000000000002,"5min":0.36099999999999999,"15min":0.29699999999999999},"max":{"1min":1.288,"5min":2.1699999999999999,"15min":4.1909999999999998},"last":0.53000000000000003}]},{"osd":5,"last update":"Fri Feb 23 00:53:44 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.46600000000000003,"15min":0.433},"min":{"1min":0.29199999999999998,"5min":0.26700000000000002,"15min":0.24399999999999999},"max":{"1min":0.66000000000000003,"5min":1.694,"15min":1.694},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.50700000000000001,"15min":0.48799999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.63200000000000001,"5min":1.663,"15min":1.663},"last":0.55300000000000005}]},{"osd":6,"last update":"Fri Feb 23 00:54:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.46100000000000002,"15min":0.44800000000000001},"min":{"1min":0.309,"5min":0.29099999999999998,"15min":0.253},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.49199999999999999,"15min":0.46200000000000002},"min":{"1min":0.30399999999999999,"5min":0.28599999999999998,"15min":0.215},"max":{"1min":1.264,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":0.59499999999999997}]},{"osd":7,"last update":"Fri Feb 23 00:54:36 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.51300000000000001,"15min":0.48799999999999999},"min":{"1min":0.32400000000000001,"5min":0.30599999999999999,"15min":0.254},"max":{"1min":1.363,"5min":2.294,"15min":2.294},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.58699999999999997,"15min":0.55200000000000005},"min":{"1min":0.39200000000000002,"5min":0.37,"15min":0.32600000000000001},"max":{"1min":1.726,"5min":2.282,"15min":2.282},"last":0.63800000000000001}]}]},{"osd":5,"up_from":39,"seq":167503724745,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":312540,"kb_used_data":3028,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93432612,"statfs":{"total":95995035648,"available":95674994688,"internally_reserved":0,"allocated":3100672,"data_stored":1400290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:54:33 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.56399999999999995,"15min":0.61699999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.246},"max":{"1min":0.74399999999999999,"5min":5.2220000000000004,"15min":23.881},"last":1.595},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.53900000000000003,"15min":0.58599999999999997},"min":{"1min":0.38400000000000001,"5min":0.29099999999999998,"15min":0.27000000000000002},"max":{"1min":0.63200000000000001,"5min":3.9249999999999998,"15min":5.4649999999999999},"last":0.83299999999999996}]},{"osd":1,"last update":"Fri Feb 23 00:54:33 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.55300000000000005,"15min":0.65600000000000003},"min":{"1min":0.379,"5min":0.35299999999999998,"15min":0.254},"max":{"1min":0.68700000000000006,"5min":3.9460000000000002,"15min":18.175999999999998},"last":1.659},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.54100000000000004,"15min":0.621},"min":{"1min":0.41899999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.63600000000000001,"5min":3.976,"15min":9.0950000000000006},"last":0.76300000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:54:33 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.60299999999999998,"15min":0.71799999999999997},"min":{"1min":0.40300000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.72599999999999998,"5min":4.5899999999999999,"15min":22.282},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.60599999999999998,"15min":0.73499999999999999},"min":{"1min":0.375,"5min":0.34399999999999997,"15min":0.29199999999999998},"max":{"1min":0.65400000000000003,"5min":3.8540000000000001,"15min":23.925000000000001},"last":1.633}]},{"osd":3,"last update":"Fri Feb 23 00:54:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.54600000000000004,"15min":0.622},"min":{"1min":0.40200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.71499999999999997,"5min":1.194,"15min":6.5730000000000004},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.61799999999999999,"15min":0.66300000000000003},"min":{"1min":0.34999999999999998,"5min":0.34699999999999998,"15min":0.33500000000000002},"max":{"1min":0.72099999999999997,"5min":3.9980000000000002,"15min":8.1760000000000002},"last":1.675}]},{"osd":4,"last update":"Fri Feb 23 00:54:33 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.435,"15min":0.44700000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.23499999999999999},"max":{"1min":0.57099999999999995,"5min":0.86099999999999999,"15min":4.593},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.47899999999999998,"15min":0.49099999999999999},"min":{"1min":0.33000000000000002,"5min":0.27900000000000003,"15min":0.27500000000000002},"max":{"1min":0.63700000000000001,"5min":1.169,"15min":3.879},"last":1.744}]},{"osd":6,"last update":"Fri Feb 23 00:54:02 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.47899999999999998,"15min":0.48699999999999999},"min":{"1min":0.34300000000000003,"5min":0.30199999999999999,"15min":0.27900000000000003},"max":{"1min":0.67100000000000004,"5min":1.1299999999999999,"15min":4.6740000000000004},"last":1.5389999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.27100000000000002},"max":{"1min":0.74299999999999999,"5min":0.79900000000000004,"15min":1.159},"last":1.7090000000000001}]},{"osd":7,"last update":"Fri Feb 23 00:54:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54500000000000004,"15min":0.55200000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.30399999999999999},"max":{"1min":0.70099999999999996,"5min":1.1870000000000001,"15min":7.0300000000000002},"last":1.756},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.51400000000000001,"15min":0.54000000000000004},"min":{"1min":0.35999999999999999,"5min":0.35699999999999998,"15min":0.29099999999999998},"max":{"1min":0.61899999999999999,"5min":0.73599999999999999,"15min":9.0579999999999998},"last":0.80100000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":2530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":20480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":16384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2031616,"data_stored":2031616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":2027520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":45056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":40960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2031616,"data_stored":2031616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":249,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":249,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":249,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":249,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":249,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":249,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-23T00:54:47.949 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-02-23T00:54:47.949 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-23T00:54:47.949 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-02-23T00:54:47.949 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd in 0 2024-02-23T00:54:48.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:47.379157+0000 osd.1 (osd.1) 293 : cluster [DBG] 2.9 scrub starts 2024-02-23T00:54:48.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:47.382050+0000 osd.1 (osd.1) 294 : cluster [DBG] 2.9 scrub ok 2024-02-23T00:54:48.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:47.390347+0000 mgr.y (mgr.14517) 635 : cluster [DBG] pgmap v1126: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.9 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 8.0 KiB/s wr, 2 op/s; 1/732 objects misplaced (0.137%); 18 B/s, 0 objects/s recovering 2024-02-23T00:54:48.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:47.411674+0000 mon.a (mon.0) 3583 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:54:48.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:47.418841+0000 mon.a (mon.0) 3584 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:54:48.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:47.418912+0000 mon.a (mon.0) 3585 : cluster [DBG] osdmap e744: 8 total, 8 up, 6 in 2024-02-23T00:54:48.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:47.379157+0000 osd.1 (osd.1) 293 : cluster [DBG] 2.9 scrub starts 2024-02-23T00:54:48.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:47.382050+0000 osd.1 (osd.1) 294 : cluster [DBG] 2.9 scrub ok 2024-02-23T00:54:48.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:47.390347+0000 mgr.y (mgr.14517) 635 : cluster [DBG] pgmap v1126: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.9 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 8.0 KiB/s wr, 2 op/s; 1/732 objects misplaced (0.137%); 18 B/s, 0 objects/s recovering 2024-02-23T00:54:48.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:47.411674+0000 mon.a (mon.0) 3583 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:54:48.560 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:47.418841+0000 mon.a (mon.0) 3584 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:54:48.560 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:47.418912+0000 mon.a (mon.0) 3585 : cluster [DBG] osdmap e744: 8 total, 8 up, 6 in 2024-02-23T00:54:48.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:47.379157+0000 osd.1 (osd.1) 293 : cluster [DBG] 2.9 scrub starts 2024-02-23T00:54:48.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:47.382050+0000 osd.1 (osd.1) 294 : cluster [DBG] 2.9 scrub ok 2024-02-23T00:54:48.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:47.390347+0000 mgr.y (mgr.14517) 635 : cluster [DBG] pgmap v1126: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.9 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 8.0 KiB/s wr, 2 op/s; 1/732 objects misplaced (0.137%); 18 B/s, 0 objects/s recovering 2024-02-23T00:54:48.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:47.411674+0000 mon.a (mon.0) 3583 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:54:48.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:47.418841+0000 mon.a (mon.0) 3584 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:54:48.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:47.418912+0000 mon.a (mon.0) 3585 : cluster [DBG] osdmap e744: 8 total, 8 up, 6 in 2024-02-23T00:54:48.724 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:54:48.727 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:48.727 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:54:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:54:48.828 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:54:48.857 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:49.006 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:54:49.009 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:49.010 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:54:49.111 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:54:49.139 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:49.236 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:54:49.239 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:49.240 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:54:49.341 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:54:49.371 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:54:49.524 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:48.039774+0000 osd.7 (osd.7) 337 : cluster [DBG] 6.8 scrub starts 2024-02-23T00:54:49.524 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:48.043988+0000 osd.7 (osd.7) 338 : cluster [DBG] 6.8 scrub ok 2024-02-23T00:54:49.524 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:48.449385+0000 osd.4 (osd.4) 309 : cluster [DBG] 4.1a scrub starts 2024-02-23T00:54:49.524 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:48.450980+0000 osd.4 (osd.4) 310 : cluster [DBG] 4.1a scrub ok 2024-02-23T00:54:49.524 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:48.039774+0000 osd.7 (osd.7) 337 : cluster [DBG] 6.8 scrub starts 2024-02-23T00:54:49.524 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:48.043988+0000 osd.7 (osd.7) 338 : cluster [DBG] 6.8 scrub ok 2024-02-23T00:54:49.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:48.449385+0000 osd.4 (osd.4) 309 : cluster [DBG] 4.1a scrub starts 2024-02-23T00:54:49.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:48.450980+0000 osd.4 (osd.4) 310 : cluster [DBG] 4.1a scrub ok 2024-02-23T00:54:49.525 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:49.526 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:54:49.626 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:54:49.658 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:49.680 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:54:49.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:48.039774+0000 osd.7 (osd.7) 337 : cluster [DBG] 6.8 scrub starts 2024-02-23T00:54:49.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:48.043988+0000 osd.7 (osd.7) 338 : cluster [DBG] 6.8 scrub ok 2024-02-23T00:54:49.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:48.449385+0000 osd.4 (osd.4) 309 : cluster [DBG] 4.1a scrub starts 2024-02-23T00:54:49.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:48.450980+0000 osd.4 (osd.4) 310 : cluster [DBG] 4.1a scrub ok 2024-02-23T00:54:49.683 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:49.683 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:54:49.784 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:54:49.812 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:49.918 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:54:49.919 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:54:49.936 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:54:49.940 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:49.941 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:54:50.042 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:54:50.074 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:50.095 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:54:50.098 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:50.098 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:54:50.199 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:54:50.228 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:50.248 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:54:50.251 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:50.252 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:54:50.352 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:54:50.381 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:50.678 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set noscrub 2024-02-23T00:54:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:49.041677+0000 osd.7 (osd.7) 339 : cluster [DBG] 5.14 scrub starts 2024-02-23T00:54:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:49.045247+0000 osd.7 (osd.7) 340 : cluster [DBG] 5.14 scrub ok 2024-02-23T00:54:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:49.370541+0000 osd.1 (osd.1) 295 : cluster [DBG] 2.4 scrub starts 2024-02-23T00:54:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:49.373451+0000 osd.1 (osd.1) 296 : cluster [DBG] 2.4 scrub ok 2024-02-23T00:54:50.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:49.390717+0000 mgr.y (mgr.14517) 636 : cluster [DBG] pgmap v1128: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.9 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 6.8 KiB/s wr, 1 op/s; 1/732 objects misplaced (0.137%); 18 B/s, 0 objects/s recovering 2024-02-23T00:54:50.705 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:49.041677+0000 osd.7 (osd.7) 339 : cluster [DBG] 5.14 scrub starts 2024-02-23T00:54:50.705 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:49.045247+0000 osd.7 (osd.7) 340 : cluster [DBG] 5.14 scrub ok 2024-02-23T00:54:50.705 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:49.370541+0000 osd.1 (osd.1) 295 : cluster [DBG] 2.4 scrub starts 2024-02-23T00:54:50.705 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:49.373451+0000 osd.1 (osd.1) 296 : cluster [DBG] 2.4 scrub ok 2024-02-23T00:54:50.705 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:49.390717+0000 mgr.y (mgr.14517) 636 : cluster [DBG] pgmap v1128: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.9 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 6.8 KiB/s wr, 1 op/s; 1/732 objects misplaced (0.137%); 18 B/s, 0 objects/s recovering 2024-02-23T00:54:50.705 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:54:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:54:50] "GET /metrics HTTP/1.1" 200 191648 "" "Prometheus/2.33.4" 2024-02-23T00:54:50.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:49.041677+0000 osd.7 (osd.7) 339 : cluster [DBG] 5.14 scrub starts 2024-02-23T00:54:50.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:49.045247+0000 osd.7 (osd.7) 340 : cluster [DBG] 5.14 scrub ok 2024-02-23T00:54:50.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:49.370541+0000 osd.1 (osd.1) 295 : cluster [DBG] 2.4 scrub starts 2024-02-23T00:54:50.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:49.373451+0000 osd.1 (osd.1) 296 : cluster [DBG] 2.4 scrub ok 2024-02-23T00:54:50.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:49.390717+0000 mgr.y (mgr.14517) 636 : cluster [DBG] pgmap v1128: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.9 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 6.8 KiB/s wr, 1 op/s; 1/732 objects misplaced (0.137%); 18 B/s, 0 objects/s recovering 2024-02-23T00:54:51.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:50.393815+0000 osd.1 (osd.1) 297 : cluster [DBG] 5.0 scrub starts 2024-02-23T00:54:51.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:50.395360+0000 osd.1 (osd.1) 298 : cluster [DBG] 5.0 scrub ok 2024-02-23T00:54:51.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:50.553582+0000 osd.6 (osd.6) 289 : cluster [DBG] 2.1a scrub starts 2024-02-23T00:54:51.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:50.554616+0000 osd.6 (osd.6) 290 : cluster [DBG] 2.1a scrub ok 2024-02-23T00:54:51.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:50.393815+0000 osd.1 (osd.1) 297 : cluster [DBG] 5.0 scrub starts 2024-02-23T00:54:51.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:50.395360+0000 osd.1 (osd.1) 298 : cluster [DBG] 5.0 scrub ok 2024-02-23T00:54:51.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:50.553582+0000 osd.6 (osd.6) 289 : cluster [DBG] 2.1a scrub starts 2024-02-23T00:54:51.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:50.554616+0000 osd.6 (osd.6) 290 : cluster [DBG] 2.1a scrub ok 2024-02-23T00:54:51.633 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:54:51.635 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:51.639 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:54:51.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:50.393815+0000 osd.1 (osd.1) 297 : cluster [DBG] 5.0 scrub starts 2024-02-23T00:54:51.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:50.395360+0000 osd.1 (osd.1) 298 : cluster [DBG] 5.0 scrub ok 2024-02-23T00:54:51.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:50.553582+0000 osd.6 (osd.6) 289 : cluster [DBG] 2.1a scrub starts 2024-02-23T00:54:51.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:50.554616+0000 osd.6 (osd.6) 290 : cluster [DBG] 2.1a scrub ok 2024-02-23T00:54:51.740 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:54:51.769 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:51.892 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:54:51.895 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:51.896 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:54:51.996 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:54:52.027 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:52.100 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:54:52.104 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:52.104 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:54:52.205 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:54:52.234 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:52.331 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:54:52.331 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-02-23T00:54:52.331 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-02-23T00:54:52.331 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-02-23T00:54:52.331 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:54:52.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:51.391189+0000 mgr.y (mgr.14517) 637 : cluster [DBG] pgmap v1129: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.9 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 732 B/s rd, 5.9 KiB/s wr, 1 op/s; 1/732 objects misplaced (0.137%); 15 B/s, 0 objects/s recovering 2024-02-23T00:54:52.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:51.399025+0000 osd.4 (osd.4) 311 : cluster [DBG] 3.1 scrub starts 2024-02-23T00:54:52.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:51.399933+0000 osd.4 (osd.4) 312 : cluster [DBG] 3.1 scrub ok 2024-02-23T00:54:52.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:51.587282+0000 osd.6 (osd.6) 291 : cluster [DBG] 5.1f scrub starts 2024-02-23T00:54:52.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:51.588567+0000 osd.6 (osd.6) 292 : cluster [DBG] 5.1f scrub ok 2024-02-23T00:54:52.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:51.391189+0000 mgr.y (mgr.14517) 637 : cluster [DBG] pgmap v1129: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.9 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 732 B/s rd, 5.9 KiB/s wr, 1 op/s; 1/732 objects misplaced (0.137%); 15 B/s, 0 objects/s recovering 2024-02-23T00:54:52.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:51.399025+0000 osd.4 (osd.4) 311 : cluster [DBG] 3.1 scrub starts 2024-02-23T00:54:52.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:51.399933+0000 osd.4 (osd.4) 312 : cluster [DBG] 3.1 scrub ok 2024-02-23T00:54:52.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:51.587282+0000 osd.6 (osd.6) 291 : cluster [DBG] 5.1f scrub starts 2024-02-23T00:54:52.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:51.588567+0000 osd.6 (osd.6) 292 : cluster [DBG] 5.1f scrub ok 2024-02-23T00:54:52.663 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:54:52.666 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:52.666 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:54:52.694 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:54:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:54:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:54:52.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:51.391189+0000 mgr.y (mgr.14517) 637 : cluster [DBG] pgmap v1129: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.9 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 732 B/s rd, 5.9 KiB/s wr, 1 op/s; 1/732 objects misplaced (0.137%); 15 B/s, 0 objects/s recovering 2024-02-23T00:54:52.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:51.399025+0000 osd.4 (osd.4) 311 : cluster [DBG] 3.1 scrub starts 2024-02-23T00:54:52.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:51.399933+0000 osd.4 (osd.4) 312 : cluster [DBG] 3.1 scrub ok 2024-02-23T00:54:52.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:51.587282+0000 osd.6 (osd.6) 291 : cluster [DBG] 5.1f scrub starts 2024-02-23T00:54:52.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:51.588567+0000 osd.6 (osd.6) 292 : cluster [DBG] 5.1f scrub ok 2024-02-23T00:54:52.767 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:54:52.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.529+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/2733392512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb280f7400 msgr2=0x7fdb28103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:52.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.529+0000 7fdb2ff9a700 1 --2- 172.21.15.47:0/2733392512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb280f7400 0x7fdb28103a20 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 rx=0x7fdb20009be0 tx=0x7fdb20004d10).stop 2024-02-23T00:54:52.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.529+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/2733392512 shutdown_connections 2024-02-23T00:54:52.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.529+0000 7fdb2ff9a700 1 --2- 172.21.15.47:0/2733392512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb280f7400 0x7fdb28103a20 unknown :-1 s=CLOSED pgs=2630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:52.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.529+0000 7fdb2ff9a700 1 --2- 172.21.15.47:0/2733392512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb280fcbb0 0x7fdb280fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:52.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.529+0000 7fdb2ff9a700 1 --2- 172.21.15.47:0/2733392512 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdb280fd510 0x7fdb280f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:52.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.529+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/2733392512 >> 172.21.15.47:0/2733392512 conn(0x7fdb280f2820 msgr2=0x7fdb280f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:52.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.529+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/2733392512 shutdown_connections 2024-02-23T00:54:52.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.529+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/2733392512 wait complete. 2024-02-23T00:54:52.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.529+0000 7fdb2ff9a700 1 librados: starting msgr at 2024-02-23T00:54:52.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.529+0000 7fdb2ff9a700 1 librados: starting objecter 2024-02-23T00:54:52.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.529+0000 7fdb2ff9a700 1 Processor -- start 2024-02-23T00:54:52.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.529+0000 7fdb2ff9a700 1 -- start start 2024-02-23T00:54:52.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.529+0000 7fdb2ff9a700 1 librados: setting wanted keys 2024-02-23T00:54:52.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.529+0000 7fdb2ff9a700 1 librados: calling monclient init 2024-02-23T00:54:52.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.530+0000 7fdb2ff9a700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdb280f7400 0x7fdb28108ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:52.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.530+0000 7fdb2ff9a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb280fcbb0 0x7fdb28109390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:52.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.530+0000 7fdb2ff9a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb280fd510 0x7fdb28109970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:52.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.530+0000 7fdb2ff9a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb28114080 con 0x7fdb280fd510 2024-02-23T00:54:52.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.530+0000 7fdb2ff9a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb281141f0 con 0x7fdb280fcbb0 2024-02-23T00:54:52.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.530+0000 7fdb2ff9a700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb281143b0 con 0x7fdb280f7400 2024-02-23T00:54:52.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.530+0000 7fdb2dd36700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdb280f7400 0x7fdb28108ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:52.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.530+0000 7fdb2e537700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb280fd510 0x7fdb28109970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:52.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.530+0000 7fdb2dd36700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdb280f7400 0x7fdb28108ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:42788/0 (socket says 172.21.15.47:42788) 2024-02-23T00:54:52.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.530+0000 7fdb2dd36700 1 -- 172.21.15.47:0/866708700 learned_addr learned my addr 172.21.15.47:0/866708700 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:54:52.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.530+0000 7fdb2d535700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb280fcbb0 0x7fdb28109390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:52.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.530+0000 7fdb2e537700 1 -- 172.21.15.47:0/866708700 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdb280f7400 msgr2=0x7fdb28108ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:52.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.530+0000 7fdb2e537700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdb280f7400 0x7fdb28108ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:52.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.530+0000 7fdb2e537700 1 -- 172.21.15.47:0/866708700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb280fcbb0 msgr2=0x7fdb28109390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:52.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.530+0000 7fdb2e537700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb280fcbb0 0x7fdb28109390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:52.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.530+0000 7fdb2e537700 1 -- 172.21.15.47:0/866708700 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb18005040 con 0x7fdb280fd510 2024-02-23T00:54:52.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.530+0000 7fdb2dd36700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdb280f7400 0x7fdb28108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:54:52.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.530+0000 7fdb2e537700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb280fd510 0x7fdb28109970 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 rx=0x7fdb20009bb0 tx=0x7fdb20004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:52.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.530+0000 7fdb1effd700 10 client.?.objecter ms_handle_connect 0x7fdb280fd510 2024-02-23T00:54:52.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.530+0000 7fdb1effd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:54:52.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.530+0000 7fdb1effd700 1 -- 172.21.15.47:0/866708700 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2000e040 con 0x7fdb280fd510 2024-02-23T00:54:52.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.530+0000 7fdb1effd700 1 -- 172.21.15.47:0/866708700 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fdb20018040 con 0x7fdb280fd510 2024-02-23T00:54:52.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.530+0000 7fdb1effd700 1 -- 172.21.15.47:0/866708700 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb20013d50 con 0x7fdb280fd510 2024-02-23T00:54:52.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.530+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb200097c0 con 0x7fdb280fd510 2024-02-23T00:54:52.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.530+0000 7fdb2ff9a700 10 client.31737.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:54:52.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.530+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb281148a0 con 0x7fdb280fd510 2024-02-23T00:54:52.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.530+0000 7fdb2ff9a700 1 librados: init done 2024-02-23T00:54:52.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.531+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fdb2805fa60 con 0x7fdb280fd510 2024-02-23T00:54:52.778 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:54:52.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.532+0000 7fdb1effd700 1 -- 172.21.15.47:0/866708700 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fdb20013420 con 0x7fdb280fd510 2024-02-23T00:54:52.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.532+0000 7fdb1effd700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdb14062cc0 0x7fdb14065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:52.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.532+0000 7fdb1effd700 1 -- 172.21.15.47:0/866708700 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(744..744 src has 621..744) v4 ==== 9807+0+0 (secure 0 0 0) 0x7fdb2007eb90 con 0x7fdb280fd510 2024-02-23T00:54:52.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.532+0000 7fdb1effd700 10 client.31737.objecter ms_dispatch 0x7fdb28001550 osd_map(744..744 src has 621..744) v4 2024-02-23T00:54:52.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.532+0000 7fdb1effd700 3 client.31737.objecter handle_osd_map got epochs [744,744] > 0 2024-02-23T00:54:52.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.532+0000 7fdb1effd700 3 client.31737.objecter handle_osd_map decoding full epoch 744 2024-02-23T00:54:52.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.532+0000 7fdb2dd36700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdb14062cc0 0x7fdb14065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:52.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.532+0000 7fdb1effd700 20 client.31737.objecter dump_active .. 0 homeless 2024-02-23T00:54:52.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.532+0000 7fdb1effd700 1 -- 172.21.15.47:0/866708700 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v744) v1 ==== 54+0+16 (secure 0 0 0) 0x7fdb2007f090 con 0x7fdb280fd510 2024-02-23T00:54:52.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.533+0000 7fdb1effd700 10 client.31737.objecter ms_handle_connect 0x7fdb14062cc0 2024-02-23T00:54:52.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.533+0000 7fdb2dd36700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdb14062cc0 0x7fdb14065180 secure :-1 s=READY pgs=3193 cs=0 l=1 rev1=1 rx=0x7fdb24004fd0 tx=0x7fdb2400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:52.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.533+0000 7fdb2ff9a700 10 client.31737.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:52.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.533+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_assign -1 1 2024-02-23T00:54:52.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.533+0000 7fdb2ff9a700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fdb280fdea0 0x7fdb280fe280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:52.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.533+0000 7fdb2ff9a700 20 client.31737.objecter _get_session s=0x7fdb2805ee80 osd=0 3 2024-02-23T00:54:52.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.533+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb2805ee80 osd=0 3 2024-02-23T00:54:52.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.533+0000 7fdb2ff9a700 20 client.31737.objecter _get_session s=0x7fdb2805ee80 osd=0 3 2024-02-23T00:54:52.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.533+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_remove -1 1 2024-02-23T00:54:52.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.533+0000 7fdb2ff9a700 20 client.31737.objecter get_session s=0x7fdb2805ee80 osd=0 3 2024-02-23T00:54:52.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.533+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_assign 0 1 2024-02-23T00:54:52.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.533+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb2805ee80 osd=0 4 2024-02-23T00:54:52.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.533+0000 7fdb2ff9a700 10 client.31737.objecter _send_command 1 2024-02-23T00:54:52.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.533+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb28105ee0 con 0x7fdb280fdea0 2024-02-23T00:54:52.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.533+0000 7fdb2d535700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fdb280fdea0 0x7fdb280fe280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:52.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.533+0000 7fdb1effd700 10 client.31737.objecter ms_handle_connect 0x7fdb280fdea0 2024-02-23T00:54:52.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.533+0000 7fdb2d535700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fdb280fdea0 0x7fdb280fe280 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:52.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.534+0000 7fdb1effd700 1 -- 172.21.15.47:0/866708700 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fdb28105ee0 con 0x7fdb280fdea0 2024-02-23T00:54:52.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.534+0000 7fdb1effd700 10 client.31737.objecter ms_dispatch 0x7fdb28001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:54:52.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.534+0000 7fdb1effd700 10 client.31737.objecter _finish_command 1 = system:0 2024-02-23T00:54:52.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.534+0000 7fdb1effd700 20 client.31737.objecter put_session s=0x7fdb2805ee80 osd=0 4 2024-02-23T00:54:52.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.534+0000 7fdb1effd700 15 client.31737.objecter _session_command_op_remove 0 1 2024-02-23T00:54:52.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.549+0000 7fdb2ff9a700 10 client.31737.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:54:52.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.549+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_assign -1 2 2024-02-23T00:54:52.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.549+0000 7fdb2ff9a700 20 client.31737.objecter _get_session s=0x7fdb2805ee80 osd=0 3 2024-02-23T00:54:52.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.549+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb2805ee80 osd=0 3 2024-02-23T00:54:52.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.549+0000 7fdb2ff9a700 20 client.31737.objecter _get_session s=0x7fdb2805ee80 osd=0 3 2024-02-23T00:54:52.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.549+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_remove -1 2 2024-02-23T00:54:52.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.549+0000 7fdb2ff9a700 20 client.31737.objecter get_session s=0x7fdb2805ee80 osd=0 3 2024-02-23T00:54:52.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.549+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_assign 0 2 2024-02-23T00:54:52.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.549+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb2805ee80 osd=0 4 2024-02-23T00:54:52.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.549+0000 7fdb2ff9a700 10 client.31737.objecter _send_command 2 2024-02-23T00:54:52.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.549+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fdb280fe9b0 con 0x7fdb280fdea0 2024-02-23T00:54:52.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.551+0000 7fdb1effd700 1 -- 172.21.15.47:0/866708700 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fdb1800f040 con 0x7fdb280fdea0 2024-02-23T00:54:52.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.551+0000 7fdb1effd700 10 client.31737.objecter ms_dispatch 0x7fdb28001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:54:52.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.551+0000 7fdb1effd700 10 client.31737.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:54:52.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.551+0000 7fdb1effd700 20 client.31737.objecter put_session s=0x7fdb2805ee80 osd=0 4 2024-02-23T00:54:52.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.551+0000 7fdb1effd700 15 client.31737.objecter _session_command_op_remove 0 2 2024-02-23T00:54:52.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.551+0000 7fdb2ff9a700 10 client.31737.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:52.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.551+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_assign -1 3 2024-02-23T00:54:52.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.551+0000 7fdb2ff9a700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fdb280598f0 0x7fdb28059d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:52.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.551+0000 7fdb2ff9a700 20 client.31737.objecter _get_session s=0x7fdb28028540 osd=1 3 2024-02-23T00:54:52.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.551+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb28028540 osd=1 3 2024-02-23T00:54:52.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.551+0000 7fdb2ff9a700 20 client.31737.objecter _get_session s=0x7fdb28028540 osd=1 3 2024-02-23T00:54:52.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.551+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_remove -1 3 2024-02-23T00:54:52.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.551+0000 7fdb2ff9a700 20 client.31737.objecter get_session s=0x7fdb28028540 osd=1 3 2024-02-23T00:54:52.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.551+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_assign 1 3 2024-02-23T00:54:52.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.551+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb28028540 osd=1 4 2024-02-23T00:54:52.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.551+0000 7fdb2ff9a700 10 client.31737.objecter _send_command 3 2024-02-23T00:54:52.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.551+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb280286b0 con 0x7fdb280598f0 2024-02-23T00:54:52.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.551+0000 7fdb2dd36700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fdb280598f0 0x7fdb28059d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:52.783 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:54:52.783 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:54:52.783 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:54:52.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.551+0000 7fdb1effd700 10 client.31737.objecter ms_handle_connect 0x7fdb280598f0 2024-02-23T00:54:52.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.551+0000 7fdb2dd36700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fdb280598f0 0x7fdb28059d30 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:52.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.552+0000 7fdb1effd700 1 -- 172.21.15.47:0/866708700 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fdb280286b0 con 0x7fdb280598f0 2024-02-23T00:54:52.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.552+0000 7fdb1effd700 10 client.31737.objecter ms_dispatch 0x7fdb28001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:54:52.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.552+0000 7fdb1effd700 10 client.31737.objecter _finish_command 3 = system:0 2024-02-23T00:54:52.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.552+0000 7fdb1effd700 20 client.31737.objecter put_session s=0x7fdb28028540 osd=1 4 2024-02-23T00:54:52.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.552+0000 7fdb1effd700 15 client.31737.objecter _session_command_op_remove 1 3 2024-02-23T00:54:52.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.567+0000 7fdb2ff9a700 10 client.31737.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:54:52.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.567+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_assign -1 4 2024-02-23T00:54:52.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.567+0000 7fdb2ff9a700 20 client.31737.objecter _get_session s=0x7fdb28028540 osd=1 3 2024-02-23T00:54:52.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.567+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb28028540 osd=1 3 2024-02-23T00:54:52.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.567+0000 7fdb2ff9a700 20 client.31737.objecter _get_session s=0x7fdb28028540 osd=1 3 2024-02-23T00:54:52.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.567+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_remove -1 4 2024-02-23T00:54:52.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.567+0000 7fdb2ff9a700 20 client.31737.objecter get_session s=0x7fdb28028540 osd=1 3 2024-02-23T00:54:52.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.567+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_assign 1 4 2024-02-23T00:54:52.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.567+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb28028540 osd=1 4 2024-02-23T00:54:52.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.567+0000 7fdb2ff9a700 10 client.31737.objecter _send_command 4 2024-02-23T00:54:52.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.567+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fdb28059360 con 0x7fdb280598f0 2024-02-23T00:54:52.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.569+0000 7fdb1effd700 1 -- 172.21.15.47:0/866708700 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fdb28059360 con 0x7fdb280598f0 2024-02-23T00:54:52.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.569+0000 7fdb1effd700 10 client.31737.objecter ms_dispatch 0x7fdb28001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:54:52.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.569+0000 7fdb1effd700 10 client.31737.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:54:52.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.569+0000 7fdb1effd700 20 client.31737.objecter put_session s=0x7fdb28028540 osd=1 4 2024-02-23T00:54:52.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.569+0000 7fdb1effd700 15 client.31737.objecter _session_command_op_remove 1 4 2024-02-23T00:54:52.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.569+0000 7fdb2ff9a700 10 client.31737.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:52.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.569+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_assign -1 5 2024-02-23T00:54:52.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.569+0000 7fdb2ff9a700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fdb28119c60 0x7fdb2811c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:52.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.569+0000 7fdb2ff9a700 20 client.31737.objecter _get_session s=0x7fdb28027b90 osd=2 3 2024-02-23T00:54:52.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.569+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb28027b90 osd=2 3 2024-02-23T00:54:52.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.569+0000 7fdb2ff9a700 20 client.31737.objecter _get_session s=0x7fdb28027b90 osd=2 3 2024-02-23T00:54:52.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.569+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_remove -1 5 2024-02-23T00:54:52.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.569+0000 7fdb2ff9a700 20 client.31737.objecter get_session s=0x7fdb28027b90 osd=2 3 2024-02-23T00:54:52.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.569+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_assign 2 5 2024-02-23T00:54:52.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.569+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb28027b90 osd=2 4 2024-02-23T00:54:52.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.569+0000 7fdb2ff9a700 10 client.31737.objecter _send_command 5 2024-02-23T00:54:52.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.569+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb280282a0 con 0x7fdb28119c60 2024-02-23T00:54:52.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.569+0000 7fdb2d535700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fdb28119c60 0x7fdb2811c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:52.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.569+0000 7fdb2d535700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fdb28119c60 0x7fdb2811c050 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:52.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.569+0000 7fdb1effd700 10 client.31737.objecter ms_handle_connect 0x7fdb28119c60 2024-02-23T00:54:52.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.570+0000 7fdb1effd700 1 -- 172.21.15.47:0/866708700 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fdb280282a0 con 0x7fdb28119c60 2024-02-23T00:54:52.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.570+0000 7fdb1effd700 10 client.31737.objecter ms_dispatch 0x7fdb28001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:54:52.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.570+0000 7fdb1effd700 10 client.31737.objecter _finish_command 5 = system:0 2024-02-23T00:54:52.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.570+0000 7fdb1effd700 20 client.31737.objecter put_session s=0x7fdb28027b90 osd=2 4 2024-02-23T00:54:52.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.570+0000 7fdb1effd700 15 client.31737.objecter _session_command_op_remove 2 5 2024-02-23T00:54:52.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.584+0000 7fdb2ff9a700 10 client.31737.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:54:52.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.584+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_assign -1 6 2024-02-23T00:54:52.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.584+0000 7fdb2ff9a700 20 client.31737.objecter _get_session s=0x7fdb28027b90 osd=2 3 2024-02-23T00:54:52.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.584+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb28027b90 osd=2 3 2024-02-23T00:54:52.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.584+0000 7fdb2ff9a700 20 client.31737.objecter _get_session s=0x7fdb28027b90 osd=2 3 2024-02-23T00:54:52.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.584+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_remove -1 6 2024-02-23T00:54:52.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.584+0000 7fdb2ff9a700 20 client.31737.objecter get_session s=0x7fdb28027b90 osd=2 3 2024-02-23T00:54:52.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.584+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_assign 2 6 2024-02-23T00:54:52.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.584+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb28027b90 osd=2 4 2024-02-23T00:54:52.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.584+0000 7fdb2ff9a700 10 client.31737.objecter _send_command 6 2024-02-23T00:54:52.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.584+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fdb28027600 con 0x7fdb28119c60 2024-02-23T00:54:52.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.586+0000 7fdb1effd700 1 -- 172.21.15.47:0/866708700 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fdb28027600 con 0x7fdb28119c60 2024-02-23T00:54:52.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.586+0000 7fdb1effd700 10 client.31737.objecter ms_dispatch 0x7fdb28001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:54:52.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.586+0000 7fdb1effd700 10 client.31737.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:54:52.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.586+0000 7fdb1effd700 20 client.31737.objecter put_session s=0x7fdb28027b90 osd=2 4 2024-02-23T00:54:52.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.586+0000 7fdb1effd700 15 client.31737.objecter _session_command_op_remove 2 6 2024-02-23T00:54:52.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.586+0000 7fdb2ff9a700 10 client.31737.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:52.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.586+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_assign -1 7 2024-02-23T00:54:52.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.586+0000 7fdb2ff9a700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fdb281197c0 0x7fdb2811f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:52.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.586+0000 7fdb2ff9a700 20 client.31737.objecter _get_session s=0x7fdb28027a10 osd=3 3 2024-02-23T00:54:52.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.586+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb28027a10 osd=3 3 2024-02-23T00:54:52.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.586+0000 7fdb2ff9a700 20 client.31737.objecter _get_session s=0x7fdb28027a10 osd=3 3 2024-02-23T00:54:52.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.586+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_remove -1 7 2024-02-23T00:54:52.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.586+0000 7fdb2ff9a700 20 client.31737.objecter get_session s=0x7fdb28027a10 osd=3 3 2024-02-23T00:54:52.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.586+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_assign 3 7 2024-02-23T00:54:52.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.586+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb28027a10 osd=3 4 2024-02-23T00:54:52.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.586+0000 7fdb2ff9a700 10 client.31737.objecter _send_command 7 2024-02-23T00:54:52.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.586+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb28122450 con 0x7fdb281197c0 2024-02-23T00:54:52.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.586+0000 7fdb2e537700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fdb281197c0 0x7fdb2811f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:52.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.586+0000 7fdb1effd700 10 client.31737.objecter ms_handle_connect 0x7fdb281197c0 2024-02-23T00:54:52.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.586+0000 7fdb2e537700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fdb281197c0 0x7fdb2811f6d0 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:52.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.587+0000 7fdb1effd700 1 -- 172.21.15.47:0/866708700 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fdb28122450 con 0x7fdb281197c0 2024-02-23T00:54:52.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.587+0000 7fdb1effd700 10 client.31737.objecter ms_dispatch 0x7fdb28001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:54:52.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.587+0000 7fdb1effd700 10 client.31737.objecter _finish_command 7 = system:0 2024-02-23T00:54:52.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.587+0000 7fdb1effd700 20 client.31737.objecter put_session s=0x7fdb28027a10 osd=3 4 2024-02-23T00:54:52.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.587+0000 7fdb1effd700 15 client.31737.objecter _session_command_op_remove 3 7 2024-02-23T00:54:52.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.601+0000 7fdb2ff9a700 10 client.31737.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:54:52.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.601+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_assign -1 8 2024-02-23T00:54:52.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.601+0000 7fdb2ff9a700 20 client.31737.objecter _get_session s=0x7fdb28027a10 osd=3 3 2024-02-23T00:54:52.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.601+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb28027a10 osd=3 3 2024-02-23T00:54:52.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.601+0000 7fdb2ff9a700 20 client.31737.objecter _get_session s=0x7fdb28027a10 osd=3 3 2024-02-23T00:54:52.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.601+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_remove -1 8 2024-02-23T00:54:52.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.601+0000 7fdb2ff9a700 20 client.31737.objecter get_session s=0x7fdb28027a10 osd=3 3 2024-02-23T00:54:52.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.601+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_assign 3 8 2024-02-23T00:54:52.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.601+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb28027a10 osd=3 4 2024-02-23T00:54:52.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.601+0000 7fdb2ff9a700 10 client.31737.objecter _send_command 8 2024-02-23T00:54:52.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.601+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fdb28118d20 con 0x7fdb281197c0 2024-02-23T00:54:52.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.603+0000 7fdb1effd700 1 -- 172.21.15.47:0/866708700 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fdb28118d20 con 0x7fdb281197c0 2024-02-23T00:54:52.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.603+0000 7fdb1effd700 10 client.31737.objecter ms_dispatch 0x7fdb28001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:54:52.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.603+0000 7fdb1effd700 10 client.31737.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:54:52.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.603+0000 7fdb1effd700 20 client.31737.objecter put_session s=0x7fdb28027a10 osd=3 4 2024-02-23T00:54:52.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.603+0000 7fdb1effd700 15 client.31737.objecter _session_command_op_remove 3 8 2024-02-23T00:54:52.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.603+0000 7fdb2ff9a700 10 client.31737.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:52.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.603+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_assign -1 9 2024-02-23T00:54:52.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.603+0000 7fdb2ff9a700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fdb2811d220 0x7fdb28121c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:52.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.603+0000 7fdb2ff9a700 20 client.31737.objecter _get_session s=0x7fdb28118d20 osd=4 3 2024-02-23T00:54:52.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.603+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb28118d20 osd=4 3 2024-02-23T00:54:52.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.603+0000 7fdb2ff9a700 20 client.31737.objecter _get_session s=0x7fdb28118d20 osd=4 3 2024-02-23T00:54:52.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.603+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_remove -1 9 2024-02-23T00:54:52.793 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:54:52.793 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:54:52.793 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:54:52.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.603+0000 7fdb2ff9a700 20 client.31737.objecter get_session s=0x7fdb28118d20 osd=4 3 2024-02-23T00:54:52.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.603+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_assign 4 9 2024-02-23T00:54:52.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.603+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb28118d20 osd=4 4 2024-02-23T00:54:52.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.603+0000 7fdb2ff9a700 10 client.31737.objecter _send_command 9 2024-02-23T00:54:52.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.603+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb281221b0 con 0x7fdb2811d220 2024-02-23T00:54:52.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.603+0000 7fdb2dd36700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fdb2811d220 0x7fdb28121c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:52.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.604+0000 7fdb1effd700 10 client.31737.objecter ms_handle_connect 0x7fdb2811d220 2024-02-23T00:54:52.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.603+0000 7fdb2dd36700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fdb2811d220 0x7fdb28121c70 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:52.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.604+0000 7fdb1effd700 1 -- 172.21.15.47:0/866708700 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fdb281221b0 con 0x7fdb2811d220 2024-02-23T00:54:52.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.604+0000 7fdb1effd700 10 client.31737.objecter ms_dispatch 0x7fdb28001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:54:52.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.604+0000 7fdb1effd700 10 client.31737.objecter _finish_command 9 = system:0 2024-02-23T00:54:52.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.604+0000 7fdb1effd700 20 client.31737.objecter put_session s=0x7fdb28118d20 osd=4 4 2024-02-23T00:54:52.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.604+0000 7fdb1effd700 15 client.31737.objecter _session_command_op_remove 4 9 2024-02-23T00:54:52.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.619+0000 7fdb2ff9a700 10 client.31737.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:54:52.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.619+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_assign -1 10 2024-02-23T00:54:52.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.619+0000 7fdb2ff9a700 20 client.31737.objecter _get_session s=0x7fdb28118d20 osd=4 3 2024-02-23T00:54:52.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.619+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb28118d20 osd=4 3 2024-02-23T00:54:52.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.619+0000 7fdb2ff9a700 20 client.31737.objecter _get_session s=0x7fdb28118d20 osd=4 3 2024-02-23T00:54:52.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.619+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_remove -1 10 2024-02-23T00:54:52.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.619+0000 7fdb2ff9a700 20 client.31737.objecter get_session s=0x7fdb28118d20 osd=4 3 2024-02-23T00:54:52.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.619+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_assign 4 10 2024-02-23T00:54:52.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.619+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb28118d20 osd=4 4 2024-02-23T00:54:52.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.619+0000 7fdb2ff9a700 10 client.31737.objecter _send_command 10 2024-02-23T00:54:52.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.619+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fdb281221b0 con 0x7fdb2811d220 2024-02-23T00:54:52.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.621+0000 7fdb1effd700 1 -- 172.21.15.47:0/866708700 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fdb281221b0 con 0x7fdb2811d220 2024-02-23T00:54:52.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.621+0000 7fdb1effd700 10 client.31737.objecter ms_dispatch 0x7fdb28001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:54:52.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.621+0000 7fdb1effd700 10 client.31737.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:54:52.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.621+0000 7fdb1effd700 20 client.31737.objecter put_session s=0x7fdb28118d20 osd=4 4 2024-02-23T00:54:52.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.621+0000 7fdb1effd700 15 client.31737.objecter _session_command_op_remove 4 10 2024-02-23T00:54:52.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.621+0000 7fdb2ff9a700 10 client.31737.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:52.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.621+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_assign -1 11 2024-02-23T00:54:52.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.621+0000 7fdb2ff9a700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fdb28108320 0x7fdb28108760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:52.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.621+0000 7fdb2ff9a700 20 client.31737.objecter _get_session s=0x7fdb281221b0 osd=5 3 2024-02-23T00:54:52.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.621+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb281221b0 osd=5 3 2024-02-23T00:54:52.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.621+0000 7fdb2ff9a700 20 client.31737.objecter _get_session s=0x7fdb281221b0 osd=5 3 2024-02-23T00:54:52.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.621+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_remove -1 11 2024-02-23T00:54:52.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.621+0000 7fdb2ff9a700 20 client.31737.objecter get_session s=0x7fdb281221b0 osd=5 3 2024-02-23T00:54:52.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.621+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_assign 5 11 2024-02-23T00:54:52.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.621+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb281221b0 osd=5 4 2024-02-23T00:54:52.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.621+0000 7fdb2ff9a700 10 client.31737.objecter _send_command 11 2024-02-23T00:54:52.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.621+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb2812b170 con 0x7fdb28108320 2024-02-23T00:54:52.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.622+0000 7fdb2d535700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fdb28108320 0x7fdb28108760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:52.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.622+0000 7fdb1effd700 10 client.31737.objecter ms_handle_connect 0x7fdb28108320 2024-02-23T00:54:52.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.622+0000 7fdb2d535700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fdb28108320 0x7fdb28108760 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:52.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.623+0000 7fdb1effd700 1 -- 172.21.15.47:0/866708700 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fdb2812b170 con 0x7fdb28108320 2024-02-23T00:54:52.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.623+0000 7fdb1effd700 10 client.31737.objecter ms_dispatch 0x7fdb28001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:54:52.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.623+0000 7fdb1effd700 10 client.31737.objecter _finish_command 11 = system:0 2024-02-23T00:54:52.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.623+0000 7fdb1effd700 20 client.31737.objecter put_session s=0x7fdb281221b0 osd=5 4 2024-02-23T00:54:52.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.623+0000 7fdb1effd700 15 client.31737.objecter _session_command_op_remove 5 11 2024-02-23T00:54:52.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.637+0000 7fdb2ff9a700 10 client.31737.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:54:52.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.637+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_assign -1 12 2024-02-23T00:54:52.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.637+0000 7fdb2ff9a700 20 client.31737.objecter _get_session s=0x7fdb281221b0 osd=5 3 2024-02-23T00:54:52.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.637+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb281221b0 osd=5 3 2024-02-23T00:54:52.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.637+0000 7fdb2ff9a700 20 client.31737.objecter _get_session s=0x7fdb281221b0 osd=5 3 2024-02-23T00:54:52.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.637+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_remove -1 12 2024-02-23T00:54:52.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.637+0000 7fdb2ff9a700 20 client.31737.objecter get_session s=0x7fdb281221b0 osd=5 3 2024-02-23T00:54:52.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.637+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_assign 5 12 2024-02-23T00:54:52.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.637+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb281221b0 osd=5 4 2024-02-23T00:54:52.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.637+0000 7fdb2ff9a700 10 client.31737.objecter _send_command 12 2024-02-23T00:54:52.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.637+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fdb28128a10 con 0x7fdb28108320 2024-02-23T00:54:52.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.639+0000 7fdb1effd700 1 -- 172.21.15.47:0/866708700 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fdb28128a10 con 0x7fdb28108320 2024-02-23T00:54:52.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.639+0000 7fdb1effd700 10 client.31737.objecter ms_dispatch 0x7fdb28001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:54:52.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.639+0000 7fdb1effd700 10 client.31737.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:54:52.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.639+0000 7fdb1effd700 20 client.31737.objecter put_session s=0x7fdb281221b0 osd=5 4 2024-02-23T00:54:52.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.639+0000 7fdb1effd700 15 client.31737.objecter _session_command_op_remove 5 12 2024-02-23T00:54:52.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.639+0000 7fdb2ff9a700 10 client.31737.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:52.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.639+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_assign -1 13 2024-02-23T00:54:52.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.639+0000 7fdb2ff9a700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fdb28129120 0x7fdb2812b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:52.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.639+0000 7fdb2ff9a700 20 client.31737.objecter _get_session s=0x7fdb28128a10 osd=6 3 2024-02-23T00:54:52.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.639+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb28128a10 osd=6 3 2024-02-23T00:54:52.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.639+0000 7fdb2ff9a700 20 client.31737.objecter _get_session s=0x7fdb28128a10 osd=6 3 2024-02-23T00:54:52.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.639+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_remove -1 13 2024-02-23T00:54:52.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.639+0000 7fdb2ff9a700 20 client.31737.objecter get_session s=0x7fdb28128a10 osd=6 3 2024-02-23T00:54:52.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.639+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_assign 6 13 2024-02-23T00:54:52.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.639+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb28128a10 osd=6 4 2024-02-23T00:54:52.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.639+0000 7fdb2ff9a700 10 client.31737.objecter _send_command 13 2024-02-23T00:54:52.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.639+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb2812e3d0 con 0x7fdb28129120 2024-02-23T00:54:52.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.640+0000 7fdb2e537700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fdb28129120 0x7fdb2812b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:52.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.640+0000 7fdb1effd700 10 client.31737.objecter ms_handle_connect 0x7fdb28129120 2024-02-23T00:54:52.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.640+0000 7fdb2e537700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fdb28129120 0x7fdb2812b550 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:52.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.641+0000 7fdb1effd700 1 -- 172.21.15.47:0/866708700 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fdb2812e3d0 con 0x7fdb28129120 2024-02-23T00:54:52.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.641+0000 7fdb1effd700 10 client.31737.objecter ms_dispatch 0x7fdb28001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:54:52.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.641+0000 7fdb1effd700 10 client.31737.objecter _finish_command 13 = system:0 2024-02-23T00:54:52.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.641+0000 7fdb1effd700 20 client.31737.objecter put_session s=0x7fdb28128a10 osd=6 4 2024-02-23T00:54:52.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.641+0000 7fdb1effd700 15 client.31737.objecter _session_command_op_remove 6 13 2024-02-23T00:54:52.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.655+0000 7fdb2ff9a700 10 client.31737.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:54:52.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.655+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_assign -1 14 2024-02-23T00:54:52.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.655+0000 7fdb2ff9a700 20 client.31737.objecter _get_session s=0x7fdb28128a10 osd=6 3 2024-02-23T00:54:52.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.655+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb28128a10 osd=6 3 2024-02-23T00:54:52.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.655+0000 7fdb2ff9a700 20 client.31737.objecter _get_session s=0x7fdb28128a10 osd=6 3 2024-02-23T00:54:52.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.655+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_remove -1 14 2024-02-23T00:54:52.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.655+0000 7fdb2ff9a700 20 client.31737.objecter get_session s=0x7fdb28128a10 osd=6 3 2024-02-23T00:54:52.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.655+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_assign 6 14 2024-02-23T00:54:52.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.655+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb28128a10 osd=6 4 2024-02-23T00:54:52.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.655+0000 7fdb2ff9a700 10 client.31737.objecter _send_command 14 2024-02-23T00:54:52.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.655+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fdb2812d160 con 0x7fdb28129120 2024-02-23T00:54:52.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.657+0000 7fdb1effd700 1 -- 172.21.15.47:0/866708700 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fdb2812d160 con 0x7fdb28129120 2024-02-23T00:54:52.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.657+0000 7fdb1effd700 10 client.31737.objecter ms_dispatch 0x7fdb28001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:54:52.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.657+0000 7fdb1effd700 10 client.31737.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:54:52.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.657+0000 7fdb1effd700 20 client.31737.objecter put_session s=0x7fdb28128a10 osd=6 4 2024-02-23T00:54:52.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.657+0000 7fdb1effd700 15 client.31737.objecter _session_command_op_remove 6 14 2024-02-23T00:54:52.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.658+0000 7fdb2ff9a700 10 client.31737.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:54:52.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.658+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_assign -1 15 2024-02-23T00:54:52.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.658+0000 7fdb2ff9a700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fdb28107460 0x7fdb281078a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:52.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.658+0000 7fdb2ff9a700 20 client.31737.objecter _get_session s=0x7fdb28000910 osd=7 3 2024-02-23T00:54:52.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.658+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb28000910 osd=7 3 2024-02-23T00:54:52.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.658+0000 7fdb2ff9a700 20 client.31737.objecter _get_session s=0x7fdb28000910 osd=7 3 2024-02-23T00:54:52.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.658+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_remove -1 15 2024-02-23T00:54:52.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.658+0000 7fdb2ff9a700 20 client.31737.objecter get_session s=0x7fdb28000910 osd=7 3 2024-02-23T00:54:52.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.658+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_assign 7 15 2024-02-23T00:54:52.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.658+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb28000910 osd=7 4 2024-02-23T00:54:52.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.658+0000 7fdb2ff9a700 10 client.31737.objecter _send_command 15 2024-02-23T00:54:52.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.658+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb2812ce30 con 0x7fdb28107460 2024-02-23T00:54:52.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.658+0000 7fdb2dd36700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fdb28107460 0x7fdb281078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:52.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.658+0000 7fdb1effd700 10 client.31737.objecter ms_handle_connect 0x7fdb28107460 2024-02-23T00:54:52.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.658+0000 7fdb2dd36700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fdb28107460 0x7fdb281078a0 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:52.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.659+0000 7fdb1effd700 1 -- 172.21.15.47:0/866708700 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fdb2812ce30 con 0x7fdb28107460 2024-02-23T00:54:52.807 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:54:52.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.659+0000 7fdb1effd700 10 client.31737.objecter ms_dispatch 0x7fdb28001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:54:52.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.659+0000 7fdb1effd700 10 client.31737.objecter _finish_command 15 = system:0 2024-02-23T00:54:52.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.659+0000 7fdb1effd700 20 client.31737.objecter put_session s=0x7fdb28000910 osd=7 4 2024-02-23T00:54:52.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.659+0000 7fdb1effd700 15 client.31737.objecter _session_command_op_remove 7 15 2024-02-23T00:54:52.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.675+0000 7fdb2ff9a700 10 client.31737.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:54:52.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.675+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_assign -1 16 2024-02-23T00:54:52.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.675+0000 7fdb2ff9a700 20 client.31737.objecter _get_session s=0x7fdb28000910 osd=7 3 2024-02-23T00:54:52.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.675+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb28000910 osd=7 3 2024-02-23T00:54:52.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.675+0000 7fdb2ff9a700 20 client.31737.objecter _get_session s=0x7fdb28000910 osd=7 3 2024-02-23T00:54:52.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.675+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_remove -1 16 2024-02-23T00:54:52.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.675+0000 7fdb2ff9a700 20 client.31737.objecter get_session s=0x7fdb28000910 osd=7 3 2024-02-23T00:54:52.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.675+0000 7fdb2ff9a700 15 client.31737.objecter _session_command_op_assign 7 16 2024-02-23T00:54:52.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.675+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb28000910 osd=7 4 2024-02-23T00:54:52.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.675+0000 7fdb2ff9a700 10 client.31737.objecter _send_command 16 2024-02-23T00:54:52.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.675+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fdb2812ce30 con 0x7fdb28107460 2024-02-23T00:54:52.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb1effd700 1 -- 172.21.15.47:0/866708700 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fdb2812ce30 con 0x7fdb28107460 2024-02-23T00:54:52.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb1effd700 10 client.31737.objecter ms_dispatch 0x7fdb28001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:54:52.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb1effd700 10 client.31737.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:54:52.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb1effd700 20 client.31737.objecter put_session s=0x7fdb28000910 osd=7 4 2024-02-23T00:54:52.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb1effd700 15 client.31737.objecter _session_command_op_remove 7 16 2024-02-23T00:54:52.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 10 librados: watch_flush enter 2024-02-23T00:54:52.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 10 librados: watch_flush exit 2024-02-23T00:54:52.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 10 client.31737.objecter close_session for osd.0 2024-02-23T00:54:52.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fdb280fdea0 msgr2=0x7fdb280fe280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:52.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fdb280fdea0 0x7fdb280fe280 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:52.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb2805ee80 osd=0 1 2024-02-23T00:54:52.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 10 client.31737.objecter close_session for osd.1 2024-02-23T00:54:52.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fdb280598f0 msgr2=0x7fdb28059d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:52.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fdb280598f0 0x7fdb28059d30 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:52.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb28028540 osd=1 1 2024-02-23T00:54:52.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 10 client.31737.objecter close_session for osd.2 2024-02-23T00:54:52.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fdb28119c60 msgr2=0x7fdb2811c050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:52.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fdb28119c60 0x7fdb2811c050 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:52.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb28027b90 osd=2 1 2024-02-23T00:54:52.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 10 client.31737.objecter close_session for osd.3 2024-02-23T00:54:52.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fdb281197c0 msgr2=0x7fdb2811f6d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:52.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fdb281197c0 0x7fdb2811f6d0 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:52.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb28027a10 osd=3 1 2024-02-23T00:54:52.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 10 client.31737.objecter close_session for osd.4 2024-02-23T00:54:52.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2e537700 1 -- 172.21.15.47:0/866708700 reap_dead start 2024-02-23T00:54:52.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fdb2811d220 msgr2=0x7fdb28121c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:52.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fdb2811d220 0x7fdb28121c70 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:52.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2e537700 1 -- 172.21.15.47:0/866708700 reap_dead start 2024-02-23T00:54:52.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb28118d20 osd=4 1 2024-02-23T00:54:52.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 10 client.31737.objecter close_session for osd.5 2024-02-23T00:54:52.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fdb28108320 msgr2=0x7fdb28108760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:52.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fdb28108320 0x7fdb28108760 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:52.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb281221b0 osd=5 1 2024-02-23T00:54:52.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 10 client.31737.objecter close_session for osd.6 2024-02-23T00:54:52.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fdb28129120 msgr2=0x7fdb2812b550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:52.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fdb28129120 0x7fdb2812b550 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:52.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb28128a10 osd=6 1 2024-02-23T00:54:52.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 10 client.31737.objecter close_session for osd.7 2024-02-23T00:54:52.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fdb28107460 msgr2=0x7fdb281078a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:52.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fdb28107460 0x7fdb281078a0 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:52.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 20 client.31737.objecter put_session s=0x7fdb28000910 osd=7 1 2024-02-23T00:54:52.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 20 client.31737.objecter shutdown clearing up homeless session... 2024-02-23T00:54:52.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 10 client.31737.objecter successfully canceled tick 2024-02-23T00:54:52.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdb14062cc0 msgr2=0x7fdb14065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:52.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fdb14062cc0 0x7fdb14065180 secure :-1 s=READY pgs=3193 cs=0 l=1 rev1=1 rx=0x7fdb24004fd0 tx=0x7fdb2400b040).stop 2024-02-23T00:54:52.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb280fd510 msgr2=0x7fdb28109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:52.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 1 --2- 172.21.15.47:0/866708700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb280fd510 0x7fdb28109970 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 rx=0x7fdb20009bb0 tx=0x7fdb20004d10).stop 2024-02-23T00:54:52.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2e537700 1 -- 172.21.15.47:0/866708700 reap_dead start 2024-02-23T00:54:52.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 shutdown_connections 2024-02-23T00:54:52.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 >> 172.21.15.47:0/866708700 conn(0x7fdb280f2820 msgr2=0x7fdb280f4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:52.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 shutdown_connections 2024-02-23T00:54:52.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 1 -- 172.21.15.47:0/866708700 wait complete. 2024-02-23T00:54:52.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:52.677+0000 7fdb2ff9a700 1 librados: shutdown 2024-02-23T00:54:52.812 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:54:52.813 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:54:52.813 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:54:52.813 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:54:52.813 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:54:52.813 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:54:52.813 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:54:52.813 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:54:52.813 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:53.436 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:54:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:54:53.437 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:52.357424+0000 osd.1 (osd.1) 299 : cluster [DBG] 2.15 scrub starts 2024-02-23T00:54:53.437 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:52.359109+0000 osd.1 (osd.1) 300 : cluster [DBG] 2.15 scrub ok 2024-02-23T00:54:53.437 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:52.532955+0000 mon.a (mon.0) 3586 : audit [DBG] from='client.? 172.21.15.47:0/866708700' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:54:53.437 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:53.354801+0000 mon.c (mon.2) 451 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:53.437 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:53.355102+0000 mon.a (mon.0) 3587 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:53.437 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:53.355627+0000 mon.c (mon.2) 452 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-130"}]: dispatch 2024-02-23T00:54:53.524 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:54:53.527 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:53.527 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:54:53.628 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:54:53.657 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:53.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:52.357424+0000 osd.1 (osd.1) 299 : cluster [DBG] 2.15 scrub starts 2024-02-23T00:54:53.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:52.359109+0000 osd.1 (osd.1) 300 : cluster [DBG] 2.15 scrub ok 2024-02-23T00:54:53.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:52.532955+0000 mon.a (mon.0) 3586 : audit [DBG] from='client.? 172.21.15.47:0/866708700' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:54:53.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:53.354801+0000 mon.c (mon.2) 451 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:53.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:53.355102+0000 mon.a (mon.0) 3587 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:53.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:53.355627+0000 mon.c (mon.2) 452 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-130"}]: dispatch 2024-02-23T00:54:53.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:53.355881+0000 mon.a (mon.0) 3588 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-130"}]: dispatch 2024-02-23T00:54:53.803 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:53.355881+0000 mon.a (mon.0) 3588 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-130"}]: dispatch 2024-02-23T00:54:53.803 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:52.357424+0000 osd.1 (osd.1) 299 : cluster [DBG] 2.15 scrub starts 2024-02-23T00:54:53.803 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:52.359109+0000 osd.1 (osd.1) 300 : cluster [DBG] 2.15 scrub ok 2024-02-23T00:54:53.803 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:52.532955+0000 mon.a (mon.0) 3586 : audit [DBG] from='client.? 172.21.15.47:0/866708700' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:54:53.803 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:53.354801+0000 mon.c (mon.2) 451 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:53.803 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:53.355102+0000 mon.a (mon.0) 3587 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:54:53.803 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:53.355627+0000 mon.c (mon.2) 452 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-130"}]: dispatch 2024-02-23T00:54:53.804 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:53.355881+0000 mon.a (mon.0) 3588 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-130"}]: dispatch 2024-02-23T00:54:53.805 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:54:53.808 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:53.809 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:54:53.909 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:54:53.938 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:54.138 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:54:54.138 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:54:54.140 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:54.141 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:54:54.241 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:54:54.270 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:54.445 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:52.951845+0000 mgr.y (mgr.14517) 638 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:54:54.445 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:53.020195+0000 osd.7 (osd.7) 341 : cluster [DBG] 2.0 scrub starts 2024-02-23T00:54:54.446 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:53.023443+0000 osd.7 (osd.7) 342 : cluster [DBG] 2.0 scrub ok 2024-02-23T00:54:54.446 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:53.376072+0000 osd.1 (osd.1) 301 : cluster [DBG] 18.4 scrub starts 2024-02-23T00:54:54.446 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:53.379038+0000 osd.1 (osd.1) 302 : cluster [DBG] 18.4 scrub ok 2024-02-23T00:54:54.446 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:53.392106+0000 mgr.y (mgr.14517) 639 : cluster [DBG] pgmap v1130: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.9 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s; 1/738 objects misplaced (0.136%); 13 B/s, 0 objects/s recovering 2024-02-23T00:54:54.446 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:53.437601+0000 mon.a (mon.0) 3589 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-130"}]': finished 2024-02-23T00:54:54.446 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:53.437680+0000 mon.a (mon.0) 3590 : cluster [DBG] osdmap e745: 8 total, 8 up, 6 in 2024-02-23T00:54:54.446 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:53.452118+0000 osd.4 (osd.4) 313 : cluster [DBG] 4.1 scrub starts 2024-02-23T00:54:54.446 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:53.455031+0000 osd.4 (osd.4) 314 : cluster [DBG] 4.1 scrub ok 2024-02-23T00:54:54.446 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:53.582811+0000 osd.6 (osd.6) 293 : cluster [DBG] 2.17 deep-scrub starts 2024-02-23T00:54:54.446 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:53.583707+0000 osd.6 (osd.6) 294 : cluster [DBG] 2.17 deep-scrub ok 2024-02-23T00:54:54.619 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:54:54.622 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:54.623 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.2 dump_historic_ops 2024-02-23T00:54:54.623 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:54:54.725 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:54:54.756 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:52.951845+0000 mgr.y (mgr.14517) 638 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:54:54.756 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:53.020195+0000 osd.7 (osd.7) 341 : cluster [DBG] 2.0 scrub starts 2024-02-23T00:54:54.756 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:53.023443+0000 osd.7 (osd.7) 342 : cluster [DBG] 2.0 scrub ok 2024-02-23T00:54:54.756 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:53.376072+0000 osd.1 (osd.1) 301 : cluster [DBG] 18.4 scrub starts 2024-02-23T00:54:54.756 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:53.379038+0000 osd.1 (osd.1) 302 : cluster [DBG] 18.4 scrub ok 2024-02-23T00:54:54.756 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:53.392106+0000 mgr.y (mgr.14517) 639 : cluster [DBG] pgmap v1130: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.9 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s; 1/738 objects misplaced (0.136%); 13 B/s, 0 objects/s recovering 2024-02-23T00:54:54.756 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:53.437601+0000 mon.a (mon.0) 3589 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-130"}]': finished 2024-02-23T00:54:54.756 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:53.437680+0000 mon.a (mon.0) 3590 : cluster [DBG] osdmap e745: 8 total, 8 up, 6 in 2024-02-23T00:54:54.757 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:53.452118+0000 osd.4 (osd.4) 313 : cluster [DBG] 4.1 scrub starts 2024-02-23T00:54:54.757 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:53.455031+0000 osd.4 (osd.4) 314 : cluster [DBG] 4.1 scrub ok 2024-02-23T00:54:54.757 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:53.582811+0000 osd.6 (osd.6) 293 : cluster [DBG] 2.17 deep-scrub starts 2024-02-23T00:54:54.757 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:53.583707+0000 osd.6 (osd.6) 294 : cluster [DBG] 2.17 deep-scrub ok 2024-02-23T00:54:54.759 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:54.780 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:54:54.782 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:54.783 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:54:54.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:52.951845+0000 mgr.y (mgr.14517) 638 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:54:54.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:53.020195+0000 osd.7 (osd.7) 341 : cluster [DBG] 2.0 scrub starts 2024-02-23T00:54:54.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:53.023443+0000 osd.7 (osd.7) 342 : cluster [DBG] 2.0 scrub ok 2024-02-23T00:54:54.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:53.376072+0000 osd.1 (osd.1) 301 : cluster [DBG] 18.4 scrub starts 2024-02-23T00:54:54.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:53.379038+0000 osd.1 (osd.1) 302 : cluster [DBG] 18.4 scrub ok 2024-02-23T00:54:54.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:53.392106+0000 mgr.y (mgr.14517) 639 : cluster [DBG] pgmap v1130: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.9 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s; 1/738 objects misplaced (0.136%); 13 B/s, 0 objects/s recovering 2024-02-23T00:54:54.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:53.437601+0000 mon.a (mon.0) 3589 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-130"}]': finished 2024-02-23T00:54:54.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:53.437680+0000 mon.a (mon.0) 3590 : cluster [DBG] osdmap e745: 8 total, 8 up, 6 in 2024-02-23T00:54:54.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:53.452118+0000 osd.4 (osd.4) 313 : cluster [DBG] 4.1 scrub starts 2024-02-23T00:54:54.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:53.455031+0000 osd.4 (osd.4) 314 : cluster [DBG] 4.1 scrub ok 2024-02-23T00:54:54.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:53.582811+0000 osd.6 (osd.6) 293 : cluster [DBG] 2.17 deep-scrub starts 2024-02-23T00:54:54.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:53.583707+0000 osd.6 (osd.6) 294 : cluster [DBG] 2.17 deep-scrub ok 2024-02-23T00:54:54.884 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:54:54.917 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:54.937 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:54:54.940 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:54.940 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:54:55.041 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:54:55.071 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:55.091 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:54:55.093 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:55.093 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:54:55.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.907+0000 7fa98e97d700 1 -- 172.21.15.47:0/863099121 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa9880f7400 msgr2=0x7fa988103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:55.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.907+0000 7fa98e97d700 1 --2- 172.21.15.47:0/863099121 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa9880f7400 0x7fa988103a20 secure :-1 s=READY pgs=2802 cs=0 l=1 rev1=1 rx=0x7fa980009be0 tx=0x7fa980004d10).stop 2024-02-23T00:54:55.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.907+0000 7fa98e97d700 1 -- 172.21.15.47:0/863099121 shutdown_connections 2024-02-23T00:54:55.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.907+0000 7fa98e97d700 1 --2- 172.21.15.47:0/863099121 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa9880f7400 0x7fa988103a20 unknown :-1 s=CLOSED pgs=2802 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:55.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.907+0000 7fa98e97d700 1 --2- 172.21.15.47:0/863099121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9880fcbb0 0x7fa9880fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:55.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.907+0000 7fa98e97d700 1 --2- 172.21.15.47:0/863099121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9880fd510 0x7fa9880f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:55.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.907+0000 7fa98e97d700 1 -- 172.21.15.47:0/863099121 >> 172.21.15.47:0/863099121 conn(0x7fa9880f2820 msgr2=0x7fa9880f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:55.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.907+0000 7fa98e97d700 1 -- 172.21.15.47:0/863099121 shutdown_connections 2024-02-23T00:54:55.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.907+0000 7fa98e97d700 1 -- 172.21.15.47:0/863099121 wait complete. 2024-02-23T00:54:55.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.908+0000 7fa98e97d700 1 librados: starting msgr at 2024-02-23T00:54:55.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.908+0000 7fa98e97d700 1 librados: starting objecter 2024-02-23T00:54:55.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.908+0000 7fa98e97d700 1 Processor -- start 2024-02-23T00:54:55.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.908+0000 7fa98e97d700 1 -- start start 2024-02-23T00:54:55.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.908+0000 7fa98e97d700 1 librados: setting wanted keys 2024-02-23T00:54:55.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.908+0000 7fa98e97d700 1 librados: calling monclient init 2024-02-23T00:54:55.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.908+0000 7fa98e97d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9880f7400 0x7fa98810de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:55.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.908+0000 7fa98e97d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9880fcbb0 0x7fa98810e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:55.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.908+0000 7fa98e97d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa9880fd510 0x7fa988108030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:55.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.908+0000 7fa98e97d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa988108520 con 0x7fa9880fcbb0 2024-02-23T00:54:55.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.908+0000 7fa98e97d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa988108690 con 0x7fa9880f7400 2024-02-23T00:54:55.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.908+0000 7fa98e97d700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa988108850 con 0x7fa9880fd510 2024-02-23T00:54:55.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.909+0000 7fa98cf1a700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa9880fd510 0x7fa988108030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:55.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.909+0000 7fa98cf1a700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa9880fd510 0x7fa988108030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:42818/0 (socket says 172.21.15.47:42818) 2024-02-23T00:54:55.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.909+0000 7fa98cf1a700 1 -- 172.21.15.47:0/2309350526 learned_addr learned my addr 172.21.15.47:0/2309350526 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:54:55.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.909+0000 7fa987fff700 1 --2- 172.21.15.47:0/2309350526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9880f7400 0x7fa98810de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:55.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.909+0000 7fa9877fe700 1 --2- 172.21.15.47:0/2309350526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9880fcbb0 0x7fa98810e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:55.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.909+0000 7fa98cf1a700 1 -- 172.21.15.47:0/2309350526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9880f7400 msgr2=0x7fa98810de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:55.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.909+0000 7fa98cf1a700 1 --2- 172.21.15.47:0/2309350526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9880f7400 0x7fa98810de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:55.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.909+0000 7fa98cf1a700 1 -- 172.21.15.47:0/2309350526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9880fcbb0 msgr2=0x7fa98810e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:55.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.909+0000 7fa98cf1a700 1 --2- 172.21.15.47:0/2309350526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9880fcbb0 0x7fa98810e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:55.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.909+0000 7fa98cf1a700 1 -- 172.21.15.47:0/2309350526 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9800097c0 con 0x7fa9880fd510 2024-02-23T00:54:55.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.909+0000 7fa98cf1a700 1 --2- 172.21.15.47:0/2309350526 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa9880fd510 0x7fa988108030 secure :-1 s=READY pgs=2803 cs=0 l=1 rev1=1 rx=0x7fa980009790 tx=0x7fa980004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:55.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.909+0000 7fa9857fa700 10 client.?.objecter ms_handle_connect 0x7fa9880fd510 2024-02-23T00:54:55.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.909+0000 7fa9857fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:54:55.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.909+0000 7fa9857fa700 1 -- 172.21.15.47:0/2309350526 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98000e040 con 0x7fa9880fd510 2024-02-23T00:54:55.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.909+0000 7fa9857fa700 1 -- 172.21.15.47:0/2309350526 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fa980018040 con 0x7fa9880fd510 2024-02-23T00:54:55.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.909+0000 7fa98e97d700 1 -- 172.21.15.47:0/2309350526 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa988108ad0 con 0x7fa9880fd510 2024-02-23T00:54:55.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.909+0000 7fa9857fa700 1 -- 172.21.15.47:0/2309350526 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa980013c30 con 0x7fa9880fd510 2024-02-23T00:54:55.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.909+0000 7fa98e97d700 10 client.42062.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:54:55.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.909+0000 7fa98e97d700 1 -- 172.21.15.47:0/2309350526 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa988114cc0 con 0x7fa9880fd510 2024-02-23T00:54:55.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.909+0000 7fa98e97d700 1 librados: init done 2024-02-23T00:54:55.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.910+0000 7fa98e97d700 1 -- 172.21.15.47:0/2309350526 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa98805fa60 con 0x7fa9880fd510 2024-02-23T00:54:55.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.911+0000 7fa9857fa700 1 -- 172.21.15.47:0/2309350526 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fa980013420 con 0x7fa9880fd510 2024-02-23T00:54:55.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.911+0000 7fa9857fa700 1 --2- 172.21.15.47:0/2309350526 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa970062cc0 0x7fa970065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:55.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.911+0000 7fa9857fa700 1 -- 172.21.15.47:0/2309350526 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(746..746 src has 621..746) v4 ==== 9400+0+0 (secure 0 0 0) 0x7fa98007f190 con 0x7fa9880fd510 2024-02-23T00:54:55.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.911+0000 7fa9857fa700 10 client.42062.objecter ms_dispatch 0x7fa988001550 osd_map(746..746 src has 621..746) v4 2024-02-23T00:54:55.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.911+0000 7fa9857fa700 3 client.42062.objecter handle_osd_map got epochs [746,746] > 0 2024-02-23T00:54:55.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.911+0000 7fa9857fa700 3 client.42062.objecter handle_osd_map decoding full epoch 746 2024-02-23T00:54:55.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.911+0000 7fa987fff700 1 --2- 172.21.15.47:0/2309350526 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa970062cc0 0x7fa970065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:55.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.911+0000 7fa9857fa700 20 client.42062.objecter dump_active .. 0 homeless 2024-02-23T00:54:55.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.911+0000 7fa9857fa700 10 client.42062.objecter ms_handle_connect 0x7fa970062cc0 2024-02-23T00:54:55.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.911+0000 7fa987fff700 1 --2- 172.21.15.47:0/2309350526 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa970062cc0 0x7fa970065180 secure :-1 s=READY pgs=3194 cs=0 l=1 rev1=1 rx=0x7fa9780048b0 tx=0x7fa97800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:55.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:54.917+0000 7fa9857fa700 1 -- 172.21.15.47:0/2309350526 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fa98004e7a0 con 0x7fa9880fd510 2024-02-23T00:54:55.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:55.111+0000 7fa98e97d700 1 -- 172.21.15.47:0/2309350526 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd in", "ids": ["0"]} v 0) v1 -- 0x7fa988105ea0 con 0x7fa9880fd510 2024-02-23T00:54:55.194 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:54:55.223 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:55.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:54:55.112+0000 7f3c89cf1700 -1 mon.a@0(leader).osd e746 definitely_dead 0 2024-02-23T00:54:55.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:55.449+0000 7fa9857fa700 1 -- 172.21.15.47:0/2309350526 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd in", "ids": ["0"]}]=0 marked in osd.0. v747) v1 ==== 85+0+0 (secure 0 0 0) 0x7fa980051dc0 con 0x7fa9880fd510 2024-02-23T00:54:55.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:55.455+0000 7fa98e97d700 10 librados: watch_flush enter 2024-02-23T00:54:55.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:55.455+0000 7fa98e97d700 10 librados: watch_flush exit 2024-02-23T00:54:55.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:55.455+0000 7fa98e97d700 20 client.42062.objecter shutdown clearing up homeless session... 2024-02-23T00:54:55.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:55.455+0000 7fa98e97d700 10 client.42062.objecter successfully canceled tick 2024-02-23T00:54:55.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:55.455+0000 7fa98e97d700 1 -- 172.21.15.47:0/2309350526 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa970062cc0 msgr2=0x7fa970065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:55.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:55.455+0000 7fa98e97d700 1 --2- 172.21.15.47:0/2309350526 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa970062cc0 0x7fa970065180 secure :-1 s=READY pgs=3194 cs=0 l=1 rev1=1 rx=0x7fa9780048b0 tx=0x7fa97800b040).stop 2024-02-23T00:54:55.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:55.455+0000 7fa98e97d700 1 -- 172.21.15.47:0/2309350526 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa9880fd510 msgr2=0x7fa988108030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:55.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:55.455+0000 7fa98e97d700 1 --2- 172.21.15.47:0/2309350526 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa9880fd510 0x7fa988108030 secure :-1 s=READY pgs=2803 cs=0 l=1 rev1=1 rx=0x7fa980009790 tx=0x7fa980004d10).stop 2024-02-23T00:54:55.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:55.457+0000 7fa98e97d700 1 -- 172.21.15.47:0/2309350526 shutdown_connections 2024-02-23T00:54:55.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:55.457+0000 7fa98e97d700 1 --2- 172.21.15.47:0/2309350526 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa970062cc0 0x7fa970065180 unknown :-1 s=CLOSED pgs=3194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:55.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:55.457+0000 7fa98e97d700 1 --2- 172.21.15.47:0/2309350526 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa9880fd510 0x7fa988108030 unknown :-1 s=CLOSED pgs=2803 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:55.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:55.457+0000 7fa98e97d700 1 --2- 172.21.15.47:0/2309350526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9880f7400 0x7fa98810de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:55.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:55.457+0000 7fa98e97d700 1 --2- 172.21.15.47:0/2309350526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9880fcbb0 0x7fa98810e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:55.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:55.457+0000 7fa98e97d700 1 -- 172.21.15.47:0/2309350526 >> 172.21.15.47:0/2309350526 conn(0x7fa9880f2820 msgr2=0x7fa988060d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:55.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:55.460+0000 7fa98e97d700 1 -- 172.21.15.47:0/2309350526 shutdown_connections 2024-02-23T00:54:55.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:55.460+0000 7fa98e97d700 1 -- 172.21.15.47:0/2309350526 wait complete. 2024-02-23T00:54:55.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:55.460+0000 7fa98e97d700 1 librados: shutdown 2024-02-23T00:54:55.473 INFO:teuthology.orchestra.run.smithi047.stderr:marked in osd.0. 2024-02-23T00:54:55.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:54.406550+0000 osd.1 (osd.1) 303 : cluster [DBG] 2.1 scrub starts 2024-02-23T00:54:55.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:54.409552+0000 osd.1 (osd.1) 304 : cluster [DBG] 2.1 scrub ok 2024-02-23T00:54:55.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:54.438051+0000 mon.a (mon.0) 3591 : cluster [DBG] osdmap e746: 8 total, 8 up, 6 in 2024-02-23T00:54:55.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:55.113116+0000 mon.c (mon.2) 453 : audit [DBG] from='client.? 172.21.15.47:0/2309350526' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["0"]}]: dispatch 2024-02-23T00:54:55.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:55.113535+0000 mon.a (mon.0) 3592 : audit [DBG] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["0"]}]: dispatch 2024-02-23T00:54:55.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:55.220080+0000 osd.5 (osd.5) 322 : cluster [DBG] 18.f scrub starts 2024-02-23T00:54:55.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:55.225779+0000 osd.5 (osd.5) 323 : cluster [DBG] 18.f scrub ok 2024-02-23T00:54:55.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:54.406550+0000 osd.1 (osd.1) 303 : cluster [DBG] 2.1 scrub starts 2024-02-23T00:54:55.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:54.409552+0000 osd.1 (osd.1) 304 : cluster [DBG] 2.1 scrub ok 2024-02-23T00:54:55.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:54.438051+0000 mon.a (mon.0) 3591 : cluster [DBG] osdmap e746: 8 total, 8 up, 6 in 2024-02-23T00:54:55.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:55.113116+0000 mon.c (mon.2) 453 : audit [DBG] from='client.? 172.21.15.47:0/2309350526' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["0"]}]: dispatch 2024-02-23T00:54:55.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:55.113535+0000 mon.a (mon.0) 3592 : audit [DBG] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["0"]}]: dispatch 2024-02-23T00:54:55.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:55.220080+0000 osd.5 (osd.5) 322 : cluster [DBG] 18.f scrub starts 2024-02-23T00:54:55.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:55.225779+0000 osd.5 (osd.5) 323 : cluster [DBG] 18.f scrub ok 2024-02-23T00:54:55.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:54.406550+0000 osd.1 (osd.1) 303 : cluster [DBG] 2.1 scrub starts 2024-02-23T00:54:55.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:54.409552+0000 osd.1 (osd.1) 304 : cluster [DBG] 2.1 scrub ok 2024-02-23T00:54:55.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:54.438051+0000 mon.a (mon.0) 3591 : cluster [DBG] osdmap e746: 8 total, 8 up, 6 in 2024-02-23T00:54:55.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:55.113116+0000 mon.c (mon.2) 453 : audit [DBG] from='client.? 172.21.15.47:0/2309350526' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["0"]}]: dispatch 2024-02-23T00:54:55.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:55.113535+0000 mon.a (mon.0) 3592 : audit [DBG] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["0"]}]: dispatch 2024-02-23T00:54:55.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:55.220080+0000 osd.5 (osd.5) 322 : cluster [DBG] 18.f scrub starts 2024-02-23T00:54:55.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:55.225779+0000 osd.5 (osd.5) 323 : cluster [DBG] 18.f scrub ok 2024-02-23T00:54:56.138 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:54:56.274 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:54:56.277 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:56.277 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:54:56.378 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:54:56.406 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:54:56.609 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:56.610 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:54:56.614 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-02-23T00:54:56.614 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:54:56.711 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:54:56.735 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:55.392574+0000 mgr.y (mgr.14517) 640 : cluster [DBG] pgmap v1133: 152 pgs: 1 active+remapped+backfill_toofull, 151 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 641 B/s rd, 0 op/s; 1/726 objects misplaced (0.138%) 2024-02-23T00:54:56.735 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:55.449836+0000 mon.a (mon.0) 3593 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["0"]}]': finished 2024-02-23T00:54:56.735 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:55.449909+0000 mon.a (mon.0) 3594 : cluster [DBG] osdmap e747: 8 total, 8 up, 7 in 2024-02-23T00:54:56.735 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:55.456823+0000 mon.c (mon.2) 454 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:54:56.735 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:55.457445+0000 mon.a (mon.0) 3595 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:54:56.735 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:55.602002+0000 osd.6 (osd.6) 295 : cluster [DBG] 2.14 scrub starts 2024-02-23T00:54:56.735 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:55.602730+0000 osd.6 (osd.6) 296 : cluster [DBG] 2.14 scrub ok 2024-02-23T00:54:56.735 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:56.311013+0000 mon.a (mon.0) 3596 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:54:56.737 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:56.756 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:54:56.759 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:56.759 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:54:56.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:55.392574+0000 mgr.y (mgr.14517) 640 : cluster [DBG] pgmap v1133: 152 pgs: 1 active+remapped+backfill_toofull, 151 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 641 B/s rd, 0 op/s; 1/726 objects misplaced (0.138%) 2024-02-23T00:54:56.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:55.449836+0000 mon.a (mon.0) 3593 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["0"]}]': finished 2024-02-23T00:54:56.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:55.449909+0000 mon.a (mon.0) 3594 : cluster [DBG] osdmap e747: 8 total, 8 up, 7 in 2024-02-23T00:54:56.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:55.456823+0000 mon.c (mon.2) 454 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:54:56.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:55.457445+0000 mon.a (mon.0) 3595 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:54:56.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:55.602002+0000 osd.6 (osd.6) 295 : cluster [DBG] 2.14 scrub starts 2024-02-23T00:54:56.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:55.602730+0000 osd.6 (osd.6) 296 : cluster [DBG] 2.14 scrub ok 2024-02-23T00:54:56.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:56.311013+0000 mon.a (mon.0) 3596 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:54:56.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:55.392574+0000 mgr.y (mgr.14517) 640 : cluster [DBG] pgmap v1133: 152 pgs: 1 active+remapped+backfill_toofull, 151 active+clean; 3.8 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 641 B/s rd, 0 op/s; 1/726 objects misplaced (0.138%) 2024-02-23T00:54:56.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:55.449836+0000 mon.a (mon.0) 3593 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["0"]}]': finished 2024-02-23T00:54:56.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:55.449909+0000 mon.a (mon.0) 3594 : cluster [DBG] osdmap e747: 8 total, 8 up, 7 in 2024-02-23T00:54:56.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:55.456823+0000 mon.c (mon.2) 454 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:54:56.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:55.457445+0000 mon.a (mon.0) 3595 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:54:56.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:55.602002+0000 osd.6 (osd.6) 295 : cluster [DBG] 2.14 scrub starts 2024-02-23T00:54:56.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:55.602730+0000 osd.6 (osd.6) 296 : cluster [DBG] 2.14 scrub ok 2024-02-23T00:54:56.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:56.311013+0000 mon.a (mon.0) 3596 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:54:56.860 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:54:56.885 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:56.904 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:54:56.906 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:56.907 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:54:57.008 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:54:57.038 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:57.117 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:54:57.120 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:57.120 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:54:57.221 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:54:57.247 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:57.266 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:54:57.269 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:57.269 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:54:57.370 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:54:57.397 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:57.416 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:54:57.418 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:57.419 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:54:57.460 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:56.454963+0000 mon.a (mon.0) 3597 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:54:57.460 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:56.455117+0000 mon.a (mon.0) 3598 : cluster [DBG] osdmap e748: 8 total, 8 up, 7 in 2024-02-23T00:54:57.460 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:56.541653+0000 mon.c (mon.2) 455 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:54:57.460 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:56.542061+0000 mon.a (mon.0) 3599 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:54:57.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.489+0000 7f8677ead700 1 -- 172.21.15.47:0/1656887451 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f86700fdbb0 msgr2=0x7f86700fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:57.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.489+0000 7f8677ead700 1 --2- 172.21.15.47:0/1656887451 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f86700fdbb0 0x7f86700fdf90 secure :-1 s=READY pgs=2804 cs=0 l=1 rev1=1 rx=0x7f86640047b0 tx=0x7f8664009cb0).stop 2024-02-23T00:54:57.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.489+0000 7f8677ead700 1 -- 172.21.15.47:0/1656887451 shutdown_connections 2024-02-23T00:54:57.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.489+0000 7f8677ead700 1 --2- 172.21.15.47:0/1656887451 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86700f8420 0x7f8670104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:54:57.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.489+0000 7f8677ead700 1 --2- 172.21.15.47:0/1656887451 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f86700fdbb0 0x7f86700fdf90 unknown :-1 s=CLOSED pgs=2804 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:57.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.489+0000 7f8677ead700 1 --2- 172.21.15.47:0/1656887451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86700fe510 0x7f86700f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:57.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.489+0000 7f8677ead700 1 -- 172.21.15.47:0/1656887451 >> 172.21.15.47:0/1656887451 conn(0x7f86700f3800 msgr2=0x7f86700f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:57.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.489+0000 7f8677ead700 1 -- 172.21.15.47:0/1656887451 shutdown_connections 2024-02-23T00:54:57.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.489+0000 7f8677ead700 1 -- 172.21.15.47:0/1656887451 wait complete. 2024-02-23T00:54:57.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.490+0000 7f8677ead700 1 librados: starting msgr at 2024-02-23T00:54:57.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.490+0000 7f8677ead700 1 librados: starting objecter 2024-02-23T00:54:57.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.490+0000 7f8677ead700 1 Processor -- start 2024-02-23T00:54:57.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.490+0000 7f8677ead700 1 -- start start 2024-02-23T00:54:57.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.490+0000 7f8677ead700 1 librados: setting wanted keys 2024-02-23T00:54:57.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.490+0000 7f8677ead700 1 librados: calling monclient init 2024-02-23T00:54:57.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.490+0000 7f8677ead700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f86700f8420 0x7f867010e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:57.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.490+0000 7f8677ead700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86700fdbb0 0x7f867010e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:57.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.490+0000 7f8677ead700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86700fe510 0x7f86701081f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:57.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.490+0000 7f8677ead700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86701086e0 con 0x7f86700fdbb0 2024-02-23T00:54:57.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.490+0000 7f8677ead700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8670108850 con 0x7f86700fe510 2024-02-23T00:54:57.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.490+0000 7f8677ead700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8670108a10 con 0x7f86700f8420 2024-02-23T00:54:57.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.490+0000 7f8675c49700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f86700f8420 0x7f867010e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:57.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.490+0000 7f8675c49700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f86700f8420 0x7f867010e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:37878/0 (socket says 172.21.15.47:37878) 2024-02-23T00:54:57.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.490+0000 7f8675c49700 1 -- 172.21.15.47:0/1384028576 learned_addr learned my addr 172.21.15.47:0/1384028576 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:54:57.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.490+0000 7f8675448700 1 --2- 172.21.15.47:0/1384028576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86700fdbb0 0x7f867010e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:57.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.491+0000 7f867644a700 1 --2- 172.21.15.47:0/1384028576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86700fe510 0x7f86701081f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:57.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.491+0000 7f8675c49700 1 -- 172.21.15.47:0/1384028576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86700fe510 msgr2=0x7f86701081f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:57.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.491+0000 7f8675c49700 1 --2- 172.21.15.47:0/1384028576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86700fe510 0x7f86701081f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:57.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.491+0000 7f8675c49700 1 -- 172.21.15.47:0/1384028576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86700fdbb0 msgr2=0x7f867010e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:57.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.491+0000 7f8675c49700 1 --2- 172.21.15.47:0/1384028576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86700fdbb0 0x7f867010e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:57.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.491+0000 7f8675c49700 1 -- 172.21.15.47:0/1384028576 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8664005040 con 0x7f86700f8420 2024-02-23T00:54:57.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.491+0000 7f8675448700 1 --2- 172.21.15.47:0/1384028576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86700fdbb0 0x7f867010e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:54:57.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.491+0000 7f8675c49700 1 --2- 172.21.15.47:0/1384028576 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f86700f8420 0x7f867010e050 secure :-1 s=READY pgs=2805 cs=0 l=1 rev1=1 rx=0x7f86640044e0 tx=0x7f8664009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:57.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.491+0000 7f8662ffd700 10 client.?.objecter ms_handle_connect 0x7f86700f8420 2024-02-23T00:54:57.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.491+0000 7f8662ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:54:57.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.491+0000 7f8662ffd700 1 -- 172.21.15.47:0/1384028576 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8664018070 con 0x7f86700f8420 2024-02-23T00:54:57.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.491+0000 7f8662ffd700 1 -- 172.21.15.47:0/1384028576 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f8664013e50 con 0x7f86700f8420 2024-02-23T00:54:57.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.491+0000 7f8677ead700 1 -- 172.21.15.47:0/1384028576 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8670108c90 con 0x7f86700f8420 2024-02-23T00:54:57.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.491+0000 7f8662ffd700 1 -- 172.21.15.47:0/1384028576 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866401db60 con 0x7f86700f8420 2024-02-23T00:54:57.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.491+0000 7f8677ead700 10 client.42068.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:54:57.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.491+0000 7f8677ead700 1 -- 172.21.15.47:0/1384028576 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8670114e80 con 0x7f86700f8420 2024-02-23T00:54:57.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.491+0000 7f8677ead700 1 librados: init done 2024-02-23T00:54:57.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.493+0000 7f8677ead700 1 -- 172.21.15.47:0/1384028576 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f867005ee80 con 0x7f86700f8420 2024-02-23T00:54:57.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.493+0000 7f8662ffd700 1 -- 172.21.15.47:0/1384028576 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f8664009240 con 0x7f86700f8420 2024-02-23T00:54:57.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.493+0000 7f8662ffd700 1 --2- 172.21.15.47:0/1384028576 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f865c062cc0 0x7f865c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:54:57.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.493+0000 7f8662ffd700 1 -- 172.21.15.47:0/1384028576 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(749..749 src has 621..749) v4 ==== 9814+0+0 (secure 0 0 0) 0x7f866407ec30 con 0x7f86700f8420 2024-02-23T00:54:57.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.493+0000 7f8662ffd700 10 client.42068.objecter ms_dispatch 0x7f8670001550 osd_map(749..749 src has 621..749) v4 2024-02-23T00:54:57.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.493+0000 7f8662ffd700 3 client.42068.objecter handle_osd_map got epochs [749,749] > 0 2024-02-23T00:54:57.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.493+0000 7f8675448700 1 --2- 172.21.15.47:0/1384028576 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f865c062cc0 0x7f865c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:54:57.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.493+0000 7f8662ffd700 3 client.42068.objecter handle_osd_map decoding full epoch 749 2024-02-23T00:54:57.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.493+0000 7f8662ffd700 20 client.42068.objecter dump_active .. 0 homeless 2024-02-23T00:54:57.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.494+0000 7f8662ffd700 10 client.42068.objecter ms_handle_connect 0x7f865c062cc0 2024-02-23T00:54:57.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.494+0000 7f8675448700 1 --2- 172.21.15.47:0/1384028576 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f865c062cc0 0x7f865c065180 secure :-1 s=READY pgs=3195 cs=0 l=1 rev1=1 rx=0x7f866c004fd0 tx=0x7f866c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:54:57.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.497+0000 7f8662ffd700 1 -- 172.21.15.47:0/1384028576 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f866404dff0 con 0x7f86700f8420 2024-02-23T00:54:57.519 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:54:57.549 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:57.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:57.685+0000 7f8677ead700 1 -- 172.21.15.47:0/1384028576 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7f8670109780 con 0x7f86700f8420 2024-02-23T00:54:57.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:56.454963+0000 mon.a (mon.0) 3597 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:54:57.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:56.455117+0000 mon.a (mon.0) 3598 : cluster [DBG] osdmap e748: 8 total, 8 up, 7 in 2024-02-23T00:54:57.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:56.541653+0000 mon.c (mon.2) 455 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:54:57.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:56.542061+0000 mon.a (mon.0) 3599 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:54:57.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:56.454963+0000 mon.a (mon.0) 3597 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:54:57.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:56.455117+0000 mon.a (mon.0) 3598 : cluster [DBG] osdmap e748: 8 total, 8 up, 7 in 2024-02-23T00:54:57.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:56.541653+0000 mon.c (mon.2) 455 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:54:57.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:56.542061+0000 mon.a (mon.0) 3599 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:54:57.898 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:54:57.915 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:57.916 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:54:58.017 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:54:58.045 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:58.193 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:54:58.196 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:58.197 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:54:58.297 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:54:58.325 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:58.462 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:57.391494+0000 osd.4 (osd.4) 315 : cluster [DBG] 4.13 scrub starts 2024-02-23T00:54:58.462 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:57.393329+0000 mgr.y (mgr.14517) 641 : cluster [DBG] pgmap v1136: 184 pgs: 1 active+remapped+backfill_toofull, 1 remapped+peering, 1 active+recovering, 181 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 2 op/s; 3/786 objects misplaced (0.382%); 977 KiB/s, 9 objects/s recovering 2024-02-23T00:54:58.462 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:57.393336+0000 osd.4 (osd.4) 316 : cluster [DBG] 4.13 scrub ok 2024-02-23T00:54:58.462 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:57.452206+0000 mon.a (mon.0) 3600 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-23T00:54:58.462 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:57.460795+0000 mon.a (mon.0) 3601 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:54:58.462 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:57.460867+0000 mon.a (mon.0) 3602 : cluster [DBG] osdmap e749: 8 total, 8 up, 7 in 2024-02-23T00:54:58.462 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:57.461317+0000 osd.1 (osd.1) 305 : cluster [DBG] 6.0 starting backfill to osd.0 from (0'0,0'0] MAX to 107'2 2024-02-23T00:54:58.462 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:57.463956+0000 mon.c (mon.2) 456 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-132"}]: dispatch 2024-02-23T00:54:58.462 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:57.464354+0000 mon.a (mon.0) 3603 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-132"}]: dispatch 2024-02-23T00:54:58.462 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:57.686933+0000 mon.c (mon.2) 457 : audit [INF] from='client.? 172.21.15.47:0/1384028576' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:54:58.462 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:57.687270+0000 mon.a (mon.0) 3604 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:54:58.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:58.466+0000 7f8662ffd700 1 -- 172.21.15.47:0/1384028576 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v750) v1 ==== 87+0+0 (secure 0 0 0) 0x7f8664051610 con 0x7f86700f8420 2024-02-23T00:54:58.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:58.469+0000 7f8677ead700 10 librados: watch_flush enter 2024-02-23T00:54:58.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:58.469+0000 7f8677ead700 10 librados: watch_flush exit 2024-02-23T00:54:58.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:58.469+0000 7f8677ead700 20 client.42068.objecter shutdown clearing up homeless session... 2024-02-23T00:54:58.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:58.469+0000 7f8677ead700 10 client.42068.objecter successfully canceled tick 2024-02-23T00:54:58.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:58.469+0000 7f8677ead700 1 -- 172.21.15.47:0/1384028576 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f865c062cc0 msgr2=0x7f865c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:58.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:58.469+0000 7f8677ead700 1 --2- 172.21.15.47:0/1384028576 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f865c062cc0 0x7f865c065180 secure :-1 s=READY pgs=3195 cs=0 l=1 rev1=1 rx=0x7f866c004fd0 tx=0x7f866c00b040).stop 2024-02-23T00:54:58.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:58.469+0000 7f8677ead700 1 -- 172.21.15.47:0/1384028576 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f86700f8420 msgr2=0x7f867010e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:54:58.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:58.469+0000 7f8677ead700 1 --2- 172.21.15.47:0/1384028576 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f86700f8420 0x7f867010e050 secure :-1 s=READY pgs=2805 cs=0 l=1 rev1=1 rx=0x7f86640044e0 tx=0x7f8664009cb0).stop 2024-02-23T00:54:58.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:58.470+0000 7f8677ead700 1 -- 172.21.15.47:0/1384028576 shutdown_connections 2024-02-23T00:54:58.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:58.470+0000 7f8677ead700 1 --2- 172.21.15.47:0/1384028576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86700fe510 0x7f86701081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:58.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:58.470+0000 7f8677ead700 1 --2- 172.21.15.47:0/1384028576 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f865c062cc0 0x7f865c065180 unknown :-1 s=CLOSED pgs=3195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:58.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:58.470+0000 7f8677ead700 1 --2- 172.21.15.47:0/1384028576 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f86700f8420 0x7f867010e050 unknown :-1 s=CLOSED pgs=2805 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:58.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:58.470+0000 7f8677ead700 1 --2- 172.21.15.47:0/1384028576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86700fdbb0 0x7f867010e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:54:58.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:58.470+0000 7f8677ead700 1 -- 172.21.15.47:0/1384028576 >> 172.21.15.47:0/1384028576 conn(0x7f86700f3800 msgr2=0x7f86700fb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:54:58.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:58.470+0000 7f8677ead700 1 -- 172.21.15.47:0/1384028576 shutdown_connections 2024-02-23T00:54:58.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:58.470+0000 7f8677ead700 1 -- 172.21.15.47:0/1384028576 wait complete. 2024-02-23T00:54:58.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:54:58.470+0000 7f8677ead700 1 librados: shutdown 2024-02-23T00:54:58.474 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is set 2024-02-23T00:54:58.475 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:54:58.477 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:58.478 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:54:58.579 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:54:58.609 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:58.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:57.391494+0000 osd.4 (osd.4) 315 : cluster [DBG] 4.13 scrub starts 2024-02-23T00:54:58.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:57.393329+0000 mgr.y (mgr.14517) 641 : cluster [DBG] pgmap v1136: 184 pgs: 1 active+remapped+backfill_toofull, 1 remapped+peering, 1 active+recovering, 181 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 2 op/s; 3/786 objects misplaced (0.382%); 977 KiB/s, 9 objects/s recovering 2024-02-23T00:54:58.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:57.393336+0000 osd.4 (osd.4) 316 : cluster [DBG] 4.13 scrub ok 2024-02-23T00:54:58.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:57.452206+0000 mon.a (mon.0) 3600 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-23T00:54:58.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:57.460795+0000 mon.a (mon.0) 3601 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:54:58.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:57.460867+0000 mon.a (mon.0) 3602 : cluster [DBG] osdmap e749: 8 total, 8 up, 7 in 2024-02-23T00:54:58.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:57.461317+0000 osd.1 (osd.1) 305 : cluster [DBG] 6.0 starting backfill to osd.0 from (0'0,0'0] MAX to 107'2 2024-02-23T00:54:58.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:57.463956+0000 mon.c (mon.2) 456 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-132"}]: dispatch 2024-02-23T00:54:58.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:57.464354+0000 mon.a (mon.0) 3603 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-132"}]: dispatch 2024-02-23T00:54:58.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:57.686933+0000 mon.c (mon.2) 457 : audit [INF] from='client.? 172.21.15.47:0/1384028576' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:54:58.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:57.687270+0000 mon.a (mon.0) 3604 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:54:58.917 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:54:58.917 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:54:58.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:57.391494+0000 osd.4 (osd.4) 315 : cluster [DBG] 4.13 scrub starts 2024-02-23T00:54:58.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:57.393329+0000 mgr.y (mgr.14517) 641 : cluster [DBG] pgmap v1136: 184 pgs: 1 active+remapped+backfill_toofull, 1 remapped+peering, 1 active+recovering, 181 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 2 op/s; 3/786 objects misplaced (0.382%); 977 KiB/s, 9 objects/s recovering 2024-02-23T00:54:58.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:57.393336+0000 osd.4 (osd.4) 316 : cluster [DBG] 4.13 scrub ok 2024-02-23T00:54:58.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:57.452206+0000 mon.a (mon.0) 3600 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-23T00:54:58.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:57.460795+0000 mon.a (mon.0) 3601 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:54:58.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:57.460867+0000 mon.a (mon.0) 3602 : cluster [DBG] osdmap e749: 8 total, 8 up, 7 in 2024-02-23T00:54:58.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:57.461317+0000 osd.1 (osd.1) 305 : cluster [DBG] 6.0 starting backfill to osd.0 from (0'0,0'0] MAX to 107'2 2024-02-23T00:54:58.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:57.463956+0000 mon.c (mon.2) 456 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-132"}]: dispatch 2024-02-23T00:54:58.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:57.464354+0000 mon.a (mon.0) 3603 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-132"}]: dispatch 2024-02-23T00:54:58.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:57.686933+0000 mon.c (mon.2) 457 : audit [INF] from='client.? 172.21.15.47:0/1384028576' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:54:58.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:57.687270+0000 mon.a (mon.0) 3604 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:54:58.948 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:54:58.957 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:54:58.961 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:58.961 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:54:59.062 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:54:59.095 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:59.116 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:54:59.119 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:59.119 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:54:59.220 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:54:59.251 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:59.272 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:54:59.275 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:59.275 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:54:59.376 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:54:59.407 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:59.616 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:54:59.618 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:59.619 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:54:59.695 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:57.849095+0000 osd.0 (osd.0) 281 : cluster [DBG] 4.b scrub starts 2024-02-23T00:54:59.695 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:57.851679+0000 osd.0 (osd.0) 282 : cluster [DBG] 4.b scrub ok 2024-02-23T00:54:59.696 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:58.025853+0000 osd.7 (osd.7) 343 : cluster [DBG] 2.8 scrub starts 2024-02-23T00:54:59.696 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:58.028860+0000 osd.7 (osd.7) 344 : cluster [DBG] 2.8 scrub ok 2024-02-23T00:54:59.696 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:58.351995+0000 osd.4 (osd.4) 317 : cluster [DBG] 4.c scrub starts 2024-02-23T00:54:59.696 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:58.353426+0000 osd.4 (osd.4) 318 : cluster [DBG] 4.c scrub ok 2024-02-23T00:54:59.696 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:58.457531+0000 mon.a (mon.0) 3605 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:54:59.696 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:58.466003+0000 mon.a (mon.0) 3606 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-132"}]': finished 2024-02-23T00:54:59.696 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:58.466092+0000 mon.a (mon.0) 3607 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:54:59.696 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:58.466174+0000 mon.a (mon.0) 3608 : cluster [DBG] osdmap e750: 8 total, 8 up, 7 in 2024-02-23T00:54:59.696 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:58.467593+0000 mon.c (mon.2) 458 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-132", "mode": "writeback"}]: dispatch 2024-02-23T00:54:59.696 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:58.468038+0000 mon.a (mon.0) 3609 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-132", "mode": "writeback"}]: dispatch 2024-02-23T00:54:59.696 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:58.614541+0000 osd.6 (osd.6) 297 : cluster [DBG] 5.2 scrub starts 2024-02-23T00:54:59.696 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:54:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:58.615555+0000 osd.6 (osd.6) 298 : cluster [DBG] 5.2 scrub ok 2024-02-23T00:54:59.697 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:57.849095+0000 osd.0 (osd.0) 281 : cluster [DBG] 4.b scrub starts 2024-02-23T00:54:59.697 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:57.851679+0000 osd.0 (osd.0) 282 : cluster [DBG] 4.b scrub ok 2024-02-23T00:54:59.697 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:58.025853+0000 osd.7 (osd.7) 343 : cluster [DBG] 2.8 scrub starts 2024-02-23T00:54:59.697 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:58.028860+0000 osd.7 (osd.7) 344 : cluster [DBG] 2.8 scrub ok 2024-02-23T00:54:59.697 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:58.351995+0000 osd.4 (osd.4) 317 : cluster [DBG] 4.c scrub starts 2024-02-23T00:54:59.697 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:58.353426+0000 osd.4 (osd.4) 318 : cluster [DBG] 4.c scrub ok 2024-02-23T00:54:59.697 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:58.457531+0000 mon.a (mon.0) 3605 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:54:59.697 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:58.466003+0000 mon.a (mon.0) 3606 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-132"}]': finished 2024-02-23T00:54:59.697 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:58.466092+0000 mon.a (mon.0) 3607 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:54:59.697 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:58.466174+0000 mon.a (mon.0) 3608 : cluster [DBG] osdmap e750: 8 total, 8 up, 7 in 2024-02-23T00:54:59.697 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:58.467593+0000 mon.c (mon.2) 458 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-132", "mode": "writeback"}]: dispatch 2024-02-23T00:54:59.697 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:58.468038+0000 mon.a (mon.0) 3609 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-132", "mode": "writeback"}]: dispatch 2024-02-23T00:54:59.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:58.614541+0000 osd.6 (osd.6) 297 : cluster [DBG] 5.2 scrub starts 2024-02-23T00:54:59.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:54:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:58.615555+0000 osd.6 (osd.6) 298 : cluster [DBG] 5.2 scrub ok 2024-02-23T00:54:59.720 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:54:59.750 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:57.849095+0000 osd.0 (osd.0) 281 : cluster [DBG] 4.b scrub starts 2024-02-23T00:54:59.750 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:57.851679+0000 osd.0 (osd.0) 282 : cluster [DBG] 4.b scrub ok 2024-02-23T00:54:59.750 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:58.025853+0000 osd.7 (osd.7) 343 : cluster [DBG] 2.8 scrub starts 2024-02-23T00:54:59.750 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:58.028860+0000 osd.7 (osd.7) 344 : cluster [DBG] 2.8 scrub ok 2024-02-23T00:54:59.750 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:58.351995+0000 osd.4 (osd.4) 317 : cluster [DBG] 4.c scrub starts 2024-02-23T00:54:59.750 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:58.353426+0000 osd.4 (osd.4) 318 : cluster [DBG] 4.c scrub ok 2024-02-23T00:54:59.750 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:58.457531+0000 mon.a (mon.0) 3605 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:54:59.751 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:58.466003+0000 mon.a (mon.0) 3606 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-132"}]': finished 2024-02-23T00:54:59.751 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:58.466092+0000 mon.a (mon.0) 3607 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:54:59.751 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:58.466174+0000 mon.a (mon.0) 3608 : cluster [DBG] osdmap e750: 8 total, 8 up, 7 in 2024-02-23T00:54:59.751 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:58.467593+0000 mon.c (mon.2) 458 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-132", "mode": "writeback"}]: dispatch 2024-02-23T00:54:59.751 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:58.468038+0000 mon.a (mon.0) 3609 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-132", "mode": "writeback"}]: dispatch 2024-02-23T00:54:59.751 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:58.614541+0000 osd.6 (osd.6) 297 : cluster [DBG] 5.2 scrub starts 2024-02-23T00:54:59.751 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:54:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:58.615555+0000 osd.6 (osd.6) 298 : cluster [DBG] 5.2 scrub ok 2024-02-23T00:54:59.752 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:59.774 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:54:59.776 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:59.777 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:54:59.878 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:54:59.908 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:54:59.983 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:54:59.986 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:54:59.986 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:55:00.006 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.3 dump_ops_in_flight 2024-02-23T00:55:00.087 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:55:00.119 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:00.140 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:55:00.142 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:00.143 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:55:00.244 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:55:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:00.299 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:55:00.302 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:00.303 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:55:00.404 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:55:00.436 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:00.457 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:55:00.460 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:00.460 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:55:00.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:59.393745+0000 mgr.y (mgr.14517) 642 : cluster [DBG] pgmap v1139: 184 pgs: 1 active+remapped+backfill_toofull, 1 remapped+peering, 1 active+recovering, 181 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1023 B/s rd, 5.0 KiB/s wr, 5 op/s; 3/786 objects misplaced (0.382%); 977 KiB/s, 9 objects/s recovering 2024-02-23T00:55:00.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:59.463118+0000 mon.a (mon.0) 3610 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:55:00.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:59.468786+0000 mon.a (mon.0) 3611 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-132", "mode": "writeback"}]': finished 2024-02-23T00:55:00.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:54:59.468913+0000 mon.a (mon.0) 3612 : cluster [DBG] osdmap e751: 8 total, 8 up, 7 in 2024-02-23T00:55:00.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:59.474553+0000 mon.c (mon.2) 459 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:55:00.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:54:59.474875+0000 mon.a (mon.0) 3613 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:55:00.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:00.305585+0000 mon.a (mon.0) 3614 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_count","val": "2"}]': finished 2024-02-23T00:55:00.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:00.305635+0000 mon.a (mon.0) 3615 : cluster [DBG] osdmap e752: 8 total, 8 up, 7 in 2024-02-23T00:55:00.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:00.307531+0000 mon.c (mon.2) 460 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:55:00.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:00.310351+0000 mon.a (mon.0) 3616 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:55:00.559 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:55:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:55:00] "GET /metrics HTTP/1.1" 200 191765 "" "Prometheus/2.33.4" 2024-02-23T00:55:00.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:59.393745+0000 mgr.y (mgr.14517) 642 : cluster [DBG] pgmap v1139: 184 pgs: 1 active+remapped+backfill_toofull, 1 remapped+peering, 1 active+recovering, 181 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1023 B/s rd, 5.0 KiB/s wr, 5 op/s; 3/786 objects misplaced (0.382%); 977 KiB/s, 9 objects/s recovering 2024-02-23T00:55:00.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:59.463118+0000 mon.a (mon.0) 3610 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:55:00.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:59.468786+0000 mon.a (mon.0) 3611 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-132", "mode": "writeback"}]': finished 2024-02-23T00:55:00.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:54:59.468913+0000 mon.a (mon.0) 3612 : cluster [DBG] osdmap e751: 8 total, 8 up, 7 in 2024-02-23T00:55:00.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:59.474553+0000 mon.c (mon.2) 459 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:55:00.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:54:59.474875+0000 mon.a (mon.0) 3613 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:55:00.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:00.305585+0000 mon.a (mon.0) 3614 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_count","val": "2"}]': finished 2024-02-23T00:55:00.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:00.305635+0000 mon.a (mon.0) 3615 : cluster [DBG] osdmap e752: 8 total, 8 up, 7 in 2024-02-23T00:55:00.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:00.307531+0000 mon.c (mon.2) 460 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:55:00.560 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:00.310351+0000 mon.a (mon.0) 3616 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:55:00.560 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:55:00.590 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:00.610 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:55:00.613 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:00.613 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:55:00.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:59.393745+0000 mgr.y (mgr.14517) 642 : cluster [DBG] pgmap v1139: 184 pgs: 1 active+remapped+backfill_toofull, 1 remapped+peering, 1 active+recovering, 181 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1023 B/s rd, 5.0 KiB/s wr, 5 op/s; 3/786 objects misplaced (0.382%); 977 KiB/s, 9 objects/s recovering 2024-02-23T00:55:00.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:59.463118+0000 mon.a (mon.0) 3610 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:55:00.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:59.468786+0000 mon.a (mon.0) 3611 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-132", "mode": "writeback"}]': finished 2024-02-23T00:55:00.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:54:59.468913+0000 mon.a (mon.0) 3612 : cluster [DBG] osdmap e751: 8 total, 8 up, 7 in 2024-02-23T00:55:00.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:59.474553+0000 mon.c (mon.2) 459 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:55:00.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:54:59.474875+0000 mon.a (mon.0) 3613 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:55:00.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:00.305585+0000 mon.a (mon.0) 3614 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_count","val": "2"}]': finished 2024-02-23T00:55:00.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:00.305635+0000 mon.a (mon.0) 3615 : cluster [DBG] osdmap e752: 8 total, 8 up, 7 in 2024-02-23T00:55:00.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:00.307531+0000 mon.c (mon.2) 460 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:55:00.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:00.310351+0000 mon.a (mon.0) 3616 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:55:00.714 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:55:00.748 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:00.770 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:55:00.771 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:55:00.773 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:00.774 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:55:00.875 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:55:00.906 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:00.927 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:55:00.929 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:00.929 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:55:01.030 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:55:01.060 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:01.081 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:55:01.083 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:01.085 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:55:01.186 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:55:01.213 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:01.233 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:55:01.235 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:01.236 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:55:01.337 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:55:01.368 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:01.441 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-23T00:55:01.567 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:55:01.570 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:01.570 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:55:01.615 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6, 7, 0] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-23T00:55:01.615 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-23T00:55:01.615 INFO:tasks.thrashosds.thrasher:Growing pool default.rgw.control 2024-02-23T00:55:01.615 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph pg dump --format=json 2024-02-23T00:55:01.671 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:55:01.700 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:01.955 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:55:01.957 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:01.958 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:55:02.059 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:55:02.087 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:02.236 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:55:02.239 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:02.240 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:55:02.340 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:55:02.371 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:02.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.546+0000 7efcd8cc1700 1 -- 172.21.15.47:0/407219712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efcd40fe480 msgr2=0x7efcd40f7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:02.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.546+0000 7efcd8cc1700 1 --2- 172.21.15.47:0/407219712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efcd40fe480 0x7efcd40f7cb0 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 rx=0x7efcc80047b0 tx=0x7efcc8009cb0).stop 2024-02-23T00:55:02.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.546+0000 7efcd8cc1700 1 -- 172.21.15.47:0/407219712 shutdown_connections 2024-02-23T00:55:02.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.546+0000 7efcd8cc1700 1 --2- 172.21.15.47:0/407219712 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efcd40f8390 0x7efcd4104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:02.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.546+0000 7efcd8cc1700 1 --2- 172.21.15.47:0/407219712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcd40fdb20 0x7efcd40fdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:02.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.546+0000 7efcd8cc1700 1 --2- 172.21.15.47:0/407219712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efcd40fe480 0x7efcd40f7cb0 unknown :-1 s=CLOSED pgs=2632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:02.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.546+0000 7efcd8cc1700 1 -- 172.21.15.47:0/407219712 >> 172.21.15.47:0/407219712 conn(0x7efcd40f3750 msgr2=0x7efcd40f5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:02.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.546+0000 7efcd8cc1700 1 -- 172.21.15.47:0/407219712 shutdown_connections 2024-02-23T00:55:02.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.546+0000 7efcd8cc1700 1 -- 172.21.15.47:0/407219712 wait complete. 2024-02-23T00:55:02.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.546+0000 7efcd8cc1700 1 librados: starting msgr at 2024-02-23T00:55:02.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.546+0000 7efcd8cc1700 1 librados: starting objecter 2024-02-23T00:55:02.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.546+0000 7efcd8cc1700 1 Processor -- start 2024-02-23T00:55:02.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.547+0000 7efcd8cc1700 1 -- start start 2024-02-23T00:55:02.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.547+0000 7efcd8cc1700 1 librados: setting wanted keys 2024-02-23T00:55:02.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.547+0000 7efcd8cc1700 1 librados: calling monclient init 2024-02-23T00:55:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.547+0000 7efcd8cc1700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcd40f8390 0x7efcd410deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.547+0000 7efcd8cc1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efcd40fdb20 0x7efcd410e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.547+0000 7efcd8cc1700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efcd40fe480 0x7efcd41080a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.547+0000 7efcd8cc1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd4108590 con 0x7efcd40fdb20 2024-02-23T00:55:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.547+0000 7efcd8cc1700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd4108700 con 0x7efcd40f8390 2024-02-23T00:55:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.547+0000 7efcd8cc1700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd41088c0 con 0x7efcd40fe480 2024-02-23T00:55:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.547+0000 7efcd1d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efcd40fdb20 0x7efcd410e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.547+0000 7efcd2d9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efcd40fe480 0x7efcd41080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.547+0000 7efcd1d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efcd40fdb20 0x7efcd410e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56728/0 (socket says 172.21.15.47:56728) 2024-02-23T00:55:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.547+0000 7efcd1d9b700 1 -- 172.21.15.47:0/4105778346 learned_addr learned my addr 172.21.15.47:0/4105778346 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:55:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.547+0000 7efcd259c700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcd40f8390 0x7efcd410deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.547+0000 7efcd1d9b700 1 -- 172.21.15.47:0/4105778346 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efcd40fe480 msgr2=0x7efcd41080a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.547+0000 7efcd1d9b700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efcd40fe480 0x7efcd41080a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.547+0000 7efcd1d9b700 1 -- 172.21.15.47:0/4105778346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcd40f8390 msgr2=0x7efcd410deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.547+0000 7efcd1d9b700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcd40f8390 0x7efcd410deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.547+0000 7efcd1d9b700 1 -- 172.21.15.47:0/4105778346 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efccc0097c0 con 0x7efcd40fdb20 2024-02-23T00:55:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.547+0000 7efcd1d9b700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efcd40fdb20 0x7efcd410e3a0 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 rx=0x7efcc80044e0 tx=0x7efcc8009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.547+0000 7efcc37fe700 10 client.?.objecter ms_handle_connect 0x7efcd40fdb20 2024-02-23T00:55:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.547+0000 7efcc37fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:55:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.548+0000 7efcc37fe700 1 -- 172.21.15.47:0/4105778346 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc8018070 con 0x7efcd40fdb20 2024-02-23T00:55:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.548+0000 7efcc37fe700 1 -- 172.21.15.47:0/4105778346 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7efcc8013e50 con 0x7efcd40fdb20 2024-02-23T00:55:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.548+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc8005040 con 0x7efcd40fdb20 2024-02-23T00:55:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.548+0000 7efcc37fe700 1 -- 172.21.15.47:0/4105778346 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc801db60 con 0x7efcd40fdb20 2024-02-23T00:55:02.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.548+0000 7efcd8cc1700 10 client.31755.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:55:02.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.548+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd4108de0 con 0x7efcd40fdb20 2024-02-23T00:55:02.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.548+0000 7efcd8cc1700 1 librados: init done 2024-02-23T00:55:02.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.548+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7efcd40606b0 con 0x7efcd40fdb20 2024-02-23T00:55:02.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.549+0000 7efcc37fe700 1 -- 172.21.15.47:0/4105778346 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7efcc8009240 con 0x7efcd40fdb20 2024-02-23T00:55:02.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.550+0000 7efcc37fe700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7efcbc062cc0 0x7efcbc065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:02.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.550+0000 7efcc37fe700 1 -- 172.21.15.47:0/4105778346 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(754..754 src has 621..754) v4 ==== 9807+0+0 (secure 0 0 0) 0x7efcc807f260 con 0x7efcd40fdb20 2024-02-23T00:55:02.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.550+0000 7efcc37fe700 10 client.31755.objecter ms_dispatch 0x7efcd4001550 osd_map(754..754 src has 621..754) v4 2024-02-23T00:55:02.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.550+0000 7efcc37fe700 3 client.31755.objecter handle_osd_map got epochs [754,754] > 0 2024-02-23T00:55:02.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.550+0000 7efcc37fe700 3 client.31755.objecter handle_osd_map decoding full epoch 754 2024-02-23T00:55:02.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.550+0000 7efcd259c700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7efcbc062cc0 0x7efcbc065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:02.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.550+0000 7efcc37fe700 20 client.31755.objecter dump_active .. 0 homeless 2024-02-23T00:55:02.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.550+0000 7efcc37fe700 1 -- 172.21.15.47:0/4105778346 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v754) v1 ==== 54+0+16 (secure 0 0 0) 0x7efcc807f740 con 0x7efcd40fdb20 2024-02-23T00:55:02.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.550+0000 7efcc37fe700 10 client.31755.objecter ms_handle_connect 0x7efcbc062cc0 2024-02-23T00:55:02.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.550+0000 7efcd259c700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7efcbc062cc0 0x7efcbc065180 secure :-1 s=READY pgs=3196 cs=0 l=1 rev1=1 rx=0x7efcc4004dc0 tx=0x7efcc400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:02.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.550+0000 7efcd8cc1700 10 client.31755.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:02.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.550+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_assign -1 1 2024-02-23T00:55:02.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.550+0000 7efcd8cc1700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7efcd4028540 0x7efcd40fe880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:02.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.550+0000 7efcd8cc1700 20 client.31755.objecter _get_session s=0x7efcd4106e40 osd=0 3 2024-02-23T00:55:02.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.550+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd4106e40 osd=0 3 2024-02-23T00:55:02.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.550+0000 7efcd8cc1700 20 client.31755.objecter _get_session s=0x7efcd4106e40 osd=0 3 2024-02-23T00:55:02.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.550+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_remove -1 1 2024-02-23T00:55:02.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.550+0000 7efcd8cc1700 20 client.31755.objecter get_session s=0x7efcd4106e40 osd=0 3 2024-02-23T00:55:02.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.550+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_assign 0 1 2024-02-23T00:55:02.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.550+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd4106e40 osd=0 4 2024-02-23T00:55:02.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.550+0000 7efcd8cc1700 10 client.31755.objecter _send_command 1 2024-02-23T00:55:02.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.550+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efcd40ffef0 con 0x7efcd4028540 2024-02-23T00:55:02.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.550+0000 7efcd2d9d700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7efcd4028540 0x7efcd40fe880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:02.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.550+0000 7efcc37fe700 10 client.31755.objecter ms_handle_connect 0x7efcd4028540 2024-02-23T00:55:02.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.550+0000 7efcd2d9d700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7efcd4028540 0x7efcd40fe880 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:02.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.551+0000 7efcc37fe700 1 -- 172.21.15.47:0/4105778346 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efcd40ffef0 con 0x7efcd4028540 2024-02-23T00:55:02.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.551+0000 7efcc37fe700 10 client.31755.objecter ms_dispatch 0x7efcd4001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:55:02.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.551+0000 7efcc37fe700 10 client.31755.objecter _finish_command 1 = system:0 2024-02-23T00:55:02.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.551+0000 7efcc37fe700 20 client.31755.objecter put_session s=0x7efcd4106e40 osd=0 4 2024-02-23T00:55:02.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.551+0000 7efcc37fe700 15 client.31755.objecter _session_command_op_remove 0 1 2024-02-23T00:55:02.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:01.312681+0000 mon.a (mon.0) 3617 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_period","val": "600"}]': finished 2024-02-23T00:55:02.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:01.312756+0000 mon.a (mon.0) 3618 : cluster [DBG] osdmap e753: 8 total, 8 up, 7 in 2024-02-23T00:55:02.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:01.314436+0000 mon.c (mon.2) 461 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-23T00:55:02.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:01.314806+0000 mon.a (mon.0) 3619 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-23T00:55:02.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:01.394150+0000 mgr.y (mgr.14517) 643 : cluster [DBG] pgmap v1143: 184 pgs: 1 active+remapped+backfill_toofull, 1 remapped+peering, 1 active+recovering, 181 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 3/786 objects misplaced (0.382%) 2024-02-23T00:55:02.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:01.312681+0000 mon.a (mon.0) 3617 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_period","val": "600"}]': finished 2024-02-23T00:55:02.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:01.312756+0000 mon.a (mon.0) 3618 : cluster [DBG] osdmap e753: 8 total, 8 up, 7 in 2024-02-23T00:55:02.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:01.314436+0000 mon.c (mon.2) 461 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-23T00:55:02.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:01.314806+0000 mon.a (mon.0) 3619 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-23T00:55:02.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:01.394150+0000 mgr.y (mgr.14517) 643 : cluster [DBG] pgmap v1143: 184 pgs: 1 active+remapped+backfill_toofull, 1 remapped+peering, 1 active+recovering, 181 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 3/786 objects misplaced (0.382%) 2024-02-23T00:55:02.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.566+0000 7efcd8cc1700 10 client.31755.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:55:02.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.566+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_assign -1 2 2024-02-23T00:55:02.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.566+0000 7efcd8cc1700 20 client.31755.objecter _get_session s=0x7efcd4106e40 osd=0 3 2024-02-23T00:55:02.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.566+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd4106e40 osd=0 3 2024-02-23T00:55:02.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.566+0000 7efcd8cc1700 20 client.31755.objecter _get_session s=0x7efcd4106e40 osd=0 3 2024-02-23T00:55:02.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.566+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_remove -1 2 2024-02-23T00:55:02.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.566+0000 7efcd8cc1700 20 client.31755.objecter get_session s=0x7efcd4106e40 osd=0 3 2024-02-23T00:55:02.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.566+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_assign 0 2 2024-02-23T00:55:02.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.566+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd4106e40 osd=0 4 2024-02-23T00:55:02.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.566+0000 7efcd8cc1700 10 client.31755.objecter _send_command 2 2024-02-23T00:55:02.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.566+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7efcd4100060 con 0x7efcd4028540 2024-02-23T00:55:02.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.568+0000 7efcc37fe700 1 -- 172.21.15.47:0/4105778346 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7efcd4100060 con 0x7efcd4028540 2024-02-23T00:55:02.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.568+0000 7efcc37fe700 10 client.31755.objecter ms_dispatch 0x7efcd4001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:55:02.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.568+0000 7efcc37fe700 10 client.31755.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:55:02.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.568+0000 7efcc37fe700 20 client.31755.objecter put_session s=0x7efcd4106e40 osd=0 4 2024-02-23T00:55:02.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.568+0000 7efcc37fe700 15 client.31755.objecter _session_command_op_remove 0 2 2024-02-23T00:55:02.570 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:55:02.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.568+0000 7efcd8cc1700 10 client.31755.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:02.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.568+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_assign -1 3 2024-02-23T00:55:02.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.568+0000 7efcd8cc1700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7efcd40ff3f0 0x7efcd40ff7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:02.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.568+0000 7efcd8cc1700 20 client.31755.objecter _get_session s=0x7efcd40fed70 osd=1 3 2024-02-23T00:55:02.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.568+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd40fed70 osd=1 3 2024-02-23T00:55:02.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.568+0000 7efcd8cc1700 20 client.31755.objecter _get_session s=0x7efcd40fed70 osd=1 3 2024-02-23T00:55:02.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.568+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_remove -1 3 2024-02-23T00:55:02.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.568+0000 7efcd8cc1700 20 client.31755.objecter get_session s=0x7efcd40fed70 osd=1 3 2024-02-23T00:55:02.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.568+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_assign 1 3 2024-02-23T00:55:02.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.568+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd40fed70 osd=1 4 2024-02-23T00:55:02.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.568+0000 7efcd8cc1700 10 client.31755.objecter _send_command 3 2024-02-23T00:55:02.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.568+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7efcd406cf20 con 0x7efcd40ff3f0 2024-02-23T00:55:02.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.568+0000 7efcd259c700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7efcd40ff3f0 0x7efcd40ff7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:02.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.568+0000 7efcc37fe700 10 client.31755.objecter ms_handle_connect 0x7efcd40ff3f0 2024-02-23T00:55:02.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.568+0000 7efcd259c700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7efcd40ff3f0 0x7efcd40ff7d0 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:02.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.569+0000 7efcc37fe700 1 -- 172.21.15.47:0/4105778346 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efcd406cf20 con 0x7efcd40ff3f0 2024-02-23T00:55:02.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.569+0000 7efcc37fe700 10 client.31755.objecter ms_dispatch 0x7efcd4001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:55:02.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.569+0000 7efcc37fe700 10 client.31755.objecter _finish_command 3 = system:0 2024-02-23T00:55:02.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.569+0000 7efcc37fe700 20 client.31755.objecter put_session s=0x7efcd40fed70 osd=1 4 2024-02-23T00:55:02.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.569+0000 7efcc37fe700 15 client.31755.objecter _session_command_op_remove 1 3 2024-02-23T00:55:02.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.584+0000 7efcd8cc1700 10 client.31755.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:55:02.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.584+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_assign -1 4 2024-02-23T00:55:02.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.584+0000 7efcd8cc1700 20 client.31755.objecter _get_session s=0x7efcd40fed70 osd=1 3 2024-02-23T00:55:02.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.584+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd40fed70 osd=1 3 2024-02-23T00:55:02.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.584+0000 7efcd8cc1700 20 client.31755.objecter _get_session s=0x7efcd40fed70 osd=1 3 2024-02-23T00:55:02.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.584+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_remove -1 4 2024-02-23T00:55:02.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.584+0000 7efcd8cc1700 20 client.31755.objecter get_session s=0x7efcd40fed70 osd=1 3 2024-02-23T00:55:02.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.584+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_assign 1 4 2024-02-23T00:55:02.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.584+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd40fed70 osd=1 4 2024-02-23T00:55:02.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.584+0000 7efcd8cc1700 10 client.31755.objecter _send_command 4 2024-02-23T00:55:02.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.584+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7efcd40ffcc0 con 0x7efcd40ff3f0 2024-02-23T00:55:02.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.585+0000 7efcc37fe700 1 -- 172.21.15.47:0/4105778346 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7efcd40ffcc0 con 0x7efcd40ff3f0 2024-02-23T00:55:02.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.585+0000 7efcc37fe700 10 client.31755.objecter ms_dispatch 0x7efcd4001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:55:02.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.585+0000 7efcc37fe700 10 client.31755.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:55:02.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.585+0000 7efcc37fe700 20 client.31755.objecter put_session s=0x7efcd40fed70 osd=1 4 2024-02-23T00:55:02.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.585+0000 7efcc37fe700 15 client.31755.objecter _session_command_op_remove 1 4 2024-02-23T00:55:02.588 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:55:02.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.585+0000 7efcd8cc1700 10 client.31755.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:02.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.585+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_assign -1 5 2024-02-23T00:55:02.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.585+0000 7efcd8cc1700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7efcd40280a0 0x7efcd411e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:02.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.585+0000 7efcd8cc1700 20 client.31755.objecter _get_session s=0x7efcd40595a0 osd=2 3 2024-02-23T00:55:02.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.585+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd40595a0 osd=2 3 2024-02-23T00:55:02.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.585+0000 7efcd8cc1700 20 client.31755.objecter _get_session s=0x7efcd40595a0 osd=2 3 2024-02-23T00:55:02.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.585+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_remove -1 5 2024-02-23T00:55:02.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.585+0000 7efcd8cc1700 20 client.31755.objecter get_session s=0x7efcd40595a0 osd=2 3 2024-02-23T00:55:02.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.585+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_assign 2 5 2024-02-23T00:55:02.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.585+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd40595a0 osd=2 4 2024-02-23T00:55:02.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.585+0000 7efcd8cc1700 10 client.31755.objecter _send_command 5 2024-02-23T00:55:02.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.585+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7efcd4120ee0 con 0x7efcd40280a0 2024-02-23T00:55:02.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.586+0000 7efcd1d9b700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7efcd40280a0 0x7efcd411e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:02.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.586+0000 7efcc37fe700 10 client.31755.objecter ms_handle_connect 0x7efcd40280a0 2024-02-23T00:55:02.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.586+0000 7efcd1d9b700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7efcd40280a0 0x7efcd411e290 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:02.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.586+0000 7efcc37fe700 1 -- 172.21.15.47:0/4105778346 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efcd4120ee0 con 0x7efcd40280a0 2024-02-23T00:55:02.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.586+0000 7efcc37fe700 10 client.31755.objecter ms_dispatch 0x7efcd4001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:55:02.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.586+0000 7efcc37fe700 10 client.31755.objecter _finish_command 5 = system:0 2024-02-23T00:55:02.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.586+0000 7efcc37fe700 20 client.31755.objecter put_session s=0x7efcd40595a0 osd=2 4 2024-02-23T00:55:02.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.586+0000 7efcc37fe700 15 client.31755.objecter _session_command_op_remove 2 5 2024-02-23T00:55:02.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.601+0000 7efcd8cc1700 10 client.31755.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:55:02.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.601+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_assign -1 6 2024-02-23T00:55:02.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.601+0000 7efcd8cc1700 20 client.31755.objecter _get_session s=0x7efcd40595a0 osd=2 3 2024-02-23T00:55:02.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.601+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd40595a0 osd=2 3 2024-02-23T00:55:02.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.601+0000 7efcd8cc1700 20 client.31755.objecter _get_session s=0x7efcd40595a0 osd=2 3 2024-02-23T00:55:02.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.601+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_remove -1 6 2024-02-23T00:55:02.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.601+0000 7efcd8cc1700 20 client.31755.objecter get_session s=0x7efcd40595a0 osd=2 3 2024-02-23T00:55:02.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.601+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_assign 2 6 2024-02-23T00:55:02.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.601+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd40595a0 osd=2 4 2024-02-23T00:55:02.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.601+0000 7efcd8cc1700 10 client.31755.objecter _send_command 6 2024-02-23T00:55:02.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.601+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7efcd4027600 con 0x7efcd40280a0 2024-02-23T00:55:02.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.602+0000 7efcc37fe700 1 -- 172.21.15.47:0/4105778346 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7efcd4027600 con 0x7efcd40280a0 2024-02-23T00:55:02.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.602+0000 7efcc37fe700 10 client.31755.objecter ms_dispatch 0x7efcd4001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:55:02.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.602+0000 7efcc37fe700 10 client.31755.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:55:02.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.602+0000 7efcc37fe700 20 client.31755.objecter put_session s=0x7efcd40595a0 osd=2 4 2024-02-23T00:55:02.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.602+0000 7efcc37fe700 15 client.31755.objecter _session_command_op_remove 2 6 2024-02-23T00:55:02.605 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:55:02.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.602+0000 7efcd8cc1700 10 client.31755.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:02.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.602+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_assign -1 7 2024-02-23T00:55:02.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.602+0000 7efcd8cc1700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7efcd411bde0 0x7efcd411f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:02.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.602+0000 7efcd8cc1700 20 client.31755.objecter _get_session s=0x7efcd40279e0 osd=3 3 2024-02-23T00:55:02.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.602+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd40279e0 osd=3 3 2024-02-23T00:55:02.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.602+0000 7efcd8cc1700 20 client.31755.objecter _get_session s=0x7efcd40279e0 osd=3 3 2024-02-23T00:55:02.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.602+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_remove -1 7 2024-02-23T00:55:02.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.603+0000 7efcd8cc1700 20 client.31755.objecter get_session s=0x7efcd40279e0 osd=3 3 2024-02-23T00:55:02.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.603+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_assign 3 7 2024-02-23T00:55:02.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.603+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd40279e0 osd=3 4 2024-02-23T00:55:02.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.603+0000 7efcd8cc1700 10 client.31755.objecter _send_command 7 2024-02-23T00:55:02.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.603+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7efcd411ff50 con 0x7efcd411bde0 2024-02-23T00:55:02.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.603+0000 7efcd2d9d700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7efcd411bde0 0x7efcd411f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:02.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.603+0000 7efcd2d9d700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7efcd411bde0 0x7efcd411f900 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:02.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.603+0000 7efcc37fe700 10 client.31755.objecter ms_handle_connect 0x7efcd411bde0 2024-02-23T00:55:02.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.603+0000 7efcc37fe700 1 -- 172.21.15.47:0/4105778346 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efcd411ff50 con 0x7efcd411bde0 2024-02-23T00:55:02.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.603+0000 7efcc37fe700 10 client.31755.objecter ms_dispatch 0x7efcd4001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:55:02.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.603+0000 7efcc37fe700 10 client.31755.objecter _finish_command 7 = system:0 2024-02-23T00:55:02.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.603+0000 7efcc37fe700 20 client.31755.objecter put_session s=0x7efcd40279e0 osd=3 4 2024-02-23T00:55:02.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.603+0000 7efcc37fe700 15 client.31755.objecter _session_command_op_remove 3 7 2024-02-23T00:55:02.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.618+0000 7efcd8cc1700 10 client.31755.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:55:02.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.618+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_assign -1 8 2024-02-23T00:55:02.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.618+0000 7efcd8cc1700 20 client.31755.objecter _get_session s=0x7efcd40279e0 osd=3 3 2024-02-23T00:55:02.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.618+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd40279e0 osd=3 3 2024-02-23T00:55:02.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.618+0000 7efcd8cc1700 20 client.31755.objecter _get_session s=0x7efcd40279e0 osd=3 3 2024-02-23T00:55:02.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.618+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_remove -1 8 2024-02-23T00:55:02.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.618+0000 7efcd8cc1700 20 client.31755.objecter get_session s=0x7efcd40279e0 osd=3 3 2024-02-23T00:55:02.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.618+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_assign 3 8 2024-02-23T00:55:02.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.618+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd40279e0 osd=3 4 2024-02-23T00:55:02.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.618+0000 7efcd8cc1700 10 client.31755.objecter _send_command 8 2024-02-23T00:55:02.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.618+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7efcd411b340 con 0x7efcd411bde0 2024-02-23T00:55:02.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.619+0000 7efcc37fe700 1 -- 172.21.15.47:0/4105778346 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7efcd411b340 con 0x7efcd411bde0 2024-02-23T00:55:02.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.619+0000 7efcc37fe700 10 client.31755.objecter ms_dispatch 0x7efcd4001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:55:02.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.619+0000 7efcc37fe700 10 client.31755.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:55:02.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.619+0000 7efcc37fe700 20 client.31755.objecter put_session s=0x7efcd40279e0 osd=3 4 2024-02-23T00:55:02.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.619+0000 7efcc37fe700 15 client.31755.objecter _session_command_op_remove 3 8 2024-02-23T00:55:02.622 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:55:02.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.620+0000 7efcd8cc1700 10 client.31755.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:02.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.620+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_assign -1 9 2024-02-23T00:55:02.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.620+0000 7efcd8cc1700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7efcd4123060 0x7efcd4125450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:02.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.620+0000 7efcd8cc1700 20 client.31755.objecter _get_session s=0x7efcd411b500 osd=4 3 2024-02-23T00:55:02.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.620+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd411b500 osd=4 3 2024-02-23T00:55:02.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.620+0000 7efcd8cc1700 20 client.31755.objecter _get_session s=0x7efcd411b500 osd=4 3 2024-02-23T00:55:02.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.620+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_remove -1 9 2024-02-23T00:55:02.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.620+0000 7efcd8cc1700 20 client.31755.objecter get_session s=0x7efcd411b500 osd=4 3 2024-02-23T00:55:02.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.620+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_assign 4 9 2024-02-23T00:55:02.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.620+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd411b500 osd=4 4 2024-02-23T00:55:02.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.620+0000 7efcd8cc1700 10 client.31755.objecter _send_command 9 2024-02-23T00:55:02.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.620+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7efcd411b670 con 0x7efcd4123060 2024-02-23T00:55:02.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.620+0000 7efcd259c700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7efcd4123060 0x7efcd4125450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:02.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.620+0000 7efcc37fe700 10 client.31755.objecter ms_handle_connect 0x7efcd4123060 2024-02-23T00:55:02.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.620+0000 7efcd259c700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7efcd4123060 0x7efcd4125450 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:02.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.621+0000 7efcc37fe700 1 -- 172.21.15.47:0/4105778346 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efcd411b670 con 0x7efcd4123060 2024-02-23T00:55:02.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.621+0000 7efcc37fe700 10 client.31755.objecter ms_dispatch 0x7efcd4001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:55:02.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.621+0000 7efcc37fe700 10 client.31755.objecter _finish_command 9 = system:0 2024-02-23T00:55:02.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.621+0000 7efcc37fe700 20 client.31755.objecter put_session s=0x7efcd411b500 osd=4 4 2024-02-23T00:55:02.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.621+0000 7efcc37fe700 15 client.31755.objecter _session_command_op_remove 4 9 2024-02-23T00:55:02.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.636+0000 7efcd8cc1700 10 client.31755.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:55:02.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.636+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_assign -1 10 2024-02-23T00:55:02.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.636+0000 7efcd8cc1700 20 client.31755.objecter _get_session s=0x7efcd411b500 osd=4 3 2024-02-23T00:55:02.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.636+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd411b500 osd=4 3 2024-02-23T00:55:02.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.636+0000 7efcd8cc1700 20 client.31755.objecter _get_session s=0x7efcd411b500 osd=4 3 2024-02-23T00:55:02.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.636+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_remove -1 10 2024-02-23T00:55:02.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.636+0000 7efcd8cc1700 20 client.31755.objecter get_session s=0x7efcd411b500 osd=4 3 2024-02-23T00:55:02.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.636+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_assign 4 10 2024-02-23T00:55:02.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.636+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd411b500 osd=4 4 2024-02-23T00:55:02.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.636+0000 7efcd8cc1700 10 client.31755.objecter _send_command 10 2024-02-23T00:55:02.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.636+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7efcd411b670 con 0x7efcd4123060 2024-02-23T00:55:02.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.638+0000 7efcc37fe700 1 -- 172.21.15.47:0/4105778346 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7efcd411b670 con 0x7efcd4123060 2024-02-23T00:55:02.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.638+0000 7efcc37fe700 10 client.31755.objecter ms_dispatch 0x7efcd4001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:55:02.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.638+0000 7efcc37fe700 10 client.31755.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:55:02.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.638+0000 7efcc37fe700 20 client.31755.objecter put_session s=0x7efcd411b500 osd=4 4 2024-02-23T00:55:02.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.638+0000 7efcc37fe700 15 client.31755.objecter _session_command_op_remove 4 10 2024-02-23T00:55:02.640 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:55:02.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.638+0000 7efcd8cc1700 10 client.31755.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:02.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.638+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_assign -1 11 2024-02-23T00:55:02.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.638+0000 7efcd8cc1700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7efcd4125e50 0x7efcd4128260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:02.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.638+0000 7efcd8cc1700 20 client.31755.objecter _get_session s=0x7efcd411b670 osd=5 3 2024-02-23T00:55:02.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.638+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd411b670 osd=5 3 2024-02-23T00:55:02.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.638+0000 7efcd8cc1700 20 client.31755.objecter _get_session s=0x7efcd411b670 osd=5 3 2024-02-23T00:55:02.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.638+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_remove -1 11 2024-02-23T00:55:02.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.638+0000 7efcd8cc1700 20 client.31755.objecter get_session s=0x7efcd411b670 osd=5 3 2024-02-23T00:55:02.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.638+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_assign 5 11 2024-02-23T00:55:02.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.638+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd411b670 osd=5 4 2024-02-23T00:55:02.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.638+0000 7efcd8cc1700 10 client.31755.objecter _send_command 11 2024-02-23T00:55:02.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.638+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7efcd412b0a0 con 0x7efcd4125e50 2024-02-23T00:55:02.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.638+0000 7efcd1d9b700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7efcd4125e50 0x7efcd4128260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:02.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.638+0000 7efcd1d9b700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7efcd4125e50 0x7efcd4128260 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:02.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.639+0000 7efcc37fe700 10 client.31755.objecter ms_handle_connect 0x7efcd4125e50 2024-02-23T00:55:02.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.639+0000 7efcc37fe700 1 -- 172.21.15.47:0/4105778346 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efcd412b0a0 con 0x7efcd4125e50 2024-02-23T00:55:02.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.639+0000 7efcc37fe700 10 client.31755.objecter ms_dispatch 0x7efcd4001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:55:02.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.639+0000 7efcc37fe700 10 client.31755.objecter _finish_command 11 = system:0 2024-02-23T00:55:02.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.639+0000 7efcc37fe700 20 client.31755.objecter put_session s=0x7efcd411b670 osd=5 4 2024-02-23T00:55:02.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.639+0000 7efcc37fe700 15 client.31755.objecter _session_command_op_remove 5 11 2024-02-23T00:55:02.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.654+0000 7efcd8cc1700 10 client.31755.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:55:02.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.654+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_assign -1 12 2024-02-23T00:55:02.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.654+0000 7efcd8cc1700 20 client.31755.objecter _get_session s=0x7efcd411b670 osd=5 3 2024-02-23T00:55:02.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.654+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd411b670 osd=5 3 2024-02-23T00:55:02.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.654+0000 7efcd8cc1700 20 client.31755.objecter _get_session s=0x7efcd411b670 osd=5 3 2024-02-23T00:55:02.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.654+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_remove -1 12 2024-02-23T00:55:02.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.654+0000 7efcd8cc1700 20 client.31755.objecter get_session s=0x7efcd411b670 osd=5 3 2024-02-23T00:55:02.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.654+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_assign 5 12 2024-02-23T00:55:02.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.654+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd411b670 osd=5 4 2024-02-23T00:55:02.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.654+0000 7efcd8cc1700 10 client.31755.objecter _send_command 12 2024-02-23T00:55:02.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.654+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7efcd412b0a0 con 0x7efcd4125e50 2024-02-23T00:55:02.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.655+0000 7efcc37fe700 1 -- 172.21.15.47:0/4105778346 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7efcd412b0a0 con 0x7efcd4125e50 2024-02-23T00:55:02.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.655+0000 7efcc37fe700 10 client.31755.objecter ms_dispatch 0x7efcd4001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:55:02.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.655+0000 7efcc37fe700 10 client.31755.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:55:02.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.655+0000 7efcc37fe700 20 client.31755.objecter put_session s=0x7efcd411b670 osd=5 4 2024-02-23T00:55:02.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.655+0000 7efcc37fe700 15 client.31755.objecter _session_command_op_remove 5 12 2024-02-23T00:55:02.658 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:55:02.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.656+0000 7efcd8cc1700 10 client.31755.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:02.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.656+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_assign -1 13 2024-02-23T00:55:02.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.656+0000 7efcd8cc1700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7efcd412a3d0 0x7efcd412a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:02.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.656+0000 7efcd8cc1700 20 client.31755.objecter _get_session s=0x7efcd412b0a0 osd=6 3 2024-02-23T00:55:02.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.656+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd412b0a0 osd=6 3 2024-02-23T00:55:02.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.656+0000 7efcd8cc1700 20 client.31755.objecter _get_session s=0x7efcd412b0a0 osd=6 3 2024-02-23T00:55:02.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.656+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_remove -1 13 2024-02-23T00:55:02.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.656+0000 7efcd8cc1700 20 client.31755.objecter get_session s=0x7efcd412b0a0 osd=6 3 2024-02-23T00:55:02.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.656+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_assign 6 13 2024-02-23T00:55:02.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.656+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd412b0a0 osd=6 4 2024-02-23T00:55:02.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.656+0000 7efcd8cc1700 10 client.31755.objecter _send_command 13 2024-02-23T00:55:02.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.656+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7efcd412e3d0 con 0x7efcd412a3d0 2024-02-23T00:55:02.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.656+0000 7efcd2d9d700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7efcd412a3d0 0x7efcd412a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:02.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.656+0000 7efcc37fe700 10 client.31755.objecter ms_handle_connect 0x7efcd412a3d0 2024-02-23T00:55:02.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.656+0000 7efcd2d9d700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7efcd412a3d0 0x7efcd412a850 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:02.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.657+0000 7efcc37fe700 1 -- 172.21.15.47:0/4105778346 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efcd412e3d0 con 0x7efcd412a3d0 2024-02-23T00:55:02.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.657+0000 7efcc37fe700 10 client.31755.objecter ms_dispatch 0x7efcd4001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:55:02.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.657+0000 7efcc37fe700 10 client.31755.objecter _finish_command 13 = system:0 2024-02-23T00:55:02.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.657+0000 7efcc37fe700 20 client.31755.objecter put_session s=0x7efcd412b0a0 osd=6 4 2024-02-23T00:55:02.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.657+0000 7efcc37fe700 15 client.31755.objecter _session_command_op_remove 6 13 2024-02-23T00:55:02.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.671+0000 7efcd8cc1700 10 client.31755.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:55:02.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.671+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_assign -1 14 2024-02-23T00:55:02.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.671+0000 7efcd8cc1700 20 client.31755.objecter _get_session s=0x7efcd412b0a0 osd=6 3 2024-02-23T00:55:02.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.671+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd412b0a0 osd=6 3 2024-02-23T00:55:02.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.671+0000 7efcd8cc1700 20 client.31755.objecter _get_session s=0x7efcd412b0a0 osd=6 3 2024-02-23T00:55:02.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.671+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_remove -1 14 2024-02-23T00:55:02.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.671+0000 7efcd8cc1700 20 client.31755.objecter get_session s=0x7efcd412b0a0 osd=6 3 2024-02-23T00:55:02.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.671+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_assign 6 14 2024-02-23T00:55:02.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.671+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd412b0a0 osd=6 4 2024-02-23T00:55:02.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.671+0000 7efcd8cc1700 10 client.31755.objecter _send_command 14 2024-02-23T00:55:02.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.671+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7efcd4128860 con 0x7efcd412a3d0 2024-02-23T00:55:02.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.673+0000 7efcc37fe700 1 -- 172.21.15.47:0/4105778346 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7efcd4128860 con 0x7efcd412a3d0 2024-02-23T00:55:02.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.673+0000 7efcc37fe700 10 client.31755.objecter ms_dispatch 0x7efcd4001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:55:02.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.673+0000 7efcc37fe700 10 client.31755.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:55:02.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.673+0000 7efcc37fe700 20 client.31755.objecter put_session s=0x7efcd412b0a0 osd=6 4 2024-02-23T00:55:02.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.673+0000 7efcc37fe700 15 client.31755.objecter _session_command_op_remove 6 14 2024-02-23T00:55:02.676 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:55:02.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.674+0000 7efcd8cc1700 10 client.31755.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:02.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.674+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_assign -1 15 2024-02-23T00:55:02.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.674+0000 7efcd8cc1700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7efcd4128ee0 0x7efcd4129340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:02.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.674+0000 7efcd8cc1700 20 client.31755.objecter _get_session s=0x7efcd4128860 osd=7 3 2024-02-23T00:55:02.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.674+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd4128860 osd=7 3 2024-02-23T00:55:02.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.674+0000 7efcd8cc1700 20 client.31755.objecter _get_session s=0x7efcd4128860 osd=7 3 2024-02-23T00:55:02.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.674+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_remove -1 15 2024-02-23T00:55:02.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.674+0000 7efcd8cc1700 20 client.31755.objecter get_session s=0x7efcd4128860 osd=7 3 2024-02-23T00:55:02.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.674+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_assign 7 15 2024-02-23T00:55:02.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.674+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd4128860 osd=7 4 2024-02-23T00:55:02.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.674+0000 7efcd8cc1700 10 client.31755.objecter _send_command 15 2024-02-23T00:55:02.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.674+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7efcd41299d0 con 0x7efcd4128ee0 2024-02-23T00:55:02.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.674+0000 7efcd259c700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7efcd4128ee0 0x7efcd4129340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:02.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.674+0000 7efcc37fe700 10 client.31755.objecter ms_handle_connect 0x7efcd4128ee0 2024-02-23T00:55:02.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.674+0000 7efcd259c700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7efcd4128ee0 0x7efcd4129340 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:02.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.675+0000 7efcc37fe700 1 -- 172.21.15.47:0/4105778346 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efcd41299d0 con 0x7efcd4128ee0 2024-02-23T00:55:02.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.675+0000 7efcc37fe700 10 client.31755.objecter ms_dispatch 0x7efcd4001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:55:02.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.675+0000 7efcc37fe700 10 client.31755.objecter _finish_command 15 = system:0 2024-02-23T00:55:02.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.675+0000 7efcc37fe700 20 client.31755.objecter put_session s=0x7efcd4128860 osd=7 4 2024-02-23T00:55:02.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.675+0000 7efcc37fe700 15 client.31755.objecter _session_command_op_remove 7 15 2024-02-23T00:55:02.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.690+0000 7efcd8cc1700 10 client.31755.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:55:02.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.690+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_assign -1 16 2024-02-23T00:55:02.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.690+0000 7efcd8cc1700 20 client.31755.objecter _get_session s=0x7efcd4128860 osd=7 3 2024-02-23T00:55:02.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.690+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd4128860 osd=7 3 2024-02-23T00:55:02.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.690+0000 7efcd8cc1700 20 client.31755.objecter _get_session s=0x7efcd4128860 osd=7 3 2024-02-23T00:55:02.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.690+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_remove -1 16 2024-02-23T00:55:02.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.690+0000 7efcd8cc1700 20 client.31755.objecter get_session s=0x7efcd4128860 osd=7 3 2024-02-23T00:55:02.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.690+0000 7efcd8cc1700 15 client.31755.objecter _session_command_op_assign 7 16 2024-02-23T00:55:02.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.690+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd4128860 osd=7 4 2024-02-23T00:55:02.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.690+0000 7efcd8cc1700 10 client.31755.objecter _send_command 16 2024-02-23T00:55:02.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.690+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7efcd41299d0 con 0x7efcd4128ee0 2024-02-23T00:55:02.694 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:55:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:55:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:55:02.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:01.312681+0000 mon.a (mon.0) 3617 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_period","val": "600"}]': finished 2024-02-23T00:55:02.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:01.312756+0000 mon.a (mon.0) 3618 : cluster [DBG] osdmap e753: 8 total, 8 up, 7 in 2024-02-23T00:55:02.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:01.314436+0000 mon.c (mon.2) 461 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-23T00:55:02.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:01.314806+0000 mon.a (mon.0) 3619 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-23T00:55:02.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:01.394150+0000 mgr.y (mgr.14517) 643 : cluster [DBG] pgmap v1143: 184 pgs: 1 active+remapped+backfill_toofull, 1 remapped+peering, 1 active+recovering, 181 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 3/786 objects misplaced (0.382%) 2024-02-23T00:55:02.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.692+0000 7efcc37fe700 1 -- 172.21.15.47:0/4105778346 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7efcd41299d0 con 0x7efcd4128ee0 2024-02-23T00:55:02.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.692+0000 7efcc37fe700 10 client.31755.objecter ms_dispatch 0x7efcd4001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:55:02.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.692+0000 7efcc37fe700 10 client.31755.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:55:02.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.692+0000 7efcc37fe700 20 client.31755.objecter put_session s=0x7efcd4128860 osd=7 4 2024-02-23T00:55:02.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.692+0000 7efcc37fe700 15 client.31755.objecter _session_command_op_remove 7 16 2024-02-23T00:55:02.695 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:55:02.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.692+0000 7efcd8cc1700 10 librados: watch_flush enter 2024-02-23T00:55:02.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.692+0000 7efcd8cc1700 10 librados: watch_flush exit 2024-02-23T00:55:02.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.692+0000 7efcd8cc1700 10 client.31755.objecter close_session for osd.0 2024-02-23T00:55:02.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.692+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7efcd4028540 msgr2=0x7efcd40fe880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:02.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.692+0000 7efcd8cc1700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7efcd4028540 0x7efcd40fe880 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:02.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd4106e40 osd=0 1 2024-02-23T00:55:02.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 10 client.31755.objecter close_session for osd.1 2024-02-23T00:55:02.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7efcd40ff3f0 msgr2=0x7efcd40ff7d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:02.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7efcd40ff3f0 0x7efcd40ff7d0 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:02.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd40fed70 osd=1 1 2024-02-23T00:55:02.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 10 client.31755.objecter close_session for osd.2 2024-02-23T00:55:02.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7efcd40280a0 msgr2=0x7efcd411e290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:02.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7efcd40280a0 0x7efcd411e290 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:02.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd40595a0 osd=2 1 2024-02-23T00:55:02.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 10 client.31755.objecter close_session for osd.3 2024-02-23T00:55:02.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7efcd411bde0 msgr2=0x7efcd411f900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:02.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7efcd411bde0 0x7efcd411f900 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:02.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd40279e0 osd=3 1 2024-02-23T00:55:02.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 10 client.31755.objecter close_session for osd.4 2024-02-23T00:55:02.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7efcd4123060 msgr2=0x7efcd4125450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:02.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7efcd4123060 0x7efcd4125450 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:02.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd411b500 osd=4 1 2024-02-23T00:55:02.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 10 client.31755.objecter close_session for osd.5 2024-02-23T00:55:02.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7efcd4125e50 msgr2=0x7efcd4128260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:02.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7efcd4125e50 0x7efcd4128260 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:02.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd411b670 osd=5 1 2024-02-23T00:55:02.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 10 client.31755.objecter close_session for osd.6 2024-02-23T00:55:02.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7efcd412a3d0 msgr2=0x7efcd412a850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:02.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7efcd412a3d0 0x7efcd412a850 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:02.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd412b0a0 osd=6 1 2024-02-23T00:55:02.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 10 client.31755.objecter close_session for osd.7 2024-02-23T00:55:02.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7efcd4128ee0 msgr2=0x7efcd4129340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:02.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7efcd4128ee0 0x7efcd4129340 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:02.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 20 client.31755.objecter put_session s=0x7efcd4128860 osd=7 1 2024-02-23T00:55:02.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 20 client.31755.objecter shutdown clearing up homeless session... 2024-02-23T00:55:02.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 10 client.31755.objecter successfully canceled tick 2024-02-23T00:55:02.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd2d9d700 1 -- 172.21.15.47:0/4105778346 reap_dead start 2024-02-23T00:55:02.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd2d9d700 1 -- 172.21.15.47:0/4105778346 reap_dead start 2024-02-23T00:55:02.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7efcbc062cc0 msgr2=0x7efcbc065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:02.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7efcbc062cc0 0x7efcbc065180 secure :-1 s=READY pgs=3196 cs=0 l=1 rev1=1 rx=0x7efcc4004dc0 tx=0x7efcc400b040).stop 2024-02-23T00:55:02.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd2d9d700 1 -- 172.21.15.47:0/4105778346 reap_dead start 2024-02-23T00:55:02.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efcd40fdb20 msgr2=0x7efcd410e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:02.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 1 --2- 172.21.15.47:0/4105778346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efcd40fdb20 0x7efcd410e3a0 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 rx=0x7efcc80044e0 tx=0x7efcc8009cb0).stop 2024-02-23T00:55:02.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd2d9d700 1 -- 172.21.15.47:0/4105778346 reap_dead start 2024-02-23T00:55:02.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd2d9d700 1 -- 172.21.15.47:0/4105778346 reap_dead start 2024-02-23T00:55:02.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 shutdown_connections 2024-02-23T00:55:02.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 >> 172.21.15.47:0/4105778346 conn(0x7efcd40f3750 msgr2=0x7efcd40fbf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:02.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 shutdown_connections 2024-02-23T00:55:02.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 1 -- 172.21.15.47:0/4105778346 wait complete. 2024-02-23T00:55:02.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:02.693+0000 7efcd8cc1700 1 librados: shutdown 2024-02-23T00:55:02.700 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:55:02.700 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:55:02.700 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:55:02.700 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:55:02.700 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:55:02.700 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:55:02.701 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:55:02.701 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:55:03.142 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:55:03.324 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:55:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:55:03.526 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:02.310408+0000 mon.a (mon.0) 3620 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:55:03.526 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:02.319453+0000 mon.a (mon.0) 3621 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_type","val": "bloom"}]': finished 2024-02-23T00:55:03.526 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:02.319567+0000 mon.a (mon.0) 3622 : cluster [DBG] osdmap e754: 8 total, 8 up, 7 in 2024-02-23T00:55:03.526 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:02.320582+0000 mon.c (mon.2) 462 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-23T00:55:03.526 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:02.320917+0000 mon.a (mon.0) 3623 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-23T00:55:03.527 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:02.549423+0000 mon.a (mon.0) 3624 : audit [DBG] from='client.? 172.21.15.47:0/4105778346' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:55:03.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:02.310408+0000 mon.a (mon.0) 3620 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:55:03.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:02.319453+0000 mon.a (mon.0) 3621 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_type","val": "bloom"}]': finished 2024-02-23T00:55:03.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:02.319567+0000 mon.a (mon.0) 3622 : cluster [DBG] osdmap e754: 8 total, 8 up, 7 in 2024-02-23T00:55:03.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:02.320582+0000 mon.c (mon.2) 462 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-23T00:55:03.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:02.320917+0000 mon.a (mon.0) 3623 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-23T00:55:03.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:02.549423+0000 mon.a (mon.0) 3624 : audit [DBG] from='client.? 172.21.15.47:0/4105778346' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:55:03.551 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:55:03.557 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:03.559 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:55:03.661 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:55:03.691 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:02.310408+0000 mon.a (mon.0) 3620 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:55:03.691 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:02.319453+0000 mon.a (mon.0) 3621 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_type","val": "bloom"}]': finished 2024-02-23T00:55:03.692 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:02.319567+0000 mon.a (mon.0) 3622 : cluster [DBG] osdmap e754: 8 total, 8 up, 7 in 2024-02-23T00:55:03.692 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:02.320582+0000 mon.c (mon.2) 462 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-23T00:55:03.692 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:02.320917+0000 mon.a (mon.0) 3623 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-23T00:55:03.692 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:02.549423+0000 mon.a (mon.0) 3624 : audit [DBG] from='client.? 172.21.15.47:0/4105778346' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:55:03.694 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:03.715 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:55:03.717 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:03.718 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:55:03.819 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:55:03.850 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:03.870 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:55:03.872 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:03.873 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:55:03.973 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:55:04.002 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:04.021 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:55:04.023 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:04.024 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:55:04.125 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:55:04.157 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:04.255 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:55:04.259 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:04.260 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:55:04.360 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:55:04.389 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:55:04.389 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-02-23T00:55:04.389 INFO:teuthology.orchestra.run.smithi047.stdout: "num_ops": 0 2024-02-23T00:55:04.389 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:55:04.389 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:04.671 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:55:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:02.957108+0000 mgr.y (mgr.14517) 644 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:55:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:03.319056+0000 mon.a (mon.0) 3625 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-23T00:55:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:03.319121+0000 mon.a (mon.0) 3626 : cluster [DBG] osdmap e755: 8 total, 8 up, 7 in 2024-02-23T00:55:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:03.319790+0000 mon.c (mon.2) 463 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-23T00:55:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:03.320548+0000 mon.a (mon.0) 3627 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-23T00:55:04.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:03.394590+0000 mgr.y (mgr.14517) 645 : cluster [DBG] pgmap v1146: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s; 1/786 objects misplaced (0.127%); 3 B/s, 0 objects/s recovering 2024-02-23T00:55:04.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:04.320624+0000 mon.a (mon.0) 3628 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-23T00:55:04.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:02.957108+0000 mgr.y (mgr.14517) 644 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:55:04.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:03.319056+0000 mon.a (mon.0) 3625 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-23T00:55:04.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:03.319121+0000 mon.a (mon.0) 3626 : cluster [DBG] osdmap e755: 8 total, 8 up, 7 in 2024-02-23T00:55:04.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:03.319790+0000 mon.c (mon.2) 463 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-23T00:55:04.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:03.320548+0000 mon.a (mon.0) 3627 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-23T00:55:04.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:03.394590+0000 mgr.y (mgr.14517) 645 : cluster [DBG] pgmap v1146: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s; 1/786 objects misplaced (0.127%); 3 B/s, 0 objects/s recovering 2024-02-23T00:55:04.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:04.320624+0000 mon.a (mon.0) 3628 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-23T00:55:04.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:02.957108+0000 mgr.y (mgr.14517) 644 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:55:04.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:03.319056+0000 mon.a (mon.0) 3625 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-23T00:55:04.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:03.319121+0000 mon.a (mon.0) 3626 : cluster [DBG] osdmap e755: 8 total, 8 up, 7 in 2024-02-23T00:55:04.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:03.319790+0000 mon.c (mon.2) 463 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-23T00:55:04.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:03.320548+0000 mon.a (mon.0) 3627 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-23T00:55:04.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:03.394590+0000 mgr.y (mgr.14517) 645 : cluster [DBG] pgmap v1146: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s; 1/786 objects misplaced (0.127%); 3 B/s, 0 objects/s recovering 2024-02-23T00:55:04.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:04.320624+0000 mon.a (mon.0) 3628 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-23T00:55:05.546 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:55:05.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:04.320716+0000 mon.a (mon.0) 3629 : cluster [DBG] osdmap e756: 8 total, 8 up, 7 in 2024-02-23T00:55:05.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:04.323370+0000 mon.c (mon.2) 464 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-23T00:55:05.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:04.323630+0000 mon.a (mon.0) 3630 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-23T00:55:05.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:04.323907+0000 mon.a (mon.0) 3631 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-23T00:55:05.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:05.322604+0000 mon.a (mon.0) 3632 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-23T00:55:05.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:05.322692+0000 mon.a (mon.0) 3633 : cluster [DBG] osdmap e757: 8 total, 8 up, 7 in 2024-02-23T00:55:05.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:04.320716+0000 mon.a (mon.0) 3629 : cluster [DBG] osdmap e756: 8 total, 8 up, 7 in 2024-02-23T00:55:05.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:04.323370+0000 mon.c (mon.2) 464 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-23T00:55:05.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:04.323630+0000 mon.a (mon.0) 3630 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-23T00:55:05.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:04.323907+0000 mon.a (mon.0) 3631 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-23T00:55:05.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:05.322604+0000 mon.a (mon.0) 3632 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-23T00:55:05.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:05.322692+0000 mon.a (mon.0) 3633 : cluster [DBG] osdmap e757: 8 total, 8 up, 7 in 2024-02-23T00:55:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:04.320716+0000 mon.a (mon.0) 3629 : cluster [DBG] osdmap e756: 8 total, 8 up, 7 in 2024-02-23T00:55:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:04.323370+0000 mon.c (mon.2) 464 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-23T00:55:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:04.323630+0000 mon.a (mon.0) 3630 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-23T00:55:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:04.323907+0000 mon.a (mon.0) 3631 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-23T00:55:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:05.322604+0000 mon.a (mon.0) 3632 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-132","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-23T00:55:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:05.322692+0000 mon.a (mon.0) 3633 : cluster [DBG] osdmap e757: 8 total, 8 up, 7 in 2024-02-23T00:55:05.722 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.3 dump_blocked_ops 2024-02-23T00:55:05.740 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:55:05.743 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:05.743 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:55:05.844 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:55:05.876 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:05.897 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:55:05.899 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:05.900 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:55:06.001 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:55:06.030 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:06.432 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:55:06.434 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:06.435 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:55:06.536 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:55:06.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:05.365048+0000 mon.c (mon.2) 465 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:06.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:05.365360+0000 mon.a (mon.0) 3634 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:06.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:05.395177+0000 mgr.y (mgr.14517) 646 : cluster [DBG] pgmap v1149: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/786 objects misplaced (0.127%); 3 B/s, 0 objects/s recovering 2024-02-23T00:55:06.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:05.365048+0000 mon.c (mon.2) 465 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:06.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:05.365360+0000 mon.a (mon.0) 3634 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:06.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:05.395177+0000 mgr.y (mgr.14517) 646 : cluster [DBG] pgmap v1149: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/786 objects misplaced (0.127%); 3 B/s, 0 objects/s recovering 2024-02-23T00:55:06.569 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:06.591 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:55:06.592 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:05.365048+0000 mon.c (mon.2) 465 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:06.593 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:05.365360+0000 mon.a (mon.0) 3634 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:06.593 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:05.395177+0000 mgr.y (mgr.14517) 646 : cluster [DBG] pgmap v1149: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/786 objects misplaced (0.127%); 3 B/s, 0 objects/s recovering 2024-02-23T00:55:06.593 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:06.594 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:55:06.695 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:55:06.725 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:06.748 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:55:06.751 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:06.751 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:55:06.852 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:55:06.881 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:06.900 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:55:06.902 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:06.903 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:55:07.004 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:55:07.031 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:07.052 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:55:07.054 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:07.054 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:55:07.155 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:55:07.183 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:07.202 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:55:07.204 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:07.205 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:55:07.305 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:55:07.340 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:07.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:06.334976+0000 mon.a (mon.0) 3635 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:55:07.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:06.335022+0000 mon.a (mon.0) 3636 : cluster [DBG] osdmap e758: 8 total, 8 up, 7 in 2024-02-23T00:55:07.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:06.335872+0000 mon.c (mon.2) 466 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-132"}]: dispatch 2024-02-23T00:55:07.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:06.336234+0000 mon.a (mon.0) 3637 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-132"}]: dispatch 2024-02-23T00:55:07.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:06.704334+0000 mon.a (mon.0) 3638 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:55:07.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:06.711951+0000 mgr.y (mgr.14517) 647 : audit [DBG] from='client.31761 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:55:07.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.530+0000 7fd834e15700 1 -- 172.21.15.47:0/2344916399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8300f8d10 msgr2=0x7fd8300f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:07.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.530+0000 7fd834e15700 1 --2- 172.21.15.47:0/2344916399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8300f8d10 0x7fd8300f90f0 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 rx=0x7fd820004870 tx=0x7fd820009cb0).stop 2024-02-23T00:55:07.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.530+0000 7fd834e15700 1 -- 172.21.15.47:0/2344916399 shutdown_connections 2024-02-23T00:55:07.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.530+0000 7fd834e15700 1 --2- 172.21.15.47:0/2344916399 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd830102270 0x7fd830104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:07.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.530+0000 7fd834e15700 1 --2- 172.21.15.47:0/2344916399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8300f8d10 0x7fd8300f90f0 unknown :-1 s=CLOSED pgs=2634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:07.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.530+0000 7fd834e15700 1 --2- 172.21.15.47:0/2344916399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8300f9670 0x7fd830101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:07.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.530+0000 7fd834e15700 1 -- 172.21.15.47:0/2344916399 >> 172.21.15.47:0/2344916399 conn(0x7fd8300f2820 msgr2=0x7fd8300f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:07.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.530+0000 7fd834e15700 1 -- 172.21.15.47:0/2344916399 shutdown_connections 2024-02-23T00:55:07.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.530+0000 7fd834e15700 1 -- 172.21.15.47:0/2344916399 wait complete. 2024-02-23T00:55:07.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.531+0000 7fd834e15700 1 librados: starting msgr at 2024-02-23T00:55:07.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.531+0000 7fd834e15700 1 librados: starting objecter 2024-02-23T00:55:07.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.531+0000 7fd834e15700 1 Processor -- start 2024-02-23T00:55:07.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.531+0000 7fd834e15700 1 -- start start 2024-02-23T00:55:07.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.531+0000 7fd834e15700 1 librados: setting wanted keys 2024-02-23T00:55:07.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.531+0000 7fd834e15700 1 librados: calling monclient init 2024-02-23T00:55:07.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.531+0000 7fd834e15700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd8300f8d10 0x7fd8300f6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.531+0000 7fd834e15700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8300f9670 0x7fd8300f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.531+0000 7fd834e15700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd830102270 0x7fd8300f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.531+0000 7fd834e15700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd830116440 con 0x7fd8300f9670 2024-02-23T00:55:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.531+0000 7fd834e15700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8301165b0 con 0x7fd830102270 2024-02-23T00:55:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.531+0000 7fd834e15700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd830116770 con 0x7fd8300f8d10 2024-02-23T00:55:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.531+0000 7fd82dd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8300f9670 0x7fd8300f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.531+0000 7fd82e59c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd8300f8d10 0x7fd8300f6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.531+0000 7fd82dd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8300f9670 0x7fd8300f71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52956/0 (socket says 172.21.15.47:52956) 2024-02-23T00:55:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.531+0000 7fd82dd9b700 1 -- 172.21.15.47:0/1860179053 learned_addr learned my addr 172.21.15.47:0/1860179053 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:55:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.532+0000 7fd82ed9d700 1 --2- 172.21.15.47:0/1860179053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd830102270 0x7fd8300f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.532+0000 7fd82dd9b700 1 -- 172.21.15.47:0/1860179053 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd8300f8d10 msgr2=0x7fd8300f6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.532+0000 7fd82dd9b700 1 --2- 172.21.15.47:0/1860179053 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd8300f8d10 0x7fd8300f6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.532+0000 7fd82dd9b700 1 -- 172.21.15.47:0/1860179053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd830102270 msgr2=0x7fd8300f76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.532+0000 7fd82dd9b700 1 --2- 172.21.15.47:0/1860179053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd830102270 0x7fd8300f76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:07.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.532+0000 7fd82dd9b700 1 -- 172.21.15.47:0/1860179053 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8280097c0 con 0x7fd8300f9670 2024-02-23T00:55:07.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.532+0000 7fd82ed9d700 1 --2- 172.21.15.47:0/1860179053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd830102270 0x7fd8300f76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-23T00:55:07.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.532+0000 7fd82dd9b700 1 --2- 172.21.15.47:0/1860179053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8300f9670 0x7fd8300f71b0 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 rx=0x7fd82400c7c0 tx=0x7fd824007ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:07.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.532+0000 7fd81f7fe700 10 client.?.objecter ms_handle_connect 0x7fd8300f9670 2024-02-23T00:55:07.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.532+0000 7fd81f7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:55:07.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.532+0000 7fd81f7fe700 1 -- 172.21.15.47:0/1860179053 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd824004430 con 0x7fd8300f9670 2024-02-23T00:55:07.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.532+0000 7fd81f7fe700 1 -- 172.21.15.47:0/1860179053 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fd82400e930 con 0x7fd8300f9670 2024-02-23T00:55:07.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.532+0000 7fd834e15700 1 -- 172.21.15.47:0/1860179053 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd820005040 con 0x7fd8300f9670 2024-02-23T00:55:07.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.532+0000 7fd81f7fe700 1 -- 172.21.15.47:0/1860179053 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd824013990 con 0x7fd8300f9670 2024-02-23T00:55:07.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.532+0000 7fd834e15700 10 client.31761.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:55:07.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.532+0000 7fd834e15700 1 -- 172.21.15.47:0/1860179053 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd830116db0 con 0x7fd8300f9670 2024-02-23T00:55:07.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.532+0000 7fd834e15700 1 librados: init done 2024-02-23T00:55:07.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.533+0000 7fd834e15700 1 -- 172.21.15.47:0/1860179053 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd83005fa60 con 0x7fd8300f9670 2024-02-23T00:55:07.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.533+0000 7fd81f7fe700 1 -- 172.21.15.47:0/1860179053 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fd82400e4a0 con 0x7fd8300f9670 2024-02-23T00:55:07.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.534+0000 7fd81f7fe700 1 --2- 172.21.15.47:0/1860179053 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd818062cc0 0x7fd818065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:07.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.534+0000 7fd81f7fe700 1 -- 172.21.15.47:0/1860179053 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(758..758 src has 621..758) v4 ==== 9807+0+0 (secure 0 0 0) 0x7fd82407db10 con 0x7fd8300f9670 2024-02-23T00:55:07.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.534+0000 7fd81f7fe700 10 client.31761.objecter ms_dispatch 0x7fd830001550 osd_map(758..758 src has 621..758) v4 2024-02-23T00:55:07.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.534+0000 7fd81f7fe700 3 client.31761.objecter handle_osd_map got epochs [758,758] > 0 2024-02-23T00:55:07.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.534+0000 7fd81f7fe700 3 client.31761.objecter handle_osd_map decoding full epoch 758 2024-02-23T00:55:07.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.534+0000 7fd82e59c700 1 --2- 172.21.15.47:0/1860179053 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd818062cc0 0x7fd818065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:07.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.534+0000 7fd81f7fe700 20 client.31761.objecter dump_active .. 0 homeless 2024-02-23T00:55:07.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.534+0000 7fd82e59c700 1 --2- 172.21.15.47:0/1860179053 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd818062cc0 0x7fd818065180 secure :-1 s=READY pgs=3197 cs=0 l=1 rev1=1 rx=0x7fd820004ee0 tx=0x7fd82000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:07.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.534+0000 7fd81f7fe700 10 client.31761.objecter ms_handle_connect 0x7fd818062cc0 2024-02-23T00:55:07.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.539+0000 7fd81f7fe700 1 -- 172.21.15.47:0/1860179053 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fd82404cee0 con 0x7fd8300f9670 2024-02-23T00:55:07.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.711+0000 7fd834e15700 1 -- 172.21.15.47:0/1860179053 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd8300f81a0 con 0x7fd818062cc0 2024-02-23T00:55:07.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:06.725+0000 7fd81f7fe700 1 -- 172.21.15.47:0/1860179053 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+477925 (secure 0 0 0) 0x7fd8300f81a0 con 0x7fd818062cc0 2024-02-23T00:55:07.462 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:55:07.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.453+0000 7fd834e15700 10 librados: watch_flush enter 2024-02-23T00:55:07.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.453+0000 7fd834e15700 10 librados: watch_flush exit 2024-02-23T00:55:07.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.453+0000 7fd834e15700 20 client.31761.objecter shutdown clearing up homeless session... 2024-02-23T00:55:07.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.453+0000 7fd834e15700 10 client.31761.objecter successfully canceled tick 2024-02-23T00:55:07.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.453+0000 7fd834e15700 1 -- 172.21.15.47:0/1860179053 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd818062cc0 msgr2=0x7fd818065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:07.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.453+0000 7fd834e15700 1 --2- 172.21.15.47:0/1860179053 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd818062cc0 0x7fd818065180 secure :-1 s=READY pgs=3197 cs=0 l=1 rev1=1 rx=0x7fd820004ee0 tx=0x7fd82000b040).stop 2024-02-23T00:55:07.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.453+0000 7fd834e15700 1 -- 172.21.15.47:0/1860179053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8300f9670 msgr2=0x7fd8300f71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:07.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.453+0000 7fd834e15700 1 --2- 172.21.15.47:0/1860179053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8300f9670 0x7fd8300f71b0 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 rx=0x7fd82400c7c0 tx=0x7fd824007ef0).stop 2024-02-23T00:55:07.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.453+0000 7fd834e15700 1 -- 172.21.15.47:0/1860179053 shutdown_connections 2024-02-23T00:55:07.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.453+0000 7fd834e15700 1 --2- 172.21.15.47:0/1860179053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd830102270 0x7fd8300f76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:07.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.453+0000 7fd834e15700 1 --2- 172.21.15.47:0/1860179053 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd818062cc0 0x7fd818065180 unknown :-1 s=CLOSED pgs=3197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:07.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.453+0000 7fd834e15700 1 --2- 172.21.15.47:0/1860179053 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd8300f8d10 0x7fd8300f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:07.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.453+0000 7fd834e15700 1 --2- 172.21.15.47:0/1860179053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8300f9670 0x7fd8300f71b0 unknown :-1 s=CLOSED pgs=2635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:07.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.453+0000 7fd834e15700 1 -- 172.21.15.47:0/1860179053 >> 172.21.15.47:0/1860179053 conn(0x7fd8300f2820 msgr2=0x7fd8300f34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:07.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.454+0000 7fd834e15700 1 -- 172.21.15.47:0/1860179053 shutdown_connections 2024-02-23T00:55:07.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.454+0000 7fd834e15700 1 -- 172.21.15.47:0/1860179053 wait complete. 2024-02-23T00:55:07.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.454+0000 7fd834e15700 1 librados: shutdown 2024-02-23T00:55:07.467 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-02-23T00:55:07.468 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:55:07.468 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:55:07.469 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:06.334976+0000 mon.a (mon.0) 3635 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:55:07.470 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:06.335022+0000 mon.a (mon.0) 3636 : cluster [DBG] osdmap e758: 8 total, 8 up, 7 in 2024-02-23T00:55:07.470 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:06.335872+0000 mon.c (mon.2) 466 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-132"}]: dispatch 2024-02-23T00:55:07.470 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:06.336234+0000 mon.a (mon.0) 3637 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-132"}]: dispatch 2024-02-23T00:55:07.470 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:06.704334+0000 mon.a (mon.0) 3638 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:55:07.470 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:06.711951+0000 mgr.y (mgr.14517) 647 : audit [DBG] from='client.31761 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:55:07.470 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:06.334976+0000 mon.a (mon.0) 3635 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:55:07.470 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:06.335022+0000 mon.a (mon.0) 3636 : cluster [DBG] osdmap e758: 8 total, 8 up, 7 in 2024-02-23T00:55:07.470 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:06.335872+0000 mon.c (mon.2) 466 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-132"}]: dispatch 2024-02-23T00:55:07.470 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:06.336234+0000 mon.a (mon.0) 3637 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-132"}]: dispatch 2024-02-23T00:55:07.470 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:06.704334+0000 mon.a (mon.0) 3638 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:55:07.471 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:06.711951+0000 mgr.y (mgr.14517) 647 : audit [DBG] from='client.31761 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:55:07.471 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:07.471 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:55:07.572 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:55:07.599 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:07.617 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:55:07.620 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:07.620 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:55:07.721 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:55:07.753 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:07.850 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:55:07.853 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:07.853 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:55:07.954 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:55:07.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.952+0000 7f167de2c700 1 -- 172.21.15.47:0/587346316 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f16780f8cb0 msgr2=0x7f16780f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:07.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.952+0000 7f167de2c700 1 --2- 172.21.15.47:0/587346316 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f16780f8cb0 0x7f16780f9090 secure :-1 s=READY pgs=2806 cs=0 l=1 rev1=1 rx=0x7f16680047b0 tx=0x7f1668009cb0).stop 2024-02-23T00:55:07.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.953+0000 7f167de2c700 1 -- 172.21.15.47:0/587346316 shutdown_connections 2024-02-23T00:55:07.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.953+0000 7f167de2c700 1 --2- 172.21.15.47:0/587346316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1678102210 0x7f1678104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:07.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.953+0000 7f167de2c700 1 --2- 172.21.15.47:0/587346316 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f16780f8cb0 0x7f16780f9090 unknown :-1 s=CLOSED pgs=2806 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:07.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.953+0000 7f167de2c700 1 --2- 172.21.15.47:0/587346316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16780f9610 0x7f1678101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:07.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.953+0000 7f167de2c700 1 -- 172.21.15.47:0/587346316 >> 172.21.15.47:0/587346316 conn(0x7f16780f2820 msgr2=0x7f16780f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:07.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.953+0000 7f167de2c700 1 -- 172.21.15.47:0/587346316 shutdown_connections 2024-02-23T00:55:07.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.953+0000 7f167de2c700 1 -- 172.21.15.47:0/587346316 wait complete. 2024-02-23T00:55:07.956 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:55:07.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.953+0000 7f167de2c700 1 librados: starting msgr at 2024-02-23T00:55:07.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.953+0000 7f167de2c700 1 librados: starting objecter 2024-02-23T00:55:07.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.953+0000 7f167de2c700 1 Processor -- start 2024-02-23T00:55:07.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.954+0000 7f167de2c700 1 -- start start 2024-02-23T00:55:07.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.954+0000 7f167de2c700 1 librados: setting wanted keys 2024-02-23T00:55:07.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.954+0000 7f167de2c700 1 librados: calling monclient init 2024-02-23T00:55:07.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.954+0000 7f167de2c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16780f8cb0 0x7f167810e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:07.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.954+0000 7f167de2c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f16780f9610 0x7f167810e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:07.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.954+0000 7f167de2c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1678102210 0x7f1678108280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:07.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.954+0000 7f167de2c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1678108770 con 0x7f16780f8cb0 2024-02-23T00:55:07.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.954+0000 7f167de2c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16781088e0 con 0x7f1678102210 2024-02-23T00:55:07.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.954+0000 7f167de2c700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1678108aa0 con 0x7f16780f9610 2024-02-23T00:55:07.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.954+0000 7f1676ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f16780f9610 0x7f167810e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:07.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.954+0000 7f1676ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f16780f9610 0x7f167810e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:60648/0 (socket says 172.21.15.47:60648) 2024-02-23T00:55:07.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.954+0000 7f1676ffd700 1 -- 172.21.15.47:0/1161354656 learned_addr learned my addr 172.21.15.47:0/1161354656 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:55:07.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.954+0000 7f16777fe700 1 --2- 172.21.15.47:0/1161354656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16780f8cb0 0x7f167810e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:07.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.954+0000 7f1677fff700 1 --2- 172.21.15.47:0/1161354656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1678102210 0x7f1678108280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:07.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.954+0000 7f1676ffd700 1 -- 172.21.15.47:0/1161354656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1678102210 msgr2=0x7f1678108280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:07.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.954+0000 7f1676ffd700 1 --2- 172.21.15.47:0/1161354656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1678102210 0x7f1678108280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:07.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.954+0000 7f1676ffd700 1 -- 172.21.15.47:0/1161354656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16780f8cb0 msgr2=0x7f167810e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:07.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.954+0000 7f1676ffd700 1 --2- 172.21.15.47:0/1161354656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16780f8cb0 0x7f167810e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:07.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.954+0000 7f1676ffd700 1 -- 172.21.15.47:0/1161354656 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16700097c0 con 0x7f16780f9610 2024-02-23T00:55:07.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.954+0000 7f16777fe700 1 --2- 172.21.15.47:0/1161354656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16780f8cb0 0x7f167810e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:55:07.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.954+0000 7f1676ffd700 1 --2- 172.21.15.47:0/1161354656 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f16780f9610 0x7f167810e5d0 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 rx=0x7f166c00e970 tx=0x7f166c00b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:07.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.954+0000 7f1674ff9700 10 client.?.objecter ms_handle_connect 0x7f16780f9610 2024-02-23T00:55:07.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.954+0000 7f1674ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:55:07.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.954+0000 7f1674ff9700 1 -- 172.21.15.47:0/1161354656 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c00ee60 con 0x7f16780f9610 2024-02-23T00:55:07.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.955+0000 7f167de2c700 1 -- 172.21.15.47:0/1161354656 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1668005040 con 0x7f16780f9610 2024-02-23T00:55:07.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.955+0000 7f1674ff9700 1 -- 172.21.15.47:0/1161354656 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f166c00f040 con 0x7f16780f9610 2024-02-23T00:55:07.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.955+0000 7f167de2c700 10 client.42086.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:55:07.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.955+0000 7f1674ff9700 1 -- 172.21.15.47:0/1161354656 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c00c5b0 con 0x7f16780f9610 2024-02-23T00:55:07.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.955+0000 7f167de2c700 1 -- 172.21.15.47:0/1161354656 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1678114f10 con 0x7f16780f9610 2024-02-23T00:55:07.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.955+0000 7f167de2c700 1 librados: init done 2024-02-23T00:55:07.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.956+0000 7f167de2c700 1 -- 172.21.15.47:0/1161354656 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1678109130 con 0x7f16780f9610 2024-02-23T00:55:07.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.956+0000 7f1674ff9700 1 -- 172.21.15.47:0/1161354656 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f166c007b40 con 0x7f16780f9610 2024-02-23T00:55:07.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.957+0000 7f1674ff9700 1 --2- 172.21.15.47:0/1161354656 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1660062cc0 0x7f1660065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:07.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.957+0000 7f1674ff9700 1 -- 172.21.15.47:0/1161354656 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(759..759 src has 621..759) v4 ==== 9773+0+0 (secure 0 0 0) 0x7f166c07d4b0 con 0x7f16780f9610 2024-02-23T00:55:07.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.957+0000 7f1674ff9700 10 client.42086.objecter ms_dispatch 0x7f1678001550 osd_map(759..759 src has 621..759) v4 2024-02-23T00:55:07.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.957+0000 7f1674ff9700 3 client.42086.objecter handle_osd_map got epochs [759,759] > 0 2024-02-23T00:55:07.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.957+0000 7f1674ff9700 3 client.42086.objecter handle_osd_map decoding full epoch 759 2024-02-23T00:55:07.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.957+0000 7f16777fe700 1 --2- 172.21.15.47:0/1161354656 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1660062cc0 0x7f1660065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:07.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.957+0000 7f1674ff9700 20 client.42086.objecter dump_active .. 0 homeless 2024-02-23T00:55:07.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.957+0000 7f1674ff9700 10 client.42086.objecter ms_handle_connect 0x7f1660062cc0 2024-02-23T00:55:07.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.957+0000 7f16777fe700 1 --2- 172.21.15.47:0/1161354656 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1660062cc0 0x7f1660065180 secure :-1 s=READY pgs=3198 cs=0 l=1 rev1=1 rx=0x7f1668004e20 tx=0x7f1668009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:07.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:07.962+0000 7f1674ff9700 1 -- 172.21.15.47:0/1161354656 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f166c04af60 con 0x7f16780f9610 2024-02-23T00:55:07.985 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:08.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:08.159+0000 7f167de2c700 1 -- 172.21.15.47:0/1161354656 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7f167805fa60 con 0x7f16780f9610 2024-02-23T00:55:08.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:08.348+0000 7f1674ff9700 1 -- 172.21.15.47:0/1161354656 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v760) v1 ==== 97+0+0 (secure 0 0 0) 0x7f166c019020 con 0x7f16780f9610 2024-02-23T00:55:08.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:08.351+0000 7f167de2c700 10 librados: watch_flush enter 2024-02-23T00:55:08.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:08.351+0000 7f167de2c700 10 librados: watch_flush exit 2024-02-23T00:55:08.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:08.351+0000 7f167de2c700 20 client.42086.objecter shutdown clearing up homeless session... 2024-02-23T00:55:08.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:08.351+0000 7f167de2c700 10 client.42086.objecter successfully canceled tick 2024-02-23T00:55:08.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:08.351+0000 7f167de2c700 1 -- 172.21.15.47:0/1161354656 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1660062cc0 msgr2=0x7f1660065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:08.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:08.351+0000 7f167de2c700 1 --2- 172.21.15.47:0/1161354656 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1660062cc0 0x7f1660065180 secure :-1 s=READY pgs=3198 cs=0 l=1 rev1=1 rx=0x7f1668004e20 tx=0x7f1668009cb0).stop 2024-02-23T00:55:08.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:08.351+0000 7f167de2c700 1 -- 172.21.15.47:0/1161354656 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f16780f9610 msgr2=0x7f167810e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:08.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:08.351+0000 7f167de2c700 1 --2- 172.21.15.47:0/1161354656 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f16780f9610 0x7f167810e5d0 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 rx=0x7f166c00e970 tx=0x7f166c00b040).stop 2024-02-23T00:55:08.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:08.352+0000 7f167de2c700 1 -- 172.21.15.47:0/1161354656 shutdown_connections 2024-02-23T00:55:08.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:08.352+0000 7f167de2c700 1 --2- 172.21.15.47:0/1161354656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1678102210 0x7f1678108280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:08.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:08.352+0000 7f167de2c700 1 --2- 172.21.15.47:0/1161354656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16780f8cb0 0x7f167810e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:08.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:08.352+0000 7f167de2c700 1 --2- 172.21.15.47:0/1161354656 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1660062cc0 0x7f1660065180 unknown :-1 s=CLOSED pgs=3198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:08.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:08.352+0000 7f167de2c700 1 --2- 172.21.15.47:0/1161354656 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f16780f9610 0x7f167810e5d0 unknown :-1 s=CLOSED pgs=2807 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:08.352+0000 7f167de2c700 1 -- 172.21.15.47:0/1161354656 >> 172.21.15.47:0/1161354656 conn(0x7f16780f2820 msgr2=0x7f16780f34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:08.352+0000 7f167de2c700 1 -- 172.21.15.47:0/1161354656 shutdown_connections 2024-02-23T00:55:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:08.352+0000 7f167de2c700 1 -- 172.21.15.47:0/1161354656 wait complete. 2024-02-23T00:55:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:08.353+0000 7f167de2c700 1 librados: shutdown 2024-02-23T00:55:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is set 2024-02-23T00:55:08.427 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:07.335624+0000 mon.a (mon.0) 3639 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-132"}]': finished 2024-02-23T00:55:08.427 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:07.335701+0000 mon.a (mon.0) 3640 : cluster [DBG] osdmap e759: 8 total, 8 up, 7 in 2024-02-23T00:55:08.427 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:07.363877+0000 mon.c (mon.2) 467 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:08.427 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:07.364252+0000 mon.a (mon.0) 3641 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:08.427 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:07.364762+0000 mon.c (mon.2) 468 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-132"}]: dispatch 2024-02-23T00:55:08.427 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:07.365018+0000 mon.a (mon.0) 3642 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-132"}]: dispatch 2024-02-23T00:55:08.427 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:07.395586+0000 mgr.y (mgr.14517) 648 : cluster [DBG] pgmap v1152: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 1 op/s; 1/789 objects misplaced (0.127%) 2024-02-23T00:55:08.427 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:08.160316+0000 mon.c (mon.2) 469 : audit [INF] from='client.? 172.21.15.47:0/1161354656' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:55:08.427 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:08.160672+0000 mon.a (mon.0) 3643 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:55:08.428 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:07.335624+0000 mon.a (mon.0) 3639 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-132"}]': finished 2024-02-23T00:55:08.428 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:07.335701+0000 mon.a (mon.0) 3640 : cluster [DBG] osdmap e759: 8 total, 8 up, 7 in 2024-02-23T00:55:08.428 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:07.363877+0000 mon.c (mon.2) 467 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:08.428 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:07.364252+0000 mon.a (mon.0) 3641 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:08.428 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:07.364762+0000 mon.c (mon.2) 468 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-132"}]: dispatch 2024-02-23T00:55:08.428 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:07.365018+0000 mon.a (mon.0) 3642 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-132"}]: dispatch 2024-02-23T00:55:08.428 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:07.395586+0000 mgr.y (mgr.14517) 648 : cluster [DBG] pgmap v1152: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 1 op/s; 1/789 objects misplaced (0.127%) 2024-02-23T00:55:08.428 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:08.160316+0000 mon.c (mon.2) 469 : audit [INF] from='client.? 172.21.15.47:0/1161354656' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:55:08.428 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:08.160672+0000 mon.a (mon.0) 3643 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:55:08.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:07.335624+0000 mon.a (mon.0) 3639 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-132"}]': finished 2024-02-23T00:55:08.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:07.335701+0000 mon.a (mon.0) 3640 : cluster [DBG] osdmap e759: 8 total, 8 up, 7 in 2024-02-23T00:55:08.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:07.363877+0000 mon.c (mon.2) 467 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:08.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:07.364252+0000 mon.a (mon.0) 3641 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:08.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:07.364762+0000 mon.c (mon.2) 468 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-132"}]: dispatch 2024-02-23T00:55:08.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:07.365018+0000 mon.a (mon.0) 3642 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-132"}]: dispatch 2024-02-23T00:55:08.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:07.395586+0000 mgr.y (mgr.14517) 648 : cluster [DBG] pgmap v1152: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 1 op/s; 1/789 objects misplaced (0.127%) 2024-02-23T00:55:08.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:08.160316+0000 mon.c (mon.2) 469 : audit [INF] from='client.? 172.21.15.47:0/1161354656' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:55:08.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:08.160672+0000 mon.a (mon.0) 3643 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:55:09.267 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":1150,"stamp":"2024-02-23T00:55:06.331339+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4006050,"num_objects":262,"num_object_clones":1,"num_object_copies":786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":0,"num_read":2475,"num_read_kb":2184,"num_write":970,"num_write_kb":4007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":4010824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1158,"ondisk_log_size":1158,"up":552,"acting":552,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":555,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":2133600,"kb_used_data":28456,"kb_used_omap":0,"kb_used_meta":2105088,"kb_avail":654082464,"statfs":{"total":671965249536,"available":669780443136,"internally_reserved":0,"allocated":29138944,"data_stored":16823961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2155610112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.016551"},"pg_stats":[{"pgid":"250.1f","version":"0'0","reported_seq":14,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:00.307143+0000","last_change":"2024-02-23T00:54:56.470448+0000","last_active":"2024-02-23T00:55:00.307143+0000","last_peered":"2024-02-23T00:55:00.307143+0000","last_clean":"2024-02-23T00:55:00.307143+0000","last_became_active":"2024-02-23T00:54:56.470127+0000","last_became_peered":"2024-02-23T00:54:56.470127+0000","last_unstale":"2024-02-23T00:55:00.307143+0000","last_undegraded":"2024-02-23T00:55:00.307143+0000","last_fullsized":"2024-02-23T00:55:00.307143+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":747,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.441916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"250.1e","version":"0'0","reported_seq":14,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:00.306935+0000","last_change":"2024-02-23T00:54:56.470926+0000","last_active":"2024-02-23T00:55:00.306935+0000","last_peered":"2024-02-23T00:55:00.306935+0000","last_clean":"2024-02-23T00:55:00.306935+0000","last_became_active":"2024-02-23T00:54:56.470447+0000","last_became_peered":"2024-02-23T00:54:56.470447+0000","last_unstale":"2024-02-23T00:55:00.306935+0000","last_undegraded":"2024-02-23T00:55:00.306935+0000","last_fullsized":"2024-02-23T00:55:00.306935+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":747,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.441916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"250.1d","version":"0'0","reported_seq":13,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.925139+0000","last_change":"2024-02-23T00:54:56.476568+0000","last_active":"2024-02-23T00:54:59.925139+0000","last_peered":"2024-02-23T00:54:59.925139+0000","last_clean":"2024-02-23T00:54:59.925139+0000","last_became_active":"2024-02-23T00:54:56.476003+0000","last_became_peered":"2024-02-23T00:54:56.476003+0000","last_unstale":"2024-02-23T00:54:59.925139+0000","last_undegraded":"2024-02-23T00:54:59.925139+0000","last_fullsized":"2024-02-23T00:54:59.925139+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":747,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.441916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"250.1c","version":"0'0","reported_seq":14,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:00.306339+0000","last_change":"2024-02-23T00:54:56.480271+0000","last_active":"2024-02-23T00:55:00.306339+0000","last_peered":"2024-02-23T00:55:00.306339+0000","last_clean":"2024-02-23T00:55:00.306339+0000","last_became_active":"2024-02-23T00:54:56.479927+0000","last_became_peered":"2024-02-23T00:54:56.479927+0000","last_unstale":"2024-02-23T00:55:00.306339+0000","last_undegraded":"2024-02-23T00:55:00.306339+0000","last_fullsized":"2024-02-23T00:55:00.306339+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":747,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.441916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"250.1b","version":"0'0","reported_seq":13,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.924075+0000","last_change":"2024-02-23T00:54:56.492323+0000","last_active":"2024-02-23T00:54:59.924075+0000","last_peered":"2024-02-23T00:54:59.924075+0000","last_clean":"2024-02-23T00:54:59.924075+0000","last_became_active":"2024-02-23T00:54:56.492008+0000","last_became_peered":"2024-02-23T00:54:56.492008+0000","last_unstale":"2024-02-23T00:54:59.924075+0000","last_undegraded":"2024-02-23T00:54:59.924075+0000","last_fullsized":"2024-02-23T00:54:59.924075+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":747,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.441916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"250.1a","version":"0'0","reported_seq":14,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:00.305989+0000","last_change":"2024-02-23T00:54:56.485516+0000","last_active":"2024-02-23T00:55:00.305989+0000","last_peered":"2024-02-23T00:55:00.305989+0000","last_clean":"2024-02-23T00:55:00.305989+0000","last_became_active":"2024-02-23T00:54:56.484985+0000","last_became_peered":"2024-02-23T00:54:56.484985+0000","last_unstale":"2024-02-23T00:55:00.305989+0000","last_undegraded":"2024-02-23T00:55:00.305989+0000","last_fullsized":"2024-02-23T00:55:00.305989+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":747,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.441916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"250.19","version":"0'0","reported_seq":14,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.110998+0000","last_change":"2024-02-23T00:54:56.484016+0000","last_active":"2024-02-23T00:55:01.110998+0000","last_peered":"2024-02-23T00:55:01.110998+0000","last_clean":"2024-02-23T00:55:01.110998+0000","last_became_active":"2024-02-23T00:54:56.483747+0000","last_became_peered":"2024-02-23T00:54:56.483747+0000","last_unstale":"2024-02-23T00:55:01.110998+0000","last_undegraded":"2024-02-23T00:55:01.110998+0000","last_fullsized":"2024-02-23T00:55:01.110998+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":747,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.441916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"250.18","version":"0'0","reported_seq":14,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.110995+0000","last_change":"2024-02-23T00:54:56.487886+0000","last_active":"2024-02-23T00:55:01.110995+0000","last_peered":"2024-02-23T00:55:01.110995+0000","last_clean":"2024-02-23T00:55:01.110995+0000","last_became_active":"2024-02-23T00:54:56.486614+0000","last_became_peered":"2024-02-23T00:54:56.486614+0000","last_unstale":"2024-02-23T00:55:01.110995+0000","last_undegraded":"2024-02-23T00:55:01.110995+0000","last_fullsized":"2024-02-23T00:55:01.110995+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":747,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.441916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"250.f","version":"0'0","reported_seq":14,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.113405+0000","last_change":"2024-02-23T00:54:56.480914+0000","last_active":"2024-02-23T00:55:01.113405+0000","last_peered":"2024-02-23T00:55:01.113405+0000","last_clean":"2024-02-23T00:55:01.113405+0000","last_became_active":"2024-02-23T00:54:56.480620+0000","last_became_peered":"2024-02-23T00:54:56.480620+0000","last_unstale":"2024-02-23T00:55:01.113405+0000","last_undegraded":"2024-02-23T00:55:01.113405+0000","last_fullsized":"2024-02-23T00:55:01.113405+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":747,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.441916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"250.e","version":"0'0","reported_seq":14,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.113256+0000","last_change":"2024-02-23T00:54:56.490600+0000","last_active":"2024-02-23T00:55:01.113256+0000","last_peered":"2024-02-23T00:55:01.113256+0000","last_clean":"2024-02-23T00:55:01.113256+0000","last_became_active":"2024-02-23T00:54:56.489594+0000","last_became_peered":"2024-02-23T00:54:56.489594+0000","last_unstale":"2024-02-23T00:55:01.113256+0000","last_undegraded":"2024-02-23T00:55:01.113256+0000","last_fullsized":"2024-02-23T00:55:01.113256+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":747,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.441916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"250.d","version":"0'0","reported_seq":13,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.926322+0000","last_change":"2024-02-23T00:54:56.475606+0000","last_active":"2024-02-23T00:54:59.926322+0000","last_peered":"2024-02-23T00:54:59.926322+0000","last_clean":"2024-02-23T00:54:59.926322+0000","last_became_active":"2024-02-23T00:54:56.475281+0000","last_became_peered":"2024-02-23T00:54:56.475281+0000","last_unstale":"2024-02-23T00:54:59.926322+0000","last_undegraded":"2024-02-23T00:54:59.926322+0000","last_fullsized":"2024-02-23T00:54:59.926322+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":747,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.441916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"250.c","version":"0'0","reported_seq":14,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.110703+0000","last_change":"2024-02-23T00:54:56.487846+0000","last_active":"2024-02-23T00:55:01.110703+0000","last_peered":"2024-02-23T00:55:01.110703+0000","last_clean":"2024-02-23T00:55:01.110703+0000","last_became_active":"2024-02-23T00:54:56.487560+0000","last_became_peered":"2024-02-23T00:54:56.487560+0000","last_unstale":"2024-02-23T00:55:01.110703+0000","last_undegraded":"2024-02-23T00:55:01.110703+0000","last_fullsized":"2024-02-23T00:55:01.110703+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":747,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.441916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,7],"acting":[5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"250.b","version":"0'0","reported_seq":14,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:00.306396+0000","last_change":"2024-02-23T00:54:56.492571+0000","last_active":"2024-02-23T00:55:00.306396+0000","last_peered":"2024-02-23T00:55:00.306396+0000","last_clean":"2024-02-23T00:55:00.306396+0000","last_became_active":"2024-02-23T00:54:56.491866+0000","last_became_peered":"2024-02-23T00:54:56.491866+0000","last_unstale":"2024-02-23T00:55:00.306396+0000","last_undegraded":"2024-02-23T00:55:00.306396+0000","last_fullsized":"2024-02-23T00:55:00.306396+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":747,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.441916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"250.a","version":"0'0","reported_seq":14,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.111154+0000","last_change":"2024-02-23T00:54:56.486140+0000","last_active":"2024-02-23T00:55:01.111154+0000","last_peered":"2024-02-23T00:55:01.111154+0000","last_clean":"2024-02-23T00:55:01.111154+0000","last_became_active":"2024-02-23T00:54:56.485839+0000","last_became_peered":"2024-02-23T00:54:56.485839+0000","last_unstale":"2024-02-23T00:55:01.111154+0000","last_undegraded":"2024-02-23T00:55:01.111154+0000","last_fullsized":"2024-02-23T00:55:01.111154+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":747,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.441916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"250.9","version":"0'0","reported_seq":14,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.111237+0000","last_change":"2024-02-23T00:54:56.483048+0000","last_active":"2024-02-23T00:55:01.111237+0000","last_peered":"2024-02-23T00:55:01.111237+0000","last_clean":"2024-02-23T00:55:01.111237+0000","last_became_active":"2024-02-23T00:54:56.482674+0000","last_became_peered":"2024-02-23T00:54:56.482674+0000","last_unstale":"2024-02-23T00:55:01.111237+0000","last_undegraded":"2024-02-23T00:55:01.111237+0000","last_fullsized":"2024-02-23T00:55:01.111237+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":747,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.441916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"250.8","version":"0'0","reported_seq":13,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.923884+0000","last_change":"2024-02-23T00:54:56.470039+0000","last_active":"2024-02-23T00:54:59.923884+0000","last_peered":"2024-02-23T00:54:59.923884+0000","last_clean":"2024-02-23T00:54:59.923884+0000","last_became_active":"2024-02-23T00:54:56.469708+0000","last_became_peered":"2024-02-23T00:54:56.469708+0000","last_unstale":"2024-02-23T00:54:59.923884+0000","last_undegraded":"2024-02-23T00:54:59.923884+0000","last_fullsized":"2024-02-23T00:54:59.923884+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":747,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.441916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"250.7","version":"0'0","reported_seq":14,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:00.307097+0000","last_change":"2024-02-23T00:54:56.470311+0000","last_active":"2024-02-23T00:55:00.307097+0000","last_peered":"2024-02-23T00:55:00.307097+0000","last_clean":"2024-02-23T00:55:00.307097+0000","last_became_active":"2024-02-23T00:54:56.469867+0000","last_became_peered":"2024-02-23T00:54:56.469867+0000","last_unstale":"2024-02-23T00:55:00.307097+0000","last_undegraded":"2024-02-23T00:55:00.307097+0000","last_fullsized":"2024-02-23T00:55:00.307097+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":747,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.441916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"250.6","version":"0'0","reported_seq":14,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.111776+0000","last_change":"2024-02-23T00:54:56.492852+0000","last_active":"2024-02-23T00:55:01.111776+0000","last_peered":"2024-02-23T00:55:01.111776+0000","last_clean":"2024-02-23T00:55:01.111776+0000","last_became_active":"2024-02-23T00:54:56.492233+0000","last_became_peered":"2024-02-23T00:54:56.492233+0000","last_unstale":"2024-02-23T00:55:01.111776+0000","last_undegraded":"2024-02-23T00:55:01.111776+0000","last_fullsized":"2024-02-23T00:55:01.111776+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":747,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.441916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"250.5","version":"0'0","reported_seq":14,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.112117+0000","last_change":"2024-02-23T00:54:56.493582+0000","last_active":"2024-02-23T00:55:01.112117+0000","last_peered":"2024-02-23T00:55:01.112117+0000","last_clean":"2024-02-23T00:55:01.112117+0000","last_became_active":"2024-02-23T00:54:56.493208+0000","last_became_peered":"2024-02-23T00:54:56.493208+0000","last_unstale":"2024-02-23T00:55:01.112117+0000","last_undegraded":"2024-02-23T00:55:01.112117+0000","last_fullsized":"2024-02-23T00:55:01.112117+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":747,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.441916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"250.4","version":"0'0","reported_seq":13,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.925132+0000","last_change":"2024-02-23T00:54:56.479758+0000","last_active":"2024-02-23T00:54:59.925132+0000","last_peered":"2024-02-23T00:54:59.925132+0000","last_clean":"2024-02-23T00:54:59.925132+0000","last_became_active":"2024-02-23T00:54:56.479467+0000","last_became_peered":"2024-02-23T00:54:56.479467+0000","last_unstale":"2024-02-23T00:54:59.925132+0000","last_undegraded":"2024-02-23T00:54:59.925132+0000","last_fullsized":"2024-02-23T00:54:59.925132+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":747,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.441916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"250.3","version":"0'0","reported_seq":14,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:00.306601+0000","last_change":"2024-02-23T00:54:56.467188+0000","last_active":"2024-02-23T00:55:00.306601+0000","last_peered":"2024-02-23T00:55:00.306601+0000","last_clean":"2024-02-23T00:55:00.306601+0000","last_became_active":"2024-02-23T00:54:56.466765+0000","last_became_peered":"2024-02-23T00:54:56.466765+0000","last_unstale":"2024-02-23T00:55:00.306601+0000","last_undegraded":"2024-02-23T00:55:00.306601+0000","last_fullsized":"2024-02-23T00:55:00.306601+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":747,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.441916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"250.2","version":"0'0","reported_seq":14,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:00.306054+0000","last_change":"2024-02-23T00:54:56.485507+0000","last_active":"2024-02-23T00:55:00.306054+0000","last_peered":"2024-02-23T00:55:00.306054+0000","last_clean":"2024-02-23T00:55:00.306054+0000","last_became_active":"2024-02-23T00:54:56.484696+0000","last_became_peered":"2024-02-23T00:54:56.484696+0000","last_unstale":"2024-02-23T00:55:00.306054+0000","last_undegraded":"2024-02-23T00:55:00.306054+0000","last_fullsized":"2024-02-23T00:55:00.306054+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":747,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.441916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"250.1","version":"0'0","reported_seq":14,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:00.305782+0000","last_change":"2024-02-23T00:54:56.484493+0000","last_active":"2024-02-23T00:55:00.305782+0000","last_peered":"2024-02-23T00:55:00.305782+0000","last_clean":"2024-02-23T00:55:00.305782+0000","last_became_active":"2024-02-23T00:54:56.483994+0000","last_became_peered":"2024-02-23T00:54:56.483994+0000","last_unstale":"2024-02-23T00:55:00.305782+0000","last_undegraded":"2024-02-23T00:55:00.305782+0000","last_fullsized":"2024-02-23T00:55:00.305782+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":747,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.441916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"250.0","version":"0'0","reported_seq":13,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.923944+0000","last_change":"2024-02-23T00:54:56.477423+0000","last_active":"2024-02-23T00:54:59.923944+0000","last_peered":"2024-02-23T00:54:59.923944+0000","last_clean":"2024-02-23T00:54:59.923944+0000","last_became_active":"2024-02-23T00:54:56.477108+0000","last_became_peered":"2024-02-23T00:54:56.477108+0000","last_unstale":"2024-02-23T00:54:59.923944+0000","last_undegraded":"2024-02-23T00:54:59.923944+0000","last_fullsized":"2024-02-23T00:54:59.923944+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":747,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.441916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"250.17","version":"0'0","reported_seq":14,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.113375+0000","last_change":"2024-02-23T00:54:56.491434+0000","last_active":"2024-02-23T00:55:01.113375+0000","last_peered":"2024-02-23T00:55:01.113375+0000","last_clean":"2024-02-23T00:55:01.113375+0000","last_became_active":"2024-02-23T00:54:56.491036+0000","last_became_peered":"2024-02-23T00:54:56.491036+0000","last_unstale":"2024-02-23T00:55:01.113375+0000","last_undegraded":"2024-02-23T00:55:01.113375+0000","last_fullsized":"2024-02-23T00:55:01.113375+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":747,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.441916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"234.7","version":"748'15","reported_seq":203,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.925847+0000","last_change":"2024-02-23T00:54:10.268034+0000","last_active":"2024-02-23T00:54:59.925847+0000","last_peered":"2024-02-23T00:54:59.925847+0000","last_clean":"2024-02-23T00:54:59.925847+0000","last_became_active":"2024-02-23T00:52:10.280357+0000","last_became_peered":"2024-02-23T00:52:10.280357+0000","last_unstale":"2024-02-23T00:54:59.925847+0000","last_undegraded":"2024-02-23T00:54:59.925847+0000","last_fullsized":"2024-02-23T00:54:59.925847+0000","mapping_epoch":600,"log_start":"0'0","ondisk_log_start":"0'0","created":600,"last_epoch_clean":601,"parent":"0.0","parent_split_bits":0,"last_scrub":"666'13","last_scrub_stamp":"2024-02-23T00:54:10.267942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_clean_scrub_stamp":"2024-02-23T00:54:10.267942+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":9,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"250.16","version":"0'0","reported_seq":14,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.110758+0000","last_change":"2024-02-23T00:54:56.486419+0000","last_active":"2024-02-23T00:55:01.110758+0000","last_peered":"2024-02-23T00:55:01.110758+0000","last_clean":"2024-02-23T00:55:01.110758+0000","last_became_active":"2024-02-23T00:54:56.483655+0000","last_became_peered":"2024-02-23T00:54:56.483655+0000","last_unstale":"2024-02-23T00:55:01.110758+0000","last_undegraded":"2024-02-23T00:55:01.110758+0000","last_fullsized":"2024-02-23T00:55:01.110758+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":747,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.441916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"234.6","version":"748'11","reported_seq":228,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.926026+0000","last_change":"2024-02-23T00:54:56.513675+0000","last_active":"2024-02-23T00:54:59.926026+0000","last_peered":"2024-02-23T00:54:59.926026+0000","last_clean":"2024-02-23T00:54:59.926026+0000","last_became_active":"2024-02-23T00:54:56.480107+0000","last_became_peered":"2024-02-23T00:54:56.480107+0000","last_unstale":"2024-02-23T00:54:59.926026+0000","last_undegraded":"2024-02-23T00:54:59.926026+0000","last_fullsized":"2024-02-23T00:54:59.926026+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":600,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"629'6","last_scrub_stamp":"2024-02-23T00:53:36.494408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_clean_scrub_stamp":"2024-02-23T00:53:36.494408+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":16,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"250.15","version":"0'0","reported_seq":13,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.925297+0000","last_change":"2024-02-23T00:54:56.493108+0000","last_active":"2024-02-23T00:54:59.925297+0000","last_peered":"2024-02-23T00:54:59.925297+0000","last_clean":"2024-02-23T00:54:59.925297+0000","last_became_active":"2024-02-23T00:54:56.492770+0000","last_became_peered":"2024-02-23T00:54:56.492770+0000","last_unstale":"2024-02-23T00:54:59.925297+0000","last_undegraded":"2024-02-23T00:54:59.925297+0000","last_fullsized":"2024-02-23T00:54:59.925297+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":747,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.441916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"234.5","version":"748'15","reported_seq":227,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.925915+0000","last_change":"2024-02-23T00:54:56.533540+0000","last_active":"2024-02-23T00:54:59.925915+0000","last_peered":"2024-02-23T00:54:59.925915+0000","last_clean":"2024-02-23T00:54:59.925915+0000","last_became_active":"2024-02-23T00:54:56.485687+0000","last_became_peered":"2024-02-23T00:54:56.485687+0000","last_unstale":"2024-02-23T00:54:59.925915+0000","last_undegraded":"2024-02-23T00:54:59.925915+0000","last_fullsized":"2024-02-23T00:54:59.925915+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":600,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"666'6","last_scrub_stamp":"2024-02-23T00:53:32.451126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_clean_scrub_stamp":"2024-02-23T00:53:32.451126+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":16,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"250.14","version":"0'0","reported_seq":14,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:00.306358+0000","last_change":"2024-02-23T00:54:56.471566+0000","last_active":"2024-02-23T00:55:00.306358+0000","last_peered":"2024-02-23T00:55:00.306358+0000","last_clean":"2024-02-23T00:55:00.306358+0000","last_became_active":"2024-02-23T00:54:56.471294+0000","last_became_peered":"2024-02-23T00:54:56.471294+0000","last_unstale":"2024-02-23T00:55:00.306358+0000","last_undegraded":"2024-02-23T00:55:00.306358+0000","last_fullsized":"2024-02-23T00:55:00.306358+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":747,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.441916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"234.4","version":"748'15","reported_seq":146,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.111312+0000","last_change":"2024-02-23T00:54:08.325933+0000","last_active":"2024-02-23T00:55:01.111312+0000","last_peered":"2024-02-23T00:55:01.111312+0000","last_clean":"2024-02-23T00:55:01.111312+0000","last_became_active":"2024-02-23T00:53:34.320586+0000","last_became_peered":"2024-02-23T00:53:34.320586+0000","last_unstale":"2024-02-23T00:55:01.111312+0000","last_undegraded":"2024-02-23T00:55:01.111312+0000","last_fullsized":"2024-02-23T00:55:01.111312+0000","mapping_epoch":676,"log_start":"0'0","ondisk_log_start":"0'0","created":600,"last_epoch_clean":677,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_clean_scrub_stamp":"2024-02-23T00:52:09.256902+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23,"num_objects":5,"num_object_clones":1,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"250.13","version":"0'0","reported_seq":13,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.924693+0000","last_change":"2024-02-23T00:54:56.495174+0000","last_active":"2024-02-23T00:54:59.924693+0000","last_peered":"2024-02-23T00:54:59.924693+0000","last_clean":"2024-02-23T00:54:59.924693+0000","last_became_active":"2024-02-23T00:54:56.494875+0000","last_became_peered":"2024-02-23T00:54:56.494875+0000","last_unstale":"2024-02-23T00:54:59.924693+0000","last_undegraded":"2024-02-23T00:54:59.924693+0000","last_fullsized":"2024-02-23T00:54:59.924693+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":747,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.441916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"234.3","version":"667'5","reported_seq":221,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.112602+0000","last_change":"2024-02-23T00:54:07.348103+0000","last_active":"2024-02-23T00:55:01.112602+0000","last_peered":"2024-02-23T00:55:01.112602+0000","last_clean":"2024-02-23T00:55:01.112602+0000","last_became_active":"2024-02-23T00:53:53.411778+0000","last_became_peered":"2024-02-23T00:53:53.411778+0000","last_unstale":"2024-02-23T00:55:01.112602+0000","last_undegraded":"2024-02-23T00:55:01.112602+0000","last_fullsized":"2024-02-23T00:55:01.112602+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":600,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"667'5","last_scrub_stamp":"2024-02-23T00:54:07.348022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_clean_scrub_stamp":"2024-02-23T00:54:07.348022+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":18,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"250.12","version":"0'0","reported_seq":14,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.111048+0000","last_change":"2024-02-23T00:54:56.487153+0000","last_active":"2024-02-23T00:55:01.111048+0000","last_peered":"2024-02-23T00:55:01.111048+0000","last_clean":"2024-02-23T00:55:01.111048+0000","last_became_active":"2024-02-23T00:54:56.486817+0000","last_became_peered":"2024-02-23T00:54:56.486817+0000","last_unstale":"2024-02-23T00:55:01.111048+0000","last_undegraded":"2024-02-23T00:55:01.111048+0000","last_fullsized":"2024-02-23T00:55:01.111048+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":747,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.441916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"234.2","version":"748'12","reported_seq":233,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.924337+0000","last_change":"2024-02-23T00:54:56.506200+0000","last_active":"2024-02-23T00:54:59.924337+0000","last_peered":"2024-02-23T00:54:59.924337+0000","last_clean":"2024-02-23T00:54:59.924337+0000","last_became_active":"2024-02-23T00:54:56.483023+0000","last_became_peered":"2024-02-23T00:54:56.483023+0000","last_unstale":"2024-02-23T00:54:59.924337+0000","last_undegraded":"2024-02-23T00:54:59.924337+0000","last_fullsized":"2024-02-23T00:54:59.924337+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":600,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"629'10","last_scrub_stamp":"2024-02-23T00:53:34.474122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_clean_scrub_stamp":"2024-02-23T00:53:34.474122+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":26,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":20,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7],"acting":[6,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"250.11","version":"0'0","reported_seq":14,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.111100+0000","last_change":"2024-02-23T00:54:56.496940+0000","last_active":"2024-02-23T00:55:01.111100+0000","last_peered":"2024-02-23T00:55:01.111100+0000","last_clean":"2024-02-23T00:55:01.111100+0000","last_became_active":"2024-02-23T00:54:56.496647+0000","last_became_peered":"2024-02-23T00:54:56.496647+0000","last_unstale":"2024-02-23T00:55:01.111100+0000","last_undegraded":"2024-02-23T00:55:01.111100+0000","last_fullsized":"2024-02-23T00:55:01.111100+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":747,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.441916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"234.1","version":"748'7","reported_seq":25,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.110613+0000","last_change":"2024-02-23T00:54:56.516215+0000","last_active":"2024-02-23T00:55:01.110613+0000","last_peered":"2024-02-23T00:55:01.110613+0000","last_clean":"2024-02-23T00:55:01.110613+0000","last_became_active":"2024-02-23T00:54:56.471953+0000","last_became_peered":"2024-02-23T00:54:56.471953+0000","last_unstale":"2024-02-23T00:55:01.110613+0000","last_undegraded":"2024-02-23T00:55:01.110613+0000","last_fullsized":"2024-02-23T00:55:01.110613+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":600,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"723'5","last_scrub_stamp":"2024-02-23T00:54:43.089897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_clean_scrub_stamp":"2024-02-23T00:54:43.089897+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4000040,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6,"num_write_kb":3912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4000032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"250.10","version":"0'0","reported_seq":14,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.111101+0000","last_change":"2024-02-23T00:54:56.487316+0000","last_active":"2024-02-23T00:55:01.111101+0000","last_peered":"2024-02-23T00:55:01.111101+0000","last_clean":"2024-02-23T00:55:01.111101+0000","last_became_active":"2024-02-23T00:54:56.486255+0000","last_became_peered":"2024-02-23T00:54:56.486255+0000","last_unstale":"2024-02-23T00:55:01.111101+0000","last_undegraded":"2024-02-23T00:55:01.111101+0000","last_fullsized":"2024-02-23T00:55:01.111101+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":747,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:55.441916+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.441916+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"234.0","version":"748'10","reported_seq":197,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:00.305709+0000","last_change":"2024-02-23T00:54:15.372319+0000","last_active":"2024-02-23T00:55:00.305709+0000","last_peered":"2024-02-23T00:55:00.305709+0000","last_clean":"2024-02-23T00:55:00.305709+0000","last_became_active":"2024-02-23T00:52:10.279573+0000","last_became_peered":"2024-02-23T00:52:10.279573+0000","last_unstale":"2024-02-23T00:55:00.305709+0000","last_undegraded":"2024-02-23T00:55:00.305709+0000","last_fullsized":"2024-02-23T00:55:00.305709+0000","mapping_epoch":600,"log_start":"0'0","ondisk_log_start":"0'0","created":600,"last_epoch_clean":601,"parent":"0.0","parent_split_bits":0,"last_scrub":"694'7","last_scrub_stamp":"2024-02-23T00:54:15.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:09.256902+0000","last_clean_scrub_stamp":"2024-02-23T00:54:15.372237+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":29,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":7,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"466'9","reported_seq":923,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.924753+0000","last_change":"2024-02-23T00:54:11.260611+0000","last_active":"2024-02-23T00:54:59.924753+0000","last_peered":"2024-02-23T00:54:59.924753+0000","last_clean":"2024-02-23T00:54:59.924753+0000","last_became_active":"2024-02-23T00:39:18.793414+0000","last_became_peered":"2024-02-23T00:39:18.793414+0000","last_unstale":"2024-02-23T00:54:59.924753+0000","last_undegraded":"2024-02-23T00:54:59.924753+0000","last_fullsized":"2024-02-23T00:54:59.924753+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'9","last_scrub_stamp":"2024-02-23T00:54:11.260489+0000","last_deep_scrub":"466'9","last_deep_scrub_stamp":"2024-02-23T00:49:43.187989+0000","last_clean_scrub_stamp":"2024-02-23T00:54:11.260489+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"470'42","reported_seq":798,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.110492+0000","last_change":"2024-02-23T00:52:17.393167+0000","last_active":"2024-02-23T00:55:01.110492+0000","last_peered":"2024-02-23T00:55:01.110492+0000","last_clean":"2024-02-23T00:55:01.110492+0000","last_became_active":"2024-02-23T00:39:16.859401+0000","last_became_peered":"2024-02-23T00:39:16.859401+0000","last_unstale":"2024-02-23T00:55:01.110492+0000","last_undegraded":"2024-02-23T00:55:01.110492+0000","last_fullsized":"2024-02-23T00:55:01.110492+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"470'42","last_scrub_stamp":"2024-02-23T00:52:17.393084+0000","last_deep_scrub":"470'42","last_deep_scrub_stamp":"2024-02-23T00:49:47.653176+0000","last_clean_scrub_stamp":"2024-02-23T00:52:17.393084+0000","log_size":42,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":112,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.925610+0000","last_change":"2024-02-23T00:53:53.424388+0000","last_active":"2024-02-23T00:54:59.925610+0000","last_peered":"2024-02-23T00:54:59.925610+0000","last_clean":"2024-02-23T00:54:59.925610+0000","last_became_active":"2024-02-23T00:53:53.424032+0000","last_became_peered":"2024-02-23T00:53:53.424032+0000","last_unstale":"2024-02-23T00:54:59.925610+0000","last_undegraded":"2024-02-23T00:54:59.925610+0000","last_fullsized":"2024-02-23T00:54:59.925610+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:40.350368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:04.330426+0000","last_clean_scrub_stamp":"2024-02-23T00:53:40.350368+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":754,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.112111+0000","last_change":"2024-02-23T00:54:56.483272+0000","last_active":"2024-02-23T00:55:01.112111+0000","last_peered":"2024-02-23T00:55:01.112111+0000","last_clean":"2024-02-23T00:55:01.112111+0000","last_became_active":"2024-02-23T00:54:56.482910+0000","last_became_peered":"2024-02-23T00:54:56.482910+0000","last_unstale":"2024-02-23T00:55:01.112111+0000","last_undegraded":"2024-02-23T00:55:01.112111+0000","last_fullsized":"2024-02-23T00:55:01.112111+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:25.491458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:52:25.491458+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":683,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.110839+0000","last_change":"2024-02-23T00:54:56.487168+0000","last_active":"2024-02-23T00:55:01.110839+0000","last_peered":"2024-02-23T00:55:01.110839+0000","last_clean":"2024-02-23T00:55:01.110839+0000","last_became_active":"2024-02-23T00:54:56.486869+0000","last_became_peered":"2024-02-23T00:54:56.486869+0000","last_unstale":"2024-02-23T00:55:01.110839+0000","last_undegraded":"2024-02-23T00:55:01.110839+0000","last_fullsized":"2024-02-23T00:55:01.110839+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:51.439818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:52:51.439818+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"457'21","reported_seq":32,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.112397+0000","last_change":"2024-02-23T00:54:57.851761+0000","last_active":"2024-02-23T00:55:01.112397+0000","last_peered":"2024-02-23T00:55:01.112397+0000","last_clean":"2024-02-23T00:55:01.112397+0000","last_became_active":"2024-02-23T00:54:56.481365+0000","last_became_peered":"2024-02-23T00:54:56.481365+0000","last_unstale":"2024-02-23T00:55:01.112397+0000","last_undegraded":"2024-02-23T00:55:01.112397+0000","last_fullsized":"2024-02-23T00:55:01.112397+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-23T00:54:57.851683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:54:57.851683+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":37,"num_read_kb":30,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":714,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.112552+0000","last_change":"2024-02-23T00:52:25.554460+0000","last_active":"2024-02-23T00:55:01.112552+0000","last_peered":"2024-02-23T00:55:01.112552+0000","last_clean":"2024-02-23T00:55:01.112552+0000","last_became_active":"2024-02-23T00:39:14.772493+0000","last_became_peered":"2024-02-23T00:39:14.772493+0000","last_unstale":"2024-02-23T00:55:01.112552+0000","last_undegraded":"2024-02-23T00:55:01.112552+0000","last_fullsized":"2024-02-23T00:55:01.112552+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:25.554392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:04.528006+0000","last_clean_scrub_stamp":"2024-02-23T00:52:25.554392+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":434,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:00.306432+0000","last_change":"2024-02-23T00:54:56.484384+0000","last_active":"2024-02-23T00:55:00.306432+0000","last_peered":"2024-02-23T00:55:00.306432+0000","last_clean":"2024-02-23T00:55:00.306432+0000","last_became_active":"2024-02-23T00:54:56.482922+0000","last_became_peered":"2024-02-23T00:54:56.482922+0000","last_unstale":"2024-02-23T00:55:00.306432+0000","last_undegraded":"2024-02-23T00:55:00.306432+0000","last_fullsized":"2024-02-23T00:55:00.306432+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:30.466466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:08.335330+0000","last_clean_scrub_stamp":"2024-02-23T00:53:30.466466+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.110506+0000","last_change":"2024-02-23T00:54:56.472469+0000","last_active":"2024-02-23T00:55:01.110506+0000","last_peered":"2024-02-23T00:55:01.110506+0000","last_clean":"2024-02-23T00:55:01.110506+0000","last_became_active":"2024-02-23T00:54:56.470791+0000","last_became_peered":"2024-02-23T00:54:56.470791+0000","last_unstale":"2024-02-23T00:55:01.110506+0000","last_undegraded":"2024-02-23T00:55:01.110506+0000","last_fullsized":"2024-02-23T00:55:01.110506+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:02.821535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:13.867716+0000","last_clean_scrub_stamp":"2024-02-23T00:53:02.821535+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":15,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.112660+0000","last_change":"2024-02-23T00:54:56.473721+0000","last_active":"2024-02-23T00:55:01.112660+0000","last_peered":"2024-02-23T00:55:01.112660+0000","last_clean":"2024-02-23T00:55:01.112660+0000","last_became_active":"2024-02-23T00:54:56.473350+0000","last_became_peered":"2024-02-23T00:54:56.473350+0000","last_unstale":"2024-02-23T00:55:01.112660+0000","last_undegraded":"2024-02-23T00:55:01.112660+0000","last_fullsized":"2024-02-23T00:55:01.112660+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:56.845554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:52:56.845554+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"457'51","reported_seq":809,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.925185+0000","last_change":"2024-02-23T00:54:44.498043+0000","last_active":"2024-02-23T00:54:59.925185+0000","last_peered":"2024-02-23T00:54:59.925185+0000","last_clean":"2024-02-23T00:54:59.925185+0000","last_became_active":"2024-02-23T00:39:16.857413+0000","last_became_peered":"2024-02-23T00:39:16.857413+0000","last_unstale":"2024-02-23T00:54:59.925185+0000","last_undegraded":"2024-02-23T00:54:59.925185+0000","last_fullsized":"2024-02-23T00:54:59.925185+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'51","last_scrub_stamp":"2024-02-23T00:54:44.497963+0000","last_deep_scrub":"88'35","last_deep_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_clean_scrub_stamp":"2024-02-23T00:54:44.497963+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":68,"num_read_kb":56,"num_write":42,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":415,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.111035+0000","last_change":"2024-02-23T00:54:56.490869+0000","last_active":"2024-02-23T00:55:01.111035+0000","last_peered":"2024-02-23T00:55:01.111035+0000","last_clean":"2024-02-23T00:55:01.111035+0000","last_became_active":"2024-02-23T00:54:56.490563+0000","last_became_peered":"2024-02-23T00:54:56.490563+0000","last_unstale":"2024-02-23T00:55:01.111035+0000","last_undegraded":"2024-02-23T00:55:01.111035+0000","last_fullsized":"2024-02-23T00:55:01.111035+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:28.314660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_clean_scrub_stamp":"2024-02-23T00:53:28.314660+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":15,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.112055+0000","last_change":"2024-02-23T00:54:56.495014+0000","last_active":"2024-02-23T00:55:01.112055+0000","last_peered":"2024-02-23T00:55:01.112055+0000","last_clean":"2024-02-23T00:55:01.112055+0000","last_became_active":"2024-02-23T00:54:56.491618+0000","last_became_peered":"2024-02-23T00:54:56.491618+0000","last_unstale":"2024-02-23T00:55:01.112055+0000","last_undegraded":"2024-02-23T00:55:01.112055+0000","last_fullsized":"2024-02-23T00:55:01.112055+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:41.132688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:03.682745+0000","last_clean_scrub_stamp":"2024-02-23T00:53:41.132688+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":747,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.923993+0000","last_change":"2024-02-23T00:54:48.044111+0000","last_active":"2024-02-23T00:54:59.923993+0000","last_peered":"2024-02-23T00:54:59.923993+0000","last_clean":"2024-02-23T00:54:59.923993+0000","last_became_active":"2024-02-23T00:47:28.215636+0000","last_became_peered":"2024-02-23T00:47:28.215636+0000","last_unstale":"2024-02-23T00:54:59.923993+0000","last_undegraded":"2024-02-23T00:54:59.923993+0000","last_fullsized":"2024-02-23T00:54:59.923993+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:48.043996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:42.150875+0000","last_clean_scrub_stamp":"2024-02-23T00:54:48.043996+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":692,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.112013+0000","last_change":"2024-02-23T00:54:56.493837+0000","last_active":"2024-02-23T00:55:01.112013+0000","last_peered":"2024-02-23T00:55:01.112013+0000","last_clean":"2024-02-23T00:55:01.112013+0000","last_became_active":"2024-02-23T00:54:56.492955+0000","last_became_peered":"2024-02-23T00:54:56.492955+0000","last_unstale":"2024-02-23T00:55:01.112013+0000","last_undegraded":"2024-02-23T00:55:01.112013+0000","last_fullsized":"2024-02-23T00:55:01.112013+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:07.482987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:53:07.482987+0000","last_clean_scrub_stamp":"2024-02-23T00:53:07.482987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"457'31","reported_seq":826,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:00.307108+0000","last_change":"2024-02-23T00:53:36.520058+0000","last_active":"2024-02-23T00:55:00.307108+0000","last_peered":"2024-02-23T00:55:00.307108+0000","last_clean":"2024-02-23T00:55:00.307108+0000","last_became_active":"2024-02-23T00:47:28.200653+0000","last_became_peered":"2024-02-23T00:47:28.200653+0000","last_unstale":"2024-02-23T00:55:00.307108+0000","last_undegraded":"2024-02-23T00:55:00.307108+0000","last_fullsized":"2024-02-23T00:55:00.307108+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:53:36.519986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:53:36.519986+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":394,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.924849+0000","last_change":"2024-02-23T00:52:55.382296+0000","last_active":"2024-02-23T00:54:59.924849+0000","last_peered":"2024-02-23T00:54:59.924849+0000","last_clean":"2024-02-23T00:54:59.924849+0000","last_became_active":"2024-02-23T00:47:28.201926+0000","last_became_peered":"2024-02-23T00:47:28.201926+0000","last_unstale":"2024-02-23T00:54:59.924849+0000","last_undegraded":"2024-02-23T00:54:59.924849+0000","last_fullsized":"2024-02-23T00:54:59.924849+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:55.382210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:55.382210+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":733,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.111844+0000","last_change":"2024-02-23T00:54:56.498144+0000","last_active":"2024-02-23T00:55:01.111844+0000","last_peered":"2024-02-23T00:55:01.111844+0000","last_clean":"2024-02-23T00:55:01.111844+0000","last_became_active":"2024-02-23T00:54:56.497533+0000","last_became_peered":"2024-02-23T00:54:56.497533+0000","last_unstale":"2024-02-23T00:55:01.111844+0000","last_undegraded":"2024-02-23T00:55:01.111844+0000","last_fullsized":"2024-02-23T00:55:01.111844+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:07.276549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:09.343884+0000","last_clean_scrub_stamp":"2024-02-23T00:54:07.276549+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":656,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.111260+0000","last_change":"2024-02-23T00:52:16.478287+0000","last_active":"2024-02-23T00:55:01.111260+0000","last_peered":"2024-02-23T00:55:01.111260+0000","last_clean":"2024-02-23T00:55:01.111260+0000","last_became_active":"2024-02-23T00:42:57.675306+0000","last_became_peered":"2024-02-23T00:42:57.675306+0000","last_unstale":"2024-02-23T00:55:01.111260+0000","last_undegraded":"2024-02-23T00:55:01.111260+0000","last_fullsized":"2024-02-23T00:55:01.111260+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:16.478219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:52:16.478219+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"457'20","reported_seq":758,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:00.306544+0000","last_change":"2024-02-23T00:54:58.353511+0000","last_active":"2024-02-23T00:55:00.306544+0000","last_peered":"2024-02-23T00:55:00.306544+0000","last_clean":"2024-02-23T00:55:00.306544+0000","last_became_active":"2024-02-23T00:39:16.777530+0000","last_became_peered":"2024-02-23T00:39:16.777530+0000","last_unstale":"2024-02-23T00:55:00.306544+0000","last_undegraded":"2024-02-23T00:55:00.306544+0000","last_fullsized":"2024-02-23T00:55:00.306544+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'20","last_scrub_stamp":"2024-02-23T00:54:58.353431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:54:58.353431+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":714,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.111594+0000","last_change":"2024-02-23T00:52:13.530732+0000","last_active":"2024-02-23T00:55:01.111594+0000","last_peered":"2024-02-23T00:55:01.111594+0000","last_clean":"2024-02-23T00:55:01.111594+0000","last_became_active":"2024-02-23T00:39:14.780379+0000","last_became_peered":"2024-02-23T00:39:14.780379+0000","last_unstale":"2024-02-23T00:55:01.111594+0000","last_undegraded":"2024-02-23T00:55:01.111594+0000","last_fullsized":"2024-02-23T00:55:01.111594+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:13.530630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:06.575242+0000","last_clean_scrub_stamp":"2024-02-23T00:52:13.530630+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"749'11","reported_seq":531,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.926046+0000","last_change":"2024-02-23T00:54:56.586057+0000","last_active":"2024-02-23T00:54:59.926046+0000","last_peered":"2024-02-23T00:54:59.926046+0000","last_clean":"2024-02-23T00:54:59.926046+0000","last_became_active":"2024-02-23T00:54:56.482145+0000","last_became_peered":"2024-02-23T00:54:56.482145+0000","last_unstale":"2024-02-23T00:54:59.926046+0000","last_undegraded":"2024-02-23T00:54:59.926046+0000","last_fullsized":"2024-02-23T00:54:59.926046+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"376'9","last_scrub_stamp":"2024-02-23T00:52:13.442894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:52:13.442894+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":111,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.111699+0000","last_change":"2024-02-23T00:54:46.332581+0000","last_active":"2024-02-23T00:55:01.111699+0000","last_peered":"2024-02-23T00:55:01.111699+0000","last_clean":"2024-02-23T00:55:01.111699+0000","last_became_active":"2024-02-23T00:53:53.411396+0000","last_became_peered":"2024-02-23T00:53:53.411396+0000","last_unstale":"2024-02-23T00:55:01.111699+0000","last_undegraded":"2024-02-23T00:55:01.111699+0000","last_fullsized":"2024-02-23T00:55:01.111699+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:46.332515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.335752+0000","last_clean_scrub_stamp":"2024-02-23T00:54:46.332515+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"457'27","reported_seq":755,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.111323+0000","last_change":"2024-02-23T00:51:48.537804+0000","last_active":"2024-02-23T00:55:01.111323+0000","last_peered":"2024-02-23T00:55:01.111323+0000","last_clean":"2024-02-23T00:55:01.111323+0000","last_became_active":"2024-02-23T00:39:16.789757+0000","last_became_peered":"2024-02-23T00:39:16.789757+0000","last_unstale":"2024-02-23T00:55:01.111323+0000","last_undegraded":"2024-02-23T00:55:01.111323+0000","last_fullsized":"2024-02-23T00:55:01.111323+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-23T00:51:48.537732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:48.537732+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":719,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.112378+0000","last_change":"2024-02-23T00:54:47.382124+0000","last_active":"2024-02-23T00:55:01.112378+0000","last_peered":"2024-02-23T00:55:01.112378+0000","last_clean":"2024-02-23T00:55:01.112378+0000","last_became_active":"2024-02-23T00:39:14.780526+0000","last_became_peered":"2024-02-23T00:39:14.780526+0000","last_unstale":"2024-02-23T00:55:01.112378+0000","last_undegraded":"2024-02-23T00:55:01.112378+0000","last_fullsized":"2024-02-23T00:55:01.112378+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:47.382054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:54:47.382054+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"749'12","reported_seq":975,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:00.307262+0000","last_change":"2024-02-23T00:54:56.516097+0000","last_active":"2024-02-23T00:55:00.307262+0000","last_peered":"2024-02-23T00:55:00.307262+0000","last_clean":"2024-02-23T00:55:00.307262+0000","last_became_active":"2024-02-23T00:54:56.484042+0000","last_became_peered":"2024-02-23T00:54:56.484042+0000","last_unstale":"2024-02-23T00:55:00.307262+0000","last_undegraded":"2024-02-23T00:55:00.307262+0000","last_fullsized":"2024-02-23T00:55:00.307262+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"694'11","last_scrub_stamp":"2024-02-23T00:54:01.484388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:54:01.484388+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":687,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:00.305829+0000","last_change":"2024-02-23T00:53:53.414508+0000","last_active":"2024-02-23T00:55:00.305829+0000","last_peered":"2024-02-23T00:55:00.305829+0000","last_clean":"2024-02-23T00:55:00.305829+0000","last_became_active":"2024-02-23T00:53:53.413978+0000","last_became_peered":"2024-02-23T00:53:53.413978+0000","last_unstale":"2024-02-23T00:55:00.305829+0000","last_undegraded":"2024-02-23T00:55:00.305829+0000","last_fullsized":"2024-02-23T00:55:00.305829+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:59.798987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:52:59.798987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"457'31","reported_seq":821,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:00.307123+0000","last_change":"2024-02-23T00:53:34.440512+0000","last_active":"2024-02-23T00:55:00.307123+0000","last_peered":"2024-02-23T00:55:00.307123+0000","last_clean":"2024-02-23T00:55:00.307123+0000","last_became_active":"2024-02-23T00:47:28.201225+0000","last_became_peered":"2024-02-23T00:47:28.201225+0000","last_unstale":"2024-02-23T00:55:00.307123+0000","last_undegraded":"2024-02-23T00:55:00.307123+0000","last_fullsized":"2024-02-23T00:55:00.307123+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:53:34.440440+0000","last_deep_scrub":"336'23","last_deep_scrub_stamp":"2024-02-23T00:46:48.320085+0000","last_clean_scrub_stamp":"2024-02-23T00:53:34.440440+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":744,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.924189+0000","last_change":"2024-02-23T00:54:58.028924+0000","last_active":"2024-02-23T00:54:59.924189+0000","last_peered":"2024-02-23T00:54:59.924189+0000","last_clean":"2024-02-23T00:54:59.924189+0000","last_became_active":"2024-02-23T00:54:56.477515+0000","last_became_peered":"2024-02-23T00:54:56.477515+0000","last_unstale":"2024-02-23T00:54:59.924189+0000","last_undegraded":"2024-02-23T00:54:59.924189+0000","last_fullsized":"2024-02-23T00:54:59.924189+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:58.028863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:54:58.028863+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":720,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.111832+0000","last_change":"2024-02-23T00:52:54.389055+0000","last_active":"2024-02-23T00:55:01.111832+0000","last_peered":"2024-02-23T00:55:01.111832+0000","last_clean":"2024-02-23T00:55:01.111832+0000","last_became_active":"2024-02-23T00:39:18.800777+0000","last_became_peered":"2024-02-23T00:39:18.800777+0000","last_unstale":"2024-02-23T00:55:01.111832+0000","last_undegraded":"2024-02-23T00:55:01.111832+0000","last_fullsized":"2024-02-23T00:55:01.111832+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:54.388949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:04.839555+0000","last_clean_scrub_stamp":"2024-02-23T00:52:54.388949+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":656,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.110612+0000","last_change":"2024-02-23T00:53:30.603155+0000","last_active":"2024-02-23T00:55:01.110612+0000","last_peered":"2024-02-23T00:55:01.110612+0000","last_clean":"2024-02-23T00:55:01.110612+0000","last_became_active":"2024-02-23T00:42:57.653279+0000","last_became_peered":"2024-02-23T00:42:57.653279+0000","last_unstale":"2024-02-23T00:55:01.110612+0000","last_undegraded":"2024-02-23T00:55:01.110612+0000","last_fullsized":"2024-02-23T00:55:01.110612+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:30.603090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:53:30.603090+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"694'10","reported_seq":156,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.110418+0000","last_change":"2024-02-23T00:54:50.395466+0000","last_active":"2024-02-23T00:55:01.110418+0000","last_peered":"2024-02-23T00:55:01.110418+0000","last_clean":"2024-02-23T00:55:01.110418+0000","last_became_active":"2024-02-23T00:53:53.407501+0000","last_became_peered":"2024-02-23T00:53:53.407501+0000","last_unstale":"2024-02-23T00:55:01.110418+0000","last_undegraded":"2024-02-23T00:55:01.110418+0000","last_fullsized":"2024-02-23T00:55:01.110418+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"694'10","last_scrub_stamp":"2024-02-23T00:54:50.395367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:54:50.395367+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"457'42","reported_seq":793,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:00.306018+0000","last_change":"2024-02-23T00:54:53.455113+0000","last_active":"2024-02-23T00:55:00.306018+0000","last_peered":"2024-02-23T00:55:00.306018+0000","last_clean":"2024-02-23T00:55:00.306018+0000","last_became_active":"2024-02-23T00:39:16.780130+0000","last_became_peered":"2024-02-23T00:39:16.780130+0000","last_unstale":"2024-02-23T00:55:00.306018+0000","last_undegraded":"2024-02-23T00:55:00.306018+0000","last_fullsized":"2024-02-23T00:55:00.306018+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'42","last_scrub_stamp":"2024-02-23T00:54:53.455036+0000","last_deep_scrub":"423'28","last_deep_scrub_stamp":"2024-02-23T00:48:17.435906+0000","last_clean_scrub_stamp":"2024-02-23T00:54:53.455036+0000","log_size":42,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":46,"num_write":34,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":742,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.926388+0000","last_change":"2024-02-23T00:54:13.439244+0000","last_active":"2024-02-23T00:54:59.926388+0000","last_peered":"2024-02-23T00:54:59.926388+0000","last_clean":"2024-02-23T00:54:59.926388+0000","last_became_active":"2024-02-23T00:47:28.237083+0000","last_became_peered":"2024-02-23T00:47:28.237083+0000","last_unstale":"2024-02-23T00:54:59.926388+0000","last_undegraded":"2024-02-23T00:54:59.926388+0000","last_fullsized":"2024-02-23T00:54:59.926388+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:13.439154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:51.391939+0000","last_clean_scrub_stamp":"2024-02-23T00:54:13.439154+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":734,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.925626+0000","last_change":"2024-02-23T00:52:20.403117+0000","last_active":"2024-02-23T00:54:59.925626+0000","last_peered":"2024-02-23T00:54:59.925626+0000","last_clean":"2024-02-23T00:54:59.925626+0000","last_became_active":"2024-02-23T00:47:30.187386+0000","last_became_peered":"2024-02-23T00:47:30.187386+0000","last_unstale":"2024-02-23T00:54:59.925626+0000","last_undegraded":"2024-02-23T00:54:59.925626+0000","last_fullsized":"2024-02-23T00:54:59.925626+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:20.403031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:46.686764+0000","last_clean_scrub_stamp":"2024-02-23T00:52:20.403031+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":674,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.111996+0000","last_change":"2024-02-23T00:53:32.258905+0000","last_active":"2024-02-23T00:55:01.111996+0000","last_peered":"2024-02-23T00:55:01.111996+0000","last_clean":"2024-02-23T00:55:01.111996+0000","last_became_active":"2024-02-23T00:47:28.212307+0000","last_became_peered":"2024-02-23T00:47:28.212307+0000","last_unstale":"2024-02-23T00:55:01.111996+0000","last_undegraded":"2024-02-23T00:55:01.111996+0000","last_fullsized":"2024-02-23T00:55:01.111996+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:32.258822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:53:32.258822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":741,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:00.306451+0000","last_change":"2024-02-23T00:53:53.414314+0000","last_active":"2024-02-23T00:55:00.306451+0000","last_peered":"2024-02-23T00:55:00.306451+0000","last_clean":"2024-02-23T00:55:00.306451+0000","last_became_active":"2024-02-23T00:53:53.413534+0000","last_became_peered":"2024-02-23T00:53:53.413534+0000","last_unstale":"2024-02-23T00:55:00.306451+0000","last_undegraded":"2024-02-23T00:55:00.306451+0000","last_fullsized":"2024-02-23T00:55:00.306451+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:51.722289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:10.493340+0000","last_clean_scrub_stamp":"2024-02-23T00:52:51.722289+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"457'52","reported_seq":691,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.924523+0000","last_change":"2024-02-23T00:54:57.411045+0000","last_active":"2024-02-23T00:54:59.924523+0000","last_peered":"2024-02-23T00:54:59.924523+0000","last_clean":"2024-02-23T00:54:59.924523+0000","last_became_active":"2024-02-23T00:54:56.476539+0000","last_became_peered":"2024-02-23T00:54:56.476539+0000","last_unstale":"2024-02-23T00:54:59.924523+0000","last_undegraded":"2024-02-23T00:54:59.924523+0000","last_fullsized":"2024-02-23T00:54:59.924523+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'52","last_scrub_stamp":"2024-02-23T00:53:02.966145+0000","last_deep_scrub":"457'52","last_deep_scrub_stamp":"2024-02-23T00:53:02.966145+0000","last_clean_scrub_stamp":"2024-02-23T00:53:02.966145+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":75,"num_read_kb":61,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":714,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.112249+0000","last_change":"2024-02-23T00:52:18.475876+0000","last_active":"2024-02-23T00:55:01.112249+0000","last_peered":"2024-02-23T00:55:01.112249+0000","last_clean":"2024-02-23T00:55:01.112249+0000","last_became_active":"2024-02-23T00:39:14.779193+0000","last_became_peered":"2024-02-23T00:39:14.779193+0000","last_unstale":"2024-02-23T00:55:01.112249+0000","last_undegraded":"2024-02-23T00:55:01.112249+0000","last_fullsized":"2024-02-23T00:55:01.112249+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:18.475801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:18.475801+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":769,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:00.306461+0000","last_change":"2024-02-23T00:54:45.474935+0000","last_active":"2024-02-23T00:55:00.306461+0000","last_peered":"2024-02-23T00:55:00.306461+0000","last_clean":"2024-02-23T00:55:00.306461+0000","last_became_active":"2024-02-23T00:53:53.408081+0000","last_became_peered":"2024-02-23T00:53:53.408081+0000","last_unstale":"2024-02-23T00:55:00.306461+0000","last_undegraded":"2024-02-23T00:55:00.306461+0000","last_fullsized":"2024-02-23T00:55:00.306461+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:45.474840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:54:45.474840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":686,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.926367+0000","last_change":"2024-02-23T00:54:56.480593+0000","last_active":"2024-02-23T00:54:59.926367+0000","last_peered":"2024-02-23T00:54:59.926367+0000","last_clean":"2024-02-23T00:54:59.926367+0000","last_became_active":"2024-02-23T00:54:56.479532+0000","last_became_peered":"2024-02-23T00:54:56.479532+0000","last_unstale":"2024-02-23T00:54:59.926367+0000","last_undegraded":"2024-02-23T00:54:59.926367+0000","last_fullsized":"2024-02-23T00:54:59.926367+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:06.983205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:53:06.983205+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":745,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.924847+0000","last_change":"2024-02-23T00:54:58.615685+0000","last_active":"2024-02-23T00:54:59.924847+0000","last_peered":"2024-02-23T00:54:59.924847+0000","last_clean":"2024-02-23T00:54:59.924847+0000","last_became_active":"2024-02-23T00:54:56.485549+0000","last_became_peered":"2024-02-23T00:54:56.485549+0000","last_unstale":"2024-02-23T00:54:59.924847+0000","last_undegraded":"2024-02-23T00:54:59.924847+0000","last_fullsized":"2024-02-23T00:54:59.924847+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:58.615563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:54:58.615563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"457'44","reported_seq":35,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.111964+0000","last_change":"2024-02-23T00:54:56.528004+0000","last_active":"2024-02-23T00:55:01.111964+0000","last_peered":"2024-02-23T00:55:01.111964+0000","last_clean":"2024-02-23T00:55:01.111964+0000","last_became_active":"2024-02-23T00:54:56.491063+0000","last_became_peered":"2024-02-23T00:54:56.491063+0000","last_unstale":"2024-02-23T00:55:01.111964+0000","last_undegraded":"2024-02-23T00:55:01.111964+0000","last_fullsized":"2024-02-23T00:55:01.111964+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'44","last_scrub_stamp":"2024-02-23T00:54:41.254929+0000","last_deep_scrub":"457'44","last_deep_scrub_stamp":"2024-02-23T00:49:46.644911+0000","last_clean_scrub_stamp":"2024-02-23T00:54:41.254929+0000","log_size":44,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":739,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.925969+0000","last_change":"2024-02-23T00:54:56.479365+0000","last_active":"2024-02-23T00:54:59.925969+0000","last_peered":"2024-02-23T00:54:59.925969+0000","last_clean":"2024-02-23T00:54:59.925969+0000","last_became_active":"2024-02-23T00:54:56.478518+0000","last_became_peered":"2024-02-23T00:54:56.478518+0000","last_unstale":"2024-02-23T00:54:59.925969+0000","last_undegraded":"2024-02-23T00:54:59.925969+0000","last_fullsized":"2024-02-23T00:54:59.925969+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:51.380778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:51.380778+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":735,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.112463+0000","last_change":"2024-02-23T00:52:07.536352+0000","last_active":"2024-02-23T00:55:01.112463+0000","last_peered":"2024-02-23T00:55:01.112463+0000","last_clean":"2024-02-23T00:55:01.112463+0000","last_became_active":"2024-02-23T00:47:37.184240+0000","last_became_peered":"2024-02-23T00:47:37.184240+0000","last_unstale":"2024-02-23T00:55:01.112463+0000","last_undegraded":"2024-02-23T00:55:01.112463+0000","last_fullsized":"2024-02-23T00:55:01.112463+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":385,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:07.536288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:52:07.536288+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":665,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.112929+0000","last_change":"2024-02-23T00:52:55.401053+0000","last_active":"2024-02-23T00:55:01.112929+0000","last_peered":"2024-02-23T00:55:01.112929+0000","last_clean":"2024-02-23T00:55:01.112929+0000","last_became_active":"2024-02-23T00:42:58.726652+0000","last_became_peered":"2024-02-23T00:42:58.726652+0000","last_unstale":"2024-02-23T00:55:01.112929+0000","last_undegraded":"2024-02-23T00:55:01.112929+0000","last_fullsized":"2024-02-23T00:55:01.112929+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:55.400946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:52:55.400946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":743,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.111833+0000","last_change":"2024-02-23T00:54:56.490633+0000","last_active":"2024-02-23T00:55:01.111833+0000","last_peered":"2024-02-23T00:55:01.111833+0000","last_clean":"2024-02-23T00:55:01.111833+0000","last_became_active":"2024-02-23T00:54:56.489603+0000","last_became_peered":"2024-02-23T00:54:56.489603+0000","last_unstale":"2024-02-23T00:55:01.111833+0000","last_undegraded":"2024-02-23T00:55:01.111833+0000","last_fullsized":"2024-02-23T00:55:01.111833+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:03.256917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:59.694033+0000","last_clean_scrub_stamp":"2024-02-23T00:54:03.256917+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"457'30","reported_seq":29,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.113115+0000","last_change":"2024-02-23T00:54:56.514535+0000","last_active":"2024-02-23T00:55:01.113115+0000","last_peered":"2024-02-23T00:55:01.113115+0000","last_clean":"2024-02-23T00:55:01.113115+0000","last_became_active":"2024-02-23T00:54:56.485739+0000","last_became_peered":"2024-02-23T00:54:56.485739+0000","last_unstale":"2024-02-23T00:55:01.113115+0000","last_undegraded":"2024-02-23T00:55:01.113115+0000","last_fullsized":"2024-02-23T00:55:01.113115+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'30","last_scrub_stamp":"2024-02-23T00:52:16.690860+0000","last_deep_scrub":"95'18","last_deep_scrub_stamp":"2024-02-23T00:42:01.195545+0000","last_clean_scrub_stamp":"2024-02-23T00:52:16.690860+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":749,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.924885+0000","last_change":"2024-02-23T00:54:56.478956+0000","last_active":"2024-02-23T00:54:59.924885+0000","last_peered":"2024-02-23T00:54:59.924885+0000","last_clean":"2024-02-23T00:54:59.924885+0000","last_became_active":"2024-02-23T00:54:56.478358+0000","last_became_peered":"2024-02-23T00:54:56.478358+0000","last_unstale":"2024-02-23T00:54:59.924885+0000","last_undegraded":"2024-02-23T00:54:59.924885+0000","last_fullsized":"2024-02-23T00:54:59.924885+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:53.023449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:54:53.023449+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.110698+0000","last_change":"2024-02-23T00:54:56.472645+0000","last_active":"2024-02-23T00:55:01.110698+0000","last_peered":"2024-02-23T00:55:01.110698+0000","last_clean":"2024-02-23T00:55:01.110698+0000","last_became_active":"2024-02-23T00:54:56.471337+0000","last_became_peered":"2024-02-23T00:54:56.471337+0000","last_unstale":"2024-02-23T00:55:01.110698+0000","last_undegraded":"2024-02-23T00:55:01.110698+0000","last_fullsized":"2024-02-23T00:55:01.110698+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:51.399939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:54:51.399939+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"67'1","reported_seq":786,"reported_epoch":752,"state":"active+remapped+backfill_toofull","last_fresh":"2024-02-23T00:55:01.112003+0000","last_change":"2024-02-23T00:54:54.407131+0000","last_active":"2024-02-23T00:55:01.112003+0000","last_peered":"2024-02-23T00:55:01.112003+0000","last_clean":"2024-02-23T00:53:51.591230+0000","last_became_active":"2024-02-23T00:53:54.403839+0000","last_became_peered":"2024-02-23T00:53:54.403839+0000","last_unstale":"2024-02-23T00:55:01.112003+0000","last_undegraded":"2024-02-23T00:55:01.112003+0000","last_fullsized":"2024-02-23T00:55:01.112003+0000","mapping_epoch":694,"log_start":"67'1","ondisk_log_start":"67'1","created":65,"last_epoch_clean":670,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:51:02.380721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:51:02.380721+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,6],"avail_no_missing":["1","5","6"],"object_location_counts":[{"shards":"1,5,6","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":691,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.924556+0000","last_change":"2024-02-23T00:54:46.510566+0000","last_active":"2024-02-23T00:54:59.924556+0000","last_peered":"2024-02-23T00:54:59.924556+0000","last_clean":"2024-02-23T00:54:59.924556+0000","last_became_active":"2024-02-23T00:53:53.420593+0000","last_became_peered":"2024-02-23T00:53:53.420593+0000","last_unstale":"2024-02-23T00:54:59.924556+0000","last_undegraded":"2024-02-23T00:54:59.924556+0000","last_fullsized":"2024-02-23T00:54:59.924556+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:46.510475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:44:35.360054+0000","last_clean_scrub_stamp":"2024-02-23T00:54:46.510475+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":749,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.925510+0000","last_change":"2024-02-23T00:54:56.487304+0000","last_active":"2024-02-23T00:54:59.925510+0000","last_peered":"2024-02-23T00:54:59.925510+0000","last_clean":"2024-02-23T00:54:59.925510+0000","last_became_active":"2024-02-23T00:54:56.486967+0000","last_became_peered":"2024-02-23T00:54:56.486967+0000","last_unstale":"2024-02-23T00:54:59.925510+0000","last_undegraded":"2024-02-23T00:54:59.925510+0000","last_fullsized":"2024-02-23T00:54:59.925510+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:38.350917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:53:38.350917+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"457'26","reported_seq":769,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.926096+0000","last_change":"2024-02-23T00:54:01.501056+0000","last_active":"2024-02-23T00:54:59.926096+0000","last_peered":"2024-02-23T00:54:59.926096+0000","last_clean":"2024-02-23T00:54:59.926096+0000","last_became_active":"2024-02-23T00:39:16.858880+0000","last_became_peered":"2024-02-23T00:39:16.858880+0000","last_unstale":"2024-02-23T00:54:59.926096+0000","last_undegraded":"2024-02-23T00:54:59.926096+0000","last_fullsized":"2024-02-23T00:54:59.926096+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-23T00:54:01.500983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:54:01.500983+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":749,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.924092+0000","last_change":"2024-02-23T00:54:56.478047+0000","last_active":"2024-02-23T00:54:59.924092+0000","last_peered":"2024-02-23T00:54:59.924092+0000","last_clean":"2024-02-23T00:54:59.924092+0000","last_became_active":"2024-02-23T00:54:56.477073+0000","last_became_peered":"2024-02-23T00:54:56.477073+0000","last_unstale":"2024-02-23T00:54:59.924092+0000","last_undegraded":"2024-02-23T00:54:59.924092+0000","last_fullsized":"2024-02-23T00:54:59.924092+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:26.398833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:49.977248+0000","last_clean_scrub_stamp":"2024-02-23T00:52:26.398833+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":733,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.112786+0000","last_change":"2024-02-23T00:52:49.465113+0000","last_active":"2024-02-23T00:55:01.112786+0000","last_peered":"2024-02-23T00:55:01.112786+0000","last_clean":"2024-02-23T00:55:01.112786+0000","last_became_active":"2024-02-23T00:47:28.240482+0000","last_became_peered":"2024-02-23T00:47:28.240482+0000","last_unstale":"2024-02-23T00:55:01.112786+0000","last_undegraded":"2024-02-23T00:55:01.112786+0000","last_fullsized":"2024-02-23T00:55:01.112786+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:49.465046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:49.465046+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":105,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.112438+0000","last_change":"2024-02-23T00:53:53.411112+0000","last_active":"2024-02-23T00:55:01.112438+0000","last_peered":"2024-02-23T00:55:01.112438+0000","last_clean":"2024-02-23T00:55:01.112438+0000","last_became_active":"2024-02-23T00:53:53.410801+0000","last_became_peered":"2024-02-23T00:53:53.410801+0000","last_unstale":"2024-02-23T00:55:01.112438+0000","last_undegraded":"2024-02-23T00:55:01.112438+0000","last_fullsized":"2024-02-23T00:55:01.112438+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:53.881152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:53.881152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":759,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.111917+0000","last_change":"2024-02-23T00:54:08.396938+0000","last_active":"2024-02-23T00:55:01.111917+0000","last_peered":"2024-02-23T00:55:01.111917+0000","last_clean":"2024-02-23T00:55:01.111917+0000","last_became_active":"2024-02-23T00:47:45.182550+0000","last_became_peered":"2024-02-23T00:47:45.182550+0000","last_unstale":"2024-02-23T00:55:01.111917+0000","last_undegraded":"2024-02-23T00:55:01.111917+0000","last_fullsized":"2024-02-23T00:55:01.111917+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:08.396857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:54:08.396857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":677,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.925450+0000","last_change":"2024-02-23T00:52:50.382543+0000","last_active":"2024-02-23T00:54:59.925450+0000","last_peered":"2024-02-23T00:54:59.925450+0000","last_clean":"2024-02-23T00:54:59.925450+0000","last_became_active":"2024-02-23T00:47:28.233475+0000","last_became_peered":"2024-02-23T00:47:28.233475+0000","last_unstale":"2024-02-23T00:54:59.925450+0000","last_undegraded":"2024-02-23T00:54:59.925450+0000","last_fullsized":"2024-02-23T00:54:59.925450+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:50.382430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:52:50.382430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":71,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.925917+0000","last_change":"2024-02-23T00:53:45.294558+0000","last_active":"2024-02-23T00:54:59.925917+0000","last_peered":"2024-02-23T00:54:59.925917+0000","last_clean":"2024-02-23T00:54:59.925917+0000","last_became_active":"2024-02-23T00:53:45.294157+0000","last_became_peered":"2024-02-23T00:53:45.294157+0000","last_unstale":"2024-02-23T00:54:59.925917+0000","last_undegraded":"2024-02-23T00:54:59.925917+0000","last_fullsized":"2024-02-23T00:54:59.925917+0000","mapping_epoch":685,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":686,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:39.124251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.264240+0000","last_clean_scrub_stamp":"2024-02-23T00:53:39.124251+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"457'23","reported_seq":806,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.111759+0000","last_change":"2024-02-23T00:54:56.954735+0000","last_active":"2024-02-23T00:55:01.111759+0000","last_peered":"2024-02-23T00:55:01.111759+0000","last_clean":"2024-02-23T00:55:01.111759+0000","last_became_active":"2024-02-23T00:54:56.496397+0000","last_became_peered":"2024-02-23T00:54:56.496397+0000","last_unstale":"2024-02-23T00:55:01.111759+0000","last_undegraded":"2024-02-23T00:55:01.111759+0000","last_fullsized":"2024-02-23T00:55:01.111759+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'23","last_scrub_stamp":"2024-02-23T00:51:42.560138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:51:42.560138+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":48,"num_read_kb":36,"num_write":26,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.110413+0000","last_change":"2024-02-23T00:54:56.488575+0000","last_active":"2024-02-23T00:55:01.110413+0000","last_peered":"2024-02-23T00:55:01.110413+0000","last_clean":"2024-02-23T00:55:01.110413+0000","last_became_active":"2024-02-23T00:54:56.487361+0000","last_became_peered":"2024-02-23T00:54:56.487361+0000","last_unstale":"2024-02-23T00:55:01.110413+0000","last_undegraded":"2024-02-23T00:55:01.110413+0000","last_fullsized":"2024-02-23T00:55:01.110413+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:54.409556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:44.659246+0000","last_clean_scrub_stamp":"2024-02-23T00:54:54.409556+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"707'6","reported_seq":1693,"reported_epoch":753,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.313938+0000","last_change":"2024-02-23T00:54:56.956351+0000","last_active":"2024-02-23T00:55:01.313938+0000","last_peered":"2024-02-23T00:55:01.313938+0000","last_clean":"2024-02-23T00:55:01.313938+0000","last_became_active":"2024-02-23T00:54:56.497240+0000","last_became_peered":"2024-02-23T00:54:56.497240+0000","last_unstale":"2024-02-23T00:55:01.313938+0000","last_undegraded":"2024-02-23T00:55:01.313938+0000","last_fullsized":"2024-02-23T00:55:01.313938+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"707'6","last_scrub_stamp":"2024-02-23T00:54:06.324310+0000","last_deep_scrub":"707'6","last_deep_scrub_stamp":"2024-02-23T00:54:06.324310+0000","last_clean_scrub_stamp":"2024-02-23T00:54:06.324310+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":934,"num_read_kb":929,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":751,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.925950+0000","last_change":"2024-02-23T00:52:10.429013+0000","last_active":"2024-02-23T00:54:59.925950+0000","last_peered":"2024-02-23T00:54:59.925950+0000","last_clean":"2024-02-23T00:54:59.925950+0000","last_became_active":"2024-02-23T00:47:52.552244+0000","last_became_peered":"2024-02-23T00:47:52.552244+0000","last_unstale":"2024-02-23T00:54:59.925950+0000","last_undegraded":"2024-02-23T00:54:59.925950+0000","last_fullsized":"2024-02-23T00:54:59.925950+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.428945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.428945+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":665,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.112088+0000","last_change":"2024-02-23T00:53:34.687443+0000","last_active":"2024-02-23T00:55:01.112088+0000","last_peered":"2024-02-23T00:55:01.112088+0000","last_clean":"2024-02-23T00:55:01.112088+0000","last_became_active":"2024-02-23T00:47:28.212516+0000","last_became_peered":"2024-02-23T00:47:28.212516+0000","last_unstale":"2024-02-23T00:55:01.112088+0000","last_undegraded":"2024-02-23T00:55:01.112088+0000","last_fullsized":"2024-02-23T00:55:01.112088+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:34.687347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:53:34.687347+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.112590+0000","last_change":"2024-02-23T00:54:56.489042+0000","last_active":"2024-02-23T00:55:01.112590+0000","last_peered":"2024-02-23T00:55:01.112590+0000","last_clean":"2024-02-23T00:55:01.112590+0000","last_became_active":"2024-02-23T00:54:56.484927+0000","last_became_peered":"2024-02-23T00:54:56.484927+0000","last_unstale":"2024-02-23T00:55:01.112590+0000","last_undegraded":"2024-02-23T00:55:01.112590+0000","last_fullsized":"2024-02-23T00:55:01.112590+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:17.704169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:52:17.704169+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"694'50","reported_seq":1012,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.110875+0000","last_change":"2024-02-23T00:53:54.363459+0000","last_active":"2024-02-23T00:55:01.110875+0000","last_peered":"2024-02-23T00:55:01.110875+0000","last_clean":"2024-02-23T00:55:01.110875+0000","last_became_active":"2024-02-23T00:53:53.412119+0000","last_became_peered":"2024-02-23T00:53:53.412119+0000","last_unstale":"2024-02-23T00:55:01.110875+0000","last_undegraded":"2024-02-23T00:55:01.110875+0000","last_fullsized":"2024-02-23T00:55:01.110875+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'48","last_scrub_stamp":"2024-02-23T00:53:02.417518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:53:02.417518+0000","log_size":50,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":78,"num_read_kb":62,"num_write":44,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":1432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":724,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.112277+0000","last_change":"2024-02-23T00:52:09.366353+0000","last_active":"2024-02-23T00:55:01.112277+0000","last_peered":"2024-02-23T00:55:01.112277+0000","last_clean":"2024-02-23T00:55:01.112277+0000","last_became_active":"2024-02-23T00:39:14.784015+0000","last_became_peered":"2024-02-23T00:39:14.784015+0000","last_unstale":"2024-02-23T00:55:01.112277+0000","last_undegraded":"2024-02-23T00:55:01.112277+0000","last_fullsized":"2024-02-23T00:55:01.112277+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:09.366281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:09.366281+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":493,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:00.307293+0000","last_change":"2024-02-23T00:53:53.417644+0000","last_active":"2024-02-23T00:55:00.307293+0000","last_peered":"2024-02-23T00:55:00.307293+0000","last_clean":"2024-02-23T00:55:00.307293+0000","last_became_active":"2024-02-23T00:53:53.417293+0000","last_became_peered":"2024-02-23T00:53:53.417293+0000","last_unstale":"2024-02-23T00:55:00.307293+0000","last_undegraded":"2024-02-23T00:55:00.307293+0000","last_fullsized":"2024-02-23T00:55:00.307293+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:33.479056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:53:33.479056+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":401,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:00.306603+0000","last_change":"2024-02-23T00:54:44.504717+0000","last_active":"2024-02-23T00:55:00.306603+0000","last_peered":"2024-02-23T00:55:00.306603+0000","last_clean":"2024-02-23T00:55:00.306603+0000","last_became_active":"2024-02-23T00:47:28.232194+0000","last_became_peered":"2024-02-23T00:47:28.232194+0000","last_unstale":"2024-02-23T00:55:00.306603+0000","last_undegraded":"2024-02-23T00:55:00.306603+0000","last_fullsized":"2024-02-23T00:55:00.306603+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:44.504627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:19.722210+0000","last_clean_scrub_stamp":"2024-02-23T00:54:44.504627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3","version":"749'11","reported_seq":20,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.112308+0000","last_change":"2024-02-23T00:54:56.527969+0000","last_active":"2024-02-23T00:55:01.112308+0000","last_peered":"2024-02-23T00:55:01.112308+0000","last_clean":"2024-02-23T00:55:01.112308+0000","last_became_active":"2024-02-23T00:54:56.492101+0000","last_became_peered":"2024-02-23T00:54:56.492101+0000","last_unstale":"2024-02-23T00:55:01.112308+0000","last_undegraded":"2024-02-23T00:55:01.112308+0000","last_fullsized":"2024-02-23T00:55:01.112308+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"656'10","last_scrub_stamp":"2024-02-23T00:54:03.457066+0000","last_deep_scrub":"91'9","last_deep_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_clean_scrub_stamp":"2024-02-23T00:54:03.457066+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"457'26","reported_seq":757,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.110078+0000","last_change":"2024-02-23T00:52:14.519112+0000","last_active":"2024-02-23T00:55:01.110078+0000","last_peered":"2024-02-23T00:55:01.110078+0000","last_clean":"2024-02-23T00:55:01.110078+0000","last_became_active":"2024-02-23T00:39:16.787468+0000","last_became_peered":"2024-02-23T00:39:16.787468+0000","last_unstale":"2024-02-23T00:55:01.110078+0000","last_undegraded":"2024-02-23T00:55:01.110078+0000","last_fullsized":"2024-02-23T00:55:01.110078+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-23T00:52:14.519049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:52:14.519049+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":36,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":755,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.111586+0000","last_change":"2024-02-23T00:54:56.493358+0000","last_active":"2024-02-23T00:55:01.111586+0000","last_peered":"2024-02-23T00:55:01.111586+0000","last_clean":"2024-02-23T00:55:01.111586+0000","last_became_active":"2024-02-23T00:54:56.492660+0000","last_became_peered":"2024-02-23T00:54:56.492660+0000","last_unstale":"2024-02-23T00:55:01.111586+0000","last_undegraded":"2024-02-23T00:55:01.111586+0000","last_fullsized":"2024-02-23T00:55:01.111586+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:49.373458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:54:49.373458+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"107'2","reported_seq":101,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.110889+0000","last_change":"2024-02-23T00:54:59.472787+0000","last_active":"2024-02-23T00:55:01.110889+0000","last_peered":"2024-02-23T00:55:01.110889+0000","last_clean":"2024-02-23T00:55:01.110889+0000","last_became_active":"2024-02-23T00:54:59.472419+0000","last_became_peered":"2024-02-23T00:54:59.472419+0000","last_unstale":"2024-02-23T00:55:01.110889+0000","last_undegraded":"2024-02-23T00:55:01.110889+0000","last_fullsized":"2024-02-23T00:55:01.110889+0000","mapping_epoch":750,"log_start":"107'2","ondisk_log_start":"107'2","created":65,"last_epoch_clean":751,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'2","last_scrub_stamp":"2024-02-23T00:54:42.271666+0000","last_deep_scrub":"107'2","last_deep_scrub_stamp":"2024-02-23T00:42:48.362739+0000","last_clean_scrub_stamp":"2024-02-23T00:54:42.271666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":695,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.925711+0000","last_change":"2024-02-23T00:54:56.479146+0000","last_active":"2024-02-23T00:54:59.925711+0000","last_peered":"2024-02-23T00:54:59.925711+0000","last_clean":"2024-02-23T00:54:59.925711+0000","last_became_active":"2024-02-23T00:54:56.478830+0000","last_became_peered":"2024-02-23T00:54:56.478830+0000","last_unstale":"2024-02-23T00:54:59.925711+0000","last_undegraded":"2024-02-23T00:54:59.925711+0000","last_fullsized":"2024-02-23T00:54:59.925711+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:00.542667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:54:00.542667+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.110802+0000","last_change":"2024-02-23T00:54:56.489053+0000","last_active":"2024-02-23T00:55:01.110802+0000","last_peered":"2024-02-23T00:55:01.110802+0000","last_clean":"2024-02-23T00:55:01.110802+0000","last_became_active":"2024-02-23T00:54:56.488230+0000","last_became_peered":"2024-02-23T00:54:56.488230+0000","last_unstale":"2024-02-23T00:55:01.110802+0000","last_undegraded":"2024-02-23T00:55:01.110802+0000","last_fullsized":"2024-02-23T00:55:01.110802+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:05.859124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_clean_scrub_stamp":"2024-02-23T00:53:05.859124+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"457'31","reported_seq":836,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:00.306326+0000","last_change":"2024-02-23T00:53:54.307638+0000","last_active":"2024-02-23T00:55:00.306326+0000","last_peered":"2024-02-23T00:55:00.306326+0000","last_clean":"2024-02-23T00:55:00.306326+0000","last_became_active":"2024-02-23T00:53:53.410425+0000","last_became_peered":"2024-02-23T00:53:53.410425+0000","last_unstale":"2024-02-23T00:55:00.306326+0000","last_undegraded":"2024-02-23T00:55:00.306326+0000","last_fullsized":"2024-02-23T00:55:00.306326+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:52:58.827562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:52:58.827562+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":58,"num_read_kb":47,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"95'4","reported_seq":775,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:00.307343+0000","last_change":"2024-02-23T00:54:56.536357+0000","last_active":"2024-02-23T00:55:00.307343+0000","last_peered":"2024-02-23T00:55:00.307343+0000","last_clean":"2024-02-23T00:55:00.307343+0000","last_became_active":"2024-02-23T00:54:56.485259+0000","last_became_peered":"2024-02-23T00:54:56.485259+0000","last_unstale":"2024-02-23T00:55:00.307343+0000","last_undegraded":"2024-02-23T00:55:00.307343+0000","last_fullsized":"2024-02-23T00:55:00.307343+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'4","last_scrub_stamp":"2024-02-23T00:53:38.580857+0000","last_deep_scrub":"95'4","last_deep_scrub_stamp":"2024-02-23T00:41:43.874397+0000","last_clean_scrub_stamp":"2024-02-23T00:53:38.580857+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":183,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.925369+0000","last_change":"2024-02-23T00:53:34.318174+0000","last_active":"2024-02-23T00:54:59.925369+0000","last_peered":"2024-02-23T00:54:59.925369+0000","last_clean":"2024-02-23T00:54:59.925369+0000","last_became_active":"2024-02-23T00:53:34.317883+0000","last_became_peered":"2024-02-23T00:53:34.317883+0000","last_unstale":"2024-02-23T00:54:59.925369+0000","last_undegraded":"2024-02-23T00:54:59.925369+0000","last_fullsized":"2024-02-23T00:54:59.925369+0000","mapping_epoch":676,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":677,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:23.776775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:23.776775+0000","last_clean_scrub_stamp":"2024-02-23T00:52:23.776775+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":696,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.926122+0000","last_change":"2024-02-23T00:54:56.481187+0000","last_active":"2024-02-23T00:54:59.926122+0000","last_peered":"2024-02-23T00:54:59.926122+0000","last_clean":"2024-02-23T00:54:59.926122+0000","last_became_active":"2024-02-23T00:54:56.480645+0000","last_became_peered":"2024-02-23T00:54:56.480645+0000","last_unstale":"2024-02-23T00:54:59.926122+0000","last_undegraded":"2024-02-23T00:54:59.926122+0000","last_fullsized":"2024-02-23T00:54:59.926122+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:02.354546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:53:02.354546+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"457'27","reported_seq":29,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.113067+0000","last_change":"2024-02-23T00:54:56.506255+0000","last_active":"2024-02-23T00:55:01.113067+0000","last_peered":"2024-02-23T00:55:01.113067+0000","last_clean":"2024-02-23T00:55:01.113067+0000","last_became_active":"2024-02-23T00:54:56.479776+0000","last_became_peered":"2024-02-23T00:54:56.479776+0000","last_unstale":"2024-02-23T00:55:01.113067+0000","last_undegraded":"2024-02-23T00:55:01.113067+0000","last_fullsized":"2024-02-23T00:55:01.113067+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-23T00:54:03.163053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:54:03.163053+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":49,"num_read_kb":41,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":407,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.110672+0000","last_change":"2024-02-23T00:54:56.492751+0000","last_active":"2024-02-23T00:55:01.110672+0000","last_peered":"2024-02-23T00:55:01.110672+0000","last_clean":"2024-02-23T00:55:01.110672+0000","last_became_active":"2024-02-23T00:54:56.492365+0000","last_became_peered":"2024-02-23T00:54:56.492365+0000","last_unstale":"2024-02-23T00:55:01.110672+0000","last_undegraded":"2024-02-23T00:55:01.110672+0000","last_fullsized":"2024-02-23T00:55:01.110672+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:21.536419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:21.536419+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":105,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.111666+0000","last_change":"2024-02-23T00:53:53.411932+0000","last_active":"2024-02-23T00:55:01.111666+0000","last_peered":"2024-02-23T00:55:01.111666+0000","last_clean":"2024-02-23T00:55:01.111666+0000","last_became_active":"2024-02-23T00:53:53.411604+0000","last_became_peered":"2024-02-23T00:53:53.411604+0000","last_unstale":"2024-02-23T00:55:01.111666+0000","last_undegraded":"2024-02-23T00:55:01.111666+0000","last_fullsized":"2024-02-23T00:55:01.111666+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:51:45.694974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:45.913526+0000","last_clean_scrub_stamp":"2024-02-23T00:51:45.694974+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.110717+0000","last_change":"2024-02-23T00:54:56.488806+0000","last_active":"2024-02-23T00:55:01.110717+0000","last_peered":"2024-02-23T00:55:01.110717+0000","last_clean":"2024-02-23T00:55:01.110717+0000","last_became_active":"2024-02-23T00:54:56.487615+0000","last_became_peered":"2024-02-23T00:54:56.487615+0000","last_unstale":"2024-02-23T00:55:01.110717+0000","last_undegraded":"2024-02-23T00:55:01.110717+0000","last_fullsized":"2024-02-23T00:55:01.110717+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:39.424888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:54:39.424888+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"457'22","reported_seq":588,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.926275+0000","last_change":"2024-02-23T00:53:54.319331+0000","last_active":"2024-02-23T00:54:59.926275+0000","last_peered":"2024-02-23T00:54:59.926275+0000","last_clean":"2024-02-23T00:54:59.926275+0000","last_became_active":"2024-02-23T00:53:53.416008+0000","last_became_peered":"2024-02-23T00:53:53.416008+0000","last_unstale":"2024-02-23T00:54:59.926275+0000","last_undegraded":"2024-02-23T00:54:59.926275+0000","last_fullsized":"2024-02-23T00:54:59.926275+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'22","last_scrub_stamp":"2024-02-23T00:53:32.317744+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-02-23T00:48:07.941101+0000","last_clean_scrub_stamp":"2024-02-23T00:53:32.317744+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":726,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.924192+0000","last_change":"2024-02-23T00:54:09.497814+0000","last_active":"2024-02-23T00:54:59.924192+0000","last_peered":"2024-02-23T00:54:59.924192+0000","last_clean":"2024-02-23T00:54:59.924192+0000","last_became_active":"2024-02-23T00:39:14.783164+0000","last_became_peered":"2024-02-23T00:39:14.783164+0000","last_unstale":"2024-02-23T00:54:59.924192+0000","last_undegraded":"2024-02-23T00:54:59.924192+0000","last_fullsized":"2024-02-23T00:54:59.924192+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:09.497701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:54:09.497701+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":725,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.110998+0000","last_change":"2024-02-23T00:54:00.288206+0000","last_active":"2024-02-23T00:55:01.110998+0000","last_peered":"2024-02-23T00:55:01.110998+0000","last_clean":"2024-02-23T00:55:01.110998+0000","last_became_active":"2024-02-23T00:39:18.800297+0000","last_became_peered":"2024-02-23T00:39:18.800297+0000","last_unstale":"2024-02-23T00:55:01.110998+0000","last_undegraded":"2024-02-23T00:55:01.110998+0000","last_fullsized":"2024-02-23T00:55:01.110998+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:00.288132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:54:00.288132+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1","version":"110'7","reported_seq":103,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.924125+0000","last_change":"2024-02-23T00:53:34.322710+0000","last_active":"2024-02-23T00:54:59.924125+0000","last_peered":"2024-02-23T00:54:59.924125+0000","last_clean":"2024-02-23T00:54:59.924125+0000","last_became_active":"2024-02-23T00:53:34.322370+0000","last_became_peered":"2024-02-23T00:53:34.322370+0000","last_unstale":"2024-02-23T00:54:59.924125+0000","last_undegraded":"2024-02-23T00:54:59.924125+0000","last_fullsized":"2024-02-23T00:54:59.924125+0000","mapping_epoch":676,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":677,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'7","last_scrub_stamp":"2024-02-23T00:53:04.958282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:53:04.958282+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"457'32","reported_seq":206,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.111765+0000","last_change":"2024-02-23T00:53:34.316882+0000","last_active":"2024-02-23T00:55:01.111765+0000","last_peered":"2024-02-23T00:55:01.111765+0000","last_clean":"2024-02-23T00:55:01.111765+0000","last_became_active":"2024-02-23T00:53:34.316552+0000","last_became_peered":"2024-02-23T00:53:34.316552+0000","last_unstale":"2024-02-23T00:55:01.111765+0000","last_undegraded":"2024-02-23T00:55:01.111765+0000","last_fullsized":"2024-02-23T00:55:01.111765+0000","mapping_epoch":676,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":677,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-23T00:53:30.199998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:53:30.199998+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":32,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":729,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.112190+0000","last_change":"2024-02-23T00:53:35.190352+0000","last_active":"2024-02-23T00:55:01.112190+0000","last_peered":"2024-02-23T00:55:01.112190+0000","last_clean":"2024-02-23T00:55:01.112190+0000","last_became_active":"2024-02-23T00:39:14.785590+0000","last_became_peered":"2024-02-23T00:39:14.785590+0000","last_unstale":"2024-02-23T00:55:01.112190+0000","last_undegraded":"2024-02-23T00:55:01.112190+0000","last_fullsized":"2024-02-23T00:55:01.112190+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:35.190288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_clean_scrub_stamp":"2024-02-23T00:53:35.190288+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"749'11","reported_seq":949,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.113156+0000","last_change":"2024-02-23T00:54:56.627427+0000","last_active":"2024-02-23T00:55:01.113156+0000","last_peered":"2024-02-23T00:55:01.113156+0000","last_clean":"2024-02-23T00:55:01.113156+0000","last_became_active":"2024-02-23T00:54:56.483713+0000","last_became_peered":"2024-02-23T00:54:56.483713+0000","last_unstale":"2024-02-23T00:55:01.113156+0000","last_undegraded":"2024-02-23T00:55:01.113156+0000","last_fullsized":"2024-02-23T00:55:01.113156+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"656'10","last_scrub_stamp":"2024-02-23T00:54:02.269193+0000","last_deep_scrub":"336'9","last_deep_scrub_stamp":"2024-02-23T00:46:49.160991+0000","last_clean_scrub_stamp":"2024-02-23T00:54:02.269193+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":682,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.924512+0000","last_change":"2024-02-23T00:54:56.478174+0000","last_active":"2024-02-23T00:54:59.924512+0000","last_peered":"2024-02-23T00:54:59.924512+0000","last_clean":"2024-02-23T00:54:59.924512+0000","last_became_active":"2024-02-23T00:54:56.477759+0000","last_became_peered":"2024-02-23T00:54:56.477759+0000","last_unstale":"2024-02-23T00:54:59.924512+0000","last_undegraded":"2024-02-23T00:54:59.924512+0000","last_fullsized":"2024-02-23T00:54:59.924512+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:00.410226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:53:00.410226+0000","last_clean_scrub_stamp":"2024-02-23T00:53:00.410226+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"457'21","reported_seq":762,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.112816+0000","last_change":"2024-02-23T00:52:27.456856+0000","last_active":"2024-02-23T00:55:01.112816+0000","last_peered":"2024-02-23T00:55:01.112816+0000","last_clean":"2024-02-23T00:55:01.112816+0000","last_became_active":"2024-02-23T00:39:16.858953+0000","last_became_peered":"2024-02-23T00:39:16.858953+0000","last_unstale":"2024-02-23T00:55:01.112816+0000","last_undegraded":"2024-02-23T00:55:01.112816+0000","last_fullsized":"2024-02-23T00:55:01.112816+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-23T00:52:27.456787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:52:27.456787+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":15,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.112214+0000","last_change":"2024-02-23T00:54:56.472692+0000","last_active":"2024-02-23T00:55:01.112214+0000","last_peered":"2024-02-23T00:55:01.112214+0000","last_clean":"2024-02-23T00:55:01.112214+0000","last_became_active":"2024-02-23T00:54:56.472343+0000","last_became_peered":"2024-02-23T00:54:56.472343+0000","last_unstale":"2024-02-23T00:55:01.112214+0000","last_undegraded":"2024-02-23T00:55:01.112214+0000","last_fullsized":"2024-02-23T00:55:01.112214+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:29.500109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:53:29.500109+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"694'11","reported_seq":155,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.925696+0000","last_change":"2024-02-23T00:54:49.045375+0000","last_active":"2024-02-23T00:54:59.925696+0000","last_peered":"2024-02-23T00:54:59.925696+0000","last_clean":"2024-02-23T00:54:59.925696+0000","last_became_active":"2024-02-23T00:53:53.415965+0000","last_became_peered":"2024-02-23T00:53:53.415965+0000","last_unstale":"2024-02-23T00:54:59.925696+0000","last_undegraded":"2024-02-23T00:54:59.925696+0000","last_fullsized":"2024-02-23T00:54:59.925696+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"694'11","last_scrub_stamp":"2024-02-23T00:54:49.045257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:54:49.045257+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":656,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.111341+0000","last_change":"2024-02-23T00:54:03.413676+0000","last_active":"2024-02-23T00:55:01.111341+0000","last_peered":"2024-02-23T00:55:01.111341+0000","last_clean":"2024-02-23T00:55:01.111341+0000","last_became_active":"2024-02-23T00:42:57.652488+0000","last_became_peered":"2024-02-23T00:42:57.652488+0000","last_unstale":"2024-02-23T00:55:01.111341+0000","last_undegraded":"2024-02-23T00:55:01.111341+0000","last_fullsized":"2024-02-23T00:55:01.111341+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:03.413609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:51.254903+0000","last_clean_scrub_stamp":"2024-02-23T00:54:03.413609+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"457'15","reported_seq":794,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.111786+0000","last_change":"2024-02-23T00:54:56.948726+0000","last_active":"2024-02-23T00:55:01.111786+0000","last_peered":"2024-02-23T00:55:01.111786+0000","last_clean":"2024-02-23T00:55:01.111786+0000","last_became_active":"2024-02-23T00:54:56.491368+0000","last_became_peered":"2024-02-23T00:54:56.491368+0000","last_unstale":"2024-02-23T00:55:01.111786+0000","last_undegraded":"2024-02-23T00:55:01.111786+0000","last_fullsized":"2024-02-23T00:55:01.111786+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'15","last_scrub_stamp":"2024-02-23T00:53:06.466487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:53:06.466487+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":745,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.925638+0000","last_change":"2024-02-23T00:54:55.602793+0000","last_active":"2024-02-23T00:54:59.925638+0000","last_peered":"2024-02-23T00:54:59.925638+0000","last_clean":"2024-02-23T00:54:59.925638+0000","last_became_active":"2024-02-23T00:53:45.293336+0000","last_became_peered":"2024-02-23T00:53:45.293336+0000","last_unstale":"2024-02-23T00:54:59.925638+0000","last_undegraded":"2024-02-23T00:54:59.925638+0000","last_fullsized":"2024-02-23T00:54:59.925638+0000","mapping_epoch":685,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":686,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.602734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.602734+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":15,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.112130+0000","last_change":"2024-02-23T00:54:56.484896+0000","last_active":"2024-02-23T00:55:01.112130+0000","last_peered":"2024-02-23T00:55:01.112130+0000","last_clean":"2024-02-23T00:55:01.112130+0000","last_became_active":"2024-02-23T00:54:56.484549+0000","last_became_peered":"2024-02-23T00:54:56.484549+0000","last_unstale":"2024-02-23T00:55:01.112130+0000","last_undegraded":"2024-02-23T00:55:01.112130+0000","last_fullsized":"2024-02-23T00:55:01.112130+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:27.761922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_clean_scrub_stamp":"2024-02-23T00:52:27.761922+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.111747+0000","last_change":"2024-02-23T00:54:56.488786+0000","last_active":"2024-02-23T00:55:01.111747+0000","last_peered":"2024-02-23T00:55:01.111747+0000","last_clean":"2024-02-23T00:55:01.111747+0000","last_became_active":"2024-02-23T00:54:56.487799+0000","last_became_peered":"2024-02-23T00:54:56.487799+0000","last_unstale":"2024-02-23T00:55:01.111747+0000","last_undegraded":"2024-02-23T00:55:01.111747+0000","last_fullsized":"2024-02-23T00:55:01.111747+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:53.379042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:54:53.379042+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"457'19","reported_seq":759,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:00.306779+0000","last_change":"2024-02-23T00:54:57.393459+0000","last_active":"2024-02-23T00:55:00.306779+0000","last_peered":"2024-02-23T00:55:00.306779+0000","last_clean":"2024-02-23T00:55:00.306779+0000","last_became_active":"2024-02-23T00:39:16.783896+0000","last_became_peered":"2024-02-23T00:39:16.783896+0000","last_unstale":"2024-02-23T00:55:00.306779+0000","last_undegraded":"2024-02-23T00:55:00.306779+0000","last_fullsized":"2024-02-23T00:55:00.306779+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'19","last_scrub_stamp":"2024-02-23T00:54:57.393344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:54:57.393344+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"202'2","reported_seq":753,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.112474+0000","last_change":"2024-02-23T00:54:56.958362+0000","last_active":"2024-02-23T00:55:01.112474+0000","last_peered":"2024-02-23T00:55:01.112474+0000","last_clean":"2024-02-23T00:55:01.112474+0000","last_became_active":"2024-02-23T00:54:56.497619+0000","last_became_peered":"2024-02-23T00:54:56.497619+0000","last_unstale":"2024-02-23T00:55:01.112474+0000","last_undegraded":"2024-02-23T00:55:01.112474+0000","last_fullsized":"2024-02-23T00:55:01.112474+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"202'2","last_scrub_stamp":"2024-02-23T00:54:52.359114+0000","last_deep_scrub":"202'2","last_deep_scrub_stamp":"2024-02-23T00:44:34.720640+0000","last_clean_scrub_stamp":"2024-02-23T00:54:52.359114+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1746,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":728,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.111874+0000","last_change":"2024-02-23T00:53:53.417039+0000","last_active":"2024-02-23T00:55:01.111874+0000","last_peered":"2024-02-23T00:55:01.111874+0000","last_clean":"2024-02-23T00:55:01.111874+0000","last_became_active":"2024-02-23T00:53:53.416685+0000","last_became_peered":"2024-02-23T00:53:53.416685+0000","last_unstale":"2024-02-23T00:55:01.111874+0000","last_undegraded":"2024-02-23T00:55:01.111874+0000","last_fullsized":"2024-02-23T00:55:01.111874+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:41.472347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:53:41.472347+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":683,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.112382+0000","last_change":"2024-02-23T00:54:56.490662+0000","last_active":"2024-02-23T00:55:01.112382+0000","last_peered":"2024-02-23T00:55:01.112382+0000","last_clean":"2024-02-23T00:55:01.112382+0000","last_became_active":"2024-02-23T00:54:56.490311+0000","last_became_peered":"2024-02-23T00:54:56.490311+0000","last_unstale":"2024-02-23T00:55:01.112382+0000","last_undegraded":"2024-02-23T00:55:01.112382+0000","last_fullsized":"2024-02-23T00:55:01.112382+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:58.516233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:46.462015+0000","last_clean_scrub_stamp":"2024-02-23T00:52:58.516233+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"457'8","reported_seq":551,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.110785+0000","last_change":"2024-02-23T00:53:53.418225+0000","last_active":"2024-02-23T00:55:01.110785+0000","last_peered":"2024-02-23T00:55:01.110785+0000","last_clean":"2024-02-23T00:55:01.110785+0000","last_became_active":"2024-02-23T00:53:53.403080+0000","last_became_peered":"2024-02-23T00:53:53.403080+0000","last_unstale":"2024-02-23T00:55:01.110785+0000","last_undegraded":"2024-02-23T00:55:01.110785+0000","last_fullsized":"2024-02-23T00:55:01.110785+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'8","last_scrub_stamp":"2024-02-23T00:52:49.890254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:52:49.890254+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":741,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.110920+0000","last_change":"2024-02-23T00:53:40.166252+0000","last_active":"2024-02-23T00:55:01.110920+0000","last_peered":"2024-02-23T00:55:01.110920+0000","last_clean":"2024-02-23T00:55:01.110920+0000","last_became_active":"2024-02-23T00:47:28.224477+0000","last_became_peered":"2024-02-23T00:47:28.224477+0000","last_unstale":"2024-02-23T00:55:01.110920+0000","last_undegraded":"2024-02-23T00:55:01.110920+0000","last_fullsized":"2024-02-23T00:55:01.110920+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:40.166169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:53:40.166169+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":736,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.924033+0000","last_change":"2024-02-23T00:53:59.499809+0000","last_active":"2024-02-23T00:54:59.924033+0000","last_peered":"2024-02-23T00:54:59.924033+0000","last_clean":"2024-02-23T00:54:59.924033+0000","last_became_active":"2024-02-23T00:47:44.230295+0000","last_became_peered":"2024-02-23T00:47:44.230295+0000","last_unstale":"2024-02-23T00:54:59.924033+0000","last_undegraded":"2024-02-23T00:54:59.924033+0000","last_fullsized":"2024-02-23T00:54:59.924033+0000","mapping_epoch":391,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":392,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:59.499666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:53:59.499666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":664,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.926082+0000","last_change":"2024-02-23T00:53:01.388275+0000","last_active":"2024-02-23T00:54:59.926082+0000","last_peered":"2024-02-23T00:54:59.926082+0000","last_clean":"2024-02-23T00:54:59.926082+0000","last_became_active":"2024-02-23T00:42:58.741365+0000","last_became_peered":"2024-02-23T00:42:58.741365+0000","last_unstale":"2024-02-23T00:54:59.926082+0000","last_undegraded":"2024-02-23T00:54:59.926082+0000","last_fullsized":"2024-02-23T00:54:59.926082+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:01.388176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:53:01.388176+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"457'25","reported_seq":553,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.924599+0000","last_change":"2024-02-23T00:53:34.322025+0000","last_active":"2024-02-23T00:54:59.924599+0000","last_peered":"2024-02-23T00:54:59.924599+0000","last_clean":"2024-02-23T00:54:59.924599+0000","last_became_active":"2024-02-23T00:53:34.321651+0000","last_became_peered":"2024-02-23T00:53:34.321651+0000","last_unstale":"2024-02-23T00:54:59.924599+0000","last_undegraded":"2024-02-23T00:54:59.924599+0000","last_fullsized":"2024-02-23T00:54:59.924599+0000","mapping_epoch":676,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":677,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-23T00:53:29.192169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:53:29.192169+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":43,"num_read_kb":35,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":740,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.926345+0000","last_change":"2024-02-23T00:54:53.583797+0000","last_active":"2024-02-23T00:54:59.926345+0000","last_peered":"2024-02-23T00:54:59.926345+0000","last_clean":"2024-02-23T00:54:59.926345+0000","last_became_active":"2024-02-23T00:47:28.240162+0000","last_became_peered":"2024-02-23T00:47:28.240162+0000","last_unstale":"2024-02-23T00:54:59.926345+0000","last_undegraded":"2024-02-23T00:54:59.926345+0000","last_fullsized":"2024-02-23T00:54:59.926345+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:53.583713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:53.583713+0000","last_clean_scrub_stamp":"2024-02-23T00:54:53.583713+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":79,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.923853+0000","last_change":"2024-02-23T00:54:12.297949+0000","last_active":"2024-02-23T00:54:59.923853+0000","last_peered":"2024-02-23T00:54:59.923853+0000","last_clean":"2024-02-23T00:54:59.923853+0000","last_became_active":"2024-02-23T00:53:45.290784+0000","last_became_peered":"2024-02-23T00:53:45.290784+0000","last_unstale":"2024-02-23T00:54:59.923853+0000","last_undegraded":"2024-02-23T00:54:59.923853+0000","last_fullsized":"2024-02-23T00:54:59.923853+0000","mapping_epoch":685,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":686,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:12.297867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:54:12.297867+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":416,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.111642+0000","last_change":"2024-02-23T00:53:53.412837+0000","last_active":"2024-02-23T00:55:01.111642+0000","last_peered":"2024-02-23T00:55:01.111642+0000","last_clean":"2024-02-23T00:55:01.111642+0000","last_became_active":"2024-02-23T00:53:53.412497+0000","last_became_peered":"2024-02-23T00:53:53.412497+0000","last_unstale":"2024-02-23T00:55:01.111642+0000","last_undegraded":"2024-02-23T00:55:01.111642+0000","last_fullsized":"2024-02-23T00:55:01.111642+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:48.449279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:52:48.449279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"457'26","reported_seq":29,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.113028+0000","last_change":"2024-02-23T00:54:56.506511+0000","last_active":"2024-02-23T00:55:01.113028+0000","last_peered":"2024-02-23T00:55:01.113028+0000","last_clean":"2024-02-23T00:55:01.113028+0000","last_became_active":"2024-02-23T00:54:56.479450+0000","last_became_peered":"2024-02-23T00:54:56.479450+0000","last_unstale":"2024-02-23T00:55:01.113028+0000","last_undegraded":"2024-02-23T00:55:01.113028+0000","last_fullsized":"2024-02-23T00:55:01.113028+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-23T00:52:53.810591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:52:53.810591+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":35,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":736,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.110583+0000","last_change":"2024-02-23T00:53:34.208099+0000","last_active":"2024-02-23T00:55:01.110583+0000","last_peered":"2024-02-23T00:55:01.110583+0000","last_clean":"2024-02-23T00:55:01.110583+0000","last_became_active":"2024-02-23T00:39:14.785363+0000","last_became_peered":"2024-02-23T00:39:14.785363+0000","last_unstale":"2024-02-23T00:55:01.110583+0000","last_undegraded":"2024-02-23T00:55:01.110583+0000","last_fullsized":"2024-02-23T00:55:01.110583+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"80'2","last_scrub_stamp":"2024-02-23T00:53:34.208030+0000","last_deep_scrub":"80'2","last_deep_scrub_stamp":"2024-02-23T00:40:15.942725+0000","last_clean_scrub_stamp":"2024-02-23T00:53:34.208030+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"93'9","reported_seq":928,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.926469+0000","last_change":"2024-02-23T00:54:51.588662+0000","last_active":"2024-02-23T00:54:59.926469+0000","last_peered":"2024-02-23T00:54:59.926469+0000","last_clean":"2024-02-23T00:54:59.926469+0000","last_became_active":"2024-02-23T00:39:18.796680+0000","last_became_peered":"2024-02-23T00:39:18.796680+0000","last_unstale":"2024-02-23T00:54:59.926469+0000","last_undegraded":"2024-02-23T00:54:59.926469+0000","last_fullsized":"2024-02-23T00:54:59.926469+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-02-23T00:54:51.588572+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-02-23T00:43:50.393713+0000","last_clean_scrub_stamp":"2024-02-23T00:54:51.588572+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.110579+0000","last_change":"2024-02-23T00:54:56.488593+0000","last_active":"2024-02-23T00:55:01.110579+0000","last_peered":"2024-02-23T00:55:01.110579+0000","last_clean":"2024-02-23T00:55:01.110579+0000","last_became_active":"2024-02-23T00:54:56.486972+0000","last_became_peered":"2024-02-23T00:54:56.486972+0000","last_unstale":"2024-02-23T00:55:01.110579+0000","last_undegraded":"2024-02-23T00:55:01.110579+0000","last_fullsized":"2024-02-23T00:55:01.110579+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:01.802405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:53:01.802405+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"457'31","reported_seq":809,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.926450+0000","last_change":"2024-02-23T00:54:45.488824+0000","last_active":"2024-02-23T00:54:59.926450+0000","last_peered":"2024-02-23T00:54:59.926450+0000","last_clean":"2024-02-23T00:54:59.926450+0000","last_became_active":"2024-02-23T00:54:45.482795+0000","last_became_peered":"2024-02-23T00:54:45.482795+0000","last_unstale":"2024-02-23T00:54:59.926450+0000","last_undegraded":"2024-02-23T00:54:59.926450+0000","last_fullsized":"2024-02-23T00:54:59.926450+0000","mapping_epoch":742,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":743,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:54:11.502514+0000","last_deep_scrub":"457'31","last_deep_scrub_stamp":"2024-02-23T00:52:27.357369+0000","last_clean_scrub_stamp":"2024-02-23T00:54:11.502514+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,7],"acting":[6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":750,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.923963+0000","last_change":"2024-02-23T00:54:56.483580+0000","last_active":"2024-02-23T00:54:59.923963+0000","last_peered":"2024-02-23T00:54:59.923963+0000","last_clean":"2024-02-23T00:54:59.923963+0000","last_became_active":"2024-02-23T00:54:56.483235+0000","last_became_peered":"2024-02-23T00:54:56.483235+0000","last_unstale":"2024-02-23T00:54:59.923963+0000","last_undegraded":"2024-02-23T00:54:59.923963+0000","last_fullsized":"2024-02-23T00:54:59.923963+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:12.460337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:54:12.460337+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":15,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.112979+0000","last_change":"2024-02-23T00:54:56.487482+0000","last_active":"2024-02-23T00:55:01.112979+0000","last_peered":"2024-02-23T00:55:01.112979+0000","last_clean":"2024-02-23T00:55:01.112979+0000","last_became_active":"2024-02-23T00:54:56.486101+0000","last_became_peered":"2024-02-23T00:54:56.486101+0000","last_unstale":"2024-02-23T00:55:01.112979+0000","last_undegraded":"2024-02-23T00:55:01.112979+0000","last_fullsized":"2024-02-23T00:55:01.112979+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:00.845179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:53.189881+0000","last_clean_scrub_stamp":"2024-02-23T00:53:00.845179+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":87,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.111503+0000","last_change":"2024-02-23T00:53:34.321705+0000","last_active":"2024-02-23T00:55:01.111503+0000","last_peered":"2024-02-23T00:55:01.111503+0000","last_clean":"2024-02-23T00:55:01.111503+0000","last_became_active":"2024-02-23T00:53:34.320314+0000","last_became_peered":"2024-02-23T00:53:34.320314+0000","last_unstale":"2024-02-23T00:55:01.111503+0000","last_undegraded":"2024-02-23T00:55:01.111503+0000","last_fullsized":"2024-02-23T00:55:01.111503+0000","mapping_epoch":676,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":677,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:57.917367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:52:57.917367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"457'27","reported_seq":560,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.111678+0000","last_change":"2024-02-23T00:54:57.267319+0000","last_active":"2024-02-23T00:55:01.111678+0000","last_peered":"2024-02-23T00:55:01.111678+0000","last_clean":"2024-02-23T00:55:01.111678+0000","last_became_active":"2024-02-23T00:54:56.493197+0000","last_became_peered":"2024-02-23T00:54:56.493197+0000","last_unstale":"2024-02-23T00:55:01.111678+0000","last_undegraded":"2024-02-23T00:55:01.111678+0000","last_fullsized":"2024-02-23T00:55:01.111678+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-23T00:52:55.480770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:52:55.480770+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":731,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.924762+0000","last_change":"2024-02-23T00:54:50.554704+0000","last_active":"2024-02-23T00:54:59.924762+0000","last_peered":"2024-02-23T00:54:59.924762+0000","last_clean":"2024-02-23T00:54:59.924762+0000","last_became_active":"2024-02-23T00:39:14.782494+0000","last_became_peered":"2024-02-23T00:39:14.782494+0000","last_unstale":"2024-02-23T00:54:59.924762+0000","last_undegraded":"2024-02-23T00:54:59.924762+0000","last_fullsized":"2024-02-23T00:54:59.924762+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:50.554622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:54:50.554622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":728,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.112713+0000","last_change":"2024-02-23T00:54:56.493951+0000","last_active":"2024-02-23T00:55:01.112713+0000","last_peered":"2024-02-23T00:55:01.112713+0000","last_clean":"2024-02-23T00:55:01.112713+0000","last_became_active":"2024-02-23T00:54:56.493367+0000","last_became_peered":"2024-02-23T00:54:56.493367+0000","last_unstale":"2024-02-23T00:55:01.112713+0000","last_undegraded":"2024-02-23T00:55:01.112713+0000","last_fullsized":"2024-02-23T00:55:01.112713+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:10.563137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.641689+0000","last_clean_scrub_stamp":"2024-02-23T00:52:10.563137+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":107,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.924853+0000","last_change":"2024-02-23T00:53:53.424895+0000","last_active":"2024-02-23T00:54:59.924853+0000","last_peered":"2024-02-23T00:54:59.924853+0000","last_clean":"2024-02-23T00:54:59.924853+0000","last_became_active":"2024-02-23T00:53:53.419653+0000","last_became_peered":"2024-02-23T00:53:53.419653+0000","last_unstale":"2024-02-23T00:54:59.924853+0000","last_undegraded":"2024-02-23T00:54:59.924853+0000","last_fullsized":"2024-02-23T00:54:59.924853+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:28.443176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:53:28.443176+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"616'30","reported_seq":771,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.112634+0000","last_change":"2024-02-23T00:53:53.426870+0000","last_active":"2024-02-23T00:55:01.112634+0000","last_peered":"2024-02-23T00:55:01.112634+0000","last_clean":"2024-02-23T00:55:01.112634+0000","last_became_active":"2024-02-23T00:53:53.405043+0000","last_became_peered":"2024-02-23T00:53:53.405043+0000","last_unstale":"2024-02-23T00:55:01.112634+0000","last_undegraded":"2024-02-23T00:55:01.112634+0000","last_fullsized":"2024-02-23T00:55:01.112634+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"616'30","last_scrub_stamp":"2024-02-23T00:52:25.764389+0000","last_deep_scrub":"616'30","last_deep_scrub_stamp":"2024-02-23T00:52:25.764389+0000","last_clean_scrub_stamp":"2024-02-23T00:52:25.764389+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":46,"num_read_kb":36,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":104,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.925006+0000","last_change":"2024-02-23T00:53:53.424319+0000","last_active":"2024-02-23T00:54:59.925006+0000","last_peered":"2024-02-23T00:54:59.925006+0000","last_clean":"2024-02-23T00:54:59.925006+0000","last_became_active":"2024-02-23T00:53:53.423915+0000","last_became_peered":"2024-02-23T00:53:53.423915+0000","last_unstale":"2024-02-23T00:54:59.925006+0000","last_undegraded":"2024-02-23T00:54:59.925006+0000","last_fullsized":"2024-02-23T00:54:59.925006+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:20.774550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:20.774550+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":76,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.925237+0000","last_change":"2024-02-23T00:53:58.483987+0000","last_active":"2024-02-23T00:54:59.925237+0000","last_peered":"2024-02-23T00:54:59.925237+0000","last_clean":"2024-02-23T00:54:59.925237+0000","last_became_active":"2024-02-23T00:53:45.291586+0000","last_became_peered":"2024-02-23T00:53:45.291586+0000","last_unstale":"2024-02-23T00:54:59.925237+0000","last_undegraded":"2024-02-23T00:54:59.925237+0000","last_fullsized":"2024-02-23T00:54:59.925237+0000","mapping_epoch":685,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":686,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:58.483860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_clean_scrub_stamp":"2024-02-23T00:53:58.483860+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":674,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.112917+0000","last_change":"2024-02-23T00:53:30.271476+0000","last_active":"2024-02-23T00:55:01.112917+0000","last_peered":"2024-02-23T00:55:01.112917+0000","last_clean":"2024-02-23T00:55:01.112917+0000","last_became_active":"2024-02-23T00:47:28.203555+0000","last_became_peered":"2024-02-23T00:47:28.203555+0000","last_unstale":"2024-02-23T00:55:01.112917+0000","last_undegraded":"2024-02-23T00:55:01.112917+0000","last_fullsized":"2024-02-23T00:55:01.112917+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:30.271410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:53:30.271410+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"457'33","reported_seq":837,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:00.306153+0000","last_change":"2024-02-23T00:54:56.523954+0000","last_active":"2024-02-23T00:55:00.306153+0000","last_peered":"2024-02-23T00:55:00.306153+0000","last_clean":"2024-02-23T00:55:00.306153+0000","last_became_active":"2024-02-23T00:54:56.483435+0000","last_became_peered":"2024-02-23T00:54:56.483435+0000","last_unstale":"2024-02-23T00:55:00.306153+0000","last_undegraded":"2024-02-23T00:55:00.306153+0000","last_fullsized":"2024-02-23T00:55:00.306153+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'33","last_scrub_stamp":"2024-02-23T00:54:48.450987+0000","last_deep_scrub":"95'27","last_deep_scrub_stamp":"2024-02-23T00:41:55.919895+0000","last_clean_scrub_stamp":"2024-02-23T00:54:48.450987+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":735,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.925234+0000","last_change":"2024-02-23T00:53:05.309846+0000","last_active":"2024-02-23T00:54:59.925234+0000","last_peered":"2024-02-23T00:54:59.925234+0000","last_clean":"2024-02-23T00:54:59.925234+0000","last_became_active":"2024-02-23T00:47:28.208293+0000","last_became_peered":"2024-02-23T00:47:28.208293+0000","last_unstale":"2024-02-23T00:54:59.925234+0000","last_undegraded":"2024-02-23T00:54:59.925234+0000","last_fullsized":"2024-02-23T00:54:59.925234+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:05.309734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:53:05.309734+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":740,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.112828+0000","last_change":"2024-02-23T00:54:57.210285+0000","last_active":"2024-02-23T00:55:01.112828+0000","last_peered":"2024-02-23T00:55:01.112828+0000","last_clean":"2024-02-23T00:55:01.112828+0000","last_became_active":"2024-02-23T00:54:56.489935+0000","last_became_peered":"2024-02-23T00:54:56.489935+0000","last_unstale":"2024-02-23T00:55:01.112828+0000","last_undegraded":"2024-02-23T00:55:01.112828+0000","last_fullsized":"2024-02-23T00:55:01.112828+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:11.328122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:52:11.328122+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":683,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.111230+0000","last_change":"2024-02-23T00:54:56.482699+0000","last_active":"2024-02-23T00:55:01.111230+0000","last_peered":"2024-02-23T00:55:01.111230+0000","last_clean":"2024-02-23T00:55:01.111230+0000","last_became_active":"2024-02-23T00:54:56.482357+0000","last_became_peered":"2024-02-23T00:54:56.482357+0000","last_unstale":"2024-02-23T00:55:01.111230+0000","last_undegraded":"2024-02-23T00:55:01.111230+0000","last_fullsized":"2024-02-23T00:55:01.111230+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:50.469857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:52:50.469857+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"457'17","reported_seq":762,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:00.306920+0000","last_change":"2024-02-23T00:54:11.404685+0000","last_active":"2024-02-23T00:55:00.306920+0000","last_peered":"2024-02-23T00:55:00.306920+0000","last_clean":"2024-02-23T00:55:00.306920+0000","last_became_active":"2024-02-23T00:39:16.782762+0000","last_became_peered":"2024-02-23T00:39:16.782762+0000","last_unstale":"2024-02-23T00:55:00.306920+0000","last_undegraded":"2024-02-23T00:55:00.306920+0000","last_fullsized":"2024-02-23T00:55:00.306920+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'17","last_scrub_stamp":"2024-02-23T00:54:11.404593+0000","last_deep_scrub":"82'15","last_deep_scrub_stamp":"2024-02-23T00:41:50.975683+0000","last_clean_scrub_stamp":"2024-02-23T00:54:11.404593+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":21,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":744,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.925799+0000","last_change":"2024-02-23T00:54:56.476177+0000","last_active":"2024-02-23T00:54:59.925799+0000","last_peered":"2024-02-23T00:54:59.925799+0000","last_clean":"2024-02-23T00:54:59.925799+0000","last_became_active":"2024-02-23T00:54:56.475828+0000","last_became_peered":"2024-02-23T00:54:56.475828+0000","last_unstale":"2024-02-23T00:54:59.925799+0000","last_undegraded":"2024-02-23T00:54:59.925799+0000","last_fullsized":"2024-02-23T00:54:59.925799+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:52.335390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:52.335390+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":724,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-23T00:54:59.924279+0000","last_change":"2024-02-23T00:52:56.397935+0000","last_active":"2024-02-23T00:54:59.924279+0000","last_peered":"2024-02-23T00:54:59.924279+0000","last_clean":"2024-02-23T00:54:59.924279+0000","last_became_active":"2024-02-23T00:39:18.796952+0000","last_became_peered":"2024-02-23T00:39:18.796952+0000","last_unstale":"2024-02-23T00:54:59.924279+0000","last_undegraded":"2024-02-23T00:54:59.924279+0000","last_fullsized":"2024-02-23T00:54:59.924279+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:56.397846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:06.491793+0000","last_clean_scrub_stamp":"2024-02-23T00:52:56.397846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":687,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:00.306909+0000","last_change":"2024-02-23T00:53:53.417926+0000","last_active":"2024-02-23T00:55:00.306909+0000","last_peered":"2024-02-23T00:55:00.306909+0000","last_clean":"2024-02-23T00:55:00.306909+0000","last_became_active":"2024-02-23T00:53:53.417585+0000","last_became_peered":"2024-02-23T00:53:53.417585+0000","last_unstale":"2024-02-23T00:55:00.306909+0000","last_undegraded":"2024-02-23T00:55:00.306909+0000","last_fullsized":"2024-02-23T00:55:00.306909+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:55.788444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:23.224663+0000","last_clean_scrub_stamp":"2024-02-23T00:52:55.788444+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"457'21","reported_seq":768,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:00.305878+0000","last_change":"2024-02-23T00:53:57.428696+0000","last_active":"2024-02-23T00:55:00.305878+0000","last_peered":"2024-02-23T00:55:00.305878+0000","last_clean":"2024-02-23T00:55:00.305878+0000","last_became_active":"2024-02-23T00:39:16.784606+0000","last_became_peered":"2024-02-23T00:39:16.784606+0000","last_unstale":"2024-02-23T00:55:00.305878+0000","last_undegraded":"2024-02-23T00:55:00.305878+0000","last_fullsized":"2024-02-23T00:55:00.305878+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-23T00:53:57.428626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:53:57.428626+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":15,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.113204+0000","last_change":"2024-02-23T00:54:56.488842+0000","last_active":"2024-02-23T00:55:01.113204+0000","last_peered":"2024-02-23T00:55:01.113204+0000","last_clean":"2024-02-23T00:55:01.113204+0000","last_became_active":"2024-02-23T00:54:56.488091+0000","last_became_peered":"2024-02-23T00:54:56.488091+0000","last_unstale":"2024-02-23T00:55:01.113204+0000","last_undegraded":"2024-02-23T00:55:01.113204+0000","last_fullsized":"2024-02-23T00:55:01.113204+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:17.778217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:52:17.778217+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":712,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.112300+0000","last_change":"2024-02-23T00:53:02.566710+0000","last_active":"2024-02-23T00:55:01.112300+0000","last_peered":"2024-02-23T00:55:01.112300+0000","last_clean":"2024-02-23T00:55:01.112300+0000","last_became_active":"2024-02-23T00:39:18.796312+0000","last_became_peered":"2024-02-23T00:39:18.796312+0000","last_unstale":"2024-02-23T00:55:01.112300+0000","last_undegraded":"2024-02-23T00:55:01.112300+0000","last_fullsized":"2024-02-23T00:55:01.112300+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:02.566644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:47.226331+0000","last_clean_scrub_stamp":"2024-02-23T00:53:02.566644+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":15,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.113146+0000","last_change":"2024-02-23T00:54:56.487744+0000","last_active":"2024-02-23T00:55:01.113146+0000","last_peered":"2024-02-23T00:55:01.113146+0000","last_clean":"2024-02-23T00:55:01.113146+0000","last_became_active":"2024-02-23T00:54:56.486558+0000","last_became_peered":"2024-02-23T00:54:56.486558+0000","last_unstale":"2024-02-23T00:55:01.113146+0000","last_undegraded":"2024-02-23T00:55:01.113146+0000","last_fullsized":"2024-02-23T00:55:01.113146+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:39.498999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:50:28.479512+0000","last_clean_scrub_stamp":"2024-02-23T00:54:39.498999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"457'62","reported_seq":39,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.110784+0000","last_change":"2024-02-23T00:54:56.526235+0000","last_active":"2024-02-23T00:55:01.110784+0000","last_peered":"2024-02-23T00:55:01.110784+0000","last_clean":"2024-02-23T00:55:01.110784+0000","last_became_active":"2024-02-23T00:54:56.487946+0000","last_became_peered":"2024-02-23T00:54:56.487946+0000","last_unstale":"2024-02-23T00:55:01.110784+0000","last_undegraded":"2024-02-23T00:55:01.110784+0000","last_fullsized":"2024-02-23T00:55:01.110784+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'62","last_scrub_stamp":"2024-02-23T00:52:58.949119+0000","last_deep_scrub":"88'50","last_deep_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_clean_scrub_stamp":"2024-02-23T00:52:58.949119+0000","log_size":62,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":92,"num_read_kb":79,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":15,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.113305+0000","last_change":"2024-02-23T00:54:56.473957+0000","last_active":"2024-02-23T00:55:01.113305+0000","last_peered":"2024-02-23T00:55:01.113305+0000","last_clean":"2024-02-23T00:55:01.113305+0000","last_became_active":"2024-02-23T00:54:56.473637+0000","last_became_peered":"2024-02-23T00:54:56.473637+0000","last_unstale":"2024-02-23T00:55:01.113305+0000","last_undegraded":"2024-02-23T00:55:01.113305+0000","last_fullsized":"2024-02-23T00:55:01.113305+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:52:59.831412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_clean_scrub_stamp":"2024-02-23T00:52:59.831412+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":721,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:00.306275+0000","last_change":"2024-02-23T00:53:35.476706+0000","last_active":"2024-02-23T00:55:00.306275+0000","last_peered":"2024-02-23T00:55:00.306275+0000","last_clean":"2024-02-23T00:55:00.306275+0000","last_became_active":"2024-02-23T00:39:18.799723+0000","last_became_peered":"2024-02-23T00:39:18.799723+0000","last_unstale":"2024-02-23T00:55:00.306275+0000","last_undegraded":"2024-02-23T00:55:00.306275+0000","last_fullsized":"2024-02-23T00:55:00.306275+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:53:35.476621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:53:35.476621+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":688,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-23T00:55:01.111511+0000","last_change":"2024-02-23T00:54:56.490816+0000","last_active":"2024-02-23T00:55:01.111511+0000","last_peered":"2024-02-23T00:55:01.111511+0000","last_clean":"2024-02-23T00:55:01.111511+0000","last_became_active":"2024-02-23T00:54:56.490281+0000","last_became_peered":"2024-02-23T00:54:56.490281+0000","last_unstale":"2024-02-23T00:55:01.111511+0000","last_undegraded":"2024-02-23T00:55:01.111511+0000","last_fullsized":"2024-02-23T00:55:01.111511+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:54:55.225786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:44.601086+0000","last_clean_scrub_stamp":"2024-02-23T00:54:55.225786+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0],"acting":[5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":250,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":7},{"poolid":234,"num_pg":8,"stat_sum":{"num_bytes":4000249,"num_objects":37,"num_object_clones":1,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":15,"num_read_kb":4,"num_write":57,"num_write_kb":3959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":4000102,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6438912,"data_stored":6438912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90,"ondisk_log_size":90,"up":24,"acting":24,"num_store_stats":7},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":12,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1133,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":36,"acting":36,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84,"ondisk_log_size":84,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1498,"num_read_kb":1223,"num_write":895,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":6492,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":963,"ondisk_log_size":963,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":934,"num_read_kb":929,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1930,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234177,"num_pgs":83,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":304504,"kb_used_data":3184,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93440648,"statfs":{"total":95995035648,"available":95683223552,"internally_reserved":0,"allocated":3260416,"data_stored":1515356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.50600000000000001,"15min":0.52500000000000002},"min":{"1min":0.38400000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.70399999999999996,"5min":0.83899999999999997,"15min":4.9420000000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.52900000000000003,"15min":0.52400000000000002},"min":{"1min":0.42999999999999999,"5min":0.309,"15min":0.24299999999999999},"max":{"1min":0.70199999999999996,"5min":0.82599999999999996,"15min":4.9729999999999999},"last":0.54600000000000004}]},{"osd":1,"last update":"Fri Feb 23 00:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.50900000000000001,"15min":0.60599999999999998},"min":{"1min":0.38400000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.69199999999999995,"5min":0.79700000000000004,"15min":10.786},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55300000000000005,"15min":0.621},"min":{"1min":0.33400000000000002,"5min":0.32500000000000001,"15min":0.28100000000000003},"max":{"1min":0.68300000000000005,"5min":2.4199999999999999,"15min":11.885999999999999},"last":0.67500000000000004}]},{"osd":2,"last update":"Fri Feb 23 00:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.51700000000000002,"15min":0.54600000000000004},"min":{"1min":0.42799999999999999,"5min":0.29999999999999999,"15min":0.28999999999999998},"max":{"1min":0.68400000000000005,"5min":0.80600000000000005,"15min":5.0789999999999997},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53400000000000003,"15min":0.54800000000000004},"min":{"1min":0.40300000000000002,"5min":0.33200000000000002,"15min":0.316},"max":{"1min":0.71599999999999997,"5min":1.786,"15min":5.1280000000000001},"last":0.57199999999999995}]},{"osd":3,"last update":"Fri Feb 23 00:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54400000000000004,"15min":0.625},"min":{"1min":0.41399999999999998,"5min":0.33100000000000002,"15min":0.29399999999999998},"max":{"1min":0.71299999999999997,"5min":2.536,"15min":15.712999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58999999999999997,"15min":0.622},"min":{"1min":0.443,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.3700000000000001,"5min":3.7000000000000002,"15min":9.4090000000000007},"last":0.68799999999999994}]},{"osd":4,"last update":"Fri Feb 23 00:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42499999999999999,"15min":0.437},"min":{"1min":0.29499999999999998,"5min":0.25800000000000001,"15min":0.23499999999999999},"max":{"1min":0.67400000000000004,"5min":0.753,"15min":2.0350000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45200000000000001,"15min":0.48899999999999999},"min":{"1min":0.34999999999999998,"5min":0.26700000000000002,"15min":0.248},"max":{"1min":0.73499999999999999,"5min":0.85299999999999998,"15min":11.236000000000001},"last":0.505}]},{"osd":5,"last update":"Fri Feb 23 00:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.46100000000000002,"15min":0.48699999999999999},"min":{"1min":0.28799999999999998,"5min":0.27500000000000002,"15min":0.27000000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":5.4000000000000004},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.498,"15min":0.503},"min":{"1min":0.32900000000000001,"5min":0.29199999999999998,"15min":0.26700000000000002},"max":{"1min":0.60399999999999998,"5min":0.79000000000000004,"15min":1.728},"last":0.58499999999999996}]},{"osd":6,"last update":"Fri Feb 23 00:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.503,"15min":0.51100000000000001},"min":{"1min":0.36399999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.67200000000000004,"5min":0.75700000000000001,"15min":1.6339999999999999},"last":0.624},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.50600000000000001,"15min":0.54800000000000004},"min":{"1min":0.316,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73999999999999999,"5min":0.82499999999999996,"15min":13.395},"last":0.56200000000000006}]}]},{"osd":6,"up_from":47,"seq":201863463112,"num_pgs":75,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":300484,"kb_used_data":3260,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93444668,"statfs":{"total":95995035648,"available":95687340032,"internally_reserved":0,"allocated":3338240,"data_stored":1585439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:54:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52300000000000002,"15min":0.67400000000000004},"min":{"1min":0.36199999999999999,"5min":0.22500000000000001,"15min":0.21099999999999999},"max":{"1min":0.81799999999999995,"5min":0.93799999999999994,"15min":30.641999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57399999999999995,"15min":0.70099999999999996},"min":{"1min":0.29899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.268,"5min":4.3609999999999998,"15min":29.292000000000002},"last":0.47199999999999998}]},{"osd":1,"last update":"Fri Feb 23 00:54:09 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.55900000000000005,"15min":0.68999999999999995},"min":{"1min":0.308,"5min":0.27100000000000002,"15min":0.247},"max":{"1min":0.79100000000000004,"5min":4.1980000000000004,"15min":28.186},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.61799999999999999,"15min":0.877},"min":{"1min":0.41299999999999998,"5min":0.34499999999999997,"15min":0.26900000000000002},"max":{"1min":1.2529999999999999,"5min":6.4370000000000003,"15min":33.104999999999997},"last":0.40899999999999997}]},{"osd":2,"last update":"Fri Feb 23 00:54:09 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.60099999999999998,"15min":0.72599999999999998},"min":{"1min":0.376,"5min":0.35299999999999998,"15min":0.27400000000000002},"max":{"1min":1.893,"5min":4.0919999999999996,"15min":19.960000000000001},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52400000000000002,"15min":0.63800000000000001},"min":{"1min":0.35499999999999998,"5min":0.32500000000000001,"15min":0.29899999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":8.5860000000000003},"last":0.42999999999999999}]},{"osd":3,"last update":"Fri Feb 23 00:54:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.63300000000000001},"min":{"1min":0.39400000000000002,"5min":0.30399999999999999,"15min":0.28599999999999998},"max":{"1min":1.236,"5min":4.1340000000000003,"15min":19.562000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.57899999999999996,"15min":0.66900000000000004},"min":{"1min":0.375,"5min":0.35699999999999998,"15min":0.33000000000000002},"max":{"1min":0.75600000000000001,"5min":3.7839999999999998,"15min":14.199},"last":0.441}]},{"osd":4,"last update":"Fri Feb 23 00:54:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.42399999999999999,"15min":0.48299999999999998},"min":{"1min":0.313,"5min":0.23200000000000001,"15min":0.218},"max":{"1min":0.54600000000000004,"5min":0.63800000000000001,"15min":11.109999999999999},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.436,"15min":0.54700000000000004},"min":{"1min":0.29799999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.56699999999999995,"5min":0.751,"15min":16.734000000000002},"last":0.38400000000000001}]},{"osd":5,"last update":"Fri Feb 23 00:54:09 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.45000000000000001,"15min":0.51900000000000002},"min":{"1min":0.311,"5min":0.308,"15min":0.249},"max":{"1min":0.54800000000000004,"5min":0.72399999999999998,"15min":14.154999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.46899999999999997,"15min":0.55800000000000005},"min":{"1min":0.32400000000000001,"5min":0.29699999999999999,"15min":0.26400000000000001},"max":{"1min":0.61099999999999999,"5min":0.73899999999999999,"15min":8.1400000000000006},"last":0.48199999999999998}]},{"osd":7,"last update":"Fri Feb 23 00:54:32 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47999999999999998,"15min":0.52600000000000002},"min":{"1min":0.34399999999999997,"5min":0.27400000000000002,"15min":0.26800000000000002},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":8.548},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.498,"5min":0.51700000000000002,"15min":0.60399999999999998},"min":{"1min":0.33600000000000002,"5min":0.32000000000000001,"15min":0.309},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":15.618},"last":0.499}]}]},{"osd":1,"up_from":13,"seq":55834575074,"num_pgs":90,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":300448,"kb_used_data":3224,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93444704,"statfs":{"total":95995035648,"available":95687376896,"internally_reserved":0,"allocated":3301376,"data_stored":1535425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:53:59 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.48299999999999998,"15min":0.70899999999999996},"min":{"1min":0.24399999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.624,"5min":7.0540000000000003,"15min":78.361000000000004},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.44,"15min":0.46500000000000002},"min":{"1min":0.26300000000000001,"5min":0.222,"15min":0.191},"max":{"1min":0.80600000000000005,"5min":5.3109999999999999,"15min":5.8849999999999998},"last":0.37}]},{"osd":2,"last update":"Fri Feb 23 00:54:20 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.52300000000000002,"15min":0.68600000000000005},"min":{"1min":0.25800000000000001,"5min":0.22700000000000001,"15min":0.188},"max":{"1min":0.60899999999999999,"5min":7.5789999999999997,"15min":48.606000000000002},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.47499999999999998,"15min":0.58899999999999997},"min":{"1min":0.29399999999999998,"5min":0.28000000000000003,"15min":0.23699999999999999},"max":{"1min":0.95199999999999996,"5min":5.2279999999999998,"15min":23.911000000000001},"last":0.34200000000000003}]},{"osd":3,"last update":"Fri Feb 23 00:53:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48199999999999998,"15min":0.60099999999999998},"min":{"1min":0.34300000000000003,"5min":0.25700000000000001,"15min":0.21199999999999999},"max":{"1min":1.01,"5min":5.2800000000000002,"15min":35.244999999999997},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55600000000000005,"15min":0.70099999999999996},"min":{"1min":0.27700000000000002,"5min":0.26900000000000002,"15min":0.248},"max":{"1min":2.2349999999999999,"5min":7.641,"15min":56.853999999999999},"last":0.36199999999999999}]},{"osd":4,"last update":"Fri Feb 23 00:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.66300000000000003,"15min":0.82099999999999995},"min":{"1min":0.35899999999999999,"5min":0.32400000000000001,"15min":0.29299999999999998},"max":{"1min":0.84099999999999997,"5min":7.6829999999999998,"15min":37.646999999999998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.71799999999999997,"15min":0.72799999999999998},"min":{"1min":0.39300000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.90500000000000003,"5min":6.9160000000000004,"15min":9.6859999999999999},"last":0.46400000000000002}]},{"osd":5,"last update":"Fri Feb 23 00:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.68000000000000005,"15min":0.82299999999999995},"min":{"1min":0.38700000000000001,"5min":0.35399999999999998,"15min":0.309},"max":{"1min":0.78400000000000003,"5min":7.742,"15min":48.792000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64100000000000001,"15min":0.69499999999999995},"min":{"1min":0.45600000000000002,"5min":0.39100000000000001,"15min":0.34100000000000003},"max":{"1min":0.998,"5min":0.998,"15min":11.872999999999999},"last":0.53800000000000003}]},{"osd":6,"last update":"Fri Feb 23 00:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.67400000000000004,"15min":0.81599999999999995},"min":{"1min":0.40500000000000003,"5min":0.37,"15min":0.318},"max":{"1min":0.87,"5min":7.7130000000000001,"15min":40.741},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61599999999999999,"15min":0.66400000000000003},"min":{"1min":0.46500000000000002,"5min":0.40999999999999998,"15min":0.34000000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":6.0069999999999997},"last":0.39500000000000002}]},{"osd":7,"last update":"Fri Feb 23 00:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63100000000000001,"15min":0.71399999999999997},"min":{"1min":0.496,"5min":0.41799999999999998,"15min":0.36699999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":20.225000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.749,"15min":0.76500000000000001},"min":{"1min":0.48999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.88300000000000001,"5min":6.8220000000000001,"15min":20.254999999999999},"last":0.52400000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738600,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":302424,"kb_used_data":5200,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93442728,"statfs":{"total":95995035648,"available":95685353472,"internally_reserved":0,"allocated":5324800,"data_stored":3564449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 23 00:54:58 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.371,"15min":0.60699999999999998},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":28.856999999999999},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.38700000000000001,"15min":0.63},"min":{"1min":0.254,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.61099999999999999,"5min":0.77000000000000002,"15min":20.577000000000002},"last":0.67900000000000005}]},{"osd":2,"last update":"Fri Feb 23 00:54:25 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.41999999999999998,"15min":0.60899999999999999},"min":{"1min":0.29399999999999998,"5min":0.222,"15min":0.222},"max":{"1min":0.70899999999999996,"5min":2.3889999999999998,"15min":13.863},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.39100000000000001,"15min":0.54300000000000004},"min":{"1min":0.30499999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":10.455},"last":0.41099999999999998}]},{"osd":3,"last update":"Fri Feb 23 00:54:55 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.38900000000000001,"15min":0.76400000000000001},"min":{"1min":0.28999999999999998,"5min":0.246,"15min":0.22700000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":35.566000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.42899999999999999,"15min":0.748},"min":{"1min":0.312,"5min":0.27300000000000002,"15min":0.26100000000000001},"max":{"1min":0.79900000000000004,"5min":0.80100000000000005,"15min":20.702999999999999},"last":0.46999999999999997}]},{"osd":4,"last update":"Fri Feb 23 00:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.56899999999999995,"15min":0.72799999999999998},"min":{"1min":0.41499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":10.584},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.749},"min":{"1min":0.41299999999999998,"5min":0.35999999999999999,"15min":0.35099999999999998},"max":{"1min":0.83599999999999997,"5min":2.5019999999999998,"15min":13.198},"last":0.626}]},{"osd":5,"last update":"Fri Feb 23 00:54:47 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.61199999999999999,"15min":0.72399999999999998},"min":{"1min":0.40300000000000002,"5min":0.34799999999999998,"15min":0.32200000000000001},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":10.397},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.65400000000000003,"15min":0.78600000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":17.297000000000001},"last":0.56200000000000006}]},{"osd":6,"last update":"Fri Feb 23 00:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.59599999999999997,"15min":0.77300000000000002},"min":{"1min":0.42699999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.84199999999999997,"5min":0.85499999999999998,"15min":17.373999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.63,"15min":0.76300000000000001},"min":{"1min":0.40699999999999997,"5min":0.38,"15min":0.36799999999999999},"max":{"1min":0.91200000000000003,"5min":2.556,"15min":13.973000000000001},"last":0.70099999999999996}]},{"osd":7,"last update":"Fri Feb 23 00:54:32 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.65600000000000003,"15min":0.78500000000000003},"min":{"1min":0.47799999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.8260000000000001,"5min":2.8260000000000001,"15min":17.399999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":1.103,"5min":0.71299999999999997,"15min":0.81000000000000005},"min":{"1min":0.41499999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":6.0229999999999997,"5min":6.0229999999999997,"15min":16.427},"last":0.73799999999999999}]}]},{"osd":2,"up_from":0,"seq":77309411549,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":26,"seq":111669149910,"num_pgs":83,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":306516,"kb_used_data":5196,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93438636,"statfs":{"total":95995035648,"available":95681163264,"internally_reserved":0,"allocated":5320704,"data_stored":3549182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:54:55 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.436,"15min":0.51600000000000001},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.20899999999999999},"max":{"1min":0.51300000000000001,"5min":8.3300000000000001,"15min":12.212999999999999},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.432,"15min":0.59699999999999998},"min":{"1min":0.27200000000000002,"5min":0.23100000000000001,"15min":0.21199999999999999},"max":{"1min":1.0469999999999999,"5min":2.9740000000000002,"15min":11.714},"last":0.375}]},{"osd":1,"last update":"Fri Feb 23 00:54:55 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42499999999999999,"15min":0.61099999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.19900000000000001},"max":{"1min":1.1539999999999999,"5min":3.056,"15min":11.742000000000001},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.45000000000000001,"15min":0.59599999999999997},"min":{"1min":0.28000000000000003,"5min":0.218,"15min":0.19900000000000001},"max":{"1min":0.85599999999999998,"5min":2.7709999999999999,"15min":8.2270000000000003},"last":0.45900000000000002}]},{"osd":2,"last update":"Fri Feb 23 00:54:55 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.49399999999999999,"15min":0.67400000000000004},"min":{"1min":0.29499999999999998,"5min":0.27700000000000002,"15min":0.255},"max":{"1min":0.92300000000000004,"5min":3.242,"15min":13.359999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.503,"15min":0.65100000000000002},"min":{"1min":0.314,"5min":0.27600000000000002,"15min":0.26100000000000001},"max":{"1min":0.71199999999999997,"5min":8.5820000000000007,"15min":21.122},"last":0.34899999999999998}]},{"osd":4,"last update":"Fri Feb 23 00:54:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59799999999999998,"15min":0.751},"min":{"1min":0.39800000000000002,"5min":0.36099999999999999,"15min":0.29899999999999999},"max":{"1min":0.97599999999999998,"5min":3.1419999999999999,"15min":11.644},"last":0.495},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61599999999999999,"15min":0.75600000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.32700000000000001},"max":{"1min":1.2969999999999999,"5min":2.9020000000000001,"15min":12.430999999999999},"last":0.501}]},{"osd":5,"last update":"Fri Feb 23 00:54:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79200000000000004,"5min":10.231999999999999,"15min":10.231999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.63700000000000001,"15min":0.79600000000000004},"min":{"1min":0.39200000000000002,"5min":0.38300000000000001,"15min":0.35499999999999998},"max":{"1min":0.77700000000000002,"5min":3.1829999999999998,"15min":11.648},"last":0.42199999999999999}]},{"osd":6,"last update":"Fri Feb 23 00:54:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64100000000000001,"15min":0.75800000000000001},"min":{"1min":0.39300000000000002,"5min":0.377,"15min":0.35199999999999998},"max":{"1min":1.3400000000000001,"5min":2.9510000000000001,"15min":6.9420000000000002},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.73499999999999999,"15min":0.72299999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.89500000000000002,"5min":10.965999999999999,"15min":10.965999999999999},"last":0.442}]},{"osd":7,"last update":"Fri Feb 23 00:54:38 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.66300000000000003,"15min":0.83199999999999996},"min":{"1min":0.432,"5min":0.40999999999999998,"15min":0.38800000000000001},"max":{"1min":0.90600000000000003,"5min":3.1030000000000002,"15min":11.676},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67600000000000005,"15min":0.81000000000000005},"min":{"1min":0.42299999999999999,"5min":0.41499999999999998,"15min":0.41299999999999998},"max":{"1min":1.375,"5min":2.8580000000000001,"15min":7.9800000000000004},"last":0.54700000000000004}]}]},{"osd":4,"up_from":32,"seq":137438953682,"num_pgs":78,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":302392,"kb_used_data":5168,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93442760,"statfs":{"total":95995035648,"available":95685386240,"internally_reserved":0,"allocated":5292032,"data_stored":3538895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.51600000000000001,"15min":0.51000000000000001},"min":{"1min":0.33100000000000002,"5min":0.28999999999999998,"15min":0.223},"max":{"1min":0.75800000000000001,"5min":1.3819999999999999,"15min":3.5219999999999998},"last":0.505},{"interface":"front","average":{"1min":0.629,"5min":0.56299999999999994,"15min":0.58199999999999996},"min":{"1min":0.32900000000000001,"5min":0.29799999999999999,"15min":0.23999999999999999},"max":{"1min":2.5859999999999999,"5min":2.5859999999999999,"15min":8.6140000000000008},"last":0.60799999999999998}]},{"osd":1,"last update":"Fri Feb 23 00:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53200000000000003,"15min":0.57199999999999995},"min":{"1min":0.36699999999999999,"5min":0.31900000000000001,"15min":0.27300000000000002},"max":{"1min":1.23,"5min":1.23,"15min":9.6110000000000007},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.504,"5min":0.55100000000000005,"15min":0.57399999999999995},"min":{"1min":0.34200000000000003,"5min":0.31900000000000001,"15min":0.255},"max":{"1min":0.71299999999999997,"5min":2.6920000000000002,"15min":6.6950000000000003},"last":0.46400000000000002}]},{"osd":2,"last update":"Fri Feb 23 00:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.54700000000000004,"15min":0.56699999999999995},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.26500000000000001},"max":{"1min":0.78600000000000003,"5min":2.9209999999999998,"15min":3.3069999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.373,"5min":0.33700000000000002,"15min":0.32500000000000001},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":5.8440000000000003},"last":0.64800000000000002}]},{"osd":3,"last update":"Fri Feb 23 00:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67800000000000005,"15min":0.64000000000000001},"min":{"1min":0.38100000000000001,"5min":0.35699999999999998,"15min":0.316},"max":{"1min":2.6150000000000002,"5min":9.1280000000000001,"15min":9.1280000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56899999999999995,"15min":0.60099999999999998},"min":{"1min":0.39700000000000002,"5min":0.36099999999999999,"15min":0.29699999999999999},"max":{"1min":1.288,"5min":2.1699999999999999,"15min":4.1909999999999998},"last":0.621}]},{"osd":5,"last update":"Fri Feb 23 00:54:47 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.48699999999999999,"15min":0.44500000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.24399999999999999},"max":{"1min":1.627,"5min":1.694,"15min":1.694},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.51600000000000001,"15min":0.497},"min":{"1min":0.34300000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.81999999999999995,"5min":1.663,"15min":1.663},"last":0.84699999999999998}]},{"osd":6,"last update":"Fri Feb 23 00:54:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.46100000000000002,"15min":0.44800000000000001},"min":{"1min":0.309,"5min":0.29099999999999998,"15min":0.253},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.49199999999999999,"15min":0.46200000000000002},"min":{"1min":0.30399999999999999,"5min":0.28599999999999998,"15min":0.215},"max":{"1min":1.264,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":0.66600000000000004}]},{"osd":7,"last update":"Fri Feb 23 00:54:36 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.51300000000000001,"15min":0.48799999999999999},"min":{"1min":0.32400000000000001,"5min":0.30599999999999999,"15min":0.254},"max":{"1min":1.363,"5min":2.294,"15min":2.294},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.58699999999999997,"15min":0.55200000000000005},"min":{"1min":0.39200000000000002,"5min":0.37,"15min":0.32600000000000001},"max":{"1min":1.726,"5min":2.282,"15min":2.282},"last":0.76100000000000001}]}]},{"osd":5,"up_from":39,"seq":167503724749,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":316832,"kb_used_data":3224,"kb_used_omap":0,"kb_used_meta":313600,"kb_avail":93428320,"statfs":{"total":95995035648,"available":95670599680,"internally_reserved":0,"allocated":3301376,"data_stored":1535215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321126400},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:54:33 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.56399999999999995,"15min":0.61699999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.246},"max":{"1min":0.74399999999999999,"5min":5.2220000000000004,"15min":23.881},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.53900000000000003,"15min":0.58599999999999997},"min":{"1min":0.38400000000000001,"5min":0.29099999999999998,"15min":0.27000000000000002},"max":{"1min":0.63200000000000001,"5min":3.9249999999999998,"15min":5.4649999999999999},"last":0.48499999999999999}]},{"osd":1,"last update":"Fri Feb 23 00:54:33 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.55300000000000005,"15min":0.65600000000000003},"min":{"1min":0.379,"5min":0.35299999999999998,"15min":0.254},"max":{"1min":0.68700000000000006,"5min":3.9460000000000002,"15min":18.175999999999998},"last":0.438},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.54100000000000004,"15min":0.621},"min":{"1min":0.41899999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.63600000000000001,"5min":3.976,"15min":9.0950000000000006},"last":0.63100000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:54:33 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.60299999999999998,"15min":0.71799999999999997},"min":{"1min":0.40300000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.72599999999999998,"5min":4.5899999999999999,"15min":22.282},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.60599999999999998,"15min":0.73499999999999999},"min":{"1min":0.375,"5min":0.34399999999999997,"15min":0.29199999999999998},"max":{"1min":0.65400000000000003,"5min":3.8540000000000001,"15min":23.925000000000001},"last":0.53400000000000003}]},{"osd":3,"last update":"Fri Feb 23 00:54:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.54600000000000004,"15min":0.622},"min":{"1min":0.40200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.71499999999999997,"5min":1.194,"15min":6.5730000000000004},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.61799999999999999,"15min":0.66300000000000003},"min":{"1min":0.34999999999999998,"5min":0.34699999999999998,"15min":0.33500000000000002},"max":{"1min":0.72099999999999997,"5min":3.9980000000000002,"15min":8.1760000000000002},"last":0.50900000000000001}]},{"osd":4,"last update":"Fri Feb 23 00:54:33 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.435,"15min":0.44700000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.23499999999999999},"max":{"1min":0.57099999999999995,"5min":0.86099999999999999,"15min":4.593},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.47899999999999998,"15min":0.49099999999999999},"min":{"1min":0.33000000000000002,"5min":0.27900000000000003,"15min":0.27500000000000002},"max":{"1min":0.63700000000000001,"5min":1.169,"15min":3.879},"last":0.58399999999999996}]},{"osd":6,"last update":"Fri Feb 23 00:54:02 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.47899999999999998,"15min":0.48699999999999999},"min":{"1min":0.34300000000000003,"5min":0.30199999999999999,"15min":0.27900000000000003},"max":{"1min":0.67100000000000004,"5min":1.1299999999999999,"15min":4.6740000000000004},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.27100000000000002},"max":{"1min":0.74299999999999999,"5min":0.79900000000000004,"15min":1.159},"last":0.64500000000000002}]},{"osd":7,"last update":"Fri Feb 23 00:54:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54500000000000004,"15min":0.55200000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.30399999999999999},"max":{"1min":0.70099999999999996,"5min":1.1870000000000001,"15min":7.0300000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.51400000000000001,"15min":0.54000000000000004},"min":{"1min":0.35999999999999999,"5min":0.35699999999999998,"15min":0.29099999999999998},"max":{"1min":0.61899999999999999,"5min":0.73599999999999999,"15min":9.0579999999999998},"last":0.59399999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2084864,"data_stored":2084864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":45056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2060288,"data_stored":2060288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2060288,"data_stored":2060288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":45056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":106496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":36864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":250,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":250,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":250,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":250,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":250,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":250,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":250,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-23T00:55:09.275 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2024-02-23T00:55:09.276 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd pool set default.rgw.control pg_num 42 2024-02-23T00:55:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:07.765067+0000 osd.0 (osd.0) 283 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:55:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:07.765275+0000 osd.0 (osd.0) 284 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:55:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:08.337682+0000 mon.a (mon.0) 3644 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:55:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:08.338489+0000 mon.a (mon.0) 3645 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:55:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:08.347499+0000 mon.a (mon.0) 3646 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:55:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:08.347568+0000 mon.a (mon.0) 3647 : cluster [DBG] osdmap e760: 8 total, 8 up, 7 in 2024-02-23T00:55:09.735 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:55:09.737 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:07.765067+0000 osd.0 (osd.0) 283 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:55:09.737 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:07.765275+0000 osd.0 (osd.0) 284 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:55:09.737 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:08.337682+0000 mon.a (mon.0) 3644 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:55:09.737 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:08.338489+0000 mon.a (mon.0) 3645 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:55:09.737 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:08.347499+0000 mon.a (mon.0) 3646 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:55:09.737 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:08.347568+0000 mon.a (mon.0) 3647 : cluster [DBG] osdmap e760: 8 total, 8 up, 7 in 2024-02-23T00:55:09.737 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:07.765067+0000 osd.0 (osd.0) 283 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:55:09.737 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:07.765275+0000 osd.0 (osd.0) 284 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:55:09.737 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:08.337682+0000 mon.a (mon.0) 3644 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:55:09.737 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:08.338489+0000 mon.a (mon.0) 3645 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:55:09.738 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:08.347499+0000 mon.a (mon.0) 3646 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:55:09.738 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:08.347568+0000 mon.a (mon.0) 3647 : cluster [DBG] osdmap e760: 8 total, 8 up, 7 in 2024-02-23T00:55:09.738 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:09.738 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:55:09.839 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:55:09.872 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:09.894 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:55:09.896 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:09.897 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:55:09.997 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:55:10.027 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:10.125 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:55:10.127 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:10.128 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:55:10.229 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:55:10.261 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:10.282 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:55:10.285 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:10.285 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:55:10.386 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:55:10.421 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:10.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:09.352903+0000 mon.a (mon.0) 3648 : cluster [DBG] osdmap e761: 8 total, 8 up, 7 in 2024-02-23T00:55:10.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:09.356689+0000 mon.c (mon.2) 470 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:55:10.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:09.357333+0000 mon.a (mon.0) 3649 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:55:10.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:09.395994+0000 mgr.y (mgr.14517) 649 : cluster [DBG] pgmap v1155: 184 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 151 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 1 op/s; 1/786 objects misplaced (0.127%) 2024-02-23T00:55:10.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:10.309906+0000 mon.a (mon.0) 3650 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:55:10.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:10.310003+0000 mon.a (mon.0) 3651 : cluster [DBG] osdmap e762: 8 total, 8 up, 7 in 2024-02-23T00:55:10.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:10.332009+0000 mon.c (mon.2) 471 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:55:10.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:10.332703+0000 mon.a (mon.0) 3652 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:55:10.807 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:55:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:55:10] "GET /metrics HTTP/1.1" 200 191765 "" "Prometheus/2.33.4" 2024-02-23T00:55:10.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:09.352903+0000 mon.a (mon.0) 3648 : cluster [DBG] osdmap e761: 8 total, 8 up, 7 in 2024-02-23T00:55:10.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:09.356689+0000 mon.c (mon.2) 470 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:55:10.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:09.357333+0000 mon.a (mon.0) 3649 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:55:10.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:09.395994+0000 mgr.y (mgr.14517) 649 : cluster [DBG] pgmap v1155: 184 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 151 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 1 op/s; 1/786 objects misplaced (0.127%) 2024-02-23T00:55:10.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:10.309906+0000 mon.a (mon.0) 3650 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:55:10.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:10.310003+0000 mon.a (mon.0) 3651 : cluster [DBG] osdmap e762: 8 total, 8 up, 7 in 2024-02-23T00:55:10.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:10.332009+0000 mon.c (mon.2) 471 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:55:10.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:10.332703+0000 mon.a (mon.0) 3652 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:55:10.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:09.352903+0000 mon.a (mon.0) 3648 : cluster [DBG] osdmap e761: 8 total, 8 up, 7 in 2024-02-23T00:55:10.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:09.356689+0000 mon.c (mon.2) 470 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:55:10.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:09.357333+0000 mon.a (mon.0) 3649 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:55:10.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:09.395994+0000 mgr.y (mgr.14517) 649 : cluster [DBG] pgmap v1155: 184 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 151 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 1 op/s; 1/786 objects misplaced (0.127%) 2024-02-23T00:55:10.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:10.309906+0000 mon.a (mon.0) 3650 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:55:10.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:10.310003+0000 mon.a (mon.0) 3651 : cluster [DBG] osdmap e762: 8 total, 8 up, 7 in 2024-02-23T00:55:10.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:10.332009+0000 mon.c (mon.2) 471 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:55:10.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:10.332703+0000 mon.a (mon.0) 3652 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:55:11.478 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:55:11.481 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:11.482 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:55:11.582 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:55:11.615 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:11.636 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:55:11.639 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:11.640 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:55:11.721 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:55:11.721 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-02-23T00:55:11.721 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-02-23T00:55:11.721 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-02-23T00:55:11.721 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:55:11.741 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:55:11.772 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:12.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.388+0000 7fe1ae561700 1 -- 172.21.15.47:0/3451596580 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe1a80fe510 msgr2=0x7fe1a80f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:12.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.388+0000 7fe1ae561700 1 --2- 172.21.15.47:0/3451596580 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe1a80fe510 0x7fe1a80f7d40 secure :-1 s=READY pgs=2808 cs=0 l=1 rev1=1 rx=0x7fe19c0047b0 tx=0x7fe19c009cb0).stop 2024-02-23T00:55:12.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.388+0000 7fe1ae561700 1 -- 172.21.15.47:0/3451596580 shutdown_connections 2024-02-23T00:55:12.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.388+0000 7fe1ae561700 1 --2- 172.21.15.47:0/3451596580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1a80f8420 0x7fe1a8104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:12.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.388+0000 7fe1ae561700 1 --2- 172.21.15.47:0/3451596580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a80fdbb0 0x7fe1a80fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:12.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.389+0000 7fe1ae561700 1 --2- 172.21.15.47:0/3451596580 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe1a80fe510 0x7fe1a80f7d40 unknown :-1 s=CLOSED pgs=2808 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:12.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.389+0000 7fe1ae561700 1 -- 172.21.15.47:0/3451596580 >> 172.21.15.47:0/3451596580 conn(0x7fe1a80f3800 msgr2=0x7fe1a80f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:12.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.389+0000 7fe1ae561700 1 -- 172.21.15.47:0/3451596580 shutdown_connections 2024-02-23T00:55:12.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.389+0000 7fe1ae561700 1 -- 172.21.15.47:0/3451596580 wait complete. 2024-02-23T00:55:12.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.389+0000 7fe1ae561700 1 librados: starting msgr at 2024-02-23T00:55:12.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.389+0000 7fe1ae561700 1 librados: starting objecter 2024-02-23T00:55:12.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.389+0000 7fe1ae561700 1 Processor -- start 2024-02-23T00:55:12.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.389+0000 7fe1ae561700 1 -- start start 2024-02-23T00:55:12.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.389+0000 7fe1ae561700 1 librados: setting wanted keys 2024-02-23T00:55:12.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.389+0000 7fe1ae561700 1 librados: calling monclient init 2024-02-23T00:55:12.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.390+0000 7fe1ae561700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe1a80f8420 0x7fe1a810df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:12.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.390+0000 7fe1ae561700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a80fdbb0 0x7fe1a810e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:12.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.390+0000 7fe1ae561700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1a80fe510 0x7fe1a81080e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:12.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.390+0000 7fe1ae561700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a81085d0 con 0x7fe1a80fe510 2024-02-23T00:55:12.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.390+0000 7fe1ae561700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a8108740 con 0x7fe1a80fdbb0 2024-02-23T00:55:12.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.390+0000 7fe1ae561700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a8108900 con 0x7fe1a80f8420 2024-02-23T00:55:12.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.390+0000 7fe1a7fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe1a80f8420 0x7fe1a810df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:12.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.390+0000 7fe1a7fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe1a80f8420 0x7fe1a810df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:60656/0 (socket says 172.21.15.47:60656) 2024-02-23T00:55:12.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.390+0000 7fe1a7fff700 1 -- 172.21.15.47:0/1461618178 learned_addr learned my addr 172.21.15.47:0/1461618178 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:55:12.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.390+0000 7fe1a77fe700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a80fdbb0 0x7fe1a810e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:12.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.390+0000 7fe1acafe700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1a80fe510 0x7fe1a81080e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:12.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.390+0000 7fe1a7fff700 1 -- 172.21.15.47:0/1461618178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a80fdbb0 msgr2=0x7fe1a810e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:12.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.390+0000 7fe1a7fff700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a80fdbb0 0x7fe1a810e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:12.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.390+0000 7fe1a7fff700 1 -- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1a80fe510 msgr2=0x7fe1a81080e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:12.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.390+0000 7fe1a7fff700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1a80fe510 0x7fe1a81080e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:12.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.390+0000 7fe1a7fff700 1 -- 172.21.15.47:0/1461618178 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe19c005040 con 0x7fe1a80f8420 2024-02-23T00:55:12.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.390+0000 7fe1acafe700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1a80fe510 0x7fe1a81080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:55:12.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.390+0000 7fe1a7fff700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe1a80f8420 0x7fe1a810df40 secure :-1 s=READY pgs=2809 cs=0 l=1 rev1=1 rx=0x7fe19800e970 tx=0x7fe19800b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:12.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.390+0000 7fe1a57fa700 10 client.?.objecter ms_handle_connect 0x7fe1a80f8420 2024-02-23T00:55:12.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.390+0000 7fe1a57fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:55:12.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.391+0000 7fe1a57fa700 1 -- 172.21.15.47:0/1461618178 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19800ee60 con 0x7fe1a80f8420 2024-02-23T00:55:12.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.391+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1a8108be0 con 0x7fe1a80f8420 2024-02-23T00:55:12.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.391+0000 7fe1a57fa700 1 -- 172.21.15.47:0/1461618178 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fe19800f040 con 0x7fe1a80f8420 2024-02-23T00:55:12.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.391+0000 7fe1a57fa700 1 -- 172.21.15.47:0/1461618178 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19800c5b0 con 0x7fe1a80f8420 2024-02-23T00:55:12.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.391+0000 7fe1ae561700 10 client.42092.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:55:12.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.391+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1a8108f60 con 0x7fe1a80f8420 2024-02-23T00:55:12.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.391+0000 7fe1ae561700 1 librados: init done 2024-02-23T00:55:12.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.392+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fe1a8106f20 con 0x7fe1a80f8420 2024-02-23T00:55:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.392+0000 7fe1a57fa700 1 -- 172.21.15.47:0/1461618178 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fe198009090 con 0x7fe1a80f8420 2024-02-23T00:55:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.393+0000 7fe1a57fa700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe190062cc0 0x7fe190065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.393+0000 7fe1a57fa700 1 -- 172.21.15.47:0/1461618178 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(764..764 src has 621..764) v4 ==== 9781+0+0 (secure 0 0 0) 0x7fe19807dac0 con 0x7fe1a80f8420 2024-02-23T00:55:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.393+0000 7fe1a57fa700 10 client.42092.objecter ms_dispatch 0x7fe1a8001550 osd_map(764..764 src has 621..764) v4 2024-02-23T00:55:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.393+0000 7fe1a57fa700 3 client.42092.objecter handle_osd_map got epochs [764,764] > 0 2024-02-23T00:55:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.393+0000 7fe1a57fa700 3 client.42092.objecter handle_osd_map decoding full epoch 764 2024-02-23T00:55:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.393+0000 7fe1a77fe700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe190062cc0 0x7fe190065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.393+0000 7fe1a57fa700 20 client.42092.objecter dump_active .. 0 homeless 2024-02-23T00:55:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.393+0000 7fe1a57fa700 1 -- 172.21.15.47:0/1461618178 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v764) v1 ==== 54+0+16 (secure 0 0 0) 0x7fe19807e0a0 con 0x7fe1a80f8420 2024-02-23T00:55:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.393+0000 7fe1a57fa700 10 client.42092.objecter ms_handle_connect 0x7fe190062cc0 2024-02-23T00:55:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.393+0000 7fe1a77fe700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe190062cc0 0x7fe190065180 secure :-1 s=READY pgs=3199 cs=0 l=1 rev1=1 rx=0x7fe1a8109880 tx=0x7fe19c009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.393+0000 7fe1ae561700 10 client.42092.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.393+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_assign -1 1 2024-02-23T00:55:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.393+0000 7fe1ae561700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fe1a80598f0 0x7fe1a8059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.393+0000 7fe1ae561700 20 client.42092.objecter _get_session s=0x7fe1a81095c0 osd=0 3 2024-02-23T00:55:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.393+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a81095c0 osd=0 3 2024-02-23T00:55:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.393+0000 7fe1ae561700 20 client.42092.objecter _get_session s=0x7fe1a81095c0 osd=0 3 2024-02-23T00:55:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.393+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_remove -1 1 2024-02-23T00:55:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.393+0000 7fe1ae561700 20 client.42092.objecter get_session s=0x7fe1a81095c0 osd=0 3 2024-02-23T00:55:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.393+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_assign 0 1 2024-02-23T00:55:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.393+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a81095c0 osd=0 4 2024-02-23T00:55:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.393+0000 7fe1ae561700 10 client.42092.objecter _send_command 1 2024-02-23T00:55:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.393+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe1a805fa60 con 0x7fe1a80598f0 2024-02-23T00:55:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.394+0000 7fe1acafe700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fe1a80598f0 0x7fe1a8059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.394+0000 7fe1a57fa700 10 client.42092.objecter ms_handle_connect 0x7fe1a80598f0 2024-02-23T00:55:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.394+0000 7fe1acafe700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fe1a80598f0 0x7fe1a8059cd0 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.394+0000 7fe1a57fa700 1 -- 172.21.15.47:0/1461618178 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe1a805fa60 con 0x7fe1a80598f0 2024-02-23T00:55:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.394+0000 7fe1a57fa700 10 client.42092.objecter ms_dispatch 0x7fe1a8001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:55:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.394+0000 7fe1a57fa700 10 client.42092.objecter _finish_command 1 = system:0 2024-02-23T00:55:12.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.394+0000 7fe1a57fa700 20 client.42092.objecter put_session s=0x7fe1a81095c0 osd=0 4 2024-02-23T00:55:12.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.394+0000 7fe1a57fa700 15 client.42092.objecter _session_command_op_remove 0 1 2024-02-23T00:55:12.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.410+0000 7fe1ae561700 10 client.42092.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:55:12.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.410+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_assign -1 2 2024-02-23T00:55:12.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.410+0000 7fe1ae561700 20 client.42092.objecter _get_session s=0x7fe1a81095c0 osd=0 3 2024-02-23T00:55:12.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.410+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a81095c0 osd=0 3 2024-02-23T00:55:12.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.410+0000 7fe1ae561700 20 client.42092.objecter _get_session s=0x7fe1a81095c0 osd=0 3 2024-02-23T00:55:12.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.410+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_remove -1 2 2024-02-23T00:55:12.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.410+0000 7fe1ae561700 20 client.42092.objecter get_session s=0x7fe1a81095c0 osd=0 3 2024-02-23T00:55:12.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.410+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_assign 0 2 2024-02-23T00:55:12.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.410+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a81095c0 osd=0 4 2024-02-23T00:55:12.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.410+0000 7fe1ae561700 10 client.42092.objecter _send_command 2 2024-02-23T00:55:12.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.410+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fe1a805fbd0 con 0x7fe1a80598f0 2024-02-23T00:55:12.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.413+0000 7fe1a57fa700 1 -- 172.21.15.47:0/1461618178 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fe1a805fbd0 con 0x7fe1a80598f0 2024-02-23T00:55:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.413+0000 7fe1a57fa700 10 client.42092.objecter ms_dispatch 0x7fe1a8001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:55:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.413+0000 7fe1a57fa700 10 client.42092.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:55:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.413+0000 7fe1a57fa700 20 client.42092.objecter put_session s=0x7fe1a81095c0 osd=0 4 2024-02-23T00:55:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.413+0000 7fe1a57fa700 15 client.42092.objecter _session_command_op_remove 0 2 2024-02-23T00:55:12.416 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:55:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.413+0000 7fe1ae561700 10 client.42092.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.413+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_assign -1 3 2024-02-23T00:55:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.413+0000 7fe1ae561700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fe1a8028540 0x7fe1a8118fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.413+0000 7fe1ae561700 20 client.42092.objecter _get_session s=0x7fe1a80595a0 osd=1 3 2024-02-23T00:55:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.413+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a80595a0 osd=1 3 2024-02-23T00:55:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.413+0000 7fe1ae561700 20 client.42092.objecter _get_session s=0x7fe1a80595a0 osd=1 3 2024-02-23T00:55:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.413+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_remove -1 3 2024-02-23T00:55:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.413+0000 7fe1ae561700 20 client.42092.objecter get_session s=0x7fe1a80595a0 osd=1 3 2024-02-23T00:55:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.413+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_assign 1 3 2024-02-23T00:55:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.413+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a80595a0 osd=1 4 2024-02-23T00:55:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.413+0000 7fe1ae561700 10 client.42092.objecter _send_command 3 2024-02-23T00:55:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.413+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe1a8059710 con 0x7fe1a8028540 2024-02-23T00:55:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.413+0000 7fe1a7fff700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fe1a8028540 0x7fe1a8118fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.414+0000 7fe1a57fa700 10 client.42092.objecter ms_handle_connect 0x7fe1a8028540 2024-02-23T00:55:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.414+0000 7fe1a7fff700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fe1a8028540 0x7fe1a8118fe0 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.414+0000 7fe1a57fa700 1 -- 172.21.15.47:0/1461618178 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe1a8059710 con 0x7fe1a8028540 2024-02-23T00:55:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.414+0000 7fe1a57fa700 10 client.42092.objecter ms_dispatch 0x7fe1a8001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:55:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.414+0000 7fe1a57fa700 10 client.42092.objecter _finish_command 3 = system:0 2024-02-23T00:55:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.414+0000 7fe1a57fa700 20 client.42092.objecter put_session s=0x7fe1a80595a0 osd=1 4 2024-02-23T00:55:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.414+0000 7fe1a57fa700 15 client.42092.objecter _session_command_op_remove 1 3 2024-02-23T00:55:12.431 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:55:12.434 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:11.312052+0000 mon.a (mon.0) 3653 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:55:12.434 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:11.322096+0000 mon.a (mon.0) 3654 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:55:12.434 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:11.322224+0000 mon.a (mon.0) 3655 : cluster [DBG] osdmap e763: 8 total, 8 up, 7 in 2024-02-23T00:55:12.434 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:11.325005+0000 mon.c (mon.2) 472 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-134"}]: dispatch 2024-02-23T00:55:12.434 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:11.325414+0000 mon.a (mon.0) 3656 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-134"}]: dispatch 2024-02-23T00:55:12.434 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:11.396367+0000 mgr.y (mgr.14517) 650 : cluster [DBG] pgmap v1158: 184 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 151 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1/786 objects misplaced (0.127%) 2024-02-23T00:55:12.435 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:11.312052+0000 mon.a (mon.0) 3653 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:55:12.435 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:11.322096+0000 mon.a (mon.0) 3654 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:55:12.435 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:11.322224+0000 mon.a (mon.0) 3655 : cluster [DBG] osdmap e763: 8 total, 8 up, 7 in 2024-02-23T00:55:12.435 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:11.325005+0000 mon.c (mon.2) 472 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-134"}]: dispatch 2024-02-23T00:55:12.435 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:11.325414+0000 mon.a (mon.0) 3656 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-134"}]: dispatch 2024-02-23T00:55:12.435 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:11.396367+0000 mgr.y (mgr.14517) 650 : cluster [DBG] pgmap v1158: 184 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 151 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1/786 objects misplaced (0.127%) 2024-02-23T00:55:12.436 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:12.436 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:55:12.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.434+0000 7fe1ae561700 10 client.42092.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:55:12.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.434+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_assign -1 4 2024-02-23T00:55:12.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.434+0000 7fe1ae561700 20 client.42092.objecter _get_session s=0x7fe1a80595a0 osd=1 3 2024-02-23T00:55:12.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.434+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a80595a0 osd=1 3 2024-02-23T00:55:12.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.434+0000 7fe1ae561700 20 client.42092.objecter _get_session s=0x7fe1a80595a0 osd=1 3 2024-02-23T00:55:12.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.434+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_remove -1 4 2024-02-23T00:55:12.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.434+0000 7fe1ae561700 20 client.42092.objecter get_session s=0x7fe1a80595a0 osd=1 3 2024-02-23T00:55:12.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.434+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_assign 1 4 2024-02-23T00:55:12.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.434+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a80595a0 osd=1 4 2024-02-23T00:55:12.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.434+0000 7fe1ae561700 10 client.42092.objecter _send_command 4 2024-02-23T00:55:12.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.434+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fe1a80ff830 con 0x7fe1a8028540 2024-02-23T00:55:12.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.436+0000 7fe1a57fa700 1 -- 172.21.15.47:0/1461618178 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fe1a80ff830 con 0x7fe1a8028540 2024-02-23T00:55:12.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.436+0000 7fe1a57fa700 10 client.42092.objecter ms_dispatch 0x7fe1a8001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:55:12.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.436+0000 7fe1a57fa700 10 client.42092.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:55:12.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.436+0000 7fe1a57fa700 20 client.42092.objecter put_session s=0x7fe1a80595a0 osd=1 4 2024-02-23T00:55:12.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.436+0000 7fe1a57fa700 15 client.42092.objecter _session_command_op_remove 1 4 2024-02-23T00:55:12.440 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:55:12.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.436+0000 7fe1ae561700 10 client.42092.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:12.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.436+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_assign -1 5 2024-02-23T00:55:12.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.436+0000 7fe1ae561700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fe1a80fecc0 0x7fe1a811c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:12.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.436+0000 7fe1ae561700 20 client.42092.objecter _get_session s=0x7fe1a80feb50 osd=2 3 2024-02-23T00:55:12.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.436+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a80feb50 osd=2 3 2024-02-23T00:55:12.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.436+0000 7fe1ae561700 20 client.42092.objecter _get_session s=0x7fe1a80feb50 osd=2 3 2024-02-23T00:55:12.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.436+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_remove -1 5 2024-02-23T00:55:12.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.436+0000 7fe1ae561700 20 client.42092.objecter get_session s=0x7fe1a80feb50 osd=2 3 2024-02-23T00:55:12.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.436+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_assign 2 5 2024-02-23T00:55:12.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.436+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a80feb50 osd=2 4 2024-02-23T00:55:12.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.436+0000 7fe1ae561700 10 client.42092.objecter _send_command 5 2024-02-23T00:55:12.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.436+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe1a80281f0 con 0x7fe1a80fecc0 2024-02-23T00:55:12.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.436+0000 7fe1a77fe700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fe1a80fecc0 0x7fe1a811c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:12.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.437+0000 7fe1a57fa700 10 client.42092.objecter ms_handle_connect 0x7fe1a80fecc0 2024-02-23T00:55:12.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.437+0000 7fe1a77fe700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fe1a80fecc0 0x7fe1a811c660 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:12.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.437+0000 7fe1a57fa700 1 -- 172.21.15.47:0/1461618178 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe1a80281f0 con 0x7fe1a80fecc0 2024-02-23T00:55:12.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.437+0000 7fe1a57fa700 10 client.42092.objecter ms_dispatch 0x7fe1a8001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:55:12.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.437+0000 7fe1a57fa700 10 client.42092.objecter _finish_command 5 = system:0 2024-02-23T00:55:12.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.437+0000 7fe1a57fa700 20 client.42092.objecter put_session s=0x7fe1a80feb50 osd=2 4 2024-02-23T00:55:12.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.437+0000 7fe1a57fa700 15 client.42092.objecter _session_command_op_remove 2 5 2024-02-23T00:55:12.448 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:55:12.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.453+0000 7fe1ae561700 10 client.42092.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:55:12.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.453+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_assign -1 6 2024-02-23T00:55:12.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.453+0000 7fe1ae561700 20 client.42092.objecter _get_session s=0x7fe1a80feb50 osd=2 3 2024-02-23T00:55:12.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.453+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a80feb50 osd=2 3 2024-02-23T00:55:12.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.453+0000 7fe1ae561700 20 client.42092.objecter _get_session s=0x7fe1a80feb50 osd=2 3 2024-02-23T00:55:12.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.453+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_remove -1 6 2024-02-23T00:55:12.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.453+0000 7fe1ae561700 20 client.42092.objecter get_session s=0x7fe1a80feb50 osd=2 3 2024-02-23T00:55:12.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.453+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_assign 2 6 2024-02-23T00:55:12.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.453+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a80feb50 osd=2 4 2024-02-23T00:55:12.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.453+0000 7fe1ae561700 10 client.42092.objecter _send_command 6 2024-02-23T00:55:12.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.453+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fe1a80283c0 con 0x7fe1a80fecc0 2024-02-23T00:55:12.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.456+0000 7fe1a57fa700 1 -- 172.21.15.47:0/1461618178 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fe19c019040 con 0x7fe1a80fecc0 2024-02-23T00:55:12.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.456+0000 7fe1a57fa700 10 client.42092.objecter ms_dispatch 0x7fe1a8001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:55:12.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.456+0000 7fe1a57fa700 10 client.42092.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:55:12.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.456+0000 7fe1a57fa700 20 client.42092.objecter put_session s=0x7fe1a80feb50 osd=2 4 2024-02-23T00:55:12.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.456+0000 7fe1a57fa700 15 client.42092.objecter _session_command_op_remove 2 6 2024-02-23T00:55:12.459 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:55:12.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.456+0000 7fe1ae561700 10 client.42092.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:12.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.456+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_assign -1 7 2024-02-23T00:55:12.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.456+0000 7fe1ae561700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fe1a811a1b0 0x7fe1a811fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:12.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.456+0000 7fe1ae561700 20 client.42092.objecter _get_session s=0x7fe1a80278f0 osd=3 3 2024-02-23T00:55:12.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.456+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a80278f0 osd=3 3 2024-02-23T00:55:12.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.456+0000 7fe1ae561700 20 client.42092.objecter _get_session s=0x7fe1a80278f0 osd=3 3 2024-02-23T00:55:12.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.456+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_remove -1 7 2024-02-23T00:55:12.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.456+0000 7fe1ae561700 20 client.42092.objecter get_session s=0x7fe1a80278f0 osd=3 3 2024-02-23T00:55:12.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.456+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_assign 3 7 2024-02-23T00:55:12.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.456+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a80278f0 osd=3 4 2024-02-23T00:55:12.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.456+0000 7fe1ae561700 10 client.42092.objecter _send_command 7 2024-02-23T00:55:12.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.456+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe1a8122930 con 0x7fe1a811a1b0 2024-02-23T00:55:12.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.456+0000 7fe1acafe700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fe1a811a1b0 0x7fe1a811fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:12.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.457+0000 7fe1a57fa700 10 client.42092.objecter ms_handle_connect 0x7fe1a811a1b0 2024-02-23T00:55:12.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.457+0000 7fe1acafe700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fe1a811a1b0 0x7fe1a811fce0 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:12.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.457+0000 7fe1a57fa700 1 -- 172.21.15.47:0/1461618178 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe1a8122930 con 0x7fe1a811a1b0 2024-02-23T00:55:12.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.457+0000 7fe1a57fa700 10 client.42092.objecter ms_dispatch 0x7fe1a8001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:55:12.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.457+0000 7fe1a57fa700 10 client.42092.objecter _finish_command 7 = system:0 2024-02-23T00:55:12.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.457+0000 7fe1a57fa700 20 client.42092.objecter put_session s=0x7fe1a80278f0 osd=3 4 2024-02-23T00:55:12.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.457+0000 7fe1a57fa700 15 client.42092.objecter _session_command_op_remove 3 7 2024-02-23T00:55:12.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.474+0000 7fe1ae561700 10 client.42092.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:55:12.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.474+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_assign -1 8 2024-02-23T00:55:12.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.474+0000 7fe1ae561700 20 client.42092.objecter _get_session s=0x7fe1a80278f0 osd=3 3 2024-02-23T00:55:12.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.474+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a80278f0 osd=3 3 2024-02-23T00:55:12.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.474+0000 7fe1ae561700 20 client.42092.objecter _get_session s=0x7fe1a80278f0 osd=3 3 2024-02-23T00:55:12.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.474+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_remove -1 8 2024-02-23T00:55:12.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.474+0000 7fe1ae561700 20 client.42092.objecter get_session s=0x7fe1a80278f0 osd=3 3 2024-02-23T00:55:12.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.474+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_assign 3 8 2024-02-23T00:55:12.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.474+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a80278f0 osd=3 4 2024-02-23T00:55:12.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.474+0000 7fe1ae561700 10 client.42092.objecter _send_command 8 2024-02-23T00:55:12.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.474+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fe1a8119710 con 0x7fe1a811a1b0 2024-02-23T00:55:12.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.475+0000 7fe1a57fa700 1 -- 172.21.15.47:0/1461618178 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fe1a8119710 con 0x7fe1a811a1b0 2024-02-23T00:55:12.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.475+0000 7fe1a57fa700 10 client.42092.objecter ms_dispatch 0x7fe1a8001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:55:12.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.475+0000 7fe1a57fa700 10 client.42092.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:55:12.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.475+0000 7fe1a57fa700 20 client.42092.objecter put_session s=0x7fe1a80278f0 osd=3 4 2024-02-23T00:55:12.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.475+0000 7fe1a57fa700 15 client.42092.objecter _session_command_op_remove 3 8 2024-02-23T00:55:12.478 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:55:12.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.475+0000 7fe1ae561700 10 client.42092.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:12.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.475+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_assign -1 9 2024-02-23T00:55:12.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.475+0000 7fe1ae561700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fe1a811d830 0x7fe1a8121350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:12.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.475+0000 7fe1ae561700 20 client.42092.objecter _get_session s=0x7fe1a81194d0 osd=4 3 2024-02-23T00:55:12.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.475+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a81194d0 osd=4 3 2024-02-23T00:55:12.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.475+0000 7fe1ae561700 20 client.42092.objecter _get_session s=0x7fe1a81194d0 osd=4 3 2024-02-23T00:55:12.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.475+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_remove -1 9 2024-02-23T00:55:12.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.475+0000 7fe1ae561700 20 client.42092.objecter get_session s=0x7fe1a81194d0 osd=4 3 2024-02-23T00:55:12.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.475+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_assign 4 9 2024-02-23T00:55:12.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.475+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a81194d0 osd=4 4 2024-02-23T00:55:12.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.475+0000 7fe1ae561700 10 client.42092.objecter _send_command 9 2024-02-23T00:55:12.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.475+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe1a81196d0 con 0x7fe1a811d830 2024-02-23T00:55:12.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.476+0000 7fe1a7fff700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fe1a811d830 0x7fe1a8121350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:12.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.476+0000 7fe1a57fa700 10 client.42092.objecter ms_handle_connect 0x7fe1a811d830 2024-02-23T00:55:12.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.476+0000 7fe1a7fff700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fe1a811d830 0x7fe1a8121350 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:12.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.477+0000 7fe1a57fa700 1 -- 172.21.15.47:0/1461618178 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe1a81196d0 con 0x7fe1a811d830 2024-02-23T00:55:12.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.477+0000 7fe1a57fa700 10 client.42092.objecter ms_dispatch 0x7fe1a8001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:55:12.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.477+0000 7fe1a57fa700 10 client.42092.objecter _finish_command 9 = system:0 2024-02-23T00:55:12.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.477+0000 7fe1a57fa700 20 client.42092.objecter put_session s=0x7fe1a81194d0 osd=4 4 2024-02-23T00:55:12.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.477+0000 7fe1a57fa700 15 client.42092.objecter _session_command_op_remove 4 9 2024-02-23T00:55:12.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.495+0000 7fe1ae561700 10 client.42092.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:55:12.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.495+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_assign -1 10 2024-02-23T00:55:12.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.495+0000 7fe1ae561700 20 client.42092.objecter _get_session s=0x7fe1a81194d0 osd=4 3 2024-02-23T00:55:12.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.495+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a81194d0 osd=4 3 2024-02-23T00:55:12.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.495+0000 7fe1ae561700 20 client.42092.objecter _get_session s=0x7fe1a81194d0 osd=4 3 2024-02-23T00:55:12.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.495+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_remove -1 10 2024-02-23T00:55:12.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.495+0000 7fe1ae561700 20 client.42092.objecter get_session s=0x7fe1a81194d0 osd=4 3 2024-02-23T00:55:12.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.495+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_assign 4 10 2024-02-23T00:55:12.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.495+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a81194d0 osd=4 4 2024-02-23T00:55:12.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.495+0000 7fe1ae561700 10 client.42092.objecter _send_command 10 2024-02-23T00:55:12.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.495+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fe1a81196d0 con 0x7fe1a811d830 2024-02-23T00:55:12.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.497+0000 7fe1a57fa700 1 -- 172.21.15.47:0/1461618178 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fe1a81196d0 con 0x7fe1a811d830 2024-02-23T00:55:12.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.497+0000 7fe1a57fa700 10 client.42092.objecter ms_dispatch 0x7fe1a8001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:55:12.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.497+0000 7fe1a57fa700 10 client.42092.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:55:12.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.497+0000 7fe1a57fa700 20 client.42092.objecter put_session s=0x7fe1a81194d0 osd=4 4 2024-02-23T00:55:12.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.497+0000 7fe1a57fa700 15 client.42092.objecter _session_command_op_remove 4 10 2024-02-23T00:55:12.500 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:55:12.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.497+0000 7fe1ae561700 10 client.42092.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:12.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.497+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_assign -1 11 2024-02-23T00:55:12.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.497+0000 7fe1ae561700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fe1a8121da0 0x7fe1a8122220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:12.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.497+0000 7fe1ae561700 20 client.42092.objecter _get_session s=0x7fe1a81196d0 osd=5 3 2024-02-23T00:55:12.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.497+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a81196d0 osd=5 3 2024-02-23T00:55:12.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.497+0000 7fe1ae561700 20 client.42092.objecter _get_session s=0x7fe1a81196d0 osd=5 3 2024-02-23T00:55:12.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.497+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_remove -1 11 2024-02-23T00:55:12.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.497+0000 7fe1ae561700 20 client.42092.objecter get_session s=0x7fe1a81196d0 osd=5 3 2024-02-23T00:55:12.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.497+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_assign 5 11 2024-02-23T00:55:12.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.497+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a81196d0 osd=5 4 2024-02-23T00:55:12.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.497+0000 7fe1ae561700 10 client.42092.objecter _send_command 11 2024-02-23T00:55:12.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.497+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe1a8129250 con 0x7fe1a8121da0 2024-02-23T00:55:12.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.498+0000 7fe1a77fe700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fe1a8121da0 0x7fe1a8122220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:12.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.498+0000 7fe1a77fe700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fe1a8121da0 0x7fe1a8122220 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:12.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.498+0000 7fe1a57fa700 10 client.42092.objecter ms_handle_connect 0x7fe1a8121da0 2024-02-23T00:55:12.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.499+0000 7fe1a57fa700 1 -- 172.21.15.47:0/1461618178 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe1a8129250 con 0x7fe1a8121da0 2024-02-23T00:55:12.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.499+0000 7fe1a57fa700 10 client.42092.objecter ms_dispatch 0x7fe1a8001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:55:12.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.499+0000 7fe1a57fa700 10 client.42092.objecter _finish_command 11 = system:0 2024-02-23T00:55:12.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.499+0000 7fe1a57fa700 20 client.42092.objecter put_session s=0x7fe1a81196d0 osd=5 4 2024-02-23T00:55:12.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.499+0000 7fe1a57fa700 15 client.42092.objecter _session_command_op_remove 5 11 2024-02-23T00:55:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.513+0000 7fe1ae561700 10 client.42092.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:55:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.513+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_assign -1 12 2024-02-23T00:55:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.513+0000 7fe1ae561700 20 client.42092.objecter _get_session s=0x7fe1a81196d0 osd=5 3 2024-02-23T00:55:12.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.513+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a81196d0 osd=5 3 2024-02-23T00:55:12.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.513+0000 7fe1ae561700 20 client.42092.objecter _get_session s=0x7fe1a81196d0 osd=5 3 2024-02-23T00:55:12.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.513+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_remove -1 12 2024-02-23T00:55:12.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.513+0000 7fe1ae561700 20 client.42092.objecter get_session s=0x7fe1a81196d0 osd=5 3 2024-02-23T00:55:12.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.513+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_assign 5 12 2024-02-23T00:55:12.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.513+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a81196d0 osd=5 4 2024-02-23T00:55:12.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.513+0000 7fe1ae561700 10 client.42092.objecter _send_command 12 2024-02-23T00:55:12.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.513+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fe1a8126af0 con 0x7fe1a8121da0 2024-02-23T00:55:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.515+0000 7fe1a57fa700 1 -- 172.21.15.47:0/1461618178 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fe1a8126af0 con 0x7fe1a8121da0 2024-02-23T00:55:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.515+0000 7fe1a57fa700 10 client.42092.objecter ms_dispatch 0x7fe1a8001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:55:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.515+0000 7fe1a57fa700 10 client.42092.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:55:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.515+0000 7fe1a57fa700 20 client.42092.objecter put_session s=0x7fe1a81196d0 osd=5 4 2024-02-23T00:55:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.515+0000 7fe1a57fa700 15 client.42092.objecter _session_command_op_remove 5 12 2024-02-23T00:55:12.519 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:55:12.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.515+0000 7fe1ae561700 10 client.42092.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:12.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.515+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_assign -1 13 2024-02-23T00:55:12.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.515+0000 7fe1ae561700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fe1a81271d0 0x7fe1a81295e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:12.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.515+0000 7fe1ae561700 20 client.42092.objecter _get_session s=0x7fe1a8126af0 osd=6 3 2024-02-23T00:55:12.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.515+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a8126af0 osd=6 3 2024-02-23T00:55:12.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.515+0000 7fe1ae561700 20 client.42092.objecter _get_session s=0x7fe1a8126af0 osd=6 3 2024-02-23T00:55:12.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.515+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_remove -1 13 2024-02-23T00:55:12.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.515+0000 7fe1ae561700 20 client.42092.objecter get_session s=0x7fe1a8126af0 osd=6 3 2024-02-23T00:55:12.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.515+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_assign 6 13 2024-02-23T00:55:12.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.515+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a8126af0 osd=6 4 2024-02-23T00:55:12.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.515+0000 7fe1ae561700 10 client.42092.objecter _send_command 13 2024-02-23T00:55:12.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.515+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe1a812c420 con 0x7fe1a81271d0 2024-02-23T00:55:12.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.516+0000 7fe1acafe700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fe1a81271d0 0x7fe1a81295e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:12.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.516+0000 7fe1a57fa700 10 client.42092.objecter ms_handle_connect 0x7fe1a81271d0 2024-02-23T00:55:12.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.516+0000 7fe1acafe700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fe1a81271d0 0x7fe1a81295e0 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:12.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.517+0000 7fe1a57fa700 1 -- 172.21.15.47:0/1461618178 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe1a812c420 con 0x7fe1a81271d0 2024-02-23T00:55:12.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.517+0000 7fe1a57fa700 10 client.42092.objecter ms_dispatch 0x7fe1a8001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:55:12.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.517+0000 7fe1a57fa700 10 client.42092.objecter _finish_command 13 = system:0 2024-02-23T00:55:12.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.517+0000 7fe1a57fa700 20 client.42092.objecter put_session s=0x7fe1a8126af0 osd=6 4 2024-02-23T00:55:12.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.517+0000 7fe1a57fa700 15 client.42092.objecter _session_command_op_remove 6 13 2024-02-23T00:55:12.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.531+0000 7fe1ae561700 10 client.42092.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:55:12.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.531+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_assign -1 14 2024-02-23T00:55:12.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.531+0000 7fe1ae561700 20 client.42092.objecter _get_session s=0x7fe1a8126af0 osd=6 3 2024-02-23T00:55:12.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.531+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a8126af0 osd=6 3 2024-02-23T00:55:12.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.531+0000 7fe1ae561700 20 client.42092.objecter _get_session s=0x7fe1a8126af0 osd=6 3 2024-02-23T00:55:12.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.531+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_remove -1 14 2024-02-23T00:55:12.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.531+0000 7fe1ae561700 20 client.42092.objecter get_session s=0x7fe1a8126af0 osd=6 3 2024-02-23T00:55:12.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.531+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_assign 6 14 2024-02-23T00:55:12.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.531+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a8126af0 osd=6 4 2024-02-23T00:55:12.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.531+0000 7fe1ae561700 10 client.42092.objecter _send_command 14 2024-02-23T00:55:12.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.531+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fe1a812c420 con 0x7fe1a81271d0 2024-02-23T00:55:12.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.533+0000 7fe1a57fa700 1 -- 172.21.15.47:0/1461618178 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fe1a812c420 con 0x7fe1a81271d0 2024-02-23T00:55:12.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.533+0000 7fe1a57fa700 10 client.42092.objecter ms_dispatch 0x7fe1a8001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:55:12.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.533+0000 7fe1a57fa700 10 client.42092.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:55:12.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.534+0000 7fe1a57fa700 20 client.42092.objecter put_session s=0x7fe1a8126af0 osd=6 4 2024-02-23T00:55:12.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.534+0000 7fe1a57fa700 15 client.42092.objecter _session_command_op_remove 6 14 2024-02-23T00:55:12.537 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:55:12.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.534+0000 7fe1ae561700 10 client.42092.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:12.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.534+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_assign -1 15 2024-02-23T00:55:12.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.534+0000 7fe1ae561700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fe1a812b750 0x7fe1a812bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:12.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.534+0000 7fe1ae561700 20 client.42092.objecter _get_session s=0x7fe1a812c420 osd=7 3 2024-02-23T00:55:12.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.534+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a812c420 osd=7 3 2024-02-23T00:55:12.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.534+0000 7fe1ae561700 20 client.42092.objecter _get_session s=0x7fe1a812c420 osd=7 3 2024-02-23T00:55:12.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.534+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_remove -1 15 2024-02-23T00:55:12.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.534+0000 7fe1ae561700 20 client.42092.objecter get_session s=0x7fe1a812c420 osd=7 3 2024-02-23T00:55:12.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.534+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_assign 7 15 2024-02-23T00:55:12.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.534+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a812c420 osd=7 4 2024-02-23T00:55:12.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.534+0000 7fe1ae561700 10 client.42092.objecter _send_command 15 2024-02-23T00:55:12.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.534+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe1a812f780 con 0x7fe1a812b750 2024-02-23T00:55:12.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.534+0000 7fe1a7fff700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fe1a812b750 0x7fe1a812bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:12.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.534+0000 7fe1a57fa700 10 client.42092.objecter ms_handle_connect 0x7fe1a812b750 2024-02-23T00:55:12.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.534+0000 7fe1a7fff700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fe1a812b750 0x7fe1a812bbd0 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:12.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.535+0000 7fe1a57fa700 1 -- 172.21.15.47:0/1461618178 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe1a812f780 con 0x7fe1a812b750 2024-02-23T00:55:12.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.535+0000 7fe1a57fa700 10 client.42092.objecter ms_dispatch 0x7fe1a8001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:55:12.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.535+0000 7fe1a57fa700 10 client.42092.objecter _finish_command 15 = system:0 2024-02-23T00:55:12.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.535+0000 7fe1a57fa700 20 client.42092.objecter put_session s=0x7fe1a812c420 osd=7 4 2024-02-23T00:55:12.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.535+0000 7fe1a57fa700 15 client.42092.objecter _session_command_op_remove 7 15 2024-02-23T00:55:12.540 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:55:12.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.551+0000 7fe1ae561700 10 client.42092.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:55:12.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.551+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_assign -1 16 2024-02-23T00:55:12.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.551+0000 7fe1ae561700 20 client.42092.objecter _get_session s=0x7fe1a812c420 osd=7 3 2024-02-23T00:55:12.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.551+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a812c420 osd=7 3 2024-02-23T00:55:12.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.551+0000 7fe1ae561700 20 client.42092.objecter _get_session s=0x7fe1a812c420 osd=7 3 2024-02-23T00:55:12.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.551+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_remove -1 16 2024-02-23T00:55:12.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.551+0000 7fe1ae561700 20 client.42092.objecter get_session s=0x7fe1a812c420 osd=7 3 2024-02-23T00:55:12.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.551+0000 7fe1ae561700 15 client.42092.objecter _session_command_op_assign 7 16 2024-02-23T00:55:12.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.551+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a812c420 osd=7 4 2024-02-23T00:55:12.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.551+0000 7fe1ae561700 10 client.42092.objecter _send_command 16 2024-02-23T00:55:12.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.551+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fe1a8129be0 con 0x7fe1a812b750 2024-02-23T00:55:12.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.553+0000 7fe1a57fa700 1 -- 172.21.15.47:0/1461618178 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fe1a8129be0 con 0x7fe1a812b750 2024-02-23T00:55:12.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.553+0000 7fe1a57fa700 10 client.42092.objecter ms_dispatch 0x7fe1a8001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:55:12.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.553+0000 7fe1a57fa700 10 client.42092.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:55:12.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.553+0000 7fe1a57fa700 20 client.42092.objecter put_session s=0x7fe1a812c420 osd=7 4 2024-02-23T00:55:12.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.553+0000 7fe1a57fa700 15 client.42092.objecter _session_command_op_remove 7 16 2024-02-23T00:55:12.556 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:55:12.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.553+0000 7fe1ae561700 10 librados: watch_flush enter 2024-02-23T00:55:12.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.553+0000 7fe1ae561700 10 librados: watch_flush exit 2024-02-23T00:55:12.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.553+0000 7fe1ae561700 10 client.42092.objecter close_session for osd.0 2024-02-23T00:55:12.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.553+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fe1a80598f0 msgr2=0x7fe1a8059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:12.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.553+0000 7fe1ae561700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fe1a80598f0 0x7fe1a8059cd0 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:12.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a81095c0 osd=0 1 2024-02-23T00:55:12.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 10 client.42092.objecter close_session for osd.1 2024-02-23T00:55:12.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fe1a8028540 msgr2=0x7fe1a8118fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:12.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fe1a8028540 0x7fe1a8118fe0 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:12.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a80595a0 osd=1 1 2024-02-23T00:55:12.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 10 client.42092.objecter close_session for osd.2 2024-02-23T00:55:12.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fe1a80fecc0 msgr2=0x7fe1a811c660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:12.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fe1a80fecc0 0x7fe1a811c660 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:12.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a80feb50 osd=2 1 2024-02-23T00:55:12.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 10 client.42092.objecter close_session for osd.3 2024-02-23T00:55:12.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fe1a811a1b0 msgr2=0x7fe1a811fce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:12.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fe1a811a1b0 0x7fe1a811fce0 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:12.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a80278f0 osd=3 1 2024-02-23T00:55:12.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 10 client.42092.objecter close_session for osd.4 2024-02-23T00:55:12.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fe1a811d830 msgr2=0x7fe1a8121350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:12.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fe1a811d830 0x7fe1a8121350 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:12.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1acafe700 1 -- 172.21.15.47:0/1461618178 reap_dead start 2024-02-23T00:55:12.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a81194d0 osd=4 1 2024-02-23T00:55:12.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 10 client.42092.objecter close_session for osd.5 2024-02-23T00:55:12.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fe1a8121da0 msgr2=0x7fe1a8122220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:12.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fe1a8121da0 0x7fe1a8122220 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:12.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1acafe700 1 -- 172.21.15.47:0/1461618178 reap_dead start 2024-02-23T00:55:12.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a81196d0 osd=5 1 2024-02-23T00:55:12.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 10 client.42092.objecter close_session for osd.6 2024-02-23T00:55:12.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fe1a81271d0 msgr2=0x7fe1a81295e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:12.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fe1a81271d0 0x7fe1a81295e0 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:12.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a8126af0 osd=6 1 2024-02-23T00:55:12.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 10 client.42092.objecter close_session for osd.7 2024-02-23T00:55:12.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fe1a812b750 msgr2=0x7fe1a812bbd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:12.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fe1a812b750 0x7fe1a812bbd0 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:12.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1acafe700 1 -- 172.21.15.47:0/1461618178 reap_dead start 2024-02-23T00:55:12.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 20 client.42092.objecter put_session s=0x7fe1a812c420 osd=7 1 2024-02-23T00:55:12.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 20 client.42092.objecter shutdown clearing up homeless session... 2024-02-23T00:55:12.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 10 client.42092.objecter successfully canceled tick 2024-02-23T00:55:12.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe190062cc0 msgr2=0x7fe190065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:12.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe190062cc0 0x7fe190065180 secure :-1 s=READY pgs=3199 cs=0 l=1 rev1=1 rx=0x7fe1a8109880 tx=0x7fe19c009cb0).stop 2024-02-23T00:55:12.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe1a80f8420 msgr2=0x7fe1a810df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:12.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe1a80f8420 0x7fe1a810df40 secure :-1 s=READY pgs=2809 cs=0 l=1 rev1=1 rx=0x7fe19800e970 tx=0x7fe19800b040).stop 2024-02-23T00:55:12.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 shutdown_connections 2024-02-23T00:55:12.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe1a80f8420 0x7fe1a810df40 unknown :-1 s=CLOSED pgs=2809 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:12.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe190062cc0 0x7fe190065180 unknown :-1 s=CLOSED pgs=3199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:12.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 1 --2- 172.21.15.47:0/1461618178 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fe1a812b750 0x7fe1a812bbd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:12.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 >> 172.21.15.47:0/1461618178 conn(0x7fe1a80f3800 msgr2=0x7fe1a80fb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:12.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 shutdown_connections 2024-02-23T00:55:12.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 1 -- 172.21.15.47:0/1461618178 wait complete. 2024-02-23T00:55:12.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:12.554+0000 7fe1ae561700 1 librados: shutdown 2024-02-23T00:55:12.562 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:55:12.562 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:55:12.562 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:55:12.562 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:55:12.563 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:55:12.563 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:55:12.563 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:55:12.563 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:55:12.572 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:12.694 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:55:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:55:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:55:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:11.312052+0000 mon.a (mon.0) 3653 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:55:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:11.322096+0000 mon.a (mon.0) 3654 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:55:12.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:11.322224+0000 mon.a (mon.0) 3655 : cluster [DBG] osdmap e763: 8 total, 8 up, 7 in 2024-02-23T00:55:12.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:11.325005+0000 mon.c (mon.2) 472 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-134"}]: dispatch 2024-02-23T00:55:12.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:11.325414+0000 mon.a (mon.0) 3656 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-134"}]: dispatch 2024-02-23T00:55:12.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:11.396367+0000 mgr.y (mgr.14517) 650 : cluster [DBG] pgmap v1158: 184 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 151 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1/786 objects misplaced (0.127%) 2024-02-23T00:55:12.921 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:55:12.924 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:12.924 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:55:12.941 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.3 dump_historic_ops 2024-02-23T00:55:13.025 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:55:13.052 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:13.333 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:55:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:55:13.503 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset noscrub 2024-02-23T00:55:13.580 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:11.565875+0000 osd.6 (osd.6) 299 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:55:13.580 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:11.566096+0000 osd.6 (osd.6) 300 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:55:13.581 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:12.325355+0000 mon.a (mon.0) 3657 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-134"}]': finished 2024-02-23T00:55:13.581 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:12.325436+0000 mon.a (mon.0) 3658 : cluster [DBG] osdmap e764: 8 total, 8 up, 7 in 2024-02-23T00:55:13.581 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:12.328598+0000 mon.c (mon.2) 473 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-134", "mode": "readproxy"}]: dispatch 2024-02-23T00:55:13.581 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:12.328872+0000 mon.a (mon.0) 3659 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-134", "mode": "readproxy"}]: dispatch 2024-02-23T00:55:13.581 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:12.393552+0000 mon.c (mon.2) 474 : audit [DBG] from='client.? 172.21.15.47:0/1461618178' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:55:13.582 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:11.565875+0000 osd.6 (osd.6) 299 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:55:13.582 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:11.566096+0000 osd.6 (osd.6) 300 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:55:13.582 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:12.325355+0000 mon.a (mon.0) 3657 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-134"}]': finished 2024-02-23T00:55:13.582 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:12.325436+0000 mon.a (mon.0) 3658 : cluster [DBG] osdmap e764: 8 total, 8 up, 7 in 2024-02-23T00:55:13.583 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:12.328598+0000 mon.c (mon.2) 473 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-134", "mode": "readproxy"}]: dispatch 2024-02-23T00:55:13.583 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:12.328872+0000 mon.a (mon.0) 3659 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-134", "mode": "readproxy"}]: dispatch 2024-02-23T00:55:13.583 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:12.393552+0000 mon.c (mon.2) 474 : audit [DBG] from='client.? 172.21.15.47:0/1461618178' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:55:13.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:11.565875+0000 osd.6 (osd.6) 299 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:55:13.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:11.566096+0000 osd.6 (osd.6) 300 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:55:13.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:12.325355+0000 mon.a (mon.0) 3657 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-134"}]': finished 2024-02-23T00:55:13.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:12.325436+0000 mon.a (mon.0) 3658 : cluster [DBG] osdmap e764: 8 total, 8 up, 7 in 2024-02-23T00:55:13.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:12.328598+0000 mon.c (mon.2) 473 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-134", "mode": "readproxy"}]: dispatch 2024-02-23T00:55:13.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:12.328872+0000 mon.a (mon.0) 3659 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-134", "mode": "readproxy"}]: dispatch 2024-02-23T00:55:13.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:12.393552+0000 mon.c (mon.2) 474 : audit [DBG] from='client.? 172.21.15.47:0/1461618178' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:55:14.652 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:12.960935+0000 mgr.y (mgr.14517) 651 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:55:14.652 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:13.322628+0000 mon.a (mon.0) 3660 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:55:14.652 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:13.326884+0000 mon.a (mon.0) 3661 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-134", "mode": "readproxy"}]': finished 2024-02-23T00:55:14.652 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:13.326967+0000 mon.a (mon.0) 3662 : cluster [DBG] osdmap e765: 8 total, 8 up, 7 in 2024-02-23T00:55:14.652 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:13.396845+0000 mgr.y (mgr.14517) 652 : cluster [DBG] pgmap v1161: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s; 1/789 objects misplaced (0.127%) 2024-02-23T00:55:14.652 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:12.960935+0000 mgr.y (mgr.14517) 651 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:55:14.652 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:13.322628+0000 mon.a (mon.0) 3660 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:55:14.653 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:13.326884+0000 mon.a (mon.0) 3661 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-134", "mode": "readproxy"}]': finished 2024-02-23T00:55:14.653 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:13.326967+0000 mon.a (mon.0) 3662 : cluster [DBG] osdmap e765: 8 total, 8 up, 7 in 2024-02-23T00:55:14.653 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:13.396845+0000 mgr.y (mgr.14517) 652 : cluster [DBG] pgmap v1161: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s; 1/789 objects misplaced (0.127%) 2024-02-23T00:55:14.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:12.960935+0000 mgr.y (mgr.14517) 651 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:55:14.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:13.322628+0000 mon.a (mon.0) 3660 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:55:14.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:13.326884+0000 mon.a (mon.0) 3661 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-134", "mode": "readproxy"}]': finished 2024-02-23T00:55:14.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:13.326967+0000 mon.a (mon.0) 3662 : cluster [DBG] osdmap e765: 8 total, 8 up, 7 in 2024-02-23T00:55:14.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:13.396845+0000 mgr.y (mgr.14517) 652 : cluster [DBG] pgmap v1161: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s; 1/789 objects misplaced (0.127%) 2024-02-23T00:55:14.827 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:55:14.937 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:55:15.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.270+0000 7f18bd897700 1 -- 172.21.15.47:0/4117186721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18b80fe540 msgr2=0x7f18b80f60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:15.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.270+0000 7f18bd897700 1 --2- 172.21.15.47:0/4117186721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18b80fe540 0x7f18b80f60c0 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 rx=0x7f18ac0047b0 tx=0x7f18ac009cb0).stop 2024-02-23T00:55:15.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.271+0000 7f18bd897700 1 -- 172.21.15.47:0/4117186721 shutdown_connections 2024-02-23T00:55:15.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.271+0000 7f18bd897700 1 --2- 172.21.15.47:0/4117186721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18b80f67a0 0x7f18b8104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:15.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.271+0000 7f18bd897700 1 --2- 172.21.15.47:0/4117186721 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f18b80fdbe0 0x7f18b80fdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:15.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.271+0000 7f18bd897700 1 --2- 172.21.15.47:0/4117186721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18b80fe540 0x7f18b80f60c0 unknown :-1 s=CLOSED pgs=2636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:15.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.271+0000 7f18bd897700 1 -- 172.21.15.47:0/4117186721 >> 172.21.15.47:0/4117186721 conn(0x7f18b806ffc0 msgr2=0x7f18b80f43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:15.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.271+0000 7f18bd897700 1 -- 172.21.15.47:0/4117186721 shutdown_connections 2024-02-23T00:55:15.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.271+0000 7f18bd897700 1 -- 172.21.15.47:0/4117186721 wait complete. 2024-02-23T00:55:15.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.271+0000 7f18bd897700 1 librados: starting msgr at 2024-02-23T00:55:15.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.271+0000 7f18bd897700 1 librados: starting objecter 2024-02-23T00:55:15.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.271+0000 7f18bd897700 1 Processor -- start 2024-02-23T00:55:15.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.271+0000 7f18bd897700 1 -- start start 2024-02-23T00:55:15.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.271+0000 7f18bd897700 1 librados: setting wanted keys 2024-02-23T00:55:15.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.271+0000 7f18bd897700 1 librados: calling monclient init 2024-02-23T00:55:15.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.272+0000 7f18bd897700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f18b80f67a0 0x7f18b80f7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:15.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.272+0000 7f18bd897700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18b80fdbe0 0x7f18b80f7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:15.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.272+0000 7f18bd897700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18b80fe540 0x7f18b80f7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:15.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.272+0000 7f18bd897700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b81164b0 con 0x7f18b80fe540 2024-02-23T00:55:15.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.272+0000 7f18bd897700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b8116620 con 0x7f18b80fdbe0 2024-02-23T00:55:15.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.272+0000 7f18bd897700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b81167e0 con 0x7f18b80f67a0 2024-02-23T00:55:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.272+0000 7f18b6ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f18b80f67a0 0x7f18b80f7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.272+0000 7f18b77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18b80fe540 0x7f18b80f7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.272+0000 7f18b6ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f18b80f67a0 0x7f18b80f7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:60674/0 (socket says 172.21.15.47:60674) 2024-02-23T00:55:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.272+0000 7f18b6ffd700 1 -- 172.21.15.47:0/1270239604 learned_addr learned my addr 172.21.15.47:0/1270239604 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:55:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.272+0000 7f18b67fc700 1 --2- 172.21.15.47:0/1270239604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18b80fdbe0 0x7f18b80f7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.272+0000 7f18b6ffd700 1 -- 172.21.15.47:0/1270239604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18b80fdbe0 msgr2=0x7f18b80f7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.272+0000 7f18b6ffd700 1 --2- 172.21.15.47:0/1270239604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18b80fdbe0 0x7f18b80f7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.272+0000 7f18b6ffd700 1 -- 172.21.15.47:0/1270239604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18b80fe540 msgr2=0x7f18b80f7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.272+0000 7f18b6ffd700 1 --2- 172.21.15.47:0/1270239604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18b80fe540 0x7f18b80f7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.272+0000 7f18b6ffd700 1 -- 172.21.15.47:0/1270239604 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a8005040 con 0x7f18b80f67a0 2024-02-23T00:55:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.272+0000 7f18b77fe700 1 --2- 172.21.15.47:0/1270239604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18b80fe540 0x7f18b80f7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:55:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.272+0000 7f18b6ffd700 1 --2- 172.21.15.47:0/1270239604 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f18b80f67a0 0x7f18b80f7390 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 rx=0x7f18a8009400 tx=0x7f18a8009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.272+0000 7f189ffff700 10 client.?.objecter ms_handle_connect 0x7f18b80f67a0 2024-02-23T00:55:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.272+0000 7f189ffff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:55:15.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.272+0000 7f189ffff700 1 -- 172.21.15.47:0/1270239604 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a8007a40 con 0x7f18b80f67a0 2024-02-23T00:55:15.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.273+0000 7f189ffff700 1 -- 172.21.15.47:0/1270239604 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f18a800f040 con 0x7f18b80f67a0 2024-02-23T00:55:15.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.273+0000 7f18bd897700 1 -- 172.21.15.47:0/1270239604 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18ac005040 con 0x7f18b80f67a0 2024-02-23T00:55:15.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.273+0000 7f189ffff700 1 -- 172.21.15.47:0/1270239604 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a8014900 con 0x7f18b80f67a0 2024-02-23T00:55:15.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.273+0000 7f18bd897700 10 client.42098.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:55:15.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.273+0000 7f18bd897700 1 -- 172.21.15.47:0/1270239604 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b8116ef0 con 0x7f18b80f67a0 2024-02-23T00:55:15.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.273+0000 7f18bd897700 1 librados: init done 2024-02-23T00:55:15.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.273+0000 7f18bd897700 1 -- 172.21.15.47:0/1270239604 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b805f9b0 con 0x7f18b80f67a0 2024-02-23T00:55:15.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.274+0000 7f189ffff700 1 -- 172.21.15.47:0/1270239604 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f18a8014d30 con 0x7f18b80f67a0 2024-02-23T00:55:15.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.274+0000 7f189ffff700 1 --2- 172.21.15.47:0/1270239604 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f18a0062cc0 0x7f18a0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:15.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.274+0000 7f18b67fc700 1 --2- 172.21.15.47:0/1270239604 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f18a0062cc0 0x7f18a0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:15.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.274+0000 7f189ffff700 1 -- 172.21.15.47:0/1270239604 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(765..765 src has 621..765) v4 ==== 9781+0+0 (secure 0 0 0) 0x7f18a807d3a0 con 0x7f18b80f67a0 2024-02-23T00:55:15.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.274+0000 7f189ffff700 10 client.42098.objecter ms_dispatch 0x7f18b8001550 osd_map(765..765 src has 621..765) v4 2024-02-23T00:55:15.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.274+0000 7f189ffff700 3 client.42098.objecter handle_osd_map got epochs [765,765] > 0 2024-02-23T00:55:15.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.274+0000 7f189ffff700 3 client.42098.objecter handle_osd_map decoding full epoch 765 2024-02-23T00:55:15.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.275+0000 7f189ffff700 20 client.42098.objecter dump_active .. 0 homeless 2024-02-23T00:55:15.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.275+0000 7f189ffff700 10 client.42098.objecter ms_handle_connect 0x7f18a0062cc0 2024-02-23T00:55:15.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.275+0000 7f18b67fc700 1 --2- 172.21.15.47:0/1270239604 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f18a0062cc0 0x7f18a0065180 secure :-1 s=READY pgs=3200 cs=0 l=1 rev1=1 rx=0x7f18ac009bc0 tx=0x7f18ac009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:15.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.279+0000 7f189ffff700 1 -- 172.21.15.47:0/1270239604 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f18a8049f60 con 0x7f18b80f67a0 2024-02-23T00:55:15.406 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:55:15.409 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:15.409 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:55:15.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:15.488+0000 7f18bd897700 1 -- 172.21.15.47:0/1270239604 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num", "val": "42"} v 0) v1 -- 0x7f18b8106e90 con 0x7f18b80f67a0 2024-02-23T00:55:15.510 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:55:15.543 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:15.628 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:15.306385+0000 mon.a (mon.0) 3663 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:55:15.628 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:15.311194+0000 mon.a (mon.0) 3664 : cluster [DBG] osdmap e766: 8 total, 8 up, 7 in 2024-02-23T00:55:15.629 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:15.306385+0000 mon.a (mon.0) 3663 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:55:15.629 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:15.311194+0000 mon.a (mon.0) 3664 : cluster [DBG] osdmap e766: 8 total, 8 up, 7 in 2024-02-23T00:55:15.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:15.306385+0000 mon.a (mon.0) 3663 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:55:15.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:15.311194+0000 mon.a (mon.0) 3664 : cluster [DBG] osdmap e766: 8 total, 8 up, 7 in 2024-02-23T00:55:15.976 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:55:16.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:16.343+0000 7f189ffff700 1 -- 172.21.15.47:0/1270239604 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num", "val": "42"}]=0 set pool 5 pg_num to 42 v767) v1 ==== 144+0+0 (secure 0 0 0) 0x7f18a804c800 con 0x7f18b80f67a0 2024-02-23T00:55:16.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:16.346+0000 7f18bd897700 10 librados: watch_flush enter 2024-02-23T00:55:16.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:16.346+0000 7f18bd897700 10 librados: watch_flush exit 2024-02-23T00:55:16.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:16.346+0000 7f18bd897700 20 client.42098.objecter shutdown clearing up homeless session... 2024-02-23T00:55:16.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:16.346+0000 7f18bd897700 10 client.42098.objecter successfully canceled tick 2024-02-23T00:55:16.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:16.346+0000 7f18bd897700 1 -- 172.21.15.47:0/1270239604 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f18a0062cc0 msgr2=0x7f18a0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:16.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:16.346+0000 7f18bd897700 1 --2- 172.21.15.47:0/1270239604 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f18a0062cc0 0x7f18a0065180 secure :-1 s=READY pgs=3200 cs=0 l=1 rev1=1 rx=0x7f18ac009bc0 tx=0x7f18ac009cb0).stop 2024-02-23T00:55:16.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:16.347+0000 7f18bd897700 1 -- 172.21.15.47:0/1270239604 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f18b80f67a0 msgr2=0x7f18b80f7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:16.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:16.347+0000 7f18bd897700 1 --2- 172.21.15.47:0/1270239604 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f18b80f67a0 0x7f18b80f7390 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 rx=0x7f18a8009400 tx=0x7f18a8009cb0).stop 2024-02-23T00:55:16.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:16.347+0000 7f18bd897700 1 -- 172.21.15.47:0/1270239604 shutdown_connections 2024-02-23T00:55:16.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:16.347+0000 7f18bd897700 1 --2- 172.21.15.47:0/1270239604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18b80fe540 0x7f18b80f7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:16.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:16.347+0000 7f18bd897700 1 --2- 172.21.15.47:0/1270239604 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f18a0062cc0 0x7f18a0065180 unknown :-1 s=CLOSED pgs=3200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:16.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:16.347+0000 7f18bd897700 1 --2- 172.21.15.47:0/1270239604 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f18b80f67a0 0x7f18b80f7390 unknown :-1 s=CLOSED pgs=2810 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:16.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:16.347+0000 7f18bd897700 1 --2- 172.21.15.47:0/1270239604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18b80fdbe0 0x7f18b80f7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:16.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:16.347+0000 7f18bd897700 1 -- 172.21.15.47:0/1270239604 >> 172.21.15.47:0/1270239604 conn(0x7f18b806ffc0 msgr2=0x7f18b80fc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:16.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:16.347+0000 7f18bd897700 1 -- 172.21.15.47:0/1270239604 shutdown_connections 2024-02-23T00:55:16.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:16.347+0000 7f18bd897700 1 -- 172.21.15.47:0/1270239604 wait complete. 2024-02-23T00:55:16.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:16.347+0000 7f18bd897700 1 librados: shutdown 2024-02-23T00:55:16.351 INFO:teuthology.orchestra.run.smithi047.stderr:set pool 5 pg_num to 42 2024-02-23T00:55:16.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:15.397310+0000 mgr.y (mgr.14517) 653 : cluster [DBG] pgmap v1163: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 251 B/s wr, 1 op/s; 1/789 objects misplaced (0.127%) 2024-02-23T00:55:16.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:15.490402+0000 mon.c (mon.2) 475 : audit [INF] from='client.? 172.21.15.47:0/1270239604' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num", "val": "42"}]: dispatch 2024-02-23T00:55:16.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:15.490749+0000 mon.a (mon.0) 3665 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num", "val": "42"}]: dispatch 2024-02-23T00:55:16.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:15.397310+0000 mgr.y (mgr.14517) 653 : cluster [DBG] pgmap v1163: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 251 B/s wr, 1 op/s; 1/789 objects misplaced (0.127%) 2024-02-23T00:55:16.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:15.490402+0000 mon.c (mon.2) 475 : audit [INF] from='client.? 172.21.15.47:0/1270239604' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num", "val": "42"}]: dispatch 2024-02-23T00:55:16.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:15.490749+0000 mon.a (mon.0) 3665 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num", "val": "42"}]: dispatch 2024-02-23T00:55:16.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:15.397310+0000 mgr.y (mgr.14517) 653 : cluster [DBG] pgmap v1163: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 251 B/s wr, 1 op/s; 1/789 objects misplaced (0.127%) 2024-02-23T00:55:16.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:15.490402+0000 mon.c (mon.2) 475 : audit [INF] from='client.? 172.21.15.47:0/1270239604' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num", "val": "42"}]: dispatch 2024-02-23T00:55:16.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:15.490749+0000 mon.a (mon.0) 3665 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num", "val": "42"}]: dispatch 2024-02-23T00:55:16.922 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:55:16.925 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:16.925 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:55:17.026 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:55:17.058 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:17.080 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:55:17.082 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:17.083 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:55:17.184 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:55:17.213 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:17.232 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:55:17.234 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:17.235 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:55:17.290 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:55:17.290 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:55:17.336 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:55:17.370 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:17.443 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:55:17.446 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:16.344209+0000 mon.a (mon.0) 3666 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num", "val": "42"}]': finished 2024-02-23T00:55:17.446 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:16.344322+0000 mon.a (mon.0) 3667 : cluster [DBG] osdmap e767: 8 total, 8 up, 7 in 2024-02-23T00:55:17.447 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:16.414665+0000 osd.7 (osd.7) 345 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:55:17.447 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:16.414967+0000 osd.7 (osd.7) 346 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:55:17.447 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:16.856928+0000 osd.1 (osd.1) 306 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:55:17.447 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:16.857064+0000 osd.1 (osd.1) 307 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:55:17.447 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:16.344209+0000 mon.a (mon.0) 3666 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num", "val": "42"}]': finished 2024-02-23T00:55:17.447 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:16.344322+0000 mon.a (mon.0) 3667 : cluster [DBG] osdmap e767: 8 total, 8 up, 7 in 2024-02-23T00:55:17.447 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:16.414665+0000 osd.7 (osd.7) 345 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:55:17.447 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:16.414967+0000 osd.7 (osd.7) 346 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:55:17.447 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:16.856928+0000 osd.1 (osd.1) 306 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:55:17.447 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:16.857064+0000 osd.1 (osd.1) 307 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:55:17.448 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:17.448 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:55:17.549 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:55:17.578 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:17.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:16.344209+0000 mon.a (mon.0) 3666 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num", "val": "42"}]': finished 2024-02-23T00:55:17.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:16.344322+0000 mon.a (mon.0) 3667 : cluster [DBG] osdmap e767: 8 total, 8 up, 7 in 2024-02-23T00:55:17.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:16.414665+0000 osd.7 (osd.7) 345 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:55:17.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:16.414967+0000 osd.7 (osd.7) 346 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:55:17.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:16.856928+0000 osd.1 (osd.1) 306 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:55:17.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:16.857064+0000 osd.1 (osd.1) 307 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:55:18.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:17.349738+0000 mon.a (mon.0) 3668 : cluster [DBG] osdmap e768: 8 total, 8 up, 7 in 2024-02-23T00:55:18.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:17.397684+0000 mgr.y (mgr.14517) 654 : cluster [DBG] pgmap v1166: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1 op/s; 1/789 objects misplaced (0.127%) 2024-02-23T00:55:18.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:17.398510+0000 mon.a (mon.0) 3669 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "42"}]: dispatch 2024-02-23T00:55:18.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:17.349738+0000 mon.a (mon.0) 3668 : cluster [DBG] osdmap e768: 8 total, 8 up, 7 in 2024-02-23T00:55:18.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:17.397684+0000 mgr.y (mgr.14517) 654 : cluster [DBG] pgmap v1166: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1 op/s; 1/789 objects misplaced (0.127%) 2024-02-23T00:55:18.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:17.398510+0000 mon.a (mon.0) 3669 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "42"}]: dispatch 2024-02-23T00:55:18.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:17.349738+0000 mon.a (mon.0) 3668 : cluster [DBG] osdmap e768: 8 total, 8 up, 7 in 2024-02-23T00:55:18.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:17.397684+0000 mgr.y (mgr.14517) 654 : cluster [DBG] pgmap v1166: 184 pgs: 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1 op/s; 1/789 objects misplaced (0.127%) 2024-02-23T00:55:18.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:17.398510+0000 mon.a (mon.0) 3669 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "42"}]: dispatch 2024-02-23T00:55:18.927 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:55:18.930 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:18.931 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:55:19.032 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:55:19.063 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:19.390 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.4 dump_ops_in_flight 2024-02-23T00:55:19.606 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:55:19.622 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:18.355249+0000 mon.a (mon.0) 3670 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "42"}]': finished 2024-02-23T00:55:19.622 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:18.355326+0000 mon.a (mon.0) 3671 : cluster [DBG] osdmap e769: 8 total, 8 up, 7 in 2024-02-23T00:55:19.622 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:19.017687+0000 mon.a (mon.0) 3672 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:55:19.622 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:19.259478+0000 mon.a (mon.0) 3673 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:55:19.622 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:18.355249+0000 mon.a (mon.0) 3670 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "42"}]': finished 2024-02-23T00:55:19.622 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:18.355326+0000 mon.a (mon.0) 3671 : cluster [DBG] osdmap e769: 8 total, 8 up, 7 in 2024-02-23T00:55:19.622 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:19.017687+0000 mon.a (mon.0) 3672 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:55:19.622 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:19.259478+0000 mon.a (mon.0) 3673 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:55:19.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:18.355249+0000 mon.a (mon.0) 3670 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pg_num_actual", "val": "42"}]': finished 2024-02-23T00:55:19.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:18.355326+0000 mon.a (mon.0) 3671 : cluster [DBG] osdmap e769: 8 total, 8 up, 7 in 2024-02-23T00:55:19.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:19.017687+0000 mon.a (mon.0) 3672 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:55:19.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:19.259478+0000 mon.a (mon.0) 3673 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:55:19.791 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:55:19.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.897+0000 7fd2d3bf2700 1 -- 172.21.15.47:0/2952665909 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2cc0f9610 msgr2=0x7fd2cc101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:19.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.897+0000 7fd2d3bf2700 1 --2- 172.21.15.47:0/2952665909 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2cc0f9610 0x7fd2cc101d20 secure :-1 s=READY pgs=2811 cs=0 l=1 rev1=1 rx=0x7fd2c8004870 tx=0x7fd2c8009cb0).stop 2024-02-23T00:55:19.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.897+0000 7fd2d3bf2700 1 -- 172.21.15.47:0/2952665909 shutdown_connections 2024-02-23T00:55:19.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.897+0000 7fd2d3bf2700 1 --2- 172.21.15.47:0/2952665909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2cc102210 0x7fd2cc104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:55:19.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.897+0000 7fd2d3bf2700 1 --2- 172.21.15.47:0/2952665909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2cc0f8cb0 0x7fd2cc0f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:19.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.897+0000 7fd2d3bf2700 1 --2- 172.21.15.47:0/2952665909 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2cc0f9610 0x7fd2cc101d20 unknown :-1 s=CLOSED pgs=2811 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:19.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.897+0000 7fd2d3bf2700 1 -- 172.21.15.47:0/2952665909 >> 172.21.15.47:0/2952665909 conn(0x7fd2cc0f2820 msgr2=0x7fd2cc0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:19.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.897+0000 7fd2d3bf2700 1 -- 172.21.15.47:0/2952665909 shutdown_connections 2024-02-23T00:55:19.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.897+0000 7fd2d3bf2700 1 -- 172.21.15.47:0/2952665909 wait complete. 2024-02-23T00:55:19.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.898+0000 7fd2d3bf2700 1 librados: starting msgr at 2024-02-23T00:55:19.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.898+0000 7fd2d3bf2700 1 librados: starting objecter 2024-02-23T00:55:19.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.898+0000 7fd2d3bf2700 1 Processor -- start 2024-02-23T00:55:19.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.898+0000 7fd2d3bf2700 1 -- start start 2024-02-23T00:55:19.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.898+0000 7fd2d3bf2700 1 librados: setting wanted keys 2024-02-23T00:55:19.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.898+0000 7fd2d3bf2700 1 librados: calling monclient init 2024-02-23T00:55:19.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.898+0000 7fd2d3bf2700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2cc0f8cb0 0x7fd2cc0f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:19.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.898+0000 7fd2d3bf2700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2cc0f9610 0x7fd2cc0f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:19.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.898+0000 7fd2d3bf2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2cc102210 0x7fd2cc0f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:19.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.898+0000 7fd2d3bf2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc1166d0 con 0x7fd2cc102210 2024-02-23T00:55:19.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.898+0000 7fd2d3bf2700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc116840 con 0x7fd2cc0f9610 2024-02-23T00:55:19.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.898+0000 7fd2d3bf2700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc116a00 con 0x7fd2cc0f8cb0 2024-02-23T00:55:19.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.898+0000 7fd2d198e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2cc0f8cb0 0x7fd2cc0f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:19.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.898+0000 7fd2d218f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2cc102210 0x7fd2cc0f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:19.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.898+0000 7fd2d198e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2cc0f8cb0 0x7fd2cc0f6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:43116/0 (socket says 172.21.15.47:43116) 2024-02-23T00:55:19.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.898+0000 7fd2d198e700 1 -- 172.21.15.47:0/2609189215 learned_addr learned my addr 172.21.15.47:0/2609189215 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:55:19.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.898+0000 7fd2d118d700 1 --2- 172.21.15.47:0/2609189215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2cc0f9610 0x7fd2cc0f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:19.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.898+0000 7fd2d198e700 1 -- 172.21.15.47:0/2609189215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2cc0f9610 msgr2=0x7fd2cc0f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:19.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.898+0000 7fd2d198e700 1 --2- 172.21.15.47:0/2609189215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2cc0f9610 0x7fd2cc0f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:19.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.898+0000 7fd2d198e700 1 -- 172.21.15.47:0/2609189215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2cc102210 msgr2=0x7fd2cc0f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:19.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.898+0000 7fd2d198e700 1 --2- 172.21.15.47:0/2609189215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2cc102210 0x7fd2cc0f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:19.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.899+0000 7fd2d198e700 1 -- 172.21.15.47:0/2609189215 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2bc005040 con 0x7fd2cc0f8cb0 2024-02-23T00:55:19.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.899+0000 7fd2d218f700 1 --2- 172.21.15.47:0/2609189215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2cc102210 0x7fd2cc0f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:55:19.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.899+0000 7fd2d198e700 1 --2- 172.21.15.47:0/2609189215 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2cc0f8cb0 0x7fd2cc0f6c60 secure :-1 s=READY pgs=2812 cs=0 l=1 rev1=1 rx=0x7fd2bc0048f0 tx=0x7fd2bc009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:19.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.899+0000 7fd2c2ffd700 10 client.?.objecter ms_handle_connect 0x7fd2cc0f8cb0 2024-02-23T00:55:19.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.899+0000 7fd2c2ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:55:19.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.899+0000 7fd2c2ffd700 1 -- 172.21.15.47:0/2609189215 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc007b00 con 0x7fd2cc0f8cb0 2024-02-23T00:55:19.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.899+0000 7fd2d3bf2700 1 -- 172.21.15.47:0/2609189215 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c8005040 con 0x7fd2cc0f8cb0 2024-02-23T00:55:19.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.899+0000 7fd2c2ffd700 1 -- 172.21.15.47:0/2609189215 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fd2bc00f040 con 0x7fd2cc0f8cb0 2024-02-23T00:55:19.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.899+0000 7fd2c2ffd700 1 -- 172.21.15.47:0/2609189215 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc014820 con 0x7fd2cc0f8cb0 2024-02-23T00:55:19.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.899+0000 7fd2d3bf2700 10 client.42104.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:55:19.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.899+0000 7fd2d3bf2700 1 -- 172.21.15.47:0/2609189215 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2cc117060 con 0x7fd2cc0f8cb0 2024-02-23T00:55:19.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.899+0000 7fd2d3bf2700 1 librados: init done 2024-02-23T00:55:19.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.900+0000 7fd2d3bf2700 1 -- 172.21.15.47:0/2609189215 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2cc05fa60 con 0x7fd2cc0f8cb0 2024-02-23T00:55:19.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.900+0000 7fd2c2ffd700 1 -- 172.21.15.47:0/2609189215 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fd2bc014c50 con 0x7fd2cc0f8cb0 2024-02-23T00:55:19.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.901+0000 7fd2c2ffd700 1 --2- 172.21.15.47:0/2609189215 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd2b8062cc0 0x7fd2b8065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:19.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.901+0000 7fd2c2ffd700 1 -- 172.21.15.47:0/2609189215 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(770..770 src has 621..770) v4 ==== 9781+0+0 (secure 0 0 0) 0x7fd2bc07d390 con 0x7fd2cc0f8cb0 2024-02-23T00:55:19.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.901+0000 7fd2c2ffd700 10 client.42104.objecter ms_dispatch 0x7fd2cc001550 osd_map(770..770 src has 621..770) v4 2024-02-23T00:55:19.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.901+0000 7fd2c2ffd700 3 client.42104.objecter handle_osd_map got epochs [770,770] > 0 2024-02-23T00:55:19.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.901+0000 7fd2c2ffd700 3 client.42104.objecter handle_osd_map decoding full epoch 770 2024-02-23T00:55:19.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.901+0000 7fd2d118d700 1 --2- 172.21.15.47:0/2609189215 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd2b8062cc0 0x7fd2b8065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:19.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.901+0000 7fd2c2ffd700 20 client.42104.objecter dump_active .. 0 homeless 2024-02-23T00:55:19.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.901+0000 7fd2c2ffd700 10 client.42104.objecter ms_handle_connect 0x7fd2b8062cc0 2024-02-23T00:55:19.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.901+0000 7fd2d118d700 1 --2- 172.21.15.47:0/2609189215 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd2b8062cc0 0x7fd2b8065180 secure :-1 s=READY pgs=3201 cs=0 l=1 rev1=1 rx=0x7fd2c8004ee0 tx=0x7fd2c800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:19.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:19.905+0000 7fd2c2ffd700 1 -- 172.21.15.47:0/2609189215 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fd2bc049f60 con 0x7fd2cc0f8cb0 2024-02-23T00:55:20.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:20.087+0000 7fd2d3bf2700 1 -- 172.21.15.47:0/2609189215 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7fd2cc05ee80 con 0x7fd2cc0f8cb0 2024-02-23T00:55:20.216 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:55:20.218 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:20.219 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:55:20.320 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:55:20.350 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:20.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:20.376+0000 7fd2c2ffd700 1 -- 172.21.15.47:0/2609189215 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v771) v1 ==== 91+0+0 (secure 0 0 0) 0x7fd2bc04c7f0 con 0x7fd2cc0f8cb0 2024-02-23T00:55:20.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:20.379+0000 7fd2d3bf2700 10 librados: watch_flush enter 2024-02-23T00:55:20.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:20.379+0000 7fd2d3bf2700 10 librados: watch_flush exit 2024-02-23T00:55:20.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:20.379+0000 7fd2d3bf2700 20 client.42104.objecter shutdown clearing up homeless session... 2024-02-23T00:55:20.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:20.379+0000 7fd2d3bf2700 10 client.42104.objecter successfully canceled tick 2024-02-23T00:55:20.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:20.379+0000 7fd2d3bf2700 1 -- 172.21.15.47:0/2609189215 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd2b8062cc0 msgr2=0x7fd2b8065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:20.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:20.379+0000 7fd2d3bf2700 1 --2- 172.21.15.47:0/2609189215 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd2b8062cc0 0x7fd2b8065180 secure :-1 s=READY pgs=3201 cs=0 l=1 rev1=1 rx=0x7fd2c8004ee0 tx=0x7fd2c800b040).stop 2024-02-23T00:55:20.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:20.379+0000 7fd2d3bf2700 1 -- 172.21.15.47:0/2609189215 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2cc0f8cb0 msgr2=0x7fd2cc0f6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:20.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:20.379+0000 7fd2d3bf2700 1 --2- 172.21.15.47:0/2609189215 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2cc0f8cb0 0x7fd2cc0f6c60 secure :-1 s=READY pgs=2812 cs=0 l=1 rev1=1 rx=0x7fd2bc0048f0 tx=0x7fd2bc009cb0).stop 2024-02-23T00:55:20.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:20.379+0000 7fd2d3bf2700 1 -- 172.21.15.47:0/2609189215 shutdown_connections 2024-02-23T00:55:20.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:20.379+0000 7fd2d3bf2700 1 --2- 172.21.15.47:0/2609189215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2cc102210 0x7fd2cc0f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:20.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:20.379+0000 7fd2d3bf2700 1 --2- 172.21.15.47:0/2609189215 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd2b8062cc0 0x7fd2b8065180 unknown :-1 s=CLOSED pgs=3201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:20.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:20.379+0000 7fd2d3bf2700 1 --2- 172.21.15.47:0/2609189215 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2cc0f8cb0 0x7fd2cc0f6c60 unknown :-1 s=CLOSED pgs=2812 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:20.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:20.379+0000 7fd2d3bf2700 1 --2- 172.21.15.47:0/2609189215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2cc0f9610 0x7fd2cc0f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:20.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:20.379+0000 7fd2d3bf2700 1 -- 172.21.15.47:0/2609189215 >> 172.21.15.47:0/2609189215 conn(0x7fd2cc0f2820 msgr2=0x7fd2cc0f34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:20.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:20.379+0000 7fd2d3bf2700 1 -- 172.21.15.47:0/2609189215 shutdown_connections 2024-02-23T00:55:20.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:20.380+0000 7fd2d3bf2700 1 -- 172.21.15.47:0/2609189215 wait complete. 2024-02-23T00:55:20.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:20.380+0000 7fd2d3bf2700 1 librados: shutdown 2024-02-23T00:55:20.384 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is unset 2024-02-23T00:55:20.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:19.356585+0000 mon.a (mon.0) 3674 : cluster [DBG] osdmap e770: 8 total, 8 up, 7 in 2024-02-23T00:55:20.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:19.398219+0000 mgr.y (mgr.14517) 655 : cluster [DBG] pgmap v1169: 194 pgs: 10 unknown, 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1 op/s; 1/789 objects misplaced (0.127%) 2024-02-23T00:55:20.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:19.399180+0000 mon.a (mon.0) 3675 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num_actual", "val": "42"}]: dispatch 2024-02-23T00:55:20.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:19.640136+0000 mon.a (mon.0) 3676 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:55:20.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:19.640792+0000 mon.a (mon.0) 3677 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:55:20.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:19.644983+0000 mon.a (mon.0) 3678 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:55:20.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:20.088946+0000 mon.c (mon.2) 476 : audit [INF] from='client.? 172.21.15.47:0/2609189215' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:55:20.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:20.089247+0000 mon.a (mon.0) 3679 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:55:20.458 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:19.356585+0000 mon.a (mon.0) 3674 : cluster [DBG] osdmap e770: 8 total, 8 up, 7 in 2024-02-23T00:55:20.458 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:19.398219+0000 mgr.y (mgr.14517) 655 : cluster [DBG] pgmap v1169: 194 pgs: 10 unknown, 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1 op/s; 1/789 objects misplaced (0.127%) 2024-02-23T00:55:20.458 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:19.399180+0000 mon.a (mon.0) 3675 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num_actual", "val": "42"}]: dispatch 2024-02-23T00:55:20.458 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:19.640136+0000 mon.a (mon.0) 3676 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:55:20.458 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:19.640792+0000 mon.a (mon.0) 3677 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:55:20.458 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:19.644983+0000 mon.a (mon.0) 3678 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:55:20.458 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:20.088946+0000 mon.c (mon.2) 476 : audit [INF] from='client.? 172.21.15.47:0/2609189215' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:55:20.458 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:20.089247+0000 mon.a (mon.0) 3679 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:55:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:19.356585+0000 mon.a (mon.0) 3674 : cluster [DBG] osdmap e770: 8 total, 8 up, 7 in 2024-02-23T00:55:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:19.398219+0000 mgr.y (mgr.14517) 655 : cluster [DBG] pgmap v1169: 194 pgs: 10 unknown, 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1 op/s; 1/789 objects misplaced (0.127%) 2024-02-23T00:55:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:19.399180+0000 mon.a (mon.0) 3675 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num_actual", "val": "42"}]: dispatch 2024-02-23T00:55:20.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:19.640136+0000 mon.a (mon.0) 3676 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:55:20.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:19.640792+0000 mon.a (mon.0) 3677 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:55:20.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:19.644983+0000 mon.a (mon.0) 3678 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:55:20.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:20.088946+0000 mon.c (mon.2) 476 : audit [INF] from='client.? 172.21.15.47:0/2609189215' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:55:20.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:20.089247+0000 mon.a (mon.0) 3679 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:55:20.793 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:55:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:55:20] "GET /metrics HTTP/1.1" 200 191881 "" "Prometheus/2.33.4" 2024-02-23T00:55:21.100 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:55:21.103 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:21.104 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:55:21.205 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:55:21.237 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:21.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:20.366598+0000 mon.a (mon.0) 3680 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:55:21.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:20.375036+0000 mon.a (mon.0) 3681 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num_actual", "val": "42"}]': finished 2024-02-23T00:55:21.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:20.375103+0000 mon.a (mon.0) 3682 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:55:21.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:20.375229+0000 mon.a (mon.0) 3683 : cluster [DBG] osdmap e771: 8 total, 8 up, 7 in 2024-02-23T00:55:21.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:21.214655+0000 osd.5 (osd.5) 324 : cluster [DBG] 5.1b scrub starts 2024-02-23T00:55:21.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:21.215424+0000 osd.5 (osd.5) 325 : cluster [DBG] 5.1b scrub ok 2024-02-23T00:55:21.389 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:20.366598+0000 mon.a (mon.0) 3680 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:55:21.389 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:20.375036+0000 mon.a (mon.0) 3681 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num_actual", "val": "42"}]': finished 2024-02-23T00:55:21.389 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:20.375103+0000 mon.a (mon.0) 3682 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:55:21.389 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:20.375229+0000 mon.a (mon.0) 3683 : cluster [DBG] osdmap e771: 8 total, 8 up, 7 in 2024-02-23T00:55:21.389 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:21.214655+0000 osd.5 (osd.5) 324 : cluster [DBG] 5.1b scrub starts 2024-02-23T00:55:21.389 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:21.215424+0000 osd.5 (osd.5) 325 : cluster [DBG] 5.1b scrub ok 2024-02-23T00:55:21.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:20.366598+0000 mon.a (mon.0) 3680 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:55:21.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:20.375036+0000 mon.a (mon.0) 3681 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num_actual", "val": "42"}]': finished 2024-02-23T00:55:21.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:20.375103+0000 mon.a (mon.0) 3682 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:55:21.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:20.375229+0000 mon.a (mon.0) 3683 : cluster [DBG] osdmap e771: 8 total, 8 up, 7 in 2024-02-23T00:55:21.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:21.214655+0000 osd.5 (osd.5) 324 : cluster [DBG] 5.1b scrub starts 2024-02-23T00:55:21.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:21.215424+0000 osd.5 (osd.5) 325 : cluster [DBG] 5.1b scrub ok 2024-02-23T00:55:21.752 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:55:21.752 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:55:22.087 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:55:22.091 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:22.091 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:55:22.192 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:55:22.225 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:22.423 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:55:22.427 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:22.428 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:55:22.443 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.4 dump_blocked_ops 2024-02-23T00:55:22.529 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:55:22.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.540+0000 7f78786d7700 1 -- 172.21.15.47:0/1937937629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78700fe510 msgr2=0x7f78700f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:22.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.540+0000 7f78786d7700 1 --2- 172.21.15.47:0/1937937629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78700fe510 0x7f78700f7d40 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 rx=0x7f7860004870 tx=0x7f7860009cb0).stop 2024-02-23T00:55:22.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.540+0000 7f78786d7700 1 -- 172.21.15.47:0/1937937629 shutdown_connections 2024-02-23T00:55:22.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.540+0000 7f78786d7700 1 --2- 172.21.15.47:0/1937937629 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f78700f8420 0x7f7870104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:22.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.540+0000 7f78786d7700 1 --2- 172.21.15.47:0/1937937629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78700fdbb0 0x7f78700fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:22.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.540+0000 7f78786d7700 1 --2- 172.21.15.47:0/1937937629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78700fe510 0x7f78700f7d40 unknown :-1 s=CLOSED pgs=2637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:22.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.540+0000 7f78786d7700 1 -- 172.21.15.47:0/1937937629 >> 172.21.15.47:0/1937937629 conn(0x7f78700f3800 msgr2=0x7f78700f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:22.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.540+0000 7f78786d7700 1 -- 172.21.15.47:0/1937937629 shutdown_connections 2024-02-23T00:55:22.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.540+0000 7f78786d7700 1 -- 172.21.15.47:0/1937937629 wait complete. 2024-02-23T00:55:22.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.541+0000 7f78786d7700 1 librados: starting msgr at 2024-02-23T00:55:22.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.541+0000 7f78786d7700 1 librados: starting objecter 2024-02-23T00:55:22.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.541+0000 7f78786d7700 1 Processor -- start 2024-02-23T00:55:22.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.541+0000 7f78786d7700 1 -- start start 2024-02-23T00:55:22.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.541+0000 7f78786d7700 1 librados: setting wanted keys 2024-02-23T00:55:22.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.541+0000 7f78786d7700 1 librados: calling monclient init 2024-02-23T00:55:22.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.541+0000 7f78786d7700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78700f8420 0x7f787010ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:22.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.541+0000 7f78786d7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78700fdbb0 0x7f787010e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:22.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.541+0000 7f78786d7700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f78700fe510 0x7f7870108070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:22.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.541+0000 7f78786d7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7870108560 con 0x7f78700fdbb0 2024-02-23T00:55:22.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.541+0000 7f78786d7700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78701086d0 con 0x7f78700f8420 2024-02-23T00:55:22.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.541+0000 7f78786d7700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7870108890 con 0x7f78700fe510 2024-02-23T00:55:22.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.541+0000 7f7875c72700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78700fdbb0 0x7f787010e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:22.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.541+0000 7f7875c72700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78700fdbb0 0x7f787010e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40746/0 (socket says 172.21.15.47:40746) 2024-02-23T00:55:22.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.541+0000 7f7875c72700 1 -- 172.21.15.47:0/2610444896 learned_addr learned my addr 172.21.15.47:0/2610444896 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:55:22.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.541+0000 7f7876473700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78700f8420 0x7f787010ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:22.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.541+0000 7f7876c74700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f78700fe510 0x7f7870108070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:22.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.542+0000 7f7875c72700 1 -- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f78700fe510 msgr2=0x7f7870108070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:22.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.542+0000 7f7875c72700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f78700fe510 0x7f7870108070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:22.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.542+0000 7f7875c72700 1 -- 172.21.15.47:0/2610444896 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78700f8420 msgr2=0x7f787010ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:22.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.542+0000 7f7875c72700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78700f8420 0x7f787010ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:22.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.542+0000 7f7875c72700 1 -- 172.21.15.47:0/2610444896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78680097c0 con 0x7f78700fdbb0 2024-02-23T00:55:22.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.542+0000 7f7875c72700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78700fdbb0 0x7f787010e3c0 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 rx=0x7f7860000e50 tx=0x7f7860009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:22.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.542+0000 7f78677fe700 10 client.?.objecter ms_handle_connect 0x7f78700fdbb0 2024-02-23T00:55:22.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.542+0000 7f78677fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:55:22.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.542+0000 7f78677fe700 1 -- 172.21.15.47:0/2610444896 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786000e030 con 0x7f78700fdbb0 2024-02-23T00:55:22.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.542+0000 7f78677fe700 1 -- 172.21.15.47:0/2610444896 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f7860013d70 con 0x7f78700fdbb0 2024-02-23T00:55:22.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.542+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7860005040 con 0x7f78700fdbb0 2024-02-23T00:55:22.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.542+0000 7f78677fe700 1 -- 172.21.15.47:0/2610444896 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7860018070 con 0x7f78700fdbb0 2024-02-23T00:55:22.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.542+0000 7f78786d7700 10 client.31791.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:55:22.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.542+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7870108e00 con 0x7f78700fdbb0 2024-02-23T00:55:22.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.542+0000 7f78786d7700 1 librados: init done 2024-02-23T00:55:22.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.543+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f787005ee80 con 0x7f78700fdbb0 2024-02-23T00:55:22.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.543+0000 7f78677fe700 1 -- 172.21.15.47:0/2610444896 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f7860009240 con 0x7f78700fdbb0 2024-02-23T00:55:22.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.544+0000 7f78677fe700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f785c062cc0 0x7f785c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:22.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.544+0000 7f78677fe700 1 -- 172.21.15.47:0/2610444896 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(772..772 src has 621..772) v4 ==== 9781+0+0 (secure 0 0 0) 0x7f786007e990 con 0x7f78700fdbb0 2024-02-23T00:55:22.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.544+0000 7f78677fe700 10 client.31791.objecter ms_dispatch 0x7f7870001550 osd_map(772..772 src has 621..772) v4 2024-02-23T00:55:22.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.544+0000 7f78677fe700 3 client.31791.objecter handle_osd_map got epochs [772,772] > 0 2024-02-23T00:55:22.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.544+0000 7f78677fe700 3 client.31791.objecter handle_osd_map decoding full epoch 772 2024-02-23T00:55:22.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.544+0000 7f7876473700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f785c062cc0 0x7f785c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:22.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.544+0000 7f78677fe700 20 client.31791.objecter dump_active .. 0 homeless 2024-02-23T00:55:22.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.544+0000 7f78677fe700 1 -- 172.21.15.47:0/2610444896 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v772) v1 ==== 54+0+16 (secure 0 0 0) 0x7f7860083220 con 0x7f78700fdbb0 2024-02-23T00:55:22.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.544+0000 7f78786d7700 10 client.31791.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:22.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.544+0000 7f78786d7700 15 client.31791.objecter _session_command_op_assign -1 1 2024-02-23T00:55:22.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.544+0000 7f78677fe700 10 client.31791.objecter ms_handle_connect 0x7f785c062cc0 2024-02-23T00:55:22.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.544+0000 7f7876473700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f785c062cc0 0x7f785c065180 secure :-1 s=READY pgs=3202 cs=0 l=1 rev1=1 rx=0x7f786c0048b0 tx=0x7f786c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:22.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.544+0000 7f78786d7700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f78700fee20 0x7f78700ff200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:22.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.544+0000 7f78786d7700 20 client.31791.objecter _get_session s=0x7f7870106e60 osd=0 3 2024-02-23T00:55:22.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.544+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f7870106e60 osd=0 3 2024-02-23T00:55:22.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.544+0000 7f78786d7700 20 client.31791.objecter _get_session s=0x7f7870106e60 osd=0 3 2024-02-23T00:55:22.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.544+0000 7f78786d7700 15 client.31791.objecter _session_command_op_remove -1 1 2024-02-23T00:55:22.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.544+0000 7f78786d7700 20 client.31791.objecter get_session s=0x7f7870106e60 osd=0 3 2024-02-23T00:55:22.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.544+0000 7f78786d7700 15 client.31791.objecter _session_command_op_assign 0 1 2024-02-23T00:55:22.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.544+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f7870106e60 osd=0 4 2024-02-23T00:55:22.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.544+0000 7f78786d7700 10 client.31791.objecter _send_command 1 2024-02-23T00:55:22.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.544+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7870109550 con 0x7f78700fee20 2024-02-23T00:55:22.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.544+0000 7f7876c74700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f78700fee20 0x7f78700ff200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:22.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.545+0000 7f78677fe700 10 client.31791.objecter ms_handle_connect 0x7f78700fee20 2024-02-23T00:55:22.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.545+0000 7f7876c74700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f78700fee20 0x7f78700ff200 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:22.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.545+0000 7f78677fe700 1 -- 172.21.15.47:0/2610444896 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7870109550 con 0x7f78700fee20 2024-02-23T00:55:22.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.545+0000 7f78677fe700 10 client.31791.objecter ms_dispatch 0x7f7870001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:55:22.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.545+0000 7f78677fe700 10 client.31791.objecter _finish_command 1 = system:0 2024-02-23T00:55:22.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.545+0000 7f78677fe700 20 client.31791.objecter put_session s=0x7f7870106e60 osd=0 4 2024-02-23T00:55:22.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.545+0000 7f78677fe700 15 client.31791.objecter _session_command_op_remove 0 1 2024-02-23T00:55:22.555 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:22.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:21.105395+0000 osd.2 (osd.2) 171 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:55:22.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:21.105640+0000 osd.2 (osd.2) 172 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:55:22.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:21.221774+0000 osd.4 (osd.4) 319 : cluster [DBG] 4.9 scrub starts 2024-02-23T00:55:22.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:21.223705+0000 osd.4 (osd.4) 320 : cluster [DBG] 4.9 scrub ok 2024-02-23T00:55:22.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:21.375178+0000 mon.a (mon.0) 3684 : cluster [DBG] osdmap e772: 8 total, 8 up, 7 in 2024-02-23T00:55:22.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:21.398819+0000 mgr.y (mgr.14517) 656 : cluster [DBG] pgmap v1172: 194 pgs: 10 unknown, 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1/789 objects misplaced (0.127%) 2024-02-23T00:55:22.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:21.105395+0000 osd.2 (osd.2) 171 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:55:22.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:21.105640+0000 osd.2 (osd.2) 172 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:55:22.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:21.221774+0000 osd.4 (osd.4) 319 : cluster [DBG] 4.9 scrub starts 2024-02-23T00:55:22.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:21.223705+0000 osd.4 (osd.4) 320 : cluster [DBG] 4.9 scrub ok 2024-02-23T00:55:22.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:21.375178+0000 mon.a (mon.0) 3684 : cluster [DBG] osdmap e772: 8 total, 8 up, 7 in 2024-02-23T00:55:22.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:21.398819+0000 mgr.y (mgr.14517) 656 : cluster [DBG] pgmap v1172: 194 pgs: 10 unknown, 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1/789 objects misplaced (0.127%) 2024-02-23T00:55:22.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.561+0000 7f78786d7700 10 client.31791.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:55:22.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.561+0000 7f78786d7700 15 client.31791.objecter _session_command_op_assign -1 2 2024-02-23T00:55:22.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.561+0000 7f78786d7700 20 client.31791.objecter _get_session s=0x7f7870106e60 osd=0 3 2024-02-23T00:55:22.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.561+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f7870106e60 osd=0 3 2024-02-23T00:55:22.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.561+0000 7f78786d7700 20 client.31791.objecter _get_session s=0x7f7870106e60 osd=0 3 2024-02-23T00:55:22.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.561+0000 7f78786d7700 15 client.31791.objecter _session_command_op_remove -1 2 2024-02-23T00:55:22.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.561+0000 7f78786d7700 20 client.31791.objecter get_session s=0x7f7870106e60 osd=0 3 2024-02-23T00:55:22.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.561+0000 7f78786d7700 15 client.31791.objecter _session_command_op_assign 0 2 2024-02-23T00:55:22.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.561+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f7870106e60 osd=0 4 2024-02-23T00:55:22.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.561+0000 7f78786d7700 10 client.31791.objecter _send_command 2 2024-02-23T00:55:22.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.561+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f78700273c0 con 0x7f78700fee20 2024-02-23T00:55:22.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.562+0000 7f78677fe700 1 -- 172.21.15.47:0/2610444896 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f78700273c0 con 0x7f78700fee20 2024-02-23T00:55:22.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.562+0000 7f78677fe700 10 client.31791.objecter ms_dispatch 0x7f7870001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:55:22.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.562+0000 7f78677fe700 10 client.31791.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:55:22.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.562+0000 7f78677fe700 20 client.31791.objecter put_session s=0x7f7870106e60 osd=0 4 2024-02-23T00:55:22.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.562+0000 7f78677fe700 15 client.31791.objecter _session_command_op_remove 0 2 2024-02-23T00:55:22.566 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:55:22.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.563+0000 7f78786d7700 10 client.31791.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:22.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.563+0000 7f78786d7700 15 client.31791.objecter _session_command_op_assign -1 3 2024-02-23T00:55:22.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.563+0000 7f78786d7700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f78700598f0 0x7f7870059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:22.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.563+0000 7f78786d7700 20 client.31791.objecter _get_session s=0x7f7870028710 osd=1 3 2024-02-23T00:55:22.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.563+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f7870028710 osd=1 3 2024-02-23T00:55:22.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.563+0000 7f78786d7700 20 client.31791.objecter _get_session s=0x7f7870028710 osd=1 3 2024-02-23T00:55:22.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.563+0000 7f78786d7700 15 client.31791.objecter _session_command_op_remove -1 3 2024-02-23T00:55:22.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.563+0000 7f78786d7700 20 client.31791.objecter get_session s=0x7f7870028710 osd=1 3 2024-02-23T00:55:22.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.563+0000 7f78786d7700 15 client.31791.objecter _session_command_op_assign 1 3 2024-02-23T00:55:22.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.563+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f7870028710 osd=1 4 2024-02-23T00:55:22.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.563+0000 7f78786d7700 10 client.31791.objecter _send_command 3 2024-02-23T00:55:22.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.563+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f787011b480 con 0x7f78700598f0 2024-02-23T00:55:22.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.563+0000 7f7876473700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f78700598f0 0x7f7870059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:22.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.563+0000 7f78677fe700 10 client.31791.objecter ms_handle_connect 0x7f78700598f0 2024-02-23T00:55:22.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.563+0000 7f7876473700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f78700598f0 0x7f7870059cd0 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:22.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.564+0000 7f78677fe700 1 -- 172.21.15.47:0/2610444896 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f787011b480 con 0x7f78700598f0 2024-02-23T00:55:22.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.564+0000 7f78677fe700 10 client.31791.objecter ms_dispatch 0x7f7870001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:55:22.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.564+0000 7f78677fe700 10 client.31791.objecter _finish_command 3 = system:0 2024-02-23T00:55:22.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.564+0000 7f78677fe700 20 client.31791.objecter put_session s=0x7f7870028710 osd=1 4 2024-02-23T00:55:22.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.564+0000 7f78677fe700 15 client.31791.objecter _session_command_op_remove 1 3 2024-02-23T00:55:22.575 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:55:22.579 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:22.579 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:55:22.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.578+0000 7f78786d7700 10 client.31791.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:55:22.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.578+0000 7f78786d7700 15 client.31791.objecter _session_command_op_assign -1 4 2024-02-23T00:55:22.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.578+0000 7f78786d7700 20 client.31791.objecter _get_session s=0x7f7870028710 osd=1 3 2024-02-23T00:55:22.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.578+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f7870028710 osd=1 3 2024-02-23T00:55:22.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.578+0000 7f78786d7700 20 client.31791.objecter _get_session s=0x7f7870028710 osd=1 3 2024-02-23T00:55:22.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.578+0000 7f78786d7700 15 client.31791.objecter _session_command_op_remove -1 4 2024-02-23T00:55:22.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.578+0000 7f78786d7700 20 client.31791.objecter get_session s=0x7f7870028710 osd=1 3 2024-02-23T00:55:22.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.578+0000 7f78786d7700 15 client.31791.objecter _session_command_op_assign 1 4 2024-02-23T00:55:22.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.578+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f7870028710 osd=1 4 2024-02-23T00:55:22.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.578+0000 7f78786d7700 10 client.31791.objecter _send_command 4 2024-02-23T00:55:22.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.578+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f7870059360 con 0x7f78700598f0 2024-02-23T00:55:22.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.580+0000 7f78677fe700 1 -- 172.21.15.47:0/2610444896 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f7870059360 con 0x7f78700598f0 2024-02-23T00:55:22.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.580+0000 7f78677fe700 10 client.31791.objecter ms_dispatch 0x7f7870001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:55:22.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.580+0000 7f78677fe700 10 client.31791.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:55:22.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.580+0000 7f78677fe700 20 client.31791.objecter put_session s=0x7f7870028710 osd=1 4 2024-02-23T00:55:22.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.580+0000 7f78677fe700 15 client.31791.objecter _session_command_op_remove 1 4 2024-02-23T00:55:22.587 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:55:22.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.580+0000 7f78786d7700 10 client.31791.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:22.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.580+0000 7f78786d7700 15 client.31791.objecter _session_command_op_assign -1 5 2024-02-23T00:55:22.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.580+0000 7f78786d7700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f7870028270 0x7f7870119ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:22.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.580+0000 7f78786d7700 20 client.31791.objecter _get_session s=0x7f7870059770 osd=2 3 2024-02-23T00:55:22.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.580+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f7870059770 osd=2 3 2024-02-23T00:55:22.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.580+0000 7f78786d7700 20 client.31791.objecter _get_session s=0x7f7870059770 osd=2 3 2024-02-23T00:55:22.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.580+0000 7f78786d7700 15 client.31791.objecter _session_command_op_remove -1 5 2024-02-23T00:55:22.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.580+0000 7f78786d7700 20 client.31791.objecter get_session s=0x7f7870059770 osd=2 3 2024-02-23T00:55:22.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.580+0000 7f78786d7700 15 client.31791.objecter _session_command_op_assign 2 5 2024-02-23T00:55:22.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.580+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f7870059770 osd=2 4 2024-02-23T00:55:22.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.580+0000 7f78786d7700 10 client.31791.objecter _send_command 5 2024-02-23T00:55:22.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.580+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f787011a420 con 0x7f7870028270 2024-02-23T00:55:22.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.580+0000 7f7875c72700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f7870028270 0x7f7870119ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:22.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.580+0000 7f7875c72700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f7870028270 0x7f7870119ea0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:22.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.580+0000 7f78677fe700 10 client.31791.objecter ms_handle_connect 0x7f7870028270 2024-02-23T00:55:22.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.581+0000 7f78677fe700 1 -- 172.21.15.47:0/2610444896 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f787011a420 con 0x7f7870028270 2024-02-23T00:55:22.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.581+0000 7f78677fe700 10 client.31791.objecter ms_dispatch 0x7f7870001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:55:22.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.581+0000 7f78677fe700 10 client.31791.objecter _finish_command 5 = system:0 2024-02-23T00:55:22.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.581+0000 7f78677fe700 20 client.31791.objecter put_session s=0x7f7870059770 osd=2 4 2024-02-23T00:55:22.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.581+0000 7f78677fe700 15 client.31791.objecter _session_command_op_remove 2 5 2024-02-23T00:55:22.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.595+0000 7f78786d7700 10 client.31791.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:55:22.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.595+0000 7f78786d7700 15 client.31791.objecter _session_command_op_assign -1 6 2024-02-23T00:55:22.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.595+0000 7f78786d7700 20 client.31791.objecter _get_session s=0x7f7870059770 osd=2 3 2024-02-23T00:55:22.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.595+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f7870059770 osd=2 3 2024-02-23T00:55:22.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.595+0000 7f78786d7700 20 client.31791.objecter _get_session s=0x7f7870059770 osd=2 3 2024-02-23T00:55:22.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.595+0000 7f78786d7700 15 client.31791.objecter _session_command_op_remove -1 6 2024-02-23T00:55:22.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.595+0000 7f78786d7700 20 client.31791.objecter get_session s=0x7f7870059770 osd=2 3 2024-02-23T00:55:22.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.595+0000 7f78786d7700 15 client.31791.objecter _session_command_op_assign 2 6 2024-02-23T00:55:22.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.595+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f7870059770 osd=2 4 2024-02-23T00:55:22.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.595+0000 7f78786d7700 10 client.31791.objecter _send_command 6 2024-02-23T00:55:22.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.595+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f78700277d0 con 0x7f7870028270 2024-02-23T00:55:22.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.597+0000 7f78677fe700 1 -- 172.21.15.47:0/2610444896 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f78700277d0 con 0x7f7870028270 2024-02-23T00:55:22.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.597+0000 7f78677fe700 10 client.31791.objecter ms_dispatch 0x7f7870001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:55:22.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.597+0000 7f78677fe700 10 client.31791.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:55:22.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.597+0000 7f78677fe700 20 client.31791.objecter put_session s=0x7f7870059770 osd=2 4 2024-02-23T00:55:22.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.597+0000 7f78677fe700 15 client.31791.objecter _session_command_op_remove 2 6 2024-02-23T00:55:22.601 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:55:22.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.597+0000 7f78786d7700 10 client.31791.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:22.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.597+0000 7f78786d7700 15 client.31791.objecter _session_command_op_assign -1 7 2024-02-23T00:55:22.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.597+0000 7f78786d7700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f787011d600 0x7f787011f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:22.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.597+0000 7f78786d7700 20 client.31791.objecter _get_session s=0x7f787011adf0 osd=3 3 2024-02-23T00:55:22.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.597+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f787011adf0 osd=3 3 2024-02-23T00:55:22.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.597+0000 7f78786d7700 20 client.31791.objecter _get_session s=0x7f787011adf0 osd=3 3 2024-02-23T00:55:22.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.597+0000 7f78786d7700 15 client.31791.objecter _session_command_op_remove -1 7 2024-02-23T00:55:22.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.597+0000 7f78786d7700 20 client.31791.objecter get_session s=0x7f787011adf0 osd=3 3 2024-02-23T00:55:22.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.597+0000 7f78786d7700 15 client.31791.objecter _session_command_op_assign 3 7 2024-02-23T00:55:22.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.597+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f787011adf0 osd=3 4 2024-02-23T00:55:22.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.597+0000 7f78786d7700 10 client.31791.objecter _send_command 7 2024-02-23T00:55:22.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.597+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7870122750 con 0x7f787011d600 2024-02-23T00:55:22.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.597+0000 7f7876c74700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f787011d600 0x7f787011f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:22.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.597+0000 7f7876c74700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f787011d600 0x7f787011f9f0 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:22.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.597+0000 7f78677fe700 10 client.31791.objecter ms_handle_connect 0x7f787011d600 2024-02-23T00:55:22.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.598+0000 7f78677fe700 1 -- 172.21.15.47:0/2610444896 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7870122750 con 0x7f787011d600 2024-02-23T00:55:22.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.598+0000 7f78677fe700 10 client.31791.objecter ms_dispatch 0x7f7870001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:55:22.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.598+0000 7f78677fe700 10 client.31791.objecter _finish_command 7 = system:0 2024-02-23T00:55:22.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.598+0000 7f78677fe700 20 client.31791.objecter put_session s=0x7f787011adf0 osd=3 4 2024-02-23T00:55:22.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.598+0000 7f78677fe700 15 client.31791.objecter _session_command_op_remove 3 7 2024-02-23T00:55:22.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.612+0000 7f78786d7700 10 client.31791.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:55:22.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.612+0000 7f78786d7700 15 client.31791.objecter _session_command_op_assign -1 8 2024-02-23T00:55:22.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.612+0000 7f78786d7700 20 client.31791.objecter _get_session s=0x7f787011adf0 osd=3 3 2024-02-23T00:55:22.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.612+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f787011adf0 osd=3 3 2024-02-23T00:55:22.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.612+0000 7f78786d7700 20 client.31791.objecter _get_session s=0x7f787011adf0 osd=3 3 2024-02-23T00:55:22.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.612+0000 7f78786d7700 15 client.31791.objecter _session_command_op_remove -1 8 2024-02-23T00:55:22.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.612+0000 7f78786d7700 20 client.31791.objecter get_session s=0x7f787011adf0 osd=3 3 2024-02-23T00:55:22.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.612+0000 7f78786d7700 15 client.31791.objecter _session_command_op_assign 3 8 2024-02-23T00:55:22.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.612+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f787011adf0 osd=3 4 2024-02-23T00:55:22.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.612+0000 7f78786d7700 10 client.31791.objecter _send_command 8 2024-02-23T00:55:22.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.612+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f787011a860 con 0x7f787011d600 2024-02-23T00:55:22.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.614+0000 7f78677fe700 1 -- 172.21.15.47:0/2610444896 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f787011a860 con 0x7f787011d600 2024-02-23T00:55:22.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.614+0000 7f78677fe700 10 client.31791.objecter ms_dispatch 0x7f7870001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:55:22.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.614+0000 7f78677fe700 10 client.31791.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:55:22.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.614+0000 7f78677fe700 20 client.31791.objecter put_session s=0x7f787011adf0 osd=3 4 2024-02-23T00:55:22.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.614+0000 7f78677fe700 15 client.31791.objecter _session_command_op_remove 3 8 2024-02-23T00:55:22.617 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:55:22.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.614+0000 7f78786d7700 10 client.31791.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:22.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.614+0000 7f78786d7700 15 client.31791.objecter _session_command_op_assign -1 9 2024-02-23T00:55:22.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.614+0000 7f78786d7700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f7870119a00 0x7f787011ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:22.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.614+0000 7f78786d7700 20 client.31791.objecter _get_session s=0x7f787011a860 osd=4 3 2024-02-23T00:55:22.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.614+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f787011a860 osd=4 3 2024-02-23T00:55:22.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.614+0000 7f78786d7700 20 client.31791.objecter _get_session s=0x7f787011a860 osd=4 3 2024-02-23T00:55:22.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.614+0000 7f78786d7700 15 client.31791.objecter _session_command_op_remove -1 9 2024-02-23T00:55:22.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.614+0000 7f78786d7700 20 client.31791.objecter get_session s=0x7f787011a860 osd=4 3 2024-02-23T00:55:22.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.614+0000 7f78786d7700 15 client.31791.objecter _session_command_op_assign 4 9 2024-02-23T00:55:22.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.614+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f787011a860 osd=4 4 2024-02-23T00:55:22.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.614+0000 7f78786d7700 10 client.31791.objecter _send_command 9 2024-02-23T00:55:22.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.614+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7870120440 con 0x7f7870119a00 2024-02-23T00:55:22.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.614+0000 7f7876473700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f7870119a00 0x7f787011ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:22.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.614+0000 7f78677fe700 10 client.31791.objecter ms_handle_connect 0x7f7870119a00 2024-02-23T00:55:22.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.614+0000 7f7876473700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f7870119a00 0x7f787011ff50 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:22.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.615+0000 7f78677fe700 1 -- 172.21.15.47:0/2610444896 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7870120440 con 0x7f7870119a00 2024-02-23T00:55:22.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.615+0000 7f78677fe700 10 client.31791.objecter ms_dispatch 0x7f7870001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:55:22.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.615+0000 7f78677fe700 10 client.31791.objecter _finish_command 9 = system:0 2024-02-23T00:55:22.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.615+0000 7f78677fe700 20 client.31791.objecter put_session s=0x7f787011a860 osd=4 4 2024-02-23T00:55:22.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.615+0000 7f78677fe700 15 client.31791.objecter _session_command_op_remove 4 9 2024-02-23T00:55:22.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.630+0000 7f78786d7700 10 client.31791.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:55:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.630+0000 7f78786d7700 15 client.31791.objecter _session_command_op_assign -1 10 2024-02-23T00:55:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.630+0000 7f78786d7700 20 client.31791.objecter _get_session s=0x7f787011a860 osd=4 3 2024-02-23T00:55:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.630+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f787011a860 osd=4 3 2024-02-23T00:55:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.630+0000 7f78786d7700 20 client.31791.objecter _get_session s=0x7f787011a860 osd=4 3 2024-02-23T00:55:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.630+0000 7f78786d7700 15 client.31791.objecter _session_command_op_remove -1 10 2024-02-23T00:55:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.630+0000 7f78786d7700 20 client.31791.objecter get_session s=0x7f787011a860 osd=4 3 2024-02-23T00:55:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.630+0000 7f78786d7700 15 client.31791.objecter _session_command_op_assign 4 10 2024-02-23T00:55:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.630+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f787011a860 osd=4 4 2024-02-23T00:55:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.630+0000 7f78786d7700 10 client.31791.objecter _send_command 10 2024-02-23T00:55:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.630+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f7870120440 con 0x7f7870119a00 2024-02-23T00:55:22.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.631+0000 7f78677fe700 1 -- 172.21.15.47:0/2610444896 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f7870120440 con 0x7f7870119a00 2024-02-23T00:55:22.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.631+0000 7f78677fe700 10 client.31791.objecter ms_dispatch 0x7f7870001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:55:22.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.631+0000 7f78677fe700 10 client.31791.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:55:22.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.631+0000 7f78677fe700 20 client.31791.objecter put_session s=0x7f787011a860 osd=4 4 2024-02-23T00:55:22.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.631+0000 7f78677fe700 15 client.31791.objecter _session_command_op_remove 4 10 2024-02-23T00:55:22.635 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:55:22.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.632+0000 7f78786d7700 10 client.31791.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:22.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.632+0000 7f78786d7700 15 client.31791.objecter _session_command_op_assign -1 11 2024-02-23T00:55:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.632+0000 7f78786d7700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f7870120af0 0x7f7870120f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.632+0000 7f78786d7700 20 client.31791.objecter _get_session s=0x7f7870120440 osd=5 3 2024-02-23T00:55:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.632+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f7870120440 osd=5 3 2024-02-23T00:55:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.632+0000 7f78786d7700 20 client.31791.objecter _get_session s=0x7f7870120440 osd=5 3 2024-02-23T00:55:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.632+0000 7f78786d7700 15 client.31791.objecter _session_command_op_remove -1 11 2024-02-23T00:55:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.632+0000 7f78786d7700 20 client.31791.objecter get_session s=0x7f7870120440 osd=5 3 2024-02-23T00:55:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.632+0000 7f78786d7700 15 client.31791.objecter _session_command_op_assign 5 11 2024-02-23T00:55:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.632+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f7870120440 osd=5 4 2024-02-23T00:55:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.632+0000 7f78786d7700 10 client.31791.objecter _send_command 11 2024-02-23T00:55:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.632+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f78701290b0 con 0x7f7870120af0 2024-02-23T00:55:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.632+0000 7f7875c72700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f7870120af0 0x7f7870120f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.632+0000 7f78677fe700 10 client.31791.objecter ms_handle_connect 0x7f7870120af0 2024-02-23T00:55:22.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.632+0000 7f7875c72700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f7870120af0 0x7f7870120f30 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:22.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.633+0000 7f78677fe700 1 -- 172.21.15.47:0/2610444896 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f78701290b0 con 0x7f7870120af0 2024-02-23T00:55:22.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.633+0000 7f78677fe700 10 client.31791.objecter ms_dispatch 0x7f7870001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:55:22.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.633+0000 7f78677fe700 10 client.31791.objecter _finish_command 11 = system:0 2024-02-23T00:55:22.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.633+0000 7f78677fe700 20 client.31791.objecter put_session s=0x7f7870120440 osd=5 4 2024-02-23T00:55:22.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.633+0000 7f78677fe700 15 client.31791.objecter _session_command_op_remove 5 11 2024-02-23T00:55:22.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.648+0000 7f78786d7700 10 client.31791.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:55:22.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.648+0000 7f78786d7700 15 client.31791.objecter _session_command_op_assign -1 12 2024-02-23T00:55:22.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.648+0000 7f78786d7700 20 client.31791.objecter _get_session s=0x7f7870120440 osd=5 3 2024-02-23T00:55:22.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.648+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f7870120440 osd=5 3 2024-02-23T00:55:22.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.648+0000 7f78786d7700 20 client.31791.objecter _get_session s=0x7f7870120440 osd=5 3 2024-02-23T00:55:22.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.648+0000 7f78786d7700 15 client.31791.objecter _session_command_op_remove -1 12 2024-02-23T00:55:22.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.648+0000 7f78786d7700 20 client.31791.objecter get_session s=0x7f7870120440 osd=5 3 2024-02-23T00:55:22.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.648+0000 7f78786d7700 15 client.31791.objecter _session_command_op_assign 5 12 2024-02-23T00:55:22.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.648+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f7870120440 osd=5 4 2024-02-23T00:55:22.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.648+0000 7f78786d7700 10 client.31791.objecter _send_command 12 2024-02-23T00:55:22.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.648+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f7870126910 con 0x7f7870120af0 2024-02-23T00:55:22.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.650+0000 7f78677fe700 1 -- 172.21.15.47:0/2610444896 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f7870126910 con 0x7f7870120af0 2024-02-23T00:55:22.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.650+0000 7f78677fe700 10 client.31791.objecter ms_dispatch 0x7f7870001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:55:22.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.650+0000 7f78677fe700 10 client.31791.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:55:22.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.650+0000 7f78677fe700 20 client.31791.objecter put_session s=0x7f7870120440 osd=5 4 2024-02-23T00:55:22.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.650+0000 7f78677fe700 15 client.31791.objecter _session_command_op_remove 5 12 2024-02-23T00:55:22.654 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:55:22.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.651+0000 7f78786d7700 10 client.31791.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:22.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.651+0000 7f78786d7700 15 client.31791.objecter _session_command_op_assign -1 13 2024-02-23T00:55:22.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.651+0000 7f78786d7700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f7870126fe0 0x7f7870127420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:22.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.651+0000 7f78786d7700 20 client.31791.objecter _get_session s=0x7f7870126910 osd=6 3 2024-02-23T00:55:22.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.651+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f7870126910 osd=6 3 2024-02-23T00:55:22.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.651+0000 7f78786d7700 20 client.31791.objecter _get_session s=0x7f7870126910 osd=6 3 2024-02-23T00:55:22.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.651+0000 7f78786d7700 15 client.31791.objecter _session_command_op_remove -1 13 2024-02-23T00:55:22.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.651+0000 7f78786d7700 20 client.31791.objecter get_session s=0x7f7870126910 osd=6 3 2024-02-23T00:55:22.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.651+0000 7f78786d7700 15 client.31791.objecter _session_command_op_assign 6 13 2024-02-23T00:55:22.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.651+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f7870126910 osd=6 4 2024-02-23T00:55:22.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.651+0000 7f78786d7700 10 client.31791.objecter _send_command 13 2024-02-23T00:55:22.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.651+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f787012c3e0 con 0x7f7870126fe0 2024-02-23T00:55:22.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.651+0000 7f7876c74700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f7870126fe0 0x7f7870127420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:22.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.651+0000 7f78677fe700 10 client.31791.objecter ms_handle_connect 0x7f7870126fe0 2024-02-23T00:55:22.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.651+0000 7f7876c74700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f7870126fe0 0x7f7870127420 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:22.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.652+0000 7f78677fe700 1 -- 172.21.15.47:0/2610444896 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f787012c3e0 con 0x7f7870126fe0 2024-02-23T00:55:22.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.652+0000 7f78677fe700 10 client.31791.objecter ms_dispatch 0x7f7870001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:55:22.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.652+0000 7f78677fe700 10 client.31791.objecter _finish_command 13 = system:0 2024-02-23T00:55:22.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.652+0000 7f78677fe700 20 client.31791.objecter put_session s=0x7f7870126910 osd=6 4 2024-02-23T00:55:22.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.652+0000 7f78677fe700 15 client.31791.objecter _session_command_op_remove 6 13 2024-02-23T00:55:22.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.666+0000 7f78786d7700 10 client.31791.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:55:22.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.666+0000 7f78786d7700 15 client.31791.objecter _session_command_op_assign -1 14 2024-02-23T00:55:22.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.666+0000 7f78786d7700 20 client.31791.objecter _get_session s=0x7f7870126910 osd=6 3 2024-02-23T00:55:22.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.666+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f7870126910 osd=6 3 2024-02-23T00:55:22.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.666+0000 7f78786d7700 20 client.31791.objecter _get_session s=0x7f7870126910 osd=6 3 2024-02-23T00:55:22.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.666+0000 7f78786d7700 15 client.31791.objecter _session_command_op_remove -1 14 2024-02-23T00:55:22.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.666+0000 7f78786d7700 20 client.31791.objecter get_session s=0x7f7870126910 osd=6 3 2024-02-23T00:55:22.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.666+0000 7f78786d7700 15 client.31791.objecter _session_command_op_assign 6 14 2024-02-23T00:55:22.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.666+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f7870126910 osd=6 4 2024-02-23T00:55:22.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.666+0000 7f78786d7700 10 client.31791.objecter _send_command 14 2024-02-23T00:55:22.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.666+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f7870127a20 con 0x7f7870126fe0 2024-02-23T00:55:22.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.667+0000 7f78677fe700 1 -- 172.21.15.47:0/2610444896 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f7870127a20 con 0x7f7870126fe0 2024-02-23T00:55:22.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.667+0000 7f78677fe700 10 client.31791.objecter ms_dispatch 0x7f7870001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:55:22.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.667+0000 7f78677fe700 10 client.31791.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:55:22.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.667+0000 7f78677fe700 20 client.31791.objecter put_session s=0x7f7870126910 osd=6 4 2024-02-23T00:55:22.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.667+0000 7f78677fe700 15 client.31791.objecter _session_command_op_remove 6 14 2024-02-23T00:55:22.671 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:55:22.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.668+0000 7f78786d7700 10 client.31791.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:22.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.668+0000 7f78786d7700 15 client.31791.objecter _session_command_op_assign -1 15 2024-02-23T00:55:22.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.668+0000 7f78786d7700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f78701280a0 0x7f78701284e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:22.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.668+0000 7f78786d7700 20 client.31791.objecter _get_session s=0x7f7870127a20 osd=7 3 2024-02-23T00:55:22.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.668+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f7870127a20 osd=7 3 2024-02-23T00:55:22.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.668+0000 7f78786d7700 20 client.31791.objecter _get_session s=0x7f7870127a20 osd=7 3 2024-02-23T00:55:22.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.668+0000 7f78786d7700 15 client.31791.objecter _session_command_op_remove -1 15 2024-02-23T00:55:22.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.668+0000 7f78786d7700 20 client.31791.objecter get_session s=0x7f7870127a20 osd=7 3 2024-02-23T00:55:22.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.668+0000 7f78786d7700 15 client.31791.objecter _session_command_op_assign 7 15 2024-02-23T00:55:22.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.668+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f7870127a20 osd=7 4 2024-02-23T00:55:22.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.668+0000 7f78786d7700 10 client.31791.objecter _send_command 15 2024-02-23T00:55:22.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.668+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7870128b20 con 0x7f78701280a0 2024-02-23T00:55:22.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.668+0000 7f7876473700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f78701280a0 0x7f78701284e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:22.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.668+0000 7f78677fe700 10 client.31791.objecter ms_handle_connect 0x7f78701280a0 2024-02-23T00:55:22.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.668+0000 7f7876473700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f78701280a0 0x7f78701284e0 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:22.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.669+0000 7f78677fe700 1 -- 172.21.15.47:0/2610444896 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7870128b20 con 0x7f78701280a0 2024-02-23T00:55:22.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.669+0000 7f78677fe700 10 client.31791.objecter ms_dispatch 0x7f7870001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:55:22.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.669+0000 7f78677fe700 10 client.31791.objecter _finish_command 15 = system:0 2024-02-23T00:55:22.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.669+0000 7f78677fe700 20 client.31791.objecter put_session s=0x7f7870127a20 osd=7 4 2024-02-23T00:55:22.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.669+0000 7f78677fe700 15 client.31791.objecter _session_command_op_remove 7 15 2024-02-23T00:55:22.680 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:55:22.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.684+0000 7f78786d7700 10 client.31791.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:55:22.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.684+0000 7f78786d7700 15 client.31791.objecter _session_command_op_assign -1 16 2024-02-23T00:55:22.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.684+0000 7f78786d7700 20 client.31791.objecter _get_session s=0x7f7870127a20 osd=7 3 2024-02-23T00:55:22.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.684+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f7870127a20 osd=7 3 2024-02-23T00:55:22.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.684+0000 7f78786d7700 20 client.31791.objecter _get_session s=0x7f7870127a20 osd=7 3 2024-02-23T00:55:22.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.684+0000 7f78786d7700 15 client.31791.objecter _session_command_op_remove -1 16 2024-02-23T00:55:22.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.684+0000 7f78786d7700 20 client.31791.objecter get_session s=0x7f7870127a20 osd=7 3 2024-02-23T00:55:22.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.684+0000 7f78786d7700 15 client.31791.objecter _session_command_op_assign 7 16 2024-02-23T00:55:22.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.684+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f7870127a20 osd=7 4 2024-02-23T00:55:22.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.684+0000 7f78786d7700 10 client.31791.objecter _send_command 16 2024-02-23T00:55:22.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.684+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f7870128b20 con 0x7f78701280a0 2024-02-23T00:55:22.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.686+0000 7f78677fe700 1 -- 172.21.15.47:0/2610444896 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f7870128b20 con 0x7f78701280a0 2024-02-23T00:55:22.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.686+0000 7f78677fe700 10 client.31791.objecter ms_dispatch 0x7f7870001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:55:22.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.686+0000 7f78677fe700 10 client.31791.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:55:22.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.686+0000 7f78677fe700 20 client.31791.objecter put_session s=0x7f7870127a20 osd=7 4 2024-02-23T00:55:22.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.686+0000 7f78677fe700 15 client.31791.objecter _session_command_op_remove 7 16 2024-02-23T00:55:22.689 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:55:22.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.686+0000 7f78786d7700 10 librados: watch_flush enter 2024-02-23T00:55:22.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.686+0000 7f78786d7700 10 librados: watch_flush exit 2024-02-23T00:55:22.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.686+0000 7f78786d7700 10 client.31791.objecter close_session for osd.0 2024-02-23T00:55:22.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.686+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f78700fee20 msgr2=0x7f78700ff200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:22.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.686+0000 7f78786d7700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f78700fee20 0x7f78700ff200 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:22.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.686+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f7870106e60 osd=0 1 2024-02-23T00:55:22.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.686+0000 7f78786d7700 10 client.31791.objecter close_session for osd.1 2024-02-23T00:55:22.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.686+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f78700598f0 msgr2=0x7f7870059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:22.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.686+0000 7f78786d7700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f78700598f0 0x7f7870059cd0 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:22.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.686+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f7870028710 osd=1 1 2024-02-23T00:55:22.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.686+0000 7f78786d7700 10 client.31791.objecter close_session for osd.2 2024-02-23T00:55:22.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.686+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f7870028270 msgr2=0x7f7870119ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:22.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.686+0000 7f78786d7700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f7870028270 0x7f7870119ea0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:22.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.686+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f7870059770 osd=2 1 2024-02-23T00:55:22.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.686+0000 7f78786d7700 10 client.31791.objecter close_session for osd.3 2024-02-23T00:55:22.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.686+0000 7f7876c74700 1 -- 172.21.15.47:0/2610444896 reap_dead start 2024-02-23T00:55:22.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.686+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f787011d600 msgr2=0x7f787011f9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:22.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.686+0000 7f78786d7700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f787011d600 0x7f787011f9f0 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:22.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.687+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f787011adf0 osd=3 1 2024-02-23T00:55:22.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.687+0000 7f78786d7700 10 client.31791.objecter close_session for osd.4 2024-02-23T00:55:22.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.687+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f7870119a00 msgr2=0x7f787011ff50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:22.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.687+0000 7f78786d7700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f7870119a00 0x7f787011ff50 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:22.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.687+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f787011a860 osd=4 1 2024-02-23T00:55:22.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.687+0000 7f78786d7700 10 client.31791.objecter close_session for osd.5 2024-02-23T00:55:22.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.687+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f7870120af0 msgr2=0x7f7870120f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:22.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.687+0000 7f78786d7700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f7870120af0 0x7f7870120f30 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:22.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.687+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f7870120440 osd=5 1 2024-02-23T00:55:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.687+0000 7f78786d7700 10 client.31791.objecter close_session for osd.6 2024-02-23T00:55:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.687+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f7870126fe0 msgr2=0x7f7870127420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.687+0000 7f78786d7700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f7870126fe0 0x7f7870127420 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.687+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f7870126910 osd=6 1 2024-02-23T00:55:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.687+0000 7f78786d7700 10 client.31791.objecter close_session for osd.7 2024-02-23T00:55:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.687+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f78701280a0 msgr2=0x7f78701284e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.687+0000 7f78786d7700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f78701280a0 0x7f78701284e0 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.687+0000 7f78786d7700 20 client.31791.objecter put_session s=0x7f7870127a20 osd=7 1 2024-02-23T00:55:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.687+0000 7f78786d7700 20 client.31791.objecter shutdown clearing up homeless session... 2024-02-23T00:55:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.687+0000 7f78786d7700 10 client.31791.objecter successfully canceled tick 2024-02-23T00:55:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.687+0000 7f7876c74700 1 -- 172.21.15.47:0/2610444896 reap_dead start 2024-02-23T00:55:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.687+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f785c062cc0 msgr2=0x7f785c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.687+0000 7f78786d7700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f785c062cc0 0x7f785c065180 secure :-1 s=READY pgs=3202 cs=0 l=1 rev1=1 rx=0x7f786c0048b0 tx=0x7f786c00b040).stop 2024-02-23T00:55:22.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.687+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78700fdbb0 msgr2=0x7f787010e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:22.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.687+0000 7f78786d7700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78700fdbb0 0x7f787010e3c0 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 rx=0x7f7860000e50 tx=0x7f7860009cb0).stop 2024-02-23T00:55:22.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.687+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 shutdown_connections 2024-02-23T00:55:22.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.687+0000 7f78786d7700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78700fdbb0 0x7f787010e3c0 unknown :-1 s=CLOSED pgs=2638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:22.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.687+0000 7f78786d7700 1 --2- 172.21.15.47:0/2610444896 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f785c062cc0 0x7f785c065180 unknown :-1 s=CLOSED pgs=3202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:22.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.687+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 >> 172.21.15.47:0/2610444896 conn(0x7f78700f3800 msgr2=0x7f78700fb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:22.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.687+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 shutdown_connections 2024-02-23T00:55:22.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.687+0000 7f78786d7700 1 -- 172.21.15.47:0/2610444896 wait complete. 2024-02-23T00:55:22.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:22.687+0000 7f78786d7700 1 librados: shutdown 2024-02-23T00:55:22.694 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:55:22.694 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:55:22.694 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:55:22.695 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:55:22.695 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:55:22.695 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:55:22.695 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:55:22.695 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:55:22.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:21.105395+0000 osd.2 (osd.2) 171 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:55:22.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:21.105640+0000 osd.2 (osd.2) 172 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:55:22.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:21.221774+0000 osd.4 (osd.4) 319 : cluster [DBG] 4.9 scrub starts 2024-02-23T00:55:22.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:21.223705+0000 osd.4 (osd.4) 320 : cluster [DBG] 4.9 scrub ok 2024-02-23T00:55:22.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:21.375178+0000 mon.a (mon.0) 3684 : cluster [DBG] osdmap e772: 8 total, 8 up, 7 in 2024-02-23T00:55:22.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:21.398819+0000 mgr.y (mgr.14517) 656 : cluster [DBG] pgmap v1172: 194 pgs: 10 unknown, 1 active+remapped+backfill_toofull, 183 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1/789 objects misplaced (0.127%) 2024-02-23T00:55:22.696 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:55:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:55:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:55:22.711 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:22.730 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:55:22.733 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:22.734 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:55:22.758 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:55:22.835 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:55:22.864 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:22.939 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:55:22.941 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:22.942 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:55:23.044 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:55:23.072 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:23.308 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:55:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:55:23.421 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:55:23.424 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:23.424 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:55:23.525 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:55:23.553 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:23.573 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:55:23.574 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:21.993864+0000 osd.7 (osd.7) 347 : cluster [DBG] 1.0 scrub starts 2024-02-23T00:55:23.574 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:21.996755+0000 osd.7 (osd.7) 348 : cluster [DBG] 1.0 scrub ok 2024-02-23T00:55:23.574 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:22.248319+0000 osd.4 (osd.4) 321 : cluster [DBG] 18.d scrub starts 2024-02-23T00:55:23.575 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:22.249684+0000 osd.4 (osd.4) 322 : cluster [DBG] 18.d scrub ok 2024-02-23T00:55:23.575 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:22.545315+0000 mon.a (mon.0) 3685 : audit [DBG] from='client.? 172.21.15.47:0/2610444896' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:55:23.575 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:22.643435+0000 osd.6 (osd.6) 301 : cluster [DBG] 18.16 scrub starts 2024-02-23T00:55:23.575 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:22.644367+0000 osd.6 (osd.6) 302 : cluster [DBG] 18.16 scrub ok 2024-02-23T00:55:23.575 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:23.576 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:55:23.677 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:55:23.705 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:21.993864+0000 osd.7 (osd.7) 347 : cluster [DBG] 1.0 scrub starts 2024-02-23T00:55:23.705 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:21.996755+0000 osd.7 (osd.7) 348 : cluster [DBG] 1.0 scrub ok 2024-02-23T00:55:23.705 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:22.248319+0000 osd.4 (osd.4) 321 : cluster [DBG] 18.d scrub starts 2024-02-23T00:55:23.706 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:22.249684+0000 osd.4 (osd.4) 322 : cluster [DBG] 18.d scrub ok 2024-02-23T00:55:23.706 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:22.545315+0000 mon.a (mon.0) 3685 : audit [DBG] from='client.? 172.21.15.47:0/2610444896' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:55:23.706 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:22.643435+0000 osd.6 (osd.6) 301 : cluster [DBG] 18.16 scrub starts 2024-02-23T00:55:23.706 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:22.644367+0000 osd.6 (osd.6) 302 : cluster [DBG] 18.16 scrub ok 2024-02-23T00:55:23.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:21.993864+0000 osd.7 (osd.7) 347 : cluster [DBG] 1.0 scrub starts 2024-02-23T00:55:23.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:21.996755+0000 osd.7 (osd.7) 348 : cluster [DBG] 1.0 scrub ok 2024-02-23T00:55:23.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:22.248319+0000 osd.4 (osd.4) 321 : cluster [DBG] 18.d scrub starts 2024-02-23T00:55:23.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:22.249684+0000 osd.4 (osd.4) 322 : cluster [DBG] 18.d scrub ok 2024-02-23T00:55:23.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:22.545315+0000 mon.a (mon.0) 3685 : audit [DBG] from='client.? 172.21.15.47:0/2610444896' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:55:23.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:22.643435+0000 osd.6 (osd.6) 301 : cluster [DBG] 18.16 scrub starts 2024-02-23T00:55:23.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:22.644367+0000 osd.6 (osd.6) 302 : cluster [DBG] 18.16 scrub ok 2024-02-23T00:55:23.707 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:23.728 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:55:23.730 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:23.731 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:55:23.832 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:55:23.862 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:23.883 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:55:23.886 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:23.886 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:55:23.987 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:55:24.016 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:24.037 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:55:24.039 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:24.040 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:55:24.063 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-23T00:55:24.140 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:55:24.168 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:24.188 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:55:24.190 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:24.191 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:55:24.292 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:55:24.321 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:24.360 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:55:24.536 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:55:24.536 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-02-23T00:55:24.536 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-02-23T00:55:24.536 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-02-23T00:55:24.536 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:55:24.560 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:22.970869+0000 mgr.y (mgr.14517) 657 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:55:24.560 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:23.037458+0000 osd.7 (osd.7) 349 : cluster [DBG] 6.5 scrub starts 2024-02-23T00:55:24.560 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:23.040498+0000 osd.7 (osd.7) 350 : cluster [DBG] 6.5 scrub ok 2024-02-23T00:55:24.561 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:23.229351+0000 osd.4 (osd.4) 323 : cluster [DBG] 18.19 scrub starts 2024-02-23T00:55:24.561 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:23.230302+0000 osd.4 (osd.4) 324 : cluster [DBG] 18.19 scrub ok 2024-02-23T00:55:24.561 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:23.399827+0000 mgr.y (mgr.14517) 658 : cluster [DBG] pgmap v1173: 194 pgs: 1 active+remapped+backfill_toofull, 193 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1014 B/s rd, 0 op/s; 1/789 objects misplaced (0.127%); 0 B/s, 0 objects/s recovering 2024-02-23T00:55:24.562 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:23.502976+0000 mon.c (mon.2) 477 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:24.562 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:23.503369+0000 mon.a (mon.0) 3686 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:24.569 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:55:24.572 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:24.573 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:55:24.662 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:22.970869+0000 mgr.y (mgr.14517) 657 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:55:24.662 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:23.037458+0000 osd.7 (osd.7) 349 : cluster [DBG] 6.5 scrub starts 2024-02-23T00:55:24.662 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:23.040498+0000 osd.7 (osd.7) 350 : cluster [DBG] 6.5 scrub ok 2024-02-23T00:55:24.662 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:23.229351+0000 osd.4 (osd.4) 323 : cluster [DBG] 18.19 scrub starts 2024-02-23T00:55:24.662 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:23.230302+0000 osd.4 (osd.4) 324 : cluster [DBG] 18.19 scrub ok 2024-02-23T00:55:24.662 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:23.399827+0000 mgr.y (mgr.14517) 658 : cluster [DBG] pgmap v1173: 194 pgs: 1 active+remapped+backfill_toofull, 193 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1014 B/s rd, 0 op/s; 1/789 objects misplaced (0.127%); 0 B/s, 0 objects/s recovering 2024-02-23T00:55:24.662 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:23.502976+0000 mon.c (mon.2) 477 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:24.662 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:23.503369+0000 mon.a (mon.0) 3686 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:24.663 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:22.970869+0000 mgr.y (mgr.14517) 657 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:55:24.663 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:23.037458+0000 osd.7 (osd.7) 349 : cluster [DBG] 6.5 scrub starts 2024-02-23T00:55:24.663 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:23.040498+0000 osd.7 (osd.7) 350 : cluster [DBG] 6.5 scrub ok 2024-02-23T00:55:24.663 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:23.229351+0000 osd.4 (osd.4) 323 : cluster [DBG] 18.19 scrub starts 2024-02-23T00:55:24.663 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:23.230302+0000 osd.4 (osd.4) 324 : cluster [DBG] 18.19 scrub ok 2024-02-23T00:55:24.663 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:23.399827+0000 mgr.y (mgr.14517) 658 : cluster [DBG] pgmap v1173: 194 pgs: 1 active+remapped+backfill_toofull, 193 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1014 B/s rd, 0 op/s; 1/789 objects misplaced (0.127%); 0 B/s, 0 objects/s recovering 2024-02-23T00:55:24.663 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:23.502976+0000 mon.c (mon.2) 477 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:24.663 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:23.503369+0000 mon.a (mon.0) 3686 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:24.674 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:55:24.702 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:24.723 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:55:24.725 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:24.726 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:55:24.827 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:55:24.855 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:25.197 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.4 dump_historic_ops 2024-02-23T00:55:25.198 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6, 7, 0] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-23T00:55:25.198 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-23T00:55:25.198 INFO:tasks.thrashosds.thrasher:Removing osd 7, in_osds are: [1, 3, 4, 5, 6, 7, 0] 2024-02-23T00:55:25.198 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd out 7 2024-02-23T00:55:25.386 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:55:25.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:23.731427+0000 osd.0 (osd.0) 285 : cluster [DBG] 5.13 scrub starts 2024-02-23T00:55:25.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:23.732292+0000 osd.0 (osd.0) 286 : cluster [DBG] 5.13 scrub ok 2024-02-23T00:55:25.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:24.004422+0000 osd.7 (osd.7) 351 : cluster [DBG] 2.1b scrub starts 2024-02-23T00:55:25.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:24.007339+0000 osd.7 (osd.7) 352 : cluster [DBG] 2.1b scrub ok 2024-02-23T00:55:25.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:24.394315+0000 mon.a (mon.0) 3687 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:55:25.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:24.394388+0000 mon.a (mon.0) 3688 : cluster [DBG] osdmap e773: 8 total, 8 up, 7 in 2024-02-23T00:55:25.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:24.395271+0000 mon.c (mon.2) 478 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-134"}]: dispatch 2024-02-23T00:55:25.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:24.396285+0000 mon.a (mon.0) 3689 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-134"}]: dispatch 2024-02-23T00:55:25.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:25.146651+0000 osd.5 (osd.5) 326 : cluster [DBG] 2.2 scrub starts 2024-02-23T00:55:25.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:25.147482+0000 osd.5 (osd.5) 327 : cluster [DBG] 2.2 scrub ok 2024-02-23T00:55:25.446 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:25.309282+0000 mon.a (mon.0) 3690 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:55:25.446 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:25.313820+0000 mon.a (mon.0) 3691 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-134"}]': finished 2024-02-23T00:55:25.446 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:25.313913+0000 mon.a (mon.0) 3692 : cluster [DBG] osdmap e774: 8 total, 8 up, 7 in 2024-02-23T00:55:25.446 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:25.340190+0000 mon.c (mon.2) 479 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:25.446 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:25.340606+0000 mon.a (mon.0) 3693 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:25.446 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:25.341274+0000 mon.c (mon.2) 480 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-134"}]: dispatch 2024-02-23T00:55:25.446 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:25.341657+0000 mon.a (mon.0) 3694 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-134"}]: dispatch 2024-02-23T00:55:25.474 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:55:25.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:23.731427+0000 osd.0 (osd.0) 285 : cluster [DBG] 5.13 scrub starts 2024-02-23T00:55:25.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:23.732292+0000 osd.0 (osd.0) 286 : cluster [DBG] 5.13 scrub ok 2024-02-23T00:55:25.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:24.004422+0000 osd.7 (osd.7) 351 : cluster [DBG] 2.1b scrub starts 2024-02-23T00:55:25.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:24.007339+0000 osd.7 (osd.7) 352 : cluster [DBG] 2.1b scrub ok 2024-02-23T00:55:25.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:24.394315+0000 mon.a (mon.0) 3687 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:55:25.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:24.394388+0000 mon.a (mon.0) 3688 : cluster [DBG] osdmap e773: 8 total, 8 up, 7 in 2024-02-23T00:55:25.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:24.395271+0000 mon.c (mon.2) 478 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-134"}]: dispatch 2024-02-23T00:55:25.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:24.396285+0000 mon.a (mon.0) 3689 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-134"}]: dispatch 2024-02-23T00:55:25.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:25.146651+0000 osd.5 (osd.5) 326 : cluster [DBG] 2.2 scrub starts 2024-02-23T00:55:25.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:25.147482+0000 osd.5 (osd.5) 327 : cluster [DBG] 2.2 scrub ok 2024-02-23T00:55:25.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:25.309282+0000 mon.a (mon.0) 3690 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:55:25.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:25.313820+0000 mon.a (mon.0) 3691 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-134"}]': finished 2024-02-23T00:55:25.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:25.313913+0000 mon.a (mon.0) 3692 : cluster [DBG] osdmap e774: 8 total, 8 up, 7 in 2024-02-23T00:55:25.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:25.340190+0000 mon.c (mon.2) 479 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:25.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:25.340606+0000 mon.a (mon.0) 3693 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:25.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:25.341274+0000 mon.c (mon.2) 480 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-134"}]: dispatch 2024-02-23T00:55:25.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:25.341657+0000 mon.a (mon.0) 3694 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-134"}]: dispatch 2024-02-23T00:55:25.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:23.731427+0000 osd.0 (osd.0) 285 : cluster [DBG] 5.13 scrub starts 2024-02-23T00:55:25.560 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:23.732292+0000 osd.0 (osd.0) 286 : cluster [DBG] 5.13 scrub ok 2024-02-23T00:55:25.560 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:24.004422+0000 osd.7 (osd.7) 351 : cluster [DBG] 2.1b scrub starts 2024-02-23T00:55:25.560 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:24.007339+0000 osd.7 (osd.7) 352 : cluster [DBG] 2.1b scrub ok 2024-02-23T00:55:25.560 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:24.394315+0000 mon.a (mon.0) 3687 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:55:25.560 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:24.394388+0000 mon.a (mon.0) 3688 : cluster [DBG] osdmap e773: 8 total, 8 up, 7 in 2024-02-23T00:55:25.560 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:24.395271+0000 mon.c (mon.2) 478 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-134"}]: dispatch 2024-02-23T00:55:25.560 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:24.396285+0000 mon.a (mon.0) 3689 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-134"}]: dispatch 2024-02-23T00:55:25.560 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:25.146651+0000 osd.5 (osd.5) 326 : cluster [DBG] 2.2 scrub starts 2024-02-23T00:55:25.560 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:25.147482+0000 osd.5 (osd.5) 327 : cluster [DBG] 2.2 scrub ok 2024-02-23T00:55:25.560 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:25.309282+0000 mon.a (mon.0) 3690 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:55:25.560 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:25.313820+0000 mon.a (mon.0) 3691 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-134"}]': finished 2024-02-23T00:55:25.561 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:25.313913+0000 mon.a (mon.0) 3692 : cluster [DBG] osdmap e774: 8 total, 8 up, 7 in 2024-02-23T00:55:25.561 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:25.340190+0000 mon.c (mon.2) 479 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:25.561 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:25.340606+0000 mon.a (mon.0) 3693 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:25.561 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:25.341274+0000 mon.c (mon.2) 480 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-134"}]: dispatch 2024-02-23T00:55:25.561 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:25.341657+0000 mon.a (mon.0) 3694 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-134"}]: dispatch 2024-02-23T00:55:25.866 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:55:26.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.376+0000 7efdf82ae700 1 -- 172.21.15.47:0/557961490 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efdf00f6a90 msgr2=0x7efdf0104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:26.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.376+0000 7efdf82ae700 1 --2- 172.21.15.47:0/557961490 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efdf00f6a90 0x7efdf0104a20 secure :-1 s=READY pgs=2813 cs=0 l=1 rev1=1 rx=0x7efde8009be0 tx=0x7efde8004d10).stop 2024-02-23T00:55:26.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.376+0000 7efdf82ae700 1 -- 172.21.15.47:0/557961490 shutdown_connections 2024-02-23T00:55:26.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.376+0000 7efdf82ae700 1 --2- 172.21.15.47:0/557961490 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efdf00f6a90 0x7efdf0104a20 unknown :-1 s=CLOSED pgs=2813 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:26.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.376+0000 7efdf82ae700 1 --2- 172.21.15.47:0/557961490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efdf00fdbd0 0x7efdf00fdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:26.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.376+0000 7efdf82ae700 1 --2- 172.21.15.47:0/557961490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdf00fe530 0x7efdf00f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:26.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.376+0000 7efdf82ae700 1 -- 172.21.15.47:0/557961490 >> 172.21.15.47:0/557961490 conn(0x7efdf006c370 msgr2=0x7efdf006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:26.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.376+0000 7efdf82ae700 1 -- 172.21.15.47:0/557961490 shutdown_connections 2024-02-23T00:55:26.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.376+0000 7efdf82ae700 1 -- 172.21.15.47:0/557961490 wait complete. 2024-02-23T00:55:26.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.376+0000 7efdf82ae700 1 librados: starting msgr at 2024-02-23T00:55:26.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.376+0000 7efdf82ae700 1 librados: starting objecter 2024-02-23T00:55:26.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.376+0000 7efdf82ae700 1 Processor -- start 2024-02-23T00:55:26.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.377+0000 7efdf82ae700 1 -- start start 2024-02-23T00:55:26.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.377+0000 7efdf82ae700 1 librados: setting wanted keys 2024-02-23T00:55:26.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.377+0000 7efdf82ae700 1 librados: calling monclient init 2024-02-23T00:55:26.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.377+0000 7efdf82ae700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efdf00f6a90 0x7efdf00f7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:26.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.377+0000 7efdf82ae700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efdf00fdbd0 0x7efdf00f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:26.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.377+0000 7efdf82ae700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdf00fe530 0x7efdf00f8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:26.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.377+0000 7efdf82ae700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf01162e0 con 0x7efdf00fe530 2024-02-23T00:55:26.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.377+0000 7efdf82ae700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf0116450 con 0x7efdf00f6a90 2024-02-23T00:55:26.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.377+0000 7efdf82ae700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf0116610 con 0x7efdf00fdbd0 2024-02-23T00:55:26.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.377+0000 7efdf5849700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efdf00fdbd0 0x7efdf00f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:26.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.377+0000 7efdf5849700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efdf00fdbd0 0x7efdf00f7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:43142/0 (socket says 172.21.15.47:43142) 2024-02-23T00:55:26.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.377+0000 7efdf5849700 1 -- 172.21.15.47:0/3438124817 learned_addr learned my addr 172.21.15.47:0/3438124817 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:55:26.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.377+0000 7efdf684b700 1 --2- 172.21.15.47:0/3438124817 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdf00fe530 0x7efdf00f8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:26.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.377+0000 7efdf604a700 1 --2- 172.21.15.47:0/3438124817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efdf00f6a90 0x7efdf00f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:26.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.377+0000 7efdf5849700 1 -- 172.21.15.47:0/3438124817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efdf00f6a90 msgr2=0x7efdf00f7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:26.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.377+0000 7efdf5849700 1 --2- 172.21.15.47:0/3438124817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efdf00f6a90 0x7efdf00f7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:26.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.377+0000 7efdf5849700 1 -- 172.21.15.47:0/3438124817 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdf00fe530 msgr2=0x7efdf00f8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:26.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.377+0000 7efdf5849700 1 --2- 172.21.15.47:0/3438124817 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdf00fe530 0x7efdf00f8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:26.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.377+0000 7efdf5849700 1 -- 172.21.15.47:0/3438124817 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde80097c0 con 0x7efdf00fdbd0 2024-02-23T00:55:26.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.377+0000 7efdf5849700 1 --2- 172.21.15.47:0/3438124817 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efdf00fdbd0 0x7efdf00f7b70 secure :-1 s=READY pgs=2814 cs=0 l=1 rev1=1 rx=0x7efde00041d0 tx=0x7efde0009f60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:26.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.377+0000 7efde7779700 10 client.?.objecter ms_handle_connect 0x7efdf00fdbd0 2024-02-23T00:55:26.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.377+0000 7efde7779700 10 client.?.objecter resend_mon_ops 2024-02-23T00:55:26.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.377+0000 7efde7779700 1 -- 172.21.15.47:0/3438124817 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde0007b40 con 0x7efdf00fdbd0 2024-02-23T00:55:26.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.377+0000 7efde7779700 1 -- 172.21.15.47:0/3438124817 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7efde000f040 con 0x7efdf00fdbd0 2024-02-23T00:55:26.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.377+0000 7efdf82ae700 1 -- 172.21.15.47:0/3438124817 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf01168f0 con 0x7efdf00fdbd0 2024-02-23T00:55:26.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.378+0000 7efde7779700 1 -- 172.21.15.47:0/3438124817 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde0014730 con 0x7efdf00fdbd0 2024-02-23T00:55:26.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.378+0000 7efdf82ae700 10 client.42116.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:55:26.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.378+0000 7efdf82ae700 1 -- 172.21.15.47:0/3438124817 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdf0116e30 con 0x7efdf00fdbd0 2024-02-23T00:55:26.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.378+0000 7efdf82ae700 1 librados: init done 2024-02-23T00:55:26.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.379+0000 7efdf82ae700 1 -- 172.21.15.47:0/3438124817 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdf005fa40 con 0x7efdf00fdbd0 2024-02-23T00:55:26.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.379+0000 7efde7779700 1 -- 172.21.15.47:0/3438124817 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7efde0009110 con 0x7efdf00fdbd0 2024-02-23T00:55:26.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.380+0000 7efde7779700 1 --2- 172.21.15.47:0/3438124817 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7efddc062cc0 0x7efddc065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:26.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.380+0000 7efde7779700 1 -- 172.21.15.47:0/3438124817 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(775..775 src has 621..775) v4 ==== 9367+0+0 (secure 0 0 0) 0x7efde007d710 con 0x7efdf00fdbd0 2024-02-23T00:55:26.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.380+0000 7efdf604a700 1 --2- 172.21.15.47:0/3438124817 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7efddc062cc0 0x7efddc065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:26.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.380+0000 7efde7779700 10 client.42116.objecter ms_dispatch 0x7efdf0001550 osd_map(775..775 src has 621..775) v4 2024-02-23T00:55:26.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.380+0000 7efde7779700 3 client.42116.objecter handle_osd_map got epochs [775,775] > 0 2024-02-23T00:55:26.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.380+0000 7efde7779700 3 client.42116.objecter handle_osd_map decoding full epoch 775 2024-02-23T00:55:26.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.380+0000 7efde7779700 20 client.42116.objecter dump_active .. 0 homeless 2024-02-23T00:55:26.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.380+0000 7efde7779700 10 client.42116.objecter ms_handle_connect 0x7efddc062cc0 2024-02-23T00:55:26.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.380+0000 7efdf604a700 1 --2- 172.21.15.47:0/3438124817 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7efddc062cc0 0x7efddc065180 secure :-1 s=READY pgs=3203 cs=0 l=1 rev1=1 rx=0x7efdec0048b0 tx=0x7efdec00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:26.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.384+0000 7efde7779700 1 -- 172.21.15.47:0/3438124817 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7efde004cc90 con 0x7efdf00fdbd0 2024-02-23T00:55:26.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:26.588+0000 7efdf82ae700 1 -- 172.21.15.47:0/3438124817 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7efdf0106e90 con 0x7efdf00fdbd0 2024-02-23T00:55:26.943 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:25.400327+0000 mgr.y (mgr.14517) 659 : cluster [DBG] pgmap v1176: 194 pgs: 1 active+remapped+backfill_toofull, 193 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1018 B/s rd, 0 op/s; 1/789 objects misplaced (0.127%); 0 B/s, 0 objects/s recovering 2024-02-23T00:55:27.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:25.400327+0000 mgr.y (mgr.14517) 659 : cluster [DBG] pgmap v1176: 194 pgs: 1 active+remapped+backfill_toofull, 193 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1018 B/s rd, 0 op/s; 1/789 objects misplaced (0.127%); 0 B/s, 0 objects/s recovering 2024-02-23T00:55:27.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:25.400327+0000 mgr.y (mgr.14517) 659 : cluster [DBG] pgmap v1176: 194 pgs: 1 active+remapped+backfill_toofull, 193 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1018 B/s rd, 0 op/s; 1/789 objects misplaced (0.127%); 0 B/s, 0 objects/s recovering 2024-02-23T00:55:27.228 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:55:27.228 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-02-23T00:55:27.228 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-02-23T00:55:27.228 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-02-23T00:55:27.228 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.228 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.40928.0:1 170.7s0 170:fd8eaea7:nspace::oid_SimpleWritePP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e266)", 2024-02-23T00:55:27.228 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:45:38.149323+0000", 2024-02-23T00:55:27.229 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 589.07411540800001, 2024-02-23T00:55:27.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0017361360000001, 2024-02-23T00:55:27.229 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:27.229 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:55:27.229 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:55:27.229 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.40928", 2024-02-23T00:55:27.229 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/3947607643", 2024-02-23T00:55:27.229 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1 2024-02-23T00:55:27.229 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.229 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:27.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:27.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:38.149323+0000", 2024-02-23T00:55:27.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:27.230 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.230 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:27.230 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:38.149323+0000", 2024-02-23T00:55:27.230 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999971 2024-02-23T00:55:27.230 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.230 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:27.230 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:38.149320+0000", 2024-02-23T00:55:27.230 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0796e-05 2024-02-23T00:55:27.230 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.230 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:27.230 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:38.149331+0000", 2024-02-23T00:55:27.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6829000000000001e-05 2024-02-23T00:55:27.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:27.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:38.149348+0000", 2024-02-23T00:55:27.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0499000000000001e-05 2024-02-23T00:55:27.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:27.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:38.149358+0000", 2024-02-23T00:55:27.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8974000000000002e-05 2024-02-23T00:55:27.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:38.149407+0000", 2024-02-23T00:55:27.232 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3039999999999998e-06 2024-02-23T00:55:27.232 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.232 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.232 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for peered", 2024-02-23T00:55:27.232 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:38.149412+0000", 2024-02-23T00:55:27.232 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.99911529799999999 2024-02-23T00:55:27.232 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.232 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:39.148527+0000", 2024-02-23T00:55:27.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6186000000000004e-05 2024-02-23T00:55:27.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:27.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:39.148623+0000", 2024-02-23T00:55:27.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040401199999999998 2024-02-23T00:55:27.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:55:27.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:39.149027+0000", 2024-02-23T00:55:27.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00044522000000000002 2024-02-23T00:55:27.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:55:27.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:39.149472+0000", 2024-02-23T00:55:27.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001539931 2024-02-23T00:55:27.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:55:27.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:39.151012+0000", 2024-02-23T00:55:27.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6737000000000003e-05 2024-02-23T00:55:27.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:27.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:45:39.151059+0000", 2024-02-23T00:55:27.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0017361360000001 2024-02-23T00:55:27.235 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.235 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:27.235 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.235 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(osd.5.0:371 175.b e288/279 175:d3d11105:test-rados-api-smithi047-82465-34::foo:14 v 288'11, mlcod=285'8)", 2024-02-23T00:55:27.236 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:46:00.171325+0000", 2024-02-23T00:55:27.236 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 567.05211338699996, 2024-02-23T00:55:27.236 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0065955129999999999, 2024-02-23T00:55:27.236 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:27.236 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:55:27.236 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:27.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.236 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:27.236 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:00.171325+0000", 2024-02-23T00:55:27.236 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:27.236 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.236 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:27.237 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:00.171325+0000", 2024-02-23T00:55:27.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999995 2024-02-23T00:55:27.237 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.237 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:27.237 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:00.171324+0000", 2024-02-23T00:55:27.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.8379999999999998e-06 2024-02-23T00:55:27.237 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.237 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:27.237 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:00.171330+0000", 2024-02-23T00:55:27.238 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0871e-05 2024-02-23T00:55:27.238 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.238 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.238 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:27.238 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:00.171341+0000", 2024-02-23T00:55:27.238 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4309999999999997e-06 2024-02-23T00:55:27.238 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.238 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.238 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:27.238 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:00.171349+0000", 2024-02-23T00:55:27.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012786699999999999 2024-02-23T00:55:27.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:00.171476+0000", 2024-02-23T00:55:27.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4640000000000001e-05 2024-02-23T00:55:27.241 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.242 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:27.242 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:00.171501+0000", 2024-02-23T00:55:27.243 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0063964579999999998 2024-02-23T00:55:27.243 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:55:27.244 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:00.177897+0000", 2024-02-23T00:55:27.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2878e-05 2024-02-23T00:55:27.244 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:27.244 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:46:00.177920+0000", 2024-02-23T00:55:27.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0065955129999999999 2024-02-23T00:55:27.244 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.245 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:27.245 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.245 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.245 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.15270.0:5372 204.6 204:678263ef:test-rados-api-smithi047-82465-50::foo:head [read 0~1] snapc 0=[] ondisk+read+known_if_redirected e382)", 2024-02-23T00:55:27.245 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:47:34.181741+0000", 2024-02-23T00:55:27.245 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 473.04169752400003, 2024-02-23T00:55:27.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010581304999999999, 2024-02-23T00:55:27.245 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:27.245 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:55:27.245 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:55:27.245 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.15270", 2024-02-23T00:55:27.245 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/4133111990", 2024-02-23T00:55:27.245 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 5372 2024-02-23T00:55:27.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.246 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:27.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.246 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:27.246 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:34.181741+0000", 2024-02-23T00:55:27.246 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:27.246 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.246 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:27.246 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:34.181741+0000", 2024-02-23T00:55:27.246 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999976 2024-02-23T00:55:27.246 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.247 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:27.247 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:34.181738+0000", 2024-02-23T00:55:27.247 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2337000000000001e-05 2024-02-23T00:55:27.247 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.247 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.247 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:27.247 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:34.181750+0000", 2024-02-23T00:55:27.247 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4881e-05 2024-02-23T00:55:27.247 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.247 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.247 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:27.247 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:34.181765+0000", 2024-02-23T00:55:27.248 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.628e-06 2024-02-23T00:55:27.248 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.248 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.248 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:27.248 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:34.181775+0000", 2024-02-23T00:55:27.248 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3414000000000001e-05 2024-02-23T00:55:27.248 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.248 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.248 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.248 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:34.181818+0000", 2024-02-23T00:55:27.248 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00019916700000000001 2024-02-23T00:55:27.248 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.248 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.249 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for blocked object", 2024-02-23T00:55:27.249 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:34.182018+0000", 2024-02-23T00:55:27.249 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0070079749999999996 2024-02-23T00:55:27.249 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.249 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.249 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.249 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:34.189026+0000", 2024-02-23T00:55:27.249 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000105541 2024-02-23T00:55:27.249 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.249 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.249 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for rw locks", 2024-02-23T00:55:27.250 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:34.189131+0000", 2024-02-23T00:55:27.250 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0010998570000000001 2024-02-23T00:55:27.250 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.250 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.250 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:34.190231+0000", 2024-02-23T00:55:27.250 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000101282 2024-02-23T00:55:27.250 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.250 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:27.250 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:34.190332+0000", 2024-02-23T00:55:27.250 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0019897859999999999 2024-02-23T00:55:27.250 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.251 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:27.251 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:47:34.192322+0000", 2024-02-23T00:55:27.251 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010581304999999999 2024-02-23T00:55:27.251 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.251 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:27.251 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.251 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.251 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.251 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.41771.0:372 238.7 238:e7fb5982:test-rados-api-smithi047-82465-109::foo:4 [read 0~1] snapc 0=[] ondisk+read+known_if_redirected e639)", 2024-02-23T00:55:27.251 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:52:56.278320+0000", 2024-02-23T00:55:27.251 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 150.94511776900001, 2024-02-23T00:55:27.251 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011305615999999999, 2024-02-23T00:55:27.251 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:27.251 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "reached pg", 2024-02-23T00:55:27.252 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:55:27.252 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.41771", 2024-02-23T00:55:27.252 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1032768195", 2024-02-23T00:55:27.252 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 372 2024-02-23T00:55:27.252 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.252 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:27.252 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.252 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:27.252 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:52:56.278320+0000", 2024-02-23T00:55:27.252 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:27.252 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.252 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.252 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:27.252 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:52:56.278320+0000", 2024-02-23T00:55:27.253 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999971 2024-02-23T00:55:27.253 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.253 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.253 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:27.253 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:52:56.278318+0000", 2024-02-23T00:55:27.253 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2294e-05 2024-02-23T00:55:27.253 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.253 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.253 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:27.253 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:52:56.278330+0000", 2024-02-23T00:55:27.253 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.6098999999999999e-05 2024-02-23T00:55:27.253 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.253 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.253 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:27.253 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:52:56.278356+0000", 2024-02-23T00:55:27.254 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0936e-05 2024-02-23T00:55:27.254 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.254 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.254 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:27.254 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:52:56.278367+0000", 2024-02-23T00:55:27.254 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.3652999999999994e-05 2024-02-23T00:55:27.254 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.254 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.254 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.254 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:52:56.278441+0000", 2024-02-23T00:55:27.254 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.8690000000000004e-06 2024-02-23T00:55:27.254 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.254 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.255 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op must wait for map", 2024-02-23T00:55:27.255 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:52:56.278449+0000", 2024-02-23T00:55:27.255 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089740700000000007 2024-02-23T00:55:27.255 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.255 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.255 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.255 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:52:56.287423+0000", 2024-02-23T00:55:27.255 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021108699999999999 2024-02-23T00:55:27.255 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.255 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.255 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for blocked object", 2024-02-23T00:55:27.255 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:52:56.287634+0000", 2024-02-23T00:55:27.255 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0018780559999999999 2024-02-23T00:55:27.255 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.255 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.256 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.256 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:52:56.289512+0000", 2024-02-23T00:55:27.256 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011434000000000001 2024-02-23T00:55:27.256 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.256 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.256 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:27.256 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:52:56.289626+0000", 2024-02-23T00:55:27.256 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011305615999999999 2024-02-23T00:55:27.256 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.256 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:27.256 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.256 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.256 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.256 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.41771.0:800 241.b 241:d2733045:test-rados-api-smithi047-82465-115::foo:head [read 0~1] snapc 0=[] ondisk+read+known_if_redirected e665)", 2024-02-23T00:55:27.257 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:53:22.306682+0000", 2024-02-23T00:55:27.257 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 124.916756722, 2024-02-23T00:55:27.257 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0074256160000000003, 2024-02-23T00:55:27.257 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:27.257 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:55:27.257 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:55:27.257 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.41771", 2024-02-23T00:55:27.257 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1032768195", 2024-02-23T00:55:27.257 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 800 2024-02-23T00:55:27.257 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.257 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:27.257 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.257 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:27.257 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.306682+0000", 2024-02-23T00:55:27.257 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:27.258 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.258 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.258 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:27.258 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.306682+0000", 2024-02-23T00:55:27.259 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999976 2024-02-23T00:55:27.259 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.259 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.259 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:27.259 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.306679+0000", 2024-02-23T00:55:27.259 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1189000000000001e-05 2024-02-23T00:55:27.259 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.259 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.259 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:27.259 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.306691+0000", 2024-02-23T00:55:27.259 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6419e-05 2024-02-23T00:55:27.259 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.259 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.259 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:27.260 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.306707+0000", 2024-02-23T00:55:27.260 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.498e-06 2024-02-23T00:55:27.260 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.260 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.260 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:27.260 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.306714+0000", 2024-02-23T00:55:27.260 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.8237e-05 2024-02-23T00:55:27.260 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.260 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.260 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.260 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.306753+0000", 2024-02-23T00:55:27.260 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5750000000000002e-06 2024-02-23T00:55:27.260 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.261 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.261 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op must wait for map", 2024-02-23T00:55:27.261 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.306757+0000", 2024-02-23T00:55:27.261 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001881116 2024-02-23T00:55:27.261 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.261 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.261 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.261 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.308638+0000", 2024-02-23T00:55:27.261 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00018778499999999999 2024-02-23T00:55:27.261 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.261 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.261 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for blocked object", 2024-02-23T00:55:27.261 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.308826+0000", 2024-02-23T00:55:27.261 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0042709610000000002 2024-02-23T00:55:27.262 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.262 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.262 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.262 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.313097+0000", 2024-02-23T00:55:27.262 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1002999999999999e-05 2024-02-23T00:55:27.262 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.262 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.262 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for rw locks", 2024-02-23T00:55:27.262 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.313148+0000", 2024-02-23T00:55:27.262 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037617500000000002 2024-02-23T00:55:27.262 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.262 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.262 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.263 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.313524+0000", 2024-02-23T00:55:27.263 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4719999999999998e-05 2024-02-23T00:55:27.263 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.263 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.263 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:27.263 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.313579+0000", 2024-02-23T00:55:27.263 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052811100000000005 2024-02-23T00:55:27.263 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.263 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.263 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:27.263 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.314107+0000", 2024-02-23T00:55:27.263 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0074256160000000003 2024-02-23T00:55:27.263 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.264 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:27.264 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.264 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.264 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.264 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.41771.0:804 241.b 241:d2733045:test-rados-api-smithi047-82465-115::foo:6 [read 0~1] snapc 0=[] ondisk+read+known_if_redirected e665)", 2024-02-23T00:55:27.264 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:53:22.321837+0000", 2024-02-23T00:55:27.264 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 124.901601103, 2024-02-23T00:55:27.264 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.002288389, 2024-02-23T00:55:27.264 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:27.264 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:55:27.264 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:55:27.264 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.41771", 2024-02-23T00:55:27.265 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1032768195", 2024-02-23T00:55:27.265 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 804 2024-02-23T00:55:27.265 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.265 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:27.265 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.265 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:27.265 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.321837+0000", 2024-02-23T00:55:27.265 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:27.265 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.265 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.265 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:27.265 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.321837+0000", 2024-02-23T00:55:27.265 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:55:27.266 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.266 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.266 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:27.266 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.321836+0000", 2024-02-23T00:55:27.266 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.7790000000000001e-06 2024-02-23T00:55:27.266 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.266 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.266 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:27.266 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.321842+0000", 2024-02-23T00:55:27.266 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0508e-05 2024-02-23T00:55:27.266 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.266 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.266 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:27.266 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.321852+0000", 2024-02-23T00:55:27.267 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5870000000000003e-06 2024-02-23T00:55:27.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.267 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.267 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:27.267 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.321857+0000", 2024-02-23T00:55:27.267 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4523999999999997e-05 2024-02-23T00:55:27.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.267 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.267 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.267 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.321892+0000", 2024-02-23T00:55:27.267 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012778799999999999 2024-02-23T00:55:27.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.267 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.267 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for blocked object", 2024-02-23T00:55:27.268 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.322019+0000", 2024-02-23T00:55:27.268 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0011425610000000001 2024-02-23T00:55:27.268 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.268 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.268 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.268 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.323162+0000", 2024-02-23T00:55:27.268 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2418999999999998e-05 2024-02-23T00:55:27.268 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.268 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.268 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for rw locks", 2024-02-23T00:55:27.268 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.323244+0000", 2024-02-23T00:55:27.268 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037892399999999999 2024-02-23T00:55:27.268 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.269 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.269 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.269 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.323623+0000", 2024-02-23T00:55:27.269 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3847000000000001e-05 2024-02-23T00:55:27.269 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.269 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.269 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:27.269 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.323677+0000", 2024-02-23T00:55:27.269 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00044837300000000001 2024-02-23T00:55:27.269 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.269 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.269 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:27.269 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.324126+0000", 2024-02-23T00:55:27.270 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.002288389 2024-02-23T00:55:27.270 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.270 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:27.270 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.270 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.270 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.270 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "MOSDECSubOpWrite(234.5s1 666/655 ECSubWrite(tid=798, reqid=osd.6.0:798, at_version=666'6, trim_to=0'0, roll_forward_to=662'4))", 2024-02-23T00:55:27.270 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:53:23.324163+0000", 2024-02-23T00:55:27.270 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 123.89927542300001, 2024-02-23T00:55:27.270 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091553550000000004, 2024-02-23T00:55:27.270 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:27.270 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "reached pg", 2024-02-23T00:55:27.270 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:27.270 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.271 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:27.271 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.324163+0000", 2024-02-23T00:55:27.271 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:27.271 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.271 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.271 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:27.271 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.324163+0000", 2024-02-23T00:55:27.271 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999995 2024-02-23T00:55:27.271 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.271 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.271 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:27.271 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.324162+0000", 2024-02-23T00:55:27.271 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9916e-05 2024-02-23T00:55:27.271 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.272 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.272 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:27.272 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.324182+0000", 2024-02-23T00:55:27.272 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00245138 2024-02-23T00:55:27.272 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.272 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.272 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:27.272 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.326634+0000", 2024-02-23T00:55:27.272 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.6330000000000002e-06 2024-02-23T00:55:27.272 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.272 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.272 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:27.272 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.326642+0000", 2024-02-23T00:55:27.273 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0026079010000000001 2024-02-23T00:55:27.273 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.273 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.273 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.273 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.329250+0000", 2024-02-23T00:55:27.273 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0893000000000001e-05 2024-02-23T00:55:27.273 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.273 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.273 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:55:27.273 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.329261+0000", 2024-02-23T00:55:27.273 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0040427400000000004 2024-02-23T00:55:27.273 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.273 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.273 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:55:27.274 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.333304+0000", 2024-02-23T00:55:27.274 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4313e-05 2024-02-23T00:55:27.274 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.274 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.274 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:27.274 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.333318+0000", 2024-02-23T00:55:27.274 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091553550000000004 2024-02-23T00:55:27.274 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.274 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:27.274 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.274 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.274 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.274 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "rep_scrubmap(5.18 e63 from shard 6)", 2024-02-23T00:55:27.274 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:53:35.472611+0000", 2024-02-23T00:55:27.274 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 111.750827117, 2024-02-23T00:55:27.275 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0037176840000000002, 2024-02-23T00:55:27.275 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:27.275 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:55:27.275 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:27.275 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.275 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:27.275 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:35.472611+0000", 2024-02-23T00:55:27.275 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:27.275 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.275 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.275 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:27.275 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:35.472611+0000", 2024-02-23T00:55:27.275 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:55:27.276 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.276 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.276 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:27.276 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:35.472610+0000", 2024-02-23T00:55:27.276 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1190000000000002e-06 2024-02-23T00:55:27.276 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.276 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.276 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:27.276 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:35.472616+0000", 2024-02-23T00:55:27.276 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5913000000000002e-05 2024-02-23T00:55:27.276 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.276 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.276 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:27.277 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:35.472632+0000", 2024-02-23T00:55:27.277 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5889999999999999e-06 2024-02-23T00:55:27.277 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.277 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.277 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:27.277 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:35.472639+0000", 2024-02-23T00:55:27.277 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.003661606 2024-02-23T00:55:27.277 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.277 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.277 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.277 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:35.476301+0000", 2024-02-23T00:55:27.277 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1960999999999999e-05 2024-02-23T00:55:27.277 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.277 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.277 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:27.278 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:35.476313+0000", 2024-02-23T00:55:27.278 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6243999999999999e-05 2024-02-23T00:55:27.278 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.278 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.278 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:27.278 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:35.476329+0000", 2024-02-23T00:55:27.278 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0037176840000000002 2024-02-23T00:55:27.278 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.278 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:27.278 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.278 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.278 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.278 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "rep_scrubmap(5.18 e63 from shard 1)", 2024-02-23T00:55:27.278 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:53:35.473421+0000", 2024-02-23T00:55:27.279 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 111.750017017, 2024-02-23T00:55:27.279 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0029861169999999999, 2024-02-23T00:55:27.279 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:27.279 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:55:27.279 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:27.279 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.279 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:27.279 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:35.473421+0000", 2024-02-23T00:55:27.279 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:27.279 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.279 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.279 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:27.279 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:35.473421+0000", 2024-02-23T00:55:27.279 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999995 2024-02-23T00:55:27.280 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.280 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.280 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:27.280 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:35.473421+0000", 2024-02-23T00:55:27.280 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.6440000000000006e-06 2024-02-23T00:55:27.280 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.280 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.280 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:27.280 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:35.473429+0000", 2024-02-23T00:55:27.280 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.9049999999999992e-06 2024-02-23T00:55:27.280 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.280 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.280 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:27.281 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:35.473439+0000", 2024-02-23T00:55:27.281 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.896e-06 2024-02-23T00:55:27.281 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.281 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.281 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:27.281 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:35.473445+0000", 2024-02-23T00:55:27.281 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0029247240000000001 2024-02-23T00:55:27.281 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.281 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.281 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.281 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:35.476370+0000", 2024-02-23T00:55:27.281 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.9680000000000001e-06 2024-02-23T00:55:27.281 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.281 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.282 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:27.282 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:35.476380+0000", 2024-02-23T00:55:27.282 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.762e-05 2024-02-23T00:55:27.282 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.282 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.282 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:27.282 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:35.476407+0000", 2024-02-23T00:55:27.282 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0029861169999999999 2024-02-23T00:55:27.282 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.282 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:27.282 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.282 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.282 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.282 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.41771.0:1533 245.4 245:20000000:test-rados-api-smithi047-82465-123:::head [pgnls start_epoch 704] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e704)", 2024-02-23T00:55:27.282 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:54:03.347433+0000", 2024-02-23T00:55:27.283 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 83.876005599999999, 2024-02-23T00:55:27.283 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0043140749999999997, 2024-02-23T00:55:27.283 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:27.283 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:55:27.283 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:55:27.283 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.41771", 2024-02-23T00:55:27.283 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1032768195", 2024-02-23T00:55:27.283 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1533 2024-02-23T00:55:27.283 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.283 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:27.283 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.283 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:27.283 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.347433+0000", 2024-02-23T00:55:27.283 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:27.283 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.284 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.284 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:27.284 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.347433+0000", 2024-02-23T00:55:27.284 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999971 2024-02-23T00:55:27.284 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.284 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.284 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:27.284 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.347430+0000", 2024-02-23T00:55:27.284 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6416999999999999e-05 2024-02-23T00:55:27.284 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.284 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.285 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:27.285 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.347446+0000", 2024-02-23T00:55:27.285 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5664e-05 2024-02-23T00:55:27.285 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.285 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.285 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:27.285 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.347472+0000", 2024-02-23T00:55:27.285 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.3959999999999995e-06 2024-02-23T00:55:27.285 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.285 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.285 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:27.285 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.347481+0000", 2024-02-23T00:55:27.285 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4196999999999995e-05 2024-02-23T00:55:27.285 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.286 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.286 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.286 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.347545+0000", 2024-02-23T00:55:27.286 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1930000000000004e-06 2024-02-23T00:55:27.286 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.286 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.286 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op must wait for map", 2024-02-23T00:55:27.286 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.347553+0000", 2024-02-23T00:55:27.286 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0040895810000000001 2024-02-23T00:55:27.286 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.286 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.286 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.286 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.351642+0000", 2024-02-23T00:55:27.286 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.6699e-05 2024-02-23T00:55:27.287 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.287 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.287 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:27.287 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.351669+0000", 2024-02-23T00:55:27.287 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.7729000000000001e-05 2024-02-23T00:55:27.287 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.287 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.287 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:27.287 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.351747+0000", 2024-02-23T00:55:27.287 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0043140749999999997 2024-02-23T00:55:27.287 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.287 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:27.287 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.288 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.288 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.288 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.41771.0:1619 245.4 245:20000000::::head [pgnls start_epoch 709] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e709)", 2024-02-23T00:55:27.288 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:54:08.302221+0000", 2024-02-23T00:55:27.288 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 78.921217591000001, 2024-02-23T00:55:27.288 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0024817860000000001, 2024-02-23T00:55:27.288 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:27.288 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:55:27.288 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:55:27.288 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.41771", 2024-02-23T00:55:27.288 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1032768195", 2024-02-23T00:55:27.288 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1619 2024-02-23T00:55:27.288 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.288 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:27.289 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.289 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:27.289 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:08.302221+0000", 2024-02-23T00:55:27.289 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:27.289 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.289 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.289 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:27.289 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:08.302221+0000", 2024-02-23T00:55:27.289 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999905 2024-02-23T00:55:27.289 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.289 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.289 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:27.289 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:08.302211+0000", 2024-02-23T00:55:27.289 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2161e-05 2024-02-23T00:55:27.290 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.290 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.290 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:27.290 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:08.302233+0000", 2024-02-23T00:55:27.290 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7458999999999999e-05 2024-02-23T00:55:27.290 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.290 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.290 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:27.290 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:08.302261+0000", 2024-02-23T00:55:27.290 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.048e-05 2024-02-23T00:55:27.290 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.290 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.290 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:27.290 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:08.302271+0000", 2024-02-23T00:55:27.291 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000167959 2024-02-23T00:55:27.291 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.291 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.291 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.291 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:08.302439+0000", 2024-02-23T00:55:27.291 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1690000000000003e-06 2024-02-23T00:55:27.291 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.291 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.291 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op must wait for map", 2024-02-23T00:55:27.291 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:08.302447+0000", 2024-02-23T00:55:27.291 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0021327859999999998 2024-02-23T00:55:27.291 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.291 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.292 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.292 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:08.304579+0000", 2024-02-23T00:55:27.292 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.1848999999999999e-05 2024-02-23T00:55:27.292 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.292 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.292 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:27.292 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:08.304611+0000", 2024-02-23T00:55:27.292 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1242999999999994e-05 2024-02-23T00:55:27.292 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.292 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.292 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:27.292 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:08.304702+0000", 2024-02-23T00:55:27.292 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0024817860000000001 2024-02-23T00:55:27.292 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.293 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:27.293 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.293 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.293 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.293 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.41771.0:1794 247.11 e723/719 247:8937cf19:test-rados-api-smithi047-82465-127::foo:head v 723'2, mlcod=0'0)", 2024-02-23T00:55:27.293 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:54:22.377713+0000", 2024-02-23T00:55:27.293 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 64.845725119999997, 2024-02-23T00:55:27.293 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.016273069000000001, 2024-02-23T00:55:27.293 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:27.293 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:55:27.293 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:27.293 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.293 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:27.293 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:22.377713+0000", 2024-02-23T00:55:27.293 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:27.294 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.294 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.294 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:27.294 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:22.377713+0000", 2024-02-23T00:55:27.294 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:55:27.294 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.294 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.294 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:27.294 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:22.377712+0000", 2024-02-23T00:55:27.294 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078924689999999992 2024-02-23T00:55:27.294 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.294 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.294 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:27.294 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:22.385604+0000", 2024-02-23T00:55:27.295 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9612000000000001e-05 2024-02-23T00:55:27.295 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.295 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.295 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:27.295 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:22.385624+0000", 2024-02-23T00:55:27.295 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9300000000000003e-06 2024-02-23T00:55:27.295 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.295 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.295 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:27.295 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:22.385632+0000", 2024-02-23T00:55:27.295 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7929000000000001e-05 2024-02-23T00:55:27.295 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.295 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.295 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.295 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:22.385680+0000", 2024-02-23T00:55:27.296 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.3130999999999998e-05 2024-02-23T00:55:27.296 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.296 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.296 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:27.296 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:22.385713+0000", 2024-02-23T00:55:27.296 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0082255520000000006 2024-02-23T00:55:27.296 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.296 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.296 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:55:27.296 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:22.393939+0000", 2024-02-23T00:55:27.296 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7549000000000002e-05 2024-02-23T00:55:27.296 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.296 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.297 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:27.297 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:22.393986+0000", 2024-02-23T00:55:27.297 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.016273069000000001 2024-02-23T00:55:27.297 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.297 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:27.297 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.297 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.297 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.297 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "MOSDECSubOpWrite(234.1s2 723/676 ECSubWrite(tid=836, reqid=osd.1.13:108, at_version=723'5, trim_to=0'0, roll_forward_to=675'3))", 2024-02-23T00:55:27.297 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:54:22.416089+0000", 2024-02-23T00:55:27.297 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 64.807348829000006, 2024-02-23T00:55:27.297 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0032845959999999999, 2024-02-23T00:55:27.297 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:27.297 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "reached pg", 2024-02-23T00:55:27.297 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:27.298 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.298 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:27.298 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:22.416089+0000", 2024-02-23T00:55:27.298 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:27.298 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.298 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.298 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:27.298 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:22.416089+0000", 2024-02-23T00:55:27.298 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967296 2024-02-23T00:55:27.298 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.298 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.298 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:27.298 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:22.416089+0000", 2024-02-23T00:55:27.298 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00059299600000000004 2024-02-23T00:55:27.299 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.299 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.299 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:27.299 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:22.416682+0000", 2024-02-23T00:55:27.299 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5812999999999999e-05 2024-02-23T00:55:27.299 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.299 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.299 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:27.299 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:22.416698+0000", 2024-02-23T00:55:27.299 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7236000000000001e-05 2024-02-23T00:55:27.299 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.299 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.299 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:27.299 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:22.416735+0000", 2024-02-23T00:55:27.299 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8781000000000002e-05 2024-02-23T00:55:27.300 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.300 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.300 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.300 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:22.416834+0000", 2024-02-23T00:55:27.300 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7972999999999999e-05 2024-02-23T00:55:27.300 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.300 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.300 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:55:27.300 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:22.416852+0000", 2024-02-23T00:55:27.300 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0024892510000000001 2024-02-23T00:55:27.300 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.300 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.300 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:55:27.300 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:22.419341+0000", 2024-02-23T00:55:27.301 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.2685999999999998e-05 2024-02-23T00:55:27.301 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.301 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.301 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:27.301 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:22.419374+0000", 2024-02-23T00:55:27.301 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0032845959999999999 2024-02-23T00:55:27.301 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.301 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:27.301 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.301 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.301 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.301 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.41771.0:1998 249.6 249:602f83fe:::foo:head [append 0~8192] snapc 0=[] ondisk+write+known_if_redirected e741)", 2024-02-23T00:55:27.301 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:54:41.304293+0000", 2024-02-23T00:55:27.301 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 45.919144772999999, 2024-02-23T00:55:27.301 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0052413640000000001, 2024-02-23T00:55:27.302 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:27.302 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:55:27.302 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:55:27.302 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.41771", 2024-02-23T00:55:27.302 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1032768195", 2024-02-23T00:55:27.302 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1998 2024-02-23T00:55:27.302 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.302 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:27.302 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.302 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:27.302 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:41.304293+0000", 2024-02-23T00:55:27.302 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:27.302 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.302 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.302 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:27.303 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:41.304293+0000", 2024-02-23T00:55:27.303 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999971 2024-02-23T00:55:27.303 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.303 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.303 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:27.303 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:41.304291+0000", 2024-02-23T00:55:27.303 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8097e-05 2024-02-23T00:55:27.303 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.303 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.303 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:27.303 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:41.304309+0000", 2024-02-23T00:55:27.303 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8564999999999999e-05 2024-02-23T00:55:27.303 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.303 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.303 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:27.304 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:41.304327+0000", 2024-02-23T00:55:27.304 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.8080000000000005e-06 2024-02-23T00:55:27.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.304 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.304 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:27.304 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:41.304335+0000", 2024-02-23T00:55:27.304 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6338999999999997e-05 2024-02-23T00:55:27.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.304 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.304 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.304 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:41.304392+0000", 2024-02-23T00:55:27.304 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8979999999999999e-06 2024-02-23T00:55:27.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.304 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.305 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op must wait for map", 2024-02-23T00:55:27.305 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:41.304399+0000", 2024-02-23T00:55:27.305 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.003800997 2024-02-23T00:55:27.305 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.305 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.305 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.305 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:41.308200+0000", 2024-02-23T00:55:27.305 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014876700000000001 2024-02-23T00:55:27.305 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.305 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.305 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:27.305 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:41.308348+0000", 2024-02-23T00:55:27.307 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00019090099999999999 2024-02-23T00:55:27.308 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.308 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.308 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1,3", 2024-02-23T00:55:27.308 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:41.308539+0000", 2024-02-23T00:55:27.308 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00048634999999999998 2024-02-23T00:55:27.308 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.308 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.308 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:55:27.308 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:41.309026+0000", 2024-02-23T00:55:27.308 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00042800100000000001 2024-02-23T00:55:27.308 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.308 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.309 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:55:27.309 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:41.309454+0000", 2024-02-23T00:55:27.309 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5636e-05 2024-02-23T00:55:27.309 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.309 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.309 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:55:27.309 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:41.309479+0000", 2024-02-23T00:55:27.309 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.1001999999999999e-05 2024-02-23T00:55:27.309 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.309 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.309 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:55:27.309 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:41.309510+0000", 2024-02-23T00:55:27.309 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4657000000000001e-05 2024-02-23T00:55:27.309 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.310 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.310 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:27.310 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:41.309535+0000", 2024-02-23T00:55:27.310 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0052413640000000001 2024-02-23T00:55:27.310 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.310 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:27.310 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.310 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.310 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.310 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.41771.0:2004 249.6 249:602f83fe:::foo:head [append 0~8192] snapc 0=[] ondisk+write+known_if_redirected e741)", 2024-02-23T00:55:27.310 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:54:44.315824+0000", 2024-02-23T00:55:27.310 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 42.907614090000003, 2024-02-23T00:55:27.310 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001862137, 2024-02-23T00:55:27.310 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:27.311 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:55:27.311 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:55:27.311 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.41771", 2024-02-23T00:55:27.311 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1032768195", 2024-02-23T00:55:27.311 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 2004 2024-02-23T00:55:27.311 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.311 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:27.311 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.311 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:27.311 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:44.315824+0000", 2024-02-23T00:55:27.311 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:27.311 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.311 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.311 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:27.312 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:44.315824+0000", 2024-02-23T00:55:27.312 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999976 2024-02-23T00:55:27.312 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.312 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.312 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:27.312 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:44.315822+0000", 2024-02-23T00:55:27.312 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0231e-05 2024-02-23T00:55:27.312 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.312 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.312 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:27.312 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:44.315842+0000", 2024-02-23T00:55:27.312 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9845000000000001e-05 2024-02-23T00:55:27.312 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.313 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.313 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:27.313 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:44.315862+0000", 2024-02-23T00:55:27.313 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.6559999999999998e-06 2024-02-23T00:55:27.313 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.313 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.313 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:27.313 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:44.315870+0000", 2024-02-23T00:55:27.313 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6212999999999999e-05 2024-02-23T00:55:27.313 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.313 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.313 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.313 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:44.315927+0000", 2024-02-23T00:55:27.313 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00072068000000000002 2024-02-23T00:55:27.313 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.314 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.314 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:27.314 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:44.316647+0000", 2024-02-23T00:55:27.314 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016980900000000001 2024-02-23T00:55:27.314 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.314 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.314 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1,3", 2024-02-23T00:55:27.314 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:44.316817+0000", 2024-02-23T00:55:27.314 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038068599999999998 2024-02-23T00:55:27.314 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.314 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.314 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:55:27.314 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:44.317198+0000", 2024-02-23T00:55:27.314 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040549500000000002 2024-02-23T00:55:27.315 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.315 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.315 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:55:27.315 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:44.317603+0000", 2024-02-23T00:55:27.315 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2742e-05 2024-02-23T00:55:27.315 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.315 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.315 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:55:27.315 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:44.317626+0000", 2024-02-23T00:55:27.315 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.3875999999999999e-05 2024-02-23T00:55:27.315 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.315 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.315 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:55:27.315 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:44.317660+0000", 2024-02-23T00:55:27.315 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.6307000000000001e-05 2024-02-23T00:55:27.316 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.316 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.316 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:27.316 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:44.317686+0000", 2024-02-23T00:55:27.316 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001862137 2024-02-23T00:55:27.316 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.316 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:27.316 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.316 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.316 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.316 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.41771.0:2010 249.6 249:602f83fe:::foo:head [append 0~8192] snapc 0=[] ondisk+write+known_if_redirected e743)", 2024-02-23T00:55:27.316 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:54:47.323256+0000", 2024-02-23T00:55:27.316 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 39.900182645000001, 2024-02-23T00:55:27.316 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0018355820000000001, 2024-02-23T00:55:27.317 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:27.317 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:55:27.317 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:55:27.317 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.41771", 2024-02-23T00:55:27.317 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1032768195", 2024-02-23T00:55:27.317 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 2010 2024-02-23T00:55:27.317 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.317 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:27.317 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.317 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:27.317 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:47.323256+0000", 2024-02-23T00:55:27.317 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:27.317 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.317 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.318 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:27.318 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:47.323256+0000", 2024-02-23T00:55:27.318 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999967 2024-02-23T00:55:27.318 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.318 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.318 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:27.318 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:47.323252+0000", 2024-02-23T00:55:27.318 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.0766000000000003e-05 2024-02-23T00:55:27.318 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.318 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.318 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:27.318 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:47.323283+0000", 2024-02-23T00:55:27.318 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.2734999999999997e-05 2024-02-23T00:55:27.318 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.318 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.319 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:27.319 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:47.323316+0000", 2024-02-23T00:55:27.319 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4406999999999999e-05 2024-02-23T00:55:27.319 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.319 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.319 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:27.319 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:47.323330+0000", 2024-02-23T00:55:27.319 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1907999999999998e-05 2024-02-23T00:55:27.319 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.319 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.319 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.319 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:47.323382+0000", 2024-02-23T00:55:27.319 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00072679400000000001 2024-02-23T00:55:27.319 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.320 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.320 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:27.320 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:47.324109+0000", 2024-02-23T00:55:27.320 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000179455 2024-02-23T00:55:27.320 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.320 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.320 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1,3", 2024-02-23T00:55:27.320 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:47.324288+0000", 2024-02-23T00:55:27.320 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00042773099999999999 2024-02-23T00:55:27.320 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.320 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.320 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:55:27.320 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:47.324716+0000", 2024-02-23T00:55:27.320 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000242117 2024-02-23T00:55:27.321 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.321 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.321 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:55:27.321 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:47.324958+0000", 2024-02-23T00:55:27.321 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.5079000000000001e-05 2024-02-23T00:55:27.321 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.321 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.321 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:55:27.321 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:47.324993+0000", 2024-02-23T00:55:27.321 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3287e-05 2024-02-23T00:55:27.321 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.321 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.321 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:55:27.321 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:47.325046+0000", 2024-02-23T00:55:27.321 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.4691000000000003e-05 2024-02-23T00:55:27.322 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.322 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.322 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:27.322 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:47.325091+0000", 2024-02-23T00:55:27.322 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0018355820000000001 2024-02-23T00:55:27.322 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.322 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:27.322 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.322 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.322 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.322 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.41771.0:2216 250.14 250:28000000:%01:::head [pgnls start_epoch 759] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e759)", 2024-02-23T00:55:27.322 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:55:07.348147+0000", 2024-02-23T00:55:27.322 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 19.875290807999999, 2024-02-23T00:55:27.322 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0022570189999999999, 2024-02-23T00:55:27.323 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:27.323 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:55:27.323 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:55:27.323 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.41771", 2024-02-23T00:55:27.323 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1032768195", 2024-02-23T00:55:27.323 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 2216 2024-02-23T00:55:27.323 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.323 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:27.323 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.323 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:27.323 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:07.348147+0000", 2024-02-23T00:55:27.323 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:27.323 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.323 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.323 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:27.324 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:07.348147+0000", 2024-02-23T00:55:27.324 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:55:27.324 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.324 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.324 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:27.324 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:07.348145+0000", 2024-02-23T00:55:27.324 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2257e-05 2024-02-23T00:55:27.324 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.324 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.324 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:27.324 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:07.348158+0000", 2024-02-23T00:55:27.324 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0169999999999998e-05 2024-02-23T00:55:27.324 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.324 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.325 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:27.325 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:07.348178+0000", 2024-02-23T00:55:27.325 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9430000000000006e-06 2024-02-23T00:55:27.325 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.325 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.325 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:27.325 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:07.348187+0000", 2024-02-23T00:55:27.325 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2175000000000006e-05 2024-02-23T00:55:27.325 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.325 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.325 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.325 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:07.348259+0000", 2024-02-23T00:55:27.325 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6919999999999996e-06 2024-02-23T00:55:27.325 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.326 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.326 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op must wait for map", 2024-02-23T00:55:27.326 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:07.348266+0000", 2024-02-23T00:55:27.326 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.002050882 2024-02-23T00:55:27.326 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.326 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.326 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.326 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:07.350317+0000", 2024-02-23T00:55:27.326 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2843000000000001e-05 2024-02-23T00:55:27.326 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.326 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.326 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:27.326 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:07.350339+0000", 2024-02-23T00:55:27.327 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5005000000000005e-05 2024-02-23T00:55:27.327 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.327 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.327 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:27.327 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:07.350404+0000", 2024-02-23T00:55:27.327 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0022570189999999999 2024-02-23T00:55:27.327 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.327 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:27.327 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.327 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.327 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.327 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "MOSDECSubOpWrite(234.5s1 762/747 ECSubWrite(tid=865, reqid=client.41771.0:2312, at_version=762'16, trim_to=0'0, roll_forward_to=748'15))", 2024-02-23T00:55:27.327 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:55:10.322542+0000", 2024-02-23T00:55:27.327 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 16.900896577000001, 2024-02-23T00:55:27.328 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072066489999999999, 2024-02-23T00:55:27.328 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:27.328 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "reached pg", 2024-02-23T00:55:27.328 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:27.328 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.328 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:27.328 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:10.322542+0000", 2024-02-23T00:55:27.328 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:27.328 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.328 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.328 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:27.328 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:10.322542+0000", 2024-02-23T00:55:27.329 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999995 2024-02-23T00:55:27.329 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.329 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.329 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:27.329 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:10.322541+0000", 2024-02-23T00:55:27.329 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5562e-05 2024-02-23T00:55:27.329 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.329 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.329 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:27.329 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:10.322567+0000", 2024-02-23T00:55:27.329 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2034e-05 2024-02-23T00:55:27.329 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.329 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.330 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:27.330 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:10.322579+0000", 2024-02-23T00:55:27.330 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.8619999999999992e-06 2024-02-23T00:55:27.330 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.330 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.330 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:27.330 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:10.322588+0000", 2024-02-23T00:55:27.330 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033073009999999999 2024-02-23T00:55:27.330 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.330 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.330 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.330 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:10.325895+0000", 2024-02-23T00:55:27.331 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1319e-05 2024-02-23T00:55:27.331 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.331 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.331 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:55:27.331 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:10.325906+0000", 2024-02-23T00:55:27.331 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0038207750000000002 2024-02-23T00:55:27.331 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.331 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.331 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:55:27.331 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:10.329727+0000", 2024-02-23T00:55:27.331 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1118999999999999e-05 2024-02-23T00:55:27.331 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.331 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.332 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:27.332 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:10.329748+0000", 2024-02-23T00:55:27.332 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072066489999999999 2024-02-23T00:55:27.332 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.332 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:27.332 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.332 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.332 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.332 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.24311.0:2286 5.23 5:c609908c:::notify.5:head [watch reconnect cookie 94587070403072 gen 3] snapc 0=[] ondisk+write+known_if_redirected e772)", 2024-02-23T00:55:27.332 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:55:23.027293+0000", 2024-02-23T00:55:27.332 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 4.1961456830000001, 2024-02-23T00:55:27.332 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001248, 2024-02-23T00:55:27.332 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:27.332 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:55:27.332 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:55:27.333 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.24311", 2024-02-23T00:55:27.333 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/3723202188", 2024-02-23T00:55:27.333 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 2286 2024-02-23T00:55:27.333 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.333 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:27.333 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.333 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:27.333 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:23.027293+0000", 2024-02-23T00:55:27.333 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:27.333 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.333 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.333 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:27.333 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:23.027293+0000", 2024-02-23T00:55:27.333 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:55:27.333 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.334 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.334 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:27.334 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:23.027292+0000", 2024-02-23T00:55:27.334 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1189999999999996e-06 2024-02-23T00:55:27.334 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.334 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.334 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:27.334 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:23.027297+0000", 2024-02-23T00:55:27.334 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0312999999999999e-05 2024-02-23T00:55:27.334 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.334 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.334 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:27.334 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:23.027307+0000", 2024-02-23T00:55:27.334 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7369999999999997e-06 2024-02-23T00:55:27.335 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.335 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.335 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:27.335 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:23.027312+0000", 2024-02-23T00:55:27.335 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.6321e-05 2024-02-23T00:55:27.335 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.335 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.335 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.335 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:23.027348+0000", 2024-02-23T00:55:27.335 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3361000000000004e-05 2024-02-23T00:55:27.335 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.335 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.335 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:27.335 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:23.027432+0000", 2024-02-23T00:55:27.335 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001108993 2024-02-23T00:55:27.336 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.336 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.336 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:27.336 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:23.028541+0000", 2024-02-23T00:55:27.336 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001248 2024-02-23T00:55:27.336 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.336 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:27.336 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.336 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.336 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.336 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.24311.0:2289 5.23 5:c609908c:::notify.5:head [watch reconnect cookie 94587070403072 gen 4] snapc 0=[] ondisk+write+known_if_redirected e772)", 2024-02-23T00:55:27.336 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:55:23.027320+0000", 2024-02-23T00:55:27.336 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 4.1961184820000001, 2024-02-23T00:55:27.336 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00133399, 2024-02-23T00:55:27.337 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:27.337 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:55:27.337 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:55:27.337 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.24311", 2024-02-23T00:55:27.337 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/3723202188", 2024-02-23T00:55:27.337 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 2289 2024-02-23T00:55:27.337 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.337 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:27.337 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.337 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:27.337 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:23.027320+0000", 2024-02-23T00:55:27.337 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:27.337 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.337 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.337 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:27.338 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:23.027320+0000", 2024-02-23T00:55:27.338 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999995 2024-02-23T00:55:27.338 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.338 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.338 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:27.338 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:23.027319+0000", 2024-02-23T00:55:27.338 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0379999999999998e-06 2024-02-23T00:55:27.338 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.338 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.338 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:27.338 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:23.027321+0000", 2024-02-23T00:55:27.338 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5709999999999999e-06 2024-02-23T00:55:27.338 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.338 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.339 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:27.339 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:23.027326+0000", 2024-02-23T00:55:27.339 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.9179999999999998e-06 2024-02-23T00:55:27.339 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.339 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.339 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:27.339 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:23.027329+0000", 2024-02-23T00:55:27.339 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033795199999999998 2024-02-23T00:55:27.339 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.339 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.339 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:27.339 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:23.027667+0000", 2024-02-23T00:55:27.339 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4610000000000006e-05 2024-02-23T00:55:27.340 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.340 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.340 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:27.340 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:23.027741+0000", 2024-02-23T00:55:27.340 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00091231999999999999 2024-02-23T00:55:27.340 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:27.340 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:27.340 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:27.340 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:23.028654+0000", 2024-02-23T00:55:27.340 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00133399 2024-02-23T00:55:27.340 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.340 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:27.340 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.340 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:27.340 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:27.341 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:55:27.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:27.326+0000 7efde7779700 1 -- 172.21.15.47:0/3438124817 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v776) v1 ==== 101+0+0 (secure 0 0 0) 0x7efde00502b0 con 0x7efdf00fdbd0 2024-02-23T00:55:27.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:27.330+0000 7efdf82ae700 10 librados: watch_flush enter 2024-02-23T00:55:27.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:27.330+0000 7efdf82ae700 10 librados: watch_flush exit 2024-02-23T00:55:27.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:27.330+0000 7efdf82ae700 20 client.42116.objecter shutdown clearing up homeless session... 2024-02-23T00:55:27.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:27.330+0000 7efdf82ae700 10 client.42116.objecter successfully canceled tick 2024-02-23T00:55:27.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:27.331+0000 7efdf82ae700 1 -- 172.21.15.47:0/3438124817 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7efddc062cc0 msgr2=0x7efddc065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:27.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:27.331+0000 7efdf82ae700 1 --2- 172.21.15.47:0/3438124817 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7efddc062cc0 0x7efddc065180 secure :-1 s=READY pgs=3203 cs=0 l=1 rev1=1 rx=0x7efdec0048b0 tx=0x7efdec00b040).stop 2024-02-23T00:55:27.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:27.331+0000 7efdf82ae700 1 -- 172.21.15.47:0/3438124817 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efdf00fdbd0 msgr2=0x7efdf00f7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:27.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:27.331+0000 7efdf82ae700 1 --2- 172.21.15.47:0/3438124817 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efdf00fdbd0 0x7efdf00f7b70 secure :-1 s=READY pgs=2814 cs=0 l=1 rev1=1 rx=0x7efde00041d0 tx=0x7efde0009f60).stop 2024-02-23T00:55:27.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:27.331+0000 7efdf82ae700 1 -- 172.21.15.47:0/3438124817 shutdown_connections 2024-02-23T00:55:27.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:27.331+0000 7efdf82ae700 1 --2- 172.21.15.47:0/3438124817 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdf00fe530 0x7efdf00f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:27.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:27.331+0000 7efdf82ae700 1 --2- 172.21.15.47:0/3438124817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efdf00f6a90 0x7efdf00f7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:27.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:27.331+0000 7efdf82ae700 1 --2- 172.21.15.47:0/3438124817 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7efddc062cc0 0x7efddc065180 unknown :-1 s=CLOSED pgs=3203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:27.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:27.331+0000 7efdf82ae700 1 --2- 172.21.15.47:0/3438124817 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efdf00fdbd0 0x7efdf00f7b70 unknown :-1 s=CLOSED pgs=2814 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:27.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:27.331+0000 7efdf82ae700 1 -- 172.21.15.47:0/3438124817 >> 172.21.15.47:0/3438124817 conn(0x7efdf006c370 msgr2=0x7efdf00f5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:27.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:27.332+0000 7efdf82ae700 1 -- 172.21.15.47:0/3438124817 shutdown_connections 2024-02-23T00:55:27.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:27.332+0000 7efdf82ae700 1 -- 172.21.15.47:0/3438124817 wait complete. 2024-02-23T00:55:27.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:27.332+0000 7efdf82ae700 1 librados: shutdown 2024-02-23T00:55:27.343 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is unset 2024-02-23T00:55:27.507 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:55:27.511 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:27.511 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:55:27.612 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:55:27.644 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:27.826 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:25.811139+0000 osd.0 (osd.0) 287 : cluster [DBG] 4.6 scrub starts 2024-02-23T00:55:27.827 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:25.812438+0000 osd.0 (osd.0) 288 : cluster [DBG] 4.6 scrub ok 2024-02-23T00:55:27.827 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:26.322944+0000 mon.a (mon.0) 3695 : cluster [DBG] osdmap e775: 8 total, 8 up, 7 in 2024-02-23T00:55:27.827 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:26.590056+0000 mon.c (mon.2) 481 : audit [INF] from='client.? 172.21.15.47:0/3438124817' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:55:27.827 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:26.738100+0000 osd.6 (osd.6) 303 : cluster [DBG] 18.1 scrub starts 2024-02-23T00:55:27.827 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:26.739528+0000 osd.6 (osd.6) 304 : cluster [DBG] 18.1 scrub ok 2024-02-23T00:55:27.827 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:26.775704+0000 mon.a (mon.0) 3696 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:55:27.827 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:27.320572+0000 mon.a (mon.0) 3697 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:55:27.827 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:27.324847+0000 mon.a (mon.0) 3698 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:55:27.827 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:27.324897+0000 mon.a (mon.0) 3699 : cluster [DBG] osdmap e776: 8 total, 8 up, 7 in 2024-02-23T00:55:27.827 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:27.328299+0000 mon.c (mon.2) 482 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:55:27.827 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:27.329000+0000 mon.a (mon.0) 3700 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:55:27.827 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:27.400704+0000 mgr.y (mgr.14517) 660 : cluster [DBG] pgmap v1179: 194 pgs: 32 unknown, 162 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 92 B/s, 0 objects/s recovering 2024-02-23T00:55:27.828 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.5 dump_ops_in_flight 2024-02-23T00:55:27.843 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:55:27.855 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:27.855 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:55:27.956 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:55:27.985 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:28.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:25.811139+0000 osd.0 (osd.0) 287 : cluster [DBG] 4.6 scrub starts 2024-02-23T00:55:28.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:25.812438+0000 osd.0 (osd.0) 288 : cluster [DBG] 4.6 scrub ok 2024-02-23T00:55:28.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:26.322944+0000 mon.a (mon.0) 3695 : cluster [DBG] osdmap e775: 8 total, 8 up, 7 in 2024-02-23T00:55:28.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:26.590056+0000 mon.c (mon.2) 481 : audit [INF] from='client.? 172.21.15.47:0/3438124817' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:55:28.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:26.738100+0000 osd.6 (osd.6) 303 : cluster [DBG] 18.1 scrub starts 2024-02-23T00:55:28.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:26.739528+0000 osd.6 (osd.6) 304 : cluster [DBG] 18.1 scrub ok 2024-02-23T00:55:28.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:26.775704+0000 mon.a (mon.0) 3696 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:55:28.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:27.320572+0000 mon.a (mon.0) 3697 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:55:28.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:27.324847+0000 mon.a (mon.0) 3698 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:55:28.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:27.324897+0000 mon.a (mon.0) 3699 : cluster [DBG] osdmap e776: 8 total, 8 up, 7 in 2024-02-23T00:55:28.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:27.328299+0000 mon.c (mon.2) 482 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:55:28.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:27.329000+0000 mon.a (mon.0) 3700 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:55:28.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:27.400704+0000 mgr.y (mgr.14517) 660 : cluster [DBG] pgmap v1179: 194 pgs: 32 unknown, 162 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 92 B/s, 0 objects/s recovering 2024-02-23T00:55:28.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:25.811139+0000 osd.0 (osd.0) 287 : cluster [DBG] 4.6 scrub starts 2024-02-23T00:55:28.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:25.812438+0000 osd.0 (osd.0) 288 : cluster [DBG] 4.6 scrub ok 2024-02-23T00:55:28.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:26.322944+0000 mon.a (mon.0) 3695 : cluster [DBG] osdmap e775: 8 total, 8 up, 7 in 2024-02-23T00:55:28.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:26.590056+0000 mon.c (mon.2) 481 : audit [INF] from='client.? 172.21.15.47:0/3438124817' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:55:28.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:26.738100+0000 osd.6 (osd.6) 303 : cluster [DBG] 18.1 scrub starts 2024-02-23T00:55:28.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:26.739528+0000 osd.6 (osd.6) 304 : cluster [DBG] 18.1 scrub ok 2024-02-23T00:55:28.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:26.775704+0000 mon.a (mon.0) 3696 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:55:28.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:27.320572+0000 mon.a (mon.0) 3697 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:55:28.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:27.324847+0000 mon.a (mon.0) 3698 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:55:28.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:27.324897+0000 mon.a (mon.0) 3699 : cluster [DBG] osdmap e776: 8 total, 8 up, 7 in 2024-02-23T00:55:28.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:27.328299+0000 mon.c (mon.2) 482 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:55:28.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:27.329000+0000 mon.a (mon.0) 3700 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:55:28.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:27.400704+0000 mgr.y (mgr.14517) 660 : cluster [DBG] pgmap v1179: 194 pgs: 32 unknown, 162 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 92 B/s, 0 objects/s recovering 2024-02-23T00:55:28.098 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:55:28.184 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:55:28.187 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:28.187 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:55:28.288 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:55:28.318 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:28.424 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:55:28.427 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:55:28.430 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:28.430 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:55:28.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.432+0000 7f0b96e94700 1 -- 172.21.15.47:0/861259598 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0b900f7330 msgr2=0x7f0b90103930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:28.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.432+0000 7f0b96e94700 1 --2- 172.21.15.47:0/861259598 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0b900f7330 0x7f0b90103930 secure :-1 s=READY pgs=2815 cs=0 l=1 rev1=1 rx=0x7f0b88009be0 tx=0x7f0b88004d10).stop 2024-02-23T00:55:28.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.432+0000 7f0b96e94700 1 -- 172.21.15.47:0/861259598 shutdown_connections 2024-02-23T00:55:28.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.432+0000 7f0b96e94700 1 --2- 172.21.15.47:0/861259598 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0b900f7330 0x7f0b90103930 unknown :-1 s=CLOSED pgs=2815 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:28.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.432+0000 7f0b96e94700 1 --2- 172.21.15.47:0/861259598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b900fcac0 0x7f0b900fcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:28.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.432+0000 7f0b96e94700 1 --2- 172.21.15.47:0/861259598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b900fd420 0x7f0b900f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:28.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.432+0000 7f0b96e94700 1 -- 172.21.15.47:0/861259598 >> 172.21.15.47:0/861259598 conn(0x7f0b900f2710 msgr2=0x7f0b900f4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:28.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.432+0000 7f0b96e94700 1 -- 172.21.15.47:0/861259598 shutdown_connections 2024-02-23T00:55:28.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.432+0000 7f0b96e94700 1 -- 172.21.15.47:0/861259598 wait complete. 2024-02-23T00:55:28.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.433+0000 7f0b96e94700 1 librados: starting msgr at 2024-02-23T00:55:28.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.433+0000 7f0b96e94700 1 librados: starting objecter 2024-02-23T00:55:28.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.433+0000 7f0b96e94700 1 Processor -- start 2024-02-23T00:55:28.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.433+0000 7f0b96e94700 1 -- start start 2024-02-23T00:55:28.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.433+0000 7f0b96e94700 1 librados: setting wanted keys 2024-02-23T00:55:28.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.433+0000 7f0b96e94700 1 librados: calling monclient init 2024-02-23T00:55:28.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.433+0000 7f0b96e94700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b900f7330 0x7f0b900f8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:28.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.433+0000 7f0b96e94700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0b900fcac0 0x7f0b900f9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:28.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.433+0000 7f0b96e94700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b900fd420 0x7f0b900f9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:28.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.433+0000 7f0b96e94700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b901162e0 con 0x7f0b900fd420 2024-02-23T00:55:28.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.433+0000 7f0b96e94700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b90116450 con 0x7f0b900f7330 2024-02-23T00:55:28.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.433+0000 7f0b96e94700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b90116610 con 0x7f0b900fcac0 2024-02-23T00:55:28.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.433+0000 7f0b8ffff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0b900fcac0 0x7f0b900f9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:28.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.433+0000 7f0b8ffff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0b900fcac0 0x7f0b900f9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:53930/0 (socket says 172.21.15.47:53930) 2024-02-23T00:55:28.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.434+0000 7f0b8ffff700 1 -- 172.21.15.47:0/2662679792 learned_addr learned my addr 172.21.15.47:0/2662679792 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:55:28.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.434+0000 7f0b95431700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b900fd420 0x7f0b900f9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:28.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.434+0000 7f0b94c30700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b900f7330 0x7f0b900f8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:28.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.434+0000 7f0b8ffff700 1 -- 172.21.15.47:0/2662679792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b900f7330 msgr2=0x7f0b900f8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:28.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.434+0000 7f0b8ffff700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b900f7330 0x7f0b900f8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:28.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.434+0000 7f0b8ffff700 1 -- 172.21.15.47:0/2662679792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b900fd420 msgr2=0x7f0b900f9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:28.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.434+0000 7f0b8ffff700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b900fd420 0x7f0b900f9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:28.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.434+0000 7f0b8ffff700 1 -- 172.21.15.47:0/2662679792 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b880097c0 con 0x7f0b900fcac0 2024-02-23T00:55:28.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.434+0000 7f0b95431700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b900fd420 0x7f0b900f9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:55:28.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.434+0000 7f0b94c30700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b900f7330 0x7f0b900f8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:55:28.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.434+0000 7f0b8ffff700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0b900fcac0 0x7f0b900f9160 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 rx=0x7f0b7c00e970 tx=0x7f0b7c00b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:28.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.434+0000 7f0b8dffb700 10 client.?.objecter ms_handle_connect 0x7f0b900fcac0 2024-02-23T00:55:28.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.434+0000 7f0b8dffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:55:28.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.434+0000 7f0b8dffb700 1 -- 172.21.15.47:0/2662679792 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c004660 con 0x7f0b900fcac0 2024-02-23T00:55:28.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.434+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b901168f0 con 0x7f0b900fcac0 2024-02-23T00:55:28.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.434+0000 7f0b8dffb700 1 -- 172.21.15.47:0/2662679792 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f0b7c00f040 con 0x7f0b900fcac0 2024-02-23T00:55:28.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.434+0000 7f0b8dffb700 1 -- 172.21.15.47:0/2662679792 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c0146f0 con 0x7f0b900fcac0 2024-02-23T00:55:28.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.435+0000 7f0b96e94700 10 client.42122.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:55:28.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.435+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b90116e30 con 0x7f0b900fcac0 2024-02-23T00:55:28.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.435+0000 7f0b96e94700 1 librados: init done 2024-02-23T00:55:28.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.435+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f0b900606d0 con 0x7f0b900fcac0 2024-02-23T00:55:28.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.436+0000 7f0b8dffb700 1 -- 172.21.15.47:0/2662679792 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f0b7c007b40 con 0x7f0b900fcac0 2024-02-23T00:55:28.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.436+0000 7f0b8dffb700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0b80062cc0 0x7f0b80065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:28.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.436+0000 7f0b8dffb700 1 -- 172.21.15.47:0/2662679792 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(777..777 src has 621..777) v4 ==== 9753+0+0 (secure 0 0 0) 0x7f0b7c07d2a0 con 0x7f0b900fcac0 2024-02-23T00:55:28.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.436+0000 7f0b8dffb700 10 client.42122.objecter ms_dispatch 0x7f0b90001550 osd_map(777..777 src has 621..777) v4 2024-02-23T00:55:28.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.436+0000 7f0b8dffb700 3 client.42122.objecter handle_osd_map got epochs [777,777] > 0 2024-02-23T00:55:28.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.436+0000 7f0b8dffb700 3 client.42122.objecter handle_osd_map decoding full epoch 777 2024-02-23T00:55:28.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.436+0000 7f0b8dffb700 20 client.42122.objecter dump_active .. 0 homeless 2024-02-23T00:55:28.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.436+0000 7f0b8dffb700 1 -- 172.21.15.47:0/2662679792 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v777) v1 ==== 54+0+16 (secure 0 0 0) 0x7f0b7c084070 con 0x7f0b900fcac0 2024-02-23T00:55:28.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.437+0000 7f0b94c30700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0b80062cc0 0x7f0b80065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:28.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.437+0000 7f0b96e94700 10 client.42122.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:28.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.437+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_assign -1 1 2024-02-23T00:55:28.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.437+0000 7f0b96e94700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f0b900ffd80 0x7f0b901001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:28.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.437+0000 7f0b96e94700 20 client.42122.objecter _get_session s=0x7f0b9005ee80 osd=0 3 2024-02-23T00:55:28.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.437+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b9005ee80 osd=0 3 2024-02-23T00:55:28.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.437+0000 7f0b96e94700 20 client.42122.objecter _get_session s=0x7f0b9005ee80 osd=0 3 2024-02-23T00:55:28.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.437+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_remove -1 1 2024-02-23T00:55:28.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.437+0000 7f0b96e94700 20 client.42122.objecter get_session s=0x7f0b9005ee80 osd=0 3 2024-02-23T00:55:28.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.437+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_assign 0 1 2024-02-23T00:55:28.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.437+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b9005ee80 osd=0 4 2024-02-23T00:55:28.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.437+0000 7f0b96e94700 10 client.42122.objecter _send_command 1 2024-02-23T00:55:28.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.437+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0b90105de0 con 0x7f0b900ffd80 2024-02-23T00:55:28.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.437+0000 7f0b8dffb700 10 client.42122.objecter ms_handle_connect 0x7f0b80062cc0 2024-02-23T00:55:28.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.437+0000 7f0b94c30700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0b80062cc0 0x7f0b80065180 secure :-1 s=READY pgs=3204 cs=0 l=1 rev1=1 rx=0x7f0b84007bf0 tx=0x7f0b8400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:28.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.437+0000 7f0b95431700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f0b900ffd80 0x7f0b901001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:28.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.437+0000 7f0b8dffb700 10 client.42122.objecter ms_handle_connect 0x7f0b900ffd80 2024-02-23T00:55:28.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.437+0000 7f0b95431700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f0b900ffd80 0x7f0b901001c0 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:28.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.438+0000 7f0b8dffb700 1 -- 172.21.15.47:0/2662679792 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0b90105de0 con 0x7f0b900ffd80 2024-02-23T00:55:28.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.438+0000 7f0b8dffb700 10 client.42122.objecter ms_dispatch 0x7f0b90001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:55:28.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.438+0000 7f0b8dffb700 10 client.42122.objecter _finish_command 1 = system:0 2024-02-23T00:55:28.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.438+0000 7f0b8dffb700 20 client.42122.objecter put_session s=0x7f0b9005ee80 osd=0 4 2024-02-23T00:55:28.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.438+0000 7f0b8dffb700 15 client.42122.objecter _session_command_op_remove 0 1 2024-02-23T00:55:28.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.454+0000 7f0b96e94700 10 client.42122.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:55:28.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.454+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_assign -1 2 2024-02-23T00:55:28.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.454+0000 7f0b96e94700 20 client.42122.objecter _get_session s=0x7f0b9005ee80 osd=0 3 2024-02-23T00:55:28.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.454+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b9005ee80 osd=0 3 2024-02-23T00:55:28.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.454+0000 7f0b96e94700 20 client.42122.objecter _get_session s=0x7f0b9005ee80 osd=0 3 2024-02-23T00:55:28.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.454+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_remove -1 2 2024-02-23T00:55:28.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.454+0000 7f0b96e94700 20 client.42122.objecter get_session s=0x7f0b9005ee80 osd=0 3 2024-02-23T00:55:28.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.454+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_assign 0 2 2024-02-23T00:55:28.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.454+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b9005ee80 osd=0 4 2024-02-23T00:55:28.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.454+0000 7f0b96e94700 10 client.42122.objecter _send_command 2 2024-02-23T00:55:28.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.454+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0b900273c0 con 0x7f0b900ffd80 2024-02-23T00:55:28.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.456+0000 7f0b8dffb700 1 -- 172.21.15.47:0/2662679792 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0b900273c0 con 0x7f0b900ffd80 2024-02-23T00:55:28.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.456+0000 7f0b8dffb700 10 client.42122.objecter ms_dispatch 0x7f0b90001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:55:28.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.456+0000 7f0b8dffb700 10 client.42122.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:55:28.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.456+0000 7f0b8dffb700 20 client.42122.objecter put_session s=0x7f0b9005ee80 osd=0 4 2024-02-23T00:55:28.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.456+0000 7f0b8dffb700 15 client.42122.objecter _session_command_op_remove 0 2 2024-02-23T00:55:28.461 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:55:28.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.456+0000 7f0b96e94700 10 client.42122.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:28.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.456+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_assign -1 3 2024-02-23T00:55:28.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.456+0000 7f0b96e94700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f0b900598f0 0x7f0b90059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:28.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.456+0000 7f0b96e94700 20 client.42122.objecter _get_session s=0x7f0b90028710 osd=1 3 2024-02-23T00:55:28.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.456+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b90028710 osd=1 3 2024-02-23T00:55:28.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.456+0000 7f0b96e94700 20 client.42122.objecter _get_session s=0x7f0b90028710 osd=1 3 2024-02-23T00:55:28.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.456+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_remove -1 3 2024-02-23T00:55:28.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.456+0000 7f0b96e94700 20 client.42122.objecter get_session s=0x7f0b90028710 osd=1 3 2024-02-23T00:55:28.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.456+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_assign 1 3 2024-02-23T00:55:28.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.456+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b90028710 osd=1 4 2024-02-23T00:55:28.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.456+0000 7f0b96e94700 10 client.42122.objecter _send_command 3 2024-02-23T00:55:28.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.456+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0b9011d7d0 con 0x7f0b900598f0 2024-02-23T00:55:28.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.456+0000 7f0b94c30700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f0b900598f0 0x7f0b90059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:28.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.457+0000 7f0b8dffb700 10 client.42122.objecter ms_handle_connect 0x7f0b900598f0 2024-02-23T00:55:28.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.457+0000 7f0b94c30700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f0b900598f0 0x7f0b90059cd0 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:28.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.457+0000 7f0b8dffb700 1 -- 172.21.15.47:0/2662679792 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0b9011d7d0 con 0x7f0b900598f0 2024-02-23T00:55:28.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.457+0000 7f0b8dffb700 10 client.42122.objecter ms_dispatch 0x7f0b90001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:55:28.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.457+0000 7f0b8dffb700 10 client.42122.objecter _finish_command 3 = system:0 2024-02-23T00:55:28.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.457+0000 7f0b8dffb700 20 client.42122.objecter put_session s=0x7f0b90028710 osd=1 4 2024-02-23T00:55:28.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.457+0000 7f0b8dffb700 15 client.42122.objecter _session_command_op_remove 1 3 2024-02-23T00:55:28.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.473+0000 7f0b96e94700 10 client.42122.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:55:28.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.473+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_assign -1 4 2024-02-23T00:55:28.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.473+0000 7f0b96e94700 20 client.42122.objecter _get_session s=0x7f0b90028710 osd=1 3 2024-02-23T00:55:28.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.473+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b90028710 osd=1 3 2024-02-23T00:55:28.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.473+0000 7f0b96e94700 20 client.42122.objecter _get_session s=0x7f0b90028710 osd=1 3 2024-02-23T00:55:28.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.473+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_remove -1 4 2024-02-23T00:55:28.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.473+0000 7f0b96e94700 20 client.42122.objecter get_session s=0x7f0b90028710 osd=1 3 2024-02-23T00:55:28.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.473+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_assign 1 4 2024-02-23T00:55:28.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.473+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b90028710 osd=1 4 2024-02-23T00:55:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.473+0000 7f0b96e94700 10 client.42122.objecter _send_command 4 2024-02-23T00:55:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.473+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0b90059360 con 0x7f0b900598f0 2024-02-23T00:55:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.475+0000 7f0b8dffb700 1 -- 172.21.15.47:0/2662679792 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0b90059360 con 0x7f0b900598f0 2024-02-23T00:55:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.475+0000 7f0b8dffb700 10 client.42122.objecter ms_dispatch 0x7f0b90001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:55:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.475+0000 7f0b8dffb700 10 client.42122.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:55:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.475+0000 7f0b8dffb700 20 client.42122.objecter put_session s=0x7f0b90028710 osd=1 4 2024-02-23T00:55:28.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.475+0000 7f0b8dffb700 15 client.42122.objecter _session_command_op_remove 1 4 2024-02-23T00:55:28.478 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:55:28.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.475+0000 7f0b96e94700 10 client.42122.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:28.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.475+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_assign -1 5 2024-02-23T00:55:28.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.475+0000 7f0b96e94700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f0b90028270 0x7f0b9011c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:28.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.475+0000 7f0b96e94700 20 client.42122.objecter _get_session s=0x7f0b90059770 osd=2 3 2024-02-23T00:55:28.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.475+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b90059770 osd=2 3 2024-02-23T00:55:28.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.475+0000 7f0b96e94700 20 client.42122.objecter _get_session s=0x7f0b90059770 osd=2 3 2024-02-23T00:55:28.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.475+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_remove -1 5 2024-02-23T00:55:28.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.475+0000 7f0b96e94700 20 client.42122.objecter get_session s=0x7f0b90059770 osd=2 3 2024-02-23T00:55:28.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.475+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_assign 2 5 2024-02-23T00:55:28.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.475+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b90059770 osd=2 4 2024-02-23T00:55:28.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.475+0000 7f0b96e94700 10 client.42122.objecter _send_command 5 2024-02-23T00:55:28.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.475+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0b9011c770 con 0x7f0b90028270 2024-02-23T00:55:28.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.475+0000 7f0b8ffff700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f0b90028270 0x7f0b9011c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:28.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.475+0000 7f0b8dffb700 10 client.42122.objecter ms_handle_connect 0x7f0b90028270 2024-02-23T00:55:28.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.475+0000 7f0b8ffff700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f0b90028270 0x7f0b9011c1f0 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:28.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.476+0000 7f0b8dffb700 1 -- 172.21.15.47:0/2662679792 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0b9011c770 con 0x7f0b90028270 2024-02-23T00:55:28.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.476+0000 7f0b8dffb700 10 client.42122.objecter ms_dispatch 0x7f0b90001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:55:28.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.476+0000 7f0b8dffb700 10 client.42122.objecter _finish_command 5 = system:0 2024-02-23T00:55:28.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.476+0000 7f0b8dffb700 20 client.42122.objecter put_session s=0x7f0b90059770 osd=2 4 2024-02-23T00:55:28.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.476+0000 7f0b8dffb700 15 client.42122.objecter _session_command_op_remove 2 5 2024-02-23T00:55:28.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.494+0000 7f0b96e94700 10 client.42122.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:55:28.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.494+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_assign -1 6 2024-02-23T00:55:28.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.494+0000 7f0b96e94700 20 client.42122.objecter _get_session s=0x7f0b90059770 osd=2 3 2024-02-23T00:55:28.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.494+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b90059770 osd=2 3 2024-02-23T00:55:28.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.494+0000 7f0b96e94700 20 client.42122.objecter _get_session s=0x7f0b90059770 osd=2 3 2024-02-23T00:55:28.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.494+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_remove -1 6 2024-02-23T00:55:28.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.494+0000 7f0b96e94700 20 client.42122.objecter get_session s=0x7f0b90059770 osd=2 3 2024-02-23T00:55:28.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.494+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_assign 2 6 2024-02-23T00:55:28.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.494+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b90059770 osd=2 4 2024-02-23T00:55:28.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.494+0000 7f0b96e94700 10 client.42122.objecter _send_command 6 2024-02-23T00:55:28.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.494+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0b900277d0 con 0x7f0b90028270 2024-02-23T00:55:28.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.496+0000 7f0b8dffb700 1 -- 172.21.15.47:0/2662679792 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0b900277d0 con 0x7f0b90028270 2024-02-23T00:55:28.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.496+0000 7f0b8dffb700 10 client.42122.objecter ms_dispatch 0x7f0b90001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:55:28.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.496+0000 7f0b8dffb700 10 client.42122.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:55:28.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.496+0000 7f0b8dffb700 20 client.42122.objecter put_session s=0x7f0b90059770 osd=2 4 2024-02-23T00:55:28.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.496+0000 7f0b8dffb700 15 client.42122.objecter _session_command_op_remove 2 6 2024-02-23T00:55:28.501 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:55:28.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.496+0000 7f0b96e94700 10 client.42122.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:28.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.496+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_assign -1 7 2024-02-23T00:55:28.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.496+0000 7f0b96e94700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f0b9011f950 0x7f0b90121d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:28.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.496+0000 7f0b96e94700 20 client.42122.objecter _get_session s=0x7f0b9011d140 osd=3 3 2024-02-23T00:55:28.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.496+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b9011d140 osd=3 3 2024-02-23T00:55:28.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.496+0000 7f0b96e94700 20 client.42122.objecter _get_session s=0x7f0b9011d140 osd=3 3 2024-02-23T00:55:28.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.496+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_remove -1 7 2024-02-23T00:55:28.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.496+0000 7f0b96e94700 20 client.42122.objecter get_session s=0x7f0b9011d140 osd=3 3 2024-02-23T00:55:28.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.496+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_assign 3 7 2024-02-23T00:55:28.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.496+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b9011d140 osd=3 4 2024-02-23T00:55:28.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.496+0000 7f0b96e94700 10 client.42122.objecter _send_command 7 2024-02-23T00:55:28.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.496+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0b90124b10 con 0x7f0b9011f950 2024-02-23T00:55:28.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.496+0000 7f0b95431700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f0b9011f950 0x7f0b90121d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:28.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.496+0000 7f0b8dffb700 10 client.42122.objecter ms_handle_connect 0x7f0b9011f950 2024-02-23T00:55:28.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.496+0000 7f0b95431700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f0b9011f950 0x7f0b90121d40 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:28.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.497+0000 7f0b8dffb700 1 -- 172.21.15.47:0/2662679792 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0b90124b10 con 0x7f0b9011f950 2024-02-23T00:55:28.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.497+0000 7f0b8dffb700 10 client.42122.objecter ms_dispatch 0x7f0b90001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:55:28.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.497+0000 7f0b8dffb700 10 client.42122.objecter _finish_command 7 = system:0 2024-02-23T00:55:28.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.497+0000 7f0b8dffb700 20 client.42122.objecter put_session s=0x7f0b9011d140 osd=3 4 2024-02-23T00:55:28.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.497+0000 7f0b8dffb700 15 client.42122.objecter _session_command_op_remove 3 7 2024-02-23T00:55:28.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.512+0000 7f0b96e94700 10 client.42122.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:55:28.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.512+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_assign -1 8 2024-02-23T00:55:28.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.512+0000 7f0b96e94700 20 client.42122.objecter _get_session s=0x7f0b9011d140 osd=3 3 2024-02-23T00:55:28.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.512+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b9011d140 osd=3 3 2024-02-23T00:55:28.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.512+0000 7f0b96e94700 20 client.42122.objecter _get_session s=0x7f0b9011d140 osd=3 3 2024-02-23T00:55:28.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.512+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_remove -1 8 2024-02-23T00:55:28.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.512+0000 7f0b96e94700 20 client.42122.objecter get_session s=0x7f0b9011d140 osd=3 3 2024-02-23T00:55:28.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.512+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_assign 3 8 2024-02-23T00:55:28.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.512+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b9011d140 osd=3 4 2024-02-23T00:55:28.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.512+0000 7f0b96e94700 10 client.42122.objecter _send_command 8 2024-02-23T00:55:28.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.512+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0b9011cbb0 con 0x7f0b9011f950 2024-02-23T00:55:28.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.513+0000 7f0b8dffb700 1 -- 172.21.15.47:0/2662679792 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0b9011cbb0 con 0x7f0b9011f950 2024-02-23T00:55:28.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.513+0000 7f0b8dffb700 10 client.42122.objecter ms_dispatch 0x7f0b90001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:55:28.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.513+0000 7f0b8dffb700 10 client.42122.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:55:28.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.513+0000 7f0b8dffb700 20 client.42122.objecter put_session s=0x7f0b9011d140 osd=3 4 2024-02-23T00:55:28.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.513+0000 7f0b8dffb700 15 client.42122.objecter _session_command_op_remove 3 8 2024-02-23T00:55:28.518 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:55:28.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.514+0000 7f0b96e94700 10 client.42122.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:28.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.514+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_assign -1 9 2024-02-23T00:55:28.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.514+0000 7f0b96e94700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f0b9011cd20 0x7f0b901222d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:28.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.514+0000 7f0b96e94700 20 client.42122.objecter _get_session s=0x7f0b9011cbb0 osd=4 3 2024-02-23T00:55:28.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.514+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b9011cbb0 osd=4 3 2024-02-23T00:55:28.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.514+0000 7f0b96e94700 20 client.42122.objecter _get_session s=0x7f0b9011cbb0 osd=4 3 2024-02-23T00:55:28.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.514+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_remove -1 9 2024-02-23T00:55:28.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.514+0000 7f0b96e94700 20 client.42122.objecter get_session s=0x7f0b9011cbb0 osd=4 3 2024-02-23T00:55:28.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.514+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_assign 4 9 2024-02-23T00:55:28.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.514+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b9011cbb0 osd=4 4 2024-02-23T00:55:28.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.514+0000 7f0b96e94700 10 client.42122.objecter _send_command 9 2024-02-23T00:55:28.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.514+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0b9011bf60 con 0x7f0b9011cd20 2024-02-23T00:55:28.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.514+0000 7f0b94c30700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f0b9011cd20 0x7f0b901222d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:28.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.514+0000 7f0b8dffb700 10 client.42122.objecter ms_handle_connect 0x7f0b9011cd20 2024-02-23T00:55:28.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.514+0000 7f0b94c30700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f0b9011cd20 0x7f0b901222d0 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:28.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.515+0000 7f0b8dffb700 1 -- 172.21.15.47:0/2662679792 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0b9011bf60 con 0x7f0b9011cd20 2024-02-23T00:55:28.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.515+0000 7f0b8dffb700 10 client.42122.objecter ms_dispatch 0x7f0b90001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:55:28.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.515+0000 7f0b8dffb700 10 client.42122.objecter _finish_command 9 = system:0 2024-02-23T00:55:28.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.515+0000 7f0b8dffb700 20 client.42122.objecter put_session s=0x7f0b9011cbb0 osd=4 4 2024-02-23T00:55:28.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.515+0000 7f0b8dffb700 15 client.42122.objecter _session_command_op_remove 4 9 2024-02-23T00:55:28.531 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:55:28.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.529+0000 7f0b96e94700 10 client.42122.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:55:28.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.529+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_assign -1 10 2024-02-23T00:55:28.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.529+0000 7f0b96e94700 20 client.42122.objecter _get_session s=0x7f0b9011cbb0 osd=4 3 2024-02-23T00:55:28.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.529+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b9011cbb0 osd=4 3 2024-02-23T00:55:28.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.529+0000 7f0b96e94700 20 client.42122.objecter _get_session s=0x7f0b9011cbb0 osd=4 3 2024-02-23T00:55:28.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.529+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_remove -1 10 2024-02-23T00:55:28.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.529+0000 7f0b96e94700 20 client.42122.objecter get_session s=0x7f0b9011cbb0 osd=4 3 2024-02-23T00:55:28.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.529+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_assign 4 10 2024-02-23T00:55:28.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.529+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b9011cbb0 osd=4 4 2024-02-23T00:55:28.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.529+0000 7f0b96e94700 10 client.42122.objecter _send_command 10 2024-02-23T00:55:28.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.529+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0b9011bf60 con 0x7f0b9011cd20 2024-02-23T00:55:28.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.531+0000 7f0b8dffb700 1 -- 172.21.15.47:0/2662679792 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0b9011bf60 con 0x7f0b9011cd20 2024-02-23T00:55:28.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.531+0000 7f0b8dffb700 10 client.42122.objecter ms_dispatch 0x7f0b90001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:55:28.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.531+0000 7f0b8dffb700 10 client.42122.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:55:28.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.531+0000 7f0b8dffb700 20 client.42122.objecter put_session s=0x7f0b9011cbb0 osd=4 4 2024-02-23T00:55:28.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.531+0000 7f0b8dffb700 15 client.42122.objecter _session_command_op_remove 4 10 2024-02-23T00:55:28.535 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:55:28.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.532+0000 7f0b96e94700 10 client.42122.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:28.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.532+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_assign -1 11 2024-02-23T00:55:28.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.532+0000 7f0b96e94700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f0b90122cd0 0x7f0b90123130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:28.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.532+0000 7f0b96e94700 20 client.42122.objecter _get_session s=0x7f0b9011bf60 osd=5 3 2024-02-23T00:55:28.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.532+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b9011bf60 osd=5 3 2024-02-23T00:55:28.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.532+0000 7f0b96e94700 20 client.42122.objecter _get_session s=0x7f0b9011bf60 osd=5 3 2024-02-23T00:55:28.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.532+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_remove -1 11 2024-02-23T00:55:28.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.532+0000 7f0b96e94700 20 client.42122.objecter get_session s=0x7f0b9011bf60 osd=5 3 2024-02-23T00:55:28.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.532+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_assign 5 11 2024-02-23T00:55:28.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.532+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b9011bf60 osd=5 4 2024-02-23T00:55:28.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.532+0000 7f0b96e94700 10 client.42122.objecter _send_command 11 2024-02-23T00:55:28.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.532+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0b9012b430 con 0x7f0b90122cd0 2024-02-23T00:55:28.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.532+0000 7f0b8ffff700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f0b90122cd0 0x7f0b90123130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:28.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.532+0000 7f0b8dffb700 10 client.42122.objecter ms_handle_connect 0x7f0b90122cd0 2024-02-23T00:55:28.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.532+0000 7f0b8ffff700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f0b90122cd0 0x7f0b90123130 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:28.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.533+0000 7f0b8dffb700 1 -- 172.21.15.47:0/2662679792 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0b9012b430 con 0x7f0b90122cd0 2024-02-23T00:55:28.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.533+0000 7f0b8dffb700 10 client.42122.objecter ms_dispatch 0x7f0b90001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:55:28.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.533+0000 7f0b8dffb700 10 client.42122.objecter _finish_command 11 = system:0 2024-02-23T00:55:28.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.533+0000 7f0b8dffb700 20 client.42122.objecter put_session s=0x7f0b9011bf60 osd=5 4 2024-02-23T00:55:28.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.533+0000 7f0b8dffb700 15 client.42122.objecter _session_command_op_remove 5 11 2024-02-23T00:55:28.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.550+0000 7f0b96e94700 10 client.42122.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:55:28.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.550+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_assign -1 12 2024-02-23T00:55:28.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.550+0000 7f0b96e94700 20 client.42122.objecter _get_session s=0x7f0b9011bf60 osd=5 3 2024-02-23T00:55:28.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.550+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b9011bf60 osd=5 3 2024-02-23T00:55:28.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.550+0000 7f0b96e94700 20 client.42122.objecter _get_session s=0x7f0b9011bf60 osd=5 3 2024-02-23T00:55:28.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.550+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_remove -1 12 2024-02-23T00:55:28.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.550+0000 7f0b96e94700 20 client.42122.objecter get_session s=0x7f0b9011bf60 osd=5 3 2024-02-23T00:55:28.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.550+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_assign 5 12 2024-02-23T00:55:28.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.550+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b9011bf60 osd=5 4 2024-02-23T00:55:28.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.550+0000 7f0b96e94700 10 client.42122.objecter _send_command 12 2024-02-23T00:55:28.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.550+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0b90128cd0 con 0x7f0b90122cd0 2024-02-23T00:55:28.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.551+0000 7f0b8dffb700 1 -- 172.21.15.47:0/2662679792 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0b90128cd0 con 0x7f0b90122cd0 2024-02-23T00:55:28.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.551+0000 7f0b8dffb700 10 client.42122.objecter ms_dispatch 0x7f0b90001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:55:28.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.551+0000 7f0b8dffb700 10 client.42122.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:55:28.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.551+0000 7f0b8dffb700 20 client.42122.objecter put_session s=0x7f0b9011bf60 osd=5 4 2024-02-23T00:55:28.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.551+0000 7f0b8dffb700 15 client.42122.objecter _session_command_op_remove 5 12 2024-02-23T00:55:28.555 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:55:28.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.552+0000 7f0b96e94700 10 client.42122.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:28.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.552+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_assign -1 13 2024-02-23T00:55:28.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.552+0000 7f0b96e94700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f0b901293b0 0x7f0b9012b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:28.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.552+0000 7f0b96e94700 20 client.42122.objecter _get_session s=0x7f0b90128cd0 osd=6 3 2024-02-23T00:55:28.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.552+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b90128cd0 osd=6 3 2024-02-23T00:55:28.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.552+0000 7f0b96e94700 20 client.42122.objecter _get_session s=0x7f0b90128cd0 osd=6 3 2024-02-23T00:55:28.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.552+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_remove -1 13 2024-02-23T00:55:28.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.552+0000 7f0b96e94700 20 client.42122.objecter get_session s=0x7f0b90128cd0 osd=6 3 2024-02-23T00:55:28.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.552+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_assign 6 13 2024-02-23T00:55:28.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.552+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b90128cd0 osd=6 4 2024-02-23T00:55:28.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.552+0000 7f0b96e94700 10 client.42122.objecter _send_command 13 2024-02-23T00:55:28.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.552+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0b9012e560 con 0x7f0b901293b0 2024-02-23T00:55:28.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.552+0000 7f0b95431700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f0b901293b0 0x7f0b9012b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:28.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.552+0000 7f0b8dffb700 10 client.42122.objecter ms_handle_connect 0x7f0b901293b0 2024-02-23T00:55:28.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.552+0000 7f0b95431700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f0b901293b0 0x7f0b9012b7a0 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:28.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.553+0000 7f0b8dffb700 1 -- 172.21.15.47:0/2662679792 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0b9012e560 con 0x7f0b901293b0 2024-02-23T00:55:28.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.553+0000 7f0b8dffb700 10 client.42122.objecter ms_dispatch 0x7f0b90001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:55:28.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.553+0000 7f0b8dffb700 10 client.42122.objecter _finish_command 13 = system:0 2024-02-23T00:55:28.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.553+0000 7f0b8dffb700 20 client.42122.objecter put_session s=0x7f0b90128cd0 osd=6 4 2024-02-23T00:55:28.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.553+0000 7f0b8dffb700 15 client.42122.objecter _session_command_op_remove 6 13 2024-02-23T00:55:28.560 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.570+0000 7f0b96e94700 10 client.42122.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:55:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.570+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_assign -1 14 2024-02-23T00:55:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.570+0000 7f0b96e94700 20 client.42122.objecter _get_session s=0x7f0b90128cd0 osd=6 3 2024-02-23T00:55:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.570+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b90128cd0 osd=6 3 2024-02-23T00:55:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.570+0000 7f0b96e94700 20 client.42122.objecter _get_session s=0x7f0b90128cd0 osd=6 3 2024-02-23T00:55:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.570+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_remove -1 14 2024-02-23T00:55:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.570+0000 7f0b96e94700 20 client.42122.objecter get_session s=0x7f0b90128cd0 osd=6 3 2024-02-23T00:55:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.570+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_assign 6 14 2024-02-23T00:55:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.570+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b90128cd0 osd=6 4 2024-02-23T00:55:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.570+0000 7f0b96e94700 10 client.42122.objecter _send_command 14 2024-02-23T00:55:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.570+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0b9012e560 con 0x7f0b901293b0 2024-02-23T00:55:28.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.572+0000 7f0b8dffb700 1 -- 172.21.15.47:0/2662679792 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0b9012e560 con 0x7f0b901293b0 2024-02-23T00:55:28.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.572+0000 7f0b8dffb700 10 client.42122.objecter ms_dispatch 0x7f0b90001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:55:28.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.572+0000 7f0b8dffb700 10 client.42122.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:55:28.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.572+0000 7f0b8dffb700 20 client.42122.objecter put_session s=0x7f0b90128cd0 osd=6 4 2024-02-23T00:55:28.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.572+0000 7f0b8dffb700 15 client.42122.objecter _session_command_op_remove 6 14 2024-02-23T00:55:28.575 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:55:28.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.572+0000 7f0b96e94700 10 client.42122.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:28.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.572+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_assign -1 15 2024-02-23T00:55:28.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.572+0000 7f0b96e94700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f0b9012d8f0 0x7f0b9012dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:28.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.572+0000 7f0b96e94700 20 client.42122.objecter _get_session s=0x7f0b9012e560 osd=7 3 2024-02-23T00:55:28.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.572+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b9012e560 osd=7 3 2024-02-23T00:55:28.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.572+0000 7f0b96e94700 20 client.42122.objecter _get_session s=0x7f0b9012e560 osd=7 3 2024-02-23T00:55:28.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.572+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_remove -1 15 2024-02-23T00:55:28.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.572+0000 7f0b96e94700 20 client.42122.objecter get_session s=0x7f0b9012e560 osd=7 3 2024-02-23T00:55:28.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.572+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_assign 7 15 2024-02-23T00:55:28.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.572+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b9012e560 osd=7 4 2024-02-23T00:55:28.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.572+0000 7f0b96e94700 10 client.42122.objecter _send_command 15 2024-02-23T00:55:28.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.572+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0b90131890 con 0x7f0b9012d8f0 2024-02-23T00:55:28.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.573+0000 7f0b94c30700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f0b9012d8f0 0x7f0b9012dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:28.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.573+0000 7f0b8dffb700 10 client.42122.objecter ms_handle_connect 0x7f0b9012d8f0 2024-02-23T00:55:28.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.573+0000 7f0b94c30700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f0b9012d8f0 0x7f0b9012dd50 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:28.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.574+0000 7f0b8dffb700 1 -- 172.21.15.47:0/2662679792 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0b90131890 con 0x7f0b9012d8f0 2024-02-23T00:55:28.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.574+0000 7f0b8dffb700 10 client.42122.objecter ms_dispatch 0x7f0b90001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:55:28.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.574+0000 7f0b8dffb700 10 client.42122.objecter _finish_command 15 = system:0 2024-02-23T00:55:28.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.574+0000 7f0b8dffb700 20 client.42122.objecter put_session s=0x7f0b9012e560 osd=7 4 2024-02-23T00:55:28.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.574+0000 7f0b8dffb700 15 client.42122.objecter _session_command_op_remove 7 15 2024-02-23T00:55:28.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.591+0000 7f0b96e94700 10 client.42122.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:55:28.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.591+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_assign -1 16 2024-02-23T00:55:28.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.591+0000 7f0b96e94700 20 client.42122.objecter _get_session s=0x7f0b9012e560 osd=7 3 2024-02-23T00:55:28.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.591+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b9012e560 osd=7 3 2024-02-23T00:55:28.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.591+0000 7f0b96e94700 20 client.42122.objecter _get_session s=0x7f0b9012e560 osd=7 3 2024-02-23T00:55:28.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.591+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_remove -1 16 2024-02-23T00:55:28.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.591+0000 7f0b96e94700 20 client.42122.objecter get_session s=0x7f0b9012e560 osd=7 3 2024-02-23T00:55:28.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.591+0000 7f0b96e94700 15 client.42122.objecter _session_command_op_assign 7 16 2024-02-23T00:55:28.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.591+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b9012e560 osd=7 4 2024-02-23T00:55:28.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.591+0000 7f0b96e94700 10 client.42122.objecter _send_command 16 2024-02-23T00:55:28.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.591+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0b9012bd80 con 0x7f0b9012d8f0 2024-02-23T00:55:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b8dffb700 1 -- 172.21.15.47:0/2662679792 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0b9012bd80 con 0x7f0b9012d8f0 2024-02-23T00:55:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b8dffb700 10 client.42122.objecter ms_dispatch 0x7f0b90001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:55:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b8dffb700 10 client.42122.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:55:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b8dffb700 20 client.42122.objecter put_session s=0x7f0b9012e560 osd=7 4 2024-02-23T00:55:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b8dffb700 15 client.42122.objecter _session_command_op_remove 7 16 2024-02-23T00:55:28.596 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:55:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 10 librados: watch_flush enter 2024-02-23T00:55:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 10 librados: watch_flush exit 2024-02-23T00:55:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 10 client.42122.objecter close_session for osd.0 2024-02-23T00:55:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f0b900ffd80 msgr2=0x7f0b901001c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f0b900ffd80 0x7f0b901001c0 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b9005ee80 osd=0 1 2024-02-23T00:55:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 10 client.42122.objecter close_session for osd.1 2024-02-23T00:55:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f0b900598f0 msgr2=0x7f0b90059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f0b900598f0 0x7f0b90059cd0 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b90028710 osd=1 1 2024-02-23T00:55:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 10 client.42122.objecter close_session for osd.2 2024-02-23T00:55:28.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f0b90028270 msgr2=0x7f0b9011c1f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:28.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f0b90028270 0x7f0b9011c1f0 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:28.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b90059770 osd=2 1 2024-02-23T00:55:28.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 10 client.42122.objecter close_session for osd.3 2024-02-23T00:55:28.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f0b9011f950 msgr2=0x7f0b90121d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:28.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b95431700 1 -- 172.21.15.47:0/2662679792 reap_dead start 2024-02-23T00:55:28.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f0b9011f950 0x7f0b90121d40 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:28.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b9011d140 osd=3 1 2024-02-23T00:55:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 10 client.42122.objecter close_session for osd.4 2024-02-23T00:55:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f0b9011cd20 msgr2=0x7f0b901222d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f0b9011cd20 0x7f0b901222d0 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b9011cbb0 osd=4 1 2024-02-23T00:55:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 10 client.42122.objecter close_session for osd.5 2024-02-23T00:55:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f0b90122cd0 msgr2=0x7f0b90123130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f0b90122cd0 0x7f0b90123130 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b9011bf60 osd=5 1 2024-02-23T00:55:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 10 client.42122.objecter close_session for osd.6 2024-02-23T00:55:28.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f0b901293b0 msgr2=0x7f0b9012b7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:28.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f0b901293b0 0x7f0b9012b7a0 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:28.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b90128cd0 osd=6 1 2024-02-23T00:55:28.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 10 client.42122.objecter close_session for osd.7 2024-02-23T00:55:28.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f0b9012d8f0 msgr2=0x7f0b9012dd50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:28.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f0b9012d8f0 0x7f0b9012dd50 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:28.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 20 client.42122.objecter put_session s=0x7f0b9012e560 osd=7 1 2024-02-23T00:55:28.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 20 client.42122.objecter shutdown clearing up homeless session... 2024-02-23T00:55:28.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 10 client.42122.objecter successfully canceled tick 2024-02-23T00:55:28.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0b80062cc0 msgr2=0x7f0b80065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:28.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0b80062cc0 0x7f0b80065180 secure :-1 s=READY pgs=3204 cs=0 l=1 rev1=1 rx=0x7f0b84007bf0 tx=0x7f0b8400b040).stop 2024-02-23T00:55:28.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0b900fcac0 msgr2=0x7f0b900f9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:28.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.593+0000 7f0b96e94700 1 --2- 172.21.15.47:0/2662679792 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0b900fcac0 0x7f0b900f9160 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 rx=0x7f0b7c00e970 tx=0x7f0b7c00b040).stop 2024-02-23T00:55:28.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.594+0000 7f0b95431700 1 -- 172.21.15.47:0/2662679792 reap_dead start 2024-02-23T00:55:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.594+0000 7f0b95431700 1 -- 172.21.15.47:0/2662679792 reap_dead start 2024-02-23T00:55:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.594+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 shutdown_connections 2024-02-23T00:55:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.594+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 >> 172.21.15.47:0/2662679792 conn(0x7f0b900f2710 msgr2=0x7f0b900facc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.594+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 shutdown_connections 2024-02-23T00:55:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.594+0000 7f0b96e94700 1 -- 172.21.15.47:0/2662679792 wait complete. 2024-02-23T00:55:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:28.594+0000 7f0b96e94700 1 librados: shutdown 2024-02-23T00:55:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:55:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:55:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:55:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:55:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:55:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:55:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:55:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:55:28.994 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:27.551423+0000 osd.1 (osd.1) 308 : cluster [DBG] 6.4 scrub starts 2024-02-23T00:55:28.994 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:27.554979+0000 osd.1 (osd.1) 309 : cluster [DBG] 6.4 scrub ok 2024-02-23T00:55:28.994 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:27.855673+0000 osd.0 (osd.0) 289 : cluster [DBG] 2.1e deep-scrub starts 2024-02-23T00:55:28.994 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:27.856490+0000 osd.0 (osd.0) 290 : cluster [DBG] 2.1e deep-scrub ok 2024-02-23T00:55:28.994 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:28.321749+0000 mon.a (mon.0) 3701 : cluster [INF] Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-02-23T00:55:28.995 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:28.329522+0000 mon.a (mon.0) 3702 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:55:28.995 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:28.329634+0000 mon.a (mon.0) 3703 : cluster [DBG] osdmap e777: 8 total, 8 up, 7 in 2024-02-23T00:55:28.995 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:28.352802+0000 mon.c (mon.2) 483 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:55:28.995 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:28.353167+0000 mon.a (mon.0) 3704 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:55:28.995 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:28.437532+0000 mon.c (mon.2) 484 : audit [DBG] from='client.? 172.21.15.47:0/2662679792' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:55:28.995 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:27.551423+0000 osd.1 (osd.1) 308 : cluster [DBG] 6.4 scrub starts 2024-02-23T00:55:28.995 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:27.554979+0000 osd.1 (osd.1) 309 : cluster [DBG] 6.4 scrub ok 2024-02-23T00:55:28.995 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:27.855673+0000 osd.0 (osd.0) 289 : cluster [DBG] 2.1e deep-scrub starts 2024-02-23T00:55:28.995 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:27.856490+0000 osd.0 (osd.0) 290 : cluster [DBG] 2.1e deep-scrub ok 2024-02-23T00:55:28.995 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:28.321749+0000 mon.a (mon.0) 3701 : cluster [INF] Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-02-23T00:55:28.995 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:28.329522+0000 mon.a (mon.0) 3702 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:55:28.996 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:28.329634+0000 mon.a (mon.0) 3703 : cluster [DBG] osdmap e777: 8 total, 8 up, 7 in 2024-02-23T00:55:28.996 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:28.352802+0000 mon.c (mon.2) 483 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:55:28.996 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:28.353167+0000 mon.a (mon.0) 3704 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:55:28.996 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:28.437532+0000 mon.c (mon.2) 484 : audit [DBG] from='client.? 172.21.15.47:0/2662679792' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:55:29.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:27.551423+0000 osd.1 (osd.1) 308 : cluster [DBG] 6.4 scrub starts 2024-02-23T00:55:29.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:27.554979+0000 osd.1 (osd.1) 309 : cluster [DBG] 6.4 scrub ok 2024-02-23T00:55:29.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:27.855673+0000 osd.0 (osd.0) 289 : cluster [DBG] 2.1e deep-scrub starts 2024-02-23T00:55:29.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:27.856490+0000 osd.0 (osd.0) 290 : cluster [DBG] 2.1e deep-scrub ok 2024-02-23T00:55:29.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:28.321749+0000 mon.a (mon.0) 3701 : cluster [INF] Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-02-23T00:55:29.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:28.329522+0000 mon.a (mon.0) 3702 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:55:29.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:28.329634+0000 mon.a (mon.0) 3703 : cluster [DBG] osdmap e777: 8 total, 8 up, 7 in 2024-02-23T00:55:29.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:28.352802+0000 mon.c (mon.2) 483 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:55:29.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:28.353167+0000 mon.a (mon.0) 3704 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:55:29.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:28.437532+0000 mon.c (mon.2) 484 : audit [DBG] from='client.? 172.21.15.47:0/2662679792' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:55:29.843 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:55:29.847 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:29.848 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:55:29.882 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:55:29.883 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:55:29.903 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:27.977773+0000 osd.7 (osd.7) 353 : cluster [DBG] 6.b scrub starts 2024-02-23T00:55:29.904 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:27.980878+0000 osd.7 (osd.7) 354 : cluster [DBG] 6.b scrub ok 2024-02-23T00:55:29.904 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:29.181725+0000 osd.5 (osd.5) 328 : cluster [DBG] 4.15 scrub starts 2024-02-23T00:55:29.904 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:29.185189+0000 osd.5 (osd.5) 329 : cluster [DBG] 4.15 scrub ok 2024-02-23T00:55:29.904 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:29.331265+0000 mon.a (mon.0) 3705 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:55:29.904 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:29.331353+0000 mon.a (mon.0) 3706 : cluster [DBG] osdmap e778: 8 total, 8 up, 7 in 2024-02-23T00:55:29.904 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:29.333930+0000 mon.c (mon.2) 485 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-136"}]: dispatch 2024-02-23T00:55:29.904 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:29.334279+0000 mon.a (mon.0) 3707 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-136"}]: dispatch 2024-02-23T00:55:29.904 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:29.401220+0000 mgr.y (mgr.14517) 661 : cluster [DBG] pgmap v1182: 194 pgs: 32 unknown, 162 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 92 B/s, 0 objects/s recovering 2024-02-23T00:55:29.949 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:55:29.978 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:30.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:27.977773+0000 osd.7 (osd.7) 353 : cluster [DBG] 6.b scrub starts 2024-02-23T00:55:30.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:27.980878+0000 osd.7 (osd.7) 354 : cluster [DBG] 6.b scrub ok 2024-02-23T00:55:30.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:29.181725+0000 osd.5 (osd.5) 328 : cluster [DBG] 4.15 scrub starts 2024-02-23T00:55:30.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:29.185189+0000 osd.5 (osd.5) 329 : cluster [DBG] 4.15 scrub ok 2024-02-23T00:55:30.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:29.331265+0000 mon.a (mon.0) 3705 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:55:30.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:29.331353+0000 mon.a (mon.0) 3706 : cluster [DBG] osdmap e778: 8 total, 8 up, 7 in 2024-02-23T00:55:30.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:29.333930+0000 mon.c (mon.2) 485 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-136"}]: dispatch 2024-02-23T00:55:30.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:29.334279+0000 mon.a (mon.0) 3707 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-136"}]: dispatch 2024-02-23T00:55:30.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:29.401220+0000 mgr.y (mgr.14517) 661 : cluster [DBG] pgmap v1182: 194 pgs: 32 unknown, 162 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 92 B/s, 0 objects/s recovering 2024-02-23T00:55:30.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:27.977773+0000 osd.7 (osd.7) 353 : cluster [DBG] 6.b scrub starts 2024-02-23T00:55:30.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:27.980878+0000 osd.7 (osd.7) 354 : cluster [DBG] 6.b scrub ok 2024-02-23T00:55:30.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:29.181725+0000 osd.5 (osd.5) 328 : cluster [DBG] 4.15 scrub starts 2024-02-23T00:55:30.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:29.185189+0000 osd.5 (osd.5) 329 : cluster [DBG] 4.15 scrub ok 2024-02-23T00:55:30.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:29.331265+0000 mon.a (mon.0) 3705 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:55:30.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:29.331353+0000 mon.a (mon.0) 3706 : cluster [DBG] osdmap e778: 8 total, 8 up, 7 in 2024-02-23T00:55:30.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:29.333930+0000 mon.c (mon.2) 485 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-136"}]: dispatch 2024-02-23T00:55:30.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:29.334279+0000 mon.a (mon.0) 3707 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-136"}]: dispatch 2024-02-23T00:55:30.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:29.401220+0000 mgr.y (mgr.14517) 661 : cluster [DBG] pgmap v1182: 194 pgs: 32 unknown, 162 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 92 B/s, 0 objects/s recovering 2024-02-23T00:55:30.100 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:55:30.104 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:30.104 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:55:30.171 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set noscrub 2024-02-23T00:55:30.205 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:55:30.249 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:30.376 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:55:30.380 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:30.381 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:55:30.482 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:55:30.511 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:30.609 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.5 dump_blocked_ops 2024-02-23T00:55:30.614 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:55:30.617 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:30.617 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:55:30.718 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:55:30.744 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:55:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:55:30] "GET /metrics HTTP/1.1" 200 191882 "" "Prometheus/2.33.4" 2024-02-23T00:55:30.746 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:30.767 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:55:30.770 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:30.770 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:55:30.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.863+0000 7f2a710a3700 1 -- 172.21.15.47:0/2936327296 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2a6c0f6b20 msgr2=0x7f2a6c0f6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:30.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.863+0000 7f2a710a3700 1 --2- 172.21.15.47:0/2936327296 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2a6c0f6b20 0x7f2a6c0f6f00 secure :-1 s=READY pgs=2817 cs=0 l=1 rev1=1 rx=0x7f2a5c004870 tx=0x7f2a5c009cb0).stop 2024-02-23T00:55:30.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.863+0000 7f2a710a3700 1 -- 172.21.15.47:0/2936327296 shutdown_connections 2024-02-23T00:55:30.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.863+0000 7f2a710a3700 1 --2- 172.21.15.47:0/2936327296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a6c104680 0x7f2a6c106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:55:30.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.863+0000 7f2a710a3700 1 --2- 172.21.15.47:0/2936327296 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2a6c0f6b20 0x7f2a6c0f6f00 unknown :-1 s=CLOSED pgs=2817 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:30.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.863+0000 7f2a710a3700 1 --2- 172.21.15.47:0/2936327296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a6c0f73f0 0x7f2a6c104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:30.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.863+0000 7f2a710a3700 1 -- 172.21.15.47:0/2936327296 >> 172.21.15.47:0/2936327296 conn(0x7f2a6c0f2820 msgr2=0x7f2a6c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:30.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.863+0000 7f2a710a3700 1 -- 172.21.15.47:0/2936327296 shutdown_connections 2024-02-23T00:55:30.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.863+0000 7f2a710a3700 1 -- 172.21.15.47:0/2936327296 wait complete. 2024-02-23T00:55:30.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.864+0000 7f2a710a3700 1 librados: starting msgr at 2024-02-23T00:55:30.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.864+0000 7f2a710a3700 1 librados: starting objecter 2024-02-23T00:55:30.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.864+0000 7f2a710a3700 1 Processor -- start 2024-02-23T00:55:30.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.865+0000 7f2a710a3700 1 -- start start 2024-02-23T00:55:30.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.865+0000 7f2a710a3700 1 librados: setting wanted keys 2024-02-23T00:55:30.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.865+0000 7f2a710a3700 1 librados: calling monclient init 2024-02-23T00:55:30.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.865+0000 7f2a710a3700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2a6c0f6b20 0x7f2a6c1102c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:30.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.865+0000 7f2a710a3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a6c0f73f0 0x7f2a6c1107b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:30.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.865+0000 7f2a710a3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a6c104680 0x7f2a6c10a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:30.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.865+0000 7f2a710a3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6c10a950 con 0x7f2a6c0f73f0 2024-02-23T00:55:30.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.865+0000 7f2a710a3700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6c10aac0 con 0x7f2a6c104680 2024-02-23T00:55:30.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.865+0000 7f2a710a3700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6c10ac80 con 0x7f2a6c0f6b20 2024-02-23T00:55:30.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.865+0000 7f2a6a59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a6c0f73f0 0x7f2a6c1107b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:30.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.865+0000 7f2a6ad9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2a6c0f6b20 0x7f2a6c1102c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:30.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.865+0000 7f2a6a59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a6c0f73f0 0x7f2a6c1107b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35710/0 (socket says 172.21.15.47:35710) 2024-02-23T00:55:30.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.865+0000 7f2a6a59c700 1 -- 172.21.15.47:0/3795516886 learned_addr learned my addr 172.21.15.47:0/3795516886 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:55:30.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.865+0000 7f2a6b59e700 1 --2- 172.21.15.47:0/3795516886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a6c104680 0x7f2a6c10a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:30.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.865+0000 7f2a6ad9d700 1 -- 172.21.15.47:0/3795516886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a6c104680 msgr2=0x7f2a6c10a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:30.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.865+0000 7f2a6ad9d700 1 --2- 172.21.15.47:0/3795516886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a6c104680 0x7f2a6c10a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:30.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.865+0000 7f2a6ad9d700 1 -- 172.21.15.47:0/3795516886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a6c0f73f0 msgr2=0x7f2a6c1107b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:30.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.865+0000 7f2a6ad9d700 1 --2- 172.21.15.47:0/3795516886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a6c0f73f0 0x7f2a6c1107b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:30.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.865+0000 7f2a6ad9d700 1 -- 172.21.15.47:0/3795516886 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5c005040 con 0x7f2a6c0f6b20 2024-02-23T00:55:30.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.865+0000 7f2a6b59e700 1 --2- 172.21.15.47:0/3795516886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a6c104680 0x7f2a6c10a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:55:30.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.865+0000 7f2a6a59c700 1 --2- 172.21.15.47:0/3795516886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a6c0f73f0 0x7f2a6c1107b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:55:30.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.866+0000 7f2a6ad9d700 1 --2- 172.21.15.47:0/3795516886 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2a6c0f6b20 0x7f2a6c1102c0 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 rx=0x7f2a5c004840 tx=0x7f2a5c009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:30.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.866+0000 7f2a53fff700 10 client.?.objecter ms_handle_connect 0x7f2a6c0f6b20 2024-02-23T00:55:30.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.866+0000 7f2a53fff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:55:30.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.866+0000 7f2a53fff700 1 -- 172.21.15.47:0/3795516886 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c00e030 con 0x7f2a6c0f6b20 2024-02-23T00:55:30.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.866+0000 7f2a710a3700 1 -- 172.21.15.47:0/3795516886 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a6c10af00 con 0x7f2a6c0f6b20 2024-02-23T00:55:30.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.866+0000 7f2a53fff700 1 -- 172.21.15.47:0/3795516886 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f2a5c013e70 con 0x7f2a6c0f6b20 2024-02-23T00:55:30.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.866+0000 7f2a53fff700 1 -- 172.21.15.47:0/3795516886 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c018070 con 0x7f2a6c0f6b20 2024-02-23T00:55:30.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.866+0000 7f2a710a3700 10 client.42128.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:55:30.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.866+0000 7f2a710a3700 1 -- 172.21.15.47:0/3795516886 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a6c1170f0 con 0x7f2a6c0f6b20 2024-02-23T00:55:30.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.866+0000 7f2a710a3700 1 librados: init done 2024-02-23T00:55:30.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.867+0000 7f2a710a3700 1 -- 172.21.15.47:0/3795516886 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a6c05fa60 con 0x7f2a6c0f6b20 2024-02-23T00:55:30.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.868+0000 7f2a53fff700 1 -- 172.21.15.47:0/3795516886 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f2a5c009240 con 0x7f2a6c0f6b20 2024-02-23T00:55:30.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.868+0000 7f2a53fff700 1 --2- 172.21.15.47:0/3795516886 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2a54062cc0 0x7f2a54065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:30.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.868+0000 7f2a53fff700 1 -- 172.21.15.47:0/3795516886 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(779..779 src has 621..779) v4 ==== 9748+0+0 (secure 0 0 0) 0x7f2a5c07ea90 con 0x7f2a6c0f6b20 2024-02-23T00:55:30.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.868+0000 7f2a53fff700 10 client.42128.objecter ms_dispatch 0x7f2a6c001550 osd_map(779..779 src has 621..779) v4 2024-02-23T00:55:30.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.868+0000 7f2a53fff700 3 client.42128.objecter handle_osd_map got epochs [779,779] > 0 2024-02-23T00:55:30.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.868+0000 7f2a53fff700 3 client.42128.objecter handle_osd_map decoding full epoch 779 2024-02-23T00:55:30.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.868+0000 7f2a6a59c700 1 --2- 172.21.15.47:0/3795516886 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2a54062cc0 0x7f2a54065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:30.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.868+0000 7f2a53fff700 20 client.42128.objecter dump_active .. 0 homeless 2024-02-23T00:55:30.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.869+0000 7f2a53fff700 10 client.42128.objecter ms_handle_connect 0x7f2a54062cc0 2024-02-23T00:55:30.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.869+0000 7f2a6a59c700 1 --2- 172.21.15.47:0/3795516886 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2a54062cc0 0x7f2a54065180 secure :-1 s=READY pgs=3205 cs=0 l=1 rev1=1 rx=0x7f2a60004fd0 tx=0x7f2a6000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:30.878 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:55:30.878 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:55:30.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:30.875+0000 7f2a53fff700 1 -- 172.21.15.47:0/3795516886 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f2a5c04de10 con 0x7f2a6c0f6b20 2024-02-23T00:55:30.904 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:55:30.909 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:30.931 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:55:30.934 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:30.934 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:55:30.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:29.827677+0000 osd.0 (osd.0) 291 : cluster [DBG] 4.1e scrub starts 2024-02-23T00:55:30.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:29.829100+0000 osd.0 (osd.0) 292 : cluster [DBG] 4.1e scrub ok 2024-02-23T00:55:30.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:29.993462+0000 osd.4 (osd.4) 325 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:55:30.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:29.993651+0000 osd.4 (osd.4) 326 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:55:30.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:30.227937+0000 osd.5 (osd.5) 330 : cluster [DBG] 6.a scrub starts 2024-02-23T00:55:30.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:30.229169+0000 osd.5 (osd.5) 331 : cluster [DBG] 6.a scrub ok 2024-02-23T00:55:30.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:30.334856+0000 mon.a (mon.0) 3708 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-136"}]': finished 2024-02-23T00:55:30.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:30.334946+0000 mon.a (mon.0) 3709 : cluster [DBG] osdmap e779: 8 total, 8 up, 7 in 2024-02-23T00:55:30.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:30.335658+0000 mon.c (mon.2) 486 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-136", "mode": "writeback"}]: dispatch 2024-02-23T00:55:30.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:30.336053+0000 mon.a (mon.0) 3710 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-136", "mode": "writeback"}]: dispatch 2024-02-23T00:55:31.036 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:55:31.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:29.827677+0000 osd.0 (osd.0) 291 : cluster [DBG] 4.1e scrub starts 2024-02-23T00:55:31.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:29.829100+0000 osd.0 (osd.0) 292 : cluster [DBG] 4.1e scrub ok 2024-02-23T00:55:31.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:29.993462+0000 osd.4 (osd.4) 325 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:55:31.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:29.993651+0000 osd.4 (osd.4) 326 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:55:31.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:30.227937+0000 osd.5 (osd.5) 330 : cluster [DBG] 6.a scrub starts 2024-02-23T00:55:31.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:30.229169+0000 osd.5 (osd.5) 331 : cluster [DBG] 6.a scrub ok 2024-02-23T00:55:31.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:30.334856+0000 mon.a (mon.0) 3708 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-136"}]': finished 2024-02-23T00:55:31.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:30.334946+0000 mon.a (mon.0) 3709 : cluster [DBG] osdmap e779: 8 total, 8 up, 7 in 2024-02-23T00:55:31.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:30.335658+0000 mon.c (mon.2) 486 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-136", "mode": "writeback"}]: dispatch 2024-02-23T00:55:31.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:30.336053+0000 mon.a (mon.0) 3710 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-136", "mode": "writeback"}]: dispatch 2024-02-23T00:55:31.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:29.827677+0000 osd.0 (osd.0) 291 : cluster [DBG] 4.1e scrub starts 2024-02-23T00:55:31.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:29.829100+0000 osd.0 (osd.0) 292 : cluster [DBG] 4.1e scrub ok 2024-02-23T00:55:31.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:29.993462+0000 osd.4 (osd.4) 325 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:55:31.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:29.993651+0000 osd.4 (osd.4) 326 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:55:31.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:30.227937+0000 osd.5 (osd.5) 330 : cluster [DBG] 6.a scrub starts 2024-02-23T00:55:31.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:30.229169+0000 osd.5 (osd.5) 331 : cluster [DBG] 6.a scrub ok 2024-02-23T00:55:31.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:30.334856+0000 mon.a (mon.0) 3708 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi047-82465-101", "overlaypool": "test-rados-api-smithi047-82465-136"}]': finished 2024-02-23T00:55:31.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:30.334946+0000 mon.a (mon.0) 3709 : cluster [DBG] osdmap e779: 8 total, 8 up, 7 in 2024-02-23T00:55:31.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:30.335658+0000 mon.c (mon.2) 486 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-136", "mode": "writeback"}]: dispatch 2024-02-23T00:55:31.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:30.336053+0000 mon.a (mon.0) 3710 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-136", "mode": "writeback"}]: dispatch 2024-02-23T00:55:31.063 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:31.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:31.064+0000 7f2a710a3700 1 -- 172.21.15.47:0/3795516886 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd out", "ids": ["7"]} v 0) v1 -- 0x7f2a6c1091b0 con 0x7f2a6c0f6b20 2024-02-23T00:55:31.135 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:55:31.137 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:31.137 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:55:31.238 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:55:31.267 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:31.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:31.341+0000 7f2a53fff700 1 -- 172.21.15.47:0/3795516886 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd out", "ids": ["7"]}]=0 marked out osd.7. v780) v1 ==== 87+0+0 (secure 0 0 0) 0x7f2a5c051430 con 0x7f2a6c0f6b20 2024-02-23T00:55:31.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:31.347+0000 7f2a710a3700 10 librados: watch_flush enter 2024-02-23T00:55:31.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:31.347+0000 7f2a710a3700 10 librados: watch_flush exit 2024-02-23T00:55:31.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:31.347+0000 7f2a710a3700 20 client.42128.objecter shutdown clearing up homeless session... 2024-02-23T00:55:31.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:31.348+0000 7f2a710a3700 10 client.42128.objecter successfully canceled tick 2024-02-23T00:55:31.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:31.348+0000 7f2a710a3700 1 -- 172.21.15.47:0/3795516886 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2a54062cc0 msgr2=0x7f2a54065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:31.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:31.348+0000 7f2a710a3700 1 --2- 172.21.15.47:0/3795516886 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2a54062cc0 0x7f2a54065180 secure :-1 s=READY pgs=3205 cs=0 l=1 rev1=1 rx=0x7f2a60004fd0 tx=0x7f2a6000b040).stop 2024-02-23T00:55:31.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:31.348+0000 7f2a710a3700 1 -- 172.21.15.47:0/3795516886 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2a6c0f6b20 msgr2=0x7f2a6c1102c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:31.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:31.348+0000 7f2a710a3700 1 --2- 172.21.15.47:0/3795516886 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2a6c0f6b20 0x7f2a6c1102c0 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 rx=0x7f2a5c004840 tx=0x7f2a5c009cb0).stop 2024-02-23T00:55:31.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:31.348+0000 7f2a710a3700 1 -- 172.21.15.47:0/3795516886 shutdown_connections 2024-02-23T00:55:31.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:31.348+0000 7f2a710a3700 1 --2- 172.21.15.47:0/3795516886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a6c104680 0x7f2a6c10a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:31.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:31.348+0000 7f2a710a3700 1 --2- 172.21.15.47:0/3795516886 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2a54062cc0 0x7f2a54065180 unknown :-1 s=CLOSED pgs=3205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:31.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:31.348+0000 7f2a710a3700 1 --2- 172.21.15.47:0/3795516886 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2a6c0f6b20 0x7f2a6c1102c0 secure :-1 s=CLOSED pgs=2818 cs=0 l=1 rev1=1 rx=0x7f2a5c004840 tx=0x7f2a5c009cb0).stop 2024-02-23T00:55:31.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:31.348+0000 7f2a710a3700 1 --2- 172.21.15.47:0/3795516886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a6c0f73f0 0x7f2a6c1107b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:31.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:31.348+0000 7f2a710a3700 1 -- 172.21.15.47:0/3795516886 >> 172.21.15.47:0/3795516886 conn(0x7f2a6c0f2820 msgr2=0x7f2a6c0f4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:31.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:31.349+0000 7f2a710a3700 1 -- 172.21.15.47:0/3795516886 shutdown_connections 2024-02-23T00:55:31.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:31.351+0000 7f2a710a3700 1 -- 172.21.15.47:0/3795516886 wait complete. 2024-02-23T00:55:31.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:31.352+0000 7f2a710a3700 1 librados: shutdown 2024-02-23T00:55:31.356 INFO:teuthology.orchestra.run.smithi047.stderr:marked out osd.7. 2024-02-23T00:55:31.366 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:55:31.368 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:31.369 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:55:31.380 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:55:31.065+0000 7f3c89cf1700 -1 mon.a@0(leader).osd e779 definitely_dead 0 2024-02-23T00:55:31.470 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:55:31.496 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:31.694 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:55:31.696 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:31.697 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:55:31.798 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:55:31.828 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:31.867 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:55:31.869 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:31.870 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:55:31.970 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:55:32.003 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:32.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:29.930692+0000 osd.7 (osd.7) 355 : cluster [DBG] 5.d scrub starts 2024-02-23T00:55:32.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:29.934179+0000 osd.7 (osd.7) 356 : cluster [DBG] 5.d scrub ok 2024-02-23T00:55:32.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:31.066081+0000 mon.c (mon.2) 487 : audit [INF] from='client.? 172.21.15.47:0/3795516886' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["7"]}]: dispatch 2024-02-23T00:55:32.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:31.066477+0000 mon.a (mon.0) 3711 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["7"]}]: dispatch 2024-02-23T00:55:32.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:31.066567+0000 mon.a (mon.0) 3712 : cluster [INF] Client client.admin marked osd.7 out, while it was still marked up 2024-02-23T00:55:32.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:31.236710+0000 osd.5 (osd.5) 332 : cluster [DBG] 4.8 deep-scrub starts 2024-02-23T00:55:32.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:31.242577+0000 osd.5 (osd.5) 333 : cluster [DBG] 4.8 deep-scrub ok 2024-02-23T00:55:32.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:31.332642+0000 mon.a (mon.0) 3713 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:55:32.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:31.336945+0000 mon.a (mon.0) 3714 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-136", "mode": "writeback"}]': finished 2024-02-23T00:55:32.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:31.337025+0000 mon.a (mon.0) 3715 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["7"]}]': finished 2024-02-23T00:55:32.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:31.337080+0000 mon.a (mon.0) 3716 : cluster [DBG] osdmap e780: 8 total, 8 up, 6 in 2024-02-23T00:55:32.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:31.401568+0000 mgr.y (mgr.14517) 662 : cluster [DBG] pgmap v1185: 194 pgs: 32 unknown, 162 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:55:32.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:29.930692+0000 osd.7 (osd.7) 355 : cluster [DBG] 5.d scrub starts 2024-02-23T00:55:32.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:29.934179+0000 osd.7 (osd.7) 356 : cluster [DBG] 5.d scrub ok 2024-02-23T00:55:32.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:31.066081+0000 mon.c (mon.2) 487 : audit [INF] from='client.? 172.21.15.47:0/3795516886' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["7"]}]: dispatch 2024-02-23T00:55:32.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:31.066477+0000 mon.a (mon.0) 3711 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["7"]}]: dispatch 2024-02-23T00:55:32.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:31.066567+0000 mon.a (mon.0) 3712 : cluster [INF] Client client.admin marked osd.7 out, while it was still marked up 2024-02-23T00:55:32.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:31.236710+0000 osd.5 (osd.5) 332 : cluster [DBG] 4.8 deep-scrub starts 2024-02-23T00:55:32.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:31.242577+0000 osd.5 (osd.5) 333 : cluster [DBG] 4.8 deep-scrub ok 2024-02-23T00:55:32.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:31.332642+0000 mon.a (mon.0) 3713 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:55:32.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:31.336945+0000 mon.a (mon.0) 3714 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-136", "mode": "writeback"}]': finished 2024-02-23T00:55:32.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:31.337025+0000 mon.a (mon.0) 3715 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["7"]}]': finished 2024-02-23T00:55:32.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:31.337080+0000 mon.a (mon.0) 3716 : cluster [DBG] osdmap e780: 8 total, 8 up, 6 in 2024-02-23T00:55:32.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:31.401568+0000 mgr.y (mgr.14517) 662 : cluster [DBG] pgmap v1185: 194 pgs: 32 unknown, 162 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:55:32.105 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:29.930692+0000 osd.7 (osd.7) 355 : cluster [DBG] 5.d scrub starts 2024-02-23T00:55:32.105 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:29.934179+0000 osd.7 (osd.7) 356 : cluster [DBG] 5.d scrub ok 2024-02-23T00:55:32.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:31.066081+0000 mon.c (mon.2) 487 : audit [INF] from='client.? 172.21.15.47:0/3795516886' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["7"]}]: dispatch 2024-02-23T00:55:32.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:31.066477+0000 mon.a (mon.0) 3711 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["7"]}]: dispatch 2024-02-23T00:55:32.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:31.066567+0000 mon.a (mon.0) 3712 : cluster [INF] Client client.admin marked osd.7 out, while it was still marked up 2024-02-23T00:55:32.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:31.236710+0000 osd.5 (osd.5) 332 : cluster [DBG] 4.8 deep-scrub starts 2024-02-23T00:55:32.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:31.242577+0000 osd.5 (osd.5) 333 : cluster [DBG] 4.8 deep-scrub ok 2024-02-23T00:55:32.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:31.332642+0000 mon.a (mon.0) 3713 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-23T00:55:32.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:31.336945+0000 mon.a (mon.0) 3714 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi047-82465-136", "mode": "writeback"}]': finished 2024-02-23T00:55:32.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:31.337025+0000 mon.a (mon.0) 3715 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["7"]}]': finished 2024-02-23T00:55:32.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:31.337080+0000 mon.a (mon.0) 3716 : cluster [DBG] osdmap e780: 8 total, 8 up, 6 in 2024-02-23T00:55:32.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:31.401568+0000 mgr.y (mgr.14517) 662 : cluster [DBG] pgmap v1185: 194 pgs: 32 unknown, 162 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:55:32.382 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:55:32.680 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:55:32.680 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:55:32.694 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:55:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:55:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:55:32.792 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:31.214514+0000 osd.4 (osd.4) 327 : cluster [DBG] 5.a scrub starts 2024-02-23T00:55:32.792 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:31.215333+0000 osd.4 (osd.4) 328 : cluster [DBG] 5.a scrub ok 2024-02-23T00:55:32.792 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:31.530520+0000 osd.1 (osd.1) 310 : cluster [DBG] 4.f scrub starts 2024-02-23T00:55:32.793 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:31.531886+0000 osd.1 (osd.1) 311 : cluster [DBG] 4.f scrub ok 2024-02-23T00:55:32.793 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:32.254674+0000 osd.3 (osd.3) 323 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:55:32.793 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:32.254797+0000 osd.3 (osd.3) 324 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:55:32.793 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:32.340392+0000 mon.a (mon.0) 3717 : cluster [DBG] osdmap e781: 8 total, 8 up, 6 in 2024-02-23T00:55:32.793 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:32.435268+0000 mon.c (mon.2) 488 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:55:32.793 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:32.435611+0000 mon.a (mon.0) 3718 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:55:32.854 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:55:32.856 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:32.857 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:55:32.958 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:55:32.984 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:31.214514+0000 osd.4 (osd.4) 327 : cluster [DBG] 5.a scrub starts 2024-02-23T00:55:32.984 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:31.215333+0000 osd.4 (osd.4) 328 : cluster [DBG] 5.a scrub ok 2024-02-23T00:55:32.984 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:31.530520+0000 osd.1 (osd.1) 310 : cluster [DBG] 4.f scrub starts 2024-02-23T00:55:32.984 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:31.531886+0000 osd.1 (osd.1) 311 : cluster [DBG] 4.f scrub ok 2024-02-23T00:55:32.984 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:32.254674+0000 osd.3 (osd.3) 323 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:55:32.984 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:32.254797+0000 osd.3 (osd.3) 324 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:55:32.984 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:32.340392+0000 mon.a (mon.0) 3717 : cluster [DBG] osdmap e781: 8 total, 8 up, 6 in 2024-02-23T00:55:32.984 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:32.435268+0000 mon.c (mon.2) 488 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:55:32.984 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:32.435611+0000 mon.a (mon.0) 3718 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:55:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:33.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:31.214514+0000 osd.4 (osd.4) 327 : cluster [DBG] 5.a scrub starts 2024-02-23T00:55:33.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:31.215333+0000 osd.4 (osd.4) 328 : cluster [DBG] 5.a scrub ok 2024-02-23T00:55:33.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:31.530520+0000 osd.1 (osd.1) 310 : cluster [DBG] 4.f scrub starts 2024-02-23T00:55:33.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:31.531886+0000 osd.1 (osd.1) 311 : cluster [DBG] 4.f scrub ok 2024-02-23T00:55:33.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:32.254674+0000 osd.3 (osd.3) 323 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:55:33.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:32.254797+0000 osd.3 (osd.3) 324 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:55:33.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:32.340392+0000 mon.a (mon.0) 3717 : cluster [DBG] osdmap e781: 8 total, 8 up, 6 in 2024-02-23T00:55:33.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:32.435268+0000 mon.c (mon.2) 488 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:55:33.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:32.435611+0000 mon.a (mon.0) 3718 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "hit_set_count","val": "2"}]: dispatch 2024-02-23T00:55:33.112 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:55:33.114 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:33.114 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:55:33.215 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:55:33.241 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:55:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:55:33.244 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:33.398 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:55:33.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.396+0000 7fbb23916700 1 -- 172.21.15.47:0/3253567126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb1c0fe510 msgr2=0x7fbb1c0f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:33.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.396+0000 7fbb23916700 1 --2- 172.21.15.47:0/3253567126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb1c0fe510 0x7fbb1c0f7d40 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 rx=0x7fbb0c004870 tx=0x7fbb0c009cb0).stop 2024-02-23T00:55:33.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.396+0000 7fbb23916700 1 -- 172.21.15.47:0/3253567126 shutdown_connections 2024-02-23T00:55:33.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.396+0000 7fbb23916700 1 --2- 172.21.15.47:0/3253567126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb1c0f8420 0x7fbb1c104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:33.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.396+0000 7fbb23916700 1 --2- 172.21.15.47:0/3253567126 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbb1c0fdbb0 0x7fbb1c0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:33.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.396+0000 7fbb23916700 1 --2- 172.21.15.47:0/3253567126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb1c0fe510 0x7fbb1c0f7d40 unknown :-1 s=CLOSED pgs=2639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:33.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.396+0000 7fbb23916700 1 -- 172.21.15.47:0/3253567126 >> 172.21.15.47:0/3253567126 conn(0x7fbb1c0f3800 msgr2=0x7fbb1c0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:33.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.396+0000 7fbb23916700 1 -- 172.21.15.47:0/3253567126 shutdown_connections 2024-02-23T00:55:33.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.396+0000 7fbb23916700 1 -- 172.21.15.47:0/3253567126 wait complete. 2024-02-23T00:55:33.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.396+0000 7fbb23916700 1 librados: starting msgr at 2024-02-23T00:55:33.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.396+0000 7fbb23916700 1 librados: starting objecter 2024-02-23T00:55:33.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.397+0000 7fbb23916700 1 Processor -- start 2024-02-23T00:55:33.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.397+0000 7fbb23916700 1 -- start start 2024-02-23T00:55:33.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.397+0000 7fbb23916700 1 librados: setting wanted keys 2024-02-23T00:55:33.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.397+0000 7fbb23916700 1 librados: calling monclient init 2024-02-23T00:55:33.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.397+0000 7fbb23916700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbb1c0f8420 0x7fbb1c06b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:33.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.397+0000 7fbb23916700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb1c0fdbb0 0x7fbb1c06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:33.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.397+0000 7fbb23916700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb1c0fe510 0x7fbb1c06bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:33.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.397+0000 7fbb23916700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c116390 con 0x7fbb1c0fe510 2024-02-23T00:55:33.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.397+0000 7fbb23916700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c116500 con 0x7fbb1c0fdbb0 2024-02-23T00:55:33.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.397+0000 7fbb23916700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c1166c0 con 0x7fbb1c0f8420 2024-02-23T00:55:33.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.397+0000 7fbb216b2700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbb1c0f8420 0x7fbb1c06b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:33.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.397+0000 7fbb216b2700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbb1c0f8420 0x7fbb1c06b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:53974/0 (socket says 172.21.15.47:53974) 2024-02-23T00:55:33.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.397+0000 7fbb216b2700 1 -- 172.21.15.47:0/505004440 learned_addr learned my addr 172.21.15.47:0/505004440 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:55:33.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.397+0000 7fbb20eb1700 1 --2- 172.21.15.47:0/505004440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb1c0fdbb0 0x7fbb1c06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:33.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.397+0000 7fbb216b2700 1 -- 172.21.15.47:0/505004440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb1c0fdbb0 msgr2=0x7fbb1c06b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:33.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.397+0000 7fbb216b2700 1 --2- 172.21.15.47:0/505004440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb1c0fdbb0 0x7fbb1c06b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:33.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.397+0000 7fbb216b2700 1 -- 172.21.15.47:0/505004440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb1c0fe510 msgr2=0x7fbb1c06bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:33.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.397+0000 7fbb21eb3700 1 --2- 172.21.15.47:0/505004440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb1c0fe510 0x7fbb1c06bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:33.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.398+0000 7fbb216b2700 1 --2- 172.21.15.47:0/505004440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb1c0fe510 0x7fbb1c06bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:33.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.398+0000 7fbb216b2700 1 -- 172.21.15.47:0/505004440 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb18005040 con 0x7fbb1c0f8420 2024-02-23T00:55:33.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.398+0000 7fbb21eb3700 1 --2- 172.21.15.47:0/505004440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb1c0fe510 0x7fbb1c06bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-23T00:55:33.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.398+0000 7fbb216b2700 1 --2- 172.21.15.47:0/505004440 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbb1c0f8420 0x7fbb1c06b0d0 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 rx=0x7fbb180048f0 tx=0x7fbb18009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:33.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.398+0000 7fbb127fc700 10 client.?.objecter ms_handle_connect 0x7fbb1c0f8420 2024-02-23T00:55:33.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.398+0000 7fbb127fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:55:33.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.398+0000 7fbb127fc700 1 -- 172.21.15.47:0/505004440 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb18007b00 con 0x7fbb1c0f8420 2024-02-23T00:55:33.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.398+0000 7fbb23916700 1 -- 172.21.15.47:0/505004440 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb0c005040 con 0x7fbb1c0f8420 2024-02-23T00:55:33.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.398+0000 7fbb127fc700 1 -- 172.21.15.47:0/505004440 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fbb1800f040 con 0x7fbb1c0f8420 2024-02-23T00:55:33.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.398+0000 7fbb127fc700 1 -- 172.21.15.47:0/505004440 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb18014890 con 0x7fbb1c0f8420 2024-02-23T00:55:33.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.398+0000 7fbb23916700 10 client.42134.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:55:33.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.398+0000 7fbb23916700 1 -- 172.21.15.47:0/505004440 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb1c116d50 con 0x7fbb1c0f8420 2024-02-23T00:55:33.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.398+0000 7fbb23916700 1 librados: init done 2024-02-23T00:55:33.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.399+0000 7fbb23916700 1 -- 172.21.15.47:0/505004440 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1c05ee80 con 0x7fbb1c0f8420 2024-02-23T00:55:33.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.399+0000 7fbb127fc700 1 -- 172.21.15.47:0/505004440 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fbb18014cc0 con 0x7fbb1c0f8420 2024-02-23T00:55:33.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.400+0000 7fbb127fc700 1 --2- 172.21.15.47:0/505004440 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbb08062cc0 0x7fbb08065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:33.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.400+0000 7fbb127fc700 1 -- 172.21.15.47:0/505004440 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(782..782 src has 621..782) v4 ==== 9719+0+0 (secure 0 0 0) 0x7fbb1807d450 con 0x7fbb1c0f8420 2024-02-23T00:55:33.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.400+0000 7fbb127fc700 10 client.42134.objecter ms_dispatch 0x7fbb1c001550 osd_map(782..782 src has 621..782) v4 2024-02-23T00:55:33.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.400+0000 7fbb127fc700 3 client.42134.objecter handle_osd_map got epochs [782,782] > 0 2024-02-23T00:55:33.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.400+0000 7fbb127fc700 3 client.42134.objecter handle_osd_map decoding full epoch 782 2024-02-23T00:55:33.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.400+0000 7fbb20eb1700 1 --2- 172.21.15.47:0/505004440 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbb08062cc0 0x7fbb08065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:33.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.400+0000 7fbb127fc700 20 client.42134.objecter dump_active .. 0 homeless 2024-02-23T00:55:33.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.400+0000 7fbb127fc700 10 client.42134.objecter ms_handle_connect 0x7fbb08062cc0 2024-02-23T00:55:33.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.400+0000 7fbb20eb1700 1 --2- 172.21.15.47:0/505004440 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbb08062cc0 0x7fbb08065180 secure :-1 s=READY pgs=3206 cs=0 l=1 rev1=1 rx=0x7fbb0c004ee0 tx=0x7fbb0c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:33.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.404+0000 7fbb127fc700 1 -- 172.21.15.47:0/505004440 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fbb18049f60 con 0x7fbb1c0f8420 2024-02-23T00:55:33.443 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:55:33.446 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:33.446 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:55:33.457 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.5 dump_historic_ops 2024-02-23T00:55:33.547 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:55:33.575 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:33.594 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:55:33.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:33.594+0000 7fbb23916700 1 -- 172.21.15.47:0/505004440 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7fbb1c05fb20 con 0x7fbb1c0f8420 2024-02-23T00:55:33.598 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:33.599 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:55:33.700 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:55:33.729 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:33.768 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:55:34.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:32.254407+0000 osd.4 (osd.4) 329 : cluster [DBG] 5.18 scrub starts 2024-02-23T00:55:34.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:32.255241+0000 osd.4 (osd.4) 330 : cluster [DBG] 5.18 scrub ok 2024-02-23T00:55:34.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:32.980728+0000 mgr.y (mgr.14517) 663 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:55:34.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:33.242664+0000 osd.5 (osd.5) 334 : cluster [DBG] 2.3 scrub starts 2024-02-23T00:55:34.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:33.243434+0000 osd.5 (osd.5) 335 : cluster [DBG] 2.3 scrub ok 2024-02-23T00:55:34.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:33.341849+0000 mon.a (mon.0) 3719 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "hit_set_count","val": "2"}]': finished 2024-02-23T00:55:34.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:33.341936+0000 mon.a (mon.0) 3720 : cluster [DBG] osdmap e782: 8 total, 8 up, 6 in 2024-02-23T00:55:34.052 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:33.345887+0000 mon.c (mon.2) 489 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:55:34.052 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:33.346255+0000 mon.a (mon.0) 3721 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:55:34.052 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:33.402183+0000 mgr.y (mgr.14517) 664 : cluster [DBG] pgmap v1188: 194 pgs: 93 peering, 101 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-23T00:55:34.052 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:33.596094+0000 mon.c (mon.2) 490 : audit [INF] from='client.? 172.21.15.47:0/505004440' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:55:34.052 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:33.596405+0000 mon.a (mon.0) 3722 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:55:34.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:32.254407+0000 osd.4 (osd.4) 329 : cluster [DBG] 5.18 scrub starts 2024-02-23T00:55:34.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:32.255241+0000 osd.4 (osd.4) 330 : cluster [DBG] 5.18 scrub ok 2024-02-23T00:55:34.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:32.980728+0000 mgr.y (mgr.14517) 663 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:55:34.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:33.242664+0000 osd.5 (osd.5) 334 : cluster [DBG] 2.3 scrub starts 2024-02-23T00:55:34.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:33.243434+0000 osd.5 (osd.5) 335 : cluster [DBG] 2.3 scrub ok 2024-02-23T00:55:34.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:33.341849+0000 mon.a (mon.0) 3719 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "hit_set_count","val": "2"}]': finished 2024-02-23T00:55:34.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:33.341936+0000 mon.a (mon.0) 3720 : cluster [DBG] osdmap e782: 8 total, 8 up, 6 in 2024-02-23T00:55:34.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:33.345887+0000 mon.c (mon.2) 489 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:55:34.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:33.346255+0000 mon.a (mon.0) 3721 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:55:34.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:33.402183+0000 mgr.y (mgr.14517) 664 : cluster [DBG] pgmap v1188: 194 pgs: 93 peering, 101 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-23T00:55:34.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:33.596094+0000 mon.c (mon.2) 490 : audit [INF] from='client.? 172.21.15.47:0/505004440' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:55:34.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:33.596405+0000 mon.a (mon.0) 3722 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:55:34.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:32.254407+0000 osd.4 (osd.4) 329 : cluster [DBG] 5.18 scrub starts 2024-02-23T00:55:34.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:32.255241+0000 osd.4 (osd.4) 330 : cluster [DBG] 5.18 scrub ok 2024-02-23T00:55:34.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:32.980728+0000 mgr.y (mgr.14517) 663 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:55:34.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:33.242664+0000 osd.5 (osd.5) 334 : cluster [DBG] 2.3 scrub starts 2024-02-23T00:55:34.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:33.243434+0000 osd.5 (osd.5) 335 : cluster [DBG] 2.3 scrub ok 2024-02-23T00:55:34.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:33.341849+0000 mon.a (mon.0) 3719 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "hit_set_count","val": "2"}]': finished 2024-02-23T00:55:34.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:33.341936+0000 mon.a (mon.0) 3720 : cluster [DBG] osdmap e782: 8 total, 8 up, 6 in 2024-02-23T00:55:34.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:33.345887+0000 mon.c (mon.2) 489 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:55:34.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:33.346255+0000 mon.a (mon.0) 3721 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "hit_set_period","val": "600"}]: dispatch 2024-02-23T00:55:34.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:33.402183+0000 mgr.y (mgr.14517) 664 : cluster [DBG] pgmap v1188: 194 pgs: 93 peering, 101 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-23T00:55:34.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:33.596094+0000 mon.c (mon.2) 490 : audit [INF] from='client.? 172.21.15.47:0/505004440' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:55:34.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:33.596405+0000 mon.a (mon.0) 3722 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:55:34.078 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:55:34.080 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:34.081 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:55:34.182 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:55:34.210 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:34.237 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:55:34.240 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:34.241 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:55:34.341 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:55:34.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:34.344+0000 7fbb127fc700 1 -- 172.21.15.47:0/505004440 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v783) v1 ==== 87+0+0 (secure 0 0 0) 0x7fbb1804c870 con 0x7fbb1c0f8420 2024-02-23T00:55:34.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:34.347+0000 7fbb23916700 10 librados: watch_flush enter 2024-02-23T00:55:34.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:34.347+0000 7fbb23916700 10 librados: watch_flush exit 2024-02-23T00:55:34.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:34.347+0000 7fbb23916700 20 client.42134.objecter shutdown clearing up homeless session... 2024-02-23T00:55:34.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:34.347+0000 7fbb23916700 10 client.42134.objecter successfully canceled tick 2024-02-23T00:55:34.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:34.347+0000 7fbb23916700 1 -- 172.21.15.47:0/505004440 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbb08062cc0 msgr2=0x7fbb08065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:34.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:34.347+0000 7fbb23916700 1 --2- 172.21.15.47:0/505004440 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbb08062cc0 0x7fbb08065180 secure :-1 s=READY pgs=3206 cs=0 l=1 rev1=1 rx=0x7fbb0c004ee0 tx=0x7fbb0c00b040).stop 2024-02-23T00:55:34.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:34.347+0000 7fbb23916700 1 -- 172.21.15.47:0/505004440 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbb1c0f8420 msgr2=0x7fbb1c06b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:34.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:34.347+0000 7fbb23916700 1 --2- 172.21.15.47:0/505004440 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbb1c0f8420 0x7fbb1c06b0d0 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 rx=0x7fbb180048f0 tx=0x7fbb18009cb0).stop 2024-02-23T00:55:34.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:34.347+0000 7fbb23916700 1 -- 172.21.15.47:0/505004440 shutdown_connections 2024-02-23T00:55:34.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:34.347+0000 7fbb23916700 1 --2- 172.21.15.47:0/505004440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb1c0fe510 0x7fbb1c06bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:34.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:34.347+0000 7fbb23916700 1 --2- 172.21.15.47:0/505004440 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fbb08062cc0 0x7fbb08065180 unknown :-1 s=CLOSED pgs=3206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:34.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:34.347+0000 7fbb23916700 1 --2- 172.21.15.47:0/505004440 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbb1c0f8420 0x7fbb1c06b0d0 unknown :-1 s=CLOSED pgs=2819 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:34.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:34.347+0000 7fbb23916700 1 --2- 172.21.15.47:0/505004440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb1c0fdbb0 0x7fbb1c06b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:34.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:34.347+0000 7fbb23916700 1 -- 172.21.15.47:0/505004440 >> 172.21.15.47:0/505004440 conn(0x7fbb1c0f3800 msgr2=0x7fbb1c0fc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:34.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:34.348+0000 7fbb23916700 1 -- 172.21.15.47:0/505004440 shutdown_connections 2024-02-23T00:55:34.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:34.348+0000 7fbb23916700 1 -- 172.21.15.47:0/505004440 wait complete. 2024-02-23T00:55:34.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:34.348+0000 7fbb23916700 1 librados: shutdown 2024-02-23T00:55:34.352 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is set 2024-02-23T00:55:34.376 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:35.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:33.251687+0000 osd.4 (osd.4) 331 : cluster [DBG] 5.1a scrub starts 2024-02-23T00:55:35.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:33.252467+0000 osd.4 (osd.4) 332 : cluster [DBG] 5.1a scrub ok 2024-02-23T00:55:35.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:34.339301+0000 mon.a (mon.0) 3723 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:55:35.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:34.339615+0000 mon.a (mon.0) 3724 : cluster [WRN] Health check failed: Reduced data availability: 14 pgs inactive, 19 pgs peering (PG_AVAILABILITY) 2024-02-23T00:55:35.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:34.343899+0000 mon.a (mon.0) 3725 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "hit_set_period","val": "600"}]': finished 2024-02-23T00:55:35.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:34.343972+0000 mon.a (mon.0) 3726 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:55:35.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:34.344035+0000 mon.a (mon.0) 3727 : cluster [DBG] osdmap e783: 8 total, 8 up, 6 in 2024-02-23T00:55:35.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:34.346091+0000 mon.c (mon.2) 491 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-23T00:55:35.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:34.348804+0000 mon.a (mon.0) 3728 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-23T00:55:35.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:33.251687+0000 osd.4 (osd.4) 331 : cluster [DBG] 5.1a scrub starts 2024-02-23T00:55:35.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:33.252467+0000 osd.4 (osd.4) 332 : cluster [DBG] 5.1a scrub ok 2024-02-23T00:55:35.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:34.339301+0000 mon.a (mon.0) 3723 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:55:35.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:34.339615+0000 mon.a (mon.0) 3724 : cluster [WRN] Health check failed: Reduced data availability: 14 pgs inactive, 19 pgs peering (PG_AVAILABILITY) 2024-02-23T00:55:35.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:34.343899+0000 mon.a (mon.0) 3725 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "hit_set_period","val": "600"}]': finished 2024-02-23T00:55:35.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:34.343972+0000 mon.a (mon.0) 3726 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:55:35.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:34.344035+0000 mon.a (mon.0) 3727 : cluster [DBG] osdmap e783: 8 total, 8 up, 6 in 2024-02-23T00:55:35.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:34.346091+0000 mon.c (mon.2) 491 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-23T00:55:35.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:34.348804+0000 mon.a (mon.0) 3728 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-23T00:55:35.089 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:33.251687+0000 osd.4 (osd.4) 331 : cluster [DBG] 5.1a scrub starts 2024-02-23T00:55:35.089 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:33.252467+0000 osd.4 (osd.4) 332 : cluster [DBG] 5.1a scrub ok 2024-02-23T00:55:35.089 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:34.339301+0000 mon.a (mon.0) 3723 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:55:35.089 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:34.339615+0000 mon.a (mon.0) 3724 : cluster [WRN] Health check failed: Reduced data availability: 14 pgs inactive, 19 pgs peering (PG_AVAILABILITY) 2024-02-23T00:55:35.089 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:34.343899+0000 mon.a (mon.0) 3725 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "hit_set_period","val": "600"}]': finished 2024-02-23T00:55:35.089 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:34.343972+0000 mon.a (mon.0) 3726 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:55:35.089 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:34.344035+0000 mon.a (mon.0) 3727 : cluster [DBG] osdmap e783: 8 total, 8 up, 6 in 2024-02-23T00:55:35.089 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:34.346091+0000 mon.c (mon.2) 491 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-23T00:55:35.089 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:34.348804+0000 mon.a (mon.0) 3728 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-23T00:55:35.127 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:55:35.129 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:35.130 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:55:35.231 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:55:35.258 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:35.378 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 00:55:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T00:55:35+0000 lvl=info msg="Database locked, sleeping then retrying" logger=sqlstore error="database is locked" retry=0 2024-02-23T00:55:35.390 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:55:35.392 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:35.393 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:55:35.493 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:55:35.525 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:35.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.599+0000 7fd34caee700 1 -- 172.21.15.47:0/1090999199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3480f8d10 msgr2=0x7fd3480f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:35.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.599+0000 7fd34caee700 1 --2- 172.21.15.47:0/1090999199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3480f8d10 0x7fd3480f90f0 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 rx=0x7fd338004870 tx=0x7fd338009cb0).stop 2024-02-23T00:55:35.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.599+0000 7fd34caee700 1 -- 172.21.15.47:0/1090999199 shutdown_connections 2024-02-23T00:55:35.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.599+0000 7fd34caee700 1 --2- 172.21.15.47:0/1090999199 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd348102270 0x7fd348104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:35.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.599+0000 7fd34caee700 1 --2- 172.21.15.47:0/1090999199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3480f8d10 0x7fd3480f90f0 unknown :-1 s=CLOSED pgs=2640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:35.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.599+0000 7fd34caee700 1 --2- 172.21.15.47:0/1090999199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3480f9670 0x7fd348101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:35.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.599+0000 7fd34caee700 1 -- 172.21.15.47:0/1090999199 >> 172.21.15.47:0/1090999199 conn(0x7fd3480f2820 msgr2=0x7fd3480f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:35.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.599+0000 7fd34caee700 1 -- 172.21.15.47:0/1090999199 shutdown_connections 2024-02-23T00:55:35.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.599+0000 7fd34caee700 1 -- 172.21.15.47:0/1090999199 wait complete. 2024-02-23T00:55:35.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.600+0000 7fd34caee700 1 librados: starting msgr at 2024-02-23T00:55:35.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.600+0000 7fd34caee700 1 librados: starting objecter 2024-02-23T00:55:35.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.600+0000 7fd34caee700 1 Processor -- start 2024-02-23T00:55:35.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.600+0000 7fd34caee700 1 -- start start 2024-02-23T00:55:35.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.600+0000 7fd34caee700 1 librados: setting wanted keys 2024-02-23T00:55:35.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.600+0000 7fd34caee700 1 librados: calling monclient init 2024-02-23T00:55:35.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.600+0000 7fd34caee700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3480f8d10 0x7fd34810e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:35.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.600+0000 7fd34caee700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3480f9670 0x7fd34810e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:35.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.600+0000 7fd34caee700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd348102270 0x7fd348108260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:35.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.600+0000 7fd34caee700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd348108750 con 0x7fd3480f9670 2024-02-23T00:55:35.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.600+0000 7fd34caee700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3481088c0 con 0x7fd3480f8d10 2024-02-23T00:55:35.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.600+0000 7fd34caee700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd348108a80 con 0x7fd348102270 2024-02-23T00:55:35.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.601+0000 7fd345d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3480f9670 0x7fd34810e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:35.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.601+0000 7fd345d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3480f9670 0x7fd34810e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35728/0 (socket says 172.21.15.47:35728) 2024-02-23T00:55:35.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.601+0000 7fd346d9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd348102270 0x7fd348108260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:35.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.601+0000 7fd34659c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3480f8d10 0x7fd34810e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:35.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.601+0000 7fd345d9b700 1 -- 172.21.15.47:0/3787533086 learned_addr learned my addr 172.21.15.47:0/3787533086 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:55:35.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.601+0000 7fd345d9b700 1 -- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd348102270 msgr2=0x7fd348108260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:35.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.601+0000 7fd345d9b700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd348102270 0x7fd348108260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:35.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.601+0000 7fd345d9b700 1 -- 172.21.15.47:0/3787533086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3480f8d10 msgr2=0x7fd34810e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:35.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.601+0000 7fd345d9b700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3480f8d10 0x7fd34810e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:35.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.601+0000 7fd345d9b700 1 -- 172.21.15.47:0/3787533086 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3400097c0 con 0x7fd3480f9670 2024-02-23T00:55:35.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.601+0000 7fd346d9d700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd348102270 0x7fd348108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:55:35.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.601+0000 7fd345d9b700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3480f9670 0x7fd34810e560 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 rx=0x7fd3300041d0 tx=0x7fd330009f60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:35.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.601+0000 7fd33f7fe700 10 client.?.objecter ms_handle_connect 0x7fd3480f9670 2024-02-23T00:55:35.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.601+0000 7fd33f7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:55:35.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.601+0000 7fd33f7fe700 1 -- 172.21.15.47:0/3787533086 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd330007b40 con 0x7fd3480f9670 2024-02-23T00:55:35.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.601+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd338005040 con 0x7fd3480f9670 2024-02-23T00:55:35.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.601+0000 7fd33f7fe700 1 -- 172.21.15.47:0/3787533086 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fd33000f040 con 0x7fd3480f9670 2024-02-23T00:55:35.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.601+0000 7fd33f7fe700 1 -- 172.21.15.47:0/3787533086 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd330014730 con 0x7fd3480f9670 2024-02-23T00:55:35.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.601+0000 7fd34caee700 10 client.31815.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:55:35.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.601+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3481090c0 con 0x7fd3480f9670 2024-02-23T00:55:35.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.601+0000 7fd34caee700 1 librados: init done 2024-02-23T00:55:35.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.602+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fd34805fa60 con 0x7fd3480f9670 2024-02-23T00:55:35.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.603+0000 7fd33f7fe700 1 -- 172.21.15.47:0/3787533086 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fd330009110 con 0x7fd3480f9670 2024-02-23T00:55:35.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.603+0000 7fd33f7fe700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd334062cc0 0x7fd334065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:35.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.603+0000 7fd33f7fe700 1 -- 172.21.15.47:0/3787533086 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(784..784 src has 621..784) v4 ==== 9745+0+0 (secure 0 0 0) 0x7fd33007d890 con 0x7fd3480f9670 2024-02-23T00:55:35.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.603+0000 7fd33f7fe700 10 client.31815.objecter ms_dispatch 0x7fd348001550 osd_map(784..784 src has 621..784) v4 2024-02-23T00:55:35.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.603+0000 7fd33f7fe700 3 client.31815.objecter handle_osd_map got epochs [784,784] > 0 2024-02-23T00:55:35.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.603+0000 7fd33f7fe700 3 client.31815.objecter handle_osd_map decoding full epoch 784 2024-02-23T00:55:35.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.603+0000 7fd34659c700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd334062cc0 0x7fd334065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:35.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.603+0000 7fd33f7fe700 20 client.31815.objecter dump_active .. 0 homeless 2024-02-23T00:55:35.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.603+0000 7fd33f7fe700 1 -- 172.21.15.47:0/3787533086 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v784) v1 ==== 54+0+16 (secure 0 0 0) 0x7fd330082220 con 0x7fd3480f9670 2024-02-23T00:55:35.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.603+0000 7fd34caee700 10 client.31815.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:35.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.603+0000 7fd34caee700 15 client.31815.objecter _session_command_op_assign -1 1 2024-02-23T00:55:35.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.603+0000 7fd34659c700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd334062cc0 0x7fd334065180 secure :-1 s=READY pgs=3207 cs=0 l=1 rev1=1 rx=0x7fd338004ee0 tx=0x7fd33800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:35.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.603+0000 7fd33f7fe700 10 client.31815.objecter ms_handle_connect 0x7fd334062cc0 2024-02-23T00:55:35.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.603+0000 7fd34caee700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fd348028540 0x7fd3480fbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:35.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.604+0000 7fd34caee700 20 client.31815.objecter _get_session s=0x7fd34805ee80 osd=0 3 2024-02-23T00:55:35.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.604+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd34805ee80 osd=0 3 2024-02-23T00:55:35.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.604+0000 7fd34caee700 20 client.31815.objecter _get_session s=0x7fd34805ee80 osd=0 3 2024-02-23T00:55:35.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.604+0000 7fd34caee700 15 client.31815.objecter _session_command_op_remove -1 1 2024-02-23T00:55:35.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.604+0000 7fd34caee700 20 client.31815.objecter get_session s=0x7fd34805ee80 osd=0 3 2024-02-23T00:55:35.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.604+0000 7fd34caee700 15 client.31815.objecter _session_command_op_assign 0 1 2024-02-23T00:55:35.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.604+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd34805ee80 osd=0 4 2024-02-23T00:55:35.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.604+0000 7fd34caee700 10 client.31815.objecter _send_command 1 2024-02-23T00:55:35.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.604+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd348107060 con 0x7fd348028540 2024-02-23T00:55:35.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.604+0000 7fd346d9d700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fd348028540 0x7fd3480fbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:35.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.604+0000 7fd33f7fe700 10 client.31815.objecter ms_handle_connect 0x7fd348028540 2024-02-23T00:55:35.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.604+0000 7fd346d9d700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fd348028540 0x7fd3480fbd30 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:35.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.604+0000 7fd33f7fe700 1 -- 172.21.15.47:0/3787533086 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd348107060 con 0x7fd348028540 2024-02-23T00:55:35.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.604+0000 7fd33f7fe700 10 client.31815.objecter ms_dispatch 0x7fd348001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:55:35.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.604+0000 7fd33f7fe700 10 client.31815.objecter _finish_command 1 = system:0 2024-02-23T00:55:35.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.604+0000 7fd33f7fe700 20 client.31815.objecter put_session s=0x7fd34805ee80 osd=0 4 2024-02-23T00:55:35.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.604+0000 7fd33f7fe700 15 client.31815.objecter _session_command_op_remove 0 1 2024-02-23T00:55:35.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.619+0000 7fd34caee700 10 client.31815.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:55:35.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.619+0000 7fd34caee700 15 client.31815.objecter _session_command_op_assign -1 2 2024-02-23T00:55:35.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.619+0000 7fd34caee700 20 client.31815.objecter _get_session s=0x7fd34805ee80 osd=0 3 2024-02-23T00:55:35.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.619+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd34805ee80 osd=0 3 2024-02-23T00:55:35.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.619+0000 7fd34caee700 20 client.31815.objecter _get_session s=0x7fd34805ee80 osd=0 3 2024-02-23T00:55:35.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.619+0000 7fd34caee700 15 client.31815.objecter _session_command_op_remove -1 2 2024-02-23T00:55:35.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.619+0000 7fd34caee700 20 client.31815.objecter get_session s=0x7fd34805ee80 osd=0 3 2024-02-23T00:55:35.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.619+0000 7fd34caee700 15 client.31815.objecter _session_command_op_assign 0 2 2024-02-23T00:55:35.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.619+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd34805ee80 osd=0 4 2024-02-23T00:55:35.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.619+0000 7fd34caee700 10 client.31815.objecter _send_command 2 2024-02-23T00:55:35.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.619+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd34805a040 con 0x7fd348028540 2024-02-23T00:55:35.626 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:55:35.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.622+0000 7fd33f7fe700 1 -- 172.21.15.47:0/3787533086 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd34805a040 con 0x7fd348028540 2024-02-23T00:55:35.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.622+0000 7fd33f7fe700 10 client.31815.objecter ms_dispatch 0x7fd348001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:55:35.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.622+0000 7fd33f7fe700 10 client.31815.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:55:35.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.622+0000 7fd33f7fe700 20 client.31815.objecter put_session s=0x7fd34805ee80 osd=0 4 2024-02-23T00:55:35.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.622+0000 7fd33f7fe700 15 client.31815.objecter _session_command_op_remove 0 2 2024-02-23T00:55:35.628 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:55:35.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.622+0000 7fd34caee700 10 client.31815.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:35.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.622+0000 7fd34caee700 15 client.31815.objecter _session_command_op_assign -1 3 2024-02-23T00:55:35.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.622+0000 7fd34caee700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fd348027b90 0x7fd348027f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:35.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.622+0000 7fd34caee700 20 client.31815.objecter _get_session s=0x7fd3480fd3a0 osd=1 3 2024-02-23T00:55:35.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.622+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd3480fd3a0 osd=1 3 2024-02-23T00:55:35.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.622+0000 7fd34caee700 20 client.31815.objecter _get_session s=0x7fd3480fd3a0 osd=1 3 2024-02-23T00:55:35.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.622+0000 7fd34caee700 15 client.31815.objecter _session_command_op_remove -1 3 2024-02-23T00:55:35.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.622+0000 7fd34caee700 20 client.31815.objecter get_session s=0x7fd3480fd3a0 osd=1 3 2024-02-23T00:55:35.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.622+0000 7fd34caee700 15 client.31815.objecter _session_command_op_assign 1 3 2024-02-23T00:55:35.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.622+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd3480fd3a0 osd=1 4 2024-02-23T00:55:35.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.622+0000 7fd34caee700 10 client.31815.objecter _send_command 3 2024-02-23T00:55:35.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.622+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd3480fd510 con 0x7fd348027b90 2024-02-23T00:55:35.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.622+0000 7fd34659c700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fd348027b90 0x7fd348027f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:35.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.622+0000 7fd33f7fe700 10 client.31815.objecter ms_handle_connect 0x7fd348027b90 2024-02-23T00:55:35.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.622+0000 7fd34659c700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fd348027b90 0x7fd348027f70 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:35.630 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:35.631 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:55:35.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.623+0000 7fd33f7fe700 1 -- 172.21.15.47:0/3787533086 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd3480fd510 con 0x7fd348027b90 2024-02-23T00:55:35.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.623+0000 7fd33f7fe700 10 client.31815.objecter ms_dispatch 0x7fd348001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:55:35.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.623+0000 7fd33f7fe700 10 client.31815.objecter _finish_command 3 = system:0 2024-02-23T00:55:35.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.623+0000 7fd33f7fe700 20 client.31815.objecter put_session s=0x7fd3480fd3a0 osd=1 4 2024-02-23T00:55:35.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.623+0000 7fd33f7fe700 15 client.31815.objecter _session_command_op_remove 1 3 2024-02-23T00:55:35.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.638+0000 7fd34caee700 10 client.31815.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:55:35.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.638+0000 7fd34caee700 15 client.31815.objecter _session_command_op_assign -1 4 2024-02-23T00:55:35.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.638+0000 7fd34caee700 20 client.31815.objecter _get_session s=0x7fd3480fd3a0 osd=1 3 2024-02-23T00:55:35.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.638+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd3480fd3a0 osd=1 3 2024-02-23T00:55:35.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.638+0000 7fd34caee700 20 client.31815.objecter _get_session s=0x7fd3480fd3a0 osd=1 3 2024-02-23T00:55:35.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.638+0000 7fd34caee700 15 client.31815.objecter _session_command_op_remove -1 4 2024-02-23T00:55:35.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.638+0000 7fd34caee700 20 client.31815.objecter get_session s=0x7fd3480fd3a0 osd=1 3 2024-02-23T00:55:35.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.638+0000 7fd34caee700 15 client.31815.objecter _session_command_op_assign 1 4 2024-02-23T00:55:35.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.638+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd3480fd3a0 osd=1 4 2024-02-23T00:55:35.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.638+0000 7fd34caee700 10 client.31815.objecter _send_command 4 2024-02-23T00:55:35.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.638+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd348027600 con 0x7fd348027b90 2024-02-23T00:55:35.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.639+0000 7fd33f7fe700 1 -- 172.21.15.47:0/3787533086 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd348027600 con 0x7fd348027b90 2024-02-23T00:55:35.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.639+0000 7fd33f7fe700 10 client.31815.objecter ms_dispatch 0x7fd348001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:55:35.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.639+0000 7fd33f7fe700 10 client.31815.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:55:35.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.639+0000 7fd33f7fe700 20 client.31815.objecter put_session s=0x7fd3480fd3a0 osd=1 4 2024-02-23T00:55:35.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.639+0000 7fd33f7fe700 15 client.31815.objecter _session_command_op_remove 1 4 2024-02-23T00:55:35.643 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:55:35.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.640+0000 7fd34caee700 10 client.31815.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:35.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.640+0000 7fd34caee700 15 client.31815.objecter _session_command_op_assign -1 5 2024-02-23T00:55:35.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.640+0000 7fd34caee700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fd3480fcf00 0x7fd34811e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:35.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.640+0000 7fd34caee700 20 client.31815.objecter _get_session s=0x7fd348027a10 osd=2 3 2024-02-23T00:55:35.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.640+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd348027a10 osd=2 3 2024-02-23T00:55:35.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.640+0000 7fd34caee700 20 client.31815.objecter _get_session s=0x7fd348027a10 osd=2 3 2024-02-23T00:55:35.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.640+0000 7fd34caee700 15 client.31815.objecter _session_command_op_remove -1 5 2024-02-23T00:55:35.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.640+0000 7fd34caee700 20 client.31815.objecter get_session s=0x7fd348027a10 osd=2 3 2024-02-23T00:55:35.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.640+0000 7fd34caee700 15 client.31815.objecter _session_command_op_assign 2 5 2024-02-23T00:55:35.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.640+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd348027a10 osd=2 4 2024-02-23T00:55:35.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.640+0000 7fd34caee700 10 client.31815.objecter _send_command 5 2024-02-23T00:55:35.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.640+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd3480f8ba0 con 0x7fd3480fcf00 2024-02-23T00:55:35.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.640+0000 7fd345d9b700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fd3480fcf00 0x7fd34811e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:35.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.640+0000 7fd33f7fe700 10 client.31815.objecter ms_handle_connect 0x7fd3480fcf00 2024-02-23T00:55:35.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.640+0000 7fd345d9b700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fd3480fcf00 0x7fd34811e510 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:35.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.641+0000 7fd33f7fe700 1 -- 172.21.15.47:0/3787533086 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd3480f8ba0 con 0x7fd3480fcf00 2024-02-23T00:55:35.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.641+0000 7fd33f7fe700 10 client.31815.objecter ms_dispatch 0x7fd348001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:55:35.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.641+0000 7fd33f7fe700 10 client.31815.objecter _finish_command 5 = system:0 2024-02-23T00:55:35.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.641+0000 7fd33f7fe700 20 client.31815.objecter put_session s=0x7fd348027a10 osd=2 4 2024-02-23T00:55:35.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.641+0000 7fd33f7fe700 15 client.31815.objecter _session_command_op_remove 2 5 2024-02-23T00:55:35.646 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:55:35.647 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:55:35.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.655+0000 7fd34caee700 10 client.31815.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:55:35.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.655+0000 7fd34caee700 15 client.31815.objecter _session_command_op_assign -1 6 2024-02-23T00:55:35.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.655+0000 7fd34caee700 20 client.31815.objecter _get_session s=0x7fd348027a10 osd=2 3 2024-02-23T00:55:35.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.655+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd348027a10 osd=2 3 2024-02-23T00:55:35.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.655+0000 7fd34caee700 20 client.31815.objecter _get_session s=0x7fd348027a10 osd=2 3 2024-02-23T00:55:35.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.655+0000 7fd34caee700 15 client.31815.objecter _session_command_op_remove -1 6 2024-02-23T00:55:35.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.655+0000 7fd34caee700 20 client.31815.objecter get_session s=0x7fd348027a10 osd=2 3 2024-02-23T00:55:35.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.655+0000 7fd34caee700 15 client.31815.objecter _session_command_op_assign 2 6 2024-02-23T00:55:35.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.655+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd348027a10 osd=2 4 2024-02-23T00:55:35.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.655+0000 7fd34caee700 10 client.31815.objecter _send_command 6 2024-02-23T00:55:35.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.655+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd3480fc460 con 0x7fd3480fcf00 2024-02-23T00:55:35.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.657+0000 7fd33f7fe700 1 -- 172.21.15.47:0/3787533086 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd3480fc460 con 0x7fd3480fcf00 2024-02-23T00:55:35.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.657+0000 7fd33f7fe700 10 client.31815.objecter ms_dispatch 0x7fd348001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:55:35.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.657+0000 7fd33f7fe700 10 client.31815.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:55:35.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.657+0000 7fd33f7fe700 20 client.31815.objecter put_session s=0x7fd348027a10 osd=2 4 2024-02-23T00:55:35.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.657+0000 7fd33f7fe700 15 client.31815.objecter _session_command_op_remove 2 6 2024-02-23T00:55:35.660 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:55:35.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.657+0000 7fd34caee700 10 client.31815.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:35.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.657+0000 7fd34caee700 15 client.31815.objecter _session_command_op_assign -1 7 2024-02-23T00:55:35.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.657+0000 7fd34caee700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fd34811fb80 0x7fd348121f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:35.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.657+0000 7fd34caee700 20 client.31815.objecter _get_session s=0x7fd348119750 osd=3 3 2024-02-23T00:55:35.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.657+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd348119750 osd=3 3 2024-02-23T00:55:35.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.657+0000 7fd34caee700 20 client.31815.objecter _get_session s=0x7fd348119750 osd=3 3 2024-02-23T00:55:35.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.657+0000 7fd34caee700 15 client.31815.objecter _session_command_op_remove -1 7 2024-02-23T00:55:35.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.657+0000 7fd34caee700 20 client.31815.objecter get_session s=0x7fd348119750 osd=3 3 2024-02-23T00:55:35.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.657+0000 7fd34caee700 15 client.31815.objecter _session_command_op_assign 3 7 2024-02-23T00:55:35.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.657+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd348119750 osd=3 4 2024-02-23T00:55:35.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.657+0000 7fd34caee700 10 client.31815.objecter _send_command 7 2024-02-23T00:55:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.657+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd348119ed0 con 0x7fd34811fb80 2024-02-23T00:55:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.657+0000 7fd346d9d700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fd34811fb80 0x7fd348121f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.657+0000 7fd33f7fe700 10 client.31815.objecter ms_handle_connect 0x7fd34811fb80 2024-02-23T00:55:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.657+0000 7fd346d9d700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fd34811fb80 0x7fd348121f70 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.658+0000 7fd33f7fe700 1 -- 172.21.15.47:0/3787533086 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd348119ed0 con 0x7fd34811fb80 2024-02-23T00:55:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.658+0000 7fd33f7fe700 10 client.31815.objecter ms_dispatch 0x7fd348001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:55:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.658+0000 7fd33f7fe700 10 client.31815.objecter _finish_command 7 = system:0 2024-02-23T00:55:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.658+0000 7fd33f7fe700 20 client.31815.objecter put_session s=0x7fd348119750 osd=3 4 2024-02-23T00:55:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.658+0000 7fd33f7fe700 15 client.31815.objecter _session_command_op_remove 3 7 2024-02-23T00:55:35.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.672+0000 7fd34caee700 10 client.31815.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:55:35.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.672+0000 7fd34caee700 15 client.31815.objecter _session_command_op_assign -1 8 2024-02-23T00:55:35.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.672+0000 7fd34caee700 20 client.31815.objecter _get_session s=0x7fd348119750 osd=3 3 2024-02-23T00:55:35.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.672+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd348119750 osd=3 3 2024-02-23T00:55:35.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.672+0000 7fd34caee700 20 client.31815.objecter _get_session s=0x7fd348119750 osd=3 3 2024-02-23T00:55:35.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.672+0000 7fd34caee700 15 client.31815.objecter _session_command_op_remove -1 8 2024-02-23T00:55:35.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.672+0000 7fd34caee700 20 client.31815.objecter get_session s=0x7fd348119750 osd=3 3 2024-02-23T00:55:35.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.672+0000 7fd34caee700 15 client.31815.objecter _session_command_op_assign 3 8 2024-02-23T00:55:35.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.672+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd348119750 osd=3 4 2024-02-23T00:55:35.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.672+0000 7fd34caee700 10 client.31815.objecter _send_command 8 2024-02-23T00:55:35.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.672+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd3481191c0 con 0x7fd34811fb80 2024-02-23T00:55:35.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.674+0000 7fd33f7fe700 1 -- 172.21.15.47:0/3787533086 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd3481191c0 con 0x7fd34811fb80 2024-02-23T00:55:35.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.674+0000 7fd33f7fe700 10 client.31815.objecter ms_dispatch 0x7fd348001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:55:35.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.674+0000 7fd33f7fe700 10 client.31815.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:55:35.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.674+0000 7fd33f7fe700 20 client.31815.objecter put_session s=0x7fd348119750 osd=3 4 2024-02-23T00:55:35.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.674+0000 7fd33f7fe700 15 client.31815.objecter _session_command_op_remove 3 8 2024-02-23T00:55:35.677 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:55:35.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.674+0000 7fd34caee700 10 client.31815.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:35.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.674+0000 7fd34caee700 15 client.31815.objecter _session_command_op_assign -1 9 2024-02-23T00:55:35.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.674+0000 7fd34caee700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fd348119330 0x7fd3481255f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:35.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.674+0000 7fd34caee700 20 client.31815.objecter _get_session s=0x7fd3481191c0 osd=4 3 2024-02-23T00:55:35.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.674+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd3481191c0 osd=4 3 2024-02-23T00:55:35.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.674+0000 7fd34caee700 20 client.31815.objecter _get_session s=0x7fd3481191c0 osd=4 3 2024-02-23T00:55:35.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.674+0000 7fd34caee700 15 client.31815.objecter _session_command_op_remove -1 9 2024-02-23T00:55:35.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.674+0000 7fd34caee700 20 client.31815.objecter get_session s=0x7fd3481191c0 osd=4 3 2024-02-23T00:55:35.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.674+0000 7fd34caee700 15 client.31815.objecter _session_command_op_assign 4 9 2024-02-23T00:55:35.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.674+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd3481191c0 osd=4 4 2024-02-23T00:55:35.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.674+0000 7fd34caee700 10 client.31815.objecter _send_command 9 2024-02-23T00:55:35.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.674+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd34811f880 con 0x7fd348119330 2024-02-23T00:55:35.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.674+0000 7fd34659c700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fd348119330 0x7fd3481255f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:35.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.675+0000 7fd33f7fe700 10 client.31815.objecter ms_handle_connect 0x7fd348119330 2024-02-23T00:55:35.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.675+0000 7fd34659c700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fd348119330 0x7fd3481255f0 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:35.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.675+0000 7fd33f7fe700 1 -- 172.21.15.47:0/3787533086 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd34811f880 con 0x7fd348119330 2024-02-23T00:55:35.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.675+0000 7fd33f7fe700 10 client.31815.objecter ms_dispatch 0x7fd348001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:55:35.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.675+0000 7fd33f7fe700 10 client.31815.objecter _finish_command 9 = system:0 2024-02-23T00:55:35.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.675+0000 7fd33f7fe700 20 client.31815.objecter put_session s=0x7fd3481191c0 osd=4 4 2024-02-23T00:55:35.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.675+0000 7fd33f7fe700 15 client.31815.objecter _session_command_op_remove 4 9 2024-02-23T00:55:35.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.690+0000 7fd34caee700 10 client.31815.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:55:35.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.690+0000 7fd34caee700 15 client.31815.objecter _session_command_op_assign -1 10 2024-02-23T00:55:35.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.690+0000 7fd34caee700 20 client.31815.objecter _get_session s=0x7fd3481191c0 osd=4 3 2024-02-23T00:55:35.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.690+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd3481191c0 osd=4 3 2024-02-23T00:55:35.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.690+0000 7fd34caee700 20 client.31815.objecter _get_session s=0x7fd3481191c0 osd=4 3 2024-02-23T00:55:35.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.690+0000 7fd34caee700 15 client.31815.objecter _session_command_op_remove -1 10 2024-02-23T00:55:35.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.690+0000 7fd34caee700 20 client.31815.objecter get_session s=0x7fd3481191c0 osd=4 3 2024-02-23T00:55:35.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.690+0000 7fd34caee700 15 client.31815.objecter _session_command_op_assign 4 10 2024-02-23T00:55:35.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.690+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd3481191c0 osd=4 4 2024-02-23T00:55:35.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.690+0000 7fd34caee700 10 client.31815.objecter _send_command 10 2024-02-23T00:55:35.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.690+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd34811f880 con 0x7fd348119330 2024-02-23T00:55:35.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.692+0000 7fd33f7fe700 1 -- 172.21.15.47:0/3787533086 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd34811f880 con 0x7fd348119330 2024-02-23T00:55:35.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.692+0000 7fd33f7fe700 10 client.31815.objecter ms_dispatch 0x7fd348001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:55:35.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.692+0000 7fd33f7fe700 10 client.31815.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:55:35.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.692+0000 7fd33f7fe700 20 client.31815.objecter put_session s=0x7fd3481191c0 osd=4 4 2024-02-23T00:55:35.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.692+0000 7fd33f7fe700 15 client.31815.objecter _session_command_op_remove 4 10 2024-02-23T00:55:35.696 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:55:35.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.692+0000 7fd34caee700 10 client.31815.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:35.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.692+0000 7fd34caee700 15 client.31815.objecter _session_command_op_assign -1 11 2024-02-23T00:55:35.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.692+0000 7fd34caee700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fd348125ff0 0x7fd3481283e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:35.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.692+0000 7fd34caee700 20 client.31815.objecter _get_session s=0x7fd34811f880 osd=5 3 2024-02-23T00:55:35.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.692+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd34811f880 osd=5 3 2024-02-23T00:55:35.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.692+0000 7fd34caee700 20 client.31815.objecter _get_session s=0x7fd34811f880 osd=5 3 2024-02-23T00:55:35.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.692+0000 7fd34caee700 15 client.31815.objecter _session_command_op_remove -1 11 2024-02-23T00:55:35.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.692+0000 7fd34caee700 20 client.31815.objecter get_session s=0x7fd34811f880 osd=5 3 2024-02-23T00:55:35.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.692+0000 7fd34caee700 15 client.31815.objecter _session_command_op_assign 5 11 2024-02-23T00:55:35.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.692+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd34811f880 osd=5 4 2024-02-23T00:55:35.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.692+0000 7fd34caee700 10 client.31815.objecter _send_command 11 2024-02-23T00:55:35.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.692+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd34812b190 con 0x7fd348125ff0 2024-02-23T00:55:35.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.692+0000 7fd345d9b700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fd348125ff0 0x7fd3481283e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.693+0000 7fd33f7fe700 10 client.31815.objecter ms_handle_connect 0x7fd348125ff0 2024-02-23T00:55:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.693+0000 7fd345d9b700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fd348125ff0 0x7fd3481283e0 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.693+0000 7fd33f7fe700 1 -- 172.21.15.47:0/3787533086 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd34812b190 con 0x7fd348125ff0 2024-02-23T00:55:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.693+0000 7fd33f7fe700 10 client.31815.objecter ms_dispatch 0x7fd348001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:55:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.693+0000 7fd33f7fe700 10 client.31815.objecter _finish_command 11 = system:0 2024-02-23T00:55:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.693+0000 7fd33f7fe700 20 client.31815.objecter put_session s=0x7fd34811f880 osd=5 4 2024-02-23T00:55:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.693+0000 7fd33f7fe700 15 client.31815.objecter _session_command_op_remove 5 11 2024-02-23T00:55:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.708+0000 7fd34caee700 10 client.31815.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:55:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.708+0000 7fd34caee700 15 client.31815.objecter _session_command_op_assign -1 12 2024-02-23T00:55:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.708+0000 7fd34caee700 20 client.31815.objecter _get_session s=0x7fd34811f880 osd=5 3 2024-02-23T00:55:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.708+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd34811f880 osd=5 3 2024-02-23T00:55:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.708+0000 7fd34caee700 20 client.31815.objecter _get_session s=0x7fd34811f880 osd=5 3 2024-02-23T00:55:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.708+0000 7fd34caee700 15 client.31815.objecter _session_command_op_remove -1 12 2024-02-23T00:55:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.708+0000 7fd34caee700 20 client.31815.objecter get_session s=0x7fd34811f880 osd=5 3 2024-02-23T00:55:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.708+0000 7fd34caee700 15 client.31815.objecter _session_command_op_assign 5 12 2024-02-23T00:55:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.708+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd34811f880 osd=5 4 2024-02-23T00:55:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.708+0000 7fd34caee700 10 client.31815.objecter _send_command 12 2024-02-23T00:55:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.708+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd34812b190 con 0x7fd348125ff0 2024-02-23T00:55:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.710+0000 7fd33f7fe700 1 -- 172.21.15.47:0/3787533086 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd34812b190 con 0x7fd348125ff0 2024-02-23T00:55:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.710+0000 7fd33f7fe700 10 client.31815.objecter ms_dispatch 0x7fd348001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:55:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.710+0000 7fd33f7fe700 10 client.31815.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:55:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.710+0000 7fd33f7fe700 20 client.31815.objecter put_session s=0x7fd34811f880 osd=5 4 2024-02-23T00:55:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.710+0000 7fd33f7fe700 15 client.31815.objecter _session_command_op_remove 5 12 2024-02-23T00:55:35.714 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:55:35.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.710+0000 7fd34caee700 10 client.31815.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:35.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.710+0000 7fd34caee700 15 client.31815.objecter _session_command_op_assign -1 13 2024-02-23T00:55:35.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.710+0000 7fd34caee700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fd34812a490 0x7fd34812a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:35.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.710+0000 7fd34caee700 20 client.31815.objecter _get_session s=0x7fd34812b190 osd=6 3 2024-02-23T00:55:35.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.710+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd34812b190 osd=6 3 2024-02-23T00:55:35.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.710+0000 7fd34caee700 20 client.31815.objecter _get_session s=0x7fd34812b190 osd=6 3 2024-02-23T00:55:35.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.710+0000 7fd34caee700 15 client.31815.objecter _session_command_op_remove -1 13 2024-02-23T00:55:35.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.710+0000 7fd34caee700 20 client.31815.objecter get_session s=0x7fd34812b190 osd=6 3 2024-02-23T00:55:35.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.710+0000 7fd34caee700 15 client.31815.objecter _session_command_op_assign 6 13 2024-02-23T00:55:35.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.710+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd34812b190 osd=6 4 2024-02-23T00:55:35.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.710+0000 7fd34caee700 10 client.31815.objecter _send_command 13 2024-02-23T00:55:35.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.710+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd34812e510 con 0x7fd34812a490 2024-02-23T00:55:35.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.710+0000 7fd346d9d700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fd34812a490 0x7fd34812a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:35.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.710+0000 7fd33f7fe700 10 client.31815.objecter ms_handle_connect 0x7fd34812a490 2024-02-23T00:55:35.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.710+0000 7fd346d9d700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fd34812a490 0x7fd34812a8f0 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:35.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.711+0000 7fd33f7fe700 1 -- 172.21.15.47:0/3787533086 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd34812e510 con 0x7fd34812a490 2024-02-23T00:55:35.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.711+0000 7fd33f7fe700 10 client.31815.objecter ms_dispatch 0x7fd348001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:55:35.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.711+0000 7fd33f7fe700 10 client.31815.objecter _finish_command 13 = system:0 2024-02-23T00:55:35.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.711+0000 7fd33f7fe700 20 client.31815.objecter put_session s=0x7fd34812b190 osd=6 4 2024-02-23T00:55:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.711+0000 7fd33f7fe700 15 client.31815.objecter _session_command_op_remove 6 13 2024-02-23T00:55:35.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.726+0000 7fd34caee700 10 client.31815.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:55:35.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.726+0000 7fd34caee700 15 client.31815.objecter _session_command_op_assign -1 14 2024-02-23T00:55:35.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.726+0000 7fd34caee700 20 client.31815.objecter _get_session s=0x7fd34812b190 osd=6 3 2024-02-23T00:55:35.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.726+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd34812b190 osd=6 3 2024-02-23T00:55:35.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.726+0000 7fd34caee700 20 client.31815.objecter _get_session s=0x7fd34812b190 osd=6 3 2024-02-23T00:55:35.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.726+0000 7fd34caee700 15 client.31815.objecter _session_command_op_remove -1 14 2024-02-23T00:55:35.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.726+0000 7fd34caee700 20 client.31815.objecter get_session s=0x7fd34812b190 osd=6 3 2024-02-23T00:55:35.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.726+0000 7fd34caee700 15 client.31815.objecter _session_command_op_assign 6 14 2024-02-23T00:55:35.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.726+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd34812b190 osd=6 4 2024-02-23T00:55:35.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.726+0000 7fd34caee700 10 client.31815.objecter _send_command 14 2024-02-23T00:55:35.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.726+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd348128970 con 0x7fd34812a490 2024-02-23T00:55:35.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.728+0000 7fd33f7fe700 1 -- 172.21.15.47:0/3787533086 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd348128970 con 0x7fd34812a490 2024-02-23T00:55:35.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.728+0000 7fd33f7fe700 10 client.31815.objecter ms_dispatch 0x7fd348001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:55:35.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.728+0000 7fd33f7fe700 10 client.31815.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:55:35.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.728+0000 7fd33f7fe700 20 client.31815.objecter put_session s=0x7fd34812b190 osd=6 4 2024-02-23T00:55:35.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.728+0000 7fd33f7fe700 15 client.31815.objecter _session_command_op_remove 6 14 2024-02-23T00:55:35.731 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:55:35.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.728+0000 7fd34caee700 10 client.31815.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:35.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.728+0000 7fd34caee700 15 client.31815.objecter _session_command_op_assign -1 15 2024-02-23T00:55:35.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.728+0000 7fd34caee700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fd348128ff0 0x7fd348129450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:35.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.728+0000 7fd34caee700 20 client.31815.objecter _get_session s=0x7fd348128970 osd=7 3 2024-02-23T00:55:35.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.728+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd348128970 osd=7 3 2024-02-23T00:55:35.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.728+0000 7fd34caee700 20 client.31815.objecter _get_session s=0x7fd348128970 osd=7 3 2024-02-23T00:55:35.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.728+0000 7fd34caee700 15 client.31815.objecter _session_command_op_remove -1 15 2024-02-23T00:55:35.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.728+0000 7fd34caee700 20 client.31815.objecter get_session s=0x7fd348128970 osd=7 3 2024-02-23T00:55:35.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.728+0000 7fd34caee700 15 client.31815.objecter _session_command_op_assign 7 15 2024-02-23T00:55:35.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.728+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd348128970 osd=7 4 2024-02-23T00:55:35.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.728+0000 7fd34caee700 10 client.31815.objecter _send_command 15 2024-02-23T00:55:35.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.728+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd348129ae0 con 0x7fd348128ff0 2024-02-23T00:55:35.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.728+0000 7fd34659c700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fd348128ff0 0x7fd348129450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:35.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.728+0000 7fd34659c700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fd348128ff0 0x7fd348129450 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:35.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.728+0000 7fd33f7fe700 10 client.31815.objecter ms_handle_connect 0x7fd348128ff0 2024-02-23T00:55:35.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.729+0000 7fd33f7fe700 1 -- 172.21.15.47:0/3787533086 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd348129ae0 con 0x7fd348128ff0 2024-02-23T00:55:35.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.729+0000 7fd33f7fe700 10 client.31815.objecter ms_dispatch 0x7fd348001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:55:35.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.729+0000 7fd33f7fe700 10 client.31815.objecter _finish_command 15 = system:0 2024-02-23T00:55:35.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.729+0000 7fd33f7fe700 20 client.31815.objecter put_session s=0x7fd348128970 osd=7 4 2024-02-23T00:55:35.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.729+0000 7fd33f7fe700 15 client.31815.objecter _session_command_op_remove 7 15 2024-02-23T00:55:35.735 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:55:35.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.744+0000 7fd34caee700 10 client.31815.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:55:35.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.744+0000 7fd34caee700 15 client.31815.objecter _session_command_op_assign -1 16 2024-02-23T00:55:35.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.744+0000 7fd34caee700 20 client.31815.objecter _get_session s=0x7fd348128970 osd=7 3 2024-02-23T00:55:35.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.744+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd348128970 osd=7 3 2024-02-23T00:55:35.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.744+0000 7fd34caee700 20 client.31815.objecter _get_session s=0x7fd348128970 osd=7 3 2024-02-23T00:55:35.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.744+0000 7fd34caee700 15 client.31815.objecter _session_command_op_remove -1 16 2024-02-23T00:55:35.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.744+0000 7fd34caee700 20 client.31815.objecter get_session s=0x7fd348128970 osd=7 3 2024-02-23T00:55:35.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.744+0000 7fd34caee700 15 client.31815.objecter _session_command_op_assign 7 16 2024-02-23T00:55:35.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.744+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd348128970 osd=7 4 2024-02-23T00:55:35.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.744+0000 7fd34caee700 10 client.31815.objecter _send_command 16 2024-02-23T00:55:35.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.744+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd348129ae0 con 0x7fd348128ff0 2024-02-23T00:55:35.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.745+0000 7fd33f7fe700 1 -- 172.21.15.47:0/3787533086 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd348129ae0 con 0x7fd348128ff0 2024-02-23T00:55:35.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.745+0000 7fd33f7fe700 10 client.31815.objecter ms_dispatch 0x7fd348001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:55:35.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.745+0000 7fd33f7fe700 10 client.31815.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:55:35.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.745+0000 7fd33f7fe700 20 client.31815.objecter put_session s=0x7fd348128970 osd=7 4 2024-02-23T00:55:35.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.745+0000 7fd33f7fe700 15 client.31815.objecter _session_command_op_remove 7 16 2024-02-23T00:55:35.749 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:55:35.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.745+0000 7fd34caee700 10 librados: watch_flush enter 2024-02-23T00:55:35.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.745+0000 7fd34caee700 10 librados: watch_flush exit 2024-02-23T00:55:35.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.745+0000 7fd34caee700 10 client.31815.objecter close_session for osd.0 2024-02-23T00:55:35.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.745+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fd348028540 msgr2=0x7fd3480fbd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:35.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.745+0000 7fd34caee700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fd348028540 0x7fd3480fbd30 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:35.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.745+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd34805ee80 osd=0 1 2024-02-23T00:55:35.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.745+0000 7fd34caee700 10 client.31815.objecter close_session for osd.1 2024-02-23T00:55:35.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.745+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fd348027b90 msgr2=0x7fd348027f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:35.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.745+0000 7fd34caee700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fd348027b90 0x7fd348027f70 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:35.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.745+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd3480fd3a0 osd=1 1 2024-02-23T00:55:35.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.745+0000 7fd34caee700 10 client.31815.objecter close_session for osd.2 2024-02-23T00:55:35.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.745+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fd3480fcf00 msgr2=0x7fd34811e510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:35.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.745+0000 7fd34caee700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fd3480fcf00 0x7fd34811e510 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:35.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.745+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd348027a10 osd=2 1 2024-02-23T00:55:35.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.745+0000 7fd34caee700 10 client.31815.objecter close_session for osd.3 2024-02-23T00:55:35.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.745+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fd34811fb80 msgr2=0x7fd348121f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:35.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.745+0000 7fd346d9d700 1 -- 172.21.15.47:0/3787533086 reap_dead start 2024-02-23T00:55:35.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.745+0000 7fd34caee700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fd34811fb80 0x7fd348121f70 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:35.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.745+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd348119750 osd=3 1 2024-02-23T00:55:35.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.745+0000 7fd34caee700 10 client.31815.objecter close_session for osd.4 2024-02-23T00:55:35.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.745+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fd348119330 msgr2=0x7fd3481255f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:35.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.746+0000 7fd34caee700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fd348119330 0x7fd3481255f0 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:35.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.746+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd3481191c0 osd=4 1 2024-02-23T00:55:35.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.746+0000 7fd34caee700 10 client.31815.objecter close_session for osd.5 2024-02-23T00:55:35.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.746+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fd348125ff0 msgr2=0x7fd3481283e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:35.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.746+0000 7fd34caee700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fd348125ff0 0x7fd3481283e0 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:35.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.746+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd34811f880 osd=5 1 2024-02-23T00:55:35.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.746+0000 7fd34caee700 10 client.31815.objecter close_session for osd.6 2024-02-23T00:55:35.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.746+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fd34812a490 msgr2=0x7fd34812a8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:35.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.746+0000 7fd34caee700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fd34812a490 0x7fd34812a8f0 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:35.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.746+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd34812b190 osd=6 1 2024-02-23T00:55:35.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.746+0000 7fd34caee700 10 client.31815.objecter close_session for osd.7 2024-02-23T00:55:35.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.746+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fd348128ff0 msgr2=0x7fd348129450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:35.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.746+0000 7fd34caee700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fd348128ff0 0x7fd348129450 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:35.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.746+0000 7fd34caee700 20 client.31815.objecter put_session s=0x7fd348128970 osd=7 1 2024-02-23T00:55:35.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.746+0000 7fd34caee700 20 client.31815.objecter shutdown clearing up homeless session... 2024-02-23T00:55:35.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.746+0000 7fd34caee700 10 client.31815.objecter successfully canceled tick 2024-02-23T00:55:35.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.746+0000 7fd346d9d700 1 -- 172.21.15.47:0/3787533086 reap_dead start 2024-02-23T00:55:35.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.746+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd334062cc0 msgr2=0x7fd334065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:35.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.746+0000 7fd34caee700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd334062cc0 0x7fd334065180 secure :-1 s=READY pgs=3207 cs=0 l=1 rev1=1 rx=0x7fd338004ee0 tx=0x7fd33800b040).stop 2024-02-23T00:55:35.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.746+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3480f9670 msgr2=0x7fd34810e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:35.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.746+0000 7fd34caee700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3480f9670 0x7fd34810e560 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 rx=0x7fd3300041d0 tx=0x7fd330009f60).stop 2024-02-23T00:55:35.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.746+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 shutdown_connections 2024-02-23T00:55:35.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.746+0000 7fd34caee700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3480f9670 0x7fd34810e560 unknown :-1 s=CLOSED pgs=2641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:35.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.746+0000 7fd34caee700 1 --2- 172.21.15.47:0/3787533086 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd334062cc0 0x7fd334065180 unknown :-1 s=CLOSED pgs=3207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:35.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.746+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 >> 172.21.15.47:0/3787533086 conn(0x7fd3480f2820 msgr2=0x7fd3480f3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:35.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.746+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 shutdown_connections 2024-02-23T00:55:35.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.746+0000 7fd34caee700 1 -- 172.21.15.47:0/3787533086 wait complete. 2024-02-23T00:55:35.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:35.746+0000 7fd34caee700 1 librados: shutdown 2024-02-23T00:55:35.755 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:55:35.755 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:55:35.756 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:55:35.756 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:55:35.756 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:55:35.756 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:55:35.756 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:55:35.756 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:55:35.766 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:35.911 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:55:35.914 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:35.914 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:55:35.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:34.889131+0000 osd.0 (osd.0) 293 : cluster [DBG] 2.5 scrub starts 2024-02-23T00:55:35.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:34.889999+0000 osd.0 (osd.0) 294 : cluster [DBG] 2.5 scrub ok 2024-02-23T00:55:35.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:35.311744+0000 mon.a (mon.0) 3729 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:55:35.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:35.316755+0000 mon.a (mon.0) 3730 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "hit_set_type","val": "bloom"}]': finished 2024-02-23T00:55:35.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:35.316838+0000 mon.a (mon.0) 3731 : cluster [DBG] osdmap e784: 8 total, 8 up, 6 in 2024-02-23T00:55:35.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:35.317774+0000 mon.c (mon.2) 492 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-23T00:55:35.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:35.320567+0000 mon.a (mon.0) 3732 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-23T00:55:35.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:35.402649+0000 mgr.y (mgr.14517) 665 : cluster [DBG] pgmap v1191: 194 pgs: 93 peering, 101 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-23T00:55:35.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:35.604480+0000 mon.a (mon.0) 3733 : audit [DBG] from='client.? 172.21.15.47:0/3787533086' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:55:36.015 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:55:36.043 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:36.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:34.889131+0000 osd.0 (osd.0) 293 : cluster [DBG] 2.5 scrub starts 2024-02-23T00:55:36.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:34.889999+0000 osd.0 (osd.0) 294 : cluster [DBG] 2.5 scrub ok 2024-02-23T00:55:36.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:35.311744+0000 mon.a (mon.0) 3729 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:55:36.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:35.316755+0000 mon.a (mon.0) 3730 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "hit_set_type","val": "bloom"}]': finished 2024-02-23T00:55:36.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:35.316838+0000 mon.a (mon.0) 3731 : cluster [DBG] osdmap e784: 8 total, 8 up, 6 in 2024-02-23T00:55:36.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:35.317774+0000 mon.c (mon.2) 492 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-23T00:55:36.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:35.320567+0000 mon.a (mon.0) 3732 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-23T00:55:36.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:35.402649+0000 mgr.y (mgr.14517) 665 : cluster [DBG] pgmap v1191: 194 pgs: 93 peering, 101 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-23T00:55:36.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:35.604480+0000 mon.a (mon.0) 3733 : audit [DBG] from='client.? 172.21.15.47:0/3787533086' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:55:36.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:34.889131+0000 osd.0 (osd.0) 293 : cluster [DBG] 2.5 scrub starts 2024-02-23T00:55:36.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:34.889999+0000 osd.0 (osd.0) 294 : cluster [DBG] 2.5 scrub ok 2024-02-23T00:55:36.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:35.311744+0000 mon.a (mon.0) 3729 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-23T00:55:36.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:35.316755+0000 mon.a (mon.0) 3730 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "hit_set_type","val": "bloom"}]': finished 2024-02-23T00:55:36.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:35.316838+0000 mon.a (mon.0) 3731 : cluster [DBG] osdmap e784: 8 total, 8 up, 6 in 2024-02-23T00:55:36.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:35.317774+0000 mon.c (mon.2) 492 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-23T00:55:36.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:35.320567+0000 mon.a (mon.0) 3732 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-23T00:55:36.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:35.402649+0000 mgr.y (mgr.14517) 665 : cluster [DBG] pgmap v1191: 194 pgs: 93 peering, 101 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-23T00:55:36.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:35.604480+0000 mon.a (mon.0) 3733 : audit [DBG] from='client.? 172.21.15.47:0/3787533086' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:55:36.081 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:55:36.085 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:36.085 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:55:36.186 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:55:36.213 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:36.446 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.6 dump_ops_in_flight 2024-02-23T00:55:36.462 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:55:36.464 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:36.464 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:55:36.565 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:55:36.597 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:36.617 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:55:36.619 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:36.620 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:55:36.720 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:55:36.731 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:55:36.748 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:36.767 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:55:36.770 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:36.770 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:55:36.871 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:55:36.899 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:36.918 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:55:36.921 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:36.922 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:55:37.023 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:55:37.051 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:37.069 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:55:37.072 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:37.072 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:55:37.089 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-23T00:55:37.174 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:55:37.202 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:37.221 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:55:37.223 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:37.223 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:55:37.325 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:55:37.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:36.324652+0000 mon.a (mon.0) 3734 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-23T00:55:37.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:36.324734+0000 mon.a (mon.0) 3735 : cluster [DBG] osdmap e785: 8 total, 8 up, 6 in 2024-02-23T00:55:37.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:36.327399+0000 mon.c (mon.2) 493 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "target_max_objects","val": "1"}]: dispatch 2024-02-23T00:55:37.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:36.328216+0000 mon.a (mon.0) 3736 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "target_max_objects","val": "1"}]: dispatch 2024-02-23T00:55:37.356 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:37.401 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:55:37.423 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6, 0] out_osds: [2, 7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-23T00:55:37.424 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-23T00:55:37.424 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-02-23T00:55:37.424 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-02-23T00:55:37.424 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-02-23T00:55:37.424 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.4 config set filestore_inject_stall 3 2024-02-23T00:55:37.639 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:36.324652+0000 mon.a (mon.0) 3734 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-23T00:55:37.639 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:36.324734+0000 mon.a (mon.0) 3735 : cluster [DBG] osdmap e785: 8 total, 8 up, 6 in 2024-02-23T00:55:37.639 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:36.327399+0000 mon.c (mon.2) 493 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "target_max_objects","val": "1"}]: dispatch 2024-02-23T00:55:37.639 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:36.328216+0000 mon.a (mon.0) 3736 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "target_max_objects","val": "1"}]: dispatch 2024-02-23T00:55:37.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:36.324652+0000 mon.a (mon.0) 3734 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-23T00:55:37.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:36.324734+0000 mon.a (mon.0) 3735 : cluster [DBG] osdmap e785: 8 total, 8 up, 6 in 2024-02-23T00:55:37.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:36.327399+0000 mon.c (mon.2) 493 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "target_max_objects","val": "1"}]: dispatch 2024-02-23T00:55:37.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:36.328216+0000 mon.a (mon.0) 3736 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "target_max_objects","val": "1"}]: dispatch 2024-02-23T00:55:38.205 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:55:38.208 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:38.208 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:55:38.309 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:55:38.338 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:38.438 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:55:38.440 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:55:38.442 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:38.442 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:55:38.453 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:55:38.453 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-02-23T00:55:38.453 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 0 2024-02-23T00:55:38.453 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:55:38.478 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:37.326583+0000 mon.a (mon.0) 3737 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "target_max_objects","val": "1"}]': finished 2024-02-23T00:55:38.478 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:37.326665+0000 mon.a (mon.0) 3738 : cluster [DBG] osdmap e786: 8 total, 8 up, 6 in 2024-02-23T00:55:38.478 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:37.403164+0000 mgr.y (mgr.14517) 666 : cluster [DBG] pgmap v1194: 194 pgs: 194 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 2.2 KiB/s wr, 6 op/s; 470 B/s, 28 objects/s recovering 2024-02-23T00:55:38.528 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:55:38.543 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:55:38.550 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:37.326583+0000 mon.a (mon.0) 3737 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "target_max_objects","val": "1"}]': finished 2024-02-23T00:55:38.550 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:37.326665+0000 mon.a (mon.0) 3738 : cluster [DBG] osdmap e786: 8 total, 8 up, 6 in 2024-02-23T00:55:38.550 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:37.403164+0000 mgr.y (mgr.14517) 666 : cluster [DBG] pgmap v1194: 194 pgs: 194 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 2.2 KiB/s wr, 6 op/s; 470 B/s, 28 objects/s recovering 2024-02-23T00:55:38.550 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:37.326583+0000 mon.a (mon.0) 3737 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-136","var": "target_max_objects","val": "1"}]': finished 2024-02-23T00:55:38.550 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:37.326665+0000 mon.a (mon.0) 3738 : cluster [DBG] osdmap e786: 8 total, 8 up, 6 in 2024-02-23T00:55:38.550 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:37.403164+0000 mgr.y (mgr.14517) 666 : cluster [DBG] pgmap v1194: 194 pgs: 194 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 2.2 KiB/s wr, 6 op/s; 470 B/s, 28 objects/s recovering 2024-02-23T00:55:38.572 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:38.694 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:55:38.697 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:38.697 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:55:38.798 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:55:38.829 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:38.850 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:55:38.853 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:38.853 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:55:38.954 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:55:38.984 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:39.085 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:55:39.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.084+0000 7f1a2cafd700 1 -- 172.21.15.47:0/2764957147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a280f6a90 msgr2=0x7f1a28104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:39.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.084+0000 7f1a2cafd700 1 --2- 172.21.15.47:0/2764957147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a280f6a90 0x7f1a28104a20 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 rx=0x7f1a20009c30 tx=0x7f1a20004c50).stop 2024-02-23T00:55:39.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.084+0000 7f1a2cafd700 1 -- 172.21.15.47:0/2764957147 shutdown_connections 2024-02-23T00:55:39.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.084+0000 7f1a2cafd700 1 --2- 172.21.15.47:0/2764957147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a280f6a90 0x7f1a28104a20 unknown :-1 s=CLOSED pgs=2642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:39.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.084+0000 7f1a2cafd700 1 --2- 172.21.15.47:0/2764957147 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1a280fdbd0 0x7f1a280fdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:39.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.084+0000 7f1a2cafd700 1 --2- 172.21.15.47:0/2764957147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a280fe530 0x7f1a280f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:39.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.085+0000 7f1a2cafd700 1 -- 172.21.15.47:0/2764957147 >> 172.21.15.47:0/2764957147 conn(0x7f1a2806c370 msgr2=0x7f1a2806c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:39.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.085+0000 7f1a2cafd700 1 -- 172.21.15.47:0/2764957147 shutdown_connections 2024-02-23T00:55:39.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.085+0000 7f1a2cafd700 1 -- 172.21.15.47:0/2764957147 wait complete. 2024-02-23T00:55:39.089 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:39.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.085+0000 7f1a2cafd700 1 librados: starting msgr at 2024-02-23T00:55:39.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.085+0000 7f1a2cafd700 1 librados: starting objecter 2024-02-23T00:55:39.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.086+0000 7f1a2cafd700 1 Processor -- start 2024-02-23T00:55:39.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.086+0000 7f1a2cafd700 1 -- start start 2024-02-23T00:55:39.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.086+0000 7f1a2cafd700 1 librados: setting wanted keys 2024-02-23T00:55:39.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.086+0000 7f1a2cafd700 1 librados: calling monclient init 2024-02-23T00:55:39.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.086+0000 7f1a2cafd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1a280f6a90 0x7f1a280f90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:39.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.086+0000 7f1a2cafd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a280fdbd0 0x7f1a280f7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:39.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.086+0000 7f1a2cafd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a280fe530 0x7f1a280f7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:39.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.086+0000 7f1a2cafd700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a280f8370 con 0x7f1a280fdbd0 2024-02-23T00:55:39.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.086+0000 7f1a2cafd700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a280f84e0 con 0x7f1a280fe530 2024-02-23T00:55:39.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.086+0000 7f1a2cafd700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2811ab70 con 0x7f1a280f6a90 2024-02-23T00:55:39.091 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:55:39.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.086+0000 7f1a25d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a280fdbd0 0x7f1a280f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:39.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.086+0000 7f1a25d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a280fdbd0 0x7f1a280f7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39992/0 (socket says 172.21.15.47:39992) 2024-02-23T00:55:39.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.086+0000 7f1a25d9b700 1 -- 172.21.15.47:0/1819199934 learned_addr learned my addr 172.21.15.47:0/1819199934 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:55:39.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.087+0000 7f1a26d9d700 1 --2- 172.21.15.47:0/1819199934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a280fe530 0x7f1a280f7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:39.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.087+0000 7f1a2659c700 1 --2- 172.21.15.47:0/1819199934 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1a280f6a90 0x7f1a280f90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:39.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.087+0000 7f1a25d9b700 1 -- 172.21.15.47:0/1819199934 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1a280f6a90 msgr2=0x7f1a280f90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:39.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.087+0000 7f1a25d9b700 1 --2- 172.21.15.47:0/1819199934 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1a280f6a90 0x7f1a280f90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:39.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.087+0000 7f1a25d9b700 1 -- 172.21.15.47:0/1819199934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a280fe530 msgr2=0x7f1a280f7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:39.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.087+0000 7f1a25d9b700 1 --2- 172.21.15.47:0/1819199934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a280fe530 0x7f1a280f7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:39.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.087+0000 7f1a25d9b700 1 -- 172.21.15.47:0/1819199934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a18005040 con 0x7f1a280fdbd0 2024-02-23T00:55:39.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.087+0000 7f1a25d9b700 1 --2- 172.21.15.47:0/1819199934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a280fdbd0 0x7f1a280f7680 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 rx=0x7f1a1c00e970 tx=0x7f1a1c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:39.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.087+0000 7f1a177fe700 10 client.?.objecter ms_handle_connect 0x7f1a280fdbd0 2024-02-23T00:55:39.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.087+0000 7f1a177fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:55:39.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.087+0000 7f1a177fe700 1 -- 172.21.15.47:0/1819199934 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c004660 con 0x7f1a280fdbd0 2024-02-23T00:55:39.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.087+0000 7f1a177fe700 1 -- 172.21.15.47:0/1819199934 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f1a1c00f040 con 0x7f1a280fdbd0 2024-02-23T00:55:39.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.087+0000 7f1a2cafd700 1 -- 172.21.15.47:0/1819199934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a200097c0 con 0x7f1a280fdbd0 2024-02-23T00:55:39.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.087+0000 7f1a177fe700 1 -- 172.21.15.47:0/1819199934 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c00c840 con 0x7f1a280fdbd0 2024-02-23T00:55:39.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.087+0000 7f1a2cafd700 10 client.31821.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:55:39.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.087+0000 7f1a2cafd700 1 -- 172.21.15.47:0/1819199934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a2811b200 con 0x7f1a280fdbd0 2024-02-23T00:55:39.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.087+0000 7f1a2cafd700 1 librados: init done 2024-02-23T00:55:39.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.088+0000 7f1a2cafd700 1 -- 172.21.15.47:0/1819199934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a2805f950 con 0x7f1a280fdbd0 2024-02-23T00:55:39.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.089+0000 7f1a177fe700 1 -- 172.21.15.47:0/1819199934 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f1a1c00cc70 con 0x7f1a280fdbd0 2024-02-23T00:55:39.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.089+0000 7f1a177fe700 1 --2- 172.21.15.47:0/1819199934 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1a10062cc0 0x7f1a10065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:39.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.089+0000 7f1a177fe700 1 -- 172.21.15.47:0/1819199934 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(786..786 src has 621..786) v4 ==== 9745+0+0 (secure 0 0 0) 0x7f1a1c07d470 con 0x7f1a280fdbd0 2024-02-23T00:55:39.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.089+0000 7f1a177fe700 10 client.31821.objecter ms_dispatch 0x7f1a28001550 osd_map(786..786 src has 621..786) v4 2024-02-23T00:55:39.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.089+0000 7f1a177fe700 3 client.31821.objecter handle_osd_map got epochs [786,786] > 0 2024-02-23T00:55:39.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.089+0000 7f1a177fe700 3 client.31821.objecter handle_osd_map decoding full epoch 786 2024-02-23T00:55:39.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.089+0000 7f1a2659c700 1 --2- 172.21.15.47:0/1819199934 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1a10062cc0 0x7f1a10065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:39.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.090+0000 7f1a177fe700 20 client.31821.objecter dump_active .. 0 homeless 2024-02-23T00:55:39.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.090+0000 7f1a2659c700 1 --2- 172.21.15.47:0/1819199934 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1a10062cc0 0x7f1a10065180 secure :-1 s=READY pgs=3208 cs=0 l=1 rev1=1 rx=0x7f1a18009f20 tx=0x7f1a18009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:39.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.090+0000 7f1a177fe700 10 client.31821.objecter ms_handle_connect 0x7f1a10062cc0 2024-02-23T00:55:39.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.094+0000 7f1a177fe700 1 -- 172.21.15.47:0/1819199934 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f1a1c04af60 con 0x7f1a280fdbd0 2024-02-23T00:55:39.128 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:55:39.191 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:55:39.221 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:39.294 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:55:39.296 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:39.297 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:55:39.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.295+0000 7f1a2cafd700 1 -- 172.21.15.47:0/1819199934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7f1a28106e90 con 0x7f1a280fdbd0 2024-02-23T00:55:39.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.342+0000 7f1a177fe700 1 -- 172.21.15.47:0/1819199934 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v787) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1a1c04caf0 con 0x7f1a280fdbd0 2024-02-23T00:55:39.348 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:38.332917+0000 mon.a (mon.0) 3739 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-23T00:55:39.349 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:38.332951+0000 mon.a (mon.0) 3740 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 14 pgs inactive, 19 pgs peering) 2024-02-23T00:55:39.349 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:39.296570+0000 mon.a (mon.0) 3741 : audit [INF] from='client.? 172.21.15.47:0/1819199934' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:55:39.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.345+0000 7f1a2cafd700 10 librados: watch_flush enter 2024-02-23T00:55:39.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.345+0000 7f1a2cafd700 10 librados: watch_flush exit 2024-02-23T00:55:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.345+0000 7f1a2cafd700 20 client.31821.objecter shutdown clearing up homeless session... 2024-02-23T00:55:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.345+0000 7f1a2cafd700 10 client.31821.objecter successfully canceled tick 2024-02-23T00:55:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.345+0000 7f1a2cafd700 1 -- 172.21.15.47:0/1819199934 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1a10062cc0 msgr2=0x7f1a10065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.345+0000 7f1a2cafd700 1 --2- 172.21.15.47:0/1819199934 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1a10062cc0 0x7f1a10065180 secure :-1 s=READY pgs=3208 cs=0 l=1 rev1=1 rx=0x7f1a18009f20 tx=0x7f1a18009cb0).stop 2024-02-23T00:55:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.345+0000 7f1a2cafd700 1 -- 172.21.15.47:0/1819199934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a280fdbd0 msgr2=0x7f1a280f7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.345+0000 7f1a2cafd700 1 --2- 172.21.15.47:0/1819199934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a280fdbd0 0x7f1a280f7680 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 rx=0x7f1a1c00e970 tx=0x7f1a1c00b040).stop 2024-02-23T00:55:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.346+0000 7f1a2cafd700 1 -- 172.21.15.47:0/1819199934 shutdown_connections 2024-02-23T00:55:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.346+0000 7f1a2cafd700 1 --2- 172.21.15.47:0/1819199934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a280fe530 0x7f1a280f7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.346+0000 7f1a2cafd700 1 --2- 172.21.15.47:0/1819199934 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1a10062cc0 0x7f1a10065180 unknown :-1 s=CLOSED pgs=3208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.346+0000 7f1a2cafd700 1 --2- 172.21.15.47:0/1819199934 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1a280f6a90 0x7f1a280f90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.346+0000 7f1a2cafd700 1 --2- 172.21.15.47:0/1819199934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a280fdbd0 0x7f1a280f7680 unknown :-1 s=CLOSED pgs=2643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.346+0000 7f1a2cafd700 1 -- 172.21.15.47:0/1819199934 >> 172.21.15.47:0/1819199934 conn(0x7f1a2806c370 msgr2=0x7f1a280f5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.346+0000 7f1a2cafd700 1 -- 172.21.15.47:0/1819199934 shutdown_connections 2024-02-23T00:55:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.346+0000 7f1a2cafd700 1 -- 172.21.15.47:0/1819199934 wait complete. 2024-02-23T00:55:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:39.346+0000 7f1a2cafd700 1 librados: shutdown 2024-02-23T00:55:39.351 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is set 2024-02-23T00:55:39.397 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:55:39.424 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:39.688 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:38.332917+0000 mon.a (mon.0) 3739 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-23T00:55:39.688 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:38.332951+0000 mon.a (mon.0) 3740 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 14 pgs inactive, 19 pgs peering) 2024-02-23T00:55:39.688 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:39.296570+0000 mon.a (mon.0) 3741 : audit [INF] from='client.? 172.21.15.47:0/1819199934' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:55:39.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:38.332917+0000 mon.a (mon.0) 3739 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-23T00:55:39.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:38.332951+0000 mon.a (mon.0) 3740 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 14 pgs inactive, 19 pgs peering) 2024-02-23T00:55:39.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:39.296570+0000 mon.a (mon.0) 3741 : audit [INF] from='client.? 172.21.15.47:0/1819199934' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:55:40.145 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.6 dump_blocked_ops 2024-02-23T00:55:40.173 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:55:40.175 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:40.176 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:55:40.277 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:55:40.302 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:40.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:39.341913+0000 mon.a (mon.0) 3742 : audit [INF] from='client.? 172.21.15.47:0/1819199934' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:55:40.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:39.341959+0000 mon.a (mon.0) 3743 : cluster [DBG] osdmap e787: 8 total, 8 up, 6 in 2024-02-23T00:55:40.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:39.403557+0000 mgr.y (mgr.14517) 667 : cluster [DBG] pgmap v1196: 194 pgs: 194 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 2.2 KiB/s wr, 6 op/s; 460 B/s, 27 objects/s recovering 2024-02-23T00:55:40.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:40.313020+0000 mon.a (mon.0) 3744 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:55:40.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:40.318701+0000 mon.a (mon.0) 3745 : cluster [DBG] osdmap e788: 8 total, 8 up, 6 in 2024-02-23T00:55:40.557 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:55:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:55:40] "GET /metrics HTTP/1.1" 200 191882 "" "Prometheus/2.33.4" 2024-02-23T00:55:40.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:39.341913+0000 mon.a (mon.0) 3742 : audit [INF] from='client.? 172.21.15.47:0/1819199934' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:55:40.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:39.341959+0000 mon.a (mon.0) 3743 : cluster [DBG] osdmap e787: 8 total, 8 up, 6 in 2024-02-23T00:55:40.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:39.403557+0000 mgr.y (mgr.14517) 667 : cluster [DBG] pgmap v1196: 194 pgs: 194 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 2.2 KiB/s wr, 6 op/s; 460 B/s, 27 objects/s recovering 2024-02-23T00:55:40.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:40.313020+0000 mon.a (mon.0) 3744 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:55:40.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:40.318701+0000 mon.a (mon.0) 3745 : cluster [DBG] osdmap e788: 8 total, 8 up, 6 in 2024-02-23T00:55:40.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:39.341913+0000 mon.a (mon.0) 3742 : audit [INF] from='client.? 172.21.15.47:0/1819199934' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:55:40.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:39.341959+0000 mon.a (mon.0) 3743 : cluster [DBG] osdmap e787: 8 total, 8 up, 6 in 2024-02-23T00:55:40.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:39.403557+0000 mgr.y (mgr.14517) 667 : cluster [DBG] pgmap v1196: 194 pgs: 194 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 2.2 KiB/s wr, 6 op/s; 460 B/s, 27 objects/s recovering 2024-02-23T00:55:40.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:40.313020+0000 mon.a (mon.0) 3744 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:55:40.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:40.318701+0000 mon.a (mon.0) 3745 : cluster [DBG] osdmap e788: 8 total, 8 up, 6 in 2024-02-23T00:55:41.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.215+0000 7f2d7209b700 1 -- 172.21.15.47:0/1250908232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d6c0f96e0 msgr2=0x7f2d6c101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:41.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.215+0000 7f2d7209b700 1 --2- 172.21.15.47:0/1250908232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d6c0f96e0 0x7f2d6c101c30 secure :-1 s=READY pgs=2644 cs=0 l=1 rev1=1 rx=0x7f2d60004870 tx=0x7f2d60009cb0).stop 2024-02-23T00:55:41.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.215+0000 7f2d7209b700 1 -- 172.21.15.47:0/1250908232 shutdown_connections 2024-02-23T00:55:41.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.215+0000 7f2d7209b700 1 --2- 172.21.15.47:0/1250908232 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2d6c102250 0x7f2d6c104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:41.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.215+0000 7f2d7209b700 1 --2- 172.21.15.47:0/1250908232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d6c0f8d80 0x7f2d6c0f9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:41.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.215+0000 7f2d7209b700 1 --2- 172.21.15.47:0/1250908232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d6c0f96e0 0x7f2d6c101c30 unknown :-1 s=CLOSED pgs=2644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:41.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.215+0000 7f2d7209b700 1 -- 172.21.15.47:0/1250908232 >> 172.21.15.47:0/1250908232 conn(0x7f2d6c0f26f0 msgr2=0x7f2d6c0f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:41.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.215+0000 7f2d7209b700 1 -- 172.21.15.47:0/1250908232 shutdown_connections 2024-02-23T00:55:41.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.215+0000 7f2d7209b700 1 -- 172.21.15.47:0/1250908232 wait complete. 2024-02-23T00:55:41.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.215+0000 7f2d7209b700 1 librados: starting msgr at 2024-02-23T00:55:41.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.215+0000 7f2d7209b700 1 librados: starting objecter 2024-02-23T00:55:41.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.215+0000 7f2d7209b700 1 Processor -- start 2024-02-23T00:55:41.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.215+0000 7f2d7209b700 1 -- start start 2024-02-23T00:55:41.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.216+0000 7f2d7209b700 1 librados: setting wanted keys 2024-02-23T00:55:41.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.216+0000 7f2d7209b700 1 librados: calling monclient init 2024-02-23T00:55:41.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.216+0000 7f2d7209b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d6c0f8d80 0x7f2d6c10e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:41.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.216+0000 7f2d7209b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d6c0f96e0 0x7f2d6c10e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:41.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.216+0000 7f2d7209b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2d6c102250 0x7f2d6c1082b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:41.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.216+0000 7f2d7209b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c1087a0 con 0x7f2d6c0f96e0 2024-02-23T00:55:41.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.216+0000 7f2d7209b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c108910 con 0x7f2d6c0f8d80 2024-02-23T00:55:41.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.216+0000 7f2d7209b700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c108ad0 con 0x7f2d6c102250 2024-02-23T00:55:41.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.216+0000 7f2d6affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d6c0f96e0 0x7f2d6c10e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:41.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.216+0000 7f2d6bfff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2d6c102250 0x7f2d6c1082b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:41.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.216+0000 7f2d6affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d6c0f96e0 0x7f2d6c10e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40002/0 (socket says 172.21.15.47:40002) 2024-02-23T00:55:41.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.216+0000 7f2d6affd700 1 -- 172.21.15.47:0/784091020 learned_addr learned my addr 172.21.15.47:0/784091020 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:55:41.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.216+0000 7f2d6b7fe700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d6c0f8d80 0x7f2d6c10e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:41.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.216+0000 7f2d6bfff700 1 -- 172.21.15.47:0/784091020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d6c0f8d80 msgr2=0x7f2d6c10e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:41.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.216+0000 7f2d6bfff700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d6c0f8d80 0x7f2d6c10e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:41.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.216+0000 7f2d6bfff700 1 -- 172.21.15.47:0/784091020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d6c0f96e0 msgr2=0x7f2d6c10e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:41.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.216+0000 7f2d6bfff700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d6c0f96e0 0x7f2d6c10e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:41.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.216+0000 7f2d6bfff700 1 -- 172.21.15.47:0/784091020 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d640097c0 con 0x7f2d6c102250 2024-02-23T00:55:41.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.216+0000 7f2d6b7fe700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d6c0f8d80 0x7f2d6c10e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:55:41.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.216+0000 7f2d6bfff700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2d6c102250 0x7f2d6c1082b0 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 rx=0x7f2d64001cb0 tx=0x7f2d64004c50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:41.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.216+0000 7f2d68ff9700 10 client.?.objecter ms_handle_connect 0x7f2d6c102250 2024-02-23T00:55:41.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.216+0000 7f2d68ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:55:41.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.216+0000 7f2d68ff9700 1 -- 172.21.15.47:0/784091020 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6400fa30 con 0x7f2d6c102250 2024-02-23T00:55:41.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.217+0000 7f2d68ff9700 1 -- 172.21.15.47:0/784091020 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f2d640090d0 con 0x7f2d6c102250 2024-02-23T00:55:41.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.217+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d60005040 con 0x7f2d6c102250 2024-02-23T00:55:41.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.217+0000 7f2d68ff9700 1 -- 172.21.15.47:0/784091020 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d64007b40 con 0x7f2d6c102250 2024-02-23T00:55:41.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.217+0000 7f2d7209b700 10 client.42152.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:55:41.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.217+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d6c1090c0 con 0x7f2d6c102250 2024-02-23T00:55:41.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.217+0000 7f2d7209b700 1 librados: init done 2024-02-23T00:55:41.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.217+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f2d6c05ee80 con 0x7f2d6c102250 2024-02-23T00:55:41.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.218+0000 7f2d68ff9700 1 -- 172.21.15.47:0/784091020 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f2d64009240 con 0x7f2d6c102250 2024-02-23T00:55:41.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.218+0000 7f2d68ff9700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2d54062cc0 0x7f2d54065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:41.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.218+0000 7f2d68ff9700 1 -- 172.21.15.47:0/784091020 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(788..788 src has 621..788) v4 ==== 9745+0+0 (secure 0 0 0) 0x7f2d6407e1b0 con 0x7f2d6c102250 2024-02-23T00:55:41.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.218+0000 7f2d68ff9700 10 client.42152.objecter ms_dispatch 0x7f2d6c001550 osd_map(788..788 src has 621..788) v4 2024-02-23T00:55:41.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.218+0000 7f2d68ff9700 3 client.42152.objecter handle_osd_map got epochs [788,788] > 0 2024-02-23T00:55:41.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.218+0000 7f2d68ff9700 3 client.42152.objecter handle_osd_map decoding full epoch 788 2024-02-23T00:55:41.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.218+0000 7f2d6b7fe700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2d54062cc0 0x7f2d54065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:41.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.218+0000 7f2d68ff9700 20 client.42152.objecter dump_active .. 0 homeless 2024-02-23T00:55:41.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.218+0000 7f2d68ff9700 1 -- 172.21.15.47:0/784091020 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v788) v1 ==== 54+0+16 (secure 0 0 0) 0x7f2d64085070 con 0x7f2d6c102250 2024-02-23T00:55:41.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.219+0000 7f2d7209b700 10 client.42152.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:41.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.219+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_assign -1 1 2024-02-23T00:55:41.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.219+0000 7f2d7209b700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f2d6c028540 0x7f2d6c0fbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:41.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.219+0000 7f2d7209b700 20 client.42152.objecter _get_session s=0x7f2d6c107010 osd=0 3 2024-02-23T00:55:41.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.219+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c107010 osd=0 3 2024-02-23T00:55:41.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.219+0000 7f2d7209b700 20 client.42152.objecter _get_session s=0x7f2d6c107010 osd=0 3 2024-02-23T00:55:41.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.219+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_remove -1 1 2024-02-23T00:55:41.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.219+0000 7f2d7209b700 20 client.42152.objecter get_session s=0x7f2d6c107010 osd=0 3 2024-02-23T00:55:41.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.219+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_assign 0 1 2024-02-23T00:55:41.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.219+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c107010 osd=0 4 2024-02-23T00:55:41.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.219+0000 7f2d68ff9700 10 client.42152.objecter ms_handle_connect 0x7f2d54062cc0 2024-02-23T00:55:41.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.219+0000 7f2d7209b700 10 client.42152.objecter _send_command 1 2024-02-23T00:55:41.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.219+0000 7f2d6b7fe700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2d54062cc0 0x7f2d54065180 secure :-1 s=READY pgs=3209 cs=0 l=1 rev1=1 rx=0x7f2d5c007fd0 tx=0x7f2d5c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:41.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.219+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2d6c0fd250 con 0x7f2d6c028540 2024-02-23T00:55:41.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.219+0000 7f2d6affd700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f2d6c028540 0x7f2d6c0fbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:41.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.219+0000 7f2d68ff9700 10 client.42152.objecter ms_handle_connect 0x7f2d6c028540 2024-02-23T00:55:41.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.219+0000 7f2d6affd700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f2d6c028540 0x7f2d6c0fbbe0 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:41.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.220+0000 7f2d68ff9700 1 -- 172.21.15.47:0/784091020 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2d6c0fd250 con 0x7f2d6c028540 2024-02-23T00:55:41.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.220+0000 7f2d68ff9700 10 client.42152.objecter ms_dispatch 0x7f2d6c001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:55:41.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.220+0000 7f2d68ff9700 10 client.42152.objecter _finish_command 1 = system:0 2024-02-23T00:55:41.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.220+0000 7f2d68ff9700 20 client.42152.objecter put_session s=0x7f2d6c107010 osd=0 4 2024-02-23T00:55:41.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.220+0000 7f2d68ff9700 15 client.42152.objecter _session_command_op_remove 0 1 2024-02-23T00:55:41.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.235+0000 7f2d7209b700 10 client.42152.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:55:41.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.235+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_assign -1 2 2024-02-23T00:55:41.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.235+0000 7f2d7209b700 20 client.42152.objecter _get_session s=0x7f2d6c107010 osd=0 3 2024-02-23T00:55:41.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.235+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c107010 osd=0 3 2024-02-23T00:55:41.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.235+0000 7f2d7209b700 20 client.42152.objecter _get_session s=0x7f2d6c107010 osd=0 3 2024-02-23T00:55:41.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.235+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_remove -1 2 2024-02-23T00:55:41.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.235+0000 7f2d7209b700 20 client.42152.objecter get_session s=0x7f2d6c107010 osd=0 3 2024-02-23T00:55:41.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.235+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_assign 0 2 2024-02-23T00:55:41.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.235+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c107010 osd=0 4 2024-02-23T00:55:41.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.235+0000 7f2d7209b700 10 client.42152.objecter _send_command 2 2024-02-23T00:55:41.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.235+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f2d6c0fd3c0 con 0x7f2d6c028540 2024-02-23T00:55:41.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.236+0000 7f2d68ff9700 1 -- 172.21.15.47:0/784091020 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f2d6c0fd3c0 con 0x7f2d6c028540 2024-02-23T00:55:41.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.236+0000 7f2d68ff9700 10 client.42152.objecter ms_dispatch 0x7f2d6c001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:55:41.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.236+0000 7f2d68ff9700 10 client.42152.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:55:41.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.236+0000 7f2d68ff9700 20 client.42152.objecter put_session s=0x7f2d6c107010 osd=0 4 2024-02-23T00:55:41.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.236+0000 7f2d68ff9700 15 client.42152.objecter _session_command_op_remove 0 2 2024-02-23T00:55:41.240 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:55:41.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.236+0000 7f2d7209b700 10 client.42152.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:41.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.236+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_assign -1 3 2024-02-23T00:55:41.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.236+0000 7f2d7209b700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f2d6c0fc750 0x7f2d6c0fcb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:41.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.237+0000 7f2d7209b700 20 client.42152.objecter _get_session s=0x7f2d6c0fc0d0 osd=1 3 2024-02-23T00:55:41.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.237+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c0fc0d0 osd=1 3 2024-02-23T00:55:41.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.237+0000 7f2d7209b700 20 client.42152.objecter _get_session s=0x7f2d6c0fc0d0 osd=1 3 2024-02-23T00:55:41.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.237+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_remove -1 3 2024-02-23T00:55:41.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.237+0000 7f2d7209b700 20 client.42152.objecter get_session s=0x7f2d6c0fc0d0 osd=1 3 2024-02-23T00:55:41.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.237+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_assign 1 3 2024-02-23T00:55:41.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.237+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c0fc0d0 osd=1 4 2024-02-23T00:55:41.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.237+0000 7f2d7209b700 10 client.42152.objecter _send_command 3 2024-02-23T00:55:41.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.237+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2d6c0fd020 con 0x7f2d6c0fc750 2024-02-23T00:55:41.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.237+0000 7f2d6b7fe700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f2d6c0fc750 0x7f2d6c0fcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:41.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.237+0000 7f2d68ff9700 10 client.42152.objecter ms_handle_connect 0x7f2d6c0fc750 2024-02-23T00:55:41.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.237+0000 7f2d6b7fe700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f2d6c0fc750 0x7f2d6c0fcb30 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:41.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.237+0000 7f2d68ff9700 1 -- 172.21.15.47:0/784091020 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2d6c0fd020 con 0x7f2d6c0fc750 2024-02-23T00:55:41.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.237+0000 7f2d68ff9700 10 client.42152.objecter ms_dispatch 0x7f2d6c001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:55:41.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.237+0000 7f2d68ff9700 10 client.42152.objecter _finish_command 3 = system:0 2024-02-23T00:55:41.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.237+0000 7f2d68ff9700 20 client.42152.objecter put_session s=0x7f2d6c0fc0d0 osd=1 4 2024-02-23T00:55:41.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.237+0000 7f2d68ff9700 15 client.42152.objecter _session_command_op_remove 1 3 2024-02-23T00:55:41.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.252+0000 7f2d7209b700 10 client.42152.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:55:41.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.252+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_assign -1 4 2024-02-23T00:55:41.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.252+0000 7f2d7209b700 20 client.42152.objecter _get_session s=0x7f2d6c0fc0d0 osd=1 3 2024-02-23T00:55:41.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.252+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c0fc0d0 osd=1 3 2024-02-23T00:55:41.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.252+0000 7f2d7209b700 20 client.42152.objecter _get_session s=0x7f2d6c0fc0d0 osd=1 3 2024-02-23T00:55:41.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.252+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_remove -1 4 2024-02-23T00:55:41.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.252+0000 7f2d7209b700 20 client.42152.objecter get_session s=0x7f2d6c0fc0d0 osd=1 3 2024-02-23T00:55:41.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.252+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_assign 1 4 2024-02-23T00:55:41.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.252+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c0fc0d0 osd=1 4 2024-02-23T00:55:41.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.252+0000 7f2d7209b700 10 client.42152.objecter _send_command 4 2024-02-23T00:55:41.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.252+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f2d6c059360 con 0x7f2d6c0fc750 2024-02-23T00:55:41.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.254+0000 7f2d68ff9700 1 -- 172.21.15.47:0/784091020 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f2d6c059360 con 0x7f2d6c0fc750 2024-02-23T00:55:41.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.254+0000 7f2d68ff9700 10 client.42152.objecter ms_dispatch 0x7f2d6c001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:55:41.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.254+0000 7f2d68ff9700 10 client.42152.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:55:41.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.254+0000 7f2d68ff9700 20 client.42152.objecter put_session s=0x7f2d6c0fc0d0 osd=1 4 2024-02-23T00:55:41.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.254+0000 7f2d68ff9700 15 client.42152.objecter _session_command_op_remove 1 4 2024-02-23T00:55:41.258 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:55:41.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.254+0000 7f2d7209b700 10 client.42152.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:41.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.254+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_assign -1 5 2024-02-23T00:55:41.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.254+0000 7f2d7209b700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f2d6c0280a0 0x7f2d6c11c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:41.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.254+0000 7f2d7209b700 20 client.42152.objecter _get_session s=0x7f2d6c059770 osd=2 3 2024-02-23T00:55:41.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.254+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c059770 osd=2 3 2024-02-23T00:55:41.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.254+0000 7f2d7209b700 20 client.42152.objecter _get_session s=0x7f2d6c059770 osd=2 3 2024-02-23T00:55:41.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.254+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_remove -1 5 2024-02-23T00:55:41.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.254+0000 7f2d7209b700 20 client.42152.objecter get_session s=0x7f2d6c059770 osd=2 3 2024-02-23T00:55:41.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.254+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_assign 2 5 2024-02-23T00:55:41.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.254+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c059770 osd=2 4 2024-02-23T00:55:41.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.254+0000 7f2d7209b700 10 client.42152.objecter _send_command 5 2024-02-23T00:55:41.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.254+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2d6c11ee40 con 0x7f2d6c0280a0 2024-02-23T00:55:41.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.254+0000 7f2d6affd700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f2d6c0280a0 0x7f2d6c11c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:41.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.254+0000 7f2d68ff9700 10 client.42152.objecter ms_handle_connect 0x7f2d6c0280a0 2024-02-23T00:55:41.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.254+0000 7f2d6affd700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f2d6c0280a0 0x7f2d6c11c160 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:41.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.255+0000 7f2d68ff9700 1 -- 172.21.15.47:0/784091020 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2d6c11ee40 con 0x7f2d6c0280a0 2024-02-23T00:55:41.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.255+0000 7f2d68ff9700 10 client.42152.objecter ms_dispatch 0x7f2d6c001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:55:41.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.255+0000 7f2d68ff9700 10 client.42152.objecter _finish_command 5 = system:0 2024-02-23T00:55:41.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.255+0000 7f2d68ff9700 20 client.42152.objecter put_session s=0x7f2d6c059770 osd=2 4 2024-02-23T00:55:41.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.255+0000 7f2d68ff9700 15 client.42152.objecter _session_command_op_remove 2 5 2024-02-23T00:55:41.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.269+0000 7f2d7209b700 10 client.42152.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:55:41.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.269+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_assign -1 6 2024-02-23T00:55:41.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.269+0000 7f2d7209b700 20 client.42152.objecter _get_session s=0x7f2d6c059770 osd=2 3 2024-02-23T00:55:41.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.269+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c059770 osd=2 3 2024-02-23T00:55:41.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.269+0000 7f2d7209b700 20 client.42152.objecter _get_session s=0x7f2d6c059770 osd=2 3 2024-02-23T00:55:41.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.269+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_remove -1 6 2024-02-23T00:55:41.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.269+0000 7f2d7209b700 20 client.42152.objecter get_session s=0x7f2d6c059770 osd=2 3 2024-02-23T00:55:41.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.269+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_assign 2 6 2024-02-23T00:55:41.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.269+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c059770 osd=2 4 2024-02-23T00:55:41.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.269+0000 7f2d7209b700 10 client.42152.objecter _send_command 6 2024-02-23T00:55:41.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.269+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f2d6c027600 con 0x7f2d6c0280a0 2024-02-23T00:55:41.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.271+0000 7f2d68ff9700 1 -- 172.21.15.47:0/784091020 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f2d6c027600 con 0x7f2d6c0280a0 2024-02-23T00:55:41.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.271+0000 7f2d68ff9700 10 client.42152.objecter ms_dispatch 0x7f2d6c001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:55:41.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.271+0000 7f2d68ff9700 10 client.42152.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:55:41.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.271+0000 7f2d68ff9700 20 client.42152.objecter put_session s=0x7f2d6c059770 osd=2 4 2024-02-23T00:55:41.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.271+0000 7f2d68ff9700 15 client.42152.objecter _session_command_op_remove 2 6 2024-02-23T00:55:41.274 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:55:41.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.271+0000 7f2d7209b700 10 client.42152.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:41.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.271+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_assign -1 7 2024-02-23T00:55:41.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.271+0000 7f2d7209b700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f2d6c119cb0 0x7f2d6c11d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:41.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.271+0000 7f2d7209b700 20 client.42152.objecter _get_session s=0x7f2d6c027a00 osd=3 3 2024-02-23T00:55:41.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.271+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c027a00 osd=3 3 2024-02-23T00:55:41.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.271+0000 7f2d7209b700 20 client.42152.objecter _get_session s=0x7f2d6c027a00 osd=3 3 2024-02-23T00:55:41.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.271+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_remove -1 7 2024-02-23T00:55:41.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.271+0000 7f2d7209b700 20 client.42152.objecter get_session s=0x7f2d6c027a00 osd=3 3 2024-02-23T00:55:41.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.271+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_assign 3 7 2024-02-23T00:55:41.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.271+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c027a00 osd=3 4 2024-02-23T00:55:41.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.271+0000 7f2d7209b700 10 client.42152.objecter _send_command 7 2024-02-23T00:55:41.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.271+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2d6c11de60 con 0x7f2d6c119cb0 2024-02-23T00:55:41.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.271+0000 7f2d6bfff700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f2d6c119cb0 0x7f2d6c11d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:41.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.271+0000 7f2d68ff9700 10 client.42152.objecter ms_handle_connect 0x7f2d6c119cb0 2024-02-23T00:55:41.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.271+0000 7f2d6bfff700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f2d6c119cb0 0x7f2d6c11d7d0 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:41.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.272+0000 7f2d68ff9700 1 -- 172.21.15.47:0/784091020 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2d6c11de60 con 0x7f2d6c119cb0 2024-02-23T00:55:41.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.272+0000 7f2d68ff9700 10 client.42152.objecter ms_dispatch 0x7f2d6c001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:55:41.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.272+0000 7f2d68ff9700 10 client.42152.objecter _finish_command 7 = system:0 2024-02-23T00:55:41.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.272+0000 7f2d68ff9700 20 client.42152.objecter put_session s=0x7f2d6c027a00 osd=3 4 2024-02-23T00:55:41.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.272+0000 7f2d68ff9700 15 client.42152.objecter _session_command_op_remove 3 7 2024-02-23T00:55:41.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.287+0000 7f2d7209b700 10 client.42152.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:55:41.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.287+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_assign -1 8 2024-02-23T00:55:41.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.287+0000 7f2d7209b700 20 client.42152.objecter _get_session s=0x7f2d6c027a00 osd=3 3 2024-02-23T00:55:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.287+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c027a00 osd=3 3 2024-02-23T00:55:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.287+0000 7f2d7209b700 20 client.42152.objecter _get_session s=0x7f2d6c027a00 osd=3 3 2024-02-23T00:55:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.287+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_remove -1 8 2024-02-23T00:55:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.287+0000 7f2d7209b700 20 client.42152.objecter get_session s=0x7f2d6c027a00 osd=3 3 2024-02-23T00:55:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.287+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_assign 3 8 2024-02-23T00:55:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.287+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c027a00 osd=3 4 2024-02-23T00:55:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.287+0000 7f2d7209b700 10 client.42152.objecter _send_command 8 2024-02-23T00:55:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.287+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f2d6c119210 con 0x7f2d6c119cb0 2024-02-23T00:55:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.288+0000 7f2d68ff9700 1 -- 172.21.15.47:0/784091020 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f2d6c119210 con 0x7f2d6c119cb0 2024-02-23T00:55:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.288+0000 7f2d68ff9700 10 client.42152.objecter ms_dispatch 0x7f2d6c001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:55:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.288+0000 7f2d68ff9700 10 client.42152.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:55:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.288+0000 7f2d68ff9700 20 client.42152.objecter put_session s=0x7f2d6c027a00 osd=3 4 2024-02-23T00:55:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.288+0000 7f2d68ff9700 15 client.42152.objecter _session_command_op_remove 3 8 2024-02-23T00:55:41.291 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:55:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.288+0000 7f2d7209b700 10 client.42152.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.288+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_assign -1 9 2024-02-23T00:55:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.288+0000 7f2d7209b700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2d6c119380 0x7f2d6c123000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.288+0000 7f2d7209b700 20 client.42152.objecter _get_session s=0x7f2d6c119210 osd=4 3 2024-02-23T00:55:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.288+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c119210 osd=4 3 2024-02-23T00:55:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.288+0000 7f2d7209b700 20 client.42152.objecter _get_session s=0x7f2d6c119210 osd=4 3 2024-02-23T00:55:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.288+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_remove -1 9 2024-02-23T00:55:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.288+0000 7f2d7209b700 20 client.42152.objecter get_session s=0x7f2d6c119210 osd=4 3 2024-02-23T00:55:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.288+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_assign 4 9 2024-02-23T00:55:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.288+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c119210 osd=4 4 2024-02-23T00:55:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.288+0000 7f2d7209b700 10 client.42152.objecter _send_command 9 2024-02-23T00:55:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.288+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2d6c125df0 con 0x7f2d6c119380 2024-02-23T00:55:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.289+0000 7f2d6b7fe700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2d6c119380 0x7f2d6c123000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.289+0000 7f2d68ff9700 10 client.42152.objecter ms_handle_connect 0x7f2d6c119380 2024-02-23T00:55:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.289+0000 7f2d6b7fe700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2d6c119380 0x7f2d6c123000 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.290+0000 7f2d68ff9700 1 -- 172.21.15.47:0/784091020 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2d6c125df0 con 0x7f2d6c119380 2024-02-23T00:55:41.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.290+0000 7f2d68ff9700 10 client.42152.objecter ms_dispatch 0x7f2d6c001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:55:41.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.290+0000 7f2d68ff9700 10 client.42152.objecter _finish_command 9 = system:0 2024-02-23T00:55:41.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.290+0000 7f2d68ff9700 20 client.42152.objecter put_session s=0x7f2d6c119210 osd=4 4 2024-02-23T00:55:41.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.290+0000 7f2d68ff9700 15 client.42152.objecter _session_command_op_remove 4 9 2024-02-23T00:55:41.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.305+0000 7f2d7209b700 10 client.42152.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:55:41.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.305+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_assign -1 10 2024-02-23T00:55:41.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.305+0000 7f2d7209b700 20 client.42152.objecter _get_session s=0x7f2d6c119210 osd=4 3 2024-02-23T00:55:41.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.305+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c119210 osd=4 3 2024-02-23T00:55:41.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.305+0000 7f2d7209b700 20 client.42152.objecter _get_session s=0x7f2d6c119210 osd=4 3 2024-02-23T00:55:41.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.305+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_remove -1 10 2024-02-23T00:55:41.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.305+0000 7f2d7209b700 20 client.42152.objecter get_session s=0x7f2d6c119210 osd=4 3 2024-02-23T00:55:41.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.305+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_assign 4 10 2024-02-23T00:55:41.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.305+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c119210 osd=4 4 2024-02-23T00:55:41.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.305+0000 7f2d7209b700 10 client.42152.objecter _send_command 10 2024-02-23T00:55:41.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.305+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f2d6c125df0 con 0x7f2d6c119380 2024-02-23T00:55:41.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.306+0000 7f2d68ff9700 1 -- 172.21.15.47:0/784091020 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f2d6c125df0 con 0x7f2d6c119380 2024-02-23T00:55:41.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.306+0000 7f2d68ff9700 10 client.42152.objecter ms_dispatch 0x7f2d6c001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:55:41.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.306+0000 7f2d68ff9700 10 client.42152.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:55:41.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.306+0000 7f2d68ff9700 20 client.42152.objecter put_session s=0x7f2d6c119210 osd=4 4 2024-02-23T00:55:41.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.306+0000 7f2d68ff9700 15 client.42152.objecter _session_command_op_remove 4 10 2024-02-23T00:55:41.310 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:55:41.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.307+0000 7f2d7209b700 10 client.42152.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:41.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.307+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_assign -1 11 2024-02-23T00:55:41.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.307+0000 7f2d7209b700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f2d6c123b40 0x7f2d6c123fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:41.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.307+0000 7f2d7209b700 20 client.42152.objecter _get_session s=0x7f2d6c125df0 osd=5 3 2024-02-23T00:55:41.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.307+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c125df0 osd=5 3 2024-02-23T00:55:41.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.307+0000 7f2d7209b700 20 client.42152.objecter _get_session s=0x7f2d6c125df0 osd=5 3 2024-02-23T00:55:41.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.307+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_remove -1 11 2024-02-23T00:55:41.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.307+0000 7f2d7209b700 20 client.42152.objecter get_session s=0x7f2d6c125df0 osd=5 3 2024-02-23T00:55:41.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.307+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_assign 5 11 2024-02-23T00:55:41.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.307+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c125df0 osd=5 4 2024-02-23T00:55:41.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.307+0000 7f2d7209b700 10 client.42152.objecter _send_command 11 2024-02-23T00:55:41.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.307+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2d6c1291a0 con 0x7f2d6c123b40 2024-02-23T00:55:41.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.307+0000 7f2d6affd700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f2d6c123b40 0x7f2d6c123fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:41.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.307+0000 7f2d68ff9700 10 client.42152.objecter ms_handle_connect 0x7f2d6c123b40 2024-02-23T00:55:41.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.307+0000 7f2d6affd700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f2d6c123b40 0x7f2d6c123fa0 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:41.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.308+0000 7f2d68ff9700 1 -- 172.21.15.47:0/784091020 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2d6c1291a0 con 0x7f2d6c123b40 2024-02-23T00:55:41.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.308+0000 7f2d68ff9700 10 client.42152.objecter ms_dispatch 0x7f2d6c001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:55:41.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.308+0000 7f2d68ff9700 10 client.42152.objecter _finish_command 11 = system:0 2024-02-23T00:55:41.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.308+0000 7f2d68ff9700 20 client.42152.objecter put_session s=0x7f2d6c125df0 osd=5 4 2024-02-23T00:55:41.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.308+0000 7f2d68ff9700 15 client.42152.objecter _session_command_op_remove 5 11 2024-02-23T00:55:41.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.323+0000 7f2d7209b700 10 client.42152.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:55:41.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.323+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_assign -1 12 2024-02-23T00:55:41.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.323+0000 7f2d7209b700 20 client.42152.objecter _get_session s=0x7f2d6c125df0 osd=5 3 2024-02-23T00:55:41.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.323+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c125df0 osd=5 3 2024-02-23T00:55:41.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.323+0000 7f2d7209b700 20 client.42152.objecter _get_session s=0x7f2d6c125df0 osd=5 3 2024-02-23T00:55:41.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.323+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_remove -1 12 2024-02-23T00:55:41.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.323+0000 7f2d7209b700 20 client.42152.objecter get_session s=0x7f2d6c125df0 osd=5 3 2024-02-23T00:55:41.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.323+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_assign 5 12 2024-02-23T00:55:41.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.323+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c125df0 osd=5 4 2024-02-23T00:55:41.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.323+0000 7f2d7209b700 10 client.42152.objecter _send_command 12 2024-02-23T00:55:41.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.323+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f2d6c1245a0 con 0x7f2d6c123b40 2024-02-23T00:55:41.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.325+0000 7f2d68ff9700 1 -- 172.21.15.47:0/784091020 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f2d6c1245a0 con 0x7f2d6c123b40 2024-02-23T00:55:41.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.325+0000 7f2d68ff9700 10 client.42152.objecter ms_dispatch 0x7f2d6c001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:55:41.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.325+0000 7f2d68ff9700 10 client.42152.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:55:41.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.325+0000 7f2d68ff9700 20 client.42152.objecter put_session s=0x7f2d6c125df0 osd=5 4 2024-02-23T00:55:41.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.325+0000 7f2d68ff9700 15 client.42152.objecter _session_command_op_remove 5 12 2024-02-23T00:55:41.328 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:55:41.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.325+0000 7f2d7209b700 10 client.42152.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:41.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.325+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_assign -1 13 2024-02-23T00:55:41.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.325+0000 7f2d7209b700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f2d6c124c20 0x7f2d6c125060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:41.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.325+0000 7f2d7209b700 20 client.42152.objecter _get_session s=0x7f2d6c1245a0 osd=6 3 2024-02-23T00:55:41.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.325+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c1245a0 osd=6 3 2024-02-23T00:55:41.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.325+0000 7f2d7209b700 20 client.42152.objecter _get_session s=0x7f2d6c1245a0 osd=6 3 2024-02-23T00:55:41.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.325+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_remove -1 13 2024-02-23T00:55:41.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.325+0000 7f2d7209b700 20 client.42152.objecter get_session s=0x7f2d6c1245a0 osd=6 3 2024-02-23T00:55:41.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.325+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_assign 6 13 2024-02-23T00:55:41.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.325+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c1245a0 osd=6 4 2024-02-23T00:55:41.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.325+0000 7f2d7209b700 10 client.42152.objecter _send_command 13 2024-02-23T00:55:41.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.325+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2d6c125710 con 0x7f2d6c124c20 2024-02-23T00:55:41.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.325+0000 7f2d6bfff700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f2d6c124c20 0x7f2d6c125060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:41.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.326+0000 7f2d68ff9700 10 client.42152.objecter ms_handle_connect 0x7f2d6c124c20 2024-02-23T00:55:41.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.326+0000 7f2d6bfff700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f2d6c124c20 0x7f2d6c125060 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:41.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.327+0000 7f2d68ff9700 1 -- 172.21.15.47:0/784091020 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2d6c125710 con 0x7f2d6c124c20 2024-02-23T00:55:41.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.327+0000 7f2d68ff9700 10 client.42152.objecter ms_dispatch 0x7f2d6c001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:55:41.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.327+0000 7f2d68ff9700 10 client.42152.objecter _finish_command 13 = system:0 2024-02-23T00:55:41.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.327+0000 7f2d68ff9700 20 client.42152.objecter put_session s=0x7f2d6c1245a0 osd=6 4 2024-02-23T00:55:41.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.327+0000 7f2d68ff9700 15 client.42152.objecter _session_command_op_remove 6 13 2024-02-23T00:55:41.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.341+0000 7f2d7209b700 10 client.42152.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:55:41.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.341+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_assign -1 14 2024-02-23T00:55:41.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.341+0000 7f2d7209b700 20 client.42152.objecter _get_session s=0x7f2d6c1245a0 osd=6 3 2024-02-23T00:55:41.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.341+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c1245a0 osd=6 3 2024-02-23T00:55:41.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.341+0000 7f2d7209b700 20 client.42152.objecter _get_session s=0x7f2d6c1245a0 osd=6 3 2024-02-23T00:55:41.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.341+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_remove -1 14 2024-02-23T00:55:41.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.341+0000 7f2d7209b700 20 client.42152.objecter get_session s=0x7f2d6c1245a0 osd=6 3 2024-02-23T00:55:41.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.341+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_assign 6 14 2024-02-23T00:55:41.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.341+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c1245a0 osd=6 4 2024-02-23T00:55:41.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.341+0000 7f2d7209b700 10 client.42152.objecter _send_command 14 2024-02-23T00:55:41.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.341+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f2d6c125710 con 0x7f2d6c124c20 2024-02-23T00:55:41.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.343+0000 7f2d68ff9700 1 -- 172.21.15.47:0/784091020 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f2d6c125710 con 0x7f2d6c124c20 2024-02-23T00:55:41.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.343+0000 7f2d68ff9700 10 client.42152.objecter ms_dispatch 0x7f2d6c001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:55:41.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.343+0000 7f2d68ff9700 10 client.42152.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:55:41.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.343+0000 7f2d68ff9700 20 client.42152.objecter put_session s=0x7f2d6c1245a0 osd=6 4 2024-02-23T00:55:41.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.343+0000 7f2d68ff9700 15 client.42152.objecter _session_command_op_remove 6 14 2024-02-23T00:55:41.347 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:55:41.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.344+0000 7f2d7209b700 10 client.42152.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:41.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.344+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_assign -1 15 2024-02-23T00:55:41.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.344+0000 7f2d7209b700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f2d6c12a540 0x7f2d6c12c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:41.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.344+0000 7f2d7209b700 20 client.42152.objecter _get_session s=0x7f2d6c125710 osd=7 3 2024-02-23T00:55:41.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.344+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c125710 osd=7 3 2024-02-23T00:55:41.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.344+0000 7f2d7209b700 20 client.42152.objecter _get_session s=0x7f2d6c125710 osd=7 3 2024-02-23T00:55:41.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.344+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_remove -1 15 2024-02-23T00:55:41.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.344+0000 7f2d7209b700 20 client.42152.objecter get_session s=0x7f2d6c125710 osd=7 3 2024-02-23T00:55:41.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.344+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_assign 7 15 2024-02-23T00:55:41.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.344+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c125710 osd=7 4 2024-02-23T00:55:41.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.344+0000 7f2d7209b700 10 client.42152.objecter _send_command 15 2024-02-23T00:55:41.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.344+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2d6c12f580 con 0x7f2d6c12a540 2024-02-23T00:55:41.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.344+0000 7f2d6b7fe700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f2d6c12a540 0x7f2d6c12c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:41.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.344+0000 7f2d68ff9700 10 client.42152.objecter ms_handle_connect 0x7f2d6c12a540 2024-02-23T00:55:41.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.344+0000 7f2d6b7fe700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f2d6c12a540 0x7f2d6c12c930 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:41.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.345+0000 7f2d68ff9700 1 -- 172.21.15.47:0/784091020 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2d6c12f580 con 0x7f2d6c12a540 2024-02-23T00:55:41.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.345+0000 7f2d68ff9700 10 client.42152.objecter ms_dispatch 0x7f2d6c001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:55:41.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.345+0000 7f2d68ff9700 10 client.42152.objecter _finish_command 15 = system:0 2024-02-23T00:55:41.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.345+0000 7f2d68ff9700 20 client.42152.objecter put_session s=0x7f2d6c125710 osd=7 4 2024-02-23T00:55:41.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.345+0000 7f2d68ff9700 15 client.42152.objecter _session_command_op_remove 7 15 2024-02-23T00:55:41.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.360+0000 7f2d7209b700 10 client.42152.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:55:41.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.360+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_assign -1 16 2024-02-23T00:55:41.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.360+0000 7f2d7209b700 20 client.42152.objecter _get_session s=0x7f2d6c125710 osd=7 3 2024-02-23T00:55:41.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.360+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c125710 osd=7 3 2024-02-23T00:55:41.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.360+0000 7f2d7209b700 20 client.42152.objecter _get_session s=0x7f2d6c125710 osd=7 3 2024-02-23T00:55:41.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.360+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_remove -1 16 2024-02-23T00:55:41.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.360+0000 7f2d7209b700 20 client.42152.objecter get_session s=0x7f2d6c125710 osd=7 3 2024-02-23T00:55:41.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.360+0000 7f2d7209b700 15 client.42152.objecter _session_command_op_assign 7 16 2024-02-23T00:55:41.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.360+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c125710 osd=7 4 2024-02-23T00:55:41.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.360+0000 7f2d7209b700 10 client.42152.objecter _send_command 16 2024-02-23T00:55:41.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.360+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f2d6c12ce20 con 0x7f2d6c12a540 2024-02-23T00:55:41.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.362+0000 7f2d68ff9700 1 -- 172.21.15.47:0/784091020 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f2d6c12ce20 con 0x7f2d6c12a540 2024-02-23T00:55:41.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.362+0000 7f2d68ff9700 10 client.42152.objecter ms_dispatch 0x7f2d6c001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:55:41.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.362+0000 7f2d68ff9700 10 client.42152.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:55:41.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.362+0000 7f2d68ff9700 20 client.42152.objecter put_session s=0x7f2d6c125710 osd=7 4 2024-02-23T00:55:41.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.362+0000 7f2d68ff9700 15 client.42152.objecter _session_command_op_remove 7 16 2024-02-23T00:55:41.366 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:55:41.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 10 librados: watch_flush enter 2024-02-23T00:55:41.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 10 librados: watch_flush exit 2024-02-23T00:55:41.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 10 client.42152.objecter close_session for osd.0 2024-02-23T00:55:41.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f2d6c028540 msgr2=0x7f2d6c0fbbe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:41.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f2d6c028540 0x7f2d6c0fbbe0 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:41.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c107010 osd=0 1 2024-02-23T00:55:41.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 10 client.42152.objecter close_session for osd.1 2024-02-23T00:55:41.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f2d6c0fc750 msgr2=0x7f2d6c0fcb30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:41.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f2d6c0fc750 0x7f2d6c0fcb30 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:41.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c0fc0d0 osd=1 1 2024-02-23T00:55:41.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 10 client.42152.objecter close_session for osd.2 2024-02-23T00:55:41.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f2d6c0280a0 msgr2=0x7f2d6c11c160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:41.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f2d6c0280a0 0x7f2d6c11c160 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:41.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c059770 osd=2 1 2024-02-23T00:55:41.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 10 client.42152.objecter close_session for osd.3 2024-02-23T00:55:41.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d6bfff700 1 -- 172.21.15.47:0/784091020 reap_dead start 2024-02-23T00:55:41.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f2d6c119cb0 msgr2=0x7f2d6c11d7d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:41.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f2d6c119cb0 0x7f2d6c11d7d0 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:41.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c027a00 osd=3 1 2024-02-23T00:55:41.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 10 client.42152.objecter close_session for osd.4 2024-02-23T00:55:41.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2d6c119380 msgr2=0x7f2d6c123000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:41.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f2d6c119380 0x7f2d6c123000 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:41.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c119210 osd=4 1 2024-02-23T00:55:41.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 10 client.42152.objecter close_session for osd.5 2024-02-23T00:55:41.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f2d6c123b40 msgr2=0x7f2d6c123fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:41.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f2d6c123b40 0x7f2d6c123fa0 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:41.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c125df0 osd=5 1 2024-02-23T00:55:41.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 10 client.42152.objecter close_session for osd.6 2024-02-23T00:55:41.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f2d6c124c20 msgr2=0x7f2d6c125060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:41.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f2d6c124c20 0x7f2d6c125060 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:41.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c1245a0 osd=6 1 2024-02-23T00:55:41.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 10 client.42152.objecter close_session for osd.7 2024-02-23T00:55:41.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f2d6c12a540 msgr2=0x7f2d6c12c930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:41.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f2d6c12a540 0x7f2d6c12c930 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:41.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 20 client.42152.objecter put_session s=0x7f2d6c125710 osd=7 1 2024-02-23T00:55:41.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 20 client.42152.objecter shutdown clearing up homeless session... 2024-02-23T00:55:41.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 10 client.42152.objecter successfully canceled tick 2024-02-23T00:55:41.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d6bfff700 1 -- 172.21.15.47:0/784091020 reap_dead start 2024-02-23T00:55:41.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2d54062cc0 msgr2=0x7f2d54065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:41.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2d54062cc0 0x7f2d54065180 secure :-1 s=READY pgs=3209 cs=0 l=1 rev1=1 rx=0x7f2d5c007fd0 tx=0x7f2d5c00b040).stop 2024-02-23T00:55:41.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2d6c102250 msgr2=0x7f2d6c1082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:41.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2d6c102250 0x7f2d6c1082b0 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 rx=0x7f2d64001cb0 tx=0x7f2d64004c50).stop 2024-02-23T00:55:41.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 shutdown_connections 2024-02-23T00:55:41.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2d6c102250 0x7f2d6c1082b0 unknown :-1 s=CLOSED pgs=2820 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:41.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 1 --2- 172.21.15.47:0/784091020 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2d54062cc0 0x7f2d54065180 unknown :-1 s=CLOSED pgs=3209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:41.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 >> 172.21.15.47:0/784091020 conn(0x7f2d6c0f26f0 msgr2=0x7f2d6c0f3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:41.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 shutdown_connections 2024-02-23T00:55:41.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 1 -- 172.21.15.47:0/784091020 wait complete. 2024-02-23T00:55:41.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:41.363+0000 7f2d7209b700 1 librados: shutdown 2024-02-23T00:55:41.371 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:55:41.371 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:55:41.371 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:55:41.371 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:55:41.371 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:55:41.371 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:55:41.371 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:55:41.371 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:55:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:41.219661+0000 mon.c (mon.2) 494 : audit [DBG] from='client.? 172.21.15.47:0/784091020' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:55:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:41.310536+0000 mon.a (mon.0) 3746 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:55:41.708 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:41.219661+0000 mon.c (mon.2) 494 : audit [DBG] from='client.? 172.21.15.47:0/784091020' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:55:41.708 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:41.310536+0000 mon.a (mon.0) 3746 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:55:41.709 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:41.219661+0000 mon.c (mon.2) 494 : audit [DBG] from='client.? 172.21.15.47:0/784091020' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:55:41.709 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:41.310536+0000 mon.a (mon.0) 3746 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:55:42.053 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:55:42.056 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:42.056 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:55:42.157 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:55:42.188 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:42.208 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:55:42.211 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:42.211 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:55:42.312 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:55:42.343 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:42.345 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:55:42.345 INFO:teuthology.orchestra.run.smithi196.stdout: "success": "filestore_inject_stall = '3' (not observed, change may require restart) " 2024-02-23T00:55:42.345 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:55:42.364 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:55:42.366 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:42.367 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:55:42.468 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:55:42.497 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:42.518 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:55:42.520 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:42.522 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:55:42.622 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:55:42.648 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:55:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:55:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:55:42.651 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:42.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:41.403929+0000 mgr.y (mgr.14517) 668 : cluster [DBG] pgmap v1198: 194 pgs: 194 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 1.8 KiB/s wr, 5 op/s; 370 B/s, 22 objects/s recovering 2024-02-23T00:55:42.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:41.738044+0000 mon.a (mon.0) 3747 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:55:42.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:41.403929+0000 mgr.y (mgr.14517) 668 : cluster [DBG] pgmap v1198: 194 pgs: 194 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 1.8 KiB/s wr, 5 op/s; 370 B/s, 22 objects/s recovering 2024-02-23T00:55:42.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:41.738044+0000 mon.a (mon.0) 3747 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:55:42.883 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset noscrub 2024-02-23T00:55:42.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:41.403929+0000 mgr.y (mgr.14517) 668 : cluster [DBG] pgmap v1198: 194 pgs: 194 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 1.8 KiB/s wr, 5 op/s; 370 B/s, 22 objects/s recovering 2024-02-23T00:55:42.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:41.738044+0000 mon.a (mon.0) 3747 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:55:43.000 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:55:43.003 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:43.004 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:55:43.105 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:55:43.119 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:55:43.135 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:43.154 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:55:43.157 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:43.157 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:55:43.164 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:55:43.258 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:55:43.282 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:55:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:55:43.284 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:43.409 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:55:43.411 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:43.412 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:55:43.513 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:55:43.541 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:43.615 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:55:43.617 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:43.618 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:55:43.719 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:55:43.748 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:43.948 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:55:43.950 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:43.950 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:55:44.051 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:55:44.079 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:44.103 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:55:44.729 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:55:44.732 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:44.733 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:55:44.834 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:55:44.864 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:44.907 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:55:44.907 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:55:44.928 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:42.990859+0000 mgr.y (mgr.14517) 669 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:55:44.928 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:43.404827+0000 mgr.y (mgr.14517) 670 : cluster [DBG] pgmap v1199: 194 pgs: 194 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 842 B/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:55:44.928 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:44.406173+0000 mon.a (mon.0) 3748 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.7", "id": [1, 6]}]: dispatch 2024-02-23T00:55:44.928 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:44.406372+0000 mon.a (mon.0) 3749 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.12", "id": [1, 6]}]: dispatch 2024-02-23T00:55:44.928 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:44.406533+0000 mon.a (mon.0) 3750 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.18", "id": [1, 3]}]: dispatch 2024-02-23T00:55:44.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.966+0000 7f95ae0d6700 1 -- 172.21.15.47:0/740367505 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f95a80fcbb0 msgr2=0x7f95a80fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:44.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.966+0000 7f95ae0d6700 1 --2- 172.21.15.47:0/740367505 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f95a80fcbb0 0x7f95a80fcf90 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 rx=0x7f95a0009be0 tx=0x7f95a0004d10).stop 2024-02-23T00:55:44.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.966+0000 7f95ae0d6700 1 -- 172.21.15.47:0/740367505 shutdown_connections 2024-02-23T00:55:44.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.966+0000 7f95ae0d6700 1 --2- 172.21.15.47:0/740367505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95a80f7400 0x7f95a8103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:44.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.966+0000 7f95ae0d6700 1 --2- 172.21.15.47:0/740367505 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f95a80fcbb0 0x7f95a80fcf90 unknown :-1 s=CLOSED pgs=2821 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:44.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.966+0000 7f95ae0d6700 1 --2- 172.21.15.47:0/740367505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95a80fd510 0x7f95a80f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:44.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.966+0000 7f95ae0d6700 1 -- 172.21.15.47:0/740367505 >> 172.21.15.47:0/740367505 conn(0x7f95a80f2820 msgr2=0x7f95a80f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:44.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.966+0000 7f95ae0d6700 1 -- 172.21.15.47:0/740367505 shutdown_connections 2024-02-23T00:55:44.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.966+0000 7f95ae0d6700 1 -- 172.21.15.47:0/740367505 wait complete. 2024-02-23T00:55:44.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.967+0000 7f95ae0d6700 1 librados: starting msgr at 2024-02-23T00:55:44.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.967+0000 7f95ae0d6700 1 librados: starting objecter 2024-02-23T00:55:44.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.967+0000 7f95ae0d6700 1 Processor -- start 2024-02-23T00:55:44.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.967+0000 7f95ae0d6700 1 -- start start 2024-02-23T00:55:44.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.967+0000 7f95ae0d6700 1 librados: setting wanted keys 2024-02-23T00:55:44.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.967+0000 7f95ae0d6700 1 librados: calling monclient init 2024-02-23T00:55:44.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.967+0000 7f95ae0d6700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95a80f7400 0x7f95a810cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:44.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.967+0000 7f95ae0d6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95a80fcbb0 0x7f95a810d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:44.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.967+0000 7f95ae0d6700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f95a80fd510 0x7f95a8107090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:44.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.967+0000 7f95ae0d6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a8107580 con 0x7f95a80fcbb0 2024-02-23T00:55:44.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.967+0000 7f95ae0d6700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a81076f0 con 0x7f95a80f7400 2024-02-23T00:55:44.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.967+0000 7f95ae0d6700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a81078b0 con 0x7f95a80fd510 2024-02-23T00:55:44.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.967+0000 7f95a7fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f95a80fd510 0x7f95a8107090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:44.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.967+0000 7f95a6ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95a80fcbb0 0x7f95a810d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:44.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.967+0000 7f95a7fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f95a80fd510 0x7f95a8107090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:41726/0 (socket says 172.21.15.47:41726) 2024-02-23T00:55:44.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.967+0000 7f95a7fff700 1 -- 172.21.15.47:0/1293224652 learned_addr learned my addr 172.21.15.47:0/1293224652 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:55:45.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.967+0000 7f95a6ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95a80fcbb0 0x7f95a810d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40028/0 (socket says 172.21.15.47:40028) 2024-02-23T00:55:45.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.967+0000 7f95a77fe700 1 --2- 172.21.15.47:0/1293224652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95a80f7400 0x7f95a810cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:45.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.968+0000 7f95a7fff700 1 -- 172.21.15.47:0/1293224652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95a80f7400 msgr2=0x7f95a810cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:45.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.968+0000 7f95a7fff700 1 --2- 172.21.15.47:0/1293224652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95a80f7400 0x7f95a810cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:45.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.968+0000 7f95a7fff700 1 -- 172.21.15.47:0/1293224652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95a80fcbb0 msgr2=0x7f95a810d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:45.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.968+0000 7f95a7fff700 1 --2- 172.21.15.47:0/1293224652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95a80fcbb0 0x7f95a810d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:45.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.968+0000 7f95a7fff700 1 -- 172.21.15.47:0/1293224652 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959c005040 con 0x7f95a80fd510 2024-02-23T00:55:45.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.968+0000 7f95a6ffd700 1 --2- 172.21.15.47:0/1293224652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95a80fcbb0 0x7f95a810d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:55:45.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.968+0000 7f95a77fe700 1 --2- 172.21.15.47:0/1293224652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95a80f7400 0x7f95a810cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:55:45.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.968+0000 7f95a7fff700 1 --2- 172.21.15.47:0/1293224652 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f95a80fd510 0x7f95a8107090 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 rx=0x7f959800c810 tx=0x7f9598007ee0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:45.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.968+0000 7f95a4ff9700 10 client.?.objecter ms_handle_connect 0x7f95a80fd510 2024-02-23T00:55:45.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.968+0000 7f95a4ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:55:45.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.968+0000 7f95a4ff9700 1 -- 172.21.15.47:0/1293224652 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959800e4a0 con 0x7f95a80fd510 2024-02-23T00:55:45.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.968+0000 7f95ae0d6700 1 -- 172.21.15.47:0/1293224652 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95a00097c0 con 0x7f95a80fd510 2024-02-23T00:55:45.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.968+0000 7f95a4ff9700 1 -- 172.21.15.47:0/1293224652 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f959800ea90 con 0x7f95a80fd510 2024-02-23T00:55:45.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.968+0000 7f95a4ff9700 1 -- 172.21.15.47:0/1293224652 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9598013a40 con 0x7f95a80fd510 2024-02-23T00:55:45.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.968+0000 7f95ae0d6700 10 client.42158.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:55:45.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.968+0000 7f95ae0d6700 1 -- 172.21.15.47:0/1293224652 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95a8107ef0 con 0x7f95a80fd510 2024-02-23T00:55:45.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.968+0000 7f95ae0d6700 1 librados: init done 2024-02-23T00:55:45.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.969+0000 7f95ae0d6700 1 -- 172.21.15.47:0/1293224652 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95a805fa60 con 0x7f95a80fd510 2024-02-23T00:55:45.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.969+0000 7f95a4ff9700 1 -- 172.21.15.47:0/1293224652 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f959800e600 con 0x7f95a80fd510 2024-02-23T00:55:45.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.970+0000 7f95a4ff9700 1 --2- 172.21.15.47:0/1293224652 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9590062cc0 0x7f9590065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:45.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.970+0000 7f95a4ff9700 1 -- 172.21.15.47:0/1293224652 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(789..789 src has 621..789) v4 ==== 9832+0+0 (secure 0 0 0) 0x7f959807d640 con 0x7f95a80fd510 2024-02-23T00:55:45.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.970+0000 7f95a4ff9700 10 client.42158.objecter ms_dispatch 0x7f95a8001550 osd_map(789..789 src has 621..789) v4 2024-02-23T00:55:45.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.970+0000 7f95a4ff9700 3 client.42158.objecter handle_osd_map got epochs [789,789] > 0 2024-02-23T00:55:45.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.970+0000 7f95a4ff9700 3 client.42158.objecter handle_osd_map decoding full epoch 789 2024-02-23T00:55:45.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.970+0000 7f95a77fe700 1 --2- 172.21.15.47:0/1293224652 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9590062cc0 0x7f9590065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:45.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.970+0000 7f95a4ff9700 20 client.42158.objecter dump_active .. 0 homeless 2024-02-23T00:55:45.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.970+0000 7f95a4ff9700 10 client.42158.objecter ms_handle_connect 0x7f9590062cc0 2024-02-23T00:55:45.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.970+0000 7f95a77fe700 1 --2- 172.21.15.47:0/1293224652 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9590062cc0 0x7f9590065180 secure :-1 s=READY pgs=3210 cs=0 l=1 rev1=1 rx=0x7f95a0004e00 tx=0x7f95a0004d10).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:45.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:44.974+0000 7f95a4ff9700 1 -- 172.21.15.47:0/1293224652 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f959804c990 con 0x7f95a80fd510 2024-02-23T00:55:45.006 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:55:45.014 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:55:45.016 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:42.990859+0000 mgr.y (mgr.14517) 669 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:55:45.016 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:43.404827+0000 mgr.y (mgr.14517) 670 : cluster [DBG] pgmap v1199: 194 pgs: 194 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 842 B/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:55:45.016 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:44.406173+0000 mon.a (mon.0) 3748 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.7", "id": [1, 6]}]: dispatch 2024-02-23T00:55:45.016 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:44.406372+0000 mon.a (mon.0) 3749 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.12", "id": [1, 6]}]: dispatch 2024-02-23T00:55:45.016 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:44.406533+0000 mon.a (mon.0) 3750 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.18", "id": [1, 3]}]: dispatch 2024-02-23T00:55:45.017 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:42.990859+0000 mgr.y (mgr.14517) 669 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:55:45.017 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:43.404827+0000 mgr.y (mgr.14517) 670 : cluster [DBG] pgmap v1199: 194 pgs: 194 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 842 B/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:55:45.017 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:44.406173+0000 mon.a (mon.0) 3748 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.7", "id": [1, 6]}]: dispatch 2024-02-23T00:55:45.017 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:44.406372+0000 mon.a (mon.0) 3749 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.12", "id": [1, 6]}]: dispatch 2024-02-23T00:55:45.017 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:44.406533+0000 mon.a (mon.0) 3750 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.18", "id": [1, 3]}]: dispatch 2024-02-23T00:55:45.017 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:45.018 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:55:45.119 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:55:45.148 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:45.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:45.173+0000 7f95ae0d6700 1 -- 172.21.15.47:0/1293224652 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7f95a8105f00 con 0x7f95a80fd510 2024-02-23T00:55:45.247 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:55:45.250 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:45.251 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:55:45.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:45.320+0000 7f95a4ff9700 1 -- 172.21.15.47:0/1293224652 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v790) v1 ==== 91+0+0 (secure 0 0 0) 0x7f959804ffb0 con 0x7f95a80fd510 2024-02-23T00:55:45.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:45.322+0000 7f95ae0d6700 10 librados: watch_flush enter 2024-02-23T00:55:45.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:45.322+0000 7f95ae0d6700 10 librados: watch_flush exit 2024-02-23T00:55:45.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:45.322+0000 7f95ae0d6700 20 client.42158.objecter shutdown clearing up homeless session... 2024-02-23T00:55:45.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:45.322+0000 7f95ae0d6700 10 client.42158.objecter successfully canceled tick 2024-02-23T00:55:45.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:45.322+0000 7f95ae0d6700 1 -- 172.21.15.47:0/1293224652 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9590062cc0 msgr2=0x7f9590065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:45.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:45.322+0000 7f95ae0d6700 1 --2- 172.21.15.47:0/1293224652 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9590062cc0 0x7f9590065180 secure :-1 s=READY pgs=3210 cs=0 l=1 rev1=1 rx=0x7f95a0004e00 tx=0x7f95a0004d10).stop 2024-02-23T00:55:45.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:45.322+0000 7f95ae0d6700 1 -- 172.21.15.47:0/1293224652 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f95a80fd510 msgr2=0x7f95a8107090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:45.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:45.322+0000 7f95ae0d6700 1 --2- 172.21.15.47:0/1293224652 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f95a80fd510 0x7f95a8107090 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 rx=0x7f959800c810 tx=0x7f9598007ee0).stop 2024-02-23T00:55:45.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:45.323+0000 7f95ae0d6700 1 -- 172.21.15.47:0/1293224652 shutdown_connections 2024-02-23T00:55:45.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:45.323+0000 7f95ae0d6700 1 --2- 172.21.15.47:0/1293224652 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9590062cc0 0x7f9590065180 unknown :-1 s=CLOSED pgs=3210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:45.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:45.323+0000 7f95ae0d6700 1 --2- 172.21.15.47:0/1293224652 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f95a80fd510 0x7f95a8107090 unknown :-1 s=CLOSED pgs=2822 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:45.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:45.323+0000 7f95ae0d6700 1 --2- 172.21.15.47:0/1293224652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95a80f7400 0x7f95a810cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:45.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:45.323+0000 7f95ae0d6700 1 --2- 172.21.15.47:0/1293224652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95a80fcbb0 0x7f95a810d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:45.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:45.323+0000 7f95ae0d6700 1 -- 172.21.15.47:0/1293224652 >> 172.21.15.47:0/1293224652 conn(0x7f95a80f2820 msgr2=0x7f95a80fa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:45.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:45.323+0000 7f95ae0d6700 1 -- 172.21.15.47:0/1293224652 shutdown_connections 2024-02-23T00:55:45.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:45.323+0000 7f95ae0d6700 1 -- 172.21.15.47:0/1293224652 wait complete. 2024-02-23T00:55:45.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:45.323+0000 7f95ae0d6700 1 librados: shutdown 2024-02-23T00:55:45.327 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is unset 2024-02-23T00:55:45.352 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:55:45.381 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:45.504 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:55:45.507 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:45.507 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:55:45.524 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.6 dump_historic_ops 2024-02-23T00:55:45.608 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:55:45.637 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:45.828 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:55:45.998 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:44.743097+0000 mon.a (mon.0) 3751 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.7", "id": [1, 6]}]': finished 2024-02-23T00:55:45.998 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:44.743223+0000 mon.a (mon.0) 3752 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.12", "id": [1, 6]}]': finished 2024-02-23T00:55:45.998 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:44.743285+0000 mon.a (mon.0) 3753 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.18", "id": [1, 3]}]': finished 2024-02-23T00:55:45.998 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:44.743335+0000 mon.a (mon.0) 3754 : cluster [DBG] osdmap e789: 8 total, 8 up, 6 in 2024-02-23T00:55:45.998 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:45.174727+0000 mon.c (mon.2) 495 : audit [INF] from='client.? 172.21.15.47:0/1293224652' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:55:45.998 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:45.175019+0000 mon.a (mon.0) 3755 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:55:45.998 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:45.315132+0000 mon.a (mon.0) 3756 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:55:45.999 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:45.319715+0000 mon.a (mon.0) 3757 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:55:45.999 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:45.319784+0000 mon.a (mon.0) 3758 : cluster [DBG] osdmap e790: 8 total, 8 up, 6 in 2024-02-23T00:55:45.999 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:45.405277+0000 mgr.y (mgr.14517) 671 : cluster [DBG] pgmap v1202: 194 pgs: 194 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:55:45.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:44.743097+0000 mon.a (mon.0) 3751 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.7", "id": [1, 6]}]': finished 2024-02-23T00:55:45.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:44.743223+0000 mon.a (mon.0) 3752 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.12", "id": [1, 6]}]': finished 2024-02-23T00:55:45.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:44.743285+0000 mon.a (mon.0) 3753 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.18", "id": [1, 3]}]': finished 2024-02-23T00:55:45.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:44.743335+0000 mon.a (mon.0) 3754 : cluster [DBG] osdmap e789: 8 total, 8 up, 6 in 2024-02-23T00:55:45.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:45.174727+0000 mon.c (mon.2) 495 : audit [INF] from='client.? 172.21.15.47:0/1293224652' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:55:45.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:45.175019+0000 mon.a (mon.0) 3755 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:55:45.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:45.315132+0000 mon.a (mon.0) 3756 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:55:46.000 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:45.319715+0000 mon.a (mon.0) 3757 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:55:46.000 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:45.319784+0000 mon.a (mon.0) 3758 : cluster [DBG] osdmap e790: 8 total, 8 up, 6 in 2024-02-23T00:55:46.000 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:45.405277+0000 mgr.y (mgr.14517) 671 : cluster [DBG] pgmap v1202: 194 pgs: 194 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:55:46.078 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:44.743097+0000 mon.a (mon.0) 3751 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.7", "id": [1, 6]}]': finished 2024-02-23T00:55:46.078 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:44.743223+0000 mon.a (mon.0) 3752 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.12", "id": [1, 6]}]': finished 2024-02-23T00:55:46.078 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:44.743285+0000 mon.a (mon.0) 3753 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.18", "id": [1, 3]}]': finished 2024-02-23T00:55:46.078 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:44.743335+0000 mon.a (mon.0) 3754 : cluster [DBG] osdmap e789: 8 total, 8 up, 6 in 2024-02-23T00:55:46.078 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:45.174727+0000 mon.c (mon.2) 495 : audit [INF] from='client.? 172.21.15.47:0/1293224652' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:55:46.079 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:45.175019+0000 mon.a (mon.0) 3755 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:55:46.079 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:45.315132+0000 mon.a (mon.0) 3756 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:55:46.079 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:45.319715+0000 mon.a (mon.0) 3757 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:55:46.079 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:45.319784+0000 mon.a (mon.0) 3758 : cluster [DBG] osdmap e790: 8 total, 8 up, 6 in 2024-02-23T00:55:46.079 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:45.405277+0000 mgr.y (mgr.14517) 671 : cluster [DBG] pgmap v1202: 194 pgs: 194 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:55:46.387 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:55:46.390 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:46.390 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:55:46.491 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:55:46.521 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:46.619 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:55:46.621 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:46.622 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:55:46.723 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:55:46.749 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:46.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:45.925004+0000 osd.5 (osd.5) 336 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:55:46.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:45.925198+0000 osd.5 (osd.5) 337 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:55:46.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:46.745363+0000 mon.a (mon.0) 3759 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:55:47.022 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:45.925004+0000 osd.5 (osd.5) 336 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:55:47.023 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:45.925198+0000 osd.5 (osd.5) 337 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:55:47.023 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:46.745363+0000 mon.a (mon.0) 3759 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:55:47.023 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:45.925004+0000 osd.5 (osd.5) 336 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:55:47.023 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:45.925198+0000 osd.5 (osd.5) 337 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:55:47.023 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:46.745363+0000 mon.a (mon.0) 3759 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:55:47.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.257+0000 7f02c8084700 1 -- 172.21.15.47:0/3573997168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02c00fcbb0 msgr2=0x7f02c00fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:47.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.257+0000 7f02c8084700 1 --2- 172.21.15.47:0/3573997168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02c00fcbb0 0x7f02c00fcf90 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 rx=0x7f02b4004870 tx=0x7f02b4009cb0).stop 2024-02-23T00:55:47.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.257+0000 7f02c8084700 1 -- 172.21.15.47:0/3573997168 shutdown_connections 2024-02-23T00:55:47.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.257+0000 7f02c8084700 1 --2- 172.21.15.47:0/3573997168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02c00f7400 0x7f02c0103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:47.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.257+0000 7f02c8084700 1 --2- 172.21.15.47:0/3573997168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02c00fcbb0 0x7f02c00fcf90 unknown :-1 s=CLOSED pgs=2645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:47.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.257+0000 7f02c8084700 1 --2- 172.21.15.47:0/3573997168 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f02c00fd510 0x7f02c00f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:47.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.257+0000 7f02c8084700 1 -- 172.21.15.47:0/3573997168 >> 172.21.15.47:0/3573997168 conn(0x7f02c00f2820 msgr2=0x7f02c00f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:47.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.257+0000 7f02c8084700 1 -- 172.21.15.47:0/3573997168 shutdown_connections 2024-02-23T00:55:47.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.257+0000 7f02c8084700 1 -- 172.21.15.47:0/3573997168 wait complete. 2024-02-23T00:55:47.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.258+0000 7f02c8084700 1 librados: starting msgr at 2024-02-23T00:55:47.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.258+0000 7f02c8084700 1 librados: starting objecter 2024-02-23T00:55:47.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.258+0000 7f02c8084700 1 Processor -- start 2024-02-23T00:55:47.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.258+0000 7f02c8084700 1 -- start start 2024-02-23T00:55:47.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.258+0000 7f02c8084700 1 librados: setting wanted keys 2024-02-23T00:55:47.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.258+0000 7f02c8084700 1 librados: calling monclient init 2024-02-23T00:55:47.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.258+0000 7f02c8084700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02c00f7400 0x7f02c00f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:47.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.258+0000 7f02c8084700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02c00fcbb0 0x7f02c00f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:47.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.258+0000 7f02c8084700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f02c00fd510 0x7f02c00f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:47.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.258+0000 7f02c8084700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c0116510 con 0x7f02c00fcbb0 2024-02-23T00:55:47.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.258+0000 7f02c8084700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c0116680 con 0x7f02c00f7400 2024-02-23T00:55:47.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.258+0000 7f02c8084700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c0116840 con 0x7f02c00fd510 2024-02-23T00:55:47.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.258+0000 7f02c6621700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f02c00fd510 0x7f02c00f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:47.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.258+0000 7f02c561f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02c00fcbb0 0x7f02c00f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:47.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.258+0000 7f02c6621700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f02c00fd510 0x7f02c00f9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:57308/0 (socket says 172.21.15.47:57308) 2024-02-23T00:55:47.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.258+0000 7f02c561f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02c00fcbb0 0x7f02c00f8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34700/0 (socket says 172.21.15.47:34700) 2024-02-23T00:55:47.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.258+0000 7f02c6621700 1 -- 172.21.15.47:0/776399190 learned_addr learned my addr 172.21.15.47:0/776399190 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:55:47.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.258+0000 7f02c5e20700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02c00f7400 0x7f02c00f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:47.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.259+0000 7f02c561f700 1 -- 172.21.15.47:0/776399190 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f02c00fd510 msgr2=0x7f02c00f9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:47.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.259+0000 7f02c561f700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f02c00fd510 0x7f02c00f9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:47.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.259+0000 7f02c561f700 1 -- 172.21.15.47:0/776399190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02c00f7400 msgr2=0x7f02c00f89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:47.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.259+0000 7f02c561f700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02c00f7400 0x7f02c00f89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:47.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.259+0000 7f02c561f700 1 -- 172.21.15.47:0/776399190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02bc005040 con 0x7f02c00fcbb0 2024-02-23T00:55:47.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.259+0000 7f02c561f700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02c00fcbb0 0x7f02c00f8ea0 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 rx=0x7f02bc004920 tx=0x7f02bc009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:47.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.259+0000 7f02b2ffd700 10 client.?.objecter ms_handle_connect 0x7f02c00fcbb0 2024-02-23T00:55:47.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.259+0000 7f02b2ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:55:47.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.259+0000 7f02b2ffd700 1 -- 172.21.15.47:0/776399190 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc00e030 con 0x7f02c00fcbb0 2024-02-23T00:55:47.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.259+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b4005040 con 0x7f02c00fcbb0 2024-02-23T00:55:47.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.259+0000 7f02b2ffd700 1 -- 172.21.15.47:0/776399190 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f02bc00f040 con 0x7f02c00fcbb0 2024-02-23T00:55:47.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.259+0000 7f02b2ffd700 1 -- 172.21.15.47:0/776399190 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc014720 con 0x7f02c00fcbb0 2024-02-23T00:55:47.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.259+0000 7f02c8084700 10 client.31839.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:55:47.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.259+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c0116e80 con 0x7f02c00fcbb0 2024-02-23T00:55:47.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.259+0000 7f02c8084700 1 librados: init done 2024-02-23T00:55:47.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.260+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f02c005fa60 con 0x7f02c00fcbb0 2024-02-23T00:55:47.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.261+0000 7f02b2ffd700 1 -- 172.21.15.47:0/776399190 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f02bc010500 con 0x7f02c00fcbb0 2024-02-23T00:55:47.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.261+0000 7f02b2ffd700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f02ac062cc0 0x7f02ac065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:47.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.261+0000 7f02b2ffd700 1 -- 172.21.15.47:0/776399190 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(791..791 src has 621..791) v4 ==== 9832+0+0 (secure 0 0 0) 0x7f02bc07df40 con 0x7f02c00fcbb0 2024-02-23T00:55:47.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.261+0000 7f02b2ffd700 10 client.31839.objecter ms_dispatch 0x7f02c0001550 osd_map(791..791 src has 621..791) v4 2024-02-23T00:55:47.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.261+0000 7f02b2ffd700 3 client.31839.objecter handle_osd_map got epochs [791,791] > 0 2024-02-23T00:55:47.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.261+0000 7f02b2ffd700 3 client.31839.objecter handle_osd_map decoding full epoch 791 2024-02-23T00:55:47.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.261+0000 7f02c5e20700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f02ac062cc0 0x7f02ac065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:47.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.261+0000 7f02b2ffd700 20 client.31839.objecter dump_active .. 0 homeless 2024-02-23T00:55:47.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.261+0000 7f02b2ffd700 1 -- 172.21.15.47:0/776399190 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v791) v1 ==== 54+0+16 (secure 0 0 0) 0x7f02bc07e3e0 con 0x7f02c00fcbb0 2024-02-23T00:55:47.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.261+0000 7f02c8084700 10 client.31839.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:47.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.261+0000 7f02c8084700 15 client.31839.objecter _session_command_op_assign -1 1 2024-02-23T00:55:47.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.261+0000 7f02c8084700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f02c00ffed0 0x7f02c01002b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:47.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.261+0000 7f02c8084700 20 client.31839.objecter _get_session s=0x7f02c005ee80 osd=0 3 2024-02-23T00:55:47.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.261+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c005ee80 osd=0 3 2024-02-23T00:55:47.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.261+0000 7f02c8084700 20 client.31839.objecter _get_session s=0x7f02c005ee80 osd=0 3 2024-02-23T00:55:47.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.261+0000 7f02c8084700 15 client.31839.objecter _session_command_op_remove -1 1 2024-02-23T00:55:47.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.261+0000 7f02c8084700 20 client.31839.objecter get_session s=0x7f02c005ee80 osd=0 3 2024-02-23T00:55:47.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.261+0000 7f02c8084700 15 client.31839.objecter _session_command_op_assign 0 1 2024-02-23T00:55:47.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.261+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c005ee80 osd=0 4 2024-02-23T00:55:47.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.261+0000 7f02c8084700 10 client.31839.objecter _send_command 1 2024-02-23T00:55:47.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.261+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f02c0105ea0 con 0x7f02c00ffed0 2024-02-23T00:55:47.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.261+0000 7f02b2ffd700 10 client.31839.objecter ms_handle_connect 0x7f02ac062cc0 2024-02-23T00:55:47.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.261+0000 7f02c5e20700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f02ac062cc0 0x7f02ac065180 secure :-1 s=READY pgs=3211 cs=0 l=1 rev1=1 rx=0x7f02b4004ee0 tx=0x7f02b400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:47.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.262+0000 7f02c6621700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f02c00ffed0 0x7f02c01002b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:47.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.262+0000 7f02c6621700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f02c00ffed0 0x7f02c01002b0 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:47.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.262+0000 7f02b2ffd700 10 client.31839.objecter ms_handle_connect 0x7f02c00ffed0 2024-02-23T00:55:47.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.262+0000 7f02b2ffd700 1 -- 172.21.15.47:0/776399190 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f02c0105ea0 con 0x7f02c00ffed0 2024-02-23T00:55:47.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.262+0000 7f02b2ffd700 10 client.31839.objecter ms_dispatch 0x7f02c0001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:55:47.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.262+0000 7f02b2ffd700 10 client.31839.objecter _finish_command 1 = system:0 2024-02-23T00:55:47.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.262+0000 7f02b2ffd700 20 client.31839.objecter put_session s=0x7f02c005ee80 osd=0 4 2024-02-23T00:55:47.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.262+0000 7f02b2ffd700 15 client.31839.objecter _session_command_op_remove 0 1 2024-02-23T00:55:47.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.277+0000 7f02c8084700 10 client.31839.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:55:47.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.277+0000 7f02c8084700 15 client.31839.objecter _session_command_op_assign -1 2 2024-02-23T00:55:47.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.277+0000 7f02c8084700 20 client.31839.objecter _get_session s=0x7f02c005ee80 osd=0 3 2024-02-23T00:55:47.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.277+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c005ee80 osd=0 3 2024-02-23T00:55:47.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.277+0000 7f02c8084700 20 client.31839.objecter _get_session s=0x7f02c005ee80 osd=0 3 2024-02-23T00:55:47.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.277+0000 7f02c8084700 15 client.31839.objecter _session_command_op_remove -1 2 2024-02-23T00:55:47.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.277+0000 7f02c8084700 20 client.31839.objecter get_session s=0x7f02c005ee80 osd=0 3 2024-02-23T00:55:47.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.277+0000 7f02c8084700 15 client.31839.objecter _session_command_op_assign 0 2 2024-02-23T00:55:47.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.277+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c005ee80 osd=0 4 2024-02-23T00:55:47.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.277+0000 7f02c8084700 10 client.31839.objecter _send_command 2 2024-02-23T00:55:47.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.277+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f02c00273c0 con 0x7f02c00ffed0 2024-02-23T00:55:47.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.279+0000 7f02b2ffd700 1 -- 172.21.15.47:0/776399190 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f02c00273c0 con 0x7f02c00ffed0 2024-02-23T00:55:47.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.279+0000 7f02b2ffd700 10 client.31839.objecter ms_dispatch 0x7f02c0001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:55:47.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.279+0000 7f02b2ffd700 10 client.31839.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:55:47.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.279+0000 7f02b2ffd700 20 client.31839.objecter put_session s=0x7f02c005ee80 osd=0 4 2024-02-23T00:55:47.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.279+0000 7f02b2ffd700 15 client.31839.objecter _session_command_op_remove 0 2 2024-02-23T00:55:47.283 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:55:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.279+0000 7f02c8084700 10 client.31839.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.279+0000 7f02c8084700 15 client.31839.objecter _session_command_op_assign -1 3 2024-02-23T00:55:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.279+0000 7f02c8084700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f02c00598f0 0x7f02c0059d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.279+0000 7f02c8084700 20 client.31839.objecter _get_session s=0x7f02c0028710 osd=1 3 2024-02-23T00:55:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.279+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c0028710 osd=1 3 2024-02-23T00:55:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.279+0000 7f02c8084700 20 client.31839.objecter _get_session s=0x7f02c0028710 osd=1 3 2024-02-23T00:55:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.279+0000 7f02c8084700 15 client.31839.objecter _session_command_op_remove -1 3 2024-02-23T00:55:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.279+0000 7f02c8084700 20 client.31839.objecter get_session s=0x7f02c0028710 osd=1 3 2024-02-23T00:55:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.279+0000 7f02c8084700 15 client.31839.objecter _session_command_op_assign 1 3 2024-02-23T00:55:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.279+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c0028710 osd=1 4 2024-02-23T00:55:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.279+0000 7f02c8084700 10 client.31839.objecter _send_command 3 2024-02-23T00:55:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.279+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f02c011d790 con 0x7f02c00598f0 2024-02-23T00:55:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.279+0000 7f02c5e20700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f02c00598f0 0x7f02c0059d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:47.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.280+0000 7f02b2ffd700 10 client.31839.objecter ms_handle_connect 0x7f02c00598f0 2024-02-23T00:55:47.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.280+0000 7f02c5e20700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f02c00598f0 0x7f02c0059d30 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:47.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.280+0000 7f02b2ffd700 1 -- 172.21.15.47:0/776399190 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f02c011d790 con 0x7f02c00598f0 2024-02-23T00:55:47.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.280+0000 7f02b2ffd700 10 client.31839.objecter ms_dispatch 0x7f02c0001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:55:47.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.280+0000 7f02b2ffd700 10 client.31839.objecter _finish_command 3 = system:0 2024-02-23T00:55:47.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.280+0000 7f02b2ffd700 20 client.31839.objecter put_session s=0x7f02c0028710 osd=1 4 2024-02-23T00:55:47.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.280+0000 7f02b2ffd700 15 client.31839.objecter _session_command_op_remove 1 3 2024-02-23T00:55:47.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.295+0000 7f02c8084700 10 client.31839.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:55:47.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.295+0000 7f02c8084700 15 client.31839.objecter _session_command_op_assign -1 4 2024-02-23T00:55:47.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.295+0000 7f02c8084700 20 client.31839.objecter _get_session s=0x7f02c0028710 osd=1 3 2024-02-23T00:55:47.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.295+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c0028710 osd=1 3 2024-02-23T00:55:47.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.295+0000 7f02c8084700 20 client.31839.objecter _get_session s=0x7f02c0028710 osd=1 3 2024-02-23T00:55:47.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.295+0000 7f02c8084700 15 client.31839.objecter _session_command_op_remove -1 4 2024-02-23T00:55:47.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.295+0000 7f02c8084700 20 client.31839.objecter get_session s=0x7f02c0028710 osd=1 3 2024-02-23T00:55:47.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.295+0000 7f02c8084700 15 client.31839.objecter _session_command_op_assign 1 4 2024-02-23T00:55:47.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.295+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c0028710 osd=1 4 2024-02-23T00:55:47.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.295+0000 7f02c8084700 10 client.31839.objecter _send_command 4 2024-02-23T00:55:47.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.295+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f02c0059360 con 0x7f02c00598f0 2024-02-23T00:55:47.299 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:55:47.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.297+0000 7f02b2ffd700 1 -- 172.21.15.47:0/776399190 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f02c0059360 con 0x7f02c00598f0 2024-02-23T00:55:47.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.297+0000 7f02b2ffd700 10 client.31839.objecter ms_dispatch 0x7f02c0001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:55:47.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.297+0000 7f02b2ffd700 10 client.31839.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:55:47.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.297+0000 7f02b2ffd700 20 client.31839.objecter put_session s=0x7f02c0028710 osd=1 4 2024-02-23T00:55:47.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.297+0000 7f02b2ffd700 15 client.31839.objecter _session_command_op_remove 1 4 2024-02-23T00:55:47.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.297+0000 7f02c8084700 10 client.31839.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:47.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.297+0000 7f02c8084700 15 client.31839.objecter _session_command_op_assign -1 5 2024-02-23T00:55:47.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.297+0000 7f02c8084700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f02c0028270 0x7f02c011c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:47.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.297+0000 7f02c8084700 20 client.31839.objecter _get_session s=0x7f02c0059710 osd=2 3 2024-02-23T00:55:47.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.297+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c0059710 osd=2 3 2024-02-23T00:55:47.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.297+0000 7f02c8084700 20 client.31839.objecter _get_session s=0x7f02c0059710 osd=2 3 2024-02-23T00:55:47.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.297+0000 7f02c8084700 15 client.31839.objecter _session_command_op_remove -1 5 2024-02-23T00:55:47.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.297+0000 7f02c8084700 20 client.31839.objecter get_session s=0x7f02c0059710 osd=2 3 2024-02-23T00:55:47.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.297+0000 7f02c8084700 15 client.31839.objecter _session_command_op_assign 2 5 2024-02-23T00:55:47.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.297+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c0059710 osd=2 4 2024-02-23T00:55:47.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.297+0000 7f02c8084700 10 client.31839.objecter _send_command 5 2024-02-23T00:55:47.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.297+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f02c011c6a0 con 0x7f02c0028270 2024-02-23T00:55:47.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.297+0000 7f02c561f700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f02c0028270 0x7f02c011c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:47.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.297+0000 7f02b2ffd700 10 client.31839.objecter ms_handle_connect 0x7f02c0028270 2024-02-23T00:55:47.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.297+0000 7f02c561f700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f02c0028270 0x7f02c011c1b0 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:47.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.298+0000 7f02b2ffd700 1 -- 172.21.15.47:0/776399190 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f02c011c6a0 con 0x7f02c0028270 2024-02-23T00:55:47.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.298+0000 7f02b2ffd700 10 client.31839.objecter ms_dispatch 0x7f02c0001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:55:47.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.298+0000 7f02b2ffd700 10 client.31839.objecter _finish_command 5 = system:0 2024-02-23T00:55:47.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.298+0000 7f02b2ffd700 20 client.31839.objecter put_session s=0x7f02c0059710 osd=2 4 2024-02-23T00:55:47.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.298+0000 7f02b2ffd700 15 client.31839.objecter _session_command_op_remove 2 5 2024-02-23T00:55:47.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.312+0000 7f02c8084700 10 client.31839.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:55:47.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.312+0000 7f02c8084700 15 client.31839.objecter _session_command_op_assign -1 6 2024-02-23T00:55:47.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.312+0000 7f02c8084700 20 client.31839.objecter _get_session s=0x7f02c0059710 osd=2 3 2024-02-23T00:55:47.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.312+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c0059710 osd=2 3 2024-02-23T00:55:47.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.312+0000 7f02c8084700 20 client.31839.objecter _get_session s=0x7f02c0059710 osd=2 3 2024-02-23T00:55:47.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.312+0000 7f02c8084700 15 client.31839.objecter _session_command_op_remove -1 6 2024-02-23T00:55:47.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.312+0000 7f02c8084700 20 client.31839.objecter get_session s=0x7f02c0059710 osd=2 3 2024-02-23T00:55:47.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.312+0000 7f02c8084700 15 client.31839.objecter _session_command_op_assign 2 6 2024-02-23T00:55:47.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.312+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c0059710 osd=2 4 2024-02-23T00:55:47.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.312+0000 7f02c8084700 10 client.31839.objecter _send_command 6 2024-02-23T00:55:47.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.312+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f02c00277d0 con 0x7f02c0028270 2024-02-23T00:55:47.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.315+0000 7f02b2ffd700 1 -- 172.21.15.47:0/776399190 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f02c00277d0 con 0x7f02c0028270 2024-02-23T00:55:47.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.315+0000 7f02b2ffd700 10 client.31839.objecter ms_dispatch 0x7f02c0001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:55:47.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.315+0000 7f02b2ffd700 10 client.31839.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:55:47.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.315+0000 7f02b2ffd700 20 client.31839.objecter put_session s=0x7f02c0059710 osd=2 4 2024-02-23T00:55:47.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.315+0000 7f02b2ffd700 15 client.31839.objecter _session_command_op_remove 2 6 2024-02-23T00:55:47.319 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:55:47.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.315+0000 7f02c8084700 10 client.31839.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:47.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.315+0000 7f02c8084700 15 client.31839.objecter _session_command_op_assign -1 7 2024-02-23T00:55:47.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.315+0000 7f02c8084700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f02c011f910 0x7f02c0121d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:47.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.315+0000 7f02c8084700 20 client.31839.objecter _get_session s=0x7f02c0027be0 osd=3 3 2024-02-23T00:55:47.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.315+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c0027be0 osd=3 3 2024-02-23T00:55:47.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.315+0000 7f02c8084700 20 client.31839.objecter _get_session s=0x7f02c0027be0 osd=3 3 2024-02-23T00:55:47.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.315+0000 7f02c8084700 15 client.31839.objecter _session_command_op_remove -1 7 2024-02-23T00:55:47.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.315+0000 7f02c8084700 20 client.31839.objecter get_session s=0x7f02c0027be0 osd=3 3 2024-02-23T00:55:47.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.315+0000 7f02c8084700 15 client.31839.objecter _session_command_op_assign 3 7 2024-02-23T00:55:47.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.315+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c0027be0 osd=3 4 2024-02-23T00:55:47.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.315+0000 7f02c8084700 10 client.31839.objecter _send_command 7 2024-02-23T00:55:47.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.315+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f02c0124950 con 0x7f02c011f910 2024-02-23T00:55:47.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.315+0000 7f02c6621700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f02c011f910 0x7f02c0121d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:47.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.315+0000 7f02b2ffd700 10 client.31839.objecter ms_handle_connect 0x7f02c011f910 2024-02-23T00:55:47.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.315+0000 7f02c6621700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f02c011f910 0x7f02c0121d00 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:47.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.315+0000 7f02b2ffd700 1 -- 172.21.15.47:0/776399190 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f02c0124950 con 0x7f02c011f910 2024-02-23T00:55:47.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.315+0000 7f02b2ffd700 10 client.31839.objecter ms_dispatch 0x7f02c0001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:55:47.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.315+0000 7f02b2ffd700 10 client.31839.objecter _finish_command 7 = system:0 2024-02-23T00:55:47.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.315+0000 7f02b2ffd700 20 client.31839.objecter put_session s=0x7f02c0027be0 osd=3 4 2024-02-23T00:55:47.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.315+0000 7f02b2ffd700 15 client.31839.objecter _session_command_op_remove 3 7 2024-02-23T00:55:47.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.330+0000 7f02c8084700 10 client.31839.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:55:47.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.330+0000 7f02c8084700 15 client.31839.objecter _session_command_op_assign -1 8 2024-02-23T00:55:47.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.330+0000 7f02c8084700 20 client.31839.objecter _get_session s=0x7f02c0027be0 osd=3 3 2024-02-23T00:55:47.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.330+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c0027be0 osd=3 3 2024-02-23T00:55:47.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.330+0000 7f02c8084700 20 client.31839.objecter _get_session s=0x7f02c0027be0 osd=3 3 2024-02-23T00:55:47.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.330+0000 7f02c8084700 15 client.31839.objecter _session_command_op_remove -1 8 2024-02-23T00:55:47.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.330+0000 7f02c8084700 20 client.31839.objecter get_session s=0x7f02c0027be0 osd=3 3 2024-02-23T00:55:47.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.330+0000 7f02c8084700 15 client.31839.objecter _session_command_op_assign 3 8 2024-02-23T00:55:47.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.330+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c0027be0 osd=3 4 2024-02-23T00:55:47.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.330+0000 7f02c8084700 10 client.31839.objecter _send_command 8 2024-02-23T00:55:47.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.330+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f02c011ca50 con 0x7f02c011f910 2024-02-23T00:55:47.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.331+0000 7f02b2ffd700 1 -- 172.21.15.47:0/776399190 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f02c011ca50 con 0x7f02c011f910 2024-02-23T00:55:47.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.331+0000 7f02b2ffd700 10 client.31839.objecter ms_dispatch 0x7f02c0001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:55:47.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.331+0000 7f02b2ffd700 10 client.31839.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:55:47.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.331+0000 7f02b2ffd700 20 client.31839.objecter put_session s=0x7f02c0027be0 osd=3 4 2024-02-23T00:55:47.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.331+0000 7f02b2ffd700 15 client.31839.objecter _session_command_op_remove 3 8 2024-02-23T00:55:47.334 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:55:47.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.331+0000 7f02c8084700 10 client.31839.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:47.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.331+0000 7f02c8084700 15 client.31839.objecter _session_command_op_assign -1 9 2024-02-23T00:55:47.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.331+0000 7f02c8084700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f02c011cbe0 0x7f02c0123370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:47.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.331+0000 7f02c8084700 20 client.31839.objecter _get_session s=0x7f02c011ca70 osd=4 3 2024-02-23T00:55:47.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.331+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c011ca70 osd=4 3 2024-02-23T00:55:47.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.331+0000 7f02c8084700 20 client.31839.objecter _get_session s=0x7f02c011ca70 osd=4 3 2024-02-23T00:55:47.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.331+0000 7f02c8084700 15 client.31839.objecter _session_command_op_remove -1 9 2024-02-23T00:55:47.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.331+0000 7f02c8084700 20 client.31839.objecter get_session s=0x7f02c011ca70 osd=4 3 2024-02-23T00:55:47.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.331+0000 7f02c8084700 15 client.31839.objecter _session_command_op_assign 4 9 2024-02-23T00:55:47.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.331+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c011ca70 osd=4 4 2024-02-23T00:55:47.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.331+0000 7f02c8084700 10 client.31839.objecter _send_command 9 2024-02-23T00:55:47.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.331+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f02c011bf40 con 0x7f02c011cbe0 2024-02-23T00:55:47.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.331+0000 7f02c5e20700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f02c011cbe0 0x7f02c0123370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:47.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.332+0000 7f02b2ffd700 10 client.31839.objecter ms_handle_connect 0x7f02c011cbe0 2024-02-23T00:55:47.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.332+0000 7f02c5e20700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f02c011cbe0 0x7f02c0123370 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:47.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.333+0000 7f02b2ffd700 1 -- 172.21.15.47:0/776399190 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f02c011bf40 con 0x7f02c011cbe0 2024-02-23T00:55:47.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.333+0000 7f02b2ffd700 10 client.31839.objecter ms_dispatch 0x7f02c0001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:55:47.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.333+0000 7f02b2ffd700 10 client.31839.objecter _finish_command 9 = system:0 2024-02-23T00:55:47.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.333+0000 7f02b2ffd700 20 client.31839.objecter put_session s=0x7f02c011ca70 osd=4 4 2024-02-23T00:55:47.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.333+0000 7f02b2ffd700 15 client.31839.objecter _session_command_op_remove 4 9 2024-02-23T00:55:47.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.349+0000 7f02c8084700 10 client.31839.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:55:47.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.349+0000 7f02c8084700 15 client.31839.objecter _session_command_op_assign -1 10 2024-02-23T00:55:47.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.349+0000 7f02c8084700 20 client.31839.objecter _get_session s=0x7f02c011ca70 osd=4 3 2024-02-23T00:55:47.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.349+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c011ca70 osd=4 3 2024-02-23T00:55:47.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.349+0000 7f02c8084700 20 client.31839.objecter _get_session s=0x7f02c011ca70 osd=4 3 2024-02-23T00:55:47.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.349+0000 7f02c8084700 15 client.31839.objecter _session_command_op_remove -1 10 2024-02-23T00:55:47.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.349+0000 7f02c8084700 20 client.31839.objecter get_session s=0x7f02c011ca70 osd=4 3 2024-02-23T00:55:47.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.349+0000 7f02c8084700 15 client.31839.objecter _session_command_op_assign 4 10 2024-02-23T00:55:47.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.349+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c011ca70 osd=4 4 2024-02-23T00:55:47.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.349+0000 7f02c8084700 10 client.31839.objecter _send_command 10 2024-02-23T00:55:47.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.349+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f02c011bf40 con 0x7f02c011cbe0 2024-02-23T00:55:47.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.352+0000 7f02b2ffd700 1 -- 172.21.15.47:0/776399190 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f02c011bf40 con 0x7f02c011cbe0 2024-02-23T00:55:47.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.352+0000 7f02b2ffd700 10 client.31839.objecter ms_dispatch 0x7f02c0001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:55:47.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.352+0000 7f02b2ffd700 10 client.31839.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:55:47.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.352+0000 7f02b2ffd700 20 client.31839.objecter put_session s=0x7f02c011ca70 osd=4 4 2024-02-23T00:55:47.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.352+0000 7f02b2ffd700 15 client.31839.objecter _session_command_op_remove 4 10 2024-02-23T00:55:47.354 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:55:47.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.352+0000 7f02c8084700 10 client.31839.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:47.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.352+0000 7f02c8084700 15 client.31839.objecter _session_command_op_assign -1 11 2024-02-23T00:55:47.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.352+0000 7f02c8084700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f02c0123d70 0x7f02c01241d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:47.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.352+0000 7f02c8084700 20 client.31839.objecter _get_session s=0x7f02c011bf40 osd=5 3 2024-02-23T00:55:47.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.352+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c011bf40 osd=5 3 2024-02-23T00:55:47.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.352+0000 7f02c8084700 20 client.31839.objecter _get_session s=0x7f02c011bf40 osd=5 3 2024-02-23T00:55:47.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.352+0000 7f02c8084700 15 client.31839.objecter _session_command_op_remove -1 11 2024-02-23T00:55:47.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.352+0000 7f02c8084700 20 client.31839.objecter get_session s=0x7f02c011bf40 osd=5 3 2024-02-23T00:55:47.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.352+0000 7f02c8084700 15 client.31839.objecter _session_command_op_assign 5 11 2024-02-23T00:55:47.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.352+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c011bf40 osd=5 4 2024-02-23T00:55:47.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.352+0000 7f02c8084700 10 client.31839.objecter _send_command 11 2024-02-23T00:55:47.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.352+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f02c012b240 con 0x7f02c0123d70 2024-02-23T00:55:47.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.352+0000 7f02c561f700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f02c0123d70 0x7f02c01241d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:47.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.352+0000 7f02b2ffd700 10 client.31839.objecter ms_handle_connect 0x7f02c0123d70 2024-02-23T00:55:47.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.352+0000 7f02c561f700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f02c0123d70 0x7f02c01241d0 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:47.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.353+0000 7f02b2ffd700 1 -- 172.21.15.47:0/776399190 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f02c012b240 con 0x7f02c0123d70 2024-02-23T00:55:47.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.353+0000 7f02b2ffd700 10 client.31839.objecter ms_dispatch 0x7f02c0001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:55:47.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.353+0000 7f02b2ffd700 10 client.31839.objecter _finish_command 11 = system:0 2024-02-23T00:55:47.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.353+0000 7f02b2ffd700 20 client.31839.objecter put_session s=0x7f02c011bf40 osd=5 4 2024-02-23T00:55:47.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.353+0000 7f02b2ffd700 15 client.31839.objecter _session_command_op_remove 5 11 2024-02-23T00:55:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.367+0000 7f02c8084700 10 client.31839.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:55:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.367+0000 7f02c8084700 15 client.31839.objecter _session_command_op_assign -1 12 2024-02-23T00:55:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.367+0000 7f02c8084700 20 client.31839.objecter _get_session s=0x7f02c011bf40 osd=5 3 2024-02-23T00:55:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.367+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c011bf40 osd=5 3 2024-02-23T00:55:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.367+0000 7f02c8084700 20 client.31839.objecter _get_session s=0x7f02c011bf40 osd=5 3 2024-02-23T00:55:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.367+0000 7f02c8084700 15 client.31839.objecter _session_command_op_remove -1 12 2024-02-23T00:55:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.367+0000 7f02c8084700 20 client.31839.objecter get_session s=0x7f02c011bf40 osd=5 3 2024-02-23T00:55:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.367+0000 7f02c8084700 15 client.31839.objecter _session_command_op_assign 5 12 2024-02-23T00:55:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.367+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c011bf40 osd=5 4 2024-02-23T00:55:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.367+0000 7f02c8084700 10 client.31839.objecter _send_command 12 2024-02-23T00:55:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.367+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f02c0128ae0 con 0x7f02c0123d70 2024-02-23T00:55:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.370+0000 7f02b2ffd700 1 -- 172.21.15.47:0/776399190 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f02c0128ae0 con 0x7f02c0123d70 2024-02-23T00:55:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.370+0000 7f02b2ffd700 10 client.31839.objecter ms_dispatch 0x7f02c0001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:55:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.370+0000 7f02b2ffd700 10 client.31839.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:55:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.370+0000 7f02b2ffd700 20 client.31839.objecter put_session s=0x7f02c011bf40 osd=5 4 2024-02-23T00:55:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.370+0000 7f02b2ffd700 15 client.31839.objecter _session_command_op_remove 5 12 2024-02-23T00:55:47.372 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:55:47.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.370+0000 7f02c8084700 10 client.31839.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:47.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.370+0000 7f02c8084700 15 client.31839.objecter _session_command_op_assign -1 13 2024-02-23T00:55:47.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.370+0000 7f02c8084700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f02c01291b0 0x7f02c012b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:47.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.370+0000 7f02c8084700 20 client.31839.objecter _get_session s=0x7f02c0128ae0 osd=6 3 2024-02-23T00:55:47.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.370+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c0128ae0 osd=6 3 2024-02-23T00:55:47.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.370+0000 7f02c8084700 20 client.31839.objecter _get_session s=0x7f02c0128ae0 osd=6 3 2024-02-23T00:55:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.370+0000 7f02c8084700 15 client.31839.objecter _session_command_op_remove -1 13 2024-02-23T00:55:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.370+0000 7f02c8084700 20 client.31839.objecter get_session s=0x7f02c0128ae0 osd=6 3 2024-02-23T00:55:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.370+0000 7f02c8084700 15 client.31839.objecter _session_command_op_assign 6 13 2024-02-23T00:55:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.370+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c0128ae0 osd=6 4 2024-02-23T00:55:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.370+0000 7f02c8084700 10 client.31839.objecter _send_command 13 2024-02-23T00:55:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.370+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f02c012e420 con 0x7f02c01291b0 2024-02-23T00:55:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.370+0000 7f02c6621700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f02c01291b0 0x7f02c012b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.370+0000 7f02b2ffd700 10 client.31839.objecter ms_handle_connect 0x7f02c01291b0 2024-02-23T00:55:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.370+0000 7f02c6621700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f02c01291b0 0x7f02c012b5e0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.372+0000 7f02b2ffd700 1 -- 172.21.15.47:0/776399190 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f02c012e420 con 0x7f02c01291b0 2024-02-23T00:55:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.372+0000 7f02b2ffd700 10 client.31839.objecter ms_dispatch 0x7f02c0001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:55:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.372+0000 7f02b2ffd700 10 client.31839.objecter _finish_command 13 = system:0 2024-02-23T00:55:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.372+0000 7f02b2ffd700 20 client.31839.objecter put_session s=0x7f02c0128ae0 osd=6 4 2024-02-23T00:55:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.372+0000 7f02b2ffd700 15 client.31839.objecter _session_command_op_remove 6 13 2024-02-23T00:55:47.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.385+0000 7f02c8084700 10 client.31839.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:55:47.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.385+0000 7f02c8084700 15 client.31839.objecter _session_command_op_assign -1 14 2024-02-23T00:55:47.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.385+0000 7f02c8084700 20 client.31839.objecter _get_session s=0x7f02c0128ae0 osd=6 3 2024-02-23T00:55:47.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.385+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c0128ae0 osd=6 3 2024-02-23T00:55:47.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.385+0000 7f02c8084700 20 client.31839.objecter _get_session s=0x7f02c0128ae0 osd=6 3 2024-02-23T00:55:47.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.385+0000 7f02c8084700 15 client.31839.objecter _session_command_op_remove -1 14 2024-02-23T00:55:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.385+0000 7f02c8084700 20 client.31839.objecter get_session s=0x7f02c0128ae0 osd=6 3 2024-02-23T00:55:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.385+0000 7f02c8084700 15 client.31839.objecter _session_command_op_assign 6 14 2024-02-23T00:55:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.385+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c0128ae0 osd=6 4 2024-02-23T00:55:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.385+0000 7f02c8084700 10 client.31839.objecter _send_command 14 2024-02-23T00:55:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.385+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f02c012d1f0 con 0x7f02c01291b0 2024-02-23T00:55:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.386+0000 7f02b2ffd700 1 -- 172.21.15.47:0/776399190 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f02c012d1f0 con 0x7f02c01291b0 2024-02-23T00:55:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.386+0000 7f02b2ffd700 10 client.31839.objecter ms_dispatch 0x7f02c0001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:55:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.386+0000 7f02b2ffd700 10 client.31839.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:55:47.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.386+0000 7f02b2ffd700 20 client.31839.objecter put_session s=0x7f02c0128ae0 osd=6 4 2024-02-23T00:55:47.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.386+0000 7f02b2ffd700 15 client.31839.objecter _session_command_op_remove 6 14 2024-02-23T00:55:47.390 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:55:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.387+0000 7f02c8084700 10 client.31839.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.387+0000 7f02c8084700 15 client.31839.objecter _session_command_op_assign -1 15 2024-02-23T00:55:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.387+0000 7f02c8084700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f02c012d8c0 0x7f02c012dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.387+0000 7f02c8084700 20 client.31839.objecter _get_session s=0x7f02c012d1f0 osd=7 3 2024-02-23T00:55:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.387+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c012d1f0 osd=7 3 2024-02-23T00:55:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.387+0000 7f02c8084700 20 client.31839.objecter _get_session s=0x7f02c012d1f0 osd=7 3 2024-02-23T00:55:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.387+0000 7f02c8084700 15 client.31839.objecter _session_command_op_remove -1 15 2024-02-23T00:55:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.387+0000 7f02c8084700 20 client.31839.objecter get_session s=0x7f02c012d1f0 osd=7 3 2024-02-23T00:55:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.387+0000 7f02c8084700 15 client.31839.objecter _session_command_op_assign 7 15 2024-02-23T00:55:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.387+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c012d1f0 osd=7 4 2024-02-23T00:55:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.387+0000 7f02c8084700 10 client.31839.objecter _send_command 15 2024-02-23T00:55:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.387+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f02c0131640 con 0x7f02c012d8c0 2024-02-23T00:55:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.387+0000 7f02c5e20700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f02c012d8c0 0x7f02c012dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.387+0000 7f02b2ffd700 10 client.31839.objecter ms_handle_connect 0x7f02c012d8c0 2024-02-23T00:55:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.387+0000 7f02c5e20700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f02c012d8c0 0x7f02c012dd40 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.388+0000 7f02b2ffd700 1 -- 172.21.15.47:0/776399190 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f02c0131640 con 0x7f02c012d8c0 2024-02-23T00:55:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.388+0000 7f02b2ffd700 10 client.31839.objecter ms_dispatch 0x7f02c0001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:55:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.388+0000 7f02b2ffd700 10 client.31839.objecter _finish_command 15 = system:0 2024-02-23T00:55:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.388+0000 7f02b2ffd700 20 client.31839.objecter put_session s=0x7f02c012d1f0 osd=7 4 2024-02-23T00:55:47.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.388+0000 7f02b2ffd700 15 client.31839.objecter _session_command_op_remove 7 15 2024-02-23T00:55:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.402+0000 7f02c8084700 10 client.31839.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:55:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.402+0000 7f02c8084700 15 client.31839.objecter _session_command_op_assign -1 16 2024-02-23T00:55:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.402+0000 7f02c8084700 20 client.31839.objecter _get_session s=0x7f02c012d1f0 osd=7 3 2024-02-23T00:55:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.402+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c012d1f0 osd=7 3 2024-02-23T00:55:47.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.402+0000 7f02c8084700 20 client.31839.objecter _get_session s=0x7f02c012d1f0 osd=7 3 2024-02-23T00:55:47.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.402+0000 7f02c8084700 15 client.31839.objecter _session_command_op_remove -1 16 2024-02-23T00:55:47.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.402+0000 7f02c8084700 20 client.31839.objecter get_session s=0x7f02c012d1f0 osd=7 3 2024-02-23T00:55:47.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.402+0000 7f02c8084700 15 client.31839.objecter _session_command_op_assign 7 16 2024-02-23T00:55:47.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.402+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c012d1f0 osd=7 4 2024-02-23T00:55:47.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.402+0000 7f02c8084700 10 client.31839.objecter _send_command 16 2024-02-23T00:55:47.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.402+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f02c0131640 con 0x7f02c012d8c0 2024-02-23T00:55:47.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02b2ffd700 1 -- 172.21.15.47:0/776399190 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f02c0131640 con 0x7f02c012d8c0 2024-02-23T00:55:47.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02b2ffd700 10 client.31839.objecter ms_dispatch 0x7f02c0001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:55:47.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02b2ffd700 10 client.31839.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:55:47.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02b2ffd700 20 client.31839.objecter put_session s=0x7f02c012d1f0 osd=7 4 2024-02-23T00:55:47.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02b2ffd700 15 client.31839.objecter _session_command_op_remove 7 16 2024-02-23T00:55:47.407 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:55:47.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 10 librados: watch_flush enter 2024-02-23T00:55:47.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 10 librados: watch_flush exit 2024-02-23T00:55:47.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 10 client.31839.objecter close_session for osd.0 2024-02-23T00:55:47.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f02c00ffed0 msgr2=0x7f02c01002b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:47.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f02c00ffed0 0x7f02c01002b0 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:47.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c005ee80 osd=0 1 2024-02-23T00:55:47.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 10 client.31839.objecter close_session for osd.1 2024-02-23T00:55:47.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f02c00598f0 msgr2=0x7f02c0059d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:47.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f02c00598f0 0x7f02c0059d30 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:47.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c0028710 osd=1 1 2024-02-23T00:55:47.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 10 client.31839.objecter close_session for osd.2 2024-02-23T00:55:47.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f02c0028270 msgr2=0x7f02c011c1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:47.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f02c0028270 0x7f02c011c1b0 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:47.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c0059710 osd=2 1 2024-02-23T00:55:47.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 10 client.31839.objecter close_session for osd.3 2024-02-23T00:55:47.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f02c011f910 msgr2=0x7f02c0121d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:47.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f02c011f910 0x7f02c0121d00 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:47.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c0027be0 osd=3 1 2024-02-23T00:55:47.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 10 client.31839.objecter close_session for osd.4 2024-02-23T00:55:47.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f02c011cbe0 msgr2=0x7f02c0123370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:47.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f02c011cbe0 0x7f02c0123370 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:47.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c011ca70 osd=4 1 2024-02-23T00:55:47.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 10 client.31839.objecter close_session for osd.5 2024-02-23T00:55:47.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f02c0123d70 msgr2=0x7f02c01241d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:47.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f02c0123d70 0x7f02c01241d0 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:47.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c6621700 1 -- 172.21.15.47:0/776399190 reap_dead start 2024-02-23T00:55:47.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c011bf40 osd=5 1 2024-02-23T00:55:47.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 10 client.31839.objecter close_session for osd.6 2024-02-23T00:55:47.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f02c01291b0 msgr2=0x7f02c012b5e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:47.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f02c01291b0 0x7f02c012b5e0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:47.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c6621700 1 -- 172.21.15.47:0/776399190 reap_dead start 2024-02-23T00:55:47.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c6621700 1 -- 172.21.15.47:0/776399190 reap_dead start 2024-02-23T00:55:47.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c6621700 1 -- 172.21.15.47:0/776399190 reap_dead start 2024-02-23T00:55:47.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c0128ae0 osd=6 1 2024-02-23T00:55:47.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 10 client.31839.objecter close_session for osd.7 2024-02-23T00:55:47.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f02c012d8c0 msgr2=0x7f02c012dd40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:47.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f02c012d8c0 0x7f02c012dd40 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:47.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 20 client.31839.objecter put_session s=0x7f02c012d1f0 osd=7 1 2024-02-23T00:55:47.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 20 client.31839.objecter shutdown clearing up homeless session... 2024-02-23T00:55:47.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.404+0000 7f02c8084700 10 client.31839.objecter successfully canceled tick 2024-02-23T00:55:47.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.405+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f02ac062cc0 msgr2=0x7f02ac065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:47.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.405+0000 7f02c8084700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f02ac062cc0 0x7f02ac065180 secure :-1 s=READY pgs=3211 cs=0 l=1 rev1=1 rx=0x7f02b4004ee0 tx=0x7f02b400b040).stop 2024-02-23T00:55:47.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.405+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02c00fcbb0 msgr2=0x7f02c00f8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:47.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.405+0000 7f02c8084700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02c00fcbb0 0x7f02c00f8ea0 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 rx=0x7f02bc004920 tx=0x7f02bc009cb0).stop 2024-02-23T00:55:47.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.405+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 shutdown_connections 2024-02-23T00:55:47.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.405+0000 7f02c8084700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f02c01291b0 0x7f02c012b5e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:47.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.405+0000 7f02c8084700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02c00fcbb0 0x7f02c00f8ea0 unknown :-1 s=CLOSED pgs=2646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:47.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.405+0000 7f02c8084700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f02ac062cc0 0x7f02ac065180 unknown :-1 s=CLOSED pgs=3211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:47.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.405+0000 7f02c8084700 1 --2- 172.21.15.47:0/776399190 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f02c012d8c0 0x7f02c012dd40 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:47.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.405+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 >> 172.21.15.47:0/776399190 conn(0x7f02c00f2820 msgr2=0x7f02c0060e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:47.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.405+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 shutdown_connections 2024-02-23T00:55:47.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.405+0000 7f02c8084700 1 -- 172.21.15.47:0/776399190 wait complete. 2024-02-23T00:55:47.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:47.405+0000 7f02c8084700 1 librados: shutdown 2024-02-23T00:55:47.415 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:55:47.415 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:55:47.415 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:55:47.415 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:55:47.415 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:55:47.415 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:55:47.415 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:55:47.415 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:55:47.587 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:55:47.587 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-02-23T00:55:47.587 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-02-23T00:55:47.587 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-02-23T00:55:47.587 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.588 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.15270.0:6372 62.1c 62:38da857d:test-rados-api-smithi047-82465-66::foo:head [writefull 0~10] snapc 0=[] ondisk+write+known_if_redirected e445)", 2024-02-23T00:55:47.588 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:49:03.384798+0000", 2024-02-23T00:55:47.588 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 404.197712653, 2024-02-23T00:55:47.588 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012165509, 2024-02-23T00:55:47.588 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:47.588 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:55:47.588 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:55:47.588 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.15270", 2024-02-23T00:55:47.588 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/4133111990", 2024-02-23T00:55:47.588 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 6372 2024-02-23T00:55:47.588 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.588 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:47.589 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.589 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:47.589 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:49:03.384798+0000", 2024-02-23T00:55:47.589 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:47.589 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.589 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.589 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:47.589 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:49:03.384798+0000", 2024-02-23T00:55:47.589 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:55:47.589 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.589 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.589 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:47.589 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:49:03.384796+0000", 2024-02-23T00:55:47.589 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0910000000000004e-06 2024-02-23T00:55:47.590 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.590 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.590 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:47.590 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:49:03.384803+0000", 2024-02-23T00:55:47.590 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00053249099999999995 2024-02-23T00:55:47.590 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.590 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.590 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:47.590 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:49:03.385336+0000", 2024-02-23T00:55:47.590 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.287e-05 2024-02-23T00:55:47.590 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.590 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.590 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:47.590 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:49:03.385349+0000", 2024-02-23T00:55:47.590 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076659679999999996 2024-02-23T00:55:47.591 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.591 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.591 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.591 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:49:03.393014+0000", 2024-02-23T00:55:47.591 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0001537 2024-02-23T00:55:47.591 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.591 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.591 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:47.591 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:49:03.393168+0000", 2024-02-23T00:55:47.591 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000191459 2024-02-23T00:55:47.591 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.591 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.591 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0,5", 2024-02-23T00:55:47.591 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:49:03.393360+0000", 2024-02-23T00:55:47.592 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00174793 2024-02-23T00:55:47.592 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.592 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.592 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:55:47.592 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:49:03.395108+0000", 2024-02-23T00:55:47.592 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9806999999999999e-05 2024-02-23T00:55:47.592 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.592 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.592 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:55:47.592 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:49:03.395127+0000", 2024-02-23T00:55:47.592 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001744174 2024-02-23T00:55:47.592 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.592 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.593 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:55:47.593 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:49:03.396872+0000", 2024-02-23T00:55:47.593 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0680000000000003e-05 2024-02-23T00:55:47.593 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.594 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.594 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:55:47.594 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:49:03.396922+0000", 2024-02-23T00:55:47.594 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0908999999999997e-05 2024-02-23T00:55:47.594 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.594 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.594 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:47.594 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:49:03.396963+0000", 2024-02-23T00:55:47.595 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012165509 2024-02-23T00:55:47.595 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.595 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:47.595 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.595 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.595 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.595 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.15270.0:7117 220.4 220:251f54bc:test-rados-api-smithi047-82465-78::foo-chunk:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected e494)", 2024-02-23T00:55:47.595 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:50:23.246166+0000", 2024-02-23T00:55:47.595 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 324.33634402600001, 2024-02-23T00:55:47.595 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078444889999999996, 2024-02-23T00:55:47.595 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:47.595 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:55:47.595 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:55:47.596 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.15270", 2024-02-23T00:55:47.600 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/4133111990", 2024-02-23T00:55:47.600 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 7117 2024-02-23T00:55:47.600 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.600 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:47.600 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.600 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:47.600 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:50:23.246166+0000", 2024-02-23T00:55:47.600 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:47.600 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.600 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.601 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:47.601 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:50:23.246166+0000", 2024-02-23T00:55:47.601 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999995 2024-02-23T00:55:47.601 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.601 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.601 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:47.601 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:50:23.246166+0000", 2024-02-23T00:55:47.601 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.3749999999999999e-06 2024-02-23T00:55:47.601 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.601 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.601 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:47.601 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:50:23.246169+0000", 2024-02-23T00:55:47.601 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.331e-06 2024-02-23T00:55:47.602 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.602 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.602 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:47.602 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:50:23.246175+0000", 2024-02-23T00:55:47.602 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7500000000000001e-06 2024-02-23T00:55:47.602 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.602 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.602 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:47.602 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:50:23.246179+0000", 2024-02-23T00:55:47.602 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1270000000000001e-05 2024-02-23T00:55:47.602 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.602 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.602 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.603 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:50:23.246200+0000", 2024-02-23T00:55:47.603 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1161999999999995e-05 2024-02-23T00:55:47.603 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.603 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.603 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:47.603 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:50:23.246261+0000", 2024-02-23T00:55:47.603 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0077492680000000001 2024-02-23T00:55:47.603 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.603 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.603 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:47.603 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:50:23.254011+0000", 2024-02-23T00:55:47.603 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078444889999999996 2024-02-23T00:55:47.603 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.603 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:47.604 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.604 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.604 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.604 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "MOSDECSubOpWrite(234.3s2 661/600 ECSubWrite(tid=519, reqid=client.41771.0:791, at_version=661'1, trim_to=0'0, roll_forward_to=0'0))", 2024-02-23T00:55:47.604 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:53:18.291212+0000", 2024-02-23T00:55:47.604 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 149.291298066, 2024-02-23T00:55:47.604 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083852100000000006, 2024-02-23T00:55:47.604 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:47.604 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "reached pg", 2024-02-23T00:55:47.604 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:47.604 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.604 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:47.604 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:18.291212+0000", 2024-02-23T00:55:47.605 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:47.605 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.605 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.605 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:47.605 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:18.291212+0000", 2024-02-23T00:55:47.605 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967296 2024-02-23T00:55:47.605 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.605 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.605 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:47.605 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:18.291212+0000", 2024-02-23T00:55:47.605 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.4376999999999999e-05 2024-02-23T00:55:47.606 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.606 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.606 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:47.606 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:18.291256+0000", 2024-02-23T00:55:47.606 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0507e-05 2024-02-23T00:55:47.606 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.606 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.606 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:47.606 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:18.291267+0000", 2024-02-23T00:55:47.606 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.9429999999999995e-06 2024-02-23T00:55:47.606 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.606 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.606 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:47.607 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:18.291277+0000", 2024-02-23T00:55:47.607 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0012096120000000001 2024-02-23T00:55:47.607 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.607 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.607 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.607 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:18.292486+0000", 2024-02-23T00:55:47.607 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0669999999999999e-05 2024-02-23T00:55:47.607 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.607 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.607 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:55:47.607 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:18.292497+0000", 2024-02-23T00:55:47.607 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0070804809999999996 2024-02-23T00:55:47.608 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.608 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.608 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:55:47.608 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:18.299578+0000", 2024-02-23T00:55:47.608 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9826e-05 2024-02-23T00:55:47.608 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.608 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.608 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:47.608 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:18.299597+0000", 2024-02-23T00:55:47.608 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083852100000000006 2024-02-23T00:55:47.608 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.608 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:47.608 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.609 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.609 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.609 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.41771.0:801 241.10 241:0d856113:test-rados-api-smithi047-82465-115::bam:head [read 0~1] snapc 0=[] ondisk+read+known_if_redirected e665)", 2024-02-23T00:55:47.609 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:53:22.315769+0000", 2024-02-23T00:55:47.609 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 145.26674153900001, 2024-02-23T00:55:47.609 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0046614559999999996, 2024-02-23T00:55:47.609 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:47.609 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:55:47.609 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:55:47.609 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.41771", 2024-02-23T00:55:47.609 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1032768195", 2024-02-23T00:55:47.609 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 801 2024-02-23T00:55:47.609 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.609 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:47.610 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.610 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:47.610 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.315769+0000", 2024-02-23T00:55:47.610 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:47.610 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.610 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.610 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:47.610 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.315769+0000", 2024-02-23T00:55:47.610 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999976 2024-02-23T00:55:47.610 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.610 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.610 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:47.610 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.315766+0000", 2024-02-23T00:55:47.610 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6509999999999996e-06 2024-02-23T00:55:47.610 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.611 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.611 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:47.611 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.315776+0000", 2024-02-23T00:55:47.611 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6694e-05 2024-02-23T00:55:47.611 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.611 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.611 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:47.611 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.315793+0000", 2024-02-23T00:55:47.611 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1029999999999998e-06 2024-02-23T00:55:47.611 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.611 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.611 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:47.611 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.315799+0000", 2024-02-23T00:55:47.612 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5592e-05 2024-02-23T00:55:47.612 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.612 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.612 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.612 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.315845+0000", 2024-02-23T00:55:47.612 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.566e-06 2024-02-23T00:55:47.612 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.612 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.612 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op must wait for map", 2024-02-23T00:55:47.612 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.315849+0000", 2024-02-23T00:55:47.612 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.002031939 2024-02-23T00:55:47.612 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.612 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.612 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.613 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.317881+0000", 2024-02-23T00:55:47.613 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016271600000000001 2024-02-23T00:55:47.613 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.613 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.613 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for blocked object", 2024-02-23T00:55:47.613 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.318044+0000", 2024-02-23T00:55:47.613 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00130802 2024-02-23T00:55:47.613 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.613 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.613 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.613 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.319352+0000", 2024-02-23T00:55:47.613 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7079000000000001e-05 2024-02-23T00:55:47.613 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.613 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.614 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for rw locks", 2024-02-23T00:55:47.614 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.319399+0000", 2024-02-23T00:55:47.614 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00043572700000000001 2024-02-23T00:55:47.614 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.614 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.614 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.614 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.319835+0000", 2024-02-23T00:55:47.614 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1360000000000003e-05 2024-02-23T00:55:47.614 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.614 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.614 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:47.614 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.319886+0000", 2024-02-23T00:55:47.614 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00054426099999999999 2024-02-23T00:55:47.614 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.614 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.615 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:47.615 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:22.320430+0000", 2024-02-23T00:55:47.615 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0046614559999999996 2024-02-23T00:55:47.615 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.615 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:47.615 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.615 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.615 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.615 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "MOSDECSubOpWrite(234.7s2 666/600 ECSubWrite(tid=831, reqid=osd.7.0:831, at_version=666'13, trim_to=0'0, roll_forward_to=662'11))", 2024-02-23T00:55:47.615 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:53:23.326198+0000", 2024-02-23T00:55:47.615 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 144.25631190999999, 2024-02-23T00:55:47.615 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0027443390000000002, 2024-02-23T00:55:47.615 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:47.615 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "reached pg", 2024-02-23T00:55:47.616 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:47.616 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.616 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:47.616 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.326198+0000", 2024-02-23T00:55:47.616 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:47.616 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.616 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.616 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:47.616 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.326198+0000", 2024-02-23T00:55:47.616 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999995 2024-02-23T00:55:47.616 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.616 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.616 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:47.617 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.326198+0000", 2024-02-23T00:55:47.617 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.8738000000000002e-05 2024-02-23T00:55:47.617 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.617 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.617 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:47.617 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.326227+0000", 2024-02-23T00:55:47.617 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3820000000000004e-06 2024-02-23T00:55:47.617 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.617 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.617 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:47.617 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.326235+0000", 2024-02-23T00:55:47.617 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1489999999999997e-06 2024-02-23T00:55:47.618 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.618 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.618 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:47.618 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.326242+0000", 2024-02-23T00:55:47.618 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000148861 2024-02-23T00:55:47.618 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.618 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.618 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.618 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.326391+0000", 2024-02-23T00:55:47.618 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1141999999999999e-05 2024-02-23T00:55:47.618 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.618 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.618 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:55:47.618 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.326402+0000", 2024-02-23T00:55:47.619 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0025200700000000001 2024-02-23T00:55:47.619 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.619 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.619 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:55:47.619 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.328922+0000", 2024-02-23T00:55:47.619 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0318000000000001e-05 2024-02-23T00:55:47.619 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.619 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.619 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:47.619 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.328943+0000", 2024-02-23T00:55:47.619 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0027443390000000002 2024-02-23T00:55:47.619 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.619 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:47.620 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.620 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.620 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.620 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "MOSDECSubOpWrite(234.0s1 666/600 ECSubWrite(tid=849, reqid=osd.4.0:849, at_version=666'5, trim_to=0'0, roll_forward_to=662'3))", 2024-02-23T00:55:47.620 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:53:23.329733+0000", 2024-02-23T00:55:47.620 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 144.25277722800001, 2024-02-23T00:55:47.620 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0036790780000000001, 2024-02-23T00:55:47.620 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:47.620 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "reached pg", 2024-02-23T00:55:47.620 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:47.620 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.620 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:47.620 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.329733+0000", 2024-02-23T00:55:47.621 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:47.621 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.621 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.621 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:47.621 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.329733+0000", 2024-02-23T00:55:47.621 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999995 2024-02-23T00:55:47.621 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.621 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.621 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:47.621 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.329733+0000", 2024-02-23T00:55:47.621 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1026000000000002e-05 2024-02-23T00:55:47.621 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.621 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.621 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:47.622 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.329754+0000", 2024-02-23T00:55:47.622 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.4950000000000008e-06 2024-02-23T00:55:47.622 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.622 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.622 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:47.622 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.329762+0000", 2024-02-23T00:55:47.622 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6229999999999999e-06 2024-02-23T00:55:47.622 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.622 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.622 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:47.622 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.329769+0000", 2024-02-23T00:55:47.622 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0023481890000000001 2024-02-23T00:55:47.622 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.622 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.623 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.623 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.332117+0000", 2024-02-23T00:55:47.623 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.8390000000000007e-06 2024-02-23T00:55:47.623 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.623 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.623 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:55:47.623 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.332125+0000", 2024-02-23T00:55:47.623 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0012570249999999999 2024-02-23T00:55:47.623 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.623 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.623 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:55:47.623 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.333382+0000", 2024-02-23T00:55:47.623 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.0125999999999999e-05 2024-02-23T00:55:47.623 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.624 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.624 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:47.624 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:23.333412+0000", 2024-02-23T00:55:47.624 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0036790780000000001 2024-02-23T00:55:47.624 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.624 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:47.624 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.624 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.624 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.624 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.41771.0:825 241.a 241:50000000:%01:::head [pgnls start_epoch 667] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e667)", 2024-02-23T00:55:47.624 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:53:24.348607+0000", 2024-02-23T00:55:47.624 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 143.233903094, 2024-02-23T00:55:47.624 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0041364720000000004, 2024-02-23T00:55:47.624 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:47.625 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:55:47.625 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:55:47.625 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.41771", 2024-02-23T00:55:47.625 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1032768195", 2024-02-23T00:55:47.625 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 825 2024-02-23T00:55:47.625 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.625 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:47.625 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.625 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:47.625 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:24.348607+0000", 2024-02-23T00:55:47.625 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:47.625 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.626 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.626 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:47.626 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:24.348607+0000", 2024-02-23T00:55:47.626 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:55:47.626 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.626 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.626 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:47.626 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:24.348606+0000", 2024-02-23T00:55:47.626 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3049999999999998e-06 2024-02-23T00:55:47.626 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.626 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.626 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:47.626 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:24.348612+0000", 2024-02-23T00:55:47.626 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0849999999999999e-05 2024-02-23T00:55:47.627 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.627 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.627 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:47.627 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:24.348623+0000", 2024-02-23T00:55:47.627 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4180000000000002e-06 2024-02-23T00:55:47.627 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.627 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.627 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:47.627 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:24.348629+0000", 2024-02-23T00:55:47.627 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0040071960000000002 2024-02-23T00:55:47.627 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.627 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.627 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.628 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:24.352637+0000", 2024-02-23T00:55:47.628 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.1749999999999999e-05 2024-02-23T00:55:47.628 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.628 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.628 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:47.628 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:24.352668+0000", 2024-02-23T00:55:47.628 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5253999999999995e-05 2024-02-23T00:55:47.628 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.628 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.628 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:47.628 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:24.352744+0000", 2024-02-23T00:55:47.628 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0041364720000000004 2024-02-23T00:55:47.628 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.629 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:47.629 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.629 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.629 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.629 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "MOSDECSubOpWrite(234.3s2 667/666 ECSubWrite(tid=531, reqid=osd.5.0:531, at_version=667'5, trim_to=0'0, roll_forward_to=0'0))", 2024-02-23T00:55:47.629 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:53:24.427454+0000", 2024-02-23T00:55:47.629 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 143.155056675, 2024-02-23T00:55:47.629 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0036459090000000001, 2024-02-23T00:55:47.629 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:47.629 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "reached pg", 2024-02-23T00:55:47.629 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:47.629 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.629 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:47.629 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:24.427454+0000", 2024-02-23T00:55:47.630 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:47.630 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.630 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.630 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:47.630 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:24.427454+0000", 2024-02-23T00:55:47.630 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999995 2024-02-23T00:55:47.630 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.630 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.630 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:47.630 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:24.427453+0000", 2024-02-23T00:55:47.630 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0168000000000001e-05 2024-02-23T00:55:47.630 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.630 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.630 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:47.631 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:24.427473+0000", 2024-02-23T00:55:47.631 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1358e-05 2024-02-23T00:55:47.631 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.631 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.631 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:47.631 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:24.427485+0000", 2024-02-23T00:55:47.631 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.311e-06 2024-02-23T00:55:47.631 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.631 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.631 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:47.631 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:24.427492+0000", 2024-02-23T00:55:47.631 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014094800000000001 2024-02-23T00:55:47.631 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.631 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.632 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.632 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:24.427633+0000", 2024-02-23T00:55:47.632 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0327999999999999e-05 2024-02-23T00:55:47.632 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.632 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.632 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:55:47.632 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:24.427653+0000", 2024-02-23T00:55:47.632 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.003423148 2024-02-23T00:55:47.632 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.632 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.632 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:55:47.632 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:24.431076+0000", 2024-02-23T00:55:47.632 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.3167e-05 2024-02-23T00:55:47.632 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.633 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.633 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:47.633 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:24.431100+0000", 2024-02-23T00:55:47.633 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0036459090000000001 2024-02-23T00:55:47.633 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.633 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:47.633 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.633 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.633 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.633 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.41771.0:915 242.1 242:817d5464:test-rados-api-smithi047-82465-117::foo:head [writefull 0~8] snapc 0=[] ondisk+write+known_if_redirected e675)", 2024-02-23T00:55:47.633 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:53:32.289744+0000", 2024-02-23T00:55:47.633 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 135.29276578899999, 2024-02-23T00:55:47.633 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0042818709999999996, 2024-02-23T00:55:47.634 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:47.634 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:55:47.634 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:55:47.634 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.41771", 2024-02-23T00:55:47.634 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1032768195", 2024-02-23T00:55:47.634 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 915 2024-02-23T00:55:47.634 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.634 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:47.634 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.634 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:47.634 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.289744+0000", 2024-02-23T00:55:47.634 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:47.634 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.634 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.635 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:47.635 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.289744+0000", 2024-02-23T00:55:47.635 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:55:47.635 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.635 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.635 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:47.635 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.289742+0000", 2024-02-23T00:55:47.635 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1698999999999999e-05 2024-02-23T00:55:47.635 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.635 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.635 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:47.635 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.289754+0000", 2024-02-23T00:55:47.635 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7660999999999999e-05 2024-02-23T00:55:47.636 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.636 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.636 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:47.636 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.289772+0000", 2024-02-23T00:55:47.636 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2080000000000003e-06 2024-02-23T00:55:47.636 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.636 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.636 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:47.636 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.289779+0000", 2024-02-23T00:55:47.636 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0646e-05 2024-02-23T00:55:47.636 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.636 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.636 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.637 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.289860+0000", 2024-02-23T00:55:47.637 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8239999999999999e-06 2024-02-23T00:55:47.637 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.637 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.637 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op must wait for map", 2024-02-23T00:55:47.637 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.289864+0000", 2024-02-23T00:55:47.637 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0017831399999999999 2024-02-23T00:55:47.637 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.637 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.637 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.637 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.291648+0000", 2024-02-23T00:55:47.637 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022167399999999999 2024-02-23T00:55:47.637 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.637 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.638 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for blocked object", 2024-02-23T00:55:47.638 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.291869+0000", 2024-02-23T00:55:47.638 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001224611 2024-02-23T00:55:47.638 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.638 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.638 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.638 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.293094+0000", 2024-02-23T00:55:47.638 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2120000000000007e-05 2024-02-23T00:55:47.638 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.638 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.638 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:47.639 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.293176+0000", 2024-02-23T00:55:47.639 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014774800000000001 2024-02-23T00:55:47.639 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.639 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.639 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1,7", 2024-02-23T00:55:47.639 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.293324+0000", 2024-02-23T00:55:47.639 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00034650300000000001 2024-02-23T00:55:47.639 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.639 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.639 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-02-23T00:55:47.639 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.293670+0000", 2024-02-23T00:55:47.639 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00020553300000000001 2024-02-23T00:55:47.640 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.640 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.640 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:55:47.640 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.293876+0000", 2024-02-23T00:55:47.640 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.8220999999999994e-05 2024-02-23T00:55:47.640 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.640 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.640 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-02-23T00:55:47.640 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.293964+0000", 2024-02-23T00:55:47.640 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.9867999999999999e-05 2024-02-23T00:55:47.640 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.641 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.641 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:55:47.641 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.293994+0000", 2024-02-23T00:55:47.641 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.2527999999999998e-05 2024-02-23T00:55:47.641 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.641 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.641 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:47.641 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.294026+0000", 2024-02-23T00:55:47.641 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0042818709999999996 2024-02-23T00:55:47.641 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.641 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:47.641 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.641 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.641 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.641 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.41771.0:960 242.1 242:817d5464:test-rados-api-smithi047-82465-117::foo:head [cache-try-flush] snapc 0=[] ondisk+read+skiprwlocks+ignore_overlay+known_if_redirected e675)", 2024-02-23T00:55:47.642 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:53:32.314444+0000", 2024-02-23T00:55:47.642 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 135.26806599, 2024-02-23T00:55:47.642 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0032990509999999999, 2024-02-23T00:55:47.642 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:47.642 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:55:47.642 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:55:47.642 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.41771", 2024-02-23T00:55:47.642 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1032768195", 2024-02-23T00:55:47.642 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 960 2024-02-23T00:55:47.642 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.643 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:47.643 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.643 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:47.643 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.314444+0000", 2024-02-23T00:55:47.643 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:47.643 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.643 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.643 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:47.643 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.314444+0000", 2024-02-23T00:55:47.643 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:55:47.643 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.643 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.644 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:47.644 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.314443+0000", 2024-02-23T00:55:47.644 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7299999999999996e-06 2024-02-23T00:55:47.644 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.644 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.644 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:47.644 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.314448+0000", 2024-02-23T00:55:47.644 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0988999999999999e-05 2024-02-23T00:55:47.644 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.644 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.644 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:47.644 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.314459+0000", 2024-02-23T00:55:47.644 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.113e-06 2024-02-23T00:55:47.645 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.645 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.645 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:47.645 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.314464+0000", 2024-02-23T00:55:47.645 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0613999999999999e-05 2024-02-23T00:55:47.645 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.645 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.645 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.645 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.314485+0000", 2024-02-23T00:55:47.645 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6028e-05 2024-02-23T00:55:47.645 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.645 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.645 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:47.646 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.314541+0000", 2024-02-23T00:55:47.646 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0030165959999999999 2024-02-23T00:55:47.646 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.646 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.646 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.646 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.317558+0000", 2024-02-23T00:55:47.646 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010406100000000001 2024-02-23T00:55:47.646 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.646 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.646 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:47.646 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.317662+0000", 2024-02-23T00:55:47.646 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1768999999999996e-05 2024-02-23T00:55:47.646 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.646 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.647 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:47.647 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:32.317743+0000", 2024-02-23T00:55:47.647 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0032990509999999999 2024-02-23T00:55:47.647 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.647 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:47.647 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.647 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.647 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.647 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "MOSDScrubReserve(5.18 RELEASE e678)", 2024-02-23T00:55:47.647 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:53:35.476785+0000", 2024-02-23T00:55:47.647 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 132.10572544199999, 2024-02-23T00:55:47.647 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0044284700000000003, 2024-02-23T00:55:47.647 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:47.647 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:55:47.647 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:47.648 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.648 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:47.648 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:35.476785+0000", 2024-02-23T00:55:47.648 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:47.648 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.648 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.648 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:47.648 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:35.476785+0000", 2024-02-23T00:55:47.648 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999995 2024-02-23T00:55:47.648 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.648 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.648 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:47.648 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:35.476784+0000", 2024-02-23T00:55:47.648 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.7429999999999999e-06 2024-02-23T00:55:47.649 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.649 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.649 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:47.649 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:35.476790+0000", 2024-02-23T00:55:47.649 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.9129999999999998e-06 2024-02-23T00:55:47.649 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.649 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.649 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:47.649 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:35.476800+0000", 2024-02-23T00:55:47.649 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.7799999999999997e-06 2024-02-23T00:55:47.649 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.649 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.649 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:47.650 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:35.476806+0000", 2024-02-23T00:55:47.650 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.7018000000000002e-05 2024-02-23T00:55:47.650 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.650 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.650 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.650 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:35.476863+0000", 2024-02-23T00:55:47.650 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5769999999999999e-06 2024-02-23T00:55:47.650 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.650 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.650 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op must wait for map", 2024-02-23T00:55:47.650 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:35.476869+0000", 2024-02-23T00:55:47.650 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00432249 2024-02-23T00:55:47.650 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.650 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.651 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.651 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:35.481192+0000", 2024-02-23T00:55:47.651 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.4530000000000002e-06 2024-02-23T00:55:47.651 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.651 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.651 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:47.651 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:35.481200+0000", 2024-02-23T00:55:47.651 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3208e-05 2024-02-23T00:55:47.651 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.651 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.651 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:47.651 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:53:35.481213+0000", 2024-02-23T00:55:47.651 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0044284700000000003 2024-02-23T00:55:47.651 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.652 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:47.652 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.652 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.652 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.652 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.41771.0:1571 245.c 245:325ee4a0:test-rados-api-smithi047-82465-123::foo:8 [cache-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected e704)", 2024-02-23T00:55:47.652 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:54:03.368317+0000", 2024-02-23T00:55:47.652 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 104.214193666, 2024-02-23T00:55:47.652 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0030795559999999998, 2024-02-23T00:55:47.652 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:47.652 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:55:47.652 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:55:47.652 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.41771", 2024-02-23T00:55:47.652 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1032768195", 2024-02-23T00:55:47.652 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1571 2024-02-23T00:55:47.653 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.653 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:47.653 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.653 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:47.653 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.368317+0000", 2024-02-23T00:55:47.653 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:47.653 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.653 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.653 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:47.653 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.368317+0000", 2024-02-23T00:55:47.653 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.999999 2024-02-23T00:55:47.653 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.653 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.654 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:47.654 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.368316+0000", 2024-02-23T00:55:47.654 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.8510000000000001e-06 2024-02-23T00:55:47.654 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.654 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.654 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:47.654 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.368320+0000", 2024-02-23T00:55:47.654 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2599999999999999e-06 2024-02-23T00:55:47.654 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.654 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.654 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:47.654 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.368327+0000", 2024-02-23T00:55:47.654 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.49e-06 2024-02-23T00:55:47.654 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.654 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.655 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:47.655 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.368330+0000", 2024-02-23T00:55:47.655 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.0657000000000001e-05 2024-02-23T00:55:47.655 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.655 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.655 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.655 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.368361+0000", 2024-02-23T00:55:47.655 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0950000000000003e-05 2024-02-23T00:55:47.655 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.655 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.655 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:47.655 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.368442+0000", 2024-02-23T00:55:47.655 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0028182849999999998 2024-02-23T00:55:47.656 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.656 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.656 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.656 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.371260+0000", 2024-02-23T00:55:47.656 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1074000000000003e-05 2024-02-23T00:55:47.656 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.656 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.656 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:47.656 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.371341+0000", 2024-02-23T00:55:47.656 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4864000000000002e-05 2024-02-23T00:55:47.656 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.656 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.656 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:47.657 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.371396+0000", 2024-02-23T00:55:47.657 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0030795559999999998 2024-02-23T00:55:47.657 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.657 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:47.657 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.657 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.657 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.657 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.41771.0:1573 245.c 245:325ee4a0:test-rados-api-smithi047-82465-123::foo:head [cache-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected e704)", 2024-02-23T00:55:47.657 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:54:03.374437+0000", 2024-02-23T00:55:47.657 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 104.208073023, 2024-02-23T00:55:47.657 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0026872829999999999, 2024-02-23T00:55:47.657 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:47.657 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:55:47.657 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:55:47.658 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.41771", 2024-02-23T00:55:47.658 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1032768195", 2024-02-23T00:55:47.658 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1573 2024-02-23T00:55:47.658 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.658 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:47.658 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.658 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:47.658 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.374437+0000", 2024-02-23T00:55:47.658 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:47.658 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.658 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.658 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:47.658 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.374437+0000", 2024-02-23T00:55:47.658 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:55:47.658 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.659 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.659 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:47.659 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.374436+0000", 2024-02-23T00:55:47.659 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1550000000000004e-06 2024-02-23T00:55:47.659 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.659 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.659 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:47.659 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.374442+0000", 2024-02-23T00:55:47.659 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.9890000000000004e-06 2024-02-23T00:55:47.659 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.659 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.659 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:47.659 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.374452+0000", 2024-02-23T00:55:47.659 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.2589999999999997e-06 2024-02-23T00:55:47.660 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.660 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.660 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:47.660 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.374458+0000", 2024-02-23T00:55:47.660 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.955e-05 2024-02-23T00:55:47.660 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.660 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.660 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.660 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.374488+0000", 2024-02-23T00:55:47.660 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8188999999999994e-05 2024-02-23T00:55:47.660 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.660 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.660 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:47.661 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.374556+0000", 2024-02-23T00:55:47.661 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0023959739999999999 2024-02-23T00:55:47.661 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.661 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.661 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.661 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.376952+0000", 2024-02-23T00:55:47.661 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000102631 2024-02-23T00:55:47.661 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.661 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.661 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:47.661 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.377055+0000", 2024-02-23T00:55:47.661 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.9809999999999997e-05 2024-02-23T00:55:47.661 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.661 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.662 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:47.662 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:03.377125+0000", 2024-02-23T00:55:47.662 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0026872829999999999 2024-02-23T00:55:47.662 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.662 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:47.662 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.662 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.662 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.662 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.41771.0:1581 245.0 245:00000000:%01:::head [pgnls start_epoch 707] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e707)", 2024-02-23T00:55:47.662 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:54:06.292972+0000", 2024-02-23T00:55:47.662 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 101.289538288, 2024-02-23T00:55:47.662 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.019065069, 2024-02-23T00:55:47.662 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:47.662 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:55:47.663 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:55:47.663 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.41771", 2024-02-23T00:55:47.663 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1032768195", 2024-02-23T00:55:47.663 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1581 2024-02-23T00:55:47.663 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.663 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:47.663 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.663 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:47.663 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:06.292972+0000", 2024-02-23T00:55:47.663 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:47.663 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.663 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.663 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:47.664 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:06.292972+0000", 2024-02-23T00:55:47.664 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999962 2024-02-23T00:55:47.664 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.664 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.664 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:47.664 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:06.292968+0000", 2024-02-23T00:55:47.664 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9758999999999998e-05 2024-02-23T00:55:47.664 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.664 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.664 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:47.664 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:06.292988+0000", 2024-02-23T00:55:47.665 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7509000000000003e-05 2024-02-23T00:55:47.665 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.665 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.665 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:47.665 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:06.293026+0000", 2024-02-23T00:55:47.665 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.2595999999999998e-05 2024-02-23T00:55:47.665 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.665 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.665 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:47.665 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:06.293058+0000", 2024-02-23T00:55:47.665 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8628000000000005e-05 2024-02-23T00:55:47.665 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.665 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.665 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.666 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:06.293157+0000", 2024-02-23T00:55:47.666 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1365999999999999e-05 2024-02-23T00:55:47.666 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.666 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.666 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op must wait for map", 2024-02-23T00:55:47.666 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:06.293168+0000", 2024-02-23T00:55:47.666 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.018633215000000002 2024-02-23T00:55:47.666 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.666 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.666 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.666 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:06.311801+0000", 2024-02-23T00:55:47.666 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1588999999999998e-05 2024-02-23T00:55:47.666 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.667 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.667 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:47.667 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:06.311843+0000", 2024-02-23T00:55:47.667 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00019413000000000001 2024-02-23T00:55:47.667 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.667 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.667 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:47.667 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:06.312037+0000", 2024-02-23T00:55:47.667 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.019065069 2024-02-23T00:55:47.667 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.667 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:47.667 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.667 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.667 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.668 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.41771.0:1615 245.0 245:00000000::::head [pgnls start_epoch 709] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e709)", 2024-02-23T00:55:47.668 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:54:08.295720+0000", 2024-02-23T00:55:47.668 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 99.286789917999997, 2024-02-23T00:55:47.668 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0030963840000000002, 2024-02-23T00:55:47.668 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:47.668 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:55:47.668 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:55:47.668 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.41771", 2024-02-23T00:55:47.668 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1032768195", 2024-02-23T00:55:47.668 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1615 2024-02-23T00:55:47.668 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.668 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:47.668 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.668 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:47.669 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:08.295720+0000", 2024-02-23T00:55:47.669 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:47.669 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.669 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.669 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:47.669 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:08.295720+0000", 2024-02-23T00:55:47.669 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999971 2024-02-23T00:55:47.669 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.669 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.669 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:47.669 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:08.295717+0000", 2024-02-23T00:55:47.669 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4161999999999999e-05 2024-02-23T00:55:47.669 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.669 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.670 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:47.670 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:08.295732+0000", 2024-02-23T00:55:47.670 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2416999999999999e-05 2024-02-23T00:55:47.670 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.670 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.670 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:47.670 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:08.295754+0000", 2024-02-23T00:55:47.670 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0933e-05 2024-02-23T00:55:47.670 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.670 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.670 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:47.670 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:08.295765+0000", 2024-02-23T00:55:47.670 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.7595e-05 2024-02-23T00:55:47.670 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.671 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.671 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.671 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:08.295823+0000", 2024-02-23T00:55:47.671 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.5389999999999998e-06 2024-02-23T00:55:47.671 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.671 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.671 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op must wait for map", 2024-02-23T00:55:47.671 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:08.295828+0000", 2024-02-23T00:55:47.671 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.002682214 2024-02-23T00:55:47.671 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.671 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.671 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.671 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:08.298510+0000", 2024-02-23T00:55:47.671 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.148e-05 2024-02-23T00:55:47.672 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.672 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.672 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:47.672 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:08.298582+0000", 2024-02-23T00:55:47.672 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00023491399999999999 2024-02-23T00:55:47.672 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.672 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.672 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:47.673 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:08.298817+0000", 2024-02-23T00:55:47.673 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0030963840000000002 2024-02-23T00:55:47.673 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.673 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:47.673 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.673 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.673 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.673 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "MOSDScrubReserve(18.0 REQUEST e739)", 2024-02-23T00:55:47.673 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:54:39.418894+0000", 2024-02-23T00:55:47.673 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 68.163616043000005, 2024-02-23T00:55:47.673 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0035206360000000002, 2024-02-23T00:55:47.673 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:47.673 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:55:47.673 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:47.673 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.674 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:47.674 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:39.418894+0000", 2024-02-23T00:55:47.674 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:47.674 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.674 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.674 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:47.674 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:39.418894+0000", 2024-02-23T00:55:47.674 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999995 2024-02-23T00:55:47.674 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.674 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.674 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:47.674 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:39.418894+0000", 2024-02-23T00:55:47.674 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5700000000000003e-06 2024-02-23T00:55:47.674 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.675 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.675 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:47.675 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:39.418899+0000", 2024-02-23T00:55:47.675 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2518999999999999e-05 2024-02-23T00:55:47.675 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.675 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.675 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:47.675 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:39.418921+0000", 2024-02-23T00:55:47.675 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0709999999999995e-06 2024-02-23T00:55:47.675 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.675 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.675 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:47.675 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:39.418929+0000", 2024-02-23T00:55:47.675 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6198000000000005e-05 2024-02-23T00:55:47.676 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.676 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.676 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.676 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:39.419005+0000", 2024-02-23T00:55:47.676 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9920000000000007e-06 2024-02-23T00:55:47.676 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.676 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.676 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op must wait for map", 2024-02-23T00:55:47.676 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:39.419013+0000", 2024-02-23T00:55:47.676 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.003351447 2024-02-23T00:55:47.676 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.677 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.677 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.677 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:39.422365+0000", 2024-02-23T00:55:47.677 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6599000000000002e-05 2024-02-23T00:55:47.677 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.677 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.677 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:47.677 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:39.422381+0000", 2024-02-23T00:55:47.677 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.3495999999999999e-05 2024-02-23T00:55:47.677 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.677 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.677 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:47.677 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:54:39.422415+0000", 2024-02-23T00:55:47.678 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0035206360000000002 2024-02-23T00:55:47.678 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.678 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:47.678 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.678 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.678 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.678 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(osd.0.8:52 234.6s0 234:75e4549c:test-rados-api-smithi047-82465-133::foo0:head [read 0~1] snapc 0=[] ondisk+read+ignore_cache+ignore_overlay+known_if_redirected e757)", 2024-02-23T00:55:47.678 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:55:05.333815+0000", 2024-02-23T00:55:47.678 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 42.248694921999999, 2024-02-23T00:55:47.678 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0037964629999999999, 2024-02-23T00:55:47.678 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:47.678 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-02-23T00:55:47.678 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:47.678 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.679 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:47.679 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:05.333815+0000", 2024-02-23T00:55:47.679 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:47.679 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.679 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.679 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:47.679 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:05.333815+0000", 2024-02-23T00:55:47.679 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999976 2024-02-23T00:55:47.679 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.679 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.679 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:47.680 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:05.333813+0000", 2024-02-23T00:55:47.680 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2361000000000001e-05 2024-02-23T00:55:47.680 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.680 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.680 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:47.680 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:05.333825+0000", 2024-02-23T00:55:47.680 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9649000000000001e-05 2024-02-23T00:55:47.680 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.680 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.680 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:47.680 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:05.333845+0000", 2024-02-23T00:55:47.680 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4340000000000003e-06 2024-02-23T00:55:47.680 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.681 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.681 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:47.681 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:05.333853+0000", 2024-02-23T00:55:47.681 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4103000000000003e-05 2024-02-23T00:55:47.681 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.681 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.681 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.681 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:05.333947+0000", 2024-02-23T00:55:47.681 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1530000000000002e-06 2024-02-23T00:55:47.681 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.681 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.681 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op must wait for map", 2024-02-23T00:55:47.681 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:05.333953+0000", 2024-02-23T00:55:47.682 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0024791879999999998 2024-02-23T00:55:47.682 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.682 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.682 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.682 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:05.336432+0000", 2024-02-23T00:55:47.682 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103047 2024-02-23T00:55:47.682 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.682 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.682 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:47.682 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:05.336535+0000", 2024-02-23T00:55:47.682 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0010768139999999999 2024-02-23T00:55:47.682 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.682 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.683 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:47.683 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:05.337612+0000", 2024-02-23T00:55:47.683 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0037964629999999999 2024-02-23T00:55:47.683 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.683 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:47.683 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.683 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.683 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.683 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.41771.0:2312 234.5s0 234:ad6b5e63:test-rados-api-smithi047-82465-135::foo:head [writefull 0~8] snapc 0=[] ondisk+write+known_if_redirected e762)", 2024-02-23T00:55:47.683 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:55:10.320891+0000", 2024-02-23T00:55:47.683 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 37.261619580999998, 2024-02-23T00:55:47.683 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010015049999999999, 2024-02-23T00:55:47.683 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:47.684 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:55:47.684 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:55:47.684 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.41771", 2024-02-23T00:55:47.684 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1032768195", 2024-02-23T00:55:47.684 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 2312 2024-02-23T00:55:47.684 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.684 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:47.684 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.684 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:47.684 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:10.320891+0000", 2024-02-23T00:55:47.684 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:47.684 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.684 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.685 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:47.685 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:10.320891+0000", 2024-02-23T00:55:47.685 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999981 2024-02-23T00:55:47.685 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.685 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.685 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:47.685 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:10.320889+0000", 2024-02-23T00:55:47.685 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0387999999999999e-05 2024-02-23T00:55:47.685 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.685 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.685 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:47.686 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:10.320899+0000", 2024-02-23T00:55:47.686 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5339e-05 2024-02-23T00:55:47.686 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.686 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.686 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:47.686 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:10.320914+0000", 2024-02-23T00:55:47.686 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.7909999999999993e-06 2024-02-23T00:55:47.686 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.686 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.686 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:47.686 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:10.320923+0000", 2024-02-23T00:55:47.686 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9072e-05 2024-02-23T00:55:47.686 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.687 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.687 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.687 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:10.320972+0000", 2024-02-23T00:55:47.687 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000157441 2024-02-23T00:55:47.687 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.687 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.687 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:47.687 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:10.321130+0000", 2024-02-23T00:55:47.687 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00053502499999999995 2024-02-23T00:55:47.687 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.687 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.687 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:55:47.687 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:10.321665+0000", 2024-02-23T00:55:47.687 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0041182939999999998 2024-02-23T00:55:47.688 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.688 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.688 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:55:47.688 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:10.325783+0000", 2024-02-23T00:55:47.688 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.005045145 2024-02-23T00:55:47.688 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.688 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.688 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:55:47.688 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:10.330828+0000", 2024-02-23T00:55:47.688 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.7595000000000005e-05 2024-02-23T00:55:47.688 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.688 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.688 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:47.689 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:10.330906+0000", 2024-02-23T00:55:47.689 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010015049999999999 2024-02-23T00:55:47.689 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.689 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:47.689 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.689 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.689 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.689 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "MOSDECSubOpWrite(234.7s2 777/600 ECSubWrite(tid=846, reqid=client.41771.0:2732, at_version=777'16, trim_to=0'0, roll_forward_to=748'15))", 2024-02-23T00:55:47.689 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:55:28.337447+0000", 2024-02-23T00:55:47.689 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 19.245063065, 2024-02-23T00:55:47.689 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0034706960000000001, 2024-02-23T00:55:47.689 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:47.689 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "reached pg", 2024-02-23T00:55:47.690 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:47.690 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.690 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:47.690 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:28.337447+0000", 2024-02-23T00:55:47.690 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:47.690 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.690 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.690 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:47.690 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:28.337447+0000", 2024-02-23T00:55:47.691 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999995 2024-02-23T00:55:47.691 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.691 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.691 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:47.691 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:28.337447+0000", 2024-02-23T00:55:47.691 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.8844999999999999e-05 2024-02-23T00:55:47.691 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.691 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.691 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:47.691 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:28.337476+0000", 2024-02-23T00:55:47.691 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1753000000000001e-05 2024-02-23T00:55:47.691 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.692 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.692 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:47.692 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:28.337488+0000", 2024-02-23T00:55:47.692 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3489999999999992e-06 2024-02-23T00:55:47.692 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.692 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.692 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:47.692 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:28.337496+0000", 2024-02-23T00:55:47.692 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0809999999999995e-05 2024-02-23T00:55:47.692 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.692 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.692 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.692 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:28.337567+0000", 2024-02-23T00:55:47.693 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.03e-05 2024-02-23T00:55:47.693 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.693 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.693 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:55:47.693 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:28.337577+0000", 2024-02-23T00:55:47.693 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033208500000000002 2024-02-23T00:55:47.693 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.693 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.693 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:55:47.693 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:28.340898+0000", 2024-02-23T00:55:47.693 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0046000000000001e-05 2024-02-23T00:55:47.693 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.694 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.694 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:47.694 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:28.340918+0000", 2024-02-23T00:55:47.694 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0034706960000000001 2024-02-23T00:55:47.694 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.694 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:47.694 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.694 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.694 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.694 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.41771.0:2733 234.5s0 234:a3391999:test-rados-api-smithi047-82465-137::bar:head [writefull 0~8] snapc 0=[] ondisk+write+known_if_redirected e777)", 2024-02-23T00:55:47.694 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-02-23T00:55:28.344366+0000", 2024-02-23T00:55:47.694 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 19.238143906000001, 2024-02-23T00:55:47.695 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0048947369999999997, 2024-02-23T00:55:47.695 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-02-23T00:55:47.695 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-23T00:55:47.695 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-02-23T00:55:47.695 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.41771", 2024-02-23T00:55:47.695 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.47:0/1032768195", 2024-02-23T00:55:47.695 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 2733 2024-02-23T00:55:47.695 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.695 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-02-23T00:55:47.695 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.695 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-02-23T00:55:47.696 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:28.344366+0000", 2024-02-23T00:55:47.696 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-02-23T00:55:47.696 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.696 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.696 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-02-23T00:55:47.696 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:28.344366+0000", 2024-02-23T00:55:47.696 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4294967295.9999986 2024-02-23T00:55:47.696 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.696 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.696 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-02-23T00:55:47.696 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:28.344365+0000", 2024-02-23T00:55:47.696 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9410000000000004e-06 2024-02-23T00:55:47.696 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.697 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.697 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-02-23T00:55:47.697 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:28.344371+0000", 2024-02-23T00:55:47.697 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1922999999999999e-05 2024-02-23T00:55:47.697 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.697 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.697 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-02-23T00:55:47.697 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:28.344383+0000", 2024-02-23T00:55:47.697 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.575e-06 2024-02-23T00:55:47.697 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.697 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.697 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-02-23T00:55:47.698 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:28.344388+0000", 2024-02-23T00:55:47.698 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0949000000000003e-05 2024-02-23T00:55:47.698 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.698 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.698 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-02-23T00:55:47.698 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:28.344439+0000", 2024-02-23T00:55:47.698 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014462200000000001 2024-02-23T00:55:47.698 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.698 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.698 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-02-23T00:55:47.698 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:28.344584+0000", 2024-02-23T00:55:47.698 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00042200400000000002 2024-02-23T00:55:47.698 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.699 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.699 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_started", 2024-02-23T00:55:47.699 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:28.345006+0000", 2024-02-23T00:55:47.699 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00096378999999999996 2024-02-23T00:55:47.699 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.699 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.699 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_committed", 2024-02-23T00:55:47.699 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:28.345970+0000", 2024-02-23T00:55:47.699 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0032105699999999998 2024-02-23T00:55:47.699 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.699 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.699 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-02-23T00:55:47.700 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:28.349180+0000", 2024-02-23T00:55:47.700 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1026999999999995e-05 2024-02-23T00:55:47.700 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-02-23T00:55:47.700 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-02-23T00:55:47.700 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-02-23T00:55:47.700 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-02-23T00:55:28.349261+0000", 2024-02-23T00:55:47.700 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0048947369999999997 2024-02-23T00:55:47.700 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.700 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:47.700 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.700 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-02-23T00:55:47.700 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-23T00:55:47.700 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:55:47.701 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:47.702 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:55:47.702 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:55:47.804 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:55:47.836 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:47.940 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:46.296604+0000 osd.4 (osd.4) 333 : cluster [DBG] 2.b scrub starts 2024-02-23T00:55:47.940 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:46.297441+0000 osd.4 (osd.4) 334 : cluster [DBG] 2.b scrub ok 2024-02-23T00:55:47.940 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:46.330035+0000 osd.5 (osd.5) 338 : cluster [DBG] 18.c scrub starts 2024-02-23T00:55:47.940 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:46.333256+0000 osd.5 (osd.5) 339 : cluster [DBG] 18.c scrub ok 2024-02-23T00:55:47.940 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:46.761001+0000 mon.a (mon.0) 3760 : cluster [DBG] osdmap e791: 8 total, 8 up, 6 in 2024-02-23T00:55:47.940 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:46.938793+0000 osd.6 (osd.6) 305 : cluster [DBG] 6.3 scrub starts 2024-02-23T00:55:47.940 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:46.939864+0000 osd.6 (osd.6) 306 : cluster [DBG] 6.3 scrub ok 2024-02-23T00:55:47.940 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:47.261767+0000 mon.a (mon.0) 3761 : audit [DBG] from='client.? 172.21.15.47:0/776399190' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:55:47.940 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:47.344009+0000 mon.c (mon.2) 496 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:47.940 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:47.344335+0000 mon.a (mon.0) 3762 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:47.941 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:47.406250+0000 mgr.y (mgr.14517) 672 : cluster [DBG] pgmap v1204: 194 pgs: 194 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:55:48.012 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6, 0] out_osds: [2, 7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-23T00:55:48.012 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-23T00:55:48.013 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-02-23T00:55:48.013 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 0.000000 2024-02-23T00:55:48.013 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd primary-affinity 1 0 2024-02-23T00:55:48.014 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:46.296604+0000 osd.4 (osd.4) 333 : cluster [DBG] 2.b scrub starts 2024-02-23T00:55:48.014 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:46.297441+0000 osd.4 (osd.4) 334 : cluster [DBG] 2.b scrub ok 2024-02-23T00:55:48.014 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:46.330035+0000 osd.5 (osd.5) 338 : cluster [DBG] 18.c scrub starts 2024-02-23T00:55:48.014 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:46.333256+0000 osd.5 (osd.5) 339 : cluster [DBG] 18.c scrub ok 2024-02-23T00:55:48.014 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:46.761001+0000 mon.a (mon.0) 3760 : cluster [DBG] osdmap e791: 8 total, 8 up, 6 in 2024-02-23T00:55:48.014 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:46.938793+0000 osd.6 (osd.6) 305 : cluster [DBG] 6.3 scrub starts 2024-02-23T00:55:48.014 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:46.939864+0000 osd.6 (osd.6) 306 : cluster [DBG] 6.3 scrub ok 2024-02-23T00:55:48.014 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:47.261767+0000 mon.a (mon.0) 3761 : audit [DBG] from='client.? 172.21.15.47:0/776399190' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:55:48.014 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:47.344009+0000 mon.c (mon.2) 496 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:48.015 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:47.344335+0000 mon.a (mon.0) 3762 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:48.015 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:47.406250+0000 mgr.y (mgr.14517) 672 : cluster [DBG] pgmap v1204: 194 pgs: 194 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:55:48.015 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:46.296604+0000 osd.4 (osd.4) 333 : cluster [DBG] 2.b scrub starts 2024-02-23T00:55:48.015 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:46.297441+0000 osd.4 (osd.4) 334 : cluster [DBG] 2.b scrub ok 2024-02-23T00:55:48.015 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:46.330035+0000 osd.5 (osd.5) 338 : cluster [DBG] 18.c scrub starts 2024-02-23T00:55:48.015 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:46.333256+0000 osd.5 (osd.5) 339 : cluster [DBG] 18.c scrub ok 2024-02-23T00:55:48.015 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:46.761001+0000 mon.a (mon.0) 3760 : cluster [DBG] osdmap e791: 8 total, 8 up, 6 in 2024-02-23T00:55:48.015 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:46.938793+0000 osd.6 (osd.6) 305 : cluster [DBG] 6.3 scrub starts 2024-02-23T00:55:48.015 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:46.939864+0000 osd.6 (osd.6) 306 : cluster [DBG] 6.3 scrub ok 2024-02-23T00:55:48.016 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:47.261767+0000 mon.a (mon.0) 3761 : audit [DBG] from='client.? 172.21.15.47:0/776399190' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:55:48.016 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:47.344009+0000 mon.c (mon.2) 496 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:48.016 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:47.344335+0000 mon.a (mon.0) 3762 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:48.016 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:47.406250+0000 mgr.y (mgr.14517) 672 : cluster [DBG] pgmap v1204: 194 pgs: 194 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-23T00:55:48.263 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.7 dump_ops_in_flight 2024-02-23T00:55:48.285 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:55:48.287 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:48.288 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:55:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:55:48.389 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:55:48.417 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:48.540 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:55:48.542 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:48.543 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:55:48.553 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:55:48.643 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:55:48.672 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:48.770 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:55:48.773 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:48.773 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:55:48.815 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-23T00:55:48.874 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:55:48.902 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:48.921 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:55:48.924 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:48.925 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:55:49.026 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:55:49.050 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:47.311144+0000 osd.5 (osd.5) 340 : cluster [DBG] 18.13 scrub starts 2024-02-23T00:55:49.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:47.312768+0000 osd.5 (osd.5) 341 : cluster [DBG] 18.13 scrub ok 2024-02-23T00:55:49.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:47.772509+0000 mon.a (mon.0) 3763 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:55:49.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:47.772588+0000 mon.a (mon.0) 3764 : cluster [DBG] osdmap e792: 8 total, 8 up, 6 in 2024-02-23T00:55:49.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:47.775558+0000 mon.c (mon.2) 497 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-136"}]: dispatch 2024-02-23T00:55:49.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:47.775917+0000 mon.a (mon.0) 3765 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-136"}]: dispatch 2024-02-23T00:55:49.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:47.915038+0000 osd.0 (osd.0) 295 : cluster [DBG] 18.1d scrub starts 2024-02-23T00:55:49.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:47.917824+0000 osd.0 (osd.0) 296 : cluster [DBG] 18.1d scrub ok 2024-02-23T00:55:49.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:47.970872+0000 osd.6 (osd.6) 307 : cluster [DBG] 2.1d scrub starts 2024-02-23T00:55:49.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:47.971667+0000 osd.6 (osd.6) 308 : cluster [DBG] 2.1d scrub ok 2024-02-23T00:55:49.052 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:49.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:47.311144+0000 osd.5 (osd.5) 340 : cluster [DBG] 18.13 scrub starts 2024-02-23T00:55:49.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:47.312768+0000 osd.5 (osd.5) 341 : cluster [DBG] 18.13 scrub ok 2024-02-23T00:55:49.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:47.772509+0000 mon.a (mon.0) 3763 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:55:49.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:47.772588+0000 mon.a (mon.0) 3764 : cluster [DBG] osdmap e792: 8 total, 8 up, 6 in 2024-02-23T00:55:49.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:47.775558+0000 mon.c (mon.2) 497 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-136"}]: dispatch 2024-02-23T00:55:49.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:47.775917+0000 mon.a (mon.0) 3765 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-136"}]: dispatch 2024-02-23T00:55:49.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:47.915038+0000 osd.0 (osd.0) 295 : cluster [DBG] 18.1d scrub starts 2024-02-23T00:55:49.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:47.917824+0000 osd.0 (osd.0) 296 : cluster [DBG] 18.1d scrub ok 2024-02-23T00:55:49.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:47.970872+0000 osd.6 (osd.6) 307 : cluster [DBG] 2.1d scrub starts 2024-02-23T00:55:49.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:47.971667+0000 osd.6 (osd.6) 308 : cluster [DBG] 2.1d scrub ok 2024-02-23T00:55:49.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:47.311144+0000 osd.5 (osd.5) 340 : cluster [DBG] 18.13 scrub starts 2024-02-23T00:55:49.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:47.312768+0000 osd.5 (osd.5) 341 : cluster [DBG] 18.13 scrub ok 2024-02-23T00:55:49.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:47.772509+0000 mon.a (mon.0) 3763 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:55:49.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:47.772588+0000 mon.a (mon.0) 3764 : cluster [DBG] osdmap e792: 8 total, 8 up, 6 in 2024-02-23T00:55:49.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:47.775558+0000 mon.c (mon.2) 497 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-136"}]: dispatch 2024-02-23T00:55:49.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:47.775917+0000 mon.a (mon.0) 3765 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-136"}]: dispatch 2024-02-23T00:55:49.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:47.915038+0000 osd.0 (osd.0) 295 : cluster [DBG] 18.1d scrub starts 2024-02-23T00:55:49.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:47.917824+0000 osd.0 (osd.0) 296 : cluster [DBG] 18.1d scrub ok 2024-02-23T00:55:49.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:47.970872+0000 osd.6 (osd.6) 307 : cluster [DBG] 2.1d scrub starts 2024-02-23T00:55:49.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:47.971667+0000 osd.6 (osd.6) 308 : cluster [DBG] 2.1d scrub ok 2024-02-23T00:55:49.813 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:49.813 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:55:49.813 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:55:49.914 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:55:49.942 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:50.033 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:48.772859+0000 mon.a (mon.0) 3766 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-136"}]': finished 2024-02-23T00:55:50.033 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:48.772933+0000 mon.a (mon.0) 3767 : cluster [DBG] osdmap e793: 8 total, 8 up, 6 in 2024-02-23T00:55:50.033 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:48.798212+0000 mon.c (mon.2) 498 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:50.033 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:48.798520+0000 mon.a (mon.0) 3768 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:50.033 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:48.798949+0000 mon.c (mon.2) 499 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-136"}]: dispatch 2024-02-23T00:55:50.033 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:48.799152+0000 mon.a (mon.0) 3769 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-136"}]: dispatch 2024-02-23T00:55:50.033 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:48.915216+0000 osd.0 (osd.0) 297 : cluster [DBG] 18.2 scrub starts 2024-02-23T00:55:50.033 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:48.916111+0000 osd.0 (osd.0) 298 : cluster [DBG] 18.2 scrub ok 2024-02-23T00:55:50.033 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:48.938131+0000 osd.6 (osd.6) 309 : cluster [DBG] 234.5 scrub starts 2024-02-23T00:55:50.033 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:48.939657+0000 osd.6 (osd.6) 310 : cluster [DBG] 234.5 scrub ok 2024-02-23T00:55:50.033 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:49.406763+0000 mgr.y (mgr.14517) 673 : cluster [DBG] pgmap v1207: 194 pgs: 194 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:55:50.038 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:55:50.040 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:55:50.042 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:50.043 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:55:50.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:48.772859+0000 mon.a (mon.0) 3766 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-136"}]': finished 2024-02-23T00:55:50.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:48.772933+0000 mon.a (mon.0) 3767 : cluster [DBG] osdmap e793: 8 total, 8 up, 6 in 2024-02-23T00:55:50.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:48.798212+0000 mon.c (mon.2) 498 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:50.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:48.798520+0000 mon.a (mon.0) 3768 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:50.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:48.798949+0000 mon.c (mon.2) 499 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-136"}]: dispatch 2024-02-23T00:55:50.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:48.799152+0000 mon.a (mon.0) 3769 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-136"}]: dispatch 2024-02-23T00:55:50.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:48.915216+0000 osd.0 (osd.0) 297 : cluster [DBG] 18.2 scrub starts 2024-02-23T00:55:50.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:48.916111+0000 osd.0 (osd.0) 298 : cluster [DBG] 18.2 scrub ok 2024-02-23T00:55:50.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:48.938131+0000 osd.6 (osd.6) 309 : cluster [DBG] 234.5 scrub starts 2024-02-23T00:55:50.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:48.939657+0000 osd.6 (osd.6) 310 : cluster [DBG] 234.5 scrub ok 2024-02-23T00:55:50.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:49.406763+0000 mgr.y (mgr.14517) 673 : cluster [DBG] pgmap v1207: 194 pgs: 194 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:55:50.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:48.772859+0000 mon.a (mon.0) 3766 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-136"}]': finished 2024-02-23T00:55:50.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:48.772933+0000 mon.a (mon.0) 3767 : cluster [DBG] osdmap e793: 8 total, 8 up, 6 in 2024-02-23T00:55:50.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:48.798212+0000 mon.c (mon.2) 498 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:50.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:48.798520+0000 mon.a (mon.0) 3768 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:50.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:48.798949+0000 mon.c (mon.2) 499 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-136"}]: dispatch 2024-02-23T00:55:50.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:48.799152+0000 mon.a (mon.0) 3769 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-136"}]: dispatch 2024-02-23T00:55:50.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:48.915216+0000 osd.0 (osd.0) 297 : cluster [DBG] 18.2 scrub starts 2024-02-23T00:55:50.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:48.916111+0000 osd.0 (osd.0) 298 : cluster [DBG] 18.2 scrub ok 2024-02-23T00:55:50.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:48.938131+0000 osd.6 (osd.6) 309 : cluster [DBG] 234.5 scrub starts 2024-02-23T00:55:50.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:48.939657+0000 osd.6 (osd.6) 310 : cluster [DBG] 234.5 scrub ok 2024-02-23T00:55:50.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:49.406763+0000 mgr.y (mgr.14517) 673 : cluster [DBG] pgmap v1207: 194 pgs: 194 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:55:50.144 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:55:50.175 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:50.194 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:55:50.196 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:50.197 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:55:50.198 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:55:50.297 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:55:50.313 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:55:50.314 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-02-23T00:55:50.314 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 0 2024-02-23T00:55:50.314 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:55:50.328 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:50.350 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:55:50.353 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:50.353 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:55:50.454 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:55:50.482 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:50.580 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:55:50.582 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:50.583 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:55:50.675 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:55:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:55:50] "GET /metrics HTTP/1.1" 200 192052 "" "Prometheus/2.33.4" 2024-02-23T00:55:50.683 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:55:50.717 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:51.009 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:49.775914+0000 mon.a (mon.0) 3770 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:55:51.009 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:49.775969+0000 mon.a (mon.0) 3771 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-23T00:55:51.009 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:49.787077+0000 mon.a (mon.0) 3772 : cluster [DBG] osdmap e794: 8 total, 8 up, 6 in 2024-02-23T00:55:51.027 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.7 dump_blocked_ops 2024-02-23T00:55:51.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:49.775914+0000 mon.a (mon.0) 3770 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:55:51.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:49.775969+0000 mon.a (mon.0) 3771 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-23T00:55:51.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:49.787077+0000 mon.a (mon.0) 3772 : cluster [DBG] osdmap e794: 8 total, 8 up, 6 in 2024-02-23T00:55:51.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:49.775914+0000 mon.a (mon.0) 3770 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:55:51.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:49.775969+0000 mon.a (mon.0) 3771 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-23T00:55:51.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:49.787077+0000 mon.a (mon.0) 3772 : cluster [DBG] osdmap e794: 8 total, 8 up, 6 in 2024-02-23T00:55:51.292 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:55:51.624 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:55:51.628 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:51.628 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:55:51.729 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:55:51.763 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:51.867 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:55:51.869 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:50.309630+0000 osd.5 (osd.5) 342 : cluster [DBG] 3.2 scrub starts 2024-02-23T00:55:51.870 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:50.310705+0000 osd.5 (osd.5) 343 : cluster [DBG] 3.2 scrub ok 2024-02-23T00:55:51.870 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:50.789748+0000 mon.a (mon.0) 3773 : cluster [DBG] osdmap e795: 8 total, 8 up, 6 in 2024-02-23T00:55:51.870 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:50.792760+0000 mon.c (mon.2) 500 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:55:51.870 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:50.798194+0000 mon.a (mon.0) 3774 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:55:51.870 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:50.977160+0000 osd.6 (osd.6) 311 : cluster [DBG] 4.11 scrub starts 2024-02-23T00:55:51.870 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:50.978670+0000 osd.6 (osd.6) 312 : cluster [DBG] 4.11 scrub ok 2024-02-23T00:55:51.870 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:51.407240+0000 mgr.y (mgr.14517) 674 : cluster [DBG] pgmap v1210: 194 pgs: 32 unknown, 162 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:55:51.870 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:50.309630+0000 osd.5 (osd.5) 342 : cluster [DBG] 3.2 scrub starts 2024-02-23T00:55:51.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:50.310705+0000 osd.5 (osd.5) 343 : cluster [DBG] 3.2 scrub ok 2024-02-23T00:55:51.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:50.789748+0000 mon.a (mon.0) 3773 : cluster [DBG] osdmap e795: 8 total, 8 up, 6 in 2024-02-23T00:55:51.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:50.792760+0000 mon.c (mon.2) 500 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:55:51.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:50.798194+0000 mon.a (mon.0) 3774 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:55:51.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:50.977160+0000 osd.6 (osd.6) 311 : cluster [DBG] 4.11 scrub starts 2024-02-23T00:55:51.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:50.978670+0000 osd.6 (osd.6) 312 : cluster [DBG] 4.11 scrub ok 2024-02-23T00:55:51.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:51.407240+0000 mgr.y (mgr.14517) 674 : cluster [DBG] pgmap v1210: 194 pgs: 32 unknown, 162 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:55:51.871 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:51.872 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:55:51.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:50.309630+0000 osd.5 (osd.5) 342 : cluster [DBG] 3.2 scrub starts 2024-02-23T00:55:51.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:50.310705+0000 osd.5 (osd.5) 343 : cluster [DBG] 3.2 scrub ok 2024-02-23T00:55:51.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:50.789748+0000 mon.a (mon.0) 3773 : cluster [DBG] osdmap e795: 8 total, 8 up, 6 in 2024-02-23T00:55:51.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:50.792760+0000 mon.c (mon.2) 500 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:55:51.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:50.798194+0000 mon.a (mon.0) 3774 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:55:51.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:50.977160+0000 osd.6 (osd.6) 311 : cluster [DBG] 4.11 scrub starts 2024-02-23T00:55:51.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:50.978670+0000 osd.6 (osd.6) 312 : cluster [DBG] 4.11 scrub ok 2024-02-23T00:55:51.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:51.407240+0000 mgr.y (mgr.14517) 674 : cluster [DBG] pgmap v1210: 194 pgs: 32 unknown, 162 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:55:51.972 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:55:52.002 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:52.100 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:55:52.103 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:52.104 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:55:52.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.117+0000 7faeba5db700 1 -- 172.21.15.47:0/4096638066 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faeb40fcac0 msgr2=0x7faeb40fcea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:52.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.117+0000 7faeba5db700 1 --2- 172.21.15.47:0/4096638066 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faeb40fcac0 0x7faeb40fcea0 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 rx=0x7faea4004870 tx=0x7faea4009cb0).stop 2024-02-23T00:55:52.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.117+0000 7faeba5db700 1 -- 172.21.15.47:0/4096638066 shutdown_connections 2024-02-23T00:55:52.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.117+0000 7faeba5db700 1 --2- 172.21.15.47:0/4096638066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faeb40f7330 0x7faeb4103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:52.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.117+0000 7faeba5db700 1 --2- 172.21.15.47:0/4096638066 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faeb40fcac0 0x7faeb40fcea0 unknown :-1 s=CLOSED pgs=2823 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:52.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.117+0000 7faeba5db700 1 --2- 172.21.15.47:0/4096638066 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faeb40fd420 0x7faeb40f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:52.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.117+0000 7faeba5db700 1 -- 172.21.15.47:0/4096638066 >> 172.21.15.47:0/4096638066 conn(0x7faeb40f26f0 msgr2=0x7faeb40f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:52.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.117+0000 7faeba5db700 1 -- 172.21.15.47:0/4096638066 shutdown_connections 2024-02-23T00:55:52.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.117+0000 7faeba5db700 1 -- 172.21.15.47:0/4096638066 wait complete. 2024-02-23T00:55:52.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.117+0000 7faeba5db700 1 librados: starting msgr at 2024-02-23T00:55:52.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.117+0000 7faeba5db700 1 librados: starting objecter 2024-02-23T00:55:52.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.117+0000 7faeba5db700 1 Processor -- start 2024-02-23T00:55:52.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.117+0000 7faeba5db700 1 -- start start 2024-02-23T00:55:52.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.117+0000 7faeba5db700 1 librados: setting wanted keys 2024-02-23T00:55:52.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.117+0000 7faeba5db700 1 librados: calling monclient init 2024-02-23T00:55:52.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.118+0000 7faeba5db700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faeb40f7330 0x7faeb4110120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:52.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.118+0000 7faeba5db700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faeb40fcac0 0x7faeb4110610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:52.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.118+0000 7faeba5db700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faeb40fd420 0x7faeb410a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:52.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.118+0000 7faeba5db700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb410a7b0 con 0x7faeb40fcac0 2024-02-23T00:55:52.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.118+0000 7faeba5db700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb410a920 con 0x7faeb40fd420 2024-02-23T00:55:52.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.118+0000 7faeba5db700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb410aae0 con 0x7faeb40f7330 2024-02-23T00:55:52.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.118+0000 7faeb37fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faeb40fcac0 0x7faeb4110610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:52.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.118+0000 7faeb37fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faeb40fcac0 0x7faeb4110610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34722/0 (socket says 172.21.15.47:34722) 2024-02-23T00:55:52.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.118+0000 7faeb37fe700 1 -- 172.21.15.47:0/1570938448 learned_addr learned my addr 172.21.15.47:0/1570938448 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:55:52.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.118+0000 7faeb8b78700 1 --2- 172.21.15.47:0/1570938448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faeb40fd420 0x7faeb410a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:52.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.118+0000 7faeb3fff700 1 --2- 172.21.15.47:0/1570938448 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faeb40f7330 0x7faeb4110120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:52.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.118+0000 7faeb3fff700 1 -- 172.21.15.47:0/1570938448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faeb40fd420 msgr2=0x7faeb410a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:52.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.118+0000 7faeb3fff700 1 --2- 172.21.15.47:0/1570938448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faeb40fd420 0x7faeb410a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:52.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.118+0000 7faeb3fff700 1 -- 172.21.15.47:0/1570938448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faeb40fcac0 msgr2=0x7faeb4110610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:52.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.118+0000 7faeb3fff700 1 --2- 172.21.15.47:0/1570938448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faeb40fcac0 0x7faeb4110610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:52.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.118+0000 7faeb3fff700 1 -- 172.21.15.47:0/1570938448 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea8005040 con 0x7faeb40f7330 2024-02-23T00:55:52.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.118+0000 7faeb8b78700 1 --2- 172.21.15.47:0/1570938448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faeb40fd420 0x7faeb410a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:55:52.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.118+0000 7faeb37fe700 1 --2- 172.21.15.47:0/1570938448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faeb40fcac0 0x7faeb4110610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:55:52.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.118+0000 7faeb3fff700 1 --2- 172.21.15.47:0/1570938448 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faeb40f7330 0x7faeb4110120 secure :-1 s=READY pgs=2824 cs=0 l=1 rev1=1 rx=0x7faea4009da0 tx=0x7faea4009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:52.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.118+0000 7faeb17fa700 10 client.?.objecter ms_handle_connect 0x7faeb40f7330 2024-02-23T00:55:52.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.118+0000 7faeb17fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:55:52.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.119+0000 7faeb17fa700 1 -- 172.21.15.47:0/1570938448 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea400e030 con 0x7faeb40f7330 2024-02-23T00:55:52.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.119+0000 7faeba5db700 1 -- 172.21.15.47:0/1570938448 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea4005040 con 0x7faeb40f7330 2024-02-23T00:55:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.119+0000 7faeb17fa700 1 -- 172.21.15.47:0/1570938448 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7faea4013d70 con 0x7faeb40f7330 2024-02-23T00:55:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.119+0000 7faeb17fa700 1 -- 172.21.15.47:0/1570938448 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea4018070 con 0x7faeb40f7330 2024-02-23T00:55:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.119+0000 7faeba5db700 10 client.42170.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:55:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.119+0000 7faeba5db700 1 -- 172.21.15.47:0/1570938448 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb410b000 con 0x7faeb40f7330 2024-02-23T00:55:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.119+0000 7faeba5db700 1 librados: init done 2024-02-23T00:55:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.120+0000 7faeba5db700 1 -- 172.21.15.47:0/1570938448 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeb405f990 con 0x7faeb40f7330 2024-02-23T00:55:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.120+0000 7faeb17fa700 1 -- 172.21.15.47:0/1570938448 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7faea4009240 con 0x7faeb40f7330 2024-02-23T00:55:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.120+0000 7faeb17fa700 1 --2- 172.21.15.47:0/1570938448 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fae9c062cc0 0x7fae9c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.120+0000 7faeb17fa700 1 -- 172.21.15.47:0/1570938448 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(796..796 src has 621..796) v4 ==== 9811+0+0 (secure 0 0 0) 0x7faea407e930 con 0x7faeb40f7330 2024-02-23T00:55:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.120+0000 7faeb17fa700 10 client.42170.objecter ms_dispatch 0x7faeb4001550 osd_map(796..796 src has 621..796) v4 2024-02-23T00:55:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.120+0000 7faeb17fa700 3 client.42170.objecter handle_osd_map got epochs [796,796] > 0 2024-02-23T00:55:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.120+0000 7faeb17fa700 3 client.42170.objecter handle_osd_map decoding full epoch 796 2024-02-23T00:55:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.120+0000 7faeb37fe700 1 --2- 172.21.15.47:0/1570938448 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fae9c062cc0 0x7fae9c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:52.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.121+0000 7faeb17fa700 20 client.42170.objecter dump_active .. 0 homeless 2024-02-23T00:55:52.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.121+0000 7faeb17fa700 10 client.42170.objecter ms_handle_connect 0x7fae9c062cc0 2024-02-23T00:55:52.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.121+0000 7faeb37fe700 1 --2- 172.21.15.47:0/1570938448 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fae9c062cc0 0x7fae9c065180 secure :-1 s=READY pgs=3212 cs=0 l=1 rev1=1 rx=0x7faea8007fd0 tx=0x7faea800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:52.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.125+0000 7faeb17fa700 1 -- 172.21.15.47:0/1570938448 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7faea404dcf0 con 0x7faeb40f7330 2024-02-23T00:55:52.205 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:55:52.235 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:52.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.316+0000 7faeba5db700 1 -- 172.21.15.47:0/1570938448 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7faeb410b7a0 con 0x7faeb40f7330 2024-02-23T00:55:52.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.328+0000 7fd958940700 1 -- 172.21.15.47:0/2731963690 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd954102210 msgr2=0x7fd954104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:52.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.328+0000 7fd958940700 1 --2- 172.21.15.47:0/2731963690 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd954102210 0x7fd954104600 secure :-1 s=READY pgs=2825 cs=0 l=1 rev1=1 rx=0x7fd94c009be0 tx=0x7fd94c004d10).stop 2024-02-23T00:55:52.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.329+0000 7fd958940700 1 -- 172.21.15.47:0/2731963690 shutdown_connections 2024-02-23T00:55:52.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.329+0000 7fd958940700 1 --2- 172.21.15.47:0/2731963690 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd954102210 0x7fd954104600 unknown :-1 s=CLOSED pgs=2825 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:52.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.329+0000 7fd958940700 1 --2- 172.21.15.47:0/2731963690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9540f8cb0 0x7fd9540f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:52.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.329+0000 7fd958940700 1 --2- 172.21.15.47:0/2731963690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9540f9610 0x7fd954101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:52.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.329+0000 7fd958940700 1 -- 172.21.15.47:0/2731963690 >> 172.21.15.47:0/2731963690 conn(0x7fd9540f2820 msgr2=0x7fd9540f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:52.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.329+0000 7fd958940700 1 -- 172.21.15.47:0/2731963690 shutdown_connections 2024-02-23T00:55:52.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.329+0000 7fd958940700 1 -- 172.21.15.47:0/2731963690 wait complete. 2024-02-23T00:55:52.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.329+0000 7fd958940700 1 librados: starting msgr at 2024-02-23T00:55:52.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.329+0000 7fd958940700 1 librados: starting objecter 2024-02-23T00:55:52.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.330+0000 7fd958940700 1 Processor -- start 2024-02-23T00:55:52.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.330+0000 7fd958940700 1 -- start start 2024-02-23T00:55:52.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.330+0000 7fd958940700 1 librados: setting wanted keys 2024-02-23T00:55:52.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.330+0000 7fd958940700 1 librados: calling monclient init 2024-02-23T00:55:52.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.330+0000 7fd958940700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9540f8cb0 0x7fd95410dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:52.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.330+0000 7fd958940700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd9540f9610 0x7fd95410e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:52.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.330+0000 7fd958940700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd954102210 0x7fd954108080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:52.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.330+0000 7fd958940700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd954108570 con 0x7fd954102210 2024-02-23T00:55:52.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.330+0000 7fd958940700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9541086e0 con 0x7fd9540f8cb0 2024-02-23T00:55:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.330+0000 7fd958940700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9541088a0 con 0x7fd9540f9610 2024-02-23T00:55:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.330+0000 7fd951d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd9540f9610 0x7fd95410e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.330+0000 7fd951d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd9540f9610 0x7fd95410e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:57334/0 (socket says 172.21.15.47:57334) 2024-02-23T00:55:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.330+0000 7fd951d9b700 1 -- 172.21.15.47:0/3498481427 learned_addr learned my addr 172.21.15.47:0/3498481427 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:55:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.330+0000 7fd952d9d700 1 --2- 172.21.15.47:0/3498481427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd954102210 0x7fd954108080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.330+0000 7fd95259c700 1 --2- 172.21.15.47:0/3498481427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9540f8cb0 0x7fd95410dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.330+0000 7fd951d9b700 1 -- 172.21.15.47:0/3498481427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9540f8cb0 msgr2=0x7fd95410dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.330+0000 7fd951d9b700 1 --2- 172.21.15.47:0/3498481427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9540f8cb0 0x7fd95410dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.330+0000 7fd951d9b700 1 -- 172.21.15.47:0/3498481427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd954102210 msgr2=0x7fd954108080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.330+0000 7fd951d9b700 1 --2- 172.21.15.47:0/3498481427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd954102210 0x7fd954108080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.330+0000 7fd951d9b700 1 -- 172.21.15.47:0/3498481427 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd948005040 con 0x7fd9540f9610 2024-02-23T00:55:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.331+0000 7fd951d9b700 1 --2- 172.21.15.47:0/3498481427 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd9540f9610 0x7fd95410e3d0 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 rx=0x7fd9480048f0 tx=0x7fd948009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.331+0000 7fd9437fe700 10 client.?.objecter ms_handle_connect 0x7fd9540f9610 2024-02-23T00:55:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.331+0000 7fd9437fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:55:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.331+0000 7fd9437fe700 1 -- 172.21.15.47:0/3498481427 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd948007b00 con 0x7fd9540f9610 2024-02-23T00:55:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.331+0000 7fd9437fe700 1 -- 172.21.15.47:0/3498481427 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fd94800f040 con 0x7fd9540f9610 2024-02-23T00:55:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.331+0000 7fd9437fe700 1 -- 172.21.15.47:0/3498481427 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd948014890 con 0x7fd9540f9610 2024-02-23T00:55:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.331+0000 7fd958940700 1 -- 172.21.15.47:0/3498481427 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd94c0097c0 con 0x7fd9540f9610 2024-02-23T00:55:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.331+0000 7fd958940700 10 client.42176.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:55:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.331+0000 7fd958940700 1 -- 172.21.15.47:0/3498481427 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd954108ee0 con 0x7fd9540f9610 2024-02-23T00:55:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.331+0000 7fd958940700 1 librados: init done 2024-02-23T00:55:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.332+0000 7fd958940700 1 -- 172.21.15.47:0/3498481427 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd95405fa60 con 0x7fd9540f9610 2024-02-23T00:55:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.332+0000 7fd9437fe700 1 -- 172.21.15.47:0/3498481427 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fd948014cc0 con 0x7fd9540f9610 2024-02-23T00:55:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.332+0000 7fd9437fe700 1 --2- 172.21.15.47:0/3498481427 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd93c062cc0 0x7fd93c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.333+0000 7fd9437fe700 1 -- 172.21.15.47:0/3498481427 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(796..796 src has 621..796) v4 ==== 9811+0+0 (secure 0 0 0) 0x7fd94807d3d0 con 0x7fd9540f9610 2024-02-23T00:55:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.333+0000 7fd9437fe700 10 client.42176.objecter ms_dispatch 0x7fd954001550 osd_map(796..796 src has 621..796) v4 2024-02-23T00:55:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.333+0000 7fd9437fe700 3 client.42176.objecter handle_osd_map got epochs [796,796] > 0 2024-02-23T00:55:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.333+0000 7fd9437fe700 3 client.42176.objecter handle_osd_map decoding full epoch 796 2024-02-23T00:55:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.333+0000 7fd95259c700 1 --2- 172.21.15.47:0/3498481427 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd93c062cc0 0x7fd93c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.333+0000 7fd9437fe700 20 client.42176.objecter dump_active .. 0 homeless 2024-02-23T00:55:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.333+0000 7fd9437fe700 10 client.42176.objecter ms_handle_connect 0x7fd93c062cc0 2024-02-23T00:55:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.333+0000 7fd95259c700 1 --2- 172.21.15.47:0/3498481427 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd93c062cc0 0x7fd93c065180 secure :-1 s=READY pgs=3213 cs=0 l=1 rev1=1 rx=0x7fd9440092f0 tx=0x7fd94400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.337+0000 7fd9437fe700 1 -- 172.21.15.47:0/3498481427 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fd948049f60 con 0x7fd9540f9610 2024-02-23T00:55:52.360 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:55:52.363 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:52.363 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:55:52.403 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:55:52.464 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:55:52.492 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:52.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.532+0000 7fd958940700 1 -- 172.21.15.47:0/3498481427 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd primary-affinity", "id": 1, "weight": 0.0} v 0) v1 -- 0x7fd954107050 con 0x7fd9540f9610 2024-02-23T00:55:52.615 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:55:52.617 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:52.617 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:55:52.718 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:55:52.743 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:55:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:55:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:55:52.745 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:52.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.796+0000 7faeb17fa700 1 -- 172.21.15.47:0/1570938448 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v797) v1 ==== 101+0+0 (secure 0 0 0) 0x7faea4051310 con 0x7faeb40f7330 2024-02-23T00:55:52.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.796+0000 7fd9437fe700 1 -- 172.21.15.47:0/3498481427 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 1, "weight": 0.0}]=0 set osd.1 primary-affinity to 0 (802) v797) v1 ==== 129+0+0 (secure 0 0 0) 0x7fd94804c810 con 0x7fd9540f9610 2024-02-23T00:55:52.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.802+0000 7fd958940700 10 librados: watch_flush enter 2024-02-23T00:55:52.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.802+0000 7fd958940700 10 librados: watch_flush exit 2024-02-23T00:55:52.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.802+0000 7fd958940700 20 client.42176.objecter shutdown clearing up homeless session... 2024-02-23T00:55:52.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.802+0000 7fd958940700 10 client.42176.objecter successfully canceled tick 2024-02-23T00:55:52.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.802+0000 7fd958940700 1 -- 172.21.15.47:0/3498481427 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd93c062cc0 msgr2=0x7fd93c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:52.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.802+0000 7fd958940700 1 --2- 172.21.15.47:0/3498481427 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd93c062cc0 0x7fd93c065180 secure :-1 s=READY pgs=3213 cs=0 l=1 rev1=1 rx=0x7fd9440092f0 tx=0x7fd94400b040).stop 2024-02-23T00:55:52.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.802+0000 7fd958940700 1 -- 172.21.15.47:0/3498481427 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd9540f9610 msgr2=0x7fd95410e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:52.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.802+0000 7fd958940700 1 --2- 172.21.15.47:0/3498481427 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd9540f9610 0x7fd95410e3d0 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 rx=0x7fd9480048f0 tx=0x7fd948009cb0).stop 2024-02-23T00:55:52.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.803+0000 7fd958940700 1 -- 172.21.15.47:0/3498481427 shutdown_connections 2024-02-23T00:55:52.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.803+0000 7fd958940700 1 --2- 172.21.15.47:0/3498481427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd954102210 0x7fd954108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:52.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.803+0000 7fd958940700 1 --2- 172.21.15.47:0/3498481427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9540f8cb0 0x7fd95410dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:52.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.803+0000 7fd958940700 1 --2- 172.21.15.47:0/3498481427 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fd93c062cc0 0x7fd93c065180 unknown :-1 s=CLOSED pgs=3213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:52.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.803+0000 7fd958940700 1 --2- 172.21.15.47:0/3498481427 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd9540f9610 0x7fd95410e3d0 secure :-1 s=CLOSED pgs=2826 cs=0 l=1 rev1=1 rx=0x7fd9480048f0 tx=0x7fd948009cb0).stop 2024-02-23T00:55:52.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.803+0000 7fd958940700 1 -- 172.21.15.47:0/3498481427 >> 172.21.15.47:0/3498481427 conn(0x7fd9540f2820 msgr2=0x7fd9540f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:52.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.807+0000 7fd958940700 1 -- 172.21.15.47:0/3498481427 shutdown_connections 2024-02-23T00:55:52.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.810+0000 7fd958940700 1 -- 172.21.15.47:0/3498481427 wait complete. 2024-02-23T00:55:52.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.813+0000 7faeba5db700 10 librados: watch_flush enter 2024-02-23T00:55:52.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.813+0000 7faeba5db700 10 librados: watch_flush exit 2024-02-23T00:55:52.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.813+0000 7faeba5db700 20 client.42170.objecter shutdown clearing up homeless session... 2024-02-23T00:55:52.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.813+0000 7faeba5db700 10 client.42170.objecter successfully canceled tick 2024-02-23T00:55:52.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.813+0000 7faeba5db700 1 -- 172.21.15.47:0/1570938448 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fae9c062cc0 msgr2=0x7fae9c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:52.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.813+0000 7faeba5db700 1 --2- 172.21.15.47:0/1570938448 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fae9c062cc0 0x7fae9c065180 secure :-1 s=READY pgs=3212 cs=0 l=1 rev1=1 rx=0x7faea8007fd0 tx=0x7faea800b040).stop 2024-02-23T00:55:52.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.814+0000 7faeba5db700 1 -- 172.21.15.47:0/1570938448 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faeb40f7330 msgr2=0x7faeb4110120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:52.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.814+0000 7faeba5db700 1 --2- 172.21.15.47:0/1570938448 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faeb40f7330 0x7faeb4110120 secure :-1 s=READY pgs=2824 cs=0 l=1 rev1=1 rx=0x7faea4009da0 tx=0x7faea4009cb0).stop 2024-02-23T00:55:52.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.814+0000 7fd958940700 1 librados: shutdown 2024-02-23T00:55:52.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.816+0000 7faeba5db700 1 -- 172.21.15.47:0/1570938448 shutdown_connections 2024-02-23T00:55:52.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.816+0000 7faeba5db700 1 --2- 172.21.15.47:0/1570938448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faeb40fd420 0x7faeb410a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:52.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.816+0000 7faeba5db700 1 --2- 172.21.15.47:0/1570938448 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fae9c062cc0 0x7fae9c065180 unknown :-1 s=CLOSED pgs=3212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:52.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.816+0000 7faeba5db700 1 --2- 172.21.15.47:0/1570938448 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faeb40f7330 0x7faeb4110120 unknown :-1 s=CLOSED pgs=2824 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:52.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.816+0000 7faeba5db700 1 --2- 172.21.15.47:0/1570938448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faeb40fcac0 0x7faeb4110610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:52.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.816+0000 7faeba5db700 1 -- 172.21.15.47:0/1570938448 >> 172.21.15.47:0/1570938448 conn(0x7faeb40f26f0 msgr2=0x7faeb4060e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:52.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.816+0000 7faeba5db700 1 -- 172.21.15.47:0/1570938448 shutdown_connections 2024-02-23T00:55:52.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.816+0000 7faeba5db700 1 -- 172.21.15.47:0/1570938448 wait complete. 2024-02-23T00:55:52.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:52.817+0000 7faeba5db700 1 librados: shutdown 2024-02-23T00:55:52.819 INFO:teuthology.orchestra.run.smithi047.stderr:set osd.1 primary-affinity to 0 (802) 2024-02-23T00:55:52.819 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is unset 2024-02-23T00:55:52.888 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:55:52.891 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:52.891 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:55:52.992 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:55:53.003 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:51.794616+0000 mon.a (mon.0) 3775 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:55:53.003 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:51.794785+0000 mon.a (mon.0) 3776 : cluster [DBG] osdmap e796: 8 total, 8 up, 6 in 2024-02-23T00:55:53.003 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:51.816319+0000 mon.c (mon.2) 501 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:55:53.003 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:51.816638+0000 mon.a (mon.0) 3777 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:55:53.003 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:51.978495+0000 osd.6 (osd.6) 313 : cluster [DBG] 234.6 scrub starts 2024-02-23T00:55:53.003 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:51.980256+0000 osd.6 (osd.6) 314 : cluster [DBG] 234.6 scrub ok 2024-02-23T00:55:53.004 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:51.989684+0000 osd.0 (osd.0) 299 : cluster [DBG] 2.1f scrub starts 2024-02-23T00:55:53.004 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:51.990521+0000 osd.0 (osd.0) 300 : cluster [DBG] 2.1f scrub ok 2024-02-23T00:55:53.004 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:52.316611+0000 mon.c (mon.2) 502 : audit [INF] from='client.? 172.21.15.47:0/1570938448' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:55:53.004 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:52.316924+0000 mon.a (mon.0) 3778 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:55:53.004 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:52.532560+0000 mon.c (mon.2) 503 : audit [INF] from='client.? 172.21.15.47:0/3498481427' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 0.0}]: dispatch 2024-02-23T00:55:53.004 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:52.532943+0000 mon.a (mon.0) 3779 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 0.0}]: dispatch 2024-02-23T00:55:53.021 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:53.041 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:55:53.043 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:53.043 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:55:53.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:51.794616+0000 mon.a (mon.0) 3775 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:55:53.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:51.794785+0000 mon.a (mon.0) 3776 : cluster [DBG] osdmap e796: 8 total, 8 up, 6 in 2024-02-23T00:55:53.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:51.816319+0000 mon.c (mon.2) 501 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:55:53.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:51.816638+0000 mon.a (mon.0) 3777 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:55:53.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:51.978495+0000 osd.6 (osd.6) 313 : cluster [DBG] 234.6 scrub starts 2024-02-23T00:55:53.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:51.980256+0000 osd.6 (osd.6) 314 : cluster [DBG] 234.6 scrub ok 2024-02-23T00:55:53.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:51.989684+0000 osd.0 (osd.0) 299 : cluster [DBG] 2.1f scrub starts 2024-02-23T00:55:53.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:51.990521+0000 osd.0 (osd.0) 300 : cluster [DBG] 2.1f scrub ok 2024-02-23T00:55:53.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:52.316611+0000 mon.c (mon.2) 502 : audit [INF] from='client.? 172.21.15.47:0/1570938448' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:55:53.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:52.316924+0000 mon.a (mon.0) 3778 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:55:53.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:52.532560+0000 mon.c (mon.2) 503 : audit [INF] from='client.? 172.21.15.47:0/3498481427' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 0.0}]: dispatch 2024-02-23T00:55:53.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:52.532943+0000 mon.a (mon.0) 3779 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 0.0}]: dispatch 2024-02-23T00:55:53.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:51.794616+0000 mon.a (mon.0) 3775 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:55:53.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:51.794785+0000 mon.a (mon.0) 3776 : cluster [DBG] osdmap e796: 8 total, 8 up, 6 in 2024-02-23T00:55:53.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:51.816319+0000 mon.c (mon.2) 501 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:55:53.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:51.816638+0000 mon.a (mon.0) 3777 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-23T00:55:53.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:51.978495+0000 osd.6 (osd.6) 313 : cluster [DBG] 234.6 scrub starts 2024-02-23T00:55:53.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:51.980256+0000 osd.6 (osd.6) 314 : cluster [DBG] 234.6 scrub ok 2024-02-23T00:55:53.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:51.989684+0000 osd.0 (osd.0) 299 : cluster [DBG] 2.1f scrub starts 2024-02-23T00:55:53.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:51.990521+0000 osd.0 (osd.0) 300 : cluster [DBG] 2.1f scrub ok 2024-02-23T00:55:53.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:52.316611+0000 mon.c (mon.2) 502 : audit [INF] from='client.? 172.21.15.47:0/1570938448' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:55:53.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:52.316924+0000 mon.a (mon.0) 3778 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:55:53.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:52.532560+0000 mon.c (mon.2) 503 : audit [INF] from='client.? 172.21.15.47:0/3498481427' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 0.0}]: dispatch 2024-02-23T00:55:53.060 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:52.532943+0000 mon.a (mon.0) 3779 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 0.0}]: dispatch 2024-02-23T00:55:53.144 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:55:53.166 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-23T00:55:53.166 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-02-23T00:55:53.166 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-02-23T00:55:53.166 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-02-23T00:55:53.166 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-23T00:55:53.175 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:53.228 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:55:53.237 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:53.238 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:55:53.339 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:55:53.367 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:53.444 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:55:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:55:53.567 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:55:53.570 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:53.570 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:55:53.671 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:55:53.703 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:53.802 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:55:53.813 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:53.814 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:55:53.818 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.7 dump_historic_ops 2024-02-23T00:55:53.914 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:55:53.940 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:53.960 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:55:53.963 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:53.963 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:55:54.064 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:55:54.089 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:55:54.090 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:52.299332+0000 osd.5 (osd.5) 344 : cluster [DBG] 18.9 scrub starts 2024-02-23T00:55:54.090 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:52.300293+0000 osd.5 (osd.5) 345 : cluster [DBG] 18.9 scrub ok 2024-02-23T00:55:54.090 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:52.731440+0000 osd.1 (osd.1) 312 : cluster [DBG] 5.17 scrub starts 2024-02-23T00:55:54.090 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:52.732280+0000 osd.1 (osd.1) 313 : cluster [DBG] 5.17 scrub ok 2024-02-23T00:55:54.090 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:52.791396+0000 mon.a (mon.0) 3780 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:55:54.090 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:52.795544+0000 mon.a (mon.0) 3781 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:55:54.090 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:52.795645+0000 mon.a (mon.0) 3782 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:55:54.090 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:52.795724+0000 mon.a (mon.0) 3783 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 0.0}]': finished 2024-02-23T00:55:54.091 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:52.795790+0000 mon.a (mon.0) 3784 : cluster [DBG] osdmap e797: 8 total, 8 up, 6 in 2024-02-23T00:55:54.091 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:53.000801+0000 mgr.y (mgr.14517) 675 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:55:54.091 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:53.016254+0000 osd.6 (osd.6) 315 : cluster [DBG] 18.a scrub starts 2024-02-23T00:55:54.091 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:53.017060+0000 osd.6 (osd.6) 316 : cluster [DBG] 18.a scrub ok 2024-02-23T00:55:54.091 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:53.269183+0000 osd.5 (osd.5) 346 : cluster [DBG] 18.7 scrub starts 2024-02-23T00:55:54.091 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:53.272784+0000 osd.5 (osd.5) 347 : cluster [DBG] 18.7 scrub ok 2024-02-23T00:55:54.091 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:53.407588+0000 mgr.y (mgr.14517) 676 : cluster [DBG] pgmap v1213: 194 pgs: 194 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 255 B/s rd, 0 op/s 2024-02-23T00:55:54.091 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:53.796841+0000 mon.a (mon.0) 3785 : cluster [DBG] osdmap e798: 8 total, 8 up, 6 in 2024-02-23T00:55:54.092 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:54.113 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:55:54.116 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:54.116 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:55:54.217 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:55:54.245 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:52.299332+0000 osd.5 (osd.5) 344 : cluster [DBG] 18.9 scrub starts 2024-02-23T00:55:54.245 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:52.300293+0000 osd.5 (osd.5) 345 : cluster [DBG] 18.9 scrub ok 2024-02-23T00:55:54.245 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:52.731440+0000 osd.1 (osd.1) 312 : cluster [DBG] 5.17 scrub starts 2024-02-23T00:55:54.245 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:52.732280+0000 osd.1 (osd.1) 313 : cluster [DBG] 5.17 scrub ok 2024-02-23T00:55:54.245 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:52.791396+0000 mon.a (mon.0) 3780 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:55:54.245 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:52.795544+0000 mon.a (mon.0) 3781 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:55:54.246 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:52.795645+0000 mon.a (mon.0) 3782 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:55:54.246 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:52.795724+0000 mon.a (mon.0) 3783 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 0.0}]': finished 2024-02-23T00:55:54.246 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:52.795790+0000 mon.a (mon.0) 3784 : cluster [DBG] osdmap e797: 8 total, 8 up, 6 in 2024-02-23T00:55:54.246 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:53.000801+0000 mgr.y (mgr.14517) 675 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:55:54.246 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:53.016254+0000 osd.6 (osd.6) 315 : cluster [DBG] 18.a scrub starts 2024-02-23T00:55:54.246 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:53.017060+0000 osd.6 (osd.6) 316 : cluster [DBG] 18.a scrub ok 2024-02-23T00:55:54.246 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:53.269183+0000 osd.5 (osd.5) 346 : cluster [DBG] 18.7 scrub starts 2024-02-23T00:55:54.246 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:53.272784+0000 osd.5 (osd.5) 347 : cluster [DBG] 18.7 scrub ok 2024-02-23T00:55:54.246 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:53.407588+0000 mgr.y (mgr.14517) 676 : cluster [DBG] pgmap v1213: 194 pgs: 194 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 255 B/s rd, 0 op/s 2024-02-23T00:55:54.246 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:53.796841+0000 mon.a (mon.0) 3785 : cluster [DBG] osdmap e798: 8 total, 8 up, 6 in 2024-02-23T00:55:54.246 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:52.299332+0000 osd.5 (osd.5) 344 : cluster [DBG] 18.9 scrub starts 2024-02-23T00:55:54.247 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:52.300293+0000 osd.5 (osd.5) 345 : cluster [DBG] 18.9 scrub ok 2024-02-23T00:55:54.247 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:52.731440+0000 osd.1 (osd.1) 312 : cluster [DBG] 5.17 scrub starts 2024-02-23T00:55:54.247 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:52.732280+0000 osd.1 (osd.1) 313 : cluster [DBG] 5.17 scrub ok 2024-02-23T00:55:54.247 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:52.791396+0000 mon.a (mon.0) 3780 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:55:54.247 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:52.795544+0000 mon.a (mon.0) 3781 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-23T00:55:54.247 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:52.795645+0000 mon.a (mon.0) 3782 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:55:54.247 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:52.795724+0000 mon.a (mon.0) 3783 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 0.0}]': finished 2024-02-23T00:55:54.247 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:52.795790+0000 mon.a (mon.0) 3784 : cluster [DBG] osdmap e797: 8 total, 8 up, 6 in 2024-02-23T00:55:54.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:53.000801+0000 mgr.y (mgr.14517) 675 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:55:54.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:53.016254+0000 osd.6 (osd.6) 315 : cluster [DBG] 18.a scrub starts 2024-02-23T00:55:54.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:53.017060+0000 osd.6 (osd.6) 316 : cluster [DBG] 18.a scrub ok 2024-02-23T00:55:54.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:53.269183+0000 osd.5 (osd.5) 346 : cluster [DBG] 18.7 scrub starts 2024-02-23T00:55:54.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:53.272784+0000 osd.5 (osd.5) 347 : cluster [DBG] 18.7 scrub ok 2024-02-23T00:55:54.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:53.407588+0000 mgr.y (mgr.14517) 676 : cluster [DBG] pgmap v1213: 194 pgs: 194 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 255 B/s rd, 0 op/s 2024-02-23T00:55:54.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:53.796841+0000 mon.a (mon.0) 3785 : cluster [DBG] osdmap e798: 8 total, 8 up, 6 in 2024-02-23T00:55:54.248 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:54.453 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:55:54.455 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:54.456 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:55:54.557 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:55:54.588 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:54.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.663+0000 7fafecf2c700 1 -- 172.21.15.47:0/172796326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafe80f8cb0 msgr2=0x7fafe80f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:54.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.663+0000 7fafecf2c700 1 --2- 172.21.15.47:0/172796326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafe80f8cb0 0x7fafe80f9090 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 rx=0x7fafd00047b0 tx=0x7fafd0009cb0).stop 2024-02-23T00:55:54.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.663+0000 7fafecf2c700 1 -- 172.21.15.47:0/172796326 shutdown_connections 2024-02-23T00:55:54.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.663+0000 7fafecf2c700 1 --2- 172.21.15.47:0/172796326 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fafe8102210 0x7fafe8104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:54.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.663+0000 7fafecf2c700 1 --2- 172.21.15.47:0/172796326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafe80f8cb0 0x7fafe80f9090 unknown :-1 s=CLOSED pgs=2647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:54.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.663+0000 7fafecf2c700 1 --2- 172.21.15.47:0/172796326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafe80f9610 0x7fafe8101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:54.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.663+0000 7fafecf2c700 1 -- 172.21.15.47:0/172796326 >> 172.21.15.47:0/172796326 conn(0x7fafe80f2820 msgr2=0x7fafe80f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:54.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.663+0000 7fafecf2c700 1 -- 172.21.15.47:0/172796326 shutdown_connections 2024-02-23T00:55:54.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.663+0000 7fafecf2c700 1 -- 172.21.15.47:0/172796326 wait complete. 2024-02-23T00:55:54.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.664+0000 7fafecf2c700 1 librados: starting msgr at 2024-02-23T00:55:54.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.664+0000 7fafecf2c700 1 librados: starting objecter 2024-02-23T00:55:54.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.664+0000 7fafecf2c700 1 Processor -- start 2024-02-23T00:55:54.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.664+0000 7fafecf2c700 1 -- start start 2024-02-23T00:55:54.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.664+0000 7fafecf2c700 1 librados: setting wanted keys 2024-02-23T00:55:54.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.664+0000 7fafecf2c700 1 librados: calling monclient init 2024-02-23T00:55:54.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.664+0000 7fafecf2c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafe80f8cb0 0x7fafe80f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:54.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.664+0000 7fafecf2c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafe80f9610 0x7fafe80f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:54.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.664+0000 7fafecf2c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fafe8102210 0x7fafe80f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:54.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.664+0000 7fafecf2c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe81165d0 con 0x7fafe80f8cb0 2024-02-23T00:55:54.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.664+0000 7fafecf2c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe8116740 con 0x7fafe80f9610 2024-02-23T00:55:54.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.664+0000 7fafecf2c700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafe8116900 con 0x7fafe8102210 2024-02-23T00:55:54.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.664+0000 7fafe659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafe80f8cb0 0x7fafe80f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:54.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.664+0000 7fafe659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafe80f8cb0 0x7fafe80f6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34758/0 (socket says 172.21.15.47:34758) 2024-02-23T00:55:54.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.664+0000 7fafe659c700 1 -- 172.21.15.47:0/2249953117 learned_addr learned my addr 172.21.15.47:0/2249953117 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:55:54.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.664+0000 7fafe5d9b700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafe80f9610 0x7fafe80f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:54.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.665+0000 7fafe6d9d700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fafe8102210 0x7fafe80f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:54.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.665+0000 7fafe659c700 1 -- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fafe8102210 msgr2=0x7fafe80f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:54.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.665+0000 7fafe659c700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fafe8102210 0x7fafe80f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:54.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.665+0000 7fafe659c700 1 -- 172.21.15.47:0/2249953117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafe80f9610 msgr2=0x7fafe80f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:54.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.665+0000 7fafe659c700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafe80f9610 0x7fafe80f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:54.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.665+0000 7fafe659c700 1 -- 172.21.15.47:0/2249953117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe00097c0 con 0x7fafe80f8cb0 2024-02-23T00:55:54.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.665+0000 7fafe659c700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafe80f8cb0 0x7fafe80f6c60 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 rx=0x7fafd0009f20 tx=0x7fafd0009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:54.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.665+0000 7fafdf7fe700 10 client.?.objecter ms_handle_connect 0x7fafe80f8cb0 2024-02-23T00:55:54.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.665+0000 7fafdf7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:55:54.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.665+0000 7fafdf7fe700 1 -- 172.21.15.47:0/2249953117 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd0018070 con 0x7fafe80f8cb0 2024-02-23T00:55:54.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.665+0000 7fafdf7fe700 1 -- 172.21.15.47:0/2249953117 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fafd0013e50 con 0x7fafe80f8cb0 2024-02-23T00:55:54.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.665+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafd0005040 con 0x7fafe80f8cb0 2024-02-23T00:55:54.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.665+0000 7fafdf7fe700 1 -- 172.21.15.47:0/2249953117 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd001dae0 con 0x7fafe80f8cb0 2024-02-23T00:55:54.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.665+0000 7fafecf2c700 10 client.31857.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:55:54.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.665+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafe8116df0 con 0x7fafe80f8cb0 2024-02-23T00:55:54.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.665+0000 7fafecf2c700 1 librados: init done 2024-02-23T00:55:54.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.666+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fafe805fa60 con 0x7fafe80f8cb0 2024-02-23T00:55:54.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.667+0000 7fafdf7fe700 1 -- 172.21.15.47:0/2249953117 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fafd0009240 con 0x7fafe80f8cb0 2024-02-23T00:55:54.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.667+0000 7fafdf7fe700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fafd4062cc0 0x7fafd4065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:54.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.667+0000 7fafdf7fe700 1 -- 172.21.15.47:0/2249953117 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(798..798 src has 621..798) v4 ==== 9806+0+0 (secure 0 0 0) 0x7fafd007f2d0 con 0x7fafe80f8cb0 2024-02-23T00:55:54.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.667+0000 7fafdf7fe700 10 client.31857.objecter ms_dispatch 0x7fafe8001550 osd_map(798..798 src has 621..798) v4 2024-02-23T00:55:54.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.667+0000 7fafdf7fe700 3 client.31857.objecter handle_osd_map got epochs [798,798] > 0 2024-02-23T00:55:54.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.667+0000 7fafdf7fe700 3 client.31857.objecter handle_osd_map decoding full epoch 798 2024-02-23T00:55:54.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.667+0000 7fafe5d9b700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fafd4062cc0 0x7fafd4065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:54.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.668+0000 7fafdf7fe700 20 client.31857.objecter dump_active .. 0 homeless 2024-02-23T00:55:54.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.668+0000 7fafdf7fe700 1 -- 172.21.15.47:0/2249953117 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v798) v1 ==== 54+0+16 (secure 0 0 0) 0x7fafd007f7d0 con 0x7fafe80f8cb0 2024-02-23T00:55:54.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.668+0000 7fafecf2c700 10 client.31857.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:54.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.668+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_assign -1 1 2024-02-23T00:55:54.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.668+0000 7fafdf7fe700 10 client.31857.objecter ms_handle_connect 0x7fafd4062cc0 2024-02-23T00:55:54.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.668+0000 7fafe5d9b700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fafd4062cc0 0x7fafd4065180 secure :-1 s=READY pgs=3214 cs=0 l=1 rev1=1 rx=0x7fafe80f8400 tx=0x7fafd8009040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:54.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.668+0000 7fafecf2c700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fafe8028540 0x7fafe80fbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:54.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.668+0000 7fafecf2c700 20 client.31857.objecter _get_session s=0x7fafe805ee80 osd=0 3 2024-02-23T00:55:54.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.668+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe805ee80 osd=0 3 2024-02-23T00:55:54.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.668+0000 7fafecf2c700 20 client.31857.objecter _get_session s=0x7fafe805ee80 osd=0 3 2024-02-23T00:55:54.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.668+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_remove -1 1 2024-02-23T00:55:54.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.668+0000 7fafecf2c700 20 client.31857.objecter get_session s=0x7fafe805ee80 osd=0 3 2024-02-23T00:55:54.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.668+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_assign 0 1 2024-02-23T00:55:54.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.668+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe805ee80 osd=0 4 2024-02-23T00:55:54.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.668+0000 7fafecf2c700 10 client.31857.objecter _send_command 1 2024-02-23T00:55:54.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.668+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fafe81070a0 con 0x7fafe8028540 2024-02-23T00:55:54.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.668+0000 7fafe6d9d700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fafe8028540 0x7fafe80fbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:54.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.668+0000 7fafdf7fe700 10 client.31857.objecter ms_handle_connect 0x7fafe8028540 2024-02-23T00:55:54.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.668+0000 7fafe6d9d700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fafe8028540 0x7fafe80fbcd0 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:54.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.669+0000 7fafdf7fe700 1 -- 172.21.15.47:0/2249953117 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fafe81070a0 con 0x7fafe8028540 2024-02-23T00:55:54.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.669+0000 7fafdf7fe700 10 client.31857.objecter ms_dispatch 0x7fafe8001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:55:54.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.669+0000 7fafdf7fe700 10 client.31857.objecter _finish_command 1 = system:0 2024-02-23T00:55:54.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.669+0000 7fafdf7fe700 20 client.31857.objecter put_session s=0x7fafe805ee80 osd=0 4 2024-02-23T00:55:54.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.669+0000 7fafdf7fe700 15 client.31857.objecter _session_command_op_remove 0 1 2024-02-23T00:55:54.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.684+0000 7fafecf2c700 10 client.31857.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:55:54.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.684+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_assign -1 2 2024-02-23T00:55:54.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.684+0000 7fafecf2c700 20 client.31857.objecter _get_session s=0x7fafe805ee80 osd=0 3 2024-02-23T00:55:54.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.684+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe805ee80 osd=0 3 2024-02-23T00:55:54.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.684+0000 7fafecf2c700 20 client.31857.objecter _get_session s=0x7fafe805ee80 osd=0 3 2024-02-23T00:55:54.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.684+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_remove -1 2 2024-02-23T00:55:54.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.684+0000 7fafecf2c700 20 client.31857.objecter get_session s=0x7fafe805ee80 osd=0 3 2024-02-23T00:55:54.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.684+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_assign 0 2 2024-02-23T00:55:54.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.684+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe805ee80 osd=0 4 2024-02-23T00:55:54.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.684+0000 7fafecf2c700 10 client.31857.objecter _send_command 2 2024-02-23T00:55:54.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.684+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fafe805a040 con 0x7fafe8028540 2024-02-23T00:55:54.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.685+0000 7fafdf7fe700 1 -- 172.21.15.47:0/2249953117 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fafe805a040 con 0x7fafe8028540 2024-02-23T00:55:54.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.685+0000 7fafdf7fe700 10 client.31857.objecter ms_dispatch 0x7fafe8001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:55:54.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.685+0000 7fafdf7fe700 10 client.31857.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:55:54.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.685+0000 7fafdf7fe700 20 client.31857.objecter put_session s=0x7fafe805ee80 osd=0 4 2024-02-23T00:55:54.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.685+0000 7fafdf7fe700 15 client.31857.objecter _session_command_op_remove 0 2 2024-02-23T00:55:54.688 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:55:54.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.685+0000 7fafecf2c700 10 client.31857.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:54.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.686+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_assign -1 3 2024-02-23T00:55:54.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.686+0000 7fafecf2c700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fafe8027b90 0x7fafe8027fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:54.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.686+0000 7fafecf2c700 20 client.31857.objecter _get_session s=0x7fafe80fd340 osd=1 3 2024-02-23T00:55:54.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.686+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe80fd340 osd=1 3 2024-02-23T00:55:54.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.686+0000 7fafecf2c700 20 client.31857.objecter _get_session s=0x7fafe80fd340 osd=1 3 2024-02-23T00:55:54.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.686+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_remove -1 3 2024-02-23T00:55:54.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.686+0000 7fafecf2c700 20 client.31857.objecter get_session s=0x7fafe80fd340 osd=1 3 2024-02-23T00:55:54.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.686+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_assign 1 3 2024-02-23T00:55:54.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.686+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe80fd340 osd=1 4 2024-02-23T00:55:54.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.686+0000 7fafecf2c700 10 client.31857.objecter _send_command 3 2024-02-23T00:55:54.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.686+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fafe80fd4b0 con 0x7fafe8027b90 2024-02-23T00:55:54.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.686+0000 7fafe659c700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fafe8027b90 0x7fafe8027fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:54.689 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:55:54.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.686+0000 7fafdf7fe700 10 client.31857.objecter ms_handle_connect 0x7fafe8027b90 2024-02-23T00:55:54.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.686+0000 7fafe659c700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fafe8027b90 0x7fafe8027fb0 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:54.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.687+0000 7fafdf7fe700 1 -- 172.21.15.47:0/2249953117 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fafe80fd4b0 con 0x7fafe8027b90 2024-02-23T00:55:54.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.687+0000 7fafdf7fe700 10 client.31857.objecter ms_dispatch 0x7fafe8001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:55:54.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.687+0000 7fafdf7fe700 10 client.31857.objecter _finish_command 3 = system:0 2024-02-23T00:55:54.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.687+0000 7fafdf7fe700 20 client.31857.objecter put_session s=0x7fafe80fd340 osd=1 4 2024-02-23T00:55:54.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.687+0000 7fafdf7fe700 15 client.31857.objecter _session_command_op_remove 1 3 2024-02-23T00:55:54.691 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:54.691 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:55:54.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.702+0000 7fafecf2c700 10 client.31857.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:55:54.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.702+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_assign -1 4 2024-02-23T00:55:54.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.702+0000 7fafecf2c700 20 client.31857.objecter _get_session s=0x7fafe80fd340 osd=1 3 2024-02-23T00:55:54.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.702+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe80fd340 osd=1 3 2024-02-23T00:55:54.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.702+0000 7fafecf2c700 20 client.31857.objecter _get_session s=0x7fafe80fd340 osd=1 3 2024-02-23T00:55:54.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.702+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_remove -1 4 2024-02-23T00:55:54.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.702+0000 7fafecf2c700 20 client.31857.objecter get_session s=0x7fafe80fd340 osd=1 3 2024-02-23T00:55:54.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.702+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_assign 1 4 2024-02-23T00:55:54.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.702+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe80fd340 osd=1 4 2024-02-23T00:55:54.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.702+0000 7fafecf2c700 10 client.31857.objecter _send_command 4 2024-02-23T00:55:54.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.702+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fafe8027600 con 0x7fafe8027b90 2024-02-23T00:55:54.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.704+0000 7fafdf7fe700 1 -- 172.21.15.47:0/2249953117 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fafe8027600 con 0x7fafe8027b90 2024-02-23T00:55:54.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.704+0000 7fafdf7fe700 10 client.31857.objecter ms_dispatch 0x7fafe8001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:55:54.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.704+0000 7fafdf7fe700 10 client.31857.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:55:54.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.704+0000 7fafdf7fe700 20 client.31857.objecter put_session s=0x7fafe80fd340 osd=1 4 2024-02-23T00:55:54.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.704+0000 7fafdf7fe700 15 client.31857.objecter _session_command_op_remove 1 4 2024-02-23T00:55:54.706 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:55:54.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.704+0000 7fafecf2c700 10 client.31857.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:54.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.704+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_assign -1 5 2024-02-23T00:55:54.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.704+0000 7fafecf2c700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fafe80fcea0 0x7fafe811e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:54.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.704+0000 7fafecf2c700 20 client.31857.objecter _get_session s=0x7fafe8027a10 osd=2 3 2024-02-23T00:55:54.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.704+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe8027a10 osd=2 3 2024-02-23T00:55:54.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.704+0000 7fafecf2c700 20 client.31857.objecter _get_session s=0x7fafe8027a10 osd=2 3 2024-02-23T00:55:54.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.704+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_remove -1 5 2024-02-23T00:55:54.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.704+0000 7fafecf2c700 20 client.31857.objecter get_session s=0x7fafe8027a10 osd=2 3 2024-02-23T00:55:54.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.704+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_assign 2 5 2024-02-23T00:55:54.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.704+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe8027a10 osd=2 4 2024-02-23T00:55:54.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.704+0000 7fafecf2c700 10 client.31857.objecter _send_command 5 2024-02-23T00:55:54.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.704+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fafe8120e60 con 0x7fafe80fcea0 2024-02-23T00:55:54.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.704+0000 7fafe5d9b700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fafe80fcea0 0x7fafe811e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:54.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.704+0000 7fafdf7fe700 10 client.31857.objecter ms_handle_connect 0x7fafe80fcea0 2024-02-23T00:55:54.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.704+0000 7fafe5d9b700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fafe80fcea0 0x7fafe811e130 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:54.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.705+0000 7fafdf7fe700 1 -- 172.21.15.47:0/2249953117 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fafe8120e60 con 0x7fafe80fcea0 2024-02-23T00:55:54.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.705+0000 7fafdf7fe700 10 client.31857.objecter ms_dispatch 0x7fafe8001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:55:54.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.705+0000 7fafdf7fe700 10 client.31857.objecter _finish_command 5 = system:0 2024-02-23T00:55:54.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.705+0000 7fafdf7fe700 20 client.31857.objecter put_session s=0x7fafe8027a10 osd=2 4 2024-02-23T00:55:54.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.705+0000 7fafdf7fe700 15 client.31857.objecter _session_command_op_remove 2 5 2024-02-23T00:55:54.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.719+0000 7fafecf2c700 10 client.31857.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:55:54.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.719+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_assign -1 6 2024-02-23T00:55:54.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.719+0000 7fafecf2c700 20 client.31857.objecter _get_session s=0x7fafe8027a10 osd=2 3 2024-02-23T00:55:54.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.719+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe8027a10 osd=2 3 2024-02-23T00:55:54.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.719+0000 7fafecf2c700 20 client.31857.objecter _get_session s=0x7fafe8027a10 osd=2 3 2024-02-23T00:55:54.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.719+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_remove -1 6 2024-02-23T00:55:54.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.719+0000 7fafecf2c700 20 client.31857.objecter get_session s=0x7fafe8027a10 osd=2 3 2024-02-23T00:55:54.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.719+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_assign 2 6 2024-02-23T00:55:54.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.719+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe8027a10 osd=2 4 2024-02-23T00:55:54.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.719+0000 7fafecf2c700 10 client.31857.objecter _send_command 6 2024-02-23T00:55:54.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.719+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fafe80fc400 con 0x7fafe80fcea0 2024-02-23T00:55:54.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.721+0000 7fafdf7fe700 1 -- 172.21.15.47:0/2249953117 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fafe80fc400 con 0x7fafe80fcea0 2024-02-23T00:55:54.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.721+0000 7fafdf7fe700 10 client.31857.objecter ms_dispatch 0x7fafe8001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:55:54.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.721+0000 7fafdf7fe700 10 client.31857.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:55:54.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.721+0000 7fafdf7fe700 20 client.31857.objecter put_session s=0x7fafe8027a10 osd=2 4 2024-02-23T00:55:54.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.721+0000 7fafdf7fe700 15 client.31857.objecter _session_command_op_remove 2 6 2024-02-23T00:55:54.723 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:55:54.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.721+0000 7fafecf2c700 10 client.31857.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:54.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.721+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_assign -1 7 2024-02-23T00:55:54.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.721+0000 7fafecf2c700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fafe811bcd0 0x7fafe811f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:54.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.721+0000 7fafecf2c700 20 client.31857.objecter _get_session s=0x7fafe80fc810 osd=3 3 2024-02-23T00:55:54.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.721+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe80fc810 osd=3 3 2024-02-23T00:55:54.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.721+0000 7fafecf2c700 20 client.31857.objecter _get_session s=0x7fafe80fc810 osd=3 3 2024-02-23T00:55:54.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.721+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_remove -1 7 2024-02-23T00:55:54.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.721+0000 7fafecf2c700 20 client.31857.objecter get_session s=0x7fafe80fc810 osd=3 3 2024-02-23T00:55:54.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.721+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_assign 3 7 2024-02-23T00:55:54.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.721+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe80fc810 osd=3 4 2024-02-23T00:55:54.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.721+0000 7fafecf2c700 10 client.31857.objecter _send_command 7 2024-02-23T00:55:54.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.721+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fafe811fe80 con 0x7fafe811bcd0 2024-02-23T00:55:54.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.721+0000 7fafe6d9d700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fafe811bcd0 0x7fafe811f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:54.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.722+0000 7fafdf7fe700 10 client.31857.objecter ms_handle_connect 0x7fafe811bcd0 2024-02-23T00:55:54.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.722+0000 7fafe6d9d700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fafe811bcd0 0x7fafe811f7e0 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:54.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.722+0000 7fafdf7fe700 1 -- 172.21.15.47:0/2249953117 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fafe811fe80 con 0x7fafe811bcd0 2024-02-23T00:55:54.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.722+0000 7fafdf7fe700 10 client.31857.objecter ms_dispatch 0x7fafe8001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:55:54.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.722+0000 7fafdf7fe700 10 client.31857.objecter _finish_command 7 = system:0 2024-02-23T00:55:54.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.722+0000 7fafdf7fe700 20 client.31857.objecter put_session s=0x7fafe80fc810 osd=3 4 2024-02-23T00:55:54.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.722+0000 7fafdf7fe700 15 client.31857.objecter _session_command_op_remove 3 7 2024-02-23T00:55:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.737+0000 7fafecf2c700 10 client.31857.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:55:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.737+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_assign -1 8 2024-02-23T00:55:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.737+0000 7fafecf2c700 20 client.31857.objecter _get_session s=0x7fafe80fc810 osd=3 3 2024-02-23T00:55:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.737+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe80fc810 osd=3 3 2024-02-23T00:55:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.737+0000 7fafecf2c700 20 client.31857.objecter _get_session s=0x7fafe80fc810 osd=3 3 2024-02-23T00:55:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.737+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_remove -1 8 2024-02-23T00:55:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.737+0000 7fafecf2c700 20 client.31857.objecter get_session s=0x7fafe80fc810 osd=3 3 2024-02-23T00:55:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.737+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_assign 3 8 2024-02-23T00:55:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.737+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe80fc810 osd=3 4 2024-02-23T00:55:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.737+0000 7fafecf2c700 10 client.31857.objecter _send_command 8 2024-02-23T00:55:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.737+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fafe8120850 con 0x7fafe811bcd0 2024-02-23T00:55:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.738+0000 7fafdf7fe700 1 -- 172.21.15.47:0/2249953117 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fafe8120850 con 0x7fafe811bcd0 2024-02-23T00:55:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.738+0000 7fafdf7fe700 10 client.31857.objecter ms_dispatch 0x7fafe8001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:55:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.738+0000 7fafdf7fe700 10 client.31857.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:55:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.738+0000 7fafdf7fe700 20 client.31857.objecter put_session s=0x7fafe80fc810 osd=3 4 2024-02-23T00:55:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.738+0000 7fafdf7fe700 15 client.31857.objecter _session_command_op_remove 3 8 2024-02-23T00:55:54.740 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:55:54.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.738+0000 7fafecf2c700 10 client.31857.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:54.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.738+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_assign -1 9 2024-02-23T00:55:54.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.738+0000 7fafecf2c700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fafe8122fe0 0x7fafe81253d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:54.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.738+0000 7fafecf2c700 20 client.31857.objecter _get_session s=0x7fafe8120080 osd=4 3 2024-02-23T00:55:54.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.738+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe8120080 osd=4 3 2024-02-23T00:55:54.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.738+0000 7fafecf2c700 20 client.31857.objecter _get_session s=0x7fafe8120080 osd=4 3 2024-02-23T00:55:54.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.738+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_remove -1 9 2024-02-23T00:55:54.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.738+0000 7fafecf2c700 20 client.31857.objecter get_session s=0x7fafe8120080 osd=4 3 2024-02-23T00:55:54.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.738+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_assign 4 9 2024-02-23T00:55:54.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.738+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe8120080 osd=4 4 2024-02-23T00:55:54.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.738+0000 7fafecf2c700 10 client.31857.objecter _send_command 9 2024-02-23T00:55:54.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.738+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fafe8120810 con 0x7fafe8122fe0 2024-02-23T00:55:54.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.739+0000 7fafe659c700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fafe8122fe0 0x7fafe81253d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:54.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.739+0000 7fafdf7fe700 10 client.31857.objecter ms_handle_connect 0x7fafe8122fe0 2024-02-23T00:55:54.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.739+0000 7fafe659c700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fafe8122fe0 0x7fafe81253d0 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:54.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.740+0000 7fafdf7fe700 1 -- 172.21.15.47:0/2249953117 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fafe8120810 con 0x7fafe8122fe0 2024-02-23T00:55:54.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.740+0000 7fafdf7fe700 10 client.31857.objecter ms_dispatch 0x7fafe8001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:55:54.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.740+0000 7fafdf7fe700 10 client.31857.objecter _finish_command 9 = system:0 2024-02-23T00:55:54.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.740+0000 7fafdf7fe700 20 client.31857.objecter put_session s=0x7fafe8120080 osd=4 4 2024-02-23T00:55:54.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.740+0000 7fafdf7fe700 15 client.31857.objecter _session_command_op_remove 4 9 2024-02-23T00:55:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.755+0000 7fafecf2c700 10 client.31857.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:55:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.755+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_assign -1 10 2024-02-23T00:55:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.755+0000 7fafecf2c700 20 client.31857.objecter _get_session s=0x7fafe8120080 osd=4 3 2024-02-23T00:55:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.755+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe8120080 osd=4 3 2024-02-23T00:55:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.755+0000 7fafecf2c700 20 client.31857.objecter _get_session s=0x7fafe8120080 osd=4 3 2024-02-23T00:55:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.755+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_remove -1 10 2024-02-23T00:55:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.755+0000 7fafecf2c700 20 client.31857.objecter get_session s=0x7fafe8120080 osd=4 3 2024-02-23T00:55:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.755+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_assign 4 10 2024-02-23T00:55:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.755+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe8120080 osd=4 4 2024-02-23T00:55:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.755+0000 7fafecf2c700 10 client.31857.objecter _send_command 10 2024-02-23T00:55:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.755+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fafe8120810 con 0x7fafe8122fe0 2024-02-23T00:55:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.756+0000 7fafdf7fe700 1 -- 172.21.15.47:0/2249953117 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fafe8120810 con 0x7fafe8122fe0 2024-02-23T00:55:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.756+0000 7fafdf7fe700 10 client.31857.objecter ms_dispatch 0x7fafe8001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:55:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.756+0000 7fafdf7fe700 10 client.31857.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:55:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.756+0000 7fafdf7fe700 20 client.31857.objecter put_session s=0x7fafe8120080 osd=4 4 2024-02-23T00:55:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.756+0000 7fafdf7fe700 15 client.31857.objecter _session_command_op_remove 4 10 2024-02-23T00:55:54.759 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:55:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.756+0000 7fafecf2c700 10 client.31857.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.756+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_assign -1 11 2024-02-23T00:55:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.756+0000 7fafecf2c700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fafe8125dd0 0x7fafe8128240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.756+0000 7fafecf2c700 20 client.31857.objecter _get_session s=0x7fafe8120810 osd=5 3 2024-02-23T00:55:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.756+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe8120810 osd=5 3 2024-02-23T00:55:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.756+0000 7fafecf2c700 20 client.31857.objecter _get_session s=0x7fafe8120810 osd=5 3 2024-02-23T00:55:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.757+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_remove -1 11 2024-02-23T00:55:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.757+0000 7fafecf2c700 20 client.31857.objecter get_session s=0x7fafe8120810 osd=5 3 2024-02-23T00:55:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.757+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_assign 5 11 2024-02-23T00:55:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.757+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe8120810 osd=5 4 2024-02-23T00:55:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.757+0000 7fafecf2c700 10 client.31857.objecter _send_command 11 2024-02-23T00:55:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.757+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fafe812b080 con 0x7fafe8125dd0 2024-02-23T00:55:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.757+0000 7fafe5d9b700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fafe8125dd0 0x7fafe8128240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.757+0000 7fafdf7fe700 10 client.31857.objecter ms_handle_connect 0x7fafe8125dd0 2024-02-23T00:55:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.757+0000 7fafe5d9b700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fafe8125dd0 0x7fafe8128240 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.758+0000 7fafdf7fe700 1 -- 172.21.15.47:0/2249953117 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fafe812b080 con 0x7fafe8125dd0 2024-02-23T00:55:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.758+0000 7fafdf7fe700 10 client.31857.objecter ms_dispatch 0x7fafe8001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:55:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.758+0000 7fafdf7fe700 10 client.31857.objecter _finish_command 11 = system:0 2024-02-23T00:55:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.758+0000 7fafdf7fe700 20 client.31857.objecter put_session s=0x7fafe8120810 osd=5 4 2024-02-23T00:55:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.758+0000 7fafdf7fe700 15 client.31857.objecter _session_command_op_remove 5 11 2024-02-23T00:55:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.773+0000 7fafecf2c700 10 client.31857.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:55:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.773+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_assign -1 12 2024-02-23T00:55:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.773+0000 7fafecf2c700 20 client.31857.objecter _get_session s=0x7fafe8120810 osd=5 3 2024-02-23T00:55:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.773+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe8120810 osd=5 3 2024-02-23T00:55:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.773+0000 7fafecf2c700 20 client.31857.objecter _get_session s=0x7fafe8120810 osd=5 3 2024-02-23T00:55:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.773+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_remove -1 12 2024-02-23T00:55:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.773+0000 7fafecf2c700 20 client.31857.objecter get_session s=0x7fafe8120810 osd=5 3 2024-02-23T00:55:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.773+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_assign 5 12 2024-02-23T00:55:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.773+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe8120810 osd=5 4 2024-02-23T00:55:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.773+0000 7fafecf2c700 10 client.31857.objecter _send_command 12 2024-02-23T00:55:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.773+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fafe812b080 con 0x7fafe8125dd0 2024-02-23T00:55:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.774+0000 7fafdf7fe700 1 -- 172.21.15.47:0/2249953117 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fafe812b080 con 0x7fafe8125dd0 2024-02-23T00:55:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.774+0000 7fafdf7fe700 10 client.31857.objecter ms_dispatch 0x7fafe8001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:55:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.774+0000 7fafdf7fe700 10 client.31857.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:55:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.774+0000 7fafdf7fe700 20 client.31857.objecter put_session s=0x7fafe8120810 osd=5 4 2024-02-23T00:55:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.774+0000 7fafdf7fe700 15 client.31857.objecter _session_command_op_remove 5 12 2024-02-23T00:55:54.777 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:55:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.775+0000 7fafecf2c700 10 client.31857.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.775+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_assign -1 13 2024-02-23T00:55:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.775+0000 7fafecf2c700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fafe812a3b0 0x7fafe812a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.775+0000 7fafecf2c700 20 client.31857.objecter _get_session s=0x7fafe812b080 osd=6 3 2024-02-23T00:55:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.775+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe812b080 osd=6 3 2024-02-23T00:55:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.775+0000 7fafecf2c700 20 client.31857.objecter _get_session s=0x7fafe812b080 osd=6 3 2024-02-23T00:55:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.775+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_remove -1 13 2024-02-23T00:55:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.775+0000 7fafecf2c700 20 client.31857.objecter get_session s=0x7fafe812b080 osd=6 3 2024-02-23T00:55:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.775+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_assign 6 13 2024-02-23T00:55:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.775+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe812b080 osd=6 4 2024-02-23T00:55:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.775+0000 7fafecf2c700 10 client.31857.objecter _send_command 13 2024-02-23T00:55:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.775+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fafe812e430 con 0x7fafe812a3b0 2024-02-23T00:55:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.775+0000 7fafe6d9d700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fafe812a3b0 0x7fafe812a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.775+0000 7fafdf7fe700 10 client.31857.objecter ms_handle_connect 0x7fafe812a3b0 2024-02-23T00:55:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.775+0000 7fafe6d9d700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fafe812a3b0 0x7fafe812a830 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.776+0000 7fafdf7fe700 1 -- 172.21.15.47:0/2249953117 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fafe812e430 con 0x7fafe812a3b0 2024-02-23T00:55:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.776+0000 7fafdf7fe700 10 client.31857.objecter ms_dispatch 0x7fafe8001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:55:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.776+0000 7fafdf7fe700 10 client.31857.objecter _finish_command 13 = system:0 2024-02-23T00:55:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.776+0000 7fafdf7fe700 20 client.31857.objecter put_session s=0x7fafe812b080 osd=6 4 2024-02-23T00:55:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.776+0000 7fafdf7fe700 15 client.31857.objecter _session_command_op_remove 6 13 2024-02-23T00:55:54.792 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:55:54.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.790+0000 7fafecf2c700 10 client.31857.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:55:54.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.790+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_assign -1 14 2024-02-23T00:55:54.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.790+0000 7fafecf2c700 20 client.31857.objecter _get_session s=0x7fafe812b080 osd=6 3 2024-02-23T00:55:54.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.790+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe812b080 osd=6 3 2024-02-23T00:55:54.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.790+0000 7fafecf2c700 20 client.31857.objecter _get_session s=0x7fafe812b080 osd=6 3 2024-02-23T00:55:54.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.790+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_remove -1 14 2024-02-23T00:55:54.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.790+0000 7fafecf2c700 20 client.31857.objecter get_session s=0x7fafe812b080 osd=6 3 2024-02-23T00:55:54.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.790+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_assign 6 14 2024-02-23T00:55:54.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.790+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe812b080 osd=6 4 2024-02-23T00:55:54.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.791+0000 7fafecf2c700 10 client.31857.objecter _send_command 14 2024-02-23T00:55:54.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.791+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fafe8128840 con 0x7fafe812a3b0 2024-02-23T00:55:54.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.792+0000 7fafdf7fe700 1 -- 172.21.15.47:0/2249953117 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fafe8128840 con 0x7fafe812a3b0 2024-02-23T00:55:54.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.792+0000 7fafdf7fe700 10 client.31857.objecter ms_dispatch 0x7fafe8001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:55:54.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.793+0000 7fafdf7fe700 10 client.31857.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:55:54.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.793+0000 7fafdf7fe700 20 client.31857.objecter put_session s=0x7fafe812b080 osd=6 4 2024-02-23T00:55:54.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.793+0000 7fafdf7fe700 15 client.31857.objecter _session_command_op_remove 6 14 2024-02-23T00:55:54.798 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:55:54.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.793+0000 7fafecf2c700 10 client.31857.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:55:54.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.793+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_assign -1 15 2024-02-23T00:55:54.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.793+0000 7fafecf2c700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fafe8128f20 0x7fafe8129380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:54.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.793+0000 7fafecf2c700 20 client.31857.objecter _get_session s=0x7fafe8128840 osd=7 3 2024-02-23T00:55:54.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.793+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe8128840 osd=7 3 2024-02-23T00:55:54.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.793+0000 7fafecf2c700 20 client.31857.objecter _get_session s=0x7fafe8128840 osd=7 3 2024-02-23T00:55:54.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.793+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_remove -1 15 2024-02-23T00:55:54.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.793+0000 7fafecf2c700 20 client.31857.objecter get_session s=0x7fafe8128840 osd=7 3 2024-02-23T00:55:54.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.793+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_assign 7 15 2024-02-23T00:55:54.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.793+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe8128840 osd=7 4 2024-02-23T00:55:54.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.793+0000 7fafecf2c700 10 client.31857.objecter _send_command 15 2024-02-23T00:55:54.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.793+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fafe8129a30 con 0x7fafe8128f20 2024-02-23T00:55:54.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.793+0000 7fafe659c700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fafe8128f20 0x7fafe8129380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:54.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.793+0000 7fafdf7fe700 10 client.31857.objecter ms_handle_connect 0x7fafe8128f20 2024-02-23T00:55:54.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.793+0000 7fafe659c700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fafe8128f20 0x7fafe8129380 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:54.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.794+0000 7fafdf7fe700 1 -- 172.21.15.47:0/2249953117 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fafe8129a30 con 0x7fafe8128f20 2024-02-23T00:55:54.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.794+0000 7fafdf7fe700 10 client.31857.objecter ms_dispatch 0x7fafe8001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:55:54.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.794+0000 7fafdf7fe700 10 client.31857.objecter _finish_command 15 = system:0 2024-02-23T00:55:54.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.794+0000 7fafdf7fe700 20 client.31857.objecter put_session s=0x7fafe8128840 osd=7 4 2024-02-23T00:55:54.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.794+0000 7fafdf7fe700 15 client.31857.objecter _session_command_op_remove 7 15 2024-02-23T00:55:54.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.809+0000 7fafecf2c700 10 client.31857.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:55:54.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.809+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_assign -1 16 2024-02-23T00:55:54.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.809+0000 7fafecf2c700 20 client.31857.objecter _get_session s=0x7fafe8128840 osd=7 3 2024-02-23T00:55:54.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.809+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe8128840 osd=7 3 2024-02-23T00:55:54.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.809+0000 7fafecf2c700 20 client.31857.objecter _get_session s=0x7fafe8128840 osd=7 3 2024-02-23T00:55:54.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.809+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_remove -1 16 2024-02-23T00:55:54.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.809+0000 7fafecf2c700 20 client.31857.objecter get_session s=0x7fafe8128840 osd=7 3 2024-02-23T00:55:54.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.809+0000 7fafecf2c700 15 client.31857.objecter _session_command_op_assign 7 16 2024-02-23T00:55:54.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.809+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe8128840 osd=7 4 2024-02-23T00:55:54.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.809+0000 7fafecf2c700 10 client.31857.objecter _send_command 16 2024-02-23T00:55:54.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.809+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fafe8129a30 con 0x7fafe8128f20 2024-02-23T00:55:54.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafdf7fe700 1 -- 172.21.15.47:0/2249953117 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fafe8129a30 con 0x7fafe8128f20 2024-02-23T00:55:54.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafdf7fe700 10 client.31857.objecter ms_dispatch 0x7fafe8001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:55:54.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafdf7fe700 10 client.31857.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:55:54.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafdf7fe700 20 client.31857.objecter put_session s=0x7fafe8128840 osd=7 4 2024-02-23T00:55:54.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafdf7fe700 15 client.31857.objecter _session_command_op_remove 7 16 2024-02-23T00:55:54.815 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:55:54.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 10 librados: watch_flush enter 2024-02-23T00:55:54.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 10 librados: watch_flush exit 2024-02-23T00:55:54.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 10 client.31857.objecter close_session for osd.0 2024-02-23T00:55:54.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fafe8028540 msgr2=0x7fafe80fbcd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:54.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7fafe8028540 0x7fafe80fbcd0 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:54.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe805ee80 osd=0 1 2024-02-23T00:55:54.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 10 client.31857.objecter close_session for osd.1 2024-02-23T00:55:54.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fafe8027b90 msgr2=0x7fafe8027fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:54.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7fafe8027b90 0x7fafe8027fb0 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:54.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe80fd340 osd=1 1 2024-02-23T00:55:54.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 10 client.31857.objecter close_session for osd.2 2024-02-23T00:55:54.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fafe80fcea0 msgr2=0x7fafe811e130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:54.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fafe80fcea0 0x7fafe811e130 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:54.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe8027a10 osd=2 1 2024-02-23T00:55:54.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 10 client.31857.objecter close_session for osd.3 2024-02-23T00:55:54.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafe6d9d700 1 -- 172.21.15.47:0/2249953117 reap_dead start 2024-02-23T00:55:54.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fafe811bcd0 msgr2=0x7fafe811f7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:54.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7fafe811bcd0 0x7fafe811f7e0 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:54.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe80fc810 osd=3 1 2024-02-23T00:55:54.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 10 client.31857.objecter close_session for osd.4 2024-02-23T00:55:54.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fafe8122fe0 msgr2=0x7fafe81253d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:54.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7fafe8122fe0 0x7fafe81253d0 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:54.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe8120080 osd=4 1 2024-02-23T00:55:54.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 10 client.31857.objecter close_session for osd.5 2024-02-23T00:55:54.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fafe8125dd0 msgr2=0x7fafe8128240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:54.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7fafe8125dd0 0x7fafe8128240 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:54.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe8120810 osd=5 1 2024-02-23T00:55:54.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 10 client.31857.objecter close_session for osd.6 2024-02-23T00:55:54.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fafe812a3b0 msgr2=0x7fafe812a830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:54.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7fafe812a3b0 0x7fafe812a830 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:54.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe812b080 osd=6 1 2024-02-23T00:55:54.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 10 client.31857.objecter close_session for osd.7 2024-02-23T00:55:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fafe8128f20 msgr2=0x7fafe8129380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7fafe8128f20 0x7fafe8129380 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 20 client.31857.objecter put_session s=0x7fafe8128840 osd=7 1 2024-02-23T00:55:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 20 client.31857.objecter shutdown clearing up homeless session... 2024-02-23T00:55:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 10 client.31857.objecter successfully canceled tick 2024-02-23T00:55:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafe6d9d700 1 -- 172.21.15.47:0/2249953117 reap_dead start 2024-02-23T00:55:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fafd4062cc0 msgr2=0x7fafd4065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fafd4062cc0 0x7fafd4065180 secure :-1 s=READY pgs=3214 cs=0 l=1 rev1=1 rx=0x7fafe80f8400 tx=0x7fafd8009040).stop 2024-02-23T00:55:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafe80f8cb0 msgr2=0x7fafe80f6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.811+0000 7fafecf2c700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafe80f8cb0 0x7fafe80f6c60 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 rx=0x7fafd0009f20 tx=0x7fafd0009cb0).stop 2024-02-23T00:55:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.812+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 shutdown_connections 2024-02-23T00:55:54.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.812+0000 7fafecf2c700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafe80f8cb0 0x7fafe80f6c60 unknown :-1 s=CLOSED pgs=2648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:54.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.812+0000 7fafecf2c700 1 --2- 172.21.15.47:0/2249953117 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fafd4062cc0 0x7fafd4065180 unknown :-1 s=CLOSED pgs=3214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:54.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.812+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 >> 172.21.15.47:0/2249953117 conn(0x7fafe80f2820 msgr2=0x7fafe80f3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:54.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.812+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 shutdown_connections 2024-02-23T00:55:54.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.812+0000 7fafecf2c700 1 -- 172.21.15.47:0/2249953117 wait complete. 2024-02-23T00:55:54.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:54.812+0000 7fafecf2c700 1 librados: shutdown 2024-02-23T00:55:54.822 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:55:54.822 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:55:54.822 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:55:54.822 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:55:54.822 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:55:54.822 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:55:54.822 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:55:54.822 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:55:54.824 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:54.837 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:53.826106+0000 mon.c (mon.2) 504 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-138"}]: dispatch 2024-02-23T00:55:54.837 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:53.826848+0000 mon.a (mon.0) 3786 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-138"}]: dispatch 2024-02-23T00:55:54.837 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:54.254646+0000 osd.5 (osd.5) 348 : cluster [DBG] 4.7 scrub starts 2024-02-23T00:55:54.837 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:54.258157+0000 osd.5 (osd.5) 349 : cluster [DBG] 4.7 scrub ok 2024-02-23T00:55:54.837 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:54.666823+0000 mon.a (mon.0) 3787 : audit [DBG] from='client.? 172.21.15.47:0/2249953117' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:55:54.838 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:53.826106+0000 mon.c (mon.2) 504 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-138"}]: dispatch 2024-02-23T00:55:54.838 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:53.826848+0000 mon.a (mon.0) 3786 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-138"}]: dispatch 2024-02-23T00:55:54.838 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:54.254646+0000 osd.5 (osd.5) 348 : cluster [DBG] 4.7 scrub starts 2024-02-23T00:55:54.838 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:54.258157+0000 osd.5 (osd.5) 349 : cluster [DBG] 4.7 scrub ok 2024-02-23T00:55:54.838 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:54.666823+0000 mon.a (mon.0) 3787 : audit [DBG] from='client.? 172.21.15.47:0/2249953117' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:55:54.899 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:55:54.902 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:54.902 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:55:55.003 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:55:55.033 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:55.100 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:53.826106+0000 mon.c (mon.2) 504 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-138"}]: dispatch 2024-02-23T00:55:55.101 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:53.826848+0000 mon.a (mon.0) 3786 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-138"}]: dispatch 2024-02-23T00:55:55.101 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:54.254646+0000 osd.5 (osd.5) 348 : cluster [DBG] 4.7 scrub starts 2024-02-23T00:55:55.101 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:54.258157+0000 osd.5 (osd.5) 349 : cluster [DBG] 4.7 scrub ok 2024-02-23T00:55:55.101 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:54.666823+0000 mon.a (mon.0) 3787 : audit [DBG] from='client.? 172.21.15.47:0/2249953117' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:55:55.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:54.432485+0000 osd.4 (osd.4) 335 : cluster [DBG] 2.d scrub starts 2024-02-23T00:55:55.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:54.433537+0000 osd.4 (osd.4) 336 : cluster [DBG] 2.d scrub ok 2024-02-23T00:55:55.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:54.825001+0000 mon.a (mon.0) 3788 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-138"}]': finished 2024-02-23T00:55:55.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:54.825089+0000 mon.a (mon.0) 3789 : cluster [DBG] osdmap e799: 8 total, 8 up, 6 in 2024-02-23T00:55:55.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:54.853534+0000 mon.c (mon.2) 505 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:55.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:54.853878+0000 mon.a (mon.0) 3790 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:55.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:54.854349+0000 mon.c (mon.2) 506 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-138"}]: dispatch 2024-02-23T00:55:55.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:54.854608+0000 mon.a (mon.0) 3791 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-138"}]: dispatch 2024-02-23T00:55:55.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:55.015111+0000 osd.6 (osd.6) 317 : cluster [DBG] 2.6 scrub starts 2024-02-23T00:55:55.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:55.015877+0000 osd.6 (osd.6) 318 : cluster [DBG] 2.6 scrub ok 2024-02-23T00:55:55.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:55.253785+0000 osd.5 (osd.5) 350 : cluster [DBG] 234.4 scrub starts 2024-02-23T00:55:55.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:55.257456+0000 osd.5 (osd.5) 351 : cluster [DBG] 234.4 scrub ok 2024-02-23T00:55:55.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:55.317905+0000 mon.a (mon.0) 3792 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:55:55.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:55.322461+0000 mon.a (mon.0) 3793 : cluster [DBG] osdmap e800: 8 total, 8 up, 6 in 2024-02-23T00:55:55.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:55 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:55.407937+0000 mgr.y (mgr.14517) 677 : cluster [DBG] pgmap v1217: 162 pgs: 162 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 283 B/s wr, 1 op/s 2024-02-23T00:55:55.948 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:55:55.948 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:55:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:55:55.984 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:55.985 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:55:56.086 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:55:56.113 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:54.432485+0000 osd.4 (osd.4) 335 : cluster [DBG] 2.d scrub starts 2024-02-23T00:55:56.113 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:54.433537+0000 osd.4 (osd.4) 336 : cluster [DBG] 2.d scrub ok 2024-02-23T00:55:56.113 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:54.825001+0000 mon.a (mon.0) 3788 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-138"}]': finished 2024-02-23T00:55:56.113 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:54.825089+0000 mon.a (mon.0) 3789 : cluster [DBG] osdmap e799: 8 total, 8 up, 6 in 2024-02-23T00:55:56.113 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:54.853534+0000 mon.c (mon.2) 505 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:56.113 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:54.853878+0000 mon.a (mon.0) 3790 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:56.113 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:54.854349+0000 mon.c (mon.2) 506 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-138"}]: dispatch 2024-02-23T00:55:56.114 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:54.854608+0000 mon.a (mon.0) 3791 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-138"}]: dispatch 2024-02-23T00:55:56.114 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:55.015111+0000 osd.6 (osd.6) 317 : cluster [DBG] 2.6 scrub starts 2024-02-23T00:55:56.114 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:55.015877+0000 osd.6 (osd.6) 318 : cluster [DBG] 2.6 scrub ok 2024-02-23T00:55:56.114 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:55.253785+0000 osd.5 (osd.5) 350 : cluster [DBG] 234.4 scrub starts 2024-02-23T00:55:56.114 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:55.257456+0000 osd.5 (osd.5) 351 : cluster [DBG] 234.4 scrub ok 2024-02-23T00:55:56.114 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:55.317905+0000 mon.a (mon.0) 3792 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:55:56.114 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:55.322461+0000 mon.a (mon.0) 3793 : cluster [DBG] osdmap e800: 8 total, 8 up, 6 in 2024-02-23T00:55:56.114 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:55.407937+0000 mgr.y (mgr.14517) 677 : cluster [DBG] pgmap v1217: 162 pgs: 162 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 283 B/s wr, 1 op/s 2024-02-23T00:55:56.114 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:54.432485+0000 osd.4 (osd.4) 335 : cluster [DBG] 2.d scrub starts 2024-02-23T00:55:56.114 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:54.433537+0000 osd.4 (osd.4) 336 : cluster [DBG] 2.d scrub ok 2024-02-23T00:55:56.114 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:54.825001+0000 mon.a (mon.0) 3788 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-138"}]': finished 2024-02-23T00:55:56.115 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:54.825089+0000 mon.a (mon.0) 3789 : cluster [DBG] osdmap e799: 8 total, 8 up, 6 in 2024-02-23T00:55:56.115 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:54.853534+0000 mon.c (mon.2) 505 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:56.115 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:54.853878+0000 mon.a (mon.0) 3790 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:56.115 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:54.854349+0000 mon.c (mon.2) 506 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-138"}]: dispatch 2024-02-23T00:55:56.115 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:54.854608+0000 mon.a (mon.0) 3791 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-138"}]: dispatch 2024-02-23T00:55:56.115 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:55.015111+0000 osd.6 (osd.6) 317 : cluster [DBG] 2.6 scrub starts 2024-02-23T00:55:56.115 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:55.015877+0000 osd.6 (osd.6) 318 : cluster [DBG] 2.6 scrub ok 2024-02-23T00:55:56.115 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:55.253785+0000 osd.5 (osd.5) 350 : cluster [DBG] 234.4 scrub starts 2024-02-23T00:55:56.115 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:55.257456+0000 osd.5 (osd.5) 351 : cluster [DBG] 234.4 scrub ok 2024-02-23T00:55:56.115 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:55.317905+0000 mon.a (mon.0) 3792 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:55:56.115 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:55.322461+0000 mon.a (mon.0) 3793 : cluster [DBG] osdmap e800: 8 total, 8 up, 6 in 2024-02-23T00:55:56.115 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:55 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:55.407937+0000 mgr.y (mgr.14517) 677 : cluster [DBG] pgmap v1217: 162 pgs: 162 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 283 B/s wr, 1 op/s 2024-02-23T00:55:56.116 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:56.134 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:55:56.139 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:56.141 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:55:56.242 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:55:56.273 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:56.293 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:55:56.303 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:56.304 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:55:56.405 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:55:56.432 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:56.475 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set noscrub 2024-02-23T00:55:56.629 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:55:56.631 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:56.632 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:55:56.638 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.0 dump_ops_in_flight 2024-02-23T00:55:56.733 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:55:56.755 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:55:56.760 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:56.780 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:55:56.781 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:56.782 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:55:56.883 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:55:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:56.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:55.422627+0000 osd.4 (osd.4) 337 : cluster [DBG] 5.1d scrub starts 2024-02-23T00:55:56.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:55.423529+0000 osd.4 (osd.4) 338 : cluster [DBG] 5.1d scrub ok 2024-02-23T00:55:56.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:56.068524+0000 osd.0 (osd.0) 301 : cluster [DBG] 2.10 scrub starts 2024-02-23T00:55:56.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:56.073624+0000 osd.0 (osd.0) 302 : cluster [DBG] 2.10 scrub ok 2024-02-23T00:55:56.947 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:56.311175+0000 mon.a (mon.0) 3794 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:55:56.947 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:56.333714+0000 mon.a (mon.0) 3795 : cluster [DBG] osdmap e801: 8 total, 8 up, 6 in 2024-02-23T00:55:56.947 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:56.337874+0000 mon.c (mon.2) 507 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:55:56.947 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:56.338323+0000 mon.a (mon.0) 3796 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:55:57.010 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:55:57.012 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:57.013 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:55:57.114 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:55:57.140 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:55.422627+0000 osd.4 (osd.4) 337 : cluster [DBG] 5.1d scrub starts 2024-02-23T00:55:57.140 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:55.423529+0000 osd.4 (osd.4) 338 : cluster [DBG] 5.1d scrub ok 2024-02-23T00:55:57.141 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:56.068524+0000 osd.0 (osd.0) 301 : cluster [DBG] 2.10 scrub starts 2024-02-23T00:55:57.141 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:56.073624+0000 osd.0 (osd.0) 302 : cluster [DBG] 2.10 scrub ok 2024-02-23T00:55:57.141 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:56.311175+0000 mon.a (mon.0) 3794 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:55:57.141 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:56.333714+0000 mon.a (mon.0) 3795 : cluster [DBG] osdmap e801: 8 total, 8 up, 6 in 2024-02-23T00:55:57.141 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:56.337874+0000 mon.c (mon.2) 507 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:55:57.141 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:56.338323+0000 mon.a (mon.0) 3796 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:55:57.142 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:55.422627+0000 osd.4 (osd.4) 337 : cluster [DBG] 5.1d scrub starts 2024-02-23T00:55:57.142 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:55.423529+0000 osd.4 (osd.4) 338 : cluster [DBG] 5.1d scrub ok 2024-02-23T00:55:57.142 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:56.068524+0000 osd.0 (osd.0) 301 : cluster [DBG] 2.10 scrub starts 2024-02-23T00:55:57.143 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:56.073624+0000 osd.0 (osd.0) 302 : cluster [DBG] 2.10 scrub ok 2024-02-23T00:55:57.143 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:56.311175+0000 mon.a (mon.0) 3794 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:55:57.143 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:56.333714+0000 mon.a (mon.0) 3795 : cluster [DBG] osdmap e801: 8 total, 8 up, 6 in 2024-02-23T00:55:57.143 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:56.337874+0000 mon.c (mon.2) 507 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:55:57.143 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:56.338323+0000 mon.a (mon.0) 3796 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:55:57.144 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:57.241 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:55:57.243 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:57.244 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:55:57.344 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:55:57.375 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:57.395 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:55:57.397 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:57.397 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:55:57.498 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:55:57.522 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-23T00:55:57.525 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:57.543 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:55:57.545 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:57.546 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:55:57.561 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:55:57.647 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:55:57.673 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:57.692 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:55:57.694 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:57.695 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:55:57.796 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:55:57.822 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:57.841 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:55:57.843 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:57.844 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:55:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:56.446869+0000 osd.4 (osd.4) 339 : cluster [DBG] 18.1b scrub starts 2024-02-23T00:55:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:56.449269+0000 osd.4 (osd.4) 340 : cluster [DBG] 18.1b scrub ok 2024-02-23T00:55:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:57.335335+0000 mon.a (mon.0) 3797 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:55:57.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:57.335457+0000 mon.a (mon.0) 3798 : cluster [DBG] osdmap e802: 8 total, 8 up, 6 in 2024-02-23T00:55:57.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:57.408502+0000 mgr.y (mgr.14517) 678 : cluster [DBG] pgmap v1220: 194 pgs: 32 creating+peering, 162 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:55:57.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:57.615120+0000 mon.c (mon.2) 508 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:57.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:57.615523+0000 mon.a (mon.0) 3799 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:57.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:57.616110+0000 mon.c (mon.2) 509 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-140"}]: dispatch 2024-02-23T00:55:57.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:57.616439+0000 mon.a (mon.0) 3800 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-140"}]: dispatch 2024-02-23T00:55:57.945 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:55:57.973 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:58.146 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:56.446869+0000 osd.4 (osd.4) 339 : cluster [DBG] 18.1b scrub starts 2024-02-23T00:55:58.146 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:56.449269+0000 osd.4 (osd.4) 340 : cluster [DBG] 18.1b scrub ok 2024-02-23T00:55:58.146 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:57.335335+0000 mon.a (mon.0) 3797 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:55:58.146 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:57.335457+0000 mon.a (mon.0) 3798 : cluster [DBG] osdmap e802: 8 total, 8 up, 6 in 2024-02-23T00:55:58.146 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:57.408502+0000 mgr.y (mgr.14517) 678 : cluster [DBG] pgmap v1220: 194 pgs: 32 creating+peering, 162 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:55:58.146 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:57.615120+0000 mon.c (mon.2) 508 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:58.146 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:57.615523+0000 mon.a (mon.0) 3799 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:58.146 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:57.616110+0000 mon.c (mon.2) 509 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-140"}]: dispatch 2024-02-23T00:55:58.146 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:57.616439+0000 mon.a (mon.0) 3800 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-140"}]: dispatch 2024-02-23T00:55:58.147 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:56.446869+0000 osd.4 (osd.4) 339 : cluster [DBG] 18.1b scrub starts 2024-02-23T00:55:58.147 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:56.449269+0000 osd.4 (osd.4) 340 : cluster [DBG] 18.1b scrub ok 2024-02-23T00:55:58.147 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:57.335335+0000 mon.a (mon.0) 3797 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:55:58.147 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:57.335457+0000 mon.a (mon.0) 3798 : cluster [DBG] osdmap e802: 8 total, 8 up, 6 in 2024-02-23T00:55:58.147 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:57.408502+0000 mgr.y (mgr.14517) 678 : cluster [DBG] pgmap v1220: 194 pgs: 32 creating+peering, 162 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:55:58.147 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:57.615120+0000 mon.c (mon.2) 508 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:58.147 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:57.615523+0000 mon.a (mon.0) 3799 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:55:58.147 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:57.616110+0000 mon.c (mon.2) 509 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-140"}]: dispatch 2024-02-23T00:55:58.147 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:57.616439+0000 mon.a (mon.0) 3800 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-140"}]: dispatch 2024-02-23T00:55:58.171 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:55:58.173 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:58.174 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:55:58.274 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:55:58.302 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:58.429 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:55:58.432 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:58.432 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:55:58.533 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:55:58.563 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:58.584 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:55:58.587 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:58.588 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:55:58.688 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:55:58.717 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:58.736 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:55:58.739 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:58.739 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:55:58.840 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:55:58.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.869+0000 7fc3c605b700 1 -- 172.21.15.47:0/1058873171 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc3c00fe510 msgr2=0x7fc3c00f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:58.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.869+0000 7fc3c605b700 1 --2- 172.21.15.47:0/1058873171 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc3c00fe510 0x7fc3c00f7d40 secure :-1 s=READY pgs=2827 cs=0 l=1 rev1=1 rx=0x7fc3a8004870 tx=0x7fc3a8009cb0).stop 2024-02-23T00:55:58.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.869+0000 7fc3c605b700 1 -- 172.21.15.47:0/1058873171 shutdown_connections 2024-02-23T00:55:58.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.869+0000 7fc3c605b700 1 --2- 172.21.15.47:0/1058873171 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3c00f8420 0x7fc3c0104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:58.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.869+0000 7fc3c605b700 1 --2- 172.21.15.47:0/1058873171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3c00fdbb0 0x7fc3c00fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:58.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.869+0000 7fc3c605b700 1 --2- 172.21.15.47:0/1058873171 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc3c00fe510 0x7fc3c00f7d40 unknown :-1 s=CLOSED pgs=2827 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:58.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.869+0000 7fc3c605b700 1 -- 172.21.15.47:0/1058873171 >> 172.21.15.47:0/1058873171 conn(0x7fc3c00f3800 msgr2=0x7fc3c00f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:58.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.870+0000 7fc3c605b700 1 -- 172.21.15.47:0/1058873171 shutdown_connections 2024-02-23T00:55:58.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.870+0000 7fc3c605b700 1 -- 172.21.15.47:0/1058873171 wait complete. 2024-02-23T00:55:58.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.870+0000 7fc3c605b700 1 librados: starting msgr at 2024-02-23T00:55:58.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.870+0000 7fc3c605b700 1 librados: starting objecter 2024-02-23T00:55:58.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.871+0000 7fc3c605b700 1 Processor -- start 2024-02-23T00:55:58.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.871+0000 7fc3c605b700 1 -- start start 2024-02-23T00:55:58.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.871+0000 7fc3c605b700 1 librados: setting wanted keys 2024-02-23T00:55:58.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.871+0000 7fc3c605b700 1 librados: calling monclient init 2024-02-23T00:55:58.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.871+0000 7fc3c605b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc3c00f8420 0x7fc3c010ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:58.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.871+0000 7fc3c605b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3c00fdbb0 0x7fc3c010e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:58.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.871+0000 7fc3c605b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3c00fe510 0x7fc3c0107f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:58.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.871+0000 7fc3c605b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c0108480 con 0x7fc3c00fdbb0 2024-02-23T00:55:58.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.871+0000 7fc3c605b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c01085f0 con 0x7fc3c00fe510 2024-02-23T00:55:58.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.871+0000 7fc3c605b700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c01087b0 con 0x7fc3c00f8420 2024-02-23T00:55:58.876 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:58.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.871+0000 7fc3beffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3c00fdbb0 0x7fc3c010e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:58.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.871+0000 7fc3beffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3c00fdbb0 0x7fc3c010e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34348/0 (socket says 172.21.15.47:34348) 2024-02-23T00:55:58.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.871+0000 7fc3beffd700 1 -- 172.21.15.47:0/619824607 learned_addr learned my addr 172.21.15.47:0/619824607 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:55:58.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.871+0000 7fc3bffff700 1 --2- 172.21.15.47:0/619824607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3c00fe510 0x7fc3c0107f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:58.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.871+0000 7fc3bf7fe700 1 --2- 172.21.15.47:0/619824607 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc3c00f8420 0x7fc3c010ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:58.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.872+0000 7fc3beffd700 1 -- 172.21.15.47:0/619824607 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc3c00f8420 msgr2=0x7fc3c010ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:58.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.872+0000 7fc3beffd700 1 --2- 172.21.15.47:0/619824607 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc3c00f8420 0x7fc3c010ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:58.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.872+0000 7fc3beffd700 1 -- 172.21.15.47:0/619824607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3c00fe510 msgr2=0x7fc3c0107f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:58.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.872+0000 7fc3beffd700 1 --2- 172.21.15.47:0/619824607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3c00fe510 0x7fc3c0107f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:58.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.872+0000 7fc3beffd700 1 -- 172.21.15.47:0/619824607 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b00097c0 con 0x7fc3c00fdbb0 2024-02-23T00:55:58.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.872+0000 7fc3bffff700 1 --2- 172.21.15.47:0/619824607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3c00fe510 0x7fc3c0107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:55:58.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.872+0000 7fc3bf7fe700 1 --2- 172.21.15.47:0/619824607 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc3c00f8420 0x7fc3c010ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:55:58.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.872+0000 7fc3beffd700 1 --2- 172.21.15.47:0/619824607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3c00fdbb0 0x7fc3c010e2e0 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 rx=0x7fc3a8009da0 tx=0x7fc3a8009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:58.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.872+0000 7fc3bcff9700 10 client.?.objecter ms_handle_connect 0x7fc3c00fdbb0 2024-02-23T00:55:58.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.872+0000 7fc3bcff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:55:58.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.872+0000 7fc3bcff9700 1 -- 172.21.15.47:0/619824607 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a800e030 con 0x7fc3c00fdbb0 2024-02-23T00:55:58.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.872+0000 7fc3bcff9700 1 -- 172.21.15.47:0/619824607 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fc3a8013e70 con 0x7fc3c00fdbb0 2024-02-23T00:55:58.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.872+0000 7fc3bcff9700 1 -- 172.21.15.47:0/619824607 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8018070 con 0x7fc3c00fdbb0 2024-02-23T00:55:58.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.872+0000 7fc3c605b700 1 -- 172.21.15.47:0/619824607 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3a8005040 con 0x7fc3c00fdbb0 2024-02-23T00:55:58.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.872+0000 7fc3c605b700 10 client.31863.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:55:58.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.872+0000 7fc3c605b700 1 -- 172.21.15.47:0/619824607 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3c0108cd0 con 0x7fc3c00fdbb0 2024-02-23T00:55:58.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.872+0000 7fc3c605b700 1 librados: init done 2024-02-23T00:55:58.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.873+0000 7fc3c605b700 1 -- 172.21.15.47:0/619824607 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3c00606d0 con 0x7fc3c00fdbb0 2024-02-23T00:55:58.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.874+0000 7fc3bcff9700 1 -- 172.21.15.47:0/619824607 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fc3a8009240 con 0x7fc3c00fdbb0 2024-02-23T00:55:58.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.874+0000 7fc3bcff9700 1 --2- 172.21.15.47:0/619824607 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc3ac062cc0 0x7fc3ac065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:55:58.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.874+0000 7fc3bcff9700 1 -- 172.21.15.47:0/619824607 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(803..803 src has 621..803) v4 ==== 9425+0+0 (secure 0 0 0) 0x7fc3a807ea50 con 0x7fc3c00fdbb0 2024-02-23T00:55:58.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.874+0000 7fc3bcff9700 10 client.31863.objecter ms_dispatch 0x7fc3c0001550 osd_map(803..803 src has 621..803) v4 2024-02-23T00:55:58.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.874+0000 7fc3bcff9700 3 client.31863.objecter handle_osd_map got epochs [803,803] > 0 2024-02-23T00:55:58.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.874+0000 7fc3bcff9700 3 client.31863.objecter handle_osd_map decoding full epoch 803 2024-02-23T00:55:58.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.874+0000 7fc3bf7fe700 1 --2- 172.21.15.47:0/619824607 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc3ac062cc0 0x7fc3ac065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:55:58.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.874+0000 7fc3bcff9700 20 client.31863.objecter dump_active .. 0 homeless 2024-02-23T00:55:58.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.875+0000 7fc3bf7fe700 1 --2- 172.21.15.47:0/619824607 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc3ac062cc0 0x7fc3ac065180 secure :-1 s=READY pgs=3215 cs=0 l=1 rev1=1 rx=0x7fc3b0001450 tx=0x7fc3b000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:55:58.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.875+0000 7fc3bcff9700 10 client.31863.objecter ms_handle_connect 0x7fc3ac062cc0 2024-02-23T00:55:58.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:58.880+0000 7fc3bcff9700 1 -- 172.21.15.47:0/619824607 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fc3a804df90 con 0x7fc3c00fdbb0 2024-02-23T00:55:58.949 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:55:58.951 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:58.952 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:55:59.053 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6 2024-02-23T00:55:59.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:59.065+0000 7fc3c605b700 1 -- 172.21.15.47:0/619824607 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7fc3c0106e20 con 0x7fc3c00fdbb0 2024-02-23T00:55:59.084 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:59.105 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:55:59.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:58.223796+0000 osd.5 (osd.5) 352 : cluster [DBG] 2.a deep-scrub starts 2024-02-23T00:55:59.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:58.224754+0000 osd.5 (osd.5) 353 : cluster [DBG] 2.a deep-scrub ok 2024-02-23T00:55:59.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:58.339682+0000 mon.a (mon.0) 3801 : cluster [DBG] osdmap e803: 8 total, 8 up, 6 in 2024-02-23T00:55:59.108 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:59.109 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6' 2024-02-23T00:55:59.210 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:55:59.239 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:59.258 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:55:59.260 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:59.261 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:55:59.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:58.223796+0000 osd.5 (osd.5) 352 : cluster [DBG] 2.a deep-scrub starts 2024-02-23T00:55:59.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:58.224754+0000 osd.5 (osd.5) 353 : cluster [DBG] 2.a deep-scrub ok 2024-02-23T00:55:59.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:58.339682+0000 mon.a (mon.0) 3801 : cluster [DBG] osdmap e803: 8 total, 8 up, 6 in 2024-02-23T00:55:59.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:58.223796+0000 osd.5 (osd.5) 352 : cluster [DBG] 2.a deep-scrub starts 2024-02-23T00:55:59.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:58.224754+0000 osd.5 (osd.5) 353 : cluster [DBG] 2.a deep-scrub ok 2024-02-23T00:55:59.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:58.339682+0000 mon.a (mon.0) 3801 : cluster [DBG] osdmap e803: 8 total, 8 up, 6 in 2024-02-23T00:55:59.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:59.341+0000 7fc3bcff9700 1 -- 172.21.15.47:0/619824607 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v804) v1 ==== 87+0+0 (secure 0 0 0) 0x7fc3a80515b0 con 0x7fc3c00fdbb0 2024-02-23T00:55:59.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:59.346+0000 7fc3c605b700 10 librados: watch_flush enter 2024-02-23T00:55:59.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:59.350+0000 7fc3c605b700 10 librados: watch_flush exit 2024-02-23T00:55:59.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:59.350+0000 7fc3c605b700 20 client.31863.objecter shutdown clearing up homeless session... 2024-02-23T00:55:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:59.350+0000 7fc3c605b700 10 client.31863.objecter successfully canceled tick 2024-02-23T00:55:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:59.350+0000 7fc3c605b700 1 -- 172.21.15.47:0/619824607 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc3ac062cc0 msgr2=0x7fc3ac065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:59.350+0000 7fc3c605b700 1 --2- 172.21.15.47:0/619824607 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc3ac062cc0 0x7fc3ac065180 secure :-1 s=READY pgs=3215 cs=0 l=1 rev1=1 rx=0x7fc3b0001450 tx=0x7fc3b000b040).stop 2024-02-23T00:55:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:59.350+0000 7fc3c605b700 1 -- 172.21.15.47:0/619824607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3c00fdbb0 msgr2=0x7fc3c010e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:55:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:59.350+0000 7fc3c605b700 1 --2- 172.21.15.47:0/619824607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3c00fdbb0 0x7fc3c010e2e0 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 rx=0x7fc3a8009da0 tx=0x7fc3a8009cb0).stop 2024-02-23T00:55:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:59.350+0000 7fc3c605b700 1 -- 172.21.15.47:0/619824607 shutdown_connections 2024-02-23T00:55:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:59.350+0000 7fc3c605b700 1 --2- 172.21.15.47:0/619824607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3c00fe510 0x7fc3c0107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:59.350+0000 7fc3c605b700 1 --2- 172.21.15.47:0/619824607 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fc3ac062cc0 0x7fc3ac065180 unknown :-1 s=CLOSED pgs=3215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:59.350+0000 7fc3c605b700 1 --2- 172.21.15.47:0/619824607 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc3c00f8420 0x7fc3c010ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:59.350+0000 7fc3c605b700 1 --2- 172.21.15.47:0/619824607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3c00fdbb0 0x7fc3c010e2e0 unknown :-1 s=CLOSED pgs=2649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:55:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:59.350+0000 7fc3c605b700 1 -- 172.21.15.47:0/619824607 >> 172.21.15.47:0/619824607 conn(0x7fc3c00f3800 msgr2=0x7fc3c00fb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:55:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:59.352+0000 7fc3c605b700 1 -- 172.21.15.47:0/619824607 shutdown_connections 2024-02-23T00:55:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:59.352+0000 7fc3c605b700 1 -- 172.21.15.47:0/619824607 wait complete. 2024-02-23T00:55:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:55:59.352+0000 7fc3c605b700 1 librados: shutdown 2024-02-23T00:55:59.358 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is set 2024-02-23T00:55:59.362 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:55:59.393 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:59.743 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:55:59.746 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:59.746 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:55:59.847 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0 2024-02-23T00:55:59.877 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:55:59.930 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:55:59.932 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:55:59.933 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0' 2024-02-23T00:56:00.034 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:56:00.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:59.066582+0000 mon.a (mon.0) 3802 : audit [INF] from='client.? 172.21.15.47:0/619824607' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:56:00.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:59.337812+0000 mon.a (mon.0) 3803 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:56:00.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:59.342281+0000 mon.a (mon.0) 3804 : audit [INF] from='client.? 172.21.15.47:0/619824607' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:56:00.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:59.342355+0000 mon.a (mon.0) 3805 : cluster [DBG] osdmap e804: 8 total, 8 up, 6 in 2024-02-23T00:56:00.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:59.346702+0000 mon.c (mon.2) 510 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:56:00.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:55:59.350727+0000 mon.a (mon.0) 3806 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:56:00.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:55:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:55:59.408877+0000 mgr.y (mgr.14517) 679 : cluster [DBG] pgmap v1223: 194 pgs: 32 unknown, 162 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-02-23T00:56:00.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:59.066582+0000 mon.a (mon.0) 3802 : audit [INF] from='client.? 172.21.15.47:0/619824607' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:56:00.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:59.337812+0000 mon.a (mon.0) 3803 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:56:00.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:59.342281+0000 mon.a (mon.0) 3804 : audit [INF] from='client.? 172.21.15.47:0/619824607' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:56:00.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:59.342355+0000 mon.a (mon.0) 3805 : cluster [DBG] osdmap e804: 8 total, 8 up, 6 in 2024-02-23T00:56:00.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:59.346702+0000 mon.c (mon.2) 510 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:56:00.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:55:59.350727+0000 mon.a (mon.0) 3806 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:56:00.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:55:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:55:59.408877+0000 mgr.y (mgr.14517) 679 : cluster [DBG] pgmap v1223: 194 pgs: 32 unknown, 162 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-02-23T00:56:00.066 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:56:00.088 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:56:00.090 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:59.066582+0000 mon.a (mon.0) 3802 : audit [INF] from='client.? 172.21.15.47:0/619824607' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-23T00:56:00.090 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:59.337812+0000 mon.a (mon.0) 3803 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:56:00.090 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:59.342281+0000 mon.a (mon.0) 3804 : audit [INF] from='client.? 172.21.15.47:0/619824607' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-23T00:56:00.090 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:59.342355+0000 mon.a (mon.0) 3805 : cluster [DBG] osdmap e804: 8 total, 8 up, 6 in 2024-02-23T00:56:00.090 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:59.346702+0000 mon.c (mon.2) 510 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:56:00.090 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:55:59.350727+0000 mon.a (mon.0) 3806 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:56:00.090 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:55:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:55:59.408877+0000 mgr.y (mgr.14517) 679 : cluster [DBG] pgmap v1223: 194 pgs: 32 unknown, 162 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-02-23T00:56:00.091 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:56:00.091 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:56:00.192 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:56:00.225 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:56:00.433 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:56:00.470 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:56:00.470 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:56:00.807 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:56:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:56:00] "GET /metrics HTTP/1.1" 200 189589 "" "Prometheus/2.33.4" 2024-02-23T00:56:00.956 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6, 0] out_osds: [2, 7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-23T00:56:00.956 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-23T00:56:00.956 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-02-23T00:56:00.957 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd in 2 2024-02-23T00:56:01.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:00.349857+0000 mon.a (mon.0) 3807 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:56:01.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:00.349949+0000 mon.a (mon.0) 3808 : cluster [DBG] osdmap e805: 8 total, 8 up, 6 in 2024-02-23T00:56:01.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:00.424620+0000 mon.c (mon.2) 511 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:56:01.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:00.424974+0000 mon.a (mon.0) 3809 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:56:01.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:00.425464+0000 mon.c (mon.2) 512 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-142"}]: dispatch 2024-02-23T00:56:01.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:00.425779+0000 mon.a (mon.0) 3810 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-142"}]: dispatch 2024-02-23T00:56:01.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:00.349857+0000 mon.a (mon.0) 3807 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:56:01.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:00.349949+0000 mon.a (mon.0) 3808 : cluster [DBG] osdmap e805: 8 total, 8 up, 6 in 2024-02-23T00:56:01.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:00.424620+0000 mon.c (mon.2) 511 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:56:01.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:00.424974+0000 mon.a (mon.0) 3809 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:56:01.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:00.425464+0000 mon.c (mon.2) 512 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-142"}]: dispatch 2024-02-23T00:56:01.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:00.425779+0000 mon.a (mon.0) 3810 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-142"}]: dispatch 2024-02-23T00:56:01.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:00.349857+0000 mon.a (mon.0) 3807 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:56:01.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:00.349949+0000 mon.a (mon.0) 3808 : cluster [DBG] osdmap e805: 8 total, 8 up, 6 in 2024-02-23T00:56:01.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:00.424620+0000 mon.c (mon.2) 511 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:56:01.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:00.424974+0000 mon.a (mon.0) 3809 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:56:01.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:00.425464+0000 mon.c (mon.2) 512 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-142"}]: dispatch 2024-02-23T00:56:01.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:00.425779+0000 mon.a (mon.0) 3810 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-142"}]: dispatch 2024-02-23T00:56:01.873 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.0 dump_blocked_ops 2024-02-23T00:56:01.980 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:56:01.984 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:56:01.985 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:56:02.085 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:56:02.114 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:56:02.372 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:01.354012+0000 mon.a (mon.0) 3811 : cluster [DBG] osdmap e806: 8 total, 8 up, 6 in 2024-02-23T00:56:02.372 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:01.409221+0000 mgr.y (mgr.14517) 680 : cluster [DBG] pgmap v1226: 162 pgs: 162 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:56:02.372 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:01.354012+0000 mon.a (mon.0) 3811 : cluster [DBG] osdmap e806: 8 total, 8 up, 6 in 2024-02-23T00:56:02.372 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:01.409221+0000 mgr.y (mgr.14517) 680 : cluster [DBG] pgmap v1226: 162 pgs: 162 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:56:02.663 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:56:02.666 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:56:02.667 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:56:02.694 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:56:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:56:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:56:02.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:01.354012+0000 mon.a (mon.0) 3811 : cluster [DBG] osdmap e806: 8 total, 8 up, 6 in 2024-02-23T00:56:02.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:01.409221+0000 mgr.y (mgr.14517) 680 : cluster [DBG] pgmap v1226: 162 pgs: 162 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:56:02.768 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:56:02.798 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:56:02.923 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:56:02.927 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:56:02.927 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:56:03.028 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:56:03.059 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:56:03.129 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-23T00:56:03.144 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:56:03.158 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:56:03.162 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:56:03.163 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:56:03.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.160+0000 7f47fc28f700 1 -- 172.21.15.47:0/1224156965 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47f40f6a90 msgr2=0x7f47f4104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:03.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.160+0000 7f47fc28f700 1 --2- 172.21.15.47:0/1224156965 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47f40f6a90 0x7f47f4104a20 secure :-1 s=READY pgs=2828 cs=0 l=1 rev1=1 rx=0x7f47ec009be0 tx=0x7f47ec004d10).stop 2024-02-23T00:56:03.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.160+0000 7f47fc28f700 1 -- 172.21.15.47:0/1224156965 shutdown_connections 2024-02-23T00:56:03.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.160+0000 7f47fc28f700 1 --2- 172.21.15.47:0/1224156965 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47f40f6a90 0x7f47f4104a20 unknown :-1 s=CLOSED pgs=2828 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:03.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.160+0000 7f47fc28f700 1 --2- 172.21.15.47:0/1224156965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47f40fdbd0 0x7f47f40fdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:03.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.160+0000 7f47fc28f700 1 --2- 172.21.15.47:0/1224156965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47f40fe530 0x7f47f40f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:03.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.160+0000 7f47fc28f700 1 -- 172.21.15.47:0/1224156965 >> 172.21.15.47:0/1224156965 conn(0x7f47f406c370 msgr2=0x7f47f406c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:56:03.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.160+0000 7f47fc28f700 1 -- 172.21.15.47:0/1224156965 shutdown_connections 2024-02-23T00:56:03.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.160+0000 7f47fc28f700 1 -- 172.21.15.47:0/1224156965 wait complete. 2024-02-23T00:56:03.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.161+0000 7f47fc28f700 1 librados: starting msgr at 2024-02-23T00:56:03.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.161+0000 7f47fc28f700 1 librados: starting objecter 2024-02-23T00:56:03.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.161+0000 7f47fc28f700 1 Processor -- start 2024-02-23T00:56:03.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.161+0000 7f47fc28f700 1 -- start start 2024-02-23T00:56:03.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.161+0000 7f47fc28f700 1 librados: setting wanted keys 2024-02-23T00:56:03.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.161+0000 7f47fc28f700 1 librados: calling monclient init 2024-02-23T00:56:03.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.161+0000 7f47fc28f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47f40f6a90 0x7f47f41143d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:03.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.162+0000 7f47fc28f700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47f40fdbd0 0x7f47f41148c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:03.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.162+0000 7f47fc28f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47f40fe530 0x7f47f410e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:03.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.162+0000 7f47fc28f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f410ea60 con 0x7f47f40f6a90 2024-02-23T00:56:03.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.162+0000 7f47fc28f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f410ebd0 con 0x7f47f40fe530 2024-02-23T00:56:03.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.162+0000 7f47fc28f700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f410ed90 con 0x7f47f40fdbd0 2024-02-23T00:56:03.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.162+0000 7f47fa02b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47f40f6a90 0x7f47f41143d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:03.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.162+0000 7f47fa02b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47f40f6a90 0x7f47f41143d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34378/0 (socket says 172.21.15.47:34378) 2024-02-23T00:56:03.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.162+0000 7f47fa02b700 1 -- 172.21.15.47:0/2958469052 learned_addr learned my addr 172.21.15.47:0/2958469052 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:56:03.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.162+0000 7f47f982a700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47f40fdbd0 0x7f47f41148c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:03.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.162+0000 7f47fa82c700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47f40fe530 0x7f47f410e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:03.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.162+0000 7f47f982a700 1 -- 172.21.15.47:0/2958469052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47f40fe530 msgr2=0x7f47f410e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:03.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.162+0000 7f47f982a700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47f40fe530 0x7f47f410e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:03.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.162+0000 7f47f982a700 1 -- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47f40f6a90 msgr2=0x7f47f41143d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:03.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.162+0000 7f47f982a700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47f40f6a90 0x7f47f41143d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:03.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.162+0000 7f47f982a700 1 -- 172.21.15.47:0/2958469052 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ec0097c0 con 0x7f47f40fdbd0 2024-02-23T00:56:03.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.162+0000 7f47f982a700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47f40fdbd0 0x7f47f41148c0 secure :-1 s=READY pgs=2829 cs=0 l=1 rev1=1 rx=0x7f47e4004920 tx=0x7f47e4009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:03.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.162+0000 7f47eb779700 10 client.?.objecter ms_handle_connect 0x7f47f40fdbd0 2024-02-23T00:56:03.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.162+0000 7f47eb779700 10 client.?.objecter resend_mon_ops 2024-02-23T00:56:03.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.163+0000 7f47eb779700 1 -- 172.21.15.47:0/2958469052 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e4007a40 con 0x7f47f40fdbd0 2024-02-23T00:56:03.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.163+0000 7f47eb779700 1 -- 172.21.15.47:0/2958469052 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f47e400f040 con 0x7f47f40fdbd0 2024-02-23T00:56:03.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.163+0000 7f47eb779700 1 -- 172.21.15.47:0/2958469052 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e40175d0 con 0x7f47f40fdbd0 2024-02-23T00:56:03.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.166+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f410f070 con 0x7f47f40fdbd0 2024-02-23T00:56:03.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.167+0000 7f47fc28f700 10 client.42194.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:56:03.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.167+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f4105ba0 con 0x7f47f40fdbd0 2024-02-23T00:56:03.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.167+0000 7f47fc28f700 1 librados: init done 2024-02-23T00:56:03.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.167+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f47f405fa60 con 0x7f47f40fdbd0 2024-02-23T00:56:03.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.169+0000 7f47eb779700 1 -- 172.21.15.47:0/2958469052 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f47e4015020 con 0x7f47f40fdbd0 2024-02-23T00:56:03.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.170+0000 7f47eb779700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f47e0062cc0 0x7f47e0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:03.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.170+0000 7f47eb779700 1 -- 172.21.15.47:0/2958469052 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(807..807 src has 621..807) v4 ==== 9798+0+0 (secure 0 0 0) 0x7f47e407d420 con 0x7f47f40fdbd0 2024-02-23T00:56:03.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.170+0000 7f47eb779700 10 client.42194.objecter ms_dispatch 0x7f47f4001550 osd_map(807..807 src has 621..807) v4 2024-02-23T00:56:03.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.170+0000 7f47eb779700 3 client.42194.objecter handle_osd_map got epochs [807,807] > 0 2024-02-23T00:56:03.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.170+0000 7f47eb779700 3 client.42194.objecter handle_osd_map decoding full epoch 807 2024-02-23T00:56:03.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.170+0000 7f47eb779700 20 client.42194.objecter dump_active .. 0 homeless 2024-02-23T00:56:03.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.170+0000 7f47eb779700 1 -- 172.21.15.47:0/2958469052 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v807) v1 ==== 54+0+16 (secure 0 0 0) 0x7f47e407d840 con 0x7f47f40fdbd0 2024-02-23T00:56:03.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.170+0000 7f47fa02b700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f47e0062cc0 0x7f47e0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:03.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.170+0000 7f47fc28f700 10 client.42194.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:56:03.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.170+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_assign -1 1 2024-02-23T00:56:03.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.170+0000 7f47fc28f700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f47f40fee60 0x7f47f40ff240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:03.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.170+0000 7f47fc28f700 20 client.42194.objecter _get_session s=0x7f47f405ee80 osd=0 3 2024-02-23T00:56:03.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.170+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f405ee80 osd=0 3 2024-02-23T00:56:03.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.170+0000 7f47fc28f700 20 client.42194.objecter _get_session s=0x7f47f405ee80 osd=0 3 2024-02-23T00:56:03.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.170+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_remove -1 1 2024-02-23T00:56:03.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.170+0000 7f47fc28f700 20 client.42194.objecter get_session s=0x7f47f405ee80 osd=0 3 2024-02-23T00:56:03.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.170+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_assign 0 1 2024-02-23T00:56:03.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.170+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f405ee80 osd=0 4 2024-02-23T00:56:03.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.170+0000 7f47fc28f700 10 client.42194.objecter _send_command 1 2024-02-23T00:56:03.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.170+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47f40f90b0 con 0x7f47f40fee60 2024-02-23T00:56:03.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.171+0000 7f47fa82c700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f47f40fee60 0x7f47f40ff240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:03.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.171+0000 7f47fa02b700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f47e0062cc0 0x7f47e0065180 secure :-1 s=READY pgs=3216 cs=0 l=1 rev1=1 rx=0x7f47f0004f30 tx=0x7f47f000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:03.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.171+0000 7f47eb779700 10 client.42194.objecter ms_handle_connect 0x7f47e0062cc0 2024-02-23T00:56:03.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.171+0000 7f47eb779700 10 client.42194.objecter ms_handle_connect 0x7f47f40fee60 2024-02-23T00:56:03.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.171+0000 7f47fa82c700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f47f40fee60 0x7f47f40ff240 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:03.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.172+0000 7f47eb779700 1 -- 172.21.15.47:0/2958469052 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f47f40f90b0 con 0x7f47f40fee60 2024-02-23T00:56:03.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.172+0000 7f47eb779700 10 client.42194.objecter ms_dispatch 0x7f47f4001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:56:03.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.172+0000 7f47eb779700 10 client.42194.objecter _finish_command 1 = system:0 2024-02-23T00:56:03.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.172+0000 7f47eb779700 20 client.42194.objecter put_session s=0x7f47f405ee80 osd=0 4 2024-02-23T00:56:03.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.172+0000 7f47eb779700 15 client.42194.objecter _session_command_op_remove 0 1 2024-02-23T00:56:03.195 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:56:03.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.193+0000 7f47fc28f700 10 client.42194.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:56:03.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.193+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_assign -1 2 2024-02-23T00:56:03.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.193+0000 7f47fc28f700 20 client.42194.objecter _get_session s=0x7f47f405ee80 osd=0 3 2024-02-23T00:56:03.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.193+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f405ee80 osd=0 3 2024-02-23T00:56:03.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.193+0000 7f47fc28f700 20 client.42194.objecter _get_session s=0x7f47f405ee80 osd=0 3 2024-02-23T00:56:03.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.193+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_remove -1 2 2024-02-23T00:56:03.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.193+0000 7f47fc28f700 20 client.42194.objecter get_session s=0x7f47f405ee80 osd=0 3 2024-02-23T00:56:03.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.193+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_assign 0 2 2024-02-23T00:56:03.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.193+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f405ee80 osd=0 4 2024-02-23T00:56:03.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.193+0000 7f47fc28f700 10 client.42194.objecter _send_command 2 2024-02-23T00:56:03.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.193+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f47f40273c0 con 0x7f47f40fee60 2024-02-23T00:56:03.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.195+0000 7f47eb779700 1 -- 172.21.15.47:0/2958469052 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f47f40273c0 con 0x7f47f40fee60 2024-02-23T00:56:03.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.195+0000 7f47eb779700 10 client.42194.objecter ms_dispatch 0x7f47f4001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:56:03.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.195+0000 7f47eb779700 10 client.42194.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:56:03.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.195+0000 7f47eb779700 20 client.42194.objecter put_session s=0x7f47f405ee80 osd=0 4 2024-02-23T00:56:03.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.195+0000 7f47eb779700 15 client.42194.objecter _session_command_op_remove 0 2 2024-02-23T00:56:03.199 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:56:03.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.196+0000 7f47fc28f700 10 client.42194.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:56:03.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.196+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_assign -1 3 2024-02-23T00:56:03.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.196+0000 7f47fc28f700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f47f40598f0 0x7f47f4059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:03.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.196+0000 7f47fc28f700 20 client.42194.objecter _get_session s=0x7f47f4028710 osd=1 3 2024-02-23T00:56:03.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.196+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f4028710 osd=1 3 2024-02-23T00:56:03.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.196+0000 7f47fc28f700 20 client.42194.objecter _get_session s=0x7f47f4028710 osd=1 3 2024-02-23T00:56:03.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.196+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_remove -1 3 2024-02-23T00:56:03.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.196+0000 7f47fc28f700 20 client.42194.objecter get_session s=0x7f47f4028710 osd=1 3 2024-02-23T00:56:03.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.196+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_assign 1 3 2024-02-23T00:56:03.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.196+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f4028710 osd=1 4 2024-02-23T00:56:03.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.196+0000 7f47fc28f700 10 client.42194.objecter _send_command 3 2024-02-23T00:56:03.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.196+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47f410c600 con 0x7f47f40598f0 2024-02-23T00:56:03.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.196+0000 7f47fa02b700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f47f40598f0 0x7f47f4059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:03.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.196+0000 7f47eb779700 10 client.42194.objecter ms_handle_connect 0x7f47f40598f0 2024-02-23T00:56:03.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.196+0000 7f47fa02b700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f47f40598f0 0x7f47f4059cd0 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:03.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.197+0000 7f47eb779700 1 -- 172.21.15.47:0/2958469052 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f47f410c600 con 0x7f47f40598f0 2024-02-23T00:56:03.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.197+0000 7f47eb779700 10 client.42194.objecter ms_dispatch 0x7f47f4001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:56:03.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.197+0000 7f47eb779700 10 client.42194.objecter _finish_command 3 = system:0 2024-02-23T00:56:03.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.197+0000 7f47eb779700 20 client.42194.objecter put_session s=0x7f47f4028710 osd=1 4 2024-02-23T00:56:03.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.197+0000 7f47eb779700 15 client.42194.objecter _session_command_op_remove 1 3 2024-02-23T00:56:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.228+0000 7f47fc28f700 10 client.42194.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:56:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.228+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_assign -1 4 2024-02-23T00:56:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.228+0000 7f47fc28f700 20 client.42194.objecter _get_session s=0x7f47f4028710 osd=1 3 2024-02-23T00:56:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.228+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f4028710 osd=1 3 2024-02-23T00:56:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.228+0000 7f47fc28f700 20 client.42194.objecter _get_session s=0x7f47f4028710 osd=1 3 2024-02-23T00:56:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.228+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_remove -1 4 2024-02-23T00:56:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.228+0000 7f47fc28f700 20 client.42194.objecter get_session s=0x7f47f4028710 osd=1 3 2024-02-23T00:56:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.228+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_assign 1 4 2024-02-23T00:56:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.228+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f4028710 osd=1 4 2024-02-23T00:56:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.228+0000 7f47fc28f700 10 client.42194.objecter _send_command 4 2024-02-23T00:56:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.228+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f47f4059360 con 0x7f47f40598f0 2024-02-23T00:56:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.229+0000 7f47eb779700 1 -- 172.21.15.47:0/2958469052 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f47f4059360 con 0x7f47f40598f0 2024-02-23T00:56:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.229+0000 7f47eb779700 10 client.42194.objecter ms_dispatch 0x7f47f4001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:56:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.229+0000 7f47eb779700 10 client.42194.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:56:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.229+0000 7f47eb779700 20 client.42194.objecter put_session s=0x7f47f4028710 osd=1 4 2024-02-23T00:56:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.229+0000 7f47eb779700 15 client.42194.objecter _session_command_op_remove 1 4 2024-02-23T00:56:03.233 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:56:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.230+0000 7f47fc28f700 10 client.42194.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:56:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.230+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_assign -1 5 2024-02-23T00:56:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.230+0000 7f47fc28f700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f47f4028270 0x7f47f410beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.230+0000 7f47fc28f700 20 client.42194.objecter _get_session s=0x7f47f4059770 osd=2 3 2024-02-23T00:56:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.230+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f4059770 osd=2 3 2024-02-23T00:56:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.230+0000 7f47fc28f700 20 client.42194.objecter _get_session s=0x7f47f4059770 osd=2 3 2024-02-23T00:56:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.230+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_remove -1 5 2024-02-23T00:56:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.230+0000 7f47fc28f700 20 client.42194.objecter get_session s=0x7f47f4059770 osd=2 3 2024-02-23T00:56:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.230+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_assign 2 5 2024-02-23T00:56:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.230+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f4059770 osd=2 4 2024-02-23T00:56:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.230+0000 7f47fc28f700 10 client.42194.objecter _send_command 5 2024-02-23T00:56:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.230+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47f410c3a0 con 0x7f47f4028270 2024-02-23T00:56:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.230+0000 7f47f982a700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f47f4028270 0x7f47f410beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.230+0000 7f47eb779700 10 client.42194.objecter ms_handle_connect 0x7f47f4028270 2024-02-23T00:56:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.230+0000 7f47f982a700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f47f4028270 0x7f47f410beb0 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.231+0000 7f47eb779700 1 -- 172.21.15.47:0/2958469052 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f47f410c3a0 con 0x7f47f4028270 2024-02-23T00:56:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.231+0000 7f47eb779700 10 client.42194.objecter ms_dispatch 0x7f47f4001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:56:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.231+0000 7f47eb779700 10 client.42194.objecter _finish_command 5 = system:0 2024-02-23T00:56:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.231+0000 7f47eb779700 20 client.42194.objecter put_session s=0x7f47f4059770 osd=2 4 2024-02-23T00:56:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.231+0000 7f47eb779700 15 client.42194.objecter _session_command_op_remove 2 5 2024-02-23T00:56:03.263 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:56:03.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.262+0000 7f47fc28f700 10 client.42194.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:56:03.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.262+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_assign -1 6 2024-02-23T00:56:03.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.262+0000 7f47fc28f700 20 client.42194.objecter _get_session s=0x7f47f4059770 osd=2 3 2024-02-23T00:56:03.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.262+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f4059770 osd=2 3 2024-02-23T00:56:03.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.262+0000 7f47fc28f700 20 client.42194.objecter _get_session s=0x7f47f4059770 osd=2 3 2024-02-23T00:56:03.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.262+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_remove -1 6 2024-02-23T00:56:03.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.262+0000 7f47fc28f700 20 client.42194.objecter get_session s=0x7f47f4059770 osd=2 3 2024-02-23T00:56:03.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.262+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_assign 2 6 2024-02-23T00:56:03.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.262+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f4059770 osd=2 4 2024-02-23T00:56:03.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.262+0000 7f47fc28f700 10 client.42194.objecter _send_command 6 2024-02-23T00:56:03.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.262+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f47f410e0c0 con 0x7f47f4028270 2024-02-23T00:56:03.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.264+0000 7f47eb779700 1 -- 172.21.15.47:0/2958469052 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f47f410e0c0 con 0x7f47f4028270 2024-02-23T00:56:03.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.264+0000 7f47eb779700 10 client.42194.objecter ms_dispatch 0x7f47f4001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:56:03.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.264+0000 7f47eb779700 10 client.42194.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:56:03.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.264+0000 7f47eb779700 20 client.42194.objecter put_session s=0x7f47f4059770 osd=2 4 2024-02-23T00:56:03.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.264+0000 7f47eb779700 15 client.42194.objecter _session_command_op_remove 2 6 2024-02-23T00:56:03.267 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:56:03.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.264+0000 7f47fc28f700 10 client.42194.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:56:03.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.264+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_assign -1 7 2024-02-23T00:56:03.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.264+0000 7f47fc28f700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f47f412c720 0x7f47f412eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:03.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.264+0000 7f47fc28f700 20 client.42194.objecter _get_session s=0x7f47f4027a10 osd=3 3 2024-02-23T00:56:03.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.264+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f4027a10 osd=3 3 2024-02-23T00:56:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.264+0000 7f47fc28f700 20 client.42194.objecter _get_session s=0x7f47f4027a10 osd=3 3 2024-02-23T00:56:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.264+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_remove -1 7 2024-02-23T00:56:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.264+0000 7f47fc28f700 20 client.42194.objecter get_session s=0x7f47f4027a10 osd=3 3 2024-02-23T00:56:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.264+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_assign 3 7 2024-02-23T00:56:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.264+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f4027a10 osd=3 4 2024-02-23T00:56:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.264+0000 7f47fc28f700 10 client.42194.objecter _send_command 7 2024-02-23T00:56:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.264+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47f4027b80 con 0x7f47f412c720 2024-02-23T00:56:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.264+0000 7f47fa82c700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f47f412c720 0x7f47f412eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.264+0000 7f47eb779700 10 client.42194.objecter ms_handle_connect 0x7f47f412c720 2024-02-23T00:56:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.264+0000 7f47fa82c700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f47f412c720 0x7f47f412eb10 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.265+0000 7f47eb779700 1 -- 172.21.15.47:0/2958469052 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f47f4027b80 con 0x7f47f412c720 2024-02-23T00:56:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.265+0000 7f47eb779700 10 client.42194.objecter ms_dispatch 0x7f47f4001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:56:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.265+0000 7f47eb779700 10 client.42194.objecter _finish_command 7 = system:0 2024-02-23T00:56:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.265+0000 7f47eb779700 20 client.42194.objecter put_session s=0x7f47f4027a10 osd=3 4 2024-02-23T00:56:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.265+0000 7f47eb779700 15 client.42194.objecter _session_command_op_remove 3 7 2024-02-23T00:56:03.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.280+0000 7f47fc28f700 10 client.42194.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:56:03.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.280+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_assign -1 8 2024-02-23T00:56:03.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.280+0000 7f47fc28f700 20 client.42194.objecter _get_session s=0x7f47f4027a10 osd=3 3 2024-02-23T00:56:03.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.280+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f4027a10 osd=3 3 2024-02-23T00:56:03.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.280+0000 7f47fc28f700 20 client.42194.objecter _get_session s=0x7f47f4027a10 osd=3 3 2024-02-23T00:56:03.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.280+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_remove -1 8 2024-02-23T00:56:03.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.280+0000 7f47fc28f700 20 client.42194.objecter get_session s=0x7f47f4027a10 osd=3 3 2024-02-23T00:56:03.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.280+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_assign 3 8 2024-02-23T00:56:03.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.280+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f4027a10 osd=3 4 2024-02-23T00:56:03.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.280+0000 7f47fc28f700 10 client.42194.objecter _send_command 8 2024-02-23T00:56:03.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.280+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f47f410d180 con 0x7f47f412c720 2024-02-23T00:56:03.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.281+0000 7f47eb779700 1 -- 172.21.15.47:0/2958469052 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f47f410d180 con 0x7f47f412c720 2024-02-23T00:56:03.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.281+0000 7f47eb779700 10 client.42194.objecter ms_dispatch 0x7f47f4001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:56:03.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.281+0000 7f47eb779700 10 client.42194.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:56:03.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.281+0000 7f47eb779700 20 client.42194.objecter put_session s=0x7f47f4027a10 osd=3 4 2024-02-23T00:56:03.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.281+0000 7f47eb779700 15 client.42194.objecter _session_command_op_remove 3 8 2024-02-23T00:56:03.285 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:56:03.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.282+0000 7f47fc28f700 10 client.42194.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:56:03.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.282+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_assign -1 9 2024-02-23T00:56:03.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.282+0000 7f47fc28f700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f47f410d800 0x7f47f4132190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:03.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.282+0000 7f47fc28f700 20 client.42194.objecter _get_session s=0x7f47f410d180 osd=4 3 2024-02-23T00:56:03.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.282+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f410d180 osd=4 3 2024-02-23T00:56:03.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.282+0000 7f47fc28f700 20 client.42194.objecter _get_session s=0x7f47f410d180 osd=4 3 2024-02-23T00:56:03.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.282+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_remove -1 9 2024-02-23T00:56:03.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.282+0000 7f47fc28f700 20 client.42194.objecter get_session s=0x7f47f410d180 osd=4 3 2024-02-23T00:56:03.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.282+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_assign 4 9 2024-02-23T00:56:03.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.282+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f410d180 osd=4 4 2024-02-23T00:56:03.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.282+0000 7f47fc28f700 10 client.42194.objecter _send_command 9 2024-02-23T00:56:03.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.282+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47f410dd20 con 0x7f47f410d800 2024-02-23T00:56:03.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.282+0000 7f47fa02b700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f47f410d800 0x7f47f4132190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:03.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.282+0000 7f47eb779700 10 client.42194.objecter ms_handle_connect 0x7f47f410d800 2024-02-23T00:56:03.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.282+0000 7f47fa02b700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f47f410d800 0x7f47f4132190 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:03.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.283+0000 7f47eb779700 1 -- 172.21.15.47:0/2958469052 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f47f410dd20 con 0x7f47f410d800 2024-02-23T00:56:03.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.283+0000 7f47eb779700 10 client.42194.objecter ms_dispatch 0x7f47f4001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:56:03.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.283+0000 7f47eb779700 10 client.42194.objecter _finish_command 9 = system:0 2024-02-23T00:56:03.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.283+0000 7f47eb779700 20 client.42194.objecter put_session s=0x7f47f410d180 osd=4 4 2024-02-23T00:56:03.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.283+0000 7f47eb779700 15 client.42194.objecter _session_command_op_remove 4 9 2024-02-23T00:56:03.293 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:56:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:56:03.296 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:56:03.317 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:56:03.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.316+0000 7f47fc28f700 10 client.42194.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:56:03.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.316+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_assign -1 10 2024-02-23T00:56:03.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.316+0000 7f47fc28f700 20 client.42194.objecter _get_session s=0x7f47f410d180 osd=4 3 2024-02-23T00:56:03.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.316+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f410d180 osd=4 3 2024-02-23T00:56:03.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.316+0000 7f47fc28f700 20 client.42194.objecter _get_session s=0x7f47f410d180 osd=4 3 2024-02-23T00:56:03.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.316+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_remove -1 10 2024-02-23T00:56:03.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.316+0000 7f47fc28f700 20 client.42194.objecter get_session s=0x7f47f410d180 osd=4 3 2024-02-23T00:56:03.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.316+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_assign 4 10 2024-02-23T00:56:03.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.316+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f410d180 osd=4 4 2024-02-23T00:56:03.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.316+0000 7f47fc28f700 10 client.42194.objecter _send_command 10 2024-02-23T00:56:03.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.316+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f47f410dd20 con 0x7f47f410d800 2024-02-23T00:56:03.320 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:56:03.320 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:56:03.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.317+0000 7f47eb779700 1 -- 172.21.15.47:0/2958469052 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f47f410dd20 con 0x7f47f410d800 2024-02-23T00:56:03.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.317+0000 7f47eb779700 10 client.42194.objecter ms_dispatch 0x7f47f4001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:56:03.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.317+0000 7f47eb779700 10 client.42194.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:56:03.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.317+0000 7f47eb779700 20 client.42194.objecter put_session s=0x7f47f410d180 osd=4 4 2024-02-23T00:56:03.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.317+0000 7f47eb779700 15 client.42194.objecter _session_command_op_remove 4 10 2024-02-23T00:56:03.322 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:56:03.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.318+0000 7f47fc28f700 10 client.42194.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:56:03.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.318+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_assign -1 11 2024-02-23T00:56:03.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.318+0000 7f47fc28f700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f47f4132b90 0x7f47f4134f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:03.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.318+0000 7f47fc28f700 20 client.42194.objecter _get_session s=0x7f47f410dd20 osd=5 3 2024-02-23T00:56:03.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.318+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f410dd20 osd=5 3 2024-02-23T00:56:03.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.318+0000 7f47fc28f700 20 client.42194.objecter _get_session s=0x7f47f410dd20 osd=5 3 2024-02-23T00:56:03.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.318+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_remove -1 11 2024-02-23T00:56:03.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.318+0000 7f47fc28f700 20 client.42194.objecter get_session s=0x7f47f410dd20 osd=5 3 2024-02-23T00:56:03.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.318+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_assign 5 11 2024-02-23T00:56:03.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.318+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f410dd20 osd=5 4 2024-02-23T00:56:03.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.318+0000 7f47fc28f700 10 client.42194.objecter _send_command 11 2024-02-23T00:56:03.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.318+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47f4137d00 con 0x7f47f4132b90 2024-02-23T00:56:03.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.318+0000 7f47f982a700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f47f4132b90 0x7f47f4134f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:03.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.318+0000 7f47eb779700 10 client.42194.objecter ms_handle_connect 0x7f47f4132b90 2024-02-23T00:56:03.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.318+0000 7f47f982a700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f47f4132b90 0x7f47f4134f80 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:03.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.319+0000 7f47eb779700 1 -- 172.21.15.47:0/2958469052 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f47f4137d00 con 0x7f47f4132b90 2024-02-23T00:56:03.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.319+0000 7f47eb779700 10 client.42194.objecter ms_dispatch 0x7f47f4001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:56:03.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.319+0000 7f47eb779700 10 client.42194.objecter _finish_command 11 = system:0 2024-02-23T00:56:03.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.319+0000 7f47eb779700 20 client.42194.objecter put_session s=0x7f47f410dd20 osd=5 4 2024-02-23T00:56:03.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.319+0000 7f47eb779700 15 client.42194.objecter _session_command_op_remove 5 11 2024-02-23T00:56:03.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.335+0000 7f47fc28f700 10 client.42194.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:56:03.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.335+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_assign -1 12 2024-02-23T00:56:03.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.335+0000 7f47fc28f700 20 client.42194.objecter _get_session s=0x7f47f410dd20 osd=5 3 2024-02-23T00:56:03.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.335+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f410dd20 osd=5 3 2024-02-23T00:56:03.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.335+0000 7f47fc28f700 20 client.42194.objecter _get_session s=0x7f47f410dd20 osd=5 3 2024-02-23T00:56:03.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.335+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_remove -1 12 2024-02-23T00:56:03.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.335+0000 7f47fc28f700 20 client.42194.objecter get_session s=0x7f47f410dd20 osd=5 3 2024-02-23T00:56:03.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.335+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_assign 5 12 2024-02-23T00:56:03.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.335+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f410dd20 osd=5 4 2024-02-23T00:56:03.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.335+0000 7f47fc28f700 10 client.42194.objecter _send_command 12 2024-02-23T00:56:03.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.335+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f47f4137d00 con 0x7f47f4132b90 2024-02-23T00:56:03.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.336+0000 7f47eb779700 1 -- 172.21.15.47:0/2958469052 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f47f4137d00 con 0x7f47f4132b90 2024-02-23T00:56:03.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.336+0000 7f47eb779700 10 client.42194.objecter ms_dispatch 0x7f47f4001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:56:03.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.336+0000 7f47eb779700 10 client.42194.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:56:03.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.336+0000 7f47eb779700 20 client.42194.objecter put_session s=0x7f47f410dd20 osd=5 4 2024-02-23T00:56:03.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.336+0000 7f47eb779700 15 client.42194.objecter _session_command_op_remove 5 12 2024-02-23T00:56:03.340 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:56:03.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.336+0000 7f47fc28f700 10 client.42194.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:56:03.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.336+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_assign -1 13 2024-02-23T00:56:03.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.336+0000 7f47fc28f700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f47f4137070 0x7f47f41374f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:03.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.336+0000 7f47fc28f700 20 client.42194.objecter _get_session s=0x7f47f4137d00 osd=6 3 2024-02-23T00:56:03.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.336+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f4137d00 osd=6 3 2024-02-23T00:56:03.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.336+0000 7f47fc28f700 20 client.42194.objecter _get_session s=0x7f47f4137d00 osd=6 3 2024-02-23T00:56:03.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.336+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_remove -1 13 2024-02-23T00:56:03.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.336+0000 7f47fc28f700 20 client.42194.objecter get_session s=0x7f47f4137d00 osd=6 3 2024-02-23T00:56:03.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.336+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_assign 6 13 2024-02-23T00:56:03.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.336+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f4137d00 osd=6 4 2024-02-23T00:56:03.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.336+0000 7f47fc28f700 10 client.42194.objecter _send_command 13 2024-02-23T00:56:03.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.337+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47f413b030 con 0x7f47f4137070 2024-02-23T00:56:03.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.337+0000 7f47fa82c700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f47f4137070 0x7f47f41374f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:03.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.337+0000 7f47eb779700 10 client.42194.objecter ms_handle_connect 0x7f47f4137070 2024-02-23T00:56:03.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.337+0000 7f47fa82c700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f47f4137070 0x7f47f41374f0 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:03.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.338+0000 7f47eb779700 1 -- 172.21.15.47:0/2958469052 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f47f413b030 con 0x7f47f4137070 2024-02-23T00:56:03.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.338+0000 7f47eb779700 10 client.42194.objecter ms_dispatch 0x7f47f4001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:56:03.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.338+0000 7f47eb779700 10 client.42194.objecter _finish_command 13 = system:0 2024-02-23T00:56:03.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.338+0000 7f47eb779700 20 client.42194.objecter put_session s=0x7f47f4137d00 osd=6 4 2024-02-23T00:56:03.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.338+0000 7f47eb779700 15 client.42194.objecter _session_command_op_remove 6 13 2024-02-23T00:56:03.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.353+0000 7f47fc28f700 10 client.42194.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:56:03.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.353+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_assign -1 14 2024-02-23T00:56:03.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.353+0000 7f47fc28f700 20 client.42194.objecter _get_session s=0x7f47f4137d00 osd=6 3 2024-02-23T00:56:03.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.353+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f4137d00 osd=6 3 2024-02-23T00:56:03.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.353+0000 7f47fc28f700 20 client.42194.objecter _get_session s=0x7f47f4137d00 osd=6 3 2024-02-23T00:56:03.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.353+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_remove -1 14 2024-02-23T00:56:03.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.353+0000 7f47fc28f700 20 client.42194.objecter get_session s=0x7f47f4137d00 osd=6 3 2024-02-23T00:56:03.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.353+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_assign 6 14 2024-02-23T00:56:03.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.353+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f4137d00 osd=6 4 2024-02-23T00:56:03.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.353+0000 7f47fc28f700 10 client.42194.objecter _send_command 14 2024-02-23T00:56:03.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.353+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f47f41354c0 con 0x7f47f4137070 2024-02-23T00:56:03.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.355+0000 7f47eb779700 1 -- 172.21.15.47:0/2958469052 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f47f41354c0 con 0x7f47f4137070 2024-02-23T00:56:03.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.355+0000 7f47eb779700 10 client.42194.objecter ms_dispatch 0x7f47f4001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:56:03.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.355+0000 7f47eb779700 10 client.42194.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:56:03.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.355+0000 7f47eb779700 20 client.42194.objecter put_session s=0x7f47f4137d00 osd=6 4 2024-02-23T00:56:03.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.355+0000 7f47eb779700 15 client.42194.objecter _session_command_op_remove 6 14 2024-02-23T00:56:03.358 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:56:03.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.355+0000 7f47fc28f700 10 client.42194.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:56:03.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.355+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_assign -1 15 2024-02-23T00:56:03.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.355+0000 7f47fc28f700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f47f4135b40 0x7f47f4135f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:03.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.355+0000 7f47fc28f700 20 client.42194.objecter _get_session s=0x7f47f41354c0 osd=7 3 2024-02-23T00:56:03.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.355+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f41354c0 osd=7 3 2024-02-23T00:56:03.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.355+0000 7f47fc28f700 20 client.42194.objecter _get_session s=0x7f47f41354c0 osd=7 3 2024-02-23T00:56:03.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.355+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_remove -1 15 2024-02-23T00:56:03.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.355+0000 7f47fc28f700 20 client.42194.objecter get_session s=0x7f47f41354c0 osd=7 3 2024-02-23T00:56:03.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.355+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_assign 7 15 2024-02-23T00:56:03.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.355+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f41354c0 osd=7 4 2024-02-23T00:56:03.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.355+0000 7f47fc28f700 10 client.42194.objecter _send_command 15 2024-02-23T00:56:03.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.355+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47f4136660 con 0x7f47f4135b40 2024-02-23T00:56:03.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.355+0000 7f47fa02b700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f47f4135b40 0x7f47f4135f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:03.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.355+0000 7f47fa02b700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f47f4135b40 0x7f47f4135f80 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:03.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.355+0000 7f47eb779700 10 client.42194.objecter ms_handle_connect 0x7f47f4135b40 2024-02-23T00:56:03.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.356+0000 7f47eb779700 1 -- 172.21.15.47:0/2958469052 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f47f4136660 con 0x7f47f4135b40 2024-02-23T00:56:03.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.356+0000 7f47eb779700 10 client.42194.objecter ms_dispatch 0x7f47f4001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:56:03.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.356+0000 7f47eb779700 10 client.42194.objecter _finish_command 15 = system:0 2024-02-23T00:56:03.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.356+0000 7f47eb779700 20 client.42194.objecter put_session s=0x7f47f41354c0 osd=7 4 2024-02-23T00:56:03.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.356+0000 7f47eb779700 15 client.42194.objecter _session_command_op_remove 7 15 2024-02-23T00:56:03.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.381+0000 7f47fc28f700 10 client.42194.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-23T00:56:03.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.381+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_assign -1 16 2024-02-23T00:56:03.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.381+0000 7f47fc28f700 20 client.42194.objecter _get_session s=0x7f47f41354c0 osd=7 3 2024-02-23T00:56:03.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.381+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f41354c0 osd=7 3 2024-02-23T00:56:03.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.381+0000 7f47fc28f700 20 client.42194.objecter _get_session s=0x7f47f41354c0 osd=7 3 2024-02-23T00:56:03.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.381+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_remove -1 16 2024-02-23T00:56:03.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.381+0000 7f47fc28f700 20 client.42194.objecter get_session s=0x7f47f41354c0 osd=7 3 2024-02-23T00:56:03.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.381+0000 7f47fc28f700 15 client.42194.objecter _session_command_op_assign 7 16 2024-02-23T00:56:03.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.381+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f41354c0 osd=7 4 2024-02-23T00:56:03.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.381+0000 7f47fc28f700 10 client.42194.objecter _send_command 16 2024-02-23T00:56:03.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.381+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f47f4136660 con 0x7f47f4135b40 2024-02-23T00:56:03.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.383+0000 7f47eb779700 1 -- 172.21.15.47:0/2958469052 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f47f4136660 con 0x7f47f4135b40 2024-02-23T00:56:03.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.383+0000 7f47eb779700 10 client.42194.objecter ms_dispatch 0x7f47f4001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-23T00:56:03.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.383+0000 7f47eb779700 10 client.42194.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-23T00:56:03.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.383+0000 7f47eb779700 20 client.42194.objecter put_session s=0x7f47f41354c0 osd=7 4 2024-02-23T00:56:03.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.383+0000 7f47eb779700 15 client.42194.objecter _session_command_op_remove 7 16 2024-02-23T00:56:03.387 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:56:03.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 10 librados: watch_flush enter 2024-02-23T00:56:03.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 10 librados: watch_flush exit 2024-02-23T00:56:03.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 10 client.42194.objecter close_session for osd.0 2024-02-23T00:56:03.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f47f40fee60 msgr2=0x7f47f40ff240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:03.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f47f40fee60 0x7f47f40ff240 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:03.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f405ee80 osd=0 1 2024-02-23T00:56:03.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 10 client.42194.objecter close_session for osd.1 2024-02-23T00:56:03.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f47f40598f0 msgr2=0x7f47f4059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:03.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f47f40598f0 0x7f47f4059cd0 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:03.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f4028710 osd=1 1 2024-02-23T00:56:03.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 10 client.42194.objecter close_session for osd.2 2024-02-23T00:56:03.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f47f4028270 msgr2=0x7f47f410beb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:03.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f47f4028270 0x7f47f410beb0 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:03.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f4059770 osd=2 1 2024-02-23T00:56:03.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 10 client.42194.objecter close_session for osd.3 2024-02-23T00:56:03.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f47f412c720 msgr2=0x7f47f412eb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:03.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f47f412c720 0x7f47f412eb10 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:03.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f4027a10 osd=3 1 2024-02-23T00:56:03.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 10 client.42194.objecter close_session for osd.4 2024-02-23T00:56:03.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f47f410d800 msgr2=0x7f47f4132190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:03.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f47f410d800 0x7f47f4132190 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:03.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fa82c700 1 -- 172.21.15.47:0/2958469052 reap_dead start 2024-02-23T00:56:03.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f410d180 osd=4 1 2024-02-23T00:56:03.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 10 client.42194.objecter close_session for osd.5 2024-02-23T00:56:03.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f47f4132b90 msgr2=0x7f47f4134f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:03.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f47f4132b90 0x7f47f4134f80 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:03.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f410dd20 osd=5 1 2024-02-23T00:56:03.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fa82c700 1 -- 172.21.15.47:0/2958469052 reap_dead start 2024-02-23T00:56:03.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 10 client.42194.objecter close_session for osd.6 2024-02-23T00:56:03.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f47f4137070 msgr2=0x7f47f41374f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:03.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f47f4137070 0x7f47f41374f0 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:03.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f4137d00 osd=6 1 2024-02-23T00:56:03.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 10 client.42194.objecter close_session for osd.7 2024-02-23T00:56:03.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f47f4135b40 msgr2=0x7f47f4135f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:03.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f47f4135b40 0x7f47f4135f80 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:03.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 20 client.42194.objecter put_session s=0x7f47f41354c0 osd=7 1 2024-02-23T00:56:03.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 20 client.42194.objecter shutdown clearing up homeless session... 2024-02-23T00:56:03.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 10 client.42194.objecter successfully canceled tick 2024-02-23T00:56:03.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fa82c700 1 -- 172.21.15.47:0/2958469052 reap_dead start 2024-02-23T00:56:03.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f47e0062cc0 msgr2=0x7f47e0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:03.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f47e0062cc0 0x7f47e0065180 secure :-1 s=READY pgs=3216 cs=0 l=1 rev1=1 rx=0x7f47f0004f30 tx=0x7f47f000b040).stop 2024-02-23T00:56:03.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47f40fdbd0 msgr2=0x7f47f41148c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:03.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47f40fdbd0 0x7f47f41148c0 secure :-1 s=READY pgs=2829 cs=0 l=1 rev1=1 rx=0x7f47e4004920 tx=0x7f47e4009cb0).stop 2024-02-23T00:56:03.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 shutdown_connections 2024-02-23T00:56:03.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47f40fdbd0 0x7f47f41148c0 unknown :-1 s=CLOSED pgs=2829 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:03.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 1 --2- 172.21.15.47:0/2958469052 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f47e0062cc0 0x7f47e0065180 unknown :-1 s=CLOSED pgs=3216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:03.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 >> 172.21.15.47:0/2958469052 conn(0x7f47f406c370 msgr2=0x7f47f40f5730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:56:03.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 shutdown_connections 2024-02-23T00:56:03.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 1 -- 172.21.15.47:0/2958469052 wait complete. 2024-02-23T00:56:03.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:03.384+0000 7f47fc28f700 1 librados: shutdown 2024-02-23T00:56:03.393 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-23T00:56:03.393 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-23T00:56:03.394 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-23T00:56:03.394 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-23T00:56:03.394 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-23T00:56:03.394 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-23T00:56:03.394 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-23T00:56:03.394 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-23T00:56:03.421 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3 2024-02-23T00:56:03.456 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:56:03.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:02.361299+0000 mon.a (mon.0) 3812 : cluster [DBG] osdmap e807: 8 total, 8 up, 6 in 2024-02-23T00:56:03.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:02.363237+0000 mon.c (mon.2) 513 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:56:03.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:02.363670+0000 mon.a (mon.0) 3813 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:56:03.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:03.169401+0000 mon.c (mon.2) 514 : audit [DBG] from='client.? 172.21.15.47:0/2958469052' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:56:03.699 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:02.361299+0000 mon.a (mon.0) 3812 : cluster [DBG] osdmap e807: 8 total, 8 up, 6 in 2024-02-23T00:56:03.700 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:02.363237+0000 mon.c (mon.2) 513 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:56:03.700 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:02.363670+0000 mon.a (mon.0) 3813 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:56:03.700 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:03.169401+0000 mon.c (mon.2) 514 : audit [DBG] from='client.? 172.21.15.47:0/2958469052' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:56:03.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:02.361299+0000 mon.a (mon.0) 3812 : cluster [DBG] osdmap e807: 8 total, 8 up, 6 in 2024-02-23T00:56:03.701 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:02.363237+0000 mon.c (mon.2) 513 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:56:03.701 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:02.363670+0000 mon.a (mon.0) 3813 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-23T00:56:03.701 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:03.169401+0000 mon.c (mon.2) 514 : audit [DBG] from='client.? 172.21.15.47:0/2958469052' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:56:04.005 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:56:04.007 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:56:04.008 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3' 2024-02-23T00:56:04.109 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1 2024-02-23T00:56:04.139 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:56:04.638 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:03.011352+0000 mgr.y (mgr.14517) 681 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:56:04.638 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:03.361672+0000 mon.a (mon.0) 3814 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:56:04.638 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:03.361792+0000 mon.a (mon.0) 3815 : cluster [DBG] osdmap e808: 8 total, 8 up, 6 in 2024-02-23T00:56:04.638 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:03.372354+0000 mon.c (mon.2) 515 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-101","var": "dedup_tier","val": "test-rados-api-smithi047-82465-144"}]: dispatch 2024-02-23T00:56:04.638 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:03.382543+0000 mon.a (mon.0) 3816 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-101","var": "dedup_tier","val": "test-rados-api-smithi047-82465-144"}]: dispatch 2024-02-23T00:56:04.638 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:03.409479+0000 mon.c (mon.2) 516 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:56:04.638 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:03.410150+0000 mgr.y (mgr.14517) 682 : cluster [DBG] pgmap v1229: 194 pgs: 32 unknown, 162 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-02-23T00:56:04.638 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:03.410569+0000 mon.a (mon.0) 3817 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:56:04.638 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:03.411319+0000 mon.c (mon.2) 517 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-144"}]: dispatch 2024-02-23T00:56:04.638 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:03.411709+0000 mon.a (mon.0) 3818 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-144"}]: dispatch 2024-02-23T00:56:04.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:03.011352+0000 mgr.y (mgr.14517) 681 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:56:04.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:03.361672+0000 mon.a (mon.0) 3814 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:56:04.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:03.361792+0000 mon.a (mon.0) 3815 : cluster [DBG] osdmap e808: 8 total, 8 up, 6 in 2024-02-23T00:56:04.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:03.372354+0000 mon.c (mon.2) 515 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-101","var": "dedup_tier","val": "test-rados-api-smithi047-82465-144"}]: dispatch 2024-02-23T00:56:04.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:03.382543+0000 mon.a (mon.0) 3816 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-101","var": "dedup_tier","val": "test-rados-api-smithi047-82465-144"}]: dispatch 2024-02-23T00:56:04.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:03.409479+0000 mon.c (mon.2) 516 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:56:04.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:03.410150+0000 mgr.y (mgr.14517) 682 : cluster [DBG] pgmap v1229: 194 pgs: 32 unknown, 162 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-02-23T00:56:04.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:03.410569+0000 mon.a (mon.0) 3817 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:56:04.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:03.411319+0000 mon.c (mon.2) 517 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-144"}]: dispatch 2024-02-23T00:56:04.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:03.411709+0000 mon.a (mon.0) 3818 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-144"}]: dispatch 2024-02-23T00:56:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:03.011352+0000 mgr.y (mgr.14517) 681 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:56:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:03.361672+0000 mon.a (mon.0) 3814 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi047-82465-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-23T00:56:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:03.361792+0000 mon.a (mon.0) 3815 : cluster [DBG] osdmap e808: 8 total, 8 up, 6 in 2024-02-23T00:56:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:03.372354+0000 mon.c (mon.2) 515 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-101","var": "dedup_tier","val": "test-rados-api-smithi047-82465-144"}]: dispatch 2024-02-23T00:56:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:03.382543+0000 mon.a (mon.0) 3816 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi047-82465-101","var": "dedup_tier","val": "test-rados-api-smithi047-82465-144"}]: dispatch 2024-02-23T00:56:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:03.409479+0000 mon.c (mon.2) 516 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:56:04.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:03.410150+0000 mgr.y (mgr.14517) 682 : cluster [DBG] pgmap v1229: 194 pgs: 32 unknown, 162 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-02-23T00:56:04.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:03.410569+0000 mon.a (mon.0) 3817 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:56:04.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:03.411319+0000 mon.c (mon.2) 517 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-144"}]: dispatch 2024-02-23T00:56:04.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:03.411709+0000 mon.a (mon.0) 3818 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi047-82465-101", "tierpool": "test-rados-api-smithi047-82465-144"}]: dispatch 2024-02-23T00:56:04.905 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:56:05.091 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:56:05.094 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:56:05.094 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1' 2024-02-23T00:56:05.195 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4 2024-02-23T00:56:05.226 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:56:05.246 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:56:05.249 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:56:05.249 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4' 2024-02-23T00:56:05.350 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7 2024-02-23T00:56:05.380 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:56:05.399 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:56:05.402 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:56:05.402 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7' 2024-02-23T00:56:05.503 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:56:05.536 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:56:05.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:04.383020+0000 mon.a (mon.0) 3819 : cluster [DBG] osdmap e809: 8 total, 8 up, 6 in 2024-02-23T00:56:05.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:05.326190+0000 mon.a (mon.0) 3820 : cluster [DBG] osdmap e810: 8 total, 8 up, 6 in 2024-02-23T00:56:05.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:05.328773+0000 mon.c (mon.2) 518 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:56:05.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:05.329453+0000 mon.a (mon.0) 3821 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:56:05.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:04.383020+0000 mon.a (mon.0) 3819 : cluster [DBG] osdmap e809: 8 total, 8 up, 6 in 2024-02-23T00:56:05.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:05.326190+0000 mon.a (mon.0) 3820 : cluster [DBG] osdmap e810: 8 total, 8 up, 6 in 2024-02-23T00:56:05.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:05.328773+0000 mon.c (mon.2) 518 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:56:05.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:05.329453+0000 mon.a (mon.0) 3821 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:56:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:04.383020+0000 mon.a (mon.0) 3819 : cluster [DBG] osdmap e809: 8 total, 8 up, 6 in 2024-02-23T00:56:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:05.326190+0000 mon.a (mon.0) 3820 : cluster [DBG] osdmap e810: 8 total, 8 up, 6 in 2024-02-23T00:56:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:05.328773+0000 mon.c (mon.2) 518 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:56:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:05.329453+0000 mon.a (mon.0) 3821 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:56:06.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.593+0000 7f872bb00700 1 -- 172.21.15.47:0/3973973282 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87240fcbb0 msgr2=0x7f87240fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.593+0000 7f872bb00700 1 --2- 172.21.15.47:0/3973973282 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87240fcbb0 0x7f87240fcf90 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 rx=0x7f8720004870 tx=0x7f8720009cb0).stop 2024-02-23T00:56:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.593+0000 7f872bb00700 1 -- 172.21.15.47:0/3973973282 shutdown_connections 2024-02-23T00:56:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.593+0000 7f872bb00700 1 --2- 172.21.15.47:0/3973973282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87240f7400 0x7f8724103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.593+0000 7f872bb00700 1 --2- 172.21.15.47:0/3973973282 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87240fcbb0 0x7f87240fcf90 unknown :-1 s=CLOSED pgs=2830 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.593+0000 7f872bb00700 1 --2- 172.21.15.47:0/3973973282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87240fd510 0x7f87240f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.593+0000 7f872bb00700 1 -- 172.21.15.47:0/3973973282 >> 172.21.15.47:0/3973973282 conn(0x7f87240f2820 msgr2=0x7f87240f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:56:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.593+0000 7f872bb00700 1 -- 172.21.15.47:0/3973973282 shutdown_connections 2024-02-23T00:56:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.593+0000 7f872bb00700 1 -- 172.21.15.47:0/3973973282 wait complete. 2024-02-23T00:56:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.593+0000 7f872bb00700 1 librados: starting msgr at 2024-02-23T00:56:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.593+0000 7f872bb00700 1 librados: starting objecter 2024-02-23T00:56:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.593+0000 7f872bb00700 1 Processor -- start 2024-02-23T00:56:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.593+0000 7f872bb00700 1 -- start start 2024-02-23T00:56:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.593+0000 7f872bb00700 1 librados: setting wanted keys 2024-02-23T00:56:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.593+0000 7f872bb00700 1 librados: calling monclient init 2024-02-23T00:56:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.594+0000 7f872bb00700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87240f7400 0x7f872410dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.594+0000 7f872bb00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87240fcbb0 0x7f872410e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.594+0000 7f872bb00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87240fd510 0x7f87241081d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.594+0000 7f872bb00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87241086c0 con 0x7f87240fcbb0 2024-02-23T00:56:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.594+0000 7f872bb00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8724108830 con 0x7f87240fd510 2024-02-23T00:56:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.594+0000 7f872bb00700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87241089f0 con 0x7f87240f7400 2024-02-23T00:56:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.594+0000 7f872989c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87240f7400 0x7f872410dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.594+0000 7f872989c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87240f7400 0x7f872410dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:42438/0 (socket says 172.21.15.47:42438) 2024-02-23T00:56:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.594+0000 7f872989c700 1 -- 172.21.15.47:0/1302302183 learned_addr learned my addr 172.21.15.47:0/1302302183 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:56:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.594+0000 7f872909b700 1 --2- 172.21.15.47:0/1302302183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87240fcbb0 0x7f872410e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.594+0000 7f872989c700 1 -- 172.21.15.47:0/1302302183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87240fd510 msgr2=0x7f87241081d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.594+0000 7f872989c700 1 --2- 172.21.15.47:0/1302302183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87240fd510 0x7f87241081d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:56:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.594+0000 7f872989c700 1 -- 172.21.15.47:0/1302302183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87240fcbb0 msgr2=0x7f872410e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.594+0000 7f872989c700 1 --2- 172.21.15.47:0/1302302183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87240fcbb0 0x7f872410e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:06.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.594+0000 7f872989c700 1 -- 172.21.15.47:0/1302302183 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871c0097c0 con 0x7f87240f7400 2024-02-23T00:56:06.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.594+0000 7f872989c700 1 --2- 172.21.15.47:0/1302302183 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87240f7400 0x7f872410dfe0 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 rx=0x7f8720009da0 tx=0x7f8720009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:06.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.594+0000 7f871affd700 10 client.?.objecter ms_handle_connect 0x7f87240f7400 2024-02-23T00:56:06.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.594+0000 7f871affd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:56:06.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.594+0000 7f871affd700 1 -- 172.21.15.47:0/1302302183 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872000e030 con 0x7f87240f7400 2024-02-23T00:56:06.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.594+0000 7f871affd700 1 -- 172.21.15.47:0/1302302183 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f8720013e70 con 0x7f87240f7400 2024-02-23T00:56:06.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.594+0000 7f872bb00700 1 -- 172.21.15.47:0/1302302183 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8720005040 con 0x7f87240f7400 2024-02-23T00:56:06.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.595+0000 7f871affd700 1 -- 172.21.15.47:0/1302302183 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8720018070 con 0x7f87240f7400 2024-02-23T00:56:06.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.595+0000 7f872bb00700 10 client.42200.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:56:06.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.595+0000 7f872bb00700 1 -- 172.21.15.47:0/1302302183 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8724108e90 con 0x7f87240f7400 2024-02-23T00:56:06.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.595+0000 7f872bb00700 1 librados: init done 2024-02-23T00:56:06.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.595+0000 7f872bb00700 1 -- 172.21.15.47:0/1302302183 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f872405fa60 con 0x7f87240f7400 2024-02-23T00:56:06.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.596+0000 7f871affd700 1 -- 172.21.15.47:0/1302302183 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f8720009240 con 0x7f87240f7400 2024-02-23T00:56:06.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.596+0000 7f871affd700 1 --2- 172.21.15.47:0/1302302183 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8710062cc0 0x7f8710065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:06.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.596+0000 7f871affd700 1 -- 172.21.15.47:0/1302302183 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(811..811 src has 621..811) v4 ==== 8715+0+0 (secure 0 0 0) 0x7f872007e6c0 con 0x7f87240f7400 2024-02-23T00:56:06.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.596+0000 7f871affd700 10 client.42200.objecter ms_dispatch 0x7f8724001550 osd_map(811..811 src has 621..811) v4 2024-02-23T00:56:06.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.596+0000 7f871affd700 3 client.42200.objecter handle_osd_map got epochs [811,811] > 0 2024-02-23T00:56:06.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.596+0000 7f871affd700 3 client.42200.objecter handle_osd_map decoding full epoch 811 2024-02-23T00:56:06.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.596+0000 7f872909b700 1 --2- 172.21.15.47:0/1302302183 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8710062cc0 0x7f8710065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:06.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.597+0000 7f871affd700 20 client.42200.objecter dump_active .. 0 homeless 2024-02-23T00:56:06.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.597+0000 7f871affd700 10 client.42200.objecter ms_handle_connect 0x7f8710062cc0 2024-02-23T00:56:06.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.597+0000 7f872909b700 1 --2- 172.21.15.47:0/1302302183 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8710062cc0 0x7f8710065180 secure :-1 s=READY pgs=3217 cs=0 l=1 rev1=1 rx=0x7f8724109920 tx=0x7f871400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:06.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.601+0000 7f871affd700 1 -- 172.21.15.47:0/1302302183 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f872004df50 con 0x7f87240f7400 2024-02-23T00:56:06.616 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:56:06.620 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:56:06.621 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:56:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:05.410458+0000 mgr.y (mgr.14517) 683 : cluster [DBG] pgmap v1232: 154 pgs: 154 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:56:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:06.327958+0000 mon.a (mon.0) 3822 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:56:06.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:06.328032+0000 mon.a (mon.0) 3823 : cluster [DBG] osdmap e811: 8 total, 8 up, 6 in 2024-02-23T00:56:06.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:06.329552+0000 mon.c (mon.2) 519 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:56:06.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:06.330199+0000 mon.a (mon.0) 3824 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:56:06.702 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:05.410458+0000 mgr.y (mgr.14517) 683 : cluster [DBG] pgmap v1232: 154 pgs: 154 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:56:06.702 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:06.327958+0000 mon.a (mon.0) 3822 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:56:06.702 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:06.328032+0000 mon.a (mon.0) 3823 : cluster [DBG] osdmap e811: 8 total, 8 up, 6 in 2024-02-23T00:56:06.703 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:06.329552+0000 mon.c (mon.2) 519 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:56:06.703 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:06.330199+0000 mon.a (mon.0) 3824 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:56:06.703 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:05.410458+0000 mgr.y (mgr.14517) 683 : cluster [DBG] pgmap v1232: 154 pgs: 154 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:56:06.703 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:06.327958+0000 mon.a (mon.0) 3822 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:56:06.703 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:06.328032+0000 mon.a (mon.0) 3823 : cluster [DBG] osdmap e811: 8 total, 8 up, 6 in 2024-02-23T00:56:06.703 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:06.329552+0000 mon.c (mon.2) 519 : audit [INF] from='client.? 172.21.15.47:0/1032768195' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:56:06.703 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:06.330199+0000 mon.a (mon.0) 3824 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82465-101"}]: dispatch 2024-02-23T00:56:06.722 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:56:06.749 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:56:06.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:06.801+0000 7f872bb00700 1 -- 172.21.15.47:0/1302302183 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd in", "ids": ["2"]} v 0) v1 -- 0x7f872405ee80 con 0x7f87240f7400 2024-02-23T00:56:06.929 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-23T00:56:06.957 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:56:06.802+0000 7f3c89cf1700 -1 mon.a@0(leader).osd e811 definitely_dead 0 2024-02-23T00:56:07.103 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:56:07.106 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:56:07.106 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:56:07.207 DEBUG:teuthology.orchestra.run.smithi047:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2 2024-02-23T00:56:07.236 INFO:teuthology.orchestra.run.smithi047.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:56:07.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:07.329+0000 7f871affd700 1 -- 172.21.15.47:0/1302302183 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd in", "ids": ["2"]}]=0 marked in osd.2. v812) v1 ==== 85+0+0 (secure 0 0 0) 0x7f872405ee80 con 0x7f87240f7400 2024-02-23T00:56:07.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:07.333+0000 7f872bb00700 10 librados: watch_flush enter 2024-02-23T00:56:07.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:07.336+0000 7f872bb00700 10 librados: watch_flush exit 2024-02-23T00:56:07.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:07.336+0000 7f872bb00700 20 client.42200.objecter shutdown clearing up homeless session... 2024-02-23T00:56:07.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:07.336+0000 7f872bb00700 10 client.42200.objecter successfully canceled tick 2024-02-23T00:56:07.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:07.338+0000 7f872bb00700 1 -- 172.21.15.47:0/1302302183 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8710062cc0 msgr2=0x7f8710065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:07.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:07.338+0000 7f872bb00700 1 --2- 172.21.15.47:0/1302302183 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8710062cc0 0x7f8710065180 secure :-1 s=READY pgs=3217 cs=0 l=1 rev1=1 rx=0x7f8724109920 tx=0x7f871400b040).stop 2024-02-23T00:56:07.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:07.338+0000 7f872bb00700 1 -- 172.21.15.47:0/1302302183 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87240f7400 msgr2=0x7f872410dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:07.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:07.338+0000 7f872bb00700 1 --2- 172.21.15.47:0/1302302183 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87240f7400 0x7f872410dfe0 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 rx=0x7f8720009da0 tx=0x7f8720009cb0).stop 2024-02-23T00:56:07.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:07.346+0000 7f872bb00700 1 -- 172.21.15.47:0/1302302183 shutdown_connections 2024-02-23T00:56:07.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:07.346+0000 7f872bb00700 1 --2- 172.21.15.47:0/1302302183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87240fd510 0x7f87241081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:56:07.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:07.346+0000 7f872bb00700 1 --2- 172.21.15.47:0/1302302183 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8710062cc0 0x7f8710065180 unknown :-1 s=CLOSED pgs=3217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:07.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:07.346+0000 7f872bb00700 1 --2- 172.21.15.47:0/1302302183 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87240f7400 0x7f872410dfe0 unknown :-1 s=CLOSED pgs=2831 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:07.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:07.346+0000 7f872bb00700 1 --2- 172.21.15.47:0/1302302183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87240fcbb0 0x7f872410e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:07.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:07.346+0000 7f872bb00700 1 -- 172.21.15.47:0/1302302183 >> 172.21.15.47:0/1302302183 conn(0x7f87240f2820 msgr2=0x7f8724060eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:56:07.352 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (9055 ms) 2024-02-23T00:56:07.353 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-02-23T00:56:07.353 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: got ls 1708649682,0 2024-02-23T00:56:07.353 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: first is 1708649682 2024-02-23T00:56:07.353 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: got ls 1708649682,0 2024-02-23T00:56:07.353 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: got ls 1708649682,0 2024-02-23T00:56:07.353 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: got ls 1708649682,1708649684,1708649685,0 2024-02-23T00:56:07.353 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: got ls 1708649682,1708649684,1708649685,0 2024-02-23T00:56:07.353 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: got ls 1708649682,1708649684,1708649685,0 2024-02-23T00:56:07.353 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: got ls 1708649682,1708649684,1708649685,1708649687,1708649688,0 2024-02-23T00:56:07.353 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: got ls 1708649682,1708649684,1708649685,1708649687,1708649688,0 2024-02-23T00:56:07.353 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: got ls 1708649682,1708649684,1708649685,1708649687,1708649688,0 2024-02-23T00:56:07.353 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: got ls 1708649682,1708649684,1708649685,1708649687,1708649688,1708649690,1708649691,0 2024-02-23T00:56:07.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: got ls 1708649682,1708649684,1708649685,1708649687,1708649688,1708649690,1708649691,0 2024-02-23T00:56:07.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: got ls 1708649682,1708649684,1708649685,1708649687,1708649688,1708649690,1708649691,0 2024-02-23T00:56:07.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: got ls 1708649685,1708649687,1708649688,1708649690,1708649691,1708649693,1708649694,0 2024-02-23T00:56:07.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: first now 1708649685, trimmed 2024-02-23T00:56:07.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (20086 ms) 2024-02-23T00:56:07.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-02-23T00:56:07.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: foo0 2024-02-23T00:56:07.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-02-23T00:56:07.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (13909 ms) 2024-02-23T00:56:07.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-02-23T00:56:07.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (17975 ms) 2024-02-23T00:56:07.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-02-23T00:56:07.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (23465 ms) 2024-02-23T00:56:07.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-02-23T00:56:07.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (5533 ms) 2024-02-23T00:56:07.354 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-02-23T00:56:07.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (3019 ms) 2024-02-23T00:56:07.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-02-23T00:56:07.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (3012 ms) 2024-02-23T00:56:07.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-02-23T00:56:07.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (3029 ms) 2024-02-23T00:56:07.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (235122 ms total) 2024-02-23T00:56:07.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: 2024-02-23T00:56:07.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [----------] Global test environment tear-down 2024-02-23T00:56:07.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [==========] 72 tests from 4 test suites ran. (800122 ms total) 2024-02-23T00:56:07.355 INFO:tasks.workunit.client.0.smithi047.stdout: api_tier_pp: [ PASSED ] 72 tests. 2024-02-23T00:56:07.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:07.348+0000 7f872bb00700 1 -- 172.21.15.47:0/1302302183 shutdown_connections 2024-02-23T00:56:07.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:07.350+0000 7f872bb00700 1 -- 172.21.15.47:0/1302302183 wait complete. 2024-02-23T00:56:07.356 INFO:tasks.workunit.client.0.smithi047.stderr:+ exit 0 2024-02-23T00:56:07.356 INFO:tasks.workunit.client.0.smithi047.stderr:+ cleanup 2024-02-23T00:56:07.356 INFO:tasks.workunit.client.0.smithi047.stderr:+ pkill -P 82368 2024-02-23T00:56:07.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:07.351+0000 7f872bb00700 1 librados: shutdown 2024-02-23T00:56:07.360 INFO:teuthology.orchestra.run.smithi047.stderr:marked in osd.2. 2024-02-23T00:56:07.384 INFO:tasks.workunit.client.0.smithi047.stderr:+ true 2024-02-23T00:56:07.385 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-23T00:56:07.385 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-23T00:56:07.456 INFO:tasks.workunit:Stopping ['rados/test.sh'] on client.0... 2024-02-23T00:56:07.456 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-02-23T00:56:07.485 INFO:teuthology.orchestra.run.smithi047.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:56:07.488 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:06.381236+0000 mon.a (mon.0) 3825 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:56:07.488 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:06.803242+0000 mon.c (mon.2) 520 : audit [DBG] from='client.? 172.21.15.47:0/1302302183' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["2"]}]: dispatch 2024-02-23T00:56:07.488 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:06.803729+0000 mon.a (mon.0) 3826 : audit [DBG] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["2"]}]: dispatch 2024-02-23T00:56:07.488 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:07.330144+0000 mon.a (mon.0) 3827 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:56:07.488 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:07.330232+0000 mon.a (mon.0) 3828 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["2"]}]': finished 2024-02-23T00:56:07.489 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:07.330317+0000 mon.a (mon.0) 3829 : cluster [DBG] osdmap e812: 8 total, 8 up, 7 in 2024-02-23T00:56:07.489 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:06.381236+0000 mon.a (mon.0) 3825 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:56:07.489 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:06.803242+0000 mon.c (mon.2) 520 : audit [DBG] from='client.? 172.21.15.47:0/1302302183' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["2"]}]: dispatch 2024-02-23T00:56:07.489 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:06.803729+0000 mon.a (mon.0) 3826 : audit [DBG] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["2"]}]: dispatch 2024-02-23T00:56:07.489 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:07.330144+0000 mon.a (mon.0) 3827 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:56:07.489 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:07.330232+0000 mon.a (mon.0) 3828 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["2"]}]': finished 2024-02-23T00:56:07.489 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:07.330317+0000 mon.a (mon.0) 3829 : cluster [DBG] osdmap e812: 8 total, 8 up, 7 in 2024-02-23T00:56:07.490 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:56:07.490 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi047 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2' 2024-02-23T00:56:07.591 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:56:07.624 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:56:07.645 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:56:07.646 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:06.381236+0000 mon.a (mon.0) 3825 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-23T00:56:07.646 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:06.803242+0000 mon.c (mon.2) 520 : audit [DBG] from='client.? 172.21.15.47:0/1302302183' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["2"]}]: dispatch 2024-02-23T00:56:07.647 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:06.803729+0000 mon.a (mon.0) 3826 : audit [DBG] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["2"]}]: dispatch 2024-02-23T00:56:07.647 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:07.330144+0000 mon.a (mon.0) 3827 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi047-82465-101"}]': finished 2024-02-23T00:56:07.647 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:07.330232+0000 mon.a (mon.0) 3828 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["2"]}]': finished 2024-02-23T00:56:07.647 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:07.330317+0000 mon.a (mon.0) 3829 : cluster [DBG] osdmap e812: 8 total, 8 up, 7 in 2024-02-23T00:56:07.647 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:56:07.648 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:56:07.749 DEBUG:teuthology.orchestra.run.smithi196:> sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5 2024-02-23T00:56:07.780 INFO:teuthology.orchestra.run.smithi196.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-23T00:56:07.781 DEBUG:teuthology.parallel:result is None 2024-02-23T00:56:07.782 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-02-23T00:56:07.799 INFO:teuthology.orchestra.run.smithi196.stderr:Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:56:07.802 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-23T00:56:07.802 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi196 with status 125: 'sudo docker kill -s 1 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5' 2024-02-23T00:56:07.812 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-02-23T00:56:07.812 DEBUG:teuthology.orchestra.run.smithi047:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-02-23T00:56:07.901 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-02-23T00:56:07.901 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-02-23T00:56:07.916 INFO:tasks.thrashosds:joining thrashosds 2024-02-23T00:56:08.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.332+0000 7f2b2dac0700 1 -- 172.21.15.47:0/3839131851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b280f7400 msgr2=0x7f2b280fbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:08.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.332+0000 7f2b2dac0700 1 --2- 172.21.15.47:0/3839131851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b280f7400 0x7f2b280fbaf0 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 rx=0x7f2b20009be0 tx=0x7f2b20004d10).stop 2024-02-23T00:56:08.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.332+0000 7f2b2dac0700 1 -- 172.21.15.47:0/3839131851 shutdown_connections 2024-02-23T00:56:08.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.332+0000 7f2b2dac0700 1 --2- 172.21.15.47:0/3839131851 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2b280fbfe0 0x7f2b280fc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:56:08.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.332+0000 7f2b2dac0700 1 --2- 172.21.15.47:0/3839131851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b280f6aa0 0x7f2b280f6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:08.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.332+0000 7f2b2dac0700 1 --2- 172.21.15.47:0/3839131851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b280f7400 0x7f2b280fbaf0 unknown :-1 s=CLOSED pgs=2650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:08.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.332+0000 7f2b2dac0700 1 -- 172.21.15.47:0/3839131851 >> 172.21.15.47:0/3839131851 conn(0x7f2b280f2820 msgr2=0x7f2b280f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:56:08.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.338+0000 7f2b2dac0700 1 -- 172.21.15.47:0/3839131851 shutdown_connections 2024-02-23T00:56:08.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.340+0000 7f2b2dac0700 1 -- 172.21.15.47:0/3839131851 wait complete. 2024-02-23T00:56:08.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.340+0000 7f2b2dac0700 1 librados: starting msgr at 2024-02-23T00:56:08.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.340+0000 7f2b2dac0700 1 librados: starting objecter 2024-02-23T00:56:08.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.341+0000 7f2b2dac0700 1 Processor -- start 2024-02-23T00:56:08.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.346+0000 7f2b2dac0700 1 -- start start 2024-02-23T00:56:08.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.346+0000 7f2b2dac0700 1 librados: setting wanted keys 2024-02-23T00:56:08.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.346+0000 7f2b2dac0700 1 librados: calling monclient init 2024-02-23T00:56:08.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.346+0000 7f2b2dac0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b280f6aa0 0x7f2b28103d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:08.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.346+0000 7f2b2dac0700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2b280f7400 0x7f2b28104270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:08.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.346+0000 7f2b2dac0700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b280fbfe0 0x7f2b28107640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:08.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.346+0000 7f2b2dac0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b28107b30 con 0x7f2b280f6aa0 2024-02-23T00:56:08.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.346+0000 7f2b2dac0700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b28107ca0 con 0x7f2b280fbfe0 2024-02-23T00:56:08.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.346+0000 7f2b2dac0700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b28107e60 con 0x7f2b280f7400 2024-02-23T00:56:08.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.347+0000 7f2b26ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2b280f7400 0x7f2b28104270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:08.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.347+0000 7f2b26ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2b280f7400 0x7f2b28104270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:42468/0 (socket says 172.21.15.47:42468) 2024-02-23T00:56:08.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.347+0000 7f2b26ffd700 1 -- 172.21.15.47:0/4203169534 learned_addr learned my addr 172.21.15.47:0/4203169534 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:56:08.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.347+0000 7f2b277fe700 1 --2- 172.21.15.47:0/4203169534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b280f6aa0 0x7f2b28103d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:08.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.348+0000 7f2b26ffd700 1 -- 172.21.15.47:0/4203169534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b280fbfe0 msgr2=0x7f2b28107640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:08.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.348+0000 7f2b26ffd700 1 --2- 172.21.15.47:0/4203169534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b280fbfe0 0x7f2b28107640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:56:08.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.348+0000 7f2b26ffd700 1 -- 172.21.15.47:0/4203169534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b280f6aa0 msgr2=0x7f2b28103d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:08.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.348+0000 7f2b26ffd700 1 --2- 172.21.15.47:0/4203169534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b280f6aa0 0x7f2b28103d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:08.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.348+0000 7f2b26ffd700 1 -- 172.21.15.47:0/4203169534 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b200097c0 con 0x7f2b280f7400 2024-02-23T00:56:08.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.349+0000 7f2b26ffd700 1 --2- 172.21.15.47:0/4203169534 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2b280f7400 0x7f2b28104270 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 rx=0x7f2b20009790 tx=0x7f2b20004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:08.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.349+0000 7f2b24f74700 10 client.?.objecter ms_handle_connect 0x7f2b280f7400 2024-02-23T00:56:08.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.349+0000 7f2b24f74700 10 client.?.objecter resend_mon_ops 2024-02-23T00:56:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.351+0000 7f2b24f74700 1 -- 172.21.15.47:0/4203169534 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2000e040 con 0x7f2b280f7400 2024-02-23T00:56:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.351+0000 7f2b24f74700 1 -- 172.21.15.47:0/4203169534 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f2b20018040 con 0x7f2b280f7400 2024-02-23T00:56:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.351+0000 7f2b24f74700 1 -- 172.21.15.47:0/4203169534 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b20013e40 con 0x7f2b280f7400 2024-02-23T00:56:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.351+0000 7f2b2dac0700 1 -- 172.21.15.47:0/4203169534 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b281080e0 con 0x7f2b280f7400 2024-02-23T00:56:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.351+0000 7f2b2dac0700 10 client.42203.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:56:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.351+0000 7f2b2dac0700 1 -- 172.21.15.47:0/4203169534 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b28108670 con 0x7f2b280f7400 2024-02-23T00:56:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.351+0000 7f2b2dac0700 1 librados: init done 2024-02-23T00:56:08.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.354+0000 7f2b24f74700 1 -- 172.21.15.47:0/4203169534 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f2b20007cf0 con 0x7f2b280f7400 2024-02-23T00:56:08.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.355+0000 7f2b24f74700 1 --2- 172.21.15.47:0/4203169534 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2b14062d60 0x7f2b14065220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:08.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.355+0000 7f2b24f74700 1 -- 172.21.15.47:0/4203169534 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(813..813 src has 621..813) v4 ==== 8638+0+0 (secure 0 0 0) 0x7f2b2007e4d0 con 0x7f2b280f7400 2024-02-23T00:56:08.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.355+0000 7f2b24f74700 10 client.42203.objecter ms_dispatch 0x7f2b28001550 osd_map(813..813 src has 621..813) v4 2024-02-23T00:56:08.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.355+0000 7f2b24f74700 3 client.42203.objecter handle_osd_map got epochs [813,813] > 0 2024-02-23T00:56:08.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.355+0000 7f2b24f74700 3 client.42203.objecter handle_osd_map decoding full epoch 813 2024-02-23T00:56:08.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.355+0000 7f2b24f74700 20 client.42203.objecter dump_active .. 0 homeless 2024-02-23T00:56:08.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.355+0000 7f2b277fe700 1 --2- 172.21.15.47:0/4203169534 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2b14062d60 0x7f2b14065220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:08.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.355+0000 7f2b2dac0700 1 -- 172.21.15.47:0/4203169534 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b2805fa60 con 0x7f2b280f7400 2024-02-23T00:56:08.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.362+0000 7f2b277fe700 1 --2- 172.21.15.47:0/4203169534 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2b14062d60 0x7f2b14065220 secure :-1 s=READY pgs=3218 cs=0 l=1 rev1=1 rx=0x7f2b18004dc0 tx=0x7f2b1800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:08.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.362+0000 7f2b24f74700 10 client.42203.objecter ms_handle_connect 0x7f2b14062d60 2024-02-23T00:56:08.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.362+0000 7f2b24f74700 1 -- 172.21.15.47:0/4203169534 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f2b20021020 con 0x7f2b280f7400 2024-02-23T00:56:08.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:08.581+0000 7f2b2dac0700 1 -- 172.21.15.47:0/4203169534 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7f2b2810d6f0 con 0x7f2b280f7400 2024-02-23T00:56:08.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:07.410913+0000 mgr.y (mgr.14517) 684 : cluster [DBG] pgmap v1235: 154 pgs: 154 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:56:08.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:08.331427+0000 mon.a (mon.0) 3830 : cluster [DBG] osdmap e813: 8 total, 8 up, 7 in 2024-02-23T00:56:08.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:07.410913+0000 mgr.y (mgr.14517) 684 : cluster [DBG] pgmap v1235: 154 pgs: 154 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:56:08.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:08.331427+0000 mon.a (mon.0) 3830 : cluster [DBG] osdmap e813: 8 total, 8 up, 7 in 2024-02-23T00:56:08.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:07.410913+0000 mgr.y (mgr.14517) 684 : cluster [DBG] pgmap v1235: 154 pgs: 154 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-23T00:56:08.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:08.331427+0000 mon.a (mon.0) 3830 : cluster [DBG] osdmap e813: 8 total, 8 up, 7 in 2024-02-23T00:56:08.815 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-02-23T00:56:08.815 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-02-23T00:56:08.815 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-02-23T00:56:08.815 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-02-23T00:56:08.815 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-02-23T00:56:09.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:09.333+0000 7f2b24f74700 1 -- 172.21.15.47:0/4203169534 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v814) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2b2004ddc0 con 0x7f2b280f7400 2024-02-23T00:56:09.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:09.335+0000 7f2b2dac0700 10 librados: watch_flush enter 2024-02-23T00:56:09.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:09.335+0000 7f2b2dac0700 10 librados: watch_flush exit 2024-02-23T00:56:09.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:09.335+0000 7f2b2dac0700 20 client.42203.objecter shutdown clearing up homeless session... 2024-02-23T00:56:09.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:09.335+0000 7f2b2dac0700 10 client.42203.objecter successfully canceled tick 2024-02-23T00:56:09.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:09.335+0000 7f2b2dac0700 1 -- 172.21.15.47:0/4203169534 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2b14062d60 msgr2=0x7f2b14065220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:09.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:09.335+0000 7f2b2dac0700 1 --2- 172.21.15.47:0/4203169534 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2b14062d60 0x7f2b14065220 secure :-1 s=READY pgs=3218 cs=0 l=1 rev1=1 rx=0x7f2b18004dc0 tx=0x7f2b1800b040).stop 2024-02-23T00:56:09.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:09.335+0000 7f2b2dac0700 1 -- 172.21.15.47:0/4203169534 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2b280f7400 msgr2=0x7f2b28104270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:09.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:09.335+0000 7f2b2dac0700 1 --2- 172.21.15.47:0/4203169534 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2b280f7400 0x7f2b28104270 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 rx=0x7f2b20009790 tx=0x7f2b20004d10).stop 2024-02-23T00:56:09.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:09.335+0000 7f2b2dac0700 1 -- 172.21.15.47:0/4203169534 shutdown_connections 2024-02-23T00:56:09.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:09.335+0000 7f2b2dac0700 1 --2- 172.21.15.47:0/4203169534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b280fbfe0 0x7f2b28107640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-23T00:56:09.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:09.335+0000 7f2b2dac0700 1 --2- 172.21.15.47:0/4203169534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b280f6aa0 0x7f2b28103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:09.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:09.335+0000 7f2b2dac0700 1 --2- 172.21.15.47:0/4203169534 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2b14062d60 0x7f2b14065220 unknown :-1 s=CLOSED pgs=3218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:09.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:09.335+0000 7f2b2dac0700 1 --2- 172.21.15.47:0/4203169534 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2b280f7400 0x7f2b28104270 unknown :-1 s=CLOSED pgs=2832 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:09.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:09.335+0000 7f2b2dac0700 1 -- 172.21.15.47:0/4203169534 >> 172.21.15.47:0/4203169534 conn(0x7f2b280f2820 msgr2=0x7f2b280f42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:56:09.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:09.335+0000 7f2b2dac0700 1 -- 172.21.15.47:0/4203169534 shutdown_connections 2024-02-23T00:56:09.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:09.335+0000 7f2b2dac0700 1 -- 172.21.15.47:0/4203169534 wait complete. 2024-02-23T00:56:09.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:09.335+0000 7f2b2dac0700 1 librados: shutdown 2024-02-23T00:56:09.340 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is set 2024-02-23T00:56:09.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:08.583385+0000 mon.c (mon.2) 521 : audit [INF] from='client.? 172.21.15.47:0/4203169534' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:56:09.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:08.583757+0000 mon.a (mon.0) 3831 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:56:09.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:09.328829+0000 mon.a (mon.0) 3832 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:56:09.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:09.333245+0000 mon.a (mon.0) 3833 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:56:09.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:09.333328+0000 mon.a (mon.0) 3834 : cluster [DBG] osdmap e814: 8 total, 8 up, 7 in 2024-02-23T00:56:09.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:08.583385+0000 mon.c (mon.2) 521 : audit [INF] from='client.? 172.21.15.47:0/4203169534' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:56:09.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:08.583757+0000 mon.a (mon.0) 3831 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:56:09.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:09.328829+0000 mon.a (mon.0) 3832 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:56:09.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:09.333245+0000 mon.a (mon.0) 3833 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:56:09.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:09.333328+0000 mon.a (mon.0) 3834 : cluster [DBG] osdmap e814: 8 total, 8 up, 7 in 2024-02-23T00:56:09.635 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.0 dump_historic_ops 2024-02-23T00:56:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:08.583385+0000 mon.c (mon.2) 521 : audit [INF] from='client.? 172.21.15.47:0/4203169534' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:56:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:08.583757+0000 mon.a (mon.0) 3831 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:56:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:09.328829+0000 mon.a (mon.0) 3832 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:56:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:09.333245+0000 mon.a (mon.0) 3833 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-23T00:56:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:09.333328+0000 mon.a (mon.0) 3834 : cluster [DBG] osdmap e814: 8 total, 8 up, 7 in 2024-02-23T00:56:10.205 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-02-23T00:56:10.207 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:56:10.450 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:09.334623+0000 osd.0 (osd.0) 303 : cluster [DBG] 6.0 starting backfill to osd.2 from (0'0,0'0] MAX to 107'2 2024-02-23T00:56:10.451 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:09.411289+0000 mgr.y (mgr.14517) 685 : cluster [DBG] pgmap v1238: 154 pgs: 154 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:56:10.451 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:09.334623+0000 osd.0 (osd.0) 303 : cluster [DBG] 6.0 starting backfill to osd.2 from (0'0,0'0] MAX to 107'2 2024-02-23T00:56:10.451 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:09.411289+0000 mgr.y (mgr.14517) 685 : cluster [DBG] pgmap v1238: 154 pgs: 154 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:56:10.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:09.334623+0000 osd.0 (osd.0) 303 : cluster [DBG] 6.0 starting backfill to osd.2 from (0'0,0'0] MAX to 107'2 2024-02-23T00:56:10.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:09.411289+0000 mgr.y (mgr.14517) 685 : cluster [DBG] pgmap v1238: 154 pgs: 154 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:56:10.807 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:56:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:56:10] "GET /metrics HTTP/1.1" 200 189589 "" "Prometheus/2.33.4" 2024-02-23T00:56:11.341 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:56:12.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.284+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/506187938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b740fe530 msgr2=0x7f1b740f63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:12.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.284+0000 7f1b7ab7b700 1 --2- 172.21.15.47:0/506187938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b740fe530 0x7f1b740f63b0 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 rx=0x7f1b60004870 tx=0x7f1b60009cb0).stop 2024-02-23T00:56:12.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.284+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/506187938 shutdown_connections 2024-02-23T00:56:12.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.284+0000 7f1b7ab7b700 1 --2- 172.21.15.47:0/506187938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b740f6a90 0x7f1b74104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:12.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.284+0000 7f1b7ab7b700 1 --2- 172.21.15.47:0/506187938 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1b740fdbd0 0x7f1b740fdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:12.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.284+0000 7f1b7ab7b700 1 --2- 172.21.15.47:0/506187938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b740fe530 0x7f1b740f63b0 unknown :-1 s=CLOSED pgs=2651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:12.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.284+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/506187938 >> 172.21.15.47:0/506187938 conn(0x7f1b7406c370 msgr2=0x7f1b7406c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:56:12.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.284+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/506187938 shutdown_connections 2024-02-23T00:56:12.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.284+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/506187938 wait complete. 2024-02-23T00:56:12.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.285+0000 7f1b7ab7b700 1 librados: starting msgr at 2024-02-23T00:56:12.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.285+0000 7f1b7ab7b700 1 librados: starting objecter 2024-02-23T00:56:12.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.285+0000 7f1b7ab7b700 1 Processor -- start 2024-02-23T00:56:12.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.285+0000 7f1b7ab7b700 1 -- start start 2024-02-23T00:56:12.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.285+0000 7f1b7ab7b700 1 librados: setting wanted keys 2024-02-23T00:56:12.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.285+0000 7f1b7ab7b700 1 librados: calling monclient init 2024-02-23T00:56:12.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.285+0000 7f1b7ab7b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1b740f6a90 0x7f1b740f7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:12.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.285+0000 7f1b7ab7b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b740fdbd0 0x7f1b740f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:12.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.285+0000 7f1b7ab7b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b740fe530 0x7f1b740f8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:12.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.285+0000 7f1b7ab7b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b741163d0 con 0x7f1b740fe530 2024-02-23T00:56:12.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.285+0000 7f1b7ab7b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b74116540 con 0x7f1b740fdbd0 2024-02-23T00:56:12.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.285+0000 7f1b7ab7b700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b74116700 con 0x7f1b740f6a90 2024-02-23T00:56:12.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.285+0000 7f1b78917700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1b740f6a90 0x7f1b740f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:12.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.285+0000 7f1b79118700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b740fe530 0x7f1b740f8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:12.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.285+0000 7f1b78917700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1b740f6a90 0x7f1b740f7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:42492/0 (socket says 172.21.15.47:42492) 2024-02-23T00:56:12.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.285+0000 7f1b79118700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b740fe530 0x7f1b740f8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51872/0 (socket says 172.21.15.47:51872) 2024-02-23T00:56:12.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.285+0000 7f1b78917700 1 -- 172.21.15.47:0/3638894407 learned_addr learned my addr 172.21.15.47:0/3638894407 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:56:12.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.285+0000 7f1b73fff700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b740fdbd0 0x7f1b740f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:12.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.286+0000 7f1b78917700 1 -- 172.21.15.47:0/3638894407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b740fdbd0 msgr2=0x7f1b740f7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:12.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.286+0000 7f1b78917700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b740fdbd0 0x7f1b740f7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:12.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.286+0000 7f1b78917700 1 -- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b740fe530 msgr2=0x7f1b740f8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.286+0000 7f1b78917700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b740fe530 0x7f1b740f8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.286+0000 7f1b78917700 1 -- 172.21.15.47:0/3638894407 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b68005040 con 0x7f1b740f6a90 2024-02-23T00:56:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.286+0000 7f1b79118700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b740fe530 0x7f1b740f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:56:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.286+0000 7f1b78917700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1b740f6a90 0x7f1b740f7680 secure :-1 s=READY pgs=2833 cs=0 l=1 rev1=1 rx=0x7f1b680048f0 tx=0x7f1b68009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.286+0000 7f1b71ffb700 10 client.?.objecter ms_handle_connect 0x7f1b740f6a90 2024-02-23T00:56:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.286+0000 7f1b71ffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:56:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.286+0000 7f1b71ffb700 1 -- 172.21.15.47:0/3638894407 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b68007b00 con 0x7f1b740f6a90 2024-02-23T00:56:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.286+0000 7f1b71ffb700 1 -- 172.21.15.47:0/3638894407 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f1b6800f040 con 0x7f1b740f6a90 2024-02-23T00:56:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.286+0000 7f1b71ffb700 1 -- 172.21.15.47:0/3638894407 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b68014890 con 0x7f1b740f6a90 2024-02-23T00:56:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.286+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b60005040 con 0x7f1b740f6a90 2024-02-23T00:56:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.286+0000 7f1b7ab7b700 10 client.42209.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:56:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.286+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b74116d40 con 0x7f1b740f6a90 2024-02-23T00:56:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.286+0000 7f1b7ab7b700 1 librados: init done 2024-02-23T00:56:12.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.287+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f1b7405f950 con 0x7f1b740f6a90 2024-02-23T00:56:12.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.287+0000 7f1b71ffb700 1 -- 172.21.15.47:0/3638894407 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f1b68014cc0 con 0x7f1b740f6a90 2024-02-23T00:56:12.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.288+0000 7f1b71ffb700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1b64062cc0 0x7f1b64065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:12.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.288+0000 7f1b71ffb700 1 -- 172.21.15.47:0/3638894407 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(814..814 src has 621..814) v4 ==== 8638+0+0 (secure 0 0 0) 0x7f1b6807cf50 con 0x7f1b740f6a90 2024-02-23T00:56:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.288+0000 7f1b71ffb700 10 client.42209.objecter ms_dispatch 0x7f1b74001550 osd_map(814..814 src has 621..814) v4 2024-02-23T00:56:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.288+0000 7f1b71ffb700 3 client.42209.objecter handle_osd_map got epochs [814,814] > 0 2024-02-23T00:56:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.288+0000 7f1b71ffb700 3 client.42209.objecter handle_osd_map decoding full epoch 814 2024-02-23T00:56:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.288+0000 7f1b73fff700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1b64062cc0 0x7f1b64065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.288+0000 7f1b71ffb700 20 client.42209.objecter dump_active .. 0 homeless 2024-02-23T00:56:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.288+0000 7f1b71ffb700 1 -- 172.21.15.47:0/3638894407 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v814) v1 ==== 54+0+16 (secure 0 0 0) 0x7f1b6807f620 con 0x7f1b740f6a90 2024-02-23T00:56:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.288+0000 7f1b7ab7b700 10 client.42209.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:56:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.288+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_assign -1 1 2024-02-23T00:56:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.288+0000 7f1b7ab7b700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f1b740feee0 0x7f1b740ff2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.288+0000 7f1b7ab7b700 20 client.42209.objecter _get_session s=0x7f1b7405ee80 osd=0 3 2024-02-23T00:56:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.288+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b7405ee80 osd=0 3 2024-02-23T00:56:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.288+0000 7f1b7ab7b700 20 client.42209.objecter _get_session s=0x7f1b7405ee80 osd=0 3 2024-02-23T00:56:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.288+0000 7f1b71ffb700 10 client.42209.objecter ms_handle_connect 0x7f1b64062cc0 2024-02-23T00:56:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.288+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_remove -1 1 2024-02-23T00:56:12.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.288+0000 7f1b7ab7b700 20 client.42209.objecter get_session s=0x7f1b7405ee80 osd=0 3 2024-02-23T00:56:12.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.288+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_assign 0 1 2024-02-23T00:56:12.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.288+0000 7f1b73fff700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1b64062cc0 0x7f1b64065180 secure :-1 s=READY pgs=3219 cs=0 l=1 rev1=1 rx=0x7f1b60004ee0 tx=0x7f1b6000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:12.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.288+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b7405ee80 osd=0 4 2024-02-23T00:56:12.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.288+0000 7f1b7ab7b700 10 client.42209.objecter _send_command 1 2024-02-23T00:56:12.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.288+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b74106ed0 con 0x7f1b740feee0 2024-02-23T00:56:12.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.288+0000 7f1b79118700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f1b740feee0 0x7f1b740ff2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:12.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.289+0000 7f1b71ffb700 10 client.42209.objecter ms_handle_connect 0x7f1b740feee0 2024-02-23T00:56:12.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.289+0000 7f1b79118700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f1b740feee0 0x7f1b740ff2c0 crc :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:12.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.289+0000 7f1b71ffb700 1 -- 172.21.15.47:0/3638894407 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1b74106ed0 con 0x7f1b740feee0 2024-02-23T00:56:12.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.289+0000 7f1b71ffb700 10 client.42209.objecter ms_dispatch 0x7f1b74001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:56:12.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.289+0000 7f1b71ffb700 10 client.42209.objecter _finish_command 1 = system:0 2024-02-23T00:56:12.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.289+0000 7f1b71ffb700 20 client.42209.objecter put_session s=0x7f1b7405ee80 osd=0 4 2024-02-23T00:56:12.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.289+0000 7f1b71ffb700 15 client.42209.objecter _session_command_op_remove 0 1 2024-02-23T00:56:12.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.304+0000 7f1b7ab7b700 10 client.42209.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:56:12.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.304+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_assign -1 2 2024-02-23T00:56:12.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.304+0000 7f1b7ab7b700 20 client.42209.objecter _get_session s=0x7f1b7405ee80 osd=0 3 2024-02-23T00:56:12.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.304+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b7405ee80 osd=0 3 2024-02-23T00:56:12.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.304+0000 7f1b7ab7b700 20 client.42209.objecter _get_session s=0x7f1b7405ee80 osd=0 3 2024-02-23T00:56:12.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.304+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_remove -1 2 2024-02-23T00:56:12.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.304+0000 7f1b7ab7b700 20 client.42209.objecter get_session s=0x7f1b7405ee80 osd=0 3 2024-02-23T00:56:12.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.304+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_assign 0 2 2024-02-23T00:56:12.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.304+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b7405ee80 osd=0 4 2024-02-23T00:56:12.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.304+0000 7f1b7ab7b700 10 client.42209.objecter _send_command 2 2024-02-23T00:56:12.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.304+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1b740273c0 con 0x7f1b740feee0 2024-02-23T00:56:12.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.306+0000 7f1b71ffb700 1 -- 172.21.15.47:0/3638894407 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1b740273c0 con 0x7f1b740feee0 2024-02-23T00:56:12.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.306+0000 7f1b71ffb700 10 client.42209.objecter ms_dispatch 0x7f1b74001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:56:12.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.306+0000 7f1b71ffb700 10 client.42209.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:56:12.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.306+0000 7f1b71ffb700 20 client.42209.objecter put_session s=0x7f1b7405ee80 osd=0 4 2024-02-23T00:56:12.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.306+0000 7f1b71ffb700 15 client.42209.objecter _session_command_op_remove 0 2 2024-02-23T00:56:12.310 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0: {} 2024-02-23T00:56:12.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.306+0000 7f1b7ab7b700 10 client.42209.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:56:12.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.306+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_assign -1 3 2024-02-23T00:56:12.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.306+0000 7f1b7ab7b700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f1b740598f0 0x7f1b74059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:12.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.306+0000 7f1b7ab7b700 20 client.42209.objecter _get_session s=0x7f1b74028710 osd=1 3 2024-02-23T00:56:12.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.306+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b74028710 osd=1 3 2024-02-23T00:56:12.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.306+0000 7f1b7ab7b700 20 client.42209.objecter _get_session s=0x7f1b74028710 osd=1 3 2024-02-23T00:56:12.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.306+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_remove -1 3 2024-02-23T00:56:12.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.306+0000 7f1b7ab7b700 20 client.42209.objecter get_session s=0x7f1b74028710 osd=1 3 2024-02-23T00:56:12.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.306+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_assign 1 3 2024-02-23T00:56:12.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.306+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b74028710 osd=1 4 2024-02-23T00:56:12.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.306+0000 7f1b7ab7b700 10 client.42209.objecter _send_command 3 2024-02-23T00:56:12.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.306+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b7411d740 con 0x7f1b740598f0 2024-02-23T00:56:12.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.306+0000 7f1b78917700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f1b740598f0 0x7f1b74059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:12.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.306+0000 7f1b71ffb700 10 client.42209.objecter ms_handle_connect 0x7f1b740598f0 2024-02-23T00:56:12.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.306+0000 7f1b78917700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f1b740598f0 0x7f1b74059cd0 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:12.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.307+0000 7f1b71ffb700 1 -- 172.21.15.47:0/3638894407 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1b7411d740 con 0x7f1b740598f0 2024-02-23T00:56:12.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.307+0000 7f1b71ffb700 10 client.42209.objecter ms_dispatch 0x7f1b74001550 command_reply(tid 3: 0 ) v1 2024-02-23T00:56:12.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.307+0000 7f1b71ffb700 10 client.42209.objecter _finish_command 3 = system:0 2024-02-23T00:56:12.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.307+0000 7f1b71ffb700 20 client.42209.objecter put_session s=0x7f1b74028710 osd=1 4 2024-02-23T00:56:12.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.307+0000 7f1b71ffb700 15 client.42209.objecter _session_command_op_remove 1 3 2024-02-23T00:56:12.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.324+0000 7f1b7ab7b700 10 client.42209.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:56:12.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.324+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_assign -1 4 2024-02-23T00:56:12.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.324+0000 7f1b7ab7b700 20 client.42209.objecter _get_session s=0x7f1b74028710 osd=1 3 2024-02-23T00:56:12.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.324+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b74028710 osd=1 3 2024-02-23T00:56:12.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.324+0000 7f1b7ab7b700 20 client.42209.objecter _get_session s=0x7f1b74028710 osd=1 3 2024-02-23T00:56:12.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.324+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_remove -1 4 2024-02-23T00:56:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.324+0000 7f1b7ab7b700 20 client.42209.objecter get_session s=0x7f1b74028710 osd=1 3 2024-02-23T00:56:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.324+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_assign 1 4 2024-02-23T00:56:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.324+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b74028710 osd=1 4 2024-02-23T00:56:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.324+0000 7f1b7ab7b700 10 client.42209.objecter _send_command 4 2024-02-23T00:56:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.324+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1b74059360 con 0x7f1b740598f0 2024-02-23T00:56:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.326+0000 7f1b71ffb700 1 -- 172.21.15.47:0/3638894407 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1b74059360 con 0x7f1b740598f0 2024-02-23T00:56:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.326+0000 7f1b71ffb700 10 client.42209.objecter ms_dispatch 0x7f1b74001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:56:12.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.326+0000 7f1b71ffb700 10 client.42209.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:56:12.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.326+0000 7f1b71ffb700 20 client.42209.objecter put_session s=0x7f1b74028710 osd=1 4 2024-02-23T00:56:12.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.326+0000 7f1b71ffb700 15 client.42209.objecter _session_command_op_remove 1 4 2024-02-23T00:56:12.329 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1: {} 2024-02-23T00:56:12.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.326+0000 7f1b7ab7b700 10 client.42209.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:56:12.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.326+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_assign -1 5 2024-02-23T00:56:12.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.326+0000 7f1b7ab7b700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f1b74028270 0x7f1b7411c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:12.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.326+0000 7f1b7ab7b700 20 client.42209.objecter _get_session s=0x7f1b74059770 osd=2 3 2024-02-23T00:56:12.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.326+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b74059770 osd=2 3 2024-02-23T00:56:12.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.326+0000 7f1b7ab7b700 20 client.42209.objecter _get_session s=0x7f1b74059770 osd=2 3 2024-02-23T00:56:12.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.326+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_remove -1 5 2024-02-23T00:56:12.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.326+0000 7f1b7ab7b700 20 client.42209.objecter get_session s=0x7f1b74059770 osd=2 3 2024-02-23T00:56:12.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.326+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_assign 2 5 2024-02-23T00:56:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.326+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b74059770 osd=2 4 2024-02-23T00:56:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.326+0000 7f1b7ab7b700 10 client.42209.objecter _send_command 5 2024-02-23T00:56:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.326+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b7411c650 con 0x7f1b74028270 2024-02-23T00:56:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.326+0000 7f1b73fff700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f1b74028270 0x7f1b7411c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.326+0000 7f1b71ffb700 10 client.42209.objecter ms_handle_connect 0x7f1b74028270 2024-02-23T00:56:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.326+0000 7f1b73fff700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f1b74028270 0x7f1b7411c160 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.327+0000 7f1b71ffb700 1 -- 172.21.15.47:0/3638894407 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1b7411c650 con 0x7f1b74028270 2024-02-23T00:56:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.327+0000 7f1b71ffb700 10 client.42209.objecter ms_dispatch 0x7f1b74001550 command_reply(tid 5: 0 ) v1 2024-02-23T00:56:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.327+0000 7f1b71ffb700 10 client.42209.objecter _finish_command 5 = system:0 2024-02-23T00:56:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.327+0000 7f1b71ffb700 20 client.42209.objecter put_session s=0x7f1b74059770 osd=2 4 2024-02-23T00:56:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.327+0000 7f1b71ffb700 15 client.42209.objecter _session_command_op_remove 2 5 2024-02-23T00:56:12.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.342+0000 7f1b7ab7b700 10 client.42209.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:56:12.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.342+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_assign -1 6 2024-02-23T00:56:12.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.342+0000 7f1b7ab7b700 20 client.42209.objecter _get_session s=0x7f1b74059770 osd=2 3 2024-02-23T00:56:12.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.342+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b74059770 osd=2 3 2024-02-23T00:56:12.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.342+0000 7f1b7ab7b700 20 client.42209.objecter _get_session s=0x7f1b74059770 osd=2 3 2024-02-23T00:56:12.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.342+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_remove -1 6 2024-02-23T00:56:12.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.342+0000 7f1b7ab7b700 20 client.42209.objecter get_session s=0x7f1b74059770 osd=2 3 2024-02-23T00:56:12.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.342+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_assign 2 6 2024-02-23T00:56:12.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.342+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b74059770 osd=2 4 2024-02-23T00:56:12.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.342+0000 7f1b7ab7b700 10 client.42209.objecter _send_command 6 2024-02-23T00:56:12.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.342+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1b740277d0 con 0x7f1b74028270 2024-02-23T00:56:12.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.343+0000 7f1b71ffb700 1 -- 172.21.15.47:0/3638894407 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1b740277d0 con 0x7f1b74028270 2024-02-23T00:56:12.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.343+0000 7f1b71ffb700 10 client.42209.objecter ms_dispatch 0x7f1b74001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:56:12.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.343+0000 7f1b71ffb700 10 client.42209.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:56:12.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.343+0000 7f1b71ffb700 20 client.42209.objecter put_session s=0x7f1b74059770 osd=2 4 2024-02-23T00:56:12.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.343+0000 7f1b71ffb700 15 client.42209.objecter _session_command_op_remove 2 6 2024-02-23T00:56:12.346 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2: {} 2024-02-23T00:56:12.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.343+0000 7f1b7ab7b700 10 client.42209.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:56:12.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.343+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_assign -1 7 2024-02-23T00:56:12.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.343+0000 7f1b7ab7b700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f1b7411f8c0 0x7f1b74121cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:12.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.343+0000 7f1b7ab7b700 20 client.42209.objecter _get_session s=0x7f1b74027bd0 osd=3 3 2024-02-23T00:56:12.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.343+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b74027bd0 osd=3 3 2024-02-23T00:56:12.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.343+0000 7f1b7ab7b700 20 client.42209.objecter _get_session s=0x7f1b74027bd0 osd=3 3 2024-02-23T00:56:12.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.343+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_remove -1 7 2024-02-23T00:56:12.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.343+0000 7f1b7ab7b700 20 client.42209.objecter get_session s=0x7f1b74027bd0 osd=3 3 2024-02-23T00:56:12.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.343+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_assign 3 7 2024-02-23T00:56:12.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.343+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b74027bd0 osd=3 4 2024-02-23T00:56:12.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.343+0000 7f1b7ab7b700 10 client.42209.objecter _send_command 7 2024-02-23T00:56:12.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.343+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b74124900 con 0x7f1b7411f8c0 2024-02-23T00:56:12.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.344+0000 7f1b79118700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f1b7411f8c0 0x7f1b74121cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:12.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.344+0000 7f1b71ffb700 10 client.42209.objecter ms_handle_connect 0x7f1b7411f8c0 2024-02-23T00:56:12.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.344+0000 7f1b79118700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f1b7411f8c0 0x7f1b74121cb0 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:12.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.344+0000 7f1b71ffb700 1 -- 172.21.15.47:0/3638894407 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1b74124900 con 0x7f1b7411f8c0 2024-02-23T00:56:12.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.344+0000 7f1b71ffb700 10 client.42209.objecter ms_dispatch 0x7f1b74001550 command_reply(tid 7: 0 ) v1 2024-02-23T00:56:12.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.344+0000 7f1b71ffb700 10 client.42209.objecter _finish_command 7 = system:0 2024-02-23T00:56:12.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.344+0000 7f1b71ffb700 20 client.42209.objecter put_session s=0x7f1b74027bd0 osd=3 4 2024-02-23T00:56:12.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.344+0000 7f1b71ffb700 15 client.42209.objecter _session_command_op_remove 3 7 2024-02-23T00:56:12.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.359+0000 7f1b7ab7b700 10 client.42209.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:56:12.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.359+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_assign -1 8 2024-02-23T00:56:12.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.359+0000 7f1b7ab7b700 20 client.42209.objecter _get_session s=0x7f1b74027bd0 osd=3 3 2024-02-23T00:56:12.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.359+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b74027bd0 osd=3 3 2024-02-23T00:56:12.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.359+0000 7f1b7ab7b700 20 client.42209.objecter _get_session s=0x7f1b74027bd0 osd=3 3 2024-02-23T00:56:12.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.359+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_remove -1 8 2024-02-23T00:56:12.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.359+0000 7f1b7ab7b700 20 client.42209.objecter get_session s=0x7f1b74027bd0 osd=3 3 2024-02-23T00:56:12.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.359+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_assign 3 8 2024-02-23T00:56:12.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.359+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b74027bd0 osd=3 4 2024-02-23T00:56:12.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.359+0000 7f1b7ab7b700 10 client.42209.objecter _send_command 8 2024-02-23T00:56:12.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.359+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1b7411ca00 con 0x7f1b7411f8c0 2024-02-23T00:56:12.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.361+0000 7f1b71ffb700 1 -- 172.21.15.47:0/3638894407 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1b7411ca00 con 0x7f1b7411f8c0 2024-02-23T00:56:12.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.361+0000 7f1b71ffb700 10 client.42209.objecter ms_dispatch 0x7f1b74001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:56:12.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.361+0000 7f1b71ffb700 10 client.42209.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:56:12.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.361+0000 7f1b71ffb700 20 client.42209.objecter put_session s=0x7f1b74027bd0 osd=3 4 2024-02-23T00:56:12.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.361+0000 7f1b71ffb700 15 client.42209.objecter _session_command_op_remove 3 8 2024-02-23T00:56:12.364 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3: {} 2024-02-23T00:56:12.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.361+0000 7f1b7ab7b700 10 client.42209.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:56:12.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.361+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_assign -1 9 2024-02-23T00:56:12.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.361+0000 7f1b7ab7b700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f1b7411cb70 0x7f1b74123320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:12.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.361+0000 7f1b7ab7b700 20 client.42209.objecter _get_session s=0x7f1b7411ca00 osd=4 3 2024-02-23T00:56:12.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.361+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b7411ca00 osd=4 3 2024-02-23T00:56:12.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.361+0000 7f1b7ab7b700 20 client.42209.objecter _get_session s=0x7f1b7411ca00 osd=4 3 2024-02-23T00:56:12.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.361+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_remove -1 9 2024-02-23T00:56:12.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.361+0000 7f1b7ab7b700 20 client.42209.objecter get_session s=0x7f1b7411ca00 osd=4 3 2024-02-23T00:56:12.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.361+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_assign 4 9 2024-02-23T00:56:12.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.361+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b7411ca00 osd=4 4 2024-02-23T00:56:12.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.361+0000 7f1b7ab7b700 10 client.42209.objecter _send_command 9 2024-02-23T00:56:12.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.361+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b7411be90 con 0x7f1b7411cb70 2024-02-23T00:56:12.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.361+0000 7f1b78917700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f1b7411cb70 0x7f1b74123320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:12.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.362+0000 7f1b71ffb700 10 client.42209.objecter ms_handle_connect 0x7f1b7411cb70 2024-02-23T00:56:12.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.362+0000 7f1b78917700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f1b7411cb70 0x7f1b74123320 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:12.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.362+0000 7f1b71ffb700 1 -- 172.21.15.47:0/3638894407 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1b7411be90 con 0x7f1b7411cb70 2024-02-23T00:56:12.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.362+0000 7f1b71ffb700 10 client.42209.objecter ms_dispatch 0x7f1b74001550 command_reply(tid 9: 0 ) v1 2024-02-23T00:56:12.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.362+0000 7f1b71ffb700 10 client.42209.objecter _finish_command 9 = system:0 2024-02-23T00:56:12.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.362+0000 7f1b71ffb700 20 client.42209.objecter put_session s=0x7f1b7411ca00 osd=4 4 2024-02-23T00:56:12.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.362+0000 7f1b71ffb700 15 client.42209.objecter _session_command_op_remove 4 9 2024-02-23T00:56:12.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.379+0000 7f1b7ab7b700 10 client.42209.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:56:12.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.379+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_assign -1 10 2024-02-23T00:56:12.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.379+0000 7f1b7ab7b700 20 client.42209.objecter _get_session s=0x7f1b7411ca00 osd=4 3 2024-02-23T00:56:12.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.379+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b7411ca00 osd=4 3 2024-02-23T00:56:12.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.379+0000 7f1b7ab7b700 20 client.42209.objecter _get_session s=0x7f1b7411ca00 osd=4 3 2024-02-23T00:56:12.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.379+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_remove -1 10 2024-02-23T00:56:12.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.379+0000 7f1b7ab7b700 20 client.42209.objecter get_session s=0x7f1b7411ca00 osd=4 3 2024-02-23T00:56:12.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.379+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_assign 4 10 2024-02-23T00:56:12.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.379+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b7411ca00 osd=4 4 2024-02-23T00:56:12.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.379+0000 7f1b7ab7b700 10 client.42209.objecter _send_command 10 2024-02-23T00:56:12.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.379+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1b7411be90 con 0x7f1b7411cb70 2024-02-23T00:56:12.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.380+0000 7f1b71ffb700 1 -- 172.21.15.47:0/3638894407 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1b7411be90 con 0x7f1b7411cb70 2024-02-23T00:56:12.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.380+0000 7f1b71ffb700 10 client.42209.objecter ms_dispatch 0x7f1b74001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:56:12.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.380+0000 7f1b71ffb700 10 client.42209.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:56:12.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.380+0000 7f1b71ffb700 20 client.42209.objecter put_session s=0x7f1b7411ca00 osd=4 4 2024-02-23T00:56:12.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.380+0000 7f1b71ffb700 15 client.42209.objecter _session_command_op_remove 4 10 2024-02-23T00:56:12.384 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4: {} 2024-02-23T00:56:12.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.381+0000 7f1b7ab7b700 10 client.42209.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:56:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.381+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_assign -1 11 2024-02-23T00:56:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.381+0000 7f1b7ab7b700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f1b74123d20 0x7f1b74124160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.381+0000 7f1b7ab7b700 20 client.42209.objecter _get_session s=0x7f1b7411be90 osd=5 3 2024-02-23T00:56:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.381+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b7411be90 osd=5 3 2024-02-23T00:56:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.381+0000 7f1b7ab7b700 20 client.42209.objecter _get_session s=0x7f1b7411be90 osd=5 3 2024-02-23T00:56:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.381+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_remove -1 11 2024-02-23T00:56:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.381+0000 7f1b7ab7b700 20 client.42209.objecter get_session s=0x7f1b7411be90 osd=5 3 2024-02-23T00:56:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.381+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_assign 5 11 2024-02-23T00:56:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.381+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b7411be90 osd=5 4 2024-02-23T00:56:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.381+0000 7f1b7ab7b700 10 client.42209.objecter _send_command 11 2024-02-23T00:56:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.381+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b7412b1f0 con 0x7f1b74123d20 2024-02-23T00:56:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.381+0000 7f1b73fff700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f1b74123d20 0x7f1b74124160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.381+0000 7f1b71ffb700 10 client.42209.objecter ms_handle_connect 0x7f1b74123d20 2024-02-23T00:56:12.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.381+0000 7f1b73fff700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f1b74123d20 0x7f1b74124160 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:12.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.382+0000 7f1b71ffb700 1 -- 172.21.15.47:0/3638894407 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1b7412b1f0 con 0x7f1b74123d20 2024-02-23T00:56:12.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.382+0000 7f1b71ffb700 10 client.42209.objecter ms_dispatch 0x7f1b74001550 command_reply(tid 11: 0 ) v1 2024-02-23T00:56:12.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.382+0000 7f1b71ffb700 10 client.42209.objecter _finish_command 11 = system:0 2024-02-23T00:56:12.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.382+0000 7f1b71ffb700 20 client.42209.objecter put_session s=0x7f1b7411be90 osd=5 4 2024-02-23T00:56:12.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.382+0000 7f1b71ffb700 15 client.42209.objecter _session_command_op_remove 5 11 2024-02-23T00:56:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.397+0000 7f1b7ab7b700 10 client.42209.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:56:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.397+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_assign -1 12 2024-02-23T00:56:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.397+0000 7f1b7ab7b700 20 client.42209.objecter _get_session s=0x7f1b7411be90 osd=5 3 2024-02-23T00:56:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.397+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b7411be90 osd=5 3 2024-02-23T00:56:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.397+0000 7f1b7ab7b700 20 client.42209.objecter _get_session s=0x7f1b7411be90 osd=5 3 2024-02-23T00:56:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.397+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_remove -1 12 2024-02-23T00:56:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.397+0000 7f1b7ab7b700 20 client.42209.objecter get_session s=0x7f1b7411be90 osd=5 3 2024-02-23T00:56:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.397+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_assign 5 12 2024-02-23T00:56:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.397+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b7411be90 osd=5 4 2024-02-23T00:56:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.397+0000 7f1b7ab7b700 10 client.42209.objecter _send_command 12 2024-02-23T00:56:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.397+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1b74128a90 con 0x7f1b74123d20 2024-02-23T00:56:12.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.399+0000 7f1b71ffb700 1 -- 172.21.15.47:0/3638894407 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1b74128a90 con 0x7f1b74123d20 2024-02-23T00:56:12.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.399+0000 7f1b71ffb700 10 client.42209.objecter ms_dispatch 0x7f1b74001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:56:12.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.399+0000 7f1b71ffb700 10 client.42209.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:56:12.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.399+0000 7f1b71ffb700 20 client.42209.objecter put_session s=0x7f1b7411be90 osd=5 4 2024-02-23T00:56:12.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.399+0000 7f1b71ffb700 15 client.42209.objecter _session_command_op_remove 5 12 2024-02-23T00:56:12.402 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5: {} 2024-02-23T00:56:12.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.399+0000 7f1b7ab7b700 10 client.42209.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:56:12.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.399+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_assign -1 13 2024-02-23T00:56:12.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.399+0000 7f1b7ab7b700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f1b74129110 0x7f1b7412b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:12.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.399+0000 7f1b7ab7b700 20 client.42209.objecter _get_session s=0x7f1b74128a90 osd=6 3 2024-02-23T00:56:12.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.399+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b74128a90 osd=6 3 2024-02-23T00:56:12.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.399+0000 7f1b7ab7b700 20 client.42209.objecter _get_session s=0x7f1b74128a90 osd=6 3 2024-02-23T00:56:12.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.399+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_remove -1 13 2024-02-23T00:56:12.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.399+0000 7f1b7ab7b700 20 client.42209.objecter get_session s=0x7f1b74128a90 osd=6 3 2024-02-23T00:56:12.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.399+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_assign 6 13 2024-02-23T00:56:12.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.399+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b74128a90 osd=6 4 2024-02-23T00:56:12.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.399+0000 7f1b7ab7b700 10 client.42209.objecter _send_command 13 2024-02-23T00:56:12.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.399+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b7412e300 con 0x7f1b74129110 2024-02-23T00:56:12.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.399+0000 7f1b79118700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f1b74129110 0x7f1b7412b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:12.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.399+0000 7f1b71ffb700 10 client.42209.objecter ms_handle_connect 0x7f1b74129110 2024-02-23T00:56:12.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.399+0000 7f1b79118700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f1b74129110 0x7f1b7412b500 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:12.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.400+0000 7f1b71ffb700 1 -- 172.21.15.47:0/3638894407 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1b7412e300 con 0x7f1b74129110 2024-02-23T00:56:12.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.400+0000 7f1b71ffb700 10 client.42209.objecter ms_dispatch 0x7f1b74001550 command_reply(tid 13: 0 ) v1 2024-02-23T00:56:12.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.400+0000 7f1b71ffb700 10 client.42209.objecter _finish_command 13 = system:0 2024-02-23T00:56:12.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.400+0000 7f1b71ffb700 20 client.42209.objecter put_session s=0x7f1b74128a90 osd=6 4 2024-02-23T00:56:12.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.400+0000 7f1b71ffb700 15 client.42209.objecter _session_command_op_remove 6 13 2024-02-23T00:56:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.415+0000 7f1b7ab7b700 10 client.42209.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:56:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.415+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_assign -1 14 2024-02-23T00:56:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.415+0000 7f1b7ab7b700 20 client.42209.objecter _get_session s=0x7f1b74128a90 osd=6 3 2024-02-23T00:56:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.415+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b74128a90 osd=6 3 2024-02-23T00:56:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.415+0000 7f1b7ab7b700 20 client.42209.objecter _get_session s=0x7f1b74128a90 osd=6 3 2024-02-23T00:56:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.415+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_remove -1 14 2024-02-23T00:56:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.415+0000 7f1b7ab7b700 20 client.42209.objecter get_session s=0x7f1b74128a90 osd=6 3 2024-02-23T00:56:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.415+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_assign 6 14 2024-02-23T00:56:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.415+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b74128a90 osd=6 4 2024-02-23T00:56:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.415+0000 7f1b7ab7b700 10 client.42209.objecter _send_command 14 2024-02-23T00:56:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.415+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1b7412d0f0 con 0x7f1b74129110 2024-02-23T00:56:12.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.416+0000 7f1b71ffb700 1 -- 172.21.15.47:0/3638894407 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1b7412d0f0 con 0x7f1b74129110 2024-02-23T00:56:12.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.417+0000 7f1b71ffb700 10 client.42209.objecter ms_dispatch 0x7f1b74001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:56:12.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.417+0000 7f1b71ffb700 10 client.42209.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:56:12.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.417+0000 7f1b71ffb700 20 client.42209.objecter put_session s=0x7f1b74128a90 osd=6 4 2024-02-23T00:56:12.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.417+0000 7f1b71ffb700 15 client.42209.objecter _session_command_op_remove 6 14 2024-02-23T00:56:12.420 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6: {} 2024-02-23T00:56:12.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.417+0000 7f1b7ab7b700 10 client.42209.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:56:12.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.417+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_assign -1 15 2024-02-23T00:56:12.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.417+0000 7f1b7ab7b700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f1b7412d810 0x7f1b7412dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:12.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.417+0000 7f1b7ab7b700 20 client.42209.objecter _get_session s=0x7f1b7412d0f0 osd=7 3 2024-02-23T00:56:12.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.417+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b7412d0f0 osd=7 3 2024-02-23T00:56:12.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.417+0000 7f1b7ab7b700 20 client.42209.objecter _get_session s=0x7f1b7412d0f0 osd=7 3 2024-02-23T00:56:12.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.417+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_remove -1 15 2024-02-23T00:56:12.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.417+0000 7f1b7ab7b700 20 client.42209.objecter get_session s=0x7f1b7412d0f0 osd=7 3 2024-02-23T00:56:12.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.417+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_assign 7 15 2024-02-23T00:56:12.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.417+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b7412d0f0 osd=7 4 2024-02-23T00:56:12.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.417+0000 7f1b7ab7b700 10 client.42209.objecter _send_command 15 2024-02-23T00:56:12.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.417+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b74131550 con 0x7f1b7412d810 2024-02-23T00:56:12.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.417+0000 7f1b78917700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f1b7412d810 0x7f1b7412dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:12.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.418+0000 7f1b71ffb700 10 client.42209.objecter ms_handle_connect 0x7f1b7412d810 2024-02-23T00:56:12.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.418+0000 7f1b78917700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f1b7412d810 0x7f1b7412dc50 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:12.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.418+0000 7f1b71ffb700 1 -- 172.21.15.47:0/3638894407 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1b74131550 con 0x7f1b7412d810 2024-02-23T00:56:12.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.418+0000 7f1b71ffb700 10 client.42209.objecter ms_dispatch 0x7f1b74001550 command_reply(tid 15: 0 ) v1 2024-02-23T00:56:12.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.418+0000 7f1b71ffb700 10 client.42209.objecter _finish_command 15 = system:0 2024-02-23T00:56:12.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.418+0000 7f1b71ffb700 20 client.42209.objecter put_session s=0x7f1b7412d0f0 osd=7 4 2024-02-23T00:56:12.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.418+0000 7f1b71ffb700 15 client.42209.objecter _session_command_op_remove 7 15 2024-02-23T00:56:12.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.435+0000 7f1b7ab7b700 10 client.42209.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-23T00:56:12.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.435+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_assign -1 16 2024-02-23T00:56:12.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.435+0000 7f1b7ab7b700 20 client.42209.objecter _get_session s=0x7f1b7412d0f0 osd=7 3 2024-02-23T00:56:12.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.435+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b7412d0f0 osd=7 3 2024-02-23T00:56:12.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.435+0000 7f1b7ab7b700 20 client.42209.objecter _get_session s=0x7f1b7412d0f0 osd=7 3 2024-02-23T00:56:12.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.435+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_remove -1 16 2024-02-23T00:56:12.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.435+0000 7f1b7ab7b700 20 client.42209.objecter get_session s=0x7f1b7412d0f0 osd=7 3 2024-02-23T00:56:12.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.435+0000 7f1b7ab7b700 15 client.42209.objecter _session_command_op_assign 7 16 2024-02-23T00:56:12.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.435+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b7412d0f0 osd=7 4 2024-02-23T00:56:12.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.435+0000 7f1b7ab7b700 10 client.42209.objecter _send_command 16 2024-02-23T00:56:12.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.435+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1b74130340 con 0x7f1b7412d810 2024-02-23T00:56:12.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.437+0000 7f1b71ffb700 1 -- 172.21.15.47:0/3638894407 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1b74130340 con 0x7f1b7412d810 2024-02-23T00:56:12.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.437+0000 7f1b71ffb700 10 client.42209.objecter ms_dispatch 0x7f1b74001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-23T00:56:12.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.437+0000 7f1b71ffb700 10 client.42209.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-23T00:56:12.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.437+0000 7f1b71ffb700 20 client.42209.objecter put_session s=0x7f1b7412d0f0 osd=7 4 2024-02-23T00:56:12.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.437+0000 7f1b71ffb700 15 client.42209.objecter _session_command_op_remove 7 16 2024-02-23T00:56:12.441 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7: {} 2024-02-23T00:56:12.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 10 librados: watch_flush enter 2024-02-23T00:56:12.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 10 librados: watch_flush exit 2024-02-23T00:56:12.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 10 client.42209.objecter close_session for osd.0 2024-02-23T00:56:12.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f1b740feee0 msgr2=0x7f1b740ff2c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:12.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f1b740feee0 0x7f1b740ff2c0 crc :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:12.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b7405ee80 osd=0 1 2024-02-23T00:56:12.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 10 client.42209.objecter close_session for osd.1 2024-02-23T00:56:12.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f1b740598f0 msgr2=0x7f1b74059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:12.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7f1b740598f0 0x7f1b74059cd0 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:12.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b74028710 osd=1 1 2024-02-23T00:56:12.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 10 client.42209.objecter close_session for osd.2 2024-02-23T00:56:12.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f1b74028270 msgr2=0x7f1b7411c160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:12.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7f1b74028270 0x7f1b7411c160 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:12.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b74059770 osd=2 1 2024-02-23T00:56:12.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 10 client.42209.objecter close_session for osd.3 2024-02-23T00:56:12.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f1b7411f8c0 msgr2=0x7f1b74121cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:12.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f1b7411f8c0 0x7f1b74121cb0 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:12.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b74027bd0 osd=3 1 2024-02-23T00:56:12.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 10 client.42209.objecter close_session for osd.4 2024-02-23T00:56:12.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f1b7411cb70 msgr2=0x7f1b74123320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:12.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f1b7411cb70 0x7f1b74123320 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:12.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b7411ca00 osd=4 1 2024-02-23T00:56:12.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 10 client.42209.objecter close_session for osd.5 2024-02-23T00:56:12.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b79118700 1 -- 172.21.15.47:0/3638894407 reap_dead start 2024-02-23T00:56:12.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f1b74123d20 msgr2=0x7f1b74124160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:12.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f1b74123d20 0x7f1b74124160 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:12.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b79118700 1 -- 172.21.15.47:0/3638894407 reap_dead start 2024-02-23T00:56:12.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b7411be90 osd=5 1 2024-02-23T00:56:12.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 10 client.42209.objecter close_session for osd.6 2024-02-23T00:56:12.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f1b74129110 msgr2=0x7f1b7412b500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:12.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f1b74129110 0x7f1b7412b500 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:12.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b74128a90 osd=6 1 2024-02-23T00:56:12.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 10 client.42209.objecter close_session for osd.7 2024-02-23T00:56:12.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f1b7412d810 msgr2=0x7f1b7412dc50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:12.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f1b7412d810 0x7f1b7412dc50 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:12.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b79118700 1 -- 172.21.15.47:0/3638894407 reap_dead start 2024-02-23T00:56:12.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 20 client.42209.objecter put_session s=0x7f1b7412d0f0 osd=7 1 2024-02-23T00:56:12.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 20 client.42209.objecter shutdown clearing up homeless session... 2024-02-23T00:56:12.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 10 client.42209.objecter successfully canceled tick 2024-02-23T00:56:12.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1b64062cc0 msgr2=0x7f1b64065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:12.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1b64062cc0 0x7f1b64065180 secure :-1 s=READY pgs=3219 cs=0 l=1 rev1=1 rx=0x7f1b60004ee0 tx=0x7f1b6000b040).stop 2024-02-23T00:56:12.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1b740f6a90 msgr2=0x7f1b740f7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:12.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1b740f6a90 0x7f1b740f7680 secure :-1 s=READY pgs=2833 cs=0 l=1 rev1=1 rx=0x7f1b680048f0 tx=0x7f1b68009cb0).stop 2024-02-23T00:56:12.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 shutdown_connections 2024-02-23T00:56:12.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1b740f6a90 0x7f1b740f7680 unknown :-1 s=CLOSED pgs=2833 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:12.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1b64062cc0 0x7f1b64065180 unknown :-1 s=CLOSED pgs=3219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:12.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 1 --2- 172.21.15.47:0/3638894407 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f1b7412d810 0x7f1b7412dc50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:12.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 >> 172.21.15.47:0/3638894407 conn(0x7f1b7406c370 msgr2=0x7f1b740fb890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:56:12.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 shutdown_connections 2024-02-23T00:56:12.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 1 -- 172.21.15.47:0/3638894407 wait complete. 2024-02-23T00:56:12.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:12.438+0000 7f1b7ab7b700 1 librados: shutdown 2024-02-23T00:56:12.446 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-23T00:56:12.446 INFO:teuthology.orchestra.run.smithi047.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-23T00:56:12.446 INFO:teuthology.orchestra.run.smithi047.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-23T00:56:12.446 INFO:teuthology.orchestra.run.smithi047.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-23T00:56:12.447 INFO:teuthology.orchestra.run.smithi047.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-23T00:56:12.447 INFO:teuthology.orchestra.run.smithi047.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-23T00:56:12.447 INFO:teuthology.orchestra.run.smithi047.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-23T00:56:12.447 INFO:teuthology.orchestra.run.smithi047.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-23T00:56:12.694 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:56:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:56:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:56:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:11.311564+0000 mon.a (mon.0) 3835 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:56:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:11.411732+0000 mgr.y (mgr.14517) 686 : cluster [DBG] pgmap v1239: 154 pgs: 154 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1006 B/s rd, 0 op/s 2024-02-23T00:56:12.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:12.289098+0000 mon.c (mon.2) 522 : audit [DBG] from='client.? 172.21.15.47:0/3638894407' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:56:12.765 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:11.311564+0000 mon.a (mon.0) 3835 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:56:12.765 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:11.411732+0000 mgr.y (mgr.14517) 686 : cluster [DBG] pgmap v1239: 154 pgs: 154 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1006 B/s rd, 0 op/s 2024-02-23T00:56:12.765 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:12.289098+0000 mon.c (mon.2) 522 : audit [DBG] from='client.? 172.21.15.47:0/3638894407' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:56:12.766 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:11.311564+0000 mon.a (mon.0) 3835 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:56:12.766 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:11.411732+0000 mgr.y (mgr.14517) 686 : cluster [DBG] pgmap v1239: 154 pgs: 154 active+clean; 5.7 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1006 B/s rd, 0 op/s 2024-02-23T00:56:12.766 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:12.289098+0000 mon.c (mon.2) 522 : audit [DBG] from='client.? 172.21.15.47:0/3638894407' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-23T00:56:13.365 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset noscrub 2024-02-23T00:56:13.444 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:56:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:56:13.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:13.463393+0000 mon.a (mon.0) 3836 : cluster [WRN] Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-02-23T00:56:13.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:13.463393+0000 mon.a (mon.0) 3836 : cluster [WRN] Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-02-23T00:56:13.905 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:56:13.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:13.463393+0000 mon.a (mon.0) 3836 : cluster [WRN] Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-02-23T00:56:14.584 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:56:14.584 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:56:14.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:13.013032+0000 mgr.y (mgr.14517) 687 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:56:14.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:13.412472+0000 mgr.y (mgr.14517) 688 : cluster [DBG] pgmap v1240: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 6.2 KiB/s rd, 6 op/s; 1/675 objects misplaced (0.148%); 213 B/s, 13 objects/s recovering 2024-02-23T00:56:14.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:13.013032+0000 mgr.y (mgr.14517) 687 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:56:14.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:13.412472+0000 mgr.y (mgr.14517) 688 : cluster [DBG] pgmap v1240: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 6.2 KiB/s rd, 6 op/s; 1/675 objects misplaced (0.148%); 213 B/s, 13 objects/s recovering 2024-02-23T00:56:14.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:13.013032+0000 mgr.y (mgr.14517) 687 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:56:14.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:13.412472+0000 mgr.y (mgr.14517) 688 : cluster [DBG] pgmap v1240: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 6.2 KiB/s rd, 6 op/s; 1/675 objects misplaced (0.148%); 213 B/s, 13 objects/s recovering 2024-02-23T00:56:15.206 INFO:tasks.thrashosds.thrasher:inning osd 2024-02-23T00:56:15.206 INFO:tasks.thrashosds.thrasher:Adding osd 7 2024-02-23T00:56:15.206 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd in 7 2024-02-23T00:56:16.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:15.412845+0000 mgr.y (mgr.14517) 689 : cluster [DBG] pgmap v1241: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 4.7 KiB/s rd, 4 op/s; 1/675 objects misplaced (0.148%); 162 B/s, 10 objects/s recovering 2024-02-23T00:56:16.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:15.412845+0000 mgr.y (mgr.14517) 689 : cluster [DBG] pgmap v1241: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 4.7 KiB/s rd, 4 op/s; 1/675 objects misplaced (0.148%); 162 B/s, 10 objects/s recovering 2024-02-23T00:56:16.668 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.1 dump_ops_in_flight 2024-02-23T00:56:16.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:15.412845+0000 mgr.y (mgr.14517) 689 : cluster [DBG] pgmap v1241: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 4.7 KiB/s rd, 4 op/s; 1/675 objects misplaced (0.148%); 162 B/s, 10 objects/s recovering 2024-02-23T00:56:17.090 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:56:17.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.106+0000 7fe36c916700 1 -- 172.21.15.47:0/406694182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3680fe510 msgr2=0x7fe3680f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:17.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.106+0000 7fe36c916700 1 --2- 172.21.15.47:0/406694182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3680fe510 0x7fe3680f7d40 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 rx=0x7fe35c004870 tx=0x7fe35c009cb0).stop 2024-02-23T00:56:17.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.106+0000 7fe36c916700 1 -- 172.21.15.47:0/406694182 shutdown_connections 2024-02-23T00:56:17.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.106+0000 7fe36c916700 1 --2- 172.21.15.47:0/406694182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3680f8420 0x7fe368104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:17.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.106+0000 7fe36c916700 1 --2- 172.21.15.47:0/406694182 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe3680fdbb0 0x7fe3680fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:17.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.106+0000 7fe36c916700 1 --2- 172.21.15.47:0/406694182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3680fe510 0x7fe3680f7d40 unknown :-1 s=CLOSED pgs=2652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:17.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.106+0000 7fe36c916700 1 -- 172.21.15.47:0/406694182 >> 172.21.15.47:0/406694182 conn(0x7fe3680f3800 msgr2=0x7fe3680f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:56:17.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.107+0000 7fe36c916700 1 -- 172.21.15.47:0/406694182 shutdown_connections 2024-02-23T00:56:17.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.107+0000 7fe36c916700 1 -- 172.21.15.47:0/406694182 wait complete. 2024-02-23T00:56:17.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.107+0000 7fe36c916700 1 librados: starting msgr at 2024-02-23T00:56:17.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.107+0000 7fe36c916700 1 librados: starting objecter 2024-02-23T00:56:17.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.107+0000 7fe36c916700 1 Processor -- start 2024-02-23T00:56:17.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.107+0000 7fe36c916700 1 -- start start 2024-02-23T00:56:17.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.107+0000 7fe36c916700 1 librados: setting wanted keys 2024-02-23T00:56:17.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.107+0000 7fe36c916700 1 librados: calling monclient init 2024-02-23T00:56:17.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.108+0000 7fe36c916700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3680f8420 0x7fe368103830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:17.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.108+0000 7fe36c916700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe3680fdbb0 0x7fe368103d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:17.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.108+0000 7fe36c916700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3680fe510 0x7fe368111be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:17.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.108+0000 7fe36c916700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3681120d0 con 0x7fe3680f8420 2024-02-23T00:56:17.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.108+0000 7fe36c916700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe368112240 con 0x7fe3680fe510 2024-02-23T00:56:17.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.108+0000 7fe36c916700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe368112400 con 0x7fe3680fdbb0 2024-02-23T00:56:17.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.108+0000 7fe36659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3680f8420 0x7fe368103830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:17.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.108+0000 7fe36659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3680f8420 0x7fe368103830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59988/0 (socket says 172.21.15.47:59988) 2024-02-23T00:56:17.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.108+0000 7fe36659c700 1 -- 172.21.15.47:0/1817804055 learned_addr learned my addr 172.21.15.47:0/1817804055 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:56:17.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.108+0000 7fe366d9d700 1 --2- 172.21.15.47:0/1817804055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3680fe510 0x7fe368111be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:17.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.108+0000 7fe365d9b700 1 --2- 172.21.15.47:0/1817804055 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe3680fdbb0 0x7fe368103d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:17.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.108+0000 7fe36659c700 1 -- 172.21.15.47:0/1817804055 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe3680fdbb0 msgr2=0x7fe368103d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:17.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.108+0000 7fe36659c700 1 --2- 172.21.15.47:0/1817804055 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe3680fdbb0 0x7fe368103d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:17.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.108+0000 7fe36659c700 1 -- 172.21.15.47:0/1817804055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3680fe510 msgr2=0x7fe368111be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:17.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.108+0000 7fe36659c700 1 --2- 172.21.15.47:0/1817804055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3680fe510 0x7fe368111be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:17.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.108+0000 7fe36659c700 1 -- 172.21.15.47:0/1817804055 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c005040 con 0x7fe3680f8420 2024-02-23T00:56:17.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.108+0000 7fe36659c700 1 --2- 172.21.15.47:0/1817804055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3680f8420 0x7fe368103830 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 rx=0x7fe35800e990 tx=0x7fe35800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:17.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.108+0000 7fe3577fe700 10 client.?.objecter ms_handle_connect 0x7fe3680f8420 2024-02-23T00:56:17.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.108+0000 7fe3577fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:56:17.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.109+0000 7fe3577fe700 1 -- 172.21.15.47:0/1817804055 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35800ee80 con 0x7fe3680f8420 2024-02-23T00:56:17.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.109+0000 7fe36c916700 1 -- 172.21.15.47:0/1817804055 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe368112680 con 0x7fe3680f8420 2024-02-23T00:56:17.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.109+0000 7fe3577fe700 1 -- 172.21.15.47:0/1817804055 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fe35800f040 con 0x7fe3680f8420 2024-02-23T00:56:17.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.109+0000 7fe3577fe700 1 -- 172.21.15.47:0/1817804055 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35800c840 con 0x7fe3680f8420 2024-02-23T00:56:17.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.109+0000 7fe36c916700 10 client.31893.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:56:17.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.109+0000 7fe36c916700 1 -- 172.21.15.47:0/1817804055 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe368112ba0 con 0x7fe3680f8420 2024-02-23T00:56:17.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.109+0000 7fe36c916700 1 librados: init done 2024-02-23T00:56:17.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.110+0000 7fe36c916700 1 -- 172.21.15.47:0/1817804055 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36805ee80 con 0x7fe3680f8420 2024-02-23T00:56:17.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.111+0000 7fe3577fe700 1 -- 172.21.15.47:0/1817804055 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fe35800cc70 con 0x7fe3680f8420 2024-02-23T00:56:17.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.111+0000 7fe3577fe700 1 --2- 172.21.15.47:0/1817804055 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe350062cc0 0x7fe350065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:17.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.111+0000 7fe3577fe700 1 -- 172.21.15.47:0/1817804055 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(814..814 src has 621..814) v4 ==== 8638+0+0 (secure 0 0 0) 0x7fe35807d180 con 0x7fe3680f8420 2024-02-23T00:56:17.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.111+0000 7fe3577fe700 10 client.31893.objecter ms_dispatch 0x7fe368001550 osd_map(814..814 src has 621..814) v4 2024-02-23T00:56:17.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.111+0000 7fe3577fe700 3 client.31893.objecter handle_osd_map got epochs [814,814] > 0 2024-02-23T00:56:17.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.111+0000 7fe3577fe700 3 client.31893.objecter handle_osd_map decoding full epoch 814 2024-02-23T00:56:17.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.112+0000 7fe3577fe700 20 client.31893.objecter dump_active .. 0 homeless 2024-02-23T00:56:17.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.112+0000 7fe365d9b700 1 --2- 172.21.15.47:0/1817804055 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe350062cc0 0x7fe350065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:17.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.112+0000 7fe365d9b700 1 --2- 172.21.15.47:0/1817804055 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe350062cc0 0x7fe350065180 secure :-1 s=READY pgs=3220 cs=0 l=1 rev1=1 rx=0x7fe3681076f0 tx=0x7fe35c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:17.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.112+0000 7fe3577fe700 10 client.31893.objecter ms_handle_connect 0x7fe350062cc0 2024-02-23T00:56:17.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.116+0000 7fe3577fe700 1 -- 172.21.15.47:0/1817804055 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe35804ca60 con 0x7fe3680f8420 2024-02-23T00:56:17.137 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:56:17.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.306+0000 7fe36c916700 1 -- 172.21.15.47:0/1817804055 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7fe368104b00 con 0x7fe3680f8420 2024-02-23T00:56:17.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.343+0000 7fe3577fe700 1 -- 172.21.15.47:0/1817804055 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v815) v1 ==== 91+0+0 (secure 0 0 0) 0x7fe3580097d0 con 0x7fe3680f8420 2024-02-23T00:56:17.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.345+0000 7fe36c916700 10 librados: watch_flush enter 2024-02-23T00:56:17.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.348+0000 7fe36c916700 10 librados: watch_flush exit 2024-02-23T00:56:17.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.348+0000 7fe36c916700 20 client.31893.objecter shutdown clearing up homeless session... 2024-02-23T00:56:17.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.348+0000 7fe36c916700 10 client.31893.objecter successfully canceled tick 2024-02-23T00:56:17.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.348+0000 7fe36c916700 1 -- 172.21.15.47:0/1817804055 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe350062cc0 msgr2=0x7fe350065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:17.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.348+0000 7fe36c916700 1 --2- 172.21.15.47:0/1817804055 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe350062cc0 0x7fe350065180 secure :-1 s=READY pgs=3220 cs=0 l=1 rev1=1 rx=0x7fe3681076f0 tx=0x7fe35c00b040).stop 2024-02-23T00:56:17.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.348+0000 7fe36c916700 1 -- 172.21.15.47:0/1817804055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3680f8420 msgr2=0x7fe368103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:17.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.348+0000 7fe36c916700 1 --2- 172.21.15.47:0/1817804055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3680f8420 0x7fe368103830 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 rx=0x7fe35800e990 tx=0x7fe35800b040).stop 2024-02-23T00:56:17.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.348+0000 7fe36c916700 1 -- 172.21.15.47:0/1817804055 shutdown_connections 2024-02-23T00:56:17.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.348+0000 7fe36c916700 1 --2- 172.21.15.47:0/1817804055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3680fe510 0x7fe368111be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:17.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.348+0000 7fe36c916700 1 --2- 172.21.15.47:0/1817804055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3680f8420 0x7fe368103830 unknown :-1 s=CLOSED pgs=2653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:17.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.348+0000 7fe36c916700 1 --2- 172.21.15.47:0/1817804055 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fe350062cc0 0x7fe350065180 unknown :-1 s=CLOSED pgs=3220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:17.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.348+0000 7fe36c916700 1 --2- 172.21.15.47:0/1817804055 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe3680fdbb0 0x7fe368103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:17.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.348+0000 7fe36c916700 1 -- 172.21.15.47:0/1817804055 >> 172.21.15.47:0/1817804055 conn(0x7fe3680f3800 msgr2=0x7fe3680f97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:56:17.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.348+0000 7fe36c916700 1 -- 172.21.15.47:0/1817804055 shutdown_connections 2024-02-23T00:56:17.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.348+0000 7fe36c916700 1 -- 172.21.15.47:0/1817804055 wait complete. 2024-02-23T00:56:17.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:17.348+0000 7fe36c916700 1 librados: shutdown 2024-02-23T00:56:17.352 INFO:teuthology.orchestra.run.smithi047.stderr:noscrub is unset 2024-02-23T00:56:17.618 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:17.308227+0000 mon.a (mon.0) 3837 : audit [INF] from='client.? 172.21.15.47:0/1817804055' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:56:17.618 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:17.308227+0000 mon.a (mon.0) 3837 : audit [INF] from='client.? 172.21.15.47:0/1817804055' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:56:17.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:17.308227+0000 mon.a (mon.0) 3837 : audit [INF] from='client.? 172.21.15.47:0/1817804055' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-23T00:56:18.614 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:17.335500+0000 mon.a (mon.0) 3838 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:56:18.614 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:17.343745+0000 mon.a (mon.0) 3839 : audit [INF] from='client.? 172.21.15.47:0/1817804055' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:56:18.614 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:17.343809+0000 mon.a (mon.0) 3840 : cluster [DBG] osdmap e815: 8 total, 8 up, 7 in 2024-02-23T00:56:18.615 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:17.413228+0000 mgr.y (mgr.14517) 690 : cluster [DBG] pgmap v1243: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 5.3 KiB/s rd, 5 op/s; 1/675 objects misplaced (0.148%); 160 B/s, 10 objects/s recovering 2024-02-23T00:56:18.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:17.335500+0000 mon.a (mon.0) 3838 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:56:18.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:17.343745+0000 mon.a (mon.0) 3839 : audit [INF] from='client.? 172.21.15.47:0/1817804055' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:56:18.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:17.343809+0000 mon.a (mon.0) 3840 : cluster [DBG] osdmap e815: 8 total, 8 up, 7 in 2024-02-23T00:56:18.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:17.413228+0000 mgr.y (mgr.14517) 690 : cluster [DBG] pgmap v1243: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 5.3 KiB/s rd, 5 op/s; 1/675 objects misplaced (0.148%); 160 B/s, 10 objects/s recovering 2024-02-23T00:56:18.635 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-23T00:56:18.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:17.335500+0000 mon.a (mon.0) 3838 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-23T00:56:18.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:17.343745+0000 mon.a (mon.0) 3839 : audit [INF] from='client.? 172.21.15.47:0/1817804055' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-23T00:56:18.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:17.343809+0000 mon.a (mon.0) 3840 : cluster [DBG] osdmap e815: 8 total, 8 up, 7 in 2024-02-23T00:56:18.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:17.413228+0000 mgr.y (mgr.14517) 690 : cluster [DBG] pgmap v1243: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 5.3 KiB/s rd, 5 op/s; 1/675 objects misplaced (0.148%); 160 B/s, 10 objects/s recovering 2024-02-23T00:56:19.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:19.149316+0000 osd.5 (osd.5) 354 : cluster [DBG] 4.2 scrub starts 2024-02-23T00:56:19.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:19.151202+0000 osd.5 (osd.5) 355 : cluster [DBG] 4.2 scrub ok 2024-02-23T00:56:19.698 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:19.149316+0000 osd.5 (osd.5) 354 : cluster [DBG] 4.2 scrub starts 2024-02-23T00:56:19.698 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:19.151202+0000 osd.5 (osd.5) 355 : cluster [DBG] 4.2 scrub ok 2024-02-23T00:56:19.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:19.149316+0000 osd.5 (osd.5) 354 : cluster [DBG] 4.2 scrub starts 2024-02-23T00:56:19.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:19.151202+0000 osd.5 (osd.5) 355 : cluster [DBG] 4.2 scrub ok 2024-02-23T00:56:20.142 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:56:20.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.359+0000 7f8c0d6dd700 1 -- 172.21.15.47:0/1704196918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c080fd510 msgr2=0x7f8c080f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:20.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.359+0000 7f8c0d6dd700 1 --2- 172.21.15.47:0/1704196918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c080fd510 0x7f8c080f6d40 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 rx=0x7f8bfc004870 tx=0x7f8bfc009cb0).stop 2024-02-23T00:56:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.359+0000 7f8c0d6dd700 1 -- 172.21.15.47:0/1704196918 shutdown_connections 2024-02-23T00:56:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.359+0000 7f8c0d6dd700 1 --2- 172.21.15.47:0/1704196918 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8c080f7400 0x7f8c08103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.359+0000 7f8c0d6dd700 1 --2- 172.21.15.47:0/1704196918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c080fcbb0 0x7f8c080fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.359+0000 7f8c0d6dd700 1 --2- 172.21.15.47:0/1704196918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c080fd510 0x7f8c080f6d40 unknown :-1 s=CLOSED pgs=2654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.359+0000 7f8c0d6dd700 1 -- 172.21.15.47:0/1704196918 >> 172.21.15.47:0/1704196918 conn(0x7f8c080f2820 msgr2=0x7f8c080f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:56:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.359+0000 7f8c0d6dd700 1 -- 172.21.15.47:0/1704196918 shutdown_connections 2024-02-23T00:56:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.359+0000 7f8c0d6dd700 1 -- 172.21.15.47:0/1704196918 wait complete. 2024-02-23T00:56:20.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.360+0000 7f8c0d6dd700 1 librados: starting msgr at 2024-02-23T00:56:20.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.360+0000 7f8c0d6dd700 1 librados: starting objecter 2024-02-23T00:56:20.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.360+0000 7f8c0d6dd700 1 Processor -- start 2024-02-23T00:56:20.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.360+0000 7f8c0d6dd700 1 -- start start 2024-02-23T00:56:20.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.360+0000 7f8c0d6dd700 1 librados: setting wanted keys 2024-02-23T00:56:20.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.360+0000 7f8c0d6dd700 1 librados: calling monclient init 2024-02-23T00:56:20.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.361+0000 7f8c0d6dd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c080f7400 0x7f8c08109130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.361+0000 7f8c0d6dd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8c080fcbb0 0x7f8c08109620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.361+0000 7f8c0d6dd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c080fd510 0x7f8c08109c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.361+0000 7f8c0d6dd700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c08114310 con 0x7f8c080f7400 2024-02-23T00:56:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.361+0000 7f8c0d6dd700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c08114480 con 0x7f8c080fd510 2024-02-23T00:56:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.361+0000 7f8c0d6dd700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c08114640 con 0x7f8c080fcbb0 2024-02-23T00:56:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.361+0000 7f8c06ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c080f7400 0x7f8c08109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.361+0000 7f8c06ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c080f7400 0x7f8c08109130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60006/0 (socket says 172.21.15.47:60006) 2024-02-23T00:56:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.361+0000 7f8c06ffd700 1 -- 172.21.15.47:0/42810872 learned_addr learned my addr 172.21.15.47:0/42810872 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:56:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.361+0000 7f8c067fc700 1 --2- 172.21.15.47:0/42810872 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8c080fcbb0 0x7f8c08109620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.361+0000 7f8c077fe700 1 --2- 172.21.15.47:0/42810872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c080fd510 0x7f8c08109c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.361+0000 7f8c06ffd700 1 -- 172.21.15.47:0/42810872 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8c080fcbb0 msgr2=0x7f8c08109620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.361+0000 7f8c06ffd700 1 --2- 172.21.15.47:0/42810872 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8c080fcbb0 0x7f8c08109620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.361+0000 7f8c06ffd700 1 -- 172.21.15.47:0/42810872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c080fd510 msgr2=0x7f8c08109c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.361+0000 7f8c06ffd700 1 --2- 172.21.15.47:0/42810872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c080fd510 0x7f8c08109c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:20.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.361+0000 7f8c06ffd700 1 -- 172.21.15.47:0/42810872 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c000097c0 con 0x7f8c080f7400 2024-02-23T00:56:20.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.361+0000 7f8c077fe700 1 --2- 172.21.15.47:0/42810872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c080fd510 0x7f8c08109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:56:20.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.361+0000 7f8c067fc700 1 --2- 172.21.15.47:0/42810872 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8c080fcbb0 0x7f8c08109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:56:20.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.361+0000 7f8c06ffd700 1 --2- 172.21.15.47:0/42810872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c080f7400 0x7f8c08109130 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 rx=0x7f8bf800e970 tx=0x7f8bf800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:20.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.361+0000 7f8beffff700 10 client.?.objecter ms_handle_connect 0x7f8c080f7400 2024-02-23T00:56:20.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.361+0000 7f8beffff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:56:20.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.362+0000 7f8beffff700 1 -- 172.21.15.47:0/42810872 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8004660 con 0x7f8c080f7400 2024-02-23T00:56:20.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.362+0000 7f8beffff700 1 -- 172.21.15.47:0/42810872 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f8bf800f040 con 0x7f8c080f7400 2024-02-23T00:56:20.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.362+0000 7f8c0d6dd700 1 -- 172.21.15.47:0/42810872 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bfc005040 con 0x7f8c080f7400 2024-02-23T00:56:20.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.362+0000 7f8c0d6dd700 10 client.31899.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:56:20.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.362+0000 7f8c0d6dd700 1 -- 172.21.15.47:0/42810872 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c08114ca0 con 0x7f8c080f7400 2024-02-23T00:56:20.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.362+0000 7f8c0d6dd700 1 librados: init done 2024-02-23T00:56:20.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.362+0000 7f8beffff700 1 -- 172.21.15.47:0/42810872 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf80146f0 con 0x7f8c080f7400 2024-02-23T00:56:20.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.363+0000 7f8c0d6dd700 1 -- 172.21.15.47:0/42810872 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf4004f50 con 0x7f8c080f7400 2024-02-23T00:56:20.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.363+0000 7f8beffff700 1 -- 172.21.15.47:0/42810872 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f8bf8007b40 con 0x7f8c080f7400 2024-02-23T00:56:20.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.364+0000 7f8beffff700 1 --2- 172.21.15.47:0/42810872 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8bf00670d0 0x7f8bf0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:20.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.364+0000 7f8beffff700 1 -- 172.21.15.47:0/42810872 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(816..816 src has 621..816) v4 ==== 8638+0+0 (secure 0 0 0) 0x7f8bf807cef0 con 0x7f8c080f7400 2024-02-23T00:56:20.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.364+0000 7f8beffff700 10 client.31899.objecter ms_dispatch 0x7f8c08001550 osd_map(816..816 src has 621..816) v4 2024-02-23T00:56:20.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.364+0000 7f8beffff700 3 client.31899.objecter handle_osd_map got epochs [816,816] > 0 2024-02-23T00:56:20.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.364+0000 7f8beffff700 3 client.31899.objecter handle_osd_map decoding full epoch 816 2024-02-23T00:56:20.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.364+0000 7f8c067fc700 1 --2- 172.21.15.47:0/42810872 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8bf00670d0 0x7f8bf0069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:20.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.364+0000 7f8beffff700 20 client.31899.objecter dump_active .. 0 homeless 2024-02-23T00:56:20.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.364+0000 7f8beffff700 10 client.31899.objecter ms_handle_connect 0x7f8bf00670d0 2024-02-23T00:56:20.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.364+0000 7f8c067fc700 1 --2- 172.21.15.47:0/42810872 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8bf00670d0 0x7f8bf0069590 secure :-1 s=READY pgs=3221 cs=0 l=1 rev1=1 rx=0x7f8bfc004ee0 tx=0x7f8bfc00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:20.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.368+0000 7f8beffff700 1 -- 172.21.15.47:0/42810872 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f8bf804af60 con 0x7f8c080f7400 2024-02-23T00:56:20.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:18.620063+0000 osd.6 (osd.6) 319 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:56:20.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:18.620316+0000 osd.6 (osd.6) 320 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:56:20.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:19.001834+0000 osd.6 (osd.6) 321 : cluster [DBG] 6.1 scrub starts 2024-02-23T00:56:20.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:19.003003+0000 osd.6 (osd.6) 322 : cluster [DBG] 6.1 scrub ok 2024-02-23T00:56:20.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:19.165771+0000 osd.2 (osd.2) 173 : cluster [DBG] 4.1c scrub starts 2024-02-23T00:56:20.457 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:19.167591+0000 osd.2 (osd.2) 174 : cluster [DBG] 4.1c scrub ok 2024-02-23T00:56:20.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:19.348691+0000 mon.a (mon.0) 3841 : cluster [DBG] osdmap e816: 8 total, 8 up, 7 in 2024-02-23T00:56:20.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:19.413572+0000 mgr.y (mgr.14517) 691 : cluster [DBG] pgmap v1245: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 5.4 KiB/s rd, 5 op/s; 1/675 objects misplaced (0.148%); 162 B/s, 10 objects/s recovering 2024-02-23T00:56:20.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:19.649528+0000 mon.a (mon.0) 3842 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:56:20.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:20.033105+0000 mon.a (mon.0) 3843 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:56:20.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:20.033780+0000 mon.a (mon.0) 3844 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:56:20.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:20.038150+0000 mon.a (mon.0) 3845 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:56:20.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:20.127507+0000 osd.5 (osd.5) 356 : cluster [DBG] 5.19 scrub starts 2024-02-23T00:56:20.458 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:20.128751+0000 osd.5 (osd.5) 357 : cluster [DBG] 5.19 scrub ok 2024-02-23T00:56:20.458 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:18.620063+0000 osd.6 (osd.6) 319 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:56:20.459 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:18.620316+0000 osd.6 (osd.6) 320 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:56:20.459 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:19.001834+0000 osd.6 (osd.6) 321 : cluster [DBG] 6.1 scrub starts 2024-02-23T00:56:20.459 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:19.003003+0000 osd.6 (osd.6) 322 : cluster [DBG] 6.1 scrub ok 2024-02-23T00:56:20.459 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:19.165771+0000 osd.2 (osd.2) 173 : cluster [DBG] 4.1c scrub starts 2024-02-23T00:56:20.459 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:19.167591+0000 osd.2 (osd.2) 174 : cluster [DBG] 4.1c scrub ok 2024-02-23T00:56:20.459 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:19.348691+0000 mon.a (mon.0) 3841 : cluster [DBG] osdmap e816: 8 total, 8 up, 7 in 2024-02-23T00:56:20.459 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:19.413572+0000 mgr.y (mgr.14517) 691 : cluster [DBG] pgmap v1245: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 5.4 KiB/s rd, 5 op/s; 1/675 objects misplaced (0.148%); 162 B/s, 10 objects/s recovering 2024-02-23T00:56:20.459 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:19.649528+0000 mon.a (mon.0) 3842 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:56:20.459 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:20.033105+0000 mon.a (mon.0) 3843 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:56:20.459 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:20.033780+0000 mon.a (mon.0) 3844 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:56:20.459 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:20.038150+0000 mon.a (mon.0) 3845 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:56:20.460 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:20.127507+0000 osd.5 (osd.5) 356 : cluster [DBG] 5.19 scrub starts 2024-02-23T00:56:20.460 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:20.128751+0000 osd.5 (osd.5) 357 : cluster [DBG] 5.19 scrub ok 2024-02-23T00:56:20.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:20.557+0000 7f8c0d6dd700 1 -- 172.21.15.47:0/42810872 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd in", "ids": ["7"]} v 0) v1 -- 0x7f8bf40054d0 con 0x7f8c080f7400 2024-02-23T00:56:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:18.620063+0000 osd.6 (osd.6) 319 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:56:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:18.620316+0000 osd.6 (osd.6) 320 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:56:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:19.001834+0000 osd.6 (osd.6) 321 : cluster [DBG] 6.1 scrub starts 2024-02-23T00:56:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:19.003003+0000 osd.6 (osd.6) 322 : cluster [DBG] 6.1 scrub ok 2024-02-23T00:56:20.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:19.165771+0000 osd.2 (osd.2) 173 : cluster [DBG] 4.1c scrub starts 2024-02-23T00:56:20.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:19.167591+0000 osd.2 (osd.2) 174 : cluster [DBG] 4.1c scrub ok 2024-02-23T00:56:20.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:19.348691+0000 mon.a (mon.0) 3841 : cluster [DBG] osdmap e816: 8 total, 8 up, 7 in 2024-02-23T00:56:20.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:19.413572+0000 mgr.y (mgr.14517) 691 : cluster [DBG] pgmap v1245: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 5.4 KiB/s rd, 5 op/s; 1/675 objects misplaced (0.148%); 162 B/s, 10 objects/s recovering 2024-02-23T00:56:20.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:19.649528+0000 mon.a (mon.0) 3842 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:56:20.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:20.033105+0000 mon.a (mon.0) 3843 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:56:20.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:20.033780+0000 mon.a (mon.0) 3844 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:56:20.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:20.038150+0000 mon.a (mon.0) 3845 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:56:20.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:20.127507+0000 osd.5 (osd.5) 356 : cluster [DBG] 5.19 scrub starts 2024-02-23T00:56:20.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:20.128751+0000 osd.5 (osd.5) 357 : cluster [DBG] 5.19 scrub ok 2024-02-23T00:56:20.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:56:20.557+0000 7f3c89cf1700 -1 mon.a@0(leader).osd e816 definitely_dead 0 2024-02-23T00:56:20.807 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:56:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:56:20] "GET /metrics HTTP/1.1" 200 187117 "" "Prometheus/2.33.4" 2024-02-23T00:56:20.813 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:56:20.813 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:56:21.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:21.372+0000 7f8beffff700 1 -- 172.21.15.47:0/42810872 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd in", "ids": ["7"]}]=0 marked in osd.7. v817) v1 ==== 85+0+0 (secure 0 0 0) 0x7f8bf8019030 con 0x7f8c080f7400 2024-02-23T00:56:21.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:21.381+0000 7f8c0d6dd700 10 librados: watch_flush enter 2024-02-23T00:56:21.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:21.381+0000 7f8c0d6dd700 10 librados: watch_flush exit 2024-02-23T00:56:21.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:21.381+0000 7f8c0d6dd700 20 client.31899.objecter shutdown clearing up homeless session... 2024-02-23T00:56:21.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:21.381+0000 7f8c0d6dd700 10 client.31899.objecter successfully canceled tick 2024-02-23T00:56:21.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:21.381+0000 7f8c0d6dd700 1 -- 172.21.15.47:0/42810872 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8bf00670d0 msgr2=0x7f8bf0069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:21.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:21.381+0000 7f8c0d6dd700 1 --2- 172.21.15.47:0/42810872 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8bf00670d0 0x7f8bf0069590 secure :-1 s=READY pgs=3221 cs=0 l=1 rev1=1 rx=0x7f8bfc004ee0 tx=0x7f8bfc00b040).stop 2024-02-23T00:56:21.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:21.381+0000 7f8c0d6dd700 1 -- 172.21.15.47:0/42810872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c080f7400 msgr2=0x7f8c08109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:21.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:21.381+0000 7f8c0d6dd700 1 --2- 172.21.15.47:0/42810872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c080f7400 0x7f8c08109130 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 rx=0x7f8bf800e970 tx=0x7f8bf800b040).stop 2024-02-23T00:56:21.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:21.382+0000 7f8c0d6dd700 1 -- 172.21.15.47:0/42810872 shutdown_connections 2024-02-23T00:56:21.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:21.382+0000 7f8c0d6dd700 1 --2- 172.21.15.47:0/42810872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c080fd510 0x7f8c08109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:21.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:21.382+0000 7f8c0d6dd700 1 --2- 172.21.15.47:0/42810872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c080f7400 0x7f8c08109130 unknown :-1 s=CLOSED pgs=2655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:21.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:21.382+0000 7f8c0d6dd700 1 --2- 172.21.15.47:0/42810872 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8bf00670d0 0x7f8bf0069590 unknown :-1 s=CLOSED pgs=3221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:21.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:21.382+0000 7f8c0d6dd700 1 --2- 172.21.15.47:0/42810872 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8c080fcbb0 0x7f8c08109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:21.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:21.382+0000 7f8c0d6dd700 1 -- 172.21.15.47:0/42810872 >> 172.21.15.47:0/42810872 conn(0x7f8c080f2820 msgr2=0x7f8c080faa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:56:21.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:21.383+0000 7f8c0d6dd700 1 -- 172.21.15.47:0/42810872 shutdown_connections 2024-02-23T00:56:21.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:21.384+0000 7f8c0d6dd700 1 -- 172.21.15.47:0/42810872 wait complete. 2024-02-23T00:56:21.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:21.384+0000 7f8c0d6dd700 1 librados: shutdown 2024-02-23T00:56:21.388 INFO:teuthology.orchestra.run.smithi047.stderr:marked in osd.7. 2024-02-23T00:56:21.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:20.017832+0000 osd.0 (osd.0) 304 : cluster [DBG] 4.0 scrub starts 2024-02-23T00:56:21.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:20.021909+0000 osd.0 (osd.0) 305 : cluster [DBG] 4.0 scrub ok 2024-02-23T00:56:21.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:20.451403+0000 osd.4 (osd.4) 341 : cluster [DBG] 4.1d scrub starts 2024-02-23T00:56:21.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:20.452803+0000 osd.4 (osd.4) 342 : cluster [DBG] 4.1d scrub ok 2024-02-23T00:56:21.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:20.558803+0000 mon.a (mon.0) 3846 : audit [DBG] from='client.? 172.21.15.47:0/42810872' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["7"]}]: dispatch 2024-02-23T00:56:21.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:20.017832+0000 osd.0 (osd.0) 304 : cluster [DBG] 4.0 scrub starts 2024-02-23T00:56:21.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:20.021909+0000 osd.0 (osd.0) 305 : cluster [DBG] 4.0 scrub ok 2024-02-23T00:56:21.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:20.451403+0000 osd.4 (osd.4) 341 : cluster [DBG] 4.1d scrub starts 2024-02-23T00:56:21.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:20.452803+0000 osd.4 (osd.4) 342 : cluster [DBG] 4.1d scrub ok 2024-02-23T00:56:21.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:20.558803+0000 mon.a (mon.0) 3846 : audit [DBG] from='client.? 172.21.15.47:0/42810872' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["7"]}]: dispatch 2024-02-23T00:56:21.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:20.017832+0000 osd.0 (osd.0) 304 : cluster [DBG] 4.0 scrub starts 2024-02-23T00:56:21.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:20.021909+0000 osd.0 (osd.0) 305 : cluster [DBG] 4.0 scrub ok 2024-02-23T00:56:21.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:20.451403+0000 osd.4 (osd.4) 341 : cluster [DBG] 4.1d scrub starts 2024-02-23T00:56:21.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:20.452803+0000 osd.4 (osd.4) 342 : cluster [DBG] 4.1d scrub ok 2024-02-23T00:56:21.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:20.558803+0000 mon.a (mon.0) 3846 : audit [DBG] from='client.? 172.21.15.47:0/42810872' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["7"]}]: dispatch 2024-02-23T00:56:22.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:21.361317+0000 mon.a (mon.0) 3847 : audit [INF] from='client.? 172.21.15.47:0/42810872' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["7"]}]': finished 2024-02-23T00:56:22.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:21.361397+0000 mon.a (mon.0) 3848 : cluster [DBG] osdmap e817: 8 total, 8 up, 8 in 2024-02-23T00:56:22.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:21.413919+0000 mgr.y (mgr.14517) 692 : cluster [DBG] pgmap v1247: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s; 1/675 objects misplaced (0.148%) 2024-02-23T00:56:22.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:22.104150+0000 osd.5 (osd.5) 358 : cluster [DBG] 4.4 scrub starts 2024-02-23T00:56:22.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:22.107689+0000 osd.5 (osd.5) 359 : cluster [DBG] 4.4 scrub ok 2024-02-23T00:56:22.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:21.361317+0000 mon.a (mon.0) 3847 : audit [INF] from='client.? 172.21.15.47:0/42810872' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["7"]}]': finished 2024-02-23T00:56:22.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:21.361397+0000 mon.a (mon.0) 3848 : cluster [DBG] osdmap e817: 8 total, 8 up, 8 in 2024-02-23T00:56:22.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:21.413919+0000 mgr.y (mgr.14517) 692 : cluster [DBG] pgmap v1247: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s; 1/675 objects misplaced (0.148%) 2024-02-23T00:56:22.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:22.104150+0000 osd.5 (osd.5) 358 : cluster [DBG] 4.4 scrub starts 2024-02-23T00:56:22.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:22.107689+0000 osd.5 (osd.5) 359 : cluster [DBG] 4.4 scrub ok 2024-02-23T00:56:22.687 INFO:tasks.thrashosds.thrasher:Added osd 7 2024-02-23T00:56:22.687 INFO:tasks.thrashosds.thrasher:fixing pg num pool default.rgw.meta 2024-02-23T00:56:22.687 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd pool set default.rgw.meta pgp_num 12 2024-02-23T00:56:22.694 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:56:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:56:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:56:22.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:21.361317+0000 mon.a (mon.0) 3847 : audit [INF] from='client.? 172.21.15.47:0/42810872' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["7"]}]': finished 2024-02-23T00:56:22.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:21.361397+0000 mon.a (mon.0) 3848 : cluster [DBG] osdmap e817: 8 total, 8 up, 8 in 2024-02-23T00:56:22.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:21.413919+0000 mgr.y (mgr.14517) 692 : cluster [DBG] pgmap v1247: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s; 1/675 objects misplaced (0.148%) 2024-02-23T00:56:22.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:22.104150+0000 osd.5 (osd.5) 358 : cluster [DBG] 4.4 scrub starts 2024-02-23T00:56:22.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:22.107689+0000 osd.5 (osd.5) 359 : cluster [DBG] 4.4 scrub ok 2024-02-23T00:56:23.366 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:56:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:56:23.464 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.1 dump_blocked_ops 2024-02-23T00:56:23.474 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:56:23.670 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:22.366962+0000 mon.a (mon.0) 3849 : cluster [DBG] osdmap e818: 8 total, 8 up, 8 in 2024-02-23T00:56:23.670 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:22.366962+0000 mon.a (mon.0) 3849 : cluster [DBG] osdmap e818: 8 total, 8 up, 8 in 2024-02-23T00:56:23.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:22.366962+0000 mon.a (mon.0) 3849 : cluster [DBG] osdmap e818: 8 total, 8 up, 8 in 2024-02-23T00:56:24.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.120+0000 7f0118754700 1 -- 172.21.15.47:0/1328882911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01100fcbb0 msgr2=0x7f01100fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:24.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.120+0000 7f0118754700 1 --2- 172.21.15.47:0/1328882911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01100fcbb0 0x7f01100fcf90 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 rx=0x7f010c004870 tx=0x7f010c009cb0).stop 2024-02-23T00:56:24.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.120+0000 7f0118754700 1 -- 172.21.15.47:0/1328882911 shutdown_connections 2024-02-23T00:56:24.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.120+0000 7f0118754700 1 --2- 172.21.15.47:0/1328882911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01100f7400 0x7f0110103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:24.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.120+0000 7f0118754700 1 --2- 172.21.15.47:0/1328882911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01100fcbb0 0x7f01100fcf90 unknown :-1 s=CLOSED pgs=2656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:24.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.120+0000 7f0118754700 1 --2- 172.21.15.47:0/1328882911 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f01100fd510 0x7f01100f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:24.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.120+0000 7f0118754700 1 -- 172.21.15.47:0/1328882911 >> 172.21.15.47:0/1328882911 conn(0x7f01100f2820 msgr2=0x7f01100f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:56:24.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.120+0000 7f0118754700 1 -- 172.21.15.47:0/1328882911 shutdown_connections 2024-02-23T00:56:24.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.120+0000 7f0118754700 1 -- 172.21.15.47:0/1328882911 wait complete. 2024-02-23T00:56:24.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.121+0000 7f0118754700 1 librados: starting msgr at 2024-02-23T00:56:24.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.121+0000 7f0118754700 1 librados: starting objecter 2024-02-23T00:56:24.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.121+0000 7f0118754700 1 Processor -- start 2024-02-23T00:56:24.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.121+0000 7f0118754700 1 -- start start 2024-02-23T00:56:24.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.121+0000 7f0118754700 1 librados: setting wanted keys 2024-02-23T00:56:24.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.121+0000 7f0118754700 1 librados: calling monclient init 2024-02-23T00:56:24.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.121+0000 7f0118754700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f01100f7400 0x7f011010e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:24.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.121+0000 7f0118754700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01100fcbb0 0x7f011010e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:24.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.121+0000 7f0118754700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01100fd510 0x7f0110108200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:24.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.121+0000 7f0118754700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01101086f0 con 0x7f01100fcbb0 2024-02-23T00:56:24.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.121+0000 7f0118754700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0110108860 con 0x7f01100fd510 2024-02-23T00:56:24.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.121+0000 7f0118754700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0110108a20 con 0x7f01100f7400 2024-02-23T00:56:24.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.121+0000 7f01164f0700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f01100f7400 0x7f011010e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:24.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.121+0000 7f0115cef700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01100fcbb0 0x7f011010e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:24.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.121+0000 7f01164f0700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f01100f7400 0x7f011010e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:36114/0 (socket says 172.21.15.47:36114) 2024-02-23T00:56:24.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.121+0000 7f01164f0700 1 -- 172.21.15.47:0/2942102728 learned_addr learned my addr 172.21.15.47:0/2942102728 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:56:24.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.121+0000 7f0116cf1700 1 --2- 172.21.15.47:0/2942102728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01100fd510 0x7f0110108200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:24.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.122+0000 7f0115cef700 1 -- 172.21.15.47:0/2942102728 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f01100f7400 msgr2=0x7f011010e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:24.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.122+0000 7f0115cef700 1 --2- 172.21.15.47:0/2942102728 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f01100f7400 0x7f011010e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:24.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.122+0000 7f0115cef700 1 -- 172.21.15.47:0/2942102728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01100fd510 msgr2=0x7f0110108200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:24.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.122+0000 7f0115cef700 1 --2- 172.21.15.47:0/2942102728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01100fd510 0x7f0110108200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:24.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.122+0000 7f0115cef700 1 -- 172.21.15.47:0/2942102728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f010c005040 con 0x7f01100fcbb0 2024-02-23T00:56:24.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.122+0000 7f01164f0700 1 --2- 172.21.15.47:0/2942102728 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f01100f7400 0x7f011010e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:56:24.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.122+0000 7f0115cef700 1 --2- 172.21.15.47:0/2942102728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01100fcbb0 0x7f011010e550 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 rx=0x7f010400e8f0 tx=0x7f010400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:24.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.122+0000 7f01037fe700 10 client.?.objecter ms_handle_connect 0x7f01100fcbb0 2024-02-23T00:56:24.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.122+0000 7f01037fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:56:24.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.122+0000 7f01037fe700 1 -- 172.21.15.47:0/2942102728 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0104004620 con 0x7f01100fcbb0 2024-02-23T00:56:24.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.122+0000 7f0118754700 1 -- 172.21.15.47:0/2942102728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0110108d00 con 0x7f01100fcbb0 2024-02-23T00:56:24.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.122+0000 7f01037fe700 1 -- 172.21.15.47:0/2942102728 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f010400f040 con 0x7f01100fcbb0 2024-02-23T00:56:24.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.122+0000 7f01037fe700 1 -- 172.21.15.47:0/2942102728 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01040146f0 con 0x7f01100fcbb0 2024-02-23T00:56:24.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.122+0000 7f0118754700 10 client.31905.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:56:24.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.122+0000 7f0118754700 1 -- 172.21.15.47:0/2942102728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0110109080 con 0x7f01100fcbb0 2024-02-23T00:56:24.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.122+0000 7f0118754700 1 librados: init done 2024-02-23T00:56:24.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.123+0000 7f0118754700 1 -- 172.21.15.47:0/2942102728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f011005fa60 con 0x7f01100fcbb0 2024-02-23T00:56:24.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.123+0000 7f01037fe700 1 -- 172.21.15.47:0/2942102728 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f0104007b40 con 0x7f01100fcbb0 2024-02-23T00:56:24.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.124+0000 7f01037fe700 1 --2- 172.21.15.47:0/2942102728 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f00fc062cc0 0x7f00fc065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:24.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.124+0000 7f01037fe700 1 -- 172.21.15.47:0/2942102728 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(818..818 src has 621..818) v4 ==== 8638+0+0 (secure 0 0 0) 0x7f010407cdb0 con 0x7f01100fcbb0 2024-02-23T00:56:24.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.124+0000 7f01037fe700 10 client.31905.objecter ms_dispatch 0x7f0110001550 osd_map(818..818 src has 621..818) v4 2024-02-23T00:56:24.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.124+0000 7f01037fe700 3 client.31905.objecter handle_osd_map got epochs [818,818] > 0 2024-02-23T00:56:24.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.124+0000 7f01037fe700 3 client.31905.objecter handle_osd_map decoding full epoch 818 2024-02-23T00:56:24.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.124+0000 7f01164f0700 1 --2- 172.21.15.47:0/2942102728 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f00fc062cc0 0x7f00fc065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:24.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.124+0000 7f01037fe700 20 client.31905.objecter dump_active .. 0 homeless 2024-02-23T00:56:24.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.124+0000 7f01037fe700 10 client.31905.objecter ms_handle_connect 0x7f00fc062cc0 2024-02-23T00:56:24.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.124+0000 7f01164f0700 1 --2- 172.21.15.47:0/2942102728 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f00fc062cc0 0x7f00fc065180 secure :-1 s=READY pgs=3222 cs=0 l=1 rev1=1 rx=0x7f010c007fd0 tx=0x7f010c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:24.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.129+0000 7f01037fe700 1 -- 172.21.15.47:0/2942102728 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f010404c730 con 0x7f01100fcbb0 2024-02-23T00:56:24.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.316+0000 7f0118754700 1 -- 172.21.15.47:0/2942102728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7f0110105ea0 con 0x7f01100fcbb0 2024-02-23T00:56:24.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.371+0000 7f01037fe700 1 -- 172.21.15.47:0/2942102728 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v819) v1 ==== 101+0+0 (secure 0 0 0) 0x7f0104019070 con 0x7f01100fcbb0 2024-02-23T00:56:24.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.373+0000 7f0118754700 10 librados: watch_flush enter 2024-02-23T00:56:24.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.373+0000 7f0118754700 10 librados: watch_flush exit 2024-02-23T00:56:24.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.373+0000 7f0118754700 20 client.31905.objecter shutdown clearing up homeless session... 2024-02-23T00:56:24.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.373+0000 7f0118754700 10 client.31905.objecter successfully canceled tick 2024-02-23T00:56:24.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.373+0000 7f0118754700 1 -- 172.21.15.47:0/2942102728 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f00fc062cc0 msgr2=0x7f00fc065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:24.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.373+0000 7f0118754700 1 --2- 172.21.15.47:0/2942102728 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f00fc062cc0 0x7f00fc065180 secure :-1 s=READY pgs=3222 cs=0 l=1 rev1=1 rx=0x7f010c007fd0 tx=0x7f010c00b040).stop 2024-02-23T00:56:24.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.373+0000 7f0118754700 1 -- 172.21.15.47:0/2942102728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01100fcbb0 msgr2=0x7f011010e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:24.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.373+0000 7f0118754700 1 --2- 172.21.15.47:0/2942102728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01100fcbb0 0x7f011010e550 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 rx=0x7f010400e8f0 tx=0x7f010400b040).stop 2024-02-23T00:56:24.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.373+0000 7f0118754700 1 -- 172.21.15.47:0/2942102728 shutdown_connections 2024-02-23T00:56:24.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.373+0000 7f0118754700 1 --2- 172.21.15.47:0/2942102728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01100fd510 0x7f0110108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:24.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.373+0000 7f0118754700 1 --2- 172.21.15.47:0/2942102728 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f00fc062cc0 0x7f00fc065180 unknown :-1 s=CLOSED pgs=3222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:24.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.373+0000 7f0118754700 1 --2- 172.21.15.47:0/2942102728 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f01100f7400 0x7f011010e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:24.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.373+0000 7f0118754700 1 --2- 172.21.15.47:0/2942102728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01100fcbb0 0x7f011010e550 unknown :-1 s=CLOSED pgs=2657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:24.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.373+0000 7f0118754700 1 -- 172.21.15.47:0/2942102728 >> 172.21.15.47:0/2942102728 conn(0x7f01100f2820 msgr2=0x7f0110060e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:56:24.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.373+0000 7f0118754700 1 -- 172.21.15.47:0/2942102728 shutdown_connections 2024-02-23T00:56:24.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.373+0000 7f0118754700 1 -- 172.21.15.47:0/2942102728 wait complete. 2024-02-23T00:56:24.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:24.373+0000 7f0118754700 1 librados: shutdown 2024-02-23T00:56:24.378 INFO:teuthology.orchestra.run.smithi047.stderr:nodeep-scrub is unset 2024-02-23T00:56:24.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:23.017234+0000 mgr.y (mgr.14517) 693 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:56:24.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:23.414381+0000 mgr.y (mgr.14517) 694 : cluster [DBG] pgmap v1249: 154 pgs: 1 active+remapped+backfill_toofull, 69 peering, 84 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1/588 objects misplaced (0.170%) 2024-02-23T00:56:24.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:23.616317+0000 osd.1 (osd.1) 314 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:56:24.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:23.616483+0000 osd.1 (osd.1) 315 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:56:24.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:24.317973+0000 mon.a (mon.0) 3850 : audit [INF] from='client.? 172.21.15.47:0/2942102728' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:56:24.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:23.017234+0000 mgr.y (mgr.14517) 693 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:56:24.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:23.414381+0000 mgr.y (mgr.14517) 694 : cluster [DBG] pgmap v1249: 154 pgs: 1 active+remapped+backfill_toofull, 69 peering, 84 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1/588 objects misplaced (0.170%) 2024-02-23T00:56:24.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:23.616317+0000 osd.1 (osd.1) 314 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:56:24.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:23.616483+0000 osd.1 (osd.1) 315 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:56:24.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:24.317973+0000 mon.a (mon.0) 3850 : audit [INF] from='client.? 172.21.15.47:0/2942102728' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:56:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:23.017234+0000 mgr.y (mgr.14517) 693 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:56:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:23.414381+0000 mgr.y (mgr.14517) 694 : cluster [DBG] pgmap v1249: 154 pgs: 1 active+remapped+backfill_toofull, 69 peering, 84 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1/588 objects misplaced (0.170%) 2024-02-23T00:56:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:23.616317+0000 osd.1 (osd.1) 314 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:56:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:23.616483+0000 osd.1 (osd.1) 315 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:56:24.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:24.317973+0000 mon.a (mon.0) 3850 : audit [INF] from='client.? 172.21.15.47:0/2942102728' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-23T00:56:25.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:24.181596+0000 osd.7 (osd.7) 357 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:56:25.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:24.181793+0000 osd.7 (osd.7) 358 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:56:25.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:24.365072+0000 mon.a (mon.0) 3851 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:56:25.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:24.372041+0000 mon.a (mon.0) 3852 : audit [INF] from='client.? 172.21.15.47:0/2942102728' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:56:25.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:24.372100+0000 mon.a (mon.0) 3853 : cluster [DBG] osdmap e819: 8 total, 8 up, 8 in 2024-02-23T00:56:25.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:25.330345+0000 mon.a (mon.0) 3854 : cluster [DBG] osdmap e820: 8 total, 8 up, 8 in 2024-02-23T00:56:25.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:24.181596+0000 osd.7 (osd.7) 357 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:56:25.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:24.181793+0000 osd.7 (osd.7) 358 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:56:25.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:24.365072+0000 mon.a (mon.0) 3851 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:56:25.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:24.372041+0000 mon.a (mon.0) 3852 : audit [INF] from='client.? 172.21.15.47:0/2942102728' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:56:25.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:24.372100+0000 mon.a (mon.0) 3853 : cluster [DBG] osdmap e819: 8 total, 8 up, 8 in 2024-02-23T00:56:25.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:25.330345+0000 mon.a (mon.0) 3854 : cluster [DBG] osdmap e820: 8 total, 8 up, 8 in 2024-02-23T00:56:25.645 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:56:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:24.181596+0000 osd.7 (osd.7) 357 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:56:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:24.181793+0000 osd.7 (osd.7) 358 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:56:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:24.365072+0000 mon.a (mon.0) 3851 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-23T00:56:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:24.372041+0000 mon.a (mon.0) 3852 : audit [INF] from='client.? 172.21.15.47:0/2942102728' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-23T00:56:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:24.372100+0000 mon.a (mon.0) 3853 : cluster [DBG] osdmap e819: 8 total, 8 up, 8 in 2024-02-23T00:56:25.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:25.330345+0000 mon.a (mon.0) 3854 : cluster [DBG] osdmap e820: 8 total, 8 up, 8 in 2024-02-23T00:56:26.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:24.919373+0000 osd.6 (osd.6) 323 : cluster [DBG] 4.10 scrub starts 2024-02-23T00:56:26.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:24.921145+0000 osd.6 (osd.6) 324 : cluster [DBG] 4.10 scrub ok 2024-02-23T00:56:26.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:25.108334+0000 osd.7 (osd.7) 359 : cluster [DBG] 18.6 scrub starts 2024-02-23T00:56:26.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:25.109169+0000 osd.7 (osd.7) 360 : cluster [DBG] 18.6 scrub ok 2024-02-23T00:56:26.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:25.207259+0000 osd.2 (osd.2) 175 : cluster [DBG] 4.19 scrub starts 2024-02-23T00:56:26.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:25.208843+0000 osd.2 (osd.2) 176 : cluster [DBG] 4.19 scrub ok 2024-02-23T00:56:26.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:25.414713+0000 mgr.y (mgr.14517) 695 : cluster [DBG] pgmap v1252: 154 pgs: 1 active+remapped+backfill_toofull, 69 peering, 84 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1/588 objects misplaced (0.170%) 2024-02-23T00:56:26.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:26.312840+0000 mon.a (mon.0) 3855 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:56:26.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:24.919373+0000 osd.6 (osd.6) 323 : cluster [DBG] 4.10 scrub starts 2024-02-23T00:56:26.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:24.921145+0000 osd.6 (osd.6) 324 : cluster [DBG] 4.10 scrub ok 2024-02-23T00:56:26.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:25.108334+0000 osd.7 (osd.7) 359 : cluster [DBG] 18.6 scrub starts 2024-02-23T00:56:26.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:25.109169+0000 osd.7 (osd.7) 360 : cluster [DBG] 18.6 scrub ok 2024-02-23T00:56:26.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:25.207259+0000 osd.2 (osd.2) 175 : cluster [DBG] 4.19 scrub starts 2024-02-23T00:56:26.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:25.208843+0000 osd.2 (osd.2) 176 : cluster [DBG] 4.19 scrub ok 2024-02-23T00:56:26.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:25.414713+0000 mgr.y (mgr.14517) 695 : cluster [DBG] pgmap v1252: 154 pgs: 1 active+remapped+backfill_toofull, 69 peering, 84 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1/588 objects misplaced (0.170%) 2024-02-23T00:56:26.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:26.312840+0000 mon.a (mon.0) 3855 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:56:26.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.467+0000 7f8de443d700 1 -- 172.21.15.47:0/2427932553 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8ddc0f7400 msgr2=0x7f8ddc103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:26.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.467+0000 7f8de443d700 1 --2- 172.21.15.47:0/2427932553 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8ddc0f7400 0x7f8ddc103a20 secure :-1 s=READY pgs=2834 cs=0 l=1 rev1=1 rx=0x7f8dd4009c30 tx=0x7f8dd4004c50).stop 2024-02-23T00:56:26.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.468+0000 7f8de443d700 1 -- 172.21.15.47:0/2427932553 shutdown_connections 2024-02-23T00:56:26.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.468+0000 7f8de443d700 1 --2- 172.21.15.47:0/2427932553 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8ddc0f7400 0x7f8ddc103a20 unknown :-1 s=CLOSED pgs=2834 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:26.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.468+0000 7f8de443d700 1 --2- 172.21.15.47:0/2427932553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ddc0fcbb0 0x7f8ddc0fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:26.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.468+0000 7f8de443d700 1 --2- 172.21.15.47:0/2427932553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ddc0fd510 0x7f8ddc0f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:26.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.468+0000 7f8de443d700 1 -- 172.21.15.47:0/2427932553 >> 172.21.15.47:0/2427932553 conn(0x7f8ddc0f2820 msgr2=0x7f8ddc0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:56:26.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.468+0000 7f8de443d700 1 -- 172.21.15.47:0/2427932553 shutdown_connections 2024-02-23T00:56:26.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.468+0000 7f8de443d700 1 -- 172.21.15.47:0/2427932553 wait complete. 2024-02-23T00:56:26.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.468+0000 7f8de443d700 1 librados: starting msgr at 2024-02-23T00:56:26.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.468+0000 7f8de443d700 1 librados: starting objecter 2024-02-23T00:56:26.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.468+0000 7f8de443d700 1 Processor -- start 2024-02-23T00:56:26.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.468+0000 7f8de443d700 1 -- start start 2024-02-23T00:56:26.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.468+0000 7f8de443d700 1 librados: setting wanted keys 2024-02-23T00:56:26.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.468+0000 7f8de443d700 1 librados: calling monclient init 2024-02-23T00:56:26.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.469+0000 7f8de443d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ddc0f7400 0x7f8ddc108eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:26.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.469+0000 7f8de443d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ddc0fcbb0 0x7f8ddc1093a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:26.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.469+0000 7f8de443d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8ddc0fd510 0x7f8ddc109980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:26.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.469+0000 7f8de443d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc114090 con 0x7f8ddc0f7400 2024-02-23T00:56:26.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.469+0000 7f8de443d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc114200 con 0x7f8ddc0fcbb0 2024-02-23T00:56:26.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.469+0000 7f8de443d700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc1143c0 con 0x7f8ddc0fd510 2024-02-23T00:56:26.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.469+0000 7f8de21d9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ddc0f7400 0x7f8ddc108eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:26.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.469+0000 7f8de29da700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8ddc0fd510 0x7f8ddc109980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:26.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.469+0000 7f8de21d9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ddc0f7400 0x7f8ddc108eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54610/0 (socket says 172.21.15.47:54610) 2024-02-23T00:56:26.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.469+0000 7f8de29da700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8ddc0fd510 0x7f8ddc109980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:59302/0 (socket says 172.21.15.47:59302) 2024-02-23T00:56:26.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.469+0000 7f8de21d9700 1 -- 172.21.15.47:0/3577097974 learned_addr learned my addr 172.21.15.47:0/3577097974 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:56:26.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.469+0000 7f8de19d8700 1 --2- 172.21.15.47:0/3577097974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ddc0fcbb0 0x7f8ddc1093a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:26.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.469+0000 7f8de21d9700 1 -- 172.21.15.47:0/3577097974 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8ddc0fd510 msgr2=0x7f8ddc109980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:26.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.469+0000 7f8de21d9700 1 --2- 172.21.15.47:0/3577097974 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8ddc0fd510 0x7f8ddc109980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:26.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.469+0000 7f8de21d9700 1 -- 172.21.15.47:0/3577097974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ddc0fcbb0 msgr2=0x7f8ddc1093a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:26.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.469+0000 7f8de21d9700 1 --2- 172.21.15.47:0/3577097974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ddc0fcbb0 0x7f8ddc1093a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:26.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.469+0000 7f8de21d9700 1 -- 172.21.15.47:0/3577097974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dcc005040 con 0x7f8ddc0f7400 2024-02-23T00:56:26.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.469+0000 7f8de29da700 1 --2- 172.21.15.47:0/3577097974 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8ddc0fd510 0x7f8ddc109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:56:26.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.469+0000 7f8de21d9700 1 --2- 172.21.15.47:0/3577097974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ddc0f7400 0x7f8ddc108eb0 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 rx=0x7f8dd80027f0 tx=0x7f8dd8002b00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:26.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.469+0000 7f8dd37fe700 10 client.?.objecter ms_handle_connect 0x7f8ddc0f7400 2024-02-23T00:56:26.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.469+0000 7f8dd37fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:56:26.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.469+0000 7f8dd37fe700 1 -- 172.21.15.47:0/3577097974 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8007b40 con 0x7f8ddc0f7400 2024-02-23T00:56:26.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.469+0000 7f8de443d700 1 -- 172.21.15.47:0/3577097974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dd40097c0 con 0x7f8ddc0f7400 2024-02-23T00:56:26.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.469+0000 7f8dd37fe700 1 -- 172.21.15.47:0/3577097974 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f8dd800f040 con 0x7f8ddc0f7400 2024-02-23T00:56:26.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.469+0000 7f8dd37fe700 1 -- 172.21.15.47:0/3577097974 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8014610 con 0x7f8ddc0f7400 2024-02-23T00:56:26.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.469+0000 7f8de443d700 10 client.31911.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:56:26.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.470+0000 7f8de443d700 1 -- 172.21.15.47:0/3577097974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ddc114a50 con 0x7f8ddc0f7400 2024-02-23T00:56:26.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.470+0000 7f8de443d700 1 librados: init done 2024-02-23T00:56:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.470+0000 7f8de443d700 1 -- 172.21.15.47:0/3577097974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ddc05fa60 con 0x7f8ddc0f7400 2024-02-23T00:56:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.471+0000 7f8dd37fe700 1 -- 172.21.15.47:0/3577097974 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f8dd8009110 con 0x7f8ddc0f7400 2024-02-23T00:56:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.472+0000 7f8dd37fe700 1 --2- 172.21.15.47:0/3577097974 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8dc8062cc0 0x7f8dc8065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.472+0000 7f8de19d8700 1 --2- 172.21.15.47:0/3577097974 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8dc8062cc0 0x7f8dc8065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.472+0000 7f8dd37fe700 1 -- 172.21.15.47:0/3577097974 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(821..821 src has 621..821) v4 ==== 8638+0+0 (secure 0 0 0) 0x7f8dd807d560 con 0x7f8ddc0f7400 2024-02-23T00:56:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.472+0000 7f8dd37fe700 10 client.31911.objecter ms_dispatch 0x7f8ddc001550 osd_map(821..821 src has 621..821) v4 2024-02-23T00:56:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.472+0000 7f8dd37fe700 3 client.31911.objecter handle_osd_map got epochs [821,821] > 0 2024-02-23T00:56:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.472+0000 7f8dd37fe700 3 client.31911.objecter handle_osd_map decoding full epoch 821 2024-02-23T00:56:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.472+0000 7f8dd37fe700 20 client.31911.objecter dump_active .. 0 homeless 2024-02-23T00:56:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.472+0000 7f8dd37fe700 10 client.31911.objecter ms_handle_connect 0x7f8dc8062cc0 2024-02-23T00:56:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.472+0000 7f8de19d8700 1 --2- 172.21.15.47:0/3577097974 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8dc8062cc0 0x7f8dc8065180 secure :-1 s=READY pgs=3223 cs=0 l=1 rev1=1 rx=0x7f8ddc061700 tx=0x7f8dcc00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.476+0000 7f8dd37fe700 1 -- 172.21.15.47:0/3577097974 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f8dd804cce0 con 0x7f8ddc0f7400 2024-02-23T00:56:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:26.645+0000 7f8de443d700 1 -- 172.21.15.47:0/3577097974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num", "val": "12"} v 0) v1 -- 0x7f8ddc05ee80 con 0x7f8ddc0f7400 2024-02-23T00:56:26.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:24.919373+0000 osd.6 (osd.6) 323 : cluster [DBG] 4.10 scrub starts 2024-02-23T00:56:26.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:24.921145+0000 osd.6 (osd.6) 324 : cluster [DBG] 4.10 scrub ok 2024-02-23T00:56:26.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:25.108334+0000 osd.7 (osd.7) 359 : cluster [DBG] 18.6 scrub starts 2024-02-23T00:56:26.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:25.109169+0000 osd.7 (osd.7) 360 : cluster [DBG] 18.6 scrub ok 2024-02-23T00:56:26.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:25.207259+0000 osd.2 (osd.2) 175 : cluster [DBG] 4.19 scrub starts 2024-02-23T00:56:26.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:25.208843+0000 osd.2 (osd.2) 176 : cluster [DBG] 4.19 scrub ok 2024-02-23T00:56:26.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:25.414713+0000 mgr.y (mgr.14517) 695 : cluster [DBG] pgmap v1252: 154 pgs: 1 active+remapped+backfill_toofull, 69 peering, 84 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1/588 objects misplaced (0.170%) 2024-02-23T00:56:26.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:26.312840+0000 mon.a (mon.0) 3855 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:56:27.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:27.377+0000 7f8dd37fe700 1 -- 172.21.15.47:0/3577097974 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num", "val": "12"}]=0 set pool 6 pgp_num to 12 v822) v1 ==== 143+0+0 (secure 0 0 0) 0x7f8dd8050300 con 0x7f8ddc0f7400 2024-02-23T00:56:27.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:27.380+0000 7f8de443d700 10 librados: watch_flush enter 2024-02-23T00:56:27.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:27.380+0000 7f8de443d700 10 librados: watch_flush exit 2024-02-23T00:56:27.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:27.380+0000 7f8de443d700 20 client.31911.objecter shutdown clearing up homeless session... 2024-02-23T00:56:27.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:27.380+0000 7f8de443d700 10 client.31911.objecter successfully canceled tick 2024-02-23T00:56:27.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:27.380+0000 7f8de443d700 1 -- 172.21.15.47:0/3577097974 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8dc8062cc0 msgr2=0x7f8dc8065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:27.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:27.380+0000 7f8de443d700 1 --2- 172.21.15.47:0/3577097974 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8dc8062cc0 0x7f8dc8065180 secure :-1 s=READY pgs=3223 cs=0 l=1 rev1=1 rx=0x7f8ddc061700 tx=0x7f8dcc00b040).stop 2024-02-23T00:56:27.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:27.380+0000 7f8de443d700 1 -- 172.21.15.47:0/3577097974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ddc0f7400 msgr2=0x7f8ddc108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:27.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:27.380+0000 7f8de443d700 1 --2- 172.21.15.47:0/3577097974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ddc0f7400 0x7f8ddc108eb0 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 rx=0x7f8dd80027f0 tx=0x7f8dd8002b00).stop 2024-02-23T00:56:27.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:27.380+0000 7f8de443d700 1 -- 172.21.15.47:0/3577097974 shutdown_connections 2024-02-23T00:56:27.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:27.380+0000 7f8de443d700 1 --2- 172.21.15.47:0/3577097974 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8dc8062cc0 0x7f8dc8065180 unknown :-1 s=CLOSED pgs=3223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:27.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:27.380+0000 7f8de443d700 1 --2- 172.21.15.47:0/3577097974 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8ddc0fd510 0x7f8ddc109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:27.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:27.380+0000 7f8de443d700 1 --2- 172.21.15.47:0/3577097974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ddc0f7400 0x7f8ddc108eb0 unknown :-1 s=CLOSED pgs=2658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:27.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:27.380+0000 7f8de443d700 1 --2- 172.21.15.47:0/3577097974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ddc0fcbb0 0x7f8ddc1093a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:27.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:27.380+0000 7f8de443d700 1 -- 172.21.15.47:0/3577097974 >> 172.21.15.47:0/3577097974 conn(0x7f8ddc0f2820 msgr2=0x7f8ddc0f41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:56:27.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:27.380+0000 7f8de443d700 1 -- 172.21.15.47:0/3577097974 shutdown_connections 2024-02-23T00:56:27.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:27.380+0000 7f8de443d700 1 -- 172.21.15.47:0/3577097974 wait complete. 2024-02-23T00:56:27.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:27.381+0000 7f8de443d700 1 librados: shutdown 2024-02-23T00:56:27.387 INFO:teuthology.orchestra.run.smithi047.stderr:set pool 6 pgp_num to 12 2024-02-23T00:56:27.523 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:26.041784+0000 osd.0 (osd.0) 306 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:56:27.523 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:26.042095+0000 osd.0 (osd.0) 307 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:56:27.523 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:26.114844+0000 osd.7 (osd.7) 361 : cluster [DBG] 18.1f scrub starts 2024-02-23T00:56:27.524 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:26.117987+0000 osd.7 (osd.7) 362 : cluster [DBG] 18.1f scrub ok 2024-02-23T00:56:27.524 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:26.377581+0000 mon.a (mon.0) 3856 : cluster [DBG] osdmap e821: 8 total, 8 up, 8 in 2024-02-23T00:56:27.524 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:26.647112+0000 mon.a (mon.0) 3857 : audit [INF] from='client.? 172.21.15.47:0/3577097974' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num", "val": "12"}]: dispatch 2024-02-23T00:56:27.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:26.041784+0000 osd.0 (osd.0) 306 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:56:27.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:26.042095+0000 osd.0 (osd.0) 307 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:56:27.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:26.114844+0000 osd.7 (osd.7) 361 : cluster [DBG] 18.1f scrub starts 2024-02-23T00:56:27.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:26.117987+0000 osd.7 (osd.7) 362 : cluster [DBG] 18.1f scrub ok 2024-02-23T00:56:27.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:26.377581+0000 mon.a (mon.0) 3856 : cluster [DBG] osdmap e821: 8 total, 8 up, 8 in 2024-02-23T00:56:27.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:26.647112+0000 mon.a (mon.0) 3857 : audit [INF] from='client.? 172.21.15.47:0/3577097974' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num", "val": "12"}]: dispatch 2024-02-23T00:56:27.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:26.041784+0000 osd.0 (osd.0) 306 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:56:27.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:26.042095+0000 osd.0 (osd.0) 307 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:56:27.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:26.114844+0000 osd.7 (osd.7) 361 : cluster [DBG] 18.1f scrub starts 2024-02-23T00:56:27.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:26.117987+0000 osd.7 (osd.7) 362 : cluster [DBG] 18.1f scrub ok 2024-02-23T00:56:27.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:26.377581+0000 mon.a (mon.0) 3856 : cluster [DBG] osdmap e821: 8 total, 8 up, 8 in 2024-02-23T00:56:27.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:26.647112+0000 mon.a (mon.0) 3857 : audit [INF] from='client.? 172.21.15.47:0/3577097974' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num", "val": "12"}]: dispatch 2024-02-23T00:56:27.887 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:56:27.887 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:56:28.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:27.173768+0000 osd.2 (osd.2) 177 : cluster [DBG] 2.c scrub starts 2024-02-23T00:56:28.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:27.174804+0000 osd.2 (osd.2) 178 : cluster [DBG] 2.c scrub ok 2024-02-23T00:56:28.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:27.378400+0000 mon.a (mon.0) 3858 : audit [INF] from='client.? 172.21.15.47:0/3577097974' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num", "val": "12"}]': finished 2024-02-23T00:56:28.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:27.378477+0000 mon.a (mon.0) 3859 : cluster [DBG] osdmap e822: 8 total, 8 up, 8 in 2024-02-23T00:56:28.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:27.415169+0000 mgr.y (mgr.14517) 696 : cluster [DBG] pgmap v1255: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 43 KiB/s rd, 767 B/s wr, 80 op/s; 1/675 objects misplaced (0.148%); 590 B/s, 34 objects/s recovering 2024-02-23T00:56:28.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:27.896062+0000 osd.6 (osd.6) 325 : cluster [DBG] 18.5 scrub starts 2024-02-23T00:56:28.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:27.898105+0000 osd.6 (osd.6) 326 : cluster [DBG] 18.5 scrub ok 2024-02-23T00:56:28.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:27.173768+0000 osd.2 (osd.2) 177 : cluster [DBG] 2.c scrub starts 2024-02-23T00:56:28.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:27.174804+0000 osd.2 (osd.2) 178 : cluster [DBG] 2.c scrub ok 2024-02-23T00:56:28.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:27.378400+0000 mon.a (mon.0) 3858 : audit [INF] from='client.? 172.21.15.47:0/3577097974' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num", "val": "12"}]': finished 2024-02-23T00:56:28.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:27.378477+0000 mon.a (mon.0) 3859 : cluster [DBG] osdmap e822: 8 total, 8 up, 8 in 2024-02-23T00:56:28.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:27.415169+0000 mgr.y (mgr.14517) 696 : cluster [DBG] pgmap v1255: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 43 KiB/s rd, 767 B/s wr, 80 op/s; 1/675 objects misplaced (0.148%); 590 B/s, 34 objects/s recovering 2024-02-23T00:56:28.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:27.896062+0000 osd.6 (osd.6) 325 : cluster [DBG] 18.5 scrub starts 2024-02-23T00:56:28.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:27.898105+0000 osd.6 (osd.6) 326 : cluster [DBG] 18.5 scrub ok 2024-02-23T00:56:28.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:27.173768+0000 osd.2 (osd.2) 177 : cluster [DBG] 2.c scrub starts 2024-02-23T00:56:28.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:27.174804+0000 osd.2 (osd.2) 178 : cluster [DBG] 2.c scrub ok 2024-02-23T00:56:28.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:27.378400+0000 mon.a (mon.0) 3858 : audit [INF] from='client.? 172.21.15.47:0/3577097974' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num", "val": "12"}]': finished 2024-02-23T00:56:28.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:27.378477+0000 mon.a (mon.0) 3859 : cluster [DBG] osdmap e822: 8 total, 8 up, 8 in 2024-02-23T00:56:28.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:27.415169+0000 mgr.y (mgr.14517) 696 : cluster [DBG] pgmap v1255: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 43 KiB/s rd, 767 B/s wr, 80 op/s; 1/675 objects misplaced (0.148%); 590 B/s, 34 objects/s recovering 2024-02-23T00:56:28.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:27.896062+0000 osd.6 (osd.6) 325 : cluster [DBG] 18.5 scrub starts 2024-02-23T00:56:28.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:27.898105+0000 osd.6 (osd.6) 326 : cluster [DBG] 18.5 scrub ok 2024-02-23T00:56:28.731 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.1 dump_historic_ops 2024-02-23T00:56:29.029 INFO:tasks.thrashosds.thrasher:fixing pg num pool default.rgw.control 2024-02-23T00:56:29.029 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd pool set default.rgw.control pgp_num 42 2024-02-23T00:56:29.080 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:56:29.604 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:28.102471+0000 osd.0 (osd.0) 308 : cluster [DBG] 6.9 scrub starts 2024-02-23T00:56:29.604 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:28.104077+0000 osd.0 (osd.0) 309 : cluster [DBG] 6.9 scrub ok 2024-02-23T00:56:29.604 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:28.306656+0000 osd.2 (osd.2) 179 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:56:29.604 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:28.307041+0000 osd.2 (osd.2) 180 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:56:29.604 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:28.388340+0000 mon.a (mon.0) 3860 : cluster [DBG] osdmap e823: 8 total, 8 up, 8 in 2024-02-23T00:56:29.604 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:28.917558+0000 osd.6 (osd.6) 327 : cluster [DBG] 4.12 deep-scrub starts 2024-02-23T00:56:29.604 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:28.921006+0000 osd.6 (osd.6) 328 : cluster [DBG] 4.12 deep-scrub ok 2024-02-23T00:56:29.604 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:29.202405+0000 osd.5 (osd.5) 360 : cluster [DBG] 18.15 scrub starts 2024-02-23T00:56:29.604 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:29.205531+0000 osd.5 (osd.5) 361 : cluster [DBG] 18.15 scrub ok 2024-02-23T00:56:29.604 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:28.102471+0000 osd.0 (osd.0) 308 : cluster [DBG] 6.9 scrub starts 2024-02-23T00:56:29.605 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:28.104077+0000 osd.0 (osd.0) 309 : cluster [DBG] 6.9 scrub ok 2024-02-23T00:56:29.605 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:28.306656+0000 osd.2 (osd.2) 179 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:56:29.605 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:28.307041+0000 osd.2 (osd.2) 180 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:56:29.605 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:28.388340+0000 mon.a (mon.0) 3860 : cluster [DBG] osdmap e823: 8 total, 8 up, 8 in 2024-02-23T00:56:29.605 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:28.917558+0000 osd.6 (osd.6) 327 : cluster [DBG] 4.12 deep-scrub starts 2024-02-23T00:56:29.605 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:28.921006+0000 osd.6 (osd.6) 328 : cluster [DBG] 4.12 deep-scrub ok 2024-02-23T00:56:29.605 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:29.202405+0000 osd.5 (osd.5) 360 : cluster [DBG] 18.15 scrub starts 2024-02-23T00:56:29.605 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:29.205531+0000 osd.5 (osd.5) 361 : cluster [DBG] 18.15 scrub ok 2024-02-23T00:56:29.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:28.102471+0000 osd.0 (osd.0) 308 : cluster [DBG] 6.9 scrub starts 2024-02-23T00:56:29.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:28.104077+0000 osd.0 (osd.0) 309 : cluster [DBG] 6.9 scrub ok 2024-02-23T00:56:29.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:28.306656+0000 osd.2 (osd.2) 179 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:56:29.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:28.307041+0000 osd.2 (osd.2) 180 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:56:29.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:28.388340+0000 mon.a (mon.0) 3860 : cluster [DBG] osdmap e823: 8 total, 8 up, 8 in 2024-02-23T00:56:29.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:28.917558+0000 osd.6 (osd.6) 327 : cluster [DBG] 4.12 deep-scrub starts 2024-02-23T00:56:29.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:28.921006+0000 osd.6 (osd.6) 328 : cluster [DBG] 4.12 deep-scrub ok 2024-02-23T00:56:29.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:29.202405+0000 osd.5 (osd.5) 360 : cluster [DBG] 18.15 scrub starts 2024-02-23T00:56:29.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:29.205531+0000 osd.5 (osd.5) 361 : cluster [DBG] 18.15 scrub ok 2024-02-23T00:56:30.416 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:56:30.662 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:56:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:56:30] "GET /metrics HTTP/1.1" 200 187122 "" "Prometheus/2.33.4" 2024-02-23T00:56:30.662 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:29.393743+0000 osd.4 (osd.4) 343 : cluster [DBG] 6.6 scrub starts 2024-02-23T00:56:30.662 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:29.394740+0000 osd.4 (osd.4) 344 : cluster [DBG] 6.6 scrub ok 2024-02-23T00:56:30.662 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:29.415513+0000 mgr.y (mgr.14517) 697 : cluster [DBG] pgmap v1257: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 42 KiB/s rd, 751 B/s wr, 79 op/s; 1/675 objects misplaced (0.148%); 578 B/s, 33 objects/s recovering 2024-02-23T00:56:30.662 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:29.923778+0000 osd.6 (osd.6) 329 : cluster [DBG] 18.18 deep-scrub starts 2024-02-23T00:56:30.663 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:29.924698+0000 osd.6 (osd.6) 330 : cluster [DBG] 18.18 deep-scrub ok 2024-02-23T00:56:30.663 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:30.332482+0000 mon.a (mon.0) 3861 : cluster [DBG] osdmap e824: 8 total, 8 up, 8 in 2024-02-23T00:56:30.663 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:29.393743+0000 osd.4 (osd.4) 343 : cluster [DBG] 6.6 scrub starts 2024-02-23T00:56:30.663 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:29.394740+0000 osd.4 (osd.4) 344 : cluster [DBG] 6.6 scrub ok 2024-02-23T00:56:30.663 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:29.415513+0000 mgr.y (mgr.14517) 697 : cluster [DBG] pgmap v1257: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 42 KiB/s rd, 751 B/s wr, 79 op/s; 1/675 objects misplaced (0.148%); 578 B/s, 33 objects/s recovering 2024-02-23T00:56:30.663 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:29.923778+0000 osd.6 (osd.6) 329 : cluster [DBG] 18.18 deep-scrub starts 2024-02-23T00:56:30.663 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:29.924698+0000 osd.6 (osd.6) 330 : cluster [DBG] 18.18 deep-scrub ok 2024-02-23T00:56:30.663 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:30.332482+0000 mon.a (mon.0) 3861 : cluster [DBG] osdmap e824: 8 total, 8 up, 8 in 2024-02-23T00:56:30.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:29.393743+0000 osd.4 (osd.4) 343 : cluster [DBG] 6.6 scrub starts 2024-02-23T00:56:30.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:29.394740+0000 osd.4 (osd.4) 344 : cluster [DBG] 6.6 scrub ok 2024-02-23T00:56:30.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:29.415513+0000 mgr.y (mgr.14517) 697 : cluster [DBG] pgmap v1257: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 42 KiB/s rd, 751 B/s wr, 79 op/s; 1/675 objects misplaced (0.148%); 578 B/s, 33 objects/s recovering 2024-02-23T00:56:30.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:29.923778+0000 osd.6 (osd.6) 329 : cluster [DBG] 18.18 deep-scrub starts 2024-02-23T00:56:30.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:29.924698+0000 osd.6 (osd.6) 330 : cluster [DBG] 18.18 deep-scrub ok 2024-02-23T00:56:30.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:30.332482+0000 mon.a (mon.0) 3861 : cluster [DBG] osdmap e824: 8 total, 8 up, 8 in 2024-02-23T00:56:31.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:30.104242+0000 osd.7 (osd.7) 363 : cluster [DBG] 2.1c scrub starts 2024-02-23T00:56:31.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:30.107193+0000 osd.7 (osd.7) 364 : cluster [DBG] 2.1c scrub ok 2024-02-23T00:56:31.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:30.181723+0000 osd.2 (osd.2) 181 : cluster [DBG] 2.e scrub starts 2024-02-23T00:56:31.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:30.182625+0000 osd.2 (osd.2) 182 : cluster [DBG] 2.e scrub ok 2024-02-23T00:56:31.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:30.391117+0000 osd.4 (osd.4) 345 : cluster [DBG] 2.f deep-scrub starts 2024-02-23T00:56:31.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:30.393517+0000 osd.4 (osd.4) 346 : cluster [DBG] 2.f deep-scrub ok 2024-02-23T00:56:31.742 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:30.104242+0000 osd.7 (osd.7) 363 : cluster [DBG] 2.1c scrub starts 2024-02-23T00:56:31.742 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:30.107193+0000 osd.7 (osd.7) 364 : cluster [DBG] 2.1c scrub ok 2024-02-23T00:56:31.742 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:30.181723+0000 osd.2 (osd.2) 181 : cluster [DBG] 2.e scrub starts 2024-02-23T00:56:31.742 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:30.182625+0000 osd.2 (osd.2) 182 : cluster [DBG] 2.e scrub ok 2024-02-23T00:56:31.742 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:30.391117+0000 osd.4 (osd.4) 345 : cluster [DBG] 2.f deep-scrub starts 2024-02-23T00:56:31.742 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:30.393517+0000 osd.4 (osd.4) 346 : cluster [DBG] 2.f deep-scrub ok 2024-02-23T00:56:31.742 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:30.104242+0000 osd.7 (osd.7) 363 : cluster [DBG] 2.1c scrub starts 2024-02-23T00:56:31.742 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:30.107193+0000 osd.7 (osd.7) 364 : cluster [DBG] 2.1c scrub ok 2024-02-23T00:56:31.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:30.181723+0000 osd.2 (osd.2) 181 : cluster [DBG] 2.e scrub starts 2024-02-23T00:56:31.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:30.182625+0000 osd.2 (osd.2) 182 : cluster [DBG] 2.e scrub ok 2024-02-23T00:56:31.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:30.391117+0000 osd.4 (osd.4) 345 : cluster [DBG] 2.f deep-scrub starts 2024-02-23T00:56:31.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:30.393517+0000 osd.4 (osd.4) 346 : cluster [DBG] 2.f deep-scrub ok 2024-02-23T00:56:32.236 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:56:32.236 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:56:32.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:31.213297+0000 osd.2 (osd.2) 183 : cluster [DBG] 5.b deep-scrub starts 2024-02-23T00:56:32.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:31.214491+0000 osd.2 (osd.2) 184 : cluster [DBG] 5.b deep-scrub ok 2024-02-23T00:56:32.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:31.415963+0000 mgr.y (mgr.14517) 698 : cluster [DBG] pgmap v1259: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 34 KiB/s rd, 609 B/s wr, 64 op/s; 1/675 objects misplaced (0.148%); 468 B/s, 27 objects/s recovering 2024-02-23T00:56:32.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:31.213297+0000 osd.2 (osd.2) 183 : cluster [DBG] 5.b deep-scrub starts 2024-02-23T00:56:32.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:31.214491+0000 osd.2 (osd.2) 184 : cluster [DBG] 5.b deep-scrub ok 2024-02-23T00:56:32.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:31.415963+0000 mgr.y (mgr.14517) 698 : cluster [DBG] pgmap v1259: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 34 KiB/s rd, 609 B/s wr, 64 op/s; 1/675 objects misplaced (0.148%); 468 B/s, 27 objects/s recovering 2024-02-23T00:56:32.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.462+0000 7f19de1dc700 1 -- 172.21.15.47:0/868548347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19d80fcbb0 msgr2=0x7f19d80fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:32.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.462+0000 7f19de1dc700 1 --2- 172.21.15.47:0/868548347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19d80fcbb0 0x7f19d80fcf90 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 rx=0x7f19c8004870 tx=0x7f19c8009cb0).stop 2024-02-23T00:56:32.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.463+0000 7f19de1dc700 1 -- 172.21.15.47:0/868548347 shutdown_connections 2024-02-23T00:56:32.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.463+0000 7f19de1dc700 1 --2- 172.21.15.47:0/868548347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19d80f7400 0x7f19d8103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:32.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.463+0000 7f19de1dc700 1 --2- 172.21.15.47:0/868548347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19d80fcbb0 0x7f19d80fcf90 unknown :-1 s=CLOSED pgs=2659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:32.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.463+0000 7f19de1dc700 1 --2- 172.21.15.47:0/868548347 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f19d80fd510 0x7f19d80f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:32.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.463+0000 7f19de1dc700 1 -- 172.21.15.47:0/868548347 >> 172.21.15.47:0/868548347 conn(0x7f19d80f2820 msgr2=0x7f19d80f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:56:32.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.463+0000 7f19de1dc700 1 -- 172.21.15.47:0/868548347 shutdown_connections 2024-02-23T00:56:32.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.463+0000 7f19de1dc700 1 -- 172.21.15.47:0/868548347 wait complete. 2024-02-23T00:56:32.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.463+0000 7f19de1dc700 1 librados: starting msgr at 2024-02-23T00:56:32.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.463+0000 7f19de1dc700 1 librados: starting objecter 2024-02-23T00:56:32.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.463+0000 7f19de1dc700 1 Processor -- start 2024-02-23T00:56:32.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.463+0000 7f19de1dc700 1 -- start start 2024-02-23T00:56:32.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.463+0000 7f19de1dc700 1 librados: setting wanted keys 2024-02-23T00:56:32.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.463+0000 7f19de1dc700 1 librados: calling monclient init 2024-02-23T00:56:32.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.464+0000 7f19de1dc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19d80f7400 0x7f19d80f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:32.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.464+0000 7f19de1dc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f19d80fcbb0 0x7f19d80f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:32.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.464+0000 7f19de1dc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19d80fd510 0x7f19d80f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:32.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.464+0000 7f19de1dc700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d81164f0 con 0x7f19d80fd510 2024-02-23T00:56:32.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.464+0000 7f19de1dc700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d8116660 con 0x7f19d80f7400 2024-02-23T00:56:32.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.464+0000 7f19de1dc700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d8116820 con 0x7f19d80fcbb0 2024-02-23T00:56:32.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.464+0000 7f19d6ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f19d80fcbb0 0x7f19d80f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:32.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.464+0000 7f19d7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19d80fd510 0x7f19d80f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:32.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.464+0000 7f19d6ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f19d80fcbb0 0x7f19d80f8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:59320/0 (socket says 172.21.15.47:59320) 2024-02-23T00:56:32.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.464+0000 7f19d6ffd700 1 -- 172.21.15.47:0/3863048025 learned_addr learned my addr 172.21.15.47:0/3863048025 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:56:32.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.464+0000 7f19d77fe700 1 --2- 172.21.15.47:0/3863048025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19d80f7400 0x7f19d80f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:32.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.464+0000 7f19d6ffd700 1 -- 172.21.15.47:0/3863048025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19d80f7400 msgr2=0x7f19d80f89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:32.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.464+0000 7f19d6ffd700 1 --2- 172.21.15.47:0/3863048025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19d80f7400 0x7f19d80f89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:32.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.464+0000 7f19d6ffd700 1 -- 172.21.15.47:0/3863048025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19d80fd510 msgr2=0x7f19d80f9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:32.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.464+0000 7f19d6ffd700 1 --2- 172.21.15.47:0/3863048025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19d80fd510 0x7f19d80f9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:32.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.464+0000 7f19d6ffd700 1 -- 172.21.15.47:0/3863048025 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19cc005040 con 0x7f19d80fcbb0 2024-02-23T00:56:32.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.464+0000 7f19d7fff700 1 --2- 172.21.15.47:0/3863048025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19d80fd510 0x7f19d80f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:56:32.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.464+0000 7f19d77fe700 1 --2- 172.21.15.47:0/3863048025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19d80f7400 0x7f19d80f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:56:32.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.464+0000 7f19d6ffd700 1 --2- 172.21.15.47:0/3863048025 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f19d80fcbb0 0x7f19d80f8ea0 secure :-1 s=READY pgs=2835 cs=0 l=1 rev1=1 rx=0x7f19cc0048f0 tx=0x7f19cc009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:32.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.464+0000 7f19d4ff9700 10 client.?.objecter ms_handle_connect 0x7f19d80fcbb0 2024-02-23T00:56:32.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.464+0000 7f19d4ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:56:32.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.464+0000 7f19d4ff9700 1 -- 172.21.15.47:0/3863048025 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc007b00 con 0x7f19d80fcbb0 2024-02-23T00:56:32.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.464+0000 7f19de1dc700 1 -- 172.21.15.47:0/3863048025 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c8005040 con 0x7f19d80fcbb0 2024-02-23T00:56:32.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.464+0000 7f19d4ff9700 1 -- 172.21.15.47:0/3863048025 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f19cc00f040 con 0x7f19d80fcbb0 2024-02-23T00:56:32.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.464+0000 7f19d4ff9700 1 -- 172.21.15.47:0/3863048025 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc014890 con 0x7f19d80fcbb0 2024-02-23T00:56:32.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.464+0000 7f19de1dc700 10 client.42239.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:56:32.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.464+0000 7f19de1dc700 1 -- 172.21.15.47:0/3863048025 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d8116e60 con 0x7f19d80fcbb0 2024-02-23T00:56:32.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.464+0000 7f19de1dc700 1 librados: init done 2024-02-23T00:56:32.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.465+0000 7f19de1dc700 1 -- 172.21.15.47:0/3863048025 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d805fa60 con 0x7f19d80fcbb0 2024-02-23T00:56:32.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.466+0000 7f19d4ff9700 1 -- 172.21.15.47:0/3863048025 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f19cc014cc0 con 0x7f19d80fcbb0 2024-02-23T00:56:32.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.466+0000 7f19d4ff9700 1 --2- 172.21.15.47:0/3863048025 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f19c0062cc0 0x7f19c0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:32.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.466+0000 7f19d4ff9700 1 -- 172.21.15.47:0/3863048025 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(824..824 src has 621..824) v4 ==== 8638+0+0 (secure 0 0 0) 0x7f19cc07cff0 con 0x7f19d80fcbb0 2024-02-23T00:56:32.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.466+0000 7f19d4ff9700 10 client.42239.objecter ms_dispatch 0x7f19d8001550 osd_map(824..824 src has 621..824) v4 2024-02-23T00:56:32.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.466+0000 7f19d4ff9700 3 client.42239.objecter handle_osd_map got epochs [824,824] > 0 2024-02-23T00:56:32.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.466+0000 7f19d4ff9700 3 client.42239.objecter handle_osd_map decoding full epoch 824 2024-02-23T00:56:32.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.466+0000 7f19d77fe700 1 --2- 172.21.15.47:0/3863048025 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f19c0062cc0 0x7f19c0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:32.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.466+0000 7f19d4ff9700 20 client.42239.objecter dump_active .. 0 homeless 2024-02-23T00:56:32.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.467+0000 7f19d4ff9700 10 client.42239.objecter ms_handle_connect 0x7f19c0062cc0 2024-02-23T00:56:32.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.467+0000 7f19d77fe700 1 --2- 172.21.15.47:0/3863048025 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f19c0062cc0 0x7f19c0065180 secure :-1 s=READY pgs=3224 cs=0 l=1 rev1=1 rx=0x7f19c8009da0 tx=0x7f19c800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:32.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.470+0000 7f19d4ff9700 1 -- 172.21.15.47:0/3863048025 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f19cc049f60 con 0x7f19d80fcbb0 2024-02-23T00:56:32.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:32.645+0000 7f19de1dc700 1 -- 172.21.15.47:0/3863048025 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num", "val": "42"} v 0) v1 -- 0x7f19d80f9e90 con 0x7f19d80fcbb0 2024-02-23T00:56:32.696 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:56:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:56:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:56:32.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:31.213297+0000 osd.2 (osd.2) 183 : cluster [DBG] 5.b deep-scrub starts 2024-02-23T00:56:32.696 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:31.214491+0000 osd.2 (osd.2) 184 : cluster [DBG] 5.b deep-scrub ok 2024-02-23T00:56:32.697 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:31.415963+0000 mgr.y (mgr.14517) 698 : cluster [DBG] pgmap v1259: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 34 KiB/s rd, 609 B/s wr, 64 op/s; 1/675 objects misplaced (0.148%); 468 B/s, 27 objects/s recovering 2024-02-23T00:56:33.061 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.2 dump_ops_in_flight 2024-02-23T00:56:33.340 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:56:33.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:33.409+0000 7f19d4ff9700 1 -- 172.21.15.47:0/3863048025 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num", "val": "42"}]=0 set pool 5 pgp_num to 42 v825) v1 ==== 146+0+0 (secure 0 0 0) 0x7f19cc050280 con 0x7f19d80fcbb0 2024-02-23T00:56:33.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:33.412+0000 7f19de1dc700 10 librados: watch_flush enter 2024-02-23T00:56:33.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:33.412+0000 7f19de1dc700 10 librados: watch_flush exit 2024-02-23T00:56:33.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:33.412+0000 7f19de1dc700 20 client.42239.objecter shutdown clearing up homeless session... 2024-02-23T00:56:33.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:33.412+0000 7f19de1dc700 10 client.42239.objecter successfully canceled tick 2024-02-23T00:56:33.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:33.412+0000 7f19de1dc700 1 -- 172.21.15.47:0/3863048025 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f19c0062cc0 msgr2=0x7f19c0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:33.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:33.412+0000 7f19de1dc700 1 --2- 172.21.15.47:0/3863048025 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f19c0062cc0 0x7f19c0065180 secure :-1 s=READY pgs=3224 cs=0 l=1 rev1=1 rx=0x7f19c8009da0 tx=0x7f19c800b040).stop 2024-02-23T00:56:33.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:33.412+0000 7f19de1dc700 1 -- 172.21.15.47:0/3863048025 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f19d80fcbb0 msgr2=0x7f19d80f8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:33.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:33.412+0000 7f19de1dc700 1 --2- 172.21.15.47:0/3863048025 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f19d80fcbb0 0x7f19d80f8ea0 secure :-1 s=READY pgs=2835 cs=0 l=1 rev1=1 rx=0x7f19cc0048f0 tx=0x7f19cc009cb0).stop 2024-02-23T00:56:33.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:33.412+0000 7f19de1dc700 1 -- 172.21.15.47:0/3863048025 shutdown_connections 2024-02-23T00:56:33.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:33.412+0000 7f19de1dc700 1 --2- 172.21.15.47:0/3863048025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19d80fd510 0x7f19d80f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:33.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:33.412+0000 7f19de1dc700 1 --2- 172.21.15.47:0/3863048025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19d80f7400 0x7f19d80f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:33.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:33.412+0000 7f19de1dc700 1 --2- 172.21.15.47:0/3863048025 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f19c0062cc0 0x7f19c0065180 unknown :-1 s=CLOSED pgs=3224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:33.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:33.412+0000 7f19de1dc700 1 --2- 172.21.15.47:0/3863048025 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f19d80fcbb0 0x7f19d80f8ea0 unknown :-1 s=CLOSED pgs=2835 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:33.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:33.412+0000 7f19de1dc700 1 -- 172.21.15.47:0/3863048025 >> 172.21.15.47:0/3863048025 conn(0x7f19d80f2820 msgr2=0x7f19d8060db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:56:33.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:33.412+0000 7f19de1dc700 1 -- 172.21.15.47:0/3863048025 shutdown_connections 2024-02-23T00:56:33.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:33.412+0000 7f19de1dc700 1 -- 172.21.15.47:0/3863048025 wait complete. 2024-02-23T00:56:33.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:33.412+0000 7f19de1dc700 1 librados: shutdown 2024-02-23T00:56:33.417 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:56:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:56:33.418 INFO:teuthology.orchestra.run.smithi047.stderr:set pool 5 pgp_num to 42 2024-02-23T00:56:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:32.068320+0000 osd.7 (osd.7) 365 : cluster [DBG] 4.14 scrub starts 2024-02-23T00:56:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:32.072038+0000 osd.7 (osd.7) 366 : cluster [DBG] 4.14 scrub ok 2024-02-23T00:56:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:32.196974+0000 osd.2 (osd.2) 185 : cluster [DBG] 3.0 scrub starts 2024-02-23T00:56:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:32.198428+0000 osd.2 (osd.2) 186 : cluster [DBG] 3.0 scrub ok 2024-02-23T00:56:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:32.435734+0000 osd.4 (osd.4) 347 : cluster [DBG] 4.e scrub starts 2024-02-23T00:56:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:32.437234+0000 osd.4 (osd.4) 348 : cluster [DBG] 4.e scrub ok 2024-02-23T00:56:33.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:32.647274+0000 mon.c (mon.2) 523 : audit [INF] from='client.? 172.21.15.47:0/3863048025' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num", "val": "42"}]: dispatch 2024-02-23T00:56:33.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:32.647590+0000 mon.a (mon.0) 3862 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num", "val": "42"}]: dispatch 2024-02-23T00:56:33.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:33.219639+0000 osd.5 (osd.5) 362 : cluster [DBG] 18.1e scrub starts 2024-02-23T00:56:33.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:33.223001+0000 osd.5 (osd.5) 363 : cluster [DBG] 18.1e scrub ok 2024-02-23T00:56:33.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:32.068320+0000 osd.7 (osd.7) 365 : cluster [DBG] 4.14 scrub starts 2024-02-23T00:56:33.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:32.072038+0000 osd.7 (osd.7) 366 : cluster [DBG] 4.14 scrub ok 2024-02-23T00:56:33.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:32.196974+0000 osd.2 (osd.2) 185 : cluster [DBG] 3.0 scrub starts 2024-02-23T00:56:33.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:32.198428+0000 osd.2 (osd.2) 186 : cluster [DBG] 3.0 scrub ok 2024-02-23T00:56:33.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:32.435734+0000 osd.4 (osd.4) 347 : cluster [DBG] 4.e scrub starts 2024-02-23T00:56:33.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:32.437234+0000 osd.4 (osd.4) 348 : cluster [DBG] 4.e scrub ok 2024-02-23T00:56:33.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:32.647274+0000 mon.c (mon.2) 523 : audit [INF] from='client.? 172.21.15.47:0/3863048025' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num", "val": "42"}]: dispatch 2024-02-23T00:56:33.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:32.647590+0000 mon.a (mon.0) 3862 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num", "val": "42"}]: dispatch 2024-02-23T00:56:33.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:33.219639+0000 osd.5 (osd.5) 362 : cluster [DBG] 18.1e scrub starts 2024-02-23T00:56:33.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:33.223001+0000 osd.5 (osd.5) 363 : cluster [DBG] 18.1e scrub ok 2024-02-23T00:56:33.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:32.068320+0000 osd.7 (osd.7) 365 : cluster [DBG] 4.14 scrub starts 2024-02-23T00:56:33.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:32.072038+0000 osd.7 (osd.7) 366 : cluster [DBG] 4.14 scrub ok 2024-02-23T00:56:33.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:32.196974+0000 osd.2 (osd.2) 185 : cluster [DBG] 3.0 scrub starts 2024-02-23T00:56:33.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:32.198428+0000 osd.2 (osd.2) 186 : cluster [DBG] 3.0 scrub ok 2024-02-23T00:56:33.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:32.435734+0000 osd.4 (osd.4) 347 : cluster [DBG] 4.e scrub starts 2024-02-23T00:56:33.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:32.437234+0000 osd.4 (osd.4) 348 : cluster [DBG] 4.e scrub ok 2024-02-23T00:56:33.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:32.647274+0000 mon.c (mon.2) 523 : audit [INF] from='client.? 172.21.15.47:0/3863048025' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num", "val": "42"}]: dispatch 2024-02-23T00:56:33.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:32.647590+0000 mon.a (mon.0) 3862 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num", "val": "42"}]: dispatch 2024-02-23T00:56:33.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:33.219639+0000 osd.5 (osd.5) 362 : cluster [DBG] 18.1e scrub starts 2024-02-23T00:56:33.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:33.223001+0000 osd.5 (osd.5) 363 : cluster [DBG] 18.1e scrub ok 2024-02-23T00:56:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:33.018678+0000 mgr.y (mgr.14517) 699 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:56:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:33.410692+0000 mon.a (mon.0) 3863 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num", "val": "42"}]': finished 2024-02-23T00:56:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:33.410796+0000 mon.a (mon.0) 3864 : cluster [DBG] osdmap e825: 8 total, 8 up, 8 in 2024-02-23T00:56:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:33.416308+0000 mgr.y (mgr.14517) 700 : cluster [DBG] pgmap v1261: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/675 objects misplaced (0.148%) 2024-02-23T00:56:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:34.022375+0000 osd.6 (osd.6) 331 : cluster [DBG] 5.11 scrub starts 2024-02-23T00:56:34.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:34.023473+0000 osd.6 (osd.6) 332 : cluster [DBG] 5.11 scrub ok 2024-02-23T00:56:34.776 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:33.018678+0000 mgr.y (mgr.14517) 699 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:56:34.776 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:33.410692+0000 mon.a (mon.0) 3863 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num", "val": "42"}]': finished 2024-02-23T00:56:34.776 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:33.410796+0000 mon.a (mon.0) 3864 : cluster [DBG] osdmap e825: 8 total, 8 up, 8 in 2024-02-23T00:56:34.776 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:33.416308+0000 mgr.y (mgr.14517) 700 : cluster [DBG] pgmap v1261: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/675 objects misplaced (0.148%) 2024-02-23T00:56:34.776 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:34.022375+0000 osd.6 (osd.6) 331 : cluster [DBG] 5.11 scrub starts 2024-02-23T00:56:34.776 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:34.023473+0000 osd.6 (osd.6) 332 : cluster [DBG] 5.11 scrub ok 2024-02-23T00:56:34.777 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:33.018678+0000 mgr.y (mgr.14517) 699 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:56:34.777 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:33.410692+0000 mon.a (mon.0) 3863 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.control", "var": "pgp_num", "val": "42"}]': finished 2024-02-23T00:56:34.777 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:33.410796+0000 mon.a (mon.0) 3864 : cluster [DBG] osdmap e825: 8 total, 8 up, 8 in 2024-02-23T00:56:34.777 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:33.416308+0000 mgr.y (mgr.14517) 700 : cluster [DBG] pgmap v1261: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/675 objects misplaced (0.148%) 2024-02-23T00:56:34.777 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:34.022375+0000 osd.6 (osd.6) 331 : cluster [DBG] 5.11 scrub starts 2024-02-23T00:56:34.777 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:34.023473+0000 osd.6 (osd.6) 332 : cluster [DBG] 5.11 scrub ok 2024-02-23T00:56:35.119 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-02-23T00:56:35.614 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:56:35.634 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:56:35.634 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:56:35.660 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:34.001739+0000 osd.7 (osd.7) 367 : cluster [DBG] 4.17 scrub starts 2024-02-23T00:56:35.660 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:34.003195+0000 osd.7 (osd.7) 368 : cluster [DBG] 4.17 scrub ok 2024-02-23T00:56:35.660 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:34.174705+0000 osd.2 (osd.2) 187 : cluster [DBG] 18.12 scrub starts 2024-02-23T00:56:35.660 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:34.176530+0000 osd.2 (osd.2) 188 : cluster [DBG] 18.12 scrub ok 2024-02-23T00:56:35.660 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:34.414446+0000 osd.4 (osd.4) 349 : cluster [DBG] 4.d scrub starts 2024-02-23T00:56:35.660 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:34.416470+0000 osd.4 (osd.4) 350 : cluster [DBG] 4.d scrub ok 2024-02-23T00:56:35.660 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:35.006554+0000 osd.6 (osd.6) 333 : cluster [DBG] 5.1c scrub starts 2024-02-23T00:56:35.660 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:35.007613+0000 osd.6 (osd.6) 334 : cluster [DBG] 5.1c scrub ok 2024-02-23T00:56:35.660 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:35.333868+0000 mon.a (mon.0) 3865 : cluster [DBG] osdmap e826: 8 total, 8 up, 8 in 2024-02-23T00:56:35.660 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:34.001739+0000 osd.7 (osd.7) 367 : cluster [DBG] 4.17 scrub starts 2024-02-23T00:56:35.661 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:34.003195+0000 osd.7 (osd.7) 368 : cluster [DBG] 4.17 scrub ok 2024-02-23T00:56:35.661 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:34.174705+0000 osd.2 (osd.2) 187 : cluster [DBG] 18.12 scrub starts 2024-02-23T00:56:35.661 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:34.176530+0000 osd.2 (osd.2) 188 : cluster [DBG] 18.12 scrub ok 2024-02-23T00:56:35.661 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:34.414446+0000 osd.4 (osd.4) 349 : cluster [DBG] 4.d scrub starts 2024-02-23T00:56:35.661 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:34.416470+0000 osd.4 (osd.4) 350 : cluster [DBG] 4.d scrub ok 2024-02-23T00:56:35.661 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:35.006554+0000 osd.6 (osd.6) 333 : cluster [DBG] 5.1c scrub starts 2024-02-23T00:56:35.661 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:35.007613+0000 osd.6 (osd.6) 334 : cluster [DBG] 5.1c scrub ok 2024-02-23T00:56:35.661 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:35.333868+0000 mon.a (mon.0) 3865 : cluster [DBG] osdmap e826: 8 total, 8 up, 8 in 2024-02-23T00:56:35.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:34.001739+0000 osd.7 (osd.7) 367 : cluster [DBG] 4.17 scrub starts 2024-02-23T00:56:35.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:34.003195+0000 osd.7 (osd.7) 368 : cluster [DBG] 4.17 scrub ok 2024-02-23T00:56:35.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:34.174705+0000 osd.2 (osd.2) 187 : cluster [DBG] 18.12 scrub starts 2024-02-23T00:56:35.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:34.176530+0000 osd.2 (osd.2) 188 : cluster [DBG] 18.12 scrub ok 2024-02-23T00:56:35.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:34.414446+0000 osd.4 (osd.4) 349 : cluster [DBG] 4.d scrub starts 2024-02-23T00:56:35.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:34.416470+0000 osd.4 (osd.4) 350 : cluster [DBG] 4.d scrub ok 2024-02-23T00:56:35.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:35.006554+0000 osd.6 (osd.6) 333 : cluster [DBG] 5.1c scrub starts 2024-02-23T00:56:35.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:35.007613+0000 osd.6 (osd.6) 334 : cluster [DBG] 5.1c scrub ok 2024-02-23T00:56:35.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:35.333868+0000 mon.a (mon.0) 3865 : cluster [DBG] osdmap e826: 8 total, 8 up, 8 in 2024-02-23T00:56:36.424 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:35.015556+0000 osd.7 (osd.7) 369 : cluster [DBG] 18.11 scrub starts 2024-02-23T00:56:36.425 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:35.018778+0000 osd.7 (osd.7) 370 : cluster [DBG] 18.11 scrub ok 2024-02-23T00:56:36.425 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:35.410288+0000 osd.4 (osd.4) 351 : cluster [DBG] 4.18 deep-scrub starts 2024-02-23T00:56:36.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:35.015556+0000 osd.7 (osd.7) 369 : cluster [DBG] 18.11 scrub starts 2024-02-23T00:56:36.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:35.018778+0000 osd.7 (osd.7) 370 : cluster [DBG] 18.11 scrub ok 2024-02-23T00:56:36.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:35.410288+0000 osd.4 (osd.4) 351 : cluster [DBG] 4.18 deep-scrub starts 2024-02-23T00:56:36.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:35.414127+0000 osd.4 (osd.4) 352 : cluster [DBG] 4.18 deep-scrub ok 2024-02-23T00:56:36.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:35.416656+0000 mgr.y (mgr.14517) 701 : cluster [DBG] pgmap v1263: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/675 objects misplaced (0.148%) 2024-02-23T00:56:36.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:35.015556+0000 osd.7 (osd.7) 369 : cluster [DBG] 18.11 scrub starts 2024-02-23T00:56:36.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:35.018778+0000 osd.7 (osd.7) 370 : cluster [DBG] 18.11 scrub ok 2024-02-23T00:56:36.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:35.410288+0000 osd.4 (osd.4) 351 : cluster [DBG] 4.18 deep-scrub starts 2024-02-23T00:56:36.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:35.414127+0000 osd.4 (osd.4) 352 : cluster [DBG] 4.18 deep-scrub ok 2024-02-23T00:56:36.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:35.416656+0000 mgr.y (mgr.14517) 701 : cluster [DBG] pgmap v1263: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/675 objects misplaced (0.148%) 2024-02-23T00:56:36.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:35.414127+0000 osd.4 (osd.4) 352 : cluster [DBG] 4.18 deep-scrub ok 2024-02-23T00:56:36.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:35.416656+0000 mgr.y (mgr.14517) 701 : cluster [DBG] pgmap v1263: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/675 objects misplaced (0.148%) 2024-02-23T00:56:37.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.405+0000 7f3d0372d700 1 -- 172.21.15.47:0/1750499176 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3cfc0fe510 msgr2=0x7f3cfc0f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:37.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.405+0000 7f3d0372d700 1 --2- 172.21.15.47:0/1750499176 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3cfc0fe510 0x7f3cfc0f7d40 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 rx=0x7f3cf0004870 tx=0x7f3cf0009cb0).stop 2024-02-23T00:56:37.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.405+0000 7f3d0372d700 1 -- 172.21.15.47:0/1750499176 shutdown_connections 2024-02-23T00:56:37.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.405+0000 7f3d0372d700 1 --2- 172.21.15.47:0/1750499176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cfc0f8420 0x7f3cfc104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:37.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.405+0000 7f3d0372d700 1 --2- 172.21.15.47:0/1750499176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cfc0fdbb0 0x7f3cfc0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:37.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.405+0000 7f3d0372d700 1 --2- 172.21.15.47:0/1750499176 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3cfc0fe510 0x7f3cfc0f7d40 unknown :-1 s=CLOSED pgs=2836 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:37.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.405+0000 7f3d0372d700 1 -- 172.21.15.47:0/1750499176 >> 172.21.15.47:0/1750499176 conn(0x7f3cfc0f3800 msgr2=0x7f3cfc0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:56:37.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.405+0000 7f3d0372d700 1 -- 172.21.15.47:0/1750499176 shutdown_connections 2024-02-23T00:56:37.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.405+0000 7f3d0372d700 1 -- 172.21.15.47:0/1750499176 wait complete. 2024-02-23T00:56:37.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.406+0000 7f3d0372d700 1 librados: starting msgr at 2024-02-23T00:56:37.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.406+0000 7f3d0372d700 1 librados: starting objecter 2024-02-23T00:56:37.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.406+0000 7f3d0372d700 1 Processor -- start 2024-02-23T00:56:37.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.406+0000 7f3d0372d700 1 -- start start 2024-02-23T00:56:37.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.406+0000 7f3d0372d700 1 librados: setting wanted keys 2024-02-23T00:56:37.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.406+0000 7f3d0372d700 1 librados: calling monclient init 2024-02-23T00:56:37.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.406+0000 7f3d0372d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3cfc0f8420 0x7f3cfc06b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:37.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.406+0000 7f3d0372d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cfc0fdbb0 0x7f3cfc06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:37.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.406+0000 7f3d0372d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cfc0fe510 0x7f3cfc06bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:37.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.406+0000 7f3d0372d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc116360 con 0x7f3cfc0fe510 2024-02-23T00:56:37.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.406+0000 7f3d0372d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc1164d0 con 0x7f3cfc0fdbb0 2024-02-23T00:56:37.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.406+0000 7f3d0372d700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc116690 con 0x7f3cfc0f8420 2024-02-23T00:56:37.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.406+0000 7f3d014c9700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3cfc0f8420 0x7f3cfc06b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:37.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.406+0000 7f3d014c9700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3cfc0f8420 0x7f3cfc06b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:46514/0 (socket says 172.21.15.47:46514) 2024-02-23T00:56:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.406+0000 7f3d014c9700 1 -- 172.21.15.47:0/3423911263 learned_addr learned my addr 172.21.15.47:0/3423911263 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:56:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.406+0000 7f3d01cca700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cfc0fe510 0x7f3cfc06bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.406+0000 7f3d00cc8700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cfc0fdbb0 0x7f3cfc06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.406+0000 7f3d014c9700 1 -- 172.21.15.47:0/3423911263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cfc0fdbb0 msgr2=0x7f3cfc06b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.406+0000 7f3d014c9700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cfc0fdbb0 0x7f3cfc06b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.406+0000 7f3d014c9700 1 -- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cfc0fe510 msgr2=0x7f3cfc06bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.406+0000 7f3d014c9700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cfc0fe510 0x7f3cfc06bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.406+0000 7f3d014c9700 1 -- 172.21.15.47:0/3423911263 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf8005040 con 0x7f3cfc0f8420 2024-02-23T00:56:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.406+0000 7f3d00cc8700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cfc0fdbb0 0x7f3cfc06b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:56:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.406+0000 7f3d01cca700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cfc0fe510 0x7f3cfc06bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:56:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.406+0000 7f3d014c9700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3cfc0f8420 0x7f3cfc06b0d0 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 rx=0x7f3cf80048f0 tx=0x7f3cf8009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.406+0000 7f3cee7fc700 10 client.?.objecter ms_handle_connect 0x7f3cfc0f8420 2024-02-23T00:56:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.406+0000 7f3cee7fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:56:37.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.407+0000 7f3cee7fc700 1 -- 172.21.15.47:0/3423911263 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8007b00 con 0x7f3cfc0f8420 2024-02-23T00:56:37.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.407+0000 7f3d0372d700 1 -- 172.21.15.47:0/3423911263 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cf0005040 con 0x7f3cfc0f8420 2024-02-23T00:56:37.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.407+0000 7f3cee7fc700 1 -- 172.21.15.47:0/3423911263 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f3cf800f040 con 0x7f3cfc0f8420 2024-02-23T00:56:37.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.407+0000 7f3cee7fc700 1 -- 172.21.15.47:0/3423911263 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8014890 con 0x7f3cfc0f8420 2024-02-23T00:56:37.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.407+0000 7f3d0372d700 10 client.42245.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:56:37.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.407+0000 7f3d0372d700 1 -- 172.21.15.47:0/3423911263 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cfc116d50 con 0x7f3cfc0f8420 2024-02-23T00:56:37.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.407+0000 7f3d0372d700 1 librados: init done 2024-02-23T00:56:37.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.407+0000 7f3d0372d700 1 -- 172.21.15.47:0/3423911263 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3cfc106e60 con 0x7f3cfc0f8420 2024-02-23T00:56:37.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.408+0000 7f3cee7fc700 1 -- 172.21.15.47:0/3423911263 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f3cf8014cc0 con 0x7f3cfc0f8420 2024-02-23T00:56:37.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.408+0000 7f3cee7fc700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3ce8062cc0 0x7f3ce8065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:37.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.408+0000 7f3cee7fc700 1 -- 172.21.15.47:0/3423911263 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(826..826 src has 621..826) v4 ==== 8638+0+0 (secure 0 0 0) 0x7f3cf807d010 con 0x7f3cfc0f8420 2024-02-23T00:56:37.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.408+0000 7f3d00cc8700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3ce8062cc0 0x7f3ce8065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:37.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.408+0000 7f3cee7fc700 10 client.42245.objecter ms_dispatch 0x7f3cfc001550 osd_map(826..826 src has 621..826) v4 2024-02-23T00:56:37.489 INFO:teuthology.orchestra.run.smithi047.stdout:mon.a: {} 2024-02-23T00:56:37.489 INFO:teuthology.orchestra.run.smithi047.stdout:mon.b: {} 2024-02-23T00:56:37.489 INFO:teuthology.orchestra.run.smithi047.stdout:mon.c: {} 2024-02-23T00:56:37.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.408+0000 7f3cee7fc700 3 client.42245.objecter handle_osd_map got epochs [826,826] > 0 2024-02-23T00:56:37.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.408+0000 7f3cee7fc700 3 client.42245.objecter handle_osd_map decoding full epoch 826 2024-02-23T00:56:37.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.409+0000 7f3cee7fc700 20 client.42245.objecter dump_active .. 0 homeless 2024-02-23T00:56:37.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.409+0000 7f3cee7fc700 1 -- 172.21.15.47:0/3423911263 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1259 (secure 0 0 0) 0x7f3cf8084090 con 0x7f3cfc0f8420 2024-02-23T00:56:37.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.409+0000 7f3cee7fc700 10 client.42245.objecter ms_handle_connect 0x7f3ce8062cc0 2024-02-23T00:56:37.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.409+0000 7f3d00cc8700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3ce8062cc0 0x7f3ce8065180 secure :-1 s=READY pgs=3225 cs=0 l=1 rev1=1 rx=0x7f3cf0004ee0 tx=0x7f3cf000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:37.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.409+0000 7f3d0372d700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cfc0fe8f0 0x7f3cfc0fecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:37.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.409+0000 7f3d0372d700 1 -- 172.21.15.47:0/3423911263 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc06c5b0 con 0x7f3cfc0fe8f0 2024-02-23T00:56:37.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.409+0000 7f3d01cca700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cfc0fe8f0 0x7f3cfc0fecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:37.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.409+0000 7f3d01cca700 1 -- 172.21.15.47:0/3423911263 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3cfc05fa60 con 0x7f3cfc0fe8f0 2024-02-23T00:56:37.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.409+0000 7f3d01cca700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cfc0fe8f0 0x7f3cfc0fecf0 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 rx=0x7f3cf400fe00 tx=0x7f3cf4009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:37.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.409+0000 7f3cee7fc700 10 client.42245.objecter ms_handle_connect 0x7f3cfc0fe8f0 2024-02-23T00:56:37.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.409+0000 7f3cee7fc700 10 client.42245.objecter resend_mon_ops 2024-02-23T00:56:37.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.409+0000 7f3cee7fc700 1 -- 172.21.15.47:0/3423911263 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf400dd40 con 0x7f3cfc0fe8f0 2024-02-23T00:56:37.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.410+0000 7f3cee7fc700 1 -- 172.21.15.47:0/3423911263 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+8167 (secure 0 0 0) 0x7f3cf400dea0 con 0x7f3cfc0fe8f0 2024-02-23T00:56:37.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.410+0000 7f3cee7fc700 1 -- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cfc0fe8f0 msgr2=0x7f3cfc0fecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:37.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.410+0000 7f3cee7fc700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cfc0fe8f0 0x7f3cfc0fecf0 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 rx=0x7f3cf400fe00 tx=0x7f3cf4009040).stop 2024-02-23T00:56:37.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.410+0000 7f3cee7fc700 1 -- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cfc0fe8f0 msgr2=0x7f3cfc0fecf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-02-23T00:56:37.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.410+0000 7f3cee7fc700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cfc0fe8f0 0x7f3cfc0fecf0 unknown :-1 s=CLOSED pgs=2660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:37.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.416+0000 7f3d0372d700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cfc0ff1e0 0x7f3cfc11aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:37.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.416+0000 7f3d0372d700 1 -- 172.21.15.47:0/3423911263 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc06c720 con 0x7f3cfc0ff1e0 2024-02-23T00:56:37.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.417+0000 7f3d01cca700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cfc0ff1e0 0x7f3cfc11aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:37.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.417+0000 7f3d01cca700 1 -- 172.21.15.47:0/3423911263 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 3: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=600"]}) v1 -- 0x7f3cfc0ff5c0 con 0x7f3cfc0ff1e0 2024-02-23T00:56:37.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.417+0000 7f3d01cca700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cfc0ff1e0 0x7f3cfc11aff0 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 rx=0x7f3cf400fe00 tx=0x7f3cf4018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:37.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.417+0000 7f3cee7fc700 10 client.42245.objecter ms_handle_connect 0x7f3cfc0ff1e0 2024-02-23T00:56:37.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.417+0000 7f3cee7fc700 10 client.42245.objecter resend_mon_ops 2024-02-23T00:56:37.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.417+0000 7f3cee7fc700 1 -- 172.21.15.47:0/3423911263 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4003bd0 con 0x7f3cfc0ff1e0 2024-02-23T00:56:37.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.419+0000 7f3cee7fc700 1 -- 172.21.15.47:0/3423911263 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 3: 0 mon_osd_down_out_interval = '600' (not observed, change may require restart) ) v1 ==== 85+0+3 (secure 0 0 0) 0x7f3cf400db30 con 0x7f3cfc0ff1e0 2024-02-23T00:56:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.419+0000 7f3cee7fc700 1 -- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cfc0ff1e0 msgr2=0x7f3cfc11aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.419+0000 7f3cee7fc700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cfc0ff1e0 0x7f3cfc11aff0 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 rx=0x7f3cf400fe00 tx=0x7f3cf4018000).stop 2024-02-23T00:56:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.419+0000 7f3cee7fc700 1 -- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cfc0ff1e0 msgr2=0x7f3cfc11aff0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-02-23T00:56:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.419+0000 7f3cee7fc700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cfc0ff1e0 0x7f3cfc11aff0 secure :-1 s=CLOSED pgs=2661 cs=0 l=1 rev1=1 rx=0x7f3cf400fe00 tx=0x7f3cf4018000).stop 2024-02-23T00:56:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.419+0000 7f3d0372d700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cfc0273c0 0x7f3cfc0277d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.419+0000 7f3d0372d700 1 -- 172.21.15.47:0/3423911263 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc0ff910 con 0x7f3cfc0273c0 2024-02-23T00:56:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.419+0000 7f3d01cca700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cfc0273c0 0x7f3cfc0277d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.420+0000 7f3d01cca700 1 -- 172.21.15.47:0/3423911263 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- command(tid 4: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3cfc027cc0 con 0x7f3cfc0273c0 2024-02-23T00:56:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.420+0000 7f3cee7fc700 10 client.42245.objecter ms_handle_connect 0x7f3cfc0273c0 2024-02-23T00:56:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.420+0000 7f3cee7fc700 10 client.42245.objecter resend_mon_ops 2024-02-23T00:56:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.420+0000 7f3d01cca700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cfc0273c0 0x7f3cfc0277d0 secure :-1 s=READY pgs=3984 cs=0 l=1 rev1=1 rx=0x7f3cf4010010 tx=0x7f3cf4021000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.420+0000 7f3cee7fc700 1 -- 172.21.15.47:0/3423911263 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4014890 con 0x7f3cfc0273c0 2024-02-23T00:56:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.420+0000 7f3cee7fc700 1 -- 172.21.15.47:0/3423911263 <== mon.1 v2:172.21.15.196:3300/0 2 ==== command_reply(tid 4: 0 ) v1 ==== 8+0+8167 (secure 0 0 0) 0x7f3cf40039c0 con 0x7f3cfc0273c0 2024-02-23T00:56:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.420+0000 7f3cee7fc700 1 -- 172.21.15.47:0/3423911263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cfc0273c0 msgr2=0x7f3cfc0277d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.420+0000 7f3cee7fc700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cfc0273c0 0x7f3cfc0277d0 secure :-1 s=READY pgs=3984 cs=0 l=1 rev1=1 rx=0x7f3cf4010010 tx=0x7f3cf4021000).stop 2024-02-23T00:56:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.420+0000 7f3cee7fc700 1 -- 172.21.15.47:0/3423911263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cfc0273c0 msgr2=0x7f3cfc0277d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-02-23T00:56:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.420+0000 7f3cee7fc700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cfc0273c0 0x7f3cfc0277d0 secure :-1 s=CLOSED pgs=3984 cs=0 l=1 rev1=1 rx=0x7f3cf4010010 tx=0x7f3cf4021000).stop 2024-02-23T00:56:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.420+0000 7f3d01cca700 1 -- 172.21.15.47:0/3423911263 reap_dead start 2024-02-23T00:56:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.427+0000 7f3d0372d700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cfc06b770 0x7f3cfc027e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.427+0000 7f3d0372d700 1 -- 172.21.15.47:0/3423911263 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc06bb50 con 0x7f3cfc06b770 2024-02-23T00:56:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.427+0000 7f3d01cca700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cfc06b770 0x7f3cfc027e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.427+0000 7f3d01cca700 1 -- 172.21.15.47:0/3423911263 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- command(tid 5: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=600"]}) v1 -- 0x7f3cfc06bce0 con 0x7f3cfc06b770 2024-02-23T00:56:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.427+0000 7f3cee7fc700 10 client.42245.objecter ms_handle_connect 0x7f3cfc06b770 2024-02-23T00:56:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.427+0000 7f3cee7fc700 10 client.42245.objecter resend_mon_ops 2024-02-23T00:56:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.427+0000 7f3d01cca700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cfc06b770 0x7f3cfc027e90 secure :-1 s=READY pgs=3985 cs=0 l=1 rev1=1 rx=0x7f3cf4007d10 tx=0x7f3cf402c000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.428+0000 7f3cee7fc700 1 -- 172.21.15.47:0/3423911263 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf402c640 con 0x7f3cfc06b770 2024-02-23T00:56:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.429+0000 7f3cee7fc700 1 -- 172.21.15.47:0/3423911263 <== mon.1 v2:172.21.15.196:3300/0 2 ==== command_reply(tid 5: 0 mon_osd_down_out_interval = '600' (not observed, change may require restart) ) v1 ==== 85+0+3 (secure 0 0 0) 0x7f3cf4003430 con 0x7f3cfc06b770 2024-02-23T00:56:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.429+0000 7f3cee7fc700 1 -- 172.21.15.47:0/3423911263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cfc06b770 msgr2=0x7f3cfc027e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.429+0000 7f3cee7fc700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cfc06b770 0x7f3cfc027e90 secure :-1 s=READY pgs=3985 cs=0 l=1 rev1=1 rx=0x7f3cf4007d10 tx=0x7f3cf402c000).stop 2024-02-23T00:56:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.429+0000 7f3cee7fc700 1 -- 172.21.15.47:0/3423911263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cfc06b770 msgr2=0x7f3cfc027e90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-02-23T00:56:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.429+0000 7f3cee7fc700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cfc06b770 0x7f3cfc027e90 secure :-1 s=CLOSED pgs=3985 cs=0 l=1 rev1=1 rx=0x7f3cf4007d10 tx=0x7f3cf402c000).stop 2024-02-23T00:56:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.429+0000 7f3d0372d700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3cfc028380 0x7f3cfc11d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.429+0000 7f3d0372d700 1 -- 172.21.15.47:0/3423911263 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc028760 con 0x7f3cfc028380 2024-02-23T00:56:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.429+0000 7f3d01cca700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3cfc028380 0x7f3cfc11d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.430+0000 7f3d01cca700 1 -- 172.21.15.47:0/3423911263 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 6: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3cfc120140 con 0x7f3cfc028380 2024-02-23T00:56:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.430+0000 7f3cee7fc700 10 client.42245.objecter ms_handle_connect 0x7f3cfc028380 2024-02-23T00:56:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.430+0000 7f3cee7fc700 10 client.42245.objecter resend_mon_ops 2024-02-23T00:56:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.430+0000 7f3d01cca700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3cfc028380 0x7f3cfc11d4f0 secure :-1 s=READY pgs=2838 cs=0 l=1 rev1=1 rx=0x7f3cf4010010 tx=0x7f3cf4037000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.430+0000 7f3cee7fc700 1 -- 172.21.15.47:0/3423911263 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf402dcf0 con 0x7f3cfc028380 2024-02-23T00:56:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.430+0000 7f3cee7fc700 1 -- 172.21.15.47:0/3423911263 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 6: 0 ) v1 ==== 8+0+8167 (secure 0 0 0) 0x7f3cf402de50 con 0x7f3cfc028380 2024-02-23T00:56:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.430+0000 7f3cee7fc700 1 -- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3cfc028380 msgr2=0x7f3cfc11d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.430+0000 7f3cee7fc700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3cfc028380 0x7f3cfc11d4f0 secure :-1 s=READY pgs=2838 cs=0 l=1 rev1=1 rx=0x7f3cf4010010 tx=0x7f3cf4037000).stop 2024-02-23T00:56:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.430+0000 7f3cee7fc700 1 -- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3cfc028380 msgr2=0x7f3cfc11d4f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-02-23T00:56:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.430+0000 7f3cee7fc700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3cfc028380 0x7f3cfc11d4f0 unknown :-1 s=CLOSED pgs=2838 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.437+0000 7f3d0372d700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3cfc120460 0x7f3cfc11f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.437+0000 7f3d0372d700 1 -- 172.21.15.47:0/3423911263 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc11ff80 con 0x7f3cfc120460 2024-02-23T00:56:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.437+0000 7f3d01cca700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3cfc120460 0x7f3cfc11f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.437+0000 7f3d01cca700 1 -- 172.21.15.47:0/3423911263 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=600"]}) v1 -- 0x7f3cfc121e20 con 0x7f3cfc120460 2024-02-23T00:56:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.437+0000 7f3cee7fc700 10 client.42245.objecter ms_handle_connect 0x7f3cfc120460 2024-02-23T00:56:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.437+0000 7f3cee7fc700 10 client.42245.objecter resend_mon_ops 2024-02-23T00:56:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.437+0000 7f3d01cca700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3cfc120460 0x7f3cfc11f9f0 secure :-1 s=READY pgs=2839 cs=0 l=1 rev1=1 rx=0x7f3cf4014e70 tx=0x7f3cf403e000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:37.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.437+0000 7f3cee7fc700 1 -- 172.21.15.47:0/3423911263 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4038e00 con 0x7f3cfc120460 2024-02-23T00:56:37.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.438+0000 7f3cee7fc700 1 -- 172.21.15.47:0/3423911263 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 7: 0 mon_osd_down_out_interval = '600' (not observed, change may require restart) ) v1 ==== 85+0+3 (secure 0 0 0) 0x7f3cf402dae0 con 0x7f3cfc120460 2024-02-23T00:56:37.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.438+0000 7f3cee7fc700 1 -- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3cfc120460 msgr2=0x7f3cfc11f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:37.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.438+0000 7f3cee7fc700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3cfc120460 0x7f3cfc11f9f0 secure :-1 s=READY pgs=2839 cs=0 l=1 rev1=1 rx=0x7f3cf4014e70 tx=0x7f3cf403e000).stop 2024-02-23T00:56:37.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.438+0000 7f3cee7fc700 1 -- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3cfc120460 msgr2=0x7f3cfc11f9f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-02-23T00:56:37.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.438+0000 7f3cee7fc700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3cfc120460 0x7f3cfc11f9f0 secure :-1 s=CLOSED pgs=2839 cs=0 l=1 rev1=1 rx=0x7f3cf4014e70 tx=0x7f3cf403e000).stop 2024-02-23T00:56:37.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.439+0000 7f3d0372d700 10 librados: watch_flush enter 2024-02-23T00:56:37.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.439+0000 7f3d0372d700 10 librados: watch_flush exit 2024-02-23T00:56:37.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.439+0000 7f3d0372d700 20 client.42245.objecter shutdown clearing up homeless session... 2024-02-23T00:56:37.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.439+0000 7f3d0372d700 10 client.42245.objecter successfully canceled tick 2024-02-23T00:56:37.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.439+0000 7f3d0372d700 1 -- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3ce8062cc0 msgr2=0x7f3ce8065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:37.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.439+0000 7f3d0372d700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3ce8062cc0 0x7f3ce8065180 secure :-1 s=READY pgs=3225 cs=0 l=1 rev1=1 rx=0x7f3cf0004ee0 tx=0x7f3cf000b040).stop 2024-02-23T00:56:37.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.439+0000 7f3d0372d700 1 -- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3cfc0f8420 msgr2=0x7f3cfc06b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:37.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.439+0000 7f3d0372d700 1 --2- 172.21.15.47:0/3423911263 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3cfc0f8420 0x7f3cfc06b0d0 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 rx=0x7f3cf80048f0 tx=0x7f3cf8009cb0).stop 2024-02-23T00:56:37.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.439+0000 7f3d01cca700 1 -- 172.21.15.47:0/3423911263 reap_dead start 2024-02-23T00:56:37.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.439+0000 7f3d0372d700 1 -- 172.21.15.47:0/3423911263 shutdown_connections 2024-02-23T00:56:37.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.439+0000 7f3d0372d700 1 -- 172.21.15.47:0/3423911263 >> 172.21.15.47:0/3423911263 conn(0x7f3cfc0f3800 msgr2=0x7f3cfc0fb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:56:37.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.439+0000 7f3d0372d700 1 -- 172.21.15.47:0/3423911263 shutdown_connections 2024-02-23T00:56:37.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.439+0000 7f3d0372d700 1 -- 172.21.15.47:0/3423911263 wait complete. 2024-02-23T00:56:37.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:37.439+0000 7f3d0372d700 1 librados: shutdown 2024-02-23T00:56:37.497 INFO:teuthology.orchestra.run.smithi047.stderr:mon.a: mon_osd_down_out_interval = '600' (not observed, change may require restart) 2024-02-23T00:56:37.497 INFO:teuthology.orchestra.run.smithi047.stderr:mon.b: mon_osd_down_out_interval = '600' (not observed, change may require restart) 2024-02-23T00:56:37.497 INFO:teuthology.orchestra.run.smithi047.stderr:mon.c: mon_osd_down_out_interval = '600' (not observed, change may require restart) 2024-02-23T00:56:37.739 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.2 dump_blocked_ops 2024-02-23T00:56:38.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:36.799127+0000 mon.a (mon.0) 3866 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:56:38.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:37.223574+0000 osd.2 (osd.2) 189 : cluster [DBG] 5.8 scrub starts 2024-02-23T00:56:38.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:37.224394+0000 osd.2 (osd.2) 190 : cluster [DBG] 5.8 scrub ok 2024-02-23T00:56:38.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:37.408832+0000 mon.c (mon.2) 524 : audit [DBG] from='client.? 172.21.15.47:0/3423911263' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-23T00:56:38.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:37.417420+0000 mgr.y (mgr.14517) 702 : cluster [DBG] pgmap v1264: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 1/675 objects misplaced (0.148%) 2024-02-23T00:56:38.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:36.799127+0000 mon.a (mon.0) 3866 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:56:38.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:37.223574+0000 osd.2 (osd.2) 189 : cluster [DBG] 5.8 scrub starts 2024-02-23T00:56:38.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:37.224394+0000 osd.2 (osd.2) 190 : cluster [DBG] 5.8 scrub ok 2024-02-23T00:56:38.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:37.408832+0000 mon.c (mon.2) 524 : audit [DBG] from='client.? 172.21.15.47:0/3423911263' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-23T00:56:38.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:37.417420+0000 mgr.y (mgr.14517) 702 : cluster [DBG] pgmap v1264: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 1/675 objects misplaced (0.148%) 2024-02-23T00:56:38.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:36.799127+0000 mon.a (mon.0) 3866 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:56:38.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:37.223574+0000 osd.2 (osd.2) 189 : cluster [DBG] 5.8 scrub starts 2024-02-23T00:56:38.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:37.224394+0000 osd.2 (osd.2) 190 : cluster [DBG] 5.8 scrub ok 2024-02-23T00:56:38.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:37.408832+0000 mon.c (mon.2) 524 : audit [DBG] from='client.? 172.21.15.47:0/3423911263' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-23T00:56:38.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:37.417420+0000 mgr.y (mgr.14517) 702 : cluster [DBG] pgmap v1264: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 1/675 objects misplaced (0.148%) 2024-02-23T00:56:38.452 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight 0 1 2024-02-23T00:56:38.550 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:56:39.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:37.058771+0000 osd.7 (osd.7) 371 : cluster [DBG] 5.10 scrub starts 2024-02-23T00:56:39.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:37.061682+0000 osd.7 (osd.7) 372 : cluster [DBG] 5.10 scrub ok 2024-02-23T00:56:39.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:38.232772+0000 osd.2 (osd.2) 191 : cluster [DBG] 2.1 scrub starts 2024-02-23T00:56:39.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:38.233582+0000 osd.2 (osd.2) 192 : cluster [DBG] 2.1 scrub ok 2024-02-23T00:56:39.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:37.058771+0000 osd.7 (osd.7) 371 : cluster [DBG] 5.10 scrub starts 2024-02-23T00:56:39.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:37.061682+0000 osd.7 (osd.7) 372 : cluster [DBG] 5.10 scrub ok 2024-02-23T00:56:39.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:38.232772+0000 osd.2 (osd.2) 191 : cluster [DBG] 2.1 scrub starts 2024-02-23T00:56:39.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:38.233582+0000 osd.2 (osd.2) 192 : cluster [DBG] 2.1 scrub ok 2024-02-23T00:56:39.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:37.058771+0000 osd.7 (osd.7) 371 : cluster [DBG] 5.10 scrub starts 2024-02-23T00:56:39.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:37.061682+0000 osd.7 (osd.7) 372 : cluster [DBG] 5.10 scrub ok 2024-02-23T00:56:39.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:38.232772+0000 osd.2 (osd.2) 191 : cluster [DBG] 2.1 scrub starts 2024-02-23T00:56:39.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:38.233582+0000 osd.2 (osd.2) 192 : cluster [DBG] 2.1 scrub ok 2024-02-23T00:56:39.808 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:56:40.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:38.030789+0000 osd.7 (osd.7) 373 : cluster [DBG] 18.3 scrub starts 2024-02-23T00:56:40.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:38.034587+0000 osd.7 (osd.7) 374 : cluster [DBG] 18.3 scrub ok 2024-02-23T00:56:40.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:38.424660+0000 osd.4 (osd.4) 353 : cluster [DBG] 5.c scrub starts 2024-02-23T00:56:40.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:38.425578+0000 osd.4 (osd.4) 354 : cluster [DBG] 5.c scrub ok 2024-02-23T00:56:40.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:38.438754+0000 osd.3 (osd.3) 325 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:56:40.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:38.438913+0000 osd.3 (osd.3) 326 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:56:40.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:38.809603+0000 mon.a (mon.0) 3867 : cluster [DBG] osdmap e827: 8 total, 8 up, 8 in 2024-02-23T00:56:40.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:38.986095+0000 osd.6 (osd.6) 335 : cluster [DBG] 5.12 scrub starts 2024-02-23T00:56:40.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:38.987089+0000 osd.6 (osd.6) 336 : cluster [DBG] 5.12 scrub ok 2024-02-23T00:56:40.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:39.103135+0000 osd.0 (osd.0) 310 : cluster [DBG] 5.1e scrub starts 2024-02-23T00:56:40.064 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:39.104054+0000 osd.0 (osd.0) 311 : cluster [DBG] 5.1e scrub ok 2024-02-23T00:56:40.065 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:39.227912+0000 osd.5 (osd.5) 364 : cluster [DBG] 5.f deep-scrub starts 2024-02-23T00:56:40.065 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:39.230820+0000 osd.5 (osd.5) 365 : cluster [DBG] 5.f deep-scrub ok 2024-02-23T00:56:40.065 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:39.417856+0000 mgr.y (mgr.14517) 703 : cluster [DBG] pgmap v1266: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 851 B/s rd, 0 op/s; 1/675 objects misplaced (0.148%) 2024-02-23T00:56:40.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:38.030789+0000 osd.7 (osd.7) 373 : cluster [DBG] 18.3 scrub starts 2024-02-23T00:56:40.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:38.034587+0000 osd.7 (osd.7) 374 : cluster [DBG] 18.3 scrub ok 2024-02-23T00:56:40.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:38.424660+0000 osd.4 (osd.4) 353 : cluster [DBG] 5.c scrub starts 2024-02-23T00:56:40.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:38.425578+0000 osd.4 (osd.4) 354 : cluster [DBG] 5.c scrub ok 2024-02-23T00:56:40.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:38.438754+0000 osd.3 (osd.3) 325 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:56:40.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:38.438913+0000 osd.3 (osd.3) 326 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:56:40.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:38.809603+0000 mon.a (mon.0) 3867 : cluster [DBG] osdmap e827: 8 total, 8 up, 8 in 2024-02-23T00:56:40.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:38.986095+0000 osd.6 (osd.6) 335 : cluster [DBG] 5.12 scrub starts 2024-02-23T00:56:40.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:38.987089+0000 osd.6 (osd.6) 336 : cluster [DBG] 5.12 scrub ok 2024-02-23T00:56:40.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:39.103135+0000 osd.0 (osd.0) 310 : cluster [DBG] 5.1e scrub starts 2024-02-23T00:56:40.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:39.104054+0000 osd.0 (osd.0) 311 : cluster [DBG] 5.1e scrub ok 2024-02-23T00:56:40.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:39.227912+0000 osd.5 (osd.5) 364 : cluster [DBG] 5.f deep-scrub starts 2024-02-23T00:56:40.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:39.230820+0000 osd.5 (osd.5) 365 : cluster [DBG] 5.f deep-scrub ok 2024-02-23T00:56:40.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:39.417856+0000 mgr.y (mgr.14517) 703 : cluster [DBG] pgmap v1266: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 851 B/s rd, 0 op/s; 1/675 objects misplaced (0.148%) 2024-02-23T00:56:40.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:38.030789+0000 osd.7 (osd.7) 373 : cluster [DBG] 18.3 scrub starts 2024-02-23T00:56:40.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:38.034587+0000 osd.7 (osd.7) 374 : cluster [DBG] 18.3 scrub ok 2024-02-23T00:56:40.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:38.424660+0000 osd.4 (osd.4) 353 : cluster [DBG] 5.c scrub starts 2024-02-23T00:56:40.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:38.425578+0000 osd.4 (osd.4) 354 : cluster [DBG] 5.c scrub ok 2024-02-23T00:56:40.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:38.438754+0000 osd.3 (osd.3) 325 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:56:40.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:38.438913+0000 osd.3 (osd.3) 326 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:56:40.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:38.809603+0000 mon.a (mon.0) 3867 : cluster [DBG] osdmap e827: 8 total, 8 up, 8 in 2024-02-23T00:56:40.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:38.986095+0000 osd.6 (osd.6) 335 : cluster [DBG] 5.12 scrub starts 2024-02-23T00:56:40.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:38.987089+0000 osd.6 (osd.6) 336 : cluster [DBG] 5.12 scrub ok 2024-02-23T00:56:40.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:39.103135+0000 osd.0 (osd.0) 310 : cluster [DBG] 5.1e scrub starts 2024-02-23T00:56:40.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:39.104054+0000 osd.0 (osd.0) 311 : cluster [DBG] 5.1e scrub ok 2024-02-23T00:56:40.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:39.227912+0000 osd.5 (osd.5) 364 : cluster [DBG] 5.f deep-scrub starts 2024-02-23T00:56:40.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:39.230820+0000 osd.5 (osd.5) 365 : cluster [DBG] 5.f deep-scrub ok 2024-02-23T00:56:40.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:39.417856+0000 mgr.y (mgr.14517) 703 : cluster [DBG] pgmap v1266: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 851 B/s rd, 0 op/s; 1/675 objects misplaced (0.148%) 2024-02-23T00:56:40.807 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:56:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:56:40] "GET /metrics HTTP/1.1" 200 187122 "" "Prometheus/2.33.4" 2024-02-23T00:56:40.847 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:56:40.847 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:56:41.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:39.072299+0000 osd.7 (osd.7) 375 : cluster [DBG] 18.1a scrub starts 2024-02-23T00:56:41.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:39.075912+0000 osd.7 (osd.7) 376 : cluster [DBG] 18.1a scrub ok 2024-02-23T00:56:41.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:39.818325+0000 mon.a (mon.0) 3868 : cluster [DBG] osdmap e828: 8 total, 8 up, 8 in 2024-02-23T00:56:41.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:40.215431+0000 osd.5 (osd.5) 366 : cluster [DBG] 2.16 scrub starts 2024-02-23T00:56:41.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:40.218358+0000 osd.5 (osd.5) 367 : cluster [DBG] 2.16 scrub ok 2024-02-23T00:56:41.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:39.072299+0000 osd.7 (osd.7) 375 : cluster [DBG] 18.1a scrub starts 2024-02-23T00:56:41.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:39.075912+0000 osd.7 (osd.7) 376 : cluster [DBG] 18.1a scrub ok 2024-02-23T00:56:41.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:39.818325+0000 mon.a (mon.0) 3868 : cluster [DBG] osdmap e828: 8 total, 8 up, 8 in 2024-02-23T00:56:41.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:40.215431+0000 osd.5 (osd.5) 366 : cluster [DBG] 2.16 scrub starts 2024-02-23T00:56:41.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:40.218358+0000 osd.5 (osd.5) 367 : cluster [DBG] 2.16 scrub ok 2024-02-23T00:56:41.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:39.072299+0000 osd.7 (osd.7) 375 : cluster [DBG] 18.1a scrub starts 2024-02-23T00:56:41.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:39.075912+0000 osd.7 (osd.7) 376 : cluster [DBG] 18.1a scrub ok 2024-02-23T00:56:41.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:39.818325+0000 mon.a (mon.0) 3868 : cluster [DBG] osdmap e828: 8 total, 8 up, 8 in 2024-02-23T00:56:41.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:40.215431+0000 osd.5 (osd.5) 366 : cluster [DBG] 2.16 scrub starts 2024-02-23T00:56:41.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:40.218358+0000 osd.5 (osd.5) 367 : cluster [DBG] 2.16 scrub ok 2024-02-23T00:56:41.831 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:40.488249+0000 osd.4 (osd.4) 355 : cluster [DBG] 5.e scrub starts 2024-02-23T00:56:41.832 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:40.489990+0000 osd.4 (osd.4) 356 : cluster [DBG] 5.e scrub ok 2024-02-23T00:56:41.832 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:40.818105+0000 mon.a (mon.0) 3869 : cluster [DBG] osdmap e829: 8 total, 8 up, 8 in 2024-02-23T00:56:41.832 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:41.183855+0000 osd.5 (osd.5) 368 : cluster [DBG] 18.17 scrub starts 2024-02-23T00:56:41.832 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:41.187986+0000 osd.5 (osd.5) 369 : cluster [DBG] 18.17 scrub ok 2024-02-23T00:56:41.832 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:41.418455+0000 mgr.y (mgr.14517) 704 : cluster [DBG] pgmap v1269: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/675 objects misplaced (0.148%) 2024-02-23T00:56:42.016 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.2 dump_historic_ops 2024-02-23T00:56:42.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:40.488249+0000 osd.4 (osd.4) 355 : cluster [DBG] 5.e scrub starts 2024-02-23T00:56:42.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:40.489990+0000 osd.4 (osd.4) 356 : cluster [DBG] 5.e scrub ok 2024-02-23T00:56:42.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:40.818105+0000 mon.a (mon.0) 3869 : cluster [DBG] osdmap e829: 8 total, 8 up, 8 in 2024-02-23T00:56:42.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:41.183855+0000 osd.5 (osd.5) 368 : cluster [DBG] 18.17 scrub starts 2024-02-23T00:56:42.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:41.187986+0000 osd.5 (osd.5) 369 : cluster [DBG] 18.17 scrub ok 2024-02-23T00:56:42.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:41.418455+0000 mgr.y (mgr.14517) 704 : cluster [DBG] pgmap v1269: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/675 objects misplaced (0.148%) 2024-02-23T00:56:42.246 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:40.488249+0000 osd.4 (osd.4) 355 : cluster [DBG] 5.e scrub starts 2024-02-23T00:56:42.246 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:40.489990+0000 osd.4 (osd.4) 356 : cluster [DBG] 5.e scrub ok 2024-02-23T00:56:42.246 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:40.818105+0000 mon.a (mon.0) 3869 : cluster [DBG] osdmap e829: 8 total, 8 up, 8 in 2024-02-23T00:56:42.246 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:41.183855+0000 osd.5 (osd.5) 368 : cluster [DBG] 18.17 scrub starts 2024-02-23T00:56:42.246 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:41.187986+0000 osd.5 (osd.5) 369 : cluster [DBG] 18.17 scrub ok 2024-02-23T00:56:42.246 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:41.418455+0000 mgr.y (mgr.14517) 704 : cluster [DBG] pgmap v1269: 154 pgs: 1 active+remapped+backfill_toofull, 153 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1/675 objects misplaced (0.148%) 2024-02-23T00:56:42.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.468+0000 7fa4c700d700 1 -- 172.21.15.47:0/620804121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4c0102270 msgr2=0x7fa4c0104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:42.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.468+0000 7fa4c700d700 1 --2- 172.21.15.47:0/620804121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4c0102270 0x7fa4c0104660 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 rx=0x7fa4b8004870 tx=0x7fa4b8009cb0).stop 2024-02-23T00:56:42.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.469+0000 7fa4c700d700 1 -- 172.21.15.47:0/620804121 shutdown_connections 2024-02-23T00:56:42.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.469+0000 7fa4c700d700 1 --2- 172.21.15.47:0/620804121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4c0102270 0x7fa4c0104660 unknown :-1 s=CLOSED pgs=2662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:42.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.469+0000 7fa4c700d700 1 --2- 172.21.15.47:0/620804121 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4c00f8d10 0x7fa4c00f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:42.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.469+0000 7fa4c700d700 1 --2- 172.21.15.47:0/620804121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c00f9670 0x7fa4c0101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:42.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.469+0000 7fa4c700d700 1 -- 172.21.15.47:0/620804121 >> 172.21.15.47:0/620804121 conn(0x7fa4c00f2820 msgr2=0x7fa4c00f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:56:42.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.469+0000 7fa4c700d700 1 -- 172.21.15.47:0/620804121 shutdown_connections 2024-02-23T00:56:42.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.469+0000 7fa4c700d700 1 -- 172.21.15.47:0/620804121 wait complete. 2024-02-23T00:56:42.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.469+0000 7fa4c700d700 1 librados: starting msgr at 2024-02-23T00:56:42.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.469+0000 7fa4c700d700 1 librados: starting objecter 2024-02-23T00:56:42.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.469+0000 7fa4c700d700 1 Processor -- start 2024-02-23T00:56:42.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.469+0000 7fa4c700d700 1 -- start start 2024-02-23T00:56:42.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.469+0000 7fa4c700d700 1 librados: setting wanted keys 2024-02-23T00:56:42.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.469+0000 7fa4c700d700 1 librados: calling monclient init 2024-02-23T00:56:42.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.469+0000 7fa4c700d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4c00f8d10 0x7fa4c00f6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:42.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.469+0000 7fa4c700d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c00f9670 0x7fa4c00f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:42.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.469+0000 7fa4c700d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4c0102270 0x7fa4c00f76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:42.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.469+0000 7fa4c700d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c0116290 con 0x7fa4c0102270 2024-02-23T00:56:42.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.469+0000 7fa4c700d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c0116400 con 0x7fa4c00f9670 2024-02-23T00:56:42.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.469+0000 7fa4c700d700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c01165c0 con 0x7fa4c00f8d10 2024-02-23T00:56:42.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.470+0000 7fa4c4da9700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4c00f8d10 0x7fa4c00f6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:42.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.470+0000 7fa4c55aa700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4c0102270 0x7fa4c00f76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:42.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.470+0000 7fa4c4da9700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4c00f8d10 0x7fa4c00f6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:46554/0 (socket says 172.21.15.47:46554) 2024-02-23T00:56:42.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.470+0000 7fa4c4da9700 1 -- 172.21.15.47:0/3703023945 learned_addr learned my addr 172.21.15.47:0/3703023945 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:56:42.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.470+0000 7fa4bffff700 1 --2- 172.21.15.47:0/3703023945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c00f9670 0x7fa4c00f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:42.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.470+0000 7fa4c4da9700 1 -- 172.21.15.47:0/3703023945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c00f9670 msgr2=0x7fa4c00f71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:42.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.470+0000 7fa4c4da9700 1 --2- 172.21.15.47:0/3703023945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c00f9670 0x7fa4c00f71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:42.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.470+0000 7fa4c4da9700 1 -- 172.21.15.47:0/3703023945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4c0102270 msgr2=0x7fa4c00f76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:42.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.470+0000 7fa4c4da9700 1 --2- 172.21.15.47:0/3703023945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4c0102270 0x7fa4c00f76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:42.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.470+0000 7fa4c4da9700 1 -- 172.21.15.47:0/3703023945 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b00097c0 con 0x7fa4c00f8d10 2024-02-23T00:56:42.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.470+0000 7fa4c55aa700 1 --2- 172.21.15.47:0/3703023945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4c0102270 0x7fa4c00f76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:56:42.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.470+0000 7fa4bffff700 1 --2- 172.21.15.47:0/3703023945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c00f9670 0x7fa4c00f71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:56:42.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.470+0000 7fa4c4da9700 1 --2- 172.21.15.47:0/3703023945 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4c00f8d10 0x7fa4c00f6cc0 secure :-1 s=READY pgs=2840 cs=0 l=1 rev1=1 rx=0x7fa4b0001e70 tx=0x7fa4b0004c50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:42.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.470+0000 7fa4bdffb700 10 client.?.objecter ms_handle_connect 0x7fa4c00f8d10 2024-02-23T00:56:42.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.470+0000 7fa4bdffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:56:42.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.470+0000 7fa4bdffb700 1 -- 172.21.15.47:0/3703023945 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b00099f0 con 0x7fa4c00f8d10 2024-02-23T00:56:42.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.470+0000 7fa4bdffb700 1 -- 172.21.15.47:0/3703023945 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fa4b00090d0 con 0x7fa4c00f8d10 2024-02-23T00:56:42.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.470+0000 7fa4c700d700 1 -- 172.21.15.47:0/3703023945 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4b8005040 con 0x7fa4c00f8d10 2024-02-23T00:56:42.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.470+0000 7fa4bdffb700 1 -- 172.21.15.47:0/3703023945 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0007c30 con 0x7fa4c00f8d10 2024-02-23T00:56:42.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.470+0000 7fa4c700d700 10 client.42257.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:56:42.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.470+0000 7fa4c700d700 1 -- 172.21.15.47:0/3703023945 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c0116c20 con 0x7fa4c00f8d10 2024-02-23T00:56:42.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.470+0000 7fa4c700d700 1 librados: init done 2024-02-23T00:56:42.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.471+0000 7fa4c700d700 1 -- 172.21.15.47:0/3703023945 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4c005fa60 con 0x7fa4c00f8d10 2024-02-23T00:56:42.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.472+0000 7fa4bdffb700 1 -- 172.21.15.47:0/3703023945 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fa4b000fb80 con 0x7fa4c00f8d10 2024-02-23T00:56:42.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.472+0000 7fa4bdffb700 1 --2- 172.21.15.47:0/3703023945 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa4a8062cc0 0x7fa4a8065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:42.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.472+0000 7fa4bdffb700 1 -- 172.21.15.47:0/3703023945 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(829..829 src has 621..829) v4 ==== 8605+0+0 (secure 0 0 0) 0x7fa4b007dd20 con 0x7fa4c00f8d10 2024-02-23T00:56:42.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.472+0000 7fa4bdffb700 10 client.42257.objecter ms_dispatch 0x7fa4c0001550 osd_map(829..829 src has 621..829) v4 2024-02-23T00:56:42.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.472+0000 7fa4bdffb700 3 client.42257.objecter handle_osd_map got epochs [829,829] > 0 2024-02-23T00:56:42.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.472+0000 7fa4bdffb700 3 client.42257.objecter handle_osd_map decoding full epoch 829 2024-02-23T00:56:42.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.472+0000 7fa4bffff700 1 --2- 172.21.15.47:0/3703023945 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa4a8062cc0 0x7fa4a8065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:42.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.473+0000 7fa4bdffb700 20 client.42257.objecter dump_active .. 0 homeless 2024-02-23T00:56:42.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.473+0000 7fa4bdffb700 10 client.42257.objecter ms_handle_connect 0x7fa4a8062cc0 2024-02-23T00:56:42.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.473+0000 7fa4bffff700 1 --2- 172.21.15.47:0/3703023945 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa4a8062cc0 0x7fa4a8065180 secure :-1 s=READY pgs=3226 cs=0 l=1 rev1=1 rx=0x7fa4c00f8460 tx=0x7fa4b4009d60).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:42.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.477+0000 7fa4bdffb700 1 -- 172.21.15.47:0/3703023945 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fa4b004d620 con 0x7fa4c00f8d10 2024-02-23T00:56:42.522 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:56:42.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.678+0000 7fa4c700d700 1 -- 172.21.15.47:0/3703023945 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd reweight", "id": 0, "weight": 1.0} v 0) v1 -- 0x7fa4c0107080 con 0x7fa4c00f8d10 2024-02-23T00:56:42.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.832+0000 7fa4bdffb700 1 -- 172.21.15.47:0/3703023945 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 0, "weight": 1.0}]=0 reweighted osd.0 to 1 (10000) v830) v1 ==== 113+0+0 (secure 0 0 0) 0x7fa4b00191b0 con 0x7fa4c00f8d10 2024-02-23T00:56:42.837 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:56:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:56:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:56:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.836+0000 7fa4c700d700 10 librados: watch_flush enter 2024-02-23T00:56:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.836+0000 7fa4c700d700 10 librados: watch_flush exit 2024-02-23T00:56:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.836+0000 7fa4c700d700 20 client.42257.objecter shutdown clearing up homeless session... 2024-02-23T00:56:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.836+0000 7fa4c700d700 10 client.42257.objecter successfully canceled tick 2024-02-23T00:56:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.836+0000 7fa4c700d700 1 -- 172.21.15.47:0/3703023945 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa4a8062cc0 msgr2=0x7fa4a8065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.836+0000 7fa4c700d700 1 --2- 172.21.15.47:0/3703023945 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa4a8062cc0 0x7fa4a8065180 secure :-1 s=READY pgs=3226 cs=0 l=1 rev1=1 rx=0x7fa4c00f8460 tx=0x7fa4b4009d60).stop 2024-02-23T00:56:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.836+0000 7fa4c700d700 1 -- 172.21.15.47:0/3703023945 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4c00f8d10 msgr2=0x7fa4c00f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.836+0000 7fa4c700d700 1 --2- 172.21.15.47:0/3703023945 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4c00f8d10 0x7fa4c00f6cc0 secure :-1 s=READY pgs=2840 cs=0 l=1 rev1=1 rx=0x7fa4b0001e70 tx=0x7fa4b0004c50).stop 2024-02-23T00:56:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.836+0000 7fa4c700d700 1 -- 172.21.15.47:0/3703023945 shutdown_connections 2024-02-23T00:56:42.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.836+0000 7fa4c700d700 1 --2- 172.21.15.47:0/3703023945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4c0102270 0x7fa4c00f76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:42.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.836+0000 7fa4c700d700 1 --2- 172.21.15.47:0/3703023945 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa4a8062cc0 0x7fa4a8065180 unknown :-1 s=CLOSED pgs=3226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:42.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.836+0000 7fa4c700d700 1 --2- 172.21.15.47:0/3703023945 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4c00f8d10 0x7fa4c00f6cc0 unknown :-1 s=CLOSED pgs=2840 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:42.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.836+0000 7fa4c700d700 1 --2- 172.21.15.47:0/3703023945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c00f9670 0x7fa4c00f71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:42.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.836+0000 7fa4c700d700 1 -- 172.21.15.47:0/3703023945 >> 172.21.15.47:0/3703023945 conn(0x7fa4c00f2820 msgr2=0x7fa4c00f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:56:42.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.836+0000 7fa4c700d700 1 -- 172.21.15.47:0/3703023945 shutdown_connections 2024-02-23T00:56:42.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.836+0000 7fa4c700d700 1 -- 172.21.15.47:0/3703023945 wait complete. 2024-02-23T00:56:42.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:42.836+0000 7fa4c700d700 1 librados: shutdown 2024-02-23T00:56:42.840 INFO:teuthology.orchestra.run.smithi047.stderr:reweighted osd.0 to 1 (10000) 2024-02-23T00:56:43.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:40.979602+0000 osd.7 (osd.7) 377 : cluster [DBG] 2.9 scrub starts 2024-02-23T00:56:43.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:40.985905+0000 osd.7 (osd.7) 378 : cluster [DBG] 2.9 scrub ok 2024-02-23T00:56:43.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:41.502833+0000 osd.4 (osd.4) 357 : cluster [DBG] 4.1b scrub starts 2024-02-23T00:56:43.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:41.504663+0000 osd.4 (osd.4) 358 : cluster [DBG] 4.1b scrub ok 2024-02-23T00:56:43.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:42.679880+0000 mon.c (mon.2) 525 : audit [INF] from='client.? 172.21.15.47:0/3703023945' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-23T00:56:43.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:42.680185+0000 mon.a (mon.0) 3870 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-23T00:56:43.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:40.979602+0000 osd.7 (osd.7) 377 : cluster [DBG] 2.9 scrub starts 2024-02-23T00:56:43.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:40.985905+0000 osd.7 (osd.7) 378 : cluster [DBG] 2.9 scrub ok 2024-02-23T00:56:43.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:41.502833+0000 osd.4 (osd.4) 357 : cluster [DBG] 4.1b scrub starts 2024-02-23T00:56:43.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:41.504663+0000 osd.4 (osd.4) 358 : cluster [DBG] 4.1b scrub ok 2024-02-23T00:56:43.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:42.679880+0000 mon.c (mon.2) 525 : audit [INF] from='client.? 172.21.15.47:0/3703023945' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-23T00:56:43.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:42.680185+0000 mon.a (mon.0) 3870 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-23T00:56:43.194 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:56:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:56:43.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:40.979602+0000 osd.7 (osd.7) 377 : cluster [DBG] 2.9 scrub starts 2024-02-23T00:56:43.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:40.985905+0000 osd.7 (osd.7) 378 : cluster [DBG] 2.9 scrub ok 2024-02-23T00:56:43.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:41.502833+0000 osd.4 (osd.4) 357 : cluster [DBG] 4.1b scrub starts 2024-02-23T00:56:43.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:41.504663+0000 osd.4 (osd.4) 358 : cluster [DBG] 4.1b scrub ok 2024-02-23T00:56:43.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:42.679880+0000 mon.c (mon.2) 525 : audit [INF] from='client.? 172.21.15.47:0/3703023945' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-23T00:56:43.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:42.680185+0000 mon.a (mon.0) 3870 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-23T00:56:43.976 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:42.006226+0000 osd.7 (osd.7) 379 : cluster [DBG] 2.8 deep-scrub starts 2024-02-23T00:56:43.976 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:42.009182+0000 osd.7 (osd.7) 380 : cluster [DBG] 2.8 deep-scrub ok 2024-02-23T00:56:43.976 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:42.514642+0000 osd.4 (osd.4) 359 : cluster [DBG] 5.0 scrub starts 2024-02-23T00:56:43.976 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:42.515825+0000 osd.4 (osd.4) 360 : cluster [DBG] 5.0 scrub ok 2024-02-23T00:56:43.976 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:42.832895+0000 mon.a (mon.0) 3871 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-02-23T00:56:43.976 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:42.832963+0000 mon.a (mon.0) 3872 : cluster [DBG] osdmap e830: 8 total, 8 up, 8 in 2024-02-23T00:56:43.976 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:42.994413+0000 osd.6 (osd.6) 337 : cluster [DBG] 4.5 scrub starts 2024-02-23T00:56:43.977 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:42.997057+0000 osd.6 (osd.6) 338 : cluster [DBG] 4.5 scrub ok 2024-02-23T00:56:43.977 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:43.029552+0000 mgr.y (mgr.14517) 705 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:56:43.977 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:43.108291+0000 osd.0 (osd.0) 312 : cluster [DBG] 18.8 scrub starts 2024-02-23T00:56:43.977 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:43.109208+0000 osd.0 (osd.0) 313 : cluster [DBG] 18.8 scrub ok 2024-02-23T00:56:43.977 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:43.176747+0000 osd.2 (osd.2) 193 : cluster [DBG] 5.6 scrub starts 2024-02-23T00:56:43.977 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:43.177768+0000 osd.2 (osd.2) 194 : cluster [DBG] 5.6 scrub ok 2024-02-23T00:56:43.977 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:43.418810+0000 mgr.y (mgr.14517) 706 : cluster [DBG] pgmap v1271: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 2 B/s, 0 objects/s recovering 2024-02-23T00:56:43.977 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:42.006226+0000 osd.7 (osd.7) 379 : cluster [DBG] 2.8 deep-scrub starts 2024-02-23T00:56:43.977 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:42.009182+0000 osd.7 (osd.7) 380 : cluster [DBG] 2.8 deep-scrub ok 2024-02-23T00:56:43.977 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:42.514642+0000 osd.4 (osd.4) 359 : cluster [DBG] 5.0 scrub starts 2024-02-23T00:56:43.977 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:42.515825+0000 osd.4 (osd.4) 360 : cluster [DBG] 5.0 scrub ok 2024-02-23T00:56:43.977 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:42.832895+0000 mon.a (mon.0) 3871 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-02-23T00:56:43.978 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:42.832963+0000 mon.a (mon.0) 3872 : cluster [DBG] osdmap e830: 8 total, 8 up, 8 in 2024-02-23T00:56:43.978 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:42.994413+0000 osd.6 (osd.6) 337 : cluster [DBG] 4.5 scrub starts 2024-02-23T00:56:43.978 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:42.997057+0000 osd.6 (osd.6) 338 : cluster [DBG] 4.5 scrub ok 2024-02-23T00:56:43.978 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:43.029552+0000 mgr.y (mgr.14517) 705 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:56:43.978 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:43.108291+0000 osd.0 (osd.0) 312 : cluster [DBG] 18.8 scrub starts 2024-02-23T00:56:43.978 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:43.109208+0000 osd.0 (osd.0) 313 : cluster [DBG] 18.8 scrub ok 2024-02-23T00:56:43.978 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:43.176747+0000 osd.2 (osd.2) 193 : cluster [DBG] 5.6 scrub starts 2024-02-23T00:56:43.978 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:43.177768+0000 osd.2 (osd.2) 194 : cluster [DBG] 5.6 scrub ok 2024-02-23T00:56:43.978 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:43.418810+0000 mgr.y (mgr.14517) 706 : cluster [DBG] pgmap v1271: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 2 B/s, 0 objects/s recovering 2024-02-23T00:56:43.998 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd primary-affinity 0 1 2024-02-23T00:56:44.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:42.006226+0000 osd.7 (osd.7) 379 : cluster [DBG] 2.8 deep-scrub starts 2024-02-23T00:56:44.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:42.009182+0000 osd.7 (osd.7) 380 : cluster [DBG] 2.8 deep-scrub ok 2024-02-23T00:56:44.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:42.514642+0000 osd.4 (osd.4) 359 : cluster [DBG] 5.0 scrub starts 2024-02-23T00:56:44.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:42.515825+0000 osd.4 (osd.4) 360 : cluster [DBG] 5.0 scrub ok 2024-02-23T00:56:44.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:42.832895+0000 mon.a (mon.0) 3871 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-02-23T00:56:44.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:42.832963+0000 mon.a (mon.0) 3872 : cluster [DBG] osdmap e830: 8 total, 8 up, 8 in 2024-02-23T00:56:44.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:42.994413+0000 osd.6 (osd.6) 337 : cluster [DBG] 4.5 scrub starts 2024-02-23T00:56:44.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:42.997057+0000 osd.6 (osd.6) 338 : cluster [DBG] 4.5 scrub ok 2024-02-23T00:56:44.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:43.029552+0000 mgr.y (mgr.14517) 705 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:56:44.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:43.108291+0000 osd.0 (osd.0) 312 : cluster [DBG] 18.8 scrub starts 2024-02-23T00:56:44.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:43.109208+0000 osd.0 (osd.0) 313 : cluster [DBG] 18.8 scrub ok 2024-02-23T00:56:44.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:43.176747+0000 osd.2 (osd.2) 193 : cluster [DBG] 5.6 scrub starts 2024-02-23T00:56:44.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:43.177768+0000 osd.2 (osd.2) 194 : cluster [DBG] 5.6 scrub ok 2024-02-23T00:56:44.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:43.418810+0000 mgr.y (mgr.14517) 706 : cluster [DBG] pgmap v1271: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 2 B/s, 0 objects/s recovering 2024-02-23T00:56:44.649 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:56:45.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:42.969571+0000 osd.7 (osd.7) 381 : cluster [DBG] 5.14 scrub starts 2024-02-23T00:56:45.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:42.973024+0000 osd.7 (osd.7) 382 : cluster [DBG] 5.14 scrub ok 2024-02-23T00:56:45.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:43.527120+0000 osd.4 (osd.4) 361 : cluster [DBG] 5.25 scrub starts 2024-02-23T00:56:45.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:43.528180+0000 osd.4 (osd.4) 362 : cluster [DBG] 5.25 scrub ok 2024-02-23T00:56:45.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:43.835209+0000 mon.a (mon.0) 3873 : cluster [INF] Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-02-23T00:56:45.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:43.964010+0000 osd.6 (osd.6) 339 : cluster [DBG] 2.19 scrub starts 2024-02-23T00:56:45.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:43.965007+0000 osd.6 (osd.6) 340 : cluster [DBG] 2.19 scrub ok 2024-02-23T00:56:45.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:44.750314+0000 mon.a (mon.0) 3874 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2024-02-23T00:56:45.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:44.750503+0000 mon.a (mon.0) 3875 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-02-23T00:56:45.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:42.969571+0000 osd.7 (osd.7) 381 : cluster [DBG] 5.14 scrub starts 2024-02-23T00:56:45.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:42.973024+0000 osd.7 (osd.7) 382 : cluster [DBG] 5.14 scrub ok 2024-02-23T00:56:45.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:43.527120+0000 osd.4 (osd.4) 361 : cluster [DBG] 5.25 scrub starts 2024-02-23T00:56:45.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:43.528180+0000 osd.4 (osd.4) 362 : cluster [DBG] 5.25 scrub ok 2024-02-23T00:56:45.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:43.835209+0000 mon.a (mon.0) 3873 : cluster [INF] Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-02-23T00:56:45.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:43.964010+0000 osd.6 (osd.6) 339 : cluster [DBG] 2.19 scrub starts 2024-02-23T00:56:45.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:43.965007+0000 osd.6 (osd.6) 340 : cluster [DBG] 2.19 scrub ok 2024-02-23T00:56:45.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:44.750314+0000 mon.a (mon.0) 3874 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2024-02-23T00:56:45.059 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:44.750503+0000 mon.a (mon.0) 3875 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-02-23T00:56:45.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:42.969571+0000 osd.7 (osd.7) 381 : cluster [DBG] 5.14 scrub starts 2024-02-23T00:56:45.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:42.973024+0000 osd.7 (osd.7) 382 : cluster [DBG] 5.14 scrub ok 2024-02-23T00:56:45.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:43.527120+0000 osd.4 (osd.4) 361 : cluster [DBG] 5.25 scrub starts 2024-02-23T00:56:45.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:43.528180+0000 osd.4 (osd.4) 362 : cluster [DBG] 5.25 scrub ok 2024-02-23T00:56:45.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:43.835209+0000 mon.a (mon.0) 3873 : cluster [INF] Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-02-23T00:56:45.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:43.964010+0000 osd.6 (osd.6) 339 : cluster [DBG] 2.19 scrub starts 2024-02-23T00:56:45.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:43.965007+0000 osd.6 (osd.6) 340 : cluster [DBG] 2.19 scrub ok 2024-02-23T00:56:45.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:44.750314+0000 mon.a (mon.0) 3874 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]: dispatch 2024-02-23T00:56:45.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:44.750503+0000 mon.a (mon.0) 3875 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-02-23T00:56:46.070 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:43.921074+0000 osd.7 (osd.7) 383 : cluster [DBG] 6.8 scrub starts 2024-02-23T00:56:46.070 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:43.924142+0000 osd.7 (osd.7) 384 : cluster [DBG] 6.8 scrub ok 2024-02-23T00:56:46.070 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:44.518768+0000 osd.4 (osd.4) 363 : cluster [DBG] 4.1a deep-scrub starts 2024-02-23T00:56:46.070 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:44.520781+0000 osd.4 (osd.4) 364 : cluster [DBG] 4.1a deep-scrub ok 2024-02-23T00:56:46.071 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:44.845946+0000 mon.a (mon.0) 3876 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]': finished 2024-02-23T00:56:46.071 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:44.846030+0000 mon.a (mon.0) 3877 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2024-02-23T00:56:46.071 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:44.846094+0000 mon.a (mon.0) 3878 : cluster [DBG] osdmap e831: 8 total, 8 up, 8 in 2024-02-23T00:56:46.071 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:44.936521+0000 osd.7 (osd.7) 385 : cluster [DBG] 2.0 scrub starts 2024-02-23T00:56:46.071 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:44.939543+0000 osd.7 (osd.7) 386 : cluster [DBG] 2.0 scrub ok 2024-02-23T00:56:46.071 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:45.337233+0000 mon.a (mon.0) 3879 : cluster [DBG] osdmap e832: 8 total, 8 up, 8 in 2024-02-23T00:56:46.071 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:45.419194+0000 mgr.y (mgr.14517) 707 : cluster [DBG] pgmap v1274: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 2 B/s, 0 objects/s recovering 2024-02-23T00:56:46.071 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:43.921074+0000 osd.7 (osd.7) 383 : cluster [DBG] 6.8 scrub starts 2024-02-23T00:56:46.071 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:43.924142+0000 osd.7 (osd.7) 384 : cluster [DBG] 6.8 scrub ok 2024-02-23T00:56:46.071 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:44.518768+0000 osd.4 (osd.4) 363 : cluster [DBG] 4.1a deep-scrub starts 2024-02-23T00:56:46.071 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:44.520781+0000 osd.4 (osd.4) 364 : cluster [DBG] 4.1a deep-scrub ok 2024-02-23T00:56:46.072 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:44.845946+0000 mon.a (mon.0) 3876 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]': finished 2024-02-23T00:56:46.072 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:44.846030+0000 mon.a (mon.0) 3877 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2024-02-23T00:56:46.072 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:44.846094+0000 mon.a (mon.0) 3878 : cluster [DBG] osdmap e831: 8 total, 8 up, 8 in 2024-02-23T00:56:46.072 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:44.936521+0000 osd.7 (osd.7) 385 : cluster [DBG] 2.0 scrub starts 2024-02-23T00:56:46.072 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:44.939543+0000 osd.7 (osd.7) 386 : cluster [DBG] 2.0 scrub ok 2024-02-23T00:56:46.072 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:45.337233+0000 mon.a (mon.0) 3879 : cluster [DBG] osdmap e832: 8 total, 8 up, 8 in 2024-02-23T00:56:46.072 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:45.419194+0000 mgr.y (mgr.14517) 707 : cluster [DBG] pgmap v1274: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 2 B/s, 0 objects/s recovering 2024-02-23T00:56:46.167 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:56:46.167 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:56:46.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:43.921074+0000 osd.7 (osd.7) 383 : cluster [DBG] 6.8 scrub starts 2024-02-23T00:56:46.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:43.924142+0000 osd.7 (osd.7) 384 : cluster [DBG] 6.8 scrub ok 2024-02-23T00:56:46.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:44.518768+0000 osd.4 (osd.4) 363 : cluster [DBG] 4.1a deep-scrub starts 2024-02-23T00:56:46.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:44.520781+0000 osd.4 (osd.4) 364 : cluster [DBG] 4.1a deep-scrub ok 2024-02-23T00:56:46.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:44.845946+0000 mon.a (mon.0) 3876 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 2]}]': finished 2024-02-23T00:56:46.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:44.846030+0000 mon.a (mon.0) 3877 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2024-02-23T00:56:46.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:44.846094+0000 mon.a (mon.0) 3878 : cluster [DBG] osdmap e831: 8 total, 8 up, 8 in 2024-02-23T00:56:46.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:44.936521+0000 osd.7 (osd.7) 385 : cluster [DBG] 2.0 scrub starts 2024-02-23T00:56:46.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:44.939543+0000 osd.7 (osd.7) 386 : cluster [DBG] 2.0 scrub ok 2024-02-23T00:56:46.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:45.337233+0000 mon.a (mon.0) 3879 : cluster [DBG] osdmap e832: 8 total, 8 up, 8 in 2024-02-23T00:56:46.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:45.419194+0000 mgr.y (mgr.14517) 707 : cluster [DBG] pgmap v1274: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 2 B/s, 0 objects/s recovering 2024-02-23T00:56:46.826 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.3 dump_ops_in_flight 2024-02-23T00:56:47.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:45.505444+0000 osd.4 (osd.4) 365 : cluster [DBG] 5.23 scrub starts 2024-02-23T00:56:47.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:45.507109+0000 osd.4 (osd.4) 366 : cluster [DBG] 5.23 scrub ok 2024-02-23T00:56:47.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:45.942365+0000 osd.6 (osd.6) 341 : cluster [DBG] 5.2 scrub starts 2024-02-23T00:56:47.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:45.943541+0000 osd.6 (osd.6) 342 : cluster [DBG] 5.2 scrub ok 2024-02-23T00:56:47.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:46.113773+0000 osd.2 (osd.2) 195 : cluster [DBG] 2.1b scrub starts 2024-02-23T00:56:47.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:46.114945+0000 osd.2 (osd.2) 196 : cluster [DBG] 2.1b scrub ok 2024-02-23T00:56:47.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:46.158250+0000 osd.0 (osd.0) 314 : cluster [DBG] 18.1c scrub starts 2024-02-23T00:56:47.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:46.159459+0000 osd.0 (osd.0) 315 : cluster [DBG] 18.1c scrub ok 2024-02-23T00:56:47.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.055+0000 7efea05cc700 1 -- 172.21.15.47:0/3464237113 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efe980f96e0 msgr2=0x7efe98101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:47.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.055+0000 7efea05cc700 1 --2- 172.21.15.47:0/3464237113 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efe980f96e0 0x7efe98101c30 secure :-1 s=READY pgs=2841 cs=0 l=1 rev1=1 rx=0x7efe88004870 tx=0x7efe88009cb0).stop 2024-02-23T00:56:47.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.055+0000 7efea05cc700 1 -- 172.21.15.47:0/3464237113 shutdown_connections 2024-02-23T00:56:47.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.055+0000 7efea05cc700 1 --2- 172.21.15.47:0/3464237113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe98102250 0x7efe98104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:47.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.055+0000 7efea05cc700 1 --2- 172.21.15.47:0/3464237113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe980f8d80 0x7efe980f9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:56:47.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.055+0000 7efea05cc700 1 --2- 172.21.15.47:0/3464237113 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efe980f96e0 0x7efe98101c30 unknown :-1 s=CLOSED pgs=2841 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:47.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.055+0000 7efea05cc700 1 -- 172.21.15.47:0/3464237113 >> 172.21.15.47:0/3464237113 conn(0x7efe980f26f0 msgr2=0x7efe980f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:56:47.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.056+0000 7efea05cc700 1 -- 172.21.15.47:0/3464237113 shutdown_connections 2024-02-23T00:56:47.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.056+0000 7efea05cc700 1 -- 172.21.15.47:0/3464237113 wait complete. 2024-02-23T00:56:47.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.056+0000 7efea05cc700 1 librados: starting msgr at 2024-02-23T00:56:47.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.056+0000 7efea05cc700 1 librados: starting objecter 2024-02-23T00:56:47.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.056+0000 7efea05cc700 1 Processor -- start 2024-02-23T00:56:47.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.056+0000 7efea05cc700 1 -- start start 2024-02-23T00:56:47.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.056+0000 7efea05cc700 1 librados: setting wanted keys 2024-02-23T00:56:47.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.057+0000 7efea05cc700 1 librados: calling monclient init 2024-02-23T00:56:47.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.057+0000 7efea05cc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efe980f96e0 0x7efe980f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:47.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.057+0000 7efea05cc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe98102250 0x7efe980f7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:47.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.057+0000 7efea05cc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe980f7550 0x7efe98116780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:47.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.057+0000 7efea05cc700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe98116c70 con 0x7efe980f7550 2024-02-23T00:56:47.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.057+0000 7efea05cc700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe98116db0 con 0x7efe98102250 2024-02-23T00:56:47.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.057+0000 7efea05cc700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe98116f20 con 0x7efe980f96e0 2024-02-23T00:56:47.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.057+0000 7efe9eb69700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe980f7550 0x7efe98116780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:47.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.057+0000 7efe9e368700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efe980f96e0 0x7efe980f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:47.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.057+0000 7efe9eb69700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe980f7550 0x7efe98116780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54618/0 (socket says 172.21.15.47:54618) 2024-02-23T00:56:47.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.057+0000 7efe9eb69700 1 -- 172.21.15.47:0/949639026 learned_addr learned my addr 172.21.15.47:0/949639026 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:56:47.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.057+0000 7efe9db67700 1 --2- 172.21.15.47:0/949639026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe98102250 0x7efe980f7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:47.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.057+0000 7efe9eb69700 1 -- 172.21.15.47:0/949639026 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efe980f96e0 msgr2=0x7efe980f6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:47.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.057+0000 7efe9eb69700 1 --2- 172.21.15.47:0/949639026 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efe980f96e0 0x7efe980f6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:47.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.057+0000 7efe9eb69700 1 -- 172.21.15.47:0/949639026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe98102250 msgr2=0x7efe980f7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:47.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.057+0000 7efe9eb69700 1 --2- 172.21.15.47:0/949639026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe98102250 0x7efe980f7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:47.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.057+0000 7efe9eb69700 1 -- 172.21.15.47:0/949639026 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe88005040 con 0x7efe980f7550 2024-02-23T00:56:47.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.057+0000 7efe9e368700 1 --2- 172.21.15.47:0/949639026 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efe980f96e0 0x7efe980f6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:56:47.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.058+0000 7efe9eb69700 1 --2- 172.21.15.47:0/949639026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe980f7550 0x7efe98116780 secure :-1 s=READY pgs=2663 cs=0 l=1 rev1=1 rx=0x7efe9000ec20 tx=0x7efe9000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:47.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.058+0000 7efe8f7fe700 10 client.?.objecter ms_handle_connect 0x7efe980f7550 2024-02-23T00:56:47.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.058+0000 7efe8f7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:56:47.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.058+0000 7efe8f7fe700 1 -- 172.21.15.47:0/949639026 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe90009950 con 0x7efe980f7550 2024-02-23T00:56:47.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.058+0000 7efe8f7fe700 1 -- 172.21.15.47:0/949639026 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7efe9000f040 con 0x7efe980f7550 2024-02-23T00:56:47.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.058+0000 7efea05cc700 1 -- 172.21.15.47:0/949639026 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe981171a0 con 0x7efe980f7550 2024-02-23T00:56:47.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.058+0000 7efe8f7fe700 1 -- 172.21.15.47:0/949639026 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe90007d10 con 0x7efe980f7550 2024-02-23T00:56:47.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.058+0000 7efea05cc700 10 client.31941.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:56:47.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.058+0000 7efea05cc700 1 -- 172.21.15.47:0/949639026 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe981176e0 con 0x7efe980f7550 2024-02-23T00:56:47.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.058+0000 7efea05cc700 1 librados: init done 2024-02-23T00:56:47.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:45.505444+0000 osd.4 (osd.4) 365 : cluster [DBG] 5.23 scrub starts 2024-02-23T00:56:47.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:45.507109+0000 osd.4 (osd.4) 366 : cluster [DBG] 5.23 scrub ok 2024-02-23T00:56:47.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:45.942365+0000 osd.6 (osd.6) 341 : cluster [DBG] 5.2 scrub starts 2024-02-23T00:56:47.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:45.943541+0000 osd.6 (osd.6) 342 : cluster [DBG] 5.2 scrub ok 2024-02-23T00:56:47.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:46.113773+0000 osd.2 (osd.2) 195 : cluster [DBG] 2.1b scrub starts 2024-02-23T00:56:47.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:46.114945+0000 osd.2 (osd.2) 196 : cluster [DBG] 2.1b scrub ok 2024-02-23T00:56:47.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:46.158250+0000 osd.0 (osd.0) 314 : cluster [DBG] 18.1c scrub starts 2024-02-23T00:56:47.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:46.159459+0000 osd.0 (osd.0) 315 : cluster [DBG] 18.1c scrub ok 2024-02-23T00:56:47.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.059+0000 7efea05cc700 1 -- 172.21.15.47:0/949639026 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe7c004f50 con 0x7efe980f7550 2024-02-23T00:56:47.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.060+0000 7efe8f7fe700 1 -- 172.21.15.47:0/949639026 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7efe90004190 con 0x7efe980f7550 2024-02-23T00:56:47.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.060+0000 7efe8f7fe700 1 --2- 172.21.15.47:0/949639026 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7efe84062cc0 0x7efe84065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:47.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.060+0000 7efe8f7fe700 1 -- 172.21.15.47:0/949639026 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(832..832 src has 621..832) v4 ==== 8663+0+0 (secure 0 0 0) 0x7efe9007d030 con 0x7efe980f7550 2024-02-23T00:56:47.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.060+0000 7efe8f7fe700 10 client.31941.objecter ms_dispatch 0x7efe98001550 osd_map(832..832 src has 621..832) v4 2024-02-23T00:56:47.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.060+0000 7efe8f7fe700 3 client.31941.objecter handle_osd_map got epochs [832,832] > 0 2024-02-23T00:56:47.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.060+0000 7efe8f7fe700 3 client.31941.objecter handle_osd_map decoding full epoch 832 2024-02-23T00:56:47.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.060+0000 7efe9e368700 1 --2- 172.21.15.47:0/949639026 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7efe84062cc0 0x7efe84065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:47.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.060+0000 7efe8f7fe700 20 client.31941.objecter dump_active .. 0 homeless 2024-02-23T00:56:47.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.061+0000 7efe8f7fe700 10 client.31941.objecter ms_handle_connect 0x7efe84062cc0 2024-02-23T00:56:47.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.061+0000 7efe9e368700 1 --2- 172.21.15.47:0/949639026 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7efe84062cc0 0x7efe84065180 secure :-1 s=READY pgs=3227 cs=0 l=1 rev1=1 rx=0x7efe98060f40 tx=0x7efe94006f90).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:47.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.065+0000 7efe8f7fe700 1 -- 172.21.15.47:0/949639026 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7efe90049f60 con 0x7efe980f7550 2024-02-23T00:56:47.131 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:56:47.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:45.505444+0000 osd.4 (osd.4) 365 : cluster [DBG] 5.23 scrub starts 2024-02-23T00:56:47.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:45.507109+0000 osd.4 (osd.4) 366 : cluster [DBG] 5.23 scrub ok 2024-02-23T00:56:47.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:45.942365+0000 osd.6 (osd.6) 341 : cluster [DBG] 5.2 scrub starts 2024-02-23T00:56:47.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:45.943541+0000 osd.6 (osd.6) 342 : cluster [DBG] 5.2 scrub ok 2024-02-23T00:56:47.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:46.113773+0000 osd.2 (osd.2) 195 : cluster [DBG] 2.1b scrub starts 2024-02-23T00:56:47.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:46.114945+0000 osd.2 (osd.2) 196 : cluster [DBG] 2.1b scrub ok 2024-02-23T00:56:47.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:46.158250+0000 osd.0 (osd.0) 314 : cluster [DBG] 18.1c scrub starts 2024-02-23T00:56:47.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:46.159459+0000 osd.0 (osd.0) 315 : cluster [DBG] 18.1c scrub ok 2024-02-23T00:56:47.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.273+0000 7efea05cc700 1 -- 172.21.15.47:0/949639026 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} v 0) v1 -- 0x7efe7c005780 con 0x7efe980f7550 2024-02-23T00:56:47.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.856+0000 7efe8f7fe700 1 -- 172.21.15.47:0/949639026 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]=0 set osd.0 primary-affinity to 1 (8655362) v833) v1 ==== 133+0+0 (secure 0 0 0) 0x7efe9004c8f0 con 0x7efe980f7550 2024-02-23T00:56:47.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.858+0000 7efea05cc700 10 librados: watch_flush enter 2024-02-23T00:56:47.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.858+0000 7efea05cc700 10 librados: watch_flush exit 2024-02-23T00:56:47.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.858+0000 7efea05cc700 20 client.31941.objecter shutdown clearing up homeless session... 2024-02-23T00:56:47.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.858+0000 7efea05cc700 10 client.31941.objecter successfully canceled tick 2024-02-23T00:56:47.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.858+0000 7efea05cc700 1 -- 172.21.15.47:0/949639026 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7efe84062cc0 msgr2=0x7efe84065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:47.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.858+0000 7efea05cc700 1 --2- 172.21.15.47:0/949639026 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7efe84062cc0 0x7efe84065180 secure :-1 s=READY pgs=3227 cs=0 l=1 rev1=1 rx=0x7efe98060f40 tx=0x7efe94006f90).stop 2024-02-23T00:56:47.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.858+0000 7efea05cc700 1 -- 172.21.15.47:0/949639026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe980f7550 msgr2=0x7efe98116780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:47.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.858+0000 7efea05cc700 1 --2- 172.21.15.47:0/949639026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe980f7550 0x7efe98116780 secure :-1 s=READY pgs=2663 cs=0 l=1 rev1=1 rx=0x7efe9000ec20 tx=0x7efe9000b040).stop 2024-02-23T00:56:47.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.861+0000 7efea05cc700 1 -- 172.21.15.47:0/949639026 shutdown_connections 2024-02-23T00:56:47.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.861+0000 7efea05cc700 1 --2- 172.21.15.47:0/949639026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe980f7550 0x7efe98116780 unknown :-1 s=CLOSED pgs=2663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:47.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.861+0000 7efea05cc700 1 --2- 172.21.15.47:0/949639026 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7efe84062cc0 0x7efe84065180 unknown :-1 s=CLOSED pgs=3227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:47.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.861+0000 7efea05cc700 1 --2- 172.21.15.47:0/949639026 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efe980f96e0 0x7efe980f6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:47.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.861+0000 7efea05cc700 1 --2- 172.21.15.47:0/949639026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe98102250 0x7efe980f7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:47.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.861+0000 7efea05cc700 1 -- 172.21.15.47:0/949639026 >> 172.21.15.47:0/949639026 conn(0x7efe980f26f0 msgr2=0x7efe980f4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:56:47.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.861+0000 7efea05cc700 1 -- 172.21.15.47:0/949639026 shutdown_connections 2024-02-23T00:56:47.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.861+0000 7efea05cc700 1 -- 172.21.15.47:0/949639026 wait complete. 2024-02-23T00:56:47.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:47.861+0000 7efea05cc700 1 librados: shutdown 2024-02-23T00:56:47.867 INFO:teuthology.orchestra.run.smithi047.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-02-23T00:56:48.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:46.536400+0000 osd.4 (osd.4) 367 : cluster [DBG] 4.c scrub starts 2024-02-23T00:56:48.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:46.538120+0000 osd.4 (osd.4) 368 : cluster [DBG] 4.c scrub ok 2024-02-23T00:56:48.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:47.172219+0000 osd.0 (osd.0) 316 : cluster [DBG] 2.13 scrub starts 2024-02-23T00:56:48.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:47.173023+0000 osd.0 (osd.0) 317 : cluster [DBG] 2.13 scrub ok 2024-02-23T00:56:48.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:47.275028+0000 mon.a (mon.0) 3880 : audit [INF] from='client.? 172.21.15.47:0/949639026' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-02-23T00:56:48.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:47.419913+0000 mgr.y (mgr.14517) 708 : cluster [DBG] pgmap v1275: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 112 B/s, 2 objects/s recovering 2024-02-23T00:56:48.197 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:46.536400+0000 osd.4 (osd.4) 367 : cluster [DBG] 4.c scrub starts 2024-02-23T00:56:48.197 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:46.538120+0000 osd.4 (osd.4) 368 : cluster [DBG] 4.c scrub ok 2024-02-23T00:56:48.197 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:47.172219+0000 osd.0 (osd.0) 316 : cluster [DBG] 2.13 scrub starts 2024-02-23T00:56:48.197 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:47.173023+0000 osd.0 (osd.0) 317 : cluster [DBG] 2.13 scrub ok 2024-02-23T00:56:48.197 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:47.275028+0000 mon.a (mon.0) 3880 : audit [INF] from='client.? 172.21.15.47:0/949639026' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-02-23T00:56:48.197 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:47.419913+0000 mgr.y (mgr.14517) 708 : cluster [DBG] pgmap v1275: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 112 B/s, 2 objects/s recovering 2024-02-23T00:56:48.197 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:46.536400+0000 osd.4 (osd.4) 367 : cluster [DBG] 4.c scrub starts 2024-02-23T00:56:48.197 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:46.538120+0000 osd.4 (osd.4) 368 : cluster [DBG] 4.c scrub ok 2024-02-23T00:56:48.197 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:47.172219+0000 osd.0 (osd.0) 316 : cluster [DBG] 2.13 scrub starts 2024-02-23T00:56:48.197 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:47.173023+0000 osd.0 (osd.0) 317 : cluster [DBG] 2.13 scrub ok 2024-02-23T00:56:48.197 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:47.275028+0000 mon.a (mon.0) 3880 : audit [INF] from='client.? 172.21.15.47:0/949639026' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-02-23T00:56:48.198 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:47.419913+0000 mgr.y (mgr.14517) 708 : cluster [DBG] pgmap v1275: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 112 B/s, 2 objects/s recovering 2024-02-23T00:56:49.000 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:47.856316+0000 mon.a (mon.0) 3881 : audit [INF] from='client.? 172.21.15.47:0/949639026' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-02-23T00:56:49.000 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:47.856409+0000 mon.a (mon.0) 3882 : cluster [DBG] osdmap e833: 8 total, 8 up, 8 in 2024-02-23T00:56:49.000 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:48.041683+0000 osd.2 (osd.2) 197 : cluster [DBG] 5.d scrub starts 2024-02-23T00:56:49.000 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:48.043478+0000 osd.2 (osd.2) 198 : cluster [DBG] 5.d scrub ok 2024-02-23T00:56:49.001 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:47.856316+0000 mon.a (mon.0) 3881 : audit [INF] from='client.? 172.21.15.47:0/949639026' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-02-23T00:56:49.001 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:47.856409+0000 mon.a (mon.0) 3882 : cluster [DBG] osdmap e833: 8 total, 8 up, 8 in 2024-02-23T00:56:49.001 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:48.041683+0000 osd.2 (osd.2) 197 : cluster [DBG] 5.d scrub starts 2024-02-23T00:56:49.001 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:48.043478+0000 osd.2 (osd.2) 198 : cluster [DBG] 5.d scrub ok 2024-02-23T00:56:49.025 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight 1 1 2024-02-23T00:56:49.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:47.856316+0000 mon.a (mon.0) 3881 : audit [INF] from='client.? 172.21.15.47:0/949639026' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-02-23T00:56:49.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:47.856409+0000 mon.a (mon.0) 3882 : cluster [DBG] osdmap e833: 8 total, 8 up, 8 in 2024-02-23T00:56:49.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:48.041683+0000 osd.2 (osd.2) 197 : cluster [DBG] 5.d scrub starts 2024-02-23T00:56:49.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:48.043478+0000 osd.2 (osd.2) 198 : cluster [DBG] 5.d scrub ok 2024-02-23T00:56:49.539 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:56:49.551 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:56:49.551 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:56:50.176 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:48.897443+0000 osd.6 (osd.6) 343 : cluster [DBG] 2.7 scrub starts 2024-02-23T00:56:50.176 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:48.898409+0000 osd.6 (osd.6) 344 : cluster [DBG] 2.7 scrub ok 2024-02-23T00:56:50.177 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:48.939050+0000 osd.7 (osd.7) 387 : cluster [DBG] 5.22 scrub starts 2024-02-23T00:56:50.177 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:48.942071+0000 osd.7 (osd.7) 388 : cluster [DBG] 5.22 scrub ok 2024-02-23T00:56:50.177 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:49.161465+0000 osd.0 (osd.0) 318 : cluster [DBG] 4.16 deep-scrub starts 2024-02-23T00:56:50.177 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:49.165582+0000 osd.0 (osd.0) 319 : cluster [DBG] 4.16 deep-scrub ok 2024-02-23T00:56:50.177 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:49.420299+0000 mgr.y (mgr.14517) 709 : cluster [DBG] pgmap v1277: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 109 B/s, 2 objects/s recovering 2024-02-23T00:56:50.177 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:48.897443+0000 osd.6 (osd.6) 343 : cluster [DBG] 2.7 scrub starts 2024-02-23T00:56:50.177 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:48.898409+0000 osd.6 (osd.6) 344 : cluster [DBG] 2.7 scrub ok 2024-02-23T00:56:50.177 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:48.939050+0000 osd.7 (osd.7) 387 : cluster [DBG] 5.22 scrub starts 2024-02-23T00:56:50.177 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:48.942071+0000 osd.7 (osd.7) 388 : cluster [DBG] 5.22 scrub ok 2024-02-23T00:56:50.177 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:49.161465+0000 osd.0 (osd.0) 318 : cluster [DBG] 4.16 deep-scrub starts 2024-02-23T00:56:50.177 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:49.165582+0000 osd.0 (osd.0) 319 : cluster [DBG] 4.16 deep-scrub ok 2024-02-23T00:56:50.178 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:49.420299+0000 mgr.y (mgr.14517) 709 : cluster [DBG] pgmap v1277: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 109 B/s, 2 objects/s recovering 2024-02-23T00:56:50.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:48.897443+0000 osd.6 (osd.6) 343 : cluster [DBG] 2.7 scrub starts 2024-02-23T00:56:50.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:48.898409+0000 osd.6 (osd.6) 344 : cluster [DBG] 2.7 scrub ok 2024-02-23T00:56:50.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:48.939050+0000 osd.7 (osd.7) 387 : cluster [DBG] 5.22 scrub starts 2024-02-23T00:56:50.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:48.942071+0000 osd.7 (osd.7) 388 : cluster [DBG] 5.22 scrub ok 2024-02-23T00:56:50.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:49.161465+0000 osd.0 (osd.0) 318 : cluster [DBG] 4.16 deep-scrub starts 2024-02-23T00:56:50.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:49.165582+0000 osd.0 (osd.0) 319 : cluster [DBG] 4.16 deep-scrub ok 2024-02-23T00:56:50.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:49.420299+0000 mgr.y (mgr.14517) 709 : cluster [DBG] pgmap v1277: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 109 B/s, 2 objects/s recovering 2024-02-23T00:56:50.652 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.3 dump_blocked_ops 2024-02-23T00:56:50.807 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:56:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:56:50] "GET /metrics HTTP/1.1" 200 187131 "" "Prometheus/2.33.4" 2024-02-23T00:56:51.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:49.545437+0000 osd.4 (osd.4) 369 : cluster [DBG] 4.13 scrub starts 2024-02-23T00:56:51.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:49.547319+0000 osd.4 (osd.4) 370 : cluster [DBG] 4.13 scrub ok 2024-02-23T00:56:51.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:49.950132+0000 osd.7 (osd.7) 389 : cluster [DBG] 5.9 deep-scrub starts 2024-02-23T00:56:51.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:49.953638+0000 osd.7 (osd.7) 390 : cluster [DBG] 5.9 deep-scrub ok 2024-02-23T00:56:51.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:50.160127+0000 osd.0 (osd.0) 320 : cluster [DBG] 5.5 scrub starts 2024-02-23T00:56:51.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:50.161307+0000 osd.0 (osd.0) 321 : cluster [DBG] 5.5 scrub ok 2024-02-23T00:56:51.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:50.338608+0000 mon.a (mon.0) 3883 : cluster [DBG] osdmap e834: 8 total, 8 up, 8 in 2024-02-23T00:56:51.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:49.545437+0000 osd.4 (osd.4) 369 : cluster [DBG] 4.13 scrub starts 2024-02-23T00:56:51.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:49.547319+0000 osd.4 (osd.4) 370 : cluster [DBG] 4.13 scrub ok 2024-02-23T00:56:51.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:49.950132+0000 osd.7 (osd.7) 389 : cluster [DBG] 5.9 deep-scrub starts 2024-02-23T00:56:51.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:49.953638+0000 osd.7 (osd.7) 390 : cluster [DBG] 5.9 deep-scrub ok 2024-02-23T00:56:51.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:50.160127+0000 osd.0 (osd.0) 320 : cluster [DBG] 5.5 scrub starts 2024-02-23T00:56:51.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:50.161307+0000 osd.0 (osd.0) 321 : cluster [DBG] 5.5 scrub ok 2024-02-23T00:56:51.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:50.338608+0000 mon.a (mon.0) 3883 : cluster [DBG] osdmap e834: 8 total, 8 up, 8 in 2024-02-23T00:56:51.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:49.545437+0000 osd.4 (osd.4) 369 : cluster [DBG] 4.13 scrub starts 2024-02-23T00:56:51.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:49.547319+0000 osd.4 (osd.4) 370 : cluster [DBG] 4.13 scrub ok 2024-02-23T00:56:51.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:49.950132+0000 osd.7 (osd.7) 389 : cluster [DBG] 5.9 deep-scrub starts 2024-02-23T00:56:51.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:49.953638+0000 osd.7 (osd.7) 390 : cluster [DBG] 5.9 deep-scrub ok 2024-02-23T00:56:51.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:50.160127+0000 osd.0 (osd.0) 320 : cluster [DBG] 5.5 scrub starts 2024-02-23T00:56:51.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:50.161307+0000 osd.0 (osd.0) 321 : cluster [DBG] 5.5 scrub ok 2024-02-23T00:56:51.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:50.338608+0000 mon.a (mon.0) 3883 : cluster [DBG] osdmap e834: 8 total, 8 up, 8 in 2024-02-23T00:56:51.986 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:56:51.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.991+0000 7f629c847700 1 -- 172.21.15.47:0/4272715427 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f62980f9620 msgr2=0x7f6298101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:51.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.991+0000 7f629c847700 1 --2- 172.21.15.47:0/4272715427 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f62980f9620 0x7f6298101dc0 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 rx=0x7f628c0047b0 tx=0x7f628c009cb0).stop 2024-02-23T00:56:51.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.991+0000 7f629c847700 1 -- 172.21.15.47:0/4272715427 shutdown_connections 2024-02-23T00:56:51.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.991+0000 7f629c847700 1 --2- 172.21.15.47:0/4272715427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62981022b0 0x7f62981046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:51.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.991+0000 7f629c847700 1 --2- 172.21.15.47:0/4272715427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62980f8d50 0x7f62980f9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:51.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.991+0000 7f629c847700 1 --2- 172.21.15.47:0/4272715427 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f62980f9620 0x7f6298101dc0 unknown :-1 s=CLOSED pgs=2842 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:51.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.991+0000 7f629c847700 1 -- 172.21.15.47:0/4272715427 >> 172.21.15.47:0/4272715427 conn(0x7f62980f2820 msgr2=0x7f62980f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:56:51.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.991+0000 7f629c847700 1 -- 172.21.15.47:0/4272715427 shutdown_connections 2024-02-23T00:56:51.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.991+0000 7f629c847700 1 -- 172.21.15.47:0/4272715427 wait complete. 2024-02-23T00:56:51.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.992+0000 7f629c847700 1 librados: starting msgr at 2024-02-23T00:56:51.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.992+0000 7f629c847700 1 librados: starting objecter 2024-02-23T00:56:51.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.992+0000 7f629c847700 1 Processor -- start 2024-02-23T00:56:51.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.992+0000 7f629c847700 1 -- start start 2024-02-23T00:56:51.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.992+0000 7f629c847700 1 librados: setting wanted keys 2024-02-23T00:56:51.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.992+0000 7f629c847700 1 librados: calling monclient init 2024-02-23T00:56:51.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.992+0000 7f629c847700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f62980f8d50 0x7f62980f6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:51.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.992+0000 7f629c847700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62980f9620 0x7f62980f71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:51.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.992+0000 7f629c847700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62981022b0 0x7f62980f76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:51.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.992+0000 7f629c847700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6298116510 con 0x7f62980f9620 2024-02-23T00:56:51.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.992+0000 7f629c847700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6298116680 con 0x7f62981022b0 2024-02-23T00:56:51.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.992+0000 7f629c847700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6298116840 con 0x7f62980f8d50 2024-02-23T00:56:51.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.993+0000 7f629659c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f62980f8d50 0x7f62980f6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:51.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.993+0000 7f6295d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62980f9620 0x7f62980f71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:51.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.993+0000 7f6295d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62980f9620 0x7f62980f71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54642/0 (socket says 172.21.15.47:54642) 2024-02-23T00:56:51.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.993+0000 7f6295d9b700 1 -- 172.21.15.47:0/3862807928 learned_addr learned my addr 172.21.15.47:0/3862807928 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:56:51.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.993+0000 7f629659c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f62980f8d50 0x7f62980f6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:34002/0 (socket says 172.21.15.47:34002) 2024-02-23T00:56:51.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.993+0000 7f6296d9d700 1 --2- 172.21.15.47:0/3862807928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62981022b0 0x7f62980f76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:51.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.993+0000 7f6295d9b700 1 -- 172.21.15.47:0/3862807928 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f62980f8d50 msgr2=0x7f62980f6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:51.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.993+0000 7f6295d9b700 1 --2- 172.21.15.47:0/3862807928 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f62980f8d50 0x7f62980f6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:51.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.993+0000 7f6295d9b700 1 -- 172.21.15.47:0/3862807928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62981022b0 msgr2=0x7f62980f76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:51.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.993+0000 7f6295d9b700 1 --2- 172.21.15.47:0/3862807928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62981022b0 0x7f62980f76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:51.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.993+0000 7f6295d9b700 1 -- 172.21.15.47:0/3862807928 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6288005040 con 0x7f62980f9620 2024-02-23T00:56:51.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.993+0000 7f629659c700 1 --2- 172.21.15.47:0/3862807928 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f62980f8d50 0x7f62980f6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:56:51.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.993+0000 7f6295d9b700 1 --2- 172.21.15.47:0/3862807928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62980f9620 0x7f62980f71f0 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 rx=0x7f628c009f20 tx=0x7f628c009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:51.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.993+0000 7f62877fe700 10 client.?.objecter ms_handle_connect 0x7f62980f9620 2024-02-23T00:56:51.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.993+0000 7f62877fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:56:51.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.993+0000 7f62877fe700 1 -- 172.21.15.47:0/3862807928 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628c018070 con 0x7f62980f9620 2024-02-23T00:56:51.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.993+0000 7f62877fe700 1 -- 172.21.15.47:0/3862807928 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f628c013dd0 con 0x7f62980f9620 2024-02-23T00:56:52.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.993+0000 7f629c847700 1 -- 172.21.15.47:0/3862807928 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f628c005040 con 0x7f62980f9620 2024-02-23T00:56:52.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.994+0000 7f62877fe700 1 -- 172.21.15.47:0/3862807928 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628c01da10 con 0x7f62980f9620 2024-02-23T00:56:52.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.994+0000 7f629c847700 10 client.31947.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:56:52.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.994+0000 7f629c847700 1 -- 172.21.15.47:0/3862807928 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6298116d30 con 0x7f62980f9620 2024-02-23T00:56:52.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.994+0000 7f629c847700 1 librados: init done 2024-02-23T00:56:52.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.995+0000 7f629c847700 1 -- 172.21.15.47:0/3862807928 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f629805fa60 con 0x7f62980f9620 2024-02-23T00:56:52.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.995+0000 7f62877fe700 1 -- 172.21.15.47:0/3862807928 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f628c009240 con 0x7f62980f9620 2024-02-23T00:56:52.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.996+0000 7f62877fe700 1 --2- 172.21.15.47:0/3862807928 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6280062cc0 0x7f6280065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:52.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.996+0000 7f62877fe700 1 -- 172.21.15.47:0/3862807928 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(835..835 src has 621..835) v4 ==== 8663+0+0 (secure 0 0 0) 0x7f628c07e690 con 0x7f62980f9620 2024-02-23T00:56:52.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.996+0000 7f62877fe700 10 client.31947.objecter ms_dispatch 0x7f6298001550 osd_map(835..835 src has 621..835) v4 2024-02-23T00:56:52.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.996+0000 7f62877fe700 3 client.31947.objecter handle_osd_map got epochs [835,835] > 0 2024-02-23T00:56:52.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.996+0000 7f62877fe700 3 client.31947.objecter handle_osd_map decoding full epoch 835 2024-02-23T00:56:52.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.996+0000 7f62877fe700 20 client.31947.objecter dump_active .. 0 homeless 2024-02-23T00:56:52.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.996+0000 7f629659c700 1 --2- 172.21.15.47:0/3862807928 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6280062cc0 0x7f6280065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:52.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.997+0000 7f62877fe700 10 client.31947.objecter ms_handle_connect 0x7f6280062cc0 2024-02-23T00:56:52.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:51.997+0000 7f629659c700 1 --2- 172.21.15.47:0/3862807928 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6280062cc0 0x7f6280065180 secure :-1 s=READY pgs=3228 cs=0 l=1 rev1=1 rx=0x7f6288004bf0 tx=0x7f628800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:52.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:52.001+0000 7f62877fe700 1 -- 172.21.15.47:0/3862807928 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f628c04ded0 con 0x7f62980f9620 2024-02-23T00:56:52.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:50.507251+0000 osd.4 (osd.4) 371 : cluster [DBG] 6.2 scrub starts 2024-02-23T00:56:52.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:50.508146+0000 osd.4 (osd.4) 372 : cluster [DBG] 6.2 scrub ok 2024-02-23T00:56:52.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:50.918787+0000 osd.2 (osd.2) 199 : cluster [DBG] 5.a deep-scrub starts 2024-02-23T00:56:52.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:50.919804+0000 osd.2 (osd.2) 200 : cluster [DBG] 5.a deep-scrub ok 2024-02-23T00:56:52.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:50.995003+0000 osd.7 (osd.7) 391 : cluster [DBG] 5.20 scrub starts 2024-02-23T00:56:52.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:50.998542+0000 osd.7 (osd.7) 392 : cluster [DBG] 5.20 scrub ok 2024-02-23T00:56:52.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:51.420765+0000 mgr.y (mgr.14517) 710 : cluster [DBG] pgmap v1279: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 841 B/s rd, 0 op/s; 108 B/s, 2 objects/s recovering 2024-02-23T00:56:52.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:51.809674+0000 mon.a (mon.0) 3884 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:56:52.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:50.507251+0000 osd.4 (osd.4) 371 : cluster [DBG] 6.2 scrub starts 2024-02-23T00:56:52.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:50.508146+0000 osd.4 (osd.4) 372 : cluster [DBG] 6.2 scrub ok 2024-02-23T00:56:52.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:50.918787+0000 osd.2 (osd.2) 199 : cluster [DBG] 5.a deep-scrub starts 2024-02-23T00:56:52.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:50.919804+0000 osd.2 (osd.2) 200 : cluster [DBG] 5.a deep-scrub ok 2024-02-23T00:56:52.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:50.995003+0000 osd.7 (osd.7) 391 : cluster [DBG] 5.20 scrub starts 2024-02-23T00:56:52.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:50.998542+0000 osd.7 (osd.7) 392 : cluster [DBG] 5.20 scrub ok 2024-02-23T00:56:52.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:51.420765+0000 mgr.y (mgr.14517) 710 : cluster [DBG] pgmap v1279: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 841 B/s rd, 0 op/s; 108 B/s, 2 objects/s recovering 2024-02-23T00:56:52.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:51 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:51.809674+0000 mon.a (mon.0) 3884 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:56:52.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:50.507251+0000 osd.4 (osd.4) 371 : cluster [DBG] 6.2 scrub starts 2024-02-23T00:56:52.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:50.508146+0000 osd.4 (osd.4) 372 : cluster [DBG] 6.2 scrub ok 2024-02-23T00:56:52.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:50.918787+0000 osd.2 (osd.2) 199 : cluster [DBG] 5.a deep-scrub starts 2024-02-23T00:56:52.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:50.919804+0000 osd.2 (osd.2) 200 : cluster [DBG] 5.a deep-scrub ok 2024-02-23T00:56:52.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:50.995003+0000 osd.7 (osd.7) 391 : cluster [DBG] 5.20 scrub starts 2024-02-23T00:56:52.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:50.998542+0000 osd.7 (osd.7) 392 : cluster [DBG] 5.20 scrub ok 2024-02-23T00:56:52.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:51.420765+0000 mgr.y (mgr.14517) 710 : cluster [DBG] pgmap v1279: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 841 B/s rd, 0 op/s; 108 B/s, 2 objects/s recovering 2024-02-23T00:56:52.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:51 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:51.809674+0000 mon.a (mon.0) 3884 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:56:52.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:52.198+0000 7f629c847700 1 -- 172.21.15.47:0/3862807928 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd reweight", "id": 1, "weight": 1.0} v 0) v1 -- 0x7f629805ee80 con 0x7f62980f9620 2024-02-23T00:56:52.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:52.877+0000 7f62877fe700 1 -- 172.21.15.47:0/3862807928 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 1, "weight": 1.0}]=0 reweighted osd.1 to 1 (10000) v836) v1 ==== 113+0+0 (secure 0 0 0) 0x7f628c0043d0 con 0x7f62980f9620 2024-02-23T00:56:52.883 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:56:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:56:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:56:52.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:52.879+0000 7f629c847700 10 librados: watch_flush enter 2024-02-23T00:56:52.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:52.880+0000 7f629c847700 10 librados: watch_flush exit 2024-02-23T00:56:52.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:52.880+0000 7f629c847700 20 client.31947.objecter shutdown clearing up homeless session... 2024-02-23T00:56:52.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:52.880+0000 7f629c847700 10 client.31947.objecter successfully canceled tick 2024-02-23T00:56:52.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:52.880+0000 7f629c847700 1 -- 172.21.15.47:0/3862807928 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6280062cc0 msgr2=0x7f6280065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:52.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:52.880+0000 7f629c847700 1 --2- 172.21.15.47:0/3862807928 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6280062cc0 0x7f6280065180 secure :-1 s=READY pgs=3228 cs=0 l=1 rev1=1 rx=0x7f6288004bf0 tx=0x7f628800b040).stop 2024-02-23T00:56:52.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:52.880+0000 7f629c847700 1 -- 172.21.15.47:0/3862807928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62980f9620 msgr2=0x7f62980f71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:52.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:52.880+0000 7f629c847700 1 --2- 172.21.15.47:0/3862807928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62980f9620 0x7f62980f71f0 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 rx=0x7f628c009f20 tx=0x7f628c009cb0).stop 2024-02-23T00:56:52.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:52.882+0000 7f629c847700 1 -- 172.21.15.47:0/3862807928 shutdown_connections 2024-02-23T00:56:52.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:52.882+0000 7f629c847700 1 --2- 172.21.15.47:0/3862807928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62981022b0 0x7f62980f76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:52.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:52.882+0000 7f629c847700 1 --2- 172.21.15.47:0/3862807928 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6280062cc0 0x7f6280065180 unknown :-1 s=CLOSED pgs=3228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:52.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:52.882+0000 7f629c847700 1 --2- 172.21.15.47:0/3862807928 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f62980f8d50 0x7f62980f6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:52.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:52.882+0000 7f629c847700 1 --2- 172.21.15.47:0/3862807928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62980f9620 0x7f62980f71f0 unknown :-1 s=CLOSED pgs=2664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:52.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:52.882+0000 7f629c847700 1 -- 172.21.15.47:0/3862807928 >> 172.21.15.47:0/3862807928 conn(0x7f62980f2820 msgr2=0x7f62980f3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:56:52.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:52.882+0000 7f629c847700 1 -- 172.21.15.47:0/3862807928 shutdown_connections 2024-02-23T00:56:52.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:52.882+0000 7f629c847700 1 -- 172.21.15.47:0/3862807928 wait complete. 2024-02-23T00:56:52.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:52.882+0000 7f629c847700 1 librados: shutdown 2024-02-23T00:56:52.886 INFO:teuthology.orchestra.run.smithi047.stderr:reweighted osd.1 to 1 (10000) 2024-02-23T00:56:53.194 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:56:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:56:53.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:51.029311+0000 osd.4 (osd.4) 373 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:56:53.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:51.029508+0000 osd.4 (osd.4) 374 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:56:53.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:51.305876+0000 osd.5 (osd.5) 370 : cluster [DBG] 2.12 scrub starts 2024-02-23T00:56:53.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:51.306803+0000 osd.5 (osd.5) 371 : cluster [DBG] 2.12 scrub ok 2024-02-23T00:56:53.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:51.841352+0000 osd.6 (osd.6) 345 : cluster [DBG] 4.1f scrub starts 2024-02-23T00:56:53.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:51.843193+0000 osd.6 (osd.6) 346 : cluster [DBG] 4.1f scrub ok 2024-02-23T00:56:53.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:51.870616+0000 mon.a (mon.0) 3885 : cluster [DBG] osdmap e835: 8 total, 8 up, 8 in 2024-02-23T00:56:53.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:52.199643+0000 mon.a (mon.0) 3886 : audit [INF] from='client.? 172.21.15.47:0/3862807928' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-23T00:56:53.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:52.278251+0000 osd.5 (osd.5) 372 : cluster [DBG] 18.b scrub starts 2024-02-23T00:56:53.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:52.280543+0000 osd.5 (osd.5) 373 : cluster [DBG] 18.b scrub ok 2024-02-23T00:56:53.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:51.029311+0000 osd.4 (osd.4) 373 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:56:53.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:51.029508+0000 osd.4 (osd.4) 374 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:56:53.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:51.305876+0000 osd.5 (osd.5) 370 : cluster [DBG] 2.12 scrub starts 2024-02-23T00:56:53.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:51.306803+0000 osd.5 (osd.5) 371 : cluster [DBG] 2.12 scrub ok 2024-02-23T00:56:53.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:51.841352+0000 osd.6 (osd.6) 345 : cluster [DBG] 4.1f scrub starts 2024-02-23T00:56:53.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:51.843193+0000 osd.6 (osd.6) 346 : cluster [DBG] 4.1f scrub ok 2024-02-23T00:56:53.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:51.870616+0000 mon.a (mon.0) 3885 : cluster [DBG] osdmap e835: 8 total, 8 up, 8 in 2024-02-23T00:56:53.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:52.199643+0000 mon.a (mon.0) 3886 : audit [INF] from='client.? 172.21.15.47:0/3862807928' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-23T00:56:53.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:52.278251+0000 osd.5 (osd.5) 372 : cluster [DBG] 18.b scrub starts 2024-02-23T00:56:53.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:52.280543+0000 osd.5 (osd.5) 373 : cluster [DBG] 18.b scrub ok 2024-02-23T00:56:53.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:51.029311+0000 osd.4 (osd.4) 373 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:56:53.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:51.029508+0000 osd.4 (osd.4) 374 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:56:53.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:51.305876+0000 osd.5 (osd.5) 370 : cluster [DBG] 2.12 scrub starts 2024-02-23T00:56:53.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:51.306803+0000 osd.5 (osd.5) 371 : cluster [DBG] 2.12 scrub ok 2024-02-23T00:56:53.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:51.841352+0000 osd.6 (osd.6) 345 : cluster [DBG] 4.1f scrub starts 2024-02-23T00:56:53.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:51.843193+0000 osd.6 (osd.6) 346 : cluster [DBG] 4.1f scrub ok 2024-02-23T00:56:53.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:51.870616+0000 mon.a (mon.0) 3885 : cluster [DBG] osdmap e835: 8 total, 8 up, 8 in 2024-02-23T00:56:53.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:52.199643+0000 mon.a (mon.0) 3886 : audit [INF] from='client.? 172.21.15.47:0/3862807928' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-23T00:56:53.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:52.278251+0000 osd.5 (osd.5) 372 : cluster [DBG] 18.b scrub starts 2024-02-23T00:56:53.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:52.280543+0000 osd.5 (osd.5) 373 : cluster [DBG] 18.b scrub ok 2024-02-23T00:56:53.921 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd primary-affinity 1 1 2024-02-23T00:56:54.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:52.878140+0000 mon.a (mon.0) 3887 : audit [INF] from='client.? 172.21.15.47:0/3862807928' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-23T00:56:54.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:52.878236+0000 mon.a (mon.0) 3888 : cluster [DBG] osdmap e836: 8 total, 8 up, 8 in 2024-02-23T00:56:54.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:52.998922+0000 osd.7 (osd.7) 393 : cluster [DBG] 5.27 scrub starts 2024-02-23T00:56:54.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:52.999998+0000 osd.7 (osd.7) 394 : cluster [DBG] 5.27 scrub ok 2024-02-23T00:56:54.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:53.040445+0000 mgr.y (mgr.14517) 711 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:56:54.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:53.114764+0000 osd.0 (osd.0) 322 : cluster [DBG] 5.3 scrub starts 2024-02-23T00:56:54.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:53.115883+0000 osd.0 (osd.0) 323 : cluster [DBG] 5.3 scrub ok 2024-02-23T00:56:54.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:53.421251+0000 mgr.y (mgr.14517) 712 : cluster [DBG] pgmap v1282: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 919 B/s rd, 0 op/s 2024-02-23T00:56:54.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:52.878140+0000 mon.a (mon.0) 3887 : audit [INF] from='client.? 172.21.15.47:0/3862807928' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-23T00:56:54.307 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:52.878236+0000 mon.a (mon.0) 3888 : cluster [DBG] osdmap e836: 8 total, 8 up, 8 in 2024-02-23T00:56:54.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:52.998922+0000 osd.7 (osd.7) 393 : cluster [DBG] 5.27 scrub starts 2024-02-23T00:56:54.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:52.999998+0000 osd.7 (osd.7) 394 : cluster [DBG] 5.27 scrub ok 2024-02-23T00:56:54.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:53.040445+0000 mgr.y (mgr.14517) 711 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:56:54.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:53.114764+0000 osd.0 (osd.0) 322 : cluster [DBG] 5.3 scrub starts 2024-02-23T00:56:54.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:53.115883+0000 osd.0 (osd.0) 323 : cluster [DBG] 5.3 scrub ok 2024-02-23T00:56:54.308 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:53.421251+0000 mgr.y (mgr.14517) 712 : cluster [DBG] pgmap v1282: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 919 B/s rd, 0 op/s 2024-02-23T00:56:54.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:52.878140+0000 mon.a (mon.0) 3887 : audit [INF] from='client.? 172.21.15.47:0/3862807928' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-23T00:56:54.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:52.878236+0000 mon.a (mon.0) 3888 : cluster [DBG] osdmap e836: 8 total, 8 up, 8 in 2024-02-23T00:56:54.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:52.998922+0000 osd.7 (osd.7) 393 : cluster [DBG] 5.27 scrub starts 2024-02-23T00:56:54.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:52.999998+0000 osd.7 (osd.7) 394 : cluster [DBG] 5.27 scrub ok 2024-02-23T00:56:54.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:53.040445+0000 mgr.y (mgr.14517) 711 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:56:54.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:53.114764+0000 osd.0 (osd.0) 322 : cluster [DBG] 5.3 scrub starts 2024-02-23T00:56:54.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:53.115883+0000 osd.0 (osd.0) 323 : cluster [DBG] 5.3 scrub ok 2024-02-23T00:56:54.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:53.421251+0000 mgr.y (mgr.14517) 712 : cluster [DBG] pgmap v1282: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 919 B/s rd, 0 op/s 2024-02-23T00:56:54.891 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:56:54.891 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:56:54.918 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:53.303307+0000 osd.5 (osd.5) 374 : cluster [DBG] 2.18 scrub starts 2024-02-23T00:56:54.918 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:53.305283+0000 osd.5 (osd.5) 375 : cluster [DBG] 2.18 scrub ok 2024-02-23T00:56:54.918 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:53.798787+0000 osd.6 (osd.6) 347 : cluster [DBG] 18.14 scrub starts 2024-02-23T00:56:54.918 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:53.799950+0000 osd.6 (osd.6) 348 : cluster [DBG] 18.14 scrub ok 2024-02-23T00:56:54.918 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:54.028044+0000 osd.7 (osd.7) 395 : cluster [DBG] 5.26 scrub starts 2024-02-23T00:56:54.918 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:54.029074+0000 osd.7 (osd.7) 396 : cluster [DBG] 5.26 scrub ok 2024-02-23T00:56:54.918 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:54.094664+0000 osd.0 (osd.0) 324 : cluster [DBG] 3.1 scrub starts 2024-02-23T00:56:54.918 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:54.095545+0000 osd.0 (osd.0) 325 : cluster [DBG] 3.1 scrub ok 2024-02-23T00:56:54.918 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:53.303307+0000 osd.5 (osd.5) 374 : cluster [DBG] 2.18 scrub starts 2024-02-23T00:56:54.918 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:53.305283+0000 osd.5 (osd.5) 375 : cluster [DBG] 2.18 scrub ok 2024-02-23T00:56:54.919 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:53.798787+0000 osd.6 (osd.6) 347 : cluster [DBG] 18.14 scrub starts 2024-02-23T00:56:54.919 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:53.799950+0000 osd.6 (osd.6) 348 : cluster [DBG] 18.14 scrub ok 2024-02-23T00:56:54.919 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:54.028044+0000 osd.7 (osd.7) 395 : cluster [DBG] 5.26 scrub starts 2024-02-23T00:56:54.919 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:54.029074+0000 osd.7 (osd.7) 396 : cluster [DBG] 5.26 scrub ok 2024-02-23T00:56:54.919 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:54.094664+0000 osd.0 (osd.0) 324 : cluster [DBG] 3.1 scrub starts 2024-02-23T00:56:54.919 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:54.095545+0000 osd.0 (osd.0) 325 : cluster [DBG] 3.1 scrub ok 2024-02-23T00:56:54.920 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:56:55.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:53.303307+0000 osd.5 (osd.5) 374 : cluster [DBG] 2.18 scrub starts 2024-02-23T00:56:55.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:53.305283+0000 osd.5 (osd.5) 375 : cluster [DBG] 2.18 scrub ok 2024-02-23T00:56:55.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:53.798787+0000 osd.6 (osd.6) 347 : cluster [DBG] 18.14 scrub starts 2024-02-23T00:56:55.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:53.799950+0000 osd.6 (osd.6) 348 : cluster [DBG] 18.14 scrub ok 2024-02-23T00:56:55.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:54.028044+0000 osd.7 (osd.7) 395 : cluster [DBG] 5.26 scrub starts 2024-02-23T00:56:55.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:54.029074+0000 osd.7 (osd.7) 396 : cluster [DBG] 5.26 scrub ok 2024-02-23T00:56:55.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:54.094664+0000 osd.0 (osd.0) 324 : cluster [DBG] 3.1 scrub starts 2024-02-23T00:56:55.195 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:54.095545+0000 osd.0 (osd.0) 325 : cluster [DBG] 3.1 scrub ok 2024-02-23T00:56:55.775 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.3 dump_historic_ops 2024-02-23T00:56:56.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:54.792474+0000 osd.6 (osd.6) 349 : cluster [DBG] 2.11 scrub starts 2024-02-23T00:56:56.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:54.793430+0000 osd.6 (osd.6) 350 : cluster [DBG] 2.11 scrub ok 2024-02-23T00:56:56.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:55.358167+0000 osd.5 (osd.5) 376 : cluster [DBG] 5.16 scrub starts 2024-02-23T00:56:56.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:55.361176+0000 osd.5 (osd.5) 377 : cluster [DBG] 5.16 scrub ok 2024-02-23T00:56:56.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:55.421617+0000 mgr.y (mgr.14517) 713 : cluster [DBG] pgmap v1283: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:56:56.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:56.312237+0000 mon.a (mon.0) 3889 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:56:56.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:54.792474+0000 osd.6 (osd.6) 349 : cluster [DBG] 2.11 scrub starts 2024-02-23T00:56:56.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:54.793430+0000 osd.6 (osd.6) 350 : cluster [DBG] 2.11 scrub ok 2024-02-23T00:56:56.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:55.358167+0000 osd.5 (osd.5) 376 : cluster [DBG] 5.16 scrub starts 2024-02-23T00:56:56.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:55.361176+0000 osd.5 (osd.5) 377 : cluster [DBG] 5.16 scrub ok 2024-02-23T00:56:56.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:55.421617+0000 mgr.y (mgr.14517) 713 : cluster [DBG] pgmap v1283: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:56:56.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:56.312237+0000 mon.a (mon.0) 3889 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:56:56.650 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:56:56.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:54.792474+0000 osd.6 (osd.6) 349 : cluster [DBG] 2.11 scrub starts 2024-02-23T00:56:56.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:54.793430+0000 osd.6 (osd.6) 350 : cluster [DBG] 2.11 scrub ok 2024-02-23T00:56:56.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:55.358167+0000 osd.5 (osd.5) 376 : cluster [DBG] 5.16 scrub starts 2024-02-23T00:56:56.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:55.361176+0000 osd.5 (osd.5) 377 : cluster [DBG] 5.16 scrub ok 2024-02-23T00:56:56.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:55.421617+0000 mgr.y (mgr.14517) 713 : cluster [DBG] pgmap v1283: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:56:56.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:56.312237+0000 mon.a (mon.0) 3889 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:56:57.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.312+0000 7f3dc7849700 1 -- 172.21.15.47:0/2490282273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3dc00fdbb0 msgr2=0x7f3dc00fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:57.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.312+0000 7f3dc7849700 1 --2- 172.21.15.47:0/2490282273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3dc00fdbb0 0x7f3dc00fdf90 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 rx=0x7f3dbc004870 tx=0x7f3dbc009cb0).stop 2024-02-23T00:56:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.312+0000 7f3dc7849700 1 -- 172.21.15.47:0/2490282273 shutdown_connections 2024-02-23T00:56:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.312+0000 7f3dc7849700 1 --2- 172.21.15.47:0/2490282273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3dc00f8420 0x7f3dc0104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.312+0000 7f3dc7849700 1 --2- 172.21.15.47:0/2490282273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3dc00fdbb0 0x7f3dc00fdf90 unknown :-1 s=CLOSED pgs=2665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.312+0000 7f3dc7849700 1 --2- 172.21.15.47:0/2490282273 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3dc00fe510 0x7f3dc00f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.312+0000 7f3dc7849700 1 -- 172.21.15.47:0/2490282273 >> 172.21.15.47:0/2490282273 conn(0x7f3dc00f3800 msgr2=0x7f3dc00f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:56:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.312+0000 7f3dc7849700 1 -- 172.21.15.47:0/2490282273 shutdown_connections 2024-02-23T00:56:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.312+0000 7f3dc7849700 1 -- 172.21.15.47:0/2490282273 wait complete. 2024-02-23T00:56:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.312+0000 7f3dc7849700 1 librados: starting msgr at 2024-02-23T00:56:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.312+0000 7f3dc7849700 1 librados: starting objecter 2024-02-23T00:56:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.313+0000 7f3dc7849700 1 Processor -- start 2024-02-23T00:56:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.313+0000 7f3dc7849700 1 -- start start 2024-02-23T00:56:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.313+0000 7f3dc7849700 1 librados: setting wanted keys 2024-02-23T00:56:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.313+0000 7f3dc7849700 1 librados: calling monclient init 2024-02-23T00:56:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.313+0000 7f3dc7849700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3dc00f8420 0x7f3dc006b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.313+0000 7f3dc7849700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3dc00fdbb0 0x7f3dc006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.313+0000 7f3dc7849700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3dc00fe510 0x7f3dc006bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.313+0000 7f3dc7849700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc0116350 con 0x7f3dc00fdbb0 2024-02-23T00:56:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.313+0000 7f3dc7849700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc01164c0 con 0x7f3dc00f8420 2024-02-23T00:56:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.313+0000 7f3dc7849700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc0116680 con 0x7f3dc00fe510 2024-02-23T00:56:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.313+0000 7f3dc4de4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3dc00fdbb0 0x7f3dc006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.313+0000 7f3dc5de6700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3dc00fe510 0x7f3dc006bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.313+0000 7f3dc4de4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3dc00fdbb0 0x7f3dc006b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54752/0 (socket says 172.21.15.47:54752) 2024-02-23T00:56:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.313+0000 7f3dc4de4700 1 -- 172.21.15.47:0/4107416527 learned_addr learned my addr 172.21.15.47:0/4107416527 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:56:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.313+0000 7f3dc55e5700 1 --2- 172.21.15.47:0/4107416527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3dc00f8420 0x7f3dc006b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.313+0000 7f3dc5de6700 1 -- 172.21.15.47:0/4107416527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3dc00f8420 msgr2=0x7f3dc006b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.313+0000 7f3dc5de6700 1 --2- 172.21.15.47:0/4107416527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3dc00f8420 0x7f3dc006b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.313+0000 7f3dc5de6700 1 -- 172.21.15.47:0/4107416527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3dc00fdbb0 msgr2=0x7f3dc006b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.313+0000 7f3dc5de6700 1 --2- 172.21.15.47:0/4107416527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3dc00fdbb0 0x7f3dc006b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.313+0000 7f3dc5de6700 1 -- 172.21.15.47:0/4107416527 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db0005040 con 0x7f3dc00fe510 2024-02-23T00:56:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.313+0000 7f3dc4de4700 1 --2- 172.21.15.47:0/4107416527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3dc00fdbb0 0x7f3dc006b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:56:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.313+0000 7f3dc5de6700 1 --2- 172.21.15.47:0/4107416527 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3dc00fe510 0x7f3dc006bab0 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 rx=0x7f3db800eb70 tx=0x7f3db800b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.313+0000 7f3db67fc700 10 client.?.objecter ms_handle_connect 0x7f3dc00fe510 2024-02-23T00:56:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.313+0000 7f3db67fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:56:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.314+0000 7f3db67fc700 1 -- 172.21.15.47:0/4107416527 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db8009870 con 0x7f3dc00fe510 2024-02-23T00:56:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.314+0000 7f3dc7849700 1 -- 172.21.15.47:0/4107416527 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dbc005040 con 0x7f3dc00fe510 2024-02-23T00:56:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.314+0000 7f3db67fc700 1 -- 172.21.15.47:0/4107416527 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f3db800f040 con 0x7f3dc00fe510 2024-02-23T00:56:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.314+0000 7f3db67fc700 1 -- 172.21.15.47:0/4107416527 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db8007c70 con 0x7f3dc00fe510 2024-02-23T00:56:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.314+0000 7f3dc7849700 10 client.42275.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:56:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.314+0000 7f3dc7849700 1 -- 172.21.15.47:0/4107416527 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc0116c90 con 0x7f3dc00fe510 2024-02-23T00:56:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.314+0000 7f3dc7849700 1 librados: init done 2024-02-23T00:56:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.315+0000 7f3dc7849700 1 -- 172.21.15.47:0/4107416527 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dc0106e20 con 0x7f3dc00fe510 2024-02-23T00:56:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.315+0000 7f3db67fc700 1 -- 172.21.15.47:0/4107416527 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f3db80040b0 con 0x7f3dc00fe510 2024-02-23T00:56:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.315+0000 7f3db67fc700 1 --2- 172.21.15.47:0/4107416527 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3dac062cc0 0x7f3dac065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:56:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.316+0000 7f3dc55e5700 1 --2- 172.21.15.47:0/4107416527 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3dac062cc0 0x7f3dac065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:56:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.316+0000 7f3db67fc700 1 -- 172.21.15.47:0/4107416527 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(836..836 src has 670..836) v4 ==== 8663+0+0 (secure 0 0 0) 0x7f3db807d1b0 con 0x7f3dc00fe510 2024-02-23T00:56:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.316+0000 7f3db67fc700 10 client.42275.objecter ms_dispatch 0x7f3dc0001550 osd_map(836..836 src has 670..836) v4 2024-02-23T00:56:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.316+0000 7f3db67fc700 3 client.42275.objecter handle_osd_map got epochs [836,836] > 0 2024-02-23T00:56:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.316+0000 7f3db67fc700 3 client.42275.objecter handle_osd_map decoding full epoch 836 2024-02-23T00:56:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.316+0000 7f3db67fc700 20 client.42275.objecter dump_active .. 0 homeless 2024-02-23T00:56:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.316+0000 7f3db67fc700 10 client.42275.objecter ms_handle_connect 0x7f3dac062cc0 2024-02-23T00:56:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.316+0000 7f3dc55e5700 1 --2- 172.21.15.47:0/4107416527 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3dac062cc0 0x7f3dac065180 secure :-1 s=READY pgs=3229 cs=0 l=1 rev1=1 rx=0x7f3dbc004ee0 tx=0x7f3dbc00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:56:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.320+0000 7f3db67fc700 1 -- 172.21.15.47:0/4107416527 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f3db8049f60 con 0x7f3dc00fe510 2024-02-23T00:56:57.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:57.510+0000 7f3dc7849700 1 -- 172.21.15.47:0/4107416527 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd primary-affinity", "id": 1, "weight": 1.0} v 0) v1 -- 0x7f3dc006c5b0 con 0x7f3dc00fe510 2024-02-23T00:56:57.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:56.816327+0000 osd.6 (osd.6) 351 : cluster [DBG] 18.10 scrub starts 2024-02-23T00:56:57.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:56.817535+0000 osd.6 (osd.6) 352 : cluster [DBG] 18.10 scrub ok 2024-02-23T00:56:57.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:56.816327+0000 osd.6 (osd.6) 351 : cluster [DBG] 18.10 scrub starts 2024-02-23T00:56:57.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:56.817535+0000 osd.6 (osd.6) 352 : cluster [DBG] 18.10 scrub ok 2024-02-23T00:56:57.776 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:56.816327+0000 osd.6 (osd.6) 351 : cluster [DBG] 18.10 scrub starts 2024-02-23T00:56:57.776 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:56.817535+0000 osd.6 (osd.6) 352 : cluster [DBG] 18.10 scrub ok 2024-02-23T00:56:58.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:58.480+0000 7f3db67fc700 1 -- 172.21.15.47:0/4107416527 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]=0 set osd.1 primary-affinity to 1 (8655362) v837) v1 ==== 133+0+0 (secure 0 0 0) 0x7f3dc006c5b0 con 0x7f3dc00fe510 2024-02-23T00:56:58.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:58.485+0000 7f3dc7849700 10 librados: watch_flush enter 2024-02-23T00:56:58.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:58.485+0000 7f3dc7849700 10 librados: watch_flush exit 2024-02-23T00:56:58.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:58.485+0000 7f3dc7849700 20 client.42275.objecter shutdown clearing up homeless session... 2024-02-23T00:56:58.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:58.485+0000 7f3dc7849700 10 client.42275.objecter successfully canceled tick 2024-02-23T00:56:58.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:58.485+0000 7f3dc7849700 1 -- 172.21.15.47:0/4107416527 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3dac062cc0 msgr2=0x7f3dac065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:58.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:58.485+0000 7f3dc7849700 1 --2- 172.21.15.47:0/4107416527 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3dac062cc0 0x7f3dac065180 secure :-1 s=READY pgs=3229 cs=0 l=1 rev1=1 rx=0x7f3dbc004ee0 tx=0x7f3dbc00b040).stop 2024-02-23T00:56:58.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:58.485+0000 7f3dc7849700 1 -- 172.21.15.47:0/4107416527 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3dc00fe510 msgr2=0x7f3dc006bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:56:58.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:58.485+0000 7f3dc7849700 1 --2- 172.21.15.47:0/4107416527 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3dc00fe510 0x7f3dc006bab0 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 rx=0x7f3db800eb70 tx=0x7f3db800b040).stop 2024-02-23T00:56:58.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:58.487+0000 7f3dc7849700 1 -- 172.21.15.47:0/4107416527 shutdown_connections 2024-02-23T00:56:58.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:58.487+0000 7f3dc7849700 1 --2- 172.21.15.47:0/4107416527 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f3dac062cc0 0x7f3dac065180 unknown :-1 s=CLOSED pgs=3229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:58.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:58.487+0000 7f3dc7849700 1 --2- 172.21.15.47:0/4107416527 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3dc00fe510 0x7f3dc006bab0 unknown :-1 s=CLOSED pgs=2843 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:58.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:58.487+0000 7f3dc7849700 1 --2- 172.21.15.47:0/4107416527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3dc00f8420 0x7f3dc006b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:58.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:58.487+0000 7f3dc7849700 1 --2- 172.21.15.47:0/4107416527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3dc00fdbb0 0x7f3dc006b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:56:58.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:58.487+0000 7f3dc7849700 1 -- 172.21.15.47:0/4107416527 >> 172.21.15.47:0/4107416527 conn(0x7f3dc00f3800 msgr2=0x7f3dc00fb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:56:58.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:58.488+0000 7f3dc7849700 1 -- 172.21.15.47:0/4107416527 shutdown_connections 2024-02-23T00:56:58.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:58.488+0000 7f3dc7849700 1 -- 172.21.15.47:0/4107416527 wait complete. 2024-02-23T00:56:58.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:56:58.488+0000 7f3dc7849700 1 librados: shutdown 2024-02-23T00:56:58.491 INFO:teuthology.orchestra.run.smithi047.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-02-23T00:56:58.703 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:57.422429+0000 mgr.y (mgr.14517) 714 : cluster [DBG] pgmap v1284: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-23T00:56:58.703 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:57.511745+0000 mon.c (mon.2) 526 : audit [INF] from='client.? 172.21.15.47:0/4107416527' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-02-23T00:56:58.703 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:57.512028+0000 mon.a (mon.0) 3890 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-02-23T00:56:58.704 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:57.565027+0000 osd.4 (osd.4) 375 : cluster [DBG] 5.1 scrub starts 2024-02-23T00:56:58.704 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:57.566014+0000 osd.4 (osd.4) 376 : cluster [DBG] 5.1 scrub ok 2024-02-23T00:56:58.704 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:57.422429+0000 mgr.y (mgr.14517) 714 : cluster [DBG] pgmap v1284: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-23T00:56:58.704 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:57.511745+0000 mon.c (mon.2) 526 : audit [INF] from='client.? 172.21.15.47:0/4107416527' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-02-23T00:56:58.704 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:57.512028+0000 mon.a (mon.0) 3890 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-02-23T00:56:58.704 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:57.565027+0000 osd.4 (osd.4) 375 : cluster [DBG] 5.1 scrub starts 2024-02-23T00:56:58.704 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:57.566014+0000 osd.4 (osd.4) 376 : cluster [DBG] 5.1 scrub ok 2024-02-23T00:56:58.844 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-02-23T00:56:58.844 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:56:58.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:57.422429+0000 mgr.y (mgr.14517) 714 : cluster [DBG] pgmap v1284: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-23T00:56:58.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:57.511745+0000 mon.c (mon.2) 526 : audit [INF] from='client.? 172.21.15.47:0/4107416527' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-02-23T00:56:58.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:57.512028+0000 mon.a (mon.0) 3890 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-02-23T00:56:58.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:57.565027+0000 osd.4 (osd.4) 375 : cluster [DBG] 5.1 scrub starts 2024-02-23T00:56:58.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:57.566014+0000 osd.4 (osd.4) 376 : cluster [DBG] 5.1 scrub ok 2024-02-23T00:56:59.492 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:56:58.478903+0000 mon.a (mon.0) 3891 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-02-23T00:56:59.492 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:58.478995+0000 mon.a (mon.0) 3892 : cluster [DBG] osdmap e837: 8 total, 8 up, 8 in 2024-02-23T00:56:59.492 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:59.292940+0000 osd.5 (osd.5) 378 : cluster [DBG] 6.7 scrub starts 2024-02-23T00:56:59.492 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:56:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:59.295983+0000 osd.5 (osd.5) 379 : cluster [DBG] 6.7 scrub ok 2024-02-23T00:56:59.644 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.4 dump_ops_in_flight 2024-02-23T00:56:59.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:56:58.478903+0000 mon.a (mon.0) 3891 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-02-23T00:56:59.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:58.478995+0000 mon.a (mon.0) 3892 : cluster [DBG] osdmap e837: 8 total, 8 up, 8 in 2024-02-23T00:56:59.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:59.292940+0000 osd.5 (osd.5) 378 : cluster [DBG] 6.7 scrub starts 2024-02-23T00:56:59.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:56:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:59.295983+0000 osd.5 (osd.5) 379 : cluster [DBG] 6.7 scrub ok 2024-02-23T00:56:59.889 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight 2 1 2024-02-23T00:56:59.919 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:56:59.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:56:58.478903+0000 mon.a (mon.0) 3891 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-02-23T00:56:59.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:58.478995+0000 mon.a (mon.0) 3892 : cluster [DBG] osdmap e837: 8 total, 8 up, 8 in 2024-02-23T00:56:59.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:59.292940+0000 osd.5 (osd.5) 378 : cluster [DBG] 6.7 scrub starts 2024-02-23T00:56:59.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:56:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:59.295983+0000 osd.5 (osd.5) 379 : cluster [DBG] 6.7 scrub ok 2024-02-23T00:57:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:57:00.807 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:57:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:57:00] "GET /metrics HTTP/1.1" 200 187130 "" "Prometheus/2.33.4" 2024-02-23T00:57:00.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:59.422963+0000 mgr.y (mgr.14517) 715 : cluster [DBG] pgmap v1286: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-23T00:57:00.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:59.492417+0000 mon.a (mon.0) 3893 : cluster [DBG] osdmap e838: 8 total, 8 up, 8 in 2024-02-23T00:57:00.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:59.532947+0000 osd.4 (osd.4) 377 : cluster [DBG] 4.1 deep-scrub starts 2024-02-23T00:57:00.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:56:59.534777+0000 osd.4 (osd.4) 378 : cluster [DBG] 4.1 deep-scrub ok 2024-02-23T00:57:00.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:00.292669+0000 osd.5 (osd.5) 380 : cluster [DBG] 5.24 scrub starts 2024-02-23T00:57:00.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:00.295878+0000 osd.5 (osd.5) 381 : cluster [DBG] 5.24 scrub ok 2024-02-23T00:57:00.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:59.422963+0000 mgr.y (mgr.14517) 715 : cluster [DBG] pgmap v1286: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-23T00:57:00.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:59.492417+0000 mon.a (mon.0) 3893 : cluster [DBG] osdmap e838: 8 total, 8 up, 8 in 2024-02-23T00:57:00.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:59.532947+0000 osd.4 (osd.4) 377 : cluster [DBG] 4.1 deep-scrub starts 2024-02-23T00:57:00.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:56:59.534777+0000 osd.4 (osd.4) 378 : cluster [DBG] 4.1 deep-scrub ok 2024-02-23T00:57:00.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:00.292669+0000 osd.5 (osd.5) 380 : cluster [DBG] 5.24 scrub starts 2024-02-23T00:57:00.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:00.295878+0000 osd.5 (osd.5) 381 : cluster [DBG] 5.24 scrub ok 2024-02-23T00:57:00.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:59.422963+0000 mgr.y (mgr.14517) 715 : cluster [DBG] pgmap v1286: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-23T00:57:00.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:59.492417+0000 mon.a (mon.0) 3893 : cluster [DBG] osdmap e838: 8 total, 8 up, 8 in 2024-02-23T00:57:00.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:59.532947+0000 osd.4 (osd.4) 377 : cluster [DBG] 4.1 deep-scrub starts 2024-02-23T00:57:00.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:56:59.534777+0000 osd.4 (osd.4) 378 : cluster [DBG] 4.1 deep-scrub ok 2024-02-23T00:57:00.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:00.292669+0000 osd.5 (osd.5) 380 : cluster [DBG] 5.24 scrub starts 2024-02-23T00:57:00.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:00.295878+0000 osd.5 (osd.5) 381 : cluster [DBG] 5.24 scrub ok 2024-02-23T00:57:01.808 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:57:01.808 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:57:01.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.880+0000 7f43686aa700 1 -- 172.21.15.47:0/2544789621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4360102210 msgr2=0x7f4360104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:01.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.880+0000 7f43686aa700 1 --2- 172.21.15.47:0/2544789621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4360102210 0x7f4360104600 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 rx=0x7f4358009c30 tx=0x7f4358004c50).stop 2024-02-23T00:57:01.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.880+0000 7f43686aa700 1 -- 172.21.15.47:0/2544789621 shutdown_connections 2024-02-23T00:57:01.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.880+0000 7f43686aa700 1 --2- 172.21.15.47:0/2544789621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4360102210 0x7f4360104600 unknown :-1 s=CLOSED pgs=2666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:01.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.880+0000 7f43686aa700 1 --2- 172.21.15.47:0/2544789621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43600f8cb0 0x7f43600f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:01.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.880+0000 7f43686aa700 1 --2- 172.21.15.47:0/2544789621 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f43600f9610 0x7f4360101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:01.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.880+0000 7f43686aa700 1 -- 172.21.15.47:0/2544789621 >> 172.21.15.47:0/2544789621 conn(0x7f43600f2820 msgr2=0x7f43600f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:01.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.880+0000 7f43686aa700 1 -- 172.21.15.47:0/2544789621 shutdown_connections 2024-02-23T00:57:01.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.880+0000 7f43686aa700 1 -- 172.21.15.47:0/2544789621 wait complete. 2024-02-23T00:57:01.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.880+0000 7f43686aa700 1 librados: starting msgr at 2024-02-23T00:57:01.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.880+0000 7f43686aa700 1 librados: starting objecter 2024-02-23T00:57:01.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.880+0000 7f43686aa700 1 Processor -- start 2024-02-23T00:57:01.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.881+0000 7f43686aa700 1 -- start start 2024-02-23T00:57:01.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.881+0000 7f43686aa700 1 librados: setting wanted keys 2024-02-23T00:57:01.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.881+0000 7f43686aa700 1 librados: calling monclient init 2024-02-23T00:57:01.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.881+0000 7f43686aa700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f43600f8cb0 0x7f436010de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:01.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.881+0000 7f43686aa700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43600f9610 0x7f436010e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:01.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.881+0000 7f43686aa700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4360102210 0x7f4360107fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:01.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.881+0000 7f43686aa700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43601084a0 con 0x7f4360102210 2024-02-23T00:57:01.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.881+0000 7f43686aa700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4360108610 con 0x7f43600f9610 2024-02-23T00:57:01.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.881+0000 7f43686aa700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43601087d0 con 0x7f43600f8cb0 2024-02-23T00:57:01.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.881+0000 7f4366c47700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4360102210 0x7f4360107fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:01.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.881+0000 7f4366446700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f43600f8cb0 0x7f436010de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:01.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.881+0000 7f4366c47700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4360102210 0x7f4360107fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54764/0 (socket says 172.21.15.47:54764) 2024-02-23T00:57:01.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.881+0000 7f4366c47700 1 -- 172.21.15.47:0/2014829875 learned_addr learned my addr 172.21.15.47:0/2014829875 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:57:01.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.881+0000 7f4365c45700 1 --2- 172.21.15.47:0/2014829875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43600f9610 0x7f436010e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:01.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.881+0000 7f4366c47700 1 -- 172.21.15.47:0/2014829875 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f43600f8cb0 msgr2=0x7f436010de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:01.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.881+0000 7f4366c47700 1 --2- 172.21.15.47:0/2014829875 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f43600f8cb0 0x7f436010de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:01.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.881+0000 7f4366c47700 1 -- 172.21.15.47:0/2014829875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43600f9610 msgr2=0x7f436010e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:01.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.881+0000 7f4366c47700 1 --2- 172.21.15.47:0/2014829875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43600f9610 0x7f436010e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:01.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.881+0000 7f4366c47700 1 -- 172.21.15.47:0/2014829875 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4350005040 con 0x7f4360102210 2024-02-23T00:57:01.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.881+0000 7f4366c47700 1 --2- 172.21.15.47:0/2014829875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4360102210 0x7f4360107fb0 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 rx=0x7f4358004890 tx=0x7f4358004c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:01.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.881+0000 7f43577fe700 10 client.?.objecter ms_handle_connect 0x7f4360102210 2024-02-23T00:57:01.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.881+0000 7f43577fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:57:01.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.881+0000 7f43577fe700 1 -- 172.21.15.47:0/2014829875 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435800e040 con 0x7f4360102210 2024-02-23T00:57:01.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.882+0000 7f43577fe700 1 -- 172.21.15.47:0/2014829875 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f4358018040 con 0x7f4360102210 2024-02-23T00:57:01.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.882+0000 7f43686aa700 1 -- 172.21.15.47:0/2014829875 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43580097c0 con 0x7f4360102210 2024-02-23T00:57:01.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.882+0000 7f43577fe700 1 -- 172.21.15.47:0/2014829875 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4358013d80 con 0x7f4360102210 2024-02-23T00:57:01.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.882+0000 7f43686aa700 10 client.31959.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:57:01.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.882+0000 7f43686aa700 1 -- 172.21.15.47:0/2014829875 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4360108c70 con 0x7f4360102210 2024-02-23T00:57:01.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.882+0000 7f43686aa700 1 librados: init done 2024-02-23T00:57:01.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.882+0000 7f43686aa700 1 -- 172.21.15.47:0/2014829875 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4360108e60 con 0x7f4360102210 2024-02-23T00:57:01.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.883+0000 7f43577fe700 1 -- 172.21.15.47:0/2014829875 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f4358007c80 con 0x7f4360102210 2024-02-23T00:57:01.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.883+0000 7f43577fe700 1 --2- 172.21.15.47:0/2014829875 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f434c062cc0 0x7f434c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:01.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.883+0000 7f43577fe700 1 -- 172.21.15.47:0/2014829875 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(838..838 src has 670..838) v4 ==== 8663+0+0 (secure 0 0 0) 0x7f435807e480 con 0x7f4360102210 2024-02-23T00:57:01.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.883+0000 7f43577fe700 10 client.31959.objecter ms_dispatch 0x7f4360001550 osd_map(838..838 src has 670..838) v4 2024-02-23T00:57:01.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.883+0000 7f43577fe700 3 client.31959.objecter handle_osd_map got epochs [838,838] > 0 2024-02-23T00:57:01.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.883+0000 7f43577fe700 3 client.31959.objecter handle_osd_map decoding full epoch 838 2024-02-23T00:57:01.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.883+0000 7f4366446700 1 --2- 172.21.15.47:0/2014829875 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f434c062cc0 0x7f434c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:01.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.883+0000 7f43577fe700 20 client.31959.objecter dump_active .. 0 homeless 2024-02-23T00:57:01.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.884+0000 7f43577fe700 10 client.31959.objecter ms_handle_connect 0x7f434c062cc0 2024-02-23T00:57:01.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.884+0000 7f4366446700 1 --2- 172.21.15.47:0/2014829875 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f434c062cc0 0x7f434c065180 secure :-1 s=READY pgs=3230 cs=0 l=1 rev1=1 rx=0x7f435c004dc0 tx=0x7f435c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:01.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:01.888+0000 7f43577fe700 1 -- 172.21.15.47:0/2014829875 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f435804ddf0 con 0x7f4360102210 2024-02-23T00:57:02.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:02.078+0000 7f43686aa700 1 -- 172.21.15.47:0/2014829875 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd reweight", "id": 2, "weight": 1.0} v 0) v1 -- 0x7f4360107050 con 0x7f4360102210 2024-02-23T00:57:02.471 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.4 dump_blocked_ops 2024-02-23T00:57:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:02.494+0000 7f43577fe700 1 -- 172.21.15.47:0/2014829875 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 2, "weight": 1.0}]=0 reweighted osd.2 to 1 (10000) v839) v1 ==== 113+0+0 (secure 0 0 0) 0x7f4358021070 con 0x7f4360102210 2024-02-23T00:57:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:02.496+0000 7f43686aa700 10 librados: watch_flush enter 2024-02-23T00:57:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:02.496+0000 7f43686aa700 10 librados: watch_flush exit 2024-02-23T00:57:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:02.496+0000 7f43686aa700 20 client.31959.objecter shutdown clearing up homeless session... 2024-02-23T00:57:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:02.496+0000 7f43686aa700 10 client.31959.objecter successfully canceled tick 2024-02-23T00:57:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:02.496+0000 7f43686aa700 1 -- 172.21.15.47:0/2014829875 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f434c062cc0 msgr2=0x7f434c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:02.496+0000 7f43686aa700 1 --2- 172.21.15.47:0/2014829875 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f434c062cc0 0x7f434c065180 secure :-1 s=READY pgs=3230 cs=0 l=1 rev1=1 rx=0x7f435c004dc0 tx=0x7f435c00b040).stop 2024-02-23T00:57:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:02.497+0000 7f43686aa700 1 -- 172.21.15.47:0/2014829875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4360102210 msgr2=0x7f4360107fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:02.497+0000 7f43686aa700 1 --2- 172.21.15.47:0/2014829875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4360102210 0x7f4360107fb0 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 rx=0x7f4358004890 tx=0x7f4358004c50).stop 2024-02-23T00:57:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:02.497+0000 7f43686aa700 1 -- 172.21.15.47:0/2014829875 shutdown_connections 2024-02-23T00:57:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:02.497+0000 7f43686aa700 1 --2- 172.21.15.47:0/2014829875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4360102210 0x7f4360107fb0 unknown :-1 s=CLOSED pgs=2667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:02.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:02.497+0000 7f43686aa700 1 --2- 172.21.15.47:0/2014829875 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f434c062cc0 0x7f434c065180 secure :-1 s=CLOSED pgs=3230 cs=0 l=1 rev1=1 rx=0x7f435c004dc0 tx=0x7f435c00b040).stop 2024-02-23T00:57:02.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:02.497+0000 7f43686aa700 1 --2- 172.21.15.47:0/2014829875 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f43600f8cb0 0x7f436010de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:02.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:02.497+0000 7f43686aa700 1 --2- 172.21.15.47:0/2014829875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43600f9610 0x7f436010e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:02.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:02.497+0000 7f43686aa700 1 -- 172.21.15.47:0/2014829875 >> 172.21.15.47:0/2014829875 conn(0x7f43600f2820 msgr2=0x7f43600f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:02.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:02.500+0000 7f43686aa700 1 -- 172.21.15.47:0/2014829875 shutdown_connections 2024-02-23T00:57:02.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:02.500+0000 7f43686aa700 1 -- 172.21.15.47:0/2014829875 wait complete. 2024-02-23T00:57:02.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:02.500+0000 7f43686aa700 1 librados: shutdown 2024-02-23T00:57:02.503 INFO:teuthology.orchestra.run.smithi047.stderr:reweighted osd.2 to 1 (10000) 2024-02-23T00:57:02.738 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:57:02.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:01.300133+0000 osd.5 (osd.5) 382 : cluster [DBG] 5.15 deep-scrub starts 2024-02-23T00:57:02.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:01.303618+0000 osd.5 (osd.5) 383 : cluster [DBG] 5.15 deep-scrub ok 2024-02-23T00:57:02.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:01.423585+0000 mgr.y (mgr.14517) 716 : cluster [DBG] pgmap v1288: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:57:02.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:02.079909+0000 mon.a (mon.0) 3894 : audit [INF] from='client.? 172.21.15.47:0/2014829875' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-23T00:57:02.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:02.256317+0000 osd.5 (osd.5) 384 : cluster [DBG] 5.7 scrub starts 2024-02-23T00:57:02.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:02.259247+0000 osd.5 (osd.5) 385 : cluster [DBG] 5.7 scrub ok 2024-02-23T00:57:02.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:01.300133+0000 osd.5 (osd.5) 382 : cluster [DBG] 5.15 deep-scrub starts 2024-02-23T00:57:02.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:01.303618+0000 osd.5 (osd.5) 383 : cluster [DBG] 5.15 deep-scrub ok 2024-02-23T00:57:02.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:01.423585+0000 mgr.y (mgr.14517) 716 : cluster [DBG] pgmap v1288: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:57:02.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:02.079909+0000 mon.a (mon.0) 3894 : audit [INF] from='client.? 172.21.15.47:0/2014829875' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-23T00:57:02.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:02.256317+0000 osd.5 (osd.5) 384 : cluster [DBG] 5.7 scrub starts 2024-02-23T00:57:02.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:02.259247+0000 osd.5 (osd.5) 385 : cluster [DBG] 5.7 scrub ok 2024-02-23T00:57:02.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:01.300133+0000 osd.5 (osd.5) 382 : cluster [DBG] 5.15 deep-scrub starts 2024-02-23T00:57:02.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:01.303618+0000 osd.5 (osd.5) 383 : cluster [DBG] 5.15 deep-scrub ok 2024-02-23T00:57:02.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:01.423585+0000 mgr.y (mgr.14517) 716 : cluster [DBG] pgmap v1288: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:57:02.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:02.079909+0000 mon.a (mon.0) 3894 : audit [INF] from='client.? 172.21.15.47:0/2014829875' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-23T00:57:02.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:02.256317+0000 osd.5 (osd.5) 384 : cluster [DBG] 5.7 scrub starts 2024-02-23T00:57:02.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:02.259247+0000 osd.5 (osd.5) 385 : cluster [DBG] 5.7 scrub ok 2024-02-23T00:57:02.945 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:57:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:57:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:57:03.201 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd primary-affinity 2 1 2024-02-23T00:57:03.444 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:57:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:57:03.483 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:57:03.505 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:02.096862+0000 osd.7 (osd.7) 397 : cluster [DBG] 5.4 deep-scrub starts 2024-02-23T00:57:03.505 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:02.099764+0000 osd.7 (osd.7) 398 : cluster [DBG] 5.4 deep-scrub ok 2024-02-23T00:57:03.505 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:02.495315+0000 mon.a (mon.0) 3895 : audit [INF] from='client.? 172.21.15.47:0/2014829875' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-02-23T00:57:03.506 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:02.096862+0000 osd.7 (osd.7) 397 : cluster [DBG] 5.4 deep-scrub starts 2024-02-23T00:57:03.506 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:02.099764+0000 osd.7 (osd.7) 398 : cluster [DBG] 5.4 deep-scrub ok 2024-02-23T00:57:03.506 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:02.495315+0000 mon.a (mon.0) 3895 : audit [INF] from='client.? 172.21.15.47:0/2014829875' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-02-23T00:57:03.506 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:02.495386+0000 mon.a (mon.0) 3896 : cluster [DBG] osdmap e839: 8 total, 8 up, 8 in 2024-02-23T00:57:03.506 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:02.760213+0000 osd.6 (osd.6) 353 : cluster [DBG] 5.1f scrub starts 2024-02-23T00:57:03.506 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:02.761449+0000 osd.6 (osd.6) 354 : cluster [DBG] 5.1f scrub ok 2024-02-23T00:57:03.506 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:02.852941+0000 osd.1 (osd.1) 316 : cluster [DBG] 2.4 scrub starts 2024-02-23T00:57:03.506 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:02.855898+0000 osd.1 (osd.1) 317 : cluster [DBG] 2.4 scrub ok 2024-02-23T00:57:03.506 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:03.264149+0000 osd.5 (osd.5) 386 : cluster [DBG] 5.28 scrub starts 2024-02-23T00:57:03.506 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:03.265145+0000 osd.5 (osd.5) 387 : cluster [DBG] 5.28 scrub ok 2024-02-23T00:57:03.762 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:02.096862+0000 osd.7 (osd.7) 397 : cluster [DBG] 5.4 deep-scrub starts 2024-02-23T00:57:03.762 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:02.099764+0000 osd.7 (osd.7) 398 : cluster [DBG] 5.4 deep-scrub ok 2024-02-23T00:57:03.762 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:02.495315+0000 mon.a (mon.0) 3895 : audit [INF] from='client.? 172.21.15.47:0/2014829875' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-02-23T00:57:03.762 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:02.495386+0000 mon.a (mon.0) 3896 : cluster [DBG] osdmap e839: 8 total, 8 up, 8 in 2024-02-23T00:57:03.762 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:02.760213+0000 osd.6 (osd.6) 353 : cluster [DBG] 5.1f scrub starts 2024-02-23T00:57:03.763 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:02.761449+0000 osd.6 (osd.6) 354 : cluster [DBG] 5.1f scrub ok 2024-02-23T00:57:03.763 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:02.852941+0000 osd.1 (osd.1) 316 : cluster [DBG] 2.4 scrub starts 2024-02-23T00:57:03.763 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:02.855898+0000 osd.1 (osd.1) 317 : cluster [DBG] 2.4 scrub ok 2024-02-23T00:57:03.763 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:03.264149+0000 osd.5 (osd.5) 386 : cluster [DBG] 5.28 scrub starts 2024-02-23T00:57:03.763 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:03.265145+0000 osd.5 (osd.5) 387 : cluster [DBG] 5.28 scrub ok 2024-02-23T00:57:03.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:02.495386+0000 mon.a (mon.0) 3896 : cluster [DBG] osdmap e839: 8 total, 8 up, 8 in 2024-02-23T00:57:03.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:02.760213+0000 osd.6 (osd.6) 353 : cluster [DBG] 5.1f scrub starts 2024-02-23T00:57:03.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:02.761449+0000 osd.6 (osd.6) 354 : cluster [DBG] 5.1f scrub ok 2024-02-23T00:57:03.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:02.852941+0000 osd.1 (osd.1) 316 : cluster [DBG] 2.4 scrub starts 2024-02-23T00:57:03.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:02.855898+0000 osd.1 (osd.1) 317 : cluster [DBG] 2.4 scrub ok 2024-02-23T00:57:03.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:03.264149+0000 osd.5 (osd.5) 386 : cluster [DBG] 5.28 scrub starts 2024-02-23T00:57:03.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:03.265145+0000 osd.5 (osd.5) 387 : cluster [DBG] 5.28 scrub ok 2024-02-23T00:57:04.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:03.050855+0000 mgr.y (mgr.14517) 717 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:57:04.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:03.102930+0000 osd.0 (osd.0) 326 : cluster [DBG] 6.0 scrub starts 2024-02-23T00:57:04.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:03.104551+0000 osd.0 (osd.0) 327 : cluster [DBG] 6.0 scrub ok 2024-02-23T00:57:04.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:03.424011+0000 mgr.y (mgr.14517) 718 : cluster [DBG] pgmap v1290: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 2 op/s; 646 B/s, 2 objects/s recovering 2024-02-23T00:57:04.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:03.050855+0000 mgr.y (mgr.14517) 717 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:57:04.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:03.102930+0000 osd.0 (osd.0) 326 : cluster [DBG] 6.0 scrub starts 2024-02-23T00:57:04.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:03.104551+0000 osd.0 (osd.0) 327 : cluster [DBG] 6.0 scrub ok 2024-02-23T00:57:04.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:03.424011+0000 mgr.y (mgr.14517) 718 : cluster [DBG] pgmap v1290: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 2 op/s; 646 B/s, 2 objects/s recovering 2024-02-23T00:57:04.574 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:57:04.575 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:57:04.592 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:03.050855+0000 mgr.y (mgr.14517) 717 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:57:04.593 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:03.102930+0000 osd.0 (osd.0) 326 : cluster [DBG] 6.0 scrub starts 2024-02-23T00:57:04.593 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:03.104551+0000 osd.0 (osd.0) 327 : cluster [DBG] 6.0 scrub ok 2024-02-23T00:57:04.593 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:03.424011+0000 mgr.y (mgr.14517) 718 : cluster [DBG] pgmap v1290: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 2 op/s; 646 B/s, 2 objects/s recovering 2024-02-23T00:57:05.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.202+0000 7f878d7b5700 1 -- 172.21.15.47:0/1822754131 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87880f8cb0 msgr2=0x7f87880f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:05.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.202+0000 7f878d7b5700 1 --2- 172.21.15.47:0/1822754131 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87880f8cb0 0x7f87880f9090 secure :-1 s=READY pgs=2844 cs=0 l=1 rev1=1 rx=0x7f8778004870 tx=0x7f8778009cb0).stop 2024-02-23T00:57:05.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.202+0000 7f878d7b5700 1 -- 172.21.15.47:0/1822754131 shutdown_connections 2024-02-23T00:57:05.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.202+0000 7f878d7b5700 1 --2- 172.21.15.47:0/1822754131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8788102210 0x7f8788104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:05.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.202+0000 7f878d7b5700 1 --2- 172.21.15.47:0/1822754131 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87880f8cb0 0x7f87880f9090 unknown :-1 s=CLOSED pgs=2844 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:05.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.202+0000 7f878d7b5700 1 --2- 172.21.15.47:0/1822754131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87880f9610 0x7f8788101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:05.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.202+0000 7f878d7b5700 1 -- 172.21.15.47:0/1822754131 >> 172.21.15.47:0/1822754131 conn(0x7f87880f2820 msgr2=0x7f87880f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:05.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.202+0000 7f878d7b5700 1 -- 172.21.15.47:0/1822754131 shutdown_connections 2024-02-23T00:57:05.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.202+0000 7f878d7b5700 1 -- 172.21.15.47:0/1822754131 wait complete. 2024-02-23T00:57:05.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.202+0000 7f878d7b5700 1 librados: starting msgr at 2024-02-23T00:57:05.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.202+0000 7f878d7b5700 1 librados: starting objecter 2024-02-23T00:57:05.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.203+0000 7f878d7b5700 1 Processor -- start 2024-02-23T00:57:05.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.203+0000 7f878d7b5700 1 -- start start 2024-02-23T00:57:05.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.203+0000 7f878d7b5700 1 librados: setting wanted keys 2024-02-23T00:57:05.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.203+0000 7f878d7b5700 1 librados: calling monclient init 2024-02-23T00:57:05.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.203+0000 7f878d7b5700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87880f8cb0 0x7f87880f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:05.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.203+0000 7f878d7b5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87880f9610 0x7f87880f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:05.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.203+0000 7f878d7b5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8788102210 0x7f87880f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:05.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.203+0000 7f878d7b5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8788116550 con 0x7f8788102210 2024-02-23T00:57:05.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.203+0000 7f878d7b5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87881166c0 con 0x7f87880f9610 2024-02-23T00:57:05.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.203+0000 7f878d7b5700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8788116880 con 0x7f87880f8cb0 2024-02-23T00:57:05.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.203+0000 7f8786ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87880f8cb0 0x7f87880f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:05.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.203+0000 7f8786ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87880f8cb0 0x7f87880f6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:50030/0 (socket says 172.21.15.47:50030) 2024-02-23T00:57:05.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.203+0000 7f8786ffd700 1 -- 172.21.15.47:0/220123456 learned_addr learned my addr 172.21.15.47:0/220123456 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:57:05.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.203+0000 7f87877fe700 1 --2- 172.21.15.47:0/220123456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8788102210 0x7f87880f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:05.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.203+0000 7f87867fc700 1 --2- 172.21.15.47:0/220123456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87880f9610 0x7f87880f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:05.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.203+0000 7f87877fe700 1 -- 172.21.15.47:0/220123456 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87880f8cb0 msgr2=0x7f87880f6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:05.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.204+0000 7f87877fe700 1 --2- 172.21.15.47:0/220123456 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87880f8cb0 0x7f87880f6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:05.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.204+0000 7f87877fe700 1 -- 172.21.15.47:0/220123456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87880f9610 msgr2=0x7f87880f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:05.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.204+0000 7f87877fe700 1 --2- 172.21.15.47:0/220123456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87880f9610 0x7f87880f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:05.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.204+0000 7f87877fe700 1 -- 172.21.15.47:0/220123456 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f877c005040 con 0x7f8788102210 2024-02-23T00:57:05.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.204+0000 7f87867fc700 1 --2- 172.21.15.47:0/220123456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87880f9610 0x7f87880f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:57:05.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.204+0000 7f8786ffd700 1 --2- 172.21.15.47:0/220123456 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87880f8cb0 0x7f87880f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:57:05.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.204+0000 7f87877fe700 1 --2- 172.21.15.47:0/220123456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8788102210 0x7f87880f7640 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 rx=0x7f878000eb40 tx=0x7f878000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:05.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.204+0000 7f876ffff700 10 client.?.objecter ms_handle_connect 0x7f8788102210 2024-02-23T00:57:05.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.204+0000 7f876ffff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:57:05.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.204+0000 7f876ffff700 1 -- 172.21.15.47:0/220123456 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8780009870 con 0x7f8788102210 2024-02-23T00:57:05.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.204+0000 7f876ffff700 1 -- 172.21.15.47:0/220123456 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f878000f040 con 0x7f8788102210 2024-02-23T00:57:05.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.204+0000 7f876ffff700 1 -- 172.21.15.47:0/220123456 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8780007c30 con 0x7f8788102210 2024-02-23T00:57:05.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.204+0000 7f878d7b5700 1 -- 172.21.15.47:0/220123456 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8778005040 con 0x7f8788102210 2024-02-23T00:57:05.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.204+0000 7f878d7b5700 10 client.31965.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:57:05.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.204+0000 7f878d7b5700 1 -- 172.21.15.47:0/220123456 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8788116e60 con 0x7f8788102210 2024-02-23T00:57:05.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.204+0000 7f878d7b5700 1 librados: init done 2024-02-23T00:57:05.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.205+0000 7f878d7b5700 1 -- 172.21.15.47:0/220123456 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f878805fa60 con 0x7f8788102210 2024-02-23T00:57:05.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.205+0000 7f876ffff700 1 -- 172.21.15.47:0/220123456 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f87800040b0 con 0x7f8788102210 2024-02-23T00:57:05.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.206+0000 7f876ffff700 1 --2- 172.21.15.47:0/220123456 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8770062cc0 0x7f8770065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:05.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.206+0000 7f876ffff700 1 -- 172.21.15.47:0/220123456 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(839..839 src has 670..839) v4 ==== 8663+0+0 (secure 0 0 0) 0x7f878007d070 con 0x7f8788102210 2024-02-23T00:57:05.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.206+0000 7f876ffff700 10 client.31965.objecter ms_dispatch 0x7f8788001550 osd_map(839..839 src has 670..839) v4 2024-02-23T00:57:05.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.206+0000 7f876ffff700 3 client.31965.objecter handle_osd_map got epochs [839,839] > 0 2024-02-23T00:57:05.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.206+0000 7f876ffff700 3 client.31965.objecter handle_osd_map decoding full epoch 839 2024-02-23T00:57:05.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.206+0000 7f8786ffd700 1 --2- 172.21.15.47:0/220123456 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8770062cc0 0x7f8770065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:05.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.206+0000 7f876ffff700 20 client.31965.objecter dump_active .. 0 homeless 2024-02-23T00:57:05.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.206+0000 7f876ffff700 10 client.31965.objecter ms_handle_connect 0x7f8770062cc0 2024-02-23T00:57:05.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.206+0000 7f8786ffd700 1 --2- 172.21.15.47:0/220123456 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8770062cc0 0x7f8770065180 secure :-1 s=READY pgs=3231 cs=0 l=1 rev1=1 rx=0x7f8778007fd0 tx=0x7f877800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:05.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.210+0000 7f876ffff700 1 -- 172.21.15.47:0/220123456 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f8780049f60 con 0x7f8788102210 2024-02-23T00:57:05.242 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.4 dump_historic_ops 2024-02-23T00:57:05.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:05.401+0000 7f878d7b5700 1 -- 172.21.15.47:0/220123456 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 2, "weight": 1.0} v 0) v1 -- 0x7f87880f8140 con 0x7f8788102210 2024-02-23T00:57:05.516 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:57:05.770 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:05.245508+0000 osd.5 (osd.5) 388 : cluster [DBG] 18.f scrub starts 2024-02-23T00:57:05.770 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:05.249825+0000 osd.5 (osd.5) 389 : cluster [DBG] 18.f scrub ok 2024-02-23T00:57:05.771 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:05.342587+0000 mon.a (mon.0) 3897 : cluster [DBG] osdmap e840: 8 total, 8 up, 8 in 2024-02-23T00:57:05.771 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:05.403184+0000 mon.a (mon.0) 3898 : audit [INF] from='client.? 172.21.15.47:0/220123456' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-02-23T00:57:05.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:05.245508+0000 osd.5 (osd.5) 388 : cluster [DBG] 18.f scrub starts 2024-02-23T00:57:05.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:05.249825+0000 osd.5 (osd.5) 389 : cluster [DBG] 18.f scrub ok 2024-02-23T00:57:05.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:05.342587+0000 mon.a (mon.0) 3897 : cluster [DBG] osdmap e840: 8 total, 8 up, 8 in 2024-02-23T00:57:05.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:05.403184+0000 mon.a (mon.0) 3898 : audit [INF] from='client.? 172.21.15.47:0/220123456' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-02-23T00:57:05.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:05.245508+0000 osd.5 (osd.5) 388 : cluster [DBG] 18.f scrub starts 2024-02-23T00:57:05.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:05.249825+0000 osd.5 (osd.5) 389 : cluster [DBG] 18.f scrub ok 2024-02-23T00:57:05.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:05.342587+0000 mon.a (mon.0) 3897 : cluster [DBG] osdmap e840: 8 total, 8 up, 8 in 2024-02-23T00:57:05.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:05.403184+0000 mon.a (mon.0) 3898 : audit [INF] from='client.? 172.21.15.47:0/220123456' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-02-23T00:57:06.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:06.350+0000 7f876ffff700 1 -- 172.21.15.47:0/220123456 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]=0 set osd.2 primary-affinity to 1 (8655362) v841) v1 ==== 133+0+0 (secure 0 0 0) 0x7f878004c8d0 con 0x7f8788102210 2024-02-23T00:57:06.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:06.352+0000 7f878d7b5700 10 librados: watch_flush enter 2024-02-23T00:57:06.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:06.352+0000 7f878d7b5700 10 librados: watch_flush exit 2024-02-23T00:57:06.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:06.352+0000 7f878d7b5700 20 client.31965.objecter shutdown clearing up homeless session... 2024-02-23T00:57:06.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:06.352+0000 7f878d7b5700 10 client.31965.objecter successfully canceled tick 2024-02-23T00:57:06.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:06.353+0000 7f878d7b5700 1 -- 172.21.15.47:0/220123456 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8770062cc0 msgr2=0x7f8770065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:06.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:06.353+0000 7f878d7b5700 1 --2- 172.21.15.47:0/220123456 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8770062cc0 0x7f8770065180 secure :-1 s=READY pgs=3231 cs=0 l=1 rev1=1 rx=0x7f8778007fd0 tx=0x7f877800b040).stop 2024-02-23T00:57:06.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:06.353+0000 7f878d7b5700 1 -- 172.21.15.47:0/220123456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8788102210 msgr2=0x7f87880f7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:06.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:06.353+0000 7f878d7b5700 1 --2- 172.21.15.47:0/220123456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8788102210 0x7f87880f7640 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 rx=0x7f878000eb40 tx=0x7f878000b040).stop 2024-02-23T00:57:06.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:06.353+0000 7f878d7b5700 1 -- 172.21.15.47:0/220123456 shutdown_connections 2024-02-23T00:57:06.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:06.353+0000 7f878d7b5700 1 --2- 172.21.15.47:0/220123456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8788102210 0x7f87880f7640 unknown :-1 s=CLOSED pgs=2668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:06.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:06.353+0000 7f878d7b5700 1 --2- 172.21.15.47:0/220123456 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8770062cc0 0x7f8770065180 unknown :-1 s=CLOSED pgs=3231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:06.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:06.353+0000 7f878d7b5700 1 --2- 172.21.15.47:0/220123456 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f87880f8cb0 0x7f87880f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:06.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:06.353+0000 7f878d7b5700 1 --2- 172.21.15.47:0/220123456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87880f9610 0x7f87880f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:06.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:06.353+0000 7f878d7b5700 1 -- 172.21.15.47:0/220123456 >> 172.21.15.47:0/220123456 conn(0x7f87880f2820 msgr2=0x7f87880f34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:06.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:06.353+0000 7f878d7b5700 1 -- 172.21.15.47:0/220123456 shutdown_connections 2024-02-23T00:57:06.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:06.353+0000 7f878d7b5700 1 -- 172.21.15.47:0/220123456 wait complete. 2024-02-23T00:57:06.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:06.353+0000 7f878d7b5700 1 librados: shutdown 2024-02-23T00:57:06.360 INFO:teuthology.orchestra.run.smithi047.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-02-23T00:57:06.730 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:05.128037+0000 osd.7 (osd.7) 399 : cluster [DBG] 6.b scrub starts 2024-02-23T00:57:06.730 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:05.131129+0000 osd.7 (osd.7) 400 : cluster [DBG] 6.b scrub ok 2024-02-23T00:57:06.730 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:05.424399+0000 mgr.y (mgr.14517) 719 : cluster [DBG] pgmap v1292: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 2 op/s; 646 B/s, 2 objects/s recovering 2024-02-23T00:57:06.730 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:05.806433+0000 osd.5 (osd.5) 390 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:57:06.730 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:05.806583+0000 osd.5 (osd.5) 391 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:57:06.731 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:06.349912+0000 mon.a (mon.0) 3899 : audit [INF] from='client.? 172.21.15.47:0/220123456' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-02-23T00:57:06.731 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:06.350016+0000 mon.a (mon.0) 3900 : cluster [DBG] osdmap e841: 8 total, 8 up, 8 in 2024-02-23T00:57:06.731 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:05.128037+0000 osd.7 (osd.7) 399 : cluster [DBG] 6.b scrub starts 2024-02-23T00:57:06.731 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:05.131129+0000 osd.7 (osd.7) 400 : cluster [DBG] 6.b scrub ok 2024-02-23T00:57:06.731 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:05.424399+0000 mgr.y (mgr.14517) 719 : cluster [DBG] pgmap v1292: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 2 op/s; 646 B/s, 2 objects/s recovering 2024-02-23T00:57:06.731 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:05.806433+0000 osd.5 (osd.5) 390 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:57:06.731 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:05.806583+0000 osd.5 (osd.5) 391 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:57:06.731 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:06.349912+0000 mon.a (mon.0) 3899 : audit [INF] from='client.? 172.21.15.47:0/220123456' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-02-23T00:57:06.731 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:06.350016+0000 mon.a (mon.0) 3900 : cluster [DBG] osdmap e841: 8 total, 8 up, 8 in 2024-02-23T00:57:06.838 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:05.128037+0000 osd.7 (osd.7) 399 : cluster [DBG] 6.b scrub starts 2024-02-23T00:57:06.838 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:05.131129+0000 osd.7 (osd.7) 400 : cluster [DBG] 6.b scrub ok 2024-02-23T00:57:06.838 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:05.424399+0000 mgr.y (mgr.14517) 719 : cluster [DBG] pgmap v1292: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 2 op/s; 646 B/s, 2 objects/s recovering 2024-02-23T00:57:06.838 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:05.806433+0000 osd.5 (osd.5) 390 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:57:06.838 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:05.806583+0000 osd.5 (osd.5) 391 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:57:06.839 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:06.349912+0000 mon.a (mon.0) 3899 : audit [INF] from='client.? 172.21.15.47:0/220123456' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-02-23T00:57:06.839 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:06.350016+0000 mon.a (mon.0) 3900 : cluster [DBG] osdmap e841: 8 total, 8 up, 8 in 2024-02-23T00:57:07.153 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight 3 1 2024-02-23T00:57:07.294 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:57:07.294 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:57:07.437 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:57:07.514 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:06.486766+0000 osd.4 (osd.4) 379 : cluster [DBG] 18.19 scrub starts 2024-02-23T00:57:07.514 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:06.488482+0000 osd.4 (osd.4) 380 : cluster [DBG] 18.19 scrub ok 2024-02-23T00:57:07.514 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:06.486766+0000 osd.4 (osd.4) 379 : cluster [DBG] 18.19 scrub starts 2024-02-23T00:57:07.514 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:06.488482+0000 osd.4 (osd.4) 380 : cluster [DBG] 18.19 scrub ok 2024-02-23T00:57:07.690 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:06.486766+0000 osd.4 (osd.4) 379 : cluster [DBG] 18.19 scrub starts 2024-02-23T00:57:07.690 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:06.488482+0000 osd.4 (osd.4) 380 : cluster [DBG] 18.19 scrub ok 2024-02-23T00:57:08.030 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.5 dump_ops_in_flight 2024-02-23T00:57:08.300 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:57:08.678 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:07.137782+0000 osd.7 (osd.7) 401 : cluster [DBG] 5.1a scrub starts 2024-02-23T00:57:08.678 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:07.140672+0000 osd.7 (osd.7) 402 : cluster [DBG] 5.1a scrub ok 2024-02-23T00:57:08.678 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:07.425053+0000 mgr.y (mgr.14517) 720 : cluster [DBG] pgmap v1294: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s; 646 B/s, 2 objects/s recovering 2024-02-23T00:57:08.678 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:07.137782+0000 osd.7 (osd.7) 401 : cluster [DBG] 5.1a scrub starts 2024-02-23T00:57:08.678 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:07.140672+0000 osd.7 (osd.7) 402 : cluster [DBG] 5.1a scrub ok 2024-02-23T00:57:08.678 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:07.425053+0000 mgr.y (mgr.14517) 720 : cluster [DBG] pgmap v1294: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s; 646 B/s, 2 objects/s recovering 2024-02-23T00:57:08.774 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:07.137782+0000 osd.7 (osd.7) 401 : cluster [DBG] 5.1a scrub starts 2024-02-23T00:57:08.774 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:07.140672+0000 osd.7 (osd.7) 402 : cluster [DBG] 5.1a scrub ok 2024-02-23T00:57:08.774 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:07.425053+0000 mgr.y (mgr.14517) 720 : cluster [DBG] pgmap v1294: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s; 646 B/s, 2 objects/s recovering 2024-02-23T00:57:09.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.181+0000 7f8e41e71700 1 -- 172.21.15.47:0/982882309 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8e3c0fe510 msgr2=0x7f8e3c0f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:09.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.181+0000 7f8e41e71700 1 --2- 172.21.15.47:0/982882309 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8e3c0fe510 0x7f8e3c0f7d40 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 rx=0x7f8e30004870 tx=0x7f8e30009cb0).stop 2024-02-23T00:57:09.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.181+0000 7f8e41e71700 1 -- 172.21.15.47:0/982882309 shutdown_connections 2024-02-23T00:57:09.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.181+0000 7f8e41e71700 1 --2- 172.21.15.47:0/982882309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e3c0f8420 0x7f8e3c104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:09.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.181+0000 7f8e41e71700 1 --2- 172.21.15.47:0/982882309 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e3c0fdbb0 0x7f8e3c0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:09.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.181+0000 7f8e41e71700 1 --2- 172.21.15.47:0/982882309 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8e3c0fe510 0x7f8e3c0f7d40 unknown :-1 s=CLOSED pgs=2845 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:09.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.181+0000 7f8e41e71700 1 -- 172.21.15.47:0/982882309 >> 172.21.15.47:0/982882309 conn(0x7f8e3c0f3800 msgr2=0x7f8e3c0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:09.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.181+0000 7f8e41e71700 1 -- 172.21.15.47:0/982882309 shutdown_connections 2024-02-23T00:57:09.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.181+0000 7f8e41e71700 1 -- 172.21.15.47:0/982882309 wait complete. 2024-02-23T00:57:09.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.182+0000 7f8e41e71700 1 librados: starting msgr at 2024-02-23T00:57:09.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.182+0000 7f8e41e71700 1 librados: starting objecter 2024-02-23T00:57:09.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.182+0000 7f8e41e71700 1 Processor -- start 2024-02-23T00:57:09.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.182+0000 7f8e41e71700 1 -- start start 2024-02-23T00:57:09.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.182+0000 7f8e41e71700 1 librados: setting wanted keys 2024-02-23T00:57:09.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.182+0000 7f8e41e71700 1 librados: calling monclient init 2024-02-23T00:57:09.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.182+0000 7f8e41e71700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e3c0f8420 0x7f8e3c10dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:09.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.182+0000 7f8e41e71700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e3c0fdbb0 0x7f8e3c10e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:09.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.182+0000 7f8e41e71700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8e3c0fe510 0x7f8e3c108060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:09.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.182+0000 7f8e41e71700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c108550 con 0x7f8e3c0f8420 2024-02-23T00:57:09.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.182+0000 7f8e41e71700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c1086c0 con 0x7f8e3c0fdbb0 2024-02-23T00:57:09.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.182+0000 7f8e41e71700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c108880 con 0x7f8e3c0fe510 2024-02-23T00:57:09.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.183+0000 7f8e3b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e3c0f8420 0x7f8e3c10dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:09.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.183+0000 7f8e3affd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e3c0fdbb0 0x7f8e3c10e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:09.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.183+0000 7f8e3bfff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8e3c0fe510 0x7f8e3c108060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:09.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.183+0000 7f8e3b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e3c0f8420 0x7f8e3c10dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58352/0 (socket says 172.21.15.47:58352) 2024-02-23T00:57:09.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.183+0000 7f8e3bfff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8e3c0fe510 0x7f8e3c108060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:50982/0 (socket says 172.21.15.47:50982) 2024-02-23T00:57:09.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.183+0000 7f8e3bfff700 1 -- 172.21.15.47:0/1162325500 learned_addr learned my addr 172.21.15.47:0/1162325500 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:57:09.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.183+0000 7f8e3affd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e3c0fdbb0 0x7f8e3c10e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.47:53276/0 (socket says 172.21.15.47:53276) 2024-02-23T00:57:09.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.183+0000 7f8e3bfff700 1 -- 172.21.15.47:0/1162325500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e3c0fdbb0 msgr2=0x7f8e3c10e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:09.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.183+0000 7f8e3bfff700 1 --2- 172.21.15.47:0/1162325500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e3c0fdbb0 0x7f8e3c10e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:09.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.183+0000 7f8e3bfff700 1 -- 172.21.15.47:0/1162325500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e3c0f8420 msgr2=0x7f8e3c10dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:09.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.183+0000 7f8e3bfff700 1 --2- 172.21.15.47:0/1162325500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e3c0f8420 0x7f8e3c10dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:09.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.183+0000 7f8e3bfff700 1 -- 172.21.15.47:0/1162325500 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2c005040 con 0x7f8e3c0fe510 2024-02-23T00:57:09.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.183+0000 7f8e3bfff700 1 --2- 172.21.15.47:0/1162325500 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8e3c0fe510 0x7f8e3c108060 secure :-1 s=READY pgs=2846 cs=0 l=1 rev1=1 rx=0x7f8e3400ec20 tx=0x7f8e3400b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:09.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.183+0000 7f8e38ff9700 10 client.?.objecter ms_handle_connect 0x7f8e3c0fe510 2024-02-23T00:57:09.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.183+0000 7f8e38ff9700 10 client.?.objecter resend_mon_ops 2024-02-23T00:57:09.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.183+0000 7f8e38ff9700 1 -- 172.21.15.47:0/1162325500 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e34009950 con 0x7f8e3c0fe510 2024-02-23T00:57:09.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.183+0000 7f8e38ff9700 1 -- 172.21.15.47:0/1162325500 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f8e3400f040 con 0x7f8e3c0fe510 2024-02-23T00:57:09.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.183+0000 7f8e41e71700 1 -- 172.21.15.47:0/1162325500 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e30005040 con 0x7f8e3c0fe510 2024-02-23T00:57:09.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.183+0000 7f8e38ff9700 1 -- 172.21.15.47:0/1162325500 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e34007d10 con 0x7f8e3c0fe510 2024-02-23T00:57:09.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.183+0000 7f8e41e71700 10 client.42293.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:57:09.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.183+0000 7f8e41e71700 1 -- 172.21.15.47:0/1162325500 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e3c108e30 con 0x7f8e3c0fe510 2024-02-23T00:57:09.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.183+0000 7f8e41e71700 1 librados: init done 2024-02-23T00:57:09.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.184+0000 7f8e41e71700 1 -- 172.21.15.47:0/1162325500 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e3c05ee80 con 0x7f8e3c0fe510 2024-02-23T00:57:09.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.184+0000 7f8e38ff9700 1 -- 172.21.15.47:0/1162325500 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f8e340092e0 con 0x7f8e3c0fe510 2024-02-23T00:57:09.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.185+0000 7f8e38ff9700 1 --2- 172.21.15.47:0/1162325500 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8e24062cc0 0x7f8e24065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:09.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.185+0000 7f8e38ff9700 1 -- 172.21.15.47:0/1162325500 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(841..841 src has 670..841) v4 ==== 8663+0+0 (secure 0 0 0) 0x7f8e3407d0d0 con 0x7f8e3c0fe510 2024-02-23T00:57:09.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.185+0000 7f8e38ff9700 10 client.42293.objecter ms_dispatch 0x7f8e3c001550 osd_map(841..841 src has 670..841) v4 2024-02-23T00:57:09.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.185+0000 7f8e38ff9700 3 client.42293.objecter handle_osd_map got epochs [841,841] > 0 2024-02-23T00:57:09.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.185+0000 7f8e38ff9700 3 client.42293.objecter handle_osd_map decoding full epoch 841 2024-02-23T00:57:09.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.185+0000 7f8e3b7fe700 1 --2- 172.21.15.47:0/1162325500 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8e24062cc0 0x7f8e24065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:09.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.185+0000 7f8e38ff9700 20 client.42293.objecter dump_active .. 0 homeless 2024-02-23T00:57:09.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.185+0000 7f8e38ff9700 10 client.42293.objecter ms_handle_connect 0x7f8e24062cc0 2024-02-23T00:57:09.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.185+0000 7f8e3b7fe700 1 --2- 172.21.15.47:0/1162325500 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8e24062cc0 0x7f8e24065180 secure :-1 s=READY pgs=3232 cs=0 l=1 rev1=1 rx=0x7f8e2c009350 tx=0x7f8e2c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:09.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.189+0000 7f8e38ff9700 1 -- 172.21.15.47:0/1162325500 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f8e34049f60 con 0x7f8e3c0fe510 2024-02-23T00:57:09.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.374+0000 7f8e41e71700 1 -- 172.21.15.47:0/1162325500 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd reweight", "id": 3, "weight": 1.0} v 0) v1 -- 0x7f8e3c109690 con 0x7f8e3c0fe510 2024-02-23T00:57:09.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.515+0000 7f8e38ff9700 1 -- 172.21.15.47:0/1162325500 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 3, "weight": 1.0}]=0 reweighted osd.3 to 1 (10000) v842) v1 ==== 113+0+0 (secure 0 0 0) 0x7f8e3c109690 con 0x7f8e3c0fe510 2024-02-23T00:57:09.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.517+0000 7f8e41e71700 10 librados: watch_flush enter 2024-02-23T00:57:09.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.517+0000 7f8e41e71700 10 librados: watch_flush exit 2024-02-23T00:57:09.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.517+0000 7f8e41e71700 20 client.42293.objecter shutdown clearing up homeless session... 2024-02-23T00:57:09.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.517+0000 7f8e41e71700 10 client.42293.objecter successfully canceled tick 2024-02-23T00:57:09.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.517+0000 7f8e41e71700 1 -- 172.21.15.47:0/1162325500 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8e24062cc0 msgr2=0x7f8e24065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:09.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.517+0000 7f8e41e71700 1 --2- 172.21.15.47:0/1162325500 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8e24062cc0 0x7f8e24065180 secure :-1 s=READY pgs=3232 cs=0 l=1 rev1=1 rx=0x7f8e2c009350 tx=0x7f8e2c00b040).stop 2024-02-23T00:57:09.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.517+0000 7f8e41e71700 1 -- 172.21.15.47:0/1162325500 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8e3c0fe510 msgr2=0x7f8e3c108060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:09.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.517+0000 7f8e41e71700 1 --2- 172.21.15.47:0/1162325500 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8e3c0fe510 0x7f8e3c108060 secure :-1 s=READY pgs=2846 cs=0 l=1 rev1=1 rx=0x7f8e3400ec20 tx=0x7f8e3400b040).stop 2024-02-23T00:57:09.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.518+0000 7f8e41e71700 1 -- 172.21.15.47:0/1162325500 shutdown_connections 2024-02-23T00:57:09.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.518+0000 7f8e41e71700 1 --2- 172.21.15.47:0/1162325500 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f8e24062cc0 0x7f8e24065180 unknown :-1 s=CLOSED pgs=3232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:09.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.518+0000 7f8e41e71700 1 --2- 172.21.15.47:0/1162325500 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8e3c0fe510 0x7f8e3c108060 unknown :-1 s=CLOSED pgs=2846 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:09.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.518+0000 7f8e41e71700 1 --2- 172.21.15.47:0/1162325500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e3c0f8420 0x7f8e3c10dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:09.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.518+0000 7f8e41e71700 1 --2- 172.21.15.47:0/1162325500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e3c0fdbb0 0x7f8e3c10e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:09.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.518+0000 7f8e41e71700 1 -- 172.21.15.47:0/1162325500 >> 172.21.15.47:0/1162325500 conn(0x7f8e3c0f3800 msgr2=0x7f8e3c0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:09.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.518+0000 7f8e41e71700 1 -- 172.21.15.47:0/1162325500 shutdown_connections 2024-02-23T00:57:09.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.518+0000 7f8e41e71700 1 -- 172.21.15.47:0/1162325500 wait complete. 2024-02-23T00:57:09.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:09.518+0000 7f8e41e71700 1 librados: shutdown 2024-02-23T00:57:09.522 INFO:teuthology.orchestra.run.smithi047.stderr:reweighted osd.3 to 1 (10000) 2024-02-23T00:57:09.776 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:08.860525+0000 osd.1 (osd.1) 318 : cluster [DBG] 2.15 scrub starts 2024-02-23T00:57:09.776 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:08.862163+0000 osd.1 (osd.1) 319 : cluster [DBG] 2.15 scrub ok 2024-02-23T00:57:09.776 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:09.375455+0000 mon.c (mon.2) 527 : audit [INF] from='client.? 172.21.15.47:0/1162325500' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-23T00:57:09.776 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:09.375759+0000 mon.a (mon.0) 3901 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-23T00:57:09.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:08.860525+0000 osd.1 (osd.1) 318 : cluster [DBG] 2.15 scrub starts 2024-02-23T00:57:09.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:08.862163+0000 osd.1 (osd.1) 319 : cluster [DBG] 2.15 scrub ok 2024-02-23T00:57:09.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:09.375455+0000 mon.c (mon.2) 527 : audit [INF] from='client.? 172.21.15.47:0/1162325500' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-23T00:57:09.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:09.375759+0000 mon.a (mon.0) 3901 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-23T00:57:09.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:08.860525+0000 osd.1 (osd.1) 318 : cluster [DBG] 2.15 scrub starts 2024-02-23T00:57:09.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:08.862163+0000 osd.1 (osd.1) 319 : cluster [DBG] 2.15 scrub ok 2024-02-23T00:57:09.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:09.375455+0000 mon.c (mon.2) 527 : audit [INF] from='client.? 172.21.15.47:0/1162325500' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-23T00:57:09.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:09.375759+0000 mon.a (mon.0) 3901 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-23T00:57:10.045 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:57:10.045 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:57:10.181 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd primary-affinity 3 1 2024-02-23T00:57:10.464 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:57:10.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:09.139063+0000 osd.7 (osd.7) 403 : cluster [DBG] 6.3 scrub starts 2024-02-23T00:57:10.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:09.141983+0000 osd.7 (osd.7) 404 : cluster [DBG] 6.3 scrub ok 2024-02-23T00:57:10.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:09.425567+0000 mgr.y (mgr.14517) 721 : cluster [DBG] pgmap v1295: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 738 B/s rd, 0 op/s 2024-02-23T00:57:10.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:09.516176+0000 mon.a (mon.0) 3902 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-23T00:57:10.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:09.516256+0000 mon.a (mon.0) 3903 : cluster [DBG] osdmap e842: 8 total, 8 up, 8 in 2024-02-23T00:57:10.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:09.749743+0000 osd.6 (osd.6) 355 : cluster [DBG] 5.21 scrub starts 2024-02-23T00:57:10.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:09.751385+0000 osd.6 (osd.6) 356 : cluster [DBG] 5.21 scrub ok 2024-02-23T00:57:10.723 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:09.139063+0000 osd.7 (osd.7) 403 : cluster [DBG] 6.3 scrub starts 2024-02-23T00:57:10.723 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:09.141983+0000 osd.7 (osd.7) 404 : cluster [DBG] 6.3 scrub ok 2024-02-23T00:57:10.723 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:09.425567+0000 mgr.y (mgr.14517) 721 : cluster [DBG] pgmap v1295: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 738 B/s rd, 0 op/s 2024-02-23T00:57:10.723 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:09.516176+0000 mon.a (mon.0) 3902 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-23T00:57:10.723 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:09.516256+0000 mon.a (mon.0) 3903 : cluster [DBG] osdmap e842: 8 total, 8 up, 8 in 2024-02-23T00:57:10.723 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:09.749743+0000 osd.6 (osd.6) 355 : cluster [DBG] 5.21 scrub starts 2024-02-23T00:57:10.723 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:09.751385+0000 osd.6 (osd.6) 356 : cluster [DBG] 5.21 scrub ok 2024-02-23T00:57:10.724 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:09.139063+0000 osd.7 (osd.7) 403 : cluster [DBG] 6.3 scrub starts 2024-02-23T00:57:10.724 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:09.141983+0000 osd.7 (osd.7) 404 : cluster [DBG] 6.3 scrub ok 2024-02-23T00:57:10.724 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:09.425567+0000 mgr.y (mgr.14517) 721 : cluster [DBG] pgmap v1295: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 738 B/s rd, 0 op/s 2024-02-23T00:57:10.724 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:09.516176+0000 mon.a (mon.0) 3902 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-23T00:57:10.724 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:09.516256+0000 mon.a (mon.0) 3903 : cluster [DBG] osdmap e842: 8 total, 8 up, 8 in 2024-02-23T00:57:10.724 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:09.749743+0000 osd.6 (osd.6) 355 : cluster [DBG] 5.21 scrub starts 2024-02-23T00:57:10.724 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:09.751385+0000 osd.6 (osd.6) 356 : cluster [DBG] 5.21 scrub ok 2024-02-23T00:57:10.725 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:57:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:57:10] "GET /metrics HTTP/1.1" 200 187130 "" "Prometheus/2.33.4" 2024-02-23T00:57:10.743 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.5 dump_blocked_ops 2024-02-23T00:57:11.008 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:57:11.775 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:10.137876+0000 osd.7 (osd.7) 405 : cluster [DBG] 6.5 scrub starts 2024-02-23T00:57:11.775 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:10.141031+0000 osd.7 (osd.7) 406 : cluster [DBG] 6.5 scrub ok 2024-02-23T00:57:11.776 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:10.144865+0000 osd.0 (osd.0) 328 : cluster [DBG] 18.e deep-scrub starts 2024-02-23T00:57:11.776 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:10.145987+0000 osd.0 (osd.0) 329 : cluster [DBG] 18.e deep-scrub ok 2024-02-23T00:57:11.776 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:10.872908+0000 osd.1 (osd.1) 320 : cluster [DBG] 6.4 scrub starts 2024-02-23T00:57:11.776 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:10.876406+0000 osd.1 (osd.1) 321 : cluster [DBG] 6.4 scrub ok 2024-02-23T00:57:11.776 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:10.985121+0000 osd.2 (osd.2) 201 : cluster [DBG] 18.7 deep-scrub starts 2024-02-23T00:57:11.776 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:10.986962+0000 osd.2 (osd.2) 202 : cluster [DBG] 18.7 deep-scrub ok 2024-02-23T00:57:11.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:10.137876+0000 osd.7 (osd.7) 405 : cluster [DBG] 6.5 scrub starts 2024-02-23T00:57:11.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:10.141031+0000 osd.7 (osd.7) 406 : cluster [DBG] 6.5 scrub ok 2024-02-23T00:57:11.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:10.144865+0000 osd.0 (osd.0) 328 : cluster [DBG] 18.e deep-scrub starts 2024-02-23T00:57:11.777 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:10.145987+0000 osd.0 (osd.0) 329 : cluster [DBG] 18.e deep-scrub ok 2024-02-23T00:57:11.777 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:10.872908+0000 osd.1 (osd.1) 320 : cluster [DBG] 6.4 scrub starts 2024-02-23T00:57:11.777 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:10.876406+0000 osd.1 (osd.1) 321 : cluster [DBG] 6.4 scrub ok 2024-02-23T00:57:11.777 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:10.985121+0000 osd.2 (osd.2) 201 : cluster [DBG] 18.7 deep-scrub starts 2024-02-23T00:57:11.777 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:10.986962+0000 osd.2 (osd.2) 202 : cluster [DBG] 18.7 deep-scrub ok 2024-02-23T00:57:11.901 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:10.137876+0000 osd.7 (osd.7) 405 : cluster [DBG] 6.5 scrub starts 2024-02-23T00:57:11.902 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:10.141031+0000 osd.7 (osd.7) 406 : cluster [DBG] 6.5 scrub ok 2024-02-23T00:57:11.902 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:10.144865+0000 osd.0 (osd.0) 328 : cluster [DBG] 18.e deep-scrub starts 2024-02-23T00:57:11.902 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:10.145987+0000 osd.0 (osd.0) 329 : cluster [DBG] 18.e deep-scrub ok 2024-02-23T00:57:11.902 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:10.872908+0000 osd.1 (osd.1) 320 : cluster [DBG] 6.4 scrub starts 2024-02-23T00:57:11.902 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:10.876406+0000 osd.1 (osd.1) 321 : cluster [DBG] 6.4 scrub ok 2024-02-23T00:57:11.902 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:10.985121+0000 osd.2 (osd.2) 201 : cluster [DBG] 18.7 deep-scrub starts 2024-02-23T00:57:11.902 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:10.986962+0000 osd.2 (osd.2) 202 : cluster [DBG] 18.7 deep-scrub ok 2024-02-23T00:57:12.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.267+0000 7f72118cd700 1 -- 172.21.15.47:0/3433482575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f720c0fe510 msgr2=0x7f720c0f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:12.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.267+0000 7f72118cd700 1 --2- 172.21.15.47:0/3433482575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f720c0fe510 0x7f720c0f7d40 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 rx=0x7f72000047b0 tx=0x7f7200009cb0).stop 2024-02-23T00:57:12.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.267+0000 7f72118cd700 1 -- 172.21.15.47:0/3433482575 shutdown_connections 2024-02-23T00:57:12.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.267+0000 7f72118cd700 1 --2- 172.21.15.47:0/3433482575 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f720c0f8420 0x7f720c104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:12.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.267+0000 7f72118cd700 1 --2- 172.21.15.47:0/3433482575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f720c0fdbb0 0x7f720c0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:12.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.267+0000 7f72118cd700 1 --2- 172.21.15.47:0/3433482575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f720c0fe510 0x7f720c0f7d40 unknown :-1 s=CLOSED pgs=2669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:12.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.267+0000 7f72118cd700 1 -- 172.21.15.47:0/3433482575 >> 172.21.15.47:0/3433482575 conn(0x7f720c0f3800 msgr2=0x7f720c0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:12.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.267+0000 7f72118cd700 1 -- 172.21.15.47:0/3433482575 shutdown_connections 2024-02-23T00:57:12.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.267+0000 7f72118cd700 1 -- 172.21.15.47:0/3433482575 wait complete. 2024-02-23T00:57:12.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.268+0000 7f72118cd700 1 librados: starting msgr at 2024-02-23T00:57:12.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.268+0000 7f72118cd700 1 librados: starting objecter 2024-02-23T00:57:12.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.268+0000 7f72118cd700 1 Processor -- start 2024-02-23T00:57:12.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.268+0000 7f72118cd700 1 -- start start 2024-02-23T00:57:12.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.268+0000 7f72118cd700 1 librados: setting wanted keys 2024-02-23T00:57:12.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.268+0000 7f72118cd700 1 librados: calling monclient init 2024-02-23T00:57:12.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.268+0000 7f72118cd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f720c0f8420 0x7f720c10df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:12.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.268+0000 7f72118cd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f720c0fdbb0 0x7f720c10e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:12.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.268+0000 7f72118cd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f720c0fe510 0x7f720c1080e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:12.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.268+0000 7f72118cd700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720c1085d0 con 0x7f720c0fdbb0 2024-02-23T00:57:12.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.268+0000 7f72118cd700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720c108740 con 0x7f720c0f8420 2024-02-23T00:57:12.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.268+0000 7f72118cd700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f720c108900 con 0x7f720c0fe510 2024-02-23T00:57:12.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.268+0000 7f720b7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f720c0fe510 0x7f720c1080e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:12.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.268+0000 7f720affd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f720c0f8420 0x7f720c10df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:12.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.268+0000 7f720a7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f720c0fdbb0 0x7f720c10e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:12.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.269+0000 7f720a7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f720c0fdbb0 0x7f720c10e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58364/0 (socket says 172.21.15.47:58364) 2024-02-23T00:57:12.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.269+0000 7f720a7fc700 1 -- 172.21.15.47:0/2291561282 learned_addr learned my addr 172.21.15.47:0/2291561282 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:57:12.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.269+0000 7f720b7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f720c0fe510 0x7f720c1080e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:51006/0 (socket says 172.21.15.47:51006) 2024-02-23T00:57:12.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.269+0000 7f720affd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f720c0f8420 0x7f720c10df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.47:53292/0 (socket says 172.21.15.47:53292) 2024-02-23T00:57:12.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.269+0000 7f720a7fc700 1 -- 172.21.15.47:0/2291561282 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f720c0fe510 msgr2=0x7f720c1080e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:12.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.269+0000 7f720a7fc700 1 --2- 172.21.15.47:0/2291561282 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f720c0fe510 0x7f720c1080e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:12.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.269+0000 7f720a7fc700 1 -- 172.21.15.47:0/2291561282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f720c0f8420 msgr2=0x7f720c10df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:12.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.269+0000 7f720a7fc700 1 --2- 172.21.15.47:0/2291561282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f720c0f8420 0x7f720c10df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:12.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.269+0000 7f720a7fc700 1 -- 172.21.15.47:0/2291561282 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72040097c0 con 0x7f720c0fdbb0 2024-02-23T00:57:12.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.269+0000 7f720b7fe700 1 --2- 172.21.15.47:0/2291561282 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f720c0fe510 0x7f720c1080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:57:12.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.269+0000 7f720affd700 1 --2- 172.21.15.47:0/2291561282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f720c0f8420 0x7f720c10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:57:12.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.269+0000 7f720a7fc700 1 --2- 172.21.15.47:0/2291561282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f720c0fdbb0 0x7f720c10e430 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 rx=0x7f72000044e0 tx=0x7f7200009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:12.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.269+0000 7f71f3fff700 10 client.?.objecter ms_handle_connect 0x7f720c0fdbb0 2024-02-23T00:57:12.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.269+0000 7f71f3fff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:57:12.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.269+0000 7f71f3fff700 1 -- 172.21.15.47:0/2291561282 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7200018070 con 0x7f720c0fdbb0 2024-02-23T00:57:12.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.269+0000 7f72118cd700 1 -- 172.21.15.47:0/2291561282 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7200005040 con 0x7f720c0fdbb0 2024-02-23T00:57:12.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.269+0000 7f71f3fff700 1 -- 172.21.15.47:0/2291561282 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f7200013e50 con 0x7f720c0fdbb0 2024-02-23T00:57:12.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.269+0000 7f71f3fff700 1 -- 172.21.15.47:0/2291561282 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720001db60 con 0x7f720c0fdbb0 2024-02-23T00:57:12.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.269+0000 7f72118cd700 10 client.31977.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:57:12.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.269+0000 7f72118cd700 1 -- 172.21.15.47:0/2291561282 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f720c108e20 con 0x7f720c0fdbb0 2024-02-23T00:57:12.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.269+0000 7f72118cd700 1 librados: init done 2024-02-23T00:57:12.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.270+0000 7f72118cd700 1 -- 172.21.15.47:0/2291561282 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f720c05ee80 con 0x7f720c0fdbb0 2024-02-23T00:57:12.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.271+0000 7f71f3fff700 1 -- 172.21.15.47:0/2291561282 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f7200009240 con 0x7f720c0fdbb0 2024-02-23T00:57:12.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.271+0000 7f71f3fff700 1 --2- 172.21.15.47:0/2291561282 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f71f4062cc0 0x7f71f4065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:12.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.271+0000 7f71f3fff700 1 -- 172.21.15.47:0/2291561282 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(842..842 src has 670..842) v4 ==== 8663+0+0 (secure 0 0 0) 0x7f720007e8a0 con 0x7f720c0fdbb0 2024-02-23T00:57:12.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.271+0000 7f71f3fff700 10 client.31977.objecter ms_dispatch 0x7f720c001550 osd_map(842..842 src has 670..842) v4 2024-02-23T00:57:12.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.271+0000 7f71f3fff700 3 client.31977.objecter handle_osd_map got epochs [842,842] > 0 2024-02-23T00:57:12.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.271+0000 7f720affd700 1 --2- 172.21.15.47:0/2291561282 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f71f4062cc0 0x7f71f4065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:12.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.271+0000 7f71f3fff700 3 client.31977.objecter handle_osd_map decoding full epoch 842 2024-02-23T00:57:12.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.271+0000 7f71f3fff700 20 client.31977.objecter dump_active .. 0 homeless 2024-02-23T00:57:12.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.271+0000 7f71f3fff700 10 client.31977.objecter ms_handle_connect 0x7f71f4062cc0 2024-02-23T00:57:12.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.271+0000 7f720affd700 1 --2- 172.21.15.47:0/2291561282 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f71f4062cc0 0x7f71f4065180 secure :-1 s=READY pgs=3233 cs=0 l=1 rev1=1 rx=0x7f71fc0103c0 tx=0x7f71fc00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:12.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.275+0000 7f71f3fff700 1 -- 172.21.15.47:0/2291561282 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f720004e0e0 con 0x7f720c0fdbb0 2024-02-23T00:57:12.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.463+0000 7f72118cd700 1 -- 172.21.15.47:0/2291561282 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} v 0) v1 -- 0x7f720c1095c0 con 0x7f720c0fdbb0 2024-02-23T00:57:12.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.526+0000 7f71f3fff700 1 -- 172.21.15.47:0/2291561282 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]=0 set osd.3 primary-affinity to 1 (8655362) v843) v1 ==== 133+0+0 (secure 0 0 0) 0x7f7200051700 con 0x7f720c0fdbb0 2024-02-23T00:57:12.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.531+0000 7f72118cd700 10 librados: watch_flush enter 2024-02-23T00:57:12.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.531+0000 7f72118cd700 10 librados: watch_flush exit 2024-02-23T00:57:12.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.531+0000 7f72118cd700 20 client.31977.objecter shutdown clearing up homeless session... 2024-02-23T00:57:12.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.531+0000 7f72118cd700 10 client.31977.objecter successfully canceled tick 2024-02-23T00:57:12.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.531+0000 7f72118cd700 1 -- 172.21.15.47:0/2291561282 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f71f4062cc0 msgr2=0x7f71f4065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:12.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.531+0000 7f72118cd700 1 --2- 172.21.15.47:0/2291561282 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f71f4062cc0 0x7f71f4065180 secure :-1 s=READY pgs=3233 cs=0 l=1 rev1=1 rx=0x7f71fc0103c0 tx=0x7f71fc00b040).stop 2024-02-23T00:57:12.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.531+0000 7f72118cd700 1 -- 172.21.15.47:0/2291561282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f720c0fdbb0 msgr2=0x7f720c10e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:12.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.531+0000 7f72118cd700 1 --2- 172.21.15.47:0/2291561282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f720c0fdbb0 0x7f720c10e430 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 rx=0x7f72000044e0 tx=0x7f7200009cb0).stop 2024-02-23T00:57:12.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.538+0000 7f72118cd700 1 -- 172.21.15.47:0/2291561282 shutdown_connections 2024-02-23T00:57:12.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.538+0000 7f72118cd700 1 --2- 172.21.15.47:0/2291561282 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f71f4062cc0 0x7f71f4065180 unknown :-1 s=CLOSED pgs=3233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:12.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.538+0000 7f72118cd700 1 --2- 172.21.15.47:0/2291561282 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f720c0fe510 0x7f720c1080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:12.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.538+0000 7f72118cd700 1 --2- 172.21.15.47:0/2291561282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f720c0f8420 0x7f720c10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:12.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.538+0000 7f72118cd700 1 --2- 172.21.15.47:0/2291561282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f720c0fdbb0 0x7f720c10e430 unknown :-1 s=CLOSED pgs=2670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:12.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.538+0000 7f72118cd700 1 -- 172.21.15.47:0/2291561282 >> 172.21.15.47:0/2291561282 conn(0x7f720c0f3800 msgr2=0x7f720c0fb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:12.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.538+0000 7f72118cd700 1 -- 172.21.15.47:0/2291561282 shutdown_connections 2024-02-23T00:57:12.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.539+0000 7f72118cd700 1 -- 172.21.15.47:0/2291561282 wait complete. 2024-02-23T00:57:12.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:12.539+0000 7f72118cd700 1 librados: shutdown 2024-02-23T00:57:12.542 INFO:teuthology.orchestra.run.smithi047.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-02-23T00:57:12.791 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:57:12.791 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:57:12.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:11.154329+0000 osd.0 (osd.0) 330 : cluster [DBG] 18.4 scrub starts 2024-02-23T00:57:12.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:11.155399+0000 osd.0 (osd.0) 331 : cluster [DBG] 18.4 scrub ok 2024-02-23T00:57:12.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:11.426190+0000 mgr.y (mgr.14517) 722 : cluster [DBG] pgmap v1297: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 841 B/s rd, 0 op/s 2024-02-23T00:57:12.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:11.776667+0000 osd.6 (osd.6) 357 : cluster [DBG] 4.a scrub starts 2024-02-23T00:57:12.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:11.778642+0000 osd.6 (osd.6) 358 : cluster [DBG] 4.a scrub ok 2024-02-23T00:57:12.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:12.464946+0000 mon.a (mon.0) 3904 : audit [INF] from='client.? 172.21.15.47:0/2291561282' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-02-23T00:57:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:11.154329+0000 osd.0 (osd.0) 330 : cluster [DBG] 18.4 scrub starts 2024-02-23T00:57:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:11.155399+0000 osd.0 (osd.0) 331 : cluster [DBG] 18.4 scrub ok 2024-02-23T00:57:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:11.426190+0000 mgr.y (mgr.14517) 722 : cluster [DBG] pgmap v1297: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 841 B/s rd, 0 op/s 2024-02-23T00:57:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:11.776667+0000 osd.6 (osd.6) 357 : cluster [DBG] 4.a scrub starts 2024-02-23T00:57:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:11.778642+0000 osd.6 (osd.6) 358 : cluster [DBG] 4.a scrub ok 2024-02-23T00:57:12.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:12.464946+0000 mon.a (mon.0) 3904 : audit [INF] from='client.? 172.21.15.47:0/2291561282' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-02-23T00:57:12.817 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:57:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:57:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:57:12.817 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:11.154329+0000 osd.0 (osd.0) 330 : cluster [DBG] 18.4 scrub starts 2024-02-23T00:57:12.818 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:11.155399+0000 osd.0 (osd.0) 331 : cluster [DBG] 18.4 scrub ok 2024-02-23T00:57:12.818 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:11.426190+0000 mgr.y (mgr.14517) 722 : cluster [DBG] pgmap v1297: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 841 B/s rd, 0 op/s 2024-02-23T00:57:12.818 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:11.776667+0000 osd.6 (osd.6) 357 : cluster [DBG] 4.a scrub starts 2024-02-23T00:57:12.818 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:11.778642+0000 osd.6 (osd.6) 358 : cluster [DBG] 4.a scrub ok 2024-02-23T00:57:12.818 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:12.464946+0000 mon.a (mon.0) 3904 : audit [INF] from='client.? 172.21.15.47:0/2291561282' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-02-23T00:57:13.141 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:57:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:57:13.254 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight 4 1 2024-02-23T00:57:13.458 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.5 dump_historic_ops 2024-02-23T00:57:13.535 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:57:13.537 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:12.114307+0000 osd.0 (osd.0) 332 : cluster [DBG] 4.3 deep-scrub starts 2024-02-23T00:57:13.537 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:12.117216+0000 osd.0 (osd.0) 333 : cluster [DBG] 4.3 deep-scrub ok 2024-02-23T00:57:13.538 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:12.194970+0000 osd.7 (osd.7) 407 : cluster [DBG] 18.2 scrub starts 2024-02-23T00:57:13.538 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:12.197977+0000 osd.7 (osd.7) 408 : cluster [DBG] 18.2 scrub ok 2024-02-23T00:57:13.538 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:12.392578+0000 osd.4 (osd.4) 381 : cluster [DBG] 5.18 scrub starts 2024-02-23T00:57:13.538 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:12.393373+0000 osd.4 (osd.4) 382 : cluster [DBG] 5.18 scrub ok 2024-02-23T00:57:13.538 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:12.527840+0000 mon.a (mon.0) 3905 : audit [INF] from='client.? 172.21.15.47:0/2291561282' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-02-23T00:57:13.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:12.114307+0000 osd.0 (osd.0) 332 : cluster [DBG] 4.3 deep-scrub starts 2024-02-23T00:57:13.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:12.117216+0000 osd.0 (osd.0) 333 : cluster [DBG] 4.3 deep-scrub ok 2024-02-23T00:57:13.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:12.194970+0000 osd.7 (osd.7) 407 : cluster [DBG] 18.2 scrub starts 2024-02-23T00:57:13.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:12.197977+0000 osd.7 (osd.7) 408 : cluster [DBG] 18.2 scrub ok 2024-02-23T00:57:13.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:12.392578+0000 osd.4 (osd.4) 381 : cluster [DBG] 5.18 scrub starts 2024-02-23T00:57:13.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:12.393373+0000 osd.4 (osd.4) 382 : cluster [DBG] 5.18 scrub ok 2024-02-23T00:57:13.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:12.527840+0000 mon.a (mon.0) 3905 : audit [INF] from='client.? 172.21.15.47:0/2291561282' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-02-23T00:57:13.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:12.527897+0000 mon.a (mon.0) 3906 : cluster [DBG] osdmap e843: 8 total, 8 up, 8 in 2024-02-23T00:57:13.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:12.769428+0000 osd.6 (osd.6) 359 : cluster [DBG] 2.1a scrub starts 2024-02-23T00:57:13.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:12.770292+0000 osd.6 (osd.6) 360 : cluster [DBG] 2.1a scrub ok 2024-02-23T00:57:13.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:12.852297+0000 osd.1 (osd.1) 322 : cluster [DBG] 4.f scrub starts 2024-02-23T00:57:13.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:12.853698+0000 osd.1 (osd.1) 323 : cluster [DBG] 4.f scrub ok 2024-02-23T00:57:13.735 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:57:13.803 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:12.527897+0000 mon.a (mon.0) 3906 : cluster [DBG] osdmap e843: 8 total, 8 up, 8 in 2024-02-23T00:57:13.803 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:12.769428+0000 osd.6 (osd.6) 359 : cluster [DBG] 2.1a scrub starts 2024-02-23T00:57:13.803 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:12.770292+0000 osd.6 (osd.6) 360 : cluster [DBG] 2.1a scrub ok 2024-02-23T00:57:13.803 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:12.852297+0000 osd.1 (osd.1) 322 : cluster [DBG] 4.f scrub starts 2024-02-23T00:57:13.803 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:12.853698+0000 osd.1 (osd.1) 323 : cluster [DBG] 4.f scrub ok 2024-02-23T00:57:13.803 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:12.114307+0000 osd.0 (osd.0) 332 : cluster [DBG] 4.3 deep-scrub starts 2024-02-23T00:57:13.804 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:12.117216+0000 osd.0 (osd.0) 333 : cluster [DBG] 4.3 deep-scrub ok 2024-02-23T00:57:13.804 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:12.194970+0000 osd.7 (osd.7) 407 : cluster [DBG] 18.2 scrub starts 2024-02-23T00:57:13.804 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:12.197977+0000 osd.7 (osd.7) 408 : cluster [DBG] 18.2 scrub ok 2024-02-23T00:57:13.804 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:12.392578+0000 osd.4 (osd.4) 381 : cluster [DBG] 5.18 scrub starts 2024-02-23T00:57:13.804 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:12.393373+0000 osd.4 (osd.4) 382 : cluster [DBG] 5.18 scrub ok 2024-02-23T00:57:13.804 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:12.527840+0000 mon.a (mon.0) 3905 : audit [INF] from='client.? 172.21.15.47:0/2291561282' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-02-23T00:57:13.804 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:12.527897+0000 mon.a (mon.0) 3906 : cluster [DBG] osdmap e843: 8 total, 8 up, 8 in 2024-02-23T00:57:13.804 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:12.769428+0000 osd.6 (osd.6) 359 : cluster [DBG] 2.1a scrub starts 2024-02-23T00:57:13.804 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:12.770292+0000 osd.6 (osd.6) 360 : cluster [DBG] 2.1a scrub ok 2024-02-23T00:57:13.804 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:12.852297+0000 osd.1 (osd.1) 322 : cluster [DBG] 4.f scrub starts 2024-02-23T00:57:13.805 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:12.853698+0000 osd.1 (osd.1) 323 : cluster [DBG] 4.f scrub ok 2024-02-23T00:57:14.673 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:13.060928+0000 mgr.y (mgr.14517) 723 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:57:14.673 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:13.426539+0000 mgr.y (mgr.14517) 724 : cluster [DBG] pgmap v1299: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-23T00:57:14.673 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:13.538577+0000 mon.a (mon.0) 3907 : cluster [DBG] osdmap e844: 8 total, 8 up, 8 in 2024-02-23T00:57:14.673 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:13.947578+0000 osd.2 (osd.2) 203 : cluster [DBG] 2.10 scrub starts 2024-02-23T00:57:14.674 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:13.948375+0000 osd.2 (osd.2) 204 : cluster [DBG] 2.10 scrub ok 2024-02-23T00:57:14.674 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:14.206947+0000 osd.5 (osd.5) 392 : cluster [DBG] 4.8 scrub starts 2024-02-23T00:57:14.674 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:14.208969+0000 osd.5 (osd.5) 393 : cluster [DBG] 4.8 scrub ok 2024-02-23T00:57:14.674 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:13.060928+0000 mgr.y (mgr.14517) 723 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:57:14.674 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:13.426539+0000 mgr.y (mgr.14517) 724 : cluster [DBG] pgmap v1299: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-23T00:57:14.674 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:13.538577+0000 mon.a (mon.0) 3907 : cluster [DBG] osdmap e844: 8 total, 8 up, 8 in 2024-02-23T00:57:14.674 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:13.947578+0000 osd.2 (osd.2) 203 : cluster [DBG] 2.10 scrub starts 2024-02-23T00:57:14.674 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:13.948375+0000 osd.2 (osd.2) 204 : cluster [DBG] 2.10 scrub ok 2024-02-23T00:57:14.674 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:14.206947+0000 osd.5 (osd.5) 392 : cluster [DBG] 4.8 scrub starts 2024-02-23T00:57:14.675 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:14.208969+0000 osd.5 (osd.5) 393 : cluster [DBG] 4.8 scrub ok 2024-02-23T00:57:14.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:13.060928+0000 mgr.y (mgr.14517) 723 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:57:14.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:13.426539+0000 mgr.y (mgr.14517) 724 : cluster [DBG] pgmap v1299: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-23T00:57:14.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:13.538577+0000 mon.a (mon.0) 3907 : cluster [DBG] osdmap e844: 8 total, 8 up, 8 in 2024-02-23T00:57:14.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:13.947578+0000 osd.2 (osd.2) 203 : cluster [DBG] 2.10 scrub starts 2024-02-23T00:57:14.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:13.948375+0000 osd.2 (osd.2) 204 : cluster [DBG] 2.10 scrub ok 2024-02-23T00:57:14.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:14.206947+0000 osd.5 (osd.5) 392 : cluster [DBG] 4.8 scrub starts 2024-02-23T00:57:14.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:14.208969+0000 osd.5 (osd.5) 393 : cluster [DBG] 4.8 scrub ok 2024-02-23T00:57:15.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.198+0000 7f00fc21f700 1 -- 172.21.15.47:0/4003621413 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f00f40fdbb0 msgr2=0x7f00f40fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:15.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.198+0000 7f00fc21f700 1 --2- 172.21.15.47:0/4003621413 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f00f40fdbb0 0x7f00f40fdf90 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 rx=0x7f00f0004870 tx=0x7f00f0009cb0).stop 2024-02-23T00:57:15.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.199+0000 7f00fc21f700 1 -- 172.21.15.47:0/4003621413 shutdown_connections 2024-02-23T00:57:15.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.199+0000 7f00fc21f700 1 --2- 172.21.15.47:0/4003621413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00f40f8420 0x7f00f4104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:15.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.199+0000 7f00fc21f700 1 --2- 172.21.15.47:0/4003621413 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f00f40fdbb0 0x7f00f40fdf90 unknown :-1 s=CLOSED pgs=2847 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:15.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.199+0000 7f00fc21f700 1 --2- 172.21.15.47:0/4003621413 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00f40fe510 0x7f00f40f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:15.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.199+0000 7f00fc21f700 1 -- 172.21.15.47:0/4003621413 >> 172.21.15.47:0/4003621413 conn(0x7f00f40f3800 msgr2=0x7f00f40f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:15.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.199+0000 7f00fc21f700 1 -- 172.21.15.47:0/4003621413 shutdown_connections 2024-02-23T00:57:15.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.199+0000 7f00fc21f700 1 -- 172.21.15.47:0/4003621413 wait complete. 2024-02-23T00:57:15.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.199+0000 7f00fc21f700 1 librados: starting msgr at 2024-02-23T00:57:15.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.199+0000 7f00fc21f700 1 librados: starting objecter 2024-02-23T00:57:15.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.199+0000 7f00fc21f700 1 Processor -- start 2024-02-23T00:57:15.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.199+0000 7f00fc21f700 1 -- start start 2024-02-23T00:57:15.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.199+0000 7f00fc21f700 1 librados: setting wanted keys 2024-02-23T00:57:15.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.199+0000 7f00fc21f700 1 librados: calling monclient init 2024-02-23T00:57:15.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.200+0000 7f00fc21f700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f00f40f8420 0x7f00f406b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:15.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.200+0000 7f00fc21f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00f40fdbb0 0x7f00f406b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:15.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.200+0000 7f00fc21f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00f40fe510 0x7f00f406bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:15.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.200+0000 7f00fc21f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f4116360 con 0x7f00f40fe510 2024-02-23T00:57:15.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.200+0000 7f00fc21f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f41164d0 con 0x7f00f40fdbb0 2024-02-23T00:57:15.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.200+0000 7f00fc21f700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00f4116690 con 0x7f00f40f8420 2024-02-23T00:57:15.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.200+0000 7f00f9fbb700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f00f40f8420 0x7f00f406b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:15.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.200+0000 7f00f9fbb700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f00f40f8420 0x7f00f406b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:51032/0 (socket says 172.21.15.47:51032) 2024-02-23T00:57:15.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.200+0000 7f00f9fbb700 1 -- 172.21.15.47:0/977123425 learned_addr learned my addr 172.21.15.47:0/977123425 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:57:15.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.200+0000 7f00f97ba700 1 --2- 172.21.15.47:0/977123425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00f40fdbb0 0x7f00f406b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:15.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.200+0000 7f00fa7bc700 1 --2- 172.21.15.47:0/977123425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00f40fe510 0x7f00f406bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:15.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.200+0000 7f00f9fbb700 1 -- 172.21.15.47:0/977123425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00f40fdbb0 msgr2=0x7f00f406b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:15.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.200+0000 7f00f9fbb700 1 --2- 172.21.15.47:0/977123425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00f40fdbb0 0x7f00f406b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:15.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.200+0000 7f00f9fbb700 1 -- 172.21.15.47:0/977123425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00f40fe510 msgr2=0x7f00f406bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:15.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.200+0000 7f00f9fbb700 1 --2- 172.21.15.47:0/977123425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00f40fe510 0x7f00f406bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:15.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.200+0000 7f00f9fbb700 1 -- 172.21.15.47:0/977123425 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e4005040 con 0x7f00f40f8420 2024-02-23T00:57:15.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.200+0000 7f00f9fbb700 1 --2- 172.21.15.47:0/977123425 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f00f40f8420 0x7f00f406b0d0 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 rx=0x7f00f0009da0 tx=0x7f00f0009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:15.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.200+0000 7f00eaffd700 10 client.?.objecter ms_handle_connect 0x7f00f40f8420 2024-02-23T00:57:15.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.200+0000 7f00eaffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:57:15.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.200+0000 7f00eaffd700 1 -- 172.21.15.47:0/977123425 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f000e030 con 0x7f00f40f8420 2024-02-23T00:57:15.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.201+0000 7f00eaffd700 1 -- 172.21.15.47:0/977123425 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f00f0013d70 con 0x7f00f40f8420 2024-02-23T00:57:15.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.201+0000 7f00fc21f700 1 -- 172.21.15.47:0/977123425 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00f0005040 con 0x7f00f40f8420 2024-02-23T00:57:15.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.201+0000 7f00eaffd700 1 -- 172.21.15.47:0/977123425 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f0018070 con 0x7f00f40f8420 2024-02-23T00:57:15.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.201+0000 7f00fc21f700 10 client.42305.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:57:15.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.201+0000 7f00fc21f700 1 -- 172.21.15.47:0/977123425 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00f4116c60 con 0x7f00f40f8420 2024-02-23T00:57:15.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.201+0000 7f00fc21f700 1 librados: init done 2024-02-23T00:57:15.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.202+0000 7f00fc21f700 1 -- 172.21.15.47:0/977123425 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00f4106e20 con 0x7f00f40f8420 2024-02-23T00:57:15.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.202+0000 7f00eaffd700 1 -- 172.21.15.47:0/977123425 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f00f0009240 con 0x7f00f40f8420 2024-02-23T00:57:15.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.203+0000 7f00eaffd700 1 --2- 172.21.15.47:0/977123425 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f00e0062cc0 0x7f00e0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:15.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.203+0000 7f00eaffd700 1 -- 172.21.15.47:0/977123425 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(844..844 src has 670..844) v4 ==== 8663+0+0 (secure 0 0 0) 0x7f00f007ec30 con 0x7f00f40f8420 2024-02-23T00:57:15.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.203+0000 7f00eaffd700 10 client.42305.objecter ms_dispatch 0x7f00f4001550 osd_map(844..844 src has 670..844) v4 2024-02-23T00:57:15.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.203+0000 7f00eaffd700 3 client.42305.objecter handle_osd_map got epochs [844,844] > 0 2024-02-23T00:57:15.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.203+0000 7f00eaffd700 3 client.42305.objecter handle_osd_map decoding full epoch 844 2024-02-23T00:57:15.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.203+0000 7f00f97ba700 1 --2- 172.21.15.47:0/977123425 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f00e0062cc0 0x7f00e0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:15.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.203+0000 7f00eaffd700 20 client.42305.objecter dump_active .. 0 homeless 2024-02-23T00:57:15.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.203+0000 7f00eaffd700 10 client.42305.objecter ms_handle_connect 0x7f00e0062cc0 2024-02-23T00:57:15.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.203+0000 7f00f97ba700 1 --2- 172.21.15.47:0/977123425 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f00e0062cc0 0x7f00e0065180 secure :-1 s=READY pgs=3234 cs=0 l=1 rev1=1 rx=0x7f00f406c870 tx=0x7f00e400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:15.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.207+0000 7f00eaffd700 1 -- 172.21.15.47:0/977123425 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f00f004e470 con 0x7f00f40f8420 2024-02-23T00:57:15.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:15.396+0000 7f00fc21f700 1 -- 172.21.15.47:0/977123425 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd reweight", "id": 4, "weight": 1.0} v 0) v1 -- 0x7f00f406c5b0 con 0x7f00f40f8420 2024-02-23T00:57:15.596 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:14.353028+0000 osd.4 (osd.4) 383 : cluster [DBG] 18.d scrub starts 2024-02-23T00:57:15.596 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:14.354697+0000 osd.4 (osd.4) 384 : cluster [DBG] 18.d scrub ok 2024-02-23T00:57:15.596 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:15.206579+0000 osd.5 (osd.5) 394 : cluster [DBG] 2.2 scrub starts 2024-02-23T00:57:15.596 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:15.207442+0000 osd.5 (osd.5) 395 : cluster [DBG] 2.2 scrub ok 2024-02-23T00:57:15.596 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:15.397393+0000 mon.c (mon.2) 528 : audit [INF] from='client.? 172.21.15.47:0/977123425' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-23T00:57:15.596 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:15.397747+0000 mon.a (mon.0) 3908 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-23T00:57:15.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:14.353028+0000 osd.4 (osd.4) 383 : cluster [DBG] 18.d scrub starts 2024-02-23T00:57:15.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:14.354697+0000 osd.4 (osd.4) 384 : cluster [DBG] 18.d scrub ok 2024-02-23T00:57:15.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:15.206579+0000 osd.5 (osd.5) 394 : cluster [DBG] 2.2 scrub starts 2024-02-23T00:57:15.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:15.207442+0000 osd.5 (osd.5) 395 : cluster [DBG] 2.2 scrub ok 2024-02-23T00:57:15.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:15.397393+0000 mon.c (mon.2) 528 : audit [INF] from='client.? 172.21.15.47:0/977123425' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-23T00:57:15.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:15.397747+0000 mon.a (mon.0) 3908 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-23T00:57:15.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:14.353028+0000 osd.4 (osd.4) 383 : cluster [DBG] 18.d scrub starts 2024-02-23T00:57:15.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:14.354697+0000 osd.4 (osd.4) 384 : cluster [DBG] 18.d scrub ok 2024-02-23T00:57:15.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:15.206579+0000 osd.5 (osd.5) 394 : cluster [DBG] 2.2 scrub starts 2024-02-23T00:57:15.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:15.207442+0000 osd.5 (osd.5) 395 : cluster [DBG] 2.2 scrub ok 2024-02-23T00:57:15.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:15.397393+0000 mon.c (mon.2) 528 : audit [INF] from='client.? 172.21.15.47:0/977123425' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-23T00:57:15.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:15.397747+0000 mon.a (mon.0) 3908 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-23T00:57:15.844 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:57:15.844 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:57:16.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:16.352+0000 7f00eaffd700 1 -- 172.21.15.47:0/977123425 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 4, "weight": 1.0}]=0 reweighted osd.4 to 1 (10000) v845) v1 ==== 113+0+0 (secure 0 0 0) 0x7f00f0051a90 con 0x7f00f40f8420 2024-02-23T00:57:16.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:16.355+0000 7f00fc21f700 10 librados: watch_flush enter 2024-02-23T00:57:16.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:16.355+0000 7f00fc21f700 10 librados: watch_flush exit 2024-02-23T00:57:16.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:16.355+0000 7f00fc21f700 20 client.42305.objecter shutdown clearing up homeless session... 2024-02-23T00:57:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:16.355+0000 7f00fc21f700 10 client.42305.objecter successfully canceled tick 2024-02-23T00:57:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:16.355+0000 7f00fc21f700 1 -- 172.21.15.47:0/977123425 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f00e0062cc0 msgr2=0x7f00e0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:16.355+0000 7f00fc21f700 1 --2- 172.21.15.47:0/977123425 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f00e0062cc0 0x7f00e0065180 secure :-1 s=READY pgs=3234 cs=0 l=1 rev1=1 rx=0x7f00f406c870 tx=0x7f00e400b040).stop 2024-02-23T00:57:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:16.355+0000 7f00fc21f700 1 -- 172.21.15.47:0/977123425 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f00f40f8420 msgr2=0x7f00f406b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:16.355+0000 7f00fc21f700 1 --2- 172.21.15.47:0/977123425 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f00f40f8420 0x7f00f406b0d0 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 rx=0x7f00f0009da0 tx=0x7f00f0009cb0).stop 2024-02-23T00:57:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:16.355+0000 7f00fc21f700 1 -- 172.21.15.47:0/977123425 shutdown_connections 2024-02-23T00:57:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:16.355+0000 7f00fc21f700 1 --2- 172.21.15.47:0/977123425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00f40fe510 0x7f00f406bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:16.355+0000 7f00fc21f700 1 --2- 172.21.15.47:0/977123425 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f00e0062cc0 0x7f00e0065180 unknown :-1 s=CLOSED pgs=3234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:16.355+0000 7f00fc21f700 1 --2- 172.21.15.47:0/977123425 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f00f40f8420 0x7f00f406b0d0 unknown :-1 s=CLOSED pgs=2848 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:16.355+0000 7f00fc21f700 1 --2- 172.21.15.47:0/977123425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00f40fdbb0 0x7f00f406b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:16.355+0000 7f00fc21f700 1 -- 172.21.15.47:0/977123425 >> 172.21.15.47:0/977123425 conn(0x7f00f40f3800 msgr2=0x7f00f40fb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:16.355+0000 7f00fc21f700 1 -- 172.21.15.47:0/977123425 shutdown_connections 2024-02-23T00:57:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:16.356+0000 7f00fc21f700 1 -- 172.21.15.47:0/977123425 wait complete. 2024-02-23T00:57:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:16.356+0000 7f00fc21f700 1 librados: shutdown 2024-02-23T00:57:16.360 INFO:teuthology.orchestra.run.smithi047.stderr:reweighted osd.4 to 1 (10000) 2024-02-23T00:57:16.522 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.6 dump_ops_in_flight 2024-02-23T00:57:16.742 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:15.089740+0000 osd.0 (osd.0) 334 : cluster [DBG] 18.0 deep-scrub starts 2024-02-23T00:57:16.742 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:15.090850+0000 osd.0 (osd.0) 335 : cluster [DBG] 18.0 deep-scrub ok 2024-02-23T00:57:16.742 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:15.426897+0000 mgr.y (mgr.14517) 725 : cluster [DBG] pgmap v1301: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:57:16.742 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:15.838291+0000 osd.1 (osd.1) 324 : cluster [DBG] 5.27 scrub starts 2024-02-23T00:57:16.742 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:15.839147+0000 osd.1 (osd.1) 325 : cluster [DBG] 5.27 scrub ok 2024-02-23T00:57:16.742 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:16.352760+0000 mon.a (mon.0) 3909 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-02-23T00:57:16.742 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:16.352861+0000 mon.a (mon.0) 3910 : cluster [DBG] osdmap e845: 8 total, 8 up, 8 in 2024-02-23T00:57:16.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:15.089740+0000 osd.0 (osd.0) 334 : cluster [DBG] 18.0 deep-scrub starts 2024-02-23T00:57:16.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:15.090850+0000 osd.0 (osd.0) 335 : cluster [DBG] 18.0 deep-scrub ok 2024-02-23T00:57:16.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:15.426897+0000 mgr.y (mgr.14517) 725 : cluster [DBG] pgmap v1301: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:57:16.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:15.838291+0000 osd.1 (osd.1) 324 : cluster [DBG] 5.27 scrub starts 2024-02-23T00:57:16.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:15.839147+0000 osd.1 (osd.1) 325 : cluster [DBG] 5.27 scrub ok 2024-02-23T00:57:16.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:16.352760+0000 mon.a (mon.0) 3909 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-02-23T00:57:16.743 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:16.352861+0000 mon.a (mon.0) 3910 : cluster [DBG] osdmap e845: 8 total, 8 up, 8 in 2024-02-23T00:57:16.795 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:57:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:15.089740+0000 osd.0 (osd.0) 334 : cluster [DBG] 18.0 deep-scrub starts 2024-02-23T00:57:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:15.090850+0000 osd.0 (osd.0) 335 : cluster [DBG] 18.0 deep-scrub ok 2024-02-23T00:57:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:15.426897+0000 mgr.y (mgr.14517) 725 : cluster [DBG] pgmap v1301: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:57:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:15.838291+0000 osd.1 (osd.1) 324 : cluster [DBG] 5.27 scrub starts 2024-02-23T00:57:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:15.839147+0000 osd.1 (osd.1) 325 : cluster [DBG] 5.27 scrub ok 2024-02-23T00:57:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:16.352760+0000 mon.a (mon.0) 3909 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-02-23T00:57:16.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:16.352861+0000 mon.a (mon.0) 3910 : cluster [DBG] osdmap e845: 8 total, 8 up, 8 in 2024-02-23T00:57:17.139 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd primary-affinity 4 1 2024-02-23T00:57:17.422 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:57:18.451 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:57:18.451 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:57:18.803 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:17.427590+0000 mgr.y (mgr.14517) 726 : cluster [DBG] pgmap v1303: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-23T00:57:18.803 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:17.819262+0000 osd.1 (osd.1) 326 : cluster [DBG] 5.25 scrub starts 2024-02-23T00:57:18.803 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:17.820124+0000 osd.1 (osd.1) 327 : cluster [DBG] 5.25 scrub ok 2024-02-23T00:57:18.803 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:18.152238+0000 osd.3 (osd.3) 327 : cluster [DBG] 18.1 scrub starts 2024-02-23T00:57:18.803 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:18.153775+0000 osd.3 (osd.3) 328 : cluster [DBG] 18.1 scrub ok 2024-02-23T00:57:18.803 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:18.243759+0000 osd.5 (osd.5) 396 : cluster [DBG] 4.15 scrub starts 2024-02-23T00:57:18.803 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:18.247232+0000 osd.5 (osd.5) 397 : cluster [DBG] 4.15 scrub ok 2024-02-23T00:57:18.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:17.427590+0000 mgr.y (mgr.14517) 726 : cluster [DBG] pgmap v1303: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-23T00:57:18.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:17.819262+0000 osd.1 (osd.1) 326 : cluster [DBG] 5.25 scrub starts 2024-02-23T00:57:18.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:17.820124+0000 osd.1 (osd.1) 327 : cluster [DBG] 5.25 scrub ok 2024-02-23T00:57:18.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:18.152238+0000 osd.3 (osd.3) 327 : cluster [DBG] 18.1 scrub starts 2024-02-23T00:57:18.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:18.153775+0000 osd.3 (osd.3) 328 : cluster [DBG] 18.1 scrub ok 2024-02-23T00:57:18.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:18.243759+0000 osd.5 (osd.5) 396 : cluster [DBG] 4.15 scrub starts 2024-02-23T00:57:18.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:18.247232+0000 osd.5 (osd.5) 397 : cluster [DBG] 4.15 scrub ok 2024-02-23T00:57:18.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:17.427590+0000 mgr.y (mgr.14517) 726 : cluster [DBG] pgmap v1303: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-23T00:57:18.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:17.819262+0000 osd.1 (osd.1) 326 : cluster [DBG] 5.25 scrub starts 2024-02-23T00:57:18.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:17.820124+0000 osd.1 (osd.1) 327 : cluster [DBG] 5.25 scrub ok 2024-02-23T00:57:18.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:18.152238+0000 osd.3 (osd.3) 327 : cluster [DBG] 18.1 scrub starts 2024-02-23T00:57:18.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:18.153775+0000 osd.3 (osd.3) 328 : cluster [DBG] 18.1 scrub ok 2024-02-23T00:57:18.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:18.243759+0000 osd.5 (osd.5) 396 : cluster [DBG] 4.15 scrub starts 2024-02-23T00:57:18.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:18.247232+0000 osd.5 (osd.5) 397 : cluster [DBG] 4.15 scrub ok 2024-02-23T00:57:19.145 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.6 dump_blocked_ops 2024-02-23T00:57:19.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.317+0000 7f67ee8fd700 1 -- 172.21.15.47:0/2098547447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67e80fe510 msgr2=0x7f67e80f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:19.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.317+0000 7f67ee8fd700 1 --2- 172.21.15.47:0/2098547447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67e80fe510 0x7f67e80f7d40 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 rx=0x7f67dc004870 tx=0x7f67dc009cb0).stop 2024-02-23T00:57:19.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.317+0000 7f67ee8fd700 1 -- 172.21.15.47:0/2098547447 shutdown_connections 2024-02-23T00:57:19.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.317+0000 7f67ee8fd700 1 --2- 172.21.15.47:0/2098547447 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f67e80f8420 0x7f67e8104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:19.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.317+0000 7f67ee8fd700 1 --2- 172.21.15.47:0/2098547447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67e80fdbb0 0x7f67e80fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:19.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.317+0000 7f67ee8fd700 1 --2- 172.21.15.47:0/2098547447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67e80fe510 0x7f67e80f7d40 unknown :-1 s=CLOSED pgs=2671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:19.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.317+0000 7f67ee8fd700 1 -- 172.21.15.47:0/2098547447 >> 172.21.15.47:0/2098547447 conn(0x7f67e80f3800 msgr2=0x7f67e80f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:19.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.317+0000 7f67ee8fd700 1 -- 172.21.15.47:0/2098547447 shutdown_connections 2024-02-23T00:57:19.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.317+0000 7f67ee8fd700 1 -- 172.21.15.47:0/2098547447 wait complete. 2024-02-23T00:57:19.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.318+0000 7f67ee8fd700 1 librados: starting msgr at 2024-02-23T00:57:19.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.318+0000 7f67ee8fd700 1 librados: starting objecter 2024-02-23T00:57:19.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.318+0000 7f67ee8fd700 1 Processor -- start 2024-02-23T00:57:19.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.318+0000 7f67ee8fd700 1 -- start start 2024-02-23T00:57:19.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.318+0000 7f67ee8fd700 1 librados: setting wanted keys 2024-02-23T00:57:19.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.318+0000 7f67ee8fd700 1 librados: calling monclient init 2024-02-23T00:57:19.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.318+0000 7f67ee8fd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f67e80f8420 0x7f67e810ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:19.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.318+0000 7f67ee8fd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67e80fdbb0 0x7f67e810e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:19.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.318+0000 7f67ee8fd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67e80fe510 0x7f67e8108070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:19.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.318+0000 7f67ee8fd700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e8108560 con 0x7f67e80fdbb0 2024-02-23T00:57:19.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.318+0000 7f67ee8fd700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e81086d0 con 0x7f67e80fe510 2024-02-23T00:57:19.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.318+0000 7f67ee8fd700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e8108890 con 0x7f67e80f8420 2024-02-23T00:57:19.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.318+0000 7f67e77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67e80fdbb0 0x7f67e810e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:19.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.318+0000 7f67e7fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f67e80f8420 0x7f67e810ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:19.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.318+0000 7f67e77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67e80fdbb0 0x7f67e810e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53092/0 (socket says 172.21.15.47:53092) 2024-02-23T00:57:19.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.318+0000 7f67e77fe700 1 -- 172.21.15.47:0/2971652166 learned_addr learned my addr 172.21.15.47:0/2971652166 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:57:19.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.319+0000 7f67ece9a700 1 --2- 172.21.15.47:0/2971652166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67e80fe510 0x7f67e8108070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:19.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.319+0000 7f67e7fff700 1 -- 172.21.15.47:0/2971652166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67e80fe510 msgr2=0x7f67e8108070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:19.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.319+0000 7f67e7fff700 1 --2- 172.21.15.47:0/2971652166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67e80fe510 0x7f67e8108070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:19.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.319+0000 7f67e7fff700 1 -- 172.21.15.47:0/2971652166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67e80fdbb0 msgr2=0x7f67e810e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:19.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.319+0000 7f67e7fff700 1 --2- 172.21.15.47:0/2971652166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67e80fdbb0 0x7f67e810e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:19.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.319+0000 7f67e7fff700 1 -- 172.21.15.47:0/2971652166 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e00097c0 con 0x7f67e80f8420 2024-02-23T00:57:19.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.319+0000 7f67e77fe700 1 --2- 172.21.15.47:0/2971652166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67e80fdbb0 0x7f67e810e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:57:19.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.319+0000 7f67e7fff700 1 --2- 172.21.15.47:0/2971652166 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f67e80f8420 0x7f67e810ded0 secure :-1 s=READY pgs=2849 cs=0 l=1 rev1=1 rx=0x7f67d800e8f0 tx=0x7f67d800b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:19.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.319+0000 7f67e57fa700 10 client.?.objecter ms_handle_connect 0x7f67e80f8420 2024-02-23T00:57:19.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.319+0000 7f67e57fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:57:19.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.319+0000 7f67e57fa700 1 -- 172.21.15.47:0/2971652166 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d8004620 con 0x7f67e80f8420 2024-02-23T00:57:19.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.319+0000 7f67ee8fd700 1 -- 172.21.15.47:0/2971652166 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67dc005040 con 0x7f67e80f8420 2024-02-23T00:57:19.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.319+0000 7f67ee8fd700 10 client.42311.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:57:19.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.319+0000 7f67ee8fd700 1 -- 172.21.15.47:0/2971652166 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e8114d00 con 0x7f67e80f8420 2024-02-23T00:57:19.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.319+0000 7f67ee8fd700 1 librados: init done 2024-02-23T00:57:19.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.319+0000 7f67e57fa700 1 -- 172.21.15.47:0/2971652166 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f67d800f040 con 0x7f67e80f8420 2024-02-23T00:57:19.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.319+0000 7f67e57fa700 1 -- 172.21.15.47:0/2971652166 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d80146f0 con 0x7f67e80f8420 2024-02-23T00:57:19.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.320+0000 7f67ee8fd700 1 -- 172.21.15.47:0/2971652166 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d4004f50 con 0x7f67e80f8420 2024-02-23T00:57:19.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.321+0000 7f67e57fa700 1 -- 172.21.15.47:0/2971652166 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f67d80093b0 con 0x7f67e80f8420 2024-02-23T00:57:19.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.321+0000 7f67e57fa700 1 --2- 172.21.15.47:0/2971652166 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f67d0062cc0 0x7f67d0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:19.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.321+0000 7f67e57fa700 1 -- 172.21.15.47:0/2971652166 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(845..845 src has 670..845) v4 ==== 8663+0+0 (secure 0 0 0) 0x7f67d807ccb0 con 0x7f67e80f8420 2024-02-23T00:57:19.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.321+0000 7f67e57fa700 10 client.42311.objecter ms_dispatch 0x7f67e8001550 osd_map(845..845 src has 670..845) v4 2024-02-23T00:57:19.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.321+0000 7f67e57fa700 3 client.42311.objecter handle_osd_map got epochs [845,845] > 0 2024-02-23T00:57:19.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.321+0000 7f67e57fa700 3 client.42311.objecter handle_osd_map decoding full epoch 845 2024-02-23T00:57:19.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.321+0000 7f67e77fe700 1 --2- 172.21.15.47:0/2971652166 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f67d0062cc0 0x7f67d0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:19.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.321+0000 7f67e57fa700 20 client.42311.objecter dump_active .. 0 homeless 2024-02-23T00:57:19.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.322+0000 7f67e77fe700 1 --2- 172.21.15.47:0/2971652166 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f67d0062cc0 0x7f67d0065180 secure :-1 s=READY pgs=3235 cs=0 l=1 rev1=1 rx=0x7f67dc007fd0 tx=0x7f67dc00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:19.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.322+0000 7f67e57fa700 10 client.42311.objecter ms_handle_connect 0x7f67d0062cc0 2024-02-23T00:57:19.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.326+0000 7f67e57fa700 1 -- 172.21.15.47:0/2971652166 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f67d804c620 con 0x7f67e80f8420 2024-02-23T00:57:19.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.510+0000 7f67ee8fd700 1 -- 172.21.15.47:0/2971652166 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd primary-affinity", "id": 4, "weight": 1.0} v 0) v1 -- 0x7f67d4005780 con 0x7f67e80f8420 2024-02-23T00:57:19.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.552+0000 7f67e57fa700 1 -- 172.21.15.47:0/2971652166 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]=0 set osd.4 primary-affinity to 1 (8655362) v846) v1 ==== 133+0+0 (secure 0 0 0) 0x7f67d8019070 con 0x7f67e80f8420 2024-02-23T00:57:19.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.554+0000 7f67ee8fd700 10 librados: watch_flush enter 2024-02-23T00:57:19.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.554+0000 7f67ee8fd700 10 librados: watch_flush exit 2024-02-23T00:57:19.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.554+0000 7f67ee8fd700 20 client.42311.objecter shutdown clearing up homeless session... 2024-02-23T00:57:19.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.554+0000 7f67ee8fd700 10 client.42311.objecter successfully canceled tick 2024-02-23T00:57:19.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.554+0000 7f67ee8fd700 1 -- 172.21.15.47:0/2971652166 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f67d0062cc0 msgr2=0x7f67d0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:19.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.554+0000 7f67ee8fd700 1 --2- 172.21.15.47:0/2971652166 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f67d0062cc0 0x7f67d0065180 secure :-1 s=READY pgs=3235 cs=0 l=1 rev1=1 rx=0x7f67dc007fd0 tx=0x7f67dc00b040).stop 2024-02-23T00:57:19.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.554+0000 7f67ee8fd700 1 -- 172.21.15.47:0/2971652166 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f67e80f8420 msgr2=0x7f67e810ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:19.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.554+0000 7f67ee8fd700 1 --2- 172.21.15.47:0/2971652166 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f67e80f8420 0x7f67e810ded0 secure :-1 s=READY pgs=2849 cs=0 l=1 rev1=1 rx=0x7f67d800e8f0 tx=0x7f67d800b040).stop 2024-02-23T00:57:19.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.554+0000 7f67ee8fd700 1 -- 172.21.15.47:0/2971652166 shutdown_connections 2024-02-23T00:57:19.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.554+0000 7f67ee8fd700 1 --2- 172.21.15.47:0/2971652166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67e80fe510 0x7f67e8108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:19.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.554+0000 7f67ee8fd700 1 --2- 172.21.15.47:0/2971652166 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f67d0062cc0 0x7f67d0065180 unknown :-1 s=CLOSED pgs=3235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:19.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.554+0000 7f67ee8fd700 1 --2- 172.21.15.47:0/2971652166 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f67e80f8420 0x7f67e810ded0 unknown :-1 s=CLOSED pgs=2849 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:19.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.554+0000 7f67ee8fd700 1 --2- 172.21.15.47:0/2971652166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67e80fdbb0 0x7f67e810e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:19.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.554+0000 7f67ee8fd700 1 -- 172.21.15.47:0/2971652166 >> 172.21.15.47:0/2971652166 conn(0x7f67e80f3800 msgr2=0x7f67e80fb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:19.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.554+0000 7f67ee8fd700 1 -- 172.21.15.47:0/2971652166 shutdown_connections 2024-02-23T00:57:19.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.554+0000 7f67ee8fd700 1 -- 172.21.15.47:0/2971652166 wait complete. 2024-02-23T00:57:19.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:19.554+0000 7f67ee8fd700 1 librados: shutdown 2024-02-23T00:57:19.557 INFO:teuthology.orchestra.run.smithi047.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-02-23T00:57:19.560 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:57:19.805 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:18.286899+0000 osd.4 (osd.4) 385 : cluster [DBG] 4.9 scrub starts 2024-02-23T00:57:19.805 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:18.288791+0000 osd.4 (osd.4) 386 : cluster [DBG] 4.9 scrub ok 2024-02-23T00:57:19.805 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:19.138685+0000 osd.3 (osd.3) 329 : cluster [DBG] 2.1e scrub starts 2024-02-23T00:57:19.805 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:19.139605+0000 osd.3 (osd.3) 330 : cluster [DBG] 2.1e scrub ok 2024-02-23T00:57:19.805 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:19.235335+0000 osd.5 (osd.5) 398 : cluster [DBG] 5.1b scrub starts 2024-02-23T00:57:19.806 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:19.236194+0000 osd.5 (osd.5) 399 : cluster [DBG] 5.1b scrub ok 2024-02-23T00:57:19.806 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:19.510852+0000 mon.c (mon.2) 529 : audit [INF] from='client.? 172.21.15.47:0/2971652166' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-02-23T00:57:19.806 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:19.511143+0000 mon.a (mon.0) 3911 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-02-23T00:57:19.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:18.286899+0000 osd.4 (osd.4) 385 : cluster [DBG] 4.9 scrub starts 2024-02-23T00:57:19.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:18.288791+0000 osd.4 (osd.4) 386 : cluster [DBG] 4.9 scrub ok 2024-02-23T00:57:19.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:19.138685+0000 osd.3 (osd.3) 329 : cluster [DBG] 2.1e scrub starts 2024-02-23T00:57:19.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:19.139605+0000 osd.3 (osd.3) 330 : cluster [DBG] 2.1e scrub ok 2024-02-23T00:57:19.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:19.235335+0000 osd.5 (osd.5) 398 : cluster [DBG] 5.1b scrub starts 2024-02-23T00:57:19.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:19.236194+0000 osd.5 (osd.5) 399 : cluster [DBG] 5.1b scrub ok 2024-02-23T00:57:19.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:19.510852+0000 mon.c (mon.2) 529 : audit [INF] from='client.? 172.21.15.47:0/2971652166' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-02-23T00:57:19.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:19.511143+0000 mon.a (mon.0) 3911 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-02-23T00:57:19.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:18.286899+0000 osd.4 (osd.4) 385 : cluster [DBG] 4.9 scrub starts 2024-02-23T00:57:19.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:18.288791+0000 osd.4 (osd.4) 386 : cluster [DBG] 4.9 scrub ok 2024-02-23T00:57:19.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:19.138685+0000 osd.3 (osd.3) 329 : cluster [DBG] 2.1e scrub starts 2024-02-23T00:57:19.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:19.139605+0000 osd.3 (osd.3) 330 : cluster [DBG] 2.1e scrub ok 2024-02-23T00:57:19.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:19.235335+0000 osd.5 (osd.5) 398 : cluster [DBG] 5.1b scrub starts 2024-02-23T00:57:19.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:19.236194+0000 osd.5 (osd.5) 399 : cluster [DBG] 5.1b scrub ok 2024-02-23T00:57:19.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:19.510852+0000 mon.c (mon.2) 529 : audit [INF] from='client.? 172.21.15.47:0/2971652166' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-02-23T00:57:19.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:19.511143+0000 mon.a (mon.0) 3911 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-02-23T00:57:20.292 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight 5 1 2024-02-23T00:57:20.551 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:57:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:57:20] "GET /metrics HTTP/1.1" 200 187132 "" "Prometheus/2.33.4" 2024-02-23T00:57:20.553 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:19.427978+0000 mgr.y (mgr.14517) 727 : cluster [DBG] pgmap v1304: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 741 B/s rd, 0 op/s 2024-02-23T00:57:20.553 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:19.551572+0000 mon.a (mon.0) 3912 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-02-23T00:57:20.553 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:19.551648+0000 mon.a (mon.0) 3913 : cluster [DBG] osdmap e846: 8 total, 8 up, 8 in 2024-02-23T00:57:20.553 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:19.657748+0000 osd.6 (osd.6) 361 : cluster [DBG] 2.17 scrub starts 2024-02-23T00:57:20.553 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:19.658528+0000 osd.6 (osd.6) 362 : cluster [DBG] 2.17 scrub ok 2024-02-23T00:57:20.553 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:20.042663+0000 mon.a (mon.0) 3914 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:57:20.553 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:20.414096+0000 mon.a (mon.0) 3915 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:57:20.553 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:20.414785+0000 mon.a (mon.0) 3916 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:57:20.553 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:20.419218+0000 mon.a (mon.0) 3917 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:57:20.576 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:57:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:19.427978+0000 mgr.y (mgr.14517) 727 : cluster [DBG] pgmap v1304: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 741 B/s rd, 0 op/s 2024-02-23T00:57:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:19.551572+0000 mon.a (mon.0) 3912 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-02-23T00:57:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:19.551648+0000 mon.a (mon.0) 3913 : cluster [DBG] osdmap e846: 8 total, 8 up, 8 in 2024-02-23T00:57:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:19.657748+0000 osd.6 (osd.6) 361 : cluster [DBG] 2.17 scrub starts 2024-02-23T00:57:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:19.658528+0000 osd.6 (osd.6) 362 : cluster [DBG] 2.17 scrub ok 2024-02-23T00:57:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:20.042663+0000 mon.a (mon.0) 3914 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:57:20.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:20.414096+0000 mon.a (mon.0) 3915 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:57:20.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:20.414785+0000 mon.a (mon.0) 3916 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:57:20.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:20.419218+0000 mon.a (mon.0) 3917 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:57:20.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:19.427978+0000 mgr.y (mgr.14517) 727 : cluster [DBG] pgmap v1304: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 741 B/s rd, 0 op/s 2024-02-23T00:57:20.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:19.551572+0000 mon.a (mon.0) 3912 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-02-23T00:57:20.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:19.551648+0000 mon.a (mon.0) 3913 : cluster [DBG] osdmap e846: 8 total, 8 up, 8 in 2024-02-23T00:57:20.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:19.657748+0000 osd.6 (osd.6) 361 : cluster [DBG] 2.17 scrub starts 2024-02-23T00:57:20.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:19.658528+0000 osd.6 (osd.6) 362 : cluster [DBG] 2.17 scrub ok 2024-02-23T00:57:20.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:20.042663+0000 mon.a (mon.0) 3914 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:57:20.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:20.414096+0000 mon.a (mon.0) 3915 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:57:20.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:20.414785+0000 mon.a (mon.0) 3916 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:57:20.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:20.419218+0000 mon.a (mon.0) 3917 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:57:21.238 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:57:21.238 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:57:21.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:20.704877+0000 osd.6 (osd.6) 363 : cluster [DBG] 2.14 deep-scrub starts 2024-02-23T00:57:21.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:20.705634+0000 osd.6 (osd.6) 364 : cluster [DBG] 2.14 deep-scrub ok 2024-02-23T00:57:21.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:20.871043+0000 osd.1 (osd.1) 328 : cluster [DBG] 18.1b scrub starts 2024-02-23T00:57:21.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:20.874933+0000 osd.1 (osd.1) 329 : cluster [DBG] 18.1b scrub ok 2024-02-23T00:57:21.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:21.255915+0000 osd.5 (osd.5) 400 : cluster [DBG] 6.a scrub starts 2024-02-23T00:57:21.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:21.257049+0000 osd.5 (osd.5) 401 : cluster [DBG] 6.a scrub ok 2024-02-23T00:57:21.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:20.704877+0000 osd.6 (osd.6) 363 : cluster [DBG] 2.14 deep-scrub starts 2024-02-23T00:57:21.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:20.705634+0000 osd.6 (osd.6) 364 : cluster [DBG] 2.14 deep-scrub ok 2024-02-23T00:57:21.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:20.871043+0000 osd.1 (osd.1) 328 : cluster [DBG] 18.1b scrub starts 2024-02-23T00:57:21.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:20.874933+0000 osd.1 (osd.1) 329 : cluster [DBG] 18.1b scrub ok 2024-02-23T00:57:21.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:21.255915+0000 osd.5 (osd.5) 400 : cluster [DBG] 6.a scrub starts 2024-02-23T00:57:21.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:21.257049+0000 osd.5 (osd.5) 401 : cluster [DBG] 6.a scrub ok 2024-02-23T00:57:21.905 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:20.704877+0000 osd.6 (osd.6) 363 : cluster [DBG] 2.14 deep-scrub starts 2024-02-23T00:57:21.906 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:20.705634+0000 osd.6 (osd.6) 364 : cluster [DBG] 2.14 deep-scrub ok 2024-02-23T00:57:21.906 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:20.871043+0000 osd.1 (osd.1) 328 : cluster [DBG] 18.1b scrub starts 2024-02-23T00:57:21.906 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:20.874933+0000 osd.1 (osd.1) 329 : cluster [DBG] 18.1b scrub ok 2024-02-23T00:57:21.906 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:21.255915+0000 osd.5 (osd.5) 400 : cluster [DBG] 6.a scrub starts 2024-02-23T00:57:21.906 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:21.257049+0000 osd.5 (osd.5) 401 : cluster [DBG] 6.a scrub ok 2024-02-23T00:57:21.925 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.6 dump_historic_ops 2024-02-23T00:57:22.192 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:57:22.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.268+0000 7f1426f6b700 1 -- 172.21.15.47:0/2172715818 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f14200fcbb0 msgr2=0x7f14200fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:22.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.268+0000 7f1426f6b700 1 --2- 172.21.15.47:0/2172715818 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f14200fcbb0 0x7f14200fcf90 secure :-1 s=READY pgs=2850 cs=0 l=1 rev1=1 rx=0x7f14140047b0 tx=0x7f1414009cb0).stop 2024-02-23T00:57:22.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.268+0000 7f1426f6b700 1 -- 172.21.15.47:0/2172715818 shutdown_connections 2024-02-23T00:57:22.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.268+0000 7f1426f6b700 1 --2- 172.21.15.47:0/2172715818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14200f7400 0x7f1420103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:22.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.268+0000 7f1426f6b700 1 --2- 172.21.15.47:0/2172715818 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f14200fcbb0 0x7f14200fcf90 unknown :-1 s=CLOSED pgs=2850 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:22.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.268+0000 7f1426f6b700 1 --2- 172.21.15.47:0/2172715818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14200fd510 0x7f14200f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:22.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.268+0000 7f1426f6b700 1 -- 172.21.15.47:0/2172715818 >> 172.21.15.47:0/2172715818 conn(0x7f14200f2820 msgr2=0x7f14200f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:22.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.268+0000 7f1426f6b700 1 -- 172.21.15.47:0/2172715818 shutdown_connections 2024-02-23T00:57:22.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.268+0000 7f1426f6b700 1 -- 172.21.15.47:0/2172715818 wait complete. 2024-02-23T00:57:22.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.268+0000 7f1426f6b700 1 librados: starting msgr at 2024-02-23T00:57:22.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.268+0000 7f1426f6b700 1 librados: starting objecter 2024-02-23T00:57:22.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.269+0000 7f1426f6b700 1 Processor -- start 2024-02-23T00:57:22.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.269+0000 7f1426f6b700 1 -- start start 2024-02-23T00:57:22.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.269+0000 7f1426f6b700 1 librados: setting wanted keys 2024-02-23T00:57:22.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.269+0000 7f1426f6b700 1 librados: calling monclient init 2024-02-23T00:57:22.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.269+0000 7f1426f6b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14200f7400 0x7f142010e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:22.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.269+0000 7f1426f6b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f14200fcbb0 0x7f142010e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:22.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.269+0000 7f1426f6b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14200fd510 0x7f1420108300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:22.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.269+0000 7f1426f6b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14201087f0 con 0x7f14200f7400 2024-02-23T00:57:22.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.269+0000 7f1426f6b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1420108960 con 0x7f14200fd510 2024-02-23T00:57:22.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.269+0000 7f1426f6b700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1420108b20 con 0x7f14200fcbb0 2024-02-23T00:57:22.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.269+0000 7f141ffff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f14200fcbb0 0x7f142010e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:22.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.269+0000 7f1424d07700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14200f7400 0x7f142010e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:22.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.269+0000 7f141ffff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f14200fcbb0 0x7f142010e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:44330/0 (socket says 172.21.15.47:44330) 2024-02-23T00:57:22.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.269+0000 7f141ffff700 1 -- 172.21.15.47:0/353268025 learned_addr learned my addr 172.21.15.47:0/353268025 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:57:22.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.269+0000 7f1424d07700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14200f7400 0x7f142010e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53122/0 (socket says 172.21.15.47:53122) 2024-02-23T00:57:22.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.269+0000 7f1425508700 1 --2- 172.21.15.47:0/353268025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14200fd510 0x7f1420108300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:22.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.269+0000 7f141ffff700 1 -- 172.21.15.47:0/353268025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14200fd510 msgr2=0x7f1420108300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:22.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.269+0000 7f141ffff700 1 --2- 172.21.15.47:0/353268025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14200fd510 0x7f1420108300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:22.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.269+0000 7f141ffff700 1 -- 172.21.15.47:0/353268025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14200f7400 msgr2=0x7f142010e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:22.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.269+0000 7f141ffff700 1 --2- 172.21.15.47:0/353268025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14200f7400 0x7f142010e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:22.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.269+0000 7f141ffff700 1 -- 172.21.15.47:0/353268025 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f140c005040 con 0x7f14200fcbb0 2024-02-23T00:57:22.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.269+0000 7f1424d07700 1 --2- 172.21.15.47:0/353268025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14200f7400 0x7f142010e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:57:22.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.270+0000 7f141ffff700 1 --2- 172.21.15.47:0/353268025 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f14200fcbb0 0x7f142010e5b0 secure :-1 s=READY pgs=2851 cs=0 l=1 rev1=1 rx=0x7f140c009400 tx=0x7f140c009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:22.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.270+0000 7f141dffb700 10 client.?.objecter ms_handle_connect 0x7f14200fcbb0 2024-02-23T00:57:22.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.270+0000 7f141dffb700 10 client.?.objecter resend_mon_ops 2024-02-23T00:57:22.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.270+0000 7f141dffb700 1 -- 172.21.15.47:0/353268025 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c007a40 con 0x7f14200fcbb0 2024-02-23T00:57:22.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.270+0000 7f141dffb700 1 -- 172.21.15.47:0/353268025 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f140c00f040 con 0x7f14200fcbb0 2024-02-23T00:57:22.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.270+0000 7f1426f6b700 1 -- 172.21.15.47:0/353268025 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1414005040 con 0x7f14200fcbb0 2024-02-23T00:57:22.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.270+0000 7f141dffb700 1 -- 172.21.15.47:0/353268025 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c014990 con 0x7f14200fcbb0 2024-02-23T00:57:22.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.270+0000 7f1426f6b700 10 client.42317.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:57:22.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.270+0000 7f1426f6b700 1 -- 172.21.15.47:0/353268025 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1420109160 con 0x7f14200fcbb0 2024-02-23T00:57:22.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.270+0000 7f1426f6b700 1 librados: init done 2024-02-23T00:57:22.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.271+0000 7f1426f6b700 1 -- 172.21.15.47:0/353268025 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f142005fa60 con 0x7f14200fcbb0 2024-02-23T00:57:22.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.271+0000 7f141dffb700 1 -- 172.21.15.47:0/353268025 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f140c00e630 con 0x7f14200fcbb0 2024-02-23T00:57:22.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.271+0000 7f141dffb700 1 --2- 172.21.15.47:0/353268025 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1410062cc0 0x7f1410065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:22.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.272+0000 7f141dffb700 1 -- 172.21.15.47:0/353268025 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(846..846 src has 670..846) v4 ==== 8663+0+0 (secure 0 0 0) 0x7f140c07d130 con 0x7f14200fcbb0 2024-02-23T00:57:22.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.272+0000 7f141dffb700 10 client.42317.objecter ms_dispatch 0x7f1420001550 osd_map(846..846 src has 670..846) v4 2024-02-23T00:57:22.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.272+0000 7f141dffb700 3 client.42317.objecter handle_osd_map got epochs [846,846] > 0 2024-02-23T00:57:22.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.272+0000 7f141dffb700 3 client.42317.objecter handle_osd_map decoding full epoch 846 2024-02-23T00:57:22.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.272+0000 7f1424d07700 1 --2- 172.21.15.47:0/353268025 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1410062cc0 0x7f1410065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:22.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.272+0000 7f141dffb700 20 client.42317.objecter dump_active .. 0 homeless 2024-02-23T00:57:22.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.272+0000 7f141dffb700 10 client.42317.objecter ms_handle_connect 0x7f1410062cc0 2024-02-23T00:57:22.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.272+0000 7f1424d07700 1 --2- 172.21.15.47:0/353268025 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1410062cc0 0x7f1410065180 secure :-1 s=READY pgs=3236 cs=0 l=1 rev1=1 rx=0x7f1414005010 tx=0x7f1414009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:22.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.276+0000 7f141dffb700 1 -- 172.21.15.47:0/353268025 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f140c049f60 con 0x7f14200fcbb0 2024-02-23T00:57:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.464+0000 7f1426f6b700 1 -- 172.21.15.47:0/353268025 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd reweight", "id": 5, "weight": 1.0} v 0) v1 -- 0x7f1420105ef0 con 0x7f14200fcbb0 2024-02-23T00:57:22.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.559+0000 7f141dffb700 1 -- 172.21.15.47:0/353268025 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 5, "weight": 1.0}]=0 reweighted osd.5 to 1 (10000) v847) v1 ==== 113+0+0 (secure 0 0 0) 0x7f140c04c970 con 0x7f14200fcbb0 2024-02-23T00:57:22.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.561+0000 7f1426f6b700 10 librados: watch_flush enter 2024-02-23T00:57:22.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.561+0000 7f1426f6b700 10 librados: watch_flush exit 2024-02-23T00:57:22.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.561+0000 7f1426f6b700 20 client.42317.objecter shutdown clearing up homeless session... 2024-02-23T00:57:22.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.561+0000 7f1426f6b700 10 client.42317.objecter successfully canceled tick 2024-02-23T00:57:22.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.561+0000 7f1426f6b700 1 -- 172.21.15.47:0/353268025 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1410062cc0 msgr2=0x7f1410065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:22.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.561+0000 7f1426f6b700 1 --2- 172.21.15.47:0/353268025 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1410062cc0 0x7f1410065180 secure :-1 s=READY pgs=3236 cs=0 l=1 rev1=1 rx=0x7f1414005010 tx=0x7f1414009cb0).stop 2024-02-23T00:57:22.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.561+0000 7f1426f6b700 1 -- 172.21.15.47:0/353268025 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f14200fcbb0 msgr2=0x7f142010e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:22.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.561+0000 7f1426f6b700 1 --2- 172.21.15.47:0/353268025 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f14200fcbb0 0x7f142010e5b0 secure :-1 s=READY pgs=2851 cs=0 l=1 rev1=1 rx=0x7f140c009400 tx=0x7f140c009cb0).stop 2024-02-23T00:57:22.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.561+0000 7f1426f6b700 1 -- 172.21.15.47:0/353268025 shutdown_connections 2024-02-23T00:57:22.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.561+0000 7f1426f6b700 1 --2- 172.21.15.47:0/353268025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14200fd510 0x7f1420108300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:22.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.561+0000 7f1426f6b700 1 --2- 172.21.15.47:0/353268025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14200f7400 0x7f142010e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:22.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.561+0000 7f1426f6b700 1 --2- 172.21.15.47:0/353268025 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1410062cc0 0x7f1410065180 unknown :-1 s=CLOSED pgs=3236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:22.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.561+0000 7f1426f6b700 1 --2- 172.21.15.47:0/353268025 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f14200fcbb0 0x7f142010e5b0 unknown :-1 s=CLOSED pgs=2851 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:22.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.561+0000 7f1426f6b700 1 -- 172.21.15.47:0/353268025 >> 172.21.15.47:0/353268025 conn(0x7f14200f2820 msgr2=0x7f1420060ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:22.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.561+0000 7f1426f6b700 1 -- 172.21.15.47:0/353268025 shutdown_connections 2024-02-23T00:57:22.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.561+0000 7f1426f6b700 1 -- 172.21.15.47:0/353268025 wait complete. 2024-02-23T00:57:22.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:22.561+0000 7f1426f6b700 1 librados: shutdown 2024-02-23T00:57:22.565 INFO:teuthology.orchestra.run.smithi047.stderr:reweighted osd.5 to 1 (10000) 2024-02-23T00:57:22.925 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:21.091166+0000 osd.0 (osd.0) 336 : cluster [DBG] 4.b deep-scrub starts 2024-02-23T00:57:22.925 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:21.095252+0000 osd.0 (osd.0) 337 : cluster [DBG] 4.b deep-scrub ok 2024-02-23T00:57:22.925 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:21.259912+0000 osd.7 (osd.7) 409 : cluster [DBG] 1.0 scrub starts 2024-02-23T00:57:22.925 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:21.262790+0000 osd.7 (osd.7) 410 : cluster [DBG] 1.0 scrub ok 2024-02-23T00:57:22.925 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:21.428341+0000 mgr.y (mgr.14517) 728 : cluster [DBG] pgmap v1306: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 648 B/s rd, 0 op/s 2024-02-23T00:57:22.925 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:21.851514+0000 osd.1 (osd.1) 330 : cluster [DBG] 2.6 scrub starts 2024-02-23T00:57:22.925 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:21.852469+0000 osd.1 (osd.1) 331 : cluster [DBG] 2.6 scrub ok 2024-02-23T00:57:22.925 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:22.057958+0000 osd.3 (osd.3) 331 : cluster [DBG] 5.13 scrub starts 2024-02-23T00:57:22.925 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:22.059017+0000 osd.3 (osd.3) 332 : cluster [DBG] 5.13 scrub ok 2024-02-23T00:57:22.925 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:22.208226+0000 osd.5 (osd.5) 402 : cluster [DBG] 2.3 scrub starts 2024-02-23T00:57:22.925 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:22.209006+0000 osd.5 (osd.5) 403 : cluster [DBG] 2.3 scrub ok 2024-02-23T00:57:22.925 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:22.465238+0000 mon.c (mon.2) 530 : audit [INF] from='client.? 172.21.15.47:0/353268025' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-23T00:57:22.926 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:22.465523+0000 mon.a (mon.0) 3918 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-23T00:57:22.926 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:21.091166+0000 osd.0 (osd.0) 336 : cluster [DBG] 4.b deep-scrub starts 2024-02-23T00:57:22.926 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:21.095252+0000 osd.0 (osd.0) 337 : cluster [DBG] 4.b deep-scrub ok 2024-02-23T00:57:22.926 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:21.259912+0000 osd.7 (osd.7) 409 : cluster [DBG] 1.0 scrub starts 2024-02-23T00:57:22.926 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:21.262790+0000 osd.7 (osd.7) 410 : cluster [DBG] 1.0 scrub ok 2024-02-23T00:57:22.926 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:21.428341+0000 mgr.y (mgr.14517) 728 : cluster [DBG] pgmap v1306: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 648 B/s rd, 0 op/s 2024-02-23T00:57:22.926 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:21.851514+0000 osd.1 (osd.1) 330 : cluster [DBG] 2.6 scrub starts 2024-02-23T00:57:22.926 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:21.852469+0000 osd.1 (osd.1) 331 : cluster [DBG] 2.6 scrub ok 2024-02-23T00:57:22.926 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:22.057958+0000 osd.3 (osd.3) 331 : cluster [DBG] 5.13 scrub starts 2024-02-23T00:57:22.926 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:22.059017+0000 osd.3 (osd.3) 332 : cluster [DBG] 5.13 scrub ok 2024-02-23T00:57:22.926 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:22.208226+0000 osd.5 (osd.5) 402 : cluster [DBG] 2.3 scrub starts 2024-02-23T00:57:22.926 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:22.209006+0000 osd.5 (osd.5) 403 : cluster [DBG] 2.3 scrub ok 2024-02-23T00:57:22.927 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:22.465238+0000 mon.c (mon.2) 530 : audit [INF] from='client.? 172.21.15.47:0/353268025' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-23T00:57:22.927 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:22.465523+0000 mon.a (mon.0) 3918 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-23T00:57:22.944 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:57:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:57:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:57:22.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:21.091166+0000 osd.0 (osd.0) 336 : cluster [DBG] 4.b deep-scrub starts 2024-02-23T00:57:22.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:21.095252+0000 osd.0 (osd.0) 337 : cluster [DBG] 4.b deep-scrub ok 2024-02-23T00:57:22.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:21.259912+0000 osd.7 (osd.7) 409 : cluster [DBG] 1.0 scrub starts 2024-02-23T00:57:22.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:21.262790+0000 osd.7 (osd.7) 410 : cluster [DBG] 1.0 scrub ok 2024-02-23T00:57:22.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:21.428341+0000 mgr.y (mgr.14517) 728 : cluster [DBG] pgmap v1306: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 648 B/s rd, 0 op/s 2024-02-23T00:57:22.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:21.851514+0000 osd.1 (osd.1) 330 : cluster [DBG] 2.6 scrub starts 2024-02-23T00:57:22.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:21.852469+0000 osd.1 (osd.1) 331 : cluster [DBG] 2.6 scrub ok 2024-02-23T00:57:22.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:22.057958+0000 osd.3 (osd.3) 331 : cluster [DBG] 5.13 scrub starts 2024-02-23T00:57:22.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:22.059017+0000 osd.3 (osd.3) 332 : cluster [DBG] 5.13 scrub ok 2024-02-23T00:57:22.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:22.208226+0000 osd.5 (osd.5) 402 : cluster [DBG] 2.3 scrub starts 2024-02-23T00:57:22.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:22.209006+0000 osd.5 (osd.5) 403 : cluster [DBG] 2.3 scrub ok 2024-02-23T00:57:22.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:22.465238+0000 mon.c (mon.2) 530 : audit [INF] from='client.? 172.21.15.47:0/353268025' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-23T00:57:22.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:22.465523+0000 mon.a (mon.0) 3918 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-23T00:57:23.354 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd primary-affinity 5 1 2024-02-23T00:57:23.403 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:57:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:57:23.643 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:57:23.668 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:22.224759+0000 osd.7 (osd.7) 411 : cluster [DBG] 2.5 scrub starts 2024-02-23T00:57:23.668 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:22.227649+0000 osd.7 (osd.7) 412 : cluster [DBG] 2.5 scrub ok 2024-02-23T00:57:23.668 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:22.558802+0000 mon.a (mon.0) 3919 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-23T00:57:23.668 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:22.558905+0000 mon.a (mon.0) 3920 : cluster [DBG] osdmap e847: 8 total, 8 up, 8 in 2024-02-23T00:57:23.669 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:23.104157+0000 osd.3 (osd.3) 333 : cluster [DBG] 18.16 scrub starts 2024-02-23T00:57:23.669 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:23.105219+0000 osd.3 (osd.3) 334 : cluster [DBG] 18.16 scrub ok 2024-02-23T00:57:23.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:22.224759+0000 osd.7 (osd.7) 411 : cluster [DBG] 2.5 scrub starts 2024-02-23T00:57:23.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:22.227649+0000 osd.7 (osd.7) 412 : cluster [DBG] 2.5 scrub ok 2024-02-23T00:57:23.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:22.558802+0000 mon.a (mon.0) 3919 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-23T00:57:23.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:22.558905+0000 mon.a (mon.0) 3920 : cluster [DBG] osdmap e847: 8 total, 8 up, 8 in 2024-02-23T00:57:23.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:23.104157+0000 osd.3 (osd.3) 333 : cluster [DBG] 18.16 scrub starts 2024-02-23T00:57:23.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:23.105219+0000 osd.3 (osd.3) 334 : cluster [DBG] 18.16 scrub ok 2024-02-23T00:57:23.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:22.224759+0000 osd.7 (osd.7) 411 : cluster [DBG] 2.5 scrub starts 2024-02-23T00:57:23.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:22.227649+0000 osd.7 (osd.7) 412 : cluster [DBG] 2.5 scrub ok 2024-02-23T00:57:23.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:22.558802+0000 mon.a (mon.0) 3919 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-23T00:57:23.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:22.558905+0000 mon.a (mon.0) 3920 : cluster [DBG] osdmap e847: 8 total, 8 up, 8 in 2024-02-23T00:57:23.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:23.104157+0000 osd.3 (osd.3) 333 : cluster [DBG] 18.16 scrub starts 2024-02-23T00:57:23.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:23.105219+0000 osd.3 (osd.3) 334 : cluster [DBG] 18.16 scrub ok 2024-02-23T00:57:23.945 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:57:23.945 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:57:24.656 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.7 dump_ops_in_flight 2024-02-23T00:57:24.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:23.061923+0000 mgr.y (mgr.14517) 729 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:57:24.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:23.244180+0000 osd.7 (osd.7) 413 : cluster [DBG] 2.b scrub starts 2024-02-23T00:57:24.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:23.247061+0000 osd.7 (osd.7) 414 : cluster [DBG] 2.b scrub ok 2024-02-23T00:57:24.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:23.428708+0000 mgr.y (mgr.14517) 730 : cluster [DBG] pgmap v1308: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-23T00:57:24.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:24.156802+0000 osd.5 (osd.5) 404 : cluster [DBG] 18.c scrub starts 2024-02-23T00:57:24.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:24.160208+0000 osd.5 (osd.5) 405 : cluster [DBG] 18.c scrub ok 2024-02-23T00:57:24.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:23.061923+0000 mgr.y (mgr.14517) 729 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:57:24.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:23.244180+0000 osd.7 (osd.7) 413 : cluster [DBG] 2.b scrub starts 2024-02-23T00:57:24.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:23.247061+0000 osd.7 (osd.7) 414 : cluster [DBG] 2.b scrub ok 2024-02-23T00:57:24.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:23.428708+0000 mgr.y (mgr.14517) 730 : cluster [DBG] pgmap v1308: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-23T00:57:24.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:24.156802+0000 osd.5 (osd.5) 404 : cluster [DBG] 18.c scrub starts 2024-02-23T00:57:24.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:24.160208+0000 osd.5 (osd.5) 405 : cluster [DBG] 18.c scrub ok 2024-02-23T00:57:24.921 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:57:24.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:23.061923+0000 mgr.y (mgr.14517) 729 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:57:24.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:23.244180+0000 osd.7 (osd.7) 413 : cluster [DBG] 2.b scrub starts 2024-02-23T00:57:24.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:23.247061+0000 osd.7 (osd.7) 414 : cluster [DBG] 2.b scrub ok 2024-02-23T00:57:24.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:23.428708+0000 mgr.y (mgr.14517) 730 : cluster [DBG] pgmap v1308: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-23T00:57:24.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:24.156802+0000 osd.5 (osd.5) 404 : cluster [DBG] 18.c scrub starts 2024-02-23T00:57:24.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:24.160208+0000 osd.5 (osd.5) 405 : cluster [DBG] 18.c scrub ok 2024-02-23T00:57:25.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.817+0000 7f0d8e463700 1 -- 172.21.15.47:0/256938411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d88102210 msgr2=0x7f0d88104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:25.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.817+0000 7f0d8e463700 1 --2- 172.21.15.47:0/256938411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d88102210 0x7f0d88104600 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 rx=0x7f0d80009be0 tx=0x7f0d80004d10).stop 2024-02-23T00:57:25.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.818+0000 7f0d8e463700 1 -- 172.21.15.47:0/256938411 shutdown_connections 2024-02-23T00:57:25.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.818+0000 7f0d8e463700 1 --2- 172.21.15.47:0/256938411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d88102210 0x7f0d88104600 unknown :-1 s=CLOSED pgs=2672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:25.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.818+0000 7f0d8e463700 1 --2- 172.21.15.47:0/256938411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d880f8cb0 0x7f0d880f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:25.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.818+0000 7f0d8e463700 1 --2- 172.21.15.47:0/256938411 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0d880f9610 0x7f0d88101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:25.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.818+0000 7f0d8e463700 1 -- 172.21.15.47:0/256938411 >> 172.21.15.47:0/256938411 conn(0x7f0d880f2820 msgr2=0x7f0d880f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:25.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.818+0000 7f0d8e463700 1 -- 172.21.15.47:0/256938411 shutdown_connections 2024-02-23T00:57:25.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.818+0000 7f0d8e463700 1 -- 172.21.15.47:0/256938411 wait complete. 2024-02-23T00:57:25.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.818+0000 7f0d8e463700 1 librados: starting msgr at 2024-02-23T00:57:25.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.818+0000 7f0d8e463700 1 librados: starting objecter 2024-02-23T00:57:25.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.818+0000 7f0d8e463700 1 Processor -- start 2024-02-23T00:57:25.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.818+0000 7f0d8e463700 1 -- start start 2024-02-23T00:57:25.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.818+0000 7f0d8e463700 1 librados: setting wanted keys 2024-02-23T00:57:25.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.818+0000 7f0d8e463700 1 librados: calling monclient init 2024-02-23T00:57:25.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.819+0000 7f0d8e463700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d880f8cb0 0x7f0d880f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:25.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.819+0000 7f0d8e463700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d880f9610 0x7f0d880f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:25.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.819+0000 7f0d8e463700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0d88102210 0x7f0d880f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:25.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.819+0000 7f0d8e463700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d881162c0 con 0x7f0d880f8cb0 2024-02-23T00:57:25.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.819+0000 7f0d8e463700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d88116430 con 0x7f0d880f9610 2024-02-23T00:57:25.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.819+0000 7f0d8e463700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d881165f0 con 0x7f0d88102210 2024-02-23T00:57:25.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.819+0000 7f0d8ca00700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0d88102210 0x7f0d880f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:25.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.819+0000 7f0d87fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d880f8cb0 0x7f0d880f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:25.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.819+0000 7f0d8ca00700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0d88102210 0x7f0d880f7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:44342/0 (socket says 172.21.15.47:44342) 2024-02-23T00:57:25.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.819+0000 7f0d8ca00700 1 -- 172.21.15.47:0/535712055 learned_addr learned my addr 172.21.15.47:0/535712055 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:57:25.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.819+0000 7f0d87fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d880f8cb0 0x7f0d880f6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53128/0 (socket says 172.21.15.47:53128) 2024-02-23T00:57:25.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.819+0000 7f0d877fe700 1 --2- 172.21.15.47:0/535712055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d880f9610 0x7f0d880f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:25.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.819+0000 7f0d8ca00700 1 -- 172.21.15.47:0/535712055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d880f9610 msgr2=0x7f0d880f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:25.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.819+0000 7f0d8ca00700 1 --2- 172.21.15.47:0/535712055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d880f9610 0x7f0d880f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:25.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.819+0000 7f0d8ca00700 1 -- 172.21.15.47:0/535712055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d880f8cb0 msgr2=0x7f0d880f6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:25.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.819+0000 7f0d8ca00700 1 --2- 172.21.15.47:0/535712055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d880f8cb0 0x7f0d880f6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:25.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.819+0000 7f0d8ca00700 1 -- 172.21.15.47:0/535712055 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d70005040 con 0x7f0d88102210 2024-02-23T00:57:25.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.819+0000 7f0d87fff700 1 --2- 172.21.15.47:0/535712055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d880f8cb0 0x7f0d880f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:57:25.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.819+0000 7f0d8ca00700 1 --2- 172.21.15.47:0/535712055 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0d88102210 0x7f0d880f7640 secure :-1 s=READY pgs=2852 cs=0 l=1 rev1=1 rx=0x7f0d80009760 tx=0x7f0d80004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:25.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.819+0000 7f0d857fa700 10 client.?.objecter ms_handle_connect 0x7f0d88102210 2024-02-23T00:57:25.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.819+0000 7f0d857fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:57:25.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.820+0000 7f0d857fa700 1 -- 172.21.15.47:0/535712055 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8000e040 con 0x7f0d88102210 2024-02-23T00:57:25.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.820+0000 7f0d857fa700 1 -- 172.21.15.47:0/535712055 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f0d80018040 con 0x7f0d88102210 2024-02-23T00:57:25.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.820+0000 7f0d8e463700 1 -- 172.21.15.47:0/535712055 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d800097c0 con 0x7f0d88102210 2024-02-23T00:57:25.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.820+0000 7f0d857fa700 1 -- 172.21.15.47:0/535712055 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d80013d30 con 0x7f0d88102210 2024-02-23T00:57:25.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.820+0000 7f0d8e463700 10 client.42323.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:57:25.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.820+0000 7f0d8e463700 1 -- 172.21.15.47:0/535712055 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d88116ae0 con 0x7f0d88102210 2024-02-23T00:57:25.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.820+0000 7f0d8e463700 1 librados: init done 2024-02-23T00:57:25.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.821+0000 7f0d8e463700 1 -- 172.21.15.47:0/535712055 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d8805fa60 con 0x7f0d88102210 2024-02-23T00:57:25.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.821+0000 7f0d857fa700 1 -- 172.21.15.47:0/535712055 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f0d80013420 con 0x7f0d88102210 2024-02-23T00:57:25.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.821+0000 7f0d857fa700 1 --2- 172.21.15.47:0/535712055 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0d74062cc0 0x7f0d74065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:25.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.822+0000 7f0d857fa700 1 -- 172.21.15.47:0/535712055 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(848..848 src has 670..848) v4 ==== 8663+0+0 (secure 0 0 0) 0x7f0d8007dfd0 con 0x7f0d88102210 2024-02-23T00:57:25.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.822+0000 7f0d857fa700 10 client.42323.objecter ms_dispatch 0x7f0d88001550 osd_map(848..848 src has 670..848) v4 2024-02-23T00:57:25.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.822+0000 7f0d857fa700 3 client.42323.objecter handle_osd_map got epochs [848,848] > 0 2024-02-23T00:57:25.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.822+0000 7f0d857fa700 3 client.42323.objecter handle_osd_map decoding full epoch 848 2024-02-23T00:57:25.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.822+0000 7f0d87fff700 1 --2- 172.21.15.47:0/535712055 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0d74062cc0 0x7f0d74065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:25.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.822+0000 7f0d857fa700 20 client.42323.objecter dump_active .. 0 homeless 2024-02-23T00:57:25.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.822+0000 7f0d857fa700 10 client.42323.objecter ms_handle_connect 0x7f0d74062cc0 2024-02-23T00:57:25.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.822+0000 7f0d87fff700 1 --2- 172.21.15.47:0/535712055 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0d74062cc0 0x7f0d74065180 secure :-1 s=READY pgs=3237 cs=0 l=1 rev1=1 rx=0x7f0d78010420 tx=0x7f0d7800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:25.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:25.826+0000 7f0d857fa700 1 -- 172.21.15.47:0/535712055 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f0d800acf60 con 0x7f0d88102210 2024-02-23T00:57:26.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:26.011+0000 7f0d8e463700 1 -- 172.21.15.47:0/535712055 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd primary-affinity", "id": 5, "weight": 1.0} v 0) v1 -- 0x7f0d880f8140 con 0x7f0d88102210 2024-02-23T00:57:26.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:26.349+0000 7f0d857fa700 1 -- 172.21.15.47:0/535712055 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]=0 set osd.5 primary-affinity to 1 (8655362) v849) v1 ==== 133+0+0 (secure 0 0 0) 0x7f0d80021070 con 0x7f0d88102210 2024-02-23T00:57:26.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:26.351+0000 7f0d8e463700 10 librados: watch_flush enter 2024-02-23T00:57:26.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:26.351+0000 7f0d8e463700 10 librados: watch_flush exit 2024-02-23T00:57:26.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:26.351+0000 7f0d8e463700 20 client.42323.objecter shutdown clearing up homeless session... 2024-02-23T00:57:26.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:26.351+0000 7f0d8e463700 10 client.42323.objecter successfully canceled tick 2024-02-23T00:57:26.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:26.351+0000 7f0d8e463700 1 -- 172.21.15.47:0/535712055 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0d74062cc0 msgr2=0x7f0d74065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:26.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:26.351+0000 7f0d8e463700 1 --2- 172.21.15.47:0/535712055 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0d74062cc0 0x7f0d74065180 secure :-1 s=READY pgs=3237 cs=0 l=1 rev1=1 rx=0x7f0d78010420 tx=0x7f0d7800b040).stop 2024-02-23T00:57:26.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:26.351+0000 7f0d8e463700 1 -- 172.21.15.47:0/535712055 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0d88102210 msgr2=0x7f0d880f7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:26.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:26.351+0000 7f0d8e463700 1 --2- 172.21.15.47:0/535712055 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0d88102210 0x7f0d880f7640 secure :-1 s=READY pgs=2852 cs=0 l=1 rev1=1 rx=0x7f0d80009760 tx=0x7f0d80004d10).stop 2024-02-23T00:57:26.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:26.351+0000 7f0d8e463700 1 -- 172.21.15.47:0/535712055 shutdown_connections 2024-02-23T00:57:26.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:26.351+0000 7f0d8e463700 1 --2- 172.21.15.47:0/535712055 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0d74062cc0 0x7f0d74065180 unknown :-1 s=CLOSED pgs=3237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:26.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:26.351+0000 7f0d8e463700 1 --2- 172.21.15.47:0/535712055 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0d88102210 0x7f0d880f7640 unknown :-1 s=CLOSED pgs=2852 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:26.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:26.351+0000 7f0d8e463700 1 --2- 172.21.15.47:0/535712055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d880f8cb0 0x7f0d880f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:26.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:26.351+0000 7f0d8e463700 1 --2- 172.21.15.47:0/535712055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d880f9610 0x7f0d880f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:26.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:26.351+0000 7f0d8e463700 1 -- 172.21.15.47:0/535712055 >> 172.21.15.47:0/535712055 conn(0x7f0d880f2820 msgr2=0x7f0d880f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:26.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:26.351+0000 7f0d8e463700 1 -- 172.21.15.47:0/535712055 shutdown_connections 2024-02-23T00:57:26.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:26.351+0000 7f0d8e463700 1 -- 172.21.15.47:0/535712055 wait complete. 2024-02-23T00:57:26.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:26.351+0000 7f0d8e463700 1 librados: shutdown 2024-02-23T00:57:26.356 INFO:teuthology.orchestra.run.smithi047.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-02-23T00:57:26.592 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:57:26.592 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:57:26.611 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:25.348560+0000 mon.a (mon.0) 3921 : cluster [DBG] osdmap e848: 8 total, 8 up, 8 in 2024-02-23T00:57:26.611 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:25.429150+0000 mgr.y (mgr.14517) 731 : cluster [DBG] pgmap v1310: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:57:26.611 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:25.843880+0000 osd.1 (osd.1) 332 : cluster [DBG] 2.d scrub starts 2024-02-23T00:57:26.611 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:25.845065+0000 osd.1 (osd.1) 333 : cluster [DBG] 2.d scrub ok 2024-02-23T00:57:26.611 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:26.013139+0000 mon.c (mon.2) 531 : audit [INF] from='client.? 172.21.15.47:0/535712055' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-02-23T00:57:26.611 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:26.013474+0000 mon.a (mon.0) 3922 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-02-23T00:57:26.611 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:26.049047+0000 osd.3 (osd.3) 335 : cluster [DBG] 5.17 scrub starts 2024-02-23T00:57:26.612 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:26.050006+0000 osd.3 (osd.3) 336 : cluster [DBG] 5.17 scrub ok 2024-02-23T00:57:26.664 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:25.348560+0000 mon.a (mon.0) 3921 : cluster [DBG] osdmap e848: 8 total, 8 up, 8 in 2024-02-23T00:57:26.664 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:25.429150+0000 mgr.y (mgr.14517) 731 : cluster [DBG] pgmap v1310: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:57:26.664 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:25.843880+0000 osd.1 (osd.1) 332 : cluster [DBG] 2.d scrub starts 2024-02-23T00:57:26.664 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:25.845065+0000 osd.1 (osd.1) 333 : cluster [DBG] 2.d scrub ok 2024-02-23T00:57:26.664 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:26.013139+0000 mon.c (mon.2) 531 : audit [INF] from='client.? 172.21.15.47:0/535712055' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-02-23T00:57:26.664 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:26.013474+0000 mon.a (mon.0) 3922 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-02-23T00:57:26.664 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:26.049047+0000 osd.3 (osd.3) 335 : cluster [DBG] 5.17 scrub starts 2024-02-23T00:57:26.664 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:26.050006+0000 osd.3 (osd.3) 336 : cluster [DBG] 5.17 scrub ok 2024-02-23T00:57:26.664 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:25.348560+0000 mon.a (mon.0) 3921 : cluster [DBG] osdmap e848: 8 total, 8 up, 8 in 2024-02-23T00:57:26.665 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:25.429150+0000 mgr.y (mgr.14517) 731 : cluster [DBG] pgmap v1310: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:57:26.665 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:25.843880+0000 osd.1 (osd.1) 332 : cluster [DBG] 2.d scrub starts 2024-02-23T00:57:26.665 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:25.845065+0000 osd.1 (osd.1) 333 : cluster [DBG] 2.d scrub ok 2024-02-23T00:57:26.665 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:26.013139+0000 mon.c (mon.2) 531 : audit [INF] from='client.? 172.21.15.47:0/535712055' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-02-23T00:57:26.665 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:26.013474+0000 mon.a (mon.0) 3922 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-02-23T00:57:26.665 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:26.049047+0000 osd.3 (osd.3) 335 : cluster [DBG] 5.17 scrub starts 2024-02-23T00:57:26.665 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:26.050006+0000 osd.3 (osd.3) 336 : cluster [DBG] 5.17 scrub ok 2024-02-23T00:57:27.102 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight 6 1 2024-02-23T00:57:27.200 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.7 dump_blocked_ops 2024-02-23T00:57:27.389 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:57:27.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:26.349805+0000 mon.a (mon.0) 3923 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-02-23T00:57:27.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:26.349897+0000 mon.a (mon.0) 3924 : cluster [DBG] osdmap e849: 8 total, 8 up, 8 in 2024-02-23T00:57:27.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:27.057599+0000 osd.3 (osd.3) 337 : cluster [DBG] 5.0 scrub starts 2024-02-23T00:57:27.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:27.060581+0000 osd.3 (osd.3) 338 : cluster [DBG] 5.0 scrub ok 2024-02-23T00:57:27.471 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:57:27.618 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:26.349805+0000 mon.a (mon.0) 3923 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-02-23T00:57:27.618 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:26.349897+0000 mon.a (mon.0) 3924 : cluster [DBG] osdmap e849: 8 total, 8 up, 8 in 2024-02-23T00:57:27.618 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:27.057599+0000 osd.3 (osd.3) 337 : cluster [DBG] 5.0 scrub starts 2024-02-23T00:57:27.618 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:27.060581+0000 osd.3 (osd.3) 338 : cluster [DBG] 5.0 scrub ok 2024-02-23T00:57:27.619 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:26.349805+0000 mon.a (mon.0) 3923 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-02-23T00:57:27.619 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:26.349897+0000 mon.a (mon.0) 3924 : cluster [DBG] osdmap e849: 8 total, 8 up, 8 in 2024-02-23T00:57:27.619 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:27.057599+0000 osd.3 (osd.3) 337 : cluster [DBG] 5.0 scrub starts 2024-02-23T00:57:27.619 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:27.060581+0000 osd.3 (osd.3) 338 : cluster [DBG] 5.0 scrub ok 2024-02-23T00:57:28.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:27.220048+0000 osd.7 (osd.7) 415 : cluster [DBG] 18.13 scrub starts 2024-02-23T00:57:28.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:27.224320+0000 osd.7 (osd.7) 416 : cluster [DBG] 18.13 scrub ok 2024-02-23T00:57:28.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:27.429957+0000 mgr.y (mgr.14517) 732 : cluster [DBG] pgmap v1312: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-23T00:57:28.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:28.008553+0000 osd.3 (osd.3) 339 : cluster [DBG] 5.26 scrub starts 2024-02-23T00:57:28.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:28.009631+0000 osd.3 (osd.3) 340 : cluster [DBG] 5.26 scrub ok 2024-02-23T00:57:28.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:27.220048+0000 osd.7 (osd.7) 415 : cluster [DBG] 18.13 scrub starts 2024-02-23T00:57:28.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:27.224320+0000 osd.7 (osd.7) 416 : cluster [DBG] 18.13 scrub ok 2024-02-23T00:57:28.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:27.429957+0000 mgr.y (mgr.14517) 732 : cluster [DBG] pgmap v1312: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-23T00:57:28.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:28.008553+0000 osd.3 (osd.3) 339 : cluster [DBG] 5.26 scrub starts 2024-02-23T00:57:28.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:28.009631+0000 osd.3 (osd.3) 340 : cluster [DBG] 5.26 scrub ok 2024-02-23T00:57:28.621 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:27.220048+0000 osd.7 (osd.7) 415 : cluster [DBG] 18.13 scrub starts 2024-02-23T00:57:28.622 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:27.224320+0000 osd.7 (osd.7) 416 : cluster [DBG] 18.13 scrub ok 2024-02-23T00:57:28.622 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:27.429957+0000 mgr.y (mgr.14517) 732 : cluster [DBG] pgmap v1312: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-23T00:57:28.622 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:28.008553+0000 osd.3 (osd.3) 339 : cluster [DBG] 5.26 scrub starts 2024-02-23T00:57:28.622 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:28.009631+0000 osd.3 (osd.3) 340 : cluster [DBG] 5.26 scrub ok 2024-02-23T00:57:29.162 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:57:29.162 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:57:29.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.212+0000 7fba37db1700 1 -- 172.21.15.47:0/847170594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba300f7400 msgr2=0x7fba30103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:29.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.212+0000 7fba37db1700 1 --2- 172.21.15.47:0/847170594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba300f7400 0x7fba30103a20 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 rx=0x7fba28009be0 tx=0x7fba28004d10).stop 2024-02-23T00:57:29.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.212+0000 7fba37db1700 1 -- 172.21.15.47:0/847170594 shutdown_connections 2024-02-23T00:57:29.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.212+0000 7fba37db1700 1 --2- 172.21.15.47:0/847170594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba300f7400 0x7fba30103a20 unknown :-1 s=CLOSED pgs=2673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:29.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.212+0000 7fba37db1700 1 --2- 172.21.15.47:0/847170594 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba300fcbb0 0x7fba300fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:29.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.212+0000 7fba37db1700 1 --2- 172.21.15.47:0/847170594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba300fd510 0x7fba300f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:29.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.212+0000 7fba37db1700 1 -- 172.21.15.47:0/847170594 >> 172.21.15.47:0/847170594 conn(0x7fba300f2820 msgr2=0x7fba300f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:29.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.212+0000 7fba37db1700 1 -- 172.21.15.47:0/847170594 shutdown_connections 2024-02-23T00:57:29.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.212+0000 7fba37db1700 1 -- 172.21.15.47:0/847170594 wait complete. 2024-02-23T00:57:29.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.213+0000 7fba37db1700 1 librados: starting msgr at 2024-02-23T00:57:29.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.213+0000 7fba37db1700 1 librados: starting objecter 2024-02-23T00:57:29.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.213+0000 7fba37db1700 1 Processor -- start 2024-02-23T00:57:29.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.213+0000 7fba37db1700 1 -- start start 2024-02-23T00:57:29.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.213+0000 7fba37db1700 1 librados: setting wanted keys 2024-02-23T00:57:29.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.213+0000 7fba37db1700 1 librados: calling monclient init 2024-02-23T00:57:29.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.213+0000 7fba37db1700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba300f7400 0x7fba3010cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:29.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.213+0000 7fba37db1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba300fcbb0 0x7fba3010d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:29.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.213+0000 7fba37db1700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba300fd510 0x7fba30106ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:29.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.213+0000 7fba37db1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba301073e0 con 0x7fba300fcbb0 2024-02-23T00:57:29.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.213+0000 7fba37db1700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba30107550 con 0x7fba300f7400 2024-02-23T00:57:29.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.213+0000 7fba37db1700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba30107710 con 0x7fba300fd510 2024-02-23T00:57:29.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.213+0000 7fba3634e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba300fd510 0x7fba30106ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:29.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.213+0000 7fba35b4d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba300f7400 0x7fba3010cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:29.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.213+0000 7fba35b4d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba300f7400 0x7fba3010cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.47:58536/0 (socket says 172.21.15.47:58536) 2024-02-23T00:57:29.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.213+0000 7fba3534c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba300fcbb0 0x7fba3010d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:29.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.213+0000 7fba35b4d700 1 -- 172.21.15.47:0/691208725 learned_addr learned my addr 172.21.15.47:0/691208725 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:57:29.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.214+0000 7fba35b4d700 1 -- 172.21.15.47:0/691208725 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba300fd510 msgr2=0x7fba30106ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:29.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.214+0000 7fba35b4d700 1 --2- 172.21.15.47:0/691208725 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba300fd510 0x7fba30106ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:29.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.214+0000 7fba35b4d700 1 -- 172.21.15.47:0/691208725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba300fcbb0 msgr2=0x7fba3010d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:29.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.214+0000 7fba35b4d700 1 --2- 172.21.15.47:0/691208725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba300fcbb0 0x7fba3010d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:29.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.214+0000 7fba35b4d700 1 -- 172.21.15.47:0/691208725 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba20005040 con 0x7fba300f7400 2024-02-23T00:57:29.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.214+0000 7fba3634e700 1 --2- 172.21.15.47:0/691208725 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba300fd510 0x7fba30106ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:57:29.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.214+0000 7fba3534c700 1 --2- 172.21.15.47:0/691208725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba300fcbb0 0x7fba3010d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:57:29.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.214+0000 7fba35b4d700 1 --2- 172.21.15.47:0/691208725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba300f7400 0x7fba3010cd50 secure :-1 s=READY pgs=3986 cs=0 l=1 rev1=1 rx=0x7fba200048f0 tx=0x7fba20009cb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:29.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.214+0000 7fba26ffd700 10 client.?.objecter ms_handle_connect 0x7fba300f7400 2024-02-23T00:57:29.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.214+0000 7fba26ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:57:29.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.214+0000 7fba26ffd700 1 -- 172.21.15.47:0/691208725 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba20017e30 con 0x7fba300f7400 2024-02-23T00:57:29.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.214+0000 7fba37db1700 1 -- 172.21.15.47:0/691208725 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba280097c0 con 0x7fba300f7400 2024-02-23T00:57:29.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.214+0000 7fba26ffd700 1 -- 172.21.15.47:0/691208725 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fba20007b00 con 0x7fba300f7400 2024-02-23T00:57:29.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.214+0000 7fba26ffd700 1 -- 172.21.15.47:0/691208725 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2001e5b0 con 0x7fba300f7400 2024-02-23T00:57:29.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.214+0000 7fba37db1700 10 client.44266.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:57:29.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.214+0000 7fba37db1700 1 -- 172.21.15.47:0/691208725 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba30107d40 con 0x7fba300f7400 2024-02-23T00:57:29.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.214+0000 7fba37db1700 1 librados: init done 2024-02-23T00:57:29.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.215+0000 7fba37db1700 1 -- 172.21.15.47:0/691208725 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba3005fa60 con 0x7fba300f7400 2024-02-23T00:57:29.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.216+0000 7fba26ffd700 1 -- 172.21.15.47:0/691208725 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fba2001e9e0 con 0x7fba300f7400 2024-02-23T00:57:29.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.216+0000 7fba26ffd700 1 --2- 172.21.15.47:0/691208725 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fba1c062cc0 0x7fba1c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:29.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.216+0000 7fba26ffd700 1 -- 172.21.15.47:0/691208725 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(850..850 src has 670..850) v4 ==== 8663+0+0 (secure 0 0 0) 0x7fba20085d80 con 0x7fba300f7400 2024-02-23T00:57:29.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.216+0000 7fba26ffd700 10 client.44266.objecter ms_dispatch 0x7fba30001550 osd_map(850..850 src has 670..850) v4 2024-02-23T00:57:29.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.216+0000 7fba26ffd700 3 client.44266.objecter handle_osd_map got epochs [850,850] > 0 2024-02-23T00:57:29.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.216+0000 7fba26ffd700 3 client.44266.objecter handle_osd_map decoding full epoch 850 2024-02-23T00:57:29.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.216+0000 7fba3534c700 1 --2- 172.21.15.47:0/691208725 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fba1c062cc0 0x7fba1c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:29.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.216+0000 7fba26ffd700 20 client.44266.objecter dump_active .. 0 homeless 2024-02-23T00:57:29.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.216+0000 7fba26ffd700 10 client.44266.objecter ms_handle_connect 0x7fba1c062cc0 2024-02-23T00:57:29.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.216+0000 7fba3534c700 1 --2- 172.21.15.47:0/691208725 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fba1c062cc0 0x7fba1c065180 secure :-1 s=READY pgs=3238 cs=0 l=1 rev1=1 rx=0x7fba30108690 tx=0x7fba2c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:29.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.220+0000 7fba26ffd700 1 -- 172.21.15.47:0/691208725 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fba200555c0 con 0x7fba300f7400 2024-02-23T00:57:29.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:29.404+0000 7fba37db1700 1 -- 172.21.15.47:0/691208725 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd reweight", "id": 6, "weight": 1.0} v 0) v1 -- 0x7fba301083d0 con 0x7fba300f7400 2024-02-23T00:57:29.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:27.946079+0000 osd.0 (osd.0) 338 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:57:29.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:27.946262+0000 osd.0 (osd.0) 339 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:57:29.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:28.057676+0000 osd.0 (osd.0) 340 : cluster [DBG] 5.29 deep-scrub starts 2024-02-23T00:57:29.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:28.058556+0000 osd.0 (osd.0) 341 : cluster [DBG] 5.29 deep-scrub ok 2024-02-23T00:57:29.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:28.362388+0000 mon.a (mon.0) 3925 : cluster [DBG] osdmap e850: 8 total, 8 up, 8 in 2024-02-23T00:57:29.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:27.946079+0000 osd.0 (osd.0) 338 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:57:29.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:27.946262+0000 osd.0 (osd.0) 339 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:57:29.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:28.057676+0000 osd.0 (osd.0) 340 : cluster [DBG] 5.29 deep-scrub starts 2024-02-23T00:57:29.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:28.058556+0000 osd.0 (osd.0) 341 : cluster [DBG] 5.29 deep-scrub ok 2024-02-23T00:57:29.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:28.362388+0000 mon.a (mon.0) 3925 : cluster [DBG] osdmap e850: 8 total, 8 up, 8 in 2024-02-23T00:57:29.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:27.946079+0000 osd.0 (osd.0) 338 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:57:29.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:27.946262+0000 osd.0 (osd.0) 339 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:57:29.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:28.057676+0000 osd.0 (osd.0) 340 : cluster [DBG] 5.29 deep-scrub starts 2024-02-23T00:57:29.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:28.058556+0000 osd.0 (osd.0) 341 : cluster [DBG] 5.29 deep-scrub ok 2024-02-23T00:57:29.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:28.362388+0000 mon.a (mon.0) 3925 : cluster [DBG] osdmap e850: 8 total, 8 up, 8 in 2024-02-23T00:57:29.855 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph daemon osd.7 dump_historic_ops 2024-02-23T00:57:30.122 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.b/config 2024-02-23T00:57:30.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:30.370+0000 7fba26ffd700 1 -- 172.21.15.47:0/691208725 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 6, "weight": 1.0}]=0 reweighted osd.6 to 1 (10000) v851) v1 ==== 113+0+0 (secure 0 0 0) 0x7fba20058be0 con 0x7fba300f7400 2024-02-23T00:57:30.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:30.373+0000 7fba37db1700 10 librados: watch_flush enter 2024-02-23T00:57:30.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:30.374+0000 7fba37db1700 10 librados: watch_flush exit 2024-02-23T00:57:30.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:30.374+0000 7fba37db1700 20 client.44266.objecter shutdown clearing up homeless session... 2024-02-23T00:57:30.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:30.374+0000 7fba37db1700 10 client.44266.objecter successfully canceled tick 2024-02-23T00:57:30.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:30.374+0000 7fba37db1700 1 -- 172.21.15.47:0/691208725 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fba1c062cc0 msgr2=0x7fba1c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:30.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:30.374+0000 7fba37db1700 1 --2- 172.21.15.47:0/691208725 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fba1c062cc0 0x7fba1c065180 secure :-1 s=READY pgs=3238 cs=0 l=1 rev1=1 rx=0x7fba30108690 tx=0x7fba2c00b040).stop 2024-02-23T00:57:30.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:30.374+0000 7fba37db1700 1 -- 172.21.15.47:0/691208725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba300f7400 msgr2=0x7fba3010cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:30.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:30.374+0000 7fba37db1700 1 --2- 172.21.15.47:0/691208725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba300f7400 0x7fba3010cd50 secure :-1 s=READY pgs=3986 cs=0 l=1 rev1=1 rx=0x7fba200048f0 tx=0x7fba20009cb0).stop 2024-02-23T00:57:30.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:30.374+0000 7fba37db1700 1 -- 172.21.15.47:0/691208725 shutdown_connections 2024-02-23T00:57:30.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:30.374+0000 7fba37db1700 1 --2- 172.21.15.47:0/691208725 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fba1c062cc0 0x7fba1c065180 unknown :-1 s=CLOSED pgs=3238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:30.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:30.374+0000 7fba37db1700 1 --2- 172.21.15.47:0/691208725 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fba300fd510 0x7fba30106ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:30.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:30.374+0000 7fba37db1700 1 --2- 172.21.15.47:0/691208725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba300f7400 0x7fba3010cd50 unknown :-1 s=CLOSED pgs=3986 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:30.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:30.374+0000 7fba37db1700 1 --2- 172.21.15.47:0/691208725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba300fcbb0 0x7fba3010d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:30.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:30.374+0000 7fba37db1700 1 -- 172.21.15.47:0/691208725 >> 172.21.15.47:0/691208725 conn(0x7fba300f2820 msgr2=0x7fba300f4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:30.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:30.374+0000 7fba37db1700 1 -- 172.21.15.47:0/691208725 shutdown_connections 2024-02-23T00:57:30.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:30.374+0000 7fba37db1700 1 -- 172.21.15.47:0/691208725 wait complete. 2024-02-23T00:57:30.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:30.374+0000 7fba37db1700 1 librados: shutdown 2024-02-23T00:57:30.379 INFO:teuthology.orchestra.run.smithi047.stderr:reweighted osd.6 to 1 (10000) 2024-02-23T00:57:30.589 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:29.405933+0000 mon.b (mon.1) 383 : audit [INF] from='client.? 172.21.15.47:0/691208725' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-23T00:57:30.589 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:29.406416+0000 mon.a (mon.0) 3926 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-23T00:57:30.589 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:29.430409+0000 mgr.y (mgr.14517) 733 : cluster [DBG] pgmap v1314: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:57:30.746 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:57:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:57:30] "GET /metrics HTTP/1.1" 200 187134 "" "Prometheus/2.33.4" 2024-02-23T00:57:30.747 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:29.405933+0000 mon.b (mon.1) 383 : audit [INF] from='client.? 172.21.15.47:0/691208725' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-23T00:57:30.747 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:29.406416+0000 mon.a (mon.0) 3926 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-23T00:57:30.747 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:29.430409+0000 mgr.y (mgr.14517) 733 : cluster [DBG] pgmap v1314: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:57:30.747 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:29.405933+0000 mon.b (mon.1) 383 : audit [INF] from='client.? 172.21.15.47:0/691208725' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-23T00:57:30.747 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:29.406416+0000 mon.a (mon.0) 3926 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-23T00:57:30.747 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:29.430409+0000 mgr.y (mgr.14517) 733 : cluster [DBG] pgmap v1314: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:57:31.162 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd primary-affinity 6 1 2024-02-23T00:57:31.448 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:57:31.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:30.368878+0000 mon.a (mon.0) 3927 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-02-23T00:57:31.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:30.368972+0000 mon.a (mon.0) 3928 : cluster [DBG] osdmap e851: 8 total, 8 up, 8 in 2024-02-23T00:57:31.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:31.170949+0000 osd.5 (osd.5) 406 : cluster [DBG] 4.4 scrub starts 2024-02-23T00:57:31.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:31.174407+0000 osd.5 (osd.5) 407 : cluster [DBG] 4.4 scrub ok 2024-02-23T00:57:31.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:30.368878+0000 mon.a (mon.0) 3927 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-02-23T00:57:31.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:30.368972+0000 mon.a (mon.0) 3928 : cluster [DBG] osdmap e851: 8 total, 8 up, 8 in 2024-02-23T00:57:31.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:31.170949+0000 osd.5 (osd.5) 406 : cluster [DBG] 4.4 scrub starts 2024-02-23T00:57:31.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:31.174407+0000 osd.5 (osd.5) 407 : cluster [DBG] 4.4 scrub ok 2024-02-23T00:57:31.633 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:30.368878+0000 mon.a (mon.0) 3927 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-02-23T00:57:31.633 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:30.368972+0000 mon.a (mon.0) 3928 : cluster [DBG] osdmap e851: 8 total, 8 up, 8 in 2024-02-23T00:57:31.633 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:31.170949+0000 osd.5 (osd.5) 406 : cluster [DBG] 4.4 scrub starts 2024-02-23T00:57:31.633 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:31.174407+0000 osd.5 (osd.5) 407 : cluster [DBG] 4.4 scrub ok 2024-02-23T00:57:31.881 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-02-23T00:57:31.881 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-23T00:57:32.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:31.241262+0000 osd.7 (osd.7) 417 : cluster [DBG] 2.1d scrub starts 2024-02-23T00:57:32.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:31.244146+0000 osd.7 (osd.7) 418 : cluster [DBG] 2.1d scrub ok 2024-02-23T00:57:32.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:31.430766+0000 mgr.y (mgr.14517) 734 : cluster [DBG] pgmap v1316: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:57:32.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:31.929157+0000 osd.1 (osd.1) 334 : cluster [DBG] 2.a scrub starts 2024-02-23T00:57:32.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:31.932037+0000 osd.1 (osd.1) 335 : cluster [DBG] 2.a scrub ok 2024-02-23T00:57:32.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:31.241262+0000 osd.7 (osd.7) 417 : cluster [DBG] 2.1d scrub starts 2024-02-23T00:57:32.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:31.244146+0000 osd.7 (osd.7) 418 : cluster [DBG] 2.1d scrub ok 2024-02-23T00:57:32.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:31.430766+0000 mgr.y (mgr.14517) 734 : cluster [DBG] pgmap v1316: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:57:32.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:31.929157+0000 osd.1 (osd.1) 334 : cluster [DBG] 2.a scrub starts 2024-02-23T00:57:32.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:31.932037+0000 osd.1 (osd.1) 335 : cluster [DBG] 2.a scrub ok 2024-02-23T00:57:32.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:31.241262+0000 osd.7 (osd.7) 417 : cluster [DBG] 2.1d scrub starts 2024-02-23T00:57:32.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:31.244146+0000 osd.7 (osd.7) 418 : cluster [DBG] 2.1d scrub ok 2024-02-23T00:57:32.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:31.430766+0000 mgr.y (mgr.14517) 734 : cluster [DBG] pgmap v1316: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:57:32.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:31.929157+0000 osd.1 (osd.1) 334 : cluster [DBG] 2.a scrub starts 2024-02-23T00:57:32.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:31.932037+0000 osd.1 (osd.1) 335 : cluster [DBG] 2.a scrub ok 2024-02-23T00:57:32.944 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:57:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:57:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:57:33.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.170+0000 7feb7ba71700 1 -- 172.21.15.47:0/942008548 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb740f67a0 msgr2=0x7feb74104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:33.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.170+0000 7feb7ba71700 1 --2- 172.21.15.47:0/942008548 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb740f67a0 0x7feb74104a50 secure :-1 s=READY pgs=2853 cs=0 l=1 rev1=1 rx=0x7feb6c009be0 tx=0x7feb6c004d10).stop 2024-02-23T00:57:33.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.170+0000 7feb7ba71700 1 -- 172.21.15.47:0/942008548 shutdown_connections 2024-02-23T00:57:33.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.170+0000 7feb7ba71700 1 --2- 172.21.15.47:0/942008548 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb740f67a0 0x7feb74104a50 unknown :-1 s=CLOSED pgs=2853 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:33.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.170+0000 7feb7ba71700 1 --2- 172.21.15.47:0/942008548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb740fdbe0 0x7feb740fdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:33.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.170+0000 7feb7ba71700 1 --2- 172.21.15.47:0/942008548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb740fe540 0x7feb740f60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-23T00:57:33.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.170+0000 7feb7ba71700 1 -- 172.21.15.47:0/942008548 >> 172.21.15.47:0/942008548 conn(0x7feb7406ffc0 msgr2=0x7feb740f43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:33.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.170+0000 7feb7ba71700 1 -- 172.21.15.47:0/942008548 shutdown_connections 2024-02-23T00:57:33.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.170+0000 7feb7ba71700 1 -- 172.21.15.47:0/942008548 wait complete. 2024-02-23T00:57:33.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.170+0000 7feb7ba71700 1 librados: starting msgr at 2024-02-23T00:57:33.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.170+0000 7feb7ba71700 1 librados: starting objecter 2024-02-23T00:57:33.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.171+0000 7feb7ba71700 1 Processor -- start 2024-02-23T00:57:33.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.171+0000 7feb7ba71700 1 -- start start 2024-02-23T00:57:33.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.171+0000 7feb7ba71700 1 librados: setting wanted keys 2024-02-23T00:57:33.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.171+0000 7feb7ba71700 1 librados: calling monclient init 2024-02-23T00:57:33.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.171+0000 7feb7ba71700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb740f67a0 0x7feb7410de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:33.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.171+0000 7feb7ba71700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb740fdbe0 0x7feb7410e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:33.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.171+0000 7feb7ba71700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb740fe540 0x7feb74107fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:33.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.171+0000 7feb7ba71700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb741084b0 con 0x7feb740fdbe0 2024-02-23T00:57:33.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.171+0000 7feb7ba71700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb74108620 con 0x7feb740fe540 2024-02-23T00:57:33.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.171+0000 7feb7ba71700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb741087e0 con 0x7feb740f67a0 2024-02-23T00:57:33.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.171+0000 7feb7900c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb740fdbe0 0x7feb7410e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:33.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.171+0000 7feb7980d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb740f67a0 0x7feb7410de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:33.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.171+0000 7feb7980d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb740f67a0 0x7feb7410de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:47400/0 (socket says 172.21.15.47:47400) 2024-02-23T00:57:33.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.171+0000 7feb7980d700 1 -- 172.21.15.47:0/3307136386 learned_addr learned my addr 172.21.15.47:0/3307136386 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:57:33.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.171+0000 7feb7900c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb740fdbe0 0x7feb7410e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41200/0 (socket says 172.21.15.47:41200) 2024-02-23T00:57:33.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.171+0000 7feb7a00e700 1 --2- 172.21.15.47:0/3307136386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb740fe540 0x7feb74107fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:33.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.171+0000 7feb7980d700 1 -- 172.21.15.47:0/3307136386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb740fe540 msgr2=0x7feb74107fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:33.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.171+0000 7feb7980d700 1 --2- 172.21.15.47:0/3307136386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb740fe540 0x7feb74107fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:33.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.171+0000 7feb7980d700 1 -- 172.21.15.47:0/3307136386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb740fdbe0 msgr2=0x7feb7410e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:33.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.171+0000 7feb7980d700 1 --2- 172.21.15.47:0/3307136386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb740fdbe0 0x7feb7410e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:33.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.171+0000 7feb7980d700 1 -- 172.21.15.47:0/3307136386 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb64005040 con 0x7feb740f67a0 2024-02-23T00:57:33.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.171+0000 7feb7980d700 1 --2- 172.21.15.47:0/3307136386 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb740f67a0 0x7feb7410de20 secure :-1 s=READY pgs=2854 cs=0 l=1 rev1=1 rx=0x7feb640048f0 tx=0x7feb64009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:33.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.171+0000 7feb6af78700 10 client.?.objecter ms_handle_connect 0x7feb740f67a0 2024-02-23T00:57:33.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.171+0000 7feb6af78700 10 client.?.objecter resend_mon_ops 2024-02-23T00:57:33.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.172+0000 7feb6af78700 1 -- 172.21.15.47:0/3307136386 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb64007b00 con 0x7feb740f67a0 2024-02-23T00:57:33.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.172+0000 7feb6af78700 1 -- 172.21.15.47:0/3307136386 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7feb6400f040 con 0x7feb740f67a0 2024-02-23T00:57:33.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.172+0000 7feb7ba71700 1 -- 172.21.15.47:0/3307136386 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb6c0097c0 con 0x7feb740f67a0 2024-02-23T00:57:33.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.172+0000 7feb6af78700 1 -- 172.21.15.47:0/3307136386 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb64014890 con 0x7feb740f67a0 2024-02-23T00:57:33.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.172+0000 7feb7ba71700 10 client.42335.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:57:33.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.172+0000 7feb7ba71700 1 -- 172.21.15.47:0/3307136386 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb74108e20 con 0x7feb740f67a0 2024-02-23T00:57:33.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.172+0000 7feb7ba71700 1 librados: init done 2024-02-23T00:57:33.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.173+0000 7feb7ba71700 1 -- 172.21.15.47:0/3307136386 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb7405f9b0 con 0x7feb740f67a0 2024-02-23T00:57:33.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.173+0000 7feb6af78700 1 -- 172.21.15.47:0/3307136386 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7feb64014cc0 con 0x7feb740f67a0 2024-02-23T00:57:33.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.173+0000 7feb6af78700 1 --2- 172.21.15.47:0/3307136386 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7feb60062cc0 0x7feb60065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:33.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.174+0000 7feb6af78700 1 -- 172.21.15.47:0/3307136386 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(851..851 src has 670..851) v4 ==== 8663+0+0 (secure 0 0 0) 0x7feb6407d050 con 0x7feb740f67a0 2024-02-23T00:57:33.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.174+0000 7feb6af78700 10 client.42335.objecter ms_dispatch 0x7feb74001550 osd_map(851..851 src has 670..851) v4 2024-02-23T00:57:33.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.174+0000 7feb7900c700 1 --2- 172.21.15.47:0/3307136386 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7feb60062cc0 0x7feb60065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:33.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.174+0000 7feb6af78700 3 client.42335.objecter handle_osd_map got epochs [851,851] > 0 2024-02-23T00:57:33.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.174+0000 7feb6af78700 3 client.42335.objecter handle_osd_map decoding full epoch 851 2024-02-23T00:57:33.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.174+0000 7feb6af78700 20 client.42335.objecter dump_active .. 0 homeless 2024-02-23T00:57:33.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.174+0000 7feb6af78700 10 client.42335.objecter ms_handle_connect 0x7feb60062cc0 2024-02-23T00:57:33.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.174+0000 7feb7900c700 1 --2- 172.21.15.47:0/3307136386 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7feb60062cc0 0x7feb60065180 secure :-1 s=READY pgs=3239 cs=0 l=1 rev1=1 rx=0x7feb74109760 tx=0x7feb70009060).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:33.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.178+0000 7feb6af78700 1 -- 172.21.15.47:0/3307136386 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7feb64049f60 con 0x7feb740f67a0 2024-02-23T00:57:33.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.364+0000 7feb7ba71700 1 -- 172.21.15.47:0/3307136386 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} v 0) v1 -- 0x7feb741094a0 con 0x7feb740f67a0 2024-02-23T00:57:33.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.375+0000 7feb6af78700 1 -- 172.21.15.47:0/3307136386 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]=0 set osd.6 primary-affinity to 1 (8655362) v852) v1 ==== 133+0+0 (secure 0 0 0) 0x7feb6404c890 con 0x7feb740f67a0 2024-02-23T00:57:33.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.377+0000 7feb7ba71700 10 librados: watch_flush enter 2024-02-23T00:57:33.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.377+0000 7feb7ba71700 10 librados: watch_flush exit 2024-02-23T00:57:33.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.377+0000 7feb7ba71700 20 client.42335.objecter shutdown clearing up homeless session... 2024-02-23T00:57:33.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.377+0000 7feb7ba71700 10 client.42335.objecter successfully canceled tick 2024-02-23T00:57:33.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.377+0000 7feb7ba71700 1 -- 172.21.15.47:0/3307136386 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7feb60062cc0 msgr2=0x7feb60065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:33.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.377+0000 7feb7ba71700 1 --2- 172.21.15.47:0/3307136386 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7feb60062cc0 0x7feb60065180 secure :-1 s=READY pgs=3239 cs=0 l=1 rev1=1 rx=0x7feb74109760 tx=0x7feb70009060).stop 2024-02-23T00:57:33.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.377+0000 7feb7ba71700 1 -- 172.21.15.47:0/3307136386 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb740f67a0 msgr2=0x7feb7410de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:33.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.377+0000 7feb7ba71700 1 --2- 172.21.15.47:0/3307136386 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb740f67a0 0x7feb7410de20 secure :-1 s=READY pgs=2854 cs=0 l=1 rev1=1 rx=0x7feb640048f0 tx=0x7feb64009cb0).stop 2024-02-23T00:57:33.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.377+0000 7feb7ba71700 1 -- 172.21.15.47:0/3307136386 shutdown_connections 2024-02-23T00:57:33.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.377+0000 7feb7ba71700 1 --2- 172.21.15.47:0/3307136386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb740fe540 0x7feb74107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:33.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.377+0000 7feb7ba71700 1 --2- 172.21.15.47:0/3307136386 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7feb60062cc0 0x7feb60065180 unknown :-1 s=CLOSED pgs=3239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:33.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.377+0000 7feb7ba71700 1 --2- 172.21.15.47:0/3307136386 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb740f67a0 0x7feb7410de20 unknown :-1 s=CLOSED pgs=2854 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:33.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.377+0000 7feb7ba71700 1 --2- 172.21.15.47:0/3307136386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb740fdbe0 0x7feb7410e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:33.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.377+0000 7feb7ba71700 1 -- 172.21.15.47:0/3307136386 >> 172.21.15.47:0/3307136386 conn(0x7feb7406ffc0 msgr2=0x7feb7406d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:33.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.377+0000 7feb7ba71700 1 -- 172.21.15.47:0/3307136386 shutdown_connections 2024-02-23T00:57:33.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.377+0000 7feb7ba71700 1 -- 172.21.15.47:0/3307136386 wait complete. 2024-02-23T00:57:33.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:33.377+0000 7feb7ba71700 1 librados: shutdown 2024-02-23T00:57:33.382 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:57:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:57:33.382 INFO:teuthology.orchestra.run.smithi047.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-02-23T00:57:33.669 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:31.991887+0000 osd.2 (osd.2) 205 : cluster [DBG] 4.1c scrub starts 2024-02-23T00:57:33.669 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:31.994097+0000 osd.2 (osd.2) 206 : cluster [DBG] 4.1c scrub ok 2024-02-23T00:57:33.669 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:32.860284+0000 osd.3 (osd.3) 341 : cluster [DBG] 3.2 scrub starts 2024-02-23T00:57:33.669 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:32.861172+0000 osd.3 (osd.3) 342 : cluster [DBG] 3.2 scrub ok 2024-02-23T00:57:33.669 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:32.966787+0000 osd.1 (osd.1) 336 : cluster [DBG] 4.7 deep-scrub starts 2024-02-23T00:57:33.669 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:32.971373+0000 osd.1 (osd.1) 337 : cluster [DBG] 4.7 deep-scrub ok 2024-02-23T00:57:33.669 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:33.366371+0000 mon.c (mon.2) 532 : audit [INF] from='client.? 172.21.15.47:0/3307136386' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-02-23T00:57:33.670 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:33.366627+0000 mon.a (mon.0) 3929 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-02-23T00:57:33.670 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:31.991887+0000 osd.2 (osd.2) 205 : cluster [DBG] 4.1c scrub starts 2024-02-23T00:57:33.670 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:31.994097+0000 osd.2 (osd.2) 206 : cluster [DBG] 4.1c scrub ok 2024-02-23T00:57:33.670 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:32.860284+0000 osd.3 (osd.3) 341 : cluster [DBG] 3.2 scrub starts 2024-02-23T00:57:33.670 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:32.861172+0000 osd.3 (osd.3) 342 : cluster [DBG] 3.2 scrub ok 2024-02-23T00:57:33.670 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:32.966787+0000 osd.1 (osd.1) 336 : cluster [DBG] 4.7 deep-scrub starts 2024-02-23T00:57:33.670 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:32.971373+0000 osd.1 (osd.1) 337 : cluster [DBG] 4.7 deep-scrub ok 2024-02-23T00:57:33.670 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:33.366371+0000 mon.c (mon.2) 532 : audit [INF] from='client.? 172.21.15.47:0/3307136386' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-02-23T00:57:33.670 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:33.366627+0000 mon.a (mon.0) 3929 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-02-23T00:57:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:31.991887+0000 osd.2 (osd.2) 205 : cluster [DBG] 4.1c scrub starts 2024-02-23T00:57:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:31.994097+0000 osd.2 (osd.2) 206 : cluster [DBG] 4.1c scrub ok 2024-02-23T00:57:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:32.860284+0000 osd.3 (osd.3) 341 : cluster [DBG] 3.2 scrub starts 2024-02-23T00:57:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:32.861172+0000 osd.3 (osd.3) 342 : cluster [DBG] 3.2 scrub ok 2024-02-23T00:57:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:32.966787+0000 osd.1 (osd.1) 336 : cluster [DBG] 4.7 deep-scrub starts 2024-02-23T00:57:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:32.971373+0000 osd.1 (osd.1) 337 : cluster [DBG] 4.7 deep-scrub ok 2024-02-23T00:57:33.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:33.366371+0000 mon.c (mon.2) 532 : audit [INF] from='client.? 172.21.15.47:0/3307136386' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-02-23T00:57:33.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:33.366627+0000 mon.a (mon.0) 3929 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-02-23T00:57:34.058 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd reweight 7 1 2024-02-23T00:57:34.344 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:57:34.648 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:33.069935+0000 mgr.y (mgr.14517) 735 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:57:34.648 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:33.180634+0000 osd.0 (osd.0) 342 : cluster [DBG] 4.6 scrub starts 2024-02-23T00:57:34.648 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:33.182111+0000 osd.0 (osd.0) 343 : cluster [DBG] 4.6 scrub ok 2024-02-23T00:57:34.648 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:33.286061+0000 osd.7 (osd.7) 419 : cluster [DBG] 18.6 scrub starts 2024-02-23T00:57:34.649 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:33.286985+0000 osd.7 (osd.7) 420 : cluster [DBG] 18.6 scrub ok 2024-02-23T00:57:34.649 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:33.375928+0000 mon.a (mon.0) 3930 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-02-23T00:57:34.649 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:33.376022+0000 mon.a (mon.0) 3931 : cluster [DBG] osdmap e852: 8 total, 8 up, 8 in 2024-02-23T00:57:34.649 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:33.431315+0000 mgr.y (mgr.14517) 736 : cluster [DBG] pgmap v1318: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:57:34.649 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:33.069935+0000 mgr.y (mgr.14517) 735 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:57:34.649 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:33.180634+0000 osd.0 (osd.0) 342 : cluster [DBG] 4.6 scrub starts 2024-02-23T00:57:34.649 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:33.182111+0000 osd.0 (osd.0) 343 : cluster [DBG] 4.6 scrub ok 2024-02-23T00:57:34.649 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:33.286061+0000 osd.7 (osd.7) 419 : cluster [DBG] 18.6 scrub starts 2024-02-23T00:57:34.649 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:33.286985+0000 osd.7 (osd.7) 420 : cluster [DBG] 18.6 scrub ok 2024-02-23T00:57:34.649 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:33.375928+0000 mon.a (mon.0) 3930 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-02-23T00:57:34.649 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:33.376022+0000 mon.a (mon.0) 3931 : cluster [DBG] osdmap e852: 8 total, 8 up, 8 in 2024-02-23T00:57:34.650 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:33.431315+0000 mgr.y (mgr.14517) 736 : cluster [DBG] pgmap v1318: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:57:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:33.069935+0000 mgr.y (mgr.14517) 735 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:57:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:33.180634+0000 osd.0 (osd.0) 342 : cluster [DBG] 4.6 scrub starts 2024-02-23T00:57:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:33.182111+0000 osd.0 (osd.0) 343 : cluster [DBG] 4.6 scrub ok 2024-02-23T00:57:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:33.286061+0000 osd.7 (osd.7) 419 : cluster [DBG] 18.6 scrub starts 2024-02-23T00:57:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:33.286985+0000 osd.7 (osd.7) 420 : cluster [DBG] 18.6 scrub ok 2024-02-23T00:57:34.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:33.375928+0000 mon.a (mon.0) 3930 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-02-23T00:57:34.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:33.376022+0000 mon.a (mon.0) 3931 : cluster [DBG] osdmap e852: 8 total, 8 up, 8 in 2024-02-23T00:57:34.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:33.431315+0000 mgr.y (mgr.14517) 736 : cluster [DBG] pgmap v1318: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:57:35.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:34.177403+0000 osd.0 (osd.0) 344 : cluster [DBG] 4.1e scrub starts 2024-02-23T00:57:35.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:34.179727+0000 osd.0 (osd.0) 345 : cluster [DBG] 4.1e scrub ok 2024-02-23T00:57:35.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:34.177403+0000 osd.0 (osd.0) 344 : cluster [DBG] 4.1e scrub starts 2024-02-23T00:57:35.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:34.179727+0000 osd.0 (osd.0) 345 : cluster [DBG] 4.1e scrub ok 2024-02-23T00:57:35.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:34.177403+0000 osd.0 (osd.0) 344 : cluster [DBG] 4.1e scrub starts 2024-02-23T00:57:35.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:34.179727+0000 osd.0 (osd.0) 345 : cluster [DBG] 4.1e scrub ok 2024-02-23T00:57:36.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.139+0000 7f6e38959700 1 -- 172.21.15.47:0/478201048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e340fe510 msgr2=0x7f6e340f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:36.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.139+0000 7f6e38959700 1 --2- 172.21.15.47:0/478201048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e340fe510 0x7f6e340f7d40 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 rx=0x7f6e280047b0 tx=0x7f6e28009cb0).stop 2024-02-23T00:57:36.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.139+0000 7f6e38959700 1 -- 172.21.15.47:0/478201048 shutdown_connections 2024-02-23T00:57:36.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.139+0000 7f6e38959700 1 --2- 172.21.15.47:0/478201048 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6e340f8420 0x7f6e34104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:36.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.139+0000 7f6e38959700 1 --2- 172.21.15.47:0/478201048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e340fdbb0 0x7f6e340fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:36.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.139+0000 7f6e38959700 1 --2- 172.21.15.47:0/478201048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e340fe510 0x7f6e340f7d40 unknown :-1 s=CLOSED pgs=2674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:36.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.139+0000 7f6e38959700 1 -- 172.21.15.47:0/478201048 >> 172.21.15.47:0/478201048 conn(0x7f6e340f3800 msgr2=0x7f6e340f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:36.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.139+0000 7f6e38959700 1 -- 172.21.15.47:0/478201048 shutdown_connections 2024-02-23T00:57:36.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.139+0000 7f6e38959700 1 -- 172.21.15.47:0/478201048 wait complete. 2024-02-23T00:57:36.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.139+0000 7f6e38959700 1 librados: starting msgr at 2024-02-23T00:57:36.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.139+0000 7f6e38959700 1 librados: starting objecter 2024-02-23T00:57:36.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.140+0000 7f6e38959700 1 Processor -- start 2024-02-23T00:57:36.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.140+0000 7f6e38959700 1 -- start start 2024-02-23T00:57:36.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.140+0000 7f6e38959700 1 librados: setting wanted keys 2024-02-23T00:57:36.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.140+0000 7f6e38959700 1 librados: calling monclient init 2024-02-23T00:57:36.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.140+0000 7f6e38959700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e340f8420 0x7f6e3410de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:36.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.140+0000 7f6e38959700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6e340fdbb0 0x7f6e3410e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:36.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.140+0000 7f6e38959700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e340fe510 0x7f6e34108070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:36.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.140+0000 7f6e38959700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e34108560 con 0x7f6e340fe510 2024-02-23T00:57:36.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.140+0000 7f6e38959700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e341086d0 con 0x7f6e340f8420 2024-02-23T00:57:36.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.140+0000 7f6e38959700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e34108890 con 0x7f6e340fdbb0 2024-02-23T00:57:36.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.140+0000 7f6e31d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6e340fdbb0 0x7f6e3410e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:36.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.140+0000 7f6e31d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6e340fdbb0 0x7f6e3410e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:47422/0 (socket says 172.21.15.47:47422) 2024-02-23T00:57:36.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.140+0000 7f6e31d9b700 1 -- 172.21.15.47:0/3037423672 learned_addr learned my addr 172.21.15.47:0/3037423672 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:57:36.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.140+0000 7f6e32d9d700 1 --2- 172.21.15.47:0/3037423672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e340fe510 0x7f6e34108070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:36.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.141+0000 7f6e3259c700 1 --2- 172.21.15.47:0/3037423672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e340f8420 0x7f6e3410de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:36.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.141+0000 7f6e31d9b700 1 -- 172.21.15.47:0/3037423672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e340f8420 msgr2=0x7f6e3410de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:36.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.141+0000 7f6e31d9b700 1 --2- 172.21.15.47:0/3037423672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e340f8420 0x7f6e3410de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:36.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.141+0000 7f6e31d9b700 1 -- 172.21.15.47:0/3037423672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e340fe510 msgr2=0x7f6e34108070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:36.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.141+0000 7f6e31d9b700 1 --2- 172.21.15.47:0/3037423672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e340fe510 0x7f6e34108070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:36.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.141+0000 7f6e31d9b700 1 -- 172.21.15.47:0/3037423672 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e2c0097c0 con 0x7f6e340fdbb0 2024-02-23T00:57:36.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.141+0000 7f6e31d9b700 1 --2- 172.21.15.47:0/3037423672 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6e340fdbb0 0x7f6e3410e350 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 rx=0x7f6e280044e0 tx=0x7f6e28009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:36.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.141+0000 7f6e237fe700 10 client.?.objecter ms_handle_connect 0x7f6e340fdbb0 2024-02-23T00:57:36.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.141+0000 7f6e237fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:57:36.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.141+0000 7f6e237fe700 1 -- 172.21.15.47:0/3037423672 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e28018070 con 0x7f6e340fdbb0 2024-02-23T00:57:36.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.141+0000 7f6e237fe700 1 -- 172.21.15.47:0/3037423672 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f6e280096d0 con 0x7f6e340fdbb0 2024-02-23T00:57:36.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.141+0000 7f6e38959700 1 -- 172.21.15.47:0/3037423672 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e28005040 con 0x7f6e340fdbb0 2024-02-23T00:57:36.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.141+0000 7f6e237fe700 1 -- 172.21.15.47:0/3037423672 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2801dc10 con 0x7f6e340fdbb0 2024-02-23T00:57:36.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.141+0000 7f6e38959700 10 client.42341.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:57:36.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.141+0000 7f6e38959700 1 -- 172.21.15.47:0/3037423672 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e34108db0 con 0x7f6e340fdbb0 2024-02-23T00:57:36.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.141+0000 7f6e38959700 1 librados: init done 2024-02-23T00:57:36.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.142+0000 7f6e38959700 1 -- 172.21.15.47:0/3037423672 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e3405ee80 con 0x7f6e340fdbb0 2024-02-23T00:57:36.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.143+0000 7f6e237fe700 1 -- 172.21.15.47:0/3037423672 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f6e28009240 con 0x7f6e340fdbb0 2024-02-23T00:57:36.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.143+0000 7f6e237fe700 1 --2- 172.21.15.47:0/3037423672 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6e1c062cc0 0x7f6e1c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:36.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.143+0000 7f6e237fe700 1 -- 172.21.15.47:0/3037423672 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(852..852 src has 670..852) v4 ==== 8663+0+0 (secure 0 0 0) 0x7f6e2807e970 con 0x7f6e340fdbb0 2024-02-23T00:57:36.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.143+0000 7f6e237fe700 10 client.42341.objecter ms_dispatch 0x7f6e34001550 osd_map(852..852 src has 670..852) v4 2024-02-23T00:57:36.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.143+0000 7f6e237fe700 3 client.42341.objecter handle_osd_map got epochs [852,852] > 0 2024-02-23T00:57:36.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.143+0000 7f6e237fe700 3 client.42341.objecter handle_osd_map decoding full epoch 852 2024-02-23T00:57:36.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.143+0000 7f6e3259c700 1 --2- 172.21.15.47:0/3037423672 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6e1c062cc0 0x7f6e1c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:36.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.143+0000 7f6e237fe700 20 client.42341.objecter dump_active .. 0 homeless 2024-02-23T00:57:36.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.143+0000 7f6e237fe700 10 client.42341.objecter ms_handle_connect 0x7f6e1c062cc0 2024-02-23T00:57:36.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.143+0000 7f6e3259c700 1 --2- 172.21.15.47:0/3037423672 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6e1c062cc0 0x7f6e1c065180 secure :-1 s=READY pgs=3240 cs=0 l=1 rev1=1 rx=0x7f6e24009170 tx=0x7f6e24009cd0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:36.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.147+0000 7f6e237fe700 1 -- 172.21.15.47:0/3037423672 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f6e2804e1b0 con 0x7f6e340fdbb0 2024-02-23T00:57:36.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.330+0000 7f6e38959700 1 -- 172.21.15.47:0/3037423672 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd reweight", "id": 7, "weight": 1.0} v 0) v1 -- 0x7f6e34106f00 con 0x7f6e340fdbb0 2024-02-23T00:57:36.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.412+0000 7f6e237fe700 1 -- 172.21.15.47:0/3037423672 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 7, "weight": 1.0}]=0 reweighted osd.7 to 1 (10000) v853) v1 ==== 113+0+0 (secure 0 0 0) 0x7f6e280517d0 con 0x7f6e340fdbb0 2024-02-23T00:57:36.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.415+0000 7f6e38959700 10 librados: watch_flush enter 2024-02-23T00:57:36.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.415+0000 7f6e38959700 10 librados: watch_flush exit 2024-02-23T00:57:36.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.415+0000 7f6e38959700 20 client.42341.objecter shutdown clearing up homeless session... 2024-02-23T00:57:36.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.415+0000 7f6e38959700 10 client.42341.objecter successfully canceled tick 2024-02-23T00:57:36.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.415+0000 7f6e38959700 1 -- 172.21.15.47:0/3037423672 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6e1c062cc0 msgr2=0x7f6e1c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:36.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.415+0000 7f6e38959700 1 --2- 172.21.15.47:0/3037423672 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6e1c062cc0 0x7f6e1c065180 secure :-1 s=READY pgs=3240 cs=0 l=1 rev1=1 rx=0x7f6e24009170 tx=0x7f6e24009cd0).stop 2024-02-23T00:57:36.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.415+0000 7f6e38959700 1 -- 172.21.15.47:0/3037423672 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6e340fdbb0 msgr2=0x7f6e3410e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:36.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.415+0000 7f6e38959700 1 --2- 172.21.15.47:0/3037423672 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6e340fdbb0 0x7f6e3410e350 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 rx=0x7f6e280044e0 tx=0x7f6e28009cb0).stop 2024-02-23T00:57:36.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.415+0000 7f6e38959700 1 -- 172.21.15.47:0/3037423672 shutdown_connections 2024-02-23T00:57:36.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.415+0000 7f6e38959700 1 --2- 172.21.15.47:0/3037423672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e340fe510 0x7f6e34108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:36.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.415+0000 7f6e38959700 1 --2- 172.21.15.47:0/3037423672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e340f8420 0x7f6e3410de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:36.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.415+0000 7f6e38959700 1 --2- 172.21.15.47:0/3037423672 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f6e1c062cc0 0x7f6e1c065180 unknown :-1 s=CLOSED pgs=3240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:36.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.415+0000 7f6e38959700 1 --2- 172.21.15.47:0/3037423672 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6e340fdbb0 0x7f6e3410e350 unknown :-1 s=CLOSED pgs=2855 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:36.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.415+0000 7f6e38959700 1 -- 172.21.15.47:0/3037423672 >> 172.21.15.47:0/3037423672 conn(0x7f6e340f3800 msgr2=0x7f6e340fb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:36.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.415+0000 7f6e38959700 1 -- 172.21.15.47:0/3037423672 shutdown_connections 2024-02-23T00:57:36.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.415+0000 7f6e38959700 1 -- 172.21.15.47:0/3037423672 wait complete. 2024-02-23T00:57:36.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:36.415+0000 7f6e38959700 1 librados: shutdown 2024-02-23T00:57:36.420 INFO:teuthology.orchestra.run.smithi047.stderr:reweighted osd.7 to 1 (10000) 2024-02-23T00:57:36.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:35.431868+0000 mgr.y (mgr.14517) 737 : cluster [DBG] pgmap v1319: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 723 B/s rd, 0 op/s 2024-02-23T00:57:36.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:35.923757+0000 osd.1 (osd.1) 338 : cluster [DBG] 5.1d scrub starts 2024-02-23T00:57:36.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:35.924997+0000 osd.1 (osd.1) 339 : cluster [DBG] 5.1d scrub ok 2024-02-23T00:57:36.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:36.332035+0000 mon.c (mon.2) 533 : audit [INF] from='client.? 172.21.15.47:0/3037423672' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-23T00:57:36.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:36.332319+0000 mon.a (mon.0) 3932 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-23T00:57:36.759 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:35.431868+0000 mgr.y (mgr.14517) 737 : cluster [DBG] pgmap v1319: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 723 B/s rd, 0 op/s 2024-02-23T00:57:36.759 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:35.923757+0000 osd.1 (osd.1) 338 : cluster [DBG] 5.1d scrub starts 2024-02-23T00:57:36.759 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:35.924997+0000 osd.1 (osd.1) 339 : cluster [DBG] 5.1d scrub ok 2024-02-23T00:57:36.759 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:36.332035+0000 mon.c (mon.2) 533 : audit [INF] from='client.? 172.21.15.47:0/3037423672' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-23T00:57:36.759 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:36.332319+0000 mon.a (mon.0) 3932 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-23T00:57:36.759 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:35.431868+0000 mgr.y (mgr.14517) 737 : cluster [DBG] pgmap v1319: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 723 B/s rd, 0 op/s 2024-02-23T00:57:36.759 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:35.923757+0000 osd.1 (osd.1) 338 : cluster [DBG] 5.1d scrub starts 2024-02-23T00:57:36.760 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:35.924997+0000 osd.1 (osd.1) 339 : cluster [DBG] 5.1d scrub ok 2024-02-23T00:57:36.760 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:36.332035+0000 mon.c (mon.2) 533 : audit [INF] from='client.? 172.21.15.47:0/3037423672' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-23T00:57:36.760 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:36.332319+0000 mon.a (mon.0) 3932 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-23T00:57:37.153 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd primary-affinity 7 1 2024-02-23T00:57:37.436 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:57:37.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:36.016090+0000 osd.2 (osd.2) 207 : cluster [DBG] 2.c scrub starts 2024-02-23T00:57:37.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:36.017112+0000 osd.2 (osd.2) 208 : cluster [DBG] 2.c scrub ok 2024-02-23T00:57:37.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:36.413137+0000 mon.a (mon.0) 3933 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-02-23T00:57:37.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:36.413214+0000 mon.a (mon.0) 3934 : cluster [DBG] osdmap e853: 8 total, 8 up, 8 in 2024-02-23T00:57:37.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:36.816329+0000 osd.3 (osd.3) 343 : cluster [DBG] 4.11 scrub starts 2024-02-23T00:57:37.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:36.818134+0000 osd.3 (osd.3) 344 : cluster [DBG] 4.11 scrub ok 2024-02-23T00:57:37.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:36.895005+0000 osd.1 (osd.1) 340 : cluster [DBG] 4.2 deep-scrub starts 2024-02-23T00:57:37.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:36.902200+0000 osd.1 (osd.1) 341 : cluster [DBG] 4.2 deep-scrub ok 2024-02-23T00:57:37.697 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:36.016090+0000 osd.2 (osd.2) 207 : cluster [DBG] 2.c scrub starts 2024-02-23T00:57:37.697 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:36.017112+0000 osd.2 (osd.2) 208 : cluster [DBG] 2.c scrub ok 2024-02-23T00:57:37.697 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:36.413137+0000 mon.a (mon.0) 3933 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-02-23T00:57:37.697 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:36.413214+0000 mon.a (mon.0) 3934 : cluster [DBG] osdmap e853: 8 total, 8 up, 8 in 2024-02-23T00:57:37.697 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:36.816329+0000 osd.3 (osd.3) 343 : cluster [DBG] 4.11 scrub starts 2024-02-23T00:57:37.697 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:36.818134+0000 osd.3 (osd.3) 344 : cluster [DBG] 4.11 scrub ok 2024-02-23T00:57:37.697 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:36.895005+0000 osd.1 (osd.1) 340 : cluster [DBG] 4.2 deep-scrub starts 2024-02-23T00:57:37.697 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:36.902200+0000 osd.1 (osd.1) 341 : cluster [DBG] 4.2 deep-scrub ok 2024-02-23T00:57:37.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:36.016090+0000 osd.2 (osd.2) 207 : cluster [DBG] 2.c scrub starts 2024-02-23T00:57:37.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:36.017112+0000 osd.2 (osd.2) 208 : cluster [DBG] 2.c scrub ok 2024-02-23T00:57:37.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:36.413137+0000 mon.a (mon.0) 3933 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-02-23T00:57:37.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:36.413214+0000 mon.a (mon.0) 3934 : cluster [DBG] osdmap e853: 8 total, 8 up, 8 in 2024-02-23T00:57:37.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:36.816329+0000 osd.3 (osd.3) 343 : cluster [DBG] 4.11 scrub starts 2024-02-23T00:57:37.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:36.818134+0000 osd.3 (osd.3) 344 : cluster [DBG] 4.11 scrub ok 2024-02-23T00:57:37.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:36.895005+0000 osd.1 (osd.1) 340 : cluster [DBG] 4.2 deep-scrub starts 2024-02-23T00:57:37.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:36.902200+0000 osd.1 (osd.1) 341 : cluster [DBG] 4.2 deep-scrub ok 2024-02-23T00:57:38.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:37.320982+0000 osd.4 (osd.4) 387 : cluster [DBG] 4.d deep-scrub starts 2024-02-23T00:57:38.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:37.326869+0000 osd.4 (osd.4) 388 : cluster [DBG] 4.d deep-scrub ok 2024-02-23T00:57:38.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:37.432543+0000 mgr.y (mgr.14517) 738 : cluster [DBG] pgmap v1321: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-23T00:57:38.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:37.320982+0000 osd.4 (osd.4) 387 : cluster [DBG] 4.d deep-scrub starts 2024-02-23T00:57:38.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:37.326869+0000 osd.4 (osd.4) 388 : cluster [DBG] 4.d deep-scrub ok 2024-02-23T00:57:38.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:37.432543+0000 mgr.y (mgr.14517) 738 : cluster [DBG] pgmap v1321: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-23T00:57:38.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:37.320982+0000 osd.4 (osd.4) 387 : cluster [DBG] 4.d deep-scrub starts 2024-02-23T00:57:38.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:37.326869+0000 osd.4 (osd.4) 388 : cluster [DBG] 4.d deep-scrub ok 2024-02-23T00:57:38.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:37.432543+0000 mgr.y (mgr.14517) 738 : cluster [DBG] pgmap v1321: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-23T00:57:39.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.151+0000 7f1e84dbd700 1 -- 172.21.15.47:0/4242381508 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e80102210 msgr2=0x7f1e80104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:39.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.151+0000 7f1e84dbd700 1 --2- 172.21.15.47:0/4242381508 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e80102210 0x7f1e80104600 secure :-1 s=READY pgs=2856 cs=0 l=1 rev1=1 rx=0x7f1e78009be0 tx=0x7f1e78004d10).stop 2024-02-23T00:57:39.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.151+0000 7f1e84dbd700 1 -- 172.21.15.47:0/4242381508 shutdown_connections 2024-02-23T00:57:39.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.151+0000 7f1e84dbd700 1 --2- 172.21.15.47:0/4242381508 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e80102210 0x7f1e80104600 unknown :-1 s=CLOSED pgs=2856 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:39.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.151+0000 7f1e84dbd700 1 --2- 172.21.15.47:0/4242381508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e800f8cb0 0x7f1e800f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:39.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.152+0000 7f1e84dbd700 1 --2- 172.21.15.47:0/4242381508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e800f9610 0x7f1e80101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:39.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.152+0000 7f1e84dbd700 1 -- 172.21.15.47:0/4242381508 >> 172.21.15.47:0/4242381508 conn(0x7f1e800f2820 msgr2=0x7f1e800f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:39.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.152+0000 7f1e84dbd700 1 -- 172.21.15.47:0/4242381508 shutdown_connections 2024-02-23T00:57:39.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.152+0000 7f1e84dbd700 1 -- 172.21.15.47:0/4242381508 wait complete. 2024-02-23T00:57:39.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.152+0000 7f1e84dbd700 1 librados: starting msgr at 2024-02-23T00:57:39.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.152+0000 7f1e84dbd700 1 librados: starting objecter 2024-02-23T00:57:39.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.152+0000 7f1e84dbd700 1 Processor -- start 2024-02-23T00:57:39.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.152+0000 7f1e84dbd700 1 -- start start 2024-02-23T00:57:39.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.152+0000 7f1e84dbd700 1 librados: setting wanted keys 2024-02-23T00:57:39.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.152+0000 7f1e84dbd700 1 librados: calling monclient init 2024-02-23T00:57:39.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.153+0000 7f1e84dbd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e800f8cb0 0x7f1e800f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:39.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.153+0000 7f1e84dbd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e800f9610 0x7f1e800f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:39.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.153+0000 7f1e84dbd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e80102210 0x7f1e800f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:39.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.153+0000 7f1e84dbd700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e80116340 con 0x7f1e80102210 2024-02-23T00:57:39.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.153+0000 7f1e84dbd700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e801164b0 con 0x7f1e800f8cb0 2024-02-23T00:57:39.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.153+0000 7f1e84dbd700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e80116670 con 0x7f1e800f9610 2024-02-23T00:57:39.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.153+0000 7f1e7ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e80102210 0x7f1e800f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:39.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.153+0000 7f1e7dd9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e800f9610 0x7f1e800f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:39.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.153+0000 7f1e7dd9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e800f9610 0x7f1e800f7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:34386/0 (socket says 172.21.15.47:34386) 2024-02-23T00:57:39.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.153+0000 7f1e7ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e80102210 0x7f1e800f7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53780/0 (socket says 172.21.15.47:53780) 2024-02-23T00:57:39.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.153+0000 7f1e7dd9b700 1 -- 172.21.15.47:0/438450978 learned_addr learned my addr 172.21.15.47:0/438450978 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:57:39.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.153+0000 7f1e7e59c700 1 --2- 172.21.15.47:0/438450978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e800f8cb0 0x7f1e800f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:39.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.153+0000 7f1e7dd9b700 1 -- 172.21.15.47:0/438450978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e800f8cb0 msgr2=0x7f1e800f6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:39.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.153+0000 7f1e7dd9b700 1 --2- 172.21.15.47:0/438450978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e800f8cb0 0x7f1e800f6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:39.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.153+0000 7f1e7dd9b700 1 -- 172.21.15.47:0/438450978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e80102210 msgr2=0x7f1e800f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:39.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.153+0000 7f1e7dd9b700 1 --2- 172.21.15.47:0/438450978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e80102210 0x7f1e800f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:39.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.153+0000 7f1e7dd9b700 1 -- 172.21.15.47:0/438450978 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e70005040 con 0x7f1e800f9610 2024-02-23T00:57:39.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.153+0000 7f1e7ed9d700 1 --2- 172.21.15.47:0/438450978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e80102210 0x7f1e800f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:57:39.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.153+0000 7f1e6f7fe700 10 client.?.objecter ms_handle_connect 0x7f1e800f9610 2024-02-23T00:57:39.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.153+0000 7f1e6f7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:57:39.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.153+0000 7f1e7dd9b700 1 --2- 172.21.15.47:0/438450978 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e800f9610 0x7f1e800f7150 secure :-1 s=READY pgs=2857 cs=0 l=1 rev1=1 rx=0x7f1e7400e970 tx=0x7f1e7400b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:39.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.153+0000 7f1e6f7fe700 1 -- 172.21.15.47:0/438450978 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e74004660 con 0x7f1e800f9610 2024-02-23T00:57:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.153+0000 7f1e6f7fe700 1 -- 172.21.15.47:0/438450978 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f1e7400f040 con 0x7f1e800f9610 2024-02-23T00:57:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.153+0000 7f1e84dbd700 1 -- 172.21.15.47:0/438450978 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e780097c0 con 0x7f1e800f9610 2024-02-23T00:57:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.153+0000 7f1e6f7fe700 1 -- 172.21.15.47:0/438450978 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e740146f0 con 0x7f1e800f9610 2024-02-23T00:57:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.154+0000 7f1e84dbd700 10 client.42347.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:57:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.154+0000 7f1e84dbd700 1 -- 172.21.15.47:0/438450978 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e80116d00 con 0x7f1e800f9610 2024-02-23T00:57:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.154+0000 7f1e84dbd700 1 librados: init done 2024-02-23T00:57:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.154+0000 7f1e84dbd700 1 -- 172.21.15.47:0/438450978 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e8005fa60 con 0x7f1e800f9610 2024-02-23T00:57:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.155+0000 7f1e6f7fe700 1 -- 172.21.15.47:0/438450978 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f1e74007b40 con 0x7f1e800f9610 2024-02-23T00:57:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.155+0000 7f1e6f7fe700 1 --2- 172.21.15.47:0/438450978 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1e68062cc0 0x7f1e68065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.155+0000 7f1e6f7fe700 1 -- 172.21.15.47:0/438450978 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(853..853 src has 670..853) v4 ==== 8663+0+0 (secure 0 0 0) 0x7f1e7407ce60 con 0x7f1e800f9610 2024-02-23T00:57:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.155+0000 7f1e6f7fe700 10 client.42347.objecter ms_dispatch 0x7f1e80001550 osd_map(853..853 src has 670..853) v4 2024-02-23T00:57:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.155+0000 7f1e6f7fe700 3 client.42347.objecter handle_osd_map got epochs [853,853] > 0 2024-02-23T00:57:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.155+0000 7f1e6f7fe700 3 client.42347.objecter handle_osd_map decoding full epoch 853 2024-02-23T00:57:39.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.155+0000 7f1e7e59c700 1 --2- 172.21.15.47:0/438450978 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1e68062cc0 0x7f1e68065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:39.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.156+0000 7f1e6f7fe700 20 client.42347.objecter dump_active .. 0 homeless 2024-02-23T00:57:39.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.156+0000 7f1e6f7fe700 10 client.42347.objecter ms_handle_connect 0x7f1e68062cc0 2024-02-23T00:57:39.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.156+0000 7f1e7e59c700 1 --2- 172.21.15.47:0/438450978 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1e68062cc0 0x7f1e68065180 secure :-1 s=READY pgs=3241 cs=0 l=1 rev1=1 rx=0x7f1e70009350 tx=0x7f1e7000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:39.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.160+0000 7f1e6f7fe700 1 -- 172.21.15.47:0/438450978 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f1e7404c750 con 0x7f1e800f9610 2024-02-23T00:57:39.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.343+0000 7f1e84dbd700 1 -- 172.21.15.47:0/438450978 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd primary-affinity", "id": 7, "weight": 1.0} v 0) v1 -- 0x7f1e80107010 con 0x7f1e800f9610 2024-02-23T00:57:39.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.430+0000 7f1e6f7fe700 1 -- 172.21.15.47:0/438450978 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]=0 set osd.7 primary-affinity to 1 (8655362) v854) v1 ==== 133+0+0 (secure 0 0 0) 0x7f1e74019070 con 0x7f1e800f9610 2024-02-23T00:57:39.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.431+0000 7f1e84dbd700 10 librados: watch_flush enter 2024-02-23T00:57:39.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.431+0000 7f1e84dbd700 10 librados: watch_flush exit 2024-02-23T00:57:39.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.431+0000 7f1e84dbd700 20 client.42347.objecter shutdown clearing up homeless session... 2024-02-23T00:57:39.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.431+0000 7f1e84dbd700 10 client.42347.objecter successfully canceled tick 2024-02-23T00:57:39.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.431+0000 7f1e84dbd700 1 -- 172.21.15.47:0/438450978 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1e68062cc0 msgr2=0x7f1e68065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:39.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.431+0000 7f1e84dbd700 1 --2- 172.21.15.47:0/438450978 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1e68062cc0 0x7f1e68065180 secure :-1 s=READY pgs=3241 cs=0 l=1 rev1=1 rx=0x7f1e70009350 tx=0x7f1e7000b040).stop 2024-02-23T00:57:39.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.431+0000 7f1e84dbd700 1 -- 172.21.15.47:0/438450978 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e800f9610 msgr2=0x7f1e800f7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:39.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.432+0000 7f1e84dbd700 1 --2- 172.21.15.47:0/438450978 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e800f9610 0x7f1e800f7150 secure :-1 s=READY pgs=2857 cs=0 l=1 rev1=1 rx=0x7f1e7400e970 tx=0x7f1e7400b040).stop 2024-02-23T00:57:39.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.432+0000 7f1e84dbd700 1 -- 172.21.15.47:0/438450978 shutdown_connections 2024-02-23T00:57:39.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.432+0000 7f1e84dbd700 1 --2- 172.21.15.47:0/438450978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e80102210 0x7f1e800f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:39.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.432+0000 7f1e84dbd700 1 --2- 172.21.15.47:0/438450978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e800f8cb0 0x7f1e800f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:39.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.432+0000 7f1e84dbd700 1 --2- 172.21.15.47:0/438450978 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1e68062cc0 0x7f1e68065180 unknown :-1 s=CLOSED pgs=3241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:39.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.432+0000 7f1e84dbd700 1 --2- 172.21.15.47:0/438450978 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e800f9610 0x7f1e800f7150 unknown :-1 s=CLOSED pgs=2857 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:39.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.432+0000 7f1e84dbd700 1 -- 172.21.15.47:0/438450978 >> 172.21.15.47:0/438450978 conn(0x7f1e800f2820 msgr2=0x7f1e800f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:39.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.432+0000 7f1e84dbd700 1 -- 172.21.15.47:0/438450978 shutdown_connections 2024-02-23T00:57:39.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.432+0000 7f1e84dbd700 1 -- 172.21.15.47:0/438450978 wait complete. 2024-02-23T00:57:39.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:39.432+0000 7f1e84dbd700 1 librados: shutdown 2024-02-23T00:57:39.436 INFO:teuthology.orchestra.run.smithi047.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-02-23T00:57:39.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:39.345196+0000 mon.c (mon.2) 534 : audit [INF] from='client.? 172.21.15.47:0/438450978' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-02-23T00:57:39.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:39.345495+0000 mon.a (mon.0) 3935 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-02-23T00:57:39.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:39.345196+0000 mon.c (mon.2) 534 : audit [INF] from='client.? 172.21.15.47:0/438450978' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-02-23T00:57:39.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:39.345495+0000 mon.a (mon.0) 3935 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-02-23T00:57:39.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:39.345196+0000 mon.c (mon.2) 534 : audit [INF] from='client.? 172.21.15.47:0/438450978' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-02-23T00:57:39.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:39.345495+0000 mon.a (mon.0) 3935 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-02-23T00:57:40.180 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-02-23T00:57:40.180 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-02-23T00:57:40.180 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-02-23T00:57:40.180 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-02-23T00:57:40.180 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-23T00:57:40.180 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd dump --format=json 2024-02-23T00:57:40.469 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:57:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:39.430303+0000 mon.a (mon.0) 3936 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-02-23T00:57:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:39.430376+0000 mon.a (mon.0) 3937 : cluster [DBG] osdmap e854: 8 total, 8 up, 8 in 2024-02-23T00:57:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:39.432876+0000 mgr.y (mgr.14517) 739 : cluster [DBG] pgmap v1323: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 844 B/s rd, 0 op/s 2024-02-23T00:57:40.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:39.843850+0000 osd.3 (osd.3) 345 : cluster [DBG] 18.9 deep-scrub starts 2024-02-23T00:57:40.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:39.845516+0000 osd.3 (osd.3) 346 : cluster [DBG] 18.9 deep-scrub ok 2024-02-23T00:57:40.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:39.908734+0000 osd.1 (osd.1) 342 : cluster [DBG] 3.0 deep-scrub starts 2024-02-23T00:57:40.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:39.910294+0000 osd.1 (osd.1) 343 : cluster [DBG] 3.0 deep-scrub ok 2024-02-23T00:57:40.727 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:57:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:57:40] "GET /metrics HTTP/1.1" 200 187134 "" "Prometheus/2.33.4" 2024-02-23T00:57:40.727 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:39.430303+0000 mon.a (mon.0) 3936 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-02-23T00:57:40.727 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:39.430376+0000 mon.a (mon.0) 3937 : cluster [DBG] osdmap e854: 8 total, 8 up, 8 in 2024-02-23T00:57:40.727 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:39.432876+0000 mgr.y (mgr.14517) 739 : cluster [DBG] pgmap v1323: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 844 B/s rd, 0 op/s 2024-02-23T00:57:40.727 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:39.843850+0000 osd.3 (osd.3) 345 : cluster [DBG] 18.9 deep-scrub starts 2024-02-23T00:57:40.727 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:39.845516+0000 osd.3 (osd.3) 346 : cluster [DBG] 18.9 deep-scrub ok 2024-02-23T00:57:40.727 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:39.908734+0000 osd.1 (osd.1) 342 : cluster [DBG] 3.0 deep-scrub starts 2024-02-23T00:57:40.728 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:39.910294+0000 osd.1 (osd.1) 343 : cluster [DBG] 3.0 deep-scrub ok 2024-02-23T00:57:40.728 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:39.430303+0000 mon.a (mon.0) 3936 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-02-23T00:57:40.728 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:39.430376+0000 mon.a (mon.0) 3937 : cluster [DBG] osdmap e854: 8 total, 8 up, 8 in 2024-02-23T00:57:40.728 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:39.432876+0000 mgr.y (mgr.14517) 739 : cluster [DBG] pgmap v1323: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 844 B/s rd, 0 op/s 2024-02-23T00:57:40.728 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:39.843850+0000 osd.3 (osd.3) 345 : cluster [DBG] 18.9 deep-scrub starts 2024-02-23T00:57:40.728 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:39.845516+0000 osd.3 (osd.3) 346 : cluster [DBG] 18.9 deep-scrub ok 2024-02-23T00:57:40.728 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:39.908734+0000 osd.1 (osd.1) 342 : cluster [DBG] 3.0 deep-scrub starts 2024-02-23T00:57:40.728 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:39.910294+0000 osd.1 (osd.1) 343 : cluster [DBG] 3.0 deep-scrub ok 2024-02-23T00:57:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:40.526054+0000 osd.6 (osd.6) 365 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:57:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:40.526278+0000 osd.6 (osd.6) 366 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:57:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:40.834460+0000 osd.3 (osd.3) 347 : cluster [DBG] 18.a deep-scrub starts 2024-02-23T00:57:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:40.835668+0000 osd.3 (osd.3) 348 : cluster [DBG] 18.a deep-scrub ok 2024-02-23T00:57:41.737 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:40.526054+0000 osd.6 (osd.6) 365 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:57:41.737 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:40.526278+0000 osd.6 (osd.6) 366 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:57:41.737 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:40.834460+0000 osd.3 (osd.3) 347 : cluster [DBG] 18.a deep-scrub starts 2024-02-23T00:57:41.737 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:40.835668+0000 osd.3 (osd.3) 348 : cluster [DBG] 18.a deep-scrub ok 2024-02-23T00:57:41.737 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:40.526054+0000 osd.6 (osd.6) 365 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:57:41.737 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:40.526278+0000 osd.6 (osd.6) 366 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:57:41.737 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:40.834460+0000 osd.3 (osd.3) 347 : cluster [DBG] 18.a deep-scrub starts 2024-02-23T00:57:41.738 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:40.835668+0000 osd.3 (osd.3) 348 : cluster [DBG] 18.a deep-scrub ok 2024-02-23T00:57:42.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.203+0000 7fcee5446700 1 -- 172.21.15.47:0/1131942879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcee00f7330 msgr2=0x7fcee0103930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:42.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.203+0000 7fcee5446700 1 --2- 172.21.15.47:0/1131942879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcee00f7330 0x7fcee0103930 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 rx=0x7fced8004870 tx=0x7fced8009cb0).stop 2024-02-23T00:57:42.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.203+0000 7fcee5446700 1 -- 172.21.15.47:0/1131942879 shutdown_connections 2024-02-23T00:57:42.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.203+0000 7fcee5446700 1 --2- 172.21.15.47:0/1131942879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcee00f7330 0x7fcee0103930 unknown :-1 s=CLOSED pgs=2675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:42.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.203+0000 7fcee5446700 1 --2- 172.21.15.47:0/1131942879 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcee00fcac0 0x7fcee00fcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:42.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.203+0000 7fcee5446700 1 --2- 172.21.15.47:0/1131942879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcee00fd420 0x7fcee00f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:42.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.203+0000 7fcee5446700 1 -- 172.21.15.47:0/1131942879 >> 172.21.15.47:0/1131942879 conn(0x7fcee00f26f0 msgr2=0x7fcee00f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:42.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.203+0000 7fcee5446700 1 -- 172.21.15.47:0/1131942879 shutdown_connections 2024-02-23T00:57:42.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.203+0000 7fcee5446700 1 -- 172.21.15.47:0/1131942879 wait complete. 2024-02-23T00:57:42.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.204+0000 7fcee5446700 1 librados: starting msgr at 2024-02-23T00:57:42.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.204+0000 7fcee5446700 1 librados: starting objecter 2024-02-23T00:57:42.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.204+0000 7fcee5446700 1 Processor -- start 2024-02-23T00:57:42.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.204+0000 7fcee5446700 1 -- start start 2024-02-23T00:57:42.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.204+0000 7fcee5446700 1 librados: setting wanted keys 2024-02-23T00:57:42.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.204+0000 7fcee5446700 1 librados: calling monclient init 2024-02-23T00:57:42.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.204+0000 7fcee5446700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcee00f7330 0x7fcee00f88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:42.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.204+0000 7fcee5446700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcee00fcac0 0x7fcee00f8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:42.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.204+0000 7fcee5446700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcee00fd420 0x7fcee00f92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:42.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.204+0000 7fcee5446700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee01160b0 con 0x7fcee00fcac0 2024-02-23T00:57:42.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.204+0000 7fcee5446700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee0116220 con 0x7fcee00fd420 2024-02-23T00:57:42.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.204+0000 7fcee5446700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee01163e0 con 0x7fcee00f7330 2024-02-23T00:57:42.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.205+0000 7fcedeffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcee00f7330 0x7fcee00f88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:42.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.205+0000 7fcedeffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcee00f7330 0x7fcee00f88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:34402/0 (socket says 172.21.15.47:34402) 2024-02-23T00:57:42.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.205+0000 7fcedeffd700 1 -- 172.21.15.47:0/2180000631 learned_addr learned my addr 172.21.15.47:0/2180000631 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:57:42.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.205+0000 7fcedf7fe700 1 --2- 172.21.15.47:0/2180000631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcee00fd420 0x7fcee00f92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:42.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.205+0000 7fcede7fc700 1 --2- 172.21.15.47:0/2180000631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcee00fcac0 0x7fcee00f8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:42.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.205+0000 7fcedeffd700 1 -- 172.21.15.47:0/2180000631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcee00fd420 msgr2=0x7fcee00f92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:42.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.205+0000 7fcedeffd700 1 --2- 172.21.15.47:0/2180000631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcee00fd420 0x7fcee00f92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:42.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.205+0000 7fcedeffd700 1 -- 172.21.15.47:0/2180000631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcee00fcac0 msgr2=0x7fcee00f8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:42.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.205+0000 7fcedeffd700 1 --2- 172.21.15.47:0/2180000631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcee00fcac0 0x7fcee00f8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:42.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.205+0000 7fcedeffd700 1 -- 172.21.15.47:0/2180000631 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced00097c0 con 0x7fcee00f7330 2024-02-23T00:57:42.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.205+0000 7fcedeffd700 1 --2- 172.21.15.47:0/2180000631 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcee00f7330 0x7fcee00f88c0 secure :-1 s=READY pgs=2858 cs=0 l=1 rev1=1 rx=0x7fced0001e70 tx=0x7fced0004c50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:42.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.205+0000 7fced7fff700 10 client.?.objecter ms_handle_connect 0x7fcee00f7330 2024-02-23T00:57:42.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.205+0000 7fced7fff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:57:42.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.205+0000 7fced7fff700 1 -- 172.21.15.47:0/2180000631 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced00099f0 con 0x7fcee00f7330 2024-02-23T00:57:42.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.205+0000 7fced7fff700 1 -- 172.21.15.47:0/2180000631 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fced00090d0 con 0x7fcee00f7330 2024-02-23T00:57:42.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.205+0000 7fcee5446700 1 -- 172.21.15.47:0/2180000631 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fced8005040 con 0x7fcee00f7330 2024-02-23T00:57:42.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.205+0000 7fced7fff700 1 -- 172.21.15.47:0/2180000631 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced0007b50 con 0x7fcee00f7330 2024-02-23T00:57:42.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.205+0000 7fcee5446700 10 client.42353.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:57:42.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.205+0000 7fcee5446700 1 -- 172.21.15.47:0/2180000631 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee0116af0 con 0x7fcee00f7330 2024-02-23T00:57:42.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.205+0000 7fcee5446700 1 librados: init done 2024-02-23T00:57:42.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.206+0000 7fcee5446700 1 -- 172.21.15.47:0/2180000631 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee005f990 con 0x7fcee00f7330 2024-02-23T00:57:42.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.207+0000 7fced7fff700 1 -- 172.21.15.47:0/2180000631 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fced000fad0 con 0x7fcee00f7330 2024-02-23T00:57:42.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.207+0000 7fced7fff700 1 --2- 172.21.15.47:0/2180000631 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcecc062cc0 0x7fcecc065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:42.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.207+0000 7fced7fff700 1 -- 172.21.15.47:0/2180000631 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(855..855 src has 670..855) v4 ==== 8663+0+0 (secure 0 0 0) 0x7fced007dcd0 con 0x7fcee00f7330 2024-02-23T00:57:42.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.207+0000 7fced7fff700 10 client.42353.objecter ms_dispatch 0x7fcee0001550 osd_map(855..855 src has 670..855) v4 2024-02-23T00:57:42.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.207+0000 7fced7fff700 3 client.42353.objecter handle_osd_map got epochs [855,855] > 0 2024-02-23T00:57:42.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.207+0000 7fced7fff700 3 client.42353.objecter handle_osd_map decoding full epoch 855 2024-02-23T00:57:42.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.207+0000 7fcede7fc700 1 --2- 172.21.15.47:0/2180000631 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcecc062cc0 0x7fcecc065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:42.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.207+0000 7fced7fff700 20 client.42353.objecter dump_active .. 0 homeless 2024-02-23T00:57:42.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.208+0000 7fced7fff700 10 client.42353.objecter ms_handle_connect 0x7fcecc062cc0 2024-02-23T00:57:42.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.208+0000 7fcede7fc700 1 --2- 172.21.15.47:0/2180000631 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcecc062cc0 0x7fcecc065180 secure :-1 s=READY pgs=3242 cs=0 l=1 rev1=1 rx=0x7fcee00fa060 tx=0x7fcec800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:42.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.212+0000 7fced7fff700 1 -- 172.21.15.47:0/2180000631 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fced004d590 con 0x7fcee00f7330 2024-02-23T00:57:42.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.387+0000 7fcee5446700 1 -- 172.21.15.47:0/2180000631 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcee00f9da0 con 0x7fcee00f7330 2024-02-23T00:57:42.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.388+0000 7fced7fff700 1 -- 172.21.15.47:0/2180000631 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v855) v1 ==== 74+0+24229 (secure 0 0 0) 0x7fced0050bb0 con 0x7fcee00f7330 2024-02-23T00:57:42.391 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:57:42.391 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":855,"fsid":"fce2617e-d1e2-11ee-95c0-87774f69a715","created":"2024-02-23T00:32:55.399437+0000","modified":"2024-02-23T00:57:41.435682+0000","last_up_change":"2024-02-23T00:39:05.662717+0000","last_in_change":"2024-02-23T00:56:21.352185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":110,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":256,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-23T00:36:52.622994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-02-23T00:39:13.277309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"iscsi","create_time":"2024-02-23T00:39:13.761312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-23T00:39:15.255656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"611","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-23T00:39:17.226393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":42,"pg_placement_num":42,"pg_placement_num_target":42,"pg_num_target":42,"pg_num_pending":42,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"825","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"769","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-23T00:39:19.300351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":12,"pg_placement_num":12,"pg_placement_num_target":12,"pg_num_target":12,"pg_num_pending":12,"last_pg_merge_meta":{"source_pgid":"6.c","ready_epoch":403,"last_epoch_started":402,"last_epoch_clean":402,"source_version":"67'1","target_version":"0'0"},"last_change":"822","last_force_op_resend":"0","last_force_op_resend_prenautilus":"403","last_force_op_resend_preluminous":"401","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}},{"pool":18,"pool_name":"ceph_test_rados_api_asio","create_time":"2024-02-23T00:42:52.665618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"380050f9-9001-46dc-bb29-037987b4c8e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":828,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":4042552860},{"type":"v1","addr":"172.21.15.47:6803","nonce":4042552860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":4042552860},{"type":"v1","addr":"172.21.15.47:6805","nonce":4042552860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":4042552860},{"type":"v1","addr":"172.21.15.47:6809","nonce":4042552860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":4042552860},{"type":"v1","addr":"172.21.15.47:6807","nonce":4042552860}]},"public_addr":"172.21.15.47:6803/4042552860","cluster_addr":"172.21.15.47:6805/4042552860","heartbeat_back_addr":"172.21.15.47:6809/4042552860","heartbeat_front_addr":"172.21.15.47:6807/4042552860","state":["exists","up"]},{"osd":1,"uuid":"b06d0cce-55d2-4a6a-a796-4db0b9bb9d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":837,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1945417179},{"type":"v1","addr":"172.21.15.47:6811","nonce":1945417179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1945417179},{"type":"v1","addr":"172.21.15.47:6813","nonce":1945417179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":1945417179},{"type":"v1","addr":"172.21.15.47:6817","nonce":1945417179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1945417179},{"type":"v1","addr":"172.21.15.47:6815","nonce":1945417179}]},"public_addr":"172.21.15.47:6811/1945417179","cluster_addr":"172.21.15.47:6813/1945417179","heartbeat_back_addr":"172.21.15.47:6817/1945417179","heartbeat_front_addr":"172.21.15.47:6815/1945417179","state":["exists","up"]},{"osd":2,"uuid":"6e44d7ca-67a1-4868-bb64-891cf477ebd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":831,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":3596356785},{"type":"v1","addr":"172.21.15.47:6819","nonce":3596356785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":3596356785},{"type":"v1","addr":"172.21.15.47:6821","nonce":3596356785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":3596356785},{"type":"v1","addr":"172.21.15.47:6825","nonce":3596356785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":3596356785},{"type":"v1","addr":"172.21.15.47:6823","nonce":3596356785}]},"public_addr":"172.21.15.47:6819/3596356785","cluster_addr":"172.21.15.47:6821/3596356785","heartbeat_back_addr":"172.21.15.47:6825/3596356785","heartbeat_front_addr":"172.21.15.47:6823/3596356785","state":["exists","up"]},{"osd":3,"uuid":"c88debab-cb0b-46f6-a7d8-c5dc61fc059f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":843,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1195438081},{"type":"v1","addr":"172.21.15.47:6827","nonce":1195438081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1195438081},{"type":"v1","addr":"172.21.15.47:6829","nonce":1195438081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1195438081},{"type":"v1","addr":"172.21.15.47:6833","nonce":1195438081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1195438081},{"type":"v1","addr":"172.21.15.47:6831","nonce":1195438081}]},"public_addr":"172.21.15.47:6827/1195438081","cluster_addr":"172.21.15.47:6829/1195438081","heartbeat_back_addr":"172.21.15.47:6833/1195438081","heartbeat_front_addr":"172.21.15.47:6831/1195438081","state":["exists","up"]},{"osd":4,"uuid":"d17966d7-eb12-4389-ac90-8d15efe290e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":817,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2530629682},{"type":"v1","addr":"172.21.15.196:6801","nonce":2530629682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2530629682},{"type":"v1","addr":"172.21.15.196:6803","nonce":2530629682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2530629682},{"type":"v1","addr":"172.21.15.196:6807","nonce":2530629682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2530629682},{"type":"v1","addr":"172.21.15.196:6805","nonce":2530629682}]},"public_addr":"172.21.15.196:6801/2530629682","cluster_addr":"172.21.15.196:6803/2530629682","heartbeat_back_addr":"172.21.15.196:6807/2530629682","heartbeat_front_addr":"172.21.15.196:6805/2530629682","state":["exists","up"]},{"osd":5,"uuid":"b729d646-4899-47b3-b917-acceba46d241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":817,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1017187491},{"type":"v1","addr":"172.21.15.196:6809","nonce":1017187491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1017187491},{"type":"v1","addr":"172.21.15.196:6811","nonce":1017187491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1017187491},{"type":"v1","addr":"172.21.15.196:6815","nonce":1017187491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1017187491},{"type":"v1","addr":"172.21.15.196:6813","nonce":1017187491}]},"public_addr":"172.21.15.196:6809/1017187491","cluster_addr":"172.21.15.196:6811/1017187491","heartbeat_back_addr":"172.21.15.196:6815/1017187491","heartbeat_front_addr":"172.21.15.196:6813/1017187491","state":["exists","up"]},{"osd":6,"uuid":"df337124-9488-4aad-98e5-3614b51fc757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":817,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3885668599},{"type":"v1","addr":"172.21.15.196:6817","nonce":3885668599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3885668599},{"type":"v1","addr":"172.21.15.196:6819","nonce":3885668599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3885668599},{"type":"v1","addr":"172.21.15.196:6823","nonce":3885668599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3885668599},{"type":"v1","addr":"172.21.15.196:6821","nonce":3885668599}]},"public_addr":"172.21.15.196:6817/3885668599","cluster_addr":"172.21.15.196:6819/3885668599","heartbeat_back_addr":"172.21.15.196:6823/3885668599","heartbeat_front_addr":"172.21.15.196:6821/3885668599","state":["exists","up"]},{"osd":7,"uuid":"84fee1f2-1a88-447a-b1f3-56ff7efe7fec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":817,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2261821675},{"type":"v1","addr":"172.21.15.196:6825","nonce":2261821675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2261821675},{"type":"v1","addr":"172.21.15.196:6827","nonce":2261821675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2261821675},{"type":"v1","addr":"172.21.15.196:6831","nonce":2261821675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2261821675},{"type":"v1","addr":"172.21.15.196:6829","nonce":2261821675}]},"public_addr":"172.21.15.196:6825/2261821675","cluster_addr":"172.21.15.196:6827/2261821675","heartbeat_back_addr":"172.21.15.196:6831/2261821675","heartbeat_front_addr":"172.21.15.196:6829/2261821675","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:57:27.946264+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:56:23.616484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:56:28.307044+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:56:38.438921+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:56:51.029513+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:57:05.806585+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:57:40.526282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:56:24.181795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.14","mappings":[{"from":3,"to":4}]},{"pgid":"2.1b","mappings":[{"from":7,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":2}]},{"pgid":"5.6","mappings":[{"from":5,"to":6}]},{"pgid":"5.7","mappings":[{"from":1,"to":6}]},{"pgid":"5.11","mappings":[{"from":4,"to":3}]},{"pgid":"5.12","mappings":[{"from":1,"to":6}]},{"pgid":"5.18","mappings":[{"from":1,"to":3}]},{"pgid":"5.1c","mappings":[{"from":4,"to":6}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6800/622994871":"2024-02-24T00:39:41.110158+0000","172.21.15.47:0/1561174431":"2024-02-24T00:39:41.110158+0000","172.21.15.47:0/3349912606":"2024-02-24T00:39:41.110158+0000","172.21.15.47:0/710432524":"2024-02-24T00:39:41.110158+0000","172.21.15.47:0/2804352395":"2024-02-24T00:33:52.061914+0000","172.21.15.47:6801/1916598887":"2024-02-24T00:33:52.061914+0000","172.21.15.47:6800/1916598887":"2024-02-24T00:33:52.061914+0000","172.21.15.47:0/792211316":"2024-02-24T00:33:27.972945+0000","172.21.15.47:0/381995198":"2024-02-24T00:33:52.061914+0000","172.21.15.47:6800/2409691510":"2024-02-24T00:33:27.972945+0000","172.21.15.47:6801/2409691510":"2024-02-24T00:33:27.972945+0000","172.21.15.47:6801/622994871":"2024-02-24T00:39:41.110158+0000","172.21.15.47:0/1448327041":"2024-02-24T00:33:27.972945+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1}]},{"pool":25,"snaps":[{"begin":9,"length":2}]},{"pool":59,"snaps":[{"begin":2,"length":2},{"begin":6,"length":2}]},{"pool":79,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1}]},{"pool":84,"snaps":[{"begin":2,"length":1}]},{"pool":93,"snaps":[{"begin":4,"length":4}]},{"pool":95,"snaps":[{"begin":6,"length":2}]},{"pool":108,"snaps":[{"begin":2,"length":1}]},{"pool":114,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1}]},{"pool":129,"snaps":[{"begin":2,"length":2},{"begin":6,"length":2},{"begin":10,"length":2}]},{"pool":132,"snaps":[{"begin":12,"length":1}]},{"pool":153,"snaps":[{"begin":14,"length":1}]},{"pool":164,"snaps":[{"begin":17,"length":1}]},{"pool":175,"snaps":[{"begin":20,"length":1}]},{"pool":237,"snaps":[{"begin":2,"length":1}]},{"pool":238,"snaps":[{"begin":4,"length":1}]},{"pool":241,"snaps":[{"begin":6,"length":1}]},{"pool":245,"snaps":[{"begin":9,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-23T00:57:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.390+0000 7fcee5446700 10 librados: watch_flush enter 2024-02-23T00:57:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.390+0000 7fcee5446700 10 librados: watch_flush exit 2024-02-23T00:57:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.390+0000 7fcee5446700 20 client.42353.objecter shutdown clearing up homeless session... 2024-02-23T00:57:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.390+0000 7fcee5446700 10 client.42353.objecter successfully canceled tick 2024-02-23T00:57:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.390+0000 7fcee5446700 1 -- 172.21.15.47:0/2180000631 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcecc062cc0 msgr2=0x7fcecc065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.390+0000 7fcee5446700 1 --2- 172.21.15.47:0/2180000631 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcecc062cc0 0x7fcecc065180 secure :-1 s=READY pgs=3242 cs=0 l=1 rev1=1 rx=0x7fcee00fa060 tx=0x7fcec800b040).stop 2024-02-23T00:57:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.390+0000 7fcee5446700 1 -- 172.21.15.47:0/2180000631 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcee00f7330 msgr2=0x7fcee00f88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.390+0000 7fcee5446700 1 --2- 172.21.15.47:0/2180000631 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcee00f7330 0x7fcee00f88c0 secure :-1 s=READY pgs=2858 cs=0 l=1 rev1=1 rx=0x7fced0001e70 tx=0x7fced0004c50).stop 2024-02-23T00:57:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.390+0000 7fcee5446700 1 -- 172.21.15.47:0/2180000631 shutdown_connections 2024-02-23T00:57:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.390+0000 7fcee5446700 1 --2- 172.21.15.47:0/2180000631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcee00fd420 0x7fcee00f92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.390+0000 7fcee5446700 1 --2- 172.21.15.47:0/2180000631 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fcecc062cc0 0x7fcecc065180 unknown :-1 s=CLOSED pgs=3242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.390+0000 7fcee5446700 1 --2- 172.21.15.47:0/2180000631 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcee00f7330 0x7fcee00f88c0 unknown :-1 s=CLOSED pgs=2858 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.390+0000 7fcee5446700 1 --2- 172.21.15.47:0/2180000631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcee00fcac0 0x7fcee00f8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.390+0000 7fcee5446700 1 -- 172.21.15.47:0/2180000631 >> 172.21.15.47:0/2180000631 conn(0x7fcee00f26f0 msgr2=0x7fcee0060ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.390+0000 7fcee5446700 1 -- 172.21.15.47:0/2180000631 shutdown_connections 2024-02-23T00:57:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.390+0000 7fcee5446700 1 -- 172.21.15.47:0/2180000631 wait complete. 2024-02-23T00:57:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:42.390+0000 7fcee5446700 1 librados: shutdown 2024-02-23T00:57:42.694 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:57:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:57:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:57:42.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:41.433321+0000 mgr.y (mgr.14517) 740 : cluster [DBG] pgmap v1324: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:57:42.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:41.443353+0000 mon.a (mon.0) 3938 : cluster [DBG] osdmap e855: 8 total, 8 up, 8 in 2024-02-23T00:57:42.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:42.388551+0000 mon.c (mon.2) 535 : audit [DBG] from='client.? 172.21.15.47:0/2180000631' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-23T00:57:42.739 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:41.433321+0000 mgr.y (mgr.14517) 740 : cluster [DBG] pgmap v1324: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:57:42.739 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:41.443353+0000 mon.a (mon.0) 3938 : cluster [DBG] osdmap e855: 8 total, 8 up, 8 in 2024-02-23T00:57:42.739 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:42.388551+0000 mon.c (mon.2) 535 : audit [DBG] from='client.? 172.21.15.47:0/2180000631' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-23T00:57:42.739 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:41.433321+0000 mgr.y (mgr.14517) 740 : cluster [DBG] pgmap v1324: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-23T00:57:42.739 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:41.443353+0000 mon.a (mon.0) 3938 : cluster [DBG] osdmap e855: 8 total, 8 up, 8 in 2024-02-23T00:57:42.739 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:42 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:42.388551+0000 mon.c (mon.2) 535 : audit [DBG] from='client.? 172.21.15.47:0/2180000631' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-23T00:57:43.159 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-23T00:57:43.159 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd dump --format=json 2024-02-23T00:57:43.441 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:57:43.444 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:57:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:57:43.448 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:42.852941+0000 osd.3 (osd.3) 349 : cluster [DBG] 4.19 scrub starts 2024-02-23T00:57:43.451 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:42.852941+0000 osd.3 (osd.3) 349 : cluster [DBG] 4.19 scrub starts 2024-02-23T00:57:43.451 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:42.854748+0000 osd.3 (osd.3) 350 : cluster [DBG] 4.19 scrub ok 2024-02-23T00:57:43.451 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:42.891643+0000 osd.1 (osd.1) 344 : cluster [DBG] 5.19 scrub starts 2024-02-23T00:57:43.451 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:42.894614+0000 osd.1 (osd.1) 345 : cluster [DBG] 5.19 scrub ok 2024-02-23T00:57:43.451 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:43.164231+0000 osd.5 (osd.5) 408 : cluster [DBG] 18.1e scrub starts 2024-02-23T00:57:43.451 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:43.167643+0000 osd.5 (osd.5) 409 : cluster [DBG] 18.1e scrub ok 2024-02-23T00:57:43.716 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:42.854748+0000 osd.3 (osd.3) 350 : cluster [DBG] 4.19 scrub ok 2024-02-23T00:57:43.716 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:42.891643+0000 osd.1 (osd.1) 344 : cluster [DBG] 5.19 scrub starts 2024-02-23T00:57:43.716 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:42.894614+0000 osd.1 (osd.1) 345 : cluster [DBG] 5.19 scrub ok 2024-02-23T00:57:43.716 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:43.164231+0000 osd.5 (osd.5) 408 : cluster [DBG] 18.1e scrub starts 2024-02-23T00:57:43.717 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:43.167643+0000 osd.5 (osd.5) 409 : cluster [DBG] 18.1e scrub ok 2024-02-23T00:57:43.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:42.852941+0000 osd.3 (osd.3) 349 : cluster [DBG] 4.19 scrub starts 2024-02-23T00:57:43.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:42.854748+0000 osd.3 (osd.3) 350 : cluster [DBG] 4.19 scrub ok 2024-02-23T00:57:43.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:42.891643+0000 osd.1 (osd.1) 344 : cluster [DBG] 5.19 scrub starts 2024-02-23T00:57:43.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:42.894614+0000 osd.1 (osd.1) 345 : cluster [DBG] 5.19 scrub ok 2024-02-23T00:57:43.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:43.164231+0000 osd.5 (osd.5) 408 : cluster [DBG] 18.1e scrub starts 2024-02-23T00:57:43.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:43.167643+0000 osd.5 (osd.5) 409 : cluster [DBG] 18.1e scrub ok 2024-02-23T00:57:44.728 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:43.080969+0000 mgr.y (mgr.14517) 741 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:57:44.728 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:43.433999+0000 mgr.y (mgr.14517) 742 : cluster [DBG] pgmap v1326: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-23T00:57:44.728 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:43.557288+0000 osd.3 (osd.3) 351 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:57:44.728 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:43.557433+0000 osd.3 (osd.3) 352 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:57:44.729 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:43.653958+0000 osd.6 (osd.6) 367 : cluster [DBG] 5.11 scrub starts 2024-02-23T00:57:44.729 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:43.654907+0000 osd.6 (osd.6) 368 : cluster [DBG] 5.11 scrub ok 2024-02-23T00:57:44.729 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:43.080969+0000 mgr.y (mgr.14517) 741 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:57:44.729 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:43.433999+0000 mgr.y (mgr.14517) 742 : cluster [DBG] pgmap v1326: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-23T00:57:44.729 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:43.557288+0000 osd.3 (osd.3) 351 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:57:44.729 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:43.557433+0000 osd.3 (osd.3) 352 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:57:44.729 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:43.653958+0000 osd.6 (osd.6) 367 : cluster [DBG] 5.11 scrub starts 2024-02-23T00:57:44.729 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:44 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:43.654907+0000 osd.6 (osd.6) 368 : cluster [DBG] 5.11 scrub ok 2024-02-23T00:57:44.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:43.080969+0000 mgr.y (mgr.14517) 741 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:57:44.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:43.433999+0000 mgr.y (mgr.14517) 742 : cluster [DBG] pgmap v1326: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-23T00:57:44.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:43.557288+0000 osd.3 (osd.3) 351 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:57:44.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:43.557433+0000 osd.3 (osd.3) 352 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:57:44.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:43.653958+0000 osd.6 (osd.6) 367 : cluster [DBG] 5.11 scrub starts 2024-02-23T00:57:44.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:43.654907+0000 osd.6 (osd.6) 368 : cluster [DBG] 5.11 scrub ok 2024-02-23T00:57:45.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.299+0000 7f0e16a1d700 1 -- 172.21.15.47:0/2320202206 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0e100fe510 msgr2=0x7f0e100f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:45.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.299+0000 7f0e16a1d700 1 --2- 172.21.15.47:0/2320202206 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0e100fe510 0x7f0e100f7d40 secure :-1 s=READY pgs=2859 cs=0 l=1 rev1=1 rx=0x7f0df8004870 tx=0x7f0df8009cb0).stop 2024-02-23T00:57:45.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.299+0000 7f0e16a1d700 1 -- 172.21.15.47:0/2320202206 shutdown_connections 2024-02-23T00:57:45.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.299+0000 7f0e16a1d700 1 --2- 172.21.15.47:0/2320202206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e100f8420 0x7f0e10104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:45.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.299+0000 7f0e16a1d700 1 --2- 172.21.15.47:0/2320202206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e100fdbb0 0x7f0e100fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:45.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.299+0000 7f0e16a1d700 1 --2- 172.21.15.47:0/2320202206 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0e100fe510 0x7f0e100f7d40 unknown :-1 s=CLOSED pgs=2859 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:45.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.299+0000 7f0e16a1d700 1 -- 172.21.15.47:0/2320202206 >> 172.21.15.47:0/2320202206 conn(0x7f0e100f3800 msgr2=0x7f0e100f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:45.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.299+0000 7f0e16a1d700 1 -- 172.21.15.47:0/2320202206 shutdown_connections 2024-02-23T00:57:45.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.299+0000 7f0e16a1d700 1 -- 172.21.15.47:0/2320202206 wait complete. 2024-02-23T00:57:45.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.299+0000 7f0e16a1d700 1 librados: starting msgr at 2024-02-23T00:57:45.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.299+0000 7f0e16a1d700 1 librados: starting objecter 2024-02-23T00:57:45.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.299+0000 7f0e16a1d700 1 Processor -- start 2024-02-23T00:57:45.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.299+0000 7f0e16a1d700 1 -- start start 2024-02-23T00:57:45.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.299+0000 7f0e16a1d700 1 librados: setting wanted keys 2024-02-23T00:57:45.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.299+0000 7f0e16a1d700 1 librados: calling monclient init 2024-02-23T00:57:45.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.300+0000 7f0e16a1d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e100f8420 0x7f0e10110070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:45.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.300+0000 7f0e16a1d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e100fdbb0 0x7f0e10110560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:45.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.300+0000 7f0e16a1d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0e100fe510 0x7f0e1010a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:45.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.300+0000 7f0e16a1d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1010a700 con 0x7f0e100fdbb0 2024-02-23T00:57:45.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.300+0000 7f0e16a1d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1010a870 con 0x7f0e100f8420 2024-02-23T00:57:45.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.300+0000 7f0e16a1d700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1010aa30 con 0x7f0e100fe510 2024-02-23T00:57:45.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.300+0000 7f0e14fba700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0e100fe510 0x7f0e1010a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:45.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.300+0000 7f0e0f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e100fdbb0 0x7f0e10110560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:45.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.300+0000 7f0e14fba700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0e100fe510 0x7f0e1010a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:34422/0 (socket says 172.21.15.47:34422) 2024-02-23T00:57:45.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.300+0000 7f0e14fba700 1 -- 172.21.15.47:0/285657423 learned_addr learned my addr 172.21.15.47:0/285657423 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:57:45.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.300+0000 7f0e0ffff700 1 --2- 172.21.15.47:0/285657423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e100f8420 0x7f0e10110070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:45.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.300+0000 7f0e14fba700 1 -- 172.21.15.47:0/285657423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e100f8420 msgr2=0x7f0e10110070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:45.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.300+0000 7f0e14fba700 1 --2- 172.21.15.47:0/285657423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e100f8420 0x7f0e10110070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:45.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.300+0000 7f0e14fba700 1 -- 172.21.15.47:0/285657423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e100fdbb0 msgr2=0x7f0e10110560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:45.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.300+0000 7f0e14fba700 1 --2- 172.21.15.47:0/285657423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e100fdbb0 0x7f0e10110560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:45.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.300+0000 7f0e14fba700 1 -- 172.21.15.47:0/285657423 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e080097c0 con 0x7f0e100fe510 2024-02-23T00:57:45.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.300+0000 7f0e0f7fe700 1 --2- 172.21.15.47:0/285657423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e100fdbb0 0x7f0e10110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:57:45.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.300+0000 7f0e14fba700 1 --2- 172.21.15.47:0/285657423 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0e100fe510 0x7f0e1010a210 secure :-1 s=READY pgs=2860 cs=0 l=1 rev1=1 rx=0x7f0e08001df0 tx=0x7f0e08004c50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:45.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.300+0000 7f0e0d7fa700 10 client.?.objecter ms_handle_connect 0x7f0e100fe510 2024-02-23T00:57:45.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.300+0000 7f0e0d7fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:57:45.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.301+0000 7f0e0d7fa700 1 -- 172.21.15.47:0/285657423 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e08009a10 con 0x7f0e100fe510 2024-02-23T00:57:45.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.301+0000 7f0e0d7fa700 1 -- 172.21.15.47:0/285657423 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f0e080090d0 con 0x7f0e100fe510 2024-02-23T00:57:45.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.301+0000 7f0e0d7fa700 1 -- 172.21.15.47:0/285657423 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e08007be0 con 0x7f0e100fe510 2024-02-23T00:57:45.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.301+0000 7f0e16a1d700 1 -- 172.21.15.47:0/285657423 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0df8005040 con 0x7f0e100fe510 2024-02-23T00:57:45.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.301+0000 7f0e16a1d700 10 client.42359.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:57:45.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.301+0000 7f0e16a1d700 1 -- 172.21.15.47:0/285657423 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e1010aff0 con 0x7f0e100fe510 2024-02-23T00:57:45.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.301+0000 7f0e16a1d700 1 librados: init done 2024-02-23T00:57:45.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.302+0000 7f0e16a1d700 1 -- 172.21.15.47:0/285657423 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e1005ee80 con 0x7f0e100fe510 2024-02-23T00:57:45.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.302+0000 7f0e0d7fa700 1 -- 172.21.15.47:0/285657423 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f0e08009240 con 0x7f0e100fe510 2024-02-23T00:57:45.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.302+0000 7f0e0d7fa700 1 --2- 172.21.15.47:0/285657423 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0dfc062cc0 0x7f0dfc065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:45.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.302+0000 7f0e0d7fa700 1 -- 172.21.15.47:0/285657423 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(856..856 src has 670..856) v4 ==== 8663+0+0 (secure 0 0 0) 0x7f0e0807dd40 con 0x7f0e100fe510 2024-02-23T00:57:45.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.302+0000 7f0e0d7fa700 10 client.42359.objecter ms_dispatch 0x7f0e10001550 osd_map(856..856 src has 670..856) v4 2024-02-23T00:57:45.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.302+0000 7f0e0d7fa700 3 client.42359.objecter handle_osd_map got epochs [856,856] > 0 2024-02-23T00:57:45.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.302+0000 7f0e0d7fa700 3 client.42359.objecter handle_osd_map decoding full epoch 856 2024-02-23T00:57:45.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.302+0000 7f0e0ffff700 1 --2- 172.21.15.47:0/285657423 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0dfc062cc0 0x7f0dfc065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:45.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.303+0000 7f0e0d7fa700 20 client.42359.objecter dump_active .. 0 homeless 2024-02-23T00:57:45.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.303+0000 7f0e0d7fa700 10 client.42359.objecter ms_handle_connect 0x7f0dfc062cc0 2024-02-23T00:57:45.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.303+0000 7f0e0ffff700 1 --2- 172.21.15.47:0/285657423 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0dfc062cc0 0x7f0dfc065180 secure :-1 s=READY pgs=3243 cs=0 l=1 rev1=1 rx=0x7f0e00004dc0 tx=0x7f0e0000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:45.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.307+0000 7f0e0d7fa700 1 -- 172.21.15.47:0/285657423 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f0e0804d600 con 0x7f0e100fe510 2024-02-23T00:57:45.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.490+0000 7f0e16a1d700 1 -- 172.21.15.47:0/285657423 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e1005fa60 con 0x7f0e100fe510 2024-02-23T00:57:45.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.491+0000 7f0e0d7fa700 1 -- 172.21.15.47:0/285657423 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v857) v1 ==== 74+0+24229 (secure 0 0 0) 0x7f0e08050c20 con 0x7f0e100fe510 2024-02-23T00:57:45.494 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:57:45.494 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":857,"fsid":"fce2617e-d1e2-11ee-95c0-87774f69a715","created":"2024-02-23T00:32:55.399437+0000","modified":"2024-02-23T00:57:45.452060+0000","last_up_change":"2024-02-23T00:39:05.662717+0000","last_in_change":"2024-02-23T00:56:21.352185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":110,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":256,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-23T00:36:52.622994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-02-23T00:39:13.277309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"iscsi","create_time":"2024-02-23T00:39:13.761312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-23T00:39:15.255656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"611","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-23T00:39:17.226393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":42,"pg_placement_num":42,"pg_placement_num_target":42,"pg_num_target":42,"pg_num_pending":42,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"825","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"769","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-23T00:39:19.300351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":12,"pg_placement_num":12,"pg_placement_num_target":12,"pg_num_target":12,"pg_num_pending":12,"last_pg_merge_meta":{"source_pgid":"6.c","ready_epoch":403,"last_epoch_started":402,"last_epoch_clean":402,"source_version":"67'1","target_version":"0'0"},"last_change":"822","last_force_op_resend":"0","last_force_op_resend_prenautilus":"403","last_force_op_resend_preluminous":"401","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}},{"pool":18,"pool_name":"ceph_test_rados_api_asio","create_time":"2024-02-23T00:42:52.665618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"380050f9-9001-46dc-bb29-037987b4c8e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":828,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":4042552860},{"type":"v1","addr":"172.21.15.47:6803","nonce":4042552860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":4042552860},{"type":"v1","addr":"172.21.15.47:6805","nonce":4042552860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":4042552860},{"type":"v1","addr":"172.21.15.47:6809","nonce":4042552860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":4042552860},{"type":"v1","addr":"172.21.15.47:6807","nonce":4042552860}]},"public_addr":"172.21.15.47:6803/4042552860","cluster_addr":"172.21.15.47:6805/4042552860","heartbeat_back_addr":"172.21.15.47:6809/4042552860","heartbeat_front_addr":"172.21.15.47:6807/4042552860","state":["exists","up"]},{"osd":1,"uuid":"b06d0cce-55d2-4a6a-a796-4db0b9bb9d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":837,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1945417179},{"type":"v1","addr":"172.21.15.47:6811","nonce":1945417179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1945417179},{"type":"v1","addr":"172.21.15.47:6813","nonce":1945417179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":1945417179},{"type":"v1","addr":"172.21.15.47:6817","nonce":1945417179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1945417179},{"type":"v1","addr":"172.21.15.47:6815","nonce":1945417179}]},"public_addr":"172.21.15.47:6811/1945417179","cluster_addr":"172.21.15.47:6813/1945417179","heartbeat_back_addr":"172.21.15.47:6817/1945417179","heartbeat_front_addr":"172.21.15.47:6815/1945417179","state":["exists","up"]},{"osd":2,"uuid":"6e44d7ca-67a1-4868-bb64-891cf477ebd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":831,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":3596356785},{"type":"v1","addr":"172.21.15.47:6819","nonce":3596356785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":3596356785},{"type":"v1","addr":"172.21.15.47:6821","nonce":3596356785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":3596356785},{"type":"v1","addr":"172.21.15.47:6825","nonce":3596356785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":3596356785},{"type":"v1","addr":"172.21.15.47:6823","nonce":3596356785}]},"public_addr":"172.21.15.47:6819/3596356785","cluster_addr":"172.21.15.47:6821/3596356785","heartbeat_back_addr":"172.21.15.47:6825/3596356785","heartbeat_front_addr":"172.21.15.47:6823/3596356785","state":["exists","up"]},{"osd":3,"uuid":"c88debab-cb0b-46f6-a7d8-c5dc61fc059f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":843,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1195438081},{"type":"v1","addr":"172.21.15.47:6827","nonce":1195438081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1195438081},{"type":"v1","addr":"172.21.15.47:6829","nonce":1195438081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1195438081},{"type":"v1","addr":"172.21.15.47:6833","nonce":1195438081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1195438081},{"type":"v1","addr":"172.21.15.47:6831","nonce":1195438081}]},"public_addr":"172.21.15.47:6827/1195438081","cluster_addr":"172.21.15.47:6829/1195438081","heartbeat_back_addr":"172.21.15.47:6833/1195438081","heartbeat_front_addr":"172.21.15.47:6831/1195438081","state":["exists","up"]},{"osd":4,"uuid":"d17966d7-eb12-4389-ac90-8d15efe290e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":817,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2530629682},{"type":"v1","addr":"172.21.15.196:6801","nonce":2530629682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2530629682},{"type":"v1","addr":"172.21.15.196:6803","nonce":2530629682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2530629682},{"type":"v1","addr":"172.21.15.196:6807","nonce":2530629682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2530629682},{"type":"v1","addr":"172.21.15.196:6805","nonce":2530629682}]},"public_addr":"172.21.15.196:6801/2530629682","cluster_addr":"172.21.15.196:6803/2530629682","heartbeat_back_addr":"172.21.15.196:6807/2530629682","heartbeat_front_addr":"172.21.15.196:6805/2530629682","state":["exists","up"]},{"osd":5,"uuid":"b729d646-4899-47b3-b917-acceba46d241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":817,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1017187491},{"type":"v1","addr":"172.21.15.196:6809","nonce":1017187491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1017187491},{"type":"v1","addr":"172.21.15.196:6811","nonce":1017187491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1017187491},{"type":"v1","addr":"172.21.15.196:6815","nonce":1017187491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1017187491},{"type":"v1","addr":"172.21.15.196:6813","nonce":1017187491}]},"public_addr":"172.21.15.196:6809/1017187491","cluster_addr":"172.21.15.196:6811/1017187491","heartbeat_back_addr":"172.21.15.196:6815/1017187491","heartbeat_front_addr":"172.21.15.196:6813/1017187491","state":["exists","up"]},{"osd":6,"uuid":"df337124-9488-4aad-98e5-3614b51fc757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":817,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3885668599},{"type":"v1","addr":"172.21.15.196:6817","nonce":3885668599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3885668599},{"type":"v1","addr":"172.21.15.196:6819","nonce":3885668599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3885668599},{"type":"v1","addr":"172.21.15.196:6823","nonce":3885668599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3885668599},{"type":"v1","addr":"172.21.15.196:6821","nonce":3885668599}]},"public_addr":"172.21.15.196:6817/3885668599","cluster_addr":"172.21.15.196:6819/3885668599","heartbeat_back_addr":"172.21.15.196:6823/3885668599","heartbeat_front_addr":"172.21.15.196:6821/3885668599","state":["exists","up"]},{"osd":7,"uuid":"84fee1f2-1a88-447a-b1f3-56ff7efe7fec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":817,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2261821675},{"type":"v1","addr":"172.21.15.196:6825","nonce":2261821675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2261821675},{"type":"v1","addr":"172.21.15.196:6827","nonce":2261821675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2261821675},{"type":"v1","addr":"172.21.15.196:6831","nonce":2261821675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2261821675},{"type":"v1","addr":"172.21.15.196:6829","nonce":2261821675}]},"public_addr":"172.21.15.196:6825/2261821675","cluster_addr":"172.21.15.196:6827/2261821675","heartbeat_back_addr":"172.21.15.196:6831/2261821675","heartbeat_front_addr":"172.21.15.196:6829/2261821675","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:57:27.946264+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:57:44.839968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:56:28.307044+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:57:43.557435+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:56:51.029513+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:57:05.806585+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:57:40.526282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-23T00:57:45.222430+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.14","mappings":[{"from":3,"to":4}]},{"pgid":"2.1b","mappings":[{"from":7,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":2}]},{"pgid":"5.6","mappings":[{"from":5,"to":6}]},{"pgid":"5.7","mappings":[{"from":1,"to":6}]},{"pgid":"5.11","mappings":[{"from":4,"to":3}]},{"pgid":"5.12","mappings":[{"from":1,"to":6}]},{"pgid":"5.18","mappings":[{"from":1,"to":3}]},{"pgid":"5.1c","mappings":[{"from":4,"to":6}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6800/622994871":"2024-02-24T00:39:41.110158+0000","172.21.15.47:0/1561174431":"2024-02-24T00:39:41.110158+0000","172.21.15.47:0/3349912606":"2024-02-24T00:39:41.110158+0000","172.21.15.47:0/710432524":"2024-02-24T00:39:41.110158+0000","172.21.15.47:0/2804352395":"2024-02-24T00:33:52.061914+0000","172.21.15.47:6801/1916598887":"2024-02-24T00:33:52.061914+0000","172.21.15.47:6800/1916598887":"2024-02-24T00:33:52.061914+0000","172.21.15.47:0/792211316":"2024-02-24T00:33:27.972945+0000","172.21.15.47:0/381995198":"2024-02-24T00:33:52.061914+0000","172.21.15.47:6800/2409691510":"2024-02-24T00:33:27.972945+0000","172.21.15.47:6801/2409691510":"2024-02-24T00:33:27.972945+0000","172.21.15.47:6801/622994871":"2024-02-24T00:39:41.110158+0000","172.21.15.47:0/1448327041":"2024-02-24T00:33:27.972945+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1}]},{"pool":25,"snaps":[{"begin":9,"length":2}]},{"pool":59,"snaps":[{"begin":2,"length":2},{"begin":6,"length":2}]},{"pool":79,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1}]},{"pool":84,"snaps":[{"begin":2,"length":1}]},{"pool":93,"snaps":[{"begin":4,"length":4}]},{"pool":95,"snaps":[{"begin":6,"length":2}]},{"pool":108,"snaps":[{"begin":2,"length":1}]},{"pool":114,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1}]},{"pool":129,"snaps":[{"begin":2,"length":2},{"begin":6,"length":2},{"begin":10,"length":2}]},{"pool":132,"snaps":[{"begin":12,"length":1}]},{"pool":153,"snaps":[{"begin":14,"length":1}]},{"pool":164,"snaps":[{"begin":17,"length":1}]},{"pool":175,"snaps":[{"begin":20,"length":1}]},{"pool":237,"snaps":[{"begin":2,"length":1}]},{"pool":238,"snaps":[{"begin":4,"length":1}]},{"pool":241,"snaps":[{"begin":6,"length":1}]},{"pool":245,"snaps":[{"begin":9,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-23T00:57:45.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.493+0000 7f0e16a1d700 10 librados: watch_flush enter 2024-02-23T00:57:45.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.493+0000 7f0e16a1d700 10 librados: watch_flush exit 2024-02-23T00:57:45.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.493+0000 7f0e16a1d700 20 client.42359.objecter shutdown clearing up homeless session... 2024-02-23T00:57:45.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.493+0000 7f0e16a1d700 10 client.42359.objecter successfully canceled tick 2024-02-23T00:57:45.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.493+0000 7f0e16a1d700 1 -- 172.21.15.47:0/285657423 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0dfc062cc0 msgr2=0x7f0dfc065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:45.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.493+0000 7f0e16a1d700 1 --2- 172.21.15.47:0/285657423 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0dfc062cc0 0x7f0dfc065180 secure :-1 s=READY pgs=3243 cs=0 l=1 rev1=1 rx=0x7f0e00004dc0 tx=0x7f0e0000b040).stop 2024-02-23T00:57:45.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.493+0000 7f0e16a1d700 1 -- 172.21.15.47:0/285657423 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0e100fe510 msgr2=0x7f0e1010a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:45.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.493+0000 7f0e16a1d700 1 --2- 172.21.15.47:0/285657423 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0e100fe510 0x7f0e1010a210 secure :-1 s=READY pgs=2860 cs=0 l=1 rev1=1 rx=0x7f0e08001df0 tx=0x7f0e08004c50).stop 2024-02-23T00:57:45.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.493+0000 7f0e16a1d700 1 -- 172.21.15.47:0/285657423 shutdown_connections 2024-02-23T00:57:45.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.493+0000 7f0e16a1d700 1 --2- 172.21.15.47:0/285657423 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0dfc062cc0 0x7f0dfc065180 unknown :-1 s=CLOSED pgs=3243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:45.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.493+0000 7f0e16a1d700 1 --2- 172.21.15.47:0/285657423 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0e100fe510 0x7f0e1010a210 unknown :-1 s=CLOSED pgs=2860 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:45.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.493+0000 7f0e16a1d700 1 --2- 172.21.15.47:0/285657423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e100f8420 0x7f0e10110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:45.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.493+0000 7f0e16a1d700 1 --2- 172.21.15.47:0/285657423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e100fdbb0 0x7f0e10110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:45.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.493+0000 7f0e16a1d700 1 -- 172.21.15.47:0/285657423 >> 172.21.15.47:0/285657423 conn(0x7f0e100f3800 msgr2=0x7f0e1006af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:45.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.493+0000 7f0e16a1d700 1 -- 172.21.15.47:0/285657423 shutdown_connections 2024-02-23T00:57:45.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.493+0000 7f0e16a1d700 1 -- 172.21.15.47:0/285657423 wait complete. 2024-02-23T00:57:45.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:45.493+0000 7f0e16a1d700 1 librados: shutdown 2024-02-23T00:57:45.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:44.347118+0000 osd.7 (osd.7) 421 : cluster [DBG] 2.8 scrub starts 2024-02-23T00:57:45.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:44.350382+0000 osd.7 (osd.7) 422 : cluster [DBG] 2.8 scrub ok 2024-02-23T00:57:45.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:44.393390+0000 osd.4 (osd.4) 389 : cluster [DBG] 4.1b scrub starts 2024-02-23T00:57:45.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:44.395099+0000 osd.4 (osd.4) 390 : cluster [DBG] 4.1b scrub ok 2024-02-23T00:57:45.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:44.454207+0000 mon.a (mon.0) 3939 : cluster [DBG] osdmap e856: 8 total, 8 up, 8 in 2024-02-23T00:57:45.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:44.839776+0000 osd.1 (osd.1) 346 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:57:45.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:44.839966+0000 osd.1 (osd.1) 347 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:57:45.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:44.347118+0000 osd.7 (osd.7) 421 : cluster [DBG] 2.8 scrub starts 2024-02-23T00:57:45.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:44.350382+0000 osd.7 (osd.7) 422 : cluster [DBG] 2.8 scrub ok 2024-02-23T00:57:45.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:44.393390+0000 osd.4 (osd.4) 389 : cluster [DBG] 4.1b scrub starts 2024-02-23T00:57:45.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:44.395099+0000 osd.4 (osd.4) 390 : cluster [DBG] 4.1b scrub ok 2024-02-23T00:57:45.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:44.454207+0000 mon.a (mon.0) 3939 : cluster [DBG] osdmap e856: 8 total, 8 up, 8 in 2024-02-23T00:57:45.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:44.839776+0000 osd.1 (osd.1) 346 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:57:45.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:45 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:44.839966+0000 osd.1 (osd.1) 347 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:57:45.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:44.347118+0000 osd.7 (osd.7) 421 : cluster [DBG] 2.8 scrub starts 2024-02-23T00:57:45.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:44.350382+0000 osd.7 (osd.7) 422 : cluster [DBG] 2.8 scrub ok 2024-02-23T00:57:45.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:44.393390+0000 osd.4 (osd.4) 389 : cluster [DBG] 4.1b scrub starts 2024-02-23T00:57:45.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:44.395099+0000 osd.4 (osd.4) 390 : cluster [DBG] 4.1b scrub ok 2024-02-23T00:57:45.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:44.454207+0000 mon.a (mon.0) 3939 : cluster [DBG] osdmap e856: 8 total, 8 up, 8 in 2024-02-23T00:57:45.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:44.839776+0000 osd.1 (osd.1) 346 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:57:45.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:44.839966+0000 osd.1 (osd.1) 347 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:57:46.278 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-23T00:57:46.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:45.222061+0000 osd.7 (osd.7) 423 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:57:46.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:45.222415+0000 osd.7 (osd.7) 424 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:57:46.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:45.401602+0000 osd.4 (osd.4) 391 : cluster [DBG] 5.e scrub starts 2024-02-23T00:57:46.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:45.403070+0000 osd.4 (osd.4) 392 : cluster [DBG] 5.e scrub ok 2024-02-23T00:57:46.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:45.434335+0000 mgr.y (mgr.14517) 743 : cluster [DBG] pgmap v1328: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-23T00:57:46.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:45.460968+0000 mon.a (mon.0) 3940 : cluster [DBG] osdmap e857: 8 total, 8 up, 8 in 2024-02-23T00:57:46.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:45.491677+0000 mon.c (mon.2) 536 : audit [DBG] from='client.? 172.21.15.47:0/285657423' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-23T00:57:46.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:45.880958+0000 osd.3 (osd.3) 353 : cluster [DBG] 4.14 scrub starts 2024-02-23T00:57:46.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:45.883354+0000 osd.3 (osd.3) 354 : cluster [DBG] 4.14 scrub ok 2024-02-23T00:57:46.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:45.222061+0000 osd.7 (osd.7) 423 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:57:46.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:45.222415+0000 osd.7 (osd.7) 424 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:57:46.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:45.401602+0000 osd.4 (osd.4) 391 : cluster [DBG] 5.e scrub starts 2024-02-23T00:57:46.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:45.403070+0000 osd.4 (osd.4) 392 : cluster [DBG] 5.e scrub ok 2024-02-23T00:57:46.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:45.434335+0000 mgr.y (mgr.14517) 743 : cluster [DBG] pgmap v1328: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-23T00:57:46.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:45.460968+0000 mon.a (mon.0) 3940 : cluster [DBG] osdmap e857: 8 total, 8 up, 8 in 2024-02-23T00:57:46.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:45.491677+0000 mon.c (mon.2) 536 : audit [DBG] from='client.? 172.21.15.47:0/285657423' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-23T00:57:46.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:45.880958+0000 osd.3 (osd.3) 353 : cluster [DBG] 4.14 scrub starts 2024-02-23T00:57:46.559 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:45.883354+0000 osd.3 (osd.3) 354 : cluster [DBG] 4.14 scrub ok 2024-02-23T00:57:46.563 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:57:46.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:45.222061+0000 osd.7 (osd.7) 423 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:57:46.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:45.222415+0000 osd.7 (osd.7) 424 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:57:46.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:45.401602+0000 osd.4 (osd.4) 391 : cluster [DBG] 5.e scrub starts 2024-02-23T00:57:46.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:45.403070+0000 osd.4 (osd.4) 392 : cluster [DBG] 5.e scrub ok 2024-02-23T00:57:46.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:45.434335+0000 mgr.y (mgr.14517) 743 : cluster [DBG] pgmap v1328: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-23T00:57:46.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:45.460968+0000 mon.a (mon.0) 3940 : cluster [DBG] osdmap e857: 8 total, 8 up, 8 in 2024-02-23T00:57:46.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:45.491677+0000 mon.c (mon.2) 536 : audit [DBG] from='client.? 172.21.15.47:0/285657423' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-23T00:57:46.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:45.880958+0000 osd.3 (osd.3) 353 : cluster [DBG] 4.14 scrub starts 2024-02-23T00:57:46.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:45.883354+0000 osd.3 (osd.3) 354 : cluster [DBG] 4.14 scrub ok 2024-02-23T00:57:47.768 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:46.281776+0000 osd.7 (osd.7) 425 : cluster [DBG] 18.1f scrub starts 2024-02-23T00:57:47.768 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:46.284432+0000 osd.7 (osd.7) 426 : cluster [DBG] 18.1f scrub ok 2024-02-23T00:57:47.768 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:46.281776+0000 osd.7 (osd.7) 425 : cluster [DBG] 18.1f scrub starts 2024-02-23T00:57:47.768 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:47 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:46.284432+0000 osd.7 (osd.7) 426 : cluster [DBG] 18.1f scrub ok 2024-02-23T00:57:47.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:46.281776+0000 osd.7 (osd.7) 425 : cluster [DBG] 18.1f scrub starts 2024-02-23T00:57:47.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:46.284432+0000 osd.7 (osd.7) 426 : cluster [DBG] 18.1f scrub ok 2024-02-23T00:57:48.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.287+0000 7f4997cc6700 1 -- 172.21.15.47:0/2674756839 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f49900fd510 msgr2=0x7f49900f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:48.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.287+0000 7f4997cc6700 1 --2- 172.21.15.47:0/2674756839 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f49900fd510 0x7f49900f6d40 secure :-1 s=READY pgs=2861 cs=0 l=1 rev1=1 rx=0x7f4984004870 tx=0x7f4984009cb0).stop 2024-02-23T00:57:48.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.287+0000 7f4997cc6700 1 -- 172.21.15.47:0/2674756839 shutdown_connections 2024-02-23T00:57:48.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.287+0000 7f4997cc6700 1 --2- 172.21.15.47:0/2674756839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49900f7400 0x7f4990103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:48.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.287+0000 7f4997cc6700 1 --2- 172.21.15.47:0/2674756839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49900fcbb0 0x7f49900fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:48.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.287+0000 7f4997cc6700 1 --2- 172.21.15.47:0/2674756839 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f49900fd510 0x7f49900f6d40 unknown :-1 s=CLOSED pgs=2861 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:48.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.287+0000 7f4997cc6700 1 -- 172.21.15.47:0/2674756839 >> 172.21.15.47:0/2674756839 conn(0x7f49900f2820 msgr2=0x7f49900f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:48.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.287+0000 7f4997cc6700 1 -- 172.21.15.47:0/2674756839 shutdown_connections 2024-02-23T00:57:48.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.287+0000 7f4997cc6700 1 -- 172.21.15.47:0/2674756839 wait complete. 2024-02-23T00:57:48.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.288+0000 7f4997cc6700 1 librados: starting msgr at 2024-02-23T00:57:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.288+0000 7f4997cc6700 1 librados: starting objecter 2024-02-23T00:57:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.288+0000 7f4997cc6700 1 Processor -- start 2024-02-23T00:57:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.288+0000 7f4997cc6700 1 -- start start 2024-02-23T00:57:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.288+0000 7f4997cc6700 1 librados: setting wanted keys 2024-02-23T00:57:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.288+0000 7f4997cc6700 1 librados: calling monclient init 2024-02-23T00:57:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.288+0000 7f4997cc6700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49900f7400 0x7f49900f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.288+0000 7f4997cc6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49900fcbb0 0x7f49900f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.288+0000 7f4997cc6700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f49900fd510 0x7f49900f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.288+0000 7f4997cc6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49901164b0 con 0x7f49900fcbb0 2024-02-23T00:57:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.288+0000 7f4997cc6700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4990116620 con 0x7f49900f7400 2024-02-23T00:57:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.288+0000 7f4997cc6700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49901167e0 con 0x7f49900fd510 2024-02-23T00:57:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.289+0000 7f4996263700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f49900fd510 0x7f49900f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.289+0000 7f4995261700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49900fcbb0 0x7f49900f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.289+0000 7f4996263700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f49900fd510 0x7f49900f9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:40322/0 (socket says 172.21.15.47:40322) 2024-02-23T00:57:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.289+0000 7f4996263700 1 -- 172.21.15.47:0/3848041133 learned_addr learned my addr 172.21.15.47:0/3848041133 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:57:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.289+0000 7f4995a62700 1 --2- 172.21.15.47:0/3848041133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49900f7400 0x7f49900f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.289+0000 7f4995261700 1 -- 172.21.15.47:0/3848041133 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f49900fd510 msgr2=0x7f49900f9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.289+0000 7f4995261700 1 --2- 172.21.15.47:0/3848041133 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f49900fd510 0x7f49900f9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.289+0000 7f4995261700 1 -- 172.21.15.47:0/3848041133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49900f7400 msgr2=0x7f49900f89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.289+0000 7f4995261700 1 --2- 172.21.15.47:0/3848041133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49900f7400 0x7f49900f89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.289+0000 7f4995261700 1 -- 172.21.15.47:0/3848041133 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4984005040 con 0x7f49900fcbb0 2024-02-23T00:57:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.289+0000 7f4996263700 1 --2- 172.21.15.47:0/3848041133 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f49900fd510 0x7f49900f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:57:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.289+0000 7f4995a62700 1 --2- 172.21.15.47:0/3848041133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49900f7400 0x7f49900f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:57:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.289+0000 7f4995261700 1 --2- 172.21.15.47:0/3848041133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49900fcbb0 0x7f49900f8ea0 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 rx=0x7f4984004710 tx=0x7f4984009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.289+0000 7f4982ffd700 10 client.?.objecter ms_handle_connect 0x7f49900fcbb0 2024-02-23T00:57:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.289+0000 7f4982ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:57:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.289+0000 7f4982ffd700 1 -- 172.21.15.47:0/3848041133 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498400e030 con 0x7f49900fcbb0 2024-02-23T00:57:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.289+0000 7f4982ffd700 1 -- 172.21.15.47:0/3848041133 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f4984013e70 con 0x7f49900fcbb0 2024-02-23T00:57:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.289+0000 7f4997cc6700 1 -- 172.21.15.47:0/3848041133 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4990116a60 con 0x7f49900fcbb0 2024-02-23T00:57:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.289+0000 7f4982ffd700 1 -- 172.21.15.47:0/3848041133 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4984018070 con 0x7f49900fcbb0 2024-02-23T00:57:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.289+0000 7f4997cc6700 10 client.32043.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:57:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.289+0000 7f4997cc6700 1 -- 172.21.15.47:0/3848041133 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4990116e10 con 0x7f49900fcbb0 2024-02-23T00:57:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.289+0000 7f4997cc6700 1 librados: init done 2024-02-23T00:57:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.290+0000 7f4997cc6700 10 client.32043.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:57:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.290+0000 7f4997cc6700 15 client.32043.objecter _session_command_op_assign -1 1 2024-02-23T00:57:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.290+0000 7f4997cc6700 20 client.32043.objecter _get_session osd=-1 returning homeless 2024-02-23T00:57:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.290+0000 7f4997cc6700 10 client.32043.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:57:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.290+0000 7f4997cc6700 1 -- 172.21.15.47:0/3848041133 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f499005fa60 con 0x7f49900fcbb0 2024-02-23T00:57:48.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.291+0000 7f4982ffd700 1 -- 172.21.15.47:0/3848041133 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f4984009240 con 0x7f49900fcbb0 2024-02-23T00:57:48.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.291+0000 7f4982ffd700 1 --2- 172.21.15.47:0/3848041133 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f497c062cc0 0x7f497c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:48.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.291+0000 7f4982ffd700 1 -- 172.21.15.47:0/3848041133 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(857..857 src has 806..857) v4 ==== 8663+0+0 (secure 0 0 0) 0x7f498407ef30 con 0x7f49900fcbb0 2024-02-23T00:57:48.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.291+0000 7f4982ffd700 10 client.32043.objecter ms_dispatch 0x7f4990001550 osd_map(857..857 src has 806..857) v4 2024-02-23T00:57:48.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.291+0000 7f4982ffd700 3 client.32043.objecter handle_osd_map got epochs [857,857] > 0 2024-02-23T00:57:48.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.291+0000 7f4982ffd700 3 client.32043.objecter handle_osd_map decoding full epoch 857 2024-02-23T00:57:48.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.291+0000 7f4995a62700 1 --2- 172.21.15.47:0/3848041133 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f497c062cc0 0x7f497c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:48.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.291+0000 7f4982ffd700 10 client.32043.objecter checking command 1 2024-02-23T00:57:48.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.291+0000 7f4982ffd700 1 --2- 172.21.15.47:0/3848041133 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f497c06e360 0x7f497c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.291+0000 7f4982ffd700 20 client.32043.objecter _get_session s=0x7f497c06dc90 osd=0 3 2024-02-23T00:57:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.291+0000 7f4982ffd700 20 client.32043.objecter put_session s=0x7f497c06dc90 osd=0 3 2024-02-23T00:57:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.291+0000 7f4982ffd700 15 client.32043.objecter _session_command_op_remove -1 1 2024-02-23T00:57:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.291+0000 7f4982ffd700 20 client.32043.objecter _get_session s=0x7f497c06dc90 osd=0 3 2024-02-23T00:57:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.291+0000 7f4982ffd700 20 client.32043.objecter get_session s=0x7f497c06dc90 osd=0 3 2024-02-23T00:57:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.291+0000 7f4982ffd700 15 client.32043.objecter _session_command_op_assign 0 1 2024-02-23T00:57:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.291+0000 7f4982ffd700 20 client.32043.objecter put_session s=0x7f497c06dc90 osd=0 4 2024-02-23T00:57:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.291+0000 7f4982ffd700 10 client.32043.objecter _send_command 1 2024-02-23T00:57:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.291+0000 7f4982ffd700 1 -- 172.21.15.47:0/3848041133 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f497c070e80 con 0x7f497c06e360 2024-02-23T00:57:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.291+0000 7f4982ffd700 20 client.32043.objecter dump_active .. 0 homeless 2024-02-23T00:57:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.291+0000 7f4982ffd700 1 -- 172.21.15.47:0/3848041133 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=857) v2 ==== 24+0+0 (secure 0 0 0) 0x7f498407f2a0 con 0x7f49900fcbb0 2024-02-23T00:57:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.291+0000 7f4996263700 1 --2- 172.21.15.47:0/3848041133 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f497c06e360 0x7f497c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.292+0000 7f4982ffd700 10 client.32043.objecter ms_handle_connect 0x7f497c062cc0 2024-02-23T00:57:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.291+0000 7f4995a62700 1 --2- 172.21.15.47:0/3848041133 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f497c062cc0 0x7f497c065180 secure :-1 s=READY pgs=3244 cs=0 l=1 rev1=1 rx=0x7f498c007fd0 tx=0x7f498c009040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.292+0000 7f4982ffd700 10 client.32043.objecter ms_handle_connect 0x7f497c06e360 2024-02-23T00:57:48.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.292+0000 7f4996263700 1 --2- 172.21.15.47:0/3848041133 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f497c06e360 0x7f497c070800 crc :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:48.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.292+0000 7f4982ffd700 1 -- 172.21.15.47:0/3848041133 <== osd.0 v2:172.21.15.47:6802/4042552860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f497c070e80 con 0x7f497c06e360 2024-02-23T00:57:48.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.292+0000 7f4982ffd700 10 client.32043.objecter ms_dispatch 0x7f4990001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:57:48.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.292+0000 7f4982ffd700 10 client.32043.objecter _finish_command 1 = system:0 2024-02-23T00:57:48.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.292+0000 7f4982ffd700 20 client.32043.objecter put_session s=0x7f497c06dc90 osd=0 4 2024-02-23T00:57:48.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.292+0000 7f4982ffd700 15 client.32043.objecter _session_command_op_remove 0 1 2024-02-23T00:57:48.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.307+0000 7f4997cc6700 10 client.32043.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-02-23T00:57:48.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.307+0000 7f4997cc6700 15 client.32043.objecter _session_command_op_assign -1 2 2024-02-23T00:57:48.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.307+0000 7f4997cc6700 20 client.32043.objecter _get_session s=0x7f497c06dc90 osd=0 3 2024-02-23T00:57:48.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.307+0000 7f4997cc6700 20 client.32043.objecter put_session s=0x7f497c06dc90 osd=0 3 2024-02-23T00:57:48.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.307+0000 7f4997cc6700 20 client.32043.objecter _get_session s=0x7f497c06dc90 osd=0 3 2024-02-23T00:57:48.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.307+0000 7f4997cc6700 15 client.32043.objecter _session_command_op_remove -1 2 2024-02-23T00:57:48.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.307+0000 7f4997cc6700 20 client.32043.objecter get_session s=0x7f497c06dc90 osd=0 3 2024-02-23T00:57:48.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.307+0000 7f4997cc6700 15 client.32043.objecter _session_command_op_assign 0 2 2024-02-23T00:57:48.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.307+0000 7f4997cc6700 20 client.32043.objecter put_session s=0x7f497c06dc90 osd=0 4 2024-02-23T00:57:48.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.307+0000 7f4997cc6700 10 client.32043.objecter _send_command 2 2024-02-23T00:57:48.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.307+0000 7f4997cc6700 1 -- 172.21.15.47:0/3848041133 --> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f499005ee80 con 0x7f497c06e360 2024-02-23T00:57:48.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.307+0000 7f4982ffd700 1 -- 172.21.15.47:0/3848041133 <== osd.0 v2:172.21.15.47:6802/4042552860 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f499005ee80 con 0x7f497c06e360 2024-02-23T00:57:48.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.307+0000 7f4982ffd700 10 client.32043.objecter ms_dispatch 0x7f4990001550 command_reply(tid 2: 0 ) v1 2024-02-23T00:57:48.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.307+0000 7f4982ffd700 10 client.32043.objecter _finish_command 2 = system:0 2024-02-23T00:57:48.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.307+0000 7f4982ffd700 20 client.32043.objecter put_session s=0x7f497c06dc90 osd=0 4 2024-02-23T00:57:48.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.307+0000 7f4982ffd700 15 client.32043.objecter _session_command_op_remove 0 2 2024-02-23T00:57:48.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.308+0000 7f4997cc6700 10 librados: watch_flush enter 2024-02-23T00:57:48.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.308+0000 7f4997cc6700 10 librados: watch_flush exit 2024-02-23T00:57:48.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.308+0000 7f4997cc6700 10 client.32043.objecter close_session for osd.0 2024-02-23T00:57:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.308+0000 7f4997cc6700 1 -- 172.21.15.47:0/3848041133 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f497c06e360 msgr2=0x7f497c070800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.308+0000 7f4997cc6700 1 --2- 172.21.15.47:0/3848041133 >> [v2:172.21.15.47:6802/4042552860,v1:172.21.15.47:6803/4042552860] conn(0x7f497c06e360 0x7f497c070800 crc :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.308+0000 7f4997cc6700 20 client.32043.objecter put_session s=0x7f497c06dc90 osd=0 1 2024-02-23T00:57:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.308+0000 7f4997cc6700 20 client.32043.objecter shutdown clearing up homeless session... 2024-02-23T00:57:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.308+0000 7f4997cc6700 10 client.32043.objecter successfully canceled tick 2024-02-23T00:57:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.308+0000 7f4997cc6700 1 -- 172.21.15.47:0/3848041133 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f497c062cc0 msgr2=0x7f497c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.308+0000 7f4997cc6700 1 --2- 172.21.15.47:0/3848041133 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f497c062cc0 0x7f497c065180 secure :-1 s=READY pgs=3244 cs=0 l=1 rev1=1 rx=0x7f498c007fd0 tx=0x7f498c009040).stop 2024-02-23T00:57:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.308+0000 7f4997cc6700 1 -- 172.21.15.47:0/3848041133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49900fcbb0 msgr2=0x7f49900f8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.308+0000 7f4997cc6700 1 --2- 172.21.15.47:0/3848041133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49900fcbb0 0x7f49900f8ea0 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 rx=0x7f4984004710 tx=0x7f4984009cb0).stop 2024-02-23T00:57:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.308+0000 7f4996263700 1 -- 172.21.15.47:0/3848041133 reap_dead start 2024-02-23T00:57:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.308+0000 7f4997cc6700 1 -- 172.21.15.47:0/3848041133 shutdown_connections 2024-02-23T00:57:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.308+0000 7f4997cc6700 1 -- 172.21.15.47:0/3848041133 >> 172.21.15.47:0/3848041133 conn(0x7f49900f2820 msgr2=0x7f49900f4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.308+0000 7f4997cc6700 1 -- 172.21.15.47:0/3848041133 shutdown_connections 2024-02-23T00:57:48.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.308+0000 7f4997cc6700 1 -- 172.21.15.47:0/3848041133 wait complete. 2024-02-23T00:57:48.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:48.308+0000 7f4997cc6700 1 librados: shutdown 2024-02-23T00:57:48.668 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:47.081460+0000 osd.2 (osd.2) 209 : cluster [DBG] 2.e scrub starts 2024-02-23T00:57:48.668 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:47.082676+0000 osd.2 (osd.2) 210 : cluster [DBG] 2.e scrub ok 2024-02-23T00:57:48.668 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:47.435165+0000 mgr.y (mgr.14517) 744 : cluster [DBG] pgmap v1330: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-23T00:57:48.668 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:47.819826+0000 osd.1 (osd.1) 348 : cluster [DBG] 2.9 scrub starts 2024-02-23T00:57:48.668 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:47.822902+0000 osd.1 (osd.1) 349 : cluster [DBG] 2.9 scrub ok 2024-02-23T00:57:48.668 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:47.853388+0000 osd.3 (osd.3) 355 : cluster [DBG] 6.6 deep-scrub starts 2024-02-23T00:57:48.668 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:47.854624+0000 osd.3 (osd.3) 356 : cluster [DBG] 6.6 deep-scrub ok 2024-02-23T00:57:48.669 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:47.081460+0000 osd.2 (osd.2) 209 : cluster [DBG] 2.e scrub starts 2024-02-23T00:57:48.669 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:47.082676+0000 osd.2 (osd.2) 210 : cluster [DBG] 2.e scrub ok 2024-02-23T00:57:48.669 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:47.435165+0000 mgr.y (mgr.14517) 744 : cluster [DBG] pgmap v1330: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-23T00:57:48.669 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:47.819826+0000 osd.1 (osd.1) 348 : cluster [DBG] 2.9 scrub starts 2024-02-23T00:57:48.669 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:47.822902+0000 osd.1 (osd.1) 349 : cluster [DBG] 2.9 scrub ok 2024-02-23T00:57:48.669 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:47.853388+0000 osd.3 (osd.3) 355 : cluster [DBG] 6.6 deep-scrub starts 2024-02-23T00:57:48.669 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:47.854624+0000 osd.3 (osd.3) 356 : cluster [DBG] 6.6 deep-scrub ok 2024-02-23T00:57:48.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:47.081460+0000 osd.2 (osd.2) 209 : cluster [DBG] 2.e scrub starts 2024-02-23T00:57:48.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:47.082676+0000 osd.2 (osd.2) 210 : cluster [DBG] 2.e scrub ok 2024-02-23T00:57:48.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:47.435165+0000 mgr.y (mgr.14517) 744 : cluster [DBG] pgmap v1330: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-23T00:57:48.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:47.819826+0000 osd.1 (osd.1) 348 : cluster [DBG] 2.9 scrub starts 2024-02-23T00:57:48.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:47.822902+0000 osd.1 (osd.1) 349 : cluster [DBG] 2.9 scrub ok 2024-02-23T00:57:48.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:47.853388+0000 osd.3 (osd.3) 355 : cluster [DBG] 6.6 deep-scrub starts 2024-02-23T00:57:48.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:48 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:47.854624+0000 osd.3 (osd.3) 356 : cluster [DBG] 6.6 deep-scrub ok 2024-02-23T00:57:49.047 INFO:teuthology.orchestra.run.smithi047.stdout:34359738634 2024-02-23T00:57:49.048 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-23T00:57:49.329 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:57:49.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:48.834205+0000 osd.3 (osd.3) 357 : cluster [DBG] 4.1d scrub starts 2024-02-23T00:57:49.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:48.835657+0000 osd.3 (osd.3) 358 : cluster [DBG] 4.1d scrub ok 2024-02-23T00:57:49.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:48.862257+0000 osd.1 (osd.1) 350 : cluster [DBG] 18.5 scrub starts 2024-02-23T00:57:49.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:48.864038+0000 osd.1 (osd.1) 351 : cluster [DBG] 18.5 scrub ok 2024-02-23T00:57:49.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:48.834205+0000 osd.3 (osd.3) 357 : cluster [DBG] 4.1d scrub starts 2024-02-23T00:57:49.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:48.835657+0000 osd.3 (osd.3) 358 : cluster [DBG] 4.1d scrub ok 2024-02-23T00:57:49.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:48.862257+0000 osd.1 (osd.1) 350 : cluster [DBG] 18.5 scrub starts 2024-02-23T00:57:49.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:48.864038+0000 osd.1 (osd.1) 351 : cluster [DBG] 18.5 scrub ok 2024-02-23T00:57:49.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:48.834205+0000 osd.3 (osd.3) 357 : cluster [DBG] 4.1d scrub starts 2024-02-23T00:57:49.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:48.835657+0000 osd.3 (osd.3) 358 : cluster [DBG] 4.1d scrub ok 2024-02-23T00:57:49.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:48.862257+0000 osd.1 (osd.1) 350 : cluster [DBG] 18.5 scrub starts 2024-02-23T00:57:49.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:49 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:48.864038+0000 osd.1 (osd.1) 351 : cluster [DBG] 18.5 scrub ok 2024-02-23T00:57:50.658 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:57:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:57:50] "GET /metrics HTTP/1.1" 200 187134 "" "Prometheus/2.33.4" 2024-02-23T00:57:50.658 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:49.204368+0000 osd.0 (osd.0) 346 : cluster [DBG] 2.1f scrub starts 2024-02-23T00:57:50.658 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:49.205247+0000 osd.0 (osd.0) 347 : cluster [DBG] 2.1f scrub ok 2024-02-23T00:57:50.658 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:49.435554+0000 mgr.y (mgr.14517) 745 : cluster [DBG] pgmap v1331: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:57:50.658 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:49.440128+0000 osd.4 (osd.4) 393 : cluster [DBG] 2.f scrub starts 2024-02-23T00:57:50.658 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:49.441305+0000 osd.4 (osd.4) 394 : cluster [DBG] 2.f scrub ok 2024-02-23T00:57:50.658 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:49.673020+0000 osd.6 (osd.6) 369 : cluster [DBG] 5.1c scrub starts 2024-02-23T00:57:50.658 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:49.674026+0000 osd.6 (osd.6) 370 : cluster [DBG] 5.1c scrub ok 2024-02-23T00:57:50.658 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:49.843654+0000 osd.1 (osd.1) 352 : cluster [DBG] 6.1 scrub starts 2024-02-23T00:57:50.659 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:49.846828+0000 osd.1 (osd.1) 353 : cluster [DBG] 6.1 scrub ok 2024-02-23T00:57:50.659 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:49.204368+0000 osd.0 (osd.0) 346 : cluster [DBG] 2.1f scrub starts 2024-02-23T00:57:50.659 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:49.205247+0000 osd.0 (osd.0) 347 : cluster [DBG] 2.1f scrub ok 2024-02-23T00:57:50.659 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:49.435554+0000 mgr.y (mgr.14517) 745 : cluster [DBG] pgmap v1331: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:57:50.659 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:49.440128+0000 osd.4 (osd.4) 393 : cluster [DBG] 2.f scrub starts 2024-02-23T00:57:50.659 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:49.441305+0000 osd.4 (osd.4) 394 : cluster [DBG] 2.f scrub ok 2024-02-23T00:57:50.659 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:49.673020+0000 osd.6 (osd.6) 369 : cluster [DBG] 5.1c scrub starts 2024-02-23T00:57:50.659 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:49.674026+0000 osd.6 (osd.6) 370 : cluster [DBG] 5.1c scrub ok 2024-02-23T00:57:50.659 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:49.843654+0000 osd.1 (osd.1) 352 : cluster [DBG] 6.1 scrub starts 2024-02-23T00:57:50.659 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:49.846828+0000 osd.1 (osd.1) 353 : cluster [DBG] 6.1 scrub ok 2024-02-23T00:57:50.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:49.204368+0000 osd.0 (osd.0) 346 : cluster [DBG] 2.1f scrub starts 2024-02-23T00:57:50.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:49.205247+0000 osd.0 (osd.0) 347 : cluster [DBG] 2.1f scrub ok 2024-02-23T00:57:50.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:49.435554+0000 mgr.y (mgr.14517) 745 : cluster [DBG] pgmap v1331: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:57:50.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:49.440128+0000 osd.4 (osd.4) 393 : cluster [DBG] 2.f scrub starts 2024-02-23T00:57:50.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:49.441305+0000 osd.4 (osd.4) 394 : cluster [DBG] 2.f scrub ok 2024-02-23T00:57:50.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:49.673020+0000 osd.6 (osd.6) 369 : cluster [DBG] 5.1c scrub starts 2024-02-23T00:57:50.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:49.674026+0000 osd.6 (osd.6) 370 : cluster [DBG] 5.1c scrub ok 2024-02-23T00:57:50.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:49.843654+0000 osd.1 (osd.1) 352 : cluster [DBG] 6.1 scrub starts 2024-02-23T00:57:50.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:50 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:49.846828+0000 osd.1 (osd.1) 353 : cluster [DBG] 6.1 scrub ok 2024-02-23T00:57:51.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.143+0000 7ff3436e7700 1 -- 172.21.15.47:0/36239178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff33c0f7400 msgr2=0x7ff33c103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:51.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.143+0000 7ff3436e7700 1 --2- 172.21.15.47:0/36239178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff33c0f7400 0x7ff33c103a20 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 rx=0x7ff334009be0 tx=0x7ff334004d10).stop 2024-02-23T00:57:51.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.143+0000 7ff3436e7700 1 -- 172.21.15.47:0/36239178 shutdown_connections 2024-02-23T00:57:51.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.143+0000 7ff3436e7700 1 --2- 172.21.15.47:0/36239178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff33c0f7400 0x7ff33c103a20 unknown :-1 s=CLOSED pgs=2677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:51.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.143+0000 7ff3436e7700 1 --2- 172.21.15.47:0/36239178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff33c0fcbb0 0x7ff33c0fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:51.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.143+0000 7ff3436e7700 1 --2- 172.21.15.47:0/36239178 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff33c0fd510 0x7ff33c0f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:51.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.143+0000 7ff3436e7700 1 -- 172.21.15.47:0/36239178 >> 172.21.15.47:0/36239178 conn(0x7ff33c0f2820 msgr2=0x7ff33c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:51.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.143+0000 7ff3436e7700 1 -- 172.21.15.47:0/36239178 shutdown_connections 2024-02-23T00:57:51.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.143+0000 7ff3436e7700 1 -- 172.21.15.47:0/36239178 wait complete. 2024-02-23T00:57:51.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.143+0000 7ff3436e7700 1 librados: starting msgr at 2024-02-23T00:57:51.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.143+0000 7ff3436e7700 1 librados: starting objecter 2024-02-23T00:57:51.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.144+0000 7ff3436e7700 1 Processor -- start 2024-02-23T00:57:51.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.144+0000 7ff3436e7700 1 -- start start 2024-02-23T00:57:51.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.144+0000 7ff3436e7700 1 librados: setting wanted keys 2024-02-23T00:57:51.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.144+0000 7ff3436e7700 1 librados: calling monclient init 2024-02-23T00:57:51.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.144+0000 7ff3436e7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff33c0f7400 0x7ff33c06aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:51.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.144+0000 7ff3436e7700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff33c0fcbb0 0x7ff33c064c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:51.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.144+0000 7ff3436e7700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff33c0fd510 0x7ff33c065340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:51.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.144+0000 7ff3436e7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c065a30 con 0x7ff33c0f7400 2024-02-23T00:57:51.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.144+0000 7ff3436e7700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c11cba0 con 0x7ff33c0fcbb0 2024-02-23T00:57:51.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.144+0000 7ff3436e7700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c11cce0 con 0x7ff33c0fd510 2024-02-23T00:57:51.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.144+0000 7ff341c84700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff33c0fd510 0x7ff33c065340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:51.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.144+0000 7ff341c84700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff33c0fd510 0x7ff33c065340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:40328/0 (socket says 172.21.15.47:40328) 2024-02-23T00:57:51.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.144+0000 7ff341c84700 1 -- 172.21.15.47:0/2604296237 learned_addr learned my addr 172.21.15.47:0/2604296237 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:57:51.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.144+0000 7ff340c82700 1 --2- 172.21.15.47:0/2604296237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff33c0fcbb0 0x7ff33c064c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:51.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.144+0000 7ff341483700 1 --2- 172.21.15.47:0/2604296237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff33c0f7400 0x7ff33c06aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:51.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.144+0000 7ff341c84700 1 -- 172.21.15.47:0/2604296237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff33c0fcbb0 msgr2=0x7ff33c064c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:51.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.144+0000 7ff341c84700 1 --2- 172.21.15.47:0/2604296237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff33c0fcbb0 0x7ff33c064c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:51.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.144+0000 7ff341c84700 1 -- 172.21.15.47:0/2604296237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff33c0f7400 msgr2=0x7ff33c06aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:51.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.144+0000 7ff341c84700 1 --2- 172.21.15.47:0/2604296237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff33c0f7400 0x7ff33c06aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:51.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.144+0000 7ff341c84700 1 -- 172.21.15.47:0/2604296237 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff338005040 con 0x7ff33c0fd510 2024-02-23T00:57:51.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.144+0000 7ff341c84700 1 --2- 172.21.15.47:0/2604296237 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff33c0fd510 0x7ff33c065340 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 rx=0x7ff334009790 tx=0x7ff334004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:51.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.144+0000 7ff32e7fc700 10 client.?.objecter ms_handle_connect 0x7ff33c0fd510 2024-02-23T00:57:51.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.144+0000 7ff32e7fc700 10 client.?.objecter resend_mon_ops 2024-02-23T00:57:51.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.145+0000 7ff32e7fc700 1 -- 172.21.15.47:0/2604296237 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33400e040 con 0x7ff33c0fd510 2024-02-23T00:57:51.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.145+0000 7ff32e7fc700 1 -- 172.21.15.47:0/2604296237 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7ff334018040 con 0x7ff33c0fd510 2024-02-23T00:57:51.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.145+0000 7ff3436e7700 1 -- 172.21.15.47:0/2604296237 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3340097c0 con 0x7ff33c0fd510 2024-02-23T00:57:51.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.145+0000 7ff32e7fc700 1 -- 172.21.15.47:0/2604296237 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff334013c30 con 0x7ff33c0fd510 2024-02-23T00:57:51.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.145+0000 7ff3436e7700 10 client.42371.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:57:51.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.145+0000 7ff3436e7700 1 -- 172.21.15.47:0/2604296237 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff33c11d180 con 0x7ff33c0fd510 2024-02-23T00:57:51.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.145+0000 7ff3436e7700 1 librados: init done 2024-02-23T00:57:51.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.145+0000 7ff3436e7700 10 client.42371.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:57:51.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.145+0000 7ff3436e7700 15 client.42371.objecter _session_command_op_assign -1 1 2024-02-23T00:57:51.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.145+0000 7ff3436e7700 20 client.42371.objecter _get_session osd=-1 returning homeless 2024-02-23T00:57:51.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.145+0000 7ff3436e7700 10 client.42371.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:57:51.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.145+0000 7ff3436e7700 1 -- 172.21.15.47:0/2604296237 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff33c05fa60 con 0x7ff33c0fd510 2024-02-23T00:57:51.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.146+0000 7ff32e7fc700 1 -- 172.21.15.47:0/2604296237 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7ff334013420 con 0x7ff33c0fd510 2024-02-23T00:57:51.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.146+0000 7ff32e7fc700 1 --2- 172.21.15.47:0/2604296237 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff328062cc0 0x7ff328065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:51.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.146+0000 7ff32e7fc700 1 -- 172.21.15.47:0/2604296237 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(857..857 src has 806..857) v4 ==== 8663+0+0 (secure 0 0 0) 0x7ff33407ef20 con 0x7ff33c0fd510 2024-02-23T00:57:51.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.146+0000 7ff32e7fc700 10 client.42371.objecter ms_dispatch 0x7ff33c001550 osd_map(857..857 src has 806..857) v4 2024-02-23T00:57:51.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.146+0000 7ff32e7fc700 3 client.42371.objecter handle_osd_map got epochs [857,857] > 0 2024-02-23T00:57:51.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.146+0000 7ff32e7fc700 3 client.42371.objecter handle_osd_map decoding full epoch 857 2024-02-23T00:57:51.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.146+0000 7ff341483700 1 --2- 172.21.15.47:0/2604296237 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff328062cc0 0x7ff328065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:51.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.147+0000 7ff32e7fc700 10 client.42371.objecter checking command 1 2024-02-23T00:57:51.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.147+0000 7ff32e7fc700 1 --2- 172.21.15.47:0/2604296237 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7ff32806e360 0x7ff328070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:51.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.147+0000 7ff32e7fc700 20 client.42371.objecter _get_session s=0x7ff32806dc90 osd=1 3 2024-02-23T00:57:51.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.147+0000 7ff32e7fc700 20 client.42371.objecter put_session s=0x7ff32806dc90 osd=1 3 2024-02-23T00:57:51.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.147+0000 7ff32e7fc700 15 client.42371.objecter _session_command_op_remove -1 1 2024-02-23T00:57:51.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.147+0000 7ff32e7fc700 20 client.42371.objecter _get_session s=0x7ff32806dc90 osd=1 3 2024-02-23T00:57:51.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.147+0000 7ff32e7fc700 20 client.42371.objecter get_session s=0x7ff32806dc90 osd=1 3 2024-02-23T00:57:51.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.147+0000 7ff32e7fc700 15 client.42371.objecter _session_command_op_assign 1 1 2024-02-23T00:57:51.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.147+0000 7ff32e7fc700 20 client.42371.objecter put_session s=0x7ff32806dc90 osd=1 4 2024-02-23T00:57:51.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.147+0000 7ff32e7fc700 10 client.42371.objecter _send_command 1 2024-02-23T00:57:51.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.147+0000 7ff32e7fc700 1 -- 172.21.15.47:0/2604296237 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff328070e80 con 0x7ff32806e360 2024-02-23T00:57:51.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.147+0000 7ff32e7fc700 20 client.42371.objecter dump_active .. 0 homeless 2024-02-23T00:57:51.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.147+0000 7ff32e7fc700 1 -- 172.21.15.47:0/2604296237 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_get_version_reply(handle=1 version=857) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff334081500 con 0x7ff33c0fd510 2024-02-23T00:57:51.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.147+0000 7ff340c82700 1 --2- 172.21.15.47:0/2604296237 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7ff32806e360 0x7ff328070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:51.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.147+0000 7ff32e7fc700 10 client.42371.objecter ms_handle_connect 0x7ff328062cc0 2024-02-23T00:57:51.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.147+0000 7ff341483700 1 --2- 172.21.15.47:0/2604296237 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff328062cc0 0x7ff328065180 secure :-1 s=READY pgs=3245 cs=0 l=1 rev1=1 rx=0x7ff3300048b0 tx=0x7ff33000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:51.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.147+0000 7ff32e7fc700 10 client.42371.objecter ms_handle_connect 0x7ff32806e360 2024-02-23T00:57:51.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.147+0000 7ff340c82700 1 --2- 172.21.15.47:0/2604296237 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7ff32806e360 0x7ff328070800 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:51.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.148+0000 7ff32e7fc700 1 -- 172.21.15.47:0/2604296237 <== osd.1 v2:172.21.15.47:6810/1945417179 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff328070e80 con 0x7ff32806e360 2024-02-23T00:57:51.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.148+0000 7ff32e7fc700 10 client.42371.objecter ms_dispatch 0x7ff33c001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:57:51.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.148+0000 7ff32e7fc700 10 client.42371.objecter _finish_command 1 = system:0 2024-02-23T00:57:51.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.148+0000 7ff32e7fc700 20 client.42371.objecter put_session s=0x7ff32806dc90 osd=1 4 2024-02-23T00:57:51.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.148+0000 7ff32e7fc700 15 client.42371.objecter _session_command_op_remove 1 1 2024-02-23T00:57:51.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.162+0000 7ff3436e7700 10 client.42371.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-02-23T00:57:51.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.162+0000 7ff3436e7700 15 client.42371.objecter _session_command_op_assign -1 2 2024-02-23T00:57:51.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.162+0000 7ff3436e7700 20 client.42371.objecter _get_session s=0x7ff32806dc90 osd=1 3 2024-02-23T00:57:51.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.162+0000 7ff3436e7700 20 client.42371.objecter put_session s=0x7ff32806dc90 osd=1 3 2024-02-23T00:57:51.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.162+0000 7ff3436e7700 20 client.42371.objecter _get_session s=0x7ff32806dc90 osd=1 3 2024-02-23T00:57:51.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.162+0000 7ff3436e7700 15 client.42371.objecter _session_command_op_remove -1 2 2024-02-23T00:57:51.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.162+0000 7ff3436e7700 20 client.42371.objecter get_session s=0x7ff32806dc90 osd=1 3 2024-02-23T00:57:51.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.162+0000 7ff3436e7700 15 client.42371.objecter _session_command_op_assign 1 2 2024-02-23T00:57:51.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.162+0000 7ff3436e7700 20 client.42371.objecter put_session s=0x7ff32806dc90 osd=1 4 2024-02-23T00:57:51.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.162+0000 7ff3436e7700 10 client.42371.objecter _send_command 2 2024-02-23T00:57:51.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.162+0000 7ff3436e7700 1 -- 172.21.15.47:0/2604296237 --> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff33c05ee80 con 0x7ff32806e360 2024-02-23T00:57:51.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.162+0000 7ff32e7fc700 1 -- 172.21.15.47:0/2604296237 <== osd.1 v2:172.21.15.47:6810/1945417179 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff33c05ee80 con 0x7ff32806e360 2024-02-23T00:57:51.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.162+0000 7ff32e7fc700 10 client.42371.objecter ms_dispatch 0x7ff33c001550 command_reply(tid 2: 0 ) v1 2024-02-23T00:57:51.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.162+0000 7ff32e7fc700 10 client.42371.objecter _finish_command 2 = system:0 2024-02-23T00:57:51.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.162+0000 7ff32e7fc700 20 client.42371.objecter put_session s=0x7ff32806dc90 osd=1 4 2024-02-23T00:57:51.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.162+0000 7ff32e7fc700 15 client.42371.objecter _session_command_op_remove 1 2 2024-02-23T00:57:51.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.163+0000 7ff3436e7700 10 librados: watch_flush enter 2024-02-23T00:57:51.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.163+0000 7ff3436e7700 10 librados: watch_flush exit 2024-02-23T00:57:51.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.163+0000 7ff3436e7700 10 client.42371.objecter close_session for osd.1 2024-02-23T00:57:51.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.163+0000 7ff3436e7700 1 -- 172.21.15.47:0/2604296237 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7ff32806e360 msgr2=0x7ff328070800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:51.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.163+0000 7ff3436e7700 1 --2- 172.21.15.47:0/2604296237 >> [v2:172.21.15.47:6810/1945417179,v1:172.21.15.47:6811/1945417179] conn(0x7ff32806e360 0x7ff328070800 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:51.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.163+0000 7ff3436e7700 20 client.42371.objecter put_session s=0x7ff32806dc90 osd=1 1 2024-02-23T00:57:51.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.163+0000 7ff3436e7700 20 client.42371.objecter shutdown clearing up homeless session... 2024-02-23T00:57:51.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.163+0000 7ff3436e7700 10 client.42371.objecter successfully canceled tick 2024-02-23T00:57:51.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.163+0000 7ff3436e7700 1 -- 172.21.15.47:0/2604296237 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff328062cc0 msgr2=0x7ff328065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:51.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.163+0000 7ff3436e7700 1 --2- 172.21.15.47:0/2604296237 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7ff328062cc0 0x7ff328065180 secure :-1 s=READY pgs=3245 cs=0 l=1 rev1=1 rx=0x7ff3300048b0 tx=0x7ff33000b040).stop 2024-02-23T00:57:51.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.163+0000 7ff3436e7700 1 -- 172.21.15.47:0/2604296237 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff33c0fd510 msgr2=0x7ff33c065340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:51.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.163+0000 7ff3436e7700 1 --2- 172.21.15.47:0/2604296237 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff33c0fd510 0x7ff33c065340 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 rx=0x7ff334009790 tx=0x7ff334004d10).stop 2024-02-23T00:57:51.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.163+0000 7ff341c84700 1 -- 172.21.15.47:0/2604296237 reap_dead start 2024-02-23T00:57:51.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.163+0000 7ff3436e7700 1 -- 172.21.15.47:0/2604296237 shutdown_connections 2024-02-23T00:57:51.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.163+0000 7ff3436e7700 1 -- 172.21.15.47:0/2604296237 >> 172.21.15.47:0/2604296237 conn(0x7ff33c0f2820 msgr2=0x7ff33c060db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:51.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.163+0000 7ff3436e7700 1 -- 172.21.15.47:0/2604296237 shutdown_connections 2024-02-23T00:57:51.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.163+0000 7ff3436e7700 1 -- 172.21.15.47:0/2604296237 wait complete. 2024-02-23T00:57:51.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:51.163+0000 7ff3436e7700 1 librados: shutdown 2024-02-23T00:57:51.920 INFO:teuthology.orchestra.run.smithi047.stdout:55834575108 2024-02-23T00:57:51.921 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-23T00:57:52.205 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:57:52.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:51.142954+0000 osd.2 (osd.2) 211 : cluster [DBG] 5.8 scrub starts 2024-02-23T00:57:52.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:51.143987+0000 osd.2 (osd.2) 212 : cluster [DBG] 5.8 scrub ok 2024-02-23T00:57:52.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:51.343729+0000 osd.2 (osd.2) 213 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:57:52.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:51.343966+0000 osd.2 (osd.2) 214 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:57:52.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:51.435986+0000 mgr.y (mgr.14517) 746 : cluster [DBG] pgmap v1332: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-23T00:57:52.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:51.479174+0000 mon.a (mon.0) 3941 : cluster [DBG] osdmap e858: 8 total, 8 up, 8 in 2024-02-23T00:57:52.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:51.837173+0000 osd.1 (osd.1) 354 : cluster [DBG] 18.11 scrub starts 2024-02-23T00:57:52.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:51.840134+0000 osd.1 (osd.1) 355 : cluster [DBG] 18.11 scrub ok 2024-02-23T00:57:52.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:51.142954+0000 osd.2 (osd.2) 211 : cluster [DBG] 5.8 scrub starts 2024-02-23T00:57:52.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:51.143987+0000 osd.2 (osd.2) 212 : cluster [DBG] 5.8 scrub ok 2024-02-23T00:57:52.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:51.343729+0000 osd.2 (osd.2) 213 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:57:52.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:51.343966+0000 osd.2 (osd.2) 214 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:57:52.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:51.435986+0000 mgr.y (mgr.14517) 746 : cluster [DBG] pgmap v1332: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-23T00:57:52.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:51.479174+0000 mon.a (mon.0) 3941 : cluster [DBG] osdmap e858: 8 total, 8 up, 8 in 2024-02-23T00:57:52.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:51.837173+0000 osd.1 (osd.1) 354 : cluster [DBG] 18.11 scrub starts 2024-02-23T00:57:52.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:52 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:51.840134+0000 osd.1 (osd.1) 355 : cluster [DBG] 18.11 scrub ok 2024-02-23T00:57:52.944 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:57:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:57:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:57:52.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:51.142954+0000 osd.2 (osd.2) 211 : cluster [DBG] 5.8 scrub starts 2024-02-23T00:57:52.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:51.143987+0000 osd.2 (osd.2) 212 : cluster [DBG] 5.8 scrub ok 2024-02-23T00:57:52.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:51.343729+0000 osd.2 (osd.2) 213 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:57:52.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:51.343966+0000 osd.2 (osd.2) 214 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:57:52.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:51.435986+0000 mgr.y (mgr.14517) 746 : cluster [DBG] pgmap v1332: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-23T00:57:52.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:51.479174+0000 mon.a (mon.0) 3941 : cluster [DBG] osdmap e858: 8 total, 8 up, 8 in 2024-02-23T00:57:52.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:51.837173+0000 osd.1 (osd.1) 354 : cluster [DBG] 18.11 scrub starts 2024-02-23T00:57:52.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:52 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:51.840134+0000 osd.1 (osd.1) 355 : cluster [DBG] 18.11 scrub ok 2024-02-23T00:57:53.444 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:57:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:57:53.688 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:52.165082+0000 osd.2 (osd.2) 215 : cluster [DBG] 18.12 scrub starts 2024-02-23T00:57:53.688 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:52.166250+0000 osd.2 (osd.2) 216 : cluster [DBG] 18.12 scrub ok 2024-02-23T00:57:53.688 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:52.208406+0000 osd.0 (osd.0) 348 : cluster [DBG] 18.1d scrub starts 2024-02-23T00:57:53.688 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:52.210572+0000 osd.0 (osd.0) 349 : cluster [DBG] 18.1d scrub ok 2024-02-23T00:57:53.688 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:52.780570+0000 osd.3 (osd.3) 359 : cluster [DBG] 4.0 scrub starts 2024-02-23T00:57:53.688 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:52.784213+0000 osd.3 (osd.3) 360 : cluster [DBG] 4.0 scrub ok 2024-02-23T00:57:53.688 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:53.187555+0000 osd.5 (osd.5) 410 : cluster [DBG] 5.f deep-scrub starts 2024-02-23T00:57:53.688 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:53.190536+0000 osd.5 (osd.5) 411 : cluster [DBG] 5.f deep-scrub ok 2024-02-23T00:57:53.688 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:52.165082+0000 osd.2 (osd.2) 215 : cluster [DBG] 18.12 scrub starts 2024-02-23T00:57:53.689 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:52.166250+0000 osd.2 (osd.2) 216 : cluster [DBG] 18.12 scrub ok 2024-02-23T00:57:53.689 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:52.208406+0000 osd.0 (osd.0) 348 : cluster [DBG] 18.1d scrub starts 2024-02-23T00:57:53.689 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:52.210572+0000 osd.0 (osd.0) 349 : cluster [DBG] 18.1d scrub ok 2024-02-23T00:57:53.689 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:52.780570+0000 osd.3 (osd.3) 359 : cluster [DBG] 4.0 scrub starts 2024-02-23T00:57:53.689 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:52.784213+0000 osd.3 (osd.3) 360 : cluster [DBG] 4.0 scrub ok 2024-02-23T00:57:53.689 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:53.187555+0000 osd.5 (osd.5) 410 : cluster [DBG] 5.f deep-scrub starts 2024-02-23T00:57:53.689 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:53.190536+0000 osd.5 (osd.5) 411 : cluster [DBG] 5.f deep-scrub ok 2024-02-23T00:57:53.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:52.165082+0000 osd.2 (osd.2) 215 : cluster [DBG] 18.12 scrub starts 2024-02-23T00:57:53.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:52.166250+0000 osd.2 (osd.2) 216 : cluster [DBG] 18.12 scrub ok 2024-02-23T00:57:53.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:52.208406+0000 osd.0 (osd.0) 348 : cluster [DBG] 18.1d scrub starts 2024-02-23T00:57:53.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:52.210572+0000 osd.0 (osd.0) 349 : cluster [DBG] 18.1d scrub ok 2024-02-23T00:57:53.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:52.780570+0000 osd.3 (osd.3) 359 : cluster [DBG] 4.0 scrub starts 2024-02-23T00:57:53.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:52.784213+0000 osd.3 (osd.3) 360 : cluster [DBG] 4.0 scrub ok 2024-02-23T00:57:53.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:53.187555+0000 osd.5 (osd.5) 410 : cluster [DBG] 5.f deep-scrub starts 2024-02-23T00:57:53.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:53 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:53.190536+0000 osd.5 (osd.5) 411 : cluster [DBG] 5.f deep-scrub ok 2024-02-23T00:57:53.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.971+0000 7fa29d95f700 1 -- 172.21.15.47:0/1922393334 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa2980fcbb0 msgr2=0x7fa2980fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:53.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.971+0000 7fa29d95f700 1 --2- 172.21.15.47:0/1922393334 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa2980fcbb0 0x7fa2980fcf90 secure :-1 s=READY pgs=2863 cs=0 l=1 rev1=1 rx=0x7fa28c004870 tx=0x7fa28c009cb0).stop 2024-02-23T00:57:53.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.971+0000 7fa29d95f700 1 -- 172.21.15.47:0/1922393334 shutdown_connections 2024-02-23T00:57:53.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.971+0000 7fa29d95f700 1 --2- 172.21.15.47:0/1922393334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2980f7400 0x7fa298103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:53.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.971+0000 7fa29d95f700 1 --2- 172.21.15.47:0/1922393334 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa2980fcbb0 0x7fa2980fcf90 unknown :-1 s=CLOSED pgs=2863 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:53.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.971+0000 7fa29d95f700 1 --2- 172.21.15.47:0/1922393334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2980fd510 0x7fa2980f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:53.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.971+0000 7fa29d95f700 1 -- 172.21.15.47:0/1922393334 >> 172.21.15.47:0/1922393334 conn(0x7fa2980f2820 msgr2=0x7fa2980f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:53.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.971+0000 7fa29d95f700 1 -- 172.21.15.47:0/1922393334 shutdown_connections 2024-02-23T00:57:53.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.971+0000 7fa29d95f700 1 -- 172.21.15.47:0/1922393334 wait complete. 2024-02-23T00:57:53.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.972+0000 7fa29d95f700 1 librados: starting msgr at 2024-02-23T00:57:53.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.972+0000 7fa29d95f700 1 librados: starting objecter 2024-02-23T00:57:53.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.972+0000 7fa29d95f700 1 Processor -- start 2024-02-23T00:57:53.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.972+0000 7fa29d95f700 1 -- start start 2024-02-23T00:57:53.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.972+0000 7fa29d95f700 1 librados: setting wanted keys 2024-02-23T00:57:53.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.972+0000 7fa29d95f700 1 librados: calling monclient init 2024-02-23T00:57:53.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.972+0000 7fa29d95f700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa2980f7400 0x7fa2980f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:53.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.972+0000 7fa29d95f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2980fcbb0 0x7fa2980f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:53.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.972+0000 7fa29d95f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2980fd510 0x7fa2980f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:53.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.972+0000 7fa29d95f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa298116540 con 0x7fa2980fcbb0 2024-02-23T00:57:53.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.972+0000 7fa29d95f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2981166b0 con 0x7fa2980fd510 2024-02-23T00:57:53.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.972+0000 7fa29d95f700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa298116870 con 0x7fa2980f7400 2024-02-23T00:57:53.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.972+0000 7fa296ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa2980f7400 0x7fa2980f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:53.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.973+0000 7fa296ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa2980f7400 0x7fa2980f89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:40344/0 (socket says 172.21.15.47:40344) 2024-02-23T00:57:53.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.973+0000 7fa2967fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2980fcbb0 0x7fa2980f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:53.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.973+0000 7fa296ffd700 1 -- 172.21.15.47:0/4223243289 learned_addr learned my addr 172.21.15.47:0/4223243289 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:57:53.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.973+0000 7fa2977fe700 1 --2- 172.21.15.47:0/4223243289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2980fd510 0x7fa2980f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:53.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.973+0000 7fa296ffd700 1 -- 172.21.15.47:0/4223243289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2980fd510 msgr2=0x7fa2980f9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:53.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.973+0000 7fa296ffd700 1 --2- 172.21.15.47:0/4223243289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2980fd510 0x7fa2980f9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:53.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.973+0000 7fa296ffd700 1 -- 172.21.15.47:0/4223243289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2980fcbb0 msgr2=0x7fa2980f8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:53.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.973+0000 7fa296ffd700 1 --2- 172.21.15.47:0/4223243289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2980fcbb0 0x7fa2980f8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.973+0000 7fa296ffd700 1 -- 172.21.15.47:0/4223243289 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa288005040 con 0x7fa2980f7400 2024-02-23T00:57:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.973+0000 7fa2967fc700 1 --2- 172.21.15.47:0/4223243289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2980fcbb0 0x7fa2980f8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:57:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.973+0000 7fa296ffd700 1 --2- 172.21.15.47:0/4223243289 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa2980f7400 0x7fa2980f89b0 secure :-1 s=READY pgs=2864 cs=0 l=1 rev1=1 rx=0x7fa28c009da0 tx=0x7fa28c009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.973+0000 7fa27ffff700 10 client.?.objecter ms_handle_connect 0x7fa2980f7400 2024-02-23T00:57:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.973+0000 7fa27ffff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:57:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.973+0000 7fa27ffff700 1 -- 172.21.15.47:0/4223243289 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28c00e030 con 0x7fa2980f7400 2024-02-23T00:57:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.973+0000 7fa27ffff700 1 -- 172.21.15.47:0/4223243289 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fa28c013e70 con 0x7fa2980f7400 2024-02-23T00:57:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.973+0000 7fa29d95f700 1 -- 172.21.15.47:0/4223243289 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa28c005040 con 0x7fa2980f7400 2024-02-23T00:57:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.973+0000 7fa27ffff700 1 -- 172.21.15.47:0/4223243289 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28c018070 con 0x7fa2980f7400 2024-02-23T00:57:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.973+0000 7fa29d95f700 10 client.42377.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:57:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.973+0000 7fa29d95f700 1 -- 172.21.15.47:0/4223243289 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa298116d60 con 0x7fa2980f7400 2024-02-23T00:57:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.973+0000 7fa29d95f700 1 librados: init done 2024-02-23T00:57:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.974+0000 7fa29d95f700 10 client.42377.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:57:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.974+0000 7fa29d95f700 15 client.42377.objecter _session_command_op_assign -1 1 2024-02-23T00:57:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.974+0000 7fa29d95f700 20 client.42377.objecter _get_session osd=-1 returning homeless 2024-02-23T00:57:53.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.974+0000 7fa29d95f700 10 client.42377.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:57:53.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.974+0000 7fa29d95f700 1 -- 172.21.15.47:0/4223243289 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa29805fa60 con 0x7fa2980f7400 2024-02-23T00:57:53.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.974+0000 7fa27ffff700 1 -- 172.21.15.47:0/4223243289 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fa28c009240 con 0x7fa2980f7400 2024-02-23T00:57:53.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.975+0000 7fa27ffff700 1 --2- 172.21.15.47:0/4223243289 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa280062cc0 0x7fa280065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:53.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.975+0000 7fa27ffff700 1 -- 172.21.15.47:0/4223243289 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(858..858 src has 806..858) v4 ==== 8663+0+0 (secure 0 0 0) 0x7fa28c07e5f0 con 0x7fa2980f7400 2024-02-23T00:57:53.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.975+0000 7fa27ffff700 10 client.42377.objecter ms_dispatch 0x7fa298001550 osd_map(858..858 src has 806..858) v4 2024-02-23T00:57:53.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.975+0000 7fa27ffff700 3 client.42377.objecter handle_osd_map got epochs [858,858] > 0 2024-02-23T00:57:53.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.975+0000 7fa27ffff700 3 client.42377.objecter handle_osd_map decoding full epoch 858 2024-02-23T00:57:53.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.975+0000 7fa2967fc700 1 --2- 172.21.15.47:0/4223243289 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa280062cc0 0x7fa280065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:53.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.975+0000 7fa27ffff700 10 client.42377.objecter checking command 1 2024-02-23T00:57:53.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.975+0000 7fa27ffff700 1 --2- 172.21.15.47:0/4223243289 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa28006e3e0 0x7fa280070880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:53.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.975+0000 7fa27ffff700 20 client.42377.objecter _get_session s=0x7fa28006dd10 osd=2 3 2024-02-23T00:57:53.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.975+0000 7fa27ffff700 20 client.42377.objecter put_session s=0x7fa28006dd10 osd=2 3 2024-02-23T00:57:53.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.975+0000 7fa27ffff700 15 client.42377.objecter _session_command_op_remove -1 1 2024-02-23T00:57:53.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.975+0000 7fa27ffff700 20 client.42377.objecter _get_session s=0x7fa28006dd10 osd=2 3 2024-02-23T00:57:53.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.975+0000 7fa27ffff700 20 client.42377.objecter get_session s=0x7fa28006dd10 osd=2 3 2024-02-23T00:57:53.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.975+0000 7fa27ffff700 15 client.42377.objecter _session_command_op_assign 2 1 2024-02-23T00:57:53.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.975+0000 7fa27ffff700 20 client.42377.objecter put_session s=0x7fa28006dd10 osd=2 4 2024-02-23T00:57:53.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.975+0000 7fa27ffff700 10 client.42377.objecter _send_command 1 2024-02-23T00:57:53.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.975+0000 7fa27ffff700 1 -- 172.21.15.47:0/4223243289 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa280070f00 con 0x7fa28006e3e0 2024-02-23T00:57:53.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.975+0000 7fa27ffff700 20 client.42377.objecter dump_active .. 0 homeless 2024-02-23T00:57:53.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.975+0000 7fa27ffff700 1 -- 172.21.15.47:0/4223243289 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_get_version_reply(handle=1 version=858) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa28c080c10 con 0x7fa2980f7400 2024-02-23T00:57:53.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.975+0000 7fa2977fe700 1 --2- 172.21.15.47:0/4223243289 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa28006e3e0 0x7fa280070880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:53.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.975+0000 7fa27ffff700 10 client.42377.objecter ms_handle_connect 0x7fa280062cc0 2024-02-23T00:57:53.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.975+0000 7fa2967fc700 1 --2- 172.21.15.47:0/4223243289 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa280062cc0 0x7fa280065180 secure :-1 s=READY pgs=3246 cs=0 l=1 rev1=1 rx=0x7fa2980fa150 tx=0x7fa28800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:53.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.976+0000 7fa27ffff700 10 client.42377.objecter ms_handle_connect 0x7fa28006e3e0 2024-02-23T00:57:53.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.976+0000 7fa2977fe700 1 --2- 172.21.15.47:0/4223243289 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa28006e3e0 0x7fa280070880 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:53.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.976+0000 7fa27ffff700 1 -- 172.21.15.47:0/4223243289 <== osd.2 v2:172.21.15.47:6818/3596356785 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa280070f00 con 0x7fa28006e3e0 2024-02-23T00:57:53.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.976+0000 7fa27ffff700 10 client.42377.objecter ms_dispatch 0x7fa298001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:57:53.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.976+0000 7fa27ffff700 10 client.42377.objecter _finish_command 1 = system:0 2024-02-23T00:57:53.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.976+0000 7fa27ffff700 20 client.42377.objecter put_session s=0x7fa28006dd10 osd=2 4 2024-02-23T00:57:53.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.976+0000 7fa27ffff700 15 client.42377.objecter _session_command_op_remove 2 1 2024-02-23T00:57:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.991+0000 7fa29d95f700 10 client.42377.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-02-23T00:57:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.991+0000 7fa29d95f700 15 client.42377.objecter _session_command_op_assign -1 2 2024-02-23T00:57:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.991+0000 7fa29d95f700 20 client.42377.objecter _get_session s=0x7fa28006dd10 osd=2 3 2024-02-23T00:57:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.991+0000 7fa29d95f700 20 client.42377.objecter put_session s=0x7fa28006dd10 osd=2 3 2024-02-23T00:57:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.991+0000 7fa29d95f700 20 client.42377.objecter _get_session s=0x7fa28006dd10 osd=2 3 2024-02-23T00:57:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.991+0000 7fa29d95f700 15 client.42377.objecter _session_command_op_remove -1 2 2024-02-23T00:57:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.991+0000 7fa29d95f700 20 client.42377.objecter get_session s=0x7fa28006dd10 osd=2 3 2024-02-23T00:57:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.991+0000 7fa29d95f700 15 client.42377.objecter _session_command_op_assign 2 2 2024-02-23T00:57:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.991+0000 7fa29d95f700 20 client.42377.objecter put_session s=0x7fa28006dd10 osd=2 4 2024-02-23T00:57:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.991+0000 7fa29d95f700 10 client.42377.objecter _send_command 2 2024-02-23T00:57:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.991+0000 7fa29d95f700 1 -- 172.21.15.47:0/4223243289 --> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa29805ee80 con 0x7fa28006e3e0 2024-02-23T00:57:53.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.991+0000 7fa27ffff700 1 -- 172.21.15.47:0/4223243289 <== osd.2 v2:172.21.15.47:6818/3596356785 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa29805ee80 con 0x7fa28006e3e0 2024-02-23T00:57:53.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.991+0000 7fa27ffff700 10 client.42377.objecter ms_dispatch 0x7fa298001550 command_reply(tid 2: 0 ) v1 2024-02-23T00:57:53.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.991+0000 7fa27ffff700 10 client.42377.objecter _finish_command 2 = system:0 2024-02-23T00:57:53.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.991+0000 7fa27ffff700 20 client.42377.objecter put_session s=0x7fa28006dd10 osd=2 4 2024-02-23T00:57:53.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.991+0000 7fa27ffff700 15 client.42377.objecter _session_command_op_remove 2 2 2024-02-23T00:57:53.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.991+0000 7fa29d95f700 10 librados: watch_flush enter 2024-02-23T00:57:53.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.991+0000 7fa29d95f700 10 librados: watch_flush exit 2024-02-23T00:57:53.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.991+0000 7fa29d95f700 10 client.42377.objecter close_session for osd.2 2024-02-23T00:57:53.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.991+0000 7fa29d95f700 1 -- 172.21.15.47:0/4223243289 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa28006e3e0 msgr2=0x7fa280070880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:53.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.991+0000 7fa29d95f700 1 --2- 172.21.15.47:0/4223243289 >> [v2:172.21.15.47:6818/3596356785,v1:172.21.15.47:6819/3596356785] conn(0x7fa28006e3e0 0x7fa280070880 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:53.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.991+0000 7fa29d95f700 20 client.42377.objecter put_session s=0x7fa28006dd10 osd=2 1 2024-02-23T00:57:53.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.991+0000 7fa29d95f700 20 client.42377.objecter shutdown clearing up homeless session... 2024-02-23T00:57:53.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.991+0000 7fa29d95f700 10 client.42377.objecter successfully canceled tick 2024-02-23T00:57:53.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.991+0000 7fa29d95f700 1 -- 172.21.15.47:0/4223243289 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa280062cc0 msgr2=0x7fa280065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:53.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.991+0000 7fa29d95f700 1 --2- 172.21.15.47:0/4223243289 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fa280062cc0 0x7fa280065180 secure :-1 s=READY pgs=3246 cs=0 l=1 rev1=1 rx=0x7fa2980fa150 tx=0x7fa28800b040).stop 2024-02-23T00:57:53.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.991+0000 7fa29d95f700 1 -- 172.21.15.47:0/4223243289 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa2980f7400 msgr2=0x7fa2980f89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:53.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.991+0000 7fa29d95f700 1 --2- 172.21.15.47:0/4223243289 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa2980f7400 0x7fa2980f89b0 secure :-1 s=READY pgs=2864 cs=0 l=1 rev1=1 rx=0x7fa28c009da0 tx=0x7fa28c009cb0).stop 2024-02-23T00:57:53.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.991+0000 7fa2977fe700 1 -- 172.21.15.47:0/4223243289 reap_dead start 2024-02-23T00:57:53.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.991+0000 7fa29d95f700 1 -- 172.21.15.47:0/4223243289 shutdown_connections 2024-02-23T00:57:53.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.991+0000 7fa29d95f700 1 -- 172.21.15.47:0/4223243289 >> 172.21.15.47:0/4223243289 conn(0x7fa2980f2820 msgr2=0x7fa298060eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:53.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.992+0000 7fa29d95f700 1 -- 172.21.15.47:0/4223243289 shutdown_connections 2024-02-23T00:57:53.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.992+0000 7fa29d95f700 1 -- 172.21.15.47:0/4223243289 wait complete. 2024-02-23T00:57:53.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:53.992+0000 7fa29d95f700 1 librados: shutdown 2024-02-23T00:57:54.671 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:57:53.091233+0000 mgr.y (mgr.14517) 747 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:57:54.671 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:53.276914+0000 osd.7 (osd.7) 427 : cluster [DBG] 5.10 scrub starts 2024-02-23T00:57:54.671 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:53.279862+0000 osd.7 (osd.7) 428 : cluster [DBG] 5.10 scrub ok 2024-02-23T00:57:54.671 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:53.436709+0000 mgr.y (mgr.14517) 748 : cluster [DBG] pgmap v1334: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:57:54.671 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:53.849225+0000 osd.1 (osd.1) 356 : cluster [DBG] 18.18 scrub starts 2024-02-23T00:57:54.671 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:53.850156+0000 osd.1 (osd.1) 357 : cluster [DBG] 18.18 scrub ok 2024-02-23T00:57:54.672 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:57:53.091233+0000 mgr.y (mgr.14517) 747 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:57:54.672 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:53.276914+0000 osd.7 (osd.7) 427 : cluster [DBG] 5.10 scrub starts 2024-02-23T00:57:54.672 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:53.279862+0000 osd.7 (osd.7) 428 : cluster [DBG] 5.10 scrub ok 2024-02-23T00:57:54.672 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:53.436709+0000 mgr.y (mgr.14517) 748 : cluster [DBG] pgmap v1334: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:57:54.672 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:53.849225+0000 osd.1 (osd.1) 356 : cluster [DBG] 18.18 scrub starts 2024-02-23T00:57:54.672 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:54 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:53.850156+0000 osd.1 (osd.1) 357 : cluster [DBG] 18.18 scrub ok 2024-02-23T00:57:54.694 INFO:teuthology.orchestra.run.smithi047.stdout:77309411584 2024-02-23T00:57:54.695 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-23T00:57:54.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:57:53.091233+0000 mgr.y (mgr.14517) 747 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:57:54.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:53.276914+0000 osd.7 (osd.7) 427 : cluster [DBG] 5.10 scrub starts 2024-02-23T00:57:54.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:53.279862+0000 osd.7 (osd.7) 428 : cluster [DBG] 5.10 scrub ok 2024-02-23T00:57:54.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:53.436709+0000 mgr.y (mgr.14517) 748 : cluster [DBG] pgmap v1334: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:57:54.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:53.849225+0000 osd.1 (osd.1) 356 : cluster [DBG] 18.18 scrub starts 2024-02-23T00:57:54.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:54 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:53.850156+0000 osd.1 (osd.1) 357 : cluster [DBG] 18.18 scrub ok 2024-02-23T00:57:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:57:56.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:55.437278+0000 mgr.y (mgr.14517) 749 : cluster [DBG] pgmap v1335: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-23T00:57:56.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:55.515209+0000 osd.4 (osd.4) 395 : cluster [DBG] 6.2 scrub starts 2024-02-23T00:57:56.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:55.516311+0000 osd.4 (osd.4) 396 : cluster [DBG] 6.2 scrub ok 2024-02-23T00:57:56.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:55.437278+0000 mgr.y (mgr.14517) 749 : cluster [DBG] pgmap v1335: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-23T00:57:56.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:55.515209+0000 osd.4 (osd.4) 395 : cluster [DBG] 6.2 scrub starts 2024-02-23T00:57:56.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:56 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:55.516311+0000 osd.4 (osd.4) 396 : cluster [DBG] 6.2 scrub ok 2024-02-23T00:57:56.712 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:55.437278+0000 mgr.y (mgr.14517) 749 : cluster [DBG] pgmap v1335: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-23T00:57:56.712 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:55.515209+0000 osd.4 (osd.4) 395 : cluster [DBG] 6.2 scrub starts 2024-02-23T00:57:56.712 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:55.516311+0000 osd.4 (osd.4) 396 : cluster [DBG] 6.2 scrub ok 2024-02-23T00:57:56.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.975+0000 7f7b89128700 1 -- 172.21.15.47:0/3721573594 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7b840fcbb0 msgr2=0x7f7b840fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:56.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.975+0000 7f7b89128700 1 --2- 172.21.15.47:0/3721573594 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7b840fcbb0 0x7f7b840fcf90 secure :-1 s=READY pgs=2865 cs=0 l=1 rev1=1 rx=0x7f7b74004870 tx=0x7f7b74009cb0).stop 2024-02-23T00:57:56.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.975+0000 7f7b89128700 1 -- 172.21.15.47:0/3721573594 shutdown_connections 2024-02-23T00:57:56.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.975+0000 7f7b89128700 1 --2- 172.21.15.47:0/3721573594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b840f7400 0x7f7b84103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:56.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.975+0000 7f7b89128700 1 --2- 172.21.15.47:0/3721573594 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7b840fcbb0 0x7f7b840fcf90 unknown :-1 s=CLOSED pgs=2865 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:56.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.975+0000 7f7b89128700 1 --2- 172.21.15.47:0/3721573594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b840fd510 0x7f7b840f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:56.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.975+0000 7f7b89128700 1 -- 172.21.15.47:0/3721573594 >> 172.21.15.47:0/3721573594 conn(0x7f7b840f2820 msgr2=0x7f7b840f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:56.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.975+0000 7f7b89128700 1 -- 172.21.15.47:0/3721573594 shutdown_connections 2024-02-23T00:57:56.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.975+0000 7f7b89128700 1 -- 172.21.15.47:0/3721573594 wait complete. 2024-02-23T00:57:56.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.976+0000 7f7b89128700 1 librados: starting msgr at 2024-02-23T00:57:56.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.976+0000 7f7b89128700 1 librados: starting objecter 2024-02-23T00:57:56.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.976+0000 7f7b89128700 1 Processor -- start 2024-02-23T00:57:56.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.976+0000 7f7b89128700 1 -- start start 2024-02-23T00:57:56.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.976+0000 7f7b89128700 1 librados: setting wanted keys 2024-02-23T00:57:56.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.976+0000 7f7b89128700 1 librados: calling monclient init 2024-02-23T00:57:56.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.977+0000 7f7b89128700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b840f7400 0x7f7b84109190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:56.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.977+0000 7f7b89128700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b840fcbb0 0x7f7b84109680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:56.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.977+0000 7f7b89128700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7b840fd510 0x7f7b84109c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:56.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.977+0000 7f7b89128700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b84114370 con 0x7f7b840fcbb0 2024-02-23T00:57:56.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.977+0000 7f7b89128700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b841144e0 con 0x7f7b840f7400 2024-02-23T00:57:56.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.977+0000 7f7b89128700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b841146a0 con 0x7f7b840fd510 2024-02-23T00:57:56.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.977+0000 7f7b8359e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7b840fd510 0x7f7b84109c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:56.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.977+0000 7f7b8359e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7b840fd510 0x7f7b84109c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:38940/0 (socket says 172.21.15.47:38940) 2024-02-23T00:57:56.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.977+0000 7f7b8359e700 1 -- 172.21.15.47:0/119587207 learned_addr learned my addr 172.21.15.47:0/119587207 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:57:56.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.977+0000 7f7b8259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b840fcbb0 0x7f7b84109680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:56.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.977+0000 7f7b82d9d700 1 --2- 172.21.15.47:0/119587207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b840f7400 0x7f7b84109190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:56.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.977+0000 7f7b8359e700 1 -- 172.21.15.47:0/119587207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b840f7400 msgr2=0x7f7b84109190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:56.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.977+0000 7f7b8359e700 1 --2- 172.21.15.47:0/119587207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b840f7400 0x7f7b84109190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:56.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.977+0000 7f7b8359e700 1 -- 172.21.15.47:0/119587207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b840fcbb0 msgr2=0x7f7b84109680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:56.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.977+0000 7f7b8359e700 1 --2- 172.21.15.47:0/119587207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b840fcbb0 0x7f7b84109680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:56.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.977+0000 7f7b8359e700 1 -- 172.21.15.47:0/119587207 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b78005040 con 0x7f7b840fd510 2024-02-23T00:57:56.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.977+0000 7f7b8259c700 1 --2- 172.21.15.47:0/119587207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b840fcbb0 0x7f7b84109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:57:56.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.977+0000 7f7b8359e700 1 --2- 172.21.15.47:0/119587207 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7b840fd510 0x7f7b84109c60 secure :-1 s=READY pgs=2866 cs=0 l=1 rev1=1 rx=0x7f7b7c00d970 tx=0x7f7b7c004e60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:56.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.977+0000 7f7b6bfff700 10 client.?.objecter ms_handle_connect 0x7f7b840fd510 2024-02-23T00:57:56.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.977+0000 7f7b6bfff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:57:56.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.977+0000 7f7b6bfff700 1 -- 172.21.15.47:0/119587207 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c004020 con 0x7f7b840fd510 2024-02-23T00:57:56.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.977+0000 7f7b89128700 1 -- 172.21.15.47:0/119587207 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b74005040 con 0x7f7b840fd510 2024-02-23T00:57:56.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.977+0000 7f7b6bfff700 1 -- 172.21.15.47:0/119587207 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f7b7c00f040 con 0x7f7b840fd510 2024-02-23T00:57:56.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.977+0000 7f7b6bfff700 1 -- 172.21.15.47:0/119587207 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c003430 con 0x7f7b840fd510 2024-02-23T00:57:56.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.977+0000 7f7b89128700 10 client.42383.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:57:56.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.977+0000 7f7b89128700 1 -- 172.21.15.47:0/119587207 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b84114c90 con 0x7f7b840fd510 2024-02-23T00:57:56.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.978+0000 7f7b89128700 1 librados: init done 2024-02-23T00:57:56.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.978+0000 7f7b89128700 10 client.42383.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:57:56.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.978+0000 7f7b89128700 15 client.42383.objecter _session_command_op_assign -1 1 2024-02-23T00:57:56.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.978+0000 7f7b89128700 20 client.42383.objecter _get_session osd=-1 returning homeless 2024-02-23T00:57:56.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.978+0000 7f7b89128700 10 client.42383.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:57:56.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.978+0000 7f7b89128700 1 -- 172.21.15.47:0/119587207 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7b8405fa60 con 0x7f7b840fd510 2024-02-23T00:57:56.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.979+0000 7f7b6bfff700 1 -- 172.21.15.47:0/119587207 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f7b7c00b510 con 0x7f7b840fd510 2024-02-23T00:57:56.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.979+0000 7f7b6bfff700 1 --2- 172.21.15.47:0/119587207 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7b6c062cc0 0x7f7b6c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:56.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.979+0000 7f7b6bfff700 1 -- 172.21.15.47:0/119587207 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(858..858 src has 806..858) v4 ==== 8663+0+0 (secure 0 0 0) 0x7f7b7c07cf80 con 0x7f7b840fd510 2024-02-23T00:57:56.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.979+0000 7f7b6bfff700 10 client.42383.objecter ms_dispatch 0x7f7b84001550 osd_map(858..858 src has 806..858) v4 2024-02-23T00:57:56.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.979+0000 7f7b6bfff700 3 client.42383.objecter handle_osd_map got epochs [858,858] > 0 2024-02-23T00:57:56.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.979+0000 7f7b6bfff700 3 client.42383.objecter handle_osd_map decoding full epoch 858 2024-02-23T00:57:56.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.979+0000 7f7b82d9d700 1 --2- 172.21.15.47:0/119587207 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7b6c062cc0 0x7f7b6c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:56.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.980+0000 7f7b6bfff700 10 client.42383.objecter checking command 1 2024-02-23T00:57:56.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.980+0000 7f7b6bfff700 1 --2- 172.21.15.47:0/119587207 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f7b6c06e360 0x7f7b6c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:56.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.980+0000 7f7b6bfff700 20 client.42383.objecter _get_session s=0x7f7b6c06dc90 osd=3 3 2024-02-23T00:57:56.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.980+0000 7f7b6bfff700 20 client.42383.objecter put_session s=0x7f7b6c06dc90 osd=3 3 2024-02-23T00:57:56.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.980+0000 7f7b6bfff700 15 client.42383.objecter _session_command_op_remove -1 1 2024-02-23T00:57:56.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.980+0000 7f7b6bfff700 20 client.42383.objecter _get_session s=0x7f7b6c06dc90 osd=3 3 2024-02-23T00:57:56.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.980+0000 7f7b6bfff700 20 client.42383.objecter get_session s=0x7f7b6c06dc90 osd=3 3 2024-02-23T00:57:56.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.980+0000 7f7b6bfff700 15 client.42383.objecter _session_command_op_assign 3 1 2024-02-23T00:57:56.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.980+0000 7f7b6bfff700 20 client.42383.objecter put_session s=0x7f7b6c06dc90 osd=3 4 2024-02-23T00:57:56.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.980+0000 7f7b6bfff700 10 client.42383.objecter _send_command 1 2024-02-23T00:57:56.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.980+0000 7f7b6bfff700 1 -- 172.21.15.47:0/119587207 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7b6c070e80 con 0x7f7b6c06e360 2024-02-23T00:57:56.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.980+0000 7f7b6bfff700 20 client.42383.objecter dump_active .. 0 homeless 2024-02-23T00:57:56.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.980+0000 7f7b6bfff700 1 -- 172.21.15.47:0/119587207 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_get_version_reply(handle=1 version=858) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b7c07f4e0 con 0x7f7b840fd510 2024-02-23T00:57:56.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.980+0000 7f7b8259c700 1 --2- 172.21.15.47:0/119587207 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f7b6c06e360 0x7f7b6c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:56.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.980+0000 7f7b6bfff700 10 client.42383.objecter ms_handle_connect 0x7f7b6c062cc0 2024-02-23T00:57:56.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.980+0000 7f7b82d9d700 1 --2- 172.21.15.47:0/119587207 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7b6c062cc0 0x7f7b6c065180 secure :-1 s=READY pgs=3247 cs=0 l=1 rev1=1 rx=0x7f7b74004ee0 tx=0x7f7b7400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:56.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.980+0000 7f7b6bfff700 10 client.42383.objecter ms_handle_connect 0x7f7b6c06e360 2024-02-23T00:57:56.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.980+0000 7f7b8259c700 1 --2- 172.21.15.47:0/119587207 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f7b6c06e360 0x7f7b6c070800 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:56.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.981+0000 7f7b6bfff700 1 -- 172.21.15.47:0/119587207 <== osd.3 v2:172.21.15.47:6826/1195438081 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7b6c070e80 con 0x7f7b6c06e360 2024-02-23T00:57:56.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.981+0000 7f7b6bfff700 10 client.42383.objecter ms_dispatch 0x7f7b84001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:57:56.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.981+0000 7f7b6bfff700 10 client.42383.objecter _finish_command 1 = system:0 2024-02-23T00:57:56.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.981+0000 7f7b6bfff700 20 client.42383.objecter put_session s=0x7f7b6c06dc90 osd=3 4 2024-02-23T00:57:56.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.981+0000 7f7b6bfff700 15 client.42383.objecter _session_command_op_remove 3 1 2024-02-23T00:57:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.995+0000 7f7b89128700 10 client.42383.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-02-23T00:57:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.995+0000 7f7b89128700 15 client.42383.objecter _session_command_op_assign -1 2 2024-02-23T00:57:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.995+0000 7f7b89128700 20 client.42383.objecter _get_session s=0x7f7b6c06dc90 osd=3 3 2024-02-23T00:57:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.995+0000 7f7b89128700 20 client.42383.objecter put_session s=0x7f7b6c06dc90 osd=3 3 2024-02-23T00:57:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.995+0000 7f7b89128700 20 client.42383.objecter _get_session s=0x7f7b6c06dc90 osd=3 3 2024-02-23T00:57:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.995+0000 7f7b89128700 15 client.42383.objecter _session_command_op_remove -1 2 2024-02-23T00:57:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.995+0000 7f7b89128700 20 client.42383.objecter get_session s=0x7f7b6c06dc90 osd=3 3 2024-02-23T00:57:56.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.995+0000 7f7b89128700 15 client.42383.objecter _session_command_op_assign 3 2 2024-02-23T00:57:56.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.995+0000 7f7b89128700 20 client.42383.objecter put_session s=0x7f7b6c06dc90 osd=3 4 2024-02-23T00:57:56.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.995+0000 7f7b89128700 10 client.42383.objecter _send_command 2 2024-02-23T00:57:56.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.995+0000 7f7b89128700 1 -- 172.21.15.47:0/119587207 --> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7b8405ee80 con 0x7f7b6c06e360 2024-02-23T00:57:56.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.995+0000 7f7b6bfff700 1 -- 172.21.15.47:0/119587207 <== osd.3 v2:172.21.15.47:6826/1195438081 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7b8405ee80 con 0x7f7b6c06e360 2024-02-23T00:57:56.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.995+0000 7f7b6bfff700 10 client.42383.objecter ms_dispatch 0x7f7b84001550 command_reply(tid 2: 0 ) v1 2024-02-23T00:57:56.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.995+0000 7f7b6bfff700 10 client.42383.objecter _finish_command 2 = system:0 2024-02-23T00:57:56.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.995+0000 7f7b6bfff700 20 client.42383.objecter put_session s=0x7f7b6c06dc90 osd=3 4 2024-02-23T00:57:56.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.995+0000 7f7b6bfff700 15 client.42383.objecter _session_command_op_remove 3 2 2024-02-23T00:57:57.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.996+0000 7f7b89128700 10 librados: watch_flush enter 2024-02-23T00:57:57.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.996+0000 7f7b89128700 10 librados: watch_flush exit 2024-02-23T00:57:57.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.996+0000 7f7b89128700 10 client.42383.objecter close_session for osd.3 2024-02-23T00:57:57.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.996+0000 7f7b89128700 1 -- 172.21.15.47:0/119587207 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f7b6c06e360 msgr2=0x7f7b6c070800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:57.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.996+0000 7f7b89128700 1 --2- 172.21.15.47:0/119587207 >> [v2:172.21.15.47:6826/1195438081,v1:172.21.15.47:6827/1195438081] conn(0x7f7b6c06e360 0x7f7b6c070800 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:57.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.996+0000 7f7b89128700 20 client.42383.objecter put_session s=0x7f7b6c06dc90 osd=3 1 2024-02-23T00:57:57.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.996+0000 7f7b89128700 20 client.42383.objecter shutdown clearing up homeless session... 2024-02-23T00:57:57.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.996+0000 7f7b89128700 10 client.42383.objecter successfully canceled tick 2024-02-23T00:57:57.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.996+0000 7f7b89128700 1 -- 172.21.15.47:0/119587207 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7b6c062cc0 msgr2=0x7f7b6c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:57.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.996+0000 7f7b89128700 1 --2- 172.21.15.47:0/119587207 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7b6c062cc0 0x7f7b6c065180 secure :-1 s=READY pgs=3247 cs=0 l=1 rev1=1 rx=0x7f7b74004ee0 tx=0x7f7b7400b040).stop 2024-02-23T00:57:57.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.996+0000 7f7b89128700 1 -- 172.21.15.47:0/119587207 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7b840fd510 msgr2=0x7f7b84109c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:57.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.996+0000 7f7b89128700 1 --2- 172.21.15.47:0/119587207 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7b840fd510 0x7f7b84109c60 secure :-1 s=READY pgs=2866 cs=0 l=1 rev1=1 rx=0x7f7b7c00d970 tx=0x7f7b7c004e60).stop 2024-02-23T00:57:57.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.996+0000 7f7b8359e700 1 -- 172.21.15.47:0/119587207 reap_dead start 2024-02-23T00:57:57.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.996+0000 7f7b89128700 1 -- 172.21.15.47:0/119587207 shutdown_connections 2024-02-23T00:57:57.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.996+0000 7f7b89128700 1 -- 172.21.15.47:0/119587207 >> 172.21.15.47:0/119587207 conn(0x7f7b840f2820 msgr2=0x7f7b840fa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:57.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.996+0000 7f7b89128700 1 -- 172.21.15.47:0/119587207 shutdown_connections 2024-02-23T00:57:57.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.996+0000 7f7b89128700 1 -- 172.21.15.47:0/119587207 wait complete. 2024-02-23T00:57:57.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:56.996+0000 7f7b89128700 1 librados: shutdown 2024-02-23T00:57:57.627 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:56.139757+0000 osd.2 (osd.2) 217 : cluster [DBG] 5.a scrub starts 2024-02-23T00:57:57.628 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:56.140895+0000 osd.2 (osd.2) 218 : cluster [DBG] 5.a scrub ok 2024-02-23T00:57:57.628 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:56.513562+0000 osd.4 (osd.4) 397 : cluster [DBG] 4.18 deep-scrub starts 2024-02-23T00:57:57.628 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:56.515460+0000 osd.4 (osd.4) 398 : cluster [DBG] 4.18 deep-scrub ok 2024-02-23T00:57:57.628 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:56.792938+0000 osd.3 (osd.3) 361 : cluster [DBG] 4.10 scrub starts 2024-02-23T00:57:57.628 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:56.796406+0000 osd.3 (osd.3) 362 : cluster [DBG] 4.10 scrub ok 2024-02-23T00:57:57.628 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:56.139757+0000 osd.2 (osd.2) 217 : cluster [DBG] 5.a scrub starts 2024-02-23T00:57:57.628 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:56.140895+0000 osd.2 (osd.2) 218 : cluster [DBG] 5.a scrub ok 2024-02-23T00:57:57.628 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:56.513562+0000 osd.4 (osd.4) 397 : cluster [DBG] 4.18 deep-scrub starts 2024-02-23T00:57:57.628 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:56.515460+0000 osd.4 (osd.4) 398 : cluster [DBG] 4.18 deep-scrub ok 2024-02-23T00:57:57.628 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:56.792938+0000 osd.3 (osd.3) 361 : cluster [DBG] 4.10 scrub starts 2024-02-23T00:57:57.628 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:57 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:56.796406+0000 osd.3 (osd.3) 362 : cluster [DBG] 4.10 scrub ok 2024-02-23T00:57:57.650 INFO:teuthology.orchestra.run.smithi047.stdout:111669149945 2024-02-23T00:57:57.651 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-23T00:57:57.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:56.139757+0000 osd.2 (osd.2) 217 : cluster [DBG] 5.a scrub starts 2024-02-23T00:57:57.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:56.140895+0000 osd.2 (osd.2) 218 : cluster [DBG] 5.a scrub ok 2024-02-23T00:57:57.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:56.513562+0000 osd.4 (osd.4) 397 : cluster [DBG] 4.18 deep-scrub starts 2024-02-23T00:57:57.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:56.515460+0000 osd.4 (osd.4) 398 : cluster [DBG] 4.18 deep-scrub ok 2024-02-23T00:57:57.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:56.792938+0000 osd.3 (osd.3) 361 : cluster [DBG] 4.10 scrub starts 2024-02-23T00:57:57.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:57 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:56.796406+0000 osd.3 (osd.3) 362 : cluster [DBG] 4.10 scrub ok 2024-02-23T00:57:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:57:58.426 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:57.127482+0000 osd.2 (osd.2) 219 : cluster [DBG] 5.b scrub starts 2024-02-23T00:57:58.426 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:57.128456+0000 osd.2 (osd.2) 220 : cluster [DBG] 5.b scrub ok 2024-02-23T00:57:58.426 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:57.268271+0000 osd.0 (osd.0) 350 : cluster [DBG] 6.9 scrub starts 2024-02-23T00:57:58.426 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:57.269420+0000 osd.0 (osd.0) 351 : cluster [DBG] 6.9 scrub ok 2024-02-23T00:57:58.426 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:57.437970+0000 mgr.y (mgr.14517) 750 : cluster [DBG] pgmap v1336: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:57:58.426 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:58.122715+0000 osd.5 (osd.5) 412 : cluster [DBG] 18.15 scrub starts 2024-02-23T00:57:58.426 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:58.125839+0000 osd.5 (osd.5) 413 : cluster [DBG] 18.15 scrub ok 2024-02-23T00:57:58.426 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:57.127482+0000 osd.2 (osd.2) 219 : cluster [DBG] 5.b scrub starts 2024-02-23T00:57:58.426 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:57.128456+0000 osd.2 (osd.2) 220 : cluster [DBG] 5.b scrub ok 2024-02-23T00:57:58.427 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:57.268271+0000 osd.0 (osd.0) 350 : cluster [DBG] 6.9 scrub starts 2024-02-23T00:57:58.427 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:57.269420+0000 osd.0 (osd.0) 351 : cluster [DBG] 6.9 scrub ok 2024-02-23T00:57:58.427 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:57.437970+0000 mgr.y (mgr.14517) 750 : cluster [DBG] pgmap v1336: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:57:58.427 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:58.122715+0000 osd.5 (osd.5) 412 : cluster [DBG] 18.15 scrub starts 2024-02-23T00:57:58.427 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:58 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:58.125839+0000 osd.5 (osd.5) 413 : cluster [DBG] 18.15 scrub ok 2024-02-23T00:57:58.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:57.127482+0000 osd.2 (osd.2) 219 : cluster [DBG] 5.b scrub starts 2024-02-23T00:57:58.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:57.128456+0000 osd.2 (osd.2) 220 : cluster [DBG] 5.b scrub ok 2024-02-23T00:57:58.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:57.268271+0000 osd.0 (osd.0) 350 : cluster [DBG] 6.9 scrub starts 2024-02-23T00:57:58.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:57.269420+0000 osd.0 (osd.0) 351 : cluster [DBG] 6.9 scrub ok 2024-02-23T00:57:58.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:57.437970+0000 mgr.y (mgr.14517) 750 : cluster [DBG] pgmap v1336: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:57:58.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:58.122715+0000 osd.5 (osd.5) 412 : cluster [DBG] 18.15 scrub starts 2024-02-23T00:57:58.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:58 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:58.125839+0000 osd.5 (osd.5) 413 : cluster [DBG] 18.15 scrub ok 2024-02-23T00:57:59.398 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:58.123081+0000 osd.2 (osd.2) 221 : cluster [DBG] 5.6 scrub starts 2024-02-23T00:57:59.399 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:58.123976+0000 osd.2 (osd.2) 222 : cluster [DBG] 5.6 scrub ok 2024-02-23T00:57:59.399 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:58.219520+0000 osd.7 (osd.7) 429 : cluster [DBG] 6.8 deep-scrub starts 2024-02-23T00:57:59.399 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:58.222410+0000 osd.7 (osd.7) 430 : cluster [DBG] 6.8 deep-scrub ok 2024-02-23T00:57:59.399 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:58.765719+0000 osd.3 (osd.3) 363 : cluster [DBG] 4.17 scrub starts 2024-02-23T00:57:59.399 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:57:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:58.767106+0000 osd.3 (osd.3) 364 : cluster [DBG] 4.17 scrub ok 2024-02-23T00:57:59.399 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:58.123081+0000 osd.2 (osd.2) 221 : cluster [DBG] 5.6 scrub starts 2024-02-23T00:57:59.399 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:58.123976+0000 osd.2 (osd.2) 222 : cluster [DBG] 5.6 scrub ok 2024-02-23T00:57:59.399 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:58.219520+0000 osd.7 (osd.7) 429 : cluster [DBG] 6.8 deep-scrub starts 2024-02-23T00:57:59.399 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:58.222410+0000 osd.7 (osd.7) 430 : cluster [DBG] 6.8 deep-scrub ok 2024-02-23T00:57:59.399 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:58.765719+0000 osd.3 (osd.3) 363 : cluster [DBG] 4.17 scrub starts 2024-02-23T00:57:59.400 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:57:59 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:58.767106+0000 osd.3 (osd.3) 364 : cluster [DBG] 4.17 scrub ok 2024-02-23T00:57:59.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.658+0000 7f918c926700 1 -- 172.21.15.47:0/4281885327 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91880fe510 msgr2=0x7f91880f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:59.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.658+0000 7f918c926700 1 --2- 172.21.15.47:0/4281885327 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91880fe510 0x7f91880f7d40 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 rx=0x7f9178004870 tx=0x7f9178009cb0).stop 2024-02-23T00:57:59.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.658+0000 7f918c926700 1 -- 172.21.15.47:0/4281885327 shutdown_connections 2024-02-23T00:57:59.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.658+0000 7f918c926700 1 --2- 172.21.15.47:0/4281885327 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f91880f8420 0x7f9188104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:59.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.658+0000 7f918c926700 1 --2- 172.21.15.47:0/4281885327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91880fdbb0 0x7f91880fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:59.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.658+0000 7f918c926700 1 --2- 172.21.15.47:0/4281885327 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91880fe510 0x7f91880f7d40 unknown :-1 s=CLOSED pgs=2678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:59.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.658+0000 7f918c926700 1 -- 172.21.15.47:0/4281885327 >> 172.21.15.47:0/4281885327 conn(0x7f91880f3800 msgr2=0x7f91880f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:59.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.658+0000 7f918c926700 1 -- 172.21.15.47:0/4281885327 shutdown_connections 2024-02-23T00:57:59.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.658+0000 7f918c926700 1 -- 172.21.15.47:0/4281885327 wait complete. 2024-02-23T00:57:59.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.658+0000 7f918c926700 1 librados: starting msgr at 2024-02-23T00:57:59.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.658+0000 7f918c926700 1 librados: starting objecter 2024-02-23T00:57:59.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.659+0000 7f918c926700 1 Processor -- start 2024-02-23T00:57:59.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.659+0000 7f918c926700 1 -- start start 2024-02-23T00:57:59.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.659+0000 7f918c926700 1 librados: setting wanted keys 2024-02-23T00:57:59.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.659+0000 7f918c926700 1 librados: calling monclient init 2024-02-23T00:57:59.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.659+0000 7f918c926700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91880f8420 0x7f918806b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:59.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.659+0000 7f918c926700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91880fdbb0 0x7f918806b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:59.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.659+0000 7f918c926700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f91880fe510 0x7f918806bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:59.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.659+0000 7f918c926700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91881162e0 con 0x7f91880f8420 2024-02-23T00:57:59.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.659+0000 7f918c926700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9188116450 con 0x7f91880fdbb0 2024-02-23T00:57:59.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.659+0000 7f918c926700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9188116610 con 0x7f91880fe510 2024-02-23T00:57:59.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.659+0000 7f9186d9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f91880fe510 0x7f918806bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:59.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.659+0000 7f9186d9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f91880fe510 0x7f918806bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:38944/0 (socket says 172.21.15.47:38944) 2024-02-23T00:57:59.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.659+0000 7f9186d9d700 1 -- 172.21.15.47:0/2239302202 learned_addr learned my addr 172.21.15.47:0/2239302202 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:57:59.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.659+0000 7f918659c700 1 --2- 172.21.15.47:0/2239302202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91880f8420 0x7f918806b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:59.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.659+0000 7f9185d9b700 1 --2- 172.21.15.47:0/2239302202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91880fdbb0 0x7f918806b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:59.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.659+0000 7f9186d9d700 1 -- 172.21.15.47:0/2239302202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91880fdbb0 msgr2=0x7f918806b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:59.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.659+0000 7f9186d9d700 1 --2- 172.21.15.47:0/2239302202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91880fdbb0 0x7f918806b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:59.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.659+0000 7f9186d9d700 1 -- 172.21.15.47:0/2239302202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91880f8420 msgr2=0x7f918806b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:59.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.659+0000 7f9186d9d700 1 --2- 172.21.15.47:0/2239302202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91880f8420 0x7f918806b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:59.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.659+0000 7f9186d9d700 1 -- 172.21.15.47:0/2239302202 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91800097c0 con 0x7f91880fe510 2024-02-23T00:57:59.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.659+0000 7f9185d9b700 1 --2- 172.21.15.47:0/2239302202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91880fdbb0 0x7f918806b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:57:59.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.660+0000 7f9186d9d700 1 --2- 172.21.15.47:0/2239302202 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f91880fe510 0x7f918806bab0 secure :-1 s=READY pgs=2867 cs=0 l=1 rev1=1 rx=0x7f9180001df0 tx=0x7f9180004c50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:59.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.660+0000 7f91777fe700 10 client.?.objecter ms_handle_connect 0x7f91880fe510 2024-02-23T00:57:59.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.660+0000 7f91777fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:57:59.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.660+0000 7f91777fe700 1 -- 172.21.15.47:0/2239302202 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180009a10 con 0x7f91880fe510 2024-02-23T00:57:59.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.660+0000 7f918c926700 1 -- 172.21.15.47:0/2239302202 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9178005040 con 0x7f91880fe510 2024-02-23T00:57:59.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.660+0000 7f91777fe700 1 -- 172.21.15.47:0/2239302202 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f9180009560 con 0x7f91880fe510 2024-02-23T00:57:59.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.660+0000 7f91777fe700 1 -- 172.21.15.47:0/2239302202 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180007c50 con 0x7f91880fe510 2024-02-23T00:57:59.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.660+0000 7f918c926700 10 client.42389.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:57:59.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.660+0000 7f918c926700 1 -- 172.21.15.47:0/2239302202 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9188116c80 con 0x7f91880fe510 2024-02-23T00:57:59.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.660+0000 7f918c926700 1 librados: init done 2024-02-23T00:57:59.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.661+0000 7f918c926700 10 client.42389.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:57:59.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.661+0000 7f918c926700 15 client.42389.objecter _session_command_op_assign -1 1 2024-02-23T00:57:59.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.661+0000 7f918c926700 20 client.42389.objecter _get_session osd=-1 returning homeless 2024-02-23T00:57:59.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.661+0000 7f918c926700 10 client.42389.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:57:59.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.661+0000 7f918c926700 1 -- 172.21.15.47:0/2239302202 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f918805ee80 con 0x7f91880fe510 2024-02-23T00:57:59.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.661+0000 7f91777fe700 1 -- 172.21.15.47:0/2239302202 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f91800090d0 con 0x7f91880fe510 2024-02-23T00:57:59.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.662+0000 7f91777fe700 1 --2- 172.21.15.47:0/2239302202 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9170062cc0 0x7f9170065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:59.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.662+0000 7f91777fe700 1 -- 172.21.15.47:0/2239302202 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(858..858 src has 806..858) v4 ==== 8663+0+0 (secure 0 0 0) 0x7f918007de80 con 0x7f91880fe510 2024-02-23T00:57:59.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.662+0000 7f91777fe700 10 client.42389.objecter ms_dispatch 0x7f9188001550 osd_map(858..858 src has 806..858) v4 2024-02-23T00:57:59.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.662+0000 7f91777fe700 3 client.42389.objecter handle_osd_map got epochs [858,858] > 0 2024-02-23T00:57:59.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.662+0000 7f91777fe700 3 client.42389.objecter handle_osd_map decoding full epoch 858 2024-02-23T00:57:59.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.662+0000 7f918659c700 1 --2- 172.21.15.47:0/2239302202 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9170062cc0 0x7f9170065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:59.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.662+0000 7f91777fe700 10 client.42389.objecter checking command 1 2024-02-23T00:57:59.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.662+0000 7f91777fe700 1 --2- 172.21.15.47:0/2239302202 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f917006e360 0x7f9170070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:57:59.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.662+0000 7f91777fe700 20 client.42389.objecter _get_session s=0x7f917006dc90 osd=4 3 2024-02-23T00:57:59.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.662+0000 7f91777fe700 20 client.42389.objecter put_session s=0x7f917006dc90 osd=4 3 2024-02-23T00:57:59.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.662+0000 7f91777fe700 15 client.42389.objecter _session_command_op_remove -1 1 2024-02-23T00:57:59.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.662+0000 7f91777fe700 20 client.42389.objecter _get_session s=0x7f917006dc90 osd=4 3 2024-02-23T00:57:59.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.662+0000 7f91777fe700 20 client.42389.objecter get_session s=0x7f917006dc90 osd=4 3 2024-02-23T00:57:59.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.662+0000 7f91777fe700 15 client.42389.objecter _session_command_op_assign 4 1 2024-02-23T00:57:59.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.662+0000 7f91777fe700 20 client.42389.objecter put_session s=0x7f917006dc90 osd=4 4 2024-02-23T00:57:59.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.662+0000 7f91777fe700 10 client.42389.objecter _send_command 1 2024-02-23T00:57:59.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.662+0000 7f91777fe700 1 -- 172.21.15.47:0/2239302202 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9170070e80 con 0x7f917006e360 2024-02-23T00:57:59.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.662+0000 7f91777fe700 20 client.42389.objecter dump_active .. 0 homeless 2024-02-23T00:57:59.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.662+0000 7f91777fe700 1 -- 172.21.15.47:0/2239302202 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_get_version_reply(handle=1 version=858) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9180080460 con 0x7f91880fe510 2024-02-23T00:57:59.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.662+0000 7f918659c700 1 --2- 172.21.15.47:0/2239302202 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9170062cc0 0x7f9170065180 secure :-1 s=READY pgs=3248 cs=0 l=1 rev1=1 rx=0x7f917c004dc0 tx=0x7f917c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:59.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.662+0000 7f91777fe700 10 client.42389.objecter ms_handle_connect 0x7f9170062cc0 2024-02-23T00:57:59.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.662+0000 7f9185d9b700 1 --2- 172.21.15.47:0/2239302202 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f917006e360 0x7f9170070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:57:59.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.662+0000 7f91777fe700 10 client.42389.objecter ms_handle_connect 0x7f917006e360 2024-02-23T00:57:59.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.662+0000 7f9185d9b700 1 --2- 172.21.15.47:0/2239302202 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f917006e360 0x7f9170070800 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:57:59.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.663+0000 7f91777fe700 1 -- 172.21.15.47:0/2239302202 <== osd.4 v2:172.21.15.196:6800/2530629682 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f9170070e80 con 0x7f917006e360 2024-02-23T00:57:59.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.663+0000 7f91777fe700 10 client.42389.objecter ms_dispatch 0x7f9188001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:57:59.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.663+0000 7f91777fe700 10 client.42389.objecter _finish_command 1 = system:0 2024-02-23T00:57:59.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.663+0000 7f91777fe700 20 client.42389.objecter put_session s=0x7f917006dc90 osd=4 4 2024-02-23T00:57:59.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.663+0000 7f91777fe700 15 client.42389.objecter _session_command_op_remove 4 1 2024-02-23T00:57:59.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.677+0000 7f918c926700 10 client.42389.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-02-23T00:57:59.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.677+0000 7f918c926700 15 client.42389.objecter _session_command_op_assign -1 2 2024-02-23T00:57:59.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.677+0000 7f918c926700 20 client.42389.objecter _get_session s=0x7f917006dc90 osd=4 3 2024-02-23T00:57:59.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.677+0000 7f918c926700 20 client.42389.objecter put_session s=0x7f917006dc90 osd=4 3 2024-02-23T00:57:59.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.677+0000 7f918c926700 20 client.42389.objecter _get_session s=0x7f917006dc90 osd=4 3 2024-02-23T00:57:59.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.677+0000 7f918c926700 15 client.42389.objecter _session_command_op_remove -1 2 2024-02-23T00:57:59.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.677+0000 7f918c926700 20 client.42389.objecter get_session s=0x7f917006dc90 osd=4 3 2024-02-23T00:57:59.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.677+0000 7f918c926700 15 client.42389.objecter _session_command_op_assign 4 2 2024-02-23T00:57:59.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.677+0000 7f918c926700 20 client.42389.objecter put_session s=0x7f917006dc90 osd=4 4 2024-02-23T00:57:59.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.677+0000 7f918c926700 10 client.42389.objecter _send_command 2 2024-02-23T00:57:59.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.677+0000 7f918c926700 1 -- 172.21.15.47:0/2239302202 --> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9188106e60 con 0x7f917006e360 2024-02-23T00:57:59.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.678+0000 7f91777fe700 1 -- 172.21.15.47:0/2239302202 <== osd.4 v2:172.21.15.196:6800/2530629682 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9188106e60 con 0x7f917006e360 2024-02-23T00:57:59.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.678+0000 7f91777fe700 10 client.42389.objecter ms_dispatch 0x7f9188001550 command_reply(tid 2: 0 ) v1 2024-02-23T00:57:59.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.678+0000 7f91777fe700 10 client.42389.objecter _finish_command 2 = system:0 2024-02-23T00:57:59.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.678+0000 7f91777fe700 20 client.42389.objecter put_session s=0x7f917006dc90 osd=4 4 2024-02-23T00:57:59.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.678+0000 7f91777fe700 15 client.42389.objecter _session_command_op_remove 4 2 2024-02-23T00:57:59.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.678+0000 7f918c926700 10 librados: watch_flush enter 2024-02-23T00:57:59.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.678+0000 7f918c926700 10 librados: watch_flush exit 2024-02-23T00:57:59.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.678+0000 7f918c926700 10 client.42389.objecter close_session for osd.4 2024-02-23T00:57:59.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.678+0000 7f918c926700 1 -- 172.21.15.47:0/2239302202 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f917006e360 msgr2=0x7f9170070800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:59.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.678+0000 7f918c926700 1 --2- 172.21.15.47:0/2239302202 >> [v2:172.21.15.196:6800/2530629682,v1:172.21.15.196:6801/2530629682] conn(0x7f917006e360 0x7f9170070800 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:57:59.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.678+0000 7f918c926700 20 client.42389.objecter put_session s=0x7f917006dc90 osd=4 1 2024-02-23T00:57:59.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.678+0000 7f918c926700 20 client.42389.objecter shutdown clearing up homeless session... 2024-02-23T00:57:59.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.678+0000 7f918c926700 10 client.42389.objecter successfully canceled tick 2024-02-23T00:57:59.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.678+0000 7f918c926700 1 -- 172.21.15.47:0/2239302202 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9170062cc0 msgr2=0x7f9170065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:59.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.678+0000 7f918c926700 1 --2- 172.21.15.47:0/2239302202 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f9170062cc0 0x7f9170065180 secure :-1 s=READY pgs=3248 cs=0 l=1 rev1=1 rx=0x7f917c004dc0 tx=0x7f917c00b040).stop 2024-02-23T00:57:59.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.678+0000 7f918c926700 1 -- 172.21.15.47:0/2239302202 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f91880fe510 msgr2=0x7f918806bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:57:59.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.678+0000 7f918c926700 1 --2- 172.21.15.47:0/2239302202 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f91880fe510 0x7f918806bab0 secure :-1 s=READY pgs=2867 cs=0 l=1 rev1=1 rx=0x7f9180001df0 tx=0x7f9180004c50).stop 2024-02-23T00:57:59.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.678+0000 7f9186d9d700 1 -- 172.21.15.47:0/2239302202 reap_dead start 2024-02-23T00:57:59.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.678+0000 7f918c926700 1 -- 172.21.15.47:0/2239302202 shutdown_connections 2024-02-23T00:57:59.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.678+0000 7f918c926700 1 -- 172.21.15.47:0/2239302202 >> 172.21.15.47:0/2239302202 conn(0x7f91880f3800 msgr2=0x7f91880fb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:57:59.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.678+0000 7f918c926700 1 -- 172.21.15.47:0/2239302202 shutdown_connections 2024-02-23T00:57:59.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.679+0000 7f918c926700 1 -- 172.21.15.47:0/2239302202 wait complete. 2024-02-23T00:57:59.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:57:59.679+0000 7f918c926700 1 librados: shutdown 2024-02-23T00:57:59.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:58.123081+0000 osd.2 (osd.2) 221 : cluster [DBG] 5.6 scrub starts 2024-02-23T00:57:59.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:58.123976+0000 osd.2 (osd.2) 222 : cluster [DBG] 5.6 scrub ok 2024-02-23T00:57:59.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:58.219520+0000 osd.7 (osd.7) 429 : cluster [DBG] 6.8 deep-scrub starts 2024-02-23T00:57:59.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:58.222410+0000 osd.7 (osd.7) 430 : cluster [DBG] 6.8 deep-scrub ok 2024-02-23T00:57:59.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:58.765719+0000 osd.3 (osd.3) 363 : cluster [DBG] 4.17 scrub starts 2024-02-23T00:57:59.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:57:59 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:58.767106+0000 osd.3 (osd.3) 364 : cluster [DBG] 4.17 scrub ok 2024-02-23T00:58:00.422 INFO:teuthology.orchestra.run.smithi047.stdout:137438953718 2024-02-23T00:58:00.423 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-23T00:58:00.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:59.438596+0000 mgr.y (mgr.14517) 751 : cluster [DBG] pgmap v1337: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:58:00.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:59.560132+0000 osd.4 (osd.4) 399 : cluster [DBG] 4.1a scrub starts 2024-02-23T00:58:00.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:57:59.561618+0000 osd.4 (osd.4) 400 : cluster [DBG] 4.1a scrub ok 2024-02-23T00:58:00.704 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:58:00.807 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:58:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:58:00] "GET /metrics HTTP/1.1" 200 187139 "" "Prometheus/2.33.4" 2024-02-23T00:58:00.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:59.438596+0000 mgr.y (mgr.14517) 751 : cluster [DBG] pgmap v1337: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:58:00.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:59.560132+0000 osd.4 (osd.4) 399 : cluster [DBG] 4.1a scrub starts 2024-02-23T00:58:00.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:57:59.561618+0000 osd.4 (osd.4) 400 : cluster [DBG] 4.1a scrub ok 2024-02-23T00:58:00.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:59.438596+0000 mgr.y (mgr.14517) 751 : cluster [DBG] pgmap v1337: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:58:00.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:59.560132+0000 osd.4 (osd.4) 399 : cluster [DBG] 4.1a scrub starts 2024-02-23T00:58:00.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:57:59.561618+0000 osd.4 (osd.4) 400 : cluster [DBG] 4.1a scrub ok 2024-02-23T00:58:01.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:00.281207+0000 osd.0 (osd.0) 352 : cluster [DBG] 18.8 scrub starts 2024-02-23T00:58:01.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:00.282395+0000 osd.0 (osd.0) 353 : cluster [DBG] 18.8 scrub ok 2024-02-23T00:58:01.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:00.378419+0000 osd.6 (osd.6) 371 : cluster [DBG] 2.7 scrub starts 2024-02-23T00:58:01.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:01 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:00.379536+0000 osd.6 (osd.6) 372 : cluster [DBG] 2.7 scrub ok 2024-02-23T00:58:01.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:00.281207+0000 osd.0 (osd.0) 352 : cluster [DBG] 18.8 scrub starts 2024-02-23T00:58:01.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:00.282395+0000 osd.0 (osd.0) 353 : cluster [DBG] 18.8 scrub ok 2024-02-23T00:58:01.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:00.378419+0000 osd.6 (osd.6) 371 : cluster [DBG] 2.7 scrub starts 2024-02-23T00:58:01.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:00.379536+0000 osd.6 (osd.6) 372 : cluster [DBG] 2.7 scrub ok 2024-02-23T00:58:01.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:00.281207+0000 osd.0 (osd.0) 352 : cluster [DBG] 18.8 scrub starts 2024-02-23T00:58:01.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:00.282395+0000 osd.0 (osd.0) 353 : cluster [DBG] 18.8 scrub ok 2024-02-23T00:58:01.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:00.378419+0000 osd.6 (osd.6) 371 : cluster [DBG] 2.7 scrub starts 2024-02-23T00:58:01.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:00.379536+0000 osd.6 (osd.6) 372 : cluster [DBG] 2.7 scrub ok 2024-02-23T00:58:02.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.491+0000 7f0da0758700 1 -- 172.21.15.47:0/3433924922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d980f9610 msgr2=0x7f0d98101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:02.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.491+0000 7f0da0758700 1 --2- 172.21.15.47:0/3433924922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d980f9610 0x7f0d98101d20 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 rx=0x7f0d880047b0 tx=0x7f0d88009cb0).stop 2024-02-23T00:58:02.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.491+0000 7f0da0758700 1 -- 172.21.15.47:0/3433924922 shutdown_connections 2024-02-23T00:58:02.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.491+0000 7f0da0758700 1 --2- 172.21.15.47:0/3433924922 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0d98102210 0x7f0d98104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:02.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.491+0000 7f0da0758700 1 --2- 172.21.15.47:0/3433924922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d980f8cb0 0x7f0d980f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:02.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.491+0000 7f0da0758700 1 --2- 172.21.15.47:0/3433924922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d980f9610 0x7f0d98101d20 unknown :-1 s=CLOSED pgs=2679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:02.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.491+0000 7f0da0758700 1 -- 172.21.15.47:0/3433924922 >> 172.21.15.47:0/3433924922 conn(0x7f0d980f2820 msgr2=0x7f0d980f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:58:02.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.491+0000 7f0da0758700 1 -- 172.21.15.47:0/3433924922 shutdown_connections 2024-02-23T00:58:02.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.491+0000 7f0da0758700 1 -- 172.21.15.47:0/3433924922 wait complete. 2024-02-23T00:58:02.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.492+0000 7f0da0758700 1 librados: starting msgr at 2024-02-23T00:58:02.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.492+0000 7f0da0758700 1 librados: starting objecter 2024-02-23T00:58:02.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.492+0000 7f0da0758700 1 Processor -- start 2024-02-23T00:58:02.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.492+0000 7f0da0758700 1 -- start start 2024-02-23T00:58:02.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.492+0000 7f0da0758700 1 librados: setting wanted keys 2024-02-23T00:58:02.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.492+0000 7f0da0758700 1 librados: calling monclient init 2024-02-23T00:58:02.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.492+0000 7f0da0758700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0d980f8cb0 0x7f0d980f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:02.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.492+0000 7f0da0758700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d980f9610 0x7f0d980f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:02.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.492+0000 7f0da0758700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d98102210 0x7f0d980f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:02.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.492+0000 7f0da0758700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d98116500 con 0x7f0d980f9610 2024-02-23T00:58:02.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.492+0000 7f0da0758700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d98116670 con 0x7f0d98102210 2024-02-23T00:58:02.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.492+0000 7f0da0758700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d98116830 con 0x7f0d980f8cb0 2024-02-23T00:58:02.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.492+0000 7f0d9dcf3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d980f9610 0x7f0d980f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:02.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.492+0000 7f0d9dcf3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d980f9610 0x7f0d980f7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47332/0 (socket says 172.21.15.47:47332) 2024-02-23T00:58:02.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.492+0000 7f0d9e4f4700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0d980f8cb0 0x7f0d980f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:02.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.492+0000 7f0d9dcf3700 1 -- 172.21.15.47:0/3544342035 learned_addr learned my addr 172.21.15.47:0/3544342035 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:58:02.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.492+0000 7f0d9ecf5700 1 --2- 172.21.15.47:0/3544342035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d98102210 0x7f0d980f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:02.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.492+0000 7f0d9dcf3700 1 -- 172.21.15.47:0/3544342035 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0d980f8cb0 msgr2=0x7f0d980f6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:02.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.492+0000 7f0d9dcf3700 1 --2- 172.21.15.47:0/3544342035 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0d980f8cb0 0x7f0d980f6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:02.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.492+0000 7f0d9dcf3700 1 -- 172.21.15.47:0/3544342035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d98102210 msgr2=0x7f0d980f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:02.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.492+0000 7f0d9dcf3700 1 --2- 172.21.15.47:0/3544342035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d98102210 0x7f0d980f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:02.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.492+0000 7f0d9dcf3700 1 -- 172.21.15.47:0/3544342035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d900097c0 con 0x7f0d980f9610 2024-02-23T00:58:02.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.492+0000 7f0d9ecf5700 1 --2- 172.21.15.47:0/3544342035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d98102210 0x7f0d980f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:58:02.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.493+0000 7f0d9dcf3700 1 --2- 172.21.15.47:0/3544342035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d980f9610 0x7f0d980f7150 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 rx=0x7f0d88009f20 tx=0x7f0d88009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:58:02.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.493+0000 7f0d8f7fe700 10 client.?.objecter ms_handle_connect 0x7f0d980f9610 2024-02-23T00:58:02.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.493+0000 7f0d8f7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:58:02.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.493+0000 7f0d8f7fe700 1 -- 172.21.15.47:0/3544342035 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d88018070 con 0x7f0d980f9610 2024-02-23T00:58:02.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.493+0000 7f0da0758700 1 -- 172.21.15.47:0/3544342035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d88005040 con 0x7f0d980f9610 2024-02-23T00:58:02.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.493+0000 7f0d8f7fe700 1 -- 172.21.15.47:0/3544342035 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f0d880096d0 con 0x7f0d980f9610 2024-02-23T00:58:02.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.493+0000 7f0d8f7fe700 1 -- 172.21.15.47:0/3544342035 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8801dc40 con 0x7f0d980f9610 2024-02-23T00:58:02.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.493+0000 7f0da0758700 10 client.32073.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:58:02.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.493+0000 7f0da0758700 1 -- 172.21.15.47:0/3544342035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d98116d20 con 0x7f0d980f9610 2024-02-23T00:58:02.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.493+0000 7f0da0758700 1 librados: init done 2024-02-23T00:58:02.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.493+0000 7f0da0758700 10 client.32073.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:58:02.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.493+0000 7f0da0758700 15 client.32073.objecter _session_command_op_assign -1 1 2024-02-23T00:58:02.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.493+0000 7f0da0758700 20 client.32073.objecter _get_session osd=-1 returning homeless 2024-02-23T00:58:02.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.493+0000 7f0da0758700 10 client.32073.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:58:02.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.493+0000 7f0da0758700 1 -- 172.21.15.47:0/3544342035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0d9805fa60 con 0x7f0d980f9610 2024-02-23T00:58:02.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.494+0000 7f0d8f7fe700 1 -- 172.21.15.47:0/3544342035 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f0d88009240 con 0x7f0d980f9610 2024-02-23T00:58:02.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.494+0000 7f0d8f7fe700 1 --2- 172.21.15.47:0/3544342035 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0d84062cc0 0x7f0d84065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:02.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.495+0000 7f0d8f7fe700 1 -- 172.21.15.47:0/3544342035 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(858..858 src has 806..858) v4 ==== 8663+0+0 (secure 0 0 0) 0x7f0d8807e8e0 con 0x7f0d980f9610 2024-02-23T00:58:02.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.495+0000 7f0d8f7fe700 10 client.32073.objecter ms_dispatch 0x7f0d98001550 osd_map(858..858 src has 806..858) v4 2024-02-23T00:58:02.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.495+0000 7f0d8f7fe700 3 client.32073.objecter handle_osd_map got epochs [858,858] > 0 2024-02-23T00:58:02.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.495+0000 7f0d8f7fe700 3 client.32073.objecter handle_osd_map decoding full epoch 858 2024-02-23T00:58:02.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.495+0000 7f0d9e4f4700 1 --2- 172.21.15.47:0/3544342035 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0d84062cc0 0x7f0d84065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:02.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.495+0000 7f0d8f7fe700 10 client.32073.objecter checking command 1 2024-02-23T00:58:02.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.495+0000 7f0d8f7fe700 1 --2- 172.21.15.47:0/3544342035 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f0d8406e360 0x7f0d84070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:02.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.495+0000 7f0d8f7fe700 20 client.32073.objecter _get_session s=0x7f0d8406dc90 osd=5 3 2024-02-23T00:58:02.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.495+0000 7f0d8f7fe700 20 client.32073.objecter put_session s=0x7f0d8406dc90 osd=5 3 2024-02-23T00:58:02.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.495+0000 7f0d8f7fe700 15 client.32073.objecter _session_command_op_remove -1 1 2024-02-23T00:58:02.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.495+0000 7f0d8f7fe700 20 client.32073.objecter _get_session s=0x7f0d8406dc90 osd=5 3 2024-02-23T00:58:02.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.495+0000 7f0d8f7fe700 20 client.32073.objecter get_session s=0x7f0d8406dc90 osd=5 3 2024-02-23T00:58:02.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.495+0000 7f0d8f7fe700 15 client.32073.objecter _session_command_op_assign 5 1 2024-02-23T00:58:02.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.495+0000 7f0d8f7fe700 20 client.32073.objecter put_session s=0x7f0d8406dc90 osd=5 4 2024-02-23T00:58:02.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.495+0000 7f0d8f7fe700 10 client.32073.objecter _send_command 1 2024-02-23T00:58:02.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.495+0000 7f0d8f7fe700 1 -- 172.21.15.47:0/3544342035 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0d84070e80 con 0x7f0d8406e360 2024-02-23T00:58:02.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.495+0000 7f0d8f7fe700 20 client.32073.objecter dump_active .. 0 homeless 2024-02-23T00:58:02.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.495+0000 7f0d8f7fe700 1 -- 172.21.15.47:0/3544342035 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=858) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d88083220 con 0x7f0d980f9610 2024-02-23T00:58:02.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.495+0000 7f0d8f7fe700 10 client.32073.objecter ms_handle_connect 0x7f0d84062cc0 2024-02-23T00:58:02.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.495+0000 7f0d9e4f4700 1 --2- 172.21.15.47:0/3544342035 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0d84062cc0 0x7f0d84065180 secure :-1 s=READY pgs=3249 cs=0 l=1 rev1=1 rx=0x7f0d94004d30 tx=0x7f0d9400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:58:02.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.495+0000 7f0d9ecf5700 1 --2- 172.21.15.47:0/3544342035 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f0d8406e360 0x7f0d84070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:02.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.495+0000 7f0d8f7fe700 10 client.32073.objecter ms_handle_connect 0x7f0d8406e360 2024-02-23T00:58:02.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.495+0000 7f0d9ecf5700 1 --2- 172.21.15.47:0/3544342035 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f0d8406e360 0x7f0d84070800 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:58:02.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.496+0000 7f0d8f7fe700 1 -- 172.21.15.47:0/3544342035 <== osd.5 v2:172.21.15.196:6808/1017187491 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0d84070e80 con 0x7f0d8406e360 2024-02-23T00:58:02.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.496+0000 7f0d8f7fe700 10 client.32073.objecter ms_dispatch 0x7f0d98001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:58:02.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.496+0000 7f0d8f7fe700 10 client.32073.objecter _finish_command 1 = system:0 2024-02-23T00:58:02.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.496+0000 7f0d8f7fe700 20 client.32073.objecter put_session s=0x7f0d8406dc90 osd=5 4 2024-02-23T00:58:02.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.496+0000 7f0d8f7fe700 15 client.32073.objecter _session_command_op_remove 5 1 2024-02-23T00:58:02.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.510+0000 7f0da0758700 10 client.32073.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-02-23T00:58:02.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.510+0000 7f0da0758700 15 client.32073.objecter _session_command_op_assign -1 2 2024-02-23T00:58:02.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.510+0000 7f0da0758700 20 client.32073.objecter _get_session s=0x7f0d8406dc90 osd=5 3 2024-02-23T00:58:02.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.510+0000 7f0da0758700 20 client.32073.objecter put_session s=0x7f0d8406dc90 osd=5 3 2024-02-23T00:58:02.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.510+0000 7f0da0758700 20 client.32073.objecter _get_session s=0x7f0d8406dc90 osd=5 3 2024-02-23T00:58:02.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.510+0000 7f0da0758700 15 client.32073.objecter _session_command_op_remove -1 2 2024-02-23T00:58:02.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.510+0000 7f0da0758700 20 client.32073.objecter get_session s=0x7f0d8406dc90 osd=5 3 2024-02-23T00:58:02.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.510+0000 7f0da0758700 15 client.32073.objecter _session_command_op_assign 5 2 2024-02-23T00:58:02.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.510+0000 7f0da0758700 20 client.32073.objecter put_session s=0x7f0d8406dc90 osd=5 4 2024-02-23T00:58:02.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.510+0000 7f0da0758700 10 client.32073.objecter _send_command 2 2024-02-23T00:58:02.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.510+0000 7f0da0758700 1 -- 172.21.15.47:0/3544342035 --> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0d9805ee80 con 0x7f0d8406e360 2024-02-23T00:58:02.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.511+0000 7f0d8f7fe700 1 -- 172.21.15.47:0/3544342035 <== osd.5 v2:172.21.15.196:6808/1017187491 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0d9805ee80 con 0x7f0d8406e360 2024-02-23T00:58:02.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.511+0000 7f0d8f7fe700 10 client.32073.objecter ms_dispatch 0x7f0d98001550 command_reply(tid 2: 0 ) v1 2024-02-23T00:58:02.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.511+0000 7f0d8f7fe700 10 client.32073.objecter _finish_command 2 = system:0 2024-02-23T00:58:02.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.511+0000 7f0d8f7fe700 20 client.32073.objecter put_session s=0x7f0d8406dc90 osd=5 4 2024-02-23T00:58:02.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.511+0000 7f0d8f7fe700 15 client.32073.objecter _session_command_op_remove 5 2 2024-02-23T00:58:02.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.512+0000 7f0da0758700 10 librados: watch_flush enter 2024-02-23T00:58:02.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.512+0000 7f0da0758700 10 librados: watch_flush exit 2024-02-23T00:58:02.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.512+0000 7f0da0758700 10 client.32073.objecter close_session for osd.5 2024-02-23T00:58:02.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.512+0000 7f0da0758700 1 -- 172.21.15.47:0/3544342035 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f0d8406e360 msgr2=0x7f0d84070800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:02.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.512+0000 7f0da0758700 1 --2- 172.21.15.47:0/3544342035 >> [v2:172.21.15.196:6808/1017187491,v1:172.21.15.196:6809/1017187491] conn(0x7f0d8406e360 0x7f0d84070800 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:02.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.512+0000 7f0da0758700 20 client.32073.objecter put_session s=0x7f0d8406dc90 osd=5 1 2024-02-23T00:58:02.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.512+0000 7f0da0758700 20 client.32073.objecter shutdown clearing up homeless session... 2024-02-23T00:58:02.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.512+0000 7f0da0758700 10 client.32073.objecter successfully canceled tick 2024-02-23T00:58:02.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.512+0000 7f0da0758700 1 -- 172.21.15.47:0/3544342035 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0d84062cc0 msgr2=0x7f0d84065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:02.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.512+0000 7f0da0758700 1 --2- 172.21.15.47:0/3544342035 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0d84062cc0 0x7f0d84065180 secure :-1 s=READY pgs=3249 cs=0 l=1 rev1=1 rx=0x7f0d94004d30 tx=0x7f0d9400b040).stop 2024-02-23T00:58:02.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.512+0000 7f0da0758700 1 -- 172.21.15.47:0/3544342035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d980f9610 msgr2=0x7f0d980f7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:02.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.512+0000 7f0da0758700 1 --2- 172.21.15.47:0/3544342035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d980f9610 0x7f0d980f7150 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 rx=0x7f0d88009f20 tx=0x7f0d88009cb0).stop 2024-02-23T00:58:02.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.512+0000 7f0d9ecf5700 1 -- 172.21.15.47:0/3544342035 reap_dead start 2024-02-23T00:58:02.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.512+0000 7f0da0758700 1 -- 172.21.15.47:0/3544342035 shutdown_connections 2024-02-23T00:58:02.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.512+0000 7f0da0758700 1 -- 172.21.15.47:0/3544342035 >> 172.21.15.47:0/3544342035 conn(0x7f0d980f2820 msgr2=0x7f0d980f34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:58:02.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.512+0000 7f0da0758700 1 -- 172.21.15.47:0/3544342035 shutdown_connections 2024-02-23T00:58:02.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.512+0000 7f0da0758700 1 -- 172.21.15.47:0/3544342035 wait complete. 2024-02-23T00:58:02.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:02.512+0000 7f0da0758700 1 librados: shutdown 2024-02-23T00:58:02.542 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:01.271421+0000 osd.0 (osd.0) 354 : cluster [DBG] 4.16 scrub starts 2024-02-23T00:58:02.542 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:01.273190+0000 osd.0 (osd.0) 355 : cluster [DBG] 4.16 scrub ok 2024-02-23T00:58:02.542 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:01.439203+0000 mgr.y (mgr.14517) 752 : cluster [DBG] pgmap v1338: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 614 B/s rd, 0 op/s 2024-02-23T00:58:02.542 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:01.535068+0000 osd.4 (osd.4) 401 : cluster [DBG] 4.e scrub starts 2024-02-23T00:58:02.542 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:01.536776+0000 osd.4 (osd.4) 402 : cluster [DBG] 4.e scrub ok 2024-02-23T00:58:02.543 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:01.271421+0000 osd.0 (osd.0) 354 : cluster [DBG] 4.16 scrub starts 2024-02-23T00:58:02.543 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:01.273190+0000 osd.0 (osd.0) 355 : cluster [DBG] 4.16 scrub ok 2024-02-23T00:58:02.543 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:01.439203+0000 mgr.y (mgr.14517) 752 : cluster [DBG] pgmap v1338: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 614 B/s rd, 0 op/s 2024-02-23T00:58:02.543 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:01.535068+0000 osd.4 (osd.4) 401 : cluster [DBG] 4.e scrub starts 2024-02-23T00:58:02.543 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:01.536776+0000 osd.4 (osd.4) 402 : cluster [DBG] 4.e scrub ok 2024-02-23T00:58:02.694 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:58:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:58:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:58:02.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:01.271421+0000 osd.0 (osd.0) 354 : cluster [DBG] 4.16 scrub starts 2024-02-23T00:58:02.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:01.273190+0000 osd.0 (osd.0) 355 : cluster [DBG] 4.16 scrub ok 2024-02-23T00:58:02.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:01.439203+0000 mgr.y (mgr.14517) 752 : cluster [DBG] pgmap v1338: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 614 B/s rd, 0 op/s 2024-02-23T00:58:02.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:01.535068+0000 osd.4 (osd.4) 401 : cluster [DBG] 4.e scrub starts 2024-02-23T00:58:02.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:02 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:01.536776+0000 osd.4 (osd.4) 402 : cluster [DBG] 4.e scrub ok 2024-02-23T00:58:03.119 INFO:teuthology.orchestra.run.smithi047.stdout:167503724786 2024-02-23T00:58:03.119 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-02-23T00:58:03.381 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:58:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:58:03.382 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:01.417156+0000 osd.6 (osd.6) 373 : cluster [DBG] 5.2 scrub starts 2024-02-23T00:58:03.382 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:01.418070+0000 osd.6 (osd.6) 374 : cluster [DBG] 5.2 scrub ok 2024-02-23T00:58:03.382 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:02.713824+0000 osd.3 (osd.3) 365 : cluster [DBG] 2.1b deep-scrub starts 2024-02-23T00:58:03.382 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:03 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:02.715190+0000 osd.3 (osd.3) 366 : cluster [DBG] 2.1b deep-scrub ok 2024-02-23T00:58:03.382 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:01.417156+0000 osd.6 (osd.6) 373 : cluster [DBG] 5.2 scrub starts 2024-02-23T00:58:03.382 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:01.418070+0000 osd.6 (osd.6) 374 : cluster [DBG] 5.2 scrub ok 2024-02-23T00:58:03.382 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:02.713824+0000 osd.3 (osd.3) 365 : cluster [DBG] 2.1b deep-scrub starts 2024-02-23T00:58:03.383 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:02.715190+0000 osd.3 (osd.3) 366 : cluster [DBG] 2.1b deep-scrub ok 2024-02-23T00:58:03.401 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:58:03.680 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:01.417156+0000 osd.6 (osd.6) 373 : cluster [DBG] 5.2 scrub starts 2024-02-23T00:58:03.680 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:01.418070+0000 osd.6 (osd.6) 374 : cluster [DBG] 5.2 scrub ok 2024-02-23T00:58:03.680 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:02.713824+0000 osd.3 (osd.3) 365 : cluster [DBG] 2.1b deep-scrub starts 2024-02-23T00:58:03.680 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:03 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:02.715190+0000 osd.3 (osd.3) 366 : cluster [DBG] 2.1b deep-scrub ok 2024-02-23T00:58:04.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:58:03.095865+0000 mgr.y (mgr.14517) 753 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:58:04.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:03.251174+0000 osd.7 (osd.7) 431 : cluster [DBG] 5.22 scrub starts 2024-02-23T00:58:04.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:03.254148+0000 osd.7 (osd.7) 432 : cluster [DBG] 5.22 scrub ok 2024-02-23T00:58:04.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:03.439989+0000 mgr.y (mgr.14517) 754 : cluster [DBG] pgmap v1339: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 941 B/s rd, 0 op/s 2024-02-23T00:58:04.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:58:03.095865+0000 mgr.y (mgr.14517) 753 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:58:04.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:03.251174+0000 osd.7 (osd.7) 431 : cluster [DBG] 5.22 scrub starts 2024-02-23T00:58:04.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:03.254148+0000 osd.7 (osd.7) 432 : cluster [DBG] 5.22 scrub ok 2024-02-23T00:58:04.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:03.439989+0000 mgr.y (mgr.14517) 754 : cluster [DBG] pgmap v1339: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 941 B/s rd, 0 op/s 2024-02-23T00:58:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:58:03.095865+0000 mgr.y (mgr.14517) 753 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:58:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:03.251174+0000 osd.7 (osd.7) 431 : cluster [DBG] 5.22 scrub starts 2024-02-23T00:58:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:03.254148+0000 osd.7 (osd.7) 432 : cluster [DBG] 5.22 scrub ok 2024-02-23T00:58:04.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:04 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:03.439989+0000 mgr.y (mgr.14517) 754 : cluster [DBG] pgmap v1339: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 941 B/s rd, 0 op/s 2024-02-23T00:58:05.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.184+0000 7f1e6fccf700 1 -- 172.21.15.47:0/1586842160 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e680fd510 msgr2=0x7f1e680f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:05.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.184+0000 7f1e6fccf700 1 --2- 172.21.15.47:0/1586842160 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e680fd510 0x7f1e680f6d40 secure :-1 s=READY pgs=2868 cs=0 l=1 rev1=1 rx=0x7f1e580047b0 tx=0x7f1e58009cb0).stop 2024-02-23T00:58:05.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.184+0000 7f1e6fccf700 1 -- 172.21.15.47:0/1586842160 shutdown_connections 2024-02-23T00:58:05.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.184+0000 7f1e6fccf700 1 --2- 172.21.15.47:0/1586842160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e680f7400 0x7f1e68103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:05.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.184+0000 7f1e6fccf700 1 --2- 172.21.15.47:0/1586842160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e680fcbb0 0x7f1e680fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:05.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.184+0000 7f1e6fccf700 1 --2- 172.21.15.47:0/1586842160 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e680fd510 0x7f1e680f6d40 unknown :-1 s=CLOSED pgs=2868 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:05.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.184+0000 7f1e6fccf700 1 -- 172.21.15.47:0/1586842160 >> 172.21.15.47:0/1586842160 conn(0x7f1e680f2820 msgr2=0x7f1e680f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:58:05.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.184+0000 7f1e6fccf700 1 -- 172.21.15.47:0/1586842160 shutdown_connections 2024-02-23T00:58:05.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.184+0000 7f1e6fccf700 1 -- 172.21.15.47:0/1586842160 wait complete. 2024-02-23T00:58:05.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.184+0000 7f1e6fccf700 1 librados: starting msgr at 2024-02-23T00:58:05.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.184+0000 7f1e6fccf700 1 librados: starting objecter 2024-02-23T00:58:05.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.184+0000 7f1e6fccf700 1 Processor -- start 2024-02-23T00:58:05.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.184+0000 7f1e6fccf700 1 -- start start 2024-02-23T00:58:05.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.184+0000 7f1e6fccf700 1 librados: setting wanted keys 2024-02-23T00:58:05.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.184+0000 7f1e6fccf700 1 librados: calling monclient init 2024-02-23T00:58:05.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.185+0000 7f1e6fccf700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e680f7400 0x7f1e680f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:05.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.185+0000 7f1e6fccf700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e680fcbb0 0x7f1e680f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:05.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.185+0000 7f1e6fccf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e680fd510 0x7f1e680f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:05.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.185+0000 7f1e6fccf700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e681164f0 con 0x7f1e680fd510 2024-02-23T00:58:05.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.185+0000 7f1e6fccf700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e68116660 con 0x7f1e680fcbb0 2024-02-23T00:58:05.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.185+0000 7f1e6fccf700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e68116820 con 0x7f1e680f7400 2024-02-23T00:58:05.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.185+0000 7f1e6e26c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e680fd510 0x7f1e680f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:05.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.185+0000 7f1e6da6b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e680f7400 0x7f1e680f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:05.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.185+0000 7f1e6e26c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e680fd510 0x7f1e680f9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47344/0 (socket says 172.21.15.47:47344) 2024-02-23T00:58:05.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.185+0000 7f1e6e26c700 1 -- 172.21.15.47:0/2245903981 learned_addr learned my addr 172.21.15.47:0/2245903981 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:58:05.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.185+0000 7f1e6d26a700 1 --2- 172.21.15.47:0/2245903981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e680fcbb0 0x7f1e680f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:05.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.185+0000 7f1e6e26c700 1 -- 172.21.15.47:0/2245903981 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e680f7400 msgr2=0x7f1e680f89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:05.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.185+0000 7f1e6e26c700 1 --2- 172.21.15.47:0/2245903981 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e680f7400 0x7f1e680f89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:05.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.185+0000 7f1e6e26c700 1 -- 172.21.15.47:0/2245903981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e680fcbb0 msgr2=0x7f1e680f8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:05.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.185+0000 7f1e6e26c700 1 --2- 172.21.15.47:0/2245903981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e680fcbb0 0x7f1e680f8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:05.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.185+0000 7f1e6e26c700 1 -- 172.21.15.47:0/2245903981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e64005040 con 0x7f1e680fd510 2024-02-23T00:58:05.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.185+0000 7f1e6da6b700 1 --2- 172.21.15.47:0/2245903981 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e680f7400 0x7f1e680f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:58:05.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.185+0000 7f1e6e26c700 1 --2- 172.21.15.47:0/2245903981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e680fd510 0x7f1e680f9390 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 rx=0x7f1e6000ec20 tx=0x7f1e6000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:58:05.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.185+0000 7f1e5effd700 10 client.?.objecter ms_handle_connect 0x7f1e680fd510 2024-02-23T00:58:05.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.185+0000 7f1e5effd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:58:05.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.185+0000 7f1e5effd700 1 -- 172.21.15.47:0/2245903981 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e60009950 con 0x7f1e680fd510 2024-02-23T00:58:05.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.185+0000 7f1e5effd700 1 -- 172.21.15.47:0/2245903981 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f1e6000f040 con 0x7f1e680fd510 2024-02-23T00:58:05.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.185+0000 7f1e6fccf700 1 -- 172.21.15.47:0/2245903981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e58005040 con 0x7f1e680fd510 2024-02-23T00:58:05.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.185+0000 7f1e5effd700 1 -- 172.21.15.47:0/2245903981 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e60007d10 con 0x7f1e680fd510 2024-02-23T00:58:05.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.185+0000 7f1e6fccf700 10 client.32079.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:58:05.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.185+0000 7f1e6fccf700 1 -- 172.21.15.47:0/2245903981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e68116e00 con 0x7f1e680fd510 2024-02-23T00:58:05.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.185+0000 7f1e6fccf700 1 librados: init done 2024-02-23T00:58:05.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.186+0000 7f1e6fccf700 10 client.32079.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:58:05.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.187+0000 7f1e6fccf700 15 client.32079.objecter _session_command_op_assign -1 1 2024-02-23T00:58:05.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.187+0000 7f1e6fccf700 20 client.32079.objecter _get_session osd=-1 returning homeless 2024-02-23T00:58:05.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.187+0000 7f1e6fccf700 10 client.32079.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:58:05.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.187+0000 7f1e6fccf700 1 -- 172.21.15.47:0/2245903981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1e6805fa60 con 0x7f1e680fd510 2024-02-23T00:58:05.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.187+0000 7f1e5effd700 1 -- 172.21.15.47:0/2245903981 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f1e600092e0 con 0x7f1e680fd510 2024-02-23T00:58:05.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.187+0000 7f1e5effd700 1 --2- 172.21.15.47:0/2245903981 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1e54062cc0 0x7f1e54065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:05.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.187+0000 7f1e5effd700 1 -- 172.21.15.47:0/2245903981 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(858..858 src has 806..858) v4 ==== 8663+0+0 (secure 0 0 0) 0x7f1e6007d130 con 0x7f1e680fd510 2024-02-23T00:58:05.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.187+0000 7f1e5effd700 10 client.32079.objecter ms_dispatch 0x7f1e68001550 osd_map(858..858 src has 806..858) v4 2024-02-23T00:58:05.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.187+0000 7f1e6da6b700 1 --2- 172.21.15.47:0/2245903981 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1e54062cc0 0x7f1e54065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:05.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.187+0000 7f1e5effd700 3 client.32079.objecter handle_osd_map got epochs [858,858] > 0 2024-02-23T00:58:05.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.187+0000 7f1e5effd700 3 client.32079.objecter handle_osd_map decoding full epoch 858 2024-02-23T00:58:05.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.187+0000 7f1e5effd700 10 client.32079.objecter checking command 1 2024-02-23T00:58:05.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.187+0000 7f1e5effd700 1 --2- 172.21.15.47:0/2245903981 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f1e5406e360 0x7f1e54070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:05.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.187+0000 7f1e5effd700 20 client.32079.objecter _get_session s=0x7f1e5406dc90 osd=6 3 2024-02-23T00:58:05.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.187+0000 7f1e5effd700 20 client.32079.objecter put_session s=0x7f1e5406dc90 osd=6 3 2024-02-23T00:58:05.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.187+0000 7f1e5effd700 15 client.32079.objecter _session_command_op_remove -1 1 2024-02-23T00:58:05.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.187+0000 7f1e5effd700 20 client.32079.objecter _get_session s=0x7f1e5406dc90 osd=6 3 2024-02-23T00:58:05.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.187+0000 7f1e5effd700 20 client.32079.objecter get_session s=0x7f1e5406dc90 osd=6 3 2024-02-23T00:58:05.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.187+0000 7f1e5effd700 15 client.32079.objecter _session_command_op_assign 6 1 2024-02-23T00:58:05.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.187+0000 7f1e5effd700 20 client.32079.objecter put_session s=0x7f1e5406dc90 osd=6 4 2024-02-23T00:58:05.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.187+0000 7f1e5effd700 10 client.32079.objecter _send_command 1 2024-02-23T00:58:05.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.187+0000 7f1e5effd700 1 -- 172.21.15.47:0/2245903981 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1e54070e80 con 0x7f1e5406e360 2024-02-23T00:58:05.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.188+0000 7f1e5effd700 20 client.32079.objecter dump_active .. 0 homeless 2024-02-23T00:58:05.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.188+0000 7f1e5effd700 1 -- 172.21.15.47:0/2245903981 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=858) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e6007f6b0 con 0x7f1e680fd510 2024-02-23T00:58:05.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.188+0000 7f1e5effd700 10 client.32079.objecter ms_handle_connect 0x7f1e54062cc0 2024-02-23T00:58:05.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.188+0000 7f1e6da6b700 1 --2- 172.21.15.47:0/2245903981 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1e54062cc0 0x7f1e54065180 secure :-1 s=READY pgs=3250 cs=0 l=1 rev1=1 rx=0x7f1e64004c80 tx=0x7f1e6400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:58:05.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.188+0000 7f1e6d26a700 1 --2- 172.21.15.47:0/2245903981 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f1e5406e360 0x7f1e54070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:05.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.188+0000 7f1e5effd700 10 client.32079.objecter ms_handle_connect 0x7f1e5406e360 2024-02-23T00:58:05.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.188+0000 7f1e6d26a700 1 --2- 172.21.15.47:0/2245903981 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f1e5406e360 0x7f1e54070800 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:58:05.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.189+0000 7f1e5effd700 1 -- 172.21.15.47:0/2245903981 <== osd.6 v2:172.21.15.196:6816/3885668599 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1e54070e80 con 0x7f1e5406e360 2024-02-23T00:58:05.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.189+0000 7f1e5effd700 10 client.32079.objecter ms_dispatch 0x7f1e68001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:58:05.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.189+0000 7f1e5effd700 10 client.32079.objecter _finish_command 1 = system:0 2024-02-23T00:58:05.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.189+0000 7f1e5effd700 20 client.32079.objecter put_session s=0x7f1e5406dc90 osd=6 4 2024-02-23T00:58:05.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.189+0000 7f1e5effd700 15 client.32079.objecter _session_command_op_remove 6 1 2024-02-23T00:58:05.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.203+0000 7f1e6fccf700 10 client.32079.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-02-23T00:58:05.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.203+0000 7f1e6fccf700 15 client.32079.objecter _session_command_op_assign -1 2 2024-02-23T00:58:05.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.203+0000 7f1e6fccf700 20 client.32079.objecter _get_session s=0x7f1e5406dc90 osd=6 3 2024-02-23T00:58:05.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.203+0000 7f1e6fccf700 20 client.32079.objecter put_session s=0x7f1e5406dc90 osd=6 3 2024-02-23T00:58:05.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.203+0000 7f1e6fccf700 20 client.32079.objecter _get_session s=0x7f1e5406dc90 osd=6 3 2024-02-23T00:58:05.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.203+0000 7f1e6fccf700 15 client.32079.objecter _session_command_op_remove -1 2 2024-02-23T00:58:05.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.203+0000 7f1e6fccf700 20 client.32079.objecter get_session s=0x7f1e5406dc90 osd=6 3 2024-02-23T00:58:05.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.203+0000 7f1e6fccf700 15 client.32079.objecter _session_command_op_assign 6 2 2024-02-23T00:58:05.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.203+0000 7f1e6fccf700 20 client.32079.objecter put_session s=0x7f1e5406dc90 osd=6 4 2024-02-23T00:58:05.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.203+0000 7f1e6fccf700 10 client.32079.objecter _send_command 2 2024-02-23T00:58:05.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.203+0000 7f1e6fccf700 1 -- 172.21.15.47:0/2245903981 --> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1e6805ee80 con 0x7f1e5406e360 2024-02-23T00:58:05.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.204+0000 7f1e5effd700 1 -- 172.21.15.47:0/2245903981 <== osd.6 v2:172.21.15.196:6816/3885668599 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1e6805ee80 con 0x7f1e5406e360 2024-02-23T00:58:05.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.204+0000 7f1e5effd700 10 client.32079.objecter ms_dispatch 0x7f1e68001550 command_reply(tid 2: 0 ) v1 2024-02-23T00:58:05.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.204+0000 7f1e5effd700 10 client.32079.objecter _finish_command 2 = system:0 2024-02-23T00:58:05.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.204+0000 7f1e5effd700 20 client.32079.objecter put_session s=0x7f1e5406dc90 osd=6 4 2024-02-23T00:58:05.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.204+0000 7f1e5effd700 15 client.32079.objecter _session_command_op_remove 6 2 2024-02-23T00:58:05.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.204+0000 7f1e6fccf700 10 librados: watch_flush enter 2024-02-23T00:58:05.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.204+0000 7f1e6fccf700 10 librados: watch_flush exit 2024-02-23T00:58:05.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.204+0000 7f1e6fccf700 10 client.32079.objecter close_session for osd.6 2024-02-23T00:58:05.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.204+0000 7f1e6fccf700 1 -- 172.21.15.47:0/2245903981 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f1e5406e360 msgr2=0x7f1e54070800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:05.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.204+0000 7f1e6fccf700 1 --2- 172.21.15.47:0/2245903981 >> [v2:172.21.15.196:6816/3885668599,v1:172.21.15.196:6817/3885668599] conn(0x7f1e5406e360 0x7f1e54070800 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:05.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.204+0000 7f1e6fccf700 20 client.32079.objecter put_session s=0x7f1e5406dc90 osd=6 1 2024-02-23T00:58:05.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.204+0000 7f1e6fccf700 20 client.32079.objecter shutdown clearing up homeless session... 2024-02-23T00:58:05.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.204+0000 7f1e6fccf700 10 client.32079.objecter successfully canceled tick 2024-02-23T00:58:05.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.204+0000 7f1e6fccf700 1 -- 172.21.15.47:0/2245903981 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1e54062cc0 msgr2=0x7f1e54065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:05.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.204+0000 7f1e6fccf700 1 --2- 172.21.15.47:0/2245903981 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1e54062cc0 0x7f1e54065180 secure :-1 s=READY pgs=3250 cs=0 l=1 rev1=1 rx=0x7f1e64004c80 tx=0x7f1e6400b040).stop 2024-02-23T00:58:05.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.204+0000 7f1e6fccf700 1 -- 172.21.15.47:0/2245903981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e680fd510 msgr2=0x7f1e680f9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:05.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.204+0000 7f1e6fccf700 1 --2- 172.21.15.47:0/2245903981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e680fd510 0x7f1e680f9390 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 rx=0x7f1e6000ec20 tx=0x7f1e6000b040).stop 2024-02-23T00:58:05.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.204+0000 7f1e6e26c700 1 -- 172.21.15.47:0/2245903981 reap_dead start 2024-02-23T00:58:05.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.204+0000 7f1e6fccf700 1 -- 172.21.15.47:0/2245903981 shutdown_connections 2024-02-23T00:58:05.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.204+0000 7f1e6fccf700 1 -- 172.21.15.47:0/2245903981 >> 172.21.15.47:0/2245903981 conn(0x7f1e680f2820 msgr2=0x7f1e680f42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:58:05.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.204+0000 7f1e6fccf700 1 -- 172.21.15.47:0/2245903981 shutdown_connections 2024-02-23T00:58:05.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.204+0000 7f1e6fccf700 1 -- 172.21.15.47:0/2245903981 wait complete. 2024-02-23T00:58:05.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:05.205+0000 7f1e6fccf700 1 librados: shutdown 2024-02-23T00:58:05.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:04.247129+0000 osd.7 (osd.7) 433 : cluster [DBG] 2.1c scrub starts 2024-02-23T00:58:05.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:04.250162+0000 osd.7 (osd.7) 434 : cluster [DBG] 2.1c scrub ok 2024-02-23T00:58:05.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:04.247129+0000 osd.7 (osd.7) 433 : cluster [DBG] 2.1c scrub starts 2024-02-23T00:58:05.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:05 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:04.250162+0000 osd.7 (osd.7) 434 : cluster [DBG] 2.1c scrub ok 2024-02-23T00:58:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:04.247129+0000 osd.7 (osd.7) 433 : cluster [DBG] 2.1c scrub starts 2024-02-23T00:58:05.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:05 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:04.250162+0000 osd.7 (osd.7) 434 : cluster [DBG] 2.1c scrub ok 2024-02-23T00:58:06.012 INFO:teuthology.orchestra.run.smithi047.stdout:201863463149 2024-02-23T00:58:06.013 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-02-23T00:58:06.294 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:58:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:05.179992+0000 osd.2 (osd.2) 223 : cluster [DBG] 5.d scrub starts 2024-02-23T00:58:06.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:05.181533+0000 osd.2 (osd.2) 224 : cluster [DBG] 5.d scrub ok 2024-02-23T00:58:06.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:05.268672+0000 osd.7 (osd.7) 435 : cluster [DBG] 2.0 scrub starts 2024-02-23T00:58:06.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:05.271589+0000 osd.7 (osd.7) 436 : cluster [DBG] 2.0 scrub ok 2024-02-23T00:58:06.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:05.440446+0000 mgr.y (mgr.14517) 755 : cluster [DBG] pgmap v1340: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:58:06.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:05.458431+0000 osd.4 (osd.4) 403 : cluster [DBG] 5.23 scrub starts 2024-02-23T00:58:06.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:05.459512+0000 osd.4 (osd.4) 404 : cluster [DBG] 5.23 scrub ok 2024-02-23T00:58:06.788 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:05.179992+0000 osd.2 (osd.2) 223 : cluster [DBG] 5.d scrub starts 2024-02-23T00:58:06.788 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:05.181533+0000 osd.2 (osd.2) 224 : cluster [DBG] 5.d scrub ok 2024-02-23T00:58:06.788 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:05.268672+0000 osd.7 (osd.7) 435 : cluster [DBG] 2.0 scrub starts 2024-02-23T00:58:06.788 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:05.271589+0000 osd.7 (osd.7) 436 : cluster [DBG] 2.0 scrub ok 2024-02-23T00:58:06.788 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:05.440446+0000 mgr.y (mgr.14517) 755 : cluster [DBG] pgmap v1340: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:58:06.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:05.458431+0000 osd.4 (osd.4) 403 : cluster [DBG] 5.23 scrub starts 2024-02-23T00:58:06.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:05.459512+0000 osd.4 (osd.4) 404 : cluster [DBG] 5.23 scrub ok 2024-02-23T00:58:06.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:05.179992+0000 osd.2 (osd.2) 223 : cluster [DBG] 5.d scrub starts 2024-02-23T00:58:06.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:05.181533+0000 osd.2 (osd.2) 224 : cluster [DBG] 5.d scrub ok 2024-02-23T00:58:06.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:05.268672+0000 osd.7 (osd.7) 435 : cluster [DBG] 2.0 scrub starts 2024-02-23T00:58:06.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:05.271589+0000 osd.7 (osd.7) 436 : cluster [DBG] 2.0 scrub ok 2024-02-23T00:58:06.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:05.440446+0000 mgr.y (mgr.14517) 755 : cluster [DBG] pgmap v1340: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:58:06.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:05.458431+0000 osd.4 (osd.4) 403 : cluster [DBG] 5.23 scrub starts 2024-02-23T00:58:06.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:06 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:05.459512+0000 osd.4 (osd.4) 404 : cluster [DBG] 5.23 scrub ok 2024-02-23T00:58:07.471 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:05.448196+0000 osd.6 (osd.6) 375 : cluster [DBG] 18.14 scrub starts 2024-02-23T00:58:07.472 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:05.449354+0000 osd.6 (osd.6) 376 : cluster [DBG] 18.14 scrub ok 2024-02-23T00:58:07.472 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:06.205669+0000 osd.2 (osd.2) 225 : cluster [DBG] 2.1 scrub starts 2024-02-23T00:58:07.472 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:06.206619+0000 osd.2 (osd.2) 226 : cluster [DBG] 2.1 scrub ok 2024-02-23T00:58:07.472 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:06.282702+0000 osd.0 (osd.0) 356 : cluster [DBG] 3.1 deep-scrub starts 2024-02-23T00:58:07.472 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:06.283717+0000 osd.0 (osd.0) 357 : cluster [DBG] 3.1 deep-scrub ok 2024-02-23T00:58:07.472 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:06.659857+0000 osd.3 (osd.3) 367 : cluster [DBG] 18.1a scrub starts 2024-02-23T00:58:07.472 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:06.660874+0000 osd.3 (osd.3) 368 : cluster [DBG] 18.1a scrub ok 2024-02-23T00:58:07.472 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:07.212407+0000 osd.5 (osd.5) 414 : cluster [DBG] 2.18 scrub starts 2024-02-23T00:58:07.472 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:07.213987+0000 osd.5 (osd.5) 415 : cluster [DBG] 2.18 scrub ok 2024-02-23T00:58:07.473 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:05.448196+0000 osd.6 (osd.6) 375 : cluster [DBG] 18.14 scrub starts 2024-02-23T00:58:07.473 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:05.449354+0000 osd.6 (osd.6) 376 : cluster [DBG] 18.14 scrub ok 2024-02-23T00:58:07.473 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:06.205669+0000 osd.2 (osd.2) 225 : cluster [DBG] 2.1 scrub starts 2024-02-23T00:58:07.473 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:06.206619+0000 osd.2 (osd.2) 226 : cluster [DBG] 2.1 scrub ok 2024-02-23T00:58:07.473 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:06.282702+0000 osd.0 (osd.0) 356 : cluster [DBG] 3.1 deep-scrub starts 2024-02-23T00:58:07.473 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:06.283717+0000 osd.0 (osd.0) 357 : cluster [DBG] 3.1 deep-scrub ok 2024-02-23T00:58:07.473 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:06.659857+0000 osd.3 (osd.3) 367 : cluster [DBG] 18.1a scrub starts 2024-02-23T00:58:07.473 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:06.660874+0000 osd.3 (osd.3) 368 : cluster [DBG] 18.1a scrub ok 2024-02-23T00:58:07.473 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:07.212407+0000 osd.5 (osd.5) 414 : cluster [DBG] 2.18 scrub starts 2024-02-23T00:58:07.473 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:07 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:07.213987+0000 osd.5 (osd.5) 415 : cluster [DBG] 2.18 scrub ok 2024-02-23T00:58:07.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:05.448196+0000 osd.6 (osd.6) 375 : cluster [DBG] 18.14 scrub starts 2024-02-23T00:58:07.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:05.449354+0000 osd.6 (osd.6) 376 : cluster [DBG] 18.14 scrub ok 2024-02-23T00:58:07.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:06.205669+0000 osd.2 (osd.2) 225 : cluster [DBG] 2.1 scrub starts 2024-02-23T00:58:07.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:06.206619+0000 osd.2 (osd.2) 226 : cluster [DBG] 2.1 scrub ok 2024-02-23T00:58:07.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:06.282702+0000 osd.0 (osd.0) 356 : cluster [DBG] 3.1 deep-scrub starts 2024-02-23T00:58:07.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:06.283717+0000 osd.0 (osd.0) 357 : cluster [DBG] 3.1 deep-scrub ok 2024-02-23T00:58:07.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:06.659857+0000 osd.3 (osd.3) 367 : cluster [DBG] 18.1a scrub starts 2024-02-23T00:58:07.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:06.660874+0000 osd.3 (osd.3) 368 : cluster [DBG] 18.1a scrub ok 2024-02-23T00:58:07.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:07.212407+0000 osd.5 (osd.5) 414 : cluster [DBG] 2.18 scrub starts 2024-02-23T00:58:07.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:07 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:07.213987+0000 osd.5 (osd.5) 415 : cluster [DBG] 2.18 scrub ok 2024-02-23T00:58:07.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.922+0000 7f37e5173700 1 -- 172.21.15.47:0/1627471951 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f37e00fcbb0 msgr2=0x7f37e00fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:07.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.922+0000 7f37e5173700 1 --2- 172.21.15.47:0/1627471951 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f37e00fcbb0 0x7f37e00fcf90 secure :-1 s=READY pgs=2869 cs=0 l=1 rev1=1 rx=0x7f37c8004870 tx=0x7f37c8009cb0).stop 2024-02-23T00:58:07.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.922+0000 7f37e5173700 1 -- 172.21.15.47:0/1627471951 shutdown_connections 2024-02-23T00:58:07.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.922+0000 7f37e5173700 1 --2- 172.21.15.47:0/1627471951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37e00f7400 0x7f37e0103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:07.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.922+0000 7f37e5173700 1 --2- 172.21.15.47:0/1627471951 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f37e00fcbb0 0x7f37e00fcf90 unknown :-1 s=CLOSED pgs=2869 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:07.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.922+0000 7f37e5173700 1 --2- 172.21.15.47:0/1627471951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37e00fd510 0x7f37e00f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:07.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.922+0000 7f37e5173700 1 -- 172.21.15.47:0/1627471951 >> 172.21.15.47:0/1627471951 conn(0x7f37e00f2820 msgr2=0x7f37e00f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:58:07.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.922+0000 7f37e5173700 1 -- 172.21.15.47:0/1627471951 shutdown_connections 2024-02-23T00:58:07.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.922+0000 7f37e5173700 1 -- 172.21.15.47:0/1627471951 wait complete. 2024-02-23T00:58:07.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.922+0000 7f37e5173700 1 librados: starting msgr at 2024-02-23T00:58:07.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.922+0000 7f37e5173700 1 librados: starting objecter 2024-02-23T00:58:07.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.923+0000 7f37e5173700 1 Processor -- start 2024-02-23T00:58:07.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.923+0000 7f37e5173700 1 -- start start 2024-02-23T00:58:07.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.923+0000 7f37e5173700 1 librados: setting wanted keys 2024-02-23T00:58:07.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.923+0000 7f37e5173700 1 librados: calling monclient init 2024-02-23T00:58:07.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.923+0000 7f37e5173700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f37e00f7400 0x7f37e010e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:07.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.923+0000 7f37e5173700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37e00fcbb0 0x7f37e010e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:07.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.923+0000 7f37e5173700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37e00fd510 0x7f37e01081b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:07.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.923+0000 7f37e5173700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e01086a0 con 0x7f37e00fd510 2024-02-23T00:58:07.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.923+0000 7f37e5173700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e0108810 con 0x7f37e00fcbb0 2024-02-23T00:58:07.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.923+0000 7f37e5173700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e01089d0 con 0x7f37e00f7400 2024-02-23T00:58:07.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.923+0000 7f37df59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37e00fd510 0x7f37e01081b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:07.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.923+0000 7f37ded9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f37e00f7400 0x7f37e010e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:07.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.923+0000 7f37df59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37e00fd510 0x7f37e01081b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50416/0 (socket says 172.21.15.47:50416) 2024-02-23T00:58:07.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.923+0000 7f37df59e700 1 -- 172.21.15.47:0/3795239260 learned_addr learned my addr 172.21.15.47:0/3795239260 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:58:07.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.924+0000 7f37de59c700 1 --2- 172.21.15.47:0/3795239260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37e00fcbb0 0x7f37e010e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:07.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.924+0000 7f37df59e700 1 -- 172.21.15.47:0/3795239260 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f37e00f7400 msgr2=0x7f37e010e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:07.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.924+0000 7f37df59e700 1 --2- 172.21.15.47:0/3795239260 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f37e00f7400 0x7f37e010e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:07.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.924+0000 7f37df59e700 1 -- 172.21.15.47:0/3795239260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37e00fcbb0 msgr2=0x7f37e010e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:07.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.924+0000 7f37df59e700 1 --2- 172.21.15.47:0/3795239260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37e00fcbb0 0x7f37e010e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:07.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.924+0000 7f37df59e700 1 -- 172.21.15.47:0/3795239260 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d80097c0 con 0x7f37e00fd510 2024-02-23T00:58:07.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.924+0000 7f37ded9d700 1 --2- 172.21.15.47:0/3795239260 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f37e00f7400 0x7f37e010e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:58:07.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.924+0000 7f37df59e700 1 --2- 172.21.15.47:0/3795239260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37e00fd510 0x7f37e01081b0 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 rx=0x7f37d8001df0 tx=0x7f37d8004c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:58:07.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.924+0000 7f37d7fff700 10 client.?.objecter ms_handle_connect 0x7f37e00fd510 2024-02-23T00:58:07.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.924+0000 7f37d7fff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:58:07.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.924+0000 7f37d7fff700 1 -- 172.21.15.47:0/3795239260 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d8009a10 con 0x7f37e00fd510 2024-02-23T00:58:07.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.924+0000 7f37d7fff700 1 -- 172.21.15.47:0/3795239260 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f37d80090d0 con 0x7f37e00fd510 2024-02-23T00:58:07.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.924+0000 7f37e5173700 1 -- 172.21.15.47:0/3795239260 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37c8005040 con 0x7f37e00fd510 2024-02-23T00:58:07.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.924+0000 7f37d7fff700 1 -- 172.21.15.47:0/3795239260 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d8007be0 con 0x7f37e00fd510 2024-02-23T00:58:07.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.924+0000 7f37e5173700 10 client.32085.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:58:07.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.924+0000 7f37e5173700 1 -- 172.21.15.47:0/3795239260 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37e0114e40 con 0x7f37e00fd510 2024-02-23T00:58:07.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.924+0000 7f37e5173700 1 librados: init done 2024-02-23T00:58:07.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.925+0000 7f37e5173700 10 client.32085.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-23T00:58:07.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.925+0000 7f37e5173700 15 client.32085.objecter _session_command_op_assign -1 1 2024-02-23T00:58:07.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.925+0000 7f37e5173700 20 client.32085.objecter _get_session osd=-1 returning homeless 2024-02-23T00:58:07.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.925+0000 7f37e5173700 10 client.32085.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:58:07.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.925+0000 7f37e5173700 1 -- 172.21.15.47:0/3795239260 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f37e005fa60 con 0x7f37e00fd510 2024-02-23T00:58:07.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.926+0000 7f37d7fff700 1 -- 172.21.15.47:0/3795239260 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f37d801f440 con 0x7f37e00fd510 2024-02-23T00:58:07.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.926+0000 7f37d7fff700 1 --2- 172.21.15.47:0/3795239260 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f37cc062cc0 0x7f37cc065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:07.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.926+0000 7f37d7fff700 1 -- 172.21.15.47:0/3795239260 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(858..858 src has 806..858) v4 ==== 8663+0+0 (secure 0 0 0) 0x7f37d807dd90 con 0x7f37e00fd510 2024-02-23T00:58:07.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.926+0000 7f37d7fff700 10 client.32085.objecter ms_dispatch 0x7f37e0001550 osd_map(858..858 src has 806..858) v4 2024-02-23T00:58:07.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.926+0000 7f37d7fff700 3 client.32085.objecter handle_osd_map got epochs [858,858] > 0 2024-02-23T00:58:07.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.926+0000 7f37d7fff700 3 client.32085.objecter handle_osd_map decoding full epoch 858 2024-02-23T00:58:07.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.926+0000 7f37ded9d700 1 --2- 172.21.15.47:0/3795239260 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f37cc062cc0 0x7f37cc065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:07.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.926+0000 7f37d7fff700 10 client.32085.objecter checking command 1 2024-02-23T00:58:07.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.926+0000 7f37d7fff700 1 --2- 172.21.15.47:0/3795239260 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f37cc06e3e0 0x7f37cc070880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:07.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.926+0000 7f37d7fff700 20 client.32085.objecter _get_session s=0x7f37cc06dd10 osd=7 3 2024-02-23T00:58:07.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.926+0000 7f37d7fff700 20 client.32085.objecter put_session s=0x7f37cc06dd10 osd=7 3 2024-02-23T00:58:07.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.926+0000 7f37d7fff700 15 client.32085.objecter _session_command_op_remove -1 1 2024-02-23T00:58:07.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.926+0000 7f37d7fff700 20 client.32085.objecter _get_session s=0x7f37cc06dd10 osd=7 3 2024-02-23T00:58:07.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.926+0000 7f37d7fff700 20 client.32085.objecter get_session s=0x7f37cc06dd10 osd=7 3 2024-02-23T00:58:07.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.926+0000 7f37d7fff700 15 client.32085.objecter _session_command_op_assign 7 1 2024-02-23T00:58:07.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.926+0000 7f37d7fff700 20 client.32085.objecter put_session s=0x7f37cc06dd10 osd=7 4 2024-02-23T00:58:07.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.926+0000 7f37d7fff700 10 client.32085.objecter _send_command 1 2024-02-23T00:58:07.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.926+0000 7f37d7fff700 1 -- 172.21.15.47:0/3795239260 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f37cc070f00 con 0x7f37cc06e3e0 2024-02-23T00:58:07.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.926+0000 7f37d7fff700 20 client.32085.objecter dump_active .. 0 homeless 2024-02-23T00:58:07.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.926+0000 7f37d7fff700 1 -- 172.21.15.47:0/3795239260 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=858) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37d80803b0 con 0x7f37e00fd510 2024-02-23T00:58:07.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.926+0000 7f37ded9d700 1 --2- 172.21.15.47:0/3795239260 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f37cc062cc0 0x7f37cc065180 secure :-1 s=READY pgs=3251 cs=0 l=1 rev1=1 rx=0x7f37c8005010 tx=0x7f37c8009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:58:07.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.926+0000 7f37d7fff700 10 client.32085.objecter ms_handle_connect 0x7f37cc062cc0 2024-02-23T00:58:07.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.927+0000 7f37de59c700 1 --2- 172.21.15.47:0/3795239260 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f37cc06e3e0 0x7f37cc070880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:07.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.927+0000 7f37d7fff700 10 client.32085.objecter ms_handle_connect 0x7f37cc06e3e0 2024-02-23T00:58:07.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.927+0000 7f37de59c700 1 --2- 172.21.15.47:0/3795239260 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f37cc06e3e0 0x7f37cc070880 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:58:07.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.928+0000 7f37d7fff700 1 -- 172.21.15.47:0/3795239260 <== osd.7 v2:172.21.15.196:6824/2261821675 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f37cc070f00 con 0x7f37cc06e3e0 2024-02-23T00:58:07.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.928+0000 7f37d7fff700 10 client.32085.objecter ms_dispatch 0x7f37e0001550 command_reply(tid 1: 0 ) v1 2024-02-23T00:58:07.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.928+0000 7f37d7fff700 10 client.32085.objecter _finish_command 1 = system:0 2024-02-23T00:58:07.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.928+0000 7f37d7fff700 20 client.32085.objecter put_session s=0x7f37cc06dd10 osd=7 4 2024-02-23T00:58:07.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.928+0000 7f37d7fff700 15 client.32085.objecter _session_command_op_remove 7 1 2024-02-23T00:58:07.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.942+0000 7f37e5173700 10 client.32085.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-02-23T00:58:07.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.942+0000 7f37e5173700 15 client.32085.objecter _session_command_op_assign -1 2 2024-02-23T00:58:07.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.942+0000 7f37e5173700 20 client.32085.objecter _get_session s=0x7f37cc06dd10 osd=7 3 2024-02-23T00:58:07.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.942+0000 7f37e5173700 20 client.32085.objecter put_session s=0x7f37cc06dd10 osd=7 3 2024-02-23T00:58:07.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.942+0000 7f37e5173700 20 client.32085.objecter _get_session s=0x7f37cc06dd10 osd=7 3 2024-02-23T00:58:07.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.942+0000 7f37e5173700 15 client.32085.objecter _session_command_op_remove -1 2 2024-02-23T00:58:07.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.942+0000 7f37e5173700 20 client.32085.objecter get_session s=0x7f37cc06dd10 osd=7 3 2024-02-23T00:58:07.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.942+0000 7f37e5173700 15 client.32085.objecter _session_command_op_assign 7 2 2024-02-23T00:58:07.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.942+0000 7f37e5173700 20 client.32085.objecter put_session s=0x7f37cc06dd10 osd=7 4 2024-02-23T00:58:07.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.942+0000 7f37e5173700 10 client.32085.objecter _send_command 2 2024-02-23T00:58:07.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.942+0000 7f37e5173700 1 -- 172.21.15.47:0/3795239260 --> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f37e005ee80 con 0x7f37cc06e3e0 2024-02-23T00:58:07.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.942+0000 7f37d7fff700 1 -- 172.21.15.47:0/3795239260 <== osd.7 v2:172.21.15.196:6824/2261821675 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f37e005ee80 con 0x7f37cc06e3e0 2024-02-23T00:58:07.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.942+0000 7f37d7fff700 10 client.32085.objecter ms_dispatch 0x7f37e0001550 command_reply(tid 2: 0 ) v1 2024-02-23T00:58:07.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.942+0000 7f37d7fff700 10 client.32085.objecter _finish_command 2 = system:0 2024-02-23T00:58:07.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.942+0000 7f37d7fff700 20 client.32085.objecter put_session s=0x7f37cc06dd10 osd=7 4 2024-02-23T00:58:07.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.942+0000 7f37d7fff700 15 client.32085.objecter _session_command_op_remove 7 2 2024-02-23T00:58:07.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.942+0000 7f37e5173700 10 librados: watch_flush enter 2024-02-23T00:58:07.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.942+0000 7f37e5173700 10 librados: watch_flush exit 2024-02-23T00:58:07.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.942+0000 7f37e5173700 10 client.32085.objecter close_session for osd.7 2024-02-23T00:58:07.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.942+0000 7f37e5173700 1 -- 172.21.15.47:0/3795239260 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f37cc06e3e0 msgr2=0x7f37cc070880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:07.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.943+0000 7f37e5173700 1 --2- 172.21.15.47:0/3795239260 >> [v2:172.21.15.196:6824/2261821675,v1:172.21.15.196:6825/2261821675] conn(0x7f37cc06e3e0 0x7f37cc070880 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:07.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.943+0000 7f37e5173700 20 client.32085.objecter put_session s=0x7f37cc06dd10 osd=7 1 2024-02-23T00:58:07.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.943+0000 7f37e5173700 20 client.32085.objecter shutdown clearing up homeless session... 2024-02-23T00:58:07.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.943+0000 7f37e5173700 10 client.32085.objecter successfully canceled tick 2024-02-23T00:58:07.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.943+0000 7f37e5173700 1 -- 172.21.15.47:0/3795239260 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f37cc062cc0 msgr2=0x7f37cc065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:07.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.943+0000 7f37e5173700 1 --2- 172.21.15.47:0/3795239260 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f37cc062cc0 0x7f37cc065180 secure :-1 s=READY pgs=3251 cs=0 l=1 rev1=1 rx=0x7f37c8005010 tx=0x7f37c8009cb0).stop 2024-02-23T00:58:07.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.943+0000 7f37e5173700 1 -- 172.21.15.47:0/3795239260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37e00fd510 msgr2=0x7f37e01081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:07.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.943+0000 7f37e5173700 1 --2- 172.21.15.47:0/3795239260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37e00fd510 0x7f37e01081b0 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 rx=0x7f37d8001df0 tx=0x7f37d8004c50).stop 2024-02-23T00:58:07.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.943+0000 7f37df59e700 1 -- 172.21.15.47:0/3795239260 reap_dead start 2024-02-23T00:58:07.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.943+0000 7f37e5173700 1 -- 172.21.15.47:0/3795239260 shutdown_connections 2024-02-23T00:58:07.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.943+0000 7f37e5173700 1 -- 172.21.15.47:0/3795239260 >> 172.21.15.47:0/3795239260 conn(0x7f37e00f2820 msgr2=0x7f37e00f4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:58:07.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.943+0000 7f37e5173700 1 -- 172.21.15.47:0/3795239260 shutdown_connections 2024-02-23T00:58:07.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.943+0000 7f37e5173700 1 -- 172.21.15.47:0/3795239260 wait complete. 2024-02-23T00:58:07.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:07.943+0000 7f37e5173700 1 librados: shutdown 2024-02-23T00:58:08.609 INFO:teuthology.orchestra.run.smithi047.stdout:231928234215 2024-02-23T00:58:08.610 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-23T00:58:08.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:06.461811+0000 osd.6 (osd.6) 377 : cluster [DBG] 4.5 scrub starts 2024-02-23T00:58:08.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:06.463814+0000 osd.6 (osd.6) 378 : cluster [DBG] 4.5 scrub ok 2024-02-23T00:58:08.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:07.415940+0000 osd.4 (osd.4) 405 : cluster [DBG] 5.1 scrub starts 2024-02-23T00:58:08.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:07.417140+0000 osd.4 (osd.4) 406 : cluster [DBG] 5.1 scrub ok 2024-02-23T00:58:08.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:08 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:07.441286+0000 mgr.y (mgr.14517) 756 : cluster [DBG] pgmap v1341: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:58:08.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:06.461811+0000 osd.6 (osd.6) 377 : cluster [DBG] 4.5 scrub starts 2024-02-23T00:58:08.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:06.463814+0000 osd.6 (osd.6) 378 : cluster [DBG] 4.5 scrub ok 2024-02-23T00:58:08.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:07.415940+0000 osd.4 (osd.4) 405 : cluster [DBG] 5.1 scrub starts 2024-02-23T00:58:08.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:07.417140+0000 osd.4 (osd.4) 406 : cluster [DBG] 5.1 scrub ok 2024-02-23T00:58:08.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:07.441286+0000 mgr.y (mgr.14517) 756 : cluster [DBG] pgmap v1341: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:58:08.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:06.461811+0000 osd.6 (osd.6) 377 : cluster [DBG] 4.5 scrub starts 2024-02-23T00:58:08.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:06.463814+0000 osd.6 (osd.6) 378 : cluster [DBG] 4.5 scrub ok 2024-02-23T00:58:08.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:07.415940+0000 osd.4 (osd.4) 405 : cluster [DBG] 5.1 scrub starts 2024-02-23T00:58:08.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:07.417140+0000 osd.4 (osd.4) 406 : cluster [DBG] 5.1 scrub ok 2024-02-23T00:58:08.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:08 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:07.441286+0000 mgr.y (mgr.14517) 756 : cluster [DBG] pgmap v1341: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:58:08.891 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:58:09.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:08.261601+0000 osd.0 (osd.0) 358 : cluster [DBG] 5.5 scrub starts 2024-02-23T00:58:09.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:08.262656+0000 osd.0 (osd.0) 359 : cluster [DBG] 5.5 scrub ok 2024-02-23T00:58:09.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:08.529816+0000 osd.6 (osd.6) 379 : cluster [DBG] 5.12 deep-scrub starts 2024-02-23T00:58:09.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:08.530787+0000 osd.6 (osd.6) 380 : cluster [DBG] 5.12 deep-scrub ok 2024-02-23T00:58:09.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:08.754269+0000 osd.3 (osd.3) 369 : cluster [DBG] 5.14 scrub starts 2024-02-23T00:58:09.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:09 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:08.755926+0000 osd.3 (osd.3) 370 : cluster [DBG] 5.14 scrub ok 2024-02-23T00:58:09.713 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:08.261601+0000 osd.0 (osd.0) 358 : cluster [DBG] 5.5 scrub starts 2024-02-23T00:58:09.714 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:08.262656+0000 osd.0 (osd.0) 359 : cluster [DBG] 5.5 scrub ok 2024-02-23T00:58:09.714 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:08.529816+0000 osd.6 (osd.6) 379 : cluster [DBG] 5.12 deep-scrub starts 2024-02-23T00:58:09.714 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:08.530787+0000 osd.6 (osd.6) 380 : cluster [DBG] 5.12 deep-scrub ok 2024-02-23T00:58:09.714 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:08.754269+0000 osd.3 (osd.3) 369 : cluster [DBG] 5.14 scrub starts 2024-02-23T00:58:09.715 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:08.755926+0000 osd.3 (osd.3) 370 : cluster [DBG] 5.14 scrub ok 2024-02-23T00:58:09.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:08.261601+0000 osd.0 (osd.0) 358 : cluster [DBG] 5.5 scrub starts 2024-02-23T00:58:09.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:08.262656+0000 osd.0 (osd.0) 359 : cluster [DBG] 5.5 scrub ok 2024-02-23T00:58:09.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:08.529816+0000 osd.6 (osd.6) 379 : cluster [DBG] 5.12 deep-scrub starts 2024-02-23T00:58:09.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:08.530787+0000 osd.6 (osd.6) 380 : cluster [DBG] 5.12 deep-scrub ok 2024-02-23T00:58:09.716 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:08.754269+0000 osd.3 (osd.3) 369 : cluster [DBG] 5.14 scrub starts 2024-02-23T00:58:09.716 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:09 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:08.755926+0000 osd.3 (osd.3) 370 : cluster [DBG] 5.14 scrub ok 2024-02-23T00:58:10.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.626+0000 7f5a7fb01700 1 -- 172.21.15.47:0/17531818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a780f8cb0 msgr2=0x7f5a780f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:10.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.626+0000 7f5a7fb01700 1 --2- 172.21.15.47:0/17531818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a780f8cb0 0x7f5a780f9090 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 rx=0x7f5a68004870 tx=0x7f5a68009cb0).stop 2024-02-23T00:58:10.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.626+0000 7f5a7fb01700 1 -- 172.21.15.47:0/17531818 shutdown_connections 2024-02-23T00:58:10.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.626+0000 7f5a7fb01700 1 --2- 172.21.15.47:0/17531818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a78102210 0x7f5a78104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:10.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.626+0000 7f5a7fb01700 1 --2- 172.21.15.47:0/17531818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a780f8cb0 0x7f5a780f9090 unknown :-1 s=CLOSED pgs=2683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:10.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.626+0000 7f5a7fb01700 1 --2- 172.21.15.47:0/17531818 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a780f9610 0x7f5a78101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:10.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.626+0000 7f5a7fb01700 1 -- 172.21.15.47:0/17531818 >> 172.21.15.47:0/17531818 conn(0x7f5a780f2820 msgr2=0x7f5a780f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:58:10.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.626+0000 7f5a7fb01700 1 -- 172.21.15.47:0/17531818 shutdown_connections 2024-02-23T00:58:10.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.626+0000 7f5a7fb01700 1 -- 172.21.15.47:0/17531818 wait complete. 2024-02-23T00:58:10.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.627+0000 7f5a7fb01700 1 librados: starting msgr at 2024-02-23T00:58:10.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.627+0000 7f5a7fb01700 1 librados: starting objecter 2024-02-23T00:58:10.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.627+0000 7f5a7fb01700 1 Processor -- start 2024-02-23T00:58:10.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.627+0000 7f5a7fb01700 1 -- start start 2024-02-23T00:58:10.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.627+0000 7f5a7fb01700 1 librados: setting wanted keys 2024-02-23T00:58:10.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.627+0000 7f5a7fb01700 1 librados: calling monclient init 2024-02-23T00:58:10.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.627+0000 7f5a7fb01700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a780f8cb0 0x7f5a7810e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:10.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.627+0000 7f5a7fb01700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a780f9610 0x7f5a7810e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:10.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.627+0000 7f5a7fb01700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a78102210 0x7f5a78108260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:10.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.627+0000 7f5a7fb01700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a78108750 con 0x7f5a780f8cb0 2024-02-23T00:58:10.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.627+0000 7f5a7fb01700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a781088c0 con 0x7f5a780f9610 2024-02-23T00:58:10.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.627+0000 7f5a7fb01700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a78108a80 con 0x7f5a78102210 2024-02-23T00:58:10.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.627+0000 7f5a7e09e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a78102210 0x7f5a78108260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:10.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.627+0000 7f5a7e09e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a78102210 0x7f5a78108260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:54996/0 (socket says 172.21.15.47:54996) 2024-02-23T00:58:10.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.627+0000 7f5a7d89d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a780f8cb0 0x7f5a7810e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:10.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.627+0000 7f5a7e09e700 1 -- 172.21.15.47:0/3028124125 learned_addr learned my addr 172.21.15.47:0/3028124125 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:58:10.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.627+0000 7f5a7d09c700 1 --2- 172.21.15.47:0/3028124125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a780f9610 0x7f5a7810e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:10.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.627+0000 7f5a7e09e700 1 -- 172.21.15.47:0/3028124125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a780f9610 msgr2=0x7f5a7810e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:10.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.627+0000 7f5a7e09e700 1 --2- 172.21.15.47:0/3028124125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a780f9610 0x7f5a7810e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:10.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.627+0000 7f5a7e09e700 1 -- 172.21.15.47:0/3028124125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a780f8cb0 msgr2=0x7f5a7810e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:10.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.627+0000 7f5a7e09e700 1 --2- 172.21.15.47:0/3028124125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a780f8cb0 0x7f5a7810e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:10.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.627+0000 7f5a7e09e700 1 -- 172.21.15.47:0/3028124125 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a68005040 con 0x7f5a78102210 2024-02-23T00:58:10.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.628+0000 7f5a7d89d700 1 --2- 172.21.15.47:0/3028124125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a780f8cb0 0x7f5a7810e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:58:10.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.628+0000 7f5a7e09e700 1 --2- 172.21.15.47:0/3028124125 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a78102210 0x7f5a78108260 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 rx=0x7f5a7000d940 tx=0x7f5a70004e40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:58:10.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.628+0000 7f5a6effd700 10 client.?.objecter ms_handle_connect 0x7f5a78102210 2024-02-23T00:58:10.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.628+0000 7f5a6effd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:58:10.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.628+0000 7f5a6effd700 1 -- 172.21.15.47:0/3028124125 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a70004020 con 0x7f5a78102210 2024-02-23T00:58:10.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.628+0000 7f5a7fb01700 1 -- 172.21.15.47:0/3028124125 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a78108d60 con 0x7f5a78102210 2024-02-23T00:58:10.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.628+0000 7f5a6effd700 1 -- 172.21.15.47:0/3028124125 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f5a7000f040 con 0x7f5a78102210 2024-02-23T00:58:10.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.628+0000 7f5a6effd700 1 -- 172.21.15.47:0/3028124125 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a70003430 con 0x7f5a78102210 2024-02-23T00:58:10.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.628+0000 7f5a7fb01700 10 client.42413.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:58:10.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.628+0000 7f5a7fb01700 1 -- 172.21.15.47:0/3028124125 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a781090e0 con 0x7f5a78102210 2024-02-23T00:58:10.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.628+0000 7f5a7fb01700 1 librados: init done 2024-02-23T00:58:10.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.629+0000 7f5a7fb01700 1 -- 172.21.15.47:0/3028124125 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a7805fa60 con 0x7f5a78102210 2024-02-23T00:58:10.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.629+0000 7f5a6effd700 1 -- 172.21.15.47:0/3028124125 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f5a70004180 con 0x7f5a78102210 2024-02-23T00:58:10.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.630+0000 7f5a6effd700 1 --2- 172.21.15.47:0/3028124125 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5a64062cc0 0x7f5a64065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:10.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.630+0000 7f5a6effd700 1 -- 172.21.15.47:0/3028124125 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(858..858 src has 806..858) v4 ==== 8663+0+0 (secure 0 0 0) 0x7f5a7007ce20 con 0x7f5a78102210 2024-02-23T00:58:10.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.630+0000 7f5a7d89d700 1 --2- 172.21.15.47:0/3028124125 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5a64062cc0 0x7f5a64065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:10.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.630+0000 7f5a6effd700 10 client.42413.objecter ms_dispatch 0x7f5a78001550 osd_map(858..858 src has 806..858) v4 2024-02-23T00:58:10.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.630+0000 7f5a6effd700 3 client.42413.objecter handle_osd_map got epochs [858,858] > 0 2024-02-23T00:58:10.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.630+0000 7f5a6effd700 3 client.42413.objecter handle_osd_map decoding full epoch 858 2024-02-23T00:58:10.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.630+0000 7f5a6effd700 20 client.42413.objecter dump_active .. 0 homeless 2024-02-23T00:58:10.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.630+0000 7f5a6effd700 10 client.42413.objecter ms_handle_connect 0x7f5a64062cc0 2024-02-23T00:58:10.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.630+0000 7f5a7d89d700 1 --2- 172.21.15.47:0/3028124125 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5a64062cc0 0x7f5a64065180 secure :-1 s=READY pgs=3252 cs=0 l=1 rev1=1 rx=0x7f5a68004ee0 tx=0x7f5a6800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:58:10.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.634+0000 7f5a6effd700 1 -- 172.21.15.47:0/3028124125 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f5a70048f60 con 0x7f5a78102210 2024-02-23T00:58:10.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:09.246230+0000 osd.0 (osd.0) 360 : cluster [DBG] 5.1e deep-scrub starts 2024-02-23T00:58:10.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:09.247196+0000 osd.0 (osd.0) 361 : cluster [DBG] 5.1e deep-scrub ok 2024-02-23T00:58:10.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:09.441821+0000 mgr.y (mgr.14517) 757 : cluster [DBG] pgmap v1342: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:58:10.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:09.801605+0000 osd.1 (osd.1) 358 : cluster [DBG] 18.1c scrub starts 2024-02-23T00:58:10.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:09.804773+0000 osd.1 (osd.1) 359 : cluster [DBG] 18.1c scrub ok 2024-02-23T00:58:10.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:10.258084+0000 osd.5 (osd.5) 416 : cluster [DBG] 5.24 scrub starts 2024-02-23T00:58:10.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:10 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:10.261107+0000 osd.5 (osd.5) 417 : cluster [DBG] 5.24 scrub ok 2024-02-23T00:58:10.807 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:58:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:58:10] "GET /metrics HTTP/1.1" 200 187139 "" "Prometheus/2.33.4" 2024-02-23T00:58:10.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:09.246230+0000 osd.0 (osd.0) 360 : cluster [DBG] 5.1e deep-scrub starts 2024-02-23T00:58:10.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:09.247196+0000 osd.0 (osd.0) 361 : cluster [DBG] 5.1e deep-scrub ok 2024-02-23T00:58:10.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:09.441821+0000 mgr.y (mgr.14517) 757 : cluster [DBG] pgmap v1342: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:58:10.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:09.801605+0000 osd.1 (osd.1) 358 : cluster [DBG] 18.1c scrub starts 2024-02-23T00:58:10.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:09.804773+0000 osd.1 (osd.1) 359 : cluster [DBG] 18.1c scrub ok 2024-02-23T00:58:10.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:10.258084+0000 osd.5 (osd.5) 416 : cluster [DBG] 5.24 scrub starts 2024-02-23T00:58:10.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:10.261107+0000 osd.5 (osd.5) 417 : cluster [DBG] 5.24 scrub ok 2024-02-23T00:58:10.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:09.246230+0000 osd.0 (osd.0) 360 : cluster [DBG] 5.1e deep-scrub starts 2024-02-23T00:58:10.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:09.247196+0000 osd.0 (osd.0) 361 : cluster [DBG] 5.1e deep-scrub ok 2024-02-23T00:58:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:09.441821+0000 mgr.y (mgr.14517) 757 : cluster [DBG] pgmap v1342: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:58:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:09.801605+0000 osd.1 (osd.1) 358 : cluster [DBG] 18.1c scrub starts 2024-02-23T00:58:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:09.804773+0000 osd.1 (osd.1) 359 : cluster [DBG] 18.1c scrub ok 2024-02-23T00:58:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:10.258084+0000 osd.5 (osd.5) 416 : cluster [DBG] 5.24 scrub starts 2024-02-23T00:58:10.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:10.261107+0000 osd.5 (osd.5) 417 : cluster [DBG] 5.24 scrub ok 2024-02-23T00:58:10.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.805+0000 7f5a7fb01700 1 -- 172.21.15.47:0/3028124125 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5a781070b0 con 0x7f5a78102210 2024-02-23T00:58:10.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.805+0000 7f5a6effd700 1 -- 172.21.15.47:0/3028124125 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5a7004c5e0 con 0x7f5a78102210 2024-02-23T00:58:10.810 INFO:teuthology.orchestra.run.smithi047.stdout:34359738638 2024-02-23T00:58:10.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.807+0000 7f5a7fb01700 10 librados: watch_flush enter 2024-02-23T00:58:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.807+0000 7f5a7fb01700 10 librados: watch_flush exit 2024-02-23T00:58:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.807+0000 7f5a7fb01700 20 client.42413.objecter shutdown clearing up homeless session... 2024-02-23T00:58:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.807+0000 7f5a7fb01700 10 client.42413.objecter successfully canceled tick 2024-02-23T00:58:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.807+0000 7f5a7fb01700 1 -- 172.21.15.47:0/3028124125 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5a64062cc0 msgr2=0x7f5a64065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.807+0000 7f5a7fb01700 1 --2- 172.21.15.47:0/3028124125 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5a64062cc0 0x7f5a64065180 secure :-1 s=READY pgs=3252 cs=0 l=1 rev1=1 rx=0x7f5a68004ee0 tx=0x7f5a6800b040).stop 2024-02-23T00:58:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.807+0000 7f5a7fb01700 1 -- 172.21.15.47:0/3028124125 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a78102210 msgr2=0x7f5a78108260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.807+0000 7f5a7fb01700 1 --2- 172.21.15.47:0/3028124125 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a78102210 0x7f5a78108260 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 rx=0x7f5a7000d940 tx=0x7f5a70004e40).stop 2024-02-23T00:58:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.807+0000 7f5a7fb01700 1 -- 172.21.15.47:0/3028124125 shutdown_connections 2024-02-23T00:58:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.807+0000 7f5a7fb01700 1 --2- 172.21.15.47:0/3028124125 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f5a64062cc0 0x7f5a64065180 unknown :-1 s=CLOSED pgs=3252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.807+0000 7f5a7fb01700 1 --2- 172.21.15.47:0/3028124125 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a78102210 0x7f5a78108260 unknown :-1 s=CLOSED pgs=2870 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.807+0000 7f5a7fb01700 1 --2- 172.21.15.47:0/3028124125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a780f8cb0 0x7f5a7810e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.807+0000 7f5a7fb01700 1 --2- 172.21.15.47:0/3028124125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a780f9610 0x7f5a7810e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.807+0000 7f5a7fb01700 1 -- 172.21.15.47:0/3028124125 >> 172.21.15.47:0/3028124125 conn(0x7f5a780f2820 msgr2=0x7f5a780f34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:58:10.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.807+0000 7f5a7fb01700 1 -- 172.21.15.47:0/3028124125 shutdown_connections 2024-02-23T00:58:10.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.807+0000 7f5a7fb01700 1 -- 172.21.15.47:0/3028124125 wait complete. 2024-02-23T00:58:10.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:10.807+0000 7f5a7fb01700 1 librados: shutdown 2024-02-23T00:58:11.552 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738634 got 34359738638 for osd.0 2024-02-23T00:58:11.552 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-23T00:58:11.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:10.758321+0000 osd.1 (osd.1) 360 : cluster [DBG] 4.12 scrub starts 2024-02-23T00:58:11.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:10.761787+0000 osd.1 (osd.1) 361 : cluster [DBG] 4.12 scrub ok 2024-02-23T00:58:11.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:58:10.806533+0000 mon.c (mon.2) 537 : audit [DBG] from='client.? 172.21.15.47:0/3028124125' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-23T00:58:11.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:11.260870+0000 osd.5 (osd.5) 418 : cluster [DBG] 2.12 scrub starts 2024-02-23T00:58:11.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:11 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:11.262034+0000 osd.5 (osd.5) 419 : cluster [DBG] 2.12 scrub ok 2024-02-23T00:58:11.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:10.758321+0000 osd.1 (osd.1) 360 : cluster [DBG] 4.12 scrub starts 2024-02-23T00:58:11.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:10.761787+0000 osd.1 (osd.1) 361 : cluster [DBG] 4.12 scrub ok 2024-02-23T00:58:11.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:58:10.806533+0000 mon.c (mon.2) 537 : audit [DBG] from='client.? 172.21.15.47:0/3028124125' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-23T00:58:11.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:11.260870+0000 osd.5 (osd.5) 418 : cluster [DBG] 2.12 scrub starts 2024-02-23T00:58:11.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:11.262034+0000 osd.5 (osd.5) 419 : cluster [DBG] 2.12 scrub ok 2024-02-23T00:58:11.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:10.758321+0000 osd.1 (osd.1) 360 : cluster [DBG] 4.12 scrub starts 2024-02-23T00:58:11.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:10.761787+0000 osd.1 (osd.1) 361 : cluster [DBG] 4.12 scrub ok 2024-02-23T00:58:11.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:58:10.806533+0000 mon.c (mon.2) 537 : audit [DBG] from='client.? 172.21.15.47:0/3028124125' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-23T00:58:11.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:11.260870+0000 osd.5 (osd.5) 418 : cluster [DBG] 2.12 scrub starts 2024-02-23T00:58:11.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:11.262034+0000 osd.5 (osd.5) 419 : cluster [DBG] 2.12 scrub ok 2024-02-23T00:58:11.834 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:58:12.694 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:58:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:58:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:58:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:11.287868+0000 osd.7 (osd.7) 437 : cluster [DBG] 5.9 scrub starts 2024-02-23T00:58:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:11.291338+0000 osd.7 (osd.7) 438 : cluster [DBG] 5.9 scrub ok 2024-02-23T00:58:12.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:11.442202+0000 mgr.y (mgr.14517) 758 : cluster [DBG] pgmap v1343: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:58:12.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:12.262268+0000 osd.5 (osd.5) 420 : cluster [DBG] 2.16 scrub starts 2024-02-23T00:58:12.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:12 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:12.265300+0000 osd.5 (osd.5) 421 : cluster [DBG] 2.16 scrub ok 2024-02-23T00:58:12.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:11.287868+0000 osd.7 (osd.7) 437 : cluster [DBG] 5.9 scrub starts 2024-02-23T00:58:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:11.291338+0000 osd.7 (osd.7) 438 : cluster [DBG] 5.9 scrub ok 2024-02-23T00:58:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:11.442202+0000 mgr.y (mgr.14517) 758 : cluster [DBG] pgmap v1343: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:58:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:12.262268+0000 osd.5 (osd.5) 420 : cluster [DBG] 2.16 scrub starts 2024-02-23T00:58:12.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:12.265300+0000 osd.5 (osd.5) 421 : cluster [DBG] 2.16 scrub ok 2024-02-23T00:58:12.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:11.287868+0000 osd.7 (osd.7) 437 : cluster [DBG] 5.9 scrub starts 2024-02-23T00:58:12.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:11.291338+0000 osd.7 (osd.7) 438 : cluster [DBG] 5.9 scrub ok 2024-02-23T00:58:12.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:11.442202+0000 mgr.y (mgr.14517) 758 : cluster [DBG] pgmap v1343: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:58:12.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:12.262268+0000 osd.5 (osd.5) 420 : cluster [DBG] 2.16 scrub starts 2024-02-23T00:58:12.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:12 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:12.265300+0000 osd.5 (osd.5) 421 : cluster [DBG] 2.16 scrub ok 2024-02-23T00:58:13.413 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:58:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:58:13.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.531+0000 7f0e71a7d700 1 -- 172.21.15.47:0/1391325656 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0e6c0f8420 msgr2=0x7f0e6c104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:13.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.531+0000 7f0e71a7d700 1 --2- 172.21.15.47:0/1391325656 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0e6c0f8420 0x7f0e6c104a20 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 rx=0x7f0e64009be0 tx=0x7f0e64004d10).stop 2024-02-23T00:58:13.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.531+0000 7f0e71a7d700 1 -- 172.21.15.47:0/1391325656 shutdown_connections 2024-02-23T00:58:13.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.531+0000 7f0e71a7d700 1 --2- 172.21.15.47:0/1391325656 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0e6c0f8420 0x7f0e6c104a20 unknown :-1 s=CLOSED pgs=2871 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:13.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.531+0000 7f0e71a7d700 1 --2- 172.21.15.47:0/1391325656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e6c0fdbb0 0x7f0e6c0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:13.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.531+0000 7f0e71a7d700 1 --2- 172.21.15.47:0/1391325656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e6c0fe510 0x7f0e6c0f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:13.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.531+0000 7f0e71a7d700 1 -- 172.21.15.47:0/1391325656 >> 172.21.15.47:0/1391325656 conn(0x7f0e6c0f3800 msgr2=0x7f0e6c0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:58:13.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.531+0000 7f0e71a7d700 1 -- 172.21.15.47:0/1391325656 shutdown_connections 2024-02-23T00:58:13.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.531+0000 7f0e71a7d700 1 -- 172.21.15.47:0/1391325656 wait complete. 2024-02-23T00:58:13.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.532+0000 7f0e71a7d700 1 librados: starting msgr at 2024-02-23T00:58:13.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.532+0000 7f0e71a7d700 1 librados: starting objecter 2024-02-23T00:58:13.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.532+0000 7f0e71a7d700 1 Processor -- start 2024-02-23T00:58:13.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.532+0000 7f0e71a7d700 1 -- start start 2024-02-23T00:58:13.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.532+0000 7f0e71a7d700 1 librados: setting wanted keys 2024-02-23T00:58:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.532+0000 7f0e71a7d700 1 librados: calling monclient init 2024-02-23T00:58:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.533+0000 7f0e71a7d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e6c0f8420 0x7f0e6c10dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.533+0000 7f0e71a7d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0e6c0fdbb0 0x7f0e6c10e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.533+0000 7f0e71a7d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e6c0fe510 0x7f0e6c107f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.533+0000 7f0e71a7d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c108430 con 0x7f0e6c0f8420 2024-02-23T00:58:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.533+0000 7f0e71a7d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c1085a0 con 0x7f0e6c0fe510 2024-02-23T00:58:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.533+0000 7f0e71a7d700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c108760 con 0x7f0e6c0fdbb0 2024-02-23T00:58:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.533+0000 7f0e6b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e6c0f8420 0x7f0e6c10dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.533+0000 7f0e6affd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0e6c0fdbb0 0x7f0e6c10e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.533+0000 7f0e6b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e6c0f8420 0x7f0e6c10dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50450/0 (socket says 172.21.15.47:50450) 2024-02-23T00:58:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.533+0000 7f0e6b7fe700 1 -- 172.21.15.47:0/2772097972 learned_addr learned my addr 172.21.15.47:0/2772097972 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:58:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.533+0000 7f0e6bfff700 1 --2- 172.21.15.47:0/2772097972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e6c0fe510 0x7f0e6c107f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:13.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.533+0000 7f0e6b7fe700 1 -- 172.21.15.47:0/2772097972 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0e6c0fdbb0 msgr2=0x7f0e6c10e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:13.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.533+0000 7f0e6b7fe700 1 --2- 172.21.15.47:0/2772097972 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0e6c0fdbb0 0x7f0e6c10e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:13.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.533+0000 7f0e6b7fe700 1 -- 172.21.15.47:0/2772097972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e6c0fe510 msgr2=0x7f0e6c107f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:13.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.533+0000 7f0e6b7fe700 1 --2- 172.21.15.47:0/2772097972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e6c0fe510 0x7f0e6c107f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:13.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.533+0000 7f0e6b7fe700 1 -- 172.21.15.47:0/2772097972 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e640097c0 con 0x7f0e6c0f8420 2024-02-23T00:58:13.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.533+0000 7f0e6b7fe700 1 --2- 172.21.15.47:0/2772097972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e6c0f8420 0x7f0e6c10dda0 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 rx=0x7f0e5c00e970 tx=0x7f0e5c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:58:13.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.533+0000 7f0e68f74700 10 client.?.objecter ms_handle_connect 0x7f0e6c0f8420 2024-02-23T00:58:13.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.533+0000 7f0e68f74700 10 client.?.objecter resend_mon_ops 2024-02-23T00:58:13.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.533+0000 7f0e68f74700 1 -- 172.21.15.47:0/2772097972 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c004660 con 0x7f0e6c0f8420 2024-02-23T00:58:13.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.534+0000 7f0e71a7d700 1 -- 172.21.15.47:0/2772097972 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e6c108a40 con 0x7f0e6c0f8420 2024-02-23T00:58:13.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.534+0000 7f0e68f74700 1 -- 172.21.15.47:0/2772097972 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f0e5c00f040 con 0x7f0e6c0f8420 2024-02-23T00:58:13.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.534+0000 7f0e68f74700 1 -- 172.21.15.47:0/2772097972 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c0146f0 con 0x7f0e6c0f8420 2024-02-23T00:58:13.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.534+0000 7f0e71a7d700 10 client.32097.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:58:13.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.534+0000 7f0e71a7d700 1 -- 172.21.15.47:0/2772097972 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e6c108dc0 con 0x7f0e6c0f8420 2024-02-23T00:58:13.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.534+0000 7f0e71a7d700 1 librados: init done 2024-02-23T00:58:13.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.534+0000 7f0e71a7d700 1 -- 172.21.15.47:0/2772097972 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e6c106ed0 con 0x7f0e6c0f8420 2024-02-23T00:58:13.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.535+0000 7f0e68f74700 1 -- 172.21.15.47:0/2772097972 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f0e5c007b40 con 0x7f0e6c0f8420 2024-02-23T00:58:13.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.535+0000 7f0e68f74700 1 --2- 172.21.15.47:0/2772097972 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0e58062cc0 0x7f0e58065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:13.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.535+0000 7f0e68f74700 1 -- 172.21.15.47:0/2772097972 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(858..858 src has 806..858) v4 ==== 8663+0+0 (secure 0 0 0) 0x7f0e5c07cf10 con 0x7f0e6c0f8420 2024-02-23T00:58:13.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.535+0000 7f0e68f74700 10 client.32097.objecter ms_dispatch 0x7f0e6c001550 osd_map(858..858 src has 806..858) v4 2024-02-23T00:58:13.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.535+0000 7f0e68f74700 3 client.32097.objecter handle_osd_map got epochs [858,858] > 0 2024-02-23T00:58:13.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.535+0000 7f0e68f74700 3 client.32097.objecter handle_osd_map decoding full epoch 858 2024-02-23T00:58:13.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.535+0000 7f0e6affd700 1 --2- 172.21.15.47:0/2772097972 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0e58062cc0 0x7f0e58065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:13.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.536+0000 7f0e68f74700 20 client.32097.objecter dump_active .. 0 homeless 2024-02-23T00:58:13.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.536+0000 7f0e68f74700 10 client.32097.objecter ms_handle_connect 0x7f0e58062cc0 2024-02-23T00:58:13.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.536+0000 7f0e6affd700 1 --2- 172.21.15.47:0/2772097972 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0e58062cc0 0x7f0e58065180 secure :-1 s=READY pgs=3253 cs=0 l=1 rev1=1 rx=0x7f0e6c1096e0 tx=0x7f0e5400b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:58:13.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.540+0000 7f0e68f74700 1 -- 172.21.15.47:0/2772097972 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f0e5c04c800 con 0x7f0e6c0f8420 2024-02-23T00:58:13.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:12.310751+0000 osd.7 (osd.7) 439 : cluster [DBG] 5.20 scrub starts 2024-02-23T00:58:13.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:12.314208+0000 osd.7 (osd.7) 440 : cluster [DBG] 5.20 scrub ok 2024-02-23T00:58:13.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:13.244483+0000 osd.5 (osd.5) 422 : cluster [DBG] 5.15 scrub starts 2024-02-23T00:58:13.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:13.247939+0000 osd.5 (osd.5) 423 : cluster [DBG] 5.15 scrub ok 2024-02-23T00:58:13.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:12.310751+0000 osd.7 (osd.7) 439 : cluster [DBG] 5.20 scrub starts 2024-02-23T00:58:13.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:12.314208+0000 osd.7 (osd.7) 440 : cluster [DBG] 5.20 scrub ok 2024-02-23T00:58:13.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:13.244483+0000 osd.5 (osd.5) 422 : cluster [DBG] 5.15 scrub starts 2024-02-23T00:58:13.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:13 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:13.247939+0000 osd.5 (osd.5) 423 : cluster [DBG] 5.15 scrub ok 2024-02-23T00:58:13.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:12.310751+0000 osd.7 (osd.7) 439 : cluster [DBG] 5.20 scrub starts 2024-02-23T00:58:13.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:12.314208+0000 osd.7 (osd.7) 440 : cluster [DBG] 5.20 scrub ok 2024-02-23T00:58:13.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:13.244483+0000 osd.5 (osd.5) 422 : cluster [DBG] 5.15 scrub starts 2024-02-23T00:58:13.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:13 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:13.247939+0000 osd.5 (osd.5) 423 : cluster [DBG] 5.15 scrub ok 2024-02-23T00:58:13.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.712+0000 7f0e71a7d700 1 -- 172.21.15.47:0/2772097972 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0e6c05fa60 con 0x7f0e6c0f8420 2024-02-23T00:58:13.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.712+0000 7f0e68f74700 1 -- 172.21.15.47:0/2772097972 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0e5c019070 con 0x7f0e6c0f8420 2024-02-23T00:58:13.715 INFO:teuthology.orchestra.run.smithi047.stdout:55834575112 2024-02-23T00:58:13.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.714+0000 7f0e71a7d700 10 librados: watch_flush enter 2024-02-23T00:58:13.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.714+0000 7f0e71a7d700 10 librados: watch_flush exit 2024-02-23T00:58:13.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.714+0000 7f0e71a7d700 20 client.32097.objecter shutdown clearing up homeless session... 2024-02-23T00:58:13.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.714+0000 7f0e71a7d700 10 client.32097.objecter successfully canceled tick 2024-02-23T00:58:13.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.714+0000 7f0e71a7d700 1 -- 172.21.15.47:0/2772097972 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0e58062cc0 msgr2=0x7f0e58065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:13.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.714+0000 7f0e71a7d700 1 --2- 172.21.15.47:0/2772097972 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0e58062cc0 0x7f0e58065180 secure :-1 s=READY pgs=3253 cs=0 l=1 rev1=1 rx=0x7f0e6c1096e0 tx=0x7f0e5400b040).stop 2024-02-23T00:58:13.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.714+0000 7f0e71a7d700 1 -- 172.21.15.47:0/2772097972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e6c0f8420 msgr2=0x7f0e6c10dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:13.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.714+0000 7f0e71a7d700 1 --2- 172.21.15.47:0/2772097972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e6c0f8420 0x7f0e6c10dda0 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 rx=0x7f0e5c00e970 tx=0x7f0e5c00b040).stop 2024-02-23T00:58:13.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.714+0000 7f0e71a7d700 1 -- 172.21.15.47:0/2772097972 shutdown_connections 2024-02-23T00:58:13.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.714+0000 7f0e71a7d700 1 --2- 172.21.15.47:0/2772097972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e6c0fe510 0x7f0e6c107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:13.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.714+0000 7f0e71a7d700 1 --2- 172.21.15.47:0/2772097972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e6c0f8420 0x7f0e6c10dda0 unknown :-1 s=CLOSED pgs=2684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:13.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.714+0000 7f0e71a7d700 1 --2- 172.21.15.47:0/2772097972 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0e58062cc0 0x7f0e58065180 unknown :-1 s=CLOSED pgs=3253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:13.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.714+0000 7f0e71a7d700 1 --2- 172.21.15.47:0/2772097972 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0e6c0fdbb0 0x7f0e6c10e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:13.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.714+0000 7f0e71a7d700 1 -- 172.21.15.47:0/2772097972 >> 172.21.15.47:0/2772097972 conn(0x7f0e6c0f3800 msgr2=0x7f0e6c0f5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:58:13.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.714+0000 7f0e71a7d700 1 -- 172.21.15.47:0/2772097972 shutdown_connections 2024-02-23T00:58:13.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.714+0000 7f0e71a7d700 1 -- 172.21.15.47:0/2772097972 wait complete. 2024-02-23T00:58:13.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:13.714+0000 7f0e71a7d700 1 librados: shutdown 2024-02-23T00:58:14.485 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834575108 got 55834575112 for osd.1 2024-02-23T00:58:14.485 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-23T00:58:14.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:58:13.106929+0000 mgr.y (mgr.14517) 759 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:58:14.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:13.442942+0000 mgr.y (mgr.14517) 760 : cluster [DBG] pgmap v1344: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:58:14.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:13.583947+0000 osd.6 (osd.6) 381 : cluster [DBG] 5.1f scrub starts 2024-02-23T00:58:14.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:13.585179+0000 osd.6 (osd.6) 382 : cluster [DBG] 5.1f scrub ok 2024-02-23T00:58:14.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:58:13.713902+0000 mon.a (mon.0) 3942 : audit [DBG] from='client.? 172.21.15.47:0/2772097972' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-23T00:58:14.766 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:58:14.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:58:13.106929+0000 mgr.y (mgr.14517) 759 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:58:14.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:13.442942+0000 mgr.y (mgr.14517) 760 : cluster [DBG] pgmap v1344: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:58:14.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:13.583947+0000 osd.6 (osd.6) 381 : cluster [DBG] 5.1f scrub starts 2024-02-23T00:58:14.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:13.585179+0000 osd.6 (osd.6) 382 : cluster [DBG] 5.1f scrub ok 2024-02-23T00:58:14.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:58:13.713902+0000 mon.a (mon.0) 3942 : audit [DBG] from='client.? 172.21.15.47:0/2772097972' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-23T00:58:14.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:58:13.106929+0000 mgr.y (mgr.14517) 759 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:58:14.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:13.442942+0000 mgr.y (mgr.14517) 760 : cluster [DBG] pgmap v1344: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:58:14.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:13.583947+0000 osd.6 (osd.6) 381 : cluster [DBG] 5.1f scrub starts 2024-02-23T00:58:14.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:13.585179+0000 osd.6 (osd.6) 382 : cluster [DBG] 5.1f scrub ok 2024-02-23T00:58:14.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:58:13.713902+0000 mon.a (mon.0) 3942 : audit [DBG] from='client.? 172.21.15.47:0/2772097972' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-23T00:58:15.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:14.274662+0000 osd.7 (osd.7) 441 : cluster [DBG] 6.5 deep-scrub starts 2024-02-23T00:58:15.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:14.277755+0000 osd.7 (osd.7) 442 : cluster [DBG] 6.5 deep-scrub ok 2024-02-23T00:58:15.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:14.775213+0000 osd.1 (osd.1) 362 : cluster [DBG] 5.c deep-scrub starts 2024-02-23T00:58:15.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:14.778286+0000 osd.1 (osd.1) 363 : cluster [DBG] 5.c deep-scrub ok 2024-02-23T00:58:15.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:14.274662+0000 osd.7 (osd.7) 441 : cluster [DBG] 6.5 deep-scrub starts 2024-02-23T00:58:15.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:14.277755+0000 osd.7 (osd.7) 442 : cluster [DBG] 6.5 deep-scrub ok 2024-02-23T00:58:15.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:14.775213+0000 osd.1 (osd.1) 362 : cluster [DBG] 5.c deep-scrub starts 2024-02-23T00:58:15.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:14.778286+0000 osd.1 (osd.1) 363 : cluster [DBG] 5.c deep-scrub ok 2024-02-23T00:58:15.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:14.274662+0000 osd.7 (osd.7) 441 : cluster [DBG] 6.5 deep-scrub starts 2024-02-23T00:58:15.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:14.277755+0000 osd.7 (osd.7) 442 : cluster [DBG] 6.5 deep-scrub ok 2024-02-23T00:58:15.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:14.775213+0000 osd.1 (osd.1) 362 : cluster [DBG] 5.c deep-scrub starts 2024-02-23T00:58:15.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:14.778286+0000 osd.1 (osd.1) 363 : cluster [DBG] 5.c deep-scrub ok 2024-02-23T00:58:16.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.432+0000 7f2823e25700 1 -- 172.21.15.47:0/71412149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f281c0fdbb0 msgr2=0x7f281c0fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:16.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.432+0000 7f2823e25700 1 --2- 172.21.15.47:0/71412149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f281c0fdbb0 0x7f281c0fdf90 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 rx=0x7f280c004870 tx=0x7f280c009cb0).stop 2024-02-23T00:58:16.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.432+0000 7f2823e25700 1 -- 172.21.15.47:0/71412149 shutdown_connections 2024-02-23T00:58:16.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.432+0000 7f2823e25700 1 --2- 172.21.15.47:0/71412149 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f281c0f8420 0x7f281c104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:16.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.432+0000 7f2823e25700 1 --2- 172.21.15.47:0/71412149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f281c0fdbb0 0x7f281c0fdf90 unknown :-1 s=CLOSED pgs=2685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:16.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.432+0000 7f2823e25700 1 --2- 172.21.15.47:0/71412149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f281c0fe510 0x7f281c0f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:16.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.432+0000 7f2823e25700 1 -- 172.21.15.47:0/71412149 >> 172.21.15.47:0/71412149 conn(0x7f281c0f3800 msgr2=0x7f281c0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:58:16.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.432+0000 7f2823e25700 1 -- 172.21.15.47:0/71412149 shutdown_connections 2024-02-23T00:58:16.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.432+0000 7f2823e25700 1 -- 172.21.15.47:0/71412149 wait complete. 2024-02-23T00:58:16.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.432+0000 7f2823e25700 1 librados: starting msgr at 2024-02-23T00:58:16.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.432+0000 7f2823e25700 1 librados: starting objecter 2024-02-23T00:58:16.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.432+0000 7f2823e25700 1 Processor -- start 2024-02-23T00:58:16.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.432+0000 7f2823e25700 1 -- start start 2024-02-23T00:58:16.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.432+0000 7f2823e25700 1 librados: setting wanted keys 2024-02-23T00:58:16.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.432+0000 7f2823e25700 1 librados: calling monclient init 2024-02-23T00:58:16.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.433+0000 7f2823e25700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f281c0f8420 0x7f281c06b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:16.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.433+0000 7f2823e25700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f281c0fdbb0 0x7f281c06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:16.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.433+0000 7f2823e25700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f281c0fe510 0x7f281c06bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:16.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.433+0000 7f2823e25700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281c116310 con 0x7f281c0fdbb0 2024-02-23T00:58:16.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.433+0000 7f2823e25700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281c116480 con 0x7f281c0f8420 2024-02-23T00:58:16.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.433+0000 7f2823e25700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f281c116640 con 0x7f281c0fe510 2024-02-23T00:58:16.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.433+0000 7f28213c0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f281c0fdbb0 0x7f281c06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:16.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.433+0000 7f28223c2700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f281c0fe510 0x7f281c06bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:16.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.433+0000 7f28213c0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f281c0fdbb0 0x7f281c06b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47120/0 (socket says 172.21.15.47:47120) 2024-02-23T00:58:16.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.433+0000 7f28213c0700 1 -- 172.21.15.47:0/2738707826 learned_addr learned my addr 172.21.15.47:0/2738707826 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:58:16.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.433+0000 7f28223c2700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f281c0fe510 0x7f281c06bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:57892/0 (socket says 172.21.15.47:57892) 2024-02-23T00:58:16.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.433+0000 7f2821bc1700 1 --2- 172.21.15.47:0/2738707826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f281c0f8420 0x7f281c06b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:16.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.433+0000 7f28213c0700 1 -- 172.21.15.47:0/2738707826 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f281c0fe510 msgr2=0x7f281c06bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:16.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.433+0000 7f28213c0700 1 --2- 172.21.15.47:0/2738707826 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f281c0fe510 0x7f281c06bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:16.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.433+0000 7f28213c0700 1 -- 172.21.15.47:0/2738707826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f281c0f8420 msgr2=0x7f281c06b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:16.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.433+0000 7f28213c0700 1 --2- 172.21.15.47:0/2738707826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f281c0f8420 0x7f281c06b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:16.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.433+0000 7f28213c0700 1 -- 172.21.15.47:0/2738707826 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f280c005040 con 0x7f281c0fdbb0 2024-02-23T00:58:16.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.433+0000 7f28213c0700 1 --2- 172.21.15.47:0/2738707826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f281c0fdbb0 0x7f281c06b5c0 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 rx=0x7f281800c810 tx=0x7f2818004330).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:58:16.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.433+0000 7f2812ffd700 10 client.?.objecter ms_handle_connect 0x7f281c0fdbb0 2024-02-23T00:58:16.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.433+0000 7f2812ffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:58:16.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.433+0000 7f2812ffd700 1 -- 172.21.15.47:0/2738707826 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28180043a0 con 0x7f281c0fdbb0 2024-02-23T00:58:16.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.433+0000 7f2812ffd700 1 -- 172.21.15.47:0/2738707826 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f28180109a0 con 0x7f281c0fdbb0 2024-02-23T00:58:16.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.433+0000 7f2823e25700 1 -- 172.21.15.47:0/2738707826 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f281c116920 con 0x7f281c0fdbb0 2024-02-23T00:58:16.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.433+0000 7f2812ffd700 1 -- 172.21.15.47:0/2738707826 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2818012940 con 0x7f281c0fdbb0 2024-02-23T00:58:16.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.434+0000 7f2823e25700 10 client.32103.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:58:16.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.434+0000 7f2823e25700 1 -- 172.21.15.47:0/2738707826 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f281c116e90 con 0x7f281c0fdbb0 2024-02-23T00:58:16.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.434+0000 7f2823e25700 1 librados: init done 2024-02-23T00:58:16.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.434+0000 7f2823e25700 1 -- 172.21.15.47:0/2738707826 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f281c106e60 con 0x7f281c0fdbb0 2024-02-23T00:58:16.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.435+0000 7f2812ffd700 1 -- 172.21.15.47:0/2738707826 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f2818010510 con 0x7f281c0fdbb0 2024-02-23T00:58:16.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.435+0000 7f2812ffd700 1 --2- 172.21.15.47:0/2738707826 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2808062cc0 0x7f2808065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:16.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.435+0000 7f2812ffd700 1 -- 172.21.15.47:0/2738707826 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(858..858 src has 806..858) v4 ==== 8663+0+0 (secure 0 0 0) 0x7f281807d340 con 0x7f281c0fdbb0 2024-02-23T00:58:16.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.435+0000 7f2812ffd700 10 client.32103.objecter ms_dispatch 0x7f281c001550 osd_map(858..858 src has 806..858) v4 2024-02-23T00:58:16.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.435+0000 7f2812ffd700 3 client.32103.objecter handle_osd_map got epochs [858,858] > 0 2024-02-23T00:58:16.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.435+0000 7f2812ffd700 3 client.32103.objecter handle_osd_map decoding full epoch 858 2024-02-23T00:58:16.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.435+0000 7f2821bc1700 1 --2- 172.21.15.47:0/2738707826 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2808062cc0 0x7f2808065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:16.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.435+0000 7f2812ffd700 20 client.32103.objecter dump_active .. 0 homeless 2024-02-23T00:58:16.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.436+0000 7f2812ffd700 10 client.32103.objecter ms_handle_connect 0x7f2808062cc0 2024-02-23T00:58:16.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.436+0000 7f2821bc1700 1 --2- 172.21.15.47:0/2738707826 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2808062cc0 0x7f2808065180 secure :-1 s=READY pgs=3254 cs=0 l=1 rev1=1 rx=0x7f280c000e50 tx=0x7f280c00b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:58:16.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.440+0000 7f2812ffd700 1 -- 172.21.15.47:0/2738707826 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f281804cb80 con 0x7f281c0fdbb0 2024-02-23T00:58:16.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.612+0000 7f2823e25700 1 -- 172.21.15.47:0/2738707826 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f281c05fb00 con 0x7f281c0fdbb0 2024-02-23T00:58:16.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.612+0000 7f2812ffd700 1 -- 172.21.15.47:0/2738707826 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f28180501a0 con 0x7f281c0fdbb0 2024-02-23T00:58:16.616 INFO:teuthology.orchestra.run.smithi047.stdout:77309411588 2024-02-23T00:58:16.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.614+0000 7f2823e25700 10 librados: watch_flush enter 2024-02-23T00:58:16.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.614+0000 7f2823e25700 10 librados: watch_flush exit 2024-02-23T00:58:16.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.614+0000 7f2823e25700 20 client.32103.objecter shutdown clearing up homeless session... 2024-02-23T00:58:16.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.614+0000 7f2823e25700 10 client.32103.objecter successfully canceled tick 2024-02-23T00:58:16.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.614+0000 7f2823e25700 1 -- 172.21.15.47:0/2738707826 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2808062cc0 msgr2=0x7f2808065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:16.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.614+0000 7f2823e25700 1 --2- 172.21.15.47:0/2738707826 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2808062cc0 0x7f2808065180 secure :-1 s=READY pgs=3254 cs=0 l=1 rev1=1 rx=0x7f280c000e50 tx=0x7f280c00b040).stop 2024-02-23T00:58:16.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.614+0000 7f2823e25700 1 -- 172.21.15.47:0/2738707826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f281c0fdbb0 msgr2=0x7f281c06b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:16.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.614+0000 7f2823e25700 1 --2- 172.21.15.47:0/2738707826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f281c0fdbb0 0x7f281c06b5c0 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 rx=0x7f281800c810 tx=0x7f2818004330).stop 2024-02-23T00:58:16.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.614+0000 7f2823e25700 1 -- 172.21.15.47:0/2738707826 shutdown_connections 2024-02-23T00:58:16.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.614+0000 7f2823e25700 1 --2- 172.21.15.47:0/2738707826 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f2808062cc0 0x7f2808065180 unknown :-1 s=CLOSED pgs=3254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:16.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.614+0000 7f2823e25700 1 --2- 172.21.15.47:0/2738707826 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f281c0fe510 0x7f281c06bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:16.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.614+0000 7f2823e25700 1 --2- 172.21.15.47:0/2738707826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f281c0f8420 0x7f281c06b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:16.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.614+0000 7f2823e25700 1 --2- 172.21.15.47:0/2738707826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f281c0fdbb0 0x7f281c06b5c0 unknown :-1 s=CLOSED pgs=2686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:16.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.614+0000 7f2823e25700 1 -- 172.21.15.47:0/2738707826 >> 172.21.15.47:0/2738707826 conn(0x7f281c0f3800 msgr2=0x7f281c0fb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:58:16.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.614+0000 7f2823e25700 1 -- 172.21.15.47:0/2738707826 shutdown_connections 2024-02-23T00:58:16.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.614+0000 7f2823e25700 1 -- 172.21.15.47:0/2738707826 wait complete. 2024-02-23T00:58:16.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:16.614+0000 7f2823e25700 1 librados: shutdown 2024-02-23T00:58:16.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:15.317257+0000 osd.7 (osd.7) 443 : cluster [DBG] 5.4 scrub starts 2024-02-23T00:58:16.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:15.320204+0000 osd.7 (osd.7) 444 : cluster [DBG] 5.4 scrub ok 2024-02-23T00:58:16.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:15.443310+0000 mgr.y (mgr.14517) 761 : cluster [DBG] pgmap v1345: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:58:16.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:15.521692+0000 osd.6 (osd.6) 383 : cluster [DBG] 18.10 deep-scrub starts 2024-02-23T00:58:16.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:15.522508+0000 osd.6 (osd.6) 384 : cluster [DBG] 18.10 deep-scrub ok 2024-02-23T00:58:16.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:16.240444+0000 osd.5 (osd.5) 424 : cluster [DBG] 18.17 scrub starts 2024-02-23T00:58:16.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:16.244668+0000 osd.5 (osd.5) 425 : cluster [DBG] 18.17 scrub ok 2024-02-23T00:58:16.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:15.317257+0000 osd.7 (osd.7) 443 : cluster [DBG] 5.4 scrub starts 2024-02-23T00:58:16.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:15.320204+0000 osd.7 (osd.7) 444 : cluster [DBG] 5.4 scrub ok 2024-02-23T00:58:16.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:15.443310+0000 mgr.y (mgr.14517) 761 : cluster [DBG] pgmap v1345: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:58:16.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:15.521692+0000 osd.6 (osd.6) 383 : cluster [DBG] 18.10 deep-scrub starts 2024-02-23T00:58:16.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:15.522508+0000 osd.6 (osd.6) 384 : cluster [DBG] 18.10 deep-scrub ok 2024-02-23T00:58:16.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:16.240444+0000 osd.5 (osd.5) 424 : cluster [DBG] 18.17 scrub starts 2024-02-23T00:58:16.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:16.244668+0000 osd.5 (osd.5) 425 : cluster [DBG] 18.17 scrub ok 2024-02-23T00:58:16.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:15.317257+0000 osd.7 (osd.7) 443 : cluster [DBG] 5.4 scrub starts 2024-02-23T00:58:16.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:15.320204+0000 osd.7 (osd.7) 444 : cluster [DBG] 5.4 scrub ok 2024-02-23T00:58:16.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:15.443310+0000 mgr.y (mgr.14517) 761 : cluster [DBG] pgmap v1345: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:58:16.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:15.521692+0000 osd.6 (osd.6) 383 : cluster [DBG] 18.10 deep-scrub starts 2024-02-23T00:58:16.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:15.522508+0000 osd.6 (osd.6) 384 : cluster [DBG] 18.10 deep-scrub ok 2024-02-23T00:58:16.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:16.240444+0000 osd.5 (osd.5) 424 : cluster [DBG] 18.17 scrub starts 2024-02-23T00:58:16.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:16 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:16.244668+0000 osd.5 (osd.5) 425 : cluster [DBG] 18.17 scrub ok 2024-02-23T00:58:17.351 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411584 got 77309411588 for osd.2 2024-02-23T00:58:17.351 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-23T00:58:17.635 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:58:17.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:16.550529+0000 osd.6 (osd.6) 385 : cluster [DBG] 4.1f scrub starts 2024-02-23T00:58:17.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:16.552530+0000 osd.6 (osd.6) 386 : cluster [DBG] 4.1f scrub ok 2024-02-23T00:58:17.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:58:16.613820+0000 mon.a (mon.0) 3943 : audit [DBG] from='client.? 172.21.15.47:0/2738707826' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-23T00:58:17.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:17.203518+0000 osd.5 (osd.5) 426 : cluster [DBG] 5.7 scrub starts 2024-02-23T00:58:17.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:17.206683+0000 osd.5 (osd.5) 427 : cluster [DBG] 5.7 scrub ok 2024-02-23T00:58:17.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:16.550529+0000 osd.6 (osd.6) 385 : cluster [DBG] 4.1f scrub starts 2024-02-23T00:58:17.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:16.552530+0000 osd.6 (osd.6) 386 : cluster [DBG] 4.1f scrub ok 2024-02-23T00:58:17.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:58:16.613820+0000 mon.a (mon.0) 3943 : audit [DBG] from='client.? 172.21.15.47:0/2738707826' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-23T00:58:17.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:17.203518+0000 osd.5 (osd.5) 426 : cluster [DBG] 5.7 scrub starts 2024-02-23T00:58:17.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:17.206683+0000 osd.5 (osd.5) 427 : cluster [DBG] 5.7 scrub ok 2024-02-23T00:58:17.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:16.550529+0000 osd.6 (osd.6) 385 : cluster [DBG] 4.1f scrub starts 2024-02-23T00:58:17.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:16.552530+0000 osd.6 (osd.6) 386 : cluster [DBG] 4.1f scrub ok 2024-02-23T00:58:17.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:58:16.613820+0000 mon.a (mon.0) 3943 : audit [DBG] from='client.? 172.21.15.47:0/2738707826' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-23T00:58:17.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:17.203518+0000 osd.5 (osd.5) 426 : cluster [DBG] 5.7 scrub starts 2024-02-23T00:58:17.809 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:17 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:17.206683+0000 osd.5 (osd.5) 427 : cluster [DBG] 5.7 scrub ok 2024-02-23T00:58:18.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:17.443986+0000 mgr.y (mgr.14517) 762 : cluster [DBG] pgmap v1346: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:58:18.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:17.443986+0000 mgr.y (mgr.14517) 762 : cluster [DBG] pgmap v1346: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:58:18.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:18 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:17.443986+0000 mgr.y (mgr.14517) 762 : cluster [DBG] pgmap v1346: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:58:19.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.337+0000 7fb4bbe65700 1 -- 172.21.15.47:0/4101109176 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb4b40fe530 msgr2=0x7fb4b40f63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:19.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.337+0000 7fb4bbe65700 1 --2- 172.21.15.47:0/4101109176 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb4b40fe530 0x7fb4b40f63b0 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 rx=0x7fb4a4004870 tx=0x7fb4a4009cb0).stop 2024-02-23T00:58:19.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.337+0000 7fb4bbe65700 1 -- 172.21.15.47:0/4101109176 shutdown_connections 2024-02-23T00:58:19.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.337+0000 7fb4bbe65700 1 --2- 172.21.15.47:0/4101109176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4b40f6a90 0x7fb4b4104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:19.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.337+0000 7fb4bbe65700 1 --2- 172.21.15.47:0/4101109176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4b40fdbd0 0x7fb4b40fdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:19.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.337+0000 7fb4bbe65700 1 --2- 172.21.15.47:0/4101109176 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb4b40fe530 0x7fb4b40f63b0 unknown :-1 s=CLOSED pgs=2872 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:19.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.337+0000 7fb4bbe65700 1 -- 172.21.15.47:0/4101109176 >> 172.21.15.47:0/4101109176 conn(0x7fb4b406c370 msgr2=0x7fb4b406c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:58:19.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.337+0000 7fb4bbe65700 1 -- 172.21.15.47:0/4101109176 shutdown_connections 2024-02-23T00:58:19.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.337+0000 7fb4bbe65700 1 -- 172.21.15.47:0/4101109176 wait complete. 2024-02-23T00:58:19.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.337+0000 7fb4bbe65700 1 librados: starting msgr at 2024-02-23T00:58:19.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.337+0000 7fb4bbe65700 1 librados: starting objecter 2024-02-23T00:58:19.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.338+0000 7fb4bbe65700 1 Processor -- start 2024-02-23T00:58:19.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.338+0000 7fb4bbe65700 1 -- start start 2024-02-23T00:58:19.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.338+0000 7fb4bbe65700 1 librados: setting wanted keys 2024-02-23T00:58:19.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.338+0000 7fb4bbe65700 1 librados: calling monclient init 2024-02-23T00:58:19.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.338+0000 7fb4bbe65700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4b40f6a90 0x7fb4b410df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:19.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.338+0000 7fb4bbe65700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb4b40fdbd0 0x7fb4b410e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:19.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.338+0000 7fb4bbe65700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4b40fe530 0x7fb4b4108120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:19.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.338+0000 7fb4bbe65700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b4108610 con 0x7fb4b40f6a90 2024-02-23T00:58:19.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.338+0000 7fb4bbe65700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b4108780 con 0x7fb4b40fe530 2024-02-23T00:58:19.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.338+0000 7fb4bbe65700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b4108940 con 0x7fb4b40fdbd0 2024-02-23T00:58:19.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.338+0000 7fb4b9400700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb4b40fdbd0 0x7fb4b410e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:19.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.338+0000 7fb4b9c01700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4b40f6a90 0x7fb4b410df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:19.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.338+0000 7fb4b9400700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb4b40fdbd0 0x7fb4b410e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:57912/0 (socket says 172.21.15.47:57912) 2024-02-23T00:58:19.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.338+0000 7fb4b9400700 1 -- 172.21.15.47:0/2718855910 learned_addr learned my addr 172.21.15.47:0/2718855910 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:58:19.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.338+0000 7fb4ba402700 1 --2- 172.21.15.47:0/2718855910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4b40fe530 0x7fb4b4108120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:19.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.338+0000 7fb4b9400700 1 -- 172.21.15.47:0/2718855910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4b40fe530 msgr2=0x7fb4b4108120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:19.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.338+0000 7fb4b9400700 1 --2- 172.21.15.47:0/2718855910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4b40fe530 0x7fb4b4108120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:19.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.338+0000 7fb4b9400700 1 -- 172.21.15.47:0/2718855910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4b40f6a90 msgr2=0x7fb4b410df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:19.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.338+0000 7fb4b9400700 1 --2- 172.21.15.47:0/2718855910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4b40f6a90 0x7fb4b410df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:19.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.338+0000 7fb4b9400700 1 -- 172.21.15.47:0/2718855910 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4ac0097c0 con 0x7fb4b40fdbd0 2024-02-23T00:58:19.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.338+0000 7fb4b9c01700 1 --2- 172.21.15.47:0/2718855910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4b40f6a90 0x7fb4b410df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:58:19.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.338+0000 7fb4ba402700 1 --2- 172.21.15.47:0/2718855910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4b40fe530 0x7fb4b4108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:58:19.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.338+0000 7fb4b9400700 1 --2- 172.21.15.47:0/2718855910 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb4b40fdbd0 0x7fb4b410e470 secure :-1 s=READY pgs=2873 cs=0 l=1 rev1=1 rx=0x7fb4a4009da0 tx=0x7fb4a4009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:58:19.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.338+0000 7fb4aaffd700 10 client.?.objecter ms_handle_connect 0x7fb4b40fdbd0 2024-02-23T00:58:19.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.338+0000 7fb4aaffd700 10 client.?.objecter resend_mon_ops 2024-02-23T00:58:19.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.339+0000 7fb4aaffd700 1 -- 172.21.15.47:0/2718855910 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a400e030 con 0x7fb4b40fdbd0 2024-02-23T00:58:19.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.339+0000 7fb4bbe65700 1 -- 172.21.15.47:0/2718855910 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4a4005040 con 0x7fb4b40fdbd0 2024-02-23T00:58:19.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.339+0000 7fb4aaffd700 1 -- 172.21.15.47:0/2718855910 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fb4a4013e70 con 0x7fb4b40fdbd0 2024-02-23T00:58:19.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.339+0000 7fb4aaffd700 1 -- 172.21.15.47:0/2718855910 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4018070 con 0x7fb4b40fdbd0 2024-02-23T00:58:19.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.339+0000 7fb4bbe65700 10 client.42431.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:58:19.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.339+0000 7fb4bbe65700 1 -- 172.21.15.47:0/2718855910 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b4108de0 con 0x7fb4b40fdbd0 2024-02-23T00:58:19.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.339+0000 7fb4bbe65700 1 librados: init done 2024-02-23T00:58:19.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.340+0000 7fb4bbe65700 1 -- 172.21.15.47:0/2718855910 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4b4108fd0 con 0x7fb4b40fdbd0 2024-02-23T00:58:19.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.340+0000 7fb4aaffd700 1 -- 172.21.15.47:0/2718855910 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fb4a4009240 con 0x7fb4b40fdbd0 2024-02-23T00:58:19.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.340+0000 7fb4aaffd700 1 --2- 172.21.15.47:0/2718855910 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb4a0062cc0 0x7fb4a0065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:19.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.340+0000 7fb4aaffd700 1 -- 172.21.15.47:0/2718855910 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(858..858 src has 806..858) v4 ==== 8663+0+0 (secure 0 0 0) 0x7fb4a407e730 con 0x7fb4b40fdbd0 2024-02-23T00:58:19.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.340+0000 7fb4aaffd700 10 client.42431.objecter ms_dispatch 0x7fb4b4001550 osd_map(858..858 src has 806..858) v4 2024-02-23T00:58:19.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.340+0000 7fb4aaffd700 3 client.42431.objecter handle_osd_map got epochs [858,858] > 0 2024-02-23T00:58:19.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.340+0000 7fb4aaffd700 3 client.42431.objecter handle_osd_map decoding full epoch 858 2024-02-23T00:58:19.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.341+0000 7fb4b9c01700 1 --2- 172.21.15.47:0/2718855910 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb4a0062cc0 0x7fb4a0065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:19.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.341+0000 7fb4aaffd700 20 client.42431.objecter dump_active .. 0 homeless 2024-02-23T00:58:19.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.341+0000 7fb4aaffd700 10 client.42431.objecter ms_handle_connect 0x7fb4a0062cc0 2024-02-23T00:58:19.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.341+0000 7fb4b9c01700 1 --2- 172.21.15.47:0/2718855910 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb4a0062cc0 0x7fb4a0065180 secure :-1 s=READY pgs=3255 cs=0 l=1 rev1=1 rx=0x7fb4b00103c0 tx=0x7fb4b000b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:58:19.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.345+0000 7fb4aaffd700 1 -- 172.21.15.47:0/2718855910 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb4a404df70 con 0x7fb4b40fdbd0 2024-02-23T00:58:19.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.520+0000 7fb4bbe65700 1 -- 172.21.15.47:0/2718855910 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb4b4106ed0 con 0x7fb4b40fdbd0 2024-02-23T00:58:19.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.521+0000 7fb4aaffd700 1 -- 172.21.15.47:0/2718855910 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb4a4051590 con 0x7fb4b40fdbd0 2024-02-23T00:58:19.524 INFO:teuthology.orchestra.run.smithi047.stdout:111669149950 2024-02-23T00:58:19.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.522+0000 7fb4bbe65700 10 librados: watch_flush enter 2024-02-23T00:58:19.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.522+0000 7fb4bbe65700 10 librados: watch_flush exit 2024-02-23T00:58:19.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.522+0000 7fb4bbe65700 20 client.42431.objecter shutdown clearing up homeless session... 2024-02-23T00:58:19.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.522+0000 7fb4bbe65700 10 client.42431.objecter successfully canceled tick 2024-02-23T00:58:19.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.522+0000 7fb4bbe65700 1 -- 172.21.15.47:0/2718855910 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb4a0062cc0 msgr2=0x7fb4a0065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:19.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.522+0000 7fb4bbe65700 1 --2- 172.21.15.47:0/2718855910 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb4a0062cc0 0x7fb4a0065180 secure :-1 s=READY pgs=3255 cs=0 l=1 rev1=1 rx=0x7fb4b00103c0 tx=0x7fb4b000b040).stop 2024-02-23T00:58:19.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.522+0000 7fb4bbe65700 1 -- 172.21.15.47:0/2718855910 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb4b40fdbd0 msgr2=0x7fb4b410e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:19.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.522+0000 7fb4bbe65700 1 --2- 172.21.15.47:0/2718855910 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb4b40fdbd0 0x7fb4b410e470 secure :-1 s=READY pgs=2873 cs=0 l=1 rev1=1 rx=0x7fb4a4009da0 tx=0x7fb4a4009cb0).stop 2024-02-23T00:58:19.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.522+0000 7fb4bbe65700 1 -- 172.21.15.47:0/2718855910 shutdown_connections 2024-02-23T00:58:19.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.522+0000 7fb4bbe65700 1 --2- 172.21.15.47:0/2718855910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4b40fe530 0x7fb4b4108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:19.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.522+0000 7fb4bbe65700 1 --2- 172.21.15.47:0/2718855910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4b40f6a90 0x7fb4b410df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:19.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.522+0000 7fb4bbe65700 1 --2- 172.21.15.47:0/2718855910 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fb4a0062cc0 0x7fb4a0065180 unknown :-1 s=CLOSED pgs=3255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:19.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.522+0000 7fb4bbe65700 1 --2- 172.21.15.47:0/2718855910 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb4b40fdbd0 0x7fb4b410e470 unknown :-1 s=CLOSED pgs=2873 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:19.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.523+0000 7fb4bbe65700 1 -- 172.21.15.47:0/2718855910 >> 172.21.15.47:0/2718855910 conn(0x7fb4b406c370 msgr2=0x7fb4b40fb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:58:19.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.523+0000 7fb4bbe65700 1 -- 172.21.15.47:0/2718855910 shutdown_connections 2024-02-23T00:58:19.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.523+0000 7fb4bbe65700 1 -- 172.21.15.47:0/2718855910 wait complete. 2024-02-23T00:58:19.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:19.523+0000 7fb4bbe65700 1 librados: shutdown 2024-02-23T00:58:19.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:18.814655+0000 osd.3 (osd.3) 371 : cluster [DBG] 2.19 deep-scrub starts 2024-02-23T00:58:19.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:18.815591+0000 osd.3 (osd.3) 372 : cluster [DBG] 2.19 deep-scrub ok 2024-02-23T00:58:19.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:18.814655+0000 osd.3 (osd.3) 371 : cluster [DBG] 2.19 deep-scrub starts 2024-02-23T00:58:19.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:18.815591+0000 osd.3 (osd.3) 372 : cluster [DBG] 2.19 deep-scrub ok 2024-02-23T00:58:19.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:18.814655+0000 osd.3 (osd.3) 371 : cluster [DBG] 2.19 deep-scrub starts 2024-02-23T00:58:19.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:19 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:18.815591+0000 osd.3 (osd.3) 372 : cluster [DBG] 2.19 deep-scrub ok 2024-02-23T00:58:20.334 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149945 got 111669149950 for osd.3 2024-02-23T00:58:20.334 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-23T00:58:20.636 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:58:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:19.444337+0000 mgr.y (mgr.14517) 763 : cluster [DBG] pgmap v1347: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:58:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:58:19.522130+0000 mon.c (mon.2) 538 : audit [DBG] from='client.? 172.21.15.47:0/2718855910' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-23T00:58:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:19.706968+0000 osd.1 (osd.1) 364 : cluster [DBG] 18.3 deep-scrub starts 2024-02-23T00:58:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:19.710956+0000 osd.1 (osd.1) 365 : cluster [DBG] 18.3 deep-scrub ok 2024-02-23T00:58:20.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:20.125395+0000 osd.5 (osd.5) 428 : cluster [DBG] 18.f scrub starts 2024-02-23T00:58:20.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:20.129753+0000 osd.5 (osd.5) 429 : cluster [DBG] 18.f scrub ok 2024-02-23T00:58:20.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:58:20.424047+0000 mon.a (mon.0) 3944 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:58:20.807 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:58:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:58:20] "GET /metrics HTTP/1.1" 200 187141 "" "Prometheus/2.33.4" 2024-02-23T00:58:20.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:19.444337+0000 mgr.y (mgr.14517) 763 : cluster [DBG] pgmap v1347: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:58:20.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:58:19.522130+0000 mon.c (mon.2) 538 : audit [DBG] from='client.? 172.21.15.47:0/2718855910' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-23T00:58:20.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:19.706968+0000 osd.1 (osd.1) 364 : cluster [DBG] 18.3 deep-scrub starts 2024-02-23T00:58:20.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:19.710956+0000 osd.1 (osd.1) 365 : cluster [DBG] 18.3 deep-scrub ok 2024-02-23T00:58:20.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:20.125395+0000 osd.5 (osd.5) 428 : cluster [DBG] 18.f scrub starts 2024-02-23T00:58:20.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:20.129753+0000 osd.5 (osd.5) 429 : cluster [DBG] 18.f scrub ok 2024-02-23T00:58:20.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:58:20.424047+0000 mon.a (mon.0) 3944 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:58:20.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:19.444337+0000 mgr.y (mgr.14517) 763 : cluster [DBG] pgmap v1347: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-23T00:58:20.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:58:19.522130+0000 mon.c (mon.2) 538 : audit [DBG] from='client.? 172.21.15.47:0/2718855910' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-23T00:58:20.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:19.706968+0000 osd.1 (osd.1) 364 : cluster [DBG] 18.3 deep-scrub starts 2024-02-23T00:58:20.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:19.710956+0000 osd.1 (osd.1) 365 : cluster [DBG] 18.3 deep-scrub ok 2024-02-23T00:58:20.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:20.125395+0000 osd.5 (osd.5) 428 : cluster [DBG] 18.f scrub starts 2024-02-23T00:58:20.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:20.129753+0000 osd.5 (osd.5) 429 : cluster [DBG] 18.f scrub ok 2024-02-23T00:58:20.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:58:20.424047+0000 mon.a (mon.0) 3944 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:58:21.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:20.187974+0000 osd.4 (osd.4) 407 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:58:21.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:20.188357+0000 osd.4 (osd.4) 408 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:58:21.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:20.438078+0000 mon.a (mon.0) 3945 : cluster [DBG] osdmap e859: 8 total, 8 up, 8 in 2024-02-23T00:58:21.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:20.564077+0000 osd.6 (osd.6) 387 : cluster [DBG] 4.a scrub starts 2024-02-23T00:58:21.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:20.566953+0000 osd.6 (osd.6) 388 : cluster [DBG] 4.a scrub ok 2024-02-23T00:58:21.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:20.672250+0000 osd.1 (osd.1) 366 : cluster [DBG] 2.15 deep-scrub starts 2024-02-23T00:58:21.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:20.674231+0000 osd.1 (osd.1) 367 : cluster [DBG] 2.15 deep-scrub ok 2024-02-23T00:58:21.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:58:20.744783+0000 mon.a (mon.0) 3946 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:58:21.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:58:20.745431+0000 mon.a (mon.0) 3947 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:58:21.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:58:20.749598+0000 mon.a (mon.0) 3948 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:58:21.767 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:20.187974+0000 osd.4 (osd.4) 407 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:58:21.767 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:20.188357+0000 osd.4 (osd.4) 408 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:58:21.767 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:20.438078+0000 mon.a (mon.0) 3945 : cluster [DBG] osdmap e859: 8 total, 8 up, 8 in 2024-02-23T00:58:21.767 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:20.564077+0000 osd.6 (osd.6) 387 : cluster [DBG] 4.a scrub starts 2024-02-23T00:58:21.767 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:20.566953+0000 osd.6 (osd.6) 388 : cluster [DBG] 4.a scrub ok 2024-02-23T00:58:21.768 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:20.672250+0000 osd.1 (osd.1) 366 : cluster [DBG] 2.15 deep-scrub starts 2024-02-23T00:58:21.768 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:20.674231+0000 osd.1 (osd.1) 367 : cluster [DBG] 2.15 deep-scrub ok 2024-02-23T00:58:21.768 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:58:20.744783+0000 mon.a (mon.0) 3946 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:58:21.768 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:58:20.745431+0000 mon.a (mon.0) 3947 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:58:21.768 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:58:20.749598+0000 mon.a (mon.0) 3948 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:58:21.768 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:20.187974+0000 osd.4 (osd.4) 407 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:58:21.768 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:20.188357+0000 osd.4 (osd.4) 408 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:58:21.768 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:20.438078+0000 mon.a (mon.0) 3945 : cluster [DBG] osdmap e859: 8 total, 8 up, 8 in 2024-02-23T00:58:21.768 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:20.564077+0000 osd.6 (osd.6) 387 : cluster [DBG] 4.a scrub starts 2024-02-23T00:58:21.768 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:20.566953+0000 osd.6 (osd.6) 388 : cluster [DBG] 4.a scrub ok 2024-02-23T00:58:21.768 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:20.672250+0000 osd.1 (osd.1) 366 : cluster [DBG] 2.15 deep-scrub starts 2024-02-23T00:58:21.769 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:20.674231+0000 osd.1 (osd.1) 367 : cluster [DBG] 2.15 deep-scrub ok 2024-02-23T00:58:21.769 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:58:20.744783+0000 mon.a (mon.0) 3946 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:58:21.769 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:58:20.745431+0000 mon.a (mon.0) 3947 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:58:21.769 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:21 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:58:20.749598+0000 mon.a (mon.0) 3948 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:58:22.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.222+0000 7f0b5e484700 1 -- 172.21.15.47:0/783655081 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0b580fcbb0 msgr2=0x7f0b580fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:22.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.222+0000 7f0b5e484700 1 --2- 172.21.15.47:0/783655081 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0b580fcbb0 0x7f0b580fcf90 secure :-1 s=READY pgs=2874 cs=0 l=1 rev1=1 rx=0x7f0b48004870 tx=0x7f0b48009cb0).stop 2024-02-23T00:58:22.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.222+0000 7f0b5e484700 1 -- 172.21.15.47:0/783655081 shutdown_connections 2024-02-23T00:58:22.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.222+0000 7f0b5e484700 1 --2- 172.21.15.47:0/783655081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b580f7400 0x7f0b58103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:22.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.222+0000 7f0b5e484700 1 --2- 172.21.15.47:0/783655081 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0b580fcbb0 0x7f0b580fcf90 unknown :-1 s=CLOSED pgs=2874 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:22.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.222+0000 7f0b5e484700 1 --2- 172.21.15.47:0/783655081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b580fd510 0x7f0b580f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:22.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.222+0000 7f0b5e484700 1 -- 172.21.15.47:0/783655081 >> 172.21.15.47:0/783655081 conn(0x7f0b580f2820 msgr2=0x7f0b580f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:58:22.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.222+0000 7f0b5e484700 1 -- 172.21.15.47:0/783655081 shutdown_connections 2024-02-23T00:58:22.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.222+0000 7f0b5e484700 1 -- 172.21.15.47:0/783655081 wait complete. 2024-02-23T00:58:22.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.222+0000 7f0b5e484700 1 librados: starting msgr at 2024-02-23T00:58:22.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.222+0000 7f0b5e484700 1 librados: starting objecter 2024-02-23T00:58:22.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.223+0000 7f0b5e484700 1 Processor -- start 2024-02-23T00:58:22.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.223+0000 7f0b5e484700 1 -- start start 2024-02-23T00:58:22.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.223+0000 7f0b5e484700 1 librados: setting wanted keys 2024-02-23T00:58:22.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.223+0000 7f0b5e484700 1 librados: calling monclient init 2024-02-23T00:58:22.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.223+0000 7f0b5e484700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b580f7400 0x7f0b581091a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:22.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.223+0000 7f0b5e484700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0b580fcbb0 0x7f0b58109690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:22.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.223+0000 7f0b5e484700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b580fd510 0x7f0b58109c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:22.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.223+0000 7f0b5e484700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b58114390 con 0x7f0b580f7400 2024-02-23T00:58:22.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.223+0000 7f0b5e484700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b58114500 con 0x7f0b580fd510 2024-02-23T00:58:22.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.223+0000 7f0b5e484700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b581146c0 con 0x7f0b580fcbb0 2024-02-23T00:58:22.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.223+0000 7f0b577fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0b580fcbb0 0x7f0b58109690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:22.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.223+0000 7f0b57fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b580f7400 0x7f0b581091a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:22.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.223+0000 7f0b577fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0b580fcbb0 0x7f0b58109690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:57928/0 (socket says 172.21.15.47:57928) 2024-02-23T00:58:22.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.223+0000 7f0b577fe700 1 -- 172.21.15.47:0/3130106062 learned_addr learned my addr 172.21.15.47:0/3130106062 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:58:22.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.223+0000 7f0b5ca21700 1 --2- 172.21.15.47:0/3130106062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b580fd510 0x7f0b58109c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:22.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.224+0000 7f0b577fe700 1 -- 172.21.15.47:0/3130106062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b580fd510 msgr2=0x7f0b58109c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:22.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.224+0000 7f0b577fe700 1 --2- 172.21.15.47:0/3130106062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b580fd510 0x7f0b58109c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:22.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.224+0000 7f0b577fe700 1 -- 172.21.15.47:0/3130106062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b580f7400 msgr2=0x7f0b581091a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:22.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.224+0000 7f0b577fe700 1 --2- 172.21.15.47:0/3130106062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b580f7400 0x7f0b581091a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:22.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.224+0000 7f0b577fe700 1 -- 172.21.15.47:0/3130106062 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b500097c0 con 0x7f0b580fcbb0 2024-02-23T00:58:22.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.224+0000 7f0b577fe700 1 --2- 172.21.15.47:0/3130106062 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0b580fcbb0 0x7f0b58109690 secure :-1 s=READY pgs=2875 cs=0 l=1 rev1=1 rx=0x7f0b4000e970 tx=0x7f0b4000b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:58:22.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.224+0000 7f0b557fa700 10 client.?.objecter ms_handle_connect 0x7f0b580fcbb0 2024-02-23T00:58:22.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.224+0000 7f0b557fa700 10 client.?.objecter resend_mon_ops 2024-02-23T00:58:22.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.224+0000 7f0b557fa700 1 -- 172.21.15.47:0/3130106062 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b40004660 con 0x7f0b580fcbb0 2024-02-23T00:58:22.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.224+0000 7f0b557fa700 1 -- 172.21.15.47:0/3130106062 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f0b4000f040 con 0x7f0b580fcbb0 2024-02-23T00:58:22.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.224+0000 7f0b5e484700 1 -- 172.21.15.47:0/3130106062 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b48005040 con 0x7f0b580fcbb0 2024-02-23T00:58:22.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.224+0000 7f0b557fa700 1 -- 172.21.15.47:0/3130106062 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b400146f0 con 0x7f0b580fcbb0 2024-02-23T00:58:22.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.224+0000 7f0b5e484700 10 client.42437.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:58:22.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.224+0000 7f0b5e484700 1 -- 172.21.15.47:0/3130106062 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b58114d50 con 0x7f0b580fcbb0 2024-02-23T00:58:22.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.224+0000 7f0b5e484700 1 librados: init done 2024-02-23T00:58:22.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.225+0000 7f0b5e484700 1 -- 172.21.15.47:0/3130106062 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b5805fa60 con 0x7f0b580fcbb0 2024-02-23T00:58:22.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.225+0000 7f0b557fa700 1 -- 172.21.15.47:0/3130106062 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f0b40007b40 con 0x7f0b580fcbb0 2024-02-23T00:58:22.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.226+0000 7f0b557fa700 1 --2- 172.21.15.47:0/3130106062 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0b44062cc0 0x7f0b44065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:22.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.226+0000 7f0b557fa700 1 -- 172.21.15.47:0/3130106062 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(859..859 src has 806..859) v4 ==== 8663+0+0 (secure 0 0 0) 0x7f0b4007ce70 con 0x7f0b580fcbb0 2024-02-23T00:58:22.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.226+0000 7f0b557fa700 10 client.42437.objecter ms_dispatch 0x7f0b58001550 osd_map(859..859 src has 806..859) v4 2024-02-23T00:58:22.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.226+0000 7f0b557fa700 3 client.42437.objecter handle_osd_map got epochs [859,859] > 0 2024-02-23T00:58:22.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.226+0000 7f0b557fa700 3 client.42437.objecter handle_osd_map decoding full epoch 859 2024-02-23T00:58:22.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.226+0000 7f0b57fff700 1 --2- 172.21.15.47:0/3130106062 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0b44062cc0 0x7f0b44065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:22.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.226+0000 7f0b557fa700 20 client.42437.objecter dump_active .. 0 homeless 2024-02-23T00:58:22.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.226+0000 7f0b57fff700 1 --2- 172.21.15.47:0/3130106062 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0b44062cc0 0x7f0b44065180 secure :-1 s=READY pgs=3256 cs=0 l=1 rev1=1 rx=0x7f0b48005010 tx=0x7f0b48009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:58:22.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.227+0000 7f0b557fa700 10 client.42437.objecter ms_handle_connect 0x7f0b44062cc0 2024-02-23T00:58:22.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.231+0000 7f0b557fa700 1 -- 172.21.15.47:0/3130106062 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f0b4004c7e0 con 0x7f0b580fcbb0 2024-02-23T00:58:22.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.404+0000 7f0b5e484700 1 -- 172.21.15.47:0/3130106062 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f0b58105ef0 con 0x7f0b580fcbb0 2024-02-23T00:58:22.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.404+0000 7f0b557fa700 1 -- 172.21.15.47:0/3130106062 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0b40019070 con 0x7f0b580fcbb0 2024-02-23T00:58:22.408 INFO:teuthology.orchestra.run.smithi047.stdout:137438953722 2024-02-23T00:58:22.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.406+0000 7f0b5e484700 10 librados: watch_flush enter 2024-02-23T00:58:22.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.406+0000 7f0b5e484700 10 librados: watch_flush exit 2024-02-23T00:58:22.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.406+0000 7f0b5e484700 20 client.42437.objecter shutdown clearing up homeless session... 2024-02-23T00:58:22.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.406+0000 7f0b5e484700 10 client.42437.objecter successfully canceled tick 2024-02-23T00:58:22.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.406+0000 7f0b5e484700 1 -- 172.21.15.47:0/3130106062 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0b44062cc0 msgr2=0x7f0b44065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:22.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.406+0000 7f0b5e484700 1 --2- 172.21.15.47:0/3130106062 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0b44062cc0 0x7f0b44065180 secure :-1 s=READY pgs=3256 cs=0 l=1 rev1=1 rx=0x7f0b48005010 tx=0x7f0b48009cb0).stop 2024-02-23T00:58:22.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.406+0000 7f0b5e484700 1 -- 172.21.15.47:0/3130106062 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0b580fcbb0 msgr2=0x7f0b58109690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:22.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.406+0000 7f0b5e484700 1 --2- 172.21.15.47:0/3130106062 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0b580fcbb0 0x7f0b58109690 secure :-1 s=READY pgs=2875 cs=0 l=1 rev1=1 rx=0x7f0b4000e970 tx=0x7f0b4000b040).stop 2024-02-23T00:58:22.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.406+0000 7f0b5e484700 1 -- 172.21.15.47:0/3130106062 shutdown_connections 2024-02-23T00:58:22.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.406+0000 7f0b5e484700 1 --2- 172.21.15.47:0/3130106062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b580fd510 0x7f0b58109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:22.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.406+0000 7f0b5e484700 1 --2- 172.21.15.47:0/3130106062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b580f7400 0x7f0b581091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:22.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.406+0000 7f0b5e484700 1 --2- 172.21.15.47:0/3130106062 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0b44062cc0 0x7f0b44065180 unknown :-1 s=CLOSED pgs=3256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:22.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.406+0000 7f0b5e484700 1 --2- 172.21.15.47:0/3130106062 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0b580fcbb0 0x7f0b58109690 unknown :-1 s=CLOSED pgs=2875 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:22.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.406+0000 7f0b5e484700 1 -- 172.21.15.47:0/3130106062 >> 172.21.15.47:0/3130106062 conn(0x7f0b580f2820 msgr2=0x7f0b580fb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:58:22.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.407+0000 7f0b5e484700 1 -- 172.21.15.47:0/3130106062 shutdown_connections 2024-02-23T00:58:22.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.407+0000 7f0b5e484700 1 -- 172.21.15.47:0/3130106062 wait complete. 2024-02-23T00:58:22.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:22.407+0000 7f0b5e484700 1 librados: shutdown 2024-02-23T00:58:22.694 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:58:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:58:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:58:22.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:21.444642+0000 mgr.y (mgr.14517) 764 : cluster [DBG] pgmap v1349: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:58:22.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:21.469365+0000 osd.7 (osd.7) 445 : cluster [DBG] 5.1a scrub starts 2024-02-23T00:58:22.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:21.472517+0000 osd.7 (osd.7) 446 : cluster [DBG] 5.1a scrub ok 2024-02-23T00:58:22.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:58:22.405802+0000 mon.c (mon.2) 539 : audit [DBG] from='client.? 172.21.15.47:0/3130106062' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-23T00:58:22.751 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:21.444642+0000 mgr.y (mgr.14517) 764 : cluster [DBG] pgmap v1349: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:58:22.752 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:21.469365+0000 osd.7 (osd.7) 445 : cluster [DBG] 5.1a scrub starts 2024-02-23T00:58:22.752 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:21.472517+0000 osd.7 (osd.7) 446 : cluster [DBG] 5.1a scrub ok 2024-02-23T00:58:22.752 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:58:22.405802+0000 mon.c (mon.2) 539 : audit [DBG] from='client.? 172.21.15.47:0/3130106062' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-23T00:58:22.752 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:21.444642+0000 mgr.y (mgr.14517) 764 : cluster [DBG] pgmap v1349: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:58:22.752 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:21.469365+0000 osd.7 (osd.7) 445 : cluster [DBG] 5.1a scrub starts 2024-02-23T00:58:22.752 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:21.472517+0000 osd.7 (osd.7) 446 : cluster [DBG] 5.1a scrub ok 2024-02-23T00:58:22.752 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:22 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:58:22.405802+0000 mon.c (mon.2) 539 : audit [DBG] from='client.? 172.21.15.47:0/3130106062' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-23T00:58:23.040 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953718 got 137438953722 for osd.4 2024-02-23T00:58:23.040 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-23T00:58:23.325 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:58:23.444 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:58:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:58:23.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:22.491351+0000 osd.7 (osd.7) 447 : cluster [DBG] 6.3 scrub starts 2024-02-23T00:58:23.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:22.494491+0000 osd.7 (osd.7) 448 : cluster [DBG] 6.3 scrub ok 2024-02-23T00:58:23.792 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:22.491351+0000 osd.7 (osd.7) 447 : cluster [DBG] 6.3 scrub starts 2024-02-23T00:58:23.792 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:22.494491+0000 osd.7 (osd.7) 448 : cluster [DBG] 6.3 scrub ok 2024-02-23T00:58:23.792 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:22.491351+0000 osd.7 (osd.7) 447 : cluster [DBG] 6.3 scrub starts 2024-02-23T00:58:23.792 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:23 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:22.494491+0000 osd.7 (osd.7) 448 : cluster [DBG] 6.3 scrub ok 2024-02-23T00:58:24.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:58:23.116585+0000 mgr.y (mgr.14517) 765 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:58:24.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:23.403550+0000 osd.2 (osd.2) 227 : cluster [DBG] 18.7 scrub starts 2024-02-23T00:58:24.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:23.405486+0000 osd.2 (osd.2) 228 : cluster [DBG] 18.7 scrub ok 2024-02-23T00:58:24.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:23.445356+0000 mgr.y (mgr.14517) 766 : cluster [DBG] pgmap v1350: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:58:24.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:23.608410+0000 osd.6 (osd.6) 389 : cluster [DBG] 2.17 scrub starts 2024-02-23T00:58:24.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:23.609377+0000 osd.6 (osd.6) 390 : cluster [DBG] 2.17 scrub ok 2024-02-23T00:58:24.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:23.774271+0000 osd.3 (osd.3) 373 : cluster [DBG] 18.1 scrub starts 2024-02-23T00:58:24.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:23.776252+0000 osd.3 (osd.3) 374 : cluster [DBG] 18.1 scrub ok 2024-02-23T00:58:24.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:58:23.116585+0000 mgr.y (mgr.14517) 765 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:58:24.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:23.403550+0000 osd.2 (osd.2) 227 : cluster [DBG] 18.7 scrub starts 2024-02-23T00:58:24.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:23.405486+0000 osd.2 (osd.2) 228 : cluster [DBG] 18.7 scrub ok 2024-02-23T00:58:24.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:23.445356+0000 mgr.y (mgr.14517) 766 : cluster [DBG] pgmap v1350: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:58:24.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:23.608410+0000 osd.6 (osd.6) 389 : cluster [DBG] 2.17 scrub starts 2024-02-23T00:58:24.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:23.609377+0000 osd.6 (osd.6) 390 : cluster [DBG] 2.17 scrub ok 2024-02-23T00:58:24.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:23.774271+0000 osd.3 (osd.3) 373 : cluster [DBG] 18.1 scrub starts 2024-02-23T00:58:24.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:24 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:23.776252+0000 osd.3 (osd.3) 374 : cluster [DBG] 18.1 scrub ok 2024-02-23T00:58:24.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:58:23.116585+0000 mgr.y (mgr.14517) 765 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:58:24.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:23.403550+0000 osd.2 (osd.2) 227 : cluster [DBG] 18.7 scrub starts 2024-02-23T00:58:24.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:23.405486+0000 osd.2 (osd.2) 228 : cluster [DBG] 18.7 scrub ok 2024-02-23T00:58:24.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:23.445356+0000 mgr.y (mgr.14517) 766 : cluster [DBG] pgmap v1350: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:58:24.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:23.608410+0000 osd.6 (osd.6) 389 : cluster [DBG] 2.17 scrub starts 2024-02-23T00:58:24.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:23.609377+0000 osd.6 (osd.6) 390 : cluster [DBG] 2.17 scrub ok 2024-02-23T00:58:24.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:23.774271+0000 osd.3 (osd.3) 373 : cluster [DBG] 18.1 scrub starts 2024-02-23T00:58:24.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:23.776252+0000 osd.3 (osd.3) 374 : cluster [DBG] 18.1 scrub ok 2024-02-23T00:58:25.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.081+0000 7f1de4deb700 1 -- 172.21.15.47:0/1026253498 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1de00fd510 msgr2=0x7f1de00f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:25.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.081+0000 7f1de4deb700 1 --2- 172.21.15.47:0/1026253498 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1de00fd510 0x7f1de00f6d40 secure :-1 s=READY pgs=2876 cs=0 l=1 rev1=1 rx=0x7f1dc8004870 tx=0x7f1dc8009cb0).stop 2024-02-23T00:58:25.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.081+0000 7f1de4deb700 1 -- 172.21.15.47:0/1026253498 shutdown_connections 2024-02-23T00:58:25.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.081+0000 7f1de4deb700 1 --2- 172.21.15.47:0/1026253498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1de00f7400 0x7f1de0103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:25.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.081+0000 7f1de4deb700 1 --2- 172.21.15.47:0/1026253498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1de00fcbb0 0x7f1de00fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:25.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.081+0000 7f1de4deb700 1 --2- 172.21.15.47:0/1026253498 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1de00fd510 0x7f1de00f6d40 unknown :-1 s=CLOSED pgs=2876 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:25.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.081+0000 7f1de4deb700 1 -- 172.21.15.47:0/1026253498 >> 172.21.15.47:0/1026253498 conn(0x7f1de00f2820 msgr2=0x7f1de00f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:58:25.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.081+0000 7f1de4deb700 1 -- 172.21.15.47:0/1026253498 shutdown_connections 2024-02-23T00:58:25.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.081+0000 7f1de4deb700 1 -- 172.21.15.47:0/1026253498 wait complete. 2024-02-23T00:58:25.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.082+0000 7f1de4deb700 1 librados: starting msgr at 2024-02-23T00:58:25.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.082+0000 7f1de4deb700 1 librados: starting objecter 2024-02-23T00:58:25.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.082+0000 7f1de4deb700 1 Processor -- start 2024-02-23T00:58:25.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.082+0000 7f1de4deb700 1 -- start start 2024-02-23T00:58:25.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.082+0000 7f1de4deb700 1 librados: setting wanted keys 2024-02-23T00:58:25.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.082+0000 7f1de4deb700 1 librados: calling monclient init 2024-02-23T00:58:25.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.083+0000 7f1de4deb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1de00f7400 0x7f1de01091b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:25.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.083+0000 7f1de4deb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1de00fcbb0 0x7f1de01096a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:25.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.083+0000 7f1de4deb700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1de00fd510 0x7f1de0109cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:25.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.083+0000 7f1de4deb700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de0114410 con 0x7f1de00f7400 2024-02-23T00:58:25.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.083+0000 7f1de4deb700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de0114580 con 0x7f1de00fcbb0 2024-02-23T00:58:25.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.083+0000 7f1de4deb700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1de0114740 con 0x7f1de00fd510 2024-02-23T00:58:25.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.083+0000 7f1dde59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1de00f7400 0x7f1de01091b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:25.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.083+0000 7f1dde59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1de00f7400 0x7f1de01091b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47186/0 (socket says 172.21.15.47:47186) 2024-02-23T00:58:25.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.083+0000 7f1dde59c700 1 -- 172.21.15.47:0/3170626374 learned_addr learned my addr 172.21.15.47:0/3170626374 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:58:25.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.083+0000 7f1dded9d700 1 --2- 172.21.15.47:0/3170626374 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1de00fd510 0x7f1de0109cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:25.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.083+0000 7f1dddd9b700 1 --2- 172.21.15.47:0/3170626374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1de00fcbb0 0x7f1de01096a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:25.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.083+0000 7f1dde59c700 1 -- 172.21.15.47:0/3170626374 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1de00fd510 msgr2=0x7f1de0109cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:25.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.083+0000 7f1dde59c700 1 --2- 172.21.15.47:0/3170626374 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1de00fd510 0x7f1de0109cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:25.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.083+0000 7f1dde59c700 1 -- 172.21.15.47:0/3170626374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1de00fcbb0 msgr2=0x7f1de01096a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:25.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.083+0000 7f1dde59c700 1 --2- 172.21.15.47:0/3170626374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1de00fcbb0 0x7f1de01096a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:25.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.083+0000 7f1dde59c700 1 -- 172.21.15.47:0/3170626374 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd0005040 con 0x7f1de00f7400 2024-02-23T00:58:25.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.083+0000 7f1dde59c700 1 --2- 172.21.15.47:0/3170626374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1de00f7400 0x7f1de01091b0 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 rx=0x7f1dd00048f0 tx=0x7f1dd0009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:58:25.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.083+0000 7f1dd77fe700 10 client.?.objecter ms_handle_connect 0x7f1de00f7400 2024-02-23T00:58:25.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.083+0000 7f1dd77fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:58:25.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.084+0000 7f1dd77fe700 1 -- 172.21.15.47:0/3170626374 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0007b00 con 0x7f1de00f7400 2024-02-23T00:58:25.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.084+0000 7f1de4deb700 1 -- 172.21.15.47:0/3170626374 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc8005040 con 0x7f1de00f7400 2024-02-23T00:58:25.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.084+0000 7f1dd77fe700 1 -- 172.21.15.47:0/3170626374 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f1dd000f040 con 0x7f1de00f7400 2024-02-23T00:58:25.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.084+0000 7f1dd77fe700 1 -- 172.21.15.47:0/3170626374 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0014890 con 0x7f1de00f7400 2024-02-23T00:58:25.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.084+0000 7f1de4deb700 10 client.32121.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:58:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.084+0000 7f1de4deb700 1 -- 172.21.15.47:0/3170626374 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1de0114d80 con 0x7f1de00f7400 2024-02-23T00:58:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.084+0000 7f1de4deb700 1 librados: init done 2024-02-23T00:58:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.084+0000 7f1de4deb700 1 -- 172.21.15.47:0/3170626374 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1de005fa60 con 0x7f1de00f7400 2024-02-23T00:58:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.085+0000 7f1dd77fe700 1 -- 172.21.15.47:0/3170626374 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f1dd0014cc0 con 0x7f1de00f7400 2024-02-23T00:58:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.086+0000 7f1dd77fe700 1 --2- 172.21.15.47:0/3170626374 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1dcc062cc0 0x7f1dcc065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.086+0000 7f1dddd9b700 1 --2- 172.21.15.47:0/3170626374 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1dcc062cc0 0x7f1dcc065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.086+0000 7f1dd77fe700 1 -- 172.21.15.47:0/3170626374 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(859..859 src has 806..859) v4 ==== 8663+0+0 (secure 0 0 0) 0x7f1dd007d050 con 0x7f1de00f7400 2024-02-23T00:58:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.086+0000 7f1dd77fe700 10 client.32121.objecter ms_dispatch 0x7f1de0001550 osd_map(859..859 src has 806..859) v4 2024-02-23T00:58:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.086+0000 7f1dd77fe700 3 client.32121.objecter handle_osd_map got epochs [859,859] > 0 2024-02-23T00:58:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.086+0000 7f1dd77fe700 3 client.32121.objecter handle_osd_map decoding full epoch 859 2024-02-23T00:58:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.086+0000 7f1dd77fe700 20 client.32121.objecter dump_active .. 0 homeless 2024-02-23T00:58:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.086+0000 7f1dd77fe700 10 client.32121.objecter ms_handle_connect 0x7f1dcc062cc0 2024-02-23T00:58:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.086+0000 7f1dddd9b700 1 --2- 172.21.15.47:0/3170626374 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1dcc062cc0 0x7f1dcc065180 secure :-1 s=READY pgs=3257 cs=0 l=1 rev1=1 rx=0x7f1dc8004ee0 tx=0x7f1dc800b040).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:58:25.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.090+0000 7f1dd77fe700 1 -- 172.21.15.47:0/3170626374 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f1dd0049f60 con 0x7f1de00f7400 2024-02-23T00:58:25.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.260+0000 7f1de4deb700 1 -- 172.21.15.47:0/3170626374 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f1de0061460 con 0x7f1de00f7400 2024-02-23T00:58:25.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.261+0000 7f1dd77fe700 1 -- 172.21.15.47:0/3170626374 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1dd004c890 con 0x7f1de00f7400 2024-02-23T00:58:25.264 INFO:teuthology.orchestra.run.smithi047.stdout:167503724790 2024-02-23T00:58:25.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.262+0000 7f1de4deb700 10 librados: watch_flush enter 2024-02-23T00:58:25.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.262+0000 7f1de4deb700 10 librados: watch_flush exit 2024-02-23T00:58:25.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.262+0000 7f1de4deb700 20 client.32121.objecter shutdown clearing up homeless session... 2024-02-23T00:58:25.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.262+0000 7f1de4deb700 10 client.32121.objecter successfully canceled tick 2024-02-23T00:58:25.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.262+0000 7f1de4deb700 1 -- 172.21.15.47:0/3170626374 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1dcc062cc0 msgr2=0x7f1dcc065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:25.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.262+0000 7f1de4deb700 1 --2- 172.21.15.47:0/3170626374 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1dcc062cc0 0x7f1dcc065180 secure :-1 s=READY pgs=3257 cs=0 l=1 rev1=1 rx=0x7f1dc8004ee0 tx=0x7f1dc800b040).stop 2024-02-23T00:58:25.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.262+0000 7f1de4deb700 1 -- 172.21.15.47:0/3170626374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1de00f7400 msgr2=0x7f1de01091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:25.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.262+0000 7f1de4deb700 1 --2- 172.21.15.47:0/3170626374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1de00f7400 0x7f1de01091b0 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 rx=0x7f1dd00048f0 tx=0x7f1dd0009cb0).stop 2024-02-23T00:58:25.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.262+0000 7f1de4deb700 1 -- 172.21.15.47:0/3170626374 shutdown_connections 2024-02-23T00:58:25.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.262+0000 7f1de4deb700 1 --2- 172.21.15.47:0/3170626374 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1dcc062cc0 0x7f1dcc065180 unknown :-1 s=CLOSED pgs=3257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:25.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.262+0000 7f1de4deb700 1 --2- 172.21.15.47:0/3170626374 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1de00fd510 0x7f1de0109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:25.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.262+0000 7f1de4deb700 1 --2- 172.21.15.47:0/3170626374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1de00f7400 0x7f1de01091b0 unknown :-1 s=CLOSED pgs=2687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:25.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.262+0000 7f1de4deb700 1 --2- 172.21.15.47:0/3170626374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1de00fcbb0 0x7f1de01096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:25.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.262+0000 7f1de4deb700 1 -- 172.21.15.47:0/3170626374 >> 172.21.15.47:0/3170626374 conn(0x7f1de00f2820 msgr2=0x7f1de00f4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:58:25.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.262+0000 7f1de4deb700 1 -- 172.21.15.47:0/3170626374 shutdown_connections 2024-02-23T00:58:25.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.262+0000 7f1de4deb700 1 -- 172.21.15.47:0/3170626374 wait complete. 2024-02-23T00:58:25.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:25.263+0000 7f1de4deb700 1 librados: shutdown 2024-02-23T00:58:25.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:24.484834+0000 osd.7 (osd.7) 449 : cluster [DBG] 18.2 scrub starts 2024-02-23T00:58:25.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:24.488327+0000 osd.7 (osd.7) 450 : cluster [DBG] 18.2 scrub ok 2024-02-23T00:58:25.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:24.797785+0000 osd.3 (osd.3) 375 : cluster [DBG] 6.b scrub starts 2024-02-23T00:58:25.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:24.799085+0000 osd.3 (osd.3) 376 : cluster [DBG] 6.b scrub ok 2024-02-23T00:58:25.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:58:25.262155+0000 mon.a (mon.0) 3949 : audit [DBG] from='client.? 172.21.15.47:0/3170626374' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-23T00:58:25.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:24.484834+0000 osd.7 (osd.7) 449 : cluster [DBG] 18.2 scrub starts 2024-02-23T00:58:25.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:24.488327+0000 osd.7 (osd.7) 450 : cluster [DBG] 18.2 scrub ok 2024-02-23T00:58:25.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:24.797785+0000 osd.3 (osd.3) 375 : cluster [DBG] 6.b scrub starts 2024-02-23T00:58:25.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:24.799085+0000 osd.3 (osd.3) 376 : cluster [DBG] 6.b scrub ok 2024-02-23T00:58:25.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:25 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:58:25.262155+0000 mon.a (mon.0) 3949 : audit [DBG] from='client.? 172.21.15.47:0/3170626374' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-23T00:58:25.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:24.484834+0000 osd.7 (osd.7) 449 : cluster [DBG] 18.2 scrub starts 2024-02-23T00:58:25.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:24.488327+0000 osd.7 (osd.7) 450 : cluster [DBG] 18.2 scrub ok 2024-02-23T00:58:25.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:24.797785+0000 osd.3 (osd.3) 375 : cluster [DBG] 6.b scrub starts 2024-02-23T00:58:25.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:24.799085+0000 osd.3 (osd.3) 376 : cluster [DBG] 6.b scrub ok 2024-02-23T00:58:25.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:58:25.262155+0000 mon.a (mon.0) 3949 : audit [DBG] from='client.? 172.21.15.47:0/3170626374' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-23T00:58:26.196 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724786 got 167503724790 for osd.5 2024-02-23T00:58:26.196 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-02-23T00:58:26.453 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:25.340866+0000 osd.4 (osd.4) 409 : cluster [DBG] 4.c scrub starts 2024-02-23T00:58:26.453 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:25.342541+0000 osd.4 (osd.4) 410 : cluster [DBG] 4.c scrub ok 2024-02-23T00:58:26.453 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:25.445709+0000 mgr.y (mgr.14517) 767 : cluster [DBG] pgmap v1351: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:58:26.453 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:25.652516+0000 osd.6 (osd.6) 391 : cluster [DBG] 2.11 scrub starts 2024-02-23T00:58:26.453 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:25.653694+0000 osd.6 (osd.6) 392 : cluster [DBG] 2.11 scrub ok 2024-02-23T00:58:26.454 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:26.131070+0000 osd.5 (osd.5) 430 : cluster [DBG] 18.b deep-scrub starts 2024-02-23T00:58:26.454 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:26.133289+0000 osd.5 (osd.5) 431 : cluster [DBG] 18.b deep-scrub ok 2024-02-23T00:58:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:58:26.721 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:25.340866+0000 osd.4 (osd.4) 409 : cluster [DBG] 4.c scrub starts 2024-02-23T00:58:26.721 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:25.342541+0000 osd.4 (osd.4) 410 : cluster [DBG] 4.c scrub ok 2024-02-23T00:58:26.721 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:25.445709+0000 mgr.y (mgr.14517) 767 : cluster [DBG] pgmap v1351: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:58:26.721 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:25.652516+0000 osd.6 (osd.6) 391 : cluster [DBG] 2.11 scrub starts 2024-02-23T00:58:26.721 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:25.653694+0000 osd.6 (osd.6) 392 : cluster [DBG] 2.11 scrub ok 2024-02-23T00:58:26.721 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:26.131070+0000 osd.5 (osd.5) 430 : cluster [DBG] 18.b deep-scrub starts 2024-02-23T00:58:26.721 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:26 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:26.133289+0000 osd.5 (osd.5) 431 : cluster [DBG] 18.b deep-scrub ok 2024-02-23T00:58:26.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:25.340866+0000 osd.4 (osd.4) 409 : cluster [DBG] 4.c scrub starts 2024-02-23T00:58:26.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:25.342541+0000 osd.4 (osd.4) 410 : cluster [DBG] 4.c scrub ok 2024-02-23T00:58:26.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:25.445709+0000 mgr.y (mgr.14517) 767 : cluster [DBG] pgmap v1351: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:58:26.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:25.652516+0000 osd.6 (osd.6) 391 : cluster [DBG] 2.11 scrub starts 2024-02-23T00:58:26.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:25.653694+0000 osd.6 (osd.6) 392 : cluster [DBG] 2.11 scrub ok 2024-02-23T00:58:26.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:26.131070+0000 osd.5 (osd.5) 430 : cluster [DBG] 18.b deep-scrub starts 2024-02-23T00:58:26.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:26.133289+0000 osd.5 (osd.5) 431 : cluster [DBG] 18.b deep-scrub ok 2024-02-23T00:58:27.731 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:25.934800+0000 osd.0 (osd.0) 362 : cluster [DBG] 6.0 deep-scrub starts 2024-02-23T00:58:27.731 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:25.936370+0000 osd.0 (osd.0) 363 : cluster [DBG] 6.0 deep-scrub ok 2024-02-23T00:58:27.731 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:27.130935+0000 osd.5 (osd.5) 432 : cluster [DBG] 2.2 scrub starts 2024-02-23T00:58:27.731 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:27.131866+0000 osd.5 (osd.5) 433 : cluster [DBG] 2.2 scrub ok 2024-02-23T00:58:27.732 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:25.934800+0000 osd.0 (osd.0) 362 : cluster [DBG] 6.0 deep-scrub starts 2024-02-23T00:58:27.732 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:25.936370+0000 osd.0 (osd.0) 363 : cluster [DBG] 6.0 deep-scrub ok 2024-02-23T00:58:27.732 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:27.130935+0000 osd.5 (osd.5) 432 : cluster [DBG] 2.2 scrub starts 2024-02-23T00:58:27.732 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:27 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:27.131866+0000 osd.5 (osd.5) 433 : cluster [DBG] 2.2 scrub ok 2024-02-23T00:58:27.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:25.934800+0000 osd.0 (osd.0) 362 : cluster [DBG] 6.0 deep-scrub starts 2024-02-23T00:58:27.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:25.936370+0000 osd.0 (osd.0) 363 : cluster [DBG] 6.0 deep-scrub ok 2024-02-23T00:58:27.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:27.130935+0000 osd.5 (osd.5) 432 : cluster [DBG] 2.2 scrub starts 2024-02-23T00:58:27.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:27.131866+0000 osd.5 (osd.5) 433 : cluster [DBG] 2.2 scrub ok 2024-02-23T00:58:28.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.256+0000 7f7f787f2700 1 -- 172.21.15.47:0/3227437777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f700f6a90 msgr2=0x7f7f70104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:28.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.256+0000 7f7f787f2700 1 --2- 172.21.15.47:0/3227437777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f700f6a90 0x7f7f70104a20 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 rx=0x7f7f68009be0 tx=0x7f7f68004d10).stop 2024-02-23T00:58:28.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.257+0000 7f7f787f2700 1 -- 172.21.15.47:0/3227437777 shutdown_connections 2024-02-23T00:58:28.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.257+0000 7f7f787f2700 1 --2- 172.21.15.47:0/3227437777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f700f6a90 0x7f7f70104a20 unknown :-1 s=CLOSED pgs=2688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:28.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.257+0000 7f7f787f2700 1 --2- 172.21.15.47:0/3227437777 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7f700fdbd0 0x7f7f700fdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:28.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.257+0000 7f7f787f2700 1 --2- 172.21.15.47:0/3227437777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f700fe530 0x7f7f700f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:28.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.257+0000 7f7f787f2700 1 -- 172.21.15.47:0/3227437777 >> 172.21.15.47:0/3227437777 conn(0x7f7f7006c370 msgr2=0x7f7f7006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:58:28.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.257+0000 7f7f787f2700 1 -- 172.21.15.47:0/3227437777 shutdown_connections 2024-02-23T00:58:28.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.257+0000 7f7f787f2700 1 -- 172.21.15.47:0/3227437777 wait complete. 2024-02-23T00:58:28.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.257+0000 7f7f787f2700 1 librados: starting msgr at 2024-02-23T00:58:28.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.257+0000 7f7f787f2700 1 librados: starting objecter 2024-02-23T00:58:28.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.257+0000 7f7f787f2700 1 Processor -- start 2024-02-23T00:58:28.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.257+0000 7f7f787f2700 1 -- start start 2024-02-23T00:58:28.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.257+0000 7f7f787f2700 1 librados: setting wanted keys 2024-02-23T00:58:28.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.257+0000 7f7f787f2700 1 librados: calling monclient init 2024-02-23T00:58:28.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.258+0000 7f7f787f2700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f700f6a90 0x7f7f700f7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:28.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.258+0000 7f7f787f2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f700fdbd0 0x7f7f700f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:28.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.258+0000 7f7f787f2700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7f700fe530 0x7f7f700f8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:28.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.258+0000 7f7f787f2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f70116200 con 0x7f7f700fdbd0 2024-02-23T00:58:28.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.258+0000 7f7f787f2700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f70116370 con 0x7f7f700f6a90 2024-02-23T00:58:28.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.258+0000 7f7f787f2700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f70116530 con 0x7f7f700fe530 2024-02-23T00:58:28.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.258+0000 7f7f75d8d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f700fdbd0 0x7f7f700f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:28.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.258+0000 7f7f76d8f700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7f700fe530 0x7f7f700f8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:28.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.258+0000 7f7f75d8d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f700fdbd0 0x7f7f700f7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50750/0 (socket says 172.21.15.47:50750) 2024-02-23T00:58:28.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.258+0000 7f7f75d8d700 1 -- 172.21.15.47:0/658049535 learned_addr learned my addr 172.21.15.47:0/658049535 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:58:28.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.258+0000 7f7f7658e700 1 --2- 172.21.15.47:0/658049535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f700f6a90 0x7f7f700f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:28.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.258+0000 7f7f75d8d700 1 -- 172.21.15.47:0/658049535 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7f700fe530 msgr2=0x7f7f700f8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:28.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.258+0000 7f7f75d8d700 1 --2- 172.21.15.47:0/658049535 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7f700fe530 0x7f7f700f8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:28.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.258+0000 7f7f75d8d700 1 -- 172.21.15.47:0/658049535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f700f6a90 msgr2=0x7f7f700f7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:28.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.258+0000 7f7f75d8d700 1 --2- 172.21.15.47:0/658049535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f700f6a90 0x7f7f700f7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:28.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.258+0000 7f7f75d8d700 1 -- 172.21.15.47:0/658049535 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6c005040 con 0x7f7f700fdbd0 2024-02-23T00:58:28.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.258+0000 7f7f7658e700 1 --2- 172.21.15.47:0/658049535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f700f6a90 0x7f7f700f7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:58:28.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.258+0000 7f7f75d8d700 1 --2- 172.21.15.47:0/658049535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f700fdbd0 0x7f7f700f7b70 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 rx=0x7f7f6000c7c0 tx=0x7f7f60007ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:58:28.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.258+0000 7f7f677fe700 10 client.?.objecter ms_handle_connect 0x7f7f700fdbd0 2024-02-23T00:58:28.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.258+0000 7f7f677fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:58:28.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.258+0000 7f7f677fe700 1 -- 172.21.15.47:0/658049535 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f60004430 con 0x7f7f700fdbd0 2024-02-23T00:58:28.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.258+0000 7f7f677fe700 1 -- 172.21.15.47:0/658049535 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f7f6000e930 con 0x7f7f700fdbd0 2024-02-23T00:58:28.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.258+0000 7f7f787f2700 1 -- 172.21.15.47:0/658049535 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f680097c0 con 0x7f7f700fdbd0 2024-02-23T00:58:28.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.258+0000 7f7f677fe700 1 -- 172.21.15.47:0/658049535 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f60013990 con 0x7f7f700fdbd0 2024-02-23T00:58:28.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.258+0000 7f7f787f2700 10 client.32127.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:58:28.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.258+0000 7f7f787f2700 1 -- 172.21.15.47:0/658049535 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f70116bc0 con 0x7f7f700fdbd0 2024-02-23T00:58:28.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.258+0000 7f7f787f2700 1 librados: init done 2024-02-23T00:58:28.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.259+0000 7f7f787f2700 1 -- 172.21.15.47:0/658049535 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f7005f950 con 0x7f7f700fdbd0 2024-02-23T00:58:28.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.260+0000 7f7f677fe700 1 -- 172.21.15.47:0/658049535 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f7f6000e4a0 con 0x7f7f700fdbd0 2024-02-23T00:58:28.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.260+0000 7f7f677fe700 1 --2- 172.21.15.47:0/658049535 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7f5c062cc0 0x7f7f5c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:28.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.260+0000 7f7f677fe700 1 -- 172.21.15.47:0/658049535 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(859..859 src has 806..859) v4 ==== 8663+0+0 (secure 0 0 0) 0x7f7f6007d610 con 0x7f7f700fdbd0 2024-02-23T00:58:28.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.260+0000 7f7f677fe700 10 client.32127.objecter ms_dispatch 0x7f7f70001550 osd_map(859..859 src has 806..859) v4 2024-02-23T00:58:28.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.260+0000 7f7f677fe700 3 client.32127.objecter handle_osd_map got epochs [859,859] > 0 2024-02-23T00:58:28.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.260+0000 7f7f677fe700 3 client.32127.objecter handle_osd_map decoding full epoch 859 2024-02-23T00:58:28.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.260+0000 7f7f7658e700 1 --2- 172.21.15.47:0/658049535 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7f5c062cc0 0x7f7f5c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:28.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.261+0000 7f7f677fe700 20 client.32127.objecter dump_active .. 0 homeless 2024-02-23T00:58:28.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.261+0000 7f7f677fe700 10 client.32127.objecter ms_handle_connect 0x7f7f5c062cc0 2024-02-23T00:58:28.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.261+0000 7f7f7658e700 1 --2- 172.21.15.47:0/658049535 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7f5c062cc0 0x7f7f5c065180 secure :-1 s=READY pgs=3258 cs=0 l=1 rev1=1 rx=0x7f7f6c007fd0 tx=0x7f7f6c009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:58:28.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.265+0000 7f7f677fe700 1 -- 172.21.15.47:0/658049535 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f7f6004ced0 con 0x7f7f700fdbd0 2024-02-23T00:58:28.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.438+0000 7f7f787f2700 1 -- 172.21.15.47:0/658049535 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f7f700f8b60 con 0x7f7f700fdbd0 2024-02-23T00:58:28.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.438+0000 7f7f677fe700 1 -- 172.21.15.47:0/658049535 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7f600504f0 con 0x7f7f700fdbd0 2024-02-23T00:58:28.442 INFO:teuthology.orchestra.run.smithi047.stdout:201863463154 2024-02-23T00:58:28.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.440+0000 7f7f787f2700 10 librados: watch_flush enter 2024-02-23T00:58:28.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.440+0000 7f7f787f2700 10 librados: watch_flush exit 2024-02-23T00:58:28.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.440+0000 7f7f787f2700 20 client.32127.objecter shutdown clearing up homeless session... 2024-02-23T00:58:28.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.440+0000 7f7f787f2700 10 client.32127.objecter successfully canceled tick 2024-02-23T00:58:28.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.440+0000 7f7f787f2700 1 -- 172.21.15.47:0/658049535 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7f5c062cc0 msgr2=0x7f7f5c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:28.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.440+0000 7f7f787f2700 1 --2- 172.21.15.47:0/658049535 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7f5c062cc0 0x7f7f5c065180 secure :-1 s=READY pgs=3258 cs=0 l=1 rev1=1 rx=0x7f7f6c007fd0 tx=0x7f7f6c009cb0).stop 2024-02-23T00:58:28.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.440+0000 7f7f787f2700 1 -- 172.21.15.47:0/658049535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f700fdbd0 msgr2=0x7f7f700f7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:28.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.440+0000 7f7f787f2700 1 --2- 172.21.15.47:0/658049535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f700fdbd0 0x7f7f700f7b70 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 rx=0x7f7f6000c7c0 tx=0x7f7f60007ef0).stop 2024-02-23T00:58:28.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.440+0000 7f7f787f2700 1 -- 172.21.15.47:0/658049535 shutdown_connections 2024-02-23T00:58:28.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.440+0000 7f7f787f2700 1 --2- 172.21.15.47:0/658049535 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f7f5c062cc0 0x7f7f5c065180 unknown :-1 s=CLOSED pgs=3258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:28.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.440+0000 7f7f787f2700 1 --2- 172.21.15.47:0/658049535 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7f700fe530 0x7f7f700f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:28.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.440+0000 7f7f787f2700 1 --2- 172.21.15.47:0/658049535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f700f6a90 0x7f7f700f7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:28.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.440+0000 7f7f787f2700 1 --2- 172.21.15.47:0/658049535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f700fdbd0 0x7f7f700f7b70 unknown :-1 s=CLOSED pgs=2689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:28.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.440+0000 7f7f787f2700 1 -- 172.21.15.47:0/658049535 >> 172.21.15.47:0/658049535 conn(0x7f7f7006c370 msgr2=0x7f7f700f5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:58:28.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.440+0000 7f7f787f2700 1 -- 172.21.15.47:0/658049535 shutdown_connections 2024-02-23T00:58:28.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.440+0000 7f7f787f2700 1 -- 172.21.15.47:0/658049535 wait complete. 2024-02-23T00:58:28.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:28.440+0000 7f7f787f2700 1 librados: shutdown 2024-02-23T00:58:28.783 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:26.895690+0000 osd.0 (osd.0) 364 : cluster [DBG] 18.4 scrub starts 2024-02-23T00:58:28.783 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:26.897092+0000 osd.0 (osd.0) 365 : cluster [DBG] 18.4 scrub ok 2024-02-23T00:58:28.783 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:27.446423+0000 mgr.y (mgr.14517) 768 : cluster [DBG] pgmap v1352: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:58:28.783 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:27.660983+0000 osd.1 (osd.1) 368 : cluster [DBG] 4.f scrub starts 2024-02-23T00:58:28.783 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:27.662747+0000 osd.1 (osd.1) 369 : cluster [DBG] 4.f scrub ok 2024-02-23T00:58:28.783 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:58:28.439893+0000 mon.a (mon.0) 3950 : audit [DBG] from='client.? 172.21.15.47:0/658049535' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-23T00:58:28.783 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:26.895690+0000 osd.0 (osd.0) 364 : cluster [DBG] 18.4 scrub starts 2024-02-23T00:58:28.783 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:26.897092+0000 osd.0 (osd.0) 365 : cluster [DBG] 18.4 scrub ok 2024-02-23T00:58:28.784 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:27.446423+0000 mgr.y (mgr.14517) 768 : cluster [DBG] pgmap v1352: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:58:28.784 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:27.660983+0000 osd.1 (osd.1) 368 : cluster [DBG] 4.f scrub starts 2024-02-23T00:58:28.784 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:27.662747+0000 osd.1 (osd.1) 369 : cluster [DBG] 4.f scrub ok 2024-02-23T00:58:28.784 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:28 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:58:28.439893+0000 mon.a (mon.0) 3950 : audit [DBG] from='client.? 172.21.15.47:0/658049535' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-23T00:58:28.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:26.895690+0000 osd.0 (osd.0) 364 : cluster [DBG] 18.4 scrub starts 2024-02-23T00:58:28.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:26.897092+0000 osd.0 (osd.0) 365 : cluster [DBG] 18.4 scrub ok 2024-02-23T00:58:28.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:27.446423+0000 mgr.y (mgr.14517) 768 : cluster [DBG] pgmap v1352: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:58:28.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:27.660983+0000 osd.1 (osd.1) 368 : cluster [DBG] 4.f scrub starts 2024-02-23T00:58:28.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:27.662747+0000 osd.1 (osd.1) 369 : cluster [DBG] 4.f scrub ok 2024-02-23T00:58:28.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:58:28.439893+0000 mon.a (mon.0) 3950 : audit [DBG] from='client.? 172.21.15.47:0/658049535' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-23T00:58:29.205 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863463149 got 201863463154 for osd.6 2024-02-23T00:58:29.205 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-02-23T00:58:29.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:27.919689+0000 osd.0 (osd.0) 366 : cluster [DBG] 2.13 scrub starts 2024-02-23T00:58:29.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:27.920653+0000 osd.0 (osd.0) 367 : cluster [DBG] 2.13 scrub ok 2024-02-23T00:58:29.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:28.325039+0000 osd.4 (osd.4) 411 : cluster [DBG] 4.13 scrub starts 2024-02-23T00:58:29.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:28.326892+0000 osd.4 (osd.4) 412 : cluster [DBG] 4.13 scrub ok 2024-02-23T00:58:29.490 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:58:29.762 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:27.919689+0000 osd.0 (osd.0) 366 : cluster [DBG] 2.13 scrub starts 2024-02-23T00:58:29.762 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:27.920653+0000 osd.0 (osd.0) 367 : cluster [DBG] 2.13 scrub ok 2024-02-23T00:58:29.762 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:28.325039+0000 osd.4 (osd.4) 411 : cluster [DBG] 4.13 scrub starts 2024-02-23T00:58:29.762 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:29 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:28.326892+0000 osd.4 (osd.4) 412 : cluster [DBG] 4.13 scrub ok 2024-02-23T00:58:29.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:27.919689+0000 osd.0 (osd.0) 366 : cluster [DBG] 2.13 scrub starts 2024-02-23T00:58:29.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:27.920653+0000 osd.0 (osd.0) 367 : cluster [DBG] 2.13 scrub ok 2024-02-23T00:58:29.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:28.325039+0000 osd.4 (osd.4) 411 : cluster [DBG] 4.13 scrub starts 2024-02-23T00:58:29.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:28.326892+0000 osd.4 (osd.4) 412 : cluster [DBG] 4.13 scrub ok 2024-02-23T00:58:30.756 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:28.925064+0000 osd.0 (osd.0) 368 : cluster [DBG] 5.3 deep-scrub starts 2024-02-23T00:58:30.756 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:28.926131+0000 osd.0 (osd.0) 369 : cluster [DBG] 5.3 deep-scrub ok 2024-02-23T00:58:30.756 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:29.446765+0000 mgr.y (mgr.14517) 769 : cluster [DBG] pgmap v1353: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:58:30.756 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:29.684673+0000 osd.1 (osd.1) 370 : cluster [DBG] 2.4 scrub starts 2024-02-23T00:58:30.756 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:29.687814+0000 osd.1 (osd.1) 371 : cluster [DBG] 2.4 scrub ok 2024-02-23T00:58:30.757 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:29.846845+0000 osd.3 (osd.3) 377 : cluster [DBG] 5.0 deep-scrub starts 2024-02-23T00:58:30.757 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:29.849888+0000 osd.3 (osd.3) 378 : cluster [DBG] 5.0 deep-scrub ok 2024-02-23T00:58:30.757 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:28.925064+0000 osd.0 (osd.0) 368 : cluster [DBG] 5.3 deep-scrub starts 2024-02-23T00:58:30.757 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:28.926131+0000 osd.0 (osd.0) 369 : cluster [DBG] 5.3 deep-scrub ok 2024-02-23T00:58:30.757 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:29.446765+0000 mgr.y (mgr.14517) 769 : cluster [DBG] pgmap v1353: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:58:30.757 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:29.684673+0000 osd.1 (osd.1) 370 : cluster [DBG] 2.4 scrub starts 2024-02-23T00:58:30.757 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:29.687814+0000 osd.1 (osd.1) 371 : cluster [DBG] 2.4 scrub ok 2024-02-23T00:58:30.757 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:29.846845+0000 osd.3 (osd.3) 377 : cluster [DBG] 5.0 deep-scrub starts 2024-02-23T00:58:30.757 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:29.849888+0000 osd.3 (osd.3) 378 : cluster [DBG] 5.0 deep-scrub ok 2024-02-23T00:58:30.758 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:58:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:58:30] "GET /metrics HTTP/1.1" 200 187143 "" "Prometheus/2.33.4" 2024-02-23T00:58:30.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:28.925064+0000 osd.0 (osd.0) 368 : cluster [DBG] 5.3 deep-scrub starts 2024-02-23T00:58:30.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:28.926131+0000 osd.0 (osd.0) 369 : cluster [DBG] 5.3 deep-scrub ok 2024-02-23T00:58:30.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:29.446765+0000 mgr.y (mgr.14517) 769 : cluster [DBG] pgmap v1353: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:58:30.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:29.684673+0000 osd.1 (osd.1) 370 : cluster [DBG] 2.4 scrub starts 2024-02-23T00:58:30.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:29.687814+0000 osd.1 (osd.1) 371 : cluster [DBG] 2.4 scrub ok 2024-02-23T00:58:30.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:29.846845+0000 osd.3 (osd.3) 377 : cluster [DBG] 5.0 deep-scrub starts 2024-02-23T00:58:30.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:29.849888+0000 osd.3 (osd.3) 378 : cluster [DBG] 5.0 deep-scrub ok 2024-02-23T00:58:31.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.278+0000 7f4e9d8df700 1 -- 172.21.15.47:0/3465855573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e98102210 msgr2=0x7f4e98104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:31.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.278+0000 7f4e9d8df700 1 --2- 172.21.15.47:0/3465855573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e98102210 0x7f4e98104600 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 rx=0x7f4e90009c30 tx=0x7f4e90004c50).stop 2024-02-23T00:58:31.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.278+0000 7f4e9d8df700 1 -- 172.21.15.47:0/3465855573 shutdown_connections 2024-02-23T00:58:31.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.278+0000 7f4e9d8df700 1 --2- 172.21.15.47:0/3465855573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e98102210 0x7f4e98104600 unknown :-1 s=CLOSED pgs=2690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:31.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.278+0000 7f4e9d8df700 1 --2- 172.21.15.47:0/3465855573 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4e980f8cb0 0x7f4e980f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:31.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.278+0000 7f4e9d8df700 1 --2- 172.21.15.47:0/3465855573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e980f9610 0x7f4e98101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:31.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.278+0000 7f4e9d8df700 1 -- 172.21.15.47:0/3465855573 >> 172.21.15.47:0/3465855573 conn(0x7f4e980f2820 msgr2=0x7f4e980f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:58:31.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.278+0000 7f4e9d8df700 1 -- 172.21.15.47:0/3465855573 shutdown_connections 2024-02-23T00:58:31.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.278+0000 7f4e9d8df700 1 -- 172.21.15.47:0/3465855573 wait complete. 2024-02-23T00:58:31.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.279+0000 7f4e9d8df700 1 librados: starting msgr at 2024-02-23T00:58:31.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.279+0000 7f4e9d8df700 1 librados: starting objecter 2024-02-23T00:58:31.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.279+0000 7f4e9d8df700 1 Processor -- start 2024-02-23T00:58:31.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.279+0000 7f4e9d8df700 1 -- start start 2024-02-23T00:58:31.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.279+0000 7f4e9d8df700 1 librados: setting wanted keys 2024-02-23T00:58:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.279+0000 7f4e9d8df700 1 librados: calling monclient init 2024-02-23T00:58:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.279+0000 7f4e9d8df700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e980f8cb0 0x7f4e980f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.279+0000 7f4e9d8df700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e980f9610 0x7f4e980f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.279+0000 7f4e9d8df700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4e98102210 0x7f4e980f7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.279+0000 7f4e9d8df700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e98116350 con 0x7f4e980f9610 2024-02-23T00:58:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.279+0000 7f4e9d8df700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e981164c0 con 0x7f4e980f8cb0 2024-02-23T00:58:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.279+0000 7f4e9d8df700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e98116680 con 0x7f4e98102210 2024-02-23T00:58:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.279+0000 7f4e967fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e980f9610 0x7f4e980f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.279+0000 7f4e977fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4e98102210 0x7f4e980f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.280+0000 7f4e967fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e980f9610 0x7f4e980f7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50764/0 (socket says 172.21.15.47:50764) 2024-02-23T00:58:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.280+0000 7f4e967fc700 1 -- 172.21.15.47:0/1503619459 learned_addr learned my addr 172.21.15.47:0/1503619459 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:58:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.280+0000 7f4e96ffd700 1 --2- 172.21.15.47:0/1503619459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e980f8cb0 0x7f4e980f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.280+0000 7f4e977fe700 1 -- 172.21.15.47:0/1503619459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e980f8cb0 msgr2=0x7f4e980f6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.280+0000 7f4e977fe700 1 --2- 172.21.15.47:0/1503619459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e980f8cb0 0x7f4e980f6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:31.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.280+0000 7f4e977fe700 1 -- 172.21.15.47:0/1503619459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e980f9610 msgr2=0x7f4e980f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:31.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.280+0000 7f4e977fe700 1 --2- 172.21.15.47:0/1503619459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e980f9610 0x7f4e980f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:31.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.280+0000 7f4e977fe700 1 -- 172.21.15.47:0/1503619459 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8c005040 con 0x7f4e98102210 2024-02-23T00:58:31.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.280+0000 7f4e967fc700 1 --2- 172.21.15.47:0/1503619459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e980f9610 0x7f4e980f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:58:31.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.280+0000 7f4e96ffd700 1 --2- 172.21.15.47:0/1503619459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e980f8cb0 0x7f4e980f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:58:31.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.280+0000 7f4e977fe700 1 --2- 172.21.15.47:0/1503619459 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4e98102210 0x7f4e980f7660 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 rx=0x7f4e900049a0 tx=0x7f4e90004c50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:58:31.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.280+0000 7f4e7ffff700 10 client.?.objecter ms_handle_connect 0x7f4e98102210 2024-02-23T00:58:31.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.280+0000 7f4e7ffff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:58:31.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.280+0000 7f4e7ffff700 1 -- 172.21.15.47:0/1503619459 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9000e040 con 0x7f4e98102210 2024-02-23T00:58:31.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.280+0000 7f4e7ffff700 1 -- 172.21.15.47:0/1503619459 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f4e90018040 con 0x7f4e98102210 2024-02-23T00:58:31.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.280+0000 7f4e9d8df700 1 -- 172.21.15.47:0/1503619459 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e900097c0 con 0x7f4e98102210 2024-02-23T00:58:31.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.280+0000 7f4e7ffff700 1 -- 172.21.15.47:0/1503619459 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e90013da0 con 0x7f4e98102210 2024-02-23T00:58:31.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.280+0000 7f4e9d8df700 10 client.42455.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:58:31.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.280+0000 7f4e9d8df700 1 -- 172.21.15.47:0/1503619459 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e98116b70 con 0x7f4e98102210 2024-02-23T00:58:31.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.280+0000 7f4e9d8df700 1 librados: init done 2024-02-23T00:58:31.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.281+0000 7f4e9d8df700 1 -- 172.21.15.47:0/1503619459 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e9805fa60 con 0x7f4e98102210 2024-02-23T00:58:31.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.282+0000 7f4e7ffff700 1 -- 172.21.15.47:0/1503619459 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f4e90013420 con 0x7f4e98102210 2024-02-23T00:58:31.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.282+0000 7f4e7ffff700 1 --2- 172.21.15.47:0/1503619459 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4e80062cc0 0x7f4e80065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:31.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.282+0000 7f4e7ffff700 1 -- 172.21.15.47:0/1503619459 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(859..859 src has 806..859) v4 ==== 8663+0+0 (secure 0 0 0) 0x7f4e9007e120 con 0x7f4e98102210 2024-02-23T00:58:31.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.282+0000 7f4e7ffff700 10 client.42455.objecter ms_dispatch 0x7f4e98001550 osd_map(859..859 src has 806..859) v4 2024-02-23T00:58:31.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.282+0000 7f4e7ffff700 3 client.42455.objecter handle_osd_map got epochs [859,859] > 0 2024-02-23T00:58:31.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.282+0000 7f4e7ffff700 3 client.42455.objecter handle_osd_map decoding full epoch 859 2024-02-23T00:58:31.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.282+0000 7f4e96ffd700 1 --2- 172.21.15.47:0/1503619459 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4e80062cc0 0x7f4e80065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:31.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.282+0000 7f4e7ffff700 20 client.42455.objecter dump_active .. 0 homeless 2024-02-23T00:58:31.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.282+0000 7f4e7ffff700 10 client.42455.objecter ms_handle_connect 0x7f4e80062cc0 2024-02-23T00:58:31.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.282+0000 7f4e96ffd700 1 --2- 172.21.15.47:0/1503619459 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4e80062cc0 0x7f4e80065180 secure :-1 s=READY pgs=3259 cs=0 l=1 rev1=1 rx=0x7f4e8c009f20 tx=0x7f4e8c009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:58:31.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.286+0000 7f4e7ffff700 1 -- 172.21.15.47:0/1503619459 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f4e9004da10 con 0x7f4e98102210 2024-02-23T00:58:31.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.458+0000 7f4e9d8df700 1 -- 172.21.15.47:0/1503619459 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f4e98107010 con 0x7f4e98102210 2024-02-23T00:58:31.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.459+0000 7f4e7ffff700 1 -- 172.21.15.47:0/1503619459 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4e90021070 con 0x7f4e98102210 2024-02-23T00:58:31.462 INFO:teuthology.orchestra.run.smithi047.stdout:231928234219 2024-02-23T00:58:31.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.460+0000 7f4e9d8df700 10 librados: watch_flush enter 2024-02-23T00:58:31.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.460+0000 7f4e9d8df700 10 librados: watch_flush exit 2024-02-23T00:58:31.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.460+0000 7f4e9d8df700 20 client.42455.objecter shutdown clearing up homeless session... 2024-02-23T00:58:31.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.460+0000 7f4e9d8df700 10 client.42455.objecter successfully canceled tick 2024-02-23T00:58:31.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.460+0000 7f4e9d8df700 1 -- 172.21.15.47:0/1503619459 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4e80062cc0 msgr2=0x7f4e80065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:31.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.460+0000 7f4e9d8df700 1 --2- 172.21.15.47:0/1503619459 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4e80062cc0 0x7f4e80065180 secure :-1 s=READY pgs=3259 cs=0 l=1 rev1=1 rx=0x7f4e8c009f20 tx=0x7f4e8c009cb0).stop 2024-02-23T00:58:31.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.460+0000 7f4e9d8df700 1 -- 172.21.15.47:0/1503619459 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4e98102210 msgr2=0x7f4e980f7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:31.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.460+0000 7f4e9d8df700 1 --2- 172.21.15.47:0/1503619459 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4e98102210 0x7f4e980f7660 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 rx=0x7f4e900049a0 tx=0x7f4e90004c50).stop 2024-02-23T00:58:31.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.460+0000 7f4e9d8df700 1 -- 172.21.15.47:0/1503619459 shutdown_connections 2024-02-23T00:58:31.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.460+0000 7f4e9d8df700 1 --2- 172.21.15.47:0/1503619459 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f4e80062cc0 0x7f4e80065180 unknown :-1 s=CLOSED pgs=3259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:31.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.460+0000 7f4e9d8df700 1 --2- 172.21.15.47:0/1503619459 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4e98102210 0x7f4e980f7660 unknown :-1 s=CLOSED pgs=2877 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:31.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.460+0000 7f4e9d8df700 1 --2- 172.21.15.47:0/1503619459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e980f8cb0 0x7f4e980f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:31.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.460+0000 7f4e9d8df700 1 --2- 172.21.15.47:0/1503619459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e980f9610 0x7f4e980f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:31.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.460+0000 7f4e9d8df700 1 -- 172.21.15.47:0/1503619459 >> 172.21.15.47:0/1503619459 conn(0x7f4e980f2820 msgr2=0x7f4e980f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:58:31.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.460+0000 7f4e9d8df700 1 -- 172.21.15.47:0/1503619459 shutdown_connections 2024-02-23T00:58:31.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.461+0000 7f4e9d8df700 1 -- 172.21.15.47:0/1503619459 wait complete. 2024-02-23T00:58:31.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:31.461+0000 7f4e9d8df700 1 librados: shutdown 2024-02-23T00:58:31.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:29.925057+0000 osd.0 (osd.0) 370 : cluster [DBG] 18.0 deep-scrub starts 2024-02-23T00:58:31.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:29.926216+0000 osd.0 (osd.0) 371 : cluster [DBG] 18.0 deep-scrub ok 2024-02-23T00:58:31.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:31.204665+0000 osd.5 (osd.5) 434 : cluster [DBG] 5.28 scrub starts 2024-02-23T00:58:31.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:31.205642+0000 osd.5 (osd.5) 435 : cluster [DBG] 5.28 scrub ok 2024-02-23T00:58:31.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:58:31.460046+0000 mon.c (mon.2) 540 : audit [DBG] from='client.? 172.21.15.47:0/1503619459' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-23T00:58:31.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:29.925057+0000 osd.0 (osd.0) 370 : cluster [DBG] 18.0 deep-scrub starts 2024-02-23T00:58:31.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:29.926216+0000 osd.0 (osd.0) 371 : cluster [DBG] 18.0 deep-scrub ok 2024-02-23T00:58:31.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:31.204665+0000 osd.5 (osd.5) 434 : cluster [DBG] 5.28 scrub starts 2024-02-23T00:58:31.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:31.205642+0000 osd.5 (osd.5) 435 : cluster [DBG] 5.28 scrub ok 2024-02-23T00:58:31.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:31 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:58:31.460046+0000 mon.c (mon.2) 540 : audit [DBG] from='client.? 172.21.15.47:0/1503619459' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-23T00:58:31.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:29.925057+0000 osd.0 (osd.0) 370 : cluster [DBG] 18.0 deep-scrub starts 2024-02-23T00:58:31.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:29.926216+0000 osd.0 (osd.0) 371 : cluster [DBG] 18.0 deep-scrub ok 2024-02-23T00:58:31.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:31.204665+0000 osd.5 (osd.5) 434 : cluster [DBG] 5.28 scrub starts 2024-02-23T00:58:31.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:31.205642+0000 osd.5 (osd.5) 435 : cluster [DBG] 5.28 scrub ok 2024-02-23T00:58:31.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:58:31.460046+0000 mon.c (mon.2) 540 : audit [DBG] from='client.? 172.21.15.47:0/1503619459' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-23T00:58:32.169 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234215 got 231928234219 for osd.7 2024-02-23T00:58:32.169 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-02-23T00:58:32.169 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph pg dump --format=json 2024-02-23T00:58:32.453 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:58:32.475 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:31.447073+0000 mgr.y (mgr.14517) 770 : cluster [DBG] pgmap v1354: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 929 B/s rd, 0 op/s 2024-02-23T00:58:32.475 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:31.609115+0000 osd.7 (osd.7) 451 : cluster [DBG] 2.b scrub starts 2024-02-23T00:58:32.475 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:31.612044+0000 osd.7 (osd.7) 452 : cluster [DBG] 2.b scrub ok 2024-02-23T00:58:32.475 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:32.197500+0000 osd.5 (osd.5) 436 : cluster [DBG] 5.16 scrub starts 2024-02-23T00:58:32.475 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:32.200434+0000 osd.5 (osd.5) 437 : cluster [DBG] 5.16 scrub ok 2024-02-23T00:58:32.807 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:31.447073+0000 mgr.y (mgr.14517) 770 : cluster [DBG] pgmap v1354: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 929 B/s rd, 0 op/s 2024-02-23T00:58:32.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:31.609115+0000 osd.7 (osd.7) 451 : cluster [DBG] 2.b scrub starts 2024-02-23T00:58:32.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:31.612044+0000 osd.7 (osd.7) 452 : cluster [DBG] 2.b scrub ok 2024-02-23T00:58:32.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:32.197500+0000 osd.5 (osd.5) 436 : cluster [DBG] 5.16 scrub starts 2024-02-23T00:58:32.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:32 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:32.200434+0000 osd.5 (osd.5) 437 : cluster [DBG] 5.16 scrub ok 2024-02-23T00:58:32.944 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:58:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:58:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:58:32.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:31.447073+0000 mgr.y (mgr.14517) 770 : cluster [DBG] pgmap v1354: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 929 B/s rd, 0 op/s 2024-02-23T00:58:32.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:31.609115+0000 osd.7 (osd.7) 451 : cluster [DBG] 2.b scrub starts 2024-02-23T00:58:32.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:31.612044+0000 osd.7 (osd.7) 452 : cluster [DBG] 2.b scrub ok 2024-02-23T00:58:32.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:32.197500+0000 osd.5 (osd.5) 436 : cluster [DBG] 5.16 scrub starts 2024-02-23T00:58:32.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:32.200434+0000 osd.5 (osd.5) 437 : cluster [DBG] 5.16 scrub ok 2024-02-23T00:58:33.444 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:58:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:58:33.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:32.559812+0000 osd.5 (osd.5) 438 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:58:33.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:32.560048+0000 osd.5 (osd.5) 439 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:58:33.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:32.559812+0000 osd.5 (osd.5) 438 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:58:33.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:32.560048+0000 osd.5 (osd.5) 439 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:58:33.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:32.559812+0000 osd.5 (osd.5) 438 : cluster [DBG] purged_snaps scrub starts 2024-02-23T00:58:33.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:32.560048+0000 osd.5 (osd.5) 439 : cluster [DBG] purged_snaps scrub ok 2024-02-23T00:58:34.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.156+0000 7f15a541c700 1 -- 172.21.15.47:0/1913364013 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f15a00fcbb0 msgr2=0x7f15a00fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:34.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.156+0000 7f15a541c700 1 --2- 172.21.15.47:0/1913364013 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f15a00fcbb0 0x7f15a00fcf90 secure :-1 s=READY pgs=2878 cs=0 l=1 rev1=1 rx=0x7f15900047b0 tx=0x7f1590009cb0).stop 2024-02-23T00:58:34.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.156+0000 7f15a541c700 1 -- 172.21.15.47:0/1913364013 shutdown_connections 2024-02-23T00:58:34.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.156+0000 7f15a541c700 1 --2- 172.21.15.47:0/1913364013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15a00f7400 0x7f15a0103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:34.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.156+0000 7f15a541c700 1 --2- 172.21.15.47:0/1913364013 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f15a00fcbb0 0x7f15a00fcf90 unknown :-1 s=CLOSED pgs=2878 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:34.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.156+0000 7f15a541c700 1 --2- 172.21.15.47:0/1913364013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15a00fd510 0x7f15a00f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:34.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.156+0000 7f15a541c700 1 -- 172.21.15.47:0/1913364013 >> 172.21.15.47:0/1913364013 conn(0x7f15a00f2820 msgr2=0x7f15a00f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:58:34.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.156+0000 7f15a541c700 1 -- 172.21.15.47:0/1913364013 shutdown_connections 2024-02-23T00:58:34.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.156+0000 7f15a541c700 1 -- 172.21.15.47:0/1913364013 wait complete. 2024-02-23T00:58:34.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.156+0000 7f15a541c700 1 librados: starting msgr at 2024-02-23T00:58:34.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.156+0000 7f15a541c700 1 librados: starting objecter 2024-02-23T00:58:34.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.156+0000 7f15a541c700 1 Processor -- start 2024-02-23T00:58:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.157+0000 7f15a541c700 1 -- start start 2024-02-23T00:58:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.157+0000 7f15a541c700 1 librados: setting wanted keys 2024-02-23T00:58:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.157+0000 7f15a541c700 1 librados: calling monclient init 2024-02-23T00:58:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.157+0000 7f15a541c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f15a00f7400 0x7f15a010e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.157+0000 7f15a541c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15a00fcbb0 0x7f15a010e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.157+0000 7f15a541c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15a00fd510 0x7f15a0108230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.157+0000 7f15a541c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a0108720 con 0x7f15a00fcbb0 2024-02-23T00:58:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.157+0000 7f15a541c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a0108890 con 0x7f15a00fd510 2024-02-23T00:58:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.157+0000 7f15a541c700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a0108a50 con 0x7f15a00f7400 2024-02-23T00:58:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.157+0000 7f159e7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15a00fcbb0 0x7f15a010e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.157+0000 7f159e7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15a00fcbb0 0x7f15a010e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50796/0 (socket says 172.21.15.47:50796) 2024-02-23T00:58:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.157+0000 7f159e7fc700 1 -- 172.21.15.47:0/2116591704 learned_addr learned my addr 172.21.15.47:0/2116591704 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:58:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.157+0000 7f159effd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f15a00f7400 0x7f15a010e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.157+0000 7f159f7fe700 1 --2- 172.21.15.47:0/2116591704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15a00fd510 0x7f15a0108230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.157+0000 7f159e7fc700 1 -- 172.21.15.47:0/2116591704 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f15a00f7400 msgr2=0x7f15a010e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.157+0000 7f159e7fc700 1 --2- 172.21.15.47:0/2116591704 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f15a00f7400 0x7f15a010e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.157+0000 7f159e7fc700 1 -- 172.21.15.47:0/2116591704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15a00fd510 msgr2=0x7f15a0108230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.157+0000 7f159e7fc700 1 --2- 172.21.15.47:0/2116591704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15a00fd510 0x7f15a0108230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.157+0000 7f159e7fc700 1 -- 172.21.15.47:0/2116591704 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1594005040 con 0x7f15a00fcbb0 2024-02-23T00:58:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.157+0000 7f159effd700 1 --2- 172.21.15.47:0/2116591704 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f15a00f7400 0x7f15a010e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:58:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.157+0000 7f159f7fe700 1 --2- 172.21.15.47:0/2116591704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15a00fd510 0x7f15a0108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:58:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.157+0000 7f159e7fc700 1 --2- 172.21.15.47:0/2116591704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15a00fcbb0 0x7f15a010e580 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 rx=0x7f1594009400 tx=0x7f1594009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:58:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.157+0000 7f1587fff700 10 client.?.objecter ms_handle_connect 0x7f15a00fcbb0 2024-02-23T00:58:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.157+0000 7f1587fff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:58:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.158+0000 7f1587fff700 1 -- 172.21.15.47:0/2116591704 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1594007a40 con 0x7f15a00fcbb0 2024-02-23T00:58:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.158+0000 7f15a541c700 1 -- 172.21.15.47:0/2116591704 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1590005040 con 0x7f15a00fcbb0 2024-02-23T00:58:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.158+0000 7f1587fff700 1 -- 172.21.15.47:0/2116591704 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f159400f040 con 0x7f15a00fcbb0 2024-02-23T00:58:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.158+0000 7f1587fff700 1 -- 172.21.15.47:0/2116591704 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1594014990 con 0x7f15a00fcbb0 2024-02-23T00:58:34.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.158+0000 7f15a541c700 10 client.32139.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:58:34.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.158+0000 7f15a541c700 1 -- 172.21.15.47:0/2116591704 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a0109090 con 0x7f15a00fcbb0 2024-02-23T00:58:34.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.158+0000 7f15a541c700 1 librados: init done 2024-02-23T00:58:34.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.159+0000 7f15a541c700 1 -- 172.21.15.47:0/2116591704 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a005fa60 con 0x7f15a00fcbb0 2024-02-23T00:58:34.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.159+0000 7f1587fff700 1 -- 172.21.15.47:0/2116591704 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f159400e630 con 0x7f15a00fcbb0 2024-02-23T00:58:34.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.160+0000 7f1587fff700 1 --2- 172.21.15.47:0/2116591704 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1588062cc0 0x7f1588065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:34.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.160+0000 7f1587fff700 1 -- 172.21.15.47:0/2116591704 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(860..860 src has 806..860) v4 ==== 8663+0+0 (secure 0 0 0) 0x7f159407d130 con 0x7f15a00fcbb0 2024-02-23T00:58:34.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.160+0000 7f1587fff700 10 client.32139.objecter ms_dispatch 0x7f15a0001550 osd_map(860..860 src has 806..860) v4 2024-02-23T00:58:34.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.160+0000 7f1587fff700 3 client.32139.objecter handle_osd_map got epochs [860,860] > 0 2024-02-23T00:58:34.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.160+0000 7f1587fff700 3 client.32139.objecter handle_osd_map decoding full epoch 860 2024-02-23T00:58:34.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.160+0000 7f159effd700 1 --2- 172.21.15.47:0/2116591704 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1588062cc0 0x7f1588065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:34.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.160+0000 7f1587fff700 20 client.32139.objecter dump_active .. 0 homeless 2024-02-23T00:58:34.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.160+0000 7f1587fff700 10 client.32139.objecter ms_handle_connect 0x7f1588062cc0 2024-02-23T00:58:34.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.160+0000 7f159effd700 1 --2- 172.21.15.47:0/2116591704 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1588062cc0 0x7f1588065180 secure :-1 s=READY pgs=3260 cs=0 l=1 rev1=1 rx=0x7f1590004e20 tx=0x7f1590009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:58:34.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.164+0000 7f1587fff700 1 -- 172.21.15.47:0/2116591704 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f1594049f60 con 0x7f15a00fcbb0 2024-02-23T00:58:34.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.331+0000 7f15a541c700 1 -- 172.21.15.47:0/2116591704 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f15a0109710 con 0x7f1588062cc0 2024-02-23T00:58:34.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.343+0000 7f1587fff700 1 -- 172.21.15.47:0/2116591704 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+410081 (secure 0 0 0) 0x7f15a0109710 con 0x7f1588062cc0 2024-02-23T00:58:34.352 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:58:34.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.346+0000 7f15a541c700 10 librados: watch_flush enter 2024-02-23T00:58:34.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.346+0000 7f15a541c700 10 librados: watch_flush exit 2024-02-23T00:58:34.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.346+0000 7f15a541c700 20 client.32139.objecter shutdown clearing up homeless session... 2024-02-23T00:58:34.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.346+0000 7f15a541c700 10 client.32139.objecter successfully canceled tick 2024-02-23T00:58:34.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.346+0000 7f15a541c700 1 -- 172.21.15.47:0/2116591704 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1588062cc0 msgr2=0x7f1588065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:34.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.346+0000 7f15a541c700 1 --2- 172.21.15.47:0/2116591704 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1588062cc0 0x7f1588065180 secure :-1 s=READY pgs=3260 cs=0 l=1 rev1=1 rx=0x7f1590004e20 tx=0x7f1590009cb0).stop 2024-02-23T00:58:34.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.346+0000 7f15a541c700 1 -- 172.21.15.47:0/2116591704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15a00fcbb0 msgr2=0x7f15a010e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:34.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.346+0000 7f15a541c700 1 --2- 172.21.15.47:0/2116591704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15a00fcbb0 0x7f15a010e580 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 rx=0x7f1594009400 tx=0x7f1594009cb0).stop 2024-02-23T00:58:34.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.346+0000 7f15a541c700 1 -- 172.21.15.47:0/2116591704 shutdown_connections 2024-02-23T00:58:34.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.346+0000 7f15a541c700 1 --2- 172.21.15.47:0/2116591704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15a00fd510 0x7f15a0108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:34.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.346+0000 7f15a541c700 1 --2- 172.21.15.47:0/2116591704 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f1588062cc0 0x7f1588065180 unknown :-1 s=CLOSED pgs=3260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:34.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.346+0000 7f15a541c700 1 --2- 172.21.15.47:0/2116591704 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f15a00f7400 0x7f15a010e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:34.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.346+0000 7f15a541c700 1 --2- 172.21.15.47:0/2116591704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15a00fcbb0 0x7f15a010e580 unknown :-1 s=CLOSED pgs=2691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:34.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.346+0000 7f15a541c700 1 -- 172.21.15.47:0/2116591704 >> 172.21.15.47:0/2116591704 conn(0x7f15a00f2820 msgr2=0x7f15a0060eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:58:34.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.346+0000 7f15a541c700 1 -- 172.21.15.47:0/2116591704 shutdown_connections 2024-02-23T00:58:34.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.346+0000 7f15a541c700 1 -- 172.21.15.47:0/2116591704 wait complete. 2024-02-23T00:58:34.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:34.346+0000 7f15a541c700 1 librados: shutdown 2024-02-23T00:58:34.357 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-02-23T00:58:34.696 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:58:33.124094+0000 mgr.y (mgr.14517) 771 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:58:34.697 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:33.447874+0000 mgr.y (mgr.14517) 772 : cluster [DBG] pgmap v1355: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:58:34.697 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:33.478647+0000 mon.a (mon.0) 3951 : cluster [DBG] osdmap e860: 8 total, 8 up, 8 in 2024-02-23T00:58:34.697 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:58:33.124094+0000 mgr.y (mgr.14517) 771 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:58:34.697 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:33.447874+0000 mgr.y (mgr.14517) 772 : cluster [DBG] pgmap v1355: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:58:34.697 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:34 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:33.478647+0000 mon.a (mon.0) 3951 : cluster [DBG] osdmap e860: 8 total, 8 up, 8 in 2024-02-23T00:58:34.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:58:33.124094+0000 mgr.y (mgr.14517) 771 : audit [DBG] from='client.24389 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-23T00:58:34.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:33.447874+0000 mgr.y (mgr.14517) 772 : cluster [DBG] pgmap v1355: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-23T00:58:34.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:33.478647+0000 mon.a (mon.0) 3951 : cluster [DBG] osdmap e860: 8 total, 8 up, 8 in 2024-02-23T00:58:35.148 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":1356,"stamp":"2024-02-23T00:58:33.475562+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5801,"num_objects":225,"num_object_clones":0,"num_object_copies":675,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":225,"num_whiteouts":0,"num_read":2603,"num_read_kb":2323,"num_write":913,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":458,"num_bytes_recovered":13770,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1167,"ondisk_log_size":1167,"up":462,"acting":462,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":462,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2440120,"kb_used_data":21368,"kb_used_omap":0,"kb_used_meta":2418688,"kb_avail":747521096,"statfs":{"total":767960285184,"available":765461602304,"internally_reserved":0,"allocated":21880832,"data_stored":8633315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2476736512},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.030709"},"pg_stats":[{"pgid":"5.29","version":"466'9","reported_seq":128,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.483147+0000","last_change":"2024-02-23T00:57:28.058623+0000","last_active":"2024-02-23T00:57:51.483147+0000","last_peered":"2024-02-23T00:57:51.483147+0000","last_clean":"2024-02-23T00:57:51.483147+0000","last_became_active":"2024-02-23T00:56:22.399258+0000","last_became_peered":"2024-02-23T00:56:22.399258+0000","last_unstale":"2024-02-23T00:57:51.483147+0000","last_undegraded":"2024-02-23T00:57:51.483147+0000","last_fullsized":"2024-02-23T00:57:51.483147+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":769,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":6,"last_scrub":"466'9","last_scrub_stamp":"2024-02-23T00:57:28.058561+0000","last_deep_scrub":"466'9","last_deep_scrub_stamp":"2024-02-23T00:57:28.058561+0000","last_clean_scrub_stamp":"2024-02-23T00:57:28.058561+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":148,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:31.205726+0000","last_change":"2024-02-23T00:58:31.205726+0000","last_active":"2024-02-23T00:58:31.205726+0000","last_peered":"2024-02-23T00:58:31.205726+0000","last_clean":"2024-02-23T00:58:31.205726+0000","last_became_active":"2024-02-23T00:56:08.352060+0000","last_became_peered":"2024-02-23T00:56:08.352060+0000","last_unstale":"2024-02-23T00:58:31.205726+0000","last_undegraded":"2024-02-23T00:58:31.205726+0000","last_fullsized":"2024-02-23T00:58:31.205726+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":769,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:31.205647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_clean_scrub_stamp":"2024-02-23T00:58:31.205647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":841,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.479802+0000","last_change":"2024-02-23T00:57:15.839233+0000","last_active":"2024-02-23T00:57:51.479802+0000","last_peered":"2024-02-23T00:57:51.479802+0000","last_clean":"2024-02-23T00:57:51.479802+0000","last_became_active":"2024-02-23T00:56:59.507959+0000","last_became_peered":"2024-02-23T00:56:59.507959+0000","last_unstale":"2024-02-23T00:57:51.479802+0000","last_undegraded":"2024-02-23T00:57:51.479802+0000","last_fullsized":"2024-02-23T00:57:51.479802+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":769,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:15.839151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:59.694033+0000","last_clean_scrub_stamp":"2024-02-23T00:57:15.839151+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":81,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.480737+0000","last_change":"2024-02-23T00:57:28.009719+0000","last_active":"2024-02-23T00:57:51.480737+0000","last_peered":"2024-02-23T00:57:51.480737+0000","last_clean":"2024-02-23T00:57:51.480737+0000","last_became_active":"2024-02-23T00:57:13.547938+0000","last_became_peered":"2024-02-23T00:57:13.547938+0000","last_unstale":"2024-02-23T00:57:51.480737+0000","last_undegraded":"2024-02-23T00:57:51.480737+0000","last_fullsized":"2024-02-23T00:57:51.480737+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":769,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:28.009637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.264240+0000","last_clean_scrub_stamp":"2024-02-23T00:57:28.009637+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":172,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.478908+0000","last_change":"2024-02-23T00:57:17.820201+0000","last_active":"2024-02-23T00:57:51.478908+0000","last_peered":"2024-02-23T00:57:51.478908+0000","last_clean":"2024-02-23T00:57:51.478908+0000","last_became_active":"2024-02-23T00:56:59.506206+0000","last_became_peered":"2024-02-23T00:56:59.506206+0000","last_unstale":"2024-02-23T00:57:51.478908+0000","last_undegraded":"2024-02-23T00:57:51.478908+0000","last_fullsized":"2024-02-23T00:57:51.478908+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":769,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:17.820129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:57:17.820129+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":873,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:10.261298+0000","last_change":"2024-02-23T00:58:10.261254+0000","last_active":"2024-02-23T00:58:10.261298+0000","last_peered":"2024-02-23T00:58:10.261298+0000","last_clean":"2024-02-23T00:58:10.261298+0000","last_became_active":"2024-02-23T00:55:21.382086+0000","last_became_peered":"2024-02-23T00:55:21.382086+0000","last_unstale":"2024-02-23T00:58:10.261298+0000","last_undegraded":"2024-02-23T00:58:10.261298+0000","last_fullsized":"2024-02-23T00:58:10.261298+0000","mapping_epoch":771,"log_start":"0'0","ondisk_log_start":"0'0","created":769,"last_epoch_clean":772,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:10.261117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:58:10.261117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"813'14","reported_seq":163,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:23.058780+0000","last_change":"2024-02-23T00:58:05.459592+0000","last_active":"2024-02-23T00:58:23.058780+0000","last_peered":"2024-02-23T00:58:23.058780+0000","last_clean":"2024-02-23T00:58:23.058780+0000","last_became_active":"2024-02-23T00:56:08.337720+0000","last_became_peered":"2024-02-23T00:56:08.337720+0000","last_unstale":"2024-02-23T00:58:23.058780+0000","last_undegraded":"2024-02-23T00:58:23.058780+0000","last_fullsized":"2024-02-23T00:58:23.058780+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":769,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":6,"last_scrub":"813'14","last_scrub_stamp":"2024-02-23T00:58:05.459517+0000","last_deep_scrub":"91'9","last_deep_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_clean_scrub_stamp":"2024-02-23T00:58:05.459517+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":62,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:03.254309+0000","last_change":"2024-02-23T00:58:03.254267+0000","last_active":"2024-02-23T00:58:03.254309+0000","last_peered":"2024-02-23T00:58:03.254309+0000","last_clean":"2024-02-23T00:58:03.254309+0000","last_became_active":"2024-02-23T00:56:22.393231+0000","last_became_peered":"2024-02-23T00:56:22.393231+0000","last_unstale":"2024-02-23T00:58:03.254309+0000","last_undegraded":"2024-02-23T00:58:03.254309+0000","last_fullsized":"2024-02-23T00:58:03.254309+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":769,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:03.254156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:58:03.254156+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":110,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.485209+0000","last_change":"2024-02-23T00:57:09.751484+0000","last_active":"2024-02-23T00:57:51.485209+0000","last_peered":"2024-02-23T00:57:51.485209+0000","last_clean":"2024-02-23T00:57:51.485209+0000","last_became_active":"2024-02-23T00:56:08.364399+0000","last_became_peered":"2024-02-23T00:56:08.364399+0000","last_unstale":"2024-02-23T00:57:51.485209+0000","last_undegraded":"2024-02-23T00:57:51.485209+0000","last_fullsized":"2024-02-23T00:57:51.485209+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":769,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:09.751392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:10.493340+0000","last_clean_scrub_stamp":"2024-02-23T00:57:09.751392+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"818'15","reported_seq":90,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:23.058757+0000","last_change":"2024-02-23T00:58:12.314312+0000","last_active":"2024-02-23T00:58:23.058757+0000","last_peered":"2024-02-23T00:58:23.058757+0000","last_clean":"2024-02-23T00:58:23.058757+0000","last_became_active":"2024-02-23T00:56:22.403485+0000","last_became_peered":"2024-02-23T00:56:22.403485+0000","last_unstale":"2024-02-23T00:58:23.058757+0000","last_undegraded":"2024-02-23T00:58:23.058757+0000","last_fullsized":"2024-02-23T00:58:23.058757+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":769,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":6,"last_scrub":"818'15","last_scrub_stamp":"2024-02-23T00:58:12.314229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:58:12.314229+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"818'12","reported_seq":90,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:23.058769+0000","last_change":"2024-02-23T00:58:11.291419+0000","last_active":"2024-02-23T00:58:23.058769+0000","last_peered":"2024-02-23T00:58:23.058769+0000","last_clean":"2024-02-23T00:58:23.058769+0000","last_became_active":"2024-02-23T00:56:22.389524+0000","last_became_peered":"2024-02-23T00:56:22.389524+0000","last_unstale":"2024-02-23T00:58:23.058769+0000","last_undegraded":"2024-02-23T00:58:23.058769+0000","last_fullsized":"2024-02-23T00:58:23.058769+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"818'12","last_scrub_stamp":"2024-02-23T00:58:11.291344+0000","last_deep_scrub":"818'12","last_deep_scrub_stamp":"2024-02-23T00:56:49.953643+0000","last_clean_scrub_stamp":"2024-02-23T00:58:11.291344+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"779'48","reported_seq":977,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.484618+0000","last_change":"2024-02-23T00:57:14.209043+0000","last_active":"2024-02-23T00:57:51.484618+0000","last_peered":"2024-02-23T00:57:51.484618+0000","last_clean":"2024-02-23T00:57:51.484618+0000","last_became_active":"2024-02-23T00:56:22.394274+0000","last_became_peered":"2024-02-23T00:56:22.394274+0000","last_unstale":"2024-02-23T00:57:51.484618+0000","last_undegraded":"2024-02-23T00:57:51.484618+0000","last_fullsized":"2024-02-23T00:57:51.484618+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"779'48","last_scrub_stamp":"2024-02-23T00:57:14.208974+0000","last_deep_scrub":"779'48","last_deep_scrub_stamp":"2024-02-23T00:55:31.242584+0000","last_clean_scrub_stamp":"2024-02-23T00:57:14.208974+0000","log_size":48,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":75,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.485197+0000","last_change":"2024-02-23T00:57:47.082744+0000","last_active":"2024-02-23T00:57:51.485197+0000","last_peered":"2024-02-23T00:57:51.485197+0000","last_clean":"2024-02-23T00:57:51.485197+0000","last_became_active":"2024-02-23T00:56:22.396348+0000","last_became_peered":"2024-02-23T00:56:22.396348+0000","last_unstale":"2024-02-23T00:57:51.485197+0000","last_undegraded":"2024-02-23T00:57:51.485197+0000","last_fullsized":"2024-02-23T00:57:51.485197+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:47.082679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:04.330426+0000","last_clean_scrub_stamp":"2024-02-23T00:57:47.082679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":869,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.485403+0000","last_change":"2024-02-23T00:57:21.257122+0000","last_active":"2024-02-23T00:57:51.485403+0000","last_peered":"2024-02-23T00:57:51.485403+0000","last_clean":"2024-02-23T00:57:51.485403+0000","last_became_active":"2024-02-23T00:54:56.482910+0000","last_became_peered":"2024-02-23T00:54:56.482910+0000","last_unstale":"2024-02-23T00:57:51.485403+0000","last_undegraded":"2024-02-23T00:57:51.485403+0000","last_fullsized":"2024-02-23T00:57:51.485403+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:21.257053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:57:21.257053+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":798,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.485609+0000","last_change":"2024-02-23T00:57:43.167765+0000","last_active":"2024-02-23T00:57:51.485609+0000","last_peered":"2024-02-23T00:57:51.485609+0000","last_clean":"2024-02-23T00:57:51.485609+0000","last_became_active":"2024-02-23T00:54:56.486869+0000","last_became_peered":"2024-02-23T00:54:56.486869+0000","last_unstale":"2024-02-23T00:57:51.485609+0000","last_undegraded":"2024-02-23T00:57:51.485609+0000","last_fullsized":"2024-02-23T00:57:51.485609+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:43.167650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:57:43.167650+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"846'26","reported_seq":147,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.484642+0000","last_change":"2024-02-23T00:57:21.095323+0000","last_active":"2024-02-23T00:57:51.484642+0000","last_peered":"2024-02-23T00:57:51.484642+0000","last_clean":"2024-02-23T00:57:51.484642+0000","last_became_active":"2024-02-23T00:54:56.481365+0000","last_became_peered":"2024-02-23T00:54:56.481365+0000","last_unstale":"2024-02-23T00:57:51.484642+0000","last_undegraded":"2024-02-23T00:57:51.484642+0000","last_fullsized":"2024-02-23T00:57:51.484642+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"846'26","last_scrub_stamp":"2024-02-23T00:57:21.095257+0000","last_deep_scrub":"846'26","last_deep_scrub_stamp":"2024-02-23T00:57:21.095257+0000","last_clean_scrub_stamp":"2024-02-23T00:57:21.095257+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":37,"num_read_kb":30,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":793,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.479318+0000","last_change":"2024-02-23T00:57:25.845138+0000","last_active":"2024-02-23T00:57:51.479318+0000","last_peered":"2024-02-23T00:57:51.479318+0000","last_clean":"2024-02-23T00:57:51.479318+0000","last_became_active":"2024-02-23T00:56:59.501902+0000","last_became_peered":"2024-02-23T00:56:59.501902+0000","last_unstale":"2024-02-23T00:57:51.479318+0000","last_undegraded":"2024-02-23T00:57:51.479318+0000","last_fullsized":"2024-02-23T00:57:51.479318+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:25.845070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:04.528006+0000","last_clean_scrub_stamp":"2024-02-23T00:57:25.845070+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":66,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:56.141048+0000","last_change":"2024-02-23T00:57:56.140991+0000","last_active":"2024-02-23T00:57:56.141048+0000","last_peered":"2024-02-23T00:57:56.141048+0000","last_clean":"2024-02-23T00:57:56.141048+0000","last_became_active":"2024-02-23T00:56:08.362289+0000","last_became_peered":"2024-02-23T00:56:08.362289+0000","last_unstale":"2024-02-23T00:57:56.141048+0000","last_undegraded":"2024-02-23T00:57:56.141048+0000","last_fullsized":"2024-02-23T00:57:56.141048+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:56.140899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:56:50.919809+0000","last_clean_scrub_stamp":"2024-02-23T00:57:56.140899+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":157,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:57.269526+0000","last_change":"2024-02-23T00:57:57.269495+0000","last_active":"2024-02-23T00:57:57.269526+0000","last_peered":"2024-02-23T00:57:57.269526+0000","last_clean":"2024-02-23T00:57:57.269526+0000","last_became_active":"2024-02-23T00:56:22.398741+0000","last_became_peered":"2024-02-23T00:56:22.398741+0000","last_unstale":"2024-02-23T00:57:57.269526+0000","last_undegraded":"2024-02-23T00:57:57.269526+0000","last_fullsized":"2024-02-23T00:57:57.269526+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:57.269424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:13.867716+0000","last_clean_scrub_stamp":"2024-02-23T00:57:57.269424+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":148,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:52.210699+0000","last_change":"2024-02-23T00:57:52.210660+0000","last_active":"2024-02-23T00:57:52.210699+0000","last_peered":"2024-02-23T00:57:52.210699+0000","last_clean":"2024-02-23T00:57:52.210699+0000","last_became_active":"2024-02-23T00:56:22.386236+0000","last_became_peered":"2024-02-23T00:56:22.386236+0000","last_unstale":"2024-02-23T00:57:52.210699+0000","last_undegraded":"2024-02-23T00:57:52.210699+0000","last_fullsized":"2024-02-23T00:57:52.210699+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:52.210576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:57:52.210576+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"457'51","reported_seq":986,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.567022+0000","last_change":"2024-02-23T00:58:20.567022+0000","last_active":"2024-02-23T00:58:20.567022+0000","last_peered":"2024-02-23T00:58:20.567022+0000","last_clean":"2024-02-23T00:58:20.567022+0000","last_became_active":"2024-02-23T00:56:22.391880+0000","last_became_peered":"2024-02-23T00:56:22.391880+0000","last_unstale":"2024-02-23T00:58:20.567022+0000","last_undegraded":"2024-02-23T00:58:20.567022+0000","last_fullsized":"2024-02-23T00:58:20.567022+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'51","last_scrub_stamp":"2024-02-23T00:58:20.566957+0000","last_deep_scrub":"88'35","last_deep_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_clean_scrub_stamp":"2024-02-23T00:58:20.566957+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":68,"num_read_kb":56,"num_write":42,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":66,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.485379+0000","last_change":"2024-02-23T00:57:36.017206+0000","last_active":"2024-02-23T00:57:51.485379+0000","last_peered":"2024-02-23T00:57:51.485379+0000","last_clean":"2024-02-23T00:57:51.485379+0000","last_became_active":"2024-02-23T00:56:08.351610+0000","last_became_peered":"2024-02-23T00:56:08.351610+0000","last_unstale":"2024-02-23T00:57:51.485379+0000","last_undegraded":"2024-02-23T00:57:51.485379+0000","last_fullsized":"2024-02-23T00:57:51.485379+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:36.017116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_clean_scrub_stamp":"2024-02-23T00:57:36.017116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":66,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:57.128552+0000","last_change":"2024-02-23T00:57:57.128526+0000","last_active":"2024-02-23T00:57:57.128552+0000","last_peered":"2024-02-23T00:57:57.128552+0000","last_clean":"2024-02-23T00:57:57.128552+0000","last_became_active":"2024-02-23T00:56:08.351215+0000","last_became_peered":"2024-02-23T00:56:08.351215+0000","last_unstale":"2024-02-23T00:57:57.128552+0000","last_undegraded":"2024-02-23T00:57:57.128552+0000","last_fullsized":"2024-02-23T00:57:57.128552+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:57.128460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:56:31.214495+0000","last_clean_scrub_stamp":"2024-02-23T00:57:57.128460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":60,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:58.222519+0000","last_change":"2024-02-23T00:57:58.222488+0000","last_active":"2024-02-23T00:57:58.222519+0000","last_peered":"2024-02-23T00:57:58.222519+0000","last_clean":"2024-02-23T00:57:58.222519+0000","last_became_active":"2024-02-23T00:56:22.404580+0000","last_became_peered":"2024-02-23T00:56:22.404580+0000","last_unstale":"2024-02-23T00:57:58.222519+0000","last_undegraded":"2024-02-23T00:57:58.222519+0000","last_fullsized":"2024-02-23T00:57:58.222519+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:58.222416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:57:58.222416+0000","last_clean_scrub_stamp":"2024-02-23T00:57:58.222416+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":788,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:09.804877+0000","last_change":"2024-02-23T00:58:09.804852+0000","last_active":"2024-02-23T00:58:09.804877+0000","last_peered":"2024-02-23T00:58:09.804877+0000","last_clean":"2024-02-23T00:58:09.804877+0000","last_became_active":"2024-02-23T00:56:59.509516+0000","last_became_peered":"2024-02-23T00:56:59.509516+0000","last_unstale":"2024-02-23T00:58:09.804877+0000","last_undegraded":"2024-02-23T00:58:09.804877+0000","last_fullsized":"2024-02-23T00:58:09.804877+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:09.804777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:53:07.482987+0000","last_clean_scrub_stamp":"2024-02-23T00:58:09.804777+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"853'39","reported_seq":976,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.483620+0000","last_change":"2024-02-23T00:57:37.326961+0000","last_active":"2024-02-23T00:57:51.483620+0000","last_peered":"2024-02-23T00:57:51.483620+0000","last_clean":"2024-02-23T00:57:51.483620+0000","last_became_active":"2024-02-23T00:56:08.363684+0000","last_became_peered":"2024-02-23T00:56:08.363684+0000","last_unstale":"2024-02-23T00:57:51.483620+0000","last_undegraded":"2024-02-23T00:57:51.483620+0000","last_fullsized":"2024-02-23T00:57:51.483620+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"853'39","last_scrub_stamp":"2024-02-23T00:57:37.326874+0000","last_deep_scrub":"853'39","last_deep_scrub_stamp":"2024-02-23T00:57:37.326874+0000","last_clean_scrub_stamp":"2024-02-23T00:57:37.326874+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":60,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:31.612130+0000","last_change":"2024-02-23T00:58:31.612130+0000","last_active":"2024-02-23T00:58:31.612130+0000","last_peered":"2024-02-23T00:58:31.612130+0000","last_clean":"2024-02-23T00:58:31.612130+0000","last_became_active":"2024-02-23T00:56:22.393096+0000","last_became_peered":"2024-02-23T00:56:22.393096+0000","last_unstale":"2024-02-23T00:58:31.612130+0000","last_undegraded":"2024-02-23T00:58:31.612130+0000","last_fullsized":"2024-02-23T00:58:31.612130+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:31.612049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:58:31.612049+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":812,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:14.778455+0000","last_change":"2024-02-23T00:58:14.778413+0000","last_active":"2024-02-23T00:58:14.778455+0000","last_peered":"2024-02-23T00:58:14.778455+0000","last_clean":"2024-02-23T00:58:14.778455+0000","last_became_active":"2024-02-23T00:56:59.504894+0000","last_became_peered":"2024-02-23T00:56:59.504894+0000","last_unstale":"2024-02-23T00:58:14.778455+0000","last_undegraded":"2024-02-23T00:58:14.778455+0000","last_fullsized":"2024-02-23T00:58:14.778455+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:14.778293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:58:14.778293+0000","last_clean_scrub_stamp":"2024-02-23T00:58:14.778293+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":735,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.478922+0000","last_change":"2024-02-23T00:57:20.875046+0000","last_active":"2024-02-23T00:57:51.478922+0000","last_peered":"2024-02-23T00:57:51.478922+0000","last_clean":"2024-02-23T00:57:51.478922+0000","last_became_active":"2024-02-23T00:56:59.502268+0000","last_became_peered":"2024-02-23T00:56:59.502268+0000","last_unstale":"2024-02-23T00:57:51.478922+0000","last_undegraded":"2024-02-23T00:57:51.478922+0000","last_fullsized":"2024-02-23T00:57:51.478922+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:20.874942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:57:20.874942+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"457'20","reported_seq":873,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:25.342628+0000","last_change":"2024-02-23T00:58:25.342628+0000","last_active":"2024-02-23T00:58:25.342628+0000","last_peered":"2024-02-23T00:58:25.342628+0000","last_clean":"2024-02-23T00:58:25.342628+0000","last_became_active":"2024-02-23T00:39:16.777530+0000","last_became_peered":"2024-02-23T00:39:16.777530+0000","last_unstale":"2024-02-23T00:58:25.342628+0000","last_undegraded":"2024-02-23T00:58:25.342628+0000","last_fullsized":"2024-02-23T00:58:25.342628+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'20","last_scrub_stamp":"2024-02-23T00:58:25.342546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:58:25.342546+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":802,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.480394+0000","last_change":"2024-02-23T00:57:31.932103+0000","last_active":"2024-02-23T00:57:51.480394+0000","last_peered":"2024-02-23T00:57:51.480394+0000","last_clean":"2024-02-23T00:57:51.480394+0000","last_became_active":"2024-02-23T00:56:59.508854+0000","last_became_peered":"2024-02-23T00:56:59.508854+0000","last_unstale":"2024-02-23T00:57:51.480394+0000","last_undegraded":"2024-02-23T00:57:51.480394+0000","last_fullsized":"2024-02-23T00:57:51.480394+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:31.932040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:55:58.224758+0000","last_clean_scrub_stamp":"2024-02-23T00:57:31.932040+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"818'14","reported_seq":112,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:23.058591+0000","last_change":"2024-02-23T00:58:05.181610+0000","last_active":"2024-02-23T00:58:23.058591+0000","last_peered":"2024-02-23T00:58:23.058591+0000","last_clean":"2024-02-23T00:58:23.058591+0000","last_became_active":"2024-02-23T00:56:22.402927+0000","last_became_peered":"2024-02-23T00:56:22.402927+0000","last_unstale":"2024-02-23T00:58:23.058591+0000","last_undegraded":"2024-02-23T00:58:23.058591+0000","last_fullsized":"2024-02-23T00:58:23.058591+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"818'14","last_scrub_stamp":"2024-02-23T00:58:05.181539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:58:05.181539+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":196,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:06.660973+0000","last_change":"2024-02-23T00:58:06.660945+0000","last_active":"2024-02-23T00:58:06.660973+0000","last_peered":"2024-02-23T00:58:06.660973+0000","last_clean":"2024-02-23T00:58:06.660973+0000","last_became_active":"2024-02-23T00:57:13.552728+0000","last_became_peered":"2024-02-23T00:57:13.552728+0000","last_unstale":"2024-02-23T00:58:06.660973+0000","last_undegraded":"2024-02-23T00:58:06.660973+0000","last_fullsized":"2024-02-23T00:58:06.660973+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:06.660878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.335752+0000","last_clean_scrub_stamp":"2024-02-23T00:58:06.660878+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"457'27","reported_seq":844,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:27.662825+0000","last_change":"2024-02-23T00:58:27.662825+0000","last_active":"2024-02-23T00:58:27.662825+0000","last_peered":"2024-02-23T00:58:27.662825+0000","last_clean":"2024-02-23T00:58:27.662825+0000","last_became_active":"2024-02-23T00:56:59.502553+0000","last_became_peered":"2024-02-23T00:56:59.502553+0000","last_unstale":"2024-02-23T00:58:27.662825+0000","last_undegraded":"2024-02-23T00:58:27.662825+0000","last_fullsized":"2024-02-23T00:58:27.662825+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-23T00:58:27.662752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:58:27.662752+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":807,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.480207+0000","last_change":"2024-02-23T00:57:47.822981+0000","last_active":"2024-02-23T00:57:51.480207+0000","last_peered":"2024-02-23T00:57:51.480207+0000","last_clean":"2024-02-23T00:57:51.480207+0000","last_became_active":"2024-02-23T00:56:59.508189+0000","last_became_peered":"2024-02-23T00:56:59.508189+0000","last_unstale":"2024-02-23T00:57:51.480207+0000","last_undegraded":"2024-02-23T00:57:51.480207+0000","last_fullsized":"2024-02-23T00:57:51.480207+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:47.822907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:57:47.822907+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"749'12","reported_seq":1132,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:23.058800+0000","last_change":"2024-02-23T00:57:45.403148+0000","last_active":"2024-02-23T00:58:23.058800+0000","last_peered":"2024-02-23T00:58:23.058800+0000","last_clean":"2024-02-23T00:58:23.058800+0000","last_became_active":"2024-02-23T00:54:56.484042+0000","last_became_peered":"2024-02-23T00:54:56.484042+0000","last_unstale":"2024-02-23T00:58:23.058800+0000","last_undegraded":"2024-02-23T00:58:23.058800+0000","last_fullsized":"2024-02-23T00:58:23.058800+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"749'12","last_scrub_stamp":"2024-02-23T00:57:45.403074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:57:45.403074+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":820,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.482416+0000","last_change":"2024-02-23T00:57:06.488551+0000","last_active":"2024-02-23T00:57:51.482416+0000","last_peered":"2024-02-23T00:57:51.482416+0000","last_clean":"2024-02-23T00:57:51.482416+0000","last_became_active":"2024-02-23T00:56:22.391505+0000","last_became_peered":"2024-02-23T00:56:22.391505+0000","last_unstale":"2024-02-23T00:57:51.482416+0000","last_undegraded":"2024-02-23T00:57:51.482416+0000","last_fullsized":"2024-02-23T00:57:51.482416+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:06.488486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:57:06.488486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"457'31","reported_seq":936,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:01.536896+0000","last_change":"2024-02-23T00:58:01.536858+0000","last_active":"2024-02-23T00:58:01.536896+0000","last_peered":"2024-02-23T00:58:01.536896+0000","last_clean":"2024-02-23T00:58:01.536896+0000","last_became_active":"2024-02-23T00:47:28.201225+0000","last_became_peered":"2024-02-23T00:47:28.201225+0000","last_unstale":"2024-02-23T00:58:01.536896+0000","last_undegraded":"2024-02-23T00:58:01.536896+0000","last_fullsized":"2024-02-23T00:58:01.536896+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:58:01.536782+0000","last_deep_scrub":"336'23","last_deep_scrub_stamp":"2024-02-23T00:46:48.320085+0000","last_clean_scrub_stamp":"2024-02-23T00:58:01.536782+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":60,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.486767+0000","last_change":"2024-02-23T00:57:44.350468+0000","last_active":"2024-02-23T00:57:51.486767+0000","last_peered":"2024-02-23T00:57:51.486767+0000","last_clean":"2024-02-23T00:57:51.486767+0000","last_became_active":"2024-02-23T00:56:22.403948+0000","last_became_peered":"2024-02-23T00:56:22.403948+0000","last_unstale":"2024-02-23T00:57:51.486767+0000","last_undegraded":"2024-02-23T00:57:51.486767+0000","last_fullsized":"2024-02-23T00:57:51.486767+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:44.350386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:56:42.009186+0000","last_clean_scrub_stamp":"2024-02-23T00:57:44.350386+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":835,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:53.190726+0000","last_change":"2024-02-23T00:57:53.190680+0000","last_active":"2024-02-23T00:57:53.190726+0000","last_peered":"2024-02-23T00:57:53.190726+0000","last_clean":"2024-02-23T00:57:53.190726+0000","last_became_active":"2024-02-23T00:39:18.800777+0000","last_became_peered":"2024-02-23T00:39:18.800777+0000","last_unstale":"2024-02-23T00:57:53.190726+0000","last_undegraded":"2024-02-23T00:57:53.190726+0000","last_fullsized":"2024-02-23T00:57:53.190726+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:53.190547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:57:53.190547+0000","last_clean_scrub_stamp":"2024-02-23T00:57:53.190547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":753,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:53.850304+0000","last_change":"2024-02-23T00:57:53.850253+0000","last_active":"2024-02-23T00:57:53.850304+0000","last_peered":"2024-02-23T00:57:53.850304+0000","last_clean":"2024-02-23T00:57:53.850304+0000","last_became_active":"2024-02-23T00:56:59.508199+0000","last_became_peered":"2024-02-23T00:56:59.508199+0000","last_unstale":"2024-02-23T00:57:53.850304+0000","last_undegraded":"2024-02-23T00:57:53.850304+0000","last_fullsized":"2024-02-23T00:57:53.850304+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:53.850171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:56:29.924704+0000","last_clean_scrub_stamp":"2024-02-23T00:57:53.850171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"694'10","reported_seq":104,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:29.849960+0000","last_change":"2024-02-23T00:58:29.849960+0000","last_active":"2024-02-23T00:58:29.849960+0000","last_peered":"2024-02-23T00:58:29.849960+0000","last_clean":"2024-02-23T00:58:29.849960+0000","last_became_active":"2024-02-23T00:57:13.552477+0000","last_became_peered":"2024-02-23T00:57:13.552477+0000","last_unstale":"2024-02-23T00:58:29.849960+0000","last_undegraded":"2024-02-23T00:58:29.849960+0000","last_fullsized":"2024-02-23T00:58:29.849960+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"694'10","last_scrub_stamp":"2024-02-23T00:58:29.849892+0000","last_deep_scrub":"694'10","last_deep_scrub_stamp":"2024-02-23T00:58:29.849892+0000","last_clean_scrub_stamp":"2024-02-23T00:58:29.849892+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"457'42","reported_seq":903,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.482562+0000","last_change":"2024-02-23T00:56:59.534857+0000","last_active":"2024-02-23T00:57:51.482562+0000","last_peered":"2024-02-23T00:57:51.482562+0000","last_clean":"2024-02-23T00:57:51.482562+0000","last_became_active":"2024-02-23T00:39:16.780130+0000","last_became_peered":"2024-02-23T00:39:16.780130+0000","last_unstale":"2024-02-23T00:57:51.482562+0000","last_undegraded":"2024-02-23T00:57:51.482562+0000","last_fullsized":"2024-02-23T00:57:51.482562+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'42","last_scrub_stamp":"2024-02-23T00:56:59.534783+0000","last_deep_scrub":"457'42","last_deep_scrub_stamp":"2024-02-23T00:56:59.534783+0000","last_clean_scrub_stamp":"2024-02-23T00:56:59.534783+0000","log_size":42,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":46,"num_write":34,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":884,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:00.379648+0000","last_change":"2024-02-23T00:58:00.379618+0000","last_active":"2024-02-23T00:58:00.379648+0000","last_peered":"2024-02-23T00:58:00.379648+0000","last_clean":"2024-02-23T00:58:00.379648+0000","last_became_active":"2024-02-23T00:56:22.397648+0000","last_became_peered":"2024-02-23T00:56:22.397648+0000","last_unstale":"2024-02-23T00:58:00.379648+0000","last_undegraded":"2024-02-23T00:58:00.379648+0000","last_fullsized":"2024-02-23T00:58:00.379648+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:00.379541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:51.391939+0000","last_clean_scrub_stamp":"2024-02-23T00:58:00.379541+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":60,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:22.494609+0000","last_change":"2024-02-23T00:58:22.494609+0000","last_active":"2024-02-23T00:58:22.494609+0000","last_peered":"2024-02-23T00:58:22.494609+0000","last_clean":"2024-02-23T00:58:22.494609+0000","last_became_active":"2024-02-23T00:56:22.405106+0000","last_became_peered":"2024-02-23T00:56:22.405106+0000","last_unstale":"2024-02-23T00:58:22.494609+0000","last_undegraded":"2024-02-23T00:58:22.494609+0000","last_fullsized":"2024-02-23T00:58:22.494609+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:22.494495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:46.686764+0000","last_clean_scrub_stamp":"2024-02-23T00:58:22.494495+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":798,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:16.244782+0000","last_change":"2024-02-23T00:58:16.244751+0000","last_active":"2024-02-23T00:58:16.244782+0000","last_peered":"2024-02-23T00:58:16.244782+0000","last_clean":"2024-02-23T00:58:16.244782+0000","last_became_active":"2024-02-23T00:56:08.359212+0000","last_became_peered":"2024-02-23T00:56:08.359212+0000","last_unstale":"2024-02-23T00:58:16.244782+0000","last_undegraded":"2024-02-23T00:58:16.244782+0000","last_fullsized":"2024-02-23T00:58:16.244782+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:16.244672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:58:16.244672+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":883,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:07.417318+0000","last_change":"2024-02-23T00:58:07.417274+0000","last_active":"2024-02-23T00:58:07.417318+0000","last_peered":"2024-02-23T00:58:07.417318+0000","last_clean":"2024-02-23T00:58:07.417318+0000","last_became_active":"2024-02-23T00:56:22.385841+0000","last_became_peered":"2024-02-23T00:56:22.385841+0000","last_unstale":"2024-02-23T00:58:07.417318+0000","last_undegraded":"2024-02-23T00:58:07.417318+0000","last_fullsized":"2024-02-23T00:58:07.417318+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:07.417149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:10.493340+0000","last_clean_scrub_stamp":"2024-02-23T00:58:07.417149+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"457'52","reported_seq":605,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:52.784313+0000","last_change":"2024-02-23T00:57:52.784287+0000","last_active":"2024-02-23T00:57:52.784313+0000","last_peered":"2024-02-23T00:57:52.784313+0000","last_clean":"2024-02-23T00:57:52.784313+0000","last_became_active":"2024-02-23T00:57:13.545133+0000","last_became_peered":"2024-02-23T00:57:13.545133+0000","last_unstale":"2024-02-23T00:57:52.784313+0000","last_undegraded":"2024-02-23T00:57:52.784313+0000","last_fullsized":"2024-02-23T00:57:52.784313+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'52","last_scrub_stamp":"2024-02-23T00:57:52.784217+0000","last_deep_scrub":"457'52","last_deep_scrub_stamp":"2024-02-23T00:53:02.966145+0000","last_clean_scrub_stamp":"2024-02-23T00:57:52.784217+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":75,"num_read_kb":61,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":793,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.479465+0000","last_change":"2024-02-23T00:57:21.852554+0000","last_active":"2024-02-23T00:57:51.479465+0000","last_peered":"2024-02-23T00:57:51.479465+0000","last_clean":"2024-02-23T00:57:51.479465+0000","last_became_active":"2024-02-23T00:56:59.505952+0000","last_became_peered":"2024-02-23T00:56:59.505952+0000","last_unstale":"2024-02-23T00:57:51.479465+0000","last_undegraded":"2024-02-23T00:57:51.479465+0000","last_fullsized":"2024-02-23T00:57:51.479465+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:21.852473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:57:21.852473+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":893,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:55.516470+0000","last_change":"2024-02-23T00:57:55.516427+0000","last_active":"2024-02-23T00:57:55.516470+0000","last_peered":"2024-02-23T00:57:55.516470+0000","last_clean":"2024-02-23T00:57:55.516470+0000","last_became_active":"2024-02-23T00:56:08.337730+0000","last_became_peered":"2024-02-23T00:56:08.337730+0000","last_unstale":"2024-02-23T00:57:55.516470+0000","last_undegraded":"2024-02-23T00:57:55.516470+0000","last_fullsized":"2024-02-23T00:57:55.516470+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:55.516320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:57:55.516320+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":705,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.481027+0000","last_change":"2024-02-23T00:57:23.105307+0000","last_active":"2024-02-23T00:57:51.481027+0000","last_peered":"2024-02-23T00:57:51.481027+0000","last_clean":"2024-02-23T00:57:51.481027+0000","last_became_active":"2024-02-23T00:57:13.548714+0000","last_became_peered":"2024-02-23T00:57:13.548714+0000","last_unstale":"2024-02-23T00:57:51.481027+0000","last_undegraded":"2024-02-23T00:57:51.481027+0000","last_fullsized":"2024-02-23T00:57:51.481027+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:23.105223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:57:23.105223+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":869,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:01.418183+0000","last_change":"2024-02-23T00:58:01.418152+0000","last_active":"2024-02-23T00:58:01.418183+0000","last_peered":"2024-02-23T00:58:01.418183+0000","last_clean":"2024-02-23T00:58:01.418183+0000","last_became_active":"2024-02-23T00:55:19.374247+0000","last_became_peered":"2024-02-23T00:55:19.374247+0000","last_unstale":"2024-02-23T00:58:01.418183+0000","last_undegraded":"2024-02-23T00:58:01.418183+0000","last_fullsized":"2024-02-23T00:58:01.418183+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":770,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:01.418074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:58:01.418074+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"457'44","reported_seq":203,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.483482+0000","last_change":"2024-02-23T00:57:12.117290+0000","last_active":"2024-02-23T00:57:51.483482+0000","last_peered":"2024-02-23T00:57:51.483482+0000","last_clean":"2024-02-23T00:57:51.483482+0000","last_became_active":"2024-02-23T00:56:22.401039+0000","last_became_peered":"2024-02-23T00:56:22.401039+0000","last_unstale":"2024-02-23T00:57:51.483482+0000","last_undegraded":"2024-02-23T00:57:51.483482+0000","last_fullsized":"2024-02-23T00:57:51.483482+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'44","last_scrub_stamp":"2024-02-23T00:57:12.117220+0000","last_deep_scrub":"457'44","last_deep_scrub_stamp":"2024-02-23T00:57:12.117220+0000","last_clean_scrub_stamp":"2024-02-23T00:57:12.117220+0000","log_size":44,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":55,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.486742+0000","last_change":"2024-02-23T00:57:22.227757+0000","last_active":"2024-02-23T00:57:51.486742+0000","last_peered":"2024-02-23T00:57:51.486742+0000","last_clean":"2024-02-23T00:57:51.486742+0000","last_became_active":"2024-02-23T00:56:22.387775+0000","last_became_peered":"2024-02-23T00:56:22.387775+0000","last_unstale":"2024-02-23T00:57:51.486742+0000","last_undegraded":"2024-02-23T00:57:51.486742+0000","last_fullsized":"2024-02-23T00:57:51.486742+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:22.227657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:57:22.227657+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":827,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.480255+0000","last_change":"2024-02-23T00:57:49.846945+0000","last_active":"2024-02-23T00:57:51.480255+0000","last_peered":"2024-02-23T00:57:51.480255+0000","last_clean":"2024-02-23T00:57:51.480255+0000","last_became_active":"2024-02-23T00:56:59.505756+0000","last_became_peered":"2024-02-23T00:56:59.505756+0000","last_unstale":"2024-02-23T00:57:51.480255+0000","last_undegraded":"2024-02-23T00:57:51.480255+0000","last_fullsized":"2024-02-23T00:57:51.480255+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:49.846837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:57:49.846837+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":782,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:58.125976+0000","last_change":"2024-02-23T00:57:58.125926+0000","last_active":"2024-02-23T00:57:58.125976+0000","last_peered":"2024-02-23T00:57:58.125976+0000","last_clean":"2024-02-23T00:57:58.125976+0000","last_became_active":"2024-02-23T00:42:58.726652+0000","last_became_peered":"2024-02-23T00:42:58.726652+0000","last_unstale":"2024-02-23T00:57:58.125976+0000","last_undegraded":"2024-02-23T00:57:58.125976+0000","last_fullsized":"2024-02-23T00:57:58.125976+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:58.125844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:57:58.125844+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":876,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:17.206815+0000","last_change":"2024-02-23T00:58:17.206770+0000","last_active":"2024-02-23T00:58:17.206815+0000","last_peered":"2024-02-23T00:58:17.206815+0000","last_clean":"2024-02-23T00:58:17.206815+0000","last_became_active":"2024-02-23T00:55:45.325453+0000","last_became_peered":"2024-02-23T00:55:45.325453+0000","last_unstale":"2024-02-23T00:58:17.206815+0000","last_undegraded":"2024-02-23T00:58:17.206815+0000","last_fullsized":"2024-02-23T00:58:17.206815+0000","mapping_epoch":789,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":790,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:17.206687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:59.694033+0000","last_clean_scrub_stamp":"2024-02-23T00:58:17.206687+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"457'30","reported_seq":192,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.484429+0000","last_change":"2024-02-23T00:57:33.182257+0000","last_active":"2024-02-23T00:57:51.484429+0000","last_peered":"2024-02-23T00:57:51.484429+0000","last_clean":"2024-02-23T00:57:51.484429+0000","last_became_active":"2024-02-23T00:56:08.353636+0000","last_became_peered":"2024-02-23T00:56:08.353636+0000","last_unstale":"2024-02-23T00:57:51.484429+0000","last_undegraded":"2024-02-23T00:57:51.484429+0000","last_fullsized":"2024-02-23T00:57:51.484429+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'30","last_scrub_stamp":"2024-02-23T00:57:33.182120+0000","last_deep_scrub":"95'18","last_deep_scrub_stamp":"2024-02-23T00:42:01.195545+0000","last_clean_scrub_stamp":"2024-02-23T00:57:33.182120+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":60,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:05.271722+0000","last_change":"2024-02-23T00:58:05.271692+0000","last_active":"2024-02-23T00:58:05.271722+0000","last_peered":"2024-02-23T00:58:05.271722+0000","last_clean":"2024-02-23T00:58:05.271722+0000","last_became_active":"2024-02-23T00:56:22.401333+0000","last_became_peered":"2024-02-23T00:56:22.401333+0000","last_unstale":"2024-02-23T00:58:05.271722+0000","last_undegraded":"2024-02-23T00:58:05.271722+0000","last_fullsized":"2024-02-23T00:58:05.271722+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:05.271595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:58:05.271595+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":130,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:06.283889+0000","last_change":"2024-02-23T00:58:06.283843+0000","last_active":"2024-02-23T00:58:06.283889+0000","last_peered":"2024-02-23T00:58:06.283889+0000","last_clean":"2024-02-23T00:58:06.283889+0000","last_became_active":"2024-02-23T00:54:56.471337+0000","last_became_peered":"2024-02-23T00:54:56.471337+0000","last_unstale":"2024-02-23T00:58:06.283889+0000","last_undegraded":"2024-02-23T00:58:06.283889+0000","last_fullsized":"2024-02-23T00:58:06.283889+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:06.283728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:58:06.283728+0000","last_clean_scrub_stamp":"2024-02-23T00:58:06.283728+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"67'1","reported_seq":887,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.480864+0000","last_change":"2024-02-23T00:57:10.876524+0000","last_active":"2024-02-23T00:57:51.480864+0000","last_peered":"2024-02-23T00:57:51.480864+0000","last_clean":"2024-02-23T00:57:51.480864+0000","last_became_active":"2024-02-23T00:56:59.509982+0000","last_became_peered":"2024-02-23T00:56:59.509982+0000","last_unstale":"2024-02-23T00:57:51.480864+0000","last_undegraded":"2024-02-23T00:57:51.480864+0000","last_fullsized":"2024-02-23T00:57:51.480864+0000","mapping_epoch":837,"log_start":"67'1","ondisk_log_start":"67'1","created":65,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:57:10.876415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:57:10.876415+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1476,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":824,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:15.522615+0000","last_change":"2024-02-23T00:58:15.522587+0000","last_active":"2024-02-23T00:58:15.522615+0000","last_peered":"2024-02-23T00:58:15.522615+0000","last_clean":"2024-02-23T00:58:15.522615+0000","last_became_active":"2024-02-23T00:56:22.387939+0000","last_became_peered":"2024-02-23T00:56:22.387939+0000","last_unstale":"2024-02-23T00:58:15.522615+0000","last_undegraded":"2024-02-23T00:58:15.522615+0000","last_fullsized":"2024-02-23T00:58:15.522615+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:15.522513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:58:15.522513+0000","last_clean_scrub_stamp":"2024-02-23T00:58:15.522513+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":60,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:15.320346+0000","last_change":"2024-02-23T00:58:15.320311+0000","last_active":"2024-02-23T00:58:15.320346+0000","last_peered":"2024-02-23T00:58:15.320346+0000","last_clean":"2024-02-23T00:58:15.320346+0000","last_became_active":"2024-02-23T00:56:22.399167+0000","last_became_peered":"2024-02-23T00:56:22.399167+0000","last_unstale":"2024-02-23T00:58:15.320346+0000","last_undegraded":"2024-02-23T00:58:15.320346+0000","last_fullsized":"2024-02-23T00:58:15.320346+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:15.320209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:57:02.099770+0000","last_clean_scrub_stamp":"2024-02-23T00:58:15.320209+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"457'26","reported_seq":936,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:06.463939+0000","last_change":"2024-02-23T00:58:06.463905+0000","last_active":"2024-02-23T00:58:06.463939+0000","last_peered":"2024-02-23T00:58:06.463939+0000","last_clean":"2024-02-23T00:58:06.463939+0000","last_became_active":"2024-02-23T00:56:22.391810+0000","last_became_peered":"2024-02-23T00:56:22.391810+0000","last_unstale":"2024-02-23T00:58:06.463939+0000","last_undegraded":"2024-02-23T00:58:06.463939+0000","last_fullsized":"2024-02-23T00:58:06.463939+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-23T00:58:06.463820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:58:06.463820+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":308,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":55,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.486213+0000","last_change":"2024-02-23T00:57:21.262854+0000","last_active":"2024-02-23T00:57:51.486213+0000","last_peered":"2024-02-23T00:57:51.486213+0000","last_clean":"2024-02-23T00:57:51.486213+0000","last_became_active":"2024-02-23T00:56:22.397990+0000","last_became_peered":"2024-02-23T00:56:22.397990+0000","last_unstale":"2024-02-23T00:57:51.486213+0000","last_undegraded":"2024-02-23T00:57:51.486213+0000","last_fullsized":"2024-02-23T00:57:51.486213+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:21.262795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:49.977248+0000","last_clean_scrub_stamp":"2024-02-23T00:57:21.262795+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":875,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.484668+0000","last_change":"2024-02-23T00:57:22.209079+0000","last_active":"2024-02-23T00:57:51.484668+0000","last_peered":"2024-02-23T00:57:51.484668+0000","last_clean":"2024-02-23T00:57:51.484668+0000","last_became_active":"2024-02-23T00:56:22.396589+0000","last_became_peered":"2024-02-23T00:56:22.396589+0000","last_unstale":"2024-02-23T00:57:51.484668+0000","last_undegraded":"2024-02-23T00:57:51.484668+0000","last_fullsized":"2024-02-23T00:57:51.484668+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:22.209010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:57:22.209010+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":71,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.479713+0000","last_change":"2024-02-23T00:57:32.861242+0000","last_active":"2024-02-23T00:57:51.479713+0000","last_peered":"2024-02-23T00:57:51.479713+0000","last_clean":"2024-02-23T00:57:51.479713+0000","last_became_active":"2024-02-23T00:57:13.548877+0000","last_became_peered":"2024-02-23T00:57:13.548877+0000","last_unstale":"2024-02-23T00:57:51.479713+0000","last_undegraded":"2024-02-23T00:57:51.479713+0000","last_fullsized":"2024-02-23T00:57:51.479713+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:32.861176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:57:32.861176+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":869,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.485447+0000","last_change":"2024-02-23T00:56:59.296066+0000","last_active":"2024-02-23T00:57:51.485447+0000","last_peered":"2024-02-23T00:57:51.485447+0000","last_clean":"2024-02-23T00:57:51.485447+0000","last_became_active":"2024-02-23T00:47:45.182550+0000","last_became_peered":"2024-02-23T00:47:45.182550+0000","last_unstale":"2024-02-23T00:57:51.485447+0000","last_undegraded":"2024-02-23T00:57:51.485447+0000","last_fullsized":"2024-02-23T00:57:51.485447+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:56:59.295987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:56:59.295987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":55,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.486059+0000","last_change":"2024-02-23T00:57:27.224397+0000","last_active":"2024-02-23T00:57:51.486059+0000","last_peered":"2024-02-23T00:57:51.486059+0000","last_clean":"2024-02-23T00:57:51.486059+0000","last_became_active":"2024-02-23T00:56:22.404754+0000","last_became_peered":"2024-02-23T00:56:22.404754+0000","last_unstale":"2024-02-23T00:57:51.486059+0000","last_undegraded":"2024-02-23T00:57:51.486059+0000","last_fullsized":"2024-02-23T00:57:51.486059+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:27.224323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:57:27.224323+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":75,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:58.124076+0000","last_change":"2024-02-23T00:57:58.124049+0000","last_active":"2024-02-23T00:57:58.124076+0000","last_peered":"2024-02-23T00:57:58.124076+0000","last_clean":"2024-02-23T00:57:58.124076+0000","last_became_active":"2024-02-23T00:56:22.396739+0000","last_became_peered":"2024-02-23T00:56:22.396739+0000","last_unstale":"2024-02-23T00:57:58.124076+0000","last_undegraded":"2024-02-23T00:57:58.124076+0000","last_fullsized":"2024-02-23T00:57:58.124076+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:58.123980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.264240+0000","last_clean_scrub_stamp":"2024-02-23T00:57:58.123980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"851'29","reported_seq":891,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.479941+0000","last_change":"2024-02-23T00:57:32.971441+0000","last_active":"2024-02-23T00:57:51.479941+0000","last_peered":"2024-02-23T00:57:51.479941+0000","last_clean":"2024-02-23T00:57:51.479941+0000","last_became_active":"2024-02-23T00:56:59.508879+0000","last_became_peered":"2024-02-23T00:56:59.508879+0000","last_unstale":"2024-02-23T00:57:51.479941+0000","last_undegraded":"2024-02-23T00:57:51.479941+0000","last_fullsized":"2024-02-23T00:57:51.479941+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"851'29","last_scrub_stamp":"2024-02-23T00:57:32.971380+0000","last_deep_scrub":"851'29","last_deep_scrub_stamp":"2024-02-23T00:57:32.971380+0000","last_clean_scrub_stamp":"2024-02-23T00:57:32.971380+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":48,"num_read_kb":36,"num_write":26,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":66,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:06.206799+0000","last_change":"2024-02-23T00:58:06.206752+0000","last_active":"2024-02-23T00:58:06.206799+0000","last_peered":"2024-02-23T00:58:06.206799+0000","last_clean":"2024-02-23T00:58:06.206799+0000","last_became_active":"2024-02-23T00:56:08.354250+0000","last_became_peered":"2024-02-23T00:56:08.354250+0000","last_unstale":"2024-02-23T00:58:06.206799+0000","last_undegraded":"2024-02-23T00:58:06.206799+0000","last_fullsized":"2024-02-23T00:58:06.206799+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:06.206629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:44.659246+0000","last_clean_scrub_stamp":"2024-02-23T00:58:06.206629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"707'6","reported_seq":1916,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:30.673118+0000","last_change":"2024-02-23T00:57:39.910370+0000","last_active":"2024-02-23T00:58:30.673118+0000","last_peered":"2024-02-23T00:58:30.673118+0000","last_clean":"2024-02-23T00:58:30.673118+0000","last_became_active":"2024-02-23T00:56:59.507898+0000","last_became_peered":"2024-02-23T00:56:59.507898+0000","last_unstale":"2024-02-23T00:58:30.673118+0000","last_undegraded":"2024-02-23T00:58:30.673118+0000","last_fullsized":"2024-02-23T00:58:30.673118+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"707'6","last_scrub_stamp":"2024-02-23T00:57:39.910298+0000","last_deep_scrub":"707'6","last_deep_scrub_stamp":"2024-02-23T00:57:39.910298+0000","last_clean_scrub_stamp":"2024-02-23T00:57:39.910298+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1077,"num_read_kb":1072,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":60,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:14.277871+0000","last_change":"2024-02-23T00:58:14.277840+0000","last_active":"2024-02-23T00:58:14.277871+0000","last_peered":"2024-02-23T00:58:14.277871+0000","last_clean":"2024-02-23T00:58:14.277871+0000","last_became_active":"2024-02-23T00:56:22.399702+0000","last_became_peered":"2024-02-23T00:56:22.399702+0000","last_unstale":"2024-02-23T00:58:14.277871+0000","last_undegraded":"2024-02-23T00:58:14.277871+0000","last_fullsized":"2024-02-23T00:58:14.277871+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:14.277759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:58:14.277759+0000","last_clean_scrub_stamp":"2024-02-23T00:58:14.277759+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":753,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.840238+0000","last_change":"2024-02-23T00:57:51.840212+0000","last_active":"2024-02-23T00:57:51.840238+0000","last_peered":"2024-02-23T00:57:51.840238+0000","last_clean":"2024-02-23T00:57:51.840238+0000","last_became_active":"2024-02-23T00:56:59.508642+0000","last_became_peered":"2024-02-23T00:56:59.508642+0000","last_unstale":"2024-02-23T00:57:51.840238+0000","last_undegraded":"2024-02-23T00:57:51.840238+0000","last_fullsized":"2024-02-23T00:57:51.840238+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:51.840138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:57:51.840138+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":139,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:08.262804+0000","last_change":"2024-02-23T00:58:08.262766+0000","last_active":"2024-02-23T00:58:08.262804+0000","last_peered":"2024-02-23T00:58:08.262804+0000","last_clean":"2024-02-23T00:58:08.262804+0000","last_became_active":"2024-02-23T00:55:19.374145+0000","last_became_peered":"2024-02-23T00:55:19.374145+0000","last_unstale":"2024-02-23T00:58:08.262804+0000","last_undegraded":"2024-02-23T00:58:08.262804+0000","last_fullsized":"2024-02-23T00:58:08.262804+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":770,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:08.262663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:58:08.262663+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"813'51","reported_seq":1202,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:23.058815+0000","last_change":"2024-02-23T00:57:31.174490+0000","last_active":"2024-02-23T00:58:23.058815+0000","last_peered":"2024-02-23T00:58:23.058815+0000","last_clean":"2024-02-23T00:58:23.058815+0000","last_became_active":"2024-02-23T00:56:08.359424+0000","last_became_peered":"2024-02-23T00:56:08.359424+0000","last_unstale":"2024-02-23T00:58:23.058815+0000","last_undegraded":"2024-02-23T00:58:23.058815+0000","last_fullsized":"2024-02-23T00:58:23.058815+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"813'51","last_scrub_stamp":"2024-02-23T00:57:31.174412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:57:31.174412+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":78,"num_read_kb":62,"num_write":44,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":1790,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":844,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:27.131954+0000","last_change":"2024-02-23T00:58:27.131954+0000","last_active":"2024-02-23T00:58:27.131954+0000","last_peered":"2024-02-23T00:58:27.131954+0000","last_clean":"2024-02-23T00:58:27.131954+0000","last_became_active":"2024-02-23T00:39:14.784015+0000","last_became_peered":"2024-02-23T00:39:14.784015+0000","last_unstale":"2024-02-23T00:58:27.131954+0000","last_undegraded":"2024-02-23T00:58:27.131954+0000","last_fullsized":"2024-02-23T00:58:27.131954+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:27.131872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:58:27.131872+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":598,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.481125+0000","last_change":"2024-02-23T00:57:47.854706+0000","last_active":"2024-02-23T00:57:51.481125+0000","last_peered":"2024-02-23T00:57:51.481125+0000","last_clean":"2024-02-23T00:57:51.481125+0000","last_became_active":"2024-02-23T00:57:13.546237+0000","last_became_peered":"2024-02-23T00:57:13.546237+0000","last_unstale":"2024-02-23T00:57:51.481125+0000","last_undegraded":"2024-02-23T00:57:51.481125+0000","last_fullsized":"2024-02-23T00:57:51.481125+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:47.854628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:57:47.854628+0000","last_clean_scrub_stamp":"2024-02-23T00:57:47.854628+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":66,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:52.166354+0000","last_change":"2024-02-23T00:57:52.166328+0000","last_active":"2024-02-23T00:57:52.166354+0000","last_peered":"2024-02-23T00:57:52.166354+0000","last_clean":"2024-02-23T00:57:52.166354+0000","last_became_active":"2024-02-23T00:56:08.362612+0000","last_became_peered":"2024-02-23T00:56:08.362612+0000","last_unstale":"2024-02-23T00:57:52.166354+0000","last_undegraded":"2024-02-23T00:57:52.166354+0000","last_fullsized":"2024-02-23T00:57:52.166354+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:52.166254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:19.722210+0000","last_clean_scrub_stamp":"2024-02-23T00:57:52.166254+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"749'11","reported_seq":148,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:28.926225+0000","last_change":"2024-02-23T00:58:28.926225+0000","last_active":"2024-02-23T00:58:28.926225+0000","last_peered":"2024-02-23T00:58:28.926225+0000","last_clean":"2024-02-23T00:58:28.926225+0000","last_became_active":"2024-02-23T00:55:19.371588+0000","last_became_peered":"2024-02-23T00:55:19.371588+0000","last_unstale":"2024-02-23T00:58:28.926225+0000","last_undegraded":"2024-02-23T00:58:28.926225+0000","last_fullsized":"2024-02-23T00:58:28.926225+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":770,"parent":"0.0","parent_split_bits":0,"last_scrub":"749'11","last_scrub_stamp":"2024-02-23T00:58:28.926136+0000","last_deep_scrub":"749'11","last_deep_scrub_stamp":"2024-02-23T00:58:28.926136+0000","last_clean_scrub_stamp":"2024-02-23T00:58:28.926136+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"853'32","reported_seq":842,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.480505+0000","last_change":"2024-02-23T00:57:36.902272+0000","last_active":"2024-02-23T00:57:51.480505+0000","last_peered":"2024-02-23T00:57:51.480505+0000","last_clean":"2024-02-23T00:57:51.480505+0000","last_became_active":"2024-02-23T00:56:59.510594+0000","last_became_peered":"2024-02-23T00:56:59.510594+0000","last_unstale":"2024-02-23T00:57:51.480505+0000","last_undegraded":"2024-02-23T00:57:51.480505+0000","last_fullsized":"2024-02-23T00:57:51.480505+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"853'32","last_scrub_stamp":"2024-02-23T00:57:36.902205+0000","last_deep_scrub":"853'32","last_deep_scrub_stamp":"2024-02-23T00:57:36.902205+0000","last_clean_scrub_stamp":"2024-02-23T00:57:36.902205+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":36,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":865,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:29.687878+0000","last_change":"2024-02-23T00:58:29.687878+0000","last_active":"2024-02-23T00:58:29.687878+0000","last_peered":"2024-02-23T00:58:29.687878+0000","last_clean":"2024-02-23T00:58:29.687878+0000","last_became_active":"2024-02-23T00:56:59.509868+0000","last_became_peered":"2024-02-23T00:56:59.509868+0000","last_unstale":"2024-02-23T00:58:29.687878+0000","last_undegraded":"2024-02-23T00:58:29.687878+0000","last_fullsized":"2024-02-23T00:58:29.687878+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:29.687817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:58:29.687817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"107'2","reported_seq":245,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:25.936451+0000","last_change":"2024-02-23T00:58:25.936451+0000","last_active":"2024-02-23T00:58:25.936451+0000","last_peered":"2024-02-23T00:58:25.936451+0000","last_clean":"2024-02-23T00:58:25.936451+0000","last_became_active":"2024-02-23T00:56:40.822752+0000","last_became_peered":"2024-02-23T00:56:40.822752+0000","last_unstale":"2024-02-23T00:58:25.936451+0000","last_undegraded":"2024-02-23T00:58:25.936451+0000","last_fullsized":"2024-02-23T00:58:25.936451+0000","mapping_epoch":828,"log_start":"107'2","ondisk_log_start":"107'2","created":65,"last_epoch_clean":829,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'2","last_scrub_stamp":"2024-02-23T00:58:25.936376+0000","last_deep_scrub":"107'2","last_deep_scrub_stamp":"2024-02-23T00:58:25.936376+0000","last_clean_scrub_stamp":"2024-02-23T00:58:25.936376+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":39,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":837,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:05.449464+0000","last_change":"2024-02-23T00:58:05.449431+0000","last_active":"2024-02-23T00:58:05.449464+0000","last_peered":"2024-02-23T00:58:05.449464+0000","last_clean":"2024-02-23T00:58:05.449464+0000","last_became_active":"2024-02-23T00:56:22.397674+0000","last_became_peered":"2024-02-23T00:56:22.397674+0000","last_unstale":"2024-02-23T00:58:05.449464+0000","last_undegraded":"2024-02-23T00:58:05.449464+0000","last_fullsized":"2024-02-23T00:58:05.449464+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:05.449359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:58:05.449359+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":66,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.483724+0000","last_change":"2024-02-23T00:57:51.144096+0000","last_active":"2024-02-23T00:57:51.483724+0000","last_peered":"2024-02-23T00:57:51.483724+0000","last_clean":"2024-02-23T00:57:51.483724+0000","last_became_active":"2024-02-23T00:56:08.363537+0000","last_became_peered":"2024-02-23T00:56:08.363537+0000","last_unstale":"2024-02-23T00:57:51.483724+0000","last_undegraded":"2024-02-23T00:57:51.483724+0000","last_fullsized":"2024-02-23T00:57:51.483724+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:51.143998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_clean_scrub_stamp":"2024-02-23T00:57:51.143998+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"457'31","reported_seq":951,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.482772+0000","last_change":"2024-02-23T00:57:18.288861+0000","last_active":"2024-02-23T00:57:51.482772+0000","last_peered":"2024-02-23T00:57:51.482772+0000","last_clean":"2024-02-23T00:57:51.482772+0000","last_became_active":"2024-02-23T00:53:53.410425+0000","last_became_peered":"2024-02-23T00:53:53.410425+0000","last_unstale":"2024-02-23T00:57:51.482772+0000","last_undegraded":"2024-02-23T00:57:51.482772+0000","last_fullsized":"2024-02-23T00:57:51.482772+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:57:18.288796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:57:18.288796+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":58,"num_read_kb":47,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"95'4","reported_seq":920,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.483714+0000","last_change":"2024-02-23T00:57:49.441397+0000","last_active":"2024-02-23T00:57:51.483714+0000","last_peered":"2024-02-23T00:57:51.483714+0000","last_clean":"2024-02-23T00:57:51.483714+0000","last_became_active":"2024-02-23T00:56:22.389241+0000","last_became_peered":"2024-02-23T00:56:22.389241+0000","last_unstale":"2024-02-23T00:57:51.483714+0000","last_undegraded":"2024-02-23T00:57:51.483714+0000","last_fullsized":"2024-02-23T00:57:51.483714+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'4","last_scrub_stamp":"2024-02-23T00:57:49.441310+0000","last_deep_scrub":"95'4","last_deep_scrub_stamp":"2024-02-23T00:56:30.393527+0000","last_clean_scrub_stamp":"2024-02-23T00:57:49.441310+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":697,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:24.799219+0000","last_change":"2024-02-23T00:58:24.799219+0000","last_active":"2024-02-23T00:58:24.799219+0000","last_peered":"2024-02-23T00:58:24.799219+0000","last_clean":"2024-02-23T00:58:24.799219+0000","last_became_active":"2024-02-23T00:57:13.553197+0000","last_became_peered":"2024-02-23T00:57:13.553197+0000","last_unstale":"2024-02-23T00:58:24.799219+0000","last_undegraded":"2024-02-23T00:58:24.799219+0000","last_fullsized":"2024-02-23T00:58:24.799219+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:24.799094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:23.776775+0000","last_clean_scrub_stamp":"2024-02-23T00:58:24.799094+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":60,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.486899+0000","last_change":"2024-02-23T00:57:46.284519+0000","last_active":"2024-02-23T00:57:51.486899+0000","last_peered":"2024-02-23T00:57:51.486899+0000","last_clean":"2024-02-23T00:57:51.486899+0000","last_became_active":"2024-02-23T00:56:22.407258+0000","last_became_peered":"2024-02-23T00:56:22.407258+0000","last_unstale":"2024-02-23T00:57:51.486899+0000","last_undegraded":"2024-02-23T00:57:51.486899+0000","last_fullsized":"2024-02-23T00:57:51.486899+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:46.284436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:57:46.284436+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"833'33","reported_seq":219,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:01.273292+0000","last_change":"2024-02-23T00:58:01.273267+0000","last_active":"2024-02-23T00:58:01.273292+0000","last_peered":"2024-02-23T00:58:01.273292+0000","last_clean":"2024-02-23T00:58:01.273292+0000","last_became_active":"2024-02-23T00:56:22.391449+0000","last_became_peered":"2024-02-23T00:56:22.391449+0000","last_unstale":"2024-02-23T00:58:01.273292+0000","last_undegraded":"2024-02-23T00:58:01.273292+0000","last_fullsized":"2024-02-23T00:58:01.273292+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"833'33","last_scrub_stamp":"2024-02-23T00:58:01.273197+0000","last_deep_scrub":"833'33","last_deep_scrub_stamp":"2024-02-23T00:56:49.165587+0000","last_clean_scrub_stamp":"2024-02-23T00:58:01.273197+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":49,"num_read_kb":41,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":61,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.483812+0000","last_change":"2024-02-23T00:57:13.948445+0000","last_active":"2024-02-23T00:57:51.483812+0000","last_peered":"2024-02-23T00:57:51.483812+0000","last_clean":"2024-02-23T00:57:51.483812+0000","last_became_active":"2024-02-23T00:56:08.364142+0000","last_became_peered":"2024-02-23T00:56:08.364142+0000","last_unstale":"2024-02-23T00:57:51.483812+0000","last_undegraded":"2024-02-23T00:57:51.483812+0000","last_fullsized":"2024-02-23T00:57:51.483812+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:13.948378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:57:13.948378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":195,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.479471+0000","last_change":"2024-02-23T00:57:26.050076+0000","last_active":"2024-02-23T00:57:51.479471+0000","last_peered":"2024-02-23T00:57:51.479471+0000","last_clean":"2024-02-23T00:57:51.479471+0000","last_became_active":"2024-02-23T00:57:13.553941+0000","last_became_peered":"2024-02-23T00:57:13.553941+0000","last_unstale":"2024-02-23T00:57:51.479471+0000","last_undegraded":"2024-02-23T00:57:51.479471+0000","last_fullsized":"2024-02-23T00:57:51.479471+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:26.050010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:45.913526+0000","last_clean_scrub_stamp":"2024-02-23T00:57:26.050010+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":130,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:29.926281+0000","last_change":"2024-02-23T00:58:29.926281+0000","last_active":"2024-02-23T00:58:29.926281+0000","last_peered":"2024-02-23T00:58:29.926281+0000","last_clean":"2024-02-23T00:58:29.926281+0000","last_became_active":"2024-02-23T00:54:56.487615+0000","last_became_peered":"2024-02-23T00:54:56.487615+0000","last_unstale":"2024-02-23T00:58:29.926281+0000","last_undegraded":"2024-02-23T00:58:29.926281+0000","last_fullsized":"2024-02-23T00:58:29.926281+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:29.926220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:58:29.926220+0000","last_clean_scrub_stamp":"2024-02-23T00:58:29.926220+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"457'22","reported_seq":545,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:58.767217+0000","last_change":"2024-02-23T00:57:58.767188+0000","last_active":"2024-02-23T00:57:58.767217+0000","last_peered":"2024-02-23T00:57:58.767217+0000","last_clean":"2024-02-23T00:57:58.767217+0000","last_became_active":"2024-02-23T00:57:13.553644+0000","last_became_peered":"2024-02-23T00:57:13.553644+0000","last_unstale":"2024-02-23T00:57:58.767217+0000","last_undegraded":"2024-02-23T00:57:58.767217+0000","last_fullsized":"2024-02-23T00:57:58.767217+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'22","last_scrub_stamp":"2024-02-23T00:57:58.767111+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-02-23T00:48:07.941101+0000","last_clean_scrub_stamp":"2024-02-23T00:57:58.767111+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":841,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:25.653778+0000","last_change":"2024-02-23T00:58:25.653778+0000","last_active":"2024-02-23T00:58:25.653778+0000","last_peered":"2024-02-23T00:58:25.653778+0000","last_clean":"2024-02-23T00:58:25.653778+0000","last_became_active":"2024-02-23T00:39:14.783164+0000","last_became_peered":"2024-02-23T00:39:14.783164+0000","last_unstale":"2024-02-23T00:58:25.653778+0000","last_undegraded":"2024-02-23T00:58:25.653778+0000","last_fullsized":"2024-02-23T00:58:25.653778+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:25.653700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:58:25.653700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":840,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:32.200528+0000","last_change":"2024-02-23T00:58:32.200528+0000","last_active":"2024-02-23T00:58:32.200528+0000","last_peered":"2024-02-23T00:58:32.200528+0000","last_clean":"2024-02-23T00:58:32.200528+0000","last_became_active":"2024-02-23T00:39:18.800297+0000","last_became_peered":"2024-02-23T00:39:18.800297+0000","last_unstale":"2024-02-23T00:58:32.200528+0000","last_undegraded":"2024-02-23T00:58:32.200528+0000","last_fullsized":"2024-02-23T00:58:32.200528+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:32.200439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:58:32.200439+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1","version":"110'7","reported_seq":644,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:23.776327+0000","last_change":"2024-02-23T00:58:23.776327+0000","last_active":"2024-02-23T00:58:23.776327+0000","last_peered":"2024-02-23T00:58:23.776327+0000","last_clean":"2024-02-23T00:58:23.776327+0000","last_became_active":"2024-02-23T00:57:13.550227+0000","last_became_peered":"2024-02-23T00:57:13.550227+0000","last_unstale":"2024-02-23T00:58:23.776327+0000","last_undegraded":"2024-02-23T00:58:23.776327+0000","last_fullsized":"2024-02-23T00:58:23.776327+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'7","last_scrub_stamp":"2024-02-23T00:58:23.776257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:58:23.776257+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"457'32","reported_seq":386,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.480287+0000","last_change":"2024-02-23T00:57:45.883434+0000","last_active":"2024-02-23T00:57:51.480287+0000","last_peered":"2024-02-23T00:57:51.480287+0000","last_clean":"2024-02-23T00:57:51.480287+0000","last_became_active":"2024-02-23T00:57:13.552747+0000","last_became_peered":"2024-02-23T00:57:13.552747+0000","last_unstale":"2024-02-23T00:57:51.480287+0000","last_undegraded":"2024-02-23T00:57:51.480287+0000","last_fullsized":"2024-02-23T00:57:51.480287+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-23T00:57:45.883360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:57:45.883360+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":52,"num_read_kb":45,"num_write":32,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":862,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:11.262157+0000","last_change":"2024-02-23T00:58:11.262121+0000","last_active":"2024-02-23T00:58:11.262157+0000","last_peered":"2024-02-23T00:58:11.262157+0000","last_clean":"2024-02-23T00:58:11.262157+0000","last_became_active":"2024-02-23T00:56:22.393719+0000","last_became_peered":"2024-02-23T00:56:22.393719+0000","last_unstale":"2024-02-23T00:58:11.262157+0000","last_undegraded":"2024-02-23T00:58:11.262157+0000","last_fullsized":"2024-02-23T00:58:11.262157+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:11.262038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_clean_scrub_stamp":"2024-02-23T00:58:11.262038+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"749'11","reported_seq":1106,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:23.058775+0000","last_change":"2024-02-23T00:58:13.248021+0000","last_active":"2024-02-23T00:58:23.058775+0000","last_peered":"2024-02-23T00:58:23.058775+0000","last_clean":"2024-02-23T00:58:23.058775+0000","last_became_active":"2024-02-23T00:54:56.483713+0000","last_became_peered":"2024-02-23T00:54:56.483713+0000","last_unstale":"2024-02-23T00:58:23.058775+0000","last_undegraded":"2024-02-23T00:58:23.058775+0000","last_fullsized":"2024-02-23T00:58:23.058775+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"749'11","last_scrub_stamp":"2024-02-23T00:58:13.247944+0000","last_deep_scrub":"749'11","last_deep_scrub_stamp":"2024-02-23T00:57:01.303627+0000","last_clean_scrub_stamp":"2024-02-23T00:58:13.247944+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":60,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:24.488463+0000","last_change":"2024-02-23T00:58:24.488463+0000","last_active":"2024-02-23T00:58:24.488463+0000","last_peered":"2024-02-23T00:58:24.488463+0000","last_clean":"2024-02-23T00:58:24.488463+0000","last_became_active":"2024-02-23T00:56:22.394948+0000","last_became_peered":"2024-02-23T00:56:22.394948+0000","last_unstale":"2024-02-23T00:58:24.488463+0000","last_undegraded":"2024-02-23T00:58:24.488463+0000","last_fullsized":"2024-02-23T00:58:24.488463+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:24.488336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:53:00.410226+0000","last_clean_scrub_stamp":"2024-02-23T00:58:24.488336+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"457'21","reported_seq":919,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.485033+0000","last_change":"2024-02-23T00:57:18.247304+0000","last_active":"2024-02-23T00:57:51.485033+0000","last_peered":"2024-02-23T00:57:51.485033+0000","last_clean":"2024-02-23T00:57:51.485033+0000","last_became_active":"2024-02-23T00:56:22.394025+0000","last_became_peered":"2024-02-23T00:56:22.394025+0000","last_unstale":"2024-02-23T00:57:51.485033+0000","last_undegraded":"2024-02-23T00:57:51.485033+0000","last_fullsized":"2024-02-23T00:57:51.485033+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-23T00:57:18.247236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:57:18.247236+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":130,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:27.920726+0000","last_change":"2024-02-23T00:58:27.920726+0000","last_active":"2024-02-23T00:58:27.920726+0000","last_peered":"2024-02-23T00:58:27.920726+0000","last_clean":"2024-02-23T00:58:27.920726+0000","last_became_active":"2024-02-23T00:54:56.472343+0000","last_became_peered":"2024-02-23T00:54:56.472343+0000","last_unstale":"2024-02-23T00:58:27.920726+0000","last_undegraded":"2024-02-23T00:58:27.920726+0000","last_fullsized":"2024-02-23T00:58:27.920726+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:27.920657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:58:27.920657+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"844'16","reported_seq":115,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:23.058583+0000","last_change":"2024-02-23T00:58:08.756001+0000","last_active":"2024-02-23T00:58:23.058583+0000","last_peered":"2024-02-23T00:58:23.058583+0000","last_clean":"2024-02-23T00:58:23.058583+0000","last_became_active":"2024-02-23T00:57:13.551557+0000","last_became_peered":"2024-02-23T00:57:13.551557+0000","last_unstale":"2024-02-23T00:58:23.058583+0000","last_undegraded":"2024-02-23T00:58:23.058583+0000","last_fullsized":"2024-02-23T00:58:23.058583+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"844'16","last_scrub_stamp":"2024-02-23T00:58:08.755930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:58:08.755930+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":744,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:19.711057+0000","last_change":"2024-02-23T00:58:19.711023+0000","last_active":"2024-02-23T00:58:19.711057+0000","last_peered":"2024-02-23T00:58:19.711057+0000","last_clean":"2024-02-23T00:58:19.711057+0000","last_became_active":"2024-02-23T00:56:59.509047+0000","last_became_peered":"2024-02-23T00:56:59.509047+0000","last_unstale":"2024-02-23T00:58:19.711057+0000","last_undegraded":"2024-02-23T00:58:19.711057+0000","last_fullsized":"2024-02-23T00:58:19.711057+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:19.710960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:58:19.710960+0000","last_clean_scrub_stamp":"2024-02-23T00:58:19.710960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"823'19","reported_seq":553,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:10.761885+0000","last_change":"2024-02-23T00:58:10.761859+0000","last_active":"2024-02-23T00:58:10.761885+0000","last_peered":"2024-02-23T00:58:10.761885+0000","last_clean":"2024-02-23T00:58:10.761885+0000","last_became_active":"2024-02-23T00:56:59.506257+0000","last_became_peered":"2024-02-23T00:56:59.506257+0000","last_unstale":"2024-02-23T00:58:10.761885+0000","last_undegraded":"2024-02-23T00:58:10.761885+0000","last_fullsized":"2024-02-23T00:58:10.761885+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"823'19","last_scrub_stamp":"2024-02-23T00:58:10.761792+0000","last_deep_scrub":"823'19","last_deep_scrub_stamp":"2024-02-23T00:56:28.921013+0000","last_clean_scrub_stamp":"2024-02-23T00:58:10.761792+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":855,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.485013+0000","last_change":"2024-02-23T00:57:20.705702+0000","last_active":"2024-02-23T00:57:51.485013+0000","last_peered":"2024-02-23T00:57:51.485013+0000","last_clean":"2024-02-23T00:57:51.485013+0000","last_became_active":"2024-02-23T00:53:45.293336+0000","last_became_peered":"2024-02-23T00:53:45.293336+0000","last_unstale":"2024-02-23T00:57:51.485013+0000","last_undegraded":"2024-02-23T00:57:51.485013+0000","last_fullsized":"2024-02-23T00:57:51.485013+0000","mapping_epoch":685,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":686,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:20.705638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:57:20.705638+0000","last_clean_scrub_stamp":"2024-02-23T00:57:20.705638+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":699,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.479074+0000","last_change":"2024-02-23T00:57:22.059152+0000","last_active":"2024-02-23T00:57:51.479074+0000","last_peered":"2024-02-23T00:57:51.479074+0000","last_clean":"2024-02-23T00:57:51.479074+0000","last_became_active":"2024-02-23T00:57:13.552787+0000","last_became_peered":"2024-02-23T00:57:13.552787+0000","last_unstale":"2024-02-23T00:57:51.479074+0000","last_undegraded":"2024-02-23T00:57:51.479074+0000","last_fullsized":"2024-02-23T00:57:51.479074+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:22.059026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_clean_scrub_stamp":"2024-02-23T00:57:22.059026+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":130,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:26.897181+0000","last_change":"2024-02-23T00:58:26.897181+0000","last_active":"2024-02-23T00:58:26.897181+0000","last_peered":"2024-02-23T00:58:26.897181+0000","last_clean":"2024-02-23T00:58:26.897181+0000","last_became_active":"2024-02-23T00:54:56.487799+0000","last_became_peered":"2024-02-23T00:54:56.487799+0000","last_unstale":"2024-02-23T00:58:26.897181+0000","last_undegraded":"2024-02-23T00:58:26.897181+0000","last_fullsized":"2024-02-23T00:58:26.897181+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:26.897097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:58:26.897097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"457'19","reported_seq":874,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:28.326978+0000","last_change":"2024-02-23T00:58:28.326978+0000","last_active":"2024-02-23T00:58:28.326978+0000","last_peered":"2024-02-23T00:58:28.326978+0000","last_clean":"2024-02-23T00:58:28.326978+0000","last_became_active":"2024-02-23T00:39:16.783896+0000","last_became_peered":"2024-02-23T00:39:16.783896+0000","last_unstale":"2024-02-23T00:58:28.326978+0000","last_undegraded":"2024-02-23T00:58:28.326978+0000","last_fullsized":"2024-02-23T00:58:28.326978+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'19","last_scrub_stamp":"2024-02-23T00:58:28.326898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:58:28.326898+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"202'2","reported_seq":837,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.674351+0000","last_change":"2024-02-23T00:58:20.674351+0000","last_active":"2024-02-23T00:58:20.674351+0000","last_peered":"2024-02-23T00:58:20.674351+0000","last_clean":"2024-02-23T00:58:20.674351+0000","last_became_active":"2024-02-23T00:56:59.510938+0000","last_became_peered":"2024-02-23T00:56:59.510938+0000","last_unstale":"2024-02-23T00:58:20.674351+0000","last_undegraded":"2024-02-23T00:58:20.674351+0000","last_fullsized":"2024-02-23T00:58:20.674351+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"202'2","last_scrub_stamp":"2024-02-23T00:58:20.674239+0000","last_deep_scrub":"202'2","last_deep_scrub_stamp":"2024-02-23T00:58:20.674239+0000","last_clean_scrub_stamp":"2024-02-23T00:58:20.674239+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1746,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":89,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:08.530954+0000","last_change":"2024-02-23T00:58:08.530911+0000","last_active":"2024-02-23T00:58:08.530954+0000","last_peered":"2024-02-23T00:58:08.530954+0000","last_clean":"2024-02-23T00:58:08.530954+0000","last_became_active":"2024-02-23T00:55:45.325510+0000","last_became_peered":"2024-02-23T00:55:45.325510+0000","last_unstale":"2024-02-23T00:58:08.530954+0000","last_undegraded":"2024-02-23T00:58:08.530954+0000","last_fullsized":"2024-02-23T00:58:08.530954+0000","mapping_epoch":789,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":790,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:08.530798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:58:08.530798+0000","last_clean_scrub_stamp":"2024-02-23T00:58:08.530798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":762,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.479149+0000","last_change":"2024-02-23T00:57:48.864113+0000","last_active":"2024-02-23T00:57:51.479149+0000","last_peered":"2024-02-23T00:57:51.479149+0000","last_clean":"2024-02-23T00:57:51.479149+0000","last_became_active":"2024-02-23T00:56:59.506213+0000","last_became_peered":"2024-02-23T00:56:59.506213+0000","last_unstale":"2024-02-23T00:57:51.479149+0000","last_undegraded":"2024-02-23T00:57:51.479149+0000","last_fullsized":"2024-02-23T00:57:51.479149+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:48.864042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:46.462015+0000","last_clean_scrub_stamp":"2024-02-23T00:57:48.864042+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"457'8","reported_seq":511,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:56.796517+0000","last_change":"2024-02-23T00:57:56.796486+0000","last_active":"2024-02-23T00:57:56.796517+0000","last_peered":"2024-02-23T00:57:56.796517+0000","last_clean":"2024-02-23T00:57:56.796517+0000","last_became_active":"2024-02-23T00:57:13.552944+0000","last_became_peered":"2024-02-23T00:57:13.552944+0000","last_unstale":"2024-02-23T00:57:56.796517+0000","last_undegraded":"2024-02-23T00:57:56.796517+0000","last_fullsized":"2024-02-23T00:57:56.796517+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'8","last_scrub_stamp":"2024-02-23T00:57:56.796410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:57:56.796410+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":865,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:12.265426+0000","last_change":"2024-02-23T00:58:12.265386+0000","last_active":"2024-02-23T00:58:12.265426+0000","last_peered":"2024-02-23T00:58:12.265426+0000","last_clean":"2024-02-23T00:58:12.265426+0000","last_became_active":"2024-02-23T00:56:08.352993+0000","last_became_peered":"2024-02-23T00:56:08.352993+0000","last_unstale":"2024-02-23T00:58:12.265426+0000","last_undegraded":"2024-02-23T00:58:12.265426+0000","last_fullsized":"2024-02-23T00:58:12.265426+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:12.265304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:58:12.265304+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":869,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.485426+0000","last_change":"2024-02-23T00:57:43.655000+0000","last_active":"2024-02-23T00:57:51.485426+0000","last_peered":"2024-02-23T00:57:51.485426+0000","last_clean":"2024-02-23T00:57:51.485426+0000","last_became_active":"2024-02-23T00:56:22.385466+0000","last_became_peered":"2024-02-23T00:56:22.385466+0000","last_unstale":"2024-02-23T00:57:51.485426+0000","last_undegraded":"2024-02-23T00:57:51.485426+0000","last_fullsized":"2024-02-23T00:57:51.485426+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:43.654912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:57:43.654912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":60,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.485443+0000","last_change":"2024-02-23T00:57:33.287063+0000","last_active":"2024-02-23T00:57:51.485443+0000","last_peered":"2024-02-23T00:57:51.485443+0000","last_clean":"2024-02-23T00:57:51.485443+0000","last_became_active":"2024-02-23T00:56:22.400114+0000","last_became_peered":"2024-02-23T00:56:22.400114+0000","last_unstale":"2024-02-23T00:57:51.485443+0000","last_undegraded":"2024-02-23T00:57:51.485443+0000","last_fullsized":"2024-02-23T00:57:51.485443+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:33.286989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:57:33.286989+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"457'25","reported_seq":734,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.479136+0000","last_change":"2024-02-23T00:57:36.818233+0000","last_active":"2024-02-23T00:57:51.479136+0000","last_peered":"2024-02-23T00:57:51.479136+0000","last_clean":"2024-02-23T00:57:51.479136+0000","last_became_active":"2024-02-23T00:57:13.549408+0000","last_became_peered":"2024-02-23T00:57:13.549408+0000","last_unstale":"2024-02-23T00:57:51.479136+0000","last_undegraded":"2024-02-23T00:57:51.479136+0000","last_fullsized":"2024-02-23T00:57:51.479136+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-23T00:57:36.818139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:57:36.818139+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":864,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:23.609466+0000","last_change":"2024-02-23T00:58:23.609466+0000","last_active":"2024-02-23T00:58:23.609466+0000","last_peered":"2024-02-23T00:58:23.609466+0000","last_clean":"2024-02-23T00:58:23.609466+0000","last_became_active":"2024-02-23T00:56:08.365524+0000","last_became_peered":"2024-02-23T00:56:08.365524+0000","last_unstale":"2024-02-23T00:58:23.609466+0000","last_undegraded":"2024-02-23T00:58:23.609466+0000","last_fullsized":"2024-02-23T00:58:23.609466+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:23.609383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:53.583713+0000","last_clean_scrub_stamp":"2024-02-23T00:58:23.609383+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":60,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:53.280013+0000","last_change":"2024-02-23T00:57:53.279971+0000","last_active":"2024-02-23T00:57:53.280013+0000","last_peered":"2024-02-23T00:57:53.280013+0000","last_clean":"2024-02-23T00:57:53.280013+0000","last_became_active":"2024-02-23T00:56:22.389716+0000","last_became_peered":"2024-02-23T00:56:22.389716+0000","last_unstale":"2024-02-23T00:57:53.280013+0000","last_undegraded":"2024-02-23T00:57:53.280013+0000","last_fullsized":"2024-02-23T00:57:53.280013+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:53.279872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:57:53.279872+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":66,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:23.405558+0000","last_change":"2024-02-23T00:58:23.405558+0000","last_active":"2024-02-23T00:58:23.405558+0000","last_peered":"2024-02-23T00:58:23.405558+0000","last_clean":"2024-02-23T00:58:23.405558+0000","last_became_active":"2024-02-23T00:56:08.359732+0000","last_became_peered":"2024-02-23T00:56:08.359732+0000","last_unstale":"2024-02-23T00:58:23.405558+0000","last_undegraded":"2024-02-23T00:58:23.405558+0000","last_fullsized":"2024-02-23T00:58:23.405558+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:23.405491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:57:10.986967+0000","last_clean_scrub_stamp":"2024-02-23T00:58:23.405491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"457'26","reported_seq":190,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.484260+0000","last_change":"2024-02-23T00:57:34.179797+0000","last_active":"2024-02-23T00:57:51.484260+0000","last_peered":"2024-02-23T00:57:51.484260+0000","last_clean":"2024-02-23T00:57:51.484260+0000","last_became_active":"2024-02-23T00:56:22.396325+0000","last_became_peered":"2024-02-23T00:56:22.396325+0000","last_unstale":"2024-02-23T00:57:51.484260+0000","last_undegraded":"2024-02-23T00:57:51.484260+0000","last_fullsized":"2024-02-23T00:57:51.484260+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-23T00:57:34.179731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:57:34.179731+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":35,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":879,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:07.214104+0000","last_change":"2024-02-23T00:58:07.214072+0000","last_active":"2024-02-23T00:58:07.214104+0000","last_peered":"2024-02-23T00:58:07.214104+0000","last_clean":"2024-02-23T00:58:07.214104+0000","last_became_active":"2024-02-23T00:56:22.394165+0000","last_became_peered":"2024-02-23T00:56:22.394165+0000","last_unstale":"2024-02-23T00:58:07.214104+0000","last_undegraded":"2024-02-23T00:58:07.214104+0000","last_fullsized":"2024-02-23T00:58:07.214104+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"80'2","last_scrub_stamp":"2024-02-23T00:58:07.213992+0000","last_deep_scrub":"80'2","last_deep_scrub_stamp":"2024-02-23T00:40:15.942725+0000","last_clean_scrub_stamp":"2024-02-23T00:58:07.213992+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"818'12","reported_seq":1124,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:23.058759+0000","last_change":"2024-02-23T00:58:13.585298+0000","last_active":"2024-02-23T00:58:23.058759+0000","last_peered":"2024-02-23T00:58:23.058759+0000","last_clean":"2024-02-23T00:58:23.058759+0000","last_became_active":"2024-02-23T00:56:22.392240+0000","last_became_peered":"2024-02-23T00:56:22.392240+0000","last_unstale":"2024-02-23T00:58:23.058759+0000","last_undegraded":"2024-02-23T00:58:23.058759+0000","last_fullsized":"2024-02-23T00:58:23.058759+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"818'12","last_scrub_stamp":"2024-02-23T00:58:13.585184+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-02-23T00:43:50.393713+0000","last_clean_scrub_stamp":"2024-02-23T00:58:13.585184+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":139,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:00.282503+0000","last_change":"2024-02-23T00:58:00.282472+0000","last_active":"2024-02-23T00:58:00.282503+0000","last_peered":"2024-02-23T00:58:00.282503+0000","last_clean":"2024-02-23T00:58:00.282503+0000","last_became_active":"2024-02-23T00:56:08.346147+0000","last_became_peered":"2024-02-23T00:56:08.346147+0000","last_unstale":"2024-02-23T00:58:00.282503+0000","last_undegraded":"2024-02-23T00:58:00.282503+0000","last_fullsized":"2024-02-23T00:58:00.282503+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:00.282399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:58:00.282399+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"457'31","reported_seq":947,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:16.552693+0000","last_change":"2024-02-23T00:58:16.552651+0000","last_active":"2024-02-23T00:58:16.552693+0000","last_peered":"2024-02-23T00:58:16.552693+0000","last_clean":"2024-02-23T00:58:16.552693+0000","last_became_active":"2024-02-23T00:55:32.369643+0000","last_became_peered":"2024-02-23T00:55:32.369643+0000","last_unstale":"2024-02-23T00:58:16.552693+0000","last_undegraded":"2024-02-23T00:58:16.552693+0000","last_fullsized":"2024-02-23T00:58:16.552693+0000","mapping_epoch":780,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":781,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:58:16.552539+0000","last_deep_scrub":"457'31","last_deep_scrub_stamp":"2024-02-23T00:52:27.357369+0000","last_clean_scrub_stamp":"2024-02-23T00:58:16.552539+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":769,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:18.815702+0000","last_change":"2024-02-23T00:58:18.815672+0000","last_active":"2024-02-23T00:58:18.815702+0000","last_peered":"2024-02-23T00:58:18.815702+0000","last_clean":"2024-02-23T00:58:18.815702+0000","last_became_active":"2024-02-23T00:57:13.549830+0000","last_became_peered":"2024-02-23T00:57:13.549830+0000","last_unstale":"2024-02-23T00:58:18.815702+0000","last_undegraded":"2024-02-23T00:58:18.815702+0000","last_fullsized":"2024-02-23T00:58:18.815702+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:18.815597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:58:18.815597+0000","last_clean_scrub_stamp":"2024-02-23T00:58:18.815597+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":157,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:09.247292+0000","last_change":"2024-02-23T00:58:09.247265+0000","last_active":"2024-02-23T00:58:09.247292+0000","last_peered":"2024-02-23T00:58:09.247292+0000","last_clean":"2024-02-23T00:58:09.247292+0000","last_became_active":"2024-02-23T00:56:22.398137+0000","last_became_peered":"2024-02-23T00:56:22.398137+0000","last_unstale":"2024-02-23T00:58:09.247292+0000","last_undegraded":"2024-02-23T00:58:09.247292+0000","last_fullsized":"2024-02-23T00:58:09.247292+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:09.247200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:58:09.247200+0000","last_clean_scrub_stamp":"2024-02-23T00:58:09.247200+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":626,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.478818+0000","last_change":"2024-02-23T00:57:39.845591+0000","last_active":"2024-02-23T00:57:51.478818+0000","last_peered":"2024-02-23T00:57:51.478818+0000","last_clean":"2024-02-23T00:57:51.478818+0000","last_became_active":"2024-02-23T00:57:13.546252+0000","last_became_peered":"2024-02-23T00:57:13.546252+0000","last_unstale":"2024-02-23T00:57:51.478818+0000","last_undegraded":"2024-02-23T00:57:51.478818+0000","last_fullsized":"2024-02-23T00:57:51.478818+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:39.845522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:57:39.845522+0000","last_clean_scrub_stamp":"2024-02-23T00:57:39.845522+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"457'27","reported_seq":82,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.485314+0000","last_change":"2024-02-23T00:57:31.994247+0000","last_active":"2024-02-23T00:57:51.485314+0000","last_peered":"2024-02-23T00:57:51.485314+0000","last_clean":"2024-02-23T00:57:51.485314+0000","last_became_active":"2024-02-23T00:56:08.371957+0000","last_became_peered":"2024-02-23T00:56:08.371957+0000","last_unstale":"2024-02-23T00:57:51.485314+0000","last_undegraded":"2024-02-23T00:57:51.485314+0000","last_fullsized":"2024-02-23T00:57:51.485314+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-23T00:57:31.994106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:57:31.994106+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":868,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.485807+0000","last_change":"2024-02-23T00:57:12.770407+0000","last_active":"2024-02-23T00:57:51.485807+0000","last_peered":"2024-02-23T00:57:51.485807+0000","last_clean":"2024-02-23T00:57:51.485807+0000","last_became_active":"2024-02-23T00:56:22.385519+0000","last_became_peered":"2024-02-23T00:56:22.385519+0000","last_unstale":"2024-02-23T00:57:51.485807+0000","last_undegraded":"2024-02-23T00:57:51.485807+0000","last_fullsized":"2024-02-23T00:57:51.485807+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:12.770301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:57:12.770301+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":807,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.479406+0000","last_change":"2024-02-23T00:57:35.925082+0000","last_active":"2024-02-23T00:57:51.479406+0000","last_peered":"2024-02-23T00:57:51.479406+0000","last_clean":"2024-02-23T00:57:51.479406+0000","last_became_active":"2024-02-23T00:56:59.504814+0000","last_became_peered":"2024-02-23T00:56:59.504814+0000","last_unstale":"2024-02-23T00:57:51.479406+0000","last_undegraded":"2024-02-23T00:57:51.479406+0000","last_fullsized":"2024-02-23T00:57:51.479406+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:35.925001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.641689+0000","last_clean_scrub_stamp":"2024-02-23T00:57:35.925001+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":77,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.479593+0000","last_change":"2024-02-23T00:57:40.835745+0000","last_active":"2024-02-23T00:57:51.479593+0000","last_peered":"2024-02-23T00:57:51.479593+0000","last_clean":"2024-02-23T00:57:51.479593+0000","last_became_active":"2024-02-23T00:57:13.548458+0000","last_became_peered":"2024-02-23T00:57:13.548458+0000","last_unstale":"2024-02-23T00:57:51.479593+0000","last_undegraded":"2024-02-23T00:57:51.479593+0000","last_fullsized":"2024-02-23T00:57:51.479593+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:40.835672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:57:40.835672+0000","last_clean_scrub_stamp":"2024-02-23T00:57:40.835672+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"616'30","reported_seq":723,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.479915+0000","last_change":"2024-02-23T00:57:48.835733+0000","last_active":"2024-02-23T00:57:51.479915+0000","last_peered":"2024-02-23T00:57:51.479915+0000","last_clean":"2024-02-23T00:57:51.479915+0000","last_became_active":"2024-02-23T00:57:13.551428+0000","last_became_peered":"2024-02-23T00:57:13.551428+0000","last_unstale":"2024-02-23T00:57:51.479915+0000","last_undegraded":"2024-02-23T00:57:51.479915+0000","last_fullsized":"2024-02-23T00:57:51.479915+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"616'30","last_scrub_stamp":"2024-02-23T00:57:48.835663+0000","last_deep_scrub":"616'30","last_deep_scrub_stamp":"2024-02-23T00:52:25.764389+0000","last_clean_scrub_stamp":"2024-02-23T00:57:48.835663+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":46,"num_read_kb":36,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":74,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:02.715283+0000","last_change":"2024-02-23T00:58:02.715258+0000","last_active":"2024-02-23T00:58:02.715283+0000","last_peered":"2024-02-23T00:58:02.715283+0000","last_clean":"2024-02-23T00:58:02.715283+0000","last_became_active":"2024-02-23T00:57:13.551625+0000","last_became_peered":"2024-02-23T00:57:13.551625+0000","last_unstale":"2024-02-23T00:58:02.715283+0000","last_undegraded":"2024-02-23T00:58:02.715283+0000","last_fullsized":"2024-02-23T00:58:02.715283+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:02.715194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:58:02.715194+0000","last_clean_scrub_stamp":"2024-02-23T00:58:02.715194+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":209,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.484414+0000","last_change":"2024-02-23T00:57:49.674116+0000","last_active":"2024-02-23T00:57:51.484414+0000","last_peered":"2024-02-23T00:57:51.484414+0000","last_clean":"2024-02-23T00:57:51.484414+0000","last_became_active":"2024-02-23T00:56:08.364467+0000","last_became_peered":"2024-02-23T00:56:08.364467+0000","last_unstale":"2024-02-23T00:57:51.484414+0000","last_undegraded":"2024-02-23T00:57:51.484414+0000","last_fullsized":"2024-02-23T00:57:51.484414+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:49.674031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_clean_scrub_stamp":"2024-02-23T00:57:49.674031+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":807,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:26.133377+0000","last_change":"2024-02-23T00:58:26.133377+0000","last_active":"2024-02-23T00:58:26.133377+0000","last_peered":"2024-02-23T00:58:26.133377+0000","last_clean":"2024-02-23T00:58:26.133377+0000","last_became_active":"2024-02-23T00:56:08.352413+0000","last_became_peered":"2024-02-23T00:56:08.352413+0000","last_unstale":"2024-02-23T00:58:26.133377+0000","last_undegraded":"2024-02-23T00:58:26.133377+0000","last_fullsized":"2024-02-23T00:58:26.133377+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:26.133296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:58:26.133296+0000","last_clean_scrub_stamp":"2024-02-23T00:58:26.133296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"457'33","reported_seq":952,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:59.561753+0000","last_change":"2024-02-23T00:57:59.561707+0000","last_active":"2024-02-23T00:57:59.561753+0000","last_peered":"2024-02-23T00:57:59.561753+0000","last_clean":"2024-02-23T00:57:59.561753+0000","last_became_active":"2024-02-23T00:54:56.483435+0000","last_became_peered":"2024-02-23T00:54:56.483435+0000","last_unstale":"2024-02-23T00:57:59.561753+0000","last_undegraded":"2024-02-23T00:57:59.561753+0000","last_fullsized":"2024-02-23T00:57:59.561753+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'33","last_scrub_stamp":"2024-02-23T00:57:59.561623+0000","last_deep_scrub":"457'33","last_deep_scrub_stamp":"2024-02-23T00:56:44.520788+0000","last_clean_scrub_stamp":"2024-02-23T00:57:59.561623+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":60,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:04.250310+0000","last_change":"2024-02-23T00:58:04.250273+0000","last_active":"2024-02-23T00:58:04.250310+0000","last_peered":"2024-02-23T00:58:04.250310+0000","last_clean":"2024-02-23T00:58:04.250310+0000","last_became_active":"2024-02-23T00:56:22.405680+0000","last_became_peered":"2024-02-23T00:56:22.405680+0000","last_unstale":"2024-02-23T00:58:04.250310+0000","last_undegraded":"2024-02-23T00:58:04.250310+0000","last_fullsized":"2024-02-23T00:58:04.250310+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:04.250166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:58:04.250166+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":873,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.484753+0000","last_change":"2024-02-23T00:57:19.236285+0000","last_active":"2024-02-23T00:57:51.484753+0000","last_peered":"2024-02-23T00:57:51.484753+0000","last_clean":"2024-02-23T00:57:51.484753+0000","last_became_active":"2024-02-23T00:56:22.405449+0000","last_became_peered":"2024-02-23T00:56:22.405449+0000","last_unstale":"2024-02-23T00:57:51.484753+0000","last_undegraded":"2024-02-23T00:57:51.484753+0000","last_fullsized":"2024-02-23T00:57:51.484753+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:19.236197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:57:19.236197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":798,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.486529+0000","last_change":"2024-02-23T00:57:24.160282+0000","last_active":"2024-02-23T00:57:51.486529+0000","last_peered":"2024-02-23T00:57:51.486529+0000","last_clean":"2024-02-23T00:57:51.486529+0000","last_became_active":"2024-02-23T00:54:56.482357+0000","last_became_peered":"2024-02-23T00:54:56.482357+0000","last_unstale":"2024-02-23T00:57:51.486529+0000","last_undegraded":"2024-02-23T00:57:51.486529+0000","last_fullsized":"2024-02-23T00:57:51.486529+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:24.160212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:57:24.160212+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"457'17","reported_seq":877,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.483034+0000","last_change":"2024-02-23T00:57:44.395181+0000","last_active":"2024-02-23T00:57:51.483034+0000","last_peered":"2024-02-23T00:57:51.483034+0000","last_clean":"2024-02-23T00:57:51.483034+0000","last_became_active":"2024-02-23T00:39:16.782762+0000","last_became_peered":"2024-02-23T00:39:16.782762+0000","last_unstale":"2024-02-23T00:57:51.483034+0000","last_undegraded":"2024-02-23T00:57:51.483034+0000","last_fullsized":"2024-02-23T00:57:51.483034+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'17","last_scrub_stamp":"2024-02-23T00:57:44.395104+0000","last_deep_scrub":"82'15","last_deep_scrub_stamp":"2024-02-23T00:41:50.975683+0000","last_clean_scrub_stamp":"2024-02-23T00:57:44.395104+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":21,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":55,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.486794+0000","last_change":"2024-02-23T00:57:31.244236+0000","last_active":"2024-02-23T00:57:51.486794+0000","last_peered":"2024-02-23T00:57:51.486794+0000","last_clean":"2024-02-23T00:57:51.486794+0000","last_became_active":"2024-02-23T00:56:22.390249+0000","last_became_peered":"2024-02-23T00:56:22.390249+0000","last_unstale":"2024-02-23T00:57:51.486794+0000","last_undegraded":"2024-02-23T00:57:51.486794+0000","last_fullsized":"2024-02-23T00:57:51.486794+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:31.244160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:57:31.244160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":60,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:21.472617+0000","last_change":"2024-02-23T00:58:21.472617+0000","last_active":"2024-02-23T00:58:21.472617+0000","last_peered":"2024-02-23T00:58:21.472617+0000","last_clean":"2024-02-23T00:58:21.472617+0000","last_became_active":"2024-02-23T00:56:22.400716+0000","last_became_peered":"2024-02-23T00:56:22.400716+0000","last_unstale":"2024-02-23T00:58:21.472617+0000","last_undegraded":"2024-02-23T00:58:21.472617+0000","last_fullsized":"2024-02-23T00:58:21.472617+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:21.472522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:06.491793+0000","last_clean_scrub_stamp":"2024-02-23T00:58:21.472522+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":802,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.483315+0000","last_change":"2024-02-23T00:57:14.354767+0000","last_active":"2024-02-23T00:57:51.483315+0000","last_peered":"2024-02-23T00:57:51.483315+0000","last_clean":"2024-02-23T00:57:51.483315+0000","last_became_active":"2024-02-23T00:53:53.417585+0000","last_became_peered":"2024-02-23T00:53:53.417585+0000","last_unstale":"2024-02-23T00:57:51.483315+0000","last_undegraded":"2024-02-23T00:57:51.483315+0000","last_fullsized":"2024-02-23T00:57:51.483315+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:14.354702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:23.224663+0000","last_clean_scrub_stamp":"2024-02-23T00:57:14.354702+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"826'26","reported_seq":888,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:56.515586+0000","last_change":"2024-02-23T00:57:56.515550+0000","last_active":"2024-02-23T00:57:56.515586+0000","last_peered":"2024-02-23T00:57:56.515586+0000","last_clean":"2024-02-23T00:57:56.515586+0000","last_became_active":"2024-02-23T00:39:16.784606+0000","last_became_peered":"2024-02-23T00:39:16.784606+0000","last_unstale":"2024-02-23T00:57:56.515586+0000","last_undegraded":"2024-02-23T00:57:56.515586+0000","last_fullsized":"2024-02-23T00:57:56.515586+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"826'26","last_scrub_stamp":"2024-02-23T00:57:56.515465+0000","last_deep_scrub":"826'26","last_deep_scrub_stamp":"2024-02-23T00:57:56.515465+0000","last_clean_scrub_stamp":"2024-02-23T00:57:56.515465+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":696,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.481289+0000","last_change":"2024-02-23T00:57:19.139683+0000","last_active":"2024-02-23T00:57:51.481289+0000","last_peered":"2024-02-23T00:57:51.481289+0000","last_clean":"2024-02-23T00:57:51.481289+0000","last_became_active":"2024-02-23T00:57:13.546182+0000","last_became_peered":"2024-02-23T00:57:13.546182+0000","last_unstale":"2024-02-23T00:57:51.481289+0000","last_undegraded":"2024-02-23T00:57:51.481289+0000","last_fullsized":"2024-02-23T00:57:51.481289+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:19.139612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:55:27.856495+0000","last_clean_scrub_stamp":"2024-02-23T00:57:19.139612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":823,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.480166+0000","last_change":"2024-02-23T00:57:42.894678+0000","last_active":"2024-02-23T00:57:51.480166+0000","last_peered":"2024-02-23T00:57:51.480166+0000","last_clean":"2024-02-23T00:57:51.480166+0000","last_became_active":"2024-02-23T00:56:59.508932+0000","last_became_peered":"2024-02-23T00:56:59.508932+0000","last_unstale":"2024-02-23T00:57:51.480166+0000","last_undegraded":"2024-02-23T00:57:51.480166+0000","last_fullsized":"2024-02-23T00:57:51.480166+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:42.894617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:47.226331+0000","last_clean_scrub_stamp":"2024-02-23T00:57:42.894617+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":125,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.484508+0000","last_change":"2024-02-23T00:57:10.146060+0000","last_active":"2024-02-23T00:57:51.484508+0000","last_peered":"2024-02-23T00:57:51.484508+0000","last_clean":"2024-02-23T00:57:51.484508+0000","last_became_active":"2024-02-23T00:54:56.486558+0000","last_became_peered":"2024-02-23T00:54:56.486558+0000","last_unstale":"2024-02-23T00:57:51.484508+0000","last_undegraded":"2024-02-23T00:57:51.484508+0000","last_fullsized":"2024-02-23T00:57:51.484508+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:10.145991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:57:10.145991+0000","last_clean_scrub_stamp":"2024-02-23T00:57:10.145991+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"457'62","reported_seq":664,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.478420+0000","last_change":"2024-02-23T00:57:42.854820+0000","last_active":"2024-02-23T00:57:51.478420+0000","last_peered":"2024-02-23T00:57:51.478420+0000","last_clean":"2024-02-23T00:57:51.478420+0000","last_became_active":"2024-02-23T00:57:13.551475+0000","last_became_peered":"2024-02-23T00:57:13.551475+0000","last_unstale":"2024-02-23T00:57:51.478420+0000","last_undegraded":"2024-02-23T00:57:51.478420+0000","last_fullsized":"2024-02-23T00:57:51.478420+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'62","last_scrub_stamp":"2024-02-23T00:57:42.854752+0000","last_deep_scrub":"88'50","last_deep_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_clean_scrub_stamp":"2024-02-23T00:57:42.854752+0000","log_size":62,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":91,"num_read_kb":78,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":148,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.483942+0000","last_change":"2024-02-23T00:57:49.205317+0000","last_active":"2024-02-23T00:57:51.483942+0000","last_peered":"2024-02-23T00:57:51.483942+0000","last_clean":"2024-02-23T00:57:51.483942+0000","last_became_active":"2024-02-23T00:56:22.396132+0000","last_became_peered":"2024-02-23T00:56:22.396132+0000","last_unstale":"2024-02-23T00:57:51.483942+0000","last_undegraded":"2024-02-23T00:57:51.483942+0000","last_fullsized":"2024-02-23T00:57:51.483942+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:49.205251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_clean_scrub_stamp":"2024-02-23T00:57:49.205251+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":845,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:57:51.482508+0000","last_change":"2024-02-23T00:57:12.393446+0000","last_active":"2024-02-23T00:57:51.482508+0000","last_peered":"2024-02-23T00:57:51.482508+0000","last_clean":"2024-02-23T00:57:51.482508+0000","last_became_active":"2024-02-23T00:55:45.323003+0000","last_became_peered":"2024-02-23T00:55:45.323003+0000","last_unstale":"2024-02-23T00:57:51.482508+0000","last_undegraded":"2024-02-23T00:57:51.482508+0000","last_fullsized":"2024-02-23T00:57:51.482508+0000","mapping_epoch":789,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":790,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:12.393376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:57:12.393376+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":830,"reported_epoch":858,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.129867+0000","last_change":"2024-02-23T00:58:20.129837+0000","last_active":"2024-02-23T00:58:20.129867+0000","last_peered":"2024-02-23T00:58:20.129867+0000","last_clean":"2024-02-23T00:58:20.129867+0000","last_became_active":"2024-02-23T00:56:22.402412+0000","last_became_peered":"2024-02-23T00:56:22.402412+0000","last_unstale":"2024-02-23T00:58:20.129867+0000","last_undegraded":"2024-02-23T00:58:20.129867+0000","last_fullsized":"2024-02-23T00:58:20.129867+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:20.129758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:44.601086+0000","last_clean_scrub_stamp":"2024-02-23T00:58:20.129758+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0],"acting":[5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":12,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":36,"acting":36,"num_store_stats":8},{"poolid":5,"num_pg":42,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":126,"acting":126,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1498,"num_read_kb":1223,"num_write":895,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":422,"num_bytes_recovered":8258,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1010,"ondisk_log_size":1010,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1077,"num_read_kb":1072,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2830,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234220,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308080,"kb_used_data":2664,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93437072,"statfs":{"total":95995035648,"available":95679561728,"internally_reserved":0,"allocated":2727936,"data_stored":1078693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:57:45 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.53200000000000003,"15min":0.52100000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.23300000000000001},"max":{"1min":0.80600000000000005,"5min":0.82999999999999996,"15min":4.9420000000000002},"last":0.495},{"interface":"front","average":{"1min":0.504,"5min":0.52900000000000003,"15min":0.52400000000000002},"min":{"1min":0.35299999999999998,"5min":0.27200000000000002,"15min":0.24299999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":4.9729999999999999},"last":0.34899999999999998}]},{"osd":1,"last update":"Fri Feb 23 00:57:45 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.54000000000000004,"15min":0.60499999999999998},"min":{"1min":0.33200000000000002,"5min":0.315,"15min":0.27200000000000002},"max":{"1min":0.875,"5min":0.88200000000000001,"15min":10.786},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.55100000000000005,"15min":0.622},"min":{"1min":0.36099999999999999,"5min":0.33400000000000002,"15min":0.28100000000000003},"max":{"1min":0.84299999999999997,"5min":1.0609999999999999,"15min":11.885999999999999},"last":0.40200000000000002}]},{"osd":2,"last update":"Fri Feb 23 00:57:45 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53900000000000003,"15min":0.54800000000000004},"min":{"1min":0.375,"5min":0.33000000000000002,"15min":0.28999999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":5.0789999999999997},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.56699999999999995,"15min":0.55700000000000005},"min":{"1min":0.38800000000000001,"5min":0.33200000000000002,"15min":0.316},"max":{"1min":3.1339999999999999,"5min":3.1339999999999999,"15min":5.1280000000000001},"last":0.41199999999999998}]},{"osd":3,"last update":"Fri Feb 23 00:57:45 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56000000000000005,"15min":0.629},"min":{"1min":0.34999999999999998,"5min":0.33600000000000002,"15min":0.29399999999999998},"max":{"1min":0.88700000000000001,"5min":1.7030000000000001,"15min":15.712999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.58599999999999997,"15min":0.626},"min":{"1min":0.379,"5min":0.35599999999999998,"15min":0.28100000000000003},"max":{"1min":0.84799999999999998,"5min":1.8720000000000001,"15min":9.4090000000000007},"last":0.56899999999999995}]},{"osd":4,"last update":"Fri Feb 23 00:57:45 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.44600000000000001,"15min":0.441},"min":{"1min":0.29799999999999999,"5min":0.25800000000000001,"15min":0.23499999999999999},"max":{"1min":0.72499999999999998,"5min":0.79500000000000004,"15min":2.0350000000000001},"last":0.433},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.46200000000000002,"15min":0.48999999999999999},"min":{"1min":0.31900000000000001,"5min":0.26700000000000002,"15min":0.248},"max":{"1min":0.71099999999999997,"5min":0.85299999999999998,"15min":11.236000000000001},"last":0.46999999999999997}]},{"osd":5,"last update":"Fri Feb 23 00:57:45 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.47399999999999998,"15min":0.48599999999999999},"min":{"1min":0.29099999999999998,"5min":0.28799999999999998,"15min":0.27000000000000002},"max":{"1min":0.77000000000000002,"5min":0.83099999999999996,"15min":5.4000000000000004},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.496,"5min":0.5,"15min":0.502},"min":{"1min":0.34699999999999998,"5min":0.29199999999999998,"15min":0.29099999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":1.728},"last":0.54700000000000004}]},{"osd":6,"last update":"Fri Feb 23 00:57:45 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.36199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.77000000000000002,"5min":0.77200000000000002,"15min":1.6339999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.50800000000000001,"15min":0.54700000000000004},"min":{"1min":0.36199999999999999,"5min":0.316,"15min":0.29299999999999998},"max":{"1min":0.73999999999999999,"5min":0.82499999999999996,"15min":13.395},"last":0.55700000000000005}]}]},{"osd":6,"up_from":47,"seq":201863463155,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":303972,"kb_used_data":2652,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93441180,"statfs":{"total":95995035648,"available":95683768320,"internally_reserved":0,"allocated":2715648,"data_stored":1078417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:58:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.53600000000000003,"15min":0.67000000000000004},"min":{"1min":0.36699999999999999,"5min":0.27800000000000002,"15min":0.21099999999999999},"max":{"1min":0.73599999999999999,"5min":0.81799999999999995,"15min":30.641999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54000000000000004,"15min":0.69999999999999996},"min":{"1min":0.374,"5min":0.29899999999999999,"15min":0.23899999999999999},"max":{"1min":0.70999999999999996,"5min":1.268,"15min":29.292000000000002},"last":0.38700000000000001}]},{"osd":1,"last update":"Fri Feb 23 00:58:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52800000000000002,"15min":0.68600000000000005},"min":{"1min":0.41299999999999998,"5min":0.27800000000000002,"15min":0.247},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":28.186},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54000000000000004,"15min":0.873},"min":{"1min":0.41599999999999998,"5min":0.33700000000000002,"15min":0.26900000000000002},"max":{"1min":0.73499999999999999,"5min":1.2529999999999999,"15min":33.104999999999997},"last":0.48299999999999998}]},{"osd":2,"last update":"Fri Feb 23 00:58:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.72299999999999998},"min":{"1min":0.40799999999999997,"5min":0.34100000000000003,"15min":0.27400000000000002},"max":{"1min":0.78800000000000003,"5min":4.1319999999999997,"15min":19.960000000000001},"last":0.44},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53700000000000003,"15min":0.624},"min":{"1min":0.38200000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68600000000000005,"5min":1.161,"15min":8.5860000000000003},"last":0.503}]},{"osd":3,"last update":"Fri Feb 23 00:58:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55200000000000005,"15min":0.627},"min":{"1min":0.436,"5min":0.37,"15min":0.28599999999999998},"max":{"1min":0.66900000000000004,"5min":1.236,"15min":19.562000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56999999999999995,"15min":0.66500000000000004},"min":{"1min":0.40899999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.76100000000000001,"5min":2.7839999999999998,"15min":14.199},"last":0.51600000000000001}]},{"osd":4,"last update":"Fri Feb 23 00:58:20 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.44500000000000001,"15min":0.48599999999999999},"min":{"1min":0.30499999999999999,"5min":0.25800000000000001,"15min":0.218},"max":{"1min":0.59299999999999997,"5min":0.72299999999999998,"15min":11.109999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.47099999999999997,"15min":0.55100000000000005},"min":{"1min":0.32500000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.66500000000000004,"5min":0.72399999999999998,"15min":16.734000000000002},"last":0.55200000000000005}]},{"osd":5,"last update":"Fri Feb 23 00:58:20 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47399999999999998,"15min":0.52600000000000002},"min":{"1min":0.33800000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.63600000000000001,"5min":0.749,"15min":14.154999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.49299999999999999,"15min":0.56299999999999994},"min":{"1min":0.30599999999999999,"5min":0.27400000000000002,"15min":0.26400000000000001},"max":{"1min":0.71399999999999997,"5min":0.74099999999999999,"15min":8.1400000000000006},"last":0.45700000000000002}]},{"osd":7,"last update":"Fri Feb 23 00:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.505,"15min":0.53100000000000003},"min":{"1min":0.35599999999999998,"5min":0.27400000000000002,"15min":0.26800000000000002},"max":{"1min":0.78900000000000003,"5min":0.80200000000000005,"15min":8.548},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54300000000000004,"15min":0.61099999999999999},"min":{"1min":0.35699999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.76300000000000001,"5min":0.77800000000000002,"15min":15.618},"last":0.49199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575117,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":304000,"kb_used_data":2680,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93441152,"statfs":{"total":95995035648,"available":95683739648,"internally_reserved":0,"allocated":2744320,"data_stored":1080270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:58:05 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36599999999999999,"15min":0.69899999999999995},"min":{"1min":0.249,"5min":0.19500000000000001,"15min":0.17999999999999999},"max":{"1min":0.63500000000000001,"5min":0.68799999999999994,"15min":78.361000000000004},"last":0.314},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40000000000000002,"15min":0.45100000000000001},"min":{"1min":0.254,"5min":0.23699999999999999,"15min":0.216},"max":{"1min":0.64900000000000002,"5min":0.80600000000000005,"15min":5.8849999999999998},"last":0.371}]},{"osd":2,"last update":"Fri Feb 23 00:58:27 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.36899999999999999,"15min":0.67400000000000004},"min":{"1min":0.26400000000000001,"5min":0.20599999999999999,"15min":0.20499999999999999},"max":{"1min":0.44400000000000001,"5min":1.3580000000000001,"15min":48.606000000000002},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.41999999999999998,"15min":0.57699999999999996},"min":{"1min":0.27400000000000002,"5min":0.25700000000000001,"15min":0.23699999999999999},"max":{"1min":0.58299999999999996,"5min":0.95199999999999996,"15min":23.911000000000001},"last":0.42499999999999999}]},{"osd":3,"last update":"Fri Feb 23 00:58:05 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.435,"15min":0.58399999999999996},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.21199999999999999},"max":{"1min":0.73399999999999999,"5min":1.01,"15min":35.244999999999997},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.45600000000000002,"15min":0.69699999999999995},"min":{"1min":0.34000000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.3919999999999999,"5min":2.2349999999999999,"15min":56.853999999999999},"last":0.33300000000000002}]},{"osd":4,"last update":"Fri Feb 23 00:58:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56999999999999995,"15min":0.77900000000000003},"min":{"1min":0.41899999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80000000000000004,"5min":0.95699999999999996,"15min":37.646999999999998},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58899999999999997,"15min":0.70099999999999996},"min":{"1min":0.435,"5min":0.35899999999999999,"15min":0.308},"max":{"1min":0.80500000000000005,"5min":0.99299999999999999,"15min":9.6859999999999999},"last":0.71099999999999997}]},{"osd":5,"last update":"Fri Feb 23 00:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.81799999999999995},"min":{"1min":0.42199999999999999,"5min":0.35199999999999998,"15min":0.309},"max":{"1min":0.90800000000000003,"5min":0.94799999999999995,"15min":48.792000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.625,"15min":0.68500000000000005},"min":{"1min":0.45200000000000001,"5min":0.371,"15min":0.34100000000000003},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":11.872999999999999},"last":0.72399999999999998}]},{"osd":6,"last update":"Fri Feb 23 00:58:22 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57299999999999995,"15min":0.79700000000000004},"min":{"1min":0.45800000000000002,"5min":0.34200000000000003,"15min":0.318},"max":{"1min":0.69799999999999995,"5min":0.87,"15min":40.741},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60099999999999998,"15min":0.64200000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.34000000000000002},"max":{"1min":0.93100000000000005,"5min":1.0389999999999999,"15min":6.0069999999999997},"last":0.76600000000000001}]},{"osd":7,"last update":"Fri Feb 23 00:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.61599999999999999,"15min":0.70499999999999996},"min":{"1min":0.44700000000000001,"5min":0.41899999999999998,"15min":0.36699999999999999},"max":{"1min":1.0680000000000001,"5min":1.0780000000000001,"15min":20.225000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60599999999999998,"15min":0.75800000000000001},"min":{"1min":0.45300000000000001,"5min":0.38800000000000001,"15min":0.35999999999999999},"max":{"1min":0.86799999999999999,"5min":0.88300000000000001,"15min":20.254999999999999},"last":0.74399999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738643,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":303992,"kb_used_data":2672,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93441160,"statfs":{"total":95995035648,"available":95683747840,"internally_reserved":0,"allocated":2736128,"data_stored":1079287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 23 00:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.379,"15min":0.58699999999999997},"min":{"1min":0.223,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.53100000000000003,"5min":0.89300000000000002,"15min":28.856999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.42099999999999999,"15min":0.63700000000000001},"min":{"1min":0.26000000000000001,"5min":0.216,"15min":0.20699999999999999},"max":{"1min":0.72899999999999998,"5min":1.8140000000000001,"15min":20.577000000000002},"last":0.33700000000000002}]},{"osd":2,"last update":"Fri Feb 23 00:57:31 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.40999999999999998,"15min":0.61399999999999999},"min":{"1min":0.31900000000000001,"5min":0.252,"15min":0.222},"max":{"1min":0.66000000000000003,"5min":0.79600000000000004,"15min":13.863},"last":0.435},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.39600000000000002,"15min":0.54400000000000004},"min":{"1min":0.27400000000000002,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.47499999999999998,"5min":1.1619999999999999,"15min":10.455},"last":0.41499999999999998}]},{"osd":3,"last update":"Fri Feb 23 00:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41299999999999998,"15min":0.73199999999999998},"min":{"1min":0.29699999999999999,"5min":0.23799999999999999,"15min":0.22700000000000001},"max":{"1min":0.66200000000000003,"5min":0.73099999999999998,"15min":35.566000000000003},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.44500000000000001,"15min":0.72299999999999998},"min":{"1min":0.34699999999999998,"5min":0.28100000000000003,"15min":0.26100000000000001},"max":{"1min":0.67400000000000004,"5min":0.80100000000000005,"15min":20.702999999999999},"last":0.38600000000000001}]},{"osd":4,"last update":"Fri Feb 23 00:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.63100000000000001,"15min":0.71099999999999997},"min":{"1min":0.34300000000000003,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.746,"5min":2.2320000000000002,"15min":10.584},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.65900000000000003,"15min":0.76300000000000001},"min":{"1min":0.45600000000000002,"5min":0.36899999999999999,"15min":0.35099999999999998},"max":{"1min":0.86699999999999999,"5min":2.2090000000000001,"15min":13.198},"last":0.64500000000000002}]},{"osd":5,"last update":"Fri Feb 23 00:57:52 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.63,"15min":0.73099999999999998},"min":{"1min":0.39600000000000002,"5min":0.33800000000000002,"15min":0.32200000000000001},"max":{"1min":0.71799999999999997,"5min":2.1150000000000002,"15min":10.397},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.68700000000000006,"15min":0.79200000000000004},"min":{"1min":0.41899999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84299999999999997,"5min":2.2509999999999999,"15min":17.297000000000001},"last":0.65900000000000003}]},{"osd":6,"last update":"Fri Feb 23 00:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65200000000000002,"15min":0.751},"min":{"1min":0.38,"5min":0.38,"15min":0.378},"max":{"1min":0.89000000000000001,"5min":1.2549999999999999,"15min":17.373999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.70799999999999996,"15min":0.78600000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.36799999999999999},"max":{"1min":0.93700000000000006,"5min":4.4859999999999998,"15min":13.973000000000001},"last":0.68700000000000006}]},{"osd":7,"last update":"Fri Feb 23 00:57:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68300000000000005,"15min":0.78900000000000003},"min":{"1min":0.432,"5min":0.38200000000000001,"15min":0.35199999999999998},"max":{"1min":0.88500000000000001,"5min":2.8260000000000001,"15min":17.399999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.749,"15min":0.82199999999999995},"min":{"1min":0.48299999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81299999999999994,"5min":6.0229999999999997,"15min":16.427},"last":0.71699999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411592,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":303976,"kb_used_data":2656,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93441176,"statfs":{"total":95995035648,"available":95683764224,"internally_reserved":0,"allocated":2719744,"data_stored":1078409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:57:27 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.44600000000000001,"15min":0.504},"min":{"1min":0.26700000000000002,"5min":0.22500000000000001,"15min":0.19700000000000001},"max":{"1min":0.47299999999999998,"5min":5.0439999999999996,"15min":13.657},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.39800000000000002,"15min":0.67800000000000005},"min":{"1min":0.27900000000000003,"5min":0.254,"15min":0.222},"max":{"1min":0.53800000000000003,"5min":1.117,"15min":42.343000000000004},"last":0.35199999999999998}]},{"osd":1,"last update":"Fri Feb 23 00:57:27 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.40100000000000002,"15min":0.57699999999999996},"min":{"1min":0.27900000000000003,"5min":0.22600000000000001,"15min":0.19700000000000001},"max":{"1min":0.52900000000000003,"5min":0.70999999999999996,"15min":18.952000000000002},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.45000000000000001,"15min":0.53300000000000003},"min":{"1min":0.29799999999999999,"5min":0.24199999999999999,"15min":0.23999999999999999},"max":{"1min":0.61099999999999999,"5min":4.9980000000000002,"15min":15.391},"last":0.33700000000000002}]},{"osd":3,"last update":"Fri Feb 23 00:58:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.44700000000000001,"15min":0.64900000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.22500000000000001},"max":{"1min":0.504,"5min":2.762,"15min":25.635999999999999},"last":0.316},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.438,"15min":0.61799999999999999},"min":{"1min":0.29899999999999999,"5min":0.28000000000000003,"15min":0.26300000000000001},"max":{"1min":0.65300000000000002,"5min":1.534,"15min":19.960000000000001},"last":0.32600000000000001}]},{"osd":4,"last update":"Fri Feb 23 00:57:27 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.61599999999999999,"15min":0.73199999999999998},"min":{"1min":0.42199999999999999,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.81899999999999995,"5min":1.224,"15min":13.673},"last":0.496},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.63300000000000001,"15min":0.754},"min":{"1min":0.436,"5min":0.315,"15min":0.315},"max":{"1min":0.83299999999999996,"5min":0.97999999999999998,"15min":10.103999999999999},"last":0.53100000000000003}]},{"osd":5,"last update":"Fri Feb 23 00:57:47 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.61699999999999999,"15min":0.69299999999999995},"min":{"1min":0.42899999999999999,"5min":0.34699999999999998,"15min":0.29899999999999999},"max":{"1min":0.85999999999999999,"5min":1.5389999999999999,"15min":15.612},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.65300000000000002,"15min":0.70299999999999996},"min":{"1min":0.41699999999999998,"5min":0.38100000000000001,"15min":0.35099999999999998},"max":{"1min":0.875,"5min":1.8839999999999999,"15min":13.632999999999999},"last":0.45000000000000001}]},{"osd":6,"last update":"Fri Feb 23 00:58:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.67200000000000004,"15min":0.68300000000000005},"min":{"1min":0.48299999999999998,"5min":0.39900000000000002,"15min":0.39000000000000001},"max":{"1min":0.81899999999999995,"5min":3.2360000000000002,"15min":11.544},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.65100000000000002,"15min":0.76300000000000001},"min":{"1min":0.46100000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81399999999999995,"5min":1.319,"15min":10.132999999999999},"last":0.44500000000000001}]},{"osd":7,"last update":"Fri Feb 23 00:57:39 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67000000000000004,"15min":0.71899999999999997},"min":{"1min":0.48499999999999999,"5min":0.40400000000000003,"15min":0.377},"max":{"1min":0.91600000000000004,"5min":1.9239999999999999,"15min":11.599},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.64600000000000002,"15min":0.78700000000000003},"min":{"1min":0.44400000000000001,"5min":0.41699999999999998,"15min":0.374},"max":{"1min":0.86499999999999999,"5min":1.1679999999999999,"15min":10.148999999999999},"last":0.50900000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149953,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308108,"kb_used_data":2692,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93437044,"statfs":{"total":95995035648,"available":95679533056,"internally_reserved":0,"allocated":2756608,"data_stored":1079274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:58:04 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.35699999999999998,"15min":0.51300000000000001},"min":{"1min":0.252,"5min":0.21199999999999999,"15min":0.20899999999999999},"max":{"1min":0.62,"5min":0.63,"15min":12.212999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.40699999999999997,"15min":0.59099999999999997},"min":{"1min":0.245,"5min":0.245,"15min":0.21199999999999999},"max":{"1min":0.53100000000000003,"5min":1.3089999999999999,"15min":11.714},"last":0.28100000000000003}]},{"osd":1,"last update":"Fri Feb 23 00:58:04 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.39800000000000002,"15min":0.60699999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.19900000000000001},"max":{"1min":0.60699999999999998,"5min":1.1539999999999999,"15min":11.742000000000001},"last":0.318},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41499999999999998,"15min":0.58999999999999997},"min":{"1min":0.309,"5min":0.218,"15min":0.19900000000000001},"max":{"1min":0.60999999999999999,"5min":1.7470000000000001,"15min":8.2270000000000003},"last":0.30399999999999999}]},{"osd":2,"last update":"Fri Feb 23 00:58:04 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.437,"15min":0.66400000000000003},"min":{"1min":0.27300000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.60399999999999998,"5min":1.2430000000000001,"15min":13.359999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.41299999999999998,"15min":0.64400000000000002},"min":{"1min":0.28999999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.60799999999999998,"5min":0.82599999999999996,"15min":21.122},"last":0.42199999999999999}]},{"osd":4,"last update":"Fri Feb 23 00:58:27 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.58999999999999997,"15min":0.745},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.29899999999999999},"max":{"1min":0.92400000000000004,"5min":2.4780000000000002,"15min":11.644},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59799999999999998,"15min":0.74199999999999999},"min":{"1min":0.40699999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.95399999999999996,"5min":2.4119999999999999,"15min":12.430999999999999},"last":0.70899999999999996}]},{"osd":5,"last update":"Fri Feb 23 00:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59399999999999997,"15min":0.68700000000000006},"min":{"1min":0.441,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.874,"5min":2.4540000000000002,"15min":10.231999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.628,"15min":0.80200000000000005},"min":{"1min":0.45300000000000001,"5min":0.36799999999999999,"15min":0.35499999999999998},"max":{"1min":0.99199999999999999,"5min":2.3679999999999999,"15min":11.648},"last":0.69899999999999995}]},{"osd":6,"last update":"Fri Feb 23 00:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.628,"15min":0.749},"min":{"1min":0.443,"5min":0.377,"15min":0.35199999999999998},"max":{"1min":0.85999999999999999,"5min":2.464,"15min":6.9420000000000002},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.629,"15min":0.72199999999999998},"min":{"1min":0.41699999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.94499999999999995,"5min":2.4249999999999998,"15min":10.965999999999999},"last":0.60799999999999998}]},{"osd":7,"last update":"Fri Feb 23 00:57:43 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66100000000000003,"15min":0.83999999999999997},"min":{"1min":0.49099999999999999,"5min":0.42999999999999999,"15min":0.38800000000000001},"max":{"1min":0.96099999999999997,"5min":2.444,"15min":11.676},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.66400000000000003,"15min":0.81599999999999995},"min":{"1min":0.433,"5min":0.41499999999999998,"15min":0.41299999999999998},"max":{"1min":0.83799999999999997,"5min":2.5009999999999999,"15min":7.9800000000000004},"last":0.73499999999999999}]}]},{"osd":4,"up_from":32,"seq":137438953725,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":303996,"kb_used_data":2676,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93441156,"statfs":{"total":95995035648,"available":95683743744,"internally_reserved":0,"allocated":2740224,"data_stored":1078918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:58:22 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.501,"15min":0.504},"min":{"1min":0.30199999999999999,"5min":0.25800000000000001,"15min":0.223},"max":{"1min":0.623,"5min":0.79400000000000004,"15min":3.5219999999999998},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55400000000000005,"15min":0.57199999999999995},"min":{"1min":0.372,"5min":0.32200000000000001,"15min":0.23999999999999999},"max":{"1min":0.67100000000000004,"5min":2.5859999999999999,"15min":8.6140000000000008},"last":0.44600000000000001}]},{"osd":1,"last update":"Fri Feb 23 00:58:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51000000000000001,"15min":0.56899999999999995},"min":{"1min":0.317,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.69699999999999995,"5min":1.23,"15min":9.6110000000000007},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53100000000000003,"15min":0.56899999999999995},"min":{"1min":0.41799999999999998,"5min":0.308,"15min":0.255},"max":{"1min":0.76700000000000002,"5min":0.79600000000000004,"15min":6.6950000000000003},"last":0.41599999999999998}]},{"osd":2,"last update":"Fri Feb 23 00:58:22 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51900000000000002,"15min":0.54900000000000004},"min":{"1min":0.34200000000000003,"5min":0.30099999999999999,"15min":0.26500000000000001},"max":{"1min":0.73599999999999999,"5min":0.78600000000000003,"15min":3.3069999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54600000000000004,"15min":0.56100000000000005},"min":{"1min":0.36399999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.70899999999999996,"5min":1.6779999999999999,"15min":5.8440000000000003},"last":0.54100000000000004}]},{"osd":3,"last update":"Fri Feb 23 00:58:22 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.57099999999999995,"15min":0.628},"min":{"1min":0.35099999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.74099999999999999,"5min":2.6150000000000002,"15min":9.1280000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54100000000000004,"15min":0.57199999999999995},"min":{"1min":0.38900000000000001,"5min":0.315,"15min":0.29699999999999999},"max":{"1min":0.68999999999999995,"5min":1.288,"15min":4.0220000000000002},"last":0.55300000000000005}]},{"osd":5,"last update":"Fri Feb 23 00:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46400000000000002,"15min":0.442},"min":{"1min":0.25600000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.73399999999999999,"5min":1.627,"15min":1.694},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48499999999999999,"15min":0.48899999999999999},"min":{"1min":0.33200000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.69199999999999995,"5min":0.84699999999999998,"15min":1.663},"last":0.52400000000000002}]},{"osd":6,"last update":"Fri Feb 23 00:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.46600000000000003,"15min":0.442},"min":{"1min":0.28399999999999997,"5min":0.28100000000000003,"15min":0.253},"max":{"1min":0.77300000000000002,"5min":1.704,"15min":1.704},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.317,"5min":0.26500000000000001,"15min":0.215},"max":{"1min":0.76700000000000002,"5min":1.264,"15min":2.2120000000000002},"last":0.58399999999999996}]},{"osd":7,"last update":"Fri Feb 23 00:57:43 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.48699999999999999,"15min":0.47999999999999998},"min":{"1min":0.33500000000000002,"5min":0.27100000000000002,"15min":0.26700000000000002},"max":{"1min":0.69199999999999995,"5min":1.363,"15min":2.294},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56000000000000005,"15min":0.54700000000000004},"min":{"1min":0.38200000000000001,"5min":0.33000000000000002,"15min":0.32600000000000001},"max":{"1min":0.89900000000000002,"5min":1.726,"15min":2.282},"last":0.60699999999999998}]}]},{"osd":5,"up_from":39,"seq":167503724792,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":303996,"kb_used_data":2676,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93441156,"statfs":{"total":95995035648,"available":95683743744,"internally_reserved":0,"allocated":2740224,"data_stored":1080047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:57:39 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.55500000000000005,"15min":0.63600000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.246},"max":{"1min":0.89100000000000001,"5min":5.6920000000000002,"15min":23.881},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.5,"5min":0.50800000000000001,"15min":0.59099999999999997},"min":{"1min":0.27200000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.79900000000000004,"5min":2.4300000000000002,"15min":5.4649999999999999},"last":0.54000000000000004}]},{"osd":1,"last update":"Fri Feb 23 00:57:39 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.52000000000000002,"15min":0.65800000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.254},"max":{"1min":0.95799999999999996,"5min":1.659,"15min":18.175999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51500000000000001,"15min":0.625},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.23899999999999999},"max":{"1min":0.72999999999999998,"5min":1.3080000000000001,"15min":9.0950000000000006},"last":0.59999999999999998}]},{"osd":2,"last update":"Fri Feb 23 00:57:39 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.55000000000000004,"15min":0.71499999999999997},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.75600000000000001,"5min":4.492,"15min":22.282},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56999999999999995,"15min":0.73999999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.29199999999999998},"max":{"1min":0.91500000000000004,"5min":2.7389999999999999,"15min":23.925000000000001},"last":0.67700000000000005}]},{"osd":3,"last update":"Fri Feb 23 00:57:39 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.53400000000000003,"15min":0.626},"min":{"1min":0.36699999999999999,"5min":0.34300000000000003,"15min":0.30399999999999999},"max":{"1min":1.446,"5min":1.446,"15min":6.5730000000000004},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.58899999999999997,"15min":0.66900000000000004},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.93700000000000006,"5min":2.8090000000000002,"15min":8.1760000000000002},"last":0.65200000000000002}]},{"osd":4,"last update":"Fri Feb 23 00:57:39 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.443,"15min":0.44900000000000001},"min":{"1min":0.23100000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.24,"5min":1.24,"15min":4.593},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.47599999999999998,"15min":0.49099999999999999},"min":{"1min":0.28399999999999997,"5min":0.255,"15min":0.255},"max":{"1min":0.65500000000000003,"5min":1.744,"15min":3.879},"last":0.47599999999999998}]},{"osd":6,"last update":"Fri Feb 23 00:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48599999999999999,"15min":0.49199999999999999},"min":{"1min":0.32000000000000001,"5min":0.28499999999999998,"15min":0.27900000000000003},"max":{"1min":0.69699999999999995,"5min":1.5389999999999999,"15min":4.6740000000000004},"last":0.628},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52400000000000002,"15min":0.498},"min":{"1min":0.36899999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.71899999999999997,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.69999999999999996}]},{"osd":7,"last update":"Fri Feb 23 00:57:39 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54800000000000004,"15min":0.55300000000000005},"min":{"1min":0.38600000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.74099999999999999,"5min":1.756,"15min":7.0300000000000002},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.52300000000000002,"15min":0.54000000000000004},"min":{"1min":0.40400000000000003,"5min":0.314,"15min":0.29099999999999998},"max":{"1min":2.1920000000000002,"5min":2.1920000000000002,"15min":9.0579999999999998},"last":0.64100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-23T00:58:35.156 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph pg dump --format=json 2024-02-23T00:58:35.462 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:58:35.483 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:58:34.333335+0000 mgr.y (mgr.14517) 773 : audit [DBG] from='client.32139 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:58:35.758 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:35 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:58:34.333335+0000 mgr.y (mgr.14517) 773 : audit [DBG] from='client.32139 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:58:35.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:58:34.333335+0000 mgr.y (mgr.14517) 773 : audit [DBG] from='client.32139 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:58:36.766 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:34.900826+0000 osd.0 (osd.0) 372 : cluster [DBG] 18.e deep-scrub starts 2024-02-23T00:58:36.766 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:34.902189+0000 osd.0 (osd.0) 373 : cluster [DBG] 18.e deep-scrub ok 2024-02-23T00:58:36.766 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:35.387266+0000 osd.4 (osd.4) 413 : cluster [DBG] 4.1 scrub starts 2024-02-23T00:58:36.766 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:35.388956+0000 osd.4 (osd.4) 414 : cluster [DBG] 4.1 scrub ok 2024-02-23T00:58:36.766 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:35.448237+0000 mgr.y (mgr.14517) 774 : cluster [DBG] pgmap v1357: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:58:36.767 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:35.654523+0000 osd.6 (osd.6) 393 : cluster [DBG] 2.1a scrub starts 2024-02-23T00:58:36.767 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:35.655431+0000 osd.6 (osd.6) 394 : cluster [DBG] 2.1a scrub ok 2024-02-23T00:58:36.767 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:34.900826+0000 osd.0 (osd.0) 372 : cluster [DBG] 18.e deep-scrub starts 2024-02-23T00:58:36.767 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:34.902189+0000 osd.0 (osd.0) 373 : cluster [DBG] 18.e deep-scrub ok 2024-02-23T00:58:36.767 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:35.387266+0000 osd.4 (osd.4) 413 : cluster [DBG] 4.1 scrub starts 2024-02-23T00:58:36.767 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:35.388956+0000 osd.4 (osd.4) 414 : cluster [DBG] 4.1 scrub ok 2024-02-23T00:58:36.767 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:35.448237+0000 mgr.y (mgr.14517) 774 : cluster [DBG] pgmap v1357: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:58:36.767 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:35.654523+0000 osd.6 (osd.6) 393 : cluster [DBG] 2.1a scrub starts 2024-02-23T00:58:36.767 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:36 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:35.655431+0000 osd.6 (osd.6) 394 : cluster [DBG] 2.1a scrub ok 2024-02-23T00:58:36.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:34.900826+0000 osd.0 (osd.0) 372 : cluster [DBG] 18.e deep-scrub starts 2024-02-23T00:58:36.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:34.902189+0000 osd.0 (osd.0) 373 : cluster [DBG] 18.e deep-scrub ok 2024-02-23T00:58:36.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:35.387266+0000 osd.4 (osd.4) 413 : cluster [DBG] 4.1 scrub starts 2024-02-23T00:58:36.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:35.388956+0000 osd.4 (osd.4) 414 : cluster [DBG] 4.1 scrub ok 2024-02-23T00:58:36.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:35.448237+0000 mgr.y (mgr.14517) 774 : cluster [DBG] pgmap v1357: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:58:36.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:35.654523+0000 osd.6 (osd.6) 393 : cluster [DBG] 2.1a scrub starts 2024-02-23T00:58:36.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:35.655431+0000 osd.6 (osd.6) 394 : cluster [DBG] 2.1a scrub ok 2024-02-23T00:58:37.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.336+0000 7f0370729700 1 -- 172.21.15.47:0/2739452465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03680fe530 msgr2=0x7f03680f63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:37.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.336+0000 7f0370729700 1 --2- 172.21.15.47:0/2739452465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03680fe530 0x7f03680f63b0 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 rx=0x7f03640047b0 tx=0x7f0364009cb0).stop 2024-02-23T00:58:37.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.336+0000 7f0370729700 1 -- 172.21.15.47:0/2739452465 shutdown_connections 2024-02-23T00:58:37.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.336+0000 7f0370729700 1 --2- 172.21.15.47:0/2739452465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03680f6a90 0x7f0368104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:37.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.336+0000 7f0370729700 1 --2- 172.21.15.47:0/2739452465 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f03680fdbd0 0x7f03680fdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:37.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.336+0000 7f0370729700 1 --2- 172.21.15.47:0/2739452465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03680fe530 0x7f03680f63b0 unknown :-1 s=CLOSED pgs=2692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:37.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.336+0000 7f0370729700 1 -- 172.21.15.47:0/2739452465 >> 172.21.15.47:0/2739452465 conn(0x7f036806c370 msgr2=0x7f036806c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:58:37.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.336+0000 7f0370729700 1 -- 172.21.15.47:0/2739452465 shutdown_connections 2024-02-23T00:58:37.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.336+0000 7f0370729700 1 -- 172.21.15.47:0/2739452465 wait complete. 2024-02-23T00:58:37.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.336+0000 7f0370729700 1 librados: starting msgr at 2024-02-23T00:58:37.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.336+0000 7f0370729700 1 librados: starting objecter 2024-02-23T00:58:37.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.336+0000 7f0370729700 1 Processor -- start 2024-02-23T00:58:37.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.336+0000 7f0370729700 1 -- start start 2024-02-23T00:58:37.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.336+0000 7f0370729700 1 librados: setting wanted keys 2024-02-23T00:58:37.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.336+0000 7f0370729700 1 librados: calling monclient init 2024-02-23T00:58:37.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.337+0000 7f0370729700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03680f6a90 0x7f036810e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:37.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.337+0000 7f0370729700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f03680fdbd0 0x7f036810e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:37.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.337+0000 7f0370729700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03680fe530 0x7f0368108200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:37.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.337+0000 7f0370729700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03681086f0 con 0x7f03680f6a90 2024-02-23T00:58:37.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.337+0000 7f0370729700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0368108860 con 0x7f03680fe530 2024-02-23T00:58:37.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.337+0000 7f0370729700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0368108a20 con 0x7f03680fdbd0 2024-02-23T00:58:37.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.337+0000 7f036e4c5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03680f6a90 0x7f036810e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:37.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.337+0000 7f036e4c5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03680f6a90 0x7f036810e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54112/0 (socket says 172.21.15.47:54112) 2024-02-23T00:58:37.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.337+0000 7f036e4c5700 1 -- 172.21.15.47:0/354475956 learned_addr learned my addr 172.21.15.47:0/354475956 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:58:37.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.337+0000 7f036dcc4700 1 --2- 172.21.15.47:0/354475956 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f03680fdbd0 0x7f036810e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:37.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.337+0000 7f036ecc6700 1 --2- 172.21.15.47:0/354475956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03680fe530 0x7f0368108200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:37.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.337+0000 7f036e4c5700 1 -- 172.21.15.47:0/354475956 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f03680fdbd0 msgr2=0x7f036810e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:37.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.337+0000 7f036e4c5700 1 --2- 172.21.15.47:0/354475956 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f03680fdbd0 0x7f036810e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:37.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.337+0000 7f036e4c5700 1 -- 172.21.15.47:0/354475956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03680fe530 msgr2=0x7f0368108200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:37.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.337+0000 7f036e4c5700 1 --2- 172.21.15.47:0/354475956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03680fe530 0x7f0368108200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:37.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.337+0000 7f036e4c5700 1 -- 172.21.15.47:0/354475956 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f035c005040 con 0x7f03680f6a90 2024-02-23T00:58:37.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.337+0000 7f036ecc6700 1 --2- 172.21.15.47:0/354475956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03680fe530 0x7f0368108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:58:37.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.337+0000 7f036e4c5700 1 --2- 172.21.15.47:0/354475956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03680f6a90 0x7f036810e060 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 rx=0x7f035c009400 tx=0x7f035c009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:58:37.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.337+0000 7f035b7fe700 10 client.?.objecter ms_handle_connect 0x7f03680f6a90 2024-02-23T00:58:37.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.337+0000 7f035b7fe700 10 client.?.objecter resend_mon_ops 2024-02-23T00:58:37.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.337+0000 7f035b7fe700 1 -- 172.21.15.47:0/354475956 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c007a40 con 0x7f03680f6a90 2024-02-23T00:58:37.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.337+0000 7f035b7fe700 1 -- 172.21.15.47:0/354475956 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f035c00f040 con 0x7f03680f6a90 2024-02-23T00:58:37.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.337+0000 7f0370729700 1 -- 172.21.15.47:0/354475956 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0364005040 con 0x7f03680f6a90 2024-02-23T00:58:37.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.338+0000 7f035b7fe700 1 -- 172.21.15.47:0/354475956 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c014990 con 0x7f03680f6a90 2024-02-23T00:58:37.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.338+0000 7f0370729700 10 client.32145.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:58:37.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.338+0000 7f0370729700 1 -- 172.21.15.47:0/354475956 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0368109060 con 0x7f03680f6a90 2024-02-23T00:58:37.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.338+0000 7f0370729700 1 librados: init done 2024-02-23T00:58:37.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.339+0000 7f0370729700 1 -- 172.21.15.47:0/354475956 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f036805f950 con 0x7f03680f6a90 2024-02-23T00:58:37.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.339+0000 7f035b7fe700 1 -- 172.21.15.47:0/354475956 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f035c00e630 con 0x7f03680f6a90 2024-02-23T00:58:37.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.339+0000 7f035b7fe700 1 --2- 172.21.15.47:0/354475956 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0354062cc0 0x7f0354065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:37.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.340+0000 7f036dcc4700 1 --2- 172.21.15.47:0/354475956 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0354062cc0 0x7f0354065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:37.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.340+0000 7f035b7fe700 1 -- 172.21.15.47:0/354475956 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(860..860 src has 806..860) v4 ==== 8663+0+0 (secure 0 0 0) 0x7f035c07d130 con 0x7f03680f6a90 2024-02-23T00:58:37.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.340+0000 7f035b7fe700 10 client.32145.objecter ms_dispatch 0x7f0368001550 osd_map(860..860 src has 806..860) v4 2024-02-23T00:58:37.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.340+0000 7f035b7fe700 3 client.32145.objecter handle_osd_map got epochs [860,860] > 0 2024-02-23T00:58:37.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.340+0000 7f035b7fe700 3 client.32145.objecter handle_osd_map decoding full epoch 860 2024-02-23T00:58:37.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.340+0000 7f035b7fe700 20 client.32145.objecter dump_active .. 0 homeless 2024-02-23T00:58:37.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.340+0000 7f035b7fe700 10 client.32145.objecter ms_handle_connect 0x7f0354062cc0 2024-02-23T00:58:37.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.340+0000 7f036dcc4700 1 --2- 172.21.15.47:0/354475956 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0354062cc0 0x7f0354065180 secure :-1 s=READY pgs=3261 cs=0 l=1 rev1=1 rx=0x7f0364009bc0 tx=0x7f0364009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:58:37.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.344+0000 7f035b7fe700 1 -- 172.21.15.47:0/354475956 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f035c049f60 con 0x7f03680f6a90 2024-02-23T00:58:37.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.515+0000 7f0370729700 1 -- 172.21.15.47:0/354475956 --> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f03681096e0 con 0x7f0354062cc0 2024-02-23T00:58:37.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.527+0000 7f035b7fe700 1 -- 172.21.15.47:0/354475956 <== mgr.14517 v2:172.21.15.47:6800/1636898748 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+409916 (secure 0 0 0) 0x7f03681096e0 con 0x7f0354062cc0 2024-02-23T00:58:37.536 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-02-23T00:58:37.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.530+0000 7f0370729700 10 librados: watch_flush enter 2024-02-23T00:58:37.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.530+0000 7f0370729700 10 librados: watch_flush exit 2024-02-23T00:58:37.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.530+0000 7f0370729700 20 client.32145.objecter shutdown clearing up homeless session... 2024-02-23T00:58:37.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.530+0000 7f0370729700 10 client.32145.objecter successfully canceled tick 2024-02-23T00:58:37.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.531+0000 7f0370729700 1 -- 172.21.15.47:0/354475956 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0354062cc0 msgr2=0x7f0354065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:37.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.531+0000 7f0370729700 1 --2- 172.21.15.47:0/354475956 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0354062cc0 0x7f0354065180 secure :-1 s=READY pgs=3261 cs=0 l=1 rev1=1 rx=0x7f0364009bc0 tx=0x7f0364009cb0).stop 2024-02-23T00:58:37.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.531+0000 7f0370729700 1 -- 172.21.15.47:0/354475956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03680f6a90 msgr2=0x7f036810e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:37.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.531+0000 7f0370729700 1 --2- 172.21.15.47:0/354475956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03680f6a90 0x7f036810e060 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 rx=0x7f035c009400 tx=0x7f035c009cb0).stop 2024-02-23T00:58:37.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.531+0000 7f0370729700 1 -- 172.21.15.47:0/354475956 shutdown_connections 2024-02-23T00:58:37.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.531+0000 7f0370729700 1 --2- 172.21.15.47:0/354475956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03680fe530 0x7f0368108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:37.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.531+0000 7f0370729700 1 --2- 172.21.15.47:0/354475956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03680f6a90 0x7f036810e060 unknown :-1 s=CLOSED pgs=2693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:37.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.531+0000 7f0370729700 1 --2- 172.21.15.47:0/354475956 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7f0354062cc0 0x7f0354065180 unknown :-1 s=CLOSED pgs=3261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:37.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.531+0000 7f0370729700 1 --2- 172.21.15.47:0/354475956 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f03680fdbd0 0x7f036810e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:37.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.531+0000 7f0370729700 1 -- 172.21.15.47:0/354475956 >> 172.21.15.47:0/354475956 conn(0x7f036806c370 msgr2=0x7f03680fba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:58:37.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.531+0000 7f0370729700 1 -- 172.21.15.47:0/354475956 shutdown_connections 2024-02-23T00:58:37.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.531+0000 7f0370729700 1 -- 172.21.15.47:0/354475956 wait complete. 2024-02-23T00:58:37.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:37.531+0000 7f0370729700 1 librados: shutdown 2024-02-23T00:58:37.540 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-02-23T00:58:37.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:36.679385+0000 osd.1 (osd.1) 372 : cluster [DBG] 5.27 scrub starts 2024-02-23T00:58:37.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:36.680666+0000 osd.1 (osd.1) 373 : cluster [DBG] 5.27 scrub ok 2024-02-23T00:58:37.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:36.679385+0000 osd.1 (osd.1) 372 : cluster [DBG] 5.27 scrub starts 2024-02-23T00:58:37.808 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:37 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:36.680666+0000 osd.1 (osd.1) 373 : cluster [DBG] 5.27 scrub ok 2024-02-23T00:58:37.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:36.679385+0000 osd.1 (osd.1) 372 : cluster [DBG] 5.27 scrub starts 2024-02-23T00:58:37.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:36.680666+0000 osd.1 (osd.1) 373 : cluster [DBG] 5.27 scrub ok 2024-02-23T00:58:38.272 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":1358,"stamp":"2024-02-23T00:58:37.448399+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5801,"num_objects":225,"num_object_clones":0,"num_object_copies":675,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":225,"num_whiteouts":0,"num_read":2608,"num_read_kb":2328,"num_write":913,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":458,"num_bytes_recovered":13770,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1167,"ondisk_log_size":1167,"up":462,"acting":462,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":462,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2438808,"kb_used_data":17656,"kb_used_omap":0,"kb_used_meta":2418688,"kb_avail":747522408,"statfs":{"total":767960285184,"available":765462945792,"internally_reserved":0,"allocated":18079744,"data_stored":6227467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2476736512},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002519"},"pg_stats":[{"pgid":"5.29","version":"466'9","reported_seq":129,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.672477+0000","last_change":"2024-02-23T00:57:28.058623+0000","last_active":"2024-02-23T00:58:20.672477+0000","last_peered":"2024-02-23T00:58:20.672477+0000","last_clean":"2024-02-23T00:58:20.672477+0000","last_became_active":"2024-02-23T00:56:22.399258+0000","last_became_peered":"2024-02-23T00:56:22.399258+0000","last_unstale":"2024-02-23T00:58:20.672477+0000","last_undegraded":"2024-02-23T00:58:20.672477+0000","last_fullsized":"2024-02-23T00:58:20.672477+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":769,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":6,"last_scrub":"466'9","last_scrub_stamp":"2024-02-23T00:57:28.058561+0000","last_deep_scrub":"466'9","last_deep_scrub_stamp":"2024-02-23T00:57:28.058561+0000","last_clean_scrub_stamp":"2024-02-23T00:57:28.058561+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":149,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:31.205757+0000","last_change":"2024-02-23T00:58:31.205726+0000","last_active":"2024-02-23T00:58:31.205757+0000","last_peered":"2024-02-23T00:58:31.205757+0000","last_clean":"2024-02-23T00:58:31.205757+0000","last_became_active":"2024-02-23T00:56:08.352060+0000","last_became_peered":"2024-02-23T00:56:08.352060+0000","last_unstale":"2024-02-23T00:58:31.205757+0000","last_undegraded":"2024-02-23T00:58:31.205757+0000","last_fullsized":"2024-02-23T00:58:31.205757+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":769,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:31.205647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_clean_scrub_stamp":"2024-02-23T00:58:31.205647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":847,"reported_epoch":860,"state":"active+clean","last_fresh":"2024-02-23T00:58:36.680750+0000","last_change":"2024-02-23T00:58:36.680750+0000","last_active":"2024-02-23T00:58:36.680750+0000","last_peered":"2024-02-23T00:58:36.680750+0000","last_clean":"2024-02-23T00:58:36.680750+0000","last_became_active":"2024-02-23T00:56:59.507959+0000","last_became_peered":"2024-02-23T00:56:59.507959+0000","last_unstale":"2024-02-23T00:58:36.680750+0000","last_undegraded":"2024-02-23T00:58:36.680750+0000","last_fullsized":"2024-02-23T00:58:36.680750+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":769,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:36.680670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:59.694033+0000","last_clean_scrub_stamp":"2024-02-23T00:58:36.680670+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":82,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.436760+0000","last_change":"2024-02-23T00:57:28.009719+0000","last_active":"2024-02-23T00:58:20.436760+0000","last_peered":"2024-02-23T00:58:20.436760+0000","last_clean":"2024-02-23T00:58:20.436760+0000","last_became_active":"2024-02-23T00:57:13.547938+0000","last_became_peered":"2024-02-23T00:57:13.547938+0000","last_unstale":"2024-02-23T00:58:20.436760+0000","last_undegraded":"2024-02-23T00:58:20.436760+0000","last_fullsized":"2024-02-23T00:58:20.436760+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":769,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:28.009637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.264240+0000","last_clean_scrub_stamp":"2024-02-23T00:57:28.009637+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":173,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.440380+0000","last_change":"2024-02-23T00:57:17.820201+0000","last_active":"2024-02-23T00:58:20.440380+0000","last_peered":"2024-02-23T00:58:20.440380+0000","last_clean":"2024-02-23T00:58:20.440380+0000","last_became_active":"2024-02-23T00:56:59.506206+0000","last_became_peered":"2024-02-23T00:56:59.506206+0000","last_unstale":"2024-02-23T00:58:20.440380+0000","last_undegraded":"2024-02-23T00:58:20.440380+0000","last_fullsized":"2024-02-23T00:58:20.440380+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":769,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:17.820129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:57:17.820129+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":874,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.615725+0000","last_change":"2024-02-23T00:58:10.261254+0000","last_active":"2024-02-23T00:58:20.615725+0000","last_peered":"2024-02-23T00:58:20.615725+0000","last_clean":"2024-02-23T00:58:20.615725+0000","last_became_active":"2024-02-23T00:55:21.382086+0000","last_became_peered":"2024-02-23T00:55:21.382086+0000","last_unstale":"2024-02-23T00:58:20.615725+0000","last_undegraded":"2024-02-23T00:58:20.615725+0000","last_fullsized":"2024-02-23T00:58:20.615725+0000","mapping_epoch":771,"log_start":"0'0","ondisk_log_start":"0'0","created":769,"last_epoch_clean":772,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:10.261117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:58:10.261117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"813'14","reported_seq":165,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:33.060872+0000","last_change":"2024-02-23T00:58:05.459592+0000","last_active":"2024-02-23T00:58:33.060872+0000","last_peered":"2024-02-23T00:58:33.060872+0000","last_clean":"2024-02-23T00:58:33.060872+0000","last_became_active":"2024-02-23T00:56:08.337720+0000","last_became_peered":"2024-02-23T00:56:08.337720+0000","last_unstale":"2024-02-23T00:58:33.060872+0000","last_undegraded":"2024-02-23T00:58:33.060872+0000","last_fullsized":"2024-02-23T00:58:33.060872+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":769,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":6,"last_scrub":"813'14","last_scrub_stamp":"2024-02-23T00:58:05.459517+0000","last_deep_scrub":"91'9","last_deep_scrub_stamp":"2024-02-23T00:41:12.843733+0000","last_clean_scrub_stamp":"2024-02-23T00:58:05.459517+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":63,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.568483+0000","last_change":"2024-02-23T00:58:03.254267+0000","last_active":"2024-02-23T00:58:20.568483+0000","last_peered":"2024-02-23T00:58:20.568483+0000","last_clean":"2024-02-23T00:58:20.568483+0000","last_became_active":"2024-02-23T00:56:22.393231+0000","last_became_peered":"2024-02-23T00:56:22.393231+0000","last_unstale":"2024-02-23T00:58:20.568483+0000","last_undegraded":"2024-02-23T00:58:20.568483+0000","last_fullsized":"2024-02-23T00:58:20.568483+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":769,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:03.254156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:58:03.254156+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":111,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.565445+0000","last_change":"2024-02-23T00:57:09.751484+0000","last_active":"2024-02-23T00:58:20.565445+0000","last_peered":"2024-02-23T00:58:20.565445+0000","last_clean":"2024-02-23T00:58:20.565445+0000","last_became_active":"2024-02-23T00:56:08.364399+0000","last_became_peered":"2024-02-23T00:56:08.364399+0000","last_unstale":"2024-02-23T00:58:20.565445+0000","last_undegraded":"2024-02-23T00:58:20.565445+0000","last_fullsized":"2024-02-23T00:58:20.565445+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":769,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:09.751392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:10.493340+0000","last_clean_scrub_stamp":"2024-02-23T00:57:09.751392+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"818'15","reported_seq":92,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:33.060799+0000","last_change":"2024-02-23T00:58:12.314312+0000","last_active":"2024-02-23T00:58:33.060799+0000","last_peered":"2024-02-23T00:58:33.060799+0000","last_clean":"2024-02-23T00:58:33.060799+0000","last_became_active":"2024-02-23T00:56:22.403485+0000","last_became_peered":"2024-02-23T00:56:22.403485+0000","last_unstale":"2024-02-23T00:58:33.060799+0000","last_undegraded":"2024-02-23T00:58:33.060799+0000","last_fullsized":"2024-02-23T00:58:33.060799+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":769,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":6,"last_scrub":"818'15","last_scrub_stamp":"2024-02-23T00:58:12.314229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:58:12.314229+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"818'12","reported_seq":92,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:33.060875+0000","last_change":"2024-02-23T00:58:11.291419+0000","last_active":"2024-02-23T00:58:33.060875+0000","last_peered":"2024-02-23T00:58:33.060875+0000","last_clean":"2024-02-23T00:58:33.060875+0000","last_became_active":"2024-02-23T00:56:22.389524+0000","last_became_peered":"2024-02-23T00:56:22.389524+0000","last_unstale":"2024-02-23T00:58:33.060875+0000","last_undegraded":"2024-02-23T00:58:33.060875+0000","last_fullsized":"2024-02-23T00:58:33.060875+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"818'12","last_scrub_stamp":"2024-02-23T00:58:11.291344+0000","last_deep_scrub":"818'12","last_deep_scrub_stamp":"2024-02-23T00:56:49.953643+0000","last_clean_scrub_stamp":"2024-02-23T00:58:11.291344+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"779'48","reported_seq":978,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.615487+0000","last_change":"2024-02-23T00:57:14.209043+0000","last_active":"2024-02-23T00:58:20.615487+0000","last_peered":"2024-02-23T00:58:20.615487+0000","last_clean":"2024-02-23T00:58:20.615487+0000","last_became_active":"2024-02-23T00:56:22.394274+0000","last_became_peered":"2024-02-23T00:56:22.394274+0000","last_unstale":"2024-02-23T00:58:20.615487+0000","last_undegraded":"2024-02-23T00:58:20.615487+0000","last_fullsized":"2024-02-23T00:58:20.615487+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"779'48","last_scrub_stamp":"2024-02-23T00:57:14.208974+0000","last_deep_scrub":"779'48","last_deep_scrub_stamp":"2024-02-23T00:55:31.242584+0000","last_clean_scrub_stamp":"2024-02-23T00:57:14.208974+0000","log_size":48,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":76,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.715113+0000","last_change":"2024-02-23T00:57:47.082744+0000","last_active":"2024-02-23T00:58:20.715113+0000","last_peered":"2024-02-23T00:58:20.715113+0000","last_clean":"2024-02-23T00:58:20.715113+0000","last_became_active":"2024-02-23T00:56:22.396348+0000","last_became_peered":"2024-02-23T00:56:22.396348+0000","last_unstale":"2024-02-23T00:58:20.715113+0000","last_undegraded":"2024-02-23T00:58:20.715113+0000","last_fullsized":"2024-02-23T00:58:20.715113+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:47.082679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:04.330426+0000","last_clean_scrub_stamp":"2024-02-23T00:57:47.082679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":870,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.615640+0000","last_change":"2024-02-23T00:57:21.257122+0000","last_active":"2024-02-23T00:58:20.615640+0000","last_peered":"2024-02-23T00:58:20.615640+0000","last_clean":"2024-02-23T00:58:20.615640+0000","last_became_active":"2024-02-23T00:54:56.482910+0000","last_became_peered":"2024-02-23T00:54:56.482910+0000","last_unstale":"2024-02-23T00:58:20.615640+0000","last_undegraded":"2024-02-23T00:58:20.615640+0000","last_fullsized":"2024-02-23T00:58:20.615640+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:21.257053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:57:21.257053+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":799,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.616203+0000","last_change":"2024-02-23T00:57:43.167765+0000","last_active":"2024-02-23T00:58:20.616203+0000","last_peered":"2024-02-23T00:58:20.616203+0000","last_clean":"2024-02-23T00:58:20.616203+0000","last_became_active":"2024-02-23T00:54:56.486869+0000","last_became_peered":"2024-02-23T00:54:56.486869+0000","last_unstale":"2024-02-23T00:58:20.616203+0000","last_undegraded":"2024-02-23T00:58:20.616203+0000","last_fullsized":"2024-02-23T00:58:20.616203+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:43.167650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:57:43.167650+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"846'26","reported_seq":148,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.671735+0000","last_change":"2024-02-23T00:57:21.095323+0000","last_active":"2024-02-23T00:58:20.671735+0000","last_peered":"2024-02-23T00:58:20.671735+0000","last_clean":"2024-02-23T00:58:20.671735+0000","last_became_active":"2024-02-23T00:54:56.481365+0000","last_became_peered":"2024-02-23T00:54:56.481365+0000","last_unstale":"2024-02-23T00:58:20.671735+0000","last_undegraded":"2024-02-23T00:58:20.671735+0000","last_fullsized":"2024-02-23T00:58:20.671735+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"846'26","last_scrub_stamp":"2024-02-23T00:57:21.095257+0000","last_deep_scrub":"846'26","last_deep_scrub_stamp":"2024-02-23T00:57:21.095257+0000","last_clean_scrub_stamp":"2024-02-23T00:57:21.095257+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":37,"num_read_kb":30,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":794,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.440843+0000","last_change":"2024-02-23T00:57:25.845138+0000","last_active":"2024-02-23T00:58:20.440843+0000","last_peered":"2024-02-23T00:58:20.440843+0000","last_clean":"2024-02-23T00:58:20.440843+0000","last_became_active":"2024-02-23T00:56:59.501902+0000","last_became_peered":"2024-02-23T00:56:59.501902+0000","last_unstale":"2024-02-23T00:58:20.440843+0000","last_undegraded":"2024-02-23T00:58:20.440843+0000","last_fullsized":"2024-02-23T00:58:20.440843+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:25.845070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:04.528006+0000","last_clean_scrub_stamp":"2024-02-23T00:57:25.845070+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":67,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.714671+0000","last_change":"2024-02-23T00:57:56.140991+0000","last_active":"2024-02-23T00:58:20.714671+0000","last_peered":"2024-02-23T00:58:20.714671+0000","last_clean":"2024-02-23T00:58:20.714671+0000","last_became_active":"2024-02-23T00:56:08.362289+0000","last_became_peered":"2024-02-23T00:56:08.362289+0000","last_unstale":"2024-02-23T00:58:20.714671+0000","last_undegraded":"2024-02-23T00:58:20.714671+0000","last_fullsized":"2024-02-23T00:58:20.714671+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:56.140899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:56:50.919809+0000","last_clean_scrub_stamp":"2024-02-23T00:57:56.140899+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":158,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.672652+0000","last_change":"2024-02-23T00:57:57.269495+0000","last_active":"2024-02-23T00:58:20.672652+0000","last_peered":"2024-02-23T00:58:20.672652+0000","last_clean":"2024-02-23T00:58:20.672652+0000","last_became_active":"2024-02-23T00:56:22.398741+0000","last_became_peered":"2024-02-23T00:56:22.398741+0000","last_unstale":"2024-02-23T00:58:20.672652+0000","last_undegraded":"2024-02-23T00:58:20.672652+0000","last_fullsized":"2024-02-23T00:58:20.672652+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:57.269424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:13.867716+0000","last_clean_scrub_stamp":"2024-02-23T00:57:57.269424+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":149,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.671965+0000","last_change":"2024-02-23T00:57:52.210660+0000","last_active":"2024-02-23T00:58:20.671965+0000","last_peered":"2024-02-23T00:58:20.671965+0000","last_clean":"2024-02-23T00:58:20.671965+0000","last_became_active":"2024-02-23T00:56:22.386236+0000","last_became_peered":"2024-02-23T00:56:22.386236+0000","last_unstale":"2024-02-23T00:58:20.671965+0000","last_undegraded":"2024-02-23T00:58:20.671965+0000","last_fullsized":"2024-02-23T00:58:20.671965+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:52.210576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:57:52.210576+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"457'51","reported_seq":987,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.567048+0000","last_change":"2024-02-23T00:58:20.567022+0000","last_active":"2024-02-23T00:58:20.567048+0000","last_peered":"2024-02-23T00:58:20.567048+0000","last_clean":"2024-02-23T00:58:20.567048+0000","last_became_active":"2024-02-23T00:56:22.391880+0000","last_became_peered":"2024-02-23T00:56:22.391880+0000","last_unstale":"2024-02-23T00:58:20.567048+0000","last_undegraded":"2024-02-23T00:58:20.567048+0000","last_fullsized":"2024-02-23T00:58:20.567048+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'51","last_scrub_stamp":"2024-02-23T00:58:20.566957+0000","last_deep_scrub":"88'35","last_deep_scrub_stamp":"2024-02-23T00:40:54.152239+0000","last_clean_scrub_stamp":"2024-02-23T00:58:20.566957+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":68,"num_read_kb":56,"num_write":42,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":67,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.714847+0000","last_change":"2024-02-23T00:57:36.017206+0000","last_active":"2024-02-23T00:58:20.714847+0000","last_peered":"2024-02-23T00:58:20.714847+0000","last_clean":"2024-02-23T00:58:20.714847+0000","last_became_active":"2024-02-23T00:56:08.351610+0000","last_became_peered":"2024-02-23T00:56:08.351610+0000","last_unstale":"2024-02-23T00:58:20.714847+0000","last_undegraded":"2024-02-23T00:58:20.714847+0000","last_fullsized":"2024-02-23T00:58:20.714847+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:36.017116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:28.450377+0000","last_clean_scrub_stamp":"2024-02-23T00:57:36.017116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":67,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.714965+0000","last_change":"2024-02-23T00:57:57.128526+0000","last_active":"2024-02-23T00:58:20.714965+0000","last_peered":"2024-02-23T00:58:20.714965+0000","last_clean":"2024-02-23T00:58:20.714965+0000","last_became_active":"2024-02-23T00:56:08.351215+0000","last_became_peered":"2024-02-23T00:56:08.351215+0000","last_unstale":"2024-02-23T00:58:20.714965+0000","last_undegraded":"2024-02-23T00:58:20.714965+0000","last_fullsized":"2024-02-23T00:58:20.714965+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:57.128460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:56:31.214495+0000","last_clean_scrub_stamp":"2024-02-23T00:57:57.128460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":61,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.568105+0000","last_change":"2024-02-23T00:57:58.222488+0000","last_active":"2024-02-23T00:58:20.568105+0000","last_peered":"2024-02-23T00:58:20.568105+0000","last_clean":"2024-02-23T00:58:20.568105+0000","last_became_active":"2024-02-23T00:56:22.404580+0000","last_became_peered":"2024-02-23T00:56:22.404580+0000","last_unstale":"2024-02-23T00:58:20.568105+0000","last_undegraded":"2024-02-23T00:58:20.568105+0000","last_fullsized":"2024-02-23T00:58:20.568105+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:58.222416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:57:58.222416+0000","last_clean_scrub_stamp":"2024-02-23T00:57:58.222416+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":789,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.441082+0000","last_change":"2024-02-23T00:58:09.804852+0000","last_active":"2024-02-23T00:58:20.441082+0000","last_peered":"2024-02-23T00:58:20.441082+0000","last_clean":"2024-02-23T00:58:20.441082+0000","last_became_active":"2024-02-23T00:56:59.509516+0000","last_became_peered":"2024-02-23T00:56:59.509516+0000","last_unstale":"2024-02-23T00:58:20.441082+0000","last_undegraded":"2024-02-23T00:58:20.441082+0000","last_fullsized":"2024-02-23T00:58:20.441082+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:09.804777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:53:07.482987+0000","last_clean_scrub_stamp":"2024-02-23T00:58:09.804777+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"853'39","reported_seq":977,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.441369+0000","last_change":"2024-02-23T00:57:37.326961+0000","last_active":"2024-02-23T00:58:20.441369+0000","last_peered":"2024-02-23T00:58:20.441369+0000","last_clean":"2024-02-23T00:58:20.441369+0000","last_became_active":"2024-02-23T00:56:08.363684+0000","last_became_peered":"2024-02-23T00:56:08.363684+0000","last_unstale":"2024-02-23T00:58:20.441369+0000","last_undegraded":"2024-02-23T00:58:20.441369+0000","last_fullsized":"2024-02-23T00:58:20.441369+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"853'39","last_scrub_stamp":"2024-02-23T00:57:37.326874+0000","last_deep_scrub":"853'39","last_deep_scrub_stamp":"2024-02-23T00:57:37.326874+0000","last_clean_scrub_stamp":"2024-02-23T00:57:37.326874+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":61,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:31.612161+0000","last_change":"2024-02-23T00:58:31.612130+0000","last_active":"2024-02-23T00:58:31.612161+0000","last_peered":"2024-02-23T00:58:31.612161+0000","last_clean":"2024-02-23T00:58:31.612161+0000","last_became_active":"2024-02-23T00:56:22.393096+0000","last_became_peered":"2024-02-23T00:56:22.393096+0000","last_unstale":"2024-02-23T00:58:31.612161+0000","last_undegraded":"2024-02-23T00:58:31.612161+0000","last_fullsized":"2024-02-23T00:58:31.612161+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:31.612049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:58:31.612049+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":813,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.441312+0000","last_change":"2024-02-23T00:58:14.778413+0000","last_active":"2024-02-23T00:58:20.441312+0000","last_peered":"2024-02-23T00:58:20.441312+0000","last_clean":"2024-02-23T00:58:20.441312+0000","last_became_active":"2024-02-23T00:56:59.504894+0000","last_became_peered":"2024-02-23T00:56:59.504894+0000","last_unstale":"2024-02-23T00:58:20.441312+0000","last_undegraded":"2024-02-23T00:58:20.441312+0000","last_fullsized":"2024-02-23T00:58:20.441312+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:14.778293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:58:14.778293+0000","last_clean_scrub_stamp":"2024-02-23T00:58:14.778293+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":736,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.441241+0000","last_change":"2024-02-23T00:57:20.875046+0000","last_active":"2024-02-23T00:58:20.441241+0000","last_peered":"2024-02-23T00:58:20.441241+0000","last_clean":"2024-02-23T00:58:20.441241+0000","last_became_active":"2024-02-23T00:56:59.502268+0000","last_became_peered":"2024-02-23T00:56:59.502268+0000","last_unstale":"2024-02-23T00:58:20.441241+0000","last_undegraded":"2024-02-23T00:58:20.441241+0000","last_fullsized":"2024-02-23T00:58:20.441241+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:20.874942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:57:20.874942+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"457'20","reported_seq":874,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:25.342661+0000","last_change":"2024-02-23T00:58:25.342628+0000","last_active":"2024-02-23T00:58:25.342661+0000","last_peered":"2024-02-23T00:58:25.342661+0000","last_clean":"2024-02-23T00:58:25.342661+0000","last_became_active":"2024-02-23T00:39:16.777530+0000","last_became_peered":"2024-02-23T00:39:16.777530+0000","last_unstale":"2024-02-23T00:58:25.342661+0000","last_undegraded":"2024-02-23T00:58:25.342661+0000","last_fullsized":"2024-02-23T00:58:25.342661+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'20","last_scrub_stamp":"2024-02-23T00:58:25.342546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:58:25.342546+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":803,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.440851+0000","last_change":"2024-02-23T00:57:31.932103+0000","last_active":"2024-02-23T00:58:20.440851+0000","last_peered":"2024-02-23T00:58:20.440851+0000","last_clean":"2024-02-23T00:58:20.440851+0000","last_became_active":"2024-02-23T00:56:59.508854+0000","last_became_peered":"2024-02-23T00:56:59.508854+0000","last_unstale":"2024-02-23T00:58:20.440851+0000","last_undegraded":"2024-02-23T00:58:20.440851+0000","last_fullsized":"2024-02-23T00:58:20.440851+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:31.932040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:55:58.224758+0000","last_clean_scrub_stamp":"2024-02-23T00:57:31.932040+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"818'14","reported_seq":114,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:33.060659+0000","last_change":"2024-02-23T00:58:05.181610+0000","last_active":"2024-02-23T00:58:33.060659+0000","last_peered":"2024-02-23T00:58:33.060659+0000","last_clean":"2024-02-23T00:58:33.060659+0000","last_became_active":"2024-02-23T00:56:22.402927+0000","last_became_peered":"2024-02-23T00:56:22.402927+0000","last_unstale":"2024-02-23T00:58:33.060659+0000","last_undegraded":"2024-02-23T00:58:33.060659+0000","last_fullsized":"2024-02-23T00:58:33.060659+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"818'14","last_scrub_stamp":"2024-02-23T00:58:05.181539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:58:05.181539+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":197,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.438907+0000","last_change":"2024-02-23T00:58:06.660945+0000","last_active":"2024-02-23T00:58:20.438907+0000","last_peered":"2024-02-23T00:58:20.438907+0000","last_clean":"2024-02-23T00:58:20.438907+0000","last_became_active":"2024-02-23T00:57:13.552728+0000","last_became_peered":"2024-02-23T00:57:13.552728+0000","last_unstale":"2024-02-23T00:58:20.438907+0000","last_undegraded":"2024-02-23T00:58:20.438907+0000","last_fullsized":"2024-02-23T00:58:20.438907+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:06.660878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:12.335752+0000","last_clean_scrub_stamp":"2024-02-23T00:58:06.660878+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"457'27","reported_seq":845,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:27.662854+0000","last_change":"2024-02-23T00:58:27.662825+0000","last_active":"2024-02-23T00:58:27.662854+0000","last_peered":"2024-02-23T00:58:27.662854+0000","last_clean":"2024-02-23T00:58:27.662854+0000","last_became_active":"2024-02-23T00:56:59.502553+0000","last_became_peered":"2024-02-23T00:56:59.502553+0000","last_unstale":"2024-02-23T00:58:27.662854+0000","last_undegraded":"2024-02-23T00:58:27.662854+0000","last_fullsized":"2024-02-23T00:58:27.662854+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-23T00:58:27.662752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:58:27.662752+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":808,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.440246+0000","last_change":"2024-02-23T00:57:47.822981+0000","last_active":"2024-02-23T00:58:20.440246+0000","last_peered":"2024-02-23T00:58:20.440246+0000","last_clean":"2024-02-23T00:58:20.440246+0000","last_became_active":"2024-02-23T00:56:59.508189+0000","last_became_peered":"2024-02-23T00:56:59.508189+0000","last_unstale":"2024-02-23T00:58:20.440246+0000","last_undegraded":"2024-02-23T00:58:20.440246+0000","last_fullsized":"2024-02-23T00:58:20.440246+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:47.822907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:57:47.822907+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"749'12","reported_seq":1134,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:33.060801+0000","last_change":"2024-02-23T00:57:45.403148+0000","last_active":"2024-02-23T00:58:33.060801+0000","last_peered":"2024-02-23T00:58:33.060801+0000","last_clean":"2024-02-23T00:58:33.060801+0000","last_became_active":"2024-02-23T00:54:56.484042+0000","last_became_peered":"2024-02-23T00:54:56.484042+0000","last_unstale":"2024-02-23T00:58:33.060801+0000","last_undegraded":"2024-02-23T00:58:33.060801+0000","last_fullsized":"2024-02-23T00:58:33.060801+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"749'12","last_scrub_stamp":"2024-02-23T00:57:45.403074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:57:45.403074+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":821,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.440225+0000","last_change":"2024-02-23T00:57:06.488551+0000","last_active":"2024-02-23T00:58:20.440225+0000","last_peered":"2024-02-23T00:58:20.440225+0000","last_clean":"2024-02-23T00:58:20.440225+0000","last_became_active":"2024-02-23T00:56:22.391505+0000","last_became_peered":"2024-02-23T00:56:22.391505+0000","last_unstale":"2024-02-23T00:58:20.440225+0000","last_undegraded":"2024-02-23T00:58:20.440225+0000","last_fullsized":"2024-02-23T00:58:20.440225+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:06.488486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:57:06.488486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"457'31","reported_seq":937,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.441069+0000","last_change":"2024-02-23T00:58:01.536858+0000","last_active":"2024-02-23T00:58:20.441069+0000","last_peered":"2024-02-23T00:58:20.441069+0000","last_clean":"2024-02-23T00:58:20.441069+0000","last_became_active":"2024-02-23T00:47:28.201225+0000","last_became_peered":"2024-02-23T00:47:28.201225+0000","last_unstale":"2024-02-23T00:58:20.441069+0000","last_undegraded":"2024-02-23T00:58:20.441069+0000","last_fullsized":"2024-02-23T00:58:20.441069+0000","mapping_epoch":375,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:58:01.536782+0000","last_deep_scrub":"336'23","last_deep_scrub_stamp":"2024-02-23T00:46:48.320085+0000","last_clean_scrub_stamp":"2024-02-23T00:58:01.536782+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":61,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.568213+0000","last_change":"2024-02-23T00:57:44.350468+0000","last_active":"2024-02-23T00:58:20.568213+0000","last_peered":"2024-02-23T00:58:20.568213+0000","last_clean":"2024-02-23T00:58:20.568213+0000","last_became_active":"2024-02-23T00:56:22.403948+0000","last_became_peered":"2024-02-23T00:56:22.403948+0000","last_unstale":"2024-02-23T00:58:20.568213+0000","last_undegraded":"2024-02-23T00:58:20.568213+0000","last_fullsized":"2024-02-23T00:58:20.568213+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:44.350386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:56:42.009186+0000","last_clean_scrub_stamp":"2024-02-23T00:57:44.350386+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":836,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.616355+0000","last_change":"2024-02-23T00:57:53.190680+0000","last_active":"2024-02-23T00:58:20.616355+0000","last_peered":"2024-02-23T00:58:20.616355+0000","last_clean":"2024-02-23T00:58:20.616355+0000","last_became_active":"2024-02-23T00:39:18.800777+0000","last_became_peered":"2024-02-23T00:39:18.800777+0000","last_unstale":"2024-02-23T00:58:20.616355+0000","last_undegraded":"2024-02-23T00:58:20.616355+0000","last_fullsized":"2024-02-23T00:58:20.616355+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:53.190547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:57:53.190547+0000","last_clean_scrub_stamp":"2024-02-23T00:57:53.190547+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":754,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.440083+0000","last_change":"2024-02-23T00:57:53.850253+0000","last_active":"2024-02-23T00:58:20.440083+0000","last_peered":"2024-02-23T00:58:20.440083+0000","last_clean":"2024-02-23T00:58:20.440083+0000","last_became_active":"2024-02-23T00:56:59.508199+0000","last_became_peered":"2024-02-23T00:56:59.508199+0000","last_unstale":"2024-02-23T00:58:20.440083+0000","last_undegraded":"2024-02-23T00:58:20.440083+0000","last_fullsized":"2024-02-23T00:58:20.440083+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:53.850171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:56:29.924704+0000","last_clean_scrub_stamp":"2024-02-23T00:57:53.850171+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"694'10","reported_seq":105,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:29.849992+0000","last_change":"2024-02-23T00:58:29.849960+0000","last_active":"2024-02-23T00:58:29.849992+0000","last_peered":"2024-02-23T00:58:29.849992+0000","last_clean":"2024-02-23T00:58:29.849992+0000","last_became_active":"2024-02-23T00:57:13.552477+0000","last_became_peered":"2024-02-23T00:57:13.552477+0000","last_unstale":"2024-02-23T00:58:29.849992+0000","last_undegraded":"2024-02-23T00:58:29.849992+0000","last_fullsized":"2024-02-23T00:58:29.849992+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"694'10","last_scrub_stamp":"2024-02-23T00:58:29.849892+0000","last_deep_scrub":"694'10","last_deep_scrub_stamp":"2024-02-23T00:58:29.849892+0000","last_clean_scrub_stamp":"2024-02-23T00:58:29.849892+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"457'42","reported_seq":909,"reported_epoch":860,"state":"active+clean","last_fresh":"2024-02-23T00:58:35.389083+0000","last_change":"2024-02-23T00:58:35.389083+0000","last_active":"2024-02-23T00:58:35.389083+0000","last_peered":"2024-02-23T00:58:35.389083+0000","last_clean":"2024-02-23T00:58:35.389083+0000","last_became_active":"2024-02-23T00:39:16.780130+0000","last_became_peered":"2024-02-23T00:39:16.780130+0000","last_unstale":"2024-02-23T00:58:35.389083+0000","last_undegraded":"2024-02-23T00:58:35.389083+0000","last_fullsized":"2024-02-23T00:58:35.389083+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'42","last_scrub_stamp":"2024-02-23T00:58:35.388966+0000","last_deep_scrub":"457'42","last_deep_scrub_stamp":"2024-02-23T00:56:59.534783+0000","last_clean_scrub_stamp":"2024-02-23T00:58:35.388966+0000","log_size":42,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":46,"num_write":34,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":885,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.566809+0000","last_change":"2024-02-23T00:58:00.379618+0000","last_active":"2024-02-23T00:58:20.566809+0000","last_peered":"2024-02-23T00:58:20.566809+0000","last_clean":"2024-02-23T00:58:20.566809+0000","last_became_active":"2024-02-23T00:56:22.397648+0000","last_became_peered":"2024-02-23T00:56:22.397648+0000","last_unstale":"2024-02-23T00:58:20.566809+0000","last_undegraded":"2024-02-23T00:58:20.566809+0000","last_fullsized":"2024-02-23T00:58:20.566809+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:00.379541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:43:51.391939+0000","last_clean_scrub_stamp":"2024-02-23T00:58:00.379541+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":61,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:22.494639+0000","last_change":"2024-02-23T00:58:22.494609+0000","last_active":"2024-02-23T00:58:22.494639+0000","last_peered":"2024-02-23T00:58:22.494639+0000","last_clean":"2024-02-23T00:58:22.494639+0000","last_became_active":"2024-02-23T00:56:22.405106+0000","last_became_peered":"2024-02-23T00:56:22.405106+0000","last_unstale":"2024-02-23T00:58:22.494639+0000","last_undegraded":"2024-02-23T00:58:22.494639+0000","last_fullsized":"2024-02-23T00:58:22.494639+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:22.494495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:46.686764+0000","last_clean_scrub_stamp":"2024-02-23T00:58:22.494495+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":799,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.616428+0000","last_change":"2024-02-23T00:58:16.244751+0000","last_active":"2024-02-23T00:58:20.616428+0000","last_peered":"2024-02-23T00:58:20.616428+0000","last_clean":"2024-02-23T00:58:20.616428+0000","last_became_active":"2024-02-23T00:56:08.359212+0000","last_became_peered":"2024-02-23T00:56:08.359212+0000","last_unstale":"2024-02-23T00:58:20.616428+0000","last_undegraded":"2024-02-23T00:58:20.616428+0000","last_fullsized":"2024-02-23T00:58:20.616428+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:16.244672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:58:16.244672+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":884,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.440811+0000","last_change":"2024-02-23T00:58:07.417274+0000","last_active":"2024-02-23T00:58:20.440811+0000","last_peered":"2024-02-23T00:58:20.440811+0000","last_clean":"2024-02-23T00:58:20.440811+0000","last_became_active":"2024-02-23T00:56:22.385841+0000","last_became_peered":"2024-02-23T00:56:22.385841+0000","last_unstale":"2024-02-23T00:58:20.440811+0000","last_undegraded":"2024-02-23T00:58:20.440811+0000","last_fullsized":"2024-02-23T00:58:20.440811+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:07.417149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:48:10.493340+0000","last_clean_scrub_stamp":"2024-02-23T00:58:07.417149+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"457'52","reported_seq":606,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.436566+0000","last_change":"2024-02-23T00:57:52.784287+0000","last_active":"2024-02-23T00:58:20.436566+0000","last_peered":"2024-02-23T00:58:20.436566+0000","last_clean":"2024-02-23T00:58:20.436566+0000","last_became_active":"2024-02-23T00:57:13.545133+0000","last_became_peered":"2024-02-23T00:57:13.545133+0000","last_unstale":"2024-02-23T00:58:20.436566+0000","last_undegraded":"2024-02-23T00:58:20.436566+0000","last_fullsized":"2024-02-23T00:58:20.436566+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'52","last_scrub_stamp":"2024-02-23T00:57:52.784217+0000","last_deep_scrub":"457'52","last_deep_scrub_stamp":"2024-02-23T00:53:02.966145+0000","last_clean_scrub_stamp":"2024-02-23T00:57:52.784217+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":75,"num_read_kb":61,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":794,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.440450+0000","last_change":"2024-02-23T00:57:21.852554+0000","last_active":"2024-02-23T00:58:20.440450+0000","last_peered":"2024-02-23T00:58:20.440450+0000","last_clean":"2024-02-23T00:58:20.440450+0000","last_became_active":"2024-02-23T00:56:59.505952+0000","last_became_peered":"2024-02-23T00:56:59.505952+0000","last_unstale":"2024-02-23T00:58:20.440450+0000","last_undegraded":"2024-02-23T00:58:20.440450+0000","last_fullsized":"2024-02-23T00:58:20.440450+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:21.852473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:57:21.852473+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":894,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.440745+0000","last_change":"2024-02-23T00:57:55.516427+0000","last_active":"2024-02-23T00:58:20.440745+0000","last_peered":"2024-02-23T00:58:20.440745+0000","last_clean":"2024-02-23T00:58:20.440745+0000","last_became_active":"2024-02-23T00:56:08.337730+0000","last_became_peered":"2024-02-23T00:56:08.337730+0000","last_unstale":"2024-02-23T00:58:20.440745+0000","last_undegraded":"2024-02-23T00:58:20.440745+0000","last_fullsized":"2024-02-23T00:58:20.440745+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:55.516320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:57:55.516320+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":706,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.437004+0000","last_change":"2024-02-23T00:57:23.105307+0000","last_active":"2024-02-23T00:58:20.437004+0000","last_peered":"2024-02-23T00:58:20.437004+0000","last_clean":"2024-02-23T00:58:20.437004+0000","last_became_active":"2024-02-23T00:57:13.548714+0000","last_became_peered":"2024-02-23T00:57:13.548714+0000","last_unstale":"2024-02-23T00:58:20.437004+0000","last_undegraded":"2024-02-23T00:58:20.437004+0000","last_fullsized":"2024-02-23T00:58:20.437004+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:23.105223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:57:23.105223+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":870,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.566092+0000","last_change":"2024-02-23T00:58:01.418152+0000","last_active":"2024-02-23T00:58:20.566092+0000","last_peered":"2024-02-23T00:58:20.566092+0000","last_clean":"2024-02-23T00:58:20.566092+0000","last_became_active":"2024-02-23T00:55:19.374247+0000","last_became_peered":"2024-02-23T00:55:19.374247+0000","last_unstale":"2024-02-23T00:58:20.566092+0000","last_undegraded":"2024-02-23T00:58:20.566092+0000","last_fullsized":"2024-02-23T00:58:20.566092+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":770,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:01.418074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:58:01.418074+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"457'44","reported_seq":204,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.671555+0000","last_change":"2024-02-23T00:57:12.117290+0000","last_active":"2024-02-23T00:58:20.671555+0000","last_peered":"2024-02-23T00:58:20.671555+0000","last_clean":"2024-02-23T00:58:20.671555+0000","last_became_active":"2024-02-23T00:56:22.401039+0000","last_became_peered":"2024-02-23T00:56:22.401039+0000","last_unstale":"2024-02-23T00:58:20.671555+0000","last_undegraded":"2024-02-23T00:58:20.671555+0000","last_fullsized":"2024-02-23T00:58:20.671555+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'44","last_scrub_stamp":"2024-02-23T00:57:12.117220+0000","last_deep_scrub":"457'44","last_deep_scrub_stamp":"2024-02-23T00:57:12.117220+0000","last_clean_scrub_stamp":"2024-02-23T00:57:12.117220+0000","log_size":44,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":56,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.568709+0000","last_change":"2024-02-23T00:57:22.227757+0000","last_active":"2024-02-23T00:58:20.568709+0000","last_peered":"2024-02-23T00:58:20.568709+0000","last_clean":"2024-02-23T00:58:20.568709+0000","last_became_active":"2024-02-23T00:56:22.387775+0000","last_became_peered":"2024-02-23T00:56:22.387775+0000","last_unstale":"2024-02-23T00:58:20.568709+0000","last_undegraded":"2024-02-23T00:58:20.568709+0000","last_fullsized":"2024-02-23T00:58:20.568709+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:22.227657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:57:22.227657+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":828,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.440277+0000","last_change":"2024-02-23T00:57:49.846945+0000","last_active":"2024-02-23T00:58:20.440277+0000","last_peered":"2024-02-23T00:58:20.440277+0000","last_clean":"2024-02-23T00:58:20.440277+0000","last_became_active":"2024-02-23T00:56:59.505756+0000","last_became_peered":"2024-02-23T00:56:59.505756+0000","last_unstale":"2024-02-23T00:58:20.440277+0000","last_undegraded":"2024-02-23T00:58:20.440277+0000","last_fullsized":"2024-02-23T00:58:20.440277+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:49.846837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:57:49.846837+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":783,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.615932+0000","last_change":"2024-02-23T00:57:58.125926+0000","last_active":"2024-02-23T00:58:20.615932+0000","last_peered":"2024-02-23T00:58:20.615932+0000","last_clean":"2024-02-23T00:58:20.615932+0000","last_became_active":"2024-02-23T00:42:58.726652+0000","last_became_peered":"2024-02-23T00:42:58.726652+0000","last_unstale":"2024-02-23T00:58:20.615932+0000","last_undegraded":"2024-02-23T00:58:20.615932+0000","last_fullsized":"2024-02-23T00:58:20.615932+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:58.125844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:57:58.125844+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":877,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.616387+0000","last_change":"2024-02-23T00:58:17.206770+0000","last_active":"2024-02-23T00:58:20.616387+0000","last_peered":"2024-02-23T00:58:20.616387+0000","last_clean":"2024-02-23T00:58:20.616387+0000","last_became_active":"2024-02-23T00:55:45.325453+0000","last_became_peered":"2024-02-23T00:55:45.325453+0000","last_unstale":"2024-02-23T00:58:20.616387+0000","last_undegraded":"2024-02-23T00:58:20.616387+0000","last_fullsized":"2024-02-23T00:58:20.616387+0000","mapping_epoch":789,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":790,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:17.206687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:59.694033+0000","last_clean_scrub_stamp":"2024-02-23T00:58:17.206687+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"457'30","reported_seq":193,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.673103+0000","last_change":"2024-02-23T00:57:33.182257+0000","last_active":"2024-02-23T00:58:20.673103+0000","last_peered":"2024-02-23T00:58:20.673103+0000","last_clean":"2024-02-23T00:58:20.673103+0000","last_became_active":"2024-02-23T00:56:08.353636+0000","last_became_peered":"2024-02-23T00:56:08.353636+0000","last_unstale":"2024-02-23T00:58:20.673103+0000","last_undegraded":"2024-02-23T00:58:20.673103+0000","last_fullsized":"2024-02-23T00:58:20.673103+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'30","last_scrub_stamp":"2024-02-23T00:57:33.182120+0000","last_deep_scrub":"95'18","last_deep_scrub_stamp":"2024-02-23T00:42:01.195545+0000","last_clean_scrub_stamp":"2024-02-23T00:57:33.182120+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":61,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.568064+0000","last_change":"2024-02-23T00:58:05.271692+0000","last_active":"2024-02-23T00:58:20.568064+0000","last_peered":"2024-02-23T00:58:20.568064+0000","last_clean":"2024-02-23T00:58:20.568064+0000","last_became_active":"2024-02-23T00:56:22.401333+0000","last_became_peered":"2024-02-23T00:56:22.401333+0000","last_unstale":"2024-02-23T00:58:20.568064+0000","last_undegraded":"2024-02-23T00:58:20.568064+0000","last_fullsized":"2024-02-23T00:58:20.568064+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:05.271595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:58:05.271595+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":131,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.672693+0000","last_change":"2024-02-23T00:58:06.283843+0000","last_active":"2024-02-23T00:58:20.672693+0000","last_peered":"2024-02-23T00:58:20.672693+0000","last_clean":"2024-02-23T00:58:20.672693+0000","last_became_active":"2024-02-23T00:54:56.471337+0000","last_became_peered":"2024-02-23T00:54:56.471337+0000","last_unstale":"2024-02-23T00:58:20.672693+0000","last_undegraded":"2024-02-23T00:58:20.672693+0000","last_fullsized":"2024-02-23T00:58:20.672693+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:06.283728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:58:06.283728+0000","last_clean_scrub_stamp":"2024-02-23T00:58:06.283728+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"67'1","reported_seq":888,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.441368+0000","last_change":"2024-02-23T00:57:10.876524+0000","last_active":"2024-02-23T00:58:20.441368+0000","last_peered":"2024-02-23T00:58:20.441368+0000","last_clean":"2024-02-23T00:58:20.441368+0000","last_became_active":"2024-02-23T00:56:59.509982+0000","last_became_peered":"2024-02-23T00:56:59.509982+0000","last_unstale":"2024-02-23T00:58:20.441368+0000","last_undegraded":"2024-02-23T00:58:20.441368+0000","last_fullsized":"2024-02-23T00:58:20.441368+0000","mapping_epoch":837,"log_start":"67'1","ondisk_log_start":"67'1","created":65,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-02-23T00:57:10.876415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:57:10.876415+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1476,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":825,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.565352+0000","last_change":"2024-02-23T00:58:15.522587+0000","last_active":"2024-02-23T00:58:20.565352+0000","last_peered":"2024-02-23T00:58:20.565352+0000","last_clean":"2024-02-23T00:58:20.565352+0000","last_became_active":"2024-02-23T00:56:22.387939+0000","last_became_peered":"2024-02-23T00:56:22.387939+0000","last_unstale":"2024-02-23T00:58:20.565352+0000","last_undegraded":"2024-02-23T00:58:20.565352+0000","last_fullsized":"2024-02-23T00:58:20.565352+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:15.522513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:58:15.522513+0000","last_clean_scrub_stamp":"2024-02-23T00:58:15.522513+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":61,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.568641+0000","last_change":"2024-02-23T00:58:15.320311+0000","last_active":"2024-02-23T00:58:20.568641+0000","last_peered":"2024-02-23T00:58:20.568641+0000","last_clean":"2024-02-23T00:58:20.568641+0000","last_became_active":"2024-02-23T00:56:22.399167+0000","last_became_peered":"2024-02-23T00:56:22.399167+0000","last_unstale":"2024-02-23T00:58:20.568641+0000","last_undegraded":"2024-02-23T00:58:20.568641+0000","last_fullsized":"2024-02-23T00:58:20.568641+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:15.320209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:57:02.099770+0000","last_clean_scrub_stamp":"2024-02-23T00:58:15.320209+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"457'26","reported_seq":937,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.566547+0000","last_change":"2024-02-23T00:58:06.463905+0000","last_active":"2024-02-23T00:58:20.566547+0000","last_peered":"2024-02-23T00:58:20.566547+0000","last_clean":"2024-02-23T00:58:20.566547+0000","last_became_active":"2024-02-23T00:56:22.391810+0000","last_became_peered":"2024-02-23T00:56:22.391810+0000","last_unstale":"2024-02-23T00:58:20.566547+0000","last_undegraded":"2024-02-23T00:58:20.566547+0000","last_fullsized":"2024-02-23T00:58:20.566547+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-23T00:58:06.463820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:58:06.463820+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":308,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":56,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.567938+0000","last_change":"2024-02-23T00:57:21.262854+0000","last_active":"2024-02-23T00:58:20.567938+0000","last_peered":"2024-02-23T00:58:20.567938+0000","last_clean":"2024-02-23T00:58:20.567938+0000","last_became_active":"2024-02-23T00:56:22.397990+0000","last_became_peered":"2024-02-23T00:56:22.397990+0000","last_unstale":"2024-02-23T00:58:20.567938+0000","last_undegraded":"2024-02-23T00:58:20.567938+0000","last_fullsized":"2024-02-23T00:58:20.567938+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:21.262795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:49.977248+0000","last_clean_scrub_stamp":"2024-02-23T00:57:21.262795+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":876,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.615791+0000","last_change":"2024-02-23T00:57:22.209079+0000","last_active":"2024-02-23T00:58:20.615791+0000","last_peered":"2024-02-23T00:58:20.615791+0000","last_clean":"2024-02-23T00:58:20.615791+0000","last_became_active":"2024-02-23T00:56:22.396589+0000","last_became_peered":"2024-02-23T00:56:22.396589+0000","last_unstale":"2024-02-23T00:58:20.615791+0000","last_undegraded":"2024-02-23T00:58:20.615791+0000","last_fullsized":"2024-02-23T00:58:20.615791+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:22.209010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:57:22.209010+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":72,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.438640+0000","last_change":"2024-02-23T00:57:32.861242+0000","last_active":"2024-02-23T00:58:20.438640+0000","last_peered":"2024-02-23T00:58:20.438640+0000","last_clean":"2024-02-23T00:58:20.438640+0000","last_became_active":"2024-02-23T00:57:13.548877+0000","last_became_peered":"2024-02-23T00:57:13.548877+0000","last_unstale":"2024-02-23T00:58:20.438640+0000","last_undegraded":"2024-02-23T00:58:20.438640+0000","last_fullsized":"2024-02-23T00:58:20.438640+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:32.861176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:57:32.861176+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":870,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.616400+0000","last_change":"2024-02-23T00:56:59.296066+0000","last_active":"2024-02-23T00:58:20.616400+0000","last_peered":"2024-02-23T00:58:20.616400+0000","last_clean":"2024-02-23T00:58:20.616400+0000","last_became_active":"2024-02-23T00:47:45.182550+0000","last_became_peered":"2024-02-23T00:47:45.182550+0000","last_unstale":"2024-02-23T00:58:20.616400+0000","last_undegraded":"2024-02-23T00:58:20.616400+0000","last_fullsized":"2024-02-23T00:58:20.616400+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:56:59.295987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:19.779989+0000","last_clean_scrub_stamp":"2024-02-23T00:56:59.295987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":56,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.568487+0000","last_change":"2024-02-23T00:57:27.224397+0000","last_active":"2024-02-23T00:58:20.568487+0000","last_peered":"2024-02-23T00:58:20.568487+0000","last_clean":"2024-02-23T00:58:20.568487+0000","last_became_active":"2024-02-23T00:56:22.404754+0000","last_became_peered":"2024-02-23T00:56:22.404754+0000","last_unstale":"2024-02-23T00:58:20.568487+0000","last_undegraded":"2024-02-23T00:58:20.568487+0000","last_fullsized":"2024-02-23T00:58:20.568487+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:27.224323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:57:27.224323+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":76,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.715136+0000","last_change":"2024-02-23T00:57:58.124049+0000","last_active":"2024-02-23T00:58:20.715136+0000","last_peered":"2024-02-23T00:58:20.715136+0000","last_clean":"2024-02-23T00:58:20.715136+0000","last_became_active":"2024-02-23T00:56:22.396739+0000","last_became_peered":"2024-02-23T00:56:22.396739+0000","last_unstale":"2024-02-23T00:58:20.715136+0000","last_undegraded":"2024-02-23T00:58:20.715136+0000","last_fullsized":"2024-02-23T00:58:20.715136+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:58.123980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.264240+0000","last_clean_scrub_stamp":"2024-02-23T00:57:58.123980+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"851'29","reported_seq":892,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.440571+0000","last_change":"2024-02-23T00:57:32.971441+0000","last_active":"2024-02-23T00:58:20.440571+0000","last_peered":"2024-02-23T00:58:20.440571+0000","last_clean":"2024-02-23T00:58:20.440571+0000","last_became_active":"2024-02-23T00:56:59.508879+0000","last_became_peered":"2024-02-23T00:56:59.508879+0000","last_unstale":"2024-02-23T00:58:20.440571+0000","last_undegraded":"2024-02-23T00:58:20.440571+0000","last_fullsized":"2024-02-23T00:58:20.440571+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"851'29","last_scrub_stamp":"2024-02-23T00:57:32.971380+0000","last_deep_scrub":"851'29","last_deep_scrub_stamp":"2024-02-23T00:57:32.971380+0000","last_clean_scrub_stamp":"2024-02-23T00:57:32.971380+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":48,"num_read_kb":36,"num_write":26,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":67,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.714468+0000","last_change":"2024-02-23T00:58:06.206752+0000","last_active":"2024-02-23T00:58:20.714468+0000","last_peered":"2024-02-23T00:58:20.714468+0000","last_clean":"2024-02-23T00:58:20.714468+0000","last_became_active":"2024-02-23T00:56:08.354250+0000","last_became_peered":"2024-02-23T00:56:08.354250+0000","last_unstale":"2024-02-23T00:58:20.714468+0000","last_undegraded":"2024-02-23T00:58:20.714468+0000","last_fullsized":"2024-02-23T00:58:20.714468+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:06.206629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:51:44.659246+0000","last_clean_scrub_stamp":"2024-02-23T00:58:06.206629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"707'6","reported_seq":1922,"reported_epoch":860,"state":"active+clean","last_fresh":"2024-02-23T00:58:35.682932+0000","last_change":"2024-02-23T00:57:39.910370+0000","last_active":"2024-02-23T00:58:35.682932+0000","last_peered":"2024-02-23T00:58:35.682932+0000","last_clean":"2024-02-23T00:58:35.682932+0000","last_became_active":"2024-02-23T00:56:59.507898+0000","last_became_peered":"2024-02-23T00:56:59.507898+0000","last_unstale":"2024-02-23T00:58:35.682932+0000","last_undegraded":"2024-02-23T00:58:35.682932+0000","last_fullsized":"2024-02-23T00:58:35.682932+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"707'6","last_scrub_stamp":"2024-02-23T00:57:39.910298+0000","last_deep_scrub":"707'6","last_deep_scrub_stamp":"2024-02-23T00:57:39.910298+0000","last_clean_scrub_stamp":"2024-02-23T00:57:39.910298+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1082,"num_read_kb":1077,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":61,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.568740+0000","last_change":"2024-02-23T00:58:14.277840+0000","last_active":"2024-02-23T00:58:20.568740+0000","last_peered":"2024-02-23T00:58:20.568740+0000","last_clean":"2024-02-23T00:58:20.568740+0000","last_became_active":"2024-02-23T00:56:22.399702+0000","last_became_peered":"2024-02-23T00:56:22.399702+0000","last_unstale":"2024-02-23T00:58:20.568740+0000","last_undegraded":"2024-02-23T00:58:20.568740+0000","last_fullsized":"2024-02-23T00:58:20.568740+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:14.277759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:58:14.277759+0000","last_clean_scrub_stamp":"2024-02-23T00:58:14.277759+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":754,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.440057+0000","last_change":"2024-02-23T00:57:51.840212+0000","last_active":"2024-02-23T00:58:20.440057+0000","last_peered":"2024-02-23T00:58:20.440057+0000","last_clean":"2024-02-23T00:58:20.440057+0000","last_became_active":"2024-02-23T00:56:59.508642+0000","last_became_peered":"2024-02-23T00:56:59.508642+0000","last_unstale":"2024-02-23T00:58:20.440057+0000","last_undegraded":"2024-02-23T00:58:20.440057+0000","last_fullsized":"2024-02-23T00:58:20.440057+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:51.840138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:57:51.840138+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":140,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.671927+0000","last_change":"2024-02-23T00:58:08.262766+0000","last_active":"2024-02-23T00:58:20.671927+0000","last_peered":"2024-02-23T00:58:20.671927+0000","last_clean":"2024-02-23T00:58:20.671927+0000","last_became_active":"2024-02-23T00:55:19.374145+0000","last_became_peered":"2024-02-23T00:55:19.374145+0000","last_unstale":"2024-02-23T00:58:20.671927+0000","last_undegraded":"2024-02-23T00:58:20.671927+0000","last_fullsized":"2024-02-23T00:58:20.671927+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":770,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:08.262663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:58:08.262663+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"813'51","reported_seq":1204,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:33.060797+0000","last_change":"2024-02-23T00:57:31.174490+0000","last_active":"2024-02-23T00:58:33.060797+0000","last_peered":"2024-02-23T00:58:33.060797+0000","last_clean":"2024-02-23T00:58:33.060797+0000","last_became_active":"2024-02-23T00:56:08.359424+0000","last_became_peered":"2024-02-23T00:56:08.359424+0000","last_unstale":"2024-02-23T00:58:33.060797+0000","last_undegraded":"2024-02-23T00:58:33.060797+0000","last_fullsized":"2024-02-23T00:58:33.060797+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"813'51","last_scrub_stamp":"2024-02-23T00:57:31.174412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:57:31.174412+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":78,"num_read_kb":62,"num_write":44,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":1790,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":845,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:27.132002+0000","last_change":"2024-02-23T00:58:27.131954+0000","last_active":"2024-02-23T00:58:27.132002+0000","last_peered":"2024-02-23T00:58:27.132002+0000","last_clean":"2024-02-23T00:58:27.132002+0000","last_became_active":"2024-02-23T00:39:14.784015+0000","last_became_peered":"2024-02-23T00:39:14.784015+0000","last_unstale":"2024-02-23T00:58:27.132002+0000","last_undegraded":"2024-02-23T00:58:27.132002+0000","last_fullsized":"2024-02-23T00:58:27.132002+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:27.131872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:58:27.131872+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":599,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.437082+0000","last_change":"2024-02-23T00:57:47.854706+0000","last_active":"2024-02-23T00:58:20.437082+0000","last_peered":"2024-02-23T00:58:20.437082+0000","last_clean":"2024-02-23T00:58:20.437082+0000","last_became_active":"2024-02-23T00:57:13.546237+0000","last_became_peered":"2024-02-23T00:57:13.546237+0000","last_unstale":"2024-02-23T00:58:20.437082+0000","last_undegraded":"2024-02-23T00:58:20.437082+0000","last_fullsized":"2024-02-23T00:58:20.437082+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:47.854628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:57:47.854628+0000","last_clean_scrub_stamp":"2024-02-23T00:57:47.854628+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":67,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.714500+0000","last_change":"2024-02-23T00:57:52.166328+0000","last_active":"2024-02-23T00:58:20.714500+0000","last_peered":"2024-02-23T00:58:20.714500+0000","last_clean":"2024-02-23T00:58:20.714500+0000","last_became_active":"2024-02-23T00:56:08.362612+0000","last_became_peered":"2024-02-23T00:56:08.362612+0000","last_unstale":"2024-02-23T00:58:20.714500+0000","last_undegraded":"2024-02-23T00:58:20.714500+0000","last_fullsized":"2024-02-23T00:58:20.714500+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:52.166254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:19.722210+0000","last_clean_scrub_stamp":"2024-02-23T00:57:52.166254+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"749'11","reported_seq":149,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:28.926252+0000","last_change":"2024-02-23T00:58:28.926225+0000","last_active":"2024-02-23T00:58:28.926252+0000","last_peered":"2024-02-23T00:58:28.926252+0000","last_clean":"2024-02-23T00:58:28.926252+0000","last_became_active":"2024-02-23T00:55:19.371588+0000","last_became_peered":"2024-02-23T00:55:19.371588+0000","last_unstale":"2024-02-23T00:58:28.926252+0000","last_undegraded":"2024-02-23T00:58:28.926252+0000","last_fullsized":"2024-02-23T00:58:28.926252+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":770,"parent":"0.0","parent_split_bits":0,"last_scrub":"749'11","last_scrub_stamp":"2024-02-23T00:58:28.926136+0000","last_deep_scrub":"749'11","last_deep_scrub_stamp":"2024-02-23T00:58:28.926136+0000","last_clean_scrub_stamp":"2024-02-23T00:58:28.926136+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"853'32","reported_seq":843,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.440976+0000","last_change":"2024-02-23T00:57:36.902272+0000","last_active":"2024-02-23T00:58:20.440976+0000","last_peered":"2024-02-23T00:58:20.440976+0000","last_clean":"2024-02-23T00:58:20.440976+0000","last_became_active":"2024-02-23T00:56:59.510594+0000","last_became_peered":"2024-02-23T00:56:59.510594+0000","last_unstale":"2024-02-23T00:58:20.440976+0000","last_undegraded":"2024-02-23T00:58:20.440976+0000","last_fullsized":"2024-02-23T00:58:20.440976+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"853'32","last_scrub_stamp":"2024-02-23T00:57:36.902205+0000","last_deep_scrub":"853'32","last_deep_scrub_stamp":"2024-02-23T00:57:36.902205+0000","last_clean_scrub_stamp":"2024-02-23T00:57:36.902205+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":36,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":866,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:29.687906+0000","last_change":"2024-02-23T00:58:29.687878+0000","last_active":"2024-02-23T00:58:29.687906+0000","last_peered":"2024-02-23T00:58:29.687906+0000","last_clean":"2024-02-23T00:58:29.687906+0000","last_became_active":"2024-02-23T00:56:59.509868+0000","last_became_peered":"2024-02-23T00:56:59.509868+0000","last_unstale":"2024-02-23T00:58:29.687906+0000","last_undegraded":"2024-02-23T00:58:29.687906+0000","last_fullsized":"2024-02-23T00:58:29.687906+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:29.687817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:58:29.687817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"107'2","reported_seq":246,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:25.936482+0000","last_change":"2024-02-23T00:58:25.936451+0000","last_active":"2024-02-23T00:58:25.936482+0000","last_peered":"2024-02-23T00:58:25.936482+0000","last_clean":"2024-02-23T00:58:25.936482+0000","last_became_active":"2024-02-23T00:56:40.822752+0000","last_became_peered":"2024-02-23T00:56:40.822752+0000","last_unstale":"2024-02-23T00:58:25.936482+0000","last_undegraded":"2024-02-23T00:58:25.936482+0000","last_fullsized":"2024-02-23T00:58:25.936482+0000","mapping_epoch":828,"log_start":"107'2","ondisk_log_start":"107'2","created":65,"last_epoch_clean":829,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'2","last_scrub_stamp":"2024-02-23T00:58:25.936376+0000","last_deep_scrub":"107'2","last_deep_scrub_stamp":"2024-02-23T00:58:25.936376+0000","last_clean_scrub_stamp":"2024-02-23T00:58:25.936376+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":39,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":838,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.565843+0000","last_change":"2024-02-23T00:58:05.449431+0000","last_active":"2024-02-23T00:58:20.565843+0000","last_peered":"2024-02-23T00:58:20.565843+0000","last_clean":"2024-02-23T00:58:20.565843+0000","last_became_active":"2024-02-23T00:56:22.397674+0000","last_became_peered":"2024-02-23T00:56:22.397674+0000","last_unstale":"2024-02-23T00:58:20.565843+0000","last_undegraded":"2024-02-23T00:58:20.565843+0000","last_fullsized":"2024-02-23T00:58:20.565843+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:05.449359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:58:05.449359+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":67,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.714343+0000","last_change":"2024-02-23T00:57:51.144096+0000","last_active":"2024-02-23T00:58:20.714343+0000","last_peered":"2024-02-23T00:58:20.714343+0000","last_clean":"2024-02-23T00:58:20.714343+0000","last_became_active":"2024-02-23T00:56:08.363537+0000","last_became_peered":"2024-02-23T00:56:08.363537+0000","last_unstale":"2024-02-23T00:58:20.714343+0000","last_undegraded":"2024-02-23T00:58:20.714343+0000","last_fullsized":"2024-02-23T00:58:20.714343+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:51.143998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:49.365711+0000","last_clean_scrub_stamp":"2024-02-23T00:57:51.143998+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"457'31","reported_seq":952,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.440806+0000","last_change":"2024-02-23T00:57:18.288861+0000","last_active":"2024-02-23T00:58:20.440806+0000","last_peered":"2024-02-23T00:58:20.440806+0000","last_clean":"2024-02-23T00:58:20.440806+0000","last_became_active":"2024-02-23T00:53:53.410425+0000","last_became_peered":"2024-02-23T00:53:53.410425+0000","last_unstale":"2024-02-23T00:58:20.440806+0000","last_undegraded":"2024-02-23T00:58:20.440806+0000","last_fullsized":"2024-02-23T00:58:20.440806+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:57:18.288796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:57:18.288796+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":58,"num_read_kb":47,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"95'4","reported_seq":921,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.441211+0000","last_change":"2024-02-23T00:57:49.441397+0000","last_active":"2024-02-23T00:58:20.441211+0000","last_peered":"2024-02-23T00:58:20.441211+0000","last_clean":"2024-02-23T00:58:20.441211+0000","last_became_active":"2024-02-23T00:56:22.389241+0000","last_became_peered":"2024-02-23T00:56:22.389241+0000","last_unstale":"2024-02-23T00:58:20.441211+0000","last_undegraded":"2024-02-23T00:58:20.441211+0000","last_fullsized":"2024-02-23T00:58:20.441211+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'4","last_scrub_stamp":"2024-02-23T00:57:49.441310+0000","last_deep_scrub":"95'4","last_deep_scrub_stamp":"2024-02-23T00:56:30.393527+0000","last_clean_scrub_stamp":"2024-02-23T00:57:49.441310+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":698,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:24.799264+0000","last_change":"2024-02-23T00:58:24.799219+0000","last_active":"2024-02-23T00:58:24.799264+0000","last_peered":"2024-02-23T00:58:24.799264+0000","last_clean":"2024-02-23T00:58:24.799264+0000","last_became_active":"2024-02-23T00:57:13.553197+0000","last_became_peered":"2024-02-23T00:57:13.553197+0000","last_unstale":"2024-02-23T00:58:24.799264+0000","last_undegraded":"2024-02-23T00:58:24.799264+0000","last_fullsized":"2024-02-23T00:58:24.799264+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:24.799094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:52:23.776775+0000","last_clean_scrub_stamp":"2024-02-23T00:58:24.799094+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":61,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.568986+0000","last_change":"2024-02-23T00:57:46.284519+0000","last_active":"2024-02-23T00:58:20.568986+0000","last_peered":"2024-02-23T00:58:20.568986+0000","last_clean":"2024-02-23T00:58:20.568986+0000","last_became_active":"2024-02-23T00:56:22.407258+0000","last_became_peered":"2024-02-23T00:56:22.407258+0000","last_unstale":"2024-02-23T00:58:20.568986+0000","last_undegraded":"2024-02-23T00:58:20.568986+0000","last_fullsized":"2024-02-23T00:58:20.568986+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:46.284436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:57:46.284436+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"833'33","reported_seq":220,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.673020+0000","last_change":"2024-02-23T00:58:01.273267+0000","last_active":"2024-02-23T00:58:20.673020+0000","last_peered":"2024-02-23T00:58:20.673020+0000","last_clean":"2024-02-23T00:58:20.673020+0000","last_became_active":"2024-02-23T00:56:22.391449+0000","last_became_peered":"2024-02-23T00:56:22.391449+0000","last_unstale":"2024-02-23T00:58:20.673020+0000","last_undegraded":"2024-02-23T00:58:20.673020+0000","last_fullsized":"2024-02-23T00:58:20.673020+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"833'33","last_scrub_stamp":"2024-02-23T00:58:01.273197+0000","last_deep_scrub":"833'33","last_deep_scrub_stamp":"2024-02-23T00:56:49.165587+0000","last_clean_scrub_stamp":"2024-02-23T00:58:01.273197+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":49,"num_read_kb":41,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":62,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.714433+0000","last_change":"2024-02-23T00:57:13.948445+0000","last_active":"2024-02-23T00:58:20.714433+0000","last_peered":"2024-02-23T00:58:20.714433+0000","last_clean":"2024-02-23T00:58:20.714433+0000","last_became_active":"2024-02-23T00:56:08.364142+0000","last_became_peered":"2024-02-23T00:56:08.364142+0000","last_unstale":"2024-02-23T00:58:20.714433+0000","last_undegraded":"2024-02-23T00:58:20.714433+0000","last_fullsized":"2024-02-23T00:58:20.714433+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:13.948378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:57:13.948378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":196,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.437808+0000","last_change":"2024-02-23T00:57:26.050076+0000","last_active":"2024-02-23T00:58:20.437808+0000","last_peered":"2024-02-23T00:58:20.437808+0000","last_clean":"2024-02-23T00:58:20.437808+0000","last_became_active":"2024-02-23T00:57:13.553941+0000","last_became_peered":"2024-02-23T00:57:13.553941+0000","last_unstale":"2024-02-23T00:58:20.437808+0000","last_undegraded":"2024-02-23T00:58:20.437808+0000","last_fullsized":"2024-02-23T00:58:20.437808+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:26.050010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:45.913526+0000","last_clean_scrub_stamp":"2024-02-23T00:57:26.050010+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":131,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:29.926306+0000","last_change":"2024-02-23T00:58:29.926281+0000","last_active":"2024-02-23T00:58:29.926306+0000","last_peered":"2024-02-23T00:58:29.926306+0000","last_clean":"2024-02-23T00:58:29.926306+0000","last_became_active":"2024-02-23T00:54:56.487615+0000","last_became_peered":"2024-02-23T00:54:56.487615+0000","last_unstale":"2024-02-23T00:58:29.926306+0000","last_undegraded":"2024-02-23T00:58:29.926306+0000","last_fullsized":"2024-02-23T00:58:29.926306+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:29.926220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:58:29.926220+0000","last_clean_scrub_stamp":"2024-02-23T00:58:29.926220+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"457'22","reported_seq":546,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.437682+0000","last_change":"2024-02-23T00:57:58.767188+0000","last_active":"2024-02-23T00:58:20.437682+0000","last_peered":"2024-02-23T00:58:20.437682+0000","last_clean":"2024-02-23T00:58:20.437682+0000","last_became_active":"2024-02-23T00:57:13.553644+0000","last_became_peered":"2024-02-23T00:57:13.553644+0000","last_unstale":"2024-02-23T00:58:20.437682+0000","last_undegraded":"2024-02-23T00:58:20.437682+0000","last_fullsized":"2024-02-23T00:58:20.437682+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'22","last_scrub_stamp":"2024-02-23T00:57:58.767111+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-02-23T00:48:07.941101+0000","last_clean_scrub_stamp":"2024-02-23T00:57:58.767111+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":842,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:25.653811+0000","last_change":"2024-02-23T00:58:25.653778+0000","last_active":"2024-02-23T00:58:25.653811+0000","last_peered":"2024-02-23T00:58:25.653811+0000","last_clean":"2024-02-23T00:58:25.653811+0000","last_became_active":"2024-02-23T00:39:14.783164+0000","last_became_peered":"2024-02-23T00:39:14.783164+0000","last_unstale":"2024-02-23T00:58:25.653811+0000","last_undegraded":"2024-02-23T00:58:25.653811+0000","last_fullsized":"2024-02-23T00:58:25.653811+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:25.653700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:58:25.653700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":841,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:32.200581+0000","last_change":"2024-02-23T00:58:32.200528+0000","last_active":"2024-02-23T00:58:32.200581+0000","last_peered":"2024-02-23T00:58:32.200581+0000","last_clean":"2024-02-23T00:58:32.200581+0000","last_became_active":"2024-02-23T00:39:18.800297+0000","last_became_peered":"2024-02-23T00:39:18.800297+0000","last_unstale":"2024-02-23T00:58:32.200581+0000","last_undegraded":"2024-02-23T00:58:32.200581+0000","last_fullsized":"2024-02-23T00:58:32.200581+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:32.200439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:58:32.200439+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1","version":"110'7","reported_seq":645,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:23.776355+0000","last_change":"2024-02-23T00:58:23.776327+0000","last_active":"2024-02-23T00:58:23.776355+0000","last_peered":"2024-02-23T00:58:23.776355+0000","last_clean":"2024-02-23T00:58:23.776355+0000","last_became_active":"2024-02-23T00:57:13.550227+0000","last_became_peered":"2024-02-23T00:57:13.550227+0000","last_unstale":"2024-02-23T00:58:23.776355+0000","last_undegraded":"2024-02-23T00:58:23.776355+0000","last_fullsized":"2024-02-23T00:58:23.776355+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'7","last_scrub_stamp":"2024-02-23T00:58:23.776257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:58:23.776257+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"457'32","reported_seq":387,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.438300+0000","last_change":"2024-02-23T00:57:45.883434+0000","last_active":"2024-02-23T00:58:20.438300+0000","last_peered":"2024-02-23T00:58:20.438300+0000","last_clean":"2024-02-23T00:58:20.438300+0000","last_became_active":"2024-02-23T00:57:13.552747+0000","last_became_peered":"2024-02-23T00:57:13.552747+0000","last_unstale":"2024-02-23T00:58:20.438300+0000","last_undegraded":"2024-02-23T00:58:20.438300+0000","last_fullsized":"2024-02-23T00:58:20.438300+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-23T00:57:45.883360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:57:45.883360+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":52,"num_read_kb":45,"num_write":32,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":863,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.615644+0000","last_change":"2024-02-23T00:58:11.262121+0000","last_active":"2024-02-23T00:58:20.615644+0000","last_peered":"2024-02-23T00:58:20.615644+0000","last_clean":"2024-02-23T00:58:20.615644+0000","last_became_active":"2024-02-23T00:56:22.393719+0000","last_became_peered":"2024-02-23T00:56:22.393719+0000","last_unstale":"2024-02-23T00:58:20.615644+0000","last_undegraded":"2024-02-23T00:58:20.615644+0000","last_fullsized":"2024-02-23T00:58:20.615644+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:11.262038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:29.990255+0000","last_clean_scrub_stamp":"2024-02-23T00:58:11.262038+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"749'11","reported_seq":1108,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:33.060858+0000","last_change":"2024-02-23T00:58:13.248021+0000","last_active":"2024-02-23T00:58:33.060858+0000","last_peered":"2024-02-23T00:58:33.060858+0000","last_clean":"2024-02-23T00:58:33.060858+0000","last_became_active":"2024-02-23T00:54:56.483713+0000","last_became_peered":"2024-02-23T00:54:56.483713+0000","last_unstale":"2024-02-23T00:58:33.060858+0000","last_undegraded":"2024-02-23T00:58:33.060858+0000","last_fullsized":"2024-02-23T00:58:33.060858+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"749'11","last_scrub_stamp":"2024-02-23T00:58:13.247944+0000","last_deep_scrub":"749'11","last_deep_scrub_stamp":"2024-02-23T00:57:01.303627+0000","last_clean_scrub_stamp":"2024-02-23T00:58:13.247944+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":61,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:24.488520+0000","last_change":"2024-02-23T00:58:24.488463+0000","last_active":"2024-02-23T00:58:24.488520+0000","last_peered":"2024-02-23T00:58:24.488520+0000","last_clean":"2024-02-23T00:58:24.488520+0000","last_became_active":"2024-02-23T00:56:22.394948+0000","last_became_peered":"2024-02-23T00:56:22.394948+0000","last_unstale":"2024-02-23T00:58:24.488520+0000","last_undegraded":"2024-02-23T00:58:24.488520+0000","last_fullsized":"2024-02-23T00:58:24.488520+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:24.488336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:53:00.410226+0000","last_clean_scrub_stamp":"2024-02-23T00:58:24.488336+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"457'21","reported_seq":920,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.616024+0000","last_change":"2024-02-23T00:57:18.247304+0000","last_active":"2024-02-23T00:58:20.616024+0000","last_peered":"2024-02-23T00:58:20.616024+0000","last_clean":"2024-02-23T00:58:20.616024+0000","last_became_active":"2024-02-23T00:56:22.394025+0000","last_became_peered":"2024-02-23T00:56:22.394025+0000","last_unstale":"2024-02-23T00:58:20.616024+0000","last_undegraded":"2024-02-23T00:58:20.616024+0000","last_fullsized":"2024-02-23T00:58:20.616024+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-23T00:57:18.247236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:57:18.247236+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":131,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:27.920753+0000","last_change":"2024-02-23T00:58:27.920726+0000","last_active":"2024-02-23T00:58:27.920753+0000","last_peered":"2024-02-23T00:58:27.920753+0000","last_clean":"2024-02-23T00:58:27.920753+0000","last_became_active":"2024-02-23T00:54:56.472343+0000","last_became_peered":"2024-02-23T00:54:56.472343+0000","last_unstale":"2024-02-23T00:58:27.920753+0000","last_undegraded":"2024-02-23T00:58:27.920753+0000","last_fullsized":"2024-02-23T00:58:27.920753+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:27.920657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:58:27.920657+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"844'16","reported_seq":117,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:33.060659+0000","last_change":"2024-02-23T00:58:08.756001+0000","last_active":"2024-02-23T00:58:33.060659+0000","last_peered":"2024-02-23T00:58:33.060659+0000","last_clean":"2024-02-23T00:58:33.060659+0000","last_became_active":"2024-02-23T00:57:13.551557+0000","last_became_peered":"2024-02-23T00:57:13.551557+0000","last_unstale":"2024-02-23T00:58:33.060659+0000","last_undegraded":"2024-02-23T00:58:33.060659+0000","last_fullsized":"2024-02-23T00:58:33.060659+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"844'16","last_scrub_stamp":"2024-02-23T00:58:08.755930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:58:08.755930+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":745,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.441318+0000","last_change":"2024-02-23T00:58:19.711023+0000","last_active":"2024-02-23T00:58:20.441318+0000","last_peered":"2024-02-23T00:58:20.441318+0000","last_clean":"2024-02-23T00:58:20.441318+0000","last_became_active":"2024-02-23T00:56:59.509047+0000","last_became_peered":"2024-02-23T00:56:59.509047+0000","last_unstale":"2024-02-23T00:58:20.441318+0000","last_undegraded":"2024-02-23T00:58:20.441318+0000","last_fullsized":"2024-02-23T00:58:20.441318+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:19.710960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:58:19.710960+0000","last_clean_scrub_stamp":"2024-02-23T00:58:19.710960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"823'19","reported_seq":554,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.440931+0000","last_change":"2024-02-23T00:58:10.761859+0000","last_active":"2024-02-23T00:58:20.440931+0000","last_peered":"2024-02-23T00:58:20.440931+0000","last_clean":"2024-02-23T00:58:20.440931+0000","last_became_active":"2024-02-23T00:56:59.506257+0000","last_became_peered":"2024-02-23T00:56:59.506257+0000","last_unstale":"2024-02-23T00:58:20.440931+0000","last_undegraded":"2024-02-23T00:58:20.440931+0000","last_fullsized":"2024-02-23T00:58:20.440931+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"823'19","last_scrub_stamp":"2024-02-23T00:58:10.761792+0000","last_deep_scrub":"823'19","last_deep_scrub_stamp":"2024-02-23T00:56:28.921013+0000","last_clean_scrub_stamp":"2024-02-23T00:58:10.761792+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":856,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.565767+0000","last_change":"2024-02-23T00:57:20.705702+0000","last_active":"2024-02-23T00:58:20.565767+0000","last_peered":"2024-02-23T00:58:20.565767+0000","last_clean":"2024-02-23T00:58:20.565767+0000","last_became_active":"2024-02-23T00:53:45.293336+0000","last_became_peered":"2024-02-23T00:53:45.293336+0000","last_unstale":"2024-02-23T00:58:20.565767+0000","last_undegraded":"2024-02-23T00:58:20.565767+0000","last_fullsized":"2024-02-23T00:58:20.565767+0000","mapping_epoch":685,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":686,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:20.705638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:57:20.705638+0000","last_clean_scrub_stamp":"2024-02-23T00:57:20.705638+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":700,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.437025+0000","last_change":"2024-02-23T00:57:22.059152+0000","last_active":"2024-02-23T00:58:20.437025+0000","last_peered":"2024-02-23T00:58:20.437025+0000","last_clean":"2024-02-23T00:58:20.437025+0000","last_became_active":"2024-02-23T00:57:13.552787+0000","last_became_peered":"2024-02-23T00:57:13.552787+0000","last_unstale":"2024-02-23T00:58:20.437025+0000","last_undegraded":"2024-02-23T00:58:20.437025+0000","last_fullsized":"2024-02-23T00:58:20.437025+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:22.059026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:17.931413+0000","last_clean_scrub_stamp":"2024-02-23T00:57:22.059026+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":131,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:26.897220+0000","last_change":"2024-02-23T00:58:26.897181+0000","last_active":"2024-02-23T00:58:26.897220+0000","last_peered":"2024-02-23T00:58:26.897220+0000","last_clean":"2024-02-23T00:58:26.897220+0000","last_became_active":"2024-02-23T00:54:56.487799+0000","last_became_peered":"2024-02-23T00:54:56.487799+0000","last_unstale":"2024-02-23T00:58:26.897220+0000","last_undegraded":"2024-02-23T00:58:26.897220+0000","last_fullsized":"2024-02-23T00:58:26.897220+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:26.897097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:58:26.897097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"457'19","reported_seq":875,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:28.327015+0000","last_change":"2024-02-23T00:58:28.326978+0000","last_active":"2024-02-23T00:58:28.327015+0000","last_peered":"2024-02-23T00:58:28.327015+0000","last_clean":"2024-02-23T00:58:28.327015+0000","last_became_active":"2024-02-23T00:39:16.783896+0000","last_became_peered":"2024-02-23T00:39:16.783896+0000","last_unstale":"2024-02-23T00:58:28.327015+0000","last_undegraded":"2024-02-23T00:58:28.327015+0000","last_fullsized":"2024-02-23T00:58:28.327015+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'19","last_scrub_stamp":"2024-02-23T00:58:28.326898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:58:28.326898+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"202'2","reported_seq":838,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.674395+0000","last_change":"2024-02-23T00:58:20.674351+0000","last_active":"2024-02-23T00:58:20.674395+0000","last_peered":"2024-02-23T00:58:20.674395+0000","last_clean":"2024-02-23T00:58:20.674395+0000","last_became_active":"2024-02-23T00:56:59.510938+0000","last_became_peered":"2024-02-23T00:56:59.510938+0000","last_unstale":"2024-02-23T00:58:20.674395+0000","last_undegraded":"2024-02-23T00:58:20.674395+0000","last_fullsized":"2024-02-23T00:58:20.674395+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"202'2","last_scrub_stamp":"2024-02-23T00:58:20.674239+0000","last_deep_scrub":"202'2","last_deep_scrub_stamp":"2024-02-23T00:58:20.674239+0000","last_clean_scrub_stamp":"2024-02-23T00:58:20.674239+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1746,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":90,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.565839+0000","last_change":"2024-02-23T00:58:08.530911+0000","last_active":"2024-02-23T00:58:20.565839+0000","last_peered":"2024-02-23T00:58:20.565839+0000","last_clean":"2024-02-23T00:58:20.565839+0000","last_became_active":"2024-02-23T00:55:45.325510+0000","last_became_peered":"2024-02-23T00:55:45.325510+0000","last_unstale":"2024-02-23T00:58:20.565839+0000","last_undegraded":"2024-02-23T00:58:20.565839+0000","last_fullsized":"2024-02-23T00:58:20.565839+0000","mapping_epoch":789,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":790,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:08.530798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:58:08.530798+0000","last_clean_scrub_stamp":"2024-02-23T00:58:08.530798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":763,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.440664+0000","last_change":"2024-02-23T00:57:48.864113+0000","last_active":"2024-02-23T00:58:20.440664+0000","last_peered":"2024-02-23T00:58:20.440664+0000","last_clean":"2024-02-23T00:58:20.440664+0000","last_became_active":"2024-02-23T00:56:59.506213+0000","last_became_peered":"2024-02-23T00:56:59.506213+0000","last_unstale":"2024-02-23T00:58:20.440664+0000","last_undegraded":"2024-02-23T00:58:20.440664+0000","last_fullsized":"2024-02-23T00:58:20.440664+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:48.864042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:46.462015+0000","last_clean_scrub_stamp":"2024-02-23T00:57:48.864042+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"457'8","reported_seq":512,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.436665+0000","last_change":"2024-02-23T00:57:56.796486+0000","last_active":"2024-02-23T00:58:20.436665+0000","last_peered":"2024-02-23T00:58:20.436665+0000","last_clean":"2024-02-23T00:58:20.436665+0000","last_became_active":"2024-02-23T00:57:13.552944+0000","last_became_peered":"2024-02-23T00:57:13.552944+0000","last_unstale":"2024-02-23T00:58:20.436665+0000","last_undegraded":"2024-02-23T00:58:20.436665+0000","last_fullsized":"2024-02-23T00:58:20.436665+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'8","last_scrub_stamp":"2024-02-23T00:57:56.796410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:57:56.796410+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":866,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.616231+0000","last_change":"2024-02-23T00:58:12.265386+0000","last_active":"2024-02-23T00:58:20.616231+0000","last_peered":"2024-02-23T00:58:20.616231+0000","last_clean":"2024-02-23T00:58:20.616231+0000","last_became_active":"2024-02-23T00:56:08.352993+0000","last_became_peered":"2024-02-23T00:56:08.352993+0000","last_unstale":"2024-02-23T00:58:20.616231+0000","last_undegraded":"2024-02-23T00:58:20.616231+0000","last_fullsized":"2024-02-23T00:58:20.616231+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:12.265304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:58:12.265304+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":870,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.565623+0000","last_change":"2024-02-23T00:57:43.655000+0000","last_active":"2024-02-23T00:58:20.565623+0000","last_peered":"2024-02-23T00:58:20.565623+0000","last_clean":"2024-02-23T00:58:20.565623+0000","last_became_active":"2024-02-23T00:56:22.385466+0000","last_became_peered":"2024-02-23T00:56:22.385466+0000","last_unstale":"2024-02-23T00:58:20.565623+0000","last_undegraded":"2024-02-23T00:58:20.565623+0000","last_fullsized":"2024-02-23T00:58:20.565623+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:43.654912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:57:43.654912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":61,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.568876+0000","last_change":"2024-02-23T00:57:33.287063+0000","last_active":"2024-02-23T00:58:20.568876+0000","last_peered":"2024-02-23T00:58:20.568876+0000","last_clean":"2024-02-23T00:58:20.568876+0000","last_became_active":"2024-02-23T00:56:22.400114+0000","last_became_peered":"2024-02-23T00:56:22.400114+0000","last_unstale":"2024-02-23T00:58:20.568876+0000","last_undegraded":"2024-02-23T00:58:20.568876+0000","last_fullsized":"2024-02-23T00:58:20.568876+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:33.286989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:57:33.286989+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"457'25","reported_seq":735,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.439225+0000","last_change":"2024-02-23T00:57:36.818233+0000","last_active":"2024-02-23T00:58:20.439225+0000","last_peered":"2024-02-23T00:58:20.439225+0000","last_clean":"2024-02-23T00:58:20.439225+0000","last_became_active":"2024-02-23T00:57:13.549408+0000","last_became_peered":"2024-02-23T00:57:13.549408+0000","last_unstale":"2024-02-23T00:58:20.439225+0000","last_undegraded":"2024-02-23T00:58:20.439225+0000","last_fullsized":"2024-02-23T00:58:20.439225+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-23T00:57:36.818139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:57:36.818139+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":865,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:23.609499+0000","last_change":"2024-02-23T00:58:23.609466+0000","last_active":"2024-02-23T00:58:23.609499+0000","last_peered":"2024-02-23T00:58:23.609499+0000","last_clean":"2024-02-23T00:58:23.609499+0000","last_became_active":"2024-02-23T00:56:08.365524+0000","last_became_peered":"2024-02-23T00:56:08.365524+0000","last_unstale":"2024-02-23T00:58:23.609499+0000","last_undegraded":"2024-02-23T00:58:23.609499+0000","last_fullsized":"2024-02-23T00:58:23.609499+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:23.609383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:54:53.583713+0000","last_clean_scrub_stamp":"2024-02-23T00:58:23.609383+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":61,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.567940+0000","last_change":"2024-02-23T00:57:53.279971+0000","last_active":"2024-02-23T00:58:20.567940+0000","last_peered":"2024-02-23T00:58:20.567940+0000","last_clean":"2024-02-23T00:58:20.567940+0000","last_became_active":"2024-02-23T00:56:22.389716+0000","last_became_peered":"2024-02-23T00:56:22.389716+0000","last_unstale":"2024-02-23T00:58:20.567940+0000","last_undegraded":"2024-02-23T00:58:20.567940+0000","last_fullsized":"2024-02-23T00:58:20.567940+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:53.279872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:57:53.279872+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":67,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:23.405588+0000","last_change":"2024-02-23T00:58:23.405558+0000","last_active":"2024-02-23T00:58:23.405588+0000","last_peered":"2024-02-23T00:58:23.405588+0000","last_clean":"2024-02-23T00:58:23.405588+0000","last_became_active":"2024-02-23T00:56:08.359732+0000","last_became_peered":"2024-02-23T00:56:08.359732+0000","last_unstale":"2024-02-23T00:58:23.405588+0000","last_undegraded":"2024-02-23T00:58:23.405588+0000","last_fullsized":"2024-02-23T00:58:23.405588+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:23.405491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:57:10.986967+0000","last_clean_scrub_stamp":"2024-02-23T00:58:23.405491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"457'26","reported_seq":191,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.673008+0000","last_change":"2024-02-23T00:57:34.179797+0000","last_active":"2024-02-23T00:58:20.673008+0000","last_peered":"2024-02-23T00:58:20.673008+0000","last_clean":"2024-02-23T00:58:20.673008+0000","last_became_active":"2024-02-23T00:56:22.396325+0000","last_became_peered":"2024-02-23T00:56:22.396325+0000","last_unstale":"2024-02-23T00:58:20.673008+0000","last_undegraded":"2024-02-23T00:58:20.673008+0000","last_fullsized":"2024-02-23T00:58:20.673008+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-23T00:57:34.179731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:57:34.179731+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":35,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":880,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.615545+0000","last_change":"2024-02-23T00:58:07.214072+0000","last_active":"2024-02-23T00:58:20.615545+0000","last_peered":"2024-02-23T00:58:20.615545+0000","last_clean":"2024-02-23T00:58:20.615545+0000","last_became_active":"2024-02-23T00:56:22.394165+0000","last_became_peered":"2024-02-23T00:56:22.394165+0000","last_unstale":"2024-02-23T00:58:20.615545+0000","last_undegraded":"2024-02-23T00:58:20.615545+0000","last_fullsized":"2024-02-23T00:58:20.615545+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"80'2","last_scrub_stamp":"2024-02-23T00:58:07.213992+0000","last_deep_scrub":"80'2","last_deep_scrub_stamp":"2024-02-23T00:40:15.942725+0000","last_clean_scrub_stamp":"2024-02-23T00:58:07.213992+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"818'12","reported_seq":1126,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:33.060951+0000","last_change":"2024-02-23T00:58:13.585298+0000","last_active":"2024-02-23T00:58:33.060951+0000","last_peered":"2024-02-23T00:58:33.060951+0000","last_clean":"2024-02-23T00:58:33.060951+0000","last_became_active":"2024-02-23T00:56:22.392240+0000","last_became_peered":"2024-02-23T00:56:22.392240+0000","last_unstale":"2024-02-23T00:58:33.060951+0000","last_undegraded":"2024-02-23T00:58:33.060951+0000","last_fullsized":"2024-02-23T00:58:33.060951+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"818'12","last_scrub_stamp":"2024-02-23T00:58:13.585184+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-02-23T00:43:50.393713+0000","last_clean_scrub_stamp":"2024-02-23T00:58:13.585184+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":140,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.671141+0000","last_change":"2024-02-23T00:58:00.282472+0000","last_active":"2024-02-23T00:58:20.671141+0000","last_peered":"2024-02-23T00:58:20.671141+0000","last_clean":"2024-02-23T00:58:20.671141+0000","last_became_active":"2024-02-23T00:56:08.346147+0000","last_became_peered":"2024-02-23T00:56:08.346147+0000","last_unstale":"2024-02-23T00:58:20.671141+0000","last_undegraded":"2024-02-23T00:58:20.671141+0000","last_fullsized":"2024-02-23T00:58:20.671141+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:00.282399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:58:00.282399+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"457'31","reported_seq":948,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.566899+0000","last_change":"2024-02-23T00:58:16.552651+0000","last_active":"2024-02-23T00:58:20.566899+0000","last_peered":"2024-02-23T00:58:20.566899+0000","last_clean":"2024-02-23T00:58:20.566899+0000","last_became_active":"2024-02-23T00:55:32.369643+0000","last_became_peered":"2024-02-23T00:55:32.369643+0000","last_unstale":"2024-02-23T00:58:20.566899+0000","last_undegraded":"2024-02-23T00:58:20.566899+0000","last_fullsized":"2024-02-23T00:58:20.566899+0000","mapping_epoch":780,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":781,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'31","last_scrub_stamp":"2024-02-23T00:58:16.552539+0000","last_deep_scrub":"457'31","last_deep_scrub_stamp":"2024-02-23T00:52:27.357369+0000","last_clean_scrub_stamp":"2024-02-23T00:58:16.552539+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":770,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.438988+0000","last_change":"2024-02-23T00:58:18.815672+0000","last_active":"2024-02-23T00:58:20.438988+0000","last_peered":"2024-02-23T00:58:20.438988+0000","last_clean":"2024-02-23T00:58:20.438988+0000","last_became_active":"2024-02-23T00:57:13.549830+0000","last_became_peered":"2024-02-23T00:57:13.549830+0000","last_unstale":"2024-02-23T00:58:20.438988+0000","last_undegraded":"2024-02-23T00:58:20.438988+0000","last_fullsized":"2024-02-23T00:58:20.438988+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:18.815597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:58:18.815597+0000","last_clean_scrub_stamp":"2024-02-23T00:58:18.815597+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":158,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.672924+0000","last_change":"2024-02-23T00:58:09.247265+0000","last_active":"2024-02-23T00:58:20.672924+0000","last_peered":"2024-02-23T00:58:20.672924+0000","last_clean":"2024-02-23T00:58:20.672924+0000","last_became_active":"2024-02-23T00:56:22.398137+0000","last_became_peered":"2024-02-23T00:56:22.398137+0000","last_unstale":"2024-02-23T00:58:20.672924+0000","last_undegraded":"2024-02-23T00:58:20.672924+0000","last_fullsized":"2024-02-23T00:58:20.672924+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:09.247200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:58:09.247200+0000","last_clean_scrub_stamp":"2024-02-23T00:58:09.247200+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":627,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.439027+0000","last_change":"2024-02-23T00:57:39.845591+0000","last_active":"2024-02-23T00:58:20.439027+0000","last_peered":"2024-02-23T00:58:20.439027+0000","last_clean":"2024-02-23T00:58:20.439027+0000","last_became_active":"2024-02-23T00:57:13.546252+0000","last_became_peered":"2024-02-23T00:57:13.546252+0000","last_unstale":"2024-02-23T00:58:20.439027+0000","last_undegraded":"2024-02-23T00:58:20.439027+0000","last_fullsized":"2024-02-23T00:58:20.439027+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:39.845522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:57:39.845522+0000","last_clean_scrub_stamp":"2024-02-23T00:57:39.845522+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"457'27","reported_seq":83,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.714770+0000","last_change":"2024-02-23T00:57:31.994247+0000","last_active":"2024-02-23T00:58:20.714770+0000","last_peered":"2024-02-23T00:58:20.714770+0000","last_clean":"2024-02-23T00:58:20.714770+0000","last_became_active":"2024-02-23T00:56:08.371957+0000","last_became_peered":"2024-02-23T00:56:08.371957+0000","last_unstale":"2024-02-23T00:58:20.714770+0000","last_undegraded":"2024-02-23T00:58:20.714770+0000","last_fullsized":"2024-02-23T00:58:20.714770+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-23T00:57:31.994106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:15.764507+0000","last_clean_scrub_stamp":"2024-02-23T00:57:31.994106+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":874,"reported_epoch":860,"state":"active+clean","last_fresh":"2024-02-23T00:58:35.655510+0000","last_change":"2024-02-23T00:58:35.655510+0000","last_active":"2024-02-23T00:58:35.655510+0000","last_peered":"2024-02-23T00:58:35.655510+0000","last_clean":"2024-02-23T00:58:35.655510+0000","last_became_active":"2024-02-23T00:56:22.385519+0000","last_became_peered":"2024-02-23T00:56:22.385519+0000","last_unstale":"2024-02-23T00:58:35.655510+0000","last_undegraded":"2024-02-23T00:58:35.655510+0000","last_fullsized":"2024-02-23T00:58:35.655510+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:35.655435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:58:35.655435+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":808,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.440926+0000","last_change":"2024-02-23T00:57:35.925082+0000","last_active":"2024-02-23T00:58:20.440926+0000","last_peered":"2024-02-23T00:58:20.440926+0000","last_clean":"2024-02-23T00:58:20.440926+0000","last_became_active":"2024-02-23T00:56:59.504814+0000","last_became_peered":"2024-02-23T00:56:59.504814+0000","last_unstale":"2024-02-23T00:58:20.440926+0000","last_undegraded":"2024-02-23T00:58:20.440926+0000","last_fullsized":"2024-02-23T00:58:20.440926+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:35.925001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:52.641689+0000","last_clean_scrub_stamp":"2024-02-23T00:57:35.925001+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":78,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.438510+0000","last_change":"2024-02-23T00:57:40.835745+0000","last_active":"2024-02-23T00:58:20.438510+0000","last_peered":"2024-02-23T00:58:20.438510+0000","last_clean":"2024-02-23T00:58:20.438510+0000","last_became_active":"2024-02-23T00:57:13.548458+0000","last_became_peered":"2024-02-23T00:57:13.548458+0000","last_unstale":"2024-02-23T00:58:20.438510+0000","last_undegraded":"2024-02-23T00:58:20.438510+0000","last_fullsized":"2024-02-23T00:58:20.438510+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:40.835672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:57:40.835672+0000","last_clean_scrub_stamp":"2024-02-23T00:57:40.835672+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"616'30","reported_seq":724,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.437245+0000","last_change":"2024-02-23T00:57:48.835733+0000","last_active":"2024-02-23T00:58:20.437245+0000","last_peered":"2024-02-23T00:58:20.437245+0000","last_clean":"2024-02-23T00:58:20.437245+0000","last_became_active":"2024-02-23T00:57:13.551428+0000","last_became_peered":"2024-02-23T00:57:13.551428+0000","last_unstale":"2024-02-23T00:58:20.437245+0000","last_undegraded":"2024-02-23T00:58:20.437245+0000","last_fullsized":"2024-02-23T00:58:20.437245+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"616'30","last_scrub_stamp":"2024-02-23T00:57:48.835663+0000","last_deep_scrub":"616'30","last_deep_scrub_stamp":"2024-02-23T00:52:25.764389+0000","last_clean_scrub_stamp":"2024-02-23T00:57:48.835663+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":46,"num_read_kb":36,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":75,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.436860+0000","last_change":"2024-02-23T00:58:02.715258+0000","last_active":"2024-02-23T00:58:20.436860+0000","last_peered":"2024-02-23T00:58:20.436860+0000","last_clean":"2024-02-23T00:58:20.436860+0000","last_became_active":"2024-02-23T00:57:13.551625+0000","last_became_peered":"2024-02-23T00:57:13.551625+0000","last_unstale":"2024-02-23T00:58:20.436860+0000","last_undegraded":"2024-02-23T00:58:20.436860+0000","last_fullsized":"2024-02-23T00:58:20.436860+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:02.715194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:58:02.715194+0000","last_clean_scrub_stamp":"2024-02-23T00:58:02.715194+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":210,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.565582+0000","last_change":"2024-02-23T00:57:49.674116+0000","last_active":"2024-02-23T00:58:20.565582+0000","last_peered":"2024-02-23T00:58:20.565582+0000","last_clean":"2024-02-23T00:58:20.565582+0000","last_became_active":"2024-02-23T00:56:08.364467+0000","last_became_peered":"2024-02-23T00:56:08.364467+0000","last_unstale":"2024-02-23T00:58:20.565582+0000","last_undegraded":"2024-02-23T00:58:20.565582+0000","last_fullsized":"2024-02-23T00:58:20.565582+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:49.674031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:41:01.063660+0000","last_clean_scrub_stamp":"2024-02-23T00:57:49.674031+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":808,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:26.133416+0000","last_change":"2024-02-23T00:58:26.133377+0000","last_active":"2024-02-23T00:58:26.133416+0000","last_peered":"2024-02-23T00:58:26.133416+0000","last_clean":"2024-02-23T00:58:26.133416+0000","last_became_active":"2024-02-23T00:56:08.352413+0000","last_became_peered":"2024-02-23T00:56:08.352413+0000","last_unstale":"2024-02-23T00:58:26.133416+0000","last_undegraded":"2024-02-23T00:58:26.133416+0000","last_fullsized":"2024-02-23T00:58:26.133416+0000","mapping_epoch":812,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":813,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:26.133296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:58:26.133296+0000","last_clean_scrub_stamp":"2024-02-23T00:58:26.133296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"457'33","reported_seq":953,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.440485+0000","last_change":"2024-02-23T00:57:59.561707+0000","last_active":"2024-02-23T00:58:20.440485+0000","last_peered":"2024-02-23T00:58:20.440485+0000","last_clean":"2024-02-23T00:58:20.440485+0000","last_became_active":"2024-02-23T00:54:56.483435+0000","last_became_peered":"2024-02-23T00:54:56.483435+0000","last_unstale":"2024-02-23T00:58:20.440485+0000","last_undegraded":"2024-02-23T00:58:20.440485+0000","last_fullsized":"2024-02-23T00:58:20.440485+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'33","last_scrub_stamp":"2024-02-23T00:57:59.561623+0000","last_deep_scrub":"457'33","last_deep_scrub_stamp":"2024-02-23T00:56:44.520788+0000","last_clean_scrub_stamp":"2024-02-23T00:57:59.561623+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":61,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.568798+0000","last_change":"2024-02-23T00:58:04.250273+0000","last_active":"2024-02-23T00:58:20.568798+0000","last_peered":"2024-02-23T00:58:20.568798+0000","last_clean":"2024-02-23T00:58:20.568798+0000","last_became_active":"2024-02-23T00:56:22.405680+0000","last_became_peered":"2024-02-23T00:56:22.405680+0000","last_unstale":"2024-02-23T00:58:20.568798+0000","last_undegraded":"2024-02-23T00:58:20.568798+0000","last_fullsized":"2024-02-23T00:58:20.568798+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:04.250166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:58:04.250166+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":874,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.615811+0000","last_change":"2024-02-23T00:57:19.236285+0000","last_active":"2024-02-23T00:58:20.615811+0000","last_peered":"2024-02-23T00:58:20.615811+0000","last_clean":"2024-02-23T00:58:20.615811+0000","last_became_active":"2024-02-23T00:56:22.405449+0000","last_became_peered":"2024-02-23T00:56:22.405449+0000","last_unstale":"2024-02-23T00:58:20.615811+0000","last_undegraded":"2024-02-23T00:58:20.615811+0000","last_fullsized":"2024-02-23T00:58:20.615811+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:19.236197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:57:19.236197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":799,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.615938+0000","last_change":"2024-02-23T00:57:24.160282+0000","last_active":"2024-02-23T00:58:20.615938+0000","last_peered":"2024-02-23T00:58:20.615938+0000","last_clean":"2024-02-23T00:58:20.615938+0000","last_became_active":"2024-02-23T00:54:56.482357+0000","last_became_peered":"2024-02-23T00:54:56.482357+0000","last_unstale":"2024-02-23T00:58:20.615938+0000","last_undegraded":"2024-02-23T00:58:20.615938+0000","last_fullsized":"2024-02-23T00:58:20.615938+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:24.160212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:42:53.113727+0000","last_clean_scrub_stamp":"2024-02-23T00:57:24.160212+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"457'17","reported_seq":878,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.440828+0000","last_change":"2024-02-23T00:57:44.395181+0000","last_active":"2024-02-23T00:58:20.440828+0000","last_peered":"2024-02-23T00:58:20.440828+0000","last_clean":"2024-02-23T00:58:20.440828+0000","last_became_active":"2024-02-23T00:39:16.782762+0000","last_became_peered":"2024-02-23T00:39:16.782762+0000","last_unstale":"2024-02-23T00:58:20.440828+0000","last_undegraded":"2024-02-23T00:58:20.440828+0000","last_fullsized":"2024-02-23T00:58:20.440828+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'17","last_scrub_stamp":"2024-02-23T00:57:44.395104+0000","last_deep_scrub":"82'15","last_deep_scrub_stamp":"2024-02-23T00:41:50.975683+0000","last_clean_scrub_stamp":"2024-02-23T00:57:44.395104+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":21,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":56,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.568786+0000","last_change":"2024-02-23T00:57:31.244236+0000","last_active":"2024-02-23T00:58:20.568786+0000","last_peered":"2024-02-23T00:58:20.568786+0000","last_clean":"2024-02-23T00:58:20.568786+0000","last_became_active":"2024-02-23T00:56:22.390249+0000","last_became_peered":"2024-02-23T00:56:22.390249+0000","last_unstale":"2024-02-23T00:58:20.568786+0000","last_undegraded":"2024-02-23T00:58:20.568786+0000","last_fullsized":"2024-02-23T00:58:20.568786+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:31.244160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:13.761515+0000","last_clean_scrub_stamp":"2024-02-23T00:57:31.244160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":61,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:21.472647+0000","last_change":"2024-02-23T00:58:21.472617+0000","last_active":"2024-02-23T00:58:21.472647+0000","last_peered":"2024-02-23T00:58:21.472647+0000","last_clean":"2024-02-23T00:58:21.472647+0000","last_became_active":"2024-02-23T00:56:22.400716+0000","last_became_peered":"2024-02-23T00:56:22.400716+0000","last_unstale":"2024-02-23T00:58:21.472647+0000","last_undegraded":"2024-02-23T00:58:21.472647+0000","last_fullsized":"2024-02-23T00:58:21.472647+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:21.472522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:06.491793+0000","last_clean_scrub_stamp":"2024-02-23T00:58:21.472522+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":803,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.441150+0000","last_change":"2024-02-23T00:57:14.354767+0000","last_active":"2024-02-23T00:58:20.441150+0000","last_peered":"2024-02-23T00:58:20.441150+0000","last_clean":"2024-02-23T00:58:20.441150+0000","last_became_active":"2024-02-23T00:53:53.417585+0000","last_became_peered":"2024-02-23T00:53:53.417585+0000","last_unstale":"2024-02-23T00:58:20.441150+0000","last_undegraded":"2024-02-23T00:58:20.441150+0000","last_fullsized":"2024-02-23T00:58:20.441150+0000","mapping_epoch":693,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":694,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:14.354702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:45:23.224663+0000","last_clean_scrub_stamp":"2024-02-23T00:57:14.354702+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"826'26","reported_seq":889,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.440335+0000","last_change":"2024-02-23T00:57:56.515550+0000","last_active":"2024-02-23T00:58:20.440335+0000","last_peered":"2024-02-23T00:58:20.440335+0000","last_clean":"2024-02-23T00:58:20.440335+0000","last_became_active":"2024-02-23T00:39:16.784606+0000","last_became_peered":"2024-02-23T00:39:16.784606+0000","last_unstale":"2024-02-23T00:58:20.440335+0000","last_undegraded":"2024-02-23T00:58:20.440335+0000","last_fullsized":"2024-02-23T00:58:20.440335+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"826'26","last_scrub_stamp":"2024-02-23T00:57:56.515465+0000","last_deep_scrub":"826'26","last_deep_scrub_stamp":"2024-02-23T00:57:56.515465+0000","last_clean_scrub_stamp":"2024-02-23T00:57:56.515465+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":697,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.437221+0000","last_change":"2024-02-23T00:57:19.139683+0000","last_active":"2024-02-23T00:58:20.437221+0000","last_peered":"2024-02-23T00:58:20.437221+0000","last_clean":"2024-02-23T00:58:20.437221+0000","last_became_active":"2024-02-23T00:57:13.546182+0000","last_became_peered":"2024-02-23T00:57:13.546182+0000","last_unstale":"2024-02-23T00:58:20.437221+0000","last_undegraded":"2024-02-23T00:58:20.437221+0000","last_fullsized":"2024-02-23T00:58:20.437221+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:19.139612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:55:27.856495+0000","last_clean_scrub_stamp":"2024-02-23T00:57:19.139612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":824,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.440182+0000","last_change":"2024-02-23T00:57:42.894678+0000","last_active":"2024-02-23T00:58:20.440182+0000","last_peered":"2024-02-23T00:58:20.440182+0000","last_clean":"2024-02-23T00:58:20.440182+0000","last_became_active":"2024-02-23T00:56:59.508932+0000","last_became_peered":"2024-02-23T00:56:59.508932+0000","last_unstale":"2024-02-23T00:58:20.440182+0000","last_undegraded":"2024-02-23T00:58:20.440182+0000","last_fullsized":"2024-02-23T00:58:20.440182+0000","mapping_epoch":837,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":838,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:42.894617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:46:47.226331+0000","last_clean_scrub_stamp":"2024-02-23T00:57:42.894617+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":131,"reported_epoch":860,"state":"active+clean","last_fresh":"2024-02-23T00:58:34.902268+0000","last_change":"2024-02-23T00:58:34.902268+0000","last_active":"2024-02-23T00:58:34.902268+0000","last_peered":"2024-02-23T00:58:34.902268+0000","last_clean":"2024-02-23T00:58:34.902268+0000","last_became_active":"2024-02-23T00:54:56.486558+0000","last_became_peered":"2024-02-23T00:54:56.486558+0000","last_unstale":"2024-02-23T00:58:34.902268+0000","last_undegraded":"2024-02-23T00:58:34.902268+0000","last_fullsized":"2024-02-23T00:58:34.902268+0000","mapping_epoch":747,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":748,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:34.902194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:58:34.902194+0000","last_clean_scrub_stamp":"2024-02-23T00:58:34.902194+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"457'62","reported_seq":665,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.438802+0000","last_change":"2024-02-23T00:57:42.854820+0000","last_active":"2024-02-23T00:58:20.438802+0000","last_peered":"2024-02-23T00:58:20.438802+0000","last_clean":"2024-02-23T00:58:20.438802+0000","last_became_active":"2024-02-23T00:57:13.551475+0000","last_became_peered":"2024-02-23T00:57:13.551475+0000","last_unstale":"2024-02-23T00:58:20.438802+0000","last_undegraded":"2024-02-23T00:58:20.438802+0000","last_fullsized":"2024-02-23T00:58:20.438802+0000","mapping_epoch":843,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":844,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'62","last_scrub_stamp":"2024-02-23T00:57:42.854752+0000","last_deep_scrub":"88'50","last_deep_scrub_stamp":"2024-02-23T00:40:52.840702+0000","last_clean_scrub_stamp":"2024-02-23T00:57:42.854752+0000","log_size":62,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":91,"num_read_kb":78,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":149,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.673180+0000","last_change":"2024-02-23T00:57:49.205317+0000","last_active":"2024-02-23T00:58:20.673180+0000","last_peered":"2024-02-23T00:58:20.673180+0000","last_clean":"2024-02-23T00:58:20.673180+0000","last_became_active":"2024-02-23T00:56:22.396132+0000","last_became_peered":"2024-02-23T00:56:22.396132+0000","last_unstale":"2024-02-23T00:58:20.673180+0000","last_undegraded":"2024-02-23T00:58:20.673180+0000","last_fullsized":"2024-02-23T00:58:20.673180+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:49.205251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:40:58.873013+0000","last_clean_scrub_stamp":"2024-02-23T00:57:49.205251+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":846,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.440447+0000","last_change":"2024-02-23T00:57:12.393446+0000","last_active":"2024-02-23T00:58:20.440447+0000","last_peered":"2024-02-23T00:58:20.440447+0000","last_clean":"2024-02-23T00:58:20.440447+0000","last_became_active":"2024-02-23T00:55:45.323003+0000","last_became_peered":"2024-02-23T00:55:45.323003+0000","last_unstale":"2024-02-23T00:58:20.440447+0000","last_undegraded":"2024-02-23T00:58:20.440447+0000","last_fullsized":"2024-02-23T00:58:20.440447+0000","mapping_epoch":789,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":790,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:57:12.393376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:39:17.770632+0000","last_clean_scrub_stamp":"2024-02-23T00:57:12.393376+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":831,"reported_epoch":859,"state":"active+clean","last_fresh":"2024-02-23T00:58:20.616248+0000","last_change":"2024-02-23T00:58:20.129837+0000","last_active":"2024-02-23T00:58:20.616248+0000","last_peered":"2024-02-23T00:58:20.616248+0000","last_clean":"2024-02-23T00:58:20.616248+0000","last_became_active":"2024-02-23T00:56:22.402412+0000","last_became_peered":"2024-02-23T00:56:22.402412+0000","last_unstale":"2024-02-23T00:58:20.616248+0000","last_undegraded":"2024-02-23T00:58:20.616248+0000","last_fullsized":"2024-02-23T00:58:20.616248+0000","mapping_epoch":817,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":818,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-23T00:58:20.129758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-23T00:49:44.601086+0000","last_clean_scrub_stamp":"2024-02-23T00:58:20.129758+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0],"acting":[5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":12,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":36,"acting":36,"num_store_stats":8},{"poolid":5,"num_pg":42,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":126,"acting":126,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1498,"num_read_kb":1223,"num_write":895,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":422,"num_bytes_recovered":8258,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1010,"ondisk_log_size":1010,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1082,"num_read_kb":1077,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2830,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234221,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308096,"kb_used_data":2200,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93437056,"statfs":{"total":95995035648,"available":95679545344,"internally_reserved":0,"allocated":2252800,"data_stored":777962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:57:45 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.53200000000000003,"15min":0.52100000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.23300000000000001},"max":{"1min":0.80600000000000005,"5min":0.82999999999999996,"15min":4.9420000000000002},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.504,"5min":0.52900000000000003,"15min":0.52400000000000002},"min":{"1min":0.35299999999999998,"5min":0.27200000000000002,"15min":0.24299999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":4.9729999999999999},"last":0.624}]},{"osd":1,"last update":"Fri Feb 23 00:57:45 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.54000000000000004,"15min":0.60499999999999998},"min":{"1min":0.33200000000000002,"5min":0.315,"15min":0.27200000000000002},"max":{"1min":0.875,"5min":0.88200000000000001,"15min":10.786},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.55100000000000005,"15min":0.622},"min":{"1min":0.36099999999999999,"5min":0.33400000000000002,"15min":0.28100000000000003},"max":{"1min":0.84299999999999997,"5min":1.0609999999999999,"15min":11.885999999999999},"last":0.66200000000000003}]},{"osd":2,"last update":"Fri Feb 23 00:57:45 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53900000000000003,"15min":0.54800000000000004},"min":{"1min":0.375,"5min":0.33000000000000002,"15min":0.28999999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":5.0789999999999997},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.56699999999999995,"15min":0.55700000000000005},"min":{"1min":0.38800000000000001,"5min":0.33200000000000002,"15min":0.316},"max":{"1min":3.1339999999999999,"5min":3.1339999999999999,"15min":5.1280000000000001},"last":0.56200000000000006}]},{"osd":3,"last update":"Fri Feb 23 00:57:45 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56000000000000005,"15min":0.629},"min":{"1min":0.34999999999999998,"5min":0.33600000000000002,"15min":0.29399999999999998},"max":{"1min":0.88700000000000001,"5min":1.7030000000000001,"15min":15.712999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.58599999999999997,"15min":0.626},"min":{"1min":0.379,"5min":0.35599999999999998,"15min":0.28100000000000003},"max":{"1min":0.84799999999999998,"5min":1.8720000000000001,"15min":9.4090000000000007},"last":0.67500000000000004}]},{"osd":4,"last update":"Fri Feb 23 00:57:45 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.44600000000000001,"15min":0.441},"min":{"1min":0.29799999999999999,"5min":0.25800000000000001,"15min":0.23499999999999999},"max":{"1min":0.72499999999999998,"5min":0.79500000000000004,"15min":2.0350000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.46200000000000002,"15min":0.48999999999999999},"min":{"1min":0.31900000000000001,"5min":0.26700000000000002,"15min":0.248},"max":{"1min":0.71099999999999997,"5min":0.85299999999999998,"15min":11.236000000000001},"last":0.40400000000000003}]},{"osd":5,"last update":"Fri Feb 23 00:57:45 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.47399999999999998,"15min":0.48599999999999999},"min":{"1min":0.29099999999999998,"5min":0.28799999999999998,"15min":0.27000000000000002},"max":{"1min":0.77000000000000002,"5min":0.83099999999999996,"15min":5.4000000000000004},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.5,"15min":0.502},"min":{"1min":0.34699999999999998,"5min":0.29199999999999998,"15min":0.29099999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":1.728},"last":0.51300000000000001}]},{"osd":6,"last update":"Fri Feb 23 00:57:45 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.36199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.77000000000000002,"5min":0.77200000000000002,"15min":1.6339999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.50800000000000001,"15min":0.54700000000000004},"min":{"1min":0.36199999999999999,"5min":0.316,"15min":0.29299999999999998},"max":{"1min":0.73999999999999999,"5min":0.82499999999999996,"15min":13.395},"last":0.58999999999999997}]}]},{"osd":6,"up_from":47,"seq":201863463156,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":303508,"kb_used_data":2188,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93441644,"statfs":{"total":95995035648,"available":95684243456,"internally_reserved":0,"allocated":2240512,"data_stored":777686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:58:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.53600000000000003,"15min":0.67000000000000004},"min":{"1min":0.36699999999999999,"5min":0.27800000000000002,"15min":0.21099999999999999},"max":{"1min":0.73599999999999999,"5min":0.81799999999999995,"15min":30.641999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54000000000000004,"15min":0.69999999999999996},"min":{"1min":0.374,"5min":0.29899999999999999,"15min":0.23899999999999999},"max":{"1min":0.70999999999999996,"5min":1.268,"15min":29.292000000000002},"last":0.60999999999999999}]},{"osd":1,"last update":"Fri Feb 23 00:58:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52800000000000002,"15min":0.68600000000000005},"min":{"1min":0.41299999999999998,"5min":0.27800000000000002,"15min":0.247},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":28.186},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54000000000000004,"15min":0.873},"min":{"1min":0.41599999999999998,"5min":0.33700000000000002,"15min":0.26900000000000002},"max":{"1min":0.73499999999999999,"5min":1.2529999999999999,"15min":33.104999999999997},"last":0.56299999999999994}]},{"osd":2,"last update":"Fri Feb 23 00:58:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.72299999999999998},"min":{"1min":0.40799999999999997,"5min":0.34100000000000003,"15min":0.27400000000000002},"max":{"1min":0.78800000000000003,"5min":4.1319999999999997,"15min":19.960000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53700000000000003,"15min":0.624},"min":{"1min":0.38200000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68600000000000005,"5min":1.161,"15min":8.5860000000000003},"last":0.45200000000000001}]},{"osd":3,"last update":"Fri Feb 23 00:58:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55200000000000005,"15min":0.627},"min":{"1min":0.436,"5min":0.37,"15min":0.28599999999999998},"max":{"1min":0.66900000000000004,"5min":1.236,"15min":19.562000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56999999999999995,"15min":0.66500000000000004},"min":{"1min":0.40899999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.76100000000000001,"5min":2.7839999999999998,"15min":14.199},"last":0.51900000000000002}]},{"osd":4,"last update":"Fri Feb 23 00:58:20 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.44500000000000001,"15min":0.48599999999999999},"min":{"1min":0.30499999999999999,"5min":0.25800000000000001,"15min":0.218},"max":{"1min":0.59299999999999997,"5min":0.72299999999999998,"15min":11.109999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.47099999999999997,"15min":0.55100000000000005},"min":{"1min":0.32500000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.66500000000000004,"5min":0.72399999999999998,"15min":16.734000000000002},"last":0.39100000000000001}]},{"osd":5,"last update":"Fri Feb 23 00:58:20 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47399999999999998,"15min":0.52600000000000002},"min":{"1min":0.33800000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.63600000000000001,"5min":0.749,"15min":14.154999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.49299999999999999,"15min":0.56299999999999994},"min":{"1min":0.30599999999999999,"5min":0.27400000000000002,"15min":0.26400000000000001},"max":{"1min":0.71399999999999997,"5min":0.74099999999999999,"15min":8.1400000000000006},"last":0.54600000000000004}]},{"osd":7,"last update":"Fri Feb 23 00:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.505,"15min":0.53100000000000003},"min":{"1min":0.35599999999999998,"5min":0.27400000000000002,"15min":0.26800000000000002},"max":{"1min":0.78900000000000003,"5min":0.80200000000000005,"15min":8.548},"last":0.499},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54300000000000004,"15min":0.61099999999999999},"min":{"1min":0.35699999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.76300000000000001,"5min":0.77800000000000002,"15min":15.618},"last":0.624}]}]},{"osd":1,"up_from":13,"seq":55834575118,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":304016,"kb_used_data":2216,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93441136,"statfs":{"total":95995035648,"available":95683723264,"internally_reserved":0,"allocated":2269184,"data_stored":779539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:58:05 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36599999999999999,"15min":0.69899999999999995},"min":{"1min":0.249,"5min":0.19500000000000001,"15min":0.17999999999999999},"max":{"1min":0.63500000000000001,"5min":0.68799999999999994,"15min":78.361000000000004},"last":0.31},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40000000000000002,"15min":0.45100000000000001},"min":{"1min":0.254,"5min":0.23699999999999999,"15min":0.216},"max":{"1min":0.64900000000000002,"5min":0.80600000000000005,"15min":5.8849999999999998},"last":0.39100000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:58:27 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.36899999999999999,"15min":0.67400000000000004},"min":{"1min":0.26400000000000001,"5min":0.20599999999999999,"15min":0.20499999999999999},"max":{"1min":0.44400000000000001,"5min":1.3580000000000001,"15min":48.606000000000002},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.41999999999999998,"15min":0.57699999999999996},"min":{"1min":0.27400000000000002,"5min":0.25700000000000001,"15min":0.23699999999999999},"max":{"1min":0.58299999999999996,"5min":0.95199999999999996,"15min":23.911000000000001},"last":0.435}]},{"osd":3,"last update":"Fri Feb 23 00:58:05 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.435,"15min":0.58399999999999996},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.21199999999999999},"max":{"1min":0.73399999999999999,"5min":1.01,"15min":35.244999999999997},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.45600000000000002,"15min":0.69699999999999995},"min":{"1min":0.34000000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.3919999999999999,"5min":2.2349999999999999,"15min":56.853999999999999},"last":0.40999999999999998}]},{"osd":4,"last update":"Fri Feb 23 00:58:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56999999999999995,"15min":0.77900000000000003},"min":{"1min":0.41899999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80000000000000004,"5min":0.95699999999999996,"15min":37.646999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58899999999999997,"15min":0.70099999999999996},"min":{"1min":0.435,"5min":0.35899999999999999,"15min":0.308},"max":{"1min":0.80500000000000005,"5min":0.99299999999999999,"15min":9.6859999999999999},"last":0.67900000000000005}]},{"osd":5,"last update":"Fri Feb 23 00:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.81799999999999995},"min":{"1min":0.42199999999999999,"5min":0.35199999999999998,"15min":0.309},"max":{"1min":0.90800000000000003,"5min":0.94799999999999995,"15min":48.792000000000002},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.625,"15min":0.68500000000000005},"min":{"1min":0.45200000000000001,"5min":0.371,"15min":0.34100000000000003},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":11.872999999999999},"last":0.69799999999999995}]},{"osd":6,"last update":"Fri Feb 23 00:58:22 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57299999999999995,"15min":0.79700000000000004},"min":{"1min":0.45800000000000002,"5min":0.34200000000000003,"15min":0.318},"max":{"1min":0.69799999999999995,"5min":0.87,"15min":40.741},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60099999999999998,"15min":0.64200000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.34000000000000002},"max":{"1min":0.93100000000000005,"5min":1.0389999999999999,"15min":6.0069999999999997},"last":0.71599999999999997}]},{"osd":7,"last update":"Fri Feb 23 00:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.61599999999999999,"15min":0.70499999999999996},"min":{"1min":0.44700000000000001,"5min":0.41899999999999998,"15min":0.36699999999999999},"max":{"1min":1.0680000000000001,"5min":1.0780000000000001,"15min":20.225000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60599999999999998,"15min":0.75800000000000001},"min":{"1min":0.45300000000000001,"5min":0.38800000000000001,"15min":0.35999999999999999},"max":{"1min":0.86799999999999999,"5min":0.88300000000000001,"15min":20.254999999999999},"last":0.70699999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738644,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":303528,"kb_used_data":2208,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93441624,"statfs":{"total":95995035648,"available":95684222976,"internally_reserved":0,"allocated":2260992,"data_stored":778556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 23 00:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.379,"15min":0.58699999999999997},"min":{"1min":0.223,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.53100000000000003,"5min":0.89300000000000002,"15min":28.856999999999999},"last":0.434},{"interface":"front","average":{"1min":0.437,"5min":0.42099999999999999,"15min":0.63700000000000001},"min":{"1min":0.26000000000000001,"5min":0.216,"15min":0.20699999999999999},"max":{"1min":0.72899999999999998,"5min":1.8140000000000001,"15min":20.577000000000002},"last":0.39200000000000002}]},{"osd":2,"last update":"Fri Feb 23 00:58:34 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.40999999999999998,"15min":0.58699999999999997},"min":{"1min":0.26700000000000002,"5min":0.252,"15min":0.222},"max":{"1min":0.53400000000000003,"5min":0.79600000000000004,"15min":13.863},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.39100000000000001,"15min":0.50900000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.21099999999999999},"max":{"1min":0.51700000000000002,"5min":1.1619999999999999,"15min":10.455},"last":0.36899999999999999}]},{"osd":3,"last update":"Fri Feb 23 00:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41299999999999998,"15min":0.73199999999999998},"min":{"1min":0.29699999999999999,"5min":0.23799999999999999,"15min":0.22700000000000001},"max":{"1min":0.66200000000000003,"5min":0.73099999999999998,"15min":35.566000000000003},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.44500000000000001,"15min":0.72299999999999998},"min":{"1min":0.34699999999999998,"5min":0.28100000000000003,"15min":0.26100000000000001},"max":{"1min":0.67400000000000004,"5min":0.80100000000000005,"15min":20.702999999999999},"last":0.44800000000000001}]},{"osd":4,"last update":"Fri Feb 23 00:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.63100000000000001,"15min":0.71099999999999997},"min":{"1min":0.34300000000000003,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.746,"5min":2.2320000000000002,"15min":10.584},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.65900000000000003,"15min":0.76300000000000001},"min":{"1min":0.45600000000000002,"5min":0.36899999999999999,"15min":0.35099999999999998},"max":{"1min":0.86699999999999999,"5min":2.2090000000000001,"15min":13.198},"last":0.54300000000000004}]},{"osd":5,"last update":"Fri Feb 23 00:57:52 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.63,"15min":0.73099999999999998},"min":{"1min":0.39600000000000002,"5min":0.33800000000000002,"15min":0.32200000000000001},"max":{"1min":0.71799999999999997,"5min":2.1150000000000002,"15min":10.397},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.68700000000000006,"15min":0.79200000000000004},"min":{"1min":0.41899999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84299999999999997,"5min":2.2509999999999999,"15min":17.297000000000001},"last":0.59299999999999997}]},{"osd":6,"last update":"Fri Feb 23 00:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65200000000000002,"15min":0.751},"min":{"1min":0.38,"5min":0.38,"15min":0.378},"max":{"1min":0.89000000000000001,"5min":1.2549999999999999,"15min":17.373999999999999},"last":0.505},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.70799999999999996,"15min":0.78600000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.36799999999999999},"max":{"1min":0.93700000000000006,"5min":4.4859999999999998,"15min":13.973000000000001},"last":0.64100000000000001}]},{"osd":7,"last update":"Fri Feb 23 00:57:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68300000000000005,"15min":0.78900000000000003},"min":{"1min":0.432,"5min":0.38200000000000001,"15min":0.35199999999999998},"max":{"1min":0.88500000000000001,"5min":2.8260000000000001,"15min":17.399999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.749,"15min":0.82199999999999995},"min":{"1min":0.48299999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81299999999999994,"5min":6.0229999999999997,"15min":16.427},"last":0.66700000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411593,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":303992,"kb_used_data":2192,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93441160,"statfs":{"total":95995035648,"available":95683747840,"internally_reserved":0,"allocated":2244608,"data_stored":777678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.437,"15min":0.502},"min":{"1min":0.26100000000000001,"5min":0.22500000000000001,"15min":0.20499999999999999},"max":{"1min":0.505,"5min":5.0439999999999996,"15min":13.657},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.39000000000000001,"15min":0.64900000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.222},"max":{"1min":0.65700000000000003,"5min":1.117,"15min":42.343000000000004},"last":0.377}]},{"osd":1,"last update":"Fri Feb 23 00:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.39000000000000001,"15min":0.56899999999999995},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.19700000000000001},"max":{"1min":0.53900000000000003,"5min":0.70999999999999996,"15min":18.952000000000002},"last":0.313},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.44900000000000001,"15min":0.52700000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.623,"5min":4.9980000000000002,"15min":15.391},"last":0.35599999999999998}]},{"osd":3,"last update":"Fri Feb 23 00:58:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.44700000000000001,"15min":0.64900000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.22500000000000001},"max":{"1min":0.504,"5min":2.762,"15min":25.635999999999999},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.438,"15min":0.61799999999999999},"min":{"1min":0.29899999999999999,"5min":0.28000000000000003,"15min":0.26300000000000001},"max":{"1min":0.65300000000000002,"5min":1.534,"15min":19.960000000000001},"last":0.36699999999999999}]},{"osd":4,"last update":"Fri Feb 23 00:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.63600000000000001,"15min":0.73099999999999998},"min":{"1min":0.496,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":1.1719999999999999,"5min":1.224,"15min":13.673},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.64900000000000002,"15min":0.755},"min":{"1min":0.443,"5min":0.34899999999999998,"15min":0.315},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":10.103999999999999},"last":0.53600000000000003}]},{"osd":5,"last update":"Fri Feb 23 00:57:47 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.61699999999999999,"15min":0.69299999999999995},"min":{"1min":0.42899999999999999,"5min":0.34699999999999998,"15min":0.29899999999999999},"max":{"1min":0.85999999999999999,"5min":1.5389999999999999,"15min":15.612},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.65300000000000002,"15min":0.70299999999999996},"min":{"1min":0.41699999999999998,"5min":0.38100000000000001,"15min":0.35099999999999998},"max":{"1min":0.875,"5min":1.8839999999999999,"15min":13.632999999999999},"last":0.60199999999999998}]},{"osd":6,"last update":"Fri Feb 23 00:58:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.67200000000000004,"15min":0.68300000000000005},"min":{"1min":0.48299999999999998,"5min":0.39900000000000002,"15min":0.39000000000000001},"max":{"1min":0.81899999999999995,"5min":3.2360000000000002,"15min":11.544},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.65100000000000002,"15min":0.76300000000000001},"min":{"1min":0.46100000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81399999999999995,"5min":1.319,"15min":10.132999999999999},"last":0.437}]},{"osd":7,"last update":"Fri Feb 23 00:57:39 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67000000000000004,"15min":0.71899999999999997},"min":{"1min":0.48499999999999999,"5min":0.40400000000000003,"15min":0.377},"max":{"1min":0.91600000000000004,"5min":1.9239999999999999,"15min":11.599},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.64600000000000002,"15min":0.78700000000000003},"min":{"1min":0.44400000000000001,"5min":0.41699999999999998,"15min":0.374},"max":{"1min":0.86499999999999999,"5min":1.1679999999999999,"15min":10.148999999999999},"last":0.57199999999999995}]}]},{"osd":3,"up_from":26,"seq":111669149954,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308124,"kb_used_data":2228,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93437028,"statfs":{"total":95995035648,"available":95679516672,"internally_reserved":0,"allocated":2281472,"data_stored":778543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:58:04 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.35699999999999998,"15min":0.51300000000000001},"min":{"1min":0.252,"5min":0.21199999999999999,"15min":0.20899999999999999},"max":{"1min":0.62,"5min":0.63,"15min":12.212999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.40699999999999997,"15min":0.59099999999999997},"min":{"1min":0.245,"5min":0.245,"15min":0.21199999999999999},"max":{"1min":0.53100000000000003,"5min":1.3089999999999999,"15min":11.714},"last":0.41199999999999998}]},{"osd":1,"last update":"Fri Feb 23 00:58:04 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.39800000000000002,"15min":0.60699999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.19900000000000001},"max":{"1min":0.60699999999999998,"5min":1.1539999999999999,"15min":11.742000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41499999999999998,"15min":0.58999999999999997},"min":{"1min":0.309,"5min":0.218,"15min":0.19900000000000001},"max":{"1min":0.60999999999999999,"5min":1.7470000000000001,"15min":8.2270000000000003},"last":0.50600000000000001}]},{"osd":2,"last update":"Fri Feb 23 00:58:04 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.437,"15min":0.66400000000000003},"min":{"1min":0.27300000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.60399999999999998,"5min":1.2430000000000001,"15min":13.359999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.41299999999999998,"15min":0.64400000000000002},"min":{"1min":0.28999999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.60799999999999998,"5min":0.82599999999999996,"15min":21.122},"last":0.439}]},{"osd":4,"last update":"Fri Feb 23 00:58:27 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.58999999999999997,"15min":0.745},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.29899999999999999},"max":{"1min":0.92400000000000004,"5min":2.4780000000000002,"15min":11.644},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59799999999999998,"15min":0.74199999999999999},"min":{"1min":0.40699999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.95399999999999996,"5min":2.4119999999999999,"15min":12.430999999999999},"last":0.997}]},{"osd":5,"last update":"Fri Feb 23 00:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59399999999999997,"15min":0.68700000000000006},"min":{"1min":0.441,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.874,"5min":2.4540000000000002,"15min":10.231999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.628,"15min":0.80200000000000005},"min":{"1min":0.45300000000000001,"5min":0.36799999999999999,"15min":0.35499999999999998},"max":{"1min":0.99199999999999999,"5min":2.3679999999999999,"15min":11.648},"last":0.76200000000000001}]},{"osd":6,"last update":"Fri Feb 23 00:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.628,"15min":0.749},"min":{"1min":0.443,"5min":0.377,"15min":0.35199999999999998},"max":{"1min":0.85999999999999999,"5min":2.464,"15min":6.9420000000000002},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.629,"15min":0.72199999999999998},"min":{"1min":0.41699999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.94499999999999995,"5min":2.4249999999999998,"15min":10.965999999999999},"last":0.67500000000000004}]},{"osd":7,"last update":"Fri Feb 23 00:57:43 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66100000000000003,"15min":0.83999999999999997},"min":{"1min":0.49099999999999999,"5min":0.42999999999999999,"15min":0.38800000000000001},"max":{"1min":0.96099999999999997,"5min":2.444,"15min":11.676},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.66400000000000003,"15min":0.81599999999999995},"min":{"1min":0.433,"5min":0.41499999999999998,"15min":0.41299999999999998},"max":{"1min":0.83799999999999997,"5min":2.5009999999999999,"15min":7.9800000000000004},"last":0.95199999999999996}]}]},{"osd":4,"up_from":32,"seq":137438953726,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":303532,"kb_used_data":2212,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93441620,"statfs":{"total":95995035648,"available":95684218880,"internally_reserved":0,"allocated":2265088,"data_stored":778187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:58:22 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.501,"15min":0.504},"min":{"1min":0.30199999999999999,"5min":0.25800000000000001,"15min":0.223},"max":{"1min":0.623,"5min":0.79400000000000004,"15min":3.5219999999999998},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55400000000000005,"15min":0.57199999999999995},"min":{"1min":0.372,"5min":0.32200000000000001,"15min":0.23999999999999999},"max":{"1min":0.67100000000000004,"5min":2.5859999999999999,"15min":8.6140000000000008},"last":0.373}]},{"osd":1,"last update":"Fri Feb 23 00:58:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51000000000000001,"15min":0.56899999999999995},"min":{"1min":0.317,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.69699999999999995,"5min":1.23,"15min":9.6110000000000007},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53100000000000003,"15min":0.56899999999999995},"min":{"1min":0.41799999999999998,"5min":0.308,"15min":0.255},"max":{"1min":0.76700000000000002,"5min":0.79600000000000004,"15min":6.6950000000000003},"last":0.48199999999999998}]},{"osd":2,"last update":"Fri Feb 23 00:58:22 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51900000000000002,"15min":0.54900000000000004},"min":{"1min":0.34200000000000003,"5min":0.30099999999999999,"15min":0.26500000000000001},"max":{"1min":0.73599999999999999,"5min":0.78600000000000003,"15min":3.3069999999999999},"last":0.438},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54600000000000004,"15min":0.56100000000000005},"min":{"1min":0.36399999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.70899999999999996,"5min":1.6779999999999999,"15min":5.8440000000000003},"last":0.44800000000000001}]},{"osd":3,"last update":"Fri Feb 23 00:58:22 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.57099999999999995,"15min":0.628},"min":{"1min":0.35099999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.74099999999999999,"5min":2.6150000000000002,"15min":9.1280000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54100000000000004,"15min":0.57199999999999995},"min":{"1min":0.38900000000000001,"5min":0.315,"15min":0.29699999999999999},"max":{"1min":0.68999999999999995,"5min":1.288,"15min":4.0220000000000002},"last":0.51400000000000001}]},{"osd":5,"last update":"Fri Feb 23 00:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46400000000000002,"15min":0.442},"min":{"1min":0.25600000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.73399999999999999,"5min":1.627,"15min":1.694},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48499999999999999,"15min":0.48899999999999999},"min":{"1min":0.33200000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.69199999999999995,"5min":0.84699999999999998,"15min":1.663},"last":0.52600000000000002}]},{"osd":6,"last update":"Fri Feb 23 00:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.46600000000000003,"15min":0.442},"min":{"1min":0.28399999999999997,"5min":0.28100000000000003,"15min":0.253},"max":{"1min":0.77300000000000002,"5min":1.704,"15min":1.704},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.317,"5min":0.26500000000000001,"15min":0.215},"max":{"1min":0.76700000000000002,"5min":1.264,"15min":2.2120000000000002},"last":0.59799999999999998}]},{"osd":7,"last update":"Fri Feb 23 00:57:43 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.48699999999999999,"15min":0.47999999999999998},"min":{"1min":0.33500000000000002,"5min":0.27100000000000002,"15min":0.26700000000000002},"max":{"1min":0.69199999999999995,"5min":1.363,"15min":2.294},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56000000000000005,"15min":0.54700000000000004},"min":{"1min":0.38200000000000001,"5min":0.33000000000000002,"15min":0.32600000000000001},"max":{"1min":0.89900000000000002,"5min":1.726,"15min":2.282},"last":0.61699999999999999}]}]},{"osd":5,"up_from":39,"seq":167503724793,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":304012,"kb_used_data":2212,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93441140,"statfs":{"total":95995035648,"available":95683727360,"internally_reserved":0,"allocated":2265088,"data_stored":779316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 23 00:57:39 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.55500000000000005,"15min":0.63600000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.246},"max":{"1min":0.89100000000000001,"5min":5.6920000000000002,"15min":23.881},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.50800000000000001,"15min":0.59099999999999997},"min":{"1min":0.27200000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.79900000000000004,"5min":2.4300000000000002,"15min":5.4649999999999999},"last":0.40200000000000002}]},{"osd":1,"last update":"Fri Feb 23 00:57:39 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.52000000000000002,"15min":0.65800000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.254},"max":{"1min":0.95799999999999996,"5min":1.659,"15min":18.175999999999998},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51500000000000001,"15min":0.625},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.23899999999999999},"max":{"1min":0.72999999999999998,"5min":1.3080000000000001,"15min":9.0950000000000006},"last":0.373}]},{"osd":2,"last update":"Fri Feb 23 00:57:39 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.55000000000000004,"15min":0.71499999999999997},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.75600000000000001,"5min":4.492,"15min":22.282},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56999999999999995,"15min":0.73999999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.29199999999999998},"max":{"1min":0.91500000000000004,"5min":2.7389999999999999,"15min":23.925000000000001},"last":0.36299999999999999}]},{"osd":3,"last update":"Fri Feb 23 00:57:39 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.53400000000000003,"15min":0.626},"min":{"1min":0.36699999999999999,"5min":0.34300000000000003,"15min":0.30399999999999999},"max":{"1min":1.446,"5min":1.446,"15min":6.5730000000000004},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.58899999999999997,"15min":0.66900000000000004},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.93700000000000006,"5min":2.8090000000000002,"15min":8.1760000000000002},"last":0.47499999999999998}]},{"osd":4,"last update":"Fri Feb 23 00:57:39 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.443,"15min":0.44900000000000001},"min":{"1min":0.23100000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.24,"5min":1.24,"15min":4.593},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.47599999999999998,"15min":0.49099999999999999},"min":{"1min":0.28399999999999997,"5min":0.255,"15min":0.255},"max":{"1min":0.65500000000000003,"5min":1.744,"15min":3.879},"last":0.41999999999999998}]},{"osd":6,"last update":"Fri Feb 23 00:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48599999999999999,"15min":0.49199999999999999},"min":{"1min":0.32000000000000001,"5min":0.28499999999999998,"15min":0.27900000000000003},"max":{"1min":0.69699999999999995,"5min":1.5389999999999999,"15min":4.6740000000000004},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52400000000000002,"15min":0.498},"min":{"1min":0.36899999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.71899999999999997,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.53500000000000003}]},{"osd":7,"last update":"Fri Feb 23 00:57:39 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54800000000000004,"15min":0.55300000000000005},"min":{"1min":0.38600000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.74099999999999999,"5min":1.756,"15min":7.0300000000000002},"last":0.495},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.52300000000000002,"15min":0.54000000000000004},"min":{"1min":0.40400000000000003,"5min":0.314,"15min":0.29099999999999998},"max":{"1min":2.1920000000000002,"5min":2.1920000000000002,"15min":9.0579999999999998},"last":0.432}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-23T00:58:38.279 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-02-23T00:58:38.279 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:eb66ed921e744301e6863be6353618d63967ea59 shell --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 -- ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-02-23T00:58:38.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:37.448917+0000 mgr.y (mgr.14517) 775 : cluster [DBG] pgmap v1358: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:58:38.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:58:37.517348+0000 mgr.y (mgr.14517) 776 : audit [DBG] from='client.32145 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:58:38.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:37.652658+0000 osd.7 (osd.7) 453 : cluster [DBG] 18.13 deep-scrub starts 2024-02-23T00:58:38.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:37.656334+0000 osd.7 (osd.7) 454 : cluster [DBG] 18.13 deep-scrub ok 2024-02-23T00:58:38.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:38.210148+0000 osd.5 (osd.5) 440 : cluster [DBG] 4.8 scrub starts 2024-02-23T00:58:38.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:38.212302+0000 osd.5 (osd.5) 441 : cluster [DBG] 4.8 scrub ok 2024-02-23T00:58:38.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:37.448917+0000 mgr.y (mgr.14517) 775 : cluster [DBG] pgmap v1358: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:58:38.558 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:58:37.517348+0000 mgr.y (mgr.14517) 776 : audit [DBG] from='client.32145 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:58:38.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:37.652658+0000 osd.7 (osd.7) 453 : cluster [DBG] 18.13 deep-scrub starts 2024-02-23T00:58:38.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:37.656334+0000 osd.7 (osd.7) 454 : cluster [DBG] 18.13 deep-scrub ok 2024-02-23T00:58:38.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:38.210148+0000 osd.5 (osd.5) 440 : cluster [DBG] 4.8 scrub starts 2024-02-23T00:58:38.559 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:38 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:38.212302+0000 osd.5 (osd.5) 441 : cluster [DBG] 4.8 scrub ok 2024-02-23T00:58:38.566 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/mon.a/config 2024-02-23T00:58:38.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:37.448917+0000 mgr.y (mgr.14517) 775 : cluster [DBG] pgmap v1358: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:58:38.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:58:37.517348+0000 mgr.y (mgr.14517) 776 : audit [DBG] from='client.32145 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-23T00:58:38.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:37.652658+0000 osd.7 (osd.7) 453 : cluster [DBG] 18.13 deep-scrub starts 2024-02-23T00:58:38.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:37.656334+0000 osd.7 (osd.7) 454 : cluster [DBG] 18.13 deep-scrub ok 2024-02-23T00:58:38.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:38.210148+0000 osd.5 (osd.5) 440 : cluster [DBG] 4.8 scrub starts 2024-02-23T00:58:38.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:38.212302+0000 osd.5 (osd.5) 441 : cluster [DBG] 4.8 scrub ok 2024-02-23T00:58:39.710 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:38.726815+0000 osd.1 (osd.1) 374 : cluster [DBG] 6.4 scrub starts 2024-02-23T00:58:39.710 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:38.730361+0000 osd.1 (osd.1) 375 : cluster [DBG] 6.4 scrub ok 2024-02-23T00:58:39.710 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:39.187303+0000 osd.5 (osd.5) 442 : cluster [DBG] 5.1b scrub starts 2024-02-23T00:58:39.710 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:39.188232+0000 osd.5 (osd.5) 443 : cluster [DBG] 5.1b scrub ok 2024-02-23T00:58:39.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:38.726815+0000 osd.1 (osd.1) 374 : cluster [DBG] 6.4 scrub starts 2024-02-23T00:58:39.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:38.730361+0000 osd.1 (osd.1) 375 : cluster [DBG] 6.4 scrub ok 2024-02-23T00:58:39.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:39.187303+0000 osd.5 (osd.5) 442 : cluster [DBG] 5.1b scrub starts 2024-02-23T00:58:39.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:39 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:39.188232+0000 osd.5 (osd.5) 443 : cluster [DBG] 5.1b scrub ok 2024-02-23T00:58:39.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:38.726815+0000 osd.1 (osd.1) 374 : cluster [DBG] 6.4 scrub starts 2024-02-23T00:58:39.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:38.730361+0000 osd.1 (osd.1) 375 : cluster [DBG] 6.4 scrub ok 2024-02-23T00:58:39.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:39.187303+0000 osd.5 (osd.5) 442 : cluster [DBG] 5.1b scrub starts 2024-02-23T00:58:39.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:39.188232+0000 osd.5 (osd.5) 443 : cluster [DBG] 5.1b scrub ok 2024-02-23T00:58:40.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.243+0000 7fda31205700 1 -- 172.21.15.47:0/913583181 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda2c0fd510 msgr2=0x7fda2c0f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:40.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.243+0000 7fda31205700 1 --2- 172.21.15.47:0/913583181 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda2c0fd510 0x7fda2c0f6d40 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 rx=0x7fda1c0047b0 tx=0x7fda1c009cb0).stop 2024-02-23T00:58:40.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.243+0000 7fda31205700 1 -- 172.21.15.47:0/913583181 shutdown_connections 2024-02-23T00:58:40.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.243+0000 7fda31205700 1 --2- 172.21.15.47:0/913583181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda2c0f7400 0x7fda2c103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:40.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.243+0000 7fda31205700 1 --2- 172.21.15.47:0/913583181 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fda2c0fcbb0 0x7fda2c0fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:40.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.243+0000 7fda31205700 1 --2- 172.21.15.47:0/913583181 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda2c0fd510 0x7fda2c0f6d40 unknown :-1 s=CLOSED pgs=2694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:40.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.243+0000 7fda31205700 1 -- 172.21.15.47:0/913583181 >> 172.21.15.47:0/913583181 conn(0x7fda2c0f2820 msgr2=0x7fda2c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:58:40.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.243+0000 7fda31205700 1 -- 172.21.15.47:0/913583181 shutdown_connections 2024-02-23T00:58:40.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.243+0000 7fda31205700 1 -- 172.21.15.47:0/913583181 wait complete. 2024-02-23T00:58:40.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.243+0000 7fda31205700 1 librados: starting msgr at 2024-02-23T00:58:40.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.243+0000 7fda31205700 1 librados: starting objecter 2024-02-23T00:58:40.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.244+0000 7fda31205700 1 Processor -- start 2024-02-23T00:58:40.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.244+0000 7fda31205700 1 -- start start 2024-02-23T00:58:40.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.244+0000 7fda31205700 1 librados: setting wanted keys 2024-02-23T00:58:40.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.244+0000 7fda31205700 1 librados: calling monclient init 2024-02-23T00:58:40.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.244+0000 7fda31205700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda2c0f7400 0x7fda2c10e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:40.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.244+0000 7fda31205700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fda2c0fcbb0 0x7fda2c10e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:40.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.244+0000 7fda31205700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda2c0fd510 0x7fda2c108280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:40.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.244+0000 7fda31205700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c108770 con 0x7fda2c0f7400 2024-02-23T00:58:40.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.244+0000 7fda31205700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c1088e0 con 0x7fda2c0fd510 2024-02-23T00:58:40.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.244+0000 7fda31205700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c108aa0 con 0x7fda2c0fcbb0 2024-02-23T00:58:40.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.244+0000 7fda2a59c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fda2c0fcbb0 0x7fda2c10e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:40.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.244+0000 7fda2ad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda2c0f7400 0x7fda2c10e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:40.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.244+0000 7fda2a59c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fda2c0fcbb0 0x7fda2c10e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52850/0 (socket says 172.21.15.47:52850) 2024-02-23T00:58:40.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.244+0000 7fda2a59c700 1 -- 172.21.15.47:0/2165950233 learned_addr learned my addr 172.21.15.47:0/2165950233 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-02-23T00:58:40.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.244+0000 7fda2b59e700 1 --2- 172.21.15.47:0/2165950233 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda2c0fd510 0x7fda2c108280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:40.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.245+0000 7fda2ad9d700 1 -- 172.21.15.47:0/2165950233 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fda2c0fcbb0 msgr2=0x7fda2c10e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:40.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.245+0000 7fda2ad9d700 1 --2- 172.21.15.47:0/2165950233 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fda2c0fcbb0 0x7fda2c10e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:40.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.245+0000 7fda2ad9d700 1 -- 172.21.15.47:0/2165950233 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda2c0fd510 msgr2=0x7fda2c108280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:40.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.245+0000 7fda2ad9d700 1 --2- 172.21.15.47:0/2165950233 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda2c0fd510 0x7fda2c108280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:40.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.245+0000 7fda2ad9d700 1 -- 172.21.15.47:0/2165950233 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda20005040 con 0x7fda2c0f7400 2024-02-23T00:58:40.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.245+0000 7fda2a59c700 1 --2- 172.21.15.47:0/2165950233 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fda2c0fcbb0 0x7fda2c10e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-23T00:58:40.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.245+0000 7fda2b59e700 1 --2- 172.21.15.47:0/2165950233 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda2c0fd510 0x7fda2c108280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-23T00:58:40.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.245+0000 7fda2ad9d700 1 --2- 172.21.15.47:0/2165950233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda2c0f7400 0x7fda2c10e090 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 rx=0x7fda200091f0 tx=0x7fda20009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:58:40.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.245+0000 7fda13fff700 10 client.?.objecter ms_handle_connect 0x7fda2c0f7400 2024-02-23T00:58:40.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.245+0000 7fda13fff700 10 client.?.objecter resend_mon_ops 2024-02-23T00:58:40.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.245+0000 7fda13fff700 1 -- 172.21.15.47:0/2165950233 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda200079e0 con 0x7fda2c0f7400 2024-02-23T00:58:40.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.245+0000 7fda13fff700 1 -- 172.21.15.47:0/2165950233 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(24 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fda2000f040 con 0x7fda2c0f7400 2024-02-23T00:58:40.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.245+0000 7fda31205700 1 -- 172.21.15.47:0/2165950233 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda1c005040 con 0x7fda2c0f7400 2024-02-23T00:58:40.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.245+0000 7fda13fff700 1 -- 172.21.15.47:0/2165950233 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda200147e0 con 0x7fda2c0f7400 2024-02-23T00:58:40.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.245+0000 7fda31205700 10 client.32151.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-23T00:58:40.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.245+0000 7fda31205700 1 -- 172.21.15.47:0/2165950233 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda2c1090e0 con 0x7fda2c0f7400 2024-02-23T00:58:40.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.245+0000 7fda31205700 1 librados: init done 2024-02-23T00:58:40.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.246+0000 7fda31205700 1 -- 172.21.15.47:0/2165950233 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda2c05fa60 con 0x7fda2c0f7400 2024-02-23T00:58:40.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.246+0000 7fda13fff700 1 -- 172.21.15.47:0/2165950233 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fda20014c10 con 0x7fda2c0f7400 2024-02-23T00:58:40.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.247+0000 7fda13fff700 1 --2- 172.21.15.47:0/2165950233 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fda14062cc0 0x7fda14065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-23T00:58:40.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.247+0000 7fda13fff700 1 -- 172.21.15.47:0/2165950233 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(860..860 src has 806..860) v4 ==== 8663+0+0 (secure 0 0 0) 0x7fda2007d050 con 0x7fda2c0f7400 2024-02-23T00:58:40.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.247+0000 7fda13fff700 10 client.32151.objecter ms_dispatch 0x7fda2c001550 osd_map(860..860 src has 806..860) v4 2024-02-23T00:58:40.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.247+0000 7fda13fff700 3 client.32151.objecter handle_osd_map got epochs [860,860] > 0 2024-02-23T00:58:40.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.247+0000 7fda13fff700 3 client.32151.objecter handle_osd_map decoding full epoch 860 2024-02-23T00:58:40.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.247+0000 7fda2a59c700 1 --2- 172.21.15.47:0/2165950233 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fda14062cc0 0x7fda14065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-23T00:58:40.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.247+0000 7fda13fff700 20 client.32151.objecter dump_active .. 0 homeless 2024-02-23T00:58:40.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.247+0000 7fda13fff700 10 client.32151.objecter ms_handle_connect 0x7fda14062cc0 2024-02-23T00:58:40.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.247+0000 7fda2a59c700 1 --2- 172.21.15.47:0/2165950233 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fda14062cc0 0x7fda14065180 secure :-1 s=READY pgs=3262 cs=0 l=1 rev1=1 rx=0x7fda1c005010 tx=0x7fda1c009cb0).ready entity=mgr.14517 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-23T00:58:40.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.251+0000 7fda13fff700 1 -- 172.21.15.47:0/2165950233 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fda20049f60 con 0x7fda2c0f7400 2024-02-23T00:58:40.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.427+0000 7fda31205700 1 -- 172.21.15.47:0/2165950233 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"} v 0) v1 -- 0x7fda2c109710 con 0x7fda2c0f7400 2024-02-23T00:58:40.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.431+0000 7fda13fff700 1 -- 172.21.15.47:0/2165950233 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]=0 v30) v1 ==== 118+0+0 (secure 0 0 0) 0x7fda2004c890 con 0x7fda2c0f7400 2024-02-23T00:58:40.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.433+0000 7fda31205700 10 librados: watch_flush enter 2024-02-23T00:58:40.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.433+0000 7fda31205700 10 librados: watch_flush exit 2024-02-23T00:58:40.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.433+0000 7fda31205700 20 client.32151.objecter shutdown clearing up homeless session... 2024-02-23T00:58:40.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.433+0000 7fda31205700 10 client.32151.objecter successfully canceled tick 2024-02-23T00:58:40.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.433+0000 7fda31205700 1 -- 172.21.15.47:0/2165950233 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fda14062cc0 msgr2=0x7fda14065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:40.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.433+0000 7fda31205700 1 --2- 172.21.15.47:0/2165950233 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fda14062cc0 0x7fda14065180 secure :-1 s=READY pgs=3262 cs=0 l=1 rev1=1 rx=0x7fda1c005010 tx=0x7fda1c009cb0).stop 2024-02-23T00:58:40.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.433+0000 7fda31205700 1 -- 172.21.15.47:0/2165950233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda2c0f7400 msgr2=0x7fda2c10e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-23T00:58:40.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.433+0000 7fda31205700 1 --2- 172.21.15.47:0/2165950233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda2c0f7400 0x7fda2c10e090 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 rx=0x7fda200091f0 tx=0x7fda20009cb0).stop 2024-02-23T00:58:40.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.434+0000 7fda31205700 1 -- 172.21.15.47:0/2165950233 shutdown_connections 2024-02-23T00:58:40.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.434+0000 7fda31205700 1 --2- 172.21.15.47:0/2165950233 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda2c0fd510 0x7fda2c108280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:40.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.434+0000 7fda31205700 1 --2- 172.21.15.47:0/2165950233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda2c0f7400 0x7fda2c10e090 unknown :-1 s=CLOSED pgs=2695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:40.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.434+0000 7fda31205700 1 --2- 172.21.15.47:0/2165950233 >> [v2:172.21.15.47:6800/1636898748,v1:172.21.15.47:6801/1636898748] conn(0x7fda14062cc0 0x7fda14065180 unknown :-1 s=CLOSED pgs=3262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:40.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.434+0000 7fda31205700 1 --2- 172.21.15.47:0/2165950233 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fda2c0fcbb0 0x7fda2c10e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-23T00:58:40.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.434+0000 7fda31205700 1 -- 172.21.15.47:0/2165950233 >> 172.21.15.47:0/2165950233 conn(0x7fda2c0f2820 msgr2=0x7fda2c060db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-23T00:58:40.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.434+0000 7fda31205700 1 -- 172.21.15.47:0/2165950233 shutdown_connections 2024-02-23T00:58:40.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.434+0000 7fda31205700 1 -- 172.21.15.47:0/2165950233 wait complete. 2024-02-23T00:58:40.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-02-23T00:58:40.434+0000 7fda31205700 1 librados: shutdown 2024-02-23T00:58:40.803 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:38.836581+0000 osd.0 (osd.0) 374 : cluster [DBG] 4.3 scrub starts 2024-02-23T00:58:40.803 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:38.838709+0000 osd.0 (osd.0) 375 : cluster [DBG] 4.3 scrub ok 2024-02-23T00:58:40.803 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:39.449491+0000 mgr.y (mgr.14517) 777 : cluster [DBG] pgmap v1359: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:58:40.803 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:39.771730+0000 osd.1 (osd.1) 376 : cluster [DBG] 18.1b deep-scrub starts 2024-02-23T00:58:40.804 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:39.775957+0000 osd.1 (osd.1) 377 : cluster [DBG] 18.1b deep-scrub ok 2024-02-23T00:58:40.804 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:40.148438+0000 osd.5 (osd.5) 444 : cluster [DBG] 6.7 deep-scrub starts 2024-02-23T00:58:40.804 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:40.151421+0000 osd.5 (osd.5) 445 : cluster [DBG] 6.7 deep-scrub ok 2024-02-23T00:58:40.804 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:58:40.428765+0000 mon.a (mon.0) 3952 : audit [INF] from='client.? 172.21.15.47:0/2165950233' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-02-23T00:58:40.804 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:58:40.432680+0000 mon.a (mon.0) 3953 : audit [INF] from='client.? 172.21.15.47:0/2165950233' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-02-23T00:58:40.804 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:58:40.464879+0000 mon.a (mon.0) 3954 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:58:40.804 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:58:40.466221+0000 mon.a (mon.0) 3955 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:58:40.804 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:58:40.466813+0000 mon.a (mon.0) 3956 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:58:40.804 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: audit 2024-02-23T00:58:40.471234+0000 mon.a (mon.0) 3957 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:58:40.805 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:58:40] ENGINE Bus STOPPING 2024-02-23T00:58:40.805 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:58:40] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-23T00:58:40.805 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:58:40] ENGINE Bus STOPPED 2024-02-23T00:58:40.805 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:58:40] ENGINE Bus STARTING 2024-02-23T00:58:40.805 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:58:40] "GET /metrics HTTP/1.1" 200 187143 "" "Prometheus/2.33.4" 2024-02-23T00:58:40.805 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:58:40] ENGINE Serving on http://:::9283 2024-02-23T00:58:40.805 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:58:40] ENGINE Bus STARTED 2024-02-23T00:58:40.805 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:38.836581+0000 osd.0 (osd.0) 374 : cluster [DBG] 4.3 scrub starts 2024-02-23T00:58:40.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:38.838709+0000 osd.0 (osd.0) 375 : cluster [DBG] 4.3 scrub ok 2024-02-23T00:58:40.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:39.449491+0000 mgr.y (mgr.14517) 777 : cluster [DBG] pgmap v1359: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:58:40.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:39.771730+0000 osd.1 (osd.1) 376 : cluster [DBG] 18.1b deep-scrub starts 2024-02-23T00:58:40.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:39.775957+0000 osd.1 (osd.1) 377 : cluster [DBG] 18.1b deep-scrub ok 2024-02-23T00:58:40.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:40.148438+0000 osd.5 (osd.5) 444 : cluster [DBG] 6.7 deep-scrub starts 2024-02-23T00:58:40.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:40.151421+0000 osd.5 (osd.5) 445 : cluster [DBG] 6.7 deep-scrub ok 2024-02-23T00:58:40.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:58:40.428765+0000 mon.a (mon.0) 3952 : audit [INF] from='client.? 172.21.15.47:0/2165950233' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-02-23T00:58:40.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:58:40.432680+0000 mon.a (mon.0) 3953 : audit [INF] from='client.? 172.21.15.47:0/2165950233' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-02-23T00:58:40.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:58:40.464879+0000 mon.a (mon.0) 3954 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:58:40.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:58:40.466221+0000 mon.a (mon.0) 3955 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:58:40.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:58:40.466813+0000 mon.a (mon.0) 3956 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:58:40.806 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: audit 2024-02-23T00:58:40.471234+0000 mon.a (mon.0) 3957 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:58:40.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:38.836581+0000 osd.0 (osd.0) 374 : cluster [DBG] 4.3 scrub starts 2024-02-23T00:58:40.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:38.838709+0000 osd.0 (osd.0) 375 : cluster [DBG] 4.3 scrub ok 2024-02-23T00:58:40.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:39.449491+0000 mgr.y (mgr.14517) 777 : cluster [DBG] pgmap v1359: 154 pgs: 154 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-23T00:58:40.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:39.771730+0000 osd.1 (osd.1) 376 : cluster [DBG] 18.1b deep-scrub starts 2024-02-23T00:58:40.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:39.775957+0000 osd.1 (osd.1) 377 : cluster [DBG] 18.1b deep-scrub ok 2024-02-23T00:58:40.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:40.148438+0000 osd.5 (osd.5) 444 : cluster [DBG] 6.7 deep-scrub starts 2024-02-23T00:58:40.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:40.151421+0000 osd.5 (osd.5) 445 : cluster [DBG] 6.7 deep-scrub ok 2024-02-23T00:58:40.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:58:40.428765+0000 mon.a (mon.0) 3952 : audit [INF] from='client.? 172.21.15.47:0/2165950233' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-02-23T00:58:40.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:58:40.432680+0000 mon.a (mon.0) 3953 : audit [INF] from='client.? 172.21.15.47:0/2165950233' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-02-23T00:58:40.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:58:40.464879+0000 mon.a (mon.0) 3954 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-23T00:58:40.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:58:40.466221+0000 mon.a (mon.0) 3955 : audit [DBG] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-23T00:58:40.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:58:40.466813+0000 mon.a (mon.0) 3956 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-23T00:58:40.945 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: audit 2024-02-23T00:58:40.471234+0000 mon.a (mon.0) 3957 : audit [INF] from='mgr.14517 172.21.15.47:0/2516441833' entity='mgr.y' 2024-02-23T00:58:41.250 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-23T00:58:41.261 INFO:tasks.cephadm:Teardown begin 2024-02-23T00:58:41.261 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-23T00:58:41.295 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-23T00:58:41.333 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-23T00:58:41.333 DEBUG:teuthology.orchestra.run.smithi047:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-23T00:58:41.355 DEBUG:teuthology.orchestra.run.smithi196:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-23T00:58:41.393 INFO:tasks.cephadm:Stopping all daemons... 2024-02-23T00:58:41.394 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-02-23T00:58:41.394 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@mon.a 2024-02-23T00:58:41.500 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:41 smithi047 systemd[1]: Stopping Ceph mon.a for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T00:58:41.500 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:41 smithi047 bash[260614]: Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon.a" found: no such container 2024-02-23T00:58:41.500 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:39.842211+0000 osd.0 (osd.0) 376 : cluster [DBG] 5.29 scrub starts 2024-02-23T00:58:41.500 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:39.843324+0000 osd.0 (osd.0) 377 : cluster [DBG] 5.29 scrub ok 2024-02-23T00:58:41.501 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:40.298180+0000 osd.4 (osd.4) 415 : cluster [DBG] 18.d scrub starts 2024-02-23T00:58:41.501 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:40.299866+0000 osd.4 (osd.4) 416 : cluster [DBG] 18.d scrub ok 2024-02-23T00:58:41.501 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:40.660928+0000 osd.6 (osd.6) 395 : cluster [DBG] 5.21 scrub starts 2024-02-23T00:58:41.501 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:40.661889+0000 osd.6 (osd.6) 396 : cluster [DBG] 5.21 scrub ok 2024-02-23T00:58:41.501 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:40.688187+0000 osd.7 (osd.7) 455 : cluster [DBG] 2.1d scrub starts 2024-02-23T00:58:41.501 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:40.691134+0000 osd.7 (osd.7) 456 : cluster [DBG] 2.1d scrub ok 2024-02-23T00:58:41.501 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:40.745341+0000 osd.1 (osd.1) 378 : cluster [DBG] 4.7 scrub starts 2024-02-23T00:58:41.501 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:40.747304+0000 osd.1 (osd.1) 379 : cluster [DBG] 4.7 scrub ok 2024-02-23T00:58:41.501 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:41.190546+0000 osd.5 (osd.5) 446 : cluster [DBG] 4.4 scrub starts 2024-02-23T00:58:41.501 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: cluster 2024-02-23T00:58:41.194140+0000 osd.5 (osd.5) 447 : cluster [DBG] 4.4 scrub ok 2024-02-23T00:58:41.502 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:39.842211+0000 osd.0 (osd.0) 376 : cluster [DBG] 5.29 scrub starts 2024-02-23T00:58:41.502 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:39.843324+0000 osd.0 (osd.0) 377 : cluster [DBG] 5.29 scrub ok 2024-02-23T00:58:41.502 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:40.298180+0000 osd.4 (osd.4) 415 : cluster [DBG] 18.d scrub starts 2024-02-23T00:58:41.502 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:40.299866+0000 osd.4 (osd.4) 416 : cluster [DBG] 18.d scrub ok 2024-02-23T00:58:41.502 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:40.660928+0000 osd.6 (osd.6) 395 : cluster [DBG] 5.21 scrub starts 2024-02-23T00:58:41.502 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:40.661889+0000 osd.6 (osd.6) 396 : cluster [DBG] 5.21 scrub ok 2024-02-23T00:58:41.502 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:40.688187+0000 osd.7 (osd.7) 455 : cluster [DBG] 2.1d scrub starts 2024-02-23T00:58:41.502 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:40.691134+0000 osd.7 (osd.7) 456 : cluster [DBG] 2.1d scrub ok 2024-02-23T00:58:41.502 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:40.745341+0000 osd.1 (osd.1) 378 : cluster [DBG] 4.7 scrub starts 2024-02-23T00:58:41.502 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:40.747304+0000 osd.1 (osd.1) 379 : cluster [DBG] 4.7 scrub ok 2024-02-23T00:58:41.503 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:41.190546+0000 osd.5 (osd.5) 446 : cluster [DBG] 4.4 scrub starts 2024-02-23T00:58:41.503 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: cluster 2024-02-23T00:58:41.194140+0000 osd.5 (osd.5) 447 : cluster [DBG] 4.4 scrub ok 2024-02-23T00:58:41.694 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:39.842211+0000 osd.0 (osd.0) 376 : cluster [DBG] 5.29 scrub starts 2024-02-23T00:58:41.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:39.843324+0000 osd.0 (osd.0) 377 : cluster [DBG] 5.29 scrub ok 2024-02-23T00:58:41.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:40.298180+0000 osd.4 (osd.4) 415 : cluster [DBG] 18.d scrub starts 2024-02-23T00:58:41.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:40.299866+0000 osd.4 (osd.4) 416 : cluster [DBG] 18.d scrub ok 2024-02-23T00:58:41.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:40.660928+0000 osd.6 (osd.6) 395 : cluster [DBG] 5.21 scrub starts 2024-02-23T00:58:41.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:40.661889+0000 osd.6 (osd.6) 396 : cluster [DBG] 5.21 scrub ok 2024-02-23T00:58:41.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:40.688187+0000 osd.7 (osd.7) 455 : cluster [DBG] 2.1d scrub starts 2024-02-23T00:58:41.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:40.691134+0000 osd.7 (osd.7) 456 : cluster [DBG] 2.1d scrub ok 2024-02-23T00:58:41.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:40.745341+0000 osd.1 (osd.1) 378 : cluster [DBG] 4.7 scrub starts 2024-02-23T00:58:41.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:40.747304+0000 osd.1 (osd.1) 379 : cluster [DBG] 4.7 scrub ok 2024-02-23T00:58:41.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:41.190546+0000 osd.5 (osd.5) 446 : cluster [DBG] 4.4 scrub starts 2024-02-23T00:58:41.695 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: cluster 2024-02-23T00:58:41.194140+0000 osd.5 (osd.5) 447 : cluster [DBG] 4.4 scrub ok 2024-02-23T00:58:41.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:58:41.702+0000 7f3c904fe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-23T00:58:41.807 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a[38306]: debug 2024-02-23T00:58:41.702+0000 7f3c904fe700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-02-23T00:58:41.808 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:58:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:58:41] ENGINE Bus STOPPING 2024-02-23T00:58:41.808 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:58:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:58:41] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-23T00:58:41.808 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:58:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:58:41] ENGINE Bus STOPPED 2024-02-23T00:58:41.808 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:58:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:58:41] ENGINE Bus STARTING 2024-02-23T00:58:42.184 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:58:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:58:41] ENGINE Serving on http://:::9283 2024-02-23T00:58:42.184 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:58:41 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:58:41] ENGINE Bus STARTED 2024-02-23T00:58:42.557 INFO:journalctl@ceph.mon.a.smithi047.stdout:Feb 23 00:58:42 smithi047 bash[260655]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-a 2024-02-23T00:58:42.752 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@mon.a.service' 2024-02-23T00:58:42.792 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-23T00:58:42.792 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-02-23T00:58:42.793 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-02-23T00:58:42.793 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@mon.c 2024-02-23T00:58:42.944 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:58:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x[41087]: ::ffff:172.21.15.196 - - [23/Feb/2024:00:58:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-23T00:58:43.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:42 smithi047 systemd[1]: Stopping Ceph mon.c for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T00:58:43.058 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:42 smithi047 bash[260787]: Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon.c" found: no such container 2024-02-23T00:58:43.444 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 00:58:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug there is no tcmu-runner data available 2024-02-23T00:58:43.557 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:58:43.157+0000 7efcba96f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-23T00:58:43.558 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c[44785]: debug 2024-02-23T00:58:43.157+0000 7efcba96f700 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-02-23T00:58:43.558 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:58:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:58:43] ENGINE Bus STOPPING 2024-02-23T00:58:43.558 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:58:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:58:43] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-23T00:58:43.558 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:58:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:58:43] ENGINE Bus STOPPED 2024-02-23T00:58:43.558 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:58:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:58:43] ENGINE Bus STARTING 2024-02-23T00:58:43.558 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:58:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:58:43] ENGINE Serving on http://:::9283 2024-02-23T00:58:43.558 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:58:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: [23/Feb/2024:00:58:43] ENGINE Bus STARTED 2024-02-23T00:58:44.057 INFO:journalctl@ceph.mon.c.smithi047.stdout:Feb 23 00:58:43 smithi047 bash[260828]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-c 2024-02-23T00:58:44.151 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@mon.c.service' 2024-02-23T00:58:44.188 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-23T00:58:44.188 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-02-23T00:58:44.188 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-02-23T00:58:44.188 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@mon.b 2024-02-23T00:58:44.604 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:44 smithi196 systemd[1]: Stopping Ceph mon.b for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T00:58:44.604 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:44 smithi196 bash[118344]: Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon.b" found: no such container 2024-02-23T00:58:44.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:58:44.600+0000 7fd3f7c72700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-23T00:58:44.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b[39619]: debug 2024-02-23T00:58:44.600+0000 7fd3f7c72700 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-02-23T00:58:45.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Feb 23 00:58:45 smithi196 bash[118384]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mon-b 2024-02-23T00:58:45.773 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@mon.b.service' 2024-02-23T00:58:45.817 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-23T00:58:45.817 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-02-23T00:58:45.817 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-02-23T00:58:45.817 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@mgr.y 2024-02-23T00:58:46.116 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:58:45 smithi047 systemd[1]: Stopping Ceph mgr.y for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T00:58:46.116 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:58:45 smithi047 bash[260958]: Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr.y" found: no such container 2024-02-23T00:58:46.418 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:58:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:58:46.113+0000 7f67974bb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-23T00:58:46.418 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:58:46 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y[38577]: debug 2024-02-23T00:58:46.113+0000 7f67974bb700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-02-23T00:58:46.807 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Feb 23 00:58:46 smithi047 bash[260998]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-y 2024-02-23T00:58:46.909 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@mgr.y.service' 2024-02-23T00:58:46.946 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-23T00:58:46.947 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-02-23T00:58:46.947 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-02-23T00:58:46.947 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@mgr.x 2024-02-23T00:58:47.444 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:58:46 smithi196 systemd[1]: Stopping Ceph mgr.x for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T00:58:47.444 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:58:47 smithi196 bash[118504]: Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr.x" found: no such container 2024-02-23T00:58:47.944 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Feb 23 00:58:47 smithi196 bash[118544]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-mgr-x 2024-02-23T00:58:48.058 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@mgr.x.service' 2024-02-23T00:58:48.094 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-23T00:58:48.094 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-02-23T00:58:48.095 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-23T00:58:48.095 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@osd.0 2024-02-23T00:58:48.411 INFO:journalctl@ceph.osd.0.smithi047.stdout:Feb 23 00:58:48 smithi047 systemd[1]: Stopping Ceph osd.0 for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T00:58:48.411 INFO:journalctl@ceph.osd.0.smithi047.stdout:Feb 23 00:58:48 smithi047 bash[261117]: Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.0" found: no such container 2024-02-23T00:58:48.807 INFO:journalctl@ceph.osd.0.smithi047.stdout:Feb 23 00:58:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-0[48294]: debug 2024-02-23T00:58:48.408+0000 7f8862ea9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-23T00:58:48.807 INFO:journalctl@ceph.osd.0.smithi047.stdout:Feb 23 00:58:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-0[48294]: debug 2024-02-23T00:58:48.408+0000 7f8862ea9700 -1 osd.0 860 *** Got signal Terminated *** 2024-02-23T00:58:48.807 INFO:journalctl@ceph.osd.0.smithi047.stdout:Feb 23 00:58:48 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-0[48294]: debug 2024-02-23T00:58:48.408+0000 7f8862ea9700 -1 osd.0 860 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-23T00:58:54.058 INFO:journalctl@ceph.osd.0.smithi047.stdout:Feb 23 00:58:53 smithi047 bash[261157]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-0 2024-02-23T00:58:55.985 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@osd.0.service' 2024-02-23T00:58:56.034 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-23T00:58:56.035 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-23T00:58:56.035 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-23T00:58:56.035 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@osd.1 2024-02-23T00:58:56.307 INFO:journalctl@ceph.osd.1.smithi047.stdout:Feb 23 00:58:56 smithi047 systemd[1]: Stopping Ceph osd.1 for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T00:58:56.308 INFO:journalctl@ceph.osd.1.smithi047.stdout:Feb 23 00:58:56 smithi047 bash[261392]: Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.1" found: no such container 2024-02-23T00:58:56.808 INFO:journalctl@ceph.osd.1.smithi047.stdout:Feb 23 00:58:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-1[52714]: debug 2024-02-23T00:58:56.330+0000 7f7fd6d66700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-23T00:58:56.808 INFO:journalctl@ceph.osd.1.smithi047.stdout:Feb 23 00:58:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-1[52714]: debug 2024-02-23T00:58:56.330+0000 7f7fd6d66700 -1 osd.1 860 *** Got signal Terminated *** 2024-02-23T00:58:56.808 INFO:journalctl@ceph.osd.1.smithi047.stdout:Feb 23 00:58:56 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-1[52714]: debug 2024-02-23T00:58:56.330+0000 7f7fd6d66700 -1 osd.1 860 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-23T00:59:00.558 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:59:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=warn ts=2024-02-23T00:59:00.142Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi047.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.47:8443: connect: connection refused" 2024-02-23T00:59:00.558 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:59:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=warn ts=2024-02-23T00:59:00.143Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-02-23T00:59:02.057 INFO:journalctl@ceph.osd.1.smithi047.stdout:Feb 23 00:59:01 smithi047 bash[261430]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-1 2024-02-23T00:59:03.737 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@osd.1.service' 2024-02-23T00:59:03.782 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-23T00:59:03.783 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-23T00:59:03.783 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-23T00:59:03.783 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@osd.2 2024-02-23T00:59:04.057 INFO:journalctl@ceph.osd.2.smithi047.stdout:Feb 23 00:59:03 smithi047 systemd[1]: Stopping Ceph osd.2 for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T00:59:04.058 INFO:journalctl@ceph.osd.2.smithi047.stdout:Feb 23 00:59:03 smithi047 bash[261677]: Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.2" found: no such container 2024-02-23T00:59:04.557 INFO:journalctl@ceph.osd.2.smithi047.stdout:Feb 23 00:59:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-2[57453]: debug 2024-02-23T00:59:04.091+0000 7f8f33085700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-23T00:59:04.558 INFO:journalctl@ceph.osd.2.smithi047.stdout:Feb 23 00:59:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-2[57453]: debug 2024-02-23T00:59:04.091+0000 7f8f33085700 -1 osd.2 860 *** Got signal Terminated *** 2024-02-23T00:59:04.558 INFO:journalctl@ceph.osd.2.smithi047.stdout:Feb 23 00:59:04 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-2[57453]: debug 2024-02-23T00:59:04.091+0000 7f8f33085700 -1 osd.2 860 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-23T00:59:09.808 INFO:journalctl@ceph.osd.2.smithi047.stdout:Feb 23 00:59:09 smithi047 bash[261715]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-2 2024-02-23T00:59:10.557 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:59:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=error ts=2024-02-23T00:59:10.142Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi047.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.47:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-02-23T00:59:10.557 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:59:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=warn ts=2024-02-23T00:59:10.142Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi047.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.47:8443: connect: connection refused" 2024-02-23T00:59:10.557 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:59:10 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=warn ts=2024-02-23T00:59:10.143Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-02-23T00:59:11.431 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@osd.2.service' 2024-02-23T00:59:11.478 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-23T00:59:11.478 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-23T00:59:11.478 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-02-23T00:59:11.479 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@osd.3 2024-02-23T00:59:11.763 INFO:journalctl@ceph.osd.3.smithi047.stdout:Feb 23 00:59:11 smithi047 systemd[1]: Stopping Ceph osd.3 for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T00:59:11.764 INFO:journalctl@ceph.osd.3.smithi047.stdout:Feb 23 00:59:11 smithi047 bash[261953]: Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.3" found: no such container 2024-02-23T00:59:12.057 INFO:journalctl@ceph.osd.3.smithi047.stdout:Feb 23 00:59:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-3[62151]: debug 2024-02-23T00:59:11.760+0000 7f7811740700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-23T00:59:12.057 INFO:journalctl@ceph.osd.3.smithi047.stdout:Feb 23 00:59:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-3[62151]: debug 2024-02-23T00:59:11.760+0000 7f7811740700 -1 osd.3 860 *** Got signal Terminated *** 2024-02-23T00:59:12.058 INFO:journalctl@ceph.osd.3.smithi047.stdout:Feb 23 00:59:11 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-3[62151]: debug 2024-02-23T00:59:11.760+0000 7f7811740700 -1 osd.3 860 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-23T00:59:14.694 INFO:journalctl@ceph.osd.4.smithi196.stdout:Feb 23 00:59:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-4[43838]: debug 2024-02-23T00:59:14.374+0000 7f1aba9e8700 -1 osd.4 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.232632+0000 front 2024-02-23T00:58:51.232700+0000 (oldest deadline 2024-02-23T00:59:13.532750+0000) 2024-02-23T00:59:14.695 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:14 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5[48426]: debug 2024-02-23T00:59:14.236+0000 7f06d96eb700 -1 osd.5 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:48.919720+0000 front 2024-02-23T00:58:48.919709+0000 (oldest deadline 2024-02-23T00:59:14.219713+0000) 2024-02-23T00:59:15.057 INFO:journalctl@ceph.osd.3.smithi047.stdout:Feb 23 00:59:14 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-3[62151]: debug 2024-02-23T00:59:14.771+0000 7f780eebe700 -1 osd.3 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:50.588709+0000 front 2024-02-23T00:58:50.588580+0000 (oldest deadline 2024-02-23T00:59:14.088856+0000) 2024-02-23T00:59:15.694 INFO:journalctl@ceph.osd.4.smithi196.stdout:Feb 23 00:59:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-4[43838]: debug 2024-02-23T00:59:15.375+0000 7f1aba9e8700 -1 osd.4 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.232632+0000 front 2024-02-23T00:58:51.232700+0000 (oldest deadline 2024-02-23T00:59:13.532750+0000) 2024-02-23T00:59:15.694 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:15 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5[48426]: debug 2024-02-23T00:59:15.212+0000 7f06d96eb700 -1 osd.5 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:48.919720+0000 front 2024-02-23T00:58:48.919709+0000 (oldest deadline 2024-02-23T00:59:14.219713+0000) 2024-02-23T00:59:16.057 INFO:journalctl@ceph.osd.3.smithi047.stdout:Feb 23 00:59:15 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-3[62151]: debug 2024-02-23T00:59:15.776+0000 7f780eebe700 -1 osd.3 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:50.588709+0000 front 2024-02-23T00:58:50.588580+0000 (oldest deadline 2024-02-23T00:59:14.088856+0000) 2024-02-23T00:59:16.694 INFO:journalctl@ceph.osd.4.smithi196.stdout:Feb 23 00:59:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-4[43838]: debug 2024-02-23T00:59:16.390+0000 7f1aba9e8700 -1 osd.4 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.232632+0000 front 2024-02-23T00:58:51.232700+0000 (oldest deadline 2024-02-23T00:59:13.532750+0000) 2024-02-23T00:59:16.694 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:16 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5[48426]: debug 2024-02-23T00:59:16.257+0000 7f06d96eb700 -1 osd.5 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:48.919720+0000 front 2024-02-23T00:58:48.919709+0000 (oldest deadline 2024-02-23T00:59:14.219713+0000) 2024-02-23T00:59:17.444 INFO:journalctl@ceph.osd.4.smithi196.stdout:Feb 23 00:59:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-4[43838]: debug 2024-02-23T00:59:17.423+0000 7f1aba9e8700 -1 osd.4 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.232632+0000 front 2024-02-23T00:58:51.232700+0000 (oldest deadline 2024-02-23T00:59:13.532750+0000) 2024-02-23T00:59:17.445 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5[48426]: debug 2024-02-23T00:59:17.216+0000 7f06d96eb700 -1 osd.5 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:48.919720+0000 front 2024-02-23T00:58:48.919709+0000 (oldest deadline 2024-02-23T00:59:14.219713+0000) 2024-02-23T00:59:17.445 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:17.188+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.861901+0000 front 2024-02-23T00:58:51.861916+0000 (oldest deadline 2024-02-23T00:59:17.161843+0000) 2024-02-23T00:59:17.557 INFO:journalctl@ceph.osd.3.smithi047.stdout:Feb 23 00:59:17 smithi047 bash[261994]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-3 2024-02-23T00:59:17.944 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:17 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:17.679+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.517693+0000 front 2024-02-23T00:58:51.517723+0000 (oldest deadline 2024-02-23T00:59:16.817507+0000) 2024-02-23T00:59:18.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Feb 23 00:59:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-4[43838]: debug 2024-02-23T00:59:18.449+0000 7f1aba9e8700 -1 osd.4 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.232632+0000 front 2024-02-23T00:58:51.232700+0000 (oldest deadline 2024-02-23T00:59:13.532750+0000) 2024-02-23T00:59:18.653 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:18.218+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.861901+0000 front 2024-02-23T00:58:51.861916+0000 (oldest deadline 2024-02-23T00:59:17.161843+0000) 2024-02-23T00:59:18.653 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5[48426]: debug 2024-02-23T00:59:18.207+0000 7f06d96eb700 -1 osd.5 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:48.919720+0000 front 2024-02-23T00:58:48.919709+0000 (oldest deadline 2024-02-23T00:59:14.219713+0000) 2024-02-23T00:59:18.944 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:18 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:18.649+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.517693+0000 front 2024-02-23T00:58:51.517723+0000 (oldest deadline 2024-02-23T00:59:16.817507+0000) 2024-02-23T00:59:18.957 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@osd.3.service' 2024-02-23T00:59:19.002 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-23T00:59:19.002 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-02-23T00:59:19.002 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-02-23T00:59:19.002 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@osd.4 2024-02-23T00:59:19.303 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:19.222+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.861901+0000 front 2024-02-23T00:58:51.861916+0000 (oldest deadline 2024-02-23T00:59:17.161843+0000) 2024-02-23T00:59:19.304 INFO:journalctl@ceph.osd.4.smithi196.stdout:Feb 23 00:59:19 smithi196 systemd[1]: Stopping Ceph osd.4 for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T00:59:19.304 INFO:journalctl@ceph.osd.4.smithi196.stdout:Feb 23 00:59:19 smithi196 bash[118677]: Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.4" found: no such container 2024-02-23T00:59:19.304 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5[48426]: debug 2024-02-23T00:59:19.233+0000 7f06d96eb700 -1 osd.5 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:48.919720+0000 front 2024-02-23T00:58:48.919709+0000 (oldest deadline 2024-02-23T00:59:14.219713+0000) 2024-02-23T00:59:19.672 INFO:journalctl@ceph.osd.4.smithi196.stdout:Feb 23 00:59:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-4[43838]: debug 2024-02-23T00:59:19.299+0000 7f1abda6b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-23T00:59:19.672 INFO:journalctl@ceph.osd.4.smithi196.stdout:Feb 23 00:59:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-4[43838]: debug 2024-02-23T00:59:19.299+0000 7f1abda6b700 -1 osd.4 860 *** Got signal Terminated *** 2024-02-23T00:59:19.672 INFO:journalctl@ceph.osd.4.smithi196.stdout:Feb 23 00:59:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-4[43838]: debug 2024-02-23T00:59:19.299+0000 7f1abda6b700 -1 osd.4 860 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-23T00:59:19.672 INFO:journalctl@ceph.osd.4.smithi196.stdout:Feb 23 00:59:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-4[43838]: debug 2024-02-23T00:59:19.427+0000 7f1aba9e8700 -1 osd.4 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.232632+0000 front 2024-02-23T00:58:51.232700+0000 (oldest deadline 2024-02-23T00:59:13.532750+0000) 2024-02-23T00:59:19.944 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:19.669+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.517693+0000 front 2024-02-23T00:58:51.517723+0000 (oldest deadline 2024-02-23T00:59:16.817507+0000) 2024-02-23T00:59:20.557 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:59:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=error ts=2024-02-23T00:59:20.142Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi047.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.47:8443: connect: connection refused" 2024-02-23T00:59:20.557 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:59:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=warn ts=2024-02-23T00:59:20.143Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi047.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.47:8443: connect: connection refused" 2024-02-23T00:59:20.558 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:59:20 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=warn ts=2024-02-23T00:59:20.143Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-02-23T00:59:20.657 INFO:journalctl@ceph.osd.4.smithi196.stdout:Feb 23 00:59:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-4[43838]: debug 2024-02-23T00:59:20.464+0000 7f1aba9e8700 -1 osd.4 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.232632+0000 front 2024-02-23T00:58:51.232700+0000 (oldest deadline 2024-02-23T00:59:13.532750+0000) 2024-02-23T00:59:20.658 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5[48426]: debug 2024-02-23T00:59:20.270+0000 7f06d96eb700 -1 osd.5 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:48.919720+0000 front 2024-02-23T00:58:48.919709+0000 (oldest deadline 2024-02-23T00:59:14.219713+0000) 2024-02-23T00:59:20.658 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:20.268+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.861901+0000 front 2024-02-23T00:58:51.861916+0000 (oldest deadline 2024-02-23T00:59:17.161843+0000) 2024-02-23T00:59:20.944 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:20 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:20.654+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.517693+0000 front 2024-02-23T00:58:51.517723+0000 (oldest deadline 2024-02-23T00:59:16.817507+0000) 2024-02-23T00:59:21.650 INFO:journalctl@ceph.osd.4.smithi196.stdout:Feb 23 00:59:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-4[43838]: debug 2024-02-23T00:59:21.427+0000 7f1aba9e8700 -1 osd.4 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.232632+0000 front 2024-02-23T00:58:51.232700+0000 (oldest deadline 2024-02-23T00:59:13.532750+0000) 2024-02-23T00:59:21.650 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5[48426]: debug 2024-02-23T00:59:21.304+0000 7f06d96eb700 -1 osd.5 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:48.919720+0000 front 2024-02-23T00:58:48.919709+0000 (oldest deadline 2024-02-23T00:59:14.219713+0000) 2024-02-23T00:59:21.650 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:21.227+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.861901+0000 front 2024-02-23T00:58:51.861916+0000 (oldest deadline 2024-02-23T00:59:17.161843+0000) 2024-02-23T00:59:21.944 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:21 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:21.647+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.517693+0000 front 2024-02-23T00:58:51.517723+0000 (oldest deadline 2024-02-23T00:59:16.817507+0000) 2024-02-23T00:59:22.674 INFO:journalctl@ceph.osd.4.smithi196.stdout:Feb 23 00:59:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-4[43838]: debug 2024-02-23T00:59:22.403+0000 7f1aba9e8700 -1 osd.4 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.232632+0000 front 2024-02-23T00:58:51.232700+0000 (oldest deadline 2024-02-23T00:59:13.532750+0000) 2024-02-23T00:59:22.675 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:22.201+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.861901+0000 front 2024-02-23T00:58:51.861916+0000 (oldest deadline 2024-02-23T00:59:17.161843+0000) 2024-02-23T00:59:22.675 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5[48426]: debug 2024-02-23T00:59:22.350+0000 7f06d96eb700 -1 osd.5 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:48.919720+0000 front 2024-02-23T00:58:48.919709+0000 (oldest deadline 2024-02-23T00:59:14.219713+0000) 2024-02-23T00:59:22.944 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:22.671+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.517693+0000 front 2024-02-23T00:58:51.517723+0000 (oldest deadline 2024-02-23T00:59:16.817507+0000) 2024-02-23T00:59:22.944 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:22 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:22.671+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:58:57.318234+0000 front 2024-02-23T00:58:57.318374+0000 (oldest deadline 2024-02-23T00:59:22.018351+0000) 2024-02-23T00:59:23.694 INFO:journalctl@ceph.osd.4.smithi196.stdout:Feb 23 00:59:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-4[43838]: debug 2024-02-23T00:59:23.363+0000 7f1aba9e8700 -1 osd.4 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.232632+0000 front 2024-02-23T00:58:51.232700+0000 (oldest deadline 2024-02-23T00:59:13.532750+0000) 2024-02-23T00:59:23.694 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5[48426]: debug 2024-02-23T00:59:23.305+0000 7f06d96eb700 -1 osd.5 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:48.919720+0000 front 2024-02-23T00:58:48.919709+0000 (oldest deadline 2024-02-23T00:59:14.219713+0000) 2024-02-23T00:59:23.695 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5[48426]: debug 2024-02-23T00:59:23.305+0000 7f06d96eb700 -1 osd.5 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.922622+0000 front 2024-02-23T00:59:00.922582+0000 (oldest deadline 2024-02-23T00:59:23.222465+0000) 2024-02-23T00:59:23.695 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:23.239+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.861901+0000 front 2024-02-23T00:58:51.861916+0000 (oldest deadline 2024-02-23T00:59:17.161843+0000) 2024-02-23T00:59:23.695 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:23.239+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.662876+0000 front 2024-02-23T00:59:00.662855+0000 (oldest deadline 2024-02-23T00:59:22.962673+0000) 2024-02-23T00:59:24.192 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:23.694+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.517693+0000 front 2024-02-23T00:58:51.517723+0000 (oldest deadline 2024-02-23T00:59:16.817507+0000) 2024-02-23T00:59:24.192 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:23 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:23.694+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:58:57.318234+0000 front 2024-02-23T00:58:57.318374+0000 (oldest deadline 2024-02-23T00:59:22.018351+0000) 2024-02-23T00:59:24.444 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5[48426]: debug 2024-02-23T00:59:24.286+0000 7f06d96eb700 -1 osd.5 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:48.919720+0000 front 2024-02-23T00:58:48.919709+0000 (oldest deadline 2024-02-23T00:59:14.219713+0000) 2024-02-23T00:59:24.444 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5[48426]: debug 2024-02-23T00:59:24.286+0000 7f06d96eb700 -1 osd.5 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.922622+0000 front 2024-02-23T00:59:00.922582+0000 (oldest deadline 2024-02-23T00:59:23.222465+0000) 2024-02-23T00:59:24.445 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:24.189+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.861901+0000 front 2024-02-23T00:58:51.861916+0000 (oldest deadline 2024-02-23T00:59:17.161843+0000) 2024-02-23T00:59:24.445 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:24.189+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.662876+0000 front 2024-02-23T00:59:00.662855+0000 (oldest deadline 2024-02-23T00:59:22.962673+0000) 2024-02-23T00:59:24.944 INFO:journalctl@ceph.osd.4.smithi196.stdout:Feb 23 00:59:24 smithi196 bash[118715]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-4 2024-02-23T00:59:24.945 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:24.692+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.517693+0000 front 2024-02-23T00:58:51.517723+0000 (oldest deadline 2024-02-23T00:59:16.817507+0000) 2024-02-23T00:59:24.945 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:24 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:24.693+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:58:57.318234+0000 front 2024-02-23T00:58:57.318374+0000 (oldest deadline 2024-02-23T00:59:22.018351+0000) 2024-02-23T00:59:25.505 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:25.219+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.861901+0000 front 2024-02-23T00:58:51.861916+0000 (oldest deadline 2024-02-23T00:59:17.161843+0000) 2024-02-23T00:59:25.505 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:25.219+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.662876+0000 front 2024-02-23T00:59:00.662855+0000 (oldest deadline 2024-02-23T00:59:22.962673+0000) 2024-02-23T00:59:25.505 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5[48426]: debug 2024-02-23T00:59:25.266+0000 7f06d96eb700 -1 osd.5 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:48.919720+0000 front 2024-02-23T00:58:48.919709+0000 (oldest deadline 2024-02-23T00:59:14.219713+0000) 2024-02-23T00:59:25.506 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5[48426]: debug 2024-02-23T00:59:25.266+0000 7f06d96eb700 -1 osd.5 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.922622+0000 front 2024-02-23T00:59:00.922582+0000 (oldest deadline 2024-02-23T00:59:23.222465+0000) 2024-02-23T00:59:25.927 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:25.666+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.517693+0000 front 2024-02-23T00:58:51.517723+0000 (oldest deadline 2024-02-23T00:59:16.817507+0000) 2024-02-23T00:59:25.927 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:25 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:25.666+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:58:57.318234+0000 front 2024-02-23T00:58:57.318374+0000 (oldest deadline 2024-02-23T00:59:22.018351+0000) 2024-02-23T00:59:26.539 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:26.228+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.861901+0000 front 2024-02-23T00:58:51.861916+0000 (oldest deadline 2024-02-23T00:59:17.161843+0000) 2024-02-23T00:59:26.539 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:26.228+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.662876+0000 front 2024-02-23T00:59:00.662855+0000 (oldest deadline 2024-02-23T00:59:22.962673+0000) 2024-02-23T00:59:26.540 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5[48426]: debug 2024-02-23T00:59:26.225+0000 7f06d96eb700 -1 osd.5 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:48.919720+0000 front 2024-02-23T00:58:48.919709+0000 (oldest deadline 2024-02-23T00:59:14.219713+0000) 2024-02-23T00:59:26.540 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5[48426]: debug 2024-02-23T00:59:26.225+0000 7f06d96eb700 -1 osd.5 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.922622+0000 front 2024-02-23T00:59:00.922582+0000 (oldest deadline 2024-02-23T00:59:23.222465+0000) 2024-02-23T00:59:26.547 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@osd.4.service' 2024-02-23T00:59:26.595 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-23T00:59:26.595 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-02-23T00:59:26.595 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-02-23T00:59:26.596 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@osd.5 2024-02-23T00:59:26.893 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:26.639+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.517693+0000 front 2024-02-23T00:58:51.517723+0000 (oldest deadline 2024-02-23T00:59:16.817507+0000) 2024-02-23T00:59:26.893 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:26.639+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:58:57.318234+0000 front 2024-02-23T00:58:57.318374+0000 (oldest deadline 2024-02-23T00:59:22.018351+0000) 2024-02-23T00:59:26.893 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:26 smithi196 systemd[1]: Stopping Ceph osd.5 for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T00:59:26.893 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:26 smithi196 bash[118953]: Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.5" found: no such container 2024-02-23T00:59:27.194 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5[48426]: debug 2024-02-23T00:59:26.889+0000 7f06de772700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-23T00:59:27.194 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5[48426]: debug 2024-02-23T00:59:26.889+0000 7f06de772700 -1 osd.5 860 *** Got signal Terminated *** 2024-02-23T00:59:27.194 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:26 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5[48426]: debug 2024-02-23T00:59:26.889+0000 7f06de772700 -1 osd.5 860 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-23T00:59:27.194 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5[48426]: debug 2024-02-23T00:59:27.191+0000 7f06d96eb700 -1 osd.5 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:48.919720+0000 front 2024-02-23T00:58:48.919709+0000 (oldest deadline 2024-02-23T00:59:14.219713+0000) 2024-02-23T00:59:27.641 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5[48426]: debug 2024-02-23T00:59:27.191+0000 7f06d96eb700 -1 osd.5 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.922622+0000 front 2024-02-23T00:59:00.922582+0000 (oldest deadline 2024-02-23T00:59:23.222465+0000) 2024-02-23T00:59:27.641 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:27.230+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.861901+0000 front 2024-02-23T00:58:51.861916+0000 (oldest deadline 2024-02-23T00:59:17.161843+0000) 2024-02-23T00:59:27.642 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:27.230+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.662876+0000 front 2024-02-23T00:59:00.662855+0000 (oldest deadline 2024-02-23T00:59:22.962673+0000) 2024-02-23T00:59:27.944 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:27.638+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.517693+0000 front 2024-02-23T00:58:51.517723+0000 (oldest deadline 2024-02-23T00:59:16.817507+0000) 2024-02-23T00:59:27.944 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:27 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:27.638+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:58:57.318234+0000 front 2024-02-23T00:58:57.318374+0000 (oldest deadline 2024-02-23T00:59:22.018351+0000) 2024-02-23T00:59:28.647 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5[48426]: debug 2024-02-23T00:59:28.226+0000 7f06d96eb700 -1 osd.5 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:48.919720+0000 front 2024-02-23T00:58:48.919709+0000 (oldest deadline 2024-02-23T00:59:14.219713+0000) 2024-02-23T00:59:28.647 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5[48426]: debug 2024-02-23T00:59:28.226+0000 7f06d96eb700 -1 osd.5 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.922622+0000 front 2024-02-23T00:59:00.922582+0000 (oldest deadline 2024-02-23T00:59:23.222465+0000) 2024-02-23T00:59:28.647 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:28.208+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.861901+0000 front 2024-02-23T00:58:51.861916+0000 (oldest deadline 2024-02-23T00:59:17.161843+0000) 2024-02-23T00:59:28.647 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:28.208+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.662876+0000 front 2024-02-23T00:59:00.662855+0000 (oldest deadline 2024-02-23T00:59:22.962673+0000) 2024-02-23T00:59:28.944 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:28.643+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.517693+0000 front 2024-02-23T00:58:51.517723+0000 (oldest deadline 2024-02-23T00:59:16.817507+0000) 2024-02-23T00:59:28.944 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:28 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:28.643+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:58:57.318234+0000 front 2024-02-23T00:58:57.318374+0000 (oldest deadline 2024-02-23T00:59:22.018351+0000) 2024-02-23T00:59:29.635 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:29.226+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.861901+0000 front 2024-02-23T00:58:51.861916+0000 (oldest deadline 2024-02-23T00:59:17.161843+0000) 2024-02-23T00:59:29.635 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:29.226+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.662876+0000 front 2024-02-23T00:59:00.662855+0000 (oldest deadline 2024-02-23T00:59:22.962673+0000) 2024-02-23T00:59:29.635 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5[48426]: debug 2024-02-23T00:59:29.275+0000 7f06d96eb700 -1 osd.5 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:48.919720+0000 front 2024-02-23T00:58:48.919709+0000 (oldest deadline 2024-02-23T00:59:14.219713+0000) 2024-02-23T00:59:29.635 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5[48426]: debug 2024-02-23T00:59:29.275+0000 7f06d96eb700 -1 osd.5 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.922622+0000 front 2024-02-23T00:59:00.922582+0000 (oldest deadline 2024-02-23T00:59:23.222465+0000) 2024-02-23T00:59:29.944 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:29.631+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.517693+0000 front 2024-02-23T00:58:51.517723+0000 (oldest deadline 2024-02-23T00:59:16.817507+0000) 2024-02-23T00:59:29.944 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:29 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:29.631+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:58:57.318234+0000 front 2024-02-23T00:58:57.318374+0000 (oldest deadline 2024-02-23T00:59:22.018351+0000) 2024-02-23T00:59:30.557 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:59:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=error ts=2024-02-23T00:59:30.143Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi047.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.47:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-02-23T00:59:30.558 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:59:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=warn ts=2024-02-23T00:59:30.143Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi047.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.47:8443: connect: connection refused" 2024-02-23T00:59:30.558 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:59:30 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=warn ts=2024-02-23T00:59:30.143Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-02-23T00:59:30.649 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5[48426]: debug 2024-02-23T00:59:30.245+0000 7f06d96eb700 -1 osd.5 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:48.919720+0000 front 2024-02-23T00:58:48.919709+0000 (oldest deadline 2024-02-23T00:59:14.219713+0000) 2024-02-23T00:59:30.649 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5[48426]: debug 2024-02-23T00:59:30.245+0000 7f06d96eb700 -1 osd.5 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.922622+0000 front 2024-02-23T00:59:00.922582+0000 (oldest deadline 2024-02-23T00:59:23.222465+0000) 2024-02-23T00:59:30.649 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:30.220+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.861901+0000 front 2024-02-23T00:58:51.861916+0000 (oldest deadline 2024-02-23T00:59:17.161843+0000) 2024-02-23T00:59:30.649 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:30.220+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.662876+0000 front 2024-02-23T00:59:00.662855+0000 (oldest deadline 2024-02-23T00:59:22.962673+0000) 2024-02-23T00:59:30.944 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:30.646+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.517693+0000 front 2024-02-23T00:58:51.517723+0000 (oldest deadline 2024-02-23T00:59:16.817507+0000) 2024-02-23T00:59:30.944 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:30 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:30.646+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:58:57.318234+0000 front 2024-02-23T00:58:57.318374+0000 (oldest deadline 2024-02-23T00:59:22.018351+0000) 2024-02-23T00:59:31.444 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5[48426]: debug 2024-02-23T00:59:31.221+0000 7f06d96eb700 -1 osd.5 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:48.919720+0000 front 2024-02-23T00:58:48.919709+0000 (oldest deadline 2024-02-23T00:59:14.219713+0000) 2024-02-23T00:59:31.444 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5[48426]: debug 2024-02-23T00:59:31.221+0000 7f06d96eb700 -1 osd.5 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.922622+0000 front 2024-02-23T00:59:00.922582+0000 (oldest deadline 2024-02-23T00:59:23.222465+0000) 2024-02-23T00:59:31.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:31.173+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.861901+0000 front 2024-02-23T00:58:51.861916+0000 (oldest deadline 2024-02-23T00:59:17.161843+0000) 2024-02-23T00:59:31.445 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:31.173+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.662876+0000 front 2024-02-23T00:59:00.662855+0000 (oldest deadline 2024-02-23T00:59:22.962673+0000) 2024-02-23T00:59:31.445 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:31.173+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6822 osd.2 since back 2024-02-23T00:59:08.263815+0000 front 2024-02-23T00:59:08.263772+0000 (oldest deadline 2024-02-23T00:59:31.163669+0000) 2024-02-23T00:59:32.131 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:31.691+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.517693+0000 front 2024-02-23T00:58:51.517723+0000 (oldest deadline 2024-02-23T00:59:16.817507+0000) 2024-02-23T00:59:32.132 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:31.691+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:58:57.318234+0000 front 2024-02-23T00:58:57.318374+0000 (oldest deadline 2024-02-23T00:59:22.018351+0000) 2024-02-23T00:59:32.132 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:31 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:31.691+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6822 osd.2 since back 2024-02-23T00:59:07.220084+0000 front 2024-02-23T00:59:07.220056+0000 (oldest deadline 2024-02-23T00:59:30.719952+0000) 2024-02-23T00:59:32.392 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:32.128+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.861901+0000 front 2024-02-23T00:58:51.861916+0000 (oldest deadline 2024-02-23T00:59:17.161843+0000) 2024-02-23T00:59:32.392 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:32.128+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.662876+0000 front 2024-02-23T00:59:00.662855+0000 (oldest deadline 2024-02-23T00:59:22.962673+0000) 2024-02-23T00:59:32.393 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:32.128+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6822 osd.2 since back 2024-02-23T00:59:08.263815+0000 front 2024-02-23T00:59:08.263772+0000 (oldest deadline 2024-02-23T00:59:31.163669+0000) 2024-02-23T00:59:32.694 INFO:journalctl@ceph.osd.5.smithi196.stdout:Feb 23 00:59:32 smithi196 bash[118992]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-5 2024-02-23T00:59:33.092 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:32.728+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.517693+0000 front 2024-02-23T00:58:51.517723+0000 (oldest deadline 2024-02-23T00:59:16.817507+0000) 2024-02-23T00:59:33.092 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:32.728+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:58:57.318234+0000 front 2024-02-23T00:58:57.318374+0000 (oldest deadline 2024-02-23T00:59:22.018351+0000) 2024-02-23T00:59:33.092 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:32 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:32.728+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6822 osd.2 since back 2024-02-23T00:59:07.220084+0000 front 2024-02-23T00:59:07.220056+0000 (oldest deadline 2024-02-23T00:59:30.719952+0000) 2024-02-23T00:59:33.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:33.122+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.861901+0000 front 2024-02-23T00:58:51.861916+0000 (oldest deadline 2024-02-23T00:59:17.161843+0000) 2024-02-23T00:59:33.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:33.122+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.662876+0000 front 2024-02-23T00:59:00.662855+0000 (oldest deadline 2024-02-23T00:59:22.962673+0000) 2024-02-23T00:59:33.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:33.122+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6822 osd.2 since back 2024-02-23T00:59:08.263815+0000 front 2024-02-23T00:59:08.263772+0000 (oldest deadline 2024-02-23T00:59:31.163669+0000) 2024-02-23T00:59:33.821 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:33.731+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.517693+0000 front 2024-02-23T00:58:51.517723+0000 (oldest deadline 2024-02-23T00:59:16.817507+0000) 2024-02-23T00:59:33.821 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:33.731+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:58:57.318234+0000 front 2024-02-23T00:58:57.318374+0000 (oldest deadline 2024-02-23T00:59:22.018351+0000) 2024-02-23T00:59:33.821 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:33 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:33.731+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6822 osd.2 since back 2024-02-23T00:59:07.220084+0000 front 2024-02-23T00:59:07.220056+0000 (oldest deadline 2024-02-23T00:59:30.719952+0000) 2024-02-23T00:59:34.057 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:59:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=warn ts=2024-02-23T00:59:33.583Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi047.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.47:8443: connect: connection refused" 2024-02-23T00:59:34.057 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:59:33 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=warn ts=2024-02-23T00:59:33.583Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-02-23T00:59:34.173 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@osd.5.service' 2024-02-23T00:59:34.213 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-23T00:59:34.213 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-02-23T00:59:34.213 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-02-23T00:59:34.214 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@osd.6 2024-02-23T00:59:34.445 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:34 smithi196 systemd[1]: Stopping Ceph osd.6 for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T00:59:34.445 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:34 smithi196 bash[119229]: Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.6" found: no such container 2024-02-23T00:59:34.445 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:34.153+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.861901+0000 front 2024-02-23T00:58:51.861916+0000 (oldest deadline 2024-02-23T00:59:17.161843+0000) 2024-02-23T00:59:34.445 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:34.153+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.662876+0000 front 2024-02-23T00:59:00.662855+0000 (oldest deadline 2024-02-23T00:59:22.962673+0000) 2024-02-23T00:59:34.445 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:34.153+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6822 osd.2 since back 2024-02-23T00:59:08.263815+0000 front 2024-02-23T00:59:08.263772+0000 (oldest deadline 2024-02-23T00:59:31.163669+0000) 2024-02-23T00:59:34.944 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:34.553+0000 7f2ceadee700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-23T00:59:34.944 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:34.553+0000 7f2ceadee700 -1 osd.6 860 *** Got signal Terminated *** 2024-02-23T00:59:34.944 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:34.553+0000 7f2ceadee700 -1 osd.6 860 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-23T00:59:34.944 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:34.706+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.517693+0000 front 2024-02-23T00:58:51.517723+0000 (oldest deadline 2024-02-23T00:59:16.817507+0000) 2024-02-23T00:59:34.945 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:34.706+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:58:57.318234+0000 front 2024-02-23T00:58:57.318374+0000 (oldest deadline 2024-02-23T00:59:22.018351+0000) 2024-02-23T00:59:34.945 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:34 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:34.706+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6822 osd.2 since back 2024-02-23T00:59:07.220084+0000 front 2024-02-23T00:59:07.220056+0000 (oldest deadline 2024-02-23T00:59:30.719952+0000) 2024-02-23T00:59:35.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:35.150+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.861901+0000 front 2024-02-23T00:58:51.861916+0000 (oldest deadline 2024-02-23T00:59:17.161843+0000) 2024-02-23T00:59:35.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:35.150+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.662876+0000 front 2024-02-23T00:59:00.662855+0000 (oldest deadline 2024-02-23T00:59:22.962673+0000) 2024-02-23T00:59:35.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:35.150+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6822 osd.2 since back 2024-02-23T00:59:08.263815+0000 front 2024-02-23T00:59:08.263772+0000 (oldest deadline 2024-02-23T00:59:31.163669+0000) 2024-02-23T00:59:35.944 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:35.662+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.517693+0000 front 2024-02-23T00:58:51.517723+0000 (oldest deadline 2024-02-23T00:59:16.817507+0000) 2024-02-23T00:59:35.944 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:35.662+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:58:57.318234+0000 front 2024-02-23T00:58:57.318374+0000 (oldest deadline 2024-02-23T00:59:22.018351+0000) 2024-02-23T00:59:35.944 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:35 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:35.662+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6822 osd.2 since back 2024-02-23T00:59:07.220084+0000 front 2024-02-23T00:59:07.220056+0000 (oldest deadline 2024-02-23T00:59:30.719952+0000) 2024-02-23T00:59:36.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:36.167+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.861901+0000 front 2024-02-23T00:58:51.861916+0000 (oldest deadline 2024-02-23T00:59:17.161843+0000) 2024-02-23T00:59:36.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:36.167+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.662876+0000 front 2024-02-23T00:59:00.662855+0000 (oldest deadline 2024-02-23T00:59:22.962673+0000) 2024-02-23T00:59:36.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:36.167+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6822 osd.2 since back 2024-02-23T00:59:08.263815+0000 front 2024-02-23T00:59:08.263772+0000 (oldest deadline 2024-02-23T00:59:31.163669+0000) 2024-02-23T00:59:37.133 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:36.712+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.517693+0000 front 2024-02-23T00:58:51.517723+0000 (oldest deadline 2024-02-23T00:59:16.817507+0000) 2024-02-23T00:59:37.133 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:36.712+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:58:57.318234+0000 front 2024-02-23T00:58:57.318374+0000 (oldest deadline 2024-02-23T00:59:22.018351+0000) 2024-02-23T00:59:37.133 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:36 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:36.712+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6822 osd.2 since back 2024-02-23T00:59:07.220084+0000 front 2024-02-23T00:59:07.220056+0000 (oldest deadline 2024-02-23T00:59:30.719952+0000) 2024-02-23T00:59:37.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:37.130+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.861901+0000 front 2024-02-23T00:58:51.861916+0000 (oldest deadline 2024-02-23T00:59:17.161843+0000) 2024-02-23T00:59:37.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:37.130+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.662876+0000 front 2024-02-23T00:59:00.662855+0000 (oldest deadline 2024-02-23T00:59:22.962673+0000) 2024-02-23T00:59:37.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:37.130+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6822 osd.2 since back 2024-02-23T00:59:08.263815+0000 front 2024-02-23T00:59:08.263772+0000 (oldest deadline 2024-02-23T00:59:31.163669+0000) 2024-02-23T00:59:38.179 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:37.750+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.517693+0000 front 2024-02-23T00:58:51.517723+0000 (oldest deadline 2024-02-23T00:59:16.817507+0000) 2024-02-23T00:59:38.179 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:37.750+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:58:57.318234+0000 front 2024-02-23T00:58:57.318374+0000 (oldest deadline 2024-02-23T00:59:22.018351+0000) 2024-02-23T00:59:38.179 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:37 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:37.750+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6822 osd.2 since back 2024-02-23T00:59:07.220084+0000 front 2024-02-23T00:59:07.220056+0000 (oldest deadline 2024-02-23T00:59:30.719952+0000) 2024-02-23T00:59:38.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:38.175+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.861901+0000 front 2024-02-23T00:58:51.861916+0000 (oldest deadline 2024-02-23T00:59:17.161843+0000) 2024-02-23T00:59:38.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:38.175+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.662876+0000 front 2024-02-23T00:59:00.662855+0000 (oldest deadline 2024-02-23T00:59:22.962673+0000) 2024-02-23T00:59:38.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:38.175+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6822 osd.2 since back 2024-02-23T00:59:08.263815+0000 front 2024-02-23T00:59:08.263772+0000 (oldest deadline 2024-02-23T00:59:31.163669+0000) 2024-02-23T00:59:39.189 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:38.776+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.517693+0000 front 2024-02-23T00:58:51.517723+0000 (oldest deadline 2024-02-23T00:59:16.817507+0000) 2024-02-23T00:59:39.189 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:38.776+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:58:57.318234+0000 front 2024-02-23T00:58:57.318374+0000 (oldest deadline 2024-02-23T00:59:22.018351+0000) 2024-02-23T00:59:39.190 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:38.776+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6822 osd.2 since back 2024-02-23T00:59:07.220084+0000 front 2024-02-23T00:59:07.220056+0000 (oldest deadline 2024-02-23T00:59:30.719952+0000) 2024-02-23T00:59:39.190 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:38 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6[52898]: debug 2024-02-23T00:59:38.776+0000 7f2ce7d6b700 -1 osd.6 860 heartbeat_check: no reply from 172.21.15.47:6830 osd.3 since back 2024-02-23T00:59:15.420823+0000 front 2024-02-23T00:59:15.420761+0000 (oldest deadline 2024-02-23T00:59:38.320772+0000) 2024-02-23T00:59:39.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:39.186+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.861901+0000 front 2024-02-23T00:58:51.861916+0000 (oldest deadline 2024-02-23T00:59:17.161843+0000) 2024-02-23T00:59:39.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:39.186+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.662876+0000 front 2024-02-23T00:59:00.662855+0000 (oldest deadline 2024-02-23T00:59:22.962673+0000) 2024-02-23T00:59:39.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:39 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:39.186+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6822 osd.2 since back 2024-02-23T00:59:08.263815+0000 front 2024-02-23T00:59:08.263772+0000 (oldest deadline 2024-02-23T00:59:31.163669+0000) 2024-02-23T00:59:40.194 INFO:journalctl@ceph.osd.6.smithi196.stdout:Feb 23 00:59:39 smithi196 bash[119271]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-6 2024-02-23T00:59:40.194 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:40.147+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.861901+0000 front 2024-02-23T00:58:51.861916+0000 (oldest deadline 2024-02-23T00:59:17.161843+0000) 2024-02-23T00:59:40.194 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:40.147+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.662876+0000 front 2024-02-23T00:59:00.662855+0000 (oldest deadline 2024-02-23T00:59:22.962673+0000) 2024-02-23T00:59:40.194 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:40 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:40.147+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6822 osd.2 since back 2024-02-23T00:59:08.263815+0000 front 2024-02-23T00:59:08.263772+0000 (oldest deadline 2024-02-23T00:59:31.163669+0000) 2024-02-23T00:59:40.557 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:59:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=error ts=2024-02-23T00:59:40.144Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi047.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.47:8443: connect: connection refused" 2024-02-23T00:59:40.558 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:59:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=warn ts=2024-02-23T00:59:40.144Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi047.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.47:8443: connect: connection refused" 2024-02-23T00:59:40.558 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:59:40 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=warn ts=2024-02-23T00:59:40.145Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-02-23T00:59:41.398 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:41.108+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.861901+0000 front 2024-02-23T00:58:51.861916+0000 (oldest deadline 2024-02-23T00:59:17.161843+0000) 2024-02-23T00:59:41.398 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:41.108+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.662876+0000 front 2024-02-23T00:59:00.662855+0000 (oldest deadline 2024-02-23T00:59:22.962673+0000) 2024-02-23T00:59:41.398 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:41 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:41.108+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6822 osd.2 since back 2024-02-23T00:59:08.263815+0000 front 2024-02-23T00:59:08.263772+0000 (oldest deadline 2024-02-23T00:59:31.163669+0000) 2024-02-23T00:59:41.682 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@osd.6.service' 2024-02-23T00:59:41.722 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-23T00:59:41.722 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-02-23T00:59:41.722 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-02-23T00:59:41.723 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@osd.7 2024-02-23T00:59:41.944 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:41 smithi196 systemd[1]: Stopping Ceph osd.7 for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T00:59:41.944 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:41 smithi196 bash[119509]: Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd.7" found: no such container 2024-02-23T00:59:42.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:42.055+0000 7f18b00a3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-23T00:59:42.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:42.055+0000 7f18b00a3700 -1 osd.7 860 *** Got signal Terminated *** 2024-02-23T00:59:42.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:42.055+0000 7f18b00a3700 -1 osd.7 860 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-23T00:59:42.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:42.063+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.861901+0000 front 2024-02-23T00:58:51.861916+0000 (oldest deadline 2024-02-23T00:59:17.161843+0000) 2024-02-23T00:59:42.445 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:42.063+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.662876+0000 front 2024-02-23T00:59:00.662855+0000 (oldest deadline 2024-02-23T00:59:22.962673+0000) 2024-02-23T00:59:42.445 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:42.063+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6822 osd.2 since back 2024-02-23T00:59:08.263815+0000 front 2024-02-23T00:59:08.263772+0000 (oldest deadline 2024-02-23T00:59:31.163669+0000) 2024-02-23T00:59:42.445 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:42 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:42.063+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6830 osd.3 since back 2024-02-23T00:59:16.464678+0000 front 2024-02-23T00:59:16.464668+0000 (oldest deadline 2024-02-23T00:59:41.764613+0000) 2024-02-23T00:59:43.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:43.039+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.861901+0000 front 2024-02-23T00:58:51.861916+0000 (oldest deadline 2024-02-23T00:59:17.161843+0000) 2024-02-23T00:59:43.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:43.039+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.662876+0000 front 2024-02-23T00:59:00.662855+0000 (oldest deadline 2024-02-23T00:59:22.962673+0000) 2024-02-23T00:59:43.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:43.039+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6822 osd.2 since back 2024-02-23T00:59:08.263815+0000 front 2024-02-23T00:59:08.263772+0000 (oldest deadline 2024-02-23T00:59:31.163669+0000) 2024-02-23T00:59:43.445 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:43 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:43.039+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6830 osd.3 since back 2024-02-23T00:59:16.464678+0000 front 2024-02-23T00:59:16.464668+0000 (oldest deadline 2024-02-23T00:59:41.764613+0000) 2024-02-23T00:59:44.057 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:59:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=error ts=2024-02-23T00:59:43.582Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=5 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi047.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.47:8443: connect: connection refused" 2024-02-23T00:59:44.057 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:59:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=warn ts=2024-02-23T00:59:43.583Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi047.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.47:8443: connect: connection refused" 2024-02-23T00:59:44.058 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:59:43 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=warn ts=2024-02-23T00:59:43.584Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-02-23T00:59:44.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:44.063+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.861901+0000 front 2024-02-23T00:58:51.861916+0000 (oldest deadline 2024-02-23T00:59:17.161843+0000) 2024-02-23T00:59:44.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:44.063+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.662876+0000 front 2024-02-23T00:59:00.662855+0000 (oldest deadline 2024-02-23T00:59:22.962673+0000) 2024-02-23T00:59:44.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:44.063+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6822 osd.2 since back 2024-02-23T00:59:08.263815+0000 front 2024-02-23T00:59:08.263772+0000 (oldest deadline 2024-02-23T00:59:31.163669+0000) 2024-02-23T00:59:44.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:44 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:44.063+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6830 osd.3 since back 2024-02-23T00:59:16.464678+0000 front 2024-02-23T00:59:16.464668+0000 (oldest deadline 2024-02-23T00:59:41.764613+0000) 2024-02-23T00:59:45.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:45.061+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.861901+0000 front 2024-02-23T00:58:51.861916+0000 (oldest deadline 2024-02-23T00:59:17.161843+0000) 2024-02-23T00:59:45.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:45.061+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.662876+0000 front 2024-02-23T00:59:00.662855+0000 (oldest deadline 2024-02-23T00:59:22.962673+0000) 2024-02-23T00:59:45.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:45.061+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6822 osd.2 since back 2024-02-23T00:59:08.263815+0000 front 2024-02-23T00:59:08.263772+0000 (oldest deadline 2024-02-23T00:59:31.163669+0000) 2024-02-23T00:59:45.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:45 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:45.061+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6830 osd.3 since back 2024-02-23T00:59:16.464678+0000 front 2024-02-23T00:59:16.464668+0000 (oldest deadline 2024-02-23T00:59:41.764613+0000) 2024-02-23T00:59:46.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:46.080+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.861901+0000 front 2024-02-23T00:58:51.861916+0000 (oldest deadline 2024-02-23T00:59:17.161843+0000) 2024-02-23T00:59:46.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:46.080+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.662876+0000 front 2024-02-23T00:59:00.662855+0000 (oldest deadline 2024-02-23T00:59:22.962673+0000) 2024-02-23T00:59:46.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:46.080+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6822 osd.2 since back 2024-02-23T00:59:08.263815+0000 front 2024-02-23T00:59:08.263772+0000 (oldest deadline 2024-02-23T00:59:31.163669+0000) 2024-02-23T00:59:46.444 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:46 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:46.080+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6830 osd.3 since back 2024-02-23T00:59:16.464678+0000 front 2024-02-23T00:59:16.464668+0000 (oldest deadline 2024-02-23T00:59:41.764613+0000) 2024-02-23T00:59:47.385 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:47.031+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-02-23T00:58:51.861901+0000 front 2024-02-23T00:58:51.861916+0000 (oldest deadline 2024-02-23T00:59:17.161843+0000) 2024-02-23T00:59:47.385 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:47.031+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6814 osd.1 since back 2024-02-23T00:59:00.662876+0000 front 2024-02-23T00:59:00.662855+0000 (oldest deadline 2024-02-23T00:59:22.962673+0000) 2024-02-23T00:59:47.385 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:47.031+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6822 osd.2 since back 2024-02-23T00:59:08.263815+0000 front 2024-02-23T00:59:08.263772+0000 (oldest deadline 2024-02-23T00:59:31.163669+0000) 2024-02-23T00:59:47.385 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:47 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7[57365]: debug 2024-02-23T00:59:47.031+0000 7f18aa01a700 -1 osd.7 860 heartbeat_check: no reply from 172.21.15.47:6830 osd.3 since back 2024-02-23T00:59:16.464678+0000 front 2024-02-23T00:59:16.464668+0000 (oldest deadline 2024-02-23T00:59:41.764613+0000) 2024-02-23T00:59:47.694 INFO:journalctl@ceph.osd.7.smithi196.stdout:Feb 23 00:59:47 smithi196 bash[119550]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-osd-7 2024-02-23T00:59:49.159 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@osd.7.service' 2024-02-23T00:59:49.201 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-23T00:59:49.201 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-02-23T00:59:49.201 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-02-23T00:59:49.202 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@rgw.foo.a 2024-02-23T00:59:49.511 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Feb 23 00:59:49 smithi047 systemd[1]: Stopping Ceph rgw.foo.a for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T00:59:49.511 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Feb 23 00:59:49 smithi047 bash[262227]: Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-rgw.foo.a" found: no such container 2024-02-23T00:59:49.807 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Feb 23 00:59:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-rgw-foo-a[64743]: debug 2024-02-23T00:59:49.508+0000 7fa212961700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-23T00:59:49.807 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Feb 23 00:59:49 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-rgw-foo-a[64743]: debug 2024-02-23T00:59:49.508+0000 7fa229a2b880 -1 shutting down 2024-02-23T00:59:50.558 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:59:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=error ts=2024-02-23T00:59:50.145Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi047.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.47:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-02-23T00:59:50.558 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:59:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=warn ts=2024-02-23T00:59:50.145Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi047.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.47:8443: connect: connection refused" 2024-02-23T00:59:50.558 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:59:50 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=warn ts=2024-02-23T00:59:50.146Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-02-23T00:59:54.057 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:59:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=error ts=2024-02-23T00:59:53.583Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=6 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi047.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.47:8443: connect: connection refused" 2024-02-23T00:59:54.058 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:59:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=warn ts=2024-02-23T00:59:53.584Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi047.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.47:8443: connect: connection refused" 2024-02-23T00:59:54.058 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 00:59:53 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=warn ts=2024-02-23T00:59:53.584Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-02-23T00:59:59.772 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Feb 23 00:59:59 smithi047 bash[262265]: time="2024-02-23T00:59:59Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-02-23T01:00:00.058 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Feb 23 00:59:59 smithi047 bash[262265]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-rgw-foo-a 2024-02-23T01:00:00.321 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@rgw.foo.a.service' 2024-02-23T01:00:00.359 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-23T01:00:00.359 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-02-23T01:00:00.359 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-02-23T01:00:00.359 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@prometheus.a 2024-02-23T01:00:00.557 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 01:00:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=error ts=2024-02-23T01:00:00.146Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi047.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.47:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-02-23T01:00:00.558 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 01:00:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=warn ts=2024-02-23T01:00:00.147Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi047.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.47:8443: connect: connection refused" 2024-02-23T01:00:00.558 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 01:00:00 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=warn ts=2024-02-23T01:00:00.147Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-02-23T01:00:00.671 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 01:00:00 smithi196 systemd[1]: Stopping Ceph prometheus.a for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T01:00:00.672 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 01:00:00 smithi196 bash[119797]: Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-prometheus.a" found: no such container 2024-02-23T01:00:00.944 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 01:00:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-prometheus-a[63150]: ts=2024-02-23T01:00:00.668Z caller=main.go:775 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-02-23T01:00:00.944 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 01:00:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-prometheus-a[63150]: ts=2024-02-23T01:00:00.669Z caller=main.go:798 level=info msg="Stopping scrape discovery manager..." 2024-02-23T01:00:00.944 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 01:00:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-prometheus-a[63150]: ts=2024-02-23T01:00:00.669Z caller=main.go:812 level=info msg="Stopping notify discovery manager..." 2024-02-23T01:00:00.945 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 01:00:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-prometheus-a[63150]: ts=2024-02-23T01:00:00.669Z caller=main.go:834 level=info msg="Stopping scrape manager..." 2024-02-23T01:00:00.945 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 01:00:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-prometheus-a[63150]: ts=2024-02-23T01:00:00.669Z caller=main.go:794 level=info msg="Scrape discovery manager stopped" 2024-02-23T01:00:00.945 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 01:00:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-prometheus-a[63150]: ts=2024-02-23T01:00:00.669Z caller=main.go:808 level=info msg="Notify discovery manager stopped" 2024-02-23T01:00:00.945 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 01:00:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-prometheus-a[63150]: ts=2024-02-23T01:00:00.669Z caller=manager.go:945 level=info component="rule manager" msg="Stopping rule manager..." 2024-02-23T01:00:00.945 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 01:00:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-prometheus-a[63150]: ts=2024-02-23T01:00:00.669Z caller=manager.go:955 level=info component="rule manager" msg="Rule manager stopped" 2024-02-23T01:00:00.945 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 01:00:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-prometheus-a[63150]: ts=2024-02-23T01:00:00.669Z caller=main.go:828 level=info msg="Scrape manager stopped" 2024-02-23T01:00:00.945 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 01:00:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-prometheus-a[63150]: ts=2024-02-23T01:00:00.669Z caller=notifier.go:600 level=info component=notifier msg="Stopping notification manager..." 2024-02-23T01:00:00.945 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 01:00:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-prometheus-a[63150]: ts=2024-02-23T01:00:00.669Z caller=main.go:1054 level=info msg="Notifier manager stopped" 2024-02-23T01:00:00.945 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 01:00:00 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-prometheus-a[63150]: ts=2024-02-23T01:00:00.669Z caller=main.go:1066 level=info msg="See you next time!" 2024-02-23T01:00:01.387 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Feb 23 01:00:01 smithi196 bash[119837]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-prometheus-a 2024-02-23T01:00:01.393 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@prometheus.a.service' 2024-02-23T01:00:01.429 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-23T01:00:01.429 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-02-23T01:00:01.429 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 --force --keep-logs 2024-02-23T01:00:01.736 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 01:00:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=warn ts=2024-02-23T01:00:01.625Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi047.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.47:8443: connect: connection refused" 2024-02-23T01:00:01.736 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 01:00:01 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=warn ts=2024-02-23T01:00:01.626Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-02-23T01:00:02.807 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 01:00:02 smithi047 systemd[1]: Stopping Ceph alertmanager.a for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T01:00:02.807 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 01:00:02 smithi047 bash[262696]: Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager.a" found: no such container 2024-02-23T01:00:03.201 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 01:00:02 smithi047 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a[71496]: level=info ts=2024-02-23T01:00:02.904Z caller=main.go:557 msg="Received SIGTERM, exiting gracefully..." 2024-02-23T01:00:03.557 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 01:00:03 smithi047 bash[262735]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-alertmanager-a 2024-02-23T01:00:04.308 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 01:00:03 smithi047 systemd[1]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@alertmanager.a.service: Succeeded. 2024-02-23T01:00:04.308 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Feb 23 01:00:03 smithi047 systemd[1]: Stopped Ceph alertmanager.a for fce2617e-d1e2-11ee-95c0-87774f69a715. 2024-02-23T01:00:04.308 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 01:00:03 smithi047 systemd[1]: Stopping Ceph node-exporter.a for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T01:00:04.308 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 01:00:04 smithi047 bash[262842]: Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter.a" found: no such container 2024-02-23T01:00:04.807 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 01:00:04 smithi047 bash[262881]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-a 2024-02-23T01:00:05.140 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 01:00:04 smithi047 systemd[1]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-02-23T01:00:05.140 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 01:00:05 smithi047 systemd[1]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@node-exporter.a.service: Failed with result 'exit-code'. 2024-02-23T01:00:05.140 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Feb 23 01:00:05 smithi047 systemd[1]: Stopped Ceph node-exporter.a for fce2617e-d1e2-11ee-95c0-87774f69a715. 2024-02-23T01:00:05.403 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 --force --keep-logs 2024-02-23T01:00:06.607 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 01:00:06 smithi196 systemd[1]: Stopping Ceph iscsi.iscsi.a for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T01:00:06.607 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 01:00:06 smithi196 bash[120191]: Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi.iscsi.a" found: no such container 2024-02-23T01:00:06.944 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 01:00:06 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a[60847]: debug Shutdown received 2024-02-23T01:00:16.944 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 01:00:16 smithi196 bash[120232]: time="2024-02-23T01:00:16Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-02-23T01:00:17.275 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 01:00:17 smithi196 bash[120232]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a 2024-02-23T01:00:17.532 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 01:00:17 smithi196 systemd[1]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-02-23T01:00:18.111 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 01:00:18 smithi196 systemd[1]: Stopping Ceph node-exporter.b for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T01:00:18.111 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 01:00:17 smithi196 bash[120307]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-iscsi-iscsi-a-tcmu 2024-02-23T01:00:18.112 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 01:00:17 smithi196 systemd[1]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-02-23T01:00:18.112 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Feb 23 01:00:17 smithi196 systemd[1]: Stopped Ceph iscsi.iscsi.a for fce2617e-d1e2-11ee-95c0-87774f69a715. 2024-02-23T01:00:18.444 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 01:00:18 smithi196 bash[120382]: Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter.b" found: no such container 2024-02-23T01:00:18.895 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 01:00:18 smithi196 bash[120423]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-node-exporter-b 2024-02-23T01:00:19.181 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 01:00:18 smithi196 systemd[1]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-02-23T01:00:19.181 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 01:00:19 smithi196 systemd[1]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@node-exporter.b.service: Failed with result 'exit-code'. 2024-02-23T01:00:19.181 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Feb 23 01:00:19 smithi196 systemd[1]: Stopped Ceph node-exporter.b for fce2617e-d1e2-11ee-95c0-87774f69a715. 2024-02-23T01:00:19.445 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 01:00:19 smithi196 systemd[1]: Stopping Ceph grafana.a for fce2617e-d1e2-11ee-95c0-87774f69a715... 2024-02-23T01:00:19.445 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 01:00:19 smithi196 bash[120559]: Error: no container with name or ID "ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana.a" found: no such container 2024-02-23T01:00:19.837 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 01:00:19 smithi196 ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a[63519]: t=2024-02-23T01:00:19+0000 lvl=info msg="Shutdown started" logger=server reason="System signal: terminated" 2024-02-23T01:00:20.184 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 01:00:19 smithi196 bash[120598]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715-grafana-a 2024-02-23T01:00:20.444 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 01:00:20 smithi196 systemd[1]: ceph-fce2617e-d1e2-11ee-95c0-87774f69a715@grafana.a.service: Succeeded. 2024-02-23T01:00:20.445 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Feb 23 01:00:20 smithi196 systemd[1]: Stopped Ceph grafana.a for fce2617e-d1e2-11ee-95c0-87774f69a715. 2024-02-23T01:00:20.551 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-23T01:00:20.591 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-23T01:00:20.620 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-23T01:00:20.622 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-02-22_21:39:39-rados-pacific-release-distro-default-smithi/7571410/remote/ubuntu@smithi047.front.sepia.ceph.com/crash 2024-02-23T01:00:20.623 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/crash -- . 2024-02-23T01:00:20.681 INFO:teuthology.orchestra.run.smithi047.stderr:tar: /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/crash: Cannot open: No such file or directory 2024-02-23T01:00:20.682 INFO:teuthology.orchestra.run.smithi047.stderr:tar: Error is not recoverable: exiting now 2024-02-23T01:00:20.683 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-02-22_21:39:39-rados-pacific-release-distro-default-smithi/7571410/remote/ubuntu@smithi196.front.sepia.ceph.com/crash 2024-02-23T01:00:20.684 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/crash -- . 2024-02-23T01:00:20.720 INFO:teuthology.orchestra.run.smithi196.stderr:tar: /var/lib/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/crash: Cannot open: No such file or directory 2024-02-23T01:00:20.720 INFO:teuthology.orchestra.run.smithi196.stderr:tar: Error is not recoverable: exiting now 2024-02-23T01:00:20.721 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-23T01:00:20.721 DEBUG:teuthology.orchestra.run.smithi047:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v '\(MON_DOWN\)' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v 'flag\(s\) set' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(PG_' | egrep -v '\(OSD_' | egrep -v '\(POOL_' | egrep -v '\(CEPHADM_STRAY_DAEMON\)' | egrep -v PG_ | egrep -v degraded | egrep -v backfill | egrep -v 'mons down' | egrep -v OSD_ | egrep -v 'is down' | egrep -v acting | egrep -v 'reached quota' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | head -n 1 2024-02-23T01:00:20.760 INFO:teuthology.orchestra.run.smithi047.stdout:2024-02-23T00:47:40.180059+0000 mon.a (mon.0) 2450 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-23T01:00:20.761 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-02-23T01:00:20.761 DEBUG:teuthology.orchestra.run.smithi047:> sudo egrep '\[SEC\]' /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v '\(MON_DOWN\)' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v 'flag\(s\) set' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(PG_' | egrep -v '\(OSD_' | egrep -v '\(POOL_' | egrep -v '\(CEPHADM_STRAY_DAEMON\)' | egrep -v PG_ | egrep -v degraded | egrep -v backfill | egrep -v 'mons down' | egrep -v OSD_ | egrep -v 'is down' | egrep -v acting | egrep -v 'reached quota' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | head -n 1 2024-02-23T01:00:20.838 DEBUG:teuthology.orchestra.run.smithi047:> sudo egrep '\[ERR\]' /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v '\(MON_DOWN\)' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v 'flag\(s\) set' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(PG_' | egrep -v '\(OSD_' | egrep -v '\(POOL_' | egrep -v '\(CEPHADM_STRAY_DAEMON\)' | egrep -v PG_ | egrep -v degraded | egrep -v backfill | egrep -v 'mons down' | egrep -v OSD_ | egrep -v 'is down' | egrep -v acting | egrep -v 'reached quota' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | head -n 1 2024-02-23T01:00:20.916 DEBUG:teuthology.orchestra.run.smithi047:> sudo egrep '\[WRN\]' /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v '\(MON_DOWN\)' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v 'flag\(s\) set' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(PG_' | egrep -v '\(OSD_' | egrep -v '\(POOL_' | egrep -v '\(CEPHADM_STRAY_DAEMON\)' | egrep -v PG_ | egrep -v degraded | egrep -v backfill | egrep -v 'mons down' | egrep -v OSD_ | egrep -v 'is down' | egrep -v acting | egrep -v 'reached quota' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | head -n 1 2024-02-23T01:00:20.991 INFO:teuthology.orchestra.run.smithi047.stdout:2024-02-23T00:47:40.180059+0000 mon.a (mon.0) 2450 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-23T01:00:20.992 INFO:tasks.cephadm:Compressing logs... 2024-02-23T01:00:20.992 DEBUG:teuthology.orchestra.run.smithi047:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-23T01:00:21.035 DEBUG:teuthology.orchestra.run.smithi196:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-23T01:00:21.063 INFO:teuthology.orchestra.run.smithi047.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-02-23T01:00:21.063 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-23T01:00:21.063 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-volume.log 2024-02-23T01:00:21.063 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-osd.2.log 2024-02-23T01:00:21.064 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-mgr.y.log 2024-02-23T01:00:21.064 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph.audit.log 2024-02-23T01:00:21.064 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-osd.2.log: /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph.cephadm.log 2024-02-23T01:00:21.065 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph.audit.log: /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph.cephadm.log: 88.9% -- replaced with /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph.cephadm.log.gz 2024-02-23T01:00:21.065 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-osd.3.log 2024-02-23T01:00:21.065 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-osd.0.log 2024-02-23T01:00:21.066 INFO:teuthology.orchestra.run.smithi196.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-23T01:00:21.066 INFO:teuthology.orchestra.run.smithi196.stderr:: No such file or directory 2024-02-23T01:00:21.066 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-osd.6.log 2024-02-23T01:00:21.066 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-volume.log 2024-02-23T01:00:21.066 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph.audit.log 2024-02-23T01:00:21.067 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph.cephadm.log 2024-02-23T01:00:21.067 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-volume.log: /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-osd.7.log 2024-02-23T01:00:21.067 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-mgr.x.log 2024-02-23T01:00:21.067 INFO:teuthology.orchestra.run.smithi196.stderr: 81.4% -- replaced with /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph.cephadm.log.gz 2024-02-23T01:00:21.067 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-osd.7.log: /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-mon.b.log 2024-02-23T01:00:21.070 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-mon.b.log: 91.4% -- replaced with /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-volume.log.gz 2024-02-23T01:00:21.070 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph.log 2024-02-23T01:00:21.071 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-osd.3.log: /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-osd.0.log: 91.5% -- replaced with /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-volume.log.gz 2024-02-23T01:00:21.072 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-client.rgw.foo.a.log 2024-02-23T01:00:21.073 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-osd.4.log 2024-02-23T01:00:21.073 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-client.rgw.foo.a.log: gzip -5 --verbose -- /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-mon.a.log 2024-02-23T01:00:21.075 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-osd.4.log: 91.9% -- replaced with /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph.audit.log.gz 2024-02-23T01:00:21.076 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-osd.5.log 2024-02-23T01:00:21.076 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-osd.5.log: 92.0% -- replaced with /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-mgr.x.log.gz 2024-02-23T01:00:21.077 INFO:teuthology.orchestra.run.smithi196.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-23T01:00:21.078 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/tcmu-runner.log 2024-02-23T01:00:21.078 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/tcmu-runner.log: 67.1% -- replaced with /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/tcmu-runner.log.gz 2024-02-23T01:00:21.080 INFO:teuthology.orchestra.run.smithi196.stderr: 85.8% -- replaced with /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph.log.gz 2024-02-23T01:00:21.084 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-mon.a.log: 95.0% -- replaced with /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph.audit.log.gz 2024-02-23T01:00:21.084 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-mon.c.log 2024-02-23T01:00:21.095 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph.log 2024-02-23T01:00:21.106 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-osd.1.log 2024-02-23T01:00:21.107 INFO:teuthology.orchestra.run.smithi047.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-23T01:00:21.129 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-osd.1.log: 92.0% -- replaced with /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph.log.gz 2024-02-23T01:00:21.314 INFO:teuthology.orchestra.run.smithi047.stderr: 93.6% -- replaced with /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-client.rgw.foo.a.log.gz 2024-02-23T01:00:21.544 INFO:teuthology.orchestra.run.smithi047.stderr: 90.8% -- replaced with /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-mgr.y.log.gz 2024-02-23T01:00:22.011 INFO:teuthology.orchestra.run.smithi196.stderr: 92.4% -- replaced with /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-mon.b.log.gz 2024-02-23T01:00:22.094 INFO:teuthology.orchestra.run.smithi047.stderr: 92.2% -- replaced with /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-mon.c.log.gz 2024-02-23T01:00:23.314 INFO:teuthology.orchestra.run.smithi047.stderr: 91.5% -- replaced with /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-mon.a.log.gz 2024-02-23T01:00:25.259 INFO:teuthology.orchestra.run.smithi047.stderr: 94.0% -- replaced with /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-osd.2.log.gz 2024-02-23T01:00:26.624 INFO:teuthology.orchestra.run.smithi196.stderr: 94.1% -- replaced with /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-osd.6.log.gz 2024-02-23T01:00:26.834 INFO:teuthology.orchestra.run.smithi196.stderr: 94.3% -- replaced with /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-osd.4.log.gz 2024-02-23T01:00:27.140 INFO:teuthology.orchestra.run.smithi047.stderr: 94.2% -- replaced with /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-osd.0.log.gz 2024-02-23T01:00:27.152 INFO:teuthology.orchestra.run.smithi196.stderr: 94.4% -- replaced with /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-osd.5.log.gz 2024-02-23T01:00:27.256 INFO:teuthology.orchestra.run.smithi047.stderr: 94.2% -- replaced with /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-osd.3.log.gz 2024-02-23T01:00:27.393 INFO:teuthology.orchestra.run.smithi196.stderr: 94.3% -- replaced with /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-osd.7.log.gz 2024-02-23T01:00:27.395 INFO:teuthology.orchestra.run.smithi196.stderr: 2024-02-23T01:00:27.395 INFO:teuthology.orchestra.run.smithi196.stderr:real 0m6.342s 2024-02-23T01:00:27.395 INFO:teuthology.orchestra.run.smithi196.stderr:user 0m23.807s 2024-02-23T01:00:27.395 INFO:teuthology.orchestra.run.smithi196.stderr:sys 0m0.903s 2024-02-23T01:00:27.788 INFO:teuthology.orchestra.run.smithi047.stderr: 94.2% -- replaced with /var/log/ceph/fce2617e-d1e2-11ee-95c0-87774f69a715/ceph-osd.1.log.gz 2024-02-23T01:00:27.790 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-02-23T01:00:27.790 INFO:teuthology.orchestra.run.smithi047.stderr:real 0m6.739s 2024-02-23T01:00:27.790 INFO:teuthology.orchestra.run.smithi047.stderr:user 0m26.236s 2024-02-23T01:00:27.790 INFO:teuthology.orchestra.run.smithi047.stderr:sys 0m0.936s 2024-02-23T01:00:27.790 INFO:tasks.cephadm:Archiving logs... 2024-02-23T01:00:27.791 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-02-22_21:39:39-rados-pacific-release-distro-default-smithi/7571410/remote/ubuntu@smithi047.front.sepia.ceph.com/log 2024-02-23T01:00:27.792 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-23T01:00:31.202 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-02-22_21:39:39-rados-pacific-release-distro-default-smithi/7571410/remote/ubuntu@smithi196.front.sepia.ceph.com/log 2024-02-23T01:00:31.203 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-23T01:00:34.556 INFO:tasks.cephadm:Removing cluster... 2024-02-23T01:00:34.556 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 --force 2024-02-23T01:00:35.070 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fce2617e-d1e2-11ee-95c0-87774f69a715 --force 2024-02-23T01:00:35.569 INFO:tasks.cephadm:Removing cephadm ... 2024-02-23T01:00:35.569 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-23T01:00:35.590 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-23T01:00:35.608 INFO:tasks.cephadm:Teardown complete 2024-02-23T01:00:35.609 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-23T01:00:35.619 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-23T01:00:35.619 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-23T01:00:35.633 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-23T01:00:35.725 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-23T01:00:35.735 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-23T01:00:35.735 DEBUG:teuthology.orchestra.run.smithi047:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-23T01:00:35.738 DEBUG:teuthology.orchestra.run.smithi196:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-23T01:00:35.754 INFO:teuthology.orchestra.run.smithi196.stderr:bash: ntpq: command not found 2024-02-23T01:00:35.754 INFO:teuthology.orchestra.run.smithi047.stderr:bash: ntpq: command not found 2024-02-23T01:00:35.758 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-23T01:00:35.758 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-02-23T01:00:35.758 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 52 +1910us[+1910us] +/- 92ms 2024-02-23T01:00:35.758 INFO:teuthology.orchestra.run.smithi196.stdout:^* hv02.front.sepia.ceph.com 3 6 377 52 -968us[-1038us] +/- 49ms 2024-02-23T01:00:35.759 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 51 -262us[ -262us] +/- 63ms 2024-02-23T01:00:35.759 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-23T01:00:35.760 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-23T01:00:35.760 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2024-02-23T01:00:35.760 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 56 +1899us[+1819us] +/- 92ms 2024-02-23T01:00:35.760 INFO:teuthology.orchestra.run.smithi047.stdout:^* hv02.front.sepia.ceph.com 3 6 377 55 -975us[-1055us] +/- 49ms 2024-02-23T01:00:35.760 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 53 -269us[ -269us] +/- 63ms 2024-02-23T01:00:35.760 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-23T01:00:35.761 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-23T01:00:35.772 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-23T01:00:35.772 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-23T01:00:35.786 DEBUG:teuthology.orchestra.run.smithi047:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-23T01:00:35.901 DEBUG:teuthology.orchestra.run.smithi196:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-23T01:00:35.983 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-23T01:00:36.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-23T01:00:36.017 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-23T01:00:36.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-23T01:00:36.048 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-23T01:00:36.058 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-23T01:00:36.073 INFO:teuthology.task.internal:Duration was 2814.891907 seconds 2024-02-23T01:00:36.073 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-23T01:00:36.085 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-23T01:00:36.085 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-23T01:00:36.087 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-23T01:00:36.133 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-23T01:00:36.133 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-23T01:00:36.317 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-23T01:00:36.317 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi047.front.sepia.ceph.com 2024-02-23T01:00:36.317 DEBUG:teuthology.orchestra.run.smithi047:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-23T01:00:36.340 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi196.front.sepia.ceph.com 2024-02-23T01:00:36.340 DEBUG:teuthology.orchestra.run.smithi196:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-23T01:00:36.362 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-23T01:00:36.362 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-23T01:00:36.382 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-23T01:00:36.500 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-23T01:00:36.500 DEBUG:teuthology.orchestra.run.smithi047:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-23T01:00:36.502 DEBUG:teuthology.orchestra.run.smithi196:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-23T01:00:36.803 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-23T01:00:36.813 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-23T01:00:36.814 DEBUG:teuthology.orchestra.run.smithi047:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-23T01:00:36.841 DEBUG:teuthology.orchestra.run.smithi196:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-23T01:00:36.868 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-23T01:00:36.966 DEBUG:teuthology.orchestra.run.smithi047:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-23T01:00:36.969 DEBUG:teuthology.orchestra.run.smithi196:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-23T01:00:36.995 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = core 2024-02-23T01:00:36.995 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = core 2024-02-23T01:00:37.019 DEBUG:teuthology.orchestra.run.smithi047:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-23T01:00:37.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-23T01:00:37.066 DEBUG:teuthology.orchestra.run.smithi196:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-23T01:00:37.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-23T01:00:37.083 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-23T01:00:37.093 INFO:teuthology.task.internal:Transferring archived files... 2024-02-23T01:00:37.094 DEBUG:teuthology.misc:Transferring archived files from smithi047:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-02-22_21:39:39-rados-pacific-release-distro-default-smithi/7571410/remote/smithi047 2024-02-23T01:00:37.095 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-23T01:00:37.167 DEBUG:teuthology.misc:Transferring archived files from smithi196:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-02-22_21:39:39-rados-pacific-release-distro-default-smithi/7571410/remote/smithi196 2024-02-23T01:00:37.168 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-23T01:00:37.216 INFO:teuthology.task.internal:Removing archive directory... 2024-02-23T01:00:37.217 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-23T01:00:37.219 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-23T01:00:37.265 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-23T01:00:37.276 INFO:teuthology.task.internal:Not uploading archives. 2024-02-23T01:00:37.276 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-23T01:00:37.286 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-23T01:00:37.287 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-23T01:00:37.289 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-23T01:00:37.305 INFO:teuthology.orchestra.run.smithi047.stdout: 526077 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 23 01:00 /home/ubuntu/cephtest 2024-02-23T01:00:37.322 INFO:teuthology.orchestra.run.smithi196.stdout: 525634 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 23 01:00 /home/ubuntu/cephtest 2024-02-23T01:00:37.324 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-23T01:00:37.335 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-23T01:00:37.358 INFO:teuthology.nuke:Checking targets against current locks 2024-02-23T01:00:37.381 DEBUG:teuthology.nuke:shortname: smithi047 2024-02-23T01:00:37.381 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-23T01:00:37.408 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi047.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-22_21:39:39-rados-pacific-release-distro-default-smithi/7571410', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-23 00:05:39.230323', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLja5BdvI+gD/T43f1uKqib+w89XuUXApahtXx6mCwgoCRsQhYYDNbC3SDQBa7ys1ksjaC4kryDPsWTIpx5hQRQ='} 2024-02-23T01:00:37.411 DEBUG:teuthology.nuke:shortname: smithi196 2024-02-23T01:00:37.411 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-23T01:00:37.431 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-22_21:39:39-rados-pacific-release-distro-default-smithi/7571410', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-23 00:05:39.227856', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL+ORhKeGKcECKIxv1tEiqI+7yxlQrpkHN2W5F/HdE9Olj7ZS9IkJm7qM7ObGbr9Z/sC15f3TGLKL76uw+nnO3s='} 2024-02-23T01:00:37.459 INFO:teuthology.orchestra.console.smithi047:Power off 2024-02-23T01:00:37.459 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-23T01:00:37.487 DEBUG:teuthology.orchestra.console.smithi047:power off output: Chassis Power Control: Down/Off 2024-02-23T01:00:37.487 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-23T01:00:37.500 INFO:teuthology.orchestra.console.smithi196:Power off 2024-02-23T01:00:37.501 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-23T01:00:37.510 DEBUG:teuthology.orchestra.console.smithi047:check power output: Chassis Power is on 2024-02-23T01:00:37.521 DEBUG:teuthology.orchestra.console.smithi196:power off output: Chassis Power Control: Down/Off 2024-02-23T01:00:37.521 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-23T01:00:37.543 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on 2024-02-23T01:00:41.512 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-23T01:00:41.544 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-23T01:00:41.622 DEBUG:teuthology.orchestra.console.smithi047:check power output: Chassis Power is on 2024-02-23T01:00:41.653 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on 2024-02-23T01:00:45.628 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-23T01:00:45.654 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-23T01:00:45.737 DEBUG:teuthology.orchestra.console.smithi047:check power output: Chassis Power is off 2024-02-23T01:00:45.763 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on 2024-02-23T01:00:45.839 INFO:teuthology.orchestra.console.smithi047:Power off completed 2024-02-23T01:00:49.764 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-23T01:00:49.875 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is off 2024-02-23T01:00:49.975 INFO:teuthology.orchestra.console.smithi196:Power off completed 2024-02-23T01:00:50.079 INFO:teuthology.run:Summary data: description: rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} duration: 2814.891907453537 failure_reason: '"2024-02-23T00:47:40.180059+0000 mon.a (mon.0) 2450 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL)" in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2024-02-23T01:00:50.080 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-23T01:00:50.171 INFO:teuthology.run:FAIL